2024-03-26T17:16:43.538 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T17:16:43.538 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} --archive /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746 --verbose --name lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi --owner scheduled_lflores@teuthology -- /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/orig.config.yaml 2024-03-26T17:16:43.596 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T17:16:43.689 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746 branch: wip-batrick-testing-20240320.145742 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7623746' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240320.145742 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - \(PG_DEGRADED - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 360516069d9393362c4cc6eb9371680fe16d66ab subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 360516069d9393362c4cc6eb9371680fe16d66ab kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-tracker-65018 sha1: 72e2437271dcafb9826f2f49bb1001426fa6de81 owner: scheduled_lflores@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4067 sha1: 360516069d9393362c4cc6eb9371680fe16d66ab sleep_before_teardown: 0 subset: 502/512 suite: fs suite_branch: wip-tracker-65018 suite_path: /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa suite_relpath: qa suite_repo: https://github.com/ljflores/ceph.git suite_sha1: 72e2437271dcafb9826f2f49bb1001426fa6de81 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJkJ/zNzhJMU8rIIvlIXthVYsZ4BrKzZ4FpTaWdJ4itQTssydhBKDBCQynlNS3fdGCiBTJFDtnYiFB4g19oNx0= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGmWgygJpbWFGl1WNLgOHy3mouoIX1uBTaGSpq9L6pDJasjwvgZBr+TMwjG7Gk1OODvDxb7FJYuOG13N5bVFggU= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGG/8Zcx0ZZbBdPTy0I0nOjTN5Ygmm++UXaETsRL3K+50Wl0R38H93gRXYIWTzQZIfmN310YtsXsjIWLW34c/8Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-26_17:10:07 tube: smithi user: lflores verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-26T17:16:43.690 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa; will attempt to use it 2024-03-26T17:16:43.690 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ljflores_ceph_72e2437271dcafb9826f2f49bb1001426fa6de81/qa/tasks 2024-03-26T17:16:43.690 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-26T17:16:43.692 INFO:teuthology.task.internal:Checking packages... 2024-03-26T17:16:43.716 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '360516069d9393362c4cc6eb9371680fe16d66ab' 2024-03-26T17:16:43.716 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-26T17:16:43.716 INFO:teuthology.packaging:ref: None 2024-03-26T17:16:43.717 INFO:teuthology.packaging:tag: None 2024-03-26T17:16:43.717 INFO:teuthology.packaging:branch: wip-batrick-testing-20240320.145742 2024-03-26T17:16:43.717 INFO:teuthology.packaging:sha1: 360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:16:43.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240320.145742 2024-03-26T17:16:43.936 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2311.g36051606 2024-03-26T17:16:43.937 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-26T17:16:43.945 INFO:teuthology.task.internal:no buildpackages task found 2024-03-26T17:16:43.945 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-26T17:16:43.953 INFO:teuthology.task.internal:Saving configuration 2024-03-26T17:16:43.973 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-26T17:16:43.980 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T17:16:44.005 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:11:42.884831', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJkJ/zNzhJMU8rIIvlIXthVYsZ4BrKzZ4FpTaWdJ4itQTssydhBKDBCQynlNS3fdGCiBTJFDtnYiFB4g19oNx0='} 2024-03-26T17:16:44.029 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:11:42.886026', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGmWgygJpbWFGl1WNLgOHy3mouoIX1uBTaGSpq9L6pDJasjwvgZBr+TMwjG7Gk1OODvDxb7FJYuOG13N5bVFggU='} 2024-03-26T17:16:44.054 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:11:42.883247', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGG/8Zcx0ZZbBdPTy0I0nOjTN5Ygmm++UXaETsRL3K+50Wl0R38H93gRXYIWTzQZIfmN310YtsXsjIWLW34c/8Y='} 2024-03-26T17:16:44.054 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-26T17:16:44.064 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-26T17:16:44.064 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-26T17:16:44.064 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-26T17:16:44.064 INFO:teuthology.run_tasks:Running task console_log... 2024-03-26T17:16:44.166 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6b7c4b1160>, signals=[15]) 2024-03-26T17:16:44.166 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-26T17:16:44.176 INFO:teuthology.task.internal:Opening connections... 2024-03-26T17:16:44.177 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T17:16:44.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:16:44.273 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-03-26T17:16:44.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:16:44.366 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-03-26T17:16:44.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:16:44.455 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-26T17:16:44.464 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-03-26T17:16:44.482 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-03-26T17:16:44.482 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-03-26T17:16:44.539 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-03-26T17:16:44.540 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T17:16:44.540 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-03-26T17:16:44.540 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T17:16:44.540 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T17:16:44.540 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T17:16:44.540 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-03-26T17:16:44.566 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-03-26T17:16:44.583 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-03-26T17:16:44.584 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-03-26T17:16:44.641 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T17:16:44.642 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T17:16:44.642 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T17:16:44.642 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-03-26T17:16:44.666 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-03-26T17:16:44.684 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-03-26T17:16:44.684 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-03-26T17:16:44.742 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-03-26T17:16:44.742 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-03-26T17:16:44.742 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-03-26T17:16:44.742 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-03-26T17:16:44.742 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-03-26T17:16:44.742 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-03-26T17:16:44.742 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T17:16:44.742 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-03-26T17:16:44.743 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-03-26T17:16:44.743 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T17:16:44.743 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-03-26T17:16:44.743 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T17:16:44.743 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T17:16:44.743 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T17:16:44.743 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-03-26T17:16:44.766 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-26T17:16:44.776 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-26T17:16:44.787 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-26T17:16:44.787 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-03-26T17:16:44.789 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-03-26T17:16:44.792 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-03-26T17:16:44.808 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-26T17:16:44.816 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-26T17:16:44.816 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-03-26T17:16:44.846 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-03-26T17:16:44.849 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-03-26T17:16:45.001 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-26T17:16:45.079 INFO:teuthology.run_tasks:Running task kernel... 2024-03-26T17:16:45.098 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-26T17:16:45.099 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-26T17:16:45.099 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T17:16:45.100 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T17:16:45.100 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T17:16:45.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:16:45.116 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-03-26T17:16:45.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:16:45.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:16:45.118 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-03-26T17:16:45.118 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-03-26T17:16:45.173 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-361.el9.x86_64 2024-03-26T17:16:45.173 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.14.0-361.el9.x86_64 2024-03-26T17:16:45.173 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2024-03-26T17:16:45.174 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-361.el9.x86_64 2024-03-26T17:16:45.174 INFO:teuthology.task.kernel:Running kernel on smithi135: 5.14.0-361.el9.x86_64 2024-03-26T17:16:45.175 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum install -y kernel 2024-03-26T17:16:45.176 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-361.el9.x86_64 2024-03-26T17:16:45.176 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.14.0-361.el9.x86_64 2024-03-26T17:16:45.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2024-03-26T17:16:47.466 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 9.9 MB/s | 6.7 MB 00:00 2024-03-26T17:16:47.589 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 8.1 MB/s | 6.7 MB 00:00 2024-03-26T17:16:47.664 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - AppStream 7.6 MB/s | 6.7 MB 00:00 2024-03-26T17:16:49.064 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-03-26T17:16:49.156 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-26T17:16:49.317 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 2.7 MB/s | 2.0 MB 00:00 2024-03-26T17:16:49.821 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-03-26T17:16:50.324 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-26T17:16:50.430 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-03-26T17:16:52.106 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-03-26T17:16:52.564 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-26T17:16:53.333 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-03-26T17:16:56.266 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.1 MB 00:00 2024-03-26T17:16:57.134 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.1 MB 00:01 2024-03-26T17:16:57.909 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-03-26T17:16:58.534 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.1 MB 00:01 2024-03-26T17:16:58.640 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-03-26T17:17:00.068 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-03-26T17:17:00.292 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-26T17:17:00.521 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 21 MB 00:02 2024-03-26T17:17:02.005 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-26T17:17:05.291 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-26T17:17:05.410 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-26T17:17:06.985 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-26T17:17:07.456 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-26T17:17:07.457 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-26T17:17:07.562 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-26T17:17:07.567 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-26T17:17:07.567 INFO:teuthology.orchestra.run.smithi135.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-26T17:17:07.569 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-26T17:17:07.570 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-03-26T17:17:07.571 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-03-26T17:17:07.571 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:17:07.571 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 76 M 2024-03-26T17:17:07.571 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 123 M 2024-03-26T17:17:07.571 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-26T17:17:07.646 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-03-26T17:17:07.655 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-03-26T17:17:07.655 INFO:teuthology.orchestra.run.smithi135.stdout: Package Architecture Version Repository Size 2024-03-26T17:17:07.655 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-03-26T17:17:07.655 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-03-26T17:17:07.655 INFO:teuthology.orchestra.run.smithi135.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-26T17:17:07.655 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-03-26T17:17:07.655 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-26T17:17:07.655 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-26T17:17:07.656 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-26T17:17:07.656 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:17:07.656 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-03-26T17:17:07.656 INFO:teuthology.orchestra.run.smithi135.stdout:================================================================================ 2024-03-26T17:17:07.656 INFO:teuthology.orchestra.run.smithi135.stdout:Install 4 Packages 2024-03-26T17:17:07.656 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:17:07.656 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 76 M 2024-03-26T17:17:07.657 INFO:teuthology.orchestra.run.smithi135.stdout:Installed size: 123 M 2024-03-26T17:17:07.657 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-03-26T17:17:07.951 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 193 kB/s | 66 kB 00:00 2024-03-26T17:17:08.004 INFO:teuthology.orchestra.run.smithi135.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 751 kB/s | 66 kB 00:00 2024-03-26T17:17:09.134 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-26T17:17:09.135 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-26T17:17:09.239 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-26T17:17:09.251 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-26T17:17:09.251 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2024-03-26T17:17:09.251 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-26T17:17:09.251 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-03-26T17:17:09.252 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-26T17:17:09.252 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-03-26T17:17:09.252 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-26T17:17:09.252 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-26T17:17:09.252 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-26T17:17:09.252 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:17:09.253 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-26T17:17:09.253 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-03-26T17:17:09.253 INFO:teuthology.orchestra.run.smithi105.stdout:Install 4 Packages 2024-03-26T17:17:09.253 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:17:09.253 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 76 M 2024-03-26T17:17:09.253 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 123 M 2024-03-26T17:17:09.254 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-26T17:17:09.279 INFO:teuthology.orchestra.run.smithi135.stdout:(2/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 25 MB/s | 33 MB 00:01 2024-03-26T17:17:09.454 INFO:teuthology.orchestra.run.smithi135.stdout:(3/4): kernel-core-5.14.0-432.el9.x86_64.rpm 9.7 MB/s | 15 MB 00:01 2024-03-26T17:17:09.520 INFO:teuthology.orchestra.run.smithi105.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 537 kB/s | 66 kB 00:00 2024-03-26T17:17:10.447 INFO:teuthology.orchestra.run.smithi135.stdout:(4/4): kernel-modules-core-5.14.0-432.el9.x86_6 11 MB/s | 27 MB 00:02 2024-03-26T17:17:10.447 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:17:10.447 INFO:teuthology.orchestra.run.smithi135.stdout:Total 27 MB/s | 76 MB 00:02 2024-03-26T17:17:10.962 INFO:teuthology.orchestra.run.smithi105.stdout:(2/4): kernel-core-5.14.0-432.el9.x86_64.rpm 9.5 MB/s | 15 MB 00:01 2024-03-26T17:17:11.115 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-03-26T17:17:11.482 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-03-26T17:17:11.482 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-03-26T17:17:12.472 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-03-26T17:17:12.472 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-03-26T17:17:12.487 INFO:teuthology.orchestra.run.smithi105.stdout:(3/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 11 MB/s | 33 MB 00:03 2024-03-26T17:17:12.729 INFO:teuthology.orchestra.run.smithi105.stdout:(4/4): kernel-modules-core-5.14.0-432.el9.x86_6 8.6 MB/s | 27 MB 00:03 2024-03-26T17:17:12.729 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:17:12.730 INFO:teuthology.orchestra.run.smithi105.stdout:Total 22 MB/s | 76 MB 00:03 2024-03-26T17:17:13.398 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-26T17:17:13.759 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-26T17:17:13.759 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-26T17:17:14.745 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-26T17:17:14.745 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-26T17:17:14.909 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-03-26T17:17:15.531 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-26T17:17:15.626 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:17:17.186 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-26T17:17:17.775 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 3.3 MB/s | 33 MB 00:10 2024-03-26T17:17:17.976 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-26T17:17:18.062 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:17:18.150 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): kernel-modules-core-5.14.0-432.el9.x86_6 2.7 MB/s | 27 MB 00:10 2024-03-26T17:17:18.349 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:17:18.503 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:17:20.350 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): kernel-core-5.14.0-432.el9.x86_64.rpm 1.2 MB/s | 15 MB 00:12 2024-03-26T17:17:20.350 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:17:20.350 INFO:teuthology.orchestra.run.smithi144.stdout:Total 5.9 MB/s | 76 MB 00:12 2024-03-26T17:17:20.774 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:17:20.907 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:17:21.023 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-26T17:17:21.379 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-26T17:17:21.379 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-26T17:17:22.391 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-26T17:17:22.391 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-26T17:17:23.177 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:17:23.213 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:24.839 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-26T17:17:25.448 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-26T17:17:25.533 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:17:25.595 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:17:25.699 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:27.939 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:28.269 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:17:28.413 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:17:30.385 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:33.118 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:17:33.221 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:37.951 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:57.416 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:57.416 INFO:teuthology.orchestra.run.smithi135.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T17:17:57.416 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:17:57.468 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:57.811 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:57.811 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-26T17:17:57.811 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:17:57.811 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:17:58.256 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-03-26T17:17:58.580 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T17:17:58.689 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T17:17:58.690 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T17:17:58.690 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-26T17:17:58.690 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-26T17:17:58.690 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-26T17:17:58.690 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-26T17:17:58.690 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi135.front.sepia.ceph.com, path=None, version=distro) 2024-03-26T17:17:58.691 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T17:17:58.770 INFO:teuthology.orchestra.run.smithi135.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T17:17:58.771 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T17:17:58.771 DEBUG:teuthology.orchestra.run.smithi135:> sudo rpm -qi grub2-tools 2024-03-26T17:17:58.814 INFO:teuthology.orchestra.run.smithi135.stdout:Name : grub2-tools 2024-03-26T17:17:58.814 INFO:teuthology.orchestra.run.smithi135.stdout:Epoch : 1 2024-03-26T17:17:58.814 INFO:teuthology.orchestra.run.smithi135.stdout:Version : 2.06 2024-03-26T17:17:58.814 INFO:teuthology.orchestra.run.smithi135.stdout:Release : 68.el9 2024-03-26T17:17:58.814 INFO:teuthology.orchestra.run.smithi135.stdout:Architecture: x86_64 2024-03-26T17:17:58.814 INFO:teuthology.orchestra.run.smithi135.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T17:17:58.814 INFO:teuthology.orchestra.run.smithi135.stdout:Group : Unspecified 2024-03-26T17:17:58.814 INFO:teuthology.orchestra.run.smithi135.stdout:Size : 8264254 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:License : GPLv3+ 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:Packager : builder@centos.org 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:Vendor : CentOS 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:Summary : Support tools for GRUB. 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:Description : 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T17:17:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T17:17:58.816 INFO:teuthology.orchestra.run.smithi135.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T17:17:58.816 INFO:teuthology.orchestra.run.smithi135.stdout:hardware devices. 2024-03-26T17:17:58.816 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:17:58.816 INFO:teuthology.orchestra.run.smithi135.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T17:17:58.817 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T17:17:58.817 INFO:teuthology.task.kernel:Updating grub on smithi135 to boot 5.14.0-432.el9.x86_64 2024-03-26T17:17:58.817 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T17:17:59.371 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:59.371 INFO:teuthology.orchestra.run.smithi105.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T17:17:59.372 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:17:59.416 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:59.459 INFO:teuthology.orchestra.run.smithi135.stderr:Generating grub configuration file ... 2024-03-26T17:17:59.812 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:17:59.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-26T17:17:59.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:17:59.812 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:18:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:18:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:18:00.224 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-03-26T17:18:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-26T17:18:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-26T17:18:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-26T17:18:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-26T17:18:00.225 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:18:00.225 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-26T17:18:00.543 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T17:18:00.646 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T17:18:00.652 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T17:18:00.652 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-26T17:18:00.652 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-26T17:18:00.652 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-26T17:18:00.652 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-26T17:18:00.653 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-03-26T17:18:00.653 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T17:18:00.736 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T17:18:00.737 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T17:18:00.737 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2024-03-26T17:18:00.785 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.06 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 68.el9 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Group : Unspecified 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 8264254 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T17:18:00.786 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : builder@centos.org 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T17:18:00.787 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T17:18:00.788 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2024-03-26T17:18:00.788 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:18:00.788 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T17:18:00.789 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T17:18:00.789 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 5.14.0-432.el9.x86_64 2024-03-26T17:18:00.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T17:18:00.856 INFO:teuthology.orchestra.run.smithi135.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T17:18:00.891 INFO:teuthology.orchestra.run.smithi135.stderr:done 2024-03-26T17:18:00.894 DEBUG:teuthology.orchestra.run.smithi135:> mktemp 2024-03-26T17:18:00.910 INFO:teuthology.orchestra.run.smithi135.stdout:/tmp/tmp.5CsXZVkkm4 2024-03-26T17:18:00.911 DEBUG:teuthology.orchestra.run.smithi135:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5CsXZVkkm4 2024-03-26T17:18:00.978 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 0666 /tmp/tmp.5CsXZVkkm4 2024-03-26T17:18:01.120 DEBUG:teuthology.orchestra.remote:smithi135:/tmp/tmp.5CsXZVkkm4 is 6KB 2024-03-26T17:18:01.168 DEBUG:teuthology.orchestra.run.smithi135:> rm -fr /tmp/tmp.5CsXZVkkm4 2024-03-26T17:18:01.182 DEBUG:teuthology.orchestra.run.smithi135:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T17:18:01.246 INFO:teuthology.orchestra.run.smithi135.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T17:18:01.247 INFO:teuthology.orchestra.run.smithi135.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T17:18:01.247 INFO:teuthology.orchestra.run.smithi135.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T17:18:01.247 INFO:teuthology.orchestra.run.smithi135.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-26T17:18:01.248 DEBUG:teuthology.orchestra.run.smithi135:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-26T17:18:01.394 DEBUG:teuthology.orchestra.run.smithi135:> sudo shutdown -r now 2024-03-26T17:18:01.398 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-03-26T17:18:02.703 INFO:teuthology.orchestra.run.smithi105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T17:18:02.731 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-03-26T17:18:02.733 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-03-26T17:18:02.748 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.PRzb2mgfkv 2024-03-26T17:18:02.749 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PRzb2mgfkv 2024-03-26T17:18:02.839 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.PRzb2mgfkv 2024-03-26T17:18:02.944 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.PRzb2mgfkv is 6KB 2024-03-26T17:18:02.992 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.PRzb2mgfkv 2024-03-26T17:18:03.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T17:18:03.078 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T17:18:03.079 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T17:18:03.079 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T17:18:03.079 INFO:teuthology.orchestra.run.smithi105.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-26T17:18:03.080 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-26T17:18:03.239 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-03-26T17:18:07.295 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:18:07.295 INFO:teuthology.orchestra.run.smithi144.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T17:18:07.295 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:18:07.341 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:18:07.722 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:18:07.722 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-26T17:18:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T17:18:07.723 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T17:18:08.130 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T17:18:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:18:08.131 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-26T17:18:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-26T17:18:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-26T17:18:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-26T17:18:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-26T17:18:08.131 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:18:08.131 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-26T17:18:08.453 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T17:18:08.601 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T17:18:08.601 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T17:18:08.601 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-26T17:18:08.601 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-26T17:18:08.601 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-26T17:18:08.601 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-03-26T17:18:08.601 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-03-26T17:18:08.602 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T17:18:08.681 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T17:18:08.681 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T17:18:08.681 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 68.el9 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8264254 2024-03-26T17:18:08.716 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:18:08.717 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T17:18:08.718 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T17:18:08.718 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T17:18:08.718 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-03-26T17:18:08.718 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:18:08.718 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T17:18:08.719 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T17:18:08.719 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 5.14.0-432.el9.x86_64 2024-03-26T17:18:08.719 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T17:18:09.344 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-03-26T17:18:10.689 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T17:18:10.723 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-03-26T17:18:10.725 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-03-26T17:18:10.741 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.0c3Ltq2uzt 2024-03-26T17:18:10.741 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0c3Ltq2uzt 2024-03-26T17:18:10.809 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.0c3Ltq2uzt 2024-03-26T17:18:10.960 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.0c3Ltq2uzt is 6KB 2024-03-26T17:18:11.011 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.0c3Ltq2uzt 2024-03-26T17:18:11.026 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T17:18:11.093 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T17:18:11.094 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T17:18:11.094 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T17:18:11.094 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-26T17:18:11.095 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-26T17:18:11.253 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-03-26T17:18:31.416 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-26T17:18:31.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-03-26T17:18:31.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:18:33.242 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-26T17:18:33.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-26T17:18:33.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:18:41.266 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-26T17:18:41.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-26T17:18:41.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:18:49.890 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.135 2024-03-26T17:18:51.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-03-26T17:18:58.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-03-26T17:18:58.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:18:59.614 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-03-26T17:19:00.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-26T17:19:00.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:19:01.950 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.135 2024-03-26T17:19:03.746 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-03-26T17:19:08.618 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-26T17:19:08.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:19:13.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-03-26T17:19:13.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:19:15.749 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-26T17:19:15.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:19:32.382 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.135 2024-03-26T17:19:47.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-03-26T17:19:47.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:19:47.633 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-26T17:19:47.827 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-03-26T17:19:48.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-26T17:19:48.032 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-432.el9.x86_64"... 2024-03-26T17:19:48.032 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-03-26T17:19:48.046 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-432.el9.x86_64 2024-03-26T17:19:48.046 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-26T17:19:48.046 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T17:19:48.046 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T17:19:48.247 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-03-26T17:19:48.247 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-432.el9.x86_64"... 2024-03-26T17:19:48.247 DEBUG:teuthology.orchestra.run.smithi135:> uname -r 2024-03-26T17:19:48.261 INFO:teuthology.orchestra.run.smithi135.stdout:5.14.0-432.el9.x86_64 2024-03-26T17:19:48.261 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-26T17:19:48.261 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T17:19:48.261 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T17:19:49.047 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-26T17:19:49.047 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T17:19:49.114 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2024-03-26T17:19:49.159 DEBUG:teuthology.parallel:result is None 2024-03-26T17:19:49.262 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-26T17:19:49.263 DEBUG:teuthology.orchestra.run.smithi135:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T17:19:49.325 INFO:teuthology.orchestra.run.smithi135.stdout:ttyS1 2024-03-26T17:19:49.370 DEBUG:teuthology.parallel:result is None 2024-03-26T17:20:08.622 DEBUG:teuthology.orchestra.remote:timed out 2024-03-26T17:20:20.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-26T17:20:20.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:20:21.036 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-26T17:20:21.393 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-26T17:20:21.394 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-432.el9.x86_64"... 2024-03-26T17:20:21.394 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-03-26T17:20:21.409 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-432.el9.x86_64 2024-03-26T17:20:21.409 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-26T17:20:21.409 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T17:20:21.409 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T17:20:22.410 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-26T17:20:22.410 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T17:20:22.480 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-03-26T17:20:22.521 DEBUG:teuthology.parallel:result is None 2024-03-26T17:20:22.521 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-26T17:20:22.528 INFO:teuthology.task.internal:Creating test directory... 2024-03-26T17:20:22.528 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T17:20:22.531 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T17:20:22.534 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T17:20:22.551 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-26T17:20:22.557 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-26T17:20:22.565 INFO:teuthology.task.internal:Creating archive directory... 2024-03-26T17:20:22.565 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T17:20:22.589 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T17:20:22.592 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T17:20:22.623 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-26T17:20:22.630 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-26T17:20:22.630 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T17:20:22.656 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T17:20:22.658 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T17:20:22.689 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:20:22.695 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:20:22.698 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:20:22.703 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:20:22.708 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:20:22.712 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T17:20:22.713 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-26T17:20:22.720 INFO:teuthology.task.internal:Configuring sudo... 2024-03-26T17:20:22.720 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T17:20:22.747 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T17:20:22.756 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T17:20:22.784 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-26T17:20:22.794 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-26T17:20:22.794 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T17:20:22.817 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T17:20:22.827 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T17:20:22.842 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:20:22.914 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:20:22.963 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:20:23.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:20:23.088 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:20:23.088 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T17:20:23.154 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:20:23.200 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:20:23.249 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:20:23.333 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:20:23.377 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:20:23.377 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T17:20:23.446 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:20:23.493 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T17:20:23.546 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:20:23.629 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T17:20:23.675 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:20:23.675 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T17:20:23.743 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-03-26T17:20:23.745 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-03-26T17:20:23.748 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-03-26T17:20:23.794 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T17:20:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T17:20:23.828 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T17:20:24.208 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-26T17:20:24.216 INFO:teuthology.task.internal:Starting timer... 2024-03-26T17:20:24.216 INFO:teuthology.run_tasks:Running task pcp... 2024-03-26T17:20:24.227 INFO:teuthology.run_tasks:Running task selinux... 2024-03-26T17:20:24.235 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-26T17:20:24.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-03-26T17:20:24.291 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-03-26T17:20:24.293 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-03-26T17:20:24.346 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-03-26T17:20:24.348 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-03-26T17:20:24.421 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-03-26T17:20:24.422 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-26T17:20:24.422 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-03-26T17:20:24.449 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-03-26T17:20:24.449 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-03-26T17:20:24.474 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-03-26T17:20:24.474 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-03-26T17:20:24.497 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-03-26T17:20:24.497 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2024-03-26T17:20:24.498 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T17:20:24.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:20:24.526 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T17:20:24.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:20:24.556 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T17:20:24.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:20:24.584 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-26T17:20:24.584 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-03-26T17:20:24.617 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-03-26T17:20:24.650 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-03-26T17:20:24.683 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-26T17:20:24.693 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-26T17:20:24.709 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-26T17:20:24.710 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi135.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-26T17:24:56.821 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-03-26T17:24:56.823 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-26T17:24:56.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:24:56.918 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-26T17:24:57.001 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-26T17:24:57.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-03-26T17:24:57.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:24:57.104 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-03-26T17:24:57.188 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-03-26T17:24:57.189 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-26T17:24:57.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T17:24:57.279 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-03-26T17:24:57.361 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-03-26T17:24:57.362 INFO:teuthology.run_tasks:Running task clock... 2024-03-26T17:24:57.372 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-26T17:24:57.373 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T17:24:57.373 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T17:24:57.375 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T17:24:57.376 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T17:24:57.378 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T17:24:57.378 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T17:24:57.406 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T17:24:57.407 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T17:24:57.423 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T17:24:57.424 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T17:24:57.432 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T17:24:57.450 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T17:24:57.453 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-03-26T17:24:57.454 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-03-26T17:24:57.467 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-03-26T17:24:57.468 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-03-26T17:24:57.479 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-03-26T17:24:57.483 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T17:24:57.484 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T17:24:57.492 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-03-26T17:24:57.499 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T17:24:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T17:24:57.509 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T17:24:57.525 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T17:24:57.557 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-26T17:24:57.559 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-03-26T17:24:57.560 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T17:24:57.560 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-26T17:24:57.560 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.560 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.560 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.561 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.562 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T17:24:57.562 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-03-26T17:24:57.562 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.562 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.562 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.562 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.592 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-26T17:24:57.596 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T17:24:57.596 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-26T17:24:57.596 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.596 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.596 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.596 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T17:24:57.597 INFO:teuthology.run_tasks:Running task install... 2024-03-26T17:24:57.606 DEBUG:teuthology.task.install:project ceph 2024-03-26T17:24:57.606 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab'}} 2024-03-26T17:24:57.606 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab'} 2024-03-26T17:24:57.606 INFO:teuthology.task.install:Using flavor: default 2024-03-26T17:24:57.612 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-26T17:24:57.612 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-26T17:24:57.613 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T17:24:57.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:24:57.615 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T17:24:57.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:24:57.616 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'tag': None, 'wait_for_package': False} 2024-03-26T17:24:57.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:24:57.772 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T17:24:57.889 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T17:24:57.954 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T17:24:57.954 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T17:24:57.966 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/ 2024-03-26T17:24:57.966 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2311.g36051606 2024-03-26T17:24:58.060 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T17:24:58.060 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:24:58.060 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T17:24:58.062 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T17:24:58.062 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:24:58.062 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T17:24:58.077 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T17:24:58.077 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:24:58.078 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T17:24:58.090 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T17:24:58.091 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T17:24:58.092 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T17:24:58.092 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T17:24:58.107 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T17:24:58.107 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/360516069d9393362c4cc6eb9371680fe16d66ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T17:24:58.158 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T17:24:58.160 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T17:24:58.175 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T17:24:58.238 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T17:24:58.240 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T17:24:58.256 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T17:24:58.305 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-03-26T17:24:58.306 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-03-26T17:24:58.307 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-03-26T17:24:58.308 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-03-26T17:24:58.323 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-03-26T17:24:58.325 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-03-26T17:24:58.548 INFO:teuthology.orchestra.run.smithi135.stdout:74 files removed 2024-03-26T17:24:58.553 INFO:teuthology.orchestra.run.smithi144.stdout:74 files removed 2024-03-26T17:24:58.577 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T17:24:58.578 INFO:teuthology.orchestra.run.smithi105.stdout:74 files removed 2024-03-26T17:24:58.579 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T17:24:58.604 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T17:24:59.261 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 216 kB/s | 83 kB 00:00 2024-03-26T17:24:59.270 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 213 kB/s | 83 kB 00:00 2024-03-26T17:24:59.387 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 171 kB/s | 83 kB 00:00 2024-03-26T17:24:59.628 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-26T17:24:59.688 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-26T17:24:59.805 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-26T17:25:00.064 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-26T17:25:00.079 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-26T17:25:00.222 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-26T17:25:00.933 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 9.9 MB/s | 6.7 MB 00:00 2024-03-26T17:25:01.249 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 5.8 MB/s | 6.7 MB 00:01 2024-03-26T17:25:01.423 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - AppStream 5.1 MB/s | 6.7 MB 00:01 2024-03-26T17:25:02.400 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-03-26T17:25:02.924 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-26T17:25:02.940 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2024-03-26T17:25:03.524 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-03-26T17:25:03.832 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-26T17:25:05.414 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.0 MB 00:02 2024-03-26T17:25:05.866 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-03-26T17:25:07.325 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-03-26T17:25:09.892 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-26T17:25:11.342 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-03-26T17:25:11.567 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.1 MB 00:00 2024-03-26T17:25:12.984 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-03-26T17:25:13.005 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-26T17:25:14.397 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-26T17:25:17.977 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-03-26T17:25:19.494 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-26T17:25:20.389 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:25:20.389 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:25:20.403 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T17:25:20.405 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T17:25:20.405 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T17:25:20.407 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T17:25:20.409 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T17:25:20.411 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T17:25:20.413 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T17:25:20.491 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-03-26T17:25:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-26T17:25:20.503 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-03-26T17:25:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T17:25:20.504 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T17:25:20.505 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:25:20.506 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T17:25:20.507 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T17:25:20.508 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T17:25:20.509 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T17:25:20.510 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T17:25:20.511 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T17:25:20.512 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T17:25:20.513 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T17:25:20.514 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T17:25:20.515 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T17:25:20.516 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T17:25:20.517 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T17:25:20.518 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout:Install 150 Packages 2024-03-26T17:25:20.519 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 24 Packages 2024-03-26T17:25:20.520 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:25:20.520 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 225 M 2024-03-26T17:25:20.520 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-03-26T17:25:21.520 INFO:teuthology.orchestra.run.smithi144.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-03-26T17:25:21.728 INFO:teuthology.orchestra.run.smithi144.stdout:(2/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 4.2 MB/s | 890 kB 00:00 2024-03-26T17:25:21.829 INFO:teuthology.orchestra.run.smithi144.stdout:(3/174): ceph-immutable-object-cache-19.0.0-231 1.4 MB/s | 145 kB 00:00 2024-03-26T17:25:21.920 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:25:21.921 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:25:21.936 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T17:25:21.937 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T17:25:21.938 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T17:25:21.940 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T17:25:21.942 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T17:25:21.945 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T17:25:21.946 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T17:25:21.997 INFO:teuthology.orchestra.run.smithi144.stdout:(4/174): ceph-base-19.0.0-2311.g36051606.el9.x8 7.7 MB/s | 5.4 MB 00:00 2024-03-26T17:25:22.028 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T17:25:22.043 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T17:25:22.044 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T17:25:22.045 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T17:25:22.046 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T17:25:22.047 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T17:25:22.048 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T17:25:22.049 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T17:25:22.050 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T17:25:22.051 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T17:25:22.052 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T17:25:22.053 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T17:25:22.054 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T17:25:22.055 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout:Install 150 Packages 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 24 Packages 2024-03-26T17:25:22.056 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:25:22.057 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 225 M 2024-03-26T17:25:22.057 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-26T17:25:22.062 INFO:teuthology.orchestra.run.smithi144.stdout:(5/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 10 MB/s | 2.4 MB 00:00 2024-03-26T17:25:22.129 INFO:teuthology.orchestra.run.smithi144.stdout:(6/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 11 MB/s | 1.5 MB 00:00 2024-03-26T17:25:22.655 INFO:teuthology.orchestra.run.smithi144.stdout:(7/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 7.9 MB/s | 4.7 MB 00:00 2024-03-26T17:25:23.029 INFO:teuthology.orchestra.run.smithi105.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 45 kB/s | 6.6 kB 00:00 2024-03-26T17:25:23.396 INFO:teuthology.orchestra.run.smithi105.stdout:(2/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 2.4 MB/s | 890 kB 00:00 2024-03-26T17:25:23.463 INFO:teuthology.orchestra.run.smithi105.stdout:(3/174): ceph-immutable-object-cache-19.0.0-231 2.1 MB/s | 145 kB 00:00 2024-03-26T17:25:23.697 INFO:teuthology.orchestra.run.smithi144.stdout:(8/174): ceph-osd-19.0.0-2311.g36051606.el9.x86 11 MB/s | 17 MB 00:01 2024-03-26T17:25:23.788 INFO:teuthology.orchestra.run.smithi144.stdout:(9/174): ceph-selinux-19.0.0-2311.g36051606.el9 275 kB/s | 25 kB 00:00 2024-03-26T17:25:24.021 INFO:teuthology.orchestra.run.smithi105.stdout:(4/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 4.2 MB/s | 2.4 MB 00:00 2024-03-26T17:25:24.371 INFO:teuthology.orchestra.run.smithi105.stdout:(5/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 4.3 MB/s | 1.5 MB 00:00 2024-03-26T17:25:24.589 INFO:teuthology.orchestra.run.smithi105.stdout:(6/174): ceph-common-19.0.0-2311.g36051606.el9. 12 MB/s | 21 MB 00:01 2024-03-26T17:25:24.947 INFO:teuthology.orchestra.run.smithi105.stdout:(7/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-03-26T17:25:25.772 INFO:teuthology.orchestra.run.smithi144.stdout:(10/174): ceph-radosgw-19.0.0-2311.g36051606.el 3.2 MB/s | 9.8 MB 00:03 2024-03-26T17:25:25.939 INFO:teuthology.orchestra.run.smithi144.stdout:(11/174): cephfs-mirror-19.0.0-2311.g36051606.e 1.3 MB/s | 221 kB 00:00 2024-03-26T17:25:26.041 INFO:teuthology.orchestra.run.smithi144.stdout:(12/174): cephfs-shell-19.0.0-2311.g36051606.el 271 kB/s | 27 kB 00:00 2024-03-26T17:25:26.114 INFO:teuthology.orchestra.run.smithi144.stdout:(13/174): libcephfs-devel-19.0.0-2311.g36051606 424 kB/s | 32 kB 00:00 2024-03-26T17:25:26.323 INFO:teuthology.orchestra.run.smithi144.stdout:(14/174): libcephfs2-19.0.0-2311.g36051606.el9. 3.5 MB/s | 746 kB 00:00 2024-03-26T17:25:26.364 INFO:teuthology.orchestra.run.smithi105.stdout:(8/174): ceph-radosgw-19.0.0-2311.g36051606.el9 6.9 MB/s | 9.8 MB 00:01 2024-03-26T17:25:26.390 INFO:teuthology.orchestra.run.smithi144.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 2.4 MB/s | 163 kB 00:00 2024-03-26T17:25:26.439 INFO:teuthology.orchestra.run.smithi105.stdout:(9/174): ceph-base-19.0.0-2311.g36051606.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-03-26T17:25:26.464 INFO:teuthology.orchestra.run.smithi105.stdout:(10/174): ceph-selinux-19.0.0-2311.g36051606.el 251 kB/s | 25 kB 00:00 2024-03-26T17:25:26.465 INFO:teuthology.orchestra.run.smithi144.stdout:(16/174): librados-devel-19.0.0-2311.g36051606. 1.7 MB/s | 127 kB 00:00 2024-03-26T17:25:26.631 INFO:teuthology.orchestra.run.smithi105.stdout:(11/174): ceph-osd-19.0.0-2311.g36051606.el9.x8 8.2 MB/s | 17 MB 00:02 2024-03-26T17:25:26.632 INFO:teuthology.orchestra.run.smithi144.stdout:(17/174): libradosstriper1-19.0.0-2311.g3605160 2.9 MB/s | 500 kB 00:00 2024-03-26T17:25:26.656 INFO:teuthology.orchestra.run.smithi105.stdout:(12/174): cephfs-mirror-19.0.0-2311.g36051606.e 1.1 MB/s | 221 kB 00:00 2024-03-26T17:25:26.698 INFO:teuthology.orchestra.run.smithi105.stdout:(13/174): cephfs-shell-19.0.0-2311.g36051606.el 407 kB/s | 27 kB 00:00 2024-03-26T17:25:26.724 INFO:teuthology.orchestra.run.smithi105.stdout:(14/174): libcephfs-devel-19.0.0-2311.g36051606 475 kB/s | 32 kB 00:00 2024-03-26T17:25:26.816 INFO:teuthology.orchestra.run.smithi105.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 1.7 MB/s | 163 kB 00:00 2024-03-26T17:25:26.866 INFO:teuthology.orchestra.run.smithi105.stdout:(16/174): libcephfs2-19.0.0-2311.g36051606.el9. 4.4 MB/s | 746 kB 00:00 2024-03-26T17:25:26.893 INFO:teuthology.orchestra.run.smithi105.stdout:(17/174): librados-devel-19.0.0-2311.g36051606. 1.6 MB/s | 127 kB 00:00 2024-03-26T17:25:26.941 INFO:teuthology.orchestra.run.smithi105.stdout:(18/174): libradosstriper1-19.0.0-2311.g3605160 6.5 MB/s | 500 kB 00:00 2024-03-26T17:25:27.517 INFO:teuthology.orchestra.run.smithi105.stdout:(19/174): python3-ceph-argparse-19.0.0-2311.g36 78 kB/s | 45 kB 00:00 2024-03-26T17:25:27.626 INFO:teuthology.orchestra.run.smithi105.stdout:(20/174): librgw2-19.0.0-2311.g36051606.el9.x86 7.0 MB/s | 5.1 MB 00:00 2024-03-26T17:25:27.659 INFO:teuthology.orchestra.run.smithi105.stdout:(21/174): python3-ceph-common-19.0.0-2311.g3605 926 kB/s | 131 kB 00:00 2024-03-26T17:25:27.718 INFO:teuthology.orchestra.run.smithi105.stdout:(22/174): python3-cephfs-19.0.0-2311.g36051606. 1.7 MB/s | 161 kB 00:00 2024-03-26T17:25:27.785 INFO:teuthology.orchestra.run.smithi105.stdout:(23/174): python3-rados-19.0.0-2311.g36051606.e 2.5 MB/s | 321 kB 00:00 2024-03-26T17:25:27.810 INFO:teuthology.orchestra.run.smithi105.stdout:(24/174): python3-rbd-19.0.0-2311.g36051606.el9 3.2 MB/s | 298 kB 00:00 2024-03-26T17:25:27.852 INFO:teuthology.orchestra.run.smithi105.stdout:(25/174): python3-rgw-19.0.0-2311.g36051606.el9 1.5 MB/s | 99 kB 00:00 2024-03-26T17:25:27.902 INFO:teuthology.orchestra.run.smithi105.stdout:(26/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 925 kB/s | 85 kB 00:00 2024-03-26T17:25:28.002 INFO:teuthology.orchestra.run.smithi105.stdout:(27/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-26T17:25:28.057 INFO:teuthology.orchestra.run.smithi144.stdout:(18/174): librgw2-19.0.0-2311.g36051606.el9.x86 3.6 MB/s | 5.1 MB 00:01 2024-03-26T17:25:28.061 INFO:teuthology.orchestra.run.smithi105.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2311.g 585 kB/s | 34 kB 00:00 2024-03-26T17:25:28.117 INFO:teuthology.orchestra.run.smithi144.stdout:(19/174): python3-ceph-argparse-19.0.0-2311.g36 775 kB/s | 45 kB 00:00 2024-03-26T17:25:28.169 INFO:teuthology.orchestra.run.smithi105.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 1.3 MB/s | 140 kB 00:00 2024-03-26T17:25:28.208 INFO:teuthology.orchestra.run.smithi144.stdout:(20/174): python3-ceph-common-19.0.0-2311.g3605 1.4 MB/s | 131 kB 00:00 2024-03-26T17:25:28.270 INFO:teuthology.orchestra.run.smithi105.stdout:(30/174): rbd-mirror-19.0.0-2311.g36051606.el9. 7.4 MB/s | 3.1 MB 00:00 2024-03-26T17:25:28.291 INFO:teuthology.orchestra.run.smithi144.stdout:(21/174): python3-cephfs-19.0.0-2311.g36051606. 1.9 MB/s | 161 kB 00:00 2024-03-26T17:25:28.437 INFO:teuthology.orchestra.run.smithi105.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2311.g36051 6.7 MB/s | 1.8 MB 00:00 2024-03-26T17:25:28.475 INFO:teuthology.orchestra.run.smithi144.stdout:(22/174): python3-rados-19.0.0-2311.g36051606.e 1.7 MB/s | 321 kB 00:00 2024-03-26T17:25:28.529 INFO:teuthology.orchestra.run.smithi105.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2311.g36 2.6 MB/s | 248 kB 00:00 2024-03-26T17:25:28.621 INFO:teuthology.orchestra.run.smithi105.stdout:(33/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 539 kB/s | 49 kB 00:00 2024-03-26T17:25:28.625 INFO:teuthology.orchestra.run.smithi144.stdout:(23/174): python3-rbd-19.0.0-2311.g36051606.el9 1.9 MB/s | 298 kB 00:00 2024-03-26T17:25:28.679 INFO:teuthology.orchestra.run.smithi105.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2311.g3 283 kB/s | 16 kB 00:00 2024-03-26T17:25:28.687 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 870 kB/s | 19 MB 00:22 2024-03-26T17:25:28.726 INFO:teuthology.orchestra.run.smithi144.stdout:(24/174): python3-rgw-19.0.0-2311.g36051606.el9 996 kB/s | 99 kB 00:00 2024-03-26T17:25:28.771 INFO:teuthology.orchestra.run.smithi105.stdout:(35/174): ceph-volume-19.0.0-2311.g36051606.el9 2.8 MB/s | 264 kB 00:00 2024-03-26T17:25:28.809 INFO:teuthology.orchestra.run.smithi144.stdout:(25/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-26T17:25:28.947 INFO:teuthology.orchestra.run.smithi105.stdout:(36/174): cephadm-19.0.0-2311.g36051606.el9.noa 4.2 MB/s | 750 kB 00:00 2024-03-26T17:25:29.022 INFO:teuthology.orchestra.run.smithi105.stdout:(37/174): cephfs-top-19.0.0-2311.g36051606.el9. 244 kB/s | 18 kB 00:00 2024-03-26T17:25:29.189 INFO:teuthology.orchestra.run.smithi105.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 662 kB 00:00 2024-03-26T17:25:29.264 INFO:teuthology.orchestra.run.smithi105.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 920 kB 00:00 2024-03-26T17:25:29.314 INFO:teuthology.orchestra.run.smithi105.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-26T17:25:29.398 INFO:teuthology.orchestra.run.smithi105.stdout:(41/174): ceph-mgr-diskprediction-local-19.0.0- 6.6 MB/s | 7.4 MB 00:01 2024-03-26T17:25:29.423 INFO:teuthology.orchestra.run.smithi105.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 307 kB 00:00 2024-03-26T17:25:29.449 INFO:teuthology.orchestra.run.smithi105.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-26T17:25:29.582 INFO:teuthology.orchestra.run.smithi105.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-03-26T17:25:29.608 INFO:teuthology.orchestra.run.smithi105.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 108 kB/s | 17 kB 00:00 2024-03-26T17:25:29.609 INFO:teuthology.orchestra.run.smithi144.stdout:(26/174): rbd-mirror-19.0.0-2311.g36051606.el9. 3.9 MB/s | 3.1 MB 00:00 2024-03-26T17:25:29.624 INFO:teuthology.orchestra.run.smithi105.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 241 kB/s | 10 kB 00:00 2024-03-26T17:25:29.667 INFO:teuthology.orchestra.run.smithi105.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-26T17:25:29.668 INFO:teuthology.orchestra.run.smithi144.stdout:(27/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-26T17:25:29.684 INFO:teuthology.orchestra.run.smithi105.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 750 kB/s | 44 kB 00:00 2024-03-26T17:25:29.718 INFO:teuthology.orchestra.run.smithi105.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-26T17:25:29.754 INFO:teuthology.orchestra.run.smithi105.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.4 MB/s | 578 kB 00:00 2024-03-26T17:25:29.776 INFO:teuthology.orchestra.run.smithi105.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 807 kB/s | 46 kB 00:00 2024-03-26T17:25:29.801 INFO:teuthology.orchestra.run.smithi105.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-26T17:25:29.827 INFO:teuthology.orchestra.run.smithi105.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T17:25:29.852 INFO:teuthology.orchestra.run.smithi105.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-26T17:25:29.877 INFO:teuthology.orchestra.run.smithi105.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 683 kB/s | 34 kB 00:00 2024-03-26T17:25:30.003 INFO:teuthology.orchestra.run.smithi105.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-26T17:25:30.036 INFO:teuthology.orchestra.run.smithi105.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-26T17:25:30.061 INFO:teuthology.orchestra.run.smithi105.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T17:25:30.093 INFO:teuthology.orchestra.run.smithi144.stdout:(28/174): ceph-test-19.0.0-2311.g36051606.el9.x 7.6 MB/s | 48 MB 00:06 2024-03-26T17:25:30.110 INFO:teuthology.orchestra.run.smithi144.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2311.g 77 kB/s | 34 kB 00:00 2024-03-26T17:25:30.144 INFO:teuthology.orchestra.run.smithi144.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 2.7 MB/s | 140 kB 00:00 2024-03-26T17:25:30.229 INFO:teuthology.orchestra.run.smithi105.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-03-26T17:25:30.254 INFO:teuthology.orchestra.run.smithi105.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 128 kB/s | 24 kB 00:00 2024-03-26T17:25:30.279 INFO:teuthology.orchestra.run.smithi105.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-03-26T17:25:30.304 INFO:teuthology.orchestra.run.smithi105.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-26T17:25:30.330 INFO:teuthology.orchestra.run.smithi105.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 630 kB/s | 32 kB 00:00 2024-03-26T17:25:30.444 INFO:teuthology.orchestra.run.smithi144.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2311.g36051 5.4 MB/s | 1.8 MB 00:00 2024-03-26T17:25:30.513 INFO:teuthology.orchestra.run.smithi105.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-03-26T17:25:30.538 INFO:teuthology.orchestra.run.smithi105.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-03-26T17:25:30.544 INFO:teuthology.orchestra.run.smithi144.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2311.g36 2.4 MB/s | 248 kB 00:00 2024-03-26T17:25:30.589 INFO:teuthology.orchestra.run.smithi105.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.3 MB/s | 174 kB 00:00 2024-03-26T17:25:30.614 INFO:teuthology.orchestra.run.smithi105.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 912 kB/s | 69 kB 00:00 2024-03-26T17:25:30.648 INFO:teuthology.orchestra.run.smithi105.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 706 kB/s | 41 kB 00:00 2024-03-26T17:25:30.661 INFO:teuthology.orchestra.run.smithi144.stdout:(33/174): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-03-26T17:25:30.678 INFO:teuthology.orchestra.run.smithi144.stdout:(34/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 369 kB/s | 49 kB 00:00 2024-03-26T17:25:30.681 INFO:teuthology.orchestra.run.smithi105.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-26T17:25:30.703 INFO:teuthology.orchestra.run.smithi144.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2311.g3 393 kB/s | 16 kB 00:00 2024-03-26T17:25:30.707 INFO:teuthology.orchestra.run.smithi105.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-26T17:25:30.749 INFO:teuthology.orchestra.run.smithi105.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-26T17:25:30.753 INFO:teuthology.orchestra.run.smithi144.stdout:(36/174): ceph-volume-19.0.0-2311.g36051606.el9 3.4 MB/s | 264 kB 00:00 2024-03-26T17:25:30.774 INFO:teuthology.orchestra.run.smithi105.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 707 kB/s | 47 kB 00:00 2024-03-26T17:25:30.799 INFO:teuthology.orchestra.run.smithi105.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 854 kB/s | 43 kB 00:00 2024-03-26T17:25:30.812 INFO:teuthology.orchestra.run.smithi144.stdout:(37/174): cephadm-19.0.0-2311.g36051606.el9.noa 6.8 MB/s | 750 kB 00:00 2024-03-26T17:25:30.829 INFO:teuthology.orchestra.run.smithi144.stdout:(38/174): cephfs-top-19.0.0-2311.g36051606.el9. 242 kB/s | 18 kB 00:00 2024-03-26T17:25:31.066 INFO:teuthology.orchestra.run.smithi105.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 145 kB/s | 39 kB 00:00 2024-03-26T17:25:31.254 INFO:teuthology.orchestra.run.smithi144.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2024-03-26T17:25:31.321 INFO:teuthology.orchestra.run.smithi144.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 920 kB 00:00 2024-03-26T17:25:31.333 INFO:teuthology.orchestra.run.smithi105.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-03-26T17:25:31.347 INFO:teuthology.orchestra.run.smithi144.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 106 kB 00:00 2024-03-26T17:25:31.359 INFO:teuthology.orchestra.run.smithi105.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 141 kB/s | 41 kB 00:00 2024-03-26T17:25:31.402 INFO:teuthology.orchestra.run.smithi105.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2024-03-26T17:25:31.405 INFO:teuthology.orchestra.run.smithi144.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-26T17:25:31.431 INFO:teuthology.orchestra.run.smithi144.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 379 kB/s | 32 kB 00:00 2024-03-26T17:25:31.434 INFO:teuthology.orchestra.run.smithi105.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 749 kB/s | 56 kB 00:00 2024-03-26T17:25:31.468 INFO:teuthology.orchestra.run.smithi105.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 940 kB/s | 63 kB 00:00 2024-03-26T17:25:31.510 INFO:teuthology.orchestra.run.smithi105.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-03-26T17:25:31.531 INFO:teuthology.orchestra.run.smithi144.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 171 kB/s | 17 kB 00:00 2024-03-26T17:25:31.535 INFO:teuthology.orchestra.run.smithi105.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 79 kB 00:00 2024-03-26T17:25:31.586 INFO:teuthology.orchestra.run.smithi105.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-26T17:25:31.623 INFO:teuthology.orchestra.run.smithi144.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-03-26T17:25:31.627 INFO:teuthology.orchestra.run.smithi105.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-26T17:25:31.648 INFO:teuthology.orchestra.run.smithi144.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 87 kB/s | 10 kB 00:00 2024-03-26T17:25:31.653 INFO:teuthology.orchestra.run.smithi105.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-26T17:25:31.673 INFO:teuthology.orchestra.run.smithi144.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-26T17:25:31.678 INFO:teuthology.orchestra.run.smithi105.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-26T17:25:31.703 INFO:teuthology.orchestra.run.smithi105.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-26T17:25:31.707 INFO:teuthology.orchestra.run.smithi144.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 756 kB/s | 44 kB 00:00 2024-03-26T17:25:31.732 INFO:teuthology.orchestra.run.smithi144.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-26T17:25:31.774 INFO:teuthology.orchestra.run.smithi144.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2024-03-26T17:25:31.799 INFO:teuthology.orchestra.run.smithi144.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 699 kB/s | 46 kB 00:00 2024-03-26T17:25:31.825 INFO:teuthology.orchestra.run.smithi144.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-26T17:25:31.850 INFO:teuthology.orchestra.run.smithi144.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T17:25:31.862 INFO:teuthology.orchestra.run.smithi105.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-26T17:25:31.875 INFO:teuthology.orchestra.run.smithi144.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-26T17:25:31.900 INFO:teuthology.orchestra.run.smithi144.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2024-03-26T17:25:31.912 INFO:teuthology.orchestra.run.smithi105.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-03-26T17:25:31.987 INFO:teuthology.orchestra.run.smithi105.stdout:(89/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-03-26T17:25:32.060 INFO:teuthology.orchestra.run.smithi144.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-03-26T17:25:32.093 INFO:teuthology.orchestra.run.smithi144.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-03-26T17:25:32.096 INFO:teuthology.orchestra.run.smithi105.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-03-26T17:25:32.118 INFO:teuthology.orchestra.run.smithi144.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T17:25:32.252 INFO:teuthology.orchestra.run.smithi144.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 37 MB/s | 5.8 MB 00:00 2024-03-26T17:25:32.269 INFO:teuthology.orchestra.run.smithi144.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 164 kB/s | 24 kB 00:00 2024-03-26T17:25:32.286 INFO:teuthology.orchestra.run.smithi144.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 541 kB/s | 18 kB 00:00 2024-03-26T17:25:32.311 INFO:teuthology.orchestra.run.smithi144.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-26T17:25:32.328 INFO:teuthology.orchestra.run.smithi144.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 760 kB/s | 32 kB 00:00 2024-03-26T17:25:32.436 INFO:teuthology.orchestra.run.smithi144.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-26T17:25:32.462 INFO:teuthology.orchestra.run.smithi144.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-26T17:25:32.487 INFO:teuthology.orchestra.run.smithi144.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-26T17:25:32.504 INFO:teuthology.orchestra.run.smithi144.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-03-26T17:25:32.529 INFO:teuthology.orchestra.run.smithi144.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 987 kB/s | 41 kB 00:00 2024-03-26T17:25:32.563 INFO:teuthology.orchestra.run.smithi144.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-26T17:25:32.588 INFO:teuthology.orchestra.run.smithi144.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-26T17:25:32.614 INFO:teuthology.orchestra.run.smithi144.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-26T17:25:32.639 INFO:teuthology.orchestra.run.smithi144.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-03-26T17:25:32.655 INFO:teuthology.orchestra.run.smithi105.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 32 kB/s | 21 kB 00:00 2024-03-26T17:25:32.664 INFO:teuthology.orchestra.run.smithi144.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-03-26T17:25:32.772 INFO:teuthology.orchestra.run.smithi105.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 38 kB/s | 25 kB 00:00 2024-03-26T17:25:32.813 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.1 MB 00:00 2024-03-26T17:25:32.831 INFO:teuthology.orchestra.run.smithi144.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 237 kB/s | 39 kB 00:00 2024-03-26T17:25:33.006 INFO:teuthology.orchestra.run.smithi144.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-03-26T17:25:33.032 INFO:teuthology.orchestra.run.smithi144.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 207 kB/s | 41 kB 00:00 2024-03-26T17:25:33.066 INFO:teuthology.orchestra.run.smithi144.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-26T17:25:33.107 INFO:teuthology.orchestra.run.smithi144.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 752 kB/s | 56 kB 00:00 2024-03-26T17:25:33.133 INFO:teuthology.orchestra.run.smithi144.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 938 kB/s | 63 kB 00:00 2024-03-26T17:25:33.158 INFO:teuthology.orchestra.run.smithi144.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-03-26T17:25:33.183 INFO:teuthology.orchestra.run.smithi144.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-26T17:25:33.208 INFO:teuthology.orchestra.run.smithi144.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-26T17:25:33.242 INFO:teuthology.orchestra.run.smithi144.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-26T17:25:33.259 INFO:teuthology.orchestra.run.smithi144.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2024-03-26T17:25:33.284 INFO:teuthology.orchestra.run.smithi144.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-03-26T17:25:33.294 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-03-26T17:25:33.318 INFO:teuthology.orchestra.run.smithi144.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-03-26T17:25:33.552 INFO:teuthology.orchestra.run.smithi144.stdout:(87/174): ceph-common-19.0.0-2311.g36051606.el9 1.7 MB/s | 21 MB 00:12 2024-03-26T17:25:33.619 INFO:teuthology.orchestra.run.smithi144.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 561 kB/s | 187 kB 00:00 2024-03-26T17:25:33.682 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] postgresql-13.14-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/postgresql-13.14-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-03-26T17:25:33.682 INFO:teuthology.orchestra.run.smithi144.stdout:(89/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-03-26T17:25:33.694 INFO:teuthology.orchestra.run.smithi144.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-03-26T17:25:33.736 INFO:teuthology.orchestra.run.smithi144.stdout:(91/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-03-26T17:25:33.813 INFO:teuthology.orchestra.run.smithi105.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-03-26T17:25:33.970 INFO:teuthology.orchestra.run.smithi144.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 71 kB/s | 21 kB 00:00 2024-03-26T17:25:33.995 INFO:teuthology.orchestra.run.smithi144.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 85 kB/s | 25 kB 00:00 2024-03-26T17:25:34.160 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-03-26T17:25:34.687 INFO:teuthology.orchestra.run.smithi144.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-03-26T17:25:34.701 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] postgresql-contrib-13.14-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.14-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-03-26T17:25:34.745 INFO:teuthology.orchestra.run.smithi144.stdout:(95/174): postgresql-contrib-13.14-1.el9.x86_64 1.2 MB/s | 911 kB 00:00 2024-03-26T17:25:34.805 INFO:teuthology.orchestra.run.smithi105.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 919 kB/s | 911 kB 00:00 2024-03-26T17:25:34.812 INFO:teuthology.orchestra.run.smithi144.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-03-26T17:25:34.954 INFO:teuthology.orchestra.run.smithi144.stdout:(97/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.4 MB/s | 4.1 MB 00:01 2024-03-26T17:25:34.979 INFO:teuthology.orchestra.run.smithi144.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-26T17:25:35.071 INFO:teuthology.orchestra.run.smithi144.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-03-26T17:25:35.105 INFO:teuthology.orchestra.run.smithi144.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-03-26T17:25:35.152 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] postgresql-private-libs-13.14-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.14-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-03-26T17:25:35.189 INFO:teuthology.orchestra.run.smithi144.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 307 kB/s | 36 kB 00:00 2024-03-26T17:25:35.197 INFO:teuthology.orchestra.run.smithi105.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 340 kB/s | 133 kB 00:00 2024-03-26T17:25:35.222 INFO:teuthology.orchestra.run.smithi144.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-03-26T17:25:35.306 INFO:teuthology.orchestra.run.smithi144.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 802 kB/s | 94 kB 00:00 2024-03-26T17:25:35.331 INFO:teuthology.orchestra.run.smithi144.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-03-26T17:25:35.373 INFO:teuthology.orchestra.run.smithi105.stdout:(96/174): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-03-26T17:25:35.423 INFO:teuthology.orchestra.run.smithi105.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-03-26T17:25:35.465 INFO:teuthology.orchestra.run.smithi105.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-03-26T17:25:35.507 INFO:teuthology.orchestra.run.smithi105.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-03-26T17:25:35.531 INFO:teuthology.orchestra.run.smithi144.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-03-26T17:25:35.549 INFO:teuthology.orchestra.run.smithi105.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 862 kB/s | 36 kB 00:00 2024-03-26T17:25:35.565 INFO:teuthology.orchestra.run.smithi144.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-03-26T17:25:35.591 INFO:teuthology.orchestra.run.smithi105.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-03-26T17:25:35.599 INFO:teuthology.orchestra.run.smithi144.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-03-26T17:25:35.613 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 11 kB/s | 16 kB 00:01 2024-03-26T17:25:35.624 INFO:teuthology.orchestra.run.smithi144.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 336 kB/s | 19 kB 00:00 2024-03-26T17:25:35.624 INFO:teuthology.orchestra.run.smithi105.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.8 MB/s | 94 kB 00:00 2024-03-26T17:25:35.774 INFO:teuthology.orchestra.run.smithi144.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-03-26T17:25:35.799 INFO:teuthology.orchestra.run.smithi144.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 149 kB/s | 26 kB 00:00 2024-03-26T17:25:35.800 INFO:teuthology.orchestra.run.smithi105.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-03-26T17:25:35.825 INFO:teuthology.orchestra.run.smithi144.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2024-03-26T17:25:35.850 INFO:teuthology.orchestra.run.smithi144.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-03-26T17:25:35.875 INFO:teuthology.orchestra.run.smithi144.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-03-26T17:25:35.917 INFO:teuthology.orchestra.run.smithi144.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-03-26T17:25:35.942 INFO:teuthology.orchestra.run.smithi144.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-03-26T17:25:35.968 INFO:teuthology.orchestra.run.smithi144.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 884 kB/s | 43 kB 00:00 2024-03-26T17:25:35.993 INFO:teuthology.orchestra.run.smithi144.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-03-26T17:25:36.008 INFO:teuthology.orchestra.run.smithi105.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-03-26T17:25:36.026 INFO:teuthology.orchestra.run.smithi144.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-03-26T17:25:36.052 INFO:teuthology.orchestra.run.smithi144.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-26T17:25:36.077 INFO:teuthology.orchestra.run.smithi144.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-03-26T17:25:36.119 INFO:teuthology.orchestra.run.smithi144.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-03-26T17:25:36.144 INFO:teuthology.orchestra.run.smithi144.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 529 kB/s | 35 kB 00:00 2024-03-26T17:25:36.186 INFO:teuthology.orchestra.run.smithi144.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.0 MB/s | 203 kB 00:00 2024-03-26T17:25:36.245 INFO:teuthology.orchestra.run.smithi144.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-03-26T17:25:36.270 INFO:teuthology.orchestra.run.smithi144.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-03-26T17:25:36.296 INFO:teuthology.orchestra.run.smithi144.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-03-26T17:25:36.321 INFO:teuthology.orchestra.run.smithi144.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-03-26T17:25:36.346 INFO:teuthology.orchestra.run.smithi144.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-03-26T17:25:36.371 INFO:teuthology.orchestra.run.smithi144.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-03-26T17:25:36.397 INFO:teuthology.orchestra.run.smithi144.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-03-26T17:25:36.439 INFO:teuthology.orchestra.run.smithi144.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-03-26T17:25:36.464 INFO:teuthology.orchestra.run.smithi144.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 696 kB/s | 46 kB 00:00 2024-03-26T17:25:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-26T17:25:36.531 INFO:teuthology.orchestra.run.smithi144.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-03-26T17:25:36.573 INFO:teuthology.orchestra.run.smithi144.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-26T17:25:36.592 INFO:teuthology.orchestra.run.smithi105.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 940 kB/s | 548 kB 00:00 2024-03-26T17:25:36.598 INFO:teuthology.orchestra.run.smithi144.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-03-26T17:25:36.640 INFO:teuthology.orchestra.run.smithi144.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-26T17:25:36.682 INFO:teuthology.orchestra.run.smithi144.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-03-26T17:25:36.716 INFO:teuthology.orchestra.run.smithi144.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-03-26T17:25:36.758 INFO:teuthology.orchestra.run.smithi144.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-26T17:25:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-03-26T17:25:36.834 INFO:teuthology.orchestra.run.smithi144.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-03-26T17:25:36.859 INFO:teuthology.orchestra.run.smithi144.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 762 kB/s | 51 kB 00:00 2024-03-26T17:25:36.901 INFO:teuthology.orchestra.run.smithi144.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-03-26T17:25:36.934 INFO:teuthology.orchestra.run.smithi144.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-26T17:25:36.976 INFO:teuthology.orchestra.run.smithi144.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-03-26T17:25:37.010 INFO:teuthology.orchestra.run.smithi144.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-03-26T17:25:37.060 INFO:teuthology.orchestra.run.smithi144.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-26T17:25:37.100 INFO:teuthology.orchestra.run.smithi105.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 608 kB/s | 309 kB 00:00 2024-03-26T17:25:37.367 INFO:teuthology.orchestra.run.smithi105.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-03-26T17:25:37.470 INFO:teuthology.orchestra.run.smithi144.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-03-26T17:25:37.711 INFO:teuthology.orchestra.run.smithi144.stdout:(150/174): postgresql-server-13.14-1.el9.x86_64 1.9 MB/s | 5.8 MB 00:02 2024-03-26T17:25:37.753 INFO:teuthology.orchestra.run.smithi144.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-26T17:25:37.936 INFO:teuthology.orchestra.run.smithi144.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-26T17:25:37.987 INFO:teuthology.orchestra.run.smithi144.stdout:(153/174): librbd1-19.0.0-2311.g36051606.el9.x8 6.1 MB/s | 3.1 MB 00:00 2024-03-26T17:25:38.028 INFO:teuthology.orchestra.run.smithi144.stdout:(154/174): NetworkManager-libnm-1.47.2-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-03-26T17:25:38.062 INFO:teuthology.orchestra.run.smithi144.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 561 kB/s | 42 kB 00:00 2024-03-26T17:25:38.087 INFO:teuthology.orchestra.run.smithi144.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-26T17:25:38.121 INFO:teuthology.orchestra.run.smithi144.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-26T17:25:38.146 INFO:teuthology.orchestra.run.smithi144.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 624 kB/s | 36 kB 00:00 2024-03-26T17:25:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-26T17:25:38.205 INFO:teuthology.orchestra.run.smithi144.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-03-26T17:25:38.230 INFO:teuthology.orchestra.run.smithi144.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 460 kB/s | 23 kB 00:00 2024-03-26T17:25:38.255 INFO:teuthology.orchestra.run.smithi144.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-26T17:25:38.281 INFO:teuthology.orchestra.run.smithi144.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-26T17:25:38.306 INFO:teuthology.orchestra.run.smithi144.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-26T17:25:38.343 INFO:teuthology.orchestra.run.smithi105.stdout:(108/174): ceph-test-19.0.0-2311.g36051606.el9. 4.0 MB/s | 48 MB 00:11 2024-03-26T17:25:38.356 INFO:teuthology.orchestra.run.smithi144.stdout:(165/174): librados2-19.0.0-2311.g36051606.el9. 2.6 MB/s | 3.4 MB 00:01 2024-03-26T17:25:38.382 INFO:teuthology.orchestra.run.smithi144.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 635 kB/s | 64 kB 00:00 2024-03-26T17:25:38.407 INFO:teuthology.orchestra.run.smithi144.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 285 kB/s | 29 kB 00:00 2024-03-26T17:25:38.482 INFO:teuthology.orchestra.run.smithi144.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 29 MB/s | 2.2 MB 00:00 2024-03-26T17:25:38.541 INFO:teuthology.orchestra.run.smithi144.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 17 MB/s | 2.3 MB 00:00 2024-03-26T17:25:38.566 INFO:teuthology.orchestra.run.smithi144.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 5.4 MB/s | 465 kB 00:00 2024-03-26T17:25:38.591 INFO:teuthology.orchestra.run.smithi144.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 621 kB/s | 31 kB 00:00 2024-03-26T17:25:38.601 INFO:teuthology.orchestra.run.smithi105.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 101 kB/s | 26 kB 00:00 2024-03-26T17:25:38.625 INFO:teuthology.orchestra.run.smithi144.stdout:(172/174): openssl-3.0.7-27.el9.x86_64.rpm 4.5 MB/s | 1.2 MB 00:00 2024-03-26T17:25:38.658 INFO:teuthology.orchestra.run.smithi144.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-03-26T17:25:38.850 INFO:teuthology.orchestra.run.smithi144.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 287 kB/s | 74 kB 00:00 2024-03-26T17:25:38.856 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:25:38.856 INFO:teuthology.orchestra.run.smithi144.stdout:Total 12 MB/s | 225 MB 00:18 2024-03-26T17:25:38.885 INFO:teuthology.orchestra.run.smithi105.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-03-26T17:25:38.943 INFO:teuthology.orchestra.run.smithi105.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-03-26T17:25:39.019 INFO:teuthology.orchestra.run.smithi105.stdout:(112/174): openssl-devel-3.0.7-27.el9.x86_64.rp 665 kB/s | 4.1 MB 00:06 2024-03-26T17:25:39.286 INFO:teuthology.orchestra.run.smithi105.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 168 kB/s | 67 kB 00:00 2024-03-26T17:25:39.386 INFO:teuthology.orchestra.run.smithi105.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 341 kB/s | 151 kB 00:00 2024-03-26T17:25:39.424 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-03-26T17:25:39.620 INFO:teuthology.orchestra.run.smithi105.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.4 MB/s | 838 kB 00:00 2024-03-26T17:25:39.701 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-03-26T17:25:39.701 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-03-26T17:25:39.728 INFO:teuthology.orchestra.run.smithi105.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-03-26T17:25:39.887 INFO:teuthology.orchestra.run.smithi105.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 120 kB/s | 32 kB 00:00 2024-03-26T17:25:39.912 INFO:teuthology.orchestra.run.smithi105.stdout:(118/174): python3-asyncssh-2.13.2-5.el9.noarch 876 kB/s | 548 kB 00:00 2024-03-26T17:25:39.996 INFO:teuthology.orchestra.run.smithi105.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-03-26T17:25:40.321 INFO:teuthology.orchestra.run.smithi105.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 396 kB/s | 172 kB 00:00 2024-03-26T17:25:40.430 INFO:teuthology.orchestra.run.smithi105.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 693 kB/s | 358 kB 00:00 2024-03-26T17:25:40.522 INFO:teuthology.orchestra.run.smithi105.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 577 kB/s | 303 kB 00:00 2024-03-26T17:25:40.589 INFO:teuthology.orchestra.run.smithi105.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-03-26T17:25:40.947 INFO:teuthology.orchestra.run.smithi105.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 392 kB/s | 203 kB 00:00 2024-03-26T17:25:41.014 INFO:teuthology.orchestra.run.smithi105.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-03-26T17:25:41.131 INFO:teuthology.orchestra.run.smithi105.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-03-26T17:25:41.281 INFO:teuthology.orchestra.run.smithi105.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-03-26T17:25:41.381 INFO:teuthology.orchestra.run.smithi105.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-03-26T17:25:41.407 INFO:teuthology.orchestra.run.smithi105.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 84 kB/s | 23 kB 00:00 2024-03-26T17:25:41.565 INFO:teuthology.orchestra.run.smithi105.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 68 kB/s | 19 kB 00:00 2024-03-26T17:25:41.640 INFO:teuthology.orchestra.run.smithi105.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 76 kB/s | 20 kB 00:00 2024-03-26T17:25:41.713 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-03-26T17:25:41.713 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-03-26T17:25:41.916 INFO:teuthology.orchestra.run.smithi105.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-03-26T17:25:41.983 INFO:teuthology.orchestra.run.smithi105.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 230 kB/s | 79 kB 00:00 2024-03-26T17:25:42.117 INFO:teuthology.orchestra.run.smithi105.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-03-26T17:25:42.270 INFO:teuthology.orchestra.run.smithi105.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-03-26T17:25:42.384 INFO:teuthology.orchestra.run.smithi105.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-03-26T17:25:42.484 INFO:teuthology.orchestra.run.smithi105.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-03-26T17:25:42.609 INFO:teuthology.orchestra.run.smithi105.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 263 kB/s | 90 kB 00:00 2024-03-26T17:25:42.660 INFO:teuthology.orchestra.run.smithi105.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 102 kB/s | 28 kB 00:00 2024-03-26T17:25:42.760 INFO:teuthology.orchestra.run.smithi105.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 112 kB/s | 31 kB 00:00 2024-03-26T17:25:43.035 INFO:teuthology.orchestra.run.smithi105.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 157 kB/s | 59 kB 00:00 2024-03-26T17:25:43.060 INFO:teuthology.orchestra.run.smithi105.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 119 kB/s | 36 kB 00:00 2024-03-26T17:25:43.085 INFO:teuthology.orchestra.run.smithi105.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 396 kB/s | 188 kB 00:00 2024-03-26T17:25:43.386 INFO:teuthology.orchestra.run.smithi105.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-03-26T17:25:43.427 INFO:teuthology.orchestra.run.smithi105.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-03-26T17:25:43.561 INFO:teuthology.orchestra.run.smithi105.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 459 kB/s | 230 kB 00:00 2024-03-26T17:25:43.695 INFO:teuthology.orchestra.run.smithi105.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-03-26T17:25:43.970 INFO:teuthology.orchestra.run.smithi105.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-03-26T17:25:43.995 INFO:teuthology.orchestra.run.smithi105.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 442 kB/s | 192 kB 00:00 2024-03-26T17:25:44.471 INFO:teuthology.orchestra.run.smithi105.stdout:(150/174): librados2-19.0.0-2311.g36051606.el9. 6.8 MB/s | 3.4 MB 00:00 2024-03-26T17:25:44.513 INFO:teuthology.orchestra.run.smithi105.stdout:(151/174): thrift-0.15.0-2.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-03-26T17:25:44.538 INFO:teuthology.orchestra.run.smithi105.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-03-26T17:25:44.872 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 2.3 MB/s | 21 MB 00:09 2024-03-26T17:25:44.946 INFO:teuthology.orchestra.run.smithi105.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.3 MB/s | 2.3 MB 00:00 2024-03-26T17:25:44.988 INFO:teuthology.orchestra.run.smithi105.stdout:(154/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.1 MB/s | 1.8 MB 00:00 2024-03-26T17:25:45.013 INFO:teuthology.orchestra.run.smithi105.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 631 kB/s | 42 kB 00:00 2024-03-26T17:25:45.055 INFO:teuthology.orchestra.run.smithi105.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 3.6 MB/s | 248 kB 00:00 2024-03-26T17:25:45.080 INFO:teuthology.orchestra.run.smithi105.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-26T17:25:45.148 INFO:teuthology.orchestra.run.smithi105.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 396 kB/s | 36 kB 00:00 2024-03-26T17:25:45.173 INFO:teuthology.orchestra.run.smithi105.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.0 MB/s | 192 kB 00:00 2024-03-26T17:25:45.215 INFO:teuthology.orchestra.run.smithi105.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 256 kB 00:00 2024-03-26T17:25:45.248 INFO:teuthology.orchestra.run.smithi105.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 305 kB/s | 23 kB 00:00 2024-03-26T17:25:45.282 INFO:teuthology.orchestra.run.smithi105.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-26T17:25:45.316 INFO:teuthology.orchestra.run.smithi105.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.3 MB/s | 155 kB 00:00 2024-03-26T17:25:45.349 INFO:teuthology.orchestra.run.smithi105.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-26T17:25:45.408 INFO:teuthology.orchestra.run.smithi105.stdout:(165/174): librbd1-19.0.0-2311.g36051606.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-03-26T17:25:45.433 INFO:teuthology.orchestra.run.smithi105.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 343 kB/s | 29 kB 00:00 2024-03-26T17:25:45.459 INFO:teuthology.orchestra.run.smithi105.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 449 kB/s | 64 kB 00:00 2024-03-26T17:25:45.659 INFO:teuthology.orchestra.run.smithi105.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-26T17:25:45.759 INFO:teuthology.orchestra.run.smithi105.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.6 MB/s | 2.2 MB 00:00 2024-03-26T17:25:45.784 INFO:teuthology.orchestra.run.smithi105.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.6 MB/s | 465 kB 00:00 2024-03-26T17:25:45.809 INFO:teuthology.orchestra.run.smithi105.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 622 kB/s | 31 kB 00:00 2024-03-26T17:25:45.860 INFO:teuthology.orchestra.run.smithi105.stdout:(172/174): openssl-3.0.7-27.el9.x86_64.rpm 2.7 MB/s | 1.2 MB 00:00 2024-03-26T17:25:45.893 INFO:teuthology.orchestra.run.smithi105.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-03-26T17:25:45.944 INFO:teuthology.orchestra.run.smithi105.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 556 kB/s | 74 kB 00:00 2024-03-26T17:25:45.948 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:25:45.948 INFO:teuthology.orchestra.run.smithi105.stdout:Total 9.4 MB/s | 225 MB 00:23 2024-03-26T17:25:46.524 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-26T17:25:46.801 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-26T17:25:46.801 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-26T17:25:48.778 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-03-26T17:25:48.896 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T17:25:48.917 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-26T17:25:48.918 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-26T17:25:48.931 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T17:25:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:25:48.986 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:25:49.033 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T17:25:49.098 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T17:25:49.138 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T17:25:49.164 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T17:25:49.217 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T17:25:49.559 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T17:25:49.612 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:25:49.746 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:25:49.772 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:25:49.862 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:25:49.881 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-26T17:25:49.901 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:25:49.944 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T17:25:50.128 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T17:25:50.191 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T17:25:50.218 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:25:50.321 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:25:50.344 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:25:50.410 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:25:50.449 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T17:25:50.479 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:25:50.495 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:25:50.683 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:25:50.711 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:25:50.784 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T17:25:50.990 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T17:25:51.122 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:25:51.166 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:25:51.259 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:25:51.318 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:25:51.547 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:25:51.658 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T17:25:51.750 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T17:25:51.789 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T17:25:51.815 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T17:25:51.850 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T17:25:51.923 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T17:25:51.978 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T17:25:52.017 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T17:25:52.080 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T17:25:52.117 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T17:25:52.160 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T17:25:52.193 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T17:25:52.220 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T17:25:52.291 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T17:25:52.314 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:25:52.315 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T17:25:52.325 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T17:25:52.329 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T17:25:52.330 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T17:25:52.331 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T17:25:52.332 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T17:25:52.334 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T17:25:52.337 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T17:25:52.338 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T17:25:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T17:25:52.397 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T17:25:52.418 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:19.0.0-2311.g36051606.el9 ceph 6.6 k 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.4 M 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 890 k 2024-03-26T17:25:52.434 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2311.g36051606.el9 ceph 145 k 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:19.0.0-2311.g36051606.el9 ceph 1.5 M 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 140 k 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 1.8 M 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 7.4 M 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 49 k 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 9.8 M 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:19.0.0-2311.g36051606.el9 ceph 48 M 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 264 k 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 750 k 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 221 k 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell x86_64 2:19.0.0-2311.g36051606.el9 ceph 27 k 2024-03-26T17:25:52.435 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 18 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 32 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 746 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:19.0.0-2311.g36051606.el9 ceph 127 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T17:25:52.436 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:19.0.0-2311.g36051606.el9 ceph 161 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:19.0.0-2311.g36051606.el9 ceph 321 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 298 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:19.0.0-2311.g36051606.el9 ceph 99 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:19.0.0-2311.g36051606.el9 ceph 85 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:19.0.0-2311.g36051606.el9 ceph 170 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T17:25:52.437 INFO:teuthology.orchestra.run.smithi135.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.4 M 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 3.1 M 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T17:25:52.438 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 21 M 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 34 k 2024-03-26T17:25:52.439 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:19.0.0-2311.g36051606.el9 ceph 2.4 M 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 248 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:19.0.0-2311.g36051606.el9 ceph 4.7 M 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:19.0.0-2311.g36051606.el9 ceph 17 M 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2311.g36051606.el9 ceph-noarch 16 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:19.0.0-2311.g36051606.el9 ceph 25 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T17:25:52.440 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:19.0.0-2311.g36051606.el9 ceph 163 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:19.0.0-2311.g36051606.el9 ceph 500 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:19.0.0-2311.g36051606.el9 ceph 5.1 M 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T17:25:52.441 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T17:25:52.442 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:19.0.0-2311.g36051606.el9 ceph 45 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:19.0.0-2311.g36051606.el9 ceph 131 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T17:25:52.443 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T17:25:52.444 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T17:25:52.445 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T17:25:52.446 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-03-26T17:25:52.447 INFO:teuthology.orchestra.run.smithi135.stdout:============================================================================================ 2024-03-26T17:25:52.448 INFO:teuthology.orchestra.run.smithi135.stdout:Install 150 Packages 2024-03-26T17:25:52.448 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 24 Packages 2024-03-26T17:25:52.448 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:25:52.448 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 225 M 2024-03-26T17:25:52.448 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-03-26T17:25:52.467 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T17:25:52.512 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T17:25:52.585 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T17:25:52.727 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T17:25:52.781 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T17:25:53.304 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T17:25:53.376 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T17:25:53.464 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T17:25:53.513 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:25:53.523 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:25:53.570 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T17:25:53.619 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T17:25:53.657 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T17:25:53.687 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T17:25:53.807 INFO:teuthology.orchestra.run.smithi135.stdout:(1/174): ceph-19.0.0-2311.g36051606.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-03-26T17:25:53.956 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T17:25:54.173 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T17:25:54.264 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T17:25:54.308 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T17:25:54.677 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T17:25:54.732 INFO:teuthology.orchestra.run.smithi135.stdout:(2/174): ceph-fuse-19.0.0-2311.g36051606.el9.x8 963 kB/s | 890 kB 00:00 2024-03-26T17:25:54.737 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T17:25:54.792 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T17:25:54.865 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T17:25:54.896 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T17:25:54.932 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T17:25:54.957 INFO:teuthology.orchestra.run.smithi135.stdout:(3/174): ceph-common-19.0.0-2311.g36051606.el9. 13 MB/s | 21 MB 00:01 2024-03-26T17:25:54.978 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T17:25:54.982 INFO:teuthology.orchestra.run.smithi135.stdout:(4/174): ceph-immutable-object-cache-19.0.0-231 580 kB/s | 145 kB 00:00 2024-03-26T17:25:55.014 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T17:25:55.057 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T17:25:55.099 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T17:25:55.107 INFO:teuthology.orchestra.run.smithi135.stdout:(5/174): ceph-mds-19.0.0-2311.g36051606.el9.x86 16 MB/s | 2.4 MB 00:00 2024-03-26T17:25:55.234 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T17:25:55.289 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T17:25:55.318 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T17:25:55.360 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T17:25:55.391 INFO:teuthology.orchestra.run.smithi135.stdout:(6/174): ceph-mon-19.0.0-2311.g36051606.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-26T17:25:55.464 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T17:25:55.502 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T17:25:55.543 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T17:25:55.568 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T17:25:55.593 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T17:25:55.616 INFO:teuthology.orchestra.run.smithi135.stdout:(7/174): ceph-mgr-19.0.0-2311.g36051606.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-03-26T17:25:55.682 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T17:25:55.810 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T17:25:55.935 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T17:25:55.976 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T17:25:56.008 INFO:teuthology.orchestra.run.smithi135.stdout:(8/174): ceph-base-19.0.0-2311.g36051606.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-03-26T17:25:56.025 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T17:25:56.081 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T17:25:56.117 INFO:teuthology.orchestra.run.smithi135.stdout:(9/174): ceph-selinux-19.0.0-2311.g36051606.el9 232 kB/s | 25 kB 00:00 2024-03-26T17:25:56.208 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-26T17:25:56.265 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T17:25:56.298 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T17:25:56.313 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T17:25:56.341 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T17:25:56.367 INFO:teuthology.orchestra.run.smithi135.stdout:(10/174): ceph-osd-19.0.0-2311.g36051606.el9.x8 17 MB/s | 17 MB 00:00 2024-03-26T17:25:56.379 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:25:56.388 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:25:56.409 INFO:teuthology.orchestra.run.smithi135.stdout:(11/174): cephfs-mirror-19.0.0-2311.g36051606.e 5.2 MB/s | 221 kB 00:00 2024-03-26T17:25:56.429 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T17:25:56.449 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T17:25:56.459 INFO:teuthology.orchestra.run.smithi135.stdout:(12/174): cephfs-shell-19.0.0-2311.g36051606.el 543 kB/s | 27 kB 00:00 2024-03-26T17:25:56.488 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T17:25:56.491 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T17:25:56.518 INFO:teuthology.orchestra.run.smithi135.stdout:(13/174): libcephfs-devel-19.0.0-2311.g36051606 545 kB/s | 32 kB 00:00 2024-03-26T17:25:56.518 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T17:25:56.524 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T17:25:56.549 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T17:25:56.558 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T17:25:56.585 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T17:25:56.608 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T17:25:56.610 INFO:teuthology.orchestra.run.smithi135.stdout:(14/174): libcephfs2-19.0.0-2311.g36051606.el9. 8.0 MB/s | 746 kB 00:00 2024-03-26T17:25:56.620 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T17:25:56.667 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T17:25:56.668 INFO:teuthology.orchestra.run.smithi135.stdout:(15/174): libcephsqlite-19.0.0-2311.g36051606.e 2.7 MB/s | 163 kB 00:00 2024-03-26T17:25:56.699 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T17:25:56.729 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T17:25:56.738 INFO:teuthology.orchestra.run.smithi135.stdout:(16/174): librados-devel-19.0.0-2311.g36051606. 1.9 MB/s | 127 kB 00:00 2024-03-26T17:25:56.804 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T17:25:56.835 INFO:teuthology.orchestra.run.smithi135.stdout:(17/174): libradosstriper1-19.0.0-2311.g3605160 4.9 MB/s | 500 kB 00:00 2024-03-26T17:25:56.839 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T17:25:56.872 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T17:25:56.914 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T17:25:56.944 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T17:25:56.949 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T17:25:56.989 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:25:57.036 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T17:25:57.111 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:25:57.119 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T17:25:57.140 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:25:57.163 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T17:25:57.195 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T17:25:57.232 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:25:57.241 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T17:25:57.267 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:25:57.303 INFO:teuthology.orchestra.run.smithi135.stdout:(18/174): librgw2-19.0.0-2311.g36051606.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-26T17:25:57.309 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T17:25:57.361 INFO:teuthology.orchestra.run.smithi135.stdout:(19/174): python3-ceph-argparse-19.0.0-2311.g36 779 kB/s | 45 kB 00:00 2024-03-26T17:25:57.372 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T17:25:57.421 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T17:25:57.428 INFO:teuthology.orchestra.run.smithi135.stdout:(20/174): python3-ceph-common-19.0.0-2311.g3605 1.9 MB/s | 131 kB 00:00 2024-03-26T17:25:57.488 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T17:25:57.495 INFO:teuthology.orchestra.run.smithi135.stdout:(21/174): python3-cephfs-19.0.0-2311.g36051606. 2.4 MB/s | 161 kB 00:00 2024-03-26T17:25:57.551 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T17:25:57.579 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:25:57.630 INFO:teuthology.orchestra.run.smithi135.stdout:(22/174): python3-rados-19.0.0-2311.g36051606.e 2.4 MB/s | 321 kB 00:00 2024-03-26T17:25:57.682 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:25:57.705 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:25:57.770 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:25:57.796 INFO:teuthology.orchestra.run.smithi135.stdout:(23/174): ceph-radosgw-19.0.0-2311.g36051606.el 4.5 MB/s | 9.8 MB 00:02 2024-03-26T17:25:57.817 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T17:25:57.821 INFO:teuthology.orchestra.run.smithi135.stdout:(24/174): python3-rbd-19.0.0-2311.g36051606.el9 1.5 MB/s | 298 kB 00:00 2024-03-26T17:25:57.840 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:25:57.855 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:25:57.871 INFO:teuthology.orchestra.run.smithi135.stdout:(25/174): python3-rgw-19.0.0-2311.g36051606.el9 1.3 MB/s | 99 kB 00:00 2024-03-26T17:25:57.896 INFO:teuthology.orchestra.run.smithi135.stdout:(26/174): rbd-fuse-19.0.0-2311.g36051606.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-26T17:25:57.972 INFO:teuthology.orchestra.run.smithi135.stdout:(27/174): rbd-nbd-19.0.0-2311.g36051606.el9.x86 2.2 MB/s | 170 kB 00:00 2024-03-26T17:25:58.039 INFO:teuthology.orchestra.run.smithi135.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2311.g 510 kB/s | 34 kB 00:00 2024-03-26T17:25:58.048 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:25:58.080 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:25:58.114 INFO:teuthology.orchestra.run.smithi135.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2311.g3605160 1.8 MB/s | 140 kB 00:00 2024-03-26T17:25:58.119 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T17:25:58.149 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T17:25:58.193 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T17:25:58.227 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T17:25:58.284 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T17:25:58.295 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T17:25:58.375 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T17:25:58.381 INFO:teuthology.orchestra.run.smithi135.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2311.g36051 6.7 MB/s | 1.8 MB 00:00 2024-03-26T17:25:58.414 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T17:25:58.441 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:25:58.508 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:25:58.548 INFO:teuthology.orchestra.run.smithi135.stdout:(31/174): rbd-mirror-19.0.0-2311.g36051606.el9. 4.6 MB/s | 3.1 MB 00:00 2024-03-26T17:25:58.591 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T17:25:58.605 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:25:58.619 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T17:25:58.651 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:25:58.656 INFO:teuthology.orchestra.run.smithi135.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2311.g36 2.2 MB/s | 248 kB 00:00 2024-03-26T17:25:58.706 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T17:25:58.740 INFO:teuthology.orchestra.run.smithi135.stdout:(33/174): ceph-mgr-rook-19.0.0-2311.g36051606.e 593 kB/s | 49 kB 00:00 2024-03-26T17:25:58.815 INFO:teuthology.orchestra.run.smithi135.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2311.g3 219 kB/s | 16 kB 00:00 2024-03-26T17:25:58.887 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:25:58.924 INFO:teuthology.orchestra.run.smithi135.stdout:(35/174): ceph-volume-19.0.0-2311.g36051606.el9 2.4 MB/s | 264 kB 00:00 2024-03-26T17:25:59.010 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T17:25:59.032 INFO:teuthology.orchestra.run.smithi135.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-03-26T17:25:59.074 INFO:teuthology.orchestra.run.smithi135.stdout:(37/174): cephfs-top-19.0.0-2311.g36051606.el9. 442 kB/s | 18 kB 00:00 2024-03-26T17:25:59.110 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T17:25:59.139 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T17:25:59.166 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T17:25:59.174 INFO:teuthology.orchestra.run.smithi135.stdout:(38/174): cephadm-19.0.0-2311.g36051606.el9.noa 2.9 MB/s | 750 kB 00:00 2024-03-26T17:25:59.210 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T17:25:59.284 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T17:25:59.338 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T17:25:59.341 INFO:teuthology.orchestra.run.smithi135.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-26T17:25:59.375 INFO:teuthology.orchestra.run.smithi135.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.5 MB/s | 920 kB 00:00 2024-03-26T17:25:59.376 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T17:25:59.392 INFO:teuthology.orchestra.run.smithi135.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-26T17:25:59.407 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T17:25:59.417 INFO:teuthology.orchestra.run.smithi135.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-26T17:25:59.441 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T17:25:59.442 INFO:teuthology.orchestra.run.smithi135.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 640 kB/s | 32 kB 00:00 2024-03-26T17:25:59.484 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T17:25:59.526 INFO:teuthology.orchestra.run.smithi135.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-26T17:25:59.527 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T17:25:59.527 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T17:25:59.551 INFO:teuthology.orchestra.run.smithi135.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 161 kB/s | 17 kB 00:00 2024-03-26T17:25:59.560 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T17:25:59.577 INFO:teuthology.orchestra.run.smithi135.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2024-03-26T17:25:59.588 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T17:25:59.602 INFO:teuthology.orchestra.run.smithi135.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-26T17:25:59.627 INFO:teuthology.orchestra.run.smithi135.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-26T17:25:59.657 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T17:25:59.661 INFO:teuthology.orchestra.run.smithi135.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-26T17:25:59.686 INFO:teuthology.orchestra.run.smithi135.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-03-26T17:25:59.692 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T17:25:59.711 INFO:teuthology.orchestra.run.smithi135.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 46 kB 00:00 2024-03-26T17:25:59.736 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T17:25:59.737 INFO:teuthology.orchestra.run.smithi135.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-26T17:25:59.762 INFO:teuthology.orchestra.run.smithi135.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T17:25:59.766 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T17:25:59.788 INFO:teuthology.orchestra.run.smithi135.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-26T17:25:59.814 INFO:teuthology.orchestra.run.smithi135.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-03-26T17:25:59.837 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T17:25:59.881 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T17:25:59.938 INFO:teuthology.orchestra.run.smithi135.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-26T17:25:59.971 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T17:25:59.980 INFO:teuthology.orchestra.run.smithi135.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-03-26T17:26:00.005 INFO:teuthology.orchestra.run.smithi135.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-26T17:26:00.108 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T17:26:00.158 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T17:26:00.206 INFO:teuthology.orchestra.run.smithi135.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-26T17:26:00.231 INFO:teuthology.orchestra.run.smithi135.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 110 kB/s | 24 kB 00:00 2024-03-26T17:26:00.273 INFO:teuthology.orchestra.run.smithi135.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 272 kB/s | 18 kB 00:00 2024-03-26T17:26:00.306 INFO:teuthology.orchestra.run.smithi135.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-26T17:26:00.348 INFO:teuthology.orchestra.run.smithi135.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 419 kB/s | 32 kB 00:00 2024-03-26T17:26:00.540 INFO:teuthology.orchestra.run.smithi135.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2024-03-26T17:26:00.565 INFO:teuthology.orchestra.run.smithi135.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2024-03-26T17:26:00.608 INFO:teuthology.orchestra.run.smithi135.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-26T17:26:00.641 INFO:teuthology.orchestra.run.smithi135.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 915 kB/s | 69 kB 00:00 2024-03-26T17:26:00.675 INFO:teuthology.orchestra.run.smithi135.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 618 kB/s | 41 kB 00:00 2024-03-26T17:26:00.681 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T17:26:00.700 INFO:teuthology.orchestra.run.smithi135.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-26T17:26:00.725 INFO:teuthology.orchestra.run.smithi135.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-26T17:26:00.750 INFO:teuthology.orchestra.run.smithi135.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-26T17:26:00.756 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T17:26:00.761 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T17:26:00.776 INFO:teuthology.orchestra.run.smithi135.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 953 kB/s | 47 kB 00:00 2024-03-26T17:26:00.801 INFO:teuthology.orchestra.run.smithi135.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-03-26T17:26:00.837 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T17:26:00.850 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T17:26:00.868 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T17:26:00.898 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:26:00.899 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T17:26:00.908 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:26:00.927 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T17:26:00.939 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T17:26:00.988 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T17:26:00.993 INFO:teuthology.orchestra.run.smithi135.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 203 kB/s | 39 kB 00:00 2024-03-26T17:26:01.025 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T17:26:01.056 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T17:26:01.239 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T17:26:01.265 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:26:01.323 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T17:26:01.343 INFO:teuthology.orchestra.run.smithi135.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-03-26T17:26:01.358 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:26:01.368 INFO:teuthology.orchestra.run.smithi135.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 110 kB/s | 41 kB 00:00 2024-03-26T17:26:01.387 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T17:26:01.394 INFO:teuthology.orchestra.run.smithi135.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-26T17:26:01.419 INFO:teuthology.orchestra.run.smithi135.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-26T17:26:01.423 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T17:26:01.444 INFO:teuthology.orchestra.run.smithi135.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-26T17:26:01.469 INFO:teuthology.orchestra.run.smithi135.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-03-26T17:26:01.495 INFO:teuthology.orchestra.run.smithi135.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-26T17:26:01.502 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T17:26:01.520 INFO:teuthology.orchestra.run.smithi135.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-26T17:26:01.562 INFO:teuthology.orchestra.run.smithi135.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-26T17:26:01.588 INFO:teuthology.orchestra.run.smithi135.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-26T17:26:01.591 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T17:26:01.613 INFO:teuthology.orchestra.run.smithi135.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-26T17:26:01.627 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T17:26:01.638 INFO:teuthology.orchestra.run.smithi135.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-26T17:26:01.763 INFO:teuthology.orchestra.run.smithi135.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-26T17:26:01.777 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T17:26:01.838 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T17:26:01.847 INFO:teuthology.orchestra.run.smithi135.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-03-26T17:26:01.888 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T17:26:01.893 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T17:26:01.897 INFO:teuthology.orchestra.run.smithi135.stdout:(89/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-26T17:26:01.918 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:26:01.922 INFO:teuthology.orchestra.run.smithi135.stdout:(90/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-03-26T17:26:01.966 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T17:26:01.972 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:26:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T17:26:02.014 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T17:26:02.033 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T17:26:02.081 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T17:26:02.149 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T17:26:02.189 INFO:teuthology.orchestra.run.smithi135.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 74 kB/s | 21 kB 00:00 2024-03-26T17:26:02.215 INFO:teuthology.orchestra.run.smithi135.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 87 kB/s | 25 kB 00:00 2024-03-26T17:26:02.226 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T17:26:02.268 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T17:26:02.401 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T17:26:02.448 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T17:26:02.479 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T17:26:02.520 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T17:26:02.614 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T17:26:02.654 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T17:26:02.694 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T17:26:02.720 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T17:26:02.744 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T17:26:02.821 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T17:26:02.932 INFO:teuthology.orchestra.run.smithi135.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-03-26T17:26:02.944 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T17:26:03.046 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T17:26:03.057 INFO:teuthology.orchestra.run.smithi135.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-03-26T17:26:03.086 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T17:26:03.135 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T17:26:03.190 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T17:26:03.238 INFO:teuthology.orchestra.run.smithi135.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 737 kB/s | 133 kB 00:00 2024-03-26T17:26:03.374 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T17:26:03.415 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T17:26:03.416 INFO:teuthology.orchestra.run.smithi135.stdout:(96/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.4 MB/s | 4.1 MB 00:01 2024-03-26T17:26:03.549 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T17:26:03.597 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T17:26:03.628 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T17:26:03.649 INFO:teuthology.orchestra.run.smithi135.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-03-26T17:26:03.668 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T17:26:03.712 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T17:26:03.747 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T17:26:03.786 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T17:26:03.817 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T17:26:03.847 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T17:26:03.916 INFO:teuthology.orchestra.run.smithi135.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 931 kB/s | 248 kB 00:00 2024-03-26T17:26:03.923 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T17:26:03.958 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T17:26:03.991 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T17:26:04.033 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T17:26:04.067 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T17:26:04.154 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T17:26:04.183 INFO:teuthology.orchestra.run.smithi135.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 828 kB/s | 221 kB 00:00 2024-03-26T17:26:04.186 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:04.233 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:04.244 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T17:26:04.283 INFO:teuthology.orchestra.run.smithi135.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 359 kB/s | 36 kB 00:00 2024-03-26T17:26:04.290 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T17:26:04.321 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T17:26:04.403 INFO:teuthology.orchestra.run.smithi135.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 475 kB/s | 47 kB 00:00 2024-03-26T17:26:04.403 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T17:26:04.458 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T17:26:04.492 INFO:teuthology.orchestra.run.smithi135.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 865 kB/s | 94 kB 00:00 2024-03-26T17:26:04.505 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T17:26:04.559 INFO:teuthology.orchestra.run.smithi135.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 335 kB/s | 22 kB 00:00 2024-03-26T17:26:04.701 INFO:teuthology.orchestra.run.smithi135.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-03-26T17:26:04.780 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:04.820 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:26:04.891 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:26:04.891 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T17:26:04.891 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T17:26:04.892 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:04.930 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:26:05.231 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T17:26:05.235 INFO:teuthology.orchestra.run.smithi135.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-26T17:26:05.278 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T17:26:05.312 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T17:26:05.359 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T17:26:05.360 INFO:teuthology.orchestra.run.smithi135.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-03-26T17:26:05.443 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T17:26:05.443 INFO:teuthology.orchestra.run.smithi135.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-03-26T17:26:05.481 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T17:26:05.656 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T17:26:05.695 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T17:26:05.773 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T17:26:05.952 INFO:teuthology.orchestra.run.smithi135.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-03-26T17:26:06.027 INFO:teuthology.orchestra.run.smithi135.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-03-26T17:26:06.119 INFO:teuthology.orchestra.run.smithi135.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-03-26T17:26:06.203 INFO:teuthology.orchestra.run.smithi135.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 808 kB/s | 67 kB 00:00 2024-03-26T17:26:06.312 INFO:teuthology.orchestra.run.smithi135.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-26T17:26:06.412 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T17:26:06.462 INFO:teuthology.orchestra.run.smithi135.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 838 kB 00:00 2024-03-26T17:26:06.533 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T17:26:06.596 INFO:teuthology.orchestra.run.smithi135.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-03-26T17:26:06.696 INFO:teuthology.orchestra.run.smithi135.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-03-26T17:26:06.780 INFO:teuthology.orchestra.run.smithi135.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 386 kB/s | 32 kB 00:00 2024-03-26T17:26:06.888 INFO:teuthology.orchestra.run.smithi135.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-03-26T17:26:06.997 INFO:teuthology.orchestra.run.smithi135.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-03-26T17:26:07.172 INFO:teuthology.orchestra.run.smithi135.stdout:(119/174): postgresql-server-13.14-1.el9.x86_64 1.5 MB/s | 5.8 MB 00:03 2024-03-26T17:26:07.206 INFO:teuthology.orchestra.run.smithi135.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-03-26T17:26:07.339 INFO:teuthology.orchestra.run.smithi135.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 265 kB/s | 35 kB 00:00 2024-03-26T17:26:07.431 INFO:teuthology.orchestra.run.smithi135.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-26T17:26:07.632 INFO:teuthology.orchestra.run.smithi135.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 660 kB/s | 303 kB 00:00 2024-03-26T17:26:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T17:26:07.757 INFO:teuthology.orchestra.run.smithi135.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.3 MB/s | 2.0 MB 00:00 2024-03-26T17:26:07.782 INFO:teuthology.orchestra.run.smithi135.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 962 kB/s | 144 kB 00:00 2024-03-26T17:26:07.821 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T17:26:07.852 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T17:26:07.857 INFO:teuthology.orchestra.run.smithi135.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-03-26T17:26:07.883 INFO:teuthology.orchestra.run.smithi135.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-26T17:26:07.883 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T17:26:07.910 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T17:26:08.222 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T17:26:08.249 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:26:08.331 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:26:08.354 INFO:teuthology.orchestra.run.smithi135.stdout:(128/174): ceph-test-19.0.0-2311.g36051606.el9. 3.9 MB/s | 48 MB 00:12 2024-03-26T17:26:08.370 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T17:26:08.406 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T17:26:08.500 INFO:teuthology.orchestra.run.smithi135.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 36 kB/s | 23 kB 00:00 2024-03-26T17:26:08.525 INFO:teuthology.orchestra.run.smithi135.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 30 kB/s | 19 kB 00:00 2024-03-26T17:26:08.650 INFO:teuthology.orchestra.run.smithi135.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 371 kB/s | 46 kB 00:00 2024-03-26T17:26:08.684 INFO:teuthology.orchestra.run.smithi135.stdout:(132/174): python3-jaraco-text-3.2.0-6.el9.noar 60 kB/s | 20 kB 00:00 2024-03-26T17:26:08.734 INFO:teuthology.orchestra.run.smithi135.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 943 kB/s | 79 kB 00:00 2024-03-26T17:26:08.826 INFO:teuthology.orchestra.run.smithi135.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-03-26T17:26:08.851 INFO:teuthology.orchestra.run.smithi135.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 345 kB/s | 58 kB 00:00 2024-03-26T17:26:08.867 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T17:26:08.918 INFO:teuthology.orchestra.run.smithi135.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-03-26T17:26:08.943 INFO:teuthology.orchestra.run.smithi135.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-03-26T17:26:08.977 INFO:teuthology.orchestra.run.smithi135.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 718 kB/s | 90 kB 00:00 2024-03-26T17:26:09.011 INFO:teuthology.orchestra.run.smithi135.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-03-26T17:26:09.044 INFO:teuthology.orchestra.run.smithi135.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-03-26T17:26:09.097 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:26:09.111 INFO:teuthology.orchestra.run.smithi135.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-03-26T17:26:09.136 INFO:teuthology.orchestra.run.smithi135.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-03-26T17:26:09.162 INFO:teuthology.orchestra.run.smithi135.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-03-26T17:26:09.195 INFO:teuthology.orchestra.run.smithi135.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 610 kB/s | 51 kB 00:00 2024-03-26T17:26:09.237 INFO:teuthology.orchestra.run.smithi135.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-26T17:26:09.263 INFO:teuthology.orchestra.run.smithi135.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-03-26T17:26:09.273 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:26:09.303 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T17:26:09.313 INFO:teuthology.orchestra.run.smithi135.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-26T17:26:09.355 INFO:teuthology.orchestra.run.smithi135.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-03-26T17:26:09.380 INFO:teuthology.orchestra.run.smithi135.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-03-26T17:26:09.547 INFO:teuthology.orchestra.run.smithi135.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-03-26T17:26:09.597 INFO:teuthology.orchestra.run.smithi135.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-26T17:26:09.881 INFO:teuthology.orchestra.run.smithi135.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 8.1 MB/s | 2.3 MB 00:00 2024-03-26T17:26:10.073 INFO:teuthology.orchestra.run.smithi135.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 9.6 MB/s | 1.8 MB 00:00 2024-03-26T17:26:10.123 INFO:teuthology.orchestra.run.smithi135.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 843 kB/s | 42 kB 00:00 2024-03-26T17:26:10.173 INFO:teuthology.orchestra.run.smithi135.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-03-26T17:26:10.232 INFO:teuthology.orchestra.run.smithi135.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-26T17:26:10.282 INFO:teuthology.orchestra.run.smithi135.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 732 kB/s | 36 kB 00:00 2024-03-26T17:26:10.332 INFO:teuthology.orchestra.run.smithi135.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-26T17:26:10.408 INFO:teuthology.orchestra.run.smithi135.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 256 kB 00:00 2024-03-26T17:26:10.449 INFO:teuthology.orchestra.run.smithi135.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 556 kB/s | 23 kB 00:00 2024-03-26T17:26:10.500 INFO:teuthology.orchestra.run.smithi135.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-26T17:26:10.558 INFO:teuthology.orchestra.run.smithi135.stdout:(162/174): librados2-19.0.0-2311.g36051606.el9. 2.8 MB/s | 3.4 MB 00:01 2024-03-26T17:26:10.583 INFO:teuthology.orchestra.run.smithi135.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-03-26T17:26:10.634 INFO:teuthology.orchestra.run.smithi135.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-26T17:26:10.667 INFO:teuthology.orchestra.run.smithi135.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-26T17:26:10.692 INFO:teuthology.orchestra.run.smithi135.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 491 kB/s | 29 kB 00:00 2024-03-26T17:26:10.801 INFO:teuthology.orchestra.run.smithi135.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 9.1 MB/s | 1.2 MB 00:00 2024-03-26T17:26:10.985 INFO:teuthology.orchestra.run.smithi135.stdout:(168/174): librbd1-19.0.0-2311.g36051606.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-03-26T17:26:11.027 INFO:teuthology.orchestra.run.smithi135.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.5 MB/s | 2.2 MB 00:00 2024-03-26T17:26:11.110 INFO:teuthology.orchestra.run.smithi135.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.4 MB/s | 2.3 MB 00:00 2024-03-26T17:26:11.136 INFO:teuthology.orchestra.run.smithi135.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 287 kB/s | 31 kB 00:00 2024-03-26T17:26:11.203 INFO:teuthology.orchestra.run.smithi135.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 2.1 MB/s | 465 kB 00:00 2024-03-26T17:26:11.420 INFO:teuthology.orchestra.run.smithi135.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.5 MB/s | 1.1 MB 00:00 2024-03-26T17:26:11.470 INFO:teuthology.orchestra.run.smithi135.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 223 kB/s | 74 kB 00:00 2024-03-26T17:26:11.473 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:26:11.474 INFO:teuthology.orchestra.run.smithi135.stdout:Total 12 MB/s | 225 MB 00:19 2024-03-26T17:26:11.502 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:11.533 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:12.028 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-03-26T17:26:12.072 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:12.120 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:26:12.208 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:26:12.208 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T17:26:12.209 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T17:26:12.209 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:26:12.302 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-03-26T17:26:12.303 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-03-26T17:26:14.393 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-03-26T17:26:14.394 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-03-26T17:26:16.885 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:26:16.886 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-03-26T17:26:16.886 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-03-26T17:26:16.886 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-03-26T17:26:16.886 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-03-26T17:26:16.886 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-03-26T17:26:16.886 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-03-26T17:26:16.886 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-03-26T17:26:16.886 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:17.051 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:26:17.067 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:26:17.067 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:17.067 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T17:26:17.067 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:18.538 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:26:18.563 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:26:18.563 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:18.563 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T17:26:18.563 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:26:18.563 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:26:18.564 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:18.788 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:26:18.811 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:26:18.811 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:18.811 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T17:26:18.812 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:26:18.812 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:26:18.812 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:19.264 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:26:19.286 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:26:19.286 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:19.287 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T17:26:19.287 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:26:19.287 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:26:19.287 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:19.374 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T17:26:19.733 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:26:19.756 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:26:19.788 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T17:26:19.848 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T17:26:19.871 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:26:20.103 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:26:20.130 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:26:21.086 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:26:21.114 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:26:21.213 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:26:21.291 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-03-26T17:26:21.367 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T17:26:21.391 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:26:21.399 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T17:26:21.414 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:26:21.414 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:21.414 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T17:26:21.414 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:26:21.414 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:26:21.414 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:21.433 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T17:26:21.450 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:26:21.461 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:26:21.464 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:26:21.474 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T17:26:21.504 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T17:26:21.513 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T17:26:21.568 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T17:26:21.601 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T17:26:21.635 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T17:26:21.689 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T17:26:22.015 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T17:26:22.078 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:26:22.200 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:26:22.237 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:26:22.329 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:26:22.358 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:26:22.384 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T17:26:22.396 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T17:26:22.410 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:26:22.433 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:26:22.434 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:22.434 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T17:26:22.434 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:26:22.434 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:26:22.434 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:22.477 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:26:22.499 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:26:22.499 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:22.499 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T17:26:22.499 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:22.545 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:26:22.567 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:26:22.567 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:22.567 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T17:26:22.567 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:26:22.567 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:26:22.567 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:22.585 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T17:26:22.639 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T17:26:22.667 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:26:22.753 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:26:22.776 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:26:22.825 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 17/198 2024-03-26T17:26:22.855 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T17:26:22.878 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:26:22.879 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:26:22.892 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T17:26:22.901 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:26:22.901 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:22.901 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T17:26:22.901 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:26:22.901 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:26:22.901 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:23.078 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:26:23.109 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T17:26:23.149 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T17:26:23.315 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T17:26:23.478 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:26:23.538 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T17:26:23.633 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:26:23.688 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:26:23.890 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T17:26:23.998 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T17:26:24.098 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T17:26:24.128 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T17:26:24.155 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T17:26:24.190 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T17:26:24.263 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:26:24.263 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-03-26T17:26:24.264 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-03-26T17:26:24.264 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-03-26T17:26:24.264 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-03-26T17:26:24.264 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-03-26T17:26:24.264 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-03-26T17:26:24.264 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-03-26T17:26:24.264 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:24.264 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T17:26:24.317 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T17:26:24.356 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T17:26:24.420 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T17:26:24.434 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:26:24.448 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:26:24.448 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:24.448 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T17:26:24.449 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:24.455 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T17:26:24.499 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T17:26:24.532 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T17:26:24.559 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T17:26:24.628 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T17:26:24.656 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 39/198 2024-03-26T17:26:24.700 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 40/198 2024-03-26T17:26:24.728 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 41/198 2024-03-26T17:26:24.796 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T17:26:24.842 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T17:26:24.915 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T17:26:25.049 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T17:26:25.102 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T17:26:25.624 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T17:26:25.698 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T17:26:25.777 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T17:26:25.818 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:26:25.828 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T17:26:25.868 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T17:26:25.908 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T17:26:25.928 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:26:25.936 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 53/198 2024-03-26T17:26:25.953 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:26:25.953 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:25.954 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T17:26:25.954 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:26:25.954 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:26:25.954 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:25.967 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T17:26:26.180 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:26:26.202 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:26:26.202 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:26.202 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T17:26:26.202 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:26:26.202 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:26:26.202 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:26.246 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T17:26:26.430 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T17:26:26.502 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T17:26:26.546 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T17:26:26.648 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:26:26.669 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:26:26.670 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:26.670 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T17:26:26.670 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:26:26.670 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:26:26.670 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:26.695 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T17:26:26.766 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T17:26:26.830 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T17:26:26.912 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T17:26:26.984 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T17:26:27.016 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T17:26:27.044 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T17:26:27.091 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T17:26:27.127 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T17:26:27.171 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T17:26:27.212 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T17:26:27.233 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:26:27.264 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:26:27.297 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T17:26:27.345 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T17:26:27.357 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T17:26:27.388 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:26:27.391 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:26:27.392 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T17:26:27.423 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T17:26:27.433 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T17:26:27.464 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T17:26:27.472 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T17:26:27.499 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T17:26:27.558 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T17:26:27.598 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T17:26:27.621 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:26:27.638 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T17:26:27.655 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:26:27.664 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T17:26:27.689 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T17:26:27.691 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T17:26:27.777 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T17:26:27.807 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T17:26:27.847 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T17:26:27.898 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T17:26:27.905 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T17:26:27.955 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T17:26:27.990 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T17:26:28.020 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T17:26:28.023 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T17:26:28.055 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T17:26:28.063 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T17:26:28.085 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T17:26:28.112 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T17:26:28.156 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T17:26:28.167 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T17:26:28.222 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T17:26:28.260 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T17:26:28.349 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T17:26:28.400 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T17:26:28.528 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T17:26:28.566 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T17:26:28.606 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T17:26:28.612 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:26:28.639 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:26:28.645 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T17:26:28.672 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T17:26:28.707 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T17:26:28.739 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:26:28.754 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T17:26:28.786 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T17:26:28.801 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:26:28.816 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T17:26:28.850 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:26:28.870 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:26:28.891 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T17:26:28.901 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T17:26:28.924 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:26:28.927 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T17:26:28.947 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:26:28.947 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:28.947 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T17:26:28.947 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:26:28.948 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:26:28.948 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:28.961 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T17:26:28.992 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:26:29.001 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T17:26:29.003 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:26:29.044 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T17:26:29.053 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T17:26:29.132 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T17:26:29.214 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T17:26:29.268 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T17:26:29.298 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T17:26:29.345 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T17:26:29.443 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T17:26:29.500 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T17:26:29.686 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T17:26:29.778 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T17:26:29.819 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T17:26:29.912 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T17:26:29.920 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T17:26:29.957 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T17:26:29.961 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:26:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T17:26:29.983 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:26:29.983 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:29.983 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T17:26:29.983 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:26:29.984 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:26:29.984 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T17:26:30.028 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:26:30.034 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T17:26:30.034 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:26:30.050 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:26:30.050 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:30.050 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T17:26:30.050 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:30.095 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:26:30.118 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:26:30.118 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:30.118 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T17:26:30.118 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:26:30.118 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:26:30.118 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:30.122 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:26:30.151 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T17:26:30.151 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:26:30.220 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:26:30.224 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T17:26:30.254 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T17:26:30.272 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T17:26:30.287 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T17:26:30.306 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T17:26:30.312 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T17:26:30.337 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T17:26:30.354 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T17:26:30.373 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T17:26:30.419 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T17:26:30.430 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:26:30.432 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T17:26:30.451 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:26:30.452 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:30.452 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T17:26:30.452 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:26:30.452 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:26:30.452 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:30.459 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T17:26:30.473 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T17:26:30.484 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T17:26:30.509 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T17:26:30.535 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:26:30.560 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T17:26:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:26:30.566 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:26:30.667 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T17:26:30.696 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T17:26:30.746 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:26:30.747 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:26:30.777 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T17:26:30.846 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:26:30.862 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:26:30.895 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T17:26:30.922 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T17:26:30.947 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T17:26:31.014 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:26:31.407 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T17:26:31.510 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T17:26:32.721 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T17:26:32.813 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T17:26:32.844 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T17:26:32.875 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T17:26:32.903 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T17:26:33.213 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T17:26:33.241 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:26:33.332 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 124/198 2024-03-26T17:26:33.362 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 125/198 2024-03-26T17:26:33.399 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T17:26:33.855 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T17:26:33.886 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:26:34.221 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 128/198 2024-03-26T17:26:34.418 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 129/198 2024-03-26T17:26:34.901 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:26:34.942 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T17:26:34.981 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T17:26:35.008 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T17:26:35.196 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T17:26:35.308 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T17:26:35.348 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T17:26:35.406 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T17:26:35.472 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T17:26:35.507 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T17:26:35.532 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T17:26:35.572 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T17:26:35.602 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T17:26:35.669 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T17:26:35.737 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T17:26:35.768 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T17:26:35.895 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T17:26:35.896 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T17:26:35.897 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T17:26:35.898 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T17:26:35.899 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T17:26:35.900 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T17:26:35.901 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T17:26:35.902 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T17:26:35.903 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T17:26:35.904 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T17:26:35.905 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T17:26:35.906 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T17:26:35.907 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T17:26:35.908 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T17:26:35.909 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T17:26:36.363 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:26:36.415 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:26:36.434 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:26:36.668 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:36.708 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:37.243 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T17:26:37.245 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 130/198 2024-03-26T17:26:37.287 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:26:37.329 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T17:26:37.370 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T17:26:37.381 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 131/198 2024-03-26T17:26:37.381 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T17:26:37.381 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T17:26:37.381 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:37.413 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:26:37.462 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T17:26:37.491 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T17:26:37.516 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T17:26:37.541 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T17:26:37.570 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T17:26:37.570 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:26:37.664 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:26:37.694 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T17:26:37.694 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:26:37.778 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:26:37.804 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T17:26:37.838 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T17:26:37.863 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T17:26:37.888 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T17:26:37.923 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T17:26:37.962 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T17:26:37.993 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T17:26:38.018 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T17:26:38.043 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T17:26:38.068 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:26:38.085 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T17:26:38.092 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:26:38.092 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:26:38.271 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:26:38.272 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:26:38.355 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:26:38.379 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:26:38.412 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T17:26:38.439 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T17:26:38.472 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T17:26:38.538 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:26:38.768 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T17:26:38.768 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:38.768 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-03-26T17:26:38.768 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T17:26:38.768 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T17:26:38.768 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T17:26:38.768 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T17:26:38.768 INFO:teuthology.orchestra.run.smithi144.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T17:26:38.769 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.770 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.771 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T17:26:38.772 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-16.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T17:26:38.773 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T17:26:38.774 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T17:26:38.775 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T17:26:38.776 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T17:26:38.777 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T17:26:38.778 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T17:26:38.779 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.780 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:26:38.781 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-03-26T17:26:39.246 DEBUG:teuthology.parallel:result is None 2024-03-26T17:26:43.338 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:26:43.338 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T17:26:43.338 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T17:26:43.338 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T17:26:43.338 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T17:26:43.338 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T17:26:43.338 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T17:26:43.339 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T17:26:43.340 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T17:26:43.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T17:26:43.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T17:26:43.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T17:26:43.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T17:26:43.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T17:26:43.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T17:26:43.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T17:26:43.357 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T17:26:43.358 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T17:26:43.359 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T17:26:43.360 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T17:26:43.361 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T17:26:43.362 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T17:26:43.363 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T17:26:43.364 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T17:26:43.365 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T17:26:43.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T17:26:43.367 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T17:26:43.368 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T17:26:43.369 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T17:26:43.370 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T17:26:43.371 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T17:26:46.442 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T17:26:46.442 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:46.442 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.443 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T17:26:46.444 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.446 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-16.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T17:26:46.447 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T17:26:46.448 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T17:26:46.449 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.450 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T17:26:46.451 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T17:26:46.452 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T17:26:46.453 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T17:26:46.454 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T17:26:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T17:26:46.456 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T17:26:46.456 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:26:46.456 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-26T17:26:46.955 DEBUG:teuthology.parallel:result is None 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 132/198 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-03-26T17:26:49.398 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:49.559 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:26:49.574 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 133/198 2024-03-26T17:26:49.574 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:49.574 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T17:26:49.575 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:51.071 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:26:51.096 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 134/198 2024-03-26T17:26:51.097 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:51.097 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T17:26:51.097 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:26:51.097 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T17:26:51.097 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:51.322 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:26:51.344 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 135/198 2024-03-26T17:26:51.344 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:51.344 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T17:26:51.344 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:26:51.344 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T17:26:51.345 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:51.797 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:26:51.819 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 136/198 2024-03-26T17:26:51.819 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:51.819 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T17:26:51.819 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:26:51.819 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T17:26:51.819 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:51.908 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T17:26:52.250 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:26:52.289 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:19.0.0-2311.g36051606.el9.noarch 138/198 2024-03-26T17:26:52.321 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 139/198 2024-03-26T17:26:52.382 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 140/198 2024-03-26T17:26:52.412 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:26:52.643 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 141/198 2024-03-26T17:26:52.671 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:26:53.634 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 142/198 2024-03-26T17:26:53.663 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:26:53.764 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 143/198 2024-03-26T17:26:53.917 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 144/198 2024-03-26T17:26:53.948 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:26:53.971 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 145/198 2024-03-26T17:26:53.971 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:53.971 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T17:26:53.971 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:26:53.971 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T17:26:53.971 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:54.016 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:26:54.026 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 146/198 2024-03-26T17:26:54.074 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:19.0.0-2311.g36051606.el9.x86_64 147/198 2024-03-26T17:26:54.945 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T17:26:54.977 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:26:54.998 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 149/198 2024-03-26T17:26:54.998 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:54.999 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T17:26:54.999 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:26:54.999 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T17:26:54.999 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:55.044 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:26:55.065 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2311.g36051 150/198 2024-03-26T17:26:55.066 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:55.066 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T17:26:55.066 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:55.111 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:26:55.133 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:26:55.133 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:55.133 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T17:26:55.134 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:26:55.134 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T17:26:55.134 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:55.464 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:26:55.486 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 152/198 2024-03-26T17:26:55.486 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T17:26:55.487 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T17:26:55.487 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:26:55.487 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T17:26:55.487 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:26:59.958 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:26:59.999 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 154/198 2024-03-26T17:27:00.037 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 155/198 2024-03-26T17:27:00.065 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T17:27:00.266 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 157/198 2024-03-26T17:27:00.373 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T17:27:00.412 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 159/198 2024-03-26T17:27:00.466 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T17:27:00.513 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T17:27:00.556 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T17:27:00.580 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 163/198 2024-03-26T17:27:00.613 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T17:27:00.642 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T17:27:00.711 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T17:27:00.771 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T17:27:00.808 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T17:27:01.424 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:27:01.464 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:27:01.483 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T17:27:02.287 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 170/198 2024-03-26T17:27:02.369 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T17:27:02.409 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T17:27:02.500 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T17:27:02.531 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T17:27:02.556 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T17:27:02.581 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T17:27:02.607 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T17:27:02.608 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:27:02.695 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T17:27:02.725 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T17:27:02.725 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:27:02.810 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T17:27:02.836 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T17:27:02.869 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T17:27:02.894 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T17:27:02.919 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T17:27:02.953 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T17:27:02.992 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T17:27:03.024 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T17:27:03.049 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T17:27:03.074 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T17:27:03.100 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:27:03.117 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T17:27:03.123 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:27:03.123 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:27:03.303 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T17:27:03.303 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:27:03.386 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T17:27:03.444 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:27:03.475 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T17:27:03.503 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T17:27:03.528 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T17:27:03.594 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:27:08.719 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:19.0.0-2311.g36051606.el9.x86_64 1/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 3/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 4/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2311.g36051 5/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 6/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 7/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 8/198 2024-03-26T17:27:08.720 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 9/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 10/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 11/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 12/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 13/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 14/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_ 15/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 16/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 17/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:19.0.0-2311.g36051606.el9.x86_6 18/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:19.0.0-2311.g36051606.el9.x86 19/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:19.0.0-2311.g36051606.el9.x86_64 20/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2311.g36051606.el 21/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:19.0.0-2311.g36051606.el9. 22/198 2024-03-26T17:27:08.721 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_6 23/198 2024-03-26T17:27:08.722 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 24/198 2024-03-26T17:27:08.722 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 25/198 2024-03-26T17:27:08.722 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 26/198 2024-03-26T17:27:08.722 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 27/198 2024-03-26T17:27:08.722 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 28/198 2024-03-26T17:27:08.722 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 29/198 2024-03-26T17:27:08.722 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2311.g36051606. 30/198 2024-03-26T17:27:08.739 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noa 31/198 2024-03-26T17:27:08.739 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.n 32/198 2024-03-26T17:27:08.739 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2311.g360 33/198 2024-03-26T17:27:08.739 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el 34/198 2024-03-26T17:27:08.739 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 35/198 2024-03-26T17:27:08.739 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2311.g36051606.e 36/198 2024-03-26T17:27:08.739 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 37/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:19.0.0-2311.g36051606.el9.noarch 38/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 39/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T17:27:08.740 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T17:27:08.741 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T17:27:08.742 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T17:27:08.743 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T17:27:08.744 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T17:27:08.745 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T17:27:08.746 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T17:27:08.747 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:19.0.0-2311.g36051606.el9.x86_64 151/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:19.0.0-2311.g36051606.el9.x86_64 153/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T17:27:08.748 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T17:27:08.749 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T17:27:08.750 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T17:27:08.751 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T17:27:08.752 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T17:27:08.752 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T17:27:08.752 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T17:27:08.752 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T17:27:08.752 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T17:27:11.648 INFO:teuthology.orchestra.run.smithi135.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.649 INFO:teuthology.orchestra.run.smithi135.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.650 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.651 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:19.0.0-2311.g36051606.el9.noarch 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: jq-1.6-16.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T17:27:11.652 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.653 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T17:27:11.654 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.655 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T17:27:11.656 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T17:27:11.657 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T17:27:11.658 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T17:27:11.659 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:19.0.0-2311.g36051606.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T17:27:11.660 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T17:27:11.661 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-03-26T17:27:11.661 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-03-26T17:27:12.117 DEBUG:teuthology.parallel:result is None 2024-03-26T17:27:12.118 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:27:12.349 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T17:27:12.375 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-2311.g36051606.el9 2024-03-26T17:27:12.375 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T17:27:12.375 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T17:27:12.377 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:27:12.600 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T17:27:12.624 INFO:teuthology.orchestra.run.smithi135.stdout:19.0.0-2311.g36051606.el9 2024-03-26T17:27:12.624 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T17:27:12.624 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T17:27:12.626 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:27:12.763 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T17:27:12.788 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-2311.g36051606.el9 2024-03-26T17:27:12.789 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2311.g36051606.el9 2024-03-26T17:27:12.789 INFO:teuthology.task.install:The correct ceph version 19.0.0-2311.g36051606 is installed. 2024-03-26T17:27:12.790 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-26T17:27:12.790 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:27:12.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T17:27:12.826 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:27:12.826 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T17:27:12.859 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:27:12.859 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T17:27:12.892 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-26T17:27:12.892 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:27:12.892 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T17:27:12.923 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T17:27:12.992 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:27:12.992 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T17:27:13.021 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T17:27:13.091 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:27:13.091 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T17:27:13.119 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T17:27:13.185 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-26T17:27:13.186 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:27:13.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T17:27:13.215 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T17:27:13.282 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:27:13.282 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T17:27:13.311 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T17:27:13.379 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:27:13.379 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T17:27:13.406 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T17:27:13.473 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-26T17:27:13.473 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:27:13.473 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T17:27:13.502 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T17:27:13.569 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:27:13.569 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T17:27:13.598 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T17:27:13.665 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:27:13.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T17:27:13.692 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T17:27:13.759 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-26T17:27:13.855 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', '\\(PG_DEGRADED', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-26T17:27:13.856 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:27:13.856 INFO:tasks.cephadm:Cluster fsid is 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:27:13.856 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-26T17:27:13.857 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.105', 'mon.b': '172.21.15.135', 'mon.c': '172.21.15.144'} 2024-03-26T17:27:13.857 INFO:tasks.cephadm:First mon is mon.a on smithi105 2024-03-26T17:27:13.857 INFO:tasks.cephadm:First mgr is x 2024-03-26T17:27:13.857 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-26T17:27:13.857 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname $(hostname -s) 2024-03-26T17:27:13.885 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname $(hostname -s) 2024-03-26T17:27:13.913 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-03-26T17:27:13.939 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-26T17:27:13.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:27:14.084 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '360516069d9393362c4cc6eb9371680fe16d66ab', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77808/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2311-g36051606', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2311.g36051606'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 17:53:08.349574', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240320.145742', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-26T17:27:14.225 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-batrick-testing-20240320.145742, sha1 360516069d9393362c4cc6eb9371680fe16d66ab from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:27:14.227 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm 2024-03-26T17:27:14.227 DEBUG:teuthology.orchestra.run.smithi105:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:14.709 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 17:27 /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:14.710 DEBUG:teuthology.orchestra.run.smithi135:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:15.179 INFO:teuthology.orchestra.run.smithi135.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 17:27 /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:15.180 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240320.145742/360516069d9393362c4cc6eb9371680fe16d66ab/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:15.667 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 769283 Mar 26 17:27 /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:15.667 DEBUG:teuthology.orchestra.run.smithi105:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:15.686 DEBUG:teuthology.orchestra.run.smithi135:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:15.705 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T17:27:15.739 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab on all hosts... 2024-03-26T17:27:15.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T17:27:15.741 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T17:27:15.747 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab pull 2024-03-26T17:27:16.554 INFO:teuthology.orchestra.run.smithi135.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:27:16.571 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:27:16.617 INFO:teuthology.orchestra.run.smithi105.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:27:53.817 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:27:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T17:27:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T17:27:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "repo_digests": [ 2024-03-26T17:27:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T17:27:53.817 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T17:27:53.817 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:28:05.852 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T17:28:05.853 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T17:28:05.853 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T17:28:05.853 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-03-26T17:28:05.853 INFO:teuthology.orchestra.run.smithi144.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T17:28:05.853 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-03-26T17:28:05.853 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T17:28:14.041 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-03-26T17:28:14.041 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T17:28:14.041 INFO:teuthology.orchestra.run.smithi135.stdout: "image_id": "497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016", 2024-03-26T17:28:14.041 INFO:teuthology.orchestra.run.smithi135.stdout: "repo_digests": [ 2024-03-26T17:28:14.041 INFO:teuthology.orchestra.run.smithi135.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9" 2024-03-26T17:28:14.041 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-03-26T17:28:14.041 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-03-26T17:28:14.065 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph 2024-03-26T17:28:14.100 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph 2024-03-26T17:28:14.129 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-03-26T17:28:14.165 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /etc/ceph 2024-03-26T17:28:14.197 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /etc/ceph 2024-03-26T17:28:14.224 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-03-26T17:28:14.255 INFO:tasks.cephadm:Writing seed config... 2024-03-26T17:28:14.258 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-26T17:28:14.258 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-26T17:28:14.258 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-26T17:28:14.259 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-26T17:28:14.259 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-26T17:28:14.259 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-26T17:28:14.259 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-26T17:28:14.259 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-26T17:28:14.260 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-26T17:28:14.260 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-26T17:28:14.260 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-26T17:28:14.260 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-26T17:28:14.260 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-26T17:28:14.260 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-26T17:28:14.261 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-26T17:28:14.261 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-26T17:28:14.261 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-26T17:28:14.261 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-26T17:28:14.261 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-26T17:28:14.261 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-26T17:28:14.262 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-26T17:28:14.262 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-26T17:28:14.262 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-03-26T17:28:14.262 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-26T17:28:14.262 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-26T17:28:14.262 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-26T17:28:14.263 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-26T17:28:14.263 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-26T17:28:14.263 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-03-26T17:28:14.263 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-26T17:28:14.263 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-26T17:28:14.263 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-26T17:28:14.264 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-26T17:28:14.264 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-26T17:28:14.264 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-26T17:28:14.264 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-26T17:28:14.264 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-26T17:28:14.264 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-26T17:28:14.264 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-26T17:28:14.265 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-26T17:28:14.265 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-26T17:28:14.265 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-26T17:28:14.265 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-26T17:28:14.265 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-26T17:28:14.265 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-26T17:28:14.265 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-26T17:28:14.266 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-26T17:28:14.266 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-26T17:28:14.267 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:28:14.267 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-26T17:28:14.285 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 15cee578-eb96-11ee-95d0-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-26T17:28:14.286 DEBUG:teuthology.orchestra.run.smithi105:mon.a> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.a.service 2024-03-26T17:28:14.329 DEBUG:teuthology.orchestra.run.smithi105:mgr.x> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.x.service 2024-03-26T17:28:14.373 INFO:tasks.cephadm:Bootstrapping... 2024-03-26T17:28:14.373 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab -v bootstrap --fsid 15cee578-eb96-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.105 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-26T17:28:14.568 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-26T17:28:14.568 INFO:teuthology.orchestra.run.smithi105.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab', '-v', 'bootstrap', '--fsid', '15cee578-eb96-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.105', '--skip-admin-label'] 2024-03-26T17:28:14.568 INFO:teuthology.orchestra.run.smithi105.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-26T17:28:14.569 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying podman|docker is present... 2024-03-26T17:28:14.594 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T17:28:14.594 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying lvm2 is present... 2024-03-26T17:28:14.594 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying time synchronization is in place... 2024-03-26T17:28:14.601 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T17:28:14.601 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T17:28:14.607 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T17:28:14.607 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-03-26T17:28:14.613 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout enabled 2024-03-26T17:28:14.619 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout active 2024-03-26T17:28:14.619 INFO:teuthology.orchestra.run.smithi105.stdout:Unit chronyd.service is enabled and running 2024-03-26T17:28:14.619 INFO:teuthology.orchestra.run.smithi105.stdout:Repeating the final host check... 2024-03-26T17:28:14.644 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T17:28:14.644 INFO:teuthology.orchestra.run.smithi105.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-26T17:28:14.644 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl is present 2024-03-26T17:28:14.644 INFO:teuthology.orchestra.run.smithi105.stdout:lvcreate is present 2024-03-26T17:28:14.650 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T17:28:14.650 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T17:28:14.656 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T17:28:14.656 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-03-26T17:28:14.662 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout enabled 2024-03-26T17:28:14.668 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout active 2024-03-26T17:28:14.668 INFO:teuthology.orchestra.run.smithi105.stdout:Unit chronyd.service is enabled and running 2024-03-26T17:28:14.668 INFO:teuthology.orchestra.run.smithi105.stdout:Host looks OK 2024-03-26T17:28:14.668 INFO:teuthology.orchestra.run.smithi105.stdout:Cluster fsid: 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:14.668 INFO:teuthology.orchestra.run.smithi105.stdout:Acquiring lock 139964840940496 on /run/cephadm/15cee578-eb96-11ee-95d0-87774f69a715.lock 2024-03-26T17:28:14.668 INFO:teuthology.orchestra.run.smithi105.stdout:Lock 139964840940496 acquired on /run/cephadm/15cee578-eb96-11ee-95d0-87774f69a715.lock 2024-03-26T17:28:14.669 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying IP 172.21.15.105 port 3300 ... 2024-03-26T17:28:14.669 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying IP 172.21.15.105 port 6789 ... 2024-03-26T17:28:14.669 INFO:teuthology.orchestra.run.smithi105.stdout:Base mon IP(s) is [172.21.15.105:3300, 172.21.15.105:6789], mon addrv is [v2:172.21.15.105:3300,v1:172.21.15.105:6789] 2024-03-26T17:28:14.672 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.105 metric 100 2024-03-26T17:28:14.672 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.105 metric 100 2024-03-26T17:28:14.675 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-26T17:28:14.675 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-26T17:28:14.675 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2024-03-26T17:28:14.677 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-26T17:28:14.677 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-26T17:28:14.677 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T17:28:14.677 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-26T17:28:14.677 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-26T17:28:14.677 INFO:teuthology.orchestra.run.smithi105.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T17:28:14.678 INFO:teuthology.orchestra.run.smithi105.stdout:Mon IP `172.21.15.105` is in CIDR network `172.21.0.0/20` 2024-03-26T17:28:14.678 INFO:teuthology.orchestra.run.smithi105.stdout:Mon IP `172.21.15.105` is in CIDR network `172.21.0.0/20` 2024-03-26T17:28:14.678 INFO:teuthology.orchestra.run.smithi105.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-26T17:28:14.679 INFO:teuthology.orchestra.run.smithi105.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-26T17:28:14.679 INFO:teuthology.orchestra.run.smithi105.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:28:15.157 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stdout 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 2024-03-26T17:28:15.157 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab... 2024-03-26T17:28:15.157 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Getting image source signatures 2024-03-26T17:28:15.157 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Copying blob sha256:0388617eb36698e8cd0b89da9480e32fffb3f5cb74210feb5d82e6789f473b79 2024-03-26T17:28:15.157 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-26T17:28:15.157 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Copying config sha256:497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 2024-03-26T17:28:15.157 INFO:teuthology.orchestra.run.smithi105.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-26T17:28:16.162 INFO:teuthology.orchestra.run.smithi105.stdout:ceph: stdout ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev) 2024-03-26T17:28:16.162 INFO:teuthology.orchestra.run.smithi105.stdout:Ceph version: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev) 2024-03-26T17:28:16.162 INFO:teuthology.orchestra.run.smithi105.stdout:Extracting ceph user uid/gid from container image... 2024-03-26T17:28:17.159 INFO:teuthology.orchestra.run.smithi105.stdout:stat: stdout 167 167 2024-03-26T17:28:17.159 INFO:teuthology.orchestra.run.smithi105.stdout:Creating initial keys... 2024-03-26T17:28:25.517 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph-authtool: stdout AQCzBQNmDK+BDBAAWp3xQpwL7aodPbpD7Ai2uQ== 2024-03-26T17:28:28.431 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph-authtool: stdout AQC7BQNmCo7FMhAA9pwdoBfK3ATJsaIVR60Utg== 2024-03-26T17:28:29.362 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph-authtool: stdout AQC8BQNmWj94LhAAX0tqkkxnZH3OhGxdP6YeUA== 2024-03-26T17:28:29.363 INFO:teuthology.orchestra.run.smithi105.stdout:Creating initial monmap... 2024-03-26T17:28:30.335 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T17:28:30.335 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-26T17:28:30.336 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:30.336 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T17:28:30.336 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool for a [v2:172.21.15.105:3300,v1:172.21.15.105:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T17:28:30.336 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = quincy 2024-03-26T17:28:30.336 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: set fsid to 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:30.336 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T17:28:30.336 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:28:30.336 INFO:teuthology.orchestra.run.smithi105.stdout:Creating mon... 2024-03-26T17:28:31.190 INFO:teuthology.orchestra.run.smithi105.stdout:create mon.a on 2024-03-26T17:28:31.396 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-26T17:28:31.535 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-26T17:28:31.703 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-15cee578-eb96-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-15cee578-eb96-11ee-95d0-87774f69a715.target. 2024-03-26T17:28:31.703 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-15cee578-eb96-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-15cee578-eb96-11ee-95d0-87774f69a715.target. 2024-03-26T17:28:31.930 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.a 2024-03-26T17:28:31.930 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Failed to reset failed state of unit ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-26T17:28:32.093 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-15cee578-eb96-11ee-95d0-87774f69a715.target.wants/ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-15cee578-eb96-11ee-95d0-87774f69a715@.service. 2024-03-26T17:28:32.639 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T17:28:32.639 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout disabled 2024-03-26T17:28:32.647 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T17:28:32.647 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-03-26T17:28:32.647 INFO:teuthology.orchestra.run.smithi105.stdout:firewalld.service is not enabled 2024-03-26T17:28:32.647 INFO:teuthology.orchestra.run.smithi105.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T17:28:32.647 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mon to start... 2024-03-26T17:28:32.647 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mon... 2024-03-26T17:28:34.518 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout cluster: 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout id: 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout services: 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 1.38406s) 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout data: 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout pgs: 2024-03-26T17:28:34.519 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.926+0000 7f2a903d9700 1 Processor -- start 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.926+0000 7f2a903d9700 1 -- start start 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.926+0000 7f2a903d9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a88108220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.926+0000 7f2a903d9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88108760 con 0x7f2a88107e00 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.927+0000 7f2a8e175700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a88108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.927+0000 7f2a8e175700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a88108220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34918/0 (socket says 172.21.15.105:34918) 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.927+0000 7f2a8e175700 1 -- 172.21.15.105:0/2159922245 learned_addr learned my addr 172.21.15.105:0/2159922245 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.927+0000 7f2a8e175700 1 -- 172.21.15.105:0/2159922245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a88108f60 con 0x7f2a88107e00 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.927+0000 7f2a8e175700 1 --2- 172.21.15.105:0/2159922245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a88108220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009e80 tx=0x7f2a8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6295963e0b20d54a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.927+0000 7f2a8d173700 1 -- 172.21.15.105:0/2159922245 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a84017070 con 0x7f2a88107e00 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.927+0000 7f2a8d173700 1 -- 172.21.15.105:0/2159922245 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84003a20 con 0x7f2a88107e00 2024-03-26T17:28:34.520 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.927+0000 7f2a8d173700 1 -- 172.21.15.105:0/2159922245 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a84003d20 con 0x7f2a88107e00 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.928+0000 7f2a903d9700 1 -- 172.21.15.105:0/2159922245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 msgr2=0x7f2a88108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.928+0000 7f2a903d9700 1 --2- 172.21.15.105:0/2159922245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a88108220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009e80 tx=0x7f2a8400b330 comp rx=0 tx=0).stop 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.928+0000 7f2a903d9700 1 -- 172.21.15.105:0/2159922245 shutdown_connections 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.928+0000 7f2a903d9700 1 --2- 172.21.15.105:0/2159922245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a88108220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.928+0000 7f2a903d9700 1 -- 172.21.15.105:0/2159922245 >> 172.21.15.105:0/2159922245 conn(0x7f2a881033a0 msgr2=0x7f2a881057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.928+0000 7f2a903d9700 1 -- 172.21.15.105:0/2159922245 shutdown_connections 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.928+0000 7f2a903d9700 1 -- 172.21.15.105:0/2159922245 wait complete. 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a903d9700 1 Processor -- start 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a903d9700 1 -- start start 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a903d9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a8819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a903d9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88109430 con 0x7f2a88107e00 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a8e175700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a8819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a8e175700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a8819b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34920/0 (socket says 172.21.15.105:34920) 2024-03-26T17:28:34.521 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a8e175700 1 -- 172.21.15.105:0/2777536625 learned_addr learned my addr 172.21.15.105:0/2777536625 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a8e175700 1 -- 172.21.15.105:0/2777536625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8819ba40 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.929+0000 7f2a8e175700 1 --2- 172.21.15.105:0/2777536625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a8819b500 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2a88108a90 tx=0x7f2a84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.930+0000 7f2a7f7fe700 1 -- 172.21.15.105:0/2777536625 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a84022020 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.930+0000 7f2a7f7fe700 1 -- 172.21.15.105:0/2777536625 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84018040 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.930+0000 7f2a7f7fe700 1 -- 172.21.15.105:0/2777536625 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a84017070 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.930+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8819bcd0 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.930+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8819e9c0 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.930+0000 7f2a7f7fe700 1 -- 172.21.15.105:0/2777536625 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a84025020 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.930+0000 7f2a7f7fe700 1 -- 172.21.15.105:0/2777536625 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a8400f050 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.930+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a880639e0 con 0x7f2a88107e00 2024-03-26T17:28:34.522 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:33.933+0000 7f2a7f7fe700 1 -- 172.21.15.105:0/2777536625 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2a8402c030 con 0x7f2a88107e00 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.003+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f2a8802dd90 con 0x7f2a88107e00 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.003+0000 7f2a7f7fe700 1 -- 172.21.15.105:0/2777536625 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f2a8401e480 con 0x7f2a88107e00 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.004+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 msgr2=0x7f2a8819b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.004+0000 7f2a903d9700 1 --2- 172.21.15.105:0/2777536625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a8819b500 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2a88108a90 tx=0x7f2a84004ad0 comp rx=0 tx=0).stop 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.004+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 shutdown_connections 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.004+0000 7f2a903d9700 1 --2- 172.21.15.105:0/2777536625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a88107e00 0x7f2a8819b500 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.004+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 >> 172.21.15.105:0/2777536625 conn(0x7f2a881033a0 msgr2=0x7f2a881920a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.004+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 shutdown_connections 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.004+0000 7f2a903d9700 1 -- 172.21.15.105:0/2777536625 wait complete. 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:mon is available 2024-03-26T17:28:34.523 INFO:teuthology.orchestra.run.smithi105.stdout:Assimilating anything we can from ceph.conf... 2024-03-26T17:28:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T17:28:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout fsid = 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.105:3300,v1:172.21.15.105:6789] 2024-03-26T17:28:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T17:28:35.575 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T17:28:35.576 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.919+0000 7f37e07dd700 1 Processor -- start 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.919+0000 7f37e07dd700 1 -- start start 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.919+0000 7f37e07dd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d8108260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.920+0000 7f37e07dd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8108830 con 0x7f37d8107e40 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.920+0000 7f37de579700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d8108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.920+0000 7f37de579700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d8108260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47080/0 (socket says 172.21.15.105:47080) 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.920+0000 7f37de579700 1 -- 172.21.15.105:0/3355088754 learned_addr learned my addr 172.21.15.105:0/3355088754 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.920+0000 7f37de579700 1 -- 172.21.15.105:0/3355088754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d8108ff0 con 0x7f37d8107e40 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.920+0000 7f37de579700 1 --2- 172.21.15.105:0/3355088754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d8108260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f37d40098a0 tx=0x7f37d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d916b781607bbf1a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37dd577700 1 -- 172.21.15.105:0/3355088754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d4004160 con 0x7f37d8107e40 2024-03-26T17:28:35.577 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37dd577700 1 -- 172.21.15.105:0/3355088754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d4004300 con 0x7f37d8107e40 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37dd577700 1 -- 172.21.15.105:0/3355088754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d40036a0 con 0x7f37d8107e40 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37e07dd700 1 -- 172.21.15.105:0/3355088754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 msgr2=0x7f37d8108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37e07dd700 1 --2- 172.21.15.105:0/3355088754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d8108260 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f37d40098a0 tx=0x7f37d4009bb0 comp rx=0 tx=0).stop 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37e07dd700 1 -- 172.21.15.105:0/3355088754 shutdown_connections 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37e07dd700 1 --2- 172.21.15.105:0/3355088754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d8108260 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37e07dd700 1 -- 172.21.15.105:0/3355088754 >> 172.21.15.105:0/3355088754 conn(0x7f37d8103400 msgr2=0x7f37d8105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37e07dd700 1 -- 172.21.15.105:0/3355088754 shutdown_connections 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.921+0000 7f37e07dd700 1 -- 172.21.15.105:0/3355088754 wait complete. 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.922+0000 7f37e07dd700 1 Processor -- start 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.922+0000 7f37e07dd700 1 -- start start 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.922+0000 7f37e07dd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.922+0000 7f37e07dd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8109520 con 0x7f37d8107e40 2024-03-26T17:28:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.922+0000 7f37de579700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.922+0000 7f37de579700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47086/0 (socket says 172.21.15.105:47086) 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.922+0000 7f37de579700 1 -- 172.21.15.105:0/4014543355 learned_addr learned my addr 172.21.15.105:0/4014543355 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.922+0000 7f37de579700 1 -- 172.21.15.105:0/4014543355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d819ba30 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.923+0000 7f37de579700 1 --2- 172.21.15.105:0/4014543355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d819b4f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f37d4000c00 tx=0x7f37d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.923+0000 7f37cf7fe700 1 -- 172.21.15.105:0/4014543355 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d4003db0 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.923+0000 7f37cf7fe700 1 -- 172.21.15.105:0/4014543355 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d4011630 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.923+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d819bcc0 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.923+0000 7f37cf7fe700 1 -- 172.21.15.105:0/4014543355 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d4011950 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.923+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d819e9b0 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.923+0000 7f37cf7fe700 1 -- 172.21.15.105:0/4014543355 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f37d4011af0 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.923+0000 7f37cf7fe700 1 -- 172.21.15.105:0/4014543355 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f37d401c450 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.924+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d80639e0 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.926+0000 7f37cf7fe700 1 -- 172.21.15.105:0/4014543355 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f37d4032020 con 0x7f37d8107e40 2024-03-26T17:28:35.579 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.992+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f37d802de10 con 0x7f37d8107e40 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.995+0000 7f37cf7fe700 1 -- 172.21.15.105:0/4014543355 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+458 (secure 0 0 0) 0x7f37d4004260 con 0x7f37d8107e40 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.996+0000 7f37cf7fe700 1 -- 172.21.15.105:0/4014543355 <== mon.0 v2:172.21.15.105:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f37d401c6a0 con 0x7f37d8107e40 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.997+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 msgr2=0x7f37d819b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.997+0000 7f37e07dd700 1 --2- 172.21.15.105:0/4014543355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d819b4f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f37d4000c00 tx=0x7f37d4004750 comp rx=0 tx=0).stop 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.997+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 shutdown_connections 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.997+0000 7f37e07dd700 1 --2- 172.21.15.105:0/4014543355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37d8107e40 0x7f37d819b4f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.997+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 >> 172.21.15.105:0/4014543355 conn(0x7f37d8103400 msgr2=0x7f37d81921f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.997+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 shutdown_connections 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:34.997+0000 7f37e07dd700 1 -- 172.21.15.105:0/4014543355 wait complete. 2024-03-26T17:28:35.580 INFO:teuthology.orchestra.run.smithi105.stdout:Generating new minimal ceph.conf... 2024-03-26T17:28:36.671 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:36 smithi105 ceph-mon[31912]: from='client.? 172.21.15.105:0/897334127' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:28:36.678 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.981+0000 7f57821bc700 1 Processor -- start 2024-03-26T17:28:36.678 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.981+0000 7f57821bc700 1 -- start start 2024-03-26T17:28:36.678 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.981+0000 7f57821bc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c106510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:36.678 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.981+0000 7f57821bc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f577c106ae0 con 0x7f577c1060f0 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.981+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c106510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.981+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c106510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47088/0 (socket says 172.21.15.105:47088) 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.981+0000 7f577b7fe700 1 -- 172.21.15.105:0/1354800834 learned_addr learned my addr 172.21.15.105:0/1354800834 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.981+0000 7f577b7fe700 1 -- 172.21.15.105:0/1354800834 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c106e10 con 0x7f577c1060f0 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f577b7fe700 1 --2- 172.21.15.105:0/1354800834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c106510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57700098a0 tx=0x7f5770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62f534be150c009 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f577a7fc700 1 -- 172.21.15.105:0/1354800834 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5770004160 con 0x7f577c1060f0 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f577a7fc700 1 -- 172.21.15.105:0/1354800834 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5770004300 con 0x7f577c1060f0 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f577a7fc700 1 -- 172.21.15.105:0/1354800834 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5770011a80 con 0x7f577c1060f0 2024-03-26T17:28:36.679 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f57821bc700 1 -- 172.21.15.105:0/1354800834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 msgr2=0x7f577c106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f57821bc700 1 --2- 172.21.15.105:0/1354800834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c106510 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57700098a0 tx=0x7f5770009bb0 comp rx=0 tx=0).stop 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f57821bc700 1 -- 172.21.15.105:0/1354800834 shutdown_connections 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f57821bc700 1 --2- 172.21.15.105:0/1354800834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c106510 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f57821bc700 1 -- 172.21.15.105:0/1354800834 >> 172.21.15.105:0/1354800834 conn(0x7f577c101a40 msgr2=0x7f577c103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f57821bc700 1 -- 172.21.15.105:0/1354800834 shutdown_connections 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.982+0000 7f57821bc700 1 -- 172.21.15.105:0/1354800834 wait complete. 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.983+0000 7f57821bc700 1 Processor -- start 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.983+0000 7f57821bc700 1 -- start start 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.983+0000 7f57821bc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.983+0000 7f57821bc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1081d0 con 0x7f577c1060f0 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.983+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:36.680 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.983+0000 7f577b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47092/0 (socket says 172.21.15.105:47092) 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.983+0000 7f577b7fe700 1 -- 172.21.15.105:0/897334127 learned_addr learned my addr 172.21.15.105:0/897334127 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f577b7fe700 1 -- 172.21.15.105:0/897334127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c19e510 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f577b7fe700 1 --2- 172.21.15.105:0/897334127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c19dfd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5770006af0 tx=0x7f5770011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f5778ff9700 1 -- 172.21.15.105:0/897334127 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5770003930 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c19e7a0 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f5778ff9700 1 -- 172.21.15.105:0/897334127 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5770003ad0 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f5778ff9700 1 -- 172.21.15.105:0/897334127 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57700195e0 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c1a1490 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f5778ff9700 1 -- 172.21.15.105:0/897334127 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5770019c70 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.984+0000 7f5778ff9700 1 -- 172.21.15.105:0/897334127 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5770024730 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.985+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577c0639e0 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:35.987+0000 7f5778ff9700 1 -- 172.21.15.105:0/897334127 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5770043dd0 con 0x7f577c1060f0 2024-03-26T17:28:36.681 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.054+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f577c197530 con 0x7f577c1060f0 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.054+0000 7f5778ff9700 1 -- 172.21.15.105:0/897334127 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f577c197530 con 0x7f577c1060f0 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.055+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 msgr2=0x7f577c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.055+0000 7f57821bc700 1 --2- 172.21.15.105:0/897334127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c19dfd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5770006af0 tx=0x7f5770011b70 comp rx=0 tx=0).stop 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.055+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 shutdown_connections 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.055+0000 7f57821bc700 1 --2- 172.21.15.105:0/897334127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f577c1060f0 0x7f577c19dfd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.055+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 >> 172.21.15.105:0/897334127 conn(0x7f577c101a40 msgr2=0x7f577c1945f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.055+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 shutdown_connections 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:36.055+0000 7f57821bc700 1 -- 172.21.15.105:0/897334127 wait complete. 2024-03-26T17:28:36.682 INFO:teuthology.orchestra.run.smithi105.stdout:Restarting the monitor... 2024-03-26T17:28:36.933 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:36 smithi105 systemd[1]: Stopping Ceph mon.a for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:28:37.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:36 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a[31908]: 2024-03-26T17:28:36.936+0000 7f96eb4d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T17:28:37.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:36 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a[31908]: 2024-03-26T17:28:36.936+0000 7f96eb4d5700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-26T17:28:37.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:37 smithi105 podman[32156]: 2024-03-26 17:28:37.040562564 +0000 UTC m=+0.202185871 container died ddd95cc6418020d55bc5b9e9331b849e2118634f43a4ba1d0beb85462cdcc3f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T17:28:37.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:37 smithi105 podman[32156]: 2024-03-26 17:28:37.404481088 +0000 UTC m=+0.566104403 container cleanup ddd95cc6418020d55bc5b9e9331b849e2118634f43a4ba1d0beb85462cdcc3f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-26T17:28:37.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:37 smithi105 bash[32156]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a 2024-03-26T17:28:38.018 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:37 smithi105 podman[32170]: 2024-03-26 17:28:37.663700608 +0000 UTC m=+0.624799047 container remove ddd95cc6418020d55bc5b9e9331b849e2118634f43a4ba1d0beb85462cdcc3f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T17:28:38.018 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:37 smithi105 systemd[1]: ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-26T17:28:38.018 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:37 smithi105 systemd[1]: Stopped Ceph mon.a for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:28:38.018 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:37 smithi105 systemd[1]: Starting Ceph mon.a for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:28:38.257 INFO:teuthology.orchestra.run.smithi105.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-26T17:28:38.273 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 podman[32261]: 2024-03-26 17:28:38.023020075 +0000 UTC m=+0.103547223 container create beae2e9cfeec292fb059fbb4fa10f199eec28b6dffceee3623bb4902f0e34be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T17:28:38.273 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 podman[32261]: 2024-03-26 17:28:37.93772021 +0000 UTC m=+0.018247315 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 podman[32261]: 2024-03-26 17:28:38.175354353 +0000 UTC m=+0.255881467 container init beae2e9cfeec292fb059fbb4fa10f199eec28b6dffceee3623bb4902f0e34be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 podman[32261]: 2024-03-26 17:28:38.178385501 +0000 UTC m=+0.258912607 container start beae2e9cfeec292fb059fbb4fa10f199eec28b6dffceee3623bb4902f0e34be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev), process ceph-mon, pid 2 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: pidfile_write: ignore empty --pid-file 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: load: jerasure load: lrc 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: RocksDB version: 7.9.2 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Git sha 0 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Compile date 2024-03-20 16:36:06 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: DB SUMMARY 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: DB Session ID: 3IZBGHNESPW24Y6RYL6J 2024-03-26T17:28:38.274 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: CURRENT file: CURRENT 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105891 ; 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.error_if_exists: 0 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.create_if_missing: 0 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.paranoid_checks: 1 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T17:28:38.275 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.env: 0x55a9a85efc60 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.info_log: 0x55a9aa3afae0 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.statistics: (nil) 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.use_fsync: 0 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_log_file_size: 0 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.allow_fallocate: 1 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T17:28:38.276 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.use_direct_reads: 0 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.db_log_dir: 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.wal_dir: 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.write_buffer_manager: 0x55a9aa3821e0 2024-03-26T17:28:38.277 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T17:28:38.278 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T17:28:38.278 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T17:28:38.278 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T17:28:38.278 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T17:28:38.279 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T17:28:38.279 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T17:28:38.279 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T17:28:38.279 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.unordered_write: 0 2024-03-26T17:28:38.279 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.row_cache: None 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.wal_filter: None 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.two_write_queues: 0 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.wal_compression: 0 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.atomic_flush: 0 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T17:28:38.280 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.log_readahead_size: 0 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_background_jobs: 2 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_background_compactions: -1 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_subcompactions: 1 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T17:28:38.281 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_open_files: -1 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_background_flushes: -1 2024-03-26T17:28:38.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Compression algorithms supported: 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: kZSTD supported: 0 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: kXpressCompression supported: 0 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: kZlibCompression supported: 1 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: kSnappyCompression supported: 1 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: kLZ4Compression supported: 1 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: kBZip2Compression supported: 0 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-26T17:28:38.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.merge_operator: 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_filter: None 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a9aa3afbe0) 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout: cache_index_and_filter_blocks: 1 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout: pin_top_level_index_and_filter: 1 2024-03-26T17:28:38.284 INFO:journalctl@ceph.mon.a.smithi105.stdout: index_type: 0 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: data_block_index_type: 0 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: index_shortening: 1 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: checksum: 4 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: no_block_cache: 0 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_cache: 0x55a9aa3b5090 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_cache_name: BinnedLRUCache 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_cache_options: 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: capacity : 536870912 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: num_shard_bits : 4 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: strict_capacity_limit : 0 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: high_pri_pool_ratio: 0.000 2024-03-26T17:28:38.285 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_cache_compressed: (nil) 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: persistent_cache: (nil) 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_size: 4096 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_size_deviation: 10 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_restart_interval: 16 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: index_block_restart_interval: 1 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: metadata_block_size: 4096 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: partition_filters: 0 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: use_delta_encoding: 1 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: filter_policy: bloomfilter 2024-03-26T17:28:38.286 INFO:journalctl@ceph.mon.a.smithi105.stdout: whole_key_filtering: 1 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: verify_compression: 0 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: read_amp_bytes_per_bit: 0 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: format_version: 5 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: enable_index_compression: 1 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: block_align: 0 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: max_auto_readahead_size: 262144 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: prepopulate_block_cache: 0 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: initial_auto_readahead_size: 8192 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression: NoCompression 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T17:28:38.287 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T17:28:38.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T17:28:38.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.num_levels: 7 2024-03-26T17:28:38.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T17:28:38.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T17:28:38.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T17:28:38.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T17:28:38.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T17:28:38.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T17:28:38.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T17:28:38.290 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T17:28:38.291 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T17:28:38.292 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.table_properties_collectors: 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.inplace_update_support: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.bloom_locality: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.max_successive_merges: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.ttl: 2592000 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T17:28:38.293 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.enable_blob_files: false 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.min_blob_size: 0 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T17:28:38.294 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-26T17:28:38.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-26T17:28:38.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b8a7e351-a4a0-4fbc-a4e8-6880661aad0e 2024-03-26T17:28:38.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711474118207364, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-26T17:28:38.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-26T17:28:38.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711474118208385, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100789, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 334, "table_properties": {"data_size": 98709, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14326, "raw_average_key_size": 49, "raw_value_size": 90936, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711474118, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b8a7e351-a4a0-4fbc-a4e8-6880661aad0e", "db_session_id": "3IZBGHNESPW24Y6RYL6J", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-26T17:28:38.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711474118208443, "job": 1, "event": "recovery_finished"} 2024-03-26T17:28:38.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-26T17:28:38.295 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a9aa4a8000 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: DB pointer 0x55a9aa492000 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: ** DB Stats ** 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: ** Compaction Stats [default] ** 2024-03-26T17:28:38.296 INFO:journalctl@ceph.mon.a.smithi105.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: L0 2/0 100.26 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: Sum 2/0 100.26 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: ** Compaction Stats [default] ** 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-03-26T17:28:38.297 INFO:journalctl@ceph.mon.a.smithi105.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: Cumulative compaction: 0.00 GB write, 31.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: Interval compaction: 0.00 GB write, 31.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: Block cache BinnedLRUCache@0x55a9aa3b5090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: starting mon.a rank 0 at public addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] at bind addrs [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:38.298 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a@-1(???) e1 preinit fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a@-1(???).mds e1 new map 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a@-1(???).mds e1 print_map 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout: e1 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout: legacy client fscid: -1 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout: 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout: No filesystems configured 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:28:38.299 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 bash[32261]: beae2e9cfeec292fb059fbb4fa10f199eec28b6dffceee3623bb4902f0e34be2 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 systemd[1]: Started Ceph mon.a for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: monmap epoch 1 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: last_changed 2024-03-26T17:28:29.725786+0000 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: created 2024-03-26T17:28:29.725786+0000 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: min_mon_release 19 (squid) 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: election_strategy: 1 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: fsmap 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T17:28:38.300 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:38 smithi105 ceph-mon[32280]: mgrmap e1: no daemons active 2024-03-26T17:28:39.209 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.679+0000 7f6bc33d6700 1 Processor -- start 2024-03-26T17:28:39.209 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.679+0000 7f6bc33d6700 1 -- start start 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.679+0000 7f6bc33d6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.679+0000 7f6bc33d6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc07c8a0 con 0x7f6bbc07de70 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.679+0000 7f6bc1172700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.679+0000 7f6bc1172700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47098/0 (socket says 172.21.15.105:47098) 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.679+0000 7f6bc1172700 1 -- 172.21.15.105:0/2721856271 learned_addr learned my addr 172.21.15.105:0/2721856271 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.680+0000 7f6bc1172700 1 -- 172.21.15.105:0/2721856271 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc07ca20 con 0x7f6bbc07de70 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.680+0000 7f6bc1172700 1 --2- 172.21.15.105:0/2721856271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098a0 tx=0x7f6bb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6462a57b7f34be1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.680+0000 7f6bb3fff700 1 -- 172.21.15.105:0/2721856271 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb8004160 con 0x7f6bbc07de70 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.680+0000 7f6bb3fff700 1 -- 172.21.15.105:0/2721856271 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6bb8004300 con 0x7f6bbc07de70 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.680+0000 7f6bb3fff700 1 -- 172.21.15.105:0/2721856271 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb8011a80 con 0x7f6bbc07de70 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2721856271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 msgr2=0x7f6bbc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 --2- 172.21.15.105:0/2721856271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098a0 tx=0x7f6bb8009bb0 comp rx=0 tx=0).stop 2024-03-26T17:28:39.210 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2721856271 shutdown_connections 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 --2- 172.21.15.105:0/2721856271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2721856271 >> 172.21.15.105:0/2721856271 conn(0x7f6bbc1033b0 msgr2=0x7f6bbc1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2721856271 shutdown_connections 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2721856271 wait complete. 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 Processor -- start 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 -- start start 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.681+0000 7f6bc33d6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc10e3c0 con 0x7f6bbc07de70 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bc1172700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bc1172700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47104/0 (socket says 172.21.15.105:47104) 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bc1172700 1 -- 172.21.15.105:0/2452748548 learned_addr learned my addr 172.21.15.105:0/2452748548 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bc1172700 1 -- 172.21.15.105:0/2452748548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc19e520 con 0x7f6bbc07de70 2024-03-26T17:28:39.211 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bc1172700 1 --2- 172.21.15.105:0/2452748548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19dfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8000c00 tx=0x7f6bb8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bb27fc700 1 -- 172.21.15.105:0/2452748548 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb80039c0 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc19e7b0 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bb27fc700 1 -- 172.21.15.105:0/2452748548 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6bb8003b60 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bb27fc700 1 -- 172.21.15.105:0/2452748548 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb801a6f0 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc1a14a0 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bb27fc700 1 -- 172.21.15.105:0/2452748548 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6bb802d3f0 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.682+0000 7f6bb27fc700 1 -- 172.21.15.105:0/2452748548 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6bb8019700 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.683+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc0639e0 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.685+0000 7f6bb27fc700 1 -- 172.21.15.105:0/2452748548 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6bb8024a40 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.752+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f6bbc1a1670 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.754+0000 7f6bb27fc700 1 -- 172.21.15.105:0/2452748548 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f6bb8032030 con 0x7f6bbc07de70 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.754+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 msgr2=0x7f6bbc19dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:39.212 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.754+0000 7f6bc33d6700 1 --2- 172.21.15.105:0/2452748548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19dfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8000c00 tx=0x7f6bb8004290 comp rx=0 tx=0).stop 2024-03-26T17:28:39.213 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.754+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 shutdown_connections 2024-03-26T17:28:39.213 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.754+0000 7f6bc33d6700 1 --2- 172.21.15.105:0/2452748548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19dfe0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:39.213 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.754+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 >> 172.21.15.105:0/2452748548 conn(0x7f6bbc1033b0 msgr2=0x7f6bbc103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:39.213 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.755+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 shutdown_connections 2024-03-26T17:28:39.213 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:38.755+0000 7f6bc33d6700 1 -- 172.21.15.105:0/2452748548 wait complete. 2024-03-26T17:28:39.213 INFO:teuthology.orchestra.run.smithi105.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-26T17:28:39.234 INFO:teuthology.orchestra.run.smithi105.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-26T17:28:39.234 INFO:teuthology.orchestra.run.smithi105.stdout:Creating mgr... 2024-03-26T17:28:39.234 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-26T17:28:39.235 INFO:teuthology.orchestra.run.smithi105.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-26T17:28:39.456 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.x 2024-03-26T17:28:39.456 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Failed to reset failed state of unit ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-26T17:28:39.598 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-15cee578-eb96-11ee-95d0-87774f69a715.target.wants/ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-15cee578-eb96-11ee-95d0-87774f69a715@.service. 2024-03-26T17:28:39.748 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:39 smithi105 systemd[1]: Starting Ceph mgr.x for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:28:40.072 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:39 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2452748548' entity='client.admin' 2024-03-26T17:28:40.072 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:39 smithi105 podman[32494]: 2024-03-26 17:28:39.73750792 +0000 UTC m=+0.017186950 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab 2024-03-26T17:28:40.072 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:39 smithi105 podman[32494]: 2024-03-26 17:28:39.873133623 +0000 UTC m=+0.152812641 container create f6be31cfa6808843b95f1a08e92759009815bb7e6ea97e9c8d6bcd186dd1ff2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True) 2024-03-26T17:28:40.072 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:40 smithi105 podman[32494]: 2024-03-26 17:28:40.077672701 +0000 UTC m=+0.357351722 container init f6be31cfa6808843b95f1a08e92759009815bb7e6ea97e9c8d6bcd186dd1ff2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T17:28:40.160 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T17:28:40.161 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout disabled 2024-03-26T17:28:40.168 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T17:28:40.168 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-03-26T17:28:40.168 INFO:teuthology.orchestra.run.smithi105.stdout:firewalld.service is not enabled 2024-03-26T17:28:40.168 INFO:teuthology.orchestra.run.smithi105.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T17:28:40.176 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T17:28:40.176 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout disabled 2024-03-26T17:28:40.184 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T17:28:40.184 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-03-26T17:28:40.184 INFO:teuthology.orchestra.run.smithi105.stdout:firewalld.service is not enabled 2024-03-26T17:28:40.184 INFO:teuthology.orchestra.run.smithi105.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-26T17:28:40.184 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mgr to start... 2024-03-26T17:28:40.184 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mgr... 2024-03-26T17:28:40.355 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:40 smithi105 podman[32494]: 2024-03-26 17:28:40.080157137 +0000 UTC m=+0.359836156 container start f6be31cfa6808843b95f1a08e92759009815bb7e6ea97e9c8d6bcd186dd1ff2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-26T17:28:40.355 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:40 smithi105 bash[32494]: f6be31cfa6808843b95f1a08e92759009815bb7e6ea97e9c8d6bcd186dd1ff2b 2024-03-26T17:28:40.355 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:40 smithi105 systemd[1]: Started Ceph mgr.x for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:28:40.962 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:40 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4225867518' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T17:28:40.962 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:40.652+0000 7f322aa03200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:28:40.962 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:40.753+0000 7f322aa03200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:28:41.258 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:41.258 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:28:41.258 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:28:41.258 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T17:28:41.258 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T17:28:41.258 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T17:28:41.258 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T17:28:41.258 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 0 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:41.259 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T17:28:41.260 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T17:28:41.261 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T17:28:32.622166+0000", 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:28:41.262 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.595+0000 7f18f4299700 1 Processor -- start 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.595+0000 7f18f4299700 1 -- start start 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.596+0000 7f18f4299700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec107110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.596+0000 7f18f4299700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec1076e0 con 0x7f18ec106cf0 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.596+0000 7f18f2035700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec107110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.596+0000 7f18f2035700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec107110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47134/0 (socket says 172.21.15.105:47134) 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.596+0000 7f18f2035700 1 -- 172.21.15.105:0/3791909271 learned_addr learned my addr 172.21.15.105:0/3791909271 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.596+0000 7f18f2035700 1 -- 172.21.15.105:0/3791909271 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec107ef0 con 0x7f18ec106cf0 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.596+0000 7f18f2035700 1 --2- 172.21.15.105:0/3791909271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec107110 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00bb40 tx=0x7f18dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827fbf9a17fb81eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.597+0000 7f18f1033700 1 -- 172.21.15.105:0/3791909271 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18dc00dbe0 con 0x7f18ec106cf0 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.597+0000 7f18f1033700 1 -- 172.21.15.105:0/3791909271 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f18dc0044c0 con 0x7f18ec106cf0 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.597+0000 7f18f1033700 1 -- 172.21.15.105:0/3791909271 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18dc010470 con 0x7f18ec106cf0 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.597+0000 7f18f4299700 1 -- 172.21.15.105:0/3791909271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 msgr2=0x7f18ec107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:41.263 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.597+0000 7f18f4299700 1 --2- 172.21.15.105:0/3791909271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec107110 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00bb40 tx=0x7f18dc00be50 comp rx=0 tx=0).stop 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.597+0000 7f18f4299700 1 -- 172.21.15.105:0/3791909271 shutdown_connections 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.597+0000 7f18f4299700 1 --2- 172.21.15.105:0/3791909271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec107110 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.597+0000 7f18f4299700 1 -- 172.21.15.105:0/3791909271 >> 172.21.15.105:0/3791909271 conn(0x7f18ec1022b0 msgr2=0x7f18ec1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f4299700 1 -- 172.21.15.105:0/3791909271 shutdown_connections 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f4299700 1 -- 172.21.15.105:0/3791909271 wait complete. 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f4299700 1 Processor -- start 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f4299700 1 -- start start 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f4299700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f4299700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec07afa0 con 0x7f18ec106cf0 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f2035700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f2035700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47144/0 (socket says 172.21.15.105:47144) 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.598+0000 7f18f2035700 1 -- 172.21.15.105:0/4225867518 learned_addr learned my addr 172.21.15.105:0/4225867518 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:41.264 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18f2035700 1 -- 172.21.15.105:0/4225867518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec19a0f0 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18f2035700 1 --2- 172.21.15.105:0/4225867518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec199bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00b340 tx=0x7f18dc003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18e37fe700 1 -- 172.21.15.105:0/4225867518 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18dc027070 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18e37fe700 1 -- 172.21.15.105:0/4225867518 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f18dc0104e0 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec19a380 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18e37fe700 1 -- 172.21.15.105:0/4225867518 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18dc0229e0 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec19d070 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18e37fe700 1 -- 172.21.15.105:0/4225867518 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f18dc02a070 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.599+0000 7f18e37fe700 1 -- 172.21.15.105:0/4225867518 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18dc01f070 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.600+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec0639e0 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.602+0000 7f18e37fe700 1 -- 172.21.15.105:0/4225867518 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f18dc031070 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.675+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f18ec193110 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.675+0000 7f18e37fe700 1 -- 172.21.15.105:0/4225867518 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f18ec193110 con 0x7f18ec106cf0 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.676+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 msgr2=0x7f18ec199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:41.265 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.676+0000 7f18f4299700 1 --2- 172.21.15.105:0/4225867518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec199bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00b340 tx=0x7f18dc003fe0 comp rx=0 tx=0).stop 2024-03-26T17:28:41.266 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.676+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 shutdown_connections 2024-03-26T17:28:41.266 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.676+0000 7f18f4299700 1 --2- 172.21.15.105:0/4225867518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18ec106cf0 0x7f18ec199bb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:41.266 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.676+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 >> 172.21.15.105:0/4225867518 conn(0x7f18ec1022b0 msgr2=0x7f18ec1901d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:41.266 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.676+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 shutdown_connections 2024-03-26T17:28:41.266 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:40.676+0000 7f18f4299700 1 -- 172.21.15.105:0/4225867518 wait complete. 2024-03-26T17:28:41.266 INFO:teuthology.orchestra.run.smithi105.stdout:mgr not available, waiting (1/15)... 2024-03-26T17:28:41.627 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:41.400+0000 7f322aa03200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:28:41.628 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:41.492+0000 7f322aa03200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:28:41.628 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:41.632+0000 7f322aa03200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:28:41.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:41.728+0000 7f322aa03200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:28:41.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:41.811+0000 7f322aa03200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:28:42.357 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:42.072+0000 7f322aa03200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:28:42.358 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:42.181+0000 7f322aa03200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:28:42.358 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:42.268+0000 7f322aa03200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:28:42.358 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:42.362+0000 7f322aa03200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:28:42.644 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:42.537+0000 7f322aa03200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:28:43.649 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:43.511+0000 7f322aa03200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:28:43.650 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:43.648+0000 7f322aa03200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:28:43.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:43 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1662108348' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T17:28:44.274 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:43.994+0000 7f322aa03200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:28:44.274 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:44.083+0000 7f322aa03200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:28:44.275 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:44.279+0000 7f322aa03200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:28:44.416 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T17:28:44.417 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T17:28:44.418 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 0 2024-03-26T17:28:44.418 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:44.418 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T17:28:44.418 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T17:28:44.418 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:44.418 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-26T17:28:44.418 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:44.419 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:44.420 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T17:28:32.622166+0000", 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:28:44.421 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.728+0000 7f3781f31700 1 Processor -- start 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.728+0000 7f3781f31700 1 -- start start 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.728+0000 7f3781f31700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c10aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.728+0000 7f3781f31700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f377c07be40 con 0x7f377c1088b0 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.728+0000 7f377b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c10aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.728+0000 7f377b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c10aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47158/0 (socket says 172.21.15.105:47158) 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.728+0000 7f377b7fe700 1 -- 172.21.15.105:0/2820301989 learned_addr learned my addr 172.21.15.105:0/2820301989 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.729+0000 7f377b7fe700 1 -- 172.21.15.105:0/2820301989 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c10b1e0 con 0x7f377c1088b0 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.729+0000 7f377b7fe700 1 --2- 172.21.15.105:0/2820301989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c10aca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f37700098a0 tx=0x7f3770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59be70dbe4ce963f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.729+0000 7f377a7fc700 1 -- 172.21.15.105:0/2820301989 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3770004160 con 0x7f377c1088b0 2024-03-26T17:28:44.422 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.729+0000 7f377a7fc700 1 -- 172.21.15.105:0/2820301989 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3770004300 con 0x7f377c1088b0 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.729+0000 7f377a7fc700 1 -- 172.21.15.105:0/2820301989 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3770011a80 con 0x7f377c1088b0 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 -- 172.21.15.105:0/2820301989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 msgr2=0x7f377c10aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 --2- 172.21.15.105:0/2820301989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c10aca0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f37700098a0 tx=0x7f3770009bb0 comp rx=0 tx=0).stop 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 -- 172.21.15.105:0/2820301989 shutdown_connections 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 --2- 172.21.15.105:0/2820301989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c10aca0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 -- 172.21.15.105:0/2820301989 >> 172.21.15.105:0/2820301989 conn(0x7f377c102300 msgr2=0x7f377c104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 -- 172.21.15.105:0/2820301989 shutdown_connections 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 -- 172.21.15.105:0/2820301989 wait complete. 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 Processor -- start 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.730+0000 7f3781f31700 1 -- start start 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f3781f31700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f3781f31700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f377c10c1a0 con 0x7f377c1088b0 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f377b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:44.423 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f377b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47164/0 (socket says 172.21.15.105:47164) 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f377b7fe700 1 -- 172.21.15.105:0/1662108348 learned_addr learned my addr 172.21.15.105:0/1662108348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f377b7fe700 1 -- 172.21.15.105:0/1662108348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c19c2d0 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f377b7fe700 1 --2- 172.21.15.105:0/1662108348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c19bd90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3770006af0 tx=0x7f3770011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f3778ff9700 1 -- 172.21.15.105:0/1662108348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3770003930 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f3778ff9700 1 -- 172.21.15.105:0/1662108348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3770003ad0 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c19c560 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f3778ff9700 1 -- 172.21.15.105:0/1662108348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37700195e0 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.731+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c19f210 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.732+0000 7f3778ff9700 1 -- 172.21.15.105:0/1662108348 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3770019c90 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.732+0000 7f3778ff9700 1 -- 172.21.15.105:0/1662108348 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3770024730 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.732+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377c0639e0 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.736+0000 7f3778ff9700 1 -- 172.21.15.105:0/1662108348 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3770043dd0 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.809+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f377c106a90 con 0x7f377c1088b0 2024-03-26T17:28:44.424 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.810+0000 7f3778ff9700 1 -- 172.21.15.105:0/1662108348 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3770032020 con 0x7f377c1088b0 2024-03-26T17:28:44.425 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.811+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 msgr2=0x7f377c19bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:44.425 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.811+0000 7f3781f31700 1 --2- 172.21.15.105:0/1662108348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c19bd90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3770006af0 tx=0x7f3770011b70 comp rx=0 tx=0).stop 2024-03-26T17:28:44.425 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.811+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 shutdown_connections 2024-03-26T17:28:44.425 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.811+0000 7f3781f31700 1 --2- 172.21.15.105:0/1662108348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f377c1088b0 0x7f377c19bd90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:44.425 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.811+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 >> 172.21.15.105:0/1662108348 conn(0x7f377c102300 msgr2=0x7f377c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:44.425 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.811+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 shutdown_connections 2024-03-26T17:28:44.425 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:43.811+0000 7f3781f31700 1 -- 172.21.15.105:0/1662108348 wait complete. 2024-03-26T17:28:44.425 INFO:teuthology.orchestra.run.smithi105.stdout:mgr not available, waiting (2/15)... 2024-03-26T17:28:44.644 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:44.466+0000 7f322aa03200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:28:45.010 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:44.750+0000 7f322aa03200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:28:45.010 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:45.015+0000 7f322aa03200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:28:45.394 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:45.249+0000 7f322aa03200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:28:46.394 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:46.046+0000 7f322aa03200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:28:46.847 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:46.624+0000 7f322aa03200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:28:46.847 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:46.724+0000 7f322aa03200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:28:47.144 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:46.944+0000 7f322aa03200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2736231093' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: Activating manager daemon x 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: mgrmap e2: x(active, starting, since 0.00302207s) 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: Manager daemon x is now available 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T17:28:47.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:46 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T17:28:47.496 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 0 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T17:28:47.497 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T17:28:47.498 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:47.499 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:47.500 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T17:28:32.622166+0000", 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.866+0000 7f0c3b2df700 1 Processor -- start 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.866+0000 7f0c3b2df700 1 -- start start 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.867+0000 7f0c3b2df700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34107120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.867+0000 7f0c3b2df700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341076f0 con 0x7f0c34106d00 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.867+0000 7f0c3907b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.867+0000 7f0c3907b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34078/0 (socket says 172.21.15.105:34078) 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.867+0000 7f0c3907b700 1 -- 172.21.15.105:0/1941891818 learned_addr learned my addr 172.21.15.105:0/1941891818 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.867+0000 7f0c3907b700 1 -- 172.21.15.105:0/1941891818 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34107f00 con 0x7f0c34106d00 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.867+0000 7f0c3907b700 1 --2- 172.21.15.105:0/1941891818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34107120 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000bb40 tx=0x7f0c3000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389e547962ff54ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:47.501 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.867+0000 7f0c2bfff700 1 -- 172.21.15.105:0/1941891818 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3000dbe0 con 0x7f0c34106d00 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.868+0000 7f0c2bfff700 1 -- 172.21.15.105:0/1941891818 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0c300044c0 con 0x7f0c34106d00 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.868+0000 7f0c2bfff700 1 -- 172.21.15.105:0/1941891818 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c30010470 con 0x7f0c34106d00 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.868+0000 7f0c3b2df700 1 -- 172.21.15.105:0/1941891818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 msgr2=0x7f0c34107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.868+0000 7f0c3b2df700 1 --2- 172.21.15.105:0/1941891818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34107120 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000bb40 tx=0x7f0c3000be50 comp rx=0 tx=0).stop 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.868+0000 7f0c3b2df700 1 -- 172.21.15.105:0/1941891818 shutdown_connections 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.868+0000 7f0c3b2df700 1 --2- 172.21.15.105:0/1941891818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34107120 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.868+0000 7f0c3b2df700 1 -- 172.21.15.105:0/1941891818 >> 172.21.15.105:0/1941891818 conn(0x7f0c341022a0 msgr2=0x7f0c341046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3b2df700 1 -- 172.21.15.105:0/1941891818 shutdown_connections 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3b2df700 1 -- 172.21.15.105:0/1941891818 wait complete. 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3b2df700 1 Processor -- start 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3b2df700 1 -- start start 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3b2df700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3b2df700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3407afa0 con 0x7f0c34106d00 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3907b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:47.502 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3907b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34092/0 (socket says 172.21.15.105:34092) 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.869+0000 7f0c3907b700 1 -- 172.21.15.105:0/2736231093 learned_addr learned my addr 172.21.15.105:0/2736231093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c3907b700 1 -- 172.21.15.105:0/2736231093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3419a140 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c3907b700 1 --2- 172.21.15.105:0/2736231093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34199c00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000bfd0 tx=0x7f0c30003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c2a7fc700 1 -- 172.21.15.105:0/2736231093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c30027070 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c2a7fc700 1 -- 172.21.15.105:0/2736231093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0c30010c00 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3419a3d0 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c2a7fc700 1 -- 172.21.15.105:0/2736231093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c30022ac0 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3419d070 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c2a7fc700 1 -- 172.21.15.105:0/2736231093 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0c3002a070 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.870+0000 7f0c2a7fc700 1 -- 172.21.15.105:0/2736231093 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0c3001f070 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.871+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c340639e0 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.873+0000 7f0c2a7fc700 1 -- 172.21.15.105:0/2736231093 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0c30031030 con 0x7f0c34106d00 2024-03-26T17:28:47.503 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.944+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0c34193160 con 0x7f0c34106d00 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.945+0000 7f0c2a7fc700 1 -- 172.21.15.105:0/2736231093 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0c30015070 con 0x7f0c34106d00 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.946+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 msgr2=0x7f0c34199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.946+0000 7f0c3b2df700 1 --2- 172.21.15.105:0/2736231093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34199c00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000bfd0 tx=0x7f0c30003fe0 comp rx=0 tx=0).stop 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.946+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 shutdown_connections 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.946+0000 7f0c3b2df700 1 --2- 172.21.15.105:0/2736231093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c34106d00 0x7f0c34199c00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.946+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 >> 172.21.15.105:0/2736231093 conn(0x7f0c341022a0 msgr2=0x7f0c34190220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.946+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 shutdown_connections 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:46.946+0000 7f0c3b2df700 1 -- 172.21.15.105:0/2736231093 wait complete. 2024-03-26T17:28:47.504 INFO:teuthology.orchestra.run.smithi105.stdout:mgr not available, waiting (3/15)... 2024-03-26T17:28:48.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:48 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' 2024-03-26T17:28:48.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:48 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' 2024-03-26T17:28:48.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:48 smithi105 ceph-mon[32280]: from='mgr.14100 172.21.15.105:0/3276794111' entity='mgr.x' 2024-03-26T17:28:48.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:48 smithi105 ceph-mon[32280]: mgrmap e3: x(active, since 1.00729s) 2024-03-26T17:28:50.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:50 smithi105 ceph-mon[32280]: mgrmap e4: x(active, since 2s) 2024-03-26T17:28:50.754 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:50.754 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:28:50.754 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:28:50.754 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T17:28:50.754 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T17:28:50.754 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T17:28:50.754 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 0 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T17:28:50.755 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T17:28:50.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T17:28:50.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ], 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T17:28:32.622166+0000", 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout }, 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:28:50.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.915+0000 7f1ca5e31700 1 Processor -- start 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.915+0000 7f1ca5e31700 1 -- start start 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.915+0000 7f1ca5e31700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca007c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.915+0000 7f1ca5e31700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca007c7d0 con 0x7f1ca007dde0 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.916+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca007c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.916+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca007c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34166/0 (socket says 172.21.15.105:34166) 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.916+0000 7f1c9f7fe700 1 -- 172.21.15.105:0/3929048056 learned_addr learned my addr 172.21.15.105:0/3929048056 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.916+0000 7f1c9f7fe700 1 -- 172.21.15.105:0/3929048056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca007c950 con 0x7f1ca007dde0 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.916+0000 7f1c9f7fe700 1 --2- 172.21.15.105:0/3929048056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca007c290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1c900098a0 tx=0x7f1c90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e7fd22ff9b72a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.916+0000 7f1c9effd700 1 -- 172.21.15.105:0/3929048056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90004160 con 0x7f1ca007dde0 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.916+0000 7f1c9effd700 1 -- 172.21.15.105:0/3929048056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1c90004300 con 0x7f1ca007dde0 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.916+0000 7f1c9effd700 1 -- 172.21.15.105:0/3929048056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90011a80 con 0x7f1ca007dde0 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 -- 172.21.15.105:0/3929048056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 msgr2=0x7f1ca007c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:50.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 --2- 172.21.15.105:0/3929048056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca007c290 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1c900098a0 tx=0x7f1c90009bb0 comp rx=0 tx=0).stop 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 -- 172.21.15.105:0/3929048056 shutdown_connections 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 --2- 172.21.15.105:0/3929048056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca007c290 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 -- 172.21.15.105:0/3929048056 >> 172.21.15.105:0/3929048056 conn(0x7f1ca0103400 msgr2=0x7f1ca0105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 -- 172.21.15.105:0/3929048056 shutdown_connections 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 -- 172.21.15.105:0/3929048056 wait complete. 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 Processor -- start 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.917+0000 7f1ca5e31700 1 -- start start 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.918+0000 7f1ca5e31700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.918+0000 7f1ca5e31700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca010e3d0 con 0x7f1ca007dde0 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.918+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.918+0000 7f1c9f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca019df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34168/0 (socket says 172.21.15.105:34168) 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.918+0000 7f1c9f7fe700 1 -- 172.21.15.105:0/2842634845 learned_addr learned my addr 172.21.15.105:0/2842634845 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:50.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.918+0000 7f1c9f7fe700 1 -- 172.21.15.105:0/2842634845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca019e4b0 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.918+0000 7f1c9f7fe700 1 --2- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca019df70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1c90006af0 tx=0x7f1c90011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.918+0000 7f1c9d7fa700 1 -- 172.21.15.105:0/2842634845 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c90003930 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.919+0000 7f1c9d7fa700 1 -- 172.21.15.105:0/2842634845 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1c90003ad0 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.919+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca019e740 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.919+0000 7f1c9d7fa700 1 -- 172.21.15.105:0/2842634845 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c900195e0 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.919+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca01a1430 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.919+0000 7f1c9d7fa700 1 -- 172.21.15.105:0/2842634845 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7f1c900198e0 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.920+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca00639e0 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.920+0000 7f1c9d7fa700 1 --2- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f1c8c0390e0 0x7f1c8c03b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.920+0000 7f1c9d7fa700 1 -- 172.21.15.105:0/2842634845 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1c9004d660 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.920+0000 7f1c96ffd700 1 --2- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f1c8c0390e0 0x7f1c8c03b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.920+0000 7f1c96ffd700 1 --2- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f1c8c0390e0 0x7f1c8c03b5a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c88006fd0 tx=0x7f1c88006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:49.925+0000 7f1c9d7fa700 1 -- 172.21.15.105:0/2842634845 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1c90020240 con 0x7f1ca007dde0 2024-03-26T17:28:50.761 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.201+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1ca01a1600 con 0x7f1ca007dde0 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.201+0000 7f1c9d7fa700 1 -- 172.21.15.105:0/2842634845 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1ca01a1600 con 0x7f1ca007dde0 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.204+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f1c8c0390e0 msgr2=0x7f1c8c03b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.204+0000 7f1ca5e31700 1 --2- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f1c8c0390e0 0x7f1c8c03b5a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c88006fd0 tx=0x7f1c88006e40 comp rx=0 tx=0).stop 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.204+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 msgr2=0x7f1ca019df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.204+0000 7f1ca5e31700 1 --2- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca019df70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1c90006af0 tx=0x7f1c90011b70 comp rx=0 tx=0).stop 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.205+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 shutdown_connections 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.205+0000 7f1ca5e31700 1 --2- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f1c8c0390e0 0x7f1c8c03b5a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.205+0000 7f1ca5e31700 1 --2- 172.21.15.105:0/2842634845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ca007dde0 0x7f1ca019df70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.205+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 >> 172.21.15.105:0/2842634845 conn(0x7f1ca0103400 msgr2=0x7f1ca0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.205+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 shutdown_connections 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:50.205+0000 7f1ca5e31700 1 -- 172.21.15.105:0/2842634845 wait complete. 2024-03-26T17:28:50.762 INFO:teuthology.orchestra.run.smithi105.stdout:mgr is available 2024-03-26T17:28:51.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:51 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2842634845' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T17:28:51.965 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:51.965 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T17:28:51.965 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout fsid = 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:28:51.965 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T17:28:51.965 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T17:28:51.966 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.181+0000 7f76e5857700 1 Processor -- start 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.181+0000 7f76e5857700 1 -- start start 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.181+0000 7f76e5857700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e010acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.181+0000 7f76e5857700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76e007be40 con 0x7f76e01088c0 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.181+0000 7f76deffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e010acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.181+0000 7f76deffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e010acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34176/0 (socket says 172.21.15.105:34176) 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.181+0000 7f76deffd700 1 -- 172.21.15.105:0/4140972919 learned_addr learned my addr 172.21.15.105:0/4140972919 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.182+0000 7f76deffd700 1 -- 172.21.15.105:0/4140972919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e010b1f0 con 0x7f76e01088c0 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.182+0000 7f76deffd700 1 --2- 172.21.15.105:0/4140972919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e010acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f76d00098a0 tx=0x7f76d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c71b57fe8c05c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.182+0000 7f76ddffb700 1 -- 172.21.15.105:0/4140972919 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d0004160 con 0x7f76e01088c0 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.182+0000 7f76ddffb700 1 -- 172.21.15.105:0/4140972919 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f76d0004300 con 0x7f76e01088c0 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.182+0000 7f76ddffb700 1 -- 172.21.15.105:0/4140972919 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d0011a80 con 0x7f76e01088c0 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.182+0000 7f76e5857700 1 -- 172.21.15.105:0/4140972919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 msgr2=0x7f76e010acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.182+0000 7f76e5857700 1 --2- 172.21.15.105:0/4140972919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e010acb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f76d00098a0 tx=0x7f76d0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:28:51.967 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 -- 172.21.15.105:0/4140972919 shutdown_connections 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 --2- 172.21.15.105:0/4140972919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e010acb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 -- 172.21.15.105:0/4140972919 >> 172.21.15.105:0/4140972919 conn(0x7f76e01022d0 msgr2=0x7f76e01046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 -- 172.21.15.105:0/4140972919 shutdown_connections 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 -- 172.21.15.105:0/4140972919 wait complete. 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 Processor -- start 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 -- start start 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.183+0000 7f76e5857700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76e010c1b0 con 0x7f76e01088c0 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76deffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76deffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34182/0 (socket says 172.21.15.105:34182) 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76deffd700 1 -- 172.21.15.105:0/1276976164 learned_addr learned my addr 172.21.15.105:0/1276976164 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76deffd700 1 -- 172.21.15.105:0/1276976164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e019c2e0 con 0x7f76e01088c0 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76deffd700 1 --2- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e019bda0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76d0011a50 tx=0x7f76d0004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76e4855700 1 -- 172.21.15.105:0/1276976164 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d00039f0 con 0x7f76e01088c0 2024-03-26T17:28:51.968 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76e4855700 1 -- 172.21.15.105:0/1276976164 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f76d0003b90 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e019c570 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76e4855700 1 -- 172.21.15.105:0/1276976164 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76d001a600 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.184+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e019f220 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.185+0000 7f76e4855700 1 -- 172.21.15.105:0/1276976164 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7f76d001a900 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.185+0000 7f76e4855700 1 --2- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f76c8039130 0x7f76c803b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.185+0000 7f76e4855700 1 -- 172.21.15.105:0/1276976164 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f76d004d5c0 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.185+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e00639e0 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.185+0000 7f76de7fc700 1 --2- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f76c8039130 0x7f76c803b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.186+0000 7f76de7fc700 1 --2- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f76c8039130 0x7f76c803b5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f76d4006fd0 tx=0x7f76d4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.190+0000 7f76e4855700 1 -- 172.21.15.105:0/1276976164 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f76d000ba30 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.382+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f76e01a0990 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.383+0000 7f76e4855700 1 -- 172.21.15.105:0/1276976164 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+400 (secure 0 0 0) 0x7f76d004cfe0 con 0x7f76e01088c0 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.385+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f76c8039130 msgr2=0x7f76c803b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:51.969 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.385+0000 7f76e5857700 1 --2- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f76c8039130 0x7f76c803b5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f76d4006fd0 tx=0x7f76d4006e40 comp rx=0 tx=0).stop 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.385+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 msgr2=0x7f76e019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.385+0000 7f76e5857700 1 --2- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e019bda0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76d0011a50 tx=0x7f76d0004290 comp rx=0 tx=0).stop 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.385+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 shutdown_connections 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.385+0000 7f76e5857700 1 --2- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f76c8039130 0x7f76c803b5f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.385+0000 7f76e5857700 1 --2- 172.21.15.105:0/1276976164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76e01088c0 0x7f76e019bda0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.385+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 >> 172.21.15.105:0/1276976164 conn(0x7f76e01022d0 msgr2=0x7f76e0102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.386+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 shutdown_connections 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:51.386+0000 7f76e5857700 1 -- 172.21.15.105:0/1276976164 wait complete. 2024-03-26T17:28:51.970 INFO:teuthology.orchestra.run.smithi105.stdout:Enabling cephadm module... 2024-03-26T17:28:52.217 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:52 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1276976164' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T17:28:53.312 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:53 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/471964377' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-26T17:28:53.313 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:53 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: ignoring --setuser ceph since I am not root 2024-03-26T17:28:53.313 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:53 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: ignoring --setgroup ceph since I am not root 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.406+0000 7ffb6a531700 1 Processor -- start 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.406+0000 7ffb6a531700 1 -- start start 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.406+0000 7ffb6a531700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb64107120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.406+0000 7ffb6a531700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641076f0 con 0x7ffb64106d00 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.407+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb64107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.407+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb64107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34188/0 (socket says 172.21.15.105:34188) 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.407+0000 7ffb63fff700 1 -- 172.21.15.105:0/4215882146 learned_addr learned my addr 172.21.15.105:0/4215882146 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.407+0000 7ffb63fff700 1 -- 172.21.15.105:0/4215882146 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb64107f00 con 0x7ffb64106d00 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.407+0000 7ffb63fff700 1 --2- 172.21.15.105:0/4215882146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb64107120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffb540098a0 tx=0x7ffb54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9997453ece60bf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.407+0000 7ffb637fe700 1 -- 172.21.15.105:0/4215882146 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54004160 con 0x7ffb64106d00 2024-03-26T17:28:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.407+0000 7ffb637fe700 1 -- 172.21.15.105:0/4215882146 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ffb54004300 con 0x7ffb64106d00 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.407+0000 7ffb637fe700 1 -- 172.21.15.105:0/4215882146 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54011a80 con 0x7ffb64106d00 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.408+0000 7ffb6a531700 1 -- 172.21.15.105:0/4215882146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 msgr2=0x7ffb64107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.408+0000 7ffb6a531700 1 --2- 172.21.15.105:0/4215882146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb64107120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffb540098a0 tx=0x7ffb54009bb0 comp rx=0 tx=0).stop 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.408+0000 7ffb6a531700 1 -- 172.21.15.105:0/4215882146 shutdown_connections 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.408+0000 7ffb6a531700 1 --2- 172.21.15.105:0/4215882146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb64107120 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.408+0000 7ffb6a531700 1 -- 172.21.15.105:0/4215882146 >> 172.21.15.105:0/4215882146 conn(0x7ffb64102300 msgr2=0x7ffb64104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.408+0000 7ffb6a531700 1 -- 172.21.15.105:0/4215882146 shutdown_connections 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.408+0000 7ffb6a531700 1 -- 172.21.15.105:0/4215882146 wait complete. 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.408+0000 7ffb6a531700 1 Processor -- start 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb6a531700 1 -- start start 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb6a531700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb6419be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb6a531700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6410b2b0 con 0x7ffb64106d00 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb6419be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:53.657 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb6419be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34190/0 (socket says 172.21.15.105:34190) 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb63fff700 1 -- 172.21.15.105:0/471964377 learned_addr learned my addr 172.21.15.105:0/471964377 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb63fff700 1 -- 172.21.15.105:0/471964377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6419c370 con 0x7ffb64106d00 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb63fff700 1 --2- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb6419be30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb54006af0 tx=0x7ffb54011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb61ffb700 1 -- 172.21.15.105:0/471964377 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54003930 con 0x7ffb64106d00 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb61ffb700 1 -- 172.21.15.105:0/471964377 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ffb54003ad0 con 0x7ffb64106d00 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6419c600 con 0x7ffb64106d00 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb61ffb700 1 -- 172.21.15.105:0/471964377 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb540195e0 con 0x7ffb64106d00 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.409+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb6419f2f0 con 0x7ffb64106d00 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.410+0000 7ffb61ffb700 1 -- 172.21.15.105:0/471964377 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) ==== 46283+0+0 (secure 0 0 0) 0x7ffb54019890 con 0x7ffb64106d00 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.410+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb640639e0 con 0x7ffb64106d00 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.410+0000 7ffb61ffb700 1 --2- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ffb50039130 0x7ffb5003b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:53.658 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.410+0000 7ffb61ffb700 1 -- 172.21.15.105:0/471964377 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb5404d7a0 con 0x7ffb64106d00 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.411+0000 7ffb5bfff700 1 --2- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ffb50039130 0x7ffb5003b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.411+0000 7ffb5bfff700 1 --2- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ffb50039130 0x7ffb5003b5f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c006fd0 tx=0x7ffb4c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.415+0000 7ffb61ffb700 1 -- 172.21.15.105:0/471964377 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffb5404d1c0 con 0x7ffb64106d00 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:52.666+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ffb640008d0 con 0x7ffb64106d00 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.011+0000 7ffb61ffb700 1 -- 172.21.15.105:0/471964377 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ffb5402d960 con 0x7ffb64106d00 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.013+0000 7ffb61ffb700 1 -- 172.21.15.105:0/471964377 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7ffb5402d440 con 0x7ffb64106d00 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ffb50039130 msgr2=0x7ffb5003b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 --2- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ffb50039130 0x7ffb5003b5f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c006fd0 tx=0x7ffb4c006e40 comp rx=0 tx=0).stop 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 msgr2=0x7ffb6419be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 --2- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb6419be30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb54006af0 tx=0x7ffb54011b70 comp rx=0 tx=0).stop 2024-03-26T17:28:53.659 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 shutdown_connections 2024-03-26T17:28:53.660 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 --2- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ffb50039130 0x7ffb5003b5f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:53.660 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 --2- 172.21.15.105:0/471964377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb64106d00 0x7ffb6419be30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:53.660 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 >> 172.21.15.105:0/471964377 conn(0x7ffb64102300 msgr2=0x7ffb64103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:53.660 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 shutdown_connections 2024-03-26T17:28:53.660 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:53.015+0000 7ffb6a531700 1 -- 172.21.15.105:0/471964377 wait complete. 2024-03-26T17:28:53.856 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:53 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:53.589+0000 7fb308f83200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:28:53.856 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:53 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:53.690+0000 7fb308f83200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:28:54.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:54 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/471964377' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-26T17:28:54.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:54 smithi105 ceph-mon[32280]: mgrmap e5: x(active, since 6s) 2024-03-26T17:28:54.583 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:54.354+0000 7fb308f83200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:28:54.583 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:54.447+0000 7fb308f83200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:28:54.583 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:54.588+0000 7fb308f83200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:28:54.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:54.683+0000 7fb308f83200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:28:54.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:54.767+0000 7fb308f83200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:28:54.946 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:28:54.952 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-26T17:28:54.952 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T17:28:54.952 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.037+0000 7ff81dec0700 1 Processor -- start 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.037+0000 7ff81dec0700 1 -- start start 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.037+0000 7ff81dec0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff818106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.037+0000 7ff81dec0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff81807bde0 con 0x7ff818106700 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.037+0000 7ff8177fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff818106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.037+0000 7ff8177fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff818106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34216/0 (socket says 172.21.15.105:34216) 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.037+0000 7ff8177fe700 1 -- 172.21.15.105:0/2808936982 learned_addr learned my addr 172.21.15.105:0/2808936982 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.038+0000 7ff8177fe700 1 -- 172.21.15.105:0/2808936982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff818107060 con 0x7ff818106700 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.038+0000 7ff8177fe700 1 --2- 172.21.15.105:0/2808936982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff818106b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff8080098a0 tx=0x7ff808009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5ebb66cd4b63b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.038+0000 7ff8167fc700 1 -- 172.21.15.105:0/2808936982 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff808004160 con 0x7ff818106700 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.038+0000 7ff8167fc700 1 -- 172.21.15.105:0/2808936982 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff808004300 con 0x7ff818106700 2024-03-26T17:28:54.953 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.038+0000 7ff8167fc700 1 -- 172.21.15.105:0/2808936982 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff808011a80 con 0x7ff818106700 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.039+0000 7ff81dec0700 1 -- 172.21.15.105:0/2808936982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 msgr2=0x7ff818106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.039+0000 7ff81dec0700 1 --2- 172.21.15.105:0/2808936982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff818106b20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff8080098a0 tx=0x7ff808009bb0 comp rx=0 tx=0).stop 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.039+0000 7ff81dec0700 1 -- 172.21.15.105:0/2808936982 shutdown_connections 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.039+0000 7ff81dec0700 1 --2- 172.21.15.105:0/2808936982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff818106b20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.039+0000 7ff81dec0700 1 -- 172.21.15.105:0/2808936982 >> 172.21.15.105:0/2808936982 conn(0x7ff818102300 msgr2=0x7ff818104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.039+0000 7ff81dec0700 1 -- 172.21.15.105:0/2808936982 shutdown_connections 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.039+0000 7ff81dec0700 1 -- 172.21.15.105:0/2808936982 wait complete. 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.040+0000 7ff81dec0700 1 Processor -- start 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.040+0000 7ff81dec0700 1 -- start start 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.040+0000 7ff81dec0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff81819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.040+0000 7ff81dec0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff818108090 con 0x7ff818106700 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.040+0000 7ff8177fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff81819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:28:54.954 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.040+0000 7ff8177fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff81819df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34218/0 (socket says 172.21.15.105:34218) 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.040+0000 7ff8177fe700 1 -- 172.21.15.105:0/4242513755 learned_addr learned my addr 172.21.15.105:0/4242513755 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.041+0000 7ff8177fe700 1 -- 172.21.15.105:0/4242513755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81819e4b0 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.041+0000 7ff8177fe700 1 --2- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff81819df70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff808006af0 tx=0x7ff808011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.041+0000 7ff814ff9700 1 -- 172.21.15.105:0/4242513755 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff808003930 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.041+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81819e740 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.041+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181a1430 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.041+0000 7ff814ff9700 1 -- 172.21.15.105:0/4242513755 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff808003ad0 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.041+0000 7ff814ff9700 1 -- 172.21.15.105:0/4242513755 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8080195e0 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.042+0000 7ff814ff9700 1 -- 172.21.15.105:0/4242513755 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7ff8080198b0 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.042+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8180639e0 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.042+0000 7ff814ff9700 1 --2- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ff8000391e0 0x7ff80003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.042+0000 7ff814ff9700 1 -- 172.21.15.105:0/4242513755 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff80804e5d0 con 0x7ff818106700 2024-03-26T17:28:54.955 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.042+0000 7ff816ffd700 1 -- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ff8000391e0 msgr2=0x7ff80003b6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/3642413152 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.042+0000 7ff816ffd700 1 --2- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ff8000391e0 0x7ff80003b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.047+0000 7ff814ff9700 1 -- 172.21.15.105:0/4242513755 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff808020240 con 0x7ff818106700 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.243+0000 7ff816ffd700 1 -- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ff8000391e0 msgr2=0x7ff80003b6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/3642413152 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.243+0000 7ff816ffd700 1 --2- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ff8000391e0 0x7ff80003b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.310+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff8180008d0 con 0x7ff818106700 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.310+0000 7ff814ff9700 1 -- 172.21.15.105:0/4242513755 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff80800bbf0 con 0x7ff818106700 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.312+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ff8000391e0 msgr2=0x7ff80003b6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.312+0000 7ff81dec0700 1 --2- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ff8000391e0 0x7ff80003b6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.312+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 msgr2=0x7ff81819df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.312+0000 7ff81dec0700 1 --2- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff81819df70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff808006af0 tx=0x7ff808011b70 comp rx=0 tx=0).stop 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.313+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 shutdown_connections 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.313+0000 7ff81dec0700 1 --2- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7ff8000391e0 0x7ff80003b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.313+0000 7ff81dec0700 1 --2- 172.21.15.105:0/4242513755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff818106700 0x7ff81819df70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:28:54.956 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.313+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 >> 172.21.15.105:0/4242513755 conn(0x7ff818102300 msgr2=0x7ff818194c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:28:54.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.313+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 shutdown_connections 2024-03-26T17:28:54.957 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:54.313+0000 7ff81dec0700 1 -- 172.21.15.105:0/4242513755 wait complete. 2024-03-26T17:28:54.957 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for the mgr to restart... 2024-03-26T17:28:54.957 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mgr epoch 5... 2024-03-26T17:28:55.203 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:55 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4242513755' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T17:28:55.203 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:55.036+0000 7fb308f83200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:28:55.203 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:55.145+0000 7fb308f83200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:28:55.500 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:55.232+0000 7fb308f83200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:28:55.500 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:55.326+0000 7fb308f83200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:28:55.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:55.505+0000 7fb308f83200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:28:56.842 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:56.454+0000 7fb308f83200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:28:56.842 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:56.583+0000 7fb308f83200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:28:57.144 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:56.908+0000 7fb308f83200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:28:57.144 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:56.995+0000 7fb308f83200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:28:57.644 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:57.186+0000 7fb308f83200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:28:57.644 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:57.368+0000 7fb308f83200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:28:57.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:57.649+0000 7fb308f83200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:28:58.395 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:57.903+0000 7fb308f83200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:28:58.395 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:58 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:58.135+0000 7fb308f83200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:28:59.395 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:58 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:58.927+0000 7fb308f83200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:28:59.798 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:59.490+0000 7fb308f83200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:28:59.798 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:59.587+0000 7fb308f83200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:28:59.798 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:28:59.803+0000 7fb308f83200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:29:00.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: Active manager daemon x restarted 2024-03-26T17:29:00.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: Activating manager daemon x 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: osdmap e2: 0 total, 0 up, 0 in 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: mgrmap e6: x(active, starting, since 0.00358209s) 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: Manager daemon x is now available 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:00.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:28:59 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:01.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:00 smithi105 ceph-mon[32280]: Found migration_current of "None". Setting to last migration. 2024-03-26T17:29:01.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:00 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:01.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:00 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T17:29:01.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:00 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T17:29:01.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:00 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:01.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:00 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:01.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:00 smithi105 ceph-mon[32280]: mgrmap e7: x(active, since 1.00789s) 2024-03-26T17:29:01.406 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:29:01.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-26T17:29:01.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T17:29:01.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:29:01.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.439+0000 7f69a71b4700 1 Processor -- start 2024-03-26T17:29:01.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.439+0000 7f69a71b4700 1 -- start start 2024-03-26T17:29:01.407 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.439+0000 7f69a71b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a0106500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.439+0000 7f69a71b4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69a0106ad0 con 0x7f69a01060e0 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.440+0000 7f69a4f50700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a0106500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.440+0000 7f69a4f50700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a0106500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41214/0 (socket says 172.21.15.105:41214) 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.440+0000 7f69a4f50700 1 -- 172.21.15.105:0/1132950042 learned_addr learned my addr 172.21.15.105:0/1132950042 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.440+0000 7f69a4f50700 1 -- 172.21.15.105:0/1132950042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a0109070 con 0x7f69a01060e0 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.440+0000 7f69a4f50700 1 --2- 172.21.15.105:0/1132950042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a0106500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f69940098a0 tx=0x7f6994009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bccc028cfbb00e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.440+0000 7f699f7fe700 1 -- 172.21.15.105:0/1132950042 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6994004160 con 0x7f69a01060e0 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.440+0000 7f699f7fe700 1 -- 172.21.15.105:0/1132950042 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6994004300 con 0x7f69a01060e0 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.440+0000 7f699f7fe700 1 -- 172.21.15.105:0/1132950042 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6994011a80 con 0x7f69a01060e0 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.441+0000 7f69a71b4700 1 -- 172.21.15.105:0/1132950042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 msgr2=0x7f69a0106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:01.408 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.441+0000 7f69a71b4700 1 --2- 172.21.15.105:0/1132950042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a0106500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f69940098a0 tx=0x7f6994009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.441+0000 7f69a71b4700 1 -- 172.21.15.105:0/1132950042 shutdown_connections 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.441+0000 7f69a71b4700 1 --2- 172.21.15.105:0/1132950042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a0106500 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.441+0000 7f69a71b4700 1 -- 172.21.15.105:0/1132950042 >> 172.21.15.105:0/1132950042 conn(0x7f69a0101a50 msgr2=0x7f69a0103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.441+0000 7f69a71b4700 1 -- 172.21.15.105:0/1132950042 shutdown_connections 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.441+0000 7f69a71b4700 1 -- 172.21.15.105:0/1132950042 wait complete. 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a71b4700 1 Processor -- start 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a71b4700 1 -- start start 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a71b4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a71b4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69a010c420 con 0x7f69a01060e0 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a4f50700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:01.409 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a4f50700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41228/0 (socket says 172.21.15.105:41228) 2024-03-26T17:29:01.410 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a4f50700 1 -- 172.21.15.105:0/1585400896 learned_addr learned my addr 172.21.15.105:0/1585400896 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:01.410 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a4f50700 1 -- 172.21.15.105:0/1585400896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a019c2f0 con 0x7f69a01060e0 2024-03-26T17:29:01.410 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.442+0000 7f69a4f50700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a019bdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6994006af0 tx=0x7f6994011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:01.410 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.443+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6994003930 con 0x7f69a01060e0 2024-03-26T17:29:01.410 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.443+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a019c580 con 0x7f69a01060e0 2024-03-26T17:29:01.410 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.443+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6994003ad0 con 0x7f69a01060e0 2024-03-26T17:29:01.410 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.443+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69940195e0 con 0x7f69a01060e0 2024-03-26T17:29:01.411 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.443+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a019f270 con 0x7f69a01060e0 2024-03-26T17:29:01.411 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.444+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 46294+0+0 (secure 0 0 0) 0x7f6994019890 con 0x7f69a01060e0 2024-03-26T17:29:01.411 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.444+0000 7f699dffb700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 0x7f699003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:01.411 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.444+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 --> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f69a00639e0 con 0x7f6990039180 2024-03-26T17:29:01.411 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.444+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f699404d630 con 0x7f69a01060e0 2024-03-26T17:29:01.411 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.444+0000 7f699ffff700 1 -- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 msgr2=0x7f699003b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/3642413152 2024-03-26T17:29:01.411 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.444+0000 7f699ffff700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 0x7f699003b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:29:01.411 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.645+0000 7f699ffff700 1 -- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 msgr2=0x7f699003b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/3642413152 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:55.645+0000 7f699ffff700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 0x7f699003b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:56.046+0000 7f699ffff700 1 -- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 msgr2=0x7f699003b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/3642413152 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:56.046+0000 7f699ffff700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 0x7f699003b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:56.847+0000 7f699ffff700 1 -- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 msgr2=0x7f699003b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/3642413152 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:56.847+0000 7f699ffff700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 0x7f699003b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:58.448+0000 7f699ffff700 1 -- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 msgr2=0x7f699003b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/3642413152 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:58.448+0000 7f699ffff700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 0x7f699003b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:59.819+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mgrmap(e 6) ==== 46061+0+0 (secure 0 0 0) 0x7f699402da50 con 0x7f69a01060e0 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:59.819+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 msgr2=0x7f699003b640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:29:01.412 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:28:59.819+0000 7f699dffb700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 0x7f699003b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.822+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 7) ==== 46188+0+0 (secure 0 0 0) 0x7f699404f220 con 0x7f69a01060e0 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.823+0000 7f699dffb700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f699003c880 0x7f699003ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.823+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f69a00639e0 con 0x7f699003c880 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.823+0000 7f699ffff700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f699003c880 0x7f699003ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.824+0000 7f699ffff700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f699003c880 0x7f699003ec70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f698c003a10 tx=0x7f698c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.824+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mgr.14120 v2:172.21.15.105:6800/1556474900 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f69a00639e0 con 0x7f699003c880 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f69a0102cf0 con 0x7f699003c880 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f699dffb700 1 -- 172.21.15.105:0/1585400896 <== mgr.14120 v2:172.21.15.105:6800/1556474900 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f69a0102cf0 con 0x7f699003c880 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f699003c880 msgr2=0x7f699003ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f699003c880 0x7f699003ec70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f698c003a10 tx=0x7f698c0092b0 comp rx=0 tx=0).stop 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 msgr2=0x7f69a019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a019bdb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6994006af0 tx=0x7f6994011b70 comp rx=0 tx=0).stop 2024-03-26T17:29:01.413 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 shutdown_connections 2024-03-26T17:29:01.414 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f699003c880 0x7f699003ec70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:01.414 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69a01060e0 0x7f69a019bdb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:01.414 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 --2- 172.21.15.105:0/1585400896 >> [v2:172.21.15.105:6800/3642413152,v1:172.21.15.105:6801/3642413152] conn(0x7f6990039180 0x7f699003b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:01.414 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 >> 172.21.15.105:0/1585400896 conn(0x7f69a0101a50 msgr2=0x7f69a01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:01.414 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 shutdown_connections 2024-03-26T17:29:01.414 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:00.831+0000 7f69a71b4700 1 -- 172.21.15.105:0/1585400896 wait complete. 2024-03-26T17:29:01.414 INFO:teuthology.orchestra.run.smithi105.stdout:mgr epoch 5 is available 2024-03-26T17:29:01.414 INFO:teuthology.orchestra.run.smithi105.stdout:Setting orchestrator backend to cephadm... 2024-03-26T17:29:02.089 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:01 smithi105 ceph-mon[32280]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T17:29:02.089 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:01 smithi105 ceph-mon[32280]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T17:29:02.089 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:01 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:02.549 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.867+0000 7f24ce9e9700 1 Processor -- start 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.867+0000 7f24ce9e9700 1 -- start start 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.867+0000 7f24ce9e9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c8108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.867+0000 7f24ce9e9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8108770 con 0x7f24c8107e10 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.867+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c8108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.867+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c8108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41280/0 (socket says 172.21.15.105:41280) 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.867+0000 7f24c7fff700 1 -- 172.21.15.105:0/2569078437 learned_addr learned my addr 172.21.15.105:0/2569078437 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24c7fff700 1 -- 172.21.15.105:0/2569078437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c8108f70 con 0x7f24c8107e10 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24c7fff700 1 --2- 172.21.15.105:0/2569078437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c8108230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24b80098a0 tx=0x7f24b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a23636b2a1c22a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24c6ffd700 1 -- 172.21.15.105:0/2569078437 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24b8004160 con 0x7f24c8107e10 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24c6ffd700 1 -- 172.21.15.105:0/2569078437 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f24b8004300 con 0x7f24c8107e10 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24c6ffd700 1 -- 172.21.15.105:0/2569078437 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24b8011a80 con 0x7f24c8107e10 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24ce9e9700 1 -- 172.21.15.105:0/2569078437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 msgr2=0x7f24c8108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:02.550 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24ce9e9700 1 --2- 172.21.15.105:0/2569078437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c8108230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24b80098a0 tx=0x7f24b8009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24ce9e9700 1 -- 172.21.15.105:0/2569078437 shutdown_connections 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24ce9e9700 1 --2- 172.21.15.105:0/2569078437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c8108230 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24ce9e9700 1 -- 172.21.15.105:0/2569078437 >> 172.21.15.105:0/2569078437 conn(0x7f24c81033b0 msgr2=0x7f24c81057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24ce9e9700 1 -- 172.21.15.105:0/2569078437 shutdown_connections 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.868+0000 7f24ce9e9700 1 -- 172.21.15.105:0/2569078437 wait complete. 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.869+0000 7f24ce9e9700 1 Processor -- start 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.869+0000 7f24ce9e9700 1 -- start start 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.869+0000 7f24ce9e9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.869+0000 7f24ce9e9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24c810c3b0 con 0x7f24c8107e10 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.869+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.869+0000 7f24c7fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41296/0 (socket says 172.21.15.105:41296) 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.869+0000 7f24c7fff700 1 -- 172.21.15.105:0/1336780078 learned_addr learned my addr 172.21.15.105:0/1336780078 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.870+0000 7f24c7fff700 1 -- 172.21.15.105:0/1336780078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c819c2e0 con 0x7f24c8107e10 2024-03-26T17:29:02.551 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.870+0000 7f24c7fff700 1 --2- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c819bda0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24b8006af0 tx=0x7f24b8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.870+0000 7f24c57fa700 1 -- 172.21.15.105:0/1336780078 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24b8003930 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.870+0000 7f24c57fa700 1 -- 172.21.15.105:0/1336780078 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f24b8003ad0 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.870+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c819c570 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.870+0000 7f24c57fa700 1 -- 172.21.15.105:0/1336780078 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24b80195e0 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.870+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c819f220 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.871+0000 7f24c57fa700 1 -- 172.21.15.105:0/1336780078 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 7) ==== 46188+0+0 (secure 0 0 0) 0x7f24b8019890 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.871+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c80639e0 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.871+0000 7f24c57fa700 1 --2- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f24b0039050 0x7f24b003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.871+0000 7f24c57fa700 1 -- 172.21.15.105:0/1336780078 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f24b804d4a0 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.871+0000 7f24c77fe700 1 --2- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f24b0039050 0x7f24b003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.871+0000 7f24c77fe700 1 --2- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f24b0039050 0x7f24b003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f24bc006fd0 tx=0x7f24bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.877+0000 7f24c57fa700 1 -- 172.21.15.105:0/1336780078 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f24b8020240 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:01.878+0000 7f24c57fa700 1 -- 172.21.15.105:0/1336780078 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f24b802d960 con 0x7f24c8107e10 2024-03-26T17:29:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.086+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f24c8104580 con 0x7f24b0039050 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.094+0000 7f24c57fa700 1 -- 172.21.15.105:0/1336780078 <== mgr.14120 v2:172.21.15.105:6800/1556474900 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f24c8104580 con 0x7f24b0039050 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f24b0039050 msgr2=0x7f24b003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 --2- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f24b0039050 0x7f24b003b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f24bc006fd0 tx=0x7f24bc006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 msgr2=0x7f24c819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 --2- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c819bda0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f24b8006af0 tx=0x7f24b8011b70 comp rx=0 tx=0).stop 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 shutdown_connections 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 --2- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f24b0039050 0x7f24b003b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 --2- 172.21.15.105:0/1336780078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24c8107e10 0x7f24c819bda0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 >> 172.21.15.105:0/1336780078 conn(0x7f24c81033b0 msgr2=0x7f24c8103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 shutdown_connections 2024-03-26T17:29:02.553 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.096+0000 7f24ce9e9700 1 -- 172.21.15.105:0/1336780078 wait complete. 2024-03-26T17:29:02.951 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:02 smithi105 ceph-mon[32280]: [26/Mar/2024:17:29:01] ENGINE Bus STARTING 2024-03-26T17:29:02.952 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:02 smithi105 ceph-mon[32280]: [26/Mar/2024:17:29:01] ENGINE Serving on http://172.21.15.105:8765 2024-03-26T17:29:02.952 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:02 smithi105 ceph-mon[32280]: [26/Mar/2024:17:29:01] ENGINE Serving on https://172.21.15.105:7150 2024-03-26T17:29:02.952 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:02 smithi105 ceph-mon[32280]: [26/Mar/2024:17:29:01] ENGINE Bus STARTED 2024-03-26T17:29:02.952 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:02 smithi105 ceph-mon[32280]: mgrmap e8: x(active, since 2s) 2024-03-26T17:29:02.952 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:02 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:02.952 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:02 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:03.782 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-26T17:29:03.782 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.964+0000 7fd0433cb700 1 Processor -- start 2024-03-26T17:29:03.782 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd0433cb700 1 -- start start 2024-03-26T17:29:03.782 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd0433cb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c10ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd0433cb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c07bda0 con 0x7fd03c108960 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd041167700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c10ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd041167700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c10ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41300/0 (socket says 172.21.15.105:41300) 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd041167700 1 -- 172.21.15.105:0/3101775725 learned_addr learned my addr 172.21.15.105:0/3101775725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd041167700 1 -- 172.21.15.105:0/3101775725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c10b290 con 0x7fd03c108960 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd041167700 1 --2- 172.21.15.105:0/3101775725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c10ad50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0380098a0 tx=0x7fd038009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafb14ddc8451348 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd033fff700 1 -- 172.21.15.105:0/3101775725 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd038004160 con 0x7fd03c108960 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd033fff700 1 -- 172.21.15.105:0/3101775725 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd038004300 con 0x7fd03c108960 2024-03-26T17:29:03.783 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.965+0000 7fd033fff700 1 -- 172.21.15.105:0/3101775725 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd038011a80 con 0x7fd03c108960 2024-03-26T17:29:03.784 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.966+0000 7fd0433cb700 1 -- 172.21.15.105:0/3101775725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 msgr2=0x7fd03c10ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:03.784 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.966+0000 7fd0433cb700 1 --2- 172.21.15.105:0/3101775725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c10ad50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0380098a0 tx=0x7fd038009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:03.784 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.966+0000 7fd0433cb700 1 -- 172.21.15.105:0/3101775725 shutdown_connections 2024-03-26T17:29:03.784 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.966+0000 7fd0433cb700 1 --2- 172.21.15.105:0/3101775725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c10ad50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:03.784 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.966+0000 7fd0433cb700 1 -- 172.21.15.105:0/3101775725 >> 172.21.15.105:0/3101775725 conn(0x7fd03c1022d0 msgr2=0x7fd03c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:03.784 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.966+0000 7fd0433cb700 1 -- 172.21.15.105:0/3101775725 shutdown_connections 2024-03-26T17:29:03.784 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.966+0000 7fd0433cb700 1 -- 172.21.15.105:0/3101775725 wait complete. 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd0433cb700 1 Processor -- start 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd0433cb700 1 -- start start 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd0433cb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd0433cb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c10c190 con 0x7fd03c108960 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd041167700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd041167700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41306/0 (socket says 172.21.15.105:41306) 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd041167700 1 -- 172.21.15.105:0/3717891087 learned_addr learned my addr 172.21.15.105:0/3717891087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd041167700 1 -- 172.21.15.105:0/3717891087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c19c300 con 0x7fd03c108960 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.967+0000 7fd041167700 1 --2- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c19bdc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd038011a50 tx=0x7fd038004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:03.785 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.968+0000 7fd0327fc700 1 -- 172.21.15.105:0/3717891087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0380039f0 con 0x7fd03c108960 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.968+0000 7fd0327fc700 1 -- 172.21.15.105:0/3717891087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd038003b90 con 0x7fd03c108960 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.968+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c19c590 con 0x7fd03c108960 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.968+0000 7fd0327fc700 1 -- 172.21.15.105:0/3717891087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd03801a600 con 0x7fd03c108960 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.968+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c19f280 con 0x7fd03c108960 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.968+0000 7fd0327fc700 1 -- 172.21.15.105:0/3717891087 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fd03801a8b0 con 0x7fd03c108960 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.969+0000 7fd0327fc700 1 --2- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fd0280391d0 0x7fd02803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.969+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03c0639e0 con 0x7fd03c108960 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.969+0000 7fd0327fc700 1 -- 172.21.15.105:0/3717891087 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd03804d700 con 0x7fd03c108960 2024-03-26T17:29:03.786 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.969+0000 7fd040966700 1 --2- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fd0280391d0 0x7fd02803b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.969+0000 7fd040966700 1 --2- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fd0280391d0 0x7fd02803b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd02c006fd0 tx=0x7fd02c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:02.974+0000 7fd0327fc700 1 -- 172.21.15.105:0/3717891087 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd03800fe60 con 0x7fd03c108960 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.172+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd03c103570 con 0x7fd0280391d0 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.172+0000 7fd0327fc700 1 -- 172.21.15.105:0/3717891087 <== mgr.14120 v2:172.21.15.105:6800/1556474900 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd03c103570 con 0x7fd0280391d0 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.174+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fd0280391d0 msgr2=0x7fd02803b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.174+0000 7fd0433cb700 1 --2- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fd0280391d0 0x7fd02803b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd02c006fd0 tx=0x7fd02c006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.174+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 msgr2=0x7fd03c19bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.174+0000 7fd0433cb700 1 --2- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c19bdc0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd038011a50 tx=0x7fd038004290 comp rx=0 tx=0).stop 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.174+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 shutdown_connections 2024-03-26T17:29:03.787 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.174+0000 7fd0433cb700 1 --2- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fd0280391d0 0x7fd02803b690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:03.788 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.174+0000 7fd0433cb700 1 --2- 172.21.15.105:0/3717891087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd03c108960 0x7fd03c19bdc0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:03.788 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.174+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 >> 172.21.15.105:0/3717891087 conn(0x7fd03c1022d0 msgr2=0x7fd03c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:03.788 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.175+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 shutdown_connections 2024-03-26T17:29:03.788 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:03.175+0000 7fd0433cb700 1 -- 172.21.15.105:0/3717891087 wait complete. 2024-03-26T17:29:03.788 INFO:teuthology.orchestra.run.smithi105.stdout:Generating ssh key... 2024-03-26T17:29:04.054 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:03 smithi105 ceph-mon[32280]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:04.645 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: Generating public/private rsa key pair. 2024-03-26T17:29:04.645 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: Your identification has been saved in /tmp/tmpajfsb5xc/key. 2024-03-26T17:29:04.645 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: Your public key has been saved in /tmp/tmpajfsb5xc/key.pub. 2024-03-26T17:29:04.645 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: The key fingerprint is: 2024-03-26T17:29:04.645 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: SHA256:K1ceaLyxdf3LxU7+pHkD/tI+KRKU5apkrr3rmmpbjIo ceph-15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:29:04.645 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: The key's randomart image is: 2024-03-26T17:29:04.645 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: +---[RSA 3072]----+ 2024-03-26T17:29:04.645 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | | 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | . | 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | + | 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | . . o o | 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | S = o . | 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | o. @ = . o | 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | ..oO o o o B| 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | . ...ooo . +.%+| 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: | E ..oo++=o . B*B| 2024-03-26T17:29:04.646 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: +----[SHA256]-----+ 2024-03-26T17:29:05.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-mon[32280]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:05.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:05.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:04 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:05.154 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f473248d700 1 Processor -- start 2024-03-26T17:29:05.154 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f473248d700 1 -- start start 2024-03-26T17:29:05.154 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f473248d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:05.154 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f473248d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f472c108770 con 0x7f472c107e10 2024-03-26T17:29:05.154 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f472bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:05.154 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f472bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41308/0 (socket says 172.21.15.105:41308) 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f472bfff700 1 -- 172.21.15.105:0/2149660960 learned_addr learned my addr 172.21.15.105:0/2149660960 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f472bfff700 1 -- 172.21.15.105:0/2149660960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c108f70 con 0x7f472c107e10 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f472bfff700 1 --2- 172.21.15.105:0/2149660960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c108230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f47200098a0 tx=0x7f4720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6137c7b3a73fa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f472affd700 1 -- 172.21.15.105:0/2149660960 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4720004160 con 0x7f472c107e10 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f472affd700 1 -- 172.21.15.105:0/2149660960 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4720004300 con 0x7f472c107e10 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.247+0000 7f472affd700 1 -- 172.21.15.105:0/2149660960 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4720011a80 con 0x7f472c107e10 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.248+0000 7f473248d700 1 -- 172.21.15.105:0/2149660960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 msgr2=0x7f472c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.248+0000 7f473248d700 1 --2- 172.21.15.105:0/2149660960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c108230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f47200098a0 tx=0x7f4720009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.248+0000 7f473248d700 1 -- 172.21.15.105:0/2149660960 shutdown_connections 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.248+0000 7f473248d700 1 --2- 172.21.15.105:0/2149660960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c108230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.248+0000 7f473248d700 1 -- 172.21.15.105:0/2149660960 >> 172.21.15.105:0/2149660960 conn(0x7f472c1033b0 msgr2=0x7f472c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.248+0000 7f473248d700 1 -- 172.21.15.105:0/2149660960 shutdown_connections 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.248+0000 7f473248d700 1 -- 172.21.15.105:0/2149660960 wait complete. 2024-03-26T17:29:05.155 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f473248d700 1 Processor -- start 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f473248d700 1 -- start start 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f473248d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f473248d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10c3b0 con 0x7f472c107e10 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f472bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f472bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41316/0 (socket says 172.21.15.105:41316) 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f472bfff700 1 -- 172.21.15.105:0/4202205889 learned_addr learned my addr 172.21.15.105:0/4202205889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f472bfff700 1 -- 172.21.15.105:0/4202205889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c19c2e0 con 0x7f472c107e10 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.249+0000 7f472bfff700 1 --2- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c19bda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4720006af0 tx=0x7f4720011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.250+0000 7f47297fa700 1 -- 172.21.15.105:0/4202205889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4720003930 con 0x7f472c107e10 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.250+0000 7f47297fa700 1 -- 172.21.15.105:0/4202205889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4720003ad0 con 0x7f472c107e10 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.250+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c19c570 con 0x7f472c107e10 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.250+0000 7f47297fa700 1 -- 172.21.15.105:0/4202205889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47200195e0 con 0x7f472c107e10 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.250+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c19f220 con 0x7f472c107e10 2024-03-26T17:29:05.156 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.250+0000 7f47297fa700 1 -- 172.21.15.105:0/4202205889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f47200198e0 con 0x7f472c107e10 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.251+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f472c0639e0 con 0x7f472c107e10 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.251+0000 7f47297fa700 1 --2- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4714039130 0x7f471403b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.251+0000 7f47297fa700 1 -- 172.21.15.105:0/4202205889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f472004d840 con 0x7f472c107e10 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.251+0000 7f472b7fe700 1 --2- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4714039130 0x7f471403b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.251+0000 7f472b7fe700 1 --2- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4714039130 0x7f471403b5f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f471c006fd0 tx=0x7f471c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.256+0000 7f47297fa700 1 -- 172.21.15.105:0/4202205889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f472002d440 con 0x7f472c107e10 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.476+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f472c104580 con 0x7f4714039130 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.577+0000 7f47297fa700 1 -- 172.21.15.105:0/4202205889 <== mgr.14120 v2:172.21.15.105:6800/1556474900 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f472c104580 con 0x7f4714039130 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4714039130 msgr2=0x7f471403b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 --2- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4714039130 0x7f471403b5f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f471c006fd0 tx=0x7f471c006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 msgr2=0x7f472c19bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 --2- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c19bda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4720006af0 tx=0x7f4720011b70 comp rx=0 tx=0).stop 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 shutdown_connections 2024-03-26T17:29:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 --2- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4714039130 0x7f471403b5f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:05.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 --2- 172.21.15.105:0/4202205889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c107e10 0x7f472c19bda0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:05.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 >> 172.21.15.105:0/4202205889 conn(0x7f472c1033b0 msgr2=0x7f472c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:05.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 shutdown_connections 2024-03-26T17:29:05.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:04.579+0000 7f473248d700 1 -- 172.21.15.105:0/4202205889 wait complete. 2024-03-26T17:29:06.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:05 smithi105 ceph-mon[32280]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:06.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:05 smithi105 ceph-mon[32280]: Generating ssh key... 2024-03-26T17:29:06.277 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCySkDhUiJMPvu2HWbpTbp7bjb1AFhF9lp1gjmX/7L4N2MUguyUOyr5EAm6i12VsM0WOjuTegG9D0muPbAF72dkjlyNBmGDJMD9+Rq4mYogl5JcuPQzBHtJNHufB2w0afmjNAogLGbt2Q4CvPaVpzG4aeSteQLBjRTtnWQOdS5Y1ePQRVIdBUpJaG+HZO0yoKtIHYp4aH/C9cFDAValjimJxoaE32jK0FyWDFigUoRgUbXuT/9+O/sAx+pJELZySijlzACBpNaewu1moYsbg/nylM0+FYv4xCvnzCqQoHnwZOhWZh7qQ6yvOtVrSOyBj/SGwO6zAAbo4LDKr3ryX2qrmTIo52C8Uxt2x/q1dvQ4ala3re8Q4aSn8+5fnpPVbGLZVCxKNc6v26m7udE+B6sgqkS8YeNMCwtVdHFmMoZfu0Aobw6pc19p5MZKujEnC7PwVO2Oz7Sx2BQ2dn3ioPwqtobldV9Z857bLISPHSE6kwhQ5kC35JMN1cBryvUgR88= ceph-15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:29:06.277 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.533+0000 7f7e5984e700 1 Processor -- start 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.533+0000 7f7e5984e700 1 -- start start 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.533+0000 7f7e5984e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e5410acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.533+0000 7f7e5984e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5407be20 con 0x7f7e541088e0 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.533+0000 7f7e52ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e5410acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.533+0000 7f7e52ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e5410acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52912/0 (socket says 172.21.15.105:52912) 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.533+0000 7f7e52ffd700 1 -- 172.21.15.105:0/4135033194 learned_addr learned my addr 172.21.15.105:0/4135033194 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.534+0000 7f7e52ffd700 1 -- 172.21.15.105:0/4135033194 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5410b210 con 0x7f7e541088e0 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.534+0000 7f7e52ffd700 1 --2- 172.21.15.105:0/4135033194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e5410acd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098a0 tx=0x7f7e44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a8717f76c65854 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.534+0000 7f7e51ffb700 1 -- 172.21.15.105:0/4135033194 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e44004160 con 0x7f7e541088e0 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.534+0000 7f7e51ffb700 1 -- 172.21.15.105:0/4135033194 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7e44004300 con 0x7f7e541088e0 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.534+0000 7f7e51ffb700 1 -- 172.21.15.105:0/4135033194 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e44011a80 con 0x7f7e541088e0 2024-03-26T17:29:06.278 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 -- 172.21.15.105:0/4135033194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 msgr2=0x7f7e5410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 --2- 172.21.15.105:0/4135033194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e5410acd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e440098a0 tx=0x7f7e44009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 -- 172.21.15.105:0/4135033194 shutdown_connections 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 --2- 172.21.15.105:0/4135033194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e5410acd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 -- 172.21.15.105:0/4135033194 >> 172.21.15.105:0/4135033194 conn(0x7f7e541022d0 msgr2=0x7f7e541046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 -- 172.21.15.105:0/4135033194 shutdown_connections 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 -- 172.21.15.105:0/4135033194 wait complete. 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 Processor -- start 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 -- start start 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e54108160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.535+0000 7f7e5984e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410c180 con 0x7f7e541088e0 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e52ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e54108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e52ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e54108160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52924/0 (socket says 172.21.15.105:52924) 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e52ffd700 1 -- 172.21.15.105:0/1610273824 learned_addr learned my addr 172.21.15.105:0/1610273824 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:06.279 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e52ffd700 1 -- 172.21.15.105:0/1610273824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e541086a0 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e52ffd700 1 --2- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e54108160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7e44011a50 tx=0x7f7e44004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e5884c700 1 -- 172.21.15.105:0/1610273824 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e440039f0 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e5884c700 1 -- 172.21.15.105:0/1610273824 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7e44003b90 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e5884c700 1 -- 172.21.15.105:0/1610273824 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e4401a600 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e54106890 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.536+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e54106dd0 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.537+0000 7f7e5884c700 1 -- 172.21.15.105:0/1610273824 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f7e4401a8b0 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.537+0000 7f7e5884c700 1 --2- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f7e3c0391d0 0x7f7e3c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.537+0000 7f7e5884c700 1 -- 172.21.15.105:0/1610273824 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7e4404d570 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.537+0000 7f7e527fc700 1 --2- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f7e3c0391d0 0x7f7e3c03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.537+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e540639e0 con 0x7f7e541088e0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.538+0000 7f7e527fc700 1 --2- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f7e3c0391d0 0x7f7e3c03b690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7e48006fd0 tx=0x7f7e48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:06.280 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.542+0000 7f7e5884c700 1 -- 172.21.15.105:0/1610273824 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7e4402a2a0 con 0x7f7e541088e0 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.736+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7e54107700 con 0x7f7e3c0391d0 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.737+0000 7f7e5884c700 1 -- 172.21.15.105:0/1610273824 <== mgr.14120 v2:172.21.15.105:6800/1556474900 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7e54107700 con 0x7f7e3c0391d0 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f7e3c0391d0 msgr2=0x7f7e3c03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 --2- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f7e3c0391d0 0x7f7e3c03b690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7e48006fd0 tx=0x7f7e48006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 msgr2=0x7f7e54108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 --2- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e54108160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7e44011a50 tx=0x7f7e44004290 comp rx=0 tx=0).stop 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 shutdown_connections 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 --2- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f7e3c0391d0 0x7f7e3c03b690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 --2- 172.21.15.105:0/1610273824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e541088e0 0x7f7e54108160 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 >> 172.21.15.105:0/1610273824 conn(0x7f7e541022d0 msgr2=0x7f7e54102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 shutdown_connections 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:05.739+0000 7f7e5984e700 1 -- 172.21.15.105:0/1610273824 wait complete. 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:Adding key to root@localhost authorized_keys... 2024-03-26T17:29:06.281 INFO:teuthology.orchestra.run.smithi105.stdout:Adding host smithi105... 2024-03-26T17:29:07.128 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:06 smithi105 ceph-mon[32280]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:08.049 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:07 smithi105 ceph-mon[32280]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi105", "addr": "172.21.15.105", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:09.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:08 smithi105 ceph-mon[32280]: Deploying cephadm binary to smithi105 2024-03-26T17:29:11.685 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout Added host 'smithi105' with addr '172.21.15.105' 2024-03-26T17:29:11.685 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.676+0000 7f5998767700 1 Processor -- start 2024-03-26T17:29:11.685 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.676+0000 7f5998767700 1 -- start start 2024-03-26T17:29:11.685 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.676+0000 7f5998767700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f5990106510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:11.685 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.676+0000 7f5998767700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5990106ae0 con 0x7f59901060f0 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.676+0000 7f5996503700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f5990106510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.676+0000 7f5996503700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f5990106510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52932/0 (socket says 172.21.15.105:52932) 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.676+0000 7f5996503700 1 -- 172.21.15.105:0/847629336 learned_addr learned my addr 172.21.15.105:0/847629336 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.677+0000 7f5996503700 1 -- 172.21.15.105:0/847629336 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5990109080 con 0x7f59901060f0 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.677+0000 7f5996503700 1 --2- 172.21.15.105:0/847629336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f5990106510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f598c0098a0 tx=0x7f598c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a844e9ade7a73d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.677+0000 7f5995501700 1 -- 172.21.15.105:0/847629336 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f598c004160 con 0x7f59901060f0 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.677+0000 7f5995501700 1 -- 172.21.15.105:0/847629336 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f598c004300 con 0x7f59901060f0 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.677+0000 7f5995501700 1 -- 172.21.15.105:0/847629336 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f598c011a80 con 0x7f59901060f0 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.677+0000 7f5998767700 1 -- 172.21.15.105:0/847629336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 msgr2=0x7f5990106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.677+0000 7f5998767700 1 --2- 172.21.15.105:0/847629336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f5990106510 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f598c0098a0 tx=0x7f598c009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 -- 172.21.15.105:0/847629336 shutdown_connections 2024-03-26T17:29:11.686 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 --2- 172.21.15.105:0/847629336 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f5990106510 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 -- 172.21.15.105:0/847629336 >> 172.21.15.105:0/847629336 conn(0x7f5990101a40 msgr2=0x7f5990103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 -- 172.21.15.105:0/847629336 shutdown_connections 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 -- 172.21.15.105:0/847629336 wait complete. 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 Processor -- start 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 -- start start 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f599019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5998767700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f599010c430 con 0x7f59901060f0 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.678+0000 7f5996503700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f599019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f5996503700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f599019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52946/0 (socket says 172.21.15.105:52946) 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f5996503700 1 -- 172.21.15.105:0/517553919 learned_addr learned my addr 172.21.15.105:0/517553919 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f5996503700 1 -- 172.21.15.105:0/517553919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599019c2e0 con 0x7f59901060f0 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f5996503700 1 --2- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f599019bda0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f598c000c00 tx=0x7f598c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f59877fe700 1 -- 172.21.15.105:0/517553919 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f598c0039c0 con 0x7f59901060f0 2024-03-26T17:29:11.687 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f59877fe700 1 -- 172.21.15.105:0/517553919 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f598c003b60 con 0x7f59901060f0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f59877fe700 1 -- 172.21.15.105:0/517553919 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f598c01a6f0 con 0x7f59901060f0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599019c570 con 0x7f59901060f0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.679+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599019f220 con 0x7f59901060f0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.680+0000 7f59877fe700 1 -- 172.21.15.105:0/517553919 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f598c01a890 con 0x7f59901060f0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.680+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59900639e0 con 0x7f59901060f0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.680+0000 7f59877fe700 1 --2- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f597c0391d0 0x7f597c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.680+0000 7f59877fe700 1 -- 172.21.15.105:0/517553919 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f598c04d590 con 0x7f59901060f0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.680+0000 7f5995d02700 1 --2- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f597c0391d0 0x7f597c03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.680+0000 7f5995d02700 1 --2- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f597c0391d0 0x7f597c03b690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5980006fd0 tx=0x7f5980006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.685+0000 7f59877fe700 1 -- 172.21.15.105:0/517553919 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f598c02d910 con 0x7f59901060f0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:06.881+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi105", "addr": "172.21.15.105", "target": ["mon-mgr", ""]}) -- 0x7f5990102c70 con 0x7f597c0391d0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.131+0000 7f59877fe700 1 -- 172.21.15.105:0/517553919 <== mgr.14120 v2:172.21.15.105:6800/1556474900 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5990102c70 con 0x7f597c0391d0 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f597c0391d0 msgr2=0x7f597c03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:11.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 --2- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f597c0391d0 0x7f597c03b690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5980006fd0 tx=0x7f5980006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 msgr2=0x7f599019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 --2- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f599019bda0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f598c000c00 tx=0x7f598c004290 comp rx=0 tx=0).stop 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 shutdown_connections 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 --2- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f597c0391d0 0x7f597c03b690 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 --2- 172.21.15.105:0/517553919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59901060f0 0x7f599019bda0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 >> 172.21.15.105:0/517553919 conn(0x7f5990101a40 msgr2=0x7f5990102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 shutdown_connections 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:11.134+0000 7f5998767700 1 -- 172.21.15.105:0/517553919 wait complete. 2024-03-26T17:29:11.689 INFO:teuthology.orchestra.run.smithi105.stdout:Deploying unmanaged mon service... 2024-03-26T17:29:12.232 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:12 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:12.233 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:12 smithi105 ceph-mon[32280]: Added host smithi105 2024-03-26T17:29:12.233 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:12 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:12.971 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-26T17:29:12.971 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa12168b700 1 Processor -- start 2024-03-26T17:29:12.971 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa12168b700 1 -- start start 2024-03-26T17:29:12.971 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa12168b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:12.971 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa12168b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c108770 con 0x7fa11c107e10 2024-03-26T17:29:12.971 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa11affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:12.971 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa11affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52956/0 (socket says 172.21.15.105:52956) 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa11affd700 1 -- 172.21.15.105:0/1162765306 learned_addr learned my addr 172.21.15.105:0/1162765306 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa11affd700 1 -- 172.21.15.105:0/1162765306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c108f70 con 0x7fa11c107e10 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.196+0000 7fa11affd700 1 --2- 172.21.15.105:0/1162765306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c108230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0098a0 tx=0x7fa10c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a3bd01e30cfc53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa119ffb700 1 -- 172.21.15.105:0/1162765306 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa10c004160 con 0x7fa11c107e10 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa119ffb700 1 -- 172.21.15.105:0/1162765306 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa10c004300 con 0x7fa11c107e10 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa119ffb700 1 -- 172.21.15.105:0/1162765306 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa10c011a80 con 0x7fa11c107e10 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa12168b700 1 -- 172.21.15.105:0/1162765306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 msgr2=0x7fa11c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa12168b700 1 --2- 172.21.15.105:0/1162765306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c108230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0098a0 tx=0x7fa10c009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:12.972 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa12168b700 1 -- 172.21.15.105:0/1162765306 shutdown_connections 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa12168b700 1 --2- 172.21.15.105:0/1162765306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c108230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa12168b700 1 -- 172.21.15.105:0/1162765306 >> 172.21.15.105:0/1162765306 conn(0x7fa11c1033b0 msgr2=0x7fa11c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa12168b700 1 -- 172.21.15.105:0/1162765306 shutdown_connections 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.197+0000 7fa12168b700 1 -- 172.21.15.105:0/1162765306 wait complete. 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa12168b700 1 Processor -- start 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa12168b700 1 -- start start 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa12168b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa12168b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c10c3b0 con 0x7fa11c107e10 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa11affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa11affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52968/0 (socket says 172.21.15.105:52968) 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa11affd700 1 -- 172.21.15.105:0/2504236010 learned_addr learned my addr 172.21.15.105:0/2504236010 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa11affd700 1 -- 172.21.15.105:0/2504236010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c19c2e0 con 0x7fa11c107e10 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.198+0000 7fa11affd700 1 --2- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c19bda0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006af0 tx=0x7fa10c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:12.973 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.199+0000 7fa103fff700 1 -- 172.21.15.105:0/2504236010 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa10c003930 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.199+0000 7fa103fff700 1 -- 172.21.15.105:0/2504236010 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa10c003ad0 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.199+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c19c570 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.199+0000 7fa103fff700 1 -- 172.21.15.105:0/2504236010 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa10c0195e0 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.199+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c19f220 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.199+0000 7fa103fff700 1 -- 172.21.15.105:0/2504236010 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fa10c0198e0 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.200+0000 7fa103fff700 1 --2- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fa104039130 0x7fa10403b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.200+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11c0639e0 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.200+0000 7fa103fff700 1 -- 172.21.15.105:0/2504236010 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa10c04d690 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.200+0000 7fa11a7fc700 1 --2- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fa104039130 0x7fa10403b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.200+0000 7fa11a7fc700 1 --2- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fa104039130 0x7fa10403b5f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa110006fd0 tx=0x7fa110006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.205+0000 7fa103fff700 1 -- 172.21.15.105:0/2504236010 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa10c0172b0 con 0x7fa11c107e10 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.419+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa11c104580 con 0x7fa104039130 2024-03-26T17:29:12.974 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.426+0000 7fa103fff700 1 -- 172.21.15.105:0/2504236010 <== mgr.14120 v2:172.21.15.105:6800/1556474900 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa11c104580 con 0x7fa104039130 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.428+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fa104039130 msgr2=0x7fa10403b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.428+0000 7fa12168b700 1 --2- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fa104039130 0x7fa10403b5f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa110006fd0 tx=0x7fa110006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.428+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 msgr2=0x7fa11c19bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.428+0000 7fa12168b700 1 --2- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c19bda0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006af0 tx=0x7fa10c011b70 comp rx=0 tx=0).stop 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.429+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 shutdown_connections 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.429+0000 7fa12168b700 1 --2- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fa104039130 0x7fa10403b5f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.429+0000 7fa12168b700 1 --2- 172.21.15.105:0/2504236010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa11c107e10 0x7fa11c19bda0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.429+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 >> 172.21.15.105:0/2504236010 conn(0x7fa11c1033b0 msgr2=0x7fa11c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.429+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 shutdown_connections 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:12.429+0000 7fa12168b700 1 -- 172.21.15.105:0/2504236010 wait complete. 2024-03-26T17:29:12.975 INFO:teuthology.orchestra.run.smithi105.stdout:Deploying unmanaged mgr service... 2024-03-26T17:29:13.627 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:13 smithi105 ceph-mon[32280]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:13.627 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:13 smithi105 ceph-mon[32280]: Saving service mon spec with placement count:5 2024-03-26T17:29:13.627 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:13 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:15.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:14 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:15.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:14 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:15.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:14 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:15.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-26T17:29:15.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.352+0000 7f4c51275700 1 Processor -- start 2024-03-26T17:29:15.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.352+0000 7f4c51275700 1 -- start start 2024-03-26T17:29:15.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c51275700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:15.158 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c51275700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c07c8c0 con 0x7f4c4c07de90 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c4affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c4affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52970/0 (socket says 172.21.15.105:52970) 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c4affd700 1 -- 172.21.15.105:0/1417427313 learned_addr learned my addr 172.21.15.105:0/1417427313 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c4affd700 1 -- 172.21.15.105:0/1417427313 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c07ca40 con 0x7f4c4c07de90 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c4affd700 1 --2- 172.21.15.105:0/1417427313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c07c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0098a0 tx=0x7f4c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cef7a8f08f25252 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c49ffb700 1 -- 172.21.15.105:0/1417427313 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3c004160 con 0x7f4c4c07de90 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c49ffb700 1 -- 172.21.15.105:0/1417427313 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c3c004300 con 0x7f4c4c07de90 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.353+0000 7f4c49ffb700 1 -- 172.21.15.105:0/1417427313 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3c011a50 con 0x7f4c4c07de90 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.354+0000 7f4c51275700 1 -- 172.21.15.105:0/1417427313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 msgr2=0x7f4c4c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.354+0000 7f4c51275700 1 --2- 172.21.15.105:0/1417427313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c07c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0098a0 tx=0x7f4c3c009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.354+0000 7f4c51275700 1 -- 172.21.15.105:0/1417427313 shutdown_connections 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.354+0000 7f4c51275700 1 --2- 172.21.15.105:0/1417427313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c07c2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.354+0000 7f4c51275700 1 -- 172.21.15.105:0/1417427313 >> 172.21.15.105:0/1417427313 conn(0x7f4c4c1033b0 msgr2=0x7f4c4c1057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:15.159 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.354+0000 7f4c51275700 1 -- 172.21.15.105:0/1417427313 shutdown_connections 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.354+0000 7f4c51275700 1 -- 172.21.15.105:0/1417427313 wait complete. 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.355+0000 7f4c51275700 1 Processor -- start 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.355+0000 7f4c51275700 1 -- start start 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.355+0000 7f4c51275700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c19df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.355+0000 7f4c51275700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c10e3f0 con 0x7f4c4c07de90 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.355+0000 7f4c4affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c19df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.355+0000 7f4c4affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c19df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52986/0 (socket says 172.21.15.105:52986) 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.355+0000 7f4c4affd700 1 -- 172.21.15.105:0/2536741657 learned_addr learned my addr 172.21.15.105:0/2536741657 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.356+0000 7f4c4affd700 1 -- 172.21.15.105:0/2536741657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c19e480 con 0x7f4c4c07de90 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.356+0000 7f4c4affd700 1 --2- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c19df40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c011a20 tx=0x7f4c3c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.356+0000 7f4c33fff700 1 -- 172.21.15.105:0/2536741657 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3c003af0 con 0x7f4c4c07de90 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.356+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c19e710 con 0x7f4c4c07de90 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.356+0000 7f4c33fff700 1 -- 172.21.15.105:0/2536741657 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c3c01a8e0 con 0x7f4c4c07de90 2024-03-26T17:29:15.160 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.356+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c1a1400 con 0x7f4c4c07de90 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.356+0000 7f4c33fff700 1 -- 172.21.15.105:0/2536741657 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3c024860 con 0x7f4c4c07de90 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.357+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c0639e0 con 0x7f4c4c07de90 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.357+0000 7f4c33fff700 1 -- 172.21.15.105:0/2536741657 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f4c3c022020 con 0x7f4c4c07de90 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.357+0000 7f4c33fff700 1 --2- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4c34039180 0x7f4c3403b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.357+0000 7f4c33fff700 1 -- 172.21.15.105:0/2536741657 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4c3c04d470 con 0x7f4c4c07de90 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.357+0000 7f4c4a7fc700 1 --2- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4c34039180 0x7f4c3403b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.358+0000 7f4c4a7fc700 1 --2- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4c34039180 0x7f4c3403b640 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c40006fd0 tx=0x7f4c40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.362+0000 7f4c33fff700 1 -- 172.21.15.105:0/2536741657 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4c3c00b3e0 con 0x7f4c4c07de90 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.590+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4c4c104570 con 0x7f4c34039180 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.598+0000 7f4c33fff700 1 -- 172.21.15.105:0/2536741657 <== mgr.14120 v2:172.21.15.105:6800/1556474900 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4c4c104570 con 0x7f4c34039180 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4c34039180 msgr2=0x7f4c3403b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 --2- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4c34039180 0x7f4c3403b640 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c40006fd0 tx=0x7f4c40006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 msgr2=0x7f4c4c19df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:15.161 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 --2- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c19df40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c011a20 tx=0x7f4c3c011b00 comp rx=0 tx=0).stop 2024-03-26T17:29:15.162 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 shutdown_connections 2024-03-26T17:29:15.162 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 --2- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f4c34039180 0x7f4c3403b640 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:15.162 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 --2- 172.21.15.105:0/2536741657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c4c07de90 0x7f4c4c19df40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:15.162 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 >> 172.21.15.105:0/2536741657 conn(0x7f4c4c1033b0 msgr2=0x7f4c4c103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:15.162 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 shutdown_connections 2024-03-26T17:29:15.162 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:14.600+0000 7f4c51275700 1 -- 172.21.15.105:0/2536741657 wait complete. 2024-03-26T17:29:16.069 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:15 smithi105 ceph-mon[32280]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:16.069 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:15 smithi105 ceph-mon[32280]: Saving service mgr spec with placement count:2 2024-03-26T17:29:16.069 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:15 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/657109096' entity='client.admin' 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.566+0000 7f91abdf3700 1 Processor -- start 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.566+0000 7f91abdf3700 1 -- start start 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91abdf3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a4108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91abdf3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4108770 con 0x7f91a4107e10 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91a9b8f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a4108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91a9b8f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a4108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36330/0 (socket says 172.21.15.105:36330) 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91a9b8f700 1 -- 172.21.15.105:0/411025454 learned_addr learned my addr 172.21.15.105:0/411025454 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91a9b8f700 1 -- 172.21.15.105:0/411025454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a4108f70 con 0x7f91a4107e10 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91a9b8f700 1 --2- 172.21.15.105:0/411025454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a4108230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91940098a0 tx=0x7f9194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a87a06a724d37ed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91a8b8d700 1 -- 172.21.15.105:0/411025454 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9194004160 con 0x7f91a4107e10 2024-03-26T17:29:16.860 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91a8b8d700 1 -- 172.21.15.105:0/411025454 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9194004300 con 0x7f91a4107e10 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.567+0000 7f91a8b8d700 1 -- 172.21.15.105:0/411025454 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9194011a50 con 0x7f91a4107e10 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.568+0000 7f91abdf3700 1 -- 172.21.15.105:0/411025454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 msgr2=0x7f91a4108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.568+0000 7f91abdf3700 1 --2- 172.21.15.105:0/411025454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a4108230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f91940098a0 tx=0x7f9194009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.568+0000 7f91abdf3700 1 -- 172.21.15.105:0/411025454 shutdown_connections 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.568+0000 7f91abdf3700 1 --2- 172.21.15.105:0/411025454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a4108230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.568+0000 7f91abdf3700 1 -- 172.21.15.105:0/411025454 >> 172.21.15.105:0/411025454 conn(0x7f91a41033b0 msgr2=0x7f91a41057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.568+0000 7f91abdf3700 1 -- 172.21.15.105:0/411025454 shutdown_connections 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.568+0000 7f91abdf3700 1 -- 172.21.15.105:0/411025454 wait complete. 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91abdf3700 1 Processor -- start 2024-03-26T17:29:16.861 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91abdf3700 1 -- start start 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91abdf3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91abdf3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f91a410c3b0 con 0x7f91a4107e10 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91a9b8f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91a9b8f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36340/0 (socket says 172.21.15.105:36340) 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91a9b8f700 1 -- 172.21.15.105:0/657109096 learned_addr learned my addr 172.21.15.105:0/657109096 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91a9b8f700 1 -- 172.21.15.105:0/657109096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a419c210 con 0x7f91a4107e10 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91a9b8f700 1 --2- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a419bcd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9194011a20 tx=0x7f9194011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f919affd700 1 -- 172.21.15.105:0/657109096 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9194003ac0 con 0x7f91a4107e10 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f919affd700 1 -- 172.21.15.105:0/657109096 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91940248e0 con 0x7f91a4107e10 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f919affd700 1 -- 172.21.15.105:0/657109096 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f919401a820 con 0x7f91a4107e10 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.569+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a419c4a0 con 0x7f91a4107e10 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.570+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a419f180 con 0x7f91a4107e10 2024-03-26T17:29:16.862 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.570+0000 7f919affd700 1 -- 172.21.15.105:0/657109096 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f9194024430 con 0x7f91a4107e10 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.571+0000 7f919affd700 1 --2- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f9190039180 0x7f919003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.571+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a410c530 con 0x7f91a4107e10 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.571+0000 7f91a938e700 1 --2- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f9190039180 0x7f919003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.571+0000 7f919affd700 1 -- 172.21.15.105:0/657109096 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f919404d8a0 con 0x7f91a4107e10 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.571+0000 7f91a938e700 1 --2- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f9190039180 0x7f919003b640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.576+0000 7f919affd700 1 -- 172.21.15.105:0/657109096 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9194022e00 con 0x7f91a4107e10 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.794+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f91a40639e0 con 0x7f91a4107e10 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.796+0000 7f919affd700 1 -- 172.21.15.105:0/657109096 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f919404d1f0 con 0x7f91a4107e10 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f9190039180 msgr2=0x7f919003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 --2- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f9190039180 0x7f919003b640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91a0006fd0 tx=0x7f91a0006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:16.863 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 msgr2=0x7f91a419bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:16.864 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 --2- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a419bcd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9194011a20 tx=0x7f9194011b00 comp rx=0 tx=0).stop 2024-03-26T17:29:16.864 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 shutdown_connections 2024-03-26T17:29:16.864 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 --2- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f9190039180 0x7f919003b640 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:16.864 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 --2- 172.21.15.105:0/657109096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91a4107e10 0x7f91a419bcd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:16.864 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 >> 172.21.15.105:0/657109096 conn(0x7f91a41033b0 msgr2=0x7f91a4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:16.864 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 shutdown_connections 2024-03-26T17:29:16.864 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:15.798+0000 7f91abdf3700 1 -- 172.21.15.105:0/657109096 wait complete. 2024-03-26T17:29:17.999 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.246+0000 7f3525eff700 1 Processor -- start 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.246+0000 7f3525eff700 1 -- start start 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f3525eff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f3520106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f3525eff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f352007be40 con 0x7f35201066a0 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f3520106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f3520106ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36348/0 (socket says 172.21.15.105:36348) 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f351f7fe700 1 -- 172.21.15.105:0/1089946294 learned_addr learned my addr 172.21.15.105:0/1089946294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f351f7fe700 1 -- 172.21.15.105:0/1089946294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3520107000 con 0x7f35201066a0 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f351f7fe700 1 --2- 172.21.15.105:0/1089946294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f3520106ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f35100098a0 tx=0x7f3510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fc6848ba53d748 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f351e7fc700 1 -- 172.21.15.105:0/1089946294 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3510004160 con 0x7f35201066a0 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f351e7fc700 1 -- 172.21.15.105:0/1089946294 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3510004300 con 0x7f35201066a0 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.247+0000 7f351e7fc700 1 -- 172.21.15.105:0/1089946294 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3510011a50 con 0x7f35201066a0 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.248+0000 7f3525eff700 1 -- 172.21.15.105:0/1089946294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 msgr2=0x7f3520106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.248+0000 7f3525eff700 1 --2- 172.21.15.105:0/1089946294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f3520106ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f35100098a0 tx=0x7f3510009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:18.000 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.248+0000 7f3525eff700 1 -- 172.21.15.105:0/1089946294 shutdown_connections 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.248+0000 7f3525eff700 1 --2- 172.21.15.105:0/1089946294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f3520106ac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.248+0000 7f3525eff700 1 -- 172.21.15.105:0/1089946294 >> 172.21.15.105:0/1089946294 conn(0x7f3520102300 msgr2=0x7f3520104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.248+0000 7f3525eff700 1 -- 172.21.15.105:0/1089946294 shutdown_connections 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.248+0000 7f3525eff700 1 -- 172.21.15.105:0/1089946294 wait complete. 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f3525eff700 1 Processor -- start 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f3525eff700 1 -- start start 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f3525eff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f352019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f3525eff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3520108070 con 0x7f35201066a0 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f352019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f352019def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36350/0 (socket says 172.21.15.105:36350) 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f351f7fe700 1 -- 172.21.15.105:0/1927467702 learned_addr learned my addr 172.21.15.105:0/1927467702 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f351f7fe700 1 -- 172.21.15.105:0/1927467702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352019e430 con 0x7f35201066a0 2024-03-26T17:29:18.001 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.249+0000 7f351f7fe700 1 --2- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f352019def0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3510011a00 tx=0x7f3510011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.250+0000 7f351cff9700 1 -- 172.21.15.105:0/1927467702 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3510003af0 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.250+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352019e6c0 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.250+0000 7f351cff9700 1 -- 172.21.15.105:0/1927467702 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351001a8e0 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.250+0000 7f351cff9700 1 -- 172.21.15.105:0/1927467702 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3510024840 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.250+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35201a13b0 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.250+0000 7f351cff9700 1 -- 172.21.15.105:0/1927467702 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7f3510022020 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.251+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35200639e0 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.251+0000 7f351cff9700 1 --2- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f3508039180 0x7f350803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.251+0000 7f351cff9700 1 -- 172.21.15.105:0/1927467702 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f351004d4c0 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.251+0000 7f351effd700 1 --2- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f3508039180 0x7f350803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.251+0000 7f351effd700 1 --2- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f3508039180 0x7f350803b640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3514006fd0 tx=0x7f3514006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.256+0000 7f351cff9700 1 -- 172.21.15.105:0/1927467702 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f351001adb0 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.465+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f352002dec0 con 0x7f35201066a0 2024-03-26T17:29:18.002 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.468+0000 7f351cff9700 1 -- 172.21.15.105:0/1927467702 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f351000bc40 con 0x7f35201066a0 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.470+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f3508039180 msgr2=0x7f350803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.470+0000 7f3525eff700 1 --2- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f3508039180 0x7f350803b640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3514006fd0 tx=0x7f3514006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.470+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 msgr2=0x7f352019def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.470+0000 7f3525eff700 1 --2- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f352019def0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3510011a00 tx=0x7f3510011ae0 comp rx=0 tx=0).stop 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.470+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 shutdown_connections 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.470+0000 7f3525eff700 1 --2- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f3508039180 0x7f350803b640 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.470+0000 7f3525eff700 1 --2- 172.21.15.105:0/1927467702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35201066a0 0x7f352019def0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.470+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 >> 172.21.15.105:0/1927467702 conn(0x7f3520102300 msgr2=0x7f3520194bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.471+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 shutdown_connections 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:17.471+0000 7f3525eff700 1 -- 172.21.15.105:0/1927467702 wait complete. 2024-03-26T17:29:18.003 INFO:teuthology.orchestra.run.smithi105.stdout:Enabling the dashboard module... 2024-03-26T17:29:18.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:18 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:18.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:18 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1927467702' entity='client.admin' 2024-03-26T17:29:19.768 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:19 smithi105 ceph-mon[32280]: from='mgr.14120 172.21.15.105:0/3491571833' entity='mgr.x' 2024-03-26T17:29:19.768 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:19 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3838150815' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-26T17:29:19.768 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:19 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: ignoring --setuser ceph since I am not root 2024-03-26T17:29:19.768 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:19 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: ignoring --setgroup ceph since I am not root 2024-03-26T17:29:20.394 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:20 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:20.237+0000 7f460fdb2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:29:20.394 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:20 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:20.329+0000 7f460fdb2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:29:20.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.470+0000 7fdab395a700 1 Processor -- start 2024-03-26T17:29:20.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.470+0000 7fdab395a700 1 -- start start 2024-03-26T17:29:20.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.470+0000 7fdab395a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac108260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:20.688 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.470+0000 7fdab395a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac108830 con 0x7fdaac107e40 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.471+0000 7fdab16f6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.471+0000 7fdab16f6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac108260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36352/0 (socket says 172.21.15.105:36352) 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.471+0000 7fdab16f6700 1 -- 172.21.15.105:0/1106307245 learned_addr learned my addr 172.21.15.105:0/1106307245 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.471+0000 7fdab16f6700 1 -- 172.21.15.105:0/1106307245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac108ff0 con 0x7fdaac107e40 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.471+0000 7fdab16f6700 1 --2- 172.21.15.105:0/1106307245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac108260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0098a0 tx=0x7fda9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bea0c02b5ce14b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.471+0000 7fdaa3fff700 1 -- 172.21.15.105:0/1106307245 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda9c004160 con 0x7fdaac107e40 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.471+0000 7fdaa3fff700 1 -- 172.21.15.105:0/1106307245 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda9c004300 con 0x7fdaac107e40 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.471+0000 7fdaa3fff700 1 -- 172.21.15.105:0/1106307245 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda9c011a50 con 0x7fdaac107e40 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.472+0000 7fdab395a700 1 -- 172.21.15.105:0/1106307245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 msgr2=0x7fdaac108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.472+0000 7fdab395a700 1 --2- 172.21.15.105:0/1106307245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac108260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0098a0 tx=0x7fda9c009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.472+0000 7fdab395a700 1 -- 172.21.15.105:0/1106307245 shutdown_connections 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.472+0000 7fdab395a700 1 --2- 172.21.15.105:0/1106307245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac108260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.472+0000 7fdab395a700 1 -- 172.21.15.105:0/1106307245 >> 172.21.15.105:0/1106307245 conn(0x7fdaac103400 msgr2=0x7fdaac105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:20.689 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.472+0000 7fdab395a700 1 -- 172.21.15.105:0/1106307245 shutdown_connections 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.472+0000 7fdab395a700 1 -- 172.21.15.105:0/1106307245 wait complete. 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.473+0000 7fdab395a700 1 Processor -- start 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.473+0000 7fdab395a700 1 -- start start 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.473+0000 7fdab395a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac07db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.473+0000 7fdab395a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac10c3a0 con 0x7fdaac107e40 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.473+0000 7fdab16f6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac07db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.473+0000 7fdab16f6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac07db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36360/0 (socket says 172.21.15.105:36360) 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.473+0000 7fdab16f6700 1 -- 172.21.15.105:0/3838150815 learned_addr learned my addr 172.21.15.105:0/3838150815 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.473+0000 7fdab16f6700 1 -- 172.21.15.105:0/3838150815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac07e080 con 0x7fdaac107e40 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.474+0000 7fdab16f6700 1 --2- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac07db40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda9c011710 tx=0x7fda9c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.474+0000 7fdaa27fc700 1 -- 172.21.15.105:0/3838150815 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda9c0039c0 con 0x7fdaac107e40 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.474+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac07c270 con 0x7fdaac107e40 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.474+0000 7fdaa27fc700 1 -- 172.21.15.105:0/3838150815 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda9c003b60 con 0x7fdaac107e40 2024-03-26T17:29:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.474+0000 7fdaa27fc700 1 -- 172.21.15.105:0/3838150815 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda9c01a690 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.474+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac07c7b0 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.475+0000 7fdaa27fc700 1 -- 172.21.15.105:0/3838150815 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 46294+0+0 (secure 0 0 0) 0x7fda9c01a9c0 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.475+0000 7fdaa27fc700 1 --2- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fda98039180 0x7fda9803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.475+0000 7fdaa27fc700 1 -- 172.21.15.105:0/3838150815 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fda9c04d750 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.475+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac0639e0 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.475+0000 7fdab0ef5700 1 --2- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fda98039180 0x7fda9803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.475+0000 7fdab0ef5700 1 --2- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fda98039180 0x7fda9803b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8006fd0 tx=0x7fdaa8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.484+0000 7fdaa27fc700 1 -- 172.21.15.105:0/3838150815 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fda9c016df0 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:18.748+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fdaac07ce30 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.679+0000 7fdaa27fc700 1 -- 172.21.15.105:0/3838150815 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fda9c00bc40 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.680+0000 7fdaa27fc700 1 -- 172.21.15.105:0/3838150815 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7fda9c01acb0 con 0x7fdaac107e40 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fda98039180 msgr2=0x7fda9803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 --2- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fda98039180 0x7fda9803b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8006fd0 tx=0x7fdaa8006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:20.691 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 msgr2=0x7fdaac07db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:20.692 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 --2- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac07db40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda9c011710 tx=0x7fda9c011ad0 comp rx=0 tx=0).stop 2024-03-26T17:29:20.692 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 shutdown_connections 2024-03-26T17:29:20.692 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 --2- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fda98039180 0x7fda9803b640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:20.692 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 --2- 172.21.15.105:0/3838150815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaac107e40 0x7fdaac07db40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:20.692 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 >> 172.21.15.105:0/3838150815 conn(0x7fdaac103400 msgr2=0x7fdaac103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:20.692 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 shutdown_connections 2024-03-26T17:29:20.692 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:19.682+0000 7fdab395a700 1 -- 172.21.15.105:0/3838150815 wait complete. 2024-03-26T17:29:20.809 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:20 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3838150815' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-26T17:29:20.809 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:20 smithi105 ceph-mon[32280]: mgrmap e9: x(active, since 19s) 2024-03-26T17:29:21.066 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:20 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:20.976+0000 7f460fdb2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:29:21.066 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:21.071+0000 7f460fdb2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:29:21.390 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:21.218+0000 7f460fdb2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:29:21.390 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:21.315+0000 7f460fdb2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:29:21.644 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:21.398+0000 7f460fdb2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:29:21.956 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2540594339' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T17:29:21.956 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:21.664+0000 7f460fdb2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:29:21.956 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:21.776+0000 7f460fdb2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:29:21.956 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:21.863+0000 7f460fdb2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:29:21.956 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:21.960+0000 7f460fdb2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:29:22.070 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:29:22.070 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-26T17:29:22.070 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T17:29:22.070 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T17:29:22.070 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T17:29:22.070 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:29:22.070 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.188+0000 7f270780d700 1 Processor -- start 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.189+0000 7f270780d700 1 -- start start 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.189+0000 7f270780d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270007c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.189+0000 7f270780d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f270007c880 con 0x7f270007de50 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.189+0000 7f27055a9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270007c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.189+0000 7f27055a9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270007c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36380/0 (socket says 172.21.15.105:36380) 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.189+0000 7f27055a9700 1 -- 172.21.15.105:0/498891659 learned_addr learned my addr 172.21.15.105:0/498891659 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.189+0000 7f27055a9700 1 -- 172.21.15.105:0/498891659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270007ca00 con 0x7f270007de50 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.189+0000 7f27055a9700 1 --2- 172.21.15.105:0/498891659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270007c2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0098a0 tx=0x7f26fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e14a74723940cce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.190+0000 7f26f7fff700 1 -- 172.21.15.105:0/498891659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26fc004160 con 0x7f270007de50 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.190+0000 7f26f7fff700 1 -- 172.21.15.105:0/498891659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26fc004300 con 0x7f270007de50 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.190+0000 7f26f7fff700 1 -- 172.21.15.105:0/498891659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26fc011a50 con 0x7f270007de50 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.191+0000 7f270780d700 1 -- 172.21.15.105:0/498891659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 msgr2=0x7f270007c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.191+0000 7f270780d700 1 --2- 172.21.15.105:0/498891659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270007c2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0098a0 tx=0x7f26fc009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:22.071 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.191+0000 7f270780d700 1 -- 172.21.15.105:0/498891659 shutdown_connections 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.191+0000 7f270780d700 1 --2- 172.21.15.105:0/498891659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270007c2b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.191+0000 7f270780d700 1 -- 172.21.15.105:0/498891659 >> 172.21.15.105:0/498891659 conn(0x7f27001033a0 msgr2=0x7f27001057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.191+0000 7f270780d700 1 -- 172.21.15.105:0/498891659 shutdown_connections 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.191+0000 7f270780d700 1 -- 172.21.15.105:0/498891659 wait complete. 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.191+0000 7f270780d700 1 Processor -- start 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f270780d700 1 -- start start 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f270780d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f270780d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f270010e420 con 0x7f270007de50 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f27055a9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f27055a9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270019df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36394/0 (socket says 172.21.15.105:36394) 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f27055a9700 1 -- 172.21.15.105:0/2540594339 learned_addr learned my addr 172.21.15.105:0/2540594339 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f27055a9700 1 -- 172.21.15.105:0/2540594339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270019e4a0 con 0x7f270007de50 2024-03-26T17:29:22.072 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f27055a9700 1 --2- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270019df60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26fc011a20 tx=0x7f26fc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f26f67fc700 1 -- 172.21.15.105:0/2540594339 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26fc0039c0 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f26f67fc700 1 -- 172.21.15.105:0/2540594339 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26fc003b60 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.192+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270019e730 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.193+0000 7f26f67fc700 1 -- 172.21.15.105:0/2540594339 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26fc01a6b0 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.193+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27001a13d0 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.194+0000 7f26f67fc700 1 -- 172.21.15.105:0/2540594339 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7f26fc01a9b0 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.194+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27000639e0 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.194+0000 7f26f67fc700 1 --2- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f26ec039180 0x7f26ec03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.194+0000 7f26f67fc700 1 -- 172.21.15.105:0/2540594339 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f26fc04d810 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.194+0000 7f2704da8700 1 -- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f26ec039180 msgr2=0x7f26ec03b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1556474900 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.194+0000 7f2704da8700 1 --2- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f26ec039180 0x7f26ec03b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.199+0000 7f26f67fc700 1 -- 172.21.15.105:0/2540594339 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26fc016df0 con 0x7f270007de50 2024-03-26T17:29:22.073 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.395+0000 7f2704da8700 1 -- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f26ec039180 msgr2=0x7f26ec03b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1556474900 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.395+0000 7f2704da8700 1 --2- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f26ec039180 0x7f26ec03b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.454+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f27001a1980 con 0x7f270007de50 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.454+0000 7f26f67fc700 1 -- 172.21.15.105:0/2540594339 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f26fc01ada0 con 0x7f270007de50 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f26ec039180 msgr2=0x7f26ec03b640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 --2- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f26ec039180 0x7f26ec03b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 msgr2=0x7f270019df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 --2- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270019df60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26fc011a20 tx=0x7f26fc011b00 comp rx=0 tx=0).stop 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 shutdown_connections 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 --2- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7f26ec039180 0x7f26ec03b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 --2- 172.21.15.105:0/2540594339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f270007de50 0x7f270019df60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 >> 172.21.15.105:0/2540594339 conn(0x7f27001033a0 msgr2=0x7f2700103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 shutdown_connections 2024-03-26T17:29:22.074 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:21.457+0000 7f270780d700 1 -- 172.21.15.105:0/2540594339 wait complete. 2024-03-26T17:29:22.075 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for the mgr to restart... 2024-03-26T17:29:22.075 INFO:teuthology.orchestra.run.smithi105.stdout:Waiting for mgr epoch 9... 2024-03-26T17:29:22.284 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:22 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:22.144+0000 7f460fdb2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:29:23.365 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:23 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:23.104+0000 7f460fdb2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:29:23.365 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:23 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:23.241+0000 7f460fdb2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:29:23.644 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:23 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:23.585+0000 7f460fdb2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:29:24.059 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:23 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:23.676+0000 7f460fdb2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:29:24.060 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:23 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:23.876+0000 7f460fdb2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:29:24.060 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:24 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:24.064+0000 7f460fdb2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:29:24.600 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:24 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:24.351+0000 7f460fdb2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:29:24.600 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:24 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:24.604+0000 7f460fdb2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:29:24.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:24 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:24.837+0000 7f460fdb2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:29:25.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:25 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:25.625+0000 7f460fdb2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:29:26.483 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:26.181+0000 7f460fdb2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:29:26.483 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:26.276+0000 7f460fdb2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:29:26.483 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:29:26.488+0000 7f460fdb2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:29:26.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: Active manager daemon x restarted 2024-03-26T17:29:26.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: Activating manager daemon x 2024-03-26T17:29:26.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: osdmap e3: 0 total, 0 up, 0 in 2024-03-26T17:29:26.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: mgrmap e10: x(active, starting, since 0.0034792s) 2024-03-26T17:29:26.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:29:26.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T17:29:26.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T17:29:26.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T17:29:26.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T17:29:26.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: Manager daemon x is now available 2024-03-26T17:29:26.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:27.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T17:29:27.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T17:29:27.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:27 smithi105 ceph-mon[32280]: mgrmap e11: x(active, since 1.00541s) 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout { 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout } 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.492+0000 7fc5a2bae700 1 Processor -- start 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.493+0000 7fc5a2bae700 1 -- start start 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.493+0000 7fc5a2bae700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c072ed0 0x7fc59c0732f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.493+0000 7fc5a2bae700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c0738c0 con 0x7fc59c072ed0 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.494+0000 7fc5a094a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c072ed0 0x7fc59c0732f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:28.373 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.494+0000 7fc5a094a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c072ed0 0x7fc59c0732f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36402/0 (socket says 172.21.15.105:36402) 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.494+0000 7fc5a094a700 1 -- 172.21.15.105:0/2367259537 learned_addr learned my addr 172.21.15.105:0/2367259537 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.494+0000 7fc5a094a700 1 -- 172.21.15.105:0/2367259537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c073a40 con 0x7fc59c072ed0 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.495+0000 7fc5a094a700 1 --2- 172.21.15.105:0/2367259537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c072ed0 0x7fc59c0732f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc590008920 tx=0x7fc59000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29daf46c245e9973 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.495+0000 7fc59b7fe700 1 -- 172.21.15.105:0/2367259537 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59001b070 con 0x7fc59c072ed0 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.495+0000 7fc59b7fe700 1 -- 172.21.15.105:0/2367259537 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc590004030 con 0x7fc59c072ed0 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.496+0000 7fc5a2bae700 1 -- 172.21.15.105:0/2367259537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c072ed0 msgr2=0x7fc59c0732f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.496+0000 7fc5a2bae700 1 --2- 172.21.15.105:0/2367259537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c072ed0 0x7fc59c0732f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc590008920 tx=0x7fc59000d330 comp rx=0 tx=0).stop 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.497+0000 7fc5a2bae700 1 -- 172.21.15.105:0/2367259537 shutdown_connections 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.497+0000 7fc5a2bae700 1 --2- 172.21.15.105:0/2367259537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c072ed0 0x7fc59c0732f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.497+0000 7fc5a2bae700 1 -- 172.21.15.105:0/2367259537 >> 172.21.15.105:0/2367259537 conn(0x7fc59c06e1a0 msgr2=0x7fc59c0705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.497+0000 7fc5a2bae700 1 -- 172.21.15.105:0/2367259537 shutdown_connections 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.497+0000 7fc5a2bae700 1 -- 172.21.15.105:0/2367259537 wait complete. 2024-03-26T17:29:28.374 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.498+0000 7fc5a2bae700 1 Processor -- start 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.498+0000 7fc5a2bae700 1 -- start start 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.498+0000 7fc5a2bae700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c1a3860 0x7fc59c1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.498+0000 7fc5a2bae700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c112740 con 0x7fc59c1a3860 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.498+0000 7fc5a094a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c1a3860 0x7fc59c1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.498+0000 7fc5a094a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c1a3860 0x7fc59c1a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36404/0 (socket says 172.21.15.105:36404) 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.498+0000 7fc5a094a700 1 -- 172.21.15.105:0/4090228743 learned_addr learned my addr 172.21.15.105:0/4090228743 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.499+0000 7fc5a094a700 1 -- 172.21.15.105:0/4090228743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c1a2510 con 0x7fc59c1a3860 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.499+0000 7fc5a094a700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c1a3860 0x7fc59c1a1fd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc59000dd40 tx=0x7fc59000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.507+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc590026070 con 0x7fc59c1a3860 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.507+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c1a2800 con 0x7fc59c1a3860 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.507+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c1a2d40 con 0x7fc59c1a3860 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.507+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5900043a0 con 0x7fc59c1a3860 2024-03-26T17:29:28.375 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.507+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59001b070 con 0x7fc59c1a3860 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.509+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 9) ==== 46307+0+0 (secure 0 0 0) 0x7fc590029070 con 0x7fc59c1a3860 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.509+0000 7fc599ffb700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 0x7fc58c03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.509+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 --> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc58c03be90 con 0x7fc58c0392c0 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.509+0000 7fc59bfff700 1 -- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 msgr2=0x7fc58c03b780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1556474900 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.510+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc590011030 con 0x7fc59c1a3860 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.510+0000 7fc59bfff700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 0x7fc58c03b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.711+0000 7fc59bfff700 1 -- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 msgr2=0x7fc58c03b780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1556474900 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:22.711+0000 7fc59bfff700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 0x7fc58c03b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:23.112+0000 7fc59bfff700 1 -- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 msgr2=0x7fc58c03b780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1556474900 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:23.112+0000 7fc59bfff700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 0x7fc58c03b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:23.913+0000 7fc59bfff700 1 -- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 msgr2=0x7fc58c03b780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1556474900 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:23.913+0000 7fc59bfff700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 0x7fc58c03b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T17:29:28.376 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:25.514+0000 7fc59bfff700 1 -- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 msgr2=0x7fc58c03b780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:6800/1556474900 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:25.514+0000 7fc59bfff700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 0x7fc58c03b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:26.503+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mgrmap(e 10) ==== 46074+0+0 (secure 0 0 0) 0x7fc59002a3b0 con 0x7fc59c1a3860 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:26.503+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 msgr2=0x7fc58c03b780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:26.503+0000 7fc599ffb700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 0x7fc58c03b780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.504+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 11) ==== 46201+0+0 (secure 0 0 0) 0x7fc59004d7e0 con 0x7fc59c1a3860 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.504+0000 7fc599ffb700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc58c03cd40 0x7fc58c03f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.505+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc590007ea0 con 0x7fc58c03cd40 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.505+0000 7fc59bfff700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc58c03cd40 0x7fc58c03f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.505+0000 7fc59bfff700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc58c03cd40 0x7fc58c03f130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc588003a10 tx=0x7fc5880092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.506+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fc590007ea0 con 0x7fc58c03cd40 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.512+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc580002600 con 0x7fc58c03cd40 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.512+0000 7fc599ffb700 1 -- 172.21.15.105:0/4090228743 <== mgr.14152 v2:172.21.15.105:6800/2294330938 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fc580002600 con 0x7fc58c03cd40 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.512+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc58c03cd40 msgr2=0x7fc58c03f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:28.377 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.512+0000 7fc5a2bae700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc58c03cd40 0x7fc58c03f130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc588003a10 tx=0x7fc5880092b0 comp rx=0 tx=0).stop 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.512+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c1a3860 msgr2=0x7fc59c1a1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.512+0000 7fc5a2bae700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c1a3860 0x7fc59c1a1fd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc59000dd40 tx=0x7fc59000de20 comp rx=0 tx=0).stop 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.513+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 shutdown_connections 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.513+0000 7fc5a2bae700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc58c03cd40 0x7fc58c03f130 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.513+0000 7fc5a2bae700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc59c1a3860 0x7fc59c1a1fd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.513+0000 7fc5a2bae700 1 --2- 172.21.15.105:0/4090228743 >> [v2:172.21.15.105:6800/1556474900,v1:172.21.15.105:6801/1556474900] conn(0x7fc58c0392c0 0x7fc58c03b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.513+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 >> 172.21.15.105:0/4090228743 conn(0x7fc59c06e1a0 msgr2=0x7fc59c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.513+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 shutdown_connections 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:27.513+0000 7fc5a2bae700 1 -- 172.21.15.105:0/4090228743 wait complete. 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:mgr epoch 9 is available 2024-03-26T17:29:28.378 INFO:teuthology.orchestra.run.smithi105.stdout:Generating a dashboard self-signed certificate... 2024-03-26T17:29:28.614 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:28 smithi105 ceph-mon[32280]: [26/Mar/2024:17:29:26] ENGINE Bus STARTING 2024-03-26T17:29:28.615 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:28 smithi105 ceph-mon[32280]: [26/Mar/2024:17:29:26] ENGINE Serving on https://172.21.15.105:7150 2024-03-26T17:29:28.615 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:28 smithi105 ceph-mon[32280]: [26/Mar/2024:17:29:26] ENGINE Serving on http://172.21.15.105:8765 2024-03-26T17:29:28.615 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:28 smithi105 ceph-mon[32280]: [26/Mar/2024:17:29:26] ENGINE Bus STARTED 2024-03-26T17:29:28.615 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:28 smithi105 ceph-mon[32280]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T17:29:28.615 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:28 smithi105 ceph-mon[32280]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T17:29:28.615 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.838+0000 7fcd9b98a700 1 Processor -- start 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.839+0000 7fcd9b98a700 1 -- start start 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.839+0000 7fcd9b98a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd94108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.839+0000 7fcd9b98a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd94108770 con 0x7fcd94107e10 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.839+0000 7fcd99726700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd94108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.839+0000 7fcd99726700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd94108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60514/0 (socket says 172.21.15.105:60514) 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.839+0000 7fcd99726700 1 -- 172.21.15.105:0/2577019122 learned_addr learned my addr 172.21.15.105:0/2577019122 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.839+0000 7fcd99726700 1 -- 172.21.15.105:0/2577019122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd94108f70 con 0x7fcd94107e10 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.840+0000 7fcd99726700 1 --2- 172.21.15.105:0/2577019122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd94108230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcd900098a0 tx=0x7fcd90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5a904a600691fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:29.695 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.840+0000 7fcd8bfff700 1 -- 172.21.15.105:0/2577019122 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd90004160 con 0x7fcd94107e10 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.840+0000 7fcd8bfff700 1 -- 172.21.15.105:0/2577019122 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd90004300 con 0x7fcd94107e10 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.840+0000 7fcd8bfff700 1 -- 172.21.15.105:0/2577019122 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd90011a50 con 0x7fcd94107e10 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.840+0000 7fcd9b98a700 1 -- 172.21.15.105:0/2577019122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 msgr2=0x7fcd94108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.840+0000 7fcd9b98a700 1 --2- 172.21.15.105:0/2577019122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd94108230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcd900098a0 tx=0x7fcd90009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 -- 172.21.15.105:0/2577019122 shutdown_connections 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 --2- 172.21.15.105:0/2577019122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd94108230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 -- 172.21.15.105:0/2577019122 >> 172.21.15.105:0/2577019122 conn(0x7fcd941033b0 msgr2=0x7fcd941057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 -- 172.21.15.105:0/2577019122 shutdown_connections 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 -- 172.21.15.105:0/2577019122 wait complete. 2024-03-26T17:29:29.696 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 Processor -- start 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 -- start start 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd9419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.841+0000 7fcd9b98a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9410c3b0 con 0x7fcd94107e10 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd99726700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd9419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd99726700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd9419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60526/0 (socket says 172.21.15.105:60526) 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd99726700 1 -- 172.21.15.105:0/3586111068 learned_addr learned my addr 172.21.15.105:0/3586111068 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd99726700 1 -- 172.21.15.105:0/3586111068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9419c210 con 0x7fcd94107e10 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd99726700 1 --2- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd9419bcd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009130 tx=0x7fcd90011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd8a7fc700 1 -- 172.21.15.105:0/3586111068 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd90003b60 con 0x7fcd94107e10 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd8a7fc700 1 -- 172.21.15.105:0/3586111068 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd9001a8e0 con 0x7fcd94107e10 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9419c4a0 con 0x7fcd94107e10 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd8a7fc700 1 -- 172.21.15.105:0/3586111068 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd90024940 con 0x7fcd94107e10 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.842+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9419f180 con 0x7fcd94107e10 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.843+0000 7fcd8a7fc700 1 -- 172.21.15.105:0/3586111068 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 11) ==== 46201+0+0 (secure 0 0 0) 0x7fcd90022020 con 0x7fcd94107e10 2024-03-26T17:29:29.697 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.843+0000 7fcd8a7fc700 1 --2- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd80039250 0x7fcd8003b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.843+0000 7fcd8a7fc700 1 -- 172.21.15.105:0/3586111068 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd9004d8a0 con 0x7fcd94107e10 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.843+0000 7fcd98f25700 1 --2- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd80039250 0x7fcd8003b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.843+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9410c530 con 0x7fcd94107e10 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.844+0000 7fcd98f25700 1 --2- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd80039250 0x7fcd8003b710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd84006fd0 tx=0x7fcd84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.849+0000 7fcd8a7fc700 1 -- 172.21.15.105:0/3586111068 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcd9001ad90 con 0x7fcd94107e10 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:28.871+0000 7fcd8a7fc700 1 -- 172.21.15.105:0/3586111068 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7fcd9002dc40 con 0x7fcd94107e10 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.061+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fcd94104530 con 0x7fcd80039250 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.119+0000 7fcd8a7fc700 1 -- 172.21.15.105:0/3586111068 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fcd94104530 con 0x7fcd80039250 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.121+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd80039250 msgr2=0x7fcd8003b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.121+0000 7fcd9b98a700 1 --2- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd80039250 0x7fcd8003b710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd84006fd0 tx=0x7fcd84006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.121+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 msgr2=0x7fcd9419bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.121+0000 7fcd9b98a700 1 --2- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd9419bcd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009130 tx=0x7fcd90011b40 comp rx=0 tx=0).stop 2024-03-26T17:29:29.698 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.121+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 shutdown_connections 2024-03-26T17:29:29.699 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.121+0000 7fcd9b98a700 1 --2- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd80039250 0x7fcd8003b710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:29.699 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.121+0000 7fcd9b98a700 1 --2- 172.21.15.105:0/3586111068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd94107e10 0x7fcd9419bcd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:29.699 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.121+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 >> 172.21.15.105:0/3586111068 conn(0x7fcd941033b0 msgr2=0x7fcd94103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:29.699 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.122+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 shutdown_connections 2024-03-26T17:29:29.699 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:29.122+0000 7fcd9b98a700 1 -- 172.21.15.105:0/3586111068 wait complete. 2024-03-26T17:29:29.699 INFO:teuthology.orchestra.run.smithi105.stdout:Creating initial admin user... 2024-03-26T17:29:30.117 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:29 smithi105 ceph-mon[32280]: mgrmap e12: x(active, since 2s) 2024-03-26T17:29:30.206 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:30.207 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:29 smithi105 ceph-mon[32280]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:30.207 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:30.207 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pYO0nXLrgiOvOkxk2YY3hOqCL3gtfSOq79QCB2Qa6ECPPY8iwm2iW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711474170, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.140+0000 7fe2c555a700 1 Processor -- start 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.140+0000 7fe2c555a700 1 -- start start 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2c555a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c0108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2c555a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c0108770 con 0x7fe2c0107e10 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2beffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c0108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2beffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c0108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60532/0 (socket says 172.21.15.105:60532) 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2beffd700 1 -- 172.21.15.105:0/3289172096 learned_addr learned my addr 172.21.15.105:0/3289172096 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2beffd700 1 -- 172.21.15.105:0/3289172096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c0108f70 con 0x7fe2c0107e10 2024-03-26T17:29:31.342 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2beffd700 1 --2- 172.21.15.105:0/3289172096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c0108230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00098a0 tx=0x7fe2b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0f4f4cd003c1f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:31.342 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2be7fc700 1 -- 172.21.15.105:0/3289172096 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b0004160 con 0x7fe2c0107e10 2024-03-26T17:29:31.342 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2be7fc700 1 -- 172.21.15.105:0/3289172096 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b0004300 con 0x7fe2c0107e10 2024-03-26T17:29:31.342 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.141+0000 7fe2be7fc700 1 -- 172.21.15.105:0/3289172096 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b0011a50 con 0x7fe2c0107e10 2024-03-26T17:29:31.342 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.142+0000 7fe2c555a700 1 -- 172.21.15.105:0/3289172096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 msgr2=0x7fe2c0108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:31.342 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.142+0000 7fe2c555a700 1 --2- 172.21.15.105:0/3289172096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c0108230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00098a0 tx=0x7fe2b0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:31.343 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.142+0000 7fe2c555a700 1 -- 172.21.15.105:0/3289172096 shutdown_connections 2024-03-26T17:29:31.343 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.142+0000 7fe2c555a700 1 --2- 172.21.15.105:0/3289172096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c0108230 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:31.343 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.142+0000 7fe2c555a700 1 -- 172.21.15.105:0/3289172096 >> 172.21.15.105:0/3289172096 conn(0x7fe2c01033b0 msgr2=0x7fe2c01057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:31.343 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.142+0000 7fe2c555a700 1 -- 172.21.15.105:0/3289172096 shutdown_connections 2024-03-26T17:29:31.344 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.142+0000 7fe2c555a700 1 -- 172.21.15.105:0/3289172096 wait complete. 2024-03-26T17:29:31.344 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2c555a700 1 Processor -- start 2024-03-26T17:29:31.344 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2c555a700 1 -- start start 2024-03-26T17:29:31.344 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2c555a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:31.344 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2c555a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c010c3b0 con 0x7fe2c0107e10 2024-03-26T17:29:31.344 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2beffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:31.344 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2beffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60540/0 (socket says 172.21.15.105:60540) 2024-03-26T17:29:31.344 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2beffd700 1 -- 172.21.15.105:0/1356011112 learned_addr learned my addr 172.21.15.105:0/1356011112 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2beffd700 1 -- 172.21.15.105:0/1356011112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c019c210 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2beffd700 1 --2- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c019bcd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0011a20 tx=0x7fe2b0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2bcff9700 1 -- 172.21.15.105:0/1356011112 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b0003ac0 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2bcff9700 1 -- 172.21.15.105:0/1356011112 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b00248e0 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.143+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c019c4a0 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.144+0000 7fe2bcff9700 1 -- 172.21.15.105:0/1356011112 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b001a820 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.144+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c019f180 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.144+0000 7fe2bcff9700 1 -- 172.21.15.105:0/1356011112 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7fe2b001aad0 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.144+0000 7fe2bcff9700 1 --2- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2ac0391e0 0x7fe2ac03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.144+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c010c530 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.145+0000 7fe2bcff9700 1 -- 172.21.15.105:0/1356011112 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe2b004d970 con 0x7fe2c0107e10 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.145+0000 7fe2b7fff700 1 --2- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2ac0391e0 0x7fe2ac03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.145+0000 7fe2b7fff700 1 --2- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2ac0391e0 0x7fe2ac03b6a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8006fd0 tx=0x7fe2a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:31.345 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.149+0000 7fe2bcff9700 1 -- 172.21.15.105:0/1356011112 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe2b0016df0 con 0x7fe2c0107e10 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.376+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe2c00639e0 con 0x7fe2ac0391e0 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.601+0000 7fe2bcff9700 1 -- 172.21.15.105:0/1356011112 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe2c00639e0 con 0x7fe2ac0391e0 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2ac0391e0 msgr2=0x7fe2ac03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 --2- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2ac0391e0 0x7fe2ac03b6a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8006fd0 tx=0x7fe2a8006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 msgr2=0x7fe2c019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 --2- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c019bcd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0011a20 tx=0x7fe2b0011b00 comp rx=0 tx=0).stop 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 shutdown_connections 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 --2- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2ac0391e0 0x7fe2ac03b6a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 --2- 172.21.15.105:0/1356011112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2c0107e10 0x7fe2c019bcd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 >> 172.21.15.105:0/1356011112 conn(0x7fe2c01033b0 msgr2=0x7fe2c0103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 shutdown_connections 2024-03-26T17:29:31.346 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:30.603+0000 7fe2c555a700 1 -- 172.21.15.105:0/1356011112 wait complete. 2024-03-26T17:29:31.347 INFO:teuthology.orchestra.run.smithi105.stdout:Fetching dashboard port number... 2024-03-26T17:29:31.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:31 smithi105 ceph-mon[32280]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:31.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:32.592 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stdout 8443 2024-03-26T17:29:32.592 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.750+0000 7f6c4e4a0700 1 Processor -- start 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.750+0000 7f6c4e4a0700 1 -- start start 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.750+0000 7f6c4e4a0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c4810ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.750+0000 7f6c4e4a0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4807be00 con 0x7f6c481088f0 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.750+0000 7f6c47fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c4810ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.750+0000 7f6c47fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c4810ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60548/0 (socket says 172.21.15.105:60548) 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.750+0000 7f6c47fff700 1 -- 172.21.15.105:0/2592068092 learned_addr learned my addr 172.21.15.105:0/2592068092 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.751+0000 7f6c47fff700 1 -- 172.21.15.105:0/2592068092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4810b220 con 0x7f6c481088f0 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.751+0000 7f6c47fff700 1 --2- 172.21.15.105:0/2592068092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c4810ace0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098a0 tx=0x7f6c3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f093a0d295fbfd41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.751+0000 7f6c46ffd700 1 -- 172.21.15.105:0/2592068092 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c004160 con 0x7f6c481088f0 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.751+0000 7f6c46ffd700 1 -- 172.21.15.105:0/2592068092 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c3c004300 con 0x7f6c481088f0 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.751+0000 7f6c46ffd700 1 -- 172.21.15.105:0/2592068092 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c011a50 con 0x7f6c481088f0 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/2592068092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 msgr2=0x7f6c4810ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:32.593 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 --2- 172.21.15.105:0/2592068092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c4810ace0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098a0 tx=0x7f6c3c009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/2592068092 shutdown_connections 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 --2- 172.21.15.105:0/2592068092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c4810ace0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/2592068092 >> 172.21.15.105:0/2592068092 conn(0x7f6c48102300 msgr2=0x7f6c48104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/2592068092 shutdown_connections 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/2592068092 wait complete. 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 Processor -- start 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.752+0000 7f6c4e4a0700 1 -- start start 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c4e4a0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c48108170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c4e4a0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4810c1c0 con 0x7f6c481088f0 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c47fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c48108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c47fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c48108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60554/0 (socket says 172.21.15.105:60554) 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c47fff700 1 -- 172.21.15.105:0/3163340481 learned_addr learned my addr 172.21.15.105:0/3163340481 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:32.594 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c47fff700 1 -- 172.21.15.105:0/3163340481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c481086b0 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c47fff700 1 --2- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c48108170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c011a20 tx=0x7f6c3c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c457fa700 1 -- 172.21.15.105:0/3163340481 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c0039c0 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c457fa700 1 -- 172.21.15.105:0/3163340481 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c3c003b60 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.753+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c481068a0 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.754+0000 7f6c457fa700 1 -- 172.21.15.105:0/3163340481 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c3c01a6b0 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.754+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c48106de0 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.754+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c480639e0 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.755+0000 7f6c457fa700 1 -- 172.21.15.105:0/3163340481 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 12) ==== 46307+0+0 (secure 0 0 0) 0x7f6c3c024430 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.755+0000 7f6c457fa700 1 --2- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c300391e0 0x7f6c3003b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.755+0000 7f6c457fa700 1 -- 172.21.15.105:0/3163340481 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c3c04d8b0 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.755+0000 7f6c477fe700 1 --2- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c300391e0 0x7f6c3003b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.755+0000 7f6c477fe700 1 --2- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c300391e0 0x7f6c3003b6a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6c38006fd0 tx=0x7f6c38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.764+0000 7f6c457fa700 1 -- 172.21.15.105:0/3163340481 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6c3c00f2e0 con 0x7f6c481088f0 2024-03-26T17:29:32.595 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.967+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f6c48107060 con 0x7f6c481088f0 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.967+0000 7f6c457fa700 1 -- 172.21.15.105:0/3163340481 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6c3c00bcc0 con 0x7f6c481088f0 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c300391e0 msgr2=0x7f6c3003b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 --2- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c300391e0 0x7f6c3003b6a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6c38006fd0 tx=0x7f6c38006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 msgr2=0x7f6c48108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 --2- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c48108170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c011a20 tx=0x7f6c3c011b00 comp rx=0 tx=0).stop 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 shutdown_connections 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 --2- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c300391e0 0x7f6c3003b6a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 --2- 172.21.15.105:0/3163340481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c481088f0 0x7f6c48108170 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 >> 172.21.15.105:0/3163340481 conn(0x7f6c48102300 msgr2=0x7f6c48102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 shutdown_connections 2024-03-26T17:29:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:31.969+0000 7f6c4e4a0700 1 -- 172.21.15.105:0/3163340481 wait complete. 2024-03-26T17:29:32.598 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T17:29:32.598 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout disabled 2024-03-26T17:29:32.605 INFO:teuthology.orchestra.run.smithi105.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T17:29:32.606 INFO:teuthology.orchestra.run.smithi105.stdout:systemctl: stdout inactive 2024-03-26T17:29:32.606 INFO:teuthology.orchestra.run.smithi105.stdout:firewalld.service is not enabled 2024-03-26T17:29:32.606 INFO:teuthology.orchestra.run.smithi105.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-26T17:29:32.607 INFO:teuthology.orchestra.run.smithi105.stdout:Ceph Dashboard is now available at: 2024-03-26T17:29:32.607 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:32.607 INFO:teuthology.orchestra.run.smithi105.stdout: URL: https://smithi105.front.sepia.ceph.com:8443/ 2024-03-26T17:29:32.608 INFO:teuthology.orchestra.run.smithi105.stdout: User: admin 2024-03-26T17:29:32.608 INFO:teuthology.orchestra.run.smithi105.stdout: Password: b42r8plq1t 2024-03-26T17:29:32.608 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:32.608 INFO:teuthology.orchestra.run.smithi105.stdout:Saving cluster configuration to /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config directory 2024-03-26T17:29:32.608 INFO:teuthology.orchestra.run.smithi105.stdout:Enabling autotune for osd_memory_target 2024-03-26T17:29:32.818 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:32 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3163340481' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-26T17:29:33.751 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:33 smithi105 ceph-mon[32280]: mgrmap e13: x(active, since 6s) 2024-03-26T17:29:33.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.001+0000 7ff8fbaed700 1 Processor -- start 2024-03-26T17:29:33.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.001+0000 7ff8fbaed700 1 -- start start 2024-03-26T17:29:33.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.001+0000 7ff8fbaed700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f4107110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:33.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.001+0000 7ff8fbaed700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41076e0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.756 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.002+0000 7ff8f9889700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f4107110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.002+0000 7ff8f9889700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f4107110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60564/0 (socket says 172.21.15.105:60564) 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.002+0000 7ff8f9889700 1 -- 172.21.15.105:0/2733163013 learned_addr learned my addr 172.21.15.105:0/2733163013 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.002+0000 7ff8f9889700 1 -- 172.21.15.105:0/2733163013 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f4107ef0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.002+0000 7ff8f9889700 1 --2- 172.21.15.105:0/2733163013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f4107110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00098a0 tx=0x7ff8f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55c72e45b873cd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.002+0000 7ff8f8887700 1 -- 172.21.15.105:0/2733163013 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f0004160 con 0x7ff8f4106cf0 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.002+0000 7ff8f8887700 1 -- 172.21.15.105:0/2733163013 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8f0004300 con 0x7ff8f4106cf0 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.002+0000 7ff8f8887700 1 -- 172.21.15.105:0/2733163013 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f0011a50 con 0x7ff8f4106cf0 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2733163013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 msgr2=0x7ff8f4107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 --2- 172.21.15.105:0/2733163013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f4107110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00098a0 tx=0x7ff8f0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2733163013 shutdown_connections 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 --2- 172.21.15.105:0/2733163013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f4107110 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2733163013 >> 172.21.15.105:0/2733163013 conn(0x7ff8f41022b0 msgr2=0x7ff8f41046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:33.757 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2733163013 shutdown_connections 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2733163013 wait complete. 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 Processor -- start 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.003+0000 7ff8fbaed700 1 -- start start 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8fbaed700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8fbaed700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f410b2a0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8f9889700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8f9889700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60570/0 (socket says 172.21.15.105:60570) 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8f9889700 1 -- 172.21.15.105:0/2205447455 learned_addr learned my addr 172.21.15.105:0/2205447455 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8f9889700 1 -- 172.21.15.105:0/2205447455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f419c2e0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8f9889700 1 --2- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f419bda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0011a20 tx=0x7ff8f0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8eaffd700 1 -- 172.21.15.105:0/2205447455 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f0003af0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8eaffd700 1 -- 172.21.15.105:0/2205447455 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8f001a8e0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.758 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8eaffd700 1 -- 172.21.15.105:0/2205447455 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8f0024860 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f419c570 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.004+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f419f210 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.005+0000 7ff8eaffd700 1 -- 172.21.15.105:0/2205447455 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7ff8f0022020 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.005+0000 7ff8eaffd700 1 --2- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8e00392d0 0x7ff8e003b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.005+0000 7ff8eaffd700 1 -- 172.21.15.105:0/2205447455 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff8f004d5a0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.005+0000 7ff8f9088700 1 --2- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8e00392d0 0x7ff8e003b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.005+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f40639e0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.006+0000 7ff8f9088700 1 --2- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8e00392d0 0x7ff8e003b790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4006fd0 tx=0x7ff8e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.010+0000 7ff8eaffd700 1 -- 172.21.15.105:0/2205447455 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff8f000b3e0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.204+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff8f402ddb0 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.204+0000 7ff8eaffd700 1 -- 172.21.15.105:0/2205447455 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff8f0032020 con 0x7ff8f4106cf0 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8e00392d0 msgr2=0x7ff8e003b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 --2- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8e00392d0 0x7ff8e003b790 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4006fd0 tx=0x7ff8e4006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:33.759 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 msgr2=0x7ff8f419bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:33.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 --2- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f419bda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0011a20 tx=0x7ff8f0011b00 comp rx=0 tx=0).stop 2024-03-26T17:29:33.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 shutdown_connections 2024-03-26T17:29:33.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 --2- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8e00392d0 0x7ff8e003b790 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:33.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 --2- 172.21.15.105:0/2205447455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8f4106cf0 0x7ff8f419bda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:33.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 >> 172.21.15.105:0/2205447455 conn(0x7ff8f41022b0 msgr2=0x7ff8f4103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:33.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.206+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 shutdown_connections 2024-03-26T17:29:33.760 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:33.207+0000 7ff8fbaed700 1 -- 172.21.15.105:0/2205447455 wait complete. 2024-03-26T17:29:35.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.208+0000 7f54cb7de700 1 Processor -- start 2024-03-26T17:29:35.097 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.208+0000 7f54cb7de700 1 -- start start 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.208+0000 7f54cb7de700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c410ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.208+0000 7f54cb7de700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f54c407be80 con 0x7f54c4108880 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.209+0000 7f54c957a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c410ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.209+0000 7f54c957a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c410ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60578/0 (socket says 172.21.15.105:60578) 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.209+0000 7f54c957a700 1 -- 172.21.15.105:0/843503022 learned_addr learned my addr 172.21.15.105:0/843503022 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.209+0000 7f54c957a700 1 -- 172.21.15.105:0/843503022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c410b1b0 con 0x7f54c4108880 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.209+0000 7f54c957a700 1 --2- 172.21.15.105:0/843503022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c410ac70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54c00098a0 tx=0x7f54c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203ef08187446e4f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.209+0000 7f54bbfff700 1 -- 172.21.15.105:0/843503022 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c0004160 con 0x7f54c4108880 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.209+0000 7f54bbfff700 1 -- 172.21.15.105:0/843503022 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54c0004300 con 0x7f54c4108880 2024-03-26T17:29:35.098 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.209+0000 7f54bbfff700 1 -- 172.21.15.105:0/843503022 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c0011a50 con 0x7f54c4108880 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 -- 172.21.15.105:0/843503022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 msgr2=0x7f54c410ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 --2- 172.21.15.105:0/843503022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c410ac70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54c00098a0 tx=0x7f54c0009bb0 comp rx=0 tx=0).stop 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 -- 172.21.15.105:0/843503022 shutdown_connections 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 --2- 172.21.15.105:0/843503022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c410ac70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 -- 172.21.15.105:0/843503022 >> 172.21.15.105:0/843503022 conn(0x7f54c41022d0 msgr2=0x7f54c41046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 -- 172.21.15.105:0/843503022 shutdown_connections 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 -- 172.21.15.105:0/843503022 wait complete. 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 Processor -- start 2024-03-26T17:29:35.099 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 -- start start 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.210+0000 7f54cb7de700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54cb7de700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f54c410c1c0 con 0x7f54c4108880 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54c957a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54c957a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c419bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60580/0 (socket says 172.21.15.105:60580) 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54c957a700 1 -- 172.21.15.105:0/970842627 learned_addr learned my addr 172.21.15.105:0/970842627 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54c957a700 1 -- 172.21.15.105:0/970842627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c419c260 con 0x7f54c4108880 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54c957a700 1 --2- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c419bd20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54c0011a00 tx=0x7f54c0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54ba7fc700 1 -- 172.21.15.105:0/970842627 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c0003af0 con 0x7f54c4108880 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54ba7fc700 1 -- 172.21.15.105:0/970842627 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54c001a8e0 con 0x7f54c4108880 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c419c4f0 con 0x7f54c4108880 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54ba7fc700 1 -- 172.21.15.105:0/970842627 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54c0024840 con 0x7f54c4108880 2024-03-26T17:29:35.100 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.211+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c419f1d0 con 0x7f54c4108880 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.212+0000 7f54ba7fc700 1 -- 172.21.15.105:0/970842627 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7f54c0022020 con 0x7f54c4108880 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.212+0000 7f54ba7fc700 1 --2- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54b0039270 0x7f54b003b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.212+0000 7f54ba7fc700 1 -- 172.21.15.105:0/970842627 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f54c004d6d0 con 0x7f54c4108880 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.212+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c40639e0 con 0x7f54c4108880 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.212+0000 7f54c8d79700 1 --2- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54b0039270 0x7f54b003b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.213+0000 7f54c8d79700 1 --2- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54b0039270 0x7f54b003b730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f54b4006fd0 tx=0x7f54b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.217+0000 7f54ba7fc700 1 -- 172.21.15.105:0/970842627 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f54c0019c70 con 0x7f54c4108880 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.527+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f54c419f3a0 con 0x7f54c4108880 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.529+0000 7f54ba7fc700 1 -- 172.21.15.105:0/970842627 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f54c0032020 con 0x7f54c4108880 2024-03-26T17:29:35.101 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.531+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54b0039270 msgr2=0x7f54b003b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.531+0000 7f54cb7de700 1 --2- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54b0039270 0x7f54b003b730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f54b4006fd0 tx=0x7f54b4006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.531+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 msgr2=0x7f54c419bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.531+0000 7f54cb7de700 1 --2- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c419bd20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54c0011a00 tx=0x7f54c0011ae0 comp rx=0 tx=0).stop 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.532+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 shutdown_connections 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.532+0000 7f54cb7de700 1 --2- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54b0039270 0x7f54b003b730 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.532+0000 7f54cb7de700 1 --2- 172.21.15.105:0/970842627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54c4108880 0x7f54c419bd20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.532+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 >> 172.21.15.105:0/970842627 conn(0x7f54c41022d0 msgr2=0x7f54c4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.532+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 shutdown_connections 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr 2024-03-26T17:29:34.532+0000 7f54cb7de700 1 -- 172.21.15.105:0/970842627 wait complete. 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-26T17:29:35.102 INFO:teuthology.orchestra.run.smithi105.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout:Or, if you are only running a single cluster on this host: 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-26T17:29:35.103 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:35.104 INFO:teuthology.orchestra.run.smithi105.stdout: ceph telemetry on 2024-03-26T17:29:35.104 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:35.104 INFO:teuthology.orchestra.run.smithi105.stdout:For more information see: 2024-03-26T17:29:35.104 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:35.104 INFO:teuthology.orchestra.run.smithi105.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-26T17:29:35.104 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:35.104 INFO:teuthology.orchestra.run.smithi105.stdout:Bootstrap complete. 2024-03-26T17:29:35.137 INFO:tasks.cephadm:Fetching config... 2024-03-26T17:29:35.137 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:29:35.138 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-26T17:29:35.155 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-26T17:29:35.155 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:29:35.155 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-26T17:29:35.212 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-26T17:29:35.212 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:29:35.212 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-26T17:29:35.279 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-26T17:29:35.279 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:29:35.279 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-26T17:29:35.334 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-26T17:29:35.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCySkDhUiJMPvu2HWbpTbp7bjb1AFhF9lp1gjmX/7L4N2MUguyUOyr5EAm6i12VsM0WOjuTegG9D0muPbAF72dkjlyNBmGDJMD9+Rq4mYogl5JcuPQzBHtJNHufB2w0afmjNAogLGbt2Q4CvPaVpzG4aeSteQLBjRTtnWQOdS5Y1ePQRVIdBUpJaG+HZO0yoKtIHYp4aH/C9cFDAValjimJxoaE32jK0FyWDFigUoRgUbXuT/9+O/sAx+pJELZySijlzACBpNaewu1moYsbg/nylM0+FYv4xCvnzCqQoHnwZOhWZh7qQ6yvOtVrSOyBj/SGwO6zAAbo4LDKr3ryX2qrmTIo52C8Uxt2x/q1dvQ4ala3re8Q4aSn8+5fnpPVbGLZVCxKNc6v26m7udE+B6sgqkS8YeNMCwtVdHFmMoZfu0Aobw6pc19p5MZKujEnC7PwVO2Oz7Sx2BQ2dn3ioPwqtobldV9Z857bLISPHSE6kwhQ5kC35JMN1cBryvUgR88= ceph-15cee578-eb96-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T17:29:35.413 INFO:teuthology.orchestra.run.smithi105.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCySkDhUiJMPvu2HWbpTbp7bjb1AFhF9lp1gjmX/7L4N2MUguyUOyr5EAm6i12VsM0WOjuTegG9D0muPbAF72dkjlyNBmGDJMD9+Rq4mYogl5JcuPQzBHtJNHufB2w0afmjNAogLGbt2Q4CvPaVpzG4aeSteQLBjRTtnWQOdS5Y1ePQRVIdBUpJaG+HZO0yoKtIHYp4aH/C9cFDAValjimJxoaE32jK0FyWDFigUoRgUbXuT/9+O/sAx+pJELZySijlzACBpNaewu1moYsbg/nylM0+FYv4xCvnzCqQoHnwZOhWZh7qQ6yvOtVrSOyBj/SGwO6zAAbo4LDKr3ryX2qrmTIo52C8Uxt2x/q1dvQ4ala3re8Q4aSn8+5fnpPVbGLZVCxKNc6v26m7udE+B6sgqkS8YeNMCwtVdHFmMoZfu0Aobw6pc19p5MZKujEnC7PwVO2Oz7Sx2BQ2dn3ioPwqtobldV9Z857bLISPHSE6kwhQ5kC35JMN1cBryvUgR88= ceph-15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:29:35.434 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCySkDhUiJMPvu2HWbpTbp7bjb1AFhF9lp1gjmX/7L4N2MUguyUOyr5EAm6i12VsM0WOjuTegG9D0muPbAF72dkjlyNBmGDJMD9+Rq4mYogl5JcuPQzBHtJNHufB2w0afmjNAogLGbt2Q4CvPaVpzG4aeSteQLBjRTtnWQOdS5Y1ePQRVIdBUpJaG+HZO0yoKtIHYp4aH/C9cFDAValjimJxoaE32jK0FyWDFigUoRgUbXuT/9+O/sAx+pJELZySijlzACBpNaewu1moYsbg/nylM0+FYv4xCvnzCqQoHnwZOhWZh7qQ6yvOtVrSOyBj/SGwO6zAAbo4LDKr3ryX2qrmTIo52C8Uxt2x/q1dvQ4ala3re8Q4aSn8+5fnpPVbGLZVCxKNc6v26m7udE+B6sgqkS8YeNMCwtVdHFmMoZfu0Aobw6pc19p5MZKujEnC7PwVO2Oz7Sx2BQ2dn3ioPwqtobldV9Z857bLISPHSE6kwhQ5kC35JMN1cBryvUgR88= ceph-15cee578-eb96-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T17:29:35.483 INFO:teuthology.orchestra.run.smithi135.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCySkDhUiJMPvu2HWbpTbp7bjb1AFhF9lp1gjmX/7L4N2MUguyUOyr5EAm6i12VsM0WOjuTegG9D0muPbAF72dkjlyNBmGDJMD9+Rq4mYogl5JcuPQzBHtJNHufB2w0afmjNAogLGbt2Q4CvPaVpzG4aeSteQLBjRTtnWQOdS5Y1ePQRVIdBUpJaG+HZO0yoKtIHYp4aH/C9cFDAValjimJxoaE32jK0FyWDFigUoRgUbXuT/9+O/sAx+pJELZySijlzACBpNaewu1moYsbg/nylM0+FYv4xCvnzCqQoHnwZOhWZh7qQ6yvOtVrSOyBj/SGwO6zAAbo4LDKr3ryX2qrmTIo52C8Uxt2x/q1dvQ4ala3re8Q4aSn8+5fnpPVbGLZVCxKNc6v26m7udE+B6sgqkS8YeNMCwtVdHFmMoZfu0Aobw6pc19p5MZKujEnC7PwVO2Oz7Sx2BQ2dn3ioPwqtobldV9Z857bLISPHSE6kwhQ5kC35JMN1cBryvUgR88= ceph-15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:29:35.498 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCySkDhUiJMPvu2HWbpTbp7bjb1AFhF9lp1gjmX/7L4N2MUguyUOyr5EAm6i12VsM0WOjuTegG9D0muPbAF72dkjlyNBmGDJMD9+Rq4mYogl5JcuPQzBHtJNHufB2w0afmjNAogLGbt2Q4CvPaVpzG4aeSteQLBjRTtnWQOdS5Y1ePQRVIdBUpJaG+HZO0yoKtIHYp4aH/C9cFDAValjimJxoaE32jK0FyWDFigUoRgUbXuT/9+O/sAx+pJELZySijlzACBpNaewu1moYsbg/nylM0+FYv4xCvnzCqQoHnwZOhWZh7qQ6yvOtVrSOyBj/SGwO6zAAbo4LDKr3ryX2qrmTIo52C8Uxt2x/q1dvQ4ala3re8Q4aSn8+5fnpPVbGLZVCxKNc6v26m7udE+B6sgqkS8YeNMCwtVdHFmMoZfu0Aobw6pc19p5MZKujEnC7PwVO2Oz7Sx2BQ2dn3ioPwqtobldV9Z857bLISPHSE6kwhQ5kC35JMN1cBryvUgR88= ceph-15cee578-eb96-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T17:29:35.546 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCySkDhUiJMPvu2HWbpTbp7bjb1AFhF9lp1gjmX/7L4N2MUguyUOyr5EAm6i12VsM0WOjuTegG9D0muPbAF72dkjlyNBmGDJMD9+Rq4mYogl5JcuPQzBHtJNHufB2w0afmjNAogLGbt2Q4CvPaVpzG4aeSteQLBjRTtnWQOdS5Y1ePQRVIdBUpJaG+HZO0yoKtIHYp4aH/C9cFDAValjimJxoaE32jK0FyWDFigUoRgUbXuT/9+O/sAx+pJELZySijlzACBpNaewu1moYsbg/nylM0+FYv4xCvnzCqQoHnwZOhWZh7qQ6yvOtVrSOyBj/SGwO6zAAbo4LDKr3ryX2qrmTIo52C8Uxt2x/q1dvQ4ala3re8Q4aSn8+5fnpPVbGLZVCxKNc6v26m7udE+B6sgqkS8YeNMCwtVdHFmMoZfu0Aobw6pc19p5MZKujEnC7PwVO2Oz7Sx2BQ2dn3ioPwqtobldV9Z857bLISPHSE6kwhQ5kC35JMN1cBryvUgR88= ceph-15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:29:35.561 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-26T17:29:35.806 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:29:35.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:35 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/970842627' entity='client.admin' 2024-03-26T17:29:38.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.301+0000 7fe19c1ba700 1 -- 172.21.15.105:0/1226416807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 msgr2=0x7fe194101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.301+0000 7fe19c1ba700 1 --2- 172.21.15.105:0/1226416807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 0x7fe194101170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe184009910 tx=0x7fe184009c20 comp rx=0 tx=0).stop 2024-03-26T17:29:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.301+0000 7fe19c1ba700 1 -- 172.21.15.105:0/1226416807 shutdown_connections 2024-03-26T17:29:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.301+0000 7fe19c1ba700 1 --2- 172.21.15.105:0/1226416807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 0x7fe194101170 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.301+0000 7fe19c1ba700 1 -- 172.21.15.105:0/1226416807 >> 172.21.15.105:0/1226416807 conn(0x7fe1940fc310 msgr2=0x7fe1940fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.301+0000 7fe19c1ba700 1 -- 172.21.15.105:0/1226416807 shutdown_connections 2024-03-26T17:29:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.301+0000 7fe19c1ba700 1 -- 172.21.15.105:0/1226416807 wait complete. 2024-03-26T17:29:38.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe19c1ba700 1 Processor -- start 2024-03-26T17:29:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe19c1ba700 1 -- start start 2024-03-26T17:29:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe19c1ba700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 0x7fe1940fe180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe19c1ba700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe194106c20 con 0x7fe194100d50 2024-03-26T17:29:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe199f56700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 0x7fe1940fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:38.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe199f56700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 0x7fe1940fe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49916/0 (socket says 172.21.15.105:49916) 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe199f56700 1 -- 172.21.15.105:0/2554936371 learned_addr learned my addr 172.21.15.105:0/2554936371 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe199f56700 1 -- 172.21.15.105:0/2554936371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1940fe6c0 con 0x7fe194100d50 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.302+0000 7fe199f56700 1 --2- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 0x7fe1940fe180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe184005800 tx=0x7fe184009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.303+0000 7fe18affd700 1 -- 172.21.15.105:0/2554936371 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe184027070 con 0x7fe194100d50 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.303+0000 7fe18affd700 1 -- 172.21.15.105:0/2554936371 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe18400be10 con 0x7fe194100d50 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.303+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1940fc8b0 con 0x7fe194100d50 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.303+0000 7fe18affd700 1 -- 172.21.15.105:0/2554936371 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe18400f460 con 0x7fe194100d50 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.303+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1940fcdf0 con 0x7fe194100d50 2024-03-26T17:29:38.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.304+0000 7fe18affd700 1 -- 172.21.15.105:0/2554936371 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7fe18402a070 con 0x7fe194100d50 2024-03-26T17:29:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.304+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1941073a0 con 0x7fe194100d50 2024-03-26T17:29:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.304+0000 7fe18affd700 1 --2- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe180039280 0x7fe18003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.304+0000 7fe18affd700 1 -- 172.21.15.105:0/2554936371 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe18401e030 con 0x7fe194100d50 2024-03-26T17:29:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.304+0000 7fe199755700 1 --2- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe180039280 0x7fe18003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:38.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.305+0000 7fe199755700 1 --2- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe180039280 0x7fe18003b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe190006fd0 tx=0x7fe190006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:38.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.309+0000 7fe18affd700 1 -- 172.21.15.105:0/2554936371 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe184013070 con 0x7fe194100d50 2024-03-26T17:29:38.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.514+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fe194190970 con 0x7fe194100d50 2024-03-26T17:29:38.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.516+0000 7fe18affd700 1 -- 172.21.15.105:0/2554936371 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fe184031070 con 0x7fe194100d50 2024-03-26T17:29:38.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe180039280 msgr2=0x7fe18003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 --2- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe180039280 0x7fe18003b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe190006fd0 tx=0x7fe190006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 msgr2=0x7fe1940fe180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 --2- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 0x7fe1940fe180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe184005800 tx=0x7fe184009f60 comp rx=0 tx=0).stop 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 shutdown_connections 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 --2- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe180039280 0x7fe18003b740 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 --2- 172.21.15.105:0/2554936371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe194100d50 0x7fe1940fe180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 >> 172.21.15.105:0/2554936371 conn(0x7fe1940fc310 msgr2=0x7fe194069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 shutdown_connections 2024-03-26T17:29:38.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:38.519+0000 7fe19c1ba700 1 -- 172.21.15.105:0/2554936371 wait complete. 2024-03-26T17:29:39.105 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-26T17:29:39.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-26T17:29:39.674 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:29:39.797 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:39 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2554936371' entity='client.admin' 2024-03-26T17:29:40.941 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:40.941 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:40.941 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:29:40.941 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:40.941 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:40.941 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.160+0000 7fa50e72f700 1 -- 172.21.15.105:0/3581811552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 msgr2=0x7fa5081010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.160+0000 7fa50e72f700 1 --2- 172.21.15.105:0/3581811552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 0x7fa5081010f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009910 tx=0x7fa4fc009c20 comp rx=0 tx=0).stop 2024-03-26T17:29:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.160+0000 7fa50e72f700 1 -- 172.21.15.105:0/3581811552 shutdown_connections 2024-03-26T17:29:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.160+0000 7fa50e72f700 1 --2- 172.21.15.105:0/3581811552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 0x7fa5081010f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.160+0000 7fa50e72f700 1 -- 172.21.15.105:0/3581811552 >> 172.21.15.105:0/3581811552 conn(0x7fa5080fc2b0 msgr2=0x7fa5080fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.160+0000 7fa50e72f700 1 -- 172.21.15.105:0/3581811552 shutdown_connections 2024-03-26T17:29:41.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.160+0000 7fa50e72f700 1 -- 172.21.15.105:0/3581811552 wait complete. 2024-03-26T17:29:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.161+0000 7fa50e72f700 1 Processor -- start 2024-03-26T17:29:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.161+0000 7fa50e72f700 1 -- start start 2024-03-26T17:29:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.161+0000 7fa50e72f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 0x7fa50806aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.161+0000 7fa50e72f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa508106c10 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.161+0000 7fa507fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 0x7fa50806aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.161+0000 7fa507fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 0x7fa50806aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49922/0 (socket says 172.21.15.105:49922) 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.161+0000 7fa507fff700 1 -- 172.21.15.105:0/3780996385 learned_addr learned my addr 172.21.15.105:0/3780996385 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.161+0000 7fa507fff700 1 -- 172.21.15.105:0/3780996385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa508197f90 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.162+0000 7fa507fff700 1 --2- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 0x7fa50806aac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc005950 tx=0x7fa4fc004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.162+0000 7fa5057fa700 1 -- 172.21.15.105:0/3780996385 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4fc026070 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.162+0000 7fa5057fa700 1 -- 172.21.15.105:0/3780996385 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4fc00bc00 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.162+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50819c5a0 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.162+0000 7fa5057fa700 1 -- 172.21.15.105:0/3780996385 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4fc0039a0 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.162+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50819c940 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.163+0000 7fa5057fa700 1 -- 172.21.15.105:0/3780996385 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7fa4fc029070 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.163+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa508107390 con 0x7fa508100cd0 2024-03-26T17:29:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.163+0000 7fa5057fa700 1 --2- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4f0039220 0x7fa4f003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:41.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.163+0000 7fa5057fa700 1 -- 172.21.15.105:0/3780996385 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa4fc01e070 con 0x7fa508100cd0 2024-03-26T17:29:41.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.163+0000 7fa5077fe700 1 --2- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4f0039220 0x7fa4f003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:41.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.163+0000 7fa5077fe700 1 --2- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4f0039220 0x7fa4f003b6e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8006fd0 tx=0x7fa4f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.168+0000 7fa5057fa700 1 -- 172.21.15.105:0/3780996385 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa4fc013070 con 0x7fa508100cd0 2024-03-26T17:29:41.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.365+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa50802dd70 con 0x7fa4f0039220 2024-03-26T17:29:41.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.369+0000 7fa5057fa700 1 -- 172.21.15.105:0/3780996385 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa50802dd70 con 0x7fa4f0039220 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4f0039220 msgr2=0x7fa4f003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 --2- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4f0039220 0x7fa4f003b6e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8006fd0 tx=0x7fa4f8006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 msgr2=0x7fa50806aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 --2- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 0x7fa50806aac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc005950 tx=0x7fa4fc004fa0 comp rx=0 tx=0).stop 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 shutdown_connections 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 --2- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4f0039220 0x7fa4f003b6e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 --2- 172.21.15.105:0/3780996385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa508100cd0 0x7fa50806aac0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 >> 172.21.15.105:0/3780996385 conn(0x7fa5080fc2b0 msgr2=0x7fa5080fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 shutdown_connections 2024-03-26T17:29:41.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:41.371+0000 7fa50e72f700 1 -- 172.21.15.105:0/3780996385 wait complete. 2024-03-26T17:29:42.048 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi135 2024-03-26T17:29:42.049 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:29:42.049 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.conf 2024-03-26T17:29:42.066 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:29:42.067 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:29:42.125 INFO:tasks.cephadm:Adding host smithi135 to orchestrator... 2024-03-26T17:29:42.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch host add smithi135 2024-03-26T17:29:42.367 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:29:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: Updating smithi105:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:42.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:43.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:43 smithi105 ceph-mon[32280]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:29:43.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:43 smithi105 ceph-mon[32280]: Updating smithi105:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:29:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.746+0000 7f8a69fcd700 1 -- 172.21.15.105:0/1191751787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 msgr2=0x7f8a64103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.746+0000 7f8a69fcd700 1 --2- 172.21.15.105:0/1191751787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 0x7f8a64103320 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009910 tx=0x7f8a54009c20 comp rx=0 tx=0).stop 2024-03-26T17:29:43.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.746+0000 7f8a69fcd700 1 -- 172.21.15.105:0/1191751787 shutdown_connections 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.746+0000 7f8a69fcd700 1 --2- 172.21.15.105:0/1191751787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 0x7f8a64103320 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.746+0000 7f8a69fcd700 1 -- 172.21.15.105:0/1191751787 >> 172.21.15.105:0/1191751787 conn(0x7f8a640fc7b0 msgr2=0x7f8a640febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.746+0000 7f8a69fcd700 1 -- 172.21.15.105:0/1191751787 shutdown_connections 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.746+0000 7f8a69fcd700 1 -- 172.21.15.105:0/1191751787 wait complete. 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.746+0000 7f8a69fcd700 1 Processor -- start 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a69fcd700 1 -- start start 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a69fcd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 0x7f8a64100ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a69fcd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6410c2c0 con 0x7f8a64102f40 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a637fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 0x7f8a64100ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a637fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 0x7f8a64100ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49936/0 (socket says 172.21.15.105:49936) 2024-03-26T17:29:43.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a637fe700 1 -- 172.21.15.105:0/2242837295 learned_addr learned my addr 172.21.15.105:0/2242837295 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a637fe700 1 -- 172.21.15.105:0/2242837295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a64102a90 con 0x7f8a64102f40 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a637fe700 1 --2- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 0x7f8a64100ba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a540056d0 tx=0x7f8a540057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.747+0000 7f8a60ff9700 1 -- 172.21.15.105:0/2242837295 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a54026070 con 0x7f8a64102f40 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.748+0000 7f8a60ff9700 1 -- 172.21.15.105:0/2242837295 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a54004370 con 0x7f8a64102f40 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.748+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a641010e0 con 0x7f8a64102f40 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.748+0000 7f8a60ff9700 1 -- 172.21.15.105:0/2242837295 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a540039b0 con 0x7f8a64102f40 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.748+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a641015a0 con 0x7f8a64102f40 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.748+0000 7f8a60ff9700 1 -- 172.21.15.105:0/2242837295 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7f8a54029070 con 0x7f8a64102f40 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.749+0000 7f8a60ff9700 1 --2- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a4c039280 0x7f8a4c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:43.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.749+0000 7f8a60ff9700 1 -- 172.21.15.105:0/2242837295 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a5401e070 con 0x7f8a64102f40 2024-03-26T17:29:43.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.749+0000 7f8a62ffd700 1 --2- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a4c039280 0x7f8a4c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:43.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.749+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6406aac0 con 0x7f8a64102f40 2024-03-26T17:29:43.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.749+0000 7f8a62ffd700 1 --2- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a4c039280 0x7f8a4c03b740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:43.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.754+0000 7f8a60ff9700 1 -- 172.21.15.105:0/2242837295 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a54030020 con 0x7f8a64102f40 2024-03-26T17:29:43.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:43.953+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}) -- 0x7f8a64101890 con 0x7f8a4c039280 2024-03-26T17:29:45.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:44 smithi105 ceph-mon[32280]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:45.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:44 smithi105 ceph-mon[32280]: Deploying cephadm binary to smithi135 2024-03-26T17:29:47.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:47 smithi105 ceph-mon[32280]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:29:48.264 INFO:teuthology.orchestra.run.smithi105.stdout:Added host 'smithi135' with addr '172.21.15.135' 2024-03-26T17:29:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.269+0000 7f8a60ff9700 1 -- 172.21.15.105:0/2242837295 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8a64101890 con 0x7f8a4c039280 2024-03-26T17:29:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.271+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a4c039280 msgr2=0x7f8a4c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.271+0000 7f8a69fcd700 1 --2- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a4c039280 0x7f8a4c03b740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.271+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 msgr2=0x7f8a64100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.271+0000 7f8a69fcd700 1 --2- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 0x7f8a64100ba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a540056d0 tx=0x7f8a540057b0 comp rx=0 tx=0).stop 2024-03-26T17:29:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.272+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 shutdown_connections 2024-03-26T17:29:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.272+0000 7f8a69fcd700 1 --2- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a4c039280 0x7f8a4c03b740 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:48.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.272+0000 7f8a69fcd700 1 --2- 172.21.15.105:0/2242837295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a64102f40 0x7f8a64100ba0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:48.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.272+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 >> 172.21.15.105:0/2242837295 conn(0x7f8a640fc7b0 msgr2=0x7f8a64103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:48.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.272+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 shutdown_connections 2024-03-26T17:29:48.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:48.272+0000 7f8a69fcd700 1 -- 172.21.15.105:0/2242837295 wait complete. 2024-03-26T17:29:48.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T17:29:49.139 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:29:49.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:49.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:49 smithi105 ceph-mon[32280]: Added host smithi135 2024-03-26T17:29:49.497 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:49.497 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:49 smithi105 ceph-mon[32280]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:29:49.497 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 -- 172.21.15.105:0/2506172319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 msgr2=0x7f86e01023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 --2- 172.21.15.105:0/2506172319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 0x7f86e01023b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009910 tx=0x7f86d4009c20 comp rx=0 tx=0).stop 2024-03-26T17:29:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 -- 172.21.15.105:0/2506172319 shutdown_connections 2024-03-26T17:29:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 --2- 172.21.15.105:0/2506172319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 0x7f86e01023b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:50.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 -- 172.21.15.105:0/2506172319 >> 172.21.15.105:0/2506172319 conn(0x7f86e00fd880 msgr2=0x7f86e00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 -- 172.21.15.105:0/2506172319 shutdown_connections 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 -- 172.21.15.105:0/2506172319 wait complete. 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 Processor -- start 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.476+0000 7f86e7017700 1 -- start start 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e7017700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 0x7f86e019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e7017700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86e010c250 con 0x7f86e0101fd0 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e4db3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 0x7f86e019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e4db3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 0x7f86e019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43856/0 (socket says 172.21.15.105:43856) 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e4db3700 1 -- 172.21.15.105:0/2454807960 learned_addr learned my addr 172.21.15.105:0/2454807960 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e4db3700 1 -- 172.21.15.105:0/2454807960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e019d970 con 0x7f86e0101fd0 2024-03-26T17:29:50.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e4db3700 1 --2- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 0x7f86e019d430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86d40056d0 tx=0x7f86d40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:50.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86ddffb700 1 -- 172.21.15.105:0/2454807960 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86d4026070 con 0x7f86e0101fd0 2024-03-26T17:29:50.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86ddffb700 1 -- 172.21.15.105:0/2454807960 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86d400bba0 con 0x7f86e0101fd0 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86ddffb700 1 -- 172.21.15.105:0/2454807960 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86d40037f0 con 0x7f86e0101fd0 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e019dc60 con 0x7f86e0101fd0 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.477+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e01975f0 con 0x7f86e0101fd0 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.478+0000 7f86ddffb700 1 -- 172.21.15.105:0/2454807960 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7f86d4029070 con 0x7f86e0101fd0 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.478+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e01034b0 con 0x7f86e0101fd0 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.478+0000 7f86ddffb700 1 --2- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86d0039220 0x7f86d003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.478+0000 7f86ddffb700 1 -- 172.21.15.105:0/2454807960 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f86d401e070 con 0x7f86e0101fd0 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.478+0000 7f86dffff700 1 --2- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86d0039220 0x7f86d003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:50.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.479+0000 7f86dffff700 1 --2- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86d0039220 0x7f86d003b6e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f86cc006fd0 tx=0x7f86cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:50.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.483+0000 7f86ddffb700 1 -- 172.21.15.105:0/2454807960 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86d4013070 con 0x7f86e0101fd0 2024-03-26T17:29:50.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:50.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.685+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86e0198b20 con 0x7f86d0039220 2024-03-26T17:29:50.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.687+0000 7f86ddffb700 1 -- 172.21.15.105:0/2454807960 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f86e0198b20 con 0x7f86d0039220 2024-03-26T17:29:50.681 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:50.681 INFO:teuthology.orchestra.run.smithi105.stdout:[{"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}] 2024-03-26T17:29:50.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86d0039220 msgr2=0x7f86d003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:50.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 --2- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86d0039220 0x7f86d003b6e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f86cc006fd0 tx=0x7f86cc006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:50.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 msgr2=0x7f86e019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:50.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 --2- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 0x7f86e019d430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f86d40056d0 tx=0x7f86d40057b0 comp rx=0 tx=0).stop 2024-03-26T17:29:50.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 shutdown_connections 2024-03-26T17:29:50.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 --2- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86d0039220 0x7f86d003b6e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:50.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 --2- 172.21.15.105:0/2454807960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86e0101fd0 0x7f86e019d430 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:50.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 >> 172.21.15.105:0/2454807960 conn(0x7f86e00fd880 msgr2=0x7f86e01039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:50.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 shutdown_connections 2024-03-26T17:29:50.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:50.689+0000 7f86e7017700 1 -- 172.21.15.105:0/2454807960 wait complete. 2024-03-26T17:29:51.298 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi144 2024-03-26T17:29:51.299 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:29:51.299 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.conf 2024-03-26T17:29:51.318 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:29:51.318 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:29:51.376 INFO:tasks.cephadm:Adding host smithi144 to orchestrator... 2024-03-26T17:29:51.376 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch host add smithi144 2024-03-26T17:29:51.611 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:29:51.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:51 smithi105 ceph-mon[32280]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:29:51.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:51 smithi105 ceph-mon[32280]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:29:51.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.106+0000 7f3f5035d700 1 -- 172.21.15.105:0/3512475339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 msgr2=0x7f3f481023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.106+0000 7f3f5035d700 1 --2- 172.21.15.105:0/3512475339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 0x7f3f481023e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009910 tx=0x7f3f44009c20 comp rx=0 tx=0).stop 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.106+0000 7f3f5035d700 1 -- 172.21.15.105:0/3512475339 shutdown_connections 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.106+0000 7f3f5035d700 1 --2- 172.21.15.105:0/3512475339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 0x7f3f481023e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.106+0000 7f3f5035d700 1 -- 172.21.15.105:0/3512475339 >> 172.21.15.105:0/3512475339 conn(0x7f3f480fd890 msgr2=0x7f3f480ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.106+0000 7f3f5035d700 1 -- 172.21.15.105:0/3512475339 shutdown_connections 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.106+0000 7f3f5035d700 1 -- 172.21.15.105:0/3512475339 wait complete. 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.107+0000 7f3f5035d700 1 Processor -- start 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.107+0000 7f3f5035d700 1 -- start start 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.107+0000 7f3f5035d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 0x7f3f4819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:53.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.107+0000 7f3f5035d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4810c280 con 0x7f3f48102000 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.107+0000 7f3f4e0f9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 0x7f3f4819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.107+0000 7f3f4e0f9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 0x7f3f4819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43878/0 (socket says 172.21.15.105:43878) 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.107+0000 7f3f4e0f9700 1 -- 172.21.15.105:0/2819918267 learned_addr learned my addr 172.21.15.105:0/2819918267 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.108+0000 7f3f4e0f9700 1 -- 172.21.15.105:0/2819918267 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4819d990 con 0x7f3f48102000 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.108+0000 7f3f4e0f9700 1 --2- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 0x7f3f4819d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006010 tx=0x7f3f44005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.108+0000 7f3f3f7fe700 1 -- 172.21.15.105:0/2819918267 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f44026070 con 0x7f3f48102000 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.108+0000 7f3f3f7fe700 1 -- 172.21.15.105:0/2819918267 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f4400bba0 con 0x7f3f48102000 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.108+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4819dc80 con 0x7f3f48102000 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.108+0000 7f3f3f7fe700 1 -- 172.21.15.105:0/2819918267 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f440038a0 con 0x7f3f48102000 2024-03-26T17:29:53.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.108+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f48197610 con 0x7f3f48102000 2024-03-26T17:29:53.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.109+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f481034e0 con 0x7f3f48102000 2024-03-26T17:29:53.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.109+0000 7f3f3f7fe700 1 -- 172.21.15.105:0/2819918267 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7f3f44029070 con 0x7f3f48102000 2024-03-26T17:29:53.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.109+0000 7f3f3f7fe700 1 --2- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f34039220 0x7f3f3403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:53.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.109+0000 7f3f3f7fe700 1 -- 172.21.15.105:0/2819918267 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f4401e070 con 0x7f3f48102000 2024-03-26T17:29:53.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.109+0000 7f3f4d8f8700 1 --2- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f34039220 0x7f3f3403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:53.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.109+0000 7f3f4d8f8700 1 --2- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f34039220 0x7f3f3403b6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:53.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.114+0000 7f3f3f7fe700 1 -- 172.21.15.105:0/2819918267 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3f44013070 con 0x7f3f48102000 2024-03-26T17:29:53.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:53.322+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}) -- 0x7f3f48198b40 con 0x7f3f34039220 2024-03-26T17:29:53.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:53 smithi105 ceph-mon[32280]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:29:54.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:54 smithi105 ceph-mon[32280]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:29:54.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:54.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:54.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:54.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:54.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:29:54.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:29:54.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:29:55.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:55 smithi105 ceph-mon[32280]: Updating smithi135:/etc/ceph/ceph.conf 2024-03-26T17:29:55.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:55 smithi105 ceph-mon[32280]: Deploying cephadm binary to smithi144 2024-03-26T17:29:55.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:55 smithi105 ceph-mon[32280]: Updating smithi135:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:29:55.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:55 smithi105 ceph-mon[32280]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:29:55.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:55 smithi105 ceph-mon[32280]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:29:57.510 INFO:teuthology.orchestra.run.smithi105.stdout:Added host 'smithi144' with addr '172.21.15.144' 2024-03-26T17:29:57.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.514+0000 7f3f3f7fe700 1 -- 172.21.15.105:0/2819918267 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3f48198b40 con 0x7f3f34039220 2024-03-26T17:29:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.516+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f34039220 msgr2=0x7f3f3403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.516+0000 7f3f5035d700 1 --2- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f34039220 0x7f3f3403b6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.516+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 msgr2=0x7f3f4819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.516+0000 7f3f5035d700 1 --2- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 0x7f3f4819d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006010 tx=0x7f3f44005780 comp rx=0 tx=0).stop 2024-03-26T17:29:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.516+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 shutdown_connections 2024-03-26T17:29:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.516+0000 7f3f5035d700 1 --2- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f34039220 0x7f3f3403b6e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.516+0000 7f3f5035d700 1 --2- 172.21.15.105:0/2819918267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f48102000 0x7f3f4819d450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:57.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.516+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 >> 172.21.15.105:0/2819918267 conn(0x7f3f480fd890 msgr2=0x7f3f48109b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:57.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.517+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 shutdown_connections 2024-03-26T17:29:57.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:57.517+0000 7f3f5035d700 1 -- 172.21.15.105:0/2819918267 wait complete. 2024-03-26T17:29:57.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:57 smithi105 ceph-mon[32280]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:29:57.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:57.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:57.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:57.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:57.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:29:58.096 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T17:29:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:29:58.722 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:58 smithi105 ceph-mon[32280]: Updating smithi135:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:29:58.722 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:58 smithi105 ceph-mon[32280]: Added host smithi144 2024-03-26T17:29:58.722 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:58.722 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:29:59.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:29:59 smithi105 ceph-mon[32280]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:29:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.751+0000 7ff433140700 1 -- 172.21.15.105:0/1368651157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 msgr2=0x7ff42c103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.751+0000 7ff433140700 1 --2- 172.21.15.105:0/1368651157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 0x7ff42c103390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff42400b0a0 tx=0x7ff42400b3b0 comp rx=0 tx=0).stop 2024-03-26T17:29:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.751+0000 7ff433140700 1 -- 172.21.15.105:0/1368651157 shutdown_connections 2024-03-26T17:29:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.751+0000 7ff433140700 1 --2- 172.21.15.105:0/1368651157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 0x7ff42c103390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.751+0000 7ff433140700 1 -- 172.21.15.105:0/1368651157 >> 172.21.15.105:0/1368651157 conn(0x7ff42c0fc780 msgr2=0x7ff42c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.751+0000 7ff433140700 1 -- 172.21.15.105:0/1368651157 shutdown_connections 2024-03-26T17:29:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.751+0000 7ff433140700 1 -- 172.21.15.105:0/1368651157 wait complete. 2024-03-26T17:29:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.752+0000 7ff433140700 1 Processor -- start 2024-03-26T17:29:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.752+0000 7ff433140700 1 -- start start 2024-03-26T17:29:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.752+0000 7ff433140700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 0x7ff42c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.752+0000 7ff433140700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c10c2e0 con 0x7ff42c102fb0 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.752+0000 7ff430edc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 0x7ff42c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.752+0000 7ff430edc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 0x7ff42c19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59810/0 (socket says 172.21.15.105:59810) 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.752+0000 7ff430edc700 1 -- 172.21.15.105:0/313385430 learned_addr learned my addr 172.21.15.105:0/313385430 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.753+0000 7ff430edc700 1 -- 172.21.15.105:0/313385430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c19fba0 con 0x7ff42c102fb0 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.753+0000 7ff430edc700 1 --2- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 0x7ff42c19f660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff4240076a0 tx=0x7ff424009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.753+0000 7ff429ffb700 1 -- 172.21.15.105:0/313385430 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff424017040 con 0x7ff42c102fb0 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.753+0000 7ff429ffb700 1 -- 172.21.15.105:0/313385430 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff424004040 con 0x7ff42c102fb0 2024-03-26T17:29:59.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.753+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c19fe30 con 0x7ff42c102fb0 2024-03-26T17:29:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.753+0000 7ff429ffb700 1 -- 172.21.15.105:0/313385430 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff42401d450 con 0x7ff42c102fb0 2024-03-26T17:29:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.753+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c199830 con 0x7ff42c102fb0 2024-03-26T17:29:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.754+0000 7ff429ffb700 1 -- 172.21.15.105:0/313385430 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7ff424024020 con 0x7ff42c102fb0 2024-03-26T17:29:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.754+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c06aa20 con 0x7ff42c102fb0 2024-03-26T17:29:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.754+0000 7ff429ffb700 1 --2- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff418039280 0x7ff41803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:29:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.754+0000 7ff429ffb700 1 -- 172.21.15.105:0/313385430 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff424022020 con 0x7ff42c102fb0 2024-03-26T17:29:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.754+0000 7ff42bfff700 1 --2- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff418039280 0x7ff41803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:29:59.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.755+0000 7ff42bfff700 1 --2- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff418039280 0x7ff41803b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff41c006fd0 tx=0x7ff41c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:29:59.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.759+0000 7ff429ffb700 1 -- 172.21.15.105:0/313385430 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff42402b020 con 0x7ff42c102fb0 2024-03-26T17:29:59.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.958+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff42c19ad10 con 0x7ff418039280 2024-03-26T17:29:59.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.960+0000 7ff429ffb700 1 -- 172.21.15.105:0/313385430 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7ff42c19ad10 con 0x7ff418039280 2024-03-26T17:29:59.955 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:29:59.955 INFO:teuthology.orchestra.run.smithi105.stdout:[{"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}, {"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}] 2024-03-26T17:29:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff418039280 msgr2=0x7ff41803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 --2- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff418039280 0x7ff41803b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff41c006fd0 tx=0x7ff41c006e40 comp rx=0 tx=0).stop 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 msgr2=0x7ff42c19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 --2- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 0x7ff42c19f660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff4240076a0 tx=0x7ff424009800 comp rx=0 tx=0).stop 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 shutdown_connections 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 --2- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff418039280 0x7ff41803b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 --2- 172.21.15.105:0/313385430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c102fb0 0x7ff42c19f660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 >> 172.21.15.105:0/313385430 conn(0x7ff42c0fc780 msgr2=0x7ff42c108170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 shutdown_connections 2024-03-26T17:29:59.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:29:59.962+0000 7ff433140700 1 -- 172.21.15.105:0/313385430 wait complete. 2024-03-26T17:30:00.532 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-26T17:30:00.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-26T17:30:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:30:01.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:01 smithi105 ceph-mon[32280]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:30:01.216 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:01.216 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:01 smithi105 ceph-mon[32280]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:02.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.190+0000 7f532d6a8700 1 -- 172.21.15.105:0/8043202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 msgr2=0x7f53281023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:02.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.190+0000 7f532d6a8700 1 --2- 172.21.15.105:0/8043202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 0x7f53281023b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5318009910 tx=0x7f5318009c20 comp rx=0 tx=0).stop 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 -- 172.21.15.105:0/8043202 shutdown_connections 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 --2- 172.21.15.105:0/8043202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 0x7f53281023b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 -- 172.21.15.105:0/8043202 >> 172.21.15.105:0/8043202 conn(0x7f53280fd880 msgr2=0x7f53280ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 -- 172.21.15.105:0/8043202 shutdown_connections 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 -- 172.21.15.105:0/8043202 wait complete. 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 Processor -- start 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 -- start start 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 0x7f53280746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.191+0000 7f532d6a8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f532810c250 con 0x7f5328101fd0 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f5326ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 0x7f53280746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f5326ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 0x7f53280746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59838/0 (socket says 172.21.15.105:59838) 2024-03-26T17:30:02.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f5326ffd700 1 -- 172.21.15.105:0/1253086667 learned_addr learned my addr 172.21.15.105:0/1253086667 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:30:02.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f5326ffd700 1 -- 172.21.15.105:0/1253086667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53280765a0 con 0x7f5328101fd0 2024-03-26T17:30:02.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f5326ffd700 1 --2- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 0x7f53280746b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53180056d0 tx=0x7f53180057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:02.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f530ffff700 1 -- 172.21.15.105:0/1253086667 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5318026070 con 0x7f5328101fd0 2024-03-26T17:30:02.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5328074c50 con 0x7f5328101fd0 2024-03-26T17:30:02.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f530ffff700 1 -- 172.21.15.105:0/1253086667 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f531800bc00 con 0x7f5328101fd0 2024-03-26T17:30:02.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f530ffff700 1 -- 172.21.15.105:0/1253086667 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53180039b0 con 0x7f5328101fd0 2024-03-26T17:30:02.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.192+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5328075110 con 0x7f5328101fd0 2024-03-26T17:30:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.193+0000 7f530ffff700 1 -- 172.21.15.105:0/1253086667 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7f5318029070 con 0x7f5328101fd0 2024-03-26T17:30:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.193+0000 7f530ffff700 1 --2- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5310039220 0x7f531003b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.193+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53281034b0 con 0x7f5328101fd0 2024-03-26T17:30:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.193+0000 7f530ffff700 1 -- 172.21.15.105:0/1253086667 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f531801e070 con 0x7f5328101fd0 2024-03-26T17:30:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.193+0000 7f53267fc700 1 --2- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5310039220 0x7f531003b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:02.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.194+0000 7f53267fc700 1 --2- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5310039220 0x7f531003b6e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f531c006fd0 tx=0x7f531c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:02.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.198+0000 7f530ffff700 1 -- 172.21.15.105:0/1253086667 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5318013070 con 0x7f5328101fd0 2024-03-26T17:30:02.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.397+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5328075c80 con 0x7f5328101fd0 2024-03-26T17:30:02.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.448+0000 7f530ffff700 1 -- 172.21.15.105:0/1253086667 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f5318030090 con 0x7f5328101fd0 2024-03-26T17:30:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.450+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5310039220 msgr2=0x7f531003b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.450+0000 7f532d6a8700 1 --2- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5310039220 0x7f531003b6e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f531c006fd0 tx=0x7f531c006e40 comp rx=0 tx=0).stop 2024-03-26T17:30:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.450+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 msgr2=0x7f53280746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.450+0000 7f532d6a8700 1 --2- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 0x7f53280746b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53180056d0 tx=0x7f53180057b0 comp rx=0 tx=0).stop 2024-03-26T17:30:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.450+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 shutdown_connections 2024-03-26T17:30:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.450+0000 7f532d6a8700 1 --2- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5310039220 0x7f531003b6e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:02.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.450+0000 7f532d6a8700 1 --2- 172.21.15.105:0/1253086667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328101fd0 0x7f53280746b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.450+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 >> 172.21.15.105:0/1253086667 conn(0x7f53280fd880 msgr2=0x7f53281039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.451+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 shutdown_connections 2024-03-26T17:30:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:02.451+0000 7f532d6a8700 1 -- 172.21.15.105:0/1253086667 wait complete. 2024-03-26T17:30:02.447 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2024-03-26T17:30:02.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:02 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1253086667' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T17:30:02.988 INFO:tasks.cephadm:Adding mon.a on smithi105 2024-03-26T17:30:02.989 INFO:tasks.cephadm:Adding mon.b on smithi135 2024-03-26T17:30:02.989 INFO:tasks.cephadm:Adding mon.c on smithi144 2024-03-26T17:30:02.989 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi105:172.21.15.105=a;smithi135:172.21.15.135=b;smithi144:172.21.15.144=c' 2024-03-26T17:30:03.252 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-26T17:30:03.297 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1253086667' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:03.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:04.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.766+0000 7ff50f7ed700 1 -- 172.21.15.144:0/2440753016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 msgr2=0x7ff508100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.766+0000 7ff50f7ed700 1 --2- 172.21.15.144:0/2440753016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 0x7ff508100a70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff504009910 tx=0x7ff504009c20 comp rx=0 tx=0).stop 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.766+0000 7ff50f7ed700 1 -- 172.21.15.144:0/2440753016 shutdown_connections 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.766+0000 7ff50f7ed700 1 --2- 172.21.15.144:0/2440753016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 0x7ff508100a70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.766+0000 7ff50f7ed700 1 -- 172.21.15.144:0/2440753016 >> 172.21.15.144:0/2440753016 conn(0x7ff5080783d0 msgr2=0x7ff5080787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.766+0000 7ff50f7ed700 1 -- 172.21.15.144:0/2440753016 shutdown_connections 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.766+0000 7ff50f7ed700 1 -- 172.21.15.144:0/2440753016 wait complete. 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.766+0000 7ff50f7ed700 1 Processor -- start 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.767+0000 7ff50f7ed700 1 -- start start 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.767+0000 7ff50f7ed700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 0x7ff508101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.767+0000 7ff50f7ed700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff50810c300 con 0x7ff508100690 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.767+0000 7ff50d589700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 0x7ff508101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:04.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.767+0000 7ff50d589700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 0x7ff508101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.144:52252/0 (socket says 172.21.15.144:52252) 2024-03-26T17:30:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.767+0000 7ff50d589700 1 -- 172.21.15.144:0/3739886906 learned_addr learned my addr 172.21.15.144:0/3739886906 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:30:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.767+0000 7ff50d589700 1 -- 172.21.15.144:0/3739886906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff508103500 con 0x7ff508100690 2024-03-26T17:30:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.767+0000 7ff50d589700 1 --2- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 0x7ff508101610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff504006010 tx=0x7ff504005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.768+0000 7ff4fe7fc700 1 -- 172.21.15.144:0/3739886906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff504026070 con 0x7ff508100690 2024-03-26T17:30:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.768+0000 7ff4fe7fc700 1 -- 172.21.15.144:0/3739886906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff50400bc00 con 0x7ff508100690 2024-03-26T17:30:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.768+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff508101bb0 con 0x7ff508100690 2024-03-26T17:30:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.768+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff508102070 con 0x7ff508100690 2024-03-26T17:30:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.768+0000 7ff4fe7fc700 1 -- 172.21.15.144:0/3739886906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff504003930 con 0x7ff508100690 2024-03-26T17:30:04.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.768+0000 7ff4fe7fc700 1 -- 172.21.15.144:0/3739886906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7ff504029070 con 0x7ff508100690 2024-03-26T17:30:04.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.768+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff508069f80 con 0x7ff508100690 2024-03-26T17:30:04.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.769+0000 7ff4fe7fc700 1 --2- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4f4039280 0x7ff4f403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:04.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.769+0000 7ff4fe7fc700 1 -- 172.21.15.144:0/3739886906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff50401e070 con 0x7ff508100690 2024-03-26T17:30:04.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.769+0000 7ff50cd88700 1 --2- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4f4039280 0x7ff4f403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:04.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.769+0000 7ff50cd88700 1 --2- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4f4039280 0x7ff4f403b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8006fd0 tx=0x7ff4f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:04.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.774+0000 7ff4fe7fc700 1 -- 172.21.15.144:0/3739886906 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff504013070 con 0x7ff508100690 2024-03-26T17:30:04.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:04 smithi105 ceph-mon[32280]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T17:30:04.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:04 smithi105 ceph-mon[32280]: Updating smithi144:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:04.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:04 smithi105 ceph-mon[32280]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:30:04.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:04 smithi105 ceph-mon[32280]: Updating smithi144:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:30:04.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:04.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:04.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:04.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.979+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi105:172.21.15.105=a;smithi135:172.21.15.135=b;smithi144:172.21.15.144=c", "target": ["mon-mgr", ""]}) -- 0x7ff508102bb0 con 0x7ff4f4039280 2024-03-26T17:30:04.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.985+0000 7ff4fe7fc700 1 -- 172.21.15.144:0/3739886906 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff508102bb0 con 0x7ff4f4039280 2024-03-26T17:30:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mon update... 2024-03-26T17:30:04.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4f4039280 msgr2=0x7ff4f403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:04.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 --2- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4f4039280 0x7ff4f403b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8006fd0 tx=0x7ff4f8006e40 comp rx=0 tx=0).stop 2024-03-26T17:30:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 msgr2=0x7ff508101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 --2- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 0x7ff508101610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff504006010 tx=0x7ff504005780 comp rx=0 tx=0).stop 2024-03-26T17:30:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 shutdown_connections 2024-03-26T17:30:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 --2- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4f4039280 0x7ff4f403b740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 --2- 172.21.15.144:0/3739886906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff508100690 0x7ff508101610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 >> 172.21.15.144:0/3739886906 conn(0x7ff5080783d0 msgr2=0x7ff508103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 shutdown_connections 2024-03-26T17:30:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:04.987+0000 7ff50f7ed700 1 -- 172.21.15.144:0/3739886906 wait complete. 2024-03-26T17:30:05.508 DEBUG:teuthology.orchestra.run.smithi135:mon.b> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.b.service 2024-03-26T17:30:05.510 DEBUG:teuthology.orchestra.run.smithi144:mon.c> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.c.service 2024-03-26T17:30:05.512 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T17:30:05.512 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T17:30:05.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:05 smithi105 ceph-mon[32280]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:05.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:05.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:05.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:05.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:05.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:05.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:05.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:06.518 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:06.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:06 smithi105 ceph-mon[32280]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi105:172.21.15.105=a;smithi135:172.21.15.135=b;smithi144:172.21.15.144=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:06.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:06 smithi105 ceph-mon[32280]: Saving service mon spec with placement smithi105:172.21.15.105=a;smithi135:172.21.15.135=b;smithi144:172.21.15.144=c;count:3 2024-03-26T17:30:06.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:06 smithi105 ceph-mon[32280]: Deploying daemon mon.c on smithi144 2024-03-26T17:30:07.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:07 smithi105 ceph-mon[32280]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.052+0000 7eff017a0700 1 -- 172.21.15.144:0/1669528704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 msgr2=0x7efefc1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.052+0000 7eff017a0700 1 --2- 172.21.15.144:0/1669528704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 0x7efefc1012b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efeec009910 tx=0x7efeec009c20 comp rx=0 tx=0).stop 2024-03-26T17:30:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.052+0000 7eff017a0700 1 -- 172.21.15.144:0/1669528704 shutdown_connections 2024-03-26T17:30:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.052+0000 7eff017a0700 1 --2- 172.21.15.144:0/1669528704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 0x7efefc1012b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.052+0000 7eff017a0700 1 -- 172.21.15.144:0/1669528704 >> 172.21.15.144:0/1669528704 conn(0x7efefc0fc7a0 msgr2=0x7efefc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.052+0000 7eff017a0700 1 -- 172.21.15.144:0/1669528704 shutdown_connections 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.052+0000 7eff017a0700 1 -- 172.21.15.144:0/1669528704 wait complete. 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.052+0000 7eff017a0700 1 Processor -- start 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.053+0000 7eff017a0700 1 -- start start 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.053+0000 7eff017a0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 0x7efefc1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.053+0000 7eff017a0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efefc10b150 con 0x7efefc100ed0 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.053+0000 7efefaffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 0x7efefc1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.053+0000 7efefaffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 0x7efefc1966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.144:52280/0 (socket says 172.21.15.144:52280) 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.053+0000 7efefaffd700 1 -- 172.21.15.144:0/1462497647 learned_addr learned my addr 172.21.15.144:0/1462497647 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:30:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.054+0000 7efefaffd700 1 -- 172.21.15.144:0/1462497647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc196c00 con 0x7efefc100ed0 2024-03-26T17:30:09.050 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.054+0000 7efefaffd700 1 --2- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 0x7efefc1966c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efeec0056d0 tx=0x7efeec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.054+0000 7efee3fff700 1 -- 172.21.15.144:0/1462497647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeec028070 con 0x7efefc100ed0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.054+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc196e90 con 0x7efefc100ed0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.054+0000 7efee3fff700 1 -- 172.21.15.144:0/1462497647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeec00bba0 con 0x7efefc100ed0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.054+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc19af90 con 0x7efefc100ed0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.054+0000 7efee3fff700 1 -- 172.21.15.144:0/1462497647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeec003860 con 0x7efefc100ed0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.055+0000 7efee3fff700 1 -- 172.21.15.144:0/1462497647 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7efeec02b070 con 0x7efefc100ed0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.055+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc1023b0 con 0x7efefc100ed0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.055+0000 7efee3fff700 1 --2- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efee4039220 0x7efee403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.055+0000 7efee3fff700 1 -- 172.21.15.144:0/1462497647 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efeec020070 con 0x7efefc100ed0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.055+0000 7efefa7fc700 1 --2- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efee4039220 0x7efee403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:09.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.056+0000 7efefa7fc700 1 --2- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efee4039220 0x7efee403b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efef0006fd0 tx=0x7efef0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:09.055 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.061+0000 7efee3fff700 1 -- 172.21.15.144:0/1462497647 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efeec013070 con 0x7efefc100ed0 2024-03-26T17:30:09.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 systemd[1]: Starting Ceph mon.c for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:30:09.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.354+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efefc197a20 con 0x7efefc100ed0 2024-03-26T17:30:09.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.355+0000 7efee3fff700 1 -- 172.21.15.144:0/1462497647 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7efeec00de00 con 0x7efefc100ed0 2024-03-26T17:30:09.351 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:30:09.351 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":1,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","modified":"2024-03-26T17:28:29.725786Z","created":"2024-03-26T17:28:29.725786Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efee4039220 msgr2=0x7efee403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 --2- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efee4039220 0x7efee403b6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efef0006fd0 tx=0x7efef0006e40 comp rx=0 tx=0).stop 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 msgr2=0x7efefc1966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 --2- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 0x7efefc1966c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efeec0056d0 tx=0x7efeec0057b0 comp rx=0 tx=0).stop 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 shutdown_connections 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 --2- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efee4039220 0x7efee403b6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 --2- 172.21.15.144:0/1462497647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efefc100ed0 0x7efefc1966c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 >> 172.21.15.144:0/1462497647 conn(0x7efefc0fc7a0 msgr2=0x7efefc1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:09.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 shutdown_connections 2024-03-26T17:30:09.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:09.358+0000 7eff017a0700 1 -- 172.21.15.144:0/1462497647 wait complete. 2024-03-26T17:30:09.354 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 1 2024-03-26T17:30:09.519 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 podman[36064]: 2024-03-26 17:30:09.268382807 +0000 UTC m=+0.103562017 container create e121d0b2a0cf4f5cf45b366975db0858524e1f9688df6536d56618e7a4f0be89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-26T17:30:09.519 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 podman[36064]: 2024-03-26 17:30:09.182553125 +0000 UTC m=+0.017732338 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:30:09.519 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 podman[36064]: 2024-03-26 17:30:09.479368379 +0000 UTC m=+0.314547589 container init e121d0b2a0cf4f5cf45b366975db0858524e1f9688df6536d56618e7a4f0be89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T17:30:09.519 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 podman[36064]: 2024-03-26 17:30:09.482731519 +0000 UTC m=+0.317910732 container start e121d0b2a0cf4f5cf45b366975db0858524e1f9688df6536d56618e7a4f0be89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-03-26T17:30:09.519 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev), process ceph-mon, pid 2 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pidfile_write: ignore empty --pid-file 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: load: jerasure load: lrc 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: RocksDB version: 7.9.2 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Git sha 0 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Compile date 2024-03-20 16:36:06 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: DB SUMMARY 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: DB Session ID: T2TVCTQANIBXSI7U3YC7 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: CURRENT file: CURRENT 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-26T17:30:09.520 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.error_if_exists: 0 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.create_if_missing: 0 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.paranoid_checks: 1 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.env: 0x55abc251dc60 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.info_log: 0x55abc3a37ae0 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T17:30:09.521 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.statistics: (nil) 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.use_fsync: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_log_file_size: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.allow_fallocate: 1 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.use_direct_reads: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T17:30:09.522 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.db_log_dir: 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.wal_dir: 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.write_buffer_manager: 0x55abc3a0a1e0 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T17:30:09.523 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T17:30:09.524 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.unordered_write: 0 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T17:30:09.525 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.row_cache: None 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.wal_filter: None 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.two_write_queues: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.wal_compression: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.atomic_flush: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.log_readahead_size: 0 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T17:30:09.526 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_background_jobs: 2 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_background_compactions: -1 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_subcompactions: 1 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T17:30:09.527 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_open_files: -1 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_background_flushes: -1 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Compression algorithms supported: 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: kZSTD supported: 0 2024-03-26T17:30:09.528 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: kXpressCompression supported: 0 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: kZlibCompression supported: 1 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: kSnappyCompression supported: 1 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: kLZ4Compression supported: 1 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: kBZip2Compression supported: 0 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.merge_operator: 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_filter: None 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T17:30:09.529 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55abc3a37be0) 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: cache_index_and_filter_blocks: 1 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: pin_top_level_index_and_filter: 1 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: index_type: 0 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: data_block_index_type: 0 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: index_shortening: 1 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: checksum: 4 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: no_block_cache: 0 2024-03-26T17:30:09.530 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_cache: 0x55abc3a3d090 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_cache_name: BinnedLRUCache 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_cache_options: 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: capacity : 536870912 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: num_shard_bits : 4 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: strict_capacity_limit : 0 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: high_pri_pool_ratio: 0.000 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_cache_compressed: (nil) 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: persistent_cache: (nil) 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_size: 4096 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_size_deviation: 10 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_restart_interval: 16 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: index_block_restart_interval: 1 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: metadata_block_size: 4096 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: partition_filters: 0 2024-03-26T17:30:09.531 INFO:journalctl@ceph.mon.c.smithi144.stdout: use_delta_encoding: 1 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: filter_policy: bloomfilter 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: whole_key_filtering: 1 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: verify_compression: 0 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: read_amp_bytes_per_bit: 0 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: format_version: 5 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: enable_index_compression: 1 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: block_align: 0 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: max_auto_readahead_size: 262144 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: prepopulate_block_cache: 0 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: initial_auto_readahead_size: 8192 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T17:30:09.532 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression: NoCompression 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.num_levels: 7 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T17:30:09.533 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T17:30:09.534 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T17:30:09.535 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T17:30:09.536 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T17:30:09.537 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.table_properties_collectors: 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.inplace_update_support: 0 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.bloom_locality: 0 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.max_successive_merges: 0 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T17:30:09.538 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.ttl: 2592000 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.enable_blob_files: false 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.min_blob_size: 0 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T17:30:09.539 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2287b589-63f6-4bb5-8617-4c8db503019e 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711474209524992, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711474209525679, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711474209, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2287b589-63f6-4bb5-8617-4c8db503019e", "db_session_id": "T2TVCTQANIBXSI7U3YC7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T17:30:09.540 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711474209525760, "job": 1, "event": "recovery_finished"} 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55abc3b30000 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: DB pointer 0x55abc3b1a000 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout: ** DB Stats ** 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T17:30:09.541 INFO:journalctl@ceph.mon.c.smithi144.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T17:30:09.542 INFO:journalctl@ceph.mon.c.smithi144.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T17:30:09.542 INFO:journalctl@ceph.mon.c.smithi144.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T17:30:09.542 INFO:journalctl@ceph.mon.c.smithi144.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T17:30:09.542 INFO:journalctl@ceph.mon.c.smithi144.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T17:30:09.542 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T17:30:09.542 INFO:journalctl@ceph.mon.c.smithi144.stdout: ** Compaction Stats [default] ** 2024-03-26T17:30:09.542 INFO:journalctl@ceph.mon.c.smithi144.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T17:30:09.542 INFO:journalctl@ceph.mon.c.smithi144.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: ** Compaction Stats [default] ** 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T17:30:09.543 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: Cumulative compaction: 0.00 GB write, 0.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: Interval compaction: 0.00 GB write, 0.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: Block cache BinnedLRUCache@0x55abc3a3d090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T17:30:09.544 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-26T17:30:09.545 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: using public_addr v2:172.21.15.144:0/0 -> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-03-26T17:30:09.826 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: starting mon.c rank -1 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(???) e0 preinit fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).mds e1 new map 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout: e1 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout: legacy client fscid: -1 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout: 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout: No filesystems configured 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T17:30:09.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.828 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Updating smithi135:/etc/ceph/ceph.conf 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Deploying cephadm binary to smithi144 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Updating smithi135:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.829 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Updating smithi135:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Added host smithi144 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1253086667' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1253086667' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:09.830 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Updating smithi144:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Updating smithi144:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.831 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi105:172.21.15.105=a;smithi135:172.21.15.135=b;smithi144:172.21.15.144=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Saving service mon spec with placement smithi105:172.21.15.105=a;smithi135:172.21.15.135=b;smithi144:172.21.15.144=c;count:3 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: Deploying daemon mon.c on smithi144 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 ceph-mon[36095]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 bash[36064]: e121d0b2a0cf4f5cf45b366975db0858524e1f9688df6536d56618e7a4f0be89 2024-03-26T17:30:09.832 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:09 smithi144 systemd[1]: Started Ceph mon.c for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:30:11.086 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T17:30:11.087 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T17:30:11.330 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:30:15.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:30:15.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: mon.a calling monitor election 2024-03-26T17:30:15.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:15.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: mon.c calling monitor election 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: monmap epoch 2 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: last_changed 2024-03-26T17:30:09.548178+0000 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: created 2024-03-26T17:28:29.725786+0000 2024-03-26T17:30:15.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: min_mon_release 19 (squid) 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: election_strategy: 1 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: fsmap 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: mgrmap e13: x(active, since 48s) 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: overall HEALTH_OK 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:15.103 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:15.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:30:15.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: mon.a calling monitor election 2024-03-26T17:30:15.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: mon.c calling monitor election 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: monmap epoch 2 2024-03-26T17:30:15.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: last_changed 2024-03-26T17:30:09.548178+0000 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: created 2024-03-26T17:28:29.725786+0000 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: min_mon_release 19 (squid) 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: election_strategy: 1 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: fsmap 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: mgrmap e13: x(active, since 48s) 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: overall HEALTH_OK 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:15.146 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:15.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.742+0000 7fb78f5e6700 1 -- 172.21.15.144:0/1893554243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 msgr2=0x7fb770005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:15.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.742+0000 7fb78f5e6700 1 --2- 172.21.15.144:0/1893554243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 0x7fb770005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb78806a670 tx=0x7fb778009e80 comp rx=0 tx=0).stop 2024-03-26T17:30:15.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.742+0000 7fb78f5e6700 1 -- 172.21.15.144:0/1893554243 shutdown_connections 2024-03-26T17:30:15.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.742+0000 7fb78f5e6700 1 --2- 172.21.15.144:0/1893554243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 0x7fb770005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:15.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.742+0000 7fb78f5e6700 1 -- 172.21.15.144:0/1893554243 >> 172.21.15.144:0/1893554243 conn(0x7fb7880fc8f0 msgr2=0x7fb7880fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:15.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.742+0000 7fb78f5e6700 1 -- 172.21.15.144:0/1893554243 shutdown_connections 2024-03-26T17:30:15.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.742+0000 7fb78f5e6700 1 -- 172.21.15.144:0/1893554243 wait complete. 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.743+0000 7fb78f5e6700 1 Processor -- start 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.743+0000 7fb78f5e6700 1 -- start start 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78f5e6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 0x7fb78819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78f5e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 0x7fb7881a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78f5e6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb788110420 con 0x7fb7700035c0 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78f5e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7881102a0 con 0x7fb78819e4f0 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78cb81700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 0x7fb7881a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78cb81700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 0x7fb7881a2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38648/0 (socket says 172.21.15.144:38648) 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78cb81700 1 -- 172.21.15.144:0/2834107235 learned_addr learned my addr 172.21.15.144:0/2834107235 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78cb81700 1 -- 172.21.15.144:0/2834107235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 msgr2=0x7fb7881a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78cb81700 1 -- 172.21.15.144:0/2834107235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 msgr2=0x7fb7881a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T17:30:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78cb81700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 0x7fb7881a2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T17:30:15.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78d382700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 0x7fb78819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:15.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78cb81700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 0x7fb7881a2880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:30:15.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78d382700 1 -- 172.21.15.144:0/2834107235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 msgr2=0x7fb7881a2880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:30:15.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78d382700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 0x7fb7881a2880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:15.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.744+0000 7fb78d382700 1 -- 172.21.15.144:0/2834107235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78819eb30 con 0x7fb7700035c0 2024-03-26T17:30:15.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.745+0000 7fb78d382700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 0x7fb78819dfb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb778007590 tx=0x7fb778017750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:15.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.745+0000 7fb7867fc700 1 -- 172.21.15.144:0/2834107235 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb778005260 con 0x7fb7700035c0 2024-03-26T17:30:15.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.745+0000 7fb7867fc700 1 -- 172.21.15.144:0/2834107235 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb778024030 con 0x7fb7700035c0 2024-03-26T17:30:15.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.745+0000 7fb7867fc700 1 -- 172.21.15.144:0/2834107235 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb778020b30 con 0x7fb7700035c0 2024-03-26T17:30:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.745+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7881a2dc0 con 0x7fb7700035c0 2024-03-26T17:30:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.745+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7881a3320 con 0x7fb7700035c0 2024-03-26T17:30:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.746+0000 7fb7867fc700 1 -- 172.21.15.144:0/2834107235 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7fb77801e070 con 0x7fb7700035c0 2024-03-26T17:30:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.746+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78806ae70 con 0x7fb7700035c0 2024-03-26T17:30:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.746+0000 7fb7867fc700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb774039010 0x7fb77403b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.746+0000 7fb7867fc700 1 -- 172.21.15.144:0/2834107235 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb77801c070 con 0x7fb7700035c0 2024-03-26T17:30:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.746+0000 7fb78cb81700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb774039010 0x7fb77403b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.747+0000 7fb78cb81700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb774039010 0x7fb77403b4d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009180 tx=0x7fb77c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:15.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:15.751+0000 7fb7867fc700 1 -- 172.21.15.144:0/2834107235 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb778025280 con 0x7fb7700035c0 2024-03-26T17:30:16.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.035+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb7881a35d0 con 0x7fb7700035c0 2024-03-26T17:30:16.030 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:15 smithi144 ceph-mon[36095]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:16.030 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:16.030 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:16.030 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:16.030 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:15 smithi144 ceph-mon[36095]: Deploying daemon mon.b on smithi135 2024-03-26T17:30:16.030 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:16.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.035+0000 7fb7867fc700 1 -- 172.21.15.144:0/2834107235 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fb778020d30 con 0x7fb7700035c0 2024-03-26T17:30:16.031 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:30:16.031 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":2,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","modified":"2024-03-26T17:30:09.548178Z","created":"2024-03-26T17:28:29.725786Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T17:30:16.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb774039010 msgr2=0x7fb77403b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:16.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb774039010 0x7fb77403b4d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009180 tx=0x7fb77c008040 comp rx=0 tx=0).stop 2024-03-26T17:30:16.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 msgr2=0x7fb78819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:16.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 0x7fb78819dfb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb778007590 tx=0x7fb778017750 comp rx=0 tx=0).stop 2024-03-26T17:30:16.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 shutdown_connections 2024-03-26T17:30:16.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb774039010 0x7fb77403b4d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:16.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7700035c0 0x7fb78819dfb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:16.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 --2- 172.21.15.144:0/2834107235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78819e4f0 0x7fb7881a2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:16.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 >> 172.21.15.144:0/2834107235 conn(0x7fb7880fc8f0 msgr2=0x7fb7880fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:16.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 shutdown_connections 2024-03-26T17:30:16.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:16.037+0000 7fb78f5e6700 1 -- 172.21.15.144:0/2834107235 wait complete. 2024-03-26T17:30:16.033 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 2 2024-03-26T17:30:16.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:15 smithi105 ceph-mon[32280]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:16.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:16.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:16.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:16.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:15 smithi105 ceph-mon[32280]: Deploying daemon mon.b on smithi135 2024-03-26T17:30:16.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:16.738 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:16 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/2834107235' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:30:16.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:16 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/2834107235' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:30:16.894 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:30:16 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:30:16.548+0000 7f45c5f57700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-26T17:30:17.767 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T17:30:17.767 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T17:30:18.001 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:30:18.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:17 smithi144 ceph-mon[36095]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:18.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:17 smithi105 ceph-mon[32280]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:23.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: mon.c calling monitor election 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: mon.a calling monitor election 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:23.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: monmap epoch 3 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: last_changed 2024-03-26T17:30:18.520833+0000 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: created 2024-03-26T17:28:29.725786+0000 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: min_mon_release 19 (squid) 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: election_strategy: 1 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-03-26T17:30:23.896 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: fsmap 2024-03-26T17:30:23.897 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:23.897 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: mgrmap e13: x(active, since 57s) 2024-03-26T17:30:23.897 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: overall HEALTH_OK 2024-03-26T17:30:23.897 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:23.897 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:23.897 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:24.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.023+0000 7f50c7742700 1 -- 172.21.15.144:0/2560582477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50ac0057d0 msgr2=0x7f50ac007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:24.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.023+0000 7f50c7742700 1 --2- 172.21.15.144:0/2560582477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50ac0057d0 0x7f50ac007c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f50c0100cb0 tx=0x7f50b000fe60 comp rx=0 tx=0).stop 2024-03-26T17:30:24.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.023+0000 7f50c7742700 1 -- 172.21.15.144:0/2560582477 shutdown_connections 2024-03-26T17:30:24.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.023+0000 7f50c7742700 1 --2- 172.21.15.144:0/2560582477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50ac0057d0 0x7f50ac007c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:24.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.023+0000 7f50c7742700 1 -- 172.21.15.144:0/2560582477 >> 172.21.15.144:0/2560582477 conn(0x7f50c00fc850 msgr2=0x7f50c00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:24.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.023+0000 7f50c7742700 1 -- 172.21.15.144:0/2560582477 shutdown_connections 2024-03-26T17:30:24.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.023+0000 7f50c7742700 1 -- 172.21.15.144:0/2560582477 wait complete. 2024-03-26T17:30:24.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.024+0000 7f50c7742700 1 Processor -- start 2024-03-26T17:30:24.019 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:24.019 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:30:24.019 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: mon.c calling monitor election 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: mon.a calling monitor election 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: monmap epoch 3 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:24.020 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: last_changed 2024-03-26T17:30:18.520833+0000 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: created 2024-03-26T17:28:29.725786+0000 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: min_mon_release 19 (squid) 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: election_strategy: 1 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: fsmap 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: mgrmap e13: x(active, since 57s) 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: overall HEALTH_OK 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:24.021 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.024+0000 7f50c7742700 1 -- start start 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.024+0000 7f50c7742700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 0x7f50c019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.024+0000 7f50c7742700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 0x7f50c0195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.024+0000 7f50c7742700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c0195bb0 0x7f50c0196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.024+0000 7f50c7742700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f50c0107c60 con 0x7f50c0195bb0 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.024+0000 7f50c7742700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f50c006b420 con 0x7f50ac0057d0 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.024+0000 7f50c7742700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50c0107de0 con 0x7f50c019bae0 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c4cdd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 0x7f50c0195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c4cdd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 0x7f50c0195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51172/0 (socket says 172.21.15.144:51172) 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c4cdd700 1 -- 172.21.15.144:0/587820588 learned_addr learned my addr 172.21.15.144:0/587820588 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c5cdf700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c0195bb0 0x7f50c0196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:24.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c54de700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 0x7f50c019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c4cdd700 1 -- 172.21.15.144:0/587820588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 msgr2=0x7f50c0195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c4cdd700 1 -- 172.21.15.144:0/587820588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 msgr2=0x7f50c0195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c4cdd700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 0x7f50c0195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c4cdd700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 0x7f50c0195670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c54de700 1 -- 172.21.15.144:0/587820588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 msgr2=0x7f50c019b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c54de700 1 -- 172.21.15.144:0/587820588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 msgr2=0x7f50c019b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c54de700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 0x7f50c019b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c54de700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 0x7f50c019b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c5cdf700 1 -- 172.21.15.144:0/587820588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 msgr2=0x7f50c0195670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c5cdf700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 0x7f50c0195670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c5cdf700 1 -- 172.21.15.144:0/587820588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 msgr2=0x7f50c019b5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c5cdf700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 0x7f50c019b5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c5cdf700 1 -- 172.21.15.144:0/587820588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c01a2350 con 0x7f50c0195bb0 2024-03-26T17:30:24.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c5cdf700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c0195bb0 0x7f50c0196060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f50b800bd50 tx=0x7f50b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50b67fc700 1 -- 172.21.15.144:0/587820588 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8019070 con 0x7f50c0195bb0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50b67fc700 1 -- 172.21.15.144:0/587820588 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50b800cd10 con 0x7f50c0195bb0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c01a25e0 con 0x7f50c0195bb0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50b67fc700 1 -- 172.21.15.144:0/587820588 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8012700 con 0x7f50c0195bb0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.025+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c01a2b20 con 0x7f50c0195bb0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.026+0000 7f50b67fc700 1 -- 172.21.15.144:0/587820588 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7f50b801c070 con 0x7f50c0195bb0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.026+0000 7f50b67fc700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50a40396b0 0x7f50a403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.026+0000 7f50b67fc700 1 -- 172.21.15.144:0/587820588 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f50b804bca0 con 0x7f50c0195bb0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.027+0000 7f50c54de700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50a40396b0 0x7f50a403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.027+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c01024c0 con 0x7f50c0195bb0 2024-03-26T17:30:24.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.027+0000 7f50c54de700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50a40396b0 0x7f50a403bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f50b0005b40 tx=0x7f50b0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:24.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.032+0000 7f50b67fc700 1 -- 172.21.15.144:0/587820588 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f50b800e0a0 con 0x7f50c0195bb0 2024-03-26T17:30:24.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.348+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f50c002dc00 con 0x7f50c0195bb0 2024-03-26T17:30:24.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.349+0000 7f50b67fc700 1 -- 172.21.15.144:0/587820588 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f50b8013c30 con 0x7f50c0195bb0 2024-03-26T17:30:24.344 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-03-26T17:30:24.344 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":3,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","modified":"2024-03-26T17:30:18.520833Z","created":"2024-03-26T17:28:29.725786Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T17:30:24.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50a40396b0 msgr2=0x7f50a403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:24.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50a40396b0 0x7f50a403bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f50b0005b40 tx=0x7f50b0005a90 comp rx=0 tx=0).stop 2024-03-26T17:30:24.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c0195bb0 msgr2=0x7f50c0196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:24.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c0195bb0 0x7f50c0196060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f50b800bd50 tx=0x7f50b800be30 comp rx=0 tx=0).stop 2024-03-26T17:30:24.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 shutdown_connections 2024-03-26T17:30:24.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50a40396b0 0x7f50a403bb70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:24.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50c0195bb0 0x7f50c0196060 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:24.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50ac0057d0 0x7f50c019b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:24.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 --2- 172.21.15.144:0/587820588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50c019bae0 0x7f50c0195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:24.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 >> 172.21.15.144:0/587820588 conn(0x7f50c00fc850 msgr2=0x7f50c00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:24.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 shutdown_connections 2024-03-26T17:30:24.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:24.352+0000 7f50c7742700 1 -- 172.21.15.144:0/587820588 wait complete. 2024-03-26T17:30:24.348 INFO:teuthology.orchestra.run.smithi144.stderr:dumped monmap epoch 3 2024-03-26T17:30:24.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:24.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:24.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:24 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/587820588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:30:24.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:24.908 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:24.908 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:24.908 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:24 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/587820588' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T17:30:24.908 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:24.935 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-26T17:30:24.935 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-26T17:30:25.177 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:30:25.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:26.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: mon.b calling monitor election 2024-03-26T17:30:26.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T17:30:26.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: Updating smithi135:/etc/ceph/ceph.conf 2024-03-26T17:30:26.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T17:30:26.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: mon.b calling monitor election 2024-03-26T17:30:26.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: mon.c calling monitor election 2024-03-26T17:30:26.541 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: mon.a calling monitor election 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: monmap epoch 3 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: last_changed 2024-03-26T17:30:18.520833+0000 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: created 2024-03-26T17:28:29.725786+0000 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: min_mon_release 19 (squid) 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: election_strategy: 1 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: fsmap 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: mgrmap e13: x(active, since 59s) 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: overall HEALTH_OK 2024-03-26T17:30:26.542 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: Updating smithi105:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: Updating smithi144:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:26.543 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.777+0000 7f525f979700 1 -- 172.21.15.105:0/1735109554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258100f10 msgr2=0x7f52581012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.777+0000 7f525f979700 1 --2- 172.21.15.105:0/1735109554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258100f10 0x7f52581012f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5248009910 tx=0x7f5248009c20 comp rx=0 tx=0).stop 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.777+0000 7f525f979700 1 -- 172.21.15.105:0/1735109554 shutdown_connections 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.777+0000 7f525f979700 1 --2- 172.21.15.105:0/1735109554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258100f10 0x7f52581012f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.777+0000 7f525f979700 1 -- 172.21.15.105:0/1735109554 >> 172.21.15.105:0/1735109554 conn(0x7f5258077a50 msgr2=0x7f52580fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.777+0000 7f525f979700 1 -- 172.21.15.105:0/1735109554 shutdown_connections 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.777+0000 7f525f979700 1 -- 172.21.15.105:0/1735109554 wait complete. 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525f979700 1 Processor -- start 2024-03-26T17:30:26.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525f979700 1 -- start start 2024-03-26T17:30:26.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525f979700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5258100f10 0x7f525819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:26.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525f979700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f525819dbe0 0x7f5258197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:26.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525f979700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258197da0 0x7f5258198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:26.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525f979700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f525810b140 con 0x7f5258197da0 2024-03-26T17:30:26.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525f979700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f525810afc0 con 0x7f525819dbe0 2024-03-26T17:30:26.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525f979700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f525810b2c0 con 0x7f5258100f10 2024-03-26T17:30:26.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525df16700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258197da0 0x7f5258198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525df16700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258197da0 0x7f5258198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48920/0 (socket says 172.21.15.105:48920) 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525df16700 1 -- 172.21.15.105:0/2753486066 learned_addr learned my addr 172.21.15.105:0/2753486066 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525d715700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5258100f10 0x7f525819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525cf14700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f525819dbe0 0x7f5258197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525df16700 1 -- 172.21.15.105:0/2753486066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5258100f10 msgr2=0x7f525819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525df16700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5258100f10 0x7f525819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525df16700 1 -- 172.21.15.105:0/2753486066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f525819dbe0 msgr2=0x7f5258197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.778+0000 7f525df16700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f525819dbe0 0x7f5258197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.779+0000 7f525df16700 1 -- 172.21.15.105:0/2753486066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52581a4570 con 0x7f5258197da0 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.779+0000 7f525cf14700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f525819dbe0 0x7f5258197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.779+0000 7f525df16700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258197da0 0x7f5258198250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f525000ba50 tx=0x7f525000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.779+0000 7f524e7fc700 1 -- 172.21.15.105:0/2753486066 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250019070 con 0x7f5258197da0 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.779+0000 7f524e7fc700 1 -- 172.21.15.105:0/2753486066 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5250014440 con 0x7f5258197da0 2024-03-26T17:30:26.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.779+0000 7f524e7fc700 1 -- 172.21.15.105:0/2753486066 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250007590 con 0x7f5258197da0 2024-03-26T17:30:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.779+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52581a47a0 con 0x7f5258197da0 2024-03-26T17:30:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.779+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52581a4ce0 con 0x7f5258197da0 2024-03-26T17:30:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.780+0000 7f524e7fc700 1 -- 172.21.15.105:0/2753486066 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7f525001c070 con 0x7f5258197da0 2024-03-26T17:30:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.780+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52580639e0 con 0x7f5258197da0 2024-03-26T17:30:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.780+0000 7f524e7fc700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52440396b0 0x7f524403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.780+0000 7f524e7fc700 1 -- 172.21.15.105:0/2753486066 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f525004b9f0 con 0x7f5258197da0 2024-03-26T17:30:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.780+0000 7f525d715700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52440396b0 0x7f524403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:26.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.780+0000 7f525d715700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52440396b0 0x7f524403bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5248009750 tx=0x7f524801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:26.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.785+0000 7f524e7fc700 1 -- 172.21.15.105:0/2753486066 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5250014980 con 0x7f5258197da0 2024-03-26T17:30:26.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: mon.b calling monitor election 2024-03-26T17:30:26.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T17:30:26.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: Updating smithi135:/etc/ceph/ceph.conf 2024-03-26T17:30:26.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T17:30:26.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: mon.b calling monitor election 2024-03-26T17:30:26.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: mon.c calling monitor election 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: mon.a calling monitor election 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: monmap epoch 3 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: last_changed 2024-03-26T17:30:18.520833+0000 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: created 2024-03-26T17:28:29.725786+0000 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: min_mon_release 19 (squid) 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: election_strategy: 1 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: fsmap 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: mgrmap e13: x(active, since 59s) 2024-03-26T17:30:26.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: overall HEALTH_OK 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: Updating smithi105:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: Updating smithi144:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:26.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:26.854 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: mon.b calling monitor election 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: Updating smithi135:/etc/ceph/ceph.conf 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: Updating smithi144:/etc/ceph/ceph.conf 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: mon.b calling monitor election 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: mon.c calling monitor election 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: mon.a calling monitor election 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: monmap epoch 3 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:26.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: last_changed 2024-03-26T17:30:18.520833+0000 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: created 2024-03-26T17:28:29.725786+0000 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: min_mon_release 19 (squid) 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: election_strategy: 1 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: 0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: 1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.c 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: 2: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.b 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: fsmap 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: mgrmap e13: x(active, since 59s) 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: overall HEALTH_OK 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: Updating smithi105:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: Updating smithi144:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.919 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:26.920 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.994+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f52581a4fd0 con 0x7f5258197da0 2024-03-26T17:30:26.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.994+0000 7f524e7fc700 1 -- 172.21.15.105:0/2753486066 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f525001d3b0 con 0x7f5258197da0 2024-03-26T17:30:26.988 INFO:teuthology.orchestra.run.smithi105.stdout:# minimal ceph.conf for 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:26.988 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-03-26T17:30:26.989 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:30:26.989 INFO:teuthology.orchestra.run.smithi105.stdout: mon_host = [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52440396b0 msgr2=0x7f524403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52440396b0 0x7f524403bb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5248009750 tx=0x7f524801a040 comp rx=0 tx=0).stop 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258197da0 msgr2=0x7f5258198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258197da0 0x7f5258198250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f525000ba50 tx=0x7f525000be10 comp rx=0 tx=0).stop 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 shutdown_connections 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52440396b0 0x7f524403bb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5258197da0 0x7f5258198250 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5258100f10 0x7f525819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 --2- 172.21.15.105:0/2753486066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f525819dbe0 0x7f5258197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 >> 172.21.15.105:0/2753486066 conn(0x7f5258077a50 msgr2=0x7f52580697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 shutdown_connections 2024-03-26T17:30:26.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:26.996+0000 7f525f979700 1 -- 172.21.15.105:0/2753486066 wait complete. 2024-03-26T17:30:27.619 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:27 smithi105 ceph-mon[32280]: Updating smithi135:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:27.619 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:27 smithi105 ceph-mon[32280]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T17:30:27.619 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:27 smithi105 ceph-mon[32280]: Reconfiguring daemon mon.a on smithi105 2024-03-26T17:30:27.620 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:27 smithi105 ceph-mon[32280]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:27.620 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:27.620 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:27.620 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:27 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2753486066' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:27.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:27 smithi144 ceph-mon[36095]: Updating smithi135:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:27.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:27 smithi144 ceph-mon[36095]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T17:30:27.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:27 smithi144 ceph-mon[36095]: Reconfiguring daemon mon.a on smithi105 2024-03-26T17:30:27.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:27 smithi144 ceph-mon[36095]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:27.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:27.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:27.852 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:27 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2753486066' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:27.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:27 smithi135 ceph-mon[35636]: Updating smithi135:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T17:30:27.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:27 smithi135 ceph-mon[35636]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T17:30:27.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:27 smithi135 ceph-mon[35636]: Reconfiguring daemon mon.a on smithi105 2024-03-26T17:30:27.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:27 smithi135 ceph-mon[35636]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:27.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:30:27.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:27.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:27 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2753486066' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:28.333 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-26T17:30:28.333 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:30:28.333 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T17:30:28.364 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:30:28.364 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:30:28.432 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:30:28.433 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T17:30:28.462 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:30:28.462 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:30:28.530 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:30:28.530 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T17:30:28.563 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:30:28.564 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T17:30:28.633 INFO:tasks.cephadm:Adding mgr.x on smithi105 2024-03-26T17:30:28.634 INFO:tasks.cephadm:Adding mgr.y on smithi135 2024-03-26T17:30:28.634 INFO:tasks.cephadm:Adding mgr.z on smithi144 2024-03-26T17:30:28.634 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi105=x;smithi135=y;smithi144=z' 2024-03-26T17:30:28.907 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:30:29.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:28 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:29.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:28 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:29.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:28 smithi135 ceph-mon[35636]: Reconfiguring mon.b (monmap changed)... 2024-03-26T17:30:29.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:28 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:29.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:28 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:29.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:28 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:29.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:28 smithi135 ceph-mon[35636]: Reconfiguring daemon mon.b on smithi135 2024-03-26T17:30:29.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:28 smithi135 ceph-mon[35636]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:29.223 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:28 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:29.223 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:28 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:29.223 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:28 smithi144 ceph-mon[36095]: Reconfiguring mon.b (monmap changed)... 2024-03-26T17:30:29.223 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:28 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:29.223 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:28 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:29.223 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:28 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:29.223 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:28 smithi144 ceph-mon[36095]: Reconfiguring daemon mon.b on smithi135 2024-03-26T17:30:29.223 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:28 smithi144 ceph-mon[36095]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:29.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:29.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:29.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:28 smithi105 ceph-mon[32280]: Reconfiguring mon.b (monmap changed)... 2024-03-26T17:30:29.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:29.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:29.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:29.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:28 smithi105 ceph-mon[32280]: Reconfiguring daemon mon.b on smithi135 2024-03-26T17:30:29.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:28 smithi105 ceph-mon[32280]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:30 smithi144 ceph-mon[36095]: Reconfiguring mon.c (monmap changed)... 2024-03-26T17:30:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:30 smithi144 ceph-mon[36095]: Reconfiguring daemon mon.c on smithi144 2024-03-26T17:30:30.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:30.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:30.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:30 smithi105 ceph-mon[32280]: Reconfiguring mon.c (monmap changed)... 2024-03-26T17:30:30.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:30.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:30.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:30.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:30 smithi105 ceph-mon[32280]: Reconfiguring daemon mon.c on smithi144 2024-03-26T17:30:30.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:30.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:30.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:30 smithi135 ceph-mon[35636]: Reconfiguring mon.c (monmap changed)... 2024-03-26T17:30:30.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T17:30:30.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T17:30:30.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:30.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:30 smithi135 ceph-mon[35636]: Reconfiguring daemon mon.c on smithi144 2024-03-26T17:30:31.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.333+0000 7f9004a58700 1 -- 172.21.15.144:0/3322383439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9000102350 msgr2=0x7f9000102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:31.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.333+0000 7f9004a58700 1 --2- 172.21.15.144:0/3322383439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9000102350 0x7f9000102770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009910 tx=0x7f8ff0009c20 comp rx=0 tx=0).stop 2024-03-26T17:30:31.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.333+0000 7f9004a58700 1 -- 172.21.15.144:0/3322383439 shutdown_connections 2024-03-26T17:30:31.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.333+0000 7f9004a58700 1 --2- 172.21.15.144:0/3322383439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9000102350 0x7f9000102770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:31.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.333+0000 7f9004a58700 1 -- 172.21.15.144:0/3322383439 >> 172.21.15.144:0/3322383439 conn(0x7f90000fbb20 msgr2=0x7f90000fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:31.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.333+0000 7f9004a58700 1 -- 172.21.15.144:0/3322383439 shutdown_connections 2024-03-26T17:30:31.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.333+0000 7f9004a58700 1 -- 172.21.15.144:0/3322383439 wait complete. 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.333+0000 7f9004a58700 1 Processor -- start 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f9004a58700 1 -- start start 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f9004a58700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9000102350 0x7f9000100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f9004a58700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900019a3a0 0x7f900019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f9004a58700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900019cd20 0x7f900019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f9004a58700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9000106cd0 con 0x7f900019cd20 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f9004a58700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9000106b50 con 0x7f900019a3a0 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f9004a58700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9000106e50 con 0x7f9000102350 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffed9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900019cd20 0x7f900019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:31.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffed9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900019cd20 0x7f900019f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.144:43276/0 (socket says 172.21.15.144:43276) 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffed9d700 1 -- 172.21.15.144:0/3370215707 learned_addr learned my addr 172.21.15.144:0/3370215707 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffe59c700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9000102350 0x7f9000100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffed9d700 1 -- 172.21.15.144:0/3370215707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9000102350 msgr2=0x7f9000100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffed9d700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9000102350 0x7f9000100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffed9d700 1 -- 172.21.15.144:0/3370215707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900019a3a0 msgr2=0x7f900019c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffed9d700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900019a3a0 0x7f900019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.334+0000 7f8ffed9d700 1 -- 172.21.15.144:0/3370215707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900019fa90 con 0x7f900019cd20 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.335+0000 7f8ffed9d700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900019cd20 0x7f900019f1c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800ba50 tx=0x7f8ff800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.335+0000 7f8fef7fe700 1 -- 172.21.15.144:0/3370215707 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8019070 con 0x7f900019cd20 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.335+0000 7f8fef7fe700 1 -- 172.21.15.144:0/3370215707 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ff8014440 con 0x7f900019cd20 2024-03-26T17:30:31.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.335+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900019fd80 con 0x7f900019cd20 2024-03-26T17:30:31.332 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.335+0000 7f8fef7fe700 1 -- 172.21.15.144:0/3370215707 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800ce60 con 0x7f900019cd20 2024-03-26T17:30:31.332 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.335+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90001a0310 con 0x7f900019cd20 2024-03-26T17:30:31.332 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.336+0000 7f8fef7fe700 1 -- 172.21.15.144:0/3370215707 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7f8ff801c070 con 0x7f900019cd20 2024-03-26T17:30:31.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.336+0000 7f8fef7fe700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fe80396b0 0x7f8fe803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:31.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.336+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9000063a20 con 0x7f900019cd20 2024-03-26T17:30:31.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.336+0000 7f8fef7fe700 1 -- 172.21.15.144:0/3370215707 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8ff804b9a0 con 0x7f900019cd20 2024-03-26T17:30:31.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.336+0000 7f8ffe59c700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fe80396b0 0x7f8fe803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:31.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.337+0000 7f8ffe59c700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fe80396b0 0x7f8fe803bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005190 tx=0x7f8ff001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:31.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.341+0000 7f8fef7fe700 1 -- 172.21.15.144:0/3370215707 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8ff8012500 con 0x7f900019cd20 2024-03-26T17:30:31.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.555+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi105=x;smithi135=y;smithi144=z", "target": ["mon-mgr", ""]}) -- 0x7f9000062880 con 0x7f8fe80396b0 2024-03-26T17:30:31.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.561+0000 7f8fef7fe700 1 -- 172.21.15.144:0/3370215707 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9000062880 con 0x7f8fe80396b0 2024-03-26T17:30:31.556 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mgr update... 2024-03-26T17:30:31.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:31 smithi144 ceph-mon[36095]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:31.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:31.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:31.556 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:31.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fe80396b0 msgr2=0x7f8fe803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:31.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fe80396b0 0x7f8fe803bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0005190 tx=0x7f8ff001a040 comp rx=0 tx=0).stop 2024-03-26T17:30:31.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900019cd20 msgr2=0x7f900019f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:31.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900019cd20 0x7f900019f1c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800ba50 tx=0x7f8ff800bd60 comp rx=0 tx=0).stop 2024-03-26T17:30:31.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 shutdown_connections 2024-03-26T17:30:31.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fe80396b0 0x7f8fe803bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:31.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f900019cd20 0x7f900019f1c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:31.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9000102350 0x7f9000100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:31.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 --2- 172.21.15.144:0/3370215707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f900019a3a0 0x7f900019c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:31.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.563+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 >> 172.21.15.144:0/3370215707 conn(0x7f90000fbb20 msgr2=0x7f90000fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:31.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.564+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 shutdown_connections 2024-03-26T17:30:31.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:30:31.564+0000 7f9004a58700 1 -- 172.21.15.144:0/3370215707 wait complete. 2024-03-26T17:30:31.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:31 smithi105 ceph-mon[32280]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:31.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:31.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:31.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:31.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:31 smithi135 ceph-mon[35636]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:31.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:31.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:31.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:32.151 DEBUG:teuthology.orchestra.run.smithi135:mgr.y> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.y.service 2024-03-26T17:30:32.154 DEBUG:teuthology.orchestra.run.smithi144:mgr.z> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.z.service 2024-03-26T17:30:32.156 INFO:tasks.cephadm:Deploying OSDs... 2024-03-26T17:30:32.156 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:30:32.156 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T17:30:32.172 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T17:30:32.172 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 17:29:38.462303200 +0000 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 17:24:16.891621393 +0000 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 17:24:16.891621393 +0000 2024-03-26T17:30:32.228 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-26 17:24:16.891621393 +0000 2024-03-26T17:30:32.229 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T17:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-26T17:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-26T17:30:32.294 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000190705 s, 2.7 MB/s 2024-03-26T17:30:32.295 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T17:30:32.351 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-03-26T17:30:32.406 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T17:30:32.406 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:32.406 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-26T17:30:32.406 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:32.406 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:32.407 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 17:29:38.491302599 +0000 2024-03-26T17:30:32.407 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 17:24:17.241614095 +0000 2024-03-26T17:30:32.407 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 17:24:17.241614095 +0000 2024-03-26T17:30:32.407 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-26 17:24:17.241614095 +0000 2024-03-26T17:30:32.407 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T17:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-26T17:30:32.471 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-26T17:30:32.472 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000192397 s, 2.7 MB/s 2024-03-26T17:30:32.473 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T17:30:32.529 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-03-26T17:30:32.585 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T17:30:32.586 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:32.586 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-26T17:30:32.586 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:32.586 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:32.586 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 17:29:38.520301997 +0000 2024-03-26T17:30:32.586 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 17:24:17.586606899 +0000 2024-03-26T17:30:32.586 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 17:24:17.586606899 +0000 2024-03-26T17:30:32.586 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-26 17:24:17.586606899 +0000 2024-03-26T17:30:32.586 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T17:30:32.653 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-26T17:30:32.730 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-26T17:30:32.730 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.00019078 s, 2.7 MB/s 2024-03-26T17:30:32.732 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T17:30:32.787 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-03-26T17:30:32.843 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T17:30:32.843 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:32.843 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-26T17:30:32.843 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:32.843 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:32.843 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 17:29:38.556301250 +0000 2024-03-26T17:30:32.844 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 17:24:17.925599830 +0000 2024-03-26T17:30:32.844 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 17:24:17.925599830 +0000 2024-03-26T17:30:32.844 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-26 17:24:17.925599830 +0000 2024-03-26T17:30:32.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi105=x;smithi135=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: Saving service mgr spec with placement smithi105=x;smithi135=y;smithi144=z;count:3 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:32.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi105=x;smithi135=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: Saving service mgr spec with placement smithi105=x;smithi135=y;smithi144=z;count:3 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:32.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:32.909 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-26T17:30:32.909 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-26T17:30:32.909 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000176095 s, 2.9 MB/s 2024-03-26T17:30:32.910 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T17:30:32.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi105=x;smithi135=y;smithi144=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: Saving service mgr spec with placement smithi105=x;smithi135=y;smithi144=z;count:3 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:32.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:32.970 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-03-26T17:30:32.970 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T17:30:32.989 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T17:30:32.989 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-03-26T17:30:33.048 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T17:30:33.048 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:33.048 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-26T17:30:33.048 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:33.048 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:33.049 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-03-26 17:30:17.999163563 +0000 2024-03-26T17:30:33.049 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-03-26 17:24:23.236721114 +0000 2024-03-26T17:30:33.049 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-03-26 17:24:23.236721114 +0000 2024-03-26T17:30:33.049 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-03-26 17:24:23.236721114 +0000 2024-03-26T17:30:33.049 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T17:30:33.117 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-03-26T17:30:33.118 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-03-26T17:30:33.118 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000289459 s, 1.8 MB/s 2024-03-26T17:30:33.119 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T17:30:33.177 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-03-26T17:30:33.234 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T17:30:33.234 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:33.235 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-26T17:30:33.235 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:33.235 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:33.235 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-03-26 17:30:18.000163541 +0000 2024-03-26T17:30:33.235 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-03-26 17:24:23.579713914 +0000 2024-03-26T17:30:33.235 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-03-26 17:24:23.579713914 +0000 2024-03-26T17:30:33.235 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-03-26 17:24:23.579713914 +0000 2024-03-26T17:30:33.236 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T17:30:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-03-26T17:30:33.302 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-03-26T17:30:33.303 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000187847 s, 2.7 MB/s 2024-03-26T17:30:33.303 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T17:30:33.360 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-03-26T17:30:33.416 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T17:30:33.416 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:33.416 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-26T17:30:33.416 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:33.417 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:33.417 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-03-26 17:30:18.000163541 +0000 2024-03-26T17:30:33.417 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-03-26 17:24:23.927706608 +0000 2024-03-26T17:30:33.417 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-03-26 17:24:23.927706608 +0000 2024-03-26T17:30:33.417 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-03-26 17:24:23.927706608 +0000 2024-03-26T17:30:33.417 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T17:30:33.482 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-03-26T17:30:33.483 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-03-26T17:30:33.483 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000199159 s, 2.6 MB/s 2024-03-26T17:30:33.486 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T17:30:33.543 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-03-26T17:30:33.597 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T17:30:33.598 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:33.598 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-26T17:30:33.598 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:33.598 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:33.598 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-03-26 17:30:18.000163541 +0000 2024-03-26T17:30:33.598 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-03-26 17:24:24.256699702 +0000 2024-03-26T17:30:33.598 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-03-26 17:24:24.256699702 +0000 2024-03-26T17:30:33.598 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-03-26 17:24:24.256699702 +0000 2024-03-26T17:30:33.599 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T17:30:33.663 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-03-26T17:30:33.663 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-03-26T17:30:33.663 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.00021444 s, 2.4 MB/s 2024-03-26T17:30:33.664 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T17:30:33.720 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-03-26T17:30:33.720 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T17:30:33.737 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T17:30:33.737 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-03-26T17:30:33.793 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T17:30:33.793 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:33.793 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-26T17:30:33.793 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:33.793 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:33.793 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-26 17:30:33.584272803 +0000 2024-03-26T17:30:33.793 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-26 17:24:16.940660908 +0000 2024-03-26T17:30:33.794 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-26 17:24:16.940660908 +0000 2024-03-26T17:30:33.794 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-26 17:24:16.940660908 +0000 2024-03-26T17:30:33.794 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T17:30:33.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:33 smithi144 ceph-mon[36095]: Deploying daemon mgr.z on smithi144 2024-03-26T17:30:33.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:33 smithi144 ceph-mon[36095]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:33.851 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:33 smithi144 systemd[1]: Starting Ceph mgr.z for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:30:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-26T17:30:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-26T17:30:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000265162 s, 1.9 MB/s 2024-03-26T17:30:33.862 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T17:30:33.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:33 smithi105 ceph-mon[32280]: Deploying daemon mgr.z on smithi144 2024-03-26T17:30:33.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:33 smithi105 ceph-mon[32280]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:33.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:33 smithi135 ceph-mon[35636]: Deploying daemon mgr.z on smithi144 2024-03-26T17:30:33.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:33 smithi135 ceph-mon[35636]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:33.918 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-03-26T17:30:33.973 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T17:30:33.973 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:33.974 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-26T17:30:33.974 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:33.974 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:33.974 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-26 17:30:33.584272803 +0000 2024-03-26T17:30:33.974 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-26 17:24:17.273654967 +0000 2024-03-26T17:30:33.974 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-26 17:24:17.273654967 +0000 2024-03-26T17:30:33.974 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-26 17:24:17.273654967 +0000 2024-03-26T17:30:33.975 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T17:30:34.039 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-26T17:30:34.039 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-26T17:30:34.039 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000214921 s, 2.4 MB/s 2024-03-26T17:30:34.040 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T17:30:34.099 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-03-26T17:30:34.154 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T17:30:34.154 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:34.155 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-26T17:30:34.155 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:34.155 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:34.155 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-26 17:30:33.584272803 +0000 2024-03-26T17:30:34.155 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-26 17:24:17.620648777 +0000 2024-03-26T17:30:34.155 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-26 17:24:17.620648777 +0000 2024-03-26T17:30:34.155 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-26 17:24:17.620648777 +0000 2024-03-26T17:30:34.155 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T17:30:34.219 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:33 smithi144 podman[37713]: 2024-03-26 17:30:33.760985894 +0000 UTC m=+0.017806305 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:30:34.219 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:33 smithi144 podman[37713]: 2024-03-26 17:30:33.870322511 +0000 UTC m=+0.127142897 container create 89c606a2afae007cb963516bb9c060a88b045f91ecd160f9cb0fd52c32a9f25e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T17:30:34.219 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:34 smithi144 podman[37713]: 2024-03-26 17:30:34.047645765 +0000 UTC m=+0.304466152 container init 89c606a2afae007cb963516bb9c060a88b045f91ecd160f9cb0fd52c32a9f25e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T17:30:34.219 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:34 smithi144 podman[37713]: 2024-03-26 17:30:34.050835128 +0000 UTC m=+0.307655521 container start 89c606a2afae007cb963516bb9c060a88b045f91ecd160f9cb0fd52c32a9f25e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T17:30:34.219 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:34 smithi144 ceph-mgr[37822]: -- 172.21.15.144:0/2700461846 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56001cc865a0 con 0x56001cc5b000 2024-03-26T17:30:34.219 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:34 smithi144 bash[37713]: 89c606a2afae007cb963516bb9c060a88b045f91ecd160f9cb0fd52c32a9f25e 2024-03-26T17:30:34.219 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:34 smithi144 systemd[1]: Started Ceph mgr.z for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:30:34.220 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-26T17:30:34.220 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-26T17:30:34.221 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000197407 s, 2.6 MB/s 2024-03-26T17:30:34.222 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T17:30:34.278 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-03-26T17:30:34.333 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T17:30:34.334 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T17:30:34.334 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-26T17:30:34.334 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:30:34.334 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T17:30:34.334 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-03-26 17:30:33.585272786 +0000 2024-03-26T17:30:34.334 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-03-26 17:24:17.959642730 +0000 2024-03-26T17:30:34.334 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-03-26 17:24:17.959642730 +0000 2024-03-26T17:30:34.334 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-03-26 17:24:17.959642730 +0000 2024-03-26T17:30:34.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T17:30:34.401 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-03-26T17:30:34.401 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-03-26T17:30:34.401 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000216938 s, 2.4 MB/s 2024-03-26T17:30:34.403 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T17:30:34.459 INFO:tasks.cephadm:Deploying osd.0 on smithi105 with /dev/vg_nvme/lv_4... 2024-03-26T17:30:34.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T17:30:34.703 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:30:35.101 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:34 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:34.609+0000 7f446911c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:30:35.101 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:34 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:34.702+0000 7f446911c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:30:35.333 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.333 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.333 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:35.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:30:35.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:35.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:35.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: Deploying daemon mgr.y on smithi135 2024-03-26T17:30:35.334 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:35 smithi135 ceph-mon[35636]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:35.432 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:35.345+0000 7f446911c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:30:35.432 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:35.437+0000 7f446911c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: Deploying daemon mgr.y on smithi135 2024-03-26T17:30:35.433 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-mon[36095]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:35.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:35.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:35.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T17:30:35.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:35.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:35.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: Deploying daemon mgr.y on smithi135 2024-03-26T17:30:35.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:35 smithi105 ceph-mon[32280]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:35.751 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:35.579+0000 7f446911c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:30:35.751 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:35.674+0000 7f446911c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:30:35.751 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:35.756+0000 7f446911c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:30:36.308 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:36.019+0000 7f446911c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:30:36.308 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:36.128+0000 7f446911c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:30:36.309 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:36.217+0000 7f446911c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:30:36.309 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:36.313+0000 7f446911c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:30:36.480 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T17:30:36.481 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T17:30:36.481 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-26T17:30:36.481 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-26T17:30:36.481 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114281 s, 918 MB/s 2024-03-26T17:30:36.481 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-26T17:30:36.600 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:36.496+0000 7f446911c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:30:36.728 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:36 smithi135 podman[37012]: 2024-03-26 17:30:36.709387995 +0000 UTC m=+0.350432855 container start ef9dc14d24170298a0dfcb18286fd453520adecbc1f4d251186d631801d7995f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-26T17:30:36.728 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:36 smithi135 ceph-mgr[37032]: -- 172.21.15.135:0/503734181 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e6d22be5a0 con 0x55e6d2293000 2024-03-26T17:30:37.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_4 2024-03-26T17:30:37.167 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:36 smithi135 bash[37012]: ef9dc14d24170298a0dfcb18286fd453520adecbc1f4d251186d631801d7995f 2024-03-26T17:30:37.167 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:36 smithi135 systemd[1]: Started Ceph mgr.y for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:30:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:30:37.562 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:37 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:37.281+0000 7f8e1edc9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T17:30:37.562 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:37 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:37.378+0000 7f8e1edc9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T17:30:37.660 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:37 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:37.525+0000 7f446911c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:30:37.661 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:37 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:37.666+0000 7f446911c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:30:37.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:37 smithi144 ceph-mon[36095]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:37.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.661 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:37.764 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:37 smithi105 ceph-mon[32280]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:37.764 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.764 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.764 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.764 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.764 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:37.858 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:37 smithi135 ceph-mon[35636]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:37.858 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.858 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.858 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.858 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:37.858 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:38.157 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:38.062+0000 7f8e1edc9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T17:30:38.303 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:38.013+0000 7f446911c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:30:38.304 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:38.103+0000 7f446911c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:30:38.304 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:38.308+0000 7f446911c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:30:38.417 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:38.162+0000 7f8e1edc9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T17:30:38.417 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:38.307+0000 7f8e1edc9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T17:30:38.417 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:38.404+0000 7f8e1edc9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T17:30:38.601 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:38.497+0000 7f446911c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:30:38.748 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:38.488+0000 7f8e1edc9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T17:30:38.748 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:38.753+0000 7f8e1edc9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T17:30:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 -- 172.21.15.105:0/339570758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 msgr2=0x7efeac102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 --2- 172.21.15.105:0/339570758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 0x7efeac102870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efea80099c0 tx=0x7efea8009cd0 comp rx=0 tx=0).stop 2024-03-26T17:30:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 -- 172.21.15.105:0/339570758 shutdown_connections 2024-03-26T17:30:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 --2- 172.21.15.105:0/339570758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeac102db0 0x7efeac109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 --2- 172.21.15.105:0/339570758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efeac101210 0x7efeac101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 --2- 172.21.15.105:0/339570758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 0x7efeac102870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 -- 172.21.15.105:0/339570758 >> 172.21.15.105:0/339570758 conn(0x7efeac0fc7d0 msgr2=0x7efeac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 -- 172.21.15.105:0/339570758 shutdown_connections 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.814+0000 7efeb319e700 1 -- 172.21.15.105:0/339570758 wait complete. 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb319e700 1 Processor -- start 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb319e700 1 -- start start 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb319e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efeac101210 0x7efeac1051a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb319e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 0x7efeac1037f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb319e700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeac102db0 0x7efeac103d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb319e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efeac10cca0 con 0x7efeac102410 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb319e700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efeac10cb20 con 0x7efeac102db0 2024-03-26T17:30:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb319e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efeac10ce20 con 0x7efeac101210 2024-03-26T17:30:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb199b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 0x7efeac1037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb219c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efeac101210 0x7efeac1051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb199b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 0x7efeac1037f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43410/0 (socket says 172.21.15.105:43410) 2024-03-26T17:30:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb199b700 1 -- 172.21.15.105:0/919396060 learned_addr learned my addr 172.21.15.105:0/919396060 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:30:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb299d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeac102db0 0x7efeac103d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb219c700 1 -- 172.21.15.105:0/919396060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeac102db0 msgr2=0x7efeac103d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.815+0000 7efeb219c700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeac102db0 0x7efeac103d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efeb219c700 1 -- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 msgr2=0x7efeac1037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efeb219c700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 0x7efeac1037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efeb219c700 1 -- 172.21.15.105:0/919396060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac1045e0 con 0x7efeac101210 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efeb219c700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efeac101210 0x7efeac1051a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00e9c0 tx=0x7efe9c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c004230 con 0x7efeac101210 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe9c0043d0 con 0x7efeac101210 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c003d90 con 0x7efeac101210 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac1a25c0 con 0x7efeac101210 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.816+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac1a2a80 con 0x7efeac101210 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.817+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) ==== 46351+0+0 (secure 0 0 0) 0x7efe9c021440 con 0x7efeac101210 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.817+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac069070 con 0x7efeac101210 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.817+0000 7efea37fe700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efe98039460 0x7efe9803b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:30:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.817+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe9c04c740 con 0x7efeac101210 2024-03-26T17:30:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.817+0000 7efeb199b700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efe98039460 0x7efe9803b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:30:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.818+0000 7efeb199b700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efe98039460 0x7efe9803b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efea8003820 tx=0x7efea8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:30:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:38.822+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efe9c01fc00 con 0x7efeac101210 2024-03-26T17:30:39.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:39.040+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7efeac1a2d40 con 0x7efe98039460 2024-03-26T17:30:39.036 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:38.862+0000 7f8e1edc9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T17:30:39.036 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:38.947+0000 7f8e1edc9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T17:30:39.036 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:39.040+0000 7f8e1edc9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T17:30:39.037 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:38.780+0000 7f446911c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:30:39.037 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:39.043+0000 7f446911c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:30:39.347 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.347 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.347 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: Reconfiguring daemon mgr.x on smithi105 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:30:39.348 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:39.350 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:39.280+0000 7f446911c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:30:39.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: Reconfiguring daemon mgr.x on smithi105 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:30:39.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:39.417 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:39.217+0000 7f8e1edc9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T17:30:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: Reconfiguring daemon mgr.x on smithi105 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:30:39.419 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:30:39.419 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:40.336 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:40.077+0000 7f446911c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:30:40.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-mon[36095]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:40.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:40.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:40.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:40.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:40 smithi105 ceph-mon[32280]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:40.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:40.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:40.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:40.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.647 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:40.188+0000 7f8e1edc9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T17:30:40.647 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:40.316+0000 7f8e1edc9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T17:30:40.647 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:40.652+0000 7f8e1edc9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T17:30:40.647 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-mon[35636]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:30:40.647 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.647 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.647 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:40.647 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:40.648 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:40.648 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:40.917 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:40.740+0000 7f8e1edc9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T17:30:40.960 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:40.650+0000 7f446911c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:30:40.960 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:40.746+0000 7f446911c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:30:40.960 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 17:30:40 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z[37784]: 2024-03-26T17:30:40.965+0000 7f446911c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:30:41.341 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:40.934+0000 7f8e1edc9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:30:41.341 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:41.117+0000 7f8e1edc9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T17:30:41.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:41.349+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 14) ==== 92075+0+0 (secure 0 0 0) 0x7efe9c0107b0 con 0x7efeac101210 2024-03-26T17:30:41.368 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:41 smithi105 ceph-mon[32280]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:41.368 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:41 smithi105 ceph-mon[32280]: Standby manager daemon z started 2024-03-26T17:30:41.368 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:41 smithi105 ceph-mon[32280]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T17:30:41.368 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:41 smithi105 ceph-mon[32280]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:30:41.368 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:41 smithi105 ceph-mon[32280]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T17:30:41.368 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:41 smithi105 ceph-mon[32280]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:30:41.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:41 smithi144 ceph-mon[36095]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:41.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:41 smithi144 ceph-mon[36095]: Standby manager daemon z started 2024-03-26T17:30:41.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:41 smithi144 ceph-mon[36095]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T17:30:41.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:41 smithi144 ceph-mon[36095]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:30:41.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:41 smithi144 ceph-mon[36095]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T17:30:41.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:41 smithi144 ceph-mon[36095]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:30:41.658 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-mon[35636]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:41.658 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-mon[35636]: Standby manager daemon z started 2024-03-26T17:30:41.658 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-mon[35636]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T17:30:41.658 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-mon[35636]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:30:41.659 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-mon[35636]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T17:30:41.659 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-mon[35636]: from='mgr.? 172.21.15.144:0/2368061110' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:30:41.659 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:41.400+0000 7f8e1edc9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T17:30:41.659 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:41.661+0000 7f8e1edc9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T17:30:41.917 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:41.895+0000 7f8e1edc9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T17:30:42.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:42 smithi144 ceph-mon[36095]: mgrmap e14: x(active, since 74s), standbys: z 2024-03-26T17:30:42.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T17:30:42.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:42.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:42 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2143552560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42769940-d924-4456-ab99-af1c060516a6"}]: dispatch 2024-03-26T17:30:42.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:42 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2143552560' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42769940-d924-4456-ab99-af1c060516a6"}]': finished 2024-03-26T17:30:42.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:42 smithi144 ceph-mon[36095]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T17:30:42.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:30:42.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:42 smithi105 ceph-mon[32280]: mgrmap e14: x(active, since 74s), standbys: z 2024-03-26T17:30:42.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T17:30:42.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:42.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:42 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2143552560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42769940-d924-4456-ab99-af1c060516a6"}]: dispatch 2024-03-26T17:30:42.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:42 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2143552560' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42769940-d924-4456-ab99-af1c060516a6"}]': finished 2024-03-26T17:30:42.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:42 smithi105 ceph-mon[32280]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T17:30:42.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:30:42.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:42 smithi135 ceph-mon[35636]: mgrmap e14: x(active, since 74s), standbys: z 2024-03-26T17:30:42.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T17:30:42.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:42.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:42 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2143552560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42769940-d924-4456-ab99-af1c060516a6"}]: dispatch 2024-03-26T17:30:42.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:42 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2143552560' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42769940-d924-4456-ab99-af1c060516a6"}]': finished 2024-03-26T17:30:42.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:42 smithi135 ceph-mon[35636]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T17:30:42.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:30:43.167 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:42.691+0000 7f8e1edc9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T17:30:43.565 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:43 smithi135 ceph-mon[35636]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:43.565 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:43 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2662315765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:30:43.565 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:43 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:43.259+0000 7f8e1edc9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T17:30:43.565 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:43 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:43.356+0000 7f8e1edc9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T17:30:43.565 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 17:30:43 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y[37028]: 2024-03-26T17:30:43.570+0000 7f8e1edc9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T17:30:43.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:43 smithi105 ceph-mon[32280]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:43.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:43 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2662315765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:30:43.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:43 smithi144 ceph-mon[36095]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:43.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:43 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2662315765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:30:44.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:44.367+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mon.1 v2:172.21.15.144:3300/0 8 ==== mgrmap(e 15) ==== 137799+0+0 (secure 0 0 0) 0x7efe9c016370 con 0x7efeac101210 2024-03-26T17:30:44.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:44 smithi105 ceph-mon[32280]: Standby manager daemon y started 2024-03-26T17:30:44.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:44 smithi105 ceph-mon[32280]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T17:30:44.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:44 smithi105 ceph-mon[32280]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:30:44.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:44 smithi105 ceph-mon[32280]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T17:30:44.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:44 smithi105 ceph-mon[32280]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:30:44.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:44 smithi135 ceph-mon[35636]: Standby manager daemon y started 2024-03-26T17:30:44.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:44 smithi135 ceph-mon[35636]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T17:30:44.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:44 smithi135 ceph-mon[35636]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:30:44.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:44 smithi135 ceph-mon[35636]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T17:30:44.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:44 smithi135 ceph-mon[35636]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:30:44.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:44 smithi144 ceph-mon[36095]: Standby manager daemon y started 2024-03-26T17:30:44.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:44 smithi144 ceph-mon[36095]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T17:30:44.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:44 smithi144 ceph-mon[36095]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T17:30:44.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:44 smithi144 ceph-mon[36095]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T17:30:44.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:44 smithi144 ceph-mon[36095]: from='mgr.? 172.21.15.135:0/3342948449' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T17:30:45.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:45 smithi105 ceph-mon[32280]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-03-26T17:30:45.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T17:30:45.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:45 smithi105 ceph-mon[32280]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:45.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:45 smithi135 ceph-mon[35636]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-03-26T17:30:45.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T17:30:45.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:45 smithi135 ceph-mon[35636]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:45.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:45 smithi144 ceph-mon[36095]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-03-26T17:30:45.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T17:30:45.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:45 smithi144 ceph-mon[36095]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:47.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:47 smithi105 ceph-mon[32280]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:47.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:47 smithi144 ceph-mon[36095]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:47 smithi135 ceph-mon[35636]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:49.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:49 smithi144 ceph-mon[36095]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:49.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T17:30:49.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:49.869 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:49 smithi105 ceph-mon[32280]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:49.869 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T17:30:49.869 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:49.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:49 smithi135 ceph-mon[35636]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:49.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T17:30:49.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:50.842 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:50 smithi105 ceph-mon[32280]: Deploying daemon osd.0 on smithi105 2024-03-26T17:30:50.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:50 smithi144 ceph-mon[36095]: Deploying daemon osd.0 on smithi105 2024-03-26T17:30:50.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:50 smithi135 ceph-mon[35636]: Deploying daemon osd.0 on smithi105 2024-03-26T17:30:51.848 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:51 smithi105 ceph-mon[32280]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:51.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:51 smithi144 ceph-mon[36095]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:51.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:51 smithi135 ceph-mon[35636]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:53.823 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:53 smithi105 ceph-mon[32280]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:53.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:53 smithi144 ceph-mon[36095]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:53.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:53 smithi135 ceph-mon[35636]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:55.619 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:55.619 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:55.619 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:55.619 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:55 smithi105 ceph-mon[32280]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:55.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:55.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:55.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:55.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:55 smithi135 ceph-mon[35636]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:55.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:55.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:55.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:55.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:55 smithi144 ceph-mon[36095]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:57.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:57 smithi144 ceph-mon[36095]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:57.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:57.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:57.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.859 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:57 smithi105 ceph-mon[32280]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:57.859 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.859 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.859 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:57.859 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:57.859 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:57 smithi135 ceph-mon[35636]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:57.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:30:57.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:30:57.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:57.939 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 0 on host 'smithi105' 2024-03-26T17:30:57.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.944+0000 7efea37fe700 1 -- 172.21.15.105:0/919396060 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efeac1a2d40 con 0x7efe98039460 2024-03-26T17:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efe98039460 msgr2=0x7efe9803b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efe98039460 0x7efe9803b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efea8003820 tx=0x7efea8005fb0 comp rx=0 tx=0).stop 2024-03-26T17:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efeac101210 msgr2=0x7efeac1051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efeac101210 0x7efeac1051a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00e9c0 tx=0x7efe9c00ecd0 comp rx=0 tx=0).stop 2024-03-26T17:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 shutdown_connections 2024-03-26T17:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efe98039460 0x7efe9803b920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeac102db0 0x7efeac103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efeac101210 0x7efeac1051a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 --2- 172.21.15.105:0/919396060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efeac102410 0x7efeac1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 >> 172.21.15.105:0/919396060 conn(0x7efeac0fc7d0 msgr2=0x7efeac0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 shutdown_connections 2024-03-26T17:30:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:30:57.946+0000 7efeb319e700 1 -- 172.21.15.105:0/919396060 wait complete. 2024-03-26T17:30:58.515 DEBUG:teuthology.orchestra.run.smithi105:osd.0> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.0.service 2024-03-26T17:30:58.518 INFO:tasks.cephadm:Deploying osd.1 on smithi135 with /dev/vg_nvme/lv_4... 2024-03-26T17:30:58.518 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T17:30:58.760 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:30:59.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:59.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:59.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:59.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:58 smithi105 ceph-mon[32280]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T17:30:59.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:58 smithi105 ceph-mon[32280]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:58 smithi135 ceph-mon[35636]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T17:30:59.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:58 smithi135 ceph-mon[35636]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:59.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:59.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:30:59.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:30:59.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:58 smithi144 ceph-mon[36095]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T17:30:59.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:58 smithi144 ceph-mon[36095]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:30:59.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:59 smithi135 ceph-mon[35636]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T17:30:59.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:59 smithi135 ceph-mon[35636]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T17:30:59.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:59 smithi135 ceph-mon[35636]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:30:59.996 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:30:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:00.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:59 smithi144 ceph-mon[36095]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T17:31:00.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:59 smithi144 ceph-mon[36095]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T17:31:00.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:59 smithi144 ceph-mon[36095]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:31:00.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:30:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:00.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:59 smithi105 ceph-mon[32280]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T17:31:00.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:59 smithi105 ceph-mon[32280]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T17:31:00.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:59 smithi105 ceph-mon[32280]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:31:00.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:30:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:00.395 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:30:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0[46771]: 2024-03-26T17:30:59.912+0000 7f416a2a1700 -1 osd.0 0 waiting for initial osdmap 2024-03-26T17:31:00.395 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:30:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0[46771]: 2024-03-26T17:30:59.918+0000 7f4166094700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:31:00.568 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T17:31:00.568 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T17:31:00.568 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-03-26T17:31:00.568 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-03-26T17:31:00.568 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111562 s, 940 MB/s 2024-03-26T17:31:00.568 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-03-26T17:31:01.010 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:00 smithi105 ceph-mon[32280]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:31:01.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:00 smithi105 ceph-mon[32280]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T17:31:01.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:00 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:01.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:00 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:01.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:00 smithi105 ceph-mon[32280]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:31:01.124 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi135:vg_nvme/lv_4 2024-03-26T17:31:01.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:00 smithi135 ceph-mon[35636]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:31:01.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:00 smithi135 ceph-mon[35636]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T17:31:01.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:00 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:01.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:00 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:01.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:00 smithi135 ceph-mon[35636]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:31:01.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:00 smithi144 ceph-mon[36095]: from='osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:31:01.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:00 smithi144 ceph-mon[36095]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T17:31:01.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:00 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:01.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:00 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:01.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:00 smithi144 ceph-mon[36095]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T17:31:01.364 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:31:02.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] boot 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi105 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi105 to 17088M 2024-03-26T17:31:02.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] boot 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T17:31:02.352 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi105 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi105 to 17088M 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:02.353 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:31:02.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:31:02.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:02.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: osd.0 [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] boot 2024-03-26T17:31:02.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T17:31:02.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi105 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi105 to 17088M 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:02.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:02.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.856+0000 7ff57b932700 1 -- 172.21.15.135:0/3960860530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741035e0 msgr2=0x7ff574103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:02.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.856+0000 7ff57b932700 1 --2- 172.21.15.135:0/3960860530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741035e0 0x7ff574103a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5640099c0 tx=0x7ff564009cd0 comp rx=0 tx=0).stop 2024-03-26T17:31:02.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.856+0000 7ff57b932700 1 -- 172.21.15.135:0/3960860530 shutdown_connections 2024-03-26T17:31:02.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.856+0000 7ff57b932700 1 --2- 172.21.15.135:0/3960860530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff574103f80 0x7ff57410a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:02.853 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.856+0000 7ff57b932700 1 --2- 172.21.15.135:0/3960860530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5741023e0 0x7ff574102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.856+0000 7ff57b932700 1 --2- 172.21.15.135:0/3960860530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741035e0 0x7ff574103a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.856+0000 7ff57b932700 1 -- 172.21.15.135:0/3960860530 >> 172.21.15.135:0/3960860530 conn(0x7ff5740fd9c0 msgr2=0x7ff5740ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 -- 172.21.15.135:0/3960860530 shutdown_connections 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 -- 172.21.15.135:0/3960860530 wait complete. 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 Processor -- start 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 -- start start 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741023e0 0x7ff574198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5741035e0 0x7ff574198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff574103f80 0x7ff57419fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff57410def0 con 0x7ff574103f80 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff57410dd70 con 0x7ff5741023e0 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff57b932700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff57410e070 con 0x7ff5741035e0 2024-03-26T17:31:02.854 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff5796ce700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741023e0 0x7ff574198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff5796ce700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741023e0 0x7ff574198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57846/0 (socket says 172.21.15.135:57846) 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.857+0000 7ff5796ce700 1 -- 172.21.15.135:0/1826552784 learned_addr learned my addr 172.21.15.135:0/1826552784 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff578ecd700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5741035e0 0x7ff574198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff579ecf700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff574103f80 0x7ff57419fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff5796ce700 1 -- 172.21.15.135:0/1826552784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5741035e0 msgr2=0x7ff574198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff5796ce700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5741035e0 0x7ff574198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff5796ce700 1 -- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff574103f80 msgr2=0x7ff57419fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff5796ce700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff574103f80 0x7ff57419fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff5796ce700 1 -- 172.21.15.135:0/1826552784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5741a0270 con 0x7ff5741023e0 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff5796ce700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741023e0 0x7ff574198610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff57000b500 tx=0x7ff57000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff56a7fc700 1 -- 172.21.15.135:0/1826552784 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570015020 con 0x7ff5741023e0 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff56a7fc700 1 -- 172.21.15.135:0/1826552784 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff570011ce0 con 0x7ff5741023e0 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5741a0560 con 0x7ff5741023e0 2024-03-26T17:31:02.855 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff56a7fc700 1 -- 172.21.15.135:0/1826552784 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570010d90 con 0x7ff5741023e0 2024-03-26T17:31:02.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.858+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741a0b70 con 0x7ff5741023e0 2024-03-26T17:31:02.856 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.859+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff574069070 con 0x7ff5741023e0 2024-03-26T17:31:02.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.860+0000 7ff56a7fc700 1 -- 172.21.15.135:0/1826552784 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 15) ==== 137799+0+0 (secure 0 0 0) 0x7ff570018020 con 0x7ff5741023e0 2024-03-26T17:31:02.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.860+0000 7ff56a7fc700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5600a2fa0 0x7ff5600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:02.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.861+0000 7ff56a7fc700 1 -- 172.21.15.135:0/1826552784 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff5700cd040 con 0x7ff5741023e0 2024-03-26T17:31:02.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.861+0000 7ff578ecd700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5600a2fa0 0x7ff5600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:02.857 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.861+0000 7ff578ecd700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5600a2fa0 0x7ff5600a5460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff564009fd0 tx=0x7ff564005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:31:02.860 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:02.865+0000 7ff56a7fc700 1 -- 172.21.15.135:0/1826552784 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff57001f020 con 0x7ff5741023e0 2024-03-26T17:31:03.063 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:03.068+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff5740627f0 con 0x7ff5600a2fa0 2024-03-26T17:31:03.353 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:02 smithi144 ceph-mon[36095]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T17:31:03.353 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:02 smithi144 ceph-mon[36095]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:03.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:02 smithi105 ceph-mon[32280]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T17:31:03.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:02 smithi105 ceph-mon[32280]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:03.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:02 smithi135 ceph-mon[35636]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T17:31:03.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:02 smithi135 ceph-mon[35636]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:04.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:03 smithi144 ceph-mon[36095]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:04.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:04.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:04.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:04.355 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:03 smithi135 ceph-mon[35636]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:04.355 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:04.355 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:04.355 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:04.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:03 smithi105 ceph-mon[32280]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:04.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:04.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:04.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:05.251 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:04 smithi144 ceph-mon[36095]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:05.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:04 smithi105 ceph-mon[32280]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:05.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:04 smithi135 ceph-mon[35636]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:06.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:05 smithi144 ceph-mon[36095]: from='client.? 172.21.15.135:0/3914716541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]: dispatch 2024-03-26T17:31:06.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:05 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]: dispatch 2024-03-26T17:31:06.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:05 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]': finished 2024-03-26T17:31:06.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:05 smithi144 ceph-mon[36095]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T17:31:06.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:05 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:06.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:05 smithi105 ceph-mon[32280]: from='client.? 172.21.15.135:0/3914716541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]: dispatch 2024-03-26T17:31:06.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:05 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]: dispatch 2024-03-26T17:31:06.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:05 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]': finished 2024-03-26T17:31:06.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:05 smithi105 ceph-mon[32280]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T17:31:06.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:06.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:05 smithi135 ceph-mon[35636]: from='client.? 172.21.15.135:0/3914716541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]: dispatch 2024-03-26T17:31:06.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:05 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]: dispatch 2024-03-26T17:31:06.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:05 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33937b10-f283-4680-a12c-ee35827a2bd2"}]': finished 2024-03-26T17:31:06.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:05 smithi135 ceph-mon[35636]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T17:31:06.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:05 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:07.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:06 smithi144 ceph-mon[36095]: from='client.? 172.21.15.135:0/2417150345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:07.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:06 smithi144 ceph-mon[36095]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:07.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:06 smithi105 ceph-mon[32280]: from='client.? 172.21.15.135:0/2417150345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:07.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:06 smithi105 ceph-mon[32280]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:07.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:06 smithi135 ceph-mon[35636]: from='client.? 172.21.15.135:0/2417150345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:07.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:06 smithi135 ceph-mon[35636]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:09.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:09 smithi144 ceph-mon[36095]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:09.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:09 smithi105 ceph-mon[32280]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:09 smithi135 ceph-mon[35636]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:11.834 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:11 smithi135 ceph-mon[35636]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:11.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:11 smithi144 ceph-mon[36095]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:11.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:11 smithi105 ceph-mon[32280]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:13.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:13 smithi144 ceph-mon[36095]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:13.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T17:31:13.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:13.890 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:13 smithi135 ceph-mon[35636]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:13.890 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T17:31:13.890 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:13.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:13 smithi105 ceph-mon[32280]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:13.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T17:31:13.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:14.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:14 smithi144 ceph-mon[36095]: Deploying daemon osd.1 on smithi135 2024-03-26T17:31:14.870 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:14 smithi135 ceph-mon[35636]: Deploying daemon osd.1 on smithi135 2024-03-26T17:31:14.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:14 smithi105 ceph-mon[32280]: Deploying daemon osd.1 on smithi135 2024-03-26T17:31:15.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:15 smithi144 ceph-mon[36095]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:15.852 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:15 smithi135 ceph-mon[35636]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:15.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:15 smithi105 ceph-mon[32280]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:17.843 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:17 smithi135 ceph-mon[35636]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:17.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:17 smithi144 ceph-mon[36095]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:17.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:17 smithi105 ceph-mon[32280]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:19 smithi105 ceph-mon[32280]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:19.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:19.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:19.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:19.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:19 smithi135 ceph-mon[35636]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:19.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:19.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:19.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:19.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:19 smithi144 ceph-mon[36095]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:21.831 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:21 smithi135 ceph-mon[35636]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:21.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.836+0000 7ff56a7fc700 1 -- 172.21.15.135:0/1826552784 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff5740627f0 con 0x7ff5600a2fa0 2024-03-26T17:31:21.832 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 1 on host 'smithi135' 2024-03-26T17:31:21.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5600a2fa0 msgr2=0x7ff5600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:21.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5600a2fa0 0x7ff5600a5460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff564009fd0 tx=0x7ff564005fb0 comp rx=0 tx=0).stop 2024-03-26T17:31:21.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741023e0 msgr2=0x7ff574198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741023e0 0x7ff574198610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff57000b500 tx=0x7ff57000b810 comp rx=0 tx=0).stop 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 shutdown_connections 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5600a2fa0 0x7ff5600a5460 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff574103f80 0x7ff57419fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5741023e0 0x7ff574198610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 --2- 172.21.15.135:0/1826552784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5741035e0 0x7ff574198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 >> 172.21.15.135:0/1826552784 conn(0x7ff5740fd9c0 msgr2=0x7ff5741069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 shutdown_connections 2024-03-26T17:31:21.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:31:21.839+0000 7ff57b932700 1 -- 172.21.15.135:0/1826552784 wait complete. 2024-03-26T17:31:21.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:21 smithi144 ceph-mon[36095]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:21.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:21 smithi105 ceph-mon[32280]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:22.493 DEBUG:teuthology.orchestra.run.smithi135:osd.1> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.1.service 2024-03-26T17:31:22.496 INFO:tasks.cephadm:Deploying osd.2 on smithi144 with /dev/vg_nvme/lv_4... 2024-03-26T17:31:22.496 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T17:31:22.738 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:31:22.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:22 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:22.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:22 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:22.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:22 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:22.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:22 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:22.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:22 smithi144 ceph-mon[36095]: from='osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:31:22.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:22 smithi144 ceph-mon[36095]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:31:22.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:22 smithi144 ceph-mon[36095]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:23.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:22 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:23.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:22 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:23.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:22 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:23.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:22 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:23.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:22 smithi105 ceph-mon[32280]: from='osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:31:23.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:22 smithi105 ceph-mon[32280]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:31:23.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:22 smithi105 ceph-mon[32280]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:23.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:22 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:23.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:22 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:23.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:22 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:23.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:22 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:23.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:22 smithi135 ceph-mon[35636]: from='osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:31:23.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:22 smithi135 ceph-mon[35636]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T17:31:23.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:22 smithi135 ceph-mon[35636]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:24.031 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:23 smithi144 ceph-mon[36095]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T17:31:24.032 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:23 smithi144 ceph-mon[36095]: from='osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:31:24.032 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:23 smithi144 ceph-mon[36095]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T17:31:24.032 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:24.032 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:23 smithi144 ceph-mon[36095]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:31:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:23 smithi105 ceph-mon[32280]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T17:31:24.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:23 smithi105 ceph-mon[32280]: from='osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:31:24.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:23 smithi105 ceph-mon[32280]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T17:31:24.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:24.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:23 smithi105 ceph-mon[32280]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:31:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:23 smithi135 ceph-mon[35636]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T17:31:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:23 smithi135 ceph-mon[35636]: from='osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:31:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:23 smithi135 ceph-mon[35636]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T17:31:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:23 smithi135 ceph-mon[35636]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:31:24.168 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:23 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1[40246]: 2024-03-26T17:31:23.887+0000 7f239218a700 -1 osd.1 0 waiting for initial osdmap 2024-03-26T17:31:24.168 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:23 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1[40246]: 2024-03-26T17:31:23.893+0000 7f238df7d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:31:24.591 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T17:31:24.591 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T17:31:24.591 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-26T17:31:24.591 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-26T17:31:24.591 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113511 s, 924 MB/s 2024-03-26T17:31:24.591 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-26T17:31:25.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:24 smithi105 ceph-mon[32280]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:31:25.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:24 smithi105 ceph-mon[32280]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T17:31:25.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:25.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:25.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:24 smithi105 ceph-mon[32280]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:25.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:24 smithi135 ceph-mon[35636]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:31:25.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:24 smithi135 ceph-mon[35636]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T17:31:25.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:25.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:25.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:24 smithi135 ceph-mon[35636]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:25.169 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:24 smithi144 ceph-mon[36095]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:31:25.169 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:24 smithi144 ceph-mon[36095]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T17:31:25.169 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:25.169 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:25.169 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:24 smithi144 ceph-mon[36095]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T17:31:25.196 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-03-26T17:31:25.443 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:31:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:31:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:31:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] boot 2024-03-26T17:31:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi135 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi135 to 17088M 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:26.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:31:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:31:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] boot 2024-03-26T17:31:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T17:31:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi135 2024-03-26T17:31:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi135 to 17088M 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:26.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:31:26.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:31:26.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:26.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: osd.1 [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] boot 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi135 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi135 to 17088M 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:26.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:26.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 -- 172.21.15.144:0/517526437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 msgr2=0x7fb5a0102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 --2- 172.21.15.144:0/517526437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 0x7fb5a0102800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb594009990 tx=0x7fb594009ca0 comp rx=0 tx=0).stop 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 -- 172.21.15.144:0/517526437 shutdown_connections 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 --2- 172.21.15.144:0/517526437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5a0103f80 0x7fb5a010a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 --2- 172.21.15.144:0/517526437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 0x7fb5a0102800 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 --2- 172.21.15.144:0/517526437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a01035e0 0x7fb5a0103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 -- 172.21.15.144:0/517526437 >> 172.21.15.144:0/517526437 conn(0x7fb5a00fd9c0 msgr2=0x7fb5a00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 -- 172.21.15.144:0/517526437 shutdown_connections 2024-03-26T17:31:26.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.870+0000 7fb5a7071700 1 -- 172.21.15.144:0/517526437 wait complete. 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a7071700 1 Processor -- start 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a7071700 1 -- start start 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a7071700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 0x7fb5a01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a7071700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5a01035e0 0x7fb5a0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a7071700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a0103f80 0x7fb5a019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a7071700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a010ded0 con 0x7fb5a0103f80 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a7071700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a010dd50 con 0x7fb5a01035e0 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a7071700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a010e050 con 0x7fb5a01023e0 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a4e0d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 0x7fb5a01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a4e0d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 0x7fb5a01985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44288/0 (socket says 172.21.15.144:44288) 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a4e0d700 1 -- 172.21.15.144:0/4095286971 learned_addr learned my addr 172.21.15.144:0/4095286971 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a560e700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a0103f80 0x7fb5a019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a4e0d700 1 -- 172.21.15.144:0/4095286971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5a01035e0 msgr2=0x7fb5a0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:26.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb59ffff700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5a01035e0 0x7fb5a0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a4e0d700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5a01035e0 0x7fb5a0198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a4e0d700 1 -- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a0103f80 msgr2=0x7fb5a019fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a4e0d700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a0103f80 0x7fb5a019fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.871+0000 7fb5a4e0d700 1 -- 172.21.15.144:0/4095286971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a01a0250 con 0x7fb5a01023e0 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.872+0000 7fb5a4e0d700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 0x7fb5a01985f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb5940047b0 tx=0x7fb594009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.872+0000 7fb59dffb700 1 -- 172.21.15.144:0/4095286971 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594026070 con 0x7fb5a01023e0 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.872+0000 7fb59dffb700 1 -- 172.21.15.144:0/4095286971 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb594004810 con 0x7fb5a01023e0 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.872+0000 7fb59dffb700 1 -- 172.21.15.144:0/4095286971 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594005390 con 0x7fb5a01023e0 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.872+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a01a04e0 con 0x7fb5a01023e0 2024-03-26T17:31:26.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.872+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a01a0960 con 0x7fb5a01023e0 2024-03-26T17:31:26.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.873+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a0069070 con 0x7fb5a01023e0 2024-03-26T17:31:26.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.874+0000 7fb59dffb700 1 -- 172.21.15.144:0/4095286971 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 15) ==== 137799+0+0 (secure 0 0 0) 0x7fb594029070 con 0x7fb5a01023e0 2024-03-26T17:31:26.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.874+0000 7fb59dffb700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb5900a2fa0 0x7fb5900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:26.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.875+0000 7fb59dffb700 1 -- 172.21.15.144:0/4095286971 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fb5940ce800 con 0x7fb5a01023e0 2024-03-26T17:31:26.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.875+0000 7fb59ffff700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb5900a2fa0 0x7fb5900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:26.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.875+0000 7fb59ffff700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb5900a2fa0 0x7fb5900a5460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb58c005950 tx=0x7fb58c005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:31:26.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:26.879+0000 7fb59dffb700 1 -- 172.21.15.144:0/4095286971 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb59405d740 con 0x7fb5a01023e0 2024-03-26T17:31:27.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:27.085+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb5a00627f0 con 0x7fb5900a2fa0 2024-03-26T17:31:27.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:27 smithi144 ceph-mon[36095]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T17:31:27.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:27 smithi144 ceph-mon[36095]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:27.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:27.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:27.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:27.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:27 smithi105 ceph-mon[32280]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T17:31:27.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:27 smithi105 ceph-mon[32280]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:27.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:27.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:27.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:27.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:27 smithi135 ceph-mon[35636]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T17:31:27.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:27 smithi135 ceph-mon[35636]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:27.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:27.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:27.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:28 smithi144 ceph-mon[36095]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:28 smithi105 ceph-mon[32280]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:28 smithi135 ceph-mon[35636]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:29.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:29 smithi144 ceph-mon[36095]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:29.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:29 smithi105 ceph-mon[32280]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:29.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:29 smithi135 ceph-mon[35636]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:30.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:30 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/3596313671' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]: dispatch 2024-03-26T17:31:30.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:30 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]: dispatch 2024-03-26T17:31:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:30 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]': finished 2024-03-26T17:31:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:30 smithi144 ceph-mon[36095]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T17:31:30.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:30.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:30 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/3596313671' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]: dispatch 2024-03-26T17:31:30.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:30 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]: dispatch 2024-03-26T17:31:30.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:30 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]': finished 2024-03-26T17:31:30.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:30 smithi105 ceph-mon[32280]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T17:31:30.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:30 smithi135 ceph-mon[35636]: from='client.? 172.21.15.144:0/3596313671' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]: dispatch 2024-03-26T17:31:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:30 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]: dispatch 2024-03-26T17:31:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:30 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f434f0a6-ad51-45e8-b9e9-411cead27014"}]': finished 2024-03-26T17:31:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:30 smithi135 ceph-mon[35636]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T17:31:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:31.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:31 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/3209488261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:31.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:31 smithi144 ceph-mon[36095]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:31.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:31 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/3209488261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:31.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:31 smithi105 ceph-mon[32280]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:31.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:31 smithi135 ceph-mon[35636]: from='client.? 172.21.15.144:0/3209488261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:31.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:31 smithi135 ceph-mon[35636]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:33.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:33 smithi144 ceph-mon[36095]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:33.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:33 smithi105 ceph-mon[32280]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:33.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:33 smithi135 ceph-mon[35636]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:35.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:35 smithi105 ceph-mon[32280]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:35.700 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:35 smithi144 ceph-mon[36095]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:35.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:35 smithi135 ceph-mon[35636]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:37.855 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:37 smithi144 ceph-mon[36095]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:37.855 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T17:31:37.855 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:37.904 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:37 smithi105 ceph-mon[32280]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:37.904 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T17:31:37.904 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:37.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:37 smithi135 ceph-mon[35636]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:37.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T17:31:37.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:38.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:38 smithi144 ceph-mon[36095]: Deploying daemon osd.2 on smithi144 2024-03-26T17:31:38.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:38 smithi105 ceph-mon[32280]: Deploying daemon osd.2 on smithi144 2024-03-26T17:31:38.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:38 smithi135 ceph-mon[35636]: Deploying daemon osd.2 on smithi144 2024-03-26T17:31:39.842 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:39 smithi144 ceph-mon[36095]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:39.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:39 smithi105 ceph-mon[32280]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:39.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:39 smithi135 ceph-mon[35636]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:41.710 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:41 smithi144 ceph-mon[36095]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:41.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:41 smithi105 ceph-mon[32280]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:41.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:41 smithi135 ceph-mon[35636]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:43.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:43.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:43 smithi144 ceph-mon[36095]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:43.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:43.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:43.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:43.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:43 smithi105 ceph-mon[32280]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:43.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:43.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:43.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:43.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:43 smithi135 ceph-mon[35636]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:45.805 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:45 smithi144 ceph-mon[36095]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:45.805 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:45.805 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:45.805 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:45.805 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:45.805 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:45 smithi105 ceph-mon[32280]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:45.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:45.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:45.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:45.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:45.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:45.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.900+0000 7fb59dffb700 1 -- 172.21.15.144:0/4095286971 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb5a00627f0 con 0x7fb5900a2fa0 2024-03-26T17:31:45.895 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 2 on host 'smithi144' 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb5900a2fa0 msgr2=0x7fb5900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb5900a2fa0 0x7fb5900a5460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb58c005950 tx=0x7fb58c005cf0 comp rx=0 tx=0).stop 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 msgr2=0x7fb5a01985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 0x7fb5a01985f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb5940047b0 tx=0x7fb594009ee0 comp rx=0 tx=0).stop 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 shutdown_connections 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb5900a2fa0 0x7fb5900a5460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5a0103f80 0x7fb5a019fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5a01023e0 0x7fb5a01985f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 --2- 172.21.15.144:0/4095286971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5a01035e0 0x7fb5a0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 >> 172.21.15.144:0/4095286971 conn(0x7fb5a00fd9c0 msgr2=0x7fb5a0106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 shutdown_connections 2024-03-26T17:31:45.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:31:45.902+0000 7fb5a7071700 1 -- 172.21.15.144:0/4095286971 wait complete. 2024-03-26T17:31:45.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:45 smithi135 ceph-mon[35636]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:45.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:45.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:45.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:45.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:45.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:46.503 DEBUG:teuthology.orchestra.run.smithi144:osd.2> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.2.service 2024-03-26T17:31:46.505 INFO:tasks.cephadm:Deploying osd.3 on smithi105 with /dev/vg_nvme/lv_3... 2024-03-26T17:31:46.505 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T17:31:46.751 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:31:47.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:47.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:47.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:47.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:46 smithi144 ceph-mon[36095]: from='osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:31:47.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:46 smithi144 ceph-mon[36095]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:31:47.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:46 smithi144 ceph-mon[36095]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:47.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:47.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:47.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:47.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:46 smithi105 ceph-mon[32280]: from='osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:31:47.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:46 smithi105 ceph-mon[32280]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:31:47.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:46 smithi105 ceph-mon[32280]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:47.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:47.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:47.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:31:47.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:46 smithi135 ceph-mon[35636]: from='osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:31:47.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:46 smithi135 ceph-mon[35636]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T17:31:47.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:46 smithi135 ceph-mon[35636]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:48.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:47 smithi144 ceph-mon[36095]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T17:31:48.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:47 smithi144 ceph-mon[36095]: from='osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:31:48.126 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:47 smithi144 ceph-mon[36095]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T17:31:48.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:48.127 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:47 smithi144 ceph-mon[36095]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:31:48.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:47 smithi105 ceph-mon[32280]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T17:31:48.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:47 smithi105 ceph-mon[32280]: from='osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:31:48.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:47 smithi105 ceph-mon[32280]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T17:31:48.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:48.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:47 smithi105 ceph-mon[32280]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:31:48.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:47 smithi135 ceph-mon[35636]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T17:31:48.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:47 smithi135 ceph-mon[35636]: from='osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:31:48.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:47 smithi135 ceph-mon[35636]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T17:31:48.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:48.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:47 smithi135 ceph-mon[35636]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:31:48.773 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T17:31:48.773 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T17:31:48.773 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-26T17:31:48.773 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-26T17:31:48.774 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115054 s, 911 MB/s 2024-03-26T17:31:48.774 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-26T17:31:49.088 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:48 smithi105 ceph-mon[32280]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:31:49.088 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:48 smithi105 ceph-mon[32280]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T17:31:49.088 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:48 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:49.088 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:48 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:49.088 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:48 smithi105 ceph-mon[32280]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:49.089 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:48 smithi105 ceph-mon[32280]: from='osd.2 ' entity='osd.2' 2024-03-26T17:31:49.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:48 smithi144 ceph-mon[36095]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:31:49.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:48 smithi144 ceph-mon[36095]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T17:31:49.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:48 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:49.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:48 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:49.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:48 smithi144 ceph-mon[36095]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:49.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:48 smithi144 ceph-mon[36095]: from='osd.2 ' entity='osd.2' 2024-03-26T17:31:49.101 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2[41252]: 2024-03-26T17:31:48.790+0000 7f87b7266700 -1 osd.2 0 waiting for initial osdmap 2024-03-26T17:31:49.101 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2[41252]: 2024-03-26T17:31:48.956+0000 7f87ae850700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:31:49.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:48 smithi135 ceph-mon[35636]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:31:49.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:48 smithi135 ceph-mon[35636]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T17:31:49.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:48 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:49.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:48 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:49.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:48 smithi135 ceph-mon[35636]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:49.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:48 smithi135 ceph-mon[35636]: from='osd.2 ' entity='osd.2' 2024-03-26T17:31:49.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_3 2024-03-26T17:31:49.658 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:31:49.975 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:31:49.975 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:31:49.975 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:49.975 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi144 2024-03-26T17:31:49.975 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:49.975 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:49.976 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:49.976 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi144 to 17088M 2024-03-26T17:31:49.976 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:49.976 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:49.976 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:49.976 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:49.976 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:50.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:31:50.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:31:50.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:50.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi144 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi144 to 17088M 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:50.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:50.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi144 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi144 to 17088M 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:31:50.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:50.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:51.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 -- 172.21.15.105:0/3137150668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0108140 msgr2=0x7f22b010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:51.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 --2- 172.21.15.105:0/3137150668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0108140 0x7f22b010a5b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22a800b0a0 tx=0x7f22a800b3b0 comp rx=0 tx=0).stop 2024-03-26T17:31:51.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 -- 172.21.15.105:0/3137150668 shutdown_connections 2024-03-26T17:31:51.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 --2- 172.21.15.105:0/3137150668 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0108140 0x7f22b010a5b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:51.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 --2- 172.21.15.105:0/3137150668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22b0102ee0 0x7f22b01052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:51.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 --2- 172.21.15.105:0/3137150668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b0105810 0x7f22b0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:51.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 -- 172.21.15.105:0/3137150668 >> 172.21.15.105:0/3137150668 conn(0x7f22b00fc910 msgr2=0x7f22b00fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:31:51.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 -- 172.21.15.105:0/3137150668 shutdown_connections 2024-03-26T17:31:51.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 -- 172.21.15.105:0/3137150668 wait complete. 2024-03-26T17:31:51.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:50 smithi105 ceph-mon[32280]: osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] boot 2024-03-26T17:31:51.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:50 smithi105 ceph-mon[32280]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T17:31:51.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:51.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:50 smithi105 ceph-mon[32280]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:51.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:31:51.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 Processor -- start 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.148+0000 7f22b64fc700 1 -- start start 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22b64fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0102ee0 0x7f22b01027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22b64fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22b0105810 0x7f22b0100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22b64fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b0108140 0x7f22b01013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22b64fc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010dec0 con 0x7f22b0102ee0 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22b64fc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010dd40 con 0x7f22b0105810 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22b64fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010e040 con 0x7f22b0108140 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0102ee0 0x7f22b01027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0102ee0 0x7f22b01027d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47068/0 (socket says 172.21.15.105:47068) 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 -- 172.21.15.105:0/1578125141 learned_addr learned my addr 172.21.15.105:0/1578125141 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22af7fe700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22b0105810 0x7f22b0100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22b4a99700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b0108140 0x7f22b01013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 -- 172.21.15.105:0/1578125141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b0108140 msgr2=0x7f22b01013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:51.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b0108140 0x7f22b01013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 -- 172.21.15.105:0/1578125141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22b0105810 msgr2=0x7f22b0100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22b0105810 0x7f22b0100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 -- 172.21.15.105:0/1578125141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b0101c10 con 0x7f22b0102ee0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22b4a99700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b0108140 0x7f22b01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.149+0000 7f22affff700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0102ee0 0x7f22b01027d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f22a400c9b0 tx=0x7f22a400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.150+0000 7f22ad7fa700 1 -- 172.21.15.105:0/1578125141 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4007b00 con 0x7f22b0102ee0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.150+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b01a25e0 con 0x7f22b0102ee0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.150+0000 7f22ad7fa700 1 -- 172.21.15.105:0/1578125141 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22a4007ca0 con 0x7f22b0102ee0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.150+0000 7f22ad7fa700 1 -- 172.21.15.105:0/1578125141 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4018660 con 0x7f22b0102ee0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.150+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b01a2ba0 con 0x7f22b0102ee0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.151+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b0069070 con 0x7f22b0102ee0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.152+0000 7f22ad7fa700 1 -- 172.21.15.105:0/1578125141 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 15) ==== 137799+0+0 (secure 0 0 0) 0x7f22a400f450 con 0x7f22b0102ee0 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.152+0000 7f22ad7fa700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22980a2f20 0x7f22980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:31:51.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.152+0000 7f22af7fe700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22980a2f20 0x7f22980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:31:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.152+0000 7f22ad7fa700 1 -- 172.21.15.105:0/1578125141 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f22a40cdf20 con 0x7f22b0102ee0 2024-03-26T17:31:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.153+0000 7f22af7fe700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22980a2f20 0x7f22980a53e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009a10 tx=0x7f22a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:31:51.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.157+0000 7f22ad7fa700 1 -- 172.21.15.105:0/1578125141 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f22a4014030 con 0x7f22b0102ee0 2024-03-26T17:31:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:50 smithi135 ceph-mon[35636]: osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] boot 2024-03-26T17:31:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:50 smithi135 ceph-mon[35636]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T17:31:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:50 smithi135 ceph-mon[35636]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:31:51.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:50 smithi144 ceph-mon[36095]: osd.2 [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] boot 2024-03-26T17:31:51.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:50 smithi144 ceph-mon[36095]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T17:31:51.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T17:31:51.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:50 smithi144 ceph-mon[36095]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T17:31:51.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:31:51.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:51.361+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f22b00627f0 con 0x7f22980a2f20 2024-03-26T17:31:52.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T17:31:52.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:51 smithi105 ceph-mon[32280]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T17:31:52.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:31:52.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:51 smithi105 ceph-mon[32280]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:52.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:52.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:52.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:52.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T17:31:52.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:51 smithi135 ceph-mon[35636]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T17:31:52.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:31:52.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:51 smithi135 ceph-mon[35636]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:52.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:52.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:52.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:52.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T17:31:52.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:51 smithi144 ceph-mon[36095]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T17:31:52.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T17:31:52.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:51 smithi144 ceph-mon[36095]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:31:52.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:31:52.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:31:52.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:31:52.487 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:31:52 smithi105 sudo[51319]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:31:52.487 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:31:52 smithi105 sudo[51319]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:52.487 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:31:52 smithi105 sudo[51319]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:52.487 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:31:52 smithi105 sudo[51319]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:52.885 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:31:52 smithi105 sudo[51322]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:31:52.886 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:31:52 smithi105 sudo[51322]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:52.886 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:31:52 smithi105 sudo[51322]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:52.886 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 17:31:52 smithi105 sudo[51322]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:53.008 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:52 smithi135 sudo[44098]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:31:53.008 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:52 smithi135 sudo[44098]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:53.008 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:52 smithi135 sudo[44098]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:53.008 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:52 smithi135 sudo[44098]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:53.008 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T17:31:53.008 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:52 smithi135 ceph-mon[35636]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T17:31:53.009 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:53.009 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:52 smithi135 ceph-mon[35636]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:53.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T17:31:53.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:52 smithi105 ceph-mon[32280]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T17:31:53.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:53.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:52 smithi105 ceph-mon[32280]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:53.274 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T17:31:53.274 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:52 smithi144 ceph-mon[36095]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T17:31:53.275 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:53.275 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:52 smithi144 ceph-mon[36095]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:53.417 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:53 smithi135 sudo[44101]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:31:53.417 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:53 smithi135 sudo[44101]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:53.417 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:53 smithi135 sudo[44101]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:53.417 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 17:31:53 smithi135 sudo[44101]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:53.529 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:53 smithi144 sudo[45280]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:31:53.530 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:53 smithi144 sudo[45280]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:53.530 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:53 smithi144 sudo[45280]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:53.530 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:53 smithi144 sudo[45280]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:53.850 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:53 smithi144 sudo[45283]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:31:53.850 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:53 smithi144 sudo[45283]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:53.850 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:53 smithi144 sudo[45283]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:53.850 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 17:31:53 smithi144 sudo[45283]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:54.062 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:53 smithi105 sudo[51383]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:31:54.062 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:53 smithi105 sudo[51383]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:54.062 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:53 smithi105 sudo[51383]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:54.062 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:53 smithi105 sudo[51383]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:54.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:31:54.095+0000 7f22ad7fa700 1 -- 172.21.15.105:0/1578125141 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f22a4098180 con 0x7f22b0102ee0 2024-03-26T17:31:54.325 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 ceph-mon[36095]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T17:31:54.326 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:54.326 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:54.326 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 ceph-mon[36095]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:54.326 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 sudo[45286]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:31:54.326 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 sudo[45286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:54.326 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 sudo[45286]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:54.326 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 sudo[45286]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:54.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:54 smithi105 sudo[51403]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:31:54.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:54 smithi105 sudo[51403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:54.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:54 smithi105 sudo[51403]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:54.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:54 smithi105 sudo[51403]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:54.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:54 smithi105 ceph-mon[32280]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T17:31:54.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:54.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:54.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:54 smithi105 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:54.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:54 smithi135 ceph-mon[35636]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T17:31:54.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:54.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:31:54.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:54 smithi135 ceph-mon[35636]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:54.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 sudo[45289]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:31:54.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 sudo[45289]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:54.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 sudo[45289]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:54.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:54 smithi144 sudo[45289]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:55.058 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:54 smithi135 sudo[44104]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T17:31:55.059 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:54 smithi135 sudo[44104]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:55.059 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:54 smithi135 sudo[44104]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:55.059 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:54 smithi135 sudo[44104]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:55.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3849617580' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52f7e5b-63cd-43ea-a2d9-5b3282dbb996"}]: dispatch 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3849617580' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f52f7e5b-63cd-43ea-a2d9-5b3282dbb996"}]': finished 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:31:55.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:55 smithi144 ceph-mon[36095]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:55.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3849617580' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52f7e5b-63cd-43ea-a2d9-5b3282dbb996"}]: dispatch 2024-03-26T17:31:55.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3849617580' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f52f7e5b-63cd-43ea-a2d9-5b3282dbb996"}]': finished 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:31:55.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:55 smithi105 ceph-mon[32280]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:55.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 sudo[44107]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T17:31:55.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 sudo[44107]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T17:31:55.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 sudo[44107]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T17:31:55.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 sudo[44107]: pam_unix(sudo:session): session closed for user root 2024-03-26T17:31:55.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:55.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3849617580' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f52f7e5b-63cd-43ea-a2d9-5b3282dbb996"}]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T17:31:55.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T17:31:55.419 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3849617580' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f52f7e5b-63cd-43ea-a2d9-5b3282dbb996"}]': finished 2024-03-26T17:31:55.419 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T17:31:55.419 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:31:55.419 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:55 smithi135 ceph-mon[35636]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T17:31:56.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:56 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3807824638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:56.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:56 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3807824638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:56.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:56 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3807824638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:31:57.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:57 smithi144 ceph-mon[36095]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:57.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:57 smithi105 ceph-mon[32280]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:57.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:57 smithi135 ceph-mon[35636]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:59.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:31:59 smithi144 ceph-mon[36095]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:59.883 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:31:59 smithi105 ceph-mon[32280]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:31:59.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:31:59 smithi135 ceph-mon[35636]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:01.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:01 smithi105 ceph-mon[32280]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:01 smithi144 ceph-mon[36095]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:01.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:01 smithi135 ceph-mon[35636]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:02.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T17:32:02.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:02.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T17:32:02.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:02.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T17:32:02.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:03.808 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:03 smithi105 ceph-mon[32280]: Deploying daemon osd.3 on smithi105 2024-03-26T17:32:03.808 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:03 smithi105 ceph-mon[32280]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:03.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:03 smithi144 ceph-mon[36095]: Deploying daemon osd.3 on smithi105 2024-03-26T17:32:03.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:03 smithi144 ceph-mon[36095]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:03.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:03 smithi135 ceph-mon[35636]: Deploying daemon osd.3 on smithi105 2024-03-26T17:32:03.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:03 smithi135 ceph-mon[35636]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:05.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:05 smithi144 ceph-mon[36095]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:05.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:05 smithi105 ceph-mon[32280]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:05.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:05 smithi135 ceph-mon[35636]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:07.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:07 smithi144 ceph-mon[36095]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:07.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:07.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:07.851 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:07.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:07 smithi105 ceph-mon[32280]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:07.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:07.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:07.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:07.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:07 smithi135 ceph-mon[35636]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:07.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:07.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:07.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:09.838 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:09 smithi105 ceph-mon[32280]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:09.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:09 smithi144 ceph-mon[36095]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:09 smithi135 ceph-mon[35636]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:10.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.766+0000 7f22ad7fa700 1 -- 172.21.15.105:0/1578125141 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f22b00627f0 con 0x7f22980a2f20 2024-03-26T17:32:10.760 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 3 on host 'smithi105' 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22980a2f20 msgr2=0x7f22980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22980a2f20 0x7f22980a53e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009a10 tx=0x7f22a0009190 comp rx=0 tx=0).stop 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0102ee0 msgr2=0x7f22b01027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0102ee0 0x7f22b01027d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f22a400c9b0 tx=0x7f22a400ccc0 comp rx=0 tx=0).stop 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 shutdown_connections 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22980a2f20 0x7f22980a53e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22b0108140 0x7f22b01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22b0102ee0 0x7f22b01027d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 --2- 172.21.15.105:0/1578125141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22b0105810 0x7f22b0100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:10.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 >> 172.21.15.105:0/1578125141 conn(0x7f22b00fc910 msgr2=0x7f22b0103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:32:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 shutdown_connections 2024-03-26T17:32:10.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:32:10.769+0000 7f22b64fc700 1 -- 172.21.15.105:0/1578125141 wait complete. 2024-03-26T17:32:11.414 DEBUG:teuthology.orchestra.run.smithi105:osd.3> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.3.service 2024-03-26T17:32:11.416 INFO:tasks.cephadm:Deploying osd.4 on smithi135 with /dev/vg_nvme/lv_3... 2024-03-26T17:32:11.416 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T17:32:11.659 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:32:12.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:11 smithi135 ceph-mon[35636]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:12.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:11 smithi135 ceph-mon[35636]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T17:32:12.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:11 smithi144 ceph-mon[36095]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:11 smithi144 ceph-mon[36095]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T17:32:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:11 smithi105 ceph-mon[32280]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:12.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:11 smithi105 ceph-mon[32280]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T17:32:12.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:12.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:13.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:12 smithi135 ceph-mon[35636]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T17:32:13.017 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:12 smithi135 ceph-mon[35636]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T17:32:13.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:12 smithi135 ceph-mon[35636]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:32:13.018 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:12 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:13.019 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:12 smithi105 ceph-mon[32280]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T17:32:13.019 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:12 smithi105 ceph-mon[32280]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T17:32:13.019 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:12 smithi105 ceph-mon[32280]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:32:13.019 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:12 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:13.019 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 17:32:12 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T17:32:12.765+0000 7fd9fca1f700 -1 osd.3 0 waiting for initial osdmap 2024-03-26T17:32:13.019 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 17:32:12 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T17:32:12.770+0000 7fd9f8812700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:32:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:12 smithi144 ceph-mon[36095]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T17:32:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:12 smithi144 ceph-mon[36095]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T17:32:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:12 smithi144 ceph-mon[36095]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:32:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:12 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:13.568 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T17:32:13.568 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T17:32:13.569 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-03-26T17:32:13.569 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-03-26T17:32:13.569 INFO:teuthology.orchestra.run.smithi135.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121035 s, 866 MB/s 2024-03-26T17:32:13.569 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-03-26T17:32:14.086 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:13 smithi105 ceph-mon[32280]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:14.086 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:13 smithi105 ceph-mon[32280]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:32:14.086 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:13 smithi105 ceph-mon[32280]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T17:32:14.086 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.086 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.086 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:13 smithi105 ceph-mon[32280]: osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] boot 2024-03-26T17:32:14.086 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:13 smithi105 ceph-mon[32280]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T17:32:14.086 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:13 smithi144 ceph-mon[36095]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:14.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:13 smithi144 ceph-mon[36095]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:32:14.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:13 smithi144 ceph-mon[36095]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T17:32:14.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:13 smithi144 ceph-mon[36095]: osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] boot 2024-03-26T17:32:14.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:13 smithi144 ceph-mon[36095]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T17:32:14.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:13 smithi135 ceph-mon[35636]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-26T17:32:14.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:13 smithi135 ceph-mon[35636]: from='osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:32:14.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:13 smithi135 ceph-mon[35636]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T17:32:14.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:13 smithi135 ceph-mon[35636]: osd.3 [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] boot 2024-03-26T17:32:14.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:13 smithi135 ceph-mon[35636]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T17:32:14.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T17:32:14.196 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi135:vg_nvme/lv_3 2024-03-26T17:32:14.434 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:32:15.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:32:15.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:32:15.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T17:32:15.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:15.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi105 2024-03-26T17:32:15.510 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi105 to 8544M 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:15.511 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:32:15.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi105 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi105 to 8544M 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:15.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:15.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:15.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:32:15.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:32:15.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T17:32:15.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi105 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi105 to 8544M 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:15.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:16.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.035+0000 7f35649b6700 1 -- 172.21.15.135:0/414522869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3560104070 msgr2=0x7f356010a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:16.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.035+0000 7f35649b6700 1 --2- 172.21.15.135:0/414522869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3560104070 0x7f356010a920 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f355800b0a0 tx=0x7f355800b3b0 comp rx=0 tx=0).stop 2024-03-26T17:32:16.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 -- 172.21.15.135:0/414522869 shutdown_connections 2024-03-26T17:32:16.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 --2- 172.21.15.135:0/414522869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3560104070 0x7f356010a920 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:16.032 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 --2- 172.21.15.135:0/414522869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35601006f0 0x7f3560100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 --2- 172.21.15.135:0/414522869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35601036d0 0x7f3560103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 -- 172.21.15.135:0/414522869 >> 172.21.15.135:0/414522869 conn(0x7f35600fc050 msgr2=0x7f35600fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 -- 172.21.15.135:0/414522869 shutdown_connections 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 -- 172.21.15.135:0/414522869 wait complete. 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 Processor -- start 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 -- start start 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35601006f0 0x7f3560103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35601036d0 0x7f3560101680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.036+0000 7f35649b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3560104070 0x7f3560101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f35649b6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f356010df30 con 0x7f35601036d0 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f35649b6700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f356010ddb0 con 0x7f35601006f0 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f35649b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f356010e0b0 con 0x7f3560104070 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35601006f0 0x7f3560103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:32:16.033 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35601006f0 0x7f3560103030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:36874/0 (socket says 172.21.15.135:36874) 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 -- 172.21.15.135:0/3360152839 learned_addr learned my addr 172.21.15.135:0/3360152839 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355dd9b700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35601036d0 0x7f3560101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 -- 172.21.15.135:0/3360152839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3560104070 msgr2=0x7f3560101bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355ed9d700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3560104070 0x7f3560101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3560104070 0x7f3560101bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 -- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35601036d0 msgr2=0x7f3560101680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35601036d0 0x7f3560101680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 -- 172.21.15.135:0/3360152839 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3560102470 con 0x7f35601006f0 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355dd9b700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35601036d0 0x7f3560101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f355e59c700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35601006f0 0x7f3560103030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f355000b500 tx=0x7f355000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f354f7fe700 1 -- 172.21.15.135:0/3360152839 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550015020 con 0x7f35601006f0 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f354f7fe700 1 -- 172.21.15.135:0/3360152839 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3550011ce0 con 0x7f35601006f0 2024-03-26T17:32:16.034 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35601a2610 con 0x7f35601006f0 2024-03-26T17:32:16.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f354f7fe700 1 -- 172.21.15.135:0/3360152839 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550010d60 con 0x7f35601006f0 2024-03-26T17:32:16.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.037+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35601a2b50 con 0x7f35601006f0 2024-03-26T17:32:16.035 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.038+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3560069070 con 0x7f35601006f0 2024-03-26T17:32:16.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.039+0000 7f354f7fe700 1 -- 172.21.15.135:0/3360152839 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f3550018020 con 0x7f35601006f0 2024-03-26T17:32:16.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.040+0000 7f354f7fe700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35480a2fb0 0x7f35480a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:32:16.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.040+0000 7f354f7fe700 1 -- 172.21.15.135:0/3360152839 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f35500cdfc0 con 0x7f35601006f0 2024-03-26T17:32:16.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.040+0000 7f355dd9b700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35480a2fb0 0x7f35480a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:32:16.036 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.041+0000 7f355dd9b700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35480a2fb0 0x7f35480a5470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f3554005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:32:16.038 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.044+0000 7f354f7fe700 1 -- 172.21.15.135:0/3360152839 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f355001f020 con 0x7f35601006f0 2024-03-26T17:32:16.243 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:16.248+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f35600627f0 con 0x7f35480a2fb0 2024-03-26T17:32:16.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:16 smithi144 ceph-mon[36095]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T17:32:16.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:16 smithi105 ceph-mon[32280]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T17:32:16.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:16 smithi135 ceph-mon[35636]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T17:32:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:17 smithi135 ceph-mon[35636]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:32:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:32:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:32:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:17 smithi135 ceph-mon[35636]: pgmap v102: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:17.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:17 smithi144 ceph-mon[36095]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:32:17.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:32:17.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:32:17.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:17.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:17 smithi144 ceph-mon[36095]: pgmap v102: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:17.646 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:17 smithi105 ceph-mon[32280]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:32:17.646 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:32:17.646 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:32:17.646 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:17.646 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:17 smithi105 ceph-mon[32280]: pgmap v102: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:18.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:18 smithi135 ceph-mon[35636]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:32:18.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:18 smithi135 ceph-mon[35636]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T17:32:18.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:18 smithi144 ceph-mon[36095]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:32:18.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:18 smithi144 ceph-mon[36095]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T17:32:18.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:18 smithi105 ceph-mon[32280]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:32:18.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:18 smithi105 ceph-mon[32280]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T17:32:19.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:19 smithi144 ceph-mon[36095]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T17:32:19.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:19 smithi144 ceph-mon[36095]: pgmap v105: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:19.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:19 smithi144 ceph-mon[36095]: from='client.? 172.21.15.135:0/192713083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]: dispatch 2024-03-26T17:32:19.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:19 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]: dispatch 2024-03-26T17:32:19.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:19 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]': finished 2024-03-26T17:32:19.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:19 smithi144 ceph-mon[36095]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T17:32:19.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:19.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:19 smithi105 ceph-mon[32280]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T17:32:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:19 smithi105 ceph-mon[32280]: pgmap v105: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:19 smithi105 ceph-mon[32280]: from='client.? 172.21.15.135:0/192713083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]: dispatch 2024-03-26T17:32:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:19 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]: dispatch 2024-03-26T17:32:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:19 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]': finished 2024-03-26T17:32:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:19 smithi105 ceph-mon[32280]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T17:32:19.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:19.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:19 smithi135 ceph-mon[35636]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T17:32:19.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:19 smithi135 ceph-mon[35636]: pgmap v105: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:19.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:19 smithi135 ceph-mon[35636]: from='client.? 172.21.15.135:0/192713083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]: dispatch 2024-03-26T17:32:19.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:19 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]: dispatch 2024-03-26T17:32:19.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:19 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9450083-abb4-4382-b2eb-96f679b03bdb"}]': finished 2024-03-26T17:32:19.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:19 smithi135 ceph-mon[35636]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T17:32:19.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:20.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:20 smithi144 ceph-mon[36095]: from='client.? 172.21.15.135:0/3891396472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:32:20.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:20 smithi105 ceph-mon[32280]: from='client.? 172.21.15.135:0/3891396472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:32:20.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:20 smithi135 ceph-mon[35636]: from='client.? 172.21.15.135:0/3891396472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:32:21.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:21 smithi144 ceph-mon[36095]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:21.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:21 smithi105 ceph-mon[32280]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:21.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:21 smithi135 ceph-mon[35636]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:32:22.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:22 smithi105 ceph-mon[32280]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T17:32:22.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:22 smithi105 ceph-mon[32280]: Cluster is now healthy 2024-03-26T17:32:22.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:22 smithi135 ceph-mon[35636]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T17:32:22.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:22 smithi135 ceph-mon[35636]: Cluster is now healthy 2024-03-26T17:32:23.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:22 smithi144 ceph-mon[36095]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T17:32:23.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:22 smithi144 ceph-mon[36095]: Cluster is now healthy 2024-03-26T17:32:23.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:23 smithi105 ceph-mon[32280]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:32:23.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:23 smithi135 ceph-mon[35636]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:32:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:23 smithi144 ceph-mon[36095]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:32:25.863 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:25 smithi135 ceph-mon[35636]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T17:32:25.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:25 smithi105 ceph-mon[32280]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T17:32:26.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:25 smithi144 ceph-mon[36095]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-26T17:32:26.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T17:32:26.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:26.910 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T17:32:26.911 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:27.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T17:32:27.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:27.810 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:27 smithi135 ceph-mon[35636]: Deploying daemon osd.4 on smithi135 2024-03-26T17:32:27.810 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:27 smithi135 ceph-mon[35636]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T17:32:27.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:27 smithi105 ceph-mon[32280]: Deploying daemon osd.4 on smithi135 2024-03-26T17:32:27.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:27 smithi105 ceph-mon[32280]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T17:32:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:27 smithi144 ceph-mon[36095]: Deploying daemon osd.4 on smithi135 2024-03-26T17:32:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:27 smithi144 ceph-mon[36095]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T17:32:29.797 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:29 smithi135 ceph-mon[35636]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:32:30.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:29 smithi144 ceph-mon[36095]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:32:30.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:29 smithi105 ceph-mon[32280]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:32:31.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:31 smithi135 ceph-mon[35636]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-03-26T17:32:31.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:31.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:31.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:32.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:31 smithi144 ceph-mon[36095]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-03-26T17:32:32.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:32.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:32.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:32.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:31 smithi105 ceph-mon[32280]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-03-26T17:32:32.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:32.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:32.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:33.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:33 smithi135 ceph-mon[35636]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:32:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:33 smithi144 ceph-mon[36095]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:32:34.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:33 smithi105 ceph-mon[32280]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:32:35.011 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:34 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.011 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:34 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.011 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:34 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:35.012 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:34 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:35.012 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:34 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.012 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:34 smithi135 ceph-mon[35636]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:35.012 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.017+0000 7f354f7fe700 1 -- 172.21.15.135:0/3360152839 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f35600627f0 con 0x7f35480a2fb0 2024-03-26T17:32:35.012 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 4 on host 'smithi135' 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.019+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35480a2fb0 msgr2=0x7f35480a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.019+0000 7f35649b6700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35480a2fb0 0x7f35480a5470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f3554005e20 comp rx=0 tx=0).stop 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.019+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35601006f0 msgr2=0x7f3560103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.019+0000 7f35649b6700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35601006f0 0x7f3560103030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f355000b500 tx=0x7f355000b8c0 comp rx=0 tx=0).stop 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.020+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 shutdown_connections 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.020+0000 7f35649b6700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35480a2fb0 0x7f35480a5470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.020+0000 7f35649b6700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3560104070 0x7f3560101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.020+0000 7f35649b6700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35601006f0 0x7f3560103030 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.020+0000 7f35649b6700 1 --2- 172.21.15.135:0/3360152839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35601036d0 0x7f3560101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.020+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 >> 172.21.15.135:0/3360152839 conn(0x7f35600fc050 msgr2=0x7f3560106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:32:35.014 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.020+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 shutdown_connections 2024-03-26T17:32:35.015 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:32:35.020+0000 7f35649b6700 1 -- 172.21.15.135:0/3360152839 wait complete. 2024-03-26T17:32:35.273 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:34 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.273 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:34 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.273 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:34 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:35.273 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:34 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:35.273 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:34 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.274 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:34 smithi144 ceph-mon[36095]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:35.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:34 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:34 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:34 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:35.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:34 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:35.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:34 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:35.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:34 smithi105 ceph-mon[32280]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:35.690 DEBUG:teuthology.orchestra.run.smithi135:osd.4> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.4.service 2024-03-26T17:32:35.693 INFO:tasks.cephadm:Deploying osd.5 on smithi144 with /dev/vg_nvme/lv_3... 2024-03-26T17:32:35.693 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T17:32:35.931 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:32:36.226 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:36.226 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:36.226 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:36.226 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:35 smithi135 ceph-mon[35636]: from='osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:32:36.226 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:35 smithi135 ceph-mon[35636]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:32:36.237 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:36.237 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:36.237 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:36.237 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:35 smithi144 ceph-mon[36095]: from='osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:32:36.237 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:35 smithi144 ceph-mon[36095]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:32:36.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:36.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:36.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:36.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:35 smithi105 ceph-mon[32280]: from='osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:32:36.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:35 smithi105 ceph-mon[32280]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T17:32:37.126 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:36 smithi135 ceph-mon[35636]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T17:32:37.126 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:36 smithi135 ceph-mon[35636]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T17:32:37.126 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:36 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:37.126 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:36 smithi135 ceph-mon[35636]: from='osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:32:37.126 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:36 smithi135 ceph-mon[35636]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:32:37.126 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:36 smithi135 ceph-mon[35636]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:37.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:36 smithi144 ceph-mon[36095]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T17:32:37.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:36 smithi144 ceph-mon[36095]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T17:32:37.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:36 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:37.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:36 smithi144 ceph-mon[36095]: from='osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:32:37.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:36 smithi144 ceph-mon[36095]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:32:37.131 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:36 smithi144 ceph-mon[36095]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:37.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:36 smithi105 ceph-mon[32280]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T17:32:37.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:36 smithi105 ceph-mon[32280]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T17:32:37.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:36 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:37.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:36 smithi105 ceph-mon[32280]: from='osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:32:37.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:36 smithi105 ceph-mon[32280]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:32:37.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:36 smithi105 ceph-mon[32280]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:37.686 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T17:32:37.686 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T17:32:37.686 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-26T17:32:37.686 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-26T17:32:37.686 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116515 s, 900 MB/s 2024-03-26T17:32:37.686 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-26T17:32:38.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:38 smithi135 ceph-mon[35636]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:32:38.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:38 smithi135 ceph-mon[35636]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T17:32:38.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:38 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:38.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:38 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:38.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:38 smithi135 ceph-mon[35636]: from='osd.4 ' entity='osd.4' 2024-03-26T17:32:38.168 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 17:32:37 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4[46915]: 2024-03-26T17:32:37.903+0000 7f3129460700 -1 osd.4 0 waiting for initial osdmap 2024-03-26T17:32:38.168 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 17:32:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4[46915]: 2024-03-26T17:32:38.116+0000 7f312124b700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:32:38.255 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:38 smithi144 ceph-mon[36095]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:32:38.255 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:38 smithi144 ceph-mon[36095]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T17:32:38.255 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:38 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:38.255 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:38 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:38.255 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:38 smithi144 ceph-mon[36095]: from='osd.4 ' entity='osd.4' 2024-03-26T17:32:38.283 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-03-26T17:32:38.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:37 smithi105 ceph-mon[32280]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:32:38.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:37 smithi105 ceph-mon[32280]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T17:32:38.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:38.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:38 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:38.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:38 smithi105 ceph-mon[32280]: from='osd.4 ' entity='osd.4' 2024-03-26T17:32:38.518 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:32:39.312 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi135 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:39.313 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:39.314 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.314 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:39.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi135 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:39.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:39.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:39.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi135 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:39.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:40.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.084+0000 7fbc88951700 1 -- 172.21.15.144:0/2742693960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc841024e0 msgr2=0x7fbc84102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.084+0000 7fbc88951700 1 --2- 172.21.15.144:0/2742693960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc841024e0 0x7fbc84102940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009990 tx=0x7fbc78009ca0 comp rx=0 tx=0).stop 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.084+0000 7fbc88951700 1 -- 172.21.15.144:0/2742693960 shutdown_connections 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.084+0000 7fbc88951700 1 --2- 172.21.15.144:0/2742693960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84102e80 0x7fbc84109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.084+0000 7fbc88951700 1 --2- 172.21.15.144:0/2742693960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc841012e0 0x7fbc84101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.084+0000 7fbc88951700 1 --2- 172.21.15.144:0/2742693960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc841024e0 0x7fbc84102940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.084+0000 7fbc88951700 1 -- 172.21.15.144:0/2742693960 >> 172.21.15.144:0/2742693960 conn(0x7fbc840fc8a0 msgr2=0x7fbc840fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.085+0000 7fbc88951700 1 -- 172.21.15.144:0/2742693960 shutdown_connections 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.085+0000 7fbc88951700 1 -- 172.21.15.144:0/2742693960 wait complete. 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.085+0000 7fbc88951700 1 Processor -- start 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.085+0000 7fbc88951700 1 -- start start 2024-03-26T17:32:40.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc88951700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc841012e0 0x7fbc84197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc88951700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc841024e0 0x7fbc84197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc88951700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84102e80 0x7fbc8419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc88951700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410cd60 con 0x7fbc841024e0 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc88951700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410cbe0 con 0x7fbc841012e0 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc88951700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410cee0 con 0x7fbc84102e80 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84102e80 0x7fbc8419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84102e80 0x7fbc8419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42886/0 (socket says 172.21.15.144:42886) 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 -- 172.21.15.144:0/131760174 learned_addr learned my addr 172.21.15.144:0/131760174 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc81d9b700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc841024e0 0x7fbc84197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc8259c700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc841012e0 0x7fbc84197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:32:40.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 -- 172.21.15.144:0/131760174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc841012e0 msgr2=0x7fbc84197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc841012e0 0x7fbc84197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 -- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc841024e0 msgr2=0x7fbc84197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc841024e0 0x7fbc84197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 -- 172.21.15.144:0/131760174 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8419efa0 con 0x7fbc84102e80 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc82d9d700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84102e80 0x7fbc8419e900 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c007f40 tx=0x7fbc7c00f5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc737fe700 1 -- 172.21.15.144:0/131760174 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c016030 con 0x7fbc84102e80 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8406ab30 con 0x7fbc84102e80 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.086+0000 7fbc737fe700 1 -- 172.21.15.144:0/131760174 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc7c00fd30 con 0x7fbc84102e80 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.087+0000 7fbc737fe700 1 -- 172.21.15.144:0/131760174 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c012db0 con 0x7fbc84102e80 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.087+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8406b070 con 0x7fbc84102e80 2024-03-26T17:32:40.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.087+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc84069070 con 0x7fbc84102e80 2024-03-26T17:32:40.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.089+0000 7fbc737fe700 1 -- 172.21.15.144:0/131760174 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fbc7c004c40 con 0x7fbc84102e80 2024-03-26T17:32:40.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.090+0000 7fbc737fe700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc6c0a2fb0 0x7fbc6c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:32:40.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.090+0000 7fbc737fe700 1 -- 172.21.15.144:0/131760174 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fbc7c0d0a10 con 0x7fbc84102e80 2024-03-26T17:32:40.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.090+0000 7fbc8259c700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc6c0a2fb0 0x7fbc6c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:32:40.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.091+0000 7fbc8259c700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc6c0a2fb0 0x7fbc6c0a5470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc84102340 tx=0x7fbc7400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:32:40.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.094+0000 7fbc737fe700 1 -- 172.21.15.144:0/131760174 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbc7c09e8b0 con 0x7fbc84102e80 2024-03-26T17:32:40.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:40.296+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbc8406b2d0 con 0x7fbc6c0a2fb0 2024-03-26T17:32:40.291 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:40 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi135 to 8544M 2024-03-26T17:32:40.291 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:40 smithi144 ceph-mon[36095]: osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] boot 2024-03-26T17:32:40.291 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:40 smithi144 ceph-mon[36095]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T17:32:40.291 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:40.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:40 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi135 to 8544M 2024-03-26T17:32:40.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:40 smithi105 ceph-mon[32280]: osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] boot 2024-03-26T17:32:40.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:40 smithi105 ceph-mon[32280]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T17:32:40.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:40.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:40 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi135 to 8544M 2024-03-26T17:32:40.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:40 smithi135 ceph-mon[35636]: osd.4 [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] boot 2024-03-26T17:32:40.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:40 smithi135 ceph-mon[35636]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T17:32:40.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T17:32:41.195 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:41 smithi144 ceph-mon[36095]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T17:32:41.195 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:41 smithi144 ceph-mon[36095]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:32:41.195 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:41 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:32:41.195 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:41 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:32:41.195 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:41 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:41.195 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:41 smithi144 ceph-mon[36095]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:41.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:41 smithi105 ceph-mon[32280]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T17:32:41.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:41 smithi105 ceph-mon[32280]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:32:41.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:41 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:32:41.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:41 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:32:41.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:41 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:41.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:41 smithi105 ceph-mon[32280]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:41.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:41 smithi135 ceph-mon[35636]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T17:32:41.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:41 smithi135 ceph-mon[35636]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:32:41.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:41 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:32:41.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:41 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:32:41.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:41 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:41.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:41 smithi135 ceph-mon[35636]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:43.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:43 smithi144 ceph-mon[36095]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:43.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:43 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/3618107115' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]: dispatch 2024-03-26T17:32:43.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:43 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]: dispatch 2024-03-26T17:32:43.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:43 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]': finished 2024-03-26T17:32:43.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:43 smithi144 ceph-mon[36095]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T17:32:43.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:32:43.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:43 smithi105 ceph-mon[32280]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:43.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:43 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/3618107115' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]: dispatch 2024-03-26T17:32:43.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:43 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]: dispatch 2024-03-26T17:32:43.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:43 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]': finished 2024-03-26T17:32:43.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:43 smithi105 ceph-mon[32280]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T17:32:43.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:32:43.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:43 smithi135 ceph-mon[35636]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:43.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:43 smithi135 ceph-mon[35636]: from='client.? 172.21.15.144:0/3618107115' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]: dispatch 2024-03-26T17:32:43.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:43 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]: dispatch 2024-03-26T17:32:43.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:43 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c13547ad-d6de-4be7-8548-7444afdf81e3"}]': finished 2024-03-26T17:32:43.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:43 smithi135 ceph-mon[35636]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T17:32:43.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:32:44.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:44 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/1023188339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:32:44.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:44 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/1023188339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:32:44.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:44 smithi135 ceph-mon[35636]: from='client.? 172.21.15.144:0/1023188339' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:32:45.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:45 smithi144 ceph-mon[36095]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:45.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:45 smithi105 ceph-mon[32280]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:45.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:45 smithi135 ceph-mon[35636]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:47.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:47 smithi135 ceph-mon[35636]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:47.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:47 smithi144 ceph-mon[36095]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:47 smithi105 ceph-mon[32280]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:49.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:49 smithi105 ceph-mon[32280]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:49.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:49 smithi135 ceph-mon[35636]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:50.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:49 smithi144 ceph-mon[36095]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:50.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T17:32:50.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:50.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T17:32:50.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:51.040 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T17:32:51.040 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:51.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:51 smithi144 ceph-mon[36095]: Deploying daemon osd.5 on smithi144 2024-03-26T17:32:51.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:51 smithi144 ceph-mon[36095]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:51.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:51 smithi105 ceph-mon[32280]: Deploying daemon osd.5 on smithi144 2024-03-26T17:32:51.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:51 smithi105 ceph-mon[32280]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:51.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:51 smithi135 ceph-mon[35636]: Deploying daemon osd.5 on smithi144 2024-03-26T17:32:51.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:51 smithi135 ceph-mon[35636]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:53.808 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:53 smithi144 ceph-mon[36095]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:53.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:53 smithi105 ceph-mon[32280]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:53.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:53 smithi135 ceph-mon[35636]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:55.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:55 smithi105 ceph-mon[32280]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:55.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:55 smithi135 ceph-mon[35636]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:56.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:55 smithi144 ceph-mon[36095]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:57.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:56 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:57.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:56 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:57.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:56 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:57.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:56 smithi144 ceph-mon[36095]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:57.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:56 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:57.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:56 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:57.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:56 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:57.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:56 smithi105 ceph-mon[32280]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:57.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:56 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:57.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:56 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:57.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:56 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:32:57.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:56 smithi135 ceph-mon[35636]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:59.686 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:59 smithi144 ceph-mon[36095]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:59.687 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:59.687 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:59.687 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:59.687 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:59.687 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:32:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:59.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.764+0000 7fbc737fe700 1 -- 172.21.15.144:0/131760174 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbc8406b2d0 con 0x7fbc6c0a2fb0 2024-03-26T17:32:59.759 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 5 on host 'smithi144' 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc6c0a2fb0 msgr2=0x7fbc6c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc6c0a2fb0 0x7fbc6c0a5470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbc84102340 tx=0x7fbc7400a3b0 comp rx=0 tx=0).stop 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84102e80 msgr2=0x7fbc8419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84102e80 0x7fbc8419e900 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c007f40 tx=0x7fbc7c00f5c0 comp rx=0 tx=0).stop 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 shutdown_connections 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc6c0a2fb0 0x7fbc6c0a5470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84102e80 0x7fbc8419e900 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc841012e0 0x7fbc84197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 --2- 172.21.15.144:0/131760174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc841024e0 0x7fbc84197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 >> 172.21.15.144:0/131760174 conn(0x7fbc840fc8a0 msgr2=0x7fbc841057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 shutdown_connections 2024-03-26T17:32:59.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:32:59.767+0000 7fbc88951700 1 -- 172.21.15.144:0/131760174 wait complete. 2024-03-26T17:32:59.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:59 smithi105 ceph-mon[32280]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:59.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:59.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:59.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:59.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:59.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:32:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:59.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:59 smithi135 ceph-mon[35636]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:32:59.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:59.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:32:59.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:32:59.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:32:59.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:32:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:00.318 DEBUG:teuthology.orchestra.run.smithi144:osd.5> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.5.service 2024-03-26T17:33:00.320 INFO:tasks.cephadm:Deploying osd.6 on smithi105 with /dev/vg_nvme/lv_2... 2024-03-26T17:33:00.320 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T17:33:00.570 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:33:00.969 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:00 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:00.969 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:00 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:00.969 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:00 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:00.969 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:00 smithi105 ceph-mon[32280]: from='osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:33:00.969 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:00 smithi105 ceph-mon[32280]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:33:01.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:00 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:01.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:00 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:01.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:00 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:01.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:00 smithi144 ceph-mon[36095]: from='osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:33:01.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:00 smithi144 ceph-mon[36095]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:33:01.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:00 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:01.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:00 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:01.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:00 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:01.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:00 smithi135 ceph-mon[35636]: from='osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:33:01.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:00 smithi135 ceph-mon[35636]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T17:33:01.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:01 smithi144 ceph-mon[36095]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:01.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:01 smithi144 ceph-mon[36095]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T17:33:01.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:01 smithi144 ceph-mon[36095]: from='osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:33:01.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:01 smithi144 ceph-mon[36095]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T17:33:01.875 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:01.876 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:01 smithi144 ceph-mon[36095]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:33:01.933 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:01 smithi105 ceph-mon[32280]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:01.933 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:01 smithi105 ceph-mon[32280]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T17:33:01.933 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:01 smithi105 ceph-mon[32280]: from='osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:33:01.933 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:01 smithi105 ceph-mon[32280]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T17:33:01.933 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:01.933 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:01 smithi105 ceph-mon[32280]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:33:02.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:01 smithi135 ceph-mon[35636]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:02.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:01 smithi135 ceph-mon[35636]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T17:33:02.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:01 smithi135 ceph-mon[35636]: from='osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:33:02.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:01 smithi135 ceph-mon[35636]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T17:33:02.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:02.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:01 smithi135 ceph-mon[35636]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:33:02.516 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T17:33:02.516 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T17:33:02.516 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-26T17:33:02.516 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-26T17:33:02.516 INFO:teuthology.orchestra.run.smithi105.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135062 s, 776 MB/s 2024-03-26T17:33:02.517 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-26T17:33:02.979 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:02 smithi144 ceph-mon[36095]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:33:02.979 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:02 smithi144 ceph-mon[36095]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T17:33:02.979 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:02.979 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:02.979 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:02 smithi144 ceph-mon[36095]: from='osd.5 ' entity='osd.5' 2024-03-26T17:33:02.980 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 17:33:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T17:33:02.641+0000 7fd226ff7700 -1 osd.5 0 waiting for initial osdmap 2024-03-26T17:33:02.980 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 17:33:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T17:33:02.888+0000 7fd21ede2700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:33:03.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:02 smithi105 ceph-mon[32280]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:33:03.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:02 smithi105 ceph-mon[32280]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T17:33:03.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:03.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:03.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:02 smithi105 ceph-mon[32280]: from='osd.5 ' entity='osd.5' 2024-03-26T17:33:03.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_2 2024-03-26T17:33:03.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:02 smithi135 ceph-mon[35636]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:33:03.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:02 smithi135 ceph-mon[35636]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T17:33:03.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:03.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:03.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:02 smithi135 ceph-mon[35636]: from='osd.5 ' entity='osd.5' 2024-03-26T17:33:03.384 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:33:03.816 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:33:03.816 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:33:03.816 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:03.816 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T17:33:03.816 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:03.816 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:03.816 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:03.817 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:03.817 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:03.817 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:03.817 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:03.817 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:03.817 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:03.817 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:04.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:33:04.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:33:04.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:04.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T17:33:04.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:04.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:04.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:04.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:04.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:04.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:04.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:04.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:04.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:04.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:04.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:04.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:04.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:04.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:04.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:04.830 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:04 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi144 2024-03-26T17:33:04.831 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:04 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi144 to 8544M 2024-03-26T17:33:04.831 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:04 smithi105 ceph-mon[32280]: osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] boot 2024-03-26T17:33:04.831 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:04 smithi105 ceph-mon[32280]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T17:33:04.831 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 -- 172.21.15.105:0/2919011354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841024e0 msgr2=0x7fc784102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/2919011354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841024e0 0x7fc784102940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7700099c0 tx=0x7fc770009cd0 comp rx=0 tx=0).stop 2024-03-26T17:33:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 -- 172.21.15.105:0/2919011354 shutdown_connections 2024-03-26T17:33:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/2919011354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc784102e80 0x7fc784109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/2919011354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7841012e0 0x7fc784101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/2919011354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841024e0 0x7fc784102940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 -- 172.21.15.105:0/2919011354 >> 172.21.15.105:0/2919011354 conn(0x7fc7840fc880 msgr2=0x7fc7840fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 -- 172.21.15.105:0/2919011354 shutdown_connections 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.941+0000 7fc78b1fa700 1 -- 172.21.15.105:0/2919011354 wait complete. 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc78b1fa700 1 Processor -- start 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc78b1fa700 1 -- start start 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc78b1fa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841012e0 0x7fc78406abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc78b1fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7841024e0 0x7fc78419d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc78b1fa700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc784102e80 0x7fc78419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc78b1fa700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc78410cd60 con 0x7fc7841012e0 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc78b1fa700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc78410cbe0 con 0x7fc784102e80 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc78b1fa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc78410cee0 con 0x7fc7841024e0 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841012e0 0x7fc78406abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841012e0 0x7fc78406abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52706/0 (socket says 172.21.15.105:52706) 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 -- 172.21.15.105:0/159079089 learned_addr learned my addr 172.21.15.105:0/159079089 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc789797700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc784102e80 0x7fc78419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc783fff700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7841024e0 0x7fc78419d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 -- 172.21.15.105:0/159079089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7841024e0 msgr2=0x7fc78419d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7841024e0 0x7fc78419d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 -- 172.21.15.105:0/159079089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc784102e80 msgr2=0x7fc78419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc784102e80 0x7fc78419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:04.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 -- 172.21.15.105:0/159079089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7841a0150 con 0x7fc7841012e0 2024-03-26T17:33:04.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc788f96700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841012e0 0x7fc78406abc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc77800c970 tx=0x7fc77800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:33:04.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.942+0000 7fc781ffb700 1 -- 172.21.15.105:0/159079089 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778004220 con 0x7fc7841012e0 2024-03-26T17:33:04.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.943+0000 7fc781ffb700 1 -- 172.21.15.105:0/159079089 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc778007d50 con 0x7fc7841012e0 2024-03-26T17:33:04.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.943+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc784197070 con 0x7fc7841012e0 2024-03-26T17:33:04.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.943+0000 7fc781ffb700 1 -- 172.21.15.105:0/159079089 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778003d00 con 0x7fc7841012e0 2024-03-26T17:33:04.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.943+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc784197600 con 0x7fc7841012e0 2024-03-26T17:33:04.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.944+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc784069070 con 0x7fc7841012e0 2024-03-26T17:33:04.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.945+0000 7fc781ffb700 1 -- 172.21.15.105:0/159079089 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fc7780078a0 con 0x7fc7841012e0 2024-03-26T17:33:04.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.945+0000 7fc781ffb700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7740a3030 0x7fc7740a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:04.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.945+0000 7fc783fff700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7740a3030 0x7fc7740a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:04.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.945+0000 7fc781ffb700 1 -- 172.21.15.105:0/159079089 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fc7780ce080 con 0x7fc7841012e0 2024-03-26T17:33:04.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.946+0000 7fc783fff700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7740a3030 0x7fc7740a54f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc78406b140 tx=0x7fc770005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:33:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:04.950+0000 7fc781ffb700 1 -- 172.21.15.105:0/159079089 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc778014030 con 0x7fc7841012e0 2024-03-26T17:33:05.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:04 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi144 2024-03-26T17:33:05.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:04 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi144 to 8544M 2024-03-26T17:33:05.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:04 smithi144 ceph-mon[36095]: osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] boot 2024-03-26T17:33:05.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:04 smithi144 ceph-mon[36095]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T17:33:05.016 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:04 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:05.067 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:04 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi144 2024-03-26T17:33:05.068 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:04 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi144 to 8544M 2024-03-26T17:33:05.068 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:04 smithi135 ceph-mon[35636]: osd.5 [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] boot 2024-03-26T17:33:05.068 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:04 smithi135 ceph-mon[35636]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T17:33:05.068 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:04 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T17:33:05.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:05.154+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc7840627f0 con 0x7fc7740a3030 2024-03-26T17:33:06.076 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:05 smithi105 ceph-mon[32280]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:06.076 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:05 smithi105 ceph-mon[32280]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T17:33:06.076 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:06.076 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:06.076 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:06.084 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:05 smithi135 ceph-mon[35636]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:06.084 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:05 smithi135 ceph-mon[35636]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T17:33:06.084 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:05 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:06.085 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:05 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:06.085 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:05 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:06.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:05 smithi144 ceph-mon[36095]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-26T17:33:06.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:05 smithi144 ceph-mon[36095]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T17:33:06.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:05 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:06.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:05 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:06.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:05 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:06.820 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:06 smithi105 ceph-mon[32280]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:06.821 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:06 smithi105 ceph-mon[32280]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T17:33:06.821 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:06 smithi105 ceph-mon[32280]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:33:07.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:06 smithi144 ceph-mon[36095]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:07.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:06 smithi144 ceph-mon[36095]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T17:33:07.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:06 smithi144 ceph-mon[36095]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:33:07.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:06 smithi135 ceph-mon[35636]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:07.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:06 smithi135 ceph-mon[35636]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T17:33:07.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:06 smithi135 ceph-mon[35636]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:33:08.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:07 smithi144 ceph-mon[36095]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T17:33:08.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:07 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2590612237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342c60b1-acab-414f-b483-bc6ab61d927e"}]: dispatch 2024-03-26T17:33:08.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:07 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2590612237' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "342c60b1-acab-414f-b483-bc6ab61d927e"}]': finished 2024-03-26T17:33:08.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:07 smithi144 ceph-mon[36095]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T17:33:08.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:07 smithi105 ceph-mon[32280]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T17:33:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:07 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2590612237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342c60b1-acab-414f-b483-bc6ab61d927e"}]: dispatch 2024-03-26T17:33:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:07 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2590612237' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "342c60b1-acab-414f-b483-bc6ab61d927e"}]': finished 2024-03-26T17:33:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:07 smithi105 ceph-mon[32280]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T17:33:08.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:08.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:07 smithi135 ceph-mon[35636]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T17:33:08.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:07 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2590612237' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342c60b1-acab-414f-b483-bc6ab61d927e"}]: dispatch 2024-03-26T17:33:08.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:07 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2590612237' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "342c60b1-acab-414f-b483-bc6ab61d927e"}]': finished 2024-03-26T17:33:08.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:07 smithi135 ceph-mon[35636]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T17:33:08.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:09.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:08 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4139368396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:09.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:08 smithi144 ceph-mon[36095]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:33:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:08 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4139368396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:08 smithi105 ceph-mon[32280]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:33:09.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:08 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4139368396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:09.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:08 smithi135 ceph-mon[35636]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-26T17:33:11.762 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:11 smithi105 ceph-mon[32280]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-26T17:33:11.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:11 smithi144 ceph-mon[36095]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-26T17:33:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:11 smithi135 ceph-mon[35636]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-26T17:33:13.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:13 smithi144 ceph-mon[36095]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T17:33:13.889 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:13 smithi105 ceph-mon[32280]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T17:33:13.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:13 smithi135 ceph-mon[35636]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T17:33:15.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:15 smithi144 ceph-mon[36095]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:15.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T17:33:15.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:15.879 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:15 smithi105 ceph-mon[32280]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:15.879 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T17:33:15.879 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:15.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:15 smithi135 ceph-mon[35636]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:15.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T17:33:15.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:16.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:16 smithi105 ceph-mon[32280]: Deploying daemon osd.6 on smithi105 2024-03-26T17:33:16.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:16 smithi135 ceph-mon[35636]: Deploying daemon osd.6 on smithi105 2024-03-26T17:33:17.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:16 smithi144 ceph-mon[36095]: Deploying daemon osd.6 on smithi105 2024-03-26T17:33:17.864 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:17 smithi105 ceph-mon[32280]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:17.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:17 smithi135 ceph-mon[35636]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:18.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:17 smithi144 ceph-mon[36095]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:19.805 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:19 smithi105 ceph-mon[32280]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:19.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:19 smithi135 ceph-mon[35636]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:20.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:19 smithi144 ceph-mon[36095]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:21.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:21.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:21.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:21.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:21 smithi144 ceph-mon[36095]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:21.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:21.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:21.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:21.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:21 smithi105 ceph-mon[32280]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:21.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:21.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:21.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:21.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:21 smithi135 ceph-mon[35636]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:23.827 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 6 on host 'smithi105' 2024-03-26T17:33:23.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.832+0000 7fc781ffb700 1 -- 172.21.15.105:0/159079089 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc7840627f0 con 0x7fc7740a3030 2024-03-26T17:33:23.828 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:23 smithi105 ceph-mon[32280]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:23.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7740a3030 msgr2=0x7fc7740a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:23.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7740a3030 0x7fc7740a54f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc78406b140 tx=0x7fc770005fd0 comp rx=0 tx=0).stop 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841012e0 msgr2=0x7fc78406abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841012e0 0x7fc78406abc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc77800c970 tx=0x7fc77800cc80 comp rx=0 tx=0).stop 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 shutdown_connections 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7740a3030 0x7fc7740a54f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc784102e80 0x7fc78419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7841012e0 0x7fc78406abc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 --2- 172.21.15.105:0/159079089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7841024e0 0x7fc78419d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 >> 172.21.15.105:0/159079089 conn(0x7fc7840fc880 msgr2=0x7fc784105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 shutdown_connections 2024-03-26T17:33:23.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:33:23.835+0000 7fc78b1fa700 1 -- 172.21.15.105:0/159079089 wait complete. 2024-03-26T17:33:23.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:23 smithi144 ceph-mon[36095]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:23.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:23 smithi135 ceph-mon[35636]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:24.421 DEBUG:teuthology.orchestra.run.smithi105:osd.6> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.6.service 2024-03-26T17:33:24.423 INFO:tasks.cephadm:Deploying osd.7 on smithi135 with /dev/vg_nvme/lv_2... 2024-03-26T17:33:24.424 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T17:33:24.670 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:33:25.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.036 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:24 smithi135 ceph-mon[35636]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T17:33:25.037 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:24 smithi135 ceph-mon[35636]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:25.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:24 smithi144 ceph-mon[36095]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T17:33:25.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:24 smithi144 ceph-mon[36095]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:25.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:25.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:24 smithi105 ceph-mon[32280]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T17:33:25.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:24 smithi105 ceph-mon[32280]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:26.002 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:25 smithi135 ceph-mon[35636]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T17:33:26.002 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:25 smithi135 ceph-mon[35636]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T17:33:26.002 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:25 smithi135 ceph-mon[35636]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:33:26.002 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:25 smithi105 ceph-mon[32280]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T17:33:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:25 smithi105 ceph-mon[32280]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T17:33:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:25 smithi105 ceph-mon[32280]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:33:26.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:26.144 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 17:33:25 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T17:33:25.933+0000 7fe32fddb700 -1 osd.6 0 waiting for initial osdmap 2024-03-26T17:33:26.144 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 17:33:25 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T17:33:25.943+0000 7fe32b3cd700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:33:26.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:25 smithi144 ceph-mon[36095]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T17:33:26.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:25 smithi144 ceph-mon[36095]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T17:33:26.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:25 smithi144 ceph-mon[36095]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:33:26.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:26.671 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T17:33:26.671 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T17:33:26.671 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-03-26T17:33:26.671 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-03-26T17:33:26.671 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113868 s, 921 MB/s 2024-03-26T17:33:26.671 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-03-26T17:33:27.262 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:26 smithi135 ceph-mon[35636]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:33:27.262 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:26 smithi135 ceph-mon[35636]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T17:33:27.262 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:27.262 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:26 smithi135 ceph-mon[35636]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:27.285 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi135:vg_nvme/lv_2 2024-03-26T17:33:27.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:26 smithi144 ceph-mon[36095]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:33:27.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:26 smithi144 ceph-mon[36095]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T17:33:27.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:27.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:26 smithi144 ceph-mon[36095]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:27.354 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:26 smithi105 ceph-mon[32280]: from='osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:33:27.354 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:26 smithi105 ceph-mon[32280]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T17:33:27.354 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:27.354 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:26 smithi105 ceph-mon[32280]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-03-26T17:33:27.536 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:33:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] boot 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi105 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi105 to 5696M 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:28.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:33:28.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:33:28.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:28.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] boot 2024-03-26T17:33:28.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T17:33:28.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi105 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi105 to 5696M 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:28.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: osd.6 [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] boot 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi105 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi105 to 5696M 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:28.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:28.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:28.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:28.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:28.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:28.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:29.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 -- 172.21.15.135:0/2339533307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc1023e0 msgr2=0x7f57dc102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:29.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 --2- 172.21.15.135:0/2339533307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc1023e0 0x7f57dc102800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0099c0 tx=0x7f57cc009cd0 comp rx=0 tx=0).stop 2024-03-26T17:33:29.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 -- 172.21.15.135:0/2339533307 shutdown_connections 2024-03-26T17:33:29.135 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 --2- 172.21.15.135:0/2339533307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57dc103f80 0x7f57dc10a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 --2- 172.21.15.135:0/2339533307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc1023e0 0x7f57dc102800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 --2- 172.21.15.135:0/2339533307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc1035e0 0x7f57dc103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 -- 172.21.15.135:0/2339533307 >> 172.21.15.135:0/2339533307 conn(0x7f57dc0fd9c0 msgr2=0x7f57dc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 -- 172.21.15.135:0/2339533307 shutdown_connections 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.138+0000 7f57e272d700 1 -- 172.21.15.135:0/2339533307 wait complete. 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e272d700 1 Processor -- start 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e272d700 1 -- start start 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e272d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57dc1023e0 0x7f57dc076150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e272d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc1035e0 0x7f57dc0747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e272d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc103f80 0x7f57dc074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e272d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10de70 con 0x7f57dc1035e0 2024-03-26T17:33:29.136 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e272d700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10dcf0 con 0x7f57dc103f80 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e272d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10dff0 con 0x7f57dc1023e0 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.139+0000 7f57e0cca700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc103f80 0x7f57dc074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e0cca700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc103f80 0x7f57dc074ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:50872/0 (socket says 172.21.15.135:50872) 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e0cca700 1 -- 172.21.15.135:0/2575531468 learned_addr learned my addr 172.21.15.135:0/2575531468 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57dbfff700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57dc1023e0 0x7f57dc076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57db7fe700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc1035e0 0x7f57dc0747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e0cca700 1 -- 172.21.15.135:0/2575531468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57dc1023e0 msgr2=0x7f57dc076150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e0cca700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57dc1023e0 0x7f57dc076150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e0cca700 1 -- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc1035e0 msgr2=0x7f57dc0747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e0cca700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc1035e0 0x7f57dc0747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e0cca700 1 -- 172.21.15.135:0/2575531468 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc075590 con 0x7f57dc103f80 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e0cca700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc103f80 0x7f57dc074ce0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57d400ada0 tx=0x7f57d400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57d97fa700 1 -- 172.21.15.135:0/2575531468 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4017020 con 0x7f57dc103f80 2024-03-26T17:33:29.137 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc1a26f0 con 0x7f57dc103f80 2024-03-26T17:33:29.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57d97fa700 1 -- 172.21.15.135:0/2575531468 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57d400cd80 con 0x7f57dc103f80 2024-03-26T17:33:29.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57d97fa700 1 -- 172.21.15.135:0/2575531468 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4005260 con 0x7f57dc103f80 2024-03-26T17:33:29.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.140+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc1a2c00 con 0x7f57dc103f80 2024-03-26T17:33:29.138 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.141+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc069070 con 0x7f57dc103f80 2024-03-26T17:33:29.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.142+0000 7f57d97fa700 1 -- 172.21.15.135:0/2575531468 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f57d400c8d0 con 0x7f57dc103f80 2024-03-26T17:33:29.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.143+0000 7f57d97fa700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57c40a2fb0 0x7f57c40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:29.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.143+0000 7f57d97fa700 1 -- 172.21.15.135:0/2575531468 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f57d40ce370 con 0x7f57dc103f80 2024-03-26T17:33:29.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.143+0000 7f57dbfff700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57c40a2fb0 0x7f57c40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:29.139 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.144+0000 7f57dbfff700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57c40a2fb0 0x7f57c40a5470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:33:29.140 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.147+0000 7f57d97fa700 1 -- 172.21.15.135:0/2575531468 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f57d4021020 con 0x7f57dc103f80 2024-03-26T17:33:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:28 smithi135 ceph-mon[35636]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T17:33:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:28 smithi135 ceph-mon[35636]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:29.343 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:29.349+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f57dc0627f0 con 0x7f57c40a2fb0 2024-03-26T17:33:29.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:28 smithi144 ceph-mon[36095]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T17:33:29.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:28 smithi144 ceph-mon[36095]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:29.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:28 smithi105 ceph-mon[32280]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T17:33:29.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:28 smithi105 ceph-mon[32280]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:30.276 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:29 smithi135 ceph-mon[35636]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:30.276 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:30.276 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:30.276 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:30.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:29 smithi144 ceph-mon[36095]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:30.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:30.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:30.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:30.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:29 smithi105 ceph-mon[32280]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:30.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:30.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:30.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:31.292 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:30 smithi135 ceph-mon[35636]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:31.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:30 smithi144 ceph-mon[36095]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:31.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:30 smithi105 ceph-mon[32280]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:32.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:32 smithi144 ceph-mon[36095]: from='client.? 172.21.15.135:0/4291939855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]: dispatch 2024-03-26T17:33:32.353 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:32 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]: dispatch 2024-03-26T17:33:32.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:32 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]': finished 2024-03-26T17:33:32.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:32 smithi144 ceph-mon[36095]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T17:33:32.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:32.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:32 smithi105 ceph-mon[32280]: from='client.? 172.21.15.135:0/4291939855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]: dispatch 2024-03-26T17:33:32.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:32 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]: dispatch 2024-03-26T17:33:32.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:32 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]': finished 2024-03-26T17:33:32.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:32 smithi105 ceph-mon[32280]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T17:33:32.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:32.396 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:32 smithi135 ceph-mon[35636]: from='client.? 172.21.15.135:0/4291939855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]: dispatch 2024-03-26T17:33:32.396 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:32 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]: dispatch 2024-03-26T17:33:32.397 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:32 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc"}]': finished 2024-03-26T17:33:32.397 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:32 smithi135 ceph-mon[35636]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T17:33:32.397 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:33.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:33 smithi144 ceph-mon[36095]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:33.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:33 smithi144 ceph-mon[36095]: from='client.? 172.21.15.135:0/2490788912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:33.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:33 smithi105 ceph-mon[32280]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:33.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:33 smithi105 ceph-mon[32280]: from='client.? 172.21.15.135:0/2490788912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:33.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:33 smithi135 ceph-mon[35636]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:33.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:33 smithi135 ceph-mon[35636]: from='client.? 172.21.15.135:0/2490788912' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:35.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:35 smithi144 ceph-mon[36095]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:35.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:35 smithi105 ceph-mon[32280]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:35.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:35 smithi135 ceph-mon[35636]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:37.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:37 smithi144 ceph-mon[36095]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:37.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:37 smithi105 ceph-mon[32280]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:37.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:37 smithi135 ceph-mon[35636]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:39.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:39 smithi105 ceph-mon[32280]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:39.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T17:33:39.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:39.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:39 smithi135 ceph-mon[35636]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:39.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T17:33:39.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:40.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:39 smithi144 ceph-mon[36095]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:40.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T17:33:40.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:40.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:40 smithi135 ceph-mon[35636]: Deploying daemon osd.7 on smithi135 2024-03-26T17:33:40.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:40 smithi105 ceph-mon[32280]: Deploying daemon osd.7 on smithi135 2024-03-26T17:33:41.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:40 smithi144 ceph-mon[36095]: Deploying daemon osd.7 on smithi135 2024-03-26T17:33:41.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:41 smithi105 ceph-mon[32280]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:41.906 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:41 smithi135 ceph-mon[35636]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:41 smithi144 ceph-mon[36095]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:43.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:43 smithi105 ceph-mon[32280]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:43.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:43 smithi135 ceph-mon[35636]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:44.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:43 smithi144 ceph-mon[36095]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:45.885 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:45 smithi135 ceph-mon[35636]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:45.885 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:45.885 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:45.885 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:45 smithi105 ceph-mon[32280]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:46.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:45 smithi144 ceph-mon[36095]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:46.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:46.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:46.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:47 smithi105 ceph-mon[32280]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:47.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:47 smithi135 ceph-mon[35636]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:47 smithi144 ceph-mon[36095]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:48.162 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.168+0000 7f57d97fa700 1 -- 172.21.15.135:0/2575531468 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f57dc0627f0 con 0x7f57c40a2fb0 2024-03-26T17:33:48.163 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 7 on host 'smithi135' 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57c40a2fb0 msgr2=0x7f57c40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57c40a2fb0 0x7f57c40a5470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc005fb0 comp rx=0 tx=0).stop 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc103f80 msgr2=0x7f57dc074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc103f80 0x7f57dc074ce0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57d400ada0 tx=0x7f57d400c330 comp rx=0 tx=0).stop 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 shutdown_connections 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57c40a2fb0 0x7f57c40a5470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57dc103f80 0x7f57dc074ce0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57dc1023e0 0x7f57dc076150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:48.165 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 --2- 172.21.15.135:0/2575531468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc1035e0 0x7f57dc0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:48.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 >> 172.21.15.135:0/2575531468 conn(0x7f57dc0fd9c0 msgr2=0x7f57dc106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:33:48.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 shutdown_connections 2024-03-26T17:33:48.166 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:33:48.171+0000 7f57e272d700 1 -- 172.21.15.135:0/2575531468 wait complete. 2024-03-26T17:33:48.807 DEBUG:teuthology.orchestra.run.smithi135:osd.7> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.7.service 2024-03-26T17:33:48.809 INFO:tasks.cephadm:Deploying osd.8 on smithi144 with /dev/vg_nvme/lv_2... 2024-03-26T17:33:48.809 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T17:33:49.058 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:33:49.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:49 smithi135 ceph-mon[35636]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:49.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:49 smithi135 ceph-mon[35636]: from='osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:33:49.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:49 smithi135 ceph-mon[35636]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:33:49.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.420 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.420 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.420 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:49 smithi144 ceph-mon[36095]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:49.420 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:49 smithi144 ceph-mon[36095]: from='osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:33:49.420 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:49 smithi144 ceph-mon[36095]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:33:49.420 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.420 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:49 smithi105 ceph-mon[32280]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:49.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:49 smithi105 ceph-mon[32280]: from='osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:33:49.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:49 smithi105 ceph-mon[32280]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T17:33:49.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:49.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:50.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:50 smithi144 ceph-mon[36095]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T17:33:50.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:50 smithi144 ceph-mon[36095]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T17:33:50.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:50.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:50 smithi144 ceph-mon[36095]: from='osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:33:50.377 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:50 smithi144 ceph-mon[36095]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:33:50.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:50 smithi135 ceph-mon[35636]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T17:33:50.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:50 smithi135 ceph-mon[35636]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T17:33:50.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:50.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:50 smithi135 ceph-mon[35636]: from='osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:33:50.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:50 smithi135 ceph-mon[35636]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:33:50.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:50 smithi105 ceph-mon[32280]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T17:33:50.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:50 smithi105 ceph-mon[32280]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T17:33:50.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:50.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:50 smithi105 ceph-mon[32280]: from='osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:33:50.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:50 smithi105 ceph-mon[32280]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:33:50.937 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T17:33:50.937 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T17:33:50.938 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-26T17:33:50.938 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-26T17:33:50.938 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113759 s, 922 MB/s 2024-03-26T17:33:50.938 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-26T17:33:51.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-mon[35636]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:33:51.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-mon[35636]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T17:33:51.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-mon[35636]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:51.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-mon[35636]: from='osd.7 ' entity='osd.7' 2024-03-26T17:33:51.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-mon[35636]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T17:33:51.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.418 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T17:33:51.061+0000 7f4f8f84f700 -1 osd.7 0 waiting for initial osdmap 2024-03-26T17:33:51.418 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 17:33:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T17:33:51.278+0000 7f4f8763a700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:33:51.580 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:51 smithi144 ceph-mon[36095]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:33:51.581 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:51 smithi144 ceph-mon[36095]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T17:33:51.581 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.581 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.581 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:51 smithi144 ceph-mon[36095]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:51.581 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:51 smithi144 ceph-mon[36095]: from='osd.7 ' entity='osd.7' 2024-03-26T17:33:51.581 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:51 smithi144 ceph-mon[36095]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T17:33:51.581 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.608 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-03-26T17:33:51.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:51 smithi105 ceph-mon[32280]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:33:51.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:51 smithi105 ceph-mon[32280]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T17:33:51.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:51 smithi105 ceph-mon[32280]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:51.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:51 smithi105 ceph-mon[32280]: from='osd.7 ' entity='osd.7' 2024-03-26T17:33:51.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:51 smithi105 ceph-mon[32280]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T17:33:51.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:51.848 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:33:52.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:52 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:33:52.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:52 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:33:52.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:52.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:52.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:52 smithi135 ceph-mon[35636]: osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] boot 2024-03-26T17:33:52.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:52 smithi135 ceph-mon[35636]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T17:33:52.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:52.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:52 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:33:52.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:52 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:33:52.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:52.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:52.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:52 smithi144 ceph-mon[36095]: osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] boot 2024-03-26T17:33:52.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:52 smithi144 ceph-mon[36095]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T17:33:52.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:52.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:52 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:33:52.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:52 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:33:52.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:52.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:52.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:52 smithi105 ceph-mon[32280]: osd.7 [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] boot 2024-03-26T17:33:52.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:52 smithi105 ceph-mon[32280]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T17:33:52.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T17:33:53.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 -- 172.21.15.144:0/3021214152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f01024e0 msgr2=0x7f95f0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:53.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 --2- 172.21.15.144:0/3021214152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f01024e0 0x7f95f0102940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009990 tx=0x7f95ec009ca0 comp rx=0 tx=0).stop 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 -- 172.21.15.144:0/3021214152 shutdown_connections 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 --2- 172.21.15.144:0/3021214152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f0102e80 0x7f95f0109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 --2- 172.21.15.144:0/3021214152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95f01012e0 0x7f95f0101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 --2- 172.21.15.144:0/3021214152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f01024e0 0x7f95f0102940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 -- 172.21.15.144:0/3021214152 >> 172.21.15.144:0/3021214152 conn(0x7f95f00fc8a0 msgr2=0x7f95f00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 -- 172.21.15.144:0/3021214152 shutdown_connections 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.418+0000 7f95f75c0700 1 -- 172.21.15.144:0/3021214152 wait complete. 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f75c0700 1 Processor -- start 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f75c0700 1 -- start start 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f75c0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f01012e0 0x7f95f0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f75c0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95f01024e0 0x7f95f0197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f75c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f0102e80 0x7f95f019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f75c0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010cdb0 con 0x7f95f01024e0 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f75c0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010cc30 con 0x7f95f01012e0 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f75c0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010cf30 con 0x7f95f0102e80 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f5b5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f0102e80 0x7f95f019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f5b5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f0102e80 0x7f95f019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60664/0 (socket says 172.21.15.144:60664) 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.419+0000 7f95f5b5d700 1 -- 172.21.15.144:0/2498982877 learned_addr learned my addr 172.21.15.144:0/2498982877 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f535c700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f01012e0 0x7f95f0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f4b5b700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95f01024e0 0x7f95f0197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f5b5d700 1 -- 172.21.15.144:0/2498982877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f01012e0 msgr2=0x7f95f0197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:53.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f5b5d700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f01012e0 0x7f95f0197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f5b5d700 1 -- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95f01024e0 msgr2=0x7f95f0197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f5b5d700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95f01024e0 0x7f95f0197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f5b5d700 1 -- 172.21.15.144:0/2498982877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f019f010 con 0x7f95f0102e80 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f5b5d700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f0102e80 0x7f95f019e970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007d40 tx=0x7f95e800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95e67fc700 1 -- 172.21.15.144:0/2498982877 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8016020 con 0x7f95f0102e80 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95e67fc700 1 -- 172.21.15.144:0/2498982877 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95e800acb0 con 0x7f95f0102e80 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f75c0700 1 -- 172.21.15.144:0/2498982877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f019f300 con 0x7f95f0102e80 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95e67fc700 1 -- 172.21.15.144:0/2498982877 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80115e0 con 0x7f95f0102e80 2024-03-26T17:33:53.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.420+0000 7f95f75c0700 1 -- 172.21.15.144:0/2498982877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f019f840 con 0x7f95f0102e80 2024-03-26T17:33:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.421+0000 7f95dbfff700 1 -- 172.21.15.144:0/2498982877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d0005330 con 0x7f95f0102e80 2024-03-26T17:33:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.422+0000 7f95e67fc700 1 -- 172.21.15.144:0/2498982877 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f95e8019020 con 0x7f95f0102e80 2024-03-26T17:33:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.423+0000 7f95e67fc700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95dc0a3030 0x7f95dc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:33:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.423+0000 7f95e67fc700 1 -- 172.21.15.144:0/2498982877 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f95e80ce750 con 0x7f95f0102e80 2024-03-26T17:33:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.423+0000 7f95f535c700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95dc0a3030 0x7f95dc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:33:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.423+0000 7f95f535c700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95dc0a3030 0x7f95dc0a54f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95f0102340 tx=0x7f95e000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:33:53.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.427+0000 7f95e67fc700 1 -- 172.21.15.144:0/2498982877 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f95e8020020 con 0x7f95f0102e80 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi135 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi135 to 5696M 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:53.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:53.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:53.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:53.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:53.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:53.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:53 smithi144 ceph-mon[36095]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T17:33:53.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:33:53.632+0000 7f95dbfff700 1 -- 172.21.15.144:0/2498982877 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f95d0000bc0 con 0x7f95dc0a3030 2024-03-26T17:33:53.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi135 2024-03-26T17:33:53.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi135 to 5696M 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:53.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:53 smithi105 ceph-mon[32280]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi135 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi135 to 5696M 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:53.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:53.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:33:53.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:53.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:33:53.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:33:53.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-26T17:33:53.668 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:53 smithi135 ceph-mon[35636]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T17:33:54.545 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:54 smithi144 ceph-mon[36095]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:54.545 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:54.545 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:54.545 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:54.545 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:54 smithi144 ceph-mon[36095]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T17:33:54.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:54 smithi105 ceph-mon[32280]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:54.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:54.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:54.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:54.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:54 smithi105 ceph-mon[32280]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T17:33:54.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:54 smithi135 ceph-mon[35636]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:33:54.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:33:54.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:33:54.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:33:54.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:54 smithi135 ceph-mon[35636]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T17:33:55.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:55 smithi135 ceph-mon[35636]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:33:55.820 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:55 smithi105 ceph-mon[32280]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:33:55.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:55 smithi144 ceph-mon[36095]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:33:56.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:56 smithi135 ceph-mon[35636]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T17:33:56.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:56 smithi135 ceph-mon[35636]: from='client.? 172.21.15.144:0/2879535374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]: dispatch 2024-03-26T17:33:56.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:56 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]: dispatch 2024-03-26T17:33:56.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:56 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]': finished 2024-03-26T17:33:56.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:56 smithi135 ceph-mon[35636]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T17:33:56.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:56 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:33:56.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:56 smithi105 ceph-mon[32280]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T17:33:56.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:56 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/2879535374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]: dispatch 2024-03-26T17:33:56.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:56 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]: dispatch 2024-03-26T17:33:56.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:56 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]': finished 2024-03-26T17:33:56.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:56 smithi105 ceph-mon[32280]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T17:33:56.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:56 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:33:56.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:56 smithi144 ceph-mon[36095]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T17:33:56.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:56 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/2879535374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]: dispatch 2024-03-26T17:33:56.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:56 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]: dispatch 2024-03-26T17:33:56.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:56 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "779c4b05-662f-4332-994a-5cb93ba4aabf"}]': finished 2024-03-26T17:33:56.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:56 smithi144 ceph-mon[36095]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T17:33:56.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:56 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:33:57.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:57 smithi135 ceph-mon[35636]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:33:57.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:57 smithi135 ceph-mon[35636]: from='client.? 172.21.15.144:0/558467054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:57.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:57 smithi144 ceph-mon[36095]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:33:57.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:57 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/558467054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:57.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:57 smithi105 ceph-mon[32280]: pgmap v181: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:33:57.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:57 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/558467054' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:33:59.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:33:59 smithi144 ceph-mon[36095]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:33:59.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:33:59 smithi105 ceph-mon[32280]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:33:59.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:33:59 smithi135 ceph-mon[35636]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:01 smithi144 ceph-mon[36095]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:01.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:01 smithi105 ceph-mon[32280]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:01.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:01 smithi135 ceph-mon[35636]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:03.821 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:03 smithi144 ceph-mon[36095]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:03 smithi105 ceph-mon[32280]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:03 smithi135 ceph-mon[35636]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:04.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:04 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T17:34:04.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:04 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T17:34:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:04.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:04 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T17:34:04.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:04 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:05 smithi105 ceph-mon[32280]: Deploying daemon osd.8 on smithi144 2024-03-26T17:34:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:05 smithi105 ceph-mon[32280]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:05.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:05 smithi135 ceph-mon[35636]: Deploying daemon osd.8 on smithi144 2024-03-26T17:34:05.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:05 smithi135 ceph-mon[35636]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:05.949 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:05 smithi144 ceph-mon[36095]: Deploying daemon osd.8 on smithi144 2024-03-26T17:34:05.949 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:05 smithi144 ceph-mon[36095]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:07.748 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:07 smithi144 ceph-mon[36095]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:07.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:07 smithi105 ceph-mon[32280]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:07.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:07 smithi135 ceph-mon[35636]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:09 smithi105 ceph-mon[32280]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:09.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:09 smithi135 ceph-mon[35636]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:09.963 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:09 smithi144 ceph-mon[36095]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:09.963 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:09.963 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:09.963 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:11.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:11 smithi144 ceph-mon[36095]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:11.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:11 smithi105 ceph-mon[32280]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:11 smithi135 ceph-mon[35636]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:12.845 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 8 on host 'smithi144' 2024-03-26T17:34:12.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.850+0000 7f95e67fc700 1 -- 172.21.15.144:0/2498982877 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f95d0000bc0 con 0x7f95dc0a3030 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 -- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95dc0a3030 msgr2=0x7f95dc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95dc0a3030 0x7f95dc0a54f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95f0102340 tx=0x7f95e000a380 comp rx=0 tx=0).stop 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 -- 172.21.15.144:0/2498982877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f0102e80 msgr2=0x7f95f019e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f0102e80 0x7f95f019e970 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007d40 tx=0x7f95e800a340 comp rx=0 tx=0).stop 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 -- 172.21.15.144:0/2498982877 shutdown_connections 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95dc0a3030 0x7f95dc0a54f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95f0102e80 0x7f95f019e970 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95f01012e0 0x7f95f0197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:12.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 --2- 172.21.15.144:0/2498982877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95f01024e0 0x7f95f0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 -- 172.21.15.144:0/2498982877 >> 172.21.15.144:0/2498982877 conn(0x7f95f00fc8a0 msgr2=0x7f95f01057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:34:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 -- 172.21.15.144:0/2498982877 shutdown_connections 2024-03-26T17:34:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:34:12.853+0000 7f95dbfff700 1 -- 172.21.15.144:0/2498982877 wait complete. 2024-03-26T17:34:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:13.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:13 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:13.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:13.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:13.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:13.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:13 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:13.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:13 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:13.534 DEBUG:teuthology.orchestra.run.smithi144:osd.8> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.8.service 2024-03-26T17:34:13.536 INFO:tasks.cephadm:Deploying osd.9 on smithi105 with /dev/vg_nvme/lv_1... 2024-03-26T17:34:13.536 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T17:34:13.786 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:34:14.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:14 smithi144 ceph-mon[36095]: from='osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:34:14.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:14 smithi144 ceph-mon[36095]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:34:14.196 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:14 smithi105 ceph-mon[32280]: from='osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:34:14.197 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:14 smithi105 ceph-mon[32280]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:34:14.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:14 smithi135 ceph-mon[35636]: from='osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:34:14.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:14 smithi135 ceph-mon[35636]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T17:34:15.202 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:15 smithi105 ceph-mon[32280]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T17:34:15.203 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:15 smithi105 ceph-mon[32280]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T17:34:15.203 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:15.203 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:15 smithi105 ceph-mon[32280]: from='osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:34:15.203 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:15 smithi105 ceph-mon[32280]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:34:15.203 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:15 smithi105 ceph-mon[32280]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:15.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:15 smithi144 ceph-mon[36095]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T17:34:15.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:15 smithi144 ceph-mon[36095]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T17:34:15.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:15.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:15 smithi144 ceph-mon[36095]: from='osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:34:15.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:15 smithi144 ceph-mon[36095]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:34:15.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:15 smithi144 ceph-mon[36095]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:15.351 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 17:34:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T17:34:15.061+0000 7fc2f99c8700 -1 osd.8 0 waiting for initial osdmap 2024-03-26T17:34:15.351 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 17:34:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T17:34:15.072+0000 7fc2f57bb700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:34:15.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:15 smithi135 ceph-mon[35636]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T17:34:15.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:15 smithi135 ceph-mon[35636]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T17:34:15.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:15.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:15 smithi135 ceph-mon[35636]: from='osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:34:15.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:15 smithi135 ceph-mon[35636]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:34:15.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:15 smithi135 ceph-mon[35636]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-03-26T17:34:15.735 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T17:34:15.736 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T17:34:15.736 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-26T17:34:15.736 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-26T17:34:15.736 INFO:teuthology.orchestra.run.smithi105.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114106 s, 919 MB/s 2024-03-26T17:34:15.736 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 2024-03-26T17:34:15.736 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-26T17:34:16.308 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:16 smithi144 ceph-mon[36095]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:34:16.308 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:16 smithi144 ceph-mon[36095]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T17:34:16.308 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:16 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:16.308 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:16 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:16.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:16 smithi105 ceph-mon[32280]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:34:16.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:16 smithi105 ceph-mon[32280]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T17:34:16.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:16 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:16.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:16 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:16.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:16 smithi135 ceph-mon[35636]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:34:16.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:16 smithi135 ceph-mon[35636]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T17:34:16.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:16 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:16.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:16 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:16.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_1 2024-03-26T17:34:16.706 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] boot 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi144 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi144 to 5696M 2024-03-26T17:34:17.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:17.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:17.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:17.351 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:17 smithi144 ceph-mon[36095]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T17:34:17.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:34:17.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:34:17.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] boot 2024-03-26T17:34:17.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi144 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi144 to 5696M 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:17.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:17.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:17 smithi105 ceph-mon[32280]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: osd.8 [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] boot 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi144 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:17.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi144 to 5696M 2024-03-26T17:34:17.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:17.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:17.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:17.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:17.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:17 smithi135 ceph-mon[35636]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 -- 172.21.15.105:0/1404220356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc101310 msgr2=0x7ff1fc101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 --2- 172.21.15.105:0/1404220356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc101310 0x7ff1fc101730 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00b0d0 tx=0x7ff1ec00b3e0 comp rx=0 tx=0).stop 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 -- 172.21.15.105:0/1404220356 shutdown_connections 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 --2- 172.21.15.105:0/1404220356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1fc102eb0 0x7ff1fc109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 --2- 172.21.15.105:0/1404220356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc101310 0x7ff1fc101730 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 --2- 172.21.15.105:0/1404220356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1fc102510 0x7ff1fc102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 -- 172.21.15.105:0/1404220356 >> 172.21.15.105:0/1404220356 conn(0x7ff1fc0fc8b0 msgr2=0x7ff1fc0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 -- 172.21.15.105:0/1404220356 shutdown_connections 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 -- 172.21.15.105:0/1404220356 wait complete. 2024-03-26T17:34:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 Processor -- start 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.330+0000 7ff20166b700 1 -- start start 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff20166b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1fc101310 0x7ff1fc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff20166b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1fc102510 0x7ff1fc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff20166b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc102eb0 0x7ff1fc19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff20166b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10cd90 con 0x7ff1fc102eb0 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff20166b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10cc10 con 0x7ff1fc102510 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff20166b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10cf10 con 0x7ff1fc101310 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc102eb0 0x7ff1fc19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc102eb0 0x7ff1fc19fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57208/0 (socket says 172.21.15.105:57208) 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 -- 172.21.15.105:0/465028384 learned_addr learned my addr 172.21.15.105:0/465028384 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1faffd700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1fc101310 0x7ff1fc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:34:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fa7fc700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1fc102510 0x7ff1fc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 -- 172.21.15.105:0/465028384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1fc101310 msgr2=0x7ff1fc198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1fc101310 0x7ff1fc198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 -- 172.21.15.105:0/465028384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1fc102510 msgr2=0x7ff1fc198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1fc102510 0x7ff1fc198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 -- 172.21.15.105:0/465028384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc1a01b0 con 0x7ff1fc102eb0 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1fb7fe700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc102eb0 0x7ff1fc19fb10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400c9a0 tx=0x7ff1f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1e3fff700 1 -- 172.21.15.105:0/465028384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4007ac0 con 0x7ff1fc102eb0 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc103790 con 0x7ff1fc102eb0 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1e3fff700 1 -- 172.21.15.105:0/465028384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1f4004ce0 con 0x7ff1fc102eb0 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.331+0000 7ff1e3fff700 1 -- 172.21.15.105:0/465028384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4005470 con 0x7ff1fc102eb0 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.332+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc103cd0 con 0x7ff1fc102eb0 2024-03-26T17:34:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.333+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1fc069070 con 0x7ff1fc102eb0 2024-03-26T17:34:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.334+0000 7ff1e3fff700 1 -- 172.21.15.105:0/465028384 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7ff1f4004750 con 0x7ff1fc102eb0 2024-03-26T17:34:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.334+0000 7ff1e3fff700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e40a2fb0 0x7ff1e40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:34:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.334+0000 7ff1faffd700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e40a2fb0 0x7ff1e40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:34:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.334+0000 7ff1e3fff700 1 -- 172.21.15.105:0/465028384 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7ff1f40ce650 con 0x7ff1fc102eb0 2024-03-26T17:34:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.335+0000 7ff1faffd700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e40a2fb0 0x7ff1e40a5470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009fd0 tx=0x7ff1ec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:34:18.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.341+0000 7ff1e3fff700 1 -- 172.21.15.105:0/465028384 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff1f4014030 con 0x7ff1fc102eb0 2024-03-26T17:34:18.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:18.545+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff1fc0feab0 con 0x7ff1e40a2fb0 2024-03-26T17:34:19.338 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:19 smithi105 ceph-mon[32280]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T17:34:19.338 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:19 smithi105 ceph-mon[32280]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:34:19.338 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:34:19.338 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:34:19.338 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:19.338 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:19 smithi105 ceph-mon[32280]: pgmap v197: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:19.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:19 smithi144 ceph-mon[36095]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T17:34:19.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:19 smithi144 ceph-mon[36095]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:34:19.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:34:19.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:34:19.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:19.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:19 smithi144 ceph-mon[36095]: pgmap v197: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:19.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:19 smithi135 ceph-mon[35636]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T17:34:19.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:19 smithi135 ceph-mon[35636]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:34:19.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:34:19.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:34:19.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:19.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:19 smithi135 ceph-mon[35636]: pgmap v197: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:20.341 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:20 smithi105 ceph-mon[32280]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:34:20.341 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:20 smithi105 ceph-mon[32280]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T17:34:20.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:20 smithi144 ceph-mon[36095]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:34:20.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:20 smithi144 ceph-mon[36095]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T17:34:20.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:20 smithi135 ceph-mon[35636]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:34:20.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:20 smithi135 ceph-mon[35636]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T17:34:21.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:21 smithi144 ceph-mon[36095]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T17:34:21.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:21 smithi144 ceph-mon[36095]: pgmap v200: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:21.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:21 smithi105 ceph-mon[32280]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T17:34:21.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:21 smithi105 ceph-mon[32280]: pgmap v200: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:21.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:21 smithi135 ceph-mon[35636]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T17:34:21.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:21 smithi135 ceph-mon[35636]: pgmap v200: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:22.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3864578543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1218e3-f488-426a-8ac6-afece38016e1"}]: dispatch 2024-03-26T17:34:22.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3864578543' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f1218e3-f488-426a-8ac6-afece38016e1"}]': finished 2024-03-26T17:34:22.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:22 smithi144 ceph-mon[36095]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T17:34:22.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:22 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:22.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2562364873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:34:22.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3864578543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1218e3-f488-426a-8ac6-afece38016e1"}]: dispatch 2024-03-26T17:34:22.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3864578543' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f1218e3-f488-426a-8ac6-afece38016e1"}]': finished 2024-03-26T17:34:22.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:22 smithi105 ceph-mon[32280]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T17:34:22.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:22 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:22.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2562364873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:34:22.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3864578543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1218e3-f488-426a-8ac6-afece38016e1"}]: dispatch 2024-03-26T17:34:22.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3864578543' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f1218e3-f488-426a-8ac6-afece38016e1"}]': finished 2024-03-26T17:34:22.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:22 smithi135 ceph-mon[35636]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T17:34:22.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:22 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:22.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2562364873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:34:23.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:23 smithi144 ceph-mon[36095]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:23.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:23 smithi105 ceph-mon[32280]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:23.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:23 smithi135 ceph-mon[35636]: pgmap v202: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T17:34:24.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:24 smithi135 ceph-mon[35636]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T17:34:24.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:24 smithi135 ceph-mon[35636]: Cluster is now healthy 2024-03-26T17:34:25.022 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:24 smithi105 ceph-mon[32280]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T17:34:25.022 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:24 smithi105 ceph-mon[32280]: Cluster is now healthy 2024-03-26T17:34:25.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:24 smithi144 ceph-mon[36095]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T17:34:25.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:24 smithi144 ceph-mon[36095]: Cluster is now healthy 2024-03-26T17:34:25.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:25 smithi135 ceph-mon[35636]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:34:26.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:25 smithi144 ceph-mon[36095]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:34:26.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:25 smithi105 ceph-mon[32280]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T17:34:27.693 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:27 smithi105 ceph-mon[32280]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T17:34:27.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:27 smithi135 ceph-mon[35636]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T17:34:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:27 smithi144 ceph-mon[36095]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-03-26T17:34:29.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:29 smithi135 ceph-mon[35636]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-26T17:34:29.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T17:34:29.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:30.081 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:29 smithi105 ceph-mon[32280]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-26T17:34:30.081 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T17:34:30.081 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:30.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:29 smithi144 ceph-mon[36095]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-26T17:34:30.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T17:34:30.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:30.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:30 smithi105 ceph-mon[32280]: Deploying daemon osd.9 on smithi105 2024-03-26T17:34:30.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:30 smithi135 ceph-mon[35636]: Deploying daemon osd.9 on smithi105 2024-03-26T17:34:31.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:30 smithi144 ceph-mon[36095]: Deploying daemon osd.9 on smithi105 2024-03-26T17:34:31.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:31 smithi135 ceph-mon[35636]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:34:32.026 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:31 smithi105 ceph-mon[32280]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:34:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:31 smithi144 ceph-mon[36095]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:34:33.873 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:33 smithi105 ceph-mon[32280]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-26T17:34:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:33 smithi144 ceph-mon[36095]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-26T17:34:34.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:33 smithi135 ceph-mon[35636]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-03-26T17:34:35.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:35.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:35.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:35.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:35 smithi144 ceph-mon[36095]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:34:35.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:35.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:35.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:35.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:35 smithi105 ceph-mon[32280]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:34:35.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:35.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:35.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:35.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:35 smithi135 ceph-mon[35636]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:34:37.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:37 smithi105 ceph-mon[32280]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:37.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:37.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:37.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:37.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:37.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:37.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:37 smithi135 ceph-mon[35636]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:37.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:37.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:37.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:37.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:37.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:38.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:37 smithi144 ceph-mon[36095]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:38.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:38.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:38.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:38.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:38.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:38.450 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 9 on host 'smithi105' 2024-03-26T17:34:38.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.456+0000 7ff1e3fff700 1 -- 172.21.15.105:0/465028384 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff1fc0feab0 con 0x7ff1e40a2fb0 2024-03-26T17:34:38.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e40a2fb0 msgr2=0x7ff1e40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:38.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e40a2fb0 0x7ff1e40a5470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009fd0 tx=0x7ff1ec00b040 comp rx=0 tx=0).stop 2024-03-26T17:34:38.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc102eb0 msgr2=0x7ff1fc19fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:38.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc102eb0 0x7ff1fc19fb10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400c9a0 tx=0x7ff1f400ccb0 comp rx=0 tx=0).stop 2024-03-26T17:34:38.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 shutdown_connections 2024-03-26T17:34:38.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e40a2fb0 0x7ff1e40a5470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:38.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1fc102eb0 0x7ff1fc19fb10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:38.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1fc101310 0x7ff1fc198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:38.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 --2- 172.21.15.105:0/465028384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1fc102510 0x7ff1fc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:38.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 >> 172.21.15.105:0/465028384 conn(0x7ff1fc0fc8b0 msgr2=0x7ff1fc0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:34:38.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 shutdown_connections 2024-03-26T17:34:38.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:34:38.459+0000 7ff20166b700 1 -- 172.21.15.105:0/465028384 wait complete. 2024-03-26T17:34:38.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:38 smithi105 ceph-mon[32280]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T17:34:38.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:38 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:38.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:38 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:38.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:38 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:38.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:38 smithi135 ceph-mon[35636]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T17:34:38.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:38 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:38.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:38 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:38.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:38 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:39.081 DEBUG:teuthology.orchestra.run.smithi105:osd.9> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.9.service 2024-03-26T17:34:39.084 INFO:tasks.cephadm:Deploying osd.10 on smithi135 with /dev/vg_nvme/lv_1... 2024-03-26T17:34:39.084 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T17:34:39.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:38 smithi144 ceph-mon[36095]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T17:34:39.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:38 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:39.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:38 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:39.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:38 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:34:39.330 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:34:39.616 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:39 smithi105 ceph-mon[32280]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:39.617 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:39 smithi105 ceph-mon[32280]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T17:34:39.617 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:39 smithi105 ceph-mon[32280]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T17:34:39.732 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:39 smithi135 ceph-mon[35636]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:39.732 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:39 smithi135 ceph-mon[35636]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T17:34:39.732 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:39 smithi135 ceph-mon[35636]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T17:34:39.732 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:39 smithi135 ceph-mon[35636]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:34:39.733 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:39.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:39 smithi105 ceph-mon[32280]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:34:39.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:40.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:39 smithi144 ceph-mon[36095]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:40.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:39 smithi144 ceph-mon[36095]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T17:34:40.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:39 smithi144 ceph-mon[36095]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T17:34:40.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:39 smithi144 ceph-mon[36095]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-26T17:34:40.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:40.776 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:40 smithi105 ceph-mon[32280]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:34:40.777 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:40 smithi105 ceph-mon[32280]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T17:34:40.777 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:40.777 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:40 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:40.777 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:40 smithi105 ceph-mon[32280]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' 2024-03-26T17:34:40.778 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 17:34:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T17:34:40.524+0000 7fb813a17700 -1 osd.9 0 waiting for initial osdmap 2024-03-26T17:34:40.778 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 17:34:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T17:34:40.672+0000 7fb80b802700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:34:40.885 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:40 smithi135 ceph-mon[35636]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:34:40.886 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:40 smithi135 ceph-mon[35636]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T17:34:40.886 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:40.886 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:40 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:40.886 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:40 smithi135 ceph-mon[35636]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' 2024-03-26T17:34:41.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:40 smithi144 ceph-mon[36095]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-26T17:34:41.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:40 smithi144 ceph-mon[36095]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T17:34:41.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:41.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:40 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:41.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:40 smithi144 ceph-mon[36095]: from='osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840]' entity='osd.9' 2024-03-26T17:34:41.542 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T17:34:41.542 INFO:teuthology.orchestra.run.smithi135.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T17:34:41.542 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10+0 records in 2024-03-26T17:34:41.542 INFO:teuthology.orchestra.run.smithi135.stderr:10+0 records out 2024-03-26T17:34:41.543 INFO:teuthology.orchestra.run.smithi135.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113254 s, 926 MB/s 2024-03-26T17:34:41.543 INFO:teuthology.orchestra.run.smithi135.stderr:--> Zapping successful for: 2024-03-26T17:34:41.863 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:41 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:34:41.863 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:41 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:34:41.863 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:41 smithi135 ceph-mon[35636]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:41.864 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:41 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:41.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:41 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:34:41.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:41 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:34:41.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:41 smithi105 ceph-mon[32280]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:41.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:41 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:41 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:34:42.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:41 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:34:42.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:41 smithi144 ceph-mon[36095]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:42.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:41 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:42.209 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi135:vg_nvme/lv_1 2024-03-26T17:34:42.445 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T17:34:42.871 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:42.871 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] boot 2024-03-26T17:34:42.871 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T17:34:42.871 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:42.871 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:42.872 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:42 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:42.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:42.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] boot 2024-03-26T17:34:42.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:42.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:42.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:42.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:42 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: osd.9 [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] boot 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:43.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:43.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:34:43.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:43.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:43.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:34:43.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:42 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:34:43.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:43 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi105 2024-03-26T17:34:43.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:43 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi105 to 4272M 2024-03-26T17:34:43.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:43 smithi105 ceph-mon[32280]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:43.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:43 smithi105 ceph-mon[32280]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T17:34:43.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:43 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi105 2024-03-26T17:34:43.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:43 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi105 to 4272M 2024-03-26T17:34:43.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:43 smithi135 ceph-mon[35636]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:43.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:43 smithi135 ceph-mon[35636]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 -- 172.21.15.135:0/1435722327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 msgr2=0x7ff1f8101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/1435722327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 0x7ff1f8101710 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80099c0 tx=0x7ff1e8009cd0 comp rx=0 tx=0).stop 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 -- 172.21.15.135:0/1435722327 shutdown_connections 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/1435722327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f8102e90 0x7ff1f8109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/1435722327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 0x7ff1f8101710 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/1435722327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1f81024f0 0x7ff1f8102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 -- 172.21.15.135:0/1435722327 >> 172.21.15.135:0/1435722327 conn(0x7ff1f80fc8b0 msgr2=0x7ff1f80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 -- 172.21.15.135:0/1435722327 shutdown_connections 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 -- 172.21.15.135:0/1435722327 wait complete. 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 Processor -- start 2024-03-26T17:34:43.947 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.953+0000 7ff1fdb66700 1 -- start start 2024-03-26T17:34:43.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1fdb66700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 0x7ff1f8105280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:34:43.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1fdb66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f81024f0 0x7ff1f81038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:34:43.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1fdb66700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1f8102e90 0x7ff1f8103e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:34:43.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1fdb66700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f810cd80 con 0x7ff1f8102e90 2024-03-26T17:34:43.949 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1fdb66700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f810cc00 con 0x7ff1f81012f0 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1fdb66700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f810cf00 con 0x7ff1f81024f0 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 0x7ff1f8105280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 0x7ff1f8105280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:49208/0 (socket says 172.21.15.135:49208) 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 -- 172.21.15.135:0/4218145817 learned_addr learned my addr 172.21.15.135:0/4218145817 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f6ffd700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f81024f0 0x7ff1f81038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f7fff700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1f8102e90 0x7ff1f8103e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 -- 172.21.15.135:0/4218145817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f81024f0 msgr2=0x7ff1f81038d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f81024f0 0x7ff1f81038d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 -- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1f8102e90 msgr2=0x7ff1f8103e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1f8102e90 0x7ff1f8103e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 -- 172.21.15.135:0/4218145817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f81046c0 con 0x7ff1f81012f0 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f7fff700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1f8102e90 0x7ff1f8103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:34:43.950 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f77fe700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 0x7ff1f8105280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c010 tx=0x7ff1e8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:34:43.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f4ff9700 1 -- 172.21.15.135:0/4218145817 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8027070 con 0x7ff1f81012f0 2024-03-26T17:34:43.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81a26a0 con 0x7ff1f81012f0 2024-03-26T17:34:43.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f4ff9700 1 -- 172.21.15.135:0/4218145817 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1e8004050 con 0x7ff1f81012f0 2024-03-26T17:34:43.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1f4ff9700 1 -- 172.21.15.135:0/4218145817 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8022cb0 con 0x7ff1f81012f0 2024-03-26T17:34:43.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.954+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a2b80 con 0x7ff1f81012f0 2024-03-26T17:34:43.951 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.955+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f8069070 con 0x7ff1f81012f0 2024-03-26T17:34:43.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.957+0000 7ff1f4ff9700 1 -- 172.21.15.135:0/4218145817 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7ff1e802a070 con 0x7ff1f81012f0 2024-03-26T17:34:43.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.957+0000 7ff1f4ff9700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e00a2fb0 0x7ff1e00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:34:43.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.957+0000 7ff1f4ff9700 1 -- 172.21.15.135:0/4218145817 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7ff1e80d0370 con 0x7ff1f81012f0 2024-03-26T17:34:43.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.957+0000 7ff1f6ffd700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e00a2fb0 0x7ff1e00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:34:43.952 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.958+0000 7ff1f6ffd700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e00a2fb0 0x7ff1e00a5470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005950 tx=0x7ff1ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:34:43.954 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:43.961+0000 7ff1f4ff9700 1 -- 172.21.15.135:0/4218145817 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff1e8031070 con 0x7ff1f81012f0 2024-03-26T17:34:44.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:43 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi105 2024-03-26T17:34:44.224 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:43 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi105 to 4272M 2024-03-26T17:34:44.225 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:43 smithi144 ceph-mon[36095]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-03-26T17:34:44.225 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:43 smithi144 ceph-mon[36095]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T17:34:44.225 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:34:44.163+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff1f80fead0 con 0x7ff1e00a2fb0 2024-03-26T17:34:44.886 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:34:44.886 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:34:44.887 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:44.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:34:44.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:34:44.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:45.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:34:45.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:34:45.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:45.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:45 smithi135 ceph-mon[35636]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:34:45.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:45 smithi135 ceph-mon[35636]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:45 smithi105 ceph-mon[32280]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:34:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:45 smithi105 ceph-mon[32280]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:45 smithi144 ceph-mon[36095]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:34:46.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:45 smithi144 ceph-mon[36095]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:47 smithi105 ceph-mon[32280]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:47 smithi105 ceph-mon[32280]: from='client.? 172.21.15.135:0/107525346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]: dispatch 2024-03-26T17:34:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:47 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]: dispatch 2024-03-26T17:34:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:47 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]': finished 2024-03-26T17:34:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:47 smithi105 ceph-mon[32280]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T17:34:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:34:47.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:47 smithi105 ceph-mon[32280]: from='client.? 172.21.15.135:0/719841524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:34:47.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:47 smithi135 ceph-mon[35636]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:47.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:47 smithi135 ceph-mon[35636]: from='client.? 172.21.15.135:0/107525346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]: dispatch 2024-03-26T17:34:47.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:47 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]: dispatch 2024-03-26T17:34:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:47 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]': finished 2024-03-26T17:34:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:47 smithi135 ceph-mon[35636]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T17:34:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:34:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:47 smithi135 ceph-mon[35636]: from='client.? 172.21.15.135:0/719841524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:34:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:47 smithi144 ceph-mon[36095]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:47 smithi144 ceph-mon[36095]: from='client.? 172.21.15.135:0/107525346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]: dispatch 2024-03-26T17:34:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:47 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]: dispatch 2024-03-26T17:34:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:47 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47f4a8a1-7c2d-4e70-a83e-03296b92743c"}]': finished 2024-03-26T17:34:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:47 smithi144 ceph-mon[36095]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T17:34:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:34:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:47 smithi144 ceph-mon[36095]: from='client.? 172.21.15.135:0/719841524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:34:49.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:49 smithi135 ceph-mon[35636]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:50.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:49 smithi144 ceph-mon[36095]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:50.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:49 smithi105 ceph-mon[32280]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:51.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:51 smithi135 ceph-mon[35636]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:52.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:51 smithi144 ceph-mon[36095]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:52.145 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:51 smithi105 ceph-mon[32280]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:53.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:53 smithi135 ceph-mon[35636]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:53 smithi144 ceph-mon[36095]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:54.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:53 smithi105 ceph-mon[32280]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:54.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T17:34:54.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:55.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T17:34:55.045 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:55.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T17:34:55.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:34:55.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:55 smithi135 ceph-mon[35636]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:55.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:55 smithi135 ceph-mon[35636]: Deploying daemon osd.10 on smithi135 2024-03-26T17:34:56.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:55 smithi144 ceph-mon[36095]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:56.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:55 smithi144 ceph-mon[36095]: Deploying daemon osd.10 on smithi135 2024-03-26T17:34:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:55 smithi105 ceph-mon[32280]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:55 smithi105 ceph-mon[32280]: Deploying daemon osd.10 on smithi135 2024-03-26T17:34:57.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:57 smithi135 ceph-mon[35636]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:58.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:57 smithi144 ceph-mon[36095]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:58.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:57 smithi105 ceph-mon[32280]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:34:59.804 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:34:59 smithi135 ceph-mon[35636]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:00.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:34:59 smithi144 ceph-mon[36095]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:00.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:34:59 smithi105 ceph-mon[32280]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:01.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:01.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:01.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:01.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:01 smithi144 ceph-mon[36095]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:01.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:01.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:01.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:01.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:01 smithi105 ceph-mon[32280]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:01.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:01.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:01.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:01.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:01 smithi135 ceph-mon[35636]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:03.471 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.477+0000 7ff1f4ff9700 1 -- 172.21.15.135:0/4218145817 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff1f80fead0 con 0x7ff1e00a2fb0 2024-03-26T17:35:03.472 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 10 on host 'smithi135' 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e00a2fb0 msgr2=0x7ff1e00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e00a2fb0 0x7ff1e00a5470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005950 tx=0x7ff1ec00b410 comp rx=0 tx=0).stop 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 msgr2=0x7ff1f8105280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 0x7ff1f8105280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c010 tx=0x7ff1e8009580 comp rx=0 tx=0).stop 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 shutdown_connections 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1e00a2fb0 0x7ff1e00a5470 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1f8102e90 0x7ff1f8103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1f81012f0 0x7ff1f8105280 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:03.474 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 --2- 172.21.15.135:0/4218145817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f81024f0 0x7ff1f81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:03.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 >> 172.21.15.135:0/4218145817 conn(0x7ff1f80fc8b0 msgr2=0x7ff1f80fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:03.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 shutdown_connections 2024-03-26T17:35:03.475 INFO:teuthology.orchestra.run.smithi135.stderr:2024-03-26T17:35:03.480+0000 7ff1fdb66700 1 -- 172.21.15.135:0/4218145817 wait complete. 2024-03-26T17:35:03.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:03 smithi135 ceph-mon[35636]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:03.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:03.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:03 smithi105 ceph-mon[32280]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:04.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:03 smithi144 ceph-mon[36095]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:04.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:04.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:04.133 DEBUG:teuthology.orchestra.run.smithi135:osd.10> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.10.service 2024-03-26T17:35:04.135 INFO:tasks.cephadm:Deploying osd.11 on smithi144 with /dev/vg_nvme/lv_1... 2024-03-26T17:35:04.136 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T17:35:04.382 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:35:04.755 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:04 smithi144 ceph-mon[36095]: from='osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:35:04.755 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:04 smithi144 ceph-mon[36095]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:35:04.755 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:04 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:04.755 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:04 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:04 smithi105 ceph-mon[32280]: from='osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:35:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:04 smithi105 ceph-mon[32280]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:35:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:04.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:04 smithi135 ceph-mon[35636]: from='osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:35:04.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:04 smithi135 ceph-mon[35636]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T17:35:04.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:04 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:04.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:04 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:05.822 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:05 smithi144 ceph-mon[36095]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:05.822 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:05 smithi144 ceph-mon[36095]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T17:35:05.822 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:05 smithi144 ceph-mon[36095]: from='osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:35:05.822 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:05 smithi144 ceph-mon[36095]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T17:35:05.822 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:05 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:05.822 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:05 smithi144 ceph-mon[36095]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:35:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:05 smithi105 ceph-mon[32280]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:05 smithi105 ceph-mon[32280]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T17:35:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:05 smithi105 ceph-mon[32280]: from='osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:35:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:05 smithi105 ceph-mon[32280]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T17:35:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:05 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:05 smithi105 ceph-mon[32280]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:35:05.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:05 smithi135 ceph-mon[35636]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:05.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:05 smithi135 ceph-mon[35636]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T17:35:05.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:05 smithi135 ceph-mon[35636]: from='osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:35:05.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:05 smithi135 ceph-mon[35636]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T17:35:05.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:05 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:05.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:05 smithi135 ceph-mon[35636]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-03-26T17:35:05.917 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 17:35:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T17:35:05.641+0000 7f5ef0bdf700 -1 osd.10 0 waiting for initial osdmap 2024-03-26T17:35:05.917 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 17:35:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T17:35:05.656+0000 7f5eec9d2700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:35:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T17:35:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T17:35:06.376 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-03-26T17:35:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-03-26T17:35:06.376 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011537 s, 909 MB/s 2024-03-26T17:35:06.376 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-03-26T17:35:06.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:06 smithi105 ceph-mon[32280]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:35:06.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:06 smithi105 ceph-mon[32280]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T17:35:06.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:06.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:06.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:06 smithi105 ceph-mon[32280]: osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] boot 2024-03-26T17:35:06.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:06 smithi105 ceph-mon[32280]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T17:35:06.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:06.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:06 smithi135 ceph-mon[35636]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:35:06.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:06 smithi135 ceph-mon[35636]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T17:35:06.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:06.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:06.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:06 smithi135 ceph-mon[35636]: osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] boot 2024-03-26T17:35:06.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:06 smithi135 ceph-mon[35636]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T17:35:06.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:07.028 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:06 smithi144 ceph-mon[36095]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-03-26T17:35:07.028 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:06 smithi144 ceph-mon[36095]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T17:35:07.028 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:07.028 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:07.028 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:06 smithi144 ceph-mon[36095]: osd.10 [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] boot 2024-03-26T17:35:07.028 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:06 smithi144 ceph-mon[36095]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T17:35:07.028 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T17:35:07.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-03-26T17:35:07.302 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:35:07.697 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:35:07.697 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:35:07.697 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:07.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:07 smithi144 ceph-mon[36095]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T17:35:07.810 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:35:07.810 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:35:07.810 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:07.810 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:07.810 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:07.811 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:07.811 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:07.811 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:07.811 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:07.811 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:07.811 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:07.812 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:07.812 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:07.812 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:07 smithi135 ceph-mon[35636]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T17:35:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:35:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:35:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-26T17:35:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:08.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:08.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:08.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:08.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:08.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:08.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:08.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:08.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:07 smithi105 ceph-mon[32280]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T17:35:08.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:08 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi135 2024-03-26T17:35:08.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:08 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi135 to 4272M 2024-03-26T17:35:08.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:08.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:08.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:08.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:08.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:08 smithi144 ceph-mon[36095]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T17:35:08.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:08 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi135 2024-03-26T17:35:08.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:08 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi135 to 4272M 2024-03-26T17:35:08.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:08.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:08.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:08.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:08.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:08 smithi135 ceph-mon[35636]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T17:35:09.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3526985578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 msgr2=0x7f33ec101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:09.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3526985578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 0x7f33ec101700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009990 tx=0x7f33dc009ca0 comp rx=0 tx=0).stop 2024-03-26T17:35:09.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3526985578 shutdown_connections 2024-03-26T17:35:09.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3526985578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33ec102e80 0x7f33ec109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:09.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3526985578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 0x7f33ec101700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:09.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3526985578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33ec1024e0 0x7f33ec102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:09.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3526985578 >> 172.21.15.144:0/3526985578 conn(0x7f33ec0fc8a0 msgr2=0x7f33ec0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:09.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3526985578 shutdown_connections 2024-03-26T17:35:09.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.063+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3526985578 wait complete. 2024-03-26T17:35:09.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.064+0000 7f33f1e7c700 1 Processor -- start 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.064+0000 7f33f1e7c700 1 -- start start 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.064+0000 7f33f1e7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 0x7f33ec1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.064+0000 7f33f1e7c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33ec1024e0 0x7f33ec197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.064+0000 7f33f1e7c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33ec102e80 0x7f33ec19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.064+0000 7f33f1e7c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10cdf0 con 0x7f33ec102e80 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.064+0000 7f33f1e7c700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10cc70 con 0x7f33ec1024e0 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.064+0000 7f33f1e7c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33ec10cf70 con 0x7f33ec1012e0 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 0x7f33ec1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:09.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 0x7f33ec1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45260/0 (socket says 172.21.15.144:45260) 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 -- 172.21.15.144:0/3715719993 learned_addr learned my addr 172.21.15.144:0/3715719993 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eaffd700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33ec1024e0 0x7f33ec197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 -- 172.21.15.144:0/3715719993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33ec1024e0 msgr2=0x7f33ec197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33ebfff700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33ec102e80 0x7f33ec19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33ec1024e0 0x7f33ec197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 -- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33ec102e80 msgr2=0x7f33ec19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33ec102e80 0x7f33ec19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 -- 172.21.15.144:0/3715719993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ec19f140 con 0x7f33ec1012e0 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33ebfff700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33ec102e80 0x7f33ec19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33eb7fe700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 0x7f33ec1974e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0047b0 tx=0x7f33dc005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33e8ff9700 1 -- 172.21.15.144:0/3715719993 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc026070 con 0x7f33ec1012e0 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ec19f3d0 con 0x7f33ec1012e0 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33e8ff9700 1 -- 172.21.15.144:0/3715719993 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33dc004100 con 0x7f33ec1012e0 2024-03-26T17:35:09.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33e8ff9700 1 -- 172.21.15.144:0/3715719993 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc005300 con 0x7f33ec1012e0 2024-03-26T17:35:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.065+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ec19f880 con 0x7f33ec1012e0 2024-03-26T17:35:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.066+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ec069070 con 0x7f33ec1012e0 2024-03-26T17:35:09.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.067+0000 7f33e8ff9700 1 -- 172.21.15.144:0/3715719993 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f33dc029070 con 0x7f33ec1012e0 2024-03-26T17:35:09.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.068+0000 7f33e8ff9700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33d40a2fb0 0x7f33d40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:09.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.068+0000 7f33e8ff9700 1 -- 172.21.15.144:0/3715719993 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7f33dc0cf7b0 con 0x7f33ec1012e0 2024-03-26T17:35:09.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.068+0000 7f33eaffd700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33d40a2fb0 0x7f33d40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:09.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.068+0000 7f33eaffd700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33d40a2fb0 0x7f33d40a5470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009a10 tx=0x7f33e0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:09.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.072+0000 7f33e8ff9700 1 -- 172.21.15.144:0/3715719993 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f33dc05d740 con 0x7f33ec1012e0 2024-03-26T17:35:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:08 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi135 2024-03-26T17:35:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:08 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi135 to 4272M 2024-03-26T17:35:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:09.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:08 smithi105 ceph-mon[32280]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T17:35:09.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:09.275+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f33ec0627f0 con 0x7f33d40a2fb0 2024-03-26T17:35:10.058 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:09 smithi144 ceph-mon[36095]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:10.058 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:35:10.058 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:35:10.058 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:10.059 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:09 smithi144 ceph-mon[36095]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T17:35:10.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:09 smithi105 ceph-mon[32280]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:10.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:35:10.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:35:10.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:10.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:09 smithi105 ceph-mon[32280]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T17:35:10.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:09 smithi135 ceph-mon[35636]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:10.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T17:35:10.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T17:35:10.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:10.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:09 smithi135 ceph-mon[35636]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T17:35:11.047 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:10 smithi144 ceph-mon[36095]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:35:11.047 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:10 smithi144 ceph-mon[36095]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T17:35:11.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:10 smithi105 ceph-mon[32280]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:35:11.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:10 smithi105 ceph-mon[32280]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T17:35:11.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:10 smithi135 ceph-mon[35636]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:35:11.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:10 smithi135 ceph-mon[35636]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T17:35:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:11 smithi144 ceph-mon[36095]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:12.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:11 smithi105 ceph-mon[32280]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:12.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:11 smithi135 ceph-mon[35636]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:13.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:12 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/162540902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]: dispatch 2024-03-26T17:35:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:12 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]: dispatch 2024-03-26T17:35:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:12 smithi144 ceph-mon[36095]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]': finished 2024-03-26T17:35:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:12 smithi144 ceph-mon[36095]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T17:35:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:12 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:12 smithi144 ceph-mon[36095]: from='client.? 172.21.15.144:0/78195292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:35:13.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:12 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/162540902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]: dispatch 2024-03-26T17:35:13.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:12 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]: dispatch 2024-03-26T17:35:13.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:12 smithi105 ceph-mon[32280]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]': finished 2024-03-26T17:35:13.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:12 smithi105 ceph-mon[32280]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T17:35:13.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:12 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:13.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:12 smithi105 ceph-mon[32280]: from='client.? 172.21.15.144:0/78195292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:35:13.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:12 smithi135 ceph-mon[35636]: from='client.? 172.21.15.144:0/162540902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]: dispatch 2024-03-26T17:35:13.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:12 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]: dispatch 2024-03-26T17:35:13.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:12 smithi135 ceph-mon[35636]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82fd99c0-5cc2-4744-b31a-45dac10ca1db"}]': finished 2024-03-26T17:35:13.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:12 smithi135 ceph-mon[35636]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T17:35:13.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:12 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:13.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:12 smithi135 ceph-mon[35636]: from='client.? 172.21.15.144:0/78195292' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T17:35:14.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:13 smithi144 ceph-mon[36095]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:14.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:13 smithi105 ceph-mon[32280]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:14.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:13 smithi135 ceph-mon[35636]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:16.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:15 smithi144 ceph-mon[36095]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-26T17:35:16.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:15 smithi105 ceph-mon[32280]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-26T17:35:16.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:15 smithi135 ceph-mon[35636]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-26T17:35:18.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:17 smithi144 ceph-mon[36095]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-26T17:35:18.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:17 smithi105 ceph-mon[32280]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-26T17:35:18.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:17 smithi135 ceph-mon[35636]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-26T17:35:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:19 smithi144 ceph-mon[36095]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T17:35:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T17:35:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:20.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:19 smithi105 ceph-mon[32280]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T17:35:20.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T17:35:20.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:20.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:19 smithi135 ceph-mon[35636]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T17:35:20.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T17:35:20.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:20.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:20 smithi144 ceph-mon[36095]: Deploying daemon osd.11 on smithi144 2024-03-26T17:35:21.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:20 smithi105 ceph-mon[32280]: Deploying daemon osd.11 on smithi144 2024-03-26T17:35:21.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:20 smithi135 ceph-mon[35636]: Deploying daemon osd.11 on smithi144 2024-03-26T17:35:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:21 smithi144 ceph-mon[36095]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:35:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:21 smithi105 ceph-mon[32280]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:35:22.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:21 smithi135 ceph-mon[35636]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T17:35:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:23 smithi144 ceph-mon[36095]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-26T17:35:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:23 smithi105 ceph-mon[32280]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-26T17:35:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:23 smithi135 ceph-mon[35636]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-26T17:35:25.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:25 smithi144 ceph-mon[36095]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:35:25.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:25.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:25.853 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:26.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:25 smithi105 ceph-mon[32280]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:35:26.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:26.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:26.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:26.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:25 smithi135 ceph-mon[35636]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T17:35:26.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:26.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:26.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:27.841 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:27 smithi144 ceph-mon[36095]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:28.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:27 smithi105 ceph-mon[32280]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:27 smithi135 ceph-mon[35636]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:28.634 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 11 on host 'smithi144' 2024-03-26T17:35:28.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.640+0000 7f33e8ff9700 1 -- 172.21.15.144:0/3715719993 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f33ec0627f0 con 0x7f33d40a2fb0 2024-03-26T17:35:28.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.642+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33d40a2fb0 msgr2=0x7f33d40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:28.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.642+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33d40a2fb0 0x7f33d40a5470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009a10 tx=0x7f33e0009190 comp rx=0 tx=0).stop 2024-03-26T17:35:28.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 msgr2=0x7f33ec1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:28.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 0x7f33ec1974e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0047b0 tx=0x7f33dc005ef0 comp rx=0 tx=0).stop 2024-03-26T17:35:28.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 shutdown_connections 2024-03-26T17:35:28.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33d40a2fb0 0x7f33d40a5470 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:28.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33ec102e80 0x7f33ec19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:28.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33ec1012e0 0x7f33ec1974e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:28.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 --2- 172.21.15.144:0/3715719993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33ec1024e0 0x7f33ec197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:28.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 >> 172.21.15.144:0/3715719993 conn(0x7f33ec0fc8a0 msgr2=0x7f33ec1058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:28.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 shutdown_connections 2024-03-26T17:35:28.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:28.643+0000 7f33f1e7c700 1 -- 172.21.15.144:0/3715719993 wait complete. 2024-03-26T17:35:29.397 DEBUG:teuthology.orchestra.run.smithi144:osd.11> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.11.service 2024-03-26T17:35:29.399 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-26T17:35:29.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T17:35:29.655 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:35:29.776 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:29 smithi105 ceph-mon[32280]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:29.776 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.776 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.777 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:29 smithi105 ceph-mon[32280]: from='osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:35:29.777 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:29 smithi105 ceph-mon[32280]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:35:29.777 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.777 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:29 smithi144 ceph-mon[36095]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:29.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:29 smithi144 ceph-mon[36095]: from='osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:35:29.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:29 smithi144 ceph-mon[36095]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:35:29.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:29 smithi135 ceph-mon[35636]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:29.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:29 smithi135 ceph-mon[35636]: from='osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:35:29.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:29 smithi135 ceph-mon[35636]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T17:35:29.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:29.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:30.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:30 smithi105 ceph-mon[32280]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T17:35:30.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:30 smithi105 ceph-mon[32280]: from='osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:35:30.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:30 smithi105 ceph-mon[32280]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T17:35:30.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:30.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:30 smithi105 ceph-mon[32280]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:35:30.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:30 smithi105 ceph-mon[32280]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 -- 172.21.15.105:0/1088423184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102540 msgr2=0x7f11ec1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 --2- 172.21.15.105:0/1088423184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102540 0x7f11ec1029a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f11e00099c0 tx=0x7f11e0009cd0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 -- 172.21.15.105:0/1088423184 shutdown_connections 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 --2- 172.21.15.105:0/1088423184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec102ee0 0x7f11ec109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 --2- 172.21.15.105:0/1088423184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ec101340 0x7f11ec101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 --2- 172.21.15.105:0/1088423184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102540 0x7f11ec1029a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 -- 172.21.15.105:0/1088423184 >> 172.21.15.105:0/1088423184 conn(0x7f11ec0fc900 msgr2=0x7f11ec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 -- 172.21.15.105:0/1088423184 shutdown_connections 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.081+0000 7f11f25fa700 1 -- 172.21.15.105:0/1088423184 wait complete. 2024-03-26T17:35:31.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f25fa700 1 Processor -- start 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f25fa700 1 -- start start 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f25fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ec101340 0x7f11ec1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f25fa700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec102540 0x7f11ec198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f25fa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102ee0 0x7f11ec19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f25fa700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10cde0 con 0x7f11ec102ee0 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f25fa700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10cc60 con 0x7f11ec102540 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f25fa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10cf60 con 0x7f11ec101340 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f0b97700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102ee0 0x7f11ec19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f0b97700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102ee0 0x7f11ec19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33438/0 (socket says 172.21.15.105:33438) 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11ebfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ec101340 0x7f11ec1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11f0b97700 1 -- 172.21.15.105:0/256953496 learned_addr learned my addr 172.21.15.105:0/256953496 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.082+0000 7f11eb7fe700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec102540 0x7f11ec198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11f0b97700 1 -- 172.21.15.105:0/256953496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ec101340 msgr2=0x7f11ec1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:31.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11f0b97700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ec101340 0x7f11ec1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11f0b97700 1 -- 172.21.15.105:0/256953496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec102540 msgr2=0x7f11ec198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11f0b97700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec102540 0x7f11ec198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11f0b97700 1 -- 172.21.15.105:0/256953496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec1a01d0 con 0x7f11ec102ee0 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11f0b97700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102ee0 0x7f11ec19fb30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f11e400b9a0 tx=0x7f11e400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11e97fa700 1 -- 172.21.15.105:0/256953496 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4019070 con 0x7f11ec102ee0 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11e97fa700 1 -- 172.21.15.105:0/256953496 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11e400cde0 con 0x7f11ec102ee0 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec1a04c0 con 0x7f11ec102ee0 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11e97fa700 1 -- 172.21.15.105:0/256953496 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4007370 con 0x7f11ec102ee0 2024-03-26T17:35:31.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.083+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec1a0a80 con 0x7f11ec102ee0 2024-03-26T17:35:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.084+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ec069070 con 0x7f11ec102ee0 2024-03-26T17:35:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.085+0000 7f11e97fa700 1 -- 172.21.15.105:0/256953496 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f11e401c070 con 0x7f11ec102ee0 2024-03-26T17:35:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.086+0000 7f11e97fa700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f11d40a2fb0 0x7f11d40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.086+0000 7f11ebfff700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f11d40a2fb0 0x7f11d40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.086+0000 7f11e97fa700 1 -- 172.21.15.105:0/256953496 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f11e40ceb90 con 0x7f11ec102ee0 2024-03-26T17:35:31.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.086+0000 7f11ebfff700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f11d40a2fb0 0x7f11d40a5470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f11ec1023a0 tx=0x7f11dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:31.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.090+0000 7f11e97fa700 1 -- 172.21.15.105:0/256953496 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f11e4023070 con 0x7f11ec102ee0 2024-03-26T17:35:31.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:30 smithi144 ceph-mon[36095]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T17:35:31.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:30 smithi144 ceph-mon[36095]: from='osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:35:31.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:30 smithi144 ceph-mon[36095]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T17:35:31.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:31.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:30 smithi144 ceph-mon[36095]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:35:31.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:30 smithi144 ceph-mon[36095]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:31.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:30 smithi135 ceph-mon[35636]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T17:35:31.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:30 smithi135 ceph-mon[35636]: from='osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:35:31.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:30 smithi135 ceph-mon[35636]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T17:35:31.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:31.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:30 smithi135 ceph-mon[35636]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-03-26T17:35:31.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:30 smithi135 ceph-mon[35636]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.286+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f11ec1a0d40 con 0x7f11ec102ee0 2024-03-26T17:35:31.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.286+0000 7f11e97fa700 1 -- 172.21.15.105:0/256953496 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f11e409bee0 con 0x7f11ec102ee0 2024-03-26T17:35:31.279 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f11d40a2fb0 msgr2=0x7f11d40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f11d40a2fb0 0x7f11d40a5470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f11ec1023a0 tx=0x7f11dc009260 comp rx=0 tx=0).stop 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102ee0 msgr2=0x7f11ec19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102ee0 0x7f11ec19fb30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f11e400b9a0 tx=0x7f11e400bcb0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 shutdown_connections 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f11d40a2fb0 0x7f11d40a5470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec102ee0 0x7f11ec19fb30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11ec101340 0x7f11ec1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 --2- 172.21.15.105:0/256953496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f11ec102540 0x7f11ec198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 >> 172.21.15.105:0/256953496 conn(0x7f11ec0fc900 msgr2=0x7f11ec0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:31.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 shutdown_connections 2024-03-26T17:35:31.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:31.289+0000 7f11f25fa700 1 -- 172.21.15.105:0/256953496 wait complete. 2024-03-26T17:35:31.969 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1711474506,"num_in_osds":12,"osd_in_since":1711474512,"num_remapped_pgs":0} 2024-03-26T17:35:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:31 smithi144 ceph-mon[36095]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:35:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:31 smithi144 ceph-mon[36095]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T17:35:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:32.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:31 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/256953496' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:35:32.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:31 smithi144 ceph-mon[36095]: from='osd.11 ' entity='osd.11' 2024-03-26T17:35:32.100 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 17:35:31 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T17:35:31.728+0000 7f1239479700 -1 osd.11 0 waiting for initial osdmap 2024-03-26T17:35:32.100 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 17:35:31 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T17:35:31.883+0000 7f1231264700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T17:35:32.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:31 smithi105 ceph-mon[32280]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:35:32.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:31 smithi105 ceph-mon[32280]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T17:35:32.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:32.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:32.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:31 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/256953496' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:35:32.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:31 smithi105 ceph-mon[32280]: from='osd.11 ' entity='osd.11' 2024-03-26T17:35:32.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:31 smithi135 ceph-mon[35636]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-03-26T17:35:32.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:31 smithi135 ceph-mon[35636]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T17:35:32.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:32.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:32.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:31 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/256953496' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:35:32.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:31 smithi135 ceph-mon[35636]: from='osd.11 ' entity='osd.11' 2024-03-26T17:35:32.970 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T17:35:33.086 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:32 smithi144 ceph-mon[36095]: purged_snaps scrub starts 2024-03-26T17:35:33.086 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:32 smithi144 ceph-mon[36095]: purged_snaps scrub ok 2024-03-26T17:35:33.086 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:33.086 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:32 smithi144 ceph-mon[36095]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:33.086 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:33.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:32 smithi105 ceph-mon[32280]: purged_snaps scrub starts 2024-03-26T17:35:33.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:32 smithi105 ceph-mon[32280]: purged_snaps scrub ok 2024-03-26T17:35:33.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:33.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:32 smithi105 ceph-mon[32280]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:33.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:33.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:32 smithi135 ceph-mon[35636]: purged_snaps scrub starts 2024-03-26T17:35:33.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:32 smithi135 ceph-mon[35636]: purged_snaps scrub ok 2024-03-26T17:35:33.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:33.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:32 smithi135 ceph-mon[35636]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-03-26T17:35:33.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:33.232 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:35:34.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] boot 2024-03-26T17:35:34.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T17:35:34.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:34.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: Detected new or changed devices on smithi144 2024-03-26T17:35:34.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi144 to 4272M 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:34.030 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] boot 2024-03-26T17:35:34.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: Detected new or changed devices on smithi144 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi144 to 4272M 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:34.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:34.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:34.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: osd.11 [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] boot 2024-03-26T17:35:34.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T17:35:34.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T17:35:34.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: Detected new or changed devices on smithi144 2024-03-26T17:35:34.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:35:34.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi144 to 4272M 2024-03-26T17:35:34.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:34.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:34.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:34.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:34.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:34.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:34.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 -- 172.21.15.105:0/1454300215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 msgr2=0x7ff0f410a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:34.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 --2- 172.21.15.105:0/1454300215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 0x7ff0f410a820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00b0d0 tx=0x7ff0ec00b3e0 comp rx=0 tx=0).stop 2024-03-26T17:35:34.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 -- 172.21.15.105:0/1454300215 shutdown_connections 2024-03-26T17:35:34.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 --2- 172.21.15.105:0/1454300215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 0x7ff0f410a820 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 --2- 172.21.15.105:0/1454300215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f41023d0 0x7ff0f41027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 --2- 172.21.15.105:0/1454300215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f41035d0 0x7ff0f4103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 -- 172.21.15.105:0/1454300215 >> 172.21.15.105:0/1454300215 conn(0x7ff0f40fd990 msgr2=0x7ff0f40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 -- 172.21.15.105:0/1454300215 shutdown_connections 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.873+0000 7ff0fc064700 1 -- 172.21.15.105:0/1454300215 wait complete. 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fc064700 1 Processor -- start 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fc064700 1 -- start start 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fc064700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f41023d0 0x7ff0f4076150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fc064700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f41035d0 0x7ff0f40747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fc064700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 0x7ff0f4074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fc064700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410de60 con 0x7ff0f4103f70 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fc064700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410dce0 con 0x7ff0f41023d0 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fc064700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410dfe0 con 0x7ff0f41035d0 2024-03-26T17:35:34.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fa601700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 0x7ff0f4074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0f95ff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f41035d0 0x7ff0f40747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fa601700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 0x7ff0f4074ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57352/0 (socket says 172.21.15.105:57352) 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fa601700 1 -- 172.21.15.105:0/4204405521 learned_addr learned my addr 172.21.15.105:0/4204405521 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0f9e00700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f41023d0 0x7ff0f4076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fa601700 1 -- 172.21.15.105:0/4204405521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f41035d0 msgr2=0x7ff0f40747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fa601700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f41035d0 0x7ff0f40747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fa601700 1 -- 172.21.15.105:0/4204405521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f41023d0 msgr2=0x7ff0f4076150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fa601700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f41023d0 0x7ff0f4076150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.874+0000 7ff0fa601700 1 -- 172.21.15.105:0/4204405521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f4075590 con 0x7ff0f4103f70 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.875+0000 7ff0fa601700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 0x7ff0f4074ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009cd0 tx=0x7ff0ec004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.875+0000 7ff0eaffd700 1 -- 172.21.15.105:0/4204405521 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec01f030 con 0x7ff0f4103f70 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.875+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f41a25e0 con 0x7ff0f4103f70 2024-03-26T17:35:34.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.875+0000 7ff0eaffd700 1 -- 172.21.15.105:0/4204405521 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0ec005040 con 0x7ff0f4103f70 2024-03-26T17:35:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.875+0000 7ff0eaffd700 1 -- 172.21.15.105:0/4204405521 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec01b690 con 0x7ff0f4103f70 2024-03-26T17:35:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.875+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41a2a90 con 0x7ff0f4103f70 2024-03-26T17:35:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.876+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f4069070 con 0x7ff0f4103f70 2024-03-26T17:35:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.877+0000 7ff0eaffd700 1 -- 172.21.15.105:0/4204405521 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7ff0ec022030 con 0x7ff0f4103f70 2024-03-26T17:35:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.877+0000 7ff0eaffd700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0e00a2fb0 0x7ff0e00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.877+0000 7ff0f9e00700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0e00a2fb0 0x7ff0e00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.878+0000 7ff0eaffd700 1 -- 172.21.15.105:0/4204405521 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff0ec00f020 con 0x7ff0f4103f70 2024-03-26T17:35:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.878+0000 7ff0f9e00700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0e00a2fb0 0x7ff0e00a5470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4103430 tx=0x7ff0f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:34.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:34.881+0000 7ff0eaffd700 1 -- 172.21.15.105:0/4204405521 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff0ec029020 con 0x7ff0f4103f70 2024-03-26T17:35:34.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:34 smithi105 ceph-mon[32280]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T17:35:34.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:34 smithi105 ceph-mon[32280]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:35.067 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:34 smithi135 ceph-mon[35636]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T17:35:35.068 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:34 smithi135 ceph-mon[35636]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:35.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.078+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff0f402dca0 con 0x7ff0f4103f70 2024-03-26T17:35:35.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.078+0000 7ff0eaffd700 1 -- 172.21.15.105:0/4204405521 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7ff0f402dca0 con 0x7ff0f4103f70 2024-03-26T17:35:35.072 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:35.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0e00a2fb0 msgr2=0x7ff0e00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:35.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0e00a2fb0 0x7ff0e00a5470 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4103430 tx=0x7ff0f0009260 comp rx=0 tx=0).stop 2024-03-26T17:35:35.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 msgr2=0x7ff0f4074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:35.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 0x7ff0f4074ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009cd0 tx=0x7ff0ec004ce0 comp rx=0 tx=0).stop 2024-03-26T17:35:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 shutdown_connections 2024-03-26T17:35:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0e00a2fb0 0x7ff0e00a5470 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f4103f70 0x7ff0f4074ce0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0f41023d0 0x7ff0f4076150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 --2- 172.21.15.105:0/4204405521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f41035d0 0x7ff0f40747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 >> 172.21.15.105:0/4204405521 conn(0x7ff0f40fd990 msgr2=0x7ff0f41068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 shutdown_connections 2024-03-26T17:35:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:35.080+0000 7ff0fc064700 1 -- 172.21.15.105:0/4204405521 wait complete. 2024-03-26T17:35:35.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:34 smithi144 ceph-mon[36095]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T17:35:35.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:34 smithi144 ceph-mon[36095]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:35.822 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711474532,"num_in_osds":12,"osd_in_since":1711474512,"num_remapped_pgs":0} 2024-03-26T17:35:35.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T17:35:36.059 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:35:36.079 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:35 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4204405521' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:35:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:35 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4204405521' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:35:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:35 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4204405521' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T17:35:37.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:36 smithi144 ceph-mon[36095]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:37.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:36 smithi105 ceph-mon[32280]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:37.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:36 smithi135 ceph-mon[35636]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 -- 172.21.15.105:0/4103612607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 msgr2=0x7f35801087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 --2- 172.21.15.105:0/4103612607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 0x7f35801087c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35700099c0 tx=0x7f3570009cd0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 -- 172.21.15.105:0/4103612607 shutdown_connections 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 --2- 172.21.15.105:0/4103612607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3580102dc0 0x7f358010f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 --2- 172.21.15.105:0/4103612607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 0x7f35801087c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 --2- 172.21.15.105:0/4103612607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3580102400 0x7f3580102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 -- 172.21.15.105:0/4103612607 >> 172.21.15.105:0/4103612607 conn(0x7f35800794e0 msgr2=0x7f35800798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 -- 172.21.15.105:0/4103612607 shutdown_connections 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.746+0000 7f35849fc700 1 -- 172.21.15.105:0/4103612607 wait complete. 2024-03-26T17:35:37.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f35849fc700 1 Processor -- start 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f35849fc700 1 -- start start 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f35849fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3580102400 0x7f358019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f35849fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3580102dc0 0x7f358019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f35849fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 0x7f3580198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f35849fc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3580112540 con 0x7f35801083e0 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f35849fc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35801123c0 con 0x7f3580102dc0 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f35849fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35801126c0 con 0x7f3580102400 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357ed9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 0x7f3580198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357ed9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 0x7f3580198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57364/0 (socket says 172.21.15.105:57364) 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357ed9d700 1 -- 172.21.15.105:0/2165890597 learned_addr learned my addr 172.21.15.105:0/2165890597 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357e59c700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3580102400 0x7f358019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357dd9b700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3580102dc0 0x7f358019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357ed9d700 1 -- 172.21.15.105:0/2165890597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3580102400 msgr2=0x7f358019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:37.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357ed9d700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3580102400 0x7f358019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357ed9d700 1 -- 172.21.15.105:0/2165890597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3580102dc0 msgr2=0x7f358019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357ed9d700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3580102dc0 0x7f358019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.747+0000 7f357ed9d700 1 -- 172.21.15.105:0/2165890597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3580198fe0 con 0x7f35801083e0 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.748+0000 7f357ed9d700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 0x7f3580198880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3578007ee0 tx=0x7f357800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.748+0000 7f356f7fe700 1 -- 172.21.15.105:0/2165890597 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578004bb0 con 0x7f35801083e0 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.748+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35801992d0 con 0x7f35801083e0 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.748+0000 7f356f7fe700 1 -- 172.21.15.105:0/2165890597 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3578004d50 con 0x7f35801083e0 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.748+0000 7f356f7fe700 1 -- 172.21.15.105:0/2165890597 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35780058e0 con 0x7f35801083e0 2024-03-26T17:35:37.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.748+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801a5590 con 0x7f35801083e0 2024-03-26T17:35:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.749+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3580069070 con 0x7f35801083e0 2024-03-26T17:35:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.750+0000 7f356f7fe700 1 -- 172.21.15.105:0/2165890597 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f357801a020 con 0x7f35801083e0 2024-03-26T17:35:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.750+0000 7f356f7fe700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35680a2fb0 0x7f35680a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.750+0000 7f357e59c700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35680a2fb0 0x7f35680a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.751+0000 7f356f7fe700 1 -- 172.21.15.105:0/2165890597 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f35780ce5e0 con 0x7f35801083e0 2024-03-26T17:35:37.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.751+0000 7f357e59c700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35680a2fb0 0x7f35680a5470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3570000c00 tx=0x7f3570005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:37.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.755+0000 7f356f7fe700 1 -- 172.21.15.105:0/2165890597 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3578010070 con 0x7f35801083e0 2024-03-26T17:35:37.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.950+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f358002dc00 con 0x7f35801083e0 2024-03-26T17:35:37.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.951+0000 7f356f7fe700 1 -- 172.21.15.105:0/2165890597 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18491 (secure 0 0 0) 0x7f357809b8b0 con 0x7f35801083e0 2024-03-26T17:35:37.945 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:37.945 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":81,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:35:33.812900+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:35:37.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35680a2fb0 msgr2=0x7f35680a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35680a2fb0 0x7f35680a5470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3570000c00 tx=0x7f3570005fb0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 msgr2=0x7f3580198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 0x7f3580198880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3578007ee0 tx=0x7f357800d3b0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 shutdown_connections 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35680a2fb0 0x7f35680a5470 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35801083e0 0x7f3580198880 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3580102400 0x7f358019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 --2- 172.21.15.105:0/2165890597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3580102dc0 0x7f358019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 >> 172.21.15.105:0/2165890597 conn(0x7f35800794e0 msgr2=0x7f3580101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 shutdown_connections 2024-03-26T17:35:37.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:37.953+0000 7f35849fc700 1 -- 172.21.15.105:0/2165890597 wait complete. 2024-03-26T17:35:38.262 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:38 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2165890597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:35:38.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:38 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2165890597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:35:38.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:38 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2165890597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:35:38.583 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:35:38.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-26T17:35:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:35:39.258 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:39 smithi105 ceph-mon[32280]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:39.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:39 smithi144 ceph-mon[36095]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:39.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:39 smithi135 ceph-mon[35636]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:40.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 -- 172.21.15.105:0/3110020206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 msgr2=0x7fc5ac1076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:40.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 --2- 172.21.15.105:0/3110020206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 0x7fc5ac1076d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009990 tx=0x7fc59c009ca0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 -- 172.21.15.105:0/3110020206 shutdown_connections 2024-03-26T17:35:40.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 --2- 172.21.15.105:0/3110020206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac101c50 0x7fc5ac10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 --2- 172.21.15.105:0/3110020206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 0x7fc5ac1076d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 --2- 172.21.15.105:0/3110020206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ac1012b0 0x7fc5ac101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 -- 172.21.15.105:0/3110020206 >> 172.21.15.105:0/3110020206 conn(0x7fc5ac0781e0 msgr2=0x7fc5ac0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:40.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 -- 172.21.15.105:0/3110020206 shutdown_connections 2024-03-26T17:35:40.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.416+0000 7fc5b2139700 1 -- 172.21.15.105:0/3110020206 wait complete. 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5b2139700 1 Processor -- start 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5b2139700 1 -- start start 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5b2139700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ac1012b0 0x7fc5ac199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5b2139700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac101c50 0x7fc5ac199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5b2139700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 0x7fc5ac1a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5b2139700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac111460 con 0x7fc5ac1072f0 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5b2139700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1112e0 con 0x7fc5ac1012b0 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5b2139700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1115e0 con 0x7fc5ac101c50 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5abfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 0x7fc5ac1a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:40.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5abfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 0x7fc5ac1a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57380/0 (socket says 172.21.15.105:57380) 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5abfff700 1 -- 172.21.15.105:0/2741226297 learned_addr learned my addr 172.21.15.105:0/2741226297 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5ab7fe700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ac1012b0 0x7fc5ac199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.417+0000 7fc5a2ffd700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac101c50 0x7fc5ac199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5abfff700 1 -- 172.21.15.105:0/2741226297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac101c50 msgr2=0x7fc5ac199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5abfff700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac101c50 0x7fc5ac199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5abfff700 1 -- 172.21.15.105:0/2741226297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ac1012b0 msgr2=0x7fc5ac199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5abfff700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ac1012b0 0x7fc5ac199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5abfff700 1 -- 172.21.15.105:0/2741226297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac1a4ec0 con 0x7fc5ac1072f0 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5abfff700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 0x7fc5ac1a4980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400b970 tx=0x7fc5a400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5a97fa700 1 -- 172.21.15.105:0/2741226297 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400d860 con 0x7fc5ac1072f0 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac1a50f0 con 0x7fc5ac1072f0 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5a97fa700 1 -- 172.21.15.105:0/2741226297 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5a400e430 con 0x7fc5ac1072f0 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5a97fa700 1 -- 172.21.15.105:0/2741226297 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4017660 con 0x7fc5ac1072f0 2024-03-26T17:35:40.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.418+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac1a5600 con 0x7fc5ac1072f0 2024-03-26T17:35:40.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.419+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac069070 con 0x7fc5ac1072f0 2024-03-26T17:35:40.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.420+0000 7fc5a97fa700 1 -- 172.21.15.105:0/2741226297 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fc5a400e5e0 con 0x7fc5ac1072f0 2024-03-26T17:35:40.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.421+0000 7fc5a97fa700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5980a2fb0 0x7fc5980a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:40.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.421+0000 7fc5ab7fe700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5980a2fb0 0x7fc5980a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:40.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.421+0000 7fc5a97fa700 1 -- 172.21.15.105:0/2741226297 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc5a40ced30 con 0x7fc5ac1072f0 2024-03-26T17:35:40.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.421+0000 7fc5ab7fe700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5980a2fb0 0x7fc5980a5470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b5c0 tx=0x7fc59c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:40.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.424+0000 7fc5a97fa700 1 -- 172.21.15.105:0/2741226297 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc5a4013030 con 0x7fc5ac1072f0 2024-03-26T17:35:40.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.618+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5ac02dc00 con 0x7fc5ac1072f0 2024-03-26T17:35:40.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.618+0000 7fc5a97fa700 1 -- 172.21.15.105:0/2741226297 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fc5a409c000 con 0x7fc5ac1072f0 2024-03-26T17:35:40.613 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:35:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5980a2fb0 msgr2=0x7fc5980a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5980a2fb0 0x7fc5980a5470 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00b5c0 tx=0x7fc59c005960 comp rx=0 tx=0).stop 2024-03-26T17:35:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 msgr2=0x7fc5ac1a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 0x7fc5ac1a4980 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400b970 tx=0x7fc5a400bc80 comp rx=0 tx=0).stop 2024-03-26T17:35:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 shutdown_connections 2024-03-26T17:35:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5980a2fb0 0x7fc5980a5470 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5ac1072f0 0x7fc5ac1a4980 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5ac1012b0 0x7fc5ac199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 --2- 172.21.15.105:0/2741226297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5ac101c50 0x7fc5ac199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 >> 172.21.15.105:0/2741226297 conn(0x7fc5ac0781e0 msgr2=0x7fc5ac1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 shutdown_connections 2024-03-26T17:35:40.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:40.621+0000 7fc5b2139700 1 -- 172.21.15.105:0/2741226297 wait complete. 2024-03-26T17:35:40.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:40 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2741226297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:35:40.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:40 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2741226297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:35:41.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:40 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2741226297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:35:41.397 INFO:tasks.cephadm:Adding mds.a on smithi105 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.d on smithi105 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.g on smithi105 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.j on smithi105 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.b on smithi135 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.e on smithi135 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.h on smithi135 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.k on smithi135 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.c on smithi144 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.f on smithi144 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.i on smithi144 2024-03-26T17:35:41.398 INFO:tasks.cephadm:Adding mds.l on smithi144 2024-03-26T17:35:41.398 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l' 2024-03-26T17:35:41.647 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T17:35:41.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:41 smithi144 ceph-mon[36095]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:41.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:41 smithi135 ceph-mon[35636]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:42.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:41 smithi105 ceph-mon[32280]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:43.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 -- 172.21.15.144:0/48368376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 msgr2=0x7f43b8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 --2- 172.21.15.144:0/48368376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 0x7f43b8102820 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009990 tx=0x7f43a8009ca0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 -- 172.21.15.144:0/48368376 shutdown_connections 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 --2- 172.21.15.144:0/48368376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b8103fa0 0x7f43b810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 --2- 172.21.15.144:0/48368376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 0x7f43b8102820 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 --2- 172.21.15.144:0/48368376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b8103600 0x7f43b8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 -- 172.21.15.144:0/48368376 >> 172.21.15.144:0/48368376 conn(0x7f43b80fd9c0 msgr2=0x7f43b80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 -- 172.21.15.144:0/48368376 shutdown_connections 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.265+0000 7f43c0694700 1 -- 172.21.15.144:0/48368376 wait complete. 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43c0694700 1 Processor -- start 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43c0694700 1 -- start start 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43c0694700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 0x7f43b8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43c0694700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b8103600 0x7f43b8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43c0694700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b8103fa0 0x7f43b819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43c0694700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810def0 con 0x7f43b8103600 2024-03-26T17:35:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43c0694700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810dd70 con 0x7f43b8103fa0 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43c0694700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810e070 con 0x7f43b8102400 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43be430700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 0x7f43b8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43be430700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 0x7f43b8198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57714/0 (socket says 172.21.15.144:57714) 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43be430700 1 -- 172.21.15.144:0/1371798272 learned_addr learned my addr 172.21.15.144:0/1371798272 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43bdc2f700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b8103600 0x7f43b8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43bec31700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b8103fa0 0x7f43b819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43be430700 1 -- 172.21.15.144:0/1371798272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b8103fa0 msgr2=0x7f43b819fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43be430700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b8103fa0 0x7f43b819fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43be430700 1 -- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b8103600 msgr2=0x7f43b8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43be430700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b8103600 0x7f43b8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.266+0000 7f43be430700 1 -- 172.21.15.144:0/1371798272 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b81a01f0 con 0x7f43b8102400 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.267+0000 7f43be430700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 0x7f43b8198590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f43a8000c00 tx=0x7f43a80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.267+0000 7f43af7fe700 1 -- 172.21.15.144:0/1371798272 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8004830 con 0x7f43b8102400 2024-03-26T17:35:43.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.267+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b81a0480 con 0x7f43b8102400 2024-03-26T17:35:43.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.267+0000 7f43af7fe700 1 -- 172.21.15.144:0/1371798272 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43a8003bd0 con 0x7f43b8102400 2024-03-26T17:35:43.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.267+0000 7f43af7fe700 1 -- 172.21.15.144:0/1371798272 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8005380 con 0x7f43b8102400 2024-03-26T17:35:43.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.267+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b81a0950 con 0x7f43b8102400 2024-03-26T17:35:43.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.268+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b8069070 con 0x7f43b8102400 2024-03-26T17:35:43.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.269+0000 7f43af7fe700 1 -- 172.21.15.144:0/1371798272 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f43a8003d80 con 0x7f43b8102400 2024-03-26T17:35:43.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.270+0000 7f43af7fe700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f43a40a2fb0 0x7f43a40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:43.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.270+0000 7f43af7fe700 1 -- 172.21.15.144:0/1371798272 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f43a80d0670 con 0x7f43b8102400 2024-03-26T17:35:43.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.270+0000 7f43bdc2f700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f43a40a2fb0 0x7f43a40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:43.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.270+0000 7f43bdc2f700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f43a40a2fb0 0x7f43a40a5470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009a10 tx=0x7f43b4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:43.267 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.273+0000 7f43af7fe700 1 -- 172.21.15.144:0/1371798272 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f43a8013070 con 0x7f43b8102400 2024-03-26T17:35:43.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.479+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}) -- 0x7f43b8062ab0 con 0x7f43a40a2fb0 2024-03-26T17:35:43.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.487+0000 7f43af7fe700 1 -- 172.21.15.144:0/1371798272 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f43b8062ab0 con 0x7f43a40a2fb0 2024-03-26T17:35:43.481 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled mds.all update... 2024-03-26T17:35:43.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f43a40a2fb0 msgr2=0x7f43a40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:43.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f43a40a2fb0 0x7f43a40a5470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009a10 tx=0x7f43b4009190 comp rx=0 tx=0).stop 2024-03-26T17:35:43.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 msgr2=0x7f43b8198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:43.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 0x7f43b8198590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f43a8000c00 tx=0x7f43a80047b0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 shutdown_connections 2024-03-26T17:35:43.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f43a40a2fb0 0x7f43a40a5470 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f43b8103fa0 0x7f43b819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43b8102400 0x7f43b8198590 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 --2- 172.21.15.144:0/1371798272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43b8103600 0x7f43b8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:43.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 >> 172.21.15.144:0/1371798272 conn(0x7f43b80fd9c0 msgr2=0x7f43b81069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:43.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 shutdown_connections 2024-03-26T17:35:43.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-03-26T17:35:43.490+0000 7f43c0694700 1 -- 172.21.15.144:0/1371798272 wait complete. 2024-03-26T17:35:43.946 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:43 smithi144 ceph-mon[36095]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:43.946 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:43.946 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:44.094 DEBUG:teuthology.orchestra.run.smithi105:mds.a> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.a.service 2024-03-26T17:35:44.097 DEBUG:teuthology.orchestra.run.smithi105:mds.d> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.d.service 2024-03-26T17:35:44.122 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:43 smithi105 ceph-mon[32280]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:44.125 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:44.125 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:44.126 DEBUG:teuthology.orchestra.run.smithi105:mds.g> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.g.service 2024-03-26T17:35:44.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:43 smithi135 ceph-mon[35636]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:44.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:44.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:35:44.168 DEBUG:teuthology.orchestra.run.smithi105:mds.j> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.j.service 2024-03-26T17:35:44.212 DEBUG:teuthology.orchestra.run.smithi135:mds.b> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.b.service 2024-03-26T17:35:44.214 DEBUG:teuthology.orchestra.run.smithi135:mds.e> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.e.service 2024-03-26T17:35:44.257 DEBUG:teuthology.orchestra.run.smithi135:mds.h> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.h.service 2024-03-26T17:35:44.300 DEBUG:teuthology.orchestra.run.smithi135:mds.k> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.k.service 2024-03-26T17:35:44.343 DEBUG:teuthology.orchestra.run.smithi144:mds.c> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.c.service 2024-03-26T17:35:44.346 DEBUG:teuthology.orchestra.run.smithi144:mds.f> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.f.service 2024-03-26T17:35:44.388 DEBUG:teuthology.orchestra.run.smithi144:mds.i> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.i.service 2024-03-26T17:35:44.431 DEBUG:teuthology.orchestra.run.smithi144:mds.l> sudo journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.l.service 2024-03-26T17:35:44.474 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-26T17:35:44.474 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.592+0000 7fa5e4177640 1 -- 172.21.15.105:0/3730668962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc0ffbb0 msgr2=0x7fa5dc0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.592+0000 7fa5e4177640 1 --2- 172.21.15.105:0/3730668962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc0ffbb0 0x7fa5dc0fffd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009930 tx=0x7fa5d802bc30 comp rx=0 tx=0).stop 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 -- 172.21.15.105:0/3730668962 shutdown_connections 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 --2- 172.21.15.105:0/3730668962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc103540 0x7fa5dc109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 --2- 172.21.15.105:0/3730668962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5dc102b80 0x7fa5dc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 --2- 172.21.15.105:0/3730668962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc0ffbb0 0x7fa5dc0fffd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 -- 172.21.15.105:0/3730668962 >> 172.21.15.105:0/3730668962 conn(0x7fa5dc0fb4a0 msgr2=0x7fa5dc0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 -- 172.21.15.105:0/3730668962 shutdown_connections 2024-03-26T17:35:44.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 -- 172.21.15.105:0/3730668962 wait complete. 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 Processor -- start 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 -- start start 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5dc0ffbb0 0x7fa5dc197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc102b80 0x7fa5dc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc103540 0x7fa5dc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc10d430 con 0x7fa5dc102b80 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc10d2b0 con 0x7fa5dc0ffbb0 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e4177640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc10d5b0 con 0x7fa5dc103540 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e1712640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc102b80 0x7fa5dc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:44.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e1712640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc102b80 0x7fa5dc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40870/0 (socket says 172.21.15.105:40870) 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.593+0000 7fa5e1712640 1 -- 172.21.15.105:0/2126390204 learned_addr learned my addr 172.21.15.105:0/2126390204 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e2714640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc103540 0x7fa5dc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e1f13640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5dc0ffbb0 0x7fa5dc197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e1712640 1 -- 172.21.15.105:0/2126390204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc103540 msgr2=0x7fa5dc19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e1712640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc103540 0x7fa5dc19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e1712640 1 -- 172.21.15.105:0/2126390204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5dc0ffbb0 msgr2=0x7fa5dc197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e1712640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5dc0ffbb0 0x7fa5dc197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e1712640 1 -- 172.21.15.105:0/2126390204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc19f760 con 0x7fa5dc102b80 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e1712640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc102b80 0x7fa5dc1980d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00d950 tx=0x7fa5cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5d2ffd640 1 -- 172.21.15.105:0/2126390204 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc017070 con 0x7fa5dc102b80 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5d2ffd640 1 -- 172.21.15.105:0/2126390204 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5cc009c30 con 0x7fa5dc102b80 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5d2ffd640 1 -- 172.21.15.105:0/2126390204 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00f620 con 0x7fa5dc102b80 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc19fa50 con 0x7fa5dc102b80 2024-03-26T17:35:44.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.594+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc19ff50 con 0x7fa5dc102b80 2024-03-26T17:35:44.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.595+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc108dd0 con 0x7fa5dc102b80 2024-03-26T17:35:44.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.596+0000 7fa5d2ffd640 1 -- 172.21.15.105:0/2126390204 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fa5cc0109a0 con 0x7fa5dc102b80 2024-03-26T17:35:44.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.596+0000 7fa5d2ffd640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c80a2f20 0x7fa5c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:44.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.597+0000 7fa5e1f13640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c80a2f20 0x7fa5c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:44.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.597+0000 7fa5d2ffd640 1 -- 172.21.15.105:0/2126390204 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa5cc0ceb40 con 0x7fa5dc102b80 2024-03-26T17:35:44.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.597+0000 7fa5e1f13640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c80a2f20 0x7fa5c80a53e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8000c00 tx=0x7fa5d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:44.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.601+0000 7fa5d2ffd640 1 -- 172.21.15.105:0/2126390204 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa5cc009de0 con 0x7fa5dc102b80 2024-03-26T17:35:44.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.782+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5dc02d490 con 0x7fa5dc102b80 2024-03-26T17:35:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.783+0000 7fa5d2ffd640 1 -- 172.21.15.105:0/2126390204 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18491 (secure 0 0 0) 0x7fa5cc09b7c0 con 0x7fa5dc102b80 2024-03-26T17:35:44.777 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:44.778 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":81,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:35:33.812900+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:35:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c80a2f20 msgr2=0x7fa5c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c80a2f20 0x7fa5c80a53e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8000c00 tx=0x7fa5d8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc102b80 msgr2=0x7fa5dc1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc102b80 0x7fa5dc1980d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00d950 tx=0x7fa5cc00de10 comp rx=0 tx=0).stop 2024-03-26T17:35:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 shutdown_connections 2024-03-26T17:35:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c80a2f20 0x7fa5c80a53e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5dc103540 0x7fa5dc19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5dc102b80 0x7fa5dc1980d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 --2- 172.21.15.105:0/2126390204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5dc0ffbb0 0x7fa5dc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 >> 172.21.15.105:0/2126390204 conn(0x7fa5dc0fb4a0 msgr2=0x7fa5dc105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 shutdown_connections 2024-03-26T17:35:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.785+0000 7fa5e4177640 1 -- 172.21.15.105:0/2126390204 wait complete. 2024-03-26T17:35:44.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:44 smithi105 ceph-mon[32280]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:35:44.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:44 smithi105 ceph-mon[32280]: Saving service mds.all spec with placement smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-03-26T17:35:44.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:44.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:44.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:44.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:44.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:44.790 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:44.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:35:44.792 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:35:44.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 -- 172.21.15.105:0/3167497685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28102c40 msgr2=0x7f0c281030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:44.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 --2- 172.21.15.105:0/3167497685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28102c40 0x7f0c281030c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009930 tx=0x7f0c1802bc30 comp rx=0 tx=0).stop 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 -- 172.21.15.105:0/3167497685 shutdown_connections 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 --2- 172.21.15.105:0/3167497685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c28103600 0x7f0c28109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 --2- 172.21.15.105:0/3167497685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28102c40 0x7f0c281030c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 --2- 172.21.15.105:0/3167497685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280ffbd0 0x7f0c280ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 -- 172.21.15.105:0/3167497685 >> 172.21.15.105:0/3167497685 conn(0x7f0c280fb4c0 msgr2=0x7f0c280fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 -- 172.21.15.105:0/3167497685 shutdown_connections 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.903+0000 7f0c30332640 1 -- 172.21.15.105:0/3167497685 wait complete. 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c30332640 1 Processor -- start 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c30332640 1 -- start start 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c30332640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280ffbd0 0x7f0c28197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c30332640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c28102c40 0x7f0c281980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c30332640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28103600 0x7f0c2819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c30332640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2810d430 con 0x7f0c28103600 2024-03-26T17:35:44.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c30332640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2810d2b0 con 0x7f0c28102c40 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c30332640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2810d5b0 con 0x7f0c280ffbd0 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c2e8cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28103600 0x7f0c2819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c2e0ce640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280ffbd0 0x7f0c28197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c2e8cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28103600 0x7f0c2819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40882/0 (socket says 172.21.15.105:40882) 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c2e8cf640 1 -- 172.21.15.105:0/1630333087 learned_addr learned my addr 172.21.15.105:0/1630333087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c2e0ce640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280ffbd0 0x7f0c28197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47186/0 (socket says 172.21.15.105:47186) 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.904+0000 7f0c2d8cd640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c28102c40 0x7f0c281980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c2e8cf640 1 -- 172.21.15.105:0/1630333087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280ffbd0 msgr2=0x7f0c28197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c2e8cf640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280ffbd0 0x7f0c28197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c2e8cf640 1 -- 172.21.15.105:0/1630333087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c28102c40 msgr2=0x7f0c281980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c2e8cf640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c28102c40 0x7f0c281980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c2e8cf640 1 -- 172.21.15.105:0/1630333087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2819f790 con 0x7f0c28103600 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c2e8cf640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28103600 0x7f0c2819f120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000cad0 tx=0x7f0c2000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:44.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/1630333087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2000ae20 con 0x7f0c28103600 2024-03-26T17:35:44.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/1630333087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c20005020 con 0x7f0c28103600 2024-03-26T17:35:44.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2819fa80 con 0x7f0c28103600 2024-03-26T17:35:44.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/1630333087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2000f6b0 con 0x7f0c28103600 2024-03-26T17:35:44.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.905+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2819ff50 con 0x7f0c28103600 2024-03-26T17:35:44.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.906+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c28108e90 con 0x7f0c28103600 2024-03-26T17:35:44.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.907+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/1630333087 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f0c200051d0 con 0x7f0c28103600 2024-03-26T17:35:44.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.907+0000 7f0c1f7fe640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c140a2f20 0x7f0c140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:44.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.907+0000 7f0c2e0ce640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c140a2f20 0x7f0c140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:44.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.908+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/1630333087 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0c200cef00 con 0x7f0c28103600 2024-03-26T17:35:44.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.908+0000 7f0c2e0ce640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c140a2f20 0x7f0c140a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c24005fd0 tx=0x7f0c2400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:44.912+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/1630333087 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0c20014030 con 0x7f0c28103600 2024-03-26T17:35:44.994 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:44 smithi144 ceph-mon[36095]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:35:44.994 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:44 smithi144 ceph-mon[36095]: Saving service mds.all spec with placement smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-03-26T17:35:44.994 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:44.994 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:44.995 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:44.995 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:44.995 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:44.995 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:45.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.085+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c2802d3f0 con 0x7f0c28103600 2024-03-26T17:35:45.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.086+0000 7f0c1f7fe640 1 -- 172.21.15.105:0/1630333087 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f0c2009bb80 con 0x7f0c28103600 2024-03-26T17:35:45.080 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c140a2f20 msgr2=0x7f0c140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c140a2f20 0x7f0c140a53e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0c24005fd0 tx=0x7f0c2400b390 comp rx=0 tx=0).stop 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28103600 msgr2=0x7f0c2819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28103600 0x7f0c2819f120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000cad0 tx=0x7f0c2000cf90 comp rx=0 tx=0).stop 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 shutdown_connections 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c140a2f20 0x7f0c140a53e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c28103600 0x7f0c2819f120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c28102c40 0x7f0c281980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 --2- 172.21.15.105:0/1630333087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280ffbd0 0x7f0c28197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 >> 172.21.15.105:0/1630333087 conn(0x7f0c280fb4c0 msgr2=0x7f0c28106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:45.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 shutdown_connections 2024-03-26T17:35:45.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.088+0000 7f0c30332640 1 -- 172.21.15.105:0/1630333087 wait complete. 2024-03-26T17:35:45.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:35:45.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:44 smithi135 ceph-mon[35636]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:35:45.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:44 smithi135 ceph-mon[35636]: Saving service mds.all spec with placement smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-03-26T17:35:45.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:45.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:35:45.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:45.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:45.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:45.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 -- 172.21.15.105:0/4181896692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 msgr2=0x7fbb00108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 --2- 172.21.15.105:0/4181896692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 0x7fbb00108ca0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009ca0 tx=0x7fbaf802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 -- 172.21.15.105:0/4181896692 shutdown_connections 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 --2- 172.21.15.105:0/4181896692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 0x7fbb00108ca0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 --2- 172.21.15.105:0/4181896692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb00101a50 0x7fbb00101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 --2- 172.21.15.105:0/4181896692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb00100860 0x7fbb00100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 -- 172.21.15.105:0/4181896692 >> 172.21.15.105:0/4181896692 conn(0x7fbb000fbdf0 msgr2=0x7fbb000fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 -- 172.21.15.105:0/4181896692 shutdown_connections 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.202+0000 7fbb04b09640 1 -- 172.21.15.105:0/4181896692 wait complete. 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbb04b09640 1 Processor -- start 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbb04b09640 1 -- start start 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbb04b09640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb00100860 0x7fbb00197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbb04b09640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb00101a50 0x7fbb00197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbb04b09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 0x7fbb0019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbb04b09640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0010c320 con 0x7fbb00102410 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbb04b09640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0010c1a0 con 0x7fbb00100860 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbb04b09640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0010c4a0 con 0x7fbb00101a50 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 0x7fbb0019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 0x7fbb0019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40888/0 (socket says 172.21.15.105:40888) 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 -- 172.21.15.105:0/1307517238 learned_addr learned my addr 172.21.15.105:0/1307517238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafe59c640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb00100860 0x7fbb00197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbaf7fff640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb00101a50 0x7fbb00197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 -- 172.21.15.105:0/1307517238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb00101a50 msgr2=0x7fbb00197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb00101a50 0x7fbb00197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 -- 172.21.15.105:0/1307517238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb00100860 msgr2=0x7fbb00197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb00100860 0x7fbb00197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 -- 172.21.15.105:0/1307517238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0019f670 con 0x7fbb00102410 2024-03-26T17:35:45.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.203+0000 7fbafed9d640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 0x7fbb0019f000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbaf802c700 tx=0x7fbaf80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.204+0000 7fbaf77fe640 1 -- 172.21.15.105:0/1307517238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8041020 con 0x7fbb00102410 2024-03-26T17:35:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.204+0000 7fbaf77fe640 1 -- 172.21.15.105:0/1307517238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaf8003e20 con 0x7fbb00102410 2024-03-26T17:35:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.204+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0019f900 con 0x7fbb00102410 2024-03-26T17:35:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.204+0000 7fbaf77fe640 1 -- 172.21.15.105:0/1307517238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8039040 con 0x7fbb00102410 2024-03-26T17:35:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.204+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0019fdf0 con 0x7fbb00102410 2024-03-26T17:35:45.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.205+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0019fff0 con 0x7fbb00102410 2024-03-26T17:35:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.206+0000 7fbaf77fe640 1 -- 172.21.15.105:0/1307517238 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fbaf8044020 con 0x7fbb00102410 2024-03-26T17:35:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.206+0000 7fbaf77fe640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbaec0a2f20 0x7fbaec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.206+0000 7fbafe59c640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbaec0a2f20 0x7fbaec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.206+0000 7fbaf77fe640 1 -- 172.21.15.105:0/1307517238 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fbaf80f13a0 con 0x7fbb00102410 2024-03-26T17:35:45.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.207+0000 7fbafe59c640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbaec0a2f20 0x7fbaec0a53e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005fd0 tx=0x7fbaf000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.210+0000 7fbaf77fe640 1 -- 172.21.15.105:0/1307517238 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbaf804b020 con 0x7fbb00102410 2024-03-26T17:35:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.423+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb001a0380 con 0x7fbb00102410 2024-03-26T17:35:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.424+0000 7fbaf77fe640 1 -- 172.21.15.105:0/1307517238 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fbaf80be670 con 0x7fbb00102410 2024-03-26T17:35:45.418 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:45.418 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-26T17:35:45.418 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 1 2024-03-26T17:35:45.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbaec0a2f20 msgr2=0x7fbaec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:45.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbaec0a2f20 0x7fbaec0a53e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005fd0 tx=0x7fbaf000b390 comp rx=0 tx=0).stop 2024-03-26T17:35:45.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 msgr2=0x7fbb0019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:45.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 0x7fbb0019f000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbaf802c700 tx=0x7fbaf80328a0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 shutdown_connections 2024-03-26T17:35:45.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbaec0a2f20 0x7fbaec0a53e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb00102410 0x7fbb0019f000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb00101a50 0x7fbb00197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 --2- 172.21.15.105:0/1307517238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb00100860 0x7fbb00197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:45.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 >> 172.21.15.105:0/1307517238 conn(0x7fbb000fbdf0 msgr2=0x7fbb000fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:45.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 shutdown_connections 2024-03-26T17:35:45.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:45.426+0000 7fbb04b09640 1 -- 172.21.15.105:0/1307517238 wait complete. 2024-03-26T17:35:45.740 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:45 smithi144 ceph-mon[36095]: Deploying daemon mds.l on smithi144 2024-03-26T17:35:45.740 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:45 smithi144 ceph-mon[36095]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:45.740 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2126390204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:35:45.741 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1630333087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:35:45.741 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1307517238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:45 smithi105 ceph-mon[32280]: Deploying daemon mds.l on smithi144 2024-03-26T17:35:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:45 smithi105 ceph-mon[32280]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2126390204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:35:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1630333087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:35:45.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1307517238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:46.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:45 smithi135 ceph-mon[35636]: Deploying daemon mds.l on smithi144 2024-03-26T17:35:46.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:45 smithi135 ceph-mon[35636]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:46.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2126390204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:35:46.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1630333087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:35:46.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1307517238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:46.349 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 systemd[1]: Starting Ceph mds.l for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:35:46.746 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 podman[66118]: 2024-03-26 17:35:46.327325776 +0000 UTC m=+0.017949297 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:35:46.746 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 podman[66118]: 2024-03-26 17:35:46.458002154 +0000 UTC m=+0.148625646 container create a7b6aeb1d2e201644d2f559532c1858d1d1cd2f92b0d9857a2fcebc2a218ab22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T17:35:46.746 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 podman[66118]: 2024-03-26 17:35:46.753492479 +0000 UTC m=+0.444115968 container init a7b6aeb1d2e201644d2f559532c1858d1d1cd2f92b0d9857a2fcebc2a218ab22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T17:35:47.100 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 podman[66118]: 2024-03-26 17:35:46.756252076 +0000 UTC m=+0.446875569 container start a7b6aeb1d2e201644d2f559532c1858d1d1cd2f92b0d9857a2fcebc2a218ab22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T17:35:47.100 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 ceph-mds[66137]: -- 172.21.15.144:0/3491705685 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f195feba40 con 0x55f195fbe800 2024-03-26T17:35:47.100 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l[66133]: starting mds.l at 2024-03-26T17:35:47.100 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 bash[66118]: a7b6aeb1d2e201644d2f559532c1858d1d1cd2f92b0d9857a2fcebc2a218ab22 2024-03-26T17:35:47.100 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 17:35:46 smithi144 systemd[1]: Started Ceph mds.l for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:35:47.432 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:35:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 -- 172.21.15.105:0/1338559494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234107660 msgr2=0x7f4234109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 --2- 172.21.15.105:0/1338559494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234107660 0x7f4234109a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f422c009ca0 tx=0x7f422c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 -- 172.21.15.105:0/1338559494 shutdown_connections 2024-03-26T17:35:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 --2- 172.21.15.105:0/1338559494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234107660 0x7f4234109a50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 --2- 172.21.15.105:0/1338559494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4234104d30 0x7f4234107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 --2- 172.21.15.105:0/1338559494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4234102400 0x7f42341047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 -- 172.21.15.105:0/1338559494 >> 172.21.15.105:0/1338559494 conn(0x7f42340fbd70 msgr2=0x7f42340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:47.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 -- 172.21.15.105:0/1338559494 shutdown_connections 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.552+0000 7f423bcfe640 1 -- 172.21.15.105:0/1338559494 wait complete. 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423bcfe640 1 Processor -- start 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423bcfe640 1 -- start start 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423bcfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4234102400 0x7f4234197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423bcfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234104d30 0x7f42341980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423bcfe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4234107660 0x7f423419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423bcfe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f423410d4a0 con 0x7f4234104d30 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423bcfe640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f423410d320 con 0x7f4234102400 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423bcfe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f423410d620 con 0x7f4234107660 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239299640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234104d30 0x7f42341980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:47.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239a9a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4234102400 0x7f4234197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239299640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234104d30 0x7f42341980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40912/0 (socket says 172.21.15.105:40912) 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239a9a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4234102400 0x7f4234197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:47580/0 (socket says 172.21.15.105:47580) 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239299640 1 -- 172.21.15.105:0/2496699148 learned_addr learned my addr 172.21.15.105:0/2496699148 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423a29b640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4234107660 0x7f423419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239299640 1 -- 172.21.15.105:0/2496699148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4234107660 msgr2=0x7f423419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239299640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4234107660 0x7f423419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239299640 1 -- 172.21.15.105:0/2496699148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4234102400 msgr2=0x7f4234197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239299640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4234102400 0x7f4234197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f4239299640 1 -- 172.21.15.105:0/2496699148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423419f7c0 con 0x7f4234104d30 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.553+0000 7f423a29b640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4234107660 0x7f423419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.554+0000 7f4239299640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234104d30 0x7f42341980e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f423000b450 tx=0x7f423000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.554+0000 7f422affd640 1 -- 172.21.15.105:0/2496699148 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230015020 con 0x7f4234104d30 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.554+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423419fab0 con 0x7f4234104d30 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.554+0000 7f422affd640 1 -- 172.21.15.105:0/2496699148 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4230011d40 con 0x7f4234104d30 2024-03-26T17:35:47.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.554+0000 7f422affd640 1 -- 172.21.15.105:0/2496699148 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423000f340 con 0x7f4234104d30 2024-03-26T17:35:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.554+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42341a0000 con 0x7f4234104d30 2024-03-26T17:35:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.555+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42340fd860 con 0x7f4234104d30 2024-03-26T17:35:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.556+0000 7f422affd640 1 -- 172.21.15.105:0/2496699148 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f4230018020 con 0x7f4234104d30 2024-03-26T17:35:47.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.556+0000 7f422affd640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42200a2f20 0x7f42200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.556+0000 7f4239a9a640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42200a2f20 0x7f42200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.557+0000 7f422affd640 1 -- 172.21.15.105:0/2496699148 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f42300cef80 con 0x7f4234104d30 2024-03-26T17:35:47.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.557+0000 7f4239a9a640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42200a2f20 0x7f42200a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4224009bd0 tx=0x7f4224005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:47.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.560+0000 7f422affd640 1 -- 172.21.15.105:0/2496699148 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f423001f020 con 0x7f4234104d30 2024-03-26T17:35:47.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.775+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f423402d240 con 0x7f4234104d30 2024-03-26T17:35:47.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.776+0000 7f422affd640 1 -- 172.21.15.105:0/2496699148 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f423000d040 con 0x7f4234104d30 2024-03-26T17:35:47.770 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:47.770 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-26T17:35:47.770 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 2 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.777+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42200a2f20 msgr2=0x7f42200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.777+0000 7f423bcfe640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42200a2f20 0x7f42200a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4224009bd0 tx=0x7f4224005f90 comp rx=0 tx=0).stop 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.777+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234104d30 msgr2=0x7f42341980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.777+0000 7f423bcfe640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234104d30 0x7f42341980e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f423000b450 tx=0x7f423000b910 comp rx=0 tx=0).stop 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.777+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 shutdown_connections 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.777+0000 7f423bcfe640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42200a2f20 0x7f42200a53e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.778+0000 7f423bcfe640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4234107660 0x7f423419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.778+0000 7f423bcfe640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4234104d30 0x7f42341980e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.778+0000 7f423bcfe640 1 --2- 172.21.15.105:0/2496699148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4234102400 0x7f4234197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.778+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 >> 172.21.15.105:0/2496699148 conn(0x7f42340fbd70 msgr2=0x7f4234102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:47.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.778+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 shutdown_connections 2024-03-26T17:35:47.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:47.778+0000 7f423bcfe640 1 -- 172.21.15.105:0/2496699148 wait complete. 2024-03-26T17:35:47.781 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6832/3569777961,v1:172.21.15.144:6833/3569777961] up:boot 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: fsmap 1 up:standby 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:47.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:47.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6832/3569777961,v1:172.21.15.144:6833/3569777961] up:boot 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: fsmap 1 up:standby 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:47.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6832/3569777961,v1:172.21.15.144:6833/3569777961] up:boot 2024-03-26T17:35:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: fsmap 1 up:standby 2024-03-26T17:35:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T17:35:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:48.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:48.983 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:48 smithi135 ceph-mon[35636]: Deploying daemon mds.k on smithi135 2024-03-26T17:35:48.983 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:48 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2496699148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:48 smithi144 ceph-mon[36095]: Deploying daemon mds.k on smithi135 2024-03-26T17:35:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:48 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2496699148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:49.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:48 smithi105 ceph-mon[32280]: Deploying daemon mds.k on smithi135 2024-03-26T17:35:49.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:48 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2496699148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:49.375 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 systemd[1]: Starting Ceph mds.k for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:35:49.634 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 podman[64745]: 2024-03-26 17:35:49.285211818 +0000 UTC m=+0.018185523 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:35:49.635 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 podman[64745]: 2024-03-26 17:35:49.387979589 +0000 UTC m=+0.120953338 container create 34eb9a26b5abaa12b788cd846c29331fedac8177f42ab09135861155d454e24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0) 2024-03-26T17:35:49.635 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 podman[64745]: 2024-03-26 17:35:49.616211495 +0000 UTC m=+0.349185180 container init 34eb9a26b5abaa12b788cd846c29331fedac8177f42ab09135861155d454e24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, ceph=True) 2024-03-26T17:35:49.635 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 podman[64745]: 2024-03-26 17:35:49.618735312 +0000 UTC m=+0.351709003 container start 34eb9a26b5abaa12b788cd846c29331fedac8177f42ab09135861155d454e24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-26T17:35:49.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:35:49.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 -- 172.21.15.105:0/2038953608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f8102390 msgr2=0x7f58f8108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:49.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 --2- 172.21.15.105:0/2038953608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f8102390 0x7f58f8108c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009900 tx=0x7f58e802bc30 comp rx=0 tx=0).stop 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 -- 172.21.15.105:0/2038953608 shutdown_connections 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 --2- 172.21.15.105:0/2038953608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f8102390 0x7f58f8108c20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 --2- 172.21.15.105:0/2038953608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f81019d0 0x7f58f8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 --2- 172.21.15.105:0/2038953608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f81007e0 0x7f58f8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 -- 172.21.15.105:0/2038953608 >> 172.21.15.105:0/2038953608 conn(0x7f58f80fbd50 msgr2=0x7f58f80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 -- 172.21.15.105:0/2038953608 shutdown_connections 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.906+0000 7f58fd510640 1 -- 172.21.15.105:0/2038953608 wait complete. 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58fd510640 1 Processor -- start 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58fd510640 1 -- start start 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58fd510640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f81007e0 0x7f58f8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58fd510640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f81019d0 0x7f58f8198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58fd510640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f8102390 0x7f58f819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58fd510640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58f810c310 con 0x7f58f81007e0 2024-03-26T17:35:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58fd510640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58f810c190 con 0x7f58f81019d0 2024-03-26T17:35:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58fd510640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58f810c490 con 0x7f58f8102390 2024-03-26T17:35:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f81007e0 0x7f58f8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58f6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f81007e0 0x7f58f8197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40930/0 (socket says 172.21.15.105:40930) 2024-03-26T17:35:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58f6ffd640 1 -- 172.21.15.105:0/3011534079 learned_addr learned my addr 172.21.15.105:0/3011534079 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.907+0000 7f58f67fc640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f81019d0 0x7f58f8198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58f77fe640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f8102390 0x7f58f819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58f6ffd640 1 -- 172.21.15.105:0/3011534079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f8102390 msgr2=0x7f58f819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58f6ffd640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f8102390 0x7f58f819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58f6ffd640 1 -- 172.21.15.105:0/3011534079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f81019d0 msgr2=0x7f58f8198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58f6ffd640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f81019d0 0x7f58f8198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58f6ffd640 1 -- 172.21.15.105:0/3011534079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f819f6e0 con 0x7f58f81007e0 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58f6ffd640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f81007e0 0x7f58f8197ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f58f000eb70 tx=0x7f58f000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58dffff640 1 -- 172.21.15.105:0/3011534079 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0003b70 con 0x7f58f81007e0 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f819f9d0 con 0x7f58f81007e0 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58dffff640 1 -- 172.21.15.105:0/3011534079 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58f000ee70 con 0x7f58f81007e0 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f819ff20 con 0x7f58f81007e0 2024-03-26T17:35:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.908+0000 7f58dffff640 1 -- 172.21.15.105:0/3011534079 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0010650 con 0x7f58f81007e0 2024-03-26T17:35:49.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.909+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f80686b0 con 0x7f58f81007e0 2024-03-26T17:35:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.910+0000 7f58dffff640 1 -- 172.21.15.105:0/3011534079 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f58f0004a40 con 0x7f58f81007e0 2024-03-26T17:35:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.911+0000 7f58dffff640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58e00a2f20 0x7f58e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.911+0000 7f58f67fc640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58e00a2f20 0x7f58e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.911+0000 7f58dffff640 1 -- 172.21.15.105:0/3011534079 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f58f00cf810 con 0x7f58f81007e0 2024-03-26T17:35:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.911+0000 7f58f67fc640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58e00a2f20 0x7f58e00a53e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f58f8198fd0 tx=0x7f58ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:49.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:49.914+0000 7f58dffff640 1 -- 172.21.15.105:0/3011534079 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f58f009ca40 con 0x7f58f81007e0 2024-03-26T17:35:49.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:49 smithi135 ceph-mon[35636]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:49.916 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 ceph-mds[64764]: -- 172.21.15.135:0/4050890998 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f8e486fa40 con 0x55f8e4842800 2024-03-26T17:35:49.916 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k[64760]: starting mds.k at 2024-03-26T17:35:49.916 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 bash[64745]: 34eb9a26b5abaa12b788cd846c29331fedac8177f42ab09135861155d454e24e 2024-03-26T17:35:49.916 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 17:35:49 smithi135 systemd[1]: Started Ceph mds.k for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:35:50.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:49 smithi144 ceph-mon[36095]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:50.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.135+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58f81a0210 con 0x7f58f81007e0 2024-03-26T17:35:50.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.135+0000 7f58dffff640 1 -- 172.21.15.105:0/3011534079 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f58f009c340 con 0x7f58f81007e0 2024-03-26T17:35:50.130 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:50.130 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T17:35:50.130 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 3 2024-03-26T17:35:50.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.137+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58e00a2f20 msgr2=0x7f58e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.137+0000 7f58fd510640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58e00a2f20 0x7f58e00a53e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f58f8198fd0 tx=0x7f58ec005f90 comp rx=0 tx=0).stop 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.137+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f81007e0 msgr2=0x7f58f8197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.137+0000 7f58fd510640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f81007e0 0x7f58f8197ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f58f000eb70 tx=0x7f58f000c6a0 comp rx=0 tx=0).stop 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.138+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 shutdown_connections 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.138+0000 7f58fd510640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58e00a2f20 0x7f58e00a53e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.138+0000 7f58fd510640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58f8102390 0x7f58f819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.138+0000 7f58fd510640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58f81019d0 0x7f58f8198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.138+0000 7f58fd510640 1 --2- 172.21.15.105:0/3011534079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58f81007e0 0x7f58f8197ac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.138+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 >> 172.21.15.105:0/3011534079 conn(0x7f58f80fbd50 msgr2=0x7f58f8068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.138+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 shutdown_connections 2024-03-26T17:35:50.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:50.138+0000 7f58fd510640 1 -- 172.21.15.105:0/3011534079 wait complete. 2024-03-26T17:35:50.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:49 smithi105 ceph-mon[32280]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:50.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6832/1499648854,v1:172.21.15.135:6833/1499648854] up:boot 2024-03-26T17:35:50.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: fsmap 2 up:standby 2024-03-26T17:35:50.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T17:35:50.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:50.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:50.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:50.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:50.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:50.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:50.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:50 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3011534079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:51.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6832/1499648854,v1:172.21.15.135:6833/1499648854] up:boot 2024-03-26T17:35:51.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: fsmap 2 up:standby 2024-03-26T17:35:51.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T17:35:51.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:51.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:51.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:51.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:51.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:51.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:51.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:50 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3011534079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:51.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6832/1499648854,v1:172.21.15.135:6833/1499648854] up:boot 2024-03-26T17:35:51.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: fsmap 2 up:standby 2024-03-26T17:35:51.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T17:35:51.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:51.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:51.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:50 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3011534079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:52.024 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:51 smithi135 ceph-mon[35636]: Deploying daemon mds.j on smithi105 2024-03-26T17:35:52.024 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:51 smithi135 ceph-mon[35636]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:52.098 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:51 smithi105 ceph-mon[32280]: Deploying daemon mds.j on smithi105 2024-03-26T17:35:52.098 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:51 smithi105 ceph-mon[32280]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:52.099 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:51 smithi105 systemd[1]: Starting Ceph mds.j for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:35:52.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:51 smithi144 ceph-mon[36095]: Deploying daemon mds.j on smithi105 2024-03-26T17:35:52.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:51 smithi144 ceph-mon[36095]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:52.144 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:35:52.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 -- 172.21.15.105:0/2717985837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf01010f0 msgr2=0x7faaf0101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:52.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 --2- 172.21.15.105:0/2717985837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf01010f0 0x7faaf0101570 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faae0009900 tx=0x7faae002bc30 comp rx=0 tx=0).stop 2024-03-26T17:35:52.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 -- 172.21.15.105:0/2717985837 shutdown_connections 2024-03-26T17:35:52.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 --2- 172.21.15.105:0/2717985837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf0101ab0 0x7faaf01083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 --2- 172.21.15.105:0/2717985837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf01010f0 0x7faaf0101570 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 --2- 172.21.15.105:0/2717985837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf00fff20 0x7faaf0100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 -- 172.21.15.105:0/2717985837 >> 172.21.15.105:0/2717985837 conn(0x7faaf00fb4e0 msgr2=0x7faaf00fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:52.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 -- 172.21.15.105:0/2717985837 shutdown_connections 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.259+0000 7faaf7913640 1 -- 172.21.15.105:0/2717985837 wait complete. 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf7913640 1 Processor -- start 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf7913640 1 -- start start 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf7913640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf00fff20 0x7faaf0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf7913640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf01010f0 0x7faaf0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf7913640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf0101ab0 0x7faaf019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf7913640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faaf010c350 con 0x7faaf0101ab0 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf7913640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faaf010c1d0 con 0x7faaf00fff20 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf7913640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faaf010c4d0 con 0x7faaf01010f0 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf5eb0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf0101ab0 0x7faaf019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf5eb0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf0101ab0 0x7faaf019f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40946/0 (socket says 172.21.15.105:40946) 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf5eb0640 1 -- 172.21.15.105:0/3289663293 learned_addr learned my addr 172.21.15.105:0/3289663293 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:52.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf56af640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf00fff20 0x7faaf0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf4eae640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf01010f0 0x7faaf0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.260+0000 7faaf5eb0640 1 -- 172.21.15.105:0/3289663293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf01010f0 msgr2=0x7faaf0197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faaf5eb0640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf01010f0 0x7faaf0197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faaf5eb0640 1 -- 172.21.15.105:0/3289663293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf00fff20 msgr2=0x7faaf0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faaf5eb0640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf00fff20 0x7faaf0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faaf5eb0640 1 -- 172.21.15.105:0/3289663293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf019f6c0 con 0x7faaf0101ab0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faaf4eae640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf01010f0 0x7faaf0197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faaf5eb0640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf0101ab0 0x7faaf019f050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faae800eaa0 tx=0x7faae800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faae67fc640 1 -- 172.21.15.105:0/3289663293 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae800cdb0 con 0x7faaf0101ab0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf019f9b0 con 0x7faaf0101ab0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faae67fc640 1 -- 172.21.15.105:0/3289663293 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faae8005020 con 0x7faaf0101ab0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faae67fc640 1 -- 172.21.15.105:0/3289663293 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae80116d0 con 0x7faaf0101ab0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.261+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf019feb0 con 0x7faaf0101ab0 2024-03-26T17:35:52.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.262+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf01a01a0 con 0x7faaf0101ab0 2024-03-26T17:35:52.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.263+0000 7faae67fc640 1 -- 172.21.15.105:0/3289663293 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7faae80051d0 con 0x7faaf0101ab0 2024-03-26T17:35:52.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.264+0000 7faae67fc640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faadc0a2f20 0x7faadc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:52.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.264+0000 7faaf56af640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faadc0a2f20 0x7faadc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:52.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.264+0000 7faae67fc640 1 -- 172.21.15.105:0/3289663293 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7faae80cf540 con 0x7faaf0101ab0 2024-03-26T17:35:52.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.264+0000 7faaf56af640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faadc0a2f20 0x7faadc0a53e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faaf00ffc40 tx=0x7faaec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:52.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.268+0000 7faae67fc640 1 -- 172.21.15.105:0/3289663293 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faae8016030 con 0x7faaf0101ab0 2024-03-26T17:35:52.360 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:52 smithi105 podman[72230]: 2024-03-26 17:35:52.008123864 +0000 UTC m=+0.018519830 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:35:52.360 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:52 smithi105 podman[72230]: 2024-03-26 17:35:52.236505983 +0000 UTC m=+0.246901958 container create 772e1ba46c6523514a84a35c4c980551467436bbb024aa5db363d908ba63b2df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS) 2024-03-26T17:35:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.487+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaf01a0370 con 0x7faaf0101ab0 2024-03-26T17:35:52.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.488+0000 7faae67fc640 1 -- 172.21.15.105:0/3289663293 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7faae809c810 con 0x7faaf0101ab0 2024-03-26T17:35:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:52.481 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T17:35:52.481 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 3 2024-03-26T17:35:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faadc0a2f20 msgr2=0x7faadc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faadc0a2f20 0x7faadc0a53e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faaf00ffc40 tx=0x7faaec005f90 comp rx=0 tx=0).stop 2024-03-26T17:35:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf0101ab0 msgr2=0x7faaf019f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf0101ab0 0x7faaf019f050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faae800eaa0 tx=0x7faae800ef60 comp rx=0 tx=0).stop 2024-03-26T17:35:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 shutdown_connections 2024-03-26T17:35:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faadc0a2f20 0x7faadc0a53e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faaf0101ab0 0x7faaf019f050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaf01010f0 0x7faaf0197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 --2- 172.21.15.105:0/3289663293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faaf00fff20 0x7faaf0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:52.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 >> 172.21.15.105:0/3289663293 conn(0x7faaf00fb4e0 msgr2=0x7faaf0108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:52.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 shutdown_connections 2024-03-26T17:35:52.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:52.490+0000 7faaf7913640 1 -- 172.21.15.105:0/3289663293 wait complete. 2024-03-26T17:35:52.643 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:52 smithi105 podman[72230]: 2024-03-26 17:35:52.47256643 +0000 UTC m=+0.482962397 container init 772e1ba46c6523514a84a35c4c980551467436bbb024aa5db363d908ba63b2df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T17:35:52.643 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:52 smithi105 podman[72230]: 2024-03-26 17:35:52.475426472 +0000 UTC m=+0.485822442 container start 772e1ba46c6523514a84a35c4c980551467436bbb024aa5db363d908ba63b2df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T17:35:52.643 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-mds[72288]: -- 172.21.15.105:0/1742905427 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5ce211a40 con 0x55d5ce1e4c00 2024-03-26T17:35:52.643 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j[72284]: starting mds.j at 2024-03-26T17:35:52.643 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:52 smithi105 bash[72230]: 772e1ba46c6523514a84a35c4c980551467436bbb024aa5db363d908ba63b2df 2024-03-26T17:35:52.643 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 17:35:52 smithi105 systemd[1]: Started Ceph mds.j for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:35:52.956 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:52 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3289663293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:52.956 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:52.956 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:52.956 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:52.956 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:52.956 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:52.957 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:53.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:52 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3289663293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:53.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3289663293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:53.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:53.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:53.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:53.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:53.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:53.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:53.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:53 smithi105 ceph-mon[32280]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:53.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:53 smithi105 ceph-mon[32280]: Deploying daemon mds.d on smithi105 2024-03-26T17:35:53.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:53 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.105:6834/3032565642,v1:172.21.15.105:6835/3032565642] up:boot 2024-03-26T17:35:53.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:53 smithi105 ceph-mon[32280]: fsmap 3 up:standby 2024-03-26T17:35:53.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T17:35:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:53 smithi144 ceph-mon[36095]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:53 smithi144 ceph-mon[36095]: Deploying daemon mds.d on smithi105 2024-03-26T17:35:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:53 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.105:6834/3032565642,v1:172.21.15.105:6835/3032565642] up:boot 2024-03-26T17:35:54.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:53 smithi144 ceph-mon[36095]: fsmap 3 up:standby 2024-03-26T17:35:54.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T17:35:54.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:53 smithi135 ceph-mon[35636]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:54.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:53 smithi135 ceph-mon[35636]: Deploying daemon mds.d on smithi105 2024-03-26T17:35:54.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:53 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.105:6834/3032565642,v1:172.21.15.105:6835/3032565642] up:boot 2024-03-26T17:35:54.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:53 smithi135 ceph-mon[35636]: fsmap 3 up:standby 2024-03-26T17:35:54.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T17:35:54.495 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:35:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 -- 172.21.15.105:0/3693104519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7638102390 msgr2=0x7f7638108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 --2- 172.21.15.105:0/3693104519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7638102390 0x7f7638108c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7630009ca0 tx=0x7f763002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 -- 172.21.15.105:0/3693104519 shutdown_connections 2024-03-26T17:35:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 --2- 172.21.15.105:0/3693104519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7638102390 0x7f7638108c20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 --2- 172.21.15.105:0/3693104519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76381019d0 0x7f7638101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 --2- 172.21.15.105:0/3693104519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76381007e0 0x7f7638100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 -- 172.21.15.105:0/3693104519 >> 172.21.15.105:0/3693104519 conn(0x7f76380fbd50 msgr2=0x7f76380fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 -- 172.21.15.105:0/3693104519 shutdown_connections 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 -- 172.21.15.105:0/3693104519 wait complete. 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.619+0000 7f763fd1b640 1 Processor -- start 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763fd1b640 1 -- start start 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763fd1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76381007e0 0x7f76381969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763fd1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76381019d0 0x7f7638196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763fd1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7638102390 0x7f763819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763fd1b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f763810c310 con 0x7f76381007e0 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763fd1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f763810c190 con 0x7f7638102390 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763fd1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f763810c490 con 0x7f76381019d0 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76381007e0 0x7f76381969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76381007e0 0x7f76381969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50450/0 (socket says 172.21.15.105:50450) 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 -- 172.21.15.105:0/2054082975 learned_addr learned my addr 172.21.15.105:0/2054082975 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:54.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763e2b8640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7638102390 0x7f763819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763d2b6640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76381019d0 0x7f7638196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 -- 172.21.15.105:0/2054082975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76381019d0 msgr2=0x7f7638196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76381019d0 0x7f7638196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 -- 172.21.15.105:0/2054082975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7638102390 msgr2=0x7f763819df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7638102390 0x7f763819df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 -- 172.21.15.105:0/2054082975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763819e5d0 con 0x7f76381007e0 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763d2b6640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76381019d0 0x7f7638196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763dab7640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76381007e0 0x7f76381969b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7634007920 tx=0x7f7634007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:54.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f762effd640 1 -- 172.21.15.105:0/2054082975 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634019070 con 0x7f76381007e0 2024-03-26T17:35:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f762effd640 1 -- 172.21.15.105:0/2054082975 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7634005020 con 0x7f76381007e0 2024-03-26T17:35:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.620+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7638068a80 con 0x7f76381007e0 2024-03-26T17:35:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.621+0000 7f762effd640 1 -- 172.21.15.105:0/2054082975 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634004a00 con 0x7f76381007e0 2024-03-26T17:35:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.621+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7638068f80 con 0x7f76381007e0 2024-03-26T17:35:54.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.621+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76380686b0 con 0x7f76381007e0 2024-03-26T17:35:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.623+0000 7f762effd640 1 -- 172.21.15.105:0/2054082975 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f763401c070 con 0x7f76381007e0 2024-03-26T17:35:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.624+0000 7f762effd640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76240a2f20 0x7f76240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.624+0000 7f763d2b6640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76240a2f20 0x7f76240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.624+0000 7f762effd640 1 -- 172.21.15.105:0/2054082975 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f76340d0120 con 0x7f76381007e0 2024-03-26T17:35:54.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.624+0000 7f763d2b6640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76240a2f20 0x7f76240a53e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7638197ec0 tx=0x7f7628005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:54.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.628+0000 7f762effd640 1 -- 172.21.15.105:0/2054082975 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f763409d3f0 con 0x7f76381007e0 2024-03-26T17:35:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.856+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76381078a0 con 0x7f76381007e0 2024-03-26T17:35:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.857+0000 7f762effd640 1 -- 172.21.15.105:0/2054082975 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7f763409ccf0 con 0x7f76381007e0 2024-03-26T17:35:54.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:54.850 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T17:35:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 4 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76240a2f20 msgr2=0x7f76240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76240a2f20 0x7f76240a53e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7638197ec0 tx=0x7f7628005ec0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76381007e0 msgr2=0x7f76381969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76381007e0 0x7f76381969b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7634007920 tx=0x7f7634007de0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 shutdown_connections 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76240a2f20 0x7f76240a53e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7638102390 0x7f763819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76381019d0 0x7f7638196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 --2- 172.21.15.105:0/2054082975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76381007e0 0x7f76381969b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 >> 172.21.15.105:0/2054082975 conn(0x7f76380fbd50 msgr2=0x7f7638104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 shutdown_connections 2024-03-26T17:35:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:54.859+0000 7f763fd1b640 1 -- 172.21.15.105:0/2054082975 wait complete. 2024-03-26T17:35:55.395 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 17:35:55 smithi105 systemd[1]: Started Ceph mds.d for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:35:56.034 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:55 smithi135 ceph-mon[35636]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:56.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:55 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2054082975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:56.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:56.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:56.035 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:56.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:55 smithi144 ceph-mon[36095]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:56.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:55 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2054082975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:56.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:56.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:56.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:55 smithi105 ceph-mon[32280]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:55 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2054082975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:56.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:56.864 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:35:56.890 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:56 smithi105 ceph-mon[32280]: Deploying daemon mds.e on smithi135 2024-03-26T17:35:56.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:56 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.105:6836/2045143293,v1:172.21.15.105:6837/2045143293] up:boot 2024-03-26T17:35:56.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:56 smithi105 ceph-mon[32280]: fsmap 4 up:standby 2024-03-26T17:35:56.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:56 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T17:35:56.958 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:56 smithi135 ceph-mon[35636]: Deploying daemon mds.e on smithi135 2024-03-26T17:35:56.958 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:56 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.105:6836/2045143293,v1:172.21.15.105:6837/2045143293] up:boot 2024-03-26T17:35:56.958 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:56 smithi135 ceph-mon[35636]: fsmap 4 up:standby 2024-03-26T17:35:56.958 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:56 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.978+0000 7f15aa26c640 1 -- 172.21.15.105:0/1537631189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 msgr2=0x7f15a4100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.978+0000 7f15aa26c640 1 --2- 172.21.15.105:0/1537631189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 0x7f15a4100c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1598009860 tx=0x7f159802bb70 comp rx=0 tx=0).stop 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 -- 172.21.15.105:0/1537631189 shutdown_connections 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 --2- 172.21.15.105:0/1537631189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a4102390 0x7f15a4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 --2- 172.21.15.105:0/1537631189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a41019d0 0x7f15a4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 --2- 172.21.15.105:0/1537631189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 0x7f15a4100c00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 -- 172.21.15.105:0/1537631189 >> 172.21.15.105:0/1537631189 conn(0x7f15a40fbd50 msgr2=0x7f15a40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 -- 172.21.15.105:0/1537631189 shutdown_connections 2024-03-26T17:35:56.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 -- 172.21.15.105:0/1537631189 wait complete. 2024-03-26T17:35:56.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 Processor -- start 2024-03-26T17:35:56.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 -- start start 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 0x7f15a40695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a41019d0 0x7f15a419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4102390 0x7f15a419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15a410c310 con 0x7f15a41007e0 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15a410c190 con 0x7f15a4102390 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.979+0000 7f15aa26c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15a410c490 con 0x7f15a41019d0 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 0x7f15a40695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 0x7f15a40695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50502/0 (socket says 172.21.15.105:50502) 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 -- 172.21.15.105:0/3236911454 learned_addr learned my addr 172.21.15.105:0/3236911454 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a8809640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4102390 0x7f15a419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 -- 172.21.15.105:0/3236911454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a41019d0 msgr2=0x7f15a419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a37fe640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a41019d0 0x7f15a419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:56.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a41019d0 0x7f15a419c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 -- 172.21.15.105:0/3236911454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4102390 msgr2=0x7f15a419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4102390 0x7f15a419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 -- 172.21.15.105:0/3236911454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a419f760 con 0x7f15a41007e0 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a3fff640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 0x7f15a40695f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1598005940 tx=0x7f15980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a17fa640 1 -- 172.21.15.105:0/3236911454 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598041020 con 0x7f15a41007e0 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a17fa640 1 -- 172.21.15.105:0/3236911454 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15980035f0 con 0x7f15a41007e0 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a419f9f0 con 0x7f15a41007e0 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15a17fa640 1 -- 172.21.15.105:0/3236911454 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598039040 con 0x7f15a41007e0 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.980+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a419fee0 con 0x7f15a41007e0 2024-03-26T17:35:56.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.982+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a4107b80 con 0x7f15a41007e0 2024-03-26T17:35:56.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.982+0000 7f15a17fa640 1 -- 172.21.15.105:0/3236911454 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f1598044020 con 0x7f15a41007e0 2024-03-26T17:35:56.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.983+0000 7f15a17fa640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f158c0a2f20 0x7f158c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:56.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.983+0000 7f15a37fe640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f158c0a2f20 0x7f158c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:56.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.983+0000 7f15a17fa640 1 -- 172.21.15.105:0/3236911454 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f15980f09a0 con 0x7f15a41007e0 2024-03-26T17:35:56.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.983+0000 7f15a37fe640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f158c0a2f20 0x7f158c0a53e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15a4069360 tx=0x7f1594009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:56.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:56.987+0000 7f15a17fa640 1 -- 172.21.15.105:0/3236911454 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f159804b020 con 0x7f15a41007e0 2024-03-26T17:35:57.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:56 smithi144 ceph-mon[36095]: Deploying daemon mds.e on smithi135 2024-03-26T17:35:57.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:56 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.105:6836/2045143293,v1:172.21.15.105:6837/2045143293] up:boot 2024-03-26T17:35:57.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:56 smithi144 ceph-mon[36095]: fsmap 4 up:standby 2024-03-26T17:35:57.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:56 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T17:35:57.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.226+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15a402d480 con 0x7f15a41007e0 2024-03-26T17:35:57.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.227+0000 7f15a17fa640 1 -- 172.21.15.105:0/3236911454 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3880 (secure 0 0 0) 0x7f15980bdc70 con 0x7f15a41007e0 2024-03-26T17:35:57.220 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:57.220 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-26T17:35:57.220 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 5 2024-03-26T17:35:57.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f158c0a2f20 msgr2=0x7f158c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:57.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f158c0a2f20 0x7f158c0a53e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15a4069360 tx=0x7f1594009260 comp rx=0 tx=0).stop 2024-03-26T17:35:57.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 msgr2=0x7f15a40695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:57.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 0x7f15a40695f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1598005940 tx=0x7f15980042f0 comp rx=0 tx=0).stop 2024-03-26T17:35:57.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 shutdown_connections 2024-03-26T17:35:57.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f158c0a2f20 0x7f158c0a53e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:57.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15a4102390 0x7f15a419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:57.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15a41019d0 0x7f15a419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:57.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 --2- 172.21.15.105:0/3236911454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a41007e0 0x7f15a40695f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:57.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 >> 172.21.15.105:0/3236911454 conn(0x7f15a40fbd50 msgr2=0x7f15a4104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:57.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 shutdown_connections 2024-03-26T17:35:57.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:57.229+0000 7f15aa26c640 1 -- 172.21.15.105:0/3236911454 wait complete. 2024-03-26T17:35:57.343 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 systemd[1]: Starting Ceph mds.e for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:35:57.666 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 podman[65068]: 2024-03-26 17:35:57.470860242 +0000 UTC m=+0.018012214 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:35:57.666 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 podman[65068]: 2024-03-26 17:35:57.568540001 +0000 UTC m=+0.115691941 container create 8a72d34a3e5ed7b4d139507d4c8ed461e8e8621867c750ef5a1c9398e070bf44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-26T17:35:57.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:57 smithi105 ceph-mon[32280]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:57.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:57 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3236911454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:58.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:57 smithi144 ceph-mon[36095]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:58.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:57 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3236911454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:58.166 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 podman[65068]: 2024-03-26 17:35:57.797486957 +0000 UTC m=+0.344638897 container init 8a72d34a3e5ed7b4d139507d4c8ed461e8e8621867c750ef5a1c9398e070bf44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-03-26T17:35:58.166 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 podman[65068]: 2024-03-26 17:35:57.800452958 +0000 UTC m=+0.347604901 container start 8a72d34a3e5ed7b4d139507d4c8ed461e8e8621867c750ef5a1c9398e070bf44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-26T17:35:58.167 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/576236726 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dad3ce1a40 con 0x55dad3cb4800 2024-03-26T17:35:58.167 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T17:35:58.167 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 bash[65068]: 8a72d34a3e5ed7b4d139507d4c8ed461e8e8621867c750ef5a1c9398e070bf44 2024-03-26T17:35:58.167 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:35:57 smithi135 systemd[1]: Started Ceph mds.e for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:35:58.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:57 smithi135 ceph-mon[35636]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:58.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:57 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3236911454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:35:59.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: Deploying daemon mds.f on smithi144 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/3121985659,v1:172.21.15.135:6835/3121985659] up:boot 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: fsmap 5 up:standby 2024-03-26T17:35:59.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:58 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:35:59.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: Deploying daemon mds.f on smithi144 2024-03-26T17:35:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/3121985659,v1:172.21.15.135:6835/3121985659] up:boot 2024-03-26T17:35:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: fsmap 5 up:standby 2024-03-26T17:35:59.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:58 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:35:59.234 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: Deploying daemon mds.f on smithi144 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/3121985659,v1:172.21.15.135:6835/3121985659] up:boot 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: fsmap 5 up:standby 2024-03-26T17:35:59.266 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:58 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:35:59.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.357+0000 7f158cd90640 1 -- 172.21.15.105:0/2726303601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1588107640 msgr2=0x7f1588109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:59.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.357+0000 7f158cd90640 1 --2- 172.21.15.105:0/2726303601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1588107640 0x7f1588109a90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1580009ca0 tx=0x7f158002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 -- 172.21.15.105:0/2726303601 shutdown_connections 2024-03-26T17:35:59.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 --2- 172.21.15.105:0/2726303601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1588107640 0x7f1588109a90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 --2- 172.21.15.105:0/2726303601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1588104d10 0x7f1588107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 --2- 172.21.15.105:0/2726303601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15881023e0 0x7f15881047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 -- 172.21.15.105:0/2726303601 >> 172.21.15.105:0/2726303601 conn(0x7f15880fbdf0 msgr2=0x7f15880fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:59.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 -- 172.21.15.105:0/2726303601 shutdown_connections 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 -- 172.21.15.105:0/2726303601 wait complete. 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 Processor -- start 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.358+0000 7f158cd90640 1 -- start start 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158cd90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15881023e0 0x7f1588197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158cd90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1588104d10 0x7f1588198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158cd90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1588107640 0x7f158819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158cd90640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f158810d490 con 0x7f15881023e0 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158cd90640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f158810d310 con 0x7f1588104d10 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158cd90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f158810d610 con 0x7f1588107640 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15881023e0 0x7f1588197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15881023e0 0x7f1588197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50518/0 (socket says 172.21.15.105:50518) 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 -- 172.21.15.105:0/3994556467 learned_addr learned my addr 172.21.15.105:0/3994556467 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f1586d9d640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1588107640 0x7f158819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:59.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 -- 172.21.15.105:0/3994556467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1588107640 msgr2=0x7f158819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1588107640 0x7f158819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 -- 172.21.15.105:0/3994556467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1588104d10 msgr2=0x7f1588198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1588104d10 0x7f1588198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 -- 172.21.15.105:0/3994556467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158819f6e0 con 0x7f15881023e0 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f1586d9d640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1588107640 0x7f158819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158659c640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15881023e0 0x7f1588197b30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f157800c920 tx=0x7f157800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f15777fe640 1 -- 172.21.15.105:0/3994556467 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578007c70 con 0x7f15881023e0 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f15777fe640 1 -- 172.21.15.105:0/3994556467 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1578007e10 con 0x7f15881023e0 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158819f9d0 con 0x7f15881023e0 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f15777fe640 1 -- 172.21.15.105:0/3994556467 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578018820 con 0x7f15881023e0 2024-03-26T17:35:59.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.359+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158819fff0 con 0x7f15881023e0 2024-03-26T17:35:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.360+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15880fd8e0 con 0x7f15881023e0 2024-03-26T17:35:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.362+0000 7f15777fe640 1 -- 172.21.15.105:0/3994556467 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f15780189c0 con 0x7f15881023e0 2024-03-26T17:35:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.362+0000 7f15777fe640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15700a2fa0 0x7f15700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:35:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.362+0000 7f1585d9b640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15700a2fa0 0x7f15700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:35:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.362+0000 7f15777fe640 1 -- 172.21.15.105:0/3994556467 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f15780cf0b0 con 0x7f15881023e0 2024-03-26T17:35:59.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.363+0000 7f1585d9b640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15700a2fa0 0x7f15700a5460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1588198fd0 tx=0x7f157c006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:35:59.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.367+0000 7f15777fe640 1 -- 172.21.15.105:0/3994556467 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1578014030 con 0x7f15881023e0 2024-03-26T17:35:59.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.580+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f158802d4f0 con 0x7f15881023e0 2024-03-26T17:35:59.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.580+0000 7f15777fe640 1 -- 172.21.15.105:0/3994556467 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4708 (secure 0 0 0) 0x7f157800de20 con 0x7f15881023e0 2024-03-26T17:35:59.573 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:35:59.574 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-26T17:35:59.574 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 6 2024-03-26T17:35:59.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15700a2fa0 msgr2=0x7f15700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15700a2fa0 0x7f15700a5460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1588198fd0 tx=0x7f157c006c90 comp rx=0 tx=0).stop 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15881023e0 msgr2=0x7f1588197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15881023e0 0x7f1588197b30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f157800c920 tx=0x7f157800cde0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 shutdown_connections 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15700a2fa0 0x7f15700a5460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1588107640 0x7f158819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1588104d10 0x7f1588198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 --2- 172.21.15.105:0/3994556467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15881023e0 0x7f1588197b30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 >> 172.21.15.105:0/3994556467 conn(0x7f15880fbdf0 msgr2=0x7f1588102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 shutdown_connections 2024-03-26T17:35:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:35:59.582+0000 7f158cd90640 1 -- 172.21.15.105:0/3994556467 wait complete. 2024-03-26T17:36:00.251 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:35:59 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3994556467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:00.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:35:59 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3994556467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:00.416 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:35:59 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3994556467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:00.599 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 17:36:00 smithi144 systemd[1]: Started Ceph mds.f for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:36:01.588 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/2030078073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81007e0 msgr2=0x7fa5e8100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/2030078073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81007e0 0x7fa5e8100c00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009900 tx=0x7fa5d802bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/2030078073 shutdown_connections 2024-03-26T17:36:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/2030078073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e8102390 0x7fa5e8108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/2030078073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e81019d0 0x7fa5e8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/2030078073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81007e0 0x7fa5e8100c00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/2030078073 >> 172.21.15.105:0/2030078073 conn(0x7fa5e80fbd50 msgr2=0x7fa5e80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:01.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/2030078073 shutdown_connections 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.708+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/2030078073 wait complete. 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5ed4f1640 1 Processor -- start 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5ed4f1640 1 -- start start 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5ed4f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e81007e0 0x7fa5e8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5ed4f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81019d0 0x7fa5e819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5ed4f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e8102390 0x7fa5e81a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5ed4f1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810c300 con 0x7fa5e81019d0 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5ed4f1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810c180 con 0x7fa5e81007e0 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5ed4f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e810c480 con 0x7fa5e8102390 2024-03-26T17:36:01.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81019d0 0x7fa5e819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81019d0 0x7fa5e819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50534/0 (socket says 172.21.15.105:50534) 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 -- 172.21.15.105:0/254645696 learned_addr learned my addr 172.21.15.105:0/254645696 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5e6ffd640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e81007e0 0x7fa5e8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5e77fe640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e8102390 0x7fa5e81a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 -- 172.21.15.105:0/254645696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e8102390 msgr2=0x7fa5e81a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e8102390 0x7fa5e81a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 -- 172.21.15.105:0/254645696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e81007e0 msgr2=0x7fa5e8069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e81007e0 0x7fa5e8069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 -- 172.21.15.105:0/254645696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e81a1a40 con 0x7fa5e81019d0 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.709+0000 7fa5dffff640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81019d0 0x7fa5e819ed00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e9f0 tx=0x7fa5d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:01.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.710+0000 7fa5e4ff9640 1 -- 172.21.15.105:0/254645696 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000cd80 con 0x7fa5e81019d0 2024-03-26T17:36:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.710+0000 7fa5e4ff9640 1 -- 172.21.15.105:0/254645696 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5d00054d0 con 0x7fa5e81019d0 2024-03-26T17:36:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.710+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e81a1cd0 con 0x7fa5e81019d0 2024-03-26T17:36:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.710+0000 7fa5e4ff9640 1 -- 172.21.15.105:0/254645696 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0010490 con 0x7fa5e81019d0 2024-03-26T17:36:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.710+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e81a21d0 con 0x7fa5e81019d0 2024-03-26T17:36:01.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.711+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e8107c20 con 0x7fa5e81019d0 2024-03-26T17:36:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.712+0000 7fa5e4ff9640 1 -- 172.21.15.105:0/254645696 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fa5d0005020 con 0x7fa5e81019d0 2024-03-26T17:36:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.712+0000 7fa5e4ff9640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5d40a2f50 0x7fa5d40a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.713+0000 7fa5e6ffd640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5d40a2f50 0x7fa5d40a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.713+0000 7fa5e4ff9640 1 -- 172.21.15.105:0/254645696 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa5d00cf310 con 0x7fa5e81019d0 2024-03-26T17:36:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.713+0000 7fa5e6ffd640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5d40a2f50 0x7fa5d40a5410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8005ab0 tx=0x7fa5d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.717+0000 7fa5e4ff9640 1 -- 172.21.15.105:0/254645696 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa5d009c5e0 con 0x7fa5e81019d0 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: Deploying daemon mds.c on smithi144 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6834/29089236,v1:172.21.15.144:6835/29089236] up:boot 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: fsmap 6 up:standby 2024-03-26T17:36:01.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:01 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T17:36:01.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:01.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:01.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:01.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:01.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: Deploying daemon mds.c on smithi144 2024-03-26T17:36:01.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6834/29089236,v1:172.21.15.144:6835/29089236] up:boot 2024-03-26T17:36:01.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: fsmap 6 up:standby 2024-03-26T17:36:01.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:01 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T17:36:01.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:01.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:01.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:01.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:01.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:01.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: Deploying daemon mds.c on smithi144 2024-03-26T17:36:01.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6834/29089236,v1:172.21.15.144:6835/29089236] up:boot 2024-03-26T17:36:01.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: fsmap 6 up:standby 2024-03-26T17:36:01.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:01 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T17:36:01.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.930+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5e802d440 con 0x7fa5e81019d0 2024-03-26T17:36:01.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.931+0000 7fa5e4ff9640 1 -- 172.21.15.105:0/254645696 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fa5d00bc0d0 con 0x7fa5e81019d0 2024-03-26T17:36:01.924 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:01.924 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-26T17:36:01.924 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 7 2024-03-26T17:36:01.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5d40a2f50 msgr2=0x7fa5d40a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:01.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5d40a2f50 0x7fa5d40a5410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8005ab0 tx=0x7fa5d8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81019d0 msgr2=0x7fa5e819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81019d0 0x7fa5e819ed00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000e9f0 tx=0x7fa5d000edf0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 shutdown_connections 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5d40a2f50 0x7fa5d40a5410 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e8102390 0x7fa5e81a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e81019d0 0x7fa5e819ed00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 --2- 172.21.15.105:0/254645696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5e81007e0 0x7fa5e8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 >> 172.21.15.105:0/254645696 conn(0x7fa5e80fbd50 msgr2=0x7fa5e8104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 shutdown_connections 2024-03-26T17:36:01.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:01.933+0000 7fa5ed4f1640 1 -- 172.21.15.105:0/254645696 wait complete. 2024-03-26T17:36:02.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:02 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/254645696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:02.850 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:02 smithi144 systemd[1]: Starting Ceph mds.c for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:36:02.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:02 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/254645696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:02.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:02 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/254645696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:03.152 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:02 smithi144 podman[66741]: 2024-03-26 17:36:02.807298876 +0000 UTC m=+0.018148698 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:36:03.153 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:02 smithi144 podman[66741]: 2024-03-26 17:36:02.915298496 +0000 UTC m=+0.126148359 container create 04c50fd54e37fc25b7f126f06859bee609c1d0704835665f871e1ecbaf2f91f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux ) 2024-03-26T17:36:03.153 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:03 smithi144 podman[66741]: 2024-03-26 17:36:03.134417211 +0000 UTC m=+0.345267031 container init 04c50fd54e37fc25b7f126f06859bee609c1d0704835665f871e1ecbaf2f91f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T17:36:03.153 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:03 smithi144 podman[66741]: 2024-03-26 17:36:03.137086982 +0000 UTC m=+0.347936803 container start 04c50fd54e37fc25b7f126f06859bee609c1d0704835665f871e1ecbaf2f91f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T17:36:03.572 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-mds[66760]: -- 172.21.15.144:0/3981621234 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7343b9a40 con 0x55f73438c800 2024-03-26T17:36:03.572 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c[66756]: starting mds.c at 2024-03-26T17:36:03.573 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:03 smithi144 bash[66741]: 04c50fd54e37fc25b7f126f06859bee609c1d0704835665f871e1ecbaf2f91f0 2024-03-26T17:36:03.573 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 17:36:03 smithi144 systemd[1]: Started Ceph mds.c for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:36:03.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-mon[36095]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:03.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:03.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:03.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:03 smithi105 ceph-mon[32280]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:03.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:03 smithi135 ceph-mon[35636]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:03.937 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 -- 172.21.15.105:0/2317474228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd654102390 msgr2=0x7fd654108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 --2- 172.21.15.105:0/2317474228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd654102390 0x7fd654108c20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009ca0 tx=0x7fd64c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 -- 172.21.15.105:0/2317474228 shutdown_connections 2024-03-26T17:36:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 --2- 172.21.15.105:0/2317474228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd654102390 0x7fd654108c20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 --2- 172.21.15.105:0/2317474228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6541019d0 0x7fd654101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 --2- 172.21.15.105:0/2317474228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6541007e0 0x7fd654100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 -- 172.21.15.105:0/2317474228 >> 172.21.15.105:0/2317474228 conn(0x7fd6540fbd50 msgr2=0x7fd6540fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 -- 172.21.15.105:0/2317474228 shutdown_connections 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.057+0000 7fd65b948640 1 -- 172.21.15.105:0/2317474228 wait complete. 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd65b948640 1 Processor -- start 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd65b948640 1 -- start start 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd65b948640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6541007e0 0x7fd6540695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd65b948640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6541019d0 0x7fd65419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd65b948640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd654102390 0x7fd65419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd65b948640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd65410c310 con 0x7fd6541007e0 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd65b948640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd65410c190 con 0x7fd6541019d0 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd65b948640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd65410c490 con 0x7fd654102390 2024-03-26T17:36:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd6596e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6541007e0 0x7fd6540695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd6596e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6541007e0 0x7fd6540695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50548/0 (socket says 172.21.15.105:50548) 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd6596e4640 1 -- 172.21.15.105:0/2990469900 learned_addr learned my addr 172.21.15.105:0/2990469900 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd658ee3640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6541019d0 0x7fd65419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd659ee5640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd654102390 0x7fd65419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd6596e4640 1 -- 172.21.15.105:0/2990469900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd654102390 msgr2=0x7fd65419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd6596e4640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd654102390 0x7fd65419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd6596e4640 1 -- 172.21.15.105:0/2990469900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6541019d0 msgr2=0x7fd65419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd6596e4640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6541019d0 0x7fd65419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.058+0000 7fd6596e4640 1 -- 172.21.15.105:0/2990469900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65419f6d0 con 0x7fd6541007e0 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.059+0000 7fd6596e4640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6541007e0 0x7fd6540695f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd65000d950 tx=0x7fd65000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.059+0000 7fd64a7fc640 1 -- 172.21.15.105:0/2990469900 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650017070 con 0x7fd6541007e0 2024-03-26T17:36:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.059+0000 7fd64a7fc640 1 -- 172.21.15.105:0/2990469900 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd650009c30 con 0x7fd6541007e0 2024-03-26T17:36:04.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.059+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65419f9c0 con 0x7fd6541007e0 2024-03-26T17:36:04.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.059+0000 7fd64a7fc640 1 -- 172.21.15.105:0/2990469900 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65000f620 con 0x7fd6541007e0 2024-03-26T17:36:04.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.059+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65419fec0 con 0x7fd6541007e0 2024-03-26T17:36:04.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.060+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd654107b80 con 0x7fd6541007e0 2024-03-26T17:36:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.061+0000 7fd64a7fc640 1 -- 172.21.15.105:0/2990469900 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fd65001a070 con 0x7fd6541007e0 2024-03-26T17:36:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.061+0000 7fd64a7fc640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6400a2f20 0x7fd6400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.061+0000 7fd658ee3640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6400a2f20 0x7fd6400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.062+0000 7fd64a7fc640 1 -- 172.21.15.105:0/2990469900 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd6500ce9a0 con 0x7fd6541007e0 2024-03-26T17:36:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.062+0000 7fd658ee3640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6400a2f20 0x7fd6400a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd654069360 tx=0x7fd64400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:04.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.066+0000 7fd64a7fc640 1 -- 172.21.15.105:0/2990469900 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd650009de0 con 0x7fd6541007e0 2024-03-26T17:36:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.280+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd65402d240 con 0x7fd6541007e0 2024-03-26T17:36:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.281+0000 7fd64a7fc640 1 -- 172.21.15.105:0/2990469900 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7fd65009bc70 con 0x7fd6541007e0 2024-03-26T17:36:04.274 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:04.274 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-26T17:36:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 8 2024-03-26T17:36:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6400a2f20 msgr2=0x7fd6400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6400a2f20 0x7fd6400a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd654069360 tx=0x7fd64400b390 comp rx=0 tx=0).stop 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6541007e0 msgr2=0x7fd6540695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6541007e0 0x7fd6540695f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd65000d950 tx=0x7fd65000de10 comp rx=0 tx=0).stop 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 shutdown_connections 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6400a2f20 0x7fd6400a53e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd654102390 0x7fd65419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6541019d0 0x7fd65419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 --2- 172.21.15.105:0/2990469900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6541007e0 0x7fd6540695f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 >> 172.21.15.105:0/2990469900 conn(0x7fd6540fbd50 msgr2=0x7fd654104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 shutdown_connections 2024-03-26T17:36:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:04.283+0000 7fd65b948640 1 -- 172.21.15.105:0/2990469900 wait complete. 2024-03-26T17:36:04.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:04 smithi144 ceph-mon[36095]: Deploying daemon mds.h on smithi135 2024-03-26T17:36:04.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:04 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] up:boot 2024-03-26T17:36:04.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:04 smithi144 ceph-mon[36095]: fsmap 7 up:standby 2024-03-26T17:36:04.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:04 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T17:36:04.850 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:04 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2990469900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:04 smithi105 ceph-mon[32280]: Deploying daemon mds.h on smithi135 2024-03-26T17:36:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:04 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] up:boot 2024-03-26T17:36:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:04 smithi105 ceph-mon[32280]: fsmap 7 up:standby 2024-03-26T17:36:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T17:36:04.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:04 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2990469900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:04.907 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:04 smithi135 ceph-mon[35636]: Deploying daemon mds.h on smithi135 2024-03-26T17:36:04.907 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:04 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] up:boot 2024-03-26T17:36:04.907 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:04 smithi135 ceph-mon[35636]: fsmap 7 up:standby 2024-03-26T17:36:04.907 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:04 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T17:36:04.907 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:04 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2990469900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:05.579 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 systemd[1]: Starting Ceph mds.h for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:36:05.830 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:05 smithi135 ceph-mon[35636]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:05.830 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 podman[65371]: 2024-03-26 17:36:05.592241623 +0000 UTC m=+0.105684073 container create 6677b0ca0560f57293849320bd45326263ae223b6c4913a895bfcb3fdf532dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-26T17:36:05.830 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 podman[65371]: 2024-03-26 17:36:05.504910113 +0000 UTC m=+0.018352571 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:36:05.830 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 podman[65371]: 2024-03-26 17:36:05.837405578 +0000 UTC m=+0.350848028 container init 6677b0ca0560f57293849320bd45326263ae223b6c4913a895bfcb3fdf532dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T17:36:05.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:05 smithi144 ceph-mon[36095]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:05.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:05 smithi105 ceph-mon[32280]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:06.167 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 podman[65371]: 2024-03-26 17:36:05.84041928 +0000 UTC m=+0.353861730 container start 6677b0ca0560f57293849320bd45326263ae223b6c4913a895bfcb3fdf532dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-03-26T17:36:06.167 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 ceph-mds[65390]: -- 172.21.15.135:0/2857000187 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ac6f9da40 con 0x561ac6f70800 2024-03-26T17:36:06.167 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h[65386]: starting mds.h at 2024-03-26T17:36:06.167 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 bash[65371]: 6677b0ca0560f57293849320bd45326263ae223b6c4913a895bfcb3fdf532dfe 2024-03-26T17:36:06.168 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 17:36:05 smithi135 systemd[1]: Started Ceph mds.h for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:36:06.288 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 -- 172.21.15.105:0/1870891740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 msgr2=0x7f92fc1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 --2- 172.21.15.105:0/1870891740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 0x7f92fc1070e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009900 tx=0x7f92f002bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 -- 172.21.15.105:0/1870891740 shutdown_connections 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 --2- 172.21.15.105:0/1870891740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92fc107620 0x7f92fc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 --2- 172.21.15.105:0/1870891740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 0x7f92fc1070e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 --2- 172.21.15.105:0/1870891740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92fc1023c0 0x7f92fc1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 -- 172.21.15.105:0/1870891740 >> 172.21.15.105:0/1870891740 conn(0x7f92fc0fbd50 msgr2=0x7f92fc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 -- 172.21.15.105:0/1870891740 shutdown_connections 2024-03-26T17:36:06.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 -- 172.21.15.105:0/1870891740 wait complete. 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 Processor -- start 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.408+0000 7f930279e640 1 -- start start 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f930279e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92fc1023c0 0x7f92fc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f930279e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 0x7f92fc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f930279e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92fc107620 0x7f92fc19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f930279e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10d400 con 0x7f92fc104cf0 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f930279e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10d280 con 0x7f92fc107620 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f930279e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc10d580 con 0x7f92fc1023c0 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 0x7f92fc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 0x7f92fc1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37040/0 (socket says 172.21.15.105:37040) 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 -- 172.21.15.105:0/1515026269 learned_addr learned my addr 172.21.15.105:0/1515026269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fbfff640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92fc1023c0 0x7f92fc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:06.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f9300d3b640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92fc107620 0x7f92fc19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 -- 172.21.15.105:0/1515026269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92fc1023c0 msgr2=0x7f92fc197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92fc1023c0 0x7f92fc197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 -- 172.21.15.105:0/1515026269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92fc107620 msgr2=0x7f92fc19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92fc107620 0x7f92fc19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 -- 172.21.15.105:0/1515026269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc19f740 con 0x7f92fc104cf0 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92fb7fe640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 0x7f92fc1980b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92f97fa640 1 -- 172.21.15.105:0/1515026269 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0043020 con 0x7f92fc104cf0 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92f97fa640 1 -- 172.21.15.105:0/1515026269 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92f0004440 con 0x7f92fc104cf0 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f92f97fa640 1 -- 172.21.15.105:0/1515026269 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0003c90 con 0x7f92fc104cf0 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc19f9d0 con 0x7f92fc104cf0 2024-03-26T17:36:06.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.409+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc19ff70 con 0x7f92fc104cf0 2024-03-26T17:36:06.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.410+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc0fd840 con 0x7f92fc104cf0 2024-03-26T17:36:06.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.411+0000 7f92f97fa640 1 -- 172.21.15.105:0/1515026269 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f92f0046020 con 0x7f92fc104cf0 2024-03-26T17:36:06.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.412+0000 7f92f97fa640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92e40a2fa0 0x7f92e40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:06.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.412+0000 7f92fbfff640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92e40a2fa0 0x7f92e40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:06.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.412+0000 7f92f97fa640 1 -- 172.21.15.105:0/1515026269 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f92f00f1190 con 0x7f92fc104cf0 2024-03-26T17:36:06.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.412+0000 7f92fbfff640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92e40a2fa0 0x7f92e40a5460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009bd0 tx=0x7f92ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:06.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.416+0000 7f92f97fa640 1 -- 172.21.15.105:0/1515026269 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f92f004d020 con 0x7f92fc104cf0 2024-03-26T17:36:06.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.632+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92fc02d490 con 0x7f92fc104cf0 2024-03-26T17:36:06.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.633+0000 7f92f97fa640 1 -- 172.21.15.105:0/1515026269 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f92f00be460 con 0x7f92fc104cf0 2024-03-26T17:36:06.626 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:06.626 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-26T17:36:06.627 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92e40a2fa0 msgr2=0x7f92e40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92e40a2fa0 0x7f92e40a5460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009bd0 tx=0x7f92ec005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 msgr2=0x7f92fc1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 0x7f92fc1980b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0004040 comp rx=0 tx=0).stop 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 shutdown_connections 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92e40a2fa0 0x7f92e40a5460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92fc107620 0x7f92fc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc104cf0 0x7f92fc1980b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 --2- 172.21.15.105:0/1515026269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92fc1023c0 0x7f92fc197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 >> 172.21.15.105:0/1515026269 conn(0x7f92fc0fbd50 msgr2=0x7f92fc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 shutdown_connections 2024-03-26T17:36:06.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:06.635+0000 7f930279e640 1 -- 172.21.15.105:0/1515026269 wait complete. 2024-03-26T17:36:07.292 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.293 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.293 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.293 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:07.293 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:07.293 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:07.294 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: Deploying daemon mds.i on smithi144 2024-03-26T17:36:07.294 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:07.294 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] up:boot 2024-03-26T17:36:07.294 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: fsmap 8 up:standby 2024-03-26T17:36:07.294 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T17:36:07.294 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:06 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1515026269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:07.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:07.394 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:07.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:07.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: Deploying daemon mds.i on smithi144 2024-03-26T17:36:07.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:07.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] up:boot 2024-03-26T17:36:07.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: fsmap 8 up:standby 2024-03-26T17:36:07.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T17:36:07.395 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:06 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1515026269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:07.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.417 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: Deploying daemon mds.i on smithi144 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] up:boot 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: fsmap 8 up:standby 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T17:36:07.418 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:06 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1515026269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:08.277 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 systemd[1]: Starting Ceph mds.i for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:36:08.548 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 podman[67060]: 2024-03-26 17:36:08.186405921 +0000 UTC m=+0.017662799 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:36:08.548 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 podman[67060]: 2024-03-26 17:36:08.302899946 +0000 UTC m=+0.134156823 container create fe945a7b850fe4a87f2948605956b4ef211c9d14ec9ec0f93026ab2773da923f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T17:36:08.549 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 podman[67060]: 2024-03-26 17:36:08.530204419 +0000 UTC m=+0.361461298 container init fe945a7b850fe4a87f2948605956b4ef211c9d14ec9ec0f93026ab2773da923f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T17:36:08.549 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 podman[67060]: 2024-03-26 17:36:08.532786131 +0000 UTC m=+0.364043012 container start fe945a7b850fe4a87f2948605956b4ef211c9d14ec9ec0f93026ab2773da923f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T17:36:08.643 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:08.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 -- 172.21.15.105:0/841985983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f4102390 msgr2=0x7f82f4108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:08.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/841985983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f4102390 0x7f82f4108c20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009ca0 tx=0x7f82ec02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 -- 172.21.15.105:0/841985983 shutdown_connections 2024-03-26T17:36:08.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/841985983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f4102390 0x7f82f4108c20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/841985983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82f41019d0 0x7f82f4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/841985983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82f41007e0 0x7f82f4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 -- 172.21.15.105:0/841985983 >> 172.21.15.105:0/841985983 conn(0x7f82f40fbd50 msgr2=0x7f82f40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 -- 172.21.15.105:0/841985983 shutdown_connections 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 -- 172.21.15.105:0/841985983 wait complete. 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 Processor -- start 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.767+0000 7f82f8d3b640 1 -- start start 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f8d3b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82f41007e0 0x7f82f40695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f8d3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f41019d0 0x7f82f419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f8d3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82f4102390 0x7f82f419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f8d3b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82f410c310 con 0x7f82f41019d0 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f8d3b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82f410c190 con 0x7f82f41007e0 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f8d3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82f410c490 con 0x7f82f4102390 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f41019d0 0x7f82f419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:08.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f41019d0 0x7f82f419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37068/0 (socket says 172.21.15.105:37068) 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 -- 172.21.15.105:0/1664315513 learned_addr learned my addr 172.21.15.105:0/1664315513 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f259c640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82f41007e0 0x7f82f40695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f2d9d640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82f4102390 0x7f82f419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 -- 172.21.15.105:0/1664315513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82f4102390 msgr2=0x7f82f419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82f4102390 0x7f82f419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 -- 172.21.15.105:0/1664315513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82f41007e0 msgr2=0x7f82f40695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82f41007e0 0x7f82f40695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 -- 172.21.15.105:0/1664315513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f419f6f0 con 0x7f82f41019d0 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f259c640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82f41007e0 0x7f82f40695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82ebfff640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f41019d0 0x7f82f419c7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00e9f0 tx=0x7f82dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82eb7fe640 1 -- 172.21.15.105:0/1664315513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00cd80 con 0x7f82f41019d0 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82eb7fe640 1 -- 172.21.15.105:0/1664315513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82dc0054d0 con 0x7f82f41019d0 2024-03-26T17:36:08.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f419f9e0 con 0x7f82f41019d0 2024-03-26T17:36:08.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82eb7fe640 1 -- 172.21.15.105:0/1664315513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc010490 con 0x7f82f41019d0 2024-03-26T17:36:08.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.768+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f419fee0 con 0x7f82f41019d0 2024-03-26T17:36:08.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.770+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f4107b00 con 0x7f82f41019d0 2024-03-26T17:36:08.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.771+0000 7f82eb7fe640 1 -- 172.21.15.105:0/1664315513 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f82dc005020 con 0x7f82f41019d0 2024-03-26T17:36:08.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.771+0000 7f82eb7fe640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82e00a2f20 0x7f82e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:08.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.772+0000 7f82f259c640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82e00a2f20 0x7f82e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:08.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.772+0000 7f82eb7fe640 1 -- 172.21.15.105:0/1664315513 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f82dc0cf290 con 0x7f82f41019d0 2024-03-26T17:36:08.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.772+0000 7f82f259c640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82e00a2f20 0x7f82e00a53e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82f4101830 tx=0x7f82e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:08.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.776+0000 7f82eb7fe640 1 -- 172.21.15.105:0/1664315513 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f82dc09c560 con 0x7f82f41019d0 2024-03-26T17:36:08.849 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 ceph-mds[67079]: -- 172.21.15.144:0/608178780 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558636507a40 con 0x5586364db000 2024-03-26T17:36:08.849 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i[67075]: starting mds.i at 2024-03-26T17:36:08.849 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 bash[67060]: fe945a7b850fe4a87f2948605956b4ef211c9d14ec9ec0f93026ab2773da923f 2024-03-26T17:36:08.849 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 17:36:08 smithi144 systemd[1]: Started Ceph mds.i for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:36:08.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.994+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82f402d420 con 0x7f82f41019d0 2024-03-26T17:36:08.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.994+0000 7f82eb7fe640 1 -- 172.21.15.105:0/1664315513 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f82dc09be60 con 0x7f82f41019d0 2024-03-26T17:36:08.988 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:08.988 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-26T17:36:08.988 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 10 2024-03-26T17:36:08.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.996+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82e00a2f20 msgr2=0x7f82e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82e00a2f20 0x7f82e00a53e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82f4101830 tx=0x7f82e4005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f41019d0 msgr2=0x7f82f419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f41019d0 0x7f82f419c7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00e9f0 tx=0x7f82dc00edf0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 shutdown_connections 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82e00a2f20 0x7f82e00a53e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82f4102390 0x7f82f419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82f41019d0 0x7f82f419c7c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 --2- 172.21.15.105:0/1664315513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82f41007e0 0x7f82f40695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 >> 172.21.15.105:0/1664315513 conn(0x7f82f40fbd50 msgr2=0x7f82f4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 shutdown_connections 2024-03-26T17:36:08.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:08.997+0000 7f82f8d3b640 1 -- 172.21.15.105:0/1664315513 wait complete. 2024-03-26T17:36:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] up:boot 2024-03-26T17:36:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: fsmap 9 up:standby 2024-03-26T17:36:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T17:36:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:09.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:09.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:09.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:09.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: Deploying daemon mds.g on smithi105 2024-03-26T17:36:09.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:09 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1664315513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:09.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:09.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] up:boot 2024-03-26T17:36:09.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: fsmap 9 up:standby 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: Deploying daemon mds.g on smithi105 2024-03-26T17:36:09.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:09 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1664315513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:10.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:10.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] up:boot 2024-03-26T17:36:10.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: fsmap 9 up:standby 2024-03-26T17:36:10.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T17:36:10.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:10.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:10.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:10.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:10.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:10.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:10.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: Deploying daemon mds.g on smithi105 2024-03-26T17:36:10.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:09 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1664315513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:11.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:11.028 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:10 smithi105 systemd[1]: Starting Ceph mds.g for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:36:11.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 -- 172.21.15.105:0/638158611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff728100800 msgr2=0x7ff728100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:11.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/638158611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff728100800 0x7ff728100c20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009900 tx=0x7ff71c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 -- 172.21.15.105:0/638158611 shutdown_connections 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/638158611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7281023b0 0x7ff728108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/638158611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7281019f0 0x7ff728101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/638158611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff728100800 0x7ff728100c20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 -- 172.21.15.105:0/638158611 >> 172.21.15.105:0/638158611 conn(0x7ff7280fbd70 msgr2=0x7ff7280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 -- 172.21.15.105:0/638158611 shutdown_connections 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.115+0000 7ff72ee7e640 1 -- 172.21.15.105:0/638158611 wait complete. 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72ee7e640 1 Processor -- start 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72ee7e640 1 -- start start 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72ee7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100800 0x7ff728197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72ee7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7281019f0 0x7ff728198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72ee7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7281023b0 0x7ff72819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72ee7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff72810d450 con 0x7ff7281023b0 2024-03-26T17:36:11.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72ee7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff72810d2d0 con 0x7ff728100800 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72ee7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff72810d5d0 con 0x7ff7281019f0 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7281023b0 0x7ff72819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7281023b0 0x7ff72819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37084/0 (socket says 172.21.15.105:37084) 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 -- 172.21.15.105:0/114505305 learned_addr learned my addr 172.21.15.105:0/114505305 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72cc1a640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100800 0x7ff728197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff727fff640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7281019f0 0x7ff728198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 -- 172.21.15.105:0/114505305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7281019f0 msgr2=0x7ff728198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7281019f0 0x7ff728198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 -- 172.21.15.105:0/114505305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100800 msgr2=0x7ff728197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100800 0x7ff728197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 -- 172.21.15.105:0/114505305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72819f770 con 0x7ff7281023b0 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72cc1a640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100800 0x7ff728197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:36:11.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff72d41b640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7281023b0 0x7ff72819f100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff72000eaa0 tx=0x7ff72000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:11.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.116+0000 7ff725ffb640 1 -- 172.21.15.105:0/114505305 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72000cdb0 con 0x7ff7281023b0 2024-03-26T17:36:11.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.117+0000 7ff725ffb640 1 -- 172.21.15.105:0/114505305 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff720005020 con 0x7ff7281023b0 2024-03-26T17:36:11.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.117+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72819fa60 con 0x7ff7281023b0 2024-03-26T17:36:11.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.117+0000 7ff725ffb640 1 -- 172.21.15.105:0/114505305 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7200116d0 con 0x7ff7281023b0 2024-03-26T17:36:11.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.117+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72819ffb0 con 0x7ff7281023b0 2024-03-26T17:36:11.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.118+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff728107bc0 con 0x7ff7281023b0 2024-03-26T17:36:11.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.118+0000 7ff725ffb640 1 -- 172.21.15.105:0/114505305 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7ff7200051d0 con 0x7ff7281023b0 2024-03-26T17:36:11.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.119+0000 7ff725ffb640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7180a2f20 0x7ff7180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:11.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.119+0000 7ff72cc1a640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7180a2f20 0x7ff7180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:11.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.119+0000 7ff725ffb640 1 -- 172.21.15.105:0/114505305 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff7200cf180 con 0x7ff7281023b0 2024-03-26T17:36:11.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.120+0000 7ff72cc1a640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7180a2f20 0x7ff7180a53e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff71c006010 tx=0x7ff71c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:11.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.123+0000 7ff725ffb640 1 -- 172.21.15.105:0/114505305 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff720016030 con 0x7ff7281023b0 2024-03-26T17:36:11.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.339+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff72802d3d0 con 0x7ff7281023b0 2024-03-26T17:36:11.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.340+0000 7ff725ffb640 1 -- 172.21.15.105:0/114505305 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7ff72009c450 con 0x7ff7281023b0 2024-03-26T17:36:11.333 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:11.333 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-26T17:36:11.333 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 10 2024-03-26T17:36:11.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7180a2f20 msgr2=0x7ff7180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:11.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7180a2f20 0x7ff7180a53e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff71c006010 tx=0x7ff71c02c7e0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7281023b0 msgr2=0x7ff72819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:11.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7281023b0 0x7ff72819f100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff72000eaa0 tx=0x7ff72000ef60 comp rx=0 tx=0).stop 2024-03-26T17:36:11.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 shutdown_connections 2024-03-26T17:36:11.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7180a2f20 0x7ff7180a53e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7281023b0 0x7ff72819f100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7281019f0 0x7ff728198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 --2- 172.21.15.105:0/114505305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100800 0x7ff728197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:11.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 >> 172.21.15.105:0/114505305 conn(0x7ff7280fbd70 msgr2=0x7ff728104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:11.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 shutdown_connections 2024-03-26T17:36:11.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:11.342+0000 7ff72ee7e640 1 -- 172.21.15.105:0/114505305 wait complete. 2024-03-26T17:36:11.345 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:11 smithi105 podman[73186]: 2024-03-26 17:36:10.996387876 +0000 UTC m=+0.017044209 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:36:11.345 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:11 smithi105 podman[73186]: 2024-03-26 17:36:11.12857939 +0000 UTC m=+0.149235710 container create ce2f700b9024bc43e1194e8bd119c9c794fe8ef12bf6ec9b20183cbab808c685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-26T17:36:11.625 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:11 smithi105 podman[73186]: 2024-03-26 17:36:11.375286048 +0000 UTC m=+0.395942368 container init ce2f700b9024bc43e1194e8bd119c9c794fe8ef12bf6ec9b20183cbab808c685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T17:36:11.625 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:11 smithi105 podman[73186]: 2024-03-26 17:36:11.377888284 +0000 UTC m=+0.398544607 container start ce2f700b9024bc43e1194e8bd119c9c794fe8ef12bf6ec9b20183cbab808c685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T17:36:11.625 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mds[73246]: -- 172.21.15.105:0/3921454941 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9de9c3a40 con 0x55e9de996800 2024-03-26T17:36:11.625 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g[73240]: starting mds.g at 2024-03-26T17:36:11.625 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:11 smithi105 bash[73186]: ce2f700b9024bc43e1194e8bd119c9c794fe8ef12bf6ec9b20183cbab808c685 2024-03-26T17:36:11.625 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 17:36:11 smithi105 systemd[1]: Started Ceph mds.g for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:36:11.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mon[32280]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:11.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/114505305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:11.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:11.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:11.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:11.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:11.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:11.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:11 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:11 smithi135 ceph-mon[35636]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:11 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/114505305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:11.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:11 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:11 smithi144 ceph-mon[36095]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:11 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/114505305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:12.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:11 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:12.695 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:12 smithi105 ceph-mon[32280]: Deploying daemon mds.a on smithi105 2024-03-26T17:36:12.695 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:12 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.105:6838/3053296748,v1:172.21.15.105:6839/3053296748] up:boot 2024-03-26T17:36:12.695 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:12 smithi105 ceph-mon[32280]: fsmap 10 up:standby 2024-03-26T17:36:12.695 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:12 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T17:36:12.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:12 smithi135 ceph-mon[35636]: Deploying daemon mds.a on smithi105 2024-03-26T17:36:12.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:12 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.105:6838/3053296748,v1:172.21.15.105:6839/3053296748] up:boot 2024-03-26T17:36:12.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:12 smithi135 ceph-mon[35636]: fsmap 10 up:standby 2024-03-26T17:36:12.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:12 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T17:36:13.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:12 smithi144 ceph-mon[36095]: Deploying daemon mds.a on smithi105 2024-03-26T17:36:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:12 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.105:6838/3053296748,v1:172.21.15.105:6839/3053296748] up:boot 2024-03-26T17:36:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:12 smithi144 ceph-mon[36095]: fsmap 10 up:standby 2024-03-26T17:36:13.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:12 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T17:36:13.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:13.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/902538008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18103520 msgr2=0x7f3a18109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:13.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/902538008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18103520 0x7f3a18109db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009cd0 tx=0x7f3a0802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/902538008 shutdown_connections 2024-03-26T17:36:13.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/902538008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18103520 0x7f3a18109db0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/902538008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a18102b60 0x7f3a18102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/902538008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a180ffb90 0x7f3a180fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/902538008 >> 172.21.15.105:0/902538008 conn(0x7f3a180fb4c0 msgr2=0x7f3a180fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/902538008 shutdown_connections 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/902538008 wait complete. 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 Processor -- start 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.614+0000 7f3a1d6a0640 1 -- start start 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a1d6a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a180ffb90 0x7f3a18197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a1d6a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18102b60 0x7f3a18198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a1d6a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a18103520 0x7f3a1819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a1d6a0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1810d420 con 0x7f3a18102b60 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a1d6a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1810d2a0 con 0x7f3a180ffb90 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a1d6a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1810d5a0 con 0x7f3a18103520 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18102b60 0x7f3a18198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:13.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18102b60 0x7f3a18198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37116/0 (socket says 172.21.15.105:37116) 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 -- 172.21.15.105:0/1181281079 learned_addr learned my addr 172.21.15.105:0/1181281079 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a177fe640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a18103520 0x7f3a1819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a16ffd640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a180ffb90 0x7f3a18197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 -- 172.21.15.105:0/1181281079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a18103520 msgr2=0x7f3a1819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a18103520 0x7f3a1819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 -- 172.21.15.105:0/1181281079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a180ffb90 msgr2=0x7f3a18197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a180ffb90 0x7f3a18197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 -- 172.21.15.105:0/1181281079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1819f720 con 0x7f3a18102b60 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a16ffd640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a180ffb90 0x7f3a18197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a167fc640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18102b60 0x7f3a18198040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000c970 tx=0x7f3a0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.615+0000 7f3a0ffff640 1 -- 172.21.15.105:0/1181281079 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00007bb0 con 0x7f3a18102b60 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.616+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1819fa10 con 0x7f3a18102b60 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.616+0000 7f3a0ffff640 1 -- 172.21.15.105:0/1181281079 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a00007d50 con 0x7f3a18102b60 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.616+0000 7f3a0ffff640 1 -- 172.21.15.105:0/1181281079 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0000f450 con 0x7f3a18102b60 2024-03-26T17:36:13.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.616+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1819ff60 con 0x7f3a18102b60 2024-03-26T17:36:13.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.617+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a18108d10 con 0x7f3a18102b60 2024-03-26T17:36:13.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.618+0000 7f3a0ffff640 1 -- 172.21.15.105:0/1181281079 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f3a00004f20 con 0x7f3a18102b60 2024-03-26T17:36:13.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.618+0000 7f3a0ffff640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3a040a2f20 0x7f3a040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:13.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.619+0000 7f3a0ffff640 1 -- 172.21.15.105:0/1181281079 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3a000cecb0 con 0x7f3a18102b60 2024-03-26T17:36:13.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.619+0000 7f3a16ffd640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3a040a2f20 0x7f3a040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:13.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.619+0000 7f3a16ffd640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3a040a2f20 0x7f3a040a53e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a18068820 tx=0x7f3a1000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:13.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.623+0000 7f3a0ffff640 1 -- 172.21.15.105:0/1181281079 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3a00014030 con 0x7f3a18102b60 2024-03-26T17:36:13.632 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:13 smithi105 systemd[1]: Starting Ceph mds.a for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:36:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.850+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a1802d410 con 0x7f3a18102b60 2024-03-26T17:36:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.850+0000 7f3a0ffff640 1 -- 172.21.15.105:0/1181281079 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f3a0009bf80 con 0x7f3a18102b60 2024-03-26T17:36:13.843 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:13.843 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-26T17:36:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 11 2024-03-26T17:36:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3a040a2f20 msgr2=0x7f3a040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3a040a2f20 0x7f3a040a53e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a18068820 tx=0x7f3a1000a560 comp rx=0 tx=0).stop 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18102b60 msgr2=0x7f3a18198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18102b60 0x7f3a18198040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000c970 tx=0x7f3a0000ce30 comp rx=0 tx=0).stop 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 shutdown_connections 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3a040a2f20 0x7f3a040a53e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a18103520 0x7f3a1819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a18102b60 0x7f3a18198040 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 --2- 172.21.15.105:0/1181281079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a180ffb90 0x7f3a18197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.852+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 >> 172.21.15.105:0/1181281079 conn(0x7f3a180fb4c0 msgr2=0x7f3a18105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.853+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 shutdown_connections 2024-03-26T17:36:13.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:13.853+0000 7f3a1d6a0640 1 -- 172.21.15.105:0/1181281079 wait complete. 2024-03-26T17:36:13.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:13 smithi105 ceph-mon[32280]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:13.891 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:13 smithi105 podman[73585]: 2024-03-26 17:36:13.77522462 +0000 UTC m=+0.115561281 container create 4276e5e49d5470b605e313f3b6bfb61efb7fc0fb43e2417160c56e5aa28a208a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-26T17:36:13.891 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:13 smithi105 podman[73585]: 2024-03-26 17:36:13.678642736 +0000 UTC m=+0.018979408 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:36:13.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:13 smithi135 ceph-mon[35636]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:14.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:13 smithi144 ceph-mon[36095]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:14.393 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:13 smithi105 podman[73585]: 2024-03-26 17:36:13.986100374 +0000 UTC m=+0.326437036 container init 4276e5e49d5470b605e313f3b6bfb61efb7fc0fb43e2417160c56e5aa28a208a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T17:36:14.393 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:13 smithi105 podman[73585]: 2024-03-26 17:36:13.988752348 +0000 UTC m=+0.329089014 container start 4276e5e49d5470b605e313f3b6bfb61efb7fc0fb43e2417160c56e5aa28a208a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T17:36:14.393 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-mds[73606]: -- 172.21.15.105:0/1346665429 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596e6f7fa40 con 0x5596e6f52c00 2024-03-26T17:36:14.393 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a[73602]: starting mds.a at 2024-03-26T17:36:14.393 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:14 smithi105 bash[73585]: 4276e5e49d5470b605e313f3b6bfb61efb7fc0fb43e2417160c56e5aa28a208a 2024-03-26T17:36:14.393 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 17:36:14 smithi105 systemd[1]: Started Ceph mds.a for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:36:14.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1181281079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:14.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:14.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:14.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:14.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:14.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:14.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:14 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:14.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:14 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1181281079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:14.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:14 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:14.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:14 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:14.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:14 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:14.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:14 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:14.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:14 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:14.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:14 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:15.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:14 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1181281079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:15.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:15.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:15.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:15.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T17:36:15.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T17:36:15.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:14 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:15.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:15 smithi135 ceph-mon[35636]: Deploying daemon mds.b on smithi135 2024-03-26T17:36:15.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:15 smithi135 ceph-mon[35636]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:15.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:15 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] up:boot 2024-03-26T17:36:15.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:15 smithi135 ceph-mon[35636]: fsmap 11 up:standby 2024-03-26T17:36:15.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T17:36:15.857 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:15.890 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:15 smithi105 ceph-mon[32280]: Deploying daemon mds.b on smithi135 2024-03-26T17:36:15.890 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:15 smithi105 ceph-mon[32280]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:15.890 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:15 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] up:boot 2024-03-26T17:36:15.890 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:15 smithi105 ceph-mon[32280]: fsmap 11 up:standby 2024-03-26T17:36:15.890 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T17:36:15.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.979+0000 7f76dda7e640 1 -- 172.21.15.105:0/1828752883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d8104d10 msgr2=0x7f76d8107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.979+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1828752883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d8104d10 0x7f76d8107100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009900 tx=0x7f76c002bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.979+0000 7f76dda7e640 1 -- 172.21.15.105:0/1828752883 shutdown_connections 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.979+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1828752883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d8107640 0x7f76d8109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.979+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1828752883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d8104d10 0x7f76d8107100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.979+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1828752883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76d81023e0 0x7f76d81047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.979+0000 7f76dda7e640 1 -- 172.21.15.105:0/1828752883 >> 172.21.15.105:0/1828752883 conn(0x7f76d80fbd70 msgr2=0x7f76d80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.979+0000 7f76dda7e640 1 -- 172.21.15.105:0/1828752883 shutdown_connections 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 -- 172.21.15.105:0/1828752883 wait complete. 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 Processor -- start 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 -- start start 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d81023e0 0x7f76d8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:15.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d8104d10 0x7f76d81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76d8107640 0x7f76d819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810d400 con 0x7f76d81023e0 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810d280 con 0x7f76d8107640 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.980+0000 7f76dda7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810d580 con 0x7f76d8104d10 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d81023e0 0x7f76d8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d81023e0 0x7f76d8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54080/0 (socket says 172.21.15.105:54080) 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 -- 172.21.15.105:0/1127451057 learned_addr learned my addr 172.21.15.105:0/1127451057 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d7fff640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76d8107640 0x7f76d819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76ceffd640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d8104d10 0x7f76d81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 -- 172.21.15.105:0/1127451057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d8104d10 msgr2=0x7f76d81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d8104d10 0x7f76d81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 -- 172.21.15.105:0/1127451057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76d8107640 msgr2=0x7f76d819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76d8107640 0x7f76d819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:15.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 -- 172.21.15.105:0/1127451057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d819f7c0 con 0x7f76d81023e0 2024-03-26T17:36:15.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d77fe640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d81023e0 0x7f76d8197ba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f76c800d950 tx=0x7f76c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:15.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d57fa640 1 -- 172.21.15.105:0/1127451057 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8017070 con 0x7f76d81023e0 2024-03-26T17:36:15.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d819fab0 con 0x7f76d81023e0 2024-03-26T17:36:15.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d57fa640 1 -- 172.21.15.105:0/1127451057 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76c8009c30 con 0x7f76d81023e0 2024-03-26T17:36:15.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76d57fa640 1 -- 172.21.15.105:0/1127451057 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c800f620 con 0x7f76d81023e0 2024-03-26T17:36:15.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.981+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d819ffd0 con 0x7f76d81023e0 2024-03-26T17:36:15.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.982+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d80fd860 con 0x7f76d81023e0 2024-03-26T17:36:15.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.983+0000 7f76d57fa640 1 -- 172.21.15.105:0/1127451057 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f76c80109a0 con 0x7f76d81023e0 2024-03-26T17:36:15.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.984+0000 7f76d57fa640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76c40a2f20 0x7f76c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:15.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.984+0000 7f76ceffd640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76c40a2f20 0x7f76c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:15.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.984+0000 7f76d57fa640 1 -- 172.21.15.105:0/1127451057 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f76c80cead0 con 0x7f76d81023e0 2024-03-26T17:36:15.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.984+0000 7f76ceffd640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76c40a2f20 0x7f76c40a53e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76c0005ab0 tx=0x7f76c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:15.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:15.988+0000 7f76d57fa640 1 -- 172.21.15.105:0/1127451057 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f76c8009de0 con 0x7f76d81023e0 2024-03-26T17:36:16.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:15 smithi144 ceph-mon[36095]: Deploying daemon mds.b on smithi135 2024-03-26T17:36:16.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:15 smithi144 ceph-mon[36095]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:16.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:15 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] up:boot 2024-03-26T17:36:16.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:15 smithi144 ceph-mon[36095]: fsmap 11 up:standby 2024-03-26T17:36:16.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T17:36:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.203+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76d802d4d0 con 0x7f76d81023e0 2024-03-26T17:36:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.203+0000 7f76d57fa640 1 -- 172.21.15.105:0/1127451057 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7f76c809b730 con 0x7f76d81023e0 2024-03-26T17:36:16.196 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:16.197 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-26T17:36:16.197 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 12 2024-03-26T17:36:16.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76c40a2f20 msgr2=0x7f76c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76c40a2f20 0x7f76c40a53e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f76c0005ab0 tx=0x7f76c0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d81023e0 msgr2=0x7f76d8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d81023e0 0x7f76d8197ba0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f76c800d950 tx=0x7f76c800de10 comp rx=0 tx=0).stop 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 shutdown_connections 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76c40a2f20 0x7f76c40a53e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76d8107640 0x7f76d819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76d8104d10 0x7f76d81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.205+0000 7f76dda7e640 1 --2- 172.21.15.105:0/1127451057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76d81023e0 0x7f76d8197ba0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.206+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 >> 172.21.15.105:0/1127451057 conn(0x7f76d80fbd70 msgr2=0x7f76d8102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.206+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 shutdown_connections 2024-03-26T17:36:16.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:16.206+0000 7f76dda7e640 1 -- 172.21.15.105:0/1127451057 wait complete. 2024-03-26T17:36:16.416 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 systemd[1]: Starting Ceph mds.b for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T17:36:16.779 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:16 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1127451057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:16.779 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 podman[65688]: 2024-03-26 17:36:16.508108305 +0000 UTC m=+0.099686102 container create 3d18c077baf57d81bd2d21245f57cfd21390f5d704dc6db981ecbdc3d7fc7e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-03-26T17:36:16.779 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 podman[65688]: 2024-03-26 17:36:16.426502382 +0000 UTC m=+0.018080174 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T17:36:16.779 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 podman[65688]: 2024-03-26 17:36:16.727834836 +0000 UTC m=+0.319412629 container init 3d18c077baf57d81bd2d21245f57cfd21390f5d704dc6db981ecbdc3d7fc7e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T17:36:16.780 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 podman[65688]: 2024-03-26 17:36:16.730538219 +0000 UTC m=+0.322116008 container start 3d18c077baf57d81bd2d21245f57cfd21390f5d704dc6db981ecbdc3d7fc7e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T17:36:16.780 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 ceph-mds[65706]: -- 172.21.15.135:0/3693521013 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef2b35ba40 con 0x55ef2b32f000 2024-03-26T17:36:16.780 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: starting mds.b at 2024-03-26T17:36:16.780 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 bash[65688]: 3d18c077baf57d81bd2d21245f57cfd21390f5d704dc6db981ecbdc3d7fc7e9a 2024-03-26T17:36:17.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:16 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1127451057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:17.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:16 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1127451057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:17.166 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:16 smithi135 systemd[1]: Started Ceph mds.b for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T17:36:17.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:17 smithi135 ceph-mon[35636]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:17.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:17 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:36:17.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:17 smithi144 ceph-mon[36095]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:17.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.922 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:17 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:36:17.937 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:17 smithi105 ceph-mon[32280]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:17.937 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.938 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.938 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.938 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.938 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:17.938 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:17 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:36:18.210 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 -- 172.21.15.105:0/648247329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc100800 msgr2=0x7f5bfc100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 --2- 172.21.15.105:0/648247329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc100800 0x7f5bfc100c20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009900 tx=0x7f5bf002bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 -- 172.21.15.105:0/648247329 shutdown_connections 2024-03-26T17:36:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 --2- 172.21.15.105:0/648247329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bfc1023b0 0x7f5bfc108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 --2- 172.21.15.105:0/648247329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bfc1019f0 0x7f5bfc101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 --2- 172.21.15.105:0/648247329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc100800 0x7f5bfc100c20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 -- 172.21.15.105:0/648247329 >> 172.21.15.105:0/648247329 conn(0x7f5bfc0fbd70 msgr2=0x7f5bfc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 -- 172.21.15.105:0/648247329 shutdown_connections 2024-03-26T17:36:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 -- 172.21.15.105:0/648247329 wait complete. 2024-03-26T17:36:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.333+0000 7f5c012e8640 1 Processor -- start 2024-03-26T17:36:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5c012e8640 1 -- start start 2024-03-26T17:36:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5c012e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bfc100800 0x7f5bfc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5c012e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bfc1019f0 0x7f5bfc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5c012e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc1023b0 0x7f5bfc19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5c012e8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc10d450 con 0x7f5bfc1023b0 2024-03-26T17:36:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5c012e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc10d2d0 con 0x7f5bfc100800 2024-03-26T17:36:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5c012e8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc10d5d0 con 0x7f5bfc1019f0 2024-03-26T17:36:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc1023b0 0x7f5bfc19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc1023b0 0x7f5bfc19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54100/0 (socket says 172.21.15.105:54100) 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 -- 172.21.15.105:0/2390338490 learned_addr learned my addr 172.21.15.105:0/2390338490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfaffd640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bfc100800 0x7f5bfc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfa7fc640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bfc1019f0 0x7f5bfc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 -- 172.21.15.105:0/2390338490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bfc1019f0 msgr2=0x7f5bfc1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bfc1019f0 0x7f5bfc1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 -- 172.21.15.105:0/2390338490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bfc100800 msgr2=0x7f5bfc197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bfc100800 0x7f5bfc197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 -- 172.21.15.105:0/2390338490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc19f790 con 0x7f5bfc1023b0 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfa7fc640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bfc1019f0 0x7f5bfc1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5bfb7fe640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc1023b0 0x7f5bfc19f120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400eaa0 tx=0x7f5bf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.334+0000 7f5be3fff640 1 -- 172.21.15.105:0/2390338490 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf400cdb0 con 0x7f5bfc1023b0 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.335+0000 7f5be3fff640 1 -- 172.21.15.105:0/2390338490 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bf4005020 con 0x7f5bfc1023b0 2024-03-26T17:36:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.335+0000 7f5be3fff640 1 -- 172.21.15.105:0/2390338490 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40116d0 con 0x7f5bfc1023b0 2024-03-26T17:36:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.335+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bfc19fa80 con 0x7f5bfc1023b0 2024-03-26T17:36:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.335+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bfc1a0020 con 0x7f5bfc1023b0 2024-03-26T17:36:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.336+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bfc107b90 con 0x7f5bfc1023b0 2024-03-26T17:36:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.337+0000 7f5be3fff640 1 -- 172.21.15.105:0/2390338490 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f5bf40051d0 con 0x7f5bfc1023b0 2024-03-26T17:36:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.337+0000 7f5be3fff640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5be40a2fa0 0x7f5be40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.338+0000 7f5bfaffd640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5be40a2fa0 0x7f5be40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.338+0000 7f5be3fff640 1 -- 172.21.15.105:0/2390338490 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5bf40cf120 con 0x7f5bfc1023b0 2024-03-26T17:36:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.338+0000 7f5bfaffd640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5be40a2fa0 0x7f5be40a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0005ab0 tx=0x7f5bf0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:18.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.342+0000 7f5be3fff640 1 -- 172.21.15.105:0/2390338490 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5bf4016030 con 0x7f5bfc1023b0 2024-03-26T17:36:18.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.611+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bfc02d240 con 0x7f5bfc1023b0 2024-03-26T17:36:18.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.611+0000 7f5be3fff640 1 -- 172.21.15.105:0/2390338490 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7f5bf409c350 con 0x7f5bfc1023b0 2024-03-26T17:36:18.604 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:18.604 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-26T17:36:18.605 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 13 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5be40a2fa0 msgr2=0x7f5be40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5be40a2fa0 0x7f5be40a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0005ab0 tx=0x7f5bf0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc1023b0 msgr2=0x7f5bfc19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc1023b0 0x7f5bfc19f120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400eaa0 tx=0x7f5bf400ef60 comp rx=0 tx=0).stop 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 shutdown_connections 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5be40a2fa0 0x7f5be40a5460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bfc1023b0 0x7f5bfc19f120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bfc1019f0 0x7f5bfc1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 --2- 172.21.15.105:0/2390338490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bfc100800 0x7f5bfc197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 >> 172.21.15.105:0/2390338490 conn(0x7f5bfc0fbd70 msgr2=0x7f5bfc104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 shutdown_connections 2024-03-26T17:36:18.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.614+0000 7f5c012e8640 1 -- 172.21.15.105:0/2390338490 wait complete. 2024-03-26T17:36:18.618 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:36:18.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 -- 172.21.15.105:0/3114174131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102390 msgr2=0x7ff700108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 --2- 172.21.15.105:0/3114174131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102390 0x7ff700108c20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009900 tx=0x7ff6f802bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 -- 172.21.15.105:0/3114174131 shutdown_connections 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 --2- 172.21.15.105:0/3114174131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff700102390 0x7ff700108c20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 --2- 172.21.15.105:0/3114174131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7001019d0 0x7ff700101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 --2- 172.21.15.105:0/3114174131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7001007e0 0x7ff700100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 -- 172.21.15.105:0/3114174131 >> 172.21.15.105:0/3114174131 conn(0x7ff7000fbd50 msgr2=0x7ff7000fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 -- 172.21.15.105:0/3114174131 shutdown_connections 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.733+0000 7ff707dcd640 1 -- 172.21.15.105:0/3114174131 wait complete. 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff707dcd640 1 Processor -- start 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff707dcd640 1 -- start start 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff707dcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7001007e0 0x7ff700197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff707dcd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7001019d0 0x7ff700197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff707dcd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff700102390 0x7ff70019f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff707dcd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010c310 con 0x7ff7001007e0 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff707dcd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010c190 con 0x7ff700102390 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff707dcd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff70010c490 con 0x7ff7001019d0 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705b69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7001007e0 0x7ff700197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705b69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7001007e0 0x7ff700197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54124/0 (socket says 172.21.15.105:54124) 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705b69640 1 -- 172.21.15.105:0/1855301563 learned_addr learned my addr 172.21.15.105:0/1855301563 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff70636a640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff700102390 0x7ff70019f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705368640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7001019d0 0x7ff700197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705b69640 1 -- 172.21.15.105:0/1855301563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7001019d0 msgr2=0x7ff700197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705b69640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7001019d0 0x7ff700197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705b69640 1 -- 172.21.15.105:0/1855301563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff700102390 msgr2=0x7ff70019f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705b69640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff700102390 0x7ff70019f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.734+0000 7ff705b69640 1 -- 172.21.15.105:0/1855301563 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70019f6d0 con 0x7ff7001007e0 2024-03-26T17:36:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.735+0000 7ff705b69640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7001007e0 0x7ff700197ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009fd0 tx=0x7ff6fc00c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.735+0000 7ff6f6ffd640 1 -- 172.21.15.105:0/1855301563 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc011470 con 0x7ff7001007e0 2024-03-26T17:36:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.735+0000 7ff6f6ffd640 1 -- 172.21.15.105:0/1855301563 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6fc011ac0 con 0x7ff7001007e0 2024-03-26T17:36:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.735+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70019f9c0 con 0x7ff7001007e0 2024-03-26T17:36:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.735+0000 7ff6f6ffd640 1 -- 172.21.15.105:0/1855301563 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc01aab0 con 0x7ff7001007e0 2024-03-26T17:36:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.735+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70019ff10 con 0x7ff7001007e0 2024-03-26T17:36:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.736+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7001a01a0 con 0x7ff7001007e0 2024-03-26T17:36:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.737+0000 7ff6f6ffd640 1 -- 172.21.15.105:0/1855301563 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7ff6fc011c70 con 0x7ff7001007e0 2024-03-26T17:36:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.738+0000 7ff6f6ffd640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6ec0a2f20 0x7ff6ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.738+0000 7ff705368640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6ec0a2f20 0x7ff6ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.738+0000 7ff6f6ffd640 1 -- 172.21.15.105:0/1855301563 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6fc0cf750 con 0x7ff7001007e0 2024-03-26T17:36:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.738+0000 7ff705368640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6ec0a2f20 0x7ff6ec0a53e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff700198fc0 tx=0x7ff6f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.742+0000 7ff6f6ffd640 1 -- 172.21.15.105:0/1855301563 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff6fc016030 con 0x7ff7001007e0 2024-03-26T17:36:18.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:18 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] up:boot 2024-03-26T17:36:18.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:18 smithi105 ceph-mon[32280]: fsmap 12 up:standby 2024-03-26T17:36:18.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:18 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T17:36:18.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:18 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:18.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:18 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:18.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:18 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2390338490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:18.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:18 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:18.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:18 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:18.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.920+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7001a0870 con 0x7ff7001007e0 2024-03-26T17:36:18.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.921+0000 7ff6f6ffd640 1 -- 172.21.15.105:0/1855301563 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18491 (secure 0 0 0) 0x7ff6fc09c980 con 0x7ff7001007e0 2024-03-26T17:36:18.914 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:18.915 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":81,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:35:33.812900+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:36:18.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:18 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] up:boot 2024-03-26T17:36:18.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:18 smithi135 ceph-mon[35636]: fsmap 12 up:standby 2024-03-26T17:36:18.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:18 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T17:36:18.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:18 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:18.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:18 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:18.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:18 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2390338490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:18.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:18 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:18.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:18 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6ec0a2f20 msgr2=0x7ff6ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6ec0a2f20 0x7ff6ec0a53e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff700198fc0 tx=0x7ff6f0005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7001007e0 msgr2=0x7ff700197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7001007e0 0x7ff700197ab0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009fd0 tx=0x7ff6fc00c270 comp rx=0 tx=0).stop 2024-03-26T17:36:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 shutdown_connections 2024-03-26T17:36:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6ec0a2f20 0x7ff6ec0a53e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff700102390 0x7ff70019f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7001019d0 0x7ff700197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 --2- 172.21.15.105:0/1855301563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7001007e0 0x7ff700197ab0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:18.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 >> 172.21.15.105:0/1855301563 conn(0x7ff7000fbd50 msgr2=0x7ff7000fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:18.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 shutdown_connections 2024-03-26T17:36:18.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:18.923+0000 7ff707dcd640 1 -- 172.21.15.105:0/1855301563 wait complete. 2024-03-26T17:36:18.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:36:18.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:36:19.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 -- 172.21.15.105:0/1691297609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98301019b0 msgr2=0x7f9830101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 --2- 172.21.15.105:0/1691297609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98301019b0 0x7f9830101e30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9820009900 tx=0x7f982002bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 -- 172.21.15.105:0/1691297609 shutdown_connections 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 --2- 172.21.15.105:0/1691297609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9830102370 0x7f9830108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 --2- 172.21.15.105:0/1691297609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98301019b0 0x7f9830101e30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 --2- 172.21.15.105:0/1691297609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98301007c0 0x7f9830100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 -- 172.21.15.105:0/1691297609 >> 172.21.15.105:0/1691297609 conn(0x7f98300fbd50 msgr2=0x7f98300fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 -- 172.21.15.105:0/1691297609 shutdown_connections 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 -- 172.21.15.105:0/1691297609 wait complete. 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.040+0000 7f9837627640 1 Processor -- start 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 -- start start 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98301007c0 0x7f9830069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98301019b0 0x7f983019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9830102370 0x7f983019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f983010c2f0 con 0x7f9830102370 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f983010c170 con 0x7f98301007c0 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f983010c470 con 0x7f98301019b0 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9830102370 0x7f983019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9830102370 0x7f983019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54156/0 (socket says 172.21.15.105:54156) 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 -- 172.21.15.105:0/472620587 learned_addr learned my addr 172.21.15.105:0/472620587 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9834bc2640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98301019b0 0x7f983019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 -- 172.21.15.105:0/472620587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98301019b0 msgr2=0x7f983019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98301019b0 0x7f983019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f98353c3640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98301007c0 0x7f9830069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 -- 172.21.15.105:0/472620587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98301007c0 msgr2=0x7f9830069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98301007c0 0x7f9830069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 -- 172.21.15.105:0/472620587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983019f720 con 0x7f9830102370 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f98353c3640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98301007c0 0x7f9830069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9835bc4640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9830102370 0x7f983019ee10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f982c00eb10 tx=0x7f982c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f98267fc640 1 -- 172.21.15.105:0/472620587 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c004b20 con 0x7f9830102370 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f98267fc640 1 -- 172.21.15.105:0/472620587 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f982c004cc0 con 0x7f9830102370 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983019f9b0 con 0x7f9830102370 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f98267fc640 1 -- 172.21.15.105:0/472620587 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c010690 con 0x7f9830102370 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.041+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f983019fed0 con 0x7f9830102370 2024-03-26T17:36:19.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.042+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9830107b50 con 0x7f9830102370 2024-03-26T17:36:19.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.043+0000 7f98267fc640 1 -- 172.21.15.105:0/472620587 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f982c005020 con 0x7f9830102370 2024-03-26T17:36:19.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.044+0000 7f98267fc640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f981c0a2f20 0x7f981c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.044+0000 7f98353c3640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f981c0a2f20 0x7f981c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.044+0000 7f98267fc640 1 -- 172.21.15.105:0/472620587 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f982c0cf6b0 con 0x7f9830102370 2024-03-26T17:36:19.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.044+0000 7f98353c3640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f981c0a2f20 0x7f981c0a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9830101810 tx=0x7f9828005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:19.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.048+0000 7f98267fc640 1 -- 172.21.15.105:0/472620587 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f982c09c980 con 0x7f9830102370 2024-03-26T17:36:19.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:18 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] up:boot 2024-03-26T17:36:19.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:18 smithi144 ceph-mon[36095]: fsmap 12 up:standby 2024-03-26T17:36:19.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:18 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T17:36:19.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:18 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:19.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:18 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:19.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:18 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2390338490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:19.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:18 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:19.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:18 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:19.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.229+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f983002d460 con 0x7f9830102370 2024-03-26T17:36:19.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.229+0000 7f98267fc640 1 -- 172.21.15.105:0/472620587 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f982c09c280 con 0x7f9830102370 2024-03-26T17:36:19.222 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:36:19.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f981c0a2f20 msgr2=0x7f981c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f981c0a2f20 0x7f981c0a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9830101810 tx=0x7f9828005db0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9830102370 msgr2=0x7f983019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9830102370 0x7f983019ee10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f982c00eb10 tx=0x7f982c00c2d0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 shutdown_connections 2024-03-26T17:36:19.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f981c0a2f20 0x7f981c0a53e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9830102370 0x7f983019ee10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98301019b0 0x7f983019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 --2- 172.21.15.105:0/472620587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98301007c0 0x7f9830069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 >> 172.21.15.105:0/472620587 conn(0x7f98300fbd50 msgr2=0x7f9830104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:19.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 shutdown_connections 2024-03-26T17:36:19.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.231+0000 7f9837627640 1 -- 172.21.15.105:0/472620587 wait complete. 2024-03-26T17:36:19.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-26T17:36:19.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.344+0000 7ff99cf1d640 1 -- 172.21.15.105:0/857867203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 msgr2=0x7ff998109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.344+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/857867203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 0x7ff998109a10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff990009ca0 tx=0x7ff99002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 -- 172.21.15.105:0/857867203 shutdown_connections 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/857867203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 0x7ff998109a10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/857867203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff998104c90 0x7ff998107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/857867203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff998102360 0x7ff998104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 -- 172.21.15.105:0/857867203 >> 172.21.15.105:0/857867203 conn(0x7ff9980fbd50 msgr2=0x7ff9980fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 -- 172.21.15.105:0/857867203 shutdown_connections 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 -- 172.21.15.105:0/857867203 wait complete. 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 Processor -- start 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 -- start start 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff998102360 0x7ff998197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff998104c90 0x7ff998197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 0x7ff99819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810d410 con 0x7ff9981075c0 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810d290 con 0x7ff998102360 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.345+0000 7ff99cf1d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810d590 con 0x7ff998104c90 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 0x7ff99819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 0x7ff99819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54178/0 (socket says 172.21.15.105:54178) 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff995d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff998104c90 0x7ff998197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 -- 172.21.15.105:0/751776580 learned_addr learned my addr 172.21.15.105:0/751776580 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff99659c640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff998102360 0x7ff998197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 -- 172.21.15.105:0/751776580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff998104c90 msgr2=0x7ff998197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff998104c90 0x7ff998197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 -- 172.21.15.105:0/751776580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff998102360 msgr2=0x7ff998197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff998102360 0x7ff998197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 -- 172.21.15.105:0/751776580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99819f680 con 0x7ff9981075c0 2024-03-26T17:36:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff996d9d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 0x7ff99819f010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff99002c700 tx=0x7ff9900328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff9877fe640 1 -- 172.21.15.105:0/751776580 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990041020 con 0x7ff9981075c0 2024-03-26T17:36:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff9877fe640 1 -- 172.21.15.105:0/751776580 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff990003e20 con 0x7ff9981075c0 2024-03-26T17:36:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99819f910 con 0x7ff9981075c0 2024-03-26T17:36:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff9877fe640 1 -- 172.21.15.105:0/751776580 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990039040 con 0x7ff9981075c0 2024-03-26T17:36:19.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.346+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99819fe50 con 0x7ff9981075c0 2024-03-26T17:36:19.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.347+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9980fd840 con 0x7ff9981075c0 2024-03-26T17:36:19.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.348+0000 7ff9877fe640 1 -- 172.21.15.105:0/751776580 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7ff990044020 con 0x7ff9981075c0 2024-03-26T17:36:19.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.349+0000 7ff9877fe640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9800a2fa0 0x7ff9800a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.349+0000 7ff99659c640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9800a2fa0 0x7ff9800a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.349+0000 7ff9877fe640 1 -- 172.21.15.105:0/751776580 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff9900f1580 con 0x7ff9981075c0 2024-03-26T17:36:19.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.349+0000 7ff99659c640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9800a2fa0 0x7ff9800a5460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff988009aa0 tx=0x7ff988005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:19.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.354+0000 7ff9877fe640 1 -- 172.21.15.105:0/751776580 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff99004b020 con 0x7ff9981075c0 2024-03-26T17:36:19.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.538+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff99802d240 con 0x7ff9981075c0 2024-03-26T17:36:19.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.539+0000 7ff9877fe640 1 -- 172.21.15.105:0/751776580 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7ff9900be850 con 0x7ff9981075c0 2024-03-26T17:36:19.533 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:19.533 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-03-26T17:36:19.533 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T17:36:19.533 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-03-26T17:36:19.534 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-03-26T17:36:19.534 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:36:19.534 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9800a2fa0 msgr2=0x7ff9800a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9800a2fa0 0x7ff9800a5460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff988009aa0 tx=0x7ff988005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 msgr2=0x7ff99819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 0x7ff99819f010 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff99002c700 tx=0x7ff9900328a0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 shutdown_connections 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9800a2fa0 0x7ff9800a5460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9981075c0 0x7ff99819f010 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff998104c90 0x7ff998197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 --2- 172.21.15.105:0/751776580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff998102360 0x7ff998197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 >> 172.21.15.105:0/751776580 conn(0x7ff9980fbd50 msgr2=0x7ff998102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 shutdown_connections 2024-03-26T17:36:19.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.542+0000 7ff99cf1d640 1 -- 172.21.15.105:0/751776580 wait complete. 2024-03-26T17:36:19.548 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-26T17:36:19.548 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-26T17:36:19.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 -- 172.21.15.105:0/1413375149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d81010e0 msgr2=0x7f38d8101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 --2- 172.21.15.105:0/1413375149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d81010e0 0x7f38d8101560 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00aa10 tx=0x7f38cc02cf70 comp rx=0 tx=0).stop 2024-03-26T17:36:19.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 -- 172.21.15.105:0/1413375149 shutdown_connections 2024-03-26T17:36:19.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 --2- 172.21.15.105:0/1413375149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38d8101aa0 0x7f38d81083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 --2- 172.21.15.105:0/1413375149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d81010e0 0x7f38d8101560 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 --2- 172.21.15.105:0/1413375149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d80fff10 0x7f38d8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 -- 172.21.15.105:0/1413375149 >> 172.21.15.105:0/1413375149 conn(0x7f38d80fb4b0 msgr2=0x7f38d80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 -- 172.21.15.105:0/1413375149 shutdown_connections 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.662+0000 7f38dd07e640 1 -- 172.21.15.105:0/1413375149 wait complete. 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38dd07e640 1 Processor -- start 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38dd07e640 1 -- start start 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38dd07e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d80fff10 0x7f38d8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38dd07e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38d81010e0 0x7f38d8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38dd07e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8101aa0 0x7f38d819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38dd07e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f38d810b210 con 0x7f38d8101aa0 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38dd07e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f38d810b090 con 0x7f38d81010e0 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38dd07e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38d810b390 con 0x7f38d80fff10 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8101aa0 0x7f38d819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d80fff10 0x7f38d8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8101aa0 0x7f38d819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54204/0 (socket says 172.21.15.105:54204) 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d759e640 1 -- 172.21.15.105:0/2859132780 learned_addr learned my addr 172.21.15.105:0/2859132780 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d659c640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38d81010e0 0x7f38d8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d6d9d640 1 -- 172.21.15.105:0/2859132780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38d81010e0 msgr2=0x7f38d8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d6d9d640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38d81010e0 0x7f38d8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d6d9d640 1 -- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8101aa0 msgr2=0x7f38d819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d6d9d640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8101aa0 0x7f38d819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.663+0000 7f38d6d9d640 1 -- 172.21.15.105:0/2859132780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d819f750 con 0x7f38d80fff10 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.664+0000 7f38d6d9d640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d80fff10 0x7f38d8197b30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38c800e9c0 tx=0x7f38c800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.664+0000 7f38bffff640 1 -- 172.21.15.105:0/2859132780 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c800ccc0 con 0x7f38d80fff10 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.664+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d819fa40 con 0x7f38d80fff10 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.664+0000 7f38bffff640 1 -- 172.21.15.105:0/2859132780 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c80035f0 con 0x7f38d80fff10 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.664+0000 7f38bffff640 1 -- 172.21.15.105:0/2859132780 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c800ce60 con 0x7f38d80fff10 2024-03-26T17:36:19.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.664+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d819ff60 con 0x7f38d80fff10 2024-03-26T17:36:19.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.665+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d81073c0 con 0x7f38d80fff10 2024-03-26T17:36:19.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.666+0000 7f38bffff640 1 -- 172.21.15.105:0/2859132780 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f38c80037a0 con 0x7f38d80fff10 2024-03-26T17:36:19.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.666+0000 7f38bffff640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38c00a2f20 0x7f38c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:19.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.666+0000 7f38d659c640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38c00a2f20 0x7f38c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:19.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.666+0000 7f38bffff640 1 -- 172.21.15.105:0/2859132780 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f38c80cf2f0 con 0x7f38d80fff10 2024-03-26T17:36:19.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.667+0000 7f38d659c640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38c00a2f20 0x7f38c00a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00a340 tx=0x7f38cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:19.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.670+0000 7f38bffff640 1 -- 172.21.15.105:0/2859132780 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f38c809c560 con 0x7f38d80fff10 2024-03-26T17:36:19.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:19.858+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f38d80624c0 con 0x7f38d80fff10 2024-03-26T17:36:19.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:19.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:19.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:19.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1855301563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:19.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:19.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:36:19.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:19.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/472620587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:36:19.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:36:19.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:19 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/751776580' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T17:36:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1855301563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:20.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:20.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:36:20.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:20.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/472620587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:36:20.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:36:20.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:19 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/751776580' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1855301563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/472620587' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:36:20.152 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:19 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/751776580' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T17:36:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.706+0000 7f38bffff640 1 -- 172.21.15.105:0/2859132780 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f38c809be60 con 0x7f38d80fff10 2024-03-26T17:36:20.700 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-03-26T17:36:20.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38c00a2f20 msgr2=0x7f38c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:20.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38c00a2f20 0x7f38c00a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00a340 tx=0x7f38cc005fd0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d80fff10 msgr2=0x7f38d8197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:20.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d80fff10 0x7f38d8197b30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38c800e9c0 tx=0x7f38c800ee80 comp rx=0 tx=0).stop 2024-03-26T17:36:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 shutdown_connections 2024-03-26T17:36:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38c00a2f20 0x7f38c00a53e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38d8101aa0 0x7f38d819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38d81010e0 0x7f38d8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 --2- 172.21.15.105:0/2859132780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d80fff10 0x7f38d8197b30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.711+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 >> 172.21.15.105:0/2859132780 conn(0x7f38d80fb4b0 msgr2=0x7f38d8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:20.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.712+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 shutdown_connections 2024-03-26T17:36:20.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.712+0000 7f38dd07e640 1 -- 172.21.15.105:0/2859132780 wait complete. 2024-03-26T17:36:20.718 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-26T17:36:20.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 -- 172.21.15.105:0/146030209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0103480 msgr2=0x7f9ef0109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:20.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/146030209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0103480 0x7f9ef0109d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8009ca0 tx=0x7f9ee802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 -- 172.21.15.105:0/146030209 shutdown_connections 2024-03-26T17:36:20.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/146030209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0103480 0x7f9ef0109d10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/146030209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ef0102ac0 0x7f9ef0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/146030209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef01018d0 0x7f9ef0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 -- 172.21.15.105:0/146030209 >> 172.21.15.105:0/146030209 conn(0x7f9ef00fce20 msgr2=0x7f9ef00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:20.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 -- 172.21.15.105:0/146030209 shutdown_connections 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 -- 172.21.15.105:0/146030209 wait complete. 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.840+0000 7f9ef72f7640 1 Processor -- start 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef72f7640 1 -- start start 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef72f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef01018d0 0x7f9ef0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef72f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0102ac0 0x7f9ef0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef72f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ef0103480 0x7f9ef019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef72f7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef010d400 con 0x7f9ef0102ac0 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef72f7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef010d280 con 0x7f9ef0103480 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef72f7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef010d580 con 0x7f9ef01018d0 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0102ac0 0x7f9ef0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0102ac0 0x7f9ef0198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54212/0 (socket says 172.21.15.105:54212) 2024-03-26T17:36:20.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 -- 172.21.15.105:0/2933565151 learned_addr learned my addr 172.21.15.105:0/2933565151 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef5093640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef01018d0 0x7f9ef0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef5894640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ef0103480 0x7f9ef019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 -- 172.21.15.105:0/2933565151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef01018d0 msgr2=0x7f9ef0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef01018d0 0x7f9ef0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 -- 172.21.15.105:0/2933565151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ef0103480 msgr2=0x7f9ef019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ef0103480 0x7f9ef019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 -- 172.21.15.105:0/2933565151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef019f720 con 0x7f9ef0102ac0 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ef4892640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0102ac0 0x7f9ef0198040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000e9f0 tx=0x7f9ee000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:20.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.841+0000 7f9ee67fc640 1 -- 172.21.15.105:0/2933565151 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee000cd80 con 0x7f9ef0102ac0 2024-03-26T17:36:20.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.842+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef019fa10 con 0x7f9ef0102ac0 2024-03-26T17:36:20.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.842+0000 7f9ee67fc640 1 -- 172.21.15.105:0/2933565151 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ee00108e0 con 0x7f9ef0102ac0 2024-03-26T17:36:20.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.842+0000 7f9ee67fc640 1 -- 172.21.15.105:0/2933565151 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0018920 con 0x7f9ef0102ac0 2024-03-26T17:36:20.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.842+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef019ffb0 con 0x7f9ef0102ac0 2024-03-26T17:36:20.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.843+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef0108d30 con 0x7f9ef0102ac0 2024-03-26T17:36:20.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.844+0000 7f9ee67fc640 1 -- 172.21.15.105:0/2933565151 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f9ee0010a90 con 0x7f9ef0102ac0 2024-03-26T17:36:20.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.845+0000 7f9ee67fc640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9edc0a2f20 0x7f9edc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:20.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.845+0000 7f9ef5093640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9edc0a2f20 0x7f9edc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:20.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.845+0000 7f9ee67fc640 1 -- 172.21.15.105:0/2933565151 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f9ee00cf800 con 0x7f9ef0102ac0 2024-03-26T17:36:20.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.845+0000 7f9ef5093640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9edc0a2f20 0x7f9edc0a53e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0102920 tx=0x7f9eec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:20.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:20.849+0000 7f9ee67fc640 1 -- 172.21.15.105:0/2933565151 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9ee009c310 con 0x7f9ef0102ac0 2024-03-26T17:36:21.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.046+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9ef002d240 con 0x7f9ef0102ac0 2024-03-26T17:36:21.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:20 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2859132780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:36:21.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:20 smithi144 ceph-mon[36095]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:36:21.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:20 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2859132780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:36:21.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:20 smithi105 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:36:21.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:20 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2859132780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:36:21.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:20 smithi135 ceph-mon[35636]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T17:36:21.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.706+0000 7f9ee67fc640 1 -- 172.21.15.105:0/2933565151 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f9ee00a03b0 con 0x7f9ef0102ac0 2024-03-26T17:36:21.701 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-03-26T17:36:21.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.709+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9edc0a2f20 msgr2=0x7f9edc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.709+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9edc0a2f20 0x7f9edc0a53e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0102920 tx=0x7f9eec005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.709+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0102ac0 msgr2=0x7f9ef0198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.709+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0102ac0 0x7f9ef0198040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000e9f0 tx=0x7f9ee000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.709+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 shutdown_connections 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.710+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9edc0a2f20 0x7f9edc0a53e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.710+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ef0103480 0x7f9ef019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.710+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ef0102ac0 0x7f9ef0198040 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.710+0000 7f9ef72f7640 1 --2- 172.21.15.105:0/2933565151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef01018d0 0x7f9ef0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.710+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 >> 172.21.15.105:0/2933565151 conn(0x7f9ef00fce20 msgr2=0x7f9ef0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.710+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 shutdown_connections 2024-03-26T17:36:21.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.710+0000 7f9ef72f7640 1 -- 172.21.15.105:0/2933565151 wait complete. 2024-03-26T17:36:21.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-26T17:36:21.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 -- 172.21.15.105:0/2387360259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd180104c70 msgr2=0x7fd180107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:21.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 --2- 172.21.15.105:0/2387360259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd180104c70 0x7fd180107060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd168009900 tx=0x7fd16802bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:21.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 -- 172.21.15.105:0/2387360259 shutdown_connections 2024-03-26T17:36:21.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 --2- 172.21.15.105:0/2387360259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1801075a0 0x7fd1801099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 --2- 172.21.15.105:0/2387360259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd180104c70 0x7fd180107060 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 --2- 172.21.15.105:0/2387360259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd180102340 0x7fd180104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 -- 172.21.15.105:0/2387360259 >> 172.21.15.105:0/2387360259 conn(0x7fd1800fbd50 msgr2=0x7fd1800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 -- 172.21.15.105:0/2387360259 shutdown_connections 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.835+0000 7fd184c52640 1 -- 172.21.15.105:0/2387360259 wait complete. 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd184c52640 1 Processor -- start 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd184c52640 1 -- start start 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd184c52640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180102340 0x7fd180197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd184c52640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd180104c70 0x7fd1801980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd184c52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1801075a0 0x7fd18019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd184c52640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010d3e0 con 0x7fd1801075a0 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd184c52640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010d260 con 0x7fd180104c70 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd184c52640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd18010d560 con 0x7fd180102340 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1801075a0 0x7fd18019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1801075a0 0x7fd18019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54226/0 (socket says 172.21.15.105:54226) 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180102340 0x7fd180197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:21.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 -- 172.21.15.105:0/399651711 learned_addr learned my addr 172.21.15.105:0/399651711 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd177fff640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd180104c70 0x7fd1801980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 -- 172.21.15.105:0/399651711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180102340 msgr2=0x7fd180197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180102340 0x7fd180197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 -- 172.21.15.105:0/399651711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd180104c70 msgr2=0x7fd1801980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd180104c70 0x7fd1801980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 -- 172.21.15.105:0/399651711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18019f750 con 0x7fd1801075a0 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd17ed9d640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1801075a0 0x7fd18019f0e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd17800eaa0 tx=0x7fd17800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.836+0000 7fd1777fe640 1 -- 172.21.15.105:0/399651711 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17800cdb0 con 0x7fd1801075a0 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.837+0000 7fd1777fe640 1 -- 172.21.15.105:0/399651711 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd178005020 con 0x7fd1801075a0 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.837+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18019fa40 con 0x7fd1801075a0 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.837+0000 7fd1777fe640 1 -- 172.21.15.105:0/399651711 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780116d0 con 0x7fd1801075a0 2024-03-26T17:36:21.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.837+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18019ff10 con 0x7fd1801075a0 2024-03-26T17:36:21.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.838+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1800fd840 con 0x7fd1801075a0 2024-03-26T17:36:21.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.839+0000 7fd1777fe640 1 -- 172.21.15.105:0/399651711 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fd1780051d0 con 0x7fd1801075a0 2024-03-26T17:36:21.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.839+0000 7fd1777fe640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd16c0a2f20 0x7fd16c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:21.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.839+0000 7fd17e59c640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd16c0a2f20 0x7fd16c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:21.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.839+0000 7fd1777fe640 1 -- 172.21.15.105:0/399651711 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fd1780cf3d0 con 0x7fd1801075a0 2024-03-26T17:36:21.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.840+0000 7fd17e59c640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd16c0a2f20 0x7fd16c0a53e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd170005d00 tx=0x7fd17000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:21.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:21.844+0000 7fd1777fe640 1 -- 172.21.15.105:0/399651711 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd178016030 con 0x7fd1801075a0 2024-03-26T17:36:22.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.057+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd1800624c0 con 0x7fd1801075a0 2024-03-26T17:36:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2933565151' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:36:22.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2933565151' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:22.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:36:22.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2933565151' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:36:22.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:36:22.172 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.172 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:36:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.726+0000 7fd1777fe640 1 -- 172.21.15.105:0/399651711 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd17809c3b0 con 0x7fd1801075a0 2024-03-26T17:36:22.720 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 2 and data pool 3 2024-03-26T17:36:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.730+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd16c0a2f20 msgr2=0x7fd16c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.730+0000 7fd184c52640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd16c0a2f20 0x7fd16c0a53e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd170005d00 tx=0x7fd17000b410 comp rx=0 tx=0).stop 2024-03-26T17:36:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.731+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1801075a0 msgr2=0x7fd18019f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.731+0000 7fd184c52640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1801075a0 0x7fd18019f0e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd17800eaa0 tx=0x7fd17800ef60 comp rx=0 tx=0).stop 2024-03-26T17:36:22.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.731+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 shutdown_connections 2024-03-26T17:36:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.731+0000 7fd184c52640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd16c0a2f20 0x7fd16c0a53e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.731+0000 7fd184c52640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1801075a0 0x7fd18019f0e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.731+0000 7fd184c52640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd180104c70 0x7fd1801980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.731+0000 7fd184c52640 1 --2- 172.21.15.105:0/399651711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd180102340 0x7fd180197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.731+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 >> 172.21.15.105:0/399651711 conn(0x7fd1800fbd50 msgr2=0x7fd1801041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.732+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 shutdown_connections 2024-03-26T17:36:22.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.733+0000 7fd184c52640 1 -- 172.21.15.105:0/399651711 wait complete. 2024-03-26T17:36:22.748 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 -- 172.21.15.105:0/2211571553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d4104d10 msgr2=0x7f53d4107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 --2- 172.21.15.105:0/2211571553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d4104d10 0x7f53d4107100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009900 tx=0x7f53c002bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 -- 172.21.15.105:0/2211571553 shutdown_connections 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 --2- 172.21.15.105:0/2211571553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d4107640 0x7f53d4109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 --2- 172.21.15.105:0/2211571553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d4104d10 0x7f53d4107100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 --2- 172.21.15.105:0/2211571553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53d41023e0 0x7f53d41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 -- 172.21.15.105:0/2211571553 >> 172.21.15.105:0/2211571553 conn(0x7f53d40fbdf0 msgr2=0x7f53d40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 -- 172.21.15.105:0/2211571553 shutdown_connections 2024-03-26T17:36:22.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.858+0000 7f53db027640 1 -- 172.21.15.105:0/2211571553 wait complete. 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53db027640 1 Processor -- start 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53db027640 1 -- start start 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53db027640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d41023e0 0x7f53d4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53db027640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53d4104d10 0x7f53d4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53db027640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d4107640 0x7f53d419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53db027640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53d410d420 con 0x7f53d41023e0 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53db027640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53d410d2a0 con 0x7f53d4104d10 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53db027640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53d410d5a0 con 0x7f53d4107640 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d41023e0 0x7f53d4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d41023e0 0x7f53d4197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54238/0 (socket says 172.21.15.105:54238) 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 -- 172.21.15.105:0/1133560348 learned_addr learned my addr 172.21.15.105:0/1133560348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d3fff640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53d4104d10 0x7f53d4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d95c4640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d4107640 0x7f53d419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:22.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 -- 172.21.15.105:0/1133560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d4107640 msgr2=0x7f53d419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d4107640 0x7f53d419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 -- 172.21.15.105:0/1133560348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53d4104d10 msgr2=0x7f53d4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53d4104d10 0x7f53d4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 -- 172.21.15.105:0/1133560348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d419f770 con 0x7f53d41023e0 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d95c4640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d4107640 0x7f53d419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d8dc3640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d41023e0 0x7f53d4197b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53c800e9f0 tx=0x7f53c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d1ffb640 1 -- 172.21.15.105:0/1133560348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c800cd80 con 0x7f53d41023e0 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.859+0000 7f53d1ffb640 1 -- 172.21.15.105:0/1133560348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53c80108e0 con 0x7f53d41023e0 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.860+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d419fa60 con 0x7f53d41023e0 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.860+0000 7f53d1ffb640 1 -- 172.21.15.105:0/1133560348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8018920 con 0x7f53d41023e0 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.860+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d419ff80 con 0x7f53d41023e0 2024-03-26T17:36:22.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.861+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d40fd8e0 con 0x7f53d41023e0 2024-03-26T17:36:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.862+0000 7f53d1ffb640 1 -- 172.21.15.105:0/1133560348 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f53c8010a90 con 0x7f53d41023e0 2024-03-26T17:36:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.863+0000 7f53d1ffb640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53c40a2f20 0x7f53c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.863+0000 7f53d3fff640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53c40a2f20 0x7f53c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.863+0000 7f53d1ffb640 1 -- 172.21.15.105:0/1133560348 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f53c80cf6d0 con 0x7f53d41023e0 2024-03-26T17:36:22.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.863+0000 7f53d3fff640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53c40a2f20 0x7f53c40a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006010 tx=0x7f53c0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:22.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:22.867+0000 7f53d1ffb640 1 -- 172.21.15.105:0/1133560348 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f53c809c650 con 0x7f53d41023e0 2024-03-26T17:36:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.065+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53d402d480 con 0x7f53d41023e0 2024-03-26T17:36:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.066+0000 7f53d1ffb640 1 -- 172.21.15.105:0/1133560348 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22267 (secure 0 0 0) 0x7f53c809bf50 con 0x7f53d41023e0 2024-03-26T17:36:23.060 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:23.061 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":84,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:36:22.059365+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:36:23.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53c40a2f20 msgr2=0x7f53c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53c40a2f20 0x7f53c40a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006010 tx=0x7f53c0005be0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d41023e0 msgr2=0x7f53d4197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d41023e0 0x7f53d4197b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53c800e9f0 tx=0x7f53c800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 shutdown_connections 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53c40a2f20 0x7f53c40a53e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53d4107640 0x7f53d419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53d4104d10 0x7f53d4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 --2- 172.21.15.105:0/1133560348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53d41023e0 0x7f53d4197b50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 >> 172.21.15.105:0/1133560348 conn(0x7f53d40fbdf0 msgr2=0x7f53d4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 shutdown_connections 2024-03-26T17:36:23.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.068+0000 7f53db027640 1 -- 172.21.15.105:0/1133560348 wait complete. 2024-03-26T17:36:23.072 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2933565151' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T17:36:23.072 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:22 smithi105 ceph-mon[32280]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T17:36:23.072 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/399651711' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T17:36:23.073 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:22 smithi105 ceph-mon[32280]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T17:36:23.074 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:36:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2933565151' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T17:36:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:22 smithi144 ceph-mon[36095]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T17:36:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/399651711' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T17:36:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:22 smithi144 ceph-mon[36095]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T17:36:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2933565151' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T17:36:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:22 smithi135 ceph-mon[35636]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T17:36:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/399651711' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T17:36:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:22 smithi135 ceph-mon[35636]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T17:36:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 -- 172.21.15.105:0/897845143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 msgr2=0x7ff204101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/897845143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 0x7ff204101e50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009900 tx=0x7ff1f002bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 -- 172.21.15.105:0/897845143 shutdown_connections 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/897845143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff204102390 0x7ff204108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/897845143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 0x7ff204101e50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/897845143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2041007e0 0x7ff204100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 -- 172.21.15.105:0/897845143 >> 172.21.15.105:0/897845143 conn(0x7ff2040fbd50 msgr2=0x7ff2040fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 -- 172.21.15.105:0/897845143 shutdown_connections 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.182+0000 7ff20ac0b640 1 -- 172.21.15.105:0/897845143 wait complete. 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff20ac0b640 1 Processor -- start 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff20ac0b640 1 -- start start 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff20ac0b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2041007e0 0x7ff204069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff20ac0b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 0x7ff20419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff20ac0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff204102390 0x7ff20419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff20ac0b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff20410c400 con 0x7ff2041019d0 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff20ac0b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff20410c280 con 0x7ff2041007e0 2024-03-26T17:36:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff20ac0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff20410c580 con 0x7ff204102390 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff203fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 0x7ff20419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff203fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 0x7ff20419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54248/0 (socket says 172.21.15.105:54248) 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff203fff640 1 -- 172.21.15.105:0/51816095 learned_addr learned my addr 172.21.15.105:0/51816095 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff2091a8640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff204102390 0x7ff20419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff2089a7640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2041007e0 0x7ff204069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff203fff640 1 -- 172.21.15.105:0/51816095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff204102390 msgr2=0x7ff20419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff203fff640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff204102390 0x7ff20419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff203fff640 1 -- 172.21.15.105:0/51816095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2041007e0 msgr2=0x7ff204069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff203fff640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2041007e0 0x7ff204069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.183+0000 7ff203fff640 1 -- 172.21.15.105:0/51816095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20419f940 con 0x7ff2041019d0 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.184+0000 7ff203fff640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 0x7ff20419ca10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0036040 tx=0x7ff1f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.184+0000 7ff201ffb640 1 -- 172.21.15.105:0/51816095 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0043020 con 0x7ff2041019d0 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.184+0000 7ff201ffb640 1 -- 172.21.15.105:0/51816095 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1f00054c0 con 0x7ff2041019d0 2024-03-26T17:36:23.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.184+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20419fb70 con 0x7ff2041019d0 2024-03-26T17:36:23.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.184+0000 7ff201ffb640 1 -- 172.21.15.105:0/51816095 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0003d70 con 0x7ff2041019d0 2024-03-26T17:36:23.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.184+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2041a00c0 con 0x7ff2041019d0 2024-03-26T17:36:23.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.185+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff204107bc0 con 0x7ff2041019d0 2024-03-26T17:36:23.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.186+0000 7ff201ffb640 1 -- 172.21.15.105:0/51816095 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7ff1f0046020 con 0x7ff2041019d0 2024-03-26T17:36:23.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.186+0000 7ff201ffb640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1f40a2f20 0x7ff1f40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:23.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.187+0000 7ff2089a7640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1f40a2f20 0x7ff1f40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:23.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.187+0000 7ff201ffb640 1 -- 172.21.15.105:0/51816095 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7ff1f00f1750 con 0x7ff2041019d0 2024-03-26T17:36:23.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.187+0000 7ff2089a7640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1f40a2f20 0x7ff1f40a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff204101830 tx=0x7ff1f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:23.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.191+0000 7ff201ffb640 1 -- 172.21.15.105:0/51816095 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff1f004d020 con 0x7ff2041019d0 2024-03-26T17:36:23.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.369+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff20402d490 con 0x7ff2041019d0 2024-03-26T17:36:23.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.370+0000 7ff201ffb640 1 -- 172.21.15.105:0/51816095 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22267 (secure 0 0 0) 0x7ff1f00be6d0 con 0x7ff2041019d0 2024-03-26T17:36:23.365 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:23.365 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":84,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:36:22.059365+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:36:23.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.372+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1f40a2f20 msgr2=0x7ff1f40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.372+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1f40a2f20 0x7ff1f40a53e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff204101830 tx=0x7ff1f8008040 comp rx=0 tx=0).stop 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.372+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 msgr2=0x7ff20419ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.372+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 0x7ff20419ca10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0036040 tx=0x7ff1f00042f0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.372+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 shutdown_connections 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.372+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1f40a2f20 0x7ff1f40a53e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.372+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff204102390 0x7ff20419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.373+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2041019d0 0x7ff20419ca10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.373+0000 7ff20ac0b640 1 --2- 172.21.15.105:0/51816095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2041007e0 0x7ff204069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.373+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 >> 172.21.15.105:0/51816095 conn(0x7ff2040fbd50 msgr2=0x7ff204104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.373+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 shutdown_connections 2024-03-26T17:36:23.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.373+0000 7ff20ac0b640 1 -- 172.21.15.105:0/51816095 wait complete. 2024-03-26T17:36:23.378 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 -- 172.21.15.105:0/233134365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541034a0 msgr2=0x7fd254109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/233134365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541034a0 0x7fd254109d30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009ca0 tx=0x7fd24c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 -- 172.21.15.105:0/233134365 shutdown_connections 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/233134365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541034a0 0x7fd254109d30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/233134365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd254102ae0 0x7fd254102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/233134365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2541018f0 0x7fd254101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 -- 172.21.15.105:0/233134365 >> 172.21.15.105:0/233134365 conn(0x7fd2540fce60 msgr2=0x7fd2540ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 -- 172.21.15.105:0/233134365 shutdown_connections 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 -- 172.21.15.105:0/233134365 wait complete. 2024-03-26T17:36:23.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 Processor -- start 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 -- start start 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541018f0 0x7fd254197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd254102ae0 0x7fd254198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.489+0000 7fd25a6e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2541034a0 0x7fd25419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd25a6e8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd25410d400 con 0x7fd2541018f0 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd25a6e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd25410d280 con 0x7fd2541034a0 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd25a6e8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd25410d580 con 0x7fd254102ae0 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541018f0 0x7fd254197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541018f0 0x7fd254197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54266/0 (socket says 172.21.15.105:54266) 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 -- 172.21.15.105:0/1532752533 learned_addr learned my addr 172.21.15.105:0/1532752533 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd258c85640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2541034a0 0x7fd25419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 -- 172.21.15.105:0/1532752533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd254102ae0 msgr2=0x7fd254198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd254102ae0 0x7fd254198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 -- 172.21.15.105:0/1532752533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2541034a0 msgr2=0x7fd25419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2541034a0 0x7fd25419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 -- 172.21.15.105:0/1532752533 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25419f6f0 con 0x7fd2541018f0 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd253fff640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541018f0 0x7fd254197ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd24400c970 tx=0x7fd24400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd2517fa640 1 -- 172.21.15.105:0/1532752533 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244004220 con 0x7fd2541018f0 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd2517fa640 1 -- 172.21.15.105:0/1532752533 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2440043c0 con 0x7fd2541018f0 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25419f9e0 con 0x7fd2541018f0 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd2517fa640 1 -- 172.21.15.105:0/1532752533 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24400f450 con 0x7fd2541018f0 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.490+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25419ff30 con 0x7fd2541018f0 2024-03-26T17:36:23.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.491+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd254108cb0 con 0x7fd2541018f0 2024-03-26T17:36:23.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.492+0000 7fd2517fa640 1 -- 172.21.15.105:0/1532752533 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fd2440079b0 con 0x7fd2541018f0 2024-03-26T17:36:23.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.493+0000 7fd2517fa640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd23c0a2f20 0x7fd23c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:23.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.493+0000 7fd2537fe640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd23c0a2f20 0x7fd23c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:23.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.493+0000 7fd2517fa640 1 -- 172.21.15.105:0/1532752533 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fd2440cf410 con 0x7fd2541018f0 2024-03-26T17:36:23.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.493+0000 7fd2537fe640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd23c0a2f20 0x7fd23c0a53e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd2480058c0 tx=0x7fd248005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:23.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.496+0000 7fd2517fa640 1 -- 172.21.15.105:0/1532752533 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd244014030 con 0x7fd2541018f0 2024-03-26T17:36:23.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:23.714+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd25402d240 con 0x7fd2541018f0 2024-03-26T17:36:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: pgmap v281: 97 pgs: 21 creating+peering, 1 active+clean, 75 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/399651711' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1133560348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] up:active 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/51816095' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1532752533' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T17:36:24.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:23 smithi144 ceph-mon[36095]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: pgmap v281: 97 pgs: 21 creating+peering, 1 active+clean, 75 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/399651711' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1133560348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] up:active 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:24.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/51816095' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:24.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1532752533' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T17:36:24.144 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:23 smithi105 ceph-mon[32280]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T17:36:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: pgmap v281: 97 pgs: 21 creating+peering, 1 active+clean, 75 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T17:36:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T17:36:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T17:36:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/399651711' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T17:36:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T17:36:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T17:36:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1133560348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] up:active 2024-03-26T17:36:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/51816095' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1532752533' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T17:36:24.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:23 smithi135 ceph-mon[35636]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T17:36:24.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.303+0000 7fd2517fa640 1 -- 172.21.15.105:0/1532752533 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fd24409c390 con 0x7fd2541018f0 2024-03-26T17:36:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd23c0a2f20 msgr2=0x7fd23c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd23c0a2f20 0x7fd23c0a53e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd2480058c0 tx=0x7fd248005850 comp rx=0 tx=0).stop 2024-03-26T17:36:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541018f0 msgr2=0x7fd254197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541018f0 0x7fd254197ad0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd24400c970 tx=0x7fd24400ce30 comp rx=0 tx=0).stop 2024-03-26T17:36:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 shutdown_connections 2024-03-26T17:36:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd23c0a2f20 0x7fd23c0a53e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2541034a0 0x7fd25419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd254102ae0 0x7fd254198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 --2- 172.21.15.105:0/1532752533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2541018f0 0x7fd254197ad0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 >> 172.21.15.105:0/1532752533 conn(0x7fd2540fce60 msgr2=0x7fd254105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 shutdown_connections 2024-03-26T17:36:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.306+0000 7fd25a6e8640 1 -- 172.21.15.105:0/1532752533 wait complete. 2024-03-26T17:36:24.313 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-26T17:36:24.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-26T17:36:24.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 -- 172.21.15.105:0/2679094436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 msgr2=0x7fca48101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:24.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/2679094436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 0x7fca48101d00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fca38009900 tx=0x7fca3802bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:24.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 -- 172.21.15.105:0/2679094436 shutdown_connections 2024-03-26T17:36:24.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/2679094436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca48103490 0x7fca48109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/2679094436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca48102ad0 0x7fca48102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/2679094436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 0x7fca48101d00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 -- 172.21.15.105:0/2679094436 >> 172.21.15.105:0/2679094436 conn(0x7fca480fce50 msgr2=0x7fca480ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 -- 172.21.15.105:0/2679094436 shutdown_connections 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 -- 172.21.15.105:0/2679094436 wait complete. 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 Processor -- start 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.427+0000 7fca4e9c6640 1 -- start start 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4e9c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 0x7fca48197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4e9c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca48102ad0 0x7fca48197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4e9c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca48103490 0x7fca4819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4e9c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fca4810d3e0 con 0x7fca481018e0 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4e9c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca4810d260 con 0x7fca48102ad0 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4e9c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca4810d560 con 0x7fca48103490 2024-03-26T17:36:24.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 0x7fca48197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 0x7fca48197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54288/0 (socket says 172.21.15.105:54288) 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 -- 172.21.15.105:0/1989055880 learned_addr learned my addr 172.21.15.105:0/1989055880 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca477fe640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca48102ad0 0x7fca48197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4cf63640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca48103490 0x7fca4819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 -- 172.21.15.105:0/1989055880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca48103490 msgr2=0x7fca4819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca48103490 0x7fca4819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 -- 172.21.15.105:0/1989055880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca48102ad0 msgr2=0x7fca48197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca48102ad0 0x7fca48197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 -- 172.21.15.105:0/1989055880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4819f6a0 con 0x7fca481018e0 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca477fe640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca48102ad0 0x7fca48197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4cf63640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca48103490 0x7fca4819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca47fff640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 0x7fca48197a80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fca38000c00 tx=0x7fca38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca457fa640 1 -- 172.21.15.105:0/1989055880 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38043020 con 0x7fca481018e0 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca457fa640 1 -- 172.21.15.105:0/1989055880 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca38004440 con 0x7fca481018e0 2024-03-26T17:36:24.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca457fa640 1 -- 172.21.15.105:0/1989055880 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38003c90 con 0x7fca481018e0 2024-03-26T17:36:24.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4819f930 con 0x7fca481018e0 2024-03-26T17:36:24.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.428+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4819fe30 con 0x7fca481018e0 2024-03-26T17:36:24.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.429+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca48108cd0 con 0x7fca481018e0 2024-03-26T17:36:24.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.430+0000 7fca457fa640 1 -- 172.21.15.105:0/1989055880 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fca38046020 con 0x7fca481018e0 2024-03-26T17:36:24.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.431+0000 7fca457fa640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fca300a2f20 0x7fca300a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:24.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.431+0000 7fca477fe640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fca300a2f20 0x7fca300a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:24.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.431+0000 7fca457fa640 1 -- 172.21.15.105:0/1989055880 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fca380f1600 con 0x7fca481018e0 2024-03-26T17:36:24.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.431+0000 7fca477fe640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fca300a2f20 0x7fca300a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fca48198f90 tx=0x7fca3c00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:24.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.435+0000 7fca457fa640 1 -- 172.21.15.105:0/1989055880 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fca3804d020 con 0x7fca481018e0 2024-03-26T17:36:24.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:24.649+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fca480624c0 con 0x7fca481018e0 2024-03-26T17:36:25.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.308+0000 7fca457fa640 1 -- 172.21.15.105:0/1989055880 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fca3803c350 con 0x7fca481018e0 2024-03-26T17:36:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fca300a2f20 msgr2=0x7fca300a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fca300a2f20 0x7fca300a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fca48198f90 tx=0x7fca3c00a5c0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 msgr2=0x7fca48197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 0x7fca48197a80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fca38000c00 tx=0x7fca38004040 comp rx=0 tx=0).stop 2024-03-26T17:36:25.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 shutdown_connections 2024-03-26T17:36:25.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fca300a2f20 0x7fca300a53e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca48103490 0x7fca4819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca48102ad0 0x7fca48197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 --2- 172.21.15.105:0/1989055880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca481018e0 0x7fca48197a80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 >> 172.21.15.105:0/1989055880 conn(0x7fca480fce50 msgr2=0x7fca48105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:25.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 shutdown_connections 2024-03-26T17:36:25.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.311+0000 7fca4e9c6640 1 -- 172.21.15.105:0/1989055880 wait complete. 2024-03-26T17:36:25.321 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 -- 172.21.15.105:0/1571372968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 msgr2=0x7f8340108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 --2- 172.21.15.105:0/1571372968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 0x7f8340108c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8338009ca0 tx=0x7f833802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 -- 172.21.15.105:0/1571372968 shutdown_connections 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 --2- 172.21.15.105:0/1571372968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 0x7f8340108c10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 --2- 172.21.15.105:0/1571372968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83401019c0 0x7f8340101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 --2- 172.21.15.105:0/1571372968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83401007d0 0x7f8340100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 -- 172.21.15.105:0/1571372968 >> 172.21.15.105:0/1571372968 conn(0x7f83400fbd60 msgr2=0x7f83400fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 -- 172.21.15.105:0/1571372968 shutdown_connections 2024-03-26T17:36:25.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.434+0000 7f8347a7e640 1 -- 172.21.15.105:0/1571372968 wait complete. 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8347a7e640 1 Processor -- start 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8347a7e640 1 -- start start 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8347a7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83401007d0 0x7f834010b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8347a7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83401019c0 0x7f8340109790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8347a7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 0x7f8340109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8347a7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f834010d460 con 0x7f8340102380 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8347a7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f834010d2e0 con 0x7f83401007d0 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8347a7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f834010d5e0 con 0x7f83401019c0 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 0x7f8340109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834581a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83401007d0 0x7f834010b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 0x7f8340109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49184/0 (socket says 172.21.15.105:49184) 2024-03-26T17:36:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 -- 172.21.15.105:0/2664111744 learned_addr learned my addr 172.21.15.105:0/2664111744 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f8345019640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83401019c0 0x7f8340109790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 -- 172.21.15.105:0/2664111744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83401019c0 msgr2=0x7f8340109790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83401019c0 0x7f8340109790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 -- 172.21.15.105:0/2664111744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83401007d0 msgr2=0x7f834010b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83401007d0 0x7f834010b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 -- 172.21.15.105:0/2664111744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834010a210 con 0x7f8340102380 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.435+0000 7f834601b640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 0x7f8340109cd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f833802c700 tx=0x7f83380328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.436+0000 7f8336ffd640 1 -- 172.21.15.105:0/2664111744 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338041020 con 0x7f8340102380 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.436+0000 7f8336ffd640 1 -- 172.21.15.105:0/2664111744 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8338003e20 con 0x7f8340102380 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.436+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834010a4a0 con 0x7f8340102380 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.436+0000 7f8336ffd640 1 -- 172.21.15.105:0/2664111744 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338039040 con 0x7f8340102380 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.436+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83401a65c0 con 0x7f8340102380 2024-03-26T17:36:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.437+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8340107af0 con 0x7f8340102380 2024-03-26T17:36:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.438+0000 7f8336ffd640 1 -- 172.21.15.105:0/2664111744 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f8338044020 con 0x7f8340102380 2024-03-26T17:36:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.438+0000 7f8336ffd640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f832c0a2fa0 0x7f832c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.438+0000 7f834581a640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f832c0a2fa0 0x7f832c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.438+0000 7f8336ffd640 1 -- 172.21.15.105:0/2664111744 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f83380f17f0 con 0x7f8340102380 2024-03-26T17:36:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.439+0000 7f834581a640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f832c0a2fa0 0x7f832c0a5460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8340101820 tx=0x7f8330005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.442+0000 7f8336ffd640 1 -- 172.21.15.105:0/2664111744 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f833804b020 con 0x7f8340102380 2024-03-26T17:36:25.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:25 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1532752533' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T17:36:25.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:25 smithi105 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:25.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:25 smithi105 ceph-mon[32280]: pgmap v284: 97 pgs: 23 creating+peering, 41 active+clean, 33 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T17:36:25.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:25 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1989055880' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T17:36:25.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:25 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1532752533' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T17:36:25.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:25 smithi144 ceph-mon[36095]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:25.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:25 smithi144 ceph-mon[36095]: pgmap v284: 97 pgs: 23 creating+peering, 41 active+clean, 33 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T17:36:25.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:25 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1989055880' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T17:36:25.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:25.660+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f834002d400 con 0x7f8340102380 2024-03-26T17:36:25.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:25 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1532752533' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T17:36:25.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:25 smithi135 ceph-mon[35636]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:25.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:25 smithi135 ceph-mon[35636]: pgmap v284: 97 pgs: 23 creating+peering, 41 active+clean, 33 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-26T17:36:25.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:25 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1989055880' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-26T17:36:26.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.315+0000 7f8336ffd640 1 -- 172.21.15.105:0/2664111744 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f834002d400 con 0x7f8340102380 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f832c0a2fa0 msgr2=0x7f832c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f832c0a2fa0 0x7f832c0a5460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8340101820 tx=0x7f8330005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 msgr2=0x7f8340109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 0x7f8340109cd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f833802c700 tx=0x7f83380328a0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 shutdown_connections 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f832c0a2fa0 0x7f832c0a5460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8340102380 0x7f8340109cd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83401019c0 0x7f8340109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 --2- 172.21.15.105:0/2664111744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83401007d0 0x7f834010b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 >> 172.21.15.105:0/2664111744 conn(0x7f83400fbd60 msgr2=0x7f8340104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:26.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 shutdown_connections 2024-03-26T17:36:26.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.317+0000 7f8347a7e640 1 -- 172.21.15.105:0/2664111744 wait complete. 2024-03-26T17:36:26.325 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 -- 172.21.15.105:0/3720241369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 msgr2=0x7f26f8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 --2- 172.21.15.105:0/3720241369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 0x7f26f8100be0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009900 tx=0x7f26e802bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 -- 172.21.15.105:0/3720241369 shutdown_connections 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 --2- 172.21.15.105:0/3720241369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f8102370 0x7f26f8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 --2- 172.21.15.105:0/3720241369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f81019b0 0x7f26f8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 --2- 172.21.15.105:0/3720241369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 0x7f26f8100be0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 -- 172.21.15.105:0/3720241369 >> 172.21.15.105:0/3720241369 conn(0x7f26f80fbd50 msgr2=0x7f26f80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 -- 172.21.15.105:0/3720241369 shutdown_connections 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.437+0000 7f26fe034640 1 -- 172.21.15.105:0/3720241369 wait complete. 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26fe034640 1 Processor -- start 2024-03-26T17:36:26.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26fe034640 1 -- start start 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26fe034640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 0x7f26f810b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26fe034640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f81019b0 0x7f26f8109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26fe034640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f8102370 0x7f26f8109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26fe034640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26f810d410 con 0x7f26f81007c0 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26fe034640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26f810d290 con 0x7f26f81019b0 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26fe034640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26f810d590 con 0x7f26f8102370 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 0x7f26f810b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 0x7f26f810b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49208/0 (socket says 172.21.15.105:49208) 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 -- 172.21.15.105:0/2671717070 learned_addr learned my addr 172.21.15.105:0/2671717070 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f7fff640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f8102370 0x7f26f8109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f6ffd640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f81019b0 0x7f26f8109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 -- 172.21.15.105:0/2671717070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f8102370 msgr2=0x7f26f8109cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:26.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f8102370 0x7f26f8109cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 -- 172.21.15.105:0/2671717070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f81019b0 msgr2=0x7f26f8109780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f81019b0 0x7f26f8109780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 -- 172.21.15.105:0/2671717070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f810a550 con 0x7f26f81007c0 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.438+0000 7f26f77fe640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 0x7f26f810b140 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f26e8000c00 tx=0x7f26e8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.439+0000 7f26f4ff9640 1 -- 172.21.15.105:0/2671717070 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8043020 con 0x7f26f81007c0 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.439+0000 7f26f4ff9640 1 -- 172.21.15.105:0/2671717070 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26e8004350 con 0x7f26f81007c0 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.439+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f81a1c80 con 0x7f26f81007c0 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.439+0000 7f26f4ff9640 1 -- 172.21.15.105:0/2671717070 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8003b80 con 0x7f26f81007c0 2024-03-26T17:36:26.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.439+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f81a21d0 con 0x7f26f81007c0 2024-03-26T17:36:26.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.440+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f8107c50 con 0x7f26f81007c0 2024-03-26T17:36:26.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.441+0000 7f26f4ff9640 1 -- 172.21.15.105:0/2671717070 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f26e8046020 con 0x7f26f81007c0 2024-03-26T17:36:26.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.441+0000 7f26f4ff9640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26e00a2f20 0x7f26e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:26.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.441+0000 7f26f6ffd640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26e00a2f20 0x7f26e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:26.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.441+0000 7f26f4ff9640 1 -- 172.21.15.105:0/2671717070 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f26e80f13d0 con 0x7f26f81007c0 2024-03-26T17:36:26.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.442+0000 7f26f6ffd640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26e00a2f20 0x7f26e00a53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26f810aeb0 tx=0x7f26ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:26.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.446+0000 7f26f4ff9640 1 -- 172.21.15.105:0/2671717070 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26e804d020 con 0x7f26f81007c0 2024-03-26T17:36:26.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T17:36:26.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: Cluster is now healthy 2024-03-26T17:36:26.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T17:36:26.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1989055880' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T17:36:26.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T17:36:26.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:26.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-26T17:36:26.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-26T17:36:26.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:26 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2664111744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: Cluster is now healthy 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1989055880' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-26T17:36:26.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:26 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2664111744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T17:36:26.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:26.666+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f26f802d460 con 0x7f26f81007c0 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: Cluster is now healthy 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1989055880' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-26T17:36:26.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:26 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2664111744' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T17:36:27.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.321+0000 7f26f4ff9640 1 -- 172.21.15.105:0/2671717070 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f26e80be350 con 0x7f26f81007c0 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26e00a2f20 msgr2=0x7f26e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26e00a2f20 0x7f26e00a53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26f810aeb0 tx=0x7f26ec005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 msgr2=0x7f26f810b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 0x7f26f810b140 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f26e8000c00 tx=0x7f26e8005dc0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 shutdown_connections 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26e00a2f20 0x7f26e00a53e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26f8102370 0x7f26f8109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26f81019b0 0x7f26f8109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 --2- 172.21.15.105:0/2671717070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26f81007c0 0x7f26f810b140 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 >> 172.21.15.105:0/2671717070 conn(0x7f26f80fbd50 msgr2=0x7f26f8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 shutdown_connections 2024-03-26T17:36:27.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.323+0000 7f26fe034640 1 -- 172.21.15.105:0/2671717070 wait complete. 2024-03-26T17:36:27.332 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-26T17:36:27.332 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-26T17:36:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 -- 172.21.15.105:0/2947516858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 msgr2=0x7fc730101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 --2- 172.21.15.105:0/2947516858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 0x7fc730101cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc724009900 tx=0x7fc72402bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 -- 172.21.15.105:0/2947516858 shutdown_connections 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 --2- 172.21.15.105:0/2947516858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc730103480 0x7fc730109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 --2- 172.21.15.105:0/2947516858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc730102ac0 0x7fc730102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 --2- 172.21.15.105:0/2947516858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 0x7fc730101cf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 -- 172.21.15.105:0/2947516858 >> 172.21.15.105:0/2947516858 conn(0x7fc7300fce60 msgr2=0x7fc7300ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 -- 172.21.15.105:0/2947516858 shutdown_connections 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 -- 172.21.15.105:0/2947516858 wait complete. 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 Processor -- start 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.445+0000 7fc736a87640 1 -- start start 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc736a87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 0x7fc730197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:27.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc736a87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc730102ac0 0x7fc7301980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc736a87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc730103480 0x7fc73019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc736a87640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc73010d450 con 0x7fc7301018d0 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc736a87640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc73010d2d0 con 0x7fc730103480 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc736a87640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc73010d5d0 con 0x7fc730102ac0 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 0x7fc730197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 0x7fc730197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49228/0 (socket says 172.21.15.105:49228) 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 -- 172.21.15.105:0/4074992035 learned_addr learned my addr 172.21.15.105:0/4074992035 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc72ffff640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc730102ac0 0x7fc7301980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc735024640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc730103480 0x7fc73019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 -- 172.21.15.105:0/4074992035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc730102ac0 msgr2=0x7fc7301980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc730102ac0 0x7fc7301980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 -- 172.21.15.105:0/4074992035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc730103480 msgr2=0x7fc73019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc730103480 0x7fc73019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 -- 172.21.15.105:0/4074992035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73019f790 con 0x7fc7301018d0 2024-03-26T17:36:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc734823640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 0x7fc730197b70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc724000c00 tx=0x7fc724005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc72dffb640 1 -- 172.21.15.105:0/4074992035 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724043020 con 0x7fc7301018d0 2024-03-26T17:36:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc72dffb640 1 -- 172.21.15.105:0/4074992035 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7240043d0 con 0x7fc7301018d0 2024-03-26T17:36:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73019fa20 con 0x7fc7301018d0 2024-03-26T17:36:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc72dffb640 1 -- 172.21.15.105:0/4074992035 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724003b40 con 0x7fc7301018d0 2024-03-26T17:36:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.446+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73019ffc0 con 0x7fc7301018d0 2024-03-26T17:36:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.447+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc730108c90 con 0x7fc7301018d0 2024-03-26T17:36:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.448+0000 7fc72dffb640 1 -- 172.21.15.105:0/4074992035 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7fc724046020 con 0x7fc7301018d0 2024-03-26T17:36:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.449+0000 7fc72dffb640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7200a2fa0 0x7fc7200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.449+0000 7fc72ffff640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7200a2fa0 0x7fc7200a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.449+0000 7fc72dffb640 1 -- 172.21.15.105:0/4074992035 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc7240f13f0 con 0x7fc7301018d0 2024-03-26T17:36:27.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.449+0000 7fc72ffff640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7200a2fa0 0x7fc7200a5460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc730199080 tx=0x7fc71c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:27.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.453+0000 7fc72dffb640 1 -- 172.21.15.105:0/4074992035 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc72404d020 con 0x7fc7301018d0 2024-03-26T17:36:27.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:27 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2664111744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T17:36:27.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:27 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] up:active 2024-03-26T17:36:27.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:27 smithi144 ceph-mon[36095]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T17:36:27.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:27 smithi144 ceph-mon[36095]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-26T17:36:27.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:27 smithi144 ceph-mon[36095]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-26T17:36:27.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:27 smithi144 ceph-mon[36095]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-26T17:36:27.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:27 smithi144 ceph-mon[36095]: pgmap v285: 97 pgs: 13 creating+peering, 80 active+clean, 4 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 7 op/s 2024-03-26T17:36:27.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:27 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2671717070' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T17:36:27.642 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2664111744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T17:36:27.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] up:active 2024-03-26T17:36:27.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-mon[32280]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T17:36:27.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-mon[32280]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-26T17:36:27.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-mon[32280]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-26T17:36:27.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-mon[32280]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-26T17:36:27.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-mon[32280]: pgmap v285: 97 pgs: 13 creating+peering, 80 active+clean, 4 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 7 op/s 2024-03-26T17:36:27.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2671717070' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T17:36:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:27.654+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc730061d00 con 0x7fc7200a2fa0 2024-03-26T17:36:27.660 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2664111744' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T17:36:27.660 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] up:active 2024-03-26T17:36:27.660 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-mon[35636]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T17:36:27.660 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-mon[35636]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-26T17:36:27.661 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-mon[35636]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-26T17:36:27.661 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-mon[35636]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-26T17:36:27.661 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-mon[35636]: pgmap v285: 97 pgs: 13 creating+peering, 80 active+clean, 4 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 7 op/s 2024-03-26T17:36:27.661 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2671717070' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T17:36:27.916 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:36:27.666+0000 7f463e5f8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-26T17:36:27.916 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:36:27.666+0000 7f463e5f8700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-26T17:36:27.916 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:27 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:36:27.666+0000 7f463e5f8700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-26T17:36:28.143 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:36:27.658+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:36:28.143 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:36:27.658+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:36:28.143 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:36:27.658+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:36:28.143 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:36:27.658+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:36:28.144 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:36:27 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:36:27.658+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:36:28.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.323+0000 7fc72dffb640 1 -- 172.21.15.105:0/4074992035 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc730061d00 con 0x7fc7200a2fa0 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7200a2fa0 msgr2=0x7fc7200a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7200a2fa0 0x7fc7200a5460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc730199080 tx=0x7fc71c005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 msgr2=0x7fc730197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 0x7fc730197b70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc724000c00 tx=0x7fc724005e60 comp rx=0 tx=0).stop 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 shutdown_connections 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc7200a2fa0 0x7fc7200a5460 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc730103480 0x7fc73019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc730102ac0 0x7fc7301980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 --2- 172.21.15.105:0/4074992035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7301018d0 0x7fc730197b70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 >> 172.21.15.105:0/4074992035 conn(0x7fc7300fce60 msgr2=0x7fc730105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 shutdown_connections 2024-03-26T17:36:28.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.326+0000 7fc736a87640 1 -- 172.21.15.105:0/4074992035 wait complete. 2024-03-26T17:36:28.333 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.449+0000 7f3550716640 1 -- 172.21.15.105:0/761139126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 msgr2=0x7f35481070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.449+0000 7f3550716640 1 --2- 172.21.15.105:0/761139126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 0x7f35481070e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3538009900 tx=0x7f353802bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 -- 172.21.15.105:0/761139126 shutdown_connections 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 --2- 172.21.15.105:0/761139126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3548107620 0x7f3548109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 --2- 172.21.15.105:0/761139126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 0x7f35481070e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 --2- 172.21.15.105:0/761139126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35481023c0 0x7f35481047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 -- 172.21.15.105:0/761139126 >> 172.21.15.105:0/761139126 conn(0x7f35480fbd70 msgr2=0x7f35480fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 -- 172.21.15.105:0/761139126 shutdown_connections 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 -- 172.21.15.105:0/761139126 wait complete. 2024-03-26T17:36:28.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 Processor -- start 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 -- start start 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35481023c0 0x7f3548197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 0x7f3548198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3548107620 0x7f354819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f354810d410 con 0x7f3548104cf0 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f354810d290 con 0x7f35481023c0 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f3550716640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f354810d590 con 0x7f3548107620 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f354dcb1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 0x7f3548198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f354dcb1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 0x7f3548198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49258/0 (socket says 172.21.15.105:49258) 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f354dcb1640 1 -- 172.21.15.105:0/54256926 learned_addr learned my addr 172.21.15.105:0/54256926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f354ecb3640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3548107620 0x7f354819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.450+0000 7f354e4b2640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35481023c0 0x7f3548197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f354dcb1640 1 -- 172.21.15.105:0/54256926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3548107620 msgr2=0x7f354819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f354dcb1640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3548107620 0x7f354819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f354dcb1640 1 -- 172.21.15.105:0/54256926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35481023c0 msgr2=0x7f3548197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f354dcb1640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35481023c0 0x7f3548197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f354dcb1640 1 -- 172.21.15.105:0/54256926 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354819f760 con 0x7f3548104cf0 2024-03-26T17:36:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f354dcb1640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 0x7f3548198080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3538000c00 tx=0x7f3538005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f353f7fe640 1 -- 172.21.15.105:0/54256926 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538043020 con 0x7f3548104cf0 2024-03-26T17:36:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f353f7fe640 1 -- 172.21.15.105:0/54256926 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3538004350 con 0x7f3548104cf0 2024-03-26T17:36:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354819f9f0 con 0x7f3548104cf0 2024-03-26T17:36:28.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f353f7fe640 1 -- 172.21.15.105:0/54256926 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538003b80 con 0x7f3548104cf0 2024-03-26T17:36:28.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.451+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354819ff90 con 0x7f3548104cf0 2024-03-26T17:36:28.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.452+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35480fd860 con 0x7f3548104cf0 2024-03-26T17:36:28.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.453+0000 7f353f7fe640 1 -- 172.21.15.105:0/54256926 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f3538046020 con 0x7f3548104cf0 2024-03-26T17:36:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.454+0000 7f353f7fe640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35340a2fa0 0x7f35340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.454+0000 7f354e4b2640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35340a2fa0 0x7f35340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.454+0000 7f353f7fe640 1 -- 172.21.15.105:0/54256926 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f35380f1740 con 0x7f3548104cf0 2024-03-26T17:36:28.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.454+0000 7f354e4b2640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35340a2fa0 0x7f35340a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3540009bd0 tx=0x7f3540005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:28.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.457+0000 7f353f7fe640 1 -- 172.21.15.105:0/54256926 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f353804d020 con 0x7f3548104cf0 2024-03-26T17:36:28.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2671717070' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T17:36:28.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] up:active 2024-03-26T17:36:28.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T17:36:28.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-26T17:36:28.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-26T17:36:28.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-26T17:36:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] up:active 2024-03-26T17:36:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T17:36:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T17:36:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: Cluster is now healthy 2024-03-26T17:36:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-03-26T17:36:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-26T17:36:28.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:28 smithi144 ceph-mon[36095]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-26T17:36:28.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2671717070' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T17:36:28.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] up:active 2024-03-26T17:36:28.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T17:36:28.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-26T17:36:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-26T17:36:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-26T17:36:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] up:active 2024-03-26T17:36:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T17:36:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T17:36:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: Cluster is now healthy 2024-03-26T17:36:28.644 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-03-26T17:36:28.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-26T17:36:28.645 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:28 smithi105 ceph-mon[32280]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-26T17:36:28.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.664+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3548061d00 con 0x7f35340a2fa0 2024-03-26T17:36:28.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2671717070' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T17:36:28.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] up:active 2024-03-26T17:36:28.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-26T17:36:28.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] up:active 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: Cluster is now healthy 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-03-26T17:36:28.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:28 smithi135 ceph-mon[35636]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-03-26T17:36:28.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.689+0000 7f353f7fe640 1 -- 172.21.15.105:0/54256926 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3548061d00 con 0x7f35340a2fa0 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35340a2fa0 msgr2=0x7f35340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35340a2fa0 0x7f35340a5460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3540009bd0 tx=0x7f3540005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 msgr2=0x7f3548198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 0x7f3548198080 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3538000c00 tx=0x7f3538005dc0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 shutdown_connections 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35340a2fa0 0x7f35340a5460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3548107620 0x7f354819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3548104cf0 0x7f3548198080 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 --2- 172.21.15.105:0/54256926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35481023c0 0x7f3548197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 >> 172.21.15.105:0/54256926 conn(0x7f35480fbd70 msgr2=0x7f3548102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:28.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 shutdown_connections 2024-03-26T17:36:28.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.691+0000 7f3550716640 1 -- 172.21.15.105:0/54256926 wait complete. 2024-03-26T17:36:28.696 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-26T17:36:28.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 -- 172.21.15.105:0/2521108846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c107620 msgr2=0x7f641c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 --2- 172.21.15.105:0/2521108846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c107620 0x7f641c109a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6414009ca0 tx=0x7f641402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 -- 172.21.15.105:0/2521108846 shutdown_connections 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 --2- 172.21.15.105:0/2521108846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c107620 0x7f641c109a10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 --2- 172.21.15.105:0/2521108846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f641c104cf0 0x7f641c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 --2- 172.21.15.105:0/2521108846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641c1023c0 0x7f641c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 -- 172.21.15.105:0/2521108846 >> 172.21.15.105:0/2521108846 conn(0x7f641c0fbd70 msgr2=0x7f641c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 -- 172.21.15.105:0/2521108846 shutdown_connections 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 -- 172.21.15.105:0/2521108846 wait complete. 2024-03-26T17:36:28.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 Processor -- start 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.810+0000 7f6421977640 1 -- start start 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6421977640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f641c1023c0 0x7f641c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6421977640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c104cf0 0x7f641c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6421977640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641c107620 0x7f641c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6421977640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f641c10d430 con 0x7f641c104cf0 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6421977640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f641c10d2b0 con 0x7f641c1023c0 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6421977640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f641c10d5b0 con 0x7f641c107620 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c104cf0 0x7f641c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c104cf0 0x7f641c198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49286/0 (socket says 172.21.15.105:49286) 2024-03-26T17:36:28.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 -- 172.21.15.105:0/1248420625 learned_addr learned my addr 172.21.15.105:0/1248420625 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f641affd640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f641c1023c0 0x7f641c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f641b7fe640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641c107620 0x7f641c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 -- 172.21.15.105:0/1248420625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641c107620 msgr2=0x7f641c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641c107620 0x7f641c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 -- 172.21.15.105:0/1248420625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f641c1023c0 msgr2=0x7f641c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f641c1023c0 0x7f641c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 -- 172.21.15.105:0/1248420625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c19f710 con 0x7f641c104cf0 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f641b7fe640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641c107620 0x7f641c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6413fff640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c104cf0 0x7f641c198030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f640400e9c0 tx=0x7f640400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6420975640 1 -- 172.21.15.105:0/1248420625 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640400cd80 con 0x7f641c104cf0 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6420975640 1 -- 172.21.15.105:0/1248420625 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64040054d0 con 0x7f641c104cf0 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c19fa00 con 0x7f641c104cf0 2024-03-26T17:36:28.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6420975640 1 -- 172.21.15.105:0/1248420625 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404010430 con 0x7f641c104cf0 2024-03-26T17:36:28.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.811+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c19ff50 con 0x7f641c104cf0 2024-03-26T17:36:28.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.812+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641c0fd860 con 0x7f641c104cf0 2024-03-26T17:36:28.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.813+0000 7f6420975640 1 -- 172.21.15.105:0/1248420625 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f6404004fd0 con 0x7f641c104cf0 2024-03-26T17:36:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.814+0000 7f6420975640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64080a2f20 0x7f64080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.814+0000 7f641affd640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64080a2f20 0x7f64080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.814+0000 7f6420975640 1 -- 172.21.15.105:0/1248420625 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f64040cf700 con 0x7f641c104cf0 2024-03-26T17:36:28.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.814+0000 7f641affd640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64080a2f20 0x7f64080a53e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f640c009bd0 tx=0x7f640c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:28.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:28.818+0000 7f6420975640 1 -- 172.21.15.105:0/1248420625 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f640409c680 con 0x7f641c104cf0 2024-03-26T17:36:29.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.023+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f641c061d00 con 0x7f64080a2f20 2024-03-26T17:36:29.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.028+0000 7f6420975640 1 -- 172.21.15.105:0/1248420625 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f641c061d00 con 0x7f64080a2f20 2024-03-26T17:36:29.022 INFO:teuthology.orchestra.run.smithi105.stdout:/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64080a2f20 msgr2=0x7f64080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64080a2f20 0x7f64080a53e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f640c009bd0 tx=0x7f640c005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c104cf0 msgr2=0x7f641c198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c104cf0 0x7f641c198030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f640400e9c0 tx=0x7f640400ee80 comp rx=0 tx=0).stop 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 shutdown_connections 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64080a2f20 0x7f64080a53e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f641c107620 0x7f641c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f641c104cf0 0x7f641c198030 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 --2- 172.21.15.105:0/1248420625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f641c1023c0 0x7f641c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 >> 172.21.15.105:0/1248420625 conn(0x7f641c0fbd70 msgr2=0x7f641c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 shutdown_connections 2024-03-26T17:36:29.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.030+0000 7f6421977640 1 -- 172.21.15.105:0/1248420625 wait complete. 2024-03-26T17:36:29.035 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-26T17:36:29.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.147+0000 7f1f59b87640 1 -- 172.21.15.105:0/970999960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 msgr2=0x7f1f54101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.147+0000 7f1f59b87640 1 --2- 172.21.15.105:0/970999960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 0x7f1f54101e40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009900 tx=0x7f1f4802bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.147+0000 7f1f59b87640 1 -- 172.21.15.105:0/970999960 shutdown_connections 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.147+0000 7f1f59b87640 1 --2- 172.21.15.105:0/970999960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f54102380 0x7f1f54108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.147+0000 7f1f59b87640 1 --2- 172.21.15.105:0/970999960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 0x7f1f54101e40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.147+0000 7f1f59b87640 1 --2- 172.21.15.105:0/970999960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f541007d0 0x7f1f54100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.147+0000 7f1f59b87640 1 -- 172.21.15.105:0/970999960 >> 172.21.15.105:0/970999960 conn(0x7f1f540fbd60 msgr2=0x7f1f540fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 -- 172.21.15.105:0/970999960 shutdown_connections 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 -- 172.21.15.105:0/970999960 wait complete. 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 Processor -- start 2024-03-26T17:36:29.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 -- start start 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f541007d0 0x7f1f54197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 0x7f1f54198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f54102380 0x7f1f5419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410c300 con 0x7f1f541019c0 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410c180 con 0x7f1f541007d0 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f59b87640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410c480 con 0x7f1f54102380 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 0x7f1f54198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 0x7f1f54198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49302/0 (socket says 172.21.15.105:49302) 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f52ffd640 1 -- 172.21.15.105:0/2252044346 learned_addr learned my addr 172.21.15.105:0/2252044346 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.148+0000 7f1f537fe640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f541007d0 0x7f1f54197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f53fff640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f54102380 0x7f1f5419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f52ffd640 1 -- 172.21.15.105:0/2252044346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f54102380 msgr2=0x7f1f5419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f52ffd640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f54102380 0x7f1f5419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f52ffd640 1 -- 172.21.15.105:0/2252044346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f541007d0 msgr2=0x7f1f54197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f52ffd640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f541007d0 0x7f1f54197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f52ffd640 1 -- 172.21.15.105:0/2252044346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5419f740 con 0x7f1f541019c0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f53fff640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f54102380 0x7f1f5419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f537fe640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f541007d0 0x7f1f54197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f52ffd640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 0x7f1f54198060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1f48006010 tx=0x7f1f48005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f50ff9640 1 -- 172.21.15.105:0/2252044346 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48043020 con 0x7f1f541019c0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5419f9d0 con 0x7f1f541019c0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f50ff9640 1 -- 172.21.15.105:0/2252044346 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f480043d0 con 0x7f1f541019c0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f50ff9640 1 -- 172.21.15.105:0/2252044346 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48003b80 con 0x7f1f541019c0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.149+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5419ff70 con 0x7f1f541019c0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.150+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f54068b00 con 0x7f1f541019c0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.151+0000 7f1f50ff9640 1 -- 172.21.15.105:0/2252044346 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) ==== 137855+0+0 (secure 0 0 0) 0x7f1f48046020 con 0x7f1f541019c0 2024-03-26T17:36:29.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.152+0000 7f1f50ff9640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f3c0a2f20 0x7f1f3c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.152+0000 7f1f50ff9640 1 -- 172.21.15.105:0/2252044346 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1f480f1b50 con 0x7f1f541019c0 2024-03-26T17:36:29.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.152+0000 7f1f537fe640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f3c0a2f20 0x7f1f3c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.152+0000 7f1f537fe640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f3c0a2f20 0x7f1f3c0a53e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1f54101820 tx=0x7f1f44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:29.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.155+0000 7f1f50ff9640 1 -- 172.21.15.105:0/2252044346 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f4804d020 con 0x7f1f541019c0 2024-03-26T17:36:29.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.342+0000 7f1f50ff9640 1 -- 172.21.15.105:0/2252044346 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f1f480baa40 con 0x7f1f541019c0 2024-03-26T17:36:29.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.381+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1f5406a5d0 con 0x7f1f3c0a2f20 2024-03-26T17:36:29.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.410+0000 7f1f50ff9640 1 -- 172.21.15.105:0/2252044346 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1f5406a5d0 con 0x7f1f3c0a2f20 2024-03-26T17:36:29.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f3c0a2f20 msgr2=0x7f1f3c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f3c0a2f20 0x7f1f3c0a53e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1f54101820 tx=0x7f1f44005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 msgr2=0x7f1f54198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 0x7f1f54198060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1f48006010 tx=0x7f1f48005e40 comp rx=0 tx=0).stop 2024-03-26T17:36:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 shutdown_connections 2024-03-26T17:36:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f3c0a2f20 0x7f1f3c0a53e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f54102380 0x7f1f5419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f541019c0 0x7f1f54198060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 --2- 172.21.15.105:0/2252044346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f541007d0 0x7f1f54197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 >> 172.21.15.105:0/2252044346 conn(0x7f1f540fbd60 msgr2=0x7f1f54069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:29.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 shutdown_connections 2024-03-26T17:36:29.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.412+0000 7f1f59b87640 1 -- 172.21.15.105:0/2252044346 wait complete. 2024-03-26T17:36:29.417 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-26T17:36:29.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 -- 172.21.15.105:0/511449812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 msgr2=0x7feef0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 --2- 172.21.15.105:0/511449812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 0x7feef0100be0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feeec009900 tx=0x7feeec02bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:29.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 -- 172.21.15.105:0/511449812 shutdown_connections 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 --2- 172.21.15.105:0/511449812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef0102370 0x7feef0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 --2- 172.21.15.105:0/511449812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef01019b0 0x7feef0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 --2- 172.21.15.105:0/511449812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 0x7feef0100be0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 -- 172.21.15.105:0/511449812 >> 172.21.15.105:0/511449812 conn(0x7feef00fbd50 msgr2=0x7feef00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 -- 172.21.15.105:0/511449812 shutdown_connections 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.543+0000 7feef8052640 1 -- 172.21.15.105:0/511449812 wait complete. 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef8052640 1 Processor -- start 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef8052640 1 -- start start 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef8052640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 0x7feef00695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef8052640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef01019b0 0x7feef019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef8052640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef0102370 0x7feef019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef8052640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feef010c2e0 con 0x7feef0102370 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef8052640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feef010c160 con 0x7feef01019b0 2024-03-26T17:36:29.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef8052640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feef010c460 con 0x7feef01007c0 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef65ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef0102370 0x7feef019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef5dee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 0x7feef00695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef5dee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 0x7feef00695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33464/0 (socket says 172.21.15.105:33464) 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef65ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef0102370 0x7feef019eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49314/0 (socket says 172.21.15.105:49314) 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef5dee640 1 -- 172.21.15.105:0/3160319685 learned_addr learned my addr 172.21.15.105:0/3160319685 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.544+0000 7feef55ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef01019b0 0x7feef019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef65ef640 1 -- 172.21.15.105:0/3160319685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 msgr2=0x7feef00695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef65ef640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 0x7feef00695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef65ef640 1 -- 172.21.15.105:0/3160319685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef01019b0 msgr2=0x7feef019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef65ef640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef01019b0 0x7feef019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef65ef640 1 -- 172.21.15.105:0/3160319685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef019f7b0 con 0x7feef0102370 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef55ed640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef01019b0 0x7feef019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef5dee640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 0x7feef00695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef65ef640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef0102370 0x7feef019eef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feee800ca30 tx=0x7feee800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feee6ffd640 1 -- 172.21.15.105:0/3160319685 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8007d90 con 0x7feef0102370 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feee6ffd640 1 -- 172.21.15.105:0/3160319685 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feee8005020 con 0x7feef0102370 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef019faa0 con 0x7feef0102370 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feee6ffd640 1 -- 172.21.15.105:0/3160319685 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee800f690 con 0x7feef0102370 2024-03-26T17:36:29.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.545+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef019fff0 con 0x7feef0102370 2024-03-26T17:36:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.546+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef0107bd0 con 0x7feef0102370 2024-03-26T17:36:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.547+0000 7feee6ffd640 1 -- 172.21.15.105:0/3160319685 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7feee80051d0 con 0x7feef0102370 2024-03-26T17:36:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.547+0000 7feee6ffd640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feedc0a3030 0x7feedc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.547+0000 7feef5dee640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feedc0a3030 0x7feedc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.547+0000 7feee6ffd640 1 -- 172.21.15.105:0/3160319685 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7feee80cf520 con 0x7feef0102370 2024-03-26T17:36:29.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.548+0000 7feef5dee640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feedc0a3030 0x7feedc0a54f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feeec02c840 tx=0x7feeec0095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:29.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.552+0000 7feee6ffd640 1 -- 172.21.15.105:0/3160319685 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feee8014070 con 0x7feef0102370 2024-03-26T17:36:29.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:29 smithi144 ceph-mon[36095]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 14 op/s 2024-03-26T17:36:29.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:29 smithi144 ceph-mon[36095]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:29.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:29 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] up:active 2024-03-26T17:36:29.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:29 smithi144 ceph-mon[36095]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:36:29.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:29 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:36:29.642 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:29 smithi105 ceph-mon[32280]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 14 op/s 2024-03-26T17:36:29.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:29 smithi105 ceph-mon[32280]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:29.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:29 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] up:active 2024-03-26T17:36:29.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:29 smithi105 ceph-mon[32280]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:36:29.643 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:29 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:36:29.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:29 smithi135 ceph-mon[35636]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 14 op/s 2024-03-26T17:36:29.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:29 smithi135 ceph-mon[35636]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:29.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:29 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] up:active 2024-03-26T17:36:29.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:29 smithi135 ceph-mon[35636]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:36:29.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:29 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:36:29.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.818+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7feef0061d00 con 0x7feedc0a3030 2024-03-26T17:36:29.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.823+0000 7feee6ffd640 1 -- 172.21.15.105:0/3160319685 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7feef0061d00 con 0x7feedc0a3030 2024-03-26T17:36:29.817 INFO:teuthology.orchestra.run.smithi105.stdout:/volumes/qa/sv_1/fe42ca20-4377-43d5-81c9-2ec349b9b53e 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feedc0a3030 msgr2=0x7feedc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feedc0a3030 0x7feedc0a54f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feeec02c840 tx=0x7feeec0095e0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef0102370 msgr2=0x7feef019eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef0102370 0x7feef019eef0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feee800ca30 tx=0x7feee800cef0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 shutdown_connections 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feedc0a3030 0x7feedc0a54f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feef0102370 0x7feef019eef0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feef01019b0 0x7feef019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 --2- 172.21.15.105:0/3160319685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef01007c0 0x7feef00695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 >> 172.21.15.105:0/3160319685 conn(0x7feef00fbd50 msgr2=0x7feef0104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:29.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 shutdown_connections 2024-03-26T17:36:29.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.826+0000 7feef8052640 1 -- 172.21.15.105:0/3160319685 wait complete. 2024-03-26T17:36:29.831 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 -- 172.21.15.105:0/1496218904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 msgr2=0x7f22dc108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 --2- 172.21.15.105:0/1496218904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 0x7f22dc108c20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009ca0 tx=0x7f22d402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 -- 172.21.15.105:0/1496218904 shutdown_connections 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 --2- 172.21.15.105:0/1496218904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 0x7f22dc108c20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 --2- 172.21.15.105:0/1496218904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22dc1019d0 0x7f22dc101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 --2- 172.21.15.105:0/1496218904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22dc1007e0 0x7f22dc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 -- 172.21.15.105:0/1496218904 >> 172.21.15.105:0/1496218904 conn(0x7f22dc0fbd50 msgr2=0x7f22dc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 -- 172.21.15.105:0/1496218904 shutdown_connections 2024-03-26T17:36:29.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 -- 172.21.15.105:0/1496218904 wait complete. 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 Processor -- start 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.940+0000 7f22e440c640 1 -- start start 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e440c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22dc1007e0 0x7f22dc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e440c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22dc1019d0 0x7f22dc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e440c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 0x7f22dc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e440c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10c310 con 0x7f22dc102390 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e440c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10c190 con 0x7f22dc1019d0 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e440c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc10c490 con 0x7f22dc1007e0 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e29a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 0x7f22dc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22dc1007e0 0x7f22dc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22dc1007e0 0x7f22dc197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33478/0 (socket says 172.21.15.105:33478) 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e29a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 0x7f22dc19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49332/0 (socket says 172.21.15.105:49332) 2024-03-26T17:36:29.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 -- 172.21.15.105:0/1116710505 learned_addr learned my addr 172.21.15.105:0/1116710505 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e19a7640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22dc1019d0 0x7f22dc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 -- 172.21.15.105:0/1116710505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22dc1019d0 msgr2=0x7f22dc198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22dc1019d0 0x7f22dc198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 -- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 msgr2=0x7f22dc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 0x7f22dc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 -- 172.21.15.105:0/1116710505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc19f730 con 0x7f22dc1007e0 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e29a9640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 0x7f22dc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e21a8640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22dc1007e0 0x7f22dc197b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22d800d950 tx=0x7f22d800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22d37fe640 1 -- 172.21.15.105:0/1116710505 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8017020 con 0x7f22dc1007e0 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc19fa20 con 0x7f22dc1007e0 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22d37fe640 1 -- 172.21.15.105:0/1116710505 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22d8009c30 con 0x7f22dc1007e0 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22d37fe640 1 -- 172.21.15.105:0/1116710505 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d800f620 con 0x7f22dc1007e0 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.941+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc19ffc0 con 0x7f22dc1007e0 2024-03-26T17:36:29.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.942+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc1a01f0 con 0x7f22dc1007e0 2024-03-26T17:36:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.943+0000 7f22d37fe640 1 -- 172.21.15.105:0/1116710505 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f22d801a020 con 0x7f22dc1007e0 2024-03-26T17:36:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.944+0000 7f22d37fe640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22c80a2fb0 0x7f22c80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.944+0000 7f22e19a7640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22c80a2fb0 0x7f22c80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.944+0000 7f22d37fe640 1 -- 172.21.15.105:0/1116710505 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f22d80cf4d0 con 0x7f22dc1007e0 2024-03-26T17:36:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.944+0000 7f22e19a7640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22c80a2fb0 0x7f22c80a5470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f22dc199020 tx=0x7f22cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:29.948+0000 7f22d37fe640 1 -- 172.21.15.105:0/1116710505 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f22d8021020 con 0x7f22dc1007e0 2024-03-26T17:36:30.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.165+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22dc1a03c0 con 0x7f22dc1007e0 2024-03-26T17:36:30.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.166+0000 7f22d37fe640 1 -- 172.21.15.105:0/1116710505 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7f22d809c250 con 0x7f22dc1007e0 2024-03-26T17:36:30.160 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:30.160 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:36:29.308004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:36:30.161 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 26 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22c80a2fb0 msgr2=0x7f22c80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22c80a2fb0 0x7f22c80a5470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f22dc199020 tx=0x7f22cc005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22dc1007e0 msgr2=0x7f22dc197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22dc1007e0 0x7f22dc197b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22d800d950 tx=0x7f22d800de10 comp rx=0 tx=0).stop 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 shutdown_connections 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22c80a2fb0 0x7f22c80a5470 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22dc102390 0x7f22dc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22dc1019d0 0x7f22dc198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 --2- 172.21.15.105:0/1116710505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22dc1007e0 0x7f22dc197b10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 >> 172.21.15.105:0/1116710505 conn(0x7f22dc0fbd50 msgr2=0x7f22dc068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:30.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 shutdown_connections 2024-03-26T17:36:30.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.169+0000 7f22e440c640 1 -- 172.21.15.105:0/1116710505 wait complete. 2024-03-26T17:36:30.174 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 -- 172.21.15.105:0/318582567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c1056b0 msgr2=0x7f909c10bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 --2- 172.21.15.105:0/318582567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c1056b0 0x7f909c10bf40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f908c009ca0 tx=0x7f908c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 -- 172.21.15.105:0/318582567 shutdown_connections 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 --2- 172.21.15.105:0/318582567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c1056b0 0x7f909c10bf40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 --2- 172.21.15.105:0/318582567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c104cf0 0x7f909c105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 --2- 172.21.15.105:0/318582567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c103b00 0x7f909c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 -- 172.21.15.105:0/318582567 >> 172.21.15.105:0/318582567 conn(0x7f909c0ff070 msgr2=0x7f909c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 -- 172.21.15.105:0/318582567 shutdown_connections 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 -- 172.21.15.105:0/318582567 wait complete. 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 Processor -- start 2024-03-26T17:36:30.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 -- start start 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.284+0000 7f90a3d97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c103b00 0x7f909c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a3d97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c104cf0 0x7f909c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a3d97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c1056b0 0x7f909c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a3d97640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f909c10f630 con 0x7f909c103b00 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a3d97640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f909c10f4b0 con 0x7f909c1056b0 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a3d97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f909c10f7b0 con 0x7f909c104cf0 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c103b00 0x7f909c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c103b00 0x7f909c199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49356/0 (socket says 172.21.15.105:49356) 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 -- 172.21.15.105:0/1299831588 learned_addr learned my addr 172.21.15.105:0/1299831588 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a2334640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c1056b0 0x7f909c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1332640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c104cf0 0x7f909c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 -- 172.21.15.105:0/1299831588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c104cf0 msgr2=0x7f909c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c104cf0 0x7f909c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 -- 172.21.15.105:0/1299831588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c1056b0 msgr2=0x7f909c1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c1056b0 0x7f909c1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 -- 172.21.15.105:0/1299831588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c1a1910 con 0x7f909c103b00 2024-03-26T17:36:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a1b33640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c103b00 0x7f909c199d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f909400e9c0 tx=0x7f909400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f909affd640 1 -- 172.21.15.105:0/1299831588 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909400cd80 con 0x7f909c103b00 2024-03-26T17:36:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f909affd640 1 -- 172.21.15.105:0/1299831588 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90940035f0 con 0x7f909c103b00 2024-03-26T17:36:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909c1a1c00 con 0x7f909c103b00 2024-03-26T17:36:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f909affd640 1 -- 172.21.15.105:0/1299831588 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094010690 con 0x7f909c103b00 2024-03-26T17:36:30.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.285+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909c1a2150 con 0x7f909c103b00 2024-03-26T17:36:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.287+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909c10aef0 con 0x7f909c103b00 2024-03-26T17:36:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.287+0000 7f909affd640 1 -- 172.21.15.105:0/1299831588 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f90940037a0 con 0x7f909c103b00 2024-03-26T17:36:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.288+0000 7f909affd640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90840a2fb0 0x7f90840a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.288+0000 7f90a1332640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90840a2fb0 0x7f90840a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.288+0000 7f909affd640 1 -- 172.21.15.105:0/1299831588 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f90940cf7e0 con 0x7f909c103b00 2024-03-26T17:36:30.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.288+0000 7f90a1332640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90840a2fb0 0x7f90840a5470 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9090009bd0 tx=0x7f9090005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:30.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.292+0000 7f909affd640 1 -- 172.21.15.105:0/1299831588 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f909409c660 con 0x7f909c103b00 2024-03-26T17:36:30.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.477+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f909c02d3e0 con 0x7f909c103b00 2024-03-26T17:36:30.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.479+0000 7f909affd640 1 -- 172.21.15.105:0/1299831588 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22254 (secure 0 0 0) 0x7f909409bf60 con 0x7f909c103b00 2024-03-26T17:36:30.473 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:30.473 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":85,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:36:23.718044+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90840a2fb0 msgr2=0x7f90840a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90840a2fb0 0x7f90840a5470 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9090009bd0 tx=0x7f9090005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c103b00 msgr2=0x7f909c199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c103b00 0x7f909c199d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f909400e9c0 tx=0x7f909400ee80 comp rx=0 tx=0).stop 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 shutdown_connections 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90840a2fb0 0x7f90840a5470 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f909c1056b0 0x7f909c1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c104cf0 0x7f909c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 --2- 172.21.15.105:0/1299831588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f909c103b00 0x7f909c199d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 >> 172.21.15.105:0/1299831588 conn(0x7f909c0ff070 msgr2=0x7f909c1080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:30.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 shutdown_connections 2024-03-26T17:36:30.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.481+0000 7f90a3d97640 1 -- 172.21.15.105:0/1299831588 wait complete. 2024-03-26T17:36:30.484 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:30 smithi105 ceph-mon[32280]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:30.485 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:30 smithi105 ceph-mon[32280]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T17:36:30.485 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:30 smithi105 ceph-mon[32280]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:30.485 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:30 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1116710505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:30.487 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-26T17:36:30.487 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T17:36:30.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- 172.21.15.105:0/3550226548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df4102390 msgr2=0x7f7df4108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/3550226548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df4102390 0x7f7df4108c20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009900 tx=0x7f7de402bc30 comp rx=0 tx=0).stop 2024-03-26T17:36:30.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- 172.21.15.105:0/3550226548 shutdown_connections 2024-03-26T17:36:30.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/3550226548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df4102390 0x7f7df4108c20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/3550226548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df41019d0 0x7f7df4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/3550226548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7df41007e0 0x7f7df4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- 172.21.15.105:0/3550226548 >> 172.21.15.105:0/3550226548 conn(0x7f7df40fbd50 msgr2=0x7f7df40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:30.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- 172.21.15.105:0/3550226548 shutdown_connections 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- 172.21.15.105:0/3550226548 wait complete. 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 Processor -- start 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- start start 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df41007e0 0x7f7df40695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7df41019d0 0x7f7df419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df4102390 0x7f7df419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7df410c310 con 0x7f7df41007e0 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7df410c190 con 0x7f7df41019d0 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.596+0000 7f7dfaa73640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7df410c490 con 0x7f7df4102390 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df41007e0 0x7f7df40695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df41007e0 0x7f7df40695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49378/0 (socket says 172.21.15.105:49378) 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 -- 172.21.15.105:0/73514889 learned_addr learned my addr 172.21.15.105:0/73514889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df3fff640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7df41019d0 0x7f7df419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df9010640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df4102390 0x7f7df419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 -- 172.21.15.105:0/73514889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df4102390 msgr2=0x7f7df419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df4102390 0x7f7df419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 -- 172.21.15.105:0/73514889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7df41019d0 msgr2=0x7f7df419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7df41019d0 0x7f7df419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 -- 172.21.15.105:0/73514889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df419f6f0 con 0x7f7df41007e0 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df9010640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df4102390 0x7f7df419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df880f640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df41007e0 0x7f7df40695f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009d20 tx=0x7f7dec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df1ffb640 1 -- 172.21.15.105:0/73514889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec010630 con 0x7f7df41007e0 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df1ffb640 1 -- 172.21.15.105:0/73514889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dec010c80 con 0x7f7df41007e0 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7df1ffb640 1 -- 172.21.15.105:0/73514889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec018d70 con 0x7f7df41007e0 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df419f9e0 con 0x7f7df41007e0 2024-03-26T17:36:30.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.597+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df419ff30 con 0x7f7df41007e0 2024-03-26T17:36:30.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.598+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df4107b00 con 0x7f7df41007e0 2024-03-26T17:36:30.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.599+0000 7f7df1ffb640 1 -- 172.21.15.105:0/73514889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f7dec01f020 con 0x7f7df41007e0 2024-03-26T17:36:30.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.600+0000 7f7df1ffb640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ddc0a2fb0 0x7f7ddc0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.600+0000 7f7df1ffb640 1 -- 172.21.15.105:0/73514889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7dec0cf8d0 con 0x7f7df41007e0 2024-03-26T17:36:30.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.600+0000 7f7df3fff640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ddc0a2fb0 0x7f7ddc0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.600+0000 7f7df3fff640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ddc0a2fb0 0x7f7ddc0a5470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7df4069360 tx=0x7f7de8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:30.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.604+0000 7f7df1ffb640 1 -- 172.21.15.105:0/73514889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7dec09c750 con 0x7f7df41007e0 2024-03-26T17:36:30.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:30 smithi144 ceph-mon[36095]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:30.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:30 smithi144 ceph-mon[36095]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T17:36:30.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:30 smithi144 ceph-mon[36095]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:30.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:30 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1116710505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:30 smithi135 ceph-mon[35636]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:30 smithi135 ceph-mon[35636]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T17:36:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:30 smithi135 ceph-mon[35636]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:30.667 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:30 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1116710505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:36:30.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.797+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7df4061d00 con 0x7f7ddc0a2fb0 2024-03-26T17:36:30.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.805+0000 7f7df1ffb640 1 -- 172.21.15.105:0/73514889 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258765 (secure 0 0 0) 0x7f7df4061d00 con 0x7f7ddc0a2fb0 2024-03-26T17:36:30.802 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-26T17:36:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ddc0a2fb0 msgr2=0x7f7ddc0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ddc0a2fb0 0x7f7ddc0a5470 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7df4069360 tx=0x7f7de8005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df41007e0 msgr2=0x7f7df40695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df41007e0 0x7f7df40695f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009d20 tx=0x7f7dec00c6a0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 shutdown_connections 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ddc0a2fb0 0x7f7ddc0a5470 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7df4102390 0x7f7df419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7df41019d0 0x7f7df419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 --2- 172.21.15.105:0/73514889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7df41007e0 0x7f7df40695f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 >> 172.21.15.105:0/73514889 conn(0x7f7df40fbd50 msgr2=0x7f7df4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 shutdown_connections 2024-03-26T17:36:30.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.808+0000 7f7dfaa73640 1 -- 172.21.15.105:0/73514889 wait complete. 2024-03-26T17:36:30.813 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-03-26T17:36:30.583327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595624,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":145,"num_read_kb":124,"num_write":321,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":267,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366036,"kb_used_data":12708,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575788,"statfs":{"total":1151940427776,"available":1151565606912,"internally_reserved":0,"allocated":13012992,"data_stored":7044567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5256,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":140,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001818"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.327606+0000","last_change":"2024-03-26T17:36:22.743815+0000","last_active":"2024-03-26T17:36:25.327606+0000","last_peered":"2024-03-26T17:36:25.327606+0000","last_clean":"2024-03-26T17:36:25.327606+0000","last_became_active":"2024-03-26T17:36:22.743446+0000","last_became_peered":"2024-03-26T17:36:22.743446+0000","last_unstale":"2024-03-26T17:36:25.327606+0000","last_undegraded":"2024-03-26T17:36:25.327606+0000","last_fullsized":"2024-03-26T17:36:25.327606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:16:56.773849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325545+0000","last_change":"2024-03-26T17:36:22.740754+0000","last_active":"2024-03-26T17:36:25.325545+0000","last_peered":"2024-03-26T17:36:25.325545+0000","last_clean":"2024-03-26T17:36:25.325545+0000","last_became_active":"2024-03-26T17:36:22.740317+0000","last_became_peered":"2024-03-26T17:36:22.740317+0000","last_unstale":"2024-03-26T17:36:25.325545+0000","last_undegraded":"2024-03-26T17:36:25.325545+0000","last_fullsized":"2024-03-26T17:36:25.325545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:50:45.321653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325516+0000","last_change":"2024-03-26T17:36:22.735079+0000","last_active":"2024-03-26T17:36:25.325516+0000","last_peered":"2024-03-26T17:36:25.325516+0000","last_clean":"2024-03-26T17:36:25.325516+0000","last_became_active":"2024-03-26T17:36:22.734646+0000","last_became_peered":"2024-03-26T17:36:22.734646+0000","last_unstale":"2024-03-26T17:36:25.325516+0000","last_undegraded":"2024-03-26T17:36:25.325516+0000","last_fullsized":"2024-03-26T17:36:25.325516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:41:09.477438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.322876+0000","last_change":"2024-03-26T17:36:22.746170+0000","last_active":"2024-03-26T17:36:25.322876+0000","last_peered":"2024-03-26T17:36:25.322876+0000","last_clean":"2024-03-26T17:36:25.322876+0000","last_became_active":"2024-03-26T17:36:22.744255+0000","last_became_peered":"2024-03-26T17:36:22.744255+0000","last_unstale":"2024-03-26T17:36:25.322876+0000","last_undegraded":"2024-03-26T17:36:25.322876+0000","last_fullsized":"2024-03-26T17:36:25.322876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:54:36.225926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749116+0000","last_change":"2024-03-26T17:36:22.737390+0000","last_active":"2024-03-26T17:36:23.749116+0000","last_peered":"2024-03-26T17:36:23.749116+0000","last_clean":"2024-03-26T17:36:23.749116+0000","last_became_active":"2024-03-26T17:36:22.737038+0000","last_became_peered":"2024-03-26T17:36:22.737038+0000","last_unstale":"2024-03-26T17:36:23.749116+0000","last_undegraded":"2024-03-26T17:36:23.749116+0000","last_fullsized":"2024-03-26T17:36:23.749116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:48.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.955420+0000","last_change":"2024-03-26T17:36:22.748097+0000","last_active":"2024-03-26T17:36:23.955420+0000","last_peered":"2024-03-26T17:36:23.955420+0000","last_clean":"2024-03-26T17:36:23.955420+0000","last_became_active":"2024-03-26T17:36:22.747730+0000","last_became_peered":"2024-03-26T17:36:22.747730+0000","last_unstale":"2024-03-26T17:36:23.955420+0000","last_undegraded":"2024-03-26T17:36:23.955420+0000","last_fullsized":"2024-03-26T17:36:23.955420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:43:01.370199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705500+0000","last_change":"2024-03-26T17:36:22.739587+0000","last_active":"2024-03-26T17:36:24.705500+0000","last_peered":"2024-03-26T17:36:24.705500+0000","last_clean":"2024-03-26T17:36:24.705500+0000","last_became_active":"2024-03-26T17:36:22.735787+0000","last_became_peered":"2024-03-26T17:36:22.735787+0000","last_unstale":"2024-03-26T17:36:24.705500+0000","last_undegraded":"2024-03-26T17:36:24.705500+0000","last_fullsized":"2024-03-26T17:36:24.705500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:41.381153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748525+0000","last_change":"2024-03-26T17:36:22.736716+0000","last_active":"2024-03-26T17:36:23.748525+0000","last_peered":"2024-03-26T17:36:23.748525+0000","last_clean":"2024-03-26T17:36:23.748525+0000","last_became_active":"2024-03-26T17:36:22.736327+0000","last_became_peered":"2024-03-26T17:36:22.736327+0000","last_unstale":"2024-03-26T17:36:23.748525+0000","last_undegraded":"2024-03-26T17:36:23.748525+0000","last_fullsized":"2024-03-26T17:36:23.748525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:10:43.439524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721997+0000","last_change":"2024-03-26T17:36:22.746355+0000","last_active":"2024-03-26T17:36:23.721997+0000","last_peered":"2024-03-26T17:36:23.721997+0000","last_clean":"2024-03-26T17:36:23.721997+0000","last_became_active":"2024-03-26T17:36:22.745363+0000","last_became_peered":"2024-03-26T17:36:22.745363+0000","last_unstale":"2024-03-26T17:36:23.721997+0000","last_undegraded":"2024-03-26T17:36:23.721997+0000","last_fullsized":"2024-03-26T17:36:23.721997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:56:33.528005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.955774+0000","last_change":"2024-03-26T17:36:22.749009+0000","last_active":"2024-03-26T17:36:23.955774+0000","last_peered":"2024-03-26T17:36:23.955774+0000","last_clean":"2024-03-26T17:36:23.955774+0000","last_became_active":"2024-03-26T17:36:22.748627+0000","last_became_peered":"2024-03-26T17:36:22.748627+0000","last_unstale":"2024-03-26T17:36:23.955774+0000","last_undegraded":"2024-03-26T17:36:23.955774+0000","last_fullsized":"2024-03-26T17:36:23.955774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:59:14.812316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705800+0000","last_change":"2024-03-26T17:36:22.741294+0000","last_active":"2024-03-26T17:36:24.705800+0000","last_peered":"2024-03-26T17:36:24.705800+0000","last_clean":"2024-03-26T17:36:24.705800+0000","last_became_active":"2024-03-26T17:36:22.740003+0000","last_became_peered":"2024-03-26T17:36:22.740003+0000","last_unstale":"2024-03-26T17:36:24.705800+0000","last_undegraded":"2024-03-26T17:36:24.705800+0000","last_fullsized":"2024-03-26T17:36:24.705800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:06.305195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.322721+0000","last_change":"2024-03-26T17:36:22.745884+0000","last_active":"2024-03-26T17:36:25.322721+0000","last_peered":"2024-03-26T17:36:25.322721+0000","last_clean":"2024-03-26T17:36:25.322721+0000","last_became_active":"2024-03-26T17:36:22.744168+0000","last_became_peered":"2024-03-26T17:36:22.744168+0000","last_unstale":"2024-03-26T17:36:25.322721+0000","last_undegraded":"2024-03-26T17:36:25.322721+0000","last_fullsized":"2024-03-26T17:36:25.322721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:50.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723833+0000","last_change":"2024-03-26T17:36:22.725956+0000","last_active":"2024-03-26T17:36:23.723833+0000","last_peered":"2024-03-26T17:36:23.723833+0000","last_clean":"2024-03-26T17:36:23.723833+0000","last_became_active":"2024-03-26T17:36:22.725513+0000","last_became_peered":"2024-03-26T17:36:22.725513+0000","last_unstale":"2024-03-26T17:36:23.723833+0000","last_undegraded":"2024-03-26T17:36:23.723833+0000","last_fullsized":"2024-03-26T17:36:23.723833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:30:10.914233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.324324+0000","last_change":"2024-03-26T17:36:22.739757+0000","last_active":"2024-03-26T17:36:25.324324+0000","last_peered":"2024-03-26T17:36:25.324324+0000","last_clean":"2024-03-26T17:36:25.324324+0000","last_became_active":"2024-03-26T17:36:22.739342+0000","last_became_peered":"2024-03-26T17:36:22.739342+0000","last_unstale":"2024-03-26T17:36:25.324324+0000","last_undegraded":"2024-03-26T17:36:25.324324+0000","last_fullsized":"2024-03-26T17:36:25.324324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:28.171325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.324063+0000","last_change":"2024-03-26T17:36:22.742988+0000","last_active":"2024-03-26T17:36:25.324063+0000","last_peered":"2024-03-26T17:36:25.324063+0000","last_clean":"2024-03-26T17:36:25.324063+0000","last_became_active":"2024-03-26T17:36:22.742639+0000","last_became_peered":"2024-03-26T17:36:22.742639+0000","last_unstale":"2024-03-26T17:36:25.324063+0000","last_undegraded":"2024-03-26T17:36:25.324063+0000","last_fullsized":"2024-03-26T17:36:25.324063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:48:44.204889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721484+0000","last_change":"2024-03-26T17:36:22.744530+0000","last_active":"2024-03-26T17:36:23.721484+0000","last_peered":"2024-03-26T17:36:23.721484+0000","last_clean":"2024-03-26T17:36:23.721484+0000","last_became_active":"2024-03-26T17:36:22.743566+0000","last_became_peered":"2024-03-26T17:36:22.743566+0000","last_unstale":"2024-03-26T17:36:23.721484+0000","last_undegraded":"2024-03-26T17:36:23.721484+0000","last_fullsized":"2024-03-26T17:36:23.721484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:42.685326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.327626+0000","last_change":"2024-03-26T17:36:22.745797+0000","last_active":"2024-03-26T17:36:25.327626+0000","last_peered":"2024-03-26T17:36:25.327626+0000","last_clean":"2024-03-26T17:36:25.327626+0000","last_became_active":"2024-03-26T17:36:22.745448+0000","last_became_peered":"2024-03-26T17:36:22.745448+0000","last_unstale":"2024-03-26T17:36:25.327626+0000","last_undegraded":"2024-03-26T17:36:25.327626+0000","last_fullsized":"2024-03-26T17:36:25.327626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:05:29.126564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.746910+0000","last_change":"2024-03-26T17:36:22.749890+0000","last_active":"2024-03-26T17:36:23.746910+0000","last_peered":"2024-03-26T17:36:23.746910+0000","last_clean":"2024-03-26T17:36:23.746910+0000","last_became_active":"2024-03-26T17:36:22.749497+0000","last_became_peered":"2024-03-26T17:36:22.749497+0000","last_unstale":"2024-03-26T17:36:23.746910+0000","last_undegraded":"2024-03-26T17:36:23.746910+0000","last_fullsized":"2024-03-26T17:36:23.746910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:12:55.069659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723160+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:36:23.723160+0000","last_peered":"2024-03-26T17:36:23.723160+0000","last_clean":"2024-03-26T17:36:23.723160+0000","last_became_active":"2024-03-26T17:36:22.740287+0000","last_became_peered":"2024-03-26T17:36:22.740287+0000","last_unstale":"2024-03-26T17:36:23.723160+0000","last_undegraded":"2024-03-26T17:36:23.723160+0000","last_fullsized":"2024-03-26T17:36:23.723160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:13:00.574246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'11","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:27.337297+0000","last_change":"2024-03-26T17:36:21.720809+0000","last_active":"2024-03-26T17:36:27.337297+0000","last_peered":"2024-03-26T17:36:27.337297+0000","last_clean":"2024-03-26T17:36:27.337297+0000","last_became_active":"2024-03-26T17:36:21.720526+0000","last_became_peered":"2024-03-26T17:36:21.720526+0000","last_unstale":"2024-03-26T17:36:27.337297+0000","last_undegraded":"2024-03-26T17:36:27.337297+0000","last_fullsized":"2024-03-26T17:36:27.337297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:48.034328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748996+0000","last_change":"2024-03-26T17:36:22.746245+0000","last_active":"2024-03-26T17:36:23.748996+0000","last_peered":"2024-03-26T17:36:23.748996+0000","last_clean":"2024-03-26T17:36:23.748996+0000","last_became_active":"2024-03-26T17:36:22.745859+0000","last_became_peered":"2024-03-26T17:36:22.745859+0000","last_unstale":"2024-03-26T17:36:23.748996+0000","last_undegraded":"2024-03-26T17:36:23.748996+0000","last_fullsized":"2024-03-26T17:36:23.748996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:39.085668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.787641+0000","last_change":"2024-03-26T17:36:21.723178+0000","last_active":"2024-03-26T17:36:23.787641+0000","last_peered":"2024-03-26T17:36:23.787641+0000","last_clean":"2024-03-26T17:36:23.787641+0000","last_became_active":"2024-03-26T17:36:21.722858+0000","last_became_peered":"2024-03-26T17:36:21.722858+0000","last_unstale":"2024-03-26T17:36:23.787641+0000","last_undegraded":"2024-03-26T17:36:23.787641+0000","last_fullsized":"2024-03-26T17:36:23.787641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:46:13.837672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325317+0000","last_change":"2024-03-26T17:36:22.736231+0000","last_active":"2024-03-26T17:36:25.325317+0000","last_peered":"2024-03-26T17:36:25.325317+0000","last_clean":"2024-03-26T17:36:25.325317+0000","last_became_active":"2024-03-26T17:36:22.735942+0000","last_became_peered":"2024-03-26T17:36:22.735942+0000","last_unstale":"2024-03-26T17:36:25.325317+0000","last_undegraded":"2024-03-26T17:36:25.325317+0000","last_fullsized":"2024-03-26T17:36:25.325317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:49.127382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749399+0000","last_change":"2024-03-26T17:36:21.731369+0000","last_active":"2024-03-26T17:36:23.749399+0000","last_peered":"2024-03-26T17:36:23.749399+0000","last_clean":"2024-03-26T17:36:23.749399+0000","last_became_active":"2024-03-26T17:36:21.731016+0000","last_became_peered":"2024-03-26T17:36:21.731016+0000","last_unstale":"2024-03-26T17:36:23.749399+0000","last_undegraded":"2024-03-26T17:36:23.749399+0000","last_fullsized":"2024-03-26T17:36:23.749399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:37:53.204825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721699+0000","last_change":"2024-03-26T17:36:22.751155+0000","last_active":"2024-03-26T17:36:23.721699+0000","last_peered":"2024-03-26T17:36:23.721699+0000","last_clean":"2024-03-26T17:36:23.721699+0000","last_became_active":"2024-03-26T17:36:22.750704+0000","last_became_peered":"2024-03-26T17:36:22.750704+0000","last_unstale":"2024-03-26T17:36:23.721699+0000","last_undegraded":"2024-03-26T17:36:23.721699+0000","last_fullsized":"2024-03-26T17:36:23.721699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:52:24.116923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749129+0000","last_change":"2024-03-26T17:36:21.726380+0000","last_active":"2024-03-26T17:36:23.749129+0000","last_peered":"2024-03-26T17:36:23.749129+0000","last_clean":"2024-03-26T17:36:23.749129+0000","last_became_active":"2024-03-26T17:36:21.726004+0000","last_became_peered":"2024-03-26T17:36:21.726004+0000","last_unstale":"2024-03-26T17:36:23.749129+0000","last_undegraded":"2024-03-26T17:36:23.749129+0000","last_fullsized":"2024-03-26T17:36:23.749129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:51:23.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909452+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:36:23.909452+0000","last_peered":"2024-03-26T17:36:23.909452+0000","last_clean":"2024-03-26T17:36:23.909452+0000","last_became_active":"2024-03-26T17:36:22.741104+0000","last_became_peered":"2024-03-26T17:36:22.741104+0000","last_unstale":"2024-03-26T17:36:23.909452+0000","last_undegraded":"2024-03-26T17:36:23.909452+0000","last_fullsized":"2024-03-26T17:36:23.909452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:53.683944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.312302+0000","last_change":"2024-03-26T17:36:21.728824+0000","last_active":"2024-03-26T17:36:28.312302+0000","last_peered":"2024-03-26T17:36:28.312302+0000","last_clean":"2024-03-26T17:36:28.312302+0000","last_became_active":"2024-03-26T17:36:21.728467+0000","last_became_peered":"2024-03-26T17:36:21.728467+0000","last_unstale":"2024-03-26T17:36:28.312302+0000","last_undegraded":"2024-03-26T17:36:28.312302+0000","last_fullsized":"2024-03-26T17:36:28.312302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:25:07.842111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749117+0000","last_change":"2024-03-26T17:36:22.740131+0000","last_active":"2024-03-26T17:36:23.749117+0000","last_peered":"2024-03-26T17:36:23.749117+0000","last_clean":"2024-03-26T17:36:23.749117+0000","last_became_active":"2024-03-26T17:36:22.739704+0000","last_became_peered":"2024-03-26T17:36:22.739704+0000","last_unstale":"2024-03-26T17:36:23.749117+0000","last_undegraded":"2024-03-26T17:36:23.749117+0000","last_fullsized":"2024-03-26T17:36:23.749117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:47.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:26.331938+0000","last_change":"2024-03-26T17:36:21.731826+0000","last_active":"2024-03-26T17:36:26.331938+0000","last_peered":"2024-03-26T17:36:26.331938+0000","last_clean":"2024-03-26T17:36:26.331938+0000","last_became_active":"2024-03-26T17:36:21.731546+0000","last_became_peered":"2024-03-26T17:36:21.731546+0000","last_unstale":"2024-03-26T17:36:26.331938+0000","last_undegraded":"2024-03-26T17:36:26.331938+0000","last_fullsized":"2024-03-26T17:36:26.331938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:08:53.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909290+0000","last_change":"2024-03-26T17:36:22.745170+0000","last_active":"2024-03-26T17:36:23.909290+0000","last_peered":"2024-03-26T17:36:23.909290+0000","last_clean":"2024-03-26T17:36:23.909290+0000","last_became_active":"2024-03-26T17:36:22.744798+0000","last_became_peered":"2024-03-26T17:36:22.744798+0000","last_unstale":"2024-03-26T17:36:23.909290+0000","last_undegraded":"2024-03-26T17:36:23.909290+0000","last_fullsized":"2024-03-26T17:36:23.909290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:32:50.418149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:26.332093+0000","last_change":"2024-03-26T17:36:21.721492+0000","last_active":"2024-03-26T17:36:26.332093+0000","last_peered":"2024-03-26T17:36:26.332093+0000","last_clean":"2024-03-26T17:36:26.332093+0000","last_became_active":"2024-03-26T17:36:21.721194+0000","last_became_peered":"2024-03-26T17:36:21.721194+0000","last_unstale":"2024-03-26T17:36:26.332093+0000","last_undegraded":"2024-03-26T17:36:26.332093+0000","last_fullsized":"2024-03-26T17:36:26.332093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:50.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748624+0000","last_change":"2024-03-26T17:36:22.740780+0000","last_active":"2024-03-26T17:36:23.748624+0000","last_peered":"2024-03-26T17:36:23.748624+0000","last_clean":"2024-03-26T17:36:23.748624+0000","last_became_active":"2024-03-26T17:36:22.740311+0000","last_became_peered":"2024-03-26T17:36:22.740311+0000","last_unstale":"2024-03-26T17:36:23.748624+0000","last_undegraded":"2024-03-26T17:36:23.748624+0000","last_fullsized":"2024-03-26T17:36:23.748624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:01:56.018416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.308270+0000","last_change":"2024-03-26T17:36:21.726289+0000","last_active":"2024-03-26T17:36:28.308270+0000","last_peered":"2024-03-26T17:36:28.308270+0000","last_clean":"2024-03-26T17:36:28.308270+0000","last_became_active":"2024-03-26T17:36:21.725898+0000","last_became_peered":"2024-03-26T17:36:21.725898+0000","last_unstale":"2024-03-26T17:36:28.308270+0000","last_undegraded":"2024-03-26T17:36:28.308270+0000","last_fullsized":"2024-03-26T17:36:28.308270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:03.893163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.746847+0000","last_change":"2024-03-26T17:36:22.750632+0000","last_active":"2024-03-26T17:36:23.746847+0000","last_peered":"2024-03-26T17:36:23.746847+0000","last_clean":"2024-03-26T17:36:23.746847+0000","last_became_active":"2024-03-26T17:36:22.750297+0000","last_became_peered":"2024-03-26T17:36:22.750297+0000","last_unstale":"2024-03-26T17:36:23.746847+0000","last_undegraded":"2024-03-26T17:36:23.746847+0000","last_fullsized":"2024-03-26T17:36:23.746847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:43:16.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.324986+0000","last_change":"2024-03-26T17:36:21.721403+0000","last_active":"2024-03-26T17:36:25.324986+0000","last_peered":"2024-03-26T17:36:25.324986+0000","last_clean":"2024-03-26T17:36:25.324986+0000","last_became_active":"2024-03-26T17:36:21.720962+0000","last_became_peered":"2024-03-26T17:36:21.720962+0000","last_unstale":"2024-03-26T17:36:25.324986+0000","last_undegraded":"2024-03-26T17:36:25.324986+0000","last_fullsized":"2024-03-26T17:36:25.324986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:02.683188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.722025+0000","last_change":"2024-03-26T17:36:22.754010+0000","last_active":"2024-03-26T17:36:23.722025+0000","last_peered":"2024-03-26T17:36:23.722025+0000","last_clean":"2024-03-26T17:36:23.722025+0000","last_became_active":"2024-03-26T17:36:22.753534+0000","last_became_peered":"2024-03-26T17:36:22.753534+0000","last_unstale":"2024-03-26T17:36:23.722025+0000","last_undegraded":"2024-03-26T17:36:23.722025+0000","last_fullsized":"2024-03-26T17:36:23.722025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:06.490733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326780+0000","last_change":"2024-03-26T17:36:21.721685+0000","last_active":"2024-03-26T17:36:25.326780+0000","last_peered":"2024-03-26T17:36:25.326780+0000","last_clean":"2024-03-26T17:36:25.326780+0000","last_became_active":"2024-03-26T17:36:21.721387+0000","last_became_peered":"2024-03-26T17:36:21.721387+0000","last_unstale":"2024-03-26T17:36:25.326780+0000","last_undegraded":"2024-03-26T17:36:25.326780+0000","last_fullsized":"2024-03-26T17:36:25.326780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:59.493172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325170+0000","last_change":"2024-03-26T17:36:22.738821+0000","last_active":"2024-03-26T17:36:25.325170+0000","last_peered":"2024-03-26T17:36:25.325170+0000","last_clean":"2024-03-26T17:36:25.325170+0000","last_became_active":"2024-03-26T17:36:22.738476+0000","last_became_peered":"2024-03-26T17:36:22.738476+0000","last_unstale":"2024-03-26T17:36:25.325170+0000","last_undegraded":"2024-03-26T17:36:25.325170+0000","last_fullsized":"2024-03-26T17:36:25.325170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:29:57.568099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:27.338385+0000","last_change":"2024-03-26T17:36:21.724041+0000","last_active":"2024-03-26T17:36:27.338385+0000","last_peered":"2024-03-26T17:36:27.338385+0000","last_clean":"2024-03-26T17:36:27.338385+0000","last_became_active":"2024-03-26T17:36:21.723624+0000","last_became_peered":"2024-03-26T17:36:21.723624+0000","last_unstale":"2024-03-26T17:36:27.338385+0000","last_undegraded":"2024-03-26T17:36:27.338385+0000","last_fullsized":"2024-03-26T17:36:27.338385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:42:24.264844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721771+0000","last_change":"2024-03-26T17:36:22.738592+0000","last_active":"2024-03-26T17:36:23.721771+0000","last_peered":"2024-03-26T17:36:23.721771+0000","last_clean":"2024-03-26T17:36:23.721771+0000","last_became_active":"2024-03-26T17:36:22.738236+0000","last_became_peered":"2024-03-26T17:36:22.738236+0000","last_unstale":"2024-03-26T17:36:23.721771+0000","last_undegraded":"2024-03-26T17:36:23.721771+0000","last_fullsized":"2024-03-26T17:36:23.721771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:56:06.977720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326035+0000","last_change":"2024-03-26T17:36:21.722793+0000","last_active":"2024-03-26T17:36:25.326035+0000","last_peered":"2024-03-26T17:36:25.326035+0000","last_clean":"2024-03-26T17:36:25.326035+0000","last_became_active":"2024-03-26T17:36:21.722482+0000","last_became_peered":"2024-03-26T17:36:21.722482+0000","last_unstale":"2024-03-26T17:36:25.326035+0000","last_undegraded":"2024-03-26T17:36:25.326035+0000","last_fullsized":"2024-03-26T17:36:25.326035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:48:32.107017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749339+0000","last_change":"2024-03-26T17:36:22.746488+0000","last_active":"2024-03-26T17:36:23.749339+0000","last_peered":"2024-03-26T17:36:23.749339+0000","last_clean":"2024-03-26T17:36:23.749339+0000","last_became_active":"2024-03-26T17:36:22.746013+0000","last_became_peered":"2024-03-26T17:36:22.746013+0000","last_unstale":"2024-03-26T17:36:23.749339+0000","last_undegraded":"2024-03-26T17:36:23.749339+0000","last_fullsized":"2024-03-26T17:36:23.749339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:14.792336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326885+0000","last_change":"2024-03-26T17:36:21.724319+0000","last_active":"2024-03-26T17:36:25.326885+0000","last_peered":"2024-03-26T17:36:25.326885+0000","last_clean":"2024-03-26T17:36:25.326885+0000","last_became_active":"2024-03-26T17:36:21.723644+0000","last_became_peered":"2024-03-26T17:36:21.723644+0000","last_unstale":"2024-03-26T17:36:25.326885+0000","last_undegraded":"2024-03-26T17:36:25.326885+0000","last_fullsized":"2024-03-26T17:36:25.326885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:31:01.132797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723695+0000","last_change":"2024-03-26T17:36:22.736535+0000","last_active":"2024-03-26T17:36:23.723695+0000","last_peered":"2024-03-26T17:36:23.723695+0000","last_clean":"2024-03-26T17:36:23.723695+0000","last_became_active":"2024-03-26T17:36:22.736229+0000","last_became_peered":"2024-03-26T17:36:22.736229+0000","last_unstale":"2024-03-26T17:36:23.723695+0000","last_undegraded":"2024-03-26T17:36:23.723695+0000","last_fullsized":"2024-03-26T17:36:23.723695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:03.938180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.316401+0000","last_change":"2024-03-26T17:36:21.726271+0000","last_active":"2024-03-26T17:36:28.316401+0000","last_peered":"2024-03-26T17:36:28.316401+0000","last_clean":"2024-03-26T17:36:28.316401+0000","last_became_active":"2024-03-26T17:36:21.725942+0000","last_became_peered":"2024-03-26T17:36:21.725942+0000","last_unstale":"2024-03-26T17:36:28.316401+0000","last_undegraded":"2024-03-26T17:36:28.316401+0000","last_fullsized":"2024-03-26T17:36:28.316401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:23:16.320106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705681+0000","last_change":"2024-03-26T17:36:22.743480+0000","last_active":"2024-03-26T17:36:24.705681+0000","last_peered":"2024-03-26T17:36:24.705681+0000","last_clean":"2024-03-26T17:36:24.705681+0000","last_became_active":"2024-03-26T17:36:22.742982+0000","last_became_peered":"2024-03-26T17:36:22.742982+0000","last_unstale":"2024-03-26T17:36:24.705681+0000","last_undegraded":"2024-03-26T17:36:24.705681+0000","last_fullsized":"2024-03-26T17:36:24.705681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:47.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.315773+0000","last_change":"2024-03-26T17:36:21.723407+0000","last_active":"2024-03-26T17:36:28.315773+0000","last_peered":"2024-03-26T17:36:28.315773+0000","last_clean":"2024-03-26T17:36:28.315773+0000","last_became_active":"2024-03-26T17:36:21.723086+0000","last_became_peered":"2024-03-26T17:36:21.723086+0000","last_unstale":"2024-03-26T17:36:28.315773+0000","last_undegraded":"2024-03-26T17:36:28.315773+0000","last_fullsized":"2024-03-26T17:36:28.315773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:26:57.886388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705465+0000","last_change":"2024-03-26T17:36:22.740705+0000","last_active":"2024-03-26T17:36:24.705465+0000","last_peered":"2024-03-26T17:36:24.705465+0000","last_clean":"2024-03-26T17:36:24.705465+0000","last_became_active":"2024-03-26T17:36:22.740360+0000","last_became_peered":"2024-03-26T17:36:22.740360+0000","last_unstale":"2024-03-26T17:36:24.705465+0000","last_undegraded":"2024-03-26T17:36:24.705465+0000","last_fullsized":"2024-03-26T17:36:24.705465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:02.879022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325945+0000","last_change":"2024-03-26T17:36:21.731427+0000","last_active":"2024-03-26T17:36:25.325945+0000","last_peered":"2024-03-26T17:36:25.325945+0000","last_clean":"2024-03-26T17:36:25.325945+0000","last_became_active":"2024-03-26T17:36:21.731124+0000","last_became_peered":"2024-03-26T17:36:21.731124+0000","last_unstale":"2024-03-26T17:36:25.325945+0000","last_undegraded":"2024-03-26T17:36:25.325945+0000","last_fullsized":"2024-03-26T17:36:25.325945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:10.084577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909251+0000","last_change":"2024-03-26T17:36:22.743791+0000","last_active":"2024-03-26T17:36:23.909251+0000","last_peered":"2024-03-26T17:36:23.909251+0000","last_clean":"2024-03-26T17:36:23.909251+0000","last_became_active":"2024-03-26T17:36:22.742780+0000","last_became_peered":"2024-03-26T17:36:22.742780+0000","last_unstale":"2024-03-26T17:36:23.909251+0000","last_undegraded":"2024-03-26T17:36:23.909251+0000","last_fullsized":"2024-03-26T17:36:23.909251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:04.387958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749382+0000","last_change":"2024-03-26T17:36:21.724289+0000","last_active":"2024-03-26T17:36:23.749382+0000","last_peered":"2024-03-26T17:36:23.749382+0000","last_clean":"2024-03-26T17:36:23.749382+0000","last_became_active":"2024-03-26T17:36:21.723847+0000","last_became_peered":"2024-03-26T17:36:21.723847+0000","last_unstale":"2024-03-26T17:36:23.749382+0000","last_undegraded":"2024-03-26T17:36:23.749382+0000","last_fullsized":"2024-03-26T17:36:23.749382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:01:59.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326947+0000","last_change":"2024-03-26T17:36:22.748323+0000","last_active":"2024-03-26T17:36:25.326947+0000","last_peered":"2024-03-26T17:36:25.326947+0000","last_clean":"2024-03-26T17:36:25.326947+0000","last_became_active":"2024-03-26T17:36:22.747970+0000","last_became_peered":"2024-03-26T17:36:22.747970+0000","last_unstale":"2024-03-26T17:36:25.326947+0000","last_undegraded":"2024-03-26T17:36:25.326947+0000","last_fullsized":"2024-03-26T17:36:25.326947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:58.886425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.746910+0000","last_change":"2024-03-26T17:36:21.726076+0000","last_active":"2024-03-26T17:36:23.746910+0000","last_peered":"2024-03-26T17:36:23.746910+0000","last_clean":"2024-03-26T17:36:23.746910+0000","last_became_active":"2024-03-26T17:36:21.722067+0000","last_became_peered":"2024-03-26T17:36:21.722067+0000","last_unstale":"2024-03-26T17:36:23.746910+0000","last_undegraded":"2024-03-26T17:36:23.746910+0000","last_fullsized":"2024-03-26T17:36:23.746910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:34:40.316110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.955622+0000","last_change":"2024-03-26T17:36:22.730890+0000","last_active":"2024-03-26T17:36:23.955622+0000","last_peered":"2024-03-26T17:36:23.955622+0000","last_clean":"2024-03-26T17:36:23.955622+0000","last_became_active":"2024-03-26T17:36:22.729635+0000","last_became_peered":"2024-03-26T17:36:22.729635+0000","last_unstale":"2024-03-26T17:36:23.955622+0000","last_undegraded":"2024-03-26T17:36:23.955622+0000","last_fullsized":"2024-03-26T17:36:23.955622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:15:09.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705644+0000","last_change":"2024-03-26T17:36:21.727587+0000","last_active":"2024-03-26T17:36:24.705644+0000","last_peered":"2024-03-26T17:36:24.705644+0000","last_clean":"2024-03-26T17:36:24.705644+0000","last_became_active":"2024-03-26T17:36:21.727302+0000","last_became_peered":"2024-03-26T17:36:21.727302+0000","last_unstale":"2024-03-26T17:36:24.705644+0000","last_undegraded":"2024-03-26T17:36:24.705644+0000","last_fullsized":"2024-03-26T17:36:24.705644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:59.404567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.955516+0000","last_change":"2024-03-26T17:36:22.750654+0000","last_active":"2024-03-26T17:36:23.955516+0000","last_peered":"2024-03-26T17:36:23.955516+0000","last_clean":"2024-03-26T17:36:23.955516+0000","last_became_active":"2024-03-26T17:36:22.750281+0000","last_became_peered":"2024-03-26T17:36:22.750281+0000","last_unstale":"2024-03-26T17:36:23.955516+0000","last_undegraded":"2024-03-26T17:36:23.955516+0000","last_fullsized":"2024-03-26T17:36:23.955516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:04.034108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909350+0000","last_change":"2024-03-26T17:36:22.739206+0000","last_active":"2024-03-26T17:36:23.909350+0000","last_peered":"2024-03-26T17:36:23.909350+0000","last_clean":"2024-03-26T17:36:23.909350+0000","last_became_active":"2024-03-26T17:36:22.738252+0000","last_became_peered":"2024-03-26T17:36:22.738252+0000","last_unstale":"2024-03-26T17:36:23.909350+0000","last_undegraded":"2024-03-26T17:36:23.909350+0000","last_fullsized":"2024-03-26T17:36:23.909350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:35.655111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:26.332241+0000","last_change":"2024-03-26T17:36:21.730812+0000","last_active":"2024-03-26T17:36:26.332241+0000","last_peered":"2024-03-26T17:36:26.332241+0000","last_clean":"2024-03-26T17:36:26.332241+0000","last_became_active":"2024-03-26T17:36:21.730499+0000","last_became_peered":"2024-03-26T17:36:21.730499+0000","last_unstale":"2024-03-26T17:36:26.332241+0000","last_undegraded":"2024-03-26T17:36:26.332241+0000","last_fullsized":"2024-03-26T17:36:26.332241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:05.666146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748691+0000","last_change":"2024-03-26T17:36:22.742344+0000","last_active":"2024-03-26T17:36:23.748691+0000","last_peered":"2024-03-26T17:36:23.748691+0000","last_clean":"2024-03-26T17:36:23.748691+0000","last_became_active":"2024-03-26T17:36:22.741864+0000","last_became_peered":"2024-03-26T17:36:22.741864+0000","last_unstale":"2024-03-26T17:36:23.748691+0000","last_undegraded":"2024-03-26T17:36:23.748691+0000","last_fullsized":"2024-03-26T17:36:23.748691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:53:53.553107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.314466+0000","last_change":"2024-03-26T17:36:21.722450+0000","last_active":"2024-03-26T17:36:28.314466+0000","last_peered":"2024-03-26T17:36:28.314466+0000","last_clean":"2024-03-26T17:36:28.314466+0000","last_became_active":"2024-03-26T17:36:21.722105+0000","last_became_peered":"2024-03-26T17:36:21.722105+0000","last_unstale":"2024-03-26T17:36:28.314466+0000","last_undegraded":"2024-03-26T17:36:28.314466+0000","last_fullsized":"2024-03-26T17:36:28.314466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:01:22.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705433+0000","last_change":"2024-03-26T17:35:10.663606+0000","last_active":"2024-03-26T17:36:24.705433+0000","last_peered":"2024-03-26T17:36:24.705433+0000","last_clean":"2024-03-26T17:36:24.705433+0000","last_became_active":"2024-03-26T17:35:10.663069+0000","last_became_peered":"2024-03-26T17:35:10.663069+0000","last_unstale":"2024-03-26T17:36:24.705433+0000","last_undegraded":"2024-03-26T17:36:24.705433+0000","last_fullsized":"2024-03-26T17:36:24.705433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_clean_scrub_stamp":"2024-03-26T17:31:50.880228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:50.817007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723514+0000","last_change":"2024-03-26T17:36:22.742987+0000","last_active":"2024-03-26T17:36:23.723514+0000","last_peered":"2024-03-26T17:36:23.723514+0000","last_clean":"2024-03-26T17:36:23.723514+0000","last_became_active":"2024-03-26T17:36:22.742661+0000","last_became_peered":"2024-03-26T17:36:22.742661+0000","last_unstale":"2024-03-26T17:36:23.723514+0000","last_undegraded":"2024-03-26T17:36:23.723514+0000","last_fullsized":"2024-03-26T17:36:23.723514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:37:39.837294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:26.332804+0000","last_change":"2024-03-26T17:36:21.723452+0000","last_active":"2024-03-26T17:36:26.332804+0000","last_peered":"2024-03-26T17:36:26.332804+0000","last_clean":"2024-03-26T17:36:26.332804+0000","last_became_active":"2024-03-26T17:36:21.723155+0000","last_became_peered":"2024-03-26T17:36:21.723155+0000","last_unstale":"2024-03-26T17:36:26.332804+0000","last_undegraded":"2024-03-26T17:36:26.332804+0000","last_fullsized":"2024-03-26T17:36:26.332804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:39:34.443624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749260+0000","last_change":"2024-03-26T17:36:21.731919+0000","last_active":"2024-03-26T17:36:23.749260+0000","last_peered":"2024-03-26T17:36:23.749260+0000","last_clean":"2024-03-26T17:36:23.749260+0000","last_became_active":"2024-03-26T17:36:21.731216+0000","last_became_peered":"2024-03-26T17:36:21.731216+0000","last_unstale":"2024-03-26T17:36:23.749260+0000","last_undegraded":"2024-03-26T17:36:23.749260+0000","last_fullsized":"2024-03-26T17:36:23.749260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:47:59.538107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723896+0000","last_change":"2024-03-26T17:36:22.736487+0000","last_active":"2024-03-26T17:36:23.723896+0000","last_peered":"2024-03-26T17:36:23.723896+0000","last_clean":"2024-03-26T17:36:23.723896+0000","last_became_active":"2024-03-26T17:36:22.735788+0000","last_became_peered":"2024-03-26T17:36:22.735788+0000","last_unstale":"2024-03-26T17:36:23.723896+0000","last_undegraded":"2024-03-26T17:36:23.723896+0000","last_fullsized":"2024-03-26T17:36:23.723896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:45.240148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:27.338662+0000","last_change":"2024-03-26T17:36:21.720755+0000","last_active":"2024-03-26T17:36:27.338662+0000","last_peered":"2024-03-26T17:36:27.338662+0000","last_clean":"2024-03-26T17:36:27.338662+0000","last_became_active":"2024-03-26T17:36:21.720175+0000","last_became_peered":"2024-03-26T17:36:21.720175+0000","last_unstale":"2024-03-26T17:36:27.338662+0000","last_undegraded":"2024-03-26T17:36:27.338662+0000","last_fullsized":"2024-03-26T17:36:27.338662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:22.458657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748935+0000","last_change":"2024-03-26T17:36:22.740731+0000","last_active":"2024-03-26T17:36:23.748935+0000","last_peered":"2024-03-26T17:36:23.748935+0000","last_clean":"2024-03-26T17:36:23.748935+0000","last_became_active":"2024-03-26T17:36:22.740283+0000","last_became_peered":"2024-03-26T17:36:22.740283+0000","last_unstale":"2024-03-26T17:36:23.748935+0000","last_undegraded":"2024-03-26T17:36:23.748935+0000","last_fullsized":"2024-03-26T17:36:23.748935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:36.403105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.313196+0000","last_change":"2024-03-26T17:36:21.721513+0000","last_active":"2024-03-26T17:36:28.313196+0000","last_peered":"2024-03-26T17:36:28.313196+0000","last_clean":"2024-03-26T17:36:28.313196+0000","last_became_active":"2024-03-26T17:36:21.721185+0000","last_became_peered":"2024-03-26T17:36:21.721185+0000","last_unstale":"2024-03-26T17:36:28.313196+0000","last_undegraded":"2024-03-26T17:36:28.313196+0000","last_fullsized":"2024-03-26T17:36:28.313196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:48:39.994980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325712+0000","last_change":"2024-03-26T17:36:22.737938+0000","last_active":"2024-03-26T17:36:25.325712+0000","last_peered":"2024-03-26T17:36:25.325712+0000","last_clean":"2024-03-26T17:36:25.325712+0000","last_became_active":"2024-03-26T17:36:22.737605+0000","last_became_peered":"2024-03-26T17:36:22.737605+0000","last_unstale":"2024-03-26T17:36:25.325712+0000","last_undegraded":"2024-03-26T17:36:25.325712+0000","last_fullsized":"2024-03-26T17:36:25.325712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:27.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325890+0000","last_change":"2024-03-26T17:36:21.720942+0000","last_active":"2024-03-26T17:36:25.325890+0000","last_peered":"2024-03-26T17:36:25.325890+0000","last_clean":"2024-03-26T17:36:25.325890+0000","last_became_active":"2024-03-26T17:36:21.720665+0000","last_became_peered":"2024-03-26T17:36:21.720665+0000","last_unstale":"2024-03-26T17:36:25.325890+0000","last_undegraded":"2024-03-26T17:36:25.325890+0000","last_fullsized":"2024-03-26T17:36:25.325890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:00.725213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748972+0000","last_change":"2024-03-26T17:36:22.747111+0000","last_active":"2024-03-26T17:36:23.748972+0000","last_peered":"2024-03-26T17:36:23.748972+0000","last_clean":"2024-03-26T17:36:23.748972+0000","last_became_active":"2024-03-26T17:36:22.746626+0000","last_became_peered":"2024-03-26T17:36:22.746626+0000","last_unstale":"2024-03-26T17:36:23.748972+0000","last_undegraded":"2024-03-26T17:36:23.748972+0000","last_fullsized":"2024-03-26T17:36:23.748972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:29:24.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.314430+0000","last_change":"2024-03-26T17:36:21.724141+0000","last_active":"2024-03-26T17:36:28.314430+0000","last_peered":"2024-03-26T17:36:28.314430+0000","last_clean":"2024-03-26T17:36:28.314430+0000","last_became_active":"2024-03-26T17:36:21.723797+0000","last_became_peered":"2024-03-26T17:36:21.723797+0000","last_unstale":"2024-03-26T17:36:28.314430+0000","last_undegraded":"2024-03-26T17:36:28.314430+0000","last_fullsized":"2024-03-26T17:36:28.314430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:06.149827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326266+0000","last_change":"2024-03-26T17:36:22.739497+0000","last_active":"2024-03-26T17:36:25.326266+0000","last_peered":"2024-03-26T17:36:25.326266+0000","last_clean":"2024-03-26T17:36:25.326266+0000","last_became_active":"2024-03-26T17:36:22.739114+0000","last_became_peered":"2024-03-26T17:36:22.739114+0000","last_unstale":"2024-03-26T17:36:25.326266+0000","last_undegraded":"2024-03-26T17:36:25.326266+0000","last_fullsized":"2024-03-26T17:36:25.326266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:03.953199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.722642+0000","last_change":"2024-03-26T17:36:21.728027+0000","last_active":"2024-03-26T17:36:23.722642+0000","last_peered":"2024-03-26T17:36:23.722642+0000","last_clean":"2024-03-26T17:36:23.722642+0000","last_became_active":"2024-03-26T17:36:21.727159+0000","last_became_peered":"2024-03-26T17:36:21.727159+0000","last_unstale":"2024-03-26T17:36:23.722642+0000","last_undegraded":"2024-03-26T17:36:23.722642+0000","last_fullsized":"2024-03-26T17:36:23.722642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:39.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748834+0000","last_change":"2024-03-26T17:36:22.733666+0000","last_active":"2024-03-26T17:36:23.748834+0000","last_peered":"2024-03-26T17:36:23.748834+0000","last_clean":"2024-03-26T17:36:23.748834+0000","last_became_active":"2024-03-26T17:36:22.733313+0000","last_became_peered":"2024-03-26T17:36:22.733313+0000","last_unstale":"2024-03-26T17:36:23.748834+0000","last_undegraded":"2024-03-26T17:36:23.748834+0000","last_fullsized":"2024-03-26T17:36:23.748834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:52:38.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325919+0000","last_change":"2024-03-26T17:36:21.730828+0000","last_active":"2024-03-26T17:36:25.325919+0000","last_peered":"2024-03-26T17:36:25.325919+0000","last_clean":"2024-03-26T17:36:25.325919+0000","last_became_active":"2024-03-26T17:36:21.730508+0000","last_became_peered":"2024-03-26T17:36:21.730508+0000","last_unstale":"2024-03-26T17:36:25.325919+0000","last_undegraded":"2024-03-26T17:36:25.325919+0000","last_fullsized":"2024-03-26T17:36:25.325919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:10:53.492725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705800+0000","last_change":"2024-03-26T17:36:22.740204+0000","last_active":"2024-03-26T17:36:24.705800+0000","last_peered":"2024-03-26T17:36:24.705800+0000","last_clean":"2024-03-26T17:36:24.705800+0000","last_became_active":"2024-03-26T17:36:22.735880+0000","last_became_peered":"2024-03-26T17:36:22.735880+0000","last_unstale":"2024-03-26T17:36:24.705800+0000","last_undegraded":"2024-03-26T17:36:24.705800+0000","last_fullsized":"2024-03-26T17:36:24.705800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:34.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723004+0000","last_change":"2024-03-26T17:36:21.720873+0000","last_active":"2024-03-26T17:36:23.723004+0000","last_peered":"2024-03-26T17:36:23.723004+0000","last_clean":"2024-03-26T17:36:23.723004+0000","last_became_active":"2024-03-26T17:36:21.720496+0000","last_became_peered":"2024-03-26T17:36:21.720496+0000","last_unstale":"2024-03-26T17:36:23.723004+0000","last_undegraded":"2024-03-26T17:36:23.723004+0000","last_fullsized":"2024-03-26T17:36:23.723004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:40:55.959064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705833+0000","last_change":"2024-03-26T17:36:22.735528+0000","last_active":"2024-03-26T17:36:24.705833+0000","last_peered":"2024-03-26T17:36:24.705833+0000","last_clean":"2024-03-26T17:36:24.705833+0000","last_became_active":"2024-03-26T17:36:22.735147+0000","last_became_peered":"2024-03-26T17:36:22.735147+0000","last_unstale":"2024-03-26T17:36:24.705833+0000","last_undegraded":"2024-03-26T17:36:24.705833+0000","last_fullsized":"2024-03-26T17:36:24.705833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:15.729853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.305262+0000","last_change":"2024-03-26T17:36:21.723246+0000","last_active":"2024-03-26T17:36:28.305262+0000","last_peered":"2024-03-26T17:36:28.305262+0000","last_clean":"2024-03-26T17:36:28.305262+0000","last_became_active":"2024-03-26T17:36:21.722839+0000","last_became_peered":"2024-03-26T17:36:21.722839+0000","last_unstale":"2024-03-26T17:36:28.305262+0000","last_undegraded":"2024-03-26T17:36:28.305262+0000","last_fullsized":"2024-03-26T17:36:28.305262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:41:05.540206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705650+0000","last_change":"2024-03-26T17:36:22.742469+0000","last_active":"2024-03-26T17:36:24.705650+0000","last_peered":"2024-03-26T17:36:24.705650+0000","last_clean":"2024-03-26T17:36:24.705650+0000","last_became_active":"2024-03-26T17:36:22.742073+0000","last_became_peered":"2024-03-26T17:36:22.742073+0000","last_unstale":"2024-03-26T17:36:24.705650+0000","last_undegraded":"2024-03-26T17:36:24.705650+0000","last_fullsized":"2024-03-26T17:36:24.705650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:19:33.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723951+0000","last_change":"2024-03-26T17:36:21.730796+0000","last_active":"2024-03-26T17:36:23.723951+0000","last_peered":"2024-03-26T17:36:23.723951+0000","last_clean":"2024-03-26T17:36:23.723951+0000","last_became_active":"2024-03-26T17:36:21.730462+0000","last_became_peered":"2024-03-26T17:36:21.730462+0000","last_unstale":"2024-03-26T17:36:23.723951+0000","last_undegraded":"2024-03-26T17:36:23.723951+0000","last_fullsized":"2024-03-26T17:36:23.723951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:47:02.582630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705885+0000","last_change":"2024-03-26T17:36:22.742614+0000","last_active":"2024-03-26T17:36:24.705885+0000","last_peered":"2024-03-26T17:36:24.705885+0000","last_clean":"2024-03-26T17:36:24.705885+0000","last_became_active":"2024-03-26T17:36:22.742278+0000","last_became_peered":"2024-03-26T17:36:22.742278+0000","last_unstale":"2024-03-26T17:36:24.705885+0000","last_undegraded":"2024-03-26T17:36:24.705885+0000","last_fullsized":"2024-03-26T17:36:24.705885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:30:14.503474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.327118+0000","last_change":"2024-03-26T17:36:22.741693+0000","last_active":"2024-03-26T17:36:25.327118+0000","last_peered":"2024-03-26T17:36:25.327118+0000","last_clean":"2024-03-26T17:36:25.327118+0000","last_became_active":"2024-03-26T17:36:22.741352+0000","last_became_peered":"2024-03-26T17:36:22.741352+0000","last_unstale":"2024-03-26T17:36:25.327118+0000","last_undegraded":"2024-03-26T17:36:25.327118+0000","last_fullsized":"2024-03-26T17:36:25.327118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:13:26.912910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.722946+0000","last_change":"2024-03-26T17:36:22.742939+0000","last_active":"2024-03-26T17:36:23.722946+0000","last_peered":"2024-03-26T17:36:23.722946+0000","last_clean":"2024-03-26T17:36:23.722946+0000","last_became_active":"2024-03-26T17:36:22.742589+0000","last_became_peered":"2024-03-26T17:36:22.742589+0000","last_unstale":"2024-03-26T17:36:23.722946+0000","last_undegraded":"2024-03-26T17:36:23.722946+0000","last_fullsized":"2024-03-26T17:36:23.722946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:21:14.043601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.722897+0000","last_change":"2024-03-26T17:36:22.748044+0000","last_active":"2024-03-26T17:36:23.722897+0000","last_peered":"2024-03-26T17:36:23.722897+0000","last_clean":"2024-03-26T17:36:23.722897+0000","last_became_active":"2024-03-26T17:36:22.747683+0000","last_became_peered":"2024-03-26T17:36:22.747683+0000","last_unstale":"2024-03-26T17:36:23.722897+0000","last_undegraded":"2024-03-26T17:36:23.722897+0000","last_fullsized":"2024-03-26T17:36:23.722897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:32:40.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.324949+0000","last_change":"2024-03-26T17:36:22.743080+0000","last_active":"2024-03-26T17:36:25.324949+0000","last_peered":"2024-03-26T17:36:25.324949+0000","last_clean":"2024-03-26T17:36:25.324949+0000","last_became_active":"2024-03-26T17:36:22.742676+0000","last_became_peered":"2024-03-26T17:36:22.742676+0000","last_unstale":"2024-03-26T17:36:25.324949+0000","last_undegraded":"2024-03-26T17:36:25.324949+0000","last_fullsized":"2024-03-26T17:36:25.324949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:58.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723838+0000","last_change":"2024-03-26T17:36:22.737908+0000","last_active":"2024-03-26T17:36:23.723838+0000","last_peered":"2024-03-26T17:36:23.723838+0000","last_clean":"2024-03-26T17:36:23.723838+0000","last_became_active":"2024-03-26T17:36:22.737055+0000","last_became_peered":"2024-03-26T17:36:22.737055+0000","last_unstale":"2024-03-26T17:36:23.723838+0000","last_undegraded":"2024-03-26T17:36:23.723838+0000","last_fullsized":"2024-03-26T17:36:23.723838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:51:13.157901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326639+0000","last_change":"2024-03-26T17:36:22.749795+0000","last_active":"2024-03-26T17:36:25.326639+0000","last_peered":"2024-03-26T17:36:25.326639+0000","last_clean":"2024-03-26T17:36:25.326639+0000","last_became_active":"2024-03-26T17:36:22.749469+0000","last_became_peered":"2024-03-26T17:36:22.749469+0000","last_unstale":"2024-03-26T17:36:25.326639+0000","last_undegraded":"2024-03-26T17:36:25.326639+0000","last_fullsized":"2024-03-26T17:36:25.326639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:19:59.126270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723356+0000","last_change":"2024-03-26T17:36:22.744101+0000","last_active":"2024-03-26T17:36:23.723356+0000","last_peered":"2024-03-26T17:36:23.723356+0000","last_clean":"2024-03-26T17:36:23.723356+0000","last_became_active":"2024-03-26T17:36:22.743761+0000","last_became_peered":"2024-03-26T17:36:22.743761+0000","last_unstale":"2024-03-26T17:36:23.723356+0000","last_undegraded":"2024-03-26T17:36:23.723356+0000","last_fullsized":"2024-03-26T17:36:23.723356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:27:19.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748926+0000","last_change":"2024-03-26T17:36:22.744800+0000","last_active":"2024-03-26T17:36:23.748926+0000","last_peered":"2024-03-26T17:36:23.748926+0000","last_clean":"2024-03-26T17:36:23.748926+0000","last_became_active":"2024-03-26T17:36:22.744437+0000","last_became_peered":"2024-03-26T17:36:22.744437+0000","last_unstale":"2024-03-26T17:36:23.748926+0000","last_undegraded":"2024-03-26T17:36:23.748926+0000","last_fullsized":"2024-03-26T17:36:23.748926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:36.459072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909372+0000","last_change":"2024-03-26T17:36:22.739421+0000","last_active":"2024-03-26T17:36:23.909372+0000","last_peered":"2024-03-26T17:36:23.909372+0000","last_clean":"2024-03-26T17:36:23.909372+0000","last_became_active":"2024-03-26T17:36:22.738837+0000","last_became_peered":"2024-03-26T17:36:22.738837+0000","last_unstale":"2024-03-26T17:36:23.909372+0000","last_undegraded":"2024-03-26T17:36:23.909372+0000","last_fullsized":"2024-03-26T17:36:23.909372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:42.907666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748612+0000","last_change":"2024-03-26T17:36:22.743211+0000","last_active":"2024-03-26T17:36:23.748612+0000","last_peered":"2024-03-26T17:36:23.748612+0000","last_clean":"2024-03-26T17:36:23.748612+0000","last_became_active":"2024-03-26T17:36:22.742869+0000","last_became_peered":"2024-03-26T17:36:22.742869+0000","last_unstale":"2024-03-26T17:36:23.748612+0000","last_undegraded":"2024-03-26T17:36:23.748612+0000","last_fullsized":"2024-03-26T17:36:23.748612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:24:04.610012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721643+0000","last_change":"2024-03-26T17:36:22.746950+0000","last_active":"2024-03-26T17:36:23.721643+0000","last_peered":"2024-03-26T17:36:23.721643+0000","last_clean":"2024-03-26T17:36:23.721643+0000","last_became_active":"2024-03-26T17:36:22.744869+0000","last_became_peered":"2024-03-26T17:36:22.744869+0000","last_unstale":"2024-03-26T17:36:23.721643+0000","last_undegraded":"2024-03-26T17:36:23.721643+0000","last_fullsized":"2024-03-26T17:36:23.721643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:09:52.283519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723768+0000","last_change":"2024-03-26T17:36:22.744627+0000","last_active":"2024-03-26T17:36:23.723768+0000","last_peered":"2024-03-26T17:36:23.723768+0000","last_clean":"2024-03-26T17:36:23.723768+0000","last_became_active":"2024-03-26T17:36:22.744299+0000","last_became_peered":"2024-03-26T17:36:22.744299+0000","last_unstale":"2024-03-26T17:36:23.723768+0000","last_undegraded":"2024-03-26T17:36:23.723768+0000","last_fullsized":"2024-03-26T17:36:23.723768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:23:37.941625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326278+0000","last_change":"2024-03-26T17:36:22.751135+0000","last_active":"2024-03-26T17:36:25.326278+0000","last_peered":"2024-03-26T17:36:25.326278+0000","last_clean":"2024-03-26T17:36:25.326278+0000","last_became_active":"2024-03-26T17:36:22.750768+0000","last_became_peered":"2024-03-26T17:36:22.750768+0000","last_unstale":"2024-03-26T17:36:25.326278+0000","last_undegraded":"2024-03-26T17:36:25.326278+0000","last_fullsized":"2024-03-26T17:36:25.326278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:15.532367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5256,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":140,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":18191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122,"ondisk_log_size":122,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645329,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1030979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":438951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1030875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":438155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":439521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":439689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":438473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":438743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32692,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712460,"statfs":{"total":95995035648,"available":95961559040,"internally_reserved":0,"allocated":1552384,"data_stored":1031231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T17:36:30.818 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T17:36:30.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 -- 172.21.15.105:0/3016580924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c1075c0 msgr2=0x7fba3c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 --2- 172.21.15.105:0/3016580924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c1075c0 0x7fba3c109a10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fba34009ca0 tx=0x7fba3402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 -- 172.21.15.105:0/3016580924 shutdown_connections 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 --2- 172.21.15.105:0/3016580924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c1075c0 0x7fba3c109a10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 --2- 172.21.15.105:0/3016580924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c104c90 0x7fba3c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 --2- 172.21.15.105:0/3016580924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3c102360 0x7fba3c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 -- 172.21.15.105:0/3016580924 >> 172.21.15.105:0/3016580924 conn(0x7fba3c0fbd50 msgr2=0x7fba3c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 -- 172.21.15.105:0/3016580924 shutdown_connections 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.963+0000 7fba409b9640 1 -- 172.21.15.105:0/3016580924 wait complete. 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba409b9640 1 Processor -- start 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba409b9640 1 -- start start 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba409b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3c102360 0x7fba3c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba409b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c104c90 0x7fba3c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba409b9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c1075c0 0x7fba3c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba409b9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c10d430 con 0x7fba3c104c90 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba409b9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c10d2b0 con 0x7fba3c1075c0 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba409b9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c10d5b0 con 0x7fba3c102360 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba39d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c104c90 0x7fba3c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba39d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c104c90 0x7fba3c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49392/0 (socket says 172.21.15.105:49392) 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba3ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c1075c0 0x7fba3c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba39d9b640 1 -- 172.21.15.105:0/3639453284 learned_addr learned my addr 172.21.15.105:0/3639453284 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.964+0000 7fba3a59c640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3c102360 0x7fba3c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba39d9b640 1 -- 172.21.15.105:0/3639453284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3c102360 msgr2=0x7fba3c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba39d9b640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3c102360 0x7fba3c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba39d9b640 1 -- 172.21.15.105:0/3639453284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c1075c0 msgr2=0x7fba3c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba39d9b640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c1075c0 0x7fba3c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba39d9b640 1 -- 172.21.15.105:0/3639453284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c19f700 con 0x7fba3c104c90 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba3a59c640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3c102360 0x7fba3c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba3ad9d640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c1075c0 0x7fba3c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba39d9b640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c104c90 0x7fba3c198020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fba3000e9c0 tx=0x7fba3000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba2b7fe640 1 -- 172.21.15.105:0/3639453284 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3000cd80 con 0x7fba3c104c90 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba2b7fe640 1 -- 172.21.15.105:0/3639453284 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba300054d0 con 0x7fba3c104c90 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c19f9f0 con 0x7fba3c104c90 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba2b7fe640 1 -- 172.21.15.105:0/3639453284 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30010430 con 0x7fba3c104c90 2024-03-26T17:36:30.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.965+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c19ff40 con 0x7fba3c104c90 2024-03-26T17:36:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.966+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3c0fd840 con 0x7fba3c104c90 2024-03-26T17:36:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.967+0000 7fba2b7fe640 1 -- 172.21.15.105:0/3639453284 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fba30004fd0 con 0x7fba3c104c90 2024-03-26T17:36:30.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.967+0000 7fba2b7fe640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba240a2fb0 0x7fba240a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.968+0000 7fba3a59c640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba240a2fb0 0x7fba240a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.968+0000 7fba2b7fe640 1 -- 172.21.15.105:0/3639453284 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fba300cf790 con 0x7fba3c104c90 2024-03-26T17:36:30.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.968+0000 7fba3a59c640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba240a2fb0 0x7fba240a5470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009fc0 tx=0x7fba2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:30.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:30.972+0000 7fba2b7fe640 1 -- 172.21.15.105:0/3639453284 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fba3009c610 con 0x7fba3c104c90 2024-03-26T17:36:31.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.148+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba3c061d00 con 0x7fba240a2fb0 2024-03-26T17:36:31.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.156+0000 7fba2b7fe640 1 -- 172.21.15.105:0/3639453284 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258765 (secure 0 0 0) 0x7fba3c061d00 con 0x7fba240a2fb0 2024-03-26T17:36:31.152 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:31.153 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-26T17:36:31.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba240a2fb0 msgr2=0x7fba240a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:31.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba240a2fb0 0x7fba240a5470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009fc0 tx=0x7fba2c005f90 comp rx=0 tx=0).stop 2024-03-26T17:36:31.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c104c90 msgr2=0x7fba3c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:31.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c104c90 0x7fba3c198020 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fba3000e9c0 tx=0x7fba3000ee80 comp rx=0 tx=0).stop 2024-03-26T17:36:31.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 shutdown_connections 2024-03-26T17:36:31.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba240a2fb0 0x7fba240a5470 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:31.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba3c1075c0 0x7fba3c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:31.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba3c104c90 0x7fba3c198020 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:31.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 --2- 172.21.15.105:0/3639453284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba3c102360 0x7fba3c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:31.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 >> 172.21.15.105:0/3639453284 conn(0x7fba3c0fbd50 msgr2=0x7fba3c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:31.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 shutdown_connections 2024-03-26T17:36:31.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:31.159+0000 7fba409b9640 1 -- 172.21.15.105:0/3639453284 wait complete. 2024-03-26T17:36:31.165 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-03-26T17:36:30.583327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595624,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":145,"num_read_kb":124,"num_write":321,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":267,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366036,"kb_used_data":12708,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575788,"statfs":{"total":1151940427776,"available":1151565606912,"internally_reserved":0,"allocated":13012992,"data_stored":7044567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5256,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":140,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001818"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.327606+0000","last_change":"2024-03-26T17:36:22.743815+0000","last_active":"2024-03-26T17:36:25.327606+0000","last_peered":"2024-03-26T17:36:25.327606+0000","last_clean":"2024-03-26T17:36:25.327606+0000","last_became_active":"2024-03-26T17:36:22.743446+0000","last_became_peered":"2024-03-26T17:36:22.743446+0000","last_unstale":"2024-03-26T17:36:25.327606+0000","last_undegraded":"2024-03-26T17:36:25.327606+0000","last_fullsized":"2024-03-26T17:36:25.327606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:16:56.773849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325545+0000","last_change":"2024-03-26T17:36:22.740754+0000","last_active":"2024-03-26T17:36:25.325545+0000","last_peered":"2024-03-26T17:36:25.325545+0000","last_clean":"2024-03-26T17:36:25.325545+0000","last_became_active":"2024-03-26T17:36:22.740317+0000","last_became_peered":"2024-03-26T17:36:22.740317+0000","last_unstale":"2024-03-26T17:36:25.325545+0000","last_undegraded":"2024-03-26T17:36:25.325545+0000","last_fullsized":"2024-03-26T17:36:25.325545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:50:45.321653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325516+0000","last_change":"2024-03-26T17:36:22.735079+0000","last_active":"2024-03-26T17:36:25.325516+0000","last_peered":"2024-03-26T17:36:25.325516+0000","last_clean":"2024-03-26T17:36:25.325516+0000","last_became_active":"2024-03-26T17:36:22.734646+0000","last_became_peered":"2024-03-26T17:36:22.734646+0000","last_unstale":"2024-03-26T17:36:25.325516+0000","last_undegraded":"2024-03-26T17:36:25.325516+0000","last_fullsized":"2024-03-26T17:36:25.325516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:41:09.477438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.322876+0000","last_change":"2024-03-26T17:36:22.746170+0000","last_active":"2024-03-26T17:36:25.322876+0000","last_peered":"2024-03-26T17:36:25.322876+0000","last_clean":"2024-03-26T17:36:25.322876+0000","last_became_active":"2024-03-26T17:36:22.744255+0000","last_became_peered":"2024-03-26T17:36:22.744255+0000","last_unstale":"2024-03-26T17:36:25.322876+0000","last_undegraded":"2024-03-26T17:36:25.322876+0000","last_fullsized":"2024-03-26T17:36:25.322876+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:54:36.225926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749116+0000","last_change":"2024-03-26T17:36:22.737390+0000","last_active":"2024-03-26T17:36:23.749116+0000","last_peered":"2024-03-26T17:36:23.749116+0000","last_clean":"2024-03-26T17:36:23.749116+0000","last_became_active":"2024-03-26T17:36:22.737038+0000","last_became_peered":"2024-03-26T17:36:22.737038+0000","last_unstale":"2024-03-26T17:36:23.749116+0000","last_undegraded":"2024-03-26T17:36:23.749116+0000","last_fullsized":"2024-03-26T17:36:23.749116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:48.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.955420+0000","last_change":"2024-03-26T17:36:22.748097+0000","last_active":"2024-03-26T17:36:23.955420+0000","last_peered":"2024-03-26T17:36:23.955420+0000","last_clean":"2024-03-26T17:36:23.955420+0000","last_became_active":"2024-03-26T17:36:22.747730+0000","last_became_peered":"2024-03-26T17:36:22.747730+0000","last_unstale":"2024-03-26T17:36:23.955420+0000","last_undegraded":"2024-03-26T17:36:23.955420+0000","last_fullsized":"2024-03-26T17:36:23.955420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:43:01.370199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705500+0000","last_change":"2024-03-26T17:36:22.739587+0000","last_active":"2024-03-26T17:36:24.705500+0000","last_peered":"2024-03-26T17:36:24.705500+0000","last_clean":"2024-03-26T17:36:24.705500+0000","last_became_active":"2024-03-26T17:36:22.735787+0000","last_became_peered":"2024-03-26T17:36:22.735787+0000","last_unstale":"2024-03-26T17:36:24.705500+0000","last_undegraded":"2024-03-26T17:36:24.705500+0000","last_fullsized":"2024-03-26T17:36:24.705500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:41.381153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748525+0000","last_change":"2024-03-26T17:36:22.736716+0000","last_active":"2024-03-26T17:36:23.748525+0000","last_peered":"2024-03-26T17:36:23.748525+0000","last_clean":"2024-03-26T17:36:23.748525+0000","last_became_active":"2024-03-26T17:36:22.736327+0000","last_became_peered":"2024-03-26T17:36:22.736327+0000","last_unstale":"2024-03-26T17:36:23.748525+0000","last_undegraded":"2024-03-26T17:36:23.748525+0000","last_fullsized":"2024-03-26T17:36:23.748525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:10:43.439524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721997+0000","last_change":"2024-03-26T17:36:22.746355+0000","last_active":"2024-03-26T17:36:23.721997+0000","last_peered":"2024-03-26T17:36:23.721997+0000","last_clean":"2024-03-26T17:36:23.721997+0000","last_became_active":"2024-03-26T17:36:22.745363+0000","last_became_peered":"2024-03-26T17:36:22.745363+0000","last_unstale":"2024-03-26T17:36:23.721997+0000","last_undegraded":"2024-03-26T17:36:23.721997+0000","last_fullsized":"2024-03-26T17:36:23.721997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:56:33.528005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.955774+0000","last_change":"2024-03-26T17:36:22.749009+0000","last_active":"2024-03-26T17:36:23.955774+0000","last_peered":"2024-03-26T17:36:23.955774+0000","last_clean":"2024-03-26T17:36:23.955774+0000","last_became_active":"2024-03-26T17:36:22.748627+0000","last_became_peered":"2024-03-26T17:36:22.748627+0000","last_unstale":"2024-03-26T17:36:23.955774+0000","last_undegraded":"2024-03-26T17:36:23.955774+0000","last_fullsized":"2024-03-26T17:36:23.955774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:59:14.812316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705800+0000","last_change":"2024-03-26T17:36:22.741294+0000","last_active":"2024-03-26T17:36:24.705800+0000","last_peered":"2024-03-26T17:36:24.705800+0000","last_clean":"2024-03-26T17:36:24.705800+0000","last_became_active":"2024-03-26T17:36:22.740003+0000","last_became_peered":"2024-03-26T17:36:22.740003+0000","last_unstale":"2024-03-26T17:36:24.705800+0000","last_undegraded":"2024-03-26T17:36:24.705800+0000","last_fullsized":"2024-03-26T17:36:24.705800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:06.305195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.322721+0000","last_change":"2024-03-26T17:36:22.745884+0000","last_active":"2024-03-26T17:36:25.322721+0000","last_peered":"2024-03-26T17:36:25.322721+0000","last_clean":"2024-03-26T17:36:25.322721+0000","last_became_active":"2024-03-26T17:36:22.744168+0000","last_became_peered":"2024-03-26T17:36:22.744168+0000","last_unstale":"2024-03-26T17:36:25.322721+0000","last_undegraded":"2024-03-26T17:36:25.322721+0000","last_fullsized":"2024-03-26T17:36:25.322721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:50.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723833+0000","last_change":"2024-03-26T17:36:22.725956+0000","last_active":"2024-03-26T17:36:23.723833+0000","last_peered":"2024-03-26T17:36:23.723833+0000","last_clean":"2024-03-26T17:36:23.723833+0000","last_became_active":"2024-03-26T17:36:22.725513+0000","last_became_peered":"2024-03-26T17:36:22.725513+0000","last_unstale":"2024-03-26T17:36:23.723833+0000","last_undegraded":"2024-03-26T17:36:23.723833+0000","last_fullsized":"2024-03-26T17:36:23.723833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:30:10.914233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.324324+0000","last_change":"2024-03-26T17:36:22.739757+0000","last_active":"2024-03-26T17:36:25.324324+0000","last_peered":"2024-03-26T17:36:25.324324+0000","last_clean":"2024-03-26T17:36:25.324324+0000","last_became_active":"2024-03-26T17:36:22.739342+0000","last_became_peered":"2024-03-26T17:36:22.739342+0000","last_unstale":"2024-03-26T17:36:25.324324+0000","last_undegraded":"2024-03-26T17:36:25.324324+0000","last_fullsized":"2024-03-26T17:36:25.324324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:28.171325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.324063+0000","last_change":"2024-03-26T17:36:22.742988+0000","last_active":"2024-03-26T17:36:25.324063+0000","last_peered":"2024-03-26T17:36:25.324063+0000","last_clean":"2024-03-26T17:36:25.324063+0000","last_became_active":"2024-03-26T17:36:22.742639+0000","last_became_peered":"2024-03-26T17:36:22.742639+0000","last_unstale":"2024-03-26T17:36:25.324063+0000","last_undegraded":"2024-03-26T17:36:25.324063+0000","last_fullsized":"2024-03-26T17:36:25.324063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:48:44.204889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721484+0000","last_change":"2024-03-26T17:36:22.744530+0000","last_active":"2024-03-26T17:36:23.721484+0000","last_peered":"2024-03-26T17:36:23.721484+0000","last_clean":"2024-03-26T17:36:23.721484+0000","last_became_active":"2024-03-26T17:36:22.743566+0000","last_became_peered":"2024-03-26T17:36:22.743566+0000","last_unstale":"2024-03-26T17:36:23.721484+0000","last_undegraded":"2024-03-26T17:36:23.721484+0000","last_fullsized":"2024-03-26T17:36:23.721484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:42.685326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.327626+0000","last_change":"2024-03-26T17:36:22.745797+0000","last_active":"2024-03-26T17:36:25.327626+0000","last_peered":"2024-03-26T17:36:25.327626+0000","last_clean":"2024-03-26T17:36:25.327626+0000","last_became_active":"2024-03-26T17:36:22.745448+0000","last_became_peered":"2024-03-26T17:36:22.745448+0000","last_unstale":"2024-03-26T17:36:25.327626+0000","last_undegraded":"2024-03-26T17:36:25.327626+0000","last_fullsized":"2024-03-26T17:36:25.327626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:05:29.126564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.746910+0000","last_change":"2024-03-26T17:36:22.749890+0000","last_active":"2024-03-26T17:36:23.746910+0000","last_peered":"2024-03-26T17:36:23.746910+0000","last_clean":"2024-03-26T17:36:23.746910+0000","last_became_active":"2024-03-26T17:36:22.749497+0000","last_became_peered":"2024-03-26T17:36:22.749497+0000","last_unstale":"2024-03-26T17:36:23.746910+0000","last_undegraded":"2024-03-26T17:36:23.746910+0000","last_fullsized":"2024-03-26T17:36:23.746910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:12:55.069659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723160+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:36:23.723160+0000","last_peered":"2024-03-26T17:36:23.723160+0000","last_clean":"2024-03-26T17:36:23.723160+0000","last_became_active":"2024-03-26T17:36:22.740287+0000","last_became_peered":"2024-03-26T17:36:22.740287+0000","last_unstale":"2024-03-26T17:36:23.723160+0000","last_undegraded":"2024-03-26T17:36:23.723160+0000","last_fullsized":"2024-03-26T17:36:23.723160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:13:00.574246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'11","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:27.337297+0000","last_change":"2024-03-26T17:36:21.720809+0000","last_active":"2024-03-26T17:36:27.337297+0000","last_peered":"2024-03-26T17:36:27.337297+0000","last_clean":"2024-03-26T17:36:27.337297+0000","last_became_active":"2024-03-26T17:36:21.720526+0000","last_became_peered":"2024-03-26T17:36:21.720526+0000","last_unstale":"2024-03-26T17:36:27.337297+0000","last_undegraded":"2024-03-26T17:36:27.337297+0000","last_fullsized":"2024-03-26T17:36:27.337297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:48.034328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748996+0000","last_change":"2024-03-26T17:36:22.746245+0000","last_active":"2024-03-26T17:36:23.748996+0000","last_peered":"2024-03-26T17:36:23.748996+0000","last_clean":"2024-03-26T17:36:23.748996+0000","last_became_active":"2024-03-26T17:36:22.745859+0000","last_became_peered":"2024-03-26T17:36:22.745859+0000","last_unstale":"2024-03-26T17:36:23.748996+0000","last_undegraded":"2024-03-26T17:36:23.748996+0000","last_fullsized":"2024-03-26T17:36:23.748996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:39.085668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.787641+0000","last_change":"2024-03-26T17:36:21.723178+0000","last_active":"2024-03-26T17:36:23.787641+0000","last_peered":"2024-03-26T17:36:23.787641+0000","last_clean":"2024-03-26T17:36:23.787641+0000","last_became_active":"2024-03-26T17:36:21.722858+0000","last_became_peered":"2024-03-26T17:36:21.722858+0000","last_unstale":"2024-03-26T17:36:23.787641+0000","last_undegraded":"2024-03-26T17:36:23.787641+0000","last_fullsized":"2024-03-26T17:36:23.787641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:46:13.837672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325317+0000","last_change":"2024-03-26T17:36:22.736231+0000","last_active":"2024-03-26T17:36:25.325317+0000","last_peered":"2024-03-26T17:36:25.325317+0000","last_clean":"2024-03-26T17:36:25.325317+0000","last_became_active":"2024-03-26T17:36:22.735942+0000","last_became_peered":"2024-03-26T17:36:22.735942+0000","last_unstale":"2024-03-26T17:36:25.325317+0000","last_undegraded":"2024-03-26T17:36:25.325317+0000","last_fullsized":"2024-03-26T17:36:25.325317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:49.127382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749399+0000","last_change":"2024-03-26T17:36:21.731369+0000","last_active":"2024-03-26T17:36:23.749399+0000","last_peered":"2024-03-26T17:36:23.749399+0000","last_clean":"2024-03-26T17:36:23.749399+0000","last_became_active":"2024-03-26T17:36:21.731016+0000","last_became_peered":"2024-03-26T17:36:21.731016+0000","last_unstale":"2024-03-26T17:36:23.749399+0000","last_undegraded":"2024-03-26T17:36:23.749399+0000","last_fullsized":"2024-03-26T17:36:23.749399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:37:53.204825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721699+0000","last_change":"2024-03-26T17:36:22.751155+0000","last_active":"2024-03-26T17:36:23.721699+0000","last_peered":"2024-03-26T17:36:23.721699+0000","last_clean":"2024-03-26T17:36:23.721699+0000","last_became_active":"2024-03-26T17:36:22.750704+0000","last_became_peered":"2024-03-26T17:36:22.750704+0000","last_unstale":"2024-03-26T17:36:23.721699+0000","last_undegraded":"2024-03-26T17:36:23.721699+0000","last_fullsized":"2024-03-26T17:36:23.721699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:52:24.116923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749129+0000","last_change":"2024-03-26T17:36:21.726380+0000","last_active":"2024-03-26T17:36:23.749129+0000","last_peered":"2024-03-26T17:36:23.749129+0000","last_clean":"2024-03-26T17:36:23.749129+0000","last_became_active":"2024-03-26T17:36:21.726004+0000","last_became_peered":"2024-03-26T17:36:21.726004+0000","last_unstale":"2024-03-26T17:36:23.749129+0000","last_undegraded":"2024-03-26T17:36:23.749129+0000","last_fullsized":"2024-03-26T17:36:23.749129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:51:23.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909452+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:36:23.909452+0000","last_peered":"2024-03-26T17:36:23.909452+0000","last_clean":"2024-03-26T17:36:23.909452+0000","last_became_active":"2024-03-26T17:36:22.741104+0000","last_became_peered":"2024-03-26T17:36:22.741104+0000","last_unstale":"2024-03-26T17:36:23.909452+0000","last_undegraded":"2024-03-26T17:36:23.909452+0000","last_fullsized":"2024-03-26T17:36:23.909452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:53.683944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.312302+0000","last_change":"2024-03-26T17:36:21.728824+0000","last_active":"2024-03-26T17:36:28.312302+0000","last_peered":"2024-03-26T17:36:28.312302+0000","last_clean":"2024-03-26T17:36:28.312302+0000","last_became_active":"2024-03-26T17:36:21.728467+0000","last_became_peered":"2024-03-26T17:36:21.728467+0000","last_unstale":"2024-03-26T17:36:28.312302+0000","last_undegraded":"2024-03-26T17:36:28.312302+0000","last_fullsized":"2024-03-26T17:36:28.312302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:25:07.842111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749117+0000","last_change":"2024-03-26T17:36:22.740131+0000","last_active":"2024-03-26T17:36:23.749117+0000","last_peered":"2024-03-26T17:36:23.749117+0000","last_clean":"2024-03-26T17:36:23.749117+0000","last_became_active":"2024-03-26T17:36:22.739704+0000","last_became_peered":"2024-03-26T17:36:22.739704+0000","last_unstale":"2024-03-26T17:36:23.749117+0000","last_undegraded":"2024-03-26T17:36:23.749117+0000","last_fullsized":"2024-03-26T17:36:23.749117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:47.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:26.331938+0000","last_change":"2024-03-26T17:36:21.731826+0000","last_active":"2024-03-26T17:36:26.331938+0000","last_peered":"2024-03-26T17:36:26.331938+0000","last_clean":"2024-03-26T17:36:26.331938+0000","last_became_active":"2024-03-26T17:36:21.731546+0000","last_became_peered":"2024-03-26T17:36:21.731546+0000","last_unstale":"2024-03-26T17:36:26.331938+0000","last_undegraded":"2024-03-26T17:36:26.331938+0000","last_fullsized":"2024-03-26T17:36:26.331938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:08:53.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909290+0000","last_change":"2024-03-26T17:36:22.745170+0000","last_active":"2024-03-26T17:36:23.909290+0000","last_peered":"2024-03-26T17:36:23.909290+0000","last_clean":"2024-03-26T17:36:23.909290+0000","last_became_active":"2024-03-26T17:36:22.744798+0000","last_became_peered":"2024-03-26T17:36:22.744798+0000","last_unstale":"2024-03-26T17:36:23.909290+0000","last_undegraded":"2024-03-26T17:36:23.909290+0000","last_fullsized":"2024-03-26T17:36:23.909290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:32:50.418149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:26.332093+0000","last_change":"2024-03-26T17:36:21.721492+0000","last_active":"2024-03-26T17:36:26.332093+0000","last_peered":"2024-03-26T17:36:26.332093+0000","last_clean":"2024-03-26T17:36:26.332093+0000","last_became_active":"2024-03-26T17:36:21.721194+0000","last_became_peered":"2024-03-26T17:36:21.721194+0000","last_unstale":"2024-03-26T17:36:26.332093+0000","last_undegraded":"2024-03-26T17:36:26.332093+0000","last_fullsized":"2024-03-26T17:36:26.332093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:50.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748624+0000","last_change":"2024-03-26T17:36:22.740780+0000","last_active":"2024-03-26T17:36:23.748624+0000","last_peered":"2024-03-26T17:36:23.748624+0000","last_clean":"2024-03-26T17:36:23.748624+0000","last_became_active":"2024-03-26T17:36:22.740311+0000","last_became_peered":"2024-03-26T17:36:22.740311+0000","last_unstale":"2024-03-26T17:36:23.748624+0000","last_undegraded":"2024-03-26T17:36:23.748624+0000","last_fullsized":"2024-03-26T17:36:23.748624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:01:56.018416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.308270+0000","last_change":"2024-03-26T17:36:21.726289+0000","last_active":"2024-03-26T17:36:28.308270+0000","last_peered":"2024-03-26T17:36:28.308270+0000","last_clean":"2024-03-26T17:36:28.308270+0000","last_became_active":"2024-03-26T17:36:21.725898+0000","last_became_peered":"2024-03-26T17:36:21.725898+0000","last_unstale":"2024-03-26T17:36:28.308270+0000","last_undegraded":"2024-03-26T17:36:28.308270+0000","last_fullsized":"2024-03-26T17:36:28.308270+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:03.893163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.746847+0000","last_change":"2024-03-26T17:36:22.750632+0000","last_active":"2024-03-26T17:36:23.746847+0000","last_peered":"2024-03-26T17:36:23.746847+0000","last_clean":"2024-03-26T17:36:23.746847+0000","last_became_active":"2024-03-26T17:36:22.750297+0000","last_became_peered":"2024-03-26T17:36:22.750297+0000","last_unstale":"2024-03-26T17:36:23.746847+0000","last_undegraded":"2024-03-26T17:36:23.746847+0000","last_fullsized":"2024-03-26T17:36:23.746847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:43:16.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.324986+0000","last_change":"2024-03-26T17:36:21.721403+0000","last_active":"2024-03-26T17:36:25.324986+0000","last_peered":"2024-03-26T17:36:25.324986+0000","last_clean":"2024-03-26T17:36:25.324986+0000","last_became_active":"2024-03-26T17:36:21.720962+0000","last_became_peered":"2024-03-26T17:36:21.720962+0000","last_unstale":"2024-03-26T17:36:25.324986+0000","last_undegraded":"2024-03-26T17:36:25.324986+0000","last_fullsized":"2024-03-26T17:36:25.324986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:02.683188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.722025+0000","last_change":"2024-03-26T17:36:22.754010+0000","last_active":"2024-03-26T17:36:23.722025+0000","last_peered":"2024-03-26T17:36:23.722025+0000","last_clean":"2024-03-26T17:36:23.722025+0000","last_became_active":"2024-03-26T17:36:22.753534+0000","last_became_peered":"2024-03-26T17:36:22.753534+0000","last_unstale":"2024-03-26T17:36:23.722025+0000","last_undegraded":"2024-03-26T17:36:23.722025+0000","last_fullsized":"2024-03-26T17:36:23.722025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:06.490733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326780+0000","last_change":"2024-03-26T17:36:21.721685+0000","last_active":"2024-03-26T17:36:25.326780+0000","last_peered":"2024-03-26T17:36:25.326780+0000","last_clean":"2024-03-26T17:36:25.326780+0000","last_became_active":"2024-03-26T17:36:21.721387+0000","last_became_peered":"2024-03-26T17:36:21.721387+0000","last_unstale":"2024-03-26T17:36:25.326780+0000","last_undegraded":"2024-03-26T17:36:25.326780+0000","last_fullsized":"2024-03-26T17:36:25.326780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:59.493172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325170+0000","last_change":"2024-03-26T17:36:22.738821+0000","last_active":"2024-03-26T17:36:25.325170+0000","last_peered":"2024-03-26T17:36:25.325170+0000","last_clean":"2024-03-26T17:36:25.325170+0000","last_became_active":"2024-03-26T17:36:22.738476+0000","last_became_peered":"2024-03-26T17:36:22.738476+0000","last_unstale":"2024-03-26T17:36:25.325170+0000","last_undegraded":"2024-03-26T17:36:25.325170+0000","last_fullsized":"2024-03-26T17:36:25.325170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:29:57.568099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:27.338385+0000","last_change":"2024-03-26T17:36:21.724041+0000","last_active":"2024-03-26T17:36:27.338385+0000","last_peered":"2024-03-26T17:36:27.338385+0000","last_clean":"2024-03-26T17:36:27.338385+0000","last_became_active":"2024-03-26T17:36:21.723624+0000","last_became_peered":"2024-03-26T17:36:21.723624+0000","last_unstale":"2024-03-26T17:36:27.338385+0000","last_undegraded":"2024-03-26T17:36:27.338385+0000","last_fullsized":"2024-03-26T17:36:27.338385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:42:24.264844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721771+0000","last_change":"2024-03-26T17:36:22.738592+0000","last_active":"2024-03-26T17:36:23.721771+0000","last_peered":"2024-03-26T17:36:23.721771+0000","last_clean":"2024-03-26T17:36:23.721771+0000","last_became_active":"2024-03-26T17:36:22.738236+0000","last_became_peered":"2024-03-26T17:36:22.738236+0000","last_unstale":"2024-03-26T17:36:23.721771+0000","last_undegraded":"2024-03-26T17:36:23.721771+0000","last_fullsized":"2024-03-26T17:36:23.721771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:56:06.977720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326035+0000","last_change":"2024-03-26T17:36:21.722793+0000","last_active":"2024-03-26T17:36:25.326035+0000","last_peered":"2024-03-26T17:36:25.326035+0000","last_clean":"2024-03-26T17:36:25.326035+0000","last_became_active":"2024-03-26T17:36:21.722482+0000","last_became_peered":"2024-03-26T17:36:21.722482+0000","last_unstale":"2024-03-26T17:36:25.326035+0000","last_undegraded":"2024-03-26T17:36:25.326035+0000","last_fullsized":"2024-03-26T17:36:25.326035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:48:32.107017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749339+0000","last_change":"2024-03-26T17:36:22.746488+0000","last_active":"2024-03-26T17:36:23.749339+0000","last_peered":"2024-03-26T17:36:23.749339+0000","last_clean":"2024-03-26T17:36:23.749339+0000","last_became_active":"2024-03-26T17:36:22.746013+0000","last_became_peered":"2024-03-26T17:36:22.746013+0000","last_unstale":"2024-03-26T17:36:23.749339+0000","last_undegraded":"2024-03-26T17:36:23.749339+0000","last_fullsized":"2024-03-26T17:36:23.749339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:14.792336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326885+0000","last_change":"2024-03-26T17:36:21.724319+0000","last_active":"2024-03-26T17:36:25.326885+0000","last_peered":"2024-03-26T17:36:25.326885+0000","last_clean":"2024-03-26T17:36:25.326885+0000","last_became_active":"2024-03-26T17:36:21.723644+0000","last_became_peered":"2024-03-26T17:36:21.723644+0000","last_unstale":"2024-03-26T17:36:25.326885+0000","last_undegraded":"2024-03-26T17:36:25.326885+0000","last_fullsized":"2024-03-26T17:36:25.326885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:31:01.132797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723695+0000","last_change":"2024-03-26T17:36:22.736535+0000","last_active":"2024-03-26T17:36:23.723695+0000","last_peered":"2024-03-26T17:36:23.723695+0000","last_clean":"2024-03-26T17:36:23.723695+0000","last_became_active":"2024-03-26T17:36:22.736229+0000","last_became_peered":"2024-03-26T17:36:22.736229+0000","last_unstale":"2024-03-26T17:36:23.723695+0000","last_undegraded":"2024-03-26T17:36:23.723695+0000","last_fullsized":"2024-03-26T17:36:23.723695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:03.938180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.316401+0000","last_change":"2024-03-26T17:36:21.726271+0000","last_active":"2024-03-26T17:36:28.316401+0000","last_peered":"2024-03-26T17:36:28.316401+0000","last_clean":"2024-03-26T17:36:28.316401+0000","last_became_active":"2024-03-26T17:36:21.725942+0000","last_became_peered":"2024-03-26T17:36:21.725942+0000","last_unstale":"2024-03-26T17:36:28.316401+0000","last_undegraded":"2024-03-26T17:36:28.316401+0000","last_fullsized":"2024-03-26T17:36:28.316401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:23:16.320106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705681+0000","last_change":"2024-03-26T17:36:22.743480+0000","last_active":"2024-03-26T17:36:24.705681+0000","last_peered":"2024-03-26T17:36:24.705681+0000","last_clean":"2024-03-26T17:36:24.705681+0000","last_became_active":"2024-03-26T17:36:22.742982+0000","last_became_peered":"2024-03-26T17:36:22.742982+0000","last_unstale":"2024-03-26T17:36:24.705681+0000","last_undegraded":"2024-03-26T17:36:24.705681+0000","last_fullsized":"2024-03-26T17:36:24.705681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:47.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.315773+0000","last_change":"2024-03-26T17:36:21.723407+0000","last_active":"2024-03-26T17:36:28.315773+0000","last_peered":"2024-03-26T17:36:28.315773+0000","last_clean":"2024-03-26T17:36:28.315773+0000","last_became_active":"2024-03-26T17:36:21.723086+0000","last_became_peered":"2024-03-26T17:36:21.723086+0000","last_unstale":"2024-03-26T17:36:28.315773+0000","last_undegraded":"2024-03-26T17:36:28.315773+0000","last_fullsized":"2024-03-26T17:36:28.315773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:26:57.886388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705465+0000","last_change":"2024-03-26T17:36:22.740705+0000","last_active":"2024-03-26T17:36:24.705465+0000","last_peered":"2024-03-26T17:36:24.705465+0000","last_clean":"2024-03-26T17:36:24.705465+0000","last_became_active":"2024-03-26T17:36:22.740360+0000","last_became_peered":"2024-03-26T17:36:22.740360+0000","last_unstale":"2024-03-26T17:36:24.705465+0000","last_undegraded":"2024-03-26T17:36:24.705465+0000","last_fullsized":"2024-03-26T17:36:24.705465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:02.879022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325945+0000","last_change":"2024-03-26T17:36:21.731427+0000","last_active":"2024-03-26T17:36:25.325945+0000","last_peered":"2024-03-26T17:36:25.325945+0000","last_clean":"2024-03-26T17:36:25.325945+0000","last_became_active":"2024-03-26T17:36:21.731124+0000","last_became_peered":"2024-03-26T17:36:21.731124+0000","last_unstale":"2024-03-26T17:36:25.325945+0000","last_undegraded":"2024-03-26T17:36:25.325945+0000","last_fullsized":"2024-03-26T17:36:25.325945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:10.084577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909251+0000","last_change":"2024-03-26T17:36:22.743791+0000","last_active":"2024-03-26T17:36:23.909251+0000","last_peered":"2024-03-26T17:36:23.909251+0000","last_clean":"2024-03-26T17:36:23.909251+0000","last_became_active":"2024-03-26T17:36:22.742780+0000","last_became_peered":"2024-03-26T17:36:22.742780+0000","last_unstale":"2024-03-26T17:36:23.909251+0000","last_undegraded":"2024-03-26T17:36:23.909251+0000","last_fullsized":"2024-03-26T17:36:23.909251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:04.387958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749382+0000","last_change":"2024-03-26T17:36:21.724289+0000","last_active":"2024-03-26T17:36:23.749382+0000","last_peered":"2024-03-26T17:36:23.749382+0000","last_clean":"2024-03-26T17:36:23.749382+0000","last_became_active":"2024-03-26T17:36:21.723847+0000","last_became_peered":"2024-03-26T17:36:21.723847+0000","last_unstale":"2024-03-26T17:36:23.749382+0000","last_undegraded":"2024-03-26T17:36:23.749382+0000","last_fullsized":"2024-03-26T17:36:23.749382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:01:59.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326947+0000","last_change":"2024-03-26T17:36:22.748323+0000","last_active":"2024-03-26T17:36:25.326947+0000","last_peered":"2024-03-26T17:36:25.326947+0000","last_clean":"2024-03-26T17:36:25.326947+0000","last_became_active":"2024-03-26T17:36:22.747970+0000","last_became_peered":"2024-03-26T17:36:22.747970+0000","last_unstale":"2024-03-26T17:36:25.326947+0000","last_undegraded":"2024-03-26T17:36:25.326947+0000","last_fullsized":"2024-03-26T17:36:25.326947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:58.886425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.746910+0000","last_change":"2024-03-26T17:36:21.726076+0000","last_active":"2024-03-26T17:36:23.746910+0000","last_peered":"2024-03-26T17:36:23.746910+0000","last_clean":"2024-03-26T17:36:23.746910+0000","last_became_active":"2024-03-26T17:36:21.722067+0000","last_became_peered":"2024-03-26T17:36:21.722067+0000","last_unstale":"2024-03-26T17:36:23.746910+0000","last_undegraded":"2024-03-26T17:36:23.746910+0000","last_fullsized":"2024-03-26T17:36:23.746910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:34:40.316110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.955622+0000","last_change":"2024-03-26T17:36:22.730890+0000","last_active":"2024-03-26T17:36:23.955622+0000","last_peered":"2024-03-26T17:36:23.955622+0000","last_clean":"2024-03-26T17:36:23.955622+0000","last_became_active":"2024-03-26T17:36:22.729635+0000","last_became_peered":"2024-03-26T17:36:22.729635+0000","last_unstale":"2024-03-26T17:36:23.955622+0000","last_undegraded":"2024-03-26T17:36:23.955622+0000","last_fullsized":"2024-03-26T17:36:23.955622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:15:09.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705644+0000","last_change":"2024-03-26T17:36:21.727587+0000","last_active":"2024-03-26T17:36:24.705644+0000","last_peered":"2024-03-26T17:36:24.705644+0000","last_clean":"2024-03-26T17:36:24.705644+0000","last_became_active":"2024-03-26T17:36:21.727302+0000","last_became_peered":"2024-03-26T17:36:21.727302+0000","last_unstale":"2024-03-26T17:36:24.705644+0000","last_undegraded":"2024-03-26T17:36:24.705644+0000","last_fullsized":"2024-03-26T17:36:24.705644+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:59.404567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.955516+0000","last_change":"2024-03-26T17:36:22.750654+0000","last_active":"2024-03-26T17:36:23.955516+0000","last_peered":"2024-03-26T17:36:23.955516+0000","last_clean":"2024-03-26T17:36:23.955516+0000","last_became_active":"2024-03-26T17:36:22.750281+0000","last_became_peered":"2024-03-26T17:36:22.750281+0000","last_unstale":"2024-03-26T17:36:23.955516+0000","last_undegraded":"2024-03-26T17:36:23.955516+0000","last_fullsized":"2024-03-26T17:36:23.955516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:04.034108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909350+0000","last_change":"2024-03-26T17:36:22.739206+0000","last_active":"2024-03-26T17:36:23.909350+0000","last_peered":"2024-03-26T17:36:23.909350+0000","last_clean":"2024-03-26T17:36:23.909350+0000","last_became_active":"2024-03-26T17:36:22.738252+0000","last_became_peered":"2024-03-26T17:36:22.738252+0000","last_unstale":"2024-03-26T17:36:23.909350+0000","last_undegraded":"2024-03-26T17:36:23.909350+0000","last_fullsized":"2024-03-26T17:36:23.909350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:35.655111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:26.332241+0000","last_change":"2024-03-26T17:36:21.730812+0000","last_active":"2024-03-26T17:36:26.332241+0000","last_peered":"2024-03-26T17:36:26.332241+0000","last_clean":"2024-03-26T17:36:26.332241+0000","last_became_active":"2024-03-26T17:36:21.730499+0000","last_became_peered":"2024-03-26T17:36:21.730499+0000","last_unstale":"2024-03-26T17:36:26.332241+0000","last_undegraded":"2024-03-26T17:36:26.332241+0000","last_fullsized":"2024-03-26T17:36:26.332241+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:05.666146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748691+0000","last_change":"2024-03-26T17:36:22.742344+0000","last_active":"2024-03-26T17:36:23.748691+0000","last_peered":"2024-03-26T17:36:23.748691+0000","last_clean":"2024-03-26T17:36:23.748691+0000","last_became_active":"2024-03-26T17:36:22.741864+0000","last_became_peered":"2024-03-26T17:36:22.741864+0000","last_unstale":"2024-03-26T17:36:23.748691+0000","last_undegraded":"2024-03-26T17:36:23.748691+0000","last_fullsized":"2024-03-26T17:36:23.748691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:53:53.553107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.314466+0000","last_change":"2024-03-26T17:36:21.722450+0000","last_active":"2024-03-26T17:36:28.314466+0000","last_peered":"2024-03-26T17:36:28.314466+0000","last_clean":"2024-03-26T17:36:28.314466+0000","last_became_active":"2024-03-26T17:36:21.722105+0000","last_became_peered":"2024-03-26T17:36:21.722105+0000","last_unstale":"2024-03-26T17:36:28.314466+0000","last_undegraded":"2024-03-26T17:36:28.314466+0000","last_fullsized":"2024-03-26T17:36:28.314466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:01:22.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":338,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705433+0000","last_change":"2024-03-26T17:35:10.663606+0000","last_active":"2024-03-26T17:36:24.705433+0000","last_peered":"2024-03-26T17:36:24.705433+0000","last_clean":"2024-03-26T17:36:24.705433+0000","last_became_active":"2024-03-26T17:35:10.663069+0000","last_became_peered":"2024-03-26T17:35:10.663069+0000","last_unstale":"2024-03-26T17:36:24.705433+0000","last_undegraded":"2024-03-26T17:36:24.705433+0000","last_fullsized":"2024-03-26T17:36:24.705433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_clean_scrub_stamp":"2024-03-26T17:31:50.880228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:50.817007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723514+0000","last_change":"2024-03-26T17:36:22.742987+0000","last_active":"2024-03-26T17:36:23.723514+0000","last_peered":"2024-03-26T17:36:23.723514+0000","last_clean":"2024-03-26T17:36:23.723514+0000","last_became_active":"2024-03-26T17:36:22.742661+0000","last_became_peered":"2024-03-26T17:36:22.742661+0000","last_unstale":"2024-03-26T17:36:23.723514+0000","last_undegraded":"2024-03-26T17:36:23.723514+0000","last_fullsized":"2024-03-26T17:36:23.723514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:37:39.837294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:26.332804+0000","last_change":"2024-03-26T17:36:21.723452+0000","last_active":"2024-03-26T17:36:26.332804+0000","last_peered":"2024-03-26T17:36:26.332804+0000","last_clean":"2024-03-26T17:36:26.332804+0000","last_became_active":"2024-03-26T17:36:21.723155+0000","last_became_peered":"2024-03-26T17:36:21.723155+0000","last_unstale":"2024-03-26T17:36:26.332804+0000","last_undegraded":"2024-03-26T17:36:26.332804+0000","last_fullsized":"2024-03-26T17:36:26.332804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:39:34.443624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.749260+0000","last_change":"2024-03-26T17:36:21.731919+0000","last_active":"2024-03-26T17:36:23.749260+0000","last_peered":"2024-03-26T17:36:23.749260+0000","last_clean":"2024-03-26T17:36:23.749260+0000","last_became_active":"2024-03-26T17:36:21.731216+0000","last_became_peered":"2024-03-26T17:36:21.731216+0000","last_unstale":"2024-03-26T17:36:23.749260+0000","last_undegraded":"2024-03-26T17:36:23.749260+0000","last_fullsized":"2024-03-26T17:36:23.749260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:47:59.538107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723896+0000","last_change":"2024-03-26T17:36:22.736487+0000","last_active":"2024-03-26T17:36:23.723896+0000","last_peered":"2024-03-26T17:36:23.723896+0000","last_clean":"2024-03-26T17:36:23.723896+0000","last_became_active":"2024-03-26T17:36:22.735788+0000","last_became_peered":"2024-03-26T17:36:22.735788+0000","last_unstale":"2024-03-26T17:36:23.723896+0000","last_undegraded":"2024-03-26T17:36:23.723896+0000","last_fullsized":"2024-03-26T17:36:23.723896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:45.240148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:27.338662+0000","last_change":"2024-03-26T17:36:21.720755+0000","last_active":"2024-03-26T17:36:27.338662+0000","last_peered":"2024-03-26T17:36:27.338662+0000","last_clean":"2024-03-26T17:36:27.338662+0000","last_became_active":"2024-03-26T17:36:21.720175+0000","last_became_peered":"2024-03-26T17:36:21.720175+0000","last_unstale":"2024-03-26T17:36:27.338662+0000","last_undegraded":"2024-03-26T17:36:27.338662+0000","last_fullsized":"2024-03-26T17:36:27.338662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:22.458657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748935+0000","last_change":"2024-03-26T17:36:22.740731+0000","last_active":"2024-03-26T17:36:23.748935+0000","last_peered":"2024-03-26T17:36:23.748935+0000","last_clean":"2024-03-26T17:36:23.748935+0000","last_became_active":"2024-03-26T17:36:22.740283+0000","last_became_peered":"2024-03-26T17:36:22.740283+0000","last_unstale":"2024-03-26T17:36:23.748935+0000","last_undegraded":"2024-03-26T17:36:23.748935+0000","last_fullsized":"2024-03-26T17:36:23.748935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:36.403105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.313196+0000","last_change":"2024-03-26T17:36:21.721513+0000","last_active":"2024-03-26T17:36:28.313196+0000","last_peered":"2024-03-26T17:36:28.313196+0000","last_clean":"2024-03-26T17:36:28.313196+0000","last_became_active":"2024-03-26T17:36:21.721185+0000","last_became_peered":"2024-03-26T17:36:21.721185+0000","last_unstale":"2024-03-26T17:36:28.313196+0000","last_undegraded":"2024-03-26T17:36:28.313196+0000","last_fullsized":"2024-03-26T17:36:28.313196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:48:39.994980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325712+0000","last_change":"2024-03-26T17:36:22.737938+0000","last_active":"2024-03-26T17:36:25.325712+0000","last_peered":"2024-03-26T17:36:25.325712+0000","last_clean":"2024-03-26T17:36:25.325712+0000","last_became_active":"2024-03-26T17:36:22.737605+0000","last_became_peered":"2024-03-26T17:36:22.737605+0000","last_unstale":"2024-03-26T17:36:25.325712+0000","last_undegraded":"2024-03-26T17:36:25.325712+0000","last_fullsized":"2024-03-26T17:36:25.325712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:27.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325890+0000","last_change":"2024-03-26T17:36:21.720942+0000","last_active":"2024-03-26T17:36:25.325890+0000","last_peered":"2024-03-26T17:36:25.325890+0000","last_clean":"2024-03-26T17:36:25.325890+0000","last_became_active":"2024-03-26T17:36:21.720665+0000","last_became_peered":"2024-03-26T17:36:21.720665+0000","last_unstale":"2024-03-26T17:36:25.325890+0000","last_undegraded":"2024-03-26T17:36:25.325890+0000","last_fullsized":"2024-03-26T17:36:25.325890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:00.725213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748972+0000","last_change":"2024-03-26T17:36:22.747111+0000","last_active":"2024-03-26T17:36:23.748972+0000","last_peered":"2024-03-26T17:36:23.748972+0000","last_clean":"2024-03-26T17:36:23.748972+0000","last_became_active":"2024-03-26T17:36:22.746626+0000","last_became_peered":"2024-03-26T17:36:22.746626+0000","last_unstale":"2024-03-26T17:36:23.748972+0000","last_undegraded":"2024-03-26T17:36:23.748972+0000","last_fullsized":"2024-03-26T17:36:23.748972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:29:24.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.314430+0000","last_change":"2024-03-26T17:36:21.724141+0000","last_active":"2024-03-26T17:36:28.314430+0000","last_peered":"2024-03-26T17:36:28.314430+0000","last_clean":"2024-03-26T17:36:28.314430+0000","last_became_active":"2024-03-26T17:36:21.723797+0000","last_became_peered":"2024-03-26T17:36:21.723797+0000","last_unstale":"2024-03-26T17:36:28.314430+0000","last_undegraded":"2024-03-26T17:36:28.314430+0000","last_fullsized":"2024-03-26T17:36:28.314430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:06.149827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326266+0000","last_change":"2024-03-26T17:36:22.739497+0000","last_active":"2024-03-26T17:36:25.326266+0000","last_peered":"2024-03-26T17:36:25.326266+0000","last_clean":"2024-03-26T17:36:25.326266+0000","last_became_active":"2024-03-26T17:36:22.739114+0000","last_became_peered":"2024-03-26T17:36:22.739114+0000","last_unstale":"2024-03-26T17:36:25.326266+0000","last_undegraded":"2024-03-26T17:36:25.326266+0000","last_fullsized":"2024-03-26T17:36:25.326266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:03.953199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.722642+0000","last_change":"2024-03-26T17:36:21.728027+0000","last_active":"2024-03-26T17:36:23.722642+0000","last_peered":"2024-03-26T17:36:23.722642+0000","last_clean":"2024-03-26T17:36:23.722642+0000","last_became_active":"2024-03-26T17:36:21.727159+0000","last_became_peered":"2024-03-26T17:36:21.727159+0000","last_unstale":"2024-03-26T17:36:23.722642+0000","last_undegraded":"2024-03-26T17:36:23.722642+0000","last_fullsized":"2024-03-26T17:36:23.722642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:39.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748834+0000","last_change":"2024-03-26T17:36:22.733666+0000","last_active":"2024-03-26T17:36:23.748834+0000","last_peered":"2024-03-26T17:36:23.748834+0000","last_clean":"2024-03-26T17:36:23.748834+0000","last_became_active":"2024-03-26T17:36:22.733313+0000","last_became_peered":"2024-03-26T17:36:22.733313+0000","last_unstale":"2024-03-26T17:36:23.748834+0000","last_undegraded":"2024-03-26T17:36:23.748834+0000","last_fullsized":"2024-03-26T17:36:23.748834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:52:38.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.325919+0000","last_change":"2024-03-26T17:36:21.730828+0000","last_active":"2024-03-26T17:36:25.325919+0000","last_peered":"2024-03-26T17:36:25.325919+0000","last_clean":"2024-03-26T17:36:25.325919+0000","last_became_active":"2024-03-26T17:36:21.730508+0000","last_became_peered":"2024-03-26T17:36:21.730508+0000","last_unstale":"2024-03-26T17:36:25.325919+0000","last_undegraded":"2024-03-26T17:36:25.325919+0000","last_fullsized":"2024-03-26T17:36:25.325919+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:10:53.492725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705800+0000","last_change":"2024-03-26T17:36:22.740204+0000","last_active":"2024-03-26T17:36:24.705800+0000","last_peered":"2024-03-26T17:36:24.705800+0000","last_clean":"2024-03-26T17:36:24.705800+0000","last_became_active":"2024-03-26T17:36:22.735880+0000","last_became_peered":"2024-03-26T17:36:22.735880+0000","last_unstale":"2024-03-26T17:36:24.705800+0000","last_undegraded":"2024-03-26T17:36:24.705800+0000","last_fullsized":"2024-03-26T17:36:24.705800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:34.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723004+0000","last_change":"2024-03-26T17:36:21.720873+0000","last_active":"2024-03-26T17:36:23.723004+0000","last_peered":"2024-03-26T17:36:23.723004+0000","last_clean":"2024-03-26T17:36:23.723004+0000","last_became_active":"2024-03-26T17:36:21.720496+0000","last_became_peered":"2024-03-26T17:36:21.720496+0000","last_unstale":"2024-03-26T17:36:23.723004+0000","last_undegraded":"2024-03-26T17:36:23.723004+0000","last_fullsized":"2024-03-26T17:36:23.723004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:40:55.959064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705833+0000","last_change":"2024-03-26T17:36:22.735528+0000","last_active":"2024-03-26T17:36:24.705833+0000","last_peered":"2024-03-26T17:36:24.705833+0000","last_clean":"2024-03-26T17:36:24.705833+0000","last_became_active":"2024-03-26T17:36:22.735147+0000","last_became_peered":"2024-03-26T17:36:22.735147+0000","last_unstale":"2024-03-26T17:36:24.705833+0000","last_undegraded":"2024-03-26T17:36:24.705833+0000","last_fullsized":"2024-03-26T17:36:24.705833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:15.729853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:28.305262+0000","last_change":"2024-03-26T17:36:21.723246+0000","last_active":"2024-03-26T17:36:28.305262+0000","last_peered":"2024-03-26T17:36:28.305262+0000","last_clean":"2024-03-26T17:36:28.305262+0000","last_became_active":"2024-03-26T17:36:21.722839+0000","last_became_peered":"2024-03-26T17:36:21.722839+0000","last_unstale":"2024-03-26T17:36:28.305262+0000","last_undegraded":"2024-03-26T17:36:28.305262+0000","last_fullsized":"2024-03-26T17:36:28.305262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:41:05.540206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705650+0000","last_change":"2024-03-26T17:36:22.742469+0000","last_active":"2024-03-26T17:36:24.705650+0000","last_peered":"2024-03-26T17:36:24.705650+0000","last_clean":"2024-03-26T17:36:24.705650+0000","last_became_active":"2024-03-26T17:36:22.742073+0000","last_became_peered":"2024-03-26T17:36:22.742073+0000","last_unstale":"2024-03-26T17:36:24.705650+0000","last_undegraded":"2024-03-26T17:36:24.705650+0000","last_fullsized":"2024-03-26T17:36:24.705650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:19:33.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723951+0000","last_change":"2024-03-26T17:36:21.730796+0000","last_active":"2024-03-26T17:36:23.723951+0000","last_peered":"2024-03-26T17:36:23.723951+0000","last_clean":"2024-03-26T17:36:23.723951+0000","last_became_active":"2024-03-26T17:36:21.730462+0000","last_became_peered":"2024-03-26T17:36:21.730462+0000","last_unstale":"2024-03-26T17:36:23.723951+0000","last_undegraded":"2024-03-26T17:36:23.723951+0000","last_fullsized":"2024-03-26T17:36:23.723951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:47:02.582630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:24.705885+0000","last_change":"2024-03-26T17:36:22.742614+0000","last_active":"2024-03-26T17:36:24.705885+0000","last_peered":"2024-03-26T17:36:24.705885+0000","last_clean":"2024-03-26T17:36:24.705885+0000","last_became_active":"2024-03-26T17:36:22.742278+0000","last_became_peered":"2024-03-26T17:36:22.742278+0000","last_unstale":"2024-03-26T17:36:24.705885+0000","last_undegraded":"2024-03-26T17:36:24.705885+0000","last_fullsized":"2024-03-26T17:36:24.705885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:30:14.503474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.327118+0000","last_change":"2024-03-26T17:36:22.741693+0000","last_active":"2024-03-26T17:36:25.327118+0000","last_peered":"2024-03-26T17:36:25.327118+0000","last_clean":"2024-03-26T17:36:25.327118+0000","last_became_active":"2024-03-26T17:36:22.741352+0000","last_became_peered":"2024-03-26T17:36:22.741352+0000","last_unstale":"2024-03-26T17:36:25.327118+0000","last_undegraded":"2024-03-26T17:36:25.327118+0000","last_fullsized":"2024-03-26T17:36:25.327118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:13:26.912910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.722946+0000","last_change":"2024-03-26T17:36:22.742939+0000","last_active":"2024-03-26T17:36:23.722946+0000","last_peered":"2024-03-26T17:36:23.722946+0000","last_clean":"2024-03-26T17:36:23.722946+0000","last_became_active":"2024-03-26T17:36:22.742589+0000","last_became_peered":"2024-03-26T17:36:22.742589+0000","last_unstale":"2024-03-26T17:36:23.722946+0000","last_undegraded":"2024-03-26T17:36:23.722946+0000","last_fullsized":"2024-03-26T17:36:23.722946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:21:14.043601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.722897+0000","last_change":"2024-03-26T17:36:22.748044+0000","last_active":"2024-03-26T17:36:23.722897+0000","last_peered":"2024-03-26T17:36:23.722897+0000","last_clean":"2024-03-26T17:36:23.722897+0000","last_became_active":"2024-03-26T17:36:22.747683+0000","last_became_peered":"2024-03-26T17:36:22.747683+0000","last_unstale":"2024-03-26T17:36:23.722897+0000","last_undegraded":"2024-03-26T17:36:23.722897+0000","last_fullsized":"2024-03-26T17:36:23.722897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:32:40.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.324949+0000","last_change":"2024-03-26T17:36:22.743080+0000","last_active":"2024-03-26T17:36:25.324949+0000","last_peered":"2024-03-26T17:36:25.324949+0000","last_clean":"2024-03-26T17:36:25.324949+0000","last_became_active":"2024-03-26T17:36:22.742676+0000","last_became_peered":"2024-03-26T17:36:22.742676+0000","last_unstale":"2024-03-26T17:36:25.324949+0000","last_undegraded":"2024-03-26T17:36:25.324949+0000","last_fullsized":"2024-03-26T17:36:25.324949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:58.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723838+0000","last_change":"2024-03-26T17:36:22.737908+0000","last_active":"2024-03-26T17:36:23.723838+0000","last_peered":"2024-03-26T17:36:23.723838+0000","last_clean":"2024-03-26T17:36:23.723838+0000","last_became_active":"2024-03-26T17:36:22.737055+0000","last_became_peered":"2024-03-26T17:36:22.737055+0000","last_unstale":"2024-03-26T17:36:23.723838+0000","last_undegraded":"2024-03-26T17:36:23.723838+0000","last_fullsized":"2024-03-26T17:36:23.723838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:51:13.157901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326639+0000","last_change":"2024-03-26T17:36:22.749795+0000","last_active":"2024-03-26T17:36:25.326639+0000","last_peered":"2024-03-26T17:36:25.326639+0000","last_clean":"2024-03-26T17:36:25.326639+0000","last_became_active":"2024-03-26T17:36:22.749469+0000","last_became_peered":"2024-03-26T17:36:22.749469+0000","last_unstale":"2024-03-26T17:36:25.326639+0000","last_undegraded":"2024-03-26T17:36:25.326639+0000","last_fullsized":"2024-03-26T17:36:25.326639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:19:59.126270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723356+0000","last_change":"2024-03-26T17:36:22.744101+0000","last_active":"2024-03-26T17:36:23.723356+0000","last_peered":"2024-03-26T17:36:23.723356+0000","last_clean":"2024-03-26T17:36:23.723356+0000","last_became_active":"2024-03-26T17:36:22.743761+0000","last_became_peered":"2024-03-26T17:36:22.743761+0000","last_unstale":"2024-03-26T17:36:23.723356+0000","last_undegraded":"2024-03-26T17:36:23.723356+0000","last_fullsized":"2024-03-26T17:36:23.723356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:27:19.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748926+0000","last_change":"2024-03-26T17:36:22.744800+0000","last_active":"2024-03-26T17:36:23.748926+0000","last_peered":"2024-03-26T17:36:23.748926+0000","last_clean":"2024-03-26T17:36:23.748926+0000","last_became_active":"2024-03-26T17:36:22.744437+0000","last_became_peered":"2024-03-26T17:36:22.744437+0000","last_unstale":"2024-03-26T17:36:23.748926+0000","last_undegraded":"2024-03-26T17:36:23.748926+0000","last_fullsized":"2024-03-26T17:36:23.748926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:36.459072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.909372+0000","last_change":"2024-03-26T17:36:22.739421+0000","last_active":"2024-03-26T17:36:23.909372+0000","last_peered":"2024-03-26T17:36:23.909372+0000","last_clean":"2024-03-26T17:36:23.909372+0000","last_became_active":"2024-03-26T17:36:22.738837+0000","last_became_peered":"2024-03-26T17:36:22.738837+0000","last_unstale":"2024-03-26T17:36:23.909372+0000","last_undegraded":"2024-03-26T17:36:23.909372+0000","last_fullsized":"2024-03-26T17:36:23.909372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:42.907666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.748612+0000","last_change":"2024-03-26T17:36:22.743211+0000","last_active":"2024-03-26T17:36:23.748612+0000","last_peered":"2024-03-26T17:36:23.748612+0000","last_clean":"2024-03-26T17:36:23.748612+0000","last_became_active":"2024-03-26T17:36:22.742869+0000","last_became_peered":"2024-03-26T17:36:22.742869+0000","last_unstale":"2024-03-26T17:36:23.748612+0000","last_undegraded":"2024-03-26T17:36:23.748612+0000","last_fullsized":"2024-03-26T17:36:23.748612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:24:04.610012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.721643+0000","last_change":"2024-03-26T17:36:22.746950+0000","last_active":"2024-03-26T17:36:23.721643+0000","last_peered":"2024-03-26T17:36:23.721643+0000","last_clean":"2024-03-26T17:36:23.721643+0000","last_became_active":"2024-03-26T17:36:22.744869+0000","last_became_peered":"2024-03-26T17:36:22.744869+0000","last_unstale":"2024-03-26T17:36:23.721643+0000","last_undegraded":"2024-03-26T17:36:23.721643+0000","last_fullsized":"2024-03-26T17:36:23.721643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:09:52.283519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:23.723768+0000","last_change":"2024-03-26T17:36:22.744627+0000","last_active":"2024-03-26T17:36:23.723768+0000","last_peered":"2024-03-26T17:36:23.723768+0000","last_clean":"2024-03-26T17:36:23.723768+0000","last_became_active":"2024-03-26T17:36:22.744299+0000","last_became_peered":"2024-03-26T17:36:22.744299+0000","last_unstale":"2024-03-26T17:36:23.723768+0000","last_undegraded":"2024-03-26T17:36:23.723768+0000","last_fullsized":"2024-03-26T17:36:23.723768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:23:37.941625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T17:36:25.326278+0000","last_change":"2024-03-26T17:36:22.751135+0000","last_active":"2024-03-26T17:36:25.326278+0000","last_peered":"2024-03-26T17:36:25.326278+0000","last_clean":"2024-03-26T17:36:25.326278+0000","last_became_active":"2024-03-26T17:36:22.750768+0000","last_became_peered":"2024-03-26T17:36:22.750768+0000","last_unstale":"2024-03-26T17:36:25.326278+0000","last_undegraded":"2024-03-26T17:36:25.326278+0000","last_fullsized":"2024-03-26T17:36:25.326278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:15.532367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5256,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":140,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":18191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122,"ondisk_log_size":122,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383695,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645329,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28524,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716628,"statfs":{"total":95995035648,"available":95965827072,"internally_reserved":0,"allocated":1544192,"data_stored":1030979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":438951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1030875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":438155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":439521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":439689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":438473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":438743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32692,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712460,"statfs":{"total":95995035648,"available":95961559040,"internally_reserved":0,"allocated":1552384,"data_stored":1031231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T17:36:31.170 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-26T17:36:31.171 INFO:tasks.cephadm:Setting up client nodes... 2024-03-26T17:36:31.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T17:36:31.458 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:31.580 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:31 smithi105 ceph-mon[32280]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:31.580 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:31 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1299831588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:31.580 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:31 smithi105 ceph-mon[32280]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.9 KiB/s wr, 19 op/s 2024-03-26T17:36:31.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:31 smithi144 ceph-mon[36095]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:31.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:31 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1299831588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:31.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:31 smithi144 ceph-mon[36095]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.9 KiB/s wr, 19 op/s 2024-03-26T17:36:31.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:31 smithi135 ceph-mon[35636]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:36:31.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:31 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1299831588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:31.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:31 smithi135 ceph-mon[35636]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.9 KiB/s wr, 19 op/s 2024-03-26T17:36:32.584 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:32 smithi105 ceph-mon[32280]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:36:32.584 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:32 smithi105 ceph-mon[32280]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:36:32.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:32 smithi144 ceph-mon[36095]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:36:32.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:32 smithi144 ceph-mon[36095]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:36:32.660 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:32 smithi135 ceph-mon[35636]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:36:32.660 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:32 smithi135 ceph-mon[35636]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:36:32.916 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:32 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:36:32.666+0000 7f463e5f8700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-26T17:36:32.916 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:36:32 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:36:32.666+0000 7f463e5f8700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-26T17:36:33.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.075+0000 7fcc19129700 1 -- 172.21.15.105:0/355045536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14101cb0 msgr2=0x7fcc14102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.075+0000 7fcc19129700 1 --2- 172.21.15.105:0/355045536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14101cb0 0x7fcc14102110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800aad0 tx=0x7fcc0800ade0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.075+0000 7fcc19129700 1 -- 172.21.15.105:0/355045536 shutdown_connections 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.075+0000 7fcc19129700 1 --2- 172.21.15.105:0/355045536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14102650 0x7fcc14108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.075+0000 7fcc19129700 1 --2- 172.21.15.105:0/355045536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc14100ab0 0x7fcc14100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.075+0000 7fcc19129700 1 --2- 172.21.15.105:0/355045536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14101cb0 0x7fcc14102110 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.075+0000 7fcc19129700 1 -- 172.21.15.105:0/355045536 >> 172.21.15.105:0/355045536 conn(0x7fcc140fc050 msgr2=0x7fcc140fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.075+0000 7fcc19129700 1 -- 172.21.15.105:0/355045536 shutdown_connections 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.076+0000 7fcc19129700 1 -- 172.21.15.105:0/355045536 wait complete. 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.076+0000 7fcc19129700 1 Processor -- start 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.076+0000 7fcc19129700 1 -- start start 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc19129700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14100ab0 0x7fcc141984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:33.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc19129700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc14101cb0 0x7fcc14198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc19129700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14102650 0x7fcc1419fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc19129700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1410bc90 con 0x7fcc14102650 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc19129700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1410bb10 con 0x7fcc14101cb0 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc19129700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1410be10 con 0x7fcc14100ab0 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc1359e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14102650 0x7fcc1419fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc1359e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14102650 0x7fcc1419fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49410/0 (socket says 172.21.15.105:49410) 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc12d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14100ab0 0x7fcc141984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc1359e700 1 -- 172.21.15.105:0/1800992906 learned_addr learned my addr 172.21.15.105:0/1800992906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc1259c700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc14101cb0 0x7fcc14198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc12d9d700 1 -- 172.21.15.105:0/1800992906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc14101cb0 msgr2=0x7fcc14198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc12d9d700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc14101cb0 0x7fcc14198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc12d9d700 1 -- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14102650 msgr2=0x7fcc1419fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc12d9d700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14102650 0x7fcc1419fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc12d9d700 1 -- 172.21.15.105:0/1800992906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc141a00d0 con 0x7fcc14100ab0 2024-03-26T17:36:33.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc1359e700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14102650 0x7fcc1419fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:33.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc1259c700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc14101cb0 0x7fcc14198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:33.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc12d9d700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14100ab0 0x7fcc141984c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400d750 tx=0x7fcc0400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:33.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcbfbfff700 1 -- 172.21.15.105:0/1800992906 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04014030 con 0x7fcc14100ab0 2024-03-26T17:36:33.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc141a03c0 con 0x7fcc14100ab0 2024-03-26T17:36:33.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.077+0000 7fcbfbfff700 1 -- 172.21.15.105:0/1800992906 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc0400fb40 con 0x7fcc14100ab0 2024-03-26T17:36:33.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.078+0000 7fcbfbfff700 1 -- 172.21.15.105:0/1800992906 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04010cc0 con 0x7fcc14100ab0 2024-03-26T17:36:33.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.078+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc141a0900 con 0x7fcc14100ab0 2024-03-26T17:36:33.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.079+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc140508c0 con 0x7fcc14100ab0 2024-03-26T17:36:33.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.079+0000 7fcbfbfff700 1 -- 172.21.15.105:0/1800992906 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fcc04017030 con 0x7fcc14100ab0 2024-03-26T17:36:33.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.080+0000 7fcbfbfff700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbfc0a30f0 0x7fcbfc0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:33.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.080+0000 7fcc1259c700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbfc0a30f0 0x7fcbfc0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:33.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.080+0000 7fcbfbfff700 1 -- 172.21.15.105:0/1800992906 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fcc040cee50 con 0x7fcc14100ab0 2024-03-26T17:36:33.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.080+0000 7fcc1259c700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbfc0a30f0 0x7fcbfc0a55b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcc08000c00 tx=0x7fcc08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.084+0000 7fcbfbfff700 1 -- 172.21.15.105:0/1800992906 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcc0409bc10 con 0x7fcc14100ab0 2024-03-26T17:36:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.390+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fcc141a0cc0 con 0x7fcc14100ab0 2024-03-26T17:36:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.397+0000 7fcbfbfff700 1 -- 172.21.15.105:0/1800992906 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fcc0409b510 con 0x7fcc14100ab0 2024-03-26T17:36:33.391 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-03-26T17:36:33.391 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQChBwNm5a5oFxAAUWQuINBMTHPx+4AkoUJSyA== 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbfc0a30f0 msgr2=0x7fcbfc0a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbfc0a30f0 0x7fcbfc0a55b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcc08000c00 tx=0x7fcc08005fb0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14100ab0 msgr2=0x7fcc141984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14100ab0 0x7fcc141984c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400d750 tx=0x7fcc0400da60 comp rx=0 tx=0).stop 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 shutdown_connections 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbfc0a30f0 0x7fcbfc0a55b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc14102650 0x7fcc1419fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14100ab0 0x7fcc141984c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 --2- 172.21.15.105:0/1800992906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc14101cb0 0x7fcc14198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 >> 172.21.15.105:0/1800992906 conn(0x7fcc140fc050 msgr2=0x7fcc141070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 shutdown_connections 2024-03-26T17:36:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:33.399+0000 7fcc19129700 1 -- 172.21.15.105:0/1800992906 wait complete. 2024-03-26T17:36:33.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:33 smithi144 ceph-mon[36095]: pgmap v288: 97 pgs: 97 active+clean; 620 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.4 KiB/s wr, 26 op/s 2024-03-26T17:36:33.642 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:33 smithi105 ceph-mon[32280]: pgmap v288: 97 pgs: 97 active+clean; 620 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.4 KiB/s wr, 26 op/s 2024-03-26T17:36:33.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:33 smithi135 ceph-mon[35636]: pgmap v288: 97 pgs: 97 active+clean; 620 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.4 KiB/s wr, 26 op/s 2024-03-26T17:36:34.028 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:36:34.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-26T17:36:34.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-26T17:36:34.070 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-26T17:36:34.071 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-26T17:36:34.071 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-26T17:36:34.356 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:34.598 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:34 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1800992906' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T17:36:34.598 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:34 smithi105 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T17:36:34.598 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:34 smithi105 ceph-mon[32280]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T17:36:34.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:34 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1800992906' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T17:36:34.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:34 smithi144 ceph-mon[36095]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T17:36:34.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:34 smithi144 ceph-mon[36095]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T17:36:34.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:34 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1800992906' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T17:36:34.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:34 smithi135 ceph-mon[35636]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T17:36:34.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:34 smithi135 ceph-mon[35636]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T17:36:35.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:35 smithi144 ceph-mon[36095]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-03-26T17:36:35.642 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:35 smithi105 ceph-mon[32280]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-03-26T17:36:35.665 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:35 smithi135 ceph-mon[35636]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.7 KiB/s wr, 26 op/s 2024-03-26T17:36:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 -- 172.21.15.105:0/4053859731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 msgr2=0x7f358c06a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 --2- 172.21.15.105:0/4053859731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 0x7f358c06a440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f357c009990 tx=0x7f357c009ca0 comp rx=0 tx=0).stop 2024-03-26T17:36:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 -- 172.21.15.105:0/4053859731 shutdown_connections 2024-03-26T17:36:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 --2- 172.21.15.105:0/4053859731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f358c10ee60 0x7f358c111250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 --2- 172.21.15.105:0/4053859731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 0x7f358c06a440 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:35.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 --2- 172.21.15.105:0/4053859731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f358c06a980 0x7f358c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 -- 172.21.15.105:0/4053859731 >> 172.21.15.105:0/4053859731 conn(0x7f358c0781e0 msgr2=0x7f358c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 -- 172.21.15.105:0/4053859731 shutdown_connections 2024-03-26T17:36:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 -- 172.21.15.105:0/4053859731 wait complete. 2024-03-26T17:36:35.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.889+0000 7f3591019700 1 Processor -- start 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f3591019700 1 -- start start 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f3591019700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 0x7f358c1a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f3591019700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f358c06a980 0x7f358c1a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f3591019700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f358c10ee60 0x7f358c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f3591019700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f358c1147d0 con 0x7f358c06a060 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f3591019700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f358c114650 con 0x7f358c06a980 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f3591019700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f358c114950 con 0x7f358c10ee60 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358ad9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 0x7f358c1a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358a59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f358c06a980 0x7f358c1a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358a59c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f358c06a980 0x7f358c1a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41570/0 (socket says 172.21.15.105:41570) 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358a59c700 1 -- 172.21.15.105:0/1855570154 learned_addr learned my addr 172.21.15.105:0/1855570154 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358b59e700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f358c10ee60 0x7f358c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:35.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358ad9d700 1 -- 172.21.15.105:0/1855570154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f358c10ee60 msgr2=0x7f358c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358ad9d700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f358c10ee60 0x7f358c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358ad9d700 1 -- 172.21.15.105:0/1855570154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f358c06a980 msgr2=0x7f358c1a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358ad9d700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f358c06a980 0x7f358c1a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.890+0000 7f358ad9d700 1 -- 172.21.15.105:0/1855570154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c19b270 con 0x7f358c06a060 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.891+0000 7f358ad9d700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 0x7f358c1a0900 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f357c000c00 tx=0x7f357c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.891+0000 7f3573fff700 1 -- 172.21.15.105:0/1855570154 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c003680 con 0x7f358c06a060 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.891+0000 7f3573fff700 1 -- 172.21.15.105:0/1855570154 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f357c004ce0 con 0x7f358c06a060 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.891+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c19b500 con 0x7f358c06a060 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.891+0000 7f3573fff700 1 -- 172.21.15.105:0/1855570154 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c003cc0 con 0x7f358c06a060 2024-03-26T17:36:35.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.891+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c19b930 con 0x7f358c06a060 2024-03-26T17:36:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.892+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c069070 con 0x7f358c06a060 2024-03-26T17:36:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.893+0000 7f3573fff700 1 -- 172.21.15.105:0/1855570154 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f357c005020 con 0x7f358c06a060 2024-03-26T17:36:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.893+0000 7f3573fff700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35740a3040 0x7f35740a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.894+0000 7f358a59c700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35740a3040 0x7f35740a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.894+0000 7f3573fff700 1 -- 172.21.15.105:0/1855570154 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f357c0d00f0 con 0x7f358c06a060 2024-03-26T17:36:35.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.894+0000 7f358a59c700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35740a3040 0x7f35740a5500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f358c100f60 tx=0x7f358000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:35.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:35.902+0000 7f3573fff700 1 -- 172.21.15.105:0/1855570154 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f357c013070 con 0x7f358c06a060 2024-03-26T17:36:36.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.231+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f358c02dc00 con 0x7f358c06a060 2024-03-26T17:36:36.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.240+0000 7f3573fff700 1 -- 172.21.15.105:0/1855570154 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263561 (secure 0 0 0) 0x7f357c09d000 con 0x7f358c06a060 2024-03-26T17:36:36.237 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35740a3040 msgr2=0x7f35740a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:36.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35740a3040 0x7f35740a5500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f358c100f60 tx=0x7f358000b500 comp rx=0 tx=0).stop 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 msgr2=0x7f358c1a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 0x7f358c1a0900 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f357c000c00 tx=0x7f357c005f10 comp rx=0 tx=0).stop 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 shutdown_connections 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35740a3040 0x7f35740a5500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f358c10ee60 0x7f358c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f358c06a060 0x7f358c1a0900 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 --2- 172.21.15.105:0/1855570154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f358c06a980 0x7f358c1a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 >> 172.21.15.105:0/1855570154 conn(0x7f358c0781e0 msgr2=0x7f358c1105f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 shutdown_connections 2024-03-26T17:36:36.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:36.243+0000 7f3591019700 1 -- 172.21.15.105:0/1855570154 wait complete. 2024-03-26T17:36:36.560 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:36 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1855570154' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T17:36:36.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:36 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1855570154' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T17:36:36.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:36 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1855570154' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T17:36:36.901 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2294330938},{"type":"v1","addr":"172.21.15.105:6801","nonce":2294330938}]},"active_addr":"172.21.15.105:6801/2294330938","active_change":"2024-03-26T17:29:26.501651+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.105:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2281139888}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1759068294}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1080855561}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":3729821015}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2807079227}]}]} 2024-03-26T17:36:36.907 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-26T17:36:36.907 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-26T17:36:36.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T17:36:37.161 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:37.598 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:37 smithi105 ceph-mon[32280]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-03-26T17:36:37.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:37 smithi135 ceph-mon[35636]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-03-26T17:36:37.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:37 smithi144 ceph-mon[36095]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 KiB/s wr, 25 op/s 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 -- 172.21.15.105:0/1039326579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1012b0 msgr2=0x7ff14c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 --2- 172.21.15.105:0/1039326579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1012b0 0x7ff14c101710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff140009990 tx=0x7ff140009ca0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 -- 172.21.15.105:0/1039326579 shutdown_connections 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 --2- 172.21.15.105:0/1039326579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff14c101c50 0x7ff14c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 --2- 172.21.15.105:0/1039326579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff14c1072f0 0x7ff14c1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 --2- 172.21.15.105:0/1039326579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1012b0 0x7ff14c101710 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 -- 172.21.15.105:0/1039326579 >> 172.21.15.105:0/1039326579 conn(0x7ff14c0781e0 msgr2=0x7ff14c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 -- 172.21.15.105:0/1039326579 shutdown_connections 2024-03-26T17:36:38.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.714+0000 7ff151245700 1 -- 172.21.15.105:0/1039326579 wait complete. 2024-03-26T17:36:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff151245700 1 Processor -- start 2024-03-26T17:36:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff151245700 1 -- start start 2024-03-26T17:36:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff151245700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff14c1012b0 0x7ff14c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:38.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff151245700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff14c101c50 0x7ff14c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff151245700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1072f0 0x7ff14c1a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff151245700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c111420 con 0x7ff14c1072f0 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff151245700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c1112a0 con 0x7ff14c1012b0 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff151245700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c1115a0 con 0x7ff14c101c50 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.715+0000 7ff14b59e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1072f0 0x7ff14c1a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff14c101c50 0x7ff14c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14ad9d700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff14c1012b0 0x7ff14c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff14c101c50 0x7ff14c199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33218/0 (socket says 172.21.15.105:33218) 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 -- 172.21.15.105:0/2289791431 learned_addr learned my addr 172.21.15.105:0/2289791431 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14b59e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1072f0 0x7ff14c1a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60210/0 (socket says 172.21.15.105:60210) 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 -- 172.21.15.105:0/2289791431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff14c1012b0 msgr2=0x7ff14c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff14c1012b0 0x7ff14c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 -- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1072f0 msgr2=0x7ff14c1a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:38.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1072f0 0x7ff14c1a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 -- 172.21.15.105:0/2289791431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c1a4f10 con 0x7ff14c101c50 2024-03-26T17:36:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14ad9d700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff14c1012b0 0x7ff14c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff14a59c700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff14c101c50 0x7ff14c199e90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff140000c00 tx=0x7ff140004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff133fff700 1 -- 172.21.15.105:0/2289791431 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140027070 con 0x7ff14c101c50 2024-03-26T17:36:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff133fff700 1 -- 172.21.15.105:0/2289791431 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff140022470 con 0x7ff14c101c50 2024-03-26T17:36:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff133fff700 1 -- 172.21.15.105:0/2289791431 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14000b610 con 0x7ff14c101c50 2024-03-26T17:36:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14c1a50e0 con 0x7ff14c101c50 2024-03-26T17:36:38.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.716+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14c1a54e0 con 0x7ff14c101c50 2024-03-26T17:36:38.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.717+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14c069070 con 0x7ff14c101c50 2024-03-26T17:36:38.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.718+0000 7ff133fff700 1 -- 172.21.15.105:0/2289791431 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7ff14002a070 con 0x7ff14c101c50 2024-03-26T17:36:38.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.719+0000 7ff133fff700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1340a3040 0x7ff1340a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:38.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.719+0000 7ff14ad9d700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1340a3040 0x7ff1340a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:38.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.719+0000 7ff133fff700 1 -- 172.21.15.105:0/2289791431 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff1400d0170 con 0x7ff14c101c50 2024-03-26T17:36:38.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.719+0000 7ff14ad9d700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1340a3040 0x7ff1340a5500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009fd0 tx=0x7ff13c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:38.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.723+0000 7ff133fff700 1 -- 172.21.15.105:0/2289791431 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff140013070 con 0x7ff14c101c50 2024-03-26T17:36:38.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.956+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff14c02ddc0 con 0x7ff14c101c50 2024-03-26T17:36:38.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.958+0000 7ff133fff700 1 -- 172.21.15.105:0/2289791431 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22254 (secure 0 0 0) 0x7ff14009cf80 con 0x7ff14c101c50 2024-03-26T17:36:38.952 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:38.952 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":85,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:36:23.718044+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1340a3040 msgr2=0x7ff1340a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1340a3040 0x7ff1340a5500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009fd0 tx=0x7ff13c009260 comp rx=0 tx=0).stop 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff14c101c50 msgr2=0x7ff14c199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff14c101c50 0x7ff14c199e90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff140000c00 tx=0x7ff140004790 comp rx=0 tx=0).stop 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 shutdown_connections 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1340a3040 0x7ff1340a5500 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff14c1072f0 0x7ff14c1a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff14c1012b0 0x7ff14c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 --2- 172.21.15.105:0/2289791431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff14c101c50 0x7ff14c199e90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:38.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 >> 172.21.15.105:0/2289791431 conn(0x7ff14c0781e0 msgr2=0x7ff14c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 shutdown_connections 2024-03-26T17:36:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:38.961+0000 7ff151245700 1 -- 172.21.15.105:0/2289791431 wait complete. 2024-03-26T17:36:39.593 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-26T17:36:39.594 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T17:36:39.848 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:39.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:39 smithi105 ceph-mon[32280]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-03-26T17:36:39.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:39 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2289791431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:39.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:39 smithi135 ceph-mon[35636]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-03-26T17:36:39.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:39 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2289791431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:40.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:39 smithi144 ceph-mon[36095]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 25 op/s 2024-03-26T17:36:40.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:39 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2289791431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:41.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 -- 172.21.15.105:0/2790248745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1012b0 msgr2=0x7ff21c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:41.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 --2- 172.21.15.105:0/2790248745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1012b0 0x7ff21c101710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff210009990 tx=0x7ff210009ca0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 -- 172.21.15.105:0/2790248745 shutdown_connections 2024-03-26T17:36:41.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 --2- 172.21.15.105:0/2790248745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff21c101c50 0x7ff21c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 --2- 172.21.15.105:0/2790248745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff21c1072f0 0x7ff21c1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 --2- 172.21.15.105:0/2790248745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1012b0 0x7ff21c101710 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 -- 172.21.15.105:0/2790248745 >> 172.21.15.105:0/2790248745 conn(0x7ff21c0781e0 msgr2=0x7ff21c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:41.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 -- 172.21.15.105:0/2790248745 shutdown_connections 2024-03-26T17:36:41.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.497+0000 7ff22212a700 1 -- 172.21.15.105:0/2790248745 wait complete. 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.498+0000 7ff22212a700 1 Processor -- start 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.498+0000 7ff22212a700 1 -- start start 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff22212a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff21c1012b0 0x7ff21c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff22212a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff21c101c50 0x7ff21c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff22212a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1072f0 0x7ff21c1a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff22212a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c111400 con 0x7ff21c1072f0 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff22212a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c111280 con 0x7ff21c101c50 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff22212a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c111580 con 0x7ff21c1012b0 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1072f0 0x7ff21c1a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff21c1012b0 0x7ff21c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:41.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff21c1012b0 0x7ff21c199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33232/0 (socket says 172.21.15.105:33232) 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21affd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff21c101c50 0x7ff21c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21b7fe700 1 -- 172.21.15.105:0/937119293 learned_addr learned my addr 172.21.15.105:0/937119293 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21bfff700 1 -- 172.21.15.105:0/937119293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff21c1012b0 msgr2=0x7ff21c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21bfff700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff21c1012b0 0x7ff21c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21bfff700 1 -- 172.21.15.105:0/937119293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff21c101c50 msgr2=0x7ff21c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21bfff700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff21c101c50 0x7ff21c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21bfff700 1 -- 172.21.15.105:0/937119293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c1a4fd0 con 0x7ff21c1072f0 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21affd700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff21c101c50 0x7ff21c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff21bfff700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1072f0 0x7ff21c1a4a90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff21400efc0 tx=0x7ff21400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.499+0000 7ff218ff9700 1 -- 172.21.15.105:0/937119293 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2140092e0 con 0x7ff21c1072f0 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.500+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c1a51a0 con 0x7ff21c1072f0 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.500+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c1a5700 con 0x7ff21c1072f0 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.500+0000 7ff218ff9700 1 -- 172.21.15.105:0/937119293 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff214004bb0 con 0x7ff21c1072f0 2024-03-26T17:36:41.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.500+0000 7ff218ff9700 1 -- 172.21.15.105:0/937119293 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214004020 con 0x7ff21c1072f0 2024-03-26T17:36:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.501+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21c069070 con 0x7ff21c1072f0 2024-03-26T17:36:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.502+0000 7ff218ff9700 1 -- 172.21.15.105:0/937119293 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7ff21401b030 con 0x7ff21c1072f0 2024-03-26T17:36:41.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.502+0000 7ff218ff9700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:41.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.503+0000 7ff21b7fe700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:41.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.503+0000 7ff218ff9700 1 -- 172.21.15.105:0/937119293 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff2140cee20 con 0x7ff21c1072f0 2024-03-26T17:36:41.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.503+0000 7ff21b7fe700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009ce0 tx=0x7ff20c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:41.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.507+0000 7ff218ff9700 1 -- 172.21.15.105:0/937119293 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff214022020 con 0x7ff21c1072f0 2024-03-26T17:36:41.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.711+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff21c02dc00 con 0x7ff21c1072f0 2024-03-26T17:36:41.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.713+0000 7ff218ff9700 1 -- 172.21.15.105:0/937119293 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22254 (secure 0 0 0) 0x7ff214010040 con 0x7ff21c1072f0 2024-03-26T17:36:41.706 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:36:41.706 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":85,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:36:23.718044+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:36:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 msgr2=0x7ff2040a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009ce0 tx=0x7ff20c009260 comp rx=0 tx=0).stop 2024-03-26T17:36:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1072f0 msgr2=0x7ff21c1a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1072f0 0x7ff21c1a4a90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff21400efc0 tx=0x7ff21400c3a0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 shutdown_connections 2024-03-26T17:36:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff21c1072f0 0x7ff21c1a4a90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff21c1012b0 0x7ff21c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 --2- 172.21.15.105:0/937119293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff21c101c50 0x7ff21c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:41.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 >> 172.21.15.105:0/937119293 conn(0x7ff21c0781e0 msgr2=0x7ff21c1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:41.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 shutdown_connections 2024-03-26T17:36:41.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:41.715+0000 7ff22212a700 1 -- 172.21.15.105:0/937119293 wait complete. 2024-03-26T17:36:41.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:41 smithi105 ceph-mon[32280]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-03-26T17:36:41.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:41 smithi135 ceph-mon[35636]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-03-26T17:36:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:41 smithi144 ceph-mon[36095]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 KiB/s wr, 21 op/s 2024-03-26T17:36:42.309 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-26T17:36:42.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-26T17:36:42.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-26T17:36:42.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-26T17:36:42.311 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-26T17:36:42.311 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-26T17:36:42.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-26T17:36:42.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-26T17:36:42.312 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-26T17:36:42.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-26T17:36:42.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-26T17:36:42.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-26T17:36:42.637 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:42 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/937119293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:42.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:42 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/937119293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:43.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:42 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/937119293' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:36:43.117 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:43.117 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:43.117 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:43.174 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:43.425 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:43.425 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:43.425 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:43.914 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:43 smithi105 ceph-mon[32280]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 19 op/s 2024-03-26T17:36:43.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:43 smithi135 ceph-mon[35636]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 19 op/s 2024-03-26T17:36:44.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:43 smithi144 ceph-mon[36095]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 19 op/s 2024-03-26T17:36:45.465 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:45.761 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:45 smithi105 ceph-mon[32280]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-26T17:36:45.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:45 smithi135 ceph-mon[35636]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-26T17:36:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:45 smithi144 ceph-mon[36095]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-03-26T17:36:47.201 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:47.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:47 smithi135 ceph-mon[35636]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-26T17:36:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:47 smithi144 ceph-mon[36095]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-26T17:36:48.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:47 smithi105 ceph-mon[32280]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-03-26T17:36:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.210+0000 7f0f0fda1700 1 -- 172.21.15.105:0/1309062505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806b500 msgr2=0x7f0f08106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.210+0000 7f0f0fda1700 1 --2- 172.21.15.105:0/1309062505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806b500 0x7f0f08106fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009990 tx=0x7f0ef8009ca0 comp rx=0 tx=0).stop 2024-03-26T17:36:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.210+0000 7f0f0fda1700 1 -- 172.21.15.105:0/1309062505 shutdown_connections 2024-03-26T17:36:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.210+0000 7f0f0fda1700 1 --2- 172.21.15.105:0/1309062505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f081076c0 0x7f0f0810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.210+0000 7f0f0fda1700 1 --2- 172.21.15.105:0/1309062505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0806ab50 0x7f0f0806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.210+0000 7f0f0fda1700 1 --2- 172.21.15.105:0/1309062505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806b500 0x7f0f08106fe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.210+0000 7f0f0fda1700 1 -- 172.21.15.105:0/1309062505 >> 172.21.15.105:0/1309062505 conn(0x7f0f08077300 msgr2=0x7f0f08077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 -- 172.21.15.105:0/1309062505 shutdown_connections 2024-03-26T17:36:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 -- 172.21.15.105:0/1309062505 wait complete. 2024-03-26T17:36:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 Processor -- start 2024-03-26T17:36:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 -- start start 2024-03-26T17:36:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806ab50 0x7f0f0819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0806b500 0x7f0f0819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f081076c0 0x7f0f08198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081124f0 con 0x7f0f0806ab50 2024-03-26T17:36:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08112370 con 0x7f0f081076c0 2024-03-26T17:36:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0fda1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08112670 con 0x7f0f0806b500 2024-03-26T17:36:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0db3d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806ab50 0x7f0f0819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:49.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0db3d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806ab50 0x7f0f0819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44814/0 (socket says 172.21.15.105:44814) 2024-03-26T17:36:49.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.211+0000 7f0f0db3d700 1 -- 172.21.15.105:0/3429845637 learned_addr learned my addr 172.21.15.105:0/3429845637 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:49.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0d33c700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0806b500 0x7f0f0819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:49.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0e33e700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f081076c0 0x7f0f08198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:49.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0db3d700 1 -- 172.21.15.105:0/3429845637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0806b500 msgr2=0x7f0f0819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:49.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0db3d700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0806b500 0x7f0f0819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:49.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0db3d700 1 -- 172.21.15.105:0/3429845637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f081076c0 msgr2=0x7f0f08198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:49.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0db3d700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f081076c0 0x7f0f08198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:49.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0db3d700 1 -- 172.21.15.105:0/3429845637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f08199010 con 0x7f0f0806ab50 2024-03-26T17:36:49.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0db3d700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806ab50 0x7f0f0819e6b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400e9a0 tx=0x7f0f0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:49.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400ca10 con 0x7f0f0806ab50 2024-03-26T17:36:49.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f0400cbb0 con 0x7f0f0806ab50 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f08199300 con 0x7f0f0806ab50 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04018680 con 0x7f0f0806ab50 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.212+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081a5510 con 0x7f0f0806ab50 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.213+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0f08062b40 con 0x7f0f0806ab50 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.214+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f0f04010430 con 0x7f0f0806ab50 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.215+0000 7f0efeffd700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ef40a3040 0x7f0ef40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.215+0000 7f0f0d33c700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ef40a3040 0x7f0ef40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.215+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0f040cfa10 con 0x7f0f0806ab50 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.215+0000 7f0efeffd700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] conn(0x7f0ef40ac820 0x7f0ef40aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:49.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.215+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 --> [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0f0400be50 con 0x7f0ef40ac820 2024-03-26T17:36:49.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.215+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f0f040cfef0 con 0x7f0f0806ab50 2024-03-26T17:36:49.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.215+0000 7f0f0d33c700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ef40a3040 0x7f0ef40a5500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006010 tx=0x7f0ef8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:49.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.215+0000 7f0f0e33e700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] conn(0x7f0ef40ac820 0x7f0ef40aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:49.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.216+0000 7f0f0e33e700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] conn(0x7f0ef40ac820 0x7f0ef40aec40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:49.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.216+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 <== osd.7 v2:172.21.15.135:6816/788617133 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0f0400be50 con 0x7f0ef40ac820 2024-03-26T17:36:49.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.232+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 --> [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0f08062f40 con 0x7f0ef40ac820 2024-03-26T17:36:49.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0efeffd700 1 -- 172.21.15.105:0/3429845637 <== osd.7 v2:172.21.15.135:6816/788617133 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0f08062f40 con 0x7f0ef40ac820 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 >> [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] conn(0x7f0ef40ac820 msgr2=0x7f0ef40aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] conn(0x7f0ef40ac820 0x7f0ef40aec40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ef40a3040 msgr2=0x7f0ef40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ef40a3040 0x7f0ef40a5500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006010 tx=0x7f0ef8005960 comp rx=0 tx=0).stop 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806ab50 msgr2=0x7f0f0819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 --2- 172.21.15.105:0/3429845637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f0806ab50 0x7f0f0819e6b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400e9a0 tx=0x7f0f0400ecb0 comp rx=0 tx=0).stop 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0e33e700 1 -- 172.21.15.105:0/3429845637 reap_dead start 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 shutdown_connections 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 >> 172.21.15.105:0/3429845637 conn(0x7f0f08077300 msgr2=0x7f0f08100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 shutdown_connections 2024-03-26T17:36:49.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:49.233+0000 7f0f0fda1700 1 -- 172.21.15.105:0/3429845637 wait complete. 2024-03-26T17:36:49.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:49 smithi105 ceph-mon[32280]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-26T17:36:49.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:49 smithi135 ceph-mon[35636]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-26T17:36:50.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:49 smithi144 ceph-mon[36095]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-26T17:36:51.483 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:51.786 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:51 smithi105 ceph-mon[32280]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-26T17:36:51.911 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:51.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:51 smithi135 ceph-mon[35636]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-26T17:36:52.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:51 smithi144 ceph-mon[36095]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 -- 172.21.15.105:0/4140847737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102df0 msgr2=0x7f473810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 --2- 172.21.15.105:0/4140847737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102df0 0x7f473810f250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f473000b0a0 tx=0x7f473000b3b0 comp rx=0 tx=0).stop 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 -- 172.21.15.105:0/4140847737 shutdown_connections 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 --2- 172.21.15.105:0/4140847737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102df0 0x7f473810f250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 --2- 172.21.15.105:0/4140847737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4738108510 0x7f47381088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 --2- 172.21.15.105:0/4140847737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4738102430 0x7f47381028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 -- 172.21.15.105:0/4140847737 >> 172.21.15.105:0/4140847737 conn(0x7f4738079470 msgr2=0x7f4738079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 -- 172.21.15.105:0/4140847737 shutdown_connections 2024-03-26T17:36:53.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.940+0000 7f4740595700 1 -- 172.21.15.105:0/4140847737 wait complete. 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f4740595700 1 Processor -- start 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f4740595700 1 -- start start 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f4740595700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102430 0x7f47380746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f4740595700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4738102df0 0x7f4738074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f4740595700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4738108510 0x7f4738075130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f4740595700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4738112540 con 0x7f4738102430 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f4740595700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47381123c0 con 0x7f4738102df0 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f4740595700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47381126c0 con 0x7f4738108510 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473e331700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102430 0x7f47380746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473e331700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102430 0x7f47380746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44830/0 (socket says 172.21.15.105:44830) 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473e331700 1 -- 172.21.15.105:0/3838886741 learned_addr learned my addr 172.21.15.105:0/3838886741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473db30700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4738102df0 0x7f4738074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473eb32700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4738108510 0x7f4738075130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:53.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473e331700 1 -- 172.21.15.105:0/3838886741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4738108510 msgr2=0x7f4738075130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473e331700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4738108510 0x7f4738075130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473e331700 1 -- 172.21.15.105:0/3838886741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4738102df0 msgr2=0x7f4738074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473e331700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4738102df0 0x7f4738074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.941+0000 7f473e331700 1 -- 172.21.15.105:0/3838886741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47381a6f20 con 0x7f4738102430 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.942+0000 7f473e331700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102430 0x7f47380746b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f473400e9a0 tx=0x7f473400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.942+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473400ca10 con 0x7f4738102430 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.942+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47381a7210 con 0x7f4738102430 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.942+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f473400cbb0 con 0x7f4738102430 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.942+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734018680 con 0x7f4738102430 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.942+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47381a77a0 con 0x7f4738102430 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.943+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4738062ac0 con 0x7f4738102430 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.945+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f4734010430 con 0x7f4738102430 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.945+0000 7f472f7fe700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47240a3040 0x7f47240a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:53.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.945+0000 7f473db30700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47240a3040 0x7f47240a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.945+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f47340d0090 con 0x7f4738102430 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.945+0000 7f472f7fe700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] conn(0x7f47240ac820 0x7f47240aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.945+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 --> [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4734014ec0 con 0x7f47240ac820 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.945+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f47340d0570 con 0x7f4738102430 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.945+0000 7f473db30700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47240a3040 0x7f47240a5500 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4738079a70 tx=0x7f4728009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.946+0000 7f473eb32700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] conn(0x7f47240ac820 0x7f47240aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.946+0000 7f473eb32700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] conn(0x7f47240ac820 0x7f47240aec40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.947+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 <== osd.5 v2:172.21.15.144:6808/3886741876 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4734014ec0 con 0x7f47240ac820 2024-03-26T17:36:53.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 --> [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4738062ec0 con 0x7f47240ac820 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f472f7fe700 1 -- 172.21.15.105:0/3838886741 <== osd.5 v2:172.21.15.144:6808/3886741876 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4738062ec0 con 0x7f47240ac820 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 >> [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] conn(0x7f47240ac820 msgr2=0x7f47240aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] conn(0x7f47240ac820 0x7f47240aec40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47240a3040 msgr2=0x7f47240a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47240a3040 0x7f47240a5500 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4738079a70 tx=0x7f4728009260 comp rx=0 tx=0).stop 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102430 msgr2=0x7f47380746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 --2- 172.21.15.105:0/3838886741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4738102430 0x7f47380746b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f473400e9a0 tx=0x7f473400ecb0 comp rx=0 tx=0).stop 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f473eb32700 1 -- 172.21.15.105:0/3838886741 reap_dead start 2024-03-26T17:36:53.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 shutdown_connections 2024-03-26T17:36:53.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 >> 172.21.15.105:0/3838886741 conn(0x7f4738079470 msgr2=0x7f4738101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:53.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.970+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 shutdown_connections 2024-03-26T17:36:53.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:52.971+0000 7f4740595700 1 -- 172.21.15.105:0/3838886741 wait complete. 2024-03-26T17:36:53.689 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:53 smithi105 ceph-mon[32280]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:36:53.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:53 smithi135 ceph-mon[35636]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:36:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:53 smithi144 ceph-mon[36095]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:36:54.268 INFO:teuthology.orchestra.run.smithi105.stdout:167503724594 2024-03-26T17:36:54.268 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-26T17:36:54.909 INFO:teuthology.orchestra.run.smithi105.stdout:223338299432 2024-03-26T17:36:54.910 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-26T17:36:55.798 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:55 smithi105 ceph-mon[32280]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:36:55.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:55 smithi135 ceph-mon[35636]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:36:55.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 -- 172.21.15.105:0/3253244564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928106630 msgr2=0x7f892810e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 --2- 172.21.15.105:0/3253244564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928106630 0x7f892810e200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f892000b0d0 tx=0x7f892000b3e0 comp rx=0 tx=0).stop 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 -- 172.21.15.105:0/3253244564 shutdown_connections 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 --2- 172.21.15.105:0/3253244564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928106630 0x7f892810e200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 --2- 172.21.15.105:0/3253244564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8928101d10 0x7f89281020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 --2- 172.21.15.105:0/3253244564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89281026c0 0x7f8928105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 -- 172.21.15.105:0/3253244564 >> 172.21.15.105:0/3253244564 conn(0x7f89280778b0 msgr2=0x7f89280fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 -- 172.21.15.105:0/3253244564 shutdown_connections 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.800+0000 7f892e8be700 1 -- 172.21.15.105:0/3253244564 wait complete. 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f892e8be700 1 Processor -- start 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f892e8be700 1 -- start start 2024-03-26T17:36:55.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f892e8be700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928101d10 0x7f892819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f892e8be700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89281026c0 0x7f892819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f892e8be700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8928106630 0x7f8928197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f892e8be700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89281114f0 con 0x7f8928101d10 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f892e8be700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8928111370 con 0x7f89281026c0 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f892e8be700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8928111670 con 0x7f8928106630 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f8927fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928101d10 0x7f892819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f8927fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928101d10 0x7f892819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44856/0 (socket says 172.21.15.105:44856) 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.801+0000 7f8927fff700 1 -- 172.21.15.105:0/3015941446 learned_addr learned my addr 172.21.15.105:0/3015941446 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f89277fe700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89281026c0 0x7f892819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f892ce5b700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8928106630 0x7f8928197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f8927fff700 1 -- 172.21.15.105:0/3015941446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8928106630 msgr2=0x7f8928197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:55.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f8927fff700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8928106630 0x7f8928197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f8927fff700 1 -- 172.21.15.105:0/3015941446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89281026c0 msgr2=0x7f892819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f8927fff700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89281026c0 0x7f892819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f8927fff700 1 -- 172.21.15.105:0/3015941446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8928197ff0 con 0x7f8928101d10 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f8927fff700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928101d10 0x7f892819d640 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f891800ead0 tx=0x7f891800ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891800b9c0 con 0x7f8928101d10 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8918004ce0 con 0x7f8928101d10 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89281982e0 con 0x7f8928101d10 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918005580 con 0x7f8928101d10 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.802+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89281009a0 con 0x7f8928101d10 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.803+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8928100ed0 con 0x7f8928101d10 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.804+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f8918004750 con 0x7f8928101d10 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.805+0000 7f89257fa700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89100a3040 0x7f89100a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.805+0000 7f89277fe700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89100a3040 0x7f89100a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:55.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.805+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f89180d1370 con 0x7f8928101d10 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.805+0000 7f89257fa700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] conn(0x7f89100ac820 0x7f89100aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.805+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 --> [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8918025e50 con 0x7f89100ac820 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.805+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f89180d1850 con 0x7f8928101d10 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.805+0000 7f89277fe700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89100a3040 0x7f89100a5500 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f89281042d0 tx=0x7f891c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.805+0000 7f892ce5b700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] conn(0x7f89100ac820 0x7f89100aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.806+0000 7f892ce5b700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] conn(0x7f89100ac820 0x7f89100aec40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.806+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 <== osd.1 v2:172.21.15.135:6800/932873598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8918025e50 con 0x7f89100ac820 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.822+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 --> [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8928101350 con 0x7f89100ac820 2024-03-26T17:36:55.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f89257fa700 1 -- 172.21.15.105:0/3015941446 <== osd.1 v2:172.21.15.135:6800/932873598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8928101350 con 0x7f89100ac820 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 >> [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] conn(0x7f89100ac820 msgr2=0x7f89100aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] conn(0x7f89100ac820 0x7f89100aec40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89100a3040 msgr2=0x7f89100a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89100a3040 0x7f89100a5500 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f89281042d0 tx=0x7f891c009260 comp rx=0 tx=0).stop 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928101d10 msgr2=0x7f892819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 --2- 172.21.15.105:0/3015941446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8928101d10 0x7f892819d640 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f891800ead0 tx=0x7f891800ede0 comp rx=0 tx=0).stop 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892ce5b700 1 -- 172.21.15.105:0/3015941446 reap_dead start 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 shutdown_connections 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 >> 172.21.15.105:0/3015941446 conn(0x7f89280778b0 msgr2=0x7f89280fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 shutdown_connections 2024-03-26T17:36:55.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:53.823+0000 7f892e8be700 1 -- 172.21.15.105:0/3015941446 wait complete. 2024-03-26T17:36:56.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:55 smithi144 ceph-mon[36095]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:36:56.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 -- 172.21.15.105:0/1426316858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81281012b0 msgr2=0x7f8128101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:56.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 --2- 172.21.15.105:0/1426316858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81281012b0 0x7f8128101710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f811c009990 tx=0x7f811c009ca0 comp rx=0 tx=0).stop 2024-03-26T17:36:56.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 -- 172.21.15.105:0/1426316858 shutdown_connections 2024-03-26T17:36:56.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 --2- 172.21.15.105:0/1426316858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8128101c50 0x7f812810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 --2- 172.21.15.105:0/1426316858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81281072f0 0x7f81281076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 --2- 172.21.15.105:0/1426316858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81281012b0 0x7f8128101710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 -- 172.21.15.105:0/1426316858 >> 172.21.15.105:0/1426316858 conn(0x7f81280781e0 msgr2=0x7f81280785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 -- 172.21.15.105:0/1426316858 shutdown_connections 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.714+0000 7f812e152700 1 -- 172.21.15.105:0/1426316858 wait complete. 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f812e152700 1 Processor -- start 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f812e152700 1 -- start start 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f812e152700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81281012b0 0x7f8128102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f812e152700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8128101c50 0x7f81281034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f812e152700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81281072f0 0x7f81281039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f812e152700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8128111400 con 0x7f8128101c50 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f812e152700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8128111280 con 0x7f81281072f0 2024-03-26T17:36:56.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f812e152700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8128111580 con 0x7f81281012b0 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f8126ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8128101c50 0x7f81281034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f8126ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8128101c50 0x7f81281034a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36196/0 (socket says 172.21.15.105:36196) 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f8126ffd700 1 -- 172.21.15.105:0/3666221939 learned_addr learned my addr 172.21.15.105:0/3666221939 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f8127fff700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81281072f0 0x7f81281039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.715+0000 7f81277fe700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81281012b0 0x7f8128102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8126ffd700 1 -- 172.21.15.105:0/3666221939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81281012b0 msgr2=0x7f8128102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8126ffd700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81281012b0 0x7f8128102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8126ffd700 1 -- 172.21.15.105:0/3666221939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81281072f0 msgr2=0x7f81281039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8126ffd700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81281072f0 0x7f81281039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8126ffd700 1 -- 172.21.15.105:0/3666221939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81281a6fb0 con 0x7f8128101c50 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8126ffd700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8128101c50 0x7f81281034a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f811c000c00 tx=0x7f811c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c027070 con 0x7f8128101c50 2024-03-26T17:36:56.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81281a7240 con 0x7f8128101c50 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f811c022470 con 0x7f8128101c50 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c00b610 con 0x7f8128101c50 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.716+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81281a76f0 con 0x7f8128101c50 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.717+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8128106380 con 0x7f8128101c50 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.718+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f811c02a070 con 0x7f8128101c50 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f8124ff9700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81100a3040 0x7f81100a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f81277fe700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81100a3040 0x7f81100a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f811c0d0850 con 0x7f8128101c50 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f8124ff9700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] conn(0x7f81100ac820 0x7f81100aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 --> [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f811c001d30 con 0x7f81100ac820 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f811c0d0d30 con 0x7f8128101c50 2024-03-26T17:36:56.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f8127fff700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] conn(0x7f81100ac820 0x7f81100aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:56.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f81277fe700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81100a3040 0x7f81100a5500 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8118009be0 tx=0x7f8118009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:56.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.719+0000 7f8127fff700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] conn(0x7f81100ac820 0x7f81100aec40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:56.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.721+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 <== osd.9 v2:172.21.15.105:6826/1324177840 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f811c001d30 con 0x7f81100ac820 2024-03-26T17:36:56.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.744+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 --> [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f81281029a0 con 0x7f81100ac820 2024-03-26T17:36:56.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f8124ff9700 1 -- 172.21.15.105:0/3666221939 <== osd.9 v2:172.21.15.105:6826/1324177840 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f81281029a0 con 0x7f81100ac820 2024-03-26T17:36:56.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] conn(0x7f81100ac820 msgr2=0x7f81100aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:56.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] conn(0x7f81100ac820 0x7f81100aec40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81100a3040 msgr2=0x7f81100a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81100a3040 0x7f81100a5500 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8118009be0 tx=0x7f8118009260 comp rx=0 tx=0).stop 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8128101c50 msgr2=0x7f81281034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 --2- 172.21.15.105:0/3666221939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8128101c50 0x7f81281034a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f811c000c00 tx=0x7f811c004790 comp rx=0 tx=0).stop 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f8127fff700 1 -- 172.21.15.105:0/3666221939 reap_dead start 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 shutdown_connections 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 >> 172.21.15.105:0/3666221939 conn(0x7f81280781e0 msgr2=0x7f812806a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 shutdown_connections 2024-03-26T17:36:56.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:56.745+0000 7f812e152700 1 -- 172.21.15.105:0/3666221939 wait complete. 2024-03-26T17:36:57.436 INFO:teuthology.orchestra.run.smithi105.stdout:287762808863 2024-03-26T17:36:57.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-26T17:36:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.722+0000 7f2f0276f700 1 -- 172.21.15.105:0/2285816042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc102430 msgr2=0x7f2efc1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.722+0000 7f2f0276f700 1 --2- 172.21.15.105:0/2285816042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc102430 0x7f2efc1028b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009990 tx=0x7f2ef0009ca0 comp rx=0 tx=0).stop 2024-03-26T17:36:57.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.722+0000 7f2f0276f700 1 -- 172.21.15.105:0/2285816042 shutdown_connections 2024-03-26T17:36:57.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.722+0000 7f2f0276f700 1 --2- 172.21.15.105:0/2285816042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2efc102df0 0x7f2efc10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:57.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.722+0000 7f2f0276f700 1 --2- 172.21.15.105:0/2285816042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2efc108480 0x7f2efc108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:57.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.722+0000 7f2f0276f700 1 --2- 172.21.15.105:0/2285816042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc102430 0x7f2efc1028b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:57.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.722+0000 7f2f0276f700 1 -- 172.21.15.105:0/2285816042 >> 172.21.15.105:0/2285816042 conn(0x7f2efc079500 msgr2=0x7f2efc079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:57.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.722+0000 7f2f0276f700 1 -- 172.21.15.105:0/2285816042 shutdown_connections 2024-03-26T17:36:57.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.723+0000 7f2f0276f700 1 -- 172.21.15.105:0/2285816042 wait complete. 2024-03-26T17:36:57.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.723+0000 7f2f0276f700 1 Processor -- start 2024-03-26T17:36:57.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.723+0000 7f2f0276f700 1 -- start start 2024-03-26T17:36:57.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.723+0000 7f2f0276f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2efc102430 0x7f2efc19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.723+0000 7f2f0276f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2efc102df0 0x7f2efc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.723+0000 7f2f0276f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc108480 0x7f2efc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.723+0000 7f2f0276f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc112570 con 0x7f2efc108480 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f0276f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1123f0 con 0x7f2efc102df0 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f0276f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1126f0 con 0x7f2efc102430 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc108480 0x7f2efc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc108480 0x7f2efc1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36212/0 (socket says 172.21.15.105:36212) 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 -- 172.21.15.105:0/1495467513 learned_addr learned my addr 172.21.15.105:0/1495467513 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2efbfff700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2efc102430 0x7f2efc19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2efb7fe700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2efc102df0 0x7f2efc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 -- 172.21.15.105:0/1495467513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2efc102430 msgr2=0x7f2efc19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2efc102430 0x7f2efc19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:57.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 -- 172.21.15.105:0/1495467513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2efc102df0 msgr2=0x7f2efc19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2efc102df0 0x7f2efc19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 -- 172.21.15.105:0/1495467513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc199050 con 0x7f2efc108480 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2efbfff700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2efc102430 0x7f2efc19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f00d0c700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc108480 0x7f2efc1988f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400efc0 tx=0x7f2ef400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40092e0 con 0x7f2efc108480 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc1992e0 con 0x7f2efc108480 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef4004bb0 con 0x7f2efc108480 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4004020 con 0x7f2efc108480 2024-03-26T17:36:57.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.724+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc1a55b0 con 0x7f2efc108480 2024-03-26T17:36:57.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.725+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2efc062b40 con 0x7f2efc108480 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.727+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f2ef401b030 con 0x7f2efc108480 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.727+0000 7f2ef97fa700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ee40a3040 0x7f2ee40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.727+0000 7f2efbfff700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ee40a3040 0x7f2ee40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.727+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f2ef40cf630 con 0x7f2efc108480 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.727+0000 7f2ef97fa700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] conn(0x7f2ee40ac820 0x7f2ee40aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.727+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 --> [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ee40af2f0 con 0x7f2ee40ac820 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.728+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f2ef40cfb10 con 0x7f2efc108480 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.728+0000 7f2efbfff700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ee40a3040 0x7f2ee40a5500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2efc100bf0 tx=0x7f2eec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:57.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.728+0000 7f2efb7fe700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] conn(0x7f2ee40ac820 0x7f2ee40aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:57.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.728+0000 7f2efb7fe700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] conn(0x7f2ee40ac820 0x7f2ee40aec40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:57.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.729+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 <== osd.4 v2:172.21.15.135:6808/507590464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ee40af2f0 con 0x7f2ee40ac820 2024-03-26T17:36:57.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 --> [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2efc062f40 con 0x7f2ee40ac820 2024-03-26T17:36:57.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2ef97fa700 1 -- 172.21.15.105:0/1495467513 <== osd.4 v2:172.21.15.135:6808/507590464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2efc062f40 con 0x7f2ee40ac820 2024-03-26T17:36:57.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 >> [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] conn(0x7f2ee40ac820 msgr2=0x7f2ee40aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] conn(0x7f2ee40ac820 0x7f2ee40aec40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ee40a3040 msgr2=0x7f2ee40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ee40a3040 0x7f2ee40a5500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2efc100bf0 tx=0x7f2eec009260 comp rx=0 tx=0).stop 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc108480 msgr2=0x7f2efc1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 --2- 172.21.15.105:0/1495467513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2efc108480 0x7f2efc1988f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400efc0 tx=0x7f2ef400c3a0 comp rx=0 tx=0).stop 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f00d0c700 1 -- 172.21.15.105:0/1495467513 reap_dead start 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 shutdown_connections 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 >> 172.21.15.105:0/1495467513 conn(0x7f2efc079500 msgr2=0x7f2efc1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 shutdown_connections 2024-03-26T17:36:57.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:57.746+0000 7f2f0276f700 1 -- 172.21.15.105:0/1495467513 wait complete. 2024-03-26T17:36:57.987 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:57 smithi105 ceph-mon[32280]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T17:36:58.015 INFO:teuthology.orchestra.run.smithi105.stdout:55834574917 2024-03-26T17:36:58.016 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-26T17:36:58.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:57 smithi144 ceph-mon[36095]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T17:36:58.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:57 smithi135 ceph-mon[35636]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-26T17:36:58.956 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:36:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.992+0000 7f4ba3c25700 1 -- 172.21.15.105:0/3558699595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101540 msgr2=0x7f4b9c101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.992+0000 7f4ba3c25700 1 --2- 172.21.15.105:0/3558699595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101540 0x7f4b9c101920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b980099c0 tx=0x7f4b98009cd0 comp rx=0 tx=0).stop 2024-03-26T17:36:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.992+0000 7f4ba3c25700 1 -- 172.21.15.105:0/3558699595 shutdown_connections 2024-03-26T17:36:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.992+0000 7f4ba3c25700 1 --2- 172.21.15.105:0/3558699595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b9c106590 0x7f4b9c10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.992+0000 7f4ba3c25700 1 --2- 172.21.15.105:0/3558699595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101540 0x7f4b9c101920 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:58.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.992+0000 7f4ba3c25700 1 --2- 172.21.15.105:0/3558699595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b9c101ef0 0x7f4b9c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:58.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.992+0000 7f4ba3c25700 1 -- 172.21.15.105:0/3558699595 >> 172.21.15.105:0/3558699595 conn(0x7f4b9c077300 msgr2=0x7f4b9c077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:58.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.993+0000 7f4ba3c25700 1 -- 172.21.15.105:0/3558699595 shutdown_connections 2024-03-26T17:36:58.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.993+0000 7f4ba3c25700 1 -- 172.21.15.105:0/3558699595 wait complete. 2024-03-26T17:36:58.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.993+0000 7f4ba3c25700 1 Processor -- start 2024-03-26T17:36:58.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.993+0000 7f4ba3c25700 1 -- start start 2024-03-26T17:36:58.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba3c25700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b9c101540 0x7f4b9c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:58.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba3c25700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101ef0 0x7f4b9c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba3c25700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b9c106590 0x7f4b9c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba3c25700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c111420 con 0x7f4b9c101ef0 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba3c25700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1112a0 con 0x7f4b9c101540 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba3c25700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1115a0 con 0x7f4b9c106590 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba11c0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101ef0 0x7f4b9c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba11c0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101ef0 0x7f4b9c19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36240/0 (socket says 172.21.15.105:36240) 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba11c0700 1 -- 172.21.15.105:0/326270271 learned_addr learned my addr 172.21.15.105:0/326270271 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba21c2700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b9c106590 0x7f4b9c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba19c1700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b9c101540 0x7f4b9c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba21c2700 1 -- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b9c101540 msgr2=0x7f4b9c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba21c2700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b9c101540 0x7f4b9c19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba21c2700 1 -- 172.21.15.105:0/326270271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101ef0 msgr2=0x7f4b9c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba21c2700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101ef0 0x7f4b9c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:58.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba21c2700 1 -- 172.21.15.105:0/326270271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c197fc0 con 0x7f4b9c106590 2024-03-26T17:36:58.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba19c1700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b9c101540 0x7f4b9c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:36:58.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.994+0000 7f4ba11c0700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b9c101ef0 0x7f4b9c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:36:58.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.995+0000 7f4ba21c2700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b9c106590 0x7f4b9c1977e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400b330 tx=0x7f4b9400b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:58.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.995+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94004bb0 con 0x7f4b9c106590 2024-03-26T17:36:58.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.995+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b94004d50 con 0x7f4b9c106590 2024-03-26T17:36:58.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.995+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94019410 con 0x7f4b9c106590 2024-03-26T17:36:58.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.995+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9c198250 con 0x7f4b9c106590 2024-03-26T17:36:58.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.995+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9c06a8f0 con 0x7f4b9c106590 2024-03-26T17:36:58.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.996+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b9c06aea0 con 0x7f4b9c106590 2024-03-26T17:36:58.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.998+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f4b94020020 con 0x7f4b9c106590 2024-03-26T17:36:58.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.998+0000 7f4b92ffd700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b880a30c0 0x7f4b880a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:58.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.998+0000 7f4ba19c1700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b880a30c0 0x7f4b880a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:58.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.999+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4b940d2a00 con 0x7f4b9c106590 2024-03-26T17:36:58.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.999+0000 7f4b92ffd700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] conn(0x7f4b880ac8a0 0x7f4b880aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:58.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.999+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 --> [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b880af370 con 0x7f4b880ac8a0 2024-03-26T17:36:58.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.999+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f4b940d72c0 con 0x7f4b9c106590 2024-03-26T17:36:58.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.999+0000 7f4ba11c0700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] conn(0x7f4b880ac8a0 0x7f4b880aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:58.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.999+0000 7f4ba19c1700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b880a30c0 0x7f4b880a5580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b98003760 tx=0x7f4b98005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:58.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:58.999+0000 7f4ba11c0700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] conn(0x7f4b880ac8a0 0x7f4b880aecc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:58.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.000+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 <== osd.10 v2:172.21.15.135:6824/3021911997 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b880af370 con 0x7f4b880ac8a0 2024-03-26T17:36:59.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 --> [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4b9c06b320 con 0x7f4b880ac8a0 2024-03-26T17:36:59.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4b92ffd700 1 -- 172.21.15.105:0/326270271 <== osd.10 v2:172.21.15.135:6824/3021911997 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4b9c06b320 con 0x7f4b880ac8a0 2024-03-26T17:36:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] conn(0x7f4b880ac8a0 msgr2=0x7f4b880aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba3c25700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] conn(0x7f4b880ac8a0 0x7f4b880aecc0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b880a30c0 msgr2=0x7f4b880a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba3c25700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b880a30c0 0x7f4b880a5580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b98003760 tx=0x7f4b98005ea0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b9c106590 msgr2=0x7f4b9c1977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba3c25700 1 --2- 172.21.15.105:0/326270271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b9c106590 0x7f4b9c1977e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400b330 tx=0x7f4b9400b640 comp rx=0 tx=0).stop 2024-03-26T17:36:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba21c2700 1 -- 172.21.15.105:0/326270271 reap_dead start 2024-03-26T17:36:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.017+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 shutdown_connections 2024-03-26T17:36:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.018+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 >> 172.21.15.105:0/326270271 conn(0x7f4b9c077300 msgr2=0x7f4b9c107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.018+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 shutdown_connections 2024-03-26T17:36:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.018+0000 7f4ba3c25700 1 -- 172.21.15.105:0/326270271 wait complete. 2024-03-26T17:36:59.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.601+0000 7f2333dae700 1 -- 172.21.15.105:0/3081288652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a980 msgr2=0x7f232c10e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.601+0000 7f2333dae700 1 --2- 172.21.15.105:0/3081288652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a980 0x7f232c10e920 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f231c009990 tx=0x7f231c009ca0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 -- 172.21.15.105:0/3081288652 shutdown_connections 2024-03-26T17:36:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 --2- 172.21.15.105:0/3081288652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f232c10ee60 0x7f232c111250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 --2- 172.21.15.105:0/3081288652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c06a060 0x7f232c06a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 --2- 172.21.15.105:0/3081288652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a980 0x7f232c10e920 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 -- 172.21.15.105:0/3081288652 >> 172.21.15.105:0/3081288652 conn(0x7f232c0781e0 msgr2=0x7f232c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 -- 172.21.15.105:0/3081288652 shutdown_connections 2024-03-26T17:36:59.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 -- 172.21.15.105:0/3081288652 wait complete. 2024-03-26T17:36:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 Processor -- start 2024-03-26T17:36:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.602+0000 7f2333dae700 1 -- start start 2024-03-26T17:36:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2333dae700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a060 0x7f232c1a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2333dae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c06a980 0x7f232c1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2333dae700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f232c10ee60 0x7f232c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2333dae700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f232c114780 con 0x7f232c06a060 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2333dae700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f232c114600 con 0x7f232c10ee60 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2333dae700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f232c114900 con 0x7f232c06a980 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a060 0x7f232c1a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a060 0x7f232c1a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36254/0 (socket says 172.21.15.105:36254) 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 -- 172.21.15.105:0/2392954765 learned_addr learned my addr 172.21.15.105:0/2392954765 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331349700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c06a980 0x7f232c1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f233234b700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f232c10ee60 0x7f232c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 -- 172.21.15.105:0/2392954765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c06a980 msgr2=0x7f232c1a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f232c06a980 0x7f232c1a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 -- 172.21.15.105:0/2392954765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f232c10ee60 msgr2=0x7f232c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f232c10ee60 0x7f232c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 -- 172.21.15.105:0/2392954765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c19b240 con 0x7f232c06a060 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2331b4a700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a060 0x7f232c1a08d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f232800e9c0 tx=0x7f232800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232800cbd0 con 0x7f232c06a060 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f232800ee60 con 0x7f232c06a060 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23280055b0 con 0x7f232c06a060 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.603+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c19b530 con 0x7f232c06a060 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.604+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1a77a0 con 0x7f232c06a060 2024-03-26T17:36:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.604+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f232c062b40 con 0x7f232c06a060 2024-03-26T17:36:59.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.606+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f2328004750 con 0x7f232c06a060 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.606+0000 7f2322ffd700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23180a30c0 0x7f23180a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.606+0000 7f2331349700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23180a30c0 0x7f23180a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.607+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f23280cfa00 con 0x7f232c06a060 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.607+0000 7f2322ffd700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] conn(0x7f23180ac8a0 0x7f23180aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.607+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 --> [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23180af370 con 0x7f23180ac8a0 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.607+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f23280cfee0 con 0x7f232c06a060 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.607+0000 7f233234b700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] conn(0x7f23180ac8a0 0x7f23180aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.607+0000 7f2331349700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23180a30c0 0x7f23180a5580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f231c00b2e0 tx=0x7f231c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.607+0000 7f233234b700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] conn(0x7f23180ac8a0 0x7f23180aecc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:36:59.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.608+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 <== osd.2 v2:172.21.15.144:6800/386966961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23180af370 con 0x7f23180ac8a0 2024-03-26T17:36:59.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.624+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 --> [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f232c062fc0 con 0x7f23180ac8a0 2024-03-26T17:36:59.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.624+0000 7f2322ffd700 1 -- 172.21.15.105:0/2392954765 <== osd.2 v2:172.21.15.144:6800/386966961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f232c062fc0 con 0x7f23180ac8a0 2024-03-26T17:36:59.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 >> [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] conn(0x7f23180ac8a0 msgr2=0x7f23180aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] conn(0x7f23180ac8a0 0x7f23180aecc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23180a30c0 msgr2=0x7f23180a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23180a30c0 0x7f23180a5580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f231c00b2e0 tx=0x7f231c005960 comp rx=0 tx=0).stop 2024-03-26T17:36:59.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a060 msgr2=0x7f232c1a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:36:59.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 --2- 172.21.15.105:0/2392954765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f232c06a060 0x7f232c1a08d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f232800e9c0 tx=0x7f232800ecd0 comp rx=0 tx=0).stop 2024-03-26T17:36:59.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f233234b700 1 -- 172.21.15.105:0/2392954765 reap_dead start 2024-03-26T17:36:59.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 shutdown_connections 2024-03-26T17:36:59.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 >> 172.21.15.105:0/2392954765 conn(0x7f232c0781e0 msgr2=0x7f232c110550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:36:59.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 shutdown_connections 2024-03-26T17:36:59.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:36:59.625+0000 7f2333dae700 1 -- 172.21.15.105:0/2392954765 wait complete. 2024-03-26T17:36:59.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:36:59 smithi105 ceph-mon[32280]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:37:00.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:36:59 smithi144 ceph-mon[36095]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:37:00.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:36:59 smithi135 ceph-mon[35636]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:37:00.532 INFO:teuthology.orchestra.run.smithi105.stdout:309237645336 2024-03-26T17:37:00.533 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-26T17:37:01.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:01 smithi105 ceph-mon[32280]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T17:37:01.954 INFO:teuthology.orchestra.run.smithi105.stdout:141733920824 2024-03-26T17:37:01.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-26T17:37:02.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:01 smithi144 ceph-mon[36095]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T17:37:02.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:01 smithi135 ceph-mon[35636]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T17:37:03.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:03 smithi105 ceph-mon[32280]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T17:37:03.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 -- 172.21.15.105:0/108232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc101c50 msgr2=0x7fb3bc10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:03.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 --2- 172.21.15.105:0/108232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc101c50 0x7fb3bc10e130 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400b0a0 tx=0x7fb3b400b3b0 comp rx=0 tx=0).stop 2024-03-26T17:37:03.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 -- 172.21.15.105:0/108232 shutdown_connections 2024-03-26T17:37:03.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 --2- 172.21.15.105:0/108232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc101c50 0x7fb3bc10e130 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:03.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 --2- 172.21.15.105:0/108232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc1072f0 0x7fb3bc1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 --2- 172.21.15.105:0/108232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012b0 0x7fb3bc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 -- 172.21.15.105:0/108232 >> 172.21.15.105:0/108232 conn(0x7fb3bc0781e0 msgr2=0x7fb3bc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 -- 172.21.15.105:0/108232 shutdown_connections 2024-03-26T17:37:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.859+0000 7fb3c2916700 1 -- 172.21.15.105:0/108232 wait complete. 2024-03-26T17:37:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c2916700 1 Processor -- start 2024-03-26T17:37:03.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c2916700 1 -- start start 2024-03-26T17:37:03.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c2916700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012b0 0x7fb3bc199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:03.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c2916700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc101c50 0x7fb3bc199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:03.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c2916700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc1072f0 0x7fb3bc1a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c2916700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc111420 con 0x7fb3bc1072f0 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c2916700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1112a0 con 0x7fb3bc101c50 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c2916700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1115a0 con 0x7fb3bc1012b0 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3bbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012b0 0x7fb3bc199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3bbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012b0 0x7fb3bc199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56840/0 (socket says 172.21.15.105:56840) 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3c0eb3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc1072f0 0x7fb3bc1a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3bbfff700 1 -- 172.21.15.105:0/1926733513 learned_addr learned my addr 172.21.15.105:0/1926733513 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.860+0000 7fb3bb7fe700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc101c50 0x7fb3bc199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3bbfff700 1 -- 172.21.15.105:0/1926733513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc101c50 msgr2=0x7fb3bc199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3bbfff700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc101c50 0x7fb3bc199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3bbfff700 1 -- 172.21.15.105:0/1926733513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc1072f0 msgr2=0x7fb3bc1a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3bbfff700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc1072f0 0x7fb3bc1a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3bbfff700 1 -- 172.21.15.105:0/1926733513 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc1a4ec0 con 0x7fb3bc1012b0 2024-03-26T17:37:03.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3bb7fe700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc101c50 0x7fb3bc199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3bbfff700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012b0 0x7fb3bc199900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00e9a0 tx=0x7fb3ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00ca10 con 0x7fb3bc1012b0 2024-03-26T17:37:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc1a50f0 con 0x7fb3bc1012b0 2024-03-26T17:37:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3ac00cbb0 con 0x7fb3bc1012b0 2024-03-26T17:37:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac018680 con 0x7fb3bc1012b0 2024-03-26T17:37:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.861+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc1a5600 con 0x7fb3bc1012b0 2024-03-26T17:37:03.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.862+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3bc062ac0 con 0x7fb3bc1012b0 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.864+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fb3ac010430 con 0x7fb3bc1012b0 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.864+0000 7fb3b97fa700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a3040 0x7fb3a40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.864+0000 7fb3bb7fe700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a3040 0x7fb3a40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.864+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb3ac0cf910 con 0x7fb3bc1012b0 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.864+0000 7fb3b97fa700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] conn(0x7fb3a40ac820 0x7fb3a40aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.864+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 --> [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3ac01ce70 con 0x7fb3a40ac820 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.864+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb3ac0cfcc0 con 0x7fb3bc1012b0 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.864+0000 7fb3bb7fe700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a3040 0x7fb3a40a5500 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc078900 tx=0x7fb3b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:03.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.865+0000 7fb3c0eb3700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] conn(0x7fb3a40ac820 0x7fb3a40aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.865+0000 7fb3c0eb3700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] conn(0x7fb3a40ac820 0x7fb3a40aec40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:03.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.866+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 <== osd.8 v2:172.21.15.144:6816/4074146904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb3ac01ce70 con 0x7fb3a40ac820 2024-03-26T17:37:03.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 --> [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb3bc062ec0 con 0x7fb3a40ac820 2024-03-26T17:37:03.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3b97fa700 1 -- 172.21.15.105:0/1926733513 <== osd.8 v2:172.21.15.144:6816/4074146904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb3bc062ec0 con 0x7fb3a40ac820 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 >> [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] conn(0x7fb3a40ac820 msgr2=0x7fb3a40aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3c2916700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] conn(0x7fb3a40ac820 0x7fb3a40aec40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a3040 msgr2=0x7fb3a40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3c2916700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a3040 0x7fb3a40a5500 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc078900 tx=0x7fb3b0009260 comp rx=0 tx=0).stop 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012b0 msgr2=0x7fb3bc199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3c2916700 1 --2- 172.21.15.105:0/1926733513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc1012b0 0x7fb3bc199900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00e9a0 tx=0x7fb3ac00ecb0 comp rx=0 tx=0).stop 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.882+0000 7fb3c0eb3700 1 -- 172.21.15.105:0/1926733513 reap_dead start 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.883+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 shutdown_connections 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.883+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 >> 172.21.15.105:0/1926733513 conn(0x7fb3bc0781e0 msgr2=0x7fb3bc100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.883+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 shutdown_connections 2024-03-26T17:37:03.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:03.883+0000 7fb3c2916700 1 -- 172.21.15.105:0/1926733513 wait complete. 2024-03-26T17:37:04.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:03 smithi144 ceph-mon[36095]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T17:37:04.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:03 smithi135 ceph-mon[35636]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-26T17:37:05.157 INFO:teuthology.orchestra.run.smithi105.stdout:77309411394 2024-03-26T17:37:05.158 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-26T17:37:05.485 INFO:teuthology.orchestra.run.smithi105.stdout:253403070500 2024-03-26T17:37:05.486 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-26T17:37:05.854 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:05 smithi105 ceph-mon[32280]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:05.968 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:05 smithi135 ceph-mon[35636]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 -- 172.21.15.105:0/3622362501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806aa00 msgr2=0x7f0eb810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 --2- 172.21.15.105:0/3622362501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806aa00 0x7f0eb810f240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000b0d0 tx=0x7f0eb000b3e0 comp rx=0 tx=0).stop 2024-03-26T17:37:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 -- 172.21.15.105:0/3622362501 shutdown_connections 2024-03-26T17:37:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 --2- 172.21.15.105:0/3622362501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806aa00 0x7f0eb810f240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 --2- 172.21.15.105:0/3622362501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eb8108400 0x7f0eb81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 --2- 172.21.15.105:0/3622362501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb806a060 0x7f0eb806a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 -- 172.21.15.105:0/3622362501 >> 172.21.15.105:0/3622362501 conn(0x7f0eb80781e0 msgr2=0x7f0eb80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 -- 172.21.15.105:0/3622362501 shutdown_connections 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.194+0000 7f0ebcc86700 1 -- 172.21.15.105:0/3622362501 wait complete. 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0ebcc86700 1 Processor -- start 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0ebcc86700 1 -- start start 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0ebcc86700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806a060 0x7f0eb819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0ebcc86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb806aa00 0x7f0eb819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0ebcc86700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eb8108400 0x7f0eb81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0ebcc86700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8112500 con 0x7f0eb806a060 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0ebcc86700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8112380 con 0x7f0eb8108400 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0ebcc86700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8112680 con 0x7f0eb806aa00 2024-03-26T17:37:05.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0eb659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806a060 0x7f0eb819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0eb659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806a060 0x7f0eb819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36304/0 (socket says 172.21.15.105:36304) 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0eb659c700 1 -- 172.21.15.105:0/1305772073 learned_addr learned my addr 172.21.15.105:0/1305772073 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0eb5d9b700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb806aa00 0x7f0eb819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.195+0000 7f0eb6d9d700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eb8108400 0x7f0eb81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0eb659c700 1 -- 172.21.15.105:0/1305772073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb806aa00 msgr2=0x7f0eb819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0eb659c700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb806aa00 0x7f0eb819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0eb659c700 1 -- 172.21.15.105:0/1305772073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eb8108400 msgr2=0x7f0eb81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0eb659c700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0eb8108400 0x7f0eb81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0eb659c700 1 -- 172.21.15.105:0/1305772073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb8199040 con 0x7f0eb806a060 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0eb659c700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806a060 0x7f0eb819e6d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800eb30 tx=0x7f0ea800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800cbd0 con 0x7f0eb806a060 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb8199330 con 0x7f0eb806a060 2024-03-26T17:37:05.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ea80044d0 con 0x7f0eb806a060 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8003e30 con 0x7f0eb806a060 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.196+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a55a0 con 0x7f0eb806a060 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.197+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0eb8062a80 con 0x7f0eb806a060 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.198+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f0ea8004000 con 0x7f0eb806a060 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.199+0000 7f0ea77fe700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ea00a3040 0x7f0ea00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.199+0000 7f0eb5d9b700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ea00a3040 0x7f0ea00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.199+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0ea80cfb20 con 0x7f0eb806a060 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.199+0000 7f0ea77fe700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] conn(0x7f0ea00ac820 0x7f0ea00aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.199+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 --> [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ea80090b0 con 0x7f0ea00ac820 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.199+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f0ea80d0000 con 0x7f0eb806a060 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.199+0000 7f0eb6d9d700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] conn(0x7f0ea00ac820 0x7f0ea00aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:05.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.199+0000 7f0eb5d9b700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ea00a3040 0x7f0ea00a5500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009680 tx=0x7f0eac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.200+0000 7f0eb6d9d700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] conn(0x7f0ea00ac820 0x7f0ea00aec40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.200+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 <== osd.6 v2:172.21.15.105:6818/3456307470 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ea80090b0 con 0x7f0ea00ac820 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.216+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 --> [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0eb8062eb0 con 0x7f0ea00ac820 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ea77fe700 1 -- 172.21.15.105:0/1305772073 <== osd.6 v2:172.21.15.105:6818/3456307470 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0eb8062eb0 con 0x7f0ea00ac820 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] conn(0x7f0ea00ac820 msgr2=0x7f0ea00aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] conn(0x7f0ea00ac820 0x7f0ea00aec40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ea00a3040 msgr2=0x7f0ea00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ea00a3040 0x7f0ea00a5500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009680 tx=0x7f0eac009260 comp rx=0 tx=0).stop 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806a060 msgr2=0x7f0eb819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 --2- 172.21.15.105:0/1305772073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb806a060 0x7f0eb819e6d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800eb30 tx=0x7f0ea800ee40 comp rx=0 tx=0).stop 2024-03-26T17:37:06.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0eb6d9d700 1 -- 172.21.15.105:0/1305772073 reap_dead start 2024-03-26T17:37:06.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 shutdown_connections 2024-03-26T17:37:06.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 >> 172.21.15.105:0/1305772073 conn(0x7f0eb80781e0 msgr2=0x7f0eb80fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:06.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 shutdown_connections 2024-03-26T17:37:06.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.217+0000 7f0ebcc86700 1 -- 172.21.15.105:0/1305772073 wait complete. 2024-03-26T17:37:06.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:05 smithi144 ceph-mon[36095]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:06.760 INFO:teuthology.orchestra.run.smithi105.stdout:197568495663 2024-03-26T17:37:06.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-26T17:37:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 -- 172.21.15.105:0/1823012360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 msgr2=0x7f570c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 --2- 172.21.15.105:0/1823012360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 0x7f570c1088f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5700009990 tx=0x7f5700009ca0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 -- 172.21.15.105:0/1823012360 shutdown_connections 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 --2- 172.21.15.105:0/1823012360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f570c102df0 0x7f570c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 --2- 172.21.15.105:0/1823012360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 0x7f570c1088f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 --2- 172.21.15.105:0/1823012360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f570c102430 0x7f570c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 -- 172.21.15.105:0/1823012360 >> 172.21.15.105:0/1823012360 conn(0x7f570c079470 msgr2=0x7f570c079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 -- 172.21.15.105:0/1823012360 shutdown_connections 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.196+0000 7f5710c83700 1 -- 172.21.15.105:0/1823012360 wait complete. 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.197+0000 7f5710c83700 1 Processor -- start 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.197+0000 7f5710c83700 1 -- start start 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.197+0000 7f5710c83700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f570c102430 0x7f570c0746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.197+0000 7f5710c83700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f570c102df0 0x7f570c074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.197+0000 7f5710c83700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 0x7f570c075130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.197+0000 7f5710c83700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f570c112520 con 0x7f570c108510 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.197+0000 7f5710c83700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f570c1123a0 con 0x7f570c102df0 2024-03-26T17:37:07.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.197+0000 7f5710c83700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f570c1126a0 con 0x7f570c102430 2024-03-26T17:37:07.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 0x7f570c075130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 0x7f570c075130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57892/0 (socket says 172.21.15.105:57892) 2024-03-26T17:37:07.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f570c102430 0x7f570c0746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f5709d9b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f570c102df0 0x7f570c074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 -- 172.21.15.105:0/2592307390 learned_addr learned my addr 172.21.15.105:0/2592307390 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 -- 172.21.15.105:0/2592307390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f570c102430 msgr2=0x7f570c0746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f570c102430 0x7f570c0746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 -- 172.21.15.105:0/2592307390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f570c102df0 msgr2=0x7f570c074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f570c102df0 0x7f570c074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 -- 172.21.15.105:0/2592307390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570c1a6f20 con 0x7f570c108510 2024-03-26T17:37:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f570ad9d700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 0x7f570c075130 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5704007ee0 tx=0x7f570400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:07.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704004bb0 con 0x7f570c108510 2024-03-26T17:37:07.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5704004d50 con 0x7f570c108510 2024-03-26T17:37:07.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57040058e0 con 0x7f570c108510 2024-03-26T17:37:07.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570c1a7210 con 0x7f570c108510 2024-03-26T17:37:07.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.198+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f570c1a7750 con 0x7f570c108510 2024-03-26T17:37:07.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.199+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f570c062ac0 con 0x7f570c108510 2024-03-26T17:37:07.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.200+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f570401a020 con 0x7f570c108510 2024-03-26T17:37:07.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.201+0000 7f56fb7fe700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56f40a30c0 0x7f56f40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.201+0000 7f570a59c700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56f40a30c0 0x7f56f40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.201+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f57040cf120 con 0x7f570c108510 2024-03-26T17:37:07.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.201+0000 7f56fb7fe700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] conn(0x7f56f40ac8a0 0x7f56f40aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.201+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 --> [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f570400ae50 con 0x7f56f40ac8a0 2024-03-26T17:37:07.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.201+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f57040cf600 con 0x7f570c108510 2024-03-26T17:37:07.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.201+0000 7f570a59c700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56f40a30c0 0x7f56f40a5580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f570000b5c0 tx=0x7f5700005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:07.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.201+0000 7f5709d9b700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] conn(0x7f56f40ac8a0 0x7f56f40aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.202+0000 7f5709d9b700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] conn(0x7f56f40ac8a0 0x7f56f40aecc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:07.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.203+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 <== osd.11 v2:172.21.15.144:6824/4293375044 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f570400ae50 con 0x7f56f40ac8a0 2024-03-26T17:37:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.228+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 --> [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f570c062f40 con 0x7f56f40ac8a0 2024-03-26T17:37:07.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.228+0000 7f56fb7fe700 1 -- 172.21.15.105:0/2592307390 <== osd.11 v2:172.21.15.144:6824/4293375044 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f570c062f40 con 0x7f56f40ac8a0 2024-03-26T17:37:07.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.228+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 >> [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] conn(0x7f56f40ac8a0 msgr2=0x7f56f40aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.228+0000 7f5710c83700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] conn(0x7f56f40ac8a0 0x7f56f40aecc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56f40a30c0 msgr2=0x7f56f40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f5710c83700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56f40a30c0 0x7f56f40a5580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f570000b5c0 tx=0x7f5700005960 comp rx=0 tx=0).stop 2024-03-26T17:37:07.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 msgr2=0x7f570c075130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f5710c83700 1 --2- 172.21.15.105:0/2592307390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f570c108510 0x7f570c075130 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5704007ee0 tx=0x7f570400d3b0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f570ad9d700 1 -- 172.21.15.105:0/2592307390 reap_dead start 2024-03-26T17:37:07.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 shutdown_connections 2024-03-26T17:37:07.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 >> 172.21.15.105:0/2592307390 conn(0x7f570c079470 msgr2=0x7f570c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:07.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 shutdown_connections 2024-03-26T17:37:07.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:07.229+0000 7f5710c83700 1 -- 172.21.15.105:0/2592307390 wait complete. 2024-03-26T17:37:07.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.077+0000 7f75474ca700 1 -- 172.21.15.105:0/1135839928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7540108500 msgr2=0x7f75401088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.077+0000 7f75474ca700 1 --2- 172.21.15.105:0/1135839928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7540108500 0x7f75401088e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f753c0098f0 tx=0x7f753c009c00 comp rx=0 tx=0).stop 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.077+0000 7f75474ca700 1 -- 172.21.15.105:0/1135839928 shutdown_connections 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.078+0000 7f75474ca700 1 --2- 172.21.15.105:0/1135839928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540101360 0x7f754010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.078+0000 7f75474ca700 1 --2- 172.21.15.105:0/1135839928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7540108500 0x7f75401088e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.078+0000 7f75474ca700 1 --2- 172.21.15.105:0/1135839928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75401009a0 0x7f7540100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.078+0000 7f75474ca700 1 -- 172.21.15.105:0/1135839928 >> 172.21.15.105:0/1135839928 conn(0x7f75400778b0 msgr2=0x7f75400fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.078+0000 7f75474ca700 1 -- 172.21.15.105:0/1135839928 shutdown_connections 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.078+0000 7f75474ca700 1 -- 172.21.15.105:0/1135839928 wait complete. 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.078+0000 7f75474ca700 1 Processor -- start 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.078+0000 7f75474ca700 1 -- start start 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f75474ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75401009a0 0x7f7540101b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f75474ca700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7540101360 0x7f75401020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f75474ca700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540108500 0x7f7540102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f75474ca700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7540112580 con 0x7f7540108500 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f75474ca700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7540112400 con 0x7f7540101360 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f75474ca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7540112700 con 0x7f75401009a0 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540108500 0x7f7540102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545266700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75401009a0 0x7f7540101b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540108500 0x7f7540102600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36294/0 (socket says 172.21.15.105:36294) 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 -- 172.21.15.105:0/4252869608 learned_addr learned my addr 172.21.15.105:0/4252869608 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7544a65700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7540101360 0x7f75401020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 -- 172.21.15.105:0/4252869608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75401009a0 msgr2=0x7f7540101b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75401009a0 0x7f7540101b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 -- 172.21.15.105:0/4252869608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7540101360 msgr2=0x7f75401020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7540101360 0x7f75401020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 -- 172.21.15.105:0/4252869608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75401a6f70 con 0x7f7540108500 2024-03-26T17:37:07.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545266700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75401009a0 0x7f7540101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:07.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f7545a67700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540108500 0x7f7540102600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f753800c110 tx=0x7f753800c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:07.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75380092e0 con 0x7f7540108500 2024-03-26T17:37:07.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.079+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75401a7200 con 0x7f7540108500 2024-03-26T17:37:07.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.080+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7538004bb0 con 0x7f7540108500 2024-03-26T17:37:07.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.080+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538007c70 con 0x7f7540108500 2024-03-26T17:37:07.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.080+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75401a7790 con 0x7f7540108500 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.080+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7540062ac0 con 0x7f7540108500 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.082+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f753801a020 con 0x7f7540108500 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.082+0000 7f75367fc700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f752c0a3040 0x7f752c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.083+0000 7f7545266700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f752c0a3040 0x7f752c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.083+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f75380cf840 con 0x7f7540108500 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.083+0000 7f75367fc700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] conn(0x7f752c0ac820 0x7f752c0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.083+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 --> [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7538008ec0 con 0x7f752c0ac820 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.083+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f75380cfd20 con 0x7f7540108500 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.083+0000 7f7544a65700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] conn(0x7f752c0ac820 0x7f752c0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.083+0000 7f7545266700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f752c0a3040 0x7f752c0a5500 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f753c009fd0 tx=0x7f753c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.083+0000 7f7544a65700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] conn(0x7f752c0ac820 0x7f752c0aec40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.084+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 <== osd.0 v2:172.21.15.105:6802/101464397 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7538008ec0 con 0x7f752c0ac820 2024-03-26T17:37:07.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.100+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 --> [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7540062ec0 con 0x7f752c0ac820 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.100+0000 7f75367fc700 1 -- 172.21.15.105:0/4252869608 <== osd.0 v2:172.21.15.105:6802/101464397 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7540062ec0 con 0x7f752c0ac820 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] conn(0x7f752c0ac820 msgr2=0x7f752c0aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] conn(0x7f752c0ac820 0x7f752c0aec40 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f752c0a3040 msgr2=0x7f752c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f752c0a3040 0x7f752c0a5500 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f753c009fd0 tx=0x7f753c019040 comp rx=0 tx=0).stop 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540108500 msgr2=0x7f7540102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 --2- 172.21.15.105:0/4252869608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7540108500 0x7f7540102600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f753800c110 tx=0x7f753800c420 comp rx=0 tx=0).stop 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f7545a67700 1 -- 172.21.15.105:0/4252869608 reap_dead start 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 shutdown_connections 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 >> 172.21.15.105:0/4252869608 conn(0x7f75400778b0 msgr2=0x7f75400ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:07.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 shutdown_connections 2024-03-26T17:37:07.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:04.101+0000 7f75474ca700 1 -- 172.21.15.105:0/4252869608 wait complete. 2024-03-26T17:37:08.050 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:07 smithi105 ceph-mon[32280]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:08.076 INFO:teuthology.orchestra.run.smithi105.stdout:34359738444 2024-03-26T17:37:08.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-26T17:37:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:07 smithi144 ceph-mon[36095]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:08.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:07 smithi135 ceph-mon[35636]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:08.812 INFO:teuthology.orchestra.run.smithi105.stdout:343597383703 2024-03-26T17:37:08.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-26T17:37:09.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 -- 172.21.15.105:0/3548491090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde0106590 msgr2=0x7fcde010e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:09.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 --2- 172.21.15.105:0/3548491090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde0106590 0x7fcde010e160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800b0d0 tx=0x7fcdd800b3e0 comp rx=0 tx=0).stop 2024-03-26T17:37:09.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 -- 172.21.15.105:0/3548491090 shutdown_connections 2024-03-26T17:37:09.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 --2- 172.21.15.105:0/3548491090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde0106590 0x7fcde010e160 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:09.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 --2- 172.21.15.105:0/3548491090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde0101540 0x7fcde0101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:09.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 --2- 172.21.15.105:0/3548491090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde0101ef0 0x7fcde0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:09.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 -- 172.21.15.105:0/3548491090 >> 172.21.15.105:0/3548491090 conn(0x7fcde0077300 msgr2=0x7fcde0077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:09.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 -- 172.21.15.105:0/3548491090 shutdown_connections 2024-03-26T17:37:09.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.259+0000 7fcde87cd700 1 -- 172.21.15.105:0/3548491090 wait complete. 2024-03-26T17:37:09.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde87cd700 1 Processor -- start 2024-03-26T17:37:09.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde87cd700 1 -- start start 2024-03-26T17:37:09.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde87cd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde0101540 0x7fcde019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:09.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde87cd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde0101ef0 0x7fcde019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde87cd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde0106590 0x7fcde01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde87cd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcde0111450 con 0x7fcde0106590 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde87cd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01112d0 con 0x7fcde0101ef0 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde87cd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcde01115d0 con 0x7fcde0101540 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde6569700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde0101540 0x7fcde019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde6569700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde0101540 0x7fcde019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40416/0 (socket says 172.21.15.105:40416) 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde6569700 1 -- 172.21.15.105:0/164105117 learned_addr learned my addr 172.21.15.105:0/164105117 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde5d68700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde0101ef0 0x7fcde019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde6d6a700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde0106590 0x7fcde01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde5d68700 1 -- 172.21.15.105:0/164105117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde0101540 msgr2=0x7fcde019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:09.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde5d68700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde0101540 0x7fcde019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:09.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde5d68700 1 -- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde0106590 msgr2=0x7fcde01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:09.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde5d68700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcde0106590 0x7fcde01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:09.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde5d68700 1 -- 172.21.15.105:0/164105117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde0198fb0 con 0x7fcde0101ef0 2024-03-26T17:37:09.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.260+0000 7fcde6569700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcde0101540 0x7fcde019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:09.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.261+0000 7fcde5d68700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde0101ef0 0x7fcde019eb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000e9c0 tx=0x7fcdd000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:09.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.261+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0004220 con 0x7fcde0101ef0 2024-03-26T17:37:09.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.261+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdd00043c0 con 0x7fcde0101ef0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.261+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde01992a0 con 0x7fcde0101ef0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.261+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0005020 con 0x7fcde0101ef0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.261+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde01a54b0 con 0x7fcde0101ef0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.261+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcde0062b40 con 0x7fcde0101ef0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.263+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fcdd0004750 con 0x7fcde0101ef0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.263+0000 7fcdd77fe700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdcc0a3040 0x7fcdcc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.263+0000 7fcde6569700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdcc0a3040 0x7fcdcc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.263+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fcdd00cf650 con 0x7fcde0101ef0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.264+0000 7fcdd77fe700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] conn(0x7fcdcc0ac820 0x7fcdcc0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.264+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 --> [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcdd001ce50 con 0x7fcdcc0ac820 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.264+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fcdd00cfa00 con 0x7fcde0101ef0 2024-03-26T17:37:09.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.264+0000 7fcde6d6a700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] conn(0x7fcdcc0ac820 0x7fcdcc0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.264+0000 7fcde6569700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdcc0a3040 0x7fcdcc0a5500 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011550 tx=0x7fcddc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.264+0000 7fcde6d6a700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] conn(0x7fcdcc0ac820 0x7fcdcc0aec40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:09.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.265+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 <== osd.3 v2:172.21.15.105:6810/2769664525 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcdd001ce50 con 0x7fcdcc0ac820 2024-03-26T17:37:09.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.282+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 --> [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcde0062fc0 con 0x7fcdcc0ac820 2024-03-26T17:37:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.282+0000 7fcdd77fe700 1 -- 172.21.15.105:0/164105117 <== osd.3 v2:172.21.15.105:6810/2769664525 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcde0062fc0 con 0x7fcdcc0ac820 2024-03-26T17:37:09.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] conn(0x7fcdcc0ac820 msgr2=0x7fcdcc0aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] conn(0x7fcdcc0ac820 0x7fcdcc0aec40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdcc0a3040 msgr2=0x7fcdcc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdcc0a3040 0x7fcdcc0a5500 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011550 tx=0x7fcddc009260 comp rx=0 tx=0).stop 2024-03-26T17:37:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde0101ef0 msgr2=0x7fcde019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 --2- 172.21.15.105:0/164105117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcde0101ef0 0x7fcde019eb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000e9c0 tx=0x7fcdd000ecd0 comp rx=0 tx=0).stop 2024-03-26T17:37:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde6d6a700 1 -- 172.21.15.105:0/164105117 reap_dead start 2024-03-26T17:37:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 shutdown_connections 2024-03-26T17:37:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 >> 172.21.15.105:0/164105117 conn(0x7fcde0077300 msgr2=0x7fcde006ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 shutdown_connections 2024-03-26T17:37:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:09.283+0000 7fcde87cd700 1 -- 172.21.15.105:0/164105117 wait complete. 2024-03-26T17:37:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:09.946 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:09 smithi105 ceph-mon[32280]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-26T17:37:09.972 INFO:teuthology.orchestra.run.smithi105.stdout:107374182462 2024-03-26T17:37:09.972 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-26T17:37:10.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:09 smithi144 ceph-mon[36095]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-26T17:37:10.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:09 smithi135 ceph-mon[35636]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-26T17:37:10.266 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:10.268 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:10.269 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:11.436 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:11.436 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:11.869 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:11 smithi105 ceph-mon[32280]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:11 smithi144 ceph-mon[36095]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:11 smithi135 ceph-mon[35636]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:37:12.420 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:13.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:13 smithi105 ceph-mon[32280]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-26T17:37:14.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:13 smithi144 ceph-mon[36095]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-26T17:37:14.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:13 smithi135 ceph-mon[35636]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-26T17:37:15.852 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:15 smithi105 ceph-mon[32280]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:16.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:15 smithi144 ceph-mon[36095]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:16.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:15 smithi135 ceph-mon[35636]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:17.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:16 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/691843082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T17:37:17.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:16 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/691843082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T17:37:17.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:16 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/691843082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T17:37:18.024 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:17 smithi105 ceph-mon[32280]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-26T17:37:18.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:17 smithi144 ceph-mon[36095]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-26T17:37:18.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:17 smithi135 ceph-mon[35636]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-03-26T17:37:18.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 -- 172.21.15.105:0/1651317046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 msgr2=0x7f795810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:18.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 --2- 172.21.15.105:0/1651317046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 0x7f795810f240 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f795000b0d0 tx=0x7f795000b3e0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 -- 172.21.15.105:0/1651317046 shutdown_connections 2024-03-26T17:37:18.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 --2- 172.21.15.105:0/1651317046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 0x7f795810f240 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 --2- 172.21.15.105:0/1651317046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7958108420 0x7f7958108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 --2- 172.21.15.105:0/1651317046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f795806aab0 0x7f795806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 -- 172.21.15.105:0/1651317046 >> 172.21.15.105:0/1651317046 conn(0x7f7958077300 msgr2=0x7f7958077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 -- 172.21.15.105:0/1651317046 shutdown_connections 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 -- 172.21.15.105:0/1651317046 wait complete. 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.624+0000 7f796066f700 1 Processor -- start 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f796066f700 1 -- start start 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f796066f700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795806aab0 0x7f79581013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f796066f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 0x7f7958101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f796066f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958108420 0x7f7958101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:18.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f796066f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7958112580 con 0x7f795806b470 2024-03-26T17:37:18.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f796066f700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7958112400 con 0x7f795806aab0 2024-03-26T17:37:18.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f796066f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7958112700 con 0x7f7958108420 2024-03-26T17:37:18.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 0x7f7958101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:18.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 0x7f7958101920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36058/0 (socket says 172.21.15.105:36058) 2024-03-26T17:37:18.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 -- 172.21.15.105:0/691843082 learned_addr learned my addr 172.21.15.105:0/691843082 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:18.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795ec0c700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958108420 0x7f7958101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:18.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795e40b700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795806aab0 0x7f79581013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:18.840 INFO:teuthology.orchestra.run.smithi105.stdout:55834574921 2024-03-26T17:37:18.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 -- 172.21.15.105:0/691843082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958108420 msgr2=0x7f7958101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:18.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958108420 0x7f7958101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 -- 172.21.15.105:0/691843082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795806aab0 msgr2=0x7f79581013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:18.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795806aab0 0x7f79581013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 -- 172.21.15.105:0/691843082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79581a6f10 con 0x7f795806b470 2024-03-26T17:37:18.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795e40b700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795806aab0 0x7f79581013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f795dc0a700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 0x7f7958101920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f794800e9c0 tx=0x7f794800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.625+0000 7f794f7fe700 1 -- 172.21.15.105:0/691843082 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794800cbd0 con 0x7f795806b470 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.626+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79581a7200 con 0x7f795806b470 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.626+0000 7f794f7fe700 1 -- 172.21.15.105:0/691843082 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f794800ee60 con 0x7f795806b470 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.626+0000 7f794f7fe700 1 -- 172.21.15.105:0/691843082 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79480055b0 con 0x7f795806b470 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.626+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79581a76f0 con 0x7f795806b470 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.627+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7958069070 con 0x7f795806b470 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.629+0000 7f794f7fe700 1 -- 172.21.15.105:0/691843082 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f7948004750 con 0x7f795806b470 2024-03-26T17:37:18.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.629+0000 7f794f7fe700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79440a3040 0x7f79440a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.629+0000 7f795e40b700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79440a3040 0x7f79440a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.629+0000 7f794f7fe700 1 -- 172.21.15.105:0/691843082 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f79480cf860 con 0x7f795806b470 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.630+0000 7f795e40b700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79440a3040 0x7f79440a5500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7954009680 tx=0x7f7954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.633+0000 7f794f7fe700 1 -- 172.21.15.105:0/691843082 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7948053740 con 0x7f795806b470 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.858+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f795802dc00 con 0x7f795806b470 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.859+0000 7f794f7fe700 1 -- 172.21.15.105:0/691843082 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f794809c6e0 con 0x7f795806b470 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79440a3040 msgr2=0x7f79440a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79440a3040 0x7f79440a5500 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7954009680 tx=0x7f7954009260 comp rx=0 tx=0).stop 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 msgr2=0x7f7958101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:18.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 0x7f7958101920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f794800e9c0 tx=0x7f794800ecd0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 shutdown_connections 2024-03-26T17:37:18.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79440a3040 0x7f79440a5500 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7958108420 0x7f7958101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f795806aab0 0x7f79581013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 --2- 172.21.15.105:0/691843082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f795806b470 0x7f7958101920 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:18.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.861+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 >> 172.21.15.105:0/691843082 conn(0x7f7958077300 msgr2=0x7f79580fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:18.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.862+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 shutdown_connections 2024-03-26T17:37:18.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:15.862+0000 7f796066f700 1 -- 172.21.15.105:0/691843082 wait complete. 2024-03-26T17:37:19.367 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574921 for osd.1 2024-03-26T17:37:19.368 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:19.852 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:19.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:19 smithi105 ceph-mon[32280]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:37:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:19 smithi144 ceph-mon[36095]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:37:20.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:19 smithi135 ceph-mon[35636]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:37:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 -- 172.21.15.105:0/4181664400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e6406aa00 msgr2=0x7f7e6410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:20.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/4181664400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e6406aa00 0x7f7e6410f240 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00b0d0 tx=0x7f7e5c00b3e0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 -- 172.21.15.105:0/4181664400 shutdown_connections 2024-03-26T17:37:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/4181664400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e6406aa00 0x7f7e6410f240 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/4181664400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e64108400 0x7f7e641087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/4181664400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6406a060 0x7f7e6406a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 -- 172.21.15.105:0/4181664400 >> 172.21.15.105:0/4181664400 conn(0x7f7e640781e0 msgr2=0x7f7e640785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 -- 172.21.15.105:0/4181664400 shutdown_connections 2024-03-26T17:37:20.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.464+0000 7f7e6b65a700 1 -- 172.21.15.105:0/4181664400 wait complete. 2024-03-26T17:37:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.465+0000 7f7e6b65a700 1 Processor -- start 2024-03-26T17:37:20.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.465+0000 7f7e6b65a700 1 -- start start 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.465+0000 7f7e6b65a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6406a060 0x7f7e6419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e6b65a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e6406aa00 0x7f7e6419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e6b65a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e64108400 0x7f7e641988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e6b65a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e64112530 con 0x7f7e64108400 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e6b65a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641123b0 con 0x7f7e6406aa00 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e6b65a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641126b0 con 0x7f7e6406a060 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e64108400 0x7f7e641988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e64108400 0x7f7e641988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36076/0 (socket says 172.21.15.105:36076) 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 -- 172.21.15.105:0/442889381 learned_addr learned my addr 172.21.15.105:0/442889381 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:20.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e68bf5700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e6406aa00 0x7f7e6419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e693f6700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6406a060 0x7f7e6419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 -- 172.21.15.105:0/442889381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6406a060 msgr2=0x7f7e6419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6406a060 0x7f7e6419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 -- 172.21.15.105:0/442889381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e6406aa00 msgr2=0x7f7e6419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e6406aa00 0x7f7e6419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 -- 172.21.15.105:0/442889381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e64199050 con 0x7f7e64108400 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e69bf7700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e64108400 0x7f7e641988f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c007a00 tx=0x7f7e5c007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e5a7fc700 1 -- 172.21.15.105:0/442889381 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c01f070 con 0x7f7e64108400 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e641992e0 con 0x7f7e64108400 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e5a7fc700 1 -- 172.21.15.105:0/442889381 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e5c004c50 con 0x7f7e64108400 2024-03-26T17:37:20.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e5a7fc700 1 -- 172.21.15.105:0/442889381 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c01b690 con 0x7f7e64108400 2024-03-26T17:37:20.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.466+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e641a55b0 con 0x7f7e64108400 2024-03-26T17:37:20.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.468+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e64069070 con 0x7f7e64108400 2024-03-26T17:37:20.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.469+0000 7f7e5a7fc700 1 -- 172.21.15.105:0/442889381 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f7e5c022070 con 0x7f7e64108400 2024-03-26T17:37:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.469+0000 7f7e5a7fc700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7e500a3040 0x7f7e500a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.469+0000 7f7e693f6700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7e500a3040 0x7f7e500a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.469+0000 7f7e5a7fc700 1 -- 172.21.15.105:0/442889381 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7e5c0cfed0 con 0x7f7e64108400 2024-03-26T17:37:20.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.470+0000 7f7e693f6700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7e500a3040 0x7f7e500a5500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e64107f60 tx=0x7f7e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:20.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.474+0000 7f7e5a7fc700 1 -- 172.21.15.105:0/442889381 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7e5c09cde0 con 0x7f7e64108400 2024-03-26T17:37:20.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.696+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7e6402dc00 con 0x7f7e64108400 2024-03-26T17:37:20.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.696+0000 7f7e5a7fc700 1 -- 172.21.15.105:0/442889381 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7e5c09c6e0 con 0x7f7e64108400 2024-03-26T17:37:20.690 INFO:teuthology.orchestra.run.smithi105.stdout:167503724599 2024-03-26T17:37:20.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7e500a3040 msgr2=0x7f7e500a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:20.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7e500a3040 0x7f7e500a5500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e64107f60 tx=0x7f7e60009260 comp rx=0 tx=0).stop 2024-03-26T17:37:20.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e64108400 msgr2=0x7f7e641988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:20.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e64108400 0x7f7e641988f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c007a00 tx=0x7f7e5c007a30 comp rx=0 tx=0).stop 2024-03-26T17:37:20.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 shutdown_connections 2024-03-26T17:37:20.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7e500a3040 0x7f7e500a5500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e64108400 0x7f7e641988f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6406a060 0x7f7e6419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 --2- 172.21.15.105:0/442889381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e6406aa00 0x7f7e6419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:20.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 >> 172.21.15.105:0/442889381 conn(0x7f7e640781e0 msgr2=0x7f7e640fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:20.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 shutdown_connections 2024-03-26T17:37:20.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:20.698+0000 7f7e6b65a700 1 -- 172.21.15.105:0/442889381 wait complete. 2024-03-26T17:37:20.751 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:20 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/442889381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T17:37:21.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:20 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/442889381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T17:37:21.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:20 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/442889381' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T17:37:21.911 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.939+0000 7f7117891700 1 -- 172.21.15.105:0/1769073385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110101360 msgr2=0x7f711010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.939+0000 7f7117891700 1 --2- 172.21.15.105:0/1769073385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110101360 0x7f711010f290 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f710800b0a0 tx=0x7f710800b3b0 comp rx=0 tx=0).stop 2024-03-26T17:37:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.939+0000 7f7117891700 1 -- 172.21.15.105:0/1769073385 shutdown_connections 2024-03-26T17:37:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.939+0000 7f7117891700 1 --2- 172.21.15.105:0/1769073385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110101360 0x7f711010f290 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.939+0000 7f7117891700 1 --2- 172.21.15.105:0/1769073385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7110108500 0x7f71101088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.939+0000 7f7117891700 1 --2- 172.21.15.105:0/1769073385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71101009a0 0x7f7110100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:21.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.939+0000 7f7117891700 1 -- 172.21.15.105:0/1769073385 >> 172.21.15.105:0/1769073385 conn(0x7f71100778b0 msgr2=0x7f71100fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:21.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.940+0000 7f7117891700 1 -- 172.21.15.105:0/1769073385 shutdown_connections 2024-03-26T17:37:21.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.940+0000 7f7117891700 1 -- 172.21.15.105:0/1769073385 wait complete. 2024-03-26T17:37:21.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.941+0000 7f7117891700 1 Processor -- start 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.941+0000 7f7117891700 1 -- start start 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.941+0000 7f7117891700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71101009a0 0x7f711019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f7117891700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7110101360 0x7f711019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f711562d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71101009a0 0x7f711019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f711562d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71101009a0 0x7f711019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53656/0 (socket says 172.21.15.105:53656) 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f7114e2c700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7110101360 0x7f711019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f7117891700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110108500 0x7f7110198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f711562d700 1 -- 172.21.15.105:0/1033172759 learned_addr learned my addr 172.21.15.105:0/1033172759 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:21.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f7115e2e700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110108500 0x7f7110198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f71101125e0 con 0x7f7110108500 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7110112460 con 0x7f7110101360 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.942+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7110112760 con 0x7f71101009a0 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.943+0000 7f711562d700 1 -- 172.21.15.105:0/1033172759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7110101360 msgr2=0x7f711019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.943+0000 7f711562d700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7110101360 0x7f711019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.943+0000 7f711562d700 1 -- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110108500 msgr2=0x7f7110198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.943+0000 7f711562d700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110108500 0x7f7110198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.943+0000 7f711562d700 1 -- 172.21.15.105:0/1033172759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7110199060 con 0x7f71101009a0 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.943+0000 7f7115e2e700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110108500 0x7f7110198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.943+0000 7f711562d700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71101009a0 0x7f711019e6b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f710c009cb0 tx=0x7f710c00b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.943+0000 7f71067fc700 1 -- 172.21.15.105:0/1033172759 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c01e070 con 0x7f71101009a0 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.944+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7110199350 con 0x7f71101009a0 2024-03-26T17:37:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.944+0000 7f71067fc700 1 -- 172.21.15.105:0/1033172759 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f710c019410 con 0x7f71101009a0 2024-03-26T17:37:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.944+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7110101ab0 con 0x7f71101009a0 2024-03-26T17:37:21.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.944+0000 7f71067fc700 1 -- 172.21.15.105:0/1033172759 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c018610 con 0x7f71101009a0 2024-03-26T17:37:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.945+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f4005330 con 0x7f71101009a0 2024-03-26T17:37:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.946+0000 7f71067fc700 1 -- 172.21.15.105:0/1033172759 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f710c021070 con 0x7f71101009a0 2024-03-26T17:37:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.946+0000 7f71067fc700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70fc0a3100 0x7f70fc0a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.947+0000 7f7114e2c700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70fc0a3100 0x7f70fc0a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.947+0000 7f71067fc700 1 -- 172.21.15.105:0/1033172759 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f710c0d2ca0 con 0x7f71101009a0 2024-03-26T17:37:21.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.947+0000 7f7114e2c700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70fc0a3100 0x7f70fc0a55c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7110107b00 tx=0x7f7100009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:21.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:21.950+0000 7f71067fc700 1 -- 172.21.15.105:0/1033172759 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f710c09fa20 con 0x7f71101009a0 2024-03-26T17:37:22.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:21 smithi105 ceph-mon[32280]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:22.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:37:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:21 smithi144 ceph-mon[36095]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:37:22.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:21 smithi135 ceph-mon[35636]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:22.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:37:22.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.174+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f70f40058f0 con 0x7f71101009a0 2024-03-26T17:37:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.174+0000 7f71067fc700 1 -- 172.21.15.105:0/1033172759 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f710c09f320 con 0x7f71101009a0 2024-03-26T17:37:22.168 INFO:teuthology.orchestra.run.smithi105.stdout:223338299438 2024-03-26T17:37:22.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70fc0a3100 msgr2=0x7f70fc0a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70fc0a3100 0x7f70fc0a55c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7110107b00 tx=0x7f7100009260 comp rx=0 tx=0).stop 2024-03-26T17:37:22.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71101009a0 msgr2=0x7f711019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71101009a0 0x7f711019e6b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f710c009cb0 tx=0x7f710c00b2c0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 shutdown_connections 2024-03-26T17:37:22.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70fc0a3100 0x7f70fc0a55c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7110108500 0x7f7110198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71101009a0 0x7f711019e6b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 --2- 172.21.15.105:0/1033172759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7110101360 0x7f711019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 >> 172.21.15.105:0/1033172759 conn(0x7f71100778b0 msgr2=0x7f71100ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:22.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 shutdown_connections 2024-03-26T17:37:22.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.177+0000 7f7117891700 1 -- 172.21.15.105:0/1033172759 wait complete. 2024-03-26T17:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 -- 172.21.15.105:0/733630313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 msgr2=0x7fb14806af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 --2- 172.21.15.105:0/733630313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 0x7fb14806af30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb1440099c0 tx=0x7fb144009cd0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 -- 172.21.15.105:0/733630313 shutdown_connections 2024-03-26T17:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 --2- 172.21.15.105:0/733630313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb148107690 0x7fb14810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 --2- 172.21.15.105:0/733630313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 0x7fb14806af30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 --2- 172.21.15.105:0/733630313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb14806b500 0x7fb148106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 -- 172.21.15.105:0/733630313 >> 172.21.15.105:0/733630313 conn(0x7fb148077300 msgr2=0x7fb148077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 -- 172.21.15.105:0/733630313 shutdown_connections 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.359+0000 7fb1503b1700 1 -- 172.21.15.105:0/733630313 wait complete. 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb1503b1700 1 Processor -- start 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb1503b1700 1 -- start start 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb1503b1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 0x7fb14819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb1503b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb14806b500 0x7fb14819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb1503b1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb148107690 0x7fb1481988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb1503b1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb148112520 con 0x7fb14806ab50 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb1503b1700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1481123a0 con 0x7fb148107690 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb1503b1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1481126a0 con 0x7fb14806b500 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb14e14d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 0x7fb14819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:22.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb14e14d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 0x7fb14819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36106/0 (socket says 172.21.15.105:36106) 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb14e14d700 1 -- 172.21.15.105:0/1010079354 learned_addr learned my addr 172.21.15.105:0/1010079354 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb14d94c700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb14806b500 0x7fb14819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.360+0000 7fb14e94e700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb148107690 0x7fb1481988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb14d94c700 1 -- 172.21.15.105:0/1010079354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb148107690 msgr2=0x7fb1481988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb14d94c700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb148107690 0x7fb1481988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb14d94c700 1 -- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 msgr2=0x7fb14819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb14d94c700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 0x7fb14819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb14d94c700 1 -- 172.21.15.105:0/1010079354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb148199020 con 0x7fb14806b500 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb14e14d700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 0x7fb14819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb14d94c700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb14806b500 0x7fb14819ebf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e9c0 tx=0x7fb13800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb13f7fe700 1 -- 172.21.15.105:0/1010079354 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800c9c0 con 0x7fb14806b500 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb148199310 con 0x7fb14806b500 2024-03-26T17:37:22.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb13f7fe700 1 -- 172.21.15.105:0/1010079354 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb13800cb60 con 0x7fb14806b500 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb13f7fe700 1 -- 172.21.15.105:0/1010079354 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138018680 con 0x7fb14806b500 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.361+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1481a5580 con 0x7fb14806b500 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.362+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb148069070 con 0x7fb14806b500 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.363+0000 7fb13f7fe700 1 -- 172.21.15.105:0/1010079354 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fb138010430 con 0x7fb14806b500 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.364+0000 7fb13f7fe700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1340a30c0 0x7fb1340a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.364+0000 7fb14e14d700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1340a30c0 0x7fb1340a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.364+0000 7fb13f7fe700 1 -- 172.21.15.105:0/1010079354 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb1380cf490 con 0x7fb14806b500 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.364+0000 7fb14e14d700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1340a30c0 0x7fb1340a5580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb144009990 tx=0x7fb144005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:22.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.369+0000 7fb13f7fe700 1 -- 172.21.15.105:0/1010079354 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb13809c290 con 0x7fb14806b500 2024-03-26T17:37:22.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.586+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb14802dc00 con 0x7fb14806b500 2024-03-26T17:37:22.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.587+0000 7fb13f7fe700 1 -- 172.21.15.105:0/1010079354 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb13809bb90 con 0x7fb14806b500 2024-03-26T17:37:22.581 INFO:teuthology.orchestra.run.smithi105.stdout:141733920829 2024-03-26T17:37:22.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1340a30c0 msgr2=0x7fb1340a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1340a30c0 0x7fb1340a5580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb144009990 tx=0x7fb144005960 comp rx=0 tx=0).stop 2024-03-26T17:37:22.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb14806b500 msgr2=0x7fb14819ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb14806b500 0x7fb14819ebf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e9c0 tx=0x7fb13800ecd0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 shutdown_connections 2024-03-26T17:37:22.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1340a30c0 0x7fb1340a5580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb148107690 0x7fb1481988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb14806ab50 0x7fb14819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 --2- 172.21.15.105:0/1010079354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb14806b500 0x7fb14819ebf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 >> 172.21.15.105:0/1010079354 conn(0x7fb148077300 msgr2=0x7fb148106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:22.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 shutdown_connections 2024-03-26T17:37:22.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.590+0000 7fb1503b1700 1 -- 172.21.15.105:0/1010079354 wait complete. 2024-03-26T17:37:22.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.618+0000 7f062eddf700 1 -- 172.21.15.105:0/1738656748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628101360 msgr2=0x7f062810f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.618+0000 7f062eddf700 1 --2- 172.21.15.105:0/1738656748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628101360 0x7f062810f290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f062000b0a0 tx=0x7f062000b3b0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.618+0000 7f062eddf700 1 -- 172.21.15.105:0/1738656748 shutdown_connections 2024-03-26T17:37:22.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.618+0000 7f062eddf700 1 --2- 172.21.15.105:0/1738656748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628101360 0x7f062810f290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.619+0000 7f062eddf700 1 --2- 172.21.15.105:0/1738656748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0628108500 0x7f06281088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.619+0000 7f062eddf700 1 --2- 172.21.15.105:0/1738656748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281009a0 0x7f0628100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.619+0000 7f062eddf700 1 -- 172.21.15.105:0/1738656748 >> 172.21.15.105:0/1738656748 conn(0x7f06280778b0 msgr2=0x7f06280fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:22.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.619+0000 7f062eddf700 1 -- 172.21.15.105:0/1738656748 shutdown_connections 2024-03-26T17:37:22.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.619+0000 7f062eddf700 1 -- 172.21.15.105:0/1738656748 wait complete. 2024-03-26T17:37:22.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.619+0000 7f062eddf700 1 Processor -- start 2024-03-26T17:37:22.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.619+0000 7f062eddf700 1 -- start start 2024-03-26T17:37:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062eddf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281009a0 0x7f062819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062eddf700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0628101360 0x7f062819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062eddf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628108500 0x7f0628198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062eddf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0628112600 con 0x7f0628108500 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062eddf700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0628112480 con 0x7f0628101360 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062eddf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0628112780 con 0x7f06281009a0 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f0627fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0628101360 0x7f062819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f0627fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0628101360 0x7f062819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42498/0 (socket says 172.21.15.105:42498) 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062d37c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628108500 0x7f0628198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f0627fff700 1 -- 172.21.15.105:0/1557949479 learned_addr learned my addr 172.21.15.105:0/1557949479 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062cb7b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281009a0 0x7f062819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:22.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062d37c700 1 -- 172.21.15.105:0/1557949479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281009a0 msgr2=0x7f062819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062d37c700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281009a0 0x7f062819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062d37c700 1 -- 172.21.15.105:0/1557949479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0628101360 msgr2=0x7f062819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062d37c700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0628101360 0x7f062819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062d37c700 1 -- 172.21.15.105:0/1557949479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0628198fe0 con 0x7f0628108500 2024-03-26T17:37:22.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f0627fff700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0628101360 0x7f062819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:22.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.620+0000 7f062d37c700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628108500 0x7f0628198880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f062000b070 tx=0x7f0620004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:22.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.621+0000 7f0625ffb700 1 -- 172.21.15.105:0/1557949479 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620021070 con 0x7f0628108500 2024-03-26T17:37:22.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.621+0000 7f0625ffb700 1 -- 172.21.15.105:0/1557949479 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0620007e30 con 0x7f0628108500 2024-03-26T17:37:22.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.621+0000 7f0625ffb700 1 -- 172.21.15.105:0/1557949479 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200045f0 con 0x7f0628108500 2024-03-26T17:37:22.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.621+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0628199270 con 0x7f0628108500 2024-03-26T17:37:22.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.621+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06281a5590 con 0x7f0628108500 2024-03-26T17:37:22.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.623+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0628069070 con 0x7f0628108500 2024-03-26T17:37:22.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.623+0000 7f0625ffb700 1 -- 172.21.15.105:0/1557949479 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f0620007980 con 0x7f0628108500 2024-03-26T17:37:22.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.624+0000 7f0625ffb700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06180a3040 0x7f06180a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:22.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.624+0000 7f062cb7b700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06180a3040 0x7f06180a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:22.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.624+0000 7f0625ffb700 1 -- 172.21.15.105:0/1557949479 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f062000f020 con 0x7f0628108500 2024-03-26T17:37:22.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.625+0000 7f062cb7b700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06180a3040 0x7f06180a5500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06281999c0 tx=0x7f061c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:22.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.632+0000 7f0625ffb700 1 -- 172.21.15.105:0/1557949479 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f062002a020 con 0x7f0628108500 2024-03-26T17:37:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.834+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f062802dc00 con 0x7f0628108500 2024-03-26T17:37:22.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.835+0000 7f0625ffb700 1 -- 172.21.15.105:0/1557949479 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f062009d560 con 0x7f0628108500 2024-03-26T17:37:22.829 INFO:teuthology.orchestra.run.smithi105.stdout:343597383706 2024-03-26T17:37:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06180a3040 msgr2=0x7f06180a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06180a3040 0x7f06180a5500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06281999c0 tx=0x7f061c009260 comp rx=0 tx=0).stop 2024-03-26T17:37:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628108500 msgr2=0x7f0628198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:22.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628108500 0x7f0628198880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f062000b070 tx=0x7f0620004c80 comp rx=0 tx=0).stop 2024-03-26T17:37:22.831 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1033172759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T17:37:22.831 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/833360695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T17:37:22.831 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1010079354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T17:37:22.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 shutdown_connections 2024-03-26T17:37:22.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06180a3040 0x7f06180a5500 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0628108500 0x7f0628198880 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06281009a0 0x7f062819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 --2- 172.21.15.105:0/1557949479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0628101360 0x7f062819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:22.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 >> 172.21.15.105:0/1557949479 conn(0x7f06280778b0 msgr2=0x7f06280fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:22.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 shutdown_connections 2024-03-26T17:37:22.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.837+0000 7f062eddf700 1 -- 172.21.15.105:0/1557949479 wait complete. 2024-03-26T17:37:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1033172759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T17:37:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/833360695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T17:37:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1010079354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T17:37:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1033172759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T17:37:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/833360695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T17:37:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1010079354' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T17:37:23.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:23 smithi105 ceph-mon[32280]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T17:37:23.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1557949479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T17:37:23.719 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:23.720 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:23.720 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:23.720 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:23.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.264+0000 7fbbf34ef700 1 -- 172.21.15.105:0/3923838036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1072f0 msgr2=0x7fbbec1076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:23.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.264+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/3923838036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1072f0 0x7fbbec1076d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009990 tx=0x7fbbe8009ca0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.264+0000 7fbbf34ef700 1 -- 172.21.15.105:0/3923838036 shutdown_connections 2024-03-26T17:37:23.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.264+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/3923838036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbec101c50 0x7fbbec10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.264+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/3923838036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1072f0 0x7fbbec1076d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.264+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/3923838036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbec1012b0 0x7fbbec101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.264+0000 7fbbf34ef700 1 -- 172.21.15.105:0/3923838036 >> 172.21.15.105:0/3923838036 conn(0x7fbbec0781e0 msgr2=0x7fbbec0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 -- 172.21.15.105:0/3923838036 shutdown_connections 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 -- 172.21.15.105:0/3923838036 wait complete. 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 Processor -- start 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 -- start start 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1012b0 0x7fbbec19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbec101c50 0x7fbbec19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbec1072f0 0x7fbbec198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec111430 con 0x7fbbec1012b0 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec1112b0 con 0x7fbbec101c50 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.265+0000 7fbbf34ef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbec1115b0 con 0x7fbbec1072f0 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1012b0 0x7fbbec19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1012b0 0x7fbbec19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36092/0 (socket says 172.21.15.105:36092) 2024-03-26T17:37:23.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 -- 172.21.15.105:0/833360695 learned_addr learned my addr 172.21.15.105:0/833360695 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf1a8c700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbec1072f0 0x7fbbec198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf0a8a700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbec101c50 0x7fbbec19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 -- 172.21.15.105:0/833360695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbec1072f0 msgr2=0x7fbbec198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbec1072f0 0x7fbbec198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 -- 172.21.15.105:0/833360695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbec101c50 msgr2=0x7fbbec19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:23.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbec101c50 0x7fbbec19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.761 INFO:teuthology.orchestra.run.smithi105.stdout:253403070503 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 -- 172.21.15.105:0/833360695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbec198fd0 con 0x7fbbec1012b0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf128b700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1012b0 0x7fbbec19e6b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80047b0 tx=0x7fbbe8009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbe27fc700 1 -- 172.21.15.105:0/833360695 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8027070 con 0x7fbbec1012b0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbe27fc700 1 -- 172.21.15.105:0/833360695 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbe800be80 con 0x7fbbec1012b0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbe27fc700 1 -- 172.21.15.105:0/833360695 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80173f0 con 0x7fbbec1012b0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbec199260 con 0x7fbbec1012b0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.266+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbec102e90 con 0x7fbbec1012b0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.267+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbec1029a0 con 0x7fbbec1012b0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.269+0000 7fbbe27fc700 1 -- 172.21.15.105:0/833360695 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fbbe802a070 con 0x7fbbec1012b0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.270+0000 7fbbe27fc700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbd80a74d0 0x7fbbd80a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.270+0000 7fbbf0a8a700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbd80a74d0 0x7fbbd80a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:23.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.270+0000 7fbbe27fc700 1 -- 172.21.15.105:0/833360695 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbbe80d0990 con 0x7fbbec1012b0 2024-03-26T17:37:23.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.271+0000 7fbbf0a8a700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbd80a74d0 0x7fbbd80a9990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0787e0 tx=0x7fbbdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:23.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.274+0000 7fbbe27fc700 1 -- 172.21.15.105:0/833360695 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbbe8031070 con 0x7fbbec1012b0 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.481+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbbec199da0 con 0x7fbbec1012b0 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.481+0000 7fbbe27fc700 1 -- 172.21.15.105:0/833360695 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbbe8013070 con 0x7fbbec1012b0 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbd80a74d0 msgr2=0x7fbbd80a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbd80a74d0 0x7fbbd80a9990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0787e0 tx=0x7fbbdc009260 comp rx=0 tx=0).stop 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1012b0 msgr2=0x7fbbec19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1012b0 0x7fbbec19e6b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80047b0 tx=0x7fbbe8009550 comp rx=0 tx=0).stop 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 shutdown_connections 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbd80a74d0 0x7fbbd80a9990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbec1072f0 0x7fbbec198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbec1012b0 0x7fbbec19e6b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 --2- 172.21.15.105:0/833360695 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbec101c50 0x7fbbec19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 >> 172.21.15.105:0/833360695 conn(0x7fbbec0781e0 msgr2=0x7fbbec06a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 shutdown_connections 2024-03-26T17:37:23.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:22.483+0000 7fbbf34ef700 1 -- 172.21.15.105:0/833360695 wait complete. 2024-03-26T17:37:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:23 smithi144 ceph-mon[36095]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T17:37:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1557949479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T17:37:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:23 smithi135 ceph-mon[35636]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T17:37:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1557949479' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T17:37:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:24.993 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070500 got 253403070503 for osd.8 2024-03-26T17:37:24.993 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:25.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383706 for osd.11 2024-03-26T17:37:25.538 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:25.763 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:25 smithi105 ceph-mon[32280]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/1872483610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a060 msgr2=0x7fb8f806a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/1872483610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a060 0x7fb8f806a440 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009990 tx=0x7fb8f4009ca0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/1872483610 shutdown_connections 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/1872483610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8f810ee60 0x7fb8f8111250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/1872483610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a060 0x7fb8f806a440 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/1872483610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f806a980 0x7fb8f810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/1872483610 >> 172.21.15.105:0/1872483610 conn(0x7fb8f80781e0 msgr2=0x7fb8f80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/1872483610 shutdown_connections 2024-03-26T17:37:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.042+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/1872483610 wait complete. 2024-03-26T17:37:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8ffdfd700 1 Processor -- start 2024-03-26T17:37:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8ffdfd700 1 -- start start 2024-03-26T17:37:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8ffdfd700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8f806a060 0x7fb8f81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8ffdfd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a980 0x7fb8f81a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8ffdfd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f810ee60 0x7fb8f819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8ffdfd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f8114780 con 0x7fb8f806a980 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8ffdfd700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f8114600 con 0x7fb8f806a060 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8ffdfd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f8114900 con 0x7fb8f810ee60 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fd398700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a980 0x7fb8f81a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fdb99700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8f806a060 0x7fb8f81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fd398700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a980 0x7fb8f81a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54438/0 (socket says 172.21.15.105:54438) 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fd398700 1 -- 172.21.15.105:0/4139829426 learned_addr learned my addr 172.21.15.105:0/4139829426 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fe39a700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f810ee60 0x7fb8f819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fdb99700 1 -- 172.21.15.105:0/4139829426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f810ee60 msgr2=0x7fb8f819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fdb99700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f810ee60 0x7fb8f819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fdb99700 1 -- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a980 msgr2=0x7fb8f81a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fdb99700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a980 0x7fb8f81a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fdb99700 1 -- 172.21.15.105:0/4139829426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f819b200 con 0x7fb8f806a060 2024-03-26T17:37:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.043+0000 7fb8fd398700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a980 0x7fb8f81a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.044+0000 7fb8fdb99700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8f806a060 0x7fb8f81a08e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40059d0 tx=0x7fb8f4004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.044+0000 7fb8eeffd700 1 -- 172.21.15.105:0/4139829426 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4027070 con 0x7fb8f806a060 2024-03-26T17:37:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.044+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f819b490 con 0x7fb8f806a060 2024-03-26T17:37:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.044+0000 7fb8eeffd700 1 -- 172.21.15.105:0/4139829426 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f4022470 con 0x7fb8f806a060 2024-03-26T17:37:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.044+0000 7fb8eeffd700 1 -- 172.21.15.105:0/4139829426 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400b610 con 0x7fb8f806a060 2024-03-26T17:37:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.044+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f81a77b0 con 0x7fb8f806a060 2024-03-26T17:37:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.045+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f8069070 con 0x7fb8f806a060 2024-03-26T17:37:26.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.046+0000 7fb8eeffd700 1 -- 172.21.15.105:0/4139829426 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fb8f402a070 con 0x7fb8f806a060 2024-03-26T17:37:26.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.046+0000 7fb8eeffd700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8e40a3040 0x7fb8e40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.046+0000 7fb8fd398700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8e40a3040 0x7fb8e40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.046+0000 7fb8eeffd700 1 -- 172.21.15.105:0/4139829426 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb8f40d0420 con 0x7fb8f806a060 2024-03-26T17:37:26.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.047+0000 7fb8fd398700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8e40a3040 0x7fb8e40a5500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8100f10 tx=0x7fb8e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:26.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.050+0000 7fb8eeffd700 1 -- 172.21.15.105:0/4139829426 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb8f4013070 con 0x7fb8f806a060 2024-03-26T17:37:26.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:25 smithi144 ceph-mon[36095]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:25 smithi135 ceph-mon[35636]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:26.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.289+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb8f802dc00 con 0x7fb8f806a060 2024-03-26T17:37:26.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.290+0000 7fb8eeffd700 1 -- 172.21.15.105:0/4139829426 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb8f409d230 con 0x7fb8f806a060 2024-03-26T17:37:26.284 INFO:teuthology.orchestra.run.smithi105.stdout:309237645341 2024-03-26T17:37:26.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8e40a3040 msgr2=0x7fb8e40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8e40a3040 0x7fb8e40a5500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8100f10 tx=0x7fb8e8009260 comp rx=0 tx=0).stop 2024-03-26T17:37:26.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8f806a060 msgr2=0x7fb8f81a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8f806a060 0x7fb8f81a08e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40059d0 tx=0x7fb8f4004790 comp rx=0 tx=0).stop 2024-03-26T17:37:26.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 shutdown_connections 2024-03-26T17:37:26.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8e40a3040 0x7fb8e40a5500 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8f810ee60 0x7fb8f819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8f806a060 0x7fb8f81a08e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 --2- 172.21.15.105:0/4139829426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8f806a980 0x7fb8f81a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 >> 172.21.15.105:0/4139829426 conn(0x7fb8f80781e0 msgr2=0x7fb8f8110580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:26.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 shutdown_connections 2024-03-26T17:37:26.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.292+0000 7fb8ffdfd700 1 -- 172.21.15.105:0/4139829426 wait complete. 2024-03-26T17:37:26.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724594 got 167503724599 for osd.5 2024-03-26T17:37:26.423 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 -- 172.21.15.105:0/2288073682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 msgr2=0x7f32c0101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 --2- 172.21.15.105:0/2288073682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 0x7f32c0101710 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009990 tx=0x7f32b4009ca0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 -- 172.21.15.105:0/2288073682 shutdown_connections 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 --2- 172.21.15.105:0/2288073682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c0101c50 0x7f32c010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 --2- 172.21.15.105:0/2288073682 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32c01072f0 0x7f32c01076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 --2- 172.21.15.105:0/2288073682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 0x7f32c0101710 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 -- 172.21.15.105:0/2288073682 >> 172.21.15.105:0/2288073682 conn(0x7f32c00781e0 msgr2=0x7f32c00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 -- 172.21.15.105:0/2288073682 shutdown_connections 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.778+0000 7f32c6863700 1 -- 172.21.15.105:0/2288073682 wait complete. 2024-03-26T17:37:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32c6863700 1 Processor -- start 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32c6863700 1 -- start start 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32c6863700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 0x7f32c019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32c6863700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c0101c50 0x7f32c019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32c6863700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32c01072f0 0x7f32c01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32c6863700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111420 con 0x7f32c01012b0 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32c6863700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01112a0 con 0x7f32c01072f0 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32c6863700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01115a0 con 0x7f32c0101c50 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.779+0000 7f32bffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 0x7f32c019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bffff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 0x7f32c019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54466/0 (socket says 172.21.15.105:54466) 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bffff700 1 -- 172.21.15.105:0/2353403414 learned_addr learned my addr 172.21.15.105:0/2353403414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bf7fe700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c0101c50 0x7f32c019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32c4e00700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32c01072f0 0x7f32c01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bffff700 1 -- 172.21.15.105:0/2353403414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c0101c50 msgr2=0x7f32c019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bffff700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c0101c50 0x7f32c019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bffff700 1 -- 172.21.15.105:0/2353403414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32c01072f0 msgr2=0x7f32c01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bffff700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32c01072f0 0x7f32c01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bffff700 1 -- 172.21.15.105:0/2353403414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c0199030 con 0x7f32c01012b0 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bf7fe700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c0101c50 0x7f32c019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bffff700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 0x7f32c019e6c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f32b000eb30 tx=0x7f32b000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bd7fa700 1 -- 172.21.15.105:0/2353403414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b000cbd0 con 0x7f32c01012b0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c0199320 con 0x7f32c01012b0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bd7fa700 1 -- 172.21.15.105:0/2353403414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32b00044d0 con 0x7f32c01012b0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32bd7fa700 1 -- 172.21.15.105:0/2353403414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0003e30 con 0x7f32c01012b0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.780+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a5590 con 0x7f32c01012b0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.781+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c0069070 con 0x7f32c01012b0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.782+0000 7f32bd7fa700 1 -- 172.21.15.105:0/2353403414 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f32b0004000 con 0x7f32c01012b0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.783+0000 7f32bd7fa700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32a80a3040 0x7f32a80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.783+0000 7f32bf7fe700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32a80a3040 0x7f32a80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.783+0000 7f32bd7fa700 1 -- 172.21.15.105:0/2353403414 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8206+0+0 (secure 0 0 0) 0x7f32b00cfac0 con 0x7f32c01012b0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.783+0000 7f32bf7fe700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32a80a3040 0x7f32a80a5500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32b400b2e0 tx=0x7f32b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.787+0000 7f32bd7fa700 1 -- 172.21.15.105:0/2353403414 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f32b0053740 con 0x7f32c01012b0 2024-03-26T17:37:26.873 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:26 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4139829426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T17:37:26.873 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-26T17:37:26.873 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-26T17:37:26.873 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]: dispatch 2024-03-26T17:37:26.874 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9, 4, 6]}]: dispatch 2024-03-26T17:37:26.874 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-26T17:37:26.874 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]: dispatch 2024-03-26T17:37:26.874 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:26 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-03-26T17:37:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 -- 172.21.15.105:0/1891362904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 msgr2=0x7f65001076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 --2- 172.21.15.105:0/1891362904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 0x7f65001076d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009990 tx=0x7f64f0009ca0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 -- 172.21.15.105:0/1891362904 shutdown_connections 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 --2- 172.21.15.105:0/1891362904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6500101c50 0x7f650010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 --2- 172.21.15.105:0/1891362904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 0x7f65001076d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 --2- 172.21.15.105:0/1891362904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65001012b0 0x7f6500101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 -- 172.21.15.105:0/1891362904 >> 172.21.15.105:0/1891362904 conn(0x7f65000781e0 msgr2=0x7f65000785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 -- 172.21.15.105:0/1891362904 shutdown_connections 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.982+0000 7f6507b25700 1 -- 172.21.15.105:0/1891362904 wait complete. 2024-03-26T17:37:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f6507b25700 1 Processor -- start 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f6507b25700 1 -- start start 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f6507b25700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65001012b0 0x7f650019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f6507b25700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6500101c50 0x7f650019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f6507b25700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 0x7f65001988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f6507b25700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6500111400 con 0x7f65001072f0 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f6507b25700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6500111280 con 0x7f65001012b0 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f6507b25700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6500111580 con 0x7f6500101c50 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f65060c2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 0x7f65001988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f65058c1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65001012b0 0x7f650019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f65050c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6500101c50 0x7f650019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f65058c1700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65001012b0 0x7f650019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36648/0 (socket says 172.21.15.105:36648) 2024-03-26T17:37:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f65060c2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 0x7f65001988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54492/0 (socket says 172.21.15.105:54492) 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.983+0000 7f65058c1700 1 -- 172.21.15.105:0/1999354019 learned_addr learned my addr 172.21.15.105:0/1999354019 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f65060c2700 1 -- 172.21.15.105:0/1999354019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6500101c50 msgr2=0x7f650019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f65060c2700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6500101c50 0x7f650019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f65060c2700 1 -- 172.21.15.105:0/1999354019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65001012b0 msgr2=0x7f650019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f65060c2700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65001012b0 0x7f650019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f65060c2700 1 -- 172.21.15.105:0/1999354019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6500199030 con 0x7f65001072f0 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f65058c1700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65001012b0 0x7f650019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f65050c0700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6500101c50 0x7f650019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f65060c2700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 0x7f65001988d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f64f800efc0 tx=0x7f64f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f64f6ffd700 1 -- 172.21.15.105:0/1999354019 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f80092e0 con 0x7f65001072f0 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f64f6ffd700 1 -- 172.21.15.105:0/1999354019 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64f8004bb0 con 0x7f65001072f0 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f64f6ffd700 1 -- 172.21.15.105:0/1999354019 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8004020 con 0x7f65001072f0 2024-03-26T17:37:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65001992c0 con 0x7f65001072f0 2024-03-26T17:37:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.984+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65001a5590 con 0x7f65001072f0 2024-03-26T17:37:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.985+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6500069070 con 0x7f65001072f0 2024-03-26T17:37:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.986+0000 7f64f6ffd700 1 -- 172.21.15.105:0/1999354019 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f64f801b030 con 0x7f65001072f0 2024-03-26T17:37:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.987+0000 7f64f6ffd700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64ec0a30c0 0x7f64ec0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.987+0000 7f65058c1700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64ec0a30c0 0x7f64ec0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.987+0000 7f64f6ffd700 1 -- 172.21.15.105:0/1999354019 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8206+0+0 (secure 0 0 0) 0x7f64f80cf930 con 0x7f65001072f0 2024-03-26T17:37:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.988+0000 7f65058c1700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64ec0a30c0 0x7f64ec0a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005f50 tx=0x7f64f001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.991+0000 7f64f6ffd700 1 -- 172.21.15.105:0/1999354019 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f64f8022020 con 0x7f65001072f0 2024-03-26T17:37:26.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.997+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f32c0199db0 con 0x7f32c01012b0 2024-03-26T17:37:26.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.997+0000 7f32bd7fa700 1 -- 172.21.15.105:0/2353403414 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f32b00bae20 con 0x7f32c01012b0 2024-03-26T17:37:26.991 INFO:teuthology.orchestra.run.smithi105.stdout:197568495667 2024-03-26T17:37:26.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32a80a3040 msgr2=0x7f32a80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32a80a3040 0x7f32a80a5500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32b400b2e0 tx=0x7f32b401a040 comp rx=0 tx=0).stop 2024-03-26T17:37:26.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 msgr2=0x7f32c019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:26.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 0x7f32c019e6c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f32b000eb30 tx=0x7f32b000ee40 comp rx=0 tx=0).stop 2024-03-26T17:37:26.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 shutdown_connections 2024-03-26T17:37:26.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32a80a3040 0x7f32a80a5500 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32c01072f0 0x7f32c01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32c01012b0 0x7f32c019e6c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 --2- 172.21.15.105:0/2353403414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32c0101c50 0x7f32c019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:26.999+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 >> 172.21.15.105:0/2353403414 conn(0x7f32c00781e0 msgr2=0x7f32c006a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.000+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 shutdown_connections 2024-03-26T17:37:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.000+0000 7f32c6863700 1 -- 172.21.15.105:0/2353403414 wait complete. 2024-03-26T17:37:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:26 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4139829426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T17:37:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-26T17:37:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-26T17:37:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]: dispatch 2024-03-26T17:37:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9, 4, 6]}]: dispatch 2024-03-26T17:37:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-26T17:37:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]: dispatch 2024-03-26T17:37:27.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:26 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-03-26T17:37:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:26 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4139829426' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T17:37:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-26T17:37:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-26T17:37:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]: dispatch 2024-03-26T17:37:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9, 4, 6]}]: dispatch 2024-03-26T17:37:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-03-26T17:37:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]: dispatch 2024-03-26T17:37:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:26 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]: dispatch 2024-03-26T17:37:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.208+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f65001a5820 con 0x7f65001072f0 2024-03-26T17:37:27.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.208+0000 7f64f6ffd700 1 -- 172.21.15.105:0/1999354019 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f64f8010040 con 0x7f65001072f0 2024-03-26T17:37:27.202 INFO:teuthology.orchestra.run.smithi105.stdout:34359738448 2024-03-26T17:37:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64ec0a30c0 msgr2=0x7f64ec0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64ec0a30c0 0x7f64ec0a5580 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f64f0005f50 tx=0x7f64f001a040 comp rx=0 tx=0).stop 2024-03-26T17:37:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 msgr2=0x7f65001988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 0x7f65001988d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f64f800efc0 tx=0x7f64f800c3a0 comp rx=0 tx=0).stop 2024-03-26T17:37:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 shutdown_connections 2024-03-26T17:37:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64ec0a30c0 0x7f64ec0a5580 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65001072f0 0x7f65001988d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65001012b0 0x7f650019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 --2- 172.21.15.105:0/1999354019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6500101c50 0x7f650019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 >> 172.21.15.105:0/1999354019 conn(0x7f65000781e0 msgr2=0x7f650006a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 shutdown_connections 2024-03-26T17:37:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:27.210+0000 7f6507b25700 1 -- 172.21.15.105:0/1999354019 wait complete. 2024-03-26T17:37:28.098 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:28.098 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-26T17:37:28.098 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-26T17:37:28.098 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-03-26T17:37:28.099 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9, 4, 6]}]': finished 2024-03-26T17:37:28.099 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-26T17:37:28.099 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]': finished 2024-03-26T17:37:28.099 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-03-26T17:37:28.099 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T17:37:28.099 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2353403414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T17:37:28.099 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:27 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1999354019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T17:37:28.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9, 4, 6]}]': finished 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]': finished 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2353403414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T17:37:28.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:27 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1999354019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T17:37:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-26T17:37:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-26T17:37:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-03-26T17:37:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9, 4, 6]}]': finished 2024-03-26T17:37:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-03-26T17:37:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 7]}]': finished 2024-03-26T17:37:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-03-26T17:37:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T17:37:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2353403414' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T17:37:28.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:27 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1999354019' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T17:37:28.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920824 got 141733920829 for osd.4 2024-03-26T17:37:28.725 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:29.096 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:28 smithi105 ceph-mon[32280]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T17:37:29.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:28 smithi144 ceph-mon[36095]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T17:37:29.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:28 smithi135 ceph-mon[35636]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T17:37:29.711 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645336 got 309237645341 for osd.10 2024-03-26T17:37:29.711 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:30.021 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:29 smithi105 ceph-mon[32280]: pgmap v318: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 2/306 objects degraded (0.654%) 2024-03-26T17:37:30.021 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:29 smithi105 ceph-mon[32280]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-26T17:37:30.021 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:29 smithi105 ceph-mon[32280]: Health check failed: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:37:30.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:29 smithi144 ceph-mon[36095]: pgmap v318: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 2/306 objects degraded (0.654%) 2024-03-26T17:37:30.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:29 smithi144 ceph-mon[36095]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-26T17:37:30.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:29 smithi144 ceph-mon[36095]: Health check failed: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:37:30.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 -- 172.21.15.105:0/3620044095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050101540 msgr2=0x7fe050101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:30.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 --2- 172.21.15.105:0/3620044095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050101540 0x7fe050101920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0400099c0 tx=0x7fe040009cd0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 -- 172.21.15.105:0/3620044095 shutdown_connections 2024-03-26T17:37:30.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 --2- 172.21.15.105:0/3620044095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050106590 0x7fe05010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 --2- 172.21.15.105:0/3620044095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050101540 0x7fe050101920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 --2- 172.21.15.105:0/3620044095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe050101ef0 0x7fe050105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 -- 172.21.15.105:0/3620044095 >> 172.21.15.105:0/3620044095 conn(0x7fe050077300 msgr2=0x7fe050077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 -- 172.21.15.105:0/3620044095 shutdown_connections 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.131+0000 7fe05627a700 1 -- 172.21.15.105:0/3620044095 wait complete. 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe05627a700 1 Processor -- start 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe05627a700 1 -- start start 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe05627a700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe050101540 0x7fe05019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe05627a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050101ef0 0x7fe05019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe05627a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050106590 0x7fe050198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe05627a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe0501114a0 con 0x7fe050106590 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe05627a700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe050111320 con 0x7fe050101540 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe05627a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe050111620 con 0x7fe050101ef0 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe054817700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050106590 0x7fe050198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe054817700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050106590 0x7fe050198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54514/0 (socket says 172.21.15.105:54514) 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.132+0000 7fe054817700 1 -- 172.21.15.105:0/2639408890 learned_addr learned my addr 172.21.15.105:0/2639408890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:30.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe04f7fe700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050101ef0 0x7fe05019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe04ffff700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe050101540 0x7fe05019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe054817700 1 -- 172.21.15.105:0/2639408890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050101ef0 msgr2=0x7fe05019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe054817700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050101ef0 0x7fe05019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe054817700 1 -- 172.21.15.105:0/2639408890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe050101540 msgr2=0x7fe05019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe054817700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe050101540 0x7fe05019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe054817700 1 -- 172.21.15.105:0/2639408890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe050199080 con 0x7fe050106590 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe054817700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050106590 0x7fe050198920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe04800c400 tx=0x7fe04800c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe04d7fa700 1 -- 172.21.15.105:0/2639408890 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe048004bb0 con 0x7fe050106590 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe04d7fa700 1 -- 172.21.15.105:0/2639408890 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe048004d50 con 0x7fe050106590 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe050199310 con 0x7fe050106590 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe04d7fa700 1 -- 172.21.15.105:0/2639408890 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0480043e0 con 0x7fe050106590 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.133+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe050102e80 con 0x7fe050106590 2024-03-26T17:37:30.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.134+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe050069070 con 0x7fe050106590 2024-03-26T17:37:30.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.136+0000 7fe04d7fa700 1 -- 172.21.15.105:0/2639408890 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fe048021020 con 0x7fe050106590 2024-03-26T17:37:30.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.136+0000 7fe04d7fa700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0380a30c0 0x7fe0380a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:30.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.137+0000 7fe04ffff700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0380a30c0 0x7fe0380a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:30.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.137+0000 7fe04d7fa700 1 -- 172.21.15.105:0/2639408890 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fe0480d2a70 con 0x7fe050106590 2024-03-26T17:37:30.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.137+0000 7fe04ffff700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0380a30c0 0x7fe0380a5580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe040000c00 tx=0x7fe040005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:30.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.143+0000 7fe04d7fa700 1 -- 172.21.15.105:0/2639408890 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe04809f760 con 0x7fe050106590 2024-03-26T17:37:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:29 smithi135 ceph-mon[35636]: pgmap v318: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 2/306 objects degraded (0.654%) 2024-03-26T17:37:30.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:29 smithi135 ceph-mon[35636]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-26T17:37:30.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:29 smithi135 ceph-mon[35636]: Health check failed: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded (PG_DEGRADED) 2024-03-26T17:37:30.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.353+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe050199e00 con 0x7fe050106590 2024-03-26T17:37:30.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.353+0000 7fe04d7fa700 1 -- 172.21.15.105:0/2639408890 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe04809f060 con 0x7fe050106590 2024-03-26T17:37:30.346 INFO:teuthology.orchestra.run.smithi105.stdout:107374182465 2024-03-26T17:37:30.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0380a30c0 msgr2=0x7fe0380a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:30.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0380a30c0 0x7fe0380a5580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe040000c00 tx=0x7fe040005fb0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050106590 msgr2=0x7fe050198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:30.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050106590 0x7fe050198920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe04800c400 tx=0x7fe04800c710 comp rx=0 tx=0).stop 2024-03-26T17:37:30.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 shutdown_connections 2024-03-26T17:37:30.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0380a30c0 0x7fe0380a5580 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe050106590 0x7fe050198920 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe050101540 0x7fe05019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 --2- 172.21.15.105:0/2639408890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050101ef0 0x7fe05019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:30.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 >> 172.21.15.105:0/2639408890 conn(0x7fe050077300 msgr2=0x7fe05006ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:30.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 shutdown_connections 2024-03-26T17:37:30.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:30.356+0000 7fe05627a700 1 -- 172.21.15.105:0/2639408890 wait complete. 2024-03-26T17:37:30.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299432 got 223338299438 for osd.7 2024-03-26T17:37:30.653 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:30.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:30 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2639408890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T17:37:31.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:30 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2639408890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T17:37:31.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:30 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2639408890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T17:37:31.263 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495667 for osd.6 2024-03-26T17:37:31.263 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:31 smithi144 ceph-mon[36095]: pgmap v319: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 2/306 objects degraded (0.654%); 27 B/s, 1 objects/s recovering 2024-03-26T17:37:32.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:31 smithi105 ceph-mon[32280]: pgmap v319: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 2/306 objects degraded (0.654%); 27 B/s, 1 objects/s recovering 2024-03-26T17:37:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:31 smithi135 ceph-mon[35636]: pgmap v319: 97 pgs: 1 activating+degraded, 2 peering, 94 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 2/306 objects degraded (0.654%); 27 B/s, 1 objects/s recovering 2024-03-26T17:37:32.645 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738448 for osd.0 2024-03-26T17:37:32.646 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:33.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 -- 172.21.15.105:0/3309307074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01072f0 msgr2=0x7f67f01076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 --2- 172.21.15.105:0/3309307074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01072f0 0x7f67f01076d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b0a0 tx=0x7f67e800b3b0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 -- 172.21.15.105:0/3309307074 shutdown_connections 2024-03-26T17:37:33.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 --2- 172.21.15.105:0/3309307074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67f0101c50 0x7f67f010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 --2- 172.21.15.105:0/3309307074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f01072f0 0x7f67f01076d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 --2- 172.21.15.105:0/3309307074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01012b0 0x7f67f0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 -- 172.21.15.105:0/3309307074 >> 172.21.15.105:0/3309307074 conn(0x7f67f00781e0 msgr2=0x7f67f00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:33.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 -- 172.21.15.105:0/3309307074 shutdown_connections 2024-03-26T17:37:33.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.302+0000 7f67f55fc700 1 -- 172.21.15.105:0/3309307074 wait complete. 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.303+0000 7f67f55fc700 1 Processor -- start 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.303+0000 7f67f55fc700 1 -- start start 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67f55fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01012b0 0x7f67f019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67f55fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f0101c50 0x7f67f019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67f55fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67f01072f0 0x7f67f01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67f55fc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0111400 con 0x7f67f0101c50 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67f55fc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0111280 con 0x7f67f01072f0 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67f55fc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67f0111580 con 0x7f67f01012b0 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67ee7fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f0101c50 0x7f67f019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01012b0 0x7f67f019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67ee7fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f0101c50 0x7f67f019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54530/0 (socket says 172.21.15.105:54530) 2024-03-26T17:37:33.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01012b0 0x7f67f019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41644/0 (socket says 172.21.15.105:41644) 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67ee7fc700 1 -- 172.21.15.105:0/769745622 learned_addr learned my addr 172.21.15.105:0/769745622 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67ef7fe700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67f01072f0 0x7f67f01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67eeffd700 1 -- 172.21.15.105:0/769745622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67f01072f0 msgr2=0x7f67f01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67eeffd700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67f01072f0 0x7f67f01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67eeffd700 1 -- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f0101c50 msgr2=0x7f67f019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67eeffd700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f0101c50 0x7f67f019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67eeffd700 1 -- 172.21.15.105:0/769745622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f0199010 con 0x7f67f01012b0 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67ee7fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f0101c50 0x7f67f019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.304+0000 7f67eeffd700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01012b0 0x7f67f019e6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b070 tx=0x7f67e8004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.305+0000 7f67e7fff700 1 -- 172.21.15.105:0/769745622 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8021070 con 0x7f67f01012b0 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.305+0000 7f67e7fff700 1 -- 172.21.15.105:0/769745622 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67e8007e30 con 0x7f67f01012b0 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.305+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f01992a0 con 0x7f67f01012b0 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.305+0000 7f67e7fff700 1 -- 172.21.15.105:0/769745622 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e80045e0 con 0x7f67f01012b0 2024-03-26T17:37:33.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.305+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01a55c0 con 0x7f67f01012b0 2024-03-26T17:37:33.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.305+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f0069070 con 0x7f67f01012b0 2024-03-26T17:37:33.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.307+0000 7f67e7fff700 1 -- 172.21.15.105:0/769745622 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f67e8003d20 con 0x7f67f01012b0 2024-03-26T17:37:33.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.307+0000 7f67e7fff700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f67dc0a3040 0x7f67dc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:33.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.307+0000 7f67ee7fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f67dc0a3040 0x7f67dc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:33.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.307+0000 7f67e7fff700 1 -- 172.21.15.105:0/769745622 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f67e800f020 con 0x7f67f01012b0 2024-03-26T17:37:33.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.308+0000 7f67ee7fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f67dc0a3040 0x7f67dc0a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f67f006b0b0 tx=0x7f67d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:33.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.311+0000 7f67e7fff700 1 -- 172.21.15.105:0/769745622 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f67e802a020 con 0x7f67f01012b0 2024-03-26T17:37:33.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 -- 172.21.15.105:0/4134178283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dd8101c50 msgr2=0x7f9dd810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 --2- 172.21.15.105:0/4134178283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dd8101c50 0x7f9dd810e130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009f90 tx=0x7f9dd000b330 comp rx=0 tx=0).stop 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 -- 172.21.15.105:0/4134178283 shutdown_connections 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 --2- 172.21.15.105:0/4134178283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dd8101c50 0x7f9dd810e130 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 --2- 172.21.15.105:0/4134178283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd81072f0 0x7f9dd81076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 --2- 172.21.15.105:0/4134178283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd81012b0 0x7f9dd8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 -- 172.21.15.105:0/4134178283 >> 172.21.15.105:0/4134178283 conn(0x7f9dd80781e0 msgr2=0x7f9dd80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 -- 172.21.15.105:0/4134178283 shutdown_connections 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 -- 172.21.15.105:0/4134178283 wait complete. 2024-03-26T17:37:33.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 Processor -- start 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.538+0000 7f9ddda39700 1 -- start start 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9ddda39700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd81012b0 0x7f9dd819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9ddda39700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd8101c50 0x7f9dd819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9ddda39700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dd81072f0 0x7f9dd81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9ddda39700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd8111420 con 0x7f9dd8101c50 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9ddda39700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd81112a0 con 0x7f9dd81072f0 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9ddda39700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd81115a0 con 0x7f9dd81012b0 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd81012b0 0x7f9dd819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd81012b0 0x7f9dd819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41658/0 (socket says 172.21.15.105:41658) 2024-03-26T17:37:33.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 -- 172.21.15.105:0/485901931 learned_addr learned my addr 172.21.15.105:0/485901931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd77fe700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dd81072f0 0x7f9dd81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd67fc700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd8101c50 0x7f9dd819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 -- 172.21.15.105:0/485901931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dd81072f0 msgr2=0x7f9dd81988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dd81072f0 0x7f9dd81988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 -- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd8101c50 msgr2=0x7f9dd819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd8101c50 0x7f9dd819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 -- 172.21.15.105:0/485901931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd8199010 con 0x7f9dd81012b0 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.539+0000 7f9dd6ffd700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd81012b0 0x7f9dd819e6f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00e9c0 tx=0x7f9dcc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.540+0000 7f9ddca37700 1 -- 172.21.15.105:0/485901931 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00cbd0 con 0x7f9dd81012b0 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.540+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd8199300 con 0x7f9dd81012b0 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.540+0000 7f9ddca37700 1 -- 172.21.15.105:0/485901931 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dcc00ee60 con 0x7f9dd81012b0 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.540+0000 7f9ddca37700 1 -- 172.21.15.105:0/485901931 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0055b0 con 0x7f9dd81012b0 2024-03-26T17:37:33.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.540+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd81a55c0 con 0x7f9dd81012b0 2024-03-26T17:37:33.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.541+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd8069070 con 0x7f9dd81012b0 2024-03-26T17:37:33.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.542+0000 7f9ddca37700 1 -- 172.21.15.105:0/485901931 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f9dcc004750 con 0x7f9dd81012b0 2024-03-26T17:37:33.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.543+0000 7f9ddca37700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dc00a30c0 0x7f9dc00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:33.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.543+0000 7f9dd67fc700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dc00a30c0 0x7f9dc00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:33.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.543+0000 7f9ddca37700 1 -- 172.21.15.105:0/485901931 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f9dcc0cf730 con 0x7f9dd81012b0 2024-03-26T17:37:33.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.543+0000 7f9dd67fc700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dc00a30c0 0x7f9dc00a5580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009750 tx=0x7f9dc8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.551+0000 7f9ddca37700 1 -- 172.21.15.105:0/485901931 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9dcc09c3c0 con 0x7f9dd81012b0 2024-03-26T17:37:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.552+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f67f01a5790 con 0x7f67f01012b0 2024-03-26T17:37:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.553+0000 7f67e7fff700 1 -- 172.21.15.105:0/769745622 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f67e8011e50 con 0x7f67f01012b0 2024-03-26T17:37:33.545 INFO:teuthology.orchestra.run.smithi105.stdout:287762808870 2024-03-26T17:37:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f67dc0a3040 msgr2=0x7f67dc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f67dc0a3040 0x7f67dc0a5500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f67f006b0b0 tx=0x7f67d8009260 comp rx=0 tx=0).stop 2024-03-26T17:37:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01012b0 msgr2=0x7f67f019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01012b0 0x7f67f019e6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67e800b070 tx=0x7f67e8004d00 comp rx=0 tx=0).stop 2024-03-26T17:37:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 shutdown_connections 2024-03-26T17:37:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f67dc0a3040 0x7f67dc0a5500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67f01072f0 0x7f67f01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67f01012b0 0x7f67f019e6f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 --2- 172.21.15.105:0/769745622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f0101c50 0x7f67f019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.555+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 >> 172.21.15.105:0/769745622 conn(0x7f67f00781e0 msgr2=0x7f67f0105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.556+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 shutdown_connections 2024-03-26T17:37:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.556+0000 7f67f55fc700 1 -- 172.21.15.105:0/769745622 wait complete. 2024-03-26T17:37:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.774+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9dd81a5850 con 0x7f9dd81012b0 2024-03-26T17:37:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.775+0000 7f9ddca37700 1 -- 172.21.15.105:0/485901931 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9dcc09bcc0 con 0x7f9dd81012b0 2024-03-26T17:37:33.768 INFO:teuthology.orchestra.run.smithi105.stdout:77309411401 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dc00a30c0 msgr2=0x7f9dc00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dc00a30c0 0x7f9dc00a5580 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009750 tx=0x7f9dc8009310 comp rx=0 tx=0).stop 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd81012b0 msgr2=0x7f9dd819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd81012b0 0x7f9dd819e6f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00e9c0 tx=0x7f9dcc00ecd0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 shutdown_connections 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dc00a30c0 0x7f9dc00a5580 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dd81072f0 0x7f9dd81988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9dd81012b0 0x7f9dd819e6f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 --2- 172.21.15.105:0/485901931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd8101c50 0x7f9dd819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 >> 172.21.15.105:0/485901931 conn(0x7f9dd80781e0 msgr2=0x7f9dd8100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:33.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 shutdown_connections 2024-03-26T17:37:33.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:33.777+0000 7f9ddda39700 1 -- 172.21.15.105:0/485901931 wait complete. 2024-03-26T17:37:33.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182465 for osd.3 2024-03-26T17:37:33.999 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:34.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:33 smithi144 ceph-mon[36095]: pgmap v320: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 2/306 objects degraded (0.654%); 27 B/s, 1 objects/s recovering 2024-03-26T17:37:34.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:33 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/769745622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T17:37:34.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:33 smithi105 ceph-mon[32280]: pgmap v320: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 2/306 objects degraded (0.654%); 27 B/s, 1 objects/s recovering 2024-03-26T17:37:34.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:33 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/769745622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T17:37:34.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:33 smithi135 ceph-mon[35636]: pgmap v320: 97 pgs: 1 activating+degraded, 1 peering, 95 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 2/306 objects degraded (0.654%); 27 B/s, 1 objects/s recovering 2024-03-26T17:37:34.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:33 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/769745622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T17:37:34.744 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411401 for osd.2 2024-03-26T17:37:34.744 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:35.067 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:34 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/485901931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T17:37:35.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:34 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/485901931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T17:37:35.115 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:34 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/485901931' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T17:37:35.142 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808863 got 287762808870 for osd.9 2024-03-26T17:37:35.142 DEBUG:teuthology.parallel:result is None 2024-03-26T17:37:35.142 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-26T17:37:35.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T17:37:35.908 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:36.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:35 smithi105 ceph-mon[32280]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 27 B/s, 1 objects/s recovering 2024-03-26T17:37:36.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:35 smithi105 ceph-mon[32280]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T17:37:36.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:35 smithi105 ceph-mon[32280]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded) 2024-03-26T17:37:36.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:35 smithi105 ceph-mon[32280]: Cluster is now healthy 2024-03-26T17:37:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:35 smithi144 ceph-mon[36095]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 27 B/s, 1 objects/s recovering 2024-03-26T17:37:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:35 smithi144 ceph-mon[36095]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T17:37:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:35 smithi144 ceph-mon[36095]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded) 2024-03-26T17:37:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:35 smithi144 ceph-mon[36095]: Cluster is now healthy 2024-03-26T17:37:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:35 smithi135 ceph-mon[35636]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 27 B/s, 1 objects/s recovering 2024-03-26T17:37:36.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:35 smithi135 ceph-mon[35636]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T17:37:36.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:35 smithi135 ceph-mon[35636]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/306 objects degraded (0.654%), 1 pg degraded) 2024-03-26T17:37:36.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:35 smithi135 ceph-mon[35636]: Cluster is now healthy 2024-03-26T17:37:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 -- 172.21.15.105:0/1247112754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101540 msgr2=0x7f09b4101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 --2- 172.21.15.105:0/1247112754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101540 0x7f09b4101920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f09a40099c0 tx=0x7f09a4009cd0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 -- 172.21.15.105:0/1247112754 shutdown_connections 2024-03-26T17:37:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 --2- 172.21.15.105:0/1247112754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09b4106590 0x7f09b410e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 --2- 172.21.15.105:0/1247112754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101540 0x7f09b4101920 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 --2- 172.21.15.105:0/1247112754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09b4101ef0 0x7f09b4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 -- 172.21.15.105:0/1247112754 >> 172.21.15.105:0/1247112754 conn(0x7f09b4077300 msgr2=0x7f09b4077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 -- 172.21.15.105:0/1247112754 shutdown_connections 2024-03-26T17:37:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.541+0000 7f09b9be2700 1 -- 172.21.15.105:0/1247112754 wait complete. 2024-03-26T17:37:37.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b9be2700 1 Processor -- start 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b9be2700 1 -- start start 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b9be2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09b4101540 0x7f09b41998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b9be2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101ef0 0x7f09b4199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b9be2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09b4106590 0x7f09b41a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b9be2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09b4111420 con 0x7f09b4101ef0 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b9be2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41112a0 con 0x7f09b4106590 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b9be2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41115a0 con 0x7f09b4101540 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b2ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101ef0 0x7f09b4199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09b4101540 0x7f09b41998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b2ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101ef0 0x7f09b4199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38660/0 (socket says 172.21.15.105:38660) 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.542+0000 7f09b2ffd700 1 -- 172.21.15.105:0/70629128 learned_addr learned my addr 172.21.15.105:0/70629128 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b3fff700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09b4106590 0x7f09b41a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b2ffd700 1 -- 172.21.15.105:0/70629128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09b4101540 msgr2=0x7f09b41998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b2ffd700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09b4101540 0x7f09b41998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b2ffd700 1 -- 172.21.15.105:0/70629128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09b4106590 msgr2=0x7f09b41a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b2ffd700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09b4106590 0x7f09b41a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b2ffd700 1 -- 172.21.15.105:0/70629128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b41a4e80 con 0x7f09b4101ef0 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b2ffd700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101ef0 0x7f09b4199e00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f09a800d950 tx=0x7f09a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b0ff9700 1 -- 172.21.15.105:0/70629128 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8016070 con 0x7f09b4101ef0 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b4197710 con 0x7f09b4101ef0 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b0ff9700 1 -- 172.21.15.105:0/70629128 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09a8004830 con 0x7f09b4101ef0 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.543+0000 7f09b0ff9700 1 -- 172.21.15.105:0/70629128 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80054a0 con 0x7f09b4101ef0 2024-03-26T17:37:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.544+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b4197ca0 con 0x7f09b4101ef0 2024-03-26T17:37:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.545+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b4069070 con 0x7f09b4101ef0 2024-03-26T17:37:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.545+0000 7f09b0ff9700 1 -- 172.21.15.105:0/70629128 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f09a8004220 con 0x7f09b4101ef0 2024-03-26T17:37:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.546+0000 7f09b0ff9700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f099c0a30c0 0x7f099c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.546+0000 7f09b37fe700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f099c0a30c0 0x7f099c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.546+0000 7f09b0ff9700 1 -- 172.21.15.105:0/70629128 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f09a80cf250 con 0x7f09b4101ef0 2024-03-26T17:37:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.547+0000 7f09b37fe700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f099c0a30c0 0x7f099c0a5580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09a4000c00 tx=0x7f09a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.551+0000 7f09b0ff9700 1 -- 172.21.15.105:0/70629128 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f09a801f020 con 0x7f09b4101ef0 2024-03-26T17:37:37.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.799+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09b40627f0 con 0x7f099c0a30c0 2024-03-26T17:37:37.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.807+0000 7f09b0ff9700 1 -- 172.21.15.105:0/70629128 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258769 (secure 0 0 0) 0x7f09b40627f0 con 0x7f099c0a30c0 2024-03-26T17:37:37.803 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f099c0a30c0 msgr2=0x7f099c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f099c0a30c0 0x7f099c0a5580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09a4000c00 tx=0x7f09a4005fb0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101ef0 msgr2=0x7f09b4199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101ef0 0x7f09b4199e00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f09a800d950 tx=0x7f09a800dc60 comp rx=0 tx=0).stop 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 shutdown_connections 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f099c0a30c0 0x7f099c0a5580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09b4106590 0x7f09b41a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09b4101540 0x7f09b41998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 --2- 172.21.15.105:0/70629128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09b4101ef0 0x7f09b4199e00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.810+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 >> 172.21.15.105:0/70629128 conn(0x7f09b4077300 msgr2=0x7f09b41051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:37.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.811+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 shutdown_connections 2024-03-26T17:37:37.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:37.811+0000 7f09b9be2700 1 -- 172.21.15.105:0/70629128 wait complete. 2024-03-26T17:37:37.807 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-26T17:37:38.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:37 smithi144 ceph-mon[36095]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 22 B/s, 1 objects/s recovering 2024-03-26T17:37:38.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:38.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:37 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:38.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:37 smithi105 ceph-mon[32280]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 22 B/s, 1 objects/s recovering 2024-03-26T17:37:38.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:38.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:37 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:37 smithi135 ceph-mon[35636]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 22 B/s, 1 objects/s recovering 2024-03-26T17:37:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:38.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:37 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:38.504 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-03-26T17:37:36.598953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658461,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":813,"num_read_kb":522,"num_write":419,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366688,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575136,"statfs":{"total":1151940427776,"available":1151564939264,"internally_reserved":0,"allocated":13627392,"data_stored":7436847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":33,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.864189"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738305+0000","last_change":"2024-03-26T17:36:22.743815+0000","last_active":"2024-03-26T17:37:27.738305+0000","last_peered":"2024-03-26T17:37:27.738305+0000","last_clean":"2024-03-26T17:37:27.738305+0000","last_became_active":"2024-03-26T17:36:22.743446+0000","last_became_peered":"2024-03-26T17:36:22.743446+0000","last_unstale":"2024-03-26T17:37:27.738305+0000","last_undegraded":"2024-03-26T17:37:27.738305+0000","last_fullsized":"2024-03-26T17:37:27.738305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:16:56.773849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736811+0000","last_change":"2024-03-26T17:36:22.740754+0000","last_active":"2024-03-26T17:37:27.736811+0000","last_peered":"2024-03-26T17:37:27.736811+0000","last_clean":"2024-03-26T17:37:27.736811+0000","last_became_active":"2024-03-26T17:36:22.740317+0000","last_became_peered":"2024-03-26T17:36:22.740317+0000","last_unstale":"2024-03-26T17:37:27.736811+0000","last_undegraded":"2024-03-26T17:37:27.736811+0000","last_fullsized":"2024-03-26T17:37:27.736811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:50:45.321653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736774+0000","last_change":"2024-03-26T17:36:22.735079+0000","last_active":"2024-03-26T17:37:27.736774+0000","last_peered":"2024-03-26T17:37:27.736774+0000","last_clean":"2024-03-26T17:37:27.736774+0000","last_became_active":"2024-03-26T17:36:22.734646+0000","last_became_peered":"2024-03-26T17:36:22.734646+0000","last_unstale":"2024-03-26T17:37:27.736774+0000","last_undegraded":"2024-03-26T17:37:27.736774+0000","last_fullsized":"2024-03-26T17:37:27.736774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:41:09.477438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.243877+0000","last_change":"2024-03-26T17:36:22.746170+0000","last_active":"2024-03-26T17:37:28.243877+0000","last_peered":"2024-03-26T17:37:28.243877+0000","last_clean":"2024-03-26T17:37:28.243877+0000","last_became_active":"2024-03-26T17:36:22.744255+0000","last_became_peered":"2024-03-26T17:36:22.744255+0000","last_unstale":"2024-03-26T17:37:28.243877+0000","last_undegraded":"2024-03-26T17:37:28.243877+0000","last_fullsized":"2024-03-26T17:37:28.243877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:54:36.225926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739480+0000","last_change":"2024-03-26T17:36:22.737390+0000","last_active":"2024-03-26T17:37:27.739480+0000","last_peered":"2024-03-26T17:37:27.739480+0000","last_clean":"2024-03-26T17:37:27.739480+0000","last_became_active":"2024-03-26T17:36:22.737038+0000","last_became_peered":"2024-03-26T17:36:22.737038+0000","last_unstale":"2024-03-26T17:37:27.739480+0000","last_undegraded":"2024-03-26T17:37:27.739480+0000","last_fullsized":"2024-03-26T17:37:27.739480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:48.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245526+0000","last_change":"2024-03-26T17:36:22.748097+0000","last_active":"2024-03-26T17:37:28.245526+0000","last_peered":"2024-03-26T17:37:28.245526+0000","last_clean":"2024-03-26T17:37:28.245526+0000","last_became_active":"2024-03-26T17:36:22.747730+0000","last_became_peered":"2024-03-26T17:36:22.747730+0000","last_unstale":"2024-03-26T17:37:28.245526+0000","last_undegraded":"2024-03-26T17:37:28.245526+0000","last_fullsized":"2024-03-26T17:37:28.245526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:43:01.370199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736622+0000","last_change":"2024-03-26T17:36:22.739587+0000","last_active":"2024-03-26T17:37:27.736622+0000","last_peered":"2024-03-26T17:37:27.736622+0000","last_clean":"2024-03-26T17:37:27.736622+0000","last_became_active":"2024-03-26T17:36:22.735787+0000","last_became_peered":"2024-03-26T17:36:22.735787+0000","last_unstale":"2024-03-26T17:37:27.736622+0000","last_undegraded":"2024-03-26T17:37:27.736622+0000","last_fullsized":"2024-03-26T17:37:27.736622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:41.381153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741378+0000","last_change":"2024-03-26T17:36:22.736716+0000","last_active":"2024-03-26T17:37:27.741378+0000","last_peered":"2024-03-26T17:37:27.741378+0000","last_clean":"2024-03-26T17:37:27.741378+0000","last_became_active":"2024-03-26T17:36:22.736327+0000","last_became_peered":"2024-03-26T17:36:22.736327+0000","last_unstale":"2024-03-26T17:37:27.741378+0000","last_undegraded":"2024-03-26T17:37:27.741378+0000","last_fullsized":"2024-03-26T17:37:27.741378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:10:43.439524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245361+0000","last_change":"2024-03-26T17:36:22.746355+0000","last_active":"2024-03-26T17:37:28.245361+0000","last_peered":"2024-03-26T17:37:28.245361+0000","last_clean":"2024-03-26T17:37:28.245361+0000","last_became_active":"2024-03-26T17:36:22.745363+0000","last_became_peered":"2024-03-26T17:36:22.745363+0000","last_unstale":"2024-03-26T17:37:28.245361+0000","last_undegraded":"2024-03-26T17:37:28.245361+0000","last_fullsized":"2024-03-26T17:37:28.245361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:56:33.528005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244350+0000","last_change":"2024-03-26T17:36:22.749009+0000","last_active":"2024-03-26T17:37:28.244350+0000","last_peered":"2024-03-26T17:37:28.244350+0000","last_clean":"2024-03-26T17:37:28.244350+0000","last_became_active":"2024-03-26T17:36:22.748627+0000","last_became_peered":"2024-03-26T17:36:22.748627+0000","last_unstale":"2024-03-26T17:37:28.244350+0000","last_undegraded":"2024-03-26T17:37:28.244350+0000","last_fullsized":"2024-03-26T17:37:28.244350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:59:14.812316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736930+0000","last_change":"2024-03-26T17:36:22.741294+0000","last_active":"2024-03-26T17:37:27.736930+0000","last_peered":"2024-03-26T17:37:27.736930+0000","last_clean":"2024-03-26T17:37:27.736930+0000","last_became_active":"2024-03-26T17:36:22.740003+0000","last_became_peered":"2024-03-26T17:36:22.740003+0000","last_unstale":"2024-03-26T17:37:27.736930+0000","last_undegraded":"2024-03-26T17:37:27.736930+0000","last_fullsized":"2024-03-26T17:37:27.736930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:06.305195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.243775+0000","last_change":"2024-03-26T17:36:22.745884+0000","last_active":"2024-03-26T17:37:28.243775+0000","last_peered":"2024-03-26T17:37:28.243775+0000","last_clean":"2024-03-26T17:37:28.243775+0000","last_became_active":"2024-03-26T17:36:22.744168+0000","last_became_peered":"2024-03-26T17:36:22.744168+0000","last_unstale":"2024-03-26T17:37:28.243775+0000","last_undegraded":"2024-03-26T17:37:28.243775+0000","last_fullsized":"2024-03-26T17:37:28.243775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:50.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740401+0000","last_change":"2024-03-26T17:36:22.725956+0000","last_active":"2024-03-26T17:37:27.740401+0000","last_peered":"2024-03-26T17:37:27.740401+0000","last_clean":"2024-03-26T17:37:27.740401+0000","last_became_active":"2024-03-26T17:36:22.725513+0000","last_became_peered":"2024-03-26T17:36:22.725513+0000","last_unstale":"2024-03-26T17:37:27.740401+0000","last_undegraded":"2024-03-26T17:37:27.740401+0000","last_fullsized":"2024-03-26T17:37:27.740401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:30:10.914233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736433+0000","last_change":"2024-03-26T17:36:22.739757+0000","last_active":"2024-03-26T17:37:27.736433+0000","last_peered":"2024-03-26T17:37:27.736433+0000","last_clean":"2024-03-26T17:37:27.736433+0000","last_became_active":"2024-03-26T17:36:22.739342+0000","last_became_peered":"2024-03-26T17:36:22.739342+0000","last_unstale":"2024-03-26T17:37:27.736433+0000","last_undegraded":"2024-03-26T17:37:27.736433+0000","last_fullsized":"2024-03-26T17:37:27.736433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:28.171325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736380+0000","last_change":"2024-03-26T17:36:22.742988+0000","last_active":"2024-03-26T17:37:27.736380+0000","last_peered":"2024-03-26T17:37:27.736380+0000","last_clean":"2024-03-26T17:37:27.736380+0000","last_became_active":"2024-03-26T17:36:22.742639+0000","last_became_peered":"2024-03-26T17:36:22.742639+0000","last_unstale":"2024-03-26T17:37:27.736380+0000","last_undegraded":"2024-03-26T17:37:27.736380+0000","last_fullsized":"2024-03-26T17:37:27.736380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:48:44.204889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245581+0000","last_change":"2024-03-26T17:36:22.744530+0000","last_active":"2024-03-26T17:37:28.245581+0000","last_peered":"2024-03-26T17:37:28.245581+0000","last_clean":"2024-03-26T17:37:28.245581+0000","last_became_active":"2024-03-26T17:36:22.743566+0000","last_became_peered":"2024-03-26T17:36:22.743566+0000","last_unstale":"2024-03-26T17:37:28.245581+0000","last_undegraded":"2024-03-26T17:37:28.245581+0000","last_fullsized":"2024-03-26T17:37:28.245581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:42.685326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738405+0000","last_change":"2024-03-26T17:36:22.745797+0000","last_active":"2024-03-26T17:37:27.738405+0000","last_peered":"2024-03-26T17:37:27.738405+0000","last_clean":"2024-03-26T17:37:27.738405+0000","last_became_active":"2024-03-26T17:36:22.745448+0000","last_became_peered":"2024-03-26T17:36:22.745448+0000","last_unstale":"2024-03-26T17:37:27.738405+0000","last_undegraded":"2024-03-26T17:37:27.738405+0000","last_fullsized":"2024-03-26T17:37:27.738405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:05:29.126564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739414+0000","last_change":"2024-03-26T17:36:22.749890+0000","last_active":"2024-03-26T17:37:27.739414+0000","last_peered":"2024-03-26T17:37:27.739414+0000","last_clean":"2024-03-26T17:37:27.739414+0000","last_became_active":"2024-03-26T17:36:22.749497+0000","last_became_peered":"2024-03-26T17:36:22.749497+0000","last_unstale":"2024-03-26T17:37:27.739414+0000","last_undegraded":"2024-03-26T17:37:27.739414+0000","last_fullsized":"2024-03-26T17:37:27.739414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:12:55.069659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742616+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:27.742616+0000","last_peered":"2024-03-26T17:37:27.742616+0000","last_clean":"2024-03-26T17:37:27.742616+0000","last_became_active":"2024-03-26T17:36:22.740287+0000","last_became_peered":"2024-03-26T17:36:22.740287+0000","last_unstale":"2024-03-26T17:37:27.742616+0000","last_undegraded":"2024-03-26T17:37:27.742616+0000","last_fullsized":"2024-03-26T17:37:27.742616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:13:00.574246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:31.378520+0000","last_change":"2024-03-26T17:36:21.720809+0000","last_active":"2024-03-26T17:37:31.378520+0000","last_peered":"2024-03-26T17:37:31.378520+0000","last_clean":"2024-03-26T17:37:31.378520+0000","last_became_active":"2024-03-26T17:36:21.720526+0000","last_became_peered":"2024-03-26T17:36:21.720526+0000","last_unstale":"2024-03-26T17:37:31.378520+0000","last_undegraded":"2024-03-26T17:37:31.378520+0000","last_fullsized":"2024-03-26T17:37:31.378520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:48.034328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741667+0000","last_change":"2024-03-26T17:36:22.746245+0000","last_active":"2024-03-26T17:37:27.741667+0000","last_peered":"2024-03-26T17:37:27.741667+0000","last_clean":"2024-03-26T17:37:27.741667+0000","last_became_active":"2024-03-26T17:36:22.745859+0000","last_became_peered":"2024-03-26T17:36:22.745859+0000","last_unstale":"2024-03-26T17:37:27.741667+0000","last_undegraded":"2024-03-26T17:37:27.741667+0000","last_fullsized":"2024-03-26T17:37:27.741667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:39.085668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741782+0000","last_change":"2024-03-26T17:36:21.723178+0000","last_active":"2024-03-26T17:37:27.741782+0000","last_peered":"2024-03-26T17:37:27.741782+0000","last_clean":"2024-03-26T17:37:27.741782+0000","last_became_active":"2024-03-26T17:36:21.722858+0000","last_became_peered":"2024-03-26T17:36:21.722858+0000","last_unstale":"2024-03-26T17:37:27.741782+0000","last_undegraded":"2024-03-26T17:37:27.741782+0000","last_fullsized":"2024-03-26T17:37:27.741782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:46:13.837672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736920+0000","last_change":"2024-03-26T17:36:22.736231+0000","last_active":"2024-03-26T17:37:27.736920+0000","last_peered":"2024-03-26T17:37:27.736920+0000","last_clean":"2024-03-26T17:37:27.736920+0000","last_became_active":"2024-03-26T17:36:22.735942+0000","last_became_peered":"2024-03-26T17:36:22.735942+0000","last_unstale":"2024-03-26T17:37:27.736920+0000","last_undegraded":"2024-03-26T17:37:27.736920+0000","last_fullsized":"2024-03-26T17:37:27.736920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:49.127382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740597+0000","last_change":"2024-03-26T17:36:21.731369+0000","last_active":"2024-03-26T17:37:27.740597+0000","last_peered":"2024-03-26T17:37:27.740597+0000","last_clean":"2024-03-26T17:37:27.740597+0000","last_became_active":"2024-03-26T17:36:21.731016+0000","last_became_peered":"2024-03-26T17:36:21.731016+0000","last_unstale":"2024-03-26T17:37:27.740597+0000","last_undegraded":"2024-03-26T17:37:27.740597+0000","last_fullsized":"2024-03-26T17:37:27.740597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:37:53.204825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244067+0000","last_change":"2024-03-26T17:36:22.751155+0000","last_active":"2024-03-26T17:37:28.244067+0000","last_peered":"2024-03-26T17:37:28.244067+0000","last_clean":"2024-03-26T17:37:28.244067+0000","last_became_active":"2024-03-26T17:36:22.750704+0000","last_became_peered":"2024-03-26T17:36:22.750704+0000","last_unstale":"2024-03-26T17:37:28.244067+0000","last_undegraded":"2024-03-26T17:37:28.244067+0000","last_fullsized":"2024-03-26T17:37:28.244067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:52:24.116923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739808+0000","last_change":"2024-03-26T17:36:21.726380+0000","last_active":"2024-03-26T17:37:27.739808+0000","last_peered":"2024-03-26T17:37:27.739808+0000","last_clean":"2024-03-26T17:37:27.739808+0000","last_became_active":"2024-03-26T17:36:21.726004+0000","last_became_peered":"2024-03-26T17:36:21.726004+0000","last_unstale":"2024-03-26T17:37:27.739808+0000","last_undegraded":"2024-03-26T17:37:27.739808+0000","last_fullsized":"2024-03-26T17:37:27.739808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:51:23.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744377+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:27.744377+0000","last_peered":"2024-03-26T17:37:27.744377+0000","last_clean":"2024-03-26T17:37:27.744377+0000","last_became_active":"2024-03-26T17:36:22.741104+0000","last_became_peered":"2024-03-26T17:36:22.741104+0000","last_unstale":"2024-03-26T17:37:27.744377+0000","last_undegraded":"2024-03-26T17:37:27.744377+0000","last_fullsized":"2024-03-26T17:37:27.744377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:53.683944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:33.377742+0000","last_change":"2024-03-26T17:36:21.728824+0000","last_active":"2024-03-26T17:37:33.377742+0000","last_peered":"2024-03-26T17:37:33.377742+0000","last_clean":"2024-03-26T17:37:33.377742+0000","last_became_active":"2024-03-26T17:36:21.728467+0000","last_became_peered":"2024-03-26T17:36:21.728467+0000","last_unstale":"2024-03-26T17:37:33.377742+0000","last_undegraded":"2024-03-26T17:37:33.377742+0000","last_fullsized":"2024-03-26T17:37:33.377742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:25:07.842111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739535+0000","last_change":"2024-03-26T17:36:22.740131+0000","last_active":"2024-03-26T17:37:27.739535+0000","last_peered":"2024-03-26T17:37:27.739535+0000","last_clean":"2024-03-26T17:37:27.739535+0000","last_became_active":"2024-03-26T17:36:22.739704+0000","last_became_peered":"2024-03-26T17:36:22.739704+0000","last_unstale":"2024-03-26T17:37:27.739535+0000","last_undegraded":"2024-03-26T17:37:27.739535+0000","last_fullsized":"2024-03-26T17:37:27.739535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:47.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245391+0000","last_change":"2024-03-26T17:36:21.731826+0000","last_active":"2024-03-26T17:37:28.245391+0000","last_peered":"2024-03-26T17:37:28.245391+0000","last_clean":"2024-03-26T17:37:28.245391+0000","last_became_active":"2024-03-26T17:36:21.731546+0000","last_became_peered":"2024-03-26T17:36:21.731546+0000","last_unstale":"2024-03-26T17:37:28.245391+0000","last_undegraded":"2024-03-26T17:37:28.245391+0000","last_fullsized":"2024-03-26T17:37:28.245391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:08:53.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742893+0000","last_change":"2024-03-26T17:36:22.745170+0000","last_active":"2024-03-26T17:37:27.742893+0000","last_peered":"2024-03-26T17:37:27.742893+0000","last_clean":"2024-03-26T17:37:27.742893+0000","last_became_active":"2024-03-26T17:36:22.744798+0000","last_became_peered":"2024-03-26T17:36:22.744798+0000","last_unstale":"2024-03-26T17:37:27.742893+0000","last_undegraded":"2024-03-26T17:37:27.742893+0000","last_fullsized":"2024-03-26T17:37:27.742893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:32:50.418149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739722+0000","last_change":"2024-03-26T17:36:21.721492+0000","last_active":"2024-03-26T17:37:27.739722+0000","last_peered":"2024-03-26T17:37:27.739722+0000","last_clean":"2024-03-26T17:37:27.739722+0000","last_became_active":"2024-03-26T17:36:21.721194+0000","last_became_peered":"2024-03-26T17:36:21.721194+0000","last_unstale":"2024-03-26T17:37:27.739722+0000","last_undegraded":"2024-03-26T17:37:27.739722+0000","last_fullsized":"2024-03-26T17:37:27.739722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:50.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741265+0000","last_change":"2024-03-26T17:36:22.740780+0000","last_active":"2024-03-26T17:37:27.741265+0000","last_peered":"2024-03-26T17:37:27.741265+0000","last_clean":"2024-03-26T17:37:27.741265+0000","last_became_active":"2024-03-26T17:36:22.740311+0000","last_became_peered":"2024-03-26T17:36:22.740311+0000","last_unstale":"2024-03-26T17:37:27.741265+0000","last_undegraded":"2024-03-26T17:37:27.741265+0000","last_fullsized":"2024-03-26T17:37:27.741265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:01:56.018416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742536+0000","last_change":"2024-03-26T17:36:21.726289+0000","last_active":"2024-03-26T17:37:27.742536+0000","last_peered":"2024-03-26T17:37:27.742536+0000","last_clean":"2024-03-26T17:37:27.742536+0000","last_became_active":"2024-03-26T17:36:21.725898+0000","last_became_peered":"2024-03-26T17:36:21.725898+0000","last_unstale":"2024-03-26T17:37:27.742536+0000","last_undegraded":"2024-03-26T17:37:27.742536+0000","last_fullsized":"2024-03-26T17:37:27.742536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:03.893163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739252+0000","last_change":"2024-03-26T17:36:22.750632+0000","last_active":"2024-03-26T17:37:27.739252+0000","last_peered":"2024-03-26T17:37:27.739252+0000","last_clean":"2024-03-26T17:37:27.739252+0000","last_became_active":"2024-03-26T17:36:22.750297+0000","last_became_peered":"2024-03-26T17:36:22.750297+0000","last_unstale":"2024-03-26T17:37:27.739252+0000","last_undegraded":"2024-03-26T17:37:27.739252+0000","last_fullsized":"2024-03-26T17:37:27.739252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:43:16.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:31.375217+0000","last_change":"2024-03-26T17:36:21.721403+0000","last_active":"2024-03-26T17:37:31.375217+0000","last_peered":"2024-03-26T17:37:31.375217+0000","last_clean":"2024-03-26T17:37:31.375217+0000","last_became_active":"2024-03-26T17:36:21.720962+0000","last_became_peered":"2024-03-26T17:36:21.720962+0000","last_unstale":"2024-03-26T17:37:31.375217+0000","last_undegraded":"2024-03-26T17:37:31.375217+0000","last_fullsized":"2024-03-26T17:37:31.375217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:02.683188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244911+0000","last_change":"2024-03-26T17:36:22.754010+0000","last_active":"2024-03-26T17:37:28.244911+0000","last_peered":"2024-03-26T17:37:28.244911+0000","last_clean":"2024-03-26T17:37:28.244911+0000","last_became_active":"2024-03-26T17:36:22.753534+0000","last_became_peered":"2024-03-26T17:36:22.753534+0000","last_unstale":"2024-03-26T17:37:28.244911+0000","last_undegraded":"2024-03-26T17:37:28.244911+0000","last_fullsized":"2024-03-26T17:37:28.244911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:06.490733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736915+0000","last_change":"2024-03-26T17:36:21.721685+0000","last_active":"2024-03-26T17:37:27.736915+0000","last_peered":"2024-03-26T17:37:27.736915+0000","last_clean":"2024-03-26T17:37:27.736915+0000","last_became_active":"2024-03-26T17:36:21.721387+0000","last_became_peered":"2024-03-26T17:36:21.721387+0000","last_unstale":"2024-03-26T17:37:27.736915+0000","last_undegraded":"2024-03-26T17:37:27.736915+0000","last_fullsized":"2024-03-26T17:37:27.736915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:59.493172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740779+0000","last_change":"2024-03-26T17:37:27.740779+0000","last_active":"2024-03-26T17:37:27.740779+0000","last_peered":"2024-03-26T17:37:27.740779+0000","last_clean":"2024-03-26T17:37:27.740779+0000","last_became_active":"2024-03-26T17:37:27.740407+0000","last_became_peered":"2024-03-26T17:37:27.740407+0000","last_unstale":"2024-03-26T17:37:27.740779+0000","last_undegraded":"2024-03-26T17:37:27.740779+0000","last_fullsized":"2024-03-26T17:37:27.740779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:39.526570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744477+0000","last_change":"2024-03-26T17:36:21.724041+0000","last_active":"2024-03-26T17:37:27.744477+0000","last_peered":"2024-03-26T17:37:27.744477+0000","last_clean":"2024-03-26T17:37:27.744477+0000","last_became_active":"2024-03-26T17:36:21.723624+0000","last_became_peered":"2024-03-26T17:36:21.723624+0000","last_unstale":"2024-03-26T17:37:27.744477+0000","last_undegraded":"2024-03-26T17:37:27.744477+0000","last_fullsized":"2024-03-26T17:37:27.744477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:42:24.264844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244229+0000","last_change":"2024-03-26T17:36:22.738592+0000","last_active":"2024-03-26T17:37:28.244229+0000","last_peered":"2024-03-26T17:37:28.244229+0000","last_clean":"2024-03-26T17:37:28.244229+0000","last_became_active":"2024-03-26T17:36:22.738236+0000","last_became_peered":"2024-03-26T17:36:22.738236+0000","last_unstale":"2024-03-26T17:37:28.244229+0000","last_undegraded":"2024-03-26T17:37:28.244229+0000","last_fullsized":"2024-03-26T17:37:28.244229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:56:06.977720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":185,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:31.375066+0000","last_change":"2024-03-26T17:36:21.722793+0000","last_active":"2024-03-26T17:37:31.375066+0000","last_peered":"2024-03-26T17:37:31.375066+0000","last_clean":"2024-03-26T17:37:31.375066+0000","last_became_active":"2024-03-26T17:36:21.722482+0000","last_became_peered":"2024-03-26T17:36:21.722482+0000","last_unstale":"2024-03-26T17:37:31.375066+0000","last_undegraded":"2024-03-26T17:37:31.375066+0000","last_fullsized":"2024-03-26T17:37:31.375066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:48:32.107017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":136,"num_read_kb":128,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739539+0000","last_change":"2024-03-26T17:36:22.746488+0000","last_active":"2024-03-26T17:37:27.739539+0000","last_peered":"2024-03-26T17:37:27.739539+0000","last_clean":"2024-03-26T17:37:27.739539+0000","last_became_active":"2024-03-26T17:36:22.746013+0000","last_became_peered":"2024-03-26T17:36:22.746013+0000","last_unstale":"2024-03-26T17:37:27.739539+0000","last_undegraded":"2024-03-26T17:37:27.739539+0000","last_fullsized":"2024-03-26T17:37:27.739539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:14.792336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742843+0000","last_change":"2024-03-26T17:36:21.724319+0000","last_active":"2024-03-26T17:37:27.742843+0000","last_peered":"2024-03-26T17:37:27.742843+0000","last_clean":"2024-03-26T17:37:27.742843+0000","last_became_active":"2024-03-26T17:36:21.723644+0000","last_became_peered":"2024-03-26T17:36:21.723644+0000","last_unstale":"2024-03-26T17:37:27.742843+0000","last_undegraded":"2024-03-26T17:37:27.742843+0000","last_fullsized":"2024-03-26T17:37:27.742843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:31:01.132797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739828+0000","last_change":"2024-03-26T17:36:22.736535+0000","last_active":"2024-03-26T17:37:27.739828+0000","last_peered":"2024-03-26T17:37:27.739828+0000","last_clean":"2024-03-26T17:37:27.739828+0000","last_became_active":"2024-03-26T17:36:22.736229+0000","last_became_peered":"2024-03-26T17:36:22.736229+0000","last_unstale":"2024-03-26T17:37:27.739828+0000","last_undegraded":"2024-03-26T17:37:27.739828+0000","last_fullsized":"2024-03-26T17:37:27.739828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:03.938180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742451+0000","last_change":"2024-03-26T17:36:21.726271+0000","last_active":"2024-03-26T17:37:27.742451+0000","last_peered":"2024-03-26T17:37:27.742451+0000","last_clean":"2024-03-26T17:37:27.742451+0000","last_became_active":"2024-03-26T17:36:21.725942+0000","last_became_peered":"2024-03-26T17:36:21.725942+0000","last_unstale":"2024-03-26T17:37:27.742451+0000","last_undegraded":"2024-03-26T17:37:27.742451+0000","last_fullsized":"2024-03-26T17:37:27.742451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:23:16.320106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736981+0000","last_change":"2024-03-26T17:36:22.743480+0000","last_active":"2024-03-26T17:37:27.736981+0000","last_peered":"2024-03-26T17:37:27.736981+0000","last_clean":"2024-03-26T17:37:27.736981+0000","last_became_active":"2024-03-26T17:36:22.742982+0000","last_became_peered":"2024-03-26T17:36:22.742982+0000","last_unstale":"2024-03-26T17:37:27.736981+0000","last_undegraded":"2024-03-26T17:37:27.736981+0000","last_fullsized":"2024-03-26T17:37:27.736981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:47.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:33.379226+0000","last_change":"2024-03-26T17:36:21.723407+0000","last_active":"2024-03-26T17:37:33.379226+0000","last_peered":"2024-03-26T17:37:33.379226+0000","last_clean":"2024-03-26T17:37:33.379226+0000","last_became_active":"2024-03-26T17:36:21.723086+0000","last_became_peered":"2024-03-26T17:36:21.723086+0000","last_unstale":"2024-03-26T17:37:33.379226+0000","last_undegraded":"2024-03-26T17:37:33.379226+0000","last_fullsized":"2024-03-26T17:37:33.379226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:26:57.886388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736567+0000","last_change":"2024-03-26T17:36:22.740705+0000","last_active":"2024-03-26T17:37:27.736567+0000","last_peered":"2024-03-26T17:37:27.736567+0000","last_clean":"2024-03-26T17:37:27.736567+0000","last_became_active":"2024-03-26T17:36:22.740360+0000","last_became_peered":"2024-03-26T17:36:22.740360+0000","last_unstale":"2024-03-26T17:37:27.736567+0000","last_undegraded":"2024-03-26T17:37:27.736567+0000","last_fullsized":"2024-03-26T17:37:27.736567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:02.879022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.266750+0000","last_change":"2024-03-26T17:37:28.266399+0000","last_active":"2024-03-26T17:37:28.266750+0000","last_peered":"2024-03-26T17:37:28.266750+0000","last_clean":"2024-03-26T17:37:28.266750+0000","last_became_active":"2024-03-26T17:37:28.246359+0000","last_became_peered":"2024-03-26T17:37:28.246359+0000","last_unstale":"2024-03-26T17:37:28.266750+0000","last_undegraded":"2024-03-26T17:37:28.266750+0000","last_fullsized":"2024-03-26T17:37:28.266750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:20:54.895081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742779+0000","last_change":"2024-03-26T17:36:22.743791+0000","last_active":"2024-03-26T17:37:27.742779+0000","last_peered":"2024-03-26T17:37:27.742779+0000","last_clean":"2024-03-26T17:37:27.742779+0000","last_became_active":"2024-03-26T17:36:22.742780+0000","last_became_peered":"2024-03-26T17:36:22.742780+0000","last_unstale":"2024-03-26T17:37:27.742779+0000","last_undegraded":"2024-03-26T17:37:27.742779+0000","last_fullsized":"2024-03-26T17:37:27.742779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:04.387958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740518+0000","last_change":"2024-03-26T17:36:21.724289+0000","last_active":"2024-03-26T17:37:27.740518+0000","last_peered":"2024-03-26T17:37:27.740518+0000","last_clean":"2024-03-26T17:37:27.740518+0000","last_became_active":"2024-03-26T17:36:21.723847+0000","last_became_peered":"2024-03-26T17:36:21.723847+0000","last_unstale":"2024-03-26T17:37:27.740518+0000","last_undegraded":"2024-03-26T17:37:27.740518+0000","last_fullsized":"2024-03-26T17:37:27.740518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:01:59.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738117+0000","last_change":"2024-03-26T17:36:22.748323+0000","last_active":"2024-03-26T17:37:27.738117+0000","last_peered":"2024-03-26T17:37:27.738117+0000","last_clean":"2024-03-26T17:37:27.738117+0000","last_became_active":"2024-03-26T17:36:22.747970+0000","last_became_peered":"2024-03-26T17:36:22.747970+0000","last_unstale":"2024-03-26T17:37:27.738117+0000","last_undegraded":"2024-03-26T17:37:27.738117+0000","last_fullsized":"2024-03-26T17:37:27.738117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:58.886425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739292+0000","last_change":"2024-03-26T17:36:21.726076+0000","last_active":"2024-03-26T17:37:27.739292+0000","last_peered":"2024-03-26T17:37:27.739292+0000","last_clean":"2024-03-26T17:37:27.739292+0000","last_became_active":"2024-03-26T17:36:21.722067+0000","last_became_peered":"2024-03-26T17:36:21.722067+0000","last_unstale":"2024-03-26T17:37:27.739292+0000","last_undegraded":"2024-03-26T17:37:27.739292+0000","last_fullsized":"2024-03-26T17:37:27.739292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:34:40.316110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.243853+0000","last_change":"2024-03-26T17:36:22.730890+0000","last_active":"2024-03-26T17:37:28.243853+0000","last_peered":"2024-03-26T17:37:28.243853+0000","last_clean":"2024-03-26T17:37:28.243853+0000","last_became_active":"2024-03-26T17:36:22.729635+0000","last_became_peered":"2024-03-26T17:36:22.729635+0000","last_unstale":"2024-03-26T17:37:28.243853+0000","last_undegraded":"2024-03-26T17:37:28.243853+0000","last_fullsized":"2024-03-26T17:37:28.243853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:15:09.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:31.378934+0000","last_change":"2024-03-26T17:36:21.727587+0000","last_active":"2024-03-26T17:37:31.378934+0000","last_peered":"2024-03-26T17:37:31.378934+0000","last_clean":"2024-03-26T17:37:31.378934+0000","last_became_active":"2024-03-26T17:36:21.727302+0000","last_became_peered":"2024-03-26T17:36:21.727302+0000","last_unstale":"2024-03-26T17:37:31.378934+0000","last_undegraded":"2024-03-26T17:37:31.378934+0000","last_fullsized":"2024-03-26T17:37:31.378934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:59.404567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245133+0000","last_change":"2024-03-26T17:36:22.750654+0000","last_active":"2024-03-26T17:37:28.245133+0000","last_peered":"2024-03-26T17:37:28.245133+0000","last_clean":"2024-03-26T17:37:28.245133+0000","last_became_active":"2024-03-26T17:36:22.750281+0000","last_became_peered":"2024-03-26T17:36:22.750281+0000","last_unstale":"2024-03-26T17:37:28.245133+0000","last_undegraded":"2024-03-26T17:37:28.245133+0000","last_fullsized":"2024-03-26T17:37:28.245133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:04.034108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.745100+0000","last_change":"2024-03-26T17:37:27.745100+0000","last_active":"2024-03-26T17:37:27.745100+0000","last_peered":"2024-03-26T17:37:27.745100+0000","last_clean":"2024-03-26T17:37:27.745100+0000","last_became_active":"2024-03-26T17:37:27.744686+0000","last_became_peered":"2024-03-26T17:37:27.744686+0000","last_unstale":"2024-03-26T17:37:27.745100+0000","last_undegraded":"2024-03-26T17:37:27.745100+0000","last_fullsized":"2024-03-26T17:37:27.745100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:38.111145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:31.377044+0000","last_change":"2024-03-26T17:36:21.730812+0000","last_active":"2024-03-26T17:37:31.377044+0000","last_peered":"2024-03-26T17:37:31.377044+0000","last_clean":"2024-03-26T17:37:31.377044+0000","last_became_active":"2024-03-26T17:36:21.730499+0000","last_became_peered":"2024-03-26T17:36:21.730499+0000","last_unstale":"2024-03-26T17:37:31.377044+0000","last_undegraded":"2024-03-26T17:37:31.377044+0000","last_fullsized":"2024-03-26T17:37:31.377044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:05.666146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741485+0000","last_change":"2024-03-26T17:36:22.742344+0000","last_active":"2024-03-26T17:37:27.741485+0000","last_peered":"2024-03-26T17:37:27.741485+0000","last_clean":"2024-03-26T17:37:27.741485+0000","last_became_active":"2024-03-26T17:36:22.741864+0000","last_became_peered":"2024-03-26T17:36:22.741864+0000","last_unstale":"2024-03-26T17:37:27.741485+0000","last_undegraded":"2024-03-26T17:37:27.741485+0000","last_fullsized":"2024-03-26T17:37:27.741485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:53:53.553107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244456+0000","last_change":"2024-03-26T17:36:21.722450+0000","last_active":"2024-03-26T17:37:28.244456+0000","last_peered":"2024-03-26T17:37:28.244456+0000","last_clean":"2024-03-26T17:37:28.244456+0000","last_became_active":"2024-03-26T17:36:21.722105+0000","last_became_peered":"2024-03-26T17:36:21.722105+0000","last_unstale":"2024-03-26T17:37:28.244456+0000","last_undegraded":"2024-03-26T17:37:28.244456+0000","last_fullsized":"2024-03-26T17:37:28.244456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:01:22.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736637+0000","last_change":"2024-03-26T17:35:10.663606+0000","last_active":"2024-03-26T17:37:27.736637+0000","last_peered":"2024-03-26T17:37:27.736637+0000","last_clean":"2024-03-26T17:37:27.736637+0000","last_became_active":"2024-03-26T17:35:10.663069+0000","last_became_peered":"2024-03-26T17:35:10.663069+0000","last_unstale":"2024-03-26T17:37:27.736637+0000","last_undegraded":"2024-03-26T17:37:27.736637+0000","last_fullsized":"2024-03-26T17:37:27.736637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_clean_scrub_stamp":"2024-03-26T17:31:50.880228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:50.817007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.745201+0000","last_change":"2024-03-26T17:37:27.745201+0000","last_active":"2024-03-26T17:37:27.745201+0000","last_peered":"2024-03-26T17:37:27.745201+0000","last_clean":"2024-03-26T17:37:27.745201+0000","last_became_active":"2024-03-26T17:37:27.744806+0000","last_became_peered":"2024-03-26T17:37:27.744806+0000","last_unstale":"2024-03-26T17:37:27.745201+0000","last_undegraded":"2024-03-26T17:37:27.745201+0000","last_fullsized":"2024-03-26T17:37:27.745201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:15:07.485013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736749+0000","last_change":"2024-03-26T17:36:21.723452+0000","last_active":"2024-03-26T17:37:27.736749+0000","last_peered":"2024-03-26T17:37:27.736749+0000","last_clean":"2024-03-26T17:37:27.736749+0000","last_became_active":"2024-03-26T17:36:21.723155+0000","last_became_peered":"2024-03-26T17:36:21.723155+0000","last_unstale":"2024-03-26T17:37:27.736749+0000","last_undegraded":"2024-03-26T17:37:27.736749+0000","last_fullsized":"2024-03-26T17:37:27.736749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:39:34.443624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.750092+0000","last_change":"2024-03-26T17:37:27.749741+0000","last_active":"2024-03-26T17:37:27.750092+0000","last_peered":"2024-03-26T17:37:27.750092+0000","last_clean":"2024-03-26T17:37:27.750092+0000","last_became_active":"2024-03-26T17:37:27.741751+0000","last_became_peered":"2024-03-26T17:37:27.741751+0000","last_unstale":"2024-03-26T17:37:27.750092+0000","last_undegraded":"2024-03-26T17:37:27.750092+0000","last_fullsized":"2024-03-26T17:37:27.750092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:42.119297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739984+0000","last_change":"2024-03-26T17:36:22.736487+0000","last_active":"2024-03-26T17:37:27.739984+0000","last_peered":"2024-03-26T17:37:27.739984+0000","last_clean":"2024-03-26T17:37:27.739984+0000","last_became_active":"2024-03-26T17:36:22.735788+0000","last_became_peered":"2024-03-26T17:36:22.735788+0000","last_unstale":"2024-03-26T17:37:27.739984+0000","last_undegraded":"2024-03-26T17:37:27.739984+0000","last_fullsized":"2024-03-26T17:37:27.739984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:45.240148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739464+0000","last_change":"2024-03-26T17:36:21.720755+0000","last_active":"2024-03-26T17:37:27.739464+0000","last_peered":"2024-03-26T17:37:27.739464+0000","last_clean":"2024-03-26T17:37:27.739464+0000","last_became_active":"2024-03-26T17:36:21.720175+0000","last_became_peered":"2024-03-26T17:36:21.720175+0000","last_unstale":"2024-03-26T17:37:27.739464+0000","last_undegraded":"2024-03-26T17:37:27.739464+0000","last_fullsized":"2024-03-26T17:37:27.739464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:22.458657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742189+0000","last_change":"2024-03-26T17:36:22.740731+0000","last_active":"2024-03-26T17:37:27.742189+0000","last_peered":"2024-03-26T17:37:27.742189+0000","last_clean":"2024-03-26T17:37:27.742189+0000","last_became_active":"2024-03-26T17:36:22.740283+0000","last_became_peered":"2024-03-26T17:36:22.740283+0000","last_unstale":"2024-03-26T17:37:27.742189+0000","last_undegraded":"2024-03-26T17:37:27.742189+0000","last_fullsized":"2024-03-26T17:37:27.742189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:36.403105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244032+0000","last_change":"2024-03-26T17:36:21.721513+0000","last_active":"2024-03-26T17:37:28.244032+0000","last_peered":"2024-03-26T17:37:28.244032+0000","last_clean":"2024-03-26T17:37:28.244032+0000","last_became_active":"2024-03-26T17:36:21.721185+0000","last_became_peered":"2024-03-26T17:36:21.721185+0000","last_unstale":"2024-03-26T17:37:28.244032+0000","last_undegraded":"2024-03-26T17:37:28.244032+0000","last_fullsized":"2024-03-26T17:37:28.244032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:48:39.994980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736856+0000","last_change":"2024-03-26T17:36:22.737938+0000","last_active":"2024-03-26T17:37:27.736856+0000","last_peered":"2024-03-26T17:37:27.736856+0000","last_clean":"2024-03-26T17:37:27.736856+0000","last_became_active":"2024-03-26T17:36:22.737605+0000","last_became_peered":"2024-03-26T17:36:22.737605+0000","last_unstale":"2024-03-26T17:37:27.736856+0000","last_undegraded":"2024-03-26T17:37:27.736856+0000","last_fullsized":"2024-03-26T17:37:27.736856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:27.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245677+0000","last_change":"2024-03-26T17:36:21.720942+0000","last_active":"2024-03-26T17:37:28.245677+0000","last_peered":"2024-03-26T17:37:28.245677+0000","last_clean":"2024-03-26T17:37:28.245677+0000","last_became_active":"2024-03-26T17:36:21.720665+0000","last_became_peered":"2024-03-26T17:36:21.720665+0000","last_unstale":"2024-03-26T17:37:28.245677+0000","last_undegraded":"2024-03-26T17:37:28.245677+0000","last_fullsized":"2024-03-26T17:37:28.245677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:00.725213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739390+0000","last_change":"2024-03-26T17:36:22.747111+0000","last_active":"2024-03-26T17:37:27.739390+0000","last_peered":"2024-03-26T17:37:27.739390+0000","last_clean":"2024-03-26T17:37:27.739390+0000","last_became_active":"2024-03-26T17:36:22.746626+0000","last_became_peered":"2024-03-26T17:36:22.746626+0000","last_unstale":"2024-03-26T17:37:27.739390+0000","last_undegraded":"2024-03-26T17:37:27.739390+0000","last_fullsized":"2024-03-26T17:37:27.739390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:29:24.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741395+0000","last_change":"2024-03-26T17:36:21.724141+0000","last_active":"2024-03-26T17:37:27.741395+0000","last_peered":"2024-03-26T17:37:27.741395+0000","last_clean":"2024-03-26T17:37:27.741395+0000","last_became_active":"2024-03-26T17:36:21.723797+0000","last_became_peered":"2024-03-26T17:36:21.723797+0000","last_unstale":"2024-03-26T17:37:27.741395+0000","last_undegraded":"2024-03-26T17:37:27.741395+0000","last_fullsized":"2024-03-26T17:37:27.741395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:06.149827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738751+0000","last_change":"2024-03-26T17:36:22.739497+0000","last_active":"2024-03-26T17:37:27.738751+0000","last_peered":"2024-03-26T17:37:27.738751+0000","last_clean":"2024-03-26T17:37:27.738751+0000","last_became_active":"2024-03-26T17:36:22.739114+0000","last_became_peered":"2024-03-26T17:36:22.739114+0000","last_unstale":"2024-03-26T17:37:27.738751+0000","last_undegraded":"2024-03-26T17:37:27.738751+0000","last_fullsized":"2024-03-26T17:37:27.738751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:03.953199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.743010+0000","last_change":"2024-03-26T17:36:21.728027+0000","last_active":"2024-03-26T17:37:27.743010+0000","last_peered":"2024-03-26T17:37:27.743010+0000","last_clean":"2024-03-26T17:37:27.743010+0000","last_became_active":"2024-03-26T17:36:21.727159+0000","last_became_peered":"2024-03-26T17:36:21.727159+0000","last_unstale":"2024-03-26T17:37:27.743010+0000","last_undegraded":"2024-03-26T17:37:27.743010+0000","last_fullsized":"2024-03-26T17:37:27.743010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:39.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741715+0000","last_change":"2024-03-26T17:36:22.733666+0000","last_active":"2024-03-26T17:37:27.741715+0000","last_peered":"2024-03-26T17:37:27.741715+0000","last_clean":"2024-03-26T17:37:27.741715+0000","last_became_active":"2024-03-26T17:36:22.733313+0000","last_became_peered":"2024-03-26T17:36:22.733313+0000","last_unstale":"2024-03-26T17:37:27.741715+0000","last_undegraded":"2024-03-26T17:37:27.741715+0000","last_fullsized":"2024-03-26T17:37:27.741715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:52:38.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739591+0000","last_change":"2024-03-26T17:36:21.730828+0000","last_active":"2024-03-26T17:37:27.739591+0000","last_peered":"2024-03-26T17:37:27.739591+0000","last_clean":"2024-03-26T17:37:27.739591+0000","last_became_active":"2024-03-26T17:36:21.730508+0000","last_became_peered":"2024-03-26T17:36:21.730508+0000","last_unstale":"2024-03-26T17:37:27.739591+0000","last_undegraded":"2024-03-26T17:37:27.739591+0000","last_fullsized":"2024-03-26T17:37:27.739591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:10:53.492725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736879+0000","last_change":"2024-03-26T17:36:22.740204+0000","last_active":"2024-03-26T17:37:27.736879+0000","last_peered":"2024-03-26T17:37:27.736879+0000","last_clean":"2024-03-26T17:37:27.736879+0000","last_became_active":"2024-03-26T17:36:22.735880+0000","last_became_peered":"2024-03-26T17:36:22.735880+0000","last_unstale":"2024-03-26T17:37:27.736879+0000","last_undegraded":"2024-03-26T17:37:27.736879+0000","last_fullsized":"2024-03-26T17:37:27.736879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:34.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:34.378569+0000","last_change":"2024-03-26T17:36:21.720873+0000","last_active":"2024-03-26T17:37:34.378569+0000","last_peered":"2024-03-26T17:37:34.378569+0000","last_clean":"2024-03-26T17:37:34.378569+0000","last_became_active":"2024-03-26T17:36:21.720496+0000","last_became_peered":"2024-03-26T17:36:21.720496+0000","last_unstale":"2024-03-26T17:37:34.378569+0000","last_undegraded":"2024-03-26T17:37:34.378569+0000","last_fullsized":"2024-03-26T17:37:34.378569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:40:55.959064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.737006+0000","last_change":"2024-03-26T17:36:22.735528+0000","last_active":"2024-03-26T17:37:27.737006+0000","last_peered":"2024-03-26T17:37:27.737006+0000","last_clean":"2024-03-26T17:37:27.737006+0000","last_became_active":"2024-03-26T17:36:22.735147+0000","last_became_peered":"2024-03-26T17:36:22.735147+0000","last_unstale":"2024-03-26T17:37:27.737006+0000","last_undegraded":"2024-03-26T17:37:27.737006+0000","last_fullsized":"2024-03-26T17:37:27.737006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:15.729853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744355+0000","last_change":"2024-03-26T17:36:21.723246+0000","last_active":"2024-03-26T17:37:27.744355+0000","last_peered":"2024-03-26T17:37:27.744355+0000","last_clean":"2024-03-26T17:37:27.744355+0000","last_became_active":"2024-03-26T17:36:21.722839+0000","last_became_peered":"2024-03-26T17:36:21.722839+0000","last_unstale":"2024-03-26T17:37:27.744355+0000","last_undegraded":"2024-03-26T17:37:27.744355+0000","last_fullsized":"2024-03-26T17:37:27.744355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:41:05.540206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736713+0000","last_change":"2024-03-26T17:36:22.742469+0000","last_active":"2024-03-26T17:37:27.736713+0000","last_peered":"2024-03-26T17:37:27.736713+0000","last_clean":"2024-03-26T17:37:27.736713+0000","last_became_active":"2024-03-26T17:36:22.742073+0000","last_became_peered":"2024-03-26T17:36:22.742073+0000","last_unstale":"2024-03-26T17:37:27.736713+0000","last_undegraded":"2024-03-26T17:37:27.736713+0000","last_fullsized":"2024-03-26T17:37:27.736713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:19:33.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.250123+0000","last_change":"2024-03-26T17:37:28.250123+0000","last_active":"2024-03-26T17:37:28.250123+0000","last_peered":"2024-03-26T17:37:28.250123+0000","last_clean":"2024-03-26T17:37:28.250123+0000","last_became_active":"2024-03-26T17:37:28.246162+0000","last_became_peered":"2024-03-26T17:37:28.246162+0000","last_unstale":"2024-03-26T17:37:28.250123+0000","last_undegraded":"2024-03-26T17:37:28.250123+0000","last_fullsized":"2024-03-26T17:37:28.250123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:33:39.002192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.737009+0000","last_change":"2024-03-26T17:36:22.742614+0000","last_active":"2024-03-26T17:37:27.737009+0000","last_peered":"2024-03-26T17:37:27.737009+0000","last_clean":"2024-03-26T17:37:27.737009+0000","last_became_active":"2024-03-26T17:36:22.742278+0000","last_became_peered":"2024-03-26T17:36:22.742278+0000","last_unstale":"2024-03-26T17:37:27.737009+0000","last_undegraded":"2024-03-26T17:37:27.737009+0000","last_fullsized":"2024-03-26T17:37:27.737009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:30:14.503474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738850+0000","last_change":"2024-03-26T17:36:22.741693+0000","last_active":"2024-03-26T17:37:27.738850+0000","last_peered":"2024-03-26T17:37:27.738850+0000","last_clean":"2024-03-26T17:37:27.738850+0000","last_became_active":"2024-03-26T17:36:22.741352+0000","last_became_peered":"2024-03-26T17:36:22.741352+0000","last_unstale":"2024-03-26T17:37:27.738850+0000","last_undegraded":"2024-03-26T17:37:27.738850+0000","last_fullsized":"2024-03-26T17:37:27.738850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:13:26.912910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742407+0000","last_change":"2024-03-26T17:36:22.742939+0000","last_active":"2024-03-26T17:37:27.742407+0000","last_peered":"2024-03-26T17:37:27.742407+0000","last_clean":"2024-03-26T17:37:27.742407+0000","last_became_active":"2024-03-26T17:36:22.742589+0000","last_became_peered":"2024-03-26T17:36:22.742589+0000","last_unstale":"2024-03-26T17:37:27.742407+0000","last_undegraded":"2024-03-26T17:37:27.742407+0000","last_fullsized":"2024-03-26T17:37:27.742407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:21:14.043601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741596+0000","last_change":"2024-03-26T17:36:22.748044+0000","last_active":"2024-03-26T17:37:27.741596+0000","last_peered":"2024-03-26T17:37:27.741596+0000","last_clean":"2024-03-26T17:37:27.741596+0000","last_became_active":"2024-03-26T17:36:22.747683+0000","last_became_peered":"2024-03-26T17:36:22.747683+0000","last_unstale":"2024-03-26T17:37:27.741596+0000","last_undegraded":"2024-03-26T17:37:27.741596+0000","last_fullsized":"2024-03-26T17:37:27.741596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:32:40.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736592+0000","last_change":"2024-03-26T17:36:22.743080+0000","last_active":"2024-03-26T17:37:27.736592+0000","last_peered":"2024-03-26T17:37:27.736592+0000","last_clean":"2024-03-26T17:37:27.736592+0000","last_became_active":"2024-03-26T17:36:22.742676+0000","last_became_peered":"2024-03-26T17:36:22.742676+0000","last_unstale":"2024-03-26T17:37:27.736592+0000","last_undegraded":"2024-03-26T17:37:27.736592+0000","last_fullsized":"2024-03-26T17:37:27.736592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:58.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739891+0000","last_change":"2024-03-26T17:36:22.737908+0000","last_active":"2024-03-26T17:37:27.739891+0000","last_peered":"2024-03-26T17:37:27.739891+0000","last_clean":"2024-03-26T17:37:27.739891+0000","last_became_active":"2024-03-26T17:36:22.737055+0000","last_became_peered":"2024-03-26T17:36:22.737055+0000","last_unstale":"2024-03-26T17:37:27.739891+0000","last_undegraded":"2024-03-26T17:37:27.739891+0000","last_fullsized":"2024-03-26T17:37:27.739891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:51:13.157901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738201+0000","last_change":"2024-03-26T17:36:22.749795+0000","last_active":"2024-03-26T17:37:27.738201+0000","last_peered":"2024-03-26T17:37:27.738201+0000","last_clean":"2024-03-26T17:37:27.738201+0000","last_became_active":"2024-03-26T17:36:22.749469+0000","last_became_peered":"2024-03-26T17:36:22.749469+0000","last_unstale":"2024-03-26T17:37:27.738201+0000","last_undegraded":"2024-03-26T17:37:27.738201+0000","last_fullsized":"2024-03-26T17:37:27.738201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:19:59.126270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.743272+0000","last_change":"2024-03-26T17:36:22.744101+0000","last_active":"2024-03-26T17:37:27.743272+0000","last_peered":"2024-03-26T17:37:27.743272+0000","last_clean":"2024-03-26T17:37:27.743272+0000","last_became_active":"2024-03-26T17:36:22.743761+0000","last_became_peered":"2024-03-26T17:36:22.743761+0000","last_unstale":"2024-03-26T17:37:27.743272+0000","last_undegraded":"2024-03-26T17:37:27.743272+0000","last_fullsized":"2024-03-26T17:37:27.743272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:27:19.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742102+0000","last_change":"2024-03-26T17:36:22.744800+0000","last_active":"2024-03-26T17:37:27.742102+0000","last_peered":"2024-03-26T17:37:27.742102+0000","last_clean":"2024-03-26T17:37:27.742102+0000","last_became_active":"2024-03-26T17:36:22.744437+0000","last_became_peered":"2024-03-26T17:36:22.744437+0000","last_unstale":"2024-03-26T17:37:27.742102+0000","last_undegraded":"2024-03-26T17:37:27.742102+0000","last_fullsized":"2024-03-26T17:37:27.742102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:36.459072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.745243+0000","last_change":"2024-03-26T17:37:27.745243+0000","last_active":"2024-03-26T17:37:27.745243+0000","last_peered":"2024-03-26T17:37:27.745243+0000","last_clean":"2024-03-26T17:37:27.745243+0000","last_became_active":"2024-03-26T17:37:27.744869+0000","last_became_peered":"2024-03-26T17:37:27.744869+0000","last_unstale":"2024-03-26T17:37:27.745243+0000","last_undegraded":"2024-03-26T17:37:27.745243+0000","last_fullsized":"2024-03-26T17:37:27.745243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:18:42.188733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741437+0000","last_change":"2024-03-26T17:36:22.743211+0000","last_active":"2024-03-26T17:37:27.741437+0000","last_peered":"2024-03-26T17:37:27.741437+0000","last_clean":"2024-03-26T17:37:27.741437+0000","last_became_active":"2024-03-26T17:36:22.742869+0000","last_became_peered":"2024-03-26T17:36:22.742869+0000","last_unstale":"2024-03-26T17:37:27.741437+0000","last_undegraded":"2024-03-26T17:37:27.741437+0000","last_fullsized":"2024-03-26T17:37:27.741437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:24:04.610012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.243871+0000","last_change":"2024-03-26T17:36:22.746950+0000","last_active":"2024-03-26T17:37:28.243871+0000","last_peered":"2024-03-26T17:37:28.243871+0000","last_clean":"2024-03-26T17:37:28.243871+0000","last_became_active":"2024-03-26T17:36:22.744869+0000","last_became_peered":"2024-03-26T17:36:22.744869+0000","last_unstale":"2024-03-26T17:37:28.243871+0000","last_undegraded":"2024-03-26T17:37:28.243871+0000","last_fullsized":"2024-03-26T17:37:28.243871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:09:52.283519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740309+0000","last_change":"2024-03-26T17:36:22.744627+0000","last_active":"2024-03-26T17:37:27.740309+0000","last_peered":"2024-03-26T17:37:27.740309+0000","last_clean":"2024-03-26T17:37:27.740309+0000","last_became_active":"2024-03-26T17:36:22.744299+0000","last_became_peered":"2024-03-26T17:36:22.744299+0000","last_unstale":"2024-03-26T17:37:27.740309+0000","last_undegraded":"2024-03-26T17:37:27.740309+0000","last_fullsized":"2024-03-26T17:37:27.740309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:23:37.941625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.737942+0000","last_change":"2024-03-26T17:36:22.751135+0000","last_active":"2024-03-26T17:37:27.737942+0000","last_peered":"2024-03-26T17:37:27.737942+0000","last_clean":"2024-03-26T17:37:27.737942+0000","last_became_active":"2024-03-26T17:36:22.750768+0000","last_became_peered":"2024-03-26T17:36:22.750768+0000","last_unstale":"2024-03-26T17:37:27.737942+0000","last_undegraded":"2024-03-26T17:37:27.737942+0000","last_fullsized":"2024-03-26T17:37:27.737942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:15.532367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67859,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":687,"num_read_kb":413,"num_write":236,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":456929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645343,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1108389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712440,"statfs":{"total":95995035648,"available":95961538560,"internally_reserved":0,"allocated":1568768,"data_stored":1048749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":455961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":518032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":457691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":518419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":455786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1581056,"data_stored":1048427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T17:37:38.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T17:37:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:39.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:38 smithi105 ceph-mon[32280]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:37:39.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:38 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:37:39.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:38 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:37:39.029 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:38 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:39.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:38 smithi144 ceph-mon[36095]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:37:39.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:38 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:37:39.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:38 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:37:39.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:38 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:39.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:38 smithi135 ceph-mon[35636]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:37:39.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:38 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:37:39.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:38 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:37:39.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:38 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:37:39.877 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:39 smithi105 ceph-mon[32280]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 20 B/s, 1 objects/s recovering 2024-03-26T17:37:40.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:39 smithi144 ceph-mon[36095]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 20 B/s, 1 objects/s recovering 2024-03-26T17:37:40.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:39 smithi135 ceph-mon[35636]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 20 B/s, 1 objects/s recovering 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 -- 172.21.15.105:0/4279304128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806b6e0 msgr2=0x7f2208107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 --2- 172.21.15.105:0/4279304128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806b6e0 0x7f2208107010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0099c0 tx=0x7f21fc009cd0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 -- 172.21.15.105:0/4279304128 shutdown_connections 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 --2- 172.21.15.105:0/4279304128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22081076f0 0x7f220810f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 --2- 172.21.15.105:0/4279304128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f220806ad30 0x7f220806b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 --2- 172.21.15.105:0/4279304128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806b6e0 0x7f2208107010 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 -- 172.21.15.105:0/4279304128 >> 172.21.15.105:0/4279304128 conn(0x7f2208078170 msgr2=0x7f2208078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 -- 172.21.15.105:0/4279304128 shutdown_connections 2024-03-26T17:37:40.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 -- 172.21.15.105:0/4279304128 wait complete. 2024-03-26T17:37:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 Processor -- start 2024-03-26T17:37:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.397+0000 7f220cf4b700 1 -- start start 2024-03-26T17:37:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220cf4b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806ad30 0x7f2208101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220cf4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220806b6e0 0x7f2208101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220cf4b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22081076f0 0x7f22081020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220cf4b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2208112560 con 0x7f220806ad30 2024-03-26T17:37:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220cf4b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f22081123e0 con 0x7f22081076f0 2024-03-26T17:37:40.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220cf4b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f22081126e0 con 0x7f220806b6e0 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806ad30 0x7f2208101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806ad30 0x7f2208101660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38678/0 (socket says 172.21.15.105:38678) 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 -- 172.21.15.105:0/683523772 learned_addr learned my addr 172.21.15.105:0/683523772 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f2206d9d700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22081076f0 0x7f22081020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f2205d9b700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220806b6e0 0x7f2208101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 -- 172.21.15.105:0/683523772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220806b6e0 msgr2=0x7f2208101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220806b6e0 0x7f2208101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 -- 172.21.15.105:0/683523772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22081076f0 msgr2=0x7f22081020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22081076f0 0x7f22081020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 -- 172.21.15.105:0/683523772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081a6fe0 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f220659c700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806ad30 0x7f2208101660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f21f800d980 tx=0x7f21f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.398+0000 7f21f77fe700 1 -- 172.21.15.105:0/683523772 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8016070 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.399+0000 7f21f77fe700 1 -- 172.21.15.105:0/683523772 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21f8004830 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.399+0000 7f220cf4b700 1 -- 172.21.15.105:0/683523772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22081a72d0 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.399+0000 7f21f77fe700 1 -- 172.21.15.105:0/683523772 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8005530 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.399+0000 7f220cf4b700 1 -- 172.21.15.105:0/683523772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22081a77e0 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.399+0000 7f21f57fa700 1 -- 172.21.15.105:0/683523772 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2208103af0 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.401+0000 7f21f77fe700 1 -- 172.21.15.105:0/683523772 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f21f80049e0 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.401+0000 7f21f77fe700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f21f00a3040 0x7f21f00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.401+0000 7f2205d9b700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f21f00a3040 0x7f21f00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.401+0000 7f21f77fe700 1 -- 172.21.15.105:0/683523772 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f21f8099e20 con 0x7f220806ad30 2024-03-26T17:37:40.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.402+0000 7f2205d9b700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f21f00a3040 0x7f21f00a5500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f21fc000c00 tx=0x7f21fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:40.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.410+0000 7f21f77fe700 1 -- 172.21.15.105:0/683523772 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f21f801f020 con 0x7f220806ad30 2024-03-26T17:37:40.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.678+0000 7f21f57fa700 1 -- 172.21.15.105:0/683523772 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2208069070 con 0x7f21f00a3040 2024-03-26T17:37:40.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.692+0000 7f21f77fe700 1 -- 172.21.15.105:0/683523772 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258773 (secure 0 0 0) 0x7f2208069070 con 0x7f21f00a3040 2024-03-26T17:37:40.690 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 -- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f21f00a3040 msgr2=0x7f21f00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f21f00a3040 0x7f21f00a5500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f21fc000c00 tx=0x7f21fc005fb0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 -- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806ad30 msgr2=0x7f2208101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806ad30 0x7f2208101660 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f21f800d980 tx=0x7f21f800dc90 comp rx=0 tx=0).stop 2024-03-26T17:37:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 -- 172.21.15.105:0/683523772 shutdown_connections 2024-03-26T17:37:40.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f21f00a3040 0x7f21f00a5500 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f22081076f0 0x7f22081020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220806ad30 0x7f2208101660 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 --2- 172.21.15.105:0/683523772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f220806b6e0 0x7f2208101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 -- 172.21.15.105:0/683523772 >> 172.21.15.105:0/683523772 conn(0x7f2208078170 msgr2=0x7f22080ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 -- 172.21.15.105:0/683523772 shutdown_connections 2024-03-26T17:37:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:40.695+0000 7f21f57fa700 1 -- 172.21.15.105:0/683523772 wait complete. 2024-03-26T17:37:40.732 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-26T17:37:41.300 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-03-26T17:37:40.599937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658461,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":858,"num_read_kb":547,"num_write":419,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366688,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575136,"statfs":{"total":1151940427776,"available":1151564939264,"internally_reserved":0,"allocated":13627392,"data_stored":7436847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":48,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002753"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738305+0000","last_change":"2024-03-26T17:36:22.743815+0000","last_active":"2024-03-26T17:37:27.738305+0000","last_peered":"2024-03-26T17:37:27.738305+0000","last_clean":"2024-03-26T17:37:27.738305+0000","last_became_active":"2024-03-26T17:36:22.743446+0000","last_became_peered":"2024-03-26T17:36:22.743446+0000","last_unstale":"2024-03-26T17:37:27.738305+0000","last_undegraded":"2024-03-26T17:37:27.738305+0000","last_fullsized":"2024-03-26T17:37:27.738305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:16:56.773849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736811+0000","last_change":"2024-03-26T17:36:22.740754+0000","last_active":"2024-03-26T17:37:27.736811+0000","last_peered":"2024-03-26T17:37:27.736811+0000","last_clean":"2024-03-26T17:37:27.736811+0000","last_became_active":"2024-03-26T17:36:22.740317+0000","last_became_peered":"2024-03-26T17:36:22.740317+0000","last_unstale":"2024-03-26T17:37:27.736811+0000","last_undegraded":"2024-03-26T17:37:27.736811+0000","last_fullsized":"2024-03-26T17:37:27.736811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:50:45.321653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736774+0000","last_change":"2024-03-26T17:36:22.735079+0000","last_active":"2024-03-26T17:37:27.736774+0000","last_peered":"2024-03-26T17:37:27.736774+0000","last_clean":"2024-03-26T17:37:27.736774+0000","last_became_active":"2024-03-26T17:36:22.734646+0000","last_became_peered":"2024-03-26T17:36:22.734646+0000","last_unstale":"2024-03-26T17:37:27.736774+0000","last_undegraded":"2024-03-26T17:37:27.736774+0000","last_fullsized":"2024-03-26T17:37:27.736774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:41:09.477438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.243877+0000","last_change":"2024-03-26T17:36:22.746170+0000","last_active":"2024-03-26T17:37:28.243877+0000","last_peered":"2024-03-26T17:37:28.243877+0000","last_clean":"2024-03-26T17:37:28.243877+0000","last_became_active":"2024-03-26T17:36:22.744255+0000","last_became_peered":"2024-03-26T17:36:22.744255+0000","last_unstale":"2024-03-26T17:37:28.243877+0000","last_undegraded":"2024-03-26T17:37:28.243877+0000","last_fullsized":"2024-03-26T17:37:28.243877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:54:36.225926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739480+0000","last_change":"2024-03-26T17:36:22.737390+0000","last_active":"2024-03-26T17:37:27.739480+0000","last_peered":"2024-03-26T17:37:27.739480+0000","last_clean":"2024-03-26T17:37:27.739480+0000","last_became_active":"2024-03-26T17:36:22.737038+0000","last_became_peered":"2024-03-26T17:36:22.737038+0000","last_unstale":"2024-03-26T17:37:27.739480+0000","last_undegraded":"2024-03-26T17:37:27.739480+0000","last_fullsized":"2024-03-26T17:37:27.739480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:48.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245526+0000","last_change":"2024-03-26T17:36:22.748097+0000","last_active":"2024-03-26T17:37:28.245526+0000","last_peered":"2024-03-26T17:37:28.245526+0000","last_clean":"2024-03-26T17:37:28.245526+0000","last_became_active":"2024-03-26T17:36:22.747730+0000","last_became_peered":"2024-03-26T17:36:22.747730+0000","last_unstale":"2024-03-26T17:37:28.245526+0000","last_undegraded":"2024-03-26T17:37:28.245526+0000","last_fullsized":"2024-03-26T17:37:28.245526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:43:01.370199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736622+0000","last_change":"2024-03-26T17:36:22.739587+0000","last_active":"2024-03-26T17:37:27.736622+0000","last_peered":"2024-03-26T17:37:27.736622+0000","last_clean":"2024-03-26T17:37:27.736622+0000","last_became_active":"2024-03-26T17:36:22.735787+0000","last_became_peered":"2024-03-26T17:36:22.735787+0000","last_unstale":"2024-03-26T17:37:27.736622+0000","last_undegraded":"2024-03-26T17:37:27.736622+0000","last_fullsized":"2024-03-26T17:37:27.736622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:41.381153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741378+0000","last_change":"2024-03-26T17:36:22.736716+0000","last_active":"2024-03-26T17:37:27.741378+0000","last_peered":"2024-03-26T17:37:27.741378+0000","last_clean":"2024-03-26T17:37:27.741378+0000","last_became_active":"2024-03-26T17:36:22.736327+0000","last_became_peered":"2024-03-26T17:36:22.736327+0000","last_unstale":"2024-03-26T17:37:27.741378+0000","last_undegraded":"2024-03-26T17:37:27.741378+0000","last_fullsized":"2024-03-26T17:37:27.741378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:10:43.439524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245361+0000","last_change":"2024-03-26T17:36:22.746355+0000","last_active":"2024-03-26T17:37:28.245361+0000","last_peered":"2024-03-26T17:37:28.245361+0000","last_clean":"2024-03-26T17:37:28.245361+0000","last_became_active":"2024-03-26T17:36:22.745363+0000","last_became_peered":"2024-03-26T17:36:22.745363+0000","last_unstale":"2024-03-26T17:37:28.245361+0000","last_undegraded":"2024-03-26T17:37:28.245361+0000","last_fullsized":"2024-03-26T17:37:28.245361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:56:33.528005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244350+0000","last_change":"2024-03-26T17:36:22.749009+0000","last_active":"2024-03-26T17:37:28.244350+0000","last_peered":"2024-03-26T17:37:28.244350+0000","last_clean":"2024-03-26T17:37:28.244350+0000","last_became_active":"2024-03-26T17:36:22.748627+0000","last_became_peered":"2024-03-26T17:36:22.748627+0000","last_unstale":"2024-03-26T17:37:28.244350+0000","last_undegraded":"2024-03-26T17:37:28.244350+0000","last_fullsized":"2024-03-26T17:37:28.244350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:59:14.812316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736930+0000","last_change":"2024-03-26T17:36:22.741294+0000","last_active":"2024-03-26T17:37:27.736930+0000","last_peered":"2024-03-26T17:37:27.736930+0000","last_clean":"2024-03-26T17:37:27.736930+0000","last_became_active":"2024-03-26T17:36:22.740003+0000","last_became_peered":"2024-03-26T17:36:22.740003+0000","last_unstale":"2024-03-26T17:37:27.736930+0000","last_undegraded":"2024-03-26T17:37:27.736930+0000","last_fullsized":"2024-03-26T17:37:27.736930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:06.305195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.243775+0000","last_change":"2024-03-26T17:36:22.745884+0000","last_active":"2024-03-26T17:37:28.243775+0000","last_peered":"2024-03-26T17:37:28.243775+0000","last_clean":"2024-03-26T17:37:28.243775+0000","last_became_active":"2024-03-26T17:36:22.744168+0000","last_became_peered":"2024-03-26T17:36:22.744168+0000","last_unstale":"2024-03-26T17:37:28.243775+0000","last_undegraded":"2024-03-26T17:37:28.243775+0000","last_fullsized":"2024-03-26T17:37:28.243775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:50.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740401+0000","last_change":"2024-03-26T17:36:22.725956+0000","last_active":"2024-03-26T17:37:27.740401+0000","last_peered":"2024-03-26T17:37:27.740401+0000","last_clean":"2024-03-26T17:37:27.740401+0000","last_became_active":"2024-03-26T17:36:22.725513+0000","last_became_peered":"2024-03-26T17:36:22.725513+0000","last_unstale":"2024-03-26T17:37:27.740401+0000","last_undegraded":"2024-03-26T17:37:27.740401+0000","last_fullsized":"2024-03-26T17:37:27.740401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:30:10.914233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736433+0000","last_change":"2024-03-26T17:36:22.739757+0000","last_active":"2024-03-26T17:37:27.736433+0000","last_peered":"2024-03-26T17:37:27.736433+0000","last_clean":"2024-03-26T17:37:27.736433+0000","last_became_active":"2024-03-26T17:36:22.739342+0000","last_became_peered":"2024-03-26T17:36:22.739342+0000","last_unstale":"2024-03-26T17:37:27.736433+0000","last_undegraded":"2024-03-26T17:37:27.736433+0000","last_fullsized":"2024-03-26T17:37:27.736433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:28.171325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736380+0000","last_change":"2024-03-26T17:36:22.742988+0000","last_active":"2024-03-26T17:37:27.736380+0000","last_peered":"2024-03-26T17:37:27.736380+0000","last_clean":"2024-03-26T17:37:27.736380+0000","last_became_active":"2024-03-26T17:36:22.742639+0000","last_became_peered":"2024-03-26T17:36:22.742639+0000","last_unstale":"2024-03-26T17:37:27.736380+0000","last_undegraded":"2024-03-26T17:37:27.736380+0000","last_fullsized":"2024-03-26T17:37:27.736380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:48:44.204889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245581+0000","last_change":"2024-03-26T17:36:22.744530+0000","last_active":"2024-03-26T17:37:28.245581+0000","last_peered":"2024-03-26T17:37:28.245581+0000","last_clean":"2024-03-26T17:37:28.245581+0000","last_became_active":"2024-03-26T17:36:22.743566+0000","last_became_peered":"2024-03-26T17:36:22.743566+0000","last_unstale":"2024-03-26T17:37:28.245581+0000","last_undegraded":"2024-03-26T17:37:28.245581+0000","last_fullsized":"2024-03-26T17:37:28.245581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:42.685326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738405+0000","last_change":"2024-03-26T17:36:22.745797+0000","last_active":"2024-03-26T17:37:27.738405+0000","last_peered":"2024-03-26T17:37:27.738405+0000","last_clean":"2024-03-26T17:37:27.738405+0000","last_became_active":"2024-03-26T17:36:22.745448+0000","last_became_peered":"2024-03-26T17:36:22.745448+0000","last_unstale":"2024-03-26T17:37:27.738405+0000","last_undegraded":"2024-03-26T17:37:27.738405+0000","last_fullsized":"2024-03-26T17:37:27.738405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:05:29.126564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739414+0000","last_change":"2024-03-26T17:36:22.749890+0000","last_active":"2024-03-26T17:37:27.739414+0000","last_peered":"2024-03-26T17:37:27.739414+0000","last_clean":"2024-03-26T17:37:27.739414+0000","last_became_active":"2024-03-26T17:36:22.749497+0000","last_became_peered":"2024-03-26T17:36:22.749497+0000","last_unstale":"2024-03-26T17:37:27.739414+0000","last_undegraded":"2024-03-26T17:37:27.739414+0000","last_fullsized":"2024-03-26T17:37:27.739414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:12:55.069659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742616+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:27.742616+0000","last_peered":"2024-03-26T17:37:27.742616+0000","last_clean":"2024-03-26T17:37:27.742616+0000","last_became_active":"2024-03-26T17:36:22.740287+0000","last_became_peered":"2024-03-26T17:36:22.740287+0000","last_unstale":"2024-03-26T17:37:27.742616+0000","last_undegraded":"2024-03-26T17:37:27.742616+0000","last_fullsized":"2024-03-26T17:37:27.742616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:13:00.574246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":102,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:36.383688+0000","last_change":"2024-03-26T17:36:21.720809+0000","last_active":"2024-03-26T17:37:36.383688+0000","last_peered":"2024-03-26T17:37:36.383688+0000","last_clean":"2024-03-26T17:37:36.383688+0000","last_became_active":"2024-03-26T17:36:21.720526+0000","last_became_peered":"2024-03-26T17:36:21.720526+0000","last_unstale":"2024-03-26T17:37:36.383688+0000","last_undegraded":"2024-03-26T17:37:36.383688+0000","last_fullsized":"2024-03-26T17:37:36.383688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:48.034328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741667+0000","last_change":"2024-03-26T17:36:22.746245+0000","last_active":"2024-03-26T17:37:27.741667+0000","last_peered":"2024-03-26T17:37:27.741667+0000","last_clean":"2024-03-26T17:37:27.741667+0000","last_became_active":"2024-03-26T17:36:22.745859+0000","last_became_peered":"2024-03-26T17:36:22.745859+0000","last_unstale":"2024-03-26T17:37:27.741667+0000","last_undegraded":"2024-03-26T17:37:27.741667+0000","last_fullsized":"2024-03-26T17:37:27.741667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:39.085668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741782+0000","last_change":"2024-03-26T17:36:21.723178+0000","last_active":"2024-03-26T17:37:27.741782+0000","last_peered":"2024-03-26T17:37:27.741782+0000","last_clean":"2024-03-26T17:37:27.741782+0000","last_became_active":"2024-03-26T17:36:21.722858+0000","last_became_peered":"2024-03-26T17:36:21.722858+0000","last_unstale":"2024-03-26T17:37:27.741782+0000","last_undegraded":"2024-03-26T17:37:27.741782+0000","last_fullsized":"2024-03-26T17:37:27.741782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:46:13.837672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736920+0000","last_change":"2024-03-26T17:36:22.736231+0000","last_active":"2024-03-26T17:37:27.736920+0000","last_peered":"2024-03-26T17:37:27.736920+0000","last_clean":"2024-03-26T17:37:27.736920+0000","last_became_active":"2024-03-26T17:36:22.735942+0000","last_became_peered":"2024-03-26T17:36:22.735942+0000","last_unstale":"2024-03-26T17:37:27.736920+0000","last_undegraded":"2024-03-26T17:37:27.736920+0000","last_fullsized":"2024-03-26T17:37:27.736920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:49.127382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740597+0000","last_change":"2024-03-26T17:36:21.731369+0000","last_active":"2024-03-26T17:37:27.740597+0000","last_peered":"2024-03-26T17:37:27.740597+0000","last_clean":"2024-03-26T17:37:27.740597+0000","last_became_active":"2024-03-26T17:36:21.731016+0000","last_became_peered":"2024-03-26T17:36:21.731016+0000","last_unstale":"2024-03-26T17:37:27.740597+0000","last_undegraded":"2024-03-26T17:37:27.740597+0000","last_fullsized":"2024-03-26T17:37:27.740597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:37:53.204825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244067+0000","last_change":"2024-03-26T17:36:22.751155+0000","last_active":"2024-03-26T17:37:28.244067+0000","last_peered":"2024-03-26T17:37:28.244067+0000","last_clean":"2024-03-26T17:37:28.244067+0000","last_became_active":"2024-03-26T17:36:22.750704+0000","last_became_peered":"2024-03-26T17:36:22.750704+0000","last_unstale":"2024-03-26T17:37:28.244067+0000","last_undegraded":"2024-03-26T17:37:28.244067+0000","last_fullsized":"2024-03-26T17:37:28.244067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:52:24.116923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739808+0000","last_change":"2024-03-26T17:36:21.726380+0000","last_active":"2024-03-26T17:37:27.739808+0000","last_peered":"2024-03-26T17:37:27.739808+0000","last_clean":"2024-03-26T17:37:27.739808+0000","last_became_active":"2024-03-26T17:36:21.726004+0000","last_became_peered":"2024-03-26T17:36:21.726004+0000","last_unstale":"2024-03-26T17:37:27.739808+0000","last_undegraded":"2024-03-26T17:37:27.739808+0000","last_fullsized":"2024-03-26T17:37:27.739808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:51:23.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744377+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:27.744377+0000","last_peered":"2024-03-26T17:37:27.744377+0000","last_clean":"2024-03-26T17:37:27.744377+0000","last_became_active":"2024-03-26T17:36:22.741104+0000","last_became_peered":"2024-03-26T17:36:22.741104+0000","last_unstale":"2024-03-26T17:37:27.744377+0000","last_undegraded":"2024-03-26T17:37:27.744377+0000","last_fullsized":"2024-03-26T17:37:27.744377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:53.683944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:38.382772+0000","last_change":"2024-03-26T17:36:21.728824+0000","last_active":"2024-03-26T17:37:38.382772+0000","last_peered":"2024-03-26T17:37:38.382772+0000","last_clean":"2024-03-26T17:37:38.382772+0000","last_became_active":"2024-03-26T17:36:21.728467+0000","last_became_peered":"2024-03-26T17:36:21.728467+0000","last_unstale":"2024-03-26T17:37:38.382772+0000","last_undegraded":"2024-03-26T17:37:38.382772+0000","last_fullsized":"2024-03-26T17:37:38.382772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:25:07.842111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739535+0000","last_change":"2024-03-26T17:36:22.740131+0000","last_active":"2024-03-26T17:37:27.739535+0000","last_peered":"2024-03-26T17:37:27.739535+0000","last_clean":"2024-03-26T17:37:27.739535+0000","last_became_active":"2024-03-26T17:36:22.739704+0000","last_became_peered":"2024-03-26T17:36:22.739704+0000","last_unstale":"2024-03-26T17:37:27.739535+0000","last_undegraded":"2024-03-26T17:37:27.739535+0000","last_fullsized":"2024-03-26T17:37:27.739535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:47.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245391+0000","last_change":"2024-03-26T17:36:21.731826+0000","last_active":"2024-03-26T17:37:28.245391+0000","last_peered":"2024-03-26T17:37:28.245391+0000","last_clean":"2024-03-26T17:37:28.245391+0000","last_became_active":"2024-03-26T17:36:21.731546+0000","last_became_peered":"2024-03-26T17:36:21.731546+0000","last_unstale":"2024-03-26T17:37:28.245391+0000","last_undegraded":"2024-03-26T17:37:28.245391+0000","last_fullsized":"2024-03-26T17:37:28.245391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:08:53.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742893+0000","last_change":"2024-03-26T17:36:22.745170+0000","last_active":"2024-03-26T17:37:27.742893+0000","last_peered":"2024-03-26T17:37:27.742893+0000","last_clean":"2024-03-26T17:37:27.742893+0000","last_became_active":"2024-03-26T17:36:22.744798+0000","last_became_peered":"2024-03-26T17:36:22.744798+0000","last_unstale":"2024-03-26T17:37:27.742893+0000","last_undegraded":"2024-03-26T17:37:27.742893+0000","last_fullsized":"2024-03-26T17:37:27.742893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:32:50.418149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739722+0000","last_change":"2024-03-26T17:36:21.721492+0000","last_active":"2024-03-26T17:37:27.739722+0000","last_peered":"2024-03-26T17:37:27.739722+0000","last_clean":"2024-03-26T17:37:27.739722+0000","last_became_active":"2024-03-26T17:36:21.721194+0000","last_became_peered":"2024-03-26T17:36:21.721194+0000","last_unstale":"2024-03-26T17:37:27.739722+0000","last_undegraded":"2024-03-26T17:37:27.739722+0000","last_fullsized":"2024-03-26T17:37:27.739722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:50.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741265+0000","last_change":"2024-03-26T17:36:22.740780+0000","last_active":"2024-03-26T17:37:27.741265+0000","last_peered":"2024-03-26T17:37:27.741265+0000","last_clean":"2024-03-26T17:37:27.741265+0000","last_became_active":"2024-03-26T17:36:22.740311+0000","last_became_peered":"2024-03-26T17:36:22.740311+0000","last_unstale":"2024-03-26T17:37:27.741265+0000","last_undegraded":"2024-03-26T17:37:27.741265+0000","last_fullsized":"2024-03-26T17:37:27.741265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:01:56.018416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742536+0000","last_change":"2024-03-26T17:36:21.726289+0000","last_active":"2024-03-26T17:37:27.742536+0000","last_peered":"2024-03-26T17:37:27.742536+0000","last_clean":"2024-03-26T17:37:27.742536+0000","last_became_active":"2024-03-26T17:36:21.725898+0000","last_became_peered":"2024-03-26T17:36:21.725898+0000","last_unstale":"2024-03-26T17:37:27.742536+0000","last_undegraded":"2024-03-26T17:37:27.742536+0000","last_fullsized":"2024-03-26T17:37:27.742536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:03.893163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739252+0000","last_change":"2024-03-26T17:36:22.750632+0000","last_active":"2024-03-26T17:37:27.739252+0000","last_peered":"2024-03-26T17:37:27.739252+0000","last_clean":"2024-03-26T17:37:27.739252+0000","last_became_active":"2024-03-26T17:36:22.750297+0000","last_became_peered":"2024-03-26T17:36:22.750297+0000","last_unstale":"2024-03-26T17:37:27.739252+0000","last_undegraded":"2024-03-26T17:37:27.739252+0000","last_fullsized":"2024-03-26T17:37:27.739252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:43:16.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":161,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:36.380165+0000","last_change":"2024-03-26T17:36:21.721403+0000","last_active":"2024-03-26T17:37:36.380165+0000","last_peered":"2024-03-26T17:37:36.380165+0000","last_clean":"2024-03-26T17:37:36.380165+0000","last_became_active":"2024-03-26T17:36:21.720962+0000","last_became_peered":"2024-03-26T17:36:21.720962+0000","last_unstale":"2024-03-26T17:37:36.380165+0000","last_undegraded":"2024-03-26T17:37:36.380165+0000","last_fullsized":"2024-03-26T17:37:36.380165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:02.683188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244911+0000","last_change":"2024-03-26T17:36:22.754010+0000","last_active":"2024-03-26T17:37:28.244911+0000","last_peered":"2024-03-26T17:37:28.244911+0000","last_clean":"2024-03-26T17:37:28.244911+0000","last_became_active":"2024-03-26T17:36:22.753534+0000","last_became_peered":"2024-03-26T17:36:22.753534+0000","last_unstale":"2024-03-26T17:37:28.244911+0000","last_undegraded":"2024-03-26T17:37:28.244911+0000","last_fullsized":"2024-03-26T17:37:28.244911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:06.490733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736915+0000","last_change":"2024-03-26T17:36:21.721685+0000","last_active":"2024-03-26T17:37:27.736915+0000","last_peered":"2024-03-26T17:37:27.736915+0000","last_clean":"2024-03-26T17:37:27.736915+0000","last_became_active":"2024-03-26T17:36:21.721387+0000","last_became_peered":"2024-03-26T17:36:21.721387+0000","last_unstale":"2024-03-26T17:37:27.736915+0000","last_undegraded":"2024-03-26T17:37:27.736915+0000","last_fullsized":"2024-03-26T17:37:27.736915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:59.493172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740779+0000","last_change":"2024-03-26T17:37:27.740779+0000","last_active":"2024-03-26T17:37:27.740779+0000","last_peered":"2024-03-26T17:37:27.740779+0000","last_clean":"2024-03-26T17:37:27.740779+0000","last_became_active":"2024-03-26T17:37:27.740407+0000","last_became_peered":"2024-03-26T17:37:27.740407+0000","last_unstale":"2024-03-26T17:37:27.740779+0000","last_undegraded":"2024-03-26T17:37:27.740779+0000","last_fullsized":"2024-03-26T17:37:27.740779+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:39.526570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744477+0000","last_change":"2024-03-26T17:36:21.724041+0000","last_active":"2024-03-26T17:37:27.744477+0000","last_peered":"2024-03-26T17:37:27.744477+0000","last_clean":"2024-03-26T17:37:27.744477+0000","last_became_active":"2024-03-26T17:36:21.723624+0000","last_became_peered":"2024-03-26T17:36:21.723624+0000","last_unstale":"2024-03-26T17:37:27.744477+0000","last_undegraded":"2024-03-26T17:37:27.744477+0000","last_fullsized":"2024-03-26T17:37:27.744477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:42:24.264844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244229+0000","last_change":"2024-03-26T17:36:22.738592+0000","last_active":"2024-03-26T17:37:28.244229+0000","last_peered":"2024-03-26T17:37:28.244229+0000","last_clean":"2024-03-26T17:37:28.244229+0000","last_became_active":"2024-03-26T17:36:22.738236+0000","last_became_peered":"2024-03-26T17:36:22.738236+0000","last_unstale":"2024-03-26T17:37:28.244229+0000","last_undegraded":"2024-03-26T17:37:28.244229+0000","last_fullsized":"2024-03-26T17:37:28.244229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:56:06.977720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":195,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:36.380182+0000","last_change":"2024-03-26T17:36:21.722793+0000","last_active":"2024-03-26T17:37:36.380182+0000","last_peered":"2024-03-26T17:37:36.380182+0000","last_clean":"2024-03-26T17:37:36.380182+0000","last_became_active":"2024-03-26T17:36:21.722482+0000","last_became_peered":"2024-03-26T17:36:21.722482+0000","last_unstale":"2024-03-26T17:37:36.380182+0000","last_undegraded":"2024-03-26T17:37:36.380182+0000","last_fullsized":"2024-03-26T17:37:36.380182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:48:32.107017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":146,"num_read_kb":133,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739539+0000","last_change":"2024-03-26T17:36:22.746488+0000","last_active":"2024-03-26T17:37:27.739539+0000","last_peered":"2024-03-26T17:37:27.739539+0000","last_clean":"2024-03-26T17:37:27.739539+0000","last_became_active":"2024-03-26T17:36:22.746013+0000","last_became_peered":"2024-03-26T17:36:22.746013+0000","last_unstale":"2024-03-26T17:37:27.739539+0000","last_undegraded":"2024-03-26T17:37:27.739539+0000","last_fullsized":"2024-03-26T17:37:27.739539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:14.792336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742843+0000","last_change":"2024-03-26T17:36:21.724319+0000","last_active":"2024-03-26T17:37:27.742843+0000","last_peered":"2024-03-26T17:37:27.742843+0000","last_clean":"2024-03-26T17:37:27.742843+0000","last_became_active":"2024-03-26T17:36:21.723644+0000","last_became_peered":"2024-03-26T17:36:21.723644+0000","last_unstale":"2024-03-26T17:37:27.742843+0000","last_undegraded":"2024-03-26T17:37:27.742843+0000","last_fullsized":"2024-03-26T17:37:27.742843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:31:01.132797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739828+0000","last_change":"2024-03-26T17:36:22.736535+0000","last_active":"2024-03-26T17:37:27.739828+0000","last_peered":"2024-03-26T17:37:27.739828+0000","last_clean":"2024-03-26T17:37:27.739828+0000","last_became_active":"2024-03-26T17:36:22.736229+0000","last_became_peered":"2024-03-26T17:36:22.736229+0000","last_unstale":"2024-03-26T17:37:27.739828+0000","last_undegraded":"2024-03-26T17:37:27.739828+0000","last_fullsized":"2024-03-26T17:37:27.739828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:03.938180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742451+0000","last_change":"2024-03-26T17:36:21.726271+0000","last_active":"2024-03-26T17:37:27.742451+0000","last_peered":"2024-03-26T17:37:27.742451+0000","last_clean":"2024-03-26T17:37:27.742451+0000","last_became_active":"2024-03-26T17:36:21.725942+0000","last_became_peered":"2024-03-26T17:36:21.725942+0000","last_unstale":"2024-03-26T17:37:27.742451+0000","last_undegraded":"2024-03-26T17:37:27.742451+0000","last_fullsized":"2024-03-26T17:37:27.742451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:23:16.320106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736981+0000","last_change":"2024-03-26T17:36:22.743480+0000","last_active":"2024-03-26T17:37:27.736981+0000","last_peered":"2024-03-26T17:37:27.736981+0000","last_clean":"2024-03-26T17:37:27.736981+0000","last_became_active":"2024-03-26T17:36:22.742982+0000","last_became_peered":"2024-03-26T17:36:22.742982+0000","last_unstale":"2024-03-26T17:37:27.736981+0000","last_undegraded":"2024-03-26T17:37:27.736981+0000","last_fullsized":"2024-03-26T17:37:27.736981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:47.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:38.383828+0000","last_change":"2024-03-26T17:36:21.723407+0000","last_active":"2024-03-26T17:37:38.383828+0000","last_peered":"2024-03-26T17:37:38.383828+0000","last_clean":"2024-03-26T17:37:38.383828+0000","last_became_active":"2024-03-26T17:36:21.723086+0000","last_became_peered":"2024-03-26T17:36:21.723086+0000","last_unstale":"2024-03-26T17:37:38.383828+0000","last_undegraded":"2024-03-26T17:37:38.383828+0000","last_fullsized":"2024-03-26T17:37:38.383828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:26:57.886388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736567+0000","last_change":"2024-03-26T17:36:22.740705+0000","last_active":"2024-03-26T17:37:27.736567+0000","last_peered":"2024-03-26T17:37:27.736567+0000","last_clean":"2024-03-26T17:37:27.736567+0000","last_became_active":"2024-03-26T17:36:22.740360+0000","last_became_peered":"2024-03-26T17:36:22.740360+0000","last_unstale":"2024-03-26T17:37:27.736567+0000","last_undegraded":"2024-03-26T17:37:27.736567+0000","last_fullsized":"2024-03-26T17:37:27.736567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:02.879022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.266750+0000","last_change":"2024-03-26T17:37:28.266399+0000","last_active":"2024-03-26T17:37:28.266750+0000","last_peered":"2024-03-26T17:37:28.266750+0000","last_clean":"2024-03-26T17:37:28.266750+0000","last_became_active":"2024-03-26T17:37:28.246359+0000","last_became_peered":"2024-03-26T17:37:28.246359+0000","last_unstale":"2024-03-26T17:37:28.266750+0000","last_undegraded":"2024-03-26T17:37:28.266750+0000","last_fullsized":"2024-03-26T17:37:28.266750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:20:54.895081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742779+0000","last_change":"2024-03-26T17:36:22.743791+0000","last_active":"2024-03-26T17:37:27.742779+0000","last_peered":"2024-03-26T17:37:27.742779+0000","last_clean":"2024-03-26T17:37:27.742779+0000","last_became_active":"2024-03-26T17:36:22.742780+0000","last_became_peered":"2024-03-26T17:36:22.742780+0000","last_unstale":"2024-03-26T17:37:27.742779+0000","last_undegraded":"2024-03-26T17:37:27.742779+0000","last_fullsized":"2024-03-26T17:37:27.742779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:04.387958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740518+0000","last_change":"2024-03-26T17:36:21.724289+0000","last_active":"2024-03-26T17:37:27.740518+0000","last_peered":"2024-03-26T17:37:27.740518+0000","last_clean":"2024-03-26T17:37:27.740518+0000","last_became_active":"2024-03-26T17:36:21.723847+0000","last_became_peered":"2024-03-26T17:36:21.723847+0000","last_unstale":"2024-03-26T17:37:27.740518+0000","last_undegraded":"2024-03-26T17:37:27.740518+0000","last_fullsized":"2024-03-26T17:37:27.740518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:01:59.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738117+0000","last_change":"2024-03-26T17:36:22.748323+0000","last_active":"2024-03-26T17:37:27.738117+0000","last_peered":"2024-03-26T17:37:27.738117+0000","last_clean":"2024-03-26T17:37:27.738117+0000","last_became_active":"2024-03-26T17:36:22.747970+0000","last_became_peered":"2024-03-26T17:36:22.747970+0000","last_unstale":"2024-03-26T17:37:27.738117+0000","last_undegraded":"2024-03-26T17:37:27.738117+0000","last_fullsized":"2024-03-26T17:37:27.738117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:58.886425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739292+0000","last_change":"2024-03-26T17:36:21.726076+0000","last_active":"2024-03-26T17:37:27.739292+0000","last_peered":"2024-03-26T17:37:27.739292+0000","last_clean":"2024-03-26T17:37:27.739292+0000","last_became_active":"2024-03-26T17:36:21.722067+0000","last_became_peered":"2024-03-26T17:36:21.722067+0000","last_unstale":"2024-03-26T17:37:27.739292+0000","last_undegraded":"2024-03-26T17:37:27.739292+0000","last_fullsized":"2024-03-26T17:37:27.739292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:34:40.316110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.243853+0000","last_change":"2024-03-26T17:36:22.730890+0000","last_active":"2024-03-26T17:37:28.243853+0000","last_peered":"2024-03-26T17:37:28.243853+0000","last_clean":"2024-03-26T17:37:28.243853+0000","last_became_active":"2024-03-26T17:36:22.729635+0000","last_became_peered":"2024-03-26T17:36:22.729635+0000","last_unstale":"2024-03-26T17:37:28.243853+0000","last_undegraded":"2024-03-26T17:37:28.243853+0000","last_fullsized":"2024-03-26T17:37:28.243853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:15:09.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:36.384117+0000","last_change":"2024-03-26T17:36:21.727587+0000","last_active":"2024-03-26T17:37:36.384117+0000","last_peered":"2024-03-26T17:37:36.384117+0000","last_clean":"2024-03-26T17:37:36.384117+0000","last_became_active":"2024-03-26T17:36:21.727302+0000","last_became_peered":"2024-03-26T17:36:21.727302+0000","last_unstale":"2024-03-26T17:37:36.384117+0000","last_undegraded":"2024-03-26T17:37:36.384117+0000","last_fullsized":"2024-03-26T17:37:36.384117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:59.404567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245133+0000","last_change":"2024-03-26T17:36:22.750654+0000","last_active":"2024-03-26T17:37:28.245133+0000","last_peered":"2024-03-26T17:37:28.245133+0000","last_clean":"2024-03-26T17:37:28.245133+0000","last_became_active":"2024-03-26T17:36:22.750281+0000","last_became_peered":"2024-03-26T17:36:22.750281+0000","last_unstale":"2024-03-26T17:37:28.245133+0000","last_undegraded":"2024-03-26T17:37:28.245133+0000","last_fullsized":"2024-03-26T17:37:28.245133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:04.034108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.745100+0000","last_change":"2024-03-26T17:37:27.745100+0000","last_active":"2024-03-26T17:37:27.745100+0000","last_peered":"2024-03-26T17:37:27.745100+0000","last_clean":"2024-03-26T17:37:27.745100+0000","last_became_active":"2024-03-26T17:37:27.744686+0000","last_became_peered":"2024-03-26T17:37:27.744686+0000","last_unstale":"2024-03-26T17:37:27.745100+0000","last_undegraded":"2024-03-26T17:37:27.745100+0000","last_fullsized":"2024-03-26T17:37:27.745100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:38.111145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:36.381698+0000","last_change":"2024-03-26T17:36:21.730812+0000","last_active":"2024-03-26T17:37:36.381698+0000","last_peered":"2024-03-26T17:37:36.381698+0000","last_clean":"2024-03-26T17:37:36.381698+0000","last_became_active":"2024-03-26T17:36:21.730499+0000","last_became_peered":"2024-03-26T17:36:21.730499+0000","last_unstale":"2024-03-26T17:37:36.381698+0000","last_undegraded":"2024-03-26T17:37:36.381698+0000","last_fullsized":"2024-03-26T17:37:36.381698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:05.666146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741485+0000","last_change":"2024-03-26T17:36:22.742344+0000","last_active":"2024-03-26T17:37:27.741485+0000","last_peered":"2024-03-26T17:37:27.741485+0000","last_clean":"2024-03-26T17:37:27.741485+0000","last_became_active":"2024-03-26T17:36:22.741864+0000","last_became_peered":"2024-03-26T17:36:22.741864+0000","last_unstale":"2024-03-26T17:37:27.741485+0000","last_undegraded":"2024-03-26T17:37:27.741485+0000","last_fullsized":"2024-03-26T17:37:27.741485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:53:53.553107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244456+0000","last_change":"2024-03-26T17:36:21.722450+0000","last_active":"2024-03-26T17:37:28.244456+0000","last_peered":"2024-03-26T17:37:28.244456+0000","last_clean":"2024-03-26T17:37:28.244456+0000","last_became_active":"2024-03-26T17:36:21.722105+0000","last_became_peered":"2024-03-26T17:36:21.722105+0000","last_unstale":"2024-03-26T17:37:28.244456+0000","last_undegraded":"2024-03-26T17:37:28.244456+0000","last_fullsized":"2024-03-26T17:37:28.244456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:01:22.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736637+0000","last_change":"2024-03-26T17:35:10.663606+0000","last_active":"2024-03-26T17:37:27.736637+0000","last_peered":"2024-03-26T17:37:27.736637+0000","last_clean":"2024-03-26T17:37:27.736637+0000","last_became_active":"2024-03-26T17:35:10.663069+0000","last_became_peered":"2024-03-26T17:35:10.663069+0000","last_unstale":"2024-03-26T17:37:27.736637+0000","last_undegraded":"2024-03-26T17:37:27.736637+0000","last_fullsized":"2024-03-26T17:37:27.736637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_clean_scrub_stamp":"2024-03-26T17:31:50.880228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:50.817007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.745201+0000","last_change":"2024-03-26T17:37:27.745201+0000","last_active":"2024-03-26T17:37:27.745201+0000","last_peered":"2024-03-26T17:37:27.745201+0000","last_clean":"2024-03-26T17:37:27.745201+0000","last_became_active":"2024-03-26T17:37:27.744806+0000","last_became_peered":"2024-03-26T17:37:27.744806+0000","last_unstale":"2024-03-26T17:37:27.745201+0000","last_undegraded":"2024-03-26T17:37:27.745201+0000","last_fullsized":"2024-03-26T17:37:27.745201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:15:07.485013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736749+0000","last_change":"2024-03-26T17:36:21.723452+0000","last_active":"2024-03-26T17:37:27.736749+0000","last_peered":"2024-03-26T17:37:27.736749+0000","last_clean":"2024-03-26T17:37:27.736749+0000","last_became_active":"2024-03-26T17:36:21.723155+0000","last_became_peered":"2024-03-26T17:36:21.723155+0000","last_unstale":"2024-03-26T17:37:27.736749+0000","last_undegraded":"2024-03-26T17:37:27.736749+0000","last_fullsized":"2024-03-26T17:37:27.736749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:39:34.443624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.750092+0000","last_change":"2024-03-26T17:37:27.749741+0000","last_active":"2024-03-26T17:37:27.750092+0000","last_peered":"2024-03-26T17:37:27.750092+0000","last_clean":"2024-03-26T17:37:27.750092+0000","last_became_active":"2024-03-26T17:37:27.741751+0000","last_became_peered":"2024-03-26T17:37:27.741751+0000","last_unstale":"2024-03-26T17:37:27.750092+0000","last_undegraded":"2024-03-26T17:37:27.750092+0000","last_fullsized":"2024-03-26T17:37:27.750092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:42.119297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739984+0000","last_change":"2024-03-26T17:36:22.736487+0000","last_active":"2024-03-26T17:37:27.739984+0000","last_peered":"2024-03-26T17:37:27.739984+0000","last_clean":"2024-03-26T17:37:27.739984+0000","last_became_active":"2024-03-26T17:36:22.735788+0000","last_became_peered":"2024-03-26T17:36:22.735788+0000","last_unstale":"2024-03-26T17:37:27.739984+0000","last_undegraded":"2024-03-26T17:37:27.739984+0000","last_fullsized":"2024-03-26T17:37:27.739984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:45.240148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739464+0000","last_change":"2024-03-26T17:36:21.720755+0000","last_active":"2024-03-26T17:37:27.739464+0000","last_peered":"2024-03-26T17:37:27.739464+0000","last_clean":"2024-03-26T17:37:27.739464+0000","last_became_active":"2024-03-26T17:36:21.720175+0000","last_became_peered":"2024-03-26T17:36:21.720175+0000","last_unstale":"2024-03-26T17:37:27.739464+0000","last_undegraded":"2024-03-26T17:37:27.739464+0000","last_fullsized":"2024-03-26T17:37:27.739464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:22.458657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742189+0000","last_change":"2024-03-26T17:36:22.740731+0000","last_active":"2024-03-26T17:37:27.742189+0000","last_peered":"2024-03-26T17:37:27.742189+0000","last_clean":"2024-03-26T17:37:27.742189+0000","last_became_active":"2024-03-26T17:36:22.740283+0000","last_became_peered":"2024-03-26T17:36:22.740283+0000","last_unstale":"2024-03-26T17:37:27.742189+0000","last_undegraded":"2024-03-26T17:37:27.742189+0000","last_fullsized":"2024-03-26T17:37:27.742189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:36.403105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244032+0000","last_change":"2024-03-26T17:36:21.721513+0000","last_active":"2024-03-26T17:37:28.244032+0000","last_peered":"2024-03-26T17:37:28.244032+0000","last_clean":"2024-03-26T17:37:28.244032+0000","last_became_active":"2024-03-26T17:36:21.721185+0000","last_became_peered":"2024-03-26T17:36:21.721185+0000","last_unstale":"2024-03-26T17:37:28.244032+0000","last_undegraded":"2024-03-26T17:37:28.244032+0000","last_fullsized":"2024-03-26T17:37:28.244032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:48:39.994980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736856+0000","last_change":"2024-03-26T17:36:22.737938+0000","last_active":"2024-03-26T17:37:27.736856+0000","last_peered":"2024-03-26T17:37:27.736856+0000","last_clean":"2024-03-26T17:37:27.736856+0000","last_became_active":"2024-03-26T17:36:22.737605+0000","last_became_peered":"2024-03-26T17:36:22.737605+0000","last_unstale":"2024-03-26T17:37:27.736856+0000","last_undegraded":"2024-03-26T17:37:27.736856+0000","last_fullsized":"2024-03-26T17:37:27.736856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:27.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245677+0000","last_change":"2024-03-26T17:36:21.720942+0000","last_active":"2024-03-26T17:37:28.245677+0000","last_peered":"2024-03-26T17:37:28.245677+0000","last_clean":"2024-03-26T17:37:28.245677+0000","last_became_active":"2024-03-26T17:36:21.720665+0000","last_became_peered":"2024-03-26T17:36:21.720665+0000","last_unstale":"2024-03-26T17:37:28.245677+0000","last_undegraded":"2024-03-26T17:37:28.245677+0000","last_fullsized":"2024-03-26T17:37:28.245677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:00.725213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739390+0000","last_change":"2024-03-26T17:36:22.747111+0000","last_active":"2024-03-26T17:37:27.739390+0000","last_peered":"2024-03-26T17:37:27.739390+0000","last_clean":"2024-03-26T17:37:27.739390+0000","last_became_active":"2024-03-26T17:36:22.746626+0000","last_became_peered":"2024-03-26T17:36:22.746626+0000","last_unstale":"2024-03-26T17:37:27.739390+0000","last_undegraded":"2024-03-26T17:37:27.739390+0000","last_fullsized":"2024-03-26T17:37:27.739390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:29:24.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741395+0000","last_change":"2024-03-26T17:36:21.724141+0000","last_active":"2024-03-26T17:37:27.741395+0000","last_peered":"2024-03-26T17:37:27.741395+0000","last_clean":"2024-03-26T17:37:27.741395+0000","last_became_active":"2024-03-26T17:36:21.723797+0000","last_became_peered":"2024-03-26T17:36:21.723797+0000","last_unstale":"2024-03-26T17:37:27.741395+0000","last_undegraded":"2024-03-26T17:37:27.741395+0000","last_fullsized":"2024-03-26T17:37:27.741395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:06.149827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738751+0000","last_change":"2024-03-26T17:36:22.739497+0000","last_active":"2024-03-26T17:37:27.738751+0000","last_peered":"2024-03-26T17:37:27.738751+0000","last_clean":"2024-03-26T17:37:27.738751+0000","last_became_active":"2024-03-26T17:36:22.739114+0000","last_became_peered":"2024-03-26T17:36:22.739114+0000","last_unstale":"2024-03-26T17:37:27.738751+0000","last_undegraded":"2024-03-26T17:37:27.738751+0000","last_fullsized":"2024-03-26T17:37:27.738751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:03.953199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.743010+0000","last_change":"2024-03-26T17:36:21.728027+0000","last_active":"2024-03-26T17:37:27.743010+0000","last_peered":"2024-03-26T17:37:27.743010+0000","last_clean":"2024-03-26T17:37:27.743010+0000","last_became_active":"2024-03-26T17:36:21.727159+0000","last_became_peered":"2024-03-26T17:36:21.727159+0000","last_unstale":"2024-03-26T17:37:27.743010+0000","last_undegraded":"2024-03-26T17:37:27.743010+0000","last_fullsized":"2024-03-26T17:37:27.743010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:39.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741715+0000","last_change":"2024-03-26T17:36:22.733666+0000","last_active":"2024-03-26T17:37:27.741715+0000","last_peered":"2024-03-26T17:37:27.741715+0000","last_clean":"2024-03-26T17:37:27.741715+0000","last_became_active":"2024-03-26T17:36:22.733313+0000","last_became_peered":"2024-03-26T17:36:22.733313+0000","last_unstale":"2024-03-26T17:37:27.741715+0000","last_undegraded":"2024-03-26T17:37:27.741715+0000","last_fullsized":"2024-03-26T17:37:27.741715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:52:38.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739591+0000","last_change":"2024-03-26T17:36:21.730828+0000","last_active":"2024-03-26T17:37:27.739591+0000","last_peered":"2024-03-26T17:37:27.739591+0000","last_clean":"2024-03-26T17:37:27.739591+0000","last_became_active":"2024-03-26T17:36:21.730508+0000","last_became_peered":"2024-03-26T17:36:21.730508+0000","last_unstale":"2024-03-26T17:37:27.739591+0000","last_undegraded":"2024-03-26T17:37:27.739591+0000","last_fullsized":"2024-03-26T17:37:27.739591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:10:53.492725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736879+0000","last_change":"2024-03-26T17:36:22.740204+0000","last_active":"2024-03-26T17:37:27.736879+0000","last_peered":"2024-03-26T17:37:27.736879+0000","last_clean":"2024-03-26T17:37:27.736879+0000","last_became_active":"2024-03-26T17:36:22.735880+0000","last_became_peered":"2024-03-26T17:36:22.735880+0000","last_unstale":"2024-03-26T17:37:27.736879+0000","last_undegraded":"2024-03-26T17:37:27.736879+0000","last_fullsized":"2024-03-26T17:37:27.736879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:34.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:34.378569+0000","last_change":"2024-03-26T17:36:21.720873+0000","last_active":"2024-03-26T17:37:34.378569+0000","last_peered":"2024-03-26T17:37:34.378569+0000","last_clean":"2024-03-26T17:37:34.378569+0000","last_became_active":"2024-03-26T17:36:21.720496+0000","last_became_peered":"2024-03-26T17:36:21.720496+0000","last_unstale":"2024-03-26T17:37:34.378569+0000","last_undegraded":"2024-03-26T17:37:34.378569+0000","last_fullsized":"2024-03-26T17:37:34.378569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:40:55.959064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.737006+0000","last_change":"2024-03-26T17:36:22.735528+0000","last_active":"2024-03-26T17:37:27.737006+0000","last_peered":"2024-03-26T17:37:27.737006+0000","last_clean":"2024-03-26T17:37:27.737006+0000","last_became_active":"2024-03-26T17:36:22.735147+0000","last_became_peered":"2024-03-26T17:36:22.735147+0000","last_unstale":"2024-03-26T17:37:27.737006+0000","last_undegraded":"2024-03-26T17:37:27.737006+0000","last_fullsized":"2024-03-26T17:37:27.737006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:15.729853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744355+0000","last_change":"2024-03-26T17:36:21.723246+0000","last_active":"2024-03-26T17:37:27.744355+0000","last_peered":"2024-03-26T17:37:27.744355+0000","last_clean":"2024-03-26T17:37:27.744355+0000","last_became_active":"2024-03-26T17:36:21.722839+0000","last_became_peered":"2024-03-26T17:36:21.722839+0000","last_unstale":"2024-03-26T17:37:27.744355+0000","last_undegraded":"2024-03-26T17:37:27.744355+0000","last_fullsized":"2024-03-26T17:37:27.744355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:41:05.540206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736713+0000","last_change":"2024-03-26T17:36:22.742469+0000","last_active":"2024-03-26T17:37:27.736713+0000","last_peered":"2024-03-26T17:37:27.736713+0000","last_clean":"2024-03-26T17:37:27.736713+0000","last_became_active":"2024-03-26T17:36:22.742073+0000","last_became_peered":"2024-03-26T17:36:22.742073+0000","last_unstale":"2024-03-26T17:37:27.736713+0000","last_undegraded":"2024-03-26T17:37:27.736713+0000","last_fullsized":"2024-03-26T17:37:27.736713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:19:33.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.250123+0000","last_change":"2024-03-26T17:37:28.250123+0000","last_active":"2024-03-26T17:37:28.250123+0000","last_peered":"2024-03-26T17:37:28.250123+0000","last_clean":"2024-03-26T17:37:28.250123+0000","last_became_active":"2024-03-26T17:37:28.246162+0000","last_became_peered":"2024-03-26T17:37:28.246162+0000","last_unstale":"2024-03-26T17:37:28.250123+0000","last_undegraded":"2024-03-26T17:37:28.250123+0000","last_fullsized":"2024-03-26T17:37:28.250123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:33:39.002192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.737009+0000","last_change":"2024-03-26T17:36:22.742614+0000","last_active":"2024-03-26T17:37:27.737009+0000","last_peered":"2024-03-26T17:37:27.737009+0000","last_clean":"2024-03-26T17:37:27.737009+0000","last_became_active":"2024-03-26T17:36:22.742278+0000","last_became_peered":"2024-03-26T17:36:22.742278+0000","last_unstale":"2024-03-26T17:37:27.737009+0000","last_undegraded":"2024-03-26T17:37:27.737009+0000","last_fullsized":"2024-03-26T17:37:27.737009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:30:14.503474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738850+0000","last_change":"2024-03-26T17:36:22.741693+0000","last_active":"2024-03-26T17:37:27.738850+0000","last_peered":"2024-03-26T17:37:27.738850+0000","last_clean":"2024-03-26T17:37:27.738850+0000","last_became_active":"2024-03-26T17:36:22.741352+0000","last_became_peered":"2024-03-26T17:36:22.741352+0000","last_unstale":"2024-03-26T17:37:27.738850+0000","last_undegraded":"2024-03-26T17:37:27.738850+0000","last_fullsized":"2024-03-26T17:37:27.738850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:13:26.912910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742407+0000","last_change":"2024-03-26T17:36:22.742939+0000","last_active":"2024-03-26T17:37:27.742407+0000","last_peered":"2024-03-26T17:37:27.742407+0000","last_clean":"2024-03-26T17:37:27.742407+0000","last_became_active":"2024-03-26T17:36:22.742589+0000","last_became_peered":"2024-03-26T17:36:22.742589+0000","last_unstale":"2024-03-26T17:37:27.742407+0000","last_undegraded":"2024-03-26T17:37:27.742407+0000","last_fullsized":"2024-03-26T17:37:27.742407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:21:14.043601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741596+0000","last_change":"2024-03-26T17:36:22.748044+0000","last_active":"2024-03-26T17:37:27.741596+0000","last_peered":"2024-03-26T17:37:27.741596+0000","last_clean":"2024-03-26T17:37:27.741596+0000","last_became_active":"2024-03-26T17:36:22.747683+0000","last_became_peered":"2024-03-26T17:36:22.747683+0000","last_unstale":"2024-03-26T17:37:27.741596+0000","last_undegraded":"2024-03-26T17:37:27.741596+0000","last_fullsized":"2024-03-26T17:37:27.741596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:32:40.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736592+0000","last_change":"2024-03-26T17:36:22.743080+0000","last_active":"2024-03-26T17:37:27.736592+0000","last_peered":"2024-03-26T17:37:27.736592+0000","last_clean":"2024-03-26T17:37:27.736592+0000","last_became_active":"2024-03-26T17:36:22.742676+0000","last_became_peered":"2024-03-26T17:36:22.742676+0000","last_unstale":"2024-03-26T17:37:27.736592+0000","last_undegraded":"2024-03-26T17:37:27.736592+0000","last_fullsized":"2024-03-26T17:37:27.736592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:58.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739891+0000","last_change":"2024-03-26T17:36:22.737908+0000","last_active":"2024-03-26T17:37:27.739891+0000","last_peered":"2024-03-26T17:37:27.739891+0000","last_clean":"2024-03-26T17:37:27.739891+0000","last_became_active":"2024-03-26T17:36:22.737055+0000","last_became_peered":"2024-03-26T17:36:22.737055+0000","last_unstale":"2024-03-26T17:37:27.739891+0000","last_undegraded":"2024-03-26T17:37:27.739891+0000","last_fullsized":"2024-03-26T17:37:27.739891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:51:13.157901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.738201+0000","last_change":"2024-03-26T17:36:22.749795+0000","last_active":"2024-03-26T17:37:27.738201+0000","last_peered":"2024-03-26T17:37:27.738201+0000","last_clean":"2024-03-26T17:37:27.738201+0000","last_became_active":"2024-03-26T17:36:22.749469+0000","last_became_peered":"2024-03-26T17:36:22.749469+0000","last_unstale":"2024-03-26T17:37:27.738201+0000","last_undegraded":"2024-03-26T17:37:27.738201+0000","last_fullsized":"2024-03-26T17:37:27.738201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:19:59.126270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.743272+0000","last_change":"2024-03-26T17:36:22.744101+0000","last_active":"2024-03-26T17:37:27.743272+0000","last_peered":"2024-03-26T17:37:27.743272+0000","last_clean":"2024-03-26T17:37:27.743272+0000","last_became_active":"2024-03-26T17:36:22.743761+0000","last_became_peered":"2024-03-26T17:36:22.743761+0000","last_unstale":"2024-03-26T17:37:27.743272+0000","last_undegraded":"2024-03-26T17:37:27.743272+0000","last_fullsized":"2024-03-26T17:37:27.743272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:27:19.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742102+0000","last_change":"2024-03-26T17:36:22.744800+0000","last_active":"2024-03-26T17:37:27.742102+0000","last_peered":"2024-03-26T17:37:27.742102+0000","last_clean":"2024-03-26T17:37:27.742102+0000","last_became_active":"2024-03-26T17:36:22.744437+0000","last_became_peered":"2024-03-26T17:36:22.744437+0000","last_unstale":"2024-03-26T17:37:27.742102+0000","last_undegraded":"2024-03-26T17:37:27.742102+0000","last_fullsized":"2024-03-26T17:37:27.742102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:36.459072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.745243+0000","last_change":"2024-03-26T17:37:27.745243+0000","last_active":"2024-03-26T17:37:27.745243+0000","last_peered":"2024-03-26T17:37:27.745243+0000","last_clean":"2024-03-26T17:37:27.745243+0000","last_became_active":"2024-03-26T17:37:27.744869+0000","last_became_peered":"2024-03-26T17:37:27.744869+0000","last_unstale":"2024-03-26T17:37:27.745243+0000","last_undegraded":"2024-03-26T17:37:27.745243+0000","last_fullsized":"2024-03-26T17:37:27.745243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:18:42.188733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741437+0000","last_change":"2024-03-26T17:36:22.743211+0000","last_active":"2024-03-26T17:37:27.741437+0000","last_peered":"2024-03-26T17:37:27.741437+0000","last_clean":"2024-03-26T17:37:27.741437+0000","last_became_active":"2024-03-26T17:36:22.742869+0000","last_became_peered":"2024-03-26T17:36:22.742869+0000","last_unstale":"2024-03-26T17:37:27.741437+0000","last_undegraded":"2024-03-26T17:37:27.741437+0000","last_fullsized":"2024-03-26T17:37:27.741437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:24:04.610012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.243871+0000","last_change":"2024-03-26T17:36:22.746950+0000","last_active":"2024-03-26T17:37:28.243871+0000","last_peered":"2024-03-26T17:37:28.243871+0000","last_clean":"2024-03-26T17:37:28.243871+0000","last_became_active":"2024-03-26T17:36:22.744869+0000","last_became_peered":"2024-03-26T17:36:22.744869+0000","last_unstale":"2024-03-26T17:37:28.243871+0000","last_undegraded":"2024-03-26T17:37:28.243871+0000","last_fullsized":"2024-03-26T17:37:28.243871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:09:52.283519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740309+0000","last_change":"2024-03-26T17:36:22.744627+0000","last_active":"2024-03-26T17:37:27.740309+0000","last_peered":"2024-03-26T17:37:27.740309+0000","last_clean":"2024-03-26T17:37:27.740309+0000","last_became_active":"2024-03-26T17:36:22.744299+0000","last_became_peered":"2024-03-26T17:36:22.744299+0000","last_unstale":"2024-03-26T17:37:27.740309+0000","last_undegraded":"2024-03-26T17:37:27.740309+0000","last_fullsized":"2024-03-26T17:37:27.740309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:23:37.941625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.737942+0000","last_change":"2024-03-26T17:36:22.751135+0000","last_active":"2024-03-26T17:37:27.737942+0000","last_peered":"2024-03-26T17:37:27.737942+0000","last_clean":"2024-03-26T17:37:27.737942+0000","last_became_active":"2024-03-26T17:36:22.750768+0000","last_became_peered":"2024-03-26T17:36:22.750768+0000","last_unstale":"2024-03-26T17:37:27.737942+0000","last_undegraded":"2024-03-26T17:37:27.737942+0000","last_fullsized":"2024-03-26T17:37:27.737942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:15.532367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67859,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":732,"num_read_kb":438,"num_write":236,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":203577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383710,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":456929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645344,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1108389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712440,"statfs":{"total":95995035648,"available":95961538560,"internally_reserved":0,"allocated":1568768,"data_stored":1048749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":455961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":518032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":457691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":518419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":455786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1581056,"data_stored":1048427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T17:37:41.304 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-26T17:37:41.304 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-26T17:37:41.304 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-26T17:37:41.305 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-26T17:37:41.588 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:42.053 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:41 smithi105 ceph-mon[32280]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 18 B/s, 1 objects/s recovering 2024-03-26T17:37:42.053 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:41 smithi105 ceph-mon[32280]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:37:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:41 smithi144 ceph-mon[36095]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 18 B/s, 1 objects/s recovering 2024-03-26T17:37:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:41 smithi144 ceph-mon[36095]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:37:42.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:41 smithi135 ceph-mon[35636]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 18 B/s, 1 objects/s recovering 2024-03-26T17:37:42.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:41 smithi135 ceph-mon[35636]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:37:43.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 -- 172.21.15.105:0/3207400184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f397006ab50 msgr2=0x7f397006af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:43.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 --2- 172.21.15.105:0/3207400184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f397006ab50 0x7f397006af30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f39600099c0 tx=0x7f3960009cd0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 -- 172.21.15.105:0/3207400184 shutdown_connections 2024-03-26T17:37:43.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 --2- 172.21.15.105:0/3207400184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3970107690 0x7f397010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 --2- 172.21.15.105:0/3207400184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f397006ab50 0x7f397006af30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 --2- 172.21.15.105:0/3207400184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397006b500 0x7f3970106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 -- 172.21.15.105:0/3207400184 >> 172.21.15.105:0/3207400184 conn(0x7f3970077300 msgr2=0x7f3970077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:43.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 -- 172.21.15.105:0/3207400184 shutdown_connections 2024-03-26T17:37:43.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.175+0000 7f39768b9700 1 -- 172.21.15.105:0/3207400184 wait complete. 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.176+0000 7f39768b9700 1 Processor -- start 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.176+0000 7f39768b9700 1 -- start start 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f39768b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397006ab50 0x7f397019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f39768b9700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f397006b500 0x7f397019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f39768b9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3970107690 0x7f3970198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f39768b9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3970112540 con 0x7f3970107690 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f39768b9700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f39701123c0 con 0x7f397006b500 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f39768b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39701126c0 con 0x7f397006ab50 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397006ab50 0x7f397019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:43.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f3974e56700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3970107690 0x7f3970198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397006ab50 0x7f397019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51814/0 (socket says 172.21.15.105:51814) 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f3974e56700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3970107690 0x7f3970198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38696/0 (socket says 172.21.15.105:38696) 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 -- 172.21.15.105:0/896161185 learned_addr learned my addr 172.21.15.105:0/896161185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396f7fe700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f397006b500 0x7f397019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 -- 172.21.15.105:0/896161185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f397006b500 msgr2=0x7f397019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f397006b500 0x7f397019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 -- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3970107690 msgr2=0x7f3970198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3970107690 0x7f3970198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 -- 172.21.15.105:0/896161185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3970198fe0 con 0x7f397006ab50 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396ffff700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397006ab50 0x7f397019e6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3960006010 tx=0x7f3960004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396d7fa700 1 -- 172.21.15.105:0/896161185 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960026070 con 0x7f397006ab50 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396d7fa700 1 -- 172.21.15.105:0/896161185 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3960004630 con 0x7f397006ab50 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f396d7fa700 1 -- 172.21.15.105:0/896161185 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396000fb20 con 0x7f397006ab50 2024-03-26T17:37:43.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3970199270 con 0x7f397006ab50 2024-03-26T17:37:43.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.177+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39701a5590 con 0x7f397006ab50 2024-03-26T17:37:43.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.178+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3970069070 con 0x7f397006ab50 2024-03-26T17:37:43.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.180+0000 7f396d7fa700 1 -- 172.21.15.105:0/896161185 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f3960029070 con 0x7f397006ab50 2024-03-26T17:37:43.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.181+0000 7f396d7fa700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39580a30c0 0x7f39580a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:43.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.181+0000 7f396f7fe700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39580a30c0 0x7f39580a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:43.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.181+0000 7f396d7fa700 1 -- 172.21.15.105:0/896161185 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f39600d0350 con 0x7f397006ab50 2024-03-26T17:37:43.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.181+0000 7f396f7fe700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39580a30c0 0x7f39580a5580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3970101000 tx=0x7f3964009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:43.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.184+0000 7f396d7fa700 1 -- 172.21.15.105:0/896161185 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3960013070 con 0x7f397006ab50 2024-03-26T17:37:43.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.480+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f397002dc00 con 0x7f397006ab50 2024-03-26T17:37:43.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.481+0000 7f396d7fa700 1 -- 172.21.15.105:0/896161185 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f396009d070 con 0x7f397006ab50 2024-03-26T17:37:43.473 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:43.474 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-26T17:37:43.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39580a30c0 msgr2=0x7f39580a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39580a30c0 0x7f39580a5580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3970101000 tx=0x7f3964009260 comp rx=0 tx=0).stop 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397006ab50 msgr2=0x7f397019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397006ab50 0x7f397019e6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3960006010 tx=0x7f3960004270 comp rx=0 tx=0).stop 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 shutdown_connections 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39580a30c0 0x7f39580a5580 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3970107690 0x7f3970198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f397006ab50 0x7f397019e6c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 --2- 172.21.15.105:0/896161185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f397006b500 0x7f397019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 >> 172.21.15.105:0/896161185 conn(0x7f3970077300 msgr2=0x7f3970106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 shutdown_connections 2024-03-26T17:37:43.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:43.483+0000 7f39768b9700 1 -- 172.21.15.105:0/896161185 wait complete. 2024-03-26T17:37:44.023 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:43 smithi105 ceph-mon[32280]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T17:37:44.023 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:43 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/896161185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T17:37:44.050 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-26T17:37:44.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:37:44.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:43 smithi144 ceph-mon[36095]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T17:37:44.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:43 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/896161185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T17:37:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:43 smithi135 ceph-mon[35636]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T17:37:44.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:43 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/896161185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T17:37:44.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 -- 172.21.15.105:0/1956388161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d41023c0 msgr2=0x7ff6d41047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/1956388161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d41023c0 0x7ff6d41047b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009900 tx=0x7ff6c402bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 -- 172.21.15.105:0/1956388161 shutdown_connections 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/1956388161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d4107620 0x7ff6d4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/1956388161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d4104cf0 0x7ff6d41070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/1956388161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d41023c0 0x7ff6d41047b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 -- 172.21.15.105:0/1956388161 >> 172.21.15.105:0/1956388161 conn(0x7ff6d40fbd50 msgr2=0x7ff6d40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 -- 172.21.15.105:0/1956388161 shutdown_connections 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.178+0000 7ff6d8a19640 1 -- 172.21.15.105:0/1956388161 wait complete. 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d8a19640 1 Processor -- start 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d8a19640 1 -- start start 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d8a19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d41023c0 0x7ff6d4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d8a19640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d4104cf0 0x7ff6d41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d8a19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d4107620 0x7ff6d419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d8a19640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410d430 con 0x7ff6d4107620 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d8a19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410d2b0 con 0x7ff6d41023c0 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d8a19640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d410d5b0 con 0x7ff6d4104cf0 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d4107620 0x7ff6d419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d4107620 0x7ff6d419f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38710/0 (socket says 172.21.15.105:38710) 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 -- 172.21.15.105:0/852279807 learned_addr learned my addr 172.21.15.105:0/852279807 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6cbfff640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d4104cf0 0x7ff6d41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d259c640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d41023c0 0x7ff6d4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 -- 172.21.15.105:0/852279807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d4104cf0 msgr2=0x7ff6d41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d4104cf0 0x7ff6d41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 -- 172.21.15.105:0/852279807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d41023c0 msgr2=0x7ff6d4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d41023c0 0x7ff6d4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 -- 172.21.15.105:0/852279807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d419f780 con 0x7ff6d4107620 2024-03-26T17:37:44.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.179+0000 7ff6d2d9d640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d4107620 0x7ff6d419f110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00efd0 tx=0x7ff6cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:44.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.180+0000 7ff6cb7fe640 1 -- 172.21.15.105:0/852279807 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc003b70 con 0x7ff6d4107620 2024-03-26T17:37:44.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.180+0000 7ff6cb7fe640 1 -- 172.21.15.105:0/852279807 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6cc003d10 con 0x7ff6d4107620 2024-03-26T17:37:44.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.180+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d419fa70 con 0x7ff6d4107620 2024-03-26T17:37:44.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.180+0000 7ff6cb7fe640 1 -- 172.21.15.105:0/852279807 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc010980 con 0x7ff6d4107620 2024-03-26T17:37:44.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.180+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d419ff70 con 0x7ff6d4107620 2024-03-26T17:37:44.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.181+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d40fd840 con 0x7ff6d4107620 2024-03-26T17:37:44.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.182+0000 7ff6cb7fe640 1 -- 172.21.15.105:0/852279807 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7ff6cc004020 con 0x7ff6d4107620 2024-03-26T17:37:44.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.182+0000 7ff6cb7fe640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c00a2fb0 0x7ff6c00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.182+0000 7ff6d259c640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c00a2fb0 0x7ff6c00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.182+0000 7ff6cb7fe640 1 -- 172.21.15.105:0/852279807 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7ff6cc0cf970 con 0x7ff6d4107620 2024-03-26T17:37:44.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.183+0000 7ff6d259c640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c00a2fb0 0x7ff6c00a5470 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009230 tx=0x7ff6c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:44.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.187+0000 7ff6cb7fe640 1 -- 172.21.15.105:0/852279807 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff6cc09c660 con 0x7ff6d4107620 2024-03-26T17:37:44.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.369+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6d40624c0 con 0x7ff6d4107620 2024-03-26T17:37:44.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.370+0000 7ff6cb7fe640 1 -- 172.21.15.105:0/852279807 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7ff6cc09bf60 con 0x7ff6d4107620 2024-03-26T17:37:44.363 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:44.363 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c00a2fb0 msgr2=0x7ff6c00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c00a2fb0 0x7ff6c00a5470 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009230 tx=0x7ff6c4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d4107620 msgr2=0x7ff6d419f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d4107620 0x7ff6d419f110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00efd0 tx=0x7ff6cc00c6a0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 shutdown_connections 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c00a2fb0 0x7ff6c00a5470 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d4107620 0x7ff6d419f110 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d4104cf0 0x7ff6d41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 --2- 172.21.15.105:0/852279807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d41023c0 0x7ff6d4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 >> 172.21.15.105:0/852279807 conn(0x7ff6d40fbd50 msgr2=0x7ff6d4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:44.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 shutdown_connections 2024-03-26T17:37:44.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.372+0000 7ff6d8a19640 1 -- 172.21.15.105:0/852279807 wait complete. 2024-03-26T17:37:44.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:37:44.379 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:37:44.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 -- 172.21.15.105:0/3353739796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1019b0 msgr2=0x7f624c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 --2- 172.21.15.105:0/3353739796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1019b0 0x7f624c101e30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6240009900 tx=0x7f624002bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 -- 172.21.15.105:0/3353739796 shutdown_connections 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 --2- 172.21.15.105:0/3353739796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c102370 0x7f624c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 --2- 172.21.15.105:0/3353739796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1019b0 0x7f624c101e30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 --2- 172.21.15.105:0/3353739796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c1007c0 0x7f624c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 -- 172.21.15.105:0/3353739796 >> 172.21.15.105:0/3353739796 conn(0x7f624c0fbd50 msgr2=0x7f624c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 -- 172.21.15.105:0/3353739796 shutdown_connections 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.492+0000 7f6252710640 1 -- 172.21.15.105:0/3353739796 wait complete. 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6252710640 1 Processor -- start 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6252710640 1 -- start start 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6252710640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1007c0 0x7f624c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6252710640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c1019b0 0x7f624c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6252710640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c102370 0x7f624c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6252710640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10c2d0 con 0x7f624c1007c0 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6252710640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10c150 con 0x7f624c1019b0 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6252710640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10c450 con 0x7f624c102370 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1007c0 0x7f624c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1007c0 0x7f624c069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34586/0 (socket says 172.21.15.105:34586) 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 -- 172.21.15.105:0/3968698060 learned_addr learned my addr 172.21.15.105:0/3968698060 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624b7fe640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c1019b0 0x7f624c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f6250cad640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c102370 0x7f624c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 -- 172.21.15.105:0/3968698060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c102370 msgr2=0x7f624c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c102370 0x7f624c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 -- 172.21.15.105:0/3968698060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c1019b0 msgr2=0x7f624c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c1019b0 0x7f624c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 -- 172.21.15.105:0/3968698060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c19f790 con 0x7f624c1007c0 2024-03-26T17:37:44.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.493+0000 7f624bfff640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1007c0 0x7f624c069590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f623c00e9c0 tx=0x7f623c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:44.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.494+0000 7f62497fa640 1 -- 172.21.15.105:0/3968698060 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c00cd80 con 0x7f624c1007c0 2024-03-26T17:37:44.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.494+0000 7f62497fa640 1 -- 172.21.15.105:0/3968698060 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f623c0035f0 con 0x7f624c1007c0 2024-03-26T17:37:44.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.494+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c19fa20 con 0x7f624c1007c0 2024-03-26T17:37:44.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.494+0000 7f62497fa640 1 -- 172.21.15.105:0/3968698060 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c010690 con 0x7f624c1007c0 2024-03-26T17:37:44.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.494+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c19ff40 con 0x7f624c1007c0 2024-03-26T17:37:44.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.495+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624c107c00 con 0x7f624c1007c0 2024-03-26T17:37:44.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.496+0000 7f62497fa640 1 -- 172.21.15.105:0/3968698060 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f623c0037a0 con 0x7f624c1007c0 2024-03-26T17:37:44.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.496+0000 7f62497fa640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fb0 0x7f62340a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.497+0000 7f624b7fe640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fb0 0x7f62340a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.497+0000 7f62497fa640 1 -- 172.21.15.105:0/3968698060 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f623c0cf9a0 con 0x7f624c1007c0 2024-03-26T17:37:44.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.497+0000 7f624b7fe640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fb0 0x7f62340a5470 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6240009230 tx=0x7f6240005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:44.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.500+0000 7f62497fa640 1 -- 172.21.15.105:0/3968698060 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f623c09c690 con 0x7f624c1007c0 2024-03-26T17:37:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.788+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f624c02d2e0 con 0x7f624c1007c0 2024-03-26T17:37:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.788+0000 7f62497fa640 1 -- 172.21.15.105:0/3968698060 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f623c09bf90 con 0x7f624c1007c0 2024-03-26T17:37:44.781 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:37:44.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fb0 msgr2=0x7f62340a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fb0 0x7f62340a5470 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6240009230 tx=0x7f6240005fd0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1007c0 msgr2=0x7f624c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1007c0 0x7f624c069590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f623c00e9c0 tx=0x7f623c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 shutdown_connections 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fb0 0x7f62340a5470 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c102370 0x7f624c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c1019b0 0x7f624c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 --2- 172.21.15.105:0/3968698060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c1007c0 0x7f624c069590 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 >> 172.21.15.105:0/3968698060 conn(0x7f624c0fbd50 msgr2=0x7f624c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 shutdown_connections 2024-03-26T17:37:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.790+0000 7f6252710640 1 -- 172.21.15.105:0/3968698060 wait complete. 2024-03-26T17:37:44.794 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T17:37:44.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 -- 172.21.15.105:0/3119247675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 msgr2=0x7f7cb0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 --2- 172.21.15.105:0/3119247675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 0x7f7cb0100be0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009900 tx=0x7f7ca402bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 -- 172.21.15.105:0/3119247675 shutdown_connections 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 --2- 172.21.15.105:0/3119247675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb0102370 0x7f7cb0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 --2- 172.21.15.105:0/3119247675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb01019b0 0x7f7cb0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 --2- 172.21.15.105:0/3119247675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 0x7f7cb0100be0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 -- 172.21.15.105:0/3119247675 >> 172.21.15.105:0/3119247675 conn(0x7f7cb00fbd50 msgr2=0x7f7cb00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 -- 172.21.15.105:0/3119247675 shutdown_connections 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 -- 172.21.15.105:0/3119247675 wait complete. 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 Processor -- start 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.911+0000 7f7cb7097640 1 -- start start 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb7097640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 0x7f7cb0196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb7097640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb01019b0 0x7f7cb0196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb7097640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb0102370 0x7f7cb019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb7097640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb010c2f0 con 0x7f7cb01007c0 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb7097640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb010c170 con 0x7f7cb0102370 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb7097640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb010c470 con 0x7f7cb01019b0 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 0x7f7cb0196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 0x7f7cb0196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34598/0 (socket says 172.21.15.105:34598) 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 -- 172.21.15.105:0/2593445845 learned_addr learned my addr 172.21.15.105:0/2593445845 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7caffff640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb01019b0 0x7f7cb0196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb5634640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb0102370 0x7f7cb019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 -- 172.21.15.105:0/2593445845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb01019b0 msgr2=0x7f7cb0196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb01019b0 0x7f7cb0196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 -- 172.21.15.105:0/2593445845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb0102370 msgr2=0x7f7cb019e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb0102370 0x7f7cb019e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 -- 172.21.15.105:0/2593445845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb019e670 con 0x7f7cb01007c0 2024-03-26T17:37:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb4e33640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 0x7f7cb0196a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4000c00 tx=0x7f7ca4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cadffb640 1 -- 172.21.15.105:0/2593445845 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4043020 con 0x7f7cb01007c0 2024-03-26T17:37:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cadffb640 1 -- 172.21.15.105:0/2593445845 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ca4004350 con 0x7f7cb01007c0 2024-03-26T17:37:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb019e900 con 0x7f7cb01007c0 2024-03-26T17:37:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.912+0000 7f7cadffb640 1 -- 172.21.15.105:0/2593445845 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4003b80 con 0x7f7cb01007c0 2024-03-26T17:37:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.913+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb019ee50 con 0x7f7cb01007c0 2024-03-26T17:37:44.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.914+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb0107c50 con 0x7f7cb01007c0 2024-03-26T17:37:44.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.915+0000 7f7cadffb640 1 -- 172.21.15.105:0/2593445845 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f7ca4046020 con 0x7f7cb01007c0 2024-03-26T17:37:44.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.916+0000 7f7cadffb640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ca00a2fb0 0x7f7ca00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:44.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.916+0000 7f7caffff640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ca00a2fb0 0x7f7ca00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:44.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.916+0000 7f7cadffb640 1 -- 172.21.15.105:0/2593445845 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f7ca40f1980 con 0x7f7cb01007c0 2024-03-26T17:37:44.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.916+0000 7f7caffff640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ca00a2fb0 0x7f7ca00a5470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0197f60 tx=0x7f7c9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:44.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:44.919+0000 7f7cadffb640 1 -- 172.21.15.105:0/2593445845 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7ca404d020 con 0x7f7cb01007c0 2024-03-26T17:37:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.101+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7cb002d3d0 con 0x7f7cb01007c0 2024-03-26T17:37:45.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.101+0000 7f7cadffb640 1 -- 172.21.15.105:0/2593445845 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f7ca40be780 con 0x7f7cb01007c0 2024-03-26T17:37:45.094 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T17:37:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ca00a2fb0 msgr2=0x7f7ca00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ca00a2fb0 0x7f7ca00a5470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0197f60 tx=0x7f7c9c005f90 comp rx=0 tx=0).stop 2024-03-26T17:37:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 msgr2=0x7f7cb0196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 0x7f7cb0196a50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4000c00 tx=0x7f7ca4005dc0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 shutdown_connections 2024-03-26T17:37:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ca00a2fb0 0x7f7ca00a5470 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7cb0102370 0x7f7cb019e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cb01019b0 0x7f7cb0196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 --2- 172.21.15.105:0/2593445845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7cb01007c0 0x7f7cb0196a50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 >> 172.21.15.105:0/2593445845 conn(0x7f7cb00fbd50 msgr2=0x7f7cb0104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 shutdown_connections 2024-03-26T17:37:45.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.103+0000 7f7cb7097640 1 -- 172.21.15.105:0/2593445845 wait complete. 2024-03-26T17:37:45.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:44 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/852279807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:45.107 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:44 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/852279807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:45.108 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T17:37:45.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:44 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/852279807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:45.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 -- 172.21.15.105:0/3157136499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4788102380 msgr2=0x7f4788108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 --2- 172.21.15.105:0/3157136499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4788102380 0x7f4788108c10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4780009be0 tx=0x7f478002bef0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 -- 172.21.15.105:0/3157136499 shutdown_connections 2024-03-26T17:37:45.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 --2- 172.21.15.105:0/3157136499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4788102380 0x7f4788108c10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 --2- 172.21.15.105:0/3157136499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47881019c0 0x7f4788101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 --2- 172.21.15.105:0/3157136499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47881007d0 0x7f4788100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 -- 172.21.15.105:0/3157136499 >> 172.21.15.105:0/3157136499 conn(0x7f47880fbd60 msgr2=0x7f47880fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 -- 172.21.15.105:0/3157136499 shutdown_connections 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.221+0000 7f478e73b640 1 -- 172.21.15.105:0/3157136499 wait complete. 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478e73b640 1 Processor -- start 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478e73b640 1 -- start start 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478e73b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47881007d0 0x7f478806a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478e73b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47881019c0 0x7f478819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478e73b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4788102380 0x7f478819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478e73b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f478810c2e0 con 0x7f47881019c0 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478e73b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f478810c160 con 0x7f47881007d0 2024-03-26T17:37:45.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478e73b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f478810c460 con 0x7f4788102380 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f47877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47881019c0 0x7f478819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f47877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47881019c0 0x7f478819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34608/0 (socket says 172.21.15.105:34608) 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f47877fe640 1 -- 172.21.15.105:0/951807390 learned_addr learned my addr 172.21.15.105:0/951807390 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f478ccd8640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4788102380 0x7f478819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.222+0000 7f4787fff640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47881007d0 0x7f478806a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f478ccd8640 1 -- 172.21.15.105:0/951807390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47881007d0 msgr2=0x7f478806a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f478ccd8640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47881007d0 0x7f478806a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f478ccd8640 1 -- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47881019c0 msgr2=0x7f478819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f478ccd8640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47881019c0 0x7f478819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f478ccd8640 1 -- 172.21.15.105:0/951807390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478819f720 con 0x7f4788102380 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f47877fe640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47881019c0 0x7f478819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f478ccd8640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4788102380 0x7f478819ee10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f478002c430 tx=0x7f4780033c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f47857fa640 1 -- 172.21.15.105:0/951807390 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780043020 con 0x7f4788102380 2024-03-26T17:37:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f47857fa640 1 -- 172.21.15.105:0/951807390 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f478002ca60 con 0x7f4788102380 2024-03-26T17:37:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f47857fa640 1 -- 172.21.15.105:0/951807390 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780034430 con 0x7f4788102380 2024-03-26T17:37:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478819f950 con 0x7f4788102380 2024-03-26T17:37:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.223+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478819fe70 con 0x7f4788102380 2024-03-26T17:37:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.225+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4788107bc0 con 0x7f4788102380 2024-03-26T17:37:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.225+0000 7f47857fa640 1 -- 172.21.15.105:0/951807390 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f4780046020 con 0x7f4788102380 2024-03-26T17:37:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.226+0000 7f47857fa640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47700a2fb0 0x7f47700a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.226+0000 7f4787fff640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47700a2fb0 0x7f47700a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.226+0000 7f47857fa640 1 -- 172.21.15.105:0/951807390 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f47800f1ac0 con 0x7f4788102380 2024-03-26T17:37:45.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.226+0000 7f4787fff640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47700a2fb0 0x7f47700a5470 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4778009530 tx=0x7f4778009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.230+0000 7f47857fa640 1 -- 172.21.15.105:0/951807390 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f478004d020 con 0x7f4788102380 2024-03-26T17:37:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.421+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f478802d3b0 con 0x7f4788102380 2024-03-26T17:37:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.421+0000 7f47857fa640 1 -- 172.21.15.105:0/951807390 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f47800be860 con 0x7f4788102380 2024-03-26T17:37:45.414 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T17:37:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47700a2fb0 msgr2=0x7f47700a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47700a2fb0 0x7f47700a5470 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4778009530 tx=0x7f4778009260 comp rx=0 tx=0).stop 2024-03-26T17:37:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4788102380 msgr2=0x7f478819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4788102380 0x7f478819ee10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f478002c430 tx=0x7f4780033c90 comp rx=0 tx=0).stop 2024-03-26T17:37:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 shutdown_connections 2024-03-26T17:37:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47700a2fb0 0x7f47700a5470 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4788102380 0x7f478819ee10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47881019c0 0x7f478819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 --2- 172.21.15.105:0/951807390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47881007d0 0x7f478806a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 >> 172.21.15.105:0/951807390 conn(0x7f47880fbd60 msgr2=0x7f4788104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 shutdown_connections 2024-03-26T17:37:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.423+0000 7f478e73b640 1 -- 172.21.15.105:0/951807390 wait complete. 2024-03-26T17:37:45.427 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.537+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/3176342082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41007e0 msgr2=0x7fe2f4100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.537+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/3176342082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41007e0 0x7fe2f4100c00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009900 tx=0x7fe2f002bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.537+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/3176342082 shutdown_connections 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.537+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/3176342082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2f4102390 0x7fe2f4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.537+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/3176342082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f41019d0 0x7fe2f4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.537+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/3176342082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41007e0 0x7fe2f4100c00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.537+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/3176342082 >> 172.21.15.105:0/3176342082 conn(0x7fe2f40fbd70 msgr2=0x7fe2f40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.538+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/3176342082 shutdown_connections 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.538+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/3176342082 wait complete. 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.538+0000 7fe2fc1e6640 1 Processor -- start 2024-03-26T17:37:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.538+0000 7fe2fc1e6640 1 -- start start 2024-03-26T17:37:45.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.538+0000 7fe2fc1e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2f41007e0 0x7fe2f4196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2fc1e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41019d0 0x7fe2f4196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2fc1e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f4102390 0x7fe2f419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2fc1e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410c300 con 0x7fe2f41019d0 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2fc1e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410c180 con 0x7fe2f41007e0 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2fc1e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410c480 con 0x7fe2f4102390 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41019d0 0x7fe2f4196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41019d0 0x7fe2f4196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34632/0 (socket says 172.21.15.105:34632) 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9f82640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2f41007e0 0x7fe2f4196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 -- 172.21.15.105:0/1243139797 learned_addr learned my addr 172.21.15.105:0/1243139797 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2fa783640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f4102390 0x7fe2f419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 -- 172.21.15.105:0/1243139797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f4102390 msgr2=0x7fe2f419e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f4102390 0x7fe2f419e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 -- 172.21.15.105:0/1243139797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2f41007e0 msgr2=0x7fe2f4196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2f41007e0 0x7fe2f4196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 -- 172.21.15.105:0/1243139797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f419e680 con 0x7fe2f41019d0 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2f9781640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41019d0 0x7fe2f4196fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400eb00 tx=0x7fe2e400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2eaffd640 1 -- 172.21.15.105:0/1243139797 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400bb70 con 0x7fe2f41019d0 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2eaffd640 1 -- 172.21.15.105:0/1243139797 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2e400bd10 con 0x7fe2f41019d0 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f4069ef0 con 0x7fe2f41019d0 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2eaffd640 1 -- 172.21.15.105:0/1243139797 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4017410 con 0x7fe2f41019d0 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.539+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f406a440 con 0x7fe2f41019d0 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.540+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f4068b00 con 0x7fe2f41019d0 2024-03-26T17:37:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.541+0000 7fe2eaffd640 1 -- 172.21.15.105:0/1243139797 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fe2e4004ec0 con 0x7fe2f41019d0 2024-03-26T17:37:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.542+0000 7fe2eaffd640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2e00a3030 0x7fe2e00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.542+0000 7fe2f9f82640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2e00a3030 0x7fe2e00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.542+0000 7fe2eaffd640 1 -- 172.21.15.105:0/1243139797 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fe2e40d0c80 con 0x7fe2f41019d0 2024-03-26T17:37:45.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.542+0000 7fe2f9f82640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2e00a3030 0x7fe2e00a54f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0005ab0 tx=0x7fe2f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:45.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.546+0000 7fe2eaffd640 1 -- 172.21.15.105:0/1243139797 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe2e409d970 con 0x7fe2f41019d0 2024-03-26T17:37:45.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.762+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2f4107850 con 0x7fe2f41019d0 2024-03-26T17:37:45.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.763+0000 7fe2eaffd640 1 -- 172.21.15.105:0/1243139797 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7fe2e409d270 con 0x7fe2f41019d0 2024-03-26T17:37:45.756 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:45.756 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:36:29.308004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:37:45.756 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 26 2024-03-26T17:37:45.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2e00a3030 msgr2=0x7fe2e00a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2e00a3030 0x7fe2e00a54f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0005ab0 tx=0x7fe2f0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41019d0 msgr2=0x7fe2f4196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41019d0 0x7fe2f4196fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400eb00 tx=0x7fe2e400ef00 comp rx=0 tx=0).stop 2024-03-26T17:37:45.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 shutdown_connections 2024-03-26T17:37:45.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2e00a3030 0x7fe2e00a54f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2f4102390 0x7fe2f419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2f41019d0 0x7fe2f4196fa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 --2- 172.21.15.105:0/1243139797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2f41007e0 0x7fe2f4196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 >> 172.21.15.105:0/1243139797 conn(0x7fe2f40fbd70 msgr2=0x7fe2f4104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:45.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 shutdown_connections 2024-03-26T17:37:45.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.765+0000 7fe2fc1e6640 1 -- 172.21.15.105:0/1243139797 wait complete. 2024-03-26T17:37:45.770 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 -- 172.21.15.105:0/1507853248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c102390 msgr2=0x7f334c108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 --2- 172.21.15.105:0/1507853248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c102390 0x7f334c108c20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f333c009900 tx=0x7f333c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 -- 172.21.15.105:0/1507853248 shutdown_connections 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 --2- 172.21.15.105:0/1507853248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c102390 0x7f334c108c20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 --2- 172.21.15.105:0/1507853248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f334c1019d0 0x7f334c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 --2- 172.21.15.105:0/1507853248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f334c1007e0 0x7f334c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 -- 172.21.15.105:0/1507853248 >> 172.21.15.105:0/1507853248 conn(0x7f334c0fbd50 msgr2=0x7f334c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 -- 172.21.15.105:0/1507853248 shutdown_connections 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.907+0000 7f3351959640 1 -- 172.21.15.105:0/1507853248 wait complete. 2024-03-26T17:37:45.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f3351959640 1 Processor -- start 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f3351959640 1 -- start start 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f3351959640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f334c1007e0 0x7f334c0695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f3351959640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c1019d0 0x7f334c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f3351959640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f334c102390 0x7f334c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f3351959640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f334c10c310 con 0x7f334c1019d0 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f3351959640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f334c10c190 con 0x7f334c102390 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f3351959640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f334c10c490 con 0x7f334c1007e0 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c1019d0 0x7f334c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c1019d0 0x7f334c19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34642/0 (socket says 172.21.15.105:34642) 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f334c1007e0 0x7f334c0695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334a7fc640 1 -- 172.21.15.105:0/2171216295 learned_addr learned my addr 172.21.15.105:0/2171216295 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334b7fe640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f334c102390 0x7f334c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334a7fc640 1 -- 172.21.15.105:0/2171216295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f334c1007e0 msgr2=0x7f334c0695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334a7fc640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f334c1007e0 0x7f334c0695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334a7fc640 1 -- 172.21.15.105:0/2171216295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f334c102390 msgr2=0x7f334c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334a7fc640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f334c102390 0x7f334c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334a7fc640 1 -- 172.21.15.105:0/2171216295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c19f6e0 con 0x7f334c1019d0 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.908+0000 7f334affd640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f334c1007e0 0x7f334c0695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.909+0000 7f334a7fc640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c1019d0 0x7f334c19c7e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f334000ea60 tx=0x7f334000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.909+0000 7f3350957640 1 -- 172.21.15.105:0/2171216295 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334000cde0 con 0x7f334c1019d0 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.909+0000 7f3350957640 1 -- 172.21.15.105:0/2171216295 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33400035f0 con 0x7f334c1019d0 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.909+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c19f9d0 con 0x7f334c1019d0 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.909+0000 7f3350957640 1 -- 172.21.15.105:0/2171216295 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340010690 con 0x7f334c1019d0 2024-03-26T17:37:45.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.909+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c19ff70 con 0x7f334c1019d0 2024-03-26T17:37:45.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.910+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f334c107b80 con 0x7f334c1019d0 2024-03-26T17:37:45.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.912+0000 7f3350957640 1 -- 172.21.15.105:0/2171216295 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f33400037a0 con 0x7f334c1019d0 2024-03-26T17:37:45.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.912+0000 7f3350957640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33340a3030 0x7f33340a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:45.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.912+0000 7f334affd640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33340a3030 0x7f33340a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:45.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.912+0000 7f3350957640 1 -- 172.21.15.105:0/2171216295 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f33400cf990 con 0x7f334c1019d0 2024-03-26T17:37:45.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.913+0000 7f334affd640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33340a3030 0x7f33340a54f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f334c101830 tx=0x7f3344009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:45.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:45.915+0000 7f3350957640 1 -- 172.21.15.105:0/2171216295 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3340053740 con 0x7f334c1019d0 2024-03-26T17:37:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.097+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f334c02d240 con 0x7f334c1019d0 2024-03-26T17:37:46.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.099+0000 7f3350957640 1 -- 172.21.15.105:0/2171216295 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f334009c680 con 0x7f334c1019d0 2024-03-26T17:37:46.092 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:46.092 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33340a3030 msgr2=0x7f33340a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33340a3030 0x7f33340a54f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f334c101830 tx=0x7f3344009500 comp rx=0 tx=0).stop 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c1019d0 msgr2=0x7f334c19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c1019d0 0x7f334c19c7e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f334000ea60 tx=0x7f334000ef20 comp rx=0 tx=0).stop 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 shutdown_connections 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33340a3030 0x7f33340a54f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f334c102390 0x7f334c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f334c1019d0 0x7f334c19c7e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 --2- 172.21.15.105:0/2171216295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f334c1007e0 0x7f334c0695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 >> 172.21.15.105:0/2171216295 conn(0x7f334c0fbd50 msgr2=0x7f334c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:46.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 shutdown_connections 2024-03-26T17:37:46.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.101+0000 7f3351959640 1 -- 172.21.15.105:0/2171216295 wait complete. 2024-03-26T17:37:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:45 smithi144 ceph-mon[36095]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T17:37:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3968698060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2593445845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/951807390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1243139797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:46.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:45 smithi105 ceph-mon[32280]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T17:37:46.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3968698060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2593445845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/951807390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.104 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1243139797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:37:46.106 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:37:46.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:45 smithi135 ceph-mon[35636]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-03-26T17:37:46.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3968698060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2593445845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/951807390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:37:46.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1243139797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:46.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 -- 172.21.15.105:0/2874618786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c98100b10 msgr2=0x7f1c98100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 --2- 172.21.15.105:0/2874618786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c98100b10 0x7f1c98100f70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009900 tx=0x7f1c8002bcd0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 -- 172.21.15.105:0/2874618786 shutdown_connections 2024-03-26T17:37:46.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 --2- 172.21.15.105:0/2874618786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c981014b0 0x7f1c98101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 --2- 172.21.15.105:0/2874618786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c98100b10 0x7f1c98100f70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 --2- 172.21.15.105:0/2874618786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c981001b0 0x7f1c981005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 -- 172.21.15.105:0/2874618786 >> 172.21.15.105:0/2874618786 conn(0x7f1c980fbd50 msgr2=0x7f1c980fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:46.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 -- 172.21.15.105:0/2874618786 shutdown_connections 2024-03-26T17:37:46.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.219+0000 7f1c9da73640 1 -- 172.21.15.105:0/2874618786 wait complete. 2024-03-26T17:37:46.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c9da73640 1 Processor -- start 2024-03-26T17:37:46.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c9da73640 1 -- start start 2024-03-26T17:37:46.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c9da73640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c981001b0 0x7f1c98199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c9da73640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98100b10 0x7f1c9819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c9da73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c981014b0 0x7f1c981a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c9da73640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9810f640 con 0x7f1c981014b0 2024-03-26T17:37:46.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c9da73640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9810f4c0 con 0x7f1c98100b10 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c9da73640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9810f7c0 con 0x7f1c981001b0 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c981014b0 0x7f1c981a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c97fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c981014b0 0x7f1c981a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34652/0 (socket says 172.21.15.105:34652) 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c97fff640 1 -- 172.21.15.105:0/1538906775 learned_addr learned my addr 172.21.15.105:0/1538906775 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c8effd640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98100b10 0x7f1c9819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.220+0000 7f1c977fe640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c981001b0 0x7f1c98199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c97fff640 1 -- 172.21.15.105:0/1538906775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c981001b0 msgr2=0x7f1c98199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c97fff640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c981001b0 0x7f1c98199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c97fff640 1 -- 172.21.15.105:0/1538906775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98100b10 msgr2=0x7f1c9819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c97fff640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98100b10 0x7f1c9819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c97fff640 1 -- 172.21.15.105:0/1538906775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c981a1970 con 0x7f1c981014b0 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c97fff640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c981014b0 0x7f1c981a1300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000eb10 tx=0x7f1c9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c957fa640 1 -- 172.21.15.105:0/1538906775 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90004b20 con 0x7f1c981014b0 2024-03-26T17:37:46.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c957fa640 1 -- 172.21.15.105:0/1538906775 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c90004cc0 con 0x7f1c981014b0 2024-03-26T17:37:46.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c957fa640 1 -- 172.21.15.105:0/1538906775 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90010690 con 0x7f1c981014b0 2024-03-26T17:37:46.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c981a1c60 con 0x7f1c981014b0 2024-03-26T17:37:46.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.221+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c981a21d0 con 0x7f1c981014b0 2024-03-26T17:37:46.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.222+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9810b140 con 0x7f1c981014b0 2024-03-26T17:37:46.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.223+0000 7f1c957fa640 1 -- 172.21.15.105:0/1538906775 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f1c90005020 con 0x7f1c981014b0 2024-03-26T17:37:46.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.224+0000 7f1c957fa640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c840a3030 0x7f1c840a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.224+0000 7f1c977fe640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c840a3030 0x7f1c840a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.224+0000 7f1c957fa640 1 -- 172.21.15.105:0/1538906775 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f1c900cfba0 con 0x7f1c981014b0 2024-03-26T17:37:46.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.224+0000 7f1c977fe640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c840a3030 0x7f1c840a54f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009bd0 tx=0x7f1c88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:46.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.228+0000 7f1c957fa640 1 -- 172.21.15.105:0/1538906775 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1c9009c890 con 0x7f1c981014b0 2024-03-26T17:37:46.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.408+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c9802d460 con 0x7f1c981014b0 2024-03-26T17:37:46.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.409+0000 7f1c957fa640 1 -- 172.21.15.105:0/1538906775 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f1c9009c190 con 0x7f1c981014b0 2024-03-26T17:37:46.402 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:37:46.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c840a3030 msgr2=0x7f1c840a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c840a3030 0x7f1c840a54f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009bd0 tx=0x7f1c88005f90 comp rx=0 tx=0).stop 2024-03-26T17:37:46.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c981014b0 msgr2=0x7f1c981a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c981014b0 0x7f1c981a1300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000eb10 tx=0x7f1c9000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 shutdown_connections 2024-03-26T17:37:46.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c840a3030 0x7f1c840a54f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c981014b0 0x7f1c981a1300 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c98100b10 0x7f1c9819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 --2- 172.21.15.105:0/1538906775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c981001b0 0x7f1c98199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 >> 172.21.15.105:0/1538906775 conn(0x7f1c980fbd50 msgr2=0x7f1c980fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:46.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 shutdown_connections 2024-03-26T17:37:46.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.411+0000 7f1c9da73640 1 -- 172.21.15.105:0/1538906775 wait complete. 2024-03-26T17:37:46.415 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T17:37:46.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 -- 172.21.15.105:0/1555067087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 msgr2=0x7fc094101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1555067087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 0x7fc094101d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc088009900 tx=0x7fc08802bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:46.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 -- 172.21.15.105:0/1555067087 shutdown_connections 2024-03-26T17:37:46.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1555067087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc094102250 0x7fc094108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1555067087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 0x7fc094101d10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1555067087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0941006a0 0x7fc094100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 -- 172.21.15.105:0/1555067087 >> 172.21.15.105:0/1555067087 conn(0x7fc0940fbc30 msgr2=0x7fc0940fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:46.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 -- 172.21.15.105:0/1555067087 shutdown_connections 2024-03-26T17:37:46.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.527+0000 7fc098f9f640 1 -- 172.21.15.105:0/1555067087 wait complete. 2024-03-26T17:37:46.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc098f9f640 1 Processor -- start 2024-03-26T17:37:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc098f9f640 1 -- start start 2024-03-26T17:37:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc098f9f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0941006a0 0x7fc094196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc098f9f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 0x7fc094196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc098f9f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc094102250 0x7fc09419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc098f9f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410c1d0 con 0x7fc0941006a0 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc098f9f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410c050 con 0x7fc094101890 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc098f9f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410c350 con 0x7fc094102250 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc0937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0941006a0 0x7fc094196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc0937fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0941006a0 0x7fc094196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34662/0 (socket says 172.21.15.105:34662) 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc093fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc094102250 0x7fc09419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc0937fe640 1 -- 172.21.15.105:0/1710075863 learned_addr learned my addr 172.21.15.105:0/1710075863 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc092ffd640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 0x7fc094196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc092ffd640 1 -- 172.21.15.105:0/1710075863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc094102250 msgr2=0x7fc09419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc092ffd640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc094102250 0x7fc09419ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc092ffd640 1 -- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0941006a0 msgr2=0x7fc094196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc092ffd640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0941006a0 0x7fc094196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.528+0000 7fc092ffd640 1 -- 172.21.15.105:0/1710075863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09419e540 con 0x7fc094101890 2024-03-26T17:37:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.529+0000 7fc0937fe640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0941006a0 0x7fc094196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:46.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.529+0000 7fc092ffd640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 0x7fc094196e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc088000c00 tx=0x7fc088005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:46.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.529+0000 7fc090ff9640 1 -- 172.21.15.105:0/1710075863 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088043020 con 0x7fc094101890 2024-03-26T17:37:46.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.529+0000 7fc090ff9640 1 -- 172.21.15.105:0/1710075863 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc088004350 con 0x7fc094101890 2024-03-26T17:37:46.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.529+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09419e7d0 con 0x7fc094101890 2024-03-26T17:37:46.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.529+0000 7fc090ff9640 1 -- 172.21.15.105:0/1710075863 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088003b50 con 0x7fc094101890 2024-03-26T17:37:46.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.529+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09419ed20 con 0x7fc094101890 2024-03-26T17:37:46.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.530+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc094107a80 con 0x7fc094101890 2024-03-26T17:37:46.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.531+0000 7fc090ff9640 1 -- 172.21.15.105:0/1710075863 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fc088046020 con 0x7fc094101890 2024-03-26T17:37:46.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.531+0000 7fc090ff9640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc07c0a2fb0 0x7fc07c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.532+0000 7fc0937fe640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc07c0a2fb0 0x7fc07c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.532+0000 7fc090ff9640 1 -- 172.21.15.105:0/1710075863 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fc0880f17f0 con 0x7fc094101890 2024-03-26T17:37:46.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.532+0000 7fc0937fe640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc07c0a2fb0 0x7fc07c0a5470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0840058c0 tx=0x7fc084005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:46.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.535+0000 7fc090ff9640 1 -- 172.21.15.105:0/1710075863 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc08804d020 con 0x7fc094101890 2024-03-26T17:37:46.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.715+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc09402d490 con 0x7fc094101890 2024-03-26T17:37:46.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.715+0000 7fc090ff9640 1 -- 172.21.15.105:0/1710075863 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fc0880be5f0 con 0x7fc094101890 2024-03-26T17:37:46.708 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T17:37:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc07c0a2fb0 msgr2=0x7fc07c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc07c0a2fb0 0x7fc07c0a5470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc0840058c0 tx=0x7fc084005f90 comp rx=0 tx=0).stop 2024-03-26T17:37:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 msgr2=0x7fc094196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 0x7fc094196e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc088000c00 tx=0x7fc088005dc0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 shutdown_connections 2024-03-26T17:37:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc07c0a2fb0 0x7fc07c0a5470 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc094102250 0x7fc09419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc094101890 0x7fc094196e60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 --2- 172.21.15.105:0/1710075863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0941006a0 0x7fc094196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.717+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 >> 172.21.15.105:0/1710075863 conn(0x7fc0940fbc30 msgr2=0x7fc094104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.718+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 shutdown_connections 2024-03-26T17:37:46.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.718+0000 7fc098f9f640 1 -- 172.21.15.105:0/1710075863 wait complete. 2024-03-26T17:37:46.721 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T17:37:46.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 -- 172.21.15.105:0/236838681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81019c0 msgr2=0x7f8bf8101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 --2- 172.21.15.105:0/236838681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81019c0 0x7f8bf8101e40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009900 tx=0x7f8be802bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:46.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 -- 172.21.15.105:0/236838681 shutdown_connections 2024-03-26T17:37:46.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 --2- 172.21.15.105:0/236838681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf8102380 0x7f8bf8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 --2- 172.21.15.105:0/236838681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81019c0 0x7f8bf8101e40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 --2- 172.21.15.105:0/236838681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bf81007d0 0x7f8bf8100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 -- 172.21.15.105:0/236838681 >> 172.21.15.105:0/236838681 conn(0x7f8bf80fbd60 msgr2=0x7f8bf80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 -- 172.21.15.105:0/236838681 shutdown_connections 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.835+0000 7f8c00275640 1 -- 172.21.15.105:0/236838681 wait complete. 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8c00275640 1 Processor -- start 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8c00275640 1 -- start start 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8c00275640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81007d0 0x7f8bf8196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8c00275640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bf81019c0 0x7f8bf8196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8c00275640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf8102380 0x7f8bf819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8c00275640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf810c300 con 0x7f8bf81007d0 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8c00275640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf810c180 con 0x7f8bf81019c0 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8c00275640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf810c480 con 0x7f8bf8102380 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfe011640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81007d0 0x7f8bf8196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfe011640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81007d0 0x7f8bf8196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34684/0 (socket says 172.21.15.105:34684) 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfe011640 1 -- 172.21.15.105:0/549637230 learned_addr learned my addr 172.21.15.105:0/549637230 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:46.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfd810640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bf81019c0 0x7f8bf8196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfe812640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf8102380 0x7f8bf819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfe011640 1 -- 172.21.15.105:0/549637230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf8102380 msgr2=0x7f8bf819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfe011640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf8102380 0x7f8bf819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfe011640 1 -- 172.21.15.105:0/549637230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bf81019c0 msgr2=0x7f8bf8196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.836+0000 7f8bfe011640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bf81019c0 0x7f8bf8196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.837+0000 7f8bfe011640 1 -- 172.21.15.105:0/549637230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf819e650 con 0x7f8bf81007d0 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.837+0000 7f8bfe011640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81007d0 0x7f8bf8196a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400e9c0 tx=0x7f8bf400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.837+0000 7f8bef7fe640 1 -- 172.21.15.105:0/549637230 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400cd80 con 0x7f8bf81007d0 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.837+0000 7f8bef7fe640 1 -- 172.21.15.105:0/549637230 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bf40035f0 con 0x7f8bf81007d0 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.837+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf819e940 con 0x7f8bf81007d0 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.837+0000 7f8bef7fe640 1 -- 172.21.15.105:0/549637230 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4010690 con 0x7f8bf81007d0 2024-03-26T17:37:46.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.837+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf819ee60 con 0x7f8bf81007d0 2024-03-26T17:37:46.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.838+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf8107b60 con 0x7f8bf81007d0 2024-03-26T17:37:46.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.839+0000 7f8bef7fe640 1 -- 172.21.15.105:0/549637230 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f8bf40037a0 con 0x7f8bf81007d0 2024-03-26T17:37:46.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.840+0000 7f8bef7fe640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8be40a2fb0 0x7f8be40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:46.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.840+0000 7f8bfd810640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8be40a2fb0 0x7f8be40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:46.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.840+0000 7f8bef7fe640 1 -- 172.21.15.105:0/549637230 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f8bf40cf9a0 con 0x7f8bf81007d0 2024-03-26T17:37:46.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.840+0000 7f8bfd810640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8be40a2fb0 0x7f8be40a5470 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009230 tx=0x7f8be8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:46.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:46.844+0000 7f8bef7fe640 1 -- 172.21.15.105:0/549637230 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8bf409c690 con 0x7f8bf81007d0 2024-03-26T17:37:47.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.023+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8bf802d2c0 con 0x7f8bf81007d0 2024-03-26T17:37:47.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.024+0000 7f8bef7fe640 1 -- 172.21.15.105:0/549637230 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f8bf409bf90 con 0x7f8bf81007d0 2024-03-26T17:37:47.017 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T17:37:47.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8be40a2fb0 msgr2=0x7f8be40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8be40a2fb0 0x7f8be40a5470 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009230 tx=0x7f8be8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81007d0 msgr2=0x7f8bf8196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81007d0 0x7f8bf8196a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400e9c0 tx=0x7f8bf400ee80 comp rx=0 tx=0).stop 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 shutdown_connections 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8be40a2fb0 0x7f8be40a5470 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf8102380 0x7f8bf819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bf81019c0 0x7f8bf8196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 --2- 172.21.15.105:0/549637230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bf81007d0 0x7f8bf8196a30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 >> 172.21.15.105:0/549637230 conn(0x7f8bf80fbd60 msgr2=0x7f8bf8104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 shutdown_connections 2024-03-26T17:37:47.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.026+0000 7f8c00275640 1 -- 172.21.15.105:0/549637230 wait complete. 2024-03-26T17:37:47.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:46 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2171216295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:47.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:46 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1538906775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:37:47.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:46 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1710075863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:37:47.030 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:37:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:46 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2171216295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:46 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1538906775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:37:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:46 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1710075863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:37:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 -- 172.21.15.105:0/88731966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effc4102390 msgr2=0x7effc4108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 --2- 172.21.15.105:0/88731966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effc4102390 0x7effc4108c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7effbc009ca0 tx=0x7effbc02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 -- 172.21.15.105:0/88731966 shutdown_connections 2024-03-26T17:37:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 --2- 172.21.15.105:0/88731966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effc4102390 0x7effc4108c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 --2- 172.21.15.105:0/88731966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc41019d0 0x7effc4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 --2- 172.21.15.105:0/88731966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc41007e0 0x7effc4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 -- 172.21.15.105:0/88731966 >> 172.21.15.105:0/88731966 conn(0x7effc40fbd50 msgr2=0x7effc40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 -- 172.21.15.105:0/88731966 shutdown_connections 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.151+0000 7effcbabd640 1 -- 172.21.15.105:0/88731966 wait complete. 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effcbabd640 1 Processor -- start 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effcbabd640 1 -- start start 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effcbabd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc41007e0 0x7effc4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effcbabd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc41019d0 0x7effc419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effcbabd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effc4102390 0x7effc419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effcbabd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7effc410c2f0 con 0x7effc41007e0 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effcbabd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effc410c170 con 0x7effc4102390 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effcbabd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7effc410c470 con 0x7effc41019d0 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effc9859640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc41007e0 0x7effc4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effc9859640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc41007e0 0x7effc4069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34694/0 (socket says 172.21.15.105:34694) 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effc9859640 1 -- 172.21.15.105:0/4204731830 learned_addr learned my addr 172.21.15.105:0/4204731830 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effc9058640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc41019d0 0x7effc419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.152+0000 7effca05a640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effc4102390 0x7effc419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effc9859640 1 -- 172.21.15.105:0/4204731830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc41019d0 msgr2=0x7effc419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effc9859640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc41019d0 0x7effc419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effc9859640 1 -- 172.21.15.105:0/4204731830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effc4102390 msgr2=0x7effc419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effc9859640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effc4102390 0x7effc419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effc9859640 1 -- 172.21.15.105:0/4204731830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc419f720 con 0x7effc41007e0 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effc9859640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc41007e0 0x7effc4069590 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7effc000e9c0 tx=0x7effc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effbaffd640 1 -- 172.21.15.105:0/4204731830 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc000cd80 con 0x7effc41007e0 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc419f9b0 con 0x7effc41007e0 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effbaffd640 1 -- 172.21.15.105:0/4204731830 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effc00035f0 con 0x7effc41007e0 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effbaffd640 1 -- 172.21.15.105:0/4204731830 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0010690 con 0x7effc41007e0 2024-03-26T17:37:47.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.153+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc419ff00 con 0x7effc41007e0 2024-03-26T17:37:47.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.154+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc4107b00 con 0x7effc41007e0 2024-03-26T17:37:47.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.155+0000 7effbaffd640 1 -- 172.21.15.105:0/4204731830 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7effc00037a0 con 0x7effc41007e0 2024-03-26T17:37:47.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.156+0000 7effbaffd640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effb00a2fb0 0x7effb00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.156+0000 7effc9058640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effb00a2fb0 0x7effb00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.156+0000 7effbaffd640 1 -- 172.21.15.105:0/4204731830 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7effc00cf8e0 con 0x7effc41007e0 2024-03-26T17:37:47.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.156+0000 7effc9058640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effb00a2fb0 0x7effb00a5470 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effc4069300 tx=0x7effb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:47.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.160+0000 7effbaffd640 1 -- 172.21.15.105:0/4204731830 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7effc009c5d0 con 0x7effc41007e0 2024-03-26T17:37:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:46 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2171216295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:46 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1538906775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:37:47.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:46 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1710075863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:37:47.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.452+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effc40624c0 con 0x7effc41007e0 2024-03-26T17:37:47.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.453+0000 7effbaffd640 1 -- 172.21.15.105:0/4204731830 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7effc00bc0c0 con 0x7effc41007e0 2024-03-26T17:37:47.445 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:47.446 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:36:29.308004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:37:47.446 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 26 2024-03-26T17:37:47.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effb00a2fb0 msgr2=0x7effb00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effb00a2fb0 0x7effb00a5470 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7effc4069300 tx=0x7effb4005f90 comp rx=0 tx=0).stop 2024-03-26T17:37:47.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc41007e0 msgr2=0x7effc4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc41007e0 0x7effc4069590 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7effc000e9c0 tx=0x7effc000ee80 comp rx=0 tx=0).stop 2024-03-26T17:37:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 shutdown_connections 2024-03-26T17:37:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effb00a2fb0 0x7effb00a5470 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effc4102390 0x7effc419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effc41019d0 0x7effc419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 --2- 172.21.15.105:0/4204731830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effc41007e0 0x7effc4069590 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 >> 172.21.15.105:0/4204731830 conn(0x7effc40fbd50 msgr2=0x7effc4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 shutdown_connections 2024-03-26T17:37:47.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.455+0000 7effcbabd640 1 -- 172.21.15.105:0/4204731830 wait complete. 2024-03-26T17:37:47.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 -- 172.21.15.105:0/2436975814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41007e0 msgr2=0x7f27d4100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 --2- 172.21.15.105:0/2436975814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41007e0 0x7f27d4100c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f27c4009900 tx=0x7f27c402bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 -- 172.21.15.105:0/2436975814 shutdown_connections 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 --2- 172.21.15.105:0/2436975814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27d4102390 0x7f27d4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 --2- 172.21.15.105:0/2436975814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d41019d0 0x7f27d4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 --2- 172.21.15.105:0/2436975814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41007e0 0x7f27d4100c00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 -- 172.21.15.105:0/2436975814 >> 172.21.15.105:0/2436975814 conn(0x7f27d40fbd50 msgr2=0x7f27d40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 -- 172.21.15.105:0/2436975814 shutdown_connections 2024-03-26T17:37:47.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.573+0000 7f27d90d1640 1 -- 172.21.15.105:0/2436975814 wait complete. 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d90d1640 1 Processor -- start 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d90d1640 1 -- start start 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d90d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d41007e0 0x7f27d4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d90d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41019d0 0x7f27d41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d90d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27d4102390 0x7f27d419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d90d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27d410c310 con 0x7f27d41019d0 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d90d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27d410c190 con 0x7f27d41007e0 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d90d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27d410c490 con 0x7f27d4102390 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41019d0 0x7f27d41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41019d0 0x7f27d41980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34720/0 (socket says 172.21.15.105:34720) 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d259c640 1 -- 172.21.15.105:0/1781844743 learned_addr learned my addr 172.21.15.105:0/1781844743 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d2d9d640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d41007e0 0x7f27d4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.574+0000 7f27d359e640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27d4102390 0x7f27d419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27d259c640 1 -- 172.21.15.105:0/1781844743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27d4102390 msgr2=0x7f27d419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27d259c640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27d4102390 0x7f27d419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27d259c640 1 -- 172.21.15.105:0/1781844743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d41007e0 msgr2=0x7f27d4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27d259c640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d41007e0 0x7f27d4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27d259c640 1 -- 172.21.15.105:0/1781844743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d419f780 con 0x7f27d41019d0 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27d259c640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41019d0 0x7f27d41980a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e9f0 tx=0x7f27c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27bbfff640 1 -- 172.21.15.105:0/1781844743 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c800cd80 con 0x7f27d41019d0 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d419fa70 con 0x7f27d41019d0 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27bbfff640 1 -- 172.21.15.105:0/1781844743 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27c80108e0 con 0x7f27d41019d0 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27bbfff640 1 -- 172.21.15.105:0/1781844743 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8018920 con 0x7f27d41019d0 2024-03-26T17:37:47.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.575+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d419ffc0 con 0x7f27d41019d0 2024-03-26T17:37:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.576+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d41a01f0 con 0x7f27d41019d0 2024-03-26T17:37:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.577+0000 7f27bbfff640 1 -- 172.21.15.105:0/1781844743 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f27c8010a90 con 0x7f27d41019d0 2024-03-26T17:37:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.578+0000 7f27bbfff640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27bc0a2fb0 0x7f27bc0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.578+0000 7f27d2d9d640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27bc0a2fb0 0x7f27bc0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.578+0000 7f27bbfff640 1 -- 172.21.15.105:0/1781844743 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f27c80cf840 con 0x7f27d41019d0 2024-03-26T17:37:47.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.578+0000 7f27d2d9d640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27bc0a2fb0 0x7f27bc0a5470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005ab0 tx=0x7f27c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:47.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.582+0000 7f27bbfff640 1 -- 172.21.15.105:0/1781844743 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f27c809bee0 con 0x7f27d41019d0 2024-03-26T17:37:47.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.763+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27d41a08e0 con 0x7f27d41019d0 2024-03-26T17:37:47.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.765+0000 7f27bbfff640 1 -- 172.21.15.105:0/1781844743 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f27c809ff80 con 0x7f27d41019d0 2024-03-26T17:37:47.758 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:47.758 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:37:47.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27bc0a2fb0 msgr2=0x7f27bc0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27bc0a2fb0 0x7f27bc0a5470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27c4005ab0 tx=0x7f27c4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41019d0 msgr2=0x7f27d41980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41019d0 0x7f27d41980a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e9f0 tx=0x7f27c800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 shutdown_connections 2024-03-26T17:37:47.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27bc0a2fb0 0x7f27bc0a5470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27d4102390 0x7f27d419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27d41019d0 0x7f27d41980a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 --2- 172.21.15.105:0/1781844743 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d41007e0 0x7f27d4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 >> 172.21.15.105:0/1781844743 conn(0x7f27d40fbd50 msgr2=0x7f27d4068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:47.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 shutdown_connections 2024-03-26T17:37:47.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.767+0000 7f27d90d1640 1 -- 172.21.15.105:0/1781844743 wait complete. 2024-03-26T17:37:47.774 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:37:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 -- 172.21.15.105:0/1113136657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e841007e0 msgr2=0x7f3e84100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1113136657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e841007e0 0x7f3e84100c00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009900 tx=0x7f3e7402bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 -- 172.21.15.105:0/1113136657 shutdown_connections 2024-03-26T17:37:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1113136657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e84102390 0x7f3e84108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1113136657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e841019d0 0x7f3e84101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1113136657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e841007e0 0x7f3e84100c00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 -- 172.21.15.105:0/1113136657 >> 172.21.15.105:0/1113136657 conn(0x7f3e840fbd50 msgr2=0x7f3e840fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 -- 172.21.15.105:0/1113136657 shutdown_connections 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.884+0000 7f3e891c6640 1 -- 172.21.15.105:0/1113136657 wait complete. 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e891c6640 1 Processor -- start 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e891c6640 1 -- start start 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e891c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e841007e0 0x7f3e84197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e891c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e841019d0 0x7f3e84198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e891c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e84102390 0x7f3e8419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e891c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8410d430 con 0x7f3e84102390 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e891c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8410d2b0 con 0x7f3e841007e0 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e891c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8410d5b0 con 0x7f3e841019d0 2024-03-26T17:37:47.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e8359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e84102390 0x7f3e8419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e82d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e841007e0 0x7f3e84197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e8359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e84102390 0x7f3e8419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34736/0 (socket says 172.21.15.105:34736) 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e8359e640 1 -- 172.21.15.105:0/1677426716 learned_addr learned my addr 172.21.15.105:0/1677426716 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e7bfff640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e841019d0 0x7f3e84198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e8359e640 1 -- 172.21.15.105:0/1677426716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e841019d0 msgr2=0x7f3e84198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e8359e640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e841019d0 0x7f3e84198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e8359e640 1 -- 172.21.15.105:0/1677426716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e841007e0 msgr2=0x7f3e84197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e8359e640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e841007e0 0x7f3e84197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e8359e640 1 -- 172.21.15.105:0/1677426716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e841095e0 con 0x7f3e84102390 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.885+0000 7f3e82d9d640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e841007e0 0x7f3e84197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.886+0000 7f3e8359e640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e84102390 0x7f3e8419f170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c00efd0 tx=0x7f3e7c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.886+0000 7f3e80d99640 1 -- 172.21.15.105:0/1677426716 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c003b70 con 0x7f3e84102390 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.886+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e841098d0 con 0x7f3e84102390 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.886+0000 7f3e80d99640 1 -- 172.21.15.105:0/1677426716 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7c003d10 con 0x7f3e84102390 2024-03-26T17:37:47.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.886+0000 7f3e80d99640 1 -- 172.21.15.105:0/1677426716 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c010980 con 0x7f3e84102390 2024-03-26T17:37:47.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.886+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e84109e20 con 0x7f3e84102390 2024-03-26T17:37:47.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.887+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e8402d240 con 0x7f3e84102390 2024-03-26T17:37:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.888+0000 7f3e80d99640 1 -- 172.21.15.105:0/1677426716 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f3e7c004b40 con 0x7f3e84102390 2024-03-26T17:37:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.888+0000 7f3e80d99640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e700a2fb0 0x7f3e700a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.889+0000 7f3e82d9d640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e700a2fb0 0x7f3e700a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.889+0000 7f3e80d99640 1 -- 172.21.15.105:0/1677426716 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f3e7c0cfd60 con 0x7f3e84102390 2024-03-26T17:37:47.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.889+0000 7f3e82d9d640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e700a2fb0 0x7f3e700a5470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005ab0 tx=0x7f3e74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:47.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:47.893+0000 7f3e80d99640 1 -- 172.21.15.105:0/1677426716 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3e7c09ca50 con 0x7f3e84102390 2024-03-26T17:37:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:47 smithi144 ceph-mon[36095]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:47 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/549637230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:37:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:47 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4204731830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:47 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1781844743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:48.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.112+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e8402d440 con 0x7f3e84102390 2024-03-26T17:37:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.113+0000 7f3e80d99640 1 -- 172.21.15.105:0/1677426716 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7f3e7c09c350 con 0x7f3e84102390 2024-03-26T17:37:48.106 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:37:48.106 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:36:29.308004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:37:48.106 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 26 2024-03-26T17:37:48.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e700a2fb0 msgr2=0x7f3e700a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e700a2fb0 0x7f3e700a5470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e74005ab0 tx=0x7f3e74005fd0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e84102390 msgr2=0x7f3e8419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e84102390 0x7f3e8419f170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c00efd0 tx=0x7f3e7c00c6a0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 shutdown_connections 2024-03-26T17:37:48.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e700a2fb0 0x7f3e700a5470 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e84102390 0x7f3e8419f170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e841019d0 0x7f3e84198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 --2- 172.21.15.105:0/1677426716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e841007e0 0x7f3e84197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 >> 172.21.15.105:0/1677426716 conn(0x7f3e840fbd50 msgr2=0x7f3e84104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 shutdown_connections 2024-03-26T17:37:48.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.116+0000 7f3e891c6640 1 -- 172.21.15.105:0/1677426716 wait complete. 2024-03-26T17:37:48.119 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:47 smithi105 ceph-mon[32280]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:48.119 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:47 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/549637230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:37:48.119 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:47 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4204731830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:48.119 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:47 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1781844743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:48.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T17:36:22.059727+0000', 'modified': '2024-03-26T17:36:29.308004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24521, 'mds_1': 24487, 'mds_2': 24482, 'mds_3': 24466, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6839/3053296748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 3053296748}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 3053296748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6841/3843959520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6840', 'nonce': 3843959520}, {'type': 'v1', 'addr': '172.21.15.105:6841', 'nonce': 3843959520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3569777961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3569777961}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3569777961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6833/1499648854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6832', 'nonce': 1499648854}, {'type': 'v1', 'addr': '172.21.15.135:6833', 'nonce': 1499648854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/29089236', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 29089236}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 29089236}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6835/3121985659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 3121985659}, {'type': 'v1', 'addr': '172.21.15.135:6835', 'nonce': 3121985659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.144:6837/184508646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 184508646}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 184508646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.135:6837/388686483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6836', 'nonce': 388686483}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 388686483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.144:6839/1175380833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 1175380833}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 1175380833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6839/338591305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 338591305}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 338591305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24482, 24521, 14628, 24466, 24487]} 2024-03-26T17:37:48.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-26T17:37:48.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-26T17:37:48.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:47 smithi135 ceph-mon[35636]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:48.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:47 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/549637230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:37:48.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:47 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4204731830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:48.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:47 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1781844743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:37:48.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 -- 172.21.15.105:0/2784926239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac104cf0 msgr2=0x7f35ac1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 --2- 172.21.15.105:0/2784926239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac104cf0 0x7f35ac1070e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3594009860 tx=0x7f359402bb70 comp rx=0 tx=0).stop 2024-03-26T17:37:48.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 -- 172.21.15.105:0/2784926239 shutdown_connections 2024-03-26T17:37:48.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 --2- 172.21.15.105:0/2784926239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ac107620 0x7f35ac109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 --2- 172.21.15.105:0/2784926239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac104cf0 0x7f35ac1070e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 --2- 172.21.15.105:0/2784926239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ac1023c0 0x7f35ac1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 -- 172.21.15.105:0/2784926239 >> 172.21.15.105:0/2784926239 conn(0x7f35ac0fbd50 msgr2=0x7f35ac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 -- 172.21.15.105:0/2784926239 shutdown_connections 2024-03-26T17:37:48.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.232+0000 7f35b1559640 1 -- 172.21.15.105:0/2784926239 wait complete. 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35b1559640 1 Processor -- start 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35b1559640 1 -- start start 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35b1559640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac1023c0 0x7f35ac197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35b1559640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ac104cf0 0x7f35ac1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35b1559640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ac107620 0x7f35ac19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35b1559640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10d3f0 con 0x7f35ac1023c0 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35b1559640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10d270 con 0x7f35ac104cf0 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35b1559640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10d570 con 0x7f35ac107620 2024-03-26T17:37:48.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac1023c0 0x7f35ac197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac1023c0 0x7f35ac197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34758/0 (socket says 172.21.15.105:34758) 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.233+0000 7f35aaffd640 1 -- 172.21.15.105:0/1235587321 learned_addr learned my addr 172.21.15.105:0/1235587321 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ac107620 0x7f35ac19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35a3fff640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ac104cf0 0x7f35ac1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35aaffd640 1 -- 172.21.15.105:0/1235587321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ac107620 msgr2=0x7f35ac19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35aaffd640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ac107620 0x7f35ac19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35aaffd640 1 -- 172.21.15.105:0/1235587321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ac104cf0 msgr2=0x7f35ac1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35aaffd640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ac104cf0 0x7f35ac1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35aaffd640 1 -- 172.21.15.105:0/1235587321 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac19f7a0 con 0x7f35ac1023c0 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35aaffd640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac1023c0 0x7f35ac197b80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f359c00ea60 tx=0x7f359c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35a8ff9640 1 -- 172.21.15.105:0/1235587321 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00cde0 con 0x7f35ac1023c0 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35b1559640 1 -- 172.21.15.105:0/1235587321 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac19fa90 con 0x7f35ac1023c0 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35a8ff9640 1 -- 172.21.15.105:0/1235587321 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f359c0035f0 con 0x7f35ac1023c0 2024-03-26T17:37:48.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35a8ff9640 1 -- 172.21.15.105:0/1235587321 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c010690 con 0x7f35ac1023c0 2024-03-26T17:37:48.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.234+0000 7f35b1559640 1 -- 172.21.15.105:0/1235587321 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac19ffb0 con 0x7f35ac1023c0 2024-03-26T17:37:48.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.236+0000 7f35a8ff9640 1 -- 172.21.15.105:0/1235587321 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f359c0037a0 con 0x7f35ac1023c0 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35b4213740 1 Processor -- start 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35b4213740 1 -- start start 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35b4213740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d33470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35b4213740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34220 0x5647e3d346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35b4213740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d3b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35b4213740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647e3c93270 con 0x5647e3d34220 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35b4213740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647e3c3bc80 con 0x5647e3d33050 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35b4213740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5647e3c8f220 con 0x5647e3d34be0 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35a8ff9640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35980a2fb0 0x7f35980a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34220 0x5647e3d346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d3b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34220 0x5647e3d346a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34774/0 (socket says 172.21.15.105:34774) 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35aaffd640 1 -- 172.21.15.105:0/2366888018 learned_addr learned my addr 172.21.15.105:0/2366888018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d33470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35980a2fb0 0x7f35980a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35a8ff9640 1 -- 172.21.15.105:0/1235587321 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f359c0cf7d0 con 0x7f35ac1023c0 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35a3fff640 1 -- 172.21.15.105:0/2366888018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 msgr2=0x5647e3d33470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.237+0000 7f35a3fff640 1 --2- 172.21.15.105:0/2366888018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d33470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35a3fff640 1 -- 172.21.15.105:0/2366888018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34220 msgr2=0x5647e3d346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35a3fff640 1 --2- 172.21.15.105:0/2366888018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34220 0x5647e3d346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35a3fff640 1 -- 172.21.15.105:0/2366888018 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e3c4dfb0 con 0x5647e3d34be0 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/2366888018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d33470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35a3fff640 1 --2- 172.21.15.105:0/2366888018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d3b470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f359402c0b0 tx=0x7f3594009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a3eb471ac903ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35980a2fb0 0x7f35980a5470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5647e3d34120 tx=0x7f35a40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35a17fa640 1 -- 172.21.15.105:0/2366888018 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594043020 con 0x5647e3d34be0 2024-03-26T17:37:48.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35a17fa640 1 -- 172.21.15.105:0/2366888018 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3594004040 con 0x5647e3d34be0 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.238+0000 7f35a17fa640 1 -- 172.21.15.105:0/2366888018 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594039040 con 0x5647e3d34be0 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- 172.21.15.105:0/2366888018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 msgr2=0x5647e3d3b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 --2- 172.21.15.105:0/2366888018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d3b470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f359402c0b0 tx=0x7f3594009490 comp rx=0 tx=0).stop 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- 172.21.15.105:0/2366888018 shutdown_connections 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 --2- 172.21.15.105:0/2366888018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d3b470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 --2- 172.21.15.105:0/2366888018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34220 0x5647e3d346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 --2- 172.21.15.105:0/2366888018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d33470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- 172.21.15.105:0/2366888018 >> 172.21.15.105:0/2366888018 conn(0x5647e3d2a030 msgr2=0x5647e3d2a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- 172.21.15.105:0/2366888018 shutdown_connections 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- 172.21.15.105:0/2366888018 wait complete. 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- start start 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 Processor -- start 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d3d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 0x5647e3d3dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34be0 0x5647e3d44af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647e3d3bb90 con 0x5647e3d33050 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647e3c4efc0 con 0x5647e3d34be0 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5647e3d3bd10 con 0x5647e3d34220 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35b4213740 10 client.0 _ll_register_callbacks cb 0x5647e3c49770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d3d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.239+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d3d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34780/0 (socket says 172.21.15.105:34780) 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35aaffd640 1 -- 172.21.15.105:0/33335269 learned_addr learned my addr 172.21.15.105:0/33335269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34be0 0x5647e3d44af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35a3fff640 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 0x5647e3d3dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35aaffd640 1 -- 172.21.15.105:0/33335269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 msgr2=0x5647e3d3dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35aaffd640 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 0x5647e3d3dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35aaffd640 1 -- 172.21.15.105:0/33335269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34be0 msgr2=0x5647e3d44af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35aaffd640 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34be0 0x5647e3d44af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35aaffd640 1 -- 172.21.15.105:0/33335269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e39a72d0 con 0x5647e3d33050 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34be0 0x5647e3d44af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35aaffd640 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d3d570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f359c00c760 tx=0x7f359c09b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e6a00f46933a81 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35927fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35927fc640 1 -- 172.21.15.105:0/33335269 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c09bdc0 con 0x5647e3d33050 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35927fc640 1 -- 172.21.15.105:0/33335269 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f359c004000 con 0x5647e3d33050 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35b4213740 1 -- 172.21.15.105:0/33335269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3594004040 con 0x5647e3d33050 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35927fc640 1 -- 172.21.15.105:0/33335269 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f359c09c4c0 con 0x5647e3d33050 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35b4213740 10 client.15042 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35b4213740 1 -- 172.21.15.105:0/33335269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647e3a716b0 con 0x5647e3d33050 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.240+0000 7f35927fc640 1 -- 172.21.15.105:0/33335269 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3594004040 con 0x5647e3d33050 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35927fc640 1 -- 172.21.15.105:0/33335269 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f359c09b210 con 0x5647e3d33050 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35b4213740 10 client.15042 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35b4213740 10 client.15042 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35b4213740 20 client.15042 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35b4213740 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5647e3d4a350 0x5647e3d4b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35b4213740 4 client.15042 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35b4213740 1 -- 172.21.15.105:0/33335269 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647e3ca07b0 con 0x5647e3d4a350 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35a3fff640 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5647e3d4a350 0x5647e3d4b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35927fc640 10 client.15042 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.241+0000 7f35a3fff640 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5647e3d4a350 0x5647e3d4b7c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1867e76c39774f8 server_cookie=6631d6600b1cc8d2 in_seq=0 out_seq=0 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35927fc640 1 -- 172.21.15.105:0/33335269 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5647e3ca07b0 con 0x5647e3d4a350 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35927fc640 10 client.15042 handle_command_reply: tid=0 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 client.15042 shutdown 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 -- 172.21.15.105:0/33335269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 msgr2=0x5647e3d3d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d3d570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f359c00c760 tx=0x7f359c09b9c0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 -- 172.21.15.105:0/33335269 shutdown_connections 2024-03-26T17:37:48.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5647e3d4a350 0x5647e3d4b7c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34be0 0x5647e3d44af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 0x5647e3d3dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35927fc640 0 client.15042 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:37:48.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 --2- 172.21.15.105:0/33335269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d3d570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 -- 172.21.15.105:0/33335269 >> 172.21.15.105:0/33335269 conn(0x5647e3d2a030 msgr2=0x5647e3d29230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 -- 172.21.15.105:0/33335269 shutdown_connections 2024-03-26T17:37:48.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 1 -- 172.21.15.105:0/33335269 wait complete. 2024-03-26T17:37:48.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.242+0000 7f35b4213740 20 client.15042 trim_cache size 0 max 16384 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.260+0000 7f35b4213740 1 Processor -- start 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.260+0000 7f35b4213740 1 -- start start 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.260+0000 7f35b4213740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d2a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.260+0000 7f35b4213740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34220 0x5647e3d44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.260+0000 7f35b4213740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.260+0000 7f35b4213740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647e3d49b00 con 0x5647e3d33050 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.260+0000 7f35b4213740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647e3d4a1d0 con 0x5647e3d34220 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.260+0000 7f35b4213740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5647e3d4a8a0 con 0x5647e3d34be0 2024-03-26T17:37:48.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d2a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34220 0x5647e3d44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d2a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34788/0 (socket says 172.21.15.105:34788) 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 -- 172.21.15.105:0/2495359818 learned_addr learned my addr 172.21.15.105:0/2495359818 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 -- 172.21.15.105:0/2495359818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 msgr2=0x5647e3d372c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d372c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 -- 172.21.15.105:0/2495359818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34220 msgr2=0x5647e3d44a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34220 0x5647e3d44a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 -- 172.21.15.105:0/2495359818 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647e3d3b0f0 con 0x5647e3d33050 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35a3fff640 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34220 0x5647e3d44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35aaffd640 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d2a410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f359c009920 tx=0x7f359c09ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8317740565286a74 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35a0ff9640 1 -- 172.21.15.105:0/2495359818 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c097e60 con 0x5647e3d33050 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35a0ff9640 1 -- 172.21.15.105:0/2495359818 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f359c09bdc0 con 0x5647e3d33050 2024-03-26T17:37:48.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.261+0000 7f35a0ff9640 1 -- 172.21.15.105:0/2495359818 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c09b210 con 0x5647e3d33050 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 -- 172.21.15.105:0/2495359818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 msgr2=0x5647e3d2a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d2a410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f359c009920 tx=0x7f359c09ba30 comp rx=0 tx=0).stop 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 -- 172.21.15.105:0/2495359818 shutdown_connections 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34be0 0x5647e3d372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d34220 0x5647e3d44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 --2- 172.21.15.105:0/2495359818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d33050 0x5647e3d2a410 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 -- 172.21.15.105:0/2495359818 >> 172.21.15.105:0/2495359818 conn(0x5647e3d2a030 msgr2=0x5647e3d3c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 -- 172.21.15.105:0/2495359818 shutdown_connections 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 -- 172.21.15.105:0/2495359818 wait complete. 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 -- start start 2024-03-26T17:37:48.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.262+0000 7f35b4213740 1 Processor -- start 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 0x5647e3d42f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34be0 0x5647e3d43480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5647e3d4b640 con 0x5647e3d34be0 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647e3d4af70 con 0x5647e3d33050 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5647e3d4bd10 con 0x5647e3d34220 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 10 client.0 _ll_register_callbacks cb 0x7f359c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34be0 0x5647e3d43480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34be0 0x5647e3d43480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34792/0 (socket says 172.21.15.105:34792) 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 -- 172.21.15.105:0/2015090512 learned_addr learned my addr 172.21.15.105:0/2015090512 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35a3fff640 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 0x5647e3d42f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35aaffd640 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 -- 172.21.15.105:0/2015090512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 msgr2=0x5647e3d42f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 0x5647e3d42f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 -- 172.21.15.105:0/2015090512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 msgr2=0x5647e3d409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 -- 172.21.15.105:0/2015090512 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647e3d43bf0 con 0x5647e3d34be0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35ab7fe640 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34be0 0x5647e3d43480 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f35a4003fb0 tx=0x7f35a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322d4238a9f8c499 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35927fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35927fc640 1 -- 172.21.15.105:0/2015090512 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a402e070 con 0x5647e3d34be0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35927fc640 1 -- 172.21.15.105:0/2015090512 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f35a401e790 con 0x5647e3d34be0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 1 -- 172.21.15.105:0/2015090512 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f359c09bdc0 con 0x5647e3d34be0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35927fc640 1 -- 172.21.15.105:0/2015090512 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f35a403a050 con 0x5647e3d34be0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 10 client.15048 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.263+0000 7f35b4213740 1 -- 172.21.15.105:0/2015090512 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647e3db00c0 con 0x5647e3d34be0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35927fc640 1 -- 172.21.15.105:0/2015090512 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f359c09bdc0 con 0x5647e3d34be0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35927fc640 1 -- 172.21.15.105:0/2015090512 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f35a400b040 con 0x5647e3d34be0 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 10 client.15048 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 10 client.15048 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 20 client.15048 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5647e3d4a350 0x5647e3db3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 4 client.15048 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 1 -- 172.21.15.105:0/2015090512 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "status"}) -- 0x5647e3c9ab00 con 0x5647e3d4a350 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35aaffd640 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5647e3d4a350 0x5647e3db3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35927fc640 10 client.15048 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35aaffd640 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5647e3d4a350 0x5647e3db3930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66090a2d1ed3a8ea server_cookie=d9ea6d82f3d120e2 in_seq=0 out_seq=0 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35927fc640 1 -- 172.21.15.105:0/2015090512 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5647e3c9ab00 con 0x5647e3d4a350 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35927fc640 10 client.15048 handle_command_reply: tid=0 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 1 client.15048 shutdown 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 1 -- 172.21.15.105:0/2015090512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34be0 msgr2=0x5647e3d43480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34be0 0x5647e3d43480 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f35a4003fb0 tx=0x7f35a4004a40 comp rx=0 tx=0).stop 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 1 -- 172.21.15.105:0/2015090512 shutdown_connections 2024-03-26T17:37:48.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5647e3d4a350 0x5647e3db3930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.264+0000 7f35b4213740 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5647e3d34be0 0x5647e3d43480 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b4213740 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5647e3d34220 0x5647e3d42f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b4213740 1 --2- 172.21.15.105:0/2015090512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647e3d33050 0x5647e3d409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b4213740 1 -- 172.21.15.105:0/2015090512 >> 172.21.15.105:0/2015090512 conn(0x5647e3d2a030 msgr2=0x5647e3d2a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35927fc640 0 client.15048 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b4213740 1 -- 172.21.15.105:0/2015090512 shutdown_connections 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b4213740 1 -- 172.21.15.105:0/2015090512 wait complete. 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b4213740 20 client.15048 trim_cache size 0 max 16384 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:37:48.264 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 4, 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 14628, 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 79.960624189000001, 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 94.250645429000002 2024-03-26T17:37:48.265 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 -- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35980a2fb0 msgr2=0x7f35980a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35980a2fb0 0x7f35980a5470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5647e3d34120 tx=0x7f35a40095c0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 -- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac1023c0 msgr2=0x7f35ac197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac1023c0 0x7f35ac197b80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f359c00ea60 tx=0x7f359c00ef20 comp rx=0 tx=0).stop 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 -- 172.21.15.105:0/1235587321 shutdown_connections 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35980a2fb0 0x7f35980a5470 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ac107620 0x7f35ac19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ac104cf0 0x7f35ac1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 --2- 172.21.15.105:0/1235587321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ac1023c0 0x7f35ac197b80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 -- 172.21.15.105:0/1235587321 >> 172.21.15.105:0/1235587321 conn(0x7f35ac0fbd50 msgr2=0x7f35ac104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 -- 172.21.15.105:0/1235587321 shutdown_connections 2024-03-26T17:37:48.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.265+0000 7f35b1559640 1 -- 172.21.15.105:0/1235587321 wait complete. 2024-03-26T17:37:48.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-26T17:37:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 -- 172.21.15.105:0/2041476716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 msgr2=0x7fb2a0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2041476716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 0x7fb2a0100be0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb298009ca0 tx=0x7fb29802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 -- 172.21.15.105:0/2041476716 shutdown_connections 2024-03-26T17:37:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2041476716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2a0102370 0x7fb2a0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2041476716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a01019b0 0x7fb2a0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2041476716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 0x7fb2a0100be0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 -- 172.21.15.105:0/2041476716 >> 172.21.15.105:0/2041476716 conn(0x7fb2a00fbd50 msgr2=0x7fb2a00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 -- 172.21.15.105:0/2041476716 shutdown_connections 2024-03-26T17:37:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.380+0000 7fb2a5737640 1 -- 172.21.15.105:0/2041476716 wait complete. 2024-03-26T17:37:48.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb2a5737640 1 Processor -- start 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb2a5737640 1 -- start start 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb2a5737640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 0x7fb2a0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb2a5737640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a01019b0 0x7fb2a019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb2a5737640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2a0102370 0x7fb2a01a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb2a5737640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a010c2c0 con 0x7fb2a01007c0 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb2a5737640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a010c140 con 0x7fb2a0102370 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb2a5737640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a010c440 con 0x7fb2a01019b0 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 0x7fb2a0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 0x7fb2a0069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34808/0 (socket says 172.21.15.105:34808) 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 -- 172.21.15.105:0/2509042228 learned_addr learned my addr 172.21.15.105:0/2509042228 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a01019b0 0x7fb2a019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29f7fe640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2a0102370 0x7fb2a01a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 -- 172.21.15.105:0/2509042228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a01019b0 msgr2=0x7fb2a019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a01019b0 0x7fb2a019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 -- 172.21.15.105:0/2509042228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2a0102370 msgr2=0x7fb2a01a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2a0102370 0x7fb2a01a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 -- 172.21.15.105:0/2509042228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a01a19c0 con 0x7fb2a01007c0 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.381+0000 7fb29effd640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 0x7fb2a0069590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb298032380 tx=0x7fb298032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.382+0000 7fb297fff640 1 -- 172.21.15.105:0/2509042228 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298041020 con 0x7fb2a01007c0 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.382+0000 7fb2a5737640 1 -- 172.21.15.105:0/2509042228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a01a1bf0 con 0x7fb2a01007c0 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.382+0000 7fb297fff640 1 -- 172.21.15.105:0/2509042228 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb298003e20 con 0x7fb2a01007c0 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.382+0000 7fb2a5737640 1 -- 172.21.15.105:0/2509042228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a01a2130 con 0x7fb2a01007c0 2024-03-26T17:37:48.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.382+0000 7fb297fff640 1 -- 172.21.15.105:0/2509042228 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298039040 con 0x7fb2a01007c0 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb297fff640 1 -- 172.21.15.105:0/2509042228 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fb298044020 con 0x7fb2a01007c0 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb2a83f8740 1 Processor -- start 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb2a83f8740 1 -- start start 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda6c40 0x56531cda7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda7e10 0x56531cda8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdaf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56531cd0b270 con 0x56531cda6c40 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56531ccb3c80 con 0x56531cda7e10 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56531cd07220 con 0x56531cda87d0 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.384+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda6c40 0x56531cda7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda6c40 0x56531cda7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34820/0 (socket says 172.21.15.105:34820) 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29f7fe640 1 -- 172.21.15.105:0/673862506 learned_addr learned my addr 172.21.15.105:0/673862506 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda7e10 0x56531cda8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdaf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb297fff640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb28c0a2fe0 0x7fb28c0a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29e7fc640 1 -- 172.21.15.105:0/673862506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda7e10 msgr2=0x56531cda8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda7e10 0x56531cda8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29e7fc640 1 -- 172.21.15.105:0/673862506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda6c40 msgr2=0x56531cda7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda6c40 0x56531cda7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29e7fc640 1 -- 172.21.15.105:0/673862506 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531ccc5fb0 con 0x56531cda87d0 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29effd640 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda7e10 0x56531cda8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29f7fe640 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda6c40 0x56531cda7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29f7fe640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb28c0a2fe0 0x7fb28c0a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb297fff640 1 -- 172.21.15.105:0/2509042228 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb2980f1410 con 0x7fb2a01007c0 2024-03-26T17:37:48.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdaf060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb288009bf0 tx=0x7fb288011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=206d589c53f415a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb2957fa640 1 -- 172.21.15.105:0/673862506 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288009da0 con 0x56531cda87d0 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb29f7fe640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb28c0a2fe0 0x7fb28c0a54a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x56531cda7d10 tx=0x7fb2900046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb2957fa640 1 -- 172.21.15.105:0/673862506 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb288005020 con 0x56531cda87d0 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.385+0000 7fb2957fa640 1 -- 172.21.15.105:0/673862506 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288004a90 con 0x56531cda87d0 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 -- 172.21.15.105:0/673862506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 msgr2=0x56531cdaf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdaf060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb288009bf0 tx=0x7fb288011810 comp rx=0 tx=0).stop 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 -- 172.21.15.105:0/673862506 shutdown_connections 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdaf060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda7e10 0x56531cda8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/673862506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda6c40 0x56531cda7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 -- 172.21.15.105:0/673862506 >> 172.21.15.105:0/673862506 conn(0x56531cda2030 msgr2=0x56531cda2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 -- 172.21.15.105:0/673862506 shutdown_connections 2024-03-26T17:37:48.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.386+0000 7fb2a83f8740 1 -- 172.21.15.105:0/673862506 wait complete. 2024-03-26T17:37:48.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 1 -- start start 2024-03-26T17:37:48.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 1 Processor -- start 2024-03-26T17:37:48.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdb1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 0x56531cdb86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56531cdaf7e0 con 0x56531cda7e10 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56531ccc6fc0 con 0x56531cda87d0 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56531cdaf960 con 0x56531cda6c40 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb2a83f8740 10 client.0 _ll_register_callbacks cb 0x56531ccc1770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34836/0 (socket says 172.21.15.105:34836) 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdb1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29e7fc640 1 -- 172.21.15.105:0/870223274 learned_addr learned my addr 172.21.15.105:0/870223274 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29f7fe640 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 0x56531cdb86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29e7fc640 1 -- 172.21.15.105:0/870223274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 msgr2=0x56531cdb1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdb1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29e7fc640 1 -- 172.21.15.105:0/870223274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 msgr2=0x56531cdb86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.387+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 0x56531cdb86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb29e7fc640 1 -- 172.21.15.105:0/870223274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531ca1f2d0 con 0x56531cda7e10 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb16a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb28801b040 tx=0x7fb288009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf713040428f6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2867fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2867fc640 1 -- 172.21.15.105:0/870223274 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288029020 con 0x56531cda7e10 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2867fc640 1 -- 172.21.15.105:0/870223274 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb288004a20 con 0x56531cda7e10 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2a83f8740 1 -- 172.21.15.105:0/870223274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb288005020 con 0x56531cda7e10 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2867fc640 1 -- 172.21.15.105:0/870223274 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb288004c20 con 0x56531cda7e10 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2a83f8740 10 client.15060 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2a83f8740 1 -- 172.21.15.105:0/870223274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56531cae96b0 con 0x56531cda7e10 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2867fc640 1 -- 172.21.15.105:0/870223274 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb288005020 con 0x56531cda7e10 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2867fc640 1 -- 172.21.15.105:0/870223274 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb288023420 con 0x56531cda7e10 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2a83f8740 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2a83f8740 10 client.15060 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2a83f8740 20 client.15060 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.388+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56531cdbdd00 0x56531cdbf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.389+0000 7fb2a83f8740 4 client.15060 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.389+0000 7fb2a83f8740 1 -- 172.21.15.105:0/870223274 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56531cd187b0 con 0x56531cdbdd00 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.389+0000 7fb29effd640 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56531cdbdd00 0x56531cdbf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.389+0000 7fb29effd640 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56531cdbdd00 0x56531cdbf170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3c60637cc704060 server_cookie=d87b5b62336f4085 in_seq=0 out_seq=0 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.389+0000 7fb2867fc640 10 client.15060 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2867fc640 1 -- 172.21.15.105:0/870223274 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56531cd187b0 con 0x56531cdbdd00 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2867fc640 10 client.15060 handle_command_reply: tid=0 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 client.15060 shutdown 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 -- 172.21.15.105:0/870223274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 msgr2=0x56531cdb16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb16a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb28801b040 tx=0x7fb288009970 comp rx=0 tx=0).stop 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 -- 172.21.15.105:0/870223274 shutdown_connections 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56531cdbdd00 0x56531cdbf170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 0x56531cdb86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb16a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2867fc640 0 client.15060 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:37:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/870223274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdb1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 -- 172.21.15.105:0/870223274 >> 172.21.15.105:0/870223274 conn(0x56531cda2030 msgr2=0x56531cda1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 -- 172.21.15.105:0/870223274 shutdown_connections 2024-03-26T17:37:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 1 -- 172.21.15.105:0/870223274 wait complete. 2024-03-26T17:37:48.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.390+0000 7fb2a83f8740 20 client.15060 trim_cache size 0 max 16384 2024-03-26T17:37:48.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb2a83f8740 1 Processor -- start 2024-03-26T17:37:48.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb2a83f8740 1 -- start start 2024-03-26T17:37:48.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdaefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 0x56531cdbcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56531cdbd4b0 con 0x56531cda7e10 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56531cdbdb80 con 0x56531cda87d0 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56531cdbe250 con 0x56531cda6c40 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb29e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb29effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdaefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb29e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34846/0 (socket says 172.21.15.105:34846) 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb29f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 0x56531cdbcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.408+0000 7fb29e7fc640 1 -- 172.21.15.105:0/3910945769 learned_addr learned my addr 172.21.15.105:0/3910945769 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb29e7fc640 1 -- 172.21.15.105:0/3910945769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 msgr2=0x56531cdaefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/3910945769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdaefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb29e7fc640 1 -- 172.21.15.105:0/3910945769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 msgr2=0x56531cdbcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/3910945769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 0x56531cdbcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb29e7fc640 1 -- 172.21.15.105:0/3910945769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531cdbe700 con 0x56531cda7e10 2024-03-26T17:37:48.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb29effd640 1 --2- 172.21.15.105:0/3910945769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdaefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/3910945769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb86a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb288023910 tx=0x7fb288009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258ea72e5ac9871e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb294ff9640 1 -- 172.21.15.105:0/3910945769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288029040 con 0x56531cda7e10 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb294ff9640 1 -- 172.21.15.105:0/3910945769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb288016030 con 0x56531cda7e10 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.409+0000 7fb294ff9640 1 -- 172.21.15.105:0/3910945769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288005020 con 0x56531cda7e10 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- 172.21.15.105:0/3910945769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 msgr2=0x56531cdb86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/3910945769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb86a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb288023910 tx=0x7fb288009a70 comp rx=0 tx=0).stop 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- 172.21.15.105:0/3910945769 shutdown_connections 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/3910945769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda87d0 0x56531cdbcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/3910945769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb86a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/3910945769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda6c40 0x56531cdaefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- 172.21.15.105:0/3910945769 >> 172.21.15.105:0/3910945769 conn(0x56531cda2030 msgr2=0x56531cda5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- 172.21.15.105:0/3910945769 shutdown_connections 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- 172.21.15.105:0/3910945769 wait complete. 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- start start 2024-03-26T17:37:48.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 Processor -- start 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda6c40 0x56531cdb3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdb72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56531cdbeff0 con 0x56531cda7e10 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56531cdbe920 con 0x56531cda6c40 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56531cdbf6c0 con 0x56531cda87d0 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb2a83f8740 10 client.0 _ll_register_callbacks cb 0x7fb290009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.410+0000 7fb29e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34858/0 (socket says 172.21.15.105:34858) 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29e7fc640 1 -- 172.21.15.105:0/1204498776 learned_addr learned my addr 172.21.15.105:0/1204498776 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29effd640 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda6c40 0x56531cdb3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29f7fe640 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdb72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29e7fc640 1 -- 172.21.15.105:0/1204498776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 msgr2=0x56531cdb72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdb72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29e7fc640 1 -- 172.21.15.105:0/1204498776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda6c40 msgr2=0x56531cdb3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda6c40 0x56531cdb3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29e7fc640 1 -- 172.21.15.105:0/1204498776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531cdb78a0 con 0x56531cda7e10 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb29e7fc640 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb4290 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb288023a00 tx=0x7fb288005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a923da26bd017d50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb2867fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb2867fc640 1 -- 172.21.15.105:0/1204498776 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28803e030 con 0x56531cda7e10 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb2867fc640 1 -- 172.21.15.105:0/1204498776 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb288004a20 con 0x56531cda7e10 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb2a83f8740 1 -- 172.21.15.105:0/1204498776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb288016030 con 0x56531cda7e10 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb2867fc640 1 -- 172.21.15.105:0/1204498776 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb288029040 con 0x56531cda7e10 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb2a83f8740 10 client.15066 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb2a83f8740 1 -- 172.21.15.105:0/1204498776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56531ce23a70 con 0x56531cda7e10 2024-03-26T17:37:48.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.411+0000 7fb2867fc640 1 -- 172.21.15.105:0/1204498776 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb288016030 con 0x56531cda7e10 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2867fc640 1 -- 172.21.15.105:0/1204498776 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb288037020 con 0x56531cda7e10 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 10 client.15066 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 10 client.15066 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 20 client.15066 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56531cdbdd00 0x56531ce26fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 4 client.15066 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 1 -- 172.21.15.105:0/1204498776 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "status"}) -- 0x56531cd12b00 con 0x56531cdbdd00 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb29effd640 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56531cdbdd00 0x56531ce26fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2867fc640 10 client.15066 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb29effd640 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56531cdbdd00 0x56531ce26fa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd8e0f887d062695 server_cookie=d2085336f902a81 in_seq=0 out_seq=0 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2867fc640 1 -- 172.21.15.105:0/1204498776 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56531cd12b00 con 0x56531cdbdd00 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2867fc640 10 client.15066 handle_command_reply: tid=0 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 1 client.15066 shutdown 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 1 -- 172.21.15.105:0/1204498776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 msgr2=0x56531cdb4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.412+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb4290 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb288023a00 tx=0x7fb288005f90 comp rx=0 tx=0).stop 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 1 -- 172.21.15.105:0/1204498776 shutdown_connections 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56531cdbdd00 0x56531ce26fa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56531cda87d0 0x56531cdb72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2867fc640 0 client.15066 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56531cda7e10 0x56531cdb4290 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 1 --2- 172.21.15.105:0/1204498776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56531cda6c40 0x56531cdb3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 1 -- 172.21.15.105:0/1204498776 >> 172.21.15.105:0/1204498776 conn(0x56531cda2030 msgr2=0x56531cda5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 1 -- 172.21.15.105:0/1204498776 shutdown_connections 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 1 -- 172.21.15.105:0/1204498776 wait complete. 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a83f8740 20 client.15066 trim_cache size 0 max 16384 2024-03-26T17:37:48.412 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 3, 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24466, 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 81.083747907000003, 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 105.250762778 2024-03-26T17:37:48.413 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:37:48.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 -- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb28c0a2fe0 msgr2=0x7fb28c0a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb28c0a2fe0 0x7fb28c0a54a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x56531cda7d10 tx=0x7fb2900046c0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 -- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 msgr2=0x7fb2a0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 0x7fb2a0069590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb298032380 tx=0x7fb298032820 comp rx=0 tx=0).stop 2024-03-26T17:37:48.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 -- 172.21.15.105:0/2509042228 shutdown_connections 2024-03-26T17:37:48.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb28c0a2fe0 0x7fb28c0a54a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb2a0102370 0x7fb2a01a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2a01019b0 0x7fb2a019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 --2- 172.21.15.105:0/2509042228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2a01007c0 0x7fb2a0069590 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.413+0000 7fb2a5737640 1 -- 172.21.15.105:0/2509042228 >> 172.21.15.105:0/2509042228 conn(0x7fb2a00fbd50 msgr2=0x7fb2a0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.414+0000 7fb2a5737640 1 -- 172.21.15.105:0/2509042228 shutdown_connections 2024-03-26T17:37:48.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.414+0000 7fb2a5737640 1 -- 172.21.15.105:0/2509042228 wait complete. 2024-03-26T17:37:48.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-26T17:37:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 -- 172.21.15.105:0/3357729633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100bb0 msgr2=0x7f4878101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 --2- 172.21.15.105:0/3357729633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100bb0 0x7f4878101010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f486c009900 tx=0x7f486c02bcd0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 -- 172.21.15.105:0/3357729633 shutdown_connections 2024-03-26T17:37:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 --2- 172.21.15.105:0/3357729633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4878101550 0x7f48781019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 --2- 172.21.15.105:0/3357729633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100bb0 0x7f4878101010 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 --2- 172.21.15.105:0/3357729633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4878100250 0x7f4878100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 -- 172.21.15.105:0/3357729633 >> 172.21.15.105:0/3357729633 conn(0x7f48780fbdd0 msgr2=0x7f48780fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 -- 172.21.15.105:0/3357729633 shutdown_connections 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.530+0000 7f487d35b640 1 -- 172.21.15.105:0/3357729633 wait complete. 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f487d35b640 1 Processor -- start 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f487d35b640 1 -- start start 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f487d35b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100250 0x7f4878199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f487d35b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4878100bb0 0x7f487819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f487d35b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4878101550 0x7f48781a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f487d35b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f487810f690 con 0x7f4878100250 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f487d35b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f487810f510 con 0x7f4878100bb0 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f487d35b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f487810f810 con 0x7f4878101550 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f4876ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100250 0x7f4878199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f4876ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100250 0x7f4878199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34868/0 (socket says 172.21.15.105:34868) 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f4876ffd640 1 -- 172.21.15.105:0/1879299735 learned_addr learned my addr 172.21.15.105:0/1879299735 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f48767fc640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4878100bb0 0x7f487819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.531+0000 7f48777fe640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4878101550 0x7f48781a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f4876ffd640 1 -- 172.21.15.105:0/1879299735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4878101550 msgr2=0x7f48781a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f4876ffd640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4878101550 0x7f48781a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f4876ffd640 1 -- 172.21.15.105:0/1879299735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4878100bb0 msgr2=0x7f487819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f4876ffd640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4878100bb0 0x7f487819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f4876ffd640 1 -- 172.21.15.105:0/1879299735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48781a19e0 con 0x7f4878100250 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f4876ffd640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100250 0x7f4878199dc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f487000ebc0 tx=0x7f487000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f485ffff640 1 -- 172.21.15.105:0/1879299735 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870010630 con 0x7f4878100250 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f487d35b640 1 -- 172.21.15.105:0/1879299735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48781a1cd0 con 0x7f4878100250 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f485ffff640 1 -- 172.21.15.105:0/1879299735 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4870010c80 con 0x7f4878100250 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f485ffff640 1 -- 172.21.15.105:0/1879299735 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870018b30 con 0x7f4878100250 2024-03-26T17:37:48.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.532+0000 7f487d35b640 1 -- 172.21.15.105:0/1879299735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48781a21a0 con 0x7f4878100250 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.534+0000 7f4880021740 1 Processor -- start 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.534+0000 7f4880021740 1 -- start start 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f4880021740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f4880021740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 0x5555c75af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f4880021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75af7d0 0x5555c75b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f4880021740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555c7512270 con 0x5555c75adc40 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f4880021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5555c74bac80 con 0x5555c75af7d0 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f4880021740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5555c750e220 con 0x5555c75aee10 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34870/0 (socket says 172.21.15.105:34870) 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 -- 172.21.15.105:0/2608418976 learned_addr learned my addr 172.21.15.105:0/2608418976 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48767fc640 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75af7d0 0x5555c75b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f485ffff640 1 -- 172.21.15.105:0/1879299735 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f4870018cd0 con 0x7f4878100250 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 -- 172.21.15.105:0/2608418976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 msgr2=0x5555c75af290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 0x5555c75af290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 -- 172.21.15.105:0/2608418976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75af7d0 msgr2=0x5555c75b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75af7d0 0x5555c75b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 -- 172.21.15.105:0/2608418976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c74ccfb0 con 0x5555c75adc40 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48767fc640 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75af7d0 0x5555c75b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f48777fe640 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75ae060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4868009ca0 tx=0x7f4868011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e82fd44d979f315 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f485d7fa640 1 -- 172.21.15.105:0/2608418976 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486801e070 con 0x5555c75adc40 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f485d7fa640 1 -- 172.21.15.105:0/2608418976 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4868009e50 con 0x5555c75adc40 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.535+0000 7f485d7fa640 1 -- 172.21.15.105:0/2608418976 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486800ea30 con 0x5555c75adc40 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f485ffff640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48600a2fb0 0x7f48600a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4876ffd640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48600a2fb0 0x7f48600a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f485ffff640 1 -- 172.21.15.105:0/1879299735 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f48700cfb40 con 0x7f4878100250 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 -- 172.21.15.105:0/2608418976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 msgr2=0x5555c75ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75ae060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4868009ca0 tx=0x7f4868011810 comp rx=0 tx=0).stop 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4876ffd640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48600a2fb0 0x7f48600a5470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f487009bee0 tx=0x7f487006de60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 -- 172.21.15.105:0/2608418976 shutdown_connections 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75af7d0 0x5555c75b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 0x5555c75af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 --2- 172.21.15.105:0/2608418976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75ae060 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 -- 172.21.15.105:0/2608418976 >> 172.21.15.105:0/2608418976 conn(0x5555c75a9030 msgr2=0x5555c75a9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 -- 172.21.15.105:0/2608418976 shutdown_connections 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 -- 172.21.15.105:0/2608418976 wait complete. 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.536+0000 7f4880021740 1 -- start start 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 1 Processor -- start 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75aee10 0x5555c75b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 0x5555c75bf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555c75b67e0 con 0x5555c75aee10 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5555c74cdfc0 con 0x5555c75adc40 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5555c75b6960 con 0x5555c75af7d0 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 10 client.0 _ll_register_callbacks cb 0x5555c74c8770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75aee10 0x5555c75b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48777fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48777fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44712/0 (socket says 172.21.15.105:44712) 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75aee10 0x5555c75b8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34880/0 (socket says 172.21.15.105:34880) 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48777fe640 1 -- 172.21.15.105:0/1054159763 learned_addr learned my addr 172.21.15.105:0/1054159763 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48777fe640 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 0x5555c75bf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48767fc640 1 -- 172.21.15.105:0/1054159763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 msgr2=0x5555c75bf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48767fc640 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 0x5555c75bf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48767fc640 1 -- 172.21.15.105:0/1054159763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 msgr2=0x5555c75b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48767fc640 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48767fc640 1 -- 172.21.15.105:0/1054159763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c72262d0 con 0x5555c75aee10 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48777fe640 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f48767fc640 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75aee10 0x5555c75b8750 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f486c000c00 tx=0x7f486c02cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223a0a50980ed32d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f485aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f485aff5640 1 -- 172.21.15.105:0/1054159763 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c043070 con 0x5555c75aee10 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f485aff5640 1 -- 172.21.15.105:0/1054159763 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f486c005250 con 0x5555c75aee10 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.537+0000 7f4880021740 1 -- 172.21.15.105:0/1054159763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4868009e50 con 0x5555c75aee10 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f485aff5640 1 -- 172.21.15.105:0/1054159763 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f486c04f050 con 0x5555c75aee10 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f4880021740 10 client.15078 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f4880021740 1 -- 172.21.15.105:0/1054159763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5555c72f06b0 con 0x5555c75aee10 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f485aff5640 1 -- 172.21.15.105:0/1054159763 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4868009e50 con 0x5555c75aee10 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f485aff5640 1 -- 172.21.15.105:0/1054159763 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f486c0035f0 con 0x5555c75aee10 2024-03-26T17:37:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f4880021740 10 client.15078 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f4880021740 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f4880021740 20 client.15078 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f4880021740 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5555c75c5030 0x5555c75c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f4880021740 4 client.15078 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f4880021740 1 -- 172.21.15.105:0/1054159763 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555c751f7b0 con 0x5555c75c5030 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.538+0000 7f48777fe640 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5555c75c5030 0x5555c75c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f48777fe640 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5555c75c5030 0x5555c75c64a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ae8f7cf7e2b89d server_cookie=49aaab97a128c7f in_seq=0 out_seq=0 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f485aff5640 10 client.15078 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f485aff5640 1 -- 172.21.15.105:0/1054159763 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5555c751f7b0 con 0x5555c75c5030 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f485aff5640 10 client.15078 handle_command_reply: tid=0 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 client.15078 shutdown 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 -- 172.21.15.105:0/1054159763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75aee10 msgr2=0x5555c75b8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75aee10 0x5555c75b8750 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f486c000c00 tx=0x7f486c02cbb0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 -- 172.21.15.105:0/1054159763 shutdown_connections 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 0x5555c75bf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75aee10 0x5555c75b8750 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5555c75c5030 0x5555c75c64a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 --2- 172.21.15.105:0/1054159763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f485aff5640 0 client.15078 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.539+0000 7f4880021740 1 -- 172.21.15.105:0/1054159763 >> 172.21.15.105:0/1054159763 conn(0x5555c75a9030 msgr2=0x5555c75a8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.540+0000 7f4880021740 1 -- 172.21.15.105:0/1054159763 shutdown_connections 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.540+0000 7f4880021740 1 -- 172.21.15.105:0/1054159763 wait complete. 2024-03-26T17:37:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.540+0000 7f4880021740 20 client.15078 trim_cache size 0 max 16384 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f4880021740 1 Processor -- start 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f4880021740 1 -- start start 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f4880021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f4880021740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 0x5555c75bf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f4880021740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75af7d0 0x5555c75b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f4880021740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555c75c47e0 con 0x5555c75af7d0 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f4880021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5555c75c4eb0 con 0x5555c75adc40 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f4880021740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5555c75c5580 con 0x5555c75aee10 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75af7d0 0x5555c75b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75af7d0 0x5555c75b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34896/0 (socket says 172.21.15.105:34896) 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 -- 172.21.15.105:0/2265587365 learned_addr learned my addr 172.21.15.105:0/2265587365 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48767fc640 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 0x5555c75bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 -- 172.21.15.105:0/2265587365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 msgr2=0x5555c75bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 0x5555c75bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 -- 172.21.15.105:0/2265587365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 msgr2=0x5555c75b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 -- 172.21.15.105:0/2265587365 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555c75ae890 con 0x5555c75af7d0 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.558+0000 7f48777fe640 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75af7d0 0x5555c75b6d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4868011d10 tx=0x7f486801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddd7c153888b541 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f485cff9640 1 -- 172.21.15.105:0/2265587365 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486801e040 con 0x5555c75af7d0 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f485cff9640 1 -- 172.21.15.105:0/2265587365 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486801f040 con 0x5555c75af7d0 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f485cff9640 1 -- 172.21.15.105:0/2265587365 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486800e820 con 0x5555c75af7d0 2024-03-26T17:37:48.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 -- 172.21.15.105:0/2265587365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75af7d0 msgr2=0x5555c75b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75af7d0 0x5555c75b6d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4868011d10 tx=0x7f486801b040 comp rx=0 tx=0).stop 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 -- 172.21.15.105:0/2265587365 shutdown_connections 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75af7d0 0x5555c75b6d90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75aee10 0x5555c75bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 --2- 172.21.15.105:0/2265587365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75adc40 0x5555c75b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 -- 172.21.15.105:0/2265587365 >> 172.21.15.105:0/2265587365 conn(0x5555c75a9030 msgr2=0x5555c75b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 -- 172.21.15.105:0/2265587365 shutdown_connections 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.559+0000 7f4880021740 1 -- 172.21.15.105:0/2265587365 wait complete. 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 1 -- start start 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 1 Processor -- start 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75aee10 0x5555c75bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 0x5555c75be410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555c75c6320 con 0x5555c75adc40 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5555c75c5c50 con 0x5555c75aee10 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5555c75c69f0 con 0x5555c75af7d0 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f4880021740 10 client.0 _ll_register_callbacks cb 0x7f486c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34898/0 (socket says 172.21.15.105:34898) 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 -- 172.21.15.105:0/1197740788 learned_addr learned my addr 172.21.15.105:0/1197740788 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48767fc640 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75aee10 0x5555c75bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 0x5555c75be410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 -- 172.21.15.105:0/1197740788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 msgr2=0x5555c75be410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 0x5555c75be410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 -- 172.21.15.105:0/1197740788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75aee10 msgr2=0x5555c75bded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75aee10 0x5555c75bded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 -- 172.21.15.105:0/1197740788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555c75beb50 con 0x5555c75adc40 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f48777fe640 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75bd990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4868009250 tx=0x7f4868009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9c273797388268 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.560+0000 7f485aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f485aff5640 1 -- 172.21.15.105:0/1197740788 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868041060 con 0x5555c75adc40 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f485aff5640 1 -- 172.21.15.105:0/1197740788 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f4868003870 con 0x5555c75adc40 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 1 -- 172.21.15.105:0/1197740788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486801f040 con 0x5555c75adc40 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f485aff5640 1 -- 172.21.15.105:0/1197740788 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f486801e040 con 0x5555c75adc40 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 10 client.15084 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 1 -- 172.21.15.105:0/1197740788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5555c762aeb0 con 0x5555c75adc40 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f485aff5640 1 -- 172.21.15.105:0/1197740788 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486801f040 con 0x5555c75adc40 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f485aff5640 1 -- 172.21.15.105:0/1197740788 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4868039440 con 0x5555c75adc40 2024-03-26T17:37:48.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 10 client.15084 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 20 client.15084 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5555c762f490 0x5555c762bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 4 client.15084 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f4880021740 1 -- 172.21.15.105:0/1197740788 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "status"}) -- 0x5555c7519b00 con 0x5555c762f490 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.561+0000 7f48767fc640 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5555c762f490 0x5555c762bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f485aff5640 10 client.15084 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f48767fc640 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5555c762f490 0x5555c762bfa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0b22f8f21dba4f4 server_cookie=bea39cf0549d3fc2 in_seq=0 out_seq=0 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f485aff5640 1 -- 172.21.15.105:0/1197740788 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5555c7519b00 con 0x5555c762f490 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f485aff5640 10 client.15084 handle_command_reply: tid=0 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 client.15084 shutdown 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 -- 172.21.15.105:0/1197740788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 msgr2=0x5555c75bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75bd990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4868009250 tx=0x7f4868009970 comp rx=0 tx=0).stop 2024-03-26T17:37:48.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 -- 172.21.15.105:0/1197740788 shutdown_connections 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555c75af7d0 0x5555c75be410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5555c762f490 0x5555c762bfa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555c75aee10 0x5555c75bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f485aff5640 0 client.15084 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 --2- 172.21.15.105:0/1197740788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555c75adc40 0x5555c75bd990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 -- 172.21.15.105:0/1197740788 >> 172.21.15.105:0/1197740788 conn(0x5555c75a9030 msgr2=0x5555c75ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 -- 172.21.15.105:0/1197740788 shutdown_connections 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 1 -- 172.21.15.105:0/1197740788 wait complete. 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.562+0000 7f4880021740 20 client.15084 trim_cache size 0 max 16384 2024-03-26T17:37:48.562 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24482, 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 82.238284820999993, 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 102.697132127 2024-03-26T17:37:48.563 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:37:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 -- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48600a2fb0 msgr2=0x7f48600a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48600a2fb0 0x7f48600a5470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f487009bee0 tx=0x7f487006de60 comp rx=0 tx=0).stop 2024-03-26T17:37:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 -- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100250 msgr2=0x7f4878199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100250 0x7f4878199dc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f487000ebc0 tx=0x7f487000c6a0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 -- 172.21.15.105:0/1879299735 shutdown_connections 2024-03-26T17:37:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48600a2fb0 0x7f48600a5470 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4878101550 0x7f48781a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4878100bb0 0x7f487819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 --2- 172.21.15.105:0/1879299735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4878100250 0x7f4878199dc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 -- 172.21.15.105:0/1879299735 >> 172.21.15.105:0/1879299735 conn(0x7f48780fbdd0 msgr2=0x7f48780fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 -- 172.21.15.105:0/1879299735 shutdown_connections 2024-03-26T17:37:48.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.563+0000 7f487d35b640 1 -- 172.21.15.105:0/1879299735 wait complete. 2024-03-26T17:37:48.566 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T17:37:48.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 -- 172.21.15.105:0/2243482231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c102370 msgr2=0x7f916c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 --2- 172.21.15.105:0/2243482231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c102370 0x7f916c108c00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9164009900 tx=0x7f916402bc30 comp rx=0 tx=0).stop 2024-03-26T17:37:48.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 -- 172.21.15.105:0/2243482231 shutdown_connections 2024-03-26T17:37:48.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 --2- 172.21.15.105:0/2243482231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c102370 0x7f916c108c00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 --2- 172.21.15.105:0/2243482231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c1019b0 0x7f916c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 --2- 172.21.15.105:0/2243482231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f916c1007c0 0x7f916c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 -- 172.21.15.105:0/2243482231 >> 172.21.15.105:0/2243482231 conn(0x7f916c0fbd50 msgr2=0x7f916c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 -- 172.21.15.105:0/2243482231 shutdown_connections 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 -- 172.21.15.105:0/2243482231 wait complete. 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 Processor -- start 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.678+0000 7f9173e36640 1 -- start start 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9173e36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f916c1007c0 0x7f916c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9173e36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c1019b0 0x7f916c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9173e36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c102370 0x7f916c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9173e36640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10c2d0 con 0x7f916c1019b0 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9173e36640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10c150 con 0x7f916c1007c0 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9173e36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10c450 con 0x7f916c102370 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c1019b0 0x7f916c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c1019b0 0x7f916c19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34928/0 (socket says 172.21.15.105:34928) 2024-03-26T17:37:48.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 -- 172.21.15.105:0/3854428983 learned_addr learned my addr 172.21.15.105:0/3854428983 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9171bd2640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f916c1007c0 0x7f916c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91723d3640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c102370 0x7f916c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 -- 172.21.15.105:0/3854428983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c102370 msgr2=0x7f916c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c102370 0x7f916c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 -- 172.21.15.105:0/3854428983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f916c1007c0 msgr2=0x7f916c069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f916c1007c0 0x7f916c069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 -- 172.21.15.105:0/3854428983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c19f780 con 0x7f916c1019b0 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f91713d1640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c1019b0 0x7f916c19c850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f915c00ea60 tx=0x7f915c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9162ffd640 1 -- 172.21.15.105:0/3854428983 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c00cde0 con 0x7f916c1019b0 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9173e36640 1 -- 172.21.15.105:0/3854428983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c19fa10 con 0x7f916c1019b0 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9162ffd640 1 -- 172.21.15.105:0/3854428983 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f915c0035f0 con 0x7f916c1019b0 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9162ffd640 1 -- 172.21.15.105:0/3854428983 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c010690 con 0x7f916c1019b0 2024-03-26T17:37:48.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.679+0000 7f9173e36640 1 -- 172.21.15.105:0/3854428983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c19ff60 con 0x7f916c1019b0 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9162ffd640 1 -- 172.21.15.105:0/3854428983 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f915c0037a0 con 0x7f916c1019b0 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9176a25740 1 Processor -- start 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9176a25740 1 -- start start 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9176a25740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a87f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9176a25740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a87fa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9176a25740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a8801060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9176a25740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c1a875d270 con 0x55c1a87f8c40 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9176a25740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c1a8705c80 con 0x55c1a87fa7d0 2024-03-26T17:37:48.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.682+0000 7f9176a25740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c1a8759220 con 0x55c1a87f9e10 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f91723d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a87f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f91723d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a87f9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34942/0 (socket says 172.21.15.105:34942) 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f91723d3640 1 -- 172.21.15.105:0/3377382195 learned_addr learned my addr 172.21.15.105:0/3377382195 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9171bd2640 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a87fa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f91713d1640 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a8801060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9162ffd640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91580a2fb0 0x7f91580a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9171bd2640 1 -- 172.21.15.105:0/3377382195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 msgr2=0x55c1a8801060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9171bd2640 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a8801060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9171bd2640 1 -- 172.21.15.105:0/3377382195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 msgr2=0x55c1a87f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9171bd2640 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a87f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9171bd2640 1 -- 172.21.15.105:0/3377382195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a8717fb0 con 0x55c1a87f9e10 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f91723d3640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91580a2fb0 0x7f91580a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9162ffd640 1 -- 172.21.15.105:0/3854428983 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f915c0cf7d0 con 0x7f916c1019b0 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9171bd2640 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a87fa290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9168005fd0 tx=0x7f91680118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ba4bc0569d5da28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9157fff640 1 -- 172.21.15.105:0/3377382195 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168004d20 con 0x55c1a87f9e10 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9157fff640 1 -- 172.21.15.105:0/3377382195 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9168003e20 con 0x55c1a87f9e10 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.683+0000 7f9157fff640 1 -- 172.21.15.105:0/3377382195 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916800f540 con 0x55c1a87f9e10 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 -- 172.21.15.105:0/3377382195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 msgr2=0x55c1a87fa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a87fa290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9168005fd0 tx=0x7f91680118d0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f91723d3640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91580a2fb0 0x7f91580a5470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f91640096c0 tx=0x7f9164009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 -- 172.21.15.105:0/3377382195 shutdown_connections 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a8801060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a87fa290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 --2- 172.21.15.105:0/3377382195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a87f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 -- 172.21.15.105:0/3377382195 >> 172.21.15.105:0/3377382195 conn(0x55c1a87f4030 msgr2=0x55c1a87f4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 -- 172.21.15.105:0/3377382195 shutdown_connections 2024-03-26T17:37:48.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 -- 172.21.15.105:0/3377382195 wait complete. 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.684+0000 7f9176a25740 1 -- start start 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9176a25740 1 Processor -- start 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9176a25740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f8c40 0x55c1a8803130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9176a25740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f9e10 0x55c1a8803670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9176a25740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a880a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9176a25740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c1a88017e0 con 0x55c1a87f9e10 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9176a25740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c1a8718fc0 con 0x55c1a87fa7d0 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9176a25740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c1a8801960 con 0x55c1a87f8c40 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9176a25740 10 client.0 _ll_register_callbacks cb 0x55c1a8713770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f9e10 0x55c1a8803670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9171bd2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f8c40 0x55c1a8803130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f9e10 0x55c1a8803670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34946/0 (socket says 172.21.15.105:34946) 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9171bd2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f8c40 0x55c1a8803130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43500/0 (socket says 172.21.15.105:43500) 2024-03-26T17:37:48.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 -- 172.21.15.105:0/680934850 learned_addr learned my addr 172.21.15.105:0/680934850 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91723d3640 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a880a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 -- 172.21.15.105:0/680934850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f8c40 msgr2=0x55c1a8803130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f8c40 0x55c1a8803130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 -- 172.21.15.105:0/680934850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 msgr2=0x55c1a880a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a880a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 -- 172.21.15.105:0/680934850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a84712d0 con 0x55c1a87f9e10 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f9171bd2640 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f8c40 0x55c1a8803130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91713d1640 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f9e10 0x55c1a8803670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f915c0a42a0 tx=0x7f915c0bc050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cdc386e0894834 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.685+0000 7f91557fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f91557fa640 1 -- 172.21.15.105:0/680934850 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c09bb90 con 0x55c1a87f9e10 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f91557fa640 1 -- 172.21.15.105:0/680934850 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f915c09b210 con 0x55c1a87f9e10 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 1 -- 172.21.15.105:0/680934850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9168003e20 con 0x55c1a87f9e10 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f91557fa640 1 -- 172.21.15.105:0/680934850 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f915c07f050 con 0x55c1a87f9e10 2024-03-26T17:37:48.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 10 client.15096 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 1 -- 172.21.15.105:0/680934850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1a853b6b0 con 0x55c1a87f9e10 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f91557fa640 1 -- 172.21.15.105:0/680934850 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9168003e20 con 0x55c1a87f9e10 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f91557fa640 1 -- 172.21.15.105:0/680934850 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f915c004000 con 0x55c1a87f9e10 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 10 client.15096 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 10 client.15096 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 20 client.15096 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55c1a880ff70 0x55c1a88113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 4 client.15096 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9176a25740 1 -- 172.21.15.105:0/680934850 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1a876a7b0 con 0x55c1a880ff70 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.686+0000 7f9171bd2640 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55c1a880ff70 0x55c1a88113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.687+0000 7f91557fa640 10 client.15096 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.687+0000 7f9171bd2640 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55c1a880ff70 0x55c1a88113e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16800f6788e6c326 server_cookie=ecc26a99ecc36382 in_seq=0 out_seq=0 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.687+0000 7f91557fa640 1 -- 172.21.15.105:0/680934850 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c1a876a7b0 con 0x55c1a880ff70 2024-03-26T17:37:48.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.687+0000 7f91557fa640 10 client.15096 handle_command_reply: tid=0 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.687+0000 7f9176a25740 1 client.15096 shutdown 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 -- 172.21.15.105:0/680934850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f9e10 msgr2=0x55c1a8803670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f9e10 0x55c1a8803670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f915c0a42a0 tx=0x7f915c0bc050 comp rx=0 tx=0).stop 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 -- 172.21.15.105:0/680934850 shutdown_connections 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55c1a880ff70 0x55c1a88113e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a880a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f91557fa640 0 client.15096 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f9e10 0x55c1a8803670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 --2- 172.21.15.105:0/680934850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f8c40 0x55c1a8803130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 -- 172.21.15.105:0/680934850 >> 172.21.15.105:0/680934850 conn(0x55c1a87f4030 msgr2=0x55c1a87f3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 -- 172.21.15.105:0/680934850 shutdown_connections 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 1 -- 172.21.15.105:0/680934850 wait complete. 2024-03-26T17:37:48.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.688+0000 7f9176a25740 20 client.15096 trim_cache size 0 max 16384 2024-03-26T17:37:48.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.705+0000 7f9176a25740 1 Processor -- start 2024-03-26T17:37:48.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.705+0000 7f9176a25740 1 -- start start 2024-03-26T17:37:48.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9176a25740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87f8c40 0x55c1a88035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9176a25740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a880a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9176a25740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87fa7d0 0x55c1a8802470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9176a25740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c1a880f720 con 0x55c1a87fa7d0 2024-03-26T17:37:48.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9176a25740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c1a880fdf0 con 0x55c1a87f8c40 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9176a25740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c1a88104c0 con 0x55c1a87f9e10 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91723d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87fa7d0 0x55c1a8802470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91713d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a880a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9171bd2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87f8c40 0x55c1a88035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91713d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a880a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43510/0 (socket says 172.21.15.105:43510) 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91723d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87fa7d0 0x55c1a8802470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34962/0 (socket says 172.21.15.105:34962) 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91713d1640 1 -- 172.21.15.105:0/1992067516 learned_addr learned my addr 172.21.15.105:0/1992067516 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9171bd2640 1 -- 172.21.15.105:0/1992067516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 msgr2=0x55c1a880a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9171bd2640 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a880a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9171bd2640 1 -- 172.21.15.105:0/1992067516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87fa7d0 msgr2=0x55c1a8802470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9171bd2640 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87fa7d0 0x55c1a8802470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9171bd2640 1 -- 172.21.15.105:0/1992067516 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a87f9890 con 0x55c1a87f8c40 2024-03-26T17:37:48.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91713d1640 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a880a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91723d3640 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87fa7d0 0x55c1a8802470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f9171bd2640 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87f8c40 0x55c1a88035d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f916801b040 tx=0x7f91680092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6287d1117b4010df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91577fe640 1 -- 172.21.15.105:0/1992067516 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916801d040 con 0x55c1a87f8c40 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91577fe640 1 -- 172.21.15.105:0/1992067516 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9168009a70 con 0x55c1a87f8c40 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.706+0000 7f91577fe640 1 -- 172.21.15.105:0/1992067516 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916802b360 con 0x55c1a87f8c40 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 -- 172.21.15.105:0/1992067516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87f8c40 msgr2=0x55c1a88035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87f8c40 0x55c1a88035d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f916801b040 tx=0x7f91680092b0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 -- 172.21.15.105:0/1992067516 shutdown_connections 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87fa7d0 0x55c1a8802470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a880a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 --2- 172.21.15.105:0/1992067516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87f8c40 0x55c1a88035d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 -- 172.21.15.105:0/1992067516 >> 172.21.15.105:0/1992067516 conn(0x55c1a87f4030 msgr2=0x55c1a8801d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 -- 172.21.15.105:0/1992067516 shutdown_connections 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 -- 172.21.15.105:0/1992067516 wait complete. 2024-03-26T17:37:48.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 -- start start 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.707+0000 7f9176a25740 1 Processor -- start 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9176a25740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a8808660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9176a25740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a8871cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9176a25740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a8874210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9176a25740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c1a8811260 con 0x55c1a87f8c40 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9176a25740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c1a8810b90 con 0x55c1a87fa7d0 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9176a25740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c1a8811930 con 0x55c1a87f9e10 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9176a25740 10 client.0 _ll_register_callbacks cb 0x7f915c00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a8808660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a8808660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34972/0 (socket says 172.21.15.105:34972) 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 -- 172.21.15.105:0/3710438528 learned_addr learned my addr 172.21.15.105:0/3710438528 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f91713d1640 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a8871cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f91723d3640 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a8874210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 -- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 msgr2=0x55c1a8871cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a8871cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 -- 172.21.15.105:0/3710438528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 msgr2=0x55c1a8874210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a8874210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 -- 172.21.15.105:0/3710438528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a8874a20 con 0x55c1a87f8c40 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f91713d1640 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a8871cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9171bd2640 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a8808660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9168005c10 tx=0x7f91680049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce315c1cdc6adb60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f91557fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f91557fa640 1 -- 172.21.15.105:0/3710438528 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168038070 con 0x55c1a87f8c40 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f91557fa640 1 -- 172.21.15.105:0/3710438528 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f9168004cd0 con 0x55c1a87f8c40 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.708+0000 7f9176a25740 1 -- 172.21.15.105:0/3710438528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9168009a70 con 0x55c1a87f8c40 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f91557fa640 1 -- 172.21.15.105:0/3710438528 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9168044050 con 0x55c1a87f8c40 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f9176a25740 10 client.15102 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f9176a25740 1 -- 172.21.15.105:0/3710438528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1a8875d20 con 0x55c1a87f8c40 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f91557fa640 1 -- 172.21.15.105:0/3710438528 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9168009a70 con 0x55c1a87f8c40 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f91557fa640 1 -- 172.21.15.105:0/3710438528 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9168005550 con 0x55c1a87f8c40 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f9176a25740 10 client.15102 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f9176a25740 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f9176a25740 20 client.15102 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f9176a25740 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55c1a887a200 0x55c1a8878f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f9176a25740 4 client.15102 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f9176a25740 1 -- 172.21.15.105:0/3710438528 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "status"}) -- 0x55c1a8764b00 con 0x55c1a887a200 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.709+0000 7f91713d1640 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55c1a887a200 0x55c1a8878f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f91557fa640 10 client.15102 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f91713d1640 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55c1a887a200 0x55c1a8878f10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3fcfec6dd84f4df server_cookie=cb5998d1211c807d in_seq=0 out_seq=0 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f91557fa640 1 -- 172.21.15.105:0/3710438528 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c1a8764b00 con 0x55c1a887a200 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f91557fa640 10 client.15102 handle_command_reply: tid=0 2024-03-26T17:37:48.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 client.15102 shutdown 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 -- 172.21.15.105:0/3710438528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 msgr2=0x55c1a8808660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a8808660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9168005c10 tx=0x7f91680049c0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 -- 172.21.15.105:0/3710438528 shutdown_connections 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55c1a887a200 0x55c1a8878f10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c1a87fa7d0 0x55c1a8874210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f91557fa640 0 client.15102 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c1a87f9e10 0x55c1a8871cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 --2- 172.21.15.105:0/3710438528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c1a87f8c40 0x55c1a8808660 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 -- 172.21.15.105:0/3710438528 >> 172.21.15.105:0/3710438528 conn(0x55c1a87f4030 msgr2=0x55c1a88025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 -- 172.21.15.105:0/3710438528 shutdown_connections 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 1 -- 172.21.15.105:0/3710438528 wait complete. 2024-03-26T17:37:48.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.710+0000 7f9176a25740 20 client.15102 trim_cache size 0 max 16384 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24487, 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 83.393814562000003, 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:37:48.710 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 100.151900341 2024-03-26T17:37:48.711 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:37:48.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 -- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91580a2fb0 msgr2=0x7f91580a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91580a2fb0 0x7f91580a5470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f91640096c0 tx=0x7f9164009260 comp rx=0 tx=0).stop 2024-03-26T17:37:48.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 -- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c1019b0 msgr2=0x7f916c19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c1019b0 0x7f916c19c850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f915c00ea60 tx=0x7f915c00ef20 comp rx=0 tx=0).stop 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 -- 172.21.15.105:0/3854428983 shutdown_connections 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91580a2fb0 0x7f91580a5470 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916c102370 0x7f916c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f916c1019b0 0x7f916c19c850 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 --2- 172.21.15.105:0/3854428983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f916c1007c0 0x7f916c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 -- 172.21.15.105:0/3854428983 >> 172.21.15.105:0/3854428983 conn(0x7f916c0fbd50 msgr2=0x7f916c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 -- 172.21.15.105:0/3854428983 shutdown_connections 2024-03-26T17:37:48.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.711+0000 7f9173e36640 1 -- 172.21.15.105:0/3854428983 wait complete. 2024-03-26T17:37:48.718 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T17:37:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.841+0000 7f8759897640 1 -- 172.21.15.105:0/523517973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 msgr2=0x7f8754108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.841+0000 7f8759897640 1 --2- 172.21.15.105:0/523517973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 0x7f8754108910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f874c009ca0 tx=0x7f874c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.841+0000 7f8759897640 1 -- 172.21.15.105:0/523517973 shutdown_connections 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.841+0000 7f8759897640 1 --2- 172.21.15.105:0/523517973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 0x7f8754108910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.841+0000 7f8759897640 1 --2- 172.21.15.105:0/523517973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541039b0 0x7f8754105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.841+0000 7f8759897640 1 --2- 172.21.15.105:0/523517973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8754100ff0 0x7f87541033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.841+0000 7f8759897640 1 -- 172.21.15.105:0/523517973 >> 172.21.15.105:0/523517973 conn(0x7f87540fb080 msgr2=0x7f87540fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 -- 172.21.15.105:0/523517973 shutdown_connections 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 -- 172.21.15.105:0/523517973 wait complete. 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 Processor -- start 2024-03-26T17:37:48.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 -- start start 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8754100ff0 0x7f8754100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541039b0 0x7f875419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 0x7f875419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f875410c350 con 0x7f8754106450 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f875410c1d0 con 0x7f87541039b0 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f8759897640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f875410c4d0 con 0x7f8754100ff0 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.842+0000 7f87537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 0x7f875419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f8752ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8754100ff0 0x7f8754100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 0x7f875419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34994/0 (socket says 172.21.15.105:34994) 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87537fe640 1 -- 172.21.15.105:0/851992108 learned_addr learned my addr 172.21.15.105:0/851992108 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87527fc640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541039b0 0x7f875419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87537fe640 1 -- 172.21.15.105:0/851992108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8754100ff0 msgr2=0x7f8754100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87537fe640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8754100ff0 0x7f8754100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87537fe640 1 -- 172.21.15.105:0/851992108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541039b0 msgr2=0x7f875419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87537fe640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541039b0 0x7f875419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87537fe640 1 -- 172.21.15.105:0/851992108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875419f780 con 0x7f8754106450 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f87537fe640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 0x7f875419eec0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f874c02c700 tx=0x7f874c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f8758895640 1 -- 172.21.15.105:0/851992108 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c041020 con 0x7f8754106450 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f8759897640 1 -- 172.21.15.105:0/851992108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875419fa10 con 0x7f8754106450 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f8758895640 1 -- 172.21.15.105:0/851992108 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f874c003e20 con 0x7f8754106450 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f8758895640 1 -- 172.21.15.105:0/851992108 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c039040 con 0x7f8754106450 2024-03-26T17:37:48.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.843+0000 7f8759897640 1 -- 172.21.15.105:0/851992108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875419ff50 con 0x7f8754106450 2024-03-26T17:37:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.845+0000 7f8758895640 1 -- 172.21.15.105:0/851992108 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f874c044020 con 0x7f8754106450 2024-03-26T17:37:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f875c483740 1 Processor -- start 2024-03-26T17:37:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f875c483740 1 -- start start 2024-03-26T17:37:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f875c483740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9d7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f875c483740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9d8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f875c483740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9df060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f875c483740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b19b93b270 con 0x55b19b9d6c40 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f875c483740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b19b8e3c80 con 0x55b19b9d87d0 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f875c483740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b19b937220 con 0x55b19b9d7e10 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f87527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9d7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f87527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9d7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35002/0 (socket says 172.21.15.105:35002) 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f87527fc640 1 -- 172.21.15.105:0/1456996206 learned_addr learned my addr 172.21.15.105:0/1456996206 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f8752ffd640 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f8758895640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f873c0a2fb0 0x7f873c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f87527fc640 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f87537fe640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f873c0a2fb0 0x7f873c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f8752ffd640 1 -- 172.21.15.105:0/1456996206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 msgr2=0x55b19b9df060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f8752ffd640 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9df060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f8752ffd640 1 -- 172.21.15.105:0/1456996206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 msgr2=0x55b19b9d7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f8752ffd640 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9d7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f8752ffd640 1 -- 172.21.15.105:0/1456996206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19b8f5fb0 con 0x55b19b9d7e10 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.846+0000 7f8758895640 1 -- 172.21.15.105:0/851992108 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f874c0f15f0 con 0x7f8754106450 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f8752ffd640 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9d8290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8744009c00 tx=0x7f8744011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa44c9fd8b42eea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f87537fe640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f873c0a2fb0 0x7f873c0a5470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f874c08f940 tx=0x7f874c0e89b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f8739ffb640 1 -- 172.21.15.105:0/1456996206 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744004230 con 0x55b19b9d7e10 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f8739ffb640 1 -- 172.21.15.105:0/1456996206 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87440043d0 con 0x55b19b9d7e10 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f8739ffb640 1 -- 172.21.15.105:0/1456996206 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744004ca0 con 0x55b19b9d7e10 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f875c483740 1 -- 172.21.15.105:0/1456996206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 msgr2=0x55b19b9d8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f875c483740 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9d8290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8744009c00 tx=0x7f8744011810 comp rx=0 tx=0).stop 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f875c483740 1 -- 172.21.15.105:0/1456996206 shutdown_connections 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f875c483740 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9df060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f875c483740 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9d8290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f875c483740 1 --2- 172.21.15.105:0/1456996206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9d7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f875c483740 1 -- 172.21.15.105:0/1456996206 >> 172.21.15.105:0/1456996206 conn(0x55b19b9d2030 msgr2=0x55b19b9d2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.847+0000 7f875c483740 1 -- 172.21.15.105:0/1456996206 shutdown_connections 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 -- 172.21.15.105:0/1456996206 wait complete. 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 -- start start 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 Processor -- start 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b19b9df7e0 con 0x55b19b9d6c40 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b19b8f6fc0 con 0x55b19b9d87d0 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b19b9df960 con 0x55b19b9d7e10 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f875c483740 10 client.0 _ll_register_callbacks cb 0x55b19b8f1770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f87527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f87527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9e11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35014/0 (socket says 172.21.15.105:35014) 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f87527fc640 1 -- 172.21.15.105:0/2443705562 learned_addr learned my addr 172.21.15.105:0/2443705562 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f8752ffd640 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.848+0000 7f87527fc640 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f87527fc640 1 -- 172.21.15.105:0/2443705562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 msgr2=0x55b19b9e16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f87527fc640 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f87527fc640 1 -- 172.21.15.105:0/2443705562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 msgr2=0x55b19b9e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f87527fc640 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f87527fc640 1 -- 172.21.15.105:0/2443705562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b19b64f2d0 con 0x55b19b9d6c40 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f87527fc640 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9e11b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8748013240 tx=0x7f8748005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538fe255c86acb2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f8732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f8732ffd640 1 -- 172.21.15.105:0/2443705562 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748004030 con 0x55b19b9d6c40 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f8732ffd640 1 -- 172.21.15.105:0/2443705562 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f87480041d0 con 0x55b19b9d6c40 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f875c483740 1 -- 172.21.15.105:0/2443705562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87440043d0 con 0x55b19b9d6c40 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f8732ffd640 1 -- 172.21.15.105:0/2443705562 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8748028020 con 0x55b19b9d6c40 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f875c483740 10 client.15114 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f875c483740 1 -- 172.21.15.105:0/2443705562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b19b7196b0 con 0x55b19b9d6c40 2024-03-26T17:37:48.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f8732ffd640 1 -- 172.21.15.105:0/2443705562 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87440043d0 con 0x55b19b9d6c40 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f8732ffd640 1 -- 172.21.15.105:0/2443705562 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f87480048c0 con 0x55b19b9d6c40 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f875c483740 10 client.15114 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f875c483740 10 client.15114 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.849+0000 7f875c483740 20 client.15114 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.850+0000 7f875c483740 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b19b9ee060 0x55b19b9ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.850+0000 7f875c483740 4 client.15114 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.850+0000 7f875c483740 1 -- 172.21.15.105:0/2443705562 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b19b9487b0 con 0x55b19b9ee060 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.850+0000 7f8752ffd640 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b19b9ee060 0x55b19b9ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.850+0000 7f8732ffd640 10 client.15114 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.850+0000 7f8752ffd640 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b19b9ee060 0x55b19b9ef4d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bef944b2c25ebd2 server_cookie=82ab1c84d5105d60 in_seq=0 out_seq=0 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f8732ffd640 1 -- 172.21.15.105:0/2443705562 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b19b9487b0 con 0x55b19b9ee060 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f8732ffd640 10 client.15114 handle_command_reply: tid=0 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 client.15114 shutdown 2024-03-26T17:37:48.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 -- 172.21.15.105:0/2443705562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 msgr2=0x55b19b9e11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9e11b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8748013240 tx=0x7f8748005850 comp rx=0 tx=0).stop 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 -- 172.21.15.105:0/2443705562 shutdown_connections 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b19b9ee060 0x55b19b9ef4d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 --2- 172.21.15.105:0/2443705562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9e11b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 -- 172.21.15.105:0/2443705562 >> 172.21.15.105:0/2443705562 conn(0x55b19b9d2030 msgr2=0x55b19b9d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f8732ffd640 0 client.15114 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 -- 172.21.15.105:0/2443705562 shutdown_connections 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.851+0000 7f875c483740 1 -- 172.21.15.105:0/2443705562 wait complete. 2024-03-26T17:37:48.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.852+0000 7f875c483740 20 client.15114 trim_cache size 0 max 16384 2024-03-26T17:37:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.869+0000 7f875c483740 1 Processor -- start 2024-03-26T17:37:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.869+0000 7f875c483740 1 -- start start 2024-03-26T17:37:48.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f875c483740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f875c483740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f875c483740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9db950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f875c483740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b19b9ed810 con 0x55b19b9d6c40 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f875c483740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b19b9edee0 con 0x55b19b9d87d0 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f875c483740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b19b9ee5b0 con 0x55b19b9d7e10 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9defa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35028/0 (socket says 172.21.15.105:35028) 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 -- 172.21.15.105:0/3081895868 learned_addr learned my addr 172.21.15.105:0/3081895868 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f8752ffd640 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9db950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 -- 172.21.15.105:0/3081895868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 msgr2=0x55b19b9e86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 -- 172.21.15.105:0/3081895868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 msgr2=0x55b19b9db950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9db950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 -- 172.21.15.105:0/3081895868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19b9d7890 con 0x55b19b9d6c40 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f8752ffd640 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87527fc640 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9defa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8748004d00 tx=0x7f87480059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf83b7cee325742f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87397fa640 1 -- 172.21.15.105:0/3081895868 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748034020 con 0x55b19b9d6c40 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87397fa640 1 -- 172.21.15.105:0/3081895868 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8748005450 con 0x55b19b9d6c40 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.870+0000 7f87397fa640 1 -- 172.21.15.105:0/3081895868 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748000e60 con 0x55b19b9d6c40 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 -- 172.21.15.105:0/3081895868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 msgr2=0x55b19b9defa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9defa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8748004d00 tx=0x7f87480059b0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 -- 172.21.15.105:0/3081895868 shutdown_connections 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d87d0 0x55b19b9db950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 --2- 172.21.15.105:0/3081895868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d6c40 0x55b19b9defa0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 -- 172.21.15.105:0/3081895868 >> 172.21.15.105:0/3081895868 conn(0x55b19b9d2030 msgr2=0x55b19b9d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 -- 172.21.15.105:0/3081895868 shutdown_connections 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 -- 172.21.15.105:0/3081895868 wait complete. 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 -- start start 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.871+0000 7f875c483740 1 Processor -- start 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f875c483740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d6c40 0x55b19b9e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f875c483740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f875c483740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d87d0 0x55b19b9e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f875c483740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b19b9ef350 con 0x55b19b9d87d0 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f875c483740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b19b9eec80 con 0x55b19b9d6c40 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f875c483740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b19b9efa20 con 0x55b19b9d7e10 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f875c483740 10 client.0 _ll_register_callbacks cb 0x7f8748013900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d87d0 0x55b19b9e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d87d0 0x55b19b9e7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35042/0 (socket says 172.21.15.105:35042) 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 -- 172.21.15.105:0/2099383800 learned_addr learned my addr 172.21.15.105:0/2099383800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:48.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f8752ffd640 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d6c40 0x55b19b9e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 -- 172.21.15.105:0/2099383800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 msgr2=0x55b19b9e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 -- 172.21.15.105:0/2099383800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d6c40 msgr2=0x55b19b9e3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d6c40 0x55b19b9e3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 -- 172.21.15.105:0/2099383800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b19b9e7730 con 0x55b19b9d87d0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f8752ffd640 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f87527fc640 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d87d0 0x55b19b9e7140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8748004ef0 tx=0x7f8748013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2814455bf5ca19 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f8732ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.872+0000 7f8732ffd640 1 -- 172.21.15.105:0/2099383800 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748034070 con 0x55b19b9d87d0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f8732ffd640 1 -- 172.21.15.105:0/2099383800 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f874802d650 con 0x55b19b9d87d0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 1 -- 172.21.15.105:0/2099383800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8748005450 con 0x55b19b9d87d0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f8732ffd640 1 -- 172.21.15.105:0/2099383800 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8748042050 con 0x55b19b9d87d0 2024-03-26T17:37:48.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 10 client.15120 fetch_fsmap learned FSMap version 26 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 1 -- 172.21.15.105:0/2099383800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b19b9e8a30 con 0x55b19b9d87d0 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f8732ffd640 1 -- 172.21.15.105:0/2099383800 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8748005450 con 0x55b19b9d87d0 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f8732ffd640 1 -- 172.21.15.105:0/2099383800 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8748039040 con 0x55b19b9d87d0 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 10 client.15120 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 20 client.15120 populate_metadata read hostname 'smithi105' 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b19ba584c0 0x55b19ba54fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 4 client.15120 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f875c483740 1 -- 172.21.15.105:0/2099383800 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "status"}) -- 0x55b19b942b00 con 0x55b19ba584c0 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.873+0000 7f8752ffd640 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b19ba584c0 0x55b19ba54fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f8732ffd640 10 client.15120 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f8752ffd640 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b19ba584c0 0x55b19ba54fd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9775e5e79cbc5252 server_cookie=c22a65bed2553c38 in_seq=0 out_seq=0 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f8732ffd640 1 -- 172.21.15.105:0/2099383800 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b19b942b00 con 0x55b19ba584c0 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f8732ffd640 10 client.15120 handle_command_reply: tid=0 2024-03-26T17:37:48.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 client.15120 shutdown 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 -- 172.21.15.105:0/2099383800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d87d0 msgr2=0x55b19b9e7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d87d0 0x55b19b9e7140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8748004ef0 tx=0x7f8748013c50 comp rx=0 tx=0).stop 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 -- 172.21.15.105:0/2099383800 shutdown_connections 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b19ba584c0 0x55b19ba54fd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b19b9d87d0 0x55b19b9e7140 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b19b9d7e10 0x55b19b9e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f8732ffd640 0 client.15120 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 --2- 172.21.15.105:0/2099383800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b19b9d6c40 0x55b19b9e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 -- 172.21.15.105:0/2099383800 >> 172.21.15.105:0/2099383800 conn(0x55b19b9d2030 msgr2=0x55b19b9db950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 -- 172.21.15.105:0/2099383800 shutdown_connections 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 1 -- 172.21.15.105:0/2099383800 wait complete. 2024-03-26T17:37:48.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.874+0000 7f875c483740 20 client.15120 trim_cache size 0 max 16384 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24521, 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 86.140956234000001, 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:37:48.875 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 92.119015949000001 2024-03-26T17:37:48.876 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:37:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 -- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f873c0a2fb0 msgr2=0x7f873c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f873c0a2fb0 0x7f873c0a5470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f874c08f940 tx=0x7f874c0e89b0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 -- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 msgr2=0x7f875419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 0x7f875419eec0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f874c02c700 tx=0x7f874c0328a0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 -- 172.21.15.105:0/851992108 shutdown_connections 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f873c0a2fb0 0x7f873c0a5470 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8754106450 0x7f875419eec0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541039b0 0x7f875419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 --2- 172.21.15.105:0/851992108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8754100ff0 0x7f8754100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 -- 172.21.15.105:0/851992108 >> 172.21.15.105:0/851992108 conn(0x7f87540fb080 msgr2=0x7f8754101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 -- 172.21.15.105:0/851992108 shutdown_connections 2024-03-26T17:37:48.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:48.875+0000 7f8759897640 1 -- 172.21.15.105:0/851992108 wait complete. 2024-03-26T17:37:48.878 INFO:tasks.cephadm:Setup complete, yielding 2024-03-26T17:37:48.879 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T17:37:48.891 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T17:37:48.891 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-26T17:37:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:48 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1677426716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:49.135 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:48 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1677426716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:49.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:48 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1677426716' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:37:49.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:49 smithi105 ceph-mon[32280]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:37:50.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:49 smithi144 ceph-mon[36095]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:37:50.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:49 smithi135 ceph-mon[35636]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:37:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 -- 172.21.15.105:0/1158452481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 msgr2=0x7f13a410a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 --2- 172.21.15.105:0/1158452481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 0x7f13a410a530 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f139c00b0a0 tx=0x7f139c00b3b0 comp rx=0 tx=0).stop 2024-03-26T17:37:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 -- 172.21.15.105:0/1158452481 shutdown_connections 2024-03-26T17:37:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 --2- 172.21.15.105:0/1158452481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 0x7f13a410a530 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 --2- 172.21.15.105:0/1158452481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a4102ea0 0x7f13a4105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 --2- 172.21.15.105:0/1158452481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a41057d0 0x7f13a4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 -- 172.21.15.105:0/1158452481 >> 172.21.15.105:0/1158452481 conn(0x7f13a40fc870 msgr2=0x7f13a40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 -- 172.21.15.105:0/1158452481 shutdown_connections 2024-03-26T17:37:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.757+0000 7f13aa135700 1 -- 172.21.15.105:0/1158452481 wait complete. 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.758+0000 7f13aa135700 1 Processor -- start 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.758+0000 7f13aa135700 1 -- start start 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.758+0000 7f13aa135700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a4102ea0 0x7f13a4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.758+0000 7f13aa135700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a41057d0 0x7f13a41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.758+0000 7f13aa135700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 0x7f13a419fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.758+0000 7f13aa135700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13a410de90 con 0x7f13a4108100 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.758+0000 7f13aa135700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13a410dd10 con 0x7f13a4102ea0 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.758+0000 7f13aa135700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13a410e010 con 0x7f13a41057d0 2024-03-26T17:37:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a3fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 0x7f13a419fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a3fff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 0x7f13a419fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35070/0 (socket says 172.21.15.105:35070) 2024-03-26T17:37:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a3fff700 1 -- 172.21.15.105:0/1391507301 learned_addr learned my addr 172.21.15.105:0/1391507301 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a41057d0 0x7f13a41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a37fe700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a4102ea0 0x7f13a4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a37fe700 1 -- 172.21.15.105:0/1391507301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a41057d0 msgr2=0x7f13a41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a37fe700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a41057d0 0x7f13a41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a37fe700 1 -- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 msgr2=0x7f13a419fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a37fe700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 0x7f13a419fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a37fe700 1 -- 172.21.15.105:0/1391507301 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a41a00d0 con 0x7f13a4102ea0 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a2ffd700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a41057d0 0x7f13a41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a3fff700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 0x7f13a419fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a37fe700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a4102ea0 0x7f13a4198470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f139400e9f0 tx=0x7f139400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a0ff9700 1 -- 172.21.15.105:0/1391507301 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394004220 con 0x7f13a4102ea0 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a0ff9700 1 -- 172.21.15.105:0/1391507301 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13940043c0 con 0x7f13a4102ea0 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a41a03c0 con 0x7f13a4102ea0 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13a0ff9700 1 -- 172.21.15.105:0/1391507301 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394005020 con 0x7f13a4102ea0 2024-03-26T17:37:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.759+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41a08b0 con 0x7f13a4102ea0 2024-03-26T17:37:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.760+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a4069070 con 0x7f13a4102ea0 2024-03-26T17:37:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.762+0000 7f13a0ff9700 1 -- 172.21.15.105:0/1391507301 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f1394042b30 con 0x7f13a4102ea0 2024-03-26T17:37:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.763+0000 7f13a0ff9700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a3040 0x7f138c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.763+0000 7f13a2ffd700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a3040 0x7f138c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.763+0000 7f13a0ff9700 1 -- 172.21.15.105:0/1391507301 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f13940cf4f0 con 0x7f13a4102ea0 2024-03-26T17:37:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.763+0000 7f13a2ffd700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a3040 0x7f138c0a5500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1398011490 tx=0x7f1398009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:50.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:50.766+0000 7f13a0ff9700 1 -- 172.21.15.105:0/1391507301 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1394042e20 con 0x7f13a4102ea0 2024-03-26T17:37:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.014+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f13a40627f0 con 0x7f138c0a3040 2024-03-26T17:37:51.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.015+0000 7f13a0ff9700 1 -- 172.21.15.105:0/1391507301 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f13a40627f0 con 0x7f138c0a3040 2024-03-26T17:37:51.008 INFO:teuthology.orchestra.run.smithi105.stdout:Backend: cephadm 2024-03-26T17:37:51.008 INFO:teuthology.orchestra.run.smithi105.stdout:Available: Yes 2024-03-26T17:37:51.008 INFO:teuthology.orchestra.run.smithi105.stdout:Paused: No 2024-03-26T17:37:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a3040 msgr2=0x7f138c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a3040 0x7f138c0a5500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1398011490 tx=0x7f1398009260 comp rx=0 tx=0).stop 2024-03-26T17:37:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a4102ea0 msgr2=0x7f13a4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a4102ea0 0x7f13a4198470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f139400e9f0 tx=0x7f139400ed00 comp rx=0 tx=0).stop 2024-03-26T17:37:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 shutdown_connections 2024-03-26T17:37:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a3040 0x7f138c0a5500 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a4108100 0x7f13a419fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:51.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a4102ea0 0x7f13a4198470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 --2- 172.21.15.105:0/1391507301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a41057d0 0x7f13a41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.017+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 >> 172.21.15.105:0/1391507301 conn(0x7f13a40fc870 msgr2=0x7f13a4103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.018+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 shutdown_connections 2024-03-26T17:37:51.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:51.018+0000 7f13aa135700 1 -- 172.21.15.105:0/1391507301 wait complete. 2024-03-26T17:37:51.659 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-26T17:37:51.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:51 smithi105 ceph-mon[32280]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:51.919 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:52.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:51 smithi144 ceph-mon[36095]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:51 smithi135 ceph-mon[35636]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:53.062 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:52 smithi105 ceph-mon[32280]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:53.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:52 smithi144 ceph-mon[36095]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:52 smithi135 ceph-mon[35636]: from='client.24995 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/2282613455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 msgr2=0x7fa6d4107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/2282613455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 0x7fa6d4107bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc009990 tx=0x7fa6bc009ca0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/2282613455 shutdown_connections 2024-03-26T17:37:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/2282613455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4108130 0x7fa6d410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/2282613455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4102ed0 0x7fa6d41052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/2282613455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 0x7fa6d4107bf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/2282613455 >> 172.21.15.105:0/2282613455 conn(0x7fa6d40fc8a0 msgr2=0x7fa6d40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/2282613455 shutdown_connections 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.610+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/2282613455 wait complete. 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d8f2b700 1 Processor -- start 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d8f2b700 1 -- start start 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d8f2b700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4102ed0 0x7fa6d4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d8f2b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 0x7fa6d4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d8f2b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4108130 0x7fa6d419fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d8f2b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410dea0 con 0x7fa6d4105800 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d8f2b700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410dd20 con 0x7fa6d4102ed0 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d8f2b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410e020 con 0x7fa6d4108130 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d1d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 0x7fa6d4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d1d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 0x7fa6d4198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35084/0 (socket says 172.21.15.105:35084) 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d1d9b700 1 -- 172.21.15.105:0/3389504537 learned_addr learned my addr 172.21.15.105:0/3389504537 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:53.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d259c700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4102ed0 0x7fa6d4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.611+0000 7fa6d2d9d700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4108130 0x7fa6d419fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6d1d9b700 1 -- 172.21.15.105:0/3389504537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4108130 msgr2=0x7fa6d419fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6d1d9b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4108130 0x7fa6d419fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6d1d9b700 1 -- 172.21.15.105:0/3389504537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4102ed0 msgr2=0x7fa6d4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6d1d9b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4102ed0 0x7fa6d4198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6d1d9b700 1 -- 172.21.15.105:0/3389504537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d41a01c0 con 0x7fa6d4105800 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6d1d9b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 0x7fa6d4198aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005910 tx=0x7fa6bc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6cb7fe700 1 -- 172.21.15.105:0/3389504537 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc026070 con 0x7fa6d4105800 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d41a0450 con 0x7fa6d4105800 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6cb7fe700 1 -- 172.21.15.105:0/3389504537 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6bc021410 con 0x7fa6d4105800 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6cb7fe700 1 -- 172.21.15.105:0/3389504537 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc017650 con 0x7fa6d4105800 2024-03-26T17:37:53.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.612+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d41a0900 con 0x7fa6d4105800 2024-03-26T17:37:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.613+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d4069070 con 0x7fa6d4105800 2024-03-26T17:37:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.615+0000 7fa6cb7fe700 1 -- 172.21.15.105:0/3389504537 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7fa6bc029070 con 0x7fa6d4105800 2024-03-26T17:37:53.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.616+0000 7fa6cb7fe700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6c00a30c0 0x7fa6c00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.616+0000 7fa6d259c700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6c00a30c0 0x7fa6c00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.616+0000 7fa6cb7fe700 1 -- 172.21.15.105:0/3389504537 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fa6bc0d0a70 con 0x7fa6d4105800 2024-03-26T17:37:53.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.617+0000 7fa6d259c700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6c00a30c0 0x7fa6c00a5580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00f540 tx=0x7fa6cc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:53.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.619+0000 7fa6cb7fe700 1 -- 172.21.15.105:0/3389504537 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa6bc030070 con 0x7fa6d4105800 2024-03-26T17:37:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.844+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa6d40627f0 con 0x7fa6c00a30c0 2024-03-26T17:37:53.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.858+0000 7fa6cb7fe700 1 -- 172.21.15.105:0/3389504537 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fa6d40627f0 con 0x7fa6c00a30c0 2024-03-26T17:37:53.851 INFO:teuthology.orchestra.run.smithi105.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-26T17:37:53.851 INFO:teuthology.orchestra.run.smithi105.stdout:mds.a smithi105 running (100s) 16s ago 99s 17.7M - 19.0.0-2311-g36051606 497bb87bc5da 4276e5e49d54 2024-03-26T17:37:53.851 INFO:teuthology.orchestra.run.smithi105.stdout:mds.b smithi135 running (97s) 30s ago 97s 26.6M - 19.0.0-2311-g36051606 497bb87bc5da 3d18c077baf5 2024-03-26T17:37:53.851 INFO:teuthology.orchestra.run.smithi105.stdout:mds.c smithi144 running (111s) 30s ago 110s 20.1M - 19.0.0-2311-g36051606 497bb87bc5da 04c50fd54e37 2024-03-26T17:37:53.851 INFO:teuthology.orchestra.run.smithi105.stdout:mds.d smithi105 running (118s) 16s ago 118s 14.0M - 19.0.0-2311-g36051606 497bb87bc5da c526cc4369ab 2024-03-26T17:37:53.851 INFO:teuthology.orchestra.run.smithi105.stdout:mds.e smithi135 running (116s) 30s ago 115s 19.2M - 19.0.0-2311-g36051606 497bb87bc5da 8a72d34a3e5e 2024-03-26T17:37:53.851 INFO:teuthology.orchestra.run.smithi105.stdout:mds.f smithi144 running (113s) 30s ago 113s 16.9M - 19.0.0-2311-g36051606 497bb87bc5da 8332b20c980f 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mds.g smithi105 running (102s) 16s ago 102s 19.4M - 19.0.0-2311-g36051606 497bb87bc5da ce2f700b9024 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mds.h smithi135 running (108s) 30s ago 107s 18.0M - 19.0.0-2311-g36051606 497bb87bc5da 6677b0ca0560 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mds.i smithi144 running (105s) 30s ago 105s 17.4M - 19.0.0-2311-g36051606 497bb87bc5da fe945a7b850f 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mds.j smithi105 running (2m) 16s ago 2m 16.5M - 19.0.0-2311-g36051606 497bb87bc5da 772e1ba46c65 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mds.k smithi135 running (2m) 30s ago 2m 16.5M - 19.0.0-2311-g36051606 497bb87bc5da 34eb9a26b5ab 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mds.l smithi144 running (2m) 30s ago 2m 19.2M - 19.0.0-2311-g36051606 497bb87bc5da a7b6aeb1d2e2 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mgr.x smithi105 *:9283,8765 running (9m) 16s ago 9m 513M - 19.0.0-2311-g36051606 497bb87bc5da f6be31cfa680 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mgr.y smithi135 *:8443,8765 running (7m) 30s ago 7m 453M - 19.0.0-2311-g36051606 497bb87bc5da ef9dc14d2417 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mgr.z smithi144 *:8443,8765 running (7m) 30s ago 7m 453M - 19.0.0-2311-g36051606 497bb87bc5da 89c606a2afae 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mon.a smithi105 running (9m) 16s ago 9m 50.0M 2048M 19.0.0-2311-g36051606 497bb87bc5da beae2e9cfeec 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mon.b smithi135 running (7m) 30s ago 7m 43.2M 2048M 19.0.0-2311-g36051606 497bb87bc5da efb123e3c39f 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:mon.c smithi144 running (7m) 30s ago 7m 42.1M 2048M 19.0.0-2311-g36051606 497bb87bc5da e121d0b2a0cf 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:osd.0 smithi105 running (6m) 16s ago 6m 51.4M 4272M 19.0.0-2311-g36051606 497bb87bc5da 1e6d9cd1e8da 2024-03-26T17:37:53.852 INFO:teuthology.orchestra.run.smithi105.stdout:osd.1 smithi135 running (6m) 30s ago 6m 48.7M 4272M 19.0.0-2311-g36051606 497bb87bc5da 2f44de1ee684 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.2 smithi144 running (6m) 30s ago 6m 61.2M 4272M 19.0.0-2311-g36051606 497bb87bc5da 23b284441229 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.3 smithi105 running (5m) 16s ago 5m 46.6M 4272M 19.0.0-2311-g36051606 497bb87bc5da 4f5aeb510415 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.4 smithi135 running (5m) 30s ago 5m 56.5M 4272M 19.0.0-2311-g36051606 497bb87bc5da 7694c0d225dd 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.5 smithi144 running (4m) 30s ago 4m 56.5M 4272M 19.0.0-2311-g36051606 497bb87bc5da 92990fba8e87 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.6 smithi105 running (4m) 16s ago 4m 49.5M 4272M 19.0.0-2311-g36051606 497bb87bc5da f05eaf211112 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.7 smithi135 running (4m) 30s ago 4m 58.1M 4272M 19.0.0-2311-g36051606 497bb87bc5da 6d56f89f29cb 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.8 smithi144 running (3m) 30s ago 3m 44.9M 4272M 19.0.0-2311-g36051606 497bb87bc5da 3777dac6b6fc 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.9 smithi105 running (3m) 16s ago 3m 50.9M 4272M 19.0.0-2311-g36051606 497bb87bc5da 70f2f136e305 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.10 smithi135 running (2m) 30s ago 2m 49.9M 4272M 19.0.0-2311-g36051606 497bb87bc5da edad98145e63 2024-03-26T17:37:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:osd.11 smithi144 running (2m) 30s ago 2m 50.4M 4272M 19.0.0-2311-g36051606 497bb87bc5da 0b651b33db3c 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6c00a30c0 msgr2=0x7fa6c00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6c00a30c0 0x7fa6c00a5580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00f540 tx=0x7fa6cc00b040 comp rx=0 tx=0).stop 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 msgr2=0x7fa6d4198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 0x7fa6d4198aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc005910 tx=0x7fa6bc009580 comp rx=0 tx=0).stop 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 shutdown_connections 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6c00a30c0 0x7fa6c00a5580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4108130 0x7fa6d419fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4102ed0 0x7fa6d4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 --2- 172.21.15.105:0/3389504537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4105800 0x7fa6d4198aa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 >> 172.21.15.105:0/3389504537 conn(0x7fa6d40fc8a0 msgr2=0x7fa6d4103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 shutdown_connections 2024-03-26T17:37:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:53.860+0000 7fa6d8f2b700 1 -- 172.21.15.105:0/3389504537 wait complete. 2024-03-26T17:37:53.864 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:53 smithi105 ceph-mon[32280]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T17:37:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:53 smithi144 ceph-mon[36095]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T17:37:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:53 smithi135 ceph-mon[35636]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T17:37:54.563 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-26T17:37:54.815 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:56.076 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:55 smithi105 ceph-mon[32280]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:56.076 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:55 smithi105 ceph-mon[32280]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:56.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:55 smithi144 ceph-mon[36095]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:56.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:55 smithi144 ceph-mon[36095]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:56.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:55 smithi135 ceph-mon[35636]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:56.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:55 smithi135 ceph-mon[35636]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:37:56.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 -- 172.21.15.105:0/576598300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75101012d0 msgr2=0x7f75101016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 --2- 172.21.15.105:0/576598300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75101012d0 0x7f75101016f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f75000099c0 tx=0x7f7500009cd0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 -- 172.21.15.105:0/576598300 shutdown_connections 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 --2- 172.21.15.105:0/576598300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7510102e70 0x7f7510109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 --2- 172.21.15.105:0/576598300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75101012d0 0x7f75101016f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 --2- 172.21.15.105:0/576598300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75101024d0 0x7f7510102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 -- 172.21.15.105:0/576598300 >> 172.21.15.105:0/576598300 conn(0x7f75100fc870 msgr2=0x7f75100fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 -- 172.21.15.105:0/576598300 shutdown_connections 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.493+0000 7f75164f0700 1 -- 172.21.15.105:0/576598300 wait complete. 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f75164f0700 1 Processor -- start 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f75164f0700 1 -- start start 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f75164f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75101012d0 0x7f7510198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:56.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f75164f0700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75101024d0 0x7f7510198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f75164f0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7510102e70 0x7f751019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f75164f0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f751010cd40 con 0x7f7510102e70 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f75164f0700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f751010cbc0 con 0x7f75101024d0 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f75164f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f751010cec0 con 0x7f75101012d0 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f7514a8d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7510102e70 0x7f751019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f7514a8d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7510102e70 0x7f751019faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57252/0 (socket says 172.21.15.105:57252) 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f7514a8d700 1 -- 172.21.15.105:0/1487250981 learned_addr learned my addr 172.21.15.105:0/1487250981 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f7507fff700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75101024d0 0x7f7510198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.494+0000 7f750ffff700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75101012d0 0x7f7510198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f7514a8d700 1 -- 172.21.15.105:0/1487250981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75101012d0 msgr2=0x7f7510198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f7514a8d700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75101012d0 0x7f7510198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f7514a8d700 1 -- 172.21.15.105:0/1487250981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75101024d0 msgr2=0x7f7510198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:56.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f7514a8d700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75101024d0 0x7f7510198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f7514a8d700 1 -- 172.21.15.105:0/1487250981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75101a0190 con 0x7f7510102e70 2024-03-26T17:37:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f7514a8d700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7510102e70 0x7f751019faf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f750800ef60 tx=0x7f750800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f750dffb700 1 -- 172.21.15.105:0/1487250981 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080092e0 con 0x7f7510102e70 2024-03-26T17:37:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f750dffb700 1 -- 172.21.15.105:0/1487250981 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7508004bb0 con 0x7f7510102e70 2024-03-26T17:37:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75101a0420 con 0x7f7510102e70 2024-03-26T17:37:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f750dffb700 1 -- 172.21.15.105:0/1487250981 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508004020 con 0x7f7510102e70 2024-03-26T17:37:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.495+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75101a0960 con 0x7f7510102e70 2024-03-26T17:37:56.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.496+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7510069070 con 0x7f7510102e70 2024-03-26T17:37:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.498+0000 7f750dffb700 1 -- 172.21.15.105:0/1487250981 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) ==== 137906+0+0 (secure 0 0 0) 0x7f750801b030 con 0x7f7510102e70 2024-03-26T17:37:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.498+0000 7f750dffb700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74fc0a30c0 0x7f74fc0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.499+0000 7f750ffff700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74fc0a30c0 0x7f74fc0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.499+0000 7f750dffb700 1 -- 172.21.15.105:0/1487250981 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f75080cf8e0 con 0x7f7510102e70 2024-03-26T17:37:56.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.499+0000 7f750ffff700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74fc0a30c0 0x7f74fc0a5580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f750000b580 tx=0x7f7500005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:56.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.502+0000 7f750dffb700 1 -- 172.21.15.105:0/1487250981 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7508022020 con 0x7f7510102e70 2024-03-26T17:37:56.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.742+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f75100fea90 con 0x7f74fc0a30c0 2024-03-26T17:37:56.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.749+0000 7f750dffb700 1 -- 172.21.15.105:0/1487250981 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f75100fea90 con 0x7f74fc0a30c0 2024-03-26T17:37:56.743 INFO:teuthology.orchestra.run.smithi105.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-26T17:37:56.743 INFO:teuthology.orchestra.run.smithi105.stdout:mds.all 12/12 33s ago 2m smithi105=a;smithi105=d;smithi105=g;smithi105=j;smithi135=b;smithi135=e;smithi135=h;smithi135=k;smithi144=c;smithi144=f;smithi144=i;smithi144=l;count:12 2024-03-26T17:37:56.743 INFO:teuthology.orchestra.run.smithi105.stdout:mgr 3/3 33s ago 7m smithi105=x;smithi135=y;smithi144=z;count:3 2024-03-26T17:37:56.743 INFO:teuthology.orchestra.run.smithi105.stdout:mon 3/3 33s ago 7m smithi105:172.21.15.105=a;smithi135:172.21.15.135=b;smithi144:172.21.15.144=c;count:3 2024-03-26T17:37:56.743 INFO:teuthology.orchestra.run.smithi105.stdout:osd 12 33s ago - 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74fc0a30c0 msgr2=0x7f74fc0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74fc0a30c0 0x7f74fc0a5580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f750000b580 tx=0x7f7500005fb0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7510102e70 msgr2=0x7f751019faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7510102e70 0x7f751019faf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f750800ef60 tx=0x7f750800ef90 comp rx=0 tx=0).stop 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 shutdown_connections 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74fc0a30c0 0x7f74fc0a5580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7510102e70 0x7f751019faf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75101012d0 0x7f7510198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 --2- 172.21.15.105:0/1487250981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75101024d0 0x7f7510198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 >> 172.21.15.105:0/1487250981 conn(0x7f75100fc870 msgr2=0x7f75100fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 shutdown_connections 2024-03-26T17:37:56.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:56.751+0000 7f75164f0700 1 -- 172.21.15.105:0/1487250981 wait complete. 2024-03-26T17:37:57.372 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-26T17:37:57.626 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:37:58.098 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:57 smithi105 ceph-mon[32280]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:58.098 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:57 smithi105 ceph-mon[32280]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:57 smithi144 ceph-mon[36095]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:58.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:57 smithi144 ceph-mon[36095]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:58.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:57 smithi135 ceph-mon[35636]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:37:58.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:57 smithi135 ceph-mon[35636]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:37:59.025 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:58 smithi105 ceph-mon[32280]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T17:37:59.025 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:58 smithi105 ceph-mon[32280]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-26T17:37:59.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:58 smithi144 ceph-mon[36095]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T17:37:59.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:58 smithi144 ceph-mon[36095]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-26T17:37:59.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:58 smithi135 ceph-mon[35636]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T17:37:59.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:58 smithi135 ceph-mon[35636]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-03-26T17:37:59.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.255+0000 7fb30a8b3700 1 -- 172.21.15.105:0/379972406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb304102e70 msgr2=0x7fb304105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:59.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.255+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/379972406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb304102e70 0x7fb304105260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009990 tx=0x7fb2f4009ca0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.255+0000 7fb30a8b3700 1 -- 172.21.15.105:0/379972406 shutdown_connections 2024-03-26T17:37:59.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.255+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/379972406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3041080d0 0x7fb30410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.255+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/379972406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb304102e70 0x7fb304105260 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.255+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/379972406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3041057a0 0x7fb304107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.255+0000 7fb30a8b3700 1 -- 172.21.15.105:0/379972406 >> 172.21.15.105:0/379972406 conn(0x7fb3040fc880 msgr2=0x7fb3040fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.255+0000 7fb30a8b3700 1 -- 172.21.15.105:0/379972406 shutdown_connections 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 -- 172.21.15.105:0/379972406 wait complete. 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 Processor -- start 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 -- start start 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb304102e70 0x7fb3041985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3041057a0 0x7fb304198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3041080d0 0x7fb30419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb30410de60 con 0x7fb3041080d0 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb30410dce0 con 0x7fb304102e70 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.256+0000 7fb30a8b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb30410dfe0 con 0x7fb3041057a0 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb308e50700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3041080d0 0x7fb30419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:59.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb304102e70 0x7fb3041985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb3037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3041057a0 0x7fb304198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb304102e70 0x7fb3041985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57534/0 (socket says 172.21.15.105:57534) 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 -- 172.21.15.105:0/3136193875 learned_addr learned my addr 172.21.15.105:0/3136193875 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb308e50700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3041080d0 0x7fb30419fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57276/0 (socket says 172.21.15.105:57276) 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 -- 172.21.15.105:0/3136193875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3041057a0 msgr2=0x7fb304198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3041057a0 0x7fb304198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 -- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3041080d0 msgr2=0x7fb30419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3041080d0 0x7fb30419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 -- 172.21.15.105:0/3136193875 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3041a01e0 con 0x7fb304102e70 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb303fff700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb304102e70 0x7fb3041985d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400b2e0 tx=0x7fb2f40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb3017fa700 1 -- 172.21.15.105:0/3136193875 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4027070 con 0x7fb304102e70 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3041a0470 con 0x7fb304102e70 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb3017fa700 1 -- 172.21.15.105:0/3136193875 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f4022470 con 0x7fb304102e70 2024-03-26T17:37:59.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb3017fa700 1 -- 172.21.15.105:0/3136193875 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4017650 con 0x7fb304102e70 2024-03-26T17:37:59.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.257+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3041a08d0 con 0x7fb304102e70 2024-03-26T17:37:59.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.258+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb304069070 con 0x7fb304102e70 2024-03-26T17:37:59.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.260+0000 7fb3017fa700 1 -- 172.21.15.105:0/3136193875 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7fb2f402a070 con 0x7fb304102e70 2024-03-26T17:37:59.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.261+0000 7fb3017fa700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb2ec0a2fb0 0x7fb2ec0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:37:59.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.261+0000 7fb3037fe700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb2ec0a2fb0 0x7fb2ec0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:37:59.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.261+0000 7fb3017fa700 1 -- 172.21.15.105:0/3136193875 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb2f40d0440 con 0x7fb304102e70 2024-03-26T17:37:59.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.261+0000 7fb3037fe700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb2ec0a2fb0 0x7fb2ec0a5470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009680 tx=0x7fb2f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:37:59.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.268+0000 7fb3017fa700 1 -- 172.21.15.105:0/3136193875 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb2f4013070 con 0x7fb304102e70 2024-03-26T17:37:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.476+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fb3040627f0 con 0x7fb2ec0a2fb0 2024-03-26T17:37:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.477+0000 7fb3017fa700 1 -- 172.21.15.105:0/3136193875 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fb3040627f0 con 0x7fb2ec0a2fb0 2024-03-26T17:37:59.471 INFO:teuthology.orchestra.run.smithi105.stdout:HOST ADDR LABELS STATUS 2024-03-26T17:37:59.471 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105 172.21.15.105 2024-03-26T17:37:59.471 INFO:teuthology.orchestra.run.smithi105.stdout:smithi135 172.21.15.135 2024-03-26T17:37:59.471 INFO:teuthology.orchestra.run.smithi105.stdout:smithi144 172.21.15.144 2024-03-26T17:37:59.472 INFO:teuthology.orchestra.run.smithi105.stdout:3 hosts in cluster 2024-03-26T17:37:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.479+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb2ec0a2fb0 msgr2=0x7fb2ec0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.479+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb2ec0a2fb0 0x7fb2ec0a5470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009680 tx=0x7fb2f8009260 comp rx=0 tx=0).stop 2024-03-26T17:37:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.479+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb304102e70 msgr2=0x7fb3041985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:37:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.479+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb304102e70 0x7fb3041985d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2f400b2e0 tx=0x7fb2f40047b0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.480+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 shutdown_connections 2024-03-26T17:37:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.480+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb2ec0a2fb0 0x7fb2ec0a5470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.480+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3041080d0 0x7fb30419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.480+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb304102e70 0x7fb3041985d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.480+0000 7fb30a8b3700 1 --2- 172.21.15.105:0/3136193875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3041057a0 0x7fb304198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:37:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.480+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 >> 172.21.15.105:0/3136193875 conn(0x7fb3040fc880 msgr2=0x7fb304104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:37:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.480+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 shutdown_connections 2024-03-26T17:37:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:37:59.480+0000 7fb30a8b3700 1 -- 172.21.15.105:0/3136193875 wait complete. 2024-03-26T17:38:00.075 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:37:59 smithi105 ceph-mon[32280]: from='client.25013 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:00.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:37:59 smithi144 ceph-mon[36095]: from='client.25013 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:00.107 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-26T17:38:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:37:59 smithi135 ceph-mon[35636]: from='client.25013 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:00.353 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:38:01.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:00 smithi144 ceph-mon[36095]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T17:38:01.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:00 smithi105 ceph-mon[32280]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T17:38:01.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:00 smithi135 ceph-mon[35636]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 -- 172.21.15.105:0/2851623674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8102e60 msgr2=0x7f65c8105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/2851623674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8102e60 0x7f65c8105250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009990 tx=0x7f65bc009ca0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 -- 172.21.15.105:0/2851623674 shutdown_connections 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/2851623674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c81080c0 0x7f65c810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/2851623674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8102e60 0x7f65c8105250 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/2851623674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c8105790 0x7f65c8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 -- 172.21.15.105:0/2851623674 >> 172.21.15.105:0/2851623674 conn(0x7f65c80fc870 msgr2=0x7f65c80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 -- 172.21.15.105:0/2851623674 shutdown_connections 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 -- 172.21.15.105:0/2851623674 wait complete. 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.137+0000 7f65cd8a2700 1 Processor -- start 2024-03-26T17:38:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65cd8a2700 1 -- start start 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65cd8a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c8102e60 0x7f65c8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65cd8a2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8105790 0x7f65c8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65cd8a2700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c81080c0 0x7f65c819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65cd8a2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810de90 con 0x7f65c8105790 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65cd8a2700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810dd10 con 0x7f65c81080c0 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65cd8a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810e010 con 0x7f65c8102e60 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c67fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8105790 0x7f65c8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c8102e60 0x7f65c8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c8102e60 0x7f65c8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44344/0 (socket says 172.21.15.105:44344) 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c6ffd700 1 -- 172.21.15.105:0/443991363 learned_addr learned my addr 172.21.15.105:0/443991363 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c77fe700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c81080c0 0x7f65c819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c67fc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8105790 0x7f65c8198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57294/0 (socket says 172.21.15.105:57294) 2024-03-26T17:38:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c6ffd700 1 -- 172.21.15.105:0/443991363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c81080c0 msgr2=0x7f65c819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c6ffd700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c81080c0 0x7f65c819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c6ffd700 1 -- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8105790 msgr2=0x7f65c8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c6ffd700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8105790 0x7f65c8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c6ffd700 1 -- 172.21.15.105:0/443991363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c81a01b0 con 0x7f65c8102e60 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.138+0000 7f65c67fc700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8105790 0x7f65c8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.139+0000 7f65c6ffd700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c8102e60 0x7f65c8198550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65bc005910 tx=0x7f65bc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.139+0000 7f65cc8a0700 1 -- 172.21.15.105:0/443991363 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc026070 con 0x7f65c8102e60 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.139+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c81a0440 con 0x7f65c8102e60 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.139+0000 7f65cc8a0700 1 -- 172.21.15.105:0/443991363 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65bc021410 con 0x7f65c8102e60 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.139+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c81a08f0 con 0x7f65c8102e60 2024-03-26T17:38:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.139+0000 7f65cc8a0700 1 -- 172.21.15.105:0/443991363 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc017650 con 0x7f65c8102e60 2024-03-26T17:38:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.140+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c8069070 con 0x7f65c8102e60 2024-03-26T17:38:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.141+0000 7f65cc8a0700 1 -- 172.21.15.105:0/443991363 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f65bc029070 con 0x7f65c8102e60 2024-03-26T17:38:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.142+0000 7f65cc8a0700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65b00a3030 0x7f65b00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.142+0000 7f65c67fc700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65b00a3030 0x7f65b00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:02.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.142+0000 7f65cc8a0700 1 -- 172.21.15.105:0/443991363 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f65bc0d0780 con 0x7f65c8102e60 2024-03-26T17:38:02.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.142+0000 7f65c67fc700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65b00a3030 0x7f65b00a54f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f65b8011490 tx=0x7f65b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:02.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.145+0000 7f65cc8a0700 1 -- 172.21.15.105:0/443991363 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f65bc013070 con 0x7f65c8102e60 2024-03-26T17:38:02.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.353+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f65c80627f0 con 0x7f65b00a3030 2024-03-26T17:38:02.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.355+0000 7f65cc8a0700 1 -- 172.21.15.105:0/443991363 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f65c80627f0 con 0x7f65b00a3030 2024-03-26T17:38:02.348 INFO:teuthology.orchestra.run.smithi105.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-26T17:38:02.349 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400TR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-26T17:38:02.349 INFO:teuthology.orchestra.run.smithi105.stdout:smithi135 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401BA400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-26T17:38:02.349 INFO:teuthology.orchestra.run.smithi105.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65b00a3030 msgr2=0x7f65b00a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65b00a3030 0x7f65b00a54f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f65b8011490 tx=0x7f65b8009260 comp rx=0 tx=0).stop 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c8102e60 msgr2=0x7f65c8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c8102e60 0x7f65c8198550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65bc005910 tx=0x7f65bc009580 comp rx=0 tx=0).stop 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 shutdown_connections 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65b00a3030 0x7f65b00a54f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65c81080c0 0x7f65c819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c8102e60 0x7f65c8198550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 --2- 172.21.15.105:0/443991363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c8105790 0x7f65c8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:02.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 >> 172.21.15.105:0/443991363 conn(0x7f65c80fc870 msgr2=0x7f65c81037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:02.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 shutdown_connections 2024-03-26T17:38:02.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:02.358+0000 7f65cd8a2700 1 -- 172.21.15.105:0/443991363 wait complete. 2024-03-26T17:38:03.192 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T17:38:03.203 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T17:38:03.203 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-26T17:38:03.456 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:38:03.836 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:03 smithi105 ceph-mon[32280]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:03.836 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:03 smithi105 ceph-mon[32280]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:38:03.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:03 smithi135 ceph-mon[35636]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:03.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:03 smithi135 ceph-mon[35636]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:38:04.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:03 smithi144 ceph-mon[36095]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:04.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:03 smithi144 ceph-mon[36095]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:38:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 -- 172.21.15.105:0/3220262551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001057b0 msgr2=0x7f4500107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 --2- 172.21.15.105:0/3220262551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001057b0 0x7f4500107ba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009990 tx=0x7f44f4009ca0 comp rx=0 tx=0).stop 2024-03-26T17:38:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 -- 172.21.15.105:0/3220262551 shutdown_connections 2024-03-26T17:38:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 --2- 172.21.15.105:0/3220262551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001080e0 0x7f450010a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:04.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 --2- 172.21.15.105:0/3220262551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102e80 0x7f4500105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 --2- 172.21.15.105:0/3220262551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001057b0 0x7f4500107ba0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 -- 172.21.15.105:0/3220262551 >> 172.21.15.105:0/3220262551 conn(0x7f45000fc8b0 msgr2=0x7f45000fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 -- 172.21.15.105:0/3220262551 shutdown_connections 2024-03-26T17:38:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.985+0000 7f45059cc700 1 -- 172.21.15.105:0/3220262551 wait complete. 2024-03-26T17:38:04.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f45059cc700 1 Processor -- start 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f45059cc700 1 -- start start 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f45059cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102e80 0x7f4500198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f45059cc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001057b0 0x7f4500198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f45059cc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001080e0 0x7f450019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f45059cc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f450010deb0 con 0x7f45001080e0 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f45059cc700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f450010dd30 con 0x7f45001057b0 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f45059cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f450010e030 con 0x7f4500102e80 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f44ff7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001080e0 0x7f450019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:04.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f44fe7fc700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001057b0 0x7f4500198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f44ff7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001080e0 0x7f450019fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33578/0 (socket says 172.21.15.105:33578) 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f44ff7fe700 1 -- 172.21.15.105:0/2043493815 learned_addr learned my addr 172.21.15.105:0/2043493815 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.986+0000 7f44feffd700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102e80 0x7f4500198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f44fe7fc700 1 -- 172.21.15.105:0/2043493815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102e80 msgr2=0x7f4500198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f44fe7fc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102e80 0x7f4500198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f44fe7fc700 1 -- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001080e0 msgr2=0x7f450019fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f44fe7fc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001080e0 0x7f450019fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f44fe7fc700 1 -- 172.21.15.105:0/2043493815 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45001a01c0 con 0x7f45001057b0 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f44ff7fe700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001080e0 0x7f450019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f44fe7fc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001057b0 0x7f4500198aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44f4000c00 tx=0x7f44f40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f45049ca700 1 -- 172.21.15.105:0/2043493815 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4027070 con 0x7f45001057b0 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f45049ca700 1 -- 172.21.15.105:0/2043493815 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44f4022470 con 0x7f45001057b0 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f45049ca700 1 -- 172.21.15.105:0/2043493815 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4017670 con 0x7f45001057b0 2024-03-26T17:38:04.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45001a0450 con 0x7f45001057b0 2024-03-26T17:38:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.987+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45001a08a0 con 0x7f45001057b0 2024-03-26T17:38:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.988+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4500069070 con 0x7f45001057b0 2024-03-26T17:38:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.989+0000 7f45049ca700 1 -- 172.21.15.105:0/2043493815 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f44f402a070 con 0x7f45001057b0 2024-03-26T17:38:04.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.990+0000 7f45049ca700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44e80a2fb0 0x7f44e80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.990+0000 7f44feffd700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44e80a2fb0 0x7f44e80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.990+0000 7f45049ca700 1 -- 172.21.15.105:0/2043493815 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f44f40d00b0 con 0x7f45001057b0 2024-03-26T17:38:04.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.990+0000 7f44feffd700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44e80a2fb0 0x7f44e80a5470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009c60 tx=0x7f44f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:04.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:04.994+0000 7f45049ca700 1 -- 172.21.15.105:0/2043493815 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f44f4013070 con 0x7f45001057b0 2024-03-26T17:38:05.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.242+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4500062fc0 con 0x7f45001057b0 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.243+0000 7f45049ca700 1 -- 172.21.15.105:0/2043493815 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3096 (secure 0 0 0) 0x7f44f409cf90 con 0x7f45001057b0 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout:e26 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout:legacy client fscid: 1 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout:Filesystem 'cephfs' (1) 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout:fs_name cephfs 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 26 2024-03-26T17:38:05.237 INFO:teuthology.orchestra.run.smithi105.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-03-26T17:36:22.059727+0000 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:modified 2024-03-26T17:36:29.308004+0000 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:tableserver 0 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:root 0 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:session_timeout 300 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:session_autoclose 300 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:max_file_size 1099511627776 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:max_xattr_size 65536 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:required_client_features {} 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:last_failure 0 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:last_failure_osd_epoch 0 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:max_mds 5 2024-03-26T17:38:05.238 INFO:teuthology.orchestra.run.smithi105.stdout:in 0,1,2,3,4 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:up {0=24521,1=24487,2=24482,3=24466,4=14628} 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:failed 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:damaged 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:stopped 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:data_pools [3] 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:metadata_pool 2 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:inline_data disabled 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:balancer 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:bal_rank_mask -1 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:standby_count_wanted 0 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:qdb_cluster leader: 24521 members: 24482,24521,14628,24466,24487 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b{0:24521} state up:active seq 3 addr [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.e{0:24458} state up:standby-replay seq 1 addr [v2:172.21.15.135:6834/3121985659,v1:172.21.15.135:6835/3121985659] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.239 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.i{1:24487} state up:active seq 6 addr [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.f{1:24454} state up:standby-replay seq 1 addr [v2:172.21.15.144:6834/29089236,v1:172.21.15.144:6835/29089236] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.h{2:24482} state up:active seq 7 addr [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.k{2:24422} state up:standby-replay seq 1 addr [v2:172.21.15.135:6832/1499648854,v1:172.21.15.135:6833/1499648854] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c{3:24466} state up:active seq 8 addr [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.l{3:24415} state up:standby-replay seq 1 addr [v2:172.21.15.144:6832/3569777961,v1:172.21.15.144:6833/3569777961] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a{4:14628} state up:active seq 5 addr [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.105:6838/3053296748,v1:172.21.15.105:6839/3053296748] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:Standby daemons: 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:05.240 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.105:6834/3032565642,v1:172.21.15.105:6835/3032565642] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.241 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.105:6836/2045143293,v1:172.21.15.105:6837/2045143293] compat {c=[1],r=[1],i=[1fff]}] 2024-03-26T17:38:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44e80a2fb0 msgr2=0x7f44e80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44e80a2fb0 0x7f44e80a5470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009c60 tx=0x7f44f0009260 comp rx=0 tx=0).stop 2024-03-26T17:38:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001057b0 msgr2=0x7f4500198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001057b0 0x7f4500198aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44f4000c00 tx=0x7f44f40047b0 comp rx=0 tx=0).stop 2024-03-26T17:38:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 shutdown_connections 2024-03-26T17:38:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44e80a2fb0 0x7f44e80a5470 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f45001080e0 0x7f450019fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102e80 0x7f4500198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 --2- 172.21.15.105:0/2043493815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001057b0 0x7f4500198aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 >> 172.21.15.105:0/2043493815 conn(0x7f45000fc8b0 msgr2=0x7f4500103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 shutdown_connections 2024-03-26T17:38:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:05.245+0000 7f45059cc700 1 -- 172.21.15.105:0/2043493815 wait complete. 2024-03-26T17:38:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 26 2024-03-26T17:38:05.917 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:05 smithi135 ceph-mon[35636]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:05.918 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:05 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2043493815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T17:38:05.923 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:05 smithi105 ceph-mon[32280]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:05.923 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:05 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2043493815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T17:38:05.949 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-26T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:05 smithi144 ceph-mon[36095]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:06.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:05 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2043493815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T17:38:06.199 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:38:07.750 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:07 smithi105 ceph-mon[32280]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:07.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:07 smithi135 ceph-mon[35636]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 -- 172.21.15.105:0/4080146373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 msgr2=0x7fecb0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 --2- 172.21.15.105:0/4080146373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 0x7fecb0107b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fec98009990 tx=0x7fec98009ca0 comp rx=0 tx=0).stop 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 -- 172.21.15.105:0/4080146373 shutdown_connections 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 --2- 172.21.15.105:0/4080146373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb01080d0 0x7fecb010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 --2- 172.21.15.105:0/4080146373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb0102e70 0x7fecb0105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 --2- 172.21.15.105:0/4080146373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 0x7fecb0107b90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 -- 172.21.15.105:0/4080146373 >> 172.21.15.105:0/4080146373 conn(0x7fecb00fc880 msgr2=0x7fecb00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 -- 172.21.15.105:0/4080146373 shutdown_connections 2024-03-26T17:38:07.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.967+0000 7fecb50bb700 1 -- 172.21.15.105:0/4080146373 wait complete. 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.968+0000 7fecb50bb700 1 Processor -- start 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.968+0000 7fecb50bb700 1 -- start start 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.968+0000 7fecb50bb700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb0102e70 0x7fecb0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecb50bb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 0x7fecb0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecb50bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb01080d0 0x7fecb019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecb50bb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010dea0 con 0x7fecb01057a0 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecb50bb700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010dd20 con 0x7fecb0102e70 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecb50bb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecb010e020 con 0x7fecb01080d0 2024-03-26T17:38:07.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7feca659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 0x7fecb0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7feca659c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 0x7fecb0198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33596/0 (socket says 172.21.15.105:33596) 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7feca659c700 1 -- 172.21.15.105:0/3235710800 learned_addr learned my addr 172.21.15.105:0/3235710800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecaed9d700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb0102e70 0x7fecb0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecaf59e700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb01080d0 0x7fecb019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecaed9d700 1 -- 172.21.15.105:0/3235710800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb01080d0 msgr2=0x7fecb019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecaed9d700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb01080d0 0x7fecb019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecaed9d700 1 -- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 msgr2=0x7fecb0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecaed9d700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 0x7fecb0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecaed9d700 1 -- 172.21.15.105:0/3235710800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb01a01d0 con 0x7fecb0102e70 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7feca659c700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 0x7fecb0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecaed9d700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb0102e70 0x7fecb0198570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feca000d950 tx=0x7feca000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecacd99700 1 -- 172.21.15.105:0/3235710800 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0016070 con 0x7fecb0102e70 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecacd99700 1 -- 172.21.15.105:0/3235710800 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feca000fd40 con 0x7fecb0102e70 2024-03-26T17:38:07.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb01a04c0 con 0x7fecb0102e70 2024-03-26T17:38:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.969+0000 7fecacd99700 1 -- 172.21.15.105:0/3235710800 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca000e3b0 con 0x7fecb0102e70 2024-03-26T17:38:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.970+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb01a0a50 con 0x7fecb0102e70 2024-03-26T17:38:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.971+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb0069070 con 0x7fecb0102e70 2024-03-26T17:38:07.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.971+0000 7fecacd99700 1 -- 172.21.15.105:0/3235710800 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7feca0019070 con 0x7fecb0102e70 2024-03-26T17:38:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.972+0000 7fecacd99700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec9c0a3030 0x7fec9c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.972+0000 7feca659c700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec9c0a3030 0x7fec9c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.972+0000 7fecacd99700 1 -- 172.21.15.105:0/3235710800 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7feca00ced50 con 0x7fecb0102e70 2024-03-26T17:38:07.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.973+0000 7feca659c700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec9c0a3030 0x7fec9c0a54f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fec98006010 tx=0x7fec98003870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:07.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:07.976+0000 7fecacd99700 1 -- 172.21.15.105:0/3235710800 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feca009bb40 con 0x7fecb0102e70 2024-03-26T17:38:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:07 smithi144 ceph-mon[36095]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:08.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.224+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fecb002dc00 con 0x7fecb0102e70 2024-03-26T17:38:08.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.225+0000 7fecacd99700 1 -- 172.21.15.105:0/3235710800 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5287 (secure 0 0 0) 0x7feca00bb070 con 0x7fecb0102e70 2024-03-26T17:38:08.219 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 87 2024-03-26T17:38:08.219 INFO:teuthology.orchestra.run.smithi105.stdout:fsid 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T17:38:08.219 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-03-26T17:28:32.621716+0000 2024-03-26T17:38:08.219 INFO:teuthology.orchestra.run.smithi105.stdout:modified 2024-03-26T17:37:27.733852+0000 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:crush_version 26 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:full_ratio 0.95 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:backfillfull_ratio 0.9 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:nearfull_ratio 0.85 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:require_min_compat_client luminous 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:min_compat_client luminous 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:require_osd_release squid 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:stretch_mode_enabled false 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-26T17:38:08.220 INFO:teuthology.orchestra.run.smithi105.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:max_osd 12 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6802/101464397,v1:172.21.15.105:6803/101464397] [v2:172.21.15.105:6804/101464397,v1:172.21.15.105:6805/101464397] exists,up 42769940-d924-4456-ab99-af1c060516a6 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/932873598,v1:172.21.15.135:6801/932873598] [v2:172.21.15.135:6802/932873598,v1:172.21.15.135:6803/932873598] exists,up 33937b10-f283-4680-a12c-ee35827a2bd2 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6800/386966961,v1:172.21.15.144:6801/386966961] [v2:172.21.15.144:6802/386966961,v1:172.21.15.144:6803/386966961] exists,up f434f0a6-ad51-45e8-b9e9-411cead27014 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6810/2769664525,v1:172.21.15.105:6811/2769664525] [v2:172.21.15.105:6812/2769664525,v1:172.21.15.105:6813/2769664525] exists,up f52f7e5b-63cd-43ea-a2d9-5b3282dbb996 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/507590464,v1:172.21.15.135:6809/507590464] [v2:172.21.15.135:6810/507590464,v1:172.21.15.135:6811/507590464] exists,up a9450083-abb4-4382-b2eb-96f679b03bdb 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.5 up in weight 1 up_from 39 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6808/3886741876,v1:172.21.15.144:6809/3886741876] [v2:172.21.15.144:6810/3886741876,v1:172.21.15.144:6811/3886741876] exists,up c13547ad-d6de-4be7-8548-7444afdf81e3 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6818/3456307470,v1:172.21.15.105:6819/3456307470] [v2:172.21.15.105:6820/3456307470,v1:172.21.15.105:6821/3456307470] exists,up 342c60b1-acab-414f-b483-bc6ab61d927e 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.7 up in weight 1 up_from 52 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/788617133,v1:172.21.15.135:6817/788617133] [v2:172.21.15.135:6818/788617133,v1:172.21.15.135:6819/788617133] exists,up 28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6816/4074146904,v1:172.21.15.144:6817/4074146904] [v2:172.21.15.144:6818/4074146904,v1:172.21.15.144:6819/4074146904] exists,up 779c4b05-662f-4332-994a-5cb93ba4aabf 2024-03-26T17:38:08.221 INFO:teuthology.orchestra.run.smithi105.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6826/1324177840,v1:172.21.15.105:6827/1324177840] [v2:172.21.15.105:6828/1324177840,v1:172.21.15.105:6829/1324177840] exists,up 0f1218e3-f488-426a-8ac6-afece38016e1 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6824/3021911997,v1:172.21.15.135:6825/3021911997] [v2:172.21.15.135:6826/3021911997,v1:172.21.15.135:6827/3021911997] exists,up 47f4a8a1-7c2d-4e70-a83e-03296b92743c 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.144:6824/4293375044,v1:172.21.15.144:6825/4293375044] [v2:172.21.15.144:6826/4293375044,v1:172.21.15.144:6827/4293375044] exists,up 82fd99c0-5cc2-4744-b31a-45dac10ca1db 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 2.5 [8,11] 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 2.e [2,4] 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 2.10 [2,1,8,11] 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.0 [5,9,4,6] 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.1 [4,9] 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.17 [4,8,5,7] 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:pg_upmap_items 3.28 [3,8] 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/1242681476 expires 2024-03-27T17:29:26.501525+0000 2024-03-26T17:38:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/976039762 expires 2024-03-27T17:29:26.501525+0000 2024-03-26T17:38:08.223 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:6801/1556474900 expires 2024-03-27T17:29:26.501525+0000 2024-03-26T17:38:08.223 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/1151470472 expires 2024-03-27T17:29:26.501525+0000 2024-03-26T17:38:08.223 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/3821116771 expires 2024-03-27T17:28:59.817767+0000 2024-03-26T17:38:08.223 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:6800/3642413152 expires 2024-03-27T17:28:59.817767+0000 2024-03-26T17:38:08.223 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/2484248025 expires 2024-03-27T17:28:59.817767+0000 2024-03-26T17:38:08.223 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:0/2017206095 expires 2024-03-27T17:28:59.817767+0000 2024-03-26T17:38:08.223 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:6801/3642413152 expires 2024-03-27T17:28:59.817767+0000 2024-03-26T17:38:08.223 INFO:teuthology.orchestra.run.smithi105.stdout:blocklist 172.21.15.105:6800/1556474900 expires 2024-03-27T17:29:26.501525+0000 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.227+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec9c0a3030 msgr2=0x7fec9c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.227+0000 7fecb50bb700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec9c0a3030 0x7fec9c0a54f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fec98006010 tx=0x7fec98003870 comp rx=0 tx=0).stop 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb0102e70 msgr2=0x7fecb0198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb0102e70 0x7fecb0198570 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feca000d950 tx=0x7feca000dc60 comp rx=0 tx=0).stop 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 shutdown_connections 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec9c0a3030 0x7fec9c0a54f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb01080d0 0x7fecb019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecb0102e70 0x7fecb0198570 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:08.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 --2- 172.21.15.105:0/3235710800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecb01057a0 0x7fecb0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:08.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 >> 172.21.15.105:0/3235710800 conn(0x7fecb00fc880 msgr2=0x7fecb0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:08.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 shutdown_connections 2024-03-26T17:38:08.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:08.228+0000 7fecb50bb700 1 -- 172.21.15.105:0/3235710800 wait complete. 2024-03-26T17:38:08.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:08 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3235710800' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T17:38:08.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:08 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3235710800' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T17:38:08.924 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-26T17:38:08.936 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:38:09.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 -- 172.21.15.105:0/452847311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 msgr2=0x7f5eb4100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 --2- 172.21.15.105:0/452847311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 0x7f5eb4100c00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009900 tx=0x7f5ea402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:09.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 -- 172.21.15.105:0/452847311 shutdown_connections 2024-03-26T17:38:09.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 --2- 172.21.15.105:0/452847311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eb4102390 0x7f5eb4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 --2- 172.21.15.105:0/452847311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb41019d0 0x7f5eb4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 --2- 172.21.15.105:0/452847311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 0x7f5eb4100c00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 -- 172.21.15.105:0/452847311 >> 172.21.15.105:0/452847311 conn(0x7f5eb40fbd50 msgr2=0x7f5eb40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 -- 172.21.15.105:0/452847311 shutdown_connections 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.052+0000 7f5ebc597640 1 -- 172.21.15.105:0/452847311 wait complete. 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5ebc597640 1 Processor -- start 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5ebc597640 1 -- start start 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5ebc597640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 0x7f5eb4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5ebc597640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eb41019d0 0x7f5eb41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5ebc597640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb4102390 0x7f5eb419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5ebc597640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb410d430 con 0x7f5eb41007e0 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5ebc597640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb410d2b0 con 0x7f5eb41019d0 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5ebc597640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb410d5b0 con 0x7f5eb4102390 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5eba333640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 0x7f5eb4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5eba333640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 0x7f5eb4197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33612/0 (socket says 172.21.15.105:33612) 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.053+0000 7f5eba333640 1 -- 172.21.15.105:0/525972110 learned_addr learned my addr 172.21.15.105:0/525972110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5ebab34640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb4102390 0x7f5eb419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eb9b32640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eb41019d0 0x7f5eb41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eba333640 1 -- 172.21.15.105:0/525972110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb4102390 msgr2=0x7f5eb419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eba333640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb4102390 0x7f5eb419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eba333640 1 -- 172.21.15.105:0/525972110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eb41019d0 msgr2=0x7f5eb41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eba333640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eb41019d0 0x7f5eb41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eba333640 1 -- 172.21.15.105:0/525972110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb419f7a0 con 0x7f5eb41007e0 2024-03-26T17:38:09.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eba333640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 0x7f5eb4197b80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000c00 tx=0x7f5ea4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:09.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eab7fe640 1 -- 172.21.15.105:0/525972110 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4043020 con 0x7f5eb41007e0 2024-03-26T17:38:09.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eab7fe640 1 -- 172.21.15.105:0/525972110 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ea4004350 con 0x7f5eb41007e0 2024-03-26T17:38:09.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb419fa30 con 0x7f5eb41007e0 2024-03-26T17:38:09.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5eab7fe640 1 -- 172.21.15.105:0/525972110 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4003b80 con 0x7f5eb41007e0 2024-03-26T17:38:09.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.054+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb419ffd0 con 0x7f5eb41007e0 2024-03-26T17:38:09.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.055+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb4107b70 con 0x7f5eb41007e0 2024-03-26T17:38:09.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.056+0000 7f5eab7fe640 1 -- 172.21.15.105:0/525972110 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f5ea4046020 con 0x7f5eb41007e0 2024-03-26T17:38:09.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.057+0000 7f5eab7fe640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ea00a2fa0 0x7f5ea00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.057+0000 7f5eb9b32640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ea00a2fa0 0x7f5ea00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.057+0000 7f5eab7fe640 1 -- 172.21.15.105:0/525972110 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f5ea40f1480 con 0x7f5eb41007e0 2024-03-26T17:38:09.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.057+0000 7f5eb9b32640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ea00a2fa0 0x7f5ea00a5460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4199090 tx=0x7f5eb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:09.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.062+0000 7f5eab7fe640 1 -- 172.21.15.105:0/525972110 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5ea404d020 con 0x7f5eb41007e0 2024-03-26T17:38:09.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:08 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3235710800' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T17:38:09.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.244+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5eb40624c0 con 0x7f5eb41007e0 2024-03-26T17:38:09.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.247+0000 7f5eab7fe640 1 -- 172.21.15.105:0/525972110 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f5ea40be2d0 con 0x7f5eb41007e0 2024-03-26T17:38:09.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:09.241 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:38:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.248+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ea00a2fa0 msgr2=0x7f5ea00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ea00a2fa0 0x7f5ea00a5460 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4199090 tx=0x7f5eb0005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:09.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 msgr2=0x7f5eb4197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 0x7f5eb4197b80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000c00 tx=0x7f5ea4005dc0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 shutdown_connections 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ea00a2fa0 0x7f5ea00a5460 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5eb4102390 0x7f5eb419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5eb41019d0 0x7f5eb41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 --2- 172.21.15.105:0/525972110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eb41007e0 0x7f5eb4197b80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 >> 172.21.15.105:0/525972110 conn(0x7f5eb40fbd50 msgr2=0x7f5eb4104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 shutdown_connections 2024-03-26T17:38:09.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.249+0000 7f5ebc597640 1 -- 172.21.15.105:0/525972110 wait complete. 2024-03-26T17:38:09.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:38:09.257 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:38:09.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.390+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1258471009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41007c0 msgr2=0x7f49d4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.390+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1258471009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41007c0 0x7f49d4100be0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009900 tx=0x7f49c402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.390+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1258471009 shutdown_connections 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.390+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1258471009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d4102370 0x7f49d4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.390+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1258471009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d41019b0 0x7f49d4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.390+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1258471009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41007c0 0x7f49d4100be0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1258471009 >> 172.21.15.105:0/1258471009 conn(0x7f49d40fbd50 msgr2=0x7f49d40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1258471009 shutdown_connections 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1258471009 wait complete. 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 Processor -- start 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 -- start start 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d41007c0 0x7f49d4196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41019b0 0x7f49d4197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d4102370 0x7f49d419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49d410c2e0 con 0x7f49d41019b0 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49d410c160 con 0x7f49d4102370 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.391+0000 7f49d9fe6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49d410c460 con 0x7f49d41007c0 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41019b0 0x7f49d4197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41019b0 0x7f49d4197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33624/0 (socket says 172.21.15.105:33624) 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 -- 172.21.15.105:0/1015648618 learned_addr learned my addr 172.21.15.105:0/1015648618 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49d37fe640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d41007c0 0x7f49d4196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 -- 172.21.15.105:0/1015648618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d41007c0 msgr2=0x7f49d4196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d41007c0 0x7f49d4196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 -- 172.21.15.105:0/1015648618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d4102370 msgr2=0x7f49d419e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d4102370 0x7f49d419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 -- 172.21.15.105:0/1015648618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d419e6e0 con 0x7f49d41019b0 2024-03-26T17:38:09.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49caffd640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41019b0 0x7f49d4197000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00e9f0 tx=0x7f49bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49d17fa640 1 -- 172.21.15.105:0/1015648618 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00cd80 con 0x7f49d41019b0 2024-03-26T17:38:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d419e9d0 con 0x7f49d41019b0 2024-03-26T17:38:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49d17fa640 1 -- 172.21.15.105:0/1015648618 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49bc0054d0 con 0x7f49d41019b0 2024-03-26T17:38:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49d17fa640 1 -- 172.21.15.105:0/1015648618 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc010490 con 0x7f49d41019b0 2024-03-26T17:38:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.392+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d419eed0 con 0x7f49d41019b0 2024-03-26T17:38:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.393+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d4107ca0 con 0x7f49d41019b0 2024-03-26T17:38:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.394+0000 7f49d17fa640 1 -- 172.21.15.105:0/1015648618 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f49bc005000 con 0x7f49d41019b0 2024-03-26T17:38:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.395+0000 7f49d17fa640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49c00a2f20 0x7f49c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.395+0000 7f49d37fe640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49c00a2f20 0x7f49c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.395+0000 7f49d17fa640 1 -- 172.21.15.105:0/1015648618 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f49bc0cf9b0 con 0x7f49d41019b0 2024-03-26T17:38:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.395+0000 7f49d37fe640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49c00a2f20 0x7f49c00a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005ab0 tx=0x7f49c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:09.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.403+0000 7f49d17fa640 1 -- 172.21.15.105:0/1015648618 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f49bc053740 con 0x7f49d41019b0 2024-03-26T17:38:09.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.583+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49d402d490 con 0x7f49d41019b0 2024-03-26T17:38:09.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.584+0000 7f49d17fa640 1 -- 172.21.15.105:0/1015648618 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f49bc0052f0 con 0x7f49d41019b0 2024-03-26T17:38:09.577 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:38:09.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.585+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49c00a2f20 msgr2=0x7f49c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.585+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49c00a2f20 0x7f49c00a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005ab0 tx=0x7f49c4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.585+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41019b0 msgr2=0x7f49d4197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.585+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41019b0 0x7f49d4197000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00e9f0 tx=0x7f49bc00edf0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.586+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 shutdown_connections 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.586+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49c00a2f20 0x7f49c00a53e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.586+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49d4102370 0x7f49d419e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.586+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49d41019b0 0x7f49d4197000 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.586+0000 7f49d9fe6640 1 --2- 172.21.15.105:0/1015648618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49d41007c0 0x7f49d4196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.586+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 >> 172.21.15.105:0/1015648618 conn(0x7f49d40fbd50 msgr2=0x7f49d4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.586+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 shutdown_connections 2024-03-26T17:38:09.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.586+0000 7f49d9fe6640 1 -- 172.21.15.105:0/1015648618 wait complete. 2024-03-26T17:38:09.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T17:38:09.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2370364277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec104cf0 msgr2=0x7f35ec1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2370364277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec104cf0 0x7f35ec1070e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009900 tx=0x7f35dc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2370364277 shutdown_connections 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2370364277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec107620 0x7f35ec109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2370364277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec104cf0 0x7f35ec1070e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2370364277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec1023c0 0x7f35ec1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2370364277 >> 172.21.15.105:0/2370364277 conn(0x7f35ec0fbd70 msgr2=0x7f35ec0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2370364277 shutdown_connections 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2370364277 wait complete. 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.712+0000 7f35f3ce8640 1 Processor -- start 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 -- start start 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1023c0 0x7f35ec101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec104cf0 0x7f35ec100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec107620 0x7f35ec1008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec10d410 con 0x7f35ec107620 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec10d290 con 0x7f35ec104cf0 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec10d590 con 0x7f35ec1023c0 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec107620 0x7f35ec1008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec107620 0x7f35ec1008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33640/0 (socket says 172.21.15.105:33640) 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 -- 172.21.15.105:0/2571266817 learned_addr learned my addr 172.21.15.105:0/2571266817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f1a84640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1023c0 0x7f35ec101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f1283640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec104cf0 0x7f35ec100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 -- 172.21.15.105:0/2571266817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1023c0 msgr2=0x7f35ec101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1023c0 0x7f35ec101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 -- 172.21.15.105:0/2571266817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec104cf0 msgr2=0x7f35ec100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec104cf0 0x7f35ec100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 -- 172.21.15.105:0/2571266817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec101140 con 0x7f35ec107620 2024-03-26T17:38:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f2285640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec107620 0x7f35ec1008b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f35e400eb10 tx=0x7f35e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:09.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35e2ffd640 1 -- 172.21.15.105:0/2571266817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4004b20 con 0x7f35ec107620 2024-03-26T17:38:09.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35e2ffd640 1 -- 172.21.15.105:0/2571266817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e4004cc0 con 0x7f35ec107620 2024-03-26T17:38:09.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec1a1c70 con 0x7f35ec107620 2024-03-26T17:38:09.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35e2ffd640 1 -- 172.21.15.105:0/2571266817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4010690 con 0x7f35ec107620 2024-03-26T17:38:09.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.713+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec1a2190 con 0x7f35ec107620 2024-03-26T17:38:09.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.714+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec068b00 con 0x7f35ec107620 2024-03-26T17:38:09.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.716+0000 7f35e2ffd640 1 -- 172.21.15.105:0/2571266817 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f35e4005000 con 0x7f35ec107620 2024-03-26T17:38:09.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.716+0000 7f35e2ffd640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d80a2f20 0x7f35d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:09.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.716+0000 7f35f1a84640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d80a2f20 0x7f35d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:09.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.716+0000 7f35e2ffd640 1 -- 172.21.15.105:0/2571266817 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f35e40cfdf0 con 0x7f35ec107620 2024-03-26T17:38:09.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.717+0000 7f35f1a84640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d80a2f20 0x7f35d80a53e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009bd0 tx=0x7f35e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:09.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.720+0000 7f35e2ffd640 1 -- 172.21.15.105:0/2571266817 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f35e409cc40 con 0x7f35ec107620 2024-03-26T17:38:09.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.898+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f35ec02d270 con 0x7f35ec107620 2024-03-26T17:38:09.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.898+0000 7f35e2ffd640 1 -- 172.21.15.105:0/2571266817 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f35e409c540 con 0x7f35ec107620 2024-03-26T17:38:09.892 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T17:38:09.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:09 smithi105 ceph-mon[32280]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:38:09.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:09 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/525972110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:09.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:09 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1015648618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d80a2f20 msgr2=0x7f35d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d80a2f20 0x7f35d80a53e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009bd0 tx=0x7f35e8005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:09.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec107620 msgr2=0x7f35ec1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec107620 0x7f35ec1008b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f35e400eb10 tx=0x7f35e400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 shutdown_connections 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d80a2f20 0x7f35d80a53e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec107620 0x7f35ec1008b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec104cf0 0x7f35ec100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 --2- 172.21.15.105:0/2571266817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1023c0 0x7f35ec101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.900+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 >> 172.21.15.105:0/2571266817 conn(0x7f35ec0fbd70 msgr2=0x7f35ec102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.901+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 shutdown_connections 2024-03-26T17:38:09.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:09.901+0000 7f35f3ce8640 1 -- 172.21.15.105:0/2571266817 wait complete. 2024-03-26T17:38:09.906 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T17:38:09.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:09 smithi135 ceph-mon[35636]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:38:09.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:09 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/525972110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:09.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:09 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1015648618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 -- 172.21.15.105:0/1478498759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc944102370 msgr2=0x7fc944108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 --2- 172.21.15.105:0/1478498759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc944102370 0x7fc944108c00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009ca0 tx=0x7fc93c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 -- 172.21.15.105:0/1478498759 shutdown_connections 2024-03-26T17:38:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 --2- 172.21.15.105:0/1478498759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc944102370 0x7fc944108c00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 --2- 172.21.15.105:0/1478498759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9441019b0 0x7fc944101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 --2- 172.21.15.105:0/1478498759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9441007c0 0x7fc944100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 -- 172.21.15.105:0/1478498759 >> 172.21.15.105:0/1478498759 conn(0x7fc9440fbd50 msgr2=0x7fc9440fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 -- 172.21.15.105:0/1478498759 shutdown_connections 2024-03-26T17:38:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.020+0000 7fc9498ff640 1 -- 172.21.15.105:0/1478498759 wait complete. 2024-03-26T17:38:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.021+0000 7fc9498ff640 1 Processor -- start 2024-03-26T17:38:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.021+0000 7fc9498ff640 1 -- start start 2024-03-26T17:38:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9498ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9441007c0 0x7fc9441969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9498ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9441019b0 0x7fc944196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9498ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc944102370 0x7fc94419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9498ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc94410c2d0 con 0x7fc9441019b0 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9498ff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc94410c150 con 0x7fc9441007c0 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9498ff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc94410c450 con 0x7fc944102370 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9441019b0 0x7fc944196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9441019b0 0x7fc944196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33648/0 (socket says 172.21.15.105:33648) 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 -- 172.21.15.105:0/3394478604 learned_addr learned my addr 172.21.15.105:0/3394478604 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc942ffd640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9441007c0 0x7fc9441969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9437fe640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc944102370 0x7fc94419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 -- 172.21.15.105:0/3394478604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc944102370 msgr2=0x7fc94419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc944102370 0x7fc94419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 -- 172.21.15.105:0/3394478604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9441007c0 msgr2=0x7fc9441969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9441007c0 0x7fc9441969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 -- 172.21.15.105:0/3394478604 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94419e5f0 con 0x7fc9441019b0 2024-03-26T17:38:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9437fe640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc944102370 0x7fc94419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc942ffd640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9441007c0 0x7fc9441969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9427fc640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9441019b0 0x7fc944196f10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc93400e9f0 tx=0x7fc93400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9488fd640 1 -- 172.21.15.105:0/3394478604 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93400cd80 con 0x7fc9441019b0 2024-03-26T17:38:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9488fd640 1 -- 172.21.15.105:0/3394478604 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9340108e0 con 0x7fc9441019b0 2024-03-26T17:38:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9488fd640 1 -- 172.21.15.105:0/3394478604 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934018920 con 0x7fc9441019b0 2024-03-26T17:38:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94419e8e0 con 0x7fc9441019b0 2024-03-26T17:38:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.022+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94419ee30 con 0x7fc9441019b0 2024-03-26T17:38:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.023+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc944107bb0 con 0x7fc9441019b0 2024-03-26T17:38:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.025+0000 7fc9488fd640 1 -- 172.21.15.105:0/3394478604 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7fc934010a90 con 0x7fc9441019b0 2024-03-26T17:38:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.026+0000 7fc9488fd640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc92c0a2f20 0x7fc92c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.026+0000 7fc942ffd640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc92c0a2f20 0x7fc92c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.026+0000 7fc9488fd640 1 -- 172.21.15.105:0/3394478604 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fc9340cf700 con 0x7fc9441019b0 2024-03-26T17:38:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.026+0000 7fc942ffd640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc92c0a2f20 0x7fc92c0a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc944101810 tx=0x7fc938005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.030+0000 7fc9488fd640 1 -- 172.21.15.105:0/3394478604 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc93409bea0 con 0x7fc9441019b0 2024-03-26T17:38:10.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:09 smithi144 ceph-mon[36095]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-26T17:38:10.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:09 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/525972110' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:10.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:09 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1015648618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:10.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.210+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc9440624c0 con 0x7fc9441019b0 2024-03-26T17:38:10.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.210+0000 7fc9488fd640 1 -- 172.21.15.105:0/3394478604 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7fc93409ff40 con 0x7fc9441019b0 2024-03-26T17:38:10.204 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T17:38:10.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc92c0a2f20 msgr2=0x7fc92c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc92c0a2f20 0x7fc92c0a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc944101810 tx=0x7fc938005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:10.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9441019b0 msgr2=0x7fc944196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9441019b0 0x7fc944196f10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc93400e9f0 tx=0x7fc93400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 shutdown_connections 2024-03-26T17:38:10.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc92c0a2f20 0x7fc92c0a53e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc944102370 0x7fc94419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9441019b0 0x7fc944196f10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 --2- 172.21.15.105:0/3394478604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9441007c0 0x7fc9441969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 >> 172.21.15.105:0/3394478604 conn(0x7fc9440fbd50 msgr2=0x7fc944104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:10.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 shutdown_connections 2024-03-26T17:38:10.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.212+0000 7fc9498ff640 1 -- 172.21.15.105:0/3394478604 wait complete. 2024-03-26T17:38:10.217 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:38:10.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 -- 172.21.15.105:0/2776675435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081007e0 msgr2=0x7f6408100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 --2- 172.21.15.105:0/2776675435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081007e0 0x7f6408100c00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6404009900 tx=0x7f640402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 -- 172.21.15.105:0/2776675435 shutdown_connections 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 --2- 172.21.15.105:0/2776675435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6408102390 0x7f6408108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 --2- 172.21.15.105:0/2776675435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64081019d0 0x7f6408101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 --2- 172.21.15.105:0/2776675435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081007e0 0x7f6408100c00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 -- 172.21.15.105:0/2776675435 >> 172.21.15.105:0/2776675435 conn(0x7f64080fbd50 msgr2=0x7f64080fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 -- 172.21.15.105:0/2776675435 shutdown_connections 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.327+0000 7f640f50a640 1 -- 172.21.15.105:0/2776675435 wait complete. 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640f50a640 1 Processor -- start 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640f50a640 1 -- start start 2024-03-26T17:38:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640f50a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64081007e0 0x7f6408197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640f50a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081019d0 0x7f64081980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640f50a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6408102390 0x7f640819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640f50a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f640810c310 con 0x7f64081019d0 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640f50a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f640810c190 con 0x7f6408102390 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640f50a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f640810c490 con 0x7f64081007e0 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640caa5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081019d0 0x7f64081980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640caa5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081019d0 0x7f64081980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33662/0 (socket says 172.21.15.105:33662) 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640caa5640 1 -- 172.21.15.105:0/2074066731 learned_addr learned my addr 172.21.15.105:0/2074066731 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640d2a6640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64081007e0 0x7f6408197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.328+0000 7f640daa7640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6408102390 0x7f640819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640caa5640 1 -- 172.21.15.105:0/2074066731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64081007e0 msgr2=0x7f6408197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640caa5640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64081007e0 0x7f6408197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640caa5640 1 -- 172.21.15.105:0/2074066731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6408102390 msgr2=0x7f640819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640caa5640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6408102390 0x7f640819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640caa5640 1 -- 172.21.15.105:0/2074066731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640819f780 con 0x7f64081019d0 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640daa7640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6408102390 0x7f640819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640d2a6640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64081007e0 0x7f6408197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640caa5640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081019d0 0x7f64081980a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e9f0 tx=0x7f63f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f63fe7fc640 1 -- 172.21.15.105:0/2074066731 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800cd80 con 0x7f64081019d0 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640819fa70 con 0x7f64081019d0 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f63fe7fc640 1 -- 172.21.15.105:0/2074066731 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63f80108e0 con 0x7f64081019d0 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f63fe7fc640 1 -- 172.21.15.105:0/2074066731 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8018920 con 0x7f64081019d0 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.329+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64081a0010 con 0x7f64081019d0 2024-03-26T17:38:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.330+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64081a01e0 con 0x7f64081019d0 2024-03-26T17:38:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.332+0000 7f63fe7fc640 1 -- 172.21.15.105:0/2074066731 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f63f8010a90 con 0x7f64081019d0 2024-03-26T17:38:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.332+0000 7f63fe7fc640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f63f40a2f20 0x7f63f40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.332+0000 7f640d2a6640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f63f40a2f20 0x7f63f40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.332+0000 7f63fe7fc640 1 -- 172.21.15.105:0/2074066731 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f63f80cfb10 con 0x7f64081019d0 2024-03-26T17:38:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.333+0000 7f640d2a6640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f63f40a2f20 0x7f63f40a53e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6404005ab0 tx=0x7f6404005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:10.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.336+0000 7f63fe7fc640 1 -- 172.21.15.105:0/2074066731 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f63f809c310 con 0x7f64081019d0 2024-03-26T17:38:10.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.555+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64081a03b0 con 0x7f64081019d0 2024-03-26T17:38:10.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.556+0000 7f63fe7fc640 1 -- 172.21.15.105:0/2074066731 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7f63f80a03b0 con 0x7f64081019d0 2024-03-26T17:38:10.550 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:10.550 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:36:29.308004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:38:10.550 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 26 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f63f40a2f20 msgr2=0x7f63f40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f63f40a2f20 0x7f63f40a53e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6404005ab0 tx=0x7f6404005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081019d0 msgr2=0x7f64081980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081019d0 0x7f64081980a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63f800e9f0 tx=0x7f63f800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 shutdown_connections 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f63f40a2f20 0x7f63f40a53e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6408102390 0x7f640819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64081019d0 0x7f64081980a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 --2- 172.21.15.105:0/2074066731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64081007e0 0x7f6408197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 >> 172.21.15.105:0/2074066731 conn(0x7f64080fbd50 msgr2=0x7f6408068b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 shutdown_connections 2024-03-26T17:38:10.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.558+0000 7f640f50a640 1 -- 172.21.15.105:0/2074066731 wait complete. 2024-03-26T17:38:10.563 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:38:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.673+0000 7f6c63f13640 1 -- 172.21.15.105:0/2819861291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1019d0 msgr2=0x7f6c5c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.673+0000 7f6c63f13640 1 --2- 172.21.15.105:0/2819861291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1019d0 0x7f6c5c101e50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009900 tx=0x7f6c4c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.673+0000 7f6c63f13640 1 -- 172.21.15.105:0/2819861291 shutdown_connections 2024-03-26T17:38:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.673+0000 7f6c63f13640 1 --2- 172.21.15.105:0/2819861291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c5c102390 0x7f6c5c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.673+0000 7f6c63f13640 1 --2- 172.21.15.105:0/2819861291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1019d0 0x7f6c5c101e50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.673+0000 7f6c63f13640 1 --2- 172.21.15.105:0/2819861291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c1007e0 0x7f6c5c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.673+0000 7f6c63f13640 1 -- 172.21.15.105:0/2819861291 >> 172.21.15.105:0/2819861291 conn(0x7f6c5c0fbd50 msgr2=0x7f6c5c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.673+0000 7f6c63f13640 1 -- 172.21.15.105:0/2819861291 shutdown_connections 2024-03-26T17:38:10.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 -- 172.21.15.105:0/2819861291 wait complete. 2024-03-26T17:38:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 Processor -- start 2024-03-26T17:38:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 -- start start 2024-03-26T17:38:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1007e0 0x7f6c5c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c1019d0 0x7f6c5c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c5c102390 0x7f6c5c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10c2f0 con 0x7f6c5c1007e0 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10c170 con 0x7f6c5c102390 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c63f13640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10c470 con 0x7f6c5c1019d0 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c61caf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1007e0 0x7f6c5c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c61caf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1007e0 0x7f6c5c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33682/0 (socket says 172.21.15.105:33682) 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c61caf640 1 -- 172.21.15.105:0/3812436587 learned_addr learned my addr 172.21.15.105:0/3812436587 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c614ae640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c1019d0 0x7f6c5c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.674+0000 7f6c624b0640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c5c102390 0x7f6c5c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.675+0000 7f6c61caf640 1 -- 172.21.15.105:0/3812436587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c1019d0 msgr2=0x7f6c5c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.675+0000 7f6c61caf640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c1019d0 0x7f6c5c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.675+0000 7f6c61caf640 1 -- 172.21.15.105:0/3812436587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c5c102390 msgr2=0x7f6c5c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.675+0000 7f6c61caf640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c5c102390 0x7f6c5c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.675+0000 7f6c61caf640 1 -- 172.21.15.105:0/3812436587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c19f760 con 0x7f6c5c1007e0 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.675+0000 7f6c614ae640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c1019d0 0x7f6c5c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.675+0000 7f6c61caf640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1007e0 0x7f6c5c197b40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800e9c0 tx=0x7f6c5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.676+0000 7f6c52ffd640 1 -- 172.21.15.105:0/3812436587 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800cd80 con 0x7f6c5c1007e0 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.676+0000 7f6c52ffd640 1 -- 172.21.15.105:0/3812436587 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c580035f0 con 0x7f6c5c1007e0 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.676+0000 7f6c52ffd640 1 -- 172.21.15.105:0/3812436587 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58010690 con 0x7f6c5c1007e0 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.676+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c19fa50 con 0x7f6c5c1007e0 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.676+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c19ff20 con 0x7f6c5c1007e0 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.677+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c1a01b0 con 0x7f6c5c1007e0 2024-03-26T17:38:10.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.678+0000 7f6c52ffd640 1 -- 172.21.15.105:0/3812436587 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f6c580037a0 con 0x7f6c5c1007e0 2024-03-26T17:38:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.679+0000 7f6c52ffd640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c480a2f20 0x7f6c480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.679+0000 7f6c614ae640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c480a2f20 0x7f6c480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.679+0000 7f6c52ffd640 1 -- 172.21.15.105:0/3812436587 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f6c580cf800 con 0x7f6c5c1007e0 2024-03-26T17:38:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.679+0000 7f6c614ae640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c480a2f20 0x7f6c480a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009230 tx=0x7f6c4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.687+0000 7f6c52ffd640 1 -- 172.21.15.105:0/3812436587 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6c5809c650 con 0x7f6c5c1007e0 2024-03-26T17:38:10.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.868+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c5c1a0450 con 0x7f6c5c1007e0 2024-03-26T17:38:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.869+0000 7f6c52ffd640 1 -- 172.21.15.105:0/3812436587 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f6c5809bf50 con 0x7f6c5c1007e0 2024-03-26T17:38:10.874 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:10.874 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:38:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c480a2f20 msgr2=0x7f6c480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c480a2f20 0x7f6c480a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009230 tx=0x7f6c4c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1007e0 msgr2=0x7f6c5c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1007e0 0x7f6c5c197b40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800e9c0 tx=0x7f6c5800ee80 comp rx=0 tx=0).stop 2024-03-26T17:38:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 shutdown_connections 2024-03-26T17:38:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c480a2f20 0x7f6c480a53e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c5c102390 0x7f6c5c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c1019d0 0x7f6c5c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 --2- 172.21.15.105:0/3812436587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c5c1007e0 0x7f6c5c197b40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 >> 172.21.15.105:0/3812436587 conn(0x7f6c5c0fbd50 msgr2=0x7f6c5c068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 shutdown_connections 2024-03-26T17:38:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.871+0000 7f6c63f13640 1 -- 172.21.15.105:0/3812436587 wait complete. 2024-03-26T17:38:10.876 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:10 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2571266817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:10.877 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:10 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3394478604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:10.877 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:10 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2074066731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:10.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:38:10.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:38:10.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 -- 172.21.15.105:0/713310813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 msgr2=0x7fb4f41099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/713310813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 0x7fb4f41099f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009ca0 tx=0x7fb4ec02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 -- 172.21.15.105:0/713310813 shutdown_connections 2024-03-26T17:38:10.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/713310813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 0x7fb4f41099f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/713310813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4f4104c70 0x7fb4f4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/713310813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4f4102340 0x7fb4f4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 -- 172.21.15.105:0/713310813 >> 172.21.15.105:0/713310813 conn(0x7fb4f40fbd50 msgr2=0x7fb4f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:10.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 -- 172.21.15.105:0/713310813 shutdown_connections 2024-03-26T17:38:10.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 -- 172.21.15.105:0/713310813 wait complete. 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 Processor -- start 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.995+0000 7fb4f95b3640 1 -- start start 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f95b3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4f4102340 0x7fb4f4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f95b3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4f4104c70 0x7fb4f4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f95b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 0x7fb4f419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f95b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f410d3f0 con 0x7fb4f41075a0 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f95b3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f410d270 con 0x7fb4f4104c70 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f95b3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f410d570 con 0x7fb4f4102340 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 0x7fb4f419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 0x7fb4f419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33690/0 (socket says 172.21.15.105:33690) 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 -- 172.21.15.105:0/1893260665 learned_addr learned my addr 172.21.15.105:0/1893260665 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:10.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f2ffd640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4f4102340 0x7fb4f4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4ebfff640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4f4104c70 0x7fb4f4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 -- 172.21.15.105:0/1893260665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4f4102340 msgr2=0x7fb4f4197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4f4102340 0x7fb4f4197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 -- 172.21.15.105:0/1893260665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4f4104c70 msgr2=0x7fb4f4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4f4104c70 0x7fb4f4198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 -- 172.21.15.105:0/1893260665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f419f720 con 0x7fb4f41075a0 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f2ffd640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4f4102340 0x7fb4f4197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f37fe640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 0x7fb4f419f0b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec02c700 tx=0x7fb4ec0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f0ff9640 1 -- 172.21.15.105:0/1893260665 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec041020 con 0x7fb4f41075a0 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f0ff9640 1 -- 172.21.15.105:0/1893260665 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4ec003e20 con 0x7fb4f41075a0 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f419f9b0 con 0x7fb4f41075a0 2024-03-26T17:38:10.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f0ff9640 1 -- 172.21.15.105:0/1893260665 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec039040 con 0x7fb4f41075a0 2024-03-26T17:38:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.996+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f419fef0 con 0x7fb4f41075a0 2024-03-26T17:38:10.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.997+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f40fd840 con 0x7fb4f41075a0 2024-03-26T17:38:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.998+0000 7fb4f0ff9640 1 -- 172.21.15.105:0/1893260665 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7fb4ec044020 con 0x7fb4f41075a0 2024-03-26T17:38:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.999+0000 7fb4f0ff9640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4e00a2fa0 0x7fb4e00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.999+0000 7fb4f2ffd640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4e00a2fa0 0x7fb4e00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.999+0000 7fb4f0ff9640 1 -- 172.21.15.105:0/1893260665 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb4ec0f14e0 con 0x7fb4f41075a0 2024-03-26T17:38:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:10.999+0000 7fb4f2ffd640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4e00a2fa0 0x7fb4e00a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40095f0 tx=0x7fb4e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:10.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.003+0000 7fb4f0ff9640 1 -- 172.21.15.105:0/1893260665 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb4ec04b020 con 0x7fb4f41075a0 2024-03-26T17:38:11.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:10 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2571266817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:11.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:10 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3394478604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:11.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:10 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2074066731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:11.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:10 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2571266817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:11.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:10 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3394478604' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:11.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:10 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2074066731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:11.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.221+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4f40624c0 con 0x7fb4f41075a0 2024-03-26T17:38:11.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.222+0000 7fb4f0ff9640 1 -- 172.21.15.105:0/1893260665 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fb4ec0be330 con 0x7fb4f41075a0 2024-03-26T17:38:11.215 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4e00a2fa0 msgr2=0x7fb4e00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4e00a2fa0 0x7fb4e00a5460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40095f0 tx=0x7fb4e4005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 msgr2=0x7fb4f419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 0x7fb4f419f0b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec02c700 tx=0x7fb4ec0328a0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 shutdown_connections 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4e00a2fa0 0x7fb4e00a5460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41075a0 0x7fb4f419f0b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4f4104c70 0x7fb4f4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 --2- 172.21.15.105:0/1893260665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4f4102340 0x7fb4f4197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 >> 172.21.15.105:0/1893260665 conn(0x7fb4f40fbd50 msgr2=0x7fb4f4102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 shutdown_connections 2024-03-26T17:38:11.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.224+0000 7fb4f95b3640 1 -- 172.21.15.105:0/1893260665 wait complete. 2024-03-26T17:38:11.231 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T17:38:11.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.343+0000 7f9568ff4640 1 -- 172.21.15.105:0/2135235705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564104d50 msgr2=0x7f9564107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.343+0000 7f9568ff4640 1 --2- 172.21.15.105:0/2135235705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564104d50 0x7f9564107140 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9558009900 tx=0x7f955802bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:11.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.343+0000 7f9568ff4640 1 -- 172.21.15.105:0/2135235705 shutdown_connections 2024-03-26T17:38:11.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.343+0000 7f9568ff4640 1 --2- 172.21.15.105:0/2135235705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9564107680 0x7f9564109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.343+0000 7f9568ff4640 1 --2- 172.21.15.105:0/2135235705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564104d50 0x7f9564107140 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.343+0000 7f9568ff4640 1 --2- 172.21.15.105:0/2135235705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564102420 0x7f9564104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.343+0000 7f9568ff4640 1 -- 172.21.15.105:0/2135235705 >> 172.21.15.105:0/2135235705 conn(0x7f95640fbdf0 msgr2=0x7f95640fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:11.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.344+0000 7f9568ff4640 1 -- 172.21.15.105:0/2135235705 shutdown_connections 2024-03-26T17:38:11.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.344+0000 7f9568ff4640 1 -- 172.21.15.105:0/2135235705 wait complete. 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.344+0000 7f9568ff4640 1 Processor -- start 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.344+0000 7f9568ff4640 1 -- start start 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9568ff4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102420 0x7f9564197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9568ff4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9564104d50 0x7f95641980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9568ff4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564107680 0x7f956419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9568ff4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f956410d3d0 con 0x7f9564102420 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9568ff4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f956410d250 con 0x7f9564104d50 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9568ff4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f956410d550 con 0x7f9564107680 2024-03-26T17:38:11.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102420 0x7f9564197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102420 0x7f9564197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33704/0 (socket says 172.21.15.105:33704) 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 -- 172.21.15.105:0/655480710 learned_addr learned my addr 172.21.15.105:0/655480710 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9561d9b640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9564104d50 0x7f95641980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9562d9d640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564107680 0x7f956419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 -- 172.21.15.105:0/655480710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564107680 msgr2=0x7f956419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564107680 0x7f956419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 -- 172.21.15.105:0/655480710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9564104d50 msgr2=0x7f95641980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9564104d50 0x7f95641980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 -- 172.21.15.105:0/655480710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956419f740 con 0x7f9564102420 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f956259c640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102420 0x7f9564197b70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f955c00db80 tx=0x7f955c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:11.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f95537fe640 1 -- 172.21.15.105:0/655480710 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c017070 con 0x7f9564102420 2024-03-26T17:38:11.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956419fa30 con 0x7f9564102420 2024-03-26T17:38:11.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f95537fe640 1 -- 172.21.15.105:0/655480710 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f955c003b70 con 0x7f9564102420 2024-03-26T17:38:11.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f95537fe640 1 -- 172.21.15.105:0/655480710 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c010480 con 0x7f9564102420 2024-03-26T17:38:11.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.345+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956419ffa0 con 0x7f9564102420 2024-03-26T17:38:11.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.346+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95640fd8e0 con 0x7f9564102420 2024-03-26T17:38:11.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.348+0000 7f95537fe640 1 -- 172.21.15.105:0/655480710 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f955c004020 con 0x7f9564102420 2024-03-26T17:38:11.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.348+0000 7f95537fe640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f954c0a2f20 0x7f954c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:11.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.348+0000 7f95537fe640 1 -- 172.21.15.105:0/655480710 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f955c0cfbe0 con 0x7f9564102420 2024-03-26T17:38:11.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.348+0000 7f9561d9b640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f954c0a2f20 0x7f954c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:11.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.349+0000 7f9561d9b640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f954c0a2f20 0x7f954c0a53e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9558005ab0 tx=0x7f9558005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:11.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.352+0000 7f95537fe640 1 -- 172.21.15.105:0/655480710 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f955c09ca30 con 0x7f9564102420 2024-03-26T17:38:11.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.532+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f956402d240 con 0x7f9564102420 2024-03-26T17:38:11.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.533+0000 7f95537fe640 1 -- 172.21.15.105:0/655480710 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f955c09c330 con 0x7f9564102420 2024-03-26T17:38:11.526 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T17:38:11.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f954c0a2f20 msgr2=0x7f954c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f954c0a2f20 0x7f954c0a53e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9558005ab0 tx=0x7f9558005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102420 msgr2=0x7f9564197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102420 0x7f9564197b70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f955c00db80 tx=0x7f955c00b630 comp rx=0 tx=0).stop 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 shutdown_connections 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f954c0a2f20 0x7f954c0a53e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564107680 0x7f956419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9564104d50 0x7f95641980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 --2- 172.21.15.105:0/655480710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102420 0x7f9564197b70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 >> 172.21.15.105:0/655480710 conn(0x7f95640fbdf0 msgr2=0x7f9564102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 shutdown_connections 2024-03-26T17:38:11.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.535+0000 7f9568ff4640 1 -- 172.21.15.105:0/655480710 wait complete. 2024-03-26T17:38:11.541 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- 172.21.15.105:0/1712529916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a01007e0 msgr2=0x7f97a0100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/1712529916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a01007e0 0x7f97a0100c00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9794009900 tx=0x7f979402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- 172.21.15.105:0/1712529916 shutdown_connections 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/1712529916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97a0102390 0x7f97a0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/1712529916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a01019d0 0x7f97a0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/1712529916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a01007e0 0x7f97a0100c00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- 172.21.15.105:0/1712529916 >> 172.21.15.105:0/1712529916 conn(0x7f97a00fbd70 msgr2=0x7f97a00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- 172.21.15.105:0/1712529916 shutdown_connections 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- 172.21.15.105:0/1712529916 wait complete. 2024-03-26T17:38:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 Processor -- start 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- start start 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a01007e0 0x7f97a0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97a01019d0 0x7f97a0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a0102390 0x7f97a019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97a010c300 con 0x7f97a0102390 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97a010c180 con 0x7f97a01019d0 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.654+0000 7f97a6eb8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97a010c480 con 0x7f97a01007e0 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a0102390 0x7f97a019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a0102390 0x7f97a019f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33722/0 (socket says 172.21.15.105:33722) 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 -- 172.21.15.105:0/3443798315 learned_addr learned my addr 172.21.15.105:0/3443798315 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f979ffff640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97a01019d0 0x7f97a0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a4c54640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a01007e0 0x7f97a0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 -- 172.21.15.105:0/3443798315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a01007e0 msgr2=0x7f97a0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a01007e0 0x7f97a0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 -- 172.21.15.105:0/3443798315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97a01019d0 msgr2=0x7f97a0198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97a01019d0 0x7f97a0198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 -- 172.21.15.105:0/3443798315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a019f770 con 0x7f97a0102390 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a5455640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a0102390 0x7f97a019f100 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f979800eb10 tx=0x7f979800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f979dffb640 1 -- 172.21.15.105:0/3443798315 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798004b20 con 0x7f97a0102390 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a019fa60 con 0x7f97a0102390 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f979dffb640 1 -- 172.21.15.105:0/3443798315 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9798004cc0 con 0x7f97a0102390 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f979dffb640 1 -- 172.21.15.105:0/3443798315 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9798010690 con 0x7f97a0102390 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.655+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a019ffb0 con 0x7f97a0102390 2024-03-26T17:38:11.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.656+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a01a0240 con 0x7f97a0102390 2024-03-26T17:38:11.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.657+0000 7f979dffb640 1 -- 172.21.15.105:0/3443798315 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f9798005020 con 0x7f97a0102390 2024-03-26T17:38:11.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.658+0000 7f979dffb640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97900a2f20 0x7f97900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:11.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.658+0000 7f97a4c54640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97900a2f20 0x7f97900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:11.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.658+0000 7f979dffb640 1 -- 172.21.15.105:0/3443798315 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f97980cf920 con 0x7f97a0102390 2024-03-26T17:38:11.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.658+0000 7f97a4c54640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97900a2f20 0x7f97900a53e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9794005ab0 tx=0x7f9794005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:11.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.662+0000 7f979dffb640 1 -- 172.21.15.105:0/3443798315 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f979809c770 con 0x7f97a0102390 2024-03-26T17:38:11.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:11 smithi105 ceph-mon[32280]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-26T17:38:11.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:11 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3812436587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:11.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:11 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1893260665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:11.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:11 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/655480710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.921+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f97a01a0410 con 0x7f97a0102390 2024-03-26T17:38:11.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.922+0000 7f979dffb640 1 -- 172.21.15.105:0/3443798315 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f979809c070 con 0x7f97a0102390 2024-03-26T17:38:11.916 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T17:38:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97900a2f20 msgr2=0x7f97900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97900a2f20 0x7f97900a53e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9794005ab0 tx=0x7f9794005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a0102390 msgr2=0x7f97a019f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a0102390 0x7f97a019f100 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f979800eb10 tx=0x7f979800c2d0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 shutdown_connections 2024-03-26T17:38:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97900a2f20 0x7f97900a53e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97a0102390 0x7f97a019f100 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97a01019d0 0x7f97a0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 --2- 172.21.15.105:0/3443798315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97a01007e0 0x7f97a0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 >> 172.21.15.105:0/3443798315 conn(0x7f97a00fbd70 msgr2=0x7f97a0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 shutdown_connections 2024-03-26T17:38:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:11.924+0000 7f97a6eb8640 1 -- 172.21.15.105:0/3443798315 wait complete. 2024-03-26T17:38:11.929 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:38:12.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.038+0000 7f4a81589640 1 -- 172.21.15.105:0/1110136764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 msgr2=0x7f4a7c1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.038+0000 7f4a81589640 1 --2- 172.21.15.105:0/1110136764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 0x7f4a7c1070e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009ca0 tx=0x7f4a7002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 -- 172.21.15.105:0/1110136764 shutdown_connections 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 --2- 172.21.15.105:0/1110136764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7c107620 0x7f4a7c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 --2- 172.21.15.105:0/1110136764 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 0x7f4a7c1070e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 --2- 172.21.15.105:0/1110136764 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a7c1023c0 0x7f4a7c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 -- 172.21.15.105:0/1110136764 >> 172.21.15.105:0/1110136764 conn(0x7f4a7c0fbd50 msgr2=0x7f4a7c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 -- 172.21.15.105:0/1110136764 shutdown_connections 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 -- 172.21.15.105:0/1110136764 wait complete. 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 Processor -- start 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 -- start start 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a7c1023c0 0x7f4a7c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 0x7f4a7c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7c107620 0x7f4a7c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c10d410 con 0x7f4a7c104cf0 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c10d290 con 0x7f4a7c1023c0 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.039+0000 7f4a81589640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c10d590 con 0x7f4a7c107620 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 0x7f4a7c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 0x7f4a7c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33738/0 (socket says 172.21.15.105:33738) 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 -- 172.21.15.105:0/3574050712 learned_addr learned my addr 172.21.15.105:0/3574050712 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7affd640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a7c1023c0 0x7f4a7c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7b7fe640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7c107620 0x7f4a7c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 -- 172.21.15.105:0/3574050712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7c107620 msgr2=0x7f4a7c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7c107620 0x7f4a7c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 -- 172.21.15.105:0/3574050712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a7c1023c0 msgr2=0x7f4a7c197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a7c1023c0 0x7f4a7c197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 -- 172.21.15.105:0/3574050712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c19f730 con 0x7f4a7c104cf0 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a7a7fc640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 0x7f4a7c198050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4a70032380 tx=0x7f4a70032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a63fff640 1 -- 172.21.15.105:0/3574050712 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70041020 con 0x7f4a7c104cf0 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a63fff640 1 -- 172.21.15.105:0/3574050712 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a70003e20 con 0x7f4a7c104cf0 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7c19f9c0 con 0x7f4a7c104cf0 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a63fff640 1 -- 172.21.15.105:0/3574050712 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70039040 con 0x7f4a7c104cf0 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.040+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7c19feb0 con 0x7f4a7c104cf0 2024-03-26T17:38:12.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.041+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7c0fd840 con 0x7f4a7c104cf0 2024-03-26T17:38:12.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.043+0000 7f4a63fff640 1 -- 172.21.15.105:0/3574050712 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f4a70044020 con 0x7f4a7c104cf0 2024-03-26T17:38:12.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.043+0000 7f4a63fff640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4a640a2f20 0x7f4a640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.044+0000 7f4a7affd640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4a640a2f20 0x7f4a640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.044+0000 7f4a63fff640 1 -- 172.21.15.105:0/3574050712 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f4a700f13e0 con 0x7f4a7c104cf0 2024-03-26T17:38:12.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.044+0000 7f4a7affd640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4a640a2f20 0x7f4a640a53e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a740095f0 tx=0x7f4a74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.049+0000 7f4a63fff640 1 -- 172.21.15.105:0/3574050712 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4a7004b020 con 0x7f4a7c104cf0 2024-03-26T17:38:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:11 smithi144 ceph-mon[36095]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-26T17:38:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:11 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3812436587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:11 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1893260665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:11 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/655480710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:11 smithi135 ceph-mon[35636]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-03-26T17:38:12.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:11 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3812436587' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:12.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:11 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1893260665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:12.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:11 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/655480710' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:12.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.272+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a7c0624f0 con 0x7f4a7c104cf0 2024-03-26T17:38:12.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.273+0000 7f4a63fff640 1 -- 172.21.15.105:0/3574050712 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11941 (secure 0 0 0) 0x7f4a700be1d0 con 0x7f4a7c104cf0 2024-03-26T17:38:12.267 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:12.267 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:36:29.308004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:38:12.267 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 26 2024-03-26T17:38:12.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4a640a2f20 msgr2=0x7f4a640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4a640a2f20 0x7f4a640a53e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a740095f0 tx=0x7f4a74005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:12.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 msgr2=0x7f4a7c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 0x7f4a7c198050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4a70032380 tx=0x7f4a70032820 comp rx=0 tx=0).stop 2024-03-26T17:38:12.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 shutdown_connections 2024-03-26T17:38:12.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4a640a2f20 0x7f4a640a53e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a7c107620 0x7f4a7c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4a7c104cf0 0x7f4a7c198050 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 --2- 172.21.15.105:0/3574050712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a7c1023c0 0x7f4a7c197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.275+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 >> 172.21.15.105:0/3574050712 conn(0x7f4a7c0fbd50 msgr2=0x7f4a7c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.276+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 shutdown_connections 2024-03-26T17:38:12.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.276+0000 7f4a81589640 1 -- 172.21.15.105:0/3574050712 wait complete. 2024-03-26T17:38:12.280 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 -- 172.21.15.105:0/971019045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 msgr2=0x7f8dfc101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 --2- 172.21.15.105:0/971019045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 0x7f8dfc101e50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009900 tx=0x7f8df002bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 -- 172.21.15.105:0/971019045 shutdown_connections 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 --2- 172.21.15.105:0/971019045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8dfc102390 0x7f8dfc108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 --2- 172.21.15.105:0/971019045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 0x7f8dfc101e50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 --2- 172.21.15.105:0/971019045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dfc1007e0 0x7f8dfc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 -- 172.21.15.105:0/971019045 >> 172.21.15.105:0/971019045 conn(0x7f8dfc0fbd50 msgr2=0x7f8dfc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 -- 172.21.15.105:0/971019045 shutdown_connections 2024-03-26T17:38:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.389+0000 7f8e02960640 1 -- 172.21.15.105:0/971019045 wait complete. 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e02960640 1 Processor -- start 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e02960640 1 -- start start 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e02960640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8dfc1007e0 0x7f8dfc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e02960640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 0x7f8dfc19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e02960640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dfc102390 0x7f8dfc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e02960640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10c310 con 0x7f8dfc1019d0 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e02960640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10c190 con 0x7f8dfc1007e0 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e02960640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc10c490 con 0x7f8dfc102390 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 0x7f8dfc19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8dfc1007e0 0x7f8dfc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 0x7f8dfc19c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33766/0 (socket says 172.21.15.105:33766) 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfb7fe640 1 -- 172.21.15.105:0/1667014169 learned_addr learned my addr 172.21.15.105:0/1667014169 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8e00efd640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dfc102390 0x7f8dfc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfb7fe640 1 -- 172.21.15.105:0/1667014169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dfc102390 msgr2=0x7f8dfc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfb7fe640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dfc102390 0x7f8dfc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfb7fe640 1 -- 172.21.15.105:0/1667014169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8dfc1007e0 msgr2=0x7f8dfc069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfb7fe640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8dfc1007e0 0x7f8dfc069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfb7fe640 1 -- 172.21.15.105:0/1667014169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc19f7c0 con 0x7f8dfc1019d0 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.390+0000 7f8dfbfff640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8dfc1007e0 0x7f8dfc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.391+0000 7f8dfb7fe640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 0x7f8dfc19c890 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8df0006010 tx=0x7f8df0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.391+0000 7f8df97fa640 1 -- 172.21.15.105:0/1667014169 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0043020 con 0x7f8dfc1019d0 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.391+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc19f9f0 con 0x7f8dfc1019d0 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.391+0000 7f8df97fa640 1 -- 172.21.15.105:0/1667014169 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df00043d0 con 0x7f8dfc1019d0 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.391+0000 7f8df97fa640 1 -- 172.21.15.105:0/1667014169 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0003b80 con 0x7f8dfc1019d0 2024-03-26T17:38:12.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.391+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc19ff90 con 0x7f8dfc1019d0 2024-03-26T17:38:12.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.392+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dfc0686b0 con 0x7f8dfc1019d0 2024-03-26T17:38:12.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.393+0000 7f8df97fa640 1 -- 172.21.15.105:0/1667014169 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f8df0046020 con 0x7f8dfc1019d0 2024-03-26T17:38:12.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.393+0000 7f8df97fa640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8de40a2f20 0x7f8de40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.394+0000 7f8dfbfff640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8de40a2f20 0x7f8de40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.394+0000 7f8df97fa640 1 -- 172.21.15.105:0/1667014169 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f8df00f1c20 con 0x7f8dfc1019d0 2024-03-26T17:38:12.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.394+0000 7f8dfbfff640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8de40a2f20 0x7f8de40a53e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc101830 tx=0x7f8dec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.401+0000 7f8df97fa640 1 -- 172.21.15.105:0/1667014169 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8df004d020 con 0x7f8dfc1019d0 2024-03-26T17:38:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.580+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8dfc0624c0 con 0x7f8dfc1019d0 2024-03-26T17:38:12.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.582+0000 7f8df97fa640 1 -- 172.21.15.105:0/1667014169 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f8df00bea70 con 0x7f8dfc1019d0 2024-03-26T17:38:12.576 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:12.576 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:38:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.584+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8de40a2f20 msgr2=0x7f8de40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.584+0000 7f8e02960640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8de40a2f20 0x7f8de40a53e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc101830 tx=0x7f8dec005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.584+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 msgr2=0x7f8dfc19c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.584+0000 7f8e02960640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 0x7f8dfc19c890 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8df0006010 tx=0x7f8df0005e40 comp rx=0 tx=0).stop 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.585+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 shutdown_connections 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.585+0000 7f8e02960640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8de40a2f20 0x7f8de40a53e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.585+0000 7f8e02960640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dfc102390 0x7f8dfc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.585+0000 7f8e02960640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8dfc1019d0 0x7f8dfc19c890 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.585+0000 7f8e02960640 1 --2- 172.21.15.105:0/1667014169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8dfc1007e0 0x7f8dfc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.585+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 >> 172.21.15.105:0/1667014169 conn(0x7f8dfc0fbd50 msgr2=0x7f8dfc104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.585+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 shutdown_connections 2024-03-26T17:38:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.585+0000 7f8e02960640 1 -- 172.21.15.105:0/1667014169 wait complete. 2024-03-26T17:38:12.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T17:36:22.059727+0000', 'modified': '2024-03-26T17:36:29.308004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24521, 'mds_1': 24487, 'mds_2': 24482, 'mds_3': 24466, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6839/3053296748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 3053296748}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 3053296748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6841/3843959520', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6840', 'nonce': 3843959520}, {'type': 'v1', 'addr': '172.21.15.105:6841', 'nonce': 3843959520}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6833/3569777961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 3569777961}, {'type': 'v1', 'addr': '172.21.15.144:6833', 'nonce': 3569777961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6833/1499648854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6832', 'nonce': 1499648854}, {'type': 'v1', 'addr': '172.21.15.135:6833', 'nonce': 1499648854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/29089236', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 29089236}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 29089236}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6835/3121985659', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 3121985659}, {'type': 'v1', 'addr': '172.21.15.135:6835', 'nonce': 3121985659}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.144:6837/184508646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 184508646}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 184508646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.135:6837/388686483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6836', 'nonce': 388686483}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 388686483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.144:6839/1175380833', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6838', 'nonce': 1175380833}, {'type': 'v1', 'addr': '172.21.15.144:6839', 'nonce': 1175380833}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.135:6839/338591305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 338591305}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 338591305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24482, 24521, 14628, 24466, 24487]} 2024-03-26T17:38:12.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-26T17:38:12.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 -- 172.21.15.105:0/1386225815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c102ae0 msgr2=0x7fb90c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 --2- 172.21.15.105:0/1386225815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c102ae0 0x7fb90c102f60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009900 tx=0x7fb8f402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 -- 172.21.15.105:0/1386225815 shutdown_connections 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 --2- 172.21.15.105:0/1386225815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb90c1034a0 0x7fb90c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 --2- 172.21.15.105:0/1386225815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c102ae0 0x7fb90c102f60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 --2- 172.21.15.105:0/1386225815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb90c1018f0 0x7fb90c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 -- 172.21.15.105:0/1386225815 >> 172.21.15.105:0/1386225815 conn(0x7fb90c0fce60 msgr2=0x7fb90c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 -- 172.21.15.105:0/1386225815 shutdown_connections 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.703+0000 7fb911909640 1 -- 172.21.15.105:0/1386225815 wait complete. 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb911909640 1 Processor -- start 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb911909640 1 -- start start 2024-03-26T17:38:12.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb911909640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb90c1018f0 0x7fb90c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb911909640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb90c102ae0 0x7fb90c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb911909640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c1034a0 0x7fb90c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb911909640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10d420 con 0x7fb90c1034a0 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb911909640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10d2a0 con 0x7fb90c1018f0 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb911909640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c10d5a0 con 0x7fb90c102ae0 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c1034a0 0x7fb90c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c1034a0 0x7fb90c19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33782/0 (socket says 172.21.15.105:33782) 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 -- 172.21.15.105:0/2063750437 learned_addr learned my addr 172.21.15.105:0/2063750437 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb903fff640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb90c102ae0 0x7fb90c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90affd640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb90c1018f0 0x7fb90c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 -- 172.21.15.105:0/2063750437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb90c102ae0 msgr2=0x7fb90c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb90c102ae0 0x7fb90c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 -- 172.21.15.105:0/2063750437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb90c1018f0 msgr2=0x7fb90c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb90c1018f0 0x7fb90c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 -- 172.21.15.105:0/2063750437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c19f780 con 0x7fb90c1034a0 2024-03-26T17:38:12.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c1034a0 0x7fb90c19f110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb90400eb10 tx=0x7fb90400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.704+0000 7fb910907640 1 -- 172.21.15.105:0/2063750437 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904004b20 con 0x7fb90c1034a0 2024-03-26T17:38:12.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.705+0000 7fb911909640 1 -- 172.21.15.105:0/2063750437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c19fa70 con 0x7fb90c1034a0 2024-03-26T17:38:12.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.705+0000 7fb910907640 1 -- 172.21.15.105:0/2063750437 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb904004cc0 con 0x7fb90c1034a0 2024-03-26T17:38:12.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.705+0000 7fb910907640 1 -- 172.21.15.105:0/2063750437 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904010690 con 0x7fb90c1034a0 2024-03-26T17:38:12.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.705+0000 7fb911909640 1 -- 172.21.15.105:0/2063750437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c19ff40 con 0x7fb90c1034a0 2024-03-26T17:38:12.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.706+0000 7fb910907640 1 -- 172.21.15.105:0/2063750437 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7fb904005020 con 0x7fb90c1034a0 2024-03-26T17:38:12.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.707+0000 7fb910907640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8f80a2f20 0x7fb8f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.707+0000 7fb90affd640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8f80a2f20 0x7fb8f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb910907640 1 -- 172.21.15.105:0/2063750437 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb9040cf940 con 0x7fb90c1034a0 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb9145d4740 1 Processor -- start 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8f80a2f20 0x7fb8f80a53e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb90c102940 tx=0x7fb8fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb9145d4740 1 -- start start 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 0x559549f30290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f307d0 0x559549f37060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb9145d4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559549e93270 con 0x559549f2ec40 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb9145d4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559549e3bc80 con 0x559549f307d0 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb9145d4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559549e8f220 con 0x559549f2fe10 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f2f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33796/0 (socket says 172.21.15.105:33796) 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 -- 172.21.15.105:0/1922824616 learned_addr learned my addr 172.21.15.105:0/1922824616 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb903fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 0x559549f30290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/1922824616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f307d0 0x559549f37060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 -- 172.21.15.105:0/1922824616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 msgr2=0x559549f30290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 --2- 172.21.15.105:0/1922824616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 0x559549f30290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 -- 172.21.15.105:0/1922824616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f307d0 msgr2=0x559549f37060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 --2- 172.21.15.105:0/1922824616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f307d0 0x559549f37060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.708+0000 7fb90affd640 1 -- 172.21.15.105:0/1922824616 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559549e4dfb0 con 0x559549f2ec40 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb90affd640 1 --2- 172.21.15.105:0/1922824616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f2f060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0058c0 tx=0x7fb8fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40567911d8f43c26 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb901ffb640 1 -- 172.21.15.105:0/1922824616 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc026430 con 0x559549f2ec40 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb901ffb640 1 -- 172.21.15.105:0/1922824616 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc01c8c0 con 0x559549f2ec40 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb901ffb640 1 -- 172.21.15.105:0/1922824616 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc01d8d0 con 0x559549f2ec40 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 -- 172.21.15.105:0/1922824616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 msgr2=0x559549f2f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1922824616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f2f060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc0058c0 tx=0x7fb8fc014000 comp rx=0 tx=0).stop 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 -- 172.21.15.105:0/1922824616 shutdown_connections 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1922824616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f307d0 0x559549f37060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1922824616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 0x559549f30290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1922824616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f2f060 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 -- 172.21.15.105:0/1922824616 >> 172.21.15.105:0/1922824616 conn(0x559549f2a030 msgr2=0x559549f2a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 -- 172.21.15.105:0/1922824616 shutdown_connections 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.709+0000 7fb9145d4740 1 -- 172.21.15.105:0/1922824616 wait complete. 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 1 -- start start 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 1 Processor -- start 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f39150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 0x559549f39690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 0x559549f406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559549f377e0 con 0x559549f2ec40 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559549e4efc0 con 0x559549f2fe10 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559549f37960 con 0x559549f307d0 2024-03-26T17:38:12.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb90affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f39150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb9145d4740 10 client.0 _ll_register_callbacks cb 0x559549e49770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.710+0000 7fb90affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f39150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33804/0 (socket says 172.21.15.105:33804) 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb90affd640 1 -- 172.21.15.105:0/4216775512 learned_addr learned my addr 172.21.15.105:0/4216775512 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 0x559549f406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb903fff640 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 0x559549f39690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb90affd640 1 -- 172.21.15.105:0/4216775512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 msgr2=0x559549f406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb90affd640 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 0x559549f406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb90affd640 1 -- 172.21.15.105:0/4216775512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 msgr2=0x559549f39690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb90affd640 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 0x559549f39690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb90affd640 1 -- 172.21.15.105:0/4216775512 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559549ba72d0 con 0x559549f2ec40 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb90affd640 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f39150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005850 tx=0x7fb8fc00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672c89e99d740c12 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb8eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb8eaffd640 1 -- 172.21.15.105:0/4216775512 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc01ca70 con 0x559549f2ec40 2024-03-26T17:38:12.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb8eaffd640 1 -- 172.21.15.105:0/4216775512 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb8fc01cc10 con 0x559549f2ec40 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb9145d4740 1 -- 172.21.15.105:0/4216775512 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc01c8c0 con 0x559549f2ec40 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb8eaffd640 1 -- 172.21.15.105:0/4216775512 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb8fc03d030 con 0x559549f2ec40 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb9145d4740 10 client.15240 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb9145d4740 1 -- 172.21.15.105:0/4216775512 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559549c716b0 con 0x559549f2ec40 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.711+0000 7fb8eaffd640 1 -- 172.21.15.105:0/4216775512 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc01c8c0 con 0x559549f2ec40 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb8eaffd640 1 -- 172.21.15.105:0/4216775512 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb8fc035070 con 0x559549f2ec40 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb9145d4740 10 client.15240 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb9145d4740 10 client.15240 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb9145d4740 20 client.15240 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb9145d4740 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559549f45b30 0x559549f46fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb9145d4740 4 client.15240 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb9145d4740 1 -- 172.21.15.105:0/4216775512 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559549ea07b0 con 0x559549f45b30 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb903fff640 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559549f45b30 0x559549f46fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb903fff640 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559549f45b30 0x559549f46fa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bbfe998c202e1624 server_cookie=ea2ebdd51caaa0eb in_seq=0 out_seq=0 2024-03-26T17:38:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.712+0000 7fb8eaffd640 10 client.15240 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb8eaffd640 1 -- 172.21.15.105:0/4216775512 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559549ea07b0 con 0x559549f45b30 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb8eaffd640 10 client.15240 handle_command_reply: tid=0 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 client.15240 shutdown 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 -- 172.21.15.105:0/4216775512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 msgr2=0x559549f39150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f39150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005850 tx=0x7fb8fc00edb0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 -- 172.21.15.105:0/4216775512 shutdown_connections 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559549f45b30 0x559549f46fa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 0x559549f406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 0x559549f39690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 --2- 172.21.15.105:0/4216775512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f39150 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb8eaffd640 0 client.15240 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 -- 172.21.15.105:0/4216775512 >> 172.21.15.105:0/4216775512 conn(0x559549f2a030 msgr2=0x559549f29230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 -- 172.21.15.105:0/4216775512 shutdown_connections 2024-03-26T17:38:12.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 1 -- 172.21.15.105:0/4216775512 wait complete. 2024-03-26T17:38:12.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.713+0000 7fb9145d4740 20 client.15240 trim_cache size 0 max 16384 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb9145d4740 1 Processor -- start 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb9145d4740 1 -- start start 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 0x559549f40660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 0x559549f395f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb9145d4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559549f452e0 con 0x559549f2ec40 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb9145d4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559549f459b0 con 0x559549f2fe10 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb9145d4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559549f46080 con 0x559549f307d0 2024-03-26T17:38:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f36f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33810/0 (socket says 172.21.15.105:33810) 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90affd640 1 -- 172.21.15.105:0/2270379113 learned_addr learned my addr 172.21.15.105:0/2270379113 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 0x559549f395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb903fff640 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 0x559549f40660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90affd640 1 -- 172.21.15.105:0/2270379113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 msgr2=0x559549f395f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90affd640 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 0x559549f395f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90affd640 1 -- 172.21.15.105:0/2270379113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 msgr2=0x559549f40660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90affd640 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 0x559549f40660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.731+0000 7fb90affd640 1 -- 172.21.15.105:0/2270379113 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559549f36bc0 con 0x559549f2ec40 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb90affd640 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f36f40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc016040 tx=0x7fb8fc00e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5dfcf9d79fbd06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9017fa640 1 -- 172.21.15.105:0/2270379113 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc01d410 con 0x559549f2ec40 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9017fa640 1 -- 172.21.15.105:0/2270379113 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc004eb0 con 0x559549f2ec40 2024-03-26T17:38:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9017fa640 1 -- 172.21.15.105:0/2270379113 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc01c6a0 con 0x559549f2ec40 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 -- 172.21.15.105:0/2270379113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 msgr2=0x559549f36f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f36f40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc016040 tx=0x7fb8fc00e520 comp rx=0 tx=0).stop 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 -- 172.21.15.105:0/2270379113 shutdown_connections 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f307d0 0x559549f395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2fe10 0x559549f40660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 --2- 172.21.15.105:0/2270379113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f2ec40 0x559549f36f40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 -- 172.21.15.105:0/2270379113 >> 172.21.15.105:0/2270379113 conn(0x559549f2a030 msgr2=0x559549f24e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 -- 172.21.15.105:0/2270379113 shutdown_connections 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.732+0000 7fb9145d4740 1 -- 172.21.15.105:0/2270379113 wait complete. 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 1 -- start start 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 1 Processor -- start 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2ec40 0x559549f3e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 0x559549fa7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f307d0 0x559549fa9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559549f46e20 con 0x559549f307d0 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559549f46750 con 0x559549f2ec40 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559549f474f0 con 0x559549f2fe10 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb9145d4740 10 client.0 _ll_register_callbacks cb 0x7fb90400cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb90b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f307d0 0x559549fa9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb90b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f307d0 0x559549fa9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33820/0 (socket says 172.21.15.105:33820) 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb90affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2ec40 0x559549f3e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb90b7fe640 1 -- 172.21.15.105:0/1346870683 learned_addr learned my addr 172.21.15.105:0/1346870683 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb903fff640 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 0x559549fa7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb90b7fe640 1 -- 172.21.15.105:0/1346870683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 msgr2=0x559549fa7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 0x559549fa7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.733+0000 7fb90b7fe640 1 -- 172.21.15.105:0/1346870683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2ec40 msgr2=0x559549f3e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2ec40 0x559549f3e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb90b7fe640 1 -- 172.21.15.105:0/1346870683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559549faa600 con 0x559549f307d0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb90affd640 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2ec40 0x559549f3e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb90b7fe640 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f307d0 0x559549fa9dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb90400cfb0 tx=0x7fb90409c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7b596f866fbbcc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb8eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb8eaffd640 1 -- 172.21.15.105:0/1346870683 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904096be0 con 0x559549f307d0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb8eaffd640 1 -- 172.21.15.105:0/1346870683 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb904096d80 con 0x559549f307d0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 1 -- 172.21.15.105:0/1346870683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc004eb0 con 0x559549f307d0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb8eaffd640 1 -- 172.21.15.105:0/1346870683 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb90409c2d0 con 0x559549f307d0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 10 client.15246 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 1 -- 172.21.15.105:0/1346870683 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559549fab9d0 con 0x559549f307d0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb8eaffd640 1 -- 172.21.15.105:0/1346870683 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc004eb0 con 0x559549f307d0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb8eaffd640 1 -- 172.21.15.105:0/1346870683 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb904083020 con 0x559549f307d0 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 10 client.15246 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 10 client.15246 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 20 client.15246 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559549fb0590 0x559549faf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 4 client.15246 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb9145d4740 1 -- 172.21.15.105:0/1346870683 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "status"}) -- 0x559549e9ab00 con 0x559549fb0590 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.734+0000 7fb90affd640 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559549fb0590 0x559549faf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb90affd640 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559549fb0590 0x559549faf2a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e04363cb4fb0034 server_cookie=15255aa83a345cb in_seq=0 out_seq=0 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb8eaffd640 10 client.15246 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb8eaffd640 1 -- 172.21.15.105:0/1346870683 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559549e9ab00 con 0x559549fb0590 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb8eaffd640 10 client.15246 handle_command_reply: tid=0 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 client.15246 shutdown 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 -- 172.21.15.105:0/1346870683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f307d0 msgr2=0x559549fa9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f307d0 0x559549fa9dd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb90400cfb0 tx=0x7fb90409c070 comp rx=0 tx=0).stop 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 -- 172.21.15.105:0/1346870683 shutdown_connections 2024-03-26T17:38:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559549fb0590 0x559549faf2a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559549f307d0 0x559549fa9dd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559549f2fe10 0x559549fa7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 --2- 172.21.15.105:0/1346870683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559549f2ec40 0x559549f3e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb8eaffd640 0 client.15246 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 -- 172.21.15.105:0/1346870683 >> 172.21.15.105:0/1346870683 conn(0x559549f2a030 msgr2=0x559549f2a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 -- 172.21.15.105:0/1346870683 shutdown_connections 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 1 -- 172.21.15.105:0/1346870683 wait complete. 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.735+0000 7fb9145d4740 20 client.15246 trim_cache size 0 max 16384 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:38:12.736 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 4, 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 14628, 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 104.43212206699999, 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 118.722144304 2024-03-26T17:38:12.737 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:12.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 -- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8f80a2f20 msgr2=0x7fb8f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8f80a2f20 0x7fb8f80a53e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb90c102940 tx=0x7fb8fc005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:12.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 -- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c1034a0 msgr2=0x7fb90c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c1034a0 0x7fb90c19f110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb90400eb10 tx=0x7fb90400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 -- 172.21.15.105:0/2063750437 shutdown_connections 2024-03-26T17:38:12.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8f80a2f20 0x7fb8f80a53e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb90c1034a0 0x7fb90c19f110 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb90c102ae0 0x7fb90c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 --2- 172.21.15.105:0/2063750437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb90c1018f0 0x7fb90c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 -- 172.21.15.105:0/2063750437 >> 172.21.15.105:0/2063750437 conn(0x7fb90c0fce60 msgr2=0x7fb90c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 -- 172.21.15.105:0/2063750437 shutdown_connections 2024-03-26T17:38:12.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.736+0000 7fb911909640 1 -- 172.21.15.105:0/2063750437 wait complete. 2024-03-26T17:38:12.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 -- 172.21.15.105:0/534008569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71441019b0 msgr2=0x7f7144101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 --2- 172.21.15.105:0/534008569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71441019b0 0x7f7144101e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7138009be0 tx=0x7f713802bef0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 -- 172.21.15.105:0/534008569 shutdown_connections 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 --2- 172.21.15.105:0/534008569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7144102370 0x7f7144108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 --2- 172.21.15.105:0/534008569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71441019b0 0x7f7144101e30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 --2- 172.21.15.105:0/534008569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71441007c0 0x7f7144100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 -- 172.21.15.105:0/534008569 >> 172.21.15.105:0/534008569 conn(0x7f71440fbd50 msgr2=0x7f71440fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 -- 172.21.15.105:0/534008569 shutdown_connections 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.853+0000 7f71489d4640 1 -- 172.21.15.105:0/534008569 wait complete. 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f71489d4640 1 Processor -- start 2024-03-26T17:38:12.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f71489d4640 1 -- start start 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f71489d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71441007c0 0x7f7144104960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f71489d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71441019b0 0x7f7144102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f71489d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7144102370 0x7f71441034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f71489d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f714410c2f0 con 0x7f7144102370 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f71489d4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f714410c170 con 0x7f71441019b0 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f71489d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f714410c470 con 0x7f71441007c0 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7142d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7144102370 0x7f71441034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f714259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71441007c0 0x7f7144104960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7142d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7144102370 0x7f71441034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33834/0 (socket says 172.21.15.105:33834) 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7142d9d640 1 -- 172.21.15.105:0/597216094 learned_addr learned my addr 172.21.15.105:0/597216094 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7141d9b640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71441019b0 0x7f7144102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7142d9d640 1 -- 172.21.15.105:0/597216094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71441007c0 msgr2=0x7f7144104960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7142d9d640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71441007c0 0x7f7144104960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7142d9d640 1 -- 172.21.15.105:0/597216094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71441019b0 msgr2=0x7f7144102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7142d9d640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71441019b0 0x7f7144102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.854+0000 7f7142d9d640 1 -- 172.21.15.105:0/597216094 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7144103d70 con 0x7f7144102370 2024-03-26T17:38:12.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.855+0000 7f7142d9d640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7144102370 0x7f71441034e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f713400e970 tx=0x7f713400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.855+0000 7f71337fe640 1 -- 172.21.15.105:0/597216094 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713400ccc0 con 0x7f7144102370 2024-03-26T17:38:12.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.855+0000 7f71489d4640 1 -- 172.21.15.105:0/597216094 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71441a1c40 con 0x7f7144102370 2024-03-26T17:38:12.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.855+0000 7f71337fe640 1 -- 172.21.15.105:0/597216094 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71340054d0 con 0x7f7144102370 2024-03-26T17:38:12.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.855+0000 7f71337fe640 1 -- 172.21.15.105:0/597216094 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134010430 con 0x7f7144102370 2024-03-26T17:38:12.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.855+0000 7f71489d4640 1 -- 172.21.15.105:0/597216094 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71441a2110 con 0x7f7144102370 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.857+0000 7f71337fe640 1 -- 172.21.15.105:0/597216094 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f7134005020 con 0x7f7144102370 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.857+0000 7f71337fe640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f712c0a2f20 0x7f712c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714259c640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f712c0a2f20 0x7f712c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f71337fe640 1 -- 172.21.15.105:0/597216094 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f71340cf8c0 con 0x7f7144102370 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714259c640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f712c0a2f20 0x7f712c0a53e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7144101810 tx=0x7f713c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714b5bf740 1 Processor -- start 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714b5bf740 1 -- start start 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef83a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef83b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef842060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714b5bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5648ef79e270 con 0x5648ef83ae10 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714b5bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648ef746c80 con 0x5648ef839c40 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.858+0000 7f714b5bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5648ef79a220 con 0x5648ef83b7d0 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f7141d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef83b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f714259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef83a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f7141d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef83b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33836/0 (socket says 172.21.15.105:33836) 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f7141d9b640 1 -- 172.21.15.105:0/2929714946 learned_addr learned my addr 172.21.15.105:0/2929714946 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f714259c640 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef83a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58992/0 (socket says 172.21.15.105:58992) 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f7142d9d640 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef842060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f714259c640 1 -- 172.21.15.105:0/2929714946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 msgr2=0x5648ef842060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f714259c640 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef842060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f714259c640 1 -- 172.21.15.105:0/2929714946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 msgr2=0x5648ef83b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f714259c640 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef83b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f714259c640 1 -- 172.21.15.105:0/2929714946 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ef758fb0 con 0x5648ef839c40 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f7141d9b640 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef83b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f714259c640 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef83a060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f713c0118d0 tx=0x7f713c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffccb58ab21673b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f7130ff9640 1 -- 172.21.15.105:0/2929714946 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c026430 con 0x5648ef839c40 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f7130ff9640 1 -- 172.21.15.105:0/2929714946 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f713c01c8c0 con 0x5648ef839c40 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.859+0000 7f7130ff9640 1 -- 172.21.15.105:0/2929714946 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c01d910 con 0x5648ef839c40 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 -- 172.21.15.105:0/2929714946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 msgr2=0x5648ef83a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef83a060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f713c0118d0 tx=0x7f713c014000 comp rx=0 tx=0).stop 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 -- 172.21.15.105:0/2929714946 shutdown_connections 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef842060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef83b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 --2- 172.21.15.105:0/2929714946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef83a060 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 -- 172.21.15.105:0/2929714946 >> 172.21.15.105:0/2929714946 conn(0x5648ef835030 msgr2=0x5648ef835430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 -- 172.21.15.105:0/2929714946 shutdown_connections 2024-03-26T17:38:12.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 -- 172.21.15.105:0/2929714946 wait complete. 2024-03-26T17:38:12.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 -- start start 2024-03-26T17:38:12.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.860+0000 7f714b5bf740 1 Processor -- start 2024-03-26T17:38:12.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef844240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83ae10 0x5648ef844780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83b7d0 0x5648ef84b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714b5bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5648ef8427e0 con 0x5648ef83b7d0 2024-03-26T17:38:12.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714b5bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648ef759fc0 con 0x5648ef839c40 2024-03-26T17:38:12.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714b5bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5648ef842960 con 0x5648ef83ae10 2024-03-26T17:38:12.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714b5bf740 10 client.0 _ll_register_callbacks cb 0x5648ef754770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:12.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef844240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef844240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59004/0 (socket says 172.21.15.105:59004) 2024-03-26T17:38:12.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f7142d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83b7d0 0x5648ef84b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 -- 172.21.15.105:0/3408616652 learned_addr learned my addr 172.21.15.105:0/3408616652 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f7141d9b640 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83ae10 0x5648ef844780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 -- 172.21.15.105:0/3408616652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83ae10 msgr2=0x5648ef844780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83ae10 0x5648ef844780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 -- 172.21.15.105:0/3408616652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83b7d0 msgr2=0x5648ef84b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83b7d0 0x5648ef84b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 -- 172.21.15.105:0/3408616652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ef4b22d0 con 0x5648ef839c40 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f7142d9d640 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83b7d0 0x5648ef84b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:12.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714259c640 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef844240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f713c00ee40 tx=0x7f713c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f0606570d39825 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f7129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f7129ffb640 1 -- 172.21.15.105:0/3408616652 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c034060 con 0x5648ef839c40 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f7129ffb640 1 -- 172.21.15.105:0/3408616652 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f713c01c510 con 0x5648ef839c40 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.861+0000 7f714b5bf740 1 -- 172.21.15.105:0/3408616652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713c01c8c0 con 0x5648ef839c40 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f7129ffb640 1 -- 172.21.15.105:0/3408616652 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f713c02b040 con 0x5648ef839c40 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f714b5bf740 10 client.25124 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f714b5bf740 1 -- 172.21.15.105:0/3408616652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5648ef57c6b0 con 0x5648ef839c40 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f7129ffb640 1 -- 172.21.15.105:0/3408616652 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f713c01c8c0 con 0x5648ef839c40 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f7129ffb640 1 -- 172.21.15.105:0/3408616652 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f713c02e030 con 0x5648ef839c40 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f714b5bf740 10 client.25124 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:12.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f714b5bf740 10 client.25124 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f714b5bf740 20 client.25124 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5648ef850b60 0x5648ef851fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f714b5bf740 4 client.25124 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f714b5bf740 1 -- 172.21.15.105:0/3408616652 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648ef7ab7b0 con 0x5648ef850b60 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.862+0000 7f7141d9b640 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5648ef850b60 0x5648ef851fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.863+0000 7f7141d9b640 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5648ef850b60 0x5648ef851fd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2fd3de8d3e21301d server_cookie=b8e929b0c3be6b07 in_seq=0 out_seq=0 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.863+0000 7f7129ffb640 10 client.25124 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f7129ffb640 1 -- 172.21.15.105:0/3408616652 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5648ef7ab7b0 con 0x5648ef850b60 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f7129ffb640 10 client.25124 handle_command_reply: tid=0 2024-03-26T17:38:12.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 client.25124 shutdown 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 -- 172.21.15.105:0/3408616652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 msgr2=0x5648ef844240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef844240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f713c00ee40 tx=0x7f713c00ee70 comp rx=0 tx=0).stop 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 -- 172.21.15.105:0/3408616652 shutdown_connections 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5648ef850b60 0x5648ef851fd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83b7d0 0x5648ef84b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83ae10 0x5648ef844780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3408616652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef844240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f7129ffb640 0 client.25124 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 -- 172.21.15.105:0/3408616652 >> 172.21.15.105:0/3408616652 conn(0x5648ef835030 msgr2=0x5648ef834230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 -- 172.21.15.105:0/3408616652 shutdown_connections 2024-03-26T17:38:12.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 1 -- 172.21.15.105:0/3408616652 wait complete. 2024-03-26T17:38:12.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.864+0000 7f714b5bf740 20 client.25124 trim_cache size 0 max 16384 2024-03-26T17:38:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.881+0000 7f714b5bf740 1 Processor -- start 2024-03-26T17:38:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.881+0000 7f714b5bf740 1 -- start start 2024-03-26T17:38:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef8426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef84b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef8446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f714b5bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5648ef850310 con 0x5648ef83ae10 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f714b5bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648ef8509e0 con 0x5648ef839c40 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f714b5bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5648ef8510b0 con 0x5648ef83b7d0 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef84b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef84b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33852/0 (socket says 172.21.15.105:33852) 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 -- 172.21.15.105:0/611127033 learned_addr learned my addr 172.21.15.105:0/611127033 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f714259c640 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef8426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7142d9d640 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef8446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 -- 172.21.15.105:0/611127033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 msgr2=0x5648ef8446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef8446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 -- 172.21.15.105:0/611127033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 msgr2=0x5648ef8426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef8426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 -- 172.21.15.105:0/611127033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ef839840 con 0x5648ef83ae10 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f7141d9b640 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef84b750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7138034ec0 tx=0x7f7138034f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5087bd313d0e1c5d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f712bfff640 1 -- 172.21.15.105:0/611127033 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713804c070 con 0x5648ef83ae10 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f712bfff640 1 -- 172.21.15.105:0/611127033 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7138033790 con 0x5648ef83ae10 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.882+0000 7f712bfff640 1 -- 172.21.15.105:0/611127033 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138003ab0 con 0x5648ef83ae10 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 -- 172.21.15.105:0/611127033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 msgr2=0x5648ef84b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef84b750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7138034ec0 tx=0x7f7138034f90 comp rx=0 tx=0).stop 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 -- 172.21.15.105:0/611127033 shutdown_connections 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef8446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef83ae10 0x5648ef84b750 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 --2- 172.21.15.105:0/611127033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef839c40 0x5648ef8426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 -- 172.21.15.105:0/611127033 >> 172.21.15.105:0/611127033 conn(0x5648ef835030 msgr2=0x5648ef842d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 -- 172.21.15.105:0/611127033 shutdown_connections 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 -- 172.21.15.105:0/611127033 wait complete. 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 -- start start 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.883+0000 7f714b5bf740 1 Processor -- start 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef839c40 0x5648ef846af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef83ae10 0x5648ef847030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef84a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5648ef851e50 con 0x5648ef839c40 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648ef851780 con 0x5648ef83ae10 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5648ef852520 con 0x5648ef83b7d0 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 10 client.0 _ll_register_callbacks cb 0x7f713c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef839c40 0x5648ef846af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef839c40 0x5648ef846af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33856/0 (socket says 172.21.15.105:33856) 2024-03-26T17:38:12.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 -- 172.21.15.105:0/3480135238 learned_addr learned my addr 172.21.15.105:0/3480135238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f7141d9b640 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef83ae10 0x5648ef847030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f7142d9d640 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef84a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 -- 172.21.15.105:0/3480135238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 msgr2=0x5648ef84a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef84a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 -- 172.21.15.105:0/3480135238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef83ae10 msgr2=0x5648ef847030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef83ae10 0x5648ef847030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 -- 172.21.15.105:0/3480135238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ef84a5f0 con 0x5648ef839c40 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714259c640 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef839c40 0x5648ef846af0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f713c0092e0 tx=0x7f713c005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f87097396e02e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f7129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f7129ffb640 1 -- 172.21.15.105:0/3480135238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c03f070 con 0x5648ef839c40 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f7129ffb640 1 -- 172.21.15.105:0/3480135238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f713c005560 con 0x5648ef839c40 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 1 -- 172.21.15.105:0/3480135238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7138033790 con 0x5648ef839c40 2024-03-26T17:38:12.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f7129ffb640 1 -- 172.21.15.105:0/3480135238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f713c034040 con 0x5648ef839c40 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 10 client.15264 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.884+0000 7f714b5bf740 1 -- 172.21.15.105:0/3480135238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5648ef84b8f0 con 0x5648ef839c40 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f7129ffb640 1 -- 172.21.15.105:0/3480135238 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7138033790 con 0x5648ef839c40 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f7129ffb640 1 -- 172.21.15.105:0/3480135238 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f713c02e040 con 0x5648ef839c40 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 10 client.15264 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 10 client.15264 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 20 client.15264 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5648ef8bb0b0 0x5648ef8b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 4 client.15264 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 1 -- 172.21.15.105:0/3480135238 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "status"}) -- 0x5648ef7a5b00 con 0x5648ef8bb0b0 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f7141d9b640 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5648ef8bb0b0 0x5648ef8b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f7129ffb640 10 client.15264 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f7141d9b640 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5648ef8bb0b0 0x5648ef8b9dc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=839ecfd65044e869 server_cookie=e1c52cbb28e658c6 in_seq=0 out_seq=0 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f7129ffb640 1 -- 172.21.15.105:0/3480135238 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5648ef7a5b00 con 0x5648ef8bb0b0 2024-03-26T17:38:12.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f7129ffb640 10 client.15264 handle_command_reply: tid=0 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 1 client.15264 shutdown 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 1 -- 172.21.15.105:0/3480135238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef839c40 msgr2=0x5648ef846af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.885+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef839c40 0x5648ef846af0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f713c0092e0 tx=0x7f713c005160 comp rx=0 tx=0).stop 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 1 -- 172.21.15.105:0/3480135238 shutdown_connections 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5648ef8bb0b0 0x5648ef8b9dc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648ef83b7d0 0x5648ef84a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648ef83ae10 0x5648ef847030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f7129ffb640 0 client.15264 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 1 --2- 172.21.15.105:0/3480135238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5648ef839c40 0x5648ef846af0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 1 -- 172.21.15.105:0/3480135238 >> 172.21.15.105:0/3480135238 conn(0x5648ef835030 msgr2=0x5648ef82fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 1 -- 172.21.15.105:0/3480135238 shutdown_connections 2024-03-26T17:38:12.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 1 -- 172.21.15.105:0/3480135238 wait complete. 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f714b5bf740 20 client.15264 trim_cache size 0 max 16384 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 3, 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24466, 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 105.55777026600001, 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:38:12.891 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:38:12.892 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:38:12.892 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 129.72479004100001 2024-03-26T17:38:12.892 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 -- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f712c0a2f20 msgr2=0x7f712c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f712c0a2f20 0x7f712c0a53e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7144101810 tx=0x7f713c005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 -- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7144102370 msgr2=0x7f71441034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7144102370 0x7f71441034e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f713400e970 tx=0x7f713400ee30 comp rx=0 tx=0).stop 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 -- 172.21.15.105:0/597216094 shutdown_connections 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f712c0a2f20 0x7f712c0a53e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7144102370 0x7f71441034e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71441019b0 0x7f7144102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 --2- 172.21.15.105:0/597216094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71441007c0 0x7f7144104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 -- 172.21.15.105:0/597216094 >> 172.21.15.105:0/597216094 conn(0x7f71440fbd50 msgr2=0x7f7144068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 -- 172.21.15.105:0/597216094 shutdown_connections 2024-03-26T17:38:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:12.886+0000 7f71489d4640 1 -- 172.21.15.105:0/597216094 wait complete. 2024-03-26T17:38:12.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:12 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3443798315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:12.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:12 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3574050712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:12.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:12 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1667014169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:12.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-26T17:38:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 -- 172.21.15.105:0/933458220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80104c90 msgr2=0x7f9b80107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 --2- 172.21.15.105:0/933458220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80104c90 0x7f9b80107080 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009900 tx=0x7f9b6802bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 -- 172.21.15.105:0/933458220 shutdown_connections 2024-03-26T17:38:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 --2- 172.21.15.105:0/933458220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b801075c0 0x7f9b80109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 --2- 172.21.15.105:0/933458220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80104c90 0x7f9b80107080 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 --2- 172.21.15.105:0/933458220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b80102360 0x7f9b80104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 -- 172.21.15.105:0/933458220 >> 172.21.15.105:0/933458220 conn(0x7f9b800fbd70 msgr2=0x7f9b800fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 -- 172.21.15.105:0/933458220 shutdown_connections 2024-03-26T17:38:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.006+0000 7f9b85f37640 1 -- 172.21.15.105:0/933458220 wait complete. 2024-03-26T17:38:13.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b85f37640 1 Processor -- start 2024-03-26T17:38:13.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b85f37640 1 -- start start 2024-03-26T17:38:13.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b85f37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80102360 0x7f9b80197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b85f37640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b80104c90 0x7f9b801980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b85f37640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b801075c0 0x7f9b8019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b85f37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8010d400 con 0x7f9b80102360 2024-03-26T17:38:13.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b85f37640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8010d280 con 0x7f9b80104c90 2024-03-26T17:38:13.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b85f37640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8010d580 con 0x7f9b801075c0 2024-03-26T17:38:13.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80102360 0x7f9b80197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80102360 0x7f9b80197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33868/0 (socket says 172.21.15.105:33868) 2024-03-26T17:38:13.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b7f7fe640 1 -- 172.21.15.105:0/1218678862 learned_addr learned my addr 172.21.15.105:0/1218678862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b7ffff640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b801075c0 0x7f9b8019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.007+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b80104c90 0x7f9b801980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7f7fe640 1 -- 172.21.15.105:0/1218678862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b801075c0 msgr2=0x7f9b8019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b801075c0 0x7f9b8019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7f7fe640 1 -- 172.21.15.105:0/1218678862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b80104c90 msgr2=0x7f9b801980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b80104c90 0x7f9b801980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7f7fe640 1 -- 172.21.15.105:0/1218678862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8019f770 con 0x7f9b80102360 2024-03-26T17:38:13.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80102360 0x7f9b80197ba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000ea60 tx=0x7f9b7000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7d7fa640 1 -- 172.21.15.105:0/1218678862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7000cde0 con 0x7f9b80102360 2024-03-26T17:38:13.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b85f37640 1 -- 172.21.15.105:0/1218678862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8019fa60 con 0x7f9b80102360 2024-03-26T17:38:13.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7d7fa640 1 -- 172.21.15.105:0/1218678862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b700035f0 con 0x7f9b80102360 2024-03-26T17:38:13.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b7d7fa640 1 -- 172.21.15.105:0/1218678862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70010690 con 0x7f9b80102360 2024-03-26T17:38:13.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.008+0000 7f9b85f37640 1 -- 172.21.15.105:0/1218678862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8019ffd0 con 0x7f9b80102360 2024-03-26T17:38:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.010+0000 7f9b7d7fa640 1 -- 172.21.15.105:0/1218678862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f9b700037a0 con 0x7f9b80102360 2024-03-26T17:38:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b88bf9740 1 Processor -- start 2024-03-26T17:38:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b88bf9740 1 -- start start 2024-03-26T17:38:13.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 0x55ddf704c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704ce10 0x55ddf704d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704d7d0 0x55ddf7054060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ddf6fb0270 con 0x55ddf704d7d0 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ddf6f58c80 con 0x55ddf704ce10 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ddf6fac220 con 0x55ddf704bc40 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b76ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704d7d0 0x55ddf7054060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b76ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704d7d0 0x55ddf7054060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33884/0 (socket says 172.21.15.105:33884) 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b7d7fa640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9b6c0a2f20 0x7f9b6c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b76ffd640 1 -- 172.21.15.105:0/1932227612 learned_addr learned my addr 172.21.15.105:0/1932227612 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b7ffff640 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 0x55ddf704c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704ce10 0x55ddf704d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b7ffff640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9b6c0a2f20 0x7f9b6c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b76ffd640 1 -- 172.21.15.105:0/1932227612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 msgr2=0x55ddf704c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 0x55ddf704c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b7d7fa640 1 -- 172.21.15.105:0/1218678862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f9b700cf690 con 0x7f9b80102360 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b76ffd640 1 -- 172.21.15.105:0/1932227612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704ce10 msgr2=0x55ddf704d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704ce10 0x55ddf704d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.011+0000 7f9b76ffd640 1 -- 172.21.15.105:0/1932227612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf6f6afb0 con 0x55ddf704d7d0 2024-03-26T17:38:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704d7d0 0x55ddf7054060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9b8006a9f0 tx=0x7f9b68009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b05a6cb7415b77 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b7ffff640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9b6c0a2f20 0x7f9b6c0a53e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55ddf704cd10 tx=0x7f9b78004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b75ffb640 1 -- 172.21.15.105:0/1932227612 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68045020 con 0x55ddf704d7d0 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b75ffb640 1 -- 172.21.15.105:0/1932227612 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b68004040 con 0x55ddf704d7d0 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b75ffb640 1 -- 172.21.15.105:0/1932227612 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680055b0 con 0x55ddf704d7d0 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1932227612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704d7d0 msgr2=0x55ddf7054060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704d7d0 0x55ddf7054060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9b8006a9f0 tx=0x7f9b68009560 comp rx=0 tx=0).stop 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1932227612 shutdown_connections 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704d7d0 0x55ddf7054060 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704ce10 0x55ddf704d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1932227612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 0x55ddf704c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.012+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1932227612 >> 172.21.15.105:0/1932227612 conn(0x55ddf7047030 msgr2=0x55ddf7047430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1932227612 shutdown_connections 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1932227612 wait complete. 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 -- start start 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 Processor -- start 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704bc40 0x55ddf70560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf7056630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704d7d0 0x55ddf705d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7054780 con 0x55ddf704ce10 2024-03-26T17:38:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ddf6f6bfc0 con 0x55ddf704bc40 2024-03-26T17:38:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7054900 con 0x55ddf704d7d0 2024-03-26T17:38:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b88bf9740 10 client.0 _ll_register_callbacks cb 0x55ddf6f66770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b76ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf7056630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b76ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf7056630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33896/0 (socket says 172.21.15.105:33896) 2024-03-26T17:38:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b76ffd640 1 -- 172.21.15.105:0/1639273974 learned_addr learned my addr 172.21.15.105:0/1639273974 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704bc40 0x55ddf70560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.013+0000 7f9b7ffff640 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704d7d0 0x55ddf705d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b76ffd640 1 -- 172.21.15.105:0/1639273974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704d7d0 msgr2=0x55ddf705d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704d7d0 0x55ddf705d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b76ffd640 1 -- 172.21.15.105:0/1639273974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704bc40 msgr2=0x55ddf70560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704bc40 0x55ddf70560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b76ffd640 1 -- 172.21.15.105:0/1639273974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf6cc42d0 con 0x55ddf704ce10 2024-03-26T17:38:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf7056630 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9b6802c170 tx=0x7f9b680055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d6a778afc271db server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b5effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b5effd640 1 -- 172.21.15.105:0/1639273974 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68045070 con 0x55ddf704ce10 2024-03-26T17:38:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b5effd640 1 -- 172.21.15.105:0/1639273974 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f9b680035f0 con 0x55ddf704ce10 2024-03-26T17:38:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1639273974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b68004040 con 0x55ddf704ce10 2024-03-26T17:38:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b5effd640 1 -- 172.21.15.105:0/1639273974 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9b6805a050 con 0x55ddf704ce10 2024-03-26T17:38:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b88bf9740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1639273974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddf6d8e6b0 con 0x55ddf704ce10 2024-03-26T17:38:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b5effd640 1 -- 172.21.15.105:0/1639273974 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b68004040 con 0x55ddf704ce10 2024-03-26T17:38:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b5effd640 1 -- 172.21.15.105:0/1639273974 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9b68004a90 con 0x55ddf704ce10 2024-03-26T17:38:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b88bf9740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b88bf9740 10 client.15276 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T17:38:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.014+0000 7f9b88bf9740 20 client.15276 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.015+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ddf7062ae0 0x55ddf7063f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.015+0000 7f9b88bf9740 4 client.15276 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.015+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1639273974 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf6fbd7b0 con 0x55ddf7062ae0 2024-03-26T17:38:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.015+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ddf7062ae0 0x55ddf7063f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.015+0000 7f9b5effd640 10 client.15276 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:38:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.015+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ddf7062ae0 0x55ddf7063f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3724575b895e94f server_cookie=b8f8164ceb22a42e in_seq=0 out_seq=0 2024-03-26T17:38:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b5effd640 1 -- 172.21.15.105:0/1639273974 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ddf6fbd7b0 con 0x55ddf7062ae0 2024-03-26T17:38:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b5effd640 10 client.15276 handle_command_reply: tid=0 2024-03-26T17:38:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 client.15276 shutdown 2024-03-26T17:38:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1639273974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 msgr2=0x55ddf7056630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf7056630 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9b6802c170 tx=0x7f9b680055e0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1639273974 shutdown_connections 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704d7d0 0x55ddf705d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf7056630 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ddf7062ae0 0x55ddf7063f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/1639273974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704bc40 0x55ddf70560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1639273974 >> 172.21.15.105:0/1639273974 conn(0x55ddf7047030 msgr2=0x55ddf7046230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b5effd640 0 client.15276 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1639273974 shutdown_connections 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 1 -- 172.21.15.105:0/1639273974 wait complete. 2024-03-26T17:38:13.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.016+0000 7f9b88bf9740 20 client.15276 trim_cache size 0 max 16384 2024-03-26T17:38:13.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b88bf9740 1 Processor -- start 2024-03-26T17:38:13.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b88bf9740 1 -- start start 2024-03-26T17:38:13.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704bc40 0x55ddf7047410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704ce10 0x55ddf705d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf7055d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7062290 con 0x55ddf704bc40 2024-03-26T17:38:13.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7062960 con 0x55ddf704d7d0 2024-03-26T17:38:13.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7063030 con 0x55ddf704ce10 2024-03-26T17:38:13.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704bc40 0x55ddf7047410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704bc40 0x55ddf7047410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33902/0 (socket says 172.21.15.105:33902) 2024-03-26T17:38:13.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b7f7fe640 1 -- 172.21.15.105:0/3295522396 learned_addr learned my addr 172.21.15.105:0/3295522396 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704ce10 0x55ddf705d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.034+0000 7f9b7ffff640 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf7055d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b7f7fe640 1 -- 172.21.15.105:0/3295522396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704ce10 msgr2=0x55ddf705d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704ce10 0x55ddf705d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b7f7fe640 1 -- 172.21.15.105:0/3295522396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 msgr2=0x55ddf7055d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf7055d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b7f7fe640 1 -- 172.21.15.105:0/3295522396 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf7047fa0 con 0x55ddf704bc40 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704bc40 0x55ddf7047410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9b7009bd10 tx=0x7f9b70098020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f127dd551bdf7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b757fa640 1 -- 172.21.15.105:0/3295522396 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7009b6e0 con 0x55ddf704bc40 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b757fa640 1 -- 172.21.15.105:0/3295522396 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b70003b30 con 0x55ddf704bc40 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b757fa640 1 -- 172.21.15.105:0/3295522396 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b700cd260 con 0x55ddf704bc40 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b88bf9740 1 -- 172.21.15.105:0/3295522396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704bc40 msgr2=0x55ddf7047410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.035+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704bc40 0x55ddf7047410 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9b7009bd10 tx=0x7f9b70098020 comp rx=0 tx=0).stop 2024-03-26T17:38:13.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 -- 172.21.15.105:0/3295522396 shutdown_connections 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf7055d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704ce10 0x55ddf705d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/3295522396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704bc40 0x55ddf7047410 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 -- 172.21.15.105:0/3295522396 >> 172.21.15.105:0/3295522396 conn(0x55ddf7047030 msgr2=0x55ddf7061b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 -- 172.21.15.105:0/3295522396 shutdown_connections 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 -- 172.21.15.105:0/3295522396 wait complete. 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 -- start start 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 Processor -- start 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 0x55ddf7059070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf70595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf705c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7063dd0 con 0x55ddf704ce10 2024-03-26T17:38:13.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7063700 con 0x55ddf704d7d0 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ddf70644a0 con 0x55ddf704bc40 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b88bf9740 10 client.0 _ll_register_callbacks cb 0x7f9b70097900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b76ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf70595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b76ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf70595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33918/0 (socket says 172.21.15.105:33918) 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b7f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 0x55ddf7059070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b76ffd640 1 -- 172.21.15.105:0/637579197 learned_addr learned my addr 172.21.15.105:0/637579197 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.036+0000 7f9b7ffff640 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf705c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b76ffd640 1 -- 172.21.15.105:0/637579197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 msgr2=0x55ddf7059070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 0x55ddf7059070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b76ffd640 1 -- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 msgr2=0x55ddf705c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf705c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b76ffd640 1 -- 172.21.15.105:0/637579197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf705cbc0 con 0x55ddf704ce10 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b7ffff640 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf705c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b76ffd640 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf70595b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009900 tx=0x7f9b680045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2a10e736d0ccea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b5effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b5effd640 1 -- 172.21.15.105:0/637579197 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68045040 con 0x55ddf704ce10 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b5effd640 1 -- 172.21.15.105:0/637579197 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f9b680048c0 con 0x55ddf704ce10 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 1 -- 172.21.15.105:0/637579197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ddf6fb7b00 con 0x55ddf704ce10 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b5effd640 1 -- 172.21.15.105:0/637579197 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9b68065030 con 0x55ddf704ce10 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 10 client.15282 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 1 -- 172.21.15.105:0/637579197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ddf70c8b00 con 0x55ddf704ce10 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b5effd640 1 -- 172.21.15.105:0/637579197 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ddf6fb7b00 con 0x55ddf704ce10 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b5effd640 1 -- 172.21.15.105:0/637579197 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9b68041c30 con 0x55ddf704ce10 2024-03-26T17:38:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 10 client.15282 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 10 client.15282 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 20 client.15282 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ddf70cd720 0x55ddf70cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 4 client.15282 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.037+0000 7f9b88bf9740 1 -- 172.21.15.105:0/637579197 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "status"}) -- 0x55ddf6f35a00 con 0x55ddf70cd720 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ddf70cd720 0x55ddf70cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b5effd640 10 client.15282 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b7f7fe640 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ddf70cd720 0x55ddf70cc430 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bf91ae38ec69bcb server_cookie=d8f3d3d6fbe5b02d in_seq=0 out_seq=0 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b5effd640 1 -- 172.21.15.105:0/637579197 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ddf6f35a00 con 0x55ddf70cd720 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b5effd640 10 client.15282 handle_command_reply: tid=0 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 client.15282 shutdown 2024-03-26T17:38:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 -- 172.21.15.105:0/637579197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 msgr2=0x55ddf70595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf70595b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009900 tx=0x7f9b680045b0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 -- 172.21.15.105:0/637579197 shutdown_connections 2024-03-26T17:38:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ddf70cd720 0x55ddf70cc430 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ddf704d7d0 0x55ddf705c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ddf704ce10 0x55ddf70595b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 --2- 172.21.15.105:0/637579197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddf704bc40 0x55ddf7059070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b5effd640 0 client.15282 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:38:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.038+0000 7f9b88bf9740 1 -- 172.21.15.105:0/637579197 >> 172.21.15.105:0/637579197 conn(0x55ddf7047030 msgr2=0x55ddf7047410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b88bf9740 1 -- 172.21.15.105:0/637579197 shutdown_connections 2024-03-26T17:38:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b88bf9740 1 -- 172.21.15.105:0/637579197 wait complete. 2024-03-26T17:38:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b88bf9740 20 client.15282 trim_cache size 0 max 16384 2024-03-26T17:38:13.046 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:13.046 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 2, 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24482, 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 106.715683967, 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 127.174534643 2024-03-26T17:38:13.047 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 -- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9b6c0a2f20 msgr2=0x7f9b6c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9b6c0a2f20 0x7f9b6c0a53e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55ddf704cd10 tx=0x7f9b78004ab0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 -- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80102360 msgr2=0x7f9b80197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80102360 0x7f9b80197ba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000ea60 tx=0x7f9b7000ef20 comp rx=0 tx=0).stop 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 -- 172.21.15.105:0/1218678862 shutdown_connections 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9b6c0a2f20 0x7f9b6c0a53e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b801075c0 0x7f9b8019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9b80104c90 0x7f9b801980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 --2- 172.21.15.105:0/1218678862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9b80102360 0x7f9b80197ba0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 -- 172.21.15.105:0/1218678862 >> 172.21.15.105:0/1218678862 conn(0x7f9b800fbd70 msgr2=0x7f9b80102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 -- 172.21.15.105:0/1218678862 shutdown_connections 2024-03-26T17:38:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.039+0000 7f9b85f37640 1 -- 172.21.15.105:0/1218678862 wait complete. 2024-03-26T17:38:13.050 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T17:38:13.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:12 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3443798315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:13.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:12 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3574050712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:13.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:12 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1667014169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 -- 172.21.15.105:0/3372327774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58141007c0 msgr2=0x7f5814100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 --2- 172.21.15.105:0/3372327774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58141007c0 0x7f5814100be0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5804009900 tx=0x7f580402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 -- 172.21.15.105:0/3372327774 shutdown_connections 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 --2- 172.21.15.105:0/3372327774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5814102370 0x7f5814108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 --2- 172.21.15.105:0/3372327774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58141019b0 0x7f5814101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 --2- 172.21.15.105:0/3372327774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58141007c0 0x7f5814100be0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 -- 172.21.15.105:0/3372327774 >> 172.21.15.105:0/3372327774 conn(0x7f58140fbd50 msgr2=0x7f58140fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 -- 172.21.15.105:0/3372327774 shutdown_connections 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.164+0000 7f581a059640 1 -- 172.21.15.105:0/3372327774 wait complete. 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f581a059640 1 Processor -- start 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f581a059640 1 -- start start 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f581a059640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58141007c0 0x7f5814199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f581a059640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58141019b0 0x7f581419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f581a059640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5814102370 0x7f58141a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f581a059640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f581410c2e0 con 0x7f5814102370 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f581a059640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f581410c160 con 0x7f58141007c0 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f581a059640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f581410c460 con 0x7f58141019b0 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5813fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5814102370 0x7f58141a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f58137fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58141007c0 0x7f5814199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5813fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5814102370 0x7f58141a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33938/0 (socket says 172.21.15.105:33938) 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5813fff640 1 -- 172.21.15.105:0/4147417866 learned_addr learned my addr 172.21.15.105:0/4147417866 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5812ffd640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58141019b0 0x7f581419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5813fff640 1 -- 172.21.15.105:0/4147417866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58141019b0 msgr2=0x7f581419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5813fff640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58141019b0 0x7f581419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5813fff640 1 -- 172.21.15.105:0/4147417866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58141007c0 msgr2=0x7f5814199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5813fff640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58141007c0 0x7f5814199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.165+0000 7f5813fff640 1 -- 172.21.15.105:0/4147417866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58141a1990 con 0x7f5814102370 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.166+0000 7f5813fff640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5814102370 0x7f58141a1320 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f580c00eb10 tx=0x7f580c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.166+0000 7f5810ff9640 1 -- 172.21.15.105:0/4147417866 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c004b20 con 0x7f5814102370 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.166+0000 7f5810ff9640 1 -- 172.21.15.105:0/4147417866 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f580c004cc0 con 0x7f5814102370 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.166+0000 7f581a059640 1 -- 172.21.15.105:0/4147417866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58141a1c80 con 0x7f5814102370 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.166+0000 7f5810ff9640 1 -- 172.21.15.105:0/4147417866 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c010690 con 0x7f5814102370 2024-03-26T17:38:13.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.166+0000 7f581a059640 1 -- 172.21.15.105:0/4147417866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58141a2180 con 0x7f5814102370 2024-03-26T17:38:13.172 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:12 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3443798315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:13.172 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:12 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3574050712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:13.172 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:12 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1667014169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f5810ff9640 1 -- 172.21.15.105:0/4147417866 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f580c005020 con 0x7f5814102370 2024-03-26T17:38:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f581cc43740 1 Processor -- start 2024-03-26T17:38:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f581cc43740 1 -- start start 2024-03-26T17:38:13.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f581cc43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998adc40 0x55e0998ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f581cc43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f581cc43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998af7d0 0x55e0998b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f581cc43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e099812270 con 0x55e0998adc40 2024-03-26T17:38:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f581cc43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e0997bac80 con 0x55e0998af7d0 2024-03-26T17:38:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f581cc43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e09980e220 con 0x55e0998aee10 2024-03-26T17:38:13.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998adc40 0x55e0998ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.168+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998adc40 0x55e0998ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33940/0 (socket says 172.21.15.105:33940) 2024-03-26T17:38:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5812ffd640 1 -- 172.21.15.105:0/4066836132 learned_addr learned my addr 172.21.15.105:0/4066836132 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5810ff9640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f58137fe640 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5812ffd640 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998af7d0 0x55e0998b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5812ffd640 1 -- 172.21.15.105:0/4066836132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 msgr2=0x55e0998af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5812ffd640 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5812ffd640 1 -- 172.21.15.105:0/4066836132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998af7d0 msgr2=0x55e0998b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5812ffd640 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998af7d0 0x55e0998b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5812ffd640 1 -- 172.21.15.105:0/4066836132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0997ccfb0 con 0x55e0998adc40 2024-03-26T17:38:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5813fff640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5810ff9640 1 -- 172.21.15.105:0/4147417866 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f580c0cf920 con 0x7f5814102370 2024-03-26T17:38:13.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5812ffd640 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998adc40 0x55e0998ae060 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5808013620 tx=0x7f5808009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66386fcb6d87246c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f57f9ffb640 1 -- 172.21.15.105:0/4066836132 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808004050 con 0x55e0998adc40 2024-03-26T17:38:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f57f9ffb640 1 -- 172.21.15.105:0/4066836132 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58080041f0 con 0x55e0998adc40 2024-03-26T17:38:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f57f9ffb640 1 -- 172.21.15.105:0/4066836132 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580800f7a0 con 0x55e0998adc40 2024-03-26T17:38:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.169+0000 7f5813fff640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f580c09bd70 tx=0x7f580c098440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 -- 172.21.15.105:0/4066836132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998adc40 msgr2=0x55e0998ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998adc40 0x55e0998ae060 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5808013620 tx=0x7f5808009830 comp rx=0 tx=0).stop 2024-03-26T17:38:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 -- 172.21.15.105:0/4066836132 shutdown_connections 2024-03-26T17:38:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998af7d0 0x55e0998b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 --2- 172.21.15.105:0/4066836132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998adc40 0x55e0998ae060 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 -- 172.21.15.105:0/4066836132 >> 172.21.15.105:0/4066836132 conn(0x55e0998a9030 msgr2=0x55e0998a9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 -- 172.21.15.105:0/4066836132 shutdown_connections 2024-03-26T17:38:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 -- 172.21.15.105:0/4066836132 wait complete. 2024-03-26T17:38:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 -- start start 2024-03-26T17:38:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.170+0000 7f581cc43740 1 Processor -- start 2024-03-26T17:38:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f581cc43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 0x55e0998b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f581cc43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f581cc43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f581cc43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e0998b67e0 con 0x55e0998af7d0 2024-03-26T17:38:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f581cc43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e0997cdfc0 con 0x55e0998aee10 2024-03-26T17:38:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f581cc43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e0998b6960 con 0x55e0998adc40 2024-03-26T17:38:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f581cc43740 10 client.0 _ll_register_callbacks cb 0x55e0997c8770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998bf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33942/0 (socket says 172.21.15.105:33942) 2024-03-26T17:38:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 -- 172.21.15.105:0/1699720559 learned_addr learned my addr 172.21.15.105:0/1699720559 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f58137fe640 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 0x55e0998b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 -- 172.21.15.105:0/1699720559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 msgr2=0x55e0998b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 0x55e0998b8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 -- 172.21.15.105:0/1699720559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 msgr2=0x55e0998b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 -- 172.21.15.105:0/1699720559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0995262d0 con 0x55e0998af7d0 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f58137fe640 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f5812ffd640 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998bf700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5808009710 tx=0x7f5808003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688296bf08e503a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f57f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f57f2ffd640 1 -- 172.21.15.105:0/1699720559 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580802e060 con 0x55e0998af7d0 2024-03-26T17:38:13.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f57f2ffd640 1 -- 172.21.15.105:0/1699720559 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f580800f580 con 0x55e0998af7d0 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.171+0000 7f581cc43740 1 -- 172.21.15.105:0/1699720559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58080041f0 con 0x55e0998af7d0 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f57f2ffd640 1 -- 172.21.15.105:0/1699720559 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f580800f780 con 0x55e0998af7d0 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f581cc43740 10 client.15294 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f581cc43740 1 -- 172.21.15.105:0/1699720559 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e0995f06b0 con 0x55e0998af7d0 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f57f2ffd640 1 -- 172.21.15.105:0/1699720559 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58080041f0 con 0x55e0998af7d0 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f57f2ffd640 1 -- 172.21.15.105:0/1699720559 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f580801e070 con 0x55e0998af7d0 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f581cc43740 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f581cc43740 10 client.15294 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f581cc43740 20 client.15294 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:13.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f581cc43740 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e0998c4c90 0x55e0998c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f581cc43740 4 client.15294 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f581cc43740 1 -- 172.21.15.105:0/1699720559 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e09981f7b0 con 0x55e0998c4c90 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.172+0000 7f58137fe640 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e0998c4c90 0x55e0998c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.173+0000 7f57f2ffd640 10 client.15294 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.173+0000 7f58137fe640 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e0998c4c90 0x55e0998c6100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a298a3da9bfae0a2 server_cookie=501eb025dd03b5c4 in_seq=0 out_seq=0 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.173+0000 7f57f2ffd640 1 -- 172.21.15.105:0/1699720559 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e09981f7b0 con 0x55e0998c4c90 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.173+0000 7f57f2ffd640 10 client.15294 handle_command_reply: tid=0 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.173+0000 7f581cc43740 1 client.15294 shutdown 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.173+0000 7f581cc43740 1 -- 172.21.15.105:0/1699720559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 msgr2=0x55e0998bf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.173+0000 7f581cc43740 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998bf700 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5808009710 tx=0x7f5808003b50 comp rx=0 tx=0).stop 2024-03-26T17:38:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 1 -- 172.21.15.105:0/1699720559 shutdown_connections 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e0998c4c90 0x55e0998c6100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998bf700 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f57f2ffd640 0 client.15294 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 1 --2- 172.21.15.105:0/1699720559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 0x55e0998b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 1 -- 172.21.15.105:0/1699720559 >> 172.21.15.105:0/1699720559 conn(0x55e0998a9030 msgr2=0x55e0998a8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 1 -- 172.21.15.105:0/1699720559 shutdown_connections 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 1 -- 172.21.15.105:0/1699720559 wait complete. 2024-03-26T17:38:13.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.174+0000 7f581cc43740 20 client.15294 trim_cache size 0 max 16384 2024-03-26T17:38:13.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f581cc43740 1 Processor -- start 2024-03-26T17:38:13.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f581cc43740 1 -- start start 2024-03-26T17:38:13.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f581cc43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 0x55e0998b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f581cc43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998bf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f581cc43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f581cc43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e0998c4440 con 0x55e0998af7d0 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f581cc43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e0998c4b10 con 0x55e0998aee10 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f581cc43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e0998c51e0 con 0x55e0998adc40 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33956/0 (socket says 172.21.15.105:33956) 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f5812ffd640 1 -- 172.21.15.105:0/1866403092 learned_addr learned my addr 172.21.15.105:0/1866403092 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f5812ffd640 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 0x55e0998b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.191+0000 7f58137fe640 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998bf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f5812ffd640 1 -- 172.21.15.105:0/1866403092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 msgr2=0x55e0998b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f5812ffd640 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 0x55e0998b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f5812ffd640 1 -- 172.21.15.105:0/1866403092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 msgr2=0x55e0998bf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f5812ffd640 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998bf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f5812ffd640 1 -- 172.21.15.105:0/1866403092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0998ae890 con 0x55e0998af7d0 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f5812ffd640 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998b72a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55e0998aede0 tx=0x7f5808005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f585c6e86d7846b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f57f97fa640 1 -- 172.21.15.105:0/1866403092 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808038030 con 0x55e0998af7d0 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f57f97fa640 1 -- 172.21.15.105:0/1866403092 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5808039040 con 0x55e0998af7d0 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f57f97fa640 1 -- 172.21.15.105:0/1866403092 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580802e040 con 0x55e0998af7d0 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f581cc43740 1 -- 172.21.15.105:0/1866403092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 msgr2=0x55e0998b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.192+0000 7f581cc43740 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998b72a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55e0998aede0 tx=0x7f5808005bf0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 -- 172.21.15.105:0/1866403092 shutdown_connections 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998b72a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998aee10 0x55e0998bf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 --2- 172.21.15.105:0/1866403092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998adc40 0x55e0998b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 -- 172.21.15.105:0/1866403092 >> 172.21.15.105:0/1866403092 conn(0x55e0998a9030 msgr2=0x55e0998a9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 -- 172.21.15.105:0/1866403092 shutdown_connections 2024-03-26T17:38:13.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 -- 172.21.15.105:0/1866403092 wait complete. 2024-03-26T17:38:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 -- start start 2024-03-26T17:38:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 Processor -- start 2024-03-26T17:38:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998adc40 0x55e0998bae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e0998c5f80 con 0x55e0998af7d0 2024-03-26T17:38:13.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e0998c58b0 con 0x55e0998adc40 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e0998c6650 con 0x55e0998aee10 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f581cc43740 10 client.0 _ll_register_callbacks cb 0x7f5808005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f5812ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998be380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33964/0 (socket says 172.21.15.105:33964) 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f5812ffd640 1 -- 172.21.15.105:0/2930902228 learned_addr learned my addr 172.21.15.105:0/2930902228 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f58137fe640 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.193+0000 7f5812ffd640 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998adc40 0x55e0998bae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f5812ffd640 1 -- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 msgr2=0x55e0998bb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f5812ffd640 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998bb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f5812ffd640 1 -- 172.21.15.105:0/2930902228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998adc40 msgr2=0x55e0998bae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f5812ffd640 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998adc40 0x55e0998bae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f5812ffd640 1 -- 172.21.15.105:0/2930902228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0998be970 con 0x55e0998af7d0 2024-03-26T17:38:13.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f58137fe640 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998bb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f5812ffd640 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998be380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f58080059f0 tx=0x7f5808009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa27fb20c5144d46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f57f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f57f2ffd640 1 -- 172.21.15.105:0/2930902228 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580802e070 con 0x55e0998af7d0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f57f2ffd640 1 -- 172.21.15.105:0/2930902228 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f58080043b0 con 0x55e0998af7d0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f581cc43740 1 -- 172.21.15.105:0/2930902228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5808039040 con 0x55e0998af7d0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f57f2ffd640 1 -- 172.21.15.105:0/2930902228 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5808038040 con 0x55e0998af7d0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f581cc43740 10 client.15300 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f581cc43740 1 -- 172.21.15.105:0/2930902228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e09992aac0 con 0x55e0998af7d0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f57f2ffd640 1 -- 172.21.15.105:0/2930902228 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5808039040 con 0x55e0998af7d0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.194+0000 7f57f2ffd640 1 -- 172.21.15.105:0/2930902228 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f580801e070 con 0x55e0998af7d0 2024-03-26T17:38:13.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 10 client.15300 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 10 client.15300 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 20 client.15300 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e09992f4a0 0x55e09992bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 4 client.15300 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 1 -- 172.21.15.105:0/2930902228 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "status"}) -- 0x55e099819b00 con 0x55e09992f4a0 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f58137fe640 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e09992f4a0 0x55e09992bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f57f2ffd640 10 client.15300 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f58137fe640 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e09992f4a0 0x55e09992bfb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15e78ae95103fc1e server_cookie=f1802e0fe6a34a7b in_seq=0 out_seq=0 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f57f2ffd640 1 -- 172.21.15.105:0/2930902228 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e099819b00 con 0x55e09992f4a0 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f57f2ffd640 10 client.15300 handle_command_reply: tid=0 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 1 client.15300 shutdown 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 1 -- 172.21.15.105:0/2930902228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 msgr2=0x55e0998be380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998be380 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f58080059f0 tx=0x7f5808009f90 comp rx=0 tx=0).stop 2024-03-26T17:38:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 1 -- 172.21.15.105:0/2930902228 shutdown_connections 2024-03-26T17:38:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e09992f4a0 0x55e09992bfb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.195+0000 7f581cc43740 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e0998af7d0 0x55e0998be380 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581cc43740 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e0998aee10 0x55e0998bb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f57f2ffd640 0 client.15300 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:38:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581cc43740 1 --2- 172.21.15.105:0/2930902228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e0998adc40 0x55e0998bae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581cc43740 1 -- 172.21.15.105:0/2930902228 >> 172.21.15.105:0/2930902228 conn(0x55e0998a9030 msgr2=0x55e0998ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581cc43740 1 -- 172.21.15.105:0/2930902228 shutdown_connections 2024-03-26T17:38:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581cc43740 1 -- 172.21.15.105:0/2930902228 wait complete. 2024-03-26T17:38:13.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581cc43740 20 client.15300 trim_cache size 0 max 16384 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 1, 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24487, 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 107.880311116, 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:38:13.204 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:38:13.205 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 124.63839666600001 2024-03-26T17:38:13.205 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 -- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57fc0a2f20 msgr2=0x7f57fc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f580c09bd70 tx=0x7f580c098440 comp rx=0 tx=0).stop 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 -- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5814102370 msgr2=0x7f58141a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5814102370 0x7f58141a1320 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f580c00eb10 tx=0x7f580c00c2d0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 -- 172.21.15.105:0/4147417866 shutdown_connections 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57fc0a2f20 0x7f57fc0a53e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5814102370 0x7f58141a1320 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58141019b0 0x7f581419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 --2- 172.21.15.105:0/4147417866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58141007c0 0x7f5814199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 -- 172.21.15.105:0/4147417866 >> 172.21.15.105:0/4147417866 conn(0x7f58140fbd50 msgr2=0x7f5814068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 -- 172.21.15.105:0/4147417866 shutdown_connections 2024-03-26T17:38:13.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.196+0000 7f581a059640 1 -- 172.21.15.105:0/4147417866 wait complete. 2024-03-26T17:38:13.208 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T17:38:13.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2620454558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 msgr2=0x7f0cf4109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2620454558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 0x7f0cf4109d10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009ca0 tx=0x7f0cec02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2620454558 shutdown_connections 2024-03-26T17:38:13.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2620454558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 0x7f0cf4109d10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2620454558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf4102ac0 0x7f0cf4102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2620454558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf41018d0 0x7f0cf4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2620454558 >> 172.21.15.105:0/2620454558 conn(0x7f0cf40fce40 msgr2=0x7f0cf40ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2620454558 shutdown_connections 2024-03-26T17:38:13.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.323+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2620454558 wait complete. 2024-03-26T17:38:13.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfc4d1640 1 Processor -- start 2024-03-26T17:38:13.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfc4d1640 1 -- start start 2024-03-26T17:38:13.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfc4d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf41018d0 0x7f0cf4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfc4d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf4102ac0 0x7f0cf4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfc4d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 0x7f0cf419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfc4d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410d3e0 con 0x7f0cf4103480 2024-03-26T17:38:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfc4d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410d260 con 0x7f0cf41018d0 2024-03-26T17:38:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfc4d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410d560 con 0x7f0cf4102ac0 2024-03-26T17:38:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfaa6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 0x7f0cf419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfaa6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 0x7f0cf419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33972/0 (socket says 172.21.15.105:33972) 2024-03-26T17:38:13.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfa26d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf41018d0 0x7f0cf4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfaa6e640 1 -- 172.21.15.105:0/2852325196 learned_addr learned my addr 172.21.15.105:0/2852325196 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf4102ac0 0x7f0cf4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfaa6e640 1 -- 172.21.15.105:0/2852325196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf4102ac0 msgr2=0x7f0cf4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfaa6e640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf4102ac0 0x7f0cf4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfaa6e640 1 -- 172.21.15.105:0/2852325196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf41018d0 msgr2=0x7f0cf4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfaa6e640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf41018d0 0x7f0cf4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfaa6e640 1 -- 172.21.15.105:0/2852325196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf419f6d0 con 0x7f0cf4103480 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.324+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf41018d0 0x7f0cf4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.325+0000 7f0cfaa6e640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 0x7f0cf419f060 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0cec02c700 tx=0x7f0cec0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.325+0000 7f0ceb7fe640 1 -- 172.21.15.105:0/2852325196 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec041020 con 0x7f0cf4103480 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.325+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2852325196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf419f960 con 0x7f0cf4103480 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.325+0000 7f0ceb7fe640 1 -- 172.21.15.105:0/2852325196 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cec003e20 con 0x7f0cf4103480 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.325+0000 7f0ceb7fe640 1 -- 172.21.15.105:0/2852325196 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec039040 con 0x7f0cf4103480 2024-03-26T17:38:13.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.325+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2852325196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf419fe50 con 0x7f0cf4103480 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.327+0000 7f0ceb7fe640 1 -- 172.21.15.105:0/2852325196 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f0cec044020 con 0x7f0cf4103480 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.327+0000 7f0cff197740 1 Processor -- start 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cff197740 1 -- start start 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cff197740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cff197740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cff197740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 0x55730c3df470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cff197740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55730c337270 con 0x55730c3d7050 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cff197740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55730c2dfc80 con 0x55730c3d8220 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cff197740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55730c333220 con 0x55730c3d8be0 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3d7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33974/0 (socket says 172.21.15.105:33974) 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/3279922568 learned_addr learned my addr 172.21.15.105:0/3279922568 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 0x55730c3df470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0ceb7fe640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/3279922568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 msgr2=0x55730c3df470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 0x55730c3df470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/3279922568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 msgr2=0x55730c3d86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3d86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/3279922568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730c2f1fb0 con 0x55730c3d7050 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3d86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.328+0000 7f0cfaa6e640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.329+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3d7470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40117e0 tx=0x7f0ce40097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2939be3a779e98 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.329+0000 7f0ceb7fe640 1 -- 172.21.15.105:0/2852325196 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f0cec0f15a0 con 0x7f0cf4103480 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.329+0000 7f0ce8bfe640 1 -- 172.21.15.105:0/3279922568 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004030 con 0x55730c3d7050 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.329+0000 7f0ce8bfe640 1 -- 172.21.15.105:0/3279922568 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce4005020 con 0x55730c3d7050 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.329+0000 7f0ce8bfe640 1 -- 172.21.15.105:0/3279922568 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004bc0 con 0x55730c3d7050 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- 172.21.15.105:0/3279922568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 msgr2=0x55730c3d7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cfaa6e640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0ef430 tx=0x7f0cec0e89b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3d7470 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40117e0 tx=0x7f0ce40097a0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- 172.21.15.105:0/3279922568 shutdown_connections 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 0x55730c3df470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3d86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 --2- 172.21.15.105:0/3279922568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3d7470 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- 172.21.15.105:0/3279922568 >> 172.21.15.105:0/3279922568 conn(0x55730c3ce030 msgr2=0x55730c3ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- 172.21.15.105:0/3279922568 shutdown_connections 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- 172.21.15.105:0/3279922568 wait complete. 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- start start 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 Processor -- start 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 0x55730c3e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55730c3dfb90 con 0x55730c3d7050 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55730c2f2fc0 con 0x55730c3d8220 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55730c3dfd10 con 0x55730c3d8be0 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.330+0000 7f0cff197740 10 client.0 _ll_register_callbacks cb 0x55730c2ed770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3e1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33976/0 (socket says 172.21.15.105:33976) 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/1028995838 learned_addr learned my addr 172.21.15.105:0/1028995838 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 0x55730c3e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/1028995838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 msgr2=0x55730c3e8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 0x55730c3e8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/1028995838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 msgr2=0x55730c3e1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3e1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/1028995838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55730c04b2d0 con 0x55730c3d7050 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3e1590 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40096a0 tx=0x7f0ce4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1243ccd17968bd72 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cddffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cddffb640 1 -- 172.21.15.105:0/1028995838 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004b80 con 0x55730c3d7050 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cddffb640 1 -- 172.21.15.105:0/1028995838 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f0ce4019040 con 0x55730c3d7050 2024-03-26T17:38:13.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cff197740 1 -- 172.21.15.105:0/1028995838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce4005020 con 0x55730c3d7050 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cddffb640 1 -- 172.21.15.105:0/1028995838 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ce4035050 con 0x55730c3d7050 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cff197740 10 client.15312 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cff197740 1 -- 172.21.15.105:0/1028995838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55730c1156b0 con 0x55730c3d7050 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.331+0000 7f0cddffb640 1 -- 172.21.15.105:0/1028995838 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce4005020 con 0x55730c3d7050 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cddffb640 1 -- 172.21.15.105:0/1028995838 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0ce400fbb0 con 0x55730c3d7050 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cff197740 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cff197740 10 client.15312 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cff197740 20 client.15312 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cff197740 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55730c3ee0b0 0x55730c3ef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cff197740 4 client.15312 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:13.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cff197740 1 -- 172.21.15.105:0/1028995838 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55730c3447b0 con 0x55730c3ee0b0 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55730c3ee0b0 0x55730c3ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cddffb640 10 client.15312 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.332+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55730c3ee0b0 0x55730c3ef520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ffb4af936271b6 server_cookie=a6a55642c6f1aed9 in_seq=0 out_seq=0 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cddffb640 1 -- 172.21.15.105:0/1028995838 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55730c3447b0 con 0x55730c3ee0b0 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cddffb640 10 client.15312 handle_command_reply: tid=0 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 client.15312 shutdown 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 -- 172.21.15.105:0/1028995838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 msgr2=0x55730c3e1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3e1590 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40096a0 tx=0x7f0ce4005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 -- 172.21.15.105:0/1028995838 shutdown_connections 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55730c3ee0b0 0x55730c3ef520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8be0 0x55730c3e8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d8220 0x55730c3e1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cddffb640 0 client.15312 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:13.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 --2- 172.21.15.105:0/1028995838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d7050 0x55730c3e1590 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 -- 172.21.15.105:0/1028995838 >> 172.21.15.105:0/1028995838 conn(0x55730c3ce030 msgr2=0x55730c3cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 -- 172.21.15.105:0/1028995838 shutdown_connections 2024-03-26T17:38:13.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 1 -- 172.21.15.105:0/1028995838 wait complete. 2024-03-26T17:38:13.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.333+0000 7f0cff197740 20 client.15312 trim_cache size 0 max 16384 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cff197740 1 Processor -- start 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cff197740 1 -- start start 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cff197740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cff197740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cff197740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cff197740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55730c3ed860 con 0x55730c3d8be0 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cff197740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55730c3edf30 con 0x55730c3d7050 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cff197740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55730c3ee600 con 0x55730c3d8220 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33980/0 (socket says 172.21.15.105:33980) 2024-03-26T17:38:13.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/3700708316 learned_addr learned my addr 172.21.15.105:0/3700708316 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.351+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/3700708316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 msgr2=0x55730c3e8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/3700708316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 msgr2=0x55730c3e11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/3700708316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730c3e0650 con 0x55730c3d8be0 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e16e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40039d0 tx=0x7f0ce40096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e176a04ee06fe07 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cdffff640 1 -- 172.21.15.105:0/3700708316 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004750 con 0x55730c3d8be0 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cdffff640 1 -- 172.21.15.105:0/3700708316 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce4004b10 con 0x55730c3d8be0 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cdffff640 1 -- 172.21.15.105:0/3700708316 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400f9f0 con 0x55730c3d8be0 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cff197740 1 -- 172.21.15.105:0/3700708316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 msgr2=0x55730c3e16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cff197740 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e16e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40039d0 tx=0x7f0ce40096d0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cff197740 1 -- 172.21.15.105:0/3700708316 shutdown_connections 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cff197740 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e16e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cff197740 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cff197740 1 --2- 172.21.15.105:0/3700708316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.352+0000 7f0cff197740 1 -- 172.21.15.105:0/3700708316 >> 172.21.15.105:0/3700708316 conn(0x55730c3ce030 msgr2=0x55730c3dbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 -- 172.21.15.105:0/3700708316 shutdown_connections 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 -- 172.21.15.105:0/3700708316 wait complete. 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 -- start start 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 Processor -- start 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55730c3ef3a0 con 0x55730c3d8be0 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55730c3eecd0 con 0x55730c3d7050 2024-03-26T17:38:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55730c3efa70 con 0x55730c3d8220 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cff197740 10 client.0 _ll_register_callbacks cb 0x7f0cf0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cfa26d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59176/0 (socket says 172.21.15.105:59176) 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/320748794 learned_addr learned my addr 172.21.15.105:0/320748794 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/320748794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 msgr2=0x55730c3e7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/320748794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 msgr2=0x55730c3e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cf9a6c640 1 -- 172.21.15.105:0/320748794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55730c3df7e0 con 0x55730c3d8be0 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.353+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cf9a6c640 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e76a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40056c0 tx=0x7f0ce4004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5deab1a2410755 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cddffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cddffb640 1 -- 172.21.15.105:0/320748794 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400f790 con 0x55730c3d8be0 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 1 -- 172.21.15.105:0/320748794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce4004b10 con 0x55730c3d8be0 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cddffb640 1 -- 172.21.15.105:0/320748794 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f0ce400f930 con 0x55730c3d8be0 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cddffb640 1 -- 172.21.15.105:0/320748794 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ce400fb30 con 0x55730c3d8be0 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 10 client.15318 fetch_fsmap learned FSMap version 26 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 1 -- 172.21.15.105:0/320748794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55730c453e20 con 0x55730c3d8be0 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cddffb640 1 -- 172.21.15.105:0/320748794 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce4004b10 con 0x55730c3d8be0 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cddffb640 1 -- 172.21.15.105:0/320748794 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0ce4045060 con 0x55730c3d8be0 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 10 client.15318 fetch_fsmap finished waiting for FSMap version 26 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 10 client.15318 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 20 client.15318 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55730c458510 0x55730c457220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 4 client.15318 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T17:38:13.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.354+0000 7f0cff197740 1 -- 172.21.15.105:0/320748794 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "status"}) -- 0x55730c33eb00 con 0x55730c458510 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55730c458510 0x55730c457220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cddffb640 10 client.15318 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cfa26d640 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55730c458510 0x55730c457220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302c77cc2e28b9ae server_cookie=a6e945529de9f9bf in_seq=0 out_seq=0 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cddffb640 1 -- 172.21.15.105:0/320748794 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55730c33eb00 con 0x55730c458510 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cddffb640 10 client.15318 handle_command_reply: tid=0 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 client.15318 shutdown 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 -- 172.21.15.105:0/320748794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 msgr2=0x55730c3e76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e76a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40056c0 tx=0x7f0ce4004a30 comp rx=0 tx=0).stop 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 -- 172.21.15.105:0/320748794 shutdown_connections 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55730c458510 0x55730c457220 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55730c3d8be0 0x55730c3e76a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55730c3d8220 0x55730c3e7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 --2- 172.21.15.105:0/320748794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55730c3d7050 0x55730c3e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cddffb640 0 client.15318 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 -- 172.21.15.105:0/320748794 >> 172.21.15.105:0/320748794 conn(0x55730c3ce030 msgr2=0x55730c3d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 -- 172.21.15.105:0/320748794 shutdown_connections 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 1 -- 172.21.15.105:0/320748794 wait complete. 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.355+0000 7f0cff197740 20 client.15318 trim_cache size 0 max 16384 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15cee578-eb96-11ee-95d0-87774f69a715", 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 24521, 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-03-26T17:38:13.360 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-03-26T17:38:13.361 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-03-26T17:38:13.361 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 110.62308985200001, 2024-03-26T17:38:13.361 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 26, 2024-03-26T17:38:13.361 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 87, 2024-03-26T17:38:13.361 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T17:38:13.361 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 116.601143828 2024-03-26T17:38:13.361 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 msgr2=0x7f0ce00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0ef430 tx=0x7f0cec0e89b0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 msgr2=0x7f0cf419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 0x7f0cf419f060 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0cec02c700 tx=0x7f0cec0328a0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2852325196 shutdown_connections 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf4103480 0x7f0cf419f060 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf4102ac0 0x7f0cf4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 --2- 172.21.15.105:0/2852325196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf41018d0 0x7f0cf4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2852325196 >> 172.21.15.105:0/2852325196 conn(0x7f0cf40fce40 msgr2=0x7f0cf4105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2852325196 shutdown_connections 2024-03-26T17:38:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:13.356+0000 7f0cfc4d1640 1 -- 172.21.15.105:0/2852325196 wait complete. 2024-03-26T17:38:13.364 INFO:teuthology.run_tasks:Running task sequential... 2024-03-26T17:38:13.373 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-26T17:38:13.373 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-26T17:38:13.374 INFO:teuthology.task.print:Enabling mgr modules 2024-03-26T17:38:13.374 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-26T17:38:13.582 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-26T17:38:13.582 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-26T17:38:13.582 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-26T17:38:13.582 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T17:38:13.583 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:13.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:13.583 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-26T17:38:13.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:13.598 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-26T17:38:13.668 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-03-26T17:38:13.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T17:38:13.670 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-26T17:38:13.670 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T17:38:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T17:38:14.096 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:13 smithi105 ceph-mon[32280]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-26T17:38:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:13 smithi144 ceph-mon[36095]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-26T17:38:14.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:13 smithi135 ceph-mon[35636]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-26T17:38:15.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 -- 172.21.15.105:0/165453801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374101540 msgr2=0x7f1374101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:15.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 --2- 172.21.15.105:0/165453801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374101540 0x7f1374101920 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f13640099c0 tx=0x7f1364009cd0 comp rx=0 tx=0).stop 2024-03-26T17:38:15.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 -- 172.21.15.105:0/165453801 shutdown_connections 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 --2- 172.21.15.105:0/165453801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374106590 0x7f137410e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 --2- 172.21.15.105:0/165453801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374101540 0x7f1374101920 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 --2- 172.21.15.105:0/165453801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1374101ef0 0x7f1374105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 -- 172.21.15.105:0/165453801 >> 172.21.15.105:0/165453801 conn(0x7f1374077300 msgr2=0x7f1374077710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 -- 172.21.15.105:0/165453801 shutdown_connections 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.794+0000 7f137aa55700 1 -- 172.21.15.105:0/165453801 wait complete. 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.795+0000 7f137aa55700 1 Processor -- start 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.795+0000 7f137aa55700 1 -- start start 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.795+0000 7f137aa55700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1374101540 0x7f137419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.795+0000 7f137aa55700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374101ef0 0x7f137419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.795+0000 7f137aa55700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374106590 0x7f1374198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:15.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.795+0000 7f137aa55700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1374111450 con 0x7f1374106590 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.795+0000 7f137aa55700 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13741112d0 con 0x7f1374101540 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f137aa55700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13741115d0 con 0x7f1374101ef0 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1374101540 0x7f137419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1378ff2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374106590 0x7f1374198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f13737fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374101ef0 0x7f137419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1374101540 0x7f137419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49402/0 (socket says 172.21.15.105:49402) 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 -- 172.21.15.105:0/886005210 learned_addr learned my addr 172.21.15.105:0/886005210 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1378ff2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374106590 0x7f1374198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52854/0 (socket says 172.21.15.105:52854) 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 -- 172.21.15.105:0/886005210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374101ef0 msgr2=0x7f137419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374101ef0 0x7f137419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 -- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374106590 msgr2=0x7f1374198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374106590 0x7f1374198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:15.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 -- 172.21.15.105:0/886005210 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1374198ff0 con 0x7f1374101540 2024-03-26T17:38:15.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f1373fff700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1374101540 0x7f137419e6d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1364006010 tx=0x7f136401ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:15.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f13717fa700 1 -- 172.21.15.105:0/886005210 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136402b070 con 0x7f1374101540 2024-03-26T17:38:15.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1374199280 con 0x7f1374101540 2024-03-26T17:38:15.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f13717fa700 1 -- 172.21.15.105:0/886005210 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f136401ce00 con 0x7f1374101540 2024-03-26T17:38:15.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f13717fa700 1 -- 172.21.15.105:0/886005210 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364026470 con 0x7f1374101540 2024-03-26T17:38:15.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.796+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1374102e80 con 0x7f1374101540 2024-03-26T17:38:15.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.797+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1374102990 con 0x7f1374101540 2024-03-26T17:38:15.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.798+0000 7f13717fa700 1 -- 172.21.15.105:0/886005210 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f136402e070 con 0x7f1374101540 2024-03-26T17:38:15.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.799+0000 7f13717fa700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f135c0a2fb0 0x7f135c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:15.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.799+0000 7f13737fe700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f135c0a2fb0 0x7f135c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:15.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.799+0000 7f13717fa700 1 -- 172.21.15.105:0/886005210 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f13640d4290 con 0x7f1374101540 2024-03-26T17:38:15.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.799+0000 7f13737fe700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f135c0a2fb0 0x7f135c0a5470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13740ffe30 tx=0x7f1368009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:15.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:15.803+0000 7f13717fa700 1 -- 172.21.15.105:0/886005210 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1364062740 con 0x7f1374101540 2024-03-26T17:38:16.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.050+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1374199dc0 con 0x7f1374101540 2024-03-26T17:38:16.043 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:15 smithi105 ceph-mon[32280]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:38:16.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.050+0000 7f13717fa700 1 -- 172.21.15.105:0/886005210 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f1364013070 con 0x7f1374101540 2024-03-26T17:38:16.044 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f135c0a2fb0 msgr2=0x7f135c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f135c0a2fb0 0x7f135c0a5470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13740ffe30 tx=0x7f1368009260 comp rx=0 tx=0).stop 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1374101540 msgr2=0x7f137419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1374101540 0x7f137419e6d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1364006010 tx=0x7f136401ca40 comp rx=0 tx=0).stop 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 shutdown_connections 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f135c0a2fb0 0x7f135c0a5470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1374106590 0x7f1374198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1374101540 0x7f137419e6d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 --2- 172.21.15.105:0/886005210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1374101ef0 0x7f137419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 >> 172.21.15.105:0/886005210 conn(0x7f1374077300 msgr2=0x7f1374107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.053+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 shutdown_connections 2024-03-26T17:38:16.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16.054+0000 7f137aa55700 1 -- 172.21.15.105:0/886005210 wait complete. 2024-03-26T17:38:16.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:15 smithi144 ceph-mon[36095]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:38:16.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:15 smithi135 ceph-mon[35636]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T17:38:16.800 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-26T17:38:16.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:16.801 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-03-26T17:38:16.818 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T17:38:16.818 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T17:38:16.818 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T17:38:16.818 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-26T17:38:16.818 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-03-26T17:38:16.818 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-03-26T17:38:16.819 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T17:38:16.820 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-03-26T17:38:16.820 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-26T17:38:16.820 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42086sec preferred_lft 42086sec 2024-03-26T17:38:16.820 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-03-26T17:38:16.820 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-03-26T17:38:16.820 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T17:38:16.821 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:38:16.821 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-26T17:38:16.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-03-26T17:38:16.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-03-26T17:38:16.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-03-26T17:38:16.821 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T17:38:16.821 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-26T17:38:16.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:38:16.977 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:16 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/886005210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T17:38:17.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:38:17.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:17.029 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T17:38:17.060 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-03-26T17:38:17.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:17.061 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-03-26T17:38:17.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:16 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/886005210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T17:38:17.135 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-03-26T17:38:17.135 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T17:38:17.135 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-26T17:38:17.135 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-26T17:38:17.136 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T17:38:17.136 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:38:17.136 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-26T17:38:17.136 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-26T17:38:17.136 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-26T17:38:17.136 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-26T17:38:17.136 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-26T17:38:17.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:16 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/886005210' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T17:38:17.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:38:17.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:38:17.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:17.447 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-26T17:38:17.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:17.464 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-03-26T17:38:17.518 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:38:17.518 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-26T17:38:17.518 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T17:38:17.518 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-26T17:38:17.519 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-26T17:38:17.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:38:17.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:38:17.641 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-26T17:38:17.641 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:38:17.641 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-26T17:38:17.641 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-26T17:38:17.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T17:38:17.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-26T17:38:17.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-26T17:38:17.642 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-26T17:38:17.642 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-26T17:38:17.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:38:17.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:38:17.788 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-26T17:38:17.788 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-03-26T17:38:17.788 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-03-26T17:38:17.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T17:38:17.789 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-03-26T17:38:17.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-26T17:38:17.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:38:17.874 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-26T17:38:17.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T17:38:17.874 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:17.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:17 smithi105 ceph-mon[32280]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:38:17.901 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-26T17:38:17.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T17:38:17.902 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:17.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-03-26T17:38:18.024 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T17:38:18.080 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-03-26T17:38:18.080 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-03-26T17:38:18.080 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-03-26T17:38:18.080 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-03-26T17:38:18.080 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-03-26T17:38:18.080 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-03-26T17:38:18.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f2e8b91e8d45a3754767f2641b0c6f2e262f151b52ac150141f3266c7f58d28e/merged 2024-03-26T17:38:18.082 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/d54973d107b1a8bf684feca433d5aae7f4fda84da92f4f25355f82d24389cb77/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/0 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/2d8e3b7b6cdbabad92cbbcee8fd792620d4e696254d6f369df85f8d48dd8786a/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/4212644b629351dd576ac52e250c31c97b988125b4a9b6cbad4e02e13979a540/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/6d750894cc5aeacb5fd55f59394ac98347e2553d604e67c4cf50f3c0373f4890/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/782043a151142270b8639e6a86ccd549cbe86bb3f31d96ea21efe0a9bc272e84/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/2993e14001fb1d61e81770863a6d439d55ac4cac95fc972b775fcaf767bfe3ec/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/1d650033c7de590f6c544f7edf85d5a1efff773361d8f4ffb8b5fde4db53c7c1/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f7fc099b95dba6bbb14cedbc001abeda13456c7f0a656c8c5f85478ea17d8b77/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/297b24121d607df23226d0bd98f01ec0bbacc144f0a27eb09917a9a64ac941d3/merged 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-03-26T17:38:18.083 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T17:38:18.084 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T17:38:18.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:18.084 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-03-26T17:38:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:17 smithi144 ceph-mon[36095]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:38:18.136 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-26T17:38:18.136 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb --client_fs=cephfs) 2024-03-26T17:38:18.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:17 smithi135 ceph-mon[35636]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T17:38:18.179 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-03-26T17:38:18.205 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-03-26T17:38:18.261 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-03-26T17:38:18.262 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f2e8b91e8d45a3754767f2641b0c6f2e262f151b52ac150141f3266c7f58d28e/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/d54973d107b1a8bf684feca433d5aae7f4fda84da92f4f25355f82d24389cb77/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/0 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/2d8e3b7b6cdbabad92cbbcee8fd792620d4e696254d6f369df85f8d48dd8786a/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/4212644b629351dd576ac52e250c31c97b988125b4a9b6cbad4e02e13979a540/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/6d750894cc5aeacb5fd55f59394ac98347e2553d604e67c4cf50f3c0373f4890/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/782043a151142270b8639e6a86ccd549cbe86bb3f31d96ea21efe0a9bc272e84/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/2993e14001fb1d61e81770863a6d439d55ac4cac95fc972b775fcaf767bfe3ec/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/1d650033c7de590f6c544f7edf85d5a1efff773361d8f4ffb8b5fde4db53c7c1/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f7fc099b95dba6bbb14cedbc001abeda13456c7f0a656c8c5f85478ea17d8b77/merged 2024-03-26T17:38:18.263 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/297b24121d607df23226d0bd98f01ec0bbacc144f0a27eb09917a9a64ac941d3/merged 2024-03-26T17:38:18.264 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-03-26T17:38:18.264 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T17:38:18.264 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T17:38:18.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:18.264 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-03-26T17:38:18.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:2024-03-26T17:38:18.291+0000 7f5c95635480 -1 init, newargv = 0x555d7ec27650 newargc=15 2024-03-26T17:38:18.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[83984]: starting ceph client 2024-03-26T17:38:18.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[83984]: starting fuse 2024-03-26T17:38:19.401 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-03-26T17:38:19.430 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T17:38:19.485 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-03-26T17:38:19.485 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-03-26T17:38:19.485 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-03-26T17:38:19.485 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-03-26T17:38:19.485 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-03-26T17:38:19.485 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-03-26T17:38:19.485 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-03-26T17:38:19.486 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f2e8b91e8d45a3754767f2641b0c6f2e262f151b52ac150141f3266c7f58d28e/merged 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/d54973d107b1a8bf684feca433d5aae7f4fda84da92f4f25355f82d24389cb77/merged 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/0 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/2d8e3b7b6cdbabad92cbbcee8fd792620d4e696254d6f369df85f8d48dd8786a/merged 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/4212644b629351dd576ac52e250c31c97b988125b4a9b6cbad4e02e13979a540/merged 2024-03-26T17:38:19.487 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/6d750894cc5aeacb5fd55f59394ac98347e2553d604e67c4cf50f3c0373f4890/merged 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/782043a151142270b8639e6a86ccd549cbe86bb3f31d96ea21efe0a9bc272e84/merged 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/2993e14001fb1d61e81770863a6d439d55ac4cac95fc972b775fcaf767bfe3ec/merged 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/1d650033c7de590f6c544f7edf85d5a1efff773361d8f4ffb8b5fde4db53c7c1/merged 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/f7fc099b95dba6bbb14cedbc001abeda13456c7f0a656c8c5f85478ea17d8b77/merged 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/containers/storage/overlay/297b24121d607df23226d0bd98f01ec0bbacc144f0a27eb09917a9a64ac941d3/merged 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T17:38:19.488 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:19.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:19.489 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-03-26T17:38:19.545 INFO:teuthology.orchestra.run.smithi105.stdout:119 2024-03-26T17:38:19.546 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-26T17:38:19.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2024-03-26T17:38:19.546 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-03-26T17:38:19.546 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-03-26T17:38:19.546 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-03-26T17:38:19.546 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-03-26T17:38:19.546 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-26T17:38:19.546 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-03-26T17:38:19.546 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-03-26T17:38:19.547 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-26T17:38:19.548 DEBUG:teuthology.orchestra.run.smithi105:> 2024-03-26T17:38:19.548 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.0")) 2024-03-26T17:38:19.548 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-03-26T17:38:19.670 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.0.83984.asok 2024-03-26T17:38:19.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T17:38:19.680 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83984.asok 2024-03-26T17:38:19.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:19.681 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83984.asok status 2024-03-26T17:38:19.818 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:19.818 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-03-26T17:38:19.818 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "360516069d9393362c4cc6eb9371680fe16d66ab", 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2311-g36051606 (360516069d9393362c4cc6eb9371680fe16d66ab) squid (dev)", 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "0", 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "83984", 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb" 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 15330, 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-03-26T17:38:19.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 15330 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3678761304 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3678761304 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T17:38:19.820 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.15330 192.168.0.1:0/3678761304", 2024-03-26T17:38:19.821 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/3678761304", 2024-03-26T17:38:19.821 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 5, 2024-03-26T17:38:19.821 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 26, 2024-03-26T17:38:19.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 87, 2024-03-26T17:38:19.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-03-26T17:38:19.821 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-03-26T17:38:19.821 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-03-26T17:38:19.821 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:19.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:19.827 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2024-03-26T17:38:19.827 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:19.842 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2024-03-26T17:38:19.843 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:19.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:19.843 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:19.911 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T17:38:19.924 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T17:38:19.924 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T17:38:19.924 DEBUG:teuthology.orchestra.run.smithi105:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-26T17:38:19.975 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:19 smithi105 ceph-mon[32280]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:38:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 -- 172.21.15.105:0/3568572289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838107600 msgr2=0x7fa838109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/3568572289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838107600 0x7fa838109a10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa830009ca0 tx=0x7fa83002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 -- 172.21.15.105:0/3568572289 shutdown_connections 2024-03-26T17:38:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/3568572289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838107600 0x7fa838109a10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/3568572289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa838104cd0 0x7fa8381070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/3568572289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8381023a0 0x7fa838104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 -- 172.21.15.105:0/3568572289 >> 172.21.15.105:0/3568572289 conn(0x7fa8380fbd50 msgr2=0x7fa8380fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 -- 172.21.15.105:0/3568572289 shutdown_connections 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 -- 172.21.15.105:0/3568572289 wait complete. 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 Processor -- start 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 -- start start 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.064+0000 7fa83e8ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8381023a0 0x7fa838101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa83e8ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838104cd0 0x7fa838100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa83e8ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa838107600 0x7fa838100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa83e8ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d430 con 0x7fa838104cd0 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa83e8ac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d2b0 con 0x7fa8381023a0 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa83e8ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa83810d5b0 con 0x7fa838107600 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838104cd0 0x7fa838100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:20.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838104cd0 0x7fa838100350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52872/0 (socket says 172.21.15.105:52872) 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 -- 172.21.15.105:0/4164607024 learned_addr learned my addr 172.21.15.105:0/4164607024 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa837fff640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8381023a0 0x7fa838101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa83ce49640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa838107600 0x7fa838100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 -- 172.21.15.105:0/4164607024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa838107600 msgr2=0x7fa838100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa838107600 0x7fa838100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 -- 172.21.15.105:0/4164607024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8381023a0 msgr2=0x7fa838101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8381023a0 0x7fa838101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 -- 172.21.15.105:0/4164607024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa838101150 con 0x7fa838104cd0 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8377fe640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838104cd0 0x7fa838100350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa82800e9f0 tx=0x7fa82800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:20.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8357fa640 1 -- 172.21.15.105:0/4164607024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82800ccc0 con 0x7fa838104cd0 2024-03-26T17:38:20.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8357fa640 1 -- 172.21.15.105:0/4164607024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8280054d0 con 0x7fa838104cd0 2024-03-26T17:38:20.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8381a1bc0 con 0x7fa838104cd0 2024-03-26T17:38:20.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa8357fa640 1 -- 172.21.15.105:0/4164607024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828010430 con 0x7fa838104cd0 2024-03-26T17:38:20.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.065+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8381a20e0 con 0x7fa838104cd0 2024-03-26T17:38:20.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.066+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8380fd840 con 0x7fa838104cd0 2024-03-26T17:38:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.067+0000 7fa8357fa640 1 -- 172.21.15.105:0/4164607024 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7fa828005000 con 0x7fa838104cd0 2024-03-26T17:38:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.068+0000 7fa8357fa640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8200a3040 0x7fa8200a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.068+0000 7fa837fff640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8200a3040 0x7fa8200a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.068+0000 7fa8357fa640 1 -- 172.21.15.105:0/4164607024 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fa8280cf8c0 con 0x7fa838104cd0 2024-03-26T17:38:20.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.068+0000 7fa837fff640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8200a3040 0x7fa8200a5500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009bd0 tx=0x7fa82c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:20.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.073+0000 7fa8357fa640 1 -- 172.21.15.105:0/4164607024 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa828053740 con 0x7fa838104cd0 2024-03-26T17:38:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:19 smithi144 ceph-mon[36095]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:38:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:19 smithi135 ceph-mon[35636]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-03-26T17:38:20.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.294+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fa83802d240 con 0x7fa838104cd0 2024-03-26T17:38:20.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.701+0000 7fa8357fa640 1 -- 172.21.15.105:0/4164607024 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7fa8280052f0 con 0x7fa838104cd0 2024-03-26T17:38:20.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8200a3040 msgr2=0x7fa8200a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:20.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8200a3040 0x7fa8200a5500 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009bd0 tx=0x7fa82c005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:20.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838104cd0 msgr2=0x7fa838100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:20.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838104cd0 0x7fa838100350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa82800e9f0 tx=0x7fa82800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 shutdown_connections 2024-03-26T17:38:20.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8200a3040 0x7fa8200a5500 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa838107600 0x7fa838100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa838104cd0 0x7fa838100350 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 --2- 172.21.15.105:0/4164607024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8381023a0 0x7fa838101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.703+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 >> 172.21.15.105:0/4164607024 conn(0x7fa8380fbd50 msgr2=0x7fa838102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:20.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.704+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 shutdown_connections 2024-03-26T17:38:20.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.704+0000 7fa83e8ac640 1 -- 172.21.15.105:0/4164607024 wait complete. 2024-03-26T17:38:20.709 DEBUG:teuthology.orchestra.run.smithi105:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1906210580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcf01007e0 msgr2=0x7fdcf0100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1906210580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcf01007e0 0x7fdcf0100c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009900 tx=0x7fdce402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1906210580 shutdown_connections 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1906210580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf0102390 0x7fdcf0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1906210580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf01019d0 0x7fdcf0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1906210580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcf01007e0 0x7fdcf0100c00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1906210580 >> 172.21.15.105:0/1906210580 conn(0x7fdcf00fbd50 msgr2=0x7fdcf00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1906210580 shutdown_connections 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1906210580 wait complete. 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 Processor -- start 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.863+0000 7fdcf6cf8640 1 -- start start 2024-03-26T17:38:20.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf6cf8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf01007e0 0x7fdcf0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf6cf8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcf01019d0 0x7fdcf0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf6cf8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf0102390 0x7fdcf019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf6cf8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf010c2e0 con 0x7fdcf01007e0 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf6cf8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf010c160 con 0x7fdcf01019d0 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf6cf8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf010c460 con 0x7fdcf0102390 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf01007e0 0x7fdcf0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf01007e0 0x7fdcf0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52884/0 (socket says 172.21.15.105:52884) 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 -- 172.21.15.105:0/1939345524 learned_addr learned my addr 172.21.15.105:0/1939345524 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdceffff640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcf01019d0 0x7fdcf0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf5295640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf0102390 0x7fdcf019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 -- 172.21.15.105:0/1939345524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf0102390 msgr2=0x7fdcf019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf0102390 0x7fdcf019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 -- 172.21.15.105:0/1939345524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcf01019d0 msgr2=0x7fdcf0198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcf01019d0 0x7fdcf0198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 -- 172.21.15.105:0/1939345524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf019f6f0 con 0x7fdcf01007e0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf4a94640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf01007e0 0x7fdcf0197ad0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdce4000c00 tx=0x7fdce4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcedffb640 1 -- 172.21.15.105:0/1939345524 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4043020 con 0x7fdcf01007e0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf019f980 con 0x7fdcf01007e0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcedffb640 1 -- 172.21.15.105:0/1939345524 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce4004350 con 0x7fdcf01007e0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.864+0000 7fdcedffb640 1 -- 172.21.15.105:0/1939345524 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003b10 con 0x7fdcf01007e0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.865+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf019fed0 con 0x7fdcf01007e0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.866+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf01a00a0 con 0x7fdcf01007e0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.867+0000 7fdcedffb640 1 -- 172.21.15.105:0/1939345524 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7fdce4046020 con 0x7fdcf01007e0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.867+0000 7fdcedffb640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdce00a2fc0 0x7fdce00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.867+0000 7fdceffff640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdce00a2fc0 0x7fdce00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:20.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.867+0000 7fdcedffb640 1 -- 172.21.15.105:0/1939345524 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fdce40f18f0 con 0x7fdcf01007e0 2024-03-26T17:38:20.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.868+0000 7fdceffff640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdce00a2fc0 0x7fdce00a5480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0198fe0 tx=0x7fdcdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:20.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:20.871+0000 7fdcedffb640 1 -- 172.21.15.105:0/1939345524 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdce404d020 con 0x7fdcf01007e0 2024-03-26T17:38:21.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.082+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fdcf0069650 con 0x7fdce00a2fc0 2024-03-26T17:38:21.080 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:20 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4164607024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T17:38:21.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.098+0000 7fdcedffb640 1 -- 172.21.15.105:0/1939345524 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fdcf0069650 con 0x7fdce00a2fc0 2024-03-26T17:38:21.090 INFO:teuthology.orchestra.run.smithi105.stdout:{} 2024-03-26T17:38:21.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdce00a2fc0 msgr2=0x7fdce00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdce00a2fc0 0x7fdce00a5480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0198fe0 tx=0x7fdcdc005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf01007e0 msgr2=0x7fdcf0197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf01007e0 0x7fdcf0197ad0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdce4000c00 tx=0x7fdce4005dc0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 shutdown_connections 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdce00a2fc0 0x7fdce00a5480 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf0102390 0x7fdcf019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdcf01019d0 0x7fdcf0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 --2- 172.21.15.105:0/1939345524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdcf01007e0 0x7fdcf0197ad0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 >> 172.21.15.105:0/1939345524 conn(0x7fdcf00fbd50 msgr2=0x7fdcf0068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 shutdown_connections 2024-03-26T17:38:21.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.101+0000 7fdcf6cf8640 1 -- 172.21.15.105:0/1939345524 wait complete. 2024-03-26T17:38:21.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:20 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4164607024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T17:38:21.105 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-26T17:38:21.146 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-26T17:38:21.160 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:38:21.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:20 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4164607024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T17:38:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 -- 172.21.15.105:0/914964744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 msgr2=0x7f6774109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 --2- 172.21.15.105:0/914964744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 0x7f6774109d30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6764009900 tx=0x7f676402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 -- 172.21.15.105:0/914964744 shutdown_connections 2024-03-26T17:38:21.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 --2- 172.21.15.105:0/914964744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 0x7f6774109d30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 --2- 172.21.15.105:0/914964744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6774102ae0 0x7f6774102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 --2- 172.21.15.105:0/914964744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67741018f0 0x7f6774101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 -- 172.21.15.105:0/914964744 >> 172.21.15.105:0/914964744 conn(0x7f67740fce60 msgr2=0x7f67740ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 -- 172.21.15.105:0/914964744 shutdown_connections 2024-03-26T17:38:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.271+0000 7f67793e6640 1 -- 172.21.15.105:0/914964744 wait complete. 2024-03-26T17:38:21.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.272+0000 7f67793e6640 1 Processor -- start 2024-03-26T17:38:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.272+0000 7f67793e6640 1 -- start start 2024-03-26T17:38:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.272+0000 7f67793e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67741018f0 0x7f6774197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.272+0000 7f67793e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6774102ae0 0x7f6774198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67793e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 0x7f677419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67793e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f677410d400 con 0x7f67741034a0 2024-03-26T17:38:21.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67793e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f677410d280 con 0x7f67741018f0 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67793e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f677410d580 con 0x7f6774102ae0 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 0x7f677419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 0x7f677419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52906/0 (socket says 172.21.15.105:52906) 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 -- 172.21.15.105:0/3599194308 learned_addr learned my addr 172.21.15.105:0/3599194308 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67727fc640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6774102ae0 0x7f6774198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f6772ffd640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67741018f0 0x7f6774197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 -- 172.21.15.105:0/3599194308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6774102ae0 msgr2=0x7f6774198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6774102ae0 0x7f6774198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 -- 172.21.15.105:0/3599194308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67741018f0 msgr2=0x7f6774197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67741018f0 0x7f6774197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 -- 172.21.15.105:0/3599194308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677419f6f0 con 0x7f67741034a0 2024-03-26T17:38:21.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67737fe640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 0x7f677419f080 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6764009230 tx=0x7f6764005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f675bfff640 1 -- 172.21.15.105:0/3599194308 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764043020 con 0x7f67741034a0 2024-03-26T17:38:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677419f980 con 0x7f67741034a0 2024-03-26T17:38:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f675bfff640 1 -- 172.21.15.105:0/3599194308 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67640043d0 con 0x7f67741034a0 2024-03-26T17:38:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f675bfff640 1 -- 172.21.15.105:0/3599194308 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764003b40 con 0x7f67741034a0 2024-03-26T17:38:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.273+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677419ff20 con 0x7f67741034a0 2024-03-26T17:38:21.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.274+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6774108c90 con 0x7f67741034a0 2024-03-26T17:38:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.275+0000 7f675bfff640 1 -- 172.21.15.105:0/3599194308 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f6764046020 con 0x7f67741034a0 2024-03-26T17:38:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.276+0000 7f675bfff640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f675c0a2f20 0x7f675c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.276+0000 7f6772ffd640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f675c0a2f20 0x7f675c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.276+0000 7f675bfff640 1 -- 172.21.15.105:0/3599194308 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f67640f1580 con 0x7f67741034a0 2024-03-26T17:38:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.277+0000 7f6772ffd640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f675c0a2f20 0x7f675c0a53e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6774102940 tx=0x7f676c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.281+0000 7f675bfff640 1 -- 172.21.15.105:0/3599194308 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f676404d020 con 0x7f67741034a0 2024-03-26T17:38:21.392 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:38:21.087+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:38:21.392 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:38:21.087+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:38:21.392 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:38:21.087+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:38:21.393 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:38:21.087+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:38:21.393 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T17:38:21.087+0000 7f4572289700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T17:38:21.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.469+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67740624c0 con 0x7f67741034a0 2024-03-26T17:38:21.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.472+0000 7f675bfff640 1 -- 172.21.15.105:0/3599194308 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f67640be3d0 con 0x7f67741034a0 2024-03-26T17:38:21.464 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:21.465 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:38:21.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f675c0a2f20 msgr2=0x7f675c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f675c0a2f20 0x7f675c0a53e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6774102940 tx=0x7f676c005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 msgr2=0x7f677419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 0x7f677419f080 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6764009230 tx=0x7f6764005e40 comp rx=0 tx=0).stop 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 shutdown_connections 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f675c0a2f20 0x7f675c0a53e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67741034a0 0x7f677419f080 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6774102ae0 0x7f6774198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 --2- 172.21.15.105:0/3599194308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67741018f0 0x7f6774197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 >> 172.21.15.105:0/3599194308 conn(0x7f67740fce60 msgr2=0x7f6774105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 shutdown_connections 2024-03-26T17:38:21.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.474+0000 7f67793e6640 1 -- 172.21.15.105:0/3599194308 wait complete. 2024-03-26T17:38:21.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:38:21.478 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:38:21.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- 172.21.15.105:0/974164871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 msgr2=0x7f7680101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 --2- 172.21.15.105:0/974164871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 0x7f7680101e30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f766c009900 tx=0x7f766c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- 172.21.15.105:0/974164871 shutdown_connections 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 --2- 172.21.15.105:0/974164871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7680102370 0x7f7680108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 --2- 172.21.15.105:0/974164871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 0x7f7680101e30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 --2- 172.21.15.105:0/974164871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76801007c0 0x7f7680100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- 172.21.15.105:0/974164871 >> 172.21.15.105:0/974164871 conn(0x7f76800fbd50 msgr2=0x7f76800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- 172.21.15.105:0/974164871 shutdown_connections 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- 172.21.15.105:0/974164871 wait complete. 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 Processor -- start 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- start start 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76801007c0 0x7f7680197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 0x7f76801980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7680102370 0x7f768019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f768010c2f0 con 0x7f76801019b0 2024-03-26T17:38:21.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f768010c170 con 0x7f7680102370 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.591+0000 7f7687259640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f768010c470 con 0x7f76801007c0 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 0x7f76801980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 0x7f76801980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52924/0 (socket says 172.21.15.105:52924) 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 -- 172.21.15.105:0/1528637578 learned_addr learned my addr 172.21.15.105:0/1528637578 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f7684ff5640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76801007c0 0x7f7680197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f76857f6640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7680102370 0x7f768019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 -- 172.21.15.105:0/1528637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76801007c0 msgr2=0x7f7680197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76801007c0 0x7f7680197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 -- 172.21.15.105:0/1528637578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7680102370 msgr2=0x7f768019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7680102370 0x7f768019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 -- 172.21.15.105:0/1528637578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768019f790 con 0x7f76801019b0 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f7684ff5640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76801007c0 0x7f7680197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f76857f6640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7680102370 0x7f768019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:21.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767ffff640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 0x7f76801980b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:21.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767dffb640 1 -- 172.21.15.105:0/1528637578 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c043020 con 0x7f76801019b0 2024-03-26T17:38:21.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768019fa20 con 0x7f76801019b0 2024-03-26T17:38:21.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767dffb640 1 -- 172.21.15.105:0/1528637578 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f766c0043d0 con 0x7f76801019b0 2024-03-26T17:38:21.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f767dffb640 1 -- 172.21.15.105:0/1528637578 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c003b80 con 0x7f76801019b0 2024-03-26T17:38:21.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.592+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768019ffc0 con 0x7f76801019b0 2024-03-26T17:38:21.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.594+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76800686b0 con 0x7f76801019b0 2024-03-26T17:38:21.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.595+0000 7f767dffb640 1 -- 172.21.15.105:0/1528637578 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) ==== 137855+0+0 (secure 0 0 0) 0x7f766c046020 con 0x7f76801019b0 2024-03-26T17:38:21.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.595+0000 7f767dffb640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76700a2f20 0x7f76700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.595+0000 7f7684ff5640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76700a2f20 0x7f76700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.595+0000 7f767dffb640 1 -- 172.21.15.105:0/1528637578 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f766c0f15a0 con 0x7f76801019b0 2024-03-26T17:38:21.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.596+0000 7f7684ff5640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76700a2f20 0x7f76700a53e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7680101810 tx=0x7f7674005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:21.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.599+0000 7f767dffb640 1 -- 172.21.15.105:0/1528637578 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f766c04d020 con 0x7f76801019b0 2024-03-26T17:38:21.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.723+0000 7f767dffb640 1 -- 172.21.15.105:0/1528637578 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f766c0ba360 con 0x7f76801019b0 2024-03-26T17:38:21.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.803+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76801a0840 con 0x7f76801019b0 2024-03-26T17:38:21.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.803+0000 7f767dffb640 1 -- 172.21.15.105:0/1528637578 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f766c0be3f0 con 0x7f76801019b0 2024-03-26T17:38:21.796 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:38:21.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76700a2f20 msgr2=0x7f76700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76700a2f20 0x7f76700a53e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7680101810 tx=0x7f7674005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 msgr2=0x7f76801980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 0x7f76801980b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c005e40 comp rx=0 tx=0).stop 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 shutdown_connections 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76700a2f20 0x7f76700a53e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7680102370 0x7f768019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76801019b0 0x7f76801980b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 --2- 172.21.15.105:0/1528637578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76801007c0 0x7f7680197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 >> 172.21.15.105:0/1528637578 conn(0x7f76800fbd50 msgr2=0x7f7680068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 shutdown_connections 2024-03-26T17:38:21.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.806+0000 7f7687259640 1 -- 172.21.15.105:0/1528637578 wait complete. 2024-03-26T17:38:21.809 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-mon[32280]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:38:21.809 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4164607024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T17:38:21.809 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:38:21.809 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:21 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3599194308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:21.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T17:38:21.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 -- 172.21.15.105:0/1689988021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 msgr2=0x7f0218101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 --2- 172.21.15.105:0/1689988021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 0x7f0218101e30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f020c009900 tx=0x7f020c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:21.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 -- 172.21.15.105:0/1689988021 shutdown_connections 2024-03-26T17:38:21.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 --2- 172.21.15.105:0/1689988021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0218102370 0x7f0218108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 --2- 172.21.15.105:0/1689988021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 0x7f0218101e30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 --2- 172.21.15.105:0/1689988021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02181007c0 0x7f0218100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 -- 172.21.15.105:0/1689988021 >> 172.21.15.105:0/1689988021 conn(0x7f02180fbd50 msgr2=0x7f02180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:21.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 -- 172.21.15.105:0/1689988021 shutdown_connections 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.928+0000 7f021def1640 1 -- 172.21.15.105:0/1689988021 wait complete. 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f021def1640 1 Processor -- start 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f021def1640 1 -- start start 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f021def1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02181007c0 0x7f0218197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f021def1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 0x7f0218198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f021def1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0218102370 0x7f021819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f021def1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f021810c2f0 con 0x7f02181019b0 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f021def1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f021810c170 con 0x7f0218102370 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f021def1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f021810c470 con 0x7f02181007c0 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f0216ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 0x7f0218198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f0216ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 0x7f0218198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52942/0 (socket says 172.21.15.105:52942) 2024-03-26T17:38:21.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f0216ffd640 1 -- 172.21.15.105:0/3821171732 learned_addr learned my addr 172.21.15.105:0/3821171732 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f0217fff640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0218102370 0x7f021819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.929+0000 7f02177fe640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02181007c0 0x7f0218197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0216ffd640 1 -- 172.21.15.105:0/3821171732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02181007c0 msgr2=0x7f0218197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0216ffd640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02181007c0 0x7f0218197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0216ffd640 1 -- 172.21.15.105:0/3821171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0218102370 msgr2=0x7f021819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0216ffd640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0218102370 0x7f021819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0216ffd640 1 -- 172.21.15.105:0/3821171732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021819f740 con 0x7f02181019b0 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0216ffd640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 0x7f0218198060 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f020c006010 tx=0x7f020c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0214ff9640 1 -- 172.21.15.105:0/3821171732 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c043020 con 0x7f02181019b0 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021819f9d0 con 0x7f02181019b0 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0214ff9640 1 -- 172.21.15.105:0/3821171732 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f020c0043d0 con 0x7f02181019b0 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f0214ff9640 1 -- 172.21.15.105:0/3821171732 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c003b80 con 0x7f02181019b0 2024-03-26T17:38:21.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.930+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021819ff20 con 0x7f02181019b0 2024-03-26T17:38:21.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.931+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02181a00f0 con 0x7f02181019b0 2024-03-26T17:38:21.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.933+0000 7f0214ff9640 1 -- 172.21.15.105:0/3821171732 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f020c046020 con 0x7f02181019b0 2024-03-26T17:38:21.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.933+0000 7f0214ff9640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02000a2fb0 0x7f02000a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:21.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.933+0000 7f02177fe640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02000a2fb0 0x7f02000a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:21.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.933+0000 7f0214ff9640 1 -- 172.21.15.105:0/3821171732 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f020c0f1630 con 0x7f02181019b0 2024-03-26T17:38:21.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.934+0000 7f02177fe640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02000a2fb0 0x7f02000a5470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0218101810 tx=0x7f0208005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:21.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:21.938+0000 7f0214ff9640 1 -- 172.21.15.105:0/3821171732 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f020c04d020 con 0x7f02181019b0 2024-03-26T17:38:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:21 smithi144 ceph-mon[36095]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:38:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:21 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4164607024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T17:38:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:21 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:38:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:21 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3599194308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:22.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.117+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f02181a02c0 con 0x7f02181019b0 2024-03-26T17:38:22.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.117+0000 7f0214ff9640 1 -- 172.21.15.105:0/3821171732 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f020c0be430 con 0x7f02181019b0 2024-03-26T17:38:22.110 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T17:38:22.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02000a2fb0 msgr2=0x7f02000a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02000a2fb0 0x7f02000a5470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0218101810 tx=0x7f0208005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:22.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 msgr2=0x7f0218198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 0x7f0218198060 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f020c006010 tx=0x7f020c005e40 comp rx=0 tx=0).stop 2024-03-26T17:38:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 shutdown_connections 2024-03-26T17:38:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02000a2fb0 0x7f02000a5470 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0218102370 0x7f021819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02181019b0 0x7f0218198060 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 --2- 172.21.15.105:0/3821171732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02181007c0 0x7f0218197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.119+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 >> 172.21.15.105:0/3821171732 conn(0x7f02180fbd50 msgr2=0x7f0218068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.120+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 shutdown_connections 2024-03-26T17:38:22.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.120+0000 7f021def1640 1 -- 172.21.15.105:0/3821171732 wait complete. 2024-03-26T17:38:22.123 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T17:38:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:21 smithi135 ceph-mon[35636]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-26T17:38:22.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:21 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4164607024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T17:38:22.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:21 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:38:22.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:21 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3599194308' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.237+0000 7f3f20a36640 1 -- 172.21.15.105:0/1759347127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 msgr2=0x7f3f1c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.237+0000 7f3f20a36640 1 --2- 172.21.15.105:0/1759347127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 0x7f3f1c101e30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009900 tx=0x7f3f1002bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 -- 172.21.15.105:0/1759347127 shutdown_connections 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 --2- 172.21.15.105:0/1759347127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f1c102370 0x7f3f1c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 --2- 172.21.15.105:0/1759347127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 0x7f3f1c101e30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 --2- 172.21.15.105:0/1759347127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f1c1007c0 0x7f3f1c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 -- 172.21.15.105:0/1759347127 >> 172.21.15.105:0/1759347127 conn(0x7f3f1c0fbd50 msgr2=0x7f3f1c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 -- 172.21.15.105:0/1759347127 shutdown_connections 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 -- 172.21.15.105:0/1759347127 wait complete. 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 Processor -- start 2024-03-26T17:38:22.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 -- start start 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.238+0000 7f3f20a36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f1c1007c0 0x7f3f1c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f20a36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 0x7f3f1c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f20a36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f1c102370 0x7f3f1c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f20a36640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10c2f0 con 0x7f3f1c1019b0 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f20a36640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10c170 con 0x7f3f1c1007c0 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f20a36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c10c470 con 0x7f3f1c102370 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 0x7f3f1c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 0x7f3f1c19ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52970/0 (socket says 172.21.15.105:52970) 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 -- 172.21.15.105:0/3804173166 learned_addr learned my addr 172.21.15.105:0/3804173166 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f1a59c640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f1c1007c0 0x7f3f1c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f1ad9d640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f1c102370 0x7f3f1c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 -- 172.21.15.105:0/3804173166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f1c102370 msgr2=0x7f3f1c1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f1c102370 0x7f3f1c1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 -- 172.21.15.105:0/3804173166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f1c1007c0 msgr2=0x7f3f1c069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f1c1007c0 0x7f3f1c069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 -- 172.21.15.105:0/3804173166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c1a19c0 con 0x7f3f1c1019b0 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f19d9b640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 0x7f3f1c19ec80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3f10006010 tx=0x7f3f10005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f0b7fe640 1 -- 172.21.15.105:0/3804173166 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10043020 con 0x7f3f1c1019b0 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1c1a1bf0 con 0x7f3f1c1019b0 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f0b7fe640 1 -- 172.21.15.105:0/3804173166 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f100043d0 con 0x7f3f1c1019b0 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f0b7fe640 1 -- 172.21.15.105:0/3804173166 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10003b80 con 0x7f3f1c1019b0 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.239+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1c1a2190 con 0x7f3f1c1019b0 2024-03-26T17:38:22.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.240+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1c107c00 con 0x7f3f1c1019b0 2024-03-26T17:38:22.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.241+0000 7f3f0b7fe640 1 -- 172.21.15.105:0/3804173166 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f3f10046020 con 0x7f3f1c1019b0 2024-03-26T17:38:22.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.242+0000 7f3f0b7fe640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f040a2fe0 0x7f3f040a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.242+0000 7f3f1a59c640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f040a2fe0 0x7f3f040a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.242+0000 7f3f0b7fe640 1 -- 172.21.15.105:0/3804173166 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f3f100f1c50 con 0x7f3f1c1019b0 2024-03-26T17:38:22.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.243+0000 7f3f1a59c640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f040a2fe0 0x7f3f040a54a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c101810 tx=0x7f3f0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:22.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.246+0000 7f3f0b7fe640 1 -- 172.21.15.105:0/3804173166 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3f1004d020 con 0x7f3f1c1019b0 2024-03-26T17:38:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.528+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f1c02d400 con 0x7f3f1c1019b0 2024-03-26T17:38:22.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.529+0000 7f3f0b7fe640 1 -- 172.21.15.105:0/3804173166 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f3f100bea50 con 0x7f3f1c1019b0 2024-03-26T17:38:22.521 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T17:38:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f040a2fe0 msgr2=0x7f3f040a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f040a2fe0 0x7f3f040a54a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c101810 tx=0x7f3f0c005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 msgr2=0x7f3f1c19ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 0x7f3f1c19ec80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3f10006010 tx=0x7f3f10005e40 comp rx=0 tx=0).stop 2024-03-26T17:38:22.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 shutdown_connections 2024-03-26T17:38:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f040a2fe0 0x7f3f040a54a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f1c102370 0x7f3f1c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f1c1019b0 0x7f3f1c19ec80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 --2- 172.21.15.105:0/3804173166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f1c1007c0 0x7f3f1c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 >> 172.21.15.105:0/3804173166 conn(0x7f3f1c0fbd50 msgr2=0x7f3f1c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 shutdown_connections 2024-03-26T17:38:22.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.531+0000 7f3f20a36640 1 -- 172.21.15.105:0/3804173166 wait complete. 2024-03-26T17:38:22.537 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:38:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 -- 172.21.15.105:0/1963955890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898102390 msgr2=0x7f9898108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 --2- 172.21.15.105:0/1963955890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898102390 0x7f9898108c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9890009ca0 tx=0x7f989002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 -- 172.21.15.105:0/1963955890 shutdown_connections 2024-03-26T17:38:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 --2- 172.21.15.105:0/1963955890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9898102390 0x7f9898108c20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 --2- 172.21.15.105:0/1963955890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98981019d0 0x7f9898101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 --2- 172.21.15.105:0/1963955890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98981007e0 0x7f9898100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 -- 172.21.15.105:0/1963955890 >> 172.21.15.105:0/1963955890 conn(0x7f98980fbd50 msgr2=0x7f98980fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:22.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 -- 172.21.15.105:0/1963955890 shutdown_connections 2024-03-26T17:38:22.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.647+0000 7f989d0a1640 1 -- 172.21.15.105:0/1963955890 wait complete. 2024-03-26T17:38:22.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.648+0000 7f989d0a1640 1 Processor -- start 2024-03-26T17:38:22.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.648+0000 7f989d0a1640 1 -- start start 2024-03-26T17:38:22.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.648+0000 7f989d0a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98981007e0 0x7f9898104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.648+0000 7f989d0a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98981019d0 0x7f9898102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.648+0000 7f989d0a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9898102390 0x7f98981032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.648+0000 7f989d0a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f989810c310 con 0x7f98981019d0 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.648+0000 7f989d0a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f989810c190 con 0x7f98981007e0 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.648+0000 7f989d0a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f989810c490 con 0x7f9898102390 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98981019d0 0x7f9898102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98981019d0 0x7f9898102db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52988/0 (socket says 172.21.15.105:52988) 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 -- 172.21.15.105:0/2826853817 learned_addr learned my addr 172.21.15.105:0/2826853817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989759e640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9898102390 0x7f98981032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f9896d9d640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98981007e0 0x7f9898104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 -- 172.21.15.105:0/2826853817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9898102390 msgr2=0x7f98981032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9898102390 0x7f98981032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 -- 172.21.15.105:0/2826853817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98981007e0 msgr2=0x7f9898104700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98981007e0 0x7f9898104700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 -- 172.21.15.105:0/2826853817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9898103b10 con 0x7f98981019d0 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f9896d9d640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98981007e0 0x7f9898104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989659c640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98981019d0 0x7f9898102db0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e9f0 tx=0x7f988c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f987ffff640 1 -- 172.21.15.105:0/2826853817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c00cd80 con 0x7f98981019d0 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f987ffff640 1 -- 172.21.15.105:0/2826853817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f988c0108e0 con 0x7f98981019d0 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98981a1c20 con 0x7f98981019d0 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f987ffff640 1 -- 172.21.15.105:0/2826853817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c018920 con 0x7f98981019d0 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.649+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98981a2170 con 0x7f98981019d0 2024-03-26T17:38:22.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.650+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98981a2340 con 0x7f98981019d0 2024-03-26T17:38:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.652+0000 7f987ffff640 1 -- 172.21.15.105:0/2826853817 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f988c010a90 con 0x7f98981019d0 2024-03-26T17:38:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.652+0000 7f987ffff640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98800a2fb0 0x7f98800a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.652+0000 7f987ffff640 1 -- 172.21.15.105:0/2826853817 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f988c0cfcb0 con 0x7f98981019d0 2024-03-26T17:38:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.652+0000 7f9896d9d640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98800a2fb0 0x7f98800a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.653+0000 7f9896d9d640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98800a2fb0 0x7f98800a5470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9898101830 tx=0x7f9888005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:22.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.656+0000 7f987ffff640 1 -- 172.21.15.105:0/2826853817 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f988c09c350 con 0x7f98981019d0 2024-03-26T17:38:22.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.851+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98981a26d0 con 0x7f98981019d0 2024-03-26T17:38:22.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.852+0000 7f987ffff640 1 -- 172.21.15.105:0/2826853817 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f988c0a03f0 con 0x7f98981019d0 2024-03-26T17:38:22.845 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:22.845 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:38:22.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.854+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98800a2fb0 msgr2=0x7f98800a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.854+0000 7f989d0a1640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98800a2fb0 0x7f98800a5470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9898101830 tx=0x7f9888005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.854+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98981019d0 msgr2=0x7f9898102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.854+0000 7f989d0a1640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98981019d0 0x7f9898102db0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e9f0 tx=0x7f988c00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.855+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 shutdown_connections 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.855+0000 7f989d0a1640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98800a2fb0 0x7f98800a5470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.855+0000 7f989d0a1640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9898102390 0x7f98981032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.855+0000 7f989d0a1640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98981019d0 0x7f9898102db0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.855+0000 7f989d0a1640 1 --2- 172.21.15.105:0/2826853817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98981007e0 0x7f9898104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.855+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 >> 172.21.15.105:0/2826853817 conn(0x7f98980fbd50 msgr2=0x7f98980fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.855+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 shutdown_connections 2024-03-26T17:38:22.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.855+0000 7f989d0a1640 1 -- 172.21.15.105:0/2826853817 wait complete. 2024-03-26T17:38:22.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:22 smithi105 ceph-mon[32280]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:22.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:22 smithi105 ceph-mon[32280]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-26T17:38:22.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1528637578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:22.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3821171732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:22.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:22 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3804173166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:22.862 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:38:22.862 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:38:22.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 -- 172.21.15.105:0/3809904425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5041018f0 msgr2=0x7fb504101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 --2- 172.21.15.105:0/3809904425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5041018f0 0x7fb504101d10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:22.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 -- 172.21.15.105:0/3809904425 shutdown_connections 2024-03-26T17:38:22.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 --2- 172.21.15.105:0/3809904425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5041034a0 0x7fb504109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 --2- 172.21.15.105:0/3809904425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb504102ae0 0x7fb504102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 --2- 172.21.15.105:0/3809904425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb5041018f0 0x7fb504101d10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 -- 172.21.15.105:0/3809904425 >> 172.21.15.105:0/3809904425 conn(0x7fb5040fce60 msgr2=0x7fb5040ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 -- 172.21.15.105:0/3809904425 shutdown_connections 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.975+0000 7fb509b2f640 1 -- 172.21.15.105:0/3809904425 wait complete. 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb509b2f640 1 Processor -- start 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb509b2f640 1 -- start start 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb509b2f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5041018f0 0x7fb504197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb509b2f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb504102ae0 0x7fb5041980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb509b2f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5041034a0 0x7fb50419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb509b2f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410d3f0 con 0x7fb504102ae0 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb509b2f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410d270 con 0x7fb5041034a0 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb509b2f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410d570 con 0x7fb5041018f0 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb502ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb504102ae0 0x7fb5041980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb502ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb504102ae0 0x7fb5041980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53010/0 (socket says 172.21.15.105:53010) 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb502ffd640 1 -- 172.21.15.105:0/1250433349 learned_addr learned my addr 172.21.15.105:0/1250433349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb5037fe640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5041018f0 0x7fb504197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.976+0000 7fb503fff640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5041034a0 0x7fb50419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb502ffd640 1 -- 172.21.15.105:0/1250433349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5041018f0 msgr2=0x7fb504197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb502ffd640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5041018f0 0x7fb504197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb502ffd640 1 -- 172.21.15.105:0/1250433349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5041034a0 msgr2=0x7fb50419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb502ffd640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5041034a0 0x7fb50419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb502ffd640 1 -- 172.21.15.105:0/1250433349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50419f790 con 0x7fb504102ae0 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb502ffd640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb504102ae0 0x7fb5041980b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800e9f0 tx=0x7fb4f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb500ff9640 1 -- 172.21.15.105:0/1250433349 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f800cd80 con 0x7fb504102ae0 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50419fa80 con 0x7fb504102ae0 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb500ff9640 1 -- 172.21.15.105:0/1250433349 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4f80108e0 con 0x7fb504102ae0 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb500ff9640 1 -- 172.21.15.105:0/1250433349 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8018920 con 0x7fb504102ae0 2024-03-26T17:38:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.977+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50419ffd0 con 0x7fb504102ae0 2024-03-26T17:38:22.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.978+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb504108d30 con 0x7fb504102ae0 2024-03-26T17:38:22.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.979+0000 7fb500ff9640 1 -- 172.21.15.105:0/1250433349 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fb4f8010a90 con 0x7fb504102ae0 2024-03-26T17:38:22.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.980+0000 7fb500ff9640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4ec0a2fb0 0x7fb4ec0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:22.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.980+0000 7fb5037fe640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4ec0a2fb0 0x7fb4ec0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:22.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.980+0000 7fb500ff9640 1 -- 172.21.15.105:0/1250433349 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fb4f80cfcb0 con 0x7fb504102ae0 2024-03-26T17:38:22.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.980+0000 7fb5037fe640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4ec0a2fb0 0x7fb4ec0a5470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005ab0 tx=0x7fb4f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:22.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:22.983+0000 7fb500ff9640 1 -- 172.21.15.105:0/1250433349 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb4f809c350 con 0x7fb504102ae0 2024-03-26T17:38:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:22 smithi144 ceph-mon[36095]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:22 smithi144 ceph-mon[36095]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-26T17:38:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1528637578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3821171732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:23.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:22 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3804173166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:23.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.164+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb50402d490 con 0x7fb504102ae0 2024-03-26T17:38:23.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.165+0000 7fb500ff9640 1 -- 172.21.15.105:0/1250433349 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fb4f80a03f0 con 0x7fb504102ae0 2024-03-26T17:38:23.157 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:38:23.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4ec0a2fb0 msgr2=0x7fb4ec0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4ec0a2fb0 0x7fb4ec0a5470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4005ab0 tx=0x7fb4f4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb504102ae0 msgr2=0x7fb5041980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb504102ae0 0x7fb5041980b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800e9f0 tx=0x7fb4f800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 shutdown_connections 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4ec0a2fb0 0x7fb4ec0a5470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5041034a0 0x7fb50419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb504102ae0 0x7fb5041980b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 --2- 172.21.15.105:0/1250433349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5041018f0 0x7fb504197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 >> 172.21.15.105:0/1250433349 conn(0x7fb5040fce60 msgr2=0x7fb504105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 shutdown_connections 2024-03-26T17:38:23.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.167+0000 7fb509b2f640 1 -- 172.21.15.105:0/1250433349 wait complete. 2024-03-26T17:38:23.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:22 smithi135 ceph-mon[35636]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T17:38:23.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:22 smithi135 ceph-mon[35636]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-03-26T17:38:23.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1528637578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3821171732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:23.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:22 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3804173166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:23.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T17:38:23.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.284+0000 7fd5271e6640 1 -- 172.21.15.105:0/1060710173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201034a0 msgr2=0x7fd520109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.284+0000 7fd5271e6640 1 --2- 172.21.15.105:0/1060710173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201034a0 0x7fd520109d30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd518009ca0 tx=0x7fd51802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.284+0000 7fd5271e6640 1 -- 172.21.15.105:0/1060710173 shutdown_connections 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.284+0000 7fd5271e6640 1 --2- 172.21.15.105:0/1060710173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5201034a0 0x7fd520109d30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.284+0000 7fd5271e6640 1 --2- 172.21.15.105:0/1060710173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd520102ae0 0x7fd520102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.284+0000 7fd5271e6640 1 --2- 172.21.15.105:0/1060710173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5201018f0 0x7fd520101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.284+0000 7fd5271e6640 1 -- 172.21.15.105:0/1060710173 >> 172.21.15.105:0/1060710173 conn(0x7fd5200fce60 msgr2=0x7fd5200ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 -- 172.21.15.105:0/1060710173 shutdown_connections 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 -- 172.21.15.105:0/1060710173 wait complete. 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 Processor -- start 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 -- start start 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5201018f0 0x7fd520197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520102ae0 0x7fd520197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5201034a0 0x7fd52019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd52010d400 con 0x7fd520102ae0 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd52010d280 con 0x7fd5201018f0 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd5271e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd52010d580 con 0x7fd5201034a0 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd51ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520102ae0 0x7fd520197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd524f82640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5201018f0 0x7fd520197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd51ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520102ae0 0x7fd520197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53032/0 (socket says 172.21.15.105:53032) 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd525783640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5201034a0 0x7fd52019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.285+0000 7fd51ffff640 1 -- 172.21.15.105:0/3631683916 learned_addr learned my addr 172.21.15.105:0/3631683916 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51ffff640 1 -- 172.21.15.105:0/3631683916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5201034a0 msgr2=0x7fd52019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51ffff640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5201034a0 0x7fd52019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51ffff640 1 -- 172.21.15.105:0/3631683916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5201018f0 msgr2=0x7fd520197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51ffff640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5201018f0 0x7fd520197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51ffff640 1 -- 172.21.15.105:0/3631683916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52019f6c0 con 0x7fd520102ae0 2024-03-26T17:38:23.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd524f82640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5201018f0 0x7fd520197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51ffff640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520102ae0 0x7fd520197fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00ea60 tx=0x7fd50c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51dffb640 1 -- 172.21.15.105:0/3631683916 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c00cde0 con 0x7fd520102ae0 2024-03-26T17:38:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51dffb640 1 -- 172.21.15.105:0/3631683916 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd50c0035f0 con 0x7fd520102ae0 2024-03-26T17:38:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52019f9b0 con 0x7fd520102ae0 2024-03-26T17:38:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd51dffb640 1 -- 172.21.15.105:0/3631683916 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c010690 con 0x7fd520102ae0 2024-03-26T17:38:23.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.286+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52019ff00 con 0x7fd520102ae0 2024-03-26T17:38:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.287+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd520108c90 con 0x7fd520102ae0 2024-03-26T17:38:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.289+0000 7fd51dffb640 1 -- 172.21.15.105:0/3631683916 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fd50c0037a0 con 0x7fd520102ae0 2024-03-26T17:38:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.289+0000 7fd51dffb640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5100a2fb0 0x7fd5100a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.290+0000 7fd51dffb640 1 -- 172.21.15.105:0/3631683916 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fd50c0cf7d0 con 0x7fd520102ae0 2024-03-26T17:38:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.290+0000 7fd524f82640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5100a2fb0 0x7fd5100a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.290+0000 7fd524f82640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5100a2fb0 0x7fd5100a5470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd520102940 tx=0x7fd514005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:23.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.294+0000 7fd51dffb640 1 -- 172.21.15.105:0/3631683916 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd50c09c4c0 con 0x7fd520102ae0 2024-03-26T17:38:23.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.478+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd52002d440 con 0x7fd520102ae0 2024-03-26T17:38:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.478+0000 7fd51dffb640 1 -- 172.21.15.105:0/3631683916 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fd50c09bdc0 con 0x7fd520102ae0 2024-03-26T17:38:23.471 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T17:38:23.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.480+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5100a2fb0 msgr2=0x7fd5100a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.480+0000 7fd5271e6640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5100a2fb0 0x7fd5100a5470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd520102940 tx=0x7fd514005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:23.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520102ae0 msgr2=0x7fd520197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520102ae0 0x7fd520197fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd50c00ea60 tx=0x7fd50c00ef20 comp rx=0 tx=0).stop 2024-03-26T17:38:23.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 shutdown_connections 2024-03-26T17:38:23.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5100a2fb0 0x7fd5100a5470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5201034a0 0x7fd52019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd520102ae0 0x7fd520197fe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 --2- 172.21.15.105:0/3631683916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5201018f0 0x7fd520197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 >> 172.21.15.105:0/3631683916 conn(0x7fd5200fce60 msgr2=0x7fd520105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:23.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 shutdown_connections 2024-03-26T17:38:23.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.481+0000 7fd5271e6640 1 -- 172.21.15.105:0/3631683916 wait complete. 2024-03-26T17:38:23.484 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T17:38:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 -- 172.21.15.105:0/745169367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c8102340 msgr2=0x7f81c8104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 --2- 172.21.15.105:0/745169367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c8102340 0x7f81c8104730 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009900 tx=0x7f81b802bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:23.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 -- 172.21.15.105:0/745169367 shutdown_connections 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 --2- 172.21.15.105:0/745169367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c81075a0 0x7f81c81099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 --2- 172.21.15.105:0/745169367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81c8104c70 0x7f81c8107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 --2- 172.21.15.105:0/745169367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c8102340 0x7f81c8104730 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 -- 172.21.15.105:0/745169367 >> 172.21.15.105:0/745169367 conn(0x7f81c80fbd50 msgr2=0x7f81c80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 -- 172.21.15.105:0/745169367 shutdown_connections 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.607+0000 7f81ce568640 1 -- 172.21.15.105:0/745169367 wait complete. 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ce568640 1 Processor -- start 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ce568640 1 -- start start 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ce568640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81c8102340 0x7f81c8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ce568640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c8104c70 0x7f81c81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ce568640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c81075a0 0x7f81c819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ce568640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f81c810d410 con 0x7f81c81075a0 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ce568640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81c810d290 con 0x7f81c8102340 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ce568640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81c810d590 con 0x7f81c8104c70 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ccb05640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c81075a0 0x7f81c819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ccb05640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c81075a0 0x7f81c819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53050/0 (socket says 172.21.15.105:53050) 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.608+0000 7f81ccb05640 1 -- 172.21.15.105:0/4003256202 learned_addr learned my addr 172.21.15.105:0/4003256202 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81c7fff640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81c8102340 0x7f81c8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81c77fe640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c8104c70 0x7f81c81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81ccb05640 1 -- 172.21.15.105:0/4003256202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c8104c70 msgr2=0x7f81c81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81ccb05640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c8104c70 0x7f81c81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81ccb05640 1 -- 172.21.15.105:0/4003256202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81c8102340 msgr2=0x7f81c8197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81ccb05640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81c8102340 0x7f81c8197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81ccb05640 1 -- 172.21.15.105:0/4003256202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c819f7a0 con 0x7f81c81075a0 2024-03-26T17:38:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81ccb05640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c81075a0 0x7f81c819f130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f81c000eb10 tx=0x7f81c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81c57fa640 1 -- 172.21.15.105:0/4003256202 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0004b20 con 0x7f81c81075a0 2024-03-26T17:38:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c819fa90 con 0x7f81c81075a0 2024-03-26T17:38:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81c57fa640 1 -- 172.21.15.105:0/4003256202 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81c0004cc0 con 0x7f81c81075a0 2024-03-26T17:38:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81c57fa640 1 -- 172.21.15.105:0/4003256202 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0010690 con 0x7f81c81075a0 2024-03-26T17:38:23.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.609+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c81a0030 con 0x7f81c81075a0 2024-03-26T17:38:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.610+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c80fd840 con 0x7f81c81075a0 2024-03-26T17:38:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.611+0000 7f81c57fa640 1 -- 172.21.15.105:0/4003256202 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f81c0005020 con 0x7f81c81075a0 2024-03-26T17:38:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.612+0000 7f81c57fa640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81b00a2fb0 0x7f81b00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.612+0000 7f81c7fff640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81b00a2fb0 0x7f81b00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.612+0000 7f81c57fa640 1 -- 172.21.15.105:0/4003256202 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f81c00cfc20 con 0x7f81c81075a0 2024-03-26T17:38:23.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.612+0000 7f81c7fff640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81b00a2fb0 0x7f81b00a5470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009230 tx=0x7f81b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:23.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.616+0000 7f81c57fa640 1 -- 172.21.15.105:0/4003256202 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f81c009c910 con 0x7f81c81075a0 2024-03-26T17:38:23.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.799+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f81c802d3f0 con 0x7f81c81075a0 2024-03-26T17:38:23.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.799+0000 7f81c57fa640 1 -- 172.21.15.105:0/4003256202 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f81c009c210 con 0x7f81c81075a0 2024-03-26T17:38:23.792 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T17:38:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81b00a2fb0 msgr2=0x7f81b00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81b00a2fb0 0x7f81b00a5470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009230 tx=0x7f81b8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c81075a0 msgr2=0x7f81c819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c81075a0 0x7f81c819f130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f81c000eb10 tx=0x7f81c000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 shutdown_connections 2024-03-26T17:38:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81b00a2fb0 0x7f81b00a5470 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81c81075a0 0x7f81c819f130 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81c8104c70 0x7f81c81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 --2- 172.21.15.105:0/4003256202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f81c8102340 0x7f81c8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 >> 172.21.15.105:0/4003256202 conn(0x7f81c80fbd50 msgr2=0x7f81c8102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 shutdown_connections 2024-03-26T17:38:23.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.801+0000 7f81ce568640 1 -- 172.21.15.105:0/4003256202 wait complete. 2024-03-26T17:38:23.804 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:23 smithi105 ceph-mon[32280]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:23.804 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2826853817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:23.804 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1250433349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:23.804 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3631683916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:23.805 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-26T17:38:23.805 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:38:23.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 -- 172.21.15.105:0/998994618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6968102390 msgr2=0x7f6968108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 --2- 172.21.15.105:0/998994618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6968102390 0x7f6968108c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6958009860 tx=0x7f695802bb70 comp rx=0 tx=0).stop 2024-03-26T17:38:23.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 -- 172.21.15.105:0/998994618 shutdown_connections 2024-03-26T17:38:23.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 --2- 172.21.15.105:0/998994618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6968102390 0x7f6968108c20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 --2- 172.21.15.105:0/998994618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69681019d0 0x7f6968101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 --2- 172.21.15.105:0/998994618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69681007e0 0x7f6968100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 -- 172.21.15.105:0/998994618 >> 172.21.15.105:0/998994618 conn(0x7f69680fbd50 msgr2=0x7f69680fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:23.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 -- 172.21.15.105:0/998994618 shutdown_connections 2024-03-26T17:38:23.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.915+0000 7f696d063640 1 -- 172.21.15.105:0/998994618 wait complete. 2024-03-26T17:38:23.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.916+0000 7f696d063640 1 Processor -- start 2024-03-26T17:38:23.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696d063640 1 -- start start 2024-03-26T17:38:23.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696d063640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69681007e0 0x7f69681969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696d063640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69681019d0 0x7f6968196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696d063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6968102390 0x7f696819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696d063640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f696810c310 con 0x7f6968102390 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696d063640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f696810c190 con 0x7f69681019d0 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696d063640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f696810c490 con 0x7f69681007e0 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6968102390 0x7f696819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6968102390 0x7f696819df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53068/0 (socket says 172.21.15.105:53068) 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 -- 172.21.15.105:0/1489888727 learned_addr learned my addr 172.21.15.105:0/1489888727 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696659c640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69681019d0 0x7f6968196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f6966d9d640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69681007e0 0x7f69681969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 -- 172.21.15.105:0/1489888727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69681007e0 msgr2=0x7f69681969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69681007e0 0x7f69681969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 -- 172.21.15.105:0/1489888727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69681019d0 msgr2=0x7f6968196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69681019d0 0x7f6968196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 -- 172.21.15.105:0/1489888727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696819e600 con 0x7f6968102390 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696659c640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69681019d0 0x7f6968196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:23.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f696759e640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6968102390 0x7f696819df90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f69580057c0 tx=0x7f69580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:23.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.917+0000 7f695ffff640 1 -- 172.21.15.105:0/1489888727 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958041020 con 0x7f6968102390 2024-03-26T17:38:23.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.918+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696819e890 con 0x7f6968102390 2024-03-26T17:38:23.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.918+0000 7f695ffff640 1 -- 172.21.15.105:0/1489888727 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69580035f0 con 0x7f6968102390 2024-03-26T17:38:23.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.918+0000 7f695ffff640 1 -- 172.21.15.105:0/1489888727 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958039040 con 0x7f6968102390 2024-03-26T17:38:23.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.918+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696819ed80 con 0x7f6968102390 2024-03-26T17:38:23.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.919+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6968107b50 con 0x7f6968102390 2024-03-26T17:38:23.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.919+0000 7f695ffff640 1 -- 172.21.15.105:0/1489888727 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f6958044020 con 0x7f6968102390 2024-03-26T17:38:23.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.920+0000 7f695ffff640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69540a2fb0 0x7f69540a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:23.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.920+0000 7f6966d9d640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69540a2fb0 0x7f69540a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:23.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.921+0000 7f695ffff640 1 -- 172.21.15.105:0/1489888727 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f69580f0cc0 con 0x7f6968102390 2024-03-26T17:38:23.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.921+0000 7f6966d9d640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69540a2fb0 0x7f69540a5470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6968101830 tx=0x7f6960009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:23.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:23.924+0000 7f695ffff640 1 -- 172.21.15.105:0/1489888727 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f695804b020 con 0x7f6968102390 2024-03-26T17:38:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:23 smithi144 ceph-mon[36095]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2826853817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1250433349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3631683916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:24.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.143+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69680624c0 con 0x7f6968102390 2024-03-26T17:38:24.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.144+0000 7f695ffff640 1 -- 172.21.15.105:0/1489888727 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7f69580bdac0 con 0x7f6968102390 2024-03-26T17:38:24.137 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:24.137 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:38:20.295050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T17:38:24.137 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-03-26T17:38:24.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69540a2fb0 msgr2=0x7f69540a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69540a2fb0 0x7f69540a5470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6968101830 tx=0x7f6960009500 comp rx=0 tx=0).stop 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6968102390 msgr2=0x7f696819df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6968102390 0x7f696819df90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f69580057c0 tx=0x7f69580042f0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 shutdown_connections 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69540a2fb0 0x7f69540a5470 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6968102390 0x7f696819df90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69681019d0 0x7f6968196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 --2- 172.21.15.105:0/1489888727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69681007e0 0x7f69681969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 >> 172.21.15.105:0/1489888727 conn(0x7f69680fbd50 msgr2=0x7f6968104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 shutdown_connections 2024-03-26T17:38:24.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.146+0000 7f696d063640 1 -- 172.21.15.105:0/1489888727 wait complete. 2024-03-26T17:38:24.150 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-26T17:38:24.150 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-26T17:38:24.150 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T17:38:24.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:23 smithi135 ceph-mon[35636]: pgmap v345: 97 pgs: 97 active+clean; 650 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2826853817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1250433349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3631683916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:24.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.264+0000 7f157a9cd640 1 -- 172.21.15.105:0/3077084674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1574104d50 msgr2=0x7f1574107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.264+0000 7f157a9cd640 1 --2- 172.21.15.105:0/3077084674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1574104d50 0x7f1574107140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1568009900 tx=0x7f156802bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:24.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 -- 172.21.15.105:0/3077084674 shutdown_connections 2024-03-26T17:38:24.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 --2- 172.21.15.105:0/3077084674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574107680 0x7f1574109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 --2- 172.21.15.105:0/3077084674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1574104d50 0x7f1574107140 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 --2- 172.21.15.105:0/3077084674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1574102420 0x7f1574104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 -- 172.21.15.105:0/3077084674 >> 172.21.15.105:0/3077084674 conn(0x7f15740fbd50 msgr2=0x7f15740fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 -- 172.21.15.105:0/3077084674 shutdown_connections 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 -- 172.21.15.105:0/3077084674 wait complete. 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 Processor -- start 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 -- start start 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574102420 0x7f1574101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1574104d50 0x7f15741003d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1574107680 0x7f1574100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f157410d440 con 0x7f1574102420 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f157410d2c0 con 0x7f1574104d50 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.265+0000 7f157a9cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f157410d5c0 con 0x7f1574107680 2024-03-26T17:38:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574102420 0x7f1574101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574102420 0x7f1574101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53082/0 (socket says 172.21.15.105:53082) 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 -- 172.21.15.105:0/2116424278 learned_addr learned my addr 172.21.15.105:0/2116424278 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f15737fe640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1574104d50 0x7f15741003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1578f6a640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1574107680 0x7f1574100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 -- 172.21.15.105:0/2116424278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1574107680 msgr2=0x7f1574100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1574107680 0x7f1574100910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 -- 172.21.15.105:0/2116424278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1574104d50 msgr2=0x7f15741003d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1574104d50 0x7f15741003d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 -- 172.21.15.105:0/2116424278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15741011a0 con 0x7f1574102420 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f1573fff640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574102420 0x7f1574101d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f156400e9f0 tx=0x7f156400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f15717fa640 1 -- 172.21.15.105:0/2116424278 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156400ccc0 con 0x7f1574102420 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15741a1c90 con 0x7f1574102420 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f15717fa640 1 -- 172.21.15.105:0/2116424278 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15640108e0 con 0x7f1574102420 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f15717fa640 1 -- 172.21.15.105:0/2116424278 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15640187c0 con 0x7f1574102420 2024-03-26T17:38:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.266+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15741a21b0 con 0x7f1574102420 2024-03-26T17:38:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.267+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15740fd840 con 0x7f1574102420 2024-03-26T17:38:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.269+0000 7f15717fa640 1 -- 172.21.15.105:0/2116424278 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f1564010a90 con 0x7f1574102420 2024-03-26T17:38:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.269+0000 7f15717fa640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f155c0a2fb0 0x7f155c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.269+0000 7f15737fe640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f155c0a2fb0 0x7f155c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.269+0000 7f15717fa640 1 -- 172.21.15.105:0/2116424278 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f15640cfdb0 con 0x7f1574102420 2024-03-26T17:38:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.270+0000 7f15737fe640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f155c0a2fb0 0x7f155c0a5470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f156802c240 tx=0x7f156802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:24.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.273+0000 7f15717fa640 1 -- 172.21.15.105:0/2116424278 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f156409caa0 con 0x7f1574102420 2024-03-26T17:38:24.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.452+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1574061d00 con 0x7f155c0a2fb0 2024-03-26T17:38:24.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.459+0000 7f15717fa640 1 -- 172.21.15.105:0/2116424278 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258895 (secure 0 0 0) 0x7f1574061d00 con 0x7f155c0a2fb0 2024-03-26T17:38:24.455 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-26T17:38:24.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f155c0a2fb0 msgr2=0x7f155c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f155c0a2fb0 0x7f155c0a5470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f156802c240 tx=0x7f156802c7e0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574102420 msgr2=0x7f1574101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574102420 0x7f1574101d90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f156400e9f0 tx=0x7f156400edf0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 shutdown_connections 2024-03-26T17:38:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f155c0a2fb0 0x7f155c0a5470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1574107680 0x7f1574100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1574104d50 0x7f15741003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 --2- 172.21.15.105:0/2116424278 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1574102420 0x7f1574101d90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 >> 172.21.15.105:0/2116424278 conn(0x7f15740fbd50 msgr2=0x7f1574102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 shutdown_connections 2024-03-26T17:38:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.463+0000 7f157a9cd640 1 -- 172.21.15.105:0/2116424278 wait complete. 2024-03-26T17:38:24.467 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-03-26T17:38:22.609221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665528,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1290,"num_read_kb":773,"num_write":438,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366620,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575204,"statfs":{"total":1151940427776,"available":1151565008896,"internally_reserved":0,"allocated":13627392,"data_stored":7443440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6677,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":74,"num_write":12,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002610"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628144+0000","last_change":"2024-03-26T17:36:22.743815+0000","last_active":"2024-03-26T17:37:56.628144+0000","last_peered":"2024-03-26T17:37:56.628144+0000","last_clean":"2024-03-26T17:37:56.628144+0000","last_became_active":"2024-03-26T17:36:22.743446+0000","last_became_peered":"2024-03-26T17:36:22.743446+0000","last_unstale":"2024-03-26T17:37:56.628144+0000","last_undegraded":"2024-03-26T17:37:56.628144+0000","last_fullsized":"2024-03-26T17:37:56.628144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:16:56.773849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628208+0000","last_change":"2024-03-26T17:36:22.740754+0000","last_active":"2024-03-26T17:37:56.628208+0000","last_peered":"2024-03-26T17:37:56.628208+0000","last_clean":"2024-03-26T17:37:56.628208+0000","last_became_active":"2024-03-26T17:36:22.740317+0000","last_became_peered":"2024-03-26T17:36:22.740317+0000","last_unstale":"2024-03-26T17:37:56.628208+0000","last_undegraded":"2024-03-26T17:37:56.628208+0000","last_fullsized":"2024-03-26T17:37:56.628208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:50:45.321653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635496+0000","last_change":"2024-03-26T17:36:22.735079+0000","last_active":"2024-03-26T17:37:56.635496+0000","last_peered":"2024-03-26T17:37:56.635496+0000","last_clean":"2024-03-26T17:37:56.635496+0000","last_became_active":"2024-03-26T17:36:22.734646+0000","last_became_peered":"2024-03-26T17:36:22.734646+0000","last_unstale":"2024-03-26T17:37:56.635496+0000","last_undegraded":"2024-03-26T17:37:56.635496+0000","last_fullsized":"2024-03-26T17:37:56.635496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:41:09.477438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.615398+0000","last_change":"2024-03-26T17:36:22.746170+0000","last_active":"2024-03-26T17:37:56.615398+0000","last_peered":"2024-03-26T17:37:56.615398+0000","last_clean":"2024-03-26T17:37:56.615398+0000","last_became_active":"2024-03-26T17:36:22.744255+0000","last_became_peered":"2024-03-26T17:36:22.744255+0000","last_unstale":"2024-03-26T17:37:56.615398+0000","last_undegraded":"2024-03-26T17:37:56.615398+0000","last_fullsized":"2024-03-26T17:37:56.615398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:54:36.225926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.607889+0000","last_change":"2024-03-26T17:36:22.737390+0000","last_active":"2024-03-26T17:37:56.607889+0000","last_peered":"2024-03-26T17:37:56.607889+0000","last_clean":"2024-03-26T17:37:56.607889+0000","last_became_active":"2024-03-26T17:36:22.737038+0000","last_became_peered":"2024-03-26T17:36:22.737038+0000","last_unstale":"2024-03-26T17:37:56.607889+0000","last_undegraded":"2024-03-26T17:37:56.607889+0000","last_fullsized":"2024-03-26T17:37:56.607889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:48.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.634748+0000","last_change":"2024-03-26T17:36:22.748097+0000","last_active":"2024-03-26T17:37:56.634748+0000","last_peered":"2024-03-26T17:37:56.634748+0000","last_clean":"2024-03-26T17:37:56.634748+0000","last_became_active":"2024-03-26T17:36:22.747730+0000","last_became_peered":"2024-03-26T17:36:22.747730+0000","last_unstale":"2024-03-26T17:37:56.634748+0000","last_undegraded":"2024-03-26T17:37:56.634748+0000","last_fullsized":"2024-03-26T17:37:56.634748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:43:01.370199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.620104+0000","last_change":"2024-03-26T17:36:22.739587+0000","last_active":"2024-03-26T17:37:56.620104+0000","last_peered":"2024-03-26T17:37:56.620104+0000","last_clean":"2024-03-26T17:37:56.620104+0000","last_became_active":"2024-03-26T17:36:22.735787+0000","last_became_peered":"2024-03-26T17:36:22.735787+0000","last_unstale":"2024-03-26T17:37:56.620104+0000","last_undegraded":"2024-03-26T17:37:56.620104+0000","last_fullsized":"2024-03-26T17:37:56.620104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:41.381153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621279+0000","last_change":"2024-03-26T17:36:22.736716+0000","last_active":"2024-03-26T17:37:56.621279+0000","last_peered":"2024-03-26T17:37:56.621279+0000","last_clean":"2024-03-26T17:37:56.621279+0000","last_became_active":"2024-03-26T17:36:22.736327+0000","last_became_peered":"2024-03-26T17:36:22.736327+0000","last_unstale":"2024-03-26T17:37:56.621279+0000","last_undegraded":"2024-03-26T17:37:56.621279+0000","last_fullsized":"2024-03-26T17:37:56.621279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:10:43.439524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635164+0000","last_change":"2024-03-26T17:36:22.746355+0000","last_active":"2024-03-26T17:37:56.635164+0000","last_peered":"2024-03-26T17:37:56.635164+0000","last_clean":"2024-03-26T17:37:56.635164+0000","last_became_active":"2024-03-26T17:36:22.745363+0000","last_became_peered":"2024-03-26T17:36:22.745363+0000","last_unstale":"2024-03-26T17:37:56.635164+0000","last_undegraded":"2024-03-26T17:37:56.635164+0000","last_fullsized":"2024-03-26T17:37:56.635164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:56:33.528005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.634951+0000","last_change":"2024-03-26T17:36:22.749009+0000","last_active":"2024-03-26T17:37:56.634951+0000","last_peered":"2024-03-26T17:37:56.634951+0000","last_clean":"2024-03-26T17:37:56.634951+0000","last_became_active":"2024-03-26T17:36:22.748627+0000","last_became_peered":"2024-03-26T17:36:22.748627+0000","last_unstale":"2024-03-26T17:37:56.634951+0000","last_undegraded":"2024-03-26T17:37:56.634951+0000","last_fullsized":"2024-03-26T17:37:56.634951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:59:14.812316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631878+0000","last_change":"2024-03-26T17:36:22.741294+0000","last_active":"2024-03-26T17:37:56.631878+0000","last_peered":"2024-03-26T17:37:56.631878+0000","last_clean":"2024-03-26T17:37:56.631878+0000","last_became_active":"2024-03-26T17:36:22.740003+0000","last_became_peered":"2024-03-26T17:36:22.740003+0000","last_unstale":"2024-03-26T17:37:56.631878+0000","last_undegraded":"2024-03-26T17:37:56.631878+0000","last_fullsized":"2024-03-26T17:37:56.631878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:06.305195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629278+0000","last_change":"2024-03-26T17:36:22.745884+0000","last_active":"2024-03-26T17:37:56.629278+0000","last_peered":"2024-03-26T17:37:56.629278+0000","last_clean":"2024-03-26T17:37:56.629278+0000","last_became_active":"2024-03-26T17:36:22.744168+0000","last_became_peered":"2024-03-26T17:36:22.744168+0000","last_unstale":"2024-03-26T17:37:56.629278+0000","last_undegraded":"2024-03-26T17:37:56.629278+0000","last_fullsized":"2024-03-26T17:37:56.629278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:50.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624550+0000","last_change":"2024-03-26T17:36:22.725956+0000","last_active":"2024-03-26T17:37:56.624550+0000","last_peered":"2024-03-26T17:37:56.624550+0000","last_clean":"2024-03-26T17:37:56.624550+0000","last_became_active":"2024-03-26T17:36:22.725513+0000","last_became_peered":"2024-03-26T17:36:22.725513+0000","last_unstale":"2024-03-26T17:37:56.624550+0000","last_undegraded":"2024-03-26T17:37:56.624550+0000","last_fullsized":"2024-03-26T17:37:56.624550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:30:10.914233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628784+0000","last_change":"2024-03-26T17:36:22.739757+0000","last_active":"2024-03-26T17:37:56.628784+0000","last_peered":"2024-03-26T17:37:56.628784+0000","last_clean":"2024-03-26T17:37:56.628784+0000","last_became_active":"2024-03-26T17:36:22.739342+0000","last_became_peered":"2024-03-26T17:36:22.739342+0000","last_unstale":"2024-03-26T17:37:56.628784+0000","last_undegraded":"2024-03-26T17:37:56.628784+0000","last_fullsized":"2024-03-26T17:37:56.628784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:28.171325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.618872+0000","last_change":"2024-03-26T17:36:22.742988+0000","last_active":"2024-03-26T17:37:56.618872+0000","last_peered":"2024-03-26T17:37:56.618872+0000","last_clean":"2024-03-26T17:37:56.618872+0000","last_became_active":"2024-03-26T17:36:22.742639+0000","last_became_peered":"2024-03-26T17:36:22.742639+0000","last_unstale":"2024-03-26T17:37:56.618872+0000","last_undegraded":"2024-03-26T17:37:56.618872+0000","last_fullsized":"2024-03-26T17:37:56.618872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:48:44.204889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635451+0000","last_change":"2024-03-26T17:36:22.744530+0000","last_active":"2024-03-26T17:37:56.635451+0000","last_peered":"2024-03-26T17:37:56.635451+0000","last_clean":"2024-03-26T17:37:56.635451+0000","last_became_active":"2024-03-26T17:36:22.743566+0000","last_became_peered":"2024-03-26T17:36:22.743566+0000","last_unstale":"2024-03-26T17:37:56.635451+0000","last_undegraded":"2024-03-26T17:37:56.635451+0000","last_fullsized":"2024-03-26T17:37:56.635451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:42.685326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629288+0000","last_change":"2024-03-26T17:36:22.745797+0000","last_active":"2024-03-26T17:37:56.629288+0000","last_peered":"2024-03-26T17:37:56.629288+0000","last_clean":"2024-03-26T17:37:56.629288+0000","last_became_active":"2024-03-26T17:36:22.745448+0000","last_became_peered":"2024-03-26T17:36:22.745448+0000","last_unstale":"2024-03-26T17:37:56.629288+0000","last_undegraded":"2024-03-26T17:37:56.629288+0000","last_fullsized":"2024-03-26T17:37:56.629288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:05:29.126564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631670+0000","last_change":"2024-03-26T17:36:22.749890+0000","last_active":"2024-03-26T17:37:56.631670+0000","last_peered":"2024-03-26T17:37:56.631670+0000","last_clean":"2024-03-26T17:37:56.631670+0000","last_became_active":"2024-03-26T17:36:22.749497+0000","last_became_peered":"2024-03-26T17:36:22.749497+0000","last_unstale":"2024-03-26T17:37:56.631670+0000","last_undegraded":"2024-03-26T17:37:56.631670+0000","last_fullsized":"2024-03-26T17:37:56.631670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:12:55.069659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.612760+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:56.612760+0000","last_peered":"2024-03-26T17:37:56.612760+0000","last_clean":"2024-03-26T17:37:56.612760+0000","last_became_active":"2024-03-26T17:36:22.740287+0000","last_became_peered":"2024-03-26T17:36:22.740287+0000","last_unstale":"2024-03-26T17:37:56.612760+0000","last_undegraded":"2024-03-26T17:37:56.612760+0000","last_fullsized":"2024-03-26T17:37:56.612760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:13:00.574246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:16.425934+0000","last_change":"2024-03-26T17:36:21.720809+0000","last_active":"2024-03-26T17:38:16.425934+0000","last_peered":"2024-03-26T17:38:16.425934+0000","last_clean":"2024-03-26T17:38:16.425934+0000","last_became_active":"2024-03-26T17:36:21.720526+0000","last_became_peered":"2024-03-26T17:36:21.720526+0000","last_unstale":"2024-03-26T17:38:16.425934+0000","last_undegraded":"2024-03-26T17:38:16.425934+0000","last_fullsized":"2024-03-26T17:38:16.425934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:48.034328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.607272+0000","last_change":"2024-03-26T17:36:22.746245+0000","last_active":"2024-03-26T17:37:56.607272+0000","last_peered":"2024-03-26T17:37:56.607272+0000","last_clean":"2024-03-26T17:37:56.607272+0000","last_became_active":"2024-03-26T17:36:22.745859+0000","last_became_peered":"2024-03-26T17:36:22.745859+0000","last_unstale":"2024-03-26T17:37:56.607272+0000","last_undegraded":"2024-03-26T17:37:56.607272+0000","last_fullsized":"2024-03-26T17:37:56.607272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:39.085668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741782+0000","last_change":"2024-03-26T17:36:21.723178+0000","last_active":"2024-03-26T17:37:27.741782+0000","last_peered":"2024-03-26T17:37:27.741782+0000","last_clean":"2024-03-26T17:37:27.741782+0000","last_became_active":"2024-03-26T17:36:21.722858+0000","last_became_peered":"2024-03-26T17:36:21.722858+0000","last_unstale":"2024-03-26T17:37:27.741782+0000","last_undegraded":"2024-03-26T17:37:27.741782+0000","last_fullsized":"2024-03-26T17:37:27.741782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:46:13.837672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635805+0000","last_change":"2024-03-26T17:36:22.736231+0000","last_active":"2024-03-26T17:37:56.635805+0000","last_peered":"2024-03-26T17:37:56.635805+0000","last_clean":"2024-03-26T17:37:56.635805+0000","last_became_active":"2024-03-26T17:36:22.735942+0000","last_became_peered":"2024-03-26T17:36:22.735942+0000","last_unstale":"2024-03-26T17:37:56.635805+0000","last_undegraded":"2024-03-26T17:37:56.635805+0000","last_fullsized":"2024-03-26T17:37:56.635805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:49.127382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740597+0000","last_change":"2024-03-26T17:36:21.731369+0000","last_active":"2024-03-26T17:37:27.740597+0000","last_peered":"2024-03-26T17:37:27.740597+0000","last_clean":"2024-03-26T17:37:27.740597+0000","last_became_active":"2024-03-26T17:36:21.731016+0000","last_became_peered":"2024-03-26T17:36:21.731016+0000","last_unstale":"2024-03-26T17:37:27.740597+0000","last_undegraded":"2024-03-26T17:37:27.740597+0000","last_fullsized":"2024-03-26T17:37:27.740597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:37:53.204825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635677+0000","last_change":"2024-03-26T17:36:22.751155+0000","last_active":"2024-03-26T17:37:56.635677+0000","last_peered":"2024-03-26T17:37:56.635677+0000","last_clean":"2024-03-26T17:37:56.635677+0000","last_became_active":"2024-03-26T17:36:22.750704+0000","last_became_peered":"2024-03-26T17:36:22.750704+0000","last_unstale":"2024-03-26T17:37:56.635677+0000","last_undegraded":"2024-03-26T17:37:56.635677+0000","last_fullsized":"2024-03-26T17:37:56.635677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:52:24.116923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739808+0000","last_change":"2024-03-26T17:36:21.726380+0000","last_active":"2024-03-26T17:37:27.739808+0000","last_peered":"2024-03-26T17:37:27.739808+0000","last_clean":"2024-03-26T17:37:27.739808+0000","last_became_active":"2024-03-26T17:36:21.726004+0000","last_became_peered":"2024-03-26T17:36:21.726004+0000","last_unstale":"2024-03-26T17:37:27.739808+0000","last_undegraded":"2024-03-26T17:37:27.739808+0000","last_fullsized":"2024-03-26T17:37:27.739808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:51:23.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.630425+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:56.630425+0000","last_peered":"2024-03-26T17:37:56.630425+0000","last_clean":"2024-03-26T17:37:56.630425+0000","last_became_active":"2024-03-26T17:36:22.741104+0000","last_became_peered":"2024-03-26T17:36:22.741104+0000","last_unstale":"2024-03-26T17:37:56.630425+0000","last_undegraded":"2024-03-26T17:37:56.630425+0000","last_fullsized":"2024-03-26T17:37:56.630425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:53.683944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:18.423824+0000","last_change":"2024-03-26T17:36:21.728824+0000","last_active":"2024-03-26T17:38:18.423824+0000","last_peered":"2024-03-26T17:38:18.423824+0000","last_clean":"2024-03-26T17:38:18.423824+0000","last_became_active":"2024-03-26T17:36:21.728467+0000","last_became_peered":"2024-03-26T17:36:21.728467+0000","last_unstale":"2024-03-26T17:38:18.423824+0000","last_undegraded":"2024-03-26T17:38:18.423824+0000","last_fullsized":"2024-03-26T17:38:18.423824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:25:07.842111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617769+0000","last_change":"2024-03-26T17:36:22.740131+0000","last_active":"2024-03-26T17:37:56.617769+0000","last_peered":"2024-03-26T17:37:56.617769+0000","last_clean":"2024-03-26T17:37:56.617769+0000","last_became_active":"2024-03-26T17:36:22.739704+0000","last_became_peered":"2024-03-26T17:36:22.739704+0000","last_unstale":"2024-03-26T17:37:56.617769+0000","last_undegraded":"2024-03-26T17:37:56.617769+0000","last_fullsized":"2024-03-26T17:37:56.617769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:47.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245391+0000","last_change":"2024-03-26T17:36:21.731826+0000","last_active":"2024-03-26T17:37:28.245391+0000","last_peered":"2024-03-26T17:37:28.245391+0000","last_clean":"2024-03-26T17:37:28.245391+0000","last_became_active":"2024-03-26T17:36:21.731546+0000","last_became_peered":"2024-03-26T17:36:21.731546+0000","last_unstale":"2024-03-26T17:37:28.245391+0000","last_undegraded":"2024-03-26T17:37:28.245391+0000","last_fullsized":"2024-03-26T17:37:28.245391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:08:53.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.614608+0000","last_change":"2024-03-26T17:36:22.745170+0000","last_active":"2024-03-26T17:37:56.614608+0000","last_peered":"2024-03-26T17:37:56.614608+0000","last_clean":"2024-03-26T17:37:56.614608+0000","last_became_active":"2024-03-26T17:36:22.744798+0000","last_became_peered":"2024-03-26T17:36:22.744798+0000","last_unstale":"2024-03-26T17:37:56.614608+0000","last_undegraded":"2024-03-26T17:37:56.614608+0000","last_fullsized":"2024-03-26T17:37:56.614608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:32:50.418149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739722+0000","last_change":"2024-03-26T17:36:21.721492+0000","last_active":"2024-03-26T17:37:27.739722+0000","last_peered":"2024-03-26T17:37:27.739722+0000","last_clean":"2024-03-26T17:37:27.739722+0000","last_became_active":"2024-03-26T17:36:21.721194+0000","last_became_peered":"2024-03-26T17:36:21.721194+0000","last_unstale":"2024-03-26T17:37:27.739722+0000","last_undegraded":"2024-03-26T17:37:27.739722+0000","last_fullsized":"2024-03-26T17:37:27.739722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:50.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624868+0000","last_change":"2024-03-26T17:36:22.740780+0000","last_active":"2024-03-26T17:37:56.624868+0000","last_peered":"2024-03-26T17:37:56.624868+0000","last_clean":"2024-03-26T17:37:56.624868+0000","last_became_active":"2024-03-26T17:36:22.740311+0000","last_became_peered":"2024-03-26T17:36:22.740311+0000","last_unstale":"2024-03-26T17:37:56.624868+0000","last_undegraded":"2024-03-26T17:37:56.624868+0000","last_fullsized":"2024-03-26T17:37:56.624868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:01:56.018416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742536+0000","last_change":"2024-03-26T17:36:21.726289+0000","last_active":"2024-03-26T17:37:27.742536+0000","last_peered":"2024-03-26T17:37:27.742536+0000","last_clean":"2024-03-26T17:37:27.742536+0000","last_became_active":"2024-03-26T17:36:21.725898+0000","last_became_peered":"2024-03-26T17:36:21.725898+0000","last_unstale":"2024-03-26T17:37:27.742536+0000","last_undegraded":"2024-03-26T17:37:27.742536+0000","last_fullsized":"2024-03-26T17:37:27.742536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:03.893163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631799+0000","last_change":"2024-03-26T17:36:22.750632+0000","last_active":"2024-03-26T17:37:56.631799+0000","last_peered":"2024-03-26T17:37:56.631799+0000","last_clean":"2024-03-26T17:37:56.631799+0000","last_became_active":"2024-03-26T17:36:22.750297+0000","last_became_peered":"2024-03-26T17:36:22.750297+0000","last_unstale":"2024-03-26T17:37:56.631799+0000","last_undegraded":"2024-03-26T17:37:56.631799+0000","last_fullsized":"2024-03-26T17:37:56.631799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:43:16.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":254,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.426474+0000","last_change":"2024-03-26T17:36:21.721403+0000","last_active":"2024-03-26T17:38:21.426474+0000","last_peered":"2024-03-26T17:38:21.426474+0000","last_clean":"2024-03-26T17:38:21.426474+0000","last_became_active":"2024-03-26T17:36:21.720962+0000","last_became_peered":"2024-03-26T17:36:21.720962+0000","last_unstale":"2024-03-26T17:38:21.426474+0000","last_undegraded":"2024-03-26T17:38:21.426474+0000","last_fullsized":"2024-03-26T17:38:21.426474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:02.683188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635445+0000","last_change":"2024-03-26T17:36:22.754010+0000","last_active":"2024-03-26T17:37:56.635445+0000","last_peered":"2024-03-26T17:37:56.635445+0000","last_clean":"2024-03-26T17:37:56.635445+0000","last_became_active":"2024-03-26T17:36:22.753534+0000","last_became_peered":"2024-03-26T17:36:22.753534+0000","last_unstale":"2024-03-26T17:37:56.635445+0000","last_undegraded":"2024-03-26T17:37:56.635445+0000","last_fullsized":"2024-03-26T17:37:56.635445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:06.490733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736915+0000","last_change":"2024-03-26T17:36:21.721685+0000","last_active":"2024-03-26T17:37:27.736915+0000","last_peered":"2024-03-26T17:37:27.736915+0000","last_clean":"2024-03-26T17:37:27.736915+0000","last_became_active":"2024-03-26T17:36:21.721387+0000","last_became_peered":"2024-03-26T17:36:21.721387+0000","last_unstale":"2024-03-26T17:37:27.736915+0000","last_undegraded":"2024-03-26T17:37:27.736915+0000","last_fullsized":"2024-03-26T17:37:27.736915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:59.493172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626882+0000","last_change":"2024-03-26T17:37:27.740779+0000","last_active":"2024-03-26T17:37:56.626882+0000","last_peered":"2024-03-26T17:37:56.626882+0000","last_clean":"2024-03-26T17:37:56.626882+0000","last_became_active":"2024-03-26T17:37:27.740407+0000","last_became_peered":"2024-03-26T17:37:27.740407+0000","last_unstale":"2024-03-26T17:37:56.626882+0000","last_undegraded":"2024-03-26T17:37:56.626882+0000","last_fullsized":"2024-03-26T17:37:56.626882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:39.526570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744477+0000","last_change":"2024-03-26T17:36:21.724041+0000","last_active":"2024-03-26T17:37:27.744477+0000","last_peered":"2024-03-26T17:37:27.744477+0000","last_clean":"2024-03-26T17:37:27.744477+0000","last_became_active":"2024-03-26T17:36:21.723624+0000","last_became_peered":"2024-03-26T17:36:21.723624+0000","last_unstale":"2024-03-26T17:37:27.744477+0000","last_undegraded":"2024-03-26T17:37:27.744477+0000","last_fullsized":"2024-03-26T17:37:27.744477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:42:24.264844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.630472+0000","last_change":"2024-03-26T17:36:22.738592+0000","last_active":"2024-03-26T17:37:56.630472+0000","last_peered":"2024-03-26T17:37:56.630472+0000","last_clean":"2024-03-26T17:37:56.630472+0000","last_became_active":"2024-03-26T17:36:22.738236+0000","last_became_peered":"2024-03-26T17:36:22.738236+0000","last_unstale":"2024-03-26T17:37:56.630472+0000","last_undegraded":"2024-03-26T17:37:56.630472+0000","last_fullsized":"2024-03-26T17:37:56.630472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:56:06.977720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":294,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.426418+0000","last_change":"2024-03-26T17:36:21.722793+0000","last_active":"2024-03-26T17:38:21.426418+0000","last_peered":"2024-03-26T17:38:21.426418+0000","last_clean":"2024-03-26T17:38:21.426418+0000","last_became_active":"2024-03-26T17:36:21.722482+0000","last_became_peered":"2024-03-26T17:36:21.722482+0000","last_unstale":"2024-03-26T17:38:21.426418+0000","last_undegraded":"2024-03-26T17:38:21.426418+0000","last_fullsized":"2024-03-26T17:38:21.426418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:48:32.107017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66493,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":185,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.607628+0000","last_change":"2024-03-26T17:36:22.746488+0000","last_active":"2024-03-26T17:37:56.607628+0000","last_peered":"2024-03-26T17:37:56.607628+0000","last_clean":"2024-03-26T17:37:56.607628+0000","last_became_active":"2024-03-26T17:36:22.746013+0000","last_became_peered":"2024-03-26T17:36:22.746013+0000","last_unstale":"2024-03-26T17:37:56.607628+0000","last_undegraded":"2024-03-26T17:37:56.607628+0000","last_fullsized":"2024-03-26T17:37:56.607628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:14.792336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742843+0000","last_change":"2024-03-26T17:36:21.724319+0000","last_active":"2024-03-26T17:37:27.742843+0000","last_peered":"2024-03-26T17:37:27.742843+0000","last_clean":"2024-03-26T17:37:27.742843+0000","last_became_active":"2024-03-26T17:36:21.723644+0000","last_became_peered":"2024-03-26T17:36:21.723644+0000","last_unstale":"2024-03-26T17:37:27.742843+0000","last_undegraded":"2024-03-26T17:37:27.742843+0000","last_fullsized":"2024-03-26T17:37:27.742843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:31:01.132797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.625070+0000","last_change":"2024-03-26T17:36:22.736535+0000","last_active":"2024-03-26T17:37:56.625070+0000","last_peered":"2024-03-26T17:37:56.625070+0000","last_clean":"2024-03-26T17:37:56.625070+0000","last_became_active":"2024-03-26T17:36:22.736229+0000","last_became_peered":"2024-03-26T17:36:22.736229+0000","last_unstale":"2024-03-26T17:37:56.625070+0000","last_undegraded":"2024-03-26T17:37:56.625070+0000","last_fullsized":"2024-03-26T17:37:56.625070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:03.938180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742451+0000","last_change":"2024-03-26T17:36:21.726271+0000","last_active":"2024-03-26T17:37:27.742451+0000","last_peered":"2024-03-26T17:37:27.742451+0000","last_clean":"2024-03-26T17:37:27.742451+0000","last_became_active":"2024-03-26T17:36:21.725942+0000","last_became_peered":"2024-03-26T17:36:21.725942+0000","last_unstale":"2024-03-26T17:37:27.742451+0000","last_undegraded":"2024-03-26T17:37:27.742451+0000","last_fullsized":"2024-03-26T17:37:27.742451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:23:16.320106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.633769+0000","last_change":"2024-03-26T17:36:22.743480+0000","last_active":"2024-03-26T17:37:56.633769+0000","last_peered":"2024-03-26T17:37:56.633769+0000","last_clean":"2024-03-26T17:37:56.633769+0000","last_became_active":"2024-03-26T17:36:22.742982+0000","last_became_peered":"2024-03-26T17:36:22.742982+0000","last_unstale":"2024-03-26T17:37:56.633769+0000","last_undegraded":"2024-03-26T17:37:56.633769+0000","last_fullsized":"2024-03-26T17:37:56.633769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:47.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:18.424299+0000","last_change":"2024-03-26T17:36:21.723407+0000","last_active":"2024-03-26T17:38:18.424299+0000","last_peered":"2024-03-26T17:38:18.424299+0000","last_clean":"2024-03-26T17:38:18.424299+0000","last_became_active":"2024-03-26T17:36:21.723086+0000","last_became_peered":"2024-03-26T17:36:21.723086+0000","last_unstale":"2024-03-26T17:38:18.424299+0000","last_undegraded":"2024-03-26T17:38:18.424299+0000","last_fullsized":"2024-03-26T17:38:18.424299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:26:57.886388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631858+0000","last_change":"2024-03-26T17:36:22.740705+0000","last_active":"2024-03-26T17:37:56.631858+0000","last_peered":"2024-03-26T17:37:56.631858+0000","last_clean":"2024-03-26T17:37:56.631858+0000","last_became_active":"2024-03-26T17:36:22.740360+0000","last_became_peered":"2024-03-26T17:36:22.740360+0000","last_unstale":"2024-03-26T17:37:56.631858+0000","last_undegraded":"2024-03-26T17:37:56.631858+0000","last_fullsized":"2024-03-26T17:37:56.631858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:02.879022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.266750+0000","last_change":"2024-03-26T17:37:28.266399+0000","last_active":"2024-03-26T17:37:28.266750+0000","last_peered":"2024-03-26T17:37:28.266750+0000","last_clean":"2024-03-26T17:37:28.266750+0000","last_became_active":"2024-03-26T17:37:28.246359+0000","last_became_peered":"2024-03-26T17:37:28.246359+0000","last_unstale":"2024-03-26T17:37:28.266750+0000","last_undegraded":"2024-03-26T17:37:28.266750+0000","last_fullsized":"2024-03-26T17:37:28.266750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:20:54.895081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.625211+0000","last_change":"2024-03-26T17:36:22.743791+0000","last_active":"2024-03-26T17:37:56.625211+0000","last_peered":"2024-03-26T17:37:56.625211+0000","last_clean":"2024-03-26T17:37:56.625211+0000","last_became_active":"2024-03-26T17:36:22.742780+0000","last_became_peered":"2024-03-26T17:36:22.742780+0000","last_unstale":"2024-03-26T17:37:56.625211+0000","last_undegraded":"2024-03-26T17:37:56.625211+0000","last_fullsized":"2024-03-26T17:37:56.625211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:04.387958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740570+0000","last_change":"2024-03-26T17:36:21.724289+0000","last_active":"2024-03-26T17:37:27.740570+0000","last_peered":"2024-03-26T17:37:27.740570+0000","last_clean":"2024-03-26T17:37:27.740570+0000","last_became_active":"2024-03-26T17:36:21.723847+0000","last_became_peered":"2024-03-26T17:36:21.723847+0000","last_unstale":"2024-03-26T17:37:27.740570+0000","last_undegraded":"2024-03-26T17:37:27.740570+0000","last_fullsized":"2024-03-26T17:37:27.740570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:01:59.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626382+0000","last_change":"2024-03-26T17:36:22.748323+0000","last_active":"2024-03-26T17:37:56.626382+0000","last_peered":"2024-03-26T17:37:56.626382+0000","last_clean":"2024-03-26T17:37:56.626382+0000","last_became_active":"2024-03-26T17:36:22.747970+0000","last_became_peered":"2024-03-26T17:36:22.747970+0000","last_unstale":"2024-03-26T17:37:56.626382+0000","last_undegraded":"2024-03-26T17:37:56.626382+0000","last_fullsized":"2024-03-26T17:37:56.626382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:58.886425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739292+0000","last_change":"2024-03-26T17:36:21.726076+0000","last_active":"2024-03-26T17:37:27.739292+0000","last_peered":"2024-03-26T17:37:27.739292+0000","last_clean":"2024-03-26T17:37:27.739292+0000","last_became_active":"2024-03-26T17:36:21.722067+0000","last_became_peered":"2024-03-26T17:36:21.722067+0000","last_unstale":"2024-03-26T17:37:27.739292+0000","last_undegraded":"2024-03-26T17:37:27.739292+0000","last_fullsized":"2024-03-26T17:37:27.739292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:34:40.316110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628515+0000","last_change":"2024-03-26T17:36:22.730890+0000","last_active":"2024-03-26T17:37:56.628515+0000","last_peered":"2024-03-26T17:37:56.628515+0000","last_clean":"2024-03-26T17:37:56.628515+0000","last_became_active":"2024-03-26T17:36:22.729635+0000","last_became_peered":"2024-03-26T17:36:22.729635+0000","last_unstale":"2024-03-26T17:37:56.628515+0000","last_undegraded":"2024-03-26T17:37:56.628515+0000","last_fullsized":"2024-03-26T17:37:56.628515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:15:09.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":134,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:16.426337+0000","last_change":"2024-03-26T17:36:21.727587+0000","last_active":"2024-03-26T17:38:16.426337+0000","last_peered":"2024-03-26T17:38:16.426337+0000","last_clean":"2024-03-26T17:38:16.426337+0000","last_became_active":"2024-03-26T17:36:21.727302+0000","last_became_peered":"2024-03-26T17:36:21.727302+0000","last_unstale":"2024-03-26T17:38:16.426337+0000","last_undegraded":"2024-03-26T17:38:16.426337+0000","last_fullsized":"2024-03-26T17:38:16.426337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:59.404567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628589+0000","last_change":"2024-03-26T17:36:22.750654+0000","last_active":"2024-03-26T17:37:56.628589+0000","last_peered":"2024-03-26T17:37:56.628589+0000","last_clean":"2024-03-26T17:37:56.628589+0000","last_became_active":"2024-03-26T17:36:22.750281+0000","last_became_peered":"2024-03-26T17:36:22.750281+0000","last_unstale":"2024-03-26T17:37:56.628589+0000","last_undegraded":"2024-03-26T17:37:56.628589+0000","last_fullsized":"2024-03-26T17:37:56.628589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:04.034108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.619451+0000","last_change":"2024-03-26T17:37:27.745100+0000","last_active":"2024-03-26T17:37:56.619451+0000","last_peered":"2024-03-26T17:37:56.619451+0000","last_clean":"2024-03-26T17:37:56.619451+0000","last_became_active":"2024-03-26T17:37:27.744686+0000","last_became_peered":"2024-03-26T17:37:27.744686+0000","last_unstale":"2024-03-26T17:37:56.619451+0000","last_undegraded":"2024-03-26T17:37:56.619451+0000","last_fullsized":"2024-03-26T17:37:56.619451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:38.111145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":135,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:16.420783+0000","last_change":"2024-03-26T17:36:21.730812+0000","last_active":"2024-03-26T17:38:16.420783+0000","last_peered":"2024-03-26T17:38:16.420783+0000","last_clean":"2024-03-26T17:38:16.420783+0000","last_became_active":"2024-03-26T17:36:21.730499+0000","last_became_peered":"2024-03-26T17:36:21.730499+0000","last_unstale":"2024-03-26T17:38:16.420783+0000","last_undegraded":"2024-03-26T17:38:16.420783+0000","last_fullsized":"2024-03-26T17:38:16.420783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:05.666146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617217+0000","last_change":"2024-03-26T17:36:22.742344+0000","last_active":"2024-03-26T17:37:56.617217+0000","last_peered":"2024-03-26T17:37:56.617217+0000","last_clean":"2024-03-26T17:37:56.617217+0000","last_became_active":"2024-03-26T17:36:22.741864+0000","last_became_peered":"2024-03-26T17:36:22.741864+0000","last_unstale":"2024-03-26T17:37:56.617217+0000","last_undegraded":"2024-03-26T17:37:56.617217+0000","last_fullsized":"2024-03-26T17:37:56.617217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:53:53.553107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244456+0000","last_change":"2024-03-26T17:36:21.722450+0000","last_active":"2024-03-26T17:37:28.244456+0000","last_peered":"2024-03-26T17:37:28.244456+0000","last_clean":"2024-03-26T17:37:28.244456+0000","last_became_active":"2024-03-26T17:36:21.722105+0000","last_became_peered":"2024-03-26T17:36:21.722105+0000","last_unstale":"2024-03-26T17:37:28.244456+0000","last_undegraded":"2024-03-26T17:37:28.244456+0000","last_fullsized":"2024-03-26T17:37:28.244456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:01:22.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736685+0000","last_change":"2024-03-26T17:35:10.663606+0000","last_active":"2024-03-26T17:37:27.736685+0000","last_peered":"2024-03-26T17:37:27.736685+0000","last_clean":"2024-03-26T17:37:27.736685+0000","last_became_active":"2024-03-26T17:35:10.663069+0000","last_became_peered":"2024-03-26T17:35:10.663069+0000","last_unstale":"2024-03-26T17:37:27.736685+0000","last_undegraded":"2024-03-26T17:37:27.736685+0000","last_fullsized":"2024-03-26T17:37:27.736685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_clean_scrub_stamp":"2024-03-26T17:31:50.880228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:50.817007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.622092+0000","last_change":"2024-03-26T17:37:27.745201+0000","last_active":"2024-03-26T17:37:56.622092+0000","last_peered":"2024-03-26T17:37:56.622092+0000","last_clean":"2024-03-26T17:37:56.622092+0000","last_became_active":"2024-03-26T17:37:27.744806+0000","last_became_peered":"2024-03-26T17:37:27.744806+0000","last_unstale":"2024-03-26T17:37:56.622092+0000","last_undegraded":"2024-03-26T17:37:56.622092+0000","last_fullsized":"2024-03-26T17:37:56.622092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:15:07.485013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736749+0000","last_change":"2024-03-26T17:36:21.723452+0000","last_active":"2024-03-26T17:37:27.736749+0000","last_peered":"2024-03-26T17:37:27.736749+0000","last_clean":"2024-03-26T17:37:27.736749+0000","last_became_active":"2024-03-26T17:36:21.723155+0000","last_became_peered":"2024-03-26T17:36:21.723155+0000","last_unstale":"2024-03-26T17:37:27.736749+0000","last_undegraded":"2024-03-26T17:37:27.736749+0000","last_fullsized":"2024-03-26T17:37:27.736749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:39:34.443624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.750092+0000","last_change":"2024-03-26T17:37:27.749741+0000","last_active":"2024-03-26T17:37:27.750092+0000","last_peered":"2024-03-26T17:37:27.750092+0000","last_clean":"2024-03-26T17:37:27.750092+0000","last_became_active":"2024-03-26T17:37:27.741751+0000","last_became_peered":"2024-03-26T17:37:27.741751+0000","last_unstale":"2024-03-26T17:37:27.750092+0000","last_undegraded":"2024-03-26T17:37:27.750092+0000","last_fullsized":"2024-03-26T17:37:27.750092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:42.119297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621855+0000","last_change":"2024-03-26T17:36:22.736487+0000","last_active":"2024-03-26T17:37:56.621855+0000","last_peered":"2024-03-26T17:37:56.621855+0000","last_clean":"2024-03-26T17:37:56.621855+0000","last_became_active":"2024-03-26T17:36:22.735788+0000","last_became_peered":"2024-03-26T17:36:22.735788+0000","last_unstale":"2024-03-26T17:37:56.621855+0000","last_undegraded":"2024-03-26T17:37:56.621855+0000","last_fullsized":"2024-03-26T17:37:56.621855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:45.240148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'13","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739515+0000","last_change":"2024-03-26T17:36:21.720755+0000","last_active":"2024-03-26T17:37:27.739515+0000","last_peered":"2024-03-26T17:37:27.739515+0000","last_clean":"2024-03-26T17:37:27.739515+0000","last_became_active":"2024-03-26T17:36:21.720175+0000","last_became_peered":"2024-03-26T17:36:21.720175+0000","last_unstale":"2024-03-26T17:37:27.739515+0000","last_undegraded":"2024-03-26T17:37:27.739515+0000","last_fullsized":"2024-03-26T17:37:27.739515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:22.458657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624275+0000","last_change":"2024-03-26T17:36:22.740731+0000","last_active":"2024-03-26T17:37:56.624275+0000","last_peered":"2024-03-26T17:37:56.624275+0000","last_clean":"2024-03-26T17:37:56.624275+0000","last_became_active":"2024-03-26T17:36:22.740283+0000","last_became_peered":"2024-03-26T17:36:22.740283+0000","last_unstale":"2024-03-26T17:37:56.624275+0000","last_undegraded":"2024-03-26T17:37:56.624275+0000","last_fullsized":"2024-03-26T17:37:56.624275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:36.403105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244032+0000","last_change":"2024-03-26T17:36:21.721513+0000","last_active":"2024-03-26T17:37:28.244032+0000","last_peered":"2024-03-26T17:37:28.244032+0000","last_clean":"2024-03-26T17:37:28.244032+0000","last_became_active":"2024-03-26T17:36:21.721185+0000","last_became_peered":"2024-03-26T17:36:21.721185+0000","last_unstale":"2024-03-26T17:37:28.244032+0000","last_undegraded":"2024-03-26T17:37:28.244032+0000","last_fullsized":"2024-03-26T17:37:28.244032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:48:39.994980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628787+0000","last_change":"2024-03-26T17:36:22.737938+0000","last_active":"2024-03-26T17:37:56.628787+0000","last_peered":"2024-03-26T17:37:56.628787+0000","last_clean":"2024-03-26T17:37:56.628787+0000","last_became_active":"2024-03-26T17:36:22.737605+0000","last_became_peered":"2024-03-26T17:36:22.737605+0000","last_unstale":"2024-03-26T17:37:56.628787+0000","last_undegraded":"2024-03-26T17:37:56.628787+0000","last_fullsized":"2024-03-26T17:37:56.628787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:27.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245677+0000","last_change":"2024-03-26T17:36:21.720942+0000","last_active":"2024-03-26T17:37:28.245677+0000","last_peered":"2024-03-26T17:37:28.245677+0000","last_clean":"2024-03-26T17:37:28.245677+0000","last_became_active":"2024-03-26T17:36:21.720665+0000","last_became_peered":"2024-03-26T17:36:21.720665+0000","last_unstale":"2024-03-26T17:37:28.245677+0000","last_undegraded":"2024-03-26T17:37:28.245677+0000","last_fullsized":"2024-03-26T17:37:28.245677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:00.725213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.587257+0000","last_change":"2024-03-26T17:36:22.747111+0000","last_active":"2024-03-26T17:37:56.587257+0000","last_peered":"2024-03-26T17:37:56.587257+0000","last_clean":"2024-03-26T17:37:56.587257+0000","last_became_active":"2024-03-26T17:36:22.746626+0000","last_became_peered":"2024-03-26T17:36:22.746626+0000","last_unstale":"2024-03-26T17:37:56.587257+0000","last_undegraded":"2024-03-26T17:37:56.587257+0000","last_fullsized":"2024-03-26T17:37:56.587257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:29:24.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741395+0000","last_change":"2024-03-26T17:36:21.724141+0000","last_active":"2024-03-26T17:37:27.741395+0000","last_peered":"2024-03-26T17:37:27.741395+0000","last_clean":"2024-03-26T17:37:27.741395+0000","last_became_active":"2024-03-26T17:36:21.723797+0000","last_became_peered":"2024-03-26T17:36:21.723797+0000","last_unstale":"2024-03-26T17:37:27.741395+0000","last_undegraded":"2024-03-26T17:37:27.741395+0000","last_fullsized":"2024-03-26T17:37:27.741395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:06.149827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.627752+0000","last_change":"2024-03-26T17:36:22.739497+0000","last_active":"2024-03-26T17:37:56.627752+0000","last_peered":"2024-03-26T17:37:56.627752+0000","last_clean":"2024-03-26T17:37:56.627752+0000","last_became_active":"2024-03-26T17:36:22.739114+0000","last_became_peered":"2024-03-26T17:36:22.739114+0000","last_unstale":"2024-03-26T17:37:56.627752+0000","last_undegraded":"2024-03-26T17:37:56.627752+0000","last_fullsized":"2024-03-26T17:37:56.627752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:03.953199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.743064+0000","last_change":"2024-03-26T17:36:21.728027+0000","last_active":"2024-03-26T17:37:27.743064+0000","last_peered":"2024-03-26T17:37:27.743064+0000","last_clean":"2024-03-26T17:37:27.743064+0000","last_became_active":"2024-03-26T17:36:21.727159+0000","last_became_peered":"2024-03-26T17:36:21.727159+0000","last_unstale":"2024-03-26T17:37:27.743064+0000","last_undegraded":"2024-03-26T17:37:27.743064+0000","last_fullsized":"2024-03-26T17:37:27.743064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:39.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628518+0000","last_change":"2024-03-26T17:36:22.733666+0000","last_active":"2024-03-26T17:37:56.628518+0000","last_peered":"2024-03-26T17:37:56.628518+0000","last_clean":"2024-03-26T17:37:56.628518+0000","last_became_active":"2024-03-26T17:36:22.733313+0000","last_became_peered":"2024-03-26T17:36:22.733313+0000","last_unstale":"2024-03-26T17:37:56.628518+0000","last_undegraded":"2024-03-26T17:37:56.628518+0000","last_fullsized":"2024-03-26T17:37:56.628518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:52:38.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739591+0000","last_change":"2024-03-26T17:36:21.730828+0000","last_active":"2024-03-26T17:37:27.739591+0000","last_peered":"2024-03-26T17:37:27.739591+0000","last_clean":"2024-03-26T17:37:27.739591+0000","last_became_active":"2024-03-26T17:36:21.730508+0000","last_became_peered":"2024-03-26T17:36:21.730508+0000","last_unstale":"2024-03-26T17:37:27.739591+0000","last_undegraded":"2024-03-26T17:37:27.739591+0000","last_fullsized":"2024-03-26T17:37:27.739591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:10:53.492725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631929+0000","last_change":"2024-03-26T17:36:22.740204+0000","last_active":"2024-03-26T17:37:56.631929+0000","last_peered":"2024-03-26T17:37:56.631929+0000","last_clean":"2024-03-26T17:37:56.631929+0000","last_became_active":"2024-03-26T17:36:22.735880+0000","last_became_peered":"2024-03-26T17:36:22.735880+0000","last_unstale":"2024-03-26T17:37:56.631929+0000","last_undegraded":"2024-03-26T17:37:56.631929+0000","last_fullsized":"2024-03-26T17:37:56.631929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:34.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":144,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:19.425044+0000","last_change":"2024-03-26T17:36:21.720873+0000","last_active":"2024-03-26T17:38:19.425044+0000","last_peered":"2024-03-26T17:38:19.425044+0000","last_clean":"2024-03-26T17:38:19.425044+0000","last_became_active":"2024-03-26T17:36:21.720496+0000","last_became_peered":"2024-03-26T17:36:21.720496+0000","last_unstale":"2024-03-26T17:38:19.425044+0000","last_undegraded":"2024-03-26T17:38:19.425044+0000","last_fullsized":"2024-03-26T17:38:19.425044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:40:55.959064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2195,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.632104+0000","last_change":"2024-03-26T17:36:22.735528+0000","last_active":"2024-03-26T17:37:56.632104+0000","last_peered":"2024-03-26T17:37:56.632104+0000","last_clean":"2024-03-26T17:37:56.632104+0000","last_became_active":"2024-03-26T17:36:22.735147+0000","last_became_peered":"2024-03-26T17:36:22.735147+0000","last_unstale":"2024-03-26T17:37:56.632104+0000","last_undegraded":"2024-03-26T17:37:56.632104+0000","last_fullsized":"2024-03-26T17:37:56.632104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:15.729853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744355+0000","last_change":"2024-03-26T17:36:21.723246+0000","last_active":"2024-03-26T17:37:27.744355+0000","last_peered":"2024-03-26T17:37:27.744355+0000","last_clean":"2024-03-26T17:37:27.744355+0000","last_became_active":"2024-03-26T17:36:21.722839+0000","last_became_peered":"2024-03-26T17:36:21.722839+0000","last_unstale":"2024-03-26T17:37:27.744355+0000","last_undegraded":"2024-03-26T17:37:27.744355+0000","last_fullsized":"2024-03-26T17:37:27.744355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:41:05.540206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629489+0000","last_change":"2024-03-26T17:36:22.742469+0000","last_active":"2024-03-26T17:37:56.629489+0000","last_peered":"2024-03-26T17:37:56.629489+0000","last_clean":"2024-03-26T17:37:56.629489+0000","last_became_active":"2024-03-26T17:36:22.742073+0000","last_became_peered":"2024-03-26T17:36:22.742073+0000","last_unstale":"2024-03-26T17:37:56.629489+0000","last_undegraded":"2024-03-26T17:37:56.629489+0000","last_fullsized":"2024-03-26T17:37:56.629489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:19:33.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.250123+0000","last_change":"2024-03-26T17:37:28.250123+0000","last_active":"2024-03-26T17:37:28.250123+0000","last_peered":"2024-03-26T17:37:28.250123+0000","last_clean":"2024-03-26T17:37:28.250123+0000","last_became_active":"2024-03-26T17:37:28.246162+0000","last_became_peered":"2024-03-26T17:37:28.246162+0000","last_unstale":"2024-03-26T17:37:28.250123+0000","last_undegraded":"2024-03-26T17:37:28.250123+0000","last_fullsized":"2024-03-26T17:37:28.250123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:33:39.002192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628828+0000","last_change":"2024-03-26T17:36:22.742614+0000","last_active":"2024-03-26T17:37:56.628828+0000","last_peered":"2024-03-26T17:37:56.628828+0000","last_clean":"2024-03-26T17:37:56.628828+0000","last_became_active":"2024-03-26T17:36:22.742278+0000","last_became_peered":"2024-03-26T17:36:22.742278+0000","last_unstale":"2024-03-26T17:37:56.628828+0000","last_undegraded":"2024-03-26T17:37:56.628828+0000","last_fullsized":"2024-03-26T17:37:56.628828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:30:14.503474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.627061+0000","last_change":"2024-03-26T17:36:22.741693+0000","last_active":"2024-03-26T17:37:56.627061+0000","last_peered":"2024-03-26T17:37:56.627061+0000","last_clean":"2024-03-26T17:37:56.627061+0000","last_became_active":"2024-03-26T17:36:22.741352+0000","last_became_peered":"2024-03-26T17:36:22.741352+0000","last_unstale":"2024-03-26T17:37:56.627061+0000","last_undegraded":"2024-03-26T17:37:56.627061+0000","last_fullsized":"2024-03-26T17:37:56.627061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:13:26.912910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.623353+0000","last_change":"2024-03-26T17:36:22.742939+0000","last_active":"2024-03-26T17:37:56.623353+0000","last_peered":"2024-03-26T17:37:56.623353+0000","last_clean":"2024-03-26T17:37:56.623353+0000","last_became_active":"2024-03-26T17:36:22.742589+0000","last_became_peered":"2024-03-26T17:36:22.742589+0000","last_unstale":"2024-03-26T17:37:56.623353+0000","last_undegraded":"2024-03-26T17:37:56.623353+0000","last_fullsized":"2024-03-26T17:37:56.623353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:21:14.043601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617673+0000","last_change":"2024-03-26T17:36:22.748044+0000","last_active":"2024-03-26T17:37:56.617673+0000","last_peered":"2024-03-26T17:37:56.617673+0000","last_clean":"2024-03-26T17:37:56.617673+0000","last_became_active":"2024-03-26T17:36:22.747683+0000","last_became_peered":"2024-03-26T17:36:22.747683+0000","last_unstale":"2024-03-26T17:37:56.617673+0000","last_undegraded":"2024-03-26T17:37:56.617673+0000","last_fullsized":"2024-03-26T17:37:56.617673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:32:40.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628303+0000","last_change":"2024-03-26T17:36:22.743080+0000","last_active":"2024-03-26T17:37:56.628303+0000","last_peered":"2024-03-26T17:37:56.628303+0000","last_clean":"2024-03-26T17:37:56.628303+0000","last_became_active":"2024-03-26T17:36:22.742676+0000","last_became_peered":"2024-03-26T17:36:22.742676+0000","last_unstale":"2024-03-26T17:37:56.628303+0000","last_undegraded":"2024-03-26T17:37:56.628303+0000","last_fullsized":"2024-03-26T17:37:56.628303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:58.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628807+0000","last_change":"2024-03-26T17:36:22.737908+0000","last_active":"2024-03-26T17:37:56.628807+0000","last_peered":"2024-03-26T17:37:56.628807+0000","last_clean":"2024-03-26T17:37:56.628807+0000","last_became_active":"2024-03-26T17:36:22.737055+0000","last_became_peered":"2024-03-26T17:36:22.737055+0000","last_unstale":"2024-03-26T17:37:56.628807+0000","last_undegraded":"2024-03-26T17:37:56.628807+0000","last_fullsized":"2024-03-26T17:37:56.628807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:51:13.157901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621037+0000","last_change":"2024-03-26T17:36:22.749795+0000","last_active":"2024-03-26T17:37:56.621037+0000","last_peered":"2024-03-26T17:37:56.621037+0000","last_clean":"2024-03-26T17:37:56.621037+0000","last_became_active":"2024-03-26T17:36:22.749469+0000","last_became_peered":"2024-03-26T17:36:22.749469+0000","last_unstale":"2024-03-26T17:37:56.621037+0000","last_undegraded":"2024-03-26T17:37:56.621037+0000","last_fullsized":"2024-03-26T17:37:56.621037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:19:59.126270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.633721+0000","last_change":"2024-03-26T17:36:22.744101+0000","last_active":"2024-03-26T17:37:56.633721+0000","last_peered":"2024-03-26T17:37:56.633721+0000","last_clean":"2024-03-26T17:37:56.633721+0000","last_became_active":"2024-03-26T17:36:22.743761+0000","last_became_peered":"2024-03-26T17:36:22.743761+0000","last_unstale":"2024-03-26T17:37:56.633721+0000","last_undegraded":"2024-03-26T17:37:56.633721+0000","last_fullsized":"2024-03-26T17:37:56.633721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:27:19.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617497+0000","last_change":"2024-03-26T17:36:22.744800+0000","last_active":"2024-03-26T17:37:56.617497+0000","last_peered":"2024-03-26T17:37:56.617497+0000","last_clean":"2024-03-26T17:37:56.617497+0000","last_became_active":"2024-03-26T17:36:22.744437+0000","last_became_peered":"2024-03-26T17:36:22.744437+0000","last_unstale":"2024-03-26T17:37:56.617497+0000","last_undegraded":"2024-03-26T17:37:56.617497+0000","last_fullsized":"2024-03-26T17:37:56.617497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:36.459072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.620441+0000","last_change":"2024-03-26T17:37:27.745243+0000","last_active":"2024-03-26T17:37:56.620441+0000","last_peered":"2024-03-26T17:37:56.620441+0000","last_clean":"2024-03-26T17:37:56.620441+0000","last_became_active":"2024-03-26T17:37:27.744869+0000","last_became_peered":"2024-03-26T17:37:27.744869+0000","last_unstale":"2024-03-26T17:37:56.620441+0000","last_undegraded":"2024-03-26T17:37:56.620441+0000","last_fullsized":"2024-03-26T17:37:56.620441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:18:42.188733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.623784+0000","last_change":"2024-03-26T17:36:22.743211+0000","last_active":"2024-03-26T17:37:56.623784+0000","last_peered":"2024-03-26T17:37:56.623784+0000","last_clean":"2024-03-26T17:37:56.623784+0000","last_became_active":"2024-03-26T17:36:22.742869+0000","last_became_peered":"2024-03-26T17:36:22.742869+0000","last_unstale":"2024-03-26T17:37:56.623784+0000","last_undegraded":"2024-03-26T17:37:56.623784+0000","last_fullsized":"2024-03-26T17:37:56.623784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:24:04.610012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635257+0000","last_change":"2024-03-26T17:36:22.746950+0000","last_active":"2024-03-26T17:37:56.635257+0000","last_peered":"2024-03-26T17:37:56.635257+0000","last_clean":"2024-03-26T17:37:56.635257+0000","last_became_active":"2024-03-26T17:36:22.744869+0000","last_became_peered":"2024-03-26T17:36:22.744869+0000","last_unstale":"2024-03-26T17:37:56.635257+0000","last_undegraded":"2024-03-26T17:37:56.635257+0000","last_fullsized":"2024-03-26T17:37:56.635257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:09:52.283519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629292+0000","last_change":"2024-03-26T17:36:22.744627+0000","last_active":"2024-03-26T17:37:56.629292+0000","last_peered":"2024-03-26T17:37:56.629292+0000","last_clean":"2024-03-26T17:37:56.629292+0000","last_became_active":"2024-03-26T17:36:22.744299+0000","last_became_peered":"2024-03-26T17:36:22.744299+0000","last_unstale":"2024-03-26T17:37:56.629292+0000","last_undegraded":"2024-03-26T17:37:56.629292+0000","last_fullsized":"2024-03-26T17:37:56.629292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:23:37.941625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626943+0000","last_change":"2024-03-26T17:36:22.751135+0000","last_active":"2024-03-26T17:37:56.626943+0000","last_peered":"2024-03-26T17:37:56.626943+0000","last_clean":"2024-03-26T17:37:56.626943+0000","last_became_active":"2024-03-26T17:36:22.750768+0000","last_became_peered":"2024-03-26T17:36:22.750768+0000","last_unstale":"2024-03-26T17:37:56.626943+0000","last_undegraded":"2024-03-26T17:37:56.626943+0000","last_fullsized":"2024-03-26T17:37:56.626943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:15.532367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74926,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1164,"num_read_kb":664,"num_write":255,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":214875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1109624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32708,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712444,"statfs":{"total":95995035648,"available":95961542656,"internally_reserved":0,"allocated":1568768,"data_stored":1048749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":455961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":519267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":458325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":520288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":455786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1048920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T17:38:24.472 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T17:38:24.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 -- 172.21.15.105:0/2365205076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 msgr2=0x7f5760101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 --2- 172.21.15.105:0/2365205076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 0x7f5760101e50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5754009900 tx=0x7f575402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:24.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 -- 172.21.15.105:0/2365205076 shutdown_connections 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 --2- 172.21.15.105:0/2365205076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5760102390 0x7f5760108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 --2- 172.21.15.105:0/2365205076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 0x7f5760101e50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 --2- 172.21.15.105:0/2365205076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57601007e0 0x7f5760100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 -- 172.21.15.105:0/2365205076 >> 172.21.15.105:0/2365205076 conn(0x7f57600fbd50 msgr2=0x7f57600fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 -- 172.21.15.105:0/2365205076 shutdown_connections 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 -- 172.21.15.105:0/2365205076 wait complete. 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.618+0000 7f5766590640 1 Processor -- start 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f5766590640 1 -- start start 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f5766590640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57601007e0 0x7f5760196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f5766590640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 0x7f5760196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:24.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f5766590640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760102390 0x7f576019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f5766590640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f576010c2f0 con 0x7f57601019d0 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f5766590640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f576010c170 con 0x7f57601007e0 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f5766590640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f576010c470 con 0x7f5760102390 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 0x7f5760196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 0x7f5760196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55834/0 (socket says 172.21.15.105:55834) 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57601007e0 0x7f5760196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575f7fe640 1 -- 172.21.15.105:0/518600303 learned_addr learned my addr 172.21.15.105:0/518600303 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f5764b2d640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760102390 0x7f576019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575f7fe640 1 -- 172.21.15.105:0/518600303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760102390 msgr2=0x7f576019dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575f7fe640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760102390 0x7f576019dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575f7fe640 1 -- 172.21.15.105:0/518600303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57601007e0 msgr2=0x7f5760196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575f7fe640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57601007e0 0x7f5760196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575f7fe640 1 -- 172.21.15.105:0/518600303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576019e650 con 0x7f57601019d0 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.619+0000 7f575ffff640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57601007e0 0x7f5760196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.620+0000 7f575f7fe640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 0x7f5760196f70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5754006010 tx=0x7f5754005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.620+0000 7f575d7fa640 1 -- 172.21.15.105:0/518600303 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754043020 con 0x7f57601019d0 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.620+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576019e8e0 con 0x7f57601019d0 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.620+0000 7f575d7fa640 1 -- 172.21.15.105:0/518600303 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57540043d0 con 0x7f57601019d0 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.620+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576019ee30 con 0x7f57601019d0 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.620+0000 7f575d7fa640 1 -- 172.21.15.105:0/518600303 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754003a20 con 0x7f57601019d0 2024-03-26T17:38:24.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.621+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5760107c20 con 0x7f57601019d0 2024-03-26T17:38:24.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.622+0000 7f575d7fa640 1 -- 172.21.15.105:0/518600303 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f5754046020 con 0x7f57601019d0 2024-03-26T17:38:24.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.623+0000 7f575d7fa640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57480a2fb0 0x7f57480a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:24.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.623+0000 7f575ffff640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57480a2fb0 0x7f57480a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:24.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.623+0000 7f575d7fa640 1 -- 172.21.15.105:0/518600303 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f57540f1c20 con 0x7f57601019d0 2024-03-26T17:38:24.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.623+0000 7f575ffff640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57480a2fb0 0x7f57480a5470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5760101830 tx=0x7f5750005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:24.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.627+0000 7f575d7fa640 1 -- 172.21.15.105:0/518600303 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f575404d020 con 0x7f57601019d0 2024-03-26T17:38:24.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.824+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5760061d00 con 0x7f57480a2fb0 2024-03-26T17:38:24.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.832+0000 7f575d7fa640 1 -- 172.21.15.105:0/518600303 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258897 (secure 0 0 0) 0x7f5760061d00 con 0x7f57480a2fb0 2024-03-26T17:38:24.827 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:24.828 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-26T17:38:24.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57480a2fb0 msgr2=0x7f57480a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57480a2fb0 0x7f57480a5470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5760101830 tx=0x7f5750005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:24.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 msgr2=0x7f5760196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:24.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 0x7f5760196f70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5754006010 tx=0x7f5754005e40 comp rx=0 tx=0).stop 2024-03-26T17:38:24.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 shutdown_connections 2024-03-26T17:38:24.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57480a2fb0 0x7f57480a5470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5760102390 0x7f576019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57601019d0 0x7f5760196f70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 --2- 172.21.15.105:0/518600303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57601007e0 0x7f5760196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:24.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 >> 172.21.15.105:0/518600303 conn(0x7f57600fbd50 msgr2=0x7f5760104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:24.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 shutdown_connections 2024-03-26T17:38:24.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:24.835+0000 7f5766590640 1 -- 172.21.15.105:0/518600303 wait complete. 2024-03-26T17:38:24.839 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-03-26T17:38:24.609618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666412,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1307,"num_read_kb":785,"num_write":446,"num_write_kb":3097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366620,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575204,"statfs":{"total":1151940427776,"available":1151565008896,"internally_reserved":0,"allocated":13627392,"data_stored":7446194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7561,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":71,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002616"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628144+0000","last_change":"2024-03-26T17:36:22.743815+0000","last_active":"2024-03-26T17:37:56.628144+0000","last_peered":"2024-03-26T17:37:56.628144+0000","last_clean":"2024-03-26T17:37:56.628144+0000","last_became_active":"2024-03-26T17:36:22.743446+0000","last_became_peered":"2024-03-26T17:36:22.743446+0000","last_unstale":"2024-03-26T17:37:56.628144+0000","last_undegraded":"2024-03-26T17:37:56.628144+0000","last_fullsized":"2024-03-26T17:37:56.628144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:16:56.773849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628208+0000","last_change":"2024-03-26T17:36:22.740754+0000","last_active":"2024-03-26T17:37:56.628208+0000","last_peered":"2024-03-26T17:37:56.628208+0000","last_clean":"2024-03-26T17:37:56.628208+0000","last_became_active":"2024-03-26T17:36:22.740317+0000","last_became_peered":"2024-03-26T17:36:22.740317+0000","last_unstale":"2024-03-26T17:37:56.628208+0000","last_undegraded":"2024-03-26T17:37:56.628208+0000","last_fullsized":"2024-03-26T17:37:56.628208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:50:45.321653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635496+0000","last_change":"2024-03-26T17:36:22.735079+0000","last_active":"2024-03-26T17:37:56.635496+0000","last_peered":"2024-03-26T17:37:56.635496+0000","last_clean":"2024-03-26T17:37:56.635496+0000","last_became_active":"2024-03-26T17:36:22.734646+0000","last_became_peered":"2024-03-26T17:36:22.734646+0000","last_unstale":"2024-03-26T17:37:56.635496+0000","last_undegraded":"2024-03-26T17:37:56.635496+0000","last_fullsized":"2024-03-26T17:37:56.635496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:41:09.477438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.615398+0000","last_change":"2024-03-26T17:36:22.746170+0000","last_active":"2024-03-26T17:37:56.615398+0000","last_peered":"2024-03-26T17:37:56.615398+0000","last_clean":"2024-03-26T17:37:56.615398+0000","last_became_active":"2024-03-26T17:36:22.744255+0000","last_became_peered":"2024-03-26T17:36:22.744255+0000","last_unstale":"2024-03-26T17:37:56.615398+0000","last_undegraded":"2024-03-26T17:37:56.615398+0000","last_fullsized":"2024-03-26T17:37:56.615398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:54:36.225926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.607889+0000","last_change":"2024-03-26T17:36:22.737390+0000","last_active":"2024-03-26T17:37:56.607889+0000","last_peered":"2024-03-26T17:37:56.607889+0000","last_clean":"2024-03-26T17:37:56.607889+0000","last_became_active":"2024-03-26T17:36:22.737038+0000","last_became_peered":"2024-03-26T17:36:22.737038+0000","last_unstale":"2024-03-26T17:37:56.607889+0000","last_undegraded":"2024-03-26T17:37:56.607889+0000","last_fullsized":"2024-03-26T17:37:56.607889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:48.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.634748+0000","last_change":"2024-03-26T17:36:22.748097+0000","last_active":"2024-03-26T17:37:56.634748+0000","last_peered":"2024-03-26T17:37:56.634748+0000","last_clean":"2024-03-26T17:37:56.634748+0000","last_became_active":"2024-03-26T17:36:22.747730+0000","last_became_peered":"2024-03-26T17:36:22.747730+0000","last_unstale":"2024-03-26T17:37:56.634748+0000","last_undegraded":"2024-03-26T17:37:56.634748+0000","last_fullsized":"2024-03-26T17:37:56.634748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:43:01.370199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.620104+0000","last_change":"2024-03-26T17:36:22.739587+0000","last_active":"2024-03-26T17:37:56.620104+0000","last_peered":"2024-03-26T17:37:56.620104+0000","last_clean":"2024-03-26T17:37:56.620104+0000","last_became_active":"2024-03-26T17:36:22.735787+0000","last_became_peered":"2024-03-26T17:36:22.735787+0000","last_unstale":"2024-03-26T17:37:56.620104+0000","last_undegraded":"2024-03-26T17:37:56.620104+0000","last_fullsized":"2024-03-26T17:37:56.620104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:41.381153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621279+0000","last_change":"2024-03-26T17:36:22.736716+0000","last_active":"2024-03-26T17:37:56.621279+0000","last_peered":"2024-03-26T17:37:56.621279+0000","last_clean":"2024-03-26T17:37:56.621279+0000","last_became_active":"2024-03-26T17:36:22.736327+0000","last_became_peered":"2024-03-26T17:36:22.736327+0000","last_unstale":"2024-03-26T17:37:56.621279+0000","last_undegraded":"2024-03-26T17:37:56.621279+0000","last_fullsized":"2024-03-26T17:37:56.621279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:10:43.439524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635164+0000","last_change":"2024-03-26T17:36:22.746355+0000","last_active":"2024-03-26T17:37:56.635164+0000","last_peered":"2024-03-26T17:37:56.635164+0000","last_clean":"2024-03-26T17:37:56.635164+0000","last_became_active":"2024-03-26T17:36:22.745363+0000","last_became_peered":"2024-03-26T17:36:22.745363+0000","last_unstale":"2024-03-26T17:37:56.635164+0000","last_undegraded":"2024-03-26T17:37:56.635164+0000","last_fullsized":"2024-03-26T17:37:56.635164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:56:33.528005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.634951+0000","last_change":"2024-03-26T17:36:22.749009+0000","last_active":"2024-03-26T17:37:56.634951+0000","last_peered":"2024-03-26T17:37:56.634951+0000","last_clean":"2024-03-26T17:37:56.634951+0000","last_became_active":"2024-03-26T17:36:22.748627+0000","last_became_peered":"2024-03-26T17:36:22.748627+0000","last_unstale":"2024-03-26T17:37:56.634951+0000","last_undegraded":"2024-03-26T17:37:56.634951+0000","last_fullsized":"2024-03-26T17:37:56.634951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:59:14.812316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631878+0000","last_change":"2024-03-26T17:36:22.741294+0000","last_active":"2024-03-26T17:37:56.631878+0000","last_peered":"2024-03-26T17:37:56.631878+0000","last_clean":"2024-03-26T17:37:56.631878+0000","last_became_active":"2024-03-26T17:36:22.740003+0000","last_became_peered":"2024-03-26T17:36:22.740003+0000","last_unstale":"2024-03-26T17:37:56.631878+0000","last_undegraded":"2024-03-26T17:37:56.631878+0000","last_fullsized":"2024-03-26T17:37:56.631878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:06.305195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629278+0000","last_change":"2024-03-26T17:36:22.745884+0000","last_active":"2024-03-26T17:37:56.629278+0000","last_peered":"2024-03-26T17:37:56.629278+0000","last_clean":"2024-03-26T17:37:56.629278+0000","last_became_active":"2024-03-26T17:36:22.744168+0000","last_became_peered":"2024-03-26T17:36:22.744168+0000","last_unstale":"2024-03-26T17:37:56.629278+0000","last_undegraded":"2024-03-26T17:37:56.629278+0000","last_fullsized":"2024-03-26T17:37:56.629278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:50.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624550+0000","last_change":"2024-03-26T17:36:22.725956+0000","last_active":"2024-03-26T17:37:56.624550+0000","last_peered":"2024-03-26T17:37:56.624550+0000","last_clean":"2024-03-26T17:37:56.624550+0000","last_became_active":"2024-03-26T17:36:22.725513+0000","last_became_peered":"2024-03-26T17:36:22.725513+0000","last_unstale":"2024-03-26T17:37:56.624550+0000","last_undegraded":"2024-03-26T17:37:56.624550+0000","last_fullsized":"2024-03-26T17:37:56.624550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:30:10.914233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628784+0000","last_change":"2024-03-26T17:36:22.739757+0000","last_active":"2024-03-26T17:37:56.628784+0000","last_peered":"2024-03-26T17:37:56.628784+0000","last_clean":"2024-03-26T17:37:56.628784+0000","last_became_active":"2024-03-26T17:36:22.739342+0000","last_became_peered":"2024-03-26T17:36:22.739342+0000","last_unstale":"2024-03-26T17:37:56.628784+0000","last_undegraded":"2024-03-26T17:37:56.628784+0000","last_fullsized":"2024-03-26T17:37:56.628784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:28.171325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.618872+0000","last_change":"2024-03-26T17:36:22.742988+0000","last_active":"2024-03-26T17:37:56.618872+0000","last_peered":"2024-03-26T17:37:56.618872+0000","last_clean":"2024-03-26T17:37:56.618872+0000","last_became_active":"2024-03-26T17:36:22.742639+0000","last_became_peered":"2024-03-26T17:36:22.742639+0000","last_unstale":"2024-03-26T17:37:56.618872+0000","last_undegraded":"2024-03-26T17:37:56.618872+0000","last_fullsized":"2024-03-26T17:37:56.618872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:48:44.204889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635451+0000","last_change":"2024-03-26T17:36:22.744530+0000","last_active":"2024-03-26T17:37:56.635451+0000","last_peered":"2024-03-26T17:37:56.635451+0000","last_clean":"2024-03-26T17:37:56.635451+0000","last_became_active":"2024-03-26T17:36:22.743566+0000","last_became_peered":"2024-03-26T17:36:22.743566+0000","last_unstale":"2024-03-26T17:37:56.635451+0000","last_undegraded":"2024-03-26T17:37:56.635451+0000","last_fullsized":"2024-03-26T17:37:56.635451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:42.685326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629288+0000","last_change":"2024-03-26T17:36:22.745797+0000","last_active":"2024-03-26T17:37:56.629288+0000","last_peered":"2024-03-26T17:37:56.629288+0000","last_clean":"2024-03-26T17:37:56.629288+0000","last_became_active":"2024-03-26T17:36:22.745448+0000","last_became_peered":"2024-03-26T17:36:22.745448+0000","last_unstale":"2024-03-26T17:37:56.629288+0000","last_undegraded":"2024-03-26T17:37:56.629288+0000","last_fullsized":"2024-03-26T17:37:56.629288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:05:29.126564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631670+0000","last_change":"2024-03-26T17:36:22.749890+0000","last_active":"2024-03-26T17:37:56.631670+0000","last_peered":"2024-03-26T17:37:56.631670+0000","last_clean":"2024-03-26T17:37:56.631670+0000","last_became_active":"2024-03-26T17:36:22.749497+0000","last_became_peered":"2024-03-26T17:36:22.749497+0000","last_unstale":"2024-03-26T17:37:56.631670+0000","last_undegraded":"2024-03-26T17:37:56.631670+0000","last_fullsized":"2024-03-26T17:37:56.631670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:12:55.069659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.612760+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:56.612760+0000","last_peered":"2024-03-26T17:37:56.612760+0000","last_clean":"2024-03-26T17:37:56.612760+0000","last_became_active":"2024-03-26T17:36:22.740287+0000","last_became_peered":"2024-03-26T17:36:22.740287+0000","last_unstale":"2024-03-26T17:37:56.612760+0000","last_undegraded":"2024-03-26T17:37:56.612760+0000","last_fullsized":"2024-03-26T17:37:56.612760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:13:00.574246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.431274+0000","last_change":"2024-03-26T17:36:21.720809+0000","last_active":"2024-03-26T17:38:21.431274+0000","last_peered":"2024-03-26T17:38:21.431274+0000","last_clean":"2024-03-26T17:38:21.431274+0000","last_became_active":"2024-03-26T17:36:21.720526+0000","last_became_peered":"2024-03-26T17:36:21.720526+0000","last_unstale":"2024-03-26T17:38:21.431274+0000","last_undegraded":"2024-03-26T17:38:21.431274+0000","last_fullsized":"2024-03-26T17:38:21.431274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:48.034328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.615928+0000","last_change":"2024-03-26T17:36:22.746245+0000","last_active":"2024-03-26T17:37:56.615928+0000","last_peered":"2024-03-26T17:37:56.615928+0000","last_clean":"2024-03-26T17:37:56.615928+0000","last_became_active":"2024-03-26T17:36:22.745859+0000","last_became_peered":"2024-03-26T17:36:22.745859+0000","last_unstale":"2024-03-26T17:37:56.615928+0000","last_undegraded":"2024-03-26T17:37:56.615928+0000","last_fullsized":"2024-03-26T17:37:56.615928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:39.085668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741831+0000","last_change":"2024-03-26T17:36:21.723178+0000","last_active":"2024-03-26T17:37:27.741831+0000","last_peered":"2024-03-26T17:37:27.741831+0000","last_clean":"2024-03-26T17:37:27.741831+0000","last_became_active":"2024-03-26T17:36:21.722858+0000","last_became_peered":"2024-03-26T17:36:21.722858+0000","last_unstale":"2024-03-26T17:37:27.741831+0000","last_undegraded":"2024-03-26T17:37:27.741831+0000","last_fullsized":"2024-03-26T17:37:27.741831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:46:13.837672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635805+0000","last_change":"2024-03-26T17:36:22.736231+0000","last_active":"2024-03-26T17:37:56.635805+0000","last_peered":"2024-03-26T17:37:56.635805+0000","last_clean":"2024-03-26T17:37:56.635805+0000","last_became_active":"2024-03-26T17:36:22.735942+0000","last_became_peered":"2024-03-26T17:36:22.735942+0000","last_unstale":"2024-03-26T17:37:56.635805+0000","last_undegraded":"2024-03-26T17:37:56.635805+0000","last_fullsized":"2024-03-26T17:37:56.635805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:49.127382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740597+0000","last_change":"2024-03-26T17:36:21.731369+0000","last_active":"2024-03-26T17:37:27.740597+0000","last_peered":"2024-03-26T17:37:27.740597+0000","last_clean":"2024-03-26T17:37:27.740597+0000","last_became_active":"2024-03-26T17:36:21.731016+0000","last_became_peered":"2024-03-26T17:36:21.731016+0000","last_unstale":"2024-03-26T17:37:27.740597+0000","last_undegraded":"2024-03-26T17:37:27.740597+0000","last_fullsized":"2024-03-26T17:37:27.740597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:37:53.204825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635677+0000","last_change":"2024-03-26T17:36:22.751155+0000","last_active":"2024-03-26T17:37:56.635677+0000","last_peered":"2024-03-26T17:37:56.635677+0000","last_clean":"2024-03-26T17:37:56.635677+0000","last_became_active":"2024-03-26T17:36:22.750704+0000","last_became_peered":"2024-03-26T17:36:22.750704+0000","last_unstale":"2024-03-26T17:37:56.635677+0000","last_undegraded":"2024-03-26T17:37:56.635677+0000","last_fullsized":"2024-03-26T17:37:56.635677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:52:24.116923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739808+0000","last_change":"2024-03-26T17:36:21.726380+0000","last_active":"2024-03-26T17:37:27.739808+0000","last_peered":"2024-03-26T17:37:27.739808+0000","last_clean":"2024-03-26T17:37:27.739808+0000","last_became_active":"2024-03-26T17:36:21.726004+0000","last_became_peered":"2024-03-26T17:36:21.726004+0000","last_unstale":"2024-03-26T17:37:27.739808+0000","last_undegraded":"2024-03-26T17:37:27.739808+0000","last_fullsized":"2024-03-26T17:37:27.739808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:51:23.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.630425+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:56.630425+0000","last_peered":"2024-03-26T17:37:56.630425+0000","last_clean":"2024-03-26T17:37:56.630425+0000","last_became_active":"2024-03-26T17:36:22.741104+0000","last_became_peered":"2024-03-26T17:36:22.741104+0000","last_unstale":"2024-03-26T17:37:56.630425+0000","last_undegraded":"2024-03-26T17:37:56.630425+0000","last_fullsized":"2024-03-26T17:37:56.630425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:53.683944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:18.423824+0000","last_change":"2024-03-26T17:36:21.728824+0000","last_active":"2024-03-26T17:38:18.423824+0000","last_peered":"2024-03-26T17:38:18.423824+0000","last_clean":"2024-03-26T17:38:18.423824+0000","last_became_active":"2024-03-26T17:36:21.728467+0000","last_became_peered":"2024-03-26T17:36:21.728467+0000","last_unstale":"2024-03-26T17:38:18.423824+0000","last_undegraded":"2024-03-26T17:38:18.423824+0000","last_fullsized":"2024-03-26T17:38:18.423824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:25:07.842111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617769+0000","last_change":"2024-03-26T17:36:22.740131+0000","last_active":"2024-03-26T17:37:56.617769+0000","last_peered":"2024-03-26T17:37:56.617769+0000","last_clean":"2024-03-26T17:37:56.617769+0000","last_became_active":"2024-03-26T17:36:22.739704+0000","last_became_peered":"2024-03-26T17:36:22.739704+0000","last_unstale":"2024-03-26T17:37:56.617769+0000","last_undegraded":"2024-03-26T17:37:56.617769+0000","last_fullsized":"2024-03-26T17:37:56.617769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:47.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245391+0000","last_change":"2024-03-26T17:36:21.731826+0000","last_active":"2024-03-26T17:37:28.245391+0000","last_peered":"2024-03-26T17:37:28.245391+0000","last_clean":"2024-03-26T17:37:28.245391+0000","last_became_active":"2024-03-26T17:36:21.731546+0000","last_became_peered":"2024-03-26T17:36:21.731546+0000","last_unstale":"2024-03-26T17:37:28.245391+0000","last_undegraded":"2024-03-26T17:37:28.245391+0000","last_fullsized":"2024-03-26T17:37:28.245391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:08:53.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.614608+0000","last_change":"2024-03-26T17:36:22.745170+0000","last_active":"2024-03-26T17:37:56.614608+0000","last_peered":"2024-03-26T17:37:56.614608+0000","last_clean":"2024-03-26T17:37:56.614608+0000","last_became_active":"2024-03-26T17:36:22.744798+0000","last_became_peered":"2024-03-26T17:36:22.744798+0000","last_unstale":"2024-03-26T17:37:56.614608+0000","last_undegraded":"2024-03-26T17:37:56.614608+0000","last_fullsized":"2024-03-26T17:37:56.614608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:32:50.418149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739722+0000","last_change":"2024-03-26T17:36:21.721492+0000","last_active":"2024-03-26T17:37:27.739722+0000","last_peered":"2024-03-26T17:37:27.739722+0000","last_clean":"2024-03-26T17:37:27.739722+0000","last_became_active":"2024-03-26T17:36:21.721194+0000","last_became_peered":"2024-03-26T17:36:21.721194+0000","last_unstale":"2024-03-26T17:37:27.739722+0000","last_undegraded":"2024-03-26T17:37:27.739722+0000","last_fullsized":"2024-03-26T17:37:27.739722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:50.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624868+0000","last_change":"2024-03-26T17:36:22.740780+0000","last_active":"2024-03-26T17:37:56.624868+0000","last_peered":"2024-03-26T17:37:56.624868+0000","last_clean":"2024-03-26T17:37:56.624868+0000","last_became_active":"2024-03-26T17:36:22.740311+0000","last_became_peered":"2024-03-26T17:36:22.740311+0000","last_unstale":"2024-03-26T17:37:56.624868+0000","last_undegraded":"2024-03-26T17:37:56.624868+0000","last_fullsized":"2024-03-26T17:37:56.624868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:01:56.018416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742536+0000","last_change":"2024-03-26T17:36:21.726289+0000","last_active":"2024-03-26T17:37:27.742536+0000","last_peered":"2024-03-26T17:37:27.742536+0000","last_clean":"2024-03-26T17:37:27.742536+0000","last_became_active":"2024-03-26T17:36:21.725898+0000","last_became_peered":"2024-03-26T17:36:21.725898+0000","last_unstale":"2024-03-26T17:37:27.742536+0000","last_undegraded":"2024-03-26T17:37:27.742536+0000","last_fullsized":"2024-03-26T17:37:27.742536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:03.893163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631799+0000","last_change":"2024-03-26T17:36:22.750632+0000","last_active":"2024-03-26T17:37:56.631799+0000","last_peered":"2024-03-26T17:37:56.631799+0000","last_clean":"2024-03-26T17:37:56.631799+0000","last_became_active":"2024-03-26T17:36:22.750297+0000","last_became_peered":"2024-03-26T17:36:22.750297+0000","last_unstale":"2024-03-26T17:37:56.631799+0000","last_undegraded":"2024-03-26T17:37:56.631799+0000","last_fullsized":"2024-03-26T17:37:56.631799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:43:16.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":254,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.426474+0000","last_change":"2024-03-26T17:36:21.721403+0000","last_active":"2024-03-26T17:38:21.426474+0000","last_peered":"2024-03-26T17:38:21.426474+0000","last_clean":"2024-03-26T17:38:21.426474+0000","last_became_active":"2024-03-26T17:36:21.720962+0000","last_became_peered":"2024-03-26T17:36:21.720962+0000","last_unstale":"2024-03-26T17:38:21.426474+0000","last_undegraded":"2024-03-26T17:38:21.426474+0000","last_fullsized":"2024-03-26T17:38:21.426474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:02.683188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635445+0000","last_change":"2024-03-26T17:36:22.754010+0000","last_active":"2024-03-26T17:37:56.635445+0000","last_peered":"2024-03-26T17:37:56.635445+0000","last_clean":"2024-03-26T17:37:56.635445+0000","last_became_active":"2024-03-26T17:36:22.753534+0000","last_became_peered":"2024-03-26T17:36:22.753534+0000","last_unstale":"2024-03-26T17:37:56.635445+0000","last_undegraded":"2024-03-26T17:37:56.635445+0000","last_fullsized":"2024-03-26T17:37:56.635445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:06.490733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736915+0000","last_change":"2024-03-26T17:36:21.721685+0000","last_active":"2024-03-26T17:37:27.736915+0000","last_peered":"2024-03-26T17:37:27.736915+0000","last_clean":"2024-03-26T17:37:27.736915+0000","last_became_active":"2024-03-26T17:36:21.721387+0000","last_became_peered":"2024-03-26T17:36:21.721387+0000","last_unstale":"2024-03-26T17:37:27.736915+0000","last_undegraded":"2024-03-26T17:37:27.736915+0000","last_fullsized":"2024-03-26T17:37:27.736915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:59.493172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626882+0000","last_change":"2024-03-26T17:37:27.740779+0000","last_active":"2024-03-26T17:37:56.626882+0000","last_peered":"2024-03-26T17:37:56.626882+0000","last_clean":"2024-03-26T17:37:56.626882+0000","last_became_active":"2024-03-26T17:37:27.740407+0000","last_became_peered":"2024-03-26T17:37:27.740407+0000","last_unstale":"2024-03-26T17:37:56.626882+0000","last_undegraded":"2024-03-26T17:37:56.626882+0000","last_fullsized":"2024-03-26T17:37:56.626882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:39.526570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744477+0000","last_change":"2024-03-26T17:36:21.724041+0000","last_active":"2024-03-26T17:37:27.744477+0000","last_peered":"2024-03-26T17:37:27.744477+0000","last_clean":"2024-03-26T17:37:27.744477+0000","last_became_active":"2024-03-26T17:36:21.723624+0000","last_became_peered":"2024-03-26T17:36:21.723624+0000","last_unstale":"2024-03-26T17:37:27.744477+0000","last_undegraded":"2024-03-26T17:37:27.744477+0000","last_fullsized":"2024-03-26T17:37:27.744477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:42:24.264844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.630472+0000","last_change":"2024-03-26T17:36:22.738592+0000","last_active":"2024-03-26T17:37:56.630472+0000","last_peered":"2024-03-26T17:37:56.630472+0000","last_clean":"2024-03-26T17:37:56.630472+0000","last_became_active":"2024-03-26T17:36:22.738236+0000","last_became_peered":"2024-03-26T17:36:22.738236+0000","last_unstale":"2024-03-26T17:37:56.630472+0000","last_undegraded":"2024-03-26T17:37:56.630472+0000","last_fullsized":"2024-03-26T17:37:56.630472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:56:06.977720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":294,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.426418+0000","last_change":"2024-03-26T17:36:21.722793+0000","last_active":"2024-03-26T17:38:21.426418+0000","last_peered":"2024-03-26T17:38:21.426418+0000","last_clean":"2024-03-26T17:38:21.426418+0000","last_became_active":"2024-03-26T17:36:21.722482+0000","last_became_peered":"2024-03-26T17:36:21.722482+0000","last_unstale":"2024-03-26T17:38:21.426418+0000","last_undegraded":"2024-03-26T17:38:21.426418+0000","last_fullsized":"2024-03-26T17:38:21.426418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:48:32.107017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66493,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":185,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.607628+0000","last_change":"2024-03-26T17:36:22.746488+0000","last_active":"2024-03-26T17:37:56.607628+0000","last_peered":"2024-03-26T17:37:56.607628+0000","last_clean":"2024-03-26T17:37:56.607628+0000","last_became_active":"2024-03-26T17:36:22.746013+0000","last_became_peered":"2024-03-26T17:36:22.746013+0000","last_unstale":"2024-03-26T17:37:56.607628+0000","last_undegraded":"2024-03-26T17:37:56.607628+0000","last_fullsized":"2024-03-26T17:37:56.607628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:14.792336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742843+0000","last_change":"2024-03-26T17:36:21.724319+0000","last_active":"2024-03-26T17:37:27.742843+0000","last_peered":"2024-03-26T17:37:27.742843+0000","last_clean":"2024-03-26T17:37:27.742843+0000","last_became_active":"2024-03-26T17:36:21.723644+0000","last_became_peered":"2024-03-26T17:36:21.723644+0000","last_unstale":"2024-03-26T17:37:27.742843+0000","last_undegraded":"2024-03-26T17:37:27.742843+0000","last_fullsized":"2024-03-26T17:37:27.742843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:31:01.132797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.625070+0000","last_change":"2024-03-26T17:36:22.736535+0000","last_active":"2024-03-26T17:37:56.625070+0000","last_peered":"2024-03-26T17:37:56.625070+0000","last_clean":"2024-03-26T17:37:56.625070+0000","last_became_active":"2024-03-26T17:36:22.736229+0000","last_became_peered":"2024-03-26T17:36:22.736229+0000","last_unstale":"2024-03-26T17:37:56.625070+0000","last_undegraded":"2024-03-26T17:37:56.625070+0000","last_fullsized":"2024-03-26T17:37:56.625070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:03.938180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742451+0000","last_change":"2024-03-26T17:36:21.726271+0000","last_active":"2024-03-26T17:37:27.742451+0000","last_peered":"2024-03-26T17:37:27.742451+0000","last_clean":"2024-03-26T17:37:27.742451+0000","last_became_active":"2024-03-26T17:36:21.725942+0000","last_became_peered":"2024-03-26T17:36:21.725942+0000","last_unstale":"2024-03-26T17:37:27.742451+0000","last_undegraded":"2024-03-26T17:37:27.742451+0000","last_fullsized":"2024-03-26T17:37:27.742451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:23:16.320106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.633769+0000","last_change":"2024-03-26T17:36:22.743480+0000","last_active":"2024-03-26T17:37:56.633769+0000","last_peered":"2024-03-26T17:37:56.633769+0000","last_clean":"2024-03-26T17:37:56.633769+0000","last_became_active":"2024-03-26T17:36:22.742982+0000","last_became_peered":"2024-03-26T17:36:22.742982+0000","last_unstale":"2024-03-26T17:37:56.633769+0000","last_undegraded":"2024-03-26T17:37:56.633769+0000","last_fullsized":"2024-03-26T17:37:56.633769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:47.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:18.424299+0000","last_change":"2024-03-26T17:36:21.723407+0000","last_active":"2024-03-26T17:38:18.424299+0000","last_peered":"2024-03-26T17:38:18.424299+0000","last_clean":"2024-03-26T17:38:18.424299+0000","last_became_active":"2024-03-26T17:36:21.723086+0000","last_became_peered":"2024-03-26T17:36:21.723086+0000","last_unstale":"2024-03-26T17:38:18.424299+0000","last_undegraded":"2024-03-26T17:38:18.424299+0000","last_fullsized":"2024-03-26T17:38:18.424299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:26:57.886388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631858+0000","last_change":"2024-03-26T17:36:22.740705+0000","last_active":"2024-03-26T17:37:56.631858+0000","last_peered":"2024-03-26T17:37:56.631858+0000","last_clean":"2024-03-26T17:37:56.631858+0000","last_became_active":"2024-03-26T17:36:22.740360+0000","last_became_peered":"2024-03-26T17:36:22.740360+0000","last_unstale":"2024-03-26T17:37:56.631858+0000","last_undegraded":"2024-03-26T17:37:56.631858+0000","last_fullsized":"2024-03-26T17:37:56.631858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:02.879022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.266750+0000","last_change":"2024-03-26T17:37:28.266399+0000","last_active":"2024-03-26T17:37:28.266750+0000","last_peered":"2024-03-26T17:37:28.266750+0000","last_clean":"2024-03-26T17:37:28.266750+0000","last_became_active":"2024-03-26T17:37:28.246359+0000","last_became_peered":"2024-03-26T17:37:28.246359+0000","last_unstale":"2024-03-26T17:37:28.266750+0000","last_undegraded":"2024-03-26T17:37:28.266750+0000","last_fullsized":"2024-03-26T17:37:28.266750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:20:54.895081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.625211+0000","last_change":"2024-03-26T17:36:22.743791+0000","last_active":"2024-03-26T17:37:56.625211+0000","last_peered":"2024-03-26T17:37:56.625211+0000","last_clean":"2024-03-26T17:37:56.625211+0000","last_became_active":"2024-03-26T17:36:22.742780+0000","last_became_peered":"2024-03-26T17:36:22.742780+0000","last_unstale":"2024-03-26T17:37:56.625211+0000","last_undegraded":"2024-03-26T17:37:56.625211+0000","last_fullsized":"2024-03-26T17:37:56.625211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:04.387958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740570+0000","last_change":"2024-03-26T17:36:21.724289+0000","last_active":"2024-03-26T17:37:27.740570+0000","last_peered":"2024-03-26T17:37:27.740570+0000","last_clean":"2024-03-26T17:37:27.740570+0000","last_became_active":"2024-03-26T17:36:21.723847+0000","last_became_peered":"2024-03-26T17:36:21.723847+0000","last_unstale":"2024-03-26T17:37:27.740570+0000","last_undegraded":"2024-03-26T17:37:27.740570+0000","last_fullsized":"2024-03-26T17:37:27.740570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:01:59.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626382+0000","last_change":"2024-03-26T17:36:22.748323+0000","last_active":"2024-03-26T17:37:56.626382+0000","last_peered":"2024-03-26T17:37:56.626382+0000","last_clean":"2024-03-26T17:37:56.626382+0000","last_became_active":"2024-03-26T17:36:22.747970+0000","last_became_peered":"2024-03-26T17:36:22.747970+0000","last_unstale":"2024-03-26T17:37:56.626382+0000","last_undegraded":"2024-03-26T17:37:56.626382+0000","last_fullsized":"2024-03-26T17:37:56.626382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:58.886425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739292+0000","last_change":"2024-03-26T17:36:21.726076+0000","last_active":"2024-03-26T17:37:27.739292+0000","last_peered":"2024-03-26T17:37:27.739292+0000","last_clean":"2024-03-26T17:37:27.739292+0000","last_became_active":"2024-03-26T17:36:21.722067+0000","last_became_peered":"2024-03-26T17:36:21.722067+0000","last_unstale":"2024-03-26T17:37:27.739292+0000","last_undegraded":"2024-03-26T17:37:27.739292+0000","last_fullsized":"2024-03-26T17:37:27.739292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:34:40.316110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628515+0000","last_change":"2024-03-26T17:36:22.730890+0000","last_active":"2024-03-26T17:37:56.628515+0000","last_peered":"2024-03-26T17:37:56.628515+0000","last_clean":"2024-03-26T17:37:56.628515+0000","last_became_active":"2024-03-26T17:36:22.729635+0000","last_became_peered":"2024-03-26T17:36:22.729635+0000","last_unstale":"2024-03-26T17:37:56.628515+0000","last_undegraded":"2024-03-26T17:37:56.628515+0000","last_fullsized":"2024-03-26T17:37:56.628515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:15:09.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'7","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.432102+0000","last_change":"2024-03-26T17:36:21.727587+0000","last_active":"2024-03-26T17:38:21.432102+0000","last_peered":"2024-03-26T17:38:21.432102+0000","last_clean":"2024-03-26T17:38:21.432102+0000","last_became_active":"2024-03-26T17:36:21.727302+0000","last_became_peered":"2024-03-26T17:36:21.727302+0000","last_unstale":"2024-03-26T17:38:21.432102+0000","last_undegraded":"2024-03-26T17:38:21.432102+0000","last_fullsized":"2024-03-26T17:38:21.432102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:59.404567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628589+0000","last_change":"2024-03-26T17:36:22.750654+0000","last_active":"2024-03-26T17:37:56.628589+0000","last_peered":"2024-03-26T17:37:56.628589+0000","last_clean":"2024-03-26T17:37:56.628589+0000","last_became_active":"2024-03-26T17:36:22.750281+0000","last_became_peered":"2024-03-26T17:36:22.750281+0000","last_unstale":"2024-03-26T17:37:56.628589+0000","last_undegraded":"2024-03-26T17:37:56.628589+0000","last_fullsized":"2024-03-26T17:37:56.628589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:04.034108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.619451+0000","last_change":"2024-03-26T17:37:27.745100+0000","last_active":"2024-03-26T17:37:56.619451+0000","last_peered":"2024-03-26T17:37:56.619451+0000","last_clean":"2024-03-26T17:37:56.619451+0000","last_became_active":"2024-03-26T17:37:27.744686+0000","last_became_peered":"2024-03-26T17:37:27.744686+0000","last_unstale":"2024-03-26T17:37:56.619451+0000","last_undegraded":"2024-03-26T17:37:56.619451+0000","last_fullsized":"2024-03-26T17:37:56.619451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:38.111145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":141,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.426775+0000","last_change":"2024-03-26T17:36:21.730812+0000","last_active":"2024-03-26T17:38:21.426775+0000","last_peered":"2024-03-26T17:38:21.426775+0000","last_clean":"2024-03-26T17:38:21.426775+0000","last_became_active":"2024-03-26T17:36:21.730499+0000","last_became_peered":"2024-03-26T17:36:21.730499+0000","last_unstale":"2024-03-26T17:38:21.426775+0000","last_undegraded":"2024-03-26T17:38:21.426775+0000","last_fullsized":"2024-03-26T17:38:21.426775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:05.666146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617217+0000","last_change":"2024-03-26T17:36:22.742344+0000","last_active":"2024-03-26T17:37:56.617217+0000","last_peered":"2024-03-26T17:37:56.617217+0000","last_clean":"2024-03-26T17:37:56.617217+0000","last_became_active":"2024-03-26T17:36:22.741864+0000","last_became_peered":"2024-03-26T17:36:22.741864+0000","last_unstale":"2024-03-26T17:37:56.617217+0000","last_undegraded":"2024-03-26T17:37:56.617217+0000","last_fullsized":"2024-03-26T17:37:56.617217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:53:53.553107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244456+0000","last_change":"2024-03-26T17:36:21.722450+0000","last_active":"2024-03-26T17:37:28.244456+0000","last_peered":"2024-03-26T17:37:28.244456+0000","last_clean":"2024-03-26T17:37:28.244456+0000","last_became_active":"2024-03-26T17:36:21.722105+0000","last_became_peered":"2024-03-26T17:36:21.722105+0000","last_unstale":"2024-03-26T17:37:28.244456+0000","last_undegraded":"2024-03-26T17:37:28.244456+0000","last_fullsized":"2024-03-26T17:37:28.244456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:01:22.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736685+0000","last_change":"2024-03-26T17:35:10.663606+0000","last_active":"2024-03-26T17:37:27.736685+0000","last_peered":"2024-03-26T17:37:27.736685+0000","last_clean":"2024-03-26T17:37:27.736685+0000","last_became_active":"2024-03-26T17:35:10.663069+0000","last_became_peered":"2024-03-26T17:35:10.663069+0000","last_unstale":"2024-03-26T17:37:27.736685+0000","last_undegraded":"2024-03-26T17:37:27.736685+0000","last_fullsized":"2024-03-26T17:37:27.736685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_clean_scrub_stamp":"2024-03-26T17:31:50.880228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:50.817007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.622092+0000","last_change":"2024-03-26T17:37:27.745201+0000","last_active":"2024-03-26T17:37:56.622092+0000","last_peered":"2024-03-26T17:37:56.622092+0000","last_clean":"2024-03-26T17:37:56.622092+0000","last_became_active":"2024-03-26T17:37:27.744806+0000","last_became_peered":"2024-03-26T17:37:27.744806+0000","last_unstale":"2024-03-26T17:37:56.622092+0000","last_undegraded":"2024-03-26T17:37:56.622092+0000","last_fullsized":"2024-03-26T17:37:56.622092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:15:07.485013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736795+0000","last_change":"2024-03-26T17:36:21.723452+0000","last_active":"2024-03-26T17:37:27.736795+0000","last_peered":"2024-03-26T17:37:27.736795+0000","last_clean":"2024-03-26T17:37:27.736795+0000","last_became_active":"2024-03-26T17:36:21.723155+0000","last_became_peered":"2024-03-26T17:36:21.723155+0000","last_unstale":"2024-03-26T17:37:27.736795+0000","last_undegraded":"2024-03-26T17:37:27.736795+0000","last_fullsized":"2024-03-26T17:37:27.736795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:39:34.443624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.750092+0000","last_change":"2024-03-26T17:37:27.749741+0000","last_active":"2024-03-26T17:37:27.750092+0000","last_peered":"2024-03-26T17:37:27.750092+0000","last_clean":"2024-03-26T17:37:27.750092+0000","last_became_active":"2024-03-26T17:37:27.741751+0000","last_became_peered":"2024-03-26T17:37:27.741751+0000","last_unstale":"2024-03-26T17:37:27.750092+0000","last_undegraded":"2024-03-26T17:37:27.750092+0000","last_fullsized":"2024-03-26T17:37:27.750092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:42.119297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621855+0000","last_change":"2024-03-26T17:36:22.736487+0000","last_active":"2024-03-26T17:37:56.621855+0000","last_peered":"2024-03-26T17:37:56.621855+0000","last_clean":"2024-03-26T17:37:56.621855+0000","last_became_active":"2024-03-26T17:36:22.735788+0000","last_became_peered":"2024-03-26T17:36:22.735788+0000","last_unstale":"2024-03-26T17:37:56.621855+0000","last_undegraded":"2024-03-26T17:37:56.621855+0000","last_fullsized":"2024-03-26T17:37:56.621855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:45.240148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:18.756601+0000","last_change":"2024-03-26T17:36:21.720755+0000","last_active":"2024-03-26T17:38:18.756601+0000","last_peered":"2024-03-26T17:38:18.756601+0000","last_clean":"2024-03-26T17:38:18.756601+0000","last_became_active":"2024-03-26T17:36:21.720175+0000","last_became_peered":"2024-03-26T17:36:21.720175+0000","last_unstale":"2024-03-26T17:38:18.756601+0000","last_undegraded":"2024-03-26T17:38:18.756601+0000","last_fullsized":"2024-03-26T17:38:18.756601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:22.458657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624275+0000","last_change":"2024-03-26T17:36:22.740731+0000","last_active":"2024-03-26T17:37:56.624275+0000","last_peered":"2024-03-26T17:37:56.624275+0000","last_clean":"2024-03-26T17:37:56.624275+0000","last_became_active":"2024-03-26T17:36:22.740283+0000","last_became_peered":"2024-03-26T17:36:22.740283+0000","last_unstale":"2024-03-26T17:37:56.624275+0000","last_undegraded":"2024-03-26T17:37:56.624275+0000","last_fullsized":"2024-03-26T17:37:56.624275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:36.403105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244032+0000","last_change":"2024-03-26T17:36:21.721513+0000","last_active":"2024-03-26T17:37:28.244032+0000","last_peered":"2024-03-26T17:37:28.244032+0000","last_clean":"2024-03-26T17:37:28.244032+0000","last_became_active":"2024-03-26T17:36:21.721185+0000","last_became_peered":"2024-03-26T17:36:21.721185+0000","last_unstale":"2024-03-26T17:37:28.244032+0000","last_undegraded":"2024-03-26T17:37:28.244032+0000","last_fullsized":"2024-03-26T17:37:28.244032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:48:39.994980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628787+0000","last_change":"2024-03-26T17:36:22.737938+0000","last_active":"2024-03-26T17:37:56.628787+0000","last_peered":"2024-03-26T17:37:56.628787+0000","last_clean":"2024-03-26T17:37:56.628787+0000","last_became_active":"2024-03-26T17:36:22.737605+0000","last_became_peered":"2024-03-26T17:36:22.737605+0000","last_unstale":"2024-03-26T17:37:56.628787+0000","last_undegraded":"2024-03-26T17:37:56.628787+0000","last_fullsized":"2024-03-26T17:37:56.628787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:27.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245677+0000","last_change":"2024-03-26T17:36:21.720942+0000","last_active":"2024-03-26T17:37:28.245677+0000","last_peered":"2024-03-26T17:37:28.245677+0000","last_clean":"2024-03-26T17:37:28.245677+0000","last_became_active":"2024-03-26T17:36:21.720665+0000","last_became_peered":"2024-03-26T17:36:21.720665+0000","last_unstale":"2024-03-26T17:37:28.245677+0000","last_undegraded":"2024-03-26T17:37:28.245677+0000","last_fullsized":"2024-03-26T17:37:28.245677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:00.725213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.587257+0000","last_change":"2024-03-26T17:36:22.747111+0000","last_active":"2024-03-26T17:37:56.587257+0000","last_peered":"2024-03-26T17:37:56.587257+0000","last_clean":"2024-03-26T17:37:56.587257+0000","last_became_active":"2024-03-26T17:36:22.746626+0000","last_became_peered":"2024-03-26T17:36:22.746626+0000","last_unstale":"2024-03-26T17:37:56.587257+0000","last_undegraded":"2024-03-26T17:37:56.587257+0000","last_fullsized":"2024-03-26T17:37:56.587257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:29:24.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741395+0000","last_change":"2024-03-26T17:36:21.724141+0000","last_active":"2024-03-26T17:37:27.741395+0000","last_peered":"2024-03-26T17:37:27.741395+0000","last_clean":"2024-03-26T17:37:27.741395+0000","last_became_active":"2024-03-26T17:36:21.723797+0000","last_became_peered":"2024-03-26T17:36:21.723797+0000","last_unstale":"2024-03-26T17:37:27.741395+0000","last_undegraded":"2024-03-26T17:37:27.741395+0000","last_fullsized":"2024-03-26T17:37:27.741395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:06.149827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.627752+0000","last_change":"2024-03-26T17:36:22.739497+0000","last_active":"2024-03-26T17:37:56.627752+0000","last_peered":"2024-03-26T17:37:56.627752+0000","last_clean":"2024-03-26T17:37:56.627752+0000","last_became_active":"2024-03-26T17:36:22.739114+0000","last_became_peered":"2024-03-26T17:36:22.739114+0000","last_unstale":"2024-03-26T17:37:56.627752+0000","last_undegraded":"2024-03-26T17:37:56.627752+0000","last_fullsized":"2024-03-26T17:37:56.627752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:03.953199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.743064+0000","last_change":"2024-03-26T17:36:21.728027+0000","last_active":"2024-03-26T17:37:27.743064+0000","last_peered":"2024-03-26T17:37:27.743064+0000","last_clean":"2024-03-26T17:37:27.743064+0000","last_became_active":"2024-03-26T17:36:21.727159+0000","last_became_peered":"2024-03-26T17:36:21.727159+0000","last_unstale":"2024-03-26T17:37:27.743064+0000","last_undegraded":"2024-03-26T17:37:27.743064+0000","last_fullsized":"2024-03-26T17:37:27.743064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:39.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628518+0000","last_change":"2024-03-26T17:36:22.733666+0000","last_active":"2024-03-26T17:37:56.628518+0000","last_peered":"2024-03-26T17:37:56.628518+0000","last_clean":"2024-03-26T17:37:56.628518+0000","last_became_active":"2024-03-26T17:36:22.733313+0000","last_became_peered":"2024-03-26T17:36:22.733313+0000","last_unstale":"2024-03-26T17:37:56.628518+0000","last_undegraded":"2024-03-26T17:37:56.628518+0000","last_fullsized":"2024-03-26T17:37:56.628518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:52:38.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739591+0000","last_change":"2024-03-26T17:36:21.730828+0000","last_active":"2024-03-26T17:37:27.739591+0000","last_peered":"2024-03-26T17:37:27.739591+0000","last_clean":"2024-03-26T17:37:27.739591+0000","last_became_active":"2024-03-26T17:36:21.730508+0000","last_became_peered":"2024-03-26T17:36:21.730508+0000","last_unstale":"2024-03-26T17:37:27.739591+0000","last_undegraded":"2024-03-26T17:37:27.739591+0000","last_fullsized":"2024-03-26T17:37:27.739591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:10:53.492725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631929+0000","last_change":"2024-03-26T17:36:22.740204+0000","last_active":"2024-03-26T17:37:56.631929+0000","last_peered":"2024-03-26T17:37:56.631929+0000","last_clean":"2024-03-26T17:37:56.631929+0000","last_became_active":"2024-03-26T17:36:22.735880+0000","last_became_peered":"2024-03-26T17:36:22.735880+0000","last_unstale":"2024-03-26T17:37:56.631929+0000","last_undegraded":"2024-03-26T17:37:56.631929+0000","last_fullsized":"2024-03-26T17:37:56.631929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:34.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":144,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:19.425044+0000","last_change":"2024-03-26T17:36:21.720873+0000","last_active":"2024-03-26T17:38:19.425044+0000","last_peered":"2024-03-26T17:38:19.425044+0000","last_clean":"2024-03-26T17:38:19.425044+0000","last_became_active":"2024-03-26T17:36:21.720496+0000","last_became_peered":"2024-03-26T17:36:21.720496+0000","last_unstale":"2024-03-26T17:38:19.425044+0000","last_undegraded":"2024-03-26T17:38:19.425044+0000","last_fullsized":"2024-03-26T17:38:19.425044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:40:55.959064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2195,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.632104+0000","last_change":"2024-03-26T17:36:22.735528+0000","last_active":"2024-03-26T17:37:56.632104+0000","last_peered":"2024-03-26T17:37:56.632104+0000","last_clean":"2024-03-26T17:37:56.632104+0000","last_became_active":"2024-03-26T17:36:22.735147+0000","last_became_peered":"2024-03-26T17:36:22.735147+0000","last_unstale":"2024-03-26T17:37:56.632104+0000","last_undegraded":"2024-03-26T17:37:56.632104+0000","last_fullsized":"2024-03-26T17:37:56.632104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:15.729853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744355+0000","last_change":"2024-03-26T17:36:21.723246+0000","last_active":"2024-03-26T17:37:27.744355+0000","last_peered":"2024-03-26T17:37:27.744355+0000","last_clean":"2024-03-26T17:37:27.744355+0000","last_became_active":"2024-03-26T17:36:21.722839+0000","last_became_peered":"2024-03-26T17:36:21.722839+0000","last_unstale":"2024-03-26T17:37:27.744355+0000","last_undegraded":"2024-03-26T17:37:27.744355+0000","last_fullsized":"2024-03-26T17:37:27.744355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:41:05.540206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629489+0000","last_change":"2024-03-26T17:36:22.742469+0000","last_active":"2024-03-26T17:37:56.629489+0000","last_peered":"2024-03-26T17:37:56.629489+0000","last_clean":"2024-03-26T17:37:56.629489+0000","last_became_active":"2024-03-26T17:36:22.742073+0000","last_became_peered":"2024-03-26T17:36:22.742073+0000","last_unstale":"2024-03-26T17:37:56.629489+0000","last_undegraded":"2024-03-26T17:37:56.629489+0000","last_fullsized":"2024-03-26T17:37:56.629489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:19:33.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.250123+0000","last_change":"2024-03-26T17:37:28.250123+0000","last_active":"2024-03-26T17:37:28.250123+0000","last_peered":"2024-03-26T17:37:28.250123+0000","last_clean":"2024-03-26T17:37:28.250123+0000","last_became_active":"2024-03-26T17:37:28.246162+0000","last_became_peered":"2024-03-26T17:37:28.246162+0000","last_unstale":"2024-03-26T17:37:28.250123+0000","last_undegraded":"2024-03-26T17:37:28.250123+0000","last_fullsized":"2024-03-26T17:37:28.250123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:33:39.002192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628828+0000","last_change":"2024-03-26T17:36:22.742614+0000","last_active":"2024-03-26T17:37:56.628828+0000","last_peered":"2024-03-26T17:37:56.628828+0000","last_clean":"2024-03-26T17:37:56.628828+0000","last_became_active":"2024-03-26T17:36:22.742278+0000","last_became_peered":"2024-03-26T17:36:22.742278+0000","last_unstale":"2024-03-26T17:37:56.628828+0000","last_undegraded":"2024-03-26T17:37:56.628828+0000","last_fullsized":"2024-03-26T17:37:56.628828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:30:14.503474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.627061+0000","last_change":"2024-03-26T17:36:22.741693+0000","last_active":"2024-03-26T17:37:56.627061+0000","last_peered":"2024-03-26T17:37:56.627061+0000","last_clean":"2024-03-26T17:37:56.627061+0000","last_became_active":"2024-03-26T17:36:22.741352+0000","last_became_peered":"2024-03-26T17:36:22.741352+0000","last_unstale":"2024-03-26T17:37:56.627061+0000","last_undegraded":"2024-03-26T17:37:56.627061+0000","last_fullsized":"2024-03-26T17:37:56.627061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:13:26.912910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.623353+0000","last_change":"2024-03-26T17:36:22.742939+0000","last_active":"2024-03-26T17:37:56.623353+0000","last_peered":"2024-03-26T17:37:56.623353+0000","last_clean":"2024-03-26T17:37:56.623353+0000","last_became_active":"2024-03-26T17:36:22.742589+0000","last_became_peered":"2024-03-26T17:36:22.742589+0000","last_unstale":"2024-03-26T17:37:56.623353+0000","last_undegraded":"2024-03-26T17:37:56.623353+0000","last_fullsized":"2024-03-26T17:37:56.623353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:21:14.043601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617673+0000","last_change":"2024-03-26T17:36:22.748044+0000","last_active":"2024-03-26T17:37:56.617673+0000","last_peered":"2024-03-26T17:37:56.617673+0000","last_clean":"2024-03-26T17:37:56.617673+0000","last_became_active":"2024-03-26T17:36:22.747683+0000","last_became_peered":"2024-03-26T17:36:22.747683+0000","last_unstale":"2024-03-26T17:37:56.617673+0000","last_undegraded":"2024-03-26T17:37:56.617673+0000","last_fullsized":"2024-03-26T17:37:56.617673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:32:40.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628303+0000","last_change":"2024-03-26T17:36:22.743080+0000","last_active":"2024-03-26T17:37:56.628303+0000","last_peered":"2024-03-26T17:37:56.628303+0000","last_clean":"2024-03-26T17:37:56.628303+0000","last_became_active":"2024-03-26T17:36:22.742676+0000","last_became_peered":"2024-03-26T17:36:22.742676+0000","last_unstale":"2024-03-26T17:37:56.628303+0000","last_undegraded":"2024-03-26T17:37:56.628303+0000","last_fullsized":"2024-03-26T17:37:56.628303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:58.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628807+0000","last_change":"2024-03-26T17:36:22.737908+0000","last_active":"2024-03-26T17:37:56.628807+0000","last_peered":"2024-03-26T17:37:56.628807+0000","last_clean":"2024-03-26T17:37:56.628807+0000","last_became_active":"2024-03-26T17:36:22.737055+0000","last_became_peered":"2024-03-26T17:36:22.737055+0000","last_unstale":"2024-03-26T17:37:56.628807+0000","last_undegraded":"2024-03-26T17:37:56.628807+0000","last_fullsized":"2024-03-26T17:37:56.628807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:51:13.157901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621037+0000","last_change":"2024-03-26T17:36:22.749795+0000","last_active":"2024-03-26T17:37:56.621037+0000","last_peered":"2024-03-26T17:37:56.621037+0000","last_clean":"2024-03-26T17:37:56.621037+0000","last_became_active":"2024-03-26T17:36:22.749469+0000","last_became_peered":"2024-03-26T17:36:22.749469+0000","last_unstale":"2024-03-26T17:37:56.621037+0000","last_undegraded":"2024-03-26T17:37:56.621037+0000","last_fullsized":"2024-03-26T17:37:56.621037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:19:59.126270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.633721+0000","last_change":"2024-03-26T17:36:22.744101+0000","last_active":"2024-03-26T17:37:56.633721+0000","last_peered":"2024-03-26T17:37:56.633721+0000","last_clean":"2024-03-26T17:37:56.633721+0000","last_became_active":"2024-03-26T17:36:22.743761+0000","last_became_peered":"2024-03-26T17:36:22.743761+0000","last_unstale":"2024-03-26T17:37:56.633721+0000","last_undegraded":"2024-03-26T17:37:56.633721+0000","last_fullsized":"2024-03-26T17:37:56.633721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:27:19.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617497+0000","last_change":"2024-03-26T17:36:22.744800+0000","last_active":"2024-03-26T17:37:56.617497+0000","last_peered":"2024-03-26T17:37:56.617497+0000","last_clean":"2024-03-26T17:37:56.617497+0000","last_became_active":"2024-03-26T17:36:22.744437+0000","last_became_peered":"2024-03-26T17:36:22.744437+0000","last_unstale":"2024-03-26T17:37:56.617497+0000","last_undegraded":"2024-03-26T17:37:56.617497+0000","last_fullsized":"2024-03-26T17:37:56.617497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:36.459072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.620441+0000","last_change":"2024-03-26T17:37:27.745243+0000","last_active":"2024-03-26T17:37:56.620441+0000","last_peered":"2024-03-26T17:37:56.620441+0000","last_clean":"2024-03-26T17:37:56.620441+0000","last_became_active":"2024-03-26T17:37:27.744869+0000","last_became_peered":"2024-03-26T17:37:27.744869+0000","last_unstale":"2024-03-26T17:37:56.620441+0000","last_undegraded":"2024-03-26T17:37:56.620441+0000","last_fullsized":"2024-03-26T17:37:56.620441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:18:42.188733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.623784+0000","last_change":"2024-03-26T17:36:22.743211+0000","last_active":"2024-03-26T17:37:56.623784+0000","last_peered":"2024-03-26T17:37:56.623784+0000","last_clean":"2024-03-26T17:37:56.623784+0000","last_became_active":"2024-03-26T17:36:22.742869+0000","last_became_peered":"2024-03-26T17:36:22.742869+0000","last_unstale":"2024-03-26T17:37:56.623784+0000","last_undegraded":"2024-03-26T17:37:56.623784+0000","last_fullsized":"2024-03-26T17:37:56.623784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:24:04.610012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635257+0000","last_change":"2024-03-26T17:36:22.746950+0000","last_active":"2024-03-26T17:37:56.635257+0000","last_peered":"2024-03-26T17:37:56.635257+0000","last_clean":"2024-03-26T17:37:56.635257+0000","last_became_active":"2024-03-26T17:36:22.744869+0000","last_became_peered":"2024-03-26T17:36:22.744869+0000","last_unstale":"2024-03-26T17:37:56.635257+0000","last_undegraded":"2024-03-26T17:37:56.635257+0000","last_fullsized":"2024-03-26T17:37:56.635257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:09:52.283519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629292+0000","last_change":"2024-03-26T17:36:22.744627+0000","last_active":"2024-03-26T17:37:56.629292+0000","last_peered":"2024-03-26T17:37:56.629292+0000","last_clean":"2024-03-26T17:37:56.629292+0000","last_became_active":"2024-03-26T17:36:22.744299+0000","last_became_peered":"2024-03-26T17:36:22.744299+0000","last_unstale":"2024-03-26T17:37:56.629292+0000","last_undegraded":"2024-03-26T17:37:56.629292+0000","last_fullsized":"2024-03-26T17:37:56.629292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:23:37.941625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626943+0000","last_change":"2024-03-26T17:36:22.751135+0000","last_active":"2024-03-26T17:37:56.626943+0000","last_peered":"2024-03-26T17:37:56.626943+0000","last_clean":"2024-03-26T17:37:56.626943+0000","last_became_active":"2024-03-26T17:36:22.750768+0000","last_became_peered":"2024-03-26T17:36:22.750768+0000","last_unstale":"2024-03-26T17:37:56.626943+0000","last_undegraded":"2024-03-26T17:37:56.626943+0000","last_fullsized":"2024-03-26T17:37:56.626943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:15.532367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75810,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1181,"num_read_kb":676,"num_write":263,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":217629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1109624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32708,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712444,"statfs":{"total":95995035648,"available":95961542656,"internally_reserved":0,"allocated":1568768,"data_stored":1050619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":519267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":458325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":520288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":455786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1048920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T17:38:24.843 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-26T17:38:24.843 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T17:38:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 -- 172.21.15.105:0/2350404405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408104cf0 msgr2=0x7f54081070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 --2- 172.21.15.105:0/2350404405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408104cf0 0x7f54081070e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009900 tx=0x7f53fc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 -- 172.21.15.105:0/2350404405 shutdown_connections 2024-03-26T17:38:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 --2- 172.21.15.105:0/2350404405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5408107620 0x7f5408109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 --2- 172.21.15.105:0/2350404405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408104cf0 0x7f54081070e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 --2- 172.21.15.105:0/2350404405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54081023c0 0x7f54081047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 -- 172.21.15.105:0/2350404405 >> 172.21.15.105:0/2350404405 conn(0x7f54080fbd50 msgr2=0x7f54080fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 -- 172.21.15.105:0/2350404405 shutdown_connections 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 -- 172.21.15.105:0/2350404405 wait complete. 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 Processor -- start 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.016+0000 7f540e826640 1 -- start start 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540e826640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54081023c0 0x7f5408197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540e826640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5408104cf0 0x7f54081980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540e826640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408107620 0x7f540819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540e826640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f540810d410 con 0x7f5408107620 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540e826640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f540810d290 con 0x7f5408104cf0 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540e826640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f540810d590 con 0x7f54081023c0 2024-03-26T17:38:25.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540cdc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408107620 0x7f540819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f5407fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54081023c0 0x7f5408197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540cdc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408107620 0x7f540819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55858/0 (socket says 172.21.15.105:55858) 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540cdc3640 1 -- 172.21.15.105:0/3091116120 learned_addr learned my addr 172.21.15.105:0/3091116120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f54077fe640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5408104cf0 0x7f54081980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f5407fff640 1 -- 172.21.15.105:0/3091116120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5408104cf0 msgr2=0x7f54081980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f5407fff640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5408104cf0 0x7f54081980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f5407fff640 1 -- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408107620 msgr2=0x7f540819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f5407fff640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408107620 0x7f540819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f5407fff640 1 -- 172.21.15.105:0/3091116120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540819f780 con 0x7f54081023c0 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f540cdc3640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408107620 0x7f540819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.017+0000 7f5407fff640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54081023c0 0x7f5408197bb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53f800e9f0 tx=0x7f53f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:25.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.018+0000 7f54057fa640 1 -- 172.21.15.105:0/3091116120 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f800ccc0 con 0x7f54081023c0 2024-03-26T17:38:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.018+0000 7f54057fa640 1 -- 172.21.15.105:0/3091116120 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f80108e0 con 0x7f54081023c0 2024-03-26T17:38:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.018+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540819fa70 con 0x7f54081023c0 2024-03-26T17:38:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.018+0000 7f54057fa640 1 -- 172.21.15.105:0/3091116120 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8018880 con 0x7f54081023c0 2024-03-26T17:38:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.018+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540819ffe0 con 0x7f54081023c0 2024-03-26T17:38:25.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.018+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54080fd840 con 0x7f54081023c0 2024-03-26T17:38:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.020+0000 7f54057fa640 1 -- 172.21.15.105:0/3091116120 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f53f8010a90 con 0x7f54081023c0 2024-03-26T17:38:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.020+0000 7f54057fa640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53f00a3030 0x7f53f00a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.020+0000 7f54077fe640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53f00a3030 0x7f53f00a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.021+0000 7f54057fa640 1 -- 172.21.15.105:0/3091116120 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f53f80cfaf0 con 0x7f54081023c0 2024-03-26T17:38:25.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.021+0000 7f54077fe640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53f00a3030 0x7f53f00a54f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005ab0 tx=0x7f53fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:25.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.024+0000 7f54057fa640 1 -- 172.21.15.105:0/3091116120 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f53f809c190 con 0x7f54081023c0 2024-03-26T17:38:25.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:24 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4003256202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:25.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:24 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1489888727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:25.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:24 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4003256202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:25.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:24 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1489888727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:25.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:24 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4003256202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:25.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:24 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1489888727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:25.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.202+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 --> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5408061d00 con 0x7f53f00a3030 2024-03-26T17:38:25.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.209+0000 7f54057fa640 1 -- 172.21.15.105:0/3091116120 <== mgr.14152 v2:172.21.15.105:6800/2294330938 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258897 (secure 0 0 0) 0x7f5408061d00 con 0x7f53f00a3030 2024-03-26T17:38:25.204 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:25.207 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53f00a3030 msgr2=0x7f53f00a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53f00a3030 0x7f53f00a54f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005ab0 tx=0x7f53fc005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54081023c0 msgr2=0x7f5408197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54081023c0 0x7f5408197bb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53f800e9f0 tx=0x7f53f800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 shutdown_connections 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53f00a3030 0x7f53f00a54f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5408107620 0x7f540819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5408104cf0 0x7f54081980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 --2- 172.21.15.105:0/3091116120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54081023c0 0x7f5408197bb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 >> 172.21.15.105:0/3091116120 conn(0x7f54080fbd50 msgr2=0x7f5408102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:25.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 shutdown_connections 2024-03-26T17:38:25.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.212+0000 7f540e826640 1 -- 172.21.15.105:0/3091116120 wait complete. 2024-03-26T17:38:25.220 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-03-26T17:38:24.609618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666412,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1307,"num_read_kb":785,"num_write":446,"num_write_kb":3097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180960,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366620,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575204,"statfs":{"total":1151940427776,"available":1151565008896,"internally_reserved":0,"allocated":13627392,"data_stored":7446194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7561,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":71,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002616"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628144+0000","last_change":"2024-03-26T17:36:22.743815+0000","last_active":"2024-03-26T17:37:56.628144+0000","last_peered":"2024-03-26T17:37:56.628144+0000","last_clean":"2024-03-26T17:37:56.628144+0000","last_became_active":"2024-03-26T17:36:22.743446+0000","last_became_peered":"2024-03-26T17:36:22.743446+0000","last_unstale":"2024-03-26T17:37:56.628144+0000","last_undegraded":"2024-03-26T17:37:56.628144+0000","last_fullsized":"2024-03-26T17:37:56.628144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:16:56.773849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628208+0000","last_change":"2024-03-26T17:36:22.740754+0000","last_active":"2024-03-26T17:37:56.628208+0000","last_peered":"2024-03-26T17:37:56.628208+0000","last_clean":"2024-03-26T17:37:56.628208+0000","last_became_active":"2024-03-26T17:36:22.740317+0000","last_became_peered":"2024-03-26T17:36:22.740317+0000","last_unstale":"2024-03-26T17:37:56.628208+0000","last_undegraded":"2024-03-26T17:37:56.628208+0000","last_fullsized":"2024-03-26T17:37:56.628208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:50:45.321653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635496+0000","last_change":"2024-03-26T17:36:22.735079+0000","last_active":"2024-03-26T17:37:56.635496+0000","last_peered":"2024-03-26T17:37:56.635496+0000","last_clean":"2024-03-26T17:37:56.635496+0000","last_became_active":"2024-03-26T17:36:22.734646+0000","last_became_peered":"2024-03-26T17:36:22.734646+0000","last_unstale":"2024-03-26T17:37:56.635496+0000","last_undegraded":"2024-03-26T17:37:56.635496+0000","last_fullsized":"2024-03-26T17:37:56.635496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:41:09.477438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.615398+0000","last_change":"2024-03-26T17:36:22.746170+0000","last_active":"2024-03-26T17:37:56.615398+0000","last_peered":"2024-03-26T17:37:56.615398+0000","last_clean":"2024-03-26T17:37:56.615398+0000","last_became_active":"2024-03-26T17:36:22.744255+0000","last_became_peered":"2024-03-26T17:36:22.744255+0000","last_unstale":"2024-03-26T17:37:56.615398+0000","last_undegraded":"2024-03-26T17:37:56.615398+0000","last_fullsized":"2024-03-26T17:37:56.615398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:54:36.225926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.607889+0000","last_change":"2024-03-26T17:36:22.737390+0000","last_active":"2024-03-26T17:37:56.607889+0000","last_peered":"2024-03-26T17:37:56.607889+0000","last_clean":"2024-03-26T17:37:56.607889+0000","last_became_active":"2024-03-26T17:36:22.737038+0000","last_became_peered":"2024-03-26T17:36:22.737038+0000","last_unstale":"2024-03-26T17:37:56.607889+0000","last_undegraded":"2024-03-26T17:37:56.607889+0000","last_fullsized":"2024-03-26T17:37:56.607889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:48.812691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.634748+0000","last_change":"2024-03-26T17:36:22.748097+0000","last_active":"2024-03-26T17:37:56.634748+0000","last_peered":"2024-03-26T17:37:56.634748+0000","last_clean":"2024-03-26T17:37:56.634748+0000","last_became_active":"2024-03-26T17:36:22.747730+0000","last_became_peered":"2024-03-26T17:36:22.747730+0000","last_unstale":"2024-03-26T17:37:56.634748+0000","last_undegraded":"2024-03-26T17:37:56.634748+0000","last_fullsized":"2024-03-26T17:37:56.634748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:43:01.370199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.620104+0000","last_change":"2024-03-26T17:36:22.739587+0000","last_active":"2024-03-26T17:37:56.620104+0000","last_peered":"2024-03-26T17:37:56.620104+0000","last_clean":"2024-03-26T17:37:56.620104+0000","last_became_active":"2024-03-26T17:36:22.735787+0000","last_became_peered":"2024-03-26T17:36:22.735787+0000","last_unstale":"2024-03-26T17:37:56.620104+0000","last_undegraded":"2024-03-26T17:37:56.620104+0000","last_fullsized":"2024-03-26T17:37:56.620104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:41.381153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621279+0000","last_change":"2024-03-26T17:36:22.736716+0000","last_active":"2024-03-26T17:37:56.621279+0000","last_peered":"2024-03-26T17:37:56.621279+0000","last_clean":"2024-03-26T17:37:56.621279+0000","last_became_active":"2024-03-26T17:36:22.736327+0000","last_became_peered":"2024-03-26T17:36:22.736327+0000","last_unstale":"2024-03-26T17:37:56.621279+0000","last_undegraded":"2024-03-26T17:37:56.621279+0000","last_fullsized":"2024-03-26T17:37:56.621279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:10:43.439524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635164+0000","last_change":"2024-03-26T17:36:22.746355+0000","last_active":"2024-03-26T17:37:56.635164+0000","last_peered":"2024-03-26T17:37:56.635164+0000","last_clean":"2024-03-26T17:37:56.635164+0000","last_became_active":"2024-03-26T17:36:22.745363+0000","last_became_peered":"2024-03-26T17:36:22.745363+0000","last_unstale":"2024-03-26T17:37:56.635164+0000","last_undegraded":"2024-03-26T17:37:56.635164+0000","last_fullsized":"2024-03-26T17:37:56.635164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:56:33.528005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.634951+0000","last_change":"2024-03-26T17:36:22.749009+0000","last_active":"2024-03-26T17:37:56.634951+0000","last_peered":"2024-03-26T17:37:56.634951+0000","last_clean":"2024-03-26T17:37:56.634951+0000","last_became_active":"2024-03-26T17:36:22.748627+0000","last_became_peered":"2024-03-26T17:36:22.748627+0000","last_unstale":"2024-03-26T17:37:56.634951+0000","last_undegraded":"2024-03-26T17:37:56.634951+0000","last_fullsized":"2024-03-26T17:37:56.634951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:59:14.812316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631878+0000","last_change":"2024-03-26T17:36:22.741294+0000","last_active":"2024-03-26T17:37:56.631878+0000","last_peered":"2024-03-26T17:37:56.631878+0000","last_clean":"2024-03-26T17:37:56.631878+0000","last_became_active":"2024-03-26T17:36:22.740003+0000","last_became_peered":"2024-03-26T17:36:22.740003+0000","last_unstale":"2024-03-26T17:37:56.631878+0000","last_undegraded":"2024-03-26T17:37:56.631878+0000","last_fullsized":"2024-03-26T17:37:56.631878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:06.305195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629278+0000","last_change":"2024-03-26T17:36:22.745884+0000","last_active":"2024-03-26T17:37:56.629278+0000","last_peered":"2024-03-26T17:37:56.629278+0000","last_clean":"2024-03-26T17:37:56.629278+0000","last_became_active":"2024-03-26T17:36:22.744168+0000","last_became_peered":"2024-03-26T17:36:22.744168+0000","last_unstale":"2024-03-26T17:37:56.629278+0000","last_undegraded":"2024-03-26T17:37:56.629278+0000","last_fullsized":"2024-03-26T17:37:56.629278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:20:50.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624550+0000","last_change":"2024-03-26T17:36:22.725956+0000","last_active":"2024-03-26T17:37:56.624550+0000","last_peered":"2024-03-26T17:37:56.624550+0000","last_clean":"2024-03-26T17:37:56.624550+0000","last_became_active":"2024-03-26T17:36:22.725513+0000","last_became_peered":"2024-03-26T17:36:22.725513+0000","last_unstale":"2024-03-26T17:37:56.624550+0000","last_undegraded":"2024-03-26T17:37:56.624550+0000","last_fullsized":"2024-03-26T17:37:56.624550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:30:10.914233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628784+0000","last_change":"2024-03-26T17:36:22.739757+0000","last_active":"2024-03-26T17:37:56.628784+0000","last_peered":"2024-03-26T17:37:56.628784+0000","last_clean":"2024-03-26T17:37:56.628784+0000","last_became_active":"2024-03-26T17:36:22.739342+0000","last_became_peered":"2024-03-26T17:36:22.739342+0000","last_unstale":"2024-03-26T17:37:56.628784+0000","last_undegraded":"2024-03-26T17:37:56.628784+0000","last_fullsized":"2024-03-26T17:37:56.628784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:28.171325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.618872+0000","last_change":"2024-03-26T17:36:22.742988+0000","last_active":"2024-03-26T17:37:56.618872+0000","last_peered":"2024-03-26T17:37:56.618872+0000","last_clean":"2024-03-26T17:37:56.618872+0000","last_became_active":"2024-03-26T17:36:22.742639+0000","last_became_peered":"2024-03-26T17:36:22.742639+0000","last_unstale":"2024-03-26T17:37:56.618872+0000","last_undegraded":"2024-03-26T17:37:56.618872+0000","last_fullsized":"2024-03-26T17:37:56.618872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:48:44.204889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635451+0000","last_change":"2024-03-26T17:36:22.744530+0000","last_active":"2024-03-26T17:37:56.635451+0000","last_peered":"2024-03-26T17:37:56.635451+0000","last_clean":"2024-03-26T17:37:56.635451+0000","last_became_active":"2024-03-26T17:36:22.743566+0000","last_became_peered":"2024-03-26T17:36:22.743566+0000","last_unstale":"2024-03-26T17:37:56.635451+0000","last_undegraded":"2024-03-26T17:37:56.635451+0000","last_fullsized":"2024-03-26T17:37:56.635451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:42.685326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629288+0000","last_change":"2024-03-26T17:36:22.745797+0000","last_active":"2024-03-26T17:37:56.629288+0000","last_peered":"2024-03-26T17:37:56.629288+0000","last_clean":"2024-03-26T17:37:56.629288+0000","last_became_active":"2024-03-26T17:36:22.745448+0000","last_became_peered":"2024-03-26T17:36:22.745448+0000","last_unstale":"2024-03-26T17:37:56.629288+0000","last_undegraded":"2024-03-26T17:37:56.629288+0000","last_fullsized":"2024-03-26T17:37:56.629288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:05:29.126564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631670+0000","last_change":"2024-03-26T17:36:22.749890+0000","last_active":"2024-03-26T17:37:56.631670+0000","last_peered":"2024-03-26T17:37:56.631670+0000","last_clean":"2024-03-26T17:37:56.631670+0000","last_became_active":"2024-03-26T17:36:22.749497+0000","last_became_peered":"2024-03-26T17:36:22.749497+0000","last_unstale":"2024-03-26T17:37:56.631670+0000","last_undegraded":"2024-03-26T17:37:56.631670+0000","last_fullsized":"2024-03-26T17:37:56.631670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:12:55.069659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.612760+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:56.612760+0000","last_peered":"2024-03-26T17:37:56.612760+0000","last_clean":"2024-03-26T17:37:56.612760+0000","last_became_active":"2024-03-26T17:36:22.740287+0000","last_became_peered":"2024-03-26T17:36:22.740287+0000","last_unstale":"2024-03-26T17:37:56.612760+0000","last_undegraded":"2024-03-26T17:37:56.612760+0000","last_fullsized":"2024-03-26T17:37:56.612760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:13:00.574246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.431274+0000","last_change":"2024-03-26T17:36:21.720809+0000","last_active":"2024-03-26T17:38:21.431274+0000","last_peered":"2024-03-26T17:38:21.431274+0000","last_clean":"2024-03-26T17:38:21.431274+0000","last_became_active":"2024-03-26T17:36:21.720526+0000","last_became_peered":"2024-03-26T17:36:21.720526+0000","last_unstale":"2024-03-26T17:38:21.431274+0000","last_undegraded":"2024-03-26T17:38:21.431274+0000","last_fullsized":"2024-03-26T17:38:21.431274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:48:48.034328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.615928+0000","last_change":"2024-03-26T17:36:22.746245+0000","last_active":"2024-03-26T17:37:56.615928+0000","last_peered":"2024-03-26T17:37:56.615928+0000","last_clean":"2024-03-26T17:37:56.615928+0000","last_became_active":"2024-03-26T17:36:22.745859+0000","last_became_peered":"2024-03-26T17:36:22.745859+0000","last_unstale":"2024-03-26T17:37:56.615928+0000","last_undegraded":"2024-03-26T17:37:56.615928+0000","last_fullsized":"2024-03-26T17:37:56.615928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:39.085668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741831+0000","last_change":"2024-03-26T17:36:21.723178+0000","last_active":"2024-03-26T17:37:27.741831+0000","last_peered":"2024-03-26T17:37:27.741831+0000","last_clean":"2024-03-26T17:37:27.741831+0000","last_became_active":"2024-03-26T17:36:21.722858+0000","last_became_peered":"2024-03-26T17:36:21.722858+0000","last_unstale":"2024-03-26T17:37:27.741831+0000","last_undegraded":"2024-03-26T17:37:27.741831+0000","last_fullsized":"2024-03-26T17:37:27.741831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:46:13.837672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635805+0000","last_change":"2024-03-26T17:36:22.736231+0000","last_active":"2024-03-26T17:37:56.635805+0000","last_peered":"2024-03-26T17:37:56.635805+0000","last_clean":"2024-03-26T17:37:56.635805+0000","last_became_active":"2024-03-26T17:36:22.735942+0000","last_became_peered":"2024-03-26T17:36:22.735942+0000","last_unstale":"2024-03-26T17:37:56.635805+0000","last_undegraded":"2024-03-26T17:37:56.635805+0000","last_fullsized":"2024-03-26T17:37:56.635805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:44:49.127382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740597+0000","last_change":"2024-03-26T17:36:21.731369+0000","last_active":"2024-03-26T17:37:27.740597+0000","last_peered":"2024-03-26T17:37:27.740597+0000","last_clean":"2024-03-26T17:37:27.740597+0000","last_became_active":"2024-03-26T17:36:21.731016+0000","last_became_peered":"2024-03-26T17:36:21.731016+0000","last_unstale":"2024-03-26T17:37:27.740597+0000","last_undegraded":"2024-03-26T17:37:27.740597+0000","last_fullsized":"2024-03-26T17:37:27.740597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:37:53.204825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635677+0000","last_change":"2024-03-26T17:36:22.751155+0000","last_active":"2024-03-26T17:37:56.635677+0000","last_peered":"2024-03-26T17:37:56.635677+0000","last_clean":"2024-03-26T17:37:56.635677+0000","last_became_active":"2024-03-26T17:36:22.750704+0000","last_became_peered":"2024-03-26T17:36:22.750704+0000","last_unstale":"2024-03-26T17:37:56.635677+0000","last_undegraded":"2024-03-26T17:37:56.635677+0000","last_fullsized":"2024-03-26T17:37:56.635677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:52:24.116923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739808+0000","last_change":"2024-03-26T17:36:21.726380+0000","last_active":"2024-03-26T17:37:27.739808+0000","last_peered":"2024-03-26T17:37:27.739808+0000","last_clean":"2024-03-26T17:37:27.739808+0000","last_became_active":"2024-03-26T17:36:21.726004+0000","last_became_peered":"2024-03-26T17:36:21.726004+0000","last_unstale":"2024-03-26T17:37:27.739808+0000","last_undegraded":"2024-03-26T17:37:27.739808+0000","last_fullsized":"2024-03-26T17:37:27.739808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:51:23.937275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.630425+0000","last_change":"2024-03-26T17:36:22.741486+0000","last_active":"2024-03-26T17:37:56.630425+0000","last_peered":"2024-03-26T17:37:56.630425+0000","last_clean":"2024-03-26T17:37:56.630425+0000","last_became_active":"2024-03-26T17:36:22.741104+0000","last_became_peered":"2024-03-26T17:36:22.741104+0000","last_unstale":"2024-03-26T17:37:56.630425+0000","last_undegraded":"2024-03-26T17:37:56.630425+0000","last_fullsized":"2024-03-26T17:37:56.630425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:53.683944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:18.423824+0000","last_change":"2024-03-26T17:36:21.728824+0000","last_active":"2024-03-26T17:38:18.423824+0000","last_peered":"2024-03-26T17:38:18.423824+0000","last_clean":"2024-03-26T17:38:18.423824+0000","last_became_active":"2024-03-26T17:36:21.728467+0000","last_became_peered":"2024-03-26T17:36:21.728467+0000","last_unstale":"2024-03-26T17:38:18.423824+0000","last_undegraded":"2024-03-26T17:38:18.423824+0000","last_fullsized":"2024-03-26T17:38:18.423824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:25:07.842111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617769+0000","last_change":"2024-03-26T17:36:22.740131+0000","last_active":"2024-03-26T17:37:56.617769+0000","last_peered":"2024-03-26T17:37:56.617769+0000","last_clean":"2024-03-26T17:37:56.617769+0000","last_became_active":"2024-03-26T17:36:22.739704+0000","last_became_peered":"2024-03-26T17:36:22.739704+0000","last_unstale":"2024-03-26T17:37:56.617769+0000","last_undegraded":"2024-03-26T17:37:56.617769+0000","last_fullsized":"2024-03-26T17:37:56.617769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:47.891743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245391+0000","last_change":"2024-03-26T17:36:21.731826+0000","last_active":"2024-03-26T17:37:28.245391+0000","last_peered":"2024-03-26T17:37:28.245391+0000","last_clean":"2024-03-26T17:37:28.245391+0000","last_became_active":"2024-03-26T17:36:21.731546+0000","last_became_peered":"2024-03-26T17:36:21.731546+0000","last_unstale":"2024-03-26T17:37:28.245391+0000","last_undegraded":"2024-03-26T17:37:28.245391+0000","last_fullsized":"2024-03-26T17:37:28.245391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:08:53.302781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.614608+0000","last_change":"2024-03-26T17:36:22.745170+0000","last_active":"2024-03-26T17:37:56.614608+0000","last_peered":"2024-03-26T17:37:56.614608+0000","last_clean":"2024-03-26T17:37:56.614608+0000","last_became_active":"2024-03-26T17:36:22.744798+0000","last_became_peered":"2024-03-26T17:36:22.744798+0000","last_unstale":"2024-03-26T17:37:56.614608+0000","last_undegraded":"2024-03-26T17:37:56.614608+0000","last_fullsized":"2024-03-26T17:37:56.614608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:32:50.418149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739722+0000","last_change":"2024-03-26T17:36:21.721492+0000","last_active":"2024-03-26T17:37:27.739722+0000","last_peered":"2024-03-26T17:37:27.739722+0000","last_clean":"2024-03-26T17:37:27.739722+0000","last_became_active":"2024-03-26T17:36:21.721194+0000","last_became_peered":"2024-03-26T17:36:21.721194+0000","last_unstale":"2024-03-26T17:37:27.739722+0000","last_undegraded":"2024-03-26T17:37:27.739722+0000","last_fullsized":"2024-03-26T17:37:27.739722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:31:50.143284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624868+0000","last_change":"2024-03-26T17:36:22.740780+0000","last_active":"2024-03-26T17:37:56.624868+0000","last_peered":"2024-03-26T17:37:56.624868+0000","last_clean":"2024-03-26T17:37:56.624868+0000","last_became_active":"2024-03-26T17:36:22.740311+0000","last_became_peered":"2024-03-26T17:36:22.740311+0000","last_unstale":"2024-03-26T17:37:56.624868+0000","last_undegraded":"2024-03-26T17:37:56.624868+0000","last_fullsized":"2024-03-26T17:37:56.624868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:01:56.018416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742536+0000","last_change":"2024-03-26T17:36:21.726289+0000","last_active":"2024-03-26T17:37:27.742536+0000","last_peered":"2024-03-26T17:37:27.742536+0000","last_clean":"2024-03-26T17:37:27.742536+0000","last_became_active":"2024-03-26T17:36:21.725898+0000","last_became_peered":"2024-03-26T17:36:21.725898+0000","last_unstale":"2024-03-26T17:37:27.742536+0000","last_undegraded":"2024-03-26T17:37:27.742536+0000","last_fullsized":"2024-03-26T17:37:27.742536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:36:03.893163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631799+0000","last_change":"2024-03-26T17:36:22.750632+0000","last_active":"2024-03-26T17:37:56.631799+0000","last_peered":"2024-03-26T17:37:56.631799+0000","last_clean":"2024-03-26T17:37:56.631799+0000","last_became_active":"2024-03-26T17:36:22.750297+0000","last_became_peered":"2024-03-26T17:36:22.750297+0000","last_unstale":"2024-03-26T17:37:56.631799+0000","last_undegraded":"2024-03-26T17:37:56.631799+0000","last_fullsized":"2024-03-26T17:37:56.631799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:43:16.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'7","reported_seq":254,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.426474+0000","last_change":"2024-03-26T17:36:21.721403+0000","last_active":"2024-03-26T17:38:21.426474+0000","last_peered":"2024-03-26T17:38:21.426474+0000","last_clean":"2024-03-26T17:38:21.426474+0000","last_became_active":"2024-03-26T17:36:21.720962+0000","last_became_peered":"2024-03-26T17:36:21.720962+0000","last_unstale":"2024-03-26T17:38:21.426474+0000","last_undegraded":"2024-03-26T17:38:21.426474+0000","last_fullsized":"2024-03-26T17:38:21.426474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:02.683188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635445+0000","last_change":"2024-03-26T17:36:22.754010+0000","last_active":"2024-03-26T17:37:56.635445+0000","last_peered":"2024-03-26T17:37:56.635445+0000","last_clean":"2024-03-26T17:37:56.635445+0000","last_became_active":"2024-03-26T17:36:22.753534+0000","last_became_peered":"2024-03-26T17:36:22.753534+0000","last_unstale":"2024-03-26T17:37:56.635445+0000","last_undegraded":"2024-03-26T17:37:56.635445+0000","last_fullsized":"2024-03-26T17:37:56.635445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:06.490733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736915+0000","last_change":"2024-03-26T17:36:21.721685+0000","last_active":"2024-03-26T17:37:27.736915+0000","last_peered":"2024-03-26T17:37:27.736915+0000","last_clean":"2024-03-26T17:37:27.736915+0000","last_became_active":"2024-03-26T17:36:21.721387+0000","last_became_peered":"2024-03-26T17:36:21.721387+0000","last_unstale":"2024-03-26T17:37:27.736915+0000","last_undegraded":"2024-03-26T17:37:27.736915+0000","last_fullsized":"2024-03-26T17:37:27.736915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:59.493172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626882+0000","last_change":"2024-03-26T17:37:27.740779+0000","last_active":"2024-03-26T17:37:56.626882+0000","last_peered":"2024-03-26T17:37:56.626882+0000","last_clean":"2024-03-26T17:37:56.626882+0000","last_became_active":"2024-03-26T17:37:27.740407+0000","last_became_peered":"2024-03-26T17:37:27.740407+0000","last_unstale":"2024-03-26T17:37:56.626882+0000","last_undegraded":"2024-03-26T17:37:56.626882+0000","last_fullsized":"2024-03-26T17:37:56.626882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:39.526570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,7],"acting":[0,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744477+0000","last_change":"2024-03-26T17:36:21.724041+0000","last_active":"2024-03-26T17:37:27.744477+0000","last_peered":"2024-03-26T17:37:27.744477+0000","last_clean":"2024-03-26T17:37:27.744477+0000","last_became_active":"2024-03-26T17:36:21.723624+0000","last_became_peered":"2024-03-26T17:36:21.723624+0000","last_unstale":"2024-03-26T17:37:27.744477+0000","last_undegraded":"2024-03-26T17:37:27.744477+0000","last_fullsized":"2024-03-26T17:37:27.744477+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:42:24.264844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.630472+0000","last_change":"2024-03-26T17:36:22.738592+0000","last_active":"2024-03-26T17:37:56.630472+0000","last_peered":"2024-03-26T17:37:56.630472+0000","last_clean":"2024-03-26T17:37:56.630472+0000","last_became_active":"2024-03-26T17:36:22.738236+0000","last_became_peered":"2024-03-26T17:36:22.738236+0000","last_unstale":"2024-03-26T17:37:56.630472+0000","last_undegraded":"2024-03-26T17:37:56.630472+0000","last_fullsized":"2024-03-26T17:37:56.630472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:56:06.977720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":294,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.426418+0000","last_change":"2024-03-26T17:36:21.722793+0000","last_active":"2024-03-26T17:38:21.426418+0000","last_peered":"2024-03-26T17:38:21.426418+0000","last_clean":"2024-03-26T17:38:21.426418+0000","last_became_active":"2024-03-26T17:36:21.722482+0000","last_became_peered":"2024-03-26T17:36:21.722482+0000","last_unstale":"2024-03-26T17:38:21.426418+0000","last_undegraded":"2024-03-26T17:38:21.426418+0000","last_fullsized":"2024-03-26T17:38:21.426418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:48:32.107017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66493,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":185,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.607628+0000","last_change":"2024-03-26T17:36:22.746488+0000","last_active":"2024-03-26T17:37:56.607628+0000","last_peered":"2024-03-26T17:37:56.607628+0000","last_clean":"2024-03-26T17:37:56.607628+0000","last_became_active":"2024-03-26T17:36:22.746013+0000","last_became_peered":"2024-03-26T17:36:22.746013+0000","last_unstale":"2024-03-26T17:37:56.607628+0000","last_undegraded":"2024-03-26T17:37:56.607628+0000","last_fullsized":"2024-03-26T17:37:56.607628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:14.792336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742843+0000","last_change":"2024-03-26T17:36:21.724319+0000","last_active":"2024-03-26T17:37:27.742843+0000","last_peered":"2024-03-26T17:37:27.742843+0000","last_clean":"2024-03-26T17:37:27.742843+0000","last_became_active":"2024-03-26T17:36:21.723644+0000","last_became_peered":"2024-03-26T17:36:21.723644+0000","last_unstale":"2024-03-26T17:37:27.742843+0000","last_undegraded":"2024-03-26T17:37:27.742843+0000","last_fullsized":"2024-03-26T17:37:27.742843+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:31:01.132797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.625070+0000","last_change":"2024-03-26T17:36:22.736535+0000","last_active":"2024-03-26T17:37:56.625070+0000","last_peered":"2024-03-26T17:37:56.625070+0000","last_clean":"2024-03-26T17:37:56.625070+0000","last_became_active":"2024-03-26T17:36:22.736229+0000","last_became_peered":"2024-03-26T17:36:22.736229+0000","last_unstale":"2024-03-26T17:37:56.625070+0000","last_undegraded":"2024-03-26T17:37:56.625070+0000","last_fullsized":"2024-03-26T17:37:56.625070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:18:03.938180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.742451+0000","last_change":"2024-03-26T17:36:21.726271+0000","last_active":"2024-03-26T17:37:27.742451+0000","last_peered":"2024-03-26T17:37:27.742451+0000","last_clean":"2024-03-26T17:37:27.742451+0000","last_became_active":"2024-03-26T17:36:21.725942+0000","last_became_peered":"2024-03-26T17:36:21.725942+0000","last_unstale":"2024-03-26T17:37:27.742451+0000","last_undegraded":"2024-03-26T17:37:27.742451+0000","last_fullsized":"2024-03-26T17:37:27.742451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:23:16.320106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.633769+0000","last_change":"2024-03-26T17:36:22.743480+0000","last_active":"2024-03-26T17:37:56.633769+0000","last_peered":"2024-03-26T17:37:56.633769+0000","last_clean":"2024-03-26T17:37:56.633769+0000","last_became_active":"2024-03-26T17:36:22.742982+0000","last_became_peered":"2024-03-26T17:36:22.742982+0000","last_unstale":"2024-03-26T17:37:56.633769+0000","last_undegraded":"2024-03-26T17:37:56.633769+0000","last_fullsized":"2024-03-26T17:37:56.633769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:47.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:18.424299+0000","last_change":"2024-03-26T17:36:21.723407+0000","last_active":"2024-03-26T17:38:18.424299+0000","last_peered":"2024-03-26T17:38:18.424299+0000","last_clean":"2024-03-26T17:38:18.424299+0000","last_became_active":"2024-03-26T17:36:21.723086+0000","last_became_peered":"2024-03-26T17:36:21.723086+0000","last_unstale":"2024-03-26T17:38:18.424299+0000","last_undegraded":"2024-03-26T17:38:18.424299+0000","last_fullsized":"2024-03-26T17:38:18.424299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:26:57.886388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631858+0000","last_change":"2024-03-26T17:36:22.740705+0000","last_active":"2024-03-26T17:37:56.631858+0000","last_peered":"2024-03-26T17:37:56.631858+0000","last_clean":"2024-03-26T17:37:56.631858+0000","last_became_active":"2024-03-26T17:36:22.740360+0000","last_became_peered":"2024-03-26T17:36:22.740360+0000","last_unstale":"2024-03-26T17:37:56.631858+0000","last_undegraded":"2024-03-26T17:37:56.631858+0000","last_fullsized":"2024-03-26T17:37:56.631858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:02.879022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.266750+0000","last_change":"2024-03-26T17:37:28.266399+0000","last_active":"2024-03-26T17:37:28.266750+0000","last_peered":"2024-03-26T17:37:28.266750+0000","last_clean":"2024-03-26T17:37:28.266750+0000","last_became_active":"2024-03-26T17:37:28.246359+0000","last_became_peered":"2024-03-26T17:37:28.246359+0000","last_unstale":"2024-03-26T17:37:28.266750+0000","last_undegraded":"2024-03-26T17:37:28.266750+0000","last_fullsized":"2024-03-26T17:37:28.266750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:20:54.895081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.625211+0000","last_change":"2024-03-26T17:36:22.743791+0000","last_active":"2024-03-26T17:37:56.625211+0000","last_peered":"2024-03-26T17:37:56.625211+0000","last_clean":"2024-03-26T17:37:56.625211+0000","last_became_active":"2024-03-26T17:36:22.742780+0000","last_became_peered":"2024-03-26T17:36:22.742780+0000","last_unstale":"2024-03-26T17:37:56.625211+0000","last_undegraded":"2024-03-26T17:37:56.625211+0000","last_fullsized":"2024-03-26T17:37:56.625211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:25:04.387958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.740570+0000","last_change":"2024-03-26T17:36:21.724289+0000","last_active":"2024-03-26T17:37:27.740570+0000","last_peered":"2024-03-26T17:37:27.740570+0000","last_clean":"2024-03-26T17:37:27.740570+0000","last_became_active":"2024-03-26T17:36:21.723847+0000","last_became_peered":"2024-03-26T17:36:21.723847+0000","last_unstale":"2024-03-26T17:37:27.740570+0000","last_undegraded":"2024-03-26T17:37:27.740570+0000","last_fullsized":"2024-03-26T17:37:27.740570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:01:59.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626382+0000","last_change":"2024-03-26T17:36:22.748323+0000","last_active":"2024-03-26T17:37:56.626382+0000","last_peered":"2024-03-26T17:37:56.626382+0000","last_clean":"2024-03-26T17:37:56.626382+0000","last_became_active":"2024-03-26T17:36:22.747970+0000","last_became_peered":"2024-03-26T17:36:22.747970+0000","last_unstale":"2024-03-26T17:37:56.626382+0000","last_undegraded":"2024-03-26T17:37:56.626382+0000","last_fullsized":"2024-03-26T17:37:56.626382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:52:58.886425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739292+0000","last_change":"2024-03-26T17:36:21.726076+0000","last_active":"2024-03-26T17:37:27.739292+0000","last_peered":"2024-03-26T17:37:27.739292+0000","last_clean":"2024-03-26T17:37:27.739292+0000","last_became_active":"2024-03-26T17:36:21.722067+0000","last_became_peered":"2024-03-26T17:36:21.722067+0000","last_unstale":"2024-03-26T17:37:27.739292+0000","last_undegraded":"2024-03-26T17:37:27.739292+0000","last_fullsized":"2024-03-26T17:37:27.739292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:34:40.316110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628515+0000","last_change":"2024-03-26T17:36:22.730890+0000","last_active":"2024-03-26T17:37:56.628515+0000","last_peered":"2024-03-26T17:37:56.628515+0000","last_clean":"2024-03-26T17:37:56.628515+0000","last_became_active":"2024-03-26T17:36:22.729635+0000","last_became_peered":"2024-03-26T17:36:22.729635+0000","last_unstale":"2024-03-26T17:37:56.628515+0000","last_undegraded":"2024-03-26T17:37:56.628515+0000","last_fullsized":"2024-03-26T17:37:56.628515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:15:09.341588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'7","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.432102+0000","last_change":"2024-03-26T17:36:21.727587+0000","last_active":"2024-03-26T17:38:21.432102+0000","last_peered":"2024-03-26T17:38:21.432102+0000","last_clean":"2024-03-26T17:38:21.432102+0000","last_became_active":"2024-03-26T17:36:21.727302+0000","last_became_peered":"2024-03-26T17:36:21.727302+0000","last_unstale":"2024-03-26T17:38:21.432102+0000","last_undegraded":"2024-03-26T17:38:21.432102+0000","last_fullsized":"2024-03-26T17:38:21.432102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:23:59.404567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628589+0000","last_change":"2024-03-26T17:36:22.750654+0000","last_active":"2024-03-26T17:37:56.628589+0000","last_peered":"2024-03-26T17:37:56.628589+0000","last_clean":"2024-03-26T17:37:56.628589+0000","last_became_active":"2024-03-26T17:36:22.750281+0000","last_became_peered":"2024-03-26T17:36:22.750281+0000","last_unstale":"2024-03-26T17:37:56.628589+0000","last_undegraded":"2024-03-26T17:37:56.628589+0000","last_fullsized":"2024-03-26T17:37:56.628589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:04.034108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.619451+0000","last_change":"2024-03-26T17:37:27.745100+0000","last_active":"2024-03-26T17:37:56.619451+0000","last_peered":"2024-03-26T17:37:56.619451+0000","last_clean":"2024-03-26T17:37:56.619451+0000","last_became_active":"2024-03-26T17:37:27.744686+0000","last_became_peered":"2024-03-26T17:37:27.744686+0000","last_unstale":"2024-03-26T17:37:56.619451+0000","last_undegraded":"2024-03-26T17:37:56.619451+0000","last_fullsized":"2024-03-26T17:37:56.619451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:38:38.111145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,6],"acting":[9,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":141,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:21.426775+0000","last_change":"2024-03-26T17:36:21.730812+0000","last_active":"2024-03-26T17:38:21.426775+0000","last_peered":"2024-03-26T17:38:21.426775+0000","last_clean":"2024-03-26T17:38:21.426775+0000","last_became_active":"2024-03-26T17:36:21.730499+0000","last_became_peered":"2024-03-26T17:36:21.730499+0000","last_unstale":"2024-03-26T17:38:21.426775+0000","last_undegraded":"2024-03-26T17:38:21.426775+0000","last_fullsized":"2024-03-26T17:38:21.426775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:39:05.666146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617217+0000","last_change":"2024-03-26T17:36:22.742344+0000","last_active":"2024-03-26T17:37:56.617217+0000","last_peered":"2024-03-26T17:37:56.617217+0000","last_clean":"2024-03-26T17:37:56.617217+0000","last_became_active":"2024-03-26T17:36:22.741864+0000","last_became_peered":"2024-03-26T17:36:22.741864+0000","last_unstale":"2024-03-26T17:37:56.617217+0000","last_undegraded":"2024-03-26T17:37:56.617217+0000","last_fullsized":"2024-03-26T17:37:56.617217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:53:53.553107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244456+0000","last_change":"2024-03-26T17:36:21.722450+0000","last_active":"2024-03-26T17:37:28.244456+0000","last_peered":"2024-03-26T17:37:28.244456+0000","last_clean":"2024-03-26T17:37:28.244456+0000","last_became_active":"2024-03-26T17:36:21.722105+0000","last_became_peered":"2024-03-26T17:36:21.722105+0000","last_unstale":"2024-03-26T17:37:28.244456+0000","last_undegraded":"2024-03-26T17:37:28.244456+0000","last_fullsized":"2024-03-26T17:37:28.244456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:01:22.983166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736685+0000","last_change":"2024-03-26T17:35:10.663606+0000","last_active":"2024-03-26T17:37:27.736685+0000","last_peered":"2024-03-26T17:37:27.736685+0000","last_clean":"2024-03-26T17:37:27.736685+0000","last_became_active":"2024-03-26T17:35:10.663069+0000","last_became_peered":"2024-03-26T17:35:10.663069+0000","last_unstale":"2024-03-26T17:37:27.736685+0000","last_undegraded":"2024-03-26T17:37:27.736685+0000","last_fullsized":"2024-03-26T17:37:27.736685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:31:50.880228+0000","last_clean_scrub_stamp":"2024-03-26T17:31:50.880228+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:50.817007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.622092+0000","last_change":"2024-03-26T17:37:27.745201+0000","last_active":"2024-03-26T17:37:56.622092+0000","last_peered":"2024-03-26T17:37:56.622092+0000","last_clean":"2024-03-26T17:37:56.622092+0000","last_became_active":"2024-03-26T17:37:27.744806+0000","last_became_peered":"2024-03-26T17:37:27.744806+0000","last_unstale":"2024-03-26T17:37:56.622092+0000","last_undegraded":"2024-03-26T17:37:56.622092+0000","last_fullsized":"2024-03-26T17:37:56.622092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:15:07.485013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.736795+0000","last_change":"2024-03-26T17:36:21.723452+0000","last_active":"2024-03-26T17:37:27.736795+0000","last_peered":"2024-03-26T17:37:27.736795+0000","last_clean":"2024-03-26T17:37:27.736795+0000","last_became_active":"2024-03-26T17:36:21.723155+0000","last_became_peered":"2024-03-26T17:36:21.723155+0000","last_unstale":"2024-03-26T17:37:27.736795+0000","last_undegraded":"2024-03-26T17:37:27.736795+0000","last_fullsized":"2024-03-26T17:37:27.736795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:39:34.443624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.750092+0000","last_change":"2024-03-26T17:37:27.749741+0000","last_active":"2024-03-26T17:37:27.750092+0000","last_peered":"2024-03-26T17:37:27.750092+0000","last_clean":"2024-03-26T17:37:27.750092+0000","last_became_active":"2024-03-26T17:37:27.741751+0000","last_became_peered":"2024-03-26T17:37:27.741751+0000","last_unstale":"2024-03-26T17:37:27.750092+0000","last_undegraded":"2024-03-26T17:37:27.750092+0000","last_fullsized":"2024-03-26T17:37:27.750092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:42.119297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621855+0000","last_change":"2024-03-26T17:36:22.736487+0000","last_active":"2024-03-26T17:37:56.621855+0000","last_peered":"2024-03-26T17:37:56.621855+0000","last_clean":"2024-03-26T17:37:56.621855+0000","last_became_active":"2024-03-26T17:36:22.735788+0000","last_became_peered":"2024-03-26T17:36:22.735788+0000","last_unstale":"2024-03-26T17:37:56.621855+0000","last_undegraded":"2024-03-26T17:37:56.621855+0000","last_fullsized":"2024-03-26T17:37:56.621855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:03:45.240148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:18.756601+0000","last_change":"2024-03-26T17:36:21.720755+0000","last_active":"2024-03-26T17:38:18.756601+0000","last_peered":"2024-03-26T17:38:18.756601+0000","last_clean":"2024-03-26T17:38:18.756601+0000","last_became_active":"2024-03-26T17:36:21.720175+0000","last_became_peered":"2024-03-26T17:36:21.720175+0000","last_unstale":"2024-03-26T17:38:18.756601+0000","last_undegraded":"2024-03-26T17:38:18.756601+0000","last_fullsized":"2024-03-26T17:38:18.756601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:48:22.458657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.624275+0000","last_change":"2024-03-26T17:36:22.740731+0000","last_active":"2024-03-26T17:37:56.624275+0000","last_peered":"2024-03-26T17:37:56.624275+0000","last_clean":"2024-03-26T17:37:56.624275+0000","last_became_active":"2024-03-26T17:36:22.740283+0000","last_became_peered":"2024-03-26T17:36:22.740283+0000","last_unstale":"2024-03-26T17:37:56.624275+0000","last_undegraded":"2024-03-26T17:37:56.624275+0000","last_fullsized":"2024-03-26T17:37:56.624275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:48:36.403105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.244032+0000","last_change":"2024-03-26T17:36:21.721513+0000","last_active":"2024-03-26T17:37:28.244032+0000","last_peered":"2024-03-26T17:37:28.244032+0000","last_clean":"2024-03-26T17:37:28.244032+0000","last_became_active":"2024-03-26T17:36:21.721185+0000","last_became_peered":"2024-03-26T17:36:21.721185+0000","last_unstale":"2024-03-26T17:37:28.244032+0000","last_undegraded":"2024-03-26T17:37:28.244032+0000","last_fullsized":"2024-03-26T17:37:28.244032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:48:39.994980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628787+0000","last_change":"2024-03-26T17:36:22.737938+0000","last_active":"2024-03-26T17:37:56.628787+0000","last_peered":"2024-03-26T17:37:56.628787+0000","last_clean":"2024-03-26T17:37:56.628787+0000","last_became_active":"2024-03-26T17:36:22.737605+0000","last_became_peered":"2024-03-26T17:36:22.737605+0000","last_unstale":"2024-03-26T17:37:56.628787+0000","last_undegraded":"2024-03-26T17:37:56.628787+0000","last_fullsized":"2024-03-26T17:37:56.628787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:27.087959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.245677+0000","last_change":"2024-03-26T17:36:21.720942+0000","last_active":"2024-03-26T17:37:28.245677+0000","last_peered":"2024-03-26T17:37:28.245677+0000","last_clean":"2024-03-26T17:37:28.245677+0000","last_became_active":"2024-03-26T17:36:21.720665+0000","last_became_peered":"2024-03-26T17:36:21.720665+0000","last_unstale":"2024-03-26T17:37:28.245677+0000","last_undegraded":"2024-03-26T17:37:28.245677+0000","last_fullsized":"2024-03-26T17:37:28.245677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:26:00.725213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.587257+0000","last_change":"2024-03-26T17:36:22.747111+0000","last_active":"2024-03-26T17:37:56.587257+0000","last_peered":"2024-03-26T17:37:56.587257+0000","last_clean":"2024-03-26T17:37:56.587257+0000","last_became_active":"2024-03-26T17:36:22.746626+0000","last_became_peered":"2024-03-26T17:36:22.746626+0000","last_unstale":"2024-03-26T17:37:56.587257+0000","last_undegraded":"2024-03-26T17:37:56.587257+0000","last_fullsized":"2024-03-26T17:37:56.587257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:29:24.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.741395+0000","last_change":"2024-03-26T17:36:21.724141+0000","last_active":"2024-03-26T17:37:27.741395+0000","last_peered":"2024-03-26T17:37:27.741395+0000","last_clean":"2024-03-26T17:37:27.741395+0000","last_became_active":"2024-03-26T17:36:21.723797+0000","last_became_peered":"2024-03-26T17:36:21.723797+0000","last_unstale":"2024-03-26T17:37:27.741395+0000","last_undegraded":"2024-03-26T17:37:27.741395+0000","last_fullsized":"2024-03-26T17:37:27.741395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:59:06.149827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.627752+0000","last_change":"2024-03-26T17:36:22.739497+0000","last_active":"2024-03-26T17:37:56.627752+0000","last_peered":"2024-03-26T17:37:56.627752+0000","last_clean":"2024-03-26T17:37:56.627752+0000","last_became_active":"2024-03-26T17:36:22.739114+0000","last_became_peered":"2024-03-26T17:36:22.739114+0000","last_unstale":"2024-03-26T17:37:56.627752+0000","last_undegraded":"2024-03-26T17:37:56.627752+0000","last_fullsized":"2024-03-26T17:37:56.627752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:03.953199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.743064+0000","last_change":"2024-03-26T17:36:21.728027+0000","last_active":"2024-03-26T17:37:27.743064+0000","last_peered":"2024-03-26T17:37:27.743064+0000","last_clean":"2024-03-26T17:37:27.743064+0000","last_became_active":"2024-03-26T17:36:21.727159+0000","last_became_peered":"2024-03-26T17:36:21.727159+0000","last_unstale":"2024-03-26T17:37:27.743064+0000","last_undegraded":"2024-03-26T17:37:27.743064+0000","last_fullsized":"2024-03-26T17:37:27.743064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:01:39.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628518+0000","last_change":"2024-03-26T17:36:22.733666+0000","last_active":"2024-03-26T17:37:56.628518+0000","last_peered":"2024-03-26T17:37:56.628518+0000","last_clean":"2024-03-26T17:37:56.628518+0000","last_became_active":"2024-03-26T17:36:22.733313+0000","last_became_peered":"2024-03-26T17:36:22.733313+0000","last_unstale":"2024-03-26T17:37:56.628518+0000","last_undegraded":"2024-03-26T17:37:56.628518+0000","last_fullsized":"2024-03-26T17:37:56.628518+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:52:38.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.739591+0000","last_change":"2024-03-26T17:36:21.730828+0000","last_active":"2024-03-26T17:37:27.739591+0000","last_peered":"2024-03-26T17:37:27.739591+0000","last_clean":"2024-03-26T17:37:27.739591+0000","last_became_active":"2024-03-26T17:36:21.730508+0000","last_became_peered":"2024-03-26T17:36:21.730508+0000","last_unstale":"2024-03-26T17:37:27.739591+0000","last_undegraded":"2024-03-26T17:37:27.739591+0000","last_fullsized":"2024-03-26T17:37:27.739591+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:10:53.492725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.631929+0000","last_change":"2024-03-26T17:36:22.740204+0000","last_active":"2024-03-26T17:37:56.631929+0000","last_peered":"2024-03-26T17:37:56.631929+0000","last_clean":"2024-03-26T17:37:56.631929+0000","last_became_active":"2024-03-26T17:36:22.735880+0000","last_became_peered":"2024-03-26T17:36:22.735880+0000","last_unstale":"2024-03-26T17:37:56.631929+0000","last_undegraded":"2024-03-26T17:37:56.631929+0000","last_fullsized":"2024-03-26T17:37:56.631929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:34.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":144,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:38:19.425044+0000","last_change":"2024-03-26T17:36:21.720873+0000","last_active":"2024-03-26T17:38:19.425044+0000","last_peered":"2024-03-26T17:38:19.425044+0000","last_clean":"2024-03-26T17:38:19.425044+0000","last_became_active":"2024-03-26T17:36:21.720496+0000","last_became_peered":"2024-03-26T17:36:21.720496+0000","last_unstale":"2024-03-26T17:38:19.425044+0000","last_undegraded":"2024-03-26T17:38:19.425044+0000","last_fullsized":"2024-03-26T17:38:19.425044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:40:55.959064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2195,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.632104+0000","last_change":"2024-03-26T17:36:22.735528+0000","last_active":"2024-03-26T17:37:56.632104+0000","last_peered":"2024-03-26T17:37:56.632104+0000","last_clean":"2024-03-26T17:37:56.632104+0000","last_became_active":"2024-03-26T17:36:22.735147+0000","last_became_peered":"2024-03-26T17:36:22.735147+0000","last_unstale":"2024-03-26T17:37:56.632104+0000","last_undegraded":"2024-03-26T17:37:56.632104+0000","last_fullsized":"2024-03-26T17:37:56.632104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:15.729853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:27.744355+0000","last_change":"2024-03-26T17:36:21.723246+0000","last_active":"2024-03-26T17:37:27.744355+0000","last_peered":"2024-03-26T17:37:27.744355+0000","last_clean":"2024-03-26T17:37:27.744355+0000","last_became_active":"2024-03-26T17:36:21.722839+0000","last_became_peered":"2024-03-26T17:36:21.722839+0000","last_unstale":"2024-03-26T17:37:27.744355+0000","last_undegraded":"2024-03-26T17:37:27.744355+0000","last_fullsized":"2024-03-26T17:37:27.744355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:41:05.540206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629489+0000","last_change":"2024-03-26T17:36:22.742469+0000","last_active":"2024-03-26T17:37:56.629489+0000","last_peered":"2024-03-26T17:37:56.629489+0000","last_clean":"2024-03-26T17:37:56.629489+0000","last_became_active":"2024-03-26T17:36:22.742073+0000","last_became_peered":"2024-03-26T17:36:22.742073+0000","last_unstale":"2024-03-26T17:37:56.629489+0000","last_undegraded":"2024-03-26T17:37:56.629489+0000","last_fullsized":"2024-03-26T17:37:56.629489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:19:33.865730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:28.250123+0000","last_change":"2024-03-26T17:37:28.250123+0000","last_active":"2024-03-26T17:37:28.250123+0000","last_peered":"2024-03-26T17:37:28.250123+0000","last_clean":"2024-03-26T17:37:28.250123+0000","last_became_active":"2024-03-26T17:37:28.246162+0000","last_became_peered":"2024-03-26T17:37:28.246162+0000","last_unstale":"2024-03-26T17:37:28.250123+0000","last_undegraded":"2024-03-26T17:37:28.250123+0000","last_fullsized":"2024-03-26T17:37:28.250123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:20.693172+0000","last_clean_scrub_stamp":"2024-03-26T17:36:20.693172+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:33:39.002192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628828+0000","last_change":"2024-03-26T17:36:22.742614+0000","last_active":"2024-03-26T17:37:56.628828+0000","last_peered":"2024-03-26T17:37:56.628828+0000","last_clean":"2024-03-26T17:37:56.628828+0000","last_became_active":"2024-03-26T17:36:22.742278+0000","last_became_peered":"2024-03-26T17:36:22.742278+0000","last_unstale":"2024-03-26T17:37:56.628828+0000","last_undegraded":"2024-03-26T17:37:56.628828+0000","last_fullsized":"2024-03-26T17:37:56.628828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:30:14.503474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.627061+0000","last_change":"2024-03-26T17:36:22.741693+0000","last_active":"2024-03-26T17:37:56.627061+0000","last_peered":"2024-03-26T17:37:56.627061+0000","last_clean":"2024-03-26T17:37:56.627061+0000","last_became_active":"2024-03-26T17:36:22.741352+0000","last_became_peered":"2024-03-26T17:36:22.741352+0000","last_unstale":"2024-03-26T17:37:56.627061+0000","last_undegraded":"2024-03-26T17:37:56.627061+0000","last_fullsized":"2024-03-26T17:37:56.627061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T03:13:26.912910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.623353+0000","last_change":"2024-03-26T17:36:22.742939+0000","last_active":"2024-03-26T17:37:56.623353+0000","last_peered":"2024-03-26T17:37:56.623353+0000","last_clean":"2024-03-26T17:37:56.623353+0000","last_became_active":"2024-03-26T17:36:22.742589+0000","last_became_peered":"2024-03-26T17:36:22.742589+0000","last_unstale":"2024-03-26T17:37:56.623353+0000","last_undegraded":"2024-03-26T17:37:56.623353+0000","last_fullsized":"2024-03-26T17:37:56.623353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T02:21:14.043601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617673+0000","last_change":"2024-03-26T17:36:22.748044+0000","last_active":"2024-03-26T17:37:56.617673+0000","last_peered":"2024-03-26T17:37:56.617673+0000","last_clean":"2024-03-26T17:37:56.617673+0000","last_became_active":"2024-03-26T17:36:22.747683+0000","last_became_peered":"2024-03-26T17:36:22.747683+0000","last_unstale":"2024-03-26T17:37:56.617673+0000","last_undegraded":"2024-03-26T17:37:56.617673+0000","last_fullsized":"2024-03-26T17:37:56.617673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:32:40.439515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628303+0000","last_change":"2024-03-26T17:36:22.743080+0000","last_active":"2024-03-26T17:37:56.628303+0000","last_peered":"2024-03-26T17:37:56.628303+0000","last_clean":"2024-03-26T17:37:56.628303+0000","last_became_active":"2024-03-26T17:36:22.742676+0000","last_became_peered":"2024-03-26T17:36:22.742676+0000","last_unstale":"2024-03-26T17:37:56.628303+0000","last_undegraded":"2024-03-26T17:37:56.628303+0000","last_fullsized":"2024-03-26T17:37:56.628303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T04:32:58.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.628807+0000","last_change":"2024-03-26T17:36:22.737908+0000","last_active":"2024-03-26T17:37:56.628807+0000","last_peered":"2024-03-26T17:37:56.628807+0000","last_clean":"2024-03-26T17:37:56.628807+0000","last_became_active":"2024-03-26T17:36:22.737055+0000","last_became_peered":"2024-03-26T17:36:22.737055+0000","last_unstale":"2024-03-26T17:37:56.628807+0000","last_undegraded":"2024-03-26T17:37:56.628807+0000","last_fullsized":"2024-03-26T17:37:56.628807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:51:13.157901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.621037+0000","last_change":"2024-03-26T17:36:22.749795+0000","last_active":"2024-03-26T17:37:56.621037+0000","last_peered":"2024-03-26T17:37:56.621037+0000","last_clean":"2024-03-26T17:37:56.621037+0000","last_became_active":"2024-03-26T17:36:22.749469+0000","last_became_peered":"2024-03-26T17:36:22.749469+0000","last_unstale":"2024-03-26T17:37:56.621037+0000","last_undegraded":"2024-03-26T17:37:56.621037+0000","last_fullsized":"2024-03-26T17:37:56.621037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:19:59.126270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.633721+0000","last_change":"2024-03-26T17:36:22.744101+0000","last_active":"2024-03-26T17:37:56.633721+0000","last_peered":"2024-03-26T17:37:56.633721+0000","last_clean":"2024-03-26T17:37:56.633721+0000","last_became_active":"2024-03-26T17:36:22.743761+0000","last_became_peered":"2024-03-26T17:36:22.743761+0000","last_unstale":"2024-03-26T17:37:56.633721+0000","last_undegraded":"2024-03-26T17:37:56.633721+0000","last_fullsized":"2024-03-26T17:37:56.633721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:27:19.257538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.617497+0000","last_change":"2024-03-26T17:36:22.744800+0000","last_active":"2024-03-26T17:37:56.617497+0000","last_peered":"2024-03-26T17:37:56.617497+0000","last_clean":"2024-03-26T17:37:56.617497+0000","last_became_active":"2024-03-26T17:36:22.744437+0000","last_became_peered":"2024-03-26T17:36:22.744437+0000","last_unstale":"2024-03-26T17:37:56.617497+0000","last_undegraded":"2024-03-26T17:37:56.617497+0000","last_fullsized":"2024-03-26T17:37:56.617497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:54:36.459072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.620441+0000","last_change":"2024-03-26T17:37:27.745243+0000","last_active":"2024-03-26T17:37:56.620441+0000","last_peered":"2024-03-26T17:37:56.620441+0000","last_clean":"2024-03-26T17:37:56.620441+0000","last_became_active":"2024-03-26T17:37:27.744869+0000","last_became_peered":"2024-03-26T17:37:27.744869+0000","last_unstale":"2024-03-26T17:37:56.620441+0000","last_undegraded":"2024-03-26T17:37:56.620441+0000","last_fullsized":"2024-03-26T17:37:56.620441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:18:42.188733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.623784+0000","last_change":"2024-03-26T17:36:22.743211+0000","last_active":"2024-03-26T17:37:56.623784+0000","last_peered":"2024-03-26T17:37:56.623784+0000","last_clean":"2024-03-26T17:37:56.623784+0000","last_became_active":"2024-03-26T17:36:22.742869+0000","last_became_peered":"2024-03-26T17:36:22.742869+0000","last_unstale":"2024-03-26T17:37:56.623784+0000","last_undegraded":"2024-03-26T17:37:56.623784+0000","last_fullsized":"2024-03-26T17:37:56.623784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T05:24:04.610012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.635257+0000","last_change":"2024-03-26T17:36:22.746950+0000","last_active":"2024-03-26T17:37:56.635257+0000","last_peered":"2024-03-26T17:37:56.635257+0000","last_clean":"2024-03-26T17:37:56.635257+0000","last_became_active":"2024-03-26T17:36:22.744869+0000","last_became_peered":"2024-03-26T17:36:22.744869+0000","last_unstale":"2024-03-26T17:37:56.635257+0000","last_undegraded":"2024-03-26T17:37:56.635257+0000","last_fullsized":"2024-03-26T17:37:56.635257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:09:52.283519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.629292+0000","last_change":"2024-03-26T17:36:22.744627+0000","last_active":"2024-03-26T17:37:56.629292+0000","last_peered":"2024-03-26T17:37:56.629292+0000","last_clean":"2024-03-26T17:37:56.629292+0000","last_became_active":"2024-03-26T17:36:22.744299+0000","last_became_peered":"2024-03-26T17:36:22.744299+0000","last_unstale":"2024-03-26T17:37:56.629292+0000","last_undegraded":"2024-03-26T17:37:56.629292+0000","last_fullsized":"2024-03-26T17:37:56.629292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:23:37.941625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T17:37:56.626943+0000","last_change":"2024-03-26T17:36:22.751135+0000","last_active":"2024-03-26T17:37:56.626943+0000","last_peered":"2024-03-26T17:37:56.626943+0000","last_clean":"2024-03-26T17:37:56.626943+0000","last_became_active":"2024-03-26T17:36:22.750768+0000","last_became_peered":"2024-03-26T17:36:22.750768+0000","last_unstale":"2024-03-26T17:37:56.626943+0000","last_undegraded":"2024-03-26T17:37:56.626943+0000","last_fullsized":"2024-03-26T17:37:56.626943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T17:36:21.704364+0000","last_clean_scrub_stamp":"2024-03-26T17:36:21.704364+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:51:15.532367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75810,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1181,"num_read_kb":676,"num_write":263,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":217629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383719,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":457563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1109624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":456244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":456398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32708,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712444,"statfs":{"total":95995035648,"available":95961542656,"internally_reserved":0,"allocated":1568768,"data_stored":1050619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":519267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":458325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":520288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":455786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1581056,"data_stored":1048920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T17:38:25.225 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:38:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 -- 172.21.15.105:0/37818057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f387c1019d0 msgr2=0x7f387c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 --2- 172.21.15.105:0/37818057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f387c1019d0 0x7f387c101e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f386c009900 tx=0x7f386c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 -- 172.21.15.105:0/37818057 shutdown_connections 2024-03-26T17:38:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 --2- 172.21.15.105:0/37818057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f387c102390 0x7f387c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 --2- 172.21.15.105:0/37818057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f387c1019d0 0x7f387c101e50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 --2- 172.21.15.105:0/37818057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f387c1007e0 0x7f387c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 -- 172.21.15.105:0/37818057 >> 172.21.15.105:0/37818057 conn(0x7f387c0fbd50 msgr2=0x7f387c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 -- 172.21.15.105:0/37818057 shutdown_connections 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.387+0000 7f3884444640 1 -- 172.21.15.105:0/37818057 wait complete. 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f3884444640 1 Processor -- start 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f3884444640 1 -- start start 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f3884444640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f387c1007e0 0x7f387c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f3884444640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f387c1019d0 0x7f387c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f3884444640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f387c102390 0x7f387c1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f3884444640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f387c10c360 con 0x7f387c1007e0 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f3884444640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f387c10c1e0 con 0x7f387c102390 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f3884444640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f387c10c4e0 con 0x7f387c1019d0 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f387c1007e0 0x7f387c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f387c1007e0 0x7f387c069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55864/0 (socket says 172.21.15.105:55864) 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 -- 172.21.15.105:0/1215047592 learned_addr learned my addr 172.21.15.105:0/1215047592 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38819df640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f387c1019d0 0x7f387c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38829e1640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f387c102390 0x7f387c1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 -- 172.21.15.105:0/1215047592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f387c1019d0 msgr2=0x7f387c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f387c1019d0 0x7f387c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 -- 172.21.15.105:0/1215047592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f387c102390 msgr2=0x7f387c1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f387c102390 0x7f387c1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 -- 172.21.15.105:0/1215047592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c1a19c0 con 0x7f387c1007e0 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.388+0000 7f38821e0640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f387c1007e0 0x7f387c069590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f387800e9c0 tx=0x7f387800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.389+0000 7f38737fe640 1 -- 172.21.15.105:0/1215047592 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387800cde0 con 0x7f387c1007e0 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.389+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387c1a1c50 con 0x7f387c1007e0 2024-03-26T17:38:25.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.389+0000 7f38737fe640 1 -- 172.21.15.105:0/1215047592 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38780108e0 con 0x7f387c1007e0 2024-03-26T17:38:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.389+0000 7f38737fe640 1 -- 172.21.15.105:0/1215047592 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38780188e0 con 0x7f387c1007e0 2024-03-26T17:38:25.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.389+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387c1a21c0 con 0x7f387c1007e0 2024-03-26T17:38:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.390+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387c107b80 con 0x7f387c1007e0 2024-03-26T17:38:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.391+0000 7f38737fe640 1 -- 172.21.15.105:0/1215047592 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f387801f020 con 0x7f387c1007e0 2024-03-26T17:38:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.392+0000 7f38737fe640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38680a2fe0 0x7f38680a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.392+0000 7f38737fe640 1 -- 172.21.15.105:0/1215047592 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f38780cf6c0 con 0x7f387c1007e0 2024-03-26T17:38:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.392+0000 7f38819df640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38680a2fe0 0x7f38680a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.392+0000 7f38819df640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38680a2fe0 0x7f38680a54a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:25.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.396+0000 7f38737fe640 1 -- 172.21.15.105:0/1215047592 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f387809c3b0 con 0x7f387c1007e0 2024-03-26T17:38:25.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.581+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f387c02d4a0 con 0x7f387c1007e0 2024-03-26T17:38:25.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.584+0000 7f38737fe640 1 -- 172.21.15.105:0/1215047592 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f387809bcb0 con 0x7f387c1007e0 2024-03-26T17:38:25.576 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:25.577 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:38:25.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.585+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38680a2fe0 msgr2=0x7f38680a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.585+0000 7f3884444640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38680a2fe0 0x7f38680a54a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c02c7e0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.585+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f387c1007e0 msgr2=0x7f387c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.585+0000 7f3884444640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f387c1007e0 0x7f387c069590 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f387800e9c0 tx=0x7f387800ee80 comp rx=0 tx=0).stop 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.586+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 shutdown_connections 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.586+0000 7f3884444640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38680a2fe0 0x7f38680a54a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.586+0000 7f3884444640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f387c102390 0x7f387c1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.586+0000 7f3884444640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f387c1019d0 0x7f387c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.586+0000 7f3884444640 1 --2- 172.21.15.105:0/1215047592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f387c1007e0 0x7f387c069590 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.586+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 >> 172.21.15.105:0/1215047592 conn(0x7f387c0fbd50 msgr2=0x7f387c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.586+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 shutdown_connections 2024-03-26T17:38:25.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.586+0000 7f3884444640 1 -- 172.21.15.105:0/1215047592 wait complete. 2024-03-26T17:38:25.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T17:38:25.594 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T17:38:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 -- 172.21.15.105:0/1451913166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed81023a0 msgr2=0x7feed8104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1451913166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed81023a0 0x7feed8104790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feed4009900 tx=0x7feed402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 -- 172.21.15.105:0/1451913166 shutdown_connections 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1451913166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feed8107600 0x7feed81099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1451913166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed8104cd0 0x7feed81070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1451913166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed81023a0 0x7feed8104790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 -- 172.21.15.105:0/1451913166 >> 172.21.15.105:0/1451913166 conn(0x7feed80fbd50 msgr2=0x7feed80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 -- 172.21.15.105:0/1451913166 shutdown_connections 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.711+0000 7feedfc4b640 1 -- 172.21.15.105:0/1451913166 wait complete. 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedfc4b640 1 Processor -- start 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedfc4b640 1 -- start start 2024-03-26T17:38:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedfc4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feed81023a0 0x7feed8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedfc4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed8104cd0 0x7feed81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedfc4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed8107600 0x7feed819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedfc4b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feed810d410 con 0x7feed81023a0 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedfc4b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feed810d290 con 0x7feed8104cd0 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedfc4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feed810d590 con 0x7feed8107600 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd9e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feed81023a0 0x7feed8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd1e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed8104cd0 0x7feed81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd9e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feed81023a0 0x7feed8197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55890/0 (socket says 172.21.15.105:55890) 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd9e7640 1 -- 172.21.15.105:0/1747782285 learned_addr learned my addr 172.21.15.105:0/1747782285 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feede1e8640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed8107600 0x7feed819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd9e7640 1 -- 172.21.15.105:0/1747782285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed8107600 msgr2=0x7feed819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd9e7640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed8107600 0x7feed819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd9e7640 1 -- 172.21.15.105:0/1747782285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed8104cd0 msgr2=0x7feed81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd9e7640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed8104cd0 0x7feed81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd9e7640 1 -- 172.21.15.105:0/1747782285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed819f780 con 0x7feed81023a0 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feede1e8640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed8107600 0x7feed819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.712+0000 7feedd1e6640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed8104cd0 0x7feed81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.713+0000 7feedd9e7640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feed81023a0 0x7feed8197bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feed4006010 tx=0x7feed4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.713+0000 7feeceffd640 1 -- 172.21.15.105:0/1747782285 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4043020 con 0x7feed81023a0 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.713+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed819fa10 con 0x7feed81023a0 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.713+0000 7feeceffd640 1 -- 172.21.15.105:0/1747782285 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feed40043d0 con 0x7feed81023a0 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.713+0000 7feeceffd640 1 -- 172.21.15.105:0/1747782285 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4003b80 con 0x7feed81023a0 2024-03-26T17:38:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.713+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed819ffb0 con 0x7feed81023a0 2024-03-26T17:38:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.714+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed80686b0 con 0x7feed81023a0 2024-03-26T17:38:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.715+0000 7feeceffd640 1 -- 172.21.15.105:0/1747782285 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7feed4046020 con 0x7feed81023a0 2024-03-26T17:38:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.715+0000 7feeceffd640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feec40a2fb0 0x7feec40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.715+0000 7feedd1e6640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feec40a2fb0 0x7feec40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.715+0000 7feeceffd640 1 -- 172.21.15.105:0/1747782285 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7feed40f1630 con 0x7feed81023a0 2024-03-26T17:38:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.716+0000 7feedd1e6640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feec40a2fb0 0x7feec40a5470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feed8199070 tx=0x7feec8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:25.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.720+0000 7feeceffd640 1 -- 172.21.15.105:0/1747782285 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feed404d020 con 0x7feed81023a0 2024-03-26T17:38:25.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.917+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feed80624c0 con 0x7feed81023a0 2024-03-26T17:38:25.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.917+0000 7feeceffd640 1 -- 172.21.15.105:0/1747782285 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7feed40be430 con 0x7feed81023a0 2024-03-26T17:38:25.910 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T17:38:25.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.919+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feec40a2fb0 msgr2=0x7feec40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.919+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feec40a2fb0 0x7feec40a5470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feed8199070 tx=0x7feec8005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:25.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.919+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feed81023a0 msgr2=0x7feed8197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.919+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feed81023a0 0x7feed8197bb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feed4006010 tx=0x7feed4005e40 comp rx=0 tx=0).stop 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.920+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 shutdown_connections 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.920+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feec40a2fb0 0x7feec40a5470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.920+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed8107600 0x7feed819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.920+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feed8104cd0 0x7feed81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.920+0000 7feedfc4b640 1 --2- 172.21.15.105:0/1747782285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feed81023a0 0x7feed8197bb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.920+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 >> 172.21.15.105:0/1747782285 conn(0x7feed80fbd50 msgr2=0x7feed8102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.920+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 shutdown_connections 2024-03-26T17:38:25.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:25.920+0000 7feedfc4b640 1 -- 172.21.15.105:0/1747782285 wait complete. 2024-03-26T17:38:25.923 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T17:38:26.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/1722999608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a41075a0 msgr2=0x7fc8a41099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/1722999608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a41075a0 0x7fc8a41099f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009ca0 tx=0x7fc89c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/1722999608 shutdown_connections 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/1722999608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a41075a0 0x7fc8a41099f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/1722999608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8a4104c70 0x7fc8a4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/1722999608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a4102340 0x7fc8a4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/1722999608 >> 172.21.15.105:0/1722999608 conn(0x7fc8a40fbd50 msgr2=0x7fc8a40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/1722999608 shutdown_connections 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.041+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/1722999608 wait complete. 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a9cc6640 1 Processor -- start 2024-03-26T17:38:26.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a9cc6640 1 -- start start 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a9cc6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8a4102340 0x7fc8a4101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a9cc6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a4104c70 0x7fc8a41002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a9cc6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a41075a0 0x7fc8a4100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a9cc6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a410d3f0 con 0x7fc8a4104c70 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a9cc6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a410d270 con 0x7fc8a4102340 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a9cc6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a410d570 con 0x7fc8a41075a0 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a4104c70 0x7fc8a41002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8a4102340 0x7fc8a4101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a4104c70 0x7fc8a41002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55914/0 (socket says 172.21.15.105:55914) 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a41075a0 0x7fc8a4100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a2ffd640 1 -- 172.21.15.105:0/2934067054 learned_addr learned my addr 172.21.15.105:0/2934067054 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:26.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a2ffd640 1 -- 172.21.15.105:0/2934067054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a41075a0 msgr2=0x7fc8a4100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a2ffd640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a41075a0 0x7fc8a4100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a2ffd640 1 -- 172.21.15.105:0/2934067054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8a4102340 msgr2=0x7fc8a4101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a2ffd640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8a4102340 0x7fc8a4101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.042+0000 7fc8a2ffd640 1 -- 172.21.15.105:0/2934067054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a41010c0 con 0x7fc8a4104c70 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.043+0000 7fc8a2ffd640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a4104c70 0x7fc8a41002f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e9a0 tx=0x7fc89800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.043+0000 7fc8a0ff9640 1 -- 172.21.15.105:0/2934067054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89800cd80 con 0x7fc8a4104c70 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.043+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a41a1c10 con 0x7fc8a4104c70 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.043+0000 7fc8a0ff9640 1 -- 172.21.15.105:0/2934067054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8980035f0 con 0x7fc8a4104c70 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.043+0000 7fc8a0ff9640 1 -- 172.21.15.105:0/2934067054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898010690 con 0x7fc8a4104c70 2024-03-26T17:38:26.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.043+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a41a21b0 con 0x7fc8a4104c70 2024-03-26T17:38:26.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.044+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a40fd840 con 0x7fc8a4104c70 2024-03-26T17:38:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.045+0000 7fc8a0ff9640 1 -- 172.21.15.105:0/2934067054 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fc8980037a0 con 0x7fc8a4104c70 2024-03-26T17:38:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.045+0000 7fc8a0ff9640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc88c0a2fb0 0x7fc88c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.046+0000 7fc8a37fe640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc88c0a2fb0 0x7fc88c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.046+0000 7fc8a0ff9640 1 -- 172.21.15.105:0/2934067054 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fc8980cf810 con 0x7fc8a4104c70 2024-03-26T17:38:26.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.046+0000 7fc8a37fe640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc88c0a2fb0 0x7fc88c0a5470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc894009bd0 tx=0x7fc894005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:26.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.050+0000 7fc8a0ff9640 1 -- 172.21.15.105:0/2934067054 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc89809c500 con 0x7fc8a4104c70 2024-03-26T17:38:26.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:25 smithi144 ceph-mon[36095]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:26.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:25 smithi144 ceph-mon[36095]: pgmap v346: 97 pgs: 97 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:38:26.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:25 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1215047592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:26.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:25 smithi105 ceph-mon[32280]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:26.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:25 smithi105 ceph-mon[32280]: pgmap v346: 97 pgs: 97 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:38:26.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:25 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1215047592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:25 smithi135 ceph-mon[35636]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:26.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:25 smithi135 ceph-mon[35636]: pgmap v346: 97 pgs: 97 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:38:26.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:25 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1215047592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:26.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.276+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8a402d410 con 0x7fc8a4104c70 2024-03-26T17:38:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.276+0000 7fc8a0ff9640 1 -- 172.21.15.105:0/2934067054 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fc89809be00 con 0x7fc8a4104c70 2024-03-26T17:38:26.269 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T17:38:26.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc88c0a2fb0 msgr2=0x7fc88c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc88c0a2fb0 0x7fc88c0a5470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc894009bd0 tx=0x7fc894005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a4104c70 msgr2=0x7fc8a41002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a4104c70 0x7fc8a41002f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e9a0 tx=0x7fc89800ee60 comp rx=0 tx=0).stop 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 shutdown_connections 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc88c0a2fb0 0x7fc88c0a5470 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8a41075a0 0x7fc8a4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8a4104c70 0x7fc8a41002f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 --2- 172.21.15.105:0/2934067054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8a4102340 0x7fc8a4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 >> 172.21.15.105:0/2934067054 conn(0x7fc8a40fbd50 msgr2=0x7fc8a4102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 shutdown_connections 2024-03-26T17:38:26.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.279+0000 7fc8a9cc6640 1 -- 172.21.15.105:0/2934067054 wait complete. 2024-03-26T17:38:26.283 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T17:38:26.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- 172.21.15.105:0/3630126956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701019b0 msgr2=0x7f0f70101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 --2- 172.21.15.105:0/3630126956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701019b0 0x7f0f70101e30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009900 tx=0x7f0f6002bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:26.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- 172.21.15.105:0/3630126956 shutdown_connections 2024-03-26T17:38:26.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 --2- 172.21.15.105:0/3630126956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f70102370 0x7f0f70108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 --2- 172.21.15.105:0/3630126956 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701019b0 0x7f0f70101e30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 --2- 172.21.15.105:0/3630126956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f701007c0 0x7f0f70100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- 172.21.15.105:0/3630126956 >> 172.21.15.105:0/3630126956 conn(0x7f0f700fbd50 msgr2=0x7f0f700fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- 172.21.15.105:0/3630126956 shutdown_connections 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- 172.21.15.105:0/3630126956 wait complete. 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 Processor -- start 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- start start 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701007c0 0x7f0f70069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f701019b0 0x7f0f7019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f70102370 0x7f0f7019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010c2f0 con 0x7f0f701007c0 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010c170 con 0x7f0f701019b0 2024-03-26T17:38:26.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.397+0000 7f0f77de6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7010c470 con 0x7f0f70102370 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701007c0 0x7f0f70069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701007c0 0x7f0f70069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55920/0 (socket says 172.21.15.105:55920) 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 -- 172.21.15.105:0/4097437354 learned_addr learned my addr 172.21.15.105:0/4097437354 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75381640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f701019b0 0x7f0f7019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f76383640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f70102370 0x7f0f7019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 -- 172.21.15.105:0/4097437354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f70102370 msgr2=0x7f0f7019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f70102370 0x7f0f7019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 -- 172.21.15.105:0/4097437354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f701019b0 msgr2=0x7f0f7019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f701019b0 0x7f0f7019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 -- 172.21.15.105:0/4097437354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7019f7f0 con 0x7f0f701007c0 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f75b82640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701007c0 0x7f0f70069590 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00e9c0 tx=0x7f0f6c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f66ffd640 1 -- 172.21.15.105:0/4097437354 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00ccc0 con 0x7f0f701007c0 2024-03-26T17:38:26.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f66ffd640 1 -- 172.21.15.105:0/4097437354 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0f6c0035f0 con 0x7f0f701007c0 2024-03-26T17:38:26.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7019fa80 con 0x7f0f701007c0 2024-03-26T17:38:26.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f66ffd640 1 -- 172.21.15.105:0/4097437354 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00ce60 con 0x7f0f701007c0 2024-03-26T17:38:26.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.398+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f7019ff50 con 0x7f0f701007c0 2024-03-26T17:38:26.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.399+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f70107ba0 con 0x7f0f701007c0 2024-03-26T17:38:26.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.401+0000 7f0f66ffd640 1 -- 172.21.15.105:0/4097437354 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f0f6c0037a0 con 0x7f0f701007c0 2024-03-26T17:38:26.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.402+0000 7f0f66ffd640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0f5c0a2fb0 0x7f0f5c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.402+0000 7f0f66ffd640 1 -- 172.21.15.105:0/4097437354 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f0f6c0cfa30 con 0x7f0f701007c0 2024-03-26T17:38:26.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.402+0000 7f0f75381640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0f5c0a2fb0 0x7f0f5c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.402+0000 7f0f75381640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0f5c0a2fb0 0x7f0f5c0a5470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009230 tx=0x7f0f60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:26.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.406+0000 7f0f66ffd640 1 -- 172.21.15.105:0/4097437354 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0f6c09c720 con 0x7f0f701007c0 2024-03-26T17:38:26.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.615+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f700624c0 con 0x7f0f701007c0 2024-03-26T17:38:26.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.615+0000 7f0f66ffd640 1 -- 172.21.15.105:0/4097437354 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f0f6c09c020 con 0x7f0f701007c0 2024-03-26T17:38:26.608 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0f5c0a2fb0 msgr2=0x7f0f5c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0f5c0a2fb0 0x7f0f5c0a5470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009230 tx=0x7f0f60005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701007c0 msgr2=0x7f0f70069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701007c0 0x7f0f70069590 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00e9c0 tx=0x7f0f6c00edc0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 shutdown_connections 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0f5c0a2fb0 0x7f0f5c0a5470 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f70102370 0x7f0f7019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f701019b0 0x7f0f7019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 --2- 172.21.15.105:0/4097437354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f701007c0 0x7f0f70069590 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 >> 172.21.15.105:0/4097437354 conn(0x7f0f700fbd50 msgr2=0x7f0f70104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 shutdown_connections 2024-03-26T17:38:26.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.617+0000 7f0f77de6640 1 -- 172.21.15.105:0/4097437354 wait complete. 2024-03-26T17:38:26.620 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:38:26.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 -- 172.21.15.105:0/440128102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 msgr2=0x7efd48108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/440128102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 0x7efd48108c20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd40009be0 tx=0x7efd4002bef0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 -- 172.21.15.105:0/440128102 shutdown_connections 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/440128102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 0x7efd48108c20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/440128102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd481019d0 0x7efd48101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/440128102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd481007e0 0x7efd48100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 -- 172.21.15.105:0/440128102 >> 172.21.15.105:0/440128102 conn(0x7efd480fbd50 msgr2=0x7efd480fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 -- 172.21.15.105:0/440128102 shutdown_connections 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 -- 172.21.15.105:0/440128102 wait complete. 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.737+0000 7efd4ebe6640 1 Processor -- start 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4ebe6640 1 -- start start 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4ebe6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd481007e0 0x7efd48197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4ebe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd481019d0 0x7efd48198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4ebe6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 0x7efd4819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4ebe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efd4810c340 con 0x7efd481019d0 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4ebe6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd4810c1c0 con 0x7efd48102390 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4ebe6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd4810c4c0 con 0x7efd481007e0 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd481019d0 0x7efd48198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4d183640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 0x7efd4819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd481019d0 0x7efd48198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55946/0 (socket says 172.21.15.105:55946) 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4c982640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd481007e0 0x7efd48197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 -- 172.21.15.105:0/3308782327 learned_addr learned my addr 172.21.15.105:0/3308782327 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 -- 172.21.15.105:0/3308782327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd481007e0 msgr2=0x7efd48197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd481007e0 0x7efd48197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 -- 172.21.15.105:0/3308782327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 msgr2=0x7efd4819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 0x7efd4819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 -- 172.21.15.105:0/3308782327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4819f6f0 con 0x7efd481019d0 2024-03-26T17:38:26.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd4d183640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 0x7efd4819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.738+0000 7efd47fff640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd481019d0 0x7efd48198010 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efd3400e9f0 tx=0x7efd3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.739+0000 7efd45ffb640 1 -- 172.21.15.105:0/3308782327 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3400cd80 con 0x7efd481019d0 2024-03-26T17:38:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.739+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4819f9e0 con 0x7efd481019d0 2024-03-26T17:38:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.739+0000 7efd45ffb640 1 -- 172.21.15.105:0/3308782327 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd340108e0 con 0x7efd481019d0 2024-03-26T17:38:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.739+0000 7efd45ffb640 1 -- 172.21.15.105:0/3308782327 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34018920 con 0x7efd481019d0 2024-03-26T17:38:26.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.739+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4819ff80 con 0x7efd481019d0 2024-03-26T17:38:26.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.740+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd481a0210 con 0x7efd481019d0 2024-03-26T17:38:26.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.741+0000 7efd45ffb640 1 -- 172.21.15.105:0/3308782327 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7efd34010a90 con 0x7efd481019d0 2024-03-26T17:38:26.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.741+0000 7efd45ffb640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd380a2fb0 0x7efd380a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:26.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.742+0000 7efd4c982640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd380a2fb0 0x7efd380a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:26.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.742+0000 7efd45ffb640 1 -- 172.21.15.105:0/3308782327 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7efd340cfd10 con 0x7efd481019d0 2024-03-26T17:38:26.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.742+0000 7efd4c982640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd380a2fb0 0x7efd380a5470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009530 tx=0x7efd3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:26.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.745+0000 7efd45ffb640 1 -- 172.21.15.105:0/3308782327 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efd3409c3b0 con 0x7efd481019d0 2024-03-26T17:38:26.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:26 smithi105 ceph-mon[32280]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:26.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:26 smithi105 ceph-mon[32280]: from='client.25282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:26.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:26 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1747782285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:26.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:26 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2934067054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:26.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:26 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4097437354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:26.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.962+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd481a03e0 con 0x7efd481019d0 2024-03-26T17:38:26.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.963+0000 7efd45ffb640 1 -- 172.21.15.105:0/3308782327 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7efd340a0450 con 0x7efd481019d0 2024-03-26T17:38:26.956 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:26.956 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:38:20.295050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T17:38:26.956 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd380a2fb0 msgr2=0x7efd380a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd380a2fb0 0x7efd380a5470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009530 tx=0x7efd3c009260 comp rx=0 tx=0).stop 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd481019d0 msgr2=0x7efd48198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd481019d0 0x7efd48198010 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efd3400e9f0 tx=0x7efd3400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 shutdown_connections 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd380a2fb0 0x7efd380a5470 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd48102390 0x7efd4819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd481019d0 0x7efd48198010 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 --2- 172.21.15.105:0/3308782327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd481007e0 0x7efd48197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 >> 172.21.15.105:0/3308782327 conn(0x7efd480fbd50 msgr2=0x7efd480fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:26.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 shutdown_connections 2024-03-26T17:38:26.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:26.965+0000 7efd4ebe6640 1 -- 172.21.15.105:0/3308782327 wait complete. 2024-03-26T17:38:26.969 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 -- 172.21.15.105:0/242143586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d81019d0 msgr2=0x7f56d8101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 --2- 172.21.15.105:0/242143586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d81019d0 0x7f56d8101e50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009900 tx=0x7f56c802bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 -- 172.21.15.105:0/242143586 shutdown_connections 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 --2- 172.21.15.105:0/242143586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d8102390 0x7f56d8108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 --2- 172.21.15.105:0/242143586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d81019d0 0x7f56d8101e50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 --2- 172.21.15.105:0/242143586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81007e0 0x7f56d8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 -- 172.21.15.105:0/242143586 >> 172.21.15.105:0/242143586 conn(0x7f56d80fbd50 msgr2=0x7f56d80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 -- 172.21.15.105:0/242143586 shutdown_connections 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.080+0000 7f56e0577640 1 -- 172.21.15.105:0/242143586 wait complete. 2024-03-26T17:38:27.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56e0577640 1 Processor -- start 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56e0577640 1 -- start start 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56e0577640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d81007e0 0x7f56d8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56e0577640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81019d0 0x7f56d81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56e0577640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d8102390 0x7f56d819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56e0577640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f56d810c2f0 con 0x7f56d8102390 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56e0577640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56d810c170 con 0x7f56d81019d0 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56e0577640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56d810c470 con 0x7f56d81007e0 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56deb14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d8102390 0x7f56d819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56deb14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d8102390 0x7f56d819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55958/0 (socket says 172.21.15.105:55958) 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56de313640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d81007e0 0x7f56d8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56deb14640 1 -- 172.21.15.105:0/3256750949 learned_addr learned my addr 172.21.15.105:0/3256750949 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56ddb12640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81019d0 0x7f56d81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56deb14640 1 -- 172.21.15.105:0/3256750949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d81007e0 msgr2=0x7f56d8197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56deb14640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d81007e0 0x7f56d8197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56deb14640 1 -- 172.21.15.105:0/3256750949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81019d0 msgr2=0x7f56d81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56deb14640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81019d0 0x7f56d81980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.081+0000 7f56deb14640 1 -- 172.21.15.105:0/3256750949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d819f780 con 0x7f56d8102390 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.082+0000 7f56deb14640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d8102390 0x7f56d819f110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f56d000eb10 tx=0x7f56d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.082+0000 7f56cf7fe640 1 -- 172.21.15.105:0/3256750949 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0004b20 con 0x7f56d8102390 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.082+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d819fa70 con 0x7f56d8102390 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.082+0000 7f56cf7fe640 1 -- 172.21.15.105:0/3256750949 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56d0004cc0 con 0x7f56d8102390 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.082+0000 7f56cf7fe640 1 -- 172.21.15.105:0/3256750949 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0010690 con 0x7f56d8102390 2024-03-26T17:38:27.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.082+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d819ff40 con 0x7f56d8102390 2024-03-26T17:38:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.083+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d81a0230 con 0x7f56d8102390 2024-03-26T17:38:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.084+0000 7f56cf7fe640 1 -- 172.21.15.105:0/3256750949 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f56d0005000 con 0x7f56d8102390 2024-03-26T17:38:27.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.085+0000 7f56cf7fe640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56c40a2fb0 0x7f56c40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.085+0000 7f56de313640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56c40a2fb0 0x7f56c40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.085+0000 7f56cf7fe640 1 -- 172.21.15.105:0/3256750949 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f56d00cfcf0 con 0x7f56d8102390 2024-03-26T17:38:27.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.085+0000 7f56de313640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56c40a2fb0 0x7f56c40a5470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56d40095f0 tx=0x7f56d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.089+0000 7f56cf7fe640 1 -- 172.21.15.105:0/3256750949 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f56d0053740 con 0x7f56d8102390 2024-03-26T17:38:27.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:26 smithi144 ceph-mon[36095]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:26 smithi144 ceph-mon[36095]: from='client.25282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:26 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1747782285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:26 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2934067054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:26 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4097437354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:26 smithi135 ceph-mon[35636]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:26 smithi135 ceph-mon[35636]: from='client.25282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T17:38:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:26 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1747782285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T17:38:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:26 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2934067054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T17:38:27.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:26 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4097437354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T17:38:27.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.272+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56d81a0400 con 0x7f56d8102390 2024-03-26T17:38:27.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.273+0000 7f56cf7fe640 1 -- 172.21.15.105:0/3256750949 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22693 (secure 0 0 0) 0x7f56d009c9e0 con 0x7f56d8102390 2024-03-26T17:38:27.266 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:27.266 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":87,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T17:37:27.733852+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T17:38:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56c40a2fb0 msgr2=0x7f56c40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56c40a2fb0 0x7f56c40a5470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56d40095f0 tx=0x7f56d4005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d8102390 msgr2=0x7f56d819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d8102390 0x7f56d819f110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f56d000eb10 tx=0x7f56d000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 shutdown_connections 2024-03-26T17:38:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56c40a2fb0 0x7f56c40a5470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d8102390 0x7f56d819f110 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56d81019d0 0x7f56d81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 --2- 172.21.15.105:0/3256750949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56d81007e0 0x7f56d8197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 >> 172.21.15.105:0/3256750949 conn(0x7f56d80fbd50 msgr2=0x7f56d8068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 shutdown_connections 2024-03-26T17:38:27.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.275+0000 7f56e0577640 1 -- 172.21.15.105:0/3256750949 wait complete. 2024-03-26T17:38:27.279 DEBUG:tasks.fwd_scrub:Yielding 2024-03-26T17:38:27.280 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-26T17:38:27.306 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T17:38:27.307 DEBUG:teuthology.orchestra.run.smithi105:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-26T17:38:27.307 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-26T17:38:27.307 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:38:27.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:38:27.308 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:38:27.310 INFO:teuthology.run_tasks:Running task workunit... 2024-03-26T17:38:27.323 INFO:tasks.workunit:Pulling workunits from ref 72e2437271dcafb9826f2f49bb1001426fa6de81 2024-03-26T17:38:27.324 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-26T17:38:27.324 INFO:tasks.workunit:timeout=3h 2024-03-26T17:38:27.324 INFO:tasks.workunit:cleanup=True 2024-03-26T17:38:27.324 DEBUG:teuthology.orchestra.run.smithi105:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-26 17:36:28.669816810 +0000 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-26 17:36:28.669816810 +0000 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-26 17:38:19.917062225 +0000 2024-03-26T17:38:27.342 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-03-26T17:38:27.343 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-26T17:38:27.343 DEBUG:teuthology.orchestra.run.smithi105:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-26T17:38:27.417 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ljflores/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 72e2437271dcafb9826f2f49bb1001426fa6de81 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 -- 172.21.15.105:0/1245009418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a281007c0 msgr2=0x7f7a28100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 --2- 172.21.15.105:0/1245009418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a281007c0 0x7f7a28100be0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009900 tx=0x7f7a1802bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 -- 172.21.15.105:0/1245009418 shutdown_connections 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 --2- 172.21.15.105:0/1245009418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a28102370 0x7f7a28108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 --2- 172.21.15.105:0/1245009418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a281019b0 0x7f7a28101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 --2- 172.21.15.105:0/1245009418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a281007c0 0x7f7a28100be0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 -- 172.21.15.105:0/1245009418 >> 172.21.15.105:0/1245009418 conn(0x7f7a280fbd50 msgr2=0x7f7a280fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 -- 172.21.15.105:0/1245009418 shutdown_connections 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.427+0000 7f7a2d159640 1 -- 172.21.15.105:0/1245009418 wait complete. 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.428+0000 7f7a2d159640 1 Processor -- start 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.428+0000 7f7a2d159640 1 -- start start 2024-03-26T17:38:27.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2d159640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a281007c0 0x7f7a28196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2d159640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a281019b0 0x7f7a28196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2d159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a28102370 0x7f7a2819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2d159640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810c2c0 con 0x7f7a28102370 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2d159640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810c140 con 0x7f7a281019b0 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2d159640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2810c440 con 0x7f7a281007c0 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a28102370 0x7f7a2819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a1e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a281019b0 0x7f7a28196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a28102370 0x7f7a2819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55982/0 (socket says 172.21.15.105:55982) 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 -- 172.21.15.105:0/3582293673 learned_addr learned my addr 172.21.15.105:0/3582293673 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a281007c0 0x7f7a28196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 -- 172.21.15.105:0/3582293673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a281007c0 msgr2=0x7f7a28196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a281007c0 0x7f7a28196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 -- 172.21.15.105:0/3582293673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a281019b0 msgr2=0x7f7a28196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a281019b0 0x7f7a28196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 -- 172.21.15.105:0/3582293673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2819e690 con 0x7f7a28102370 2024-03-26T17:38:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a281019b0 0x7f7a28196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2759e640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a28102370 0x7f7a2819e020 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000efd0 tx=0x7f7a2000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a24d99640 1 -- 172.21.15.105:0/3582293673 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20003b70 con 0x7f7a28102370 2024-03-26T17:38:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2d159640 1 -- 172.21.15.105:0/3582293673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a28068a80 con 0x7f7a28102370 2024-03-26T17:38:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a24d99640 1 -- 172.21.15.105:0/3582293673 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a20003d10 con 0x7f7a28102370 2024-03-26T17:38:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a24d99640 1 -- 172.21.15.105:0/3582293673 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20010980 con 0x7f7a28102370 2024-03-26T17:38:27.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.429+0000 7f7a2d159640 1 -- 172.21.15.105:0/3582293673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a28068f80 con 0x7f7a28102370 2024-03-26T17:38:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.432+0000 7f7a24d99640 1 -- 172.21.15.105:0/3582293673 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f7a20004020 con 0x7f7a28102370 2024-03-26T17:38:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.432+0000 7f7a2fe11740 1 Processor -- start 2024-03-26T17:38:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.432+0000 7f7a2fe11740 1 -- start start 2024-03-26T17:38:27.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.432+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.432+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7dc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5638ba739630 con 0x5638ba7d9cf0 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5638ba6e1e10 con 0x5638ba7d4190 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638ba7355e0 con 0x5638ba7d4b80 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a24d99640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a140a2fb0 0x7f7a140a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a1e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7d9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44508/0 (socket says 172.21.15.105:44508) 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 -- 172.21.15.105:0/3721236603 learned_addr learned my addr 172.21.15.105:0/3721236603 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a1e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7d45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38440/0 (socket says 172.21.15.105:38440) 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a2759e640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a140a2fb0 0x7f7a140a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a24d99640 1 -- 172.21.15.105:0/3582293673 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f7a200cf990 con 0x7f7a28102370 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 -- 172.21.15.105:0/3721236603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 msgr2=0x5638ba7d45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7d45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 -- 172.21.15.105:0/3721236603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 msgr2=0x5638ba7dc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7dc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 -- 172.21.15.105:0/3721236603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ba6f8740 con 0x5638ba7d4b80 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a2759e640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a140a2fb0 0x7f7a140a5470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a20098930 tx=0x7f7a20010430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.433+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7d9680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009900 tx=0x7f7a18009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbcc645600706465 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.434+0000 7f7a1d59a640 1 -- 172.21.15.105:0/3721236603 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18045020 con 0x5638ba7d4b80 2024-03-26T17:38:27.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.434+0000 7f7a1d59a640 1 -- 172.21.15.105:0/3721236603 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a18004040 con 0x5638ba7d4b80 2024-03-26T17:38:27.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.434+0000 7f7a1d59a640 1 -- 172.21.15.105:0/3721236603 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18005500 con 0x5638ba7d4b80 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- 172.21.15.105:0/3721236603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 msgr2=0x5638ba7d9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7d9680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009900 tx=0x7f7a18009560 comp rx=0 tx=0).stop 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- 172.21.15.105:0/3721236603 shutdown_connections 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7dc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7d9680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/3721236603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- 172.21.15.105:0/3721236603 >> 172.21.15.105:0/3721236603 conn(0x5638ba7cf700 msgr2=0x5638ba7cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- 172.21.15.105:0/3721236603 shutdown_connections 2024-03-26T17:38:27.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- 172.21.15.105:0/3721236603 wait complete. 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- start start 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 Processor -- start 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4190 0x5638ba7de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4b80 0x5638ba7dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5638ba7dcc40 con 0x5638ba7d9cf0 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5638ba6f51a0 con 0x5638ba7d4b80 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638ba7dcdc0 con 0x5638ba7d4190 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.435+0000 7f7a2fe11740 10 client.0 _ll_register_callbacks cb 0x5638ba44cc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4b80 0x5638ba7dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4b80 0x5638ba7dec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38450/0 (socket says 172.21.15.105:38450) 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 -- 172.21.15.105:0/706357576 learned_addr learned my addr 172.21.15.105:0/706357576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4190 0x5638ba7de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 -- 172.21.15.105:0/706357576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4190 msgr2=0x5638ba7de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4190 0x5638ba7de6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 -- 172.21.15.105:0/706357576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 msgr2=0x5638ba7e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 -- 172.21.15.105:0/706357576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ba5157e0 con 0x5638ba7d4b80 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4b80 0x5638ba7dec10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a18000c00 tx=0x7f7a18005550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe584efeb6315ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/706357576 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18045070 con 0x5638ba7d4b80 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/706357576 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f7a180046c0 con 0x5638ba7d4b80 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.436+0000 7f7a2fe11740 1 -- 172.21.15.105:0/706357576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a18004040 con 0x5638ba7d4b80 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/706357576 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a1805a050 con 0x5638ba7d4b80 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a2fe11740 10 client.25334 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a2fe11740 1 -- 172.21.15.105:0/706357576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638ba575f70 con 0x5638ba7d4b80 2024-03-26T17:38:27.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/706357576 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a18004d70 con 0x5638ba7d4b80 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/706357576 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7a18003a90 con 0x5638ba7d4b80 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a2fe11740 10 client.25334 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a2fe11740 10 client.25334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a2fe11740 20 client.25334 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638ba7ea140 0x5638ba7ec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a2fe11740 4 client.25334 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a2fe11740 1 -- 172.21.15.105:0/706357576 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7a18004040 con 0x5638ba7ea140 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.437+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638ba7ea140 0x5638ba7ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638ba7ea140 0x5638ba7ec550 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f43405fd38ce06 server_cookie=33c687d98ee849c7 in_seq=0 out_seq=0 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a0e7fc640 10 client.25334 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/706357576 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f7a18004040 con 0x5638ba7ea140 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a0e7fc640 10 client.25334 handle_command_reply: tid=0 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a2fe11740 1 client.25334 shutdown 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a2fe11740 1 -- 172.21.15.105:0/706357576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4b80 msgr2=0x5638ba7dec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4b80 0x5638ba7dec10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a18000c00 tx=0x7f7a18005550 comp rx=0 tx=0).stop 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a2fe11740 1 -- 172.21.15.105:0/706357576 shutdown_connections 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638ba7ea140 0x5638ba7ec550 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.438+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.439+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4b80 0x5638ba7dec10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.439+0000 7f7a0e7fc640 0 client.25334 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.439+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/706357576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4190 0x5638ba7de6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.439+0000 7f7a2fe11740 1 -- 172.21.15.105:0/706357576 >> 172.21.15.105:0/706357576 conn(0x5638ba7cf700 msgr2=0x5638ba7ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.439+0000 7f7a2fe11740 1 -- 172.21.15.105:0/706357576 shutdown_connections 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.439+0000 7f7a2fe11740 1 -- 172.21.15.105:0/706357576 wait complete. 2024-03-26T17:38:27.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.439+0000 7f7a2fe11740 20 client.25334 trim_cache size 0 max 16384 2024-03-26T17:38:27.438 INFO:tasks.workunit.client.0.smithi105.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.462+0000 7f7a2fe11740 1 Processor -- start 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.462+0000 7f7a2fe11740 1 -- start start 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.462+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4190 0x5638ba7ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.462+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.462+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d9cf0 0x5638ba7ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.462+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a68e0 con 0x5638ba7d4190 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.462+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a8420 con 0x5638ba7d9cf0 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.462+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a6fb0 con 0x5638ba7d4b80 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a1e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4190 0x5638ba7ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a1e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4190 0x5638ba7ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56018/0 (socket says 172.21.15.105:56018) 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a26d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a1e59c640 1 -- 172.21.15.105:0/4242382902 learned_addr learned my addr 172.21.15.105:0/4242382902 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d9cf0 0x5638ba7ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a26d9d640 1 -- 172.21.15.105:0/4242382902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d9cf0 msgr2=0x5638ba7ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d9cf0 0x5638ba7ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a26d9d640 1 -- 172.21.15.105:0/4242382902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4190 msgr2=0x5638ba7ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4190 0x5638ba7ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a26d9d640 1 -- 172.21.15.105:0/4242382902 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ba40d170 con 0x5638ba7d4b80 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4190 0x5638ba7ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7e5c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a1802c170 tx=0x7f7a18009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dba2700b9c6e490 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a1cd99640 1 -- 172.21.15.105:0/4242382902 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18057020 con 0x5638ba7d4b80 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a1cd99640 1 -- 172.21.15.105:0/4242382902 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a180055c0 con 0x5638ba7d4b80 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.463+0000 7f7a1cd99640 1 -- 172.21.15.105:0/4242382902 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18045040 con 0x5638ba7d4b80 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 -- 172.21.15.105:0/4242382902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 msgr2=0x5638ba7e5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7e5c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a1802c170 tx=0x7f7a18009670 comp rx=0 tx=0).stop 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 -- 172.21.15.105:0/4242382902 shutdown_connections 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d9cf0 0x5638ba7ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d4b80 0x5638ba7e5c10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/4242382902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4190 0x5638ba7ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 -- 172.21.15.105:0/4242382902 >> 172.21.15.105:0/4242382902 conn(0x5638ba7cf700 msgr2=0x5638ba7dd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 -- 172.21.15.105:0/4242382902 shutdown_connections 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 -- 172.21.15.105:0/4242382902 wait complete. 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 -- start start 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.464+0000 7f7a2fe11740 1 Processor -- start 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7e3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4b80 0x5638ba7e3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a7d50 con 0x5638ba7d4b80 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a7680 con 0x5638ba7d4190 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a281a48c0 con 0x5638ba7d9cf0 2024-03-26T17:38:27.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 10 client.0 _ll_register_callbacks cb 0x7f7a10017690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4b80 0x5638ba7e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4b80 0x5638ba7e3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56020/0 (socket says 172.21.15.105:56020) 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 -- 172.21.15.105:0/2686873174 learned_addr learned my addr 172.21.15.105:0/2686873174 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 -- 172.21.15.105:0/2686873174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d9cf0 msgr2=0x5638ba7e49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 -- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 msgr2=0x5638ba7e3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7e3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 -- 172.21.15.105:0/2686873174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ba7e4f90 con 0x5638ba7d4b80 2024-03-26T17:38:27.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7e3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a26d9d640 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4b80 0x5638ba7e3990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005520 tx=0x7f7a1802cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c8329c11c73ecc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/2686873174 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18057030 con 0x5638ba7d4b80 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/2686873174 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7f7a18003a00 con 0x5638ba7d4b80 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 1 -- 172.21.15.105:0/2686873174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638ba746b70 con 0x5638ba7d4b80 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/2686873174 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a1806e050 con 0x5638ba7d4b80 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.465+0000 7f7a2fe11740 10 client.15474 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a2fe11740 1 -- 172.21.15.105:0/2686873174 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5638ba8505f0 con 0x5638ba7d4b80 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/2686873174 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5638ba746b70 con 0x5638ba7d4b80 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/2686873174 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7a18034560 con 0x5638ba7d4b80 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a2fe11740 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a2fe11740 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a2fe11740 20 client.15474 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:27.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638ba850ee0 0x5638ba8532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a2fe11740 4 client.15474 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:38:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a2fe11740 1 -- 172.21.15.105:0/2686873174 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5638ba740ec0 con 0x5638ba850ee0 2024-03-26T17:38:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638ba850ee0 0x5638ba8532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a0e7fc640 10 client.15474 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:27.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.466+0000 7f7a1e59c640 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638ba850ee0 0x5638ba8532d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44b5299090ce0f1 server_cookie=dc5642d4d525c312 in_seq=0 out_seq=0 2024-03-26T17:38:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.467+0000 7f7a0e7fc640 1 -- 172.21.15.105:0/2686873174 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5638ba740ec0 con 0x5638ba850ee0 2024-03-26T17:38:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.467+0000 7f7a0e7fc640 10 client.15474 handle_command_reply: tid=0 2024-03-26T17:38:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.467+0000 7f7a2fe11740 1 client.15474 shutdown 2024-03-26T17:38:27.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.467+0000 7f7a2fe11740 1 -- 172.21.15.105:0/2686873174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4b80 msgr2=0x5638ba7e3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.467+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4b80 0x5638ba7e3990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005520 tx=0x7f7a1802cd20 comp rx=0 tx=0).stop 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 1 -- 172.21.15.105:0/2686873174 shutdown_connections 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638ba850ee0 0x5638ba8532d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638ba7d9cf0 0x5638ba7e49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a0e7fc640 0 client.15474 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638ba7d4b80 0x5638ba7e3990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 1 --2- 172.21.15.105:0/2686873174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638ba7d4190 0x5638ba7e3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 1 -- 172.21.15.105:0/2686873174 >> 172.21.15.105:0/2686873174 conn(0x5638ba7cf700 msgr2=0x5638ba7d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 1 -- 172.21.15.105:0/2686873174 shutdown_connections 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 1 -- 172.21.15.105:0/2686873174 wait complete. 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2fe11740 20 client.15474 trim_cache size 0 max 16384 2024-03-26T17:38:27.466 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:27.467 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:38:27.467 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "c02e800b-0148-4fd7-823f-0b84e91bc47c", 2024-03-26T17:38:27.467 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:38:27.467 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 -- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a140a2fb0 msgr2=0x7f7a140a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a140a2fb0 0x7f7a140a5470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a20098930 tx=0x7f7a20010430 comp rx=0 tx=0).stop 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 -- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a28102370 msgr2=0x7f7a2819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a28102370 0x7f7a2819e020 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000efd0 tx=0x7f7a2000c6a0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 -- 172.21.15.105:0/3582293673 shutdown_connections 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a140a2fb0 0x7f7a140a5470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a28102370 0x7f7a2819e020 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a281019b0 0x7f7a28196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.468+0000 7f7a2d159640 1 --2- 172.21.15.105:0/3582293673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a281007c0 0x7f7a28196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.469+0000 7f7a2d159640 1 -- 172.21.15.105:0/3582293673 >> 172.21.15.105:0/3582293673 conn(0x7f7a280fbd50 msgr2=0x7f7a28104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.469+0000 7f7a2d159640 1 -- 172.21.15.105:0/3582293673 shutdown_connections 2024-03-26T17:38:27.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.469+0000 7f7a2d159640 1 -- 172.21.15.105:0/3582293673 wait complete. 2024-03-26T17:38:27.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:38:27.472 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:38:27.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 -- 172.21.15.105:0/1958746885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c1018d0 msgr2=0x7fe54c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 --2- 172.21.15.105:0/1958746885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c1018d0 0x7fe54c101cf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009900 tx=0x7fe53c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 -- 172.21.15.105:0/1958746885 shutdown_connections 2024-03-26T17:38:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 --2- 172.21.15.105:0/1958746885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe54c103480 0x7fe54c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 --2- 172.21.15.105:0/1958746885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe54c102ac0 0x7fe54c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 --2- 172.21.15.105:0/1958746885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c1018d0 0x7fe54c101cf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 -- 172.21.15.105:0/1958746885 >> 172.21.15.105:0/1958746885 conn(0x7fe54c0fce60 msgr2=0x7fe54c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 -- 172.21.15.105:0/1958746885 shutdown_connections 2024-03-26T17:38:27.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 -- 172.21.15.105:0/1958746885 wait complete. 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 Processor -- start 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.629+0000 7fe5517cd640 1 -- start start 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe5517cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe54c1018d0 0x7fe54c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe5517cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe54c102ac0 0x7fe54c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe5517cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c103480 0x7fe54c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe5517cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c10d400 con 0x7fe54c103480 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe5517cd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c10d280 con 0x7fe54c1018d0 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe5517cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c10d580 con 0x7fe54c102ac0 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c103480 0x7fe54c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c103480 0x7fe54c19f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56032/0 (socket says 172.21.15.105:56032) 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe543fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe54c102ac0 0x7fe54c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe54b7fe640 1 -- 172.21.15.105:0/3495527969 learned_addr learned my addr 172.21.15.105:0/3495527969 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe54affd640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe54c1018d0 0x7fe54c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe543fff640 1 -- 172.21.15.105:0/3495527969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe54c1018d0 msgr2=0x7fe54c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe543fff640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe54c1018d0 0x7fe54c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe543fff640 1 -- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c103480 msgr2=0x7fe54c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe543fff640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c103480 0x7fe54c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe543fff640 1 -- 172.21.15.105:0/3495527969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c19f750 con 0x7fe54c102ac0 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe543fff640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe54c102ac0 0x7fe54c198070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe53400e9f0 tx=0x7fe53400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe548ff9640 1 -- 172.21.15.105:0/3495527969 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53400ccc0 con 0x7fe54c102ac0 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe548ff9640 1 -- 172.21.15.105:0/3495527969 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5340108e0 con 0x7fe54c102ac0 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe5517cd640 1 -- 172.21.15.105:0/3495527969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c19fa40 con 0x7fe54c102ac0 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe548ff9640 1 -- 172.21.15.105:0/3495527969 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534018880 con 0x7fe54c102ac0 2024-03-26T17:38:27.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.630+0000 7fe5517cd640 1 -- 172.21.15.105:0/3495527969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c19ff90 con 0x7fe54c102ac0 2024-03-26T17:38:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.632+0000 7fe548ff9640 1 -- 172.21.15.105:0/3495527969 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fe534010a90 con 0x7fe54c102ac0 2024-03-26T17:38:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe5543bb740 1 Processor -- start 2024-03-26T17:38:27.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe548ff9640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5380a2fb0 0x7fe5380a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe5543bb740 1 -- start start 2024-03-26T17:38:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5380a2fb0 0x7fe5380a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe548ff9640 1 -- 172.21.15.105:0/3495527969 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fe5340cf860 con 0x7fe54c102ac0 2024-03-26T17:38:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab2dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe5543bb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56486aa8d980 con 0x56486ab2e450 2024-03-26T17:38:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe5543bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56486aa35be0 con 0x56486ab292e0 2024-03-26T17:38:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe5543bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56486aa89930 con 0x56486ab288f0 2024-03-26T17:38:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.633+0000 7fe54affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab28d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44554/0 (socket says 172.21.15.105:44554) 2024-03-26T17:38:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54affd640 1 -- 172.21.15.105:0/3438364575 learned_addr learned my addr 172.21.15.105:0/3438364575 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe543fff640 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab2dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5380a2fb0 0x7fe5380a5470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe5440049d0 tx=0x7fe544011920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54affd640 1 -- 172.21.15.105:0/3438364575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 msgr2=0x56486ab2dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54affd640 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab2dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54affd640 1 -- 172.21.15.105:0/3438364575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 msgr2=0x56486ab308e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54affd640 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab308e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54affd640 1 -- 172.21.15.105:0/3438364575 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56486a86b7d0 con 0x56486ab288f0 2024-03-26T17:38:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe54affd640 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab28d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009900 tx=0x7fe53c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=412f73ee200b9642 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe5417fa640 1 -- 172.21.15.105:0/3438364575 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c045020 con 0x56486ab288f0 2024-03-26T17:38:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe5417fa640 1 -- 172.21.15.105:0/3438364575 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe53c004040 con 0x56486ab288f0 2024-03-26T17:38:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.634+0000 7fe5417fa640 1 -- 172.21.15.105:0/3438364575 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c005570 con 0x56486ab288f0 2024-03-26T17:38:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- 172.21.15.105:0/3438364575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 msgr2=0x56486ab28d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab28d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009900 tx=0x7fe53c009560 comp rx=0 tx=0).stop 2024-03-26T17:38:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- 172.21.15.105:0/3438364575 shutdown_connections 2024-03-26T17:38:27.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab308e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab2dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 --2- 172.21.15.105:0/3438364575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab28d10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- 172.21.15.105:0/3438364575 >> 172.21.15.105:0/3438364575 conn(0x56486ab23e60 msgr2=0x56486ab24260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- 172.21.15.105:0/3438364575 shutdown_connections 2024-03-26T17:38:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- 172.21.15.105:0/3438364575 wait complete. 2024-03-26T17:38:27.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- start start 2024-03-26T17:38:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 Processor -- start 2024-03-26T17:38:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab288f0 0x56486ab32b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab33040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab2e450 0x56486ab3a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56486ab313a0 con 0x56486ab288f0 2024-03-26T17:38:27.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56486aa49160 con 0x56486ab292e0 2024-03-26T17:38:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.635+0000 7fe5543bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56486ab31520 con 0x56486ab2e450 2024-03-26T17:38:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe5543bb740 10 client.0 _ll_register_callbacks cb 0x56486aa85720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab288f0 0x56486ab32b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab2e450 0x56486ab3a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab288f0 0x56486ab32b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56054/0 (socket says 172.21.15.105:56054) 2024-03-26T17:38:27.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 -- 172.21.15.105:0/2382089077 learned_addr learned my addr 172.21.15.105:0/2382089077 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab2e450 0x56486ab3a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44558/0 (socket says 172.21.15.105:44558) 2024-03-26T17:38:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe543fff640 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab33040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 -- 172.21.15.105:0/2382089077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab2e450 msgr2=0x56486ab3a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab2e450 0x56486ab3a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 -- 172.21.15.105:0/2382089077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 msgr2=0x56486ab33040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab33040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 -- 172.21.15.105:0/2382089077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56486a7a13f0 con 0x56486ab288f0 2024-03-26T17:38:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab2e450 0x56486ab3a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe54affd640 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab288f0 0x56486ab32b00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe53c02c170 tx=0x7fe53c005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb2d27c0368b718 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe5327fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe5327fc640 1 -- 172.21.15.105:0/2382089077 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c04e070 con 0x56486ab288f0 2024-03-26T17:38:27.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe5327fc640 1 -- 172.21.15.105:0/2382089077 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fe53c0035f0 con 0x56486ab288f0 2024-03-26T17:38:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe5543bb740 1 -- 172.21.15.105:0/2382089077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe53c004040 con 0x56486ab288f0 2024-03-26T17:38:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe5327fc640 1 -- 172.21.15.105:0/2382089077 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe53c003830 con 0x56486ab288f0 2024-03-26T17:38:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe5543bb740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.636+0000 7fe5543bb740 1 -- 172.21.15.105:0/2382089077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56486a7610f0 con 0x56486ab288f0 2024-03-26T17:38:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5327fc640 1 -- 172.21.15.105:0/2382089077 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe53c004040 con 0x56486ab288f0 2024-03-26T17:38:27.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5327fc640 1 -- 172.21.15.105:0/2382089077 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe53c05c070 con 0x56486ab288f0 2024-03-26T17:38:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5543bb740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5543bb740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5543bb740 20 client.15486 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56486ab3e5c0 0x56486ab409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5543bb740 4 client.15486 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:27.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5543bb740 1 -- 172.21.15.105:0/2382089077 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56486aa9aec0 con 0x56486ab3e5c0 2024-03-26T17:38:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe543fff640 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56486ab3e5c0 0x56486ab409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe5327fc640 10 client.15486 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.637+0000 7fe543fff640 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56486ab3e5c0 0x56486ab409d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32cb571fbc5d46a7 server_cookie=6a5c01c8d383d2cc in_seq=0 out_seq=0 2024-03-26T17:38:27.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5327fc640 1 -- 172.21.15.105:0/2382089077 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56486aa9aec0 con 0x56486ab3e5c0 2024-03-26T17:38:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5327fc640 10 client.15486 handle_command_reply: tid=0 2024-03-26T17:38:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 client.15486 shutdown 2024-03-26T17:38:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 -- 172.21.15.105:0/2382089077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab288f0 msgr2=0x56486ab32b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab288f0 0x56486ab32b00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe53c02c170 tx=0x7fe53c005540 comp rx=0 tx=0).stop 2024-03-26T17:38:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 -- 172.21.15.105:0/2382089077 shutdown_connections 2024-03-26T17:38:27.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56486ab3e5c0 0x56486ab409d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab2e450 0x56486ab3a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab33040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2382089077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab288f0 0x56486ab32b00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 -- 172.21.15.105:0/2382089077 >> 172.21.15.105:0/2382089077 conn(0x56486ab23e60 msgr2=0x56486ab23080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5327fc640 0 client.15486 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:27.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 -- 172.21.15.105:0/2382089077 shutdown_connections 2024-03-26T17:38:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.638+0000 7fe5543bb740 1 -- 172.21.15.105:0/2382089077 wait complete. 2024-03-26T17:38:27.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.639+0000 7fe5543bb740 20 client.15486 trim_cache size 0 max 16384 2024-03-26T17:38:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.655+0000 7fe5543bb740 1 Processor -- start 2024-03-26T17:38:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.655+0000 7fe5543bb740 1 -- start start 2024-03-26T17:38:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab318c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab227f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab22d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe5543bb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1a8b00 con 0x56486ab2e450 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe5543bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1aa640 con 0x56486ab292e0 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe5543bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1a91d0 con 0x56486ab288f0 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab22d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab22d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56060/0 (socket says 172.21.15.105:56060) 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 -- 172.21.15.105:0/4177671327 learned_addr learned my addr 172.21.15.105:0/4177671327 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe543fff640 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54affd640 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 -- 172.21.15.105:0/4177671327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 msgr2=0x56486ab318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab318c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 -- 172.21.15.105:0/4177671327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 msgr2=0x56486ab227f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab227f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 -- 172.21.15.105:0/4177671327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56486ab31f20 con 0x56486ab2e450 2024-03-26T17:38:27.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe543fff640 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab227f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.656+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab22d30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe54400fca0 tx=0x7fe54401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9868582b75f0b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe540ff9640 1 -- 172.21.15.105:0/4177671327 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54402c070 con 0x56486ab2e450 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe540ff9640 1 -- 172.21.15.105:0/4177671327 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe544019050 con 0x56486ab2e450 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe540ff9640 1 -- 172.21.15.105:0/4177671327 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544017040 con 0x56486ab2e450 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 -- 172.21.15.105:0/4177671327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 msgr2=0x56486ab22d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab22d30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe54400fca0 tx=0x7fe54401c000 comp rx=0 tx=0).stop 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 -- 172.21.15.105:0/4177671327 shutdown_connections 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486ab22d30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab292e0 0x56486ab227f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 --2- 172.21.15.105:0/4177671327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab288f0 0x56486ab318c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 -- 172.21.15.105:0/4177671327 >> 172.21.15.105:0/4177671327 conn(0x56486ab23e60 msgr2=0x56486ab2a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 -- 172.21.15.105:0/4177671327 shutdown_connections 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.657+0000 7fe5543bb740 1 -- 172.21.15.105:0/4177671327 wait complete. 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 1 -- start start 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 1 Processor -- start 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab288f0 0x56486ab9f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab292e0 0x56486ab9f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486aba67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1a9f70 con 0x56486ab2e450 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1a98a0 con 0x56486ab288f0 2024-03-26T17:38:27.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1a6ae0 con 0x56486ab292e0 2024-03-26T17:38:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5543bb740 10 client.0 _ll_register_callbacks cb 0x7fe544009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486aba67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe543fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab292e0 0x56486ab9f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486aba67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56062/0 (socket says 172.21.15.105:56062) 2024-03-26T17:38:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab288f0 0x56486ab9f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 -- 172.21.15.105:0/2137869685 learned_addr learned my addr 172.21.15.105:0/2137869685 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 -- 172.21.15.105:0/2137869685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab292e0 msgr2=0x56486ab9f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab292e0 0x56486ab9f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 -- 172.21.15.105:0/2137869685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab288f0 msgr2=0x56486ab9f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab288f0 0x56486ab9f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 -- 172.21.15.105:0/2137869685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56486aba6de0 con 0x56486ab2e450 2024-03-26T17:38:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe543fff640 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab292e0 0x56486ab9f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:27.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe54b7fe640 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486aba67f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe544000f80 tx=0x7fe544004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901a828bb8797e2e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.658+0000 7fe5327fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5327fc640 1 -- 172.21.15.105:0/2137869685 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54402c070 con 0x56486ab2e450 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 1 -- 172.21.15.105:0/2137869685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe544019050 con 0x56486ab2e450 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5327fc640 1 -- 172.21.15.105:0/2137869685 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8206+0+0 (secure 0 0 0) 0x7fe544009040 con 0x56486ab2e450 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5327fc640 1 -- 172.21.15.105:0/2137869685 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe544036030 con 0x56486ab2e450 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 1 -- 172.21.15.105:0/2137869685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56486aba80f0 con 0x56486ab2e450 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5327fc640 1 -- 172.21.15.105:0/2137869685 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe544019050 con 0x56486ab2e450 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5327fc640 1 -- 172.21.15.105:0/2137869685 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe544027c80 con 0x56486ab2e450 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:27.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 20 client.15492 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56486aba8a60 0x56486abaae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 4 client.15492 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe5543bb740 1 -- 172.21.15.105:0/2137869685 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56486aa95210 con 0x56486aba8a60 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.659+0000 7fe54affd640 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56486aba8a60 0x56486abaae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe54affd640 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56486aba8a60 0x56486abaae80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623cf442969e4fbc server_cookie=16980ba655e9ea75 in_seq=0 out_seq=0 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5327fc640 10 client.15492 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5327fc640 1 -- 172.21.15.105:0/2137869685 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56486aa95210 con 0x56486aba8a60 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5327fc640 10 client.15492 handle_command_reply: tid=0 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 client.15492 shutdown 2024-03-26T17:38:27.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 -- 172.21.15.105:0/2137869685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 msgr2=0x56486aba67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486aba67f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe544000f80 tx=0x7fe544004f30 comp rx=0 tx=0).stop 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 -- 172.21.15.105:0/2137869685 shutdown_connections 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56486aba8a60 0x56486abaae80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56486ab2e450 0x56486aba67f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56486ab292e0 0x56486ab9f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5327fc640 0 client.15492 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 --2- 172.21.15.105:0/2137869685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56486ab288f0 0x56486ab9f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 -- 172.21.15.105:0/2137869685 >> 172.21.15.105:0/2137869685 conn(0x56486ab23e60 msgr2=0x56486ab270c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 -- 172.21.15.105:0/2137869685 shutdown_connections 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 1 -- 172.21.15.105:0/2137869685 wait complete. 2024-03-26T17:38:27.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.660+0000 7fe5543bb740 20 client.15492 trim_cache size 0 max 16384 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "c02e800b-0148-4fd7-823f-0b84e91bc47c": { 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "c02e800b-0148-4fd7-823f-0b84e91bc47c", 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:38:27.672 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:27.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 -- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5380a2fb0 msgr2=0x7fe5380a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5380a2fb0 0x7fe5380a5470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe5440049d0 tx=0x7fe544011920 comp rx=0 tx=0).stop 2024-03-26T17:38:27.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 -- 172.21.15.105:0/3495527969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe54c102ac0 msgr2=0x7fe54c198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:27.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe54c102ac0 0x7fe54c198070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe53400e9f0 tx=0x7fe53400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 -- 172.21.15.105:0/3495527969 shutdown_connections 2024-03-26T17:38:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5380a2fb0 0x7fe5380a5470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe54c103480 0x7fe54c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe54c102ac0 0x7fe54c198070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 --2- 172.21.15.105:0/3495527969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe54c1018d0 0x7fe54c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 -- 172.21.15.105:0/3495527969 >> 172.21.15.105:0/3495527969 conn(0x7fe54c0fce60 msgr2=0x7fe54c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 -- 172.21.15.105:0/3495527969 shutdown_connections 2024-03-26T17:38:27.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:27.661+0000 7fe5517cd640 1 -- 172.21.15.105:0/3495527969 wait complete. 2024-03-26T17:38:27.676 INFO:tasks.cephfs.filesystem:scrub status for tag:c02e800b-0148-4fd7-823f-0b84e91bc47c - {'path': '/', 'tag': 'c02e800b-0148-4fd7-823f-0b84e91bc47c', 'options': 'recursive,force'} 2024-03-26T17:38:27.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:27 smithi105 ceph-mon[32280]: pgmap v347: 97 pgs: 97 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:38:27.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-26T17:38:27.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-03-26T17:38:27.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3f", "id": [1, 9]}]: dispatch 2024-03-26T17:38:27.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:27 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3308782327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:27.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:27 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3256750949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:28.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:27 smithi144 ceph-mon[36095]: pgmap v347: 97 pgs: 97 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:38:28.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-26T17:38:28.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-03-26T17:38:28.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3f", "id": [1, 9]}]: dispatch 2024-03-26T17:38:28.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:27 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3308782327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:28.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:27 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3256750949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:28.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:27 smithi135 ceph-mon[35636]: pgmap v347: 97 pgs: 97 active+clean; 651 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-26T17:38:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-03-26T17:38:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-03-26T17:38:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3f", "id": [1, 9]}]: dispatch 2024-03-26T17:38:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:27 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3308782327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:28.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:27 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3256750949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T17:38:29.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:28 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:38:29.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:28 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:38:29.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:28 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:38:29.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:28 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:38:29.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:28 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-26T17:38:29.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:28 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-03-26T17:38:29.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:28 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3f", "id": [1, 9]}]': finished 2024-03-26T17:38:29.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:28 smithi144 ceph-mon[36095]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T17:38:29.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:28 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:38:29.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:28 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:38:29.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:28 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:38:29.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:28 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:38:29.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-26T17:38:29.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-03-26T17:38:29.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:28 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3f", "id": [1, 9]}]': finished 2024-03-26T17:38:29.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:28 smithi105 ceph-mon[32280]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T17:38:29.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:28 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:38:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:28 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:38:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:28 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:38:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:28 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:38:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:28 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-03-26T17:38:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:28 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-03-26T17:38:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:28 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3f", "id": [1, 9]}]': finished 2024-03-26T17:38:29.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:28 smithi135 ceph-mon[35636]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T17:38:30.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:29 smithi144 ceph-mon[36095]: pgmap v349: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-03-26T17:38:30.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:29 smithi144 ceph-mon[36095]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T17:38:30.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:29 smithi105 ceph-mon[32280]: pgmap v349: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-03-26T17:38:30.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:29 smithi105 ceph-mon[32280]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T17:38:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:29 smithi135 ceph-mon[35636]: pgmap v349: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.4 KiB/s wr, 13 op/s 2024-03-26T17:38:30.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:29 smithi135 ceph-mon[35636]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T17:38:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:31 smithi144 ceph-mon[36095]: pgmap v351: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:32.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:31 smithi105 ceph-mon[32280]: pgmap v351: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:31 smithi135 ceph-mon[35636]: pgmap v351: 97 pgs: 97 active+clean; 653 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:33.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:32 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:38:33.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:32 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:38:33.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:32 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:38:34.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:33 smithi144 ceph-mon[36095]: pgmap v352: 97 pgs: 97 active+clean; 659 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:34.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:33 smithi105 ceph-mon[32280]: pgmap v352: 97 pgs: 97 active+clean; 659 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:34.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:33 smithi135 ceph-mon[35636]: pgmap v352: 97 pgs: 97 active+clean; 659 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:35 smithi144 ceph-mon[36095]: pgmap v353: 97 pgs: 97 active+clean; 659 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:35 smithi105 ceph-mon[32280]: pgmap v353: 97 pgs: 97 active+clean; 659 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:35 smithi135 ceph-mon[35636]: pgmap v353: 97 pgs: 97 active+clean; 659 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-26T17:38:38.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:37 smithi144 ceph-mon[36095]: pgmap v354: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 807 B/s wr, 9 op/s 2024-03-26T17:38:38.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:37 smithi105 ceph-mon[32280]: pgmap v354: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 807 B/s wr, 9 op/s 2024-03-26T17:38:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:37 smithi135 ceph-mon[35636]: pgmap v354: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 807 B/s wr, 9 op/s 2024-03-26T17:38:40.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:39 smithi144 ceph-mon[36095]: pgmap v355: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 818 B/s wr, 10 op/s 2024-03-26T17:38:40.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:39 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:38:40.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:39 smithi105 ceph-mon[32280]: pgmap v355: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 818 B/s wr, 10 op/s 2024-03-26T17:38:40.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:39 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:38:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:39 smithi135 ceph-mon[35636]: pgmap v355: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 818 B/s wr, 10 op/s 2024-03-26T17:38:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:39 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 -- 172.21.15.105:0/4257057103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a0103530 msgr2=0x7fe3a0109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/4257057103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a0103530 0x7fe3a0109dc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe398009cd0 tx=0x7fe39802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 -- 172.21.15.105:0/4257057103 shutdown_connections 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/4257057103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a0103530 0x7fe3a0109dc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/4257057103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0102b70 0x7fe3a0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/4257057103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a00ffba0 0x7fe3a00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 -- 172.21.15.105:0/4257057103 >> 172.21.15.105:0/4257057103 conn(0x7fe3a00fb510 msgr2=0x7fe3a00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 -- 172.21.15.105:0/4257057103 shutdown_connections 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 -- 172.21.15.105:0/4257057103 wait complete. 2024-03-26T17:38:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.472+0000 7fe3a52b2640 1 Processor -- start 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe3a52b2640 1 -- start start 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe3a52b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a00ffba0 0x7fe3a0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe3a52b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0102b70 0x7fe3a0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe3a52b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a0103530 0x7fe3a019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe3a52b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010d400 con 0x7fe3a00ffba0 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe3a52b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010d280 con 0x7fe3a0102b70 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe3a52b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010d580 con 0x7fe3a0103530 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a00ffba0 0x7fe3a0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a00ffba0 0x7fe3a0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43330/0 (socket says 172.21.15.105:43330) 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39effd640 1 -- 172.21.15.105:0/1295600957 learned_addr learned my addr 172.21.15.105:0/1295600957 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe3967fc640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0102b70 0x7fe3a0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a0103530 0x7fe3a019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39effd640 1 -- 172.21.15.105:0/1295600957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a0103530 msgr2=0x7fe3a019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39effd640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a0103530 0x7fe3a019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39effd640 1 -- 172.21.15.105:0/1295600957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0102b70 msgr2=0x7fe3a0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39effd640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0102b70 0x7fe3a0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.473+0000 7fe39effd640 1 -- 172.21.15.105:0/1295600957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a019f690 con 0x7fe3a00ffba0 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.474+0000 7fe39effd640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a00ffba0 0x7fe3a0197a70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d950 tx=0x7fe39000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.474+0000 7fe39cff9640 1 -- 172.21.15.105:0/1295600957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390017020 con 0x7fe3a00ffba0 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.474+0000 7fe39cff9640 1 -- 172.21.15.105:0/1295600957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe390009c30 con 0x7fe3a00ffba0 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.474+0000 7fe3a52b2640 1 -- 172.21.15.105:0/1295600957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a019f980 con 0x7fe3a00ffba0 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.474+0000 7fe39cff9640 1 -- 172.21.15.105:0/1295600957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39000f6b0 con 0x7fe3a00ffba0 2024-03-26T17:38:40.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.474+0000 7fe3a52b2640 1 -- 172.21.15.105:0/1295600957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a019fed0 con 0x7fe3a00ffba0 2024-03-26T17:38:40.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.476+0000 7fe39cff9640 1 -- 172.21.15.105:0/1295600957 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fe39001a020 con 0x7fe3a00ffba0 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.476+0000 7fe3a7f7b740 1 Processor -- start 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3a7f7b740 1 -- start start 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe39cff9640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe38c0a3060 0x7fe38c0a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049bb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0049200b0 con 0x55a0049c1130 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0048c8960 con 0x55a0049bbfc0 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a00491c060 con 0x55a0049bb5d0 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe38c0a3060 0x7fe38c0a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3967fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe39effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049bb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3967fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58846/0 (socket says 172.21.15.105:58846) 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3967fc640 1 -- 172.21.15.105:0/683948939 learned_addr learned my addr 172.21.15.105:0/683948939 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe39cff9640 1 -- 172.21.15.105:0/1295600957 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fe3900cf230 con 0x7fe3a00ffba0 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3967fc640 1 -- 172.21.15.105:0/683948939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 msgr2=0x55a0049bb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3967fc640 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049bb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.477+0000 7fe3967fc640 1 -- 172.21.15.105:0/683948939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 msgr2=0x55a0049c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.478+0000 7fe3967fc640 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.478+0000 7fe3967fc640 1 -- 172.21.15.105:0/683948939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0048dac60 con 0x55a0049bbfc0 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.478+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.478+0000 7fe3967fc640 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c0ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe388009c10 tx=0x7fe388011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68b37c889c9d8c99 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.478+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe38c0a3060 0x7fe38c0a5520 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe398009ca0 tx=0x7fe39802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.478+0000 7fe3957fa640 1 -- 172.21.15.105:0/683948939 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388004230 con 0x55a0049bbfc0 2024-03-26T17:38:40.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.478+0000 7fe3957fa640 1 -- 172.21.15.105:0/683948939 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3880043d0 con 0x55a0049bbfc0 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.478+0000 7fe3957fa640 1 -- 172.21.15.105:0/683948939 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388004de0 con 0x55a0049bbfc0 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/683948939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 msgr2=0x55a0049c0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c0ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe388009c10 tx=0x7fe388011810 comp rx=0 tx=0).stop 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/683948939 shutdown_connections 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c0ac0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/683948939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049bb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/683948939 >> 172.21.15.105:0/683948939 conn(0x55a0049b6b40 msgr2=0x55a0049b6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/683948939 shutdown_connections 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/683948939 wait complete. 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 -- start start 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.479+0000 7fe3a7f7b740 1 Processor -- start 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bb5d0 0x55a0049c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049bbfc0 0x55a0049c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049c1130 0x55a0049cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0049c40e0 con 0x55a0049bbfc0 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0048dbdb0 con 0x55a0049bb5d0 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0049c4260 con 0x55a0049c1130 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3a7f7b740 10 client.0 _ll_register_callbacks cb 0x55a0048e0110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3967fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049bbfc0 0x55a0049c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3967fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049bbfc0 0x55a0049c5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43354/0 (socket says 172.21.15.105:43354) 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3967fc640 1 -- 172.21.15.105:0/3570064453 learned_addr learned my addr 172.21.15.105:0/3570064453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe39effd640 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bb5d0 0x55a0049c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049c1130 0x55a0049cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe39effd640 1 -- 172.21.15.105:0/3570064453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049c1130 msgr2=0x55a0049cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe39effd640 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049c1130 0x55a0049cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe39effd640 1 -- 172.21.15.105:0/3570064453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049bbfc0 msgr2=0x55a0049c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe39effd640 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049bbfc0 0x55a0049c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe39effd640 1 -- 172.21.15.105:0/3570064453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a004634210 con 0x55a0049bb5d0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3967fc640 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049bbfc0 0x55a0049c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe39effd640 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bb5d0 0x55a0049c53f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ccb40 tx=0x7fe3900073b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4858dd31e2e1d743 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3867fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3867fc640 1 -- 172.21.15.105:0/3570064453 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390099d20 con 0x55a0049bb5d0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3867fc640 1 -- 172.21.15.105:0/3570064453 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fe3900860b0 con 0x55a0049bb5d0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.480+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/3570064453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3880043d0 con 0x55a0049bb5d0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3867fc640 1 -- 172.21.15.105:0/3570064453 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe3900a38e0 con 0x55a0049bb5d0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3a7f7b740 10 client.25370 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/3570064453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a004700620 con 0x55a0049bb5d0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3867fc640 1 -- 172.21.15.105:0/3570064453 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3880043d0 con 0x55a0049bb5d0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3867fc640 1 -- 172.21.15.105:0/3570064453 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe390024060 con 0x55a0049bb5d0 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3a7f7b740 10 client.25370 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3a7f7b740 10 client.25370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3a7f7b740 20 client.25370 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:40.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0049d11f0 0x55a0049d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3a7f7b740 4 client.25370 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/3570064453 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a00492d5f0 con 0x55a0049d11f0 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.481+0000 7fe3967fc640 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0049d11f0 0x55a0049d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.482+0000 7fe3867fc640 10 client.25370 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.482+0000 7fe3967fc640 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0049d11f0 0x55a0049d3680 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e165dc4c49a972 server_cookie=d429892915350af7 in_seq=0 out_seq=0 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.482+0000 7fe3867fc640 1 -- 172.21.15.105:0/3570064453 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a00492d5f0 con 0x55a0049d11f0 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3867fc640 10 client.25370 handle_command_reply: tid=0 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 client.25370 shutdown 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bb5d0 msgr2=0x55a0049c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bb5d0 0x55a0049c53f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe3900ccb40 tx=0x7fe3900073b0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/3570064453 shutdown_connections 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0049d11f0 0x55a0049d3680 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049c1130 0x55a0049cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.482 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049bbfc0 0x55a0049c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/3570064453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bb5d0 0x55a0049c53f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/3570064453 >> 172.21.15.105:0/3570064453 conn(0x55a0049b6b40 msgr2=0x55a0049b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3867fc640 0 client.25370 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/3570064453 shutdown_connections 2024-03-26T17:38:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/3570064453 wait complete. 2024-03-26T17:38:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.483+0000 7fe3a7f7b740 20 client.25370 trim_cache size 0 max 16384 2024-03-26T17:38:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.497+0000 7fe3a7f7b740 1 Processor -- start 2024-03-26T17:38:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.497+0000 7fe3a7f7b740 1 -- start start 2024-03-26T17:38:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049bca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01a8d00 con 0x55a0049c1130 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01aa840 con 0x55a0049bbfc0 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01a93d0 con 0x55a0049bb5d0 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049bca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43366/0 (socket says 172.21.15.105:43366) 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 -- 172.21.15.105:0/4197131200 learned_addr learned my addr 172.21.15.105:0/4197131200 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe3967fc640 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39effd640 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 -- 172.21.15.105:0/4197131200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 msgr2=0x55a0049d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 -- 172.21.15.105:0/4197131200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 msgr2=0x55a0049c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 -- 172.21.15.105:0/4197131200 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0049bc3a0 con 0x55a0049c1130 2024-03-26T17:38:40.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.498+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049bca30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe39802c700 tx=0x7fe3980324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8cd740c6ded98e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe394ff9640 1 -- 172.21.15.105:0/4197131200 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398052070 con 0x55a0049c1130 2024-03-26T17:38:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe394ff9640 1 -- 172.21.15.105:0/4197131200 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe398003df0 con 0x55a0049c1130 2024-03-26T17:38:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe394ff9640 1 -- 172.21.15.105:0/4197131200 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398039040 con 0x55a0049c1130 2024-03-26T17:38:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/4197131200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 msgr2=0x55a0049bca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049bca30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe39802c700 tx=0x7fe3980324f0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/4197131200 shutdown_connections 2024-03-26T17:38:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049bca30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/4197131200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/4197131200 >> 172.21.15.105:0/4197131200 conn(0x55a0049b6b40 msgr2=0x55a0049c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:40.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.499+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/4197131200 shutdown_connections 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/4197131200 wait complete. 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 -- start start 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 Processor -- start 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01aa170 con 0x55a0049c1130 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01a9aa0 con 0x55a0049bbfc0 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a01a6ce0 con 0x55a0049bb5d0 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3a7f7b740 10 client.0 _ll_register_callbacks cb 0x7fe390097370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43380/0 (socket says 172.21.15.105:43380) 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39f7fe640 1 -- 172.21.15.105:0/771307198 learned_addr learned my addr 172.21.15.105:0/771307198 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39effd640 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe3967fc640 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39f7fe640 1 -- 172.21.15.105:0/771307198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 msgr2=0x55a0049c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39f7fe640 1 -- 172.21.15.105:0/771307198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 msgr2=0x55a0049c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.500+0000 7fe39f7fe640 1 -- 172.21.15.105:0/771307198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0049cb570 con 0x55a0049c1130 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe39f7fe640 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049caf80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe39802c130 tx=0x7fe398035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3629f3da9fea0784 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe3867fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe3867fc640 1 -- 172.21.15.105:0/771307198 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39805d070 con 0x55a0049c1130 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe3867fc640 1 -- 172.21.15.105:0/771307198 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fe3980041b0 con 0x55a0049c1130 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/771307198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe398003df0 con 0x55a0049c1130 2024-03-26T17:38:40.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe3867fc640 1 -- 172.21.15.105:0/771307198 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe398052040 con 0x55a0049c1130 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe3a7f7b740 10 client.15510 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/771307198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0049cc8f0 con 0x55a0049c1130 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.501+0000 7fe3867fc640 1 -- 172.21.15.105:0/771307198 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe398003df0 con 0x55a0049c1130 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe3867fc640 1 -- 172.21.15.105:0/771307198 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe39803f070 con 0x55a0049c1130 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe3a7f7b740 10 client.15510 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe3a7f7b740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe3a7f7b740 20 client.15510 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a004a37d00 0x55a004a3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe3a7f7b740 4 client.15510 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/771307198 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a004927940 con 0x55a004a37d00 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe39effd640 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a004a37d00 0x55a004a3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe39effd640 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a004a37d00 0x55a004a3a120 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15f989e90bef2a1 server_cookie=2f91c8c1d9e0022c in_seq=0 out_seq=0 2024-03-26T17:38:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.502+0000 7fe3867fc640 10 client.15510 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3867fc640 1 -- 172.21.15.105:0/771307198 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a004927940 con 0x55a004a37d00 2024-03-26T17:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3867fc640 10 client.15510 handle_command_reply: tid=0 2024-03-26T17:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 client.15510 shutdown 2024-03-26T17:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/771307198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 msgr2=0x55a0049caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049caf80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe39802c130 tx=0x7fe398035d10 comp rx=0 tx=0).stop 2024-03-26T17:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/771307198 shutdown_connections 2024-03-26T17:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a004a37d00 0x55a004a3a120 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0049c1130 0x55a0049caf80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0049bbfc0 0x55a0049c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 --2- 172.21.15.105:0/771307198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0049bb5d0 0x55a0049c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/771307198 >> 172.21.15.105:0/771307198 conn(0x55a0049b6b40 msgr2=0x55a0049b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3867fc640 0 client.15510 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/771307198 shutdown_connections 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 1 -- 172.21.15.105:0/771307198 wait complete. 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.513+0000 7fe3a7f7b740 20 client.15510 trim_cache size 0 max 16384 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:38:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:38:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:38:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 -- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe38c0a3060 msgr2=0x7fe38c0a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe38c0a3060 0x7fe38c0a5520 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe398009ca0 tx=0x7fe39802e040 comp rx=0 tx=0).stop 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 -- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a00ffba0 msgr2=0x7fe3a0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a00ffba0 0x7fe3a0197a70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d950 tx=0x7fe39000de10 comp rx=0 tx=0).stop 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 -- 172.21.15.105:0/1295600957 shutdown_connections 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe38c0a3060 0x7fe38c0a5520 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3a0103530 0x7fe3a019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3a0102b70 0x7fe3a0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 --2- 172.21.15.105:0/1295600957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3a00ffba0 0x7fe3a0197a70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 -- 172.21.15.105:0/1295600957 >> 172.21.15.105:0/1295600957 conn(0x7fe3a00fb510 msgr2=0x7fe3a0105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 -- 172.21.15.105:0/1295600957 shutdown_connections 2024-03-26T17:38:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:40.514+0000 7fe3a52b2640 1 -- 172.21.15.105:0/1295600957 wait complete. 2024-03-26T17:38:41.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:41 smithi105 ceph-mon[32280]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 690 B/s wr, 9 op/s 2024-03-26T17:38:41.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:41 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:41.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:41 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:41.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:41 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:41.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:41 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:41.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:41 smithi135 ceph-mon[35636]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 690 B/s wr, 9 op/s 2024-03-26T17:38:41.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:41 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:41.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:41 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:41.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:41 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:41.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:41 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:41 smithi144 ceph-mon[36095]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 690 B/s wr, 9 op/s 2024-03-26T17:38:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:41 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:41 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:41 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:42.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:41 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:43.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:43 smithi105 ceph-mon[32280]: pgmap v357: 97 pgs: 97 active+clean; 665 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-26T17:38:43.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:43 smithi135 ceph-mon[35636]: pgmap v357: 97 pgs: 97 active+clean; 665 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-26T17:38:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:43 smithi144 ceph-mon[36095]: pgmap v357: 97 pgs: 97 active+clean; 665 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-03-26T17:38:44.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:44.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:44.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:38:44.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:38:44.893 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:44.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:44.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:44.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:38:44.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:38:44.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:45.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:45.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:45.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:38:45.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:38:45.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:38:45.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:45 smithi105 ceph-mon[32280]: pgmap v358: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T17:38:45.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:45 smithi135 ceph-mon[35636]: pgmap v358: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T17:38:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:45 smithi144 ceph-mon[36095]: pgmap v358: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-26T17:38:47.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:47 smithi105 ceph-mon[32280]: pgmap v359: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-26T17:38:47.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:47 smithi135 ceph-mon[35636]: pgmap v359: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-26T17:38:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:47 smithi144 ceph-mon[36095]: pgmap v359: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-03-26T17:38:49.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:49 smithi105 ceph-mon[32280]: pgmap v360: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-03-26T17:38:49.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:49 smithi135 ceph-mon[35636]: pgmap v360: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-03-26T17:38:50.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:49 smithi144 ceph-mon[36095]: pgmap v360: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-03-26T17:38:51.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:51 smithi105 ceph-mon[32280]: pgmap v361: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-03-26T17:38:51.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:51 smithi135 ceph-mon[35636]: pgmap v361: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-03-26T17:38:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:51 smithi144 ceph-mon[36095]: pgmap v361: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-03-26T17:38:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 -- 172.21.15.105:0/2907165117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7714074320 msgr2=0x7f77140747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 --2- 172.21.15.105:0/2907165117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7714074320 0x7f77140747a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7704009930 tx=0x7f770402bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 -- 172.21.15.105:0/2907165117 shutdown_connections 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 --2- 172.21.15.105:0/2907165117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f771410dee0 0x7f77141102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 --2- 172.21.15.105:0/2907165117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7714074320 0x7f77140747a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 --2- 172.21.15.105:0/2907165117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7714075900 0x7f7714073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 -- 172.21.15.105:0/2907165117 >> 172.21.15.105:0/2907165117 conn(0x7f77140fce80 msgr2=0x7f77140ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 -- 172.21.15.105:0/2907165117 shutdown_connections 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.642+0000 7f771ae8c640 1 -- 172.21.15.105:0/2907165117 wait complete. 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.643+0000 7f771ae8c640 1 Processor -- start 2024-03-26T17:38:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.643+0000 7f771ae8c640 1 -- start start 2024-03-26T17:38:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.643+0000 7f771ae8c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714075900 0x7f771419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.643+0000 7f771ae8c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f771410dee0 0x7f771419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.643+0000 7f771ae8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77141a3880 0x7f77141a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.643+0000 7f771ae8c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7714113a80 con 0x7f77141a3880 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f771ae8c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7714113900 con 0x7f7714075900 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f771ae8c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7714113c00 con 0x7f771410dee0 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7718c28640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714075900 0x7f771419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7718c28640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714075900 0x7f771419e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:47262/0 (socket says 172.21.15.105:47262) 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7718c28640 1 -- 172.21.15.105:0/1370374829 learned_addr learned my addr 172.21.15.105:0/1370374829 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7719429640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77141a3880 0x7f77141a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7713fff640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f771410dee0 0x7f771419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7719429640 1 -- 172.21.15.105:0/1370374829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f771410dee0 msgr2=0x7f771419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7719429640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f771410dee0 0x7f771419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7719429640 1 -- 172.21.15.105:0/1370374829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714075900 msgr2=0x7f771419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7719429640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714075900 0x7f771419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7719429640 1 -- 172.21.15.105:0/1370374829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77141a62e0 con 0x7f77141a3880 2024-03-26T17:38:53.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7713fff640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f771410dee0 0x7f771419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7719429640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77141a3880 0x7f77141a5c70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f770c009fd0 tx=0x7f770c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.644+0000 7f7711ffb640 1 -- 172.21.15.105:0/1370374829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c00ce20 con 0x7f77141a3880 2024-03-26T17:38:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.645+0000 7f771ae8c640 1 -- 172.21.15.105:0/1370374829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77141a65d0 con 0x7f77141a3880 2024-03-26T17:38:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.645+0000 7f7711ffb640 1 -- 172.21.15.105:0/1370374829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f770c005020 con 0x7f77141a3880 2024-03-26T17:38:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.645+0000 7f7711ffb640 1 -- 172.21.15.105:0/1370374829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c010690 con 0x7f77141a3880 2024-03-26T17:38:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.645+0000 7f771ae8c640 1 -- 172.21.15.105:0/1370374829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77141a6af0 con 0x7f77141a3880 2024-03-26T17:38:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.648+0000 7f771da7e740 1 Processor -- start 2024-03-26T17:38:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.648+0000 7f7711ffb640 1 -- 172.21.15.105:0/1370374829 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f770c0051d0 con 0x7f77141a3880 2024-03-26T17:38:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.648+0000 7f771da7e740 1 -- start start 2024-03-26T17:38:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.648+0000 7f771da7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7096325d0 0x55c7096329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.648+0000 7f771da7e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c709637ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.648+0000 7f771da7e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c709638130 0x55c70963a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f771da7e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c7095970b0 con 0x55c709638130 2024-03-26T17:38:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f771da7e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c70953f960 con 0x55c709632fc0 2024-03-26T17:38:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f771da7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c709593060 con 0x55c7096325d0 2024-03-26T17:38:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f7713fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c709638130 0x55c70963a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f7713fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c709638130 0x55c70963a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37398/0 (socket says 172.21.15.105:37398) 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f7713fff640 1 -- 172.21.15.105:0/1715033157 learned_addr learned my addr 172.21.15.105:0/1715033157 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f7711ffb640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76fc0a3160 0x7f76fc0a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f7711ffb640 1 -- 172.21.15.105:0/1370374829 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f770c0cf8c0 con 0x7f77141a3880 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.649+0000 7f7718c28640 1 --2- 172.21.15.105:0/1715033157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c709637ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f7713fff640 1 -- 172.21.15.105:0/1715033157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7096325d0 msgr2=0x55c7096329f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f7713fff640 1 --2- 172.21.15.105:0/1715033157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7096325d0 0x55c7096329f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f7713fff640 1 -- 172.21.15.105:0/1715033157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 msgr2=0x55c709637ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f7713fff640 1 --2- 172.21.15.105:0/1715033157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c709637ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f7713fff640 1 -- 172.21.15.105:0/1715033157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c709551c60 con 0x55c709638130 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f7719429640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76fc0a3160 0x7f76fc0a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f7713fff640 1 --2- 172.21.15.105:0/1715033157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c709638130 0x55c70963a5c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7704036040 tx=0x7f77040095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd155d8c370f1152 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f76faffd640 1 -- 172.21.15.105:0/1715033157 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704045030 con 0x55c709638130 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f76faffd640 1 -- 172.21.15.105:0/1715033157 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f770402cc90 con 0x55c709638130 2024-03-26T17:38:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.650+0000 7f76faffd640 1 -- 172.21.15.105:0/1715033157 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040046c0 con 0x55c709638130 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 -- 172.21.15.105:0/1715033157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c709638130 msgr2=0x55c70963a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 --2- 172.21.15.105:0/1715033157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c709638130 0x55c70963a5c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7704036040 tx=0x7f77040095f0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f7719429640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76fc0a3160 0x7f76fc0a5620 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f770c01a040 tx=0x7f770c00c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 -- 172.21.15.105:0/1715033157 shutdown_connections 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 --2- 172.21.15.105:0/1715033157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c709638130 0x55c70963a5c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 --2- 172.21.15.105:0/1715033157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c709637ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 --2- 172.21.15.105:0/1715033157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7096325d0 0x55c7096329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 -- 172.21.15.105:0/1715033157 >> 172.21.15.105:0/1715033157 conn(0x55c70962db40 msgr2=0x55c70962df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 -- 172.21.15.105:0/1715033157 shutdown_connections 2024-03-26T17:38:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.651+0000 7f771da7e740 1 -- 172.21.15.105:0/1715033157 wait complete. 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.652+0000 7f771da7e740 1 -- start start 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.652+0000 7f771da7e740 1 Processor -- start 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.652+0000 7f771da7e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37410/0 (socket says 172.21.15.105:37410) 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 -- 172.21.15.105:0/4039229388 learned_addr learned my addr 172.21.15.105:0/4039229388 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f771da7e740 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c70964da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f771da7e740 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 0x55c709654e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c70964c090 con 0x55c7096325d0 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c709552db0 con 0x55c709632fc0 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c70964c210 con 0x55c709652a90 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7718c28640 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c70964da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 -- 172.21.15.105:0/4039229388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 msgr2=0x55c709654e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 0x55c709654e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 -- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 msgr2=0x55c70964da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c70964da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 -- 172.21.15.105:0/4039229388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7092ab210 con 0x55c7096325d0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f7713fff640 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f770402c240 tx=0x7f7704003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ac3b483c1fa3ba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.653+0000 7f76f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 10 client.0 _ll_register_callbacks cb 0x55c709557110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f76f3fff640 1 -- 172.21.15.105:0/4039229388 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770404d030 con 0x55c7096325d0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770402cc90 con 0x55c7096325d0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f76f3fff640 1 -- 172.21.15.105:0/4039229388 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f7704003a10 con 0x55c7096325d0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f76f3fff640 1 -- 172.21.15.105:0/4039229388 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7704045070 con 0x55c7096325d0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55c709377620 con 0x55c7096325d0 2024-03-26T17:38:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f76f3fff640 1 -- 172.21.15.105:0/4039229388 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7704003c10 con 0x55c7096325d0 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 20 client.15522 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c709659740 0x55c70965bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 4 client.15522 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.654+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f770402cc90 con 0x55c709659740 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.655+0000 7f7718c28640 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c709659740 0x55c70965bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.655+0000 7f76f3fff640 10 client.15522 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.655+0000 7f7718c28640 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c709659740 0x55c70965bbd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20512ac460594d1a server_cookie=4cccca847ed5ad2d in_seq=0 out_seq=0 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f76f3fff640 1 -- 172.21.15.105:0/4039229388 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f770402cc90 con 0x55c709659740 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f76f3fff640 10 client.15522 handle_command_reply: tid=0 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 client.15522 shutdown 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 msgr2=0x55c70964d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d520 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f770402c240 tx=0x7f7704003820 comp rx=0 tx=0).stop 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 shutdown_connections 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c709659740 0x55c70965bbd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 0x55c709654e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c70964da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 --2- 172.21.15.105:0/4039229388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d520 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 >> 172.21.15.105:0/4039229388 conn(0x55c70962db40 msgr2=0x55c70962cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f76f3fff640 0 client.15522 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 shutdown_connections 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 1 -- 172.21.15.105:0/4039229388 wait complete. 2024-03-26T17:38:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.656+0000 7f771da7e740 20 client.15522 trim_cache size 0 max 16384 2024-03-26T17:38:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f771da7e740 1 Processor -- start 2024-03-26T17:38:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f771da7e740 1 -- start start 2024-03-26T17:38:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f771da7e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f771da7e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c70964d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f771da7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 0x55c70962c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f771da7e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77141057a0 con 0x55c7096325d0 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f771da7e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77141072e0 con 0x55c709632fc0 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f771da7e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7714105e70 con 0x55c709652a90 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7713fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7713fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37418/0 (socket says 172.21.15.105:37418) 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7713fff640 1 -- 172.21.15.105:0/106733653 learned_addr learned my addr 172.21.15.105:0/106733653 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7713fff640 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 0x55c70962c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7718c28640 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c70964d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7713fff640 1 -- 172.21.15.105:0/106733653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 msgr2=0x55c70962c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7713fff640 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 0x55c70962c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7713fff640 1 -- 172.21.15.105:0/106733653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 msgr2=0x55c70964d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.685+0000 7f7713fff640 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c70964d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.686+0000 7f7713fff640 1 -- 172.21.15.105:0/106733653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70962e8b0 con 0x55c7096325d0 2024-03-26T17:38:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.686+0000 7f7713fff640 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d130 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7704006010 tx=0x7f7704046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2a4f7014e2ee2f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.686+0000 7f76fa7fc640 1 -- 172.21.15.105:0/106733653 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770404d070 con 0x55c7096325d0 2024-03-26T17:38:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.686+0000 7f76fa7fc640 1 -- 172.21.15.105:0/106733653 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77040054e0 con 0x55c7096325d0 2024-03-26T17:38:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.686+0000 7f76fa7fc640 1 -- 172.21.15.105:0/106733653 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704041610 con 0x55c7096325d0 2024-03-26T17:38:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 -- 172.21.15.105:0/106733653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 msgr2=0x55c70964d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d130 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7704006010 tx=0x7f7704046000 comp rx=0 tx=0).stop 2024-03-26T17:38:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 -- 172.21.15.105:0/106733653 shutdown_connections 2024-03-26T17:38:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709652a90 0x55c70962c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709632fc0 0x55c70964d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 --2- 172.21.15.105:0/106733653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c70964d130 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 -- 172.21.15.105:0/106733653 >> 172.21.15.105:0/106733653 conn(0x55c70962db40 msgr2=0x55c709630da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 -- 172.21.15.105:0/106733653 shutdown_connections 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 -- 172.21.15.105:0/106733653 wait complete. 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.687+0000 7f771da7e740 1 -- start start 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f771da7e740 1 Processor -- start 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f771da7e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c709669ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f771da7e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709632fc0 0x55c709669ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f7713fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c709669ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f771da7e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709652a90 0x55c7096bd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f771da7e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7714106c10 con 0x55c7096325d0 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f7713fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c709669ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37426/0 (socket says 172.21.15.105:37426) 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f7713fff640 1 -- 172.21.15.105:0/1472407918 learned_addr learned my addr 172.21.15.105:0/1472407918 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f771da7e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7714106540 con 0x55c709652a90 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7714071b90 con 0x55c709632fc0 2024-03-26T17:38:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f7718c28640 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709632fc0 0x55c709669ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f771da7e740 10 client.0 _ll_register_callbacks cb 0x7f7704009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f7713fff640 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709652a90 0x55c7096bd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f7713fff640 1 -- 172.21.15.105:0/1472407918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709632fc0 msgr2=0x55c709669ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f7713fff640 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709632fc0 0x55c709669ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.688+0000 7f7713fff640 1 -- 172.21.15.105:0/1472407918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709652a90 msgr2=0x55c7096bd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f7713fff640 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709652a90 0x55c7096bd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f7713fff640 1 -- 172.21.15.105:0/1472407918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70966ce20 con 0x55c7096325d0 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f7718c28640 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709632fc0 0x55c709669ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f7713fff640 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c709669ab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7704005910 tx=0x7f770402cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cad10a099ae1b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f76f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f76f3fff640 1 -- 172.21.15.105:0/1472407918 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770404d070 con 0x55c7096325d0 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f76f3fff640 1 -- 172.21.15.105:0/1472407918 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f77040051f0 con 0x55c7096325d0 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77040054e0 con 0x55c7096325d0 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f76f3fff640 1 -- 172.21.15.105:0/1472407918 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77040415c0 con 0x55c7096325d0 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f771da7e740 10 client.15528 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.689+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c7096be8b0 con 0x55c7096325d0 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f76f3fff640 1 -- 172.21.15.105:0/1472407918 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77040348f0 con 0x55c7096325d0 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f76f3fff640 1 -- 172.21.15.105:0/1472407918 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f770403f6b0 con 0x55c7096325d0 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f771da7e740 10 client.15528 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f771da7e740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f771da7e740 20 client.15528 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f771da7e740 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c7096bf060 0x55c7096c1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f771da7e740 4 client.15528 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:38:53.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f77040054e0 con 0x55c7096bf060 2024-03-26T17:38:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f7718c28640 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c7096bf060 0x55c7096c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f76f3fff640 10 client.15528 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:53.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.690+0000 7f7718c28640 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c7096bf060 0x55c7096c1450 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3186a84cb90c81b6 server_cookie=e1d7565f79758fe6 in_seq=0 out_seq=0 2024-03-26T17:38:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.693+0000 7f76f3fff640 1 -- 172.21.15.105:0/1472407918 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f77040054e0 con 0x55c7096bf060 2024-03-26T17:38:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.693+0000 7f76f3fff640 10 client.15528 handle_command_reply: tid=0 2024-03-26T17:38:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.693+0000 7f771da7e740 1 client.15528 shutdown 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 msgr2=0x55c709669ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c709669ab0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7704005910 tx=0x7f770402cc90 comp rx=0 tx=0).stop 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 shutdown_connections 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c7096bf060 0x55c7096c1450 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c709652a90 0x55c7096bd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c709632fc0 0x55c709669ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 --2- 172.21.15.105:0/1472407918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7096325d0 0x55c709669ab0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 >> 172.21.15.105:0/1472407918 conn(0x55c70962db40 msgr2=0x55c70964d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f76f3fff640 0 client.15528 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 shutdown_connections 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 1 -- 172.21.15.105:0/1472407918 wait complete. 2024-03-26T17:38:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.694+0000 7f771da7e740 20 client.15528 trim_cache size 0 max 16384 2024-03-26T17:38:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:38:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:38:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:38:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:38:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 -- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76fc0a3160 msgr2=0x7f76fc0a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76fc0a3160 0x7f76fc0a5620 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f770c01a040 tx=0x7f770c00c890 comp rx=0 tx=0).stop 2024-03-26T17:38:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 -- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77141a3880 msgr2=0x7f77141a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77141a3880 0x7f77141a5c70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f770c009fd0 tx=0x7f770c00ef10 comp rx=0 tx=0).stop 2024-03-26T17:38:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 -- 172.21.15.105:0/1370374829 shutdown_connections 2024-03-26T17:38:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76fc0a3160 0x7f76fc0a5620 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77141a3880 0x7f77141a5c70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f771410dee0 0x7f771419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 --2- 172.21.15.105:0/1370374829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7714075900 0x7f771419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 -- 172.21.15.105:0/1370374829 >> 172.21.15.105:0/1370374829 conn(0x7f77140fce80 msgr2=0x7f7714101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 -- 172.21.15.105:0/1370374829 shutdown_connections 2024-03-26T17:38:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:38:53.695+0000 7f771ae8c640 1 -- 172.21.15.105:0/1370374829 wait complete. 2024-03-26T17:38:53.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:53 smithi105 ceph-mon[32280]: pgmap v362: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-03-26T17:38:53.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:53 smithi135 ceph-mon[35636]: pgmap v362: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-03-26T17:38:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:53 smithi144 ceph-mon[36095]: pgmap v362: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-03-26T17:38:55.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:55 smithi105 ceph-mon[32280]: pgmap v363: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-26T17:38:55.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:55 smithi135 ceph-mon[35636]: pgmap v363: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-26T17:38:56.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:55 smithi144 ceph-mon[36095]: pgmap v363: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-26T17:38:57.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:38:57.678 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:38:57.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 -- 172.21.15.105:0/284839288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101b30 msgr2=0x7fe7201083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 --2- 172.21.15.105:0/284839288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101b30 0x7fe7201083c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe718009ca0 tx=0x7fe71802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 -- 172.21.15.105:0/284839288 shutdown_connections 2024-03-26T17:38:57.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 --2- 172.21.15.105:0/284839288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101b30 0x7fe7201083c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 --2- 172.21.15.105:0/284839288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe720101170 0x7fe7201015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 --2- 172.21.15.105:0/284839288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7200fff80 0x7fe7201003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 -- 172.21.15.105:0/284839288 >> 172.21.15.105:0/284839288 conn(0x7fe7200fb4f0 msgr2=0x7fe7200fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:57.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 -- 172.21.15.105:0/284839288 shutdown_connections 2024-03-26T17:38:57.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.799+0000 7fe727397640 1 -- 172.21.15.105:0/284839288 wait complete. 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe727397640 1 Processor -- start 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe727397640 1 -- start start 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe727397640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7200fff80 0x7fe720197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe727397640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101170 0x7fe720197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe727397640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe720101b30 0x7fe72019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe727397640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe72010d4a0 con 0x7fe7200fff80 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe727397640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe72010d320 con 0x7fe720101b30 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe727397640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe72010d620 con 0x7fe720101170 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe725133640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7200fff80 0x7fe720197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe725133640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7200fff80 0x7fe720197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51998/0 (socket says 172.21.15.105:51998) 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe725133640 1 -- 172.21.15.105:0/1013254121 learned_addr learned my addr 172.21.15.105:0/1013254121 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe725934640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe720101b30 0x7fe72019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.800+0000 7fe724932640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101170 0x7fe720197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.801+0000 7fe724932640 1 -- 172.21.15.105:0/1013254121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe720101b30 msgr2=0x7fe72019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.801+0000 7fe724932640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe720101b30 0x7fe72019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.801+0000 7fe724932640 1 -- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7200fff80 msgr2=0x7fe720197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.801+0000 7fe724932640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7200fff80 0x7fe720197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.801+0000 7fe724932640 1 -- 172.21.15.105:0/1013254121 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72006b410 con 0x7fe720101170 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.801+0000 7fe725133640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7200fff80 0x7fe720197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:57.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.802+0000 7fe724932640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101170 0x7fe720197f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9f0 tx=0x7fe71000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.802+0000 7fe7167fc640 1 -- 172.21.15.105:0/1013254121 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71000ccc0 con 0x7fe720101170 2024-03-26T17:38:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.802+0000 7fe7167fc640 1 -- 172.21.15.105:0/1013254121 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe710005020 con 0x7fe720101170 2024-03-26T17:38:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.802+0000 7fe727397640 1 -- 172.21.15.105:0/1013254121 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72006b700 con 0x7fe720101170 2024-03-26T17:38:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.802+0000 7fe7167fc640 1 -- 172.21.15.105:0/1013254121 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710004a00 con 0x7fe720101170 2024-03-26T17:38:57.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.802+0000 7fe727397640 1 -- 172.21.15.105:0/1013254121 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72006bc50 con 0x7fe720101170 2024-03-26T17:38:57.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.806+0000 7fe7167fc640 1 -- 172.21.15.105:0/1013254121 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fe710004ba0 con 0x7fe720101170 2024-03-26T17:38:57.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.806+0000 7fe729f83740 1 Processor -- start 2024-03-26T17:38:57.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.806+0000 7fe729f83740 1 -- start start 2024-03-26T17:38:57.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.806+0000 7fe729f83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7448f0 0x56295d744d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.807+0000 7fe7167fc640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe70c0a30e0 0x7fe70c0a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.807+0000 7fe7167fc640 1 -- 172.21.15.105:0/1013254121 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fe7100cfac0 con 0x7fe720101170 2024-03-26T17:38:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.807+0000 7fe725133640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe70c0a30e0 0x7fe70c0a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.807+0000 7fe725934640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7448f0 0x56295d744d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.807+0000 7fe725934640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7448f0 0x56295d744d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52004/0 (socket says 172.21.15.105:52004) 2024-03-26T17:38:57.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.807+0000 7fe725934640 1 -- 172.21.15.105:0/3216482961 learned_addr learned my addr 172.21.15.105:0/3216482961 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.808+0000 7fe725133640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe70c0a30e0 0x7fe70c0a55a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0058c0 tx=0x7fe71c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.808+0000 7fe729f83740 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7452e0 0x56295d749de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.808+0000 7fe729f83740 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 0x56295d74c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.808+0000 7fe729f83740 1 -- 172.21.15.105:0/3216482961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56295d6a9980 con 0x56295d7448f0 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.808+0000 7fe729f83740 1 -- 172.21.15.105:0/3216482961 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56295d651be0 con 0x56295d74a450 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.808+0000 7fe724932640 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7452e0 0x56295d749de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.808+0000 7fe729f83740 1 -- 172.21.15.105:0/3216482961 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56295d6a5930 con 0x56295d7452e0 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe724932640 1 -- 172.21.15.105:0/3216482961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 msgr2=0x56295d74c8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe724932640 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 0x56295d74c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe724932640 1 -- 172.21.15.105:0/3216482961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7448f0 msgr2=0x56295d744d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe724932640 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7448f0 0x56295d744d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe724932640 1 -- 172.21.15.105:0/3216482961 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295d4877d0 con 0x56295d7452e0 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe725934640 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7448f0 0x56295d744d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:57.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe724932640 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7452e0 0x56295d749de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe7100cd3d0 tx=0x7fe71009bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73238a347801126 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:57.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe70b7fe640 1 -- 172.21.15.105:0/3216482961 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710005560 con 0x56295d7452e0 2024-03-26T17:38:57.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe70b7fe640 1 -- 172.21.15.105:0/3216482961 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe71009a2f0 con 0x56295d7452e0 2024-03-26T17:38:57.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.809+0000 7fe70b7fe640 1 -- 172.21.15.105:0/3216482961 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7100cd620 con 0x56295d7452e0 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.812+0000 7fe729f83740 1 -- 172.21.15.105:0/3216482961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7452e0 msgr2=0x56295d749de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.812+0000 7fe729f83740 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7452e0 0x56295d749de0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe7100cd3d0 tx=0x7fe71009bff0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.813+0000 7fe729f83740 1 -- 172.21.15.105:0/3216482961 shutdown_connections 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.813+0000 7fe729f83740 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 0x56295d74c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.813+0000 7fe729f83740 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7452e0 0x56295d749de0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.813+0000 7fe729f83740 1 --2- 172.21.15.105:0/3216482961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7448f0 0x56295d744d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.813+0000 7fe729f83740 1 -- 172.21.15.105:0/3216482961 >> 172.21.15.105:0/3216482961 conn(0x56295d73fe60 msgr2=0x56295d740260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.813+0000 7fe729f83740 1 -- 172.21.15.105:0/3216482961 shutdown_connections 2024-03-26T17:38:57.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.813+0000 7fe729f83740 1 -- 172.21.15.105:0/3216482961 wait complete. 2024-03-26T17:38:57.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.814+0000 7fe729f83740 1 -- start start 2024-03-26T17:38:57.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.814+0000 7fe729f83740 1 Processor -- start 2024-03-26T17:38:57.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe729f83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe729f83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d74f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe725133640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe724932640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d74f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe725133640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55254/0 (socket says 172.21.15.105:55254) 2024-03-26T17:38:57.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe724932640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d74f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52010/0 (socket says 172.21.15.105:52010) 2024-03-26T17:38:57.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe725133640 1 -- 172.21.15.105:0/3800146159 learned_addr learned my addr 172.21.15.105:0/3800146159 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:57.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe729f83740 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 0x56295d756170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.815+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56295d74d3a0 con 0x56295d7452e0 2024-03-26T17:38:57.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56295d665160 con 0x56295d74a450 2024-03-26T17:38:57.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56295d74d520 con 0x56295d7448f0 2024-03-26T17:38:57.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe725934640 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 0x56295d756170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe729f83740 10 client.0 _ll_register_callbacks cb 0x56295d6a1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:57.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe725934640 1 -- 172.21.15.105:0/3800146159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 msgr2=0x56295d74ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:57.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe725934640 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe725934640 1 -- 172.21.15.105:0/3800146159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 msgr2=0x56295d74f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:57.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe725934640 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d74f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe725934640 1 -- 172.21.15.105:0/3800146159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295d3bd3f0 con 0x56295d74a450 2024-03-26T17:38:57.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe725934640 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 0x56295d756170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe718032420 tx=0x7fe718032bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31600af66b97aa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.816+0000 7fe708ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.817+0000 7fe708ff9640 1 -- 172.21.15.105:0/3800146159 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71804b060 con 0x56295d74a450 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.817+0000 7fe708ff9640 1 -- 172.21.15.105:0/3800146159 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fe718003b70 con 0x56295d74a450 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.817+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe71009a2f0 con 0x56295d74a450 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.817+0000 7fe708ff9640 1 -- 172.21.15.105:0/3800146159 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe718039040 con 0x56295d74a450 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.817+0000 7fe729f83740 10 client.25397 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.817+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56295d37d0f0 con 0x56295d74a450 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.817+0000 7fe708ff9640 1 -- 172.21.15.105:0/3800146159 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe718003d70 con 0x56295d74a450 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.818+0000 7fe708ff9640 1 -- 172.21.15.105:0/3800146159 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe71803d870 con 0x56295d74a450 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.818+0000 7fe729f83740 10 client.25397 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.818+0000 7fe729f83740 10 client.25397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.818+0000 7fe729f83740 20 client.25397 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:57.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.818+0000 7fe729f83740 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56295d75aa50 0x56295d75ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.818+0000 7fe729f83740 4 client.25397 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:57.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.818+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe71009a2f0 con 0x56295d75aa50 2024-03-26T17:38:57.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.818+0000 7fe725133640 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56295d75aa50 0x56295d75ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.819+0000 7fe725133640 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56295d75aa50 0x56295d75ce40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f2455d5b8ab4f8 server_cookie=2a6cab3e5f0b2a62 in_seq=0 out_seq=0 2024-03-26T17:38:57.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.819+0000 7fe708ff9640 10 client.25397 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:57.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.819+0000 7fe708ff9640 1 -- 172.21.15.105:0/3800146159 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fe71009a2f0 con 0x56295d75aa50 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.819+0000 7fe708ff9640 10 client.25397 handle_command_reply: tid=0 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.819+0000 7fe729f83740 1 client.25397 shutdown 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 msgr2=0x56295d756170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 0x56295d756170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe718032420 tx=0x7fe718032bd0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 shutdown_connections 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56295d75aa50 0x56295d75ce40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d74a450 0x56295d756170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe708ff9640 0 client.25397 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d74f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 --2- 172.21.15.105:0/3800146159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 >> 172.21.15.105:0/3800146159 conn(0x56295d73fe60 msgr2=0x56295d73f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 shutdown_connections 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 1 -- 172.21.15.105:0/3800146159 wait complete. 2024-03-26T17:38:57.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.820+0000 7fe729f83740 20 client.25397 trim_cache size 0 max 16384 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.854+0000 7fe729f83740 1 Processor -- start 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.854+0000 7fe729f83740 1 -- start start 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe729f83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe729f83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d756130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe729f83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d74a450 0x56295d74f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe729f83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe720072900 con 0x56295d7452e0 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe725133640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe725133640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60528/0 (socket says 172.21.15.105:60528) 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe725133640 1 -- 172.21.15.105:0/2748341542 learned_addr learned my addr 172.21.15.105:0/2748341542 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe724932640 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d756130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe725934640 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d74a450 0x56295d74f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.855+0000 7fe729f83740 1 -- 172.21.15.105:0/2748341542 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7201b0e90 con 0x56295d7448f0 2024-03-26T17:38:57.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.856+0000 7fe729f83740 1 -- 172.21.15.105:0/2748341542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe720072fd0 con 0x56295d74a450 2024-03-26T17:38:57.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.856+0000 7fe725934640 1 -- 172.21.15.105:0/2748341542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7448f0 msgr2=0x56295d74ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.856+0000 7fe725934640 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.856+0000 7fe725934640 1 -- 172.21.15.105:0/2748341542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 msgr2=0x56295d756130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.856+0000 7fe725934640 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d756130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.856+0000 7fe725934640 1 -- 172.21.15.105:0/2748341542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295d73b8d0 con 0x56295d74a450 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.856+0000 7fe724932640 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d756130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.856+0000 7fe725934640 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d74a450 0x56295d74f1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe71803dd60 tx=0x7fe71803df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59be3b55495b49de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.857+0000 7fe70affd640 1 -- 172.21.15.105:0/2748341542 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71804b070 con 0x56295d74a450 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.857+0000 7fe70affd640 1 -- 172.21.15.105:0/2748341542 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe71803daa0 con 0x56295d74a450 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.857+0000 7fe70affd640 1 -- 172.21.15.105:0/2748341542 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe718034a90 con 0x56295d74a450 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 -- 172.21.15.105:0/2748341542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d74a450 msgr2=0x56295d74f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d74a450 0x56295d74f1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe71803dd60 tx=0x7fe71803df90 comp rx=0 tx=0).stop 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 -- 172.21.15.105:0/2748341542 shutdown_connections 2024-03-26T17:38:57.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d74a450 0x56295d74f1a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d7452e0 0x56295d756130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 --2- 172.21.15.105:0/2748341542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7448f0 0x56295d74ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 -- 172.21.15.105:0/2748341542 >> 172.21.15.105:0/2748341542 conn(0x56295d73fe60 msgr2=0x56295d73ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 -- 172.21.15.105:0/2748341542 shutdown_connections 2024-03-26T17:38:57.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.858+0000 7fe729f83740 1 -- 172.21.15.105:0/2748341542 wait complete. 2024-03-26T17:38:57.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.859+0000 7fe729f83740 1 -- start start 2024-03-26T17:38:57.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.859+0000 7fe729f83740 1 Processor -- start 2024-03-26T17:38:57.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe729f83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d753b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe729f83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7452e0 0x56295d7540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe729f83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d74a450 0x56295d7550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe725133640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d753b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe725133640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d753b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55268/0 (socket says 172.21.15.105:55268) 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe724932640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7452e0 0x56295d7540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe724932640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7452e0 0x56295d7540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60532/0 (socket says 172.21.15.105:60532) 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe725934640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d74a450 0x56295d7550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe725934640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d74a450 0x56295d7550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52024/0 (socket says 172.21.15.105:52024) 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe725133640 1 -- 172.21.15.105:0/1742194466 learned_addr learned my addr 172.21.15.105:0/1742194466 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7201b07c0 con 0x56295d74a450 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7200736a0 con 0x56295d7452e0 2024-03-26T17:38:57.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.860+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7200708e0 con 0x56295d7448f0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe729f83740 10 client.0 _ll_register_callbacks cb 0x7fe710005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe724932640 1 -- 172.21.15.105:0/1742194466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 msgr2=0x56295d753b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe724932640 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d753b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe724932640 1 -- 172.21.15.105:0/1742194466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d74a450 msgr2=0x56295d7550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe724932640 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d74a450 0x56295d7550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe724932640 1 -- 172.21.15.105:0/1742194466 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295d7556a0 con 0x56295d7452e0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe724932640 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7452e0 0x56295d7540a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x56295d745d50 tx=0x7fe71009bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4038f57fb61bd35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe708ff9640 10 client.25403 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe71803daa0 con 0x56295d7452e0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe708ff9640 1 -- 172.21.15.105:0/1742194466 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71007ae60 con 0x56295d7452e0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe708ff9640 1 -- 172.21.15.105:0/1742194466 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fe7100bc140 con 0x56295d7452e0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe708ff9640 1 -- 172.21.15.105:0/1742194466 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe710098290 con 0x56295d7452e0 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.861+0000 7fe729f83740 10 client.25403 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:57.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=27}) -- 0x56295d7c0ea0 con 0x56295d7452e0 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe708ff9640 1 -- 172.21.15.105:0/1742194466 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe71001f070 con 0x56295d7452e0 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe729f83740 10 client.25403 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe729f83740 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe729f83740 20 client.25403 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe729f83740 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56295d75aa50 0x56295d7c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe729f83740 4 client.25403 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe71803daa0 con 0x56295d75aa50 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.862+0000 7fe725133640 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56295d75aa50 0x56295d7c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe725133640 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56295d75aa50 0x56295d7c32c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22de47a5d2eb0d67 server_cookie=755b102185958ece in_seq=0 out_seq=0 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe708ff9640 10 client.25403 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe708ff9640 1 -- 172.21.15.105:0/1742194466 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe71803daa0 con 0x56295d75aa50 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe708ff9640 10 client.25403 handle_command_reply: tid=0 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe729f83740 1 client.25403 shutdown 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7452e0 msgr2=0x56295d7540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe729f83740 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7452e0 0x56295d7540a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x56295d745d50 tx=0x7fe71009bef0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 shutdown_connections 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe729f83740 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56295d75aa50 0x56295d7c32c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe729f83740 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56295d74a450 0x56295d7550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.863+0000 7fe708ff9640 0 client.25403 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.864+0000 7fe729f83740 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56295d7452e0 0x56295d7540a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.864+0000 7fe729f83740 1 --2- 172.21.15.105:0/1742194466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56295d7448f0 0x56295d753b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.864+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 >> 172.21.15.105:0/1742194466 conn(0x56295d73fe60 msgr2=0x56295d74ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.864+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 shutdown_connections 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.864+0000 7fe729f83740 1 -- 172.21.15.105:0/1742194466 wait complete. 2024-03-26T17:38:57.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.864+0000 7fe729f83740 20 client.25403 trim_cache size 0 max 16384 2024-03-26T17:38:57.862 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:38:57.862 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:38:57.862 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:38:57.862 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 -- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe70c0a30e0 msgr2=0x7fe70c0a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe70c0a30e0 0x7fe70c0a55a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0058c0 tx=0x7fe71c005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 -- 172.21.15.105:0/1013254121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101170 msgr2=0x7fe720197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101170 0x7fe720197f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe71000e9f0 tx=0x7fe71000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 -- 172.21.15.105:0/1013254121 shutdown_connections 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe70c0a30e0 0x7fe70c0a55a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe720101b30 0x7fe72019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe720101170 0x7fe720197f80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 --2- 172.21.15.105:0/1013254121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7200fff80 0x7fe720197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 -- 172.21.15.105:0/1013254121 >> 172.21.15.105:0/1013254121 conn(0x7fe7200fb4f0 msgr2=0x7fe720104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 -- 172.21.15.105:0/1013254121 shutdown_connections 2024-03-26T17:38:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:57.866+0000 7fe727397640 1 -- 172.21.15.105:0/1013254121 wait complete. 2024-03-26T17:38:57.874 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:38:57.875 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:38:57.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:57 smithi105 ceph-mon[32280]: pgmap v364: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:57.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:57 smithi135 ceph-mon[35636]: pgmap v364: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:58.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 -- 172.21.15.105:0/1386593400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 msgr2=0x7fa7c010c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/1386593400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 0x7fa7c010c710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009900 tx=0x7fa7b002bc60 comp rx=0 tx=0).stop 2024-03-26T17:38:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 -- 172.21.15.105:0/1386593400 shutdown_connections 2024-03-26T17:38:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/1386593400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c010dea0 0x7fa7c0114730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/1386593400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c010d4e0 0x7fa7c010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/1386593400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 0x7fa7c010c710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 -- 172.21.15.105:0/1386593400 >> 172.21.15.105:0/1386593400 conn(0x7fa7c006c320 msgr2=0x7fa7c006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 -- 172.21.15.105:0/1386593400 shutdown_connections 2024-03-26T17:38:58.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.040+0000 7fa7c6c62640 1 -- 172.21.15.105:0/1386593400 wait complete. 2024-03-26T17:38:58.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c6c62640 1 Processor -- start 2024-03-26T17:38:58.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c6c62640 1 -- start start 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c6c62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 0x7fa7c01a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c6c62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c010d4e0 0x7fa7c01a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c6c62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c010dea0 0x7fa7c01a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c6c62640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0117e30 con 0x7fa7c010dea0 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c6c62640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0117cb0 con 0x7fa7c010d4e0 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c6c62640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0117fb0 con 0x7fa7c010c2f0 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c49fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 0x7fa7c01a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7bffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c010d4e0 0x7fa7c01a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c49fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 0x7fa7c01a2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55282/0 (socket says 172.21.15.105:55282) 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c49fe640 1 -- 172.21.15.105:0/2538394562 learned_addr learned my addr 172.21.15.105:0/2538394562 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7c51ff640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c010dea0 0x7fa7c01a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7bffff640 1 -- 172.21.15.105:0/2538394562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 msgr2=0x7fa7c01a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7bffff640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 0x7fa7c01a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7bffff640 1 -- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c010dea0 msgr2=0x7fa7c01a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7bffff640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c010dea0 0x7fa7c01a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.041+0000 7fa7bffff640 1 -- 172.21.15.105:0/2538394562 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c01aa190 con 0x7fa7c010d4e0 2024-03-26T17:38:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.042+0000 7fa7bffff640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c010d4e0 0x7fa7c01a2ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d6d0 tx=0x7fa7b400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.042+0000 7fa7bdffb640 1 -- 172.21.15.105:0/2538394562 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4017070 con 0x7fa7c010d4e0 2024-03-26T17:38:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.042+0000 7fa7bdffb640 1 -- 172.21.15.105:0/2538394562 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7b4009820 con 0x7fa7c010d4e0 2024-03-26T17:38:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.042+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c01aa480 con 0x7fa7c010d4e0 2024-03-26T17:38:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.042+0000 7fa7bdffb640 1 -- 172.21.15.105:0/2538394562 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4004c40 con 0x7fa7c010d4e0 2024-03-26T17:38:58.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.042+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c01aaa20 con 0x7fa7c010d4e0 2024-03-26T17:38:58.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.044+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c01136e0 con 0x7fa7c010d4e0 2024-03-26T17:38:58.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.046+0000 7fa7bdffb640 1 -- 172.21.15.105:0/2538394562 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fa7b40099d0 con 0x7fa7c010d4e0 2024-03-26T17:38:58.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.048+0000 7fa7bdffb640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7a80a3030 0x7fa7a80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.048+0000 7fa7bdffb640 1 -- 172.21.15.105:0/2538394562 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fa7b40cf740 con 0x7fa7c010d4e0 2024-03-26T17:38:58.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.048+0000 7fa7c49fe640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7a80a3030 0x7fa7a80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.048+0000 7fa7c49fe640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7a80a3030 0x7fa7a80a54f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0004c10 tx=0x7fa7b0009640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.054+0000 7fa7bdffb640 1 -- 172.21.15.105:0/2538394562 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa7b409c3f0 con 0x7fa7c010d4e0 2024-03-26T17:38:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:57 smithi144 ceph-mon[36095]: pgmap v364: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-26T17:38:58.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.445+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7c002d490 con 0x7fa7c010d4e0 2024-03-26T17:38:58.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.446+0000 7fa7bdffb640 1 -- 172.21.15.105:0/2538394562 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7fa7b409bcf0 con 0x7fa7c010d4e0 2024-03-26T17:38:58.440 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:38:58.440 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:38:20.295050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T17:38:58.440 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7a80a3030 msgr2=0x7fa7a80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7a80a3030 0x7fa7a80a54f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0004c10 tx=0x7fa7b0009640 comp rx=0 tx=0).stop 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c010d4e0 msgr2=0x7fa7c01a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c010d4e0 0x7fa7c01a2ab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d6d0 tx=0x7fa7b400db90 comp rx=0 tx=0).stop 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 shutdown_connections 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7a80a3030 0x7fa7a80a54f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7c010dea0 0x7fa7c01a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c010d4e0 0x7fa7c01a2ab0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 --2- 172.21.15.105:0/2538394562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c010c2f0 0x7fa7c01a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 >> 172.21.15.105:0/2538394562 conn(0x7fa7c006c320 msgr2=0x7fa7c01108b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 shutdown_connections 2024-03-26T17:38:58.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.449+0000 7fa7c6c62640 1 -- 172.21.15.105:0/2538394562 wait complete. 2024-03-26T17:38:58.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:38:58.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 -- 172.21.15.105:0/3485975341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 msgr2=0x7f359c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 --2- 172.21.15.105:0/3485975341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 0x7f359c109d30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3594009be0 tx=0x7f359402bef0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 -- 172.21.15.105:0/3485975341 shutdown_connections 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 --2- 172.21.15.105:0/3485975341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 0x7f359c109d30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 --2- 172.21.15.105:0/3485975341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359c102ae0 0x7f359c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 --2- 172.21.15.105:0/3485975341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f359c1018f0 0x7f359c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 -- 172.21.15.105:0/3485975341 >> 172.21.15.105:0/3485975341 conn(0x7f359c0fce60 msgr2=0x7f359c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 -- 172.21.15.105:0/3485975341 shutdown_connections 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.619+0000 7f35a339e640 1 -- 172.21.15.105:0/3485975341 wait complete. 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a339e640 1 Processor -- start 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a339e640 1 -- start start 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a339e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359c1018f0 0x7f359c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a339e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f359c102ae0 0x7f359c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a339e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 0x7f359c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a339e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10d490 con 0x7f359c102ae0 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a339e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10d310 con 0x7f359c1018f0 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a339e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10d610 con 0x7f359c1034a0 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a193b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 0x7f359c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a113a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359c1018f0 0x7f359c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a193b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 0x7f359c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55290/0 (socket says 172.21.15.105:55290) 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a193b640 1 -- 172.21.15.105:0/930783535 learned_addr learned my addr 172.21.15.105:0/930783535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a0939640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f359c102ae0 0x7f359c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a113a640 1 -- 172.21.15.105:0/930783535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 msgr2=0x7f359c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a113a640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 0x7f359c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a113a640 1 -- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f359c102ae0 msgr2=0x7f359c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a113a640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f359c102ae0 0x7f359c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.620+0000 7f35a113a640 1 -- 172.21.15.105:0/930783535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c19f770 con 0x7f359c1018f0 2024-03-26T17:38:58.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.621+0000 7f35a113a640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359c1018f0 0x7f359c197b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f359800b3c0 tx=0x7f359800b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.621+0000 7f35927fc640 1 -- 172.21.15.105:0/930783535 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598011830 con 0x7f359c1018f0 2024-03-26T17:38:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.621+0000 7f35927fc640 1 -- 172.21.15.105:0/930783535 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3598016070 con 0x7f359c1018f0 2024-03-26T17:38:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.621+0000 7f35a339e640 1 -- 172.21.15.105:0/930783535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c19fa60 con 0x7f359c1018f0 2024-03-26T17:38:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.621+0000 7f35927fc640 1 -- 172.21.15.105:0/930783535 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598010e30 con 0x7f359c1018f0 2024-03-26T17:38:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.621+0000 7f35a339e640 1 -- 172.21.15.105:0/930783535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c19ff80 con 0x7f359c1018f0 2024-03-26T17:38:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a5f88740 1 Processor -- start 2024-03-26T17:38:58.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a5f88740 1 -- start start 2024-03-26T17:38:58.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35927fc640 1 -- 172.21.15.105:0/930783535 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f35980119d0 con 0x7f359c1018f0 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10d9b0 0x56322b10ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b110e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b11e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a5f88740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56322b072970 con 0x56322b10e3a0 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a5f88740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56322b01ad70 con 0x56322b10d9b0 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a5f88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56322b06e920 con 0x56322b111500 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a113a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b110e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.624+0000 7f35a193b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10d9b0 0x56322b10ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a113a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b110e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52060/0 (socket says 172.21.15.105:52060) 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a113a640 1 -- 172.21.15.105:0/3626488010 learned_addr learned my addr 172.21.15.105:0/3626488010 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a193b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10d9b0 0x56322b10ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60568/0 (socket says 172.21.15.105:60568) 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a0939640 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b11e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a193b640 1 -- 172.21.15.105:0/3626488010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 msgr2=0x56322b11e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35927fc640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35880a2fb0 0x7f35880a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a193b640 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b11e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a193b640 1 -- 172.21.15.105:0/3626488010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 msgr2=0x56322b110e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a193b640 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b110e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a193b640 1 -- 172.21.15.105:0/3626488010 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56322ad863f0 con 0x56322b10d9b0 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a113a640 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35927fc640 1 -- 172.21.15.105:0/930783535 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f35980ceb30 con 0x7f359c1018f0 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a0939640 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b11e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:38:58.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.625+0000 7f35a193b640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35880a2fb0 0x7f35880a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.626+0000 7f35a193b640 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10d9b0 0x56322b10ddd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3594004af0 tx=0x7f3594009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671bde3590c54df9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.626+0000 7f35a193b640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35880a2fb0 0x7f35880a5470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3594046000 tx=0x7f3594048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.626+0000 7f35877fe640 1 -- 172.21.15.105:0/3626488010 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594050070 con 0x56322b10d9b0 2024-03-26T17:38:58.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.626+0000 7f35877fe640 1 -- 172.21.15.105:0/3626488010 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3594033750 con 0x56322b10d9b0 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.626+0000 7f35877fe640 1 -- 172.21.15.105:0/3626488010 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594034ca0 con 0x56322b10d9b0 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 -- 172.21.15.105:0/3626488010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10d9b0 msgr2=0x56322b10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10d9b0 0x56322b10ddd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3594004af0 tx=0x7f3594009960 comp rx=0 tx=0).stop 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 -- 172.21.15.105:0/3626488010 shutdown_connections 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b11e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 --2- 172.21.15.105:0/3626488010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10d9b0 0x56322b10ddd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 -- 172.21.15.105:0/3626488010 >> 172.21.15.105:0/3626488010 conn(0x56322b108fa0 msgr2=0x56322b1093a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 -- 172.21.15.105:0/3626488010 shutdown_connections 2024-03-26T17:38:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 -- 172.21.15.105:0/3626488010 wait complete. 2024-03-26T17:38:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 -- start start 2024-03-26T17:38:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.627+0000 7f35a5f88740 1 Processor -- start 2024-03-26T17:38:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b1207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b120ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b111500 0x56322b127d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a5f88740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56322b11ecb0 con 0x56322b111500 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a5f88740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56322b02e310 con 0x56322b10e3a0 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a5f88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56322b11ee30 con 0x56322b10d9b0 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a5f88740 10 client.0 _ll_register_callbacks cb 0x56322b069c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b1207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a193b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b111500 0x56322b127d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b1207a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55308/0 (socket says 172.21.15.105:55308) 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 -- 172.21.15.105:0/354787959 learned_addr learned my addr 172.21.15.105:0/354787959 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a0939640 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b120ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 -- 172.21.15.105:0/354787959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 msgr2=0x56322b120ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b120ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 -- 172.21.15.105:0/354787959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b111500 msgr2=0x56322b127d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b111500 0x56322b127d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 -- 172.21.15.105:0/354787959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56322ae507d0 con 0x56322b10d9b0 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a0939640 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b120ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:38:58.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a193b640 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b111500 0x56322b127d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.628+0000 7f35a113a640 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b1207a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3598000c00 tx=0x7f35980cc540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90810667cb9571d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f3584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f3584ff9640 1 -- 172.21.15.105:0/354787959 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359809acb0 con 0x56322b10d9b0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f3584ff9640 1 -- 172.21.15.105:0/354787959 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f35980cc690 con 0x56322b10d9b0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 1 -- 172.21.15.105:0/354787959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3594033750 con 0x56322b10d9b0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f3584ff9640 1 -- 172.21.15.105:0/354787959 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f359809aad0 con 0x56322b10d9b0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 10 client.25417 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 1 -- 172.21.15.105:0/354787959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56322b12b070 con 0x56322b10d9b0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f3584ff9640 1 -- 172.21.15.105:0/354787959 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35980d4070 con 0x56322b10d9b0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f3584ff9640 1 -- 172.21.15.105:0/354787959 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3598095ce0 con 0x56322b10d9b0 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 10 client.25417 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 10 client.25417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:38:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 20 client.25417 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56322b12c770 0x56322b12ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 4 client.25417 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.629+0000 7f35a5f88740 1 -- 172.21.15.105:0/354787959 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3594033750 con 0x56322b12c770 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.630+0000 7f35a0939640 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56322b12c770 0x56322b12ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.630+0000 7f35a0939640 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56322b12c770 0x56322b12ebe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=770335590b3e07f server_cookie=5b92a071007d9831 in_seq=0 out_seq=0 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.630+0000 7f3584ff9640 10 client.25417 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f3584ff9640 1 -- 172.21.15.105:0/354787959 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f3594033750 con 0x56322b12c770 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f3584ff9640 10 client.25417 handle_command_reply: tid=0 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 client.25417 shutdown 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 -- 172.21.15.105:0/354787959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 msgr2=0x56322b1207a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b1207a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3598000c00 tx=0x7f35980cc540 comp rx=0 tx=0).stop 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 -- 172.21.15.105:0/354787959 shutdown_connections 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56322b12c770 0x56322b12ebe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f3584ff9640 0 client.25417 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:38:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b111500 0x56322b127d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b120ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 --2- 172.21.15.105:0/354787959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b1207a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 -- 172.21.15.105:0/354787959 >> 172.21.15.105:0/354787959 conn(0x56322b108fa0 msgr2=0x56322b1081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 -- 172.21.15.105:0/354787959 shutdown_connections 2024-03-26T17:38:58.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 1 -- 172.21.15.105:0/354787959 wait complete. 2024-03-26T17:38:58.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.631+0000 7f35a5f88740 20 client.25417 trim_cache size 0 max 16384 2024-03-26T17:38:58.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:58 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2538394562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:58.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a5f88740 1 Processor -- start 2024-03-26T17:38:58.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a5f88740 1 -- start start 2024-03-26T17:38:58.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10d9b0 0x56322b1203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a113a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10d9b0 0x56322b1203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a113a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10d9b0 0x56322b1203b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52074/0 (socket says 172.21.15.105:52074) 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b127ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a113a640 1 -- 172.21.15.105:0/2728733051 learned_addr learned my addr 172.21.15.105:0/2728733051 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b1208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a5f88740 1 -- 172.21.15.105:0/2728733051 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a6c30 con 0x56322b10d9b0 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a5f88740 1 -- 172.21.15.105:0/2728733051 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a8770 con 0x56322b10e3a0 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.658+0000 7f35a5f88740 1 -- 172.21.15.105:0/2728733051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a7300 con 0x56322b111500 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a0939640 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b127ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a193b640 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b1208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a0939640 1 -- 172.21.15.105:0/2728733051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 msgr2=0x56322b1208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a0939640 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b1208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a0939640 1 -- 172.21.15.105:0/2728733051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10d9b0 msgr2=0x56322b1203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a0939640 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10d9b0 0x56322b1203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a0939640 1 -- 172.21.15.105:0/2728733051 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56322b109d10 con 0x56322b10e3a0 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a113a640 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10d9b0 0x56322b1203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f35a0939640 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b127ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x56322b121c80 tx=0x7f358c01df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281ba7213e84defa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f3586ffd640 1 -- 172.21.15.105:0/2728733051 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c022020 con 0x56322b10e3a0 2024-03-26T17:38:58.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.659+0000 7f3586ffd640 1 -- 172.21.15.105:0/2728733051 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c01d730 con 0x56322b10e3a0 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 -- 172.21.15.105:0/2728733051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 msgr2=0x56322b127ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b127ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x56322b121c80 tx=0x7f358c01df60 comp rx=0 tx=0).stop 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f3586ffd640 1 -- 172.21.15.105:0/2728733051 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c003850 con 0x56322b10e3a0 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 -- 172.21.15.105:0/2728733051 shutdown_connections 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b111500 0x56322b1208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b10e3a0 0x56322b127ce0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2728733051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10d9b0 0x56322b1203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 -- 172.21.15.105:0/2728733051 >> 172.21.15.105:0/2728733051 conn(0x56322b108fa0 msgr2=0x56322b11f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 -- 172.21.15.105:0/2728733051 shutdown_connections 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.660+0000 7f35a5f88740 1 -- 172.21.15.105:0/2728733051 wait complete. 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 1 -- start start 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 1 Processor -- start 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b124d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b1252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b111500 0x56322b190ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a80a0 con 0x56322b10e3a0 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a79d0 con 0x56322b111500 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f359c1a4c10 con 0x56322b10d9b0 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a5f88740 10 client.0 _ll_register_callbacks cb 0x7f35980d22d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a113a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b124d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a193b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b111500 0x56322b190ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a113a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b124d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55332/0 (socket says 172.21.15.105:55332) 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.661+0000 7f35a113a640 1 -- 172.21.15.105:0/2630756167 learned_addr learned my addr 172.21.15.105:0/2630756167 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a0939640 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b1252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a193b640 1 -- 172.21.15.105:0/2630756167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 msgr2=0x56322b124d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a193b640 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b124d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a193b640 1 -- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 msgr2=0x56322b1252c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a193b640 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b1252c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a193b640 1 -- 172.21.15.105:0/2630756167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56322b191090 con 0x56322b111500 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a113a640 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b124d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a0939640 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b1252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a193b640 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b111500 0x56322b190ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3594006010 tx=0x7f35940348e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e4a502c2894362 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f3584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f3584ff9640 1 -- 172.21.15.105:0/2630756167 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594050040 con 0x56322b111500 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a5f88740 1 -- 172.21.15.105:0/2630756167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f358c01d730 con 0x56322b111500 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f3584ff9640 1 -- 172.21.15.105:0/2630756167 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f359402cca0 con 0x56322b111500 2024-03-26T17:38:58.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f3584ff9640 1 -- 172.21.15.105:0/2630756167 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f359404c520 con 0x56322b111500 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a5f88740 10 client.25424 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.662+0000 7f35a5f88740 1 -- 172.21.15.105:0/2630756167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56322b1923e0 con 0x56322b111500 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.663+0000 7f3584ff9640 1 -- 172.21.15.105:0/2630756167 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f358c01d730 con 0x56322b111500 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.663+0000 7f3584ff9640 1 -- 172.21.15.105:0/2630756167 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3594049070 con 0x56322b111500 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.663+0000 7f35a5f88740 10 client.25424 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.663+0000 7f35a5f88740 10 client.25424 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.663+0000 7f35a5f88740 20 client.25424 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.663+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56322b1928e0 0x56322b194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.664+0000 7f35a5f88740 4 client.25424 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.664+0000 7f35a5f88740 1 -- 172.21.15.105:0/2630756167 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56322b07feb0 con 0x56322b1928e0 2024-03-26T17:38:58.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.664+0000 7f35a113a640 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56322b1928e0 0x56322b194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.665+0000 7f3584ff9640 10 client.25424 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:38:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.665+0000 7f35a113a640 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56322b1928e0 0x56322b194cd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9262539ca16dfff server_cookie=3a7da519b3cec6d0 in_seq=0 out_seq=0 2024-03-26T17:38:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f3584ff9640 1 -- 172.21.15.105:0/2630756167 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56322b07feb0 con 0x56322b1928e0 2024-03-26T17:38:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f3584ff9640 10 client.25424 handle_command_reply: tid=0 2024-03-26T17:38:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 client.25424 shutdown 2024-03-26T17:38:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 -- 172.21.15.105:0/2630756167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b111500 msgr2=0x56322b190ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b111500 0x56322b190ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3594006010 tx=0x7f35940348e0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 -- 172.21.15.105:0/2630756167 shutdown_connections 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56322b1928e0 0x56322b194cd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56322b111500 0x56322b190ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f3584ff9640 0 client.25424 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56322b10e3a0 0x56322b1252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 --2- 172.21.15.105:0/2630756167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56322b10d9b0 0x56322b124d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.666+0000 7f35a5f88740 1 -- 172.21.15.105:0/2630756167 >> 172.21.15.105:0/2630756167 conn(0x56322b108fa0 msgr2=0x56322b1203b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.667+0000 7f35a5f88740 1 -- 172.21.15.105:0/2630756167 shutdown_connections 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.667+0000 7f35a5f88740 1 -- 172.21.15.105:0/2630756167 wait complete. 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.667+0000 7f35a5f88740 20 client.25424 trim_cache size 0 max 16384 2024-03-26T17:38:58.664 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:38:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.667+0000 7f35a339e640 1 -- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35880a2fb0 msgr2=0x7f35880a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.667+0000 7f35a339e640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35880a2fb0 0x7f35880a5470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3594046000 tx=0x7f3594048000 comp rx=0 tx=0).stop 2024-03-26T17:38:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.667+0000 7f35a339e640 1 -- 172.21.15.105:0/930783535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359c1018f0 msgr2=0x7f359c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.667+0000 7f35a339e640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359c1018f0 0x7f359c197b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f359800b3c0 tx=0x7f359800b880 comp rx=0 tx=0).stop 2024-03-26T17:38:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.668+0000 7f35a339e640 1 -- 172.21.15.105:0/930783535 shutdown_connections 2024-03-26T17:38:58.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.668+0000 7f35a339e640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35880a2fb0 0x7f35880a5470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.668+0000 7f35a339e640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f359c1034a0 0x7f359c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.668+0000 7f35a339e640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f359c102ae0 0x7f359c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.668+0000 7f35a339e640 1 --2- 172.21.15.105:0/930783535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f359c1018f0 0x7f359c197b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.668+0000 7f35a339e640 1 -- 172.21.15.105:0/930783535 >> 172.21.15.105:0/930783535 conn(0x7f359c0fce60 msgr2=0x7f359c105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.668+0000 7f35a339e640 1 -- 172.21.15.105:0/930783535 shutdown_connections 2024-03-26T17:38:58.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.668+0000 7f35a339e640 1 -- 172.21.15.105:0/930783535 wait complete. 2024-03-26T17:38:58.673 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 -- 172.21.15.105:0/4129547215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 msgr2=0x7fa4fc109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 --2- 172.21.15.105:0/4129547215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 0x7fa4fc109a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009be0 tx=0x7fa4f402bef0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 -- 172.21.15.105:0/4129547215 shutdown_connections 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 --2- 172.21.15.105:0/4129547215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 0x7fa4fc109a50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 --2- 172.21.15.105:0/4129547215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc104d30 0x7fa4fc107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 --2- 172.21.15.105:0/4129547215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4fc102400 0x7fa4fc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 -- 172.21.15.105:0/4129547215 >> 172.21.15.105:0/4129547215 conn(0x7fa4fc0fbd50 msgr2=0x7fa4fc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 -- 172.21.15.105:0/4129547215 shutdown_connections 2024-03-26T17:38:58.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.828+0000 7fa5010d3640 1 -- 172.21.15.105:0/4129547215 wait complete. 2024-03-26T17:38:58.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.829+0000 7fa5010d3640 1 Processor -- start 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.829+0000 7fa5010d3640 1 -- start start 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.829+0000 7fa5010d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc102400 0x7fa4fc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.829+0000 7fa5010d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4fc104d30 0x7fa4fc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.829+0000 7fa5010d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 0x7fa4fc19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.829+0000 7fa4fad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc102400 0x7fa4fc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc102400 0x7fa4fc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55352/0 (socket says 172.21.15.105:55352) 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fad9d640 1 -- 172.21.15.105:0/3479868187 learned_addr learned my addr 172.21.15.105:0/3479868187 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc10d450 con 0x7fa4fc104d30 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc10d2d0 con 0x7fa4fc107660 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc10d5d0 con 0x7fa4fc102400 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4fc104d30 0x7fa4fc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fb59e640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 0x7fa4fc19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fad9d640 1 -- 172.21.15.105:0/3479868187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 msgr2=0x7fa4fc19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 0x7fa4fc19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fad9d640 1 -- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4fc104d30 msgr2=0x7fa4fc198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4fc104d30 0x7fa4fc198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fad9d640 1 -- 172.21.15.105:0/3479868187 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc19f740 con 0x7fa4fc102400 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fb59e640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 0x7fa4fc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4fc104d30 0x7fa4fc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc102400 0x7fa4fc197b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00e9c0 tx=0x7fa4ec00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4e3fff640 1 -- 172.21.15.105:0/3479868187 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00ccc0 con 0x7fa4fc102400 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc19fa30 con 0x7fa4fc102400 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4e3fff640 1 -- 172.21.15.105:0/3479868187 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4ec0054d0 con 0x7fa4fc102400 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.830+0000 7fa4e3fff640 1 -- 172.21.15.105:0/3479868187 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec010430 con 0x7fa4fc102400 2024-03-26T17:38:58.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.831+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc19ff30 con 0x7fa4fc102400 2024-03-26T17:38:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.833+0000 7fa503d95740 1 Processor -- start 2024-03-26T17:38:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.833+0000 7fa4e3fff640 1 -- 172.21.15.105:0/3479868187 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fa4ec004f50 con 0x7fa4fc102400 2024-03-26T17:38:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.833+0000 7fa503d95740 1 -- start start 2024-03-26T17:38:58.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa503d95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf480a30 0x55addf480e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa503d95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf485f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa503d95740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf488a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa503d95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55addf3e5970 con 0x55addf481420 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa503d95740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55addf38dd70 con 0x55addf486590 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa503d95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55addf3e1920 con 0x55addf480a30 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf480a30 0x55addf480e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf485f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf480a30 0x55addf480e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55360/0 (socket says 172.21.15.105:55360) 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fb59e640 1 -- 172.21.15.105:0/2486017960 learned_addr learned my addr 172.21.15.105:0/2486017960 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/2486017960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf488a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4e3fff640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4e40a2fb0 0x7fa4e40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fa59c640 1 -- 172.21.15.105:0/2486017960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf480a30 msgr2=0x55addf480e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/2486017960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf480a30 0x55addf480e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fa59c640 1 -- 172.21.15.105:0/2486017960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 msgr2=0x55addf485f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/2486017960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf485f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fa59c640 1 -- 172.21.15.105:0/2486017960 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55addf0f93f0 con 0x55addf486590 2024-03-26T17:38:58.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4e3fff640 1 -- 172.21.15.105:0/3479868187 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fa4ec0cfaa0 con 0x7fa4fc102400 2024-03-26T17:38:58.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fb59e640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4e40a2fb0 0x7fa4e40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.834+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/2486017960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf488a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0694c0 tx=0x7fa4f00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d2fc14ddff6742 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.835+0000 7fa4e17fa640 1 -- 172.21.15.105:0/2486017960 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0004020 con 0x55addf486590 2024-03-26T17:38:58.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.835+0000 7fa4e17fa640 1 -- 172.21.15.105:0/2486017960 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4f00054d0 con 0x55addf486590 2024-03-26T17:38:58.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.835+0000 7fa4e17fa640 1 -- 172.21.15.105:0/2486017960 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0004d70 con 0x55addf486590 2024-03-26T17:38:58.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- 172.21.15.105:0/2486017960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 msgr2=0x55addf488a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa4fb59e640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4e40a2fb0 0x7fa4e40a5470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4005f00 tx=0x7fa4f402bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 --2- 172.21.15.105:0/2486017960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf488a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0694c0 tx=0x7fa4f00118d0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- 172.21.15.105:0/2486017960 shutdown_connections 2024-03-26T17:38:58.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 --2- 172.21.15.105:0/2486017960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf488a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 --2- 172.21.15.105:0/2486017960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf485f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 --2- 172.21.15.105:0/2486017960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf480a30 0x55addf480e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- 172.21.15.105:0/2486017960 >> 172.21.15.105:0/2486017960 conn(0x55addf47bfa0 msgr2=0x55addf47c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- 172.21.15.105:0/2486017960 shutdown_connections 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- 172.21.15.105:0/2486017960 wait complete. 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- start start 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 Processor -- start 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf480a30 0x55addf48b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf481420 0x55addf48b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf486590 0x55addf4925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55addf489540 con 0x55addf480a30 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55addf3a1310 con 0x55addf481420 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55addf4896c0 con 0x55addf486590 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.836+0000 7fa503d95740 10 client.0 _ll_register_callbacks cb 0x55addf3dcc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf480a30 0x55addf48b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf480a30 0x55addf48b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52106/0 (socket says 172.21.15.105:52106) 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 -- 172.21.15.105:0/2297731734 learned_addr learned my addr 172.21.15.105:0/2297731734 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fa59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf481420 0x55addf48b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fb59e640 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf486590 0x55addf4925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 -- 172.21.15.105:0/2297731734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf486590 msgr2=0x55addf4925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf486590 0x55addf4925b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 -- 172.21.15.105:0/2297731734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf481420 msgr2=0x55addf48b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf481420 0x55addf48b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 -- 172.21.15.105:0/2297731734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55addf1c37d0 con 0x55addf480a30 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf481420 0x55addf48b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf480a30 0x55addf48b030 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec098430 tx=0x7fa4ec09bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a95805a2fe6cd2d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4deff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4deff5640 1 -- 172.21.15.105:0/2297731734 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec005680 con 0x55addf480a30 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4deff5640 1 -- 172.21.15.105:0/2297731734 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fa4ec0bc0c0 con 0x55addf480a30 2024-03-26T17:38:58.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa503d95740 1 -- 172.21.15.105:0/2297731734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f00054d0 con 0x55addf480a30 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa4deff5640 1 -- 172.21.15.105:0/2297731734 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4ec09c6d0 con 0x55addf480a30 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa503d95740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.837+0000 7fa503d95740 1 -- 172.21.15.105:0/2297731734 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55addf495990 con 0x55addf480a30 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa4deff5640 1 -- 172.21.15.105:0/2297731734 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4f00054d0 con 0x55addf480a30 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa4deff5640 1 -- 172.21.15.105:0/2297731734 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4ec09b420 con 0x55addf480a30 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa503d95740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa503d95740 10 client.15576 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa503d95740 20 client.15576 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa503d95740 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55addf497030 0x55addf4994c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa503d95740 4 client.15576 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa503d95740 1 -- 172.21.15.105:0/2297731734 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55addf3f2eb0 con 0x55addf497030 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.838+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55addf497030 0x55addf4994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.839+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55addf497030 0x55addf4994c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bce842c87552b66b server_cookie=7c27132fe21ee378 in_seq=0 out_seq=0 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.839+0000 7fa4deff5640 10 client.15576 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:38:58.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa4deff5640 1 -- 172.21.15.105:0/2297731734 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55addf3f2eb0 con 0x55addf497030 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa4deff5640 10 client.15576 handle_command_reply: tid=0 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 client.15576 shutdown 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 -- 172.21.15.105:0/2297731734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf480a30 msgr2=0x55addf48b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf480a30 0x55addf48b030 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec098430 tx=0x7fa4ec09bda0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 -- 172.21.15.105:0/2297731734 shutdown_connections 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55addf497030 0x55addf4994c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf486590 0x55addf4925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf481420 0x55addf48b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 --2- 172.21.15.105:0/2297731734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf480a30 0x55addf48b030 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 -- 172.21.15.105:0/2297731734 >> 172.21.15.105:0/2297731734 conn(0x55addf47bfa0 msgr2=0x55addf47b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa4deff5640 0 client.15576 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 -- 172.21.15.105:0/2297731734 shutdown_connections 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 1 -- 172.21.15.105:0/2297731734 wait complete. 2024-03-26T17:38:58.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.840+0000 7fa503d95740 20 client.15576 trim_cache size 0 max 16384 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.853+0000 7fa503d95740 1 Processor -- start 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.853+0000 7fa503d95740 1 -- start start 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.853+0000 7fa503d95740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf480a30 0x55addf489a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.853+0000 7fa503d95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf481420 0x55addf492570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.853+0000 7fa503d95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf486590 0x55addf489fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.853+0000 7fa503d95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ab0a0 con 0x55addf486590 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.853+0000 7fa503d95740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1acbe0 con 0x55addf480a30 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.853+0000 7fa503d95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ab770 con 0x55addf481420 2024-03-26T17:38:58.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf480a30 0x55addf489a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fa59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf481420 0x55addf492570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf486590 0x55addf489fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf480a30 0x55addf489a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60624/0 (socket says 172.21.15.105:60624) 2024-03-26T17:38:58.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 -- 172.21.15.105:0/1402715817 learned_addr learned my addr 172.21.15.105:0/1402715817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 -- 172.21.15.105:0/1402715817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf481420 msgr2=0x55addf492570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/1402715817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf481420 0x55addf492570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 -- 172.21.15.105:0/1402715817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf486590 msgr2=0x55addf489fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/1402715817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf486590 0x55addf489fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 -- 172.21.15.105:0/1402715817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55addf481800 con 0x55addf480a30 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/1402715817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf481420 0x55addf492570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/1402715817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf480a30 0x55addf489a90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec09bf60 tx=0x7fa4ec09bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81b9b27f863af50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4e0ff9640 1 -- 172.21.15.105:0/1402715817 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec005680 con 0x55addf480a30 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4e0ff9640 1 -- 172.21.15.105:0/1402715817 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4ec0d8070 con 0x55addf480a30 2024-03-26T17:38:58.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.854+0000 7fa4e0ff9640 1 -- 172.21.15.105:0/1402715817 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0bc070 con 0x55addf480a30 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 -- 172.21.15.105:0/1402715817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf480a30 msgr2=0x55addf489a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 --2- 172.21.15.105:0/1402715817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf480a30 0x55addf489a90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec09bf60 tx=0x7fa4ec09bed0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 -- 172.21.15.105:0/1402715817 shutdown_connections 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 --2- 172.21.15.105:0/1402715817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf486590 0x55addf489fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 --2- 172.21.15.105:0/1402715817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf481420 0x55addf492570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 --2- 172.21.15.105:0/1402715817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf480a30 0x55addf489a90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 -- 172.21.15.105:0/1402715817 >> 172.21.15.105:0/1402715817 conn(0x55addf47bfa0 msgr2=0x55addf48ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 -- 172.21.15.105:0/1402715817 shutdown_connections 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.855+0000 7fa503d95740 1 -- 172.21.15.105:0/1402715817 wait complete. 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa503d95740 1 -- start start 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa503d95740 1 Processor -- start 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa503d95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf48f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa503d95740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf48f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa503d95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf4f9370 0x55addf4fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa503d95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ac510 con 0x55addf481420 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa503d95740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1abe40 con 0x55addf486590 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa503d95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1a9080 con 0x55addf4f9370 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.856+0000 7fa4fa59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf48f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fa59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf48f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60636/0 (socket says 172.21.15.105:60636) 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fa59c640 1 -- 172.21.15.105:0/1209284580 learned_addr learned my addr 172.21.15.105:0/1209284580 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fb59e640 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf4f9370 0x55addf4fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa503d95740 10 client.0 _ll_register_callbacks cb 0x7fa4ec004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fa59c640 1 -- 172.21.15.105:0/1209284580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf4f9370 msgr2=0x55addf4fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf4f9370 0x55addf4fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fa59c640 1 -- 172.21.15.105:0/1209284580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 msgr2=0x55addf48f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf48f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf48f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fa59c640 1 -- 172.21.15.105:0/1209284580 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55addf48fe30 con 0x55addf486590 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf48f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4fa59c640 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf48f840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0011dd0 tx=0x7fa4f0004810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1fb1c42f20f381f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4deff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4deff5640 1 -- 172.21.15.105:0/1209284580 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f002f020 con 0x55addf486590 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.857+0000 7fa4deff5640 1 -- 172.21.15.105:0/1209284580 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fa4f002ab60 con 0x55addf486590 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 1 -- 172.21.15.105:0/1209284580 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ec0d8070 con 0x55addf486590 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa4deff5640 1 -- 172.21.15.105:0/1209284580 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa4f003a050 con 0x55addf486590 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 10 client.25442 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 1 -- 172.21.15.105:0/1209284580 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55addf4fcd90 con 0x55addf486590 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa4deff5640 1 -- 172.21.15.105:0/1209284580 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4ec0d8070 con 0x55addf486590 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa4deff5640 1 -- 172.21.15.105:0/1209284580 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa4f00035f0 con 0x55addf486590 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 10 client.25442 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 10 client.25442 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 20 client.25442 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:58.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55addf4fd5c0 0x55addf4ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 4 client.25442 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:38:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.858+0000 7fa503d95740 1 -- 172.21.15.105:0/1209284580 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55addf3ed200 con 0x55addf4fd5c0 2024-03-26T17:38:58.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.859+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55addf4fd5c0 0x55addf4ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.859+0000 7fa4fad9d640 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55addf4fd5c0 0x55addf4ff9e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10079f3788adbea3 server_cookie=d3605444e4fb1441 in_seq=0 out_seq=0 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.859+0000 7fa4deff5640 10 client.25442 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.859+0000 7fa4deff5640 1 -- 172.21.15.105:0/1209284580 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55addf3ed200 con 0x55addf4fd5c0 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.859+0000 7fa4deff5640 10 client.25442 handle_command_reply: tid=0 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.859+0000 7fa503d95740 1 client.25442 shutdown 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.859+0000 7fa503d95740 1 -- 172.21.15.105:0/1209284580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 msgr2=0x55addf48f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf48f840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0011dd0 tx=0x7fa4f0004810 comp rx=0 tx=0).stop 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 -- 172.21.15.105:0/1209284580 shutdown_connections 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55addf4fd5c0 0x55addf4ff9e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55addf4f9370 0x55addf4fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa4deff5640 0 client.25442 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55addf486590 0x55addf48f840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 --2- 172.21.15.105:0/1209284580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55addf481420 0x55addf48f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 -- 172.21.15.105:0/1209284580 >> 172.21.15.105:0/1209284580 conn(0x55addf47bfa0 msgr2=0x55addf47fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 -- 172.21.15.105:0/1209284580 shutdown_connections 2024-03-26T17:38:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 1 -- 172.21.15.105:0/1209284580 wait complete. 2024-03-26T17:38:58.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.860+0000 7fa503d95740 20 client.25442 trim_cache size 0 max 16384 2024-03-26T17:38:58.859 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4e40a2fb0 msgr2=0x7fa4e40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4e40a2fb0 0x7fa4e40a5470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4005f00 tx=0x7fa4f402bef0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc102400 msgr2=0x7fa4fc197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc102400 0x7fa4fc197b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00e9c0 tx=0x7fa4ec00ee80 comp rx=0 tx=0).stop 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 shutdown_connections 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4e40a2fb0 0x7fa4e40a5470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4fc107660 0x7fa4fc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4fc104d30 0x7fa4fc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 --2- 172.21.15.105:0/3479868187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc102400 0x7fa4fc197b20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:58.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 >> 172.21.15.105:0/3479868187 conn(0x7fa4fc0fbd50 msgr2=0x7fa4fc102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 shutdown_connections 2024-03-26T17:38:58.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:58.861+0000 7fa5010d3640 1 -- 172.21.15.105:0/3479868187 wait complete. 2024-03-26T17:38:58.868 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:38:58.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:58 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2538394562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:59.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 -- 172.21.15.105:0/1240615803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c8102370 msgr2=0x7f07c8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 --2- 172.21.15.105:0/1240615803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c8102370 0x7f07c8108c00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009ca0 tx=0x7f07c002c100 comp rx=0 tx=0).stop 2024-03-26T17:38:59.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 -- 172.21.15.105:0/1240615803 shutdown_connections 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 --2- 172.21.15.105:0/1240615803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c8102370 0x7f07c8108c00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 --2- 172.21.15.105:0/1240615803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07c81019b0 0x7f07c8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 --2- 172.21.15.105:0/1240615803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07c81007c0 0x7f07c8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 -- 172.21.15.105:0/1240615803 >> 172.21.15.105:0/1240615803 conn(0x7f07c80fbd50 msgr2=0x7f07c80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 -- 172.21.15.105:0/1240615803 shutdown_connections 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.009+0000 7f07cfeff640 1 -- 172.21.15.105:0/1240615803 wait complete. 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cfeff640 1 Processor -- start 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cfeff640 1 -- start start 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cfeff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c81007c0 0x7f07c8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cfeff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07c81019b0 0x7f07c819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cfeff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07c8102370 0x7f07c819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cfeff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810c350 con 0x7f07c81019b0 2024-03-26T17:38:59.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cfeff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810c1d0 con 0x7f07c81007c0 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cfeff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810c4d0 con 0x7f07c8102370 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.010+0000 7f07cdc9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c81007c0 0x7f07c8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cdc9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c81007c0 0x7f07c8069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60650/0 (socket says 172.21.15.105:60650) 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cdc9b640 1 -- 172.21.15.105:0/2174183186 learned_addr learned my addr 172.21.15.105:0/2174183186 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cd49a640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07c81019b0 0x7f07c819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07ce49c640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07c8102370 0x7f07c819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cdc9b640 1 -- 172.21.15.105:0/2174183186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07c8102370 msgr2=0x7f07c819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cdc9b640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07c8102370 0x7f07c819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cdc9b640 1 -- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07c81019b0 msgr2=0x7f07c819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cdc9b640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07c81019b0 0x7f07c819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cdc9b640 1 -- 172.21.15.105:0/2174183186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c819f5c0 con 0x7f07c81007c0 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07ce49c640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07c8102370 0x7f07c819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cd49a640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07c81019b0 0x7f07c819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cdc9b640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c81007c0 0x7f07c8069590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07c400e9c0 tx=0x7f07c400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07beffd640 1 -- 172.21.15.105:0/2174183186 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c400ccc0 con 0x7f07c81007c0 2024-03-26T17:38:59.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cfeff640 1 -- 172.21.15.105:0/2174183186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c819f850 con 0x7f07c81007c0 2024-03-26T17:38:59.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07beffd640 1 -- 172.21.15.105:0/2174183186 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07c40035f0 con 0x7f07c81007c0 2024-03-26T17:38:59.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07beffd640 1 -- 172.21.15.105:0/2174183186 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c400ce60 con 0x7f07c81007c0 2024-03-26T17:38:59.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.011+0000 7f07cfeff640 1 -- 172.21.15.105:0/2174183186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c819fe10 con 0x7f07c81007c0 2024-03-26T17:38:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.014+0000 7f07beffd640 1 -- 172.21.15.105:0/2174183186 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f07c40037a0 con 0x7f07c81007c0 2024-03-26T17:38:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07d2bca740 1 Processor -- start 2024-03-26T17:38:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07d2bca740 1 -- start start 2024-03-26T17:38:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fde7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdeaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07d2bca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55707fd47970 con 0x55707fde8590 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07d2bca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55707fcefd70 con 0x55707fde2a30 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07d2bca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55707fd43920 con 0x55707fde3420 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07ce49c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07cdc9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fde7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07cd49a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdeaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07ce49c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60660/0 (socket says 172.21.15.105:60660) 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.015+0000 7f07ce49c640 1 -- 172.21.15.105:0/1034154557 learned_addr learned my addr 172.21.15.105:0/1034154557 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07ce49c640 1 -- 172.21.15.105:0/1034154557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 msgr2=0x55707fde7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fde7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07ce49c640 1 -- 172.21.15.105:0/1034154557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 msgr2=0x55707fdeaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdeaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07ce49c640 1 -- 172.21.15.105:0/1034154557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707fa5b3f0 con 0x55707fde2a30 2024-03-26T17:38:59.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07cdc9b640 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fde7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07cd49a640 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdeaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde2e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07c0036000 tx=0x7f07c0032cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58c685f63ff13e6a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07b3fff640 1 -- 172.21.15.105:0/1034154557 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0043020 con 0x55707fde2a30 2024-03-26T17:38:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07b3fff640 1 -- 172.21.15.105:0/1034154557 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07c0003e20 con 0x55707fde2a30 2024-03-26T17:38:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07b3fff640 1 -- 172.21.15.105:0/1034154557 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0039040 con 0x55707fde2a30 2024-03-26T17:38:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.016+0000 7f07beffd640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f07b40a3030 0x7f07b40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.017+0000 7f07cdc9b640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f07b40a3030 0x7f07b40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.017+0000 7f07beffd640 1 -- 172.21.15.105:0/2174183186 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f07c40cf9d0 con 0x7f07c81007c0 2024-03-26T17:38:59.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 -- 172.21.15.105:0/1034154557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 msgr2=0x55707fde2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07cdc9b640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f07b40a3030 0x7f07b40a54f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55707fde3e90 tx=0x7f07c40d34a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde2e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07c0036000 tx=0x7f07c0032cd0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 -- 172.21.15.105:0/1034154557 shutdown_connections 2024-03-26T17:38:59.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdeaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fde7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1034154557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde2e50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 -- 172.21.15.105:0/1034154557 >> 172.21.15.105:0/1034154557 conn(0x55707fdddfa0 msgr2=0x55707fdde3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 -- 172.21.15.105:0/1034154557 shutdown_connections 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 -- 172.21.15.105:0/1034154557 wait complete. 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 -- start start 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.018+0000 7f07d2bca740 1 Processor -- start 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07d2bca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55707fdef8f0 con 0x55707fde8590 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07d2bca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55707fd03310 con 0x55707fde2a30 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07d2bca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55707fdefa70 con 0x55707fde3420 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07ce49c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07ce49c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52164/0 (socket says 172.21.15.105:52164) 2024-03-26T17:38:59.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07ce49c640 1 -- 172.21.15.105:0/2093003909 learned_addr learned my addr 172.21.15.105:0/2093003909 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07ce49c640 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07cd49a640 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.019+0000 7f07d2bca740 10 client.0 _ll_register_callbacks cb 0x55707fd3ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07ce49c640 1 -- 172.21.15.105:0/2093003909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 msgr2=0x55707fdf18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07ce49c640 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07ce49c640 1 -- 172.21.15.105:0/2093003909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 msgr2=0x55707fdf1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07ce49c640 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07ce49c640 1 -- 172.21.15.105:0/2093003909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707fb257d0 con 0x55707fde8590 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07ce49c640 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf8870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f07c0033000 tx=0x7f07c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bd3b322c001276 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07b17fa640 1 -- 172.21.15.105:0/2093003909 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0043070 con 0x55707fde8590 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07b17fa640 1 -- 172.21.15.105:0/2093003909 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f07c003f470 con 0x55707fde8590 2024-03-26T17:38:59.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.020+0000 7f07d2bca740 1 -- 172.21.15.105:0/2093003909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c0003e20 con 0x55707fde8590 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.021+0000 7f07b17fa640 1 -- 172.21.15.105:0/2093003909 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07c0039040 con 0x55707fde8590 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.021+0000 7f07d2bca740 10 client.15591 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.021+0000 7f07d2bca740 1 -- 172.21.15.105:0/2093003909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55707fdfbba0 con 0x55707fde8590 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.021+0000 7f07b17fa640 1 -- 172.21.15.105:0/2093003909 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07c0003e20 con 0x55707fde8590 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.021+0000 7f07b17fa640 1 -- 172.21.15.105:0/2093003909 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f07c0034440 con 0x55707fde8590 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07d2bca740 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07d2bca740 10 client.15591 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07d2bca740 20 client.15591 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07d2bca740 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55707fdfcef0 0x55707fdff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07d2bca740 4 client.15591 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07d2bca740 1 -- 172.21.15.105:0/2093003909 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55707fd54eb0 con 0x55707fdfcef0 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07cd49a640 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55707fdfcef0 0x55707fdff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07cd49a640 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55707fdfcef0 0x55707fdff2e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8eb51868ebd92747 server_cookie=c1c2c26a2bcbc7e1 in_seq=0 out_seq=0 2024-03-26T17:38:59.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.022+0000 7f07b17fa640 10 client.15591 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07b17fa640 1 -- 172.21.15.105:0/2093003909 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55707fd54eb0 con 0x55707fdfcef0 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07b17fa640 10 client.15591 handle_command_reply: tid=0 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 client.15591 shutdown 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 -- 172.21.15.105:0/2093003909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 msgr2=0x55707fdf8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf8870 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f07c0033000 tx=0x7f07c0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 -- 172.21.15.105:0/2093003909 shutdown_connections 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf8870 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55707fdfcef0 0x55707fdff2e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 --2- 172.21.15.105:0/2093003909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07d2bca740 1 -- 172.21.15.105:0/2093003909 >> 172.21.15.105:0/2093003909 conn(0x55707fdddfa0 msgr2=0x55707fddd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.023+0000 7f07b17fa640 0 client.15591 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:38:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.024+0000 7f07d2bca740 1 -- 172.21.15.105:0/2093003909 shutdown_connections 2024-03-26T17:38:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.024+0000 7f07d2bca740 1 -- 172.21.15.105:0/2093003909 wait complete. 2024-03-26T17:38:59.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.024+0000 7f07d2bca740 20 client.15591 trim_cache size 0 max 16384 2024-03-26T17:38:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.041+0000 7f07d2bca740 1 Processor -- start 2024-03-26T17:38:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.041+0000 7f07d2bca740 1 -- start start 2024-03-26T17:38:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde3420 0x55707fde4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde8590 0x55707fdf8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07d2bca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a6980 con 0x55707fde3420 2024-03-26T17:38:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07d2bca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a84c0 con 0x55707fde2a30 2024-03-26T17:38:59.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07d2bca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a7050 con 0x55707fde8590 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60670/0 (socket says 172.21.15.105:60670) 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 -- 172.21.15.105:0/1463143256 learned_addr learned my addr 172.21.15.105:0/1463143256 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07cd49a640 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde3420 0x55707fde4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde8590 0x55707fdf8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 -- 172.21.15.105:0/1463143256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde8590 msgr2=0x55707fdf8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde8590 0x55707fdf8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 -- 172.21.15.105:0/1463143256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde3420 msgr2=0x55707fde4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde3420 0x55707fde4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 -- 172.21.15.105:0/1463143256 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707fdefbd0 con 0x55707fde2a30 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde4960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009860 tx=0x7f07c002c100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e7ff1f569ce4b0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07b37fe640 1 -- 172.21.15.105:0/1463143256 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0043070 con 0x55707fde2a30 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07b37fe640 1 -- 172.21.15.105:0/1463143256 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07c00043c0 con 0x55707fde2a30 2024-03-26T17:38:59.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.042+0000 7f07b37fe640 1 -- 172.21.15.105:0/1463143256 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c00041c0 con 0x55707fde2a30 2024-03-26T17:38:59.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 -- 172.21.15.105:0/1463143256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 msgr2=0x55707fde4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde4960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009860 tx=0x7f07c002c100 comp rx=0 tx=0).stop 2024-03-26T17:38:59.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 -- 172.21.15.105:0/1463143256 shutdown_connections 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde8590 0x55707fdf8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde3420 0x55707fde4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1463143256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fde4960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 -- 172.21.15.105:0/1463143256 >> 172.21.15.105:0/1463143256 conn(0x55707fdddfa0 msgr2=0x55707fdde380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 -- 172.21.15.105:0/1463143256 shutdown_connections 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.043+0000 7f07d2bca740 1 -- 172.21.15.105:0/1463143256 wait complete. 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 1 -- start start 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 1 Processor -- start 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07ce49c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a7df0 con 0x55707fde8590 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a7720 con 0x55707fde2a30 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07ce49c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60684/0 (socket says 172.21.15.105:60684) 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a4960 con 0x55707fde3420 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07ce49c640 1 -- 172.21.15.105:0/1313754607 learned_addr learned my addr 172.21.15.105:0/1313754607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07cd49a640 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.044+0000 7f07d2bca740 10 client.0 _ll_register_callbacks cb 0x7f07b8009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07cd49a640 1 -- 172.21.15.105:0/1313754607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 msgr2=0x55707fdf5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07cd49a640 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07cd49a640 1 -- 172.21.15.105:0/1313754607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 msgr2=0x55707fdf7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07cd49a640 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07cd49a640 1 -- 172.21.15.105:0/1313754607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707fdf7970 con 0x55707fde3420 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07cd49a640 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf6370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55707fdffa80 tx=0x7f07b8005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96947101fcbf1694 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07b17fa640 1 -- 172.21.15.105:0/1313754607 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b801fe60 con 0x55707fde3420 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07d2bca740 1 -- 172.21.15.105:0/1313754607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55707fd4f200 con 0x55707fde3420 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.045+0000 7f07b17fa640 1 -- 172.21.15.105:0/1313754607 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f07b8006720 con 0x55707fde3420 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07b17fa640 1 -- 172.21.15.105:0/1313754607 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07b802f050 con 0x55707fde3420 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07d2bca740 10 client.25456 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07d2bca740 1 -- 172.21.15.105:0/1313754607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55707fdeb1c0 con 0x55707fde3420 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07b17fa640 1 -- 172.21.15.105:0/1313754607 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55707fd4f200 con 0x55707fde3420 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07b17fa640 1 -- 172.21.15.105:0/1313754607 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f07b80148d0 con 0x55707fde3420 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07d2bca740 10 client.25456 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07d2bca740 10 client.25456 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:38:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07d2bca740 20 client.25456 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55707fdebb10 0x55707fdedf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07d2bca740 4 client.25456 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07d2bca740 1 -- 172.21.15.105:0/1313754607 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55707fccc6a0 con 0x55707fdebb10 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.046+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55707fdebb10 0x55707fdedf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07b17fa640 10 client.25456 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07ce49c640 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55707fdebb10 0x55707fdedf00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1536793901fc1ee server_cookie=887a7fff22ead43f in_seq=0 out_seq=0 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07b17fa640 1 -- 172.21.15.105:0/1313754607 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55707fccc6a0 con 0x55707fdebb10 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07b17fa640 10 client.25456 handle_command_reply: tid=0 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 client.25456 shutdown 2024-03-26T17:38:59.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 -- 172.21.15.105:0/1313754607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 msgr2=0x55707fdf6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf6370 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55707fdffa80 tx=0x7f07b8005d90 comp rx=0 tx=0).stop 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 -- 172.21.15.105:0/1313754607 shutdown_connections 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55707fde8590 0x55707fdf7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55707fde3420 0x55707fdf6370 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55707fdebb10 0x55707fdedf00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07b17fa640 0 client.25456 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 --2- 172.21.15.105:0/1313754607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55707fde2a30 0x55707fdf5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 -- 172.21.15.105:0/1313754607 >> 172.21.15.105:0/1313754607 conn(0x55707fdddfa0 msgr2=0x55707fde4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 -- 172.21.15.105:0/1313754607 shutdown_connections 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 1 -- 172.21.15.105:0/1313754607 wait complete. 2024-03-26T17:38:59.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.047+0000 7f07d2bca740 20 client.25456 trim_cache size 0 max 16384 2024-03-26T17:38:59.048 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:38:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 -- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f07b40a3030 msgr2=0x7f07b40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f07b40a3030 0x7f07b40a54f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55707fde3e90 tx=0x7f07c40d34a0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 -- 172.21.15.105:0/2174183186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c81007c0 msgr2=0x7f07c8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c81007c0 0x7f07c8069590 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07c400e9c0 tx=0x7f07c400edc0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 -- 172.21.15.105:0/2174183186 shutdown_connections 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f07b40a3030 0x7f07b40a54f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07c8102370 0x7f07c819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07c81019b0 0x7f07c819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 --2- 172.21.15.105:0/2174183186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07c81007c0 0x7f07c8069590 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07c400e9c0 tx=0x7f07c400edc0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 -- 172.21.15.105:0/2174183186 >> 172.21.15.105:0/2174183186 conn(0x7f07c80fbd50 msgr2=0x7f07c8104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 -- 172.21.15.105:0/2174183186 shutdown_connections 2024-03-26T17:38:59.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.048+0000 7f07cfeff640 1 -- 172.21.15.105:0/2174183186 wait complete. 2024-03-26T17:38:59.054 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:38:59.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:58 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2538394562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 -- 172.21.15.105:0/817916320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 msgr2=0x7fd638102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/817916320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 0x7fd638102f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009900 tx=0x7fd62c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 -- 172.21.15.105:0/817916320 shutdown_connections 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/817916320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6381034a0 0x7fd638109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/817916320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 0x7fd638102f60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/817916320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6381018f0 0x7fd638101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 -- 172.21.15.105:0/817916320 >> 172.21.15.105:0/817916320 conn(0x7fd6380fce60 msgr2=0x7fd6380ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 -- 172.21.15.105:0/817916320 shutdown_connections 2024-03-26T17:38:59.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.202+0000 7fd63e9cd640 1 -- 172.21.15.105:0/817916320 wait complete. 2024-03-26T17:38:59.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63e9cd640 1 Processor -- start 2024-03-26T17:38:59.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63e9cd640 1 -- start start 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63e9cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6381018f0 0x7fd638197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63e9cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 0x7fd638197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63e9cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6381034a0 0x7fd63819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63e9cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd63810d470 con 0x7fd6381034a0 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63e9cd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd63810d2f0 con 0x7fd6381018f0 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63e9cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd63810d5f0 con 0x7fd638102ae0 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd6377fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 0x7fd638197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd637fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6381018f0 0x7fd638197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd63cf6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6381034a0 0x7fd63819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd637fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6381018f0 0x7fd638197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60706/0 (socket says 172.21.15.105:60706) 2024-03-26T17:38:59.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.203+0000 7fd637fff640 1 -- 172.21.15.105:0/376224742 learned_addr learned my addr 172.21.15.105:0/376224742 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd637fff640 1 -- 172.21.15.105:0/376224742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 msgr2=0x7fd638197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd637fff640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 0x7fd638197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd637fff640 1 -- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6381034a0 msgr2=0x7fd63819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd637fff640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6381034a0 0x7fd63819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd637fff640 1 -- 172.21.15.105:0/376224742 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63819f5b0 con 0x7fd6381018f0 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd6377fe640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 0x7fd638197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd63cf6a640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6381034a0 0x7fd63819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd637fff640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6381018f0 0x7fd638197990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd62800e9c0 tx=0x7fd62800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd6357fa640 1 -- 172.21.15.105:0/376224742 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62800cd80 con 0x7fd6381018f0 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd63e9cd640 1 -- 172.21.15.105:0/376224742 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63819f8a0 con 0x7fd6381018f0 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.204+0000 7fd63e9cd640 1 -- 172.21.15.105:0/376224742 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63819fdc0 con 0x7fd6381018f0 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.205+0000 7fd6357fa640 1 -- 172.21.15.105:0/376224742 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6280035f0 con 0x7fd6381018f0 2024-03-26T17:38:59.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.205+0000 7fd6357fa640 1 -- 172.21.15.105:0/376224742 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628010690 con 0x7fd6381018f0 2024-03-26T17:38:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.207+0000 7fd6415ba740 1 Processor -- start 2024-03-26T17:38:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.207+0000 7fd6415ba740 1 -- start start 2024-03-26T17:38:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6357fa640 1 -- 172.21.15.105:0/376224742 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fd628010830 con 0x7fd6381018f0 2024-03-26T17:38:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a5a30 0x561a125a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a6420 0x561a125aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125ab590 0x561a125ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6415ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561a1250a970 con 0x561a125a6420 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6415ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561a124b2d70 con 0x561a125ab590 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6415ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a12506920 con 0x561a125a5a30 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a6420 0x561a125aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a6420 0x561a125aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52218/0 (socket says 172.21.15.105:52218) 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 -- 172.21.15.105:0/3155220882 learned_addr learned my addr 172.21.15.105:0/3155220882 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6377fe640 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125ab590 0x561a125ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 -- 172.21.15.105:0/3155220882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a5a30 msgr2=0x561a125a5e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a5a30 0x561a125a5e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 -- 172.21.15.105:0/3155220882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125ab590 msgr2=0x561a125ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125ab590 0x561a125ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 -- 172.21.15.105:0/3155220882 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1221e3f0 con 0x561a125a6420 2024-03-26T17:38:59.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6377fe640 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125ab590 0x561a125ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd637fff640 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a6420 0x561a125aaf20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd6280cd230 tx=0x7fd6280d5ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37e525ca6ac7d41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.208+0000 7fd6357fa640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6200a3030 0x7fd6200a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.209+0000 7fd6357fa640 1 -- 172.21.15.105:0/376224742 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fd6280cf980 con 0x7fd6381018f0 2024-03-26T17:38:59.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.209+0000 7fd63cf6a640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6200a3030 0x7fd6200a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.209+0000 7fd61e7fc640 1 -- 172.21.15.105:0/3155220882 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6280de070 con 0x561a125a6420 2024-03-26T17:38:59.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.209+0000 7fd61e7fc640 1 -- 172.21.15.105:0/3155220882 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd62809c610 con 0x561a125a6420 2024-03-26T17:38:59.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.209+0000 7fd61e7fc640 1 -- 172.21.15.105:0/3155220882 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62809bf10 con 0x561a125a6420 2024-03-26T17:38:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 -- 172.21.15.105:0/3155220882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a6420 msgr2=0x561a125aaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd63cf6a640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6200a3030 0x7fd6200a54f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x561a125a6c80 tx=0x7fd630009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a6420 0x561a125aaf20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd6280cd230 tx=0x7fd6280d5ef0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 -- 172.21.15.105:0/3155220882 shutdown_connections 2024-03-26T17:38:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125ab590 0x561a125ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a6420 0x561a125aaf20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3155220882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a5a30 0x561a125a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 -- 172.21.15.105:0/3155220882 >> 172.21.15.105:0/3155220882 conn(0x561a125a0fa0 msgr2=0x561a125a13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 -- 172.21.15.105:0/3155220882 shutdown_connections 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 -- 172.21.15.105:0/3155220882 wait complete. 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.210+0000 7fd6415ba740 1 -- start start 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6415ba740 1 Processor -- start 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 0x561a125bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6415ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561a125b28c0 con 0x561a125a5a30 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6415ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561a124c6310 con 0x561a125a6420 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6415ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a125b2a40 con 0x561a125ab590 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd637fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6377fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd637fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52220/0 (socket says 172.21.15.105:52220) 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6377fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60732/0 (socket says 172.21.15.105:60732) 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd637fff640 1 -- 172.21.15.105:0/4022861689 learned_addr learned my addr 172.21.15.105:0/4022861689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd63cf6a640 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 0x561a125bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6415ba740 10 client.0 _ll_register_callbacks cb 0x561a12501c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6377fe640 1 -- 172.21.15.105:0/4022861689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 msgr2=0x561a125bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6377fe640 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 0x561a125bb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6377fe640 1 -- 172.21.15.105:0/4022861689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 msgr2=0x561a125b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.211+0000 7fd6377fe640 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6377fe640 1 -- 172.21.15.105:0/4022861689 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a122e87d0 con 0x561a125a6420 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd637fff640 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6377fe640 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125b4900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd62c036040 tx=0x7fd62c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beab3acd3c98f8ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6177fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:59.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6177fe640 1 -- 172.21.15.105:0/4022861689 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c003800 con 0x561a125a6420 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6177fe640 1 -- 172.21.15.105:0/4022861689 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fd62c0039a0 con 0x561a125a6420 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6415ba740 1 -- 172.21.15.105:0/4022861689 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62809c610 con 0x561a125a6420 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6177fe640 1 -- 172.21.15.105:0/4022861689 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd62c003ba0 con 0x561a125a6420 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6415ba740 10 client.25472 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6415ba740 1 -- 172.21.15.105:0/4022861689 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a125becc0 con 0x561a125a6420 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.212+0000 7fd6177fe640 1 -- 172.21.15.105:0/4022861689 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd62809c610 con 0x561a125a6420 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd6177fe640 1 -- 172.21.15.105:0/4022861689 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd62c040420 con 0x561a125a6420 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd6415ba740 10 client.25472 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd6415ba740 10 client.25472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd6415ba740 20 client.25472 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd6415ba740 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561a125c0340 0x561a125c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd6415ba740 4 client.25472 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:59.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd6415ba740 1 -- 172.21.15.105:0/4022861689 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a12517eb0 con 0x561a125c0340 2024-03-26T17:38:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd637fff640 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561a125c0340 0x561a125c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd637fff640 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561a125c0340 0x561a125c27a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df2958d2aa6b1b64 server_cookie=d7b5e366edf4de87 in_seq=0 out_seq=0 2024-03-26T17:38:59.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.213+0000 7fd6177fe640 10 client.25472 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.214+0000 7fd6177fe640 1 -- 172.21.15.105:0/4022861689 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561a12517eb0 con 0x561a125c0340 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.214+0000 7fd6177fe640 10 client.25472 handle_command_reply: tid=0 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.214+0000 7fd6415ba740 1 client.25472 shutdown 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 -- 172.21.15.105:0/4022861689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 msgr2=0x561a125b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125b4900 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd62c036040 tx=0x7fd62c0045e0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 -- 172.21.15.105:0/4022861689 shutdown_connections 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561a125c0340 0x561a125c27a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 0x561a125bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125b4900 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 --2- 172.21.15.105:0/4022861689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 -- 172.21.15.105:0/4022861689 >> 172.21.15.105:0/4022861689 conn(0x561a125a0fa0 msgr2=0x561a125a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6177fe640 0 client.25472 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:38:59.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 -- 172.21.15.105:0/4022861689 shutdown_connections 2024-03-26T17:38:59.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 1 -- 172.21.15.105:0/4022861689 wait complete. 2024-03-26T17:38:59.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.215+0000 7fd6415ba740 20 client.25472 trim_cache size 0 max 16384 2024-03-26T17:38:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.240+0000 7fd6415ba740 1 Processor -- start 2024-03-26T17:38:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.240+0000 7fd6415ba740 1 -- start start 2024-03-26T17:38:59.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.240+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 0x561a125a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd6415ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd62400e520 con 0x561a125a5a30 2024-03-26T17:38:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd6415ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd624010060 con 0x561a125a6420 2024-03-26T17:38:59.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd6415ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd62400ebf0 con 0x561a125ab590 2024-03-26T17:38:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125a7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52228/0 (socket says 172.21.15.105:52228) 2024-03-26T17:38:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 -- 172.21.15.105:0/754286483 learned_addr learned my addr 172.21.15.105:0/754286483 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd63cf6a640 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 0x561a125a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd6377fe640 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 -- 172.21.15.105:0/754286483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 msgr2=0x561a125a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 0x561a125a7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 -- 172.21.15.105:0/754286483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 msgr2=0x561a125bb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125bb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 -- 172.21.15.105:0/754286483 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a125adde0 con 0x561a125a5a30 2024-03-26T17:38:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd6377fe640 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:38:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.241+0000 7fd637fff640 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125a7920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd62809bea0 tx=0x7fd62809ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdcd763780ff8a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.242+0000 7fd61dffb640 1 -- 172.21.15.105:0/754286483 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6280de070 con 0x561a125a5a30 2024-03-26T17:38:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.242+0000 7fd61dffb640 1 -- 172.21.15.105:0/754286483 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd62809c610 con 0x561a125a5a30 2024-03-26T17:38:59.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.242+0000 7fd61dffb640 1 -- 172.21.15.105:0/754286483 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62809bf10 con 0x561a125a5a30 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 -- 172.21.15.105:0/754286483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 msgr2=0x561a125a7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125a7920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd62809bea0 tx=0x7fd62809ba50 comp rx=0 tx=0).stop 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 -- 172.21.15.105:0/754286483 shutdown_connections 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125ab590 0x561a125a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a6420 0x561a125bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 --2- 172.21.15.105:0/754286483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125a5a30 0x561a125a7920 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 -- 172.21.15.105:0/754286483 >> 172.21.15.105:0/754286483 conn(0x561a125a0fa0 msgr2=0x561a125b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 -- 172.21.15.105:0/754286483 shutdown_connections 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 -- 172.21.15.105:0/754286483 wait complete. 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 -- start start 2024-03-26T17:38:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.243+0000 7fd6415ba740 1 Processor -- start 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a5a30 0x561a125b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a6420 0x561a125b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6415ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125ab590 0x561a125bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6415ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd62400f990 con 0x561a125ab590 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6415ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd62400f2c0 con 0x561a125a5a30 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd637fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a5a30 0x561a125b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd637fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a5a30 0x561a125b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60750/0 (socket says 172.21.15.105:60750) 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd637fff640 1 -- 172.21.15.105:0/3705131277 learned_addr learned my addr 172.21.15.105:0/3705131277 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd62400c500 con 0x561a125a6420 2024-03-26T17:38:59.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6377fe640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a6420 0x561a125b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd63cf6a640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125ab590 0x561a125bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6415ba740 10 client.0 _ll_register_callbacks cb 0x7fd62c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:59.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6377fe640 1 -- 172.21.15.105:0/3705131277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a5a30 msgr2=0x561a125b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6377fe640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a5a30 0x561a125b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6377fe640 1 -- 172.21.15.105:0/3705131277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125ab590 msgr2=0x561a125bb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6377fe640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125ab590 0x561a125bb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd6377fe640 1 -- 172.21.15.105:0/3705131277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a126316c0 con 0x561a125a6420 2024-03-26T17:38:59.242 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:38:59.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd637fff640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a5a30 0x561a125b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.244+0000 7fd63cf6a640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125ab590 0x561a125bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6377fe640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a6420 0x561a125b16b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd62c040890 tx=0x7fd62c004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a850574fc3f3225 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6177fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6177fe640 1 -- 172.21.15.105:0/3705131277 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c009520 con 0x561a125a6420 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6177fe640 1 -- 172.21.15.105:0/3705131277 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fd62c040a80 con 0x561a125a6420 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62809c610 con 0x561a125a6420 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6177fe640 1 -- 172.21.15.105:0/3705131277 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd62c040ca0 con 0x561a125a6420 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6415ba740 10 client.25471 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561a12643910 con 0x561a125a6420 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6177fe640 1 -- 172.21.15.105:0/3705131277 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd62809c610 con 0x561a125a6420 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.245+0000 7fd6177fe640 1 -- 172.21.15.105:0/3705131277 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd62c05d030 con 0x561a125a6420 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6415ba740 10 client.25471 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6415ba740 10 client.25471 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6415ba740 20 client.25471 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561a126442b0 0x561a12646730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6415ba740 4 client.25471 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a125b41b0 con 0x561a126442b0 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd637fff640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561a126442b0 0x561a12646730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd637fff640 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561a126442b0 0x561a12646730 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84584baff043442c server_cookie=32c4a7314e6a326 in_seq=0 out_seq=0 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6177fe640 10 client.25471 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6177fe640 1 -- 172.21.15.105:0/3705131277 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a125b41b0 con 0x561a126442b0 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6177fe640 10 client.25471 handle_command_reply: tid=0 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.246+0000 7fd6415ba740 1 client.25471 shutdown 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a6420 msgr2=0x561a125b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a6420 0x561a125b16b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd62c040890 tx=0x7fd62c004250 comp rx=0 tx=0).stop 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 shutdown_connections 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561a126442b0 0x561a12646730 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561a125ab590 0x561a125bb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a125a6420 0x561a125b16b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 --2- 172.21.15.105:0/3705131277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a125a5a30 0x561a125b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 >> 172.21.15.105:0/3705131277 conn(0x561a125a0fa0 msgr2=0x561a125a7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6177fe640 0 client.25471 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 shutdown_connections 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 1 -- 172.21.15.105:0/3705131277 wait complete. 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd6415ba740 20 client.25471 trim_cache size 0 max 16384 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd63e9cd640 1 -- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6200a3030 msgr2=0x7fd6200a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6200a3030 0x7fd6200a54f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x561a125a6c80 tx=0x7fd630009500 comp rx=0 tx=0).stop 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd63e9cd640 1 -- 172.21.15.105:0/376224742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6381018f0 msgr2=0x7fd638197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.247+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6381018f0 0x7fd638197990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd62800e9c0 tx=0x7fd62800ee80 comp rx=0 tx=0).stop 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.248+0000 7fd63e9cd640 1 -- 172.21.15.105:0/376224742 shutdown_connections 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.248+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6200a3030 0x7fd6200a54f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.248+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6381034a0 0x7fd63819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.248+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd638102ae0 0x7fd638197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.248+0000 7fd63e9cd640 1 --2- 172.21.15.105:0/376224742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6381018f0 0x7fd638197990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.248+0000 7fd63e9cd640 1 -- 172.21.15.105:0/376224742 >> 172.21.15.105:0/376224742 conn(0x7fd6380fce60 msgr2=0x7fd638105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.248+0000 7fd63e9cd640 1 -- 172.21.15.105:0/376224742 shutdown_connections 2024-03-26T17:38:59.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.248+0000 7fd63e9cd640 1 -- 172.21.15.105:0/376224742 wait complete. 2024-03-26T17:38:59.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:38:59.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.376+0000 7fbc893c4640 1 -- 172.21.15.105:0/3534966415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 msgr2=0x7fbc84100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.376+0000 7fbc893c4640 1 --2- 172.21.15.105:0/3534966415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 0x7fbc84100550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009860 tx=0x7fbc7402bb70 comp rx=0 tx=0).stop 2024-03-26T17:38:59.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 -- 172.21.15.105:0/3534966415 shutdown_connections 2024-03-26T17:38:59.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 --2- 172.21.15.105:0/3534966415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc84101430 0x7fbc841018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 --2- 172.21.15.105:0/3534966415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84100a90 0x7fbc84100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 --2- 172.21.15.105:0/3534966415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 0x7fbc84100550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 -- 172.21.15.105:0/3534966415 >> 172.21.15.105:0/3534966415 conn(0x7fbc840fbd50 msgr2=0x7fbc840fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 -- 172.21.15.105:0/3534966415 shutdown_connections 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 -- 172.21.15.105:0/3534966415 wait complete. 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 Processor -- start 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.377+0000 7fbc893c4640 1 -- start start 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc893c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 0x7fbc84199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc893c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc84100a90 0x7fbc8419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc893c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84101430 0x7fbc841a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc893c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410f5b0 con 0x7fbc84100a90 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc893c4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410f430 con 0x7fbc84100130 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc893c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8410f730 con 0x7fbc84101430 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc82ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 0x7fbc84199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84101430 0x7fbc841a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc82ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 0x7fbc84199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60770/0 (socket says 172.21.15.105:60770) 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84101430 0x7fbc841a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55524/0 (socket says 172.21.15.105:55524) 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc82ffd640 1 -- 172.21.15.105:0/2126811015 learned_addr learned my addr 172.21.15.105:0/2126811015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc84100a90 0x7fbc8419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc837fe640 1 -- 172.21.15.105:0/2126811015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 msgr2=0x7fbc84199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc837fe640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 0x7fbc84199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc837fe640 1 -- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc84100a90 msgr2=0x7fbc8419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc837fe640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc84100a90 0x7fbc8419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc837fe640 1 -- 172.21.15.105:0/2126811015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc841a1920 con 0x7fbc84101430 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.378+0000 7fbc837fe640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84101430 0x7fbc841a1280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009fd0 tx=0x7fbc7c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.379+0000 7fbc80ff9640 1 -- 172.21.15.105:0/2126811015 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00cd60 con 0x7fbc84101430 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.379+0000 7fbc80ff9640 1 -- 172.21.15.105:0/2126811015 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc7c005020 con 0x7fbc84101430 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.379+0000 7fbc893c4640 1 -- 172.21.15.105:0/2126811015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc841023a0 con 0x7fbc84101430 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.380+0000 7fbc80ff9640 1 -- 172.21.15.105:0/2126811015 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c010690 con 0x7fbc84101430 2024-03-26T17:38:59.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.380+0000 7fbc893c4640 1 -- 172.21.15.105:0/2126811015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc84102890 con 0x7fbc84101430 2024-03-26T17:38:59.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.382+0000 7fbc80ff9640 1 -- 172.21.15.105:0/2126811015 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fbc7c0051d0 con 0x7fbc84101430 2024-03-26T17:38:59.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc8bfad740 1 Processor -- start 2024-03-26T17:38:59.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc8bfad740 1 -- start start 2024-03-26T17:38:59.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f520a30 0x55581f520e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f525f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f528a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55581f485970 con 0x55581f520a30 2024-03-26T17:38:59.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55581f42dd70 con 0x55581f521420 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55581f481920 con 0x55581f526590 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.383+0000 7fbc80ff9640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc700a2f10 0x7fbc700a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc82ffd640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc700a2f10 0x7fbc700a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f528a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f520a30 0x55581f520e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f520a30 0x55581f520e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52274/0 (socket says 172.21.15.105:52274) 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 -- 172.21.15.105:0/2397272689 learned_addr learned my addr 172.21.15.105:0/2397272689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/2397272689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f525f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 -- 172.21.15.105:0/2397272689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 msgr2=0x55581f528a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/2397272689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f528a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 -- 172.21.15.105:0/2397272689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 msgr2=0x55581f525f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/2397272689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f525f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 -- 172.21.15.105:0/2397272689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581f1993f0 con 0x55581f520a30 2024-03-26T17:38:59.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc80ff9640 1 -- 172.21.15.105:0/2126811015 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fbc7c0cf850 con 0x7fbc84101430 2024-03-26T17:38:59.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc82ffd640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc700a2f10 0x7fbc700a53d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbc740095f0 tx=0x7fbc74034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.384+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/2397272689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f520a30 0x55581f520e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55581f521e90 tx=0x7fbc6c0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364e90cee6d083d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.385+0000 7fbc797fa640 1 -- 172.21.15.105:0/2397272689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c00f790 con 0x55581f520a30 2024-03-26T17:38:59.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.385+0000 7fbc797fa640 1 -- 172.21.15.105:0/2397272689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc6c00fde0 con 0x55581f520a30 2024-03-26T17:38:59.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.385+0000 7fbc797fa640 1 -- 172.21.15.105:0/2397272689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c01fd10 con 0x55581f520a30 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2397272689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f520a30 msgr2=0x55581f520e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2397272689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f520a30 0x55581f520e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55581f521e90 tx=0x7fbc6c0097e0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2397272689 shutdown_connections 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2397272689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f528a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2397272689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f525f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2397272689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f520a30 0x55581f520e50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2397272689 >> 172.21.15.105:0/2397272689 conn(0x55581f51bfa0 msgr2=0x55581f51c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2397272689 shutdown_connections 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.386+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2397272689 wait complete. 2024-03-26T17:38:59.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.387+0000 7fbc8bfad740 1 -- start start 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.387+0000 7fbc8bfad740 1 Processor -- start 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.387+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f520a30 0x55581f52af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f52b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f526590 0x55581f5324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55581f5294e0 con 0x55581f526590 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55581f441310 con 0x55581f521420 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f52b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f52b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60780/0 (socket says 172.21.15.105:60780) 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc7bfff640 1 -- 172.21.15.105:0/2389723345 learned_addr learned my addr 172.21.15.105:0/2389723345 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc837fe640 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f526590 0x55581f5324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc82ffd640 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f520a30 0x55581f52af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55581f529660 con 0x55581f520a30 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc8bfad740 10 client.0 _ll_register_callbacks cb 0x55581f47cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc7bfff640 1 -- 172.21.15.105:0/2389723345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f520a30 msgr2=0x55581f52af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f520a30 0x55581f52af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc7bfff640 1 -- 172.21.15.105:0/2389723345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f526590 msgr2=0x55581f5324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f526590 0x55581f5324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.388+0000 7fbc7bfff640 1 -- 172.21.15.105:0/2389723345 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581f2637d0 con 0x55581f521420 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.389+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f52b470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0099a0 tx=0x7fbc6c009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45765aaca49dc936 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.389+0000 7fbc6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.389+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/2389723345 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c020e70 con 0x55581f521420 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.389+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc6c00fde0 con 0x55581f521420 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.389+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/2389723345 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fbc6c00f790 con 0x55581f521420 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.389+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/2389723345 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c00f990 con 0x55581f521420 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.389+0000 7fbc8bfad740 10 client.25490 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.389+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55581f535750 con 0x55581f521420 2024-03-26T17:38:59.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/2389723345 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc6c00fde0 con 0x55581f521420 2024-03-26T17:38:59.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/2389723345 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbc6c031030 con 0x55581f521420 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc8bfad740 10 client.25490 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc8bfad740 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc8bfad740 20 client.25490 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55581f536d10 0x55581f539140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc8bfad740 4 client.25490 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55581f492eb0 con 0x55581f536d10 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.390+0000 7fbc82ffd640 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55581f536d10 0x55581f539140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.391+0000 7fbc6a7fc640 10 client.25490 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.391+0000 7fbc82ffd640 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55581f536d10 0x55581f539140 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831f657077fe3610 server_cookie=cb7191436c2ce92f in_seq=0 out_seq=0 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.391+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/2389723345 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55581f492eb0 con 0x55581f536d10 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.391+0000 7fbc6a7fc640 10 client.25490 handle_command_reply: tid=0 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 client.25490 shutdown 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 msgr2=0x55581f52b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f52b470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0099a0 tx=0x7fbc6c009e70 comp rx=0 tx=0).stop 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 shutdown_connections 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55581f536d10 0x55581f539140 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f526590 0x55581f5324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f521420 0x55581f52b470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/2389723345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f520a30 0x55581f52af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 >> 172.21.15.105:0/2389723345 conn(0x55581f51bfa0 msgr2=0x55581f51b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc6a7fc640 0 client.25490 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 shutdown_connections 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 1 -- 172.21.15.105:0/2389723345 wait complete. 2024-03-26T17:38:59.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.392+0000 7fbc8bfad740 20 client.25490 trim_cache size 0 max 16384 2024-03-26T17:38:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.407+0000 7fbc8bfad740 1 Processor -- start 2024-03-26T17:38:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.407+0000 7fbc8bfad740 1 -- start start 2024-03-26T17:38:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.407+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f529980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.407+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f532470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.407+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f529ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a6a20 con 0x55581f521420 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc82ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f529980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc82ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f529980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60796/0 (socket says 172.21.15.105:60796) 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc82ffd640 1 -- 172.21.15.105:0/1121170323 learned_addr learned my addr 172.21.15.105:0/1121170323 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc8bfad740 1 -- 172.21.15.105:0/1121170323 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a8560 con 0x55581f520a30 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc8bfad740 1 -- 172.21.15.105:0/1121170323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a70f0 con 0x55581f526590 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f532470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc837fe640 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f529ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc837fe640 1 -- 172.21.15.105:0/1121170323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 msgr2=0x55581f529980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc837fe640 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f529980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc837fe640 1 -- 172.21.15.105:0/1121170323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 msgr2=0x55581f532470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc837fe640 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f532470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc837fe640 1 -- 172.21.15.105:0/1121170323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581f51ca10 con 0x55581f526590 2024-03-26T17:38:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.408+0000 7fbc837fe640 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f529ec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0cd1c0 tx=0x7fbc7c00c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d2708a7c6e21e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.409+0000 7fbc78ff9640 1 -- 172.21.15.105:0/1121170323 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0b7070 con 0x55581f526590 2024-03-26T17:38:59.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.409+0000 7fbc78ff9640 1 -- 172.21.15.105:0/1121170323 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc7c09f040 con 0x55581f526590 2024-03-26T17:38:59.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.409+0000 7fbc78ff9640 1 -- 172.21.15.105:0/1121170323 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0a4360 con 0x55581f526590 2024-03-26T17:38:59.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 -- 172.21.15.105:0/1121170323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 msgr2=0x55581f529ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f529ec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0cd1c0 tx=0x7fbc7c00c7d0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 -- 172.21.15.105:0/1121170323 shutdown_connections 2024-03-26T17:38:59.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f529ec0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f532470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/1121170323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f529980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 -- 172.21.15.105:0/1121170323 >> 172.21.15.105:0/1121170323 conn(0x55581f51bfa0 msgr2=0x55581f5229f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 -- 172.21.15.105:0/1121170323 shutdown_connections 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 -- 172.21.15.105:0/1121170323 wait complete. 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 -- start start 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.410+0000 7fbc8bfad740 1 Processor -- start 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f52eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f52f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc8bfad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f59b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a7e90 con 0x55581f521420 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a77c0 con 0x55581f520a30 2024-03-26T17:38:59.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc8bfad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc841a4a00 con 0x55581f526590 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc8bfad740 10 client.0 _ll_register_callbacks cb 0x7fbc6c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f52eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f52f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f52eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60812/0 (socket says 172.21.15.105:60812) 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f52f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52300/0 (socket says 172.21.15.105:52300) 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 -- 172.21.15.105:0/4131433826 learned_addr learned my addr 172.21.15.105:0/4131433826 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc837fe640 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f59b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 -- 172.21.15.105:0/4131433826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 msgr2=0x55581f59b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f59b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 -- 172.21.15.105:0/4131433826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 msgr2=0x55581f52f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f52f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 -- 172.21.15.105:0/4131433826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581f532600 con 0x55581f520a30 2024-03-26T17:38:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f52f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.411+0000 7fbc82ffd640 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f52eff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc74000c00 tx=0x7fbc74005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33060f89d9b4d387 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/4131433826 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74057040 con 0x55581f520a30 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/4131433826 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fbc74004440 con 0x55581f520a30 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc8bfad740 1 -- 172.21.15.105:0/4131433826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc7c09f040 con 0x55581f520a30 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/4131433826 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbc74039040 con 0x55581f520a30 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc8bfad740 10 client.25496 fetch_fsmap learned FSMap version 27 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc8bfad740 1 -- 172.21.15.105:0/4131433826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55581f59c530 con 0x55581f520a30 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/4131433826 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc7c09f040 con 0x55581f520a30 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.412+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/4131433826 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbc74062130 con 0x55581f520a30 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 10 client.25496 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 10 client.25496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:38:59.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 20 client.25496 populate_metadata read hostname 'smithi105' 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55581f59ce60 0x55581f59f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 4 client.25496 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 1 -- 172.21.15.105:0/4131433826 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55581f52ad20 con 0x55581f59ce60 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55581f59ce60 0x55581f59f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc7bfff640 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55581f59ce60 0x55581f59f280 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb36e18fb34cb0da server_cookie=656d788056500548 in_seq=0 out_seq=0 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc6a7fc640 10 client.25496 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc6a7fc640 1 -- 172.21.15.105:0/4131433826 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55581f52ad20 con 0x55581f59ce60 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc6a7fc640 10 client.25496 handle_command_reply: tid=0 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 1 client.25496 shutdown 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 1 -- 172.21.15.105:0/4131433826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 msgr2=0x55581f52eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.413+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f52eff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc74000c00 tx=0x7fbc74005f90 comp rx=0 tx=0).stop 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 1 -- 172.21.15.105:0/4131433826 shutdown_connections 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55581f59ce60 0x55581f59f280 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55581f526590 0x55581f59b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55581f521420 0x55581f52f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 1 --2- 172.21.15.105:0/4131433826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55581f520a30 0x55581f52eff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc6a7fc640 0 client.25496 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:38:59.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 1 -- 172.21.15.105:0/4131433826 >> 172.21.15.105:0/4131433826 conn(0x55581f51bfa0 msgr2=0x55581f5229f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 1 -- 172.21.15.105:0/4131433826 shutdown_connections 2024-03-26T17:38:59.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 1 -- 172.21.15.105:0/4131433826 wait complete. 2024-03-26T17:38:59.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc8bfad740 20 client.25496 trim_cache size 0 max 16384 2024-03-26T17:38:59.413 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:38:59.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 -- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc700a2f10 msgr2=0x7fbc700a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc700a2f10 0x7fbc700a53d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbc740095f0 tx=0x7fbc74034980 comp rx=0 tx=0).stop 2024-03-26T17:38:59.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 -- 172.21.15.105:0/2126811015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84101430 msgr2=0x7fbc841a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:38:59.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84101430 0x7fbc841a1280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009fd0 tx=0x7fbc7c00ee50 comp rx=0 tx=0).stop 2024-03-26T17:38:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 -- 172.21.15.105:0/2126811015 shutdown_connections 2024-03-26T17:38:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc700a2f10 0x7fbc700a53d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc84101430 0x7fbc841a1280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc84100a90 0x7fbc8419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 --2- 172.21.15.105:0/2126811015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc84100130 0x7fbc84199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:38:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.414+0000 7fbc893c4640 1 -- 172.21.15.105:0/2126811015 >> 172.21.15.105:0/2126811015 conn(0x7fbc840fbd50 msgr2=0x7fbc840fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:38:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.415+0000 7fbc893c4640 1 -- 172.21.15.105:0/2126811015 shutdown_connections 2024-03-26T17:38:59.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:38:59.415+0000 7fbc893c4640 1 -- 172.21.15.105:0/2126811015 wait complete. 2024-03-26T17:38:59.640 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:38:59 smithi105 ceph-mon[32280]: pgmap v365: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:38:59.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:38:59 smithi135 ceph-mon[35636]: pgmap v365: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:00.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:38:59 smithi144 ceph-mon[36095]: pgmap v365: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:00.420 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:39:00.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:39:00.421 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:39:00.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 -- 172.21.15.105:0/1105110123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 msgr2=0x7f0400100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 --2- 172.21.15.105:0/1105110123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 0x7f0400100be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009900 tx=0x7f03f002bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:00.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 -- 172.21.15.105:0/1105110123 shutdown_connections 2024-03-26T17:39:00.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 --2- 172.21.15.105:0/1105110123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0400102370 0x7f0400108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 --2- 172.21.15.105:0/1105110123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04001019b0 0x7f0400101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 --2- 172.21.15.105:0/1105110123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 0x7f0400100be0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 -- 172.21.15.105:0/1105110123 >> 172.21.15.105:0/1105110123 conn(0x7f04000fbd50 msgr2=0x7f04000fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 -- 172.21.15.105:0/1105110123 shutdown_connections 2024-03-26T17:39:00.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.558+0000 7f0405e34640 1 -- 172.21.15.105:0/1105110123 wait complete. 2024-03-26T17:39:00.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.559+0000 7f0405e34640 1 Processor -- start 2024-03-26T17:39:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.559+0000 7f0405e34640 1 -- start start 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.559+0000 7f0405e34640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 0x7f04000695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.559+0000 7f0405e34640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04001019b0 0x7f040019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.559+0000 7f0405e34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0400102370 0x7f040019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.559+0000 7f0405e34640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f040010c310 con 0x7f0400102370 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.559+0000 7f0405e34640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f040010c190 con 0x7f04001007c0 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.559+0000 7f0405e34640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f040010c490 con 0x7f04001019b0 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 0x7f04000695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 0x7f04000695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60828/0 (socket says 172.21.15.105:60828) 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03ff7fe640 1 -- 172.21.15.105:0/1706825337 learned_addr learned my addr 172.21.15.105:0/1706825337 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03feffd640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04001019b0 0x7f040019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03fffff640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0400102370 0x7f040019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03feffd640 1 -- 172.21.15.105:0/1706825337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 msgr2=0x7f04000695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03feffd640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 0x7f04000695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03feffd640 1 -- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0400102370 msgr2=0x7f040019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03feffd640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0400102370 0x7f040019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03feffd640 1 -- 172.21.15.105:0/1706825337 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040019f770 con 0x7f04001019b0 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 0x7f04000695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03feffd640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04001019b0 0x7f040019c840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03f400e9f0 tx=0x7f03f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.560+0000 7f03fcff9640 1 -- 172.21.15.105:0/1706825337 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f400ccc0 con 0x7f04001019b0 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.561+0000 7f03fcff9640 1 -- 172.21.15.105:0/1706825337 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03f4005020 con 0x7f04001019b0 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.561+0000 7f0405e34640 1 -- 172.21.15.105:0/1706825337 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040019fa60 con 0x7f04001019b0 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.561+0000 7f03fcff9640 1 -- 172.21.15.105:0/1706825337 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4004a00 con 0x7f04001019b0 2024-03-26T17:39:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.561+0000 7f0405e34640 1 -- 172.21.15.105:0/1706825337 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040019ffb0 con 0x7f04001019b0 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.563+0000 7f0408a19740 1 Processor -- start 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f03fcff9640 1 -- 172.21.15.105:0/1706825337 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f03f4004ba0 con 0x7f04001019b0 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f0408a19740 1 -- start start 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f0408a19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefb03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f0408a19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefb4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f0408a19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 0x558feefb79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f0408a19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558feef0c630 con 0x558feefb0980 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f0408a19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558feeeb4e10 con 0x558feefaff90 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f0408a19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558feef085e0 con 0x558feefb5580 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f03fffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefb03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f03feffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 0x558feefb79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f03fffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefb03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60836/0 (socket says 172.21.15.105:60836) 2024-03-26T17:39:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f03fffff640 1 -- 172.21.15.105:0/660633718 learned_addr learned my addr 172.21.15.105:0/660633718 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.564+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/660633718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefb4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03ff7fe640 1 -- 172.21.15.105:0/660633718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 msgr2=0x558feefb79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/660633718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 0x558feefb79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03ff7fe640 1 -- 172.21.15.105:0/660633718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 msgr2=0x558feefb03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/660633718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefb03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03ff7fe640 1 -- 172.21.15.105:0/660633718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558feeecb740 con 0x558feefb0980 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03fcff9640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03e80a3030 0x7f03e80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/660633718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefb4f10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03f0006010 tx=0x7f03f0009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eec2c486af1ef6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03fcff9640 1 -- 172.21.15.105:0/1706825337 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f03f40cf4b0 con 0x7f04001019b0 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03fffff640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03e80a3030 0x7f03e80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03e5ffb640 1 -- 172.21.15.105:0/660633718 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0045020 con 0x558feefb0980 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03e5ffb640 1 -- 172.21.15.105:0/660633718 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03f002cc90 con 0x558feefb0980 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.565+0000 7f03e5ffb640 1 -- 172.21.15.105:0/660633718 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f00048e0 con 0x558feefb0980 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 -- 172.21.15.105:0/660633718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 msgr2=0x558feefb4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f03fffff640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03e80a3030 0x7f03e80a54f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x558feefa6340 tx=0x7f03f8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 --2- 172.21.15.105:0/660633718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefb4f10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f03f0006010 tx=0x7f03f0009660 comp rx=0 tx=0).stop 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 -- 172.21.15.105:0/660633718 shutdown_connections 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 --2- 172.21.15.105:0/660633718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 0x558feefb79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 --2- 172.21.15.105:0/660633718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefb4f10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 --2- 172.21.15.105:0/660633718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefb03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 -- 172.21.15.105:0/660633718 >> 172.21.15.105:0/660633718 conn(0x558feefa2680 msgr2=0x558feefa2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 -- 172.21.15.105:0/660633718 shutdown_connections 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.566+0000 7f0408a19740 1 -- 172.21.15.105:0/660633718 wait complete. 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 1 -- start start 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 1 Processor -- start 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefb9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 0x558feefc1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558feefb84b0 con 0x558feefaff90 2024-03-26T17:39:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558feeec81a0 con 0x558feefb5580 2024-03-26T17:39:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558feefb8630 con 0x558feefb0980 2024-03-26T17:39:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f0408a19740 10 client.0 _ll_register_callbacks cb 0x558feec1fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f03feffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f03feffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefba210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55572/0 (socket says 172.21.15.105:55572) 2024-03-26T17:39:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f03feffd640 1 -- 172.21.15.105:0/3835920123 learned_addr learned my addr 172.21.15.105:0/3835920123 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.567+0000 7f03fffff640 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 0x558feefc1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefb9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03feffd640 1 -- 172.21.15.105:0/3835920123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 msgr2=0x558feefc1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03feffd640 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 0x558feefc1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03feffd640 1 -- 172.21.15.105:0/3835920123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 msgr2=0x558feefb9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03feffd640 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefb9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03feffd640 1 -- 172.21.15.105:0/3835920123 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558feece87e0 con 0x558feefb0980 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefb9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03feffd640 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefba210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f03f4084fd0 tx=0x7f03f409b580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4232c5371d997166 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03deffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03deffd640 1 -- 172.21.15.105:0/3835920123 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f401c070 con 0x558feefb0980 2024-03-26T17:39:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f0408a19740 1 -- 172.21.15.105:0/3835920123 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f002cc90 con 0x558feefb0980 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03deffd640 1 -- 172.21.15.105:0/3835920123 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f03f40051d0 con 0x558feefb0980 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03deffd640 1 -- 172.21.15.105:0/3835920123 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03f409c140 con 0x558feefb0980 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f0408a19740 10 client.25501 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f0408a19740 1 -- 172.21.15.105:0/3835920123 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558feed48f70 con 0x558feefb0980 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.568+0000 7f03deffd640 1 -- 172.21.15.105:0/3835920123 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03f002cc90 con 0x558feefb0980 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f03deffd640 1 -- 172.21.15.105:0/3835920123 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03f40cf0d0 con 0x558feefb0980 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f0408a19740 10 client.25501 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f0408a19740 10 client.25501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f0408a19740 20 client.25501 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f0408a19740 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558feefc5a70 0x558feefc7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f0408a19740 4 client.25501 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f0408a19740 1 -- 172.21.15.105:0/3835920123 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558feef19b70 con 0x558feefc5a70 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558feefc5a70 0x558feefc7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f03deffd640 10 client.25501 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.569+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558feefc5a70 0x558feefc7f10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7beb61d5451c4fa server_cookie=bb881a33dac2fc95 in_seq=0 out_seq=0 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f03deffd640 1 -- 172.21.15.105:0/3835920123 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558feef19b70 con 0x558feefc5a70 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f03deffd640 10 client.25501 handle_command_reply: tid=0 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 client.25501 shutdown 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 -- 172.21.15.105:0/3835920123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 msgr2=0x558feefba210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefba210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f03f4084fd0 tx=0x7f03f409b580 comp rx=0 tx=0).stop 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 -- 172.21.15.105:0/3835920123 shutdown_connections 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558feefc5a70 0x558feefc7f10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 0x558feefc1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefba210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f03deffd640 0 client.25501 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 --2- 172.21.15.105:0/3835920123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefb9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 -- 172.21.15.105:0/3835920123 >> 172.21.15.105:0/3835920123 conn(0x558feefa2680 msgr2=0x558feefaf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 -- 172.21.15.105:0/3835920123 shutdown_connections 2024-03-26T17:39:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 1 -- 172.21.15.105:0/3835920123 wait complete. 2024-03-26T17:39:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.570+0000 7f0408a19740 20 client.25501 trim_cache size 0 max 16384 2024-03-26T17:39:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f0408a19740 1 Processor -- start 2024-03-26T17:39:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f0408a19740 1 -- start start 2024-03-26T17:39:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f0408a19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefaf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f0408a19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefc1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f0408a19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 0x558feefcc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f0408a19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04001a6a90 con 0x558feefb0980 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f0408a19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04001a85d0 con 0x558feefaff90 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f0408a19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04001a7160 con 0x558feefb5580 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f03feffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefc1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f03fffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 0x558feefcc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f03feffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefc1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52332/0 (socket says 172.21.15.105:52332) 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f03feffd640 1 -- 172.21.15.105:0/353452693 learned_addr learned my addr 172.21.15.105:0/353452693 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.588+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/353452693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefaf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03ff7fe640 1 -- 172.21.15.105:0/353452693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 msgr2=0x558feefcc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/353452693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 0x558feefcc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03ff7fe640 1 -- 172.21.15.105:0/353452693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 msgr2=0x558feefc1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/353452693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefc1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03ff7fe640 1 -- 172.21.15.105:0/353452693 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558feebe0170 con 0x558feefaff90 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/353452693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefaf4a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x558feefc8780 tx=0x7f03f0005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4538e9c423f74e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03e57fa640 1 -- 172.21.15.105:0/353452693 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0053030 con 0x558feefaff90 2024-03-26T17:39:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03e57fa640 1 -- 172.21.15.105:0/353452693 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03f002cc90 con 0x558feefaff90 2024-03-26T17:39:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.589+0000 7f03e57fa640 1 -- 172.21.15.105:0/353452693 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0045040 con 0x558feefaff90 2024-03-26T17:39:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 -- 172.21.15.105:0/353452693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 msgr2=0x558feefaf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 --2- 172.21.15.105:0/353452693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefaf4a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x558feefc8780 tx=0x7f03f0005e50 comp rx=0 tx=0).stop 2024-03-26T17:39:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 -- 172.21.15.105:0/353452693 shutdown_connections 2024-03-26T17:39:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 --2- 172.21.15.105:0/353452693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb5580 0x558feefcc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 --2- 172.21.15.105:0/353452693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefb0980 0x558feefc1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 --2- 172.21.15.105:0/353452693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefaff90 0x558feefaf4a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 -- 172.21.15.105:0/353452693 >> 172.21.15.105:0/353452693 conn(0x558feefa2680 msgr2=0x558feefaed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 -- 172.21.15.105:0/353452693 shutdown_connections 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 -- 172.21.15.105:0/353452693 wait complete. 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 -- start start 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.590+0000 7f0408a19740 1 Processor -- start 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f0408a19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefc0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f0408a19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefbf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefc0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefc0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52336/0 (socket says 172.21.15.105:52336) 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f0408a19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 0x558feefbfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f0408a19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04001a7f00 con 0x558feefaff90 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f0408a19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04001a7830 con 0x558feefb5580 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f0408a19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04001a4a70 con 0x558feefb0980 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 -- 172.21.15.105:0/899610850 learned_addr learned my addr 172.21.15.105:0/899610850 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f0408a19740 10 client.0 _ll_register_callbacks cb 0x7f03f40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03feffd640 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefbf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03fffff640 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 0x558feefbfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 -- 172.21.15.105:0/899610850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 msgr2=0x558feefbf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefbf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 -- 172.21.15.105:0/899610850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 msgr2=0x558feefbfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 0x558feefbfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 -- 172.21.15.105:0/899610850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558feefc00d0 con 0x558feefaff90 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03ff7fe640 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefc0ef0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f03f002c840 tx=0x7f03f004f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3500d8da41373854 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.591+0000 7f03deffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f03deffd640 1 -- 172.21.15.105:0/899610850 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0053070 con 0x558feefaff90 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 1 -- 172.21.15.105:0/899610850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f002cc90 con 0x558feefaff90 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f03deffd640 1 -- 172.21.15.105:0/899610850 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f03f0005450 con 0x558feefaff90 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f03deffd640 1 -- 172.21.15.105:0/899610850 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03f0045040 con 0x558feefaff90 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 1 -- 172.21.15.105:0/899610850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558fef02cc80 con 0x558feefaff90 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f03deffd640 1 -- 172.21.15.105:0/899610850 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03f002cc90 con 0x558feefaff90 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f03deffd640 1 -- 172.21.15.105:0/899610850 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f03f0041480 con 0x558feefaff90 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 20 client.15651 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558fef02d1b0 0x558fef02f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 4 client.15651 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.592+0000 7f0408a19740 1 -- 172.21.15.105:0/899610850 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558feef13ec0 con 0x558fef02d1b0 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.593+0000 7f03feffd640 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558fef02d1b0 0x558fef02f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.593+0000 7f03feffd640 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558fef02d1b0 0x558fef02f5a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288810fbd8064812 server_cookie=b3b706f01b7796e8 in_seq=0 out_seq=0 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.593+0000 7f03deffd640 10 client.15651 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f03deffd640 1 -- 172.21.15.105:0/899610850 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558feef13ec0 con 0x558fef02d1b0 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f03deffd640 10 client.15651 handle_command_reply: tid=0 2024-03-26T17:39:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 client.15651 shutdown 2024-03-26T17:39:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 -- 172.21.15.105:0/899610850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 msgr2=0x558feefc0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefc0ef0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f03f002c840 tx=0x7f03f004f830 comp rx=0 tx=0).stop 2024-03-26T17:39:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 -- 172.21.15.105:0/899610850 shutdown_connections 2024-03-26T17:39:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558fef02d1b0 0x558fef02f5a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f03deffd640 0 client.15651 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558feefb5580 0x558feefbfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558feefb0980 0x558feefbf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 --2- 172.21.15.105:0/899610850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558feefaff90 0x558feefc0ef0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 -- 172.21.15.105:0/899610850 >> 172.21.15.105:0/899610850 conn(0x558feefa2680 msgr2=0x558feefaf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 -- 172.21.15.105:0/899610850 shutdown_connections 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 1 -- 172.21.15.105:0/899610850 wait complete. 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.594+0000 7f0408a19740 20 client.15651 trim_cache size 0 max 16384 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "aa75b414-0e69-4c89-9a5d-aaa3a92781d2", 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:39:00.592 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 -- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03e80a3030 msgr2=0x7f03e80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03e80a3030 0x7f03e80a54f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x558feefa6340 tx=0x7f03f8009500 comp rx=0 tx=0).stop 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 -- 172.21.15.105:0/1706825337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04001019b0 msgr2=0x7f040019c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04001019b0 0x7f040019c840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03f400e9f0 tx=0x7f03f400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 -- 172.21.15.105:0/1706825337 shutdown_connections 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03e80a3030 0x7f03e80a54f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0400102370 0x7f040019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04001019b0 0x7f040019c840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 --2- 172.21.15.105:0/1706825337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04001007c0 0x7f04000695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.595+0000 7f0405e34640 1 -- 172.21.15.105:0/1706825337 >> 172.21.15.105:0/1706825337 conn(0x7f04000fbd50 msgr2=0x7f0400104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.596+0000 7f0405e34640 1 -- 172.21.15.105:0/1706825337 shutdown_connections 2024-03-26T17:39:00.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.596+0000 7f0405e34640 1 -- 172.21.15.105:0/1706825337 wait complete. 2024-03-26T17:39:00.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:39:00.604 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:39:00.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 -- 172.21.15.105:0/2262857082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 msgr2=0x7f27fc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 --2- 172.21.15.105:0/2262857082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 0x7f27fc100be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00dc90 tx=0x7f27ec02fbd0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 -- 172.21.15.105:0/2262857082 shutdown_connections 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 --2- 172.21.15.105:0/2262857082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc102370 0x7f27fc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 --2- 172.21.15.105:0/2262857082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27fc1019b0 0x7f27fc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 --2- 172.21.15.105:0/2262857082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 0x7f27fc100be0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 -- 172.21.15.105:0/2262857082 >> 172.21.15.105:0/2262857082 conn(0x7f27fc0fbd50 msgr2=0x7f27fc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 -- 172.21.15.105:0/2262857082 shutdown_connections 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.725+0000 7f280240b640 1 -- 172.21.15.105:0/2262857082 wait complete. 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.726+0000 7f280240b640 1 Processor -- start 2024-03-26T17:39:00.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.726+0000 7f280240b640 1 -- start start 2024-03-26T17:39:00.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f280240b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 0x7f27fc195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f280240b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc1019b0 0x7f27fc195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f280240b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27fc102370 0x7f27fc19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc1019b0 0x7f27fc195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 0x7f27fc195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f280240b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc10b200 con 0x7f27fc1019b0 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 0x7f27fc195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55614/0 (socket says 172.21.15.105:55614) 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fbfff640 1 -- 172.21.15.105:0/4278411903 learned_addr learned my addr 172.21.15.105:0/4278411903 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc10b080 con 0x7f27fc102370 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc10b380 con 0x7f27fc1007c0 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f28009a8640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27fc102370 0x7f27fc19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fb7fe640 1 -- 172.21.15.105:0/4278411903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 msgr2=0x7f27fc195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 0x7f27fc195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fb7fe640 1 -- 172.21.15.105:0/4278411903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27fc102370 msgr2=0x7f27fc19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27fc102370 0x7f27fc19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fb7fe640 1 -- 172.21.15.105:0/4278411903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc19d590 con 0x7f27fc1019b0 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.727+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc1019b0 0x7f27fc195eb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e9f0 tx=0x7f27f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.728+0000 7f27f97fa640 1 -- 172.21.15.105:0/4278411903 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f000ccc0 con 0x7f27fc1019b0 2024-03-26T17:39:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.728+0000 7f27f97fa640 1 -- 172.21.15.105:0/4278411903 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27f0005020 con 0x7f27fc1019b0 2024-03-26T17:39:00.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.728+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc19d880 con 0x7f27fc1019b0 2024-03-26T17:39:00.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.728+0000 7f27f97fa640 1 -- 172.21.15.105:0/4278411903 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0004a00 con 0x7f27fc1019b0 2024-03-26T17:39:00.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.728+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc19dd80 con 0x7f27fc1019b0 2024-03-26T17:39:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.730+0000 7f27f97fa640 1 -- 172.21.15.105:0/4278411903 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f27f0004ba0 con 0x7f27fc1019b0 2024-03-26T17:39:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.731+0000 7f27f97fa640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27e40a2fb0 0x7f27e40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.731+0000 7f27f97fa640 1 -- 172.21.15.105:0/4278411903 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f27f0099c80 con 0x7f27fc1019b0 2024-03-26T17:39:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.731+0000 7f27fbfff640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27e40a2fb0 0x7f27e40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.731+0000 7f2804ffa740 1 Processor -- start 2024-03-26T17:39:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f2804ffa740 1 -- start start 2024-03-26T17:39:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b48f0 0x5633794b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b52e0 0x5633794b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794ba450 0x5633794bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f2804ffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563379419980 con 0x5633794ba450 2024-03-26T17:39:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f2804ffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5633793c1be0 con 0x5633794b48f0 2024-03-26T17:39:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f2804ffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563379415930 con 0x5633794b52e0 2024-03-26T17:39:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f27fbfff640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27e40a2fb0 0x7f27e40a5470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00dad0 tx=0x7f27ec032040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f28009a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794ba450 0x5633794bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f27fbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b48f0 0x5633794b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f28009a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794ba450 0x5633794bc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52372/0 (socket says 172.21.15.105:52372) 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f28009a8640 1 -- 172.21.15.105:0/3894175181 learned_addr learned my addr 172.21.15.105:0/3894175181 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b52e0 0x5633794b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f28009a8640 1 -- 172.21.15.105:0/3894175181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b52e0 msgr2=0x5633794b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f28009a8640 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b52e0 0x5633794b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f28009a8640 1 -- 172.21.15.105:0/3894175181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b48f0 msgr2=0x5633794b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f28009a8640 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b48f0 0x5633794b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f28009a8640 1 -- 172.21.15.105:0/3894175181 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633791f77d0 con 0x5633794ba450 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.732+0000 7f27fbfff640 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b48f0 0x5633794b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.733+0000 7f28009a8640 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794ba450 0x5633794bc8e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009fd0 tx=0x7f27f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f46768c19099271 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.733+0000 7f27e27fc640 1 -- 172.21.15.105:0/3894175181 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4004d20 con 0x5633794ba450 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.733+0000 7f27e27fc640 1 -- 172.21.15.105:0/3894175181 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27f4004050 con 0x5633794ba450 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.733+0000 7f27e27fc640 1 -- 172.21.15.105:0/3894175181 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f4005280 con 0x5633794ba450 2024-03-26T17:39:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.733+0000 7f2804ffa740 1 -- 172.21.15.105:0/3894175181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794ba450 msgr2=0x5633794bc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.733+0000 7f2804ffa740 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794ba450 0x5633794bc8e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009fd0 tx=0x7f27f40118d0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 -- 172.21.15.105:0/3894175181 shutdown_connections 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794ba450 0x5633794bc8e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b52e0 0x5633794b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 --2- 172.21.15.105:0/3894175181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b48f0 0x5633794b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 -- 172.21.15.105:0/3894175181 >> 172.21.15.105:0/3894175181 conn(0x5633794afe60 msgr2=0x5633794b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 -- 172.21.15.105:0/3894175181 shutdown_connections 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 -- 172.21.15.105:0/3894175181 wait complete. 2024-03-26T17:39:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 -- start start 2024-03-26T17:39:00.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 Processor -- start 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.734+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 0x5633794ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52380/0 (socket says 172.21.15.105:52380) 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55636/0 (socket says 172.21.15.105:55636) 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 -- 172.21.15.105:0/4272047220 learned_addr learned my addr 172.21.15.105:0/4272047220 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5633794bd400 con 0x5633794b52e0 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5633793d5160 con 0x5633794ba450 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5633794bd580 con 0x5633794b48f0 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f2804ffa740 10 client.0 _ll_register_callbacks cb 0x563379411720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f28009a8640 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 0x5633794ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 -- 172.21.15.105:0/4272047220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 msgr2=0x5633794ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 0x5633794ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 -- 172.21.15.105:0/4272047220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 msgr2=0x5633794c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 -- 172.21.15.105:0/4272047220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337912d3f0 con 0x5633794b48f0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27fbfff640 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794c7310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00d940 tx=0x7f27ec00d670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d36662de00339781 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27db7fe640 1 -- 172.21.15.105:0/4272047220 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec03d040 con 0x5633794b48f0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f27db7fe640 1 -- 172.21.15.105:0/4272047220 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f27ec003a00 con 0x5633794b48f0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.735+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f4004050 con 0x5633794b48f0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f27db7fe640 1 -- 172.21.15.105:0/4272047220 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27ec003c00 con 0x5633794b48f0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f2804ffa740 10 client.25519 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633790ed0f0 con 0x5633794b48f0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f27db7fe640 1 -- 172.21.15.105:0/4272047220 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27f4004050 con 0x5633794b48f0 2024-03-26T17:39:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f27db7fe640 1 -- 172.21.15.105:0/4272047220 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f27ec04d070 con 0x5633794b48f0 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f2804ffa740 10 client.25519 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f2804ffa740 10 client.25519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f2804ffa740 20 client.25519 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f2804ffa740 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5633794d3180 0x5633794d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f2804ffa740 4 client.25519 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563379426ec0 con 0x5633794d3180 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5633794d3180 0x5633794d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.736+0000 7f27db7fe640 10 client.25519 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5633794d3180 0x5633794d5640 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46927b905b93b69c server_cookie=4f07fc63ee777892 in_seq=0 out_seq=0 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f27db7fe640 1 -- 172.21.15.105:0/4272047220 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563379426ec0 con 0x5633794d3180 2024-03-26T17:39:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f27db7fe640 10 client.25519 handle_command_reply: tid=0 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 client.25519 shutdown 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 msgr2=0x5633794c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794c7310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00d940 tx=0x7f27ec00d670 comp rx=0 tx=0).stop 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 shutdown_connections 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5633794d3180 0x5633794d5640 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 0x5633794ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f27db7fe640 0 client.25519 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 --2- 172.21.15.105:0/4272047220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794c7310 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.737+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 >> 172.21.15.105:0/4272047220 conn(0x5633794afe60 msgr2=0x5633794af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.738+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 shutdown_connections 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.738+0000 7f2804ffa740 1 -- 172.21.15.105:0/4272047220 wait complete. 2024-03-26T17:39:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.738+0000 7f2804ffa740 20 client.25519 trim_cache size 0 max 16384 2024-03-26T17:39:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f2804ffa740 1 Processor -- start 2024-03-26T17:39:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f2804ffa740 1 -- start start 2024-03-26T17:39:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b48f0 0x5633794c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b52e0 0x5633794ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794ba450 0x5633794ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f2804ffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc1a6c70 con 0x5633794b48f0 2024-03-26T17:39:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f2804ffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc1a87b0 con 0x5633794b52e0 2024-03-26T17:39:00.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f2804ffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc1a7340 con 0x5633794ba450 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b52e0 0x5633794ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f27fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b48f0 0x5633794c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b52e0 0x5633794ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60906/0 (socket says 172.21.15.105:60906) 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f27fb7fe640 1 -- 172.21.15.105:0/1645866187 learned_addr learned my addr 172.21.15.105:0/1645866187 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.760+0000 7f28009a8640 1 --2- 172.21.15.105:0/1645866187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794ba450 0x5633794ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27fb7fe640 1 -- 172.21.15.105:0/1645866187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794ba450 msgr2=0x5633794ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/1645866187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794ba450 0x5633794ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27fb7fe640 1 -- 172.21.15.105:0/1645866187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b48f0 msgr2=0x5633794c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/1645866187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b48f0 0x5633794c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27fb7fe640 1 -- 172.21.15.105:0/1645866187 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633794ba1a0 con 0x5633794b52e0 2024-03-26T17:39:00.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/1645866187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b52e0 0x5633794ce850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e9c0 tx=0x7f27f009ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8ab0911873989e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27e1ffb640 1 -- 172.21.15.105:0/1645866187 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0098610 con 0x5633794b52e0 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27e1ffb640 1 -- 172.21.15.105:0/1645866187 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27f009c2e0 con 0x5633794b52e0 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.761+0000 7f27e1ffb640 1 -- 172.21.15.105:0/1645866187 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00987b0 con 0x5633794b52e0 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 -- 172.21.15.105:0/1645866187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b52e0 msgr2=0x5633794ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 --2- 172.21.15.105:0/1645866187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b52e0 0x5633794ce850 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e9c0 tx=0x7f27f009ff20 comp rx=0 tx=0).stop 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 -- 172.21.15.105:0/1645866187 shutdown_connections 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 --2- 172.21.15.105:0/1645866187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794ba450 0x5633794ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 --2- 172.21.15.105:0/1645866187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794b52e0 0x5633794ce850 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 --2- 172.21.15.105:0/1645866187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b48f0 0x5633794c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 -- 172.21.15.105:0/1645866187 >> 172.21.15.105:0/1645866187 conn(0x5633794afe60 msgr2=0x5633794b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 -- 172.21.15.105:0/1645866187 shutdown_connections 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.762+0000 7f2804ffa740 1 -- 172.21.15.105:0/1645866187 wait complete. 2024-03-26T17:39:00.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.763+0000 7f2804ffa740 1 -- start start 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.763+0000 7f2804ffa740 1 Processor -- start 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f2804ffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 0x5633794cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f27fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f27fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55648/0 (socket says 172.21.15.105:55648) 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f27fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52402/0 (socket says 172.21.15.105:52402) 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f27fbfff640 1 -- 172.21.15.105:0/2065863353 learned_addr learned my addr 172.21.15.105:0/2065863353 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc1a80e0 con 0x5633794b52e0 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f28009a8640 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 0x5633794cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc1a7a10 con 0x5633794ba450 2024-03-26T17:39:00.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.764+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc1a4c50 con 0x5633794b48f0 2024-03-26T17:39:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f2804ffa740 10 client.0 _ll_register_callbacks cb 0x7f27f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:00.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27fb7fe640 1 -- 172.21.15.105:0/2065863353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 msgr2=0x5633794cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27fb7fe640 1 -- 172.21.15.105:0/2065863353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 msgr2=0x5633794cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 0x5633794cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27fb7fe640 1 -- 172.21.15.105:0/2065863353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633794cdc70 con 0x5633794b52e0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27fb7fe640 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794cc730 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f27f00bb000 tx=0x7f27f0005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f9465bd30fd439 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27db7fe640 1 -- 172.21.15.105:0/2065863353 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00b9030 con 0x5633794b52e0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27f009c2e0 con 0x5633794b52e0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.765+0000 7f27db7fe640 1 -- 172.21.15.105:0/2065863353 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f27f009b030 con 0x5633794b52e0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f27db7fe640 1 -- 172.21.15.105:0/2065863353 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27f0098950 con 0x5633794b52e0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f2804ffa740 10 client.15669 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=27}) -- 0x563379539620 con 0x5633794b52e0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f27db7fe640 1 -- 172.21.15.105:0/2065863353 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f27f0098250 con 0x5633794b52e0 2024-03-26T17:39:00.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f2804ffa740 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f2804ffa740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f2804ffa740 20 client.15669 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f2804ffa740 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563379539e60 0x56337953c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f2804ffa740 4 client.15669 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.766+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563379421210 con 0x563379539e60 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.767+0000 7f27fbfff640 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563379539e60 0x56337953c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.767+0000 7f27db7fe640 10 client.15669 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.767+0000 7f27fbfff640 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563379539e60 0x56337953c280 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29138b0d5fcec7d6 server_cookie=83fadfe20717b040 in_seq=0 out_seq=0 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.767+0000 7f27db7fe640 1 -- 172.21.15.105:0/2065863353 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563379421210 con 0x563379539e60 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.767+0000 7f27db7fe640 10 client.15669 handle_command_reply: tid=0 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.767+0000 7f2804ffa740 1 client.15669 shutdown 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 msgr2=0x5633794cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794cc730 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f27f00bb000 tx=0x7f27f0005cc0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 shutdown_connections 2024-03-26T17:39:00.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563379539e60 0x56337953c280 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f27db7fe640 0 client.15669 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633794ba450 0x5633794cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633794b52e0 0x5633794cc730 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 --2- 172.21.15.105:0/2065863353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633794b48f0 0x5633794cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 >> 172.21.15.105:0/2065863353 conn(0x5633794afe60 msgr2=0x5633794b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 shutdown_connections 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 1 -- 172.21.15.105:0/2065863353 wait complete. 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.768+0000 7f2804ffa740 20 client.15669 trim_cache size 0 max 16384 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:39:00.765 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:39:00.766 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:39:00.766 INFO:teuthology.orchestra.run.smithi105.stdout: "aa75b414-0e69-4c89-9a5d-aaa3a92781d2": { 2024-03-26T17:39:00.766 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:39:00.766 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "aa75b414-0e69-4c89-9a5d-aaa3a92781d2", 2024-03-26T17:39:00.766 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:39:00.766 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:39:00.766 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:39:00.766 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:39:00.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27e40a2fb0 msgr2=0x7f27e40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27e40a2fb0 0x7f27e40a5470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f27ec00dad0 tx=0x7f27ec032040 comp rx=0 tx=0).stop 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc1019b0 msgr2=0x7f27fc195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc1019b0 0x7f27fc195eb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f27f000e9f0 tx=0x7f27f000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 shutdown_connections 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f27e40a2fb0 0x7f27e40a5470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27fc102370 0x7f27fc19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f27fc1019b0 0x7f27fc195eb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 --2- 172.21.15.105:0/4278411903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27fc1007c0 0x7f27fc195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.769+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 >> 172.21.15.105:0/4278411903 conn(0x7f27fc0fbd50 msgr2=0x7f27fc104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.770+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 shutdown_connections 2024-03-26T17:39:00.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:00.770+0000 7f280240b640 1 -- 172.21.15.105:0/4278411903 wait complete. 2024-03-26T17:39:00.777 INFO:tasks.cephfs.filesystem:scrub status for tag:aa75b414-0e69-4c89-9a5d-aaa3a92781d2 - {'path': '/', 'tag': 'aa75b414-0e69-4c89-9a5d-aaa3a92781d2', 'options': 'recursive,force'} 2024-03-26T17:39:01.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:01 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:39:01.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:01.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:39:01.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:01.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:01 smithi135 ceph-mon[35636]: pgmap v366: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T17:39:02.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:01 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:39:02.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:02.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:39:02.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:02.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:01 smithi144 ceph-mon[36095]: pgmap v366: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T17:39:02.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:01 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:39:02.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:02.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:39:02.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:02.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:01 smithi105 ceph-mon[32280]: pgmap v366: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-26T17:39:02.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:02 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:39:03.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:02 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:39:03.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:02 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:39:03.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:03 smithi135 ceph-mon[35636]: pgmap v367: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:03 smithi144 ceph-mon[36095]: pgmap v367: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:04.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:03 smithi105 ceph-mon[32280]: pgmap v367: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:05.909 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:05 smithi144 ceph-mon[36095]: pgmap v368: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:05.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:05 smithi135 ceph-mon[35636]: pgmap v368: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:06.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:05 smithi105 ceph-mon[32280]: pgmap v368: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 -- 172.21.15.105:0/1722430562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c1009c0 msgr2=0x7f130c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 --2- 172.21.15.105:0/1722430562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c1009c0 0x7f130c100e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1300009860 tx=0x7f130002bb70 comp rx=0 tx=0).stop 2024-03-26T17:39:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 -- 172.21.15.105:0/1722430562 shutdown_connections 2024-03-26T17:39:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 --2- 172.21.15.105:0/1722430562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f130c101380 0x7f130c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 --2- 172.21.15.105:0/1722430562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c1009c0 0x7f130c100e40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 --2- 172.21.15.105:0/1722430562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c101dc0 0x7f130c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 -- 172.21.15.105:0/1722430562 >> 172.21.15.105:0/1722430562 conn(0x7f130c0fb510 msgr2=0x7f130c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:06.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 -- 172.21.15.105:0/1722430562 shutdown_connections 2024-03-26T17:39:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.826+0000 7f13121d4640 1 -- 172.21.15.105:0/1722430562 wait complete. 2024-03-26T17:39:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f13121d4640 1 Processor -- start 2024-03-26T17:39:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f13121d4640 1 -- start start 2024-03-26T17:39:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f13121d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f130c1009c0 0x7f130c195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f13121d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c101380 0x7f130c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f13121d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c101dc0 0x7f130c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f13121d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f130c10b210 con 0x7f130c1009c0 2024-03-26T17:39:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f13121d4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f130c10b090 con 0x7f130c101dc0 2024-03-26T17:39:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f13121d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f130c10b390 con 0x7f130c101380 2024-03-26T17:39:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f130c1009c0 0x7f130c195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f130bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c101dc0 0x7f130c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f130c1009c0 0x7f130c195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54636/0 (socket says 172.21.15.105:54636) 2024-03-26T17:39:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f130b7fe640 1 -- 172.21.15.105:0/3155394152 learned_addr learned my addr 172.21.15.105:0/3155394152 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.827+0000 7f130affd640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c101380 0x7f130c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f130bfff640 1 -- 172.21.15.105:0/3155394152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c101380 msgr2=0x7f130c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f130bfff640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c101380 0x7f130c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f130bfff640 1 -- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f130c1009c0 msgr2=0x7f130c195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f130bfff640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f130c1009c0 0x7f130c195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f130bfff640 1 -- 172.21.15.105:0/3155394152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c19d490 con 0x7f130c101dc0 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f130affd640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c101380 0x7f130c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f130b7fe640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f130c1009c0 0x7f130c195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f130bfff640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c101dc0 0x7f130c19ce20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f130400ec50 tx=0x7f130400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f1308ff9640 1 -- 172.21.15.105:0/3155394152 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304003b70 con 0x7f130c101dc0 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f1308ff9640 1 -- 172.21.15.105:0/3155394152 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1304003d10 con 0x7f130c101dc0 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f13121d4640 1 -- 172.21.15.105:0/3155394152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c19d780 con 0x7f130c101dc0 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f1308ff9640 1 -- 172.21.15.105:0/3155394152 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304010760 con 0x7f130c101dc0 2024-03-26T17:39:06.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.828+0000 7f13121d4640 1 -- 172.21.15.105:0/3155394152 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c19dcd0 con 0x7f130c101dc0 2024-03-26T17:39:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.831+0000 7f1308ff9640 1 -- 172.21.15.105:0/3155394152 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7f1304004020 con 0x7f130c101dc0 2024-03-26T17:39:06.823 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.831+0000 7f1308ff9640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12f40a3160 0x7f12f40a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.831+0000 7f1308ff9640 1 -- 172.21.15.105:0/3155394152 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f13040cfbf0 con 0x7f130c101dc0 2024-03-26T17:39:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.831+0000 7f130b7fe640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12f40a3160 0x7f12f40a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f1314dc1740 1 Processor -- start 2024-03-26T17:39:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f1314dc1740 1 -- start start 2024-03-26T17:39:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030170ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d0301735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f1314dc1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d0300d00b0 con 0x55d03016b5d0 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f1314dc1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d030078960 con 0x55d030171130 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f1314dc1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0300cc060 con 0x55d03016bfc0 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f130affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030170ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f130b7fe640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12f40a3160 0x7f12f40a5620 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0104e0 tx=0x7f12fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f130affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030170ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47336/0 (socket says 172.21.15.105:47336) 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f130affd640 1 -- 172.21.15.105:0/4182840369 learned_addr learned my addr 172.21.15.105:0/4182840369 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.832+0000 7f130bfff640 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d0301735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f130affd640 1 -- 172.21.15.105:0/4182840369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 msgr2=0x55d0301735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f130affd640 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d0301735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f130affd640 1 -- 172.21.15.105:0/4182840369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 msgr2=0x55d03016b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f130affd640 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f130affd640 1 -- 172.21.15.105:0/4182840369 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03008ac60 con 0x55d03016bfc0 2024-03-26T17:39:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f130b7fe640 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f130affd640 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030170ac0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1300000c00 tx=0x7f1300034440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8b5e33197d95f2d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f12f1ffb640 1 -- 172.21.15.105:0/4182840369 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300043070 con 0x55d03016bfc0 2024-03-26T17:39:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f12f1ffb640 1 -- 172.21.15.105:0/4182840369 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f130003a040 con 0x55d03016bfc0 2024-03-26T17:39:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.833+0000 7f12f1ffb640 1 -- 172.21.15.105:0/4182840369 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300039070 con 0x55d03016bfc0 2024-03-26T17:39:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 -- 172.21.15.105:0/4182840369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 msgr2=0x55d030170ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030170ac0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1300000c00 tx=0x7f1300034440 comp rx=0 tx=0).stop 2024-03-26T17:39:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 -- 172.21.15.105:0/4182840369 shutdown_connections 2024-03-26T17:39:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d0301735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030170ac0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4182840369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 -- 172.21.15.105:0/4182840369 >> 172.21.15.105:0/4182840369 conn(0x55d030166b40 msgr2=0x55d030166f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 -- 172.21.15.105:0/4182840369 shutdown_connections 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.834+0000 7f1314dc1740 1 -- 172.21.15.105:0/4182840369 wait complete. 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.835+0000 7f1314dc1740 1 -- start start 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.835+0000 7f1314dc1740 1 Processor -- start 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.835+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d03017dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.835+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d03017e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.835+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030185140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.835+0000 7f130affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d03017e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.835+0000 7f130affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d03017e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54656/0 (socket says 172.21.15.105:54656) 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130affd640 1 -- 172.21.15.105:0/4159074422 learned_addr learned my addr 172.21.15.105:0/4159074422 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.835+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d03017dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d03017c8e0 con 0x55d03016bfc0 2024-03-26T17:39:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130bfff640 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030185140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d03008bdb0 con 0x55d030171130 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d03017ca60 con 0x55d03016b5d0 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130affd640 1 -- 172.21.15.105:0/4159074422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 msgr2=0x55d03017dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130affd640 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d03017dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130affd640 1 -- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 msgr2=0x55d030185140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130affd640 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030185140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130affd640 1 -- 172.21.15.105:0/4159074422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d02fde4210 con 0x55d03016bfc0 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130bfff640 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030185140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f1314dc1740 10 client.0 _ll_register_callbacks cb 0x55d030090110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130003a040 con 0x55d03016bfc0 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f12eaffd640 10 client.15681 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.836+0000 7f130affd640 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d03017e100 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1300041040 tx=0x7f1300005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cf74db60c58a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.837+0000 7f12eaffd640 1 -- 172.21.15.105:0/4159074422 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130004e060 con 0x55d03016bfc0 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.837+0000 7f12eaffd640 1 -- 172.21.15.105:0/4159074422 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f13000046c0 con 0x55d03016bfc0 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.837+0000 7f12eaffd640 1 -- 172.21.15.105:0/4159074422 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1300043070 con 0x55d03016bfc0 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.837+0000 7f1314dc1740 10 client.15681 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.837+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55d02feb0620 con 0x55d03016bfc0 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.837+0000 7f12eaffd640 1 -- 172.21.15.105:0/4159074422 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f13000048c0 con 0x55d03016bfc0 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.838+0000 7f1314dc1740 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.838+0000 7f1314dc1740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.838+0000 7f1314dc1740 20 client.15681 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.838+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d030189360 0x55d03018b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.838+0000 7f1314dc1740 4 client.15681 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.838+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0300dd5f0 con 0x55d030189360 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.838+0000 7f130b7fe640 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d030189360 0x55d03018b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.839+0000 7f12eaffd640 10 client.15681 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.839+0000 7f130b7fe640 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d030189360 0x55d03018b750 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b80f111eea24311 server_cookie=eede0654296e976f in_seq=0 out_seq=0 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f12eaffd640 1 -- 172.21.15.105:0/4159074422 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d0300dd5f0 con 0x55d030189360 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f12eaffd640 10 client.15681 handle_command_reply: tid=0 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 client.15681 shutdown 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 msgr2=0x55d03017e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d03017e100 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1300041040 tx=0x7f1300005f90 comp rx=0 tx=0).stop 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 shutdown_connections 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d030189360 0x55d03018b750 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030185140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f12eaffd640 0 client.15681 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d03017e100 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 --2- 172.21.15.105:0/4159074422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d03017dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 >> 172.21.15.105:0/4159074422 conn(0x55d030166b40 msgr2=0x55d030165df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 shutdown_connections 2024-03-26T17:39:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 1 -- 172.21.15.105:0/4159074422 wait complete. 2024-03-26T17:39:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.840+0000 7f1314dc1740 20 client.15681 trim_cache size 0 max 16384 2024-03-26T17:39:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.869+0000 7f1314dc1740 1 Processor -- start 2024-03-26T17:39:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.869+0000 7f1314dc1740 1 -- start start 2024-03-26T17:39:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.869+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.869+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030185100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.869+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030165560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030185100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030185100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47352/0 (socket says 172.21.15.105:47352) 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54670/0 (socket says 172.21.15.105:54670) 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 -- 172.21.15.105:0/1585737530 learned_addr learned my addr 172.21.15.105:0/1585737530 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f1314dc1740 1 -- 172.21.15.105:0/1585737530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1a6ae0 con 0x55d03016b5d0 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f1314dc1740 1 -- 172.21.15.105:0/1585737530 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1a8620 con 0x55d030171130 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f1314dc1740 1 -- 172.21.15.105:0/1585737530 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1a71b0 con 0x55d03016bfc0 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130bfff640 1 --2- 172.21.15.105:0/1585737530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030165560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 -- 172.21.15.105:0/1585737530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 msgr2=0x55d030165560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 --2- 172.21.15.105:0/1585737530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030165560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 -- 172.21.15.105:0/1585737530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 msgr2=0x55d03016d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 --2- 172.21.15.105:0/1585737530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 -- 172.21.15.105:0/1585737530 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03018bac0 con 0x55d03016bfc0 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.870+0000 7f130affd640 1 --2- 172.21.15.105:0/1585737530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030185100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f130002c0b0 tx=0x7f1300046000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c94bcf097adf1437 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.871+0000 7f12f17fa640 1 -- 172.21.15.105:0/1585737530 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300039040 con 0x55d03016bfc0 2024-03-26T17:39:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.871+0000 7f12f17fa640 1 -- 172.21.15.105:0/1585737530 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f130003a040 con 0x55d03016bfc0 2024-03-26T17:39:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.871+0000 7f12f17fa640 1 -- 172.21.15.105:0/1585737530 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130002c800 con 0x55d03016bfc0 2024-03-26T17:39:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.871+0000 7f1314dc1740 1 -- 172.21.15.105:0/1585737530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 msgr2=0x55d030185100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.871+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1585737530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030185100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f130002c0b0 tx=0x7f1300046000 comp rx=0 tx=0).stop 2024-03-26T17:39:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 -- 172.21.15.105:0/1585737530 shutdown_connections 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1585737530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030165560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1585737530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016bfc0 0x55d030185100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1585737530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016b5d0 0x55d03016d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 -- 172.21.15.105:0/1585737530 >> 172.21.15.105:0/1585737530 conn(0x55d030166b40 msgr2=0x55d03016ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 -- 172.21.15.105:0/1585737530 shutdown_connections 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 -- 172.21.15.105:0/1585737530 wait complete. 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 -- start start 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 Processor -- start 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d030182b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d030183040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030184050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1a7f50 con 0x55d03016bfc0 2024-03-26T17:39:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1a7880 con 0x55d030171130 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.872+0000 7f1314dc1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f130c1a4ac0 con 0x55d03016b5d0 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f1314dc1740 10 client.0 _ll_register_callbacks cb 0x7f1300041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d030182b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d030183040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d030182b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47358/0 (socket says 172.21.15.105:47358) 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 -- 172.21.15.105:0/1061672020 learned_addr learned my addr 172.21.15.105:0/1061672020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130bfff640 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030184050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 -- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 msgr2=0x55d030184050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030184050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 -- 172.21.15.105:0/1061672020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 msgr2=0x55d030183040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d030183040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 -- 172.21.15.105:0/1061672020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d030184640 con 0x55d03016b5d0 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130bfff640 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030184050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f130b7fe640 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d030182b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55d03016c820 tx=0x7f12fc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3fe03fe43854a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f12eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.873+0000 7f12eaffd640 1 -- 172.21.15.105:0/1061672020 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc02ae60 con 0x55d03016b5d0 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f12eaffd640 1 -- 172.21.15.105:0/1061672020 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f12fc033670 con 0x55d03016b5d0 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 1 -- 172.21.15.105:0/1061672020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130003a040 con 0x55d03016b5d0 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f12eaffd640 1 -- 172.21.15.105:0/1061672020 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f12fc0338b0 con 0x55d03016b5d0 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 10 client.25540 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 1 -- 172.21.15.105:0/1061672020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d0301efa60 con 0x55d03016b5d0 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f12eaffd640 1 -- 172.21.15.105:0/1061672020 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12fc02e070 con 0x55d03016b5d0 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f12eaffd640 1 -- 172.21.15.105:0/1061672020 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f12fc03d3f0 con 0x55d03016b5d0 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 10 client.25540 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 10 client.25540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 20 client.25540 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d030189360 0x55d0301f2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 4 client.25540 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:39:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.874+0000 7f1314dc1740 1 -- 172.21.15.105:0/1061672020 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f130003a040 con 0x55d030189360 2024-03-26T17:39:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.875+0000 7f130affd640 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d030189360 0x55d0301f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.875+0000 7f12eaffd640 10 client.25540 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.875+0000 7f130affd640 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d030189360 0x55d0301f2470 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df6793969493029 server_cookie=f4ef4c96c1613607 in_seq=0 out_seq=0 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f12eaffd640 1 -- 172.21.15.105:0/1061672020 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f130003a040 con 0x55d030189360 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f12eaffd640 10 client.25540 handle_command_reply: tid=0 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 client.25540 shutdown 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 -- 172.21.15.105:0/1061672020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 msgr2=0x55d030182b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d030182b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55d03016c820 tx=0x7f12fc004730 comp rx=0 tx=0).stop 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 -- 172.21.15.105:0/1061672020 shutdown_connections 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d030189360 0x55d0301f2470 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d030171130 0x55d030184050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d03016bfc0 0x55d030183040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 --2- 172.21.15.105:0/1061672020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d03016b5d0 0x55d030182b00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 -- 172.21.15.105:0/1061672020 >> 172.21.15.105:0/1061672020 conn(0x55d030166b40 msgr2=0x55d030165560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f12eaffd640 0 client.25540 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 -- 172.21.15.105:0/1061672020 shutdown_connections 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.878+0000 7f1314dc1740 1 -- 172.21.15.105:0/1061672020 wait complete. 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f1314dc1740 20 client.25540 trim_cache size 0 max 16384 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:39:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:39:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 -- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12f40a3160 msgr2=0x7f12f40a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12f40a3160 0x7f12f40a5620 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0104e0 tx=0x7f12fc00b410 comp rx=0 tx=0).stop 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 -- 172.21.15.105:0/3155394152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c101dc0 msgr2=0x7f130c19ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c101dc0 0x7f130c19ce20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f130400ec50 tx=0x7f130400c6a0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 -- 172.21.15.105:0/3155394152 shutdown_connections 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12f40a3160 0x7f12f40a5620 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f130c101dc0 0x7f130c19ce20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c101380 0x7f130c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 --2- 172.21.15.105:0/3155394152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f130c1009c0 0x7f130c195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.879+0000 7f13121d4640 1 -- 172.21.15.105:0/3155394152 >> 172.21.15.105:0/3155394152 conn(0x7f130c0fb510 msgr2=0x7f130c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.880+0000 7f13121d4640 1 -- 172.21.15.105:0/3155394152 shutdown_connections 2024-03-26T17:39:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:06.880+0000 7f13121d4640 1 -- 172.21.15.105:0/3155394152 wait complete. 2024-03-26T17:39:07.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:07 smithi105 ceph-mon[32280]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:07.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:07 smithi135 ceph-mon[35636]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:07 smithi144 ceph-mon[36095]: pgmap v369: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:09.916 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:09 smithi135 ceph-mon[35636]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T17:39:10.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:09 smithi144 ceph-mon[36095]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T17:39:10.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:09 smithi105 ceph-mon[32280]: pgmap v370: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-26T17:39:11.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:11 smithi135 ceph-mon[35636]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T17:39:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:11 smithi144 ceph-mon[36095]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T17:39:12.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:11 smithi105 ceph-mon[32280]: pgmap v371: 97 pgs: 97 active+clean; 665 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-26T17:39:13.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:13 smithi135 ceph-mon[35636]: pgmap v372: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:14.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:13 smithi144 ceph-mon[36095]: pgmap v372: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:14.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:13 smithi105 ceph-mon[32280]: pgmap v372: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-26T17:39:15.834 INFO:tasks.workunit.client.0.smithi105.stderr:Note: switching to '72e2437271dcafb9826f2f49bb1001426fa6de81'. 2024-03-26T17:39:15.834 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-03-26T17:39:15.834 INFO:tasks.workunit.client.0.smithi105.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-26T17:39:15.834 INFO:tasks.workunit.client.0.smithi105.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-26T17:39:15.834 INFO:tasks.workunit.client.0.smithi105.stderr:state without impacting any branches by switching back to a branch. 2024-03-26T17:39:15.834 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-03-26T17:39:15.834 INFO:tasks.workunit.client.0.smithi105.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr: git switch -c 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr:Or undo this operation with: 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr: git switch - 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-03-26T17:39:15.835 INFO:tasks.workunit.client.0.smithi105.stderr:HEAD is now at 72e2437271d qa/cephfs/overrides: ignore expected PG_DEGRADED health warning 2024-03-26T17:39:15.838 DEBUG:teuthology.orchestra.run.smithi105:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-26T17:39:15.865 INFO:tasks.workunit.client.0.smithi105.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-26T17:39:15.867 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T17:39:15.867 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-26T17:39:15.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:15 smithi135 ceph-mon[35636]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:15 smithi144 ceph-mon[36095]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:16.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:15 smithi105 ceph-mon[32280]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:16.619 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-26T17:39:16.699 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-26T17:39:16.738 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T17:39:16.739 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T17:39:16.739 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-26T17:39:16.778 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T17:39:16.781 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T17:39:16.781 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-26T17:39:16.837 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-03-26T17:39:16.839 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-03-26T17:39:16.839 DEBUG:teuthology.orchestra.run.smithi105:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=72e2437271dcafb9826f2f49bb1001426fa6de81 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-03-26T17:39:16.903 INFO:tasks.workunit.client.0.smithi105.stderr:+ dbench 1 2024-03-26T17:39:16.908 INFO:tasks.workunit.client.0.smithi105.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-26T17:39:16.908 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-03-26T17:39:16.908 INFO:tasks.workunit.client.0.smithi105.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-26T17:39:16.917 INFO:tasks.workunit.client.0.smithi105.stdout:0 of 1 processes prepared for launch 0 sec 2024-03-26T17:39:17.017 INFO:tasks.workunit.client.0.smithi105.stdout:1 of 1 processes prepared for launch 0 sec 2024-03-26T17:39:17.017 INFO:tasks.workunit.client.0.smithi105.stdout:releasing clients 2024-03-26T17:39:17.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:17 smithi135 ceph-mon[35636]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:18.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 862 21.58 MB/sec warmup 1 sec latency 15.125 ms 2024-03-26T17:39:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:17 smithi144 ceph-mon[36095]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:18.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:17 smithi105 ceph-mon[32280]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-26T17:39:19.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 1497 13.02 MB/sec warmup 2 sec latency 16.123 ms 2024-03-26T17:39:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 -- 172.21.15.105:0/2574233745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc014100d00 msgr2=0x7fc014101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 --2- 172.21.15.105:0/2574233745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc014100d00 0x7fc014101120 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc004009900 tx=0x7fc00402bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 -- 172.21.15.105:0/2574233745 shutdown_connections 2024-03-26T17:39:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 --2- 172.21.15.105:0/2574233745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0141023d0 0x7fc014108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 --2- 172.21.15.105:0/2574233745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc014101a10 0x7fc014101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 --2- 172.21.15.105:0/2574233745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc014100d00 0x7fc014101120 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 -- 172.21.15.105:0/2574233745 >> 172.21.15.105:0/2574233745 conn(0x7fc0140fb510 msgr2=0x7fc0140fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 -- 172.21.15.105:0/2574233745 shutdown_connections 2024-03-26T17:39:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.979+0000 7fc01980b640 1 -- 172.21.15.105:0/2574233745 wait complete. 2024-03-26T17:39:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc01980b640 1 Processor -- start 2024-03-26T17:39:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc01980b640 1 -- start start 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc01980b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc014100d00 0x7fc014197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc01980b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc014101a10 0x7fc014198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc01980b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0141023d0 0x7fc01419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc01980b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc01410d490 con 0x7fc0141023d0 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc01980b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc01410d310 con 0x7fc014100d00 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc01980b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc01410d610 con 0x7fc014101a10 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0141023d0 0x7fc01419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc014100d00 0x7fc014197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc014101a10 0x7fc014198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc014100d00 0x7fc014197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50848/0 (socket says 172.21.15.105:50848) 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc014101a10 0x7fc014198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58278/0 (socket says 172.21.15.105:58278) 2024-03-26T17:39:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.980+0000 7fc012ffd640 1 -- 172.21.15.105:0/1637284650 learned_addr learned my addr 172.21.15.105:0/1637284650 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc0137fe640 1 -- 172.21.15.105:0/1637284650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc014101a10 msgr2=0x7fc014198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc0137fe640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc014101a10 0x7fc014198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc0137fe640 1 -- 172.21.15.105:0/1637284650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc014100d00 msgr2=0x7fc014197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc0137fe640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc014100d00 0x7fc014197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc0137fe640 1 -- 172.21.15.105:0/1637284650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01419f6e0 con 0x7fc0141023d0 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc00bfff640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc014101a10 0x7fc014198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc012ffd640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc014100d00 0x7fc014197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc0137fe640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0141023d0 0x7fc01419f070 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009fd0 tx=0x7fc00c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc018809640 1 -- 172.21.15.105:0/1637284650 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00ce20 con 0x7fc0141023d0 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc018809640 1 -- 172.21.15.105:0/1637284650 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc00c005020 con 0x7fc0141023d0 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc01980b640 1 -- 172.21.15.105:0/1637284650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01419f9d0 con 0x7fc0141023d0 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc018809640 1 -- 172.21.15.105:0/1637284650 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c010690 con 0x7fc0141023d0 2024-03-26T17:39:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.981+0000 7fc01980b640 1 -- 172.21.15.105:0/1637284650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01419fed0 con 0x7fc0141023d0 2024-03-26T17:39:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc01c3f3740 1 Processor -- start 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc018809640 1 -- 172.21.15.105:0/1637284650 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 19) ==== 137906+0+0 (secure 0 0 0) 0x7fc00c0051d0 con 0x7fc0141023d0 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc01c3f3740 1 -- start start 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebd19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebd95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559daeb360b0 con 0x559daebd7130 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559daeade960 con 0x559daebd15d0 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559daeb32060 con 0x559daebd1fc0 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebd19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebd95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc018809640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0000ab870 0x7fc0000add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebd95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43494/0 (socket says 172.21.15.105:43494) 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.984+0000 7fc00bfff640 1 -- 172.21.15.105:0/2396657858 learned_addr learned my addr 172.21.15.105:0/2396657858 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc018809640 1 -- 172.21.15.105:0/1637284650 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fc00c0cfc90 con 0x7fc0141023d0 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc0137fe640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0000ab870 0x7fc0000add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc012ffd640 1 -- 172.21.15.105:0/2396657858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 msgr2=0x559daebd19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc012ffd640 1 --2- 172.21.15.105:0/2396657858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebd19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc012ffd640 1 -- 172.21.15.105:0/2396657858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 msgr2=0x559daebd95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc012ffd640 1 --2- 172.21.15.105:0/2396657858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebd95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc012ffd640 1 -- 172.21.15.105:0/2396657858 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559daeaf0c60 con 0x559daebd1fc0 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2396657858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebd95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc012ffd640 1 --2- 172.21.15.105:0/2396657858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd6ac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc004005ab0 tx=0x7fc004009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf680fb0141eac0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc0137fe640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0000ab870 0x7fc0000add30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc00c098680 tx=0x7fc00c005810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc009ffb640 1 -- 172.21.15.105:0/2396657858 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004045020 con 0x559daebd1fc0 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc009ffb640 1 -- 172.21.15.105:0/2396657858 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc00402cc90 con 0x559daebd1fc0 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.985+0000 7fc009ffb640 1 -- 172.21.15.105:0/2396657858 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004004920 con 0x559daebd1fc0 2024-03-26T17:39:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2396657858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 msgr2=0x559daebd6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2396657858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd6ac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc004005ab0 tx=0x7fc004009670 comp rx=0 tx=0).stop 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2396657858 shutdown_connections 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2396657858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebd95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2396657858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd6ac0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2396657858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebd19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2396657858 >> 172.21.15.105:0/2396657858 conn(0x559daebccb40 msgr2=0x559daebccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2396657858 shutdown_connections 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2396657858 wait complete. 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 -- start start 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.986+0000 7fc01c3f3740 1 Processor -- start 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd15d0 0x559daebd4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd7130 0x559daebd55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559daebd9f40 con 0x559daebd15d0 2024-03-26T17:39:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559daeaf1db0 con 0x559daebd7130 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559daebda0c0 con 0x559daebd1fc0 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc01c3f3740 10 client.0 _ll_register_callbacks cb 0x559daeaf6110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd15d0 0x559daebd4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd15d0 0x559daebd4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43510/0 (socket says 172.21.15.105:43510) 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc00bfff640 1 -- 172.21.15.105:0/921903931 learned_addr learned my addr 172.21.15.105:0/921903931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc012ffd640 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc00bfff640 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd7130 0x559daebd55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc012ffd640 1 -- 172.21.15.105:0/921903931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd7130 msgr2=0x559daebd55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc012ffd640 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd7130 0x559daebd55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc012ffd640 1 -- 172.21.15.105:0/921903931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd15d0 msgr2=0x559daebd4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc012ffd640 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd15d0 0x559daebd4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc012ffd640 1 -- 172.21.15.105:0/921903931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dae84a210 con 0x559daebd1fc0 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc00bfff640 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd15d0 0x559daebd4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fc012ffd640 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd4650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x559daebcdab0 tx=0x7fc0040056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eccffd7562d62983 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fbff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fbff2ffd640 1 -- 172.21.15.105:0/921903931 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004045040 con 0x559daebd1fc0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.987+0000 7fbff2ffd640 1 -- 172.21.15.105:0/921903931 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fc004003b00 con 0x559daebd1fc0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 1 -- 172.21.15.105:0/921903931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc00402cc90 con 0x559daebd1fc0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fbff2ffd640 1 -- 172.21.15.105:0/921903931 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc004003d00 con 0x559daebd1fc0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 10 client.25552 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 1 -- 172.21.15.105:0/921903931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559dae916620 con 0x559daebd1fc0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fbff2ffd640 1 -- 172.21.15.105:0/921903931 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc00402cc90 con 0x559daebd1fc0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fbff2ffd640 1 -- 172.21.15.105:0/921903931 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc00404c460 con 0x559daebd1fc0 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 10 client.25552 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 10 client.25552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 20 client.25552 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559daebe6d70 0x559daebe9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 4 client.25552 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc01c3f3740 1 -- 172.21.15.105:0/921903931 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559daeb435f0 con 0x559daebe6d70 2024-03-26T17:39:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.988+0000 7fc00bfff640 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559daebe6d70 0x559daebe9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.989+0000 7fbff2ffd640 10 client.25552 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.989+0000 7fc00bfff640 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559daebe6d70 0x559daebe9180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b415a8032694d2 server_cookie=1bfbcd5270daefb4 in_seq=0 out_seq=0 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fbff2ffd640 1 -- 172.21.15.105:0/921903931 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559daeb435f0 con 0x559daebe6d70 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fbff2ffd640 10 client.25552 handle_command_reply: tid=0 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 client.25552 shutdown 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 -- 172.21.15.105:0/921903931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 msgr2=0x559daebd4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd4650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x559daebcdab0 tx=0x7fc0040056b0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 -- 172.21.15.105:0/921903931 shutdown_connections 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559daebe6d70 0x559daebe9180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd7130 0x559daebd55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fbff2ffd640 0 client.25552 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebd4650 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/921903931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd15d0 0x559daebd4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 -- 172.21.15.105:0/921903931 >> 172.21.15.105:0/921903931 conn(0x559daebccb40 msgr2=0x559daebcbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 -- 172.21.15.105:0/921903931 shutdown_connections 2024-03-26T17:39:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 1 -- 172.21.15.105:0/921903931 wait complete. 2024-03-26T17:39:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:19.991+0000 7fc01c3f3740 20 client.25552 trim_cache size 0 max 16384 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.007+0000 7fc01c3f3740 1 Processor -- start 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.007+0000 7fc01c3f3740 1 -- start start 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebd2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebdabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebcb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a6990 con 0x559daebd7130 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a84d0 con 0x559daebd15d0 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a7060 con 0x559daebd1fc0 2024-03-26T17:39:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebcb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebdabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebdabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58312/0 (socket says 172.21.15.105:58312) 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebcb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43514/0 (socket says 172.21.15.105:43514) 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc012ffd640 1 -- 172.21.15.105:0/2802277263 learned_addr learned my addr 172.21.15.105:0/2802277263 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebd2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 -- 172.21.15.105:0/2802277263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 msgr2=0x559daebdabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebdabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 -- 172.21.15.105:0/2802277263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 msgr2=0x559daebd2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebd2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 -- 172.21.15.105:0/2802277263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559daebe9800 con 0x559daebd7130 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc012ffd640 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebdabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebcb560 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559daeaf1f00 tx=0x7fbffc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f22459f898d3aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc0097fa640 1 -- 172.21.15.105:0/2802277263 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc029040 con 0x559daebd7130 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc0097fa640 1 -- 172.21.15.105:0/2802277263 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbffc02a040 con 0x559daebd7130 2024-03-26T17:39:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.008+0000 7fc0097fa640 1 -- 172.21.15.105:0/2802277263 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc004a00 con 0x559daebd7130 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2802277263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 msgr2=0x559daebcb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebcb560 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559daeaf1f00 tx=0x7fbffc005f90 comp rx=0 tx=0).stop 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2802277263 shutdown_connections 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd7130 0x559daebcb560 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd1fc0 0x559daebdabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2802277263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebd2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2802277263 >> 172.21.15.105:0/2802277263 conn(0x559daebccb40 msgr2=0x559daebda4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2802277263 shutdown_connections 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.009+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2802277263 wait complete. 2024-03-26T17:39:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 1 -- start start 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 1 Processor -- start 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebe1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd1fc0 0x559daebe2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd7130 0x559daebe3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a7e00 con 0x559daebd1fc0 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a7730 con 0x559daebd15d0 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141a4970 con 0x559daebd7130 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc01c3f3740 10 client.0 _ll_register_callbacks cb 0x7fbffc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd1fc0 0x559daebe2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebe1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd1fc0 0x559daebe2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43520/0 (socket says 172.21.15.105:43520) 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc012ffd640 1 -- 172.21.15.105:0/2505060240 learned_addr learned my addr 172.21.15.105:0/2505060240 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.010+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd7130 0x559daebe3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc012ffd640 1 -- 172.21.15.105:0/2505060240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd7130 msgr2=0x559daebe3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc012ffd640 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd7130 0x559daebe3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc012ffd640 1 -- 172.21.15.105:0/2505060240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 msgr2=0x559daebe1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc012ffd640 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebe1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc012ffd640 1 -- 172.21.15.105:0/2505060240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559daebcd5b0 con 0x559daebd1fc0 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebe1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc012ffd640 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd1fc0 0x559daebe2120 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc004004650 tx=0x7fc00404c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c862f61b108e52d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fbff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fbff2ffd640 1 -- 172.21.15.105:0/2505060240 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004045070 con 0x559daebd1fc0 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fbff2ffd640 1 -- 172.21.15.105:0/2505060240 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fc004003b00 con 0x559daebd1fc0 2024-03-26T17:39:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2505060240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbffc02a040 con 0x559daebd1fc0 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fbff2ffd640 1 -- 172.21.15.105:0/2505060240 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc00404ccd0 con 0x559daebd1fc0 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc01c3f3740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2505060240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559daec4d6e0 con 0x559daebd1fc0 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.011+0000 7fbff2ffd640 1 -- 172.21.15.105:0/2505060240 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbffc02a040 con 0x559daebd1fc0 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fbff2ffd640 1 -- 172.21.15.105:0/2505060240 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc00405d030 con 0x559daebd1fc0 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fc01c3f3740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fc01c3f3740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fc01c3f3740 20 client.15705 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559daec4e110 0x559daec50530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fc01c3f3740 4 client.15705 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2505060240 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559daeb3d940 con 0x559daec4e110 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559daec4e110 0x559daec50530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fbff2ffd640 10 client.15705 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.012+0000 7fc00bfff640 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559daec4e110 0x559daec50530 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4678aecd2056bd24 server_cookie=dd703cb6718663e2 in_seq=0 out_seq=0 2024-03-26T17:39:20.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 2146 8.90 MB/sec warmup 3 sec latency 15.735 ms 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fbff2ffd640 1 -- 172.21.15.105:0/2505060240 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559daeb3d940 con 0x559daec4e110 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fbff2ffd640 10 client.15705 handle_command_reply: tid=0 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 client.15705 shutdown 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2505060240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd1fc0 msgr2=0x559daebe2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd1fc0 0x559daebe2120 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc004004650 tx=0x7fc00404c770 comp rx=0 tx=0).stop 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2505060240 shutdown_connections 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559daec4e110 0x559daec50530 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559daebd7130 0x559daebe3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559daebd1fc0 0x559daebe2120 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 --2- 172.21.15.105:0/2505060240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559daebd15d0 0x559daebe1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2505060240 >> 172.21.15.105:0/2505060240 conn(0x559daebccb40 msgr2=0x559daebcfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.052+0000 7fbff2ffd640 0 client.15705 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2505060240 shutdown_connections 2024-03-26T17:39:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01c3f3740 1 -- 172.21.15.105:0/2505060240 wait complete. 2024-03-26T17:39:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01c3f3740 20 client.15705 trim_cache size 0 max 16384 2024-03-26T17:39:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:39:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:39:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:39:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:39:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 -- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0000ab870 msgr2=0x7fc0000add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0000ab870 0x7fc0000add30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc00c098680 tx=0x7fc00c005810 comp rx=0 tx=0).stop 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 -- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0141023d0 msgr2=0x7fc01419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0141023d0 0x7fc01419f070 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009fd0 tx=0x7fc00c00ef10 comp rx=0 tx=0).stop 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 -- 172.21.15.105:0/1637284650 shutdown_connections 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0000ab870 0x7fc0000add30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0141023d0 0x7fc01419f070 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc014101a10 0x7fc014198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 --2- 172.21.15.105:0/1637284650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc014100d00 0x7fc014197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 -- 172.21.15.105:0/1637284650 >> 172.21.15.105:0/1637284650 conn(0x7fc0140fb510 msgr2=0x7fc014104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 -- 172.21.15.105:0/1637284650 shutdown_connections 2024-03-26T17:39:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:20.053+0000 7fc01980b640 1 -- 172.21.15.105:0/1637284650 wait complete. 2024-03-26T17:39:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:19 smithi144 ceph-mon[36095]: pgmap v375: 97 pgs: 97 active+clean; 6.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 515 KiB/s wr, 13 op/s 2024-03-26T17:39:20.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:19 smithi105 ceph-mon[32280]: pgmap v375: 97 pgs: 97 active+clean; 6.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 515 KiB/s wr, 13 op/s 2024-03-26T17:39:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:19 smithi135 ceph-mon[35636]: pgmap v375: 97 pgs: 97 active+clean; 6.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 515 KiB/s wr, 13 op/s 2024-03-26T17:39:21.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 2775 7.99 MB/sec warmup 4 sec latency 40.493 ms 2024-03-26T17:39:22.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 3365 7.09 MB/sec warmup 5 sec latency 15.508 ms 2024-03-26T17:39:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:21 smithi144 ceph-mon[36095]: pgmap v376: 97 pgs: 97 active+clean; 13 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-03-26T17:39:22.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:21 smithi105 ceph-mon[32280]: pgmap v376: 97 pgs: 97 active+clean; 13 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-03-26T17:39:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:21 smithi135 ceph-mon[35636]: pgmap v376: 97 pgs: 97 active+clean; 13 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 MiB/s wr, 14 op/s 2024-03-26T17:39:23.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 3955 6.68 MB/sec warmup 6 sec latency 14.771 ms 2024-03-26T17:39:24.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 4433 5.81 MB/sec warmup 7 sec latency 24.995 ms 2024-03-26T17:39:24.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:23 smithi144 ceph-mon[36095]: pgmap v377: 97 pgs: 97 active+clean; 22 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.9 MiB/s wr, 47 op/s 2024-03-26T17:39:24.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:23 smithi105 ceph-mon[32280]: pgmap v377: 97 pgs: 97 active+clean; 22 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.9 MiB/s wr, 47 op/s 2024-03-26T17:39:24.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:23 smithi135 ceph-mon[35636]: pgmap v377: 97 pgs: 97 active+clean; 22 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.9 MiB/s wr, 47 op/s 2024-03-26T17:39:25.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 5068 5.65 MB/sec warmup 8 sec latency 16.205 ms 2024-03-26T17:39:25.942 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:25 smithi105 ceph-mon[32280]: pgmap v378: 97 pgs: 97 active+clean; 24 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.3 MiB/s wr, 65 op/s 2024-03-26T17:39:26.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 5737 5.10 MB/sec warmup 9 sec latency 17.597 ms 2024-03-26T17:39:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:25 smithi144 ceph-mon[36095]: pgmap v378: 97 pgs: 97 active+clean; 24 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.3 MiB/s wr, 65 op/s 2024-03-26T17:39:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:25 smithi135 ceph-mon[35636]: pgmap v378: 97 pgs: 97 active+clean; 24 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.3 MiB/s wr, 65 op/s 2024-03-26T17:39:27.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 6428 5.13 MB/sec warmup 10 sec latency 16.991 ms 2024-03-26T17:39:28.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 6914 4.96 MB/sec warmup 11 sec latency 19.887 ms 2024-03-26T17:39:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:27 smithi144 ceph-mon[36095]: pgmap v379: 97 pgs: 97 active+clean; 24 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-03-26T17:39:28.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:27 smithi105 ceph-mon[32280]: pgmap v379: 97 pgs: 97 active+clean; 24 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-03-26T17:39:28.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:27 smithi135 ceph-mon[35636]: pgmap v379: 97 pgs: 97 active+clean; 24 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 2.4 MiB/s wr, 70 op/s 2024-03-26T17:39:29.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 7480 4.93 MB/sec warmup 12 sec latency 15.352 ms 2024-03-26T17:39:30.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 7965 4.60 MB/sec warmup 13 sec latency 18.103 ms 2024-03-26T17:39:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:29 smithi144 ceph-mon[36095]: pgmap v380: 97 pgs: 97 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-03-26T17:39:30.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:29 smithi144 ceph-mon[36095]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-26T17:39:30.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:29 smithi105 ceph-mon[32280]: pgmap v380: 97 pgs: 97 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-03-26T17:39:30.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:29 smithi105 ceph-mon[32280]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-26T17:39:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:29 smithi135 ceph-mon[35636]: pgmap v380: 97 pgs: 97 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 2.6 MiB/s wr, 98 op/s 2024-03-26T17:39:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:29 smithi135 ceph-mon[35636]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-26T17:39:30.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:39:30.778 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:39:30.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 -- 172.21.15.105:0/420146186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c104d10 msgr2=0x7ff66c107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 --2- 172.21.15.105:0/420146186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c104d10 0x7ff66c107100 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009900 tx=0x7ff65c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 -- 172.21.15.105:0/420146186 shutdown_connections 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 --2- 172.21.15.105:0/420146186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c107640 0x7ff66c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 --2- 172.21.15.105:0/420146186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c104d10 0x7ff66c107100 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 --2- 172.21.15.105:0/420146186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff66c1023e0 0x7ff66c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 -- 172.21.15.105:0/420146186 >> 172.21.15.105:0/420146186 conn(0x7ff66c0fbd50 msgr2=0x7ff66c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 -- 172.21.15.105:0/420146186 shutdown_connections 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.915+0000 7ff673c7e640 1 -- 172.21.15.105:0/420146186 wait complete. 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff673c7e640 1 Processor -- start 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff673c7e640 1 -- start start 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff673c7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff66c1023e0 0x7ff66c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff673c7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c104d10 0x7ff66c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff673c7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c107640 0x7ff66c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff673c7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10d3f0 con 0x7ff66c107640 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff673c7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10d270 con 0x7ff66c1023e0 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff673c7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c10d570 con 0x7ff66c104d10 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c107640 0x7ff66c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff66c1023e0 0x7ff66c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c107640 0x7ff66c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49062/0 (socket says 172.21.15.105:49062) 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff66c1023e0 0x7ff66c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41028/0 (socket says 172.21.15.105:41028) 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 -- 172.21.15.105:0/3046581744 learned_addr learned my addr 172.21.15.105:0/3046581744 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff671219640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c104d10 0x7ff66c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 -- 172.21.15.105:0/3046581744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c104d10 msgr2=0x7ff66c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c104d10 0x7ff66c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 -- 172.21.15.105:0/3046581744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff66c1023e0 msgr2=0x7ff66c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff66c1023e0 0x7ff66c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 -- 172.21.15.105:0/3046581744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66c19f7a0 con 0x7ff66c107640 2024-03-26T17:39:30.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.916+0000 7ff67221b640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c107640 0x7ff66c19f130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff66400eb10 tx=0x7ff66400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.917+0000 7ff662ffd640 1 -- 172.21.15.105:0/3046581744 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664004b20 con 0x7ff66c107640 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.917+0000 7ff662ffd640 1 -- 172.21.15.105:0/3046581744 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff664004cc0 con 0x7ff66c107640 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.917+0000 7ff673c7e640 1 -- 172.21.15.105:0/3046581744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66c19fa90 con 0x7ff66c107640 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.917+0000 7ff662ffd640 1 -- 172.21.15.105:0/3046581744 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664010690 con 0x7ff66c107640 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.917+0000 7ff673c7e640 1 -- 172.21.15.105:0/3046581744 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66c1a0000 con 0x7ff66c107640 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.919+0000 7ff662ffd640 1 -- 172.21.15.105:0/3046581744 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff664005020 con 0x7ff66c107640 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.920+0000 7ff662ffd640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6580a2fa0 0x7ff6580a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.920+0000 7ff676865740 1 Processor -- start 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.920+0000 7ff671a1a640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6580a2fa0 0x7ff6580a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.920+0000 7ff662ffd640 1 -- 172.21.15.105:0/3046581744 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7ff6640cfa60 con 0x7ff66c107640 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.920+0000 7ff676865740 1 -- start start 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.920+0000 7ff676865740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff676865740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff676865740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff676865740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56464191b980 con 0x5646419bc450 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff676865740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646418c3be0 con 0x5646419b68f0 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff676865740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564641917930 con 0x5646419b72e0 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6580a2fa0 0x7ff6580a5460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6680095f0 tx=0x7ff668005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41034/0 (socket says 172.21.15.105:41034) 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671219640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 -- 172.21.15.105:0/2880506027 learned_addr learned my addr 172.21.15.105:0/2880506027 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff67221b640 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 -- 172.21.15.105:0/2880506027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 msgr2=0x5646419bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 -- 172.21.15.105:0/2880506027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 msgr2=0x5646419be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:30.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419be8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 -- 172.21.15.105:0/2880506027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646416f97d0 con 0x5646419b68f0 2024-03-26T17:39:30.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671219640 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:30.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff67221b640 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419be8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:30.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.921+0000 7ff671a1a640 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419b6d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6680097e0 tx=0x7ff668014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fa3dfe4a0f2604f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:30.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.922+0000 7ff657fff640 1 -- 172.21.15.105:0/2880506027 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668004020 con 0x5646419b68f0 2024-03-26T17:39:30.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.922+0000 7ff657fff640 1 -- 172.21.15.105:0/2880506027 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6680041c0 con 0x5646419b68f0 2024-03-26T17:39:30.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.922+0000 7ff657fff640 1 -- 172.21.15.105:0/2880506027 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6680266a0 con 0x5646419b68f0 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 -- 172.21.15.105:0/2880506027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 msgr2=0x5646419b6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419b6d10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6680097e0 tx=0x7ff668014000 comp rx=0 tx=0).stop 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 -- 172.21.15.105:0/2880506027 shutdown_connections 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419be8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 --2- 172.21.15.105:0/2880506027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419b6d10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 -- 172.21.15.105:0/2880506027 >> 172.21.15.105:0/2880506027 conn(0x5646419b1e60 msgr2=0x5646419b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 -- 172.21.15.105:0/2880506027 shutdown_connections 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 -- 172.21.15.105:0/2880506027 wait complete. 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 -- start start 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.923+0000 7ff676865740 1 Processor -- start 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff676865740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff676865740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff676865740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41046/0 (socket says 172.21.15.105:41046) 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff671219640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff671219640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41488/0 (socket says 172.21.15.105:41488) 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff671219640 1 -- 172.21.15.105:0/177034576 learned_addr learned my addr 172.21.15.105:0/177034576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff67221b640 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5646419bf400 con 0x5646419bc450 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646418d7160 con 0x5646419b68f0 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5646419bf580 con 0x5646419b72e0 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff67221b640 1 -- 172.21.15.105:0/177034576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 msgr2=0x5646419c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff67221b640 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff67221b640 1 -- 172.21.15.105:0/177034576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 msgr2=0x5646419c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:30.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff67221b640 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff67221b640 1 -- 172.21.15.105:0/177034576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56464162f3f0 con 0x5646419bc450 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff671219640 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.924+0000 7ff676865740 10 client.0 _ll_register_callbacks cb 0x564641913720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff67221b640 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c8080 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff6640cd890 tx=0x7ff66409bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afedb2647d01c033 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff6557fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff6557fa640 1 -- 172.21.15.105:0/177034576 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640bc2c0 con 0x5646419bc450 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6680041c0 con 0x5646419bc450 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff6557fa640 1 -- 172.21.15.105:0/177034576 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7ff6640b51d0 con 0x5646419bc450 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff6557fa640 1 -- 172.21.15.105:0/177034576 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6640a02a0 con 0x5646419bc450 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 10 client.15717 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5646415ef0f0 con 0x5646419bc450 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff6557fa640 1 -- 172.21.15.105:0/177034576 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff664005310 con 0x5646419bc450 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:30.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 20 client.15717 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646419cc7d0 0x5646419cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 4 client.15717 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.925+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff6680041c0 con 0x5646419cc7d0 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.926+0000 7ff671a1a640 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646419cc7d0 0x5646419cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.926+0000 7ff6557fa640 10 client.15717 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.926+0000 7ff671a1a640 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646419cc7d0 0x5646419cec90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208d75f1cc3f50ff server_cookie=54455e325b426f23 in_seq=0 out_seq=0 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff6557fa640 1 -- 172.21.15.105:0/177034576 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7ff6680041c0 con 0x5646419cc7d0 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff6557fa640 10 client.15717 handle_command_reply: tid=0 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 client.15717 shutdown 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 msgr2=0x5646419c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c8080 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff6640cd890 tx=0x7ff66409bb70 comp rx=0 tx=0).stop 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 shutdown_connections 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646419cc7d0 0x5646419cec90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c8080 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 --2- 172.21.15.105:0/177034576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.927+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 >> 172.21.15.105:0/177034576 conn(0x5646419b1e60 msgr2=0x5646419b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:30.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.928+0000 7ff6557fa640 0 client.15717 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:30.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.928+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 shutdown_connections 2024-03-26T17:39:30.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.928+0000 7ff676865740 1 -- 172.21.15.105:0/177034576 wait complete. 2024-03-26T17:39:30.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.928+0000 7ff676865740 20 client.15717 trim_cache size 0 max 16384 2024-03-26T17:39:30.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff676865740 1 Processor -- start 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff676865740 1 -- start start 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff676865740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419b68f0 0x5646419c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff676865740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b72e0 0x5646419c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff676865740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419bc450 0x5646419b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff676865740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1a6c30 con 0x5646419b68f0 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff676865740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1a8770 con 0x5646419b72e0 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff676865740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1a7300 con 0x5646419bc450 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419b68f0 0x5646419c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff67221b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419bc450 0x5646419b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419b68f0 0x5646419c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49106/0 (socket says 172.21.15.105:49106) 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff671219640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b72e0 0x5646419c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff671a1a640 1 -- 172.21.15.105:0/1226911388 learned_addr learned my addr 172.21.15.105:0/1226911388 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.953+0000 7ff67221b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419bc450 0x5646419b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41504/0 (socket says 172.21.15.105:41504) 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff671a1a640 1 -- 172.21.15.105:0/1226911388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419bc450 msgr2=0x5646419b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff671a1a640 1 --2- 172.21.15.105:0/1226911388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419bc450 0x5646419b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff671a1a640 1 -- 172.21.15.105:0/1226911388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b72e0 msgr2=0x5646419c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff671a1a640 1 --2- 172.21.15.105:0/1226911388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b72e0 0x5646419c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff671a1a640 1 -- 172.21.15.105:0/1226911388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646419b17b0 con 0x5646419b68f0 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff67221b640 1 --2- 172.21.15.105:0/1226911388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419bc450 0x5646419b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff671a1a640 1 --2- 172.21.15.105:0/1226911388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419b68f0 0x5646419c8ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff6680097e0 tx=0x7ff668029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48657f32dda2fba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff6577fe640 1 -- 172.21.15.105:0/1226911388 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668032b90 con 0x5646419b68f0 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff6577fe640 1 -- 172.21.15.105:0/1226911388 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff668026480 con 0x5646419b68f0 2024-03-26T17:39:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.954+0000 7ff6577fe640 1 -- 172.21.15.105:0/1226911388 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6680269a0 con 0x5646419b68f0 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 -- 172.21.15.105:0/1226911388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419b68f0 msgr2=0x5646419c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 --2- 172.21.15.105:0/1226911388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419b68f0 0x5646419c8ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff6680097e0 tx=0x7ff668029040 comp rx=0 tx=0).stop 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 -- 172.21.15.105:0/1226911388 shutdown_connections 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 --2- 172.21.15.105:0/1226911388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419bc450 0x5646419b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 --2- 172.21.15.105:0/1226911388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b72e0 0x5646419c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 --2- 172.21.15.105:0/1226911388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419b68f0 0x5646419c8ad0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 -- 172.21.15.105:0/1226911388 >> 172.21.15.105:0/1226911388 conn(0x5646419b1e60 msgr2=0x5646419bbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 -- 172.21.15.105:0/1226911388 shutdown_connections 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 -- 172.21.15.105:0/1226911388 wait complete. 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 -- start start 2024-03-26T17:39:30.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.955+0000 7ff676865740 1 Processor -- start 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff676865740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff676865740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff676865740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff676865740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1a80a0 con 0x5646419bc450 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff676865740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1a79d0 con 0x5646419b68f0 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff676865740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff66c1a4c10 con 0x5646419b72e0 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff676865740 10 client.0 _ll_register_callbacks cb 0x7ff66409bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff671219640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff67221b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff671a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41076/0 (socket says 172.21.15.105:41076) 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff671219640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41518/0 (socket says 172.21.15.105:41518) 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff671a1a640 1 -- 172.21.15.105:0/2267787652 learned_addr learned my addr 172.21.15.105:0/2267787652 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff67221b640 1 -- 172.21.15.105:0/2267787652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 msgr2=0x5646419c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:30.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff67221b640 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff67221b640 1 -- 172.21.15.105:0/2267787652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 msgr2=0x5646419c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff67221b640 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff67221b640 1 -- 172.21.15.105:0/2267787652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646419c6bc0 con 0x5646419bc450 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff671a1a640 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff67221b640 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c65d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff6640049d0 tx=0x7ff6640cd3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00306e8ed6bfc16 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.956+0000 7ff6557fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff6557fa640 1 -- 172.21.15.105:0/2267787652 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66409bf20 con 0x5646419bc450 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff6557fa640 1 -- 172.21.15.105:0/2267787652 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7ff664097ea0 con 0x5646419bc450 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 1 -- 172.21.15.105:0/2267787652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff668026480 con 0x5646419bc450 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff6557fa640 1 -- 172.21.15.105:0/2267787652 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6640a4b60 con 0x5646419bc450 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 10 client.15723 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 1 -- 172.21.15.105:0/2267787652 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5646419c7f40 con 0x5646419bc450 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff6557fa640 1 -- 172.21.15.105:0/2267787652 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff668026480 con 0x5646419bc450 2024-03-26T17:39:30.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff6557fa640 1 -- 172.21.15.105:0/2267787652 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6640a1430 con 0x5646419bc450 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 20 client.15723 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564641a32f10 0x564641a35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 4 client.15723 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.957+0000 7ff676865740 1 -- 172.21.15.105:0/2267787652 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564641928ec0 con 0x564641a32f10 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.958+0000 7ff671a1a640 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564641a32f10 0x564641a35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.958+0000 7ff6557fa640 10 client.15723 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.958+0000 7ff671a1a640 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564641a32f10 0x564641a35330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e36ae87171fe571 server_cookie=914c941cafefc652 in_seq=0 out_seq=0 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.958+0000 7ff6557fa640 1 -- 172.21.15.105:0/2267787652 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564641928ec0 con 0x564641a32f10 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.958+0000 7ff6557fa640 10 client.15723 handle_command_reply: tid=0 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.958+0000 7ff676865740 1 client.15723 shutdown 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 -- 172.21.15.105:0/2267787652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 msgr2=0x5646419c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c65d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff6640049d0 tx=0x7ff6640cd3f0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 -- 172.21.15.105:0/2267787652 shutdown_connections 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564641a32f10 0x564641a35330 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646419bc450 0x5646419c65d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff6557fa640 0 client.15723 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646419b72e0 0x5646419c55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 --2- 172.21.15.105:0/2267787652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646419b68f0 0x5646419c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 -- 172.21.15.105:0/2267787652 >> 172.21.15.105:0/2267787652 conn(0x5646419b1e60 msgr2=0x5646419c0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 -- 172.21.15.105:0/2267787652 shutdown_connections 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 1 -- 172.21.15.105:0/2267787652 wait complete. 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.959+0000 7ff676865740 20 client.15723 trim_cache size 0 max 16384 2024-03-26T17:39:30.958 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:39:30.959 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:39:30.959 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:39:30.959 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 -- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6580a2fa0 msgr2=0x7ff6580a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6580a2fa0 0x7ff6580a5460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6680095f0 tx=0x7ff668005f90 comp rx=0 tx=0).stop 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 -- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c107640 msgr2=0x7ff66c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c107640 0x7ff66c19f130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff66400eb10 tx=0x7ff66400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 -- 172.21.15.105:0/3046581744 shutdown_connections 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6580a2fa0 0x7ff6580a5460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff66c107640 0x7ff66c19f130 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66c104d10 0x7ff66c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 --2- 172.21.15.105:0/3046581744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff66c1023e0 0x7ff66c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:30.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 -- 172.21.15.105:0/3046581744 >> 172.21.15.105:0/3046581744 conn(0x7ff66c0fbd50 msgr2=0x7ff66c102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 -- 172.21.15.105:0/3046581744 shutdown_connections 2024-03-26T17:39:30.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:30.960+0000 7ff673c7e640 1 -- 172.21.15.105:0/3046581744 wait complete. 2024-03-26T17:39:30.965 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:39:30.965 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:39:31.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 8594 4.59 MB/sec warmup 14 sec latency 18.839 ms 2024-03-26T17:39:31.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 -- 172.21.15.105:0/1213001678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201007e0 msgr2=0x7fa420100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 --2- 172.21.15.105:0/1213001678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201007e0 0x7fa420100c00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa410009860 tx=0x7fa41002bb70 comp rx=0 tx=0).stop 2024-03-26T17:39:31.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 -- 172.21.15.105:0/1213001678 shutdown_connections 2024-03-26T17:39:31.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 --2- 172.21.15.105:0/1213001678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa420102390 0x7fa420108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 --2- 172.21.15.105:0/1213001678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4201019d0 0x7fa420101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 --2- 172.21.15.105:0/1213001678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201007e0 0x7fa420100c00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 -- 172.21.15.105:0/1213001678 >> 172.21.15.105:0/1213001678 conn(0x7fa4200fbd50 msgr2=0x7fa4200fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 -- 172.21.15.105:0/1213001678 shutdown_connections 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.085+0000 7fa425e4b640 1 -- 172.21.15.105:0/1213001678 wait complete. 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa425e4b640 1 Processor -- start 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa425e4b640 1 -- start start 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa425e4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4201007e0 0x7fa420195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa425e4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201019d0 0x7fa420195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa425e4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa420102390 0x7fa42019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa425e4b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa42010b220 con 0x7fa4201007e0 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa425e4b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa42010b0a0 con 0x7fa420102390 2024-03-26T17:39:31.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa425e4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa42010b3a0 con 0x7fa4201019d0 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4201007e0 0x7fa420195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201019d0 0x7fa420195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4201007e0 0x7fa420195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49132/0 (socket says 172.21.15.105:49132) 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 -- 172.21.15.105:0/2517393570 learned_addr learned my addr 172.21.15.105:0/2517393570 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa420102390 0x7fa42019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 -- 172.21.15.105:0/2517393570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201019d0 msgr2=0x7fa420195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201019d0 0x7fa420195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 -- 172.21.15.105:0/2517393570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa420102390 msgr2=0x7fa42019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa420102390 0x7fa42019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 -- 172.21.15.105:0/2517393570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42019d530 con 0x7fa4201007e0 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41effd640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201019d0 0x7fa420195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.086+0000 7fa41f7fe640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4201007e0 0x7fa420195910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa410005940 tx=0x7fa4100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.087+0000 7fa41cff9640 1 -- 172.21.15.105:0/2517393570 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410041020 con 0x7fa4201007e0 2024-03-26T17:39:31.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.087+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42019d7c0 con 0x7fa4201007e0 2024-03-26T17:39:31.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.087+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42019dd00 con 0x7fa4201007e0 2024-03-26T17:39:31.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.087+0000 7fa41cff9640 1 -- 172.21.15.105:0/2517393570 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4100035f0 con 0x7fa4201007e0 2024-03-26T17:39:31.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.087+0000 7fa41cff9640 1 -- 172.21.15.105:0/2517393570 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410039040 con 0x7fa4201007e0 2024-03-26T17:39:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.088+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c0052c0 con 0x7fa4201007e0 2024-03-26T17:39:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.089+0000 7fa41cff9640 1 -- 172.21.15.105:0/2517393570 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa410044020 con 0x7fa4201007e0 2024-03-26T17:39:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.090+0000 7fa41cff9640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4080a2fa0 0x7fa4080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.090+0000 7fa41effd640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4080a2fa0 0x7fa4080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.090+0000 7fa41cff9640 1 -- 172.21.15.105:0/2517393570 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fa4100f0c80 con 0x7fa4201007e0 2024-03-26T17:39:31.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.090+0000 7fa41effd640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4080a2fa0 0x7fa4080a5460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa420196e20 tx=0x7fa414005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.095+0000 7fa41cff9640 1 -- 172.21.15.105:0/2517393570 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa41004b020 con 0x7fa4201007e0 2024-03-26T17:39:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.373+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40c0055b0 con 0x7fa4201007e0 2024-03-26T17:39:31.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.374+0000 7fa41cff9640 1 -- 172.21.15.105:0/2517393570 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11941 (secure 0 0 0) 0x7fa4100bda70 con 0x7fa4201007e0 2024-03-26T17:39:31.368 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:39:31.368 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:38:20.295050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3121985659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3121985659},{"type":"v1","addr":"172.21.15.135:6835","nonce":3121985659}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T17:39:31.368 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 27 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4080a2fa0 msgr2=0x7fa4080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4080a2fa0 0x7fa4080a5460 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa420196e20 tx=0x7fa414005f90 comp rx=0 tx=0).stop 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4201007e0 msgr2=0x7fa420195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4201007e0 0x7fa420195910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa410005940 tx=0x7fa4100042f0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 shutdown_connections 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4080a2fa0 0x7fa4080a5460 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa420102390 0x7fa42019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201019d0 0x7fa420195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 --2- 172.21.15.105:0/2517393570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4201007e0 0x7fa420195910 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 >> 172.21.15.105:0/2517393570 conn(0x7fa4200fbd50 msgr2=0x7fa420104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 shutdown_connections 2024-03-26T17:39:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.376+0000 7fa425e4b640 1 -- 172.21.15.105:0/2517393570 wait complete. 2024-03-26T17:39:31.382 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:39:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.507+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3888888535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe01018d0 msgr2=0x7fbbe0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.507+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3888888535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe01018d0 0x7fbbe0101cf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009900 tx=0x7fbbd002bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3888888535 shutdown_connections 2024-03-26T17:39:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3888888535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbe0103480 0x7fbbe0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3888888535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe0102ac0 0x7fbbe0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3888888535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe01018d0 0x7fbbe0101cf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3888888535 >> 172.21.15.105:0/3888888535 conn(0x7fbbe00fce40 msgr2=0x7fbbe00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3888888535 shutdown_connections 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3888888535 wait complete. 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 Processor -- start 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 -- start start 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbe01018d0 0x7fbbe0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe0102ac0 0x7fbbe01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.508+0000 7fbbe54ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe0103480 0x7fbbe019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbe54ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe010d3f0 con 0x7fbbe0103480 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbe54ea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe010d270 con 0x7fbbe01018d0 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbe54ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe010d570 con 0x7fbbe0102ac0 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbe01018d0 0x7fbbe0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe0103480 0x7fbbe019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe0102ac0 0x7fbbe01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbe01018d0 0x7fbbe0197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41104/0 (socket says 172.21.15.105:41104) 2024-03-26T17:39:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe0103480 0x7fbbe019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49158/0 (socket says 172.21.15.105:49158) 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdeffd640 1 -- 172.21.15.105:0/3010733187 learned_addr learned my addr 172.21.15.105:0/3010733187 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe0102ac0 0x7fbbe01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41552/0 (socket says 172.21.15.105:41552) 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdf7fe640 1 -- 172.21.15.105:0/3010733187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe0102ac0 msgr2=0x7fbbe01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdf7fe640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe0102ac0 0x7fbbe01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdf7fe640 1 -- 172.21.15.105:0/3010733187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbe01018d0 msgr2=0x7fbbe0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdf7fe640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbe01018d0 0x7fbbe0197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdf7fe640 1 -- 172.21.15.105:0/3010733187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe019f7a0 con 0x7fbbe0103480 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdf7fe640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe0103480 0x7fbbe019f130 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800da10 tx=0x7fbbd800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.509+0000 7fbbdcff9640 1 -- 172.21.15.105:0/3010733187 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800bd70 con 0x7fbbe0103480 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.510+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3010733187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe019fa90 con 0x7fbbe0103480 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.510+0000 7fbbdcff9640 1 -- 172.21.15.105:0/3010733187 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbd8003e20 con 0x7fbbe0103480 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.510+0000 7fbbdcff9640 1 -- 172.21.15.105:0/3010733187 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80106c0 con 0x7fbbe0103480 2024-03-26T17:39:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.510+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3010733187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe019ff90 con 0x7fbbe0103480 2024-03-26T17:39:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.512+0000 7fbbdcff9640 1 -- 172.21.15.105:0/3010733187 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbbd8003970 con 0x7fbbe0103480 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbe81b5740 1 Processor -- start 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbe81b5740 1 -- start start 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89813a30 0x558e89813e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89818f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e8981ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558e89778970 con 0x558e89813a30 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558e89720d70 con 0x558e89814420 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558e89774920 con 0x558e89819590 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbdcff9640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbcc0a2f20 0x7fbbcc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89813a30 0x558e89813e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89813a30 0x558e89813e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49164/0 (socket says 172.21.15.105:49164) 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbd7fff640 1 -- 172.21.15.105:0/2312357040 learned_addr learned my addr 172.21.15.105:0/2312357040 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.513+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89818f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdf7fe640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbcc0a2f20 0x7fbbcc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbd7fff640 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e8981ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdcff9640 1 -- 172.21.15.105:0/3010733187 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fbbd80cf780 con 0x7fbbe0103480 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdeffd640 1 -- 172.21.15.105:0/2312357040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 msgr2=0x558e8981ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e8981ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdeffd640 1 -- 172.21.15.105:0/2312357040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89813a30 msgr2=0x558e89813e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89813a30 0x558e89813e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdeffd640 1 -- 172.21.15.105:0/2312357040 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e8948c3f0 con 0x558e89814420 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbd7fff640 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e8981ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdf7fe640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbcc0a2f20 0x7fbbcc0a53e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbd809be00 tx=0x7fbbd806eba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89818f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0006010 tx=0x7fbbd0005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd9635d2ab0a448 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbd57fa640 1 -- 172.21.15.105:0/2312357040 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0045020 con 0x558e89814420 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbd57fa640 1 -- 172.21.15.105:0/2312357040 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbd0004040 con 0x558e89814420 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.514+0000 7fbbd57fa640 1 -- 172.21.15.105:0/2312357040 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00046c0 con 0x558e89814420 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 -- 172.21.15.105:0/2312357040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 msgr2=0x558e89818f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89818f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0006010 tx=0x7fbbd0005e00 comp rx=0 tx=0).stop 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 -- 172.21.15.105:0/2312357040 shutdown_connections 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e8981ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89818f20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/2312357040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89813a30 0x558e89813e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 -- 172.21.15.105:0/2312357040 >> 172.21.15.105:0/2312357040 conn(0x558e8980efa0 msgr2=0x558e8980f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 -- 172.21.15.105:0/2312357040 shutdown_connections 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.515+0000 7fbbe81b5740 1 -- 172.21.15.105:0/2312357040 wait complete. 2024-03-26T17:39:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 1 -- start start 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 1 Processor -- start 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 0x558e8981df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89814420 0x558e8981e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e89825490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558e8981c4e0 con 0x558e89814420 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558e89734310 con 0x558e89813a30 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558e8981c660 con 0x558e89819590 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89814420 0x558e8981e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 0x558e8981df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89814420 0x558e8981e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49166/0 (socket says 172.21.15.105:49166) 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbe81b5740 10 client.0 _ll_register_callbacks cb 0x558e8976fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 -- 172.21.15.105:0/1466946335 learned_addr learned my addr 172.21.15.105:0/1466946335 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbd7fff640 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e89825490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 -- 172.21.15.105:0/1466946335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 msgr2=0x558e89825490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e89825490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 -- 172.21.15.105:0/1466946335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 msgr2=0x558e8981df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 0x558e8981df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 -- 172.21.15.105:0/1466946335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e895567d0 con 0x558e89814420 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.516+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89814420 0x558e8981e450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005770 tx=0x7fbbd002c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a7b6dcc90e5d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbc67fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1466946335 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd004e030 con 0x558e89814420 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1466946335 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fbbd00035f0 con 0x558e89814420 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1466946335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd0004040 con 0x558e89814420 2024-03-26T17:39:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1466946335 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbd0045070 con 0x558e89814420 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 10 client.15741 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1466946335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e898287e0 con 0x558e89814420 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1466946335 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbd0004040 con 0x558e89814420 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1466946335 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbbd00419e0 con 0x558e89814420 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 10 client.15741 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 20 client.15741 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558e89829b30 0x558e8982bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 4 client.15741 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.517+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1466946335 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e89785eb0 con 0x558e89829b30 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.518+0000 7fbbd7fff640 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558e89829b30 0x558e8982bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.518+0000 7fbbd7fff640 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558e89829b30 0x558e8982bfa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=958d51288b639427 server_cookie=95b91868ff012e79 in_seq=0 out_seq=0 2024-03-26T17:39:31.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.518+0000 7fbbc67fc640 10 client.15741 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:39:31.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1466946335 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558e89785eb0 con 0x558e89829b30 2024-03-26T17:39:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbc67fc640 10 client.15741 handle_command_reply: tid=0 2024-03-26T17:39:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 client.15741 shutdown 2024-03-26T17:39:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1466946335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89814420 msgr2=0x558e8981e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89814420 0x558e8981e450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005770 tx=0x7fbbd002c9e0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1466946335 shutdown_connections 2024-03-26T17:39:31.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558e89829b30 0x558e8982bfa0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89819590 0x558e89825490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89814420 0x558e8981e450 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1466946335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 0x558e8981df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1466946335 >> 172.21.15.105:0/1466946335 conn(0x558e8980efa0 msgr2=0x558e8980e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbc67fc640 0 client.15741 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:39:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1466946335 shutdown_connections 2024-03-26T17:39:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1466946335 wait complete. 2024-03-26T17:39:31.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.519+0000 7fbbe81b5740 20 client.15741 trim_cache size 0 max 16384 2024-03-26T17:39:31.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbe81b5740 1 Processor -- start 2024-03-26T17:39:31.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbe81b5740 1 -- start start 2024-03-26T17:39:31.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 0x558e89815960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89814420 0x558e89815ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 0x558e89825450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe01a8ca0 con 0x558e89819590 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe01aa7e0 con 0x558e89813a30 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe01a9370 con 0x558e89814420 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 0x558e89825450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89814420 0x558e89815ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 0x558e89815960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89814420 0x558e89815ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41598/0 (socket says 172.21.15.105:41598) 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbdeffd640 1 -- 172.21.15.105:0/3841503800 learned_addr learned my addr 172.21.15.105:0/3841503800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbdeffd640 1 -- 172.21.15.105:0/3841503800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 msgr2=0x558e89815960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/3841503800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 0x558e89815960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbdeffd640 1 -- 172.21.15.105:0/3841503800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 msgr2=0x558e89825450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/3841503800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 0x558e89825450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.533+0000 7fbbdeffd640 1 -- 172.21.15.105:0/3841503800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e898095d0 con 0x558e89814420 2024-03-26T17:39:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.534+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/3841503800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89814420 0x558e89815ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbbd002c840 tx=0x7fbbd00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d89f4ba9a1c74bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.534+0000 7fbbd4ff9640 1 -- 172.21.15.105:0/3841503800 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd004e070 con 0x558e89814420 2024-03-26T17:39:31.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.534+0000 7fbbd4ff9640 1 -- 172.21.15.105:0/3841503800 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbd004a650 con 0x558e89814420 2024-03-26T17:39:31.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.534+0000 7fbbd4ff9640 1 -- 172.21.15.105:0/3841503800 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0041800 con 0x558e89814420 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- 172.21.15.105:0/3841503800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89814420 msgr2=0x558e89815ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/3841503800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89814420 0x558e89815ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbbd002c840 tx=0x7fbbd00057f0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- 172.21.15.105:0/3841503800 shutdown_connections 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/3841503800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 0x558e89825450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/3841503800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89814420 0x558e89815ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/3841503800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89813a30 0x558e89815960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- 172.21.15.105:0/3841503800 >> 172.21.15.105:0/3841503800 conn(0x558e8980efa0 msgr2=0x558e8981ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- 172.21.15.105:0/3841503800 shutdown_connections 2024-03-26T17:39:31.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- 172.21.15.105:0/3841503800 wait complete. 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- start start 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 Processor -- start 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89813a30 0x558e898224f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89822a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 0x558e89823a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe01aa110 con 0x558e89819590 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe01a9a40 con 0x558e89814420 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe01a6c80 con 0x558e89813a30 2024-03-26T17:39:31.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.535+0000 7fbbe81b5740 10 client.0 _ll_register_callbacks cb 0x7fbbc8009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 0x558e89823a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89813a30 0x558e898224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89822a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbd7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89813a30 0x558e898224f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41612/0 (socket says 172.21.15.105:41612) 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89822a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41160/0 (socket says 172.21.15.105:41160) 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbd7fff640 1 -- 172.21.15.105:0/1939781783 learned_addr learned my addr 172.21.15.105:0/1939781783 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbdeffd640 1 -- 172.21.15.105:0/1939781783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89813a30 msgr2=0x558e898224f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89813a30 0x558e898224f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbdeffd640 1 -- 172.21.15.105:0/1939781783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 msgr2=0x558e89823a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 0x558e89823a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbdeffd640 1 -- 172.21.15.105:0/1939781783 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e89824030 con 0x558e89814420 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbd7fff640 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89813a30 0x558e898224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbdeffd640 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89822a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0041e20 tx=0x7fbbd004ae90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c6e98fe1043365b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbc67fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:39:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1939781783 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd004e040 con 0x558e89814420 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1939781783 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fbbd0041580 con 0x558e89814420 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.536+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1939781783 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd004a650 con 0x558e89814420 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1939781783 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbd00417a0 con 0x558e89814420 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbe81b5740 10 client.25610 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1939781783 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558e89825330 con 0x558e89814420 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1939781783 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbd004a650 con 0x558e89814420 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1939781783 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbbd0034440 con 0x558e89814420 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbe81b5740 10 client.25610 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbe81b5740 10 client.25610 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbe81b5740 20 client.25610 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558e8988ff10 0x558e89892300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbe81b5740 4 client.25610 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1939781783 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e89780200 con 0x558e8988ff10 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbd7fff640 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558e8988ff10 0x558e89892300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbc67fc640 10 client.25610 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.537+0000 7fbbd7fff640 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558e8988ff10 0x558e89892300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7780350ba6363c4b server_cookie=dcffc51343cb6551 in_seq=0 out_seq=0 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbc67fc640 1 -- 172.21.15.105:0/1939781783 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e89780200 con 0x558e8988ff10 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbc67fc640 10 client.25610 handle_command_reply: tid=0 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 client.25610 shutdown 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1939781783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 msgr2=0x558e89822a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89822a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0041e20 tx=0x7fbbd004ae90 comp rx=0 tx=0).stop 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1939781783 shutdown_connections 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558e8988ff10 0x558e89892300 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558e89819590 0x558e89823a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558e89814420 0x558e89822a30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbc67fc640 0 client.25610 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 --2- 172.21.15.105:0/1939781783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558e89813a30 0x558e898224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1939781783 >> 172.21.15.105:0/1939781783 conn(0x558e8980efa0 msgr2=0x558e8981def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1939781783 shutdown_connections 2024-03-26T17:39:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 1 -- 172.21.15.105:0/1939781783 wait complete. 2024-03-26T17:39:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe81b5740 20 client.25610 trim_cache size 0 max 16384 2024-03-26T17:39:31.538 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:39:31.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbcc0a2f20 msgr2=0x7fbbcc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbcc0a2f20 0x7fbbcc0a53e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbbd809be00 tx=0x7fbbd806eba0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.538+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe0103480 msgr2=0x7fbbe019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe0103480 0x7fbbe019f130 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800da10 tx=0x7fbbd800ded0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3010733187 shutdown_connections 2024-03-26T17:39:31.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbcc0a2f20 0x7fbbcc0a53e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbe0103480 0x7fbbe019f130 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbe0102ac0 0x7fbbe01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 --2- 172.21.15.105:0/3010733187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbbe01018d0 0x7fbbe0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3010733187 >> 172.21.15.105:0/3010733187 conn(0x7fbbe00fce40 msgr2=0x7fbbe0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3010733187 shutdown_connections 2024-03-26T17:39:31.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.539+0000 7fbbe54ea640 1 -- 172.21.15.105:0/3010733187 wait complete. 2024-03-26T17:39:31.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 -- 172.21.15.105:0/4186106443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 msgr2=0x7fb104101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 --2- 172.21.15.105:0/4186106443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 0x7fb104101e30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009900 tx=0x7fb0f802bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 -- 172.21.15.105:0/4186106443 shutdown_connections 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 --2- 172.21.15.105:0/4186106443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb104102370 0x7fb104108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 --2- 172.21.15.105:0/4186106443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 0x7fb104101e30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 --2- 172.21.15.105:0/4186106443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1041007c0 0x7fb104100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 -- 172.21.15.105:0/4186106443 >> 172.21.15.105:0/4186106443 conn(0x7fb1040fbd50 msgr2=0x7fb1040fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 -- 172.21.15.105:0/4186106443 shutdown_connections 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 -- 172.21.15.105:0/4186106443 wait complete. 2024-03-26T17:39:31.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 Processor -- start 2024-03-26T17:39:31.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.671+0000 7fb10a587640 1 -- start start 2024-03-26T17:39:31.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb10a587640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1041007c0 0x7fb1040695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb10a587640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 0x7fb10419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb10a587640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb104102370 0x7fb10419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb10a587640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb10410c2d0 con 0x7fb1041019b0 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb10a587640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb10410c150 con 0x7fb104102370 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb10a587640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb10410c450 con 0x7fb1041007c0 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 0x7fb10419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 0x7fb10419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49204/0 (socket says 172.21.15.105:49204) 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 -- 172.21.15.105:0/3492058435 learned_addr learned my addr 172.21.15.105:0/3492058435 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb103fff640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1041007c0 0x7fb1040695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb108b24640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb104102370 0x7fb10419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 -- 172.21.15.105:0/3492058435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1041007c0 msgr2=0x7fb1040695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1041007c0 0x7fb1040695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 -- 172.21.15.105:0/3492058435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb104102370 msgr2=0x7fb10419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb104102370 0x7fb10419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 -- 172.21.15.105:0/3492058435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10419f770 con 0x7fb1041019b0 2024-03-26T17:39:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.672+0000 7fb1037fe640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 0x7fb10419c870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8006010 tx=0x7fb0f8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.673+0000 7fb1017fa640 1 -- 172.21.15.105:0/3492058435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8043020 con 0x7fb1041019b0 2024-03-26T17:39:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.673+0000 7fb1017fa640 1 -- 172.21.15.105:0/3492058435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0f80043d0 con 0x7fb1041019b0 2024-03-26T17:39:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.673+0000 7fb10a587640 1 -- 172.21.15.105:0/3492058435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10419fa00 con 0x7fb1041019b0 2024-03-26T17:39:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.673+0000 7fb1017fa640 1 -- 172.21.15.105:0/3492058435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8003b80 con 0x7fb1041019b0 2024-03-26T17:39:31.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.673+0000 7fb10a587640 1 -- 172.21.15.105:0/3492058435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10419ff50 con 0x7fb1041019b0 2024-03-26T17:39:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.675+0000 7fb10d175740 1 Processor -- start 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.675+0000 7fb10d175740 1 -- start start 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.675+0000 7fb10d175740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x564634034e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.675+0000 7fb10d175740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x564634039f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.675+0000 7fb10d175740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x56463403ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.675+0000 7fb10d175740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564633f99970 con 0x564634034a30 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb10d175740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564633f41d70 con 0x56463403a590 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb10d175740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564633f95920 con 0x564634035420 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb103fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x564634039f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb108b24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x564634034e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb103fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x564634039f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41648/0 (socket says 172.21.15.105:41648) 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb103fff640 1 -- 172.21.15.105:0/2949639337 learned_addr learned my addr 172.21.15.105:0/2949639337 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb1017fa640 1 -- 172.21.15.105:0/3492058435 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb0f8046020 con 0x7fb1041019b0 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb1037fe640 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x56463403ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb108b24640 1 -- 172.21.15.105:0/2949639337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 msgr2=0x564634039f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb108b24640 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x564634039f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb108b24640 1 -- 172.21.15.105:0/2949639337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 msgr2=0x56463403ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb108b24640 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x56463403ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb108b24640 1 -- 172.21.15.105:0/2949639337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564633cad3f0 con 0x564634034a30 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb1037fe640 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x56463403ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb108b24640 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x564634034e50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009f00 tx=0x7fb0fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad9b30de990b8e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb0ea7fc640 1 -- 172.21.15.105:0/2949639337 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc004d20 con 0x564634034a30 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb0ea7fc640 1 -- 172.21.15.105:0/2949639337 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0fc004050 con 0x564634034a30 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb0ea7fc640 1 -- 172.21.15.105:0/2949639337 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc005280 con 0x564634034a30 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.676+0000 7fb1017fa640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb0ec0a2fa0 0x7fb0ec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb103fff640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb0ec0a2fa0 0x7fb0ec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb1017fa640 1 -- 172.21.15.105:0/3492058435 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fb0f80f1650 con 0x7fb1041019b0 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 -- 172.21.15.105:0/2949639337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 msgr2=0x564634034e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x564634034e50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009f00 tx=0x7fb0fc011810 comp rx=0 tx=0).stop 2024-03-26T17:39:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb103fff640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb0ec0a2fa0 0x7fb0ec0a5460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x564634035820 tx=0x7fb0f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 -- 172.21.15.105:0/2949639337 shutdown_connections 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x56463403ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x564634039f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 --2- 172.21.15.105:0/2949639337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x564634034e50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 -- 172.21.15.105:0/2949639337 >> 172.21.15.105:0/2949639337 conn(0x56463402ffa0 msgr2=0x5646340303a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 -- 172.21.15.105:0/2949639337 shutdown_connections 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.677+0000 7fb10d175740 1 -- 172.21.15.105:0/2949639337 wait complete. 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 1 -- start start 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 1 Processor -- start 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x56463403ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x56463403f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x5646340464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56463403d540 con 0x564634034a30 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564633f55310 con 0x56463403a590 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56463403d6c0 con 0x564634035420 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb103fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x56463403ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb103fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x56463403ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49222/0 (socket says 172.21.15.105:49222) 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb103fff640 1 -- 172.21.15.105:0/604238868 learned_addr learned my addr 172.21.15.105:0/604238868 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb10d175740 10 client.0 _ll_register_callbacks cb 0x564633f90c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.678+0000 7fb1037fe640 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x56463403f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb108b24640 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x5646340464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb1037fe640 1 -- 172.21.15.105:0/604238868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 msgr2=0x5646340464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb1037fe640 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x5646340464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb1037fe640 1 -- 172.21.15.105:0/604238868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 msgr2=0x56463403ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb1037fe640 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x56463403ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb1037fe640 1 -- 172.21.15.105:0/604238868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564633d777d0 con 0x564634035420 2024-03-26T17:39:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb103fff640 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x56463403ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb1037fe640 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x56463403f4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80bd9e0 tx=0x7fb0f80bdb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faad3dd61cbc76c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb0e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb0e37fe640 1 -- 172.21.15.105:0/604238868 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80be3e0 con 0x564634035420 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb10d175740 1 -- 172.21.15.105:0/604238868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0fc004050 con 0x564634035420 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb0e37fe640 1 -- 172.21.15.105:0/604238868 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fb0f80c6260 con 0x564634035420 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb0e37fe640 1 -- 172.21.15.105:0/604238868 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0f80bdce0 con 0x564634035420 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb10d175740 10 client.25612 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb10d175740 1 -- 172.21.15.105:0/604238868 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56463403d600 con 0x564634035420 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.679+0000 7fb0e37fe640 1 -- 172.21.15.105:0/604238868 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0fc004050 con 0x564634035420 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb0e37fe640 1 -- 172.21.15.105:0/604238868 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb0f80bd130 con 0x564634035420 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb10d175740 10 client.25612 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb10d175740 10 client.25612 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb10d175740 20 client.25612 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb10d175740 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56463404ae70 0x56463404d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb10d175740 4 client.25612 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb10d175740 1 -- 172.21.15.105:0/604238868 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564633fa6eb0 con 0x56463404ae70 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb103fff640 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56463404ae70 0x56463404d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb0e37fe640 10 client.25612 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.680+0000 7fb103fff640 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56463404ae70 0x56463404d2f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=84dbd807e71a9c7d server_cookie=5e6bb1ac670ecdd in_seq=0 out_seq=0 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb0e37fe640 1 -- 172.21.15.105:0/604238868 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564633fa6eb0 con 0x56463404ae70 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb0e37fe640 10 client.25612 handle_command_reply: tid=0 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 client.25612 shutdown 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 -- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 msgr2=0x56463403f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x56463403f4b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80bd9e0 tx=0x7fb0f80bdb70 comp rx=0 tx=0).stop 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 -- 172.21.15.105:0/604238868 shutdown_connections 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56463404ae70 0x56463404d2f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x5646340464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x56463403f4b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 --2- 172.21.15.105:0/604238868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634034a30 0x56463403ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb0e37fe640 0 client.25612 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:39:31.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 -- 172.21.15.105:0/604238868 >> 172.21.15.105:0/604238868 conn(0x56463402ffa0 msgr2=0x56463402f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 -- 172.21.15.105:0/604238868 shutdown_connections 2024-03-26T17:39:31.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 1 -- 172.21.15.105:0/604238868 wait complete. 2024-03-26T17:39:31.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.681+0000 7fb10d175740 20 client.25612 trim_cache size 0 max 16384 2024-03-26T17:39:31.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb10d175740 1 Processor -- start 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb10d175740 1 -- start start 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb10d175740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564634034a30 0x56463402e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb10d175740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x5646340464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb10d175740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56463403a590 0x56463402ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb10d175740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041a8df0 con 0x56463403a590 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb10d175740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041aa930 con 0x564634034a30 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb10d175740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041a94c0 con 0x564634035420 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb108b24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56463403a590 0x56463402ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x5646340464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb103fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564634034a30 0x56463402e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb108b24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56463403a590 0x56463402ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49234/0 (socket says 172.21.15.105:49234) 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.698+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x5646340464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41670/0 (socket says 172.21.15.105:41670) 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb108b24640 1 -- 172.21.15.105:0/4098792828 learned_addr learned my addr 172.21.15.105:0/4098792828 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb108b24640 1 -- 172.21.15.105:0/4098792828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 msgr2=0x5646340464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb108b24640 1 --2- 172.21.15.105:0/4098792828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x5646340464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb108b24640 1 -- 172.21.15.105:0/4098792828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564634034a30 msgr2=0x56463402e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb108b24640 1 --2- 172.21.15.105:0/4098792828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564634034a30 0x56463402e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb108b24640 1 -- 172.21.15.105:0/4098792828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463404d970 con 0x56463403a590 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb103fff640 1 --2- 172.21.15.105:0/4098792828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564634034a30 0x56463402e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb108b24640 1 --2- 172.21.15.105:0/4098792828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56463403a590 0x56463402ee70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc01b040 tx=0x7fb0fc003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853dea3932c333b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb0e9ffb640 1 -- 172.21.15.105:0/4098792828 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0050c0 con 0x56463403a590 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb0e9ffb640 1 -- 172.21.15.105:0/4098792828 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0fc005710 con 0x56463403a590 2024-03-26T17:39:31.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.699+0000 7fb0e9ffb640 1 -- 172.21.15.105:0/4098792828 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc019cb0 con 0x56463403a590 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 -- 172.21.15.105:0/4098792828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56463403a590 msgr2=0x56463402ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 --2- 172.21.15.105:0/4098792828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56463403a590 0x56463402ee70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc01b040 tx=0x7fb0fc003b00 comp rx=0 tx=0).stop 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 -- 172.21.15.105:0/4098792828 shutdown_connections 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 --2- 172.21.15.105:0/4098792828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56463403a590 0x56463402ee70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 --2- 172.21.15.105:0/4098792828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634035420 0x5646340464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 --2- 172.21.15.105:0/4098792828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564634034a30 0x56463402e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 -- 172.21.15.105:0/4098792828 >> 172.21.15.105:0/4098792828 conn(0x56463402ffa0 msgr2=0x564634036920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 -- 172.21.15.105:0/4098792828 shutdown_connections 2024-03-26T17:39:31.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 -- 172.21.15.105:0/4098792828 wait complete. 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.700+0000 7fb10d175740 1 -- start start 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb10d175740 1 Processor -- start 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb10d175740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634034a30 0x564634043740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb10d175740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634035420 0x564634043c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb10d175740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x564634044c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb10d175740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041aa260 con 0x564634035420 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb10d175740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041a9b90 con 0x56463403a590 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb10d175740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041a6dd0 con 0x564634034a30 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb10d175740 10 client.0 _ll_register_callbacks cb 0x7fb0f8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634035420 0x564634043c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634035420 0x564634043c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49246/0 (socket says 172.21.15.105:49246) 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 -- 172.21.15.105:0/936806634 learned_addr learned my addr 172.21.15.105:0/936806634 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb103fff640 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634034a30 0x564634043740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb108b24640 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x564634044c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 -- 172.21.15.105:0/936806634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634034a30 msgr2=0x564634043740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634034a30 0x564634043740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 -- 172.21.15.105:0/936806634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 msgr2=0x564634044c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x564634044c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 -- 172.21.15.105:0/936806634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463402fdd0 con 0x564634035420 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb1037fe640 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634035420 0x564634043c80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80ba6b0 tx=0x7fb0f80095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a1ece68b0bc436 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.701+0000 7fb0e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb0e37fe640 1 -- 172.21.15.105:0/936806634 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80c1e30 con 0x564634035420 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb0e37fe640 1 -- 172.21.15.105:0/936806634 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fb0f80b9a80 con 0x564634035420 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 1 -- 172.21.15.105:0/936806634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0fc005710 con 0x564634035420 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb0e37fe640 1 -- 172.21.15.105:0/936806634 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0f80c66f0 con 0x564634035420 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 1 -- 172.21.15.105:0/936806634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564634046420 con 0x564634035420 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb0e37fe640 1 -- 172.21.15.105:0/936806634 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0fc005710 con 0x564634035420 2024-03-26T17:39:31.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb0e37fe640 1 -- 172.21.15.105:0/936806634 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb0f80c2fc0 con 0x564634035420 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 10 client.15765 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 20 client.15765 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5646340b14b0 0x5646340b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 4 client.15765 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb10d175740 1 -- 172.21.15.105:0/936806634 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564633fa1200 con 0x5646340b14b0 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.702+0000 7fb103fff640 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5646340b14b0 0x5646340b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb103fff640 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5646340b14b0 0x5646340b38a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=89b46b64211d8fb3 server_cookie=eb0d4207baa2afe8 in_seq=0 out_seq=0 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb0e37fe640 10 client.15765 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb0e37fe640 1 -- 172.21.15.105:0/936806634 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564633fa1200 con 0x5646340b14b0 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb0e37fe640 10 client.15765 handle_command_reply: tid=0 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 client.15765 shutdown 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 -- 172.21.15.105:0/936806634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634035420 msgr2=0x564634043c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634035420 0x564634043c80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80ba6b0 tx=0x7fb0f80095a0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 -- 172.21.15.105:0/936806634 shutdown_connections 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5646340b14b0 0x5646340b38a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463403a590 0x564634044c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564634035420 0x564634043c80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 --2- 172.21.15.105:0/936806634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564634034a30 0x564634043740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 -- 172.21.15.105:0/936806634 >> 172.21.15.105:0/936806634 conn(0x56463402ffa0 msgr2=0x564634033200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb0e37fe640 0 client.15765 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 -- 172.21.15.105:0/936806634 shutdown_connections 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 1 -- 172.21.15.105:0/936806634 wait complete. 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.703+0000 7fb10d175740 20 client.15765 trim_cache size 0 max 16384 2024-03-26T17:39:31.702 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:39:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 -- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb0ec0a2fa0 msgr2=0x7fb0ec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb0ec0a2fa0 0x7fb0ec0a5460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x564634035820 tx=0x7fb0f4005f90 comp rx=0 tx=0).stop 2024-03-26T17:39:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 -- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 msgr2=0x7fb10419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 0x7fb10419c870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8006010 tx=0x7fb0f8005e40 comp rx=0 tx=0).stop 2024-03-26T17:39:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 -- 172.21.15.105:0/3492058435 shutdown_connections 2024-03-26T17:39:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb0ec0a2fa0 0x7fb0ec0a5460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb104102370 0x7fb10419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1041019b0 0x7fb10419c870 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 --2- 172.21.15.105:0/3492058435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1041007c0 0x7fb1040695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 -- 172.21.15.105:0/3492058435 >> 172.21.15.105:0/3492058435 conn(0x7fb1040fbd50 msgr2=0x7fb104104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 -- 172.21.15.105:0/3492058435 shutdown_connections 2024-03-26T17:39:31.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.704+0000 7fb10a587640 1 -- 172.21.15.105:0/3492058435 wait complete. 2024-03-26T17:39:31.710 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:39:31.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 -- 172.21.15.105:0/1602717583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 msgr2=0x7fc5c0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 --2- 172.21.15.105:0/1602717583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 0x7fc5c0108c00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009ca0 tx=0x7fc5b802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 -- 172.21.15.105:0/1602717583 shutdown_connections 2024-03-26T17:39:31.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 --2- 172.21.15.105:0/1602717583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 0x7fc5c0108c00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 --2- 172.21.15.105:0/1602717583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5c01019b0 0x7fc5c0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 --2- 172.21.15.105:0/1602717583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c01007c0 0x7fc5c0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 -- 172.21.15.105:0/1602717583 >> 172.21.15.105:0/1602717583 conn(0x7fc5c00fbd50 msgr2=0x7fc5c00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 -- 172.21.15.105:0/1602717583 shutdown_connections 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 -- 172.21.15.105:0/1602717583 wait complete. 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 Processor -- start 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.845+0000 7fc5c5729640 1 -- start start 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5c5729640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5c01007c0 0x7fc5c0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5c5729640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c01019b0 0x7fc5c0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5c5729640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 0x7fc5c019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5c5729640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c010c2f0 con 0x7fc5c0102370 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5c5729640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c010c170 con 0x7fc5c01007c0 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5c5729640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c010c470 con 0x7fc5c01019b0 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 0x7fc5c019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5c01007c0 0x7fc5c0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 0x7fc5c019f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49274/0 (socket says 172.21.15.105:49274) 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5c01007c0 0x7fc5c0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41244/0 (socket says 172.21.15.105:41244) 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 -- 172.21.15.105:0/765105645 learned_addr learned my addr 172.21.15.105:0/765105645 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c01019b0 0x7fc5c0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 -- 172.21.15.105:0/765105645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c01019b0 msgr2=0x7fc5c0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c01019b0 0x7fc5c0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 -- 172.21.15.105:0/765105645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5c01007c0 msgr2=0x7fc5c0197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5c01007c0 0x7fc5c0197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 -- 172.21.15.105:0/765105645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c019f730 con 0x7fc5c0102370 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5bf7fe640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 0x7fc5c019f0c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc5b802c700 tx=0x7fc5b80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5a7fff640 1 -- 172.21.15.105:0/765105645 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8041020 con 0x7fc5c0102370 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5a7fff640 1 -- 172.21.15.105:0/765105645 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5b8003e20 con 0x7fc5c0102370 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5c5729640 1 -- 172.21.15.105:0/765105645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c019f9c0 con 0x7fc5c0102370 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.846+0000 7fc5a7fff640 1 -- 172.21.15.105:0/765105645 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8039040 con 0x7fc5c0102370 2024-03-26T17:39:31.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.847+0000 7fc5c5729640 1 -- 172.21.15.105:0/765105645 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c019feb0 con 0x7fc5c0102370 2024-03-26T17:39:31.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.849+0000 7fc5c83f2740 1 Processor -- start 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.849+0000 7fc5a7fff640 1 -- 172.21.15.105:0/765105645 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc5b8044020 con 0x7fc5c0102370 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.849+0000 7fc5c83f2740 1 -- start start 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.849+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c3fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c44f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 0x55a0b6c47a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0b6ba4970 con 0x55a0b6c40420 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0b6b4cd70 con 0x55a0b6c3fa30 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0b6ba0920 con 0x55a0b6c45590 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c44f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5be7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c3fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c44f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49282/0 (socket says 172.21.15.105:49282) 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 -- 172.21.15.105:0/4135014386 learned_addr learned my addr 172.21.15.105:0/4135014386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5be7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c3fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41258/0 (socket says 172.21.15.105:41258) 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 0x55a0b6c47a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5a7fff640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5a80a2f20 0x7fc5a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5bf7fe640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5a80a2f20 0x7fc5a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 -- 172.21.15.105:0/4135014386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 msgr2=0x55a0b6c47a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 0x55a0b6c47a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 -- 172.21.15.105:0/4135014386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 msgr2=0x55a0b6c3fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c3fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 -- 172.21.15.105:0/4135014386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b68b83f0 con 0x55a0b6c40420 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c3fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5a7fff640 1 -- 172.21.15.105:0/765105645 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fc5b80f15c0 con 0x7fc5c0102370 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5beffd640 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c44f20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009ca0 tx=0x7fc5b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e0892fa51e600d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.850+0000 7fc5a57fa640 1 -- 172.21.15.105:0/4135014386 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0004220 con 0x55a0b6c40420 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.851+0000 7fc5a57fa640 1 -- 172.21.15.105:0/4135014386 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5b00043c0 con 0x55a0b6c40420 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.851+0000 7fc5a57fa640 1 -- 172.21.15.105:0/4135014386 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0019760 con 0x55a0b6c40420 2024-03-26T17:39:31.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.851+0000 7fc5bf7fe640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5a80a2f20 0x7fc5a80a53e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ef0d0 tx=0x7fc5b8088800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.851+0000 7fc5c83f2740 1 -- 172.21.15.105:0/4135014386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 msgr2=0x55a0b6c44f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.851+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c44f20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009ca0 tx=0x7fc5b0011810 comp rx=0 tx=0).stop 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 -- 172.21.15.105:0/4135014386 shutdown_connections 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 0x55a0b6c47a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c44f20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/4135014386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c3fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 -- 172.21.15.105:0/4135014386 >> 172.21.15.105:0/4135014386 conn(0x55a0b6c3afa0 msgr2=0x55a0b6c3b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 -- 172.21.15.105:0/4135014386 shutdown_connections 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 -- 172.21.15.105:0/4135014386 wait complete. 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 -- start start 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 Processor -- start 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c49f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c4a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c45590 0x55a0b6c514c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0b6c48540 con 0x55a0b6c40420 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0b6b60310 con 0x55a0b6c45590 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0b6c486c0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5c83f2740 10 client.0 _ll_register_callbacks cb 0x55a0b6b9bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c4a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5be7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c49f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5be7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c49f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41724/0 (socket says 172.21.15.105:41724) 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c4a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49294/0 (socket says 172.21.15.105:49294) 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5be7fc640 1 -- 172.21.15.105:0/842256418 learned_addr learned my addr 172.21.15.105:0/842256418 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.852+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c45590 0x55a0b6c514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5be7fc640 1 -- 172.21.15.105:0/842256418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c45590 msgr2=0x55a0b6c514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c45590 0x55a0b6c514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5be7fc640 1 -- 172.21.15.105:0/842256418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 msgr2=0x55a0b6c4a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c4a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5be7fc640 1 -- 172.21.15.105:0/842256418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b69827d0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c49f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4012d10 tx=0x7fc5b4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85cc540c3c518af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/842256418 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b401a040 con 0x55a0b6c3fa30 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/842256418 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fc5b4011250 con 0x55a0b6c3fa30 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5c83f2740 1 -- 172.21.15.105:0/842256418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b00043c0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/842256418 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5b400fe60 con 0x55a0b6c3fa30 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5c83f2740 10 client.25630 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5c83f2740 1 -- 172.21.15.105:0/842256418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0b6c548a0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.853+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/842256418 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5b00043c0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/842256418 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc5b40047a0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5c83f2740 10 client.25630 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:31.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5c83f2740 10 client.25630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:39:31.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5c83f2740 20 client.25630 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:31.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0b6c56300 0x55a0b6c587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5c83f2740 4 client.25630 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:31.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5c83f2740 1 -- 172.21.15.105:0/842256418 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0b6bb1eb0 con 0x55a0b6c56300 2024-03-26T17:39:31.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5beffd640 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0b6c56300 0x55a0b6c587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5a2ff5640 10 client.25630 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:39:31.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.854+0000 7fc5beffd640 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0b6c56300 0x55a0b6c587a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1fe15cb28994c719 server_cookie=6fd590301c85f8d5 in_seq=0 out_seq=0 2024-03-26T17:39:31.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.855+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/842256418 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a0b6bb1eb0 con 0x55a0b6c56300 2024-03-26T17:39:31.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.855+0000 7fc5a2ff5640 10 client.25630 handle_command_reply: tid=0 2024-03-26T17:39:31.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.855+0000 7fc5c83f2740 1 client.25630 shutdown 2024-03-26T17:39:31.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.855+0000 7fc5c83f2740 1 -- 172.21.15.105:0/842256418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c3fa30 msgr2=0x55a0b6c49f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.855+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c49f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4012d10 tx=0x7fc5b4004730 comp rx=0 tx=0).stop 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 1 -- 172.21.15.105:0/842256418 shutdown_connections 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0b6c56300 0x55a0b6c587a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c45590 0x55a0b6c514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c40420 0x55a0b6c4a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/842256418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c49f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 1 -- 172.21.15.105:0/842256418 >> 172.21.15.105:0/842256418 conn(0x55a0b6c3afa0 msgr2=0x55a0b6c3a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5a2ff5640 0 client.25630 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 1 -- 172.21.15.105:0/842256418 shutdown_connections 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 1 -- 172.21.15.105:0/842256418 wait complete. 2024-03-26T17:39:31.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.856+0000 7fc5c83f2740 20 client.25630 trim_cache size 0 max 16384 2024-03-26T17:39:31.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5c83f2740 1 Processor -- start 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5c83f2740 1 -- start start 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c4a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c40420 0x55a0b6c39930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 0x55a0b6c39e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01a6bc0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01a8700 con 0x55a0b6c40420 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01a7290 con 0x55a0b6c45590 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c4a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c40420 0x55a0b6c39930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c4a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49300/0 (socket says 172.21.15.105:49300) 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 -- 172.21.15.105:0/3302124110 learned_addr learned my addr 172.21.15.105:0/3302124110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c40420 0x55a0b6c39930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41276/0 (socket says 172.21.15.105:41276) 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 0x55a0b6c39e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 -- 172.21.15.105:0/3302124110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 msgr2=0x55a0b6c39e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 0x55a0b6c39e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 -- 172.21.15.105:0/3302124110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c40420 msgr2=0x55a0b6c39930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c40420 0x55a0b6c39930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 -- 172.21.15.105:0/3302124110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b6c3a8f0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5beffd640 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c40420 0x55a0b6c39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.870+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c4a260 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4006010 tx=0x7fc5b4012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d480c0104f8360bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5a4ff9640 1 -- 172.21.15.105:0/3302124110 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4011270 con 0x55a0b6c3fa30 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5a4ff9640 1 -- 172.21.15.105:0/3302124110 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5b40052e0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5a4ff9640 1 -- 172.21.15.105:0/3302124110 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b402b6b0 con 0x55a0b6c3fa30 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 -- 172.21.15.105:0/3302124110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c3fa30 msgr2=0x55a0b6c4a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c4a260 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4006010 tx=0x7fc5b4012b10 comp rx=0 tx=0).stop 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 -- 172.21.15.105:0/3302124110 shutdown_connections 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c45590 0x55a0b6c39e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c40420 0x55a0b6c39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/3302124110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c4a260 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 -- 172.21.15.105:0/3302124110 >> 172.21.15.105:0/3302124110 conn(0x55a0b6c3afa0 msgr2=0x55a0b6c49b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 -- 172.21.15.105:0/3302124110 shutdown_connections 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.871+0000 7fc5c83f2740 1 -- 172.21.15.105:0/3302124110 wait complete. 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 1 -- start start 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 1 Processor -- start 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c5da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c40420 0x55a0b6c5dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c45590 0x55a0b6cbac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01a8030 con 0x55a0b6c45590 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01a7960 con 0x55a0b6c3fa30 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01a4ba0 con 0x55a0b6c40420 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5c83f2740 10 client.0 _ll_register_callbacks cb 0x7fc5b4012560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:31.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c45590 0x55a0b6cbac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c45590 0x55a0b6cbac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49310/0 (socket says 172.21.15.105:49310) 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5beffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c40420 0x55a0b6c5dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 -- 172.21.15.105:0/1664944263 learned_addr learned my addr 172.21.15.105:0/1664944263 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c5da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 -- 172.21.15.105:0/1664944263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c40420 msgr2=0x55a0b6c5dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c40420 0x55a0b6c5dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 -- 172.21.15.105:0/1664944263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 msgr2=0x55a0b6c5da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c5da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5be7fc640 1 -- 172.21.15.105:0/1664944263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b6c518e0 con 0x55a0b6c45590 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.872+0000 7fc5beffd640 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c40420 0x55a0b6c5dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5be7fc640 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c45590 0x55a0b6cbac00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40097c0 tx=0x7fc5b4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9382f76628771856 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/1664944263 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4011020 con 0x55a0b6c45590 2024-03-26T17:39:31.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/1664944263 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fc5b40111c0 con 0x55a0b6c45590 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 1 -- 172.21.15.105:0/1664944263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b40052e0 con 0x55a0b6c45590 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/1664944263 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5b4012020 con 0x55a0b6c45590 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 1 -- 172.21.15.105:0/1664944263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0b6cbc320 con 0x55a0b6c45590 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/1664944263 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5b40052e0 con 0x55a0b6c45590 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/1664944263 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc5b4036040 con 0x55a0b6c45590 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 10 client.15783 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 20 client.15783 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0b6cbcca0 0x55a0b6cbf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 4 client.15783 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.873+0000 7fc5c83f2740 1 -- 172.21.15.105:0/1664944263 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0b6bac200 con 0x55a0b6cbcca0 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5beffd640 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0b6cbcca0 0x55a0b6cbf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5beffd640 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0b6cbcca0 0x55a0b6cbf090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=990ab359b6e14e8b server_cookie=e1834f46e189b755 in_seq=0 out_seq=0 2024-03-26T17:39:31.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5a2ff5640 10 client.15783 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5a2ff5640 1 -- 172.21.15.105:0/1664944263 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0b6bac200 con 0x55a0b6cbcca0 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5a2ff5640 10 client.15783 handle_command_reply: tid=0 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 client.15783 shutdown 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 -- 172.21.15.105:0/1664944263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c45590 msgr2=0x55a0b6cbac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c45590 0x55a0b6cbac00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40097c0 tx=0x7fc5b4009310 comp rx=0 tx=0).stop 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 -- 172.21.15.105:0/1664944263 shutdown_connections 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0b6c45590 0x55a0b6cbac00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0b6c40420 0x55a0b6c5dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0b6cbcca0 0x55a0b6cbf090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 --2- 172.21.15.105:0/1664944263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0b6c3fa30 0x55a0b6c5da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5a2ff5640 0 client.15783 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 -- 172.21.15.105:0/1664944263 >> 172.21.15.105:0/1664944263 conn(0x55a0b6c3afa0 msgr2=0x55a0b6c39930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 -- 172.21.15.105:0/1664944263 shutdown_connections 2024-03-26T17:39:31.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 1 -- 172.21.15.105:0/1664944263 wait complete. 2024-03-26T17:39:31.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.874+0000 7fc5c83f2740 20 client.15783 trim_cache size 0 max 16384 2024-03-26T17:39:31.875 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 -- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5a80a2f20 msgr2=0x7fc5a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5a80a2f20 0x7fc5a80a53e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ef0d0 tx=0x7fc5b8088800 comp rx=0 tx=0).stop 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 -- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 msgr2=0x7fc5c019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 0x7fc5c019f0c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc5b802c700 tx=0x7fc5b80328a0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 -- 172.21.15.105:0/765105645 shutdown_connections 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5a80a2f20 0x7fc5a80a53e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5c0102370 0x7fc5c019f0c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c01019b0 0x7fc5c0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 --2- 172.21.15.105:0/765105645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5c01007c0 0x7fc5c0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 -- 172.21.15.105:0/765105645 >> 172.21.15.105:0/765105645 conn(0x7fc5c00fbd50 msgr2=0x7fc5c00fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 -- 172.21.15.105:0/765105645 shutdown_connections 2024-03-26T17:39:31.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:31.875+0000 7fc5c5729640 1 -- 172.21.15.105:0/765105645 wait complete. 2024-03-26T17:39:31.881 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:39:32.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 -- 172.21.15.105:0/1909299435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 msgr2=0x7f5fe0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 --2- 172.21.15.105:0/1909299435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 0x7f5fe0100be0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009900 tx=0x7f5fdc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:32.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 -- 172.21.15.105:0/1909299435 shutdown_connections 2024-03-26T17:39:32.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 --2- 172.21.15.105:0/1909299435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe0102370 0x7f5fe0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 --2- 172.21.15.105:0/1909299435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fe01019b0 0x7f5fe0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 --2- 172.21.15.105:0/1909299435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 0x7f5fe0100be0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 -- 172.21.15.105:0/1909299435 >> 172.21.15.105:0/1909299435 conn(0x7f5fe00fbd50 msgr2=0x7f5fe00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 -- 172.21.15.105:0/1909299435 shutdown_connections 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.011+0000 7f5fe868c640 1 -- 172.21.15.105:0/1909299435 wait complete. 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe868c640 1 Processor -- start 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe868c640 1 -- start start 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe868c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 0x7f5fe0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe868c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fe01019b0 0x7f5fe019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe868c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe0102370 0x7f5fe019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe868c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe010c2c0 con 0x7f5fe01007c0 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe868c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe010c140 con 0x7f5fe01019b0 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe868c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe010c440 con 0x7f5fe0102370 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe6428640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 0x7f5fe0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe6428640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 0x7f5fe0069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49338/0 (socket says 172.21.15.105:49338) 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe6428640 1 -- 172.21.15.105:0/2976534735 learned_addr learned my addr 172.21.15.105:0/2976534735 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fe01019b0 0x7f5fe019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.012+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe0102370 0x7f5fe019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe6428640 1 -- 172.21.15.105:0/2976534735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe0102370 msgr2=0x7f5fe019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe6428640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe0102370 0x7f5fe019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe6428640 1 -- 172.21.15.105:0/2976534735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fe01019b0 msgr2=0x7f5fe019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe6428640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fe01019b0 0x7f5fe019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe6428640 1 -- 172.21.15.105:0/2976534735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe019f7b0 con 0x7f5fe01007c0 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fe01019b0 0x7f5fe019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe6428640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 0x7f5fe0069590 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc000c00 tx=0x7f5fdc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fd77fe640 1 -- 172.21.15.105:0/2976534735 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc043020 con 0x7f5fe01007c0 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fd77fe640 1 -- 172.21.15.105:0/2976534735 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fdc004350 con 0x7f5fe01007c0 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe868c640 1 -- 172.21.15.105:0/2976534735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe019f9e0 con 0x7f5fe01007c0 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fd77fe640 1 -- 172.21.15.105:0/2976534735 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc003b80 con 0x7f5fe01007c0 2024-03-26T17:39:32.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.013+0000 7f5fe868c640 1 -- 172.21.15.105:0/2976534735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe019ff30 con 0x7f5fe01007c0 2024-03-26T17:39:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.015+0000 7f5fd77fe640 1 -- 172.21.15.105:0/2976534735 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5fdc046020 con 0x7f5fe01007c0 2024-03-26T17:39:32.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5feb27f740 1 Processor -- start 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5feb27f740 1 -- start start 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf236e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf23bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf23ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5feb27f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5565bf19b970 con 0x5565bf23c590 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5feb27f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5565bf143d70 con 0x5565bf237420 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5feb27f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5565bf197920 con 0x5565bf236a30 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5fd77fe640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fcc0a2f20 0x7f5fcc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5fe5c27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf23ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5fe6428640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf23bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5fe6c29640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf236e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5fe5c27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf23ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49346/0 (socket says 172.21.15.105:49346) 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5fe5c27640 1 -- 172.21.15.105:0/4012903356 learned_addr learned my addr 172.21.15.105:0/4012903356 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fcc0a2f20 0x7f5fcc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.016+0000 7f5fd77fe640 1 -- 172.21.15.105:0/2976534735 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f5fdc0f1580 con 0x7f5fe01007c0 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe6c29640 1 -- 172.21.15.105:0/4012903356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 msgr2=0x5565bf23bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf23bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe6c29640 1 -- 172.21.15.105:0/4012903356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 msgr2=0x5565bf23ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf23ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe6c29640 1 -- 172.21.15.105:0/4012903356 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565beeaf3f0 con 0x5565bf236a30 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf23ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe6428640 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf23bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf236e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ddc8194382f922 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fcc0a2f20 0x7f5fcc0a53e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800fc90 tx=0x7f5fd801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fd4ff9640 1 -- 172.21.15.105:0/4012903356 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8011e30 con 0x5565bf236a30 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fd4ff9640 1 -- 172.21.15.105:0/4012903356 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fd801dbf0 con 0x5565bf236a30 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.017+0000 7f5fd4ff9640 1 -- 172.21.15.105:0/4012903356 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80042b0 con 0x5565bf236a30 2024-03-26T17:39:32.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 -- 172.21.15.105:0/4012903356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 msgr2=0x5565bf236e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf236e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd800f790 comp rx=0 tx=0).stop 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 -- 172.21.15.105:0/4012903356 shutdown_connections 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf23ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf23bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 --2- 172.21.15.105:0/4012903356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf236e50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 -- 172.21.15.105:0/4012903356 >> 172.21.15.105:0/4012903356 conn(0x5565bf231fa0 msgr2=0x5565bf2323a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 -- 172.21.15.105:0/4012903356 shutdown_connections 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.018+0000 7f5feb27f740 1 -- 172.21.15.105:0/4012903356 wait complete. 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 1 -- start start 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 1 Processor -- start 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf23c590 0x5565bf2483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5565bf23f4e0 con 0x5565bf236a30 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5565bf157310 con 0x5565bf237420 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5565bf23f660 con 0x5565bf23c590 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5feb27f740 10 client.0 _ll_register_callbacks cb 0x5565bf192c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6428640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6428640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49354/0 (socket says 172.21.15.105:49354) 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6428640 1 -- 172.21.15.105:0/1256140019 learned_addr learned my addr 172.21.15.105:0/1256140019 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf23c590 0x5565bf2483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6c29640 1 -- 172.21.15.105:0/1256140019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 msgr2=0x5565bf2413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6c29640 1 -- 172.21.15.105:0/1256140019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf236a30 msgr2=0x5565bf240e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6c29640 1 -- 172.21.15.105:0/1256140019 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565bef797d0 con 0x5565bf23c590 2024-03-26T17:39:32.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe6428640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.019+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf23c590 0x5565bf2483e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd8005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee1765a24bc94ed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5fc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/1256140019 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8011e30 con 0x5565bf23c590 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 1 -- 172.21.15.105:0/1256140019 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd801dbf0 con 0x5565bf23c590 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/1256140019 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f5fd80055c0 con 0x5565bf23c590 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/1256140019 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5fd801dda0 con 0x5565bf23c590 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 10 client.25648 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 1 -- 172.21.15.105:0/1256140019 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565bf24b7c0 con 0x5565bf23c590 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/1256140019 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fd801dbf0 con 0x5565bf23c590 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/1256140019 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5fd8026a20 con 0x5565bf23c590 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 10 client.25648 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 10 client.25648 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 20 client.25648 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:32.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5565bf24ce50 0x5565bf24f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 4 client.25648 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.020+0000 7f5feb27f740 1 -- 172.21.15.105:0/1256140019 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565bf1a8eb0 con 0x5565bf24ce50 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.021+0000 7f5fe6428640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5565bf24ce50 0x5565bf24f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.021+0000 7f5fc9ffb640 10 client.25648 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.021+0000 7f5fe6428640 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5565bf24ce50 0x5565bf24f290 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45651497c8c2c812 server_cookie=f582a22dd0c43944 in_seq=0 out_seq=0 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/1256140019 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5565bf1a8eb0 con 0x5565bf24ce50 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5fc9ffb640 10 client.25648 handle_command_reply: tid=0 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 client.25648 shutdown 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 -- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf23c590 msgr2=0x5565bf2483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf23c590 0x5565bf2483e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd8005450 comp rx=0 tx=0).stop 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 -- 172.21.15.105:0/1256140019 shutdown_connections 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5565bf24ce50 0x5565bf24f290 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf23c590 0x5565bf2483e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5fc9ffb640 0 client.25648 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:39:32.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 --2- 172.21.15.105:0/1256140019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 -- 172.21.15.105:0/1256140019 >> 172.21.15.105:0/1256140019 conn(0x5565bf231fa0 msgr2=0x5565bf2311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 -- 172.21.15.105:0/1256140019 shutdown_connections 2024-03-26T17:39:32.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 1 -- 172.21.15.105:0/1256140019 wait complete. 2024-03-26T17:39:32.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.022+0000 7f5feb27f740 20 client.25648 trim_cache size 0 max 16384 2024-03-26T17:39:32.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 9252 4.33 MB/sec warmup 15 sec latency 19.200 ms 2024-03-26T17:39:32.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5feb27f740 1 Processor -- start 2024-03-26T17:39:32.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5feb27f740 1 -- start start 2024-03-26T17:39:32.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf23c590 0x5565bf2388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5feb27f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01a8d70 con 0x5565bf237420 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5feb27f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01aa8b0 con 0x5565bf23c590 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5feb27f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01a9440 con 0x5565bf236a30 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe5c27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe5c27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49360/0 (socket says 172.21.15.105:49360) 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe6428640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe5c27640 1 -- 172.21.15.105:0/3102019358 learned_addr learned my addr 172.21.15.105:0/3102019358 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf23c590 0x5565bf2388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe6428640 1 -- 172.21.15.105:0/3102019358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf23c590 msgr2=0x5565bf2388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe6428640 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf23c590 0x5565bf2388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe6428640 1 -- 172.21.15.105:0/3102019358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf237420 msgr2=0x5565bf2413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe6428640 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe6428640 1 -- 172.21.15.105:0/3102019358 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565bf22c5d0 con 0x5565bf236a30 2024-03-26T17:39:32.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.038+0000 7f5fe6428640 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf240e60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ef040 tx=0x7f5fdc0eef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f4da2c246bd02ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5fcbfff640 1 -- 172.21.15.105:0/3102019358 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0bd7b0 con 0x5565bf236a30 2024-03-26T17:39:32.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5fcbfff640 1 -- 172.21.15.105:0/3102019358 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fdc0be370 con 0x5565bf236a30 2024-03-26T17:39:32.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5fcbfff640 1 -- 172.21.15.105:0/3102019358 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0bd5d0 con 0x5565bf236a30 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 -- 172.21.15.105:0/3102019358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 msgr2=0x5565bf240e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf240e60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ef040 tx=0x7f5fdc0eef10 comp rx=0 tx=0).stop 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 -- 172.21.15.105:0/3102019358 shutdown_connections 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf23c590 0x5565bf2388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf237420 0x5565bf2413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 --2- 172.21.15.105:0/3102019358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf236a30 0x5565bf240e60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 -- 172.21.15.105:0/3102019358 >> 172.21.15.105:0/3102019358 conn(0x5565bf231fa0 msgr2=0x5565bf23fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 -- 172.21.15.105:0/3102019358 shutdown_connections 2024-03-26T17:39:32.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.039+0000 7f5feb27f740 1 -- 172.21.15.105:0/3102019358 wait complete. 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 1 -- start start 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 1 Processor -- start 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf236a30 0x5565bf2ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf237420 0x5565bf2adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf2b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01aa1e0 con 0x5565bf23c590 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01a9b10 con 0x5565bf236a30 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01a6d50 con 0x5565bf237420 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5feb27f740 10 client.0 _ll_register_callbacks cb 0x7f5fd8011b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe6428640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf236a30 0x5565bf2ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe6c29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf2b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe6428640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf236a30 0x5565bf2ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41342/0 (socket says 172.21.15.105:41342) 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe5c27640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf237420 0x5565bf2adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe6428640 1 -- 172.21.15.105:0/408030827 learned_addr learned my addr 172.21.15.105:0/408030827 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe6428640 1 -- 172.21.15.105:0/408030827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf237420 msgr2=0x5565bf2adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe6428640 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf237420 0x5565bf2adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe6428640 1 -- 172.21.15.105:0/408030827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 msgr2=0x5565bf2b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.040+0000 7f5fe6428640 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf2b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fe6428640 1 -- 172.21.15.105:0/408030827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565bf2b50c0 con 0x5565bf236a30 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fe6c29640 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf2b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf237420 0x5565bf2adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fe6428640 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf236a30 0x5565bf2ad600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ba640 tx=0x7f5fdc0c1eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5503ad23f4aad635 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/408030827 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0db060 con 0x5565bf236a30 2024-03-26T17:39:32.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5feb27f740 1 -- 172.21.15.105:0/408030827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fdc0be370 con 0x5565bf236a30 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/408030827 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f5fdc003d20 con 0x5565bf236a30 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/408030827 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc0bdc70 con 0x5565bf236a30 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5feb27f740 10 client.25661 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5feb27f740 1 -- 172.21.15.105:0/408030827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565bf2b63c0 con 0x5565bf236a30 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/408030827 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fdc0be370 con 0x5565bf236a30 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.041+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/408030827 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5fdc0f11a0 con 0x5565bf236a30 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5feb27f740 10 client.25661 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5feb27f740 10 client.25661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5feb27f740 20 client.25661 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5feb27f740 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5565bf24ce50 0x5565bf2b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5feb27f740 4 client.25661 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:39:32.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5feb27f740 1 -- 172.21.15.105:0/408030827 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565bf1a3200 con 0x5565bf24ce50 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5565bf24ce50 0x5565bf2b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5fc9ffb640 10 client.25661 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5fe5c27640 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5565bf24ce50 0x5565bf2b8d90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71fe1273f6e46eb4 server_cookie=dd08dd946df67da4 in_seq=0 out_seq=0 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5fc9ffb640 1 -- 172.21.15.105:0/408030827 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565bf1a3200 con 0x5565bf24ce50 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5fc9ffb640 10 client.25661 handle_command_reply: tid=0 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.042+0000 7f5feb27f740 1 client.25661 shutdown 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 -- 172.21.15.105:0/408030827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf236a30 msgr2=0x5565bf2ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf236a30 0x5565bf2ad600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ba640 tx=0x7f5fdc0c1eb0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 -- 172.21.15.105:0/408030827 shutdown_connections 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5565bf24ce50 0x5565bf2b8d90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5565bf23c590 0x5565bf2b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5565bf237420 0x5565bf2adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fc9ffb640 0 client.25661 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:39:32.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 --2- 172.21.15.105:0/408030827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565bf236a30 0x5565bf2ad600 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 -- 172.21.15.105:0/408030827 >> 172.21.15.105:0/408030827 conn(0x5565bf231fa0 msgr2=0x5565bf235a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 -- 172.21.15.105:0/408030827 shutdown_connections 2024-03-26T17:39:32.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 1 -- 172.21.15.105:0/408030827 wait complete. 2024-03-26T17:39:32.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5feb27f740 20 client.25661 trim_cache size 0 max 16384 2024-03-26T17:39:32.044 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 -- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fcc0a2f20 msgr2=0x7f5fcc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fcc0a2f20 0x7f5fcc0a53e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800fc90 tx=0x7f5fd801e000 comp rx=0 tx=0).stop 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 -- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 msgr2=0x7f5fe0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 0x7f5fe0069590 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc000c00 tx=0x7f5fdc005dc0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 -- 172.21.15.105:0/2976534735 shutdown_connections 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fcc0a2f20 0x7f5fcc0a53e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe0102370 0x7f5fe019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fe01019b0 0x7f5fe019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 --2- 172.21.15.105:0/2976534735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fe01007c0 0x7f5fe0069590 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.043+0000 7f5fe868c640 1 -- 172.21.15.105:0/2976534735 >> 172.21.15.105:0/2976534735 conn(0x7f5fe00fbd50 msgr2=0x7f5fe0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.044+0000 7f5fe868c640 1 -- 172.21.15.105:0/2976534735 shutdown_connections 2024-03-26T17:39:32.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.044+0000 7f5fe868c640 1 -- 172.21.15.105:0/2976534735 wait complete. 2024-03-26T17:39:32.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:31 smithi105 ceph-mon[32280]: pgmap v381: 97 pgs: 97 active+clean; 24 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 2.3 MiB/s wr, 99 op/s 2024-03-26T17:39:32.047 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:31 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2517393570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:39:32.048 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:39:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:31 smithi144 ceph-mon[36095]: pgmap v381: 97 pgs: 97 active+clean; 24 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 2.3 MiB/s wr, 99 op/s 2024-03-26T17:39:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:31 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2517393570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:39:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:31 smithi135 ceph-mon[35636]: pgmap v381: 97 pgs: 97 active+clean; 24 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 2.3 MiB/s wr, 99 op/s 2024-03-26T17:39:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:31 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2517393570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:39:32.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/2241035479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 msgr2=0x7f9f18109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/2241035479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 0x7f9f18109a10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009ca0 tx=0x7f9f1002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/2241035479 shutdown_connections 2024-03-26T17:39:32.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/2241035479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 0x7f9f18109a10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/2241035479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18104c90 0x7f9f18107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/2241035479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18102360 0x7f9f18104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/2241035479 >> 172.21.15.105:0/2241035479 conn(0x7f9f180fbd70 msgr2=0x7f9f180fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/2241035479 shutdown_connections 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/2241035479 wait complete. 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 Processor -- start 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.179+0000 7f9f1e7f4640 1 -- start start 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1e7f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18102360 0x7f9f18197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1e7f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18104c90 0x7f9f18198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1e7f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 0x7f9f1819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1e7f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810d410 con 0x7f9f181075c0 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1e7f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810d290 con 0x7f9f18102360 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1e7f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810d590 con 0x7f9f18104c90 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 0x7f9f1819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18104c90 0x7f9f18198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18102360 0x7f9f18197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 0x7f9f1819f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49398/0 (socket says 172.21.15.105:49398) 2024-03-26T17:39:32.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 -- 172.21.15.105:0/1870583607 learned_addr learned my addr 172.21.15.105:0/1870583607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 -- 172.21.15.105:0/1870583607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18104c90 msgr2=0x7f9f18198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18104c90 0x7f9f18198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 -- 172.21.15.105:0/1870583607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18102360 msgr2=0x7f9f18197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18102360 0x7f9f18197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 -- 172.21.15.105:0/1870583607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1819f700 con 0x7f9f181075c0 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f177fe640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18104c90 0x7f9f18198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18102360 0x7f9f18197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 0x7f9f1819f090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9f10032380 tx=0x7f9f10032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f157fa640 1 -- 172.21.15.105:0/1870583607 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10041020 con 0x7f9f181075c0 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f157fa640 1 -- 172.21.15.105:0/1870583607 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f10003e20 con 0x7f9f181075c0 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.180+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/1870583607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1819f990 con 0x7f9f181075c0 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.181+0000 7f9f157fa640 1 -- 172.21.15.105:0/1870583607 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10039040 con 0x7f9f181075c0 2024-03-26T17:39:32.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.181+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/1870583607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1819fe80 con 0x7f9f181075c0 2024-03-26T17:39:32.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.183+0000 7f9f157fa640 1 -- 172.21.15.105:0/1870583607 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9f10044020 con 0x7f9f181075c0 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f213e3740 1 Processor -- start 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f157fa640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f000a2f20 0x7f9f000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f213e3740 1 -- start start 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f177fe640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f000a2f20 0x7f9f000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f157fa640 1 -- 172.21.15.105:0/1870583607 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f9f100f1500 con 0x7f9f181075c0 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00feb9b0 0x555f00febdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fec3a0 0x555f00feee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f00ffc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f213e3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555f00f50970 con 0x555f00feb9b0 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f213e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555f00ef8d70 con 0x555f00fef500 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f213e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555f00f4c920 con 0x555f00fec3a0 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00feb9b0 0x555f00febdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00feb9b0 0x555f00febdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49400/0 (socket says 172.21.15.105:49400) 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f17fff640 1 -- 172.21.15.105:0/1731799938 learned_addr learned my addr 172.21.15.105:0/1731799938 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.184+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f00ffc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f1cd91640 1 -- 172.21.15.105:0/1731799938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fec3a0 msgr2=0x555f00feee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f177fe640 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fec3a0 0x555f00feee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fec3a0 0x555f00feee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f1cd91640 1 -- 172.21.15.105:0/1731799938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00feb9b0 msgr2=0x555f00febdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00feb9b0 0x555f00febdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f1cd91640 1 -- 172.21.15.105:0/1731799938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f00c643f0 con 0x555f00fef500 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f177fe640 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fec3a0 0x555f00feee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00feb9b0 0x555f00febdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f177fe640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f000a2f20 0x7f9f000a53e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9f18198ff0 tx=0x7f9f0c011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f00ffc370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f100eefc0 tx=0x7f9f1002c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e02ef18cd339ab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9efe7fc640 1 -- 172.21.15.105:0/1731799938 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100be2f0 con 0x555f00fef500 2024-03-26T17:39:32.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9efe7fc640 1 -- 172.21.15.105:0/1731799938 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f100a8050 con 0x555f00fef500 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.185+0000 7f9efe7fc640 1 -- 172.21.15.105:0/1731799938 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100ba260 con 0x555f00fef500 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 -- 172.21.15.105:0/1731799938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 msgr2=0x555f00ffc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f00ffc370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f100eefc0 tx=0x7f9f1002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 -- 172.21.15.105:0/1731799938 shutdown_connections 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f00ffc370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fec3a0 0x555f00feee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1731799938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00feb9b0 0x555f00febdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 -- 172.21.15.105:0/1731799938 >> 172.21.15.105:0/1731799938 conn(0x555f00fe6fa0 msgr2=0x555f00fe73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 -- 172.21.15.105:0/1731799938 shutdown_connections 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 -- 172.21.15.105:0/1731799938 wait complete. 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.186+0000 7f9f213e3740 1 -- start start 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f213e3740 1 Processor -- start 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 0x555f00ffe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ffea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f01005aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f213e3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555f00ffcb30 con 0x555f00fec3a0 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f213e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555f00f0c310 con 0x555f00fef500 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f213e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555f00ffccb0 con 0x555f00feb9b0 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ffea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f213e3740 10 client.0 _ll_register_callbacks cb 0x555f00f47c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ffea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49414/0 (socket says 172.21.15.105:49414) 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 -- 172.21.15.105:0/1708857395 learned_addr learned my addr 172.21.15.105:0/1708857395 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 0x555f00ffe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f01005aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 -- 172.21.15.105:0/1708857395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 msgr2=0x555f00ffe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 0x555f00ffe520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 -- 172.21.15.105:0/1708857395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 msgr2=0x555f01005aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f01005aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 -- 172.21.15.105:0/1708857395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f00d2e7d0 con 0x555f00fec3a0 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9f177fe640 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ffea60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00fb60 tx=0x7f9f0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936985809bd3bb65 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.187+0000 7f9ef77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.188+0000 7f9ef77fe640 1 -- 172.21.15.105:0/1708857395 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c017040 con 0x555f00fec3a0 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.188+0000 7f9ef77fe640 1 -- 172.21.15.105:0/1708857395 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f9f0c004030 con 0x555f00fec3a0 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.188+0000 7f9f213e3740 1 -- 172.21.15.105:0/1708857395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f100a8050 con 0x555f00fec3a0 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.188+0000 7f9ef77fe640 1 -- 172.21.15.105:0/1708857395 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f0c004230 con 0x555f00fec3a0 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.188+0000 7f9f213e3740 10 client.15813 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.188+0000 7f9f213e3740 1 -- 172.21.15.105:0/1708857395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f01008e50 con 0x555f00fec3a0 2024-03-26T17:39:32.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.188+0000 7f9ef77fe640 1 -- 172.21.15.105:0/1708857395 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f100a8050 con 0x555f00fec3a0 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9ef77fe640 1 -- 172.21.15.105:0/1708857395 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9f0c01f410 con 0x555f00fec3a0 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9f213e3740 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9f213e3740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9f213e3740 20 client.15813 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555f0100a5c0 0x555f0100ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9f213e3740 4 client.15813 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9f213e3740 1 -- 172.21.15.105:0/1708857395 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f00f5deb0 con 0x555f0100a5c0 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555f0100a5c0 0x555f0100ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555f0100a5c0 0x555f0100ca00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18305ab9bc9cd78 server_cookie=2e18fde89d28114a in_seq=0 out_seq=0 2024-03-26T17:39:32.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.189+0000 7f9ef77fe640 10 client.15813 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:32.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9ef77fe640 1 -- 172.21.15.105:0/1708857395 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555f00f5deb0 con 0x555f0100a5c0 2024-03-26T17:39:32.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9ef77fe640 10 client.15813 handle_command_reply: tid=0 2024-03-26T17:39:32.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 client.15813 shutdown 2024-03-26T17:39:32.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 -- 172.21.15.105:0/1708857395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 msgr2=0x555f00ffea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ffea60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00fb60 tx=0x7f9f0c016000 comp rx=0 tx=0).stop 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 -- 172.21.15.105:0/1708857395 shutdown_connections 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555f0100a5c0 0x555f0100ca00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f01005aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ffea60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9ef77fe640 0 client.15813 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 --2- 172.21.15.105:0/1708857395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 0x555f00ffe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 -- 172.21.15.105:0/1708857395 >> 172.21.15.105:0/1708857395 conn(0x555f00fe6fa0 msgr2=0x555f00fe61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 -- 172.21.15.105:0/1708857395 shutdown_connections 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.190+0000 7f9f213e3740 1 -- 172.21.15.105:0/1708857395 wait complete. 2024-03-26T17:39:32.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.191+0000 7f9f213e3740 20 client.15813 trim_cache size 0 max 16384 2024-03-26T17:39:32.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.204+0000 7f9f213e3740 1 Processor -- start 2024-03-26T17:39:32.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.204+0000 7f9f213e3740 1 -- start start 2024-03-26T17:39:32.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.204+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 0x555f00ff8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ff92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f01005a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f213e3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a6b30 con 0x555f00fec3a0 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f213e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a8670 con 0x555f00fef500 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f213e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a7200 con 0x555f00feb9b0 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ff92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ff92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49426/0 (socket says 172.21.15.105:49426) 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 -- 172.21.15.105:0/49670264 learned_addr learned my addr 172.21.15.105:0/49670264 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f17fff640 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 0x555f00ff8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f01005a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 -- 172.21.15.105:0/49670264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 msgr2=0x555f00ff8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 0x555f00ff8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 -- 172.21.15.105:0/49670264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 msgr2=0x555f01005a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f01005a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 -- 172.21.15.105:0/49670264 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f00fec780 con 0x555f00fec3a0 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9f177fe640 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ff92f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c01f900 tx=0x7f9f0c009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38ac8f34af5242f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9efdffb640 1 -- 172.21.15.105:0/49670264 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c017070 con 0x555f00fec3a0 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9efdffb640 1 -- 172.21.15.105:0/49670264 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f0c02d030 con 0x555f00fec3a0 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.205+0000 7f9efdffb640 1 -- 172.21.15.105:0/49670264 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c0326b0 con 0x555f00fec3a0 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 -- 172.21.15.105:0/49670264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 msgr2=0x555f00ff92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ff92f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c01f900 tx=0x7f9f0c009d40 comp rx=0 tx=0).stop 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 -- 172.21.15.105:0/49670264 shutdown_connections 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00fef500 0x555f01005a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f00ff92f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 --2- 172.21.15.105:0/49670264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00feb9b0 0x555f00ff8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 -- 172.21.15.105:0/49670264 >> 172.21.15.105:0/49670264 conn(0x555f00fe6fa0 msgr2=0x555f00ffe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 -- 172.21.15.105:0/49670264 shutdown_connections 2024-03-26T17:39:32.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 -- 172.21.15.105:0/49670264 wait complete. 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.206+0000 7f9f213e3740 1 -- start start 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f213e3740 1 Processor -- start 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00feb9b0 0x555f01002c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f01003150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f213e3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fef500 0x555f01004160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f213e3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a7fa0 con 0x555f00fec3a0 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f213e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a78d0 con 0x555f00feb9b0 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f213e3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a4b10 con 0x555f00fef500 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f213e3740 10 client.0 _ll_register_callbacks cb 0x7f9f0c00f540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f01003150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00feb9b0 0x555f01002c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f01003150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49442/0 (socket says 172.21.15.105:49442) 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f1cd91640 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fef500 0x555f01004160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 -- 172.21.15.105:0/887045991 learned_addr learned my addr 172.21.15.105:0/887045991 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:32.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 -- 172.21.15.105:0/887045991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fef500 msgr2=0x555f01004160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fef500 0x555f01004160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 -- 172.21.15.105:0/887045991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00feb9b0 msgr2=0x555f01002c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00feb9b0 0x555f01002c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 -- 172.21.15.105:0/887045991 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f01004750 con 0x555f00fec3a0 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f17fff640 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00feb9b0 0x555f01002c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9f177fe640 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f01003150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c01f880 tx=0x7f9f0c00ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12874f1298c87b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.207+0000 7f9ef77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9ef77fe640 1 -- 172.21.15.105:0/887045991 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c041070 con 0x555f00fec3a0 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9ef77fe640 1 -- 172.21.15.105:0/887045991 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f9f0c00ed80 con 0x555f00fec3a0 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 1 -- 172.21.15.105:0/887045991 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f0c02d030 con 0x555f00fec3a0 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9ef77fe640 1 -- 172.21.15.105:0/887045991 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9f0c04d050 con 0x555f00fec3a0 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 1 -- 172.21.15.105:0/887045991 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f01005a50 con 0x555f00fec3a0 2024-03-26T17:39:32.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9ef77fe640 1 -- 172.21.15.105:0/887045991 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f0c02d030 con 0x555f00fec3a0 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9ef77fe640 1 -- 172.21.15.105:0/887045991 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9f0c03a070 con 0x555f00fec3a0 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 20 client.15819 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555f0100a5c0 0x555f01072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 4 client.15819 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f213e3740 1 -- 172.21.15.105:0/887045991 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f00f58200 con 0x555f0100a5c0 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.208+0000 7f9f17fff640 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555f0100a5c0 0x555f01072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f17fff640 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555f0100a5c0 0x555f01072ca0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1aec095a947b26 server_cookie=d2878151325b3cda in_seq=0 out_seq=0 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9ef77fe640 10 client.15819 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9ef77fe640 1 -- 172.21.15.105:0/887045991 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f00f58200 con 0x555f0100a5c0 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9ef77fe640 10 client.15819 handle_command_reply: tid=0 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 client.15819 shutdown 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 -- 172.21.15.105:0/887045991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 msgr2=0x555f01003150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f01003150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c01f880 tx=0x7f9f0c00ff60 comp rx=0 tx=0).stop 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 -- 172.21.15.105:0/887045991 shutdown_connections 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555f0100a5c0 0x555f01072ca0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f00fef500 0x555f01004160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f00fec3a0 0x555f01003150 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 --2- 172.21.15.105:0/887045991 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f00feb9b0 0x555f01002c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9f213e3740 1 -- 172.21.15.105:0/887045991 >> 172.21.15.105:0/887045991 conn(0x555f00fe6fa0 msgr2=0x555f00ffe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.209+0000 7f9ef77fe640 0 client.15819 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f213e3740 1 -- 172.21.15.105:0/887045991 shutdown_connections 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f213e3740 1 -- 172.21.15.105:0/887045991 wait complete. 2024-03-26T17:39:32.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f213e3740 20 client.15819 trim_cache size 0 max 16384 2024-03-26T17:39:32.211 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:39:32.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f000a2f20 msgr2=0x7f9f000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f000a2f20 0x7f9f000a53e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9f18198ff0 tx=0x7f9f0c011810 comp rx=0 tx=0).stop 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 msgr2=0x7f9f1819f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 0x7f9f1819f090 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9f10032380 tx=0x7f9f10032820 comp rx=0 tx=0).stop 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/1870583607 shutdown_connections 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f000a2f20 0x7f9f000a53e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181075c0 0x7f9f1819f090 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18104c90 0x7f9f18198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 --2- 172.21.15.105:0/1870583607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18102360 0x7f9f18197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/1870583607 >> 172.21.15.105:0/1870583607 conn(0x7f9f180fbd70 msgr2=0x7f9f18102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/1870583607 shutdown_connections 2024-03-26T17:39:32.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:32.210+0000 7f9f1e7f4640 1 -- 172.21.15.105:0/1870583607 wait complete. 2024-03-26T17:39:33.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 9880 4.39 MB/sec warmup 16 sec latency 17.832 ms 2024-03-26T17:39:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.150+0000 7f66c4ee8640 1 -- 172.21.15.105:0/1635348852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 msgr2=0x7f66c0101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.150+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/1635348852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 0x7f66c0101d40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009900 tx=0x7f66b002bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.150+0000 7f66c4ee8640 1 -- 172.21.15.105:0/1635348852 shutdown_connections 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.150+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/1635348852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01034d0 0x7f66c0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.150+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/1635348852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c0102b10 0x7f66c0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.150+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/1635348852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 0x7f66c0101d40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.150+0000 7f66c4ee8640 1 -- 172.21.15.105:0/1635348852 >> 172.21.15.105:0/1635348852 conn(0x7f66c00fce90 msgr2=0x7f66c00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.151+0000 7f66c4ee8640 1 -- 172.21.15.105:0/1635348852 shutdown_connections 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.151+0000 7f66c4ee8640 1 -- 172.21.15.105:0/1635348852 wait complete. 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.151+0000 7f66c4ee8640 1 Processor -- start 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.151+0000 7f66c4ee8640 1 -- start start 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.151+0000 7f66c4ee8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 0x7f66c0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.151+0000 7f66c4ee8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c0102b10 0x7f66c0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66c4ee8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01034d0 0x7f66c019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66c4ee8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010d420 con 0x7f66c0101920 2024-03-26T17:39:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66c4ee8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010d2a0 con 0x7f66c0102b10 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66c4ee8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010d5a0 con 0x7f66c01034d0 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66be59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 0x7f66c0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66be59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 0x7f66c0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49470/0 (socket says 172.21.15.105:49470) 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bdd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c0102b10 0x7f66c0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66be59c640 1 -- 172.21.15.105:0/2039941416 learned_addr learned my addr 172.21.15.105:0/2039941416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01034d0 0x7f66c019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bdd9b640 1 -- 172.21.15.105:0/2039941416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01034d0 msgr2=0x7f66c019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01034d0 0x7f66c019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bdd9b640 1 -- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 msgr2=0x7f66c0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 0x7f66c0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bdd9b640 1 -- 172.21.15.105:0/2039941416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c019f6e0 con 0x7f66c0102b10 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66be59c640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 0x7f66c0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bed9d640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01034d0 0x7f66c019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c0102b10 0x7f66c0198000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f66b400e9f0 tx=0x7f66b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66af7fe640 1 -- 172.21.15.105:0/2039941416 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b400ccc0 con 0x7f66c0102b10 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66c4ee8640 1 -- 172.21.15.105:0/2039941416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c019f9d0 con 0x7f66c0102b10 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66af7fe640 1 -- 172.21.15.105:0/2039941416 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b40108e0 con 0x7f66c0102b10 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.152+0000 7f66af7fe640 1 -- 172.21.15.105:0/2039941416 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4018880 con 0x7f66c0102b10 2024-03-26T17:39:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.153+0000 7f66c4ee8640 1 -- 172.21.15.105:0/2039941416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c019ff70 con 0x7f66c0102b10 2024-03-26T17:39:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.155+0000 7f66af7fe640 1 -- 172.21.15.105:0/2039941416 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f66b4010a90 con 0x7f66c0102b10 2024-03-26T17:39:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.155+0000 7f66c7ba9740 1 Processor -- start 2024-03-26T17:39:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.155+0000 7f66c7ba9740 1 -- start start 2024-03-26T17:39:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.155+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a75d0 0x5591d84a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.155+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.155+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84ad130 0x5591d84af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.155+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5591d840c0b0 con 0x5591d84a75d0 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.155+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5591d83b4960 con 0x5591d84ad130 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5591d8408060 con 0x5591d84a7fc0 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66af7fe640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66a80a2fc0 0x7f66a80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a75d0 0x5591d84a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bdd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84ad130 0x5591d84af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a75d0 0x5591d84a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49486/0 (socket says 172.21.15.105:49486) 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 -- 172.21.15.105:0/3383892873 learned_addr learned my addr 172.21.15.105:0/3383892873 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84ad130 0x5591d84af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41410/0 (socket says 172.21.15.105:41410) 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66be59c640 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66a80a2fc0 0x7f66a80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66af7fe640 1 -- 172.21.15.105:0/2039941416 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f66b40cf760 con 0x7f66c0102b10 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 -- 172.21.15.105:0/3383892873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 msgr2=0x5591d84acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 -- 172.21.15.105:0/3383892873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84ad130 msgr2=0x5591d84af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84ad130 0x5591d84af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 -- 172.21.15.105:0/3383892873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591d83c6c60 con 0x5591d84a75d0 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a75d0 0x5591d84a79f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f66b800f5c0 tx=0x7f66b800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9998f5d82c07432e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66bed9d640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66a80a2fc0 0x7f66a80a5480 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f66b80099e0 tx=0x7f66b801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.156+0000 7f66acff9640 1 -- 172.21.15.105:0/3383892873 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b802b070 con 0x5591d84a75d0 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66acff9640 1 -- 172.21.15.105:0/3383892873 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b8011630 con 0x5591d84a75d0 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66acff9640 1 -- 172.21.15.105:0/3383892873 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8022070 con 0x5591d84a75d0 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3383892873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a75d0 msgr2=0x5591d84a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a75d0 0x5591d84a79f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f66b800f5c0 tx=0x7f66b800fa80 comp rx=0 tx=0).stop 2024-03-26T17:39:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3383892873 shutdown_connections 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84ad130 0x5591d84af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/3383892873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a75d0 0x5591d84a79f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.157+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3383892873 >> 172.21.15.105:0/3383892873 conn(0x5591d84a2b40 msgr2=0x5591d84a2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3383892873 shutdown_connections 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3383892873 wait complete. 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 -- start start 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 Processor -- start 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 0x5591d84b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5591d84b0080 con 0x5591d84ad130 2024-03-26T17:39:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5591d83c7db0 con 0x5591d84a75d0 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5591d84b0200 con 0x5591d84a7fc0 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66c7ba9740 10 client.0 _ll_register_callbacks cb 0x5591d83cc110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66be59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66bed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 0x5591d84b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66bdd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66be59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41416/0 (socket says 172.21.15.105:41416) 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.158+0000 7f66be59c640 1 -- 172.21.15.105:0/569046899 learned_addr learned my addr 172.21.15.105:0/569046899 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66be59c640 1 -- 172.21.15.105:0/569046899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 msgr2=0x5591d84b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66be59c640 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66be59c640 1 -- 172.21.15.105:0/569046899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 msgr2=0x5591d84b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66be59c640 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 0x5591d84b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66be59c640 1 -- 172.21.15.105:0/569046899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591d8120210 con 0x5591d84a75d0 2024-03-26T17:39:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66be59c640 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b1320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f903449a2710b5b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66a5ffb640 1 -- 172.21.15.105:0/569046899 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0005d50 con 0x5591d84a75d0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66c7ba9740 1 -- 172.21.15.105:0/569046899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b8011630 con 0x5591d84a75d0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66a5ffb640 1 -- 172.21.15.105:0/569046899 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f66b0005570 con 0x5591d84a75d0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66a5ffb640 1 -- 172.21.15.105:0/569046899 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66b002cc90 con 0x5591d84a75d0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66c7ba9740 10 client.25691 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.159+0000 7f66c7ba9740 1 -- 172.21.15.105:0/569046899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5591d81ec620 con 0x5591d84a75d0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66a5ffb640 1 -- 172.21.15.105:0/569046899 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b8011630 con 0x5591d84a75d0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66a5ffb640 1 -- 172.21.15.105:0/569046899 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f66b0003a70 con 0x5591d84a75d0 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66c7ba9740 10 client.25691 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66c7ba9740 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66c7ba9740 20 client.25691 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5591d84bd120 0x5591d84bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66c7ba9740 4 client.25691 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66c7ba9740 1 -- 172.21.15.105:0/569046899 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591d84195f0 con 0x5591d84bd120 2024-03-26T17:39:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5591d84bd120 0x5591d84bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5591d84bd120 0x5591d84bf590 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650ee846d72ce7c8 server_cookie=cfe3ea0f38aabb11 in_seq=0 out_seq=0 2024-03-26T17:39:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.160+0000 7f66a5ffb640 10 client.25691 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66a5ffb640 1 -- 172.21.15.105:0/569046899 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5591d84195f0 con 0x5591d84bd120 2024-03-26T17:39:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66a5ffb640 10 client.25691 handle_command_reply: tid=0 2024-03-26T17:39:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 client.25691 shutdown 2024-03-26T17:39:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 -- 172.21.15.105:0/569046899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 msgr2=0x5591d84b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b1320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b0005270 comp rx=0 tx=0).stop 2024-03-26T17:39:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 -- 172.21.15.105:0/569046899 shutdown_connections 2024-03-26T17:39:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5591d84bd120 0x5591d84bf590 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 0x5591d84b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a7fc0 0x5591d84b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66a5ffb640 0 client.25691 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/569046899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b1320 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 -- 172.21.15.105:0/569046899 >> 172.21.15.105:0/569046899 conn(0x5591d84a2b40 msgr2=0x5591d84a1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 -- 172.21.15.105:0/569046899 shutdown_connections 2024-03-26T17:39:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 1 -- 172.21.15.105:0/569046899 wait complete. 2024-03-26T17:39:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.162+0000 7f66c7ba9740 20 client.25691 trim_cache size 0 max 16384 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66c7ba9740 1 Processor -- start 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66c7ba9740 1 -- start start 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a75d0 0x5591d84a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a7fc0 0x5591d84b8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 0x5591d84a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a8d80 con 0x5591d84ad130 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01aa8c0 con 0x5591d84a7fc0 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a9450 con 0x5591d84a75d0 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66bdd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a7fc0 0x5591d84b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66be59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a75d0 0x5591d84a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66bdd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a7fc0 0x5591d84b8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41430/0 (socket says 172.21.15.105:41430) 2024-03-26T17:39:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66be59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a75d0 0x5591d84a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41888/0 (socket says 172.21.15.105:41888) 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66bdd9b640 1 -- 172.21.15.105:0/3618408857 learned_addr learned my addr 172.21.15.105:0/3618408857 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.177+0000 7f66bed9d640 1 --2- 172.21.15.105:0/3618408857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 0x5591d84a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66be59c640 1 -- 172.21.15.105:0/3618408857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a7fc0 msgr2=0x5591d84b8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66be59c640 1 --2- 172.21.15.105:0/3618408857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a7fc0 0x5591d84b8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66be59c640 1 -- 172.21.15.105:0/3618408857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 msgr2=0x5591d84a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66be59c640 1 --2- 172.21.15.105:0/3618408857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 0x5591d84a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66be59c640 1 -- 172.21.15.105:0/3618408857 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591d84b0300 con 0x5591d84a75d0 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66be59c640 1 --2- 172.21.15.105:0/3618408857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a75d0 0x5591d84a8850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009850 tx=0x7f66b002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e543e9c206439562 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66a7fff640 1 -- 172.21.15.105:0/3618408857 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b002cc90 con 0x5591d84a75d0 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66a7fff640 1 -- 172.21.15.105:0/3618408857 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b0034d70 con 0x5591d84a75d0 2024-03-26T17:39:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.178+0000 7f66a7fff640 1 -- 172.21.15.105:0/3618408857 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0003a20 con 0x5591d84a75d0 2024-03-26T17:39:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3618408857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a75d0 msgr2=0x5591d84a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/3618408857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a75d0 0x5591d84a8850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009850 tx=0x7f66b002bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3618408857 shutdown_connections 2024-03-26T17:39:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/3618408857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84ad130 0x5591d84a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/3618408857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a7fc0 0x5591d84b8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/3618408857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84a75d0 0x5591d84a8850 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3618408857 >> 172.21.15.105:0/3618408857 conn(0x5591d84a2b40 msgr2=0x5591d84a1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3618408857 shutdown_connections 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- 172.21.15.105:0/3618408857 wait complete. 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- start start 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 Processor -- start 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a7fc0 0x5591d84b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84ad130 0x5591d84b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01aa1f0 con 0x5591d84a7fc0 2024-03-26T17:39:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a9b20 con 0x5591d84a75d0 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.179+0000 7f66c7ba9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a6d60 con 0x5591d84ad130 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66c7ba9740 10 client.0 _ll_register_callbacks cb 0x7f66b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66bdd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a7fc0 0x5591d84b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66be59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66bdd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a7fc0 0x5591d84b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49510/0 (socket says 172.21.15.105:49510) 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66bdd9b640 1 -- 172.21.15.105:0/4064686362 learned_addr learned my addr 172.21.15.105:0/4064686362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66bed9d640 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84ad130 0x5591d84b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66be59c640 1 -- 172.21.15.105:0/4064686362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84ad130 msgr2=0x5591d84b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66be59c640 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84ad130 0x5591d84b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66be59c640 1 -- 172.21.15.105:0/4064686362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a7fc0 msgr2=0x5591d84b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66be59c640 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a7fc0 0x5591d84b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66be59c640 1 -- 172.21.15.105:0/4064686362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591d84b7490 con 0x5591d84a75d0 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a7fc0 0x5591d84b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66be59c640 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b3940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66b002c840 tx=0x7f66b0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb355e510cdb11f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66a5ffb640 1 -- 172.21.15.105:0/4064686362 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0003760 con 0x5591d84a75d0 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66a5ffb640 1 -- 172.21.15.105:0/4064686362 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f66b0003900 con 0x5591d84a75d0 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.180+0000 7f66c7ba9740 1 -- 172.21.15.105:0/4064686362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b0034d70 con 0x5591d84a75d0 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66a5ffb640 1 -- 172.21.15.105:0/4064686362 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66b0003b40 con 0x5591d84a75d0 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66c7ba9740 10 client.25697 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66c7ba9740 1 -- 172.21.15.105:0/4064686362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5591d84b87a0 con 0x5591d84a75d0 2024-03-26T17:39:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66a5ffb640 1 -- 172.21.15.105:0/4064686362 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b0034d70 con 0x5591d84a75d0 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66a5ffb640 1 -- 172.21.15.105:0/4064686362 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f66b0065670 con 0x5591d84a75d0 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66c7ba9740 10 client.25697 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66c7ba9740 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66c7ba9740 20 client.25697 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5591d8527bd0 0x5591d8529fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66c7ba9740 4 client.25697 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66c7ba9740 1 -- 172.21.15.105:0/4064686362 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591d8413940 con 0x5591d8527bd0 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5591d8527bd0 0x5591d8529fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66bdd9b640 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5591d8527bd0 0x5591d8529fc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77d35c49cb7e1b4 server_cookie=8ad81b133854e674 in_seq=0 out_seq=0 2024-03-26T17:39:33.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.181+0000 7f66a5ffb640 10 client.25697 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66a5ffb640 1 -- 172.21.15.105:0/4064686362 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591d8413940 con 0x5591d8527bd0 2024-03-26T17:39:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66a5ffb640 10 client.25697 handle_command_reply: tid=0 2024-03-26T17:39:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 client.25697 shutdown 2024-03-26T17:39:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 -- 172.21.15.105:0/4064686362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 msgr2=0x5591d84b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b3940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66b002c840 tx=0x7f66b0005270 comp rx=0 tx=0).stop 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 -- 172.21.15.105:0/4064686362 shutdown_connections 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5591d8527bd0 0x5591d8529fc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591d84ad130 0x5591d84b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66a5ffb640 0 client.25697 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591d84a7fc0 0x5591d84b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 --2- 172.21.15.105:0/4064686362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591d84a75d0 0x5591d84b3940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 -- 172.21.15.105:0/4064686362 >> 172.21.15.105:0/4064686362 conn(0x5591d84a2b40 msgr2=0x5591d84a8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 -- 172.21.15.105:0/4064686362 shutdown_connections 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 1 -- 172.21.15.105:0/4064686362 wait complete. 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.220+0000 7f66c7ba9740 20 client.25697 trim_cache size 0 max 16384 2024-03-26T17:39:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:39:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:39:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:39:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:39:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 -- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66a80a2fc0 msgr2=0x7f66a80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66a80a2fc0 0x7f66a80a5480 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f66b80099e0 tx=0x7f66b801e000 comp rx=0 tx=0).stop 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 -- 172.21.15.105:0/2039941416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c0102b10 msgr2=0x7f66c0198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c0102b10 0x7f66c0198000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f66b400e9f0 tx=0x7f66b400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 -- 172.21.15.105:0/2039941416 shutdown_connections 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66a80a2fc0 0x7f66a80a5480 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01034d0 0x7f66c019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c0102b10 0x7f66c0198000 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 --2- 172.21.15.105:0/2039941416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0101920 0x7f66c0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 -- 172.21.15.105:0/2039941416 >> 172.21.15.105:0/2039941416 conn(0x7f66c00fce90 msgr2=0x7f66c0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 -- 172.21.15.105:0/2039941416 shutdown_connections 2024-03-26T17:39:33.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:33.221+0000 7f66c4ee8640 1 -- 172.21.15.105:0/2039941416 wait complete. 2024-03-26T17:39:33.217 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:39:33.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:39:33.218 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 -- 172.21.15.105:0/4210741496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 msgr2=0x7f7adc1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 --2- 172.21.15.105:0/4210741496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 0x7f7adc1099f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009ca0 tx=0x7f7ad402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 -- 172.21.15.105:0/4210741496 shutdown_connections 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 --2- 172.21.15.105:0/4210741496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 0x7f7adc1099f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 --2- 172.21.15.105:0/4210741496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7adc104c70 0x7f7adc107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 --2- 172.21.15.105:0/4210741496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc102340 0x7f7adc104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 -- 172.21.15.105:0/4210741496 >> 172.21.15.105:0/4210741496 conn(0x7f7adc0fbd50 msgr2=0x7f7adc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 -- 172.21.15.105:0/4210741496 shutdown_connections 2024-03-26T17:39:33.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.341+0000 7f7ae226c640 1 -- 172.21.15.105:0/4210741496 wait complete. 2024-03-26T17:39:33.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae226c640 1 Processor -- start 2024-03-26T17:39:33.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae226c640 1 -- start start 2024-03-26T17:39:33.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae226c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7adc102340 0x7f7adc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae226c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc104c70 0x7f7adc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae226c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 0x7f7adc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae226c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc10d3f0 con 0x7f7adc1075a0 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae226c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc10d270 con 0x7f7adc102340 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae226c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc10d570 con 0x7f7adc104c70 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae0809640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 0x7f7adc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae0809640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 0x7f7adc19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49524/0 (socket says 172.21.15.105:49524) 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae0809640 1 -- 172.21.15.105:0/3646024257 learned_addr learned my addr 172.21.15.105:0/3646024257 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc104c70 0x7f7adc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7adbfff640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7adc102340 0x7f7adc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae0809640 1 -- 172.21.15.105:0/3646024257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc104c70 msgr2=0x7f7adc198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae0809640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc104c70 0x7f7adc198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae0809640 1 -- 172.21.15.105:0/3646024257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7adc102340 msgr2=0x7f7adc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.342+0000 7f7ae0809640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7adc102340 0x7f7adc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.343+0000 7f7ae0809640 1 -- 172.21.15.105:0/3646024257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc19f710 con 0x7f7adc1075a0 2024-03-26T17:39:33.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.343+0000 7f7ae0809640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 0x7f7adc19f0a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7ad402c700 tx=0x7f7ad40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.343+0000 7f7ad97fa640 1 -- 172.21.15.105:0/3646024257 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4041020 con 0x7f7adc1075a0 2024-03-26T17:39:33.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.343+0000 7f7ad97fa640 1 -- 172.21.15.105:0/3646024257 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ad4003e20 con 0x7f7adc1075a0 2024-03-26T17:39:33.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.343+0000 7f7ad97fa640 1 -- 172.21.15.105:0/3646024257 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4039040 con 0x7f7adc1075a0 2024-03-26T17:39:33.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.343+0000 7f7ae226c640 1 -- 172.21.15.105:0/3646024257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc19f9a0 con 0x7f7adc1075a0 2024-03-26T17:39:33.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.343+0000 7f7ae226c640 1 -- 172.21.15.105:0/3646024257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc19fe40 con 0x7f7adc1075a0 2024-03-26T17:39:33.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ad97fa640 1 -- 172.21.15.105:0/3646024257 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7ad4044020 con 0x7f7adc1075a0 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ae4e4c740 1 Processor -- start 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ae4e4c740 1 -- start start 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977edc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ad97fa640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ac40a2f20 0x7f7ac40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977ee4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c977e41630 con 0x55c977edc190 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c977de9e10 con 0x55c977edcb80 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c977e3d5e0 con 0x55c977ee1cf0 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977edc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977edc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49528/0 (socket says 172.21.15.105:49528) 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.346+0000 7f7adb7fe640 1 -- 172.21.15.105:0/2817958288 learned_addr learned my addr 172.21.15.105:0/2817958288 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ac40a2f20 0x7f7ac40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7ae0809640 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977ee4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7ad97fa640 1 -- 172.21.15.105:0/3646024257 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f7ad40f1600 con 0x7f7adc1075a0 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adbfff640 1 -- 172.21.15.105:0/2817958288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 msgr2=0x55c977ee4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adbfff640 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977ee4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adbfff640 1 -- 172.21.15.105:0/2817958288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 msgr2=0x55c977edc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adbfff640 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977edc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adbfff640 1 -- 172.21.15.105:0/2817958288 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c977e00740 con 0x55c977edcb80 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977edc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adbfff640 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee1680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009c20 tx=0x7f7acc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89c863cadf00128e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ac40a2f20 0x7f7ac40a53e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55c977edd3e0 tx=0x7f7ad0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7ac27fc640 1 -- 172.21.15.105:0/2817958288 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc004050 con 0x55c977edcb80 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7ac27fc640 1 -- 172.21.15.105:0/2817958288 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7acc0054d0 con 0x55c977edcb80 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.347+0000 7f7ac27fc640 1 -- 172.21.15.105:0/2817958288 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc004c30 con 0x55c977edcb80 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/2817958288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 msgr2=0x55c977ee1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee1680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009c20 tx=0x7f7acc011810 comp rx=0 tx=0).stop 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/2817958288 shutdown_connections 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977ee4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee1680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/2817958288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977edc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/2817958288 >> 172.21.15.105:0/2817958288 conn(0x55c977ed7700 msgr2=0x55c977ed7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/2817958288 shutdown_connections 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/2817958288 wait complete. 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.348+0000 7f7ae4e4c740 1 -- start start 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae4e4c740 1 Processor -- start 2024-03-26T17:39:33.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ee6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977eedc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c977ee4ca0 con 0x55c977edc190 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c977dfd1a0 con 0x55c977edcb80 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c977ee4e20 con 0x55c977ee1cf0 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae4e4c740 10 client.0 _ll_register_callbacks cb 0x55c977b54c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ee6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ee6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49536/0 (socket says 172.21.15.105:49536) 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adbfff640 1 -- 172.21.15.105:0/1771127886 learned_addr learned my addr 172.21.15.105:0/1771127886 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7ae0809640 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977eedc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adbfff640 1 -- 172.21.15.105:0/1771127886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 msgr2=0x55c977eedc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adbfff640 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977eedc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adbfff640 1 -- 172.21.15.105:0/1771127886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 msgr2=0x55c977ee6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adbfff640 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.349+0000 7f7adbfff640 1 -- 172.21.15.105:0/1771127886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c977c1d7e0 con 0x55c977edc190 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7adbfff640 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ee6680 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000c00 tx=0x7f7acc004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416819462d0769ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7abb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7abb7fe640 1 -- 172.21.15.105:0/1771127886 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc029070 con 0x55c977edc190 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7abb7fe640 1 -- 172.21.15.105:0/1771127886 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f7acc003a00 con 0x55c977edc190 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/1771127886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7acc0054d0 con 0x55c977edc190 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7abb7fe640 1 -- 172.21.15.105:0/1771127886 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7acc035050 con 0x55c977edc190 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7ae4e4c740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/1771127886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c977c7df70 con 0x55c977edc190 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7abb7fe640 1 -- 172.21.15.105:0/1771127886 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7acc0054d0 con 0x55c977edc190 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7abb7fe640 1 -- 172.21.15.105:0/1771127886 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7acc0050f0 con 0x55c977edc190 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7ae4e4c740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:33.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.350+0000 7f7ae4e4c740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:33.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.351+0000 7f7ae4e4c740 20 client.15849 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:33.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.351+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c977ef2290 0x55c977ef46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.351+0000 7f7ae4e4c740 4 client.15849 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:33.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.351+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/1771127886 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c977e4eb70 con 0x55c977ef2290 2024-03-26T17:39:33.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.351+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c977ef2290 0x55c977ef46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.351+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c977ef2290 0x55c977ef46c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785066dc6be254e5 server_cookie=1960e4d7eb1d0a1f in_seq=0 out_seq=0 2024-03-26T17:39:33.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.351+0000 7f7abb7fe640 10 client.15849 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.352+0000 7f7abb7fe640 1 -- 172.21.15.105:0/1771127886 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c977e4eb70 con 0x55c977ef2290 2024-03-26T17:39:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.352+0000 7f7abb7fe640 10 client.15849 handle_command_reply: tid=0 2024-03-26T17:39:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 client.15849 shutdown 2024-03-26T17:39:33.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/1771127886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 msgr2=0x55c977ee6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ee6680 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000c00 tx=0x7f7acc004300 comp rx=0 tx=0).stop 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/1771127886 shutdown_connections 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c977ef2290 0x55c977ef46c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977eedc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977ee6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/1771127886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ee6680 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7abb7fe640 0 client.15849 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/1771127886 >> 172.21.15.105:0/1771127886 conn(0x55c977ed7700 msgr2=0x55c977ed6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/1771127886 shutdown_connections 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/1771127886 wait complete. 2024-03-26T17:39:33.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.353+0000 7f7ae4e4c740 20 client.15849 trim_cache size 0 max 16384 2024-03-26T17:39:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae4e4c740 1 Processor -- start 2024-03-26T17:39:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae4e4c740 1 -- start start 2024-03-26T17:39:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ef8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977edcb80 0x55c977eedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977ee1cf0 0x55c977ef9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1a8d40 con 0x55c977edc190 2024-03-26T17:39:33.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1aa880 con 0x55c977ee1cf0 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1a9410 con 0x55c977edcb80 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977edcb80 0x55c977eedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ef8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae0809640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977ee1cf0 0x55c977ef9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977edcb80 0x55c977eedbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41936/0 (socket says 172.21.15.105:41936) 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adb7fe640 1 -- 172.21.15.105:0/3999346646 learned_addr learned my addr 172.21.15.105:0/3999346646 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adbfff640 1 -- 172.21.15.105:0/3999346646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977edcb80 msgr2=0x55c977eedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adbfff640 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977edcb80 0x55c977eedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adbfff640 1 -- 172.21.15.105:0/3999346646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977ee1cf0 msgr2=0x55c977ef9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adbfff640 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977ee1cf0 0x55c977ef9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adbfff640 1 -- 172.21.15.105:0/3999346646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c977b15170 con 0x55c977edc190 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977edcb80 0x55c977eedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.374+0000 7f7ae0809640 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977ee1cf0 0x55c977ef9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.375+0000 7f7adbfff640 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ef8fc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7acc011d10 tx=0x7f7acc009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fb97c08d63b62c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.375+0000 7f7ac1ffb640 1 -- 172.21.15.105:0/3999346646 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc029070 con 0x55c977edc190 2024-03-26T17:39:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.375+0000 7f7ac1ffb640 1 -- 172.21.15.105:0/3999346646 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7acc0054f0 con 0x55c977edc190 2024-03-26T17:39:33.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.375+0000 7f7ac1ffb640 1 -- 172.21.15.105:0/3999346646 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc003a60 con 0x55c977edc190 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3999346646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 msgr2=0x55c977ef8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ef8fc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7acc011d10 tx=0x7f7acc009a70 comp rx=0 tx=0).stop 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3999346646 shutdown_connections 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977ee1cf0 0x55c977ef9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977edcb80 0x55c977eedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3999346646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977ef8fc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3999346646 >> 172.21.15.105:0/3999346646 conn(0x55c977ed7700 msgr2=0x55c977ed6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3999346646 shutdown_connections 2024-03-26T17:39:33.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3999346646 wait complete. 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 -- start start 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.376+0000 7f7ae4e4c740 1 Processor -- start 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977eed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977eebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae4e4c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977eec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1aa1b0 con 0x55c977edc190 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1a9ae0 con 0x55c977edcb80 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae4e4c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc1a6d20 con 0x55c977ee1cf0 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae4e4c740 10 client.0 _ll_register_callbacks cb 0x7f7ad0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977eed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977eed9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49564/0 (socket says 172.21.15.105:49564) 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 -- 172.21.15.105:0/3241078642 learned_addr learned my addr 172.21.15.105:0/3241078642 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977eebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae0809640 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977eec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 -- 172.21.15.105:0/3241078642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 msgr2=0x55c977eec520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977eec520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 -- 172.21.15.105:0/3241078642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 msgr2=0x55c977eebfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977eebfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 -- 172.21.15.105:0/3241078642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c977eecb60 con 0x55c977edc190 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7adbfff640 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977eed9d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0059f0 tx=0x7f7acc009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9ad525abd6db96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7abb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7abb7fe640 1 -- 172.21.15.105:0/3241078642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc029030 con 0x55c977edc190 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7abb7fe640 1 -- 172.21.15.105:0/3241078642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7f7acc005240 con 0x55c977edc190 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.377+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3241078642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7acc0054f0 con 0x55c977edc190 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7abb7fe640 1 -- 172.21.15.105:0/3241078642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7acc003800 con 0x55c977edc190 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7ae4e4c740 10 client.15855 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3241078642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c977f594b0 con 0x55c977edc190 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7abb7fe640 1 -- 172.21.15.105:0/3241078642 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7acc003e50 con 0x55c977edc190 2024-03-26T17:39:33.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7abb7fe640 1 -- 172.21.15.105:0/3241078642 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7acc004bf0 con 0x55c977edc190 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7ae4e4c740 10 client.15855 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7ae4e4c740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7ae4e4c740 20 client.15855 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c977f59d50 0x55c977f5c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7ae4e4c740 4 client.15855 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3241078642 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7acc0054f0 con 0x55c977f59d50 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.378+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c977f59d50 0x55c977f5c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.379+0000 7f7abb7fe640 10 client.15855 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.379+0000 7f7adb7fe640 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c977f59d50 0x55c977f5c170 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3100fa99acd341 server_cookie=f6d407ebd95bdd8a in_seq=0 out_seq=0 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7abb7fe640 1 -- 172.21.15.105:0/3241078642 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7acc0054f0 con 0x55c977f59d50 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7abb7fe640 10 client.15855 handle_command_reply: tid=0 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 client.15855 shutdown 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3241078642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 msgr2=0x55c977eed9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977eed9d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0059f0 tx=0x7f7acc009820 comp rx=0 tx=0).stop 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3241078642 shutdown_connections 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c977f59d50 0x55c977f5c170 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7abb7fe640 0 client.15855 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c977ee1cf0 0x55c977eec520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c977edcb80 0x55c977eebfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 --2- 172.21.15.105:0/3241078642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c977edc190 0x55c977eed9d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3241078642 >> 172.21.15.105:0/3241078642 conn(0x55c977ed7700 msgr2=0x55c977ee5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3241078642 shutdown_connections 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.380+0000 7f7ae4e4c740 1 -- 172.21.15.105:0/3241078642 wait complete. 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae4e4c740 20 client.15855 trim_cache size 0 max 16384 2024-03-26T17:39:33.378 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:39:33.379 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:39:33.379 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "7633c63b-902c-4ad3-a185-105a8f15adc5", 2024-03-26T17:39:33.379 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:39:33.379 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 -- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ac40a2f20 msgr2=0x7f7ac40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ac40a2f20 0x7f7ac40a53e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55c977edd3e0 tx=0x7f7ad0005f90 comp rx=0 tx=0).stop 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 -- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 msgr2=0x7f7adc19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 0x7f7adc19f0a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7ad402c700 tx=0x7f7ad40328a0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 -- 172.21.15.105:0/3646024257 shutdown_connections 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ac40a2f20 0x7f7ac40a53e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7adc1075a0 0x7f7adc19f0a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc104c70 0x7f7adc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 --2- 172.21.15.105:0/3646024257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7adc102340 0x7f7adc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 -- 172.21.15.105:0/3646024257 >> 172.21.15.105:0/3646024257 conn(0x7f7adc0fbd50 msgr2=0x7f7adc102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 -- 172.21.15.105:0/3646024257 shutdown_connections 2024-03-26T17:39:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.381+0000 7f7ae226c640 1 -- 172.21.15.105:0/3646024257 wait complete. 2024-03-26T17:39:33.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:39:33.386 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:39:33.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 -- 172.21.15.105:0/3593432968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 msgr2=0x7fed9c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 --2- 172.21.15.105:0/3593432968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 0x7fed9c100be0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fed98009900 tx=0x7fed9802bc30 comp rx=0 tx=0).stop 2024-03-26T17:39:33.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 -- 172.21.15.105:0/3593432968 shutdown_connections 2024-03-26T17:39:33.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 --2- 172.21.15.105:0/3593432968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed9c102370 0x7fed9c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 --2- 172.21.15.105:0/3593432968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed9c1019b0 0x7fed9c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 --2- 172.21.15.105:0/3593432968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 0x7fed9c100be0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 -- 172.21.15.105:0/3593432968 >> 172.21.15.105:0/3593432968 conn(0x7fed9c0fbd30 msgr2=0x7fed9c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 -- 172.21.15.105:0/3593432968 shutdown_connections 2024-03-26T17:39:33.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.511+0000 7feda3a34640 1 -- 172.21.15.105:0/3593432968 wait complete. 2024-03-26T17:39:33.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda3a34640 1 Processor -- start 2024-03-26T17:39:33.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda3a34640 1 -- start start 2024-03-26T17:39:33.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda3a34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 0x7fed9c196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda3a34640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed9c1019b0 0x7fed9c196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda3a34640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed9c102370 0x7fed9c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda3a34640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c10c2c0 con 0x7fed9c1007c0 2024-03-26T17:39:33.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda3a34640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c10c140 con 0x7fed9c102370 2024-03-26T17:39:33.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda3a34640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c10c440 con 0x7fed9c1019b0 2024-03-26T17:39:33.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda17d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 0x7fed9c196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda0fcf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed9c1019b0 0x7fed9c196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda17d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 0x7fed9c196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49588/0 (socket says 172.21.15.105:49588) 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda17d0640 1 -- 172.21.15.105:0/3717096537 learned_addr learned my addr 172.21.15.105:0/3717096537 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.512+0000 7feda1fd1640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed9c102370 0x7fed9c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7feda17d0640 1 -- 172.21.15.105:0/3717096537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed9c1019b0 msgr2=0x7fed9c196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7feda17d0640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed9c1019b0 0x7fed9c196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7feda17d0640 1 -- 172.21.15.105:0/3717096537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed9c102370 msgr2=0x7fed9c19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7feda17d0640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed9c102370 0x7fed9c19dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7feda17d0640 1 -- 172.21.15.105:0/3717096537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c19e650 con 0x7fed9c1007c0 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7feda17d0640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 0x7fed9c196a30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fed98000c00 tx=0x7fed98005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7fed927fc640 1 -- 172.21.15.105:0/3717096537 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98043020 con 0x7fed9c1007c0 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7feda3a34640 1 -- 172.21.15.105:0/3717096537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c19e8e0 con 0x7fed9c1007c0 2024-03-26T17:39:33.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7fed927fc640 1 -- 172.21.15.105:0/3717096537 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed98004350 con 0x7fed9c1007c0 2024-03-26T17:39:33.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7fed927fc640 1 -- 172.21.15.105:0/3717096537 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98003b80 con 0x7fed9c1007c0 2024-03-26T17:39:33.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.513+0000 7feda3a34640 1 -- 172.21.15.105:0/3717096537 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c19ee30 con 0x7fed9c1007c0 2024-03-26T17:39:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.515+0000 7fed927fc640 1 -- 172.21.15.105:0/3717096537 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fed98046020 con 0x7fed9c1007c0 2024-03-26T17:39:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.516+0000 7fed927fc640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed880a2f20 0x7fed880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.516+0000 7fed927fc640 1 -- 172.21.15.105:0/3717096537 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fed980f1550 con 0x7fed9c1007c0 2024-03-26T17:39:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.516+0000 7feda0fcf640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed880a2f20 0x7fed880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.516+0000 7feda661c740 1 Processor -- start 2024-03-26T17:39:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.516+0000 7feda661c740 1 -- start start 2024-03-26T17:39:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.516+0000 7feda661c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ee2d0 0x561d892eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda661c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d892ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda661c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892fbc10 0x561d892fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda661c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561d89256980 con 0x561d892ee2d0 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda661c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561d891febe0 con 0x561d892ecf60 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda661c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561d89252930 con 0x561d892fbc10 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ee2d0 0x561d892eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ee2d0 0x561d892eca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49590/0 (socket says 172.21.15.105:49590) 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 -- 172.21.15.105:0/2689959571 learned_addr learned my addr 172.21.15.105:0/2689959571 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda1fd1640 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892fbc10 0x561d892fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda0fcf640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed880a2f20 0x7fed880a53e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fed9c197f40 tx=0x7fed8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 -- 172.21.15.105:0/2689959571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892fbc10 msgr2=0x561d892fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892fbc10 0x561d892fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda0fcf640 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d892ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 -- 172.21.15.105:0/2689959571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 msgr2=0x561d892ed3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d892ed3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 -- 172.21.15.105:0/2689959571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d890347d0 con 0x561d892ee2d0 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda1fd1640 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892fbc10 0x561d892fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda0fcf640 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d892ed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7feda17d0640 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ee2d0 0x561d892eca20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fed980bdbd0 tx=0x7fed980bd780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcba7ada7ac82170 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7fed877fe640 1 -- 172.21.15.105:0/2689959571 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed980bdc40 con 0x561d892ee2d0 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7fed877fe640 1 -- 172.21.15.105:0/2689959571 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed980c1d90 con 0x561d892ee2d0 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.517+0000 7fed877fe640 1 -- 172.21.15.105:0/2689959571 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98003d20 con 0x561d892ee2d0 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 -- 172.21.15.105:0/2689959571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ee2d0 msgr2=0x561d892eca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ee2d0 0x561d892eca20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fed980bdbd0 tx=0x7fed980bd780 comp rx=0 tx=0).stop 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 -- 172.21.15.105:0/2689959571 shutdown_connections 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892fbc10 0x561d892fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d892ed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 --2- 172.21.15.105:0/2689959571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ee2d0 0x561d892eca20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 -- 172.21.15.105:0/2689959571 >> 172.21.15.105:0/2689959571 conn(0x561d892ef070 msgr2=0x561d892ef470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 -- 172.21.15.105:0/2689959571 shutdown_connections 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.518+0000 7feda661c740 1 -- 172.21.15.105:0/2689959571 wait complete. 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 1 -- start start 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 1 Processor -- start 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d892ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89300170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892fbc10 0x561d893071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561d892fe600 con 0x561d892ecf60 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561d89212160 con 0x561d892fbc10 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561d892fe780 con 0x561d892ee2d0 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda17d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d892ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.519+0000 7feda661c740 10 client.0 _ll_register_callbacks cb 0x561d8924e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:33.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda17d0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d892ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49600/0 (socket says 172.21.15.105:49600) 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda17d0640 1 -- 172.21.15.105:0/419812538 learned_addr learned my addr 172.21.15.105:0/419812538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda0fcf640 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89300170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda17d0640 1 -- 172.21.15.105:0/419812538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 msgr2=0x561d89300170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda17d0640 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89300170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda17d0640 1 -- 172.21.15.105:0/419812538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892fbc10 msgr2=0x561d893071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda17d0640 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892fbc10 0x561d893071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda17d0640 1 -- 172.21.15.105:0/419812538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d88f6a3f0 con 0x561d892ecf60 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda0fcf640 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89300170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda17d0640 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d892ffc30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fed980f1870 tx=0x7fed98005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedb5d002c0398bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7fed84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7fed84ff9640 1 -- 172.21.15.105:0/419812538 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed980da070 con 0x561d892ecf60 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7fed84ff9640 1 -- 172.21.15.105:0/419812538 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fed980c61c0 con 0x561d892ecf60 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda661c740 1 -- 172.21.15.105:0/419812538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed980c1d90 con 0x561d892ecf60 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7fed84ff9640 1 -- 172.21.15.105:0/419812538 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed980bbce0 con 0x561d892ecf60 2024-03-26T17:39:33.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda661c740 10 client.15867 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.520+0000 7feda661c740 1 -- 172.21.15.105:0/419812538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d88f2a0f0 con 0x561d892ecf60 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7fed84ff9640 1 -- 172.21.15.105:0/419812538 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed980c1d90 con 0x561d892ecf60 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7fed84ff9640 1 -- 172.21.15.105:0/419812538 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed980d4070 con 0x561d892ecf60 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7feda661c740 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7feda661c740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7feda661c740 20 client.15867 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7feda661c740 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d8930b870 0x561d8930dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7feda661c740 4 client.15867 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7feda661c740 1 -- 172.21.15.105:0/419812538 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d89263ec0 con 0x561d8930b870 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7feda0fcf640 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d8930b870 0x561d8930dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7fed84ff9640 10 client.15867 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.521+0000 7feda0fcf640 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d8930b870 0x561d8930dd30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc9ea2bef34e3d server_cookie=2734b41dcd67d48a in_seq=0 out_seq=0 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.522+0000 7fed84ff9640 1 -- 172.21.15.105:0/419812538 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561d89263ec0 con 0x561d8930b870 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.522+0000 7fed84ff9640 10 client.15867 handle_command_reply: tid=0 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.522+0000 7feda661c740 1 client.15867 shutdown 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 -- 172.21.15.105:0/419812538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 msgr2=0x561d892ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d892ffc30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fed980f1870 tx=0x7fed98005700 comp rx=0 tx=0).stop 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 -- 172.21.15.105:0/419812538 shutdown_connections 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d8930b870 0x561d8930dd30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892fbc10 0x561d893071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7fed84ff9640 0 client.15867 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89300170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 --2- 172.21.15.105:0/419812538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d892ffc30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 -- 172.21.15.105:0/419812538 >> 172.21.15.105:0/419812538 conn(0x561d892ef070 msgr2=0x561d892ec080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 -- 172.21.15.105:0/419812538 shutdown_connections 2024-03-26T17:39:33.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 1 -- 172.21.15.105:0/419812538 wait complete. 2024-03-26T17:39:33.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.523+0000 7feda661c740 20 client.15867 trim_cache size 0 max 16384 2024-03-26T17:39:33.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda661c740 1 Processor -- start 2024-03-26T17:39:33.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda661c740 1 -- start start 2024-03-26T17:39:33.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda661c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ef070 0x561d893000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda661c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d89307140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda661c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ee2d0 0x561d892eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda661c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1a6bf0 con 0x561d892ecf60 2024-03-26T17:39:33.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda661c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1a8730 con 0x561d892ee2d0 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda661c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1a72c0 con 0x561d892ef070 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d89307140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda17d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ef070 0x561d893000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d89307140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49606/0 (socket says 172.21.15.105:49606) 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 -- 172.21.15.105:0/1661248130 learned_addr learned my addr 172.21.15.105:0/1661248130 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda1fd1640 1 --2- 172.21.15.105:0/1661248130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ee2d0 0x561d892eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 -- 172.21.15.105:0/1661248130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ef070 msgr2=0x561d893000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 --2- 172.21.15.105:0/1661248130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ef070 0x561d893000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 -- 172.21.15.105:0/1661248130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ee2d0 msgr2=0x561d892eb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 --2- 172.21.15.105:0/1661248130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ee2d0 0x561d892eb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 -- 172.21.15.105:0/1661248130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d892efde0 con 0x561d892ecf60 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.543+0000 7feda0fcf640 1 --2- 172.21.15.105:0/1661248130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d89307140 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fed8c01d680 tx=0x7fed8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16d873935ddf440 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7fed86ffd640 1 -- 172.21.15.105:0/1661248130 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c02e070 con 0x561d892ecf60 2024-03-26T17:39:33.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7fed86ffd640 1 -- 172.21.15.105:0/1661248130 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed8c01dd60 con 0x561d892ecf60 2024-03-26T17:39:33.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7fed86ffd640 1 -- 172.21.15.105:0/1661248130 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c0038c0 con 0x561d892ecf60 2024-03-26T17:39:33.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7feda661c740 1 -- 172.21.15.105:0/1661248130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 msgr2=0x561d89307140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7feda661c740 1 --2- 172.21.15.105:0/1661248130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d89307140 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fed8c01d680 tx=0x7fed8c014000 comp rx=0 tx=0).stop 2024-03-26T17:39:33.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7feda661c740 1 -- 172.21.15.105:0/1661248130 shutdown_connections 2024-03-26T17:39:33.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7feda661c740 1 --2- 172.21.15.105:0/1661248130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ee2d0 0x561d892eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7feda661c740 1 --2- 172.21.15.105:0/1661248130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ecf60 0x561d89307140 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7feda661c740 1 --2- 172.21.15.105:0/1661248130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ef070 0x561d893000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.544+0000 7feda661c740 1 -- 172.21.15.105:0/1661248130 >> 172.21.15.105:0/1661248130 conn(0x561d892fb7c0 msgr2=0x561d892ef450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 -- 172.21.15.105:0/1661248130 shutdown_connections 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 -- 172.21.15.105:0/1661248130 wait complete. 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 -- start start 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 Processor -- start 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d89304960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89304ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ef070 0x561d89305e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1a8060 con 0x561d892ef070 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1a7990 con 0x561d892ecf60 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.545+0000 7feda661c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1a4bd0 con 0x561d892ee2d0 2024-03-26T17:39:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda661c740 10 client.0 _ll_register_callbacks cb 0x7fed94009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda17d0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d89304960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda1fd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ef070 0x561d89305e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda0fcf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89304ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda17d0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d89304960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41558/0 (socket says 172.21.15.105:41558) 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda1fd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ef070 0x561d89305e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49608/0 (socket says 172.21.15.105:49608) 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda17d0640 1 -- 172.21.15.105:0/4205832183 learned_addr learned my addr 172.21.15.105:0/4205832183 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda1fd1640 1 -- 172.21.15.105:0/4205832183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 msgr2=0x561d89304ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda1fd1640 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89304ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda1fd1640 1 -- 172.21.15.105:0/4205832183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 msgr2=0x561d89304960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda1fd1640 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d89304960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda1fd1640 1 -- 172.21.15.105:0/4205832183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d89306450 con 0x561d892ef070 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda0fcf640 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89304ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda1fd1640 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ef070 0x561d89305e60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x561d892effe0 tx=0x7fed94005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359ece07c356609d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7fed84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7fed84ff9640 1 -- 172.21.15.105:0/4205832183 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94020060 con 0x561d892ef070 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7fed84ff9640 1 -- 172.21.15.105:0/4205832183 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8293+0+0 (secure 0 0 0) 0x7fed94011ca0 con 0x561d892ef070 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda661c740 1 -- 172.21.15.105:0/4205832183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed8c01dd60 con 0x561d892ef070 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7fed84ff9640 1 -- 172.21.15.105:0/4205832183 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed94005980 con 0x561d892ef070 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda661c740 10 client.15873 fetch_fsmap learned FSMap version 27 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.546+0000 7feda661c740 1 -- 172.21.15.105:0/4205832183 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561d89371df0 con 0x561d892ef070 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7fed84ff9640 1 -- 172.21.15.105:0/4205832183 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed8c01dd60 con 0x561d892ef070 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7fed84ff9640 1 -- 172.21.15.105:0/4205832183 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed9401a070 con 0x561d892ef070 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7feda661c740 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7feda661c740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7feda661c740 20 client.15873 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7feda661c740 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d892fbc10 0x561d89374860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7feda661c740 4 client.15873 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:39:33.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7feda661c740 1 -- 172.21.15.105:0/4205832183 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d8925e210 con 0x561d892fbc10 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.547+0000 7feda17d0640 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d892fbc10 0x561d89374860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7fed84ff9640 10 client.15873 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda17d0640 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d892fbc10 0x561d89374860 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b076b525c29183 server_cookie=741e753926cbccd4 in_seq=0 out_seq=0 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7fed84ff9640 1 -- 172.21.15.105:0/4205832183 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561d8925e210 con 0x561d892fbc10 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7fed84ff9640 10 client.15873 handle_command_reply: tid=0 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 client.15873 shutdown 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 -- 172.21.15.105:0/4205832183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ef070 msgr2=0x561d89305e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ef070 0x561d89305e60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x561d892effe0 tx=0x7fed94005730 comp rx=0 tx=0).stop 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 -- 172.21.15.105:0/4205832183 shutdown_connections 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d892fbc10 0x561d89374860 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d892ef070 0x561d89305e60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d892ee2d0 0x561d89304ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 --2- 172.21.15.105:0/4205832183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d892ecf60 0x561d89304960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 -- 172.21.15.105:0/4205832183 >> 172.21.15.105:0/4205832183 conn(0x561d892fb7c0 msgr2=0x561d892eb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7fed84ff9640 0 client.15873 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 -- 172.21.15.105:0/4205832183 shutdown_connections 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 1 -- 172.21.15.105:0/4205832183 wait complete. 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.548+0000 7feda661c740 20 client.15873 trim_cache size 0 max 16384 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stdout: "7633c63b-902c-4ad3-a185-105a8f15adc5": { 2024-03-26T17:39:33.548 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:39:33.549 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "7633c63b-902c-4ad3-a185-105a8f15adc5", 2024-03-26T17:39:33.549 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:39:33.549 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:39:33.549 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:39:33.549 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 -- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed880a2f20 msgr2=0x7fed880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed880a2f20 0x7fed880a53e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fed9c197f40 tx=0x7fed8c005f90 comp rx=0 tx=0).stop 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 -- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 msgr2=0x7fed9c196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 0x7fed9c196a30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fed98000c00 tx=0x7fed98005dc0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 -- 172.21.15.105:0/3717096537 shutdown_connections 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed880a2f20 0x7fed880a53e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed9c102370 0x7fed9c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed9c1019b0 0x7fed9c196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 --2- 172.21.15.105:0/3717096537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed9c1007c0 0x7fed9c196a30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 -- 172.21.15.105:0/3717096537 >> 172.21.15.105:0/3717096537 conn(0x7fed9c0fbd30 msgr2=0x7fed9c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:33.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 -- 172.21.15.105:0/3717096537 shutdown_connections 2024-03-26T17:39:33.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:39:33.549+0000 7feda3a34640 1 -- 172.21.15.105:0/3717096537 wait complete. 2024-03-26T17:39:33.554 INFO:tasks.cephfs.filesystem:scrub status for tag:7633c63b-902c-4ad3-a185-105a8f15adc5 - {'path': '/', 'tag': '7633c63b-902c-4ad3-a185-105a8f15adc5', 'options': 'recursive,force'} 2024-03-26T17:39:33.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:33 smithi105 ceph-mon[32280]: pgmap v382: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 2.1 MiB/s wr, 120 op/s 2024-03-26T17:39:34.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 10435 4.33 MB/sec warmup 17 sec latency 24.171 ms 2024-03-26T17:39:34.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:33 smithi144 ceph-mon[36095]: pgmap v382: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 2.1 MiB/s wr, 120 op/s 2024-03-26T17:39:34.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:33 smithi135 ceph-mon[35636]: pgmap v382: 97 pgs: 97 active+clean; 27 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 2.1 MiB/s wr, 120 op/s 2024-03-26T17:39:34.166 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:39:33 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setuser ceph since I am not root 2024-03-26T17:39:34.166 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:39:33 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setgroup ceph since I am not root 2024-03-26T17:39:34.166 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:39:33 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/2135695292 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556833937a40 con 0x55683390a800 2024-03-26T17:39:34.166 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:39:33 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T17:39:35.000 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:34 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:39:35.000 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:34 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:35.000 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:34 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:39:35.000 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:34 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:35.000 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:34 smithi144 ceph-mon[36095]: daemon mds.e restarted 2024-03-26T17:39:35.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 11005 4.35 MB/sec warmup 18 sec latency 15.983 ms 2024-03-26T17:39:35.066 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:34 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:39:35.066 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:34 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:35.066 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:34 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:39:35.066 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:34 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:35.066 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:34 smithi135 ceph-mon[35636]: daemon mds.e restarted 2024-03-26T17:39:35.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:34 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:39:35.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:34 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:35.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:34 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:39:35.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:34 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:39:35.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:34 smithi105 ceph-mon[32280]: daemon mds.e restarted 2024-03-26T17:39:36.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 11438 4.14 MB/sec warmup 19 sec latency 22.095 ms 2024-03-26T17:39:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:35 smithi144 ceph-mon[36095]: pgmap v383: 97 pgs: 97 active+clean; 27 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:39:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:35 smithi144 ceph-mon[36095]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T17:39:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:35 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/2307931519,v1:172.21.15.135:6835/2307931519] up:boot 2024-03-26T17:39:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:35 smithi144 ceph-mon[36095]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:39:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:39:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:35 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:39:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:35 smithi105 ceph-mon[32280]: pgmap v383: 97 pgs: 97 active+clean; 27 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:39:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:35 smithi105 ceph-mon[32280]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T17:39:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:35 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/2307931519,v1:172.21.15.135:6835/2307931519] up:boot 2024-03-26T17:39:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:35 smithi105 ceph-mon[32280]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:39:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:39:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:35 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:39:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:35 smithi135 ceph-mon[35636]: pgmap v383: 97 pgs: 97 active+clean; 27 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:39:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:35 smithi135 ceph-mon[35636]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T17:39:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:35 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/2307931519,v1:172.21.15.135:6835/2307931519] up:boot 2024-03-26T17:39:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:35 smithi135 ceph-mon[35636]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:39:36.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:39:36.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:35 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:39:37.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 12084 4.17 MB/sec warmup 20 sec latency 22.911 ms 2024-03-26T17:39:38.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 12744 4.00 MB/sec warmup 21 sec latency 14.929 ms 2024-03-26T17:39:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:37 smithi144 ceph-mon[36095]: pgmap v385: 97 pgs: 97 active+clean; 26 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:39:38.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:37 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:39:38.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:37 smithi105 ceph-mon[32280]: pgmap v385: 97 pgs: 97 active+clean; 26 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:39:38.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:37 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:39:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:37 smithi135 ceph-mon[35636]: pgmap v385: 97 pgs: 97 active+clean; 26 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:39:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:37 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:39:39.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 13412 4.06 MB/sec warmup 22 sec latency 15.968 ms 2024-03-26T17:39:40.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 13986 4.03 MB/sec warmup 23 sec latency 15.824 ms 2024-03-26T17:39:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:39 smithi144 ceph-mon[36095]: pgmap v386: 97 pgs: 97 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:39:40.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:39 smithi105 ceph-mon[32280]: pgmap v386: 97 pgs: 97 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:39:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:39 smithi135 ceph-mon[35636]: pgmap v386: 97 pgs: 97 active+clean; 24 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:39:41.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 14576 4.06 MB/sec warmup 24 sec latency 15.608 ms 2024-03-26T17:39:42.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 15057 3.92 MB/sec warmup 25 sec latency 21.512 ms 2024-03-26T17:39:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:41 smithi144 ceph-mon[36095]: pgmap v387: 97 pgs: 97 active+clean; 26 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-03-26T17:39:42.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:41 smithi105 ceph-mon[32280]: pgmap v387: 97 pgs: 97 active+clean; 26 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-03-26T17:39:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:41 smithi135 ceph-mon[35636]: pgmap v387: 97 pgs: 97 active+clean; 26 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-03-26T17:39:43.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 15687 3.94 MB/sec warmup 26 sec latency 17.464 ms 2024-03-26T17:39:44.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 16371 3.82 MB/sec warmup 27 sec latency 16.154 ms 2024-03-26T17:39:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:43 smithi144 ceph-mon[36095]: pgmap v388: 97 pgs: 97 active+clean; 24 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:39:44.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:43 smithi105 ceph-mon[32280]: pgmap v388: 97 pgs: 97 active+clean; 24 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:39:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:43 smithi135 ceph-mon[35636]: pgmap v388: 97 pgs: 97 active+clean; 24 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:39:45.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 17064 3.88 MB/sec warmup 28 sec latency 16.867 ms 2024-03-26T17:39:45.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:44 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:39:45.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:44 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:39:45.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:44 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:39:46.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 17526 3.86 MB/sec warmup 29 sec latency 16.646 ms 2024-03-26T17:39:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:45 smithi144 ceph-mon[36095]: pgmap v389: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:39:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:46.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:45 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:46.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:45 smithi105 ceph-mon[32280]: pgmap v389: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:39:46.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:46.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:45 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:45 smithi135 ceph-mon[35636]: pgmap v389: 97 pgs: 97 active+clean; 25 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:39:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:45 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 -- 172.21.15.105:0/4036737127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63701034c0 msgr2=0x7f6370109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 --2- 172.21.15.105:0/4036737127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63701034c0 0x7f6370109d50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6368009ca0 tx=0x7f636802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 -- 172.21.15.105:0/4036737127 shutdown_connections 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 --2- 172.21.15.105:0/4036737127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63701034c0 0x7f6370109d50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 --2- 172.21.15.105:0/4036737127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6370102b00 0x7f6370102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 --2- 172.21.15.105:0/4036737127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6370101910 0x7f6370101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 -- 172.21.15.105:0/4036737127 >> 172.21.15.105:0/4036737127 conn(0x7f63700fce80 msgr2=0x7f63700ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 -- 172.21.15.105:0/4036737127 shutdown_connections 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.314+0000 7f63773f4640 1 -- 172.21.15.105:0/4036737127 wait complete. 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f63773f4640 1 Processor -- start 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f63773f4640 1 -- start start 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f63773f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6370101910 0x7f6370197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f63773f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6370102b00 0x7f6370197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f63773f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63701034c0 0x7f637019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f63773f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f637010d440 con 0x7f6370101910 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f63773f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f637010d2c0 con 0x7f63701034c0 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f63773f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f637010d5c0 con 0x7f6370102b00 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6370101910 0x7f6370197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6370101910 0x7f6370197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47884/0 (socket says 172.21.15.105:47884) 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 -- 172.21.15.105:0/422349482 learned_addr learned my addr 172.21.15.105:0/422349482 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375991640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63701034c0 0x7f637019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f637498f640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6370102b00 0x7f6370197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 -- 172.21.15.105:0/422349482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6370102b00 msgr2=0x7f6370197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6370102b00 0x7f6370197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 -- 172.21.15.105:0/422349482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63701034c0 msgr2=0x7f637019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63701034c0 0x7f637019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 -- 172.21.15.105:0/422349482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637019f670 con 0x7f6370101910 2024-03-26T17:39:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.315+0000 7f6375190640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6370101910 0x7f6370197a50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f636c00e9c0 tx=0x7f636c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.316+0000 7f63667fc640 1 -- 172.21.15.105:0/422349482 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c00cd80 con 0x7f6370101910 2024-03-26T17:39:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.316+0000 7f63773f4640 1 -- 172.21.15.105:0/422349482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637019f960 con 0x7f6370101910 2024-03-26T17:39:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.316+0000 7f63667fc640 1 -- 172.21.15.105:0/422349482 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f636c0035f0 con 0x7f6370101910 2024-03-26T17:39:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.316+0000 7f63667fc640 1 -- 172.21.15.105:0/422349482 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c010690 con 0x7f6370101910 2024-03-26T17:39:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.316+0000 7f63773f4640 1 -- 172.21.15.105:0/422349482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637019ff00 con 0x7f6370101910 2024-03-26T17:39:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.318+0000 7f63667fc640 1 -- 172.21.15.105:0/422349482 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f636c0037a0 con 0x7f6370101910 2024-03-26T17:39:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.318+0000 7f6379fdd740 1 Processor -- start 2024-03-26T17:39:46.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.318+0000 7f6379fdd740 1 -- start start 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f3fc0 0x562edd5f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6379fdd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562edd5580b0 con 0x562edd5f35d0 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6379fdd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562edd500960 con 0x562edd5f9130 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6379fdd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562edd554060 con 0x562edd5f3fc0 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6375991640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6375991640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5f39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47888/0 (socket says 172.21.15.105:47888) 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6375991640 1 -- 172.21.15.105:0/1393706828 learned_addr learned my addr 172.21.15.105:0/1393706828 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6375190640 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f3fc0 0x562edd5f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f637498f640 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f63667fc640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f635c0a2fc0 0x7f635c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f637498f640 1 -- 172.21.15.105:0/1393706828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f3fc0 msgr2=0x562edd5f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f637498f640 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f3fc0 0x562edd5f8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f637498f640 1 -- 172.21.15.105:0/1393706828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 msgr2=0x562edd5f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f637498f640 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f637498f640 1 -- 172.21.15.105:0/1393706828 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562edd512c60 con 0x562edd5f9130 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f6375991640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f635c0a2fc0 0x7f635c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f637498f640 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fb5c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6360009bd0 tx=0x7f6360011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba09114d6a7c74eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.319+0000 7f63667fc640 1 -- 172.21.15.105:0/422349482 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f636c0cf7a0 con 0x7f6370101910 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f635b7fe640 1 -- 172.21.15.105:0/1393706828 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360004230 con 0x562edd5f9130 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f635b7fe640 1 -- 172.21.15.105:0/1393706828 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6360009e40 con 0x562edd5f9130 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f635b7fe640 1 -- 172.21.15.105:0/1393706828 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360004ca0 con 0x562edd5f9130 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f6379fdd740 1 -- 172.21.15.105:0/1393706828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 msgr2=0x562edd5fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f6375991640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f635c0a2fc0 0x7f635c0a5480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x562edd5f4820 tx=0x7f636802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f6379fdd740 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fb5c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6360009bd0 tx=0x7f6360011810 comp rx=0 tx=0).stop 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f6379fdd740 1 -- 172.21.15.105:0/1393706828 shutdown_connections 2024-03-26T17:39:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f6379fdd740 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fb5c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f6379fdd740 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f3fc0 0x562edd5f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f6379fdd740 1 --2- 172.21.15.105:0/1393706828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.320+0000 7f6379fdd740 1 -- 172.21.15.105:0/1393706828 >> 172.21.15.105:0/1393706828 conn(0x562edd5eeb40 msgr2=0x562edd5eef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 -- 172.21.15.105:0/1393706828 shutdown_connections 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 -- 172.21.15.105:0/1393706828 wait complete. 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 -- start start 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 Processor -- start 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f3fc0 0x562edd5fd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd6048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562edd5fc0e0 con 0x562edd5f9130 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562edd513db0 con 0x562edd5f3fc0 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562edd5fc260 con 0x562edd5f35d0 2024-03-26T17:39:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6379fdd740 10 client.0 _ll_register_callbacks cb 0x562edd518110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.321+0000 7f6375991640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd6048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f3fc0 0x562edd5fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f3fc0 0x562edd5fd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:47316/0 (socket says 172.21.15.105:47316) 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f6375991640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd6048f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47900/0 (socket says 172.21.15.105:47900) 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 -- 172.21.15.105:0/4118305802 learned_addr learned my addr 172.21.15.105:0/4118305802 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f6375190640 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 -- 172.21.15.105:0/4118305802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f35d0 msgr2=0x562edd5fd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 -- 172.21.15.105:0/4118305802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 msgr2=0x562edd6048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd6048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 -- 172.21.15.105:0/4118305802 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edd26c210 con 0x562edd5f3fc0 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f6375190640 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f6375991640 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd6048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f637498f640 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f3fc0 0x562edd5fd920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6360005bc0 tx=0x7f6360003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a79e0da654c21a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f6358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f6358ff9640 1 -- 172.21.15.105:0/4118305802 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360027020 con 0x562edd5f3fc0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f6358ff9640 1 -- 172.21.15.105:0/4118305802 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f6360003c10 con 0x562edd5f3fc0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.322+0000 7f6379fdd740 1 -- 172.21.15.105:0/4118305802 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562edd5655f0 con 0x562edd5f3fc0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6358ff9640 1 -- 172.21.15.105:0/4118305802 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6360032050 con 0x562edd5f3fc0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6379fdd740 10 client.25745 fetch_fsmap learned FSMap version 29 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6379fdd740 1 -- 172.21.15.105:0/4118305802 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562edd338620 con 0x562edd5f3fc0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6358ff9640 1 -- 172.21.15.105:0/4118305802 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562edd5655f0 con 0x562edd5f3fc0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6358ff9640 1 -- 172.21.15.105:0/4118305802 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6360018070 con 0x562edd5f3fc0 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6379fdd740 10 client.25745 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6379fdd740 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6379fdd740 20 client.25745 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6379fdd740 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562edd6091c0 0x562edd60b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6379fdd740 4 client.25745 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6379fdd740 1 -- 172.21.15.105:0/4118305802 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562edd55f940 con 0x562edd6091c0 2024-03-26T17:39:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.323+0000 7f6375190640 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562edd6091c0 0x562edd60b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.324+0000 7f6358ff9640 10 client.25745 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.324+0000 7f6375190640 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562edd6091c0 0x562edd60b5b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ded18c01a58c5b server_cookie=7985d91d9ecce192 in_seq=0 out_seq=0 2024-03-26T17:39:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6358ff9640 1 -- 172.21.15.105:0/4118305802 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562edd55f940 con 0x562edd6091c0 2024-03-26T17:39:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6358ff9640 10 client.25745 handle_command_reply: tid=0 2024-03-26T17:39:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 client.25745 shutdown 2024-03-26T17:39:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 -- 172.21.15.105:0/4118305802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f3fc0 msgr2=0x562edd5fd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f3fc0 0x562edd5fd920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6360005bc0 tx=0x7f6360003820 comp rx=0 tx=0).stop 2024-03-26T17:39:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 -- 172.21.15.105:0/4118305802 shutdown_connections 2024-03-26T17:39:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562edd6091c0 0x562edd60b5b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd6048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6358ff9640 0 client.25745 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f3fc0 0x562edd5fd920 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 --2- 172.21.15.105:0/4118305802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 -- 172.21.15.105:0/4118305802 >> 172.21.15.105:0/4118305802 conn(0x562edd5eeb40 msgr2=0x562edd5eddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 -- 172.21.15.105:0/4118305802 shutdown_connections 2024-03-26T17:39:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 1 -- 172.21.15.105:0/4118305802 wait complete. 2024-03-26T17:39:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.325+0000 7f6379fdd740 20 client.25745 trim_cache size 0 max 16384 2024-03-26T17:39:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.351+0000 7f6379fdd740 1 Processor -- start 2024-03-26T17:39:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.351+0000 7f6379fdd740 1 -- start start 2024-03-26T17:39:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 0x562edd5f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6379fdd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63701a8d40 con 0x562edd5f35d0 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6379fdd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63701aa880 con 0x562edd5f9130 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6379fdd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63701a9410 con 0x562edd6048b0 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47916/0 (socket says 172.21.15.105:47916) 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 -- 172.21.15.105:0/2471901325 learned_addr learned my addr 172.21.15.105:0/2471901325 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f637498f640 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375991640 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 0x562edd5f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 -- 172.21.15.105:0/2471901325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 msgr2=0x562edd5f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 0x562edd5f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 -- 172.21.15.105:0/2471901325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 msgr2=0x562edd5fd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 -- 172.21.15.105:0/2471901325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562edd604d10 con 0x562edd5f35d0 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f637498f640 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f6375190640 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f636c098810 tx=0x7f636c0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156ce1eae914e68f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f635affd640 1 -- 172.21.15.105:0/2471901325 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c0bc0b0 con 0x562edd5f35d0 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f635affd640 1 -- 172.21.15.105:0/2471901325 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f636c003b30 con 0x562edd5f35d0 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.352+0000 7f635affd640 1 -- 172.21.15.105:0/2471901325 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c09a190 con 0x562edd5f35d0 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 -- 172.21.15.105:0/2471901325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 msgr2=0x562edd5fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f636c098810 tx=0x7f636c0bc040 comp rx=0 tx=0).stop 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 -- 172.21.15.105:0/2471901325 shutdown_connections 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 0x562edd5f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f9130 0x562edd5fd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 --2- 172.21.15.105:0/2471901325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f35d0 0x562edd5fd3e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 -- 172.21.15.105:0/2471901325 >> 172.21.15.105:0/2471901325 conn(0x562edd5eeb40 msgr2=0x562edd5f1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 -- 172.21.15.105:0/2471901325 shutdown_connections 2024-03-26T17:39:46.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 -- 172.21.15.105:0/2471901325 wait complete. 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.353+0000 7f6379fdd740 1 -- start start 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6379fdd740 1 Processor -- start 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f35d0 0x562edd6011f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd601730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6379fdd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 0x562edd66d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6379fdd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63701aa1b0 con 0x562edd5f9130 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6379fdd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63701a9ae0 con 0x562edd5f35d0 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6379fdd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63701a6d20 con 0x562edd6048b0 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6379fdd740 10 client.0 _ll_register_callbacks cb 0x7f636c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd601730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd601730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47928/0 (socket says 172.21.15.105:47928) 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 -- 172.21.15.105:0/3863958739 learned_addr learned my addr 172.21.15.105:0/3863958739 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6375190640 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f35d0 0x562edd6011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6375991640 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 0x562edd66d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 -- 172.21.15.105:0/3863958739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 msgr2=0x562edd66d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 0x562edd66d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 -- 172.21.15.105:0/3863958739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f35d0 msgr2=0x562edd6011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f35d0 0x562edd6011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 -- 172.21.15.105:0/3863958739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edd66d7d0 con 0x562edd5f9130 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f637498f640 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd601730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6360005e00 tx=0x7f6360009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a28237479c6577 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.354+0000 7f6358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6358ff9640 1 -- 172.21.15.105:0/3863958739 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360015020 con 0x562edd5f9130 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6358ff9640 1 -- 172.21.15.105:0/3863958739 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f6360005240 con 0x562edd5f9130 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 1 -- 172.21.15.105:0/3863958739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562edd4dd770 con 0x562edd5f9130 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6358ff9640 1 -- 172.21.15.105:0/3863958739 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6360027040 con 0x562edd5f9130 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 10 client.15897 fetch_fsmap learned FSMap version 29 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 1 -- 172.21.15.105:0/3863958739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562edd66ea20 con 0x562edd5f9130 2024-03-26T17:39:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6358ff9640 1 -- 172.21.15.105:0/3863958739 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562edd4dd770 con 0x562edd5f9130 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6358ff9640 1 -- 172.21.15.105:0/3863958739 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6360018030 con 0x562edd5f9130 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 10 client.15897 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 20 client.15897 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562edd66f3d0 0x562edd6717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 4 client.15897 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6379fdd740 1 -- 172.21.15.105:0/3863958739 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562edd529930 con 0x562edd66f3d0 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.355+0000 7f6375190640 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562edd66f3d0 0x562edd6717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.356+0000 7f6358ff9640 10 client.15897 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.356+0000 7f6375190640 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562edd66f3d0 0x562edd6717f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47a9021eb2f1886b server_cookie=726526d8541fc1ab in_seq=0 out_seq=0 2024-03-26T17:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.398+0000 7f6358ff9640 1 -- 172.21.15.105:0/3863958739 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562edd529930 con 0x562edd66f3d0 2024-03-26T17:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.398+0000 7f6358ff9640 10 client.15897 handle_command_reply: tid=0 2024-03-26T17:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 client.15897 shutdown 2024-03-26T17:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 -- 172.21.15.105:0/3863958739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 msgr2=0x562edd601730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd601730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6360005e00 tx=0x7f6360009ad0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 -- 172.21.15.105:0/3863958739 shutdown_connections 2024-03-26T17:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562edd66f3d0 0x562edd6717f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6358ff9640 0 client.15897 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562edd6048b0 0x562edd66d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562edd5f9130 0x562edd601730 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 --2- 172.21.15.105:0/3863958739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562edd5f35d0 0x562edd6011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 -- 172.21.15.105:0/3863958739 >> 172.21.15.105:0/3863958739 conn(0x562edd5eeb40 msgr2=0x562edd5f48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 -- 172.21.15.105:0/3863958739 shutdown_connections 2024-03-26T17:39:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 1 -- 172.21.15.105:0/3863958739 wait complete. 2024-03-26T17:39:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.399+0000 7f6379fdd740 20 client.15897 trim_cache size 0 max 16384 2024-03-26T17:39:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:39:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:39:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:39:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:39:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 -- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f635c0a2fc0 msgr2=0x7f635c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f635c0a2fc0 0x7f635c0a5480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x562edd5f4820 tx=0x7f636802e040 comp rx=0 tx=0).stop 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 -- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6370101910 msgr2=0x7f6370197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6370101910 0x7f6370197a50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f636c00e9c0 tx=0x7f636c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 -- 172.21.15.105:0/422349482 shutdown_connections 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f635c0a2fc0 0x7f635c0a5480 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63701034c0 0x7f637019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6370102b00 0x7f6370197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 --2- 172.21.15.105:0/422349482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6370101910 0x7f6370197a50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 -- 172.21.15.105:0/422349482 >> 172.21.15.105:0/422349482 conn(0x7f63700fce80 msgr2=0x7f6370105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 -- 172.21.15.105:0/422349482 shutdown_connections 2024-03-26T17:39:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:46.400+0000 7f63773f4640 1 -- 172.21.15.105:0/422349482 wait complete. 2024-03-26T17:39:47.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 18081 3.88 MB/sec warmup 30 sec latency 17.183 ms 2024-03-26T17:39:47.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:39:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:39:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:47.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:39:47.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:39:47.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:39:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:39:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:39:48.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 18533 3.77 MB/sec warmup 31 sec latency 18.526 ms 2024-03-26T17:39:48.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:47 smithi144 ceph-mon[36095]: pgmap v390: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:39:48.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:47 smithi105 ceph-mon[32280]: pgmap v390: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:39:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:47 smithi135 ceph-mon[35636]: pgmap v390: 97 pgs: 97 active+clean; 26 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:39:49.017 INFO:tasks.workunit.client.0.smithi105.stdout: 1 19145 3.80 MB/sec warmup 32 sec latency 16.832 ms 2024-03-26T17:39:50.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 19783 3.70 MB/sec warmup 33 sec latency 22.718 ms 2024-03-26T17:39:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:49 smithi144 ceph-mon[36095]: pgmap v391: 97 pgs: 97 active+clean; 27 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-03-26T17:39:50.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:49 smithi105 ceph-mon[32280]: pgmap v391: 97 pgs: 97 active+clean; 27 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-03-26T17:39:50.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:49 smithi135 ceph-mon[35636]: pgmap v391: 97 pgs: 97 active+clean; 27 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-03-26T17:39:51.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 20400 3.67 MB/sec warmup 34 sec latency 19.228 ms 2024-03-26T17:39:52.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 21108 3.74 MB/sec warmup 35 sec latency 16.780 ms 2024-03-26T17:39:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:51 smithi144 ceph-mon[36095]: pgmap v392: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-03-26T17:39:52.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:51 smithi105 ceph-mon[32280]: pgmap v392: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-03-26T17:39:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:51 smithi135 ceph-mon[35636]: pgmap v392: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-03-26T17:39:53.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 21666 3.77 MB/sec warmup 36 sec latency 16.131 ms 2024-03-26T17:39:54.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 22134 3.68 MB/sec warmup 37 sec latency 22.869 ms 2024-03-26T17:39:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:53 smithi144 ceph-mon[36095]: pgmap v393: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:39:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:53 smithi105 ceph-mon[32280]: pgmap v393: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:39:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:53 smithi135 ceph-mon[35636]: pgmap v393: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:39:55.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 22748 3.70 MB/sec warmup 38 sec latency 19.925 ms 2024-03-26T17:39:56.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 23406 3.63 MB/sec warmup 39 sec latency 17.098 ms 2024-03-26T17:39:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:55 smithi144 ceph-mon[36095]: pgmap v394: 97 pgs: 97 active+clean; 27 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:39:56.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:55 smithi105 ceph-mon[32280]: pgmap v394: 97 pgs: 97 active+clean; 27 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:39:56.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:55 smithi135 ceph-mon[35636]: pgmap v394: 97 pgs: 97 active+clean; 27 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:39:57.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 24043 3.67 MB/sec warmup 40 sec latency 17.075 ms 2024-03-26T17:39:58.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 24656 3.66 MB/sec warmup 41 sec latency 15.862 ms 2024-03-26T17:39:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:57 smithi144 ceph-mon[36095]: pgmap v395: 97 pgs: 97 active+clean; 28 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-03-26T17:39:58.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:57 smithi105 ceph-mon[32280]: pgmap v395: 97 pgs: 97 active+clean; 28 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-03-26T17:39:58.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:57 smithi135 ceph-mon[35636]: pgmap v395: 97 pgs: 97 active+clean; 28 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-03-26T17:39:59.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 25229 3.69 MB/sec warmup 42 sec latency 16.611 ms 2024-03-26T17:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 -- 172.21.15.105:0/1859748213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 msgr2=0x7ff6dc109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 --2- 172.21.15.105:0/1859748213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 0x7ff6dc109a10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4009ca0 tx=0x7ff6d402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 -- 172.21.15.105:0/1859748213 shutdown_connections 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 --2- 172.21.15.105:0/1859748213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 0x7ff6dc109a10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 --2- 172.21.15.105:0/1859748213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6dc104c90 0x7ff6dc107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 --2- 172.21.15.105:0/1859748213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6dc102360 0x7ff6dc104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 -- 172.21.15.105:0/1859748213 >> 172.21.15.105:0/1859748213 conn(0x7ff6dc0fbd50 msgr2=0x7ff6dc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 -- 172.21.15.105:0/1859748213 shutdown_connections 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.501+0000 7ff6e393b640 1 -- 172.21.15.105:0/1859748213 wait complete. 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e393b640 1 Processor -- start 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e393b640 1 -- start start 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e393b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6dc102360 0x7ff6dc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e393b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6dc104c90 0x7ff6dc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e393b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 0x7ff6dc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e393b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc10d430 con 0x7ff6dc1075c0 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e393b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc10d2b0 con 0x7ff6dc102360 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e393b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc10d5b0 con 0x7ff6dc104c90 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e1ed8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 0x7ff6dc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e0ed6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6dc104c90 0x7ff6dc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e1ed8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 0x7ff6dc19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51272/0 (socket says 172.21.15.105:51272) 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/229840401 learned_addr learned my addr 172.21.15.105:0/229840401 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6dc102360 0x7ff6dc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/229840401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6dc104c90 msgr2=0x7ff6dc197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6dc104c90 0x7ff6dc197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/229840401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6dc102360 msgr2=0x7ff6dc197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6dc102360 0x7ff6dc197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.502+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/229840401 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc19f660 con 0x7ff6dc1075c0 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.503+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 0x7ff6dc19eff0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff6d402c700 tx=0x7ff6d40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.503+0000 7ff6d27fc640 1 -- 172.21.15.105:0/229840401 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4041020 con 0x7ff6dc1075c0 2024-03-26T17:39:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.503+0000 7ff6d27fc640 1 -- 172.21.15.105:0/229840401 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6d4003e20 con 0x7ff6dc1075c0 2024-03-26T17:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.503+0000 7ff6e393b640 1 -- 172.21.15.105:0/229840401 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc19f8f0 con 0x7ff6dc1075c0 2024-03-26T17:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.503+0000 7ff6d27fc640 1 -- 172.21.15.105:0/229840401 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4039040 con 0x7ff6dc1075c0 2024-03-26T17:39:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.503+0000 7ff6e393b640 1 -- 172.21.15.105:0/229840401 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc19fde0 con 0x7ff6dc1075c0 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.505+0000 7ff6e6604740 1 Processor -- start 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6d27fc640 1 -- 172.21.15.105:0/229840401 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff6d4044020 con 0x7ff6dc1075c0 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e6604740 1 -- start start 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a6eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a6f130 0x556e10a715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e6604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556e109ce0b0 con 0x556e10a69fc0 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e6604740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556e10976960 con 0x556e10a6f130 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e6604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e109ca060 con 0x556e10a695d0 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e16d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a6eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e0ed6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e0ed6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a699f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49110/0 (socket says 172.21.15.105:49110) 2024-03-26T17:39:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/1049117892 learned_addr learned my addr 172.21.15.105:0/1049117892 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e16d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a6eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51278/0 (socket says 172.21.15.105:51278) 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.506+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/1049117892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a6f130 0x556e10a715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/1049117892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a6f130 msgr2=0x556e10a715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/1049117892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a6f130 0x556e10a715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/1049117892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 msgr2=0x556e10a6eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/1049117892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a6eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/1049117892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e10988c60 con 0x556e10a695d0 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6d27fc640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c80a2fc0 0x7ff6c80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c80a2fc0 0x7ff6c80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/1049117892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a699f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0177e0 tx=0x7ff6cc005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565203746b7afc51 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6d27fc640 1 -- 172.21.15.105:0/229840401 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7ff6d40f1600 con 0x7ff6dc1075c0 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6c77fe640 1 -- 172.21.15.105:0/1049117892 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc017d60 con 0x556e10a695d0 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6c77fe640 1 -- 172.21.15.105:0/1049117892 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6cc005020 con 0x556e10a695d0 2024-03-26T17:39:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.507+0000 7ff6c77fe640 1 -- 172.21.15.105:0/1049117892 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc011a80 con 0x556e10a695d0 2024-03-26T17:39:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c80a2fc0 0x7ff6c80a5480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x556e10a6a3c0 tx=0x7ff6d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 -- 172.21.15.105:0/1049117892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 msgr2=0x556e10a699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1049117892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a699f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0177e0 tx=0x7ff6cc005850 comp rx=0 tx=0).stop 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 -- 172.21.15.105:0/1049117892 shutdown_connections 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1049117892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a6f130 0x556e10a715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1049117892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a6eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1049117892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a699f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 -- 172.21.15.105:0/1049117892 >> 172.21.15.105:0/1049117892 conn(0x556e10a64b40 msgr2=0x556e10a64f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 -- 172.21.15.105:0/1049117892 shutdown_connections 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.508+0000 7ff6e6604740 1 -- 172.21.15.105:0/1049117892 wait complete. 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 1 -- start start 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 1 Processor -- start 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a73920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a6f130 0x556e10a7a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556e10a720e0 con 0x556e10a69fc0 2024-03-26T17:39:59.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556e10989db0 con 0x556e10a695d0 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e10a72260 con 0x556e10a6f130 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e6604740 10 client.0 _ll_register_callbacks cb 0x556e1098e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e0ed6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a73920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e16d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e0ed6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a73920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51282/0 (socket says 172.21.15.105:51282) 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/1530825429 learned_addr learned my addr 172.21.15.105:0/1530825429 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.509+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a6f130 0x556e10a7a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/1530825429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a6f130 msgr2=0x556e10a7a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a6f130 0x556e10a7a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/1530825429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 msgr2=0x556e10a733e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a733e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/1530825429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e106e2210 con 0x556e10a69fc0 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a6f130 0x556e10a7a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a73920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009db0 tx=0x7ff6cc011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b32db7cc30c67c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/1530825429 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc017d60 con 0x556e10a69fc0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/1530825429 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7ff6cc004550 con 0x556e10a69fc0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e6604740 1 -- 172.21.15.105:0/1530825429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6cc005020 con 0x556e10a69fc0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/1530825429 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff6cc004750 con 0x556e10a69fc0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e6604740 10 client.15909 fetch_fsmap learned FSMap version 29 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e6604740 1 -- 172.21.15.105:0/1530825429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e107ae620 con 0x556e10a69fc0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/1530825429 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6cc005020 con 0x556e10a69fc0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/1530825429 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff6cc024b30 con 0x556e10a69fc0 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e6604740 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.510+0000 7ff6e6604740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.511+0000 7ff6e6604740 20 client.15909 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.511+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e10a7ef10 0x556e10a81300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.511+0000 7ff6e6604740 4 client.15909 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.511+0000 7ff6e6604740 1 -- 172.21.15.105:0/1530825429 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e109db5f0 con 0x556e10a7ef10 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.511+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e10a7ef10 0x556e10a81300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.511+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e10a7ef10 0x556e10a81300 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef794aa8ca47d7a1 server_cookie=6ad617dbf01a2f6d in_seq=0 out_seq=0 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.511+0000 7ff6c4ff9640 10 client.15909 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/1530825429 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556e109db5f0 con 0x556e10a7ef10 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6c4ff9640 10 client.15909 handle_command_reply: tid=0 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 client.15909 shutdown 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 -- 172.21.15.105:0/1530825429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 msgr2=0x556e10a73920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a73920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009db0 tx=0x7ff6cc011a70 comp rx=0 tx=0).stop 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 -- 172.21.15.105:0/1530825429 shutdown_connections 2024-03-26T17:39:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e10a7ef10 0x556e10a81300 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a6f130 0x556e10a7a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6c4ff9640 0 client.15909 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a69fc0 0x556e10a73920 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 --2- 172.21.15.105:0/1530825429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 -- 172.21.15.105:0/1530825429 >> 172.21.15.105:0/1530825429 conn(0x556e10a64b40 msgr2=0x556e10a63df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.512+0000 7ff6e6604740 1 -- 172.21.15.105:0/1530825429 shutdown_connections 2024-03-26T17:39:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.513+0000 7ff6e6604740 1 -- 172.21.15.105:0/1530825429 wait complete. 2024-03-26T17:39:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.513+0000 7ff6e6604740 20 client.15909 trim_cache size 0 max 16384 2024-03-26T17:39:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e6604740 1 Processor -- start 2024-03-26T17:39:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e6604740 1 -- start start 2024-03-26T17:39:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a6eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a69fc0 0x556e10a7a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10a6aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e6604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1a6ab0 con 0x556e10a6f130 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e6604740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1a85f0 con 0x556e10a695d0 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e6604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1a7180 con 0x556e10a69fc0 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e1ed8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10a6aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e1ed8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10a6aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51286/0 (socket says 172.21.15.105:51286) 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/4134808940 learned_addr learned my addr 172.21.15.105:0/4134808940 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a69fc0 0x556e10a7a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a6eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/4134808940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a69fc0 msgr2=0x556e10a7a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a69fc0 0x556e10a7a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/4134808940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 msgr2=0x556e10a6eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a6eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.535+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/4134808940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e10a81980 con 0x556e10a6f130 2024-03-26T17:39:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.536+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10a6aa60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40ba670 tx=0x7ff6d40bd650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafd5b5ff301fade server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.536+0000 7ff6c6ffd640 1 -- 172.21.15.105:0/4134808940 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40bdcf0 con 0x556e10a6f130 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.536+0000 7ff6c6ffd640 1 -- 172.21.15.105:0/4134808940 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6d40c1e40 con 0x556e10a6f130 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.536+0000 7ff6c6ffd640 1 -- 172.21.15.105:0/4134808940 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4004da0 con 0x556e10a6f130 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.536+0000 7ff6e6604740 1 -- 172.21.15.105:0/4134808940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 msgr2=0x556e10a6aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.536+0000 7ff6e6604740 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10a6aa60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40ba670 tx=0x7ff6d40bd650 comp rx=0 tx=0).stop 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 -- 172.21.15.105:0/4134808940 shutdown_connections 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10a6aa60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a69fc0 0x556e10a7a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 --2- 172.21.15.105:0/4134808940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a695d0 0x556e10a6eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 -- 172.21.15.105:0/4134808940 >> 172.21.15.105:0/4134808940 conn(0x556e10a64b40 msgr2=0x556e10a6e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 -- 172.21.15.105:0/4134808940 shutdown_connections 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 -- 172.21.15.105:0/4134808940 wait complete. 2024-03-26T17:39:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 -- start start 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 Processor -- start 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a69fc0 0x556e10a77a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10ae3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1a7f20 con 0x556e10a6f130 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1a7850 con 0x556e10a69fc0 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc1a4a90 con 0x556e10a695d0 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.537+0000 7ff6e6604740 10 client.0 _ll_register_callbacks cb 0x7ff6d80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e0ed6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a69fc0 0x556e10a77a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e1ed8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10ae3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e1ed8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10ae3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51300/0 (socket says 172.21.15.105:51300) 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e0ed6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a69fc0 0x556e10a77a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50514/0 (socket says 172.21.15.105:50514) 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e16d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e1ed8640 1 -- 172.21.15.105:0/598635205 learned_addr learned my addr 172.21.15.105:0/598635205 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:39:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/598635205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 msgr2=0x556e10a774c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a774c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/598635205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 msgr2=0x556e10ae3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10ae3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e0ed6640 1 -- 172.21.15.105:0/598635205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e10a7aad0 con 0x556e10a69fc0 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e1ed8640 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10ae3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a774c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e0ed6640 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a69fc0 0x556e10a77a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc024ea0 tx=0x7ff6cc02c5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa59732fa3d0aecf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/598635205 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc03b070 con 0x556e10a69fc0 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/598635205 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7ff6cc004070 con 0x556e10a69fc0 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.538+0000 7ff6e6604740 1 -- 172.21.15.105:0/598635205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d40c1e40 con 0x556e10a69fc0 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/598635205 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff6cc02c6d0 con 0x556e10a69fc0 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e6604740 10 client.25769 fetch_fsmap learned FSMap version 29 2024-03-26T17:39:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e6604740 1 -- 172.21.15.105:0/598635205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e10ae46f0 con 0x556e10a69fc0 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/598635205 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6d40c1e40 con 0x556e10a69fc0 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/598635205 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff6cc02ca70 con 0x556e10a69fc0 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e6604740 10 client.25769 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e6604740 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e6604740 20 client.25769 populate_metadata read hostname 'smithi105' 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e6604740 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e10ae5010 0x556e10ae7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e6604740 4 client.25769 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e6604740 1 -- 172.21.15.105:0/598635205 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e10a731d0 con 0x556e10ae5010 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.539+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e10ae5010 0x556e10ae7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.540+0000 7ff6c4ff9640 10 client.25769 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.540+0000 7ff6e16d7640 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e10ae5010 0x556e10ae7430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b1a887a698b387 server_cookie=3049e849c73bb839 in_seq=0 out_seq=0 2024-03-26T17:39:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6c4ff9640 1 -- 172.21.15.105:0/598635205 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e10a731d0 con 0x556e10ae5010 2024-03-26T17:39:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6c4ff9640 10 client.25769 handle_command_reply: tid=0 2024-03-26T17:39:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 client.25769 shutdown 2024-03-26T17:39:59.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 -- 172.21.15.105:0/598635205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a69fc0 msgr2=0x556e10a77a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a69fc0 0x556e10a77a00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc024ea0 tx=0x7ff6cc02c5c0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 -- 172.21.15.105:0/598635205 shutdown_connections 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e10ae5010 0x556e10ae7430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e10a6f130 0x556e10ae3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e10a69fc0 0x556e10a77a00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 --2- 172.21.15.105:0/598635205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e10a695d0 0x556e10a774c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6c4ff9640 0 client.25769 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 -- 172.21.15.105:0/598635205 >> 172.21.15.105:0/598635205 conn(0x556e10a64b40 msgr2=0x556e10a63560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 -- 172.21.15.105:0/598635205 shutdown_connections 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.783+0000 7ff6e6604740 1 -- 172.21.15.105:0/598635205 wait complete. 2024-03-26T17:39:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e6604740 20 client.25769 trim_cache size 0 max 16384 2024-03-26T17:39:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:39:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:39:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:39:59.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:39:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 -- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c80a2fc0 msgr2=0x7ff6c80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c80a2fc0 0x7ff6c80a5480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x556e10a6a3c0 tx=0x7ff6d8005f90 comp rx=0 tx=0).stop 2024-03-26T17:39:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 -- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 msgr2=0x7ff6dc19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:39:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 0x7ff6dc19eff0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff6d402c700 tx=0x7ff6d40328a0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 -- 172.21.15.105:0/229840401 shutdown_connections 2024-03-26T17:39:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6c80a2fc0 0x7ff6c80a5480 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6dc1075c0 0x7ff6dc19eff0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6dc104c90 0x7ff6dc197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 --2- 172.21.15.105:0/229840401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6dc102360 0x7ff6dc197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:39:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 -- 172.21.15.105:0/229840401 >> 172.21.15.105:0/229840401 conn(0x7ff6dc0fbd50 msgr2=0x7ff6dc102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:39:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 -- 172.21.15.105:0/229840401 shutdown_connections 2024-03-26T17:39:59.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:39:59.784+0000 7ff6e393b640 1 -- 172.21.15.105:0/229840401 wait complete. 2024-03-26T17:40:00.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 25672 3.61 MB/sec warmup 43 sec latency 41.799 ms 2024-03-26T17:40:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:39:59 smithi144 ceph-mon[36095]: pgmap v396: 97 pgs: 97 active+clean; 28 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:40:00.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:39:59 smithi105 ceph-mon[32280]: pgmap v396: 97 pgs: 97 active+clean; 28 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:40:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:39:59 smithi135 ceph-mon[35636]: pgmap v396: 97 pgs: 97 active+clean; 28 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:40:01.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 26284 3.63 MB/sec warmup 44 sec latency 17.713 ms 2024-03-26T17:40:01.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:00 smithi144 ceph-mon[36095]: overall HEALTH_OK 2024-03-26T17:40:01.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:00 smithi105 ceph-mon[32280]: overall HEALTH_OK 2024-03-26T17:40:01.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:00 smithi135 ceph-mon[35636]: overall HEALTH_OK 2024-03-26T17:40:02.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 26957 3.57 MB/sec warmup 45 sec latency 16.780 ms 2024-03-26T17:40:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:01 smithi144 ceph-mon[36095]: pgmap v397: 97 pgs: 97 active+clean; 28 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:40:02.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:01 smithi105 ceph-mon[32280]: pgmap v397: 97 pgs: 97 active+clean; 28 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:40:02.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:01 smithi135 ceph-mon[35636]: pgmap v397: 97 pgs: 97 active+clean; 28 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:40:03.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 27591 3.61 MB/sec warmup 46 sec latency 17.012 ms 2024-03-26T17:40:03.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:40:03.555 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:40:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 -- 172.21.15.105:0/3955642791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17601075a0 msgr2=0x7f17601099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 --2- 172.21.15.105:0/3955642791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17601075a0 0x7f17601099f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1758009be0 tx=0x7f175802bef0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 -- 172.21.15.105:0/3955642791 shutdown_connections 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 --2- 172.21.15.105:0/3955642791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17601075a0 0x7f17601099f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 --2- 172.21.15.105:0/3955642791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1760104c70 0x7f1760107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 --2- 172.21.15.105:0/3955642791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1760102340 0x7f1760104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 -- 172.21.15.105:0/3955642791 >> 172.21.15.105:0/3955642791 conn(0x7f17600fbd50 msgr2=0x7f17600fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 -- 172.21.15.105:0/3955642791 shutdown_connections 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 -- 172.21.15.105:0/3955642791 wait complete. 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.674+0000 7f176673b640 1 Processor -- start 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f176673b640 1 -- start start 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f176673b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1760102340 0x7f1760197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f176673b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1760104c70 0x7f1760198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f176673b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17601075a0 0x7f176019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f176673b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f176010d3f0 con 0x7f17601075a0 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f176673b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f176010d270 con 0x7f1760102340 2024-03-26T17:40:03.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f176673b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f176010d570 con 0x7f1760104c70 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f1764cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17601075a0 0x7f176019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1760104c70 0x7f1760198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f1764cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17601075a0 0x7f176019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51328/0 (socket says 172.21.15.105:51328) 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f175ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1760102340 0x7f1760197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f1764cd8640 1 -- 172.21.15.105:0/3204754986 learned_addr learned my addr 172.21.15.105:0/3204754986 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f1764cd8640 1 -- 172.21.15.105:0/3204754986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1760104c70 msgr2=0x7f1760198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f1764cd8640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1760104c70 0x7f1760198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f1764cd8640 1 -- 172.21.15.105:0/3204754986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1760102340 msgr2=0x7f1760197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f1764cd8640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1760102340 0x7f1760197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f1764cd8640 1 -- 172.21.15.105:0/3204754986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176019f710 con 0x7f17601075a0 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f175ffff640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1760102340 0x7f1760197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.675+0000 7f175f7fe640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1760104c70 0x7f1760198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.676+0000 7f1764cd8640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17601075a0 0x7f176019f0a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1758004af0 tx=0x7f1758033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:03.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.676+0000 7f175d7fa640 1 -- 172.21.15.105:0/3204754986 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758043020 con 0x7f17601075a0 2024-03-26T17:40:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.676+0000 7f175d7fa640 1 -- 172.21.15.105:0/3204754986 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f175802ca60 con 0x7f17601075a0 2024-03-26T17:40:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.676+0000 7f175d7fa640 1 -- 172.21.15.105:0/3204754986 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758034430 con 0x7f17601075a0 2024-03-26T17:40:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.676+0000 7f176673b640 1 -- 172.21.15.105:0/3204754986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176019f9a0 con 0x7f17601075a0 2024-03-26T17:40:03.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.676+0000 7f176673b640 1 -- 172.21.15.105:0/3204754986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176019ff40 con 0x7f17601075a0 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.678+0000 7f175d7fa640 1 -- 172.21.15.105:0/3204754986 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1758046020 con 0x7f17601075a0 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f17693ff740 1 Processor -- start 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f175d7fa640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17480a2fa0 0x7f17480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f17693ff740 1 -- start start 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f175f7fe640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17480a2fa0 0x7f17480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f17693ff740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f15cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f175d7fa640 1 -- 172.21.15.105:0/3204754986 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f17580f1a10 con 0x7f17601075a0 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f17693ff740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15d2e0 0x56392f161de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f17693ff740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f17693ff740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56392f0c1980 con 0x56392f162450 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f17693ff740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56392f069be0 con 0x56392f15d2e0 2024-03-26T17:40:03.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f17693ff740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56392f0bd930 con 0x56392f15c8f0 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15d2e0 0x56392f161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f175ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f15cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f1764cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f1764cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1648e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51344/0 (socket says 172.21.15.105:51344) 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f175ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f15cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49160/0 (socket says 172.21.15.105:49160) 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15d2e0 0x56392f161de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50546/0 (socket says 172.21.15.105:50546) 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f1764cd8640 1 -- 172.21.15.105:0/453804252 learned_addr learned my addr 172.21.15.105:0/453804252 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f1764cd8640 1 -- 172.21.15.105:0/453804252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 msgr2=0x56392f15cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f1764cd8640 1 --2- 172.21.15.105:0/453804252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f15cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f1764cd8640 1 -- 172.21.15.105:0/453804252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15d2e0 msgr2=0x56392f161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f175f7fe640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17480a2fa0 0x7f17480a5460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1760199000 tx=0x7f1754011cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f1764cd8640 1 --2- 172.21.15.105:0/453804252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15d2e0 0x56392f161de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.679+0000 7f1764cd8640 1 -- 172.21.15.105:0/453804252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56392ee9f7d0 con 0x56392f162450 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.680+0000 7f1764cd8640 1 --2- 172.21.15.105:0/453804252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1648e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f17580c65e0 tx=0x7f17580ba4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00f04a5b3bb5dd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:03.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.680+0000 7f17467fc640 1 -- 172.21.15.105:0/453804252 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758033e80 con 0x56392f162450 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.680+0000 7f17467fc640 1 -- 172.21.15.105:0/453804252 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17580be800 con 0x56392f162450 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.680+0000 7f17467fc640 1 -- 172.21.15.105:0/453804252 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758003ca0 con 0x56392f162450 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 -- 172.21.15.105:0/453804252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 msgr2=0x56392f1648e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 --2- 172.21.15.105:0/453804252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1648e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f17580c65e0 tx=0x7f17580ba4c0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 -- 172.21.15.105:0/453804252 shutdown_connections 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 --2- 172.21.15.105:0/453804252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1648e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 --2- 172.21.15.105:0/453804252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15d2e0 0x56392f161de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 --2- 172.21.15.105:0/453804252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f15cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 -- 172.21.15.105:0/453804252 >> 172.21.15.105:0/453804252 conn(0x56392f157e60 msgr2=0x56392f158260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 -- 172.21.15.105:0/453804252 shutdown_connections 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 -- 172.21.15.105:0/453804252 wait complete. 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 -- start start 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.681+0000 7f17693ff740 1 Processor -- start 2024-03-26T17:40:03.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f17693ff740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15c8f0 0x56392f16ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f17693ff740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f16b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f17693ff740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f17693ff740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56392f1695d0 con 0x56392f162450 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f17693ff740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56392f07d160 con 0x56392f15c8f0 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f17693ff740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56392f169750 con 0x56392f15d2e0 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f17693ff740 10 client.0 _ll_register_callbacks cb 0x56392f0b9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f1764cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f175ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15c8f0 0x56392f16ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f1764cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51348/0 (socket says 172.21.15.105:51348) 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f1764cd8640 1 -- 172.21.15.105:0/623480499 learned_addr learned my addr 172.21.15.105:0/623480499 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f175f7fe640 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f16b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f175ffff640 1 -- 172.21.15.105:0/623480499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 msgr2=0x56392f16b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f175ffff640 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f16b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f175ffff640 1 -- 172.21.15.105:0/623480499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 msgr2=0x56392f1721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f175ffff640 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f175ffff640 1 -- 172.21.15.105:0/623480499 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56392edd53f0 con 0x56392f15c8f0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f1764cd8640 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f175ffff640 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15c8f0 0x56392f16ac20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x56392f15db40 tx=0x7f175000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b44a26200ef73f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f173f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f173f7fe640 1 -- 172.21.15.105:0/623480499 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750012110 con 0x56392f15c8f0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f173f7fe640 1 -- 172.21.15.105:0/623480499 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f1750004030 con 0x56392f15c8f0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.682+0000 7f17693ff740 1 -- 172.21.15.105:0/623480499 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17580be800 con 0x56392f15c8f0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f173f7fe640 1 -- 172.21.15.105:0/623480499 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17500122b0 con 0x56392f15c8f0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f17693ff740 10 client.25781 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f17693ff740 1 -- 172.21.15.105:0/623480499 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56392ed950f0 con 0x56392f15c8f0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f173f7fe640 1 -- 172.21.15.105:0/623480499 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17580be800 con 0x56392f15c8f0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f173f7fe640 1 -- 172.21.15.105:0/623480499 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1750010430 con 0x56392f15c8f0 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f17693ff740 10 client.25781 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f17693ff740 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f17693ff740 20 client.25781 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:03.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f17693ff740 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56392f176a80 0x56392f178f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f17693ff740 4 client.25781 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f17693ff740 1 -- 172.21.15.105:0/623480499 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56392f0ceec0 con 0x56392f176a80 2024-03-26T17:40:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.683+0000 7f175f7fe640 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56392f176a80 0x56392f178f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f173f7fe640 10 client.25781 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:03.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f175f7fe640 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56392f176a80 0x56392f178f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1c054f14d1fb84 server_cookie=68d204a7c518c2ae in_seq=0 out_seq=0 2024-03-26T17:40:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f173f7fe640 1 -- 172.21.15.105:0/623480499 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56392f0ceec0 con 0x56392f176a80 2024-03-26T17:40:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f173f7fe640 10 client.25781 handle_command_reply: tid=0 2024-03-26T17:40:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 client.25781 shutdown 2024-03-26T17:40:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 -- 172.21.15.105:0/623480499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15c8f0 msgr2=0x56392f16ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15c8f0 0x56392f16ac20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x56392f15db40 tx=0x7f175000f790 comp rx=0 tx=0).stop 2024-03-26T17:40:03.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 -- 172.21.15.105:0/623480499 shutdown_connections 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56392f176a80 0x56392f178f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f162450 0x56392f1721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f173f7fe640 0 client.25781 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f16b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 --2- 172.21.15.105:0/623480499 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f15c8f0 0x56392f16ac20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.684+0000 7f17693ff740 1 -- 172.21.15.105:0/623480499 >> 172.21.15.105:0/623480499 conn(0x56392f157e60 msgr2=0x56392f157080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.685+0000 7f17693ff740 1 -- 172.21.15.105:0/623480499 shutdown_connections 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.685+0000 7f17693ff740 1 -- 172.21.15.105:0/623480499 wait complete. 2024-03-26T17:40:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.685+0000 7f17693ff740 20 client.25781 trim_cache size 0 max 16384 2024-03-26T17:40:03.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.704+0000 7f17693ff740 1 Processor -- start 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.704+0000 7f17693ff740 1 -- start start 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.704+0000 7f17693ff740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15c8f0 0x56392f161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.704+0000 7f17693ff740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f172160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.704+0000 7f17693ff740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f16a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.704+0000 7f17693ff740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17601a6c20 con 0x56392f15c8f0 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.704+0000 7f17693ff740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17601a8760 con 0x56392f162450 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.704+0000 7f17693ff740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17601a72f0 con 0x56392f15d2e0 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15c8f0 0x56392f161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15c8f0 0x56392f161b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51350/0 (socket says 172.21.15.105:51350) 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 -- 172.21.15.105:0/3331151906 learned_addr learned my addr 172.21.15.105:0/3331151906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175f7fe640 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f172160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f1764cd8640 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f16a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 -- 172.21.15.105:0/3331151906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 msgr2=0x56392f172160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f172160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 -- 172.21.15.105:0/3331151906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 msgr2=0x56392f16a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f16a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 -- 172.21.15.105:0/3331151906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56392f1538d0 con 0x56392f15c8f0 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175f7fe640 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f172160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f175ffff640 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15c8f0 0x56392f161b70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f17500108a0 tx=0x7f1750005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636dfc3d85e01083 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f1745ffb640 1 -- 172.21.15.105:0/3331151906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750000e60 con 0x56392f15c8f0 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f1745ffb640 1 -- 172.21.15.105:0/3331151906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1750010430 con 0x56392f15c8f0 2024-03-26T17:40:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.705+0000 7f1745ffb640 1 -- 172.21.15.105:0/3331151906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750003850 con 0x56392f15c8f0 2024-03-26T17:40:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 -- 172.21.15.105:0/3331151906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15c8f0 msgr2=0x56392f161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15c8f0 0x56392f161b70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f17500108a0 tx=0x7f1750005970 comp rx=0 tx=0).stop 2024-03-26T17:40:03.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 -- 172.21.15.105:0/3331151906 shutdown_connections 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f16a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15d2e0 0x56392f172160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 --2- 172.21.15.105:0/3331151906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15c8f0 0x56392f161b70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 -- 172.21.15.105:0/3331151906 >> 172.21.15.105:0/3331151906 conn(0x56392f157e60 msgr2=0x56392f15b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 -- 172.21.15.105:0/3331151906 shutdown_connections 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 -- 172.21.15.105:0/3331151906 wait complete. 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.706+0000 7f17693ff740 1 -- start start 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f17693ff740 1 Processor -- start 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f17693ff740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f16f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f17693ff740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15d2e0 0x56392f16fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f17693ff740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f170c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f17693ff740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17601a8090 con 0x56392f15d2e0 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f17693ff740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17601a79c0 con 0x56392f162450 2024-03-26T17:40:03.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f17693ff740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17601a4c00 con 0x56392f15c8f0 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f17693ff740 10 client.0 _ll_register_callbacks cb 0x7f1750009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f1764cd8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f170c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f16f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15d2e0 0x56392f16fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f16f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49180/0 (socket says 172.21.15.105:49180) 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 -- 172.21.15.105:0/2216696271 learned_addr learned my addr 172.21.15.105:0/2216696271 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f1764cd8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f170c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50586/0 (socket says 172.21.15.105:50586) 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 -- 172.21.15.105:0/2216696271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 msgr2=0x56392f170c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f170c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 -- 172.21.15.105:0/2216696271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15d2e0 msgr2=0x56392f16fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15d2e0 0x56392f16fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 -- 172.21.15.105:0/2216696271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56392f171240 con 0x56392f15c8f0 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175f7fe640 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15d2e0 0x56392f16fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f175ffff640 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f16f700 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1750010810 tx=0x7f1750010920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9600f443d947cd11 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:03.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.707+0000 7f173f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f173f7fe640 1 -- 172.21.15.105:0/2216696271 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750010ca0 con 0x56392f15c8f0 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f173f7fe640 1 -- 172.21.15.105:0/2216696271 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f17500035f0 con 0x56392f15c8f0 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 1 -- 172.21.15.105:0/2216696271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1750010430 con 0x56392f15c8f0 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f173f7fe640 1 -- 172.21.15.105:0/2216696271 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f175003f050 con 0x56392f15c8f0 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 10 client.25783 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 1 -- 172.21.15.105:0/2216696271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56392f1dce00 con 0x56392f15c8f0 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f173f7fe640 1 -- 172.21.15.105:0/2216696271 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1750010430 con 0x56392f15c8f0 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f173f7fe640 1 -- 172.21.15.105:0/2216696271 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1750038060 con 0x56392f15c8f0 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 10 client.25783 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 20 client.25783 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56392f1dd5f0 0x56392f1dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 4 client.25783 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f17693ff740 1 -- 172.21.15.105:0/2216696271 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56392f0c9210 con 0x56392f1dd5f0 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.708+0000 7f175f7fe640 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56392f1dd5f0 0x56392f1dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.709+0000 7f173f7fe640 10 client.25783 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.709+0000 7f175f7fe640 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56392f1dd5f0 0x56392f1dfa10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9765a048c5f5d4 server_cookie=59e21965ae5bec48 in_seq=0 out_seq=0 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f173f7fe640 1 -- 172.21.15.105:0/2216696271 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56392f0c9210 con 0x56392f1dd5f0 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f173f7fe640 10 client.25783 handle_command_reply: tid=0 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 client.25783 shutdown 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 -- 172.21.15.105:0/2216696271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 msgr2=0x56392f16f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f16f700 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1750010810 tx=0x7f1750010920 comp rx=0 tx=0).stop 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 -- 172.21.15.105:0/2216696271 shutdown_connections 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56392f1dd5f0 0x56392f1dfa10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56392f162450 0x56392f170c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f173f7fe640 0 client.25783 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56392f15d2e0 0x56392f16fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 --2- 172.21.15.105:0/2216696271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56392f15c8f0 0x56392f16f700 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 -- 172.21.15.105:0/2216696271 >> 172.21.15.105:0/2216696271 conn(0x56392f157e60 msgr2=0x56392f15b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.710+0000 7f17693ff740 1 -- 172.21.15.105:0/2216696271 shutdown_connections 2024-03-26T17:40:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f17693ff740 1 -- 172.21.15.105:0/2216696271 wait complete. 2024-03-26T17:40:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f17693ff740 20 client.25783 trim_cache size 0 max 16384 2024-03-26T17:40:03.708 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:40:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:40:03.708 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:40:03.708 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:40:03.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 -- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17480a2fa0 msgr2=0x7f17480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17480a2fa0 0x7f17480a5460 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1760199000 tx=0x7f1754011cc0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 -- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17601075a0 msgr2=0x7f176019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17601075a0 0x7f176019f0a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1758004af0 tx=0x7f1758033c90 comp rx=0 tx=0).stop 2024-03-26T17:40:03.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 -- 172.21.15.105:0/3204754986 shutdown_connections 2024-03-26T17:40:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17480a2fa0 0x7f17480a5460 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17601075a0 0x7f176019f0a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1760104c70 0x7f1760198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 --2- 172.21.15.105:0/3204754986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1760102340 0x7f1760197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 -- 172.21.15.105:0/3204754986 >> 172.21.15.105:0/3204754986 conn(0x7f17600fbd50 msgr2=0x7f1760102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 -- 172.21.15.105:0/3204754986 shutdown_connections 2024-03-26T17:40:03.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.711+0000 7f176673b640 1 -- 172.21.15.105:0/3204754986 wait complete. 2024-03-26T17:40:03.715 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:40:03.715 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:40:03.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.846+0000 7fda71d28640 1 -- 172.21.15.105:0/4000506895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c102390 msgr2=0x7fda6c108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.846+0000 7fda71d28640 1 --2- 172.21.15.105:0/4000506895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c102390 0x7fda6c108c20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fda64009ca0 tx=0x7fda6402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.847+0000 7fda71d28640 1 -- 172.21.15.105:0/4000506895 shutdown_connections 2024-03-26T17:40:03.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.847+0000 7fda71d28640 1 --2- 172.21.15.105:0/4000506895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c102390 0x7fda6c108c20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.847+0000 7fda71d28640 1 --2- 172.21.15.105:0/4000506895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c1019d0 0x7fda6c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.847+0000 7fda71d28640 1 --2- 172.21.15.105:0/4000506895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda6c1007e0 0x7fda6c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.847+0000 7fda71d28640 1 -- 172.21.15.105:0/4000506895 >> 172.21.15.105:0/4000506895 conn(0x7fda6c0fbd50 msgr2=0x7fda6c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.847+0000 7fda71d28640 1 -- 172.21.15.105:0/4000506895 shutdown_connections 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.847+0000 7fda71d28640 1 -- 172.21.15.105:0/4000506895 wait complete. 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.847+0000 7fda71d28640 1 Processor -- start 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda71d28640 1 -- start start 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda71d28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c1007e0 0x7fda6c0695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda71d28640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda6c1019d0 0x7fda6c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda71d28640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c102390 0x7fda6c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda71d28640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10c310 con 0x7fda6c1007e0 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda71d28640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10c190 con 0x7fda6c1019d0 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda71d28640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10c490 con 0x7fda6c102390 2024-03-26T17:40:03.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c1007e0 0x7fda6c0695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c1007e0 0x7fda6c0695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51382/0 (socket says 172.21.15.105:51382) 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 -- 172.21.15.105:0/2079009552 learned_addr learned my addr 172.21.15.105:0/2079009552 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6affd640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda6c1019d0 0x7fda6c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6bfff640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c102390 0x7fda6c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 -- 172.21.15.105:0/2079009552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c102390 msgr2=0x7fda6c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c102390 0x7fda6c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 -- 172.21.15.105:0/2079009552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda6c1019d0 msgr2=0x7fda6c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda6c1019d0 0x7fda6c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 -- 172.21.15.105:0/2079009552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c19f700 con 0x7fda6c1007e0 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda6b7fe640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c1007e0 0x7fda6c0695b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00e9c0 tx=0x7fda5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.848+0000 7fda68ff9640 1 -- 172.21.15.105:0/2079009552 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00cd80 con 0x7fda6c1007e0 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.849+0000 7fda68ff9640 1 -- 172.21.15.105:0/2079009552 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda5c0035f0 con 0x7fda6c1007e0 2024-03-26T17:40:03.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.849+0000 7fda68ff9640 1 -- 172.21.15.105:0/2079009552 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c010690 con 0x7fda6c1007e0 2024-03-26T17:40:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.849+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c19f9f0 con 0x7fda6c1007e0 2024-03-26T17:40:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.849+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c19ff90 con 0x7fda6c1007e0 2024-03-26T17:40:03.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.850+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c107b00 con 0x7fda6c1007e0 2024-03-26T17:40:03.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.851+0000 7fda68ff9640 1 -- 172.21.15.105:0/2079009552 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fda5c0037a0 con 0x7fda6c1007e0 2024-03-26T17:40:03.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.851+0000 7fda68ff9640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda540a2fa0 0x7fda540a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:03.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.852+0000 7fda6affd640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda540a2fa0 0x7fda540a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:03.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.852+0000 7fda68ff9640 1 -- 172.21.15.105:0/2079009552 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fda5c0cf6f0 con 0x7fda6c1007e0 2024-03-26T17:40:03.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.852+0000 7fda6affd640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda540a2fa0 0x7fda540a5460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda6c069320 tx=0x7fda60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:03.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:03.856+0000 7fda68ff9640 1 -- 172.21.15.105:0/2079009552 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fda5c09c4e0 con 0x7fda6c1007e0 2024-03-26T17:40:03.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:03 smithi105 ceph-mon[32280]: pgmap v398: 97 pgs: 97 active+clean; 30 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:40:04.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 28211 3.60 MB/sec warmup 47 sec latency 15.785 ms 2024-03-26T17:40:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:03 smithi144 ceph-mon[36095]: pgmap v398: 97 pgs: 97 active+clean; 30 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:40:04.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.125+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda6c0624c0 con 0x7fda6c1007e0 2024-03-26T17:40:04.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.126+0000 7fda68ff9640 1 -- 172.21.15.105:0/2079009552 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7fda5c09bde0 con 0x7fda6c1007e0 2024-03-26T17:40:04.118 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:40:04.119 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:39:34.702865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_15879":{"gid":15879,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/2307931519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2307931519},{"type":"v1","addr":"172.21.15.135:6835","nonce":2307931519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:40:04.119 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 29 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda540a2fa0 msgr2=0x7fda540a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda540a2fa0 0x7fda540a5460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda6c069320 tx=0x7fda60005f90 comp rx=0 tx=0).stop 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c1007e0 msgr2=0x7fda6c0695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c1007e0 0x7fda6c0695b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00e9c0 tx=0x7fda5c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 shutdown_connections 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda540a2fa0 0x7fda540a5460 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda6c102390 0x7fda6c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda6c1019d0 0x7fda6c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 --2- 172.21.15.105:0/2079009552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda6c1007e0 0x7fda6c0695b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 >> 172.21.15.105:0/2079009552 conn(0x7fda6c0fbd50 msgr2=0x7fda6c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 shutdown_connections 2024-03-26T17:40:04.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.128+0000 7fda71d28640 1 -- 172.21.15.105:0/2079009552 wait complete. 2024-03-26T17:40:04.133 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:40:04.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:03 smithi135 ceph-mon[35636]: pgmap v398: 97 pgs: 97 active+clean; 30 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:40:04.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 -- 172.21.15.105:0/2406714103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1034a0 msgr2=0x7fa24c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 --2- 172.21.15.105:0/2406714103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1034a0 0x7fa24c109d30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa244009ca0 tx=0x7fa24402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 -- 172.21.15.105:0/2406714103 shutdown_connections 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 --2- 172.21.15.105:0/2406714103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1034a0 0x7fa24c109d30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 --2- 172.21.15.105:0/2406714103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa24c102ae0 0x7fa24c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 --2- 172.21.15.105:0/2406714103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa24c1018f0 0x7fa24c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 -- 172.21.15.105:0/2406714103 >> 172.21.15.105:0/2406714103 conn(0x7fa24c0fce60 msgr2=0x7fa24c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 -- 172.21.15.105:0/2406714103 shutdown_connections 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 -- 172.21.15.105:0/2406714103 wait complete. 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 Processor -- start 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.263+0000 7fa252b76640 1 -- start start 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa252b76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1018f0 0x7fa24c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa252b76640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa24c102ae0 0x7fa24c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa252b76640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa24c1034a0 0x7fa24c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa252b76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c10d400 con 0x7fa24c1018f0 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa252b76640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c10d280 con 0x7fa24c102ae0 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa252b76640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c10d580 con 0x7fa24c1034a0 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1018f0 0x7fa24c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1018f0 0x7fa24c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51404/0 (socket says 172.21.15.105:51404) 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 -- 172.21.15.105:0/3790626340 learned_addr learned my addr 172.21.15.105:0/3790626340 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa24c102ae0 0x7fa24c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa251113640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa24c1034a0 0x7fa24c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 -- 172.21.15.105:0/3790626340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa24c1034a0 msgr2=0x7fa24c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa24c1034a0 0x7fa24c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 -- 172.21.15.105:0/3790626340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa24c102ae0 msgr2=0x7fa24c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa24c102ae0 0x7fa24c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 -- 172.21.15.105:0/3790626340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c19f6f0 con 0x7fa24c1018f0 2024-03-26T17:40:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa250912640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1018f0 0x7fa24c197ad0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9c0 tx=0x7fa24000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa249ffb640 1 -- 172.21.15.105:0/3790626340 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24000cd80 con 0x7fa24c1018f0 2024-03-26T17:40:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa249ffb640 1 -- 172.21.15.105:0/3790626340 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2400035f0 con 0x7fa24c1018f0 2024-03-26T17:40:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa252b76640 1 -- 172.21.15.105:0/3790626340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c19f9e0 con 0x7fa24c1018f0 2024-03-26T17:40:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.264+0000 7fa249ffb640 1 -- 172.21.15.105:0/3790626340 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240010690 con 0x7fa24c1018f0 2024-03-26T17:40:04.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.265+0000 7fa252b76640 1 -- 172.21.15.105:0/3790626340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c19ff30 con 0x7fa24c1018f0 2024-03-26T17:40:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.267+0000 7fa249ffb640 1 -- 172.21.15.105:0/3790626340 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa2400037a0 con 0x7fa24c1018f0 2024-03-26T17:40:04.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.267+0000 7fa249ffb640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa249ffb640 1 -- 172.21.15.105:0/3790626340 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa2400cf7a0 con 0x7fa24c1018f0 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa255833740 1 Processor -- start 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa255833740 1 -- start start 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa255833740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a37e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa255833740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a3cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa255833740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a3fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa255833740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563c7299c970 con 0x563c72a38420 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa255833740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563c72944d70 con 0x563c72a3d590 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa255833740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563c72998920 con 0x563c72a37a30 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa250912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a37e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa251113640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a3fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa250912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a37e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49224/0 (socket says 172.21.15.105:49224) 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa250912640 1 -- 172.21.15.105:0/4067847213 learned_addr learned my addr 172.21.15.105:0/4067847213 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa24bfff640 1 --2- 172.21.15.105:0/4067847213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a3cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa2380095f0 tx=0x7fa238005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa250912640 1 -- 172.21.15.105:0/4067847213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 msgr2=0x563c72a3fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.268+0000 7fa250912640 1 --2- 172.21.15.105:0/4067847213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a3fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.269+0000 7fa250912640 1 -- 172.21.15.105:0/4067847213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 msgr2=0x563c72a3cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.269+0000 7fa250912640 1 --2- 172.21.15.105:0/4067847213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a3cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.269+0000 7fa250912640 1 -- 172.21.15.105:0/4067847213 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c726b03f0 con 0x563c72a37a30 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.269+0000 7fa250912640 1 --2- 172.21.15.105:0/4067847213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a37e50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2400a4370 tx=0x7fa24009ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7aef80a419a3c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.269+0000 7fa236ffd640 1 -- 172.21.15.105:0/4067847213 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24009be90 con 0x563c72a37a30 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.269+0000 7fa236ffd640 1 -- 172.21.15.105:0/4067847213 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24009ffe0 con 0x563c72a37a30 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.269+0000 7fa236ffd640 1 -- 172.21.15.105:0/4067847213 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400a4410 con 0x563c72a37a30 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 -- 172.21.15.105:0/4067847213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 msgr2=0x563c72a37e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 --2- 172.21.15.105:0/4067847213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a37e50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2400a4370 tx=0x7fa24009ba90 comp rx=0 tx=0).stop 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 -- 172.21.15.105:0/4067847213 shutdown_connections 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 --2- 172.21.15.105:0/4067847213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a3fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 --2- 172.21.15.105:0/4067847213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a3cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 --2- 172.21.15.105:0/4067847213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a37e50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 -- 172.21.15.105:0/4067847213 >> 172.21.15.105:0/4067847213 conn(0x563c72a32fa0 msgr2=0x563c72a333a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 -- 172.21.15.105:0/4067847213 shutdown_connections 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 -- 172.21.15.105:0/4067847213 wait complete. 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 -- start start 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 Processor -- start 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.270+0000 7fa255833740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a41f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa255833740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a42440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a41f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa255833740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a49480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a41f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49232/0 (socket says 172.21.15.105:49232) 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 -- 172.21.15.105:0/1618629184 learned_addr learned my addr 172.21.15.105:0/1618629184 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563c72a404e0 con 0x563c72a38420 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563c72958310 con 0x563c72a3d590 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563c72a40660 con 0x563c72a37a30 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa24bfff640 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a42440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa251113640 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a49480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa255833740 10 client.0 _ll_register_callbacks cb 0x563c72993c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 -- 172.21.15.105:0/1618629184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 msgr2=0x563c72a49480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a49480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 -- 172.21.15.105:0/1618629184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 msgr2=0x563c72a42440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a42440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 -- 172.21.15.105:0/1618629184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7277a7d0 con 0x563c72a37a30 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa251113640 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a49480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa250912640 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a41f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2400982f0 tx=0x7fa2400046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f552848aa3b0fd4f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa22bfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa22bfff640 1 -- 172.21.15.105:0/1618629184 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24009b4e0 con 0x563c72a37a30 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24009ffe0 con 0x563c72a37a30 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.271+0000 7fa22bfff640 1 -- 172.21.15.105:0/1618629184 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa240097be0 con 0x563c72a37a30 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa22bfff640 1 -- 172.21.15.105:0/1618629184 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa2400a48a0 con 0x563c72a37a30 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa255833740 10 client.25801 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c72a40530 con 0x563c72a37a30 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa22bfff640 1 -- 172.21.15.105:0/1618629184 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa24009ffe0 con 0x563c72a37a30 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa22bfff640 1 -- 172.21.15.105:0/1618629184 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa24009aee0 con 0x563c72a37a30 2024-03-26T17:40:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa255833740 10 client.25801 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa255833740 10 client.25801 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa255833740 20 client.25801 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa255833740 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x563c72a4d9a0 0x563c72a4fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa255833740 4 client.25801 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c729a9eb0 con 0x563c72a4d9a0 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa24bfff640 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x563c72a4d9a0 0x563c72a4fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.272+0000 7fa22bfff640 10 client.25801 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa24bfff640 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x563c72a4d9a0 0x563c72a4fdd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=38f581ff187dce6f server_cookie=e8bafbaaf4d9014a in_seq=0 out_seq=0 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa22bfff640 1 -- 172.21.15.105:0/1618629184 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563c729a9eb0 con 0x563c72a4d9a0 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa22bfff640 10 client.25801 handle_command_reply: tid=0 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 client.25801 shutdown 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 msgr2=0x563c72a41f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a41f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2400982f0 tx=0x7fa2400046c0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 shutdown_connections 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x563c72a4d9a0 0x563c72a4fdd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a3d590 0x563c72a49480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a42440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 --2- 172.21.15.105:0/1618629184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a41f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 >> 172.21.15.105:0/1618629184 conn(0x563c72a32fa0 msgr2=0x563c72a321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.273+0000 7fa22bfff640 0 client.25801 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.274+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 shutdown_connections 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.274+0000 7fa255833740 1 -- 172.21.15.105:0/1618629184 wait complete. 2024-03-26T17:40:04.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.274+0000 7fa255833740 20 client.25801 trim_cache size 0 max 16384 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa255833740 1 Processor -- start 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa255833740 1 -- start start 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa255833740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a42220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa255833740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a38420 0x563c72a49410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa255833740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a3d590 0x563c72a31930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa255833740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c1a6bd0 con 0x563c72a3d590 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa255833740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c1a8710 con 0x563c72a38420 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa255833740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c1a72a0 con 0x563c72a37a30 2024-03-26T17:40:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa24bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a38420 0x563c72a49410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa250912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a42220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa24bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a38420 0x563c72a49410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50638/0 (socket says 172.21.15.105:50638) 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa24bfff640 1 -- 172.21.15.105:0/3868405035 learned_addr learned my addr 172.21.15.105:0/3868405035 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa250912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a42220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49234/0 (socket says 172.21.15.105:49234) 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.288+0000 7fa251113640 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a3d590 0x563c72a31930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa250912640 1 -- 172.21.15.105:0/3868405035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a38420 msgr2=0x563c72a49410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa250912640 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a38420 0x563c72a49410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa250912640 1 -- 172.21.15.105:0/3868405035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a3d590 msgr2=0x563c72a31930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa250912640 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a3d590 0x563c72a31930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa250912640 1 -- 172.21.15.105:0/3868405035 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c72a407c0 con 0x563c72a37a30 2024-03-26T17:40:04.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a38420 0x563c72a49410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa250912640 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a42220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa24009be20 tx=0x7fa240004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7160a6113efac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa2367fc640 1 -- 172.21.15.105:0/3868405035 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24009ffe0 con 0x563c72a37a30 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa2367fc640 1 -- 172.21.15.105:0/3868405035 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa240080030 con 0x563c72a37a30 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.289+0000 7fa2367fc640 1 -- 172.21.15.105:0/3868405035 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24009b540 con 0x563c72a37a30 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 -- 172.21.15.105:0/3868405035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 msgr2=0x563c72a42220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a42220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa24009be20 tx=0x7fa240004730 comp rx=0 tx=0).stop 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 -- 172.21.15.105:0/3868405035 shutdown_connections 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a3d590 0x563c72a31930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a38420 0x563c72a49410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 --2- 172.21.15.105:0/3868405035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a37a30 0x563c72a42220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 -- 172.21.15.105:0/3868405035 >> 172.21.15.105:0/3868405035 conn(0x563c72a32fa0 msgr2=0x563c72a41b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 -- 172.21.15.105:0/3868405035 shutdown_connections 2024-03-26T17:40:04.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 -- 172.21.15.105:0/3868405035 wait complete. 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 -- start start 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.290+0000 7fa255833740 1 Processor -- start 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa255833740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a37a30 0x563c72a46260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa255833740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa255833740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a3d590 0x563c72ab1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa255833740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c1a8040 con 0x563c72a38420 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa255833740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c1a7970 con 0x563c72a37a30 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa255833740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c1a4bb0 con 0x563c72a3d590 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa255833740 10 client.0 _ll_register_callbacks cb 0x7fa238009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa24bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa250912640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a37a30 0x563c72a46260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa24bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51434/0 (socket says 172.21.15.105:51434) 2024-03-26T17:40:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa250912640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a37a30 0x563c72a46260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50646/0 (socket says 172.21.15.105:50646) 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa24bfff640 1 -- 172.21.15.105:0/3356144429 learned_addr learned my addr 172.21.15.105:0/3356144429 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa251113640 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a3d590 0x563c72ab1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa250912640 1 -- 172.21.15.105:0/3356144429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a3d590 msgr2=0x563c72ab1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa250912640 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a3d590 0x563c72ab1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa250912640 1 -- 172.21.15.105:0/3356144429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 msgr2=0x563c72a467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa250912640 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa250912640 1 -- 172.21.15.105:0/3356144429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c72ab2200 con 0x563c72a37a30 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa250912640 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a37a30 0x563c72a46260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa24009bb90 tx=0x7fa2400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f7b6e4fc3213512 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.291+0000 7fa22bfff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa22bfff640 1 -- 172.21.15.105:0/3356144429 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24009b4e0 con 0x563c72a37a30 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa22bfff640 1 -- 172.21.15.105:0/3356144429 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa240097be0 con 0x563c72a37a30 2024-03-26T17:40:04.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 1 -- 172.21.15.105:0/3356144429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa240080030 con 0x563c72a37a30 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa22bfff640 1 -- 172.21.15.105:0/3356144429 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa2400a11f0 con 0x563c72a37a30 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 10 client.25811 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 1 -- 172.21.15.105:0/3356144429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563c72ab33c0 con 0x563c72a37a30 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa22bfff640 1 -- 172.21.15.105:0/3356144429 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa240080030 con 0x563c72a37a30 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa22bfff640 1 -- 172.21.15.105:0/3356144429 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa2400b00d0 con 0x563c72a37a30 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 10 client.25811 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 10 client.25811 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 20 client.25811 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x563c72ab3ac0 0x563c72ab5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 4 client.25811 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa255833740 1 -- 172.21.15.105:0/3356144429 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c729a4200 con 0x563c72ab3ac0 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.292+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x563c72ab3ac0 0x563c72ab5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa24bfff640 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x563c72ab3ac0 0x563c72ab5eb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=47d03072caf9e8db server_cookie=329fa0fa51856f23 in_seq=0 out_seq=0 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa22bfff640 10 client.25811 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:40:04.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa22bfff640 1 -- 172.21.15.105:0/3356144429 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c729a4200 con 0x563c72ab3ac0 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa22bfff640 10 client.25811 handle_command_reply: tid=0 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 client.25811 shutdown 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 -- 172.21.15.105:0/3356144429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a37a30 msgr2=0x563c72a46260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a37a30 0x563c72a46260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa24009bb90 tx=0x7fa2400046c0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 -- 172.21.15.105:0/3356144429 shutdown_connections 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x563c72ab3ac0 0x563c72ab5eb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563c72a3d590 0x563c72ab1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa22bfff640 0 client.25811 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563c72a38420 0x563c72a467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 --2- 172.21.15.105:0/3356144429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563c72a37a30 0x563c72a46260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 -- 172.21.15.105:0/3356144429 >> 172.21.15.105:0/3356144429 conn(0x563c72a32fa0 msgr2=0x563c72a36a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 -- 172.21.15.105:0/3356144429 shutdown_connections 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 1 -- 172.21.15.105:0/3356144429 wait complete. 2024-03-26T17:40:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.293+0000 7fa255833740 20 client.25811 trim_cache size 0 max 16384 2024-03-26T17:40:04.297 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 -- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa23c0a2f20 msgr2=0x7fa23c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa2380095f0 tx=0x7fa238005f90 comp rx=0 tx=0).stop 2024-03-26T17:40:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 -- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1018f0 msgr2=0x7fa24c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1018f0 0x7fa24c197ad0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa24000e9c0 tx=0x7fa24000ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:04.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 -- 172.21.15.105:0/3790626340 shutdown_connections 2024-03-26T17:40:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa23c0a2f20 0x7fa23c0a53e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa24c1034a0 0x7fa24c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa24c102ae0 0x7fa24c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 --2- 172.21.15.105:0/3790626340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa24c1018f0 0x7fa24c197ad0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 -- 172.21.15.105:0/3790626340 >> 172.21.15.105:0/3790626340 conn(0x7fa24c0fce60 msgr2=0x7fa24c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 -- 172.21.15.105:0/3790626340 shutdown_connections 2024-03-26T17:40:04.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.294+0000 7fa252b76640 1 -- 172.21.15.105:0/3790626340 wait complete. 2024-03-26T17:40:04.299 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 -- 172.21.15.105:0/3861393937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 msgr2=0x7f4b30073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/3861393937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 0x7f4b30073d50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009900 tx=0x7f4b2002bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 -- 172.21.15.105:0/3861393937 shutdown_connections 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/3861393937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b301097b0 0x7f4b3010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/3861393937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b30074320 0x7f4b300747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/3861393937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 0x7f4b30073d50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 -- 172.21.15.105:0/3861393937 >> 172.21.15.105:0/3861393937 conn(0x7f4b300fce60 msgr2=0x7f4b300ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 -- 172.21.15.105:0/3861393937 shutdown_connections 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.423+0000 7f4b35ebd640 1 -- 172.21.15.105:0/3861393937 wait complete. 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b35ebd640 1 Processor -- start 2024-03-26T17:40:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b35ebd640 1 -- start start 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b35ebd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b30074320 0x7f4b30199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b35ebd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 0x7f4b3019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b35ebd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b301097b0 0x7f4b301a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b35ebd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010f600 con 0x7f4b30075900 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b35ebd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010f480 con 0x7f4b30074320 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b35ebd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3010f780 con 0x7f4b301097b0 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 0x7f4b3019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b30074320 0x7f4b30199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b30074320 0x7f4b30199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50672/0 (socket says 172.21.15.105:50672) 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 0x7f4b3019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51456/0 (socket says 172.21.15.105:51456) 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1828182903 learned_addr learned my addr 172.21.15.105:0/1828182903 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.424+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b301097b0 0x7f4b301a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1828182903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b301097b0 msgr2=0x7f4b301a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b301097b0 0x7f4b301a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 msgr2=0x7f4b3019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 0x7f4b3019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1828182903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b301a1970 con 0x7f4b30074320 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2effd640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 0x7f4b3019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b30074320 0x7f4b30199d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b20005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2cff9640 1 -- 172.21.15.105:0/1828182903 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20043020 con 0x7f4b30074320 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b35ebd640 1 -- 172.21.15.105:0/1828182903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b301a1c00 con 0x7f4b30074320 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2cff9640 1 -- 172.21.15.105:0/1828182903 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b200043d0 con 0x7f4b30074320 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b2cff9640 1 -- 172.21.15.105:0/1828182903 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20003b80 con 0x7f4b30074320 2024-03-26T17:40:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.425+0000 7f4b35ebd640 1 -- 172.21.15.105:0/1828182903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b301a2150 con 0x7f4b30074320 2024-03-26T17:40:04.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b2cff9640 1 -- 172.21.15.105:0/1828182903 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4b20046020 con 0x7f4b30074320 2024-03-26T17:40:04.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b38b83740 1 Processor -- start 2024-03-26T17:40:04.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b38b83740 1 -- start start 2024-03-26T17:40:04.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccdbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdc420 0x558bccce0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccce3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b38b83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558bccc40970 con 0x558bcccdc420 2024-03-26T17:40:04.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b38b83740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558bccbe8d70 con 0x558bccce1590 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b38b83740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558bccc3c920 con 0x558bcccdba30 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdc420 0x558bccce0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b2ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccdbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b2effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccce3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdc420 0x558bccce0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51458/0 (socket says 172.21.15.105:51458) 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1655807873 learned_addr learned my addr 172.21.15.105:0/1655807873 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.428+0000 7f4b2cff9640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b180a2f20 0x7f4b180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b180a2f20 0x7f4b180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2cff9640 1 -- 172.21.15.105:0/1828182903 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f4b200f1660 con 0x7f4b30074320 2024-03-26T17:40:04.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1655807873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 msgr2=0x558bcccdbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccdbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1655807873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 msgr2=0x558bccce3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccce3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1655807873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bcc9543f0 con 0x558bcccdc420 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2effd640 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccce3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccdbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdc420 0x558bccce0f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4b200bdc80 tx=0x7f4b2002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e5b22a009466f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b15ffb640 1 -- 172.21.15.105:0/1655807873 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b200bdac0 con 0x558bcccdc420 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b180a2f20 0x7f4b180a53e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x558bcccdcc80 tx=0x7f4b28004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b15ffb640 1 -- 172.21.15.105:0/1655807873 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b200be3f0 con 0x558bcccdc420 2024-03-26T17:40:04.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.429+0000 7f4b15ffb640 1 -- 172.21.15.105:0/1655807873 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b200bd650 con 0x558bcccdc420 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 -- 172.21.15.105:0/1655807873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdc420 msgr2=0x558bccce0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdc420 0x558bccce0f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4b200bdc80 tx=0x7f4b2002bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 -- 172.21.15.105:0/1655807873 shutdown_connections 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccce3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdc420 0x558bccce0f20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1655807873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccdbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 -- 172.21.15.105:0/1655807873 >> 172.21.15.105:0/1655807873 conn(0x558bcccd6fa0 msgr2=0x558bcccd73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 -- 172.21.15.105:0/1655807873 shutdown_connections 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 -- 172.21.15.105:0/1655807873 wait complete. 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.430+0000 7f4b38b83740 1 -- start start 2024-03-26T17:40:04.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b38b83740 1 Processor -- start 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdba30 0x558bccce5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdc420 0x558bccce6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccced170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b38b83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558bccc40970 con 0x558bcccdba30 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b38b83740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558bccbfc310 con 0x558bccce1590 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b38b83740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558bccce4300 con 0x558bcccdc420 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b38b83740 10 client.0 _ll_register_callbacks cb 0x558bccc37c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdba30 0x558bccce5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdba30 0x558bccce5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51470/0 (socket says 172.21.15.105:51470) 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1752605111 learned_addr learned my addr 172.21.15.105:0/1752605111 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccced170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2effd640 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdc420 0x558bccce6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1752605111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdc420 msgr2=0x558bccce6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdc420 0x558bccce6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1752605111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 msgr2=0x558bccced170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccced170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1752605111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bcca1e7d0 con 0x558bcccdba30 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccced170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdba30 0x558bccce5bf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4b200c61d0 tx=0x7f4b20003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c76c4ed011b98f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.431+0000 7f4b0effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b0effd640 1 -- 172.21.15.105:0/1752605111 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2009c070 con 0x558bcccdba30 2024-03-26T17:40:04.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b0effd640 1 -- 172.21.15.105:0/1752605111 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f4b200c6270 con 0x558bcccdba30 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 1 -- 172.21.15.105:0/1752605111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b200be3f0 con 0x558bcccdba30 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b0effd640 1 -- 172.21.15.105:0/1752605111 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b200bd830 con 0x558bcccdba30 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 10 client.15969 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 1 -- 172.21.15.105:0/1752605111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bcccf04e0 con 0x558bcccdba30 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b0effd640 1 -- 172.21.15.105:0/1752605111 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b200be3f0 con 0x558bcccdba30 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b0effd640 1 -- 172.21.15.105:0/1752605111 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4b2009cb60 con 0x558bcccdba30 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 10 client.15969 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 10 client.15969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 20 client.15969 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558bcccf19d0 0x558bcccf3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 4 client.15969 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.432+0000 7f4b38b83740 1 -- 172.21.15.105:0/1752605111 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bccc4deb0 con 0x558bcccf19d0 2024-03-26T17:40:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.433+0000 7f4b2effd640 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558bcccf19d0 0x558bcccf3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.433+0000 7f4b2effd640 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558bcccf19d0 0x558bcccf3e00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c6d49106bfab9bd1 server_cookie=ef83f4e5427a02a9 in_seq=0 out_seq=0 2024-03-26T17:40:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.433+0000 7f4b0effd640 10 client.15969 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:40:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.433+0000 7f4b0effd640 1 -- 172.21.15.105:0/1752605111 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558bccc4deb0 con 0x558bcccf19d0 2024-03-26T17:40:04.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.433+0000 7f4b0effd640 10 client.15969 handle_command_reply: tid=0 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.433+0000 7f4b38b83740 1 client.15969 shutdown 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 -- 172.21.15.105:0/1752605111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdba30 msgr2=0x558bccce5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdba30 0x558bccce5bf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4b200c61d0 tx=0x7f4b20003f50 comp rx=0 tx=0).stop 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 -- 172.21.15.105:0/1752605111 shutdown_connections 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558bcccf19d0 0x558bcccf3e00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bccce1590 0x558bccced170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdc420 0x558bccce6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b0effd640 0 client.15969 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1752605111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bcccdba30 0x558bccce5bf0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 -- 172.21.15.105:0/1752605111 >> 172.21.15.105:0/1752605111 conn(0x558bcccd6fa0 msgr2=0x558bcccd61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 -- 172.21.15.105:0/1752605111 shutdown_connections 2024-03-26T17:40:04.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 1 -- 172.21.15.105:0/1752605111 wait complete. 2024-03-26T17:40:04.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.434+0000 7f4b38b83740 20 client.15969 trim_cache size 0 max 16384 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b38b83740 1 Processor -- start 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b38b83740 1 -- start start 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bccce6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccced130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccd6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b38b83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ab020 con 0x558bccce1590 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b38b83740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301acb60 con 0x558bcccdc420 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b38b83740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ab6f0 con 0x558bcccdba30 2024-03-26T17:40:04.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccd6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bccce6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccd6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51476/0 (socket says 172.21.15.105:51476) 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.456+0000 7f4b2ffff640 1 -- 172.21.15.105:0/143036006 learned_addr learned my addr 172.21.15.105:0/143036006 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2effd640 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccced130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2ffff640 1 -- 172.21.15.105:0/143036006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 msgr2=0x558bccce6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bccce6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2ffff640 1 -- 172.21.15.105:0/143036006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 msgr2=0x558bccced130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccced130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2ffff640 1 -- 172.21.15.105:0/143036006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bcccd2820 con 0x558bccce1590 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2effd640 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccced130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bccce6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccd6040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800fcf0 tx=0x7f4b28005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df756807ac38581 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b157fa640 1 -- 172.21.15.105:0/143036006 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280045d0 con 0x558bccce1590 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b157fa640 1 -- 172.21.15.105:0/143036006 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b28026070 con 0x558bccce1590 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.457+0000 7f4b157fa640 1 -- 172.21.15.105:0/143036006 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28022450 con 0x558bccce1590 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 -- 172.21.15.105:0/143036006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 msgr2=0x558bcccd6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccd6040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800fcf0 tx=0x7f4b28005bd0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 -- 172.21.15.105:0/143036006 shutdown_connections 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccd6040 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccced130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 --2- 172.21.15.105:0/143036006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bccce6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 -- 172.21.15.105:0/143036006 >> 172.21.15.105:0/143036006 conn(0x558bcccd6fa0 msgr2=0x558bcccd5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 -- 172.21.15.105:0/143036006 shutdown_connections 2024-03-26T17:40:04.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 -- 172.21.15.105:0/143036006 wait complete. 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.458+0000 7f4b38b83740 1 -- start start 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b38b83740 1 Processor -- start 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccceac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b38b83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccebc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b38b83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ac490 con 0x558bccce1590 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b38b83740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301abdc0 con 0x558bcccdc420 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b38b83740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301a9000 con 0x558bcccdba30 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b38b83740 10 client.0 _ll_register_callbacks cb 0x7f4b24009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccceac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccceac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50702/0 (socket says 172.21.15.105:50702) 2024-03-26T17:40:04.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2effd640 1 -- 172.21.15.105:0/1657642315 learned_addr learned my addr 172.21.15.105:0/1657642315 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1657642315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 msgr2=0x558bccceac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccceac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1657642315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 msgr2=0x558bcccebc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccebc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2f7fe640 1 -- 172.21.15.105:0/1657642315 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bcccec220 con 0x558bcccdba30 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2effd640 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccceac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.459+0000 7f4b2ffff640 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccebc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b2f7fe640 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccea6e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4b200bbfa0 tx=0x7f4b200b9de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a729fef0cfa4756 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b0effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b0effd640 1 -- 172.21.15.105:0/1657642315 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2009c040 con 0x558bcccdba30 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b0effd640 1 -- 172.21.15.105:0/1657642315 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f4b200b9a40 con 0x558bcccdba30 2024-03-26T17:40:04.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 1 -- 172.21.15.105:0/1657642315 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b28026070 con 0x558bcccdba30 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b0effd640 1 -- 172.21.15.105:0/1657642315 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b200bd650 con 0x558bcccdba30 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 10 client.25825 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 1 -- 172.21.15.105:0/1657642315 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bcccec220 con 0x558bcccdba30 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b0effd640 1 -- 172.21.15.105:0/1657642315 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b28026070 con 0x558bcccdba30 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b0effd640 1 -- 172.21.15.105:0/1657642315 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4b200f1100 con 0x558bcccdba30 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 10 client.25825 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 10 client.25825 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 20 client.25825 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558bccd580c0 0x558bccd5a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 4 client.25825 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b38b83740 1 -- 172.21.15.105:0/1657642315 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bccc48200 con 0x558bccd580c0 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.460+0000 7f4b2effd640 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558bccd580c0 0x558bccd5a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b0effd640 10 client.25825 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b2effd640 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558bccd580c0 0x558bccd5a4e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a23e24e7b2dbd507 server_cookie=d86028f540c4e5e7 in_seq=0 out_seq=0 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b0effd640 1 -- 172.21.15.105:0/1657642315 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bccc48200 con 0x558bccd580c0 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b0effd640 10 client.25825 handle_command_reply: tid=0 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 client.25825 shutdown 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 -- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 msgr2=0x558bcccea6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccea6e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4b200bbfa0 tx=0x7f4b200b9de0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 -- 172.21.15.105:0/1657642315 shutdown_connections 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558bccd580c0 0x558bccd5a4e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558bccce1590 0x558bcccebc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558bcccdc420 0x558bccceac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b0effd640 0 client.25825 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 --2- 172.21.15.105:0/1657642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bcccdba30 0x558bcccea6e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 -- 172.21.15.105:0/1657642315 >> 172.21.15.105:0/1657642315 conn(0x558bcccd6fa0 msgr2=0x558bcccdaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 -- 172.21.15.105:0/1657642315 shutdown_connections 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 1 -- 172.21.15.105:0/1657642315 wait complete. 2024-03-26T17:40:04.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.461+0000 7f4b38b83740 20 client.25825 trim_cache size 0 max 16384 2024-03-26T17:40:04.463 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 -- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b180a2f20 msgr2=0x7f4b180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b180a2f20 0x7f4b180a53e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x558bcccdcc80 tx=0x7f4b28004ab0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 -- 172.21.15.105:0/1828182903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b30074320 msgr2=0x7f4b30199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b30074320 0x7f4b30199d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b20005e40 comp rx=0 tx=0).stop 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 -- 172.21.15.105:0/1828182903 shutdown_connections 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b180a2f20 0x7f4b180a53e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b301097b0 0x7f4b301a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b30075900 0x7f4b3019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 --2- 172.21.15.105:0/1828182903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b30074320 0x7f4b30199d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 -- 172.21.15.105:0/1828182903 >> 172.21.15.105:0/1828182903 conn(0x7f4b300fce60 msgr2=0x7f4b301056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 -- 172.21.15.105:0/1828182903 shutdown_connections 2024-03-26T17:40:04.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.462+0000 7f4b35ebd640 1 -- 172.21.15.105:0/1828182903 wait complete. 2024-03-26T17:40:04.466 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:40:04.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 -- 172.21.15.105:0/2040056443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 msgr2=0x7f0190107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 --2- 172.21.15.105:0/2040056443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 0x7f0190107060 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0180009900 tx=0x7f018002bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:04.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 -- 172.21.15.105:0/2040056443 shutdown_connections 2024-03-26T17:40:04.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 --2- 172.21.15.105:0/2040056443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01901075a0 0x7f01901099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 --2- 172.21.15.105:0/2040056443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 0x7f0190107060 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 --2- 172.21.15.105:0/2040056443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0190102340 0x7f0190104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 -- 172.21.15.105:0/2040056443 >> 172.21.15.105:0/2040056443 conn(0x7f01900fbd50 msgr2=0x7f01900fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 -- 172.21.15.105:0/2040056443 shutdown_connections 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 -- 172.21.15.105:0/2040056443 wait complete. 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 Processor -- start 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.598+0000 7f01973f1640 1 -- start start 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01973f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0190102340 0x7f0190197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01973f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 0x7f0190198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01973f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01901075a0 0x7f019019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01973f1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f019010d3e0 con 0x7f0190104c70 2024-03-26T17:40:04.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01973f1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f019010d260 con 0x7f01901075a0 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01973f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f019010d560 con 0x7f0190102340 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 0x7f0190198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 0x7f0190198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36948/0 (socket says 172.21.15.105:36948) 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 -- 172.21.15.105:0/3430411638 learned_addr learned my addr 172.21.15.105:0/3430411638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019518d640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0190102340 0x7f0190197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019598e640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01901075a0 0x7f019019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 -- 172.21.15.105:0/3430411638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0190102340 msgr2=0x7f0190197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0190102340 0x7f0190197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 -- 172.21.15.105:0/3430411638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01901075a0 msgr2=0x7f019019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01901075a0 0x7f019019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 -- 172.21.15.105:0/3430411638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019019f760 con 0x7f0190104c70 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f019498c640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 0x7f0190198080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0180000c00 tx=0x7f0180004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01867fc640 1 -- 172.21.15.105:0/3430411638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180043020 con 0x7f0190104c70 2024-03-26T17:40:04.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01867fc640 1 -- 172.21.15.105:0/3430411638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0180004440 con 0x7f0190104c70 2024-03-26T17:40:04.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01973f1640 1 -- 172.21.15.105:0/3430411638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019019f9f0 con 0x7f0190104c70 2024-03-26T17:40:04.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.599+0000 7f01867fc640 1 -- 172.21.15.105:0/3430411638 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180003c90 con 0x7f0190104c70 2024-03-26T17:40:04.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.600+0000 7f01973f1640 1 -- 172.21.15.105:0/3430411638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019019ff40 con 0x7f0190104c70 2024-03-26T17:40:04.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.602+0000 7f01867fc640 1 -- 172.21.15.105:0/3430411638 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0180046020 con 0x7f0190104c70 2024-03-26T17:40:04.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f0199fd8740 1 Processor -- start 2024-03-26T17:40:04.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f01867fc640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f017c0a2f20 0x7f017c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f0199fd8740 1 -- start start 2024-03-26T17:40:04.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffcac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019598e640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f017c0a2f20 0x7f017c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcb3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f0199fd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0ffc0c970 con 0x55a0ffcabe40 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f0199fd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0ffbb4d70 con 0x55a0ffcac830 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f0199fd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0ffc08920 con 0x55a0ffcb19a0 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f01867fc640 1 -- 172.21.15.105:0/3430411638 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f01800f1600 con 0x7f0190104c70 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019518d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffcac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019598e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcb3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019518d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffcac260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36954/0 (socket says 172.21.15.105:36954) 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019518d640 1 -- 172.21.15.105:0/1003058376 learned_addr learned my addr 172.21.15.105:0/1003058376 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019498c640 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019518d640 1 -- 172.21.15.105:0/1003058376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcb19a0 msgr2=0x55a0ffcb3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019518d640 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcb3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019518d640 1 -- 172.21.15.105:0/1003058376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 msgr2=0x55a0ffcb1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019518d640 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019518d640 1 -- 172.21.15.105:0/1003058376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ff9203f0 con 0x55a0ffcabe40 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.603+0000 7f019598e640 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcb3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.604+0000 7f019598e640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f017c0a2f20 0x7f017c0a53e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0188005fd0 tx=0x7f0188009e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.604+0000 7f019518d640 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffcac260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f018c00bd10 tx=0x7f018c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dbf919a00f5f34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.604+0000 7f017b7fe640 1 -- 172.21.15.105:0/1003058376 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c004030 con 0x55a0ffcabe40 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.604+0000 7f017b7fe640 1 -- 172.21.15.105:0/1003058376 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f018c0035f0 con 0x55a0ffcabe40 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.604+0000 7f017b7fe640 1 -- 172.21.15.105:0/1003058376 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c011640 con 0x55a0ffcabe40 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- 172.21.15.105:0/1003058376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 msgr2=0x55a0ffcac260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffcac260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f018c00bd10 tx=0x7f018c0107a0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- 172.21.15.105:0/1003058376 shutdown_connections 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcb3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 --2- 172.21.15.105:0/1003058376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffcac260 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- 172.21.15.105:0/1003058376 >> 172.21.15.105:0/1003058376 conn(0x55a0ffca73b0 msgr2=0x55a0ffca77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- 172.21.15.105:0/1003058376 shutdown_connections 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- 172.21.15.105:0/1003058376 wait complete. 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- start start 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 Processor -- start 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcb6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcbd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0ffcb4950 con 0x55a0ffcb19a0 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0ffbc8310 con 0x55a0ffcac830 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0ffcb4ad0 con 0x55a0ffcabe40 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.605+0000 7f0199fd8740 10 client.0 _ll_register_callbacks cb 0x55a0ffc03c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019598e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcbd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcb6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcb6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58834/0 (socket says 172.21.15.105:58834) 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 -- 172.21.15.105:0/3960550089 learned_addr learned my addr 172.21.15.105:0/3960550089 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019498c640 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 -- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 msgr2=0x55a0ffcb6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 -- 172.21.15.105:0/3960550089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcb19a0 msgr2=0x55a0ffcbd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcbd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 -- 172.21.15.105:0/3960550089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ff9ea7d0 con 0x55a0ffcabe40 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019498c640 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019598e640 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcbd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f019518d640 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcb6330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f018c010ca0 tx=0x7f018c00be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6950dcced289c52 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f0178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f0178ff9640 1 -- 172.21.15.105:0/3960550089 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c029060 con 0x55a0ffcabe40 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f0178ff9640 1 -- 172.21.15.105:0/3960550089 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f018c003a30 con 0x55a0ffcabe40 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f0199fd8740 1 -- 172.21.15.105:0/3960550089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018c0035f0 con 0x55a0ffcabe40 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.606+0000 7f0178ff9640 1 -- 172.21.15.105:0/3960550089 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f018c004000 con 0x55a0ffcabe40 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0199fd8740 10 client.25837 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0199fd8740 1 -- 172.21.15.105:0/3960550089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0ffcc0c20 con 0x55a0ffcabe40 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0178ff9640 1 -- 172.21.15.105:0/3960550089 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f018c0035f0 con 0x55a0ffcabe40 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0178ff9640 1 -- 172.21.15.105:0/3960550089 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f018c0047e0 con 0x55a0ffcabe40 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0199fd8740 10 client.25837 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0199fd8740 10 client.25837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0199fd8740 20 client.25837 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0ffcc2080 0x55a0ffcc4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0199fd8740 4 client.25837 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f0199fd8740 1 -- 172.21.15.105:0/3960550089 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0ffc19eb0 con 0x55a0ffcc2080 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.607+0000 7f019498c640 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0ffcc2080 0x55a0ffcc4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.608+0000 7f019498c640 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0ffcc2080 0x55a0ffcc4470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a02a6b4e01538d7 server_cookie=780ef248da4f277f in_seq=0 out_seq=0 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.608+0000 7f0178ff9640 10 client.25837 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.608+0000 7f0178ff9640 1 -- 172.21.15.105:0/3960550089 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a0ffc19eb0 con 0x55a0ffcc2080 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.608+0000 7f0178ff9640 10 client.25837 handle_command_reply: tid=0 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 client.25837 shutdown 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 -- 172.21.15.105:0/3960550089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 msgr2=0x55a0ffcb6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcb6330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f018c010ca0 tx=0x7f018c00be70 comp rx=0 tx=0).stop 2024-03-26T17:40:04.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 -- 172.21.15.105:0/3960550089 shutdown_connections 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcbd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0ffcc2080 0x55a0ffcc4470 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcac830 0x55a0ffcb6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3960550089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcb6330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0178ff9640 0 client.25837 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 -- 172.21.15.105:0/3960550089 >> 172.21.15.105:0/3960550089 conn(0x55a0ffca73b0 msgr2=0x55a0ffca21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 -- 172.21.15.105:0/3960550089 shutdown_connections 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 1 -- 172.21.15.105:0/3960550089 wait complete. 2024-03-26T17:40:04.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.609+0000 7f0199fd8740 20 client.25837 trim_cache size 0 max 16384 2024-03-26T17:40:04.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f0199fd8740 1 Processor -- start 2024-03-26T17:40:04.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f0199fd8740 1 -- start start 2024-03-26T17:40:04.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffca1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcac830 0x55a0ffcbd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 0x55a0ffca1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f0199fd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01901a6ba0 con 0x55a0ffcabe40 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f0199fd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f01901a86e0 con 0x55a0ffcb19a0 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f0199fd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01901a7270 con 0x55a0ffcac830 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f019518d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffca1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f019518d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffca1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36974/0 (socket says 172.21.15.105:36974) 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f019518d640 1 -- 172.21.15.105:0/1148432587 learned_addr learned my addr 172.21.15.105:0/1148432587 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f019598e640 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 0x55a0ffca1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.624+0000 7f019498c640 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcac830 0x55a0ffcbd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f019518d640 1 -- 172.21.15.105:0/1148432587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcac830 msgr2=0x55a0ffcbd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f019518d640 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcac830 0x55a0ffcbd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f019518d640 1 -- 172.21.15.105:0/1148432587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 msgr2=0x55a0ffca1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f019518d640 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 0x55a0ffca1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f019518d640 1 -- 172.21.15.105:0/1148432587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ffcb74b0 con 0x55a0ffcabe40 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f019518d640 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffca1930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f018c00b3d0 tx=0x7f018c00bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5964bffc6586d65 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f017affd640 1 -- 172.21.15.105:0/1148432587 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c029070 con 0x55a0ffcabe40 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f017affd640 1 -- 172.21.15.105:0/1148432587 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f018c003e30 con 0x55a0ffcabe40 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.625+0000 7f017affd640 1 -- 172.21.15.105:0/1148432587 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c011680 con 0x55a0ffcabe40 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 -- 172.21.15.105:0/1148432587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 msgr2=0x55a0ffca1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffca1930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f018c00b3d0 tx=0x7f018c00bfa0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 -- 172.21.15.105:0/1148432587 shutdown_connections 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 0x55a0ffca1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcac830 0x55a0ffcbd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 --2- 172.21.15.105:0/1148432587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcabe40 0x55a0ffca1930 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 -- 172.21.15.105:0/1148432587 >> 172.21.15.105:0/1148432587 conn(0x55a0ffca73b0 msgr2=0x55a0ffcadd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 -- 172.21.15.105:0/1148432587 shutdown_connections 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 -- 172.21.15.105:0/1148432587 wait complete. 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 -- start start 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 Processor -- start 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcbd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcac830 0x55a0ffcba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.626+0000 7f0199fd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01901a8010 con 0x55a0ffcac830 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0199fd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f01901a7940 con 0x55a0ffcb19a0 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0199fd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01901a4b80 con 0x55a0ffcabe40 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0199fd8740 10 client.0 _ll_register_callbacks cb 0x7f0180005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcac830 0x55a0ffcba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcac830 0x55a0ffcba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36986/0 (socket says 172.21.15.105:36986) 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 -- 172.21.15.105:0/3733598155 learned_addr learned my addr 172.21.15.105:0/3733598155 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019598e640 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019518d640 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcbd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 -- 172.21.15.105:0/3733598155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 msgr2=0x55a0ffcbd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcbd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 -- 172.21.15.105:0/3733598155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 msgr2=0x55a0ffcba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcba780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 -- 172.21.15.105:0/3733598155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ffd26310 con 0x55a0ffcac830 2024-03-26T17:40:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f019498c640 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcac830 0x55a0ffcba240 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0180006010 tx=0x7f018002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00d63ad3de41f60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:04.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0178ff9640 1 -- 172.21.15.105:0/3733598155 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01800c1e40 con 0x55a0ffcac830 2024-03-26T17:40:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0178ff9640 1 -- 172.21.15.105:0/3733598155 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f0180005590 con 0x55a0ffcac830 2024-03-26T17:40:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0199fd8740 1 -- 172.21.15.105:0/3733598155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018c003e30 con 0x55a0ffcac830 2024-03-26T17:40:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0178ff9640 1 -- 172.21.15.105:0/3733598155 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01800c6700 con 0x55a0ffcac830 2024-03-26T17:40:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0199fd8740 10 client.15993 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.627+0000 7f0199fd8740 1 -- 172.21.15.105:0/3733598155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0ffd27690 con 0x55a0ffcac830 2024-03-26T17:40:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0178ff9640 1 -- 172.21.15.105:0/3733598155 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f018c003e30 con 0x55a0ffcac830 2024-03-26T17:40:04.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0178ff9640 1 -- 172.21.15.105:0/3733598155 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f01800b9a40 con 0x55a0ffcac830 2024-03-26T17:40:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0199fd8740 10 client.15993 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0199fd8740 10 client.15993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:40:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0199fd8740 20 client.15993 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0ffd28040 0x55a0ffd2a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0199fd8740 4 client.15993 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0199fd8740 1 -- 172.21.15.105:0/3733598155 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0ffc14200 con 0x55a0ffd28040 2024-03-26T17:40:04.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f019518d640 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0ffd28040 0x55a0ffd2a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f0178ff9640 10 client.15993 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:40:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.628+0000 7f019518d640 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0ffd28040 0x55a0ffd2a460 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=687d1c6c39b72ea4 server_cookie=15351566cb4c3c2e in_seq=0 out_seq=0 2024-03-26T17:40:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0178ff9640 1 -- 172.21.15.105:0/3733598155 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0ffc14200 con 0x55a0ffd28040 2024-03-26T17:40:04.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0178ff9640 10 client.15993 handle_command_reply: tid=0 2024-03-26T17:40:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 client.15993 shutdown 2024-03-26T17:40:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 -- 172.21.15.105:0/3733598155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcac830 msgr2=0x55a0ffcba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcac830 0x55a0ffcba240 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0180006010 tx=0x7f018002bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 -- 172.21.15.105:0/3733598155 shutdown_connections 2024-03-26T17:40:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a0ffd28040 0x55a0ffd2a460 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0ffcb19a0 0x55a0ffcba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0ffcac830 0x55a0ffcba240 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 --2- 172.21.15.105:0/3733598155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0ffcabe40 0x55a0ffcbd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 -- 172.21.15.105:0/3733598155 >> 172.21.15.105:0/3733598155 conn(0x55a0ffca73b0 msgr2=0x55a0ffcaaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0178ff9640 0 client.15993 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:40:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 -- 172.21.15.105:0/3733598155 shutdown_connections 2024-03-26T17:40:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 1 -- 172.21.15.105:0/3733598155 wait complete. 2024-03-26T17:40:04.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.629+0000 7f0199fd8740 20 client.15993 trim_cache size 0 max 16384 2024-03-26T17:40:04.633 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 -- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f017c0a2f20 msgr2=0x7f017c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f017c0a2f20 0x7f017c0a53e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0188005fd0 tx=0x7f0188009e00 comp rx=0 tx=0).stop 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 -- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 msgr2=0x7f0190198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 0x7f0190198080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0180000c00 tx=0x7f0180004040 comp rx=0 tx=0).stop 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 -- 172.21.15.105:0/3430411638 shutdown_connections 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f017c0a2f20 0x7f017c0a53e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01901075a0 0x7f019019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0190104c70 0x7f0190198080 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 --2- 172.21.15.105:0/3430411638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0190102340 0x7f0190197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 -- 172.21.15.105:0/3430411638 >> 172.21.15.105:0/3430411638 conn(0x7f01900fbd50 msgr2=0x7f01901041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 -- 172.21.15.105:0/3430411638 shutdown_connections 2024-03-26T17:40:04.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.630+0000 7f01973f1640 1 -- 172.21.15.105:0/3430411638 wait complete. 2024-03-26T17:40:04.635 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:40:04.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 -- 172.21.15.105:0/4017536287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec1034a0 msgr2=0x7fa3ec109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 --2- 172.21.15.105:0/4017536287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec1034a0 0x7fa3ec109d30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4009ca0 tx=0x7fa3e402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 -- 172.21.15.105:0/4017536287 shutdown_connections 2024-03-26T17:40:04.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 --2- 172.21.15.105:0/4017536287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec1034a0 0x7fa3ec109d30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 --2- 172.21.15.105:0/4017536287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec102ae0 0x7fa3ec102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 --2- 172.21.15.105:0/4017536287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3ec1018f0 0x7fa3ec101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 -- 172.21.15.105:0/4017536287 >> 172.21.15.105:0/4017536287 conn(0x7fa3ec0fce60 msgr2=0x7fa3ec0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 -- 172.21.15.105:0/4017536287 shutdown_connections 2024-03-26T17:40:04.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.766+0000 7fa3f0812640 1 -- 172.21.15.105:0/4017536287 wait complete. 2024-03-26T17:40:04.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3f0812640 1 Processor -- start 2024-03-26T17:40:04.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3f0812640 1 -- start start 2024-03-26T17:40:04.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3f0812640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3ec1018f0 0x7fa3ec197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3f0812640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec102ae0 0x7fa3ec198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3f0812640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec1034a0 0x7fa3ec19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3f0812640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec10d420 con 0x7fa3ec102ae0 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3f0812640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec10d2a0 con 0x7fa3ec1018f0 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3f0812640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec10d5a0 con 0x7fa3ec1034a0 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec102ae0 0x7fa3ec198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3ea59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3ec1018f0 0x7fa3ec197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec102ae0 0x7fa3ec198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37012/0 (socket says 172.21.15.105:37012) 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3e3fff640 1 -- 172.21.15.105:0/1173495003 learned_addr learned my addr 172.21.15.105:0/1173495003 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.767+0000 7fa3ead9d640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec1034a0 0x7fa3ec19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e3fff640 1 -- 172.21.15.105:0/1173495003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec1034a0 msgr2=0x7fa3ec19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec1034a0 0x7fa3ec19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e3fff640 1 -- 172.21.15.105:0/1173495003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3ec1018f0 msgr2=0x7fa3ec197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3ec1018f0 0x7fa3ec197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e3fff640 1 -- 172.21.15.105:0/1173495003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec19f6e0 con 0x7fa3ec102ae0 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3ec1018f0 0x7fa3ec197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec102ae0 0x7fa3ec198000 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400e9f0 tx=0x7fa3d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e37fe640 1 -- 172.21.15.105:0/1173495003 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400cd80 con 0x7fa3ec102ae0 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e37fe640 1 -- 172.21.15.105:0/1173495003 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3d40054d0 con 0x7fa3ec102ae0 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3e37fe640 1 -- 172.21.15.105:0/1173495003 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4010490 con 0x7fa3ec102ae0 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3f0812640 1 -- 172.21.15.105:0/1173495003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ec19f9d0 con 0x7fa3ec102ae0 2024-03-26T17:40:04.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.768+0000 7fa3f0812640 1 -- 172.21.15.105:0/1173495003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ec19fed0 con 0x7fa3ec102ae0 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3f3404740 1 Processor -- start 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3e37fe640 1 -- 172.21.15.105:0/1173495003 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa3d4005020 con 0x7fa3ec102ae0 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3f3404740 1 -- start start 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6ca30 0x559990a6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a71f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3f3404740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5599909d1970 con 0x559990a6d420 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3f3404740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559990979d70 con 0x559990a72590 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.771+0000 7fa3f3404740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599909cd920 con 0x559990a6ca30 2024-03-26T17:40:04.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a71f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a74a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ead9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6ca30 0x559990a6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a71f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37028/0 (socket says 172.21.15.105:37028) 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 -- 172.21.15.105:0/1278971391 learned_addr learned my addr 172.21.15.105:0/1278971391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3e37fe640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3d80a2f20 0x7fa3d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 -- 172.21.15.105:0/1278971391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6ca30 msgr2=0x559990a6ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/1278971391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6ca30 0x559990a6ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ead9d640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3d80a2f20 0x7fa3d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 -- 172.21.15.105:0/1278971391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 msgr2=0x559990a74a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/1278971391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a74a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 -- 172.21.15.105:0/1278971391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599906e53f0 con 0x559990a6d420 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/1278971391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3e37fe640 1 -- 172.21.15.105:0/1173495003 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa3d40cf790 con 0x7fa3ec102ae0 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/1278971391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a71f20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0114c0 tx=0x7fa3dc011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358e3d41c4773550 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.772+0000 7fa3e0ff9640 1 -- 172.21.15.105:0/1278971391 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc004020 con 0x559990a6d420 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.773+0000 7fa3e0ff9640 1 -- 172.21.15.105:0/1278971391 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3dc0054d0 con 0x559990a6d420 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.773+0000 7fa3e0ff9640 1 -- 172.21.15.105:0/1278971391 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc004ce0 con 0x559990a6d420 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.773+0000 7fa3ead9d640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3d80a2f20 0x7fa3d80a53e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa3e402c630 tx=0x7fa3e402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.773+0000 7fa3f3404740 1 -- 172.21.15.105:0/1278971391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 msgr2=0x559990a71f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.773+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1278971391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a71f20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0114c0 tx=0x7fa3dc011980 comp rx=0 tx=0).stop 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 -- 172.21.15.105:0/1278971391 shutdown_connections 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1278971391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1278971391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a71f20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1278971391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6ca30 0x559990a6ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 -- 172.21.15.105:0/1278971391 >> 172.21.15.105:0/1278971391 conn(0x559990a67fa0 msgr2=0x559990a683a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 -- 172.21.15.105:0/1278971391 shutdown_connections 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 -- 172.21.15.105:0/1278971391 wait complete. 2024-03-26T17:40:04.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 -- start start 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 Processor -- start 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6ca30 0x559990a76f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a7e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6ca30 0x559990a76f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559990a75540 con 0x559990a6ca30 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55999098d310 con 0x559990a72590 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3f3404740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559990a756c0 con 0x559990a6d420 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6ca30 0x559990a76f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37044/0 (socket says 172.21.15.105:37044) 2024-03-26T17:40:04.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.774+0000 7fa3ea59c640 1 -- 172.21.15.105:0/1107384055 learned_addr learned my addr 172.21.15.105:0/1107384055 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3ead9d640 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a7e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3f3404740 10 client.0 _ll_register_callbacks cb 0x5599909c8c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3ea59c640 1 -- 172.21.15.105:0/1107384055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 msgr2=0x559990a774b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a774b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3ea59c640 1 -- 172.21.15.105:0/1107384055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 msgr2=0x559990a7e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a7e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3ea59c640 1 -- 172.21.15.105:0/1107384055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599907af7d0 con 0x559990a6ca30 2024-03-26T17:40:04.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6ca30 0x559990a76f70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc011e80 tx=0x7fa3dc00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696e5f31f4a19a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/1107384055 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc028030 con 0x559990a6ca30 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/1107384055 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa3dc016040 con 0x559990a6ca30 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3f3404740 1 -- 172.21.15.105:0/1107384055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3dc0054d0 con 0x559990a6ca30 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/1107384055 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3dc01d040 con 0x559990a6ca30 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3f3404740 10 client.16005 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.775+0000 7fa3f3404740 1 -- 172.21.15.105:0/1107384055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559990a81840 con 0x559990a6ca30 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/1107384055 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3dc0054d0 con 0x559990a6ca30 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/1107384055 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa3dc0050f0 con 0x559990a6ca30 2024-03-26T17:40:04.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3f3404740 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3f3404740 10 client.16005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3f3404740 20 client.16005 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559990a82c90 0x559990a850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3f3404740 4 client.16005 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3f3404740 1 -- 172.21.15.105:0/1107384055 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599909deeb0 con 0x559990a82c90 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559990a82c90 0x559990a850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3d1ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.776+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559990a82c90 0x559990a850e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f284b70a04209df server_cookie=31254c77caea3866 in_seq=0 out_seq=0 2024-03-26T17:40:04.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/1107384055 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5599909deeb0 con 0x559990a82c90 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3d1ffb640 10 client.16005 handle_command_reply: tid=0 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 client.16005 shutdown 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 -- 172.21.15.105:0/1107384055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6ca30 msgr2=0x559990a76f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6ca30 0x559990a76f70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc011e80 tx=0x7fa3dc00fc90 comp rx=0 tx=0).stop 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 -- 172.21.15.105:0/1107384055 shutdown_connections 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559990a82c90 0x559990a850e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a72590 0x559990a7e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 --2- 172.21.15.105:0/1107384055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6ca30 0x559990a76f70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 -- 172.21.15.105:0/1107384055 >> 172.21.15.105:0/1107384055 conn(0x559990a67fa0 msgr2=0x559990a671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3d1ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:40:04.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 -- 172.21.15.105:0/1107384055 shutdown_connections 2024-03-26T17:40:04.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.777+0000 7fa3f3404740 1 -- 172.21.15.105:0/1107384055 wait complete. 2024-03-26T17:40:04.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.778+0000 7fa3f3404740 20 client.16005 trim_cache size 0 max 16384 2024-03-26T17:40:04.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3f3404740 1 Processor -- start 2024-03-26T17:40:04.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3f3404740 1 -- start start 2024-03-26T17:40:04.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a6b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a7e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a72590 0x559990a899a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3f3404740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1a8de0 con 0x559990a72590 2024-03-26T17:40:04.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3f3404740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1aa920 con 0x559990a6ca30 2024-03-26T17:40:04.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3f3404740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1a94b0 con 0x559990a6d420 2024-03-26T17:40:04.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a72590 0x559990a899a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3ea59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a6b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a72590 0x559990a899a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37060/0 (socket says 172.21.15.105:37060) 2024-03-26T17:40:04.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3ea59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a6b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58136/0 (socket says 172.21.15.105:58136) 2024-03-26T17:40:04.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3ead9d640 1 -- 172.21.15.105:0/2492598770 learned_addr learned my addr 172.21.15.105:0/2492598770 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.792+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a7e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3ead9d640 1 -- 172.21.15.105:0/2492598770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 msgr2=0x559990a7e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3ead9d640 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a7e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3ead9d640 1 -- 172.21.15.105:0/2492598770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 msgr2=0x559990a6b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3ead9d640 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a6b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3ead9d640 1 -- 172.21.15.105:0/2492598770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559990a63820 con 0x559990a72590 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a6b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a7e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3ead9d640 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a72590 0x559990a899a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa3e402c700 tx=0x7fa3e40093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeba0ae07171 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3d3fff640 1 -- 172.21.15.105:0/2492598770 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4039040 con 0x559990a72590 2024-03-26T17:40:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3d3fff640 1 -- 172.21.15.105:0/2492598770 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3e4053070 con 0x559990a72590 2024-03-26T17:40:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.793+0000 7fa3d3fff640 1 -- 172.21.15.105:0/2492598770 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4042830 con 0x559990a72590 2024-03-26T17:40:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 -- 172.21.15.105:0/2492598770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a72590 msgr2=0x559990a899a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a72590 0x559990a899a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa3e402c700 tx=0x7fa3e40093f0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 -- 172.21.15.105:0/2492598770 shutdown_connections 2024-03-26T17:40:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a72590 0x559990a899a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a6d420 0x559990a7e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 --2- 172.21.15.105:0/2492598770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a6b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 -- 172.21.15.105:0/2492598770 >> 172.21.15.105:0/2492598770 conn(0x559990a67fa0 msgr2=0x559990a6b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 -- 172.21.15.105:0/2492598770 shutdown_connections 2024-03-26T17:40:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 -- 172.21.15.105:0/2492598770 wait complete. 2024-03-26T17:40:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.794+0000 7fa3f3404740 1 -- start start 2024-03-26T17:40:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3f3404740 1 Processor -- start 2024-03-26T17:40:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a7b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a7b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3f3404740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a72590 0x559990ae6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3f3404740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1aa250 con 0x559990a6d420 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3f3404740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1a9b80 con 0x559990a6ca30 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3f3404740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1a6dc0 con 0x559990a72590 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3f3404740 10 client.0 _ll_register_callbacks cb 0x7fa3d4098120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a7b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a7b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37074/0 (socket says 172.21.15.105:37074) 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 -- 172.21.15.105:0/424306014 learned_addr learned my addr 172.21.15.105:0/424306014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a7b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3ead9d640 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a72590 0x559990ae6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 -- 172.21.15.105:0/424306014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a72590 msgr2=0x559990ae6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a72590 0x559990ae6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 -- 172.21.15.105:0/424306014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 msgr2=0x559990a7b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a7b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 -- 172.21.15.105:0/424306014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559990a7e700 con 0x559990a6d420 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3e3fff640 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a7b630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40cd5c0 tx=0x7fa3d409b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90194d8399fd11dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.795+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/424306014 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4005680 con 0x559990a6d420 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/424306014 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa3d40046c0 con 0x559990a6d420 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 1 -- 172.21.15.105:0/424306014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e4053070 con 0x559990a6d420 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/424306014 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3d409c580 con 0x559990a6d420 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 10 client.16011 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 1 -- 172.21.15.105:0/424306014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559990ae85b0 con 0x559990a6d420 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/424306014 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3e4053070 con 0x559990a6d420 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/424306014 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa3d409c0e0 con 0x559990a6d420 2024-03-26T17:40:04.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 10 client.16011 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 10 client.16011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 20 client.16011 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559990ae8fb0 0x559990aeb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 4 client.16011 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3f3404740 1 -- 172.21.15.105:0/424306014 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599909d9200 con 0x559990ae8fb0 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.796+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559990ae8fb0 0x559990aeb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3d1ffb640 10 client.16011 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3ea59c640 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559990ae8fb0 0x559990aeb3d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85524a9d25fa3cae server_cookie=bff33bc973ea36d4 in_seq=0 out_seq=0 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3d1ffb640 1 -- 172.21.15.105:0/424306014 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599909d9200 con 0x559990ae8fb0 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3d1ffb640 10 client.16011 handle_command_reply: tid=0 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 client.16011 shutdown 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 -- 172.21.15.105:0/424306014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 msgr2=0x559990a7b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a7b630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40cd5c0 tx=0x7fa3d409b8f0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 -- 172.21.15.105:0/424306014 shutdown_connections 2024-03-26T17:40:04.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559990ae8fb0 0x559990aeb3d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559990a72590 0x559990ae6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559990a6d420 0x559990a7b630 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 --2- 172.21.15.105:0/424306014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559990a6ca30 0x559990a7b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 -- 172.21.15.105:0/424306014 >> 172.21.15.105:0/424306014 conn(0x559990a67fa0 msgr2=0x559990a6ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3d1ffb640 0 client.16011 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 -- 172.21.15.105:0/424306014 shutdown_connections 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 1 -- 172.21.15.105:0/424306014 wait complete. 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.797+0000 7fa3f3404740 20 client.16011 trim_cache size 0 max 16384 2024-03-26T17:40:04.810 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 -- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3d80a2f20 msgr2=0x7fa3d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3d80a2f20 0x7fa3d80a53e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa3e402c630 tx=0x7fa3e402e040 comp rx=0 tx=0).stop 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 -- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec102ae0 msgr2=0x7fa3ec198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec102ae0 0x7fa3ec198000 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400e9f0 tx=0x7fa3d400edf0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 -- 172.21.15.105:0/1173495003 shutdown_connections 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3d80a2f20 0x7fa3d80a53e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3ec1034a0 0x7fa3ec19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3ec102ae0 0x7fa3ec198000 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 --2- 172.21.15.105:0/1173495003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3ec1018f0 0x7fa3ec197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 -- 172.21.15.105:0/1173495003 >> 172.21.15.105:0/1173495003 conn(0x7fa3ec0fce60 msgr2=0x7fa3ec105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 -- 172.21.15.105:0/1173495003 shutdown_connections 2024-03-26T17:40:04.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.798+0000 7fa3f0812640 1 -- 172.21.15.105:0/1173495003 wait complete. 2024-03-26T17:40:04.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 -- 172.21.15.105:0/287109662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 msgr2=0x7fbb9c107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 --2- 172.21.15.105:0/287109662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 0x7fbb9c107060 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009900 tx=0x7fbb9002bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 -- 172.21.15.105:0/287109662 shutdown_connections 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 --2- 172.21.15.105:0/287109662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9c1075a0 0x7fbb9c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 --2- 172.21.15.105:0/287109662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 0x7fbb9c107060 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 --2- 172.21.15.105:0/287109662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c102340 0x7fbb9c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 -- 172.21.15.105:0/287109662 >> 172.21.15.105:0/287109662 conn(0x7fbb9c0fbd50 msgr2=0x7fbb9c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 -- 172.21.15.105:0/287109662 shutdown_connections 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.941+0000 7fbba2885640 1 -- 172.21.15.105:0/287109662 wait complete. 2024-03-26T17:40:04.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba2885640 1 Processor -- start 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba2885640 1 -- start start 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba2885640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c102340 0x7fbb9c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba2885640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 0x7fbb9c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba2885640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9c1075a0 0x7fbb9c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba2885640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10d400 con 0x7fbb9c104c70 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba2885640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10d280 con 0x7fbb9c1075a0 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba2885640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10d580 con 0x7fbb9c102340 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c102340 0x7fbb9c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbb9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 0x7fbb9c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c102340 0x7fbb9c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58912/0 (socket says 172.21.15.105:58912) 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbb9bfff640 1 -- 172.21.15.105:0/3995112421 learned_addr learned my addr 172.21.15.105:0/3995112421 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.942+0000 7fbba0e22640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9c1075a0 0x7fbb9c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/3995112421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c102340 msgr2=0x7fbb9c197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c102340 0x7fbb9c197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/3995112421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9c1075a0 msgr2=0x7fbb9c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9c1075a0 0x7fbb9c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/3995112421 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c19f770 con 0x7fbb9c104c70 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbba0e22640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9c1075a0 0x7fbb9c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 0x7fbb9c1980e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbb90036040 tx=0x7fbb90005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbb997fa640 1 -- 172.21.15.105:0/3995112421 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90043020 con 0x7fbb9c104c70 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbba2885640 1 -- 172.21.15.105:0/3995112421 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c19fa00 con 0x7fbb9c104c70 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbba2885640 1 -- 172.21.15.105:0/3995112421 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c19ff70 con 0x7fbb9c104c70 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.943+0000 7fbb997fa640 1 -- 172.21.15.105:0/3995112421 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb90004350 con 0x7fbb9c104c70 2024-03-26T17:40:04.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.944+0000 7fbb997fa640 1 -- 172.21.15.105:0/3995112421 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90003ab0 con 0x7fbb9c104c70 2024-03-26T17:40:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.945+0000 7fbb997fa640 1 -- 172.21.15.105:0/3995112421 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbb90046020 con 0x7fbb9c104c70 2024-03-26T17:40:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.946+0000 7fbb997fa640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbb840a2f20 0x7fbb840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.946+0000 7fbb997fa640 1 -- 172.21.15.105:0/3995112421 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fbb900f0e60 con 0x7fbb9c104c70 2024-03-26T17:40:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.946+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbb840a2f20 0x7fbb840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.946+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbb840a2f20 0x7fbb840a53e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009cd0 tx=0x7fbb8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.946+0000 7fbba5473740 1 Processor -- start 2024-03-26T17:40:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.946+0000 7fbba5473740 1 -- start start 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbba5473740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b7be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbba5473740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b80f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbba5473740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b83a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbba5473740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55df80ae0970 con 0x55df80b7c420 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbba5473740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55df80a88d70 con 0x55df80b81590 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbba5473740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55df80adc920 con 0x55df80b7ba30 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b80f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b80f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37104/0 (socket says 172.21.15.105:37104) 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b7be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/3117807155 learned_addr learned my addr 172.21.15.105:0/3117807155 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbba0e22640 1 --2- 172.21.15.105:0/3117807155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b83a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/3117807155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 msgr2=0x55df80b7be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/3117807155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b7be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/3117807155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 msgr2=0x55df80b83a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/3117807155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b83a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/3117807155 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df807f43f0 con 0x55df80b7c420 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/3117807155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b80f20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbb900bd250 tx=0x7fbb9002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8646ddb613f28de3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb827fc640 1 -- 172.21.15.105:0/3117807155 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb900bd550 con 0x55df80b7c420 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb827fc640 1 -- 172.21.15.105:0/3117807155 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb900c16a0 con 0x55df80b7c420 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.947+0000 7fbb827fc640 1 -- 172.21.15.105:0/3117807155 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90005420 con 0x55df80b7c420 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 -- 172.21.15.105:0/3117807155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 msgr2=0x55df80b80f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 --2- 172.21.15.105:0/3117807155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b80f20 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbb900bd250 tx=0x7fbb9002bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 -- 172.21.15.105:0/3117807155 shutdown_connections 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 --2- 172.21.15.105:0/3117807155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b83a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 --2- 172.21.15.105:0/3117807155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b80f20 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 --2- 172.21.15.105:0/3117807155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b7be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 -- 172.21.15.105:0/3117807155 >> 172.21.15.105:0/3117807155 conn(0x55df80b76fa0 msgr2=0x55df80b773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 -- 172.21.15.105:0/3117807155 shutdown_connections 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.948+0000 7fbba5473740 1 -- 172.21.15.105:0/3117807155 wait complete. 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 1 -- start start 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 1 Processor -- start 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b89150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b89690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b85c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55df80b84300 con 0x55df80b7c420 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55df80a9c310 con 0x55df80b81590 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55df80b84480 con 0x55df80b7ba30 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba5473740 10 client.0 _ll_register_callbacks cb 0x55df80ad7c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbb9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b89690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b89150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba0e22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b85c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba0e22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b85c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58194/0 (socket says 172.21.15.105:58194) 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba0e22640 1 -- 172.21.15.105:0/3498119728 learned_addr learned my addr 172.21.15.105:0/3498119728 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b89150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58932/0 (socket says 172.21.15.105:58932) 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba0e22640 1 -- 172.21.15.105:0/3498119728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 msgr2=0x55df80b89150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba0e22640 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b89150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba0e22640 1 -- 172.21.15.105:0/3498119728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 msgr2=0x55df80b89690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.949+0000 7fbba0e22640 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b89690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbba0e22640 1 -- 172.21.15.105:0/3498119728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df808be7d0 con 0x55df80b81590 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbba0e22640 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b85c10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400fe40 tx=0x7fbb94012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de359b33a9f8b2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbb7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/3498119728 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94003970 con 0x55df80b81590 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbba5473740 1 -- 172.21.15.105:0/3498119728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb900c16a0 con 0x55df80b81590 2024-03-26T17:40:04.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/3498119728 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fbb94003b10 con 0x55df80b81590 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/3498119728 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb94003d10 con 0x55df80b81590 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbba5473740 10 client.25877 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbba5473740 1 -- 172.21.15.105:0/3498119728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df80b98e90 con 0x55df80b81590 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.950+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/3498119728 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb900c16a0 con 0x55df80b81590 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/3498119728 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbb94010610 con 0x55df80b81590 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbba5473740 10 client.25877 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbba5473740 10 client.25877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbba5473740 20 client.25877 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbba5473740 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55df80b9a500 0x55df80b9c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbba5473740 4 client.25877 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbba5473740 1 -- 172.21.15.105:0/3498119728 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df80aedeb0 con 0x55df80b9a500 2024-03-26T17:40:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55df80b9a500 0x55df80b9c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55df80b9a500 0x55df80b9c930 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848c087df8d541ed server_cookie=86fbf6d27425665a in_seq=0 out_seq=0 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.951+0000 7fbb7b7fe640 10 client.25877 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/3498119728 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55df80aedeb0 con 0x55df80b9a500 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbb7b7fe640 10 client.25877 handle_command_reply: tid=0 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 client.25877 shutdown 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 -- 172.21.15.105:0/3498119728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 msgr2=0x55df80b85c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b85c10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbb9400fe40 tx=0x7fbb94012820 comp rx=0 tx=0).stop 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 -- 172.21.15.105:0/3498119728 shutdown_connections 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55df80b9a500 0x55df80b9c930 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80b85c10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80b89690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbb7b7fe640 0 client.25877 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:04.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 --2- 172.21.15.105:0/3498119728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7ba30 0x55df80b89150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 -- 172.21.15.105:0/3498119728 >> 172.21.15.105:0/3498119728 conn(0x55df80b76fa0 msgr2=0x55df80b761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 -- 172.21.15.105:0/3498119728 shutdown_connections 2024-03-26T17:40:04.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 1 -- 172.21.15.105:0/3498119728 wait complete. 2024-03-26T17:40:04.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.952+0000 7fbba5473740 20 client.25877 trim_cache size 0 max 16384 2024-03-26T17:40:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbba5473740 1 Processor -- start 2024-03-26T17:40:04.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbba5473740 1 -- start start 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbba5473740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b7ba30 0x55df80b7a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbba5473740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80ba11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbba5473740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b81590 0x55df80ba1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbba5473740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1a6b50 con 0x55df80b7c420 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbba5473740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1a8690 con 0x55df80b7ba30 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbba5473740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1a7220 con 0x55df80b81590 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbb9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80ba11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbb9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80ba11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37122/0 (socket says 172.21.15.105:37122) 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/2426271502 learned_addr learned my addr 172.21.15.105:0/2426271502 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.970+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b7ba30 0x55df80b7a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbba0e22640 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b81590 0x55df80ba1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/2426271502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b81590 msgr2=0x55df80ba1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b81590 0x55df80ba1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/2426271502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b7ba30 msgr2=0x55df80b7a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b7ba30 0x55df80b7a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb9b7fe640 1 -- 172.21.15.105:0/2426271502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df80b72820 con 0x55df80b7c420 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80ba11f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbb900b99b0 tx=0x7fbb90042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a99f469d89709f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb81ffb640 1 -- 172.21.15.105:0/2426271502 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb900da070 con 0x55df80b7c420 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb81ffb640 1 -- 172.21.15.105:0/2426271502 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb900c5f60 con 0x55df80b7c420 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.971+0000 7fbb81ffb640 1 -- 172.21.15.105:0/2426271502 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb900b9ce0 con 0x55df80b7c420 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 -- 172.21.15.105:0/2426271502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 msgr2=0x55df80ba11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80ba11f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbb900b99b0 tx=0x7fbb90042000 comp rx=0 tx=0).stop 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 -- 172.21.15.105:0/2426271502 shutdown_connections 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b81590 0x55df80ba1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7c420 0x55df80ba11f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 --2- 172.21.15.105:0/2426271502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b7ba30 0x55df80b7a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 -- 172.21.15.105:0/2426271502 >> 172.21.15.105:0/2426271502 conn(0x55df80b76fa0 msgr2=0x55df80b7a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 -- 172.21.15.105:0/2426271502 shutdown_connections 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 -- 172.21.15.105:0/2426271502 wait complete. 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.972+0000 7fbba5473740 1 -- start start 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba5473740 1 Processor -- start 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba5473740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7ba30 0x55df80bab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba5473740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7c420 0x55df80babe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba5473740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80bace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba5473740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1a7fc0 con 0x55df80b7ba30 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba5473740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1a78f0 con 0x55df80b81590 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba5473740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1a4b30 con 0x55df80b7c420 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba5473740 10 client.0 _ll_register_callbacks cb 0x7fbb9400f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7ba30 0x55df80bab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7ba30 0x55df80bab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37128/0 (socket says 172.21.15.105:37128) 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9bfff640 1 -- 172.21.15.105:0/956240082 learned_addr learned my addr 172.21.15.105:0/956240082 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7c420 0x55df80babe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbba0e22640 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80bace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9bfff640 1 -- 172.21.15.105:0/956240082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7c420 msgr2=0x55df80babe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7c420 0x55df80babe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9bfff640 1 -- 172.21.15.105:0/956240082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 msgr2=0x55df80bace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80bace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.973+0000 7fbb9bfff640 1 -- 172.21.15.105:0/956240082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df80bad420 con 0x55df80b7ba30 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbb9bfff640 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7ba30 0x55df80bab8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x55df80b9d1a0 tx=0x7fbb8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95d99cb7ac462a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbb7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/956240082 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c01be60 con 0x55df80b7ba30 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/956240082 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fbb8c00f810 con 0x55df80b7ba30 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 1 -- 172.21.15.105:0/956240082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb900c5f60 con 0x55df80b7ba30 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/956240082 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb8c00fa10 con 0x55df80b7ba30 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 10 client.16029 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:04.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 1 -- 172.21.15.105:0/956240082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df80c00a80 con 0x55df80b7ba30 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/956240082 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb900c5f60 con 0x55df80b7ba30 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/956240082 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbb8c02a490 con 0x55df80b7ba30 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 10 client.16029 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 20 client.16029 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55df80c01460 0x55df80c03880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 4 client.16029 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.974+0000 7fbba5473740 1 -- 172.21.15.105:0/956240082 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df80ae8200 con 0x55df80c01460 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.975+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55df80c01460 0x55df80c03880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.975+0000 7fbb9b7fe640 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55df80c01460 0x55df80c03880 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb84f32b1d172b0 server_cookie=ce64af032eac8410 in_seq=0 out_seq=0 2024-03-26T17:40:04.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.975+0000 7fbb7b7fe640 10 client.16029 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.975+0000 7fbb7b7fe640 1 -- 172.21.15.105:0/956240082 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df80ae8200 con 0x55df80c01460 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.975+0000 7fbb7b7fe640 10 client.16029 handle_command_reply: tid=0 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.975+0000 7fbba5473740 1 client.16029 shutdown 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.975+0000 7fbba5473740 1 -- 172.21.15.105:0/956240082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7ba30 msgr2=0x55df80bab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.975+0000 7fbba5473740 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7ba30 0x55df80bab8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x55df80b9d1a0 tx=0x7fbb8c004730 comp rx=0 tx=0).stop 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 1 -- 172.21.15.105:0/956240082 shutdown_connections 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55df80c01460 0x55df80c03880 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55df80b81590 0x55df80bace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df80b7c420 0x55df80babe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 1 --2- 172.21.15.105:0/956240082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55df80b7ba30 0x55df80bab8e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 1 -- 172.21.15.105:0/956240082 >> 172.21.15.105:0/956240082 conn(0x55df80b76fa0 msgr2=0x55df80b7aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbb7b7fe640 0 client.16029 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 1 -- 172.21.15.105:0/956240082 shutdown_connections 2024-03-26T17:40:04.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 1 -- 172.21.15.105:0/956240082 wait complete. 2024-03-26T17:40:04.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba5473740 20 client.16029 trim_cache size 0 max 16384 2024-03-26T17:40:04.975 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 -- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbb840a2f20 msgr2=0x7fbb840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbb840a2f20 0x7fbb840a53e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009cd0 tx=0x7fbb8c005f90 comp rx=0 tx=0).stop 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 -- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 msgr2=0x7fbb9c1980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 0x7fbb9c1980e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbb90036040 tx=0x7fbb90005dc0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 -- 172.21.15.105:0/3995112421 shutdown_connections 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbb840a2f20 0x7fbb840a53e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbb9c1075a0 0x7fbb9c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbb9c104c70 0x7fbb9c1980e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 --2- 172.21.15.105:0/3995112421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb9c102340 0x7fbb9c197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:04.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.976+0000 7fbba2885640 1 -- 172.21.15.105:0/3995112421 >> 172.21.15.105:0/3995112421 conn(0x7fbb9c0fbd50 msgr2=0x7fbb9c1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.977+0000 7fbba2885640 1 -- 172.21.15.105:0/3995112421 shutdown_connections 2024-03-26T17:40:04.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:04.977+0000 7fbba2885640 1 -- 172.21.15.105:0/3995112421 wait complete. 2024-03-26T17:40:04.980 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:04 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2079009552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:40:04.999 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:04 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2079009552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:40:05.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 28785 3.62 MB/sec warmup 48 sec latency 17.124 ms 2024-03-26T17:40:05.066 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:04 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2079009552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:40:05.982 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:40:05.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:40:05.983 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:40:06.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:05 smithi105 ceph-mon[32280]: pgmap v399: 97 pgs: 97 active+clean; 30 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:40:06.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 29241 3.56 MB/sec warmup 49 sec latency 25.191 ms 2024-03-26T17:40:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.103+0000 7f8679ac0640 1 -- 172.21.15.105:0/1679926108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741019b0 msgr2=0x7f8674101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.103+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1679926108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741019b0 0x7f8674101e30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8668009900 tx=0x7f866802bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.103+0000 7f8679ac0640 1 -- 172.21.15.105:0/1679926108 shutdown_connections 2024-03-26T17:40:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.103+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1679926108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8674102370 0x7f8674108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.103+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1679926108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741019b0 0x7f8674101e30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.103+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1679926108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86741007c0 0x7f8674100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.103+0000 7f8679ac0640 1 -- 172.21.15.105:0/1679926108 >> 172.21.15.105:0/1679926108 conn(0x7f86740fbd50 msgr2=0x7f86740fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.104+0000 7f8679ac0640 1 -- 172.21.15.105:0/1679926108 shutdown_connections 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.104+0000 7f8679ac0640 1 -- 172.21.15.105:0/1679926108 wait complete. 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.104+0000 7f8679ac0640 1 Processor -- start 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.104+0000 7f8679ac0640 1 -- start start 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.104+0000 7f8679ac0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741007c0 0x7f8674197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.104+0000 7f8679ac0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741019b0 0x7f86741980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.104+0000 7f8679ac0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8674102370 0x7f867419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8679ac0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f867410c2f0 con 0x7f86741007c0 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8679ac0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f867410c170 con 0x7f8674102370 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8679ac0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f867410c470 con 0x7f86741019b0 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741007c0 0x7f8674197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741007c0 0x7f8674197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37156/0 (socket says 172.21.15.105:37156) 2024-03-26T17:40:06.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 -- 172.21.15.105:0/1262672171 learned_addr learned my addr 172.21.15.105:0/1262672171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8672ffd640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741019b0 0x7f86741980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8673fff640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8674102370 0x7f867419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 -- 172.21.15.105:0/1262672171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741019b0 msgr2=0x7f86741980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741019b0 0x7f86741980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 -- 172.21.15.105:0/1262672171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8674102370 msgr2=0x7f867419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8674102370 0x7f867419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 -- 172.21.15.105:0/1262672171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867419f750 con 0x7f86741007c0 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8672ffd640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741019b0 0x7f86741980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f86737fe640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741007c0 0x7f8674197b80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f866400e9c0 tx=0x7f866400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8670ff9640 1 -- 172.21.15.105:0/1262672171 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866400cd80 con 0x7f86741007c0 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8670ff9640 1 -- 172.21.15.105:0/1262672171 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86640035f0 con 0x7f86741007c0 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8679ac0640 1 -- 172.21.15.105:0/1262672171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867419fa40 con 0x7f86741007c0 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8670ff9640 1 -- 172.21.15.105:0/1262672171 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664010690 con 0x7f86741007c0 2024-03-26T17:40:06.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.105+0000 7f8679ac0640 1 -- 172.21.15.105:0/1262672171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867419ffb0 con 0x7f86741007c0 2024-03-26T17:40:06.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:05 smithi144 ceph-mon[36095]: pgmap v399: 97 pgs: 97 active+clean; 30 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:40:06.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f8670ff9640 1 -- 172.21.15.105:0/1262672171 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f86640037a0 con 0x7f86741007c0 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f867c787740 1 Processor -- start 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f867c787740 1 -- start start 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f867c787740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd919c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f867c787740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd91f90 0x55a0acd96a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f867c787740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd97100 0x55a0acd99590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f867c787740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0accf2630 con 0x55a0acd91f90 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f867c787740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0acc9ae10 con 0x55a0acd97100 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f867c787740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0accee5e0 con 0x55a0acd915a0 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f8673fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd919c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f86737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd91f90 0x55a0acd96a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f8673fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd919c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58980/0 (socket says 172.21.15.105:58980) 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f8673fff640 1 -- 172.21.15.105:0/297668022 learned_addr learned my addr 172.21.15.105:0/297668022 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.108+0000 7f8672ffd640 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd97100 0x55a0acd99590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8670ff9640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f865c0a2fa0 0x7f865c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8673fff640 1 -- 172.21.15.105:0/297668022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd97100 msgr2=0x55a0acd99590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8673fff640 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd97100 0x55a0acd99590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8673fff640 1 -- 172.21.15.105:0/297668022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd91f90 msgr2=0x55a0acd96a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8673fff640 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd91f90 0x55a0acd96a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8673fff640 1 -- 172.21.15.105:0/297668022 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0accb1740 con 0x55a0acd915a0 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f86737fe640 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd91f90 0x55a0acd96a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8670ff9640 1 -- 172.21.15.105:0/1262672171 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f86640cf720 con 0x7f86741007c0 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8673fff640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f865c0a2fa0 0x7f865c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8673fff640 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd919c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f866c009fd0 tx=0x7f866c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e48af2fcaa852f0d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8659ffb640 1 -- 172.21.15.105:0/297668022 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c004050 con 0x55a0acd915a0 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8659ffb640 1 -- 172.21.15.105:0/297668022 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f866c003e70 con 0x55a0acd915a0 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.109+0000 7f8659ffb640 1 -- 172.21.15.105:0/297668022 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c004050 con 0x55a0acd915a0 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.110+0000 7f867c787740 1 -- 172.21.15.105:0/297668022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 msgr2=0x55a0acd919c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.110+0000 7f867c787740 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd919c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f866c009fd0 tx=0x7f866c0118d0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.110+0000 7f8673fff640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f865c0a2fa0 0x7f865c0a5460 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f866c01d000 tx=0x7f866c00fea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.110+0000 7f867c787740 1 -- 172.21.15.105:0/297668022 shutdown_connections 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.110+0000 7f867c787740 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd97100 0x55a0acd99590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.110+0000 7f867c787740 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd91f90 0x55a0acd96a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.110+0000 7f867c787740 1 --2- 172.21.15.105:0/297668022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd919c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.110+0000 7f867c787740 1 -- 172.21.15.105:0/297668022 >> 172.21.15.105:0/297668022 conn(0x55a0acd88700 msgr2=0x55a0acd88b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 -- 172.21.15.105:0/297668022 shutdown_connections 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 -- 172.21.15.105:0/297668022 wait complete. 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 -- start start 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 Processor -- start 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd9b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acd9bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acda2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0acd9a0b0 con 0x55a0acd97100 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0accae1a0 con 0x55a0acd91f90 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0acd9a230 con 0x55a0acd915a0 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.111+0000 7f867c787740 10 client.0 _ll_register_callbacks cb 0x55a0aca05c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd9b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f8673fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acda2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd9b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58986/0 (socket says 172.21.15.105:58986) 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 -- 172.21.15.105:0/293853735 learned_addr learned my addr 172.21.15.105:0/293853735 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f8672ffd640 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acd9bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 -- 172.21.15.105:0/293853735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 msgr2=0x55a0acd9bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acd9bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 -- 172.21.15.105:0/293853735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 msgr2=0x55a0acda2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acda2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 -- 172.21.15.105:0/293853735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0acace7e0 con 0x55a0acd915a0 2024-03-26T17:40:06.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f86737fe640 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd9b9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f866400c6a0 tx=0x7f8664009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d4ca594110aa7a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f8652ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f8652ffd640 1 -- 172.21.15.105:0/293853735 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86640980b0 con 0x55a0acd915a0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f8652ffd640 1 -- 172.21.15.105:0/293853735 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f866409c270 con 0x55a0acd915a0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f867c787740 1 -- 172.21.15.105:0/293853735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f866c003e70 con 0x55a0acd915a0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f8652ffd640 1 -- 172.21.15.105:0/293853735 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f866409b770 con 0x55a0acd915a0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f867c787740 10 client.25891 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.112+0000 7f867c787740 1 -- 172.21.15.105:0/293853735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0acb2ef70 con 0x55a0acd915a0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f8652ffd640 1 -- 172.21.15.105:0/293853735 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f866c003e70 con 0x55a0acd915a0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f8652ffd640 1 -- 172.21.15.105:0/293853735 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f86640cfbd0 con 0x55a0acd915a0 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f867c787740 10 client.25891 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f867c787740 10 client.25891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f867c787740 20 client.25891 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f867c787740 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0acda78b0 0x55a0acda9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f867c787740 4 client.25891 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:06.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f867c787740 1 -- 172.21.15.105:0/293853735 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0accffb70 con 0x55a0acda78b0 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f8672ffd640 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0acda78b0 0x55a0acda9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f8652ffd640 10 client.25891 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.113+0000 7f8672ffd640 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0acda78b0 0x55a0acda9d40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1d4074e270e056 server_cookie=139d6bb5b99f9a62 in_seq=0 out_seq=0 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.114+0000 7f8652ffd640 1 -- 172.21.15.105:0/293853735 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a0accffb70 con 0x55a0acda78b0 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f8652ffd640 10 client.25891 handle_command_reply: tid=0 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 client.25891 shutdown 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 -- 172.21.15.105:0/293853735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 msgr2=0x55a0acd9b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd9b9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f866400c6a0 tx=0x7f8664009700 comp rx=0 tx=0).stop 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 -- 172.21.15.105:0/293853735 shutdown_connections 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0acda78b0 0x55a0acda9d40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acda2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acd9bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f8652ffd640 0 client.25891 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:06.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 --2- 172.21.15.105:0/293853735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd9b9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 -- 172.21.15.105:0/293853735 >> 172.21.15.105:0/293853735 conn(0x55a0acd88700 msgr2=0x55a0acd87900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 -- 172.21.15.105:0/293853735 shutdown_connections 2024-03-26T17:40:06.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 1 -- 172.21.15.105:0/293853735 wait complete. 2024-03-26T17:40:06.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.115+0000 7f867c787740 20 client.25891 trim_cache size 0 max 16384 2024-03-26T17:40:06.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f867c787740 1 Processor -- start 2024-03-26T17:40:06.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f867c787740 1 -- start start 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f867c787740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd87070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f867c787740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acd875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f867c787740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acd87af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f867c787740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86741a6c10 con 0x55a0acd97100 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f867c787740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86741a8750 con 0x55a0acd91f90 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f867c787740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86741a72e0 con 0x55a0acd915a0 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f8673fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acd87af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f86737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd87070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f8672ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acd875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f8673fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acd87af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37178/0 (socket says 172.21.15.105:37178) 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f8673fff640 1 -- 172.21.15.105:0/1243944905 learned_addr learned my addr 172.21.15.105:0/1243944905 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.135+0000 7f86737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd87070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59000/0 (socket says 172.21.15.105:59000) 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86737fe640 1 -- 172.21.15.105:0/1243944905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 msgr2=0x55a0acd875b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86737fe640 1 --2- 172.21.15.105:0/1243944905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acd875b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86737fe640 1 -- 172.21.15.105:0/1243944905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 msgr2=0x55a0acd87af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86737fe640 1 --2- 172.21.15.105:0/1243944905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acd87af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86737fe640 1 -- 172.21.15.105:0/1243944905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ac9c6170 con 0x55a0acd915a0 2024-03-26T17:40:06.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f8673fff640 1 --2- 172.21.15.105:0/1243944905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acd87af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:06.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86737fe640 1 --2- 172.21.15.105:0/1243944905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd87070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f866409a0c0 tx=0x7f866409a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a475986db700ce09 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86597fa640 1 -- 172.21.15.105:0/1243944905 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866409b950 con 0x55a0acd915a0 2024-03-26T17:40:06.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86597fa640 1 -- 172.21.15.105:0/1243944905 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f866407e070 con 0x55a0acd915a0 2024-03-26T17:40:06.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.136+0000 7f86597fa640 1 -- 172.21.15.105:0/1243944905 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866409bbe0 con 0x55a0acd915a0 2024-03-26T17:40:06.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 -- 172.21.15.105:0/1243944905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 msgr2=0x55a0acd87070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 --2- 172.21.15.105:0/1243944905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd87070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f866409a0c0 tx=0x7f866409a320 comp rx=0 tx=0).stop 2024-03-26T17:40:06.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 -- 172.21.15.105:0/1243944905 shutdown_connections 2024-03-26T17:40:06.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 --2- 172.21.15.105:0/1243944905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acd87af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 --2- 172.21.15.105:0/1243944905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acd875b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 --2- 172.21.15.105:0/1243944905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acd87070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 -- 172.21.15.105:0/1243944905 >> 172.21.15.105:0/1243944905 conn(0x55a0acd88700 msgr2=0x55a0acd9ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 -- 172.21.15.105:0/1243944905 shutdown_connections 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 -- 172.21.15.105:0/1243944905 wait complete. 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 -- start start 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.137+0000 7f867c787740 1 Processor -- start 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f867c787740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acda08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f867c787740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acda0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f867c787740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acda1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f867c787740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86741a8080 con 0x55a0acd97100 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f867c787740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86741a79b0 con 0x55a0acd91f90 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f867c787740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86741a4bf0 con 0x55a0acd915a0 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f867c787740 10 client.0 _ll_register_callbacks cb 0x7f8664003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:06.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acda08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f8672ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acda0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acda08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59006/0 (socket says 172.21.15.105:59006) 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 -- 172.21.15.105:0/2022975051 learned_addr learned my addr 172.21.15.105:0/2022975051 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f8673fff640 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acda1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 -- 172.21.15.105:0/2022975051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 msgr2=0x55a0acda0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acda0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 -- 172.21.15.105:0/2022975051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 msgr2=0x55a0acda1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acda1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 -- 172.21.15.105:0/2022975051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0acda23a0 con 0x55a0acd915a0 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f86737fe640 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acda08b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f86640058c0 tx=0x7f86640a0060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd3607bbd7974636 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f8652ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.138+0000 7f8652ffd640 1 -- 172.21.15.105:0/2022975051 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866409b950 con 0x55a0acd915a0 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 1 -- 172.21.15.105:0/2022975051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f866407e070 con 0x55a0acd915a0 2024-03-26T17:40:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f8652ffd640 1 -- 172.21.15.105:0/2022975051 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f86640a43e0 con 0x55a0acd915a0 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f8652ffd640 1 -- 172.21.15.105:0/2022975051 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86640980b0 con 0x55a0acd915a0 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 10 client.25897 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 1 -- 172.21.15.105:0/2022975051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0ace0e1a0 con 0x55a0acd915a0 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f8652ffd640 1 -- 172.21.15.105:0/2022975051 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f866407e070 con 0x55a0acd915a0 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f8652ffd640 1 -- 172.21.15.105:0/2022975051 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f866409c0e0 con 0x55a0acd915a0 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 10 client.25897 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 10 client.25897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 20 client.25897 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0ace0e8c0 0x55a0ace10ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 4 client.25897 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f867c787740 1 -- 172.21.15.105:0/2022975051 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a0accf9ec0 con 0x55a0ace0e8c0 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.139+0000 7f8672ffd640 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0ace0e8c0 0x55a0ace10ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.140+0000 7f8652ffd640 10 client.25897 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.140+0000 7f8672ffd640 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0ace0e8c0 0x55a0ace10ce0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9cce8d29061d01f server_cookie=6316a20539751254 in_seq=0 out_seq=0 2024-03-26T17:40:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f8652ffd640 1 -- 172.21.15.105:0/2022975051 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a0accf9ec0 con 0x55a0ace0e8c0 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f8652ffd640 10 client.25897 handle_command_reply: tid=0 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 client.25897 shutdown 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 -- 172.21.15.105:0/2022975051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 msgr2=0x55a0acda08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acda08b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f86640058c0 tx=0x7f86640a0060 comp rx=0 tx=0).stop 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 -- 172.21.15.105:0/2022975051 shutdown_connections 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0ace0e8c0 0x55a0ace10ce0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0acd97100 0x55a0acda1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0acd91f90 0x55a0acda0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f8652ffd640 0 client.25897 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 --2- 172.21.15.105:0/2022975051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0acd915a0 0x55a0acda08b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 -- 172.21.15.105:0/2022975051 >> 172.21.15.105:0/2022975051 conn(0x55a0acd88700 msgr2=0x55a0acd8b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 -- 172.21.15.105:0/2022975051 shutdown_connections 2024-03-26T17:40:06.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 1 -- 172.21.15.105:0/2022975051 wait complete. 2024-03-26T17:40:06.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.141+0000 7f867c787740 20 client.25897 trim_cache size 0 max 16384 2024-03-26T17:40:06.142 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:40:06.142 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:40:06.142 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "32b4378b-67c1-488b-ad99-6bc591cf9b95", 2024-03-26T17:40:06.142 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:40:06.142 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 -- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f865c0a2fa0 msgr2=0x7f865c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f865c0a2fa0 0x7f865c0a5460 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f866c01d000 tx=0x7f866c00fea0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 -- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741007c0 msgr2=0x7f8674197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741007c0 0x7f8674197b80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f866400e9c0 tx=0x7f866400ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 -- 172.21.15.105:0/1262672171 shutdown_connections 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f865c0a2fa0 0x7f865c0a5460 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8674102370 0x7f867419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86741019b0 0x7f86741980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 --2- 172.21.15.105:0/1262672171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86741007c0 0x7f8674197b80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 -- 172.21.15.105:0/1262672171 >> 172.21.15.105:0/1262672171 conn(0x7f86740fbd50 msgr2=0x7f8674068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 -- 172.21.15.105:0/1262672171 shutdown_connections 2024-03-26T17:40:06.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.142+0000 7f8679ac0640 1 -- 172.21.15.105:0/1262672171 wait complete. 2024-03-26T17:40:06.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:40:06.148 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:40:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:05 smithi135 ceph-mon[35636]: pgmap v399: 97 pgs: 97 active+clean; 30 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:40:06.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 -- 172.21.15.105:0/2386307782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e70102ac0 msgr2=0x7f9e70102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 --2- 172.21.15.105:0/2386307782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e70102ac0 0x7f9e70102f40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009900 tx=0x7f9e6402bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:06.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 -- 172.21.15.105:0/2386307782 shutdown_connections 2024-03-26T17:40:06.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 --2- 172.21.15.105:0/2386307782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70103480 0x7f9e70109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 --2- 172.21.15.105:0/2386307782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e70102ac0 0x7f9e70102f40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 --2- 172.21.15.105:0/2386307782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e701018d0 0x7f9e70101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 -- 172.21.15.105:0/2386307782 >> 172.21.15.105:0/2386307782 conn(0x7f9e700fce60 msgr2=0x7f9e700ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 -- 172.21.15.105:0/2386307782 shutdown_connections 2024-03-26T17:40:06.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.317+0000 7f9e74e61640 1 -- 172.21.15.105:0/2386307782 wait complete. 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.318+0000 7f9e74e61640 1 Processor -- start 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.318+0000 7f9e74e61640 1 -- start start 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.318+0000 7f9e74e61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e701018d0 0x7f9e70197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.318+0000 7f9e74e61640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e70102ac0 0x7f9e70198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.318+0000 7f9e74e61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70103480 0x7f9e7019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.318+0000 7f9e74e61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010d400 con 0x7f9e701018d0 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.318+0000 7f9e74e61640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010d280 con 0x7f9e70102ac0 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.318+0000 7f9e74e61640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7010d580 con 0x7f9e70103480 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e701018d0 0x7f9e70197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e70102ac0 0x7f9e70198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e70102ac0 0x7f9e70198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58306/0 (socket says 172.21.15.105:58306) 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/4168251114 learned_addr learned my addr 172.21.15.105:0/4168251114 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70103480 0x7f9e7019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/4168251114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70103480 msgr2=0x7f9e7019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70103480 0x7f9e7019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e701018d0 msgr2=0x7f9e70197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e701018d0 0x7f9e70197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/4168251114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7019f730 con 0x7f9e70102ac0 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6e59c640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e701018d0 0x7f9e70197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e70102ac0 0x7f9e70198050 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006010 tx=0x7f9e64005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e5f7fe640 1 -- 172.21.15.105:0/4168251114 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64043020 con 0x7f9e70102ac0 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e74e61640 1 -- 172.21.15.105:0/4168251114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7019f9c0 con 0x7f9e70102ac0 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e5f7fe640 1 -- 172.21.15.105:0/4168251114 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e640043d0 con 0x7f9e70102ac0 2024-03-26T17:40:06.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e5f7fe640 1 -- 172.21.15.105:0/4168251114 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64003b80 con 0x7f9e70102ac0 2024-03-26T17:40:06.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.319+0000 7f9e74e61640 1 -- 172.21.15.105:0/4168251114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7019ff60 con 0x7f9e70102ac0 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.322+0000 7f9e5f7fe640 1 -- 172.21.15.105:0/4168251114 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9e64046020 con 0x7f9e70102ac0 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e77a51740 1 Processor -- start 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e77a51740 1 -- start start 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aa6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96aab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aab860 0x558b96aadcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e5f7fe640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e580a2f20 0x7f9e580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e77a51740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558b96a06980 con 0x558b96aa5d00 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e77a51740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558b969aebe0 con 0x558b96aab860 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e77a51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558b96a02930 con 0x558b96aa66f0 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aa6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96aab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aa6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37226/0 (socket says 172.21.15.105:37226) 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e6ed9d640 1 -- 172.21.15.105:0/113634276 learned_addr learned my addr 172.21.15.105:0/113634276 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aab860 0x558b96aadcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e580a2f20 0x7f9e580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.323+0000 7f9e5f7fe640 1 -- 172.21.15.105:0/4168251114 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f9e640f1600 con 0x7f9e70102ac0 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e6ed9d640 1 -- 172.21.15.105:0/113634276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 msgr2=0x558b96aab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96aab1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e6ed9d640 1 -- 172.21.15.105:0/113634276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aab860 msgr2=0x558b96aadcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aab860 0x558b96aadcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e6ed9d640 1 -- 172.21.15.105:0/113634276 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b967e47d0 con 0x558b96aa5d00 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e6e59c640 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96aab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aa6120 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9e68011e20 tx=0x7f9e6800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a716e8ad0643a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e580a2f20 0x7f9e580a53e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800fc90 tx=0x7f9e6801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e5cff9640 1 -- 172.21.15.105:0/113634276 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6801dcb0 con 0x558b96aa5d00 2024-03-26T17:40:06.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e5cff9640 1 -- 172.21.15.105:0/113634276 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e6801de50 con 0x558b96aa5d00 2024-03-26T17:40:06.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.324+0000 7f9e5cff9640 1 -- 172.21.15.105:0/113634276 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68004280 con 0x558b96aa5d00 2024-03-26T17:40:06.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 -- 172.21.15.105:0/113634276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 msgr2=0x558b96aa6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aa6120 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9e68011e20 tx=0x7f9e6800f790 comp rx=0 tx=0).stop 2024-03-26T17:40:06.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 -- 172.21.15.105:0/113634276 shutdown_connections 2024-03-26T17:40:06.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aab860 0x558b96aadcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96aab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 --2- 172.21.15.105:0/113634276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aa6120 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 -- 172.21.15.105:0/113634276 >> 172.21.15.105:0/113634276 conn(0x558b96a9ce60 msgr2=0x558b96a9d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 -- 172.21.15.105:0/113634276 shutdown_connections 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 -- 172.21.15.105:0/113634276 wait complete. 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.325+0000 7f9e77a51740 1 -- start start 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e77a51740 1 Processor -- start 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aaffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa66f0 0x558b96ab04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aab860 0x558b96ab7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e77a51740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558b96aae750 con 0x558b96aa5d00 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e77a51740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558b969c2160 con 0x558b96aa66f0 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e77a51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558b96aae8d0 con 0x558b96aab860 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e77a51740 10 client.0 _ll_register_callbacks cb 0x558b969fe720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aaffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aab860 0x558b96ab7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa66f0 0x558b96ab04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aaffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37230/0 (socket says 172.21.15.105:37230) 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aab860 0x558b96ab7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59038/0 (socket says 172.21.15.105:59038) 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6e59c640 1 -- 172.21.15.105:0/2091163776 learned_addr learned my addr 172.21.15.105:0/2091163776 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/2091163776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aab860 msgr2=0x558b96ab7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aab860 0x558b96ab7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/2091163776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 msgr2=0x558b96aaffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aaffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/2091163776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9671a3f0 con 0x558b96aa66f0 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6e59c640 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aaffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa66f0 0x558b96ab04e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9e640ef430 tx=0x7f9e640454b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdbccb52f4a10ece server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.326+0000 7f9e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e55ffb640 1 -- 172.21.15.105:0/2091163776 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640bd830 con 0x558b96aa66f0 2024-03-26T17:40:06.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 1 -- 172.21.15.105:0/2091163776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6801de50 con 0x558b96aa66f0 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e55ffb640 1 -- 172.21.15.105:0/2091163776 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f9e64003d20 con 0x558b96aa66f0 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e55ffb640 1 -- 172.21.15.105:0/2091163776 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e640d0960 con 0x558b96aa66f0 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 10 client.25913 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 1 -- 172.21.15.105:0/2091163776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b966da0f0 con 0x558b96aa66f0 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e55ffb640 1 -- 172.21.15.105:0/2091163776 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e6801de50 con 0x558b96aa66f0 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e55ffb640 1 -- 172.21.15.105:0/2091163776 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9e640054b0 con 0x558b96aa66f0 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 10 client.25913 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 10 client.25913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 20 client.25913 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558b96abbcb0 0x558b96abe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 4 client.25913 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e77a51740 1 -- 172.21.15.105:0/2091163776 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b96a13ec0 con 0x558b96abbcb0 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.327+0000 7f9e6e59c640 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558b96abbcb0 0x558b96abe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.328+0000 7f9e6e59c640 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558b96abbcb0 0x558b96abe130 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e437601a532b1 server_cookie=a1480c067cfd8001 in_seq=0 out_seq=0 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.328+0000 7f9e55ffb640 10 client.25913 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e55ffb640 1 -- 172.21.15.105:0/2091163776 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558b96a13ec0 con 0x558b96abbcb0 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e55ffb640 10 client.25913 handle_command_reply: tid=0 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 client.25913 shutdown 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 -- 172.21.15.105:0/2091163776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa66f0 msgr2=0x558b96ab04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa66f0 0x558b96ab04e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9e640ef430 tx=0x7f9e640454b0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 -- 172.21.15.105:0/2091163776 shutdown_connections 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558b96abbcb0 0x558b96abe130 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aab860 0x558b96ab7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa66f0 0x558b96ab04e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2091163776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa5d00 0x558b96aaffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e55ffb640 0 client.25913 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 -- 172.21.15.105:0/2091163776 >> 172.21.15.105:0/2091163776 conn(0x558b96a9ce60 msgr2=0x558b96a9c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 -- 172.21.15.105:0/2091163776 shutdown_connections 2024-03-26T17:40:06.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 1 -- 172.21.15.105:0/2091163776 wait complete. 2024-03-26T17:40:06.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.329+0000 7f9e77a51740 20 client.25913 trim_cache size 0 max 16384 2024-03-26T17:40:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.347+0000 7f9e77a51740 1 Processor -- start 2024-03-26T17:40:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.347+0000 7f9e77a51740 1 -- start start 2024-03-26T17:40:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.347+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96a9ce60 0x558b96a9b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.347+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96a9bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e77a51740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701a6be0 con 0x558b96a9ce60 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e77a51740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701a8720 con 0x558b96aa5d00 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e77a51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701a72b0 con 0x558b96aa66f0 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96a9ce60 0x558b96a9b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96a9ce60 0x558b96a9b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37240/0 (socket says 172.21.15.105:37240) 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6e59c640 1 -- 172.21.15.105:0/2991407058 learned_addr learned my addr 172.21.15.105:0/2991407058 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96a9bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/2991407058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 msgr2=0x558b96a9bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96a9bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/2991407058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96a9ce60 msgr2=0x558b96a9b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96a9ce60 0x558b96a9b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6dd9b640 1 -- 172.21.15.105:0/2991407058 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b96abe7b0 con 0x558b96aa5d00 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6e59c640 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96a9ce60 0x558b96a9b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab74e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e64005ab0 tx=0x7f9e640e9e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84192a8090085db0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e57fff640 1 -- 172.21.15.105:0/2991407058 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640bd650 con 0x558b96aa5d00 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e57fff640 1 -- 172.21.15.105:0/2991407058 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e640d20b0 con 0x558b96aa5d00 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.348+0000 7f9e57fff640 1 -- 172.21.15.105:0/2991407058 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640054b0 con 0x558b96aa5d00 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 -- 172.21.15.105:0/2991407058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 msgr2=0x558b96ab74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab74e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e64005ab0 tx=0x7f9e640e9e30 comp rx=0 tx=0).stop 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 -- 172.21.15.105:0/2991407058 shutdown_connections 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96aa66f0 0x558b96a9bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab74e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 --2- 172.21.15.105:0/2991407058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96a9ce60 0x558b96a9b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 -- 172.21.15.105:0/2991407058 >> 172.21.15.105:0/2991407058 conn(0x558b96aab3a0 msgr2=0x558b96aaffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 -- 172.21.15.105:0/2991407058 shutdown_connections 2024-03-26T17:40:06.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 -- 172.21.15.105:0/2991407058 wait complete. 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 -- start start 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.349+0000 7f9e77a51740 1 Processor -- start 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96a9ce60 0x558b96ab4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e77a51740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa66f0 0x558b96ab6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e77a51740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701a8050 con 0x558b96aa66f0 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e77a51740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701a7980 con 0x558b96aa5d00 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e77a51740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701a4bc0 con 0x558b96a9ce60 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e77a51740 10 client.0 _ll_register_callbacks cb 0x7f9e64009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:06.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa66f0 0x558b96ab6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96a9ce60 0x558b96ab4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa66f0 0x558b96ab6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37242/0 (socket says 172.21.15.105:37242) 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 -- 172.21.15.105:0/1261322793 learned_addr learned my addr 172.21.15.105:0/1261322793 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 -- 172.21.15.105:0/1261322793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96a9ce60 msgr2=0x558b96ab4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96a9ce60 0x558b96ab4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 -- 172.21.15.105:0/1261322793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 msgr2=0x558b96ab50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 -- 172.21.15.105:0/1261322793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b96ab65f0 con 0x558b96aa66f0 2024-03-26T17:40:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6dd9b640 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e6ed9d640 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa66f0 0x558b96ab6000 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x558b96aa6f50 tx=0x7f9e68009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0701690dc1d3aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e55ffb640 1 -- 172.21.15.105:0/1261322793 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6802a040 con 0x558b96aa66f0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.350+0000 7f9e55ffb640 1 -- 172.21.15.105:0/1261322793 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f9e68011030 con 0x558b96aa66f0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 1 -- 172.21.15.105:0/1261322793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e640d20b0 con 0x558b96aa66f0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e55ffb640 1 -- 172.21.15.105:0/1261322793 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e68041050 con 0x558b96aa66f0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 10 client.16065 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 1 -- 172.21.15.105:0/1261322793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b96b22070 con 0x558b96aa66f0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e55ffb640 1 -- 172.21.15.105:0/1261322793 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e640d20b0 con 0x558b96aa66f0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e55ffb640 1 -- 172.21.15.105:0/1261322793 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9e68005350 con 0x558b96aa66f0 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 10 client.16065 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 20 client.16065 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558b96aab860 0x558b96b24850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 4 client.16065 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e77a51740 1 -- 172.21.15.105:0/1261322793 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b96a0e210 con 0x558b96aab860 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.351+0000 7f9e6e59c640 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558b96aab860 0x558b96b24850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e6e59c640 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558b96aab860 0x558b96b24850 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b01f1e64d905ff server_cookie=15210a0803af55a3 in_seq=0 out_seq=0 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e55ffb640 10 client.16065 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e55ffb640 1 -- 172.21.15.105:0/1261322793 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558b96a0e210 con 0x558b96aab860 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e55ffb640 10 client.16065 handle_command_reply: tid=0 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 client.16065 shutdown 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 -- 172.21.15.105:0/1261322793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa66f0 msgr2=0x558b96ab6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa66f0 0x558b96ab6000 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x558b96aa6f50 tx=0x7f9e68009f90 comp rx=0 tx=0).stop 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 -- 172.21.15.105:0/1261322793 shutdown_connections 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558b96aab860 0x558b96b24850 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558b96aa66f0 0x558b96ab6000 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b96aa5d00 0x558b96ab50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e55ffb640 0 client.16065 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 --2- 172.21.15.105:0/1261322793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558b96a9ce60 0x558b96ab4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 -- 172.21.15.105:0/1261322793 >> 172.21.15.105:0/1261322793 conn(0x558b96aab3a0 msgr2=0x558b96aa00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 -- 172.21.15.105:0/1261322793 shutdown_connections 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 1 -- 172.21.15.105:0/1261322793 wait complete. 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.352+0000 7f9e77a51740 20 client.16065 trim_cache size 0 max 16384 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:40:06.353 INFO:teuthology.orchestra.run.smithi105.stdout: "32b4378b-67c1-488b-ad99-6bc591cf9b95": { 2024-03-26T17:40:06.354 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:40:06.354 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "32b4378b-67c1-488b-ad99-6bc591cf9b95", 2024-03-26T17:40:06.354 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:40:06.354 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:40:06.354 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:40:06.354 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 -- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e580a2f20 msgr2=0x7f9e580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e580a2f20 0x7f9e580a53e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800fc90 tx=0x7f9e6801e000 comp rx=0 tx=0).stop 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 -- 172.21.15.105:0/4168251114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e70102ac0 msgr2=0x7f9e70198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e70102ac0 0x7f9e70198050 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e64006010 tx=0x7f9e64005e40 comp rx=0 tx=0).stop 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 -- 172.21.15.105:0/4168251114 shutdown_connections 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e580a2f20 0x7f9e580a53e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e70103480 0x7f9e7019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e70102ac0 0x7f9e70198050 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 --2- 172.21.15.105:0/4168251114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e701018d0 0x7f9e70197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 -- 172.21.15.105:0/4168251114 >> 172.21.15.105:0/4168251114 conn(0x7f9e700fce60 msgr2=0x7f9e70105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 -- 172.21.15.105:0/4168251114 shutdown_connections 2024-03-26T17:40:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:06.353+0000 7f9e74e61640 1 -- 172.21.15.105:0/4168251114 wait complete. 2024-03-26T17:40:06.357 INFO:tasks.cephfs.filesystem:scrub status for tag:32b4378b-67c1-488b-ad99-6bc591cf9b95 - {'path': '/', 'tag': '32b4378b-67c1-488b-ad99-6bc591cf9b95', 'options': 'recursive,force'} 2024-03-26T17:40:07.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 29852 3.58 MB/sec warmup 50 sec latency 19.457 ms 2024-03-26T17:40:08.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 30514 3.52 MB/sec warmup 51 sec latency 15.075 ms 2024-03-26T17:40:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:07 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:40:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:07 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:40:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:07 smithi144 ceph-mon[36095]: pgmap v400: 97 pgs: 97 active+clean; 27 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:40:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:07 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:40:08.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:07 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:40:08.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:08.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:07 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:40:08.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:08.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:07 smithi105 ceph-mon[32280]: pgmap v400: 97 pgs: 97 active+clean; 27 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:40:08.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:07 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:40:08.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:07 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:40:08.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:08.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:07 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:40:08.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:08.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:07 smithi135 ceph-mon[35636]: pgmap v400: 97 pgs: 97 active+clean; 27 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:40:08.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:07 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:40:09.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 31175 3.56 MB/sec warmup 52 sec latency 17.236 ms 2024-03-26T17:40:10.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 31751 3.55 MB/sec warmup 53 sec latency 53.282 ms 2024-03-26T17:40:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:09 smithi144 ceph-mon[36095]: pgmap v401: 97 pgs: 97 active+clean; 28 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:40:10.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:09 smithi105 ceph-mon[32280]: pgmap v401: 97 pgs: 97 active+clean; 28 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:40:10.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:09 smithi135 ceph-mon[35636]: pgmap v401: 97 pgs: 97 active+clean; 28 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:40:11.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 32338 3.58 MB/sec warmup 54 sec latency 16.303 ms 2024-03-26T17:40:12.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 32789 3.52 MB/sec warmup 55 sec latency 22.780 ms 2024-03-26T17:40:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:11 smithi144 ceph-mon[36095]: pgmap v402: 97 pgs: 97 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:40:12.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:11 smithi105 ceph-mon[32280]: pgmap v402: 97 pgs: 97 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:40:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:11 smithi135 ceph-mon[35636]: pgmap v402: 97 pgs: 97 active+clean; 28 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:40:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 -- 172.21.15.105:0/1027660612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a541023b0 msgr2=0x7f5a54108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 --2- 172.21.15.105:0/1027660612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a541023b0 0x7f5a54108c40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009ca0 tx=0x7f5a4c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 -- 172.21.15.105:0/1027660612 shutdown_connections 2024-03-26T17:40:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 --2- 172.21.15.105:0/1027660612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a541023b0 0x7f5a54108c40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 --2- 172.21.15.105:0/1027660612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a541019f0 0x7f5a54101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 --2- 172.21.15.105:0/1027660612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a54100800 0x7f5a54100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 -- 172.21.15.105:0/1027660612 >> 172.21.15.105:0/1027660612 conn(0x7f5a540fbd50 msgr2=0x7f5a540fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 -- 172.21.15.105:0/1027660612 shutdown_connections 2024-03-26T17:40:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 -- 172.21.15.105:0/1027660612 wait complete. 2024-03-26T17:40:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.880+0000 7f5a59e19640 1 Processor -- start 2024-03-26T17:40:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a59e19640 1 -- start start 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a59e19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a54100800 0x7f5a54197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a59e19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a541019f0 0x7f5a54198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a59e19640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a541023b0 0x7f5a5419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a59e19640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5410c330 con 0x7f5a54100800 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a59e19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5410c1b0 con 0x7f5a541019f0 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a59e19640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5410c4b0 con 0x7f5a541023b0 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a54100800 0x7f5a54197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a54100800 0x7f5a54197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37254/0 (socket says 172.21.15.105:37254) 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a537fe640 1 -- 172.21.15.105:0/2848776714 learned_addr learned my addr 172.21.15.105:0/2848776714 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a52ffd640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a541019f0 0x7f5a54198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a53fff640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a541023b0 0x7f5a5419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a537fe640 1 -- 172.21.15.105:0/2848776714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a541023b0 msgr2=0x7f5a5419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a537fe640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a541023b0 0x7f5a5419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a537fe640 1 -- 172.21.15.105:0/2848776714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a541019f0 msgr2=0x7f5a54198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a537fe640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a541019f0 0x7f5a54198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.881+0000 7f5a537fe640 1 -- 172.21.15.105:0/2848776714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5419f6a0 con 0x7f5a54100800 2024-03-26T17:40:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.882+0000 7f5a537fe640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a54100800 0x7f5a54197ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400e9c0 tx=0x7f5a4400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.882+0000 7f5a50ff9640 1 -- 172.21.15.105:0/2848776714 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4400cd80 con 0x7f5a54100800 2024-03-26T17:40:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.882+0000 7f5a50ff9640 1 -- 172.21.15.105:0/2848776714 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a440035f0 con 0x7f5a54100800 2024-03-26T17:40:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.882+0000 7f5a50ff9640 1 -- 172.21.15.105:0/2848776714 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44010690 con 0x7f5a54100800 2024-03-26T17:40:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.882+0000 7f5a59e19640 1 -- 172.21.15.105:0/2848776714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5419f990 con 0x7f5a54100800 2024-03-26T17:40:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.882+0000 7f5a59e19640 1 -- 172.21.15.105:0/2848776714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5419ff30 con 0x7f5a54100800 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.885+0000 7f5a50ff9640 1 -- 172.21.15.105:0/2848776714 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5a440037a0 con 0x7f5a54100800 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a5ca06740 1 Processor -- start 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a5ca06740 1 -- start start 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a50ff9640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a3c0a3040 0x7f5a3c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962b5d0 0x55b4a962b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 0x55b4a9630ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a96335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b4a95900b0 con 0x55b4a9631130 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b4a9538960 con 0x55b4a962b5d0 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b4a958c060 con 0x55b4a962bfc0 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a96335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a96335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37270/0 (socket says 172.21.15.105:37270) 2024-03-26T17:40:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a53fff640 1 -- 172.21.15.105:0/552036132 learned_addr learned my addr 172.21.15.105:0/552036132 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a537fe640 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962b5d0 0x55b4a962b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a52ffd640 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 0x55b4a9630ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a53fff640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a3c0a3040 0x7f5a3c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a53fff640 1 -- 172.21.15.105:0/552036132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 msgr2=0x55b4a9630ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a53fff640 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 0x55b4a9630ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.886+0000 7f5a53fff640 1 -- 172.21.15.105:0/552036132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962b5d0 msgr2=0x55b4a962b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.887+0000 7f5a50ff9640 1 -- 172.21.15.105:0/2848776714 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f5a440cf800 con 0x7f5a54100800 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.887+0000 7f5a53fff640 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962b5d0 0x55b4a962b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.887+0000 7f5a53fff640 1 -- 172.21.15.105:0/552036132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a954ac60 con 0x55b4a9631130 2024-03-26T17:40:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.887+0000 7f5a537fe640 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962b5d0 0x55b4a962b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:40:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.887+0000 7f5a53fff640 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a96335c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009430 tx=0x7f5a4c032890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bfa3e41c45c009 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.887+0000 7f5a53fff640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a3c0a3040 0x7f5a3c0a5500 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55b4a9633cb0 tx=0x7f5a4c047000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.887+0000 7f5a39ffb640 1 -- 172.21.15.105:0/552036132 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c039030 con 0x55b4a9631130 2024-03-26T17:40:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.887+0000 7f5a39ffb640 1 -- 172.21.15.105:0/552036132 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a4c048030 con 0x55b4a9631130 2024-03-26T17:40:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.888+0000 7f5a39ffb640 1 -- 172.21.15.105:0/552036132 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c04d420 con 0x55b4a9631130 2024-03-26T17:40:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.888+0000 7f5a5ca06740 1 -- 172.21.15.105:0/552036132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 msgr2=0x55b4a96335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.888+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a96335c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009430 tx=0x7f5a4c032890 comp rx=0 tx=0).stop 2024-03-26T17:40:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.889+0000 7f5a5ca06740 1 -- 172.21.15.105:0/552036132 shutdown_connections 2024-03-26T17:40:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.889+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a96335c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.889+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 0x55b4a9630ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.889+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/552036132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962b5d0 0x55b4a962b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.882 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.889+0000 7f5a5ca06740 1 -- 172.21.15.105:0/552036132 >> 172.21.15.105:0/552036132 conn(0x55b4a9626b40 msgr2=0x55b4a9626f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.889+0000 7f5a5ca06740 1 -- 172.21.15.105:0/552036132 shutdown_connections 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.890+0000 7f5a5ca06740 1 -- 172.21.15.105:0/552036132 wait complete. 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.890+0000 7f5a5ca06740 1 -- start start 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.890+0000 7f5a5ca06740 1 Processor -- start 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a96397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962bfc0 0x55b4a9639d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a9640d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b4a96384f0 con 0x55b4a9631130 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b4a954bdb0 con 0x55b4a962bfc0 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b4a9638670 con 0x55b4a962b5d0 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a5ca06740 10 client.0 _ll_register_callbacks cb 0x55b4a9550110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a9640d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a52ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962bfc0 0x55b4a9639d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a9640d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37286/0 (socket says 172.21.15.105:37286) 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a96397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 -- 172.21.15.105:0/3867051468 learned_addr learned my addr 172.21.15.105:0/3867051468 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 -- 172.21.15.105:0/3867051468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 msgr2=0x55b4a96397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a96397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 -- 172.21.15.105:0/3867051468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962bfc0 msgr2=0x55b4a9639d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962bfc0 0x55b4a9639d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 -- 172.21.15.105:0/3867051468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a92a4210 con 0x55b4a9631130 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a537fe640 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a96397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a53fff640 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a9640d60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009bf0 tx=0x7f5a4c034840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ebc1df265be004 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.891+0000 7f5a32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a32ffd640 1 -- 172.21.15.105:0/3867051468 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c052070 con 0x55b4a9631130 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a32ffd640 1 -- 172.21.15.105:0/3867051468 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f5a4c034d80 con 0x55b4a9631130 2024-03-26T17:40:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 1 -- 172.21.15.105:0/3867051468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a4c048030 con 0x55b4a9631130 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a32ffd640 1 -- 172.21.15.105:0/3867051468 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a4c039040 con 0x55b4a9631130 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 10 client.16077 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 1 -- 172.21.15.105:0/3867051468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4a9370620 con 0x55b4a9631130 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a32ffd640 1 -- 172.21.15.105:0/3867051468 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a4c048030 con 0x55b4a9631130 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a32ffd640 1 -- 172.21.15.105:0/3867051468 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5a4c04da10 con 0x55b4a9631130 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 10 client.16077 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 20 client.16077 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b4a9645250 0x55b4a9647660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 4 client.16077 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.892+0000 7f5a5ca06740 1 -- 172.21.15.105:0/3867051468 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4a959d5f0 con 0x55b4a9645250 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.893+0000 7f5a537fe640 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b4a9645250 0x55b4a9647660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.893+0000 7f5a537fe640 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b4a9645250 0x55b4a9647660 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdeab69da14ec5bc server_cookie=47c9d07d06078b5e in_seq=0 out_seq=0 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.893+0000 7f5a32ffd640 10 client.16077 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a32ffd640 1 -- 172.21.15.105:0/3867051468 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b4a959d5f0 con 0x55b4a9645250 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a32ffd640 10 client.16077 handle_command_reply: tid=0 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 client.16077 shutdown 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 -- 172.21.15.105:0/3867051468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 msgr2=0x55b4a9640d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a9640d60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009bf0 tx=0x7f5a4c034840 comp rx=0 tx=0).stop 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 -- 172.21.15.105:0/3867051468 shutdown_connections 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b4a9645250 0x55b4a9647660 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a9631130 0x55b4a9640d60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a962bfc0 0x55b4a9639d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/3867051468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a96397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 -- 172.21.15.105:0/3867051468 >> 172.21.15.105:0/3867051468 conn(0x55b4a9626b40 msgr2=0x55b4a9625df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a32ffd640 0 client.16077 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 -- 172.21.15.105:0/3867051468 shutdown_connections 2024-03-26T17:40:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 1 -- 172.21.15.105:0/3867051468 wait complete. 2024-03-26T17:40:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.894+0000 7f5a5ca06740 20 client.16077 trim_cache size 0 max 16384 2024-03-26T17:40:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a5ca06740 1 Processor -- start 2024-03-26T17:40:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a5ca06740 1 -- start start 2024-03-26T17:40:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962b5d0 0x55b4a9647260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 0x55b4a9640d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a96303a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541a6bb0 con 0x55b4a962b5d0 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541a86f0 con 0x55b4a9631130 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541a7280 con 0x55b4a962bfc0 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962b5d0 0x55b4a9647260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962b5d0 0x55b4a9647260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37292/0 (socket says 172.21.15.105:37292) 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a537fe640 1 -- 172.21.15.105:0/2737817481 learned_addr learned my addr 172.21.15.105:0/2737817481 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a53fff640 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a96303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.919+0000 7f5a52ffd640 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 0x55b4a9640d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a537fe640 1 -- 172.21.15.105:0/2737817481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 msgr2=0x55b4a9640d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a537fe640 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 0x55b4a9640d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a537fe640 1 -- 172.21.15.105:0/2737817481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 msgr2=0x55b4a96303a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a537fe640 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a96303a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a537fe640 1 -- 172.21.15.105:0/2737817481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a96421e0 con 0x55b4a962b5d0 2024-03-26T17:40:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a53fff640 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a96303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a537fe640 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962b5d0 0x55b4a9647260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5a44004770 tx=0x7f5a440cd190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5173946cb05dc702 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a397fa640 1 -- 172.21.15.105:0/2737817481 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440bc0b0 con 0x55b4a962b5d0 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a397fa640 1 -- 172.21.15.105:0/2737817481 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a4409c5f0 con 0x55b4a962b5d0 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.920+0000 7f5a397fa640 1 -- 172.21.15.105:0/2737817481 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4409a1f0 con 0x55b4a962b5d0 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 -- 172.21.15.105:0/2737817481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962b5d0 msgr2=0x55b4a9647260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962b5d0 0x55b4a9647260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5a44004770 tx=0x7f5a440cd190 comp rx=0 tx=0).stop 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 -- 172.21.15.105:0/2737817481 shutdown_connections 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a96303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962bfc0 0x55b4a9640d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/2737817481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962b5d0 0x55b4a9647260 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 -- 172.21.15.105:0/2737817481 >> 172.21.15.105:0/2737817481 conn(0x55b4a9626b40 msgr2=0x55b4a96397e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 -- 172.21.15.105:0/2737817481 shutdown_connections 2024-03-26T17:40:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 -- 172.21.15.105:0/2737817481 wait complete. 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 -- start start 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.921+0000 7f5a5ca06740 1 Processor -- start 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a963e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962bfc0 0x55b4a963e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a5ca06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a963f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541a8020 con 0x55b4a962bfc0 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541a7950 con 0x55b4a9631130 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a5ca06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a541a4b90 con 0x55b4a962b5d0 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a963e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a963e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59116/0 (socket says 172.21.15.105:59116) 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 -- 172.21.15.105:0/1091178175 learned_addr learned my addr 172.21.15.105:0/1091178175 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a52ffd640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962bfc0 0x55b4a963e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a53fff640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a963f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a5ca06740 10 client.0 _ll_register_callbacks cb 0x7f5a48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 -- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 msgr2=0x55b4a963f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a963f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 -- 172.21.15.105:0/1091178175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962bfc0 msgr2=0x55b4a963e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962bfc0 0x55b4a963e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 -- 172.21.15.105:0/1091178175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a963fce0 con 0x55b4a962b5d0 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a53fff640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a963f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a52ffd640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962bfc0 0x55b4a963e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a537fe640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a963e1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400cfd0 tx=0x7f5a440badd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11eb467e9352787 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.922+0000 7f5a32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a32ffd640 1 -- 172.21.15.105:0/1091178175 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a440b4020 con 0x55b4a962b5d0 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 1 -- 172.21.15.105:0/1091178175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a4409c5f0 con 0x55b4a962b5d0 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a32ffd640 1 -- 172.21.15.105:0/1091178175 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f5a44098560 con 0x55b4a962b5d0 2024-03-26T17:40:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a32ffd640 1 -- 172.21.15.105:0/1091178175 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a440bc0b0 con 0x55b4a962b5d0 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 10 client.25933 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 1 -- 172.21.15.105:0/1091178175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b4a9641060 con 0x55b4a962b5d0 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a32ffd640 1 -- 172.21.15.105:0/1091178175 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a4409c5f0 con 0x55b4a962b5d0 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a32ffd640 1 -- 172.21.15.105:0/1091178175 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5a440b7b70 con 0x55b4a962b5d0 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 10 client.25933 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 10 client.25933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 20 client.25933 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b4a96abbe0 0x55b4a96ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 4 client.25933 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.923+0000 7f5a5ca06740 1 -- 172.21.15.105:0/1091178175 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4a9597940 con 0x55b4a96abbe0 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.924+0000 7f5a52ffd640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b4a96abbe0 0x55b4a96ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.924+0000 7f5a32ffd640 10 client.25933 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.924+0000 7f5a52ffd640 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b4a96abbe0 0x55b4a96ae000 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ff4dfdccb796b7 server_cookie=532e6f7cce7a8d0e in_seq=0 out_seq=0 2024-03-26T17:40:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a32ffd640 1 -- 172.21.15.105:0/1091178175 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4a9597940 con 0x55b4a96abbe0 2024-03-26T17:40:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a32ffd640 10 client.25933 handle_command_reply: tid=0 2024-03-26T17:40:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 client.25933 shutdown 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 -- 172.21.15.105:0/1091178175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 msgr2=0x55b4a963e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a963e1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400cfd0 tx=0x7f5a440badd0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 -- 172.21.15.105:0/1091178175 shutdown_connections 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b4a96abbe0 0x55b4a96ae000 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b4a9631130 0x55b4a963f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b4a962bfc0 0x55b4a963e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 --2- 172.21.15.105:0/1091178175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b4a962b5d0 0x55b4a963e1a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a32ffd640 0 client.25933 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 -- 172.21.15.105:0/1091178175 >> 172.21.15.105:0/1091178175 conn(0x55b4a9626b40 msgr2=0x55b4a9630360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 -- 172.21.15.105:0/1091178175 shutdown_connections 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.977+0000 7f5a5ca06740 1 -- 172.21.15.105:0/1091178175 wait complete. 2024-03-26T17:40:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a5ca06740 20 client.25933 trim_cache size 0 max 16384 2024-03-26T17:40:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:40:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:40:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:40:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:40:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 -- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a3c0a3040 msgr2=0x7f5a3c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a3c0a3040 0x7f5a3c0a5500 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55b4a9633cb0 tx=0x7f5a4c047000 comp rx=0 tx=0).stop 2024-03-26T17:40:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 -- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a54100800 msgr2=0x7f5a54197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a54100800 0x7f5a54197ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400e9c0 tx=0x7f5a4400ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:12.973 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 -- 172.21.15.105:0/2848776714 shutdown_connections 2024-03-26T17:40:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a3c0a3040 0x7f5a3c0a5500 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a541023b0 0x7f5a5419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a541019f0 0x7f5a54198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 --2- 172.21.15.105:0/2848776714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a54100800 0x7f5a54197ad0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 -- 172.21.15.105:0/2848776714 >> 172.21.15.105:0/2848776714 conn(0x7f5a540fbd50 msgr2=0x7f5a54069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 -- 172.21.15.105:0/2848776714 shutdown_connections 2024-03-26T17:40:12.974 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:12.978+0000 7f5a59e19640 1 -- 172.21.15.105:0/2848776714 wait complete. 2024-03-26T17:40:13.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 33417 3.54 MB/sec warmup 56 sec latency 28.032 ms 2024-03-26T17:40:14.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 34001 3.49 MB/sec warmup 57 sec latency 141.097 ms 2024-03-26T17:40:14.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:13 smithi144 ceph-mon[36095]: pgmap v403: 97 pgs: 97 active+clean; 28 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:40:14.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:13 smithi105 ceph-mon[32280]: pgmap v403: 97 pgs: 97 active+clean; 28 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:40:14.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:13 smithi135 ceph-mon[35636]: pgmap v403: 97 pgs: 97 active+clean; 28 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:40:15.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 34568 3.47 MB/sec warmup 58 sec latency 15.509 ms 2024-03-26T17:40:16.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 35293 3.52 MB/sec warmup 59 sec latency 14.997 ms 2024-03-26T17:40:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:15 smithi144 ceph-mon[36095]: pgmap v404: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-03-26T17:40:16.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:15 smithi105 ceph-mon[32280]: pgmap v404: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-03-26T17:40:16.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:15 smithi135 ceph-mon[35636]: pgmap v404: 97 pgs: 97 active+clean; 30 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-03-26T17:40:17.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 35827 3.53 MB/sec warmup 60 sec latency 89.443 ms 2024-03-26T17:40:18.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 36288 3.49 MB/sec warmup 61 sec latency 24.564 ms 2024-03-26T17:40:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:17 smithi144 ceph-mon[36095]: pgmap v405: 97 pgs: 97 active+clean; 26 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-03-26T17:40:18.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:17 smithi105 ceph-mon[32280]: pgmap v405: 97 pgs: 97 active+clean; 26 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-03-26T17:40:18.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:17 smithi135 ceph-mon[35636]: pgmap v405: 97 pgs: 97 active+clean; 26 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-03-26T17:40:19.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 36932 3.50 MB/sec warmup 62 sec latency 16.397 ms 2024-03-26T17:40:20.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 37590 3.46 MB/sec warmup 63 sec latency 16.737 ms 2024-03-26T17:40:20.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:19 smithi144 ceph-mon[36095]: pgmap v406: 97 pgs: 97 active+clean; 28 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:40:20.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:19 smithi105 ceph-mon[32280]: pgmap v406: 97 pgs: 97 active+clean; 28 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:40:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:19 smithi135 ceph-mon[35636]: pgmap v406: 97 pgs: 97 active+clean; 28 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:40:21.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 38265 3.49 MB/sec warmup 64 sec latency 16.665 ms 2024-03-26T17:40:22.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 38847 3.49 MB/sec warmup 65 sec latency 20.006 ms 2024-03-26T17:40:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:21 smithi144 ceph-mon[36095]: pgmap v407: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:40:22.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:21 smithi105 ceph-mon[32280]: pgmap v407: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:40:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:21 smithi135 ceph-mon[35636]: pgmap v407: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:40:23.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 39443 3.50 MB/sec warmup 66 sec latency 15.215 ms 2024-03-26T17:40:24.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 39928 3.46 MB/sec warmup 67 sec latency 20.786 ms 2024-03-26T17:40:24.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:23 smithi144 ceph-mon[36095]: pgmap v408: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:40:24.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:23 smithi105 ceph-mon[32280]: pgmap v408: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:40:24.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:23 smithi135 ceph-mon[35636]: pgmap v408: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:40:25.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 40565 3.48 MB/sec warmup 68 sec latency 16.329 ms 2024-03-26T17:40:26.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 41240 3.44 MB/sec warmup 69 sec latency 17.958 ms 2024-03-26T17:40:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.073+0000 7f4cffabf640 1 -- 172.21.15.105:0/1899905156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf8102390 msgr2=0x7f4cf8108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.073+0000 7f4cffabf640 1 --2- 172.21.15.105:0/1899905156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf8102390 0x7f4cf8108c20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009ca0 tx=0x7f4cf002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.073+0000 7f4cffabf640 1 -- 172.21.15.105:0/1899905156 shutdown_connections 2024-03-26T17:40:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.073+0000 7f4cffabf640 1 --2- 172.21.15.105:0/1899905156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf8102390 0x7f4cf8108c20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.073+0000 7f4cffabf640 1 --2- 172.21.15.105:0/1899905156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cf81019d0 0x7f4cf8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.073+0000 7f4cffabf640 1 --2- 172.21.15.105:0/1899905156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf81007e0 0x7f4cf8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.073+0000 7f4cffabf640 1 -- 172.21.15.105:0/1899905156 >> 172.21.15.105:0/1899905156 conn(0x7f4cf80fbd50 msgr2=0x7f4cf80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 -- 172.21.15.105:0/1899905156 shutdown_connections 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 -- 172.21.15.105:0/1899905156 wait complete. 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 Processor -- start 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 -- start start 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf81007e0 0x7f4cf8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf81019d0 0x7f4cf8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cf8102390 0x7f4cf819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf810c310 con 0x7f4cf81007e0 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf810c190 con 0x7f4cf8102390 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cffabf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf810c490 con 0x7f4cf81019d0 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cfd85b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf81007e0 0x7f4cf8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cfd85b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf81007e0 0x7f4cf8197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43974/0 (socket says 172.21.15.105:43974) 2024-03-26T17:40:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cfd85b640 1 -- 172.21.15.105:0/2684671552 learned_addr learned my addr 172.21.15.105:0/2684671552 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf81019d0 0x7f4cf8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.074+0000 7f4cfe05c640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cf8102390 0x7f4cf819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4cfd85b640 1 -- 172.21.15.105:0/2684671552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf81019d0 msgr2=0x7f4cf8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf81019d0 0x7f4cf8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4cfd85b640 1 -- 172.21.15.105:0/2684671552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cf8102390 msgr2=0x7f4cf819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cf8102390 0x7f4cf819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4cfd85b640 1 -- 172.21.15.105:0/2684671552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf819f640 con 0x7f4cf81007e0 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf81007e0 0x7f4cf8197a20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400e9c0 tx=0x7f4cf400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4ceeffd640 1 -- 172.21.15.105:0/2684671552 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400cd80 con 0x7f4cf81007e0 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4ceeffd640 1 -- 172.21.15.105:0/2684671552 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cf40035f0 con 0x7f4cf81007e0 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4cffabf640 1 -- 172.21.15.105:0/2684671552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf819f930 con 0x7f4cf81007e0 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4ceeffd640 1 -- 172.21.15.105:0/2684671552 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4010690 con 0x7f4cf81007e0 2024-03-26T17:40:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.075+0000 7f4cffabf640 1 -- 172.21.15.105:0/2684671552 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf819fe80 con 0x7f4cf81007e0 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.078+0000 7f4ceeffd640 1 -- 172.21.15.105:0/2684671552 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4cf40037a0 con 0x7f4cf81007e0 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.078+0000 7f4d02781740 1 Processor -- start 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4d02781740 1 -- start start 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4d02781740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fbfe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4d02781740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc03ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4d02781740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fc065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4d02781740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d30fb630b0 con 0x55d30fbfefc0 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4d02781740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d30fb0b960 con 0x55d30fc04130 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4d02781740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d30fb5f060 con 0x55d30fbfe5d0 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfe05c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fbfe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfd05a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fc065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfd85b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc03ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfe05c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fbfe9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56300/0 (socket says 172.21.15.105:56300) 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfe05c640 1 -- 172.21.15.105:0/3210735879 learned_addr learned my addr 172.21.15.105:0/3210735879 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4ceeffd640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ce40a3040 0x7f4ce40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfd05a640 1 -- 172.21.15.105:0/3210735879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 msgr2=0x55d30fbfe9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/3210735879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fbfe9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfe05c640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ce40a3040 0x7f4ce40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfd05a640 1 -- 172.21.15.105:0/3210735879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 msgr2=0x55d30fc03ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/3210735879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc03ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4cfd05a640 1 -- 172.21.15.105:0/3210735879 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30fb1dc60 con 0x55d30fc04130 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.079+0000 7f4ceeffd640 1 -- 172.21.15.105:0/2684671552 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f4cf40cf8c0 con 0x7f4cf81007e0 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.080+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/3210735879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fc065c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009bf0 tx=0x7f4ce8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d06f9feb5f2829c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.080+0000 7f4ce3fff640 1 -- 172.21.15.105:0/3210735879 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8009da0 con 0x55d30fc04130 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.080+0000 7f4cfe05c640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ce40a3040 0x7f4ce40a5500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55d30fbff820 tx=0x7f4cf002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.080+0000 7f4ce3fff640 1 -- 172.21.15.105:0/3210735879 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ce8005020 con 0x55d30fc04130 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.080+0000 7f4ce3fff640 1 -- 172.21.15.105:0/3210735879 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8004b20 con 0x55d30fc04130 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 -- 172.21.15.105:0/3210735879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 msgr2=0x55d30fc065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 --2- 172.21.15.105:0/3210735879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fc065c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009bf0 tx=0x7f4ce8011810 comp rx=0 tx=0).stop 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 -- 172.21.15.105:0/3210735879 shutdown_connections 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 --2- 172.21.15.105:0/3210735879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fc065c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 --2- 172.21.15.105:0/3210735879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc03ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 --2- 172.21.15.105:0/3210735879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fbfe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 -- 172.21.15.105:0/3210735879 >> 172.21.15.105:0/3210735879 conn(0x55d30fbf9b40 msgr2=0x55d30fbf9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 -- 172.21.15.105:0/3210735879 shutdown_connections 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 -- 172.21.15.105:0/3210735879 wait complete. 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 -- start start 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.081+0000 7f4d02781740 1 Processor -- start 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4d02781740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fc082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4d02781740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfefc0 0x55d30fc08820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4d02781740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fc04130 0x55d30fc0f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4d02781740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d30fc070e0 con 0x55d30fc04130 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4d02781740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d30fb1edb0 con 0x55d30fbfefc0 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4d02781740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d30fc07260 con 0x55d30fbfe5d0 2024-03-26T17:40:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fc082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fc082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56304/0 (socket says 172.21.15.105:56304) 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 -- 172.21.15.105:0/1672664270 learned_addr learned my addr 172.21.15.105:0/1672664270 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfefc0 0x55d30fc08820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4d02781740 10 client.0 _ll_register_callbacks cb 0x55d30fb23110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfe05c640 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fc04130 0x55d30fc0f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 -- 172.21.15.105:0/1672664270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfefc0 msgr2=0x55d30fc08820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfefc0 0x55d30fc08820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 -- 172.21.15.105:0/1672664270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fc04130 msgr2=0x55d30fc0f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fc04130 0x55d30fc0f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 -- 172.21.15.105:0/1672664270 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30f877210 con 0x55d30fbfe5d0 2024-03-26T17:40:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfe05c640 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fc04130 0x55d30fc0f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fc082e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4098980 tx=0x7f4cf409be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21e969c93e9b152f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.082+0000 7f4ce17fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.083+0000 7f4ce17fa640 1 -- 172.21.15.105:0/1672664270 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf409c6b0 con 0x55d30fbfe5d0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.083+0000 7f4ce17fa640 1 -- 172.21.15.105:0/1672664270 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f4cf4097d00 con 0x55d30fbfe5d0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.083+0000 7f4d02781740 1 -- 172.21.15.105:0/1672664270 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce8005020 con 0x55d30fbfe5d0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.083+0000 7f4ce17fa640 1 -- 172.21.15.105:0/1672664270 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cf409bfb0 con 0x55d30fbfe5d0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.083+0000 7f4d02781740 10 client.25945 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.083+0000 7f4d02781740 1 -- 172.21.15.105:0/1672664270 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d30f943620 con 0x55d30fbfe5d0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.083+0000 7f4ce17fa640 1 -- 172.21.15.105:0/1672664270 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ce8005020 con 0x55d30fbfe5d0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4ce17fa640 1 -- 172.21.15.105:0/1672664270 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4cf4096b20 con 0x55d30fbfe5d0 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4d02781740 10 client.25945 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4d02781740 10 client.25945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:26.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4d02781740 20 client.25945 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4d02781740 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d30fc145c0 0x55d30fc169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4d02781740 4 client.25945 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4d02781740 1 -- 172.21.15.105:0/1672664270 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d30fb705f0 con 0x55d30fc145c0 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d30fc145c0 0x55d30fc169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d30fc145c0 0x55d30fc169b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1723ef3693cc62a4 server_cookie=7e906fd07efc5e6 in_seq=0 out_seq=0 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.084+0000 7f4ce17fa640 10 client.25945 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4ce17fa640 1 -- 172.21.15.105:0/1672664270 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d30fb705f0 con 0x55d30fc145c0 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4ce17fa640 10 client.25945 handle_command_reply: tid=0 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 client.25945 shutdown 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 -- 172.21.15.105:0/1672664270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 msgr2=0x55d30fc082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fc082e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4098980 tx=0x7f4cf409be40 comp rx=0 tx=0).stop 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 -- 172.21.15.105:0/1672664270 shutdown_connections 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d30fc145c0 0x55d30fc169b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fc04130 0x55d30fc0f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfefc0 0x55d30fc08820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4ce17fa640 0 client.25945 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 --2- 172.21.15.105:0/1672664270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfe5d0 0x55d30fc082e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 -- 172.21.15.105:0/1672664270 >> 172.21.15.105:0/1672664270 conn(0x55d30fbf9b40 msgr2=0x55d30fbf8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 -- 172.21.15.105:0/1672664270 shutdown_connections 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 1 -- 172.21.15.105:0/1672664270 wait complete. 2024-03-26T17:40:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.085+0000 7f4d02781740 20 client.25945 trim_cache size 0 max 16384 2024-03-26T17:40:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:25 smithi144 ceph-mon[36095]: pgmap v409: 97 pgs: 97 active+clean; 27 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-03-26T17:40:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4d02781740 1 Processor -- start 2024-03-26T17:40:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4d02781740 1 -- start start 2024-03-26T17:40:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4d02781740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfe5d0 0x55d30fc08600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4d02781740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfefc0 0x55d30fbff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4d02781740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fbffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4d02781740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81a6a90 con 0x55d30fbfe5d0 2024-03-26T17:40:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4d02781740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81a85d0 con 0x55d30fc04130 2024-03-26T17:40:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4d02781740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81a7160 con 0x55d30fbfefc0 2024-03-26T17:40:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfe5d0 0x55d30fc08600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfe5d0 0x55d30fc08600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44002/0 (socket says 172.21.15.105:44002) 2024-03-26T17:40:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 -- 172.21.15.105:0/3538282124 learned_addr learned my addr 172.21.15.105:0/3538282124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfe05c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fbffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfefc0 0x55d30fbff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 -- 172.21.15.105:0/3538282124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfefc0 msgr2=0x55d30fbff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfefc0 0x55d30fbff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 -- 172.21.15.105:0/3538282124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 msgr2=0x55d30fbffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fbffd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 -- 172.21.15.105:0/3538282124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d30fc140f0 con 0x55d30fbfe5d0 2024-03-26T17:40:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfe05c640 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fbffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.107+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfe5d0 0x55d30fc08600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4098930 tx=0x7f4cf409beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9989d8d5ed494f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.108+0000 7f4ce37fe640 1 -- 172.21.15.105:0/3538282124 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf409b910 con 0x55d30fbfe5d0 2024-03-26T17:40:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.108+0000 7f4ce37fe640 1 -- 172.21.15.105:0/3538282124 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cf409c6b0 con 0x55d30fbfe5d0 2024-03-26T17:40:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.108+0000 7f4ce37fe640 1 -- 172.21.15.105:0/3538282124 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4097d00 con 0x55d30fbfe5d0 2024-03-26T17:40:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 -- 172.21.15.105:0/3538282124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfe5d0 msgr2=0x55d30fc08600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfe5d0 0x55d30fc08600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4098930 tx=0x7f4cf409beb0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 -- 172.21.15.105:0/3538282124 shutdown_connections 2024-03-26T17:40:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fc04130 0x55d30fbffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fbfefc0 0x55d30fbff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 --2- 172.21.15.105:0/3538282124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfe5d0 0x55d30fc08600 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 -- 172.21.15.105:0/3538282124 >> 172.21.15.105:0/3538282124 conn(0x55d30fbf9b40 msgr2=0x55d30fc07ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 -- 172.21.15.105:0/3538282124 shutdown_connections 2024-03-26T17:40:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 -- 172.21.15.105:0/3538282124 wait complete. 2024-03-26T17:40:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 -- start start 2024-03-26T17:40:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.109+0000 7f4d02781740 1 Processor -- start 2024-03-26T17:40:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4d02781740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfe5d0 0x55d30fc74660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4d02781740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc76bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4d02781740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fc04130 0x55d30fc79100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4d02781740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81a7f00 con 0x55d30fbfefc0 2024-03-26T17:40:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4d02781740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81a7830 con 0x55d30fbfe5d0 2024-03-26T17:40:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4d02781740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf81a4a70 con 0x55d30fc04130 2024-03-26T17:40:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4d02781740 10 client.0 _ll_register_callbacks cb 0x7f4cf4003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc76bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfe05c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fc04130 0x55d30fc79100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd85b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfe5d0 0x55d30fc74660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc76bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44014/0 (socket says 172.21.15.105:44014) 2024-03-26T17:40:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 -- 172.21.15.105:0/70242593 learned_addr learned my addr 172.21.15.105:0/70242593 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 -- 172.21.15.105:0/70242593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fc04130 msgr2=0x55d30fc79100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fc04130 0x55d30fc79100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 -- 172.21.15.105:0/70242593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfe5d0 msgr2=0x55d30fc74660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfe5d0 0x55d30fc74660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 -- 172.21.15.105:0/70242593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30fc79930 con 0x55d30fbfefc0 2024-03-26T17:40:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfe05c640 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fc04130 0x55d30fc79100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4cfd05a640 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc76bb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8014040 tx=0x7f4ce8009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f887f71f6e7cbc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4ce17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4ce17fa640 1 -- 172.21.15.105:0/70242593 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8005530 con 0x55d30fbfefc0 2024-03-26T17:40:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.110+0000 7f4ce17fa640 1 -- 172.21.15.105:0/70242593 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f4ce802ba30 con 0x55d30fbfefc0 2024-03-26T17:40:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 1 -- 172.21.15.105:0/70242593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cf409c6b0 con 0x55d30fbfefc0 2024-03-26T17:40:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4ce17fa640 1 -- 172.21.15.105:0/70242593 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ce803b050 con 0x55d30fbfefc0 2024-03-26T17:40:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 10 client.16101 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 1 -- 172.21.15.105:0/70242593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d30fc7acb0 con 0x55d30fbfefc0 2024-03-26T17:40:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4ce17fa640 1 -- 172.21.15.105:0/70242593 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cf409c6b0 con 0x55d30fbfefc0 2024-03-26T17:40:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4ce17fa640 1 -- 172.21.15.105:0/70242593 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ce8004040 con 0x55d30fbfefc0 2024-03-26T17:40:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 20 client.16101 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d30fc145c0 0x55d30fc7d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 4 client.16101 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:40:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4d02781740 1 -- 172.21.15.105:0/70242593 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d30fb6a940 con 0x55d30fc145c0 2024-03-26T17:40:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.111+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d30fc145c0 0x55d30fc7d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.112+0000 7f4ce17fa640 10 client.16101 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.112+0000 7f4cfd85b640 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d30fc145c0 0x55d30fc7d500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac0bcb7dc491248 server_cookie=da69ddef4f2f2642 in_seq=0 out_seq=0 2024-03-26T17:40:26.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:25 smithi105 ceph-mon[32280]: pgmap v409: 97 pgs: 97 active+clean; 27 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-03-26T17:40:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4ce17fa640 1 -- 172.21.15.105:0/70242593 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d30fb6a940 con 0x55d30fc145c0 2024-03-26T17:40:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4ce17fa640 10 client.16101 handle_command_reply: tid=0 2024-03-26T17:40:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 client.16101 shutdown 2024-03-26T17:40:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 -- 172.21.15.105:0/70242593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 msgr2=0x55d30fc76bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc76bb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8014040 tx=0x7f4ce8009ce0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 -- 172.21.15.105:0/70242593 shutdown_connections 2024-03-26T17:40:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d30fc145c0 0x55d30fc7d500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d30fc04130 0x55d30fc79100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d30fbfefc0 0x55d30fc76bb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 --2- 172.21.15.105:0/70242593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d30fbfe5d0 0x55d30fc74660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4d02781740 1 -- 172.21.15.105:0/70242593 >> 172.21.15.105:0/70242593 conn(0x55d30fbf9b40 msgr2=0x55d30fbff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.152+0000 7f4ce17fa640 0 client.16101 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4d02781740 1 -- 172.21.15.105:0/70242593 shutdown_connections 2024-03-26T17:40:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4d02781740 1 -- 172.21.15.105:0/70242593 wait complete. 2024-03-26T17:40:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4d02781740 20 client.16101 trim_cache size 0 max 16384 2024-03-26T17:40:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:40:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:40:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:40:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:40:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 -- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ce40a3040 msgr2=0x7f4ce40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ce40a3040 0x7f4ce40a5500 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55d30fbff820 tx=0x7f4cf002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 -- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf81007e0 msgr2=0x7f4cf8197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf81007e0 0x7f4cf8197a20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400e9c0 tx=0x7f4cf400ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 -- 172.21.15.105:0/2684671552 shutdown_connections 2024-03-26T17:40:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ce40a3040 0x7f4ce40a5500 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cf8102390 0x7f4cf819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cf81019d0 0x7f4cf8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 --2- 172.21.15.105:0/2684671552 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf81007e0 0x7f4cf8197a20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 -- 172.21.15.105:0/2684671552 >> 172.21.15.105:0/2684671552 conn(0x7f4cf80fbd50 msgr2=0x7f4cf80fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 -- 172.21.15.105:0/2684671552 shutdown_connections 2024-03-26T17:40:26.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:26.153+0000 7f4cffabf640 1 -- 172.21.15.105:0/2684671552 wait complete. 2024-03-26T17:40:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:25 smithi135 ceph-mon[35636]: pgmap v409: 97 pgs: 97 active+clean; 27 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-03-26T17:40:27.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 41933 3.47 MB/sec warmup 70 sec latency 29.984 ms 2024-03-26T17:40:28.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 42574 3.50 MB/sec warmup 71 sec latency 15.756 ms 2024-03-26T17:40:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:27 smithi144 ceph-mon[36095]: pgmap v410: 97 pgs: 97 active+clean; 28 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2024-03-26T17:40:28.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:27 smithi105 ceph-mon[32280]: pgmap v410: 97 pgs: 97 active+clean; 28 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2024-03-26T17:40:28.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:27 smithi135 ceph-mon[35636]: pgmap v410: 97 pgs: 97 active+clean; 28 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.3 MiB/s wr, 88 op/s 2024-03-26T17:40:29.018 INFO:tasks.workunit.client.0.smithi105.stdout: 1 43037 3.48 MB/sec warmup 72 sec latency 13.782 ms 2024-03-26T17:40:30.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 43536 3.44 MB/sec warmup 73 sec latency 24.297 ms 2024-03-26T17:40:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:29 smithi144 ceph-mon[36095]: pgmap v411: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-03-26T17:40:30.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:29 smithi105 ceph-mon[32280]: pgmap v411: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-03-26T17:40:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:29 smithi135 ceph-mon[35636]: pgmap v411: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-03-26T17:40:31.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 44158 3.45 MB/sec warmup 74 sec latency 16.382 ms 2024-03-26T17:40:32.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 44902 3.43 MB/sec warmup 75 sec latency 16.309 ms 2024-03-26T17:40:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:31 smithi144 ceph-mon[36095]: pgmap v412: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.5 MiB/s wr, 89 op/s 2024-03-26T17:40:32.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:31 smithi105 ceph-mon[32280]: pgmap v412: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.5 MiB/s wr, 89 op/s 2024-03-26T17:40:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:31 smithi135 ceph-mon[35636]: pgmap v412: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.5 MiB/s wr, 89 op/s 2024-03-26T17:40:33.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 45610 3.46 MB/sec warmup 76 sec latency 16.913 ms 2024-03-26T17:40:34.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 46228 3.48 MB/sec warmup 77 sec latency 20.510 ms 2024-03-26T17:40:34.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:33 smithi144 ceph-mon[36095]: pgmap v413: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:40:34.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:33 smithi105 ceph-mon[32280]: pgmap v413: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:40:34.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:33 smithi135 ceph-mon[35636]: pgmap v413: 97 pgs: 97 active+clean; 26 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:40:35.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 46646 3.46 MB/sec warmup 78 sec latency 15.676 ms 2024-03-26T17:40:36.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 47203 3.44 MB/sec warmup 79 sec latency 27.343 ms 2024-03-26T17:40:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:35 smithi144 ceph-mon[36095]: pgmap v414: 97 pgs: 97 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-03-26T17:40:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:35 smithi105 ceph-mon[32280]: pgmap v414: 97 pgs: 97 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-03-26T17:40:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:35 smithi135 ceph-mon[35636]: pgmap v414: 97 pgs: 97 active+clean; 25 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.7 MiB/s wr, 109 op/s 2024-03-26T17:40:36.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:40:36.358 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:40:36.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 -- 172.21.15.105:0/2628113107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e181075a0 msgr2=0x7f0e181099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/2628113107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e181075a0 0x7f0e181099f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009ca0 tx=0x7f0e1002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 -- 172.21.15.105:0/2628113107 shutdown_connections 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/2628113107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e181075a0 0x7f0e181099f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/2628113107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e18104c70 0x7f0e18107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/2628113107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e18102340 0x7f0e18104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 -- 172.21.15.105:0/2628113107 >> 172.21.15.105:0/2628113107 conn(0x7f0e180fbd50 msgr2=0x7f0e180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 -- 172.21.15.105:0/2628113107 shutdown_connections 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.486+0000 7f0e1fff4640 1 -- 172.21.15.105:0/2628113107 wait complete. 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1fff4640 1 Processor -- start 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1fff4640 1 -- start start 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1fff4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e18102340 0x7f0e18197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1fff4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18104c70 0x7f0e18198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1fff4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e181075a0 0x7f0e1819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1fff4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1810d410 con 0x7f0e18104c70 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1fff4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1810d290 con 0x7f0e18102340 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1fff4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1810d590 con 0x7f0e181075a0 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1d58f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18104c70 0x7f0e18198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1dd90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e18102340 0x7f0e18197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1d58f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18104c70 0x7f0e18198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58822/0 (socket says 172.21.15.105:58822) 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1d58f640 1 -- 172.21.15.105:0/3582145097 learned_addr learned my addr 172.21.15.105:0/3582145097 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1e591640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e181075a0 0x7f0e1819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1d58f640 1 -- 172.21.15.105:0/3582145097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e181075a0 msgr2=0x7f0e1819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e181075a0 0x7f0e1819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1d58f640 1 -- 172.21.15.105:0/3582145097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e18102340 msgr2=0x7f0e18197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e18102340 0x7f0e18197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.487+0000 7f0e1d58f640 1 -- 172.21.15.105:0/3582145097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1819f760 con 0x7f0e18104c70 2024-03-26T17:40:36.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.488+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18104c70 0x7f0e18198080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800e9f0 tx=0x7f0e0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:36.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.488+0000 7f0e0effd640 1 -- 172.21.15.105:0/3582145097 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0800ccc0 con 0x7f0e18104c70 2024-03-26T17:40:36.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.488+0000 7f0e0effd640 1 -- 172.21.15.105:0/3582145097 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e080054d0 con 0x7f0e18104c70 2024-03-26T17:40:36.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.488+0000 7f0e1fff4640 1 -- 172.21.15.105:0/3582145097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1819fa50 con 0x7f0e18104c70 2024-03-26T17:40:36.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.488+0000 7f0e0effd640 1 -- 172.21.15.105:0/3582145097 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08010430 con 0x7f0e18104c70 2024-03-26T17:40:36.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.488+0000 7f0e1fff4640 1 -- 172.21.15.105:0/3582145097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1819ffa0 con 0x7f0e18104c70 2024-03-26T17:40:36.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.490+0000 7f0e0effd640 1 -- 172.21.15.105:0/3582145097 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0e08005020 con 0x7f0e18104c70 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e22be4740 1 Processor -- start 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e22be4740 1 -- start start 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e0effd640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0e040a2fa0 0x7f0e040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133c8f0 0x56217133cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 0x562171341de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e22be4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5621712a1980 con 0x562171342450 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e22be4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562171249be0 con 0x56217133c8f0 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e22be4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56217129d930 con 0x56217133d2e0 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e1e591640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0e040a2fa0 0x7f0e040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.491+0000 7f0e0effd640 1 -- 172.21.15.105:0/3582145097 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f0e080cf740 con 0x7f0e18104c70 2024-03-26T17:40:36.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133c8f0 0x56217133cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133c8f0 0x56217133cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43900/0 (socket says 172.21.15.105:43900) 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 -- 172.21.15.105:0/2741115928 learned_addr learned my addr 172.21.15.105:0/2741115928 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 0x562171341de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1e591640 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 -- 172.21.15.105:0/2741115928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 msgr2=0x562171341de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 0x562171341de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 -- 172.21.15.105:0/2741115928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 msgr2=0x5621713448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 -- 172.21.15.105:0/2741115928 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217107f7d0 con 0x56217133c8f0 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1e591640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0e040a2fa0 0x7f0e040a5460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e10006010 tx=0x7f0e10032ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133c8f0 0x56217133cd10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009cf0 tx=0x7f0e14011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eca04daa6e394ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e03fff640 1 -- 172.21.15.105:0/2741115928 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14004230 con 0x56217133c8f0 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e03fff640 1 -- 172.21.15.105:0/2741115928 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e140043d0 con 0x56217133c8f0 2024-03-26T17:40:36.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.492+0000 7f0e03fff640 1 -- 172.21.15.105:0/2741115928 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140197c0 con 0x56217133c8f0 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 -- 172.21.15.105:0/2741115928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133c8f0 msgr2=0x56217133cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133c8f0 0x56217133cd10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009cf0 tx=0x7f0e14011810 comp rx=0 tx=0).stop 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 -- 172.21.15.105:0/2741115928 shutdown_connections 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 0x562171341de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2741115928 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133c8f0 0x56217133cd10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 -- 172.21.15.105:0/2741115928 >> 172.21.15.105:0/2741115928 conn(0x562171337e60 msgr2=0x562171338260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 -- 172.21.15.105:0/2741115928 shutdown_connections 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.493+0000 7f0e22be4740 1 -- 172.21.15.105:0/2741115928 wait complete. 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 1 -- start start 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 1 Processor -- start 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171346aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x562171346fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x56217134e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562171345400 con 0x562171342450 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56217125d160 con 0x56217133d2e0 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562171345580 con 0x56217133c8f0 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e22be4740 10 client.0 _ll_register_callbacks cb 0x562171299720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e1dd90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171346aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e1d58f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x562171346fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e1dd90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171346aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41638/0 (socket says 172.21.15.105:41638) 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e1dd90640 1 -- 172.21.15.105:0/1323489175 learned_addr learned my addr 172.21.15.105:0/1323489175 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.494+0000 7f0e1e591640 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x56217134e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e1dd90640 1 -- 172.21.15.105:0/1323489175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 msgr2=0x562171346fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x562171346fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e1dd90640 1 -- 172.21.15.105:0/1323489175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 msgr2=0x56217134e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x56217134e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e1dd90640 1 -- 172.21.15.105:0/1323489175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562170fb53f0 con 0x56217133c8f0 2024-03-26T17:40:36.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x562171346fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171346aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e14004630 tx=0x7f0e140038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9621de173a729ed6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e017fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e017fa640 1 -- 172.21.15.105:0/1323489175 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14027070 con 0x56217133c8f0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e22be4740 1 -- 172.21.15.105:0/1323489175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e140043d0 con 0x56217133c8f0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e017fa640 1 -- 172.21.15.105:0/1323489175 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f0e14005450 con 0x56217133c8f0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e017fa640 1 -- 172.21.15.105:0/1323489175 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e14033050 con 0x56217133c8f0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e22be4740 10 client.25963 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e22be4740 1 -- 172.21.15.105:0/1323489175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562170f750f0 con 0x56217133c8f0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.495+0000 7f0e017fa640 1 -- 172.21.15.105:0/1323489175 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e140043d0 con 0x56217133c8f0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e017fa640 1 -- 172.21.15.105:0/1323489175 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0e14019500 con 0x56217133c8f0 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e22be4740 10 client.25963 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e22be4740 10 client.25963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e22be4740 20 client.25963 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e22be4740 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621713526a0 0x562171354af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e22be4740 4 client.25963 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:36.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e22be4740 1 -- 172.21.15.105:0/1323489175 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621712aeec0 con 0x5621713526a0 2024-03-26T17:40:36.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621713526a0 0x562171354af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e017fa640 10 client.25963 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:36.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.496+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621713526a0 0x562171354af0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfc7464fa8c13da server_cookie=3adbb0476eba23a in_seq=0 out_seq=0 2024-03-26T17:40:36.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e017fa640 1 -- 172.21.15.105:0/1323489175 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5621712aeec0 con 0x5621713526a0 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e017fa640 10 client.25963 handle_command_reply: tid=0 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 client.25963 shutdown 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 -- 172.21.15.105:0/1323489175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 msgr2=0x562171346aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171346aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e14004630 tx=0x7f0e140038a0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 -- 172.21.15.105:0/1323489175 shutdown_connections 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621713526a0 0x562171354af0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x56217134e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x562171346fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 --2- 172.21.15.105:0/1323489175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171346aa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e017fa640 0 client.25963 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 -- 172.21.15.105:0/1323489175 >> 172.21.15.105:0/1323489175 conn(0x562171337e60 msgr2=0x562171337080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 -- 172.21.15.105:0/1323489175 shutdown_connections 2024-03-26T17:40:36.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 1 -- 172.21.15.105:0/1323489175 wait complete. 2024-03-26T17:40:36.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.498+0000 7f0e22be4740 20 client.25963 trim_cache size 0 max 16384 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.516+0000 7f0e22be4740 1 Processor -- start 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e22be4740 1 -- start start 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171338240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x56217134dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e22be4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181a8e30 con 0x562171342450 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e22be4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181aa970 con 0x56217133d2e0 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e22be4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181a9500 con 0x56217133c8f0 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1dd90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171338240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58858/0 (socket says 172.21.15.105:58858) 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 -- 172.21.15.105:0/390817554 learned_addr learned my addr 172.21.15.105:0/390817554 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x56217134dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 -- 172.21.15.105:0/390817554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 msgr2=0x562171338240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171338240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 -- 172.21.15.105:0/390817554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 msgr2=0x56217134dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x56217134dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 -- 172.21.15.105:0/390817554 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217133d6c0 con 0x562171342450 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171338240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x56217134dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.517+0000 7f0e1e591640 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713460c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009ca0 tx=0x7f0e1002e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bd085151e6d15a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e037fe640 1 -- 172.21.15.105:0/390817554 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10039040 con 0x562171342450 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e037fe640 1 -- 172.21.15.105:0/390817554 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e10053070 con 0x562171342450 2024-03-26T17:40:36.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e037fe640 1 -- 172.21.15.105:0/390817554 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e100348a0 con 0x562171342450 2024-03-26T17:40:36.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 -- 172.21.15.105:0/390817554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 msgr2=0x5621713460c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713460c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009ca0 tx=0x7f0e1002e050 comp rx=0 tx=0).stop 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 -- 172.21.15.105:0/390817554 shutdown_connections 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562171342450 0x5621713460c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56217133d2e0 0x56217134dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 --2- 172.21.15.105:0/390817554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133c8f0 0x562171338240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 -- 172.21.15.105:0/390817554 >> 172.21.15.105:0/390817554 conn(0x562171337e60 msgr2=0x56217133b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 -- 172.21.15.105:0/390817554 shutdown_connections 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.518+0000 7f0e22be4740 1 -- 172.21.15.105:0/390817554 wait complete. 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 1 -- start start 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 1 Processor -- start 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56217133c8f0 0x56217134b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 0x56217134ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562171342450 0x56217134ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181aa2a0 con 0x56217133c8f0 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181a9bd0 con 0x562171342450 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181a6e10 con 0x56217133d2e0 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e22be4740 10 client.0 _ll_register_callbacks cb 0x7f0e080980d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e1dd90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56217133c8f0 0x56217134b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e1dd90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56217133c8f0 0x56217134b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58874/0 (socket says 172.21.15.105:58874) 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e1dd90640 1 -- 172.21.15.105:0/2740216760 learned_addr learned my addr 172.21.15.105:0/2740216760 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e1e591640 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562171342450 0x56217134ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.519+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 0x56217134ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e1dd90640 1 -- 172.21.15.105:0/2740216760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 msgr2=0x56217134ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 0x56217134ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e1dd90640 1 -- 172.21.15.105:0/2740216760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562171342450 msgr2=0x56217134ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:36.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562171342450 0x56217134ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e1dd90640 1 -- 172.21.15.105:0/2740216760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56217134d020 con 0x56217133c8f0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e1dd90640 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56217133c8f0 0x56217134b4e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0e14011850 tx=0x7f0e14009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e411305eb8be202 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e017fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e017fa640 1 -- 172.21.15.105:0/2740216760 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1403c060 con 0x56217133c8f0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e017fa640 1 -- 172.21.15.105:0/2740216760 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f0e14019420 con 0x56217133c8f0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e22be4740 1 -- 172.21.15.105:0/2740216760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e10053070 con 0x56217133c8f0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e017fa640 1 -- 172.21.15.105:0/2740216760 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e14027050 con 0x56217133c8f0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e22be4740 10 client.16119 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e22be4740 1 -- 172.21.15.105:0/2740216760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56217134e330 con 0x56217133c8f0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.520+0000 7f0e017fa640 1 -- 172.21.15.105:0/2740216760 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e10053070 con 0x56217133c8f0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e017fa640 1 -- 172.21.15.105:0/2740216760 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f0e14004660 con 0x56217133c8f0 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e22be4740 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e22be4740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:36.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e22be4740 20 client.16119 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621713b90e0 0x5621713bb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e22be4740 4 client.16119 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e22be4740 1 -- 172.21.15.105:0/2740216760 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5621712a9210 con 0x5621713b90e0 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621713b90e0 0x5621713bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e017fa640 10 client.16119 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.521+0000 7f0e1d58f640 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621713b90e0 0x5621713bb500 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc2d1b6bfa15412 server_cookie=b0d444a73c800fd8 in_seq=0 out_seq=0 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e017fa640 1 -- 172.21.15.105:0/2740216760 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5621712a9210 con 0x5621713b90e0 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e017fa640 10 client.16119 handle_command_reply: tid=0 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 client.16119 shutdown 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 -- 172.21.15.105:0/2740216760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56217133c8f0 msgr2=0x56217134b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56217133c8f0 0x56217134b4e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0e14011850 tx=0x7f0e14009640 comp rx=0 tx=0).stop 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 -- 172.21.15.105:0/2740216760 shutdown_connections 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621713b90e0 0x5621713bb500 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562171342450 0x56217134ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56217133d2e0 0x56217134ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e017fa640 0 client.16119 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 --2- 172.21.15.105:0/2740216760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56217133c8f0 0x56217134b4e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 -- 172.21.15.105:0/2740216760 >> 172.21.15.105:0/2740216760 conn(0x562171337e60 msgr2=0x5621713367f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 -- 172.21.15.105:0/2740216760 shutdown_connections 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 1 -- 172.21.15.105:0/2740216760 wait complete. 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.522+0000 7f0e22be4740 20 client.16119 trim_cache size 0 max 16384 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:40:36.521 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:40:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 -- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0e040a2fa0 msgr2=0x7f0e040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0e040a2fa0 0x7f0e040a5460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e10006010 tx=0x7f0e10032ec0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 -- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18104c70 msgr2=0x7f0e18198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18104c70 0x7f0e18198080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800e9f0 tx=0x7f0e0800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 -- 172.21.15.105:0/3582145097 shutdown_connections 2024-03-26T17:40:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0e040a2fa0 0x7f0e040a5460 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e181075a0 0x7f0e1819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e18104c70 0x7f0e18198080 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 --2- 172.21.15.105:0/3582145097 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e18102340 0x7f0e18197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 -- 172.21.15.105:0/3582145097 >> 172.21.15.105:0/3582145097 conn(0x7f0e180fbd50 msgr2=0x7f0e18102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 -- 172.21.15.105:0/3582145097 shutdown_connections 2024-03-26T17:40:36.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.523+0000 7f0e1fff4640 1 -- 172.21.15.105:0/3582145097 wait complete. 2024-03-26T17:40:36.527 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:40:36.527 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:40:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.648+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3929460954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac102390 msgr2=0x7fa7ac108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.648+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3929460954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac102390 0x7fa7ac108c20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009ca0 tx=0x7fa7a402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.648+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3929460954 shutdown_connections 2024-03-26T17:40:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.648+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3929460954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac102390 0x7fa7ac108c20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.648+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3929460954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7ac1019d0 0x7fa7ac101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.648+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3929460954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac1007e0 0x7fa7ac100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.648+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3929460954 >> 172.21.15.105:0/3929460954 conn(0x7fa7ac0fbd50 msgr2=0x7fa7ac0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3929460954 shutdown_connections 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3929460954 wait complete. 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 Processor -- start 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 -- start start 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac1007e0 0x7fa7ac197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac1019d0 0x7fa7ac198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7ac102390 0x7fa7ac19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac10c2f0 con 0x7fa7ac1007e0 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac10c170 con 0x7fa7ac1019d0 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b3f1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac10c470 con 0x7fa7ac102390 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.649+0000 7fa7b1cb7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac1007e0 0x7fa7ac197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b14b6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac1019d0 0x7fa7ac198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b1cb7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac1007e0 0x7fa7ac197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58892/0 (socket says 172.21.15.105:58892) 2024-03-26T17:40:36.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b1cb7640 1 -- 172.21.15.105:0/3129508937 learned_addr learned my addr 172.21.15.105:0/3129508937 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b24b8640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7ac102390 0x7fa7ac19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b1cb7640 1 -- 172.21.15.105:0/3129508937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7ac102390 msgr2=0x7fa7ac19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b1cb7640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7ac102390 0x7fa7ac19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b1cb7640 1 -- 172.21.15.105:0/3129508937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac1019d0 msgr2=0x7fa7ac198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b1cb7640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac1019d0 0x7fa7ac198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b1cb7640 1 -- 172.21.15.105:0/3129508937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac19f6e0 con 0x7fa7ac1007e0 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b14b6640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac1019d0 0x7fa7ac198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b1cb7640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac1007e0 0x7fa7ac197ac0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800e9c0 tx=0x7fa7a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7a2ffd640 1 -- 172.21.15.105:0/3129508937 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800cde0 con 0x7fa7ac1007e0 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7a2ffd640 1 -- 172.21.15.105:0/3129508937 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7a80108e0 con 0x7fa7ac1007e0 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ac19f9d0 con 0x7fa7ac1007e0 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7a2ffd640 1 -- 172.21.15.105:0/3129508937 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8018990 con 0x7fa7ac1007e0 2024-03-26T17:40:36.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.650+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ac19ff20 con 0x7fa7ac1007e0 2024-03-26T17:40:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.651+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ac1a0150 con 0x7fa7ac1007e0 2024-03-26T17:40:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.653+0000 7fa7a2ffd640 1 -- 172.21.15.105:0/3129508937 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa7a801f020 con 0x7fa7ac1007e0 2024-03-26T17:40:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.653+0000 7fa7a2ffd640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7980a2f20 0x7fa7980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:36.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.654+0000 7fa7b14b6640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7980a2f20 0x7fa7980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.654+0000 7fa7a2ffd640 1 -- 172.21.15.105:0/3129508937 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa7a80cfbe0 con 0x7fa7ac1007e0 2024-03-26T17:40:36.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.654+0000 7fa7b14b6640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7980a2f20 0x7fa7980a53e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0095f0 tx=0x7fa79c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:36.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.658+0000 7fa7a2ffd640 1 -- 172.21.15.105:0/3129508937 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa7a809c9d0 con 0x7fa7ac1007e0 2024-03-26T17:40:36.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.935+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7ac1a0320 con 0x7fa7ac1007e0 2024-03-26T17:40:36.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.936+0000 7fa7a2ffd640 1 -- 172.21.15.105:0/3129508937 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7fa7a809c2d0 con 0x7fa7ac1007e0 2024-03-26T17:40:36.928 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:40:36.929 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:39:34.702865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_15879":{"gid":15879,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/2307931519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2307931519},{"type":"v1","addr":"172.21.15.135:6835","nonce":2307931519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:40:36.929 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 29 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.938+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7980a2f20 msgr2=0x7fa7980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.938+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7980a2f20 0x7fa7980a53e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0095f0 tx=0x7fa79c005f90 comp rx=0 tx=0).stop 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.938+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac1007e0 msgr2=0x7fa7ac197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.938+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac1007e0 0x7fa7ac197ac0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa7a800e9c0 tx=0x7fa7a800ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.938+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 shutdown_connections 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.939+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7980a2f20 0x7fa7980a53e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.939+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7ac102390 0x7fa7ac19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.939+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7ac1019d0 0x7fa7ac198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.939+0000 7fa7b3f1b640 1 --2- 172.21.15.105:0/3129508937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7ac1007e0 0x7fa7ac197ac0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.939+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 >> 172.21.15.105:0/3129508937 conn(0x7fa7ac0fbd50 msgr2=0x7fa7ac0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.939+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 shutdown_connections 2024-03-26T17:40:36.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:36.939+0000 7fa7b3f1b640 1 -- 172.21.15.105:0/3129508937 wait complete. 2024-03-26T17:40:36.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:40:37.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 47797 3.44 MB/sec warmup 80 sec latency 19.214 ms 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.071+0000 7f13de990640 1 -- 172.21.15.105:0/3871849880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d81007c0 msgr2=0x7f13d8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.071+0000 7f13de990640 1 --2- 172.21.15.105:0/3871849880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d81007c0 0x7f13d8100be0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009900 tx=0x7f13c802bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.071+0000 7f13de990640 1 -- 172.21.15.105:0/3871849880 shutdown_connections 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.071+0000 7f13de990640 1 --2- 172.21.15.105:0/3871849880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13d8102370 0x7f13d8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.071+0000 7f13de990640 1 --2- 172.21.15.105:0/3871849880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13d81019b0 0x7f13d8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.071+0000 7f13de990640 1 --2- 172.21.15.105:0/3871849880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d81007c0 0x7f13d8100be0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.071+0000 7f13de990640 1 -- 172.21.15.105:0/3871849880 >> 172.21.15.105:0/3871849880 conn(0x7f13d80fbd50 msgr2=0x7f13d80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 -- 172.21.15.105:0/3871849880 shutdown_connections 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 -- 172.21.15.105:0/3871849880 wait complete. 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 Processor -- start 2024-03-26T17:40:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 -- start start 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13d81007c0 0x7f13d8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13d81019b0 0x7f13d81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d8102370 0x7f13d819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13d810c2e0 con 0x7f13d8102370 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13d810c160 con 0x7f13d81007c0 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.072+0000 7f13de990640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13d810c460 con 0x7f13d81019b0 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13dcf2d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d8102370 0x7f13d819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13d81007c0 0x7f13d8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13d81019b0 0x7f13d81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13d81007c0 0x7f13d8197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43986/0 (socket says 172.21.15.105:43986) 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13dcf2d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d8102370 0x7f13d819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58916/0 (socket says 172.21.15.105:58916) 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13d81019b0 0x7f13d81980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41670/0 (socket says 172.21.15.105:41670) 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d7fff640 1 -- 172.21.15.105:0/1672811346 learned_addr learned my addr 172.21.15.105:0/1672811346 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d77fe640 1 -- 172.21.15.105:0/1672811346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13d81007c0 msgr2=0x7f13d8197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d77fe640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13d81007c0 0x7f13d8197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d77fe640 1 -- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d8102370 msgr2=0x7f13d819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d77fe640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d8102370 0x7f13d819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d77fe640 1 -- 172.21.15.105:0/1672811346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d819f780 con 0x7f13d81019b0 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d77fe640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13d81019b0 0x7f13d81980a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00e9f0 tx=0x7f13cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d57fa640 1 -- 172.21.15.105:0/1672811346 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00ccc0 con 0x7f13d81019b0 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13de990640 1 -- 172.21.15.105:0/1672811346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d819fa70 con 0x7f13d81019b0 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d57fa640 1 -- 172.21.15.105:0/1672811346 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13cc005020 con 0x7f13d81019b0 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13d57fa640 1 -- 172.21.15.105:0/1672811346 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc004a00 con 0x7f13d81019b0 2024-03-26T17:40:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.073+0000 7f13de990640 1 -- 172.21.15.105:0/1672811346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d819ffc0 con 0x7f13d81019b0 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.075+0000 7f13d57fa640 1 -- 172.21.15.105:0/1672811346 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f13cc004ba0 con 0x7f13d81019b0 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.076+0000 7f13d57fa640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13c00a2f20 0x7f13c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.076+0000 7f13d7fff640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13c00a2f20 0x7f13c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.076+0000 7f13d57fa640 1 -- 172.21.15.105:0/1672811346 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f13cc0cf3f0 con 0x7f13d81019b0 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d7fff640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13c00a2f20 0x7f13c00a53e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f13c802c840 tx=0x7f13c80095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13e1577740 1 Processor -- start 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13e1577740 1 -- start start 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13e1577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7da30 0x55e35dd7de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13e1577740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13e1577740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 0x55e35dd85a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13e1577740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e35dce2970 con 0x55e35dd7e420 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13e1577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e35dc8ad70 con 0x55e35dd7da30 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13e1577740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e35dcde920 con 0x55e35dd83590 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13dcf2d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 0x55e35dd85a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd82f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58918/0 (socket says 172.21.15.105:58918) 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7da30 0x55e35dd7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d77fe640 1 -- 172.21.15.105:0/276061783 learned_addr learned my addr 172.21.15.105:0/276061783 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d77fe640 1 -- 172.21.15.105:0/276061783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 msgr2=0x55e35dd85a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d77fe640 1 --2- 172.21.15.105:0/276061783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 0x55e35dd85a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d77fe640 1 -- 172.21.15.105:0/276061783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7da30 msgr2=0x55e35dd7de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d77fe640 1 --2- 172.21.15.105:0/276061783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7da30 0x55e35dd7de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d77fe640 1 -- 172.21.15.105:0/276061783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e35d9f63f0 con 0x55e35dd7e420 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.077+0000 7f13d7fff640 1 --2- 172.21.15.105:0/276061783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7da30 0x55e35dd7de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.078+0000 7f13d77fe640 1 --2- 172.21.15.105:0/276061783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd82f20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f13cc098450 tx=0x7f13cc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c12850a37823d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.078+0000 7f13be7fc640 1 -- 172.21.15.105:0/276061783 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc01c040 con 0x55e35dd7e420 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.078+0000 7f13be7fc640 1 -- 172.21.15.105:0/276061783 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13cc0a4490 con 0x55e35dd7e420 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.078+0000 7f13be7fc640 1 -- 172.21.15.105:0/276061783 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0052f0 con 0x55e35dd7e420 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.078+0000 7f13e1577740 1 -- 172.21.15.105:0/276061783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 msgr2=0x55e35dd82f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.078+0000 7f13e1577740 1 --2- 172.21.15.105:0/276061783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd82f20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f13cc098450 tx=0x7f13cc009f60 comp rx=0 tx=0).stop 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 -- 172.21.15.105:0/276061783 shutdown_connections 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 --2- 172.21.15.105:0/276061783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 0x55e35dd85a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 --2- 172.21.15.105:0/276061783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd82f20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 --2- 172.21.15.105:0/276061783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7da30 0x55e35dd7de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 -- 172.21.15.105:0/276061783 >> 172.21.15.105:0/276061783 conn(0x55e35dd78fa0 msgr2=0x55e35dd793a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 -- 172.21.15.105:0/276061783 shutdown_connections 2024-03-26T17:40:37.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 -- 172.21.15.105:0/276061783 wait complete. 2024-03-26T17:40:37.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 -- start start 2024-03-26T17:40:37.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.079+0000 7f13e1577740 1 Processor -- start 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13e1577740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd8c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13e1577740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7e420 0x55e35dd8c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13e1577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13e1577740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e35dd8a950 con 0x55e35dd7da30 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13e1577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e35dc9e310 con 0x55e35dd83590 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13e1577740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e35dd8aad0 con 0x55e35dd7e420 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13e1577740 10 client.0 _ll_register_callbacks cb 0x55e35dcd9c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7e420 0x55e35dd8c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7e420 0x55e35dd8c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41702/0 (socket says 172.21.15.105:41702) 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d77fe640 1 -- 172.21.15.105:0/1575920198 learned_addr learned my addr 172.21.15.105:0/1575920198 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13dcf2d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d7fff640 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd8c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d77fe640 1 -- 172.21.15.105:0/1575920198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 msgr2=0x55e35dd938b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d77fe640 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd938b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d77fe640 1 -- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 msgr2=0x55e35dd8c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d77fe640 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd8c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d77fe640 1 -- 172.21.15.105:0/1575920198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e35dac07d0 con 0x55e35dd7e420 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.080+0000 7f13d7fff640 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd8c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13d77fe640 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7e420 0x55e35dd8c870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f13cc099fd0 tx=0x7f13cc099df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc2ea233427f1233 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13b77fe640 1 -- 172.21.15.105:0/1575920198 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc01c030 con 0x55e35dd7e420 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13b77fe640 1 -- 172.21.15.105:0/1575920198 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f13cc0a4000 con 0x55e35dd7e420 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 1 -- 172.21.15.105:0/1575920198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13cc0a4490 con 0x55e35dd7e420 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13b77fe640 1 -- 172.21.15.105:0/1575920198 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13cc0ba050 con 0x55e35dd7e420 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 10 client.25987 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 1 -- 172.21.15.105:0/1575920198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e35dd8a950 con 0x55e35dd7e420 2024-03-26T17:40:37.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13b77fe640 1 -- 172.21.15.105:0/1575920198 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13cc0a4490 con 0x55e35dd7e420 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13b77fe640 1 -- 172.21.15.105:0/1575920198 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13cc09a940 con 0x55e35dd7e420 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 10 client.25987 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 10 client.25987 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 20 client.25987 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e35dd97e40 0x55e35dd9a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 4 client.25987 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.081+0000 7f13e1577740 1 -- 172.21.15.105:0/1575920198 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e35dcefeb0 con 0x55e35dd97e40 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.082+0000 7f13d7fff640 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e35dd97e40 0x55e35dd9a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.082+0000 7f13d7fff640 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e35dd97e40 0x55e35dd9a250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6b26e991e301269 server_cookie=b3814c7b8d861384 in_seq=0 out_seq=0 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.082+0000 7f13b77fe640 10 client.25987 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13b77fe640 1 -- 172.21.15.105:0/1575920198 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e35dcefeb0 con 0x55e35dd97e40 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13b77fe640 10 client.25987 handle_command_reply: tid=0 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 client.25987 shutdown 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 -- 172.21.15.105:0/1575920198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7e420 msgr2=0x55e35dd8c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7e420 0x55e35dd8c870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f13cc099fd0 tx=0x7f13cc099df0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 -- 172.21.15.105:0/1575920198 shutdown_connections 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e35dd97e40 0x55e35dd9a250 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd938b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13b77fe640 0 client.25987 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7e420 0x55e35dd8c870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 --2- 172.21.15.105:0/1575920198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd8c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 -- 172.21.15.105:0/1575920198 >> 172.21.15.105:0/1575920198 conn(0x55e35dd78fa0 msgr2=0x55e35dd781c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 -- 172.21.15.105:0/1575920198 shutdown_connections 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 1 -- 172.21.15.105:0/1575920198 wait complete. 2024-03-26T17:40:37.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.083+0000 7f13e1577740 20 client.25987 trim_cache size 0 max 16384 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13e1577740 1 Processor -- start 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13e1577740 1 -- start start 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13e1577740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7da30 0x55e35dd77930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13e1577740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd93870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13e1577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd77e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13e1577740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81a6c40 con 0x55e35dd7e420 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13e1577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81a8780 con 0x55e35dd83590 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13e1577740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81a7310 con 0x55e35dd7da30 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7da30 0x55e35dd77930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd93870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13dcf2d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd77e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7da30 0x55e35dd77930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41716/0 (socket says 172.21.15.105:41716) 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13d7fff640 1 -- 172.21.15.105:0/2497529197 learned_addr learned my addr 172.21.15.105:0/2497529197 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.099+0000 7f13d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd93870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58946/0 (socket says 172.21.15.105:58946) 2024-03-26T17:40:37.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13dcf2d640 1 -- 172.21.15.105:0/2497529197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7da30 msgr2=0x55e35dd77930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13dcf2d640 1 --2- 172.21.15.105:0/2497529197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7da30 0x55e35dd77930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13dcf2d640 1 -- 172.21.15.105:0/2497529197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 msgr2=0x55e35dd93870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13dcf2d640 1 --2- 172.21.15.105:0/2497529197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd93870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13dcf2d640 1 -- 172.21.15.105:0/2497529197 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e35dd82bd0 con 0x55e35dd83590 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13d77fe640 1 --2- 172.21.15.105:0/2497529197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd93870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13dcf2d640 1 --2- 172.21.15.105:0/2497529197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd77e70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f13d0005fd0 tx=0x7f13d0004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9613bd637d775e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13bdffb640 1 -- 172.21.15.105:0/2497529197 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0004b20 con 0x55e35dd83590 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13bdffb640 1 -- 172.21.15.105:0/2497529197 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13d00108e0 con 0x55e35dd83590 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.100+0000 7f13bdffb640 1 -- 172.21.15.105:0/2497529197 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d001b890 con 0x55e35dd83590 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 -- 172.21.15.105:0/2497529197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 msgr2=0x55e35dd77e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 --2- 172.21.15.105:0/2497529197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd77e70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f13d0005fd0 tx=0x7f13d0004410 comp rx=0 tx=0).stop 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 -- 172.21.15.105:0/2497529197 shutdown_connections 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 --2- 172.21.15.105:0/2497529197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd83590 0x55e35dd77e70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 --2- 172.21.15.105:0/2497529197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7e420 0x55e35dd93870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 --2- 172.21.15.105:0/2497529197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd7da30 0x55e35dd77930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 -- 172.21.15.105:0/2497529197 >> 172.21.15.105:0/2497529197 conn(0x55e35dd78fa0 msgr2=0x55e35dd7ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 -- 172.21.15.105:0/2497529197 shutdown_connections 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 -- 172.21.15.105:0/2497529197 wait complete. 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.101+0000 7f13e1577740 1 -- start start 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13e1577740 1 Processor -- start 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13e1577740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd90e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13e1577740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7e420 0x55e35dd91360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13e1577740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 0x55e35dd92300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13e1577740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81a80b0 con 0x55e35dd7da30 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13e1577740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81a79e0 con 0x55e35dd7e420 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13e1577740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13d81a4c20 con 0x55e35dd83590 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13e1577740 10 client.0 _ll_register_callbacks cb 0x7f13cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd90e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd90e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58962/0 (socket says 172.21.15.105:58962) 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d7fff640 1 -- 172.21.15.105:0/2794843364 learned_addr learned my addr 172.21.15.105:0/2794843364 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d77fe640 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7e420 0x55e35dd91360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13dcf2d640 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 0x55e35dd92300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d7fff640 1 -- 172.21.15.105:0/2794843364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 msgr2=0x55e35dd92300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d7fff640 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 0x55e35dd92300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d7fff640 1 -- 172.21.15.105:0/2794843364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7e420 msgr2=0x55e35dd91360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d7fff640 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7e420 0x55e35dd91360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d7fff640 1 -- 172.21.15.105:0/2794843364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e35dd928f0 con 0x55e35dd7da30 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.102+0000 7f13d77fe640 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7e420 0x55e35dd91360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13d7fff640 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd90e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f13c80042e0 tx=0x7f13c802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0a211e75e2fa56 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13b77fe640 1 -- 172.21.15.105:0/2794843364 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c805f060 con 0x55e35dd7da30 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13b77fe640 1 -- 172.21.15.105:0/2794843364 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f13c80046c0 con 0x55e35dd7da30 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 1 -- 172.21.15.105:0/2794843364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d00108e0 con 0x55e35dd7da30 2024-03-26T17:40:37.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13b77fe640 1 -- 172.21.15.105:0/2794843364 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f13c80048c0 con 0x55e35dd7da30 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 10 client.16140 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 1 -- 172.21.15.105:0/2794843364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e35ddfe150 con 0x55e35dd7da30 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13b77fe640 1 -- 172.21.15.105:0/2794843364 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13d00108e0 con 0x55e35dd7da30 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13b77fe640 1 -- 172.21.15.105:0/2794843364 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13c805a480 con 0x55e35dd7da30 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 10 client.16140 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 10 client.16140 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 20 client.16140 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e35ddfe9e0 0x55e35de00e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 4 client.16140 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.103+0000 7f13e1577740 1 -- 172.21.15.105:0/2794843364 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e35dcea200 con 0x55e35ddfe9e0 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13d77fe640 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e35ddfe9e0 0x55e35de00e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13b77fe640 10 client.16140 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13d77fe640 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e35ddfe9e0 0x55e35de00e00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d665e8ee1fae225 server_cookie=ec690b4e172c421 in_seq=0 out_seq=0 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13b77fe640 1 -- 172.21.15.105:0/2794843364 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e35dcea200 con 0x55e35ddfe9e0 2024-03-26T17:40:37.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13b77fe640 10 client.16140 handle_command_reply: tid=0 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 client.16140 shutdown 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 -- 172.21.15.105:0/2794843364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 msgr2=0x55e35dd90e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd90e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f13c80042e0 tx=0x7f13c802bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 -- 172.21.15.105:0/2794843364 shutdown_connections 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e35ddfe9e0 0x55e35de00e00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e35dd83590 0x55e35dd92300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e35dd7e420 0x55e35dd91360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13b77fe640 0 client.16140 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 --2- 172.21.15.105:0/2794843364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e35dd7da30 0x55e35dd90e20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 -- 172.21.15.105:0/2794843364 >> 172.21.15.105:0/2794843364 conn(0x55e35dd78fa0 msgr2=0x55e35dd8ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 -- 172.21.15.105:0/2794843364 shutdown_connections 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.104+0000 7f13e1577740 1 -- 172.21.15.105:0/2794843364 wait complete. 2024-03-26T17:40:37.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13e1577740 20 client.16140 trim_cache size 0 max 16384 2024-03-26T17:40:37.103 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 -- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13c00a2f20 msgr2=0x7f13c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13c00a2f20 0x7f13c00a53e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f13c802c840 tx=0x7f13c80095a0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 -- 172.21.15.105:0/1672811346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13d81019b0 msgr2=0x7f13d81980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13d81019b0 0x7f13d81980a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00e9f0 tx=0x7f13cc00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 -- 172.21.15.105:0/1672811346 shutdown_connections 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13c00a2f20 0x7f13c00a53e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d8102370 0x7f13d819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13d81019b0 0x7f13d81980a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 --2- 172.21.15.105:0/1672811346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13d81007c0 0x7f13d8197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 -- 172.21.15.105:0/1672811346 >> 172.21.15.105:0/1672811346 conn(0x7f13d80fbd50 msgr2=0x7f13d8068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 -- 172.21.15.105:0/1672811346 shutdown_connections 2024-03-26T17:40:37.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.105+0000 7f13de990640 1 -- 172.21.15.105:0/1672811346 wait complete. 2024-03-26T17:40:37.109 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:40:37.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 -- 172.21.15.105:0/3485160081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01019b0 msgr2=0x7fa4f0101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3485160081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01019b0 0x7fa4f0101e30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009860 tx=0x7fa4d802bb70 comp rx=0 tx=0).stop 2024-03-26T17:40:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 -- 172.21.15.105:0/3485160081 shutdown_connections 2024-03-26T17:40:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3485160081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f0102370 0x7fa4f0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3485160081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01019b0 0x7fa4f0101e30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3485160081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f01007c0 0x7fa4f0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 -- 172.21.15.105:0/3485160081 >> 172.21.15.105:0/3485160081 conn(0x7fa4f00fbd50 msgr2=0x7fa4f00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 -- 172.21.15.105:0/3485160081 shutdown_connections 2024-03-26T17:40:37.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.232+0000 7fa4f4962640 1 -- 172.21.15.105:0/3485160081 wait complete. 2024-03-26T17:40:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4f4962640 1 Processor -- start 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4f4962640 1 -- start start 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4f4962640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01007c0 0x7fa4f00695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4f4962640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f01019b0 0x7fa4f019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4f4962640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f0102370 0x7fa4f019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4f4962640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f010c2d0 con 0x7fa4f0102370 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4f4962640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f010c150 con 0x7fa4f01019b0 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4f4962640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f010c450 con 0x7fa4f01007c0 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4ee59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01007c0 0x7fa4f00695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f0102370 0x7fa4f019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4ee59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01007c0 0x7fa4f00695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41736/0 (socket says 172.21.15.105:41736) 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f01019b0 0x7fa4f019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4ee59c640 1 -- 172.21.15.105:0/3947094784 learned_addr learned my addr 172.21.15.105:0/3947094784 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4ee59c640 1 -- 172.21.15.105:0/3947094784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f01019b0 msgr2=0x7fa4f019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f01019b0 0x7fa4f019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4ee59c640 1 -- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f0102370 msgr2=0x7fa4f019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f0102370 0x7fa4f019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4ee59c640 1 -- 172.21.15.105:0/3947094784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f019f730 con 0x7fa4f01007c0 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.233+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f01019b0 0x7fa4f019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.234+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01007c0 0x7fa4f00695f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000e9f0 tx=0x7fa4e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.234+0000 7fa4e77fe640 1 -- 172.21.15.105:0/3947094784 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000ccc0 con 0x7fa4f01007c0 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.234+0000 7fa4e77fe640 1 -- 172.21.15.105:0/3947094784 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4e00035f0 con 0x7fa4f01007c0 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.234+0000 7fa4f4962640 1 -- 172.21.15.105:0/3947094784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f019fa20 con 0x7fa4f01007c0 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.234+0000 7fa4e77fe640 1 -- 172.21.15.105:0/3947094784 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000ce60 con 0x7fa4f01007c0 2024-03-26T17:40:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.234+0000 7fa4f4962640 1 -- 172.21.15.105:0/3947094784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f019ff90 con 0x7fa4f01007c0 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.236+0000 7fa4e77fe640 1 -- 172.21.15.105:0/3947094784 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa4e00037a0 con 0x7fa4f01007c0 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4f762b740 1 Processor -- start 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4f762b740 1 -- start start 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e5a30 0x559c4d1e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e6420 0x559c4d1eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1eda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4f762b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559c4d14a970 con 0x559c4d1eb590 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4f762b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559c4d0f2d70 con 0x559c4d1e6420 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4f762b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559c4d146920 con 0x559c4d1e5a30 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4e77fe640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4dc0a2fa0 0x7fa4dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e5a30 0x559c4d1e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e6420 0x559c4d1eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1eda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58988/0 (socket says 172.21.15.105:58988) 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4e7fff640 1 -- 172.21.15.105:0/2010055726 learned_addr learned my addr 172.21.15.105:0/2010055726 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4eed9d640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4dc0a2fa0 0x7fa4dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.237+0000 7fa4e77fe640 1 -- 172.21.15.105:0/3947094784 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa4e00cf580 con 0x7fa4f01007c0 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4ee59c640 1 -- 172.21.15.105:0/2010055726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e5a30 msgr2=0x559c4d1e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/2010055726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e5a30 0x559c4d1e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4ee59c640 1 -- 172.21.15.105:0/2010055726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 msgr2=0x559c4d1eda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/2010055726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1eda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4ee59c640 1 -- 172.21.15.105:0/2010055726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4ce5e3f0 con 0x559c4d1e6420 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/2010055726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1eda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/2010055726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e6420 0x559c4d1eaf20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4e009bc00 tx=0x7fa4e0091150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c36a1131c38ef53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4eed9d640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4dc0a2fa0 0x7fa4dc0a5460 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x559c4d1e6c80 tx=0x7fa4e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4e4ff9640 1 -- 172.21.15.105:0/2010055726 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e009b5d0 con 0x559c4d1e6420 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4e4ff9640 1 -- 172.21.15.105:0/2010055726 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4e0003b10 con 0x559c4d1e6420 2024-03-26T17:40:37.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.238+0000 7fa4e4ff9640 1 -- 172.21.15.105:0/2010055726 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00982e0 con 0x559c4d1e6420 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 -- 172.21.15.105:0/2010055726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e6420 msgr2=0x559c4d1eaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2010055726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e6420 0x559c4d1eaf20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4e009bc00 tx=0x7fa4e0091150 comp rx=0 tx=0).stop 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 -- 172.21.15.105:0/2010055726 shutdown_connections 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2010055726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1eda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2010055726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e6420 0x559c4d1eaf20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2010055726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e5a30 0x559c4d1e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 -- 172.21.15.105:0/2010055726 >> 172.21.15.105:0/2010055726 conn(0x559c4d1e0fa0 msgr2=0x559c4d1e13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 -- 172.21.15.105:0/2010055726 shutdown_connections 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.239+0000 7fa4f762b740 1 -- 172.21.15.105:0/2010055726 wait complete. 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 1 -- start start 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 1 Processor -- start 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1eff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559c4d1ee4e0 con 0x559c4d1eb590 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559c4d106310 con 0x559c4d1e5a30 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559c4d1ee660 con 0x559c4d1e6420 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4f762b740 10 client.0 _ll_register_callbacks cb 0x559c4d141c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59002/0 (socket says 172.21.15.105:59002) 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4eed9d640 1 -- 172.21.15.105:0/2019857613 learned_addr learned my addr 172.21.15.105:0/2019857613 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41762/0 (socket says 172.21.15.105:41762) 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4eed9d640 1 -- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 msgr2=0x559c4d1f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4eed9d640 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4eed9d640 1 -- 172.21.15.105:0/2019857613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 msgr2=0x559c4d1eff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4eed9d640 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1eff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.240+0000 7fa4eed9d640 1 -- 172.21.15.105:0/2019857613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4cf287d0 con 0x559c4d1eb590 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4eed9d640 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f7490 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8011ee0 tx=0x7fa4e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bcb5ab1185550d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/2019857613 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e801e410 con 0x559c4d1eb590 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/2019857613 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa4e801e5b0 con 0x559c4d1eb590 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4f762b740 1 -- 172.21.15.105:0/2019857613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c4d157eb0 con 0x559c4d1eb590 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/2019857613 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4e801e7b0 con 0x559c4d1eb590 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4f762b740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4f762b740 1 -- 172.21.15.105:0/2019857613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c4d1fa870 con 0x559c4d1eb590 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.241+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/2019857613 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559c4d157eb0 con 0x559c4d1eb590 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/2019857613 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa4e801d070 con 0x559c4d1eb590 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4f762b740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4f762b740 10 client.16152 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4f762b740 20 client.16152 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559c4d1fc000 0x559c4d1fe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4f762b740 4 client.16152 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4f762b740 1 -- 172.21.15.105:0/2019857613 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c4d152200 con 0x559c4d1fc000 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559c4d1fc000 0x559c4d1fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4d5ffb640 10 client.16152 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.242+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559c4d1fc000 0x559c4d1fe460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39b2c9fd2b0bda43 server_cookie=239b26bd2fac9174 in_seq=0 out_seq=0 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/2019857613 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559c4d152200 con 0x559c4d1fc000 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4d5ffb640 10 client.16152 handle_command_reply: tid=0 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 client.16152 shutdown 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 -- 172.21.15.105:0/2019857613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 msgr2=0x559c4d1f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f7490 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8011ee0 tx=0x7fa4e8004ab0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 -- 172.21.15.105:0/2019857613 shutdown_connections 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559c4d1fc000 0x559c4d1fe460 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f7490 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4d5ffb640 0 client.16152 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:40:37.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 --2- 172.21.15.105:0/2019857613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1eff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 -- 172.21.15.105:0/2019857613 >> 172.21.15.105:0/2019857613 conn(0x559c4d1e0fa0 msgr2=0x559c4d1e01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 -- 172.21.15.105:0/2019857613 shutdown_connections 2024-03-26T17:40:37.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 1 -- 172.21.15.105:0/2019857613 wait complete. 2024-03-26T17:40:37.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.243+0000 7fa4f762b740 20 client.16152 trim_cache size 0 max 16384 2024-03-26T17:40:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4f762b740 1 Processor -- start 2024-03-26T17:40:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4f762b740 1 -- start start 2024-03-26T17:40:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1e7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1e6420 0x559c4d1e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1eb590 0x559c4d1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4f762b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01a8dc0 con 0x559c4d1e6420 2024-03-26T17:40:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4f762b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01aa900 con 0x559c4d1e5a30 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4f762b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01a9490 con 0x559c4d1eb590 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1e6420 0x559c4d1e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1e6420 0x559c4d1e7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59016/0 (socket says 172.21.15.105:59016) 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4e7fff640 1 -- 172.21.15.105:0/4134601382 learned_addr learned my addr 172.21.15.105:0/4134601382 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4eed9d640 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1eb590 0x559c4d1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4e7fff640 1 -- 172.21.15.105:0/4134601382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1eb590 msgr2=0x559c4d1f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1eb590 0x559c4d1f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.259+0000 7fa4e7fff640 1 -- 172.21.15.105:0/4134601382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 msgr2=0x559c4d1e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.260+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.260+0000 7fa4e7fff640 1 -- 172.21.15.105:0/4134601382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4d1dc820 con 0x559c4d1e6420 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.260+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1e6420 0x559c4d1e7e90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8000c00 tx=0x7fa4d8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6063ecc5028eb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.260+0000 7fa4d7fff640 1 -- 172.21.15.105:0/4134601382 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8043030 con 0x559c4d1e6420 2024-03-26T17:40:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.260+0000 7fa4d7fff640 1 -- 172.21.15.105:0/4134601382 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d802c9c0 con 0x559c4d1e6420 2024-03-26T17:40:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.260+0000 7fa4d7fff640 1 -- 172.21.15.105:0/4134601382 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8039040 con 0x559c4d1e6420 2024-03-26T17:40:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 -- 172.21.15.105:0/4134601382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1e6420 msgr2=0x559c4d1e7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1e6420 0x559c4d1e7e90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8000c00 tx=0x7fa4d8004040 comp rx=0 tx=0).stop 2024-03-26T17:40:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 -- 172.21.15.105:0/4134601382 shutdown_connections 2024-03-26T17:40:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1eb590 0x559c4d1f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1e6420 0x559c4d1e7e90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 --2- 172.21.15.105:0/4134601382 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 -- 172.21.15.105:0/4134601382 >> 172.21.15.105:0/4134601382 conn(0x559c4d1e0fa0 msgr2=0x559c4d1eea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 -- 172.21.15.105:0/4134601382 shutdown_connections 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 -- 172.21.15.105:0/4134601382 wait complete. 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 -- start start 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.261+0000 7fa4f762b740 1 Processor -- start 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4f762b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4f762b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01aa230 con 0x559c4d1eb590 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4f762b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01a9b60 con 0x559c4d1e5a30 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4f762b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01a6da0 con 0x559c4d1e6420 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4f762b740 10 client.0 _ll_register_callbacks cb 0x7fa4e8011da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44074/0 (socket says 172.21.15.105:44074) 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4e7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41784/0 (socket says 172.21.15.105:41784) 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4eed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59018/0 (socket says 172.21.15.105:59018) 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 -- 172.21.15.105:0/3200556491 learned_addr learned my addr 172.21.15.105:0/3200556491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 -- 172.21.15.105:0/3200556491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 msgr2=0x559c4d1f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 -- 172.21.15.105:0/3200556491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 msgr2=0x559c4d1f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 -- 172.21.15.105:0/3200556491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4d1f4600 con 0x559c4d1e5a30 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4eed9d640 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4ee59c640 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1f7090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00cd090 tx=0x7fa4e00cf780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5296abf0798b2055 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/3200556491 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e009c370 con 0x559c4d1e5a30 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/3200556491 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fa4e0096440 con 0x559c4d1e5a30 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.262+0000 7fa4f762b740 1 -- 172.21.15.105:0/3200556491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d802c9c0 con 0x559c4d1e5a30 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/3200556491 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4e00982e0 con 0x559c4d1e5a30 2024-03-26T17:40:37.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4f762b740 10 client.26015 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4f762b740 1 -- 172.21.15.105:0/3200556491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c4d261350 con 0x559c4d1e5a30 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/3200556491 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d802c9c0 con 0x559c4d1e5a30 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/3200556491 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa4e00a41f0 con 0x559c4d1e5a30 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4f762b740 10 client.26015 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4f762b740 10 client.26015 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4f762b740 20 client.26015 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4f762b740 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559c4d1fc000 0x559c4d263840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4f762b740 4 client.26015 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4f762b740 1 -- 172.21.15.105:0/3200556491 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c4d0cf6a0 con 0x559c4d1fc000 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559c4d1fc000 0x559c4d263840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.263+0000 7fa4d5ffb640 10 client.26015 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4e7fff640 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559c4d1fc000 0x559c4d263840 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=557e59e35e86c48e server_cookie=b2d1bf7c85a0d41a in_seq=0 out_seq=0 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4d5ffb640 1 -- 172.21.15.105:0/3200556491 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c4d0cf6a0 con 0x559c4d1fc000 2024-03-26T17:40:37.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4d5ffb640 10 client.26015 handle_command_reply: tid=0 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 client.26015 shutdown 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 -- 172.21.15.105:0/3200556491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 msgr2=0x559c4d1f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1f7090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00cd090 tx=0x7fa4e00cf780 comp rx=0 tx=0).stop 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 -- 172.21.15.105:0/3200556491 shutdown_connections 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559c4d1fc000 0x559c4d263840 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c4d1eb590 0x559c4d1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c4d1e6420 0x559c4d1f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 --2- 172.21.15.105:0/3200556491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c4d1e5a30 0x559c4d1f7090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4d5ffb640 0 client.26015 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 -- 172.21.15.105:0/3200556491 >> 172.21.15.105:0/3200556491 conn(0x559c4d1e0fa0 msgr2=0x559c4d1eff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 -- 172.21.15.105:0/3200556491 shutdown_connections 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 1 -- 172.21.15.105:0/3200556491 wait complete. 2024-03-26T17:40:37.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.264+0000 7fa4f762b740 20 client.26015 trim_cache size 0 max 16384 2024-03-26T17:40:37.263 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 -- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4dc0a2fa0 msgr2=0x7fa4dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4dc0a2fa0 0x7fa4dc0a5460 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x559c4d1e6c80 tx=0x7fa4e8009500 comp rx=0 tx=0).stop 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 -- 172.21.15.105:0/3947094784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01007c0 msgr2=0x7fa4f00695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01007c0 0x7fa4f00695f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000e9f0 tx=0x7fa4e000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 -- 172.21.15.105:0/3947094784 shutdown_connections 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa4dc0a2fa0 0x7fa4dc0a5460 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4f0102370 0x7fa4f019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f01019b0 0x7fa4f019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 --2- 172.21.15.105:0/3947094784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4f01007c0 0x7fa4f00695f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 -- 172.21.15.105:0/3947094784 >> 172.21.15.105:0/3947094784 conn(0x7fa4f00fbd50 msgr2=0x7fa4f0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 -- 172.21.15.105:0/3947094784 shutdown_connections 2024-03-26T17:40:37.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.265+0000 7fa4f4962640 1 -- 172.21.15.105:0/3947094784 wait complete. 2024-03-26T17:40:37.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 -- 172.21.15.105:0/1056310143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2908102370 msgr2=0x7f2908108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 --2- 172.21.15.105:0/1056310143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2908102370 0x7f2908108c00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2900009ca0 tx=0x7f290002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 -- 172.21.15.105:0/1056310143 shutdown_connections 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 --2- 172.21.15.105:0/1056310143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2908102370 0x7f2908108c00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 --2- 172.21.15.105:0/1056310143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29081019b0 0x7f2908101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 --2- 172.21.15.105:0/1056310143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29081007c0 0x7f2908100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 -- 172.21.15.105:0/1056310143 >> 172.21.15.105:0/1056310143 conn(0x7f29080fbd50 msgr2=0x7f29080fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 -- 172.21.15.105:0/1056310143 shutdown_connections 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.402+0000 7f290e820640 1 -- 172.21.15.105:0/1056310143 wait complete. 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290e820640 1 Processor -- start 2024-03-26T17:40:37.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290e820640 1 -- start start 2024-03-26T17:40:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290e820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29081007c0 0x7f2908069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290e820640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29081019b0 0x7f290819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290e820640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2908102370 0x7f290819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290e820640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f290810c2f0 con 0x7f29081007c0 2024-03-26T17:40:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290e820640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f290810c170 con 0x7f29081019b0 2024-03-26T17:40:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290e820640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f290810c470 con 0x7f2908102370 2024-03-26T17:40:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f2907fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29081007c0 0x7f2908069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f2907fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29081007c0 0x7f2908069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59032/0 (socket says 172.21.15.105:59032) 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f2907fff640 1 -- 172.21.15.105:0/2542172461 learned_addr learned my addr 172.21.15.105:0/2542172461 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f29077fe640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29081019b0 0x7f290819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f290cdbd640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2908102370 0x7f290819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f2907fff640 1 -- 172.21.15.105:0/2542172461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2908102370 msgr2=0x7f290819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f2907fff640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2908102370 0x7f290819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f2907fff640 1 -- 172.21.15.105:0/2542172461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29081019b0 msgr2=0x7f290819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f2907fff640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29081019b0 0x7f290819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.403+0000 7f2907fff640 1 -- 172.21.15.105:0/2542172461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f290819f740 con 0x7f29081007c0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.404+0000 7f2907fff640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29081007c0 0x7f2908069590 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9c0 tx=0x7f28f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.404+0000 7f29057fa640 1 -- 172.21.15.105:0/2542172461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f800cd80 con 0x7f29081007c0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.404+0000 7f29057fa640 1 -- 172.21.15.105:0/2542172461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f80035f0 con 0x7f29081007c0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.404+0000 7f290e820640 1 -- 172.21.15.105:0/2542172461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f290819f9d0 con 0x7f29081007c0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.404+0000 7f29057fa640 1 -- 172.21.15.105:0/2542172461 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8010690 con 0x7f29081007c0 2024-03-26T17:40:37.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.404+0000 7f290e820640 1 -- 172.21.15.105:0/2542172461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f290819ff70 con 0x7f29081007c0 2024-03-26T17:40:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.406+0000 7f29057fa640 1 -- 172.21.15.105:0/2542172461 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f28f80037a0 con 0x7f29081007c0 2024-03-26T17:40:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.406+0000 7f291140d740 1 Processor -- start 2024-03-26T17:40:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f291140d740 1 -- start start 2024-03-26T17:40:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f291140d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbbbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f291140d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbc420 0x55fffdbc0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f291140d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 0x55fffdbc3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f291140d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fffdb20970 con 0x55fffdbbc420 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f291140d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fffdac8d70 con 0x55fffdbc1590 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f291140d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fffdb1c920 con 0x55fffdbbba30 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f2907fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbc420 0x55fffdbc0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f290cdbd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbbbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f2907fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbc420 0x55fffdbc0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59038/0 (socket says 172.21.15.105:59038) 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f2907fff640 1 -- 172.21.15.105:0/2231628080 learned_addr learned my addr 172.21.15.105:0/2231628080 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f29077fe640 1 --2- 172.21.15.105:0/2231628080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 0x55fffdbc3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f29057fa640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28f00a2fa0 0x7f28f00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f2907fff640 1 -- 172.21.15.105:0/2231628080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 msgr2=0x55fffdbbbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f2907fff640 1 --2- 172.21.15.105:0/2231628080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbbbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f2907fff640 1 -- 172.21.15.105:0/2231628080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 msgr2=0x55fffdbc3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f2907fff640 1 --2- 172.21.15.105:0/2231628080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 0x55fffdbc3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f2907fff640 1 -- 172.21.15.105:0/2231628080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffd8343f0 con 0x55fffdbbc420 2024-03-26T17:40:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.407+0000 7f290cdbd640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28f00a2fa0 0x7f28f00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.408+0000 7f29057fa640 1 -- 172.21.15.105:0/2542172461 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f28f80cf770 con 0x7f29081007c0 2024-03-26T17:40:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.408+0000 7f2907fff640 1 --2- 172.21.15.105:0/2231628080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbc420 0x55fffdbc0f20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f28f80829e0 tx=0x7f28f80669b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3817169c0b4680 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.408+0000 7f290cdbd640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28f00a2fa0 0x7f28f00a5460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2900006010 tx=0x7f290002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.408+0000 7f28ee7fc640 1 -- 172.21.15.105:0/2231628080 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f809b9a0 con 0x55fffdbbc420 2024-03-26T17:40:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.408+0000 7f28ee7fc640 1 -- 172.21.15.105:0/2231628080 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28f8003b30 con 0x55fffdbbc420 2024-03-26T17:40:37.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.408+0000 7f28ee7fc640 1 -- 172.21.15.105:0/2231628080 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f809b7c0 con 0x55fffdbbc420 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 -- 172.21.15.105:0/2231628080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbc420 msgr2=0x55fffdbc0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 --2- 172.21.15.105:0/2231628080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbc420 0x55fffdbc0f20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f28f80829e0 tx=0x7f28f80669b0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 -- 172.21.15.105:0/2231628080 shutdown_connections 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 --2- 172.21.15.105:0/2231628080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 0x55fffdbc3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 --2- 172.21.15.105:0/2231628080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbc420 0x55fffdbc0f20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 --2- 172.21.15.105:0/2231628080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbbbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 -- 172.21.15.105:0/2231628080 >> 172.21.15.105:0/2231628080 conn(0x55fffdbb6fa0 msgr2=0x55fffdbb73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 -- 172.21.15.105:0/2231628080 shutdown_connections 2024-03-26T17:40:37.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.409+0000 7f291140d740 1 -- 172.21.15.105:0/2231628080 wait complete. 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 1 -- start start 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 1 Processor -- start 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbc63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbcd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fffdbc4480 con 0x55fffdbc1590 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fffdadc310 con 0x55fffdbbc420 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fffdbc4600 con 0x55fffdbbba30 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f291140d740 10 client.0 _ll_register_callbacks cb 0x55fffdb17c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f2907fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f2907fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41818/0 (socket says 172.21.15.105:41818) 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f2907fff640 1 -- 172.21.15.105:0/4024924360 learned_addr learned my addr 172.21.15.105:0/4024924360 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f29077fe640 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbc63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.410+0000 7f290cdbd640 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbcd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f2907fff640 1 -- 172.21.15.105:0/4024924360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 msgr2=0x55fffdbc63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f2907fff640 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbc63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f2907fff640 1 -- 172.21.15.105:0/4024924360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 msgr2=0x55fffdbcd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f2907fff640 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbcd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f2907fff640 1 -- 172.21.15.105:0/4024924360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffd8fe7d0 con 0x55fffdbbba30 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f2907fff640 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc5e60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cd280 tx=0x7f28f80cd1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef651a05ef8e5f6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f28e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f28e77fe640 1 -- 172.21.15.105:0/4024924360 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f809b7c0 con 0x55fffdbbba30 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f28e77fe640 1 -- 172.21.15.105:0/4024924360 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f28f80850d0 con 0x55fffdbbba30 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f291140d740 1 -- 172.21.15.105:0/4024924360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fffdb2deb0 con 0x55fffdbbba30 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f28e77fe640 1 -- 172.21.15.105:0/4024924360 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28f809b9a0 con 0x55fffdbbba30 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f291140d740 10 client.26023 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f291140d740 1 -- 172.21.15.105:0/4024924360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fffdbc44f0 con 0x55fffdbbba30 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.411+0000 7f28e77fe640 1 -- 172.21.15.105:0/4024924360 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55fffdb2deb0 con 0x55fffdbbba30 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f28e77fe640 1 -- 172.21.15.105:0/4024924360 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f28f80a43e0 con 0x55fffdbbba30 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f291140d740 10 client.26023 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f291140d740 10 client.26023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f291140d740 20 client.26023 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f291140d740 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55fffdbd5d80 0x55fffdbd81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f291140d740 4 client.26023 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f291140d740 1 -- 172.21.15.105:0/4024924360 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fffdb28200 con 0x55fffdbd5d80 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f29077fe640 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55fffdbd5d80 0x55fffdbd81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f28e77fe640 10 client.26023 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.412+0000 7f29077fe640 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55fffdbd5d80 0x55fffdbd81f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf874cba19626f10 server_cookie=91636d4c67ae1e4a in_seq=0 out_seq=0 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f28e77fe640 1 -- 172.21.15.105:0/4024924360 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fffdb28200 con 0x55fffdbd5d80 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f28e77fe640 10 client.26023 handle_command_reply: tid=0 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 client.26023 shutdown 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 -- 172.21.15.105:0/4024924360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 msgr2=0x55fffdbc5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc5e60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28f80cd280 tx=0x7f28f80cd1a0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 -- 172.21.15.105:0/4024924360 shutdown_connections 2024-03-26T17:40:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbcd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55fffdbd5d80 0x55fffdbd81f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbc63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 --2- 172.21.15.105:0/4024924360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc5e60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f28e77fe640 0 client.26023 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:40:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.413+0000 7f291140d740 1 -- 172.21.15.105:0/4024924360 >> 172.21.15.105:0/4024924360 conn(0x55fffdbb6fa0 msgr2=0x55fffdbb61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.414+0000 7f291140d740 1 -- 172.21.15.105:0/4024924360 shutdown_connections 2024-03-26T17:40:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.414+0000 7f291140d740 1 -- 172.21.15.105:0/4024924360 wait complete. 2024-03-26T17:40:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.414+0000 7f291140d740 20 client.26023 trim_cache size 0 max 16384 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.428+0000 7f291140d740 1 Processor -- start 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.428+0000 7f291140d740 1 -- start start 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.428+0000 7f291140d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.428+0000 7f291140d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbcd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.428+0000 7f291140d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbdcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.428+0000 7f291140d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f29081a6cb0 con 0x55fffdbc1590 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.428+0000 7f291140d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29081a87f0 con 0x55fffdbbc420 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.428+0000 7f291140d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29081a7380 con 0x55fffdbbba30 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f290cdbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbdcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f29077fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbcd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f290cdbd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbdcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59054/0 (socket says 172.21.15.105:59054) 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f290cdbd640 1 -- 172.21.15.105:0/579741989 learned_addr learned my addr 172.21.15.105:0/579741989 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f2907fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f29077fe640 1 -- 172.21.15.105:0/579741989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 msgr2=0x55fffdbc60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f29077fe640 1 --2- 172.21.15.105:0/579741989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f29077fe640 1 -- 172.21.15.105:0/579741989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 msgr2=0x55fffdbdcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f29077fe640 1 --2- 172.21.15.105:0/579741989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbdcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f29077fe640 1 -- 172.21.15.105:0/579741989 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffdbd8870 con 0x55fffdbbc420 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f2907fff640 1 --2- 172.21.15.105:0/579741989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f29077fe640 1 --2- 172.21.15.105:0/579741989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbcd3a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55fffdbd8a40 tx=0x7f28fc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e575084c37170778 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f28edffb640 1 -- 172.21.15.105:0/579741989 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc01f050 con 0x55fffdbbc420 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f28edffb640 1 -- 172.21.15.105:0/579741989 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28fc024070 con 0x55fffdbbc420 2024-03-26T17:40:37.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.429+0000 7f28edffb640 1 -- 172.21.15.105:0/579741989 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc010a60 con 0x55fffdbbc420 2024-03-26T17:40:37.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 -- 172.21.15.105:0/579741989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 msgr2=0x55fffdbcd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 --2- 172.21.15.105:0/579741989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbcd3a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55fffdbd8a40 tx=0x7f28fc005f90 comp rx=0 tx=0).stop 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 -- 172.21.15.105:0/579741989 shutdown_connections 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 --2- 172.21.15.105:0/579741989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbc1590 0x55fffdbdcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 --2- 172.21.15.105:0/579741989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbbc420 0x55fffdbcd3a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 --2- 172.21.15.105:0/579741989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbba30 0x55fffdbc60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 -- 172.21.15.105:0/579741989 >> 172.21.15.105:0/579741989 conn(0x55fffdbb6fa0 msgr2=0x55fffdbbaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 -- 172.21.15.105:0/579741989 shutdown_connections 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 -- 172.21.15.105:0/579741989 wait complete. 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.430+0000 7f291140d740 1 -- start start 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f291140d740 1 Processor -- start 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f291140d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbba30 0x55fffdbcb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f291140d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbc420 0x55fffdbcb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f291140d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 0x55fffdbcc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f291140d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f29081a8120 con 0x55fffdbbba30 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f291140d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f29081a7a50 con 0x55fffdbc1590 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f291140d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29081a4c90 con 0x55fffdbbc420 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f291140d740 10 client.0 _ll_register_callbacks cb 0x7f28fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f2907fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbba30 0x55fffdbcb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f2907fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbba30 0x55fffdbcb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59066/0 (socket says 172.21.15.105:59066) 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f2907fff640 1 -- 172.21.15.105:0/3934793890 learned_addr learned my addr 172.21.15.105:0/3934793890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f29077fe640 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbc420 0x55fffdbcb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f290cdbd640 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 0x55fffdbcc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f2907fff640 1 -- 172.21.15.105:0/3934793890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbc420 msgr2=0x55fffdbcb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f2907fff640 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbc420 0x55fffdbcb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f2907fff640 1 -- 172.21.15.105:0/3934793890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 msgr2=0x55fffdbcc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f2907fff640 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 0x55fffdbcc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.431+0000 7f2907fff640 1 -- 172.21.15.105:0/3934793890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffdbccd20 con 0x55fffdbbba30 2024-03-26T17:40:37.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f2907fff640 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbba30 0x55fffdbcb230 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f28f800cfd0 tx=0x7f28f80cd2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc72d523bc6d132 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f28e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f28e77fe640 1 -- 172.21.15.105:0/3934793890 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8081070 con 0x55fffdbbba30 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f28e77fe640 1 -- 172.21.15.105:0/3934793890 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f28f80850e0 con 0x55fffdbbba30 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 1 -- 172.21.15.105:0/3934793890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28fc024070 con 0x55fffdbbba30 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f28e77fe640 1 -- 172.21.15.105:0/3934793890 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28f809b7c0 con 0x55fffdbbba30 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 10 client.16176 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 1 -- 172.21.15.105:0/3934793890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fffdc3c870 con 0x55fffdbbba30 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f28e77fe640 1 -- 172.21.15.105:0/3934793890 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28fc024070 con 0x55fffdbbba30 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f28e77fe640 1 -- 172.21.15.105:0/3934793890 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f28f80babd0 con 0x55fffdbbba30 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 10 client.16176 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 20 client.16176 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55fffdc3d0e0 0x55fffdc3f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 4 client.16176 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.432+0000 7f291140d740 1 -- 172.21.15.105:0/3934793890 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fffdaa56a0 con 0x55fffdc3d0e0 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f29077fe640 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55fffdc3d0e0 0x55fffdc3f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f29077fe640 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55fffdc3d0e0 0x55fffdc3f500 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=baaaca26dad330f0 server_cookie=5cc49b446c4a550e in_seq=0 out_seq=0 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f28e77fe640 10 client.16176 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f28e77fe640 1 -- 172.21.15.105:0/3934793890 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fffdaa56a0 con 0x55fffdc3d0e0 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f28e77fe640 10 client.16176 handle_command_reply: tid=0 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 client.16176 shutdown 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 -- 172.21.15.105:0/3934793890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbba30 msgr2=0x55fffdbcb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbba30 0x55fffdbcb230 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f28f800cfd0 tx=0x7f28f80cd2b0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 -- 172.21.15.105:0/3934793890 shutdown_connections 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55fffdc3d0e0 0x55fffdc3f500 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fffdbc1590 0x55fffdbcc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffdbbc420 0x55fffdbcb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 --2- 172.21.15.105:0/3934793890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fffdbbba30 0x55fffdbcb230 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 -- 172.21.15.105:0/3934793890 >> 172.21.15.105:0/3934793890 conn(0x55fffdbb6fa0 msgr2=0x55fffdbbaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f28e77fe640 0 client.16176 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:40:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 -- 172.21.15.105:0/3934793890 shutdown_connections 2024-03-26T17:40:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.433+0000 7f291140d740 1 -- 172.21.15.105:0/3934793890 wait complete. 2024-03-26T17:40:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f291140d740 20 client.16176 trim_cache size 0 max 16384 2024-03-26T17:40:37.432 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 -- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28f00a2fa0 msgr2=0x7f28f00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28f00a2fa0 0x7f28f00a5460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2900006010 tx=0x7f290002e040 comp rx=0 tx=0).stop 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 -- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29081007c0 msgr2=0x7f2908069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29081007c0 0x7f2908069590 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f28f800e9c0 tx=0x7f28f800ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 -- 172.21.15.105:0/2542172461 shutdown_connections 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28f00a2fa0 0x7f28f00a5460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2908102370 0x7f290819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29081019b0 0x7f290819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 --2- 172.21.15.105:0/2542172461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29081007c0 0x7f2908069590 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 -- 172.21.15.105:0/2542172461 >> 172.21.15.105:0/2542172461 conn(0x7f29080fbd50 msgr2=0x7f2908104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 -- 172.21.15.105:0/2542172461 shutdown_connections 2024-03-26T17:40:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.434+0000 7f290e820640 1 -- 172.21.15.105:0/2542172461 wait complete. 2024-03-26T17:40:37.439 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:40:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 -- 172.21.15.105:0/2122064372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18841007e0 msgr2=0x7f1884100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 --2- 172.21.15.105:0/2122064372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18841007e0 0x7f1884100c00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1880009900 tx=0x7f188002bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 -- 172.21.15.105:0/2122064372 shutdown_connections 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 --2- 172.21.15.105:0/2122064372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1884102390 0x7f1884108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 --2- 172.21.15.105:0/2122064372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18841019d0 0x7f1884101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 --2- 172.21.15.105:0/2122064372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18841007e0 0x7f1884100c00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 -- 172.21.15.105:0/2122064372 >> 172.21.15.105:0/2122064372 conn(0x7f18840fbd70 msgr2=0x7f18840fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 -- 172.21.15.105:0/2122064372 shutdown_connections 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 -- 172.21.15.105:0/2122064372 wait complete. 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.586+0000 7f188b819640 1 Processor -- start 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f188b819640 1 -- start start 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f188b819640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18841007e0 0x7f1884197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f188b819640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18841019d0 0x7f18841980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f188b819640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1884102390 0x7f188419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f188b819640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f188410c330 con 0x7f18841007e0 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f188b819640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f188410c1b0 con 0x7f1884102390 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f188b819640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f188410c4b0 con 0x7f18841019d0 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18841007e0 0x7f1884197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18841007e0 0x7f1884197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59082/0 (socket says 172.21.15.105:59082) 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 -- 172.21.15.105:0/503420842 learned_addr learned my addr 172.21.15.105:0/503420842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f1889db6640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1884102390 0x7f188419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f1888db4640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18841019d0 0x7f18841980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 -- 172.21.15.105:0/503420842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18841019d0 msgr2=0x7f18841980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18841019d0 0x7f18841980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 -- 172.21.15.105:0/503420842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1884102390 msgr2=0x7f188419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1884102390 0x7f188419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 -- 172.21.15.105:0/503420842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188419f760 con 0x7f18841007e0 2024-03-26T17:40:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.587+0000 7f18895b5640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18841007e0 0x7f1884197b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1880000c00 tx=0x7f1880005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.588+0000 7f187a7fc640 1 -- 172.21.15.105:0/503420842 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1880043020 con 0x7f18841007e0 2024-03-26T17:40:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.588+0000 7f187a7fc640 1 -- 172.21.15.105:0/503420842 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18800043d0 con 0x7f18841007e0 2024-03-26T17:40:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.588+0000 7f188b819640 1 -- 172.21.15.105:0/503420842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188419f9f0 con 0x7f18841007e0 2024-03-26T17:40:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.588+0000 7f187a7fc640 1 -- 172.21.15.105:0/503420842 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1880003b40 con 0x7f18841007e0 2024-03-26T17:40:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.588+0000 7f188b819640 1 -- 172.21.15.105:0/503420842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188419ff40 con 0x7f18841007e0 2024-03-26T17:40:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.590+0000 7f188e408740 1 Processor -- start 2024-03-26T17:40:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.590+0000 7f188e408740 1 -- start start 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f188e408740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eadfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f188e408740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eae4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f188e408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eae7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f188e408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a42ea44970 con 0x55a42eae5590 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f188e408740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a42e9ecd70 con 0x55a42eae0420 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f188e408740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a42ea40920 con 0x55a42eadfa30 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f187a7fc640 1 -- 172.21.15.105:0/503420842 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1880046020 con 0x7f18841007e0 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1888db4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eae7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1889db6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eadfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1888db4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eae7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59088/0 (socket says 172.21.15.105:59088) 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f18895b5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eae4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1889db6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eadfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41870/0 (socket says 172.21.15.105:41870) 2024-03-26T17:40:37.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1888db4640 1 -- 172.21.15.105:0/3518069166 learned_addr learned my addr 172.21.15.105:0/3518069166 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1889db6640 1 -- 172.21.15.105:0/3518069166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 msgr2=0x55a42eae4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1889db6640 1 --2- 172.21.15.105:0/3518069166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eae4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1889db6640 1 -- 172.21.15.105:0/3518069166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 msgr2=0x55a42eae7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1889db6640 1 --2- 172.21.15.105:0/3518069166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eae7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1889db6640 1 -- 172.21.15.105:0/3518069166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42e7583f0 con 0x55a42eadfa30 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f1889db6640 1 --2- 172.21.15.105:0/3518069166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eadfe50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f187c009f00 tx=0x7f187c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=301d5c82eed47b85 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f187a7fc640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f18700a2fa0 0x7f18700a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.591+0000 7f186f7fe640 1 -- 172.21.15.105:0/3518069166 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c004d20 con 0x55a42eadfa30 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f186f7fe640 1 -- 172.21.15.105:0/3518069166 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f187c004030 con 0x55a42eadfa30 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f186f7fe640 1 -- 172.21.15.105:0/3518069166 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c005280 con 0x55a42eadfa30 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f18895b5640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f18700a2fa0 0x7f18700a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f187a7fc640 1 -- 172.21.15.105:0/503420842 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f18800f15c0 con 0x7f18841007e0 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f18895b5640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f18700a2fa0 0x7f18700a5460 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f188004e000 tx=0x7f1880009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f188e408740 1 -- 172.21.15.105:0/3518069166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 msgr2=0x55a42eadfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f188e408740 1 --2- 172.21.15.105:0/3518069166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eadfe50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f187c009f00 tx=0x7f187c011810 comp rx=0 tx=0).stop 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f188e408740 1 -- 172.21.15.105:0/3518069166 shutdown_connections 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f188e408740 1 --2- 172.21.15.105:0/3518069166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eae7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f188e408740 1 --2- 172.21.15.105:0/3518069166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eae4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.592+0000 7f188e408740 1 --2- 172.21.15.105:0/3518069166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eadfe50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 -- 172.21.15.105:0/3518069166 >> 172.21.15.105:0/3518069166 conn(0x55a42eadafa0 msgr2=0x55a42eadb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 -- 172.21.15.105:0/3518069166 shutdown_connections 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 -- 172.21.15.105:0/3518069166 wait complete. 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 -- start start 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 Processor -- start 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eadfa30 0x55a42eae9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae0420 0x55a42eaea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eaf14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a42eae8540 con 0x55a42eae5590 2024-03-26T17:40:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a42ea00310 con 0x55a42eadfa30 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a42eae86c0 con 0x55a42eae0420 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f188e408740 10 client.0 _ll_register_callbacks cb 0x55a42ea3bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f1889db6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eadfa30 0x55a42eae9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f1889db6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eadfa30 0x55a42eae9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44180/0 (socket says 172.21.15.105:44180) 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.593+0000 7f1889db6640 1 -- 172.21.15.105:0/790667598 learned_addr learned my addr 172.21.15.105:0/790667598 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f1888db4640 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae0420 0x55a42eaea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f1889db6640 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eaf14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f1889db6640 1 -- 172.21.15.105:0/790667598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae0420 msgr2=0x55a42eaea4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f1889db6640 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae0420 0x55a42eaea4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f1889db6640 1 -- 172.21.15.105:0/790667598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 msgr2=0x55a42eaf14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f1889db6640 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eaf14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f1889db6640 1 -- 172.21.15.105:0/790667598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42e8227d0 con 0x55a42eadfa30 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f1889db6640 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eadfa30 0x55a42eae9f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f187c0046c0 tx=0x7f187c003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a696366105550e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f186cff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f186cff9640 1 -- 172.21.15.105:0/790667598 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c02d060 con 0x55a42eadfa30 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f186cff9640 1 -- 172.21.15.105:0/790667598 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f187c004a40 con 0x55a42eadfa30 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f188e408740 1 -- 172.21.15.105:0/790667598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f187c004030 con 0x55a42eadfa30 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f186cff9640 1 -- 172.21.15.105:0/790667598 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f187c01e040 con 0x55a42eadfa30 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f188e408740 10 client.26045 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.594+0000 7f188e408740 1 -- 172.21.15.105:0/790667598 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a42eaf4860 con 0x55a42eadfa30 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f186cff9640 1 -- 172.21.15.105:0/790667598 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f187c004030 con 0x55a42eadfa30 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f186cff9640 1 -- 172.21.15.105:0/790667598 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f187c024070 con 0x55a42eadfa30 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f188e408740 10 client.26045 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f188e408740 10 client.26045 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f188e408740 20 client.26045 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f188e408740 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a42eaf5d60 0x55a42eaf8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f188e408740 4 client.26045 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f188e408740 1 -- 172.21.15.105:0/790667598 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a42ea51eb0 con 0x55a42eaf5d60 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f1888db4640 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a42eaf5d60 0x55a42eaf8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f186cff9640 10 client.26045 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.595+0000 7f1888db4640 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a42eaf5d60 0x55a42eaf8170 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f1844b988308452 server_cookie=47dd8aec24223c88 in_seq=0 out_seq=0 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f186cff9640 1 -- 172.21.15.105:0/790667598 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a42ea51eb0 con 0x55a42eaf5d60 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f186cff9640 10 client.26045 handle_command_reply: tid=0 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 client.26045 shutdown 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 -- 172.21.15.105:0/790667598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eadfa30 msgr2=0x55a42eae9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eadfa30 0x55a42eae9f70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f187c0046c0 tx=0x7f187c003f20 comp rx=0 tx=0).stop 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 -- 172.21.15.105:0/790667598 shutdown_connections 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a42eaf5d60 0x55a42eaf8170 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eaf14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f186cff9640 0 client.26045 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae0420 0x55a42eaea4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 --2- 172.21.15.105:0/790667598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eadfa30 0x55a42eae9f70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 -- 172.21.15.105:0/790667598 >> 172.21.15.105:0/790667598 conn(0x55a42eadafa0 msgr2=0x55a42eada1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 -- 172.21.15.105:0/790667598 shutdown_connections 2024-03-26T17:40:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.596+0000 7f188e408740 1 -- 172.21.15.105:0/790667598 wait complete. 2024-03-26T17:40:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.597+0000 7f188e408740 20 client.26045 trim_cache size 0 max 16384 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f188e408740 1 Processor -- start 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f188e408740 1 -- start start 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f188e408740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eaea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f188e408740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaf1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f188e408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eade910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f188e408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a8e20 con 0x55a42eae5590 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f188e408740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f18841aa960 con 0x55a42eae0420 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f188e408740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a94f0 con 0x55a42eadfa30 2024-03-26T17:40:37.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f1889db6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eade910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f1889db6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eade910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59092/0 (socket says 172.21.15.105:59092) 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f1889db6640 1 -- 172.21.15.105:0/3489129141 learned_addr learned my addr 172.21.15.105:0/3489129141 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f1888db4640 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaf1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.617+0000 7f1889db6640 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eaea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f1889db6640 1 -- 172.21.15.105:0/3489129141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 msgr2=0x55a42eaea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f1889db6640 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eaea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f1889db6640 1 -- 172.21.15.105:0/3489129141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 msgr2=0x55a42eaf1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f1889db6640 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaf1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f1889db6640 1 -- 172.21.15.105:0/3489129141 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42eae8840 con 0x55a42eae5590 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f1889db6640 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eade910 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f187c005d70 tx=0x7f187c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bf7affe34e5f4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f186effd640 1 -- 172.21.15.105:0/3489129141 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c037040 con 0x55a42eae5590 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f186effd640 1 -- 172.21.15.105:0/3489129141 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f187c0058e0 con 0x55a42eae5590 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.618+0000 7f186effd640 1 -- 172.21.15.105:0/3489129141 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c02d040 con 0x55a42eae5590 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- 172.21.15.105:0/3489129141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 msgr2=0x55a42eade910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eade910 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f187c005d70 tx=0x7f187c014040 comp rx=0 tx=0).stop 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- 172.21.15.105:0/3489129141 shutdown_connections 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eae5590 0x55a42eade910 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaf1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 --2- 172.21.15.105:0/3489129141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eadfa30 0x55a42eaea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- 172.21.15.105:0/3489129141 >> 172.21.15.105:0/3489129141 conn(0x55a42eadafa0 msgr2=0x55a42eade200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- 172.21.15.105:0/3489129141 shutdown_connections 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- 172.21.15.105:0/3489129141 wait complete. 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- start start 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 Processor -- start 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eadfa30 0x55a42eaee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae5590 0x55a42eb5a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f18841aa290 con 0x55a42eadfa30 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a9bc0 con 0x55a42eae0420 2024-03-26T17:40:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.619+0000 7f188e408740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18841a6e00 con 0x55a42eae5590 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f188e408740 10 client.0 _ll_register_callbacks cb 0x7f18740097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eadfa30 0x55a42eaee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1888db4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eadfa30 0x55a42eaee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59106/0 (socket says 172.21.15.105:59106) 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 -- 172.21.15.105:0/2215800131 learned_addr learned my addr 172.21.15.105:0/2215800131 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae5590 0x55a42eb5a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 -- 172.21.15.105:0/2215800131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae5590 msgr2=0x55a42eb5a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae5590 0x55a42eb5a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 -- 172.21.15.105:0/2215800131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 msgr2=0x55a42eaee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaee880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 -- 172.21.15.105:0/2215800131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42eb5a5a0 con 0x55a42eadfa30 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1888db4640 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f1889db6640 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eadfa30 0x55a42eaee340 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f187c006010 tx=0x7f187c00ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621a85cdee2482c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f186cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f186cff9640 1 -- 172.21.15.105:0/2215800131 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c037070 con 0x55a42eadfa30 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f186cff9640 1 -- 172.21.15.105:0/2215800131 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f187c00fa80 con 0x55a42eadfa30 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f188e408740 1 -- 172.21.15.105:0/2215800131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f187c0058e0 con 0x55a42eadfa30 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f186cff9640 1 -- 172.21.15.105:0/2215800131 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f187c02d050 con 0x55a42eadfa30 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f188e408740 10 client.16194 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.620+0000 7f188e408740 1 -- 172.21.15.105:0/2215800131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a42eb5b820 con 0x55a42eadfa30 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f186cff9640 1 -- 172.21.15.105:0/2215800131 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f187c0058e0 con 0x55a42eadfa30 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f186cff9640 1 -- 172.21.15.105:0/2215800131 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f187c024070 con 0x55a42eadfa30 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f188e408740 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f188e408740 10 client.16194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f188e408740 20 client.16194 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f188e408740 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a42eb5c1d0 0x55a42eb5e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f188e408740 4 client.16194 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f188e408740 1 -- 172.21.15.105:0/2215800131 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a42ea4c200 con 0x55a42eb5c1d0 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f1888db4640 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a42eb5c1d0 0x55a42eb5e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f186cff9640 10 client.16194 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.621+0000 7f1888db4640 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a42eb5c1d0 0x55a42eb5e5f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39e3e078d7bd8ca8 server_cookie=8a91f3b4bde8295b in_seq=0 out_seq=0 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f186cff9640 1 -- 172.21.15.105:0/2215800131 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a42ea4c200 con 0x55a42eb5c1d0 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f186cff9640 10 client.16194 handle_command_reply: tid=0 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 client.16194 shutdown 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 -- 172.21.15.105:0/2215800131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eadfa30 msgr2=0x55a42eaee340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eadfa30 0x55a42eaee340 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f187c006010 tx=0x7f187c00ffa0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 -- 172.21.15.105:0/2215800131 shutdown_connections 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a42eb5c1d0 0x55a42eb5e5f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42eae5590 0x55a42eb5a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42eae0420 0x55a42eaee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f186cff9640 0 client.16194 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 --2- 172.21.15.105:0/2215800131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42eadfa30 0x55a42eaee340 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 -- 172.21.15.105:0/2215800131 >> 172.21.15.105:0/2215800131 conn(0x55a42eadafa0 msgr2=0x55a42eadea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 -- 172.21.15.105:0/2215800131 shutdown_connections 2024-03-26T17:40:37.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 1 -- 172.21.15.105:0/2215800131 wait complete. 2024-03-26T17:40:37.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.622+0000 7f188e408740 20 client.16194 trim_cache size 0 max 16384 2024-03-26T17:40:37.621 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 -- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f18700a2fa0 msgr2=0x7f18700a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f18700a2fa0 0x7f18700a5460 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f188004e000 tx=0x7f1880009260 comp rx=0 tx=0).stop 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 -- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18841007e0 msgr2=0x7f1884197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18841007e0 0x7f1884197b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1880000c00 tx=0x7f1880005e60 comp rx=0 tx=0).stop 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 -- 172.21.15.105:0/503420842 shutdown_connections 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f18700a2fa0 0x7f18700a5460 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1884102390 0x7f188419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18841019d0 0x7f18841980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 --2- 172.21.15.105:0/503420842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18841007e0 0x7f1884197b90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 -- 172.21.15.105:0/503420842 >> 172.21.15.105:0/503420842 conn(0x7f18840fbd70 msgr2=0x7f1884069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 -- 172.21.15.105:0/503420842 shutdown_connections 2024-03-26T17:40:37.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.623+0000 7f188b819640 1 -- 172.21.15.105:0/503420842 wait complete. 2024-03-26T17:40:37.626 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:40:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 -- 172.21.15.105:0/2126304270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e41007c0 msgr2=0x7fc6e4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 --2- 172.21.15.105:0/2126304270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e41007c0 0x7fc6e4100be0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009900 tx=0x7fc6d402bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 -- 172.21.15.105:0/2126304270 shutdown_connections 2024-03-26T17:40:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 --2- 172.21.15.105:0/2126304270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e4102370 0x7fc6e4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 --2- 172.21.15.105:0/2126304270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41019b0 0x7fc6e4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 --2- 172.21.15.105:0/2126304270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e41007c0 0x7fc6e4100be0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 -- 172.21.15.105:0/2126304270 >> 172.21.15.105:0/2126304270 conn(0x7fc6e40fbd50 msgr2=0x7fc6e40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 -- 172.21.15.105:0/2126304270 shutdown_connections 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.755+0000 7fc6e8834640 1 -- 172.21.15.105:0/2126304270 wait complete. 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e8834640 1 Processor -- start 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e8834640 1 -- start start 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e8834640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41007c0 0x7fc6e4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e8834640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41019b0 0x7fc6e4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e8834640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e4102370 0x7fc6e419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e8834640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410c2f0 con 0x7fc6e4102370 2024-03-26T17:40:37.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e8834640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410c170 con 0x7fc6e41007c0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e8834640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410c470 con 0x7fc6e41019b0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41007c0 0x7fc6e4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41007c0 0x7fc6e4196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44206/0 (socket says 172.21.15.105:44206) 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 -- 172.21.15.105:0/992076689 learned_addr learned my addr 172.21.15.105:0/992076689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41019b0 0x7fc6e4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e4102370 0x7fc6e419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 -- 172.21.15.105:0/992076689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41019b0 msgr2=0x7fc6e4196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41019b0 0x7fc6e4196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 -- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e4102370 msgr2=0x7fc6e419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e4102370 0x7fc6e419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 -- 172.21.15.105:0/992076689 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e419e670 con 0x7fc6e41007c0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.756+0000 7fc6e259c640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41007c0 0x7fc6e4196a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4000c00 tx=0x7fc6d4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.757+0000 7fc6db7fe640 1 -- 172.21.15.105:0/992076689 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4004e00 con 0x7fc6e41007c0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.757+0000 7fc6e8834640 1 -- 172.21.15.105:0/992076689 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e4068a20 con 0x7fc6e41007c0 2024-03-26T17:40:37.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.757+0000 7fc6e8834640 1 -- 172.21.15.105:0/992076689 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e4068fc0 con 0x7fc6e41007c0 2024-03-26T17:40:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.757+0000 7fc6db7fe640 1 -- 172.21.15.105:0/992076689 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d4005450 con 0x7fc6e41007c0 2024-03-26T17:40:37.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.757+0000 7fc6db7fe640 1 -- 172.21.15.105:0/992076689 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4047480 con 0x7fc6e41007c0 2024-03-26T17:40:37.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.759+0000 7fc6eb41b740 1 Processor -- start 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6eb41b740 1 -- start start 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6db7fe640 1 -- 172.21.15.105:0/992076689 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc6d40035f0 con 0x7fc6e41007c0 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f0a30 0x5595913f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f6590 0x5595913f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559591355970 con 0x5595913f0a30 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5595912fdd70 con 0x5595913f1420 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559591351920 con 0x5595913f6590 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6e2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f0a30 0x5595913f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6e259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f6590 0x5595913f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f6590 0x5595913f8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41904/0 (socket says 172.21.15.105:41904) 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6e259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44222/0 (socket says 172.21.15.105:44222) 2024-03-26T17:40:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6dbfff640 1 -- 172.21.15.105:0/2827288188 learned_addr learned my addr 172.21.15.105:0/2827288188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6dbfff640 1 -- 172.21.15.105:0/2827288188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 msgr2=0x5595913f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6dbfff640 1 --2- 172.21.15.105:0/2827288188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913f5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6dbfff640 1 -- 172.21.15.105:0/2827288188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f0a30 msgr2=0x5595913f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6dbfff640 1 --2- 172.21.15.105:0/2827288188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f0a30 0x5595913f0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.760+0000 7fc6dbfff640 1 -- 172.21.15.105:0/2827288188 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595910693f0 con 0x5595913f6590 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.761+0000 7fc6dbfff640 1 --2- 172.21.15.105:0/2827288188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f6590 0x5595913f8a20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009c90 tx=0x7fc6cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a62e7fb0d4caf22b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.761+0000 7fc6db7fe640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a2f20 0x7fc6d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.761+0000 7fc6d8ff9640 1 -- 172.21.15.105:0/2827288188 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc01e070 con 0x5595913f6590 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.761+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a2f20 0x7fc6d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.761+0000 7fc6d8ff9640 1 -- 172.21.15.105:0/2827288188 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6cc019ae0 con 0x5595913f6590 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.761+0000 7fc6db7fe640 1 -- 172.21.15.105:0/992076689 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fc6d40f34c0 con 0x7fc6e41007c0 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.761+0000 7fc6d8ff9640 1 -- 172.21.15.105:0/2827288188 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00e970 con 0x5595913f6590 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- 172.21.15.105:0/2827288188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f6590 msgr2=0x5595913f8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/2827288188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f6590 0x5595913f8a20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009c90 tx=0x7fc6cc011810 comp rx=0 tx=0).stop 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a2f20 0x7fc6d00a53e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5595913f1c80 tx=0x7fc6dc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- 172.21.15.105:0/2827288188 shutdown_connections 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/2827288188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f6590 0x5595913f8a20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/2827288188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/2827288188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f0a30 0x5595913f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- 172.21.15.105:0/2827288188 >> 172.21.15.105:0/2827288188 conn(0x5595913ebfa0 msgr2=0x5595913ec3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- 172.21.15.105:0/2827288188 shutdown_connections 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- 172.21.15.105:0/2827288188 wait complete. 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- start start 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 Processor -- start 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913faf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f1420 0x5595913fb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f6590 0x5595914024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5595913f9540 con 0x5595913f1420 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559591311310 con 0x5595913f6590 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5595913f96c0 con 0x5595913f0a30 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6eb41b740 10 client.0 _ll_register_callbacks cb 0x55959134cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f1420 0x5595913fb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6e259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.762+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f1420 0x5595913fb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59160/0 (socket says 172.21.15.105:59160) 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f6590 0x5595914024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6dbfff640 1 -- 172.21.15.105:0/353664378 learned_addr learned my addr 172.21.15.105:0/353664378 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6e259c640 1 -- 172.21.15.105:0/353664378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f6590 msgr2=0x5595914024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6e259c640 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f6590 0x5595914024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6e259c640 1 -- 172.21.15.105:0/353664378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f1420 msgr2=0x5595913fb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6e259c640 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f1420 0x5595913fb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6e259c640 1 -- 172.21.15.105:0/353664378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595911337d0 con 0x5595913f0a30 2024-03-26T17:40:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6dbfff640 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f1420 0x5595913fb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6e259c640 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913faf50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40f0fd0 tx=0x7fc6d4004550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a9a075ce24d1199 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/353664378 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d40bf980 con 0x5595913f0a30 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/353664378 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fc6d40f32c0 con 0x5595913f0a30 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6eb41b740 1 -- 172.21.15.105:0/353664378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6cc019ae0 con 0x5595913f0a30 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/353664378 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6d40bf510 con 0x5595913f0a30 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6eb41b740 10 client.26059 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.763+0000 7fc6eb41b740 1 -- 172.21.15.105:0/353664378 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5595913f95b0 con 0x5595913f0a30 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/353664378 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6cc019ae0 con 0x5595913f0a30 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/353664378 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc6d404f030 con 0x5595913f0a30 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6eb41b740 10 client.26059 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6eb41b740 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6eb41b740 20 client.26059 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559591407100 0x5595914095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6eb41b740 4 client.26059 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6eb41b740 1 -- 172.21.15.105:0/353664378 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559591362eb0 con 0x559591407100 2024-03-26T17:40:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6dbfff640 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559591407100 0x5595914095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6c9ffb640 10 client.26059 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.764+0000 7fc6dbfff640 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559591407100 0x5595914095c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db18c27d00d5873 server_cookie=77d18f135ecc1949 in_seq=0 out_seq=0 2024-03-26T17:40:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.765+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/353664378 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559591362eb0 con 0x559591407100 2024-03-26T17:40:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.765+0000 7fc6c9ffb640 10 client.26059 handle_command_reply: tid=0 2024-03-26T17:40:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.765+0000 7fc6eb41b740 1 client.26059 shutdown 2024-03-26T17:40:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 -- 172.21.15.105:0/353664378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 msgr2=0x5595913faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913faf50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40f0fd0 tx=0x7fc6d4004550 comp rx=0 tx=0).stop 2024-03-26T17:40:37.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 -- 172.21.15.105:0/353664378 shutdown_connections 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559591407100 0x5595914095c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f6590 0x5595914024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f1420 0x5595913fb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/353664378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913faf50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6c9ffb640 0 client.26059 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 -- 172.21.15.105:0/353664378 >> 172.21.15.105:0/353664378 conn(0x5595913ebfa0 msgr2=0x5595913eb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 -- 172.21.15.105:0/353664378 shutdown_connections 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 1 -- 172.21.15.105:0/353664378 wait complete. 2024-03-26T17:40:37.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.766+0000 7fc6eb41b740 20 client.26059 trim_cache size 0 max 16384 2024-03-26T17:40:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6eb41b740 1 Processor -- start 2024-03-26T17:40:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6eb41b740 1 -- start start 2024-03-26T17:40:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913fb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x559591402490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x5595913ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c400e4e0 con 0x5595913f6590 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4010020 con 0x5595913f1420 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c400ebb0 con 0x5595913f0a30 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6e2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x5595913ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6e259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913fb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6e2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x5595913ea930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59170/0 (socket says 172.21.15.105:59170) 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6e2d9d640 1 -- 172.21.15.105:0/1899879515 learned_addr learned my addr 172.21.15.105:0/1899879515 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.782+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x559591402490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6e2d9d640 1 -- 172.21.15.105:0/1899879515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 msgr2=0x5595913fb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913fb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6e2d9d640 1 -- 172.21.15.105:0/1899879515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 msgr2=0x559591402490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x559591402490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6e2d9d640 1 -- 172.21.15.105:0/1899879515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595913e7820 con 0x5595913f6590 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6e259c640 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6dbfff640 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x559591402490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x5595913ea930 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc003f70 tx=0x7fc6dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def13e7be4b8f904 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6cbfff640 1 -- 172.21.15.105:0/1899879515 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00f6e0 con 0x5595913f6590 2024-03-26T17:40:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6cbfff640 1 -- 172.21.15.105:0/1899879515 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6dc00fd30 con 0x5595913f6590 2024-03-26T17:40:37.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.783+0000 7fc6cbfff640 1 -- 172.21.15.105:0/1899879515 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc029e10 con 0x5595913f6590 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1899879515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 msgr2=0x5595913ea930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x5595913ea930 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc003f70 tx=0x7fc6dc01a000 comp rx=0 tx=0).stop 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1899879515 shutdown_connections 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x5595913ea930 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x559591402490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1899879515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913fb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1899879515 >> 172.21.15.105:0/1899879515 conn(0x5595913ebfa0 msgr2=0x5595913fab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1899879515 shutdown_connections 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1899879515 wait complete. 2024-03-26T17:40:37.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- start start 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 Processor -- start 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913ff930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x559591400e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c400f950 con 0x5595913f6590 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c400f280 con 0x5595913f1420 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c400c4c0 con 0x5595913f0a30 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.784+0000 7fc6eb41b740 10 client.0 _ll_register_callbacks cb 0x7fc6d4003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x559591400e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6dbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x559591400e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59172/0 (socket says 172.21.15.105:59172) 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 -- 172.21.15.105:0/1566028040 learned_addr learned my addr 172.21.15.105:0/1566028040 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:37.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 -- 172.21.15.105:0/1566028040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 msgr2=0x5595913ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 -- 172.21.15.105:0/1566028040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 msgr2=0x5595913ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 -- 172.21.15.105:0/1566028040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559591401470 con 0x5595913f6590 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6dbfff640 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6e2d9d640 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x559591400e80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc018040 tx=0x7fc6dc00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58fc57aed9fbd92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/1566028040 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc029d90 con 0x5595913f6590 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/1566028040 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7fc6dc004a40 con 0x5595913f6590 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1566028040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6dc00fd30 con 0x5595913f6590 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/1566028040 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6dc004c40 con 0x5595913f6590 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6eb41b740 10 client.16212 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.785+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1566028040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559591402770 con 0x5595913f6590 2024-03-26T17:40:37.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/1566028040 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6dc00fd30 con 0x5595913f6590 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/1566028040 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc6dc030070 con 0x5595913f6590 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6eb41b740 10 client.16212 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6eb41b740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6eb41b740 20 client.16212 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55959146db90 0x55959146ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6eb41b740 4 client.16212 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1566028040 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55959135d200 con 0x55959146db90 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6e259c640 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55959146db90 0x55959146ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6c9ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.786+0000 7fc6e259c640 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55959146db90 0x55959146ffb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576af64b80f53497 server_cookie=40082c5c31112edf in_seq=0 out_seq=0 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6c9ffb640 1 -- 172.21.15.105:0/1566028040 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55959135d200 con 0x55959146db90 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6c9ffb640 10 client.16212 handle_command_reply: tid=0 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 client.16212 shutdown 2024-03-26T17:40:37.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1566028040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 msgr2=0x559591400e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x559591400e80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc018040 tx=0x7fc6dc00ff70 comp rx=0 tx=0).stop 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1566028040 shutdown_connections 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55959146db90 0x55959146ffb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6c9ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5595913f6590 0x559591400e80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595913f1420 0x5595913ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 --2- 172.21.15.105:0/1566028040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595913f0a30 0x5595913ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1566028040 >> 172.21.15.105:0/1566028040 conn(0x5595913ebfa0 msgr2=0x5595913ef200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1566028040 shutdown_connections 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.787+0000 7fc6eb41b740 1 -- 172.21.15.105:0/1566028040 wait complete. 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.788+0000 7fc6eb41b740 20 client.16212 trim_cache size 0 max 16384 2024-03-26T17:40:37.785 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:40:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.788+0000 7fc6e8834640 1 -- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a2f20 msgr2=0x7fc6d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.788+0000 7fc6e8834640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a2f20 0x7fc6d00a53e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5595913f1c80 tx=0x7fc6dc009500 comp rx=0 tx=0).stop 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.788+0000 7fc6e8834640 1 -- 172.21.15.105:0/992076689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41007c0 msgr2=0x7fc6e4196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.788+0000 7fc6e8834640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41007c0 0x7fc6e4196a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4000c00 tx=0x7fc6d4005dc0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.789+0000 7fc6e8834640 1 -- 172.21.15.105:0/992076689 shutdown_connections 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.789+0000 7fc6e8834640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a2f20 0x7fc6d00a53e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.789+0000 7fc6e8834640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e4102370 0x7fc6e419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.789+0000 7fc6e8834640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41019b0 0x7fc6e4196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.789+0000 7fc6e8834640 1 --2- 172.21.15.105:0/992076689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41007c0 0x7fc6e4196a50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.789+0000 7fc6e8834640 1 -- 172.21.15.105:0/992076689 >> 172.21.15.105:0/992076689 conn(0x7fc6e40fbd50 msgr2=0x7fc6e4104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.789+0000 7fc6e8834640 1 -- 172.21.15.105:0/992076689 shutdown_connections 2024-03-26T17:40:37.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:37.789+0000 7fc6e8834640 1 -- 172.21.15.105:0/992076689 wait complete. 2024-03-26T17:40:37.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:37 smithi105 ceph-mon[32280]: pgmap v415: 97 pgs: 97 active+clean; 26 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-03-26T17:40:37.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:37 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3129508937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:40:38.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 48483 3.41 MB/sec warmup 81 sec latency 17.966 ms 2024-03-26T17:40:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:37 smithi144 ceph-mon[36095]: pgmap v415: 97 pgs: 97 active+clean; 26 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-03-26T17:40:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:37 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3129508937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:40:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:37 smithi135 ceph-mon[35636]: pgmap v415: 97 pgs: 97 active+clean; 26 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-03-26T17:40:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:37 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3129508937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:40:38.792 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:40:38.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:40:38.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:40:38.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 -- 172.21.15.105:0/1260410785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8278102370 msgr2=0x7f8278108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 --2- 172.21.15.105:0/1260410785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8278102370 0x7f8278108c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8270009ca0 tx=0x7f827002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 -- 172.21.15.105:0/1260410785 shutdown_connections 2024-03-26T17:40:38.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 --2- 172.21.15.105:0/1260410785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8278102370 0x7f8278108c00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 --2- 172.21.15.105:0/1260410785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82781019b0 0x7f8278101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 --2- 172.21.15.105:0/1260410785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82781007c0 0x7f8278100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 -- 172.21.15.105:0/1260410785 >> 172.21.15.105:0/1260410785 conn(0x7f82780fbd50 msgr2=0x7f82780fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:38.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 -- 172.21.15.105:0/1260410785 shutdown_connections 2024-03-26T17:40:38.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.931+0000 7f827da92640 1 -- 172.21.15.105:0/1260410785 wait complete. 2024-03-26T17:40:38.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f827da92640 1 Processor -- start 2024-03-26T17:40:38.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f827da92640 1 -- start start 2024-03-26T17:40:38.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f827da92640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82781007c0 0x7f8278104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f827da92640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82781019b0 0x7f8278102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f827da92640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8278102370 0x7f82781032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f827da92640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f827810c2d0 con 0x7f82781019b0 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f827da92640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f827810c150 con 0x7f8278102370 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f827da92640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f827810c450 con 0x7f82781007c0 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82781019b0 0x7f8278102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82781019b0 0x7f8278102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59192/0 (socket says 172.21.15.105:59192) 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f8276ffd640 1 -- 172.21.15.105:0/3692695299 learned_addr learned my addr 172.21.15.105:0/3692695299 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f82777fe640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82781007c0 0x7f8278104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.932+0000 7f8277fff640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8278102370 0x7f82781032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f82777fe640 1 -- 172.21.15.105:0/3692695299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8278102370 msgr2=0x7f82781032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f82777fe640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8278102370 0x7f82781032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f82777fe640 1 -- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82781019b0 msgr2=0x7f8278102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f82777fe640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82781019b0 0x7f8278102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f82777fe640 1 -- 172.21.15.105:0/3692695299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8278103b60 con 0x7f82781007c0 2024-03-26T17:40:38.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f8276ffd640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82781019b0 0x7f8278102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:38.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f82777fe640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82781007c0 0x7f8278104750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f826800e9c0 tx=0x7f826800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:38.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f8274ff9640 1 -- 172.21.15.105:0/3692695299 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826800ccc0 con 0x7f82781007c0 2024-03-26T17:40:38.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f827da92640 1 -- 172.21.15.105:0/3692695299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82781a1bd0 con 0x7f82781007c0 2024-03-26T17:40:38.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f8274ff9640 1 -- 172.21.15.105:0/3692695299 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82680035f0 con 0x7f82781007c0 2024-03-26T17:40:38.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f8274ff9640 1 -- 172.21.15.105:0/3692695299 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826800ce60 con 0x7f82781007c0 2024-03-26T17:40:38.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.933+0000 7f827da92640 1 -- 172.21.15.105:0/3692695299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781a2170 con 0x7f82781007c0 2024-03-26T17:40:38.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.935+0000 7f8274ff9640 1 -- 172.21.15.105:0/3692695299 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f82680037a0 con 0x7f82781007c0 2024-03-26T17:40:38.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.936+0000 7f8274ff9640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82600a2f20 0x7f82600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.936+0000 7f8274ff9640 1 -- 172.21.15.105:0/3692695299 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f82680cf890 con 0x7f82781007c0 2024-03-26T17:40:38.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.936+0000 7f8276ffd640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82600a2f20 0x7f82600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.936+0000 7f8276ffd640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82600a2f20 0x7f82600a53e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f82781044c0 tx=0x7f826c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:38.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.936+0000 7f828067d740 1 Processor -- start 2024-03-26T17:40:38.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.936+0000 7f828067d740 1 -- start start 2024-03-26T17:40:38.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f828067d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de1190 0x560f31de15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f828067d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31de6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f828067d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de6cf0 0x560f31de9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f828067d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560f31d46630 con 0x560f31de6cf0 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f828067d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560f31ceee10 con 0x560f31de1b80 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f828067d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560f31d425e0 con 0x560f31de1190 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31de6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de6cf0 0x560f31de9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de6cf0 0x560f31de9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59208/0 (socket says 172.21.15.105:59208) 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31de6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44278/0 (socket says 172.21.15.105:44278) 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 -- 172.21.15.105:0/1762482059 learned_addr learned my addr 172.21.15.105:0/1762482059 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f82777fe640 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de1190 0x560f31de15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 -- 172.21.15.105:0/1762482059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de1190 msgr2=0x560f31de15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de1190 0x560f31de15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 -- 172.21.15.105:0/1762482059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 msgr2=0x560f31de6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:38.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31de6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 -- 172.21.15.105:0/1762482059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f31d05740 con 0x560f31de6cf0 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8276ffd640 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31de6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f82777fe640 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de1190 0x560f31de15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f8277fff640 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de6cf0 0x560f31de9180 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8270005e30 tx=0x7f8270009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27c271c7fe9560 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f825dffb640 1 -- 172.21.15.105:0/1762482059 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270043070 con 0x560f31de6cf0 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f825dffb640 1 -- 172.21.15.105:0/1762482059 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8270032db0 con 0x560f31de6cf0 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.937+0000 7f825dffb640 1 -- 172.21.15.105:0/1762482059 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270039040 con 0x560f31de6cf0 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.938+0000 7f828067d740 1 -- 172.21.15.105:0/1762482059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de6cf0 msgr2=0x560f31de9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.938+0000 7f828067d740 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de6cf0 0x560f31de9180 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8270005e30 tx=0x7f8270009800 comp rx=0 tx=0).stop 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.938+0000 7f828067d740 1 -- 172.21.15.105:0/1762482059 shutdown_connections 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.938+0000 7f828067d740 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de6cf0 0x560f31de9180 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.938+0000 7f828067d740 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31de6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.938+0000 7f828067d740 1 --2- 172.21.15.105:0/1762482059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de1190 0x560f31de15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.938+0000 7f828067d740 1 -- 172.21.15.105:0/1762482059 >> 172.21.15.105:0/1762482059 conn(0x560f31ddc700 msgr2=0x560f31ddcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:38.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.939+0000 7f828067d740 1 -- 172.21.15.105:0/1762482059 shutdown_connections 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.939+0000 7f828067d740 1 -- 172.21.15.105:0/1762482059 wait complete. 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.939+0000 7f828067d740 1 -- start start 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.939+0000 7f828067d740 1 Processor -- start 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.939+0000 7f828067d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1190 0x560f31deb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.939+0000 7f828067d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31debb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.939+0000 7f828067d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de6cf0 0x560f31df2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f828067d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560f31de9c40 con 0x560f31de1190 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f828067d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560f31d021a0 con 0x560f31de1b80 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f828067d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560f31de9dc0 con 0x560f31de6cf0 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1190 0x560f31deb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1190 0x560f31deb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59210/0 (socket says 172.21.15.105:59210) 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 -- 172.21.15.105:0/1116752989 learned_addr learned my addr 172.21.15.105:0/1116752989 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f828067d740 10 client.0 _ll_register_callbacks cb 0x560f31a59c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:38.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f8276ffd640 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31debb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f8277fff640 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de6cf0 0x560f31df2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 -- 172.21.15.105:0/1116752989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de6cf0 msgr2=0x560f31df2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de6cf0 0x560f31df2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 -- 172.21.15.105:0/1116752989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 msgr2=0x560f31debb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31debb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 -- 172.21.15.105:0/1116752989 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f31b227e0 con 0x560f31de1190 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f82777fe640 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1190 0x560f31deb630 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f82680cd350 tx=0x7f826809c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d23ea4ced47d2dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f8256ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f8256ffd640 1 -- 172.21.15.105:0/1116752989 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680a4990 con 0x560f31de1190 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f8256ffd640 1 -- 172.21.15.105:0/1116752989 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f82680985f0 con 0x560f31de1190 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f828067d740 1 -- 172.21.15.105:0/1116752989 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8270032db0 con 0x560f31de1190 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f8256ffd640 1 -- 172.21.15.105:0/1116752989 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f826809b3d0 con 0x560f31de1190 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f828067d740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f828067d740 1 -- 172.21.15.105:0/1116752989 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f31b82f70 con 0x560f31de1190 2024-03-26T17:40:38.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.940+0000 7f8256ffd640 1 -- 172.21.15.105:0/1116752989 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8270032db0 con 0x560f31de1190 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f8256ffd640 1 -- 172.21.15.105:0/1116752989 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f826809afd0 con 0x560f31de1190 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f828067d740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f828067d740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f828067d740 20 client.16224 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f828067d740 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560f31df7540 0x560f31df9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f828067d740 4 client.16224 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f828067d740 1 -- 172.21.15.105:0/1116752989 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f31d53b70 con 0x560f31df7540 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f8276ffd640 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560f31df7540 0x560f31df9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f8276ffd640 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560f31df7540 0x560f31df9a00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc4ca2e845bafe2 server_cookie=ac50c716a53ccdf2 in_seq=0 out_seq=0 2024-03-26T17:40:38.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.941+0000 7f8256ffd640 10 client.16224 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:38.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f8256ffd640 1 -- 172.21.15.105:0/1116752989 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560f31d53b70 con 0x560f31df7540 2024-03-26T17:40:38.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f8256ffd640 10 client.16224 handle_command_reply: tid=0 2024-03-26T17:40:38.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 client.16224 shutdown 2024-03-26T17:40:38.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 -- 172.21.15.105:0/1116752989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1190 msgr2=0x560f31deb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1190 0x560f31deb630 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f82680cd350 tx=0x7f826809c700 comp rx=0 tx=0).stop 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 -- 172.21.15.105:0/1116752989 shutdown_connections 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560f31df7540 0x560f31df9a00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31de6cf0 0x560f31df2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1b80 0x560f31debb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f8256ffd640 0 client.16224 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 --2- 172.21.15.105:0/1116752989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1190 0x560f31deb630 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 -- 172.21.15.105:0/1116752989 >> 172.21.15.105:0/1116752989 conn(0x560f31ddc700 msgr2=0x560f31ddb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 -- 172.21.15.105:0/1116752989 shutdown_connections 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 1 -- 172.21.15.105:0/1116752989 wait complete. 2024-03-26T17:40:38.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.943+0000 7f828067d740 20 client.16224 trim_cache size 0 max 16384 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.961+0000 7f828067d740 1 Processor -- start 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.961+0000 7f828067d740 1 -- start start 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.961+0000 7f828067d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31dff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.961+0000 7f828067d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31ddb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.961+0000 7f828067d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31ddbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.961+0000 7f828067d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82781aaf70 con 0x560f31de1b80 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.961+0000 7f828067d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82781acab0 con 0x560f31de1190 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.961+0000 7f828067d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82781ab640 con 0x560f31ddc700 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8277fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31ddbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f82777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31dff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f82777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31dff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41974/0 (socket says 172.21.15.105:41974) 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31ddb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8277fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31ddbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59224/0 (socket says 172.21.15.105:59224) 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f82777fe640 1 -- 172.21.15.105:0/2668768582 learned_addr learned my addr 172.21.15.105:0/2668768582 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8277fff640 1 -- 172.21.15.105:0/2668768582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 msgr2=0x560f31dff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8277fff640 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31dff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8277fff640 1 -- 172.21.15.105:0/2668768582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 msgr2=0x560f31ddb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8277fff640 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31ddb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8277fff640 1 -- 172.21.15.105:0/2668768582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f31a1a170 con 0x560f31de1b80 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8276ffd640 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31ddb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f82777fe640 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31dff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f8277fff640 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31ddbcc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f827002c700 tx=0x7f827002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec0270ed788aa97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:38.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f825d7fa640 1 -- 172.21.15.105:0/2668768582 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270039050 con 0x560f31de1b80 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f825d7fa640 1 -- 172.21.15.105:0/2668768582 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8270032db0 con 0x560f31de1b80 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.962+0000 7f825d7fa640 1 -- 172.21.15.105:0/2668768582 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270043040 con 0x560f31de1b80 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 -- 172.21.15.105:0/2668768582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 msgr2=0x560f31ddbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31ddbcc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f827002c700 tx=0x7f827002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 -- 172.21.15.105:0/2668768582 shutdown_connections 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31ddbcc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31ddb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 --2- 172.21.15.105:0/2668768582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31dff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 -- 172.21.15.105:0/2668768582 >> 172.21.15.105:0/2668768582 conn(0x560f31dd7780 msgr2=0x560f31ddb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 -- 172.21.15.105:0/2668768582 shutdown_connections 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.963+0000 7f828067d740 1 -- 172.21.15.105:0/2668768582 wait complete. 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 1 -- start start 2024-03-26T17:40:38.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 1 Processor -- start 2024-03-26T17:40:38.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31df1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31df1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31e5d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82781ac3e0 con 0x560f31de1b80 2024-03-26T17:40:38.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82781abd10 con 0x560f31de1190 2024-03-26T17:40:38.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82781a8f50 con 0x560f31ddc700 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f828067d740 10 client.0 _ll_register_callbacks cb 0x7f82680046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f82777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31df1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f8277fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31e5d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f82777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31df1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41976/0 (socket says 172.21.15.105:41976) 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f82777fe640 1 -- 172.21.15.105:0/2049630140 learned_addr learned my addr 172.21.15.105:0/2049630140 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f8276ffd640 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31df1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f8277fff640 1 -- 172.21.15.105:0/2049630140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 msgr2=0x560f31df1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f8277fff640 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31df1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f8277fff640 1 -- 172.21.15.105:0/2049630140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 msgr2=0x560f31df1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.964+0000 7f8277fff640 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31df1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f8277fff640 1 -- 172.21.15.105:0/2049630140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f31e5de40 con 0x560f31de1b80 2024-03-26T17:40:38.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f82777fe640 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31df1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f8277fff640 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31e5d900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8270006010 tx=0x7f8270032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51313845a83dcda server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f8256ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f8256ffd640 1 -- 172.21.15.105:0/2049630140 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270055070 con 0x560f31de1b80 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f8256ffd640 1 -- 172.21.15.105:0/2049630140 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f82700327f0 con 0x560f31de1b80 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f828067d740 1 -- 172.21.15.105:0/2049630140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8270032db0 con 0x560f31de1b80 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f8256ffd640 1 -- 172.21.15.105:0/2049630140 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8270039070 con 0x560f31de1b80 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f828067d740 10 client.16230 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f828067d740 1 -- 172.21.15.105:0/2049630140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560f31e5f0e0 con 0x560f31de1b80 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f8256ffd640 1 -- 172.21.15.105:0/2049630140 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8270032db0 con 0x560f31de1b80 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f8256ffd640 1 -- 172.21.15.105:0/2049630140 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f827005a460 con 0x560f31de1b80 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.965+0000 7f828067d740 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.966+0000 7f828067d740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.966+0000 7f828067d740 20 client.16230 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:38.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.966+0000 7f828067d740 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560f31de6cf0 0x560f31e61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.966+0000 7f828067d740 4 client.16230 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.966+0000 7f828067d740 1 -- 172.21.15.105:0/2049630140 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560f31d4dec0 con 0x560f31de6cf0 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.966+0000 7f82777fe640 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560f31de6cf0 0x560f31e61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.966+0000 7f8256ffd640 10 client.16230 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.966+0000 7f82777fe640 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560f31de6cf0 0x560f31e61570 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27525f8284e39a4 server_cookie=813964ae391e7464 in_seq=0 out_seq=0 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.967+0000 7f8256ffd640 1 -- 172.21.15.105:0/2049630140 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560f31d4dec0 con 0x560f31de6cf0 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.967+0000 7f8256ffd640 10 client.16230 handle_command_reply: tid=0 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.967+0000 7f828067d740 1 client.16230 shutdown 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.967+0000 7f828067d740 1 -- 172.21.15.105:0/2049630140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 msgr2=0x560f31e5d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.967+0000 7f828067d740 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31e5d900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8270006010 tx=0x7f8270032fc0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 1 -- 172.21.15.105:0/2049630140 shutdown_connections 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560f31de6cf0 0x560f31e61570 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560f31de1b80 0x560f31e5d900 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560f31de1190 0x560f31df1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 1 --2- 172.21.15.105:0/2049630140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560f31ddc700 0x560f31df1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 1 -- 172.21.15.105:0/2049630140 >> 172.21.15.105:0/2049630140 conn(0x560f31dd7780 msgr2=0x560f31ddb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f8256ffd640 0 client.16230 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 1 -- 172.21.15.105:0/2049630140 shutdown_connections 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 1 -- 172.21.15.105:0/2049630140 wait complete. 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f828067d740 20 client.16230 trim_cache size 0 max 16384 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "d68144a2-2262-4221-88f2-7bdac5ae6d60", 2024-03-26T17:40:38.966 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:40:38.967 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:40:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 -- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82600a2f20 msgr2=0x7f82600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82600a2f20 0x7f82600a53e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f82781044c0 tx=0x7f826c005f90 comp rx=0 tx=0).stop 2024-03-26T17:40:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 -- 172.21.15.105:0/3692695299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82781007c0 msgr2=0x7f8278104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82781007c0 0x7f8278104750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f826800e9c0 tx=0x7f826800ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:38.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 -- 172.21.15.105:0/3692695299 shutdown_connections 2024-03-26T17:40:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82600a2f20 0x7f82600a53e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8278102370 0x7f82781032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82781019b0 0x7f8278102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 --2- 172.21.15.105:0/3692695299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82781007c0 0x7f8278104750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 -- 172.21.15.105:0/3692695299 >> 172.21.15.105:0/3692695299 conn(0x7f82780fbd50 msgr2=0x7f82780fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 -- 172.21.15.105:0/3692695299 shutdown_connections 2024-03-26T17:40:38.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:38.968+0000 7f827da92640 1 -- 172.21.15.105:0/3692695299 wait complete. 2024-03-26T17:40:38.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:40:38.972 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:40:39.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 49138 3.44 MB/sec warmup 82 sec latency 18.456 ms 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 -- 172.21.15.105:0/2614507849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4104c90 msgr2=0x7f28e4107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 --2- 172.21.15.105:0/2614507849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4104c90 0x7f28e4107080 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009900 tx=0x7f28d802bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 -- 172.21.15.105:0/2614507849 shutdown_connections 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 --2- 172.21.15.105:0/2614507849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e41075c0 0x7f28e4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 --2- 172.21.15.105:0/2614507849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4104c90 0x7f28e4107080 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 --2- 172.21.15.105:0/2614507849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28e4102360 0x7f28e4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 -- 172.21.15.105:0/2614507849 >> 172.21.15.105:0/2614507849 conn(0x7f28e40fbd70 msgr2=0x7f28e40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 -- 172.21.15.105:0/2614507849 shutdown_connections 2024-03-26T17:40:39.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.099+0000 7f28ea190640 1 -- 172.21.15.105:0/2614507849 wait complete. 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.100+0000 7f28ea190640 1 Processor -- start 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.100+0000 7f28ea190640 1 -- start start 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.100+0000 7f28ea190640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4102360 0x7f28e4197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.100+0000 7f28ea190640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e4104c90 0x7f28e4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.100+0000 7f28ea190640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28e41075c0 0x7f28e419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.100+0000 7f28ea190640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28e410d400 con 0x7f28e4102360 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.100+0000 7f28ea190640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28e410d280 con 0x7f28e41075c0 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.100+0000 7f28ea190640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28e410d580 con 0x7f28e4104c90 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4102360 0x7f28e4197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4102360 0x7f28e4197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59252/0 (socket says 172.21.15.105:59252) 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 -- 172.21.15.105:0/2247585971 learned_addr learned my addr 172.21.15.105:0/2247585971 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e4104c90 0x7f28e4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e3fff640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28e41075c0 0x7f28e419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 -- 172.21.15.105:0/2247585971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e4104c90 msgr2=0x7f28e4198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e4104c90 0x7f28e4198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 -- 172.21.15.105:0/2247585971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28e41075c0 msgr2=0x7f28e419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28e41075c0 0x7f28e419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 -- 172.21.15.105:0/2247585971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e419f760 con 0x7f28e4102360 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e37fe640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4102360 0x7f28e4197b40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f28d400e9f0 tx=0x7f28d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e0ff9640 1 -- 172.21.15.105:0/2247585971 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d400cd80 con 0x7f28e4102360 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e0ff9640 1 -- 172.21.15.105:0/2247585971 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28d40108e0 con 0x7f28e4102360 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28ea190640 1 -- 172.21.15.105:0/2247585971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e41001b0 con 0x7f28e4102360 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28e0ff9640 1 -- 172.21.15.105:0/2247585971 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4018920 con 0x7f28e4102360 2024-03-26T17:40:39.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.101+0000 7f28ea190640 1 -- 172.21.15.105:0/2247585971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e41006d0 con 0x7f28e4102360 2024-03-26T17:40:39.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28e0ff9640 1 -- 172.21.15.105:0/2247585971 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f28d4010a90 con 0x7f28e4102360 2024-03-26T17:40:39.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28ecd78740 1 Processor -- start 2024-03-26T17:40:39.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28ecd78740 1 -- start start 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7f100 0x556e07e7f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28e0ff9640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28cc0a2f20 0x7f28cc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7faf0 0x556e07e845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e84c60 0x556e07e870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28ecd78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556e07ddb980 con 0x556e07e7faf0 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28ecd78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556e07d83be0 con 0x556e07e7f100 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.104+0000 7f28ecd78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e07dd7930 con 0x556e07e84c60 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e37fe640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28cc0a2f20 0x7f28cc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7f100 0x556e07e7f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7f100 0x556e07e7f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44332/0 (socket says 172.21.15.105:44332) 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 -- 172.21.15.105:0/1268169871 learned_addr learned my addr 172.21.15.105:0/1268169871 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7faf0 0x556e07e845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e84c60 0x556e07e870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e0ff9640 1 -- 172.21.15.105:0/2247585971 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f28d40cf700 con 0x7f28e4102360 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 -- 172.21.15.105:0/1268169871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e84c60 msgr2=0x556e07e870f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e84c60 0x556e07e870f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 -- 172.21.15.105:0/1268169871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7faf0 msgr2=0x556e07e845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7faf0 0x556e07e845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 -- 172.21.15.105:0/1268169871 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e07bb97d0 con 0x556e07e7f100 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e3fff640 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7f100 0x556e07e7f520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005f40 tx=0x7f28dc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b386168b8264785 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28e37fe640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28cc0a2f20 0x7f28cc0a53e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f28d4012020 tx=0x7f28d40c6ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28c9ffb640 1 -- 172.21.15.105:0/1268169871 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc004b20 con 0x556e07e7f100 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28c9ffb640 1 -- 172.21.15.105:0/1268169871 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28dc00fd70 con 0x556e07e7f100 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.105+0000 7f28c9ffb640 1 -- 172.21.15.105:0/1268169871 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc004b20 con 0x556e07e7f100 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 -- 172.21.15.105:0/1268169871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7f100 msgr2=0x556e07e7f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7f100 0x556e07e7f520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005f40 tx=0x7f28dc009c30 comp rx=0 tx=0).stop 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 -- 172.21.15.105:0/1268169871 shutdown_connections 2024-03-26T17:40:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e84c60 0x556e07e870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7faf0 0x556e07e845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1268169871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7f100 0x556e07e7f520 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 -- 172.21.15.105:0/1268169871 >> 172.21.15.105:0/1268169871 conn(0x556e07e71e60 msgr2=0x556e07e72260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 -- 172.21.15.105:0/1268169871 shutdown_connections 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.106+0000 7f28ecd78740 1 -- 172.21.15.105:0/1268169871 wait complete. 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 1 -- start start 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 1 Processor -- start 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7f100 0x556e07e892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7faf0 0x556e07e89830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e84c60 0x556e07e90870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556e07e87c10 con 0x556e07e84c60 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556e07d97160 con 0x556e07e7faf0 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556e07e87d90 con 0x556e07e7f100 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e84c60 0x556e07e90870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28ecd78740 10 client.0 _ll_register_callbacks cb 0x556e07dd3720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28e2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7faf0 0x556e07e89830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e84c60 0x556e07e90870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59276/0 (socket says 172.21.15.105:59276) 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28e3fff640 1 -- 172.21.15.105:0/3318750414 learned_addr learned my addr 172.21.15.105:0/3318750414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.107+0000 7f28e3fff640 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7f100 0x556e07e892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28e3fff640 1 -- 172.21.15.105:0/3318750414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7f100 msgr2=0x556e07e892f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28e3fff640 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7f100 0x556e07e892f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28e3fff640 1 -- 172.21.15.105:0/3318750414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7faf0 msgr2=0x556e07e89830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28e3fff640 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7faf0 0x556e07e89830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28e3fff640 1 -- 172.21.15.105:0/3318750414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e07aef3f0 con 0x556e07e84c60 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7faf0 0x556e07e89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28e3fff640 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e84c60 0x556e07e90870 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005fd0 tx=0x7f28dc0115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622c54298fe33d71 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28c2ffd640 1 -- 172.21.15.105:0/3318750414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc004b20 con 0x556e07e84c60 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28c2ffd640 1 -- 172.21.15.105:0/3318750414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f28dc01a030 con 0x556e07e84c60 2024-03-26T17:40:39.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28ecd78740 1 -- 172.21.15.105:0/3318750414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28dc00fd70 con 0x556e07e84c60 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28c2ffd640 1 -- 172.21.15.105:0/3318750414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28dc035050 con 0x556e07e84c60 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28ecd78740 10 client.16242 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28ecd78740 1 -- 172.21.15.105:0/3318750414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e07aaf0f0 con 0x556e07e84c60 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.108+0000 7f28c2ffd640 1 -- 172.21.15.105:0/3318750414 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28dc00fd70 con 0x556e07e84c60 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28c2ffd640 1 -- 172.21.15.105:0/3318750414 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f28dc01e9f0 con 0x556e07e84c60 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28ecd78740 10 client.16242 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28ecd78740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28ecd78740 20 client.16242 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28ecd78740 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e07e94e80 0x556e07e972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28ecd78740 4 client.16242 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28ecd78740 1 -- 172.21.15.105:0/3318750414 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e07de8ec0 con 0x556e07e94e80 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e07e94e80 0x556e07e972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28c2ffd640 10 client.16242 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:39.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.109+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e07e94e80 0x556e07e972d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5b16b0a2f4f935 server_cookie=16f2335d124bc912 in_seq=0 out_seq=0 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.110+0000 7f28c2ffd640 1 -- 172.21.15.105:0/3318750414 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556e07de8ec0 con 0x556e07e94e80 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.110+0000 7f28c2ffd640 10 client.16242 handle_command_reply: tid=0 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.110+0000 7f28ecd78740 1 client.16242 shutdown 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 -- 172.21.15.105:0/3318750414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e84c60 msgr2=0x556e07e90870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e84c60 0x556e07e90870 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f28dc005fd0 tx=0x7f28dc0115f0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 -- 172.21.15.105:0/3318750414 shutdown_connections 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e07e94e80 0x556e07e972d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e84c60 0x556e07e90870 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e7faf0 0x556e07e89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28c2ffd640 0 client.16242 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 --2- 172.21.15.105:0/3318750414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7f100 0x556e07e892f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 -- 172.21.15.105:0/3318750414 >> 172.21.15.105:0/3318750414 conn(0x556e07e71e60 msgr2=0x556e07e71080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 -- 172.21.15.105:0/3318750414 shutdown_connections 2024-03-26T17:40:39.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 1 -- 172.21.15.105:0/3318750414 wait complete. 2024-03-26T17:40:39.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.111+0000 7f28ecd78740 20 client.16242 trim_cache size 0 max 16384 2024-03-26T17:40:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.133+0000 7f28ecd78740 1 Processor -- start 2024-03-26T17:40:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.133+0000 7f28ecd78740 1 -- start start 2024-03-26T17:40:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.133+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e72240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.133+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 0x556e07e90830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.133+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 0x556e07e707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.133+0000 7f28ecd78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41a6bf0 con 0x556e07e7f100 2024-03-26T17:40:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.133+0000 7f28ecd78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41a8730 con 0x556e07e84c60 2024-03-26T17:40:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.133+0000 7f28ecd78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41a72c0 con 0x556e07e7faf0 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e72240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e72240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59278/0 (socket says 172.21.15.105:59278) 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 -- 172.21.15.105:0/2886560056 learned_addr learned my addr 172.21.15.105:0/2886560056 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 0x556e07e90830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 0x556e07e707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 -- 172.21.15.105:0/2886560056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 msgr2=0x556e07e90830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 0x556e07e90830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 -- 172.21.15.105:0/2886560056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 msgr2=0x556e07e707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 0x556e07e707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 -- 172.21.15.105:0/2886560056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e07e83fc0 con 0x556e07e7f100 2024-03-26T17:40:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28e3fff640 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e72240 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00fc50 tx=0x7f28dc009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d62ea9f7cace10 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28c97fa640 1 -- 172.21.15.105:0/2886560056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc009500 con 0x556e07e7f100 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28c97fa640 1 -- 172.21.15.105:0/2886560056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28dc00fd70 con 0x556e07e7f100 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.134+0000 7f28c97fa640 1 -- 172.21.15.105:0/2886560056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc01e420 con 0x556e07e7f100 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 -- 172.21.15.105:0/2886560056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 msgr2=0x556e07e72240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e72240 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00fc50 tx=0x7f28dc009230 comp rx=0 tx=0).stop 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 -- 172.21.15.105:0/2886560056 shutdown_connections 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 0x556e07e707f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 0x556e07e90830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 --2- 172.21.15.105:0/2886560056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e72240 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 -- 172.21.15.105:0/2886560056 >> 172.21.15.105:0/2886560056 conn(0x556e07e71e60 msgr2=0x556e07e84340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 -- 172.21.15.105:0/2886560056 shutdown_connections 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.135+0000 7f28ecd78740 1 -- 172.21.15.105:0/2886560056 wait complete. 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 1 -- start start 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 1 Processor -- start 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e8dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 0x556e07e8e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 0x556e07e8f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41a8060 con 0x556e07e7f100 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41a7990 con 0x556e07e84c60 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28e41a4bd0 con 0x556e07e7faf0 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28ecd78740 10 client.0 _ll_register_callbacks cb 0x7f28dc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e8dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e8dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59292/0 (socket says 172.21.15.105:59292) 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 -- 172.21.15.105:0/1798591892 learned_addr learned my addr 172.21.15.105:0/1798591892 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 0x556e07e8e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 0x556e07e8f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 -- 172.21.15.105:0/1798591892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 msgr2=0x556e07e8e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 0x556e07e8e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 -- 172.21.15.105:0/1798591892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 msgr2=0x556e07e8f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 0x556e07e8f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.136+0000 7f28e3fff640 1 -- 172.21.15.105:0/1798591892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e07e8fae0 con 0x556e07e7f100 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28e3fff640 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e8dfa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009a70 tx=0x7f28dc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc067342470d38b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28c2ffd640 1 -- 172.21.15.105:0/1798591892 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc040020 con 0x556e07e7f100 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28c2ffd640 1 -- 172.21.15.105:0/1798591892 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f28dc011030 con 0x556e07e7f100 2024-03-26T17:40:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28ecd78740 1 -- 172.21.15.105:0/1798591892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28dc00fd70 con 0x556e07e7f100 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28c2ffd640 1 -- 172.21.15.105:0/1798591892 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28dc04b050 con 0x556e07e7f100 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28ecd78740 10 client.16248 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28ecd78740 1 -- 172.21.15.105:0/1798591892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e07efb1d0 con 0x556e07e7f100 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.137+0000 7f28c2ffd640 1 -- 172.21.15.105:0/1798591892 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28dc00fd70 con 0x556e07e7f100 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28c2ffd640 1 -- 172.21.15.105:0/1798591892 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f28dc01a070 con 0x556e07e7f100 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28ecd78740 10 client.16248 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28ecd78740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28ecd78740 20 client.16248 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e07efba00 0x556e07efddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28ecd78740 4 client.16248 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28ecd78740 1 -- 172.21.15.105:0/1798591892 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e07de3210 con 0x556e07efba00 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e07efba00 0x556e07efddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28c2ffd640 10 client.16248 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.138+0000 7f28e2ffd640 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e07efba00 0x556e07efddf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9606655aa3b90117 server_cookie=6fd70093597466c in_seq=0 out_seq=0 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28c2ffd640 1 -- 172.21.15.105:0/1798591892 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556e07de3210 con 0x556e07efba00 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28c2ffd640 10 client.16248 handle_command_reply: tid=0 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 client.16248 shutdown 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 -- 172.21.15.105:0/1798591892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 msgr2=0x556e07e8dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e8dfa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009a70 tx=0x7f28dc009fd0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 -- 172.21.15.105:0/1798591892 shutdown_connections 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556e07efba00 0x556e07efddf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556e07e84c60 0x556e07e8f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556e07e7faf0 0x556e07e8e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 --2- 172.21.15.105:0/1798591892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556e07e7f100 0x556e07e8dfa0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28c2ffd640 0 client.16248 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 -- 172.21.15.105:0/1798591892 >> 172.21.15.105:0/1798591892 conn(0x556e07e71e60 msgr2=0x556e07e72240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 -- 172.21.15.105:0/1798591892 shutdown_connections 2024-03-26T17:40:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 1 -- 172.21.15.105:0/1798591892 wait complete. 2024-03-26T17:40:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.139+0000 7f28ecd78740 20 client.16248 trim_cache size 0 max 16384 2024-03-26T17:40:39.140 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:40:39.140 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:40:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:40:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: "d68144a2-2262-4221-88f2-7bdac5ae6d60": { 2024-03-26T17:40:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:40:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "d68144a2-2262-4221-88f2-7bdac5ae6d60", 2024-03-26T17:40:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:40:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:40:39.141 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:40:39.141 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 -- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28cc0a2f20 msgr2=0x7f28cc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28cc0a2f20 0x7f28cc0a53e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f28d4012020 tx=0x7f28d40c6ab0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 -- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4102360 msgr2=0x7f28e4197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4102360 0x7f28e4197b40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f28d400e9f0 tx=0x7f28d400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 -- 172.21.15.105:0/2247585971 shutdown_connections 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28cc0a2f20 0x7f28cc0a53e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28e41075c0 0x7f28e419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28e4104c90 0x7f28e4198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 --2- 172.21.15.105:0/2247585971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28e4102360 0x7f28e4197b40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 -- 172.21.15.105:0/2247585971 >> 172.21.15.105:0/2247585971 conn(0x7f28e40fbd70 msgr2=0x7f28e41041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 -- 172.21.15.105:0/2247585971 shutdown_connections 2024-03-26T17:40:39.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:40:39.140+0000 7f28ea190640 1 -- 172.21.15.105:0/2247585971 wait complete. 2024-03-26T17:40:39.144 INFO:tasks.cephfs.filesystem:scrub status for tag:d68144a2-2262-4221-88f2-7bdac5ae6d60 - {'path': '/', 'tag': 'd68144a2-2262-4221-88f2-7bdac5ae6d60', 'options': 'recursive,force'} 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 -- 172.21.15.105:0/691160575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 msgr2=0x7f624c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 --2- 172.21.15.105:0/691160575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 0x7f624c101e90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6240009900 tx=0x7f624002bc30 comp rx=0 tx=0).stop 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 -- 172.21.15.105:0/691160575 shutdown_connections 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 --2- 172.21.15.105:0/691160575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c1023d0 0x7f624c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 --2- 172.21.15.105:0/691160575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 0x7f624c101e90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 --2- 172.21.15.105:0/691160575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c100820 0x7f624c100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 -- 172.21.15.105:0/691160575 >> 172.21.15.105:0/691160575 conn(0x7f624c0fbd90 msgr2=0x7f624c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 -- 172.21.15.105:0/691160575 shutdown_connections 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.251+0000 7f6251d52640 1 -- 172.21.15.105:0/691160575 wait complete. 2024-03-26T17:40:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f6251d52640 1 Processor -- start 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f6251d52640 1 -- start start 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f6251d52640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c100820 0x7f624c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f6251d52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 0x7f624c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f6251d52640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c1023d0 0x7f624c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f6251d52640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10d430 con 0x7f624c101a10 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f6251d52640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10d2b0 con 0x7f624c100820 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f6251d52640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f624c10d5b0 con 0x7f624c1023d0 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f624affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 0x7f624c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c100820 0x7f624c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c100820 0x7f624c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44370/0 (socket says 172.21.15.105:44370) 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f624affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 0x7f624c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59314/0 (socket says 172.21.15.105:59314) 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f624b7fe640 1 -- 172.21.15.105:0/3066064402 learned_addr learned my addr 172.21.15.105:0/3066064402 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.252+0000 7f624bfff640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c1023d0 0x7f624c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f624b7fe640 1 -- 172.21.15.105:0/3066064402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c1023d0 msgr2=0x7f624c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f624b7fe640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c1023d0 0x7f624c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f624b7fe640 1 -- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 msgr2=0x7f624c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f624b7fe640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 0x7f624c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f624b7fe640 1 -- 172.21.15.105:0/3066064402 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c19f6c0 con 0x7f624c100820 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f624b7fe640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c100820 0x7f624c197aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f623c00e9c0 tx=0x7f623c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f6248ff9640 1 -- 172.21.15.105:0/3066064402 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c00cde0 con 0x7f624c100820 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f6251d52640 1 -- 172.21.15.105:0/3066064402 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c19f9b0 con 0x7f624c100820 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f6248ff9640 1 -- 172.21.15.105:0/3066064402 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f623c0108e0 con 0x7f624c100820 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f6248ff9640 1 -- 172.21.15.105:0/3066064402 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c0188e0 con 0x7f624c100820 2024-03-26T17:40:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.253+0000 7f6251d52640 1 -- 172.21.15.105:0/3066064402 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c19fed0 con 0x7f624c100820 2024-03-26T17:40:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6248ff9640 1 -- 172.21.15.105:0/3066064402 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f623c01f020 con 0x7f624c100820 2024-03-26T17:40:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6254a10740 1 Processor -- start 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6254a10740 1 -- start start 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6254a10740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x56097351e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6254a10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x560973523ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6254a10740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x5609735265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6254a10740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609734830b0 con 0x56097351e5d0 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6254a10740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56097342b960 con 0x560973524130 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f6254a10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56097347f060 con 0x56097351efc0 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f624bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x56097351e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f624bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x56097351e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59328/0 (socket says 172.21.15.105:59328) 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f624bfff640 1 -- 172.21.15.105:0/515598467 learned_addr learned my addr 172.21.15.105:0/515598467 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f624affd640 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x5609735265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.256+0000 7f624b7fe640 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x560973523ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f624bfff640 1 -- 172.21.15.105:0/515598467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 msgr2=0x560973523ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f624bfff640 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x560973523ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f6248ff9640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fc0 0x7f62340a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f624bfff640 1 -- 172.21.15.105:0/515598467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 msgr2=0x5609735265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f624bfff640 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x5609735265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f624bfff640 1 -- 172.21.15.105:0/515598467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097343dc60 con 0x56097351e5d0 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f624affd640 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x5609735265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f6248ff9640 1 -- 172.21.15.105:0/3066064402 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f623c0cf5b0 con 0x7f624c100820 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f624bfff640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fc0 0x7f62340a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f624bfff640 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x56097351e9f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6244009fd0 tx=0x7f62440118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cc8398bc78d589 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f6231ffb640 1 -- 172.21.15.105:0/515598467 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244004050 con 0x56097351e5d0 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f6231ffb640 1 -- 172.21.15.105:0/515598467 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6244004720 con 0x56097351e5d0 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.257+0000 7f6231ffb640 1 -- 172.21.15.105:0/515598467 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244019400 con 0x56097351e5d0 2024-03-26T17:40:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 -- 172.21.15.105:0/515598467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 msgr2=0x56097351e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x56097351e9f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6244009fd0 tx=0x7f62440118d0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f624bfff640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fc0 0x7f62340a5480 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f624401d000 tx=0x7f62440196e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 -- 172.21.15.105:0/515598467 shutdown_connections 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x5609735265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x560973523ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 --2- 172.21.15.105:0/515598467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x56097351e9f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 -- 172.21.15.105:0/515598467 >> 172.21.15.105:0/515598467 conn(0x560973519b40 msgr2=0x560973519f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 -- 172.21.15.105:0/515598467 shutdown_connections 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 -- 172.21.15.105:0/515598467 wait complete. 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 -- start start 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.258+0000 7f6254a10740 1 Processor -- start 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f6254a10740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x560973528310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f6254a10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x560973528850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f6254a10740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x56097352f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f6254a10740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609735270e0 con 0x56097351e5d0 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f6254a10740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56097343edb0 con 0x560973524130 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f6254a10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560973527260 con 0x56097351efc0 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x560973528310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f6254a10740 10 client.0 _ll_register_callbacks cb 0x560973443110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x560973528310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59342/0 (socket says 172.21.15.105:59342) 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 -- 172.21.15.105:0/530789835 learned_addr learned my addr 172.21.15.105:0/530789835 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624affd640 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x560973528850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624bfff640 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x56097352f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 -- 172.21.15.105:0/530789835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 msgr2=0x560973528850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x560973528850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 -- 172.21.15.105:0/530789835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 msgr2=0x56097352f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x56097352f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 -- 172.21.15.105:0/530789835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560973197210 con 0x56097351e5d0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f624b7fe640 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x560973528310 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f623c098670 tx=0x7f623c09b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebe396d1a62c510 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f622affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.259+0000 7f622affd640 1 -- 172.21.15.105:0/530789835 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c09c3a0 con 0x56097351e5d0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f622affd640 1 -- 172.21.15.105:0/530789835 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f623c0a4220 con 0x56097351e5d0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 1 -- 172.21.15.105:0/530789835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6244004720 con 0x56097351e5d0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f622affd640 1 -- 172.21.15.105:0/530789835 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f623c09bca0 con 0x56097351e5d0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 1 -- 172.21.15.105:0/530789835 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560973263620 con 0x56097351e5d0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f622affd640 1 -- 172.21.15.105:0/530789835 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6244004720 con 0x56097351e5d0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f622affd640 1 -- 172.21.15.105:0/530789835 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f623c0b1030 con 0x56097351e5d0 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 20 client.16260 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560973534190 0x5609735365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 4 client.16260 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.260+0000 7f6254a10740 1 -- 172.21.15.105:0/530789835 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609734905f0 con 0x560973534190 2024-03-26T17:40:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.261+0000 7f624affd640 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560973534190 0x5609735365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.261+0000 7f622affd640 10 client.16260 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.261+0000 7f624affd640 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560973534190 0x5609735365a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d078c1ea0f3583 server_cookie=202cf5da4ece4351 in_seq=0 out_seq=0 2024-03-26T17:40:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f622affd640 1 -- 172.21.15.105:0/530789835 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5609734905f0 con 0x560973534190 2024-03-26T17:40:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f622affd640 10 client.16260 handle_command_reply: tid=0 2024-03-26T17:40:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 client.16260 shutdown 2024-03-26T17:40:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 -- 172.21.15.105:0/530789835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 msgr2=0x560973528310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x560973528310 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f623c098670 tx=0x7f623c09b8a0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 -- 172.21.15.105:0/530789835 shutdown_connections 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560973534190 0x5609735365a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x56097352f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x560973528850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 --2- 172.21.15.105:0/530789835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351e5d0 0x560973528310 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 -- 172.21.15.105:0/530789835 >> 172.21.15.105:0/530789835 conn(0x560973519b40 msgr2=0x560973518df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f622affd640 0 client.16260 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 -- 172.21.15.105:0/530789835 shutdown_connections 2024-03-26T17:40:39.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 1 -- 172.21.15.105:0/530789835 wait complete. 2024-03-26T17:40:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.262+0000 7f6254a10740 20 client.16260 trim_cache size 0 max 16384 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.280+0000 7f6254a10740 1 Processor -- start 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.280+0000 7f6254a10740 1 -- start start 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.280+0000 7f6254a10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351e5d0 0x5609735287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.280+0000 7f6254a10740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351efc0 0x56097352f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.280+0000 7f6254a10740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x560973520050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.280+0000 7f6254a10740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f624c1a6ba0 con 0x56097351efc0 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.280+0000 7f6254a10740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f624c1a86e0 con 0x560973524130 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.280+0000 7f6254a10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f624c1a7270 con 0x56097351e5d0 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351efc0 0x56097352f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351efc0 0x56097352f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59350/0 (socket says 172.21.15.105:59350) 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624affd640 1 -- 172.21.15.105:0/2672556276 learned_addr learned my addr 172.21.15.105:0/2672556276 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624b7fe640 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351e5d0 0x5609735287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624bfff640 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x560973520050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624b7fe640 1 -- 172.21.15.105:0/2672556276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 msgr2=0x560973520050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624b7fe640 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x560973520050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624b7fe640 1 -- 172.21.15.105:0/2672556276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351efc0 msgr2=0x56097352f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624b7fe640 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351efc0 0x56097352f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624b7fe640 1 -- 172.21.15.105:0/2672556276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56097351f3a0 con 0x56097351e5d0 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624affd640 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351efc0 0x56097352f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f624b7fe640 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351e5d0 0x5609735287b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f623c0d2e30 tx=0x7f623c00efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15c02b9fa01d1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f62317fa640 1 -- 172.21.15.105:0/2672556276 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c09ba70 con 0x56097351e5d0 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f62317fa640 1 -- 172.21.15.105:0/2672556276 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f623c09c3a0 con 0x56097351e5d0 2024-03-26T17:40:39.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.281+0000 7f62317fa640 1 -- 172.21.15.105:0/2672556276 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c0cf110 con 0x56097351e5d0 2024-03-26T17:40:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 -- 172.21.15.105:0/2672556276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351e5d0 msgr2=0x5609735287b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351e5d0 0x5609735287b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f623c0d2e30 tx=0x7f623c00efb0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 -- 172.21.15.105:0/2672556276 shutdown_connections 2024-03-26T17:40:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560973524130 0x560973520050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56097351efc0 0x56097352f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 --2- 172.21.15.105:0/2672556276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351e5d0 0x5609735287b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 -- 172.21.15.105:0/2672556276 >> 172.21.15.105:0/2672556276 conn(0x560973519b40 msgr2=0x56097351f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 -- 172.21.15.105:0/2672556276 shutdown_connections 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 -- 172.21.15.105:0/2672556276 wait complete. 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.282+0000 7f6254a10740 1 -- start start 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f6254a10740 1 Processor -- start 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f6254a10740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097351e5d0 0x56097352cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f6254a10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x56097352d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f6254a10740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560973524130 0x56097352e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f6254a10740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f624c1a8010 con 0x560973524130 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f6254a10740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f624c1a7940 con 0x56097351e5d0 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f6254a10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f624c1a4b80 con 0x56097351efc0 2024-03-26T17:40:39.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f6254a10740 10 client.0 _ll_register_callbacks cb 0x7f623c097f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x56097352d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560973524130 0x56097352e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097351e5d0 0x56097352cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560973524130 0x56097352e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59364/0 (socket says 172.21.15.105:59364) 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x56097352d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42094/0 (socket says 172.21.15.105:42094) 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097351e5d0 0x56097352cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44414/0 (socket says 172.21.15.105:44414) 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 -- 172.21.15.105:0/4174867739 learned_addr learned my addr 172.21.15.105:0/4174867739 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 -- 172.21.15.105:0/4174867739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 msgr2=0x56097352d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x56097352d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 -- 172.21.15.105:0/4174867739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097351e5d0 msgr2=0x56097352cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097351e5d0 0x56097352cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 -- 172.21.15.105:0/4174867739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56097352e730 con 0x560973524130 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.283+0000 7f624bfff640 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560973524130 0x56097352e140 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6244009230 tx=0x7f6244025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835d5635bc1a9e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f622affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f622affd640 1 -- 172.21.15.105:0/4174867739 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244019400 con 0x560973524130 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f622affd640 1 -- 172.21.15.105:0/4174867739 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f6244009b30 con 0x560973524130 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 1 -- 172.21.15.105:0/4174867739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f623c09c3a0 con 0x560973524130 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f622affd640 1 -- 172.21.15.105:0/4174867739 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6244003d10 con 0x560973524130 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 1 -- 172.21.15.105:0/4174867739 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56097352fab0 con 0x560973524130 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f622affd640 1 -- 172.21.15.105:0/4174867739 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f623c09c3a0 con 0x560973524130 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f622affd640 1 -- 172.21.15.105:0/4174867739 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f624402dba0 con 0x560973524130 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 20 client.16266 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56097359aa80 0x56097359cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 4 client.16266 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:40:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f6254a10740 1 -- 172.21.15.105:0/4174867739 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56097348a940 con 0x56097359aa80 2024-03-26T17:40:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.284+0000 7f624b7fe640 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56097359aa80 0x56097359cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.285+0000 7f622affd640 10 client.16266 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:39.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.285+0000 7f624b7fe640 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56097359aa80 0x56097359cea0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e6749f604c1245 server_cookie=86a3c8253ac51f58 in_seq=0 out_seq=0 2024-03-26T17:40:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.329+0000 7f622affd640 1 -- 172.21.15.105:0/4174867739 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56097348a940 con 0x56097359aa80 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.329+0000 7f622affd640 10 client.16266 handle_command_reply: tid=0 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.329+0000 7f6254a10740 1 client.16266 shutdown 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 -- 172.21.15.105:0/4174867739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560973524130 msgr2=0x56097352e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560973524130 0x56097352e140 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6244009230 tx=0x7f6244025000 comp rx=0 tx=0).stop 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 -- 172.21.15.105:0/4174867739 shutdown_connections 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56097359aa80 0x56097359cea0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560973524130 0x56097352e140 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56097351efc0 0x56097352d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 --2- 172.21.15.105:0/4174867739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56097351e5d0 0x56097352cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 -- 172.21.15.105:0/4174867739 >> 172.21.15.105:0/4174867739 conn(0x560973519b40 msgr2=0x5609735285c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f622affd640 0 client.16266 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 -- 172.21.15.105:0/4174867739 shutdown_connections 2024-03-26T17:40:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 1 -- 172.21.15.105:0/4174867739 wait complete. 2024-03-26T17:40:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6254a10740 20 client.16266 trim_cache size 0 max 16384 2024-03-26T17:40:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:40:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:40:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:40:39.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:40:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6251d52640 1 -- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fc0 msgr2=0x7f62340a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.330+0000 7f6251d52640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fc0 0x7f62340a5480 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f624401d000 tx=0x7f62440196e0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 -- 172.21.15.105:0/3066064402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c100820 msgr2=0x7f624c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c100820 0x7f624c197aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f623c00e9c0 tx=0x7f623c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:40:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 -- 172.21.15.105:0/3066064402 shutdown_connections 2024-03-26T17:40:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f62340a2fc0 0x7f62340a5480 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f624c1023d0 0x7f624c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f624c101a10 0x7f624c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 --2- 172.21.15.105:0/3066064402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f624c100820 0x7f624c197aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 -- 172.21.15.105:0/3066064402 >> 172.21.15.105:0/3066064402 conn(0x7f624c0fbd90 msgr2=0x7f624c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 -- 172.21.15.105:0/3066064402 shutdown_connections 2024-03-26T17:40:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:39.331+0000 7f6251d52640 1 -- 172.21.15.105:0/3066064402 wait complete. 2024-03-26T17:40:40.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 49669 3.46 MB/sec warmup 83 sec latency 29.642 ms 2024-03-26T17:40:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:39 smithi144 ceph-mon[36095]: pgmap v416: 97 pgs: 97 active+clean; 27 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:40:40.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:39 smithi105 ceph-mon[32280]: pgmap v416: 97 pgs: 97 active+clean; 27 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:40:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:39 smithi135 ceph-mon[35636]: pgmap v416: 97 pgs: 97 active+clean; 27 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:40:41.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 50130 3.44 MB/sec warmup 84 sec latency 16.108 ms 2024-03-26T17:40:41.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:40 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:40:41.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:40 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:41.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:40 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:40:41.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:40 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:41.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:40 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:40:41.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:40 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:41.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:40 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:40:41.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:40 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:41.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:40 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:40:41.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:40 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:41.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:40 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:40:41.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:40 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:40:42.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 50664 3.41 MB/sec warmup 85 sec latency 19.892 ms 2024-03-26T17:40:42.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:41 smithi144 ceph-mon[36095]: pgmap v417: 97 pgs: 97 active+clean; 28 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:40:42.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:41 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:40:42.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:41 smithi105 ceph-mon[32280]: pgmap v417: 97 pgs: 97 active+clean; 28 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:40:42.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:41 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:40:42.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:41 smithi135 ceph-mon[35636]: pgmap v417: 97 pgs: 97 active+clean; 28 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:40:42.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:41 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:40:43.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 51313 3.42 MB/sec warmup 86 sec latency 17.384 ms 2024-03-26T17:40:44.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 52025 3.40 MB/sec warmup 87 sec latency 19.587 ms 2024-03-26T17:40:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:43 smithi144 ceph-mon[36095]: pgmap v418: 97 pgs: 97 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:40:44.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:43 smithi105 ceph-mon[32280]: pgmap v418: 97 pgs: 97 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:40:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:43 smithi135 ceph-mon[35636]: pgmap v418: 97 pgs: 97 active+clean; 24 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:40:45.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 52683 3.42 MB/sec warmup 88 sec latency 61.209 ms 2024-03-26T17:40:46.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 53261 3.44 MB/sec warmup 89 sec latency 15.415 ms 2024-03-26T17:40:46.101 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:45 smithi144 ceph-mon[36095]: pgmap v419: 97 pgs: 97 active+clean; 26 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:40:46.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:45 smithi105 ceph-mon[32280]: pgmap v419: 97 pgs: 97 active+clean; 26 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:40:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:45 smithi135 ceph-mon[35636]: pgmap v419: 97 pgs: 97 active+clean; 26 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:40:47.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 53687 3.42 MB/sec warmup 90 sec latency 14.898 ms 2024-03-26T17:40:47.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:40:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:46 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:40:47.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:46 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:40:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:47.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:46 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:48.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 54178 3.39 MB/sec warmup 91 sec latency 17.315 ms 2024-03-26T17:40:48.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:47 smithi144 ceph-mon[36095]: pgmap v420: 97 pgs: 97 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:40:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:40:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:40:48.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:47 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:48.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:47 smithi105 ceph-mon[32280]: pgmap v420: 97 pgs: 97 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:40:48.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:40:48.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:40:48.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:47 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:47 smithi135 ceph-mon[35636]: pgmap v420: 97 pgs: 97 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:40:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:40:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:40:48.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:47 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:40:49.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 54848 3.40 MB/sec warmup 92 sec latency 17.049 ms 2024-03-26T17:40:50.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 55565 3.38 MB/sec warmup 93 sec latency 16.308 ms 2024-03-26T17:40:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:49 smithi144 ceph-mon[36095]: pgmap v421: 97 pgs: 97 active+clean; 25 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:40:50.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:49 smithi105 ceph-mon[32280]: pgmap v421: 97 pgs: 97 active+clean; 25 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:40:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:49 smithi135 ceph-mon[35636]: pgmap v421: 97 pgs: 97 active+clean; 25 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:40:51.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 56250 3.41 MB/sec warmup 94 sec latency 17.260 ms 2024-03-26T17:40:52.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 56632 3.39 MB/sec warmup 95 sec latency 136.256 ms 2024-03-26T17:40:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:51 smithi144 ceph-mon[36095]: pgmap v422: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:40:52.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:51 smithi105 ceph-mon[32280]: pgmap v422: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:40:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:51 smithi135 ceph-mon[35636]: pgmap v422: 97 pgs: 97 active+clean; 27 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:40:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 -- 172.21.15.105:0/2500100581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109a30 msgr2=0x7f2620110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 --2- 172.21.15.105:0/2500100581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109a30 0x7f2620110350 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2618009ca0 tx=0x7f261802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 -- 172.21.15.105:0/2500100581 shutdown_connections 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 --2- 172.21.15.105:0/2500100581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109a30 0x7f2620110350 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 --2- 172.21.15.105:0/2500100581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2620109070 0x7f26201094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 --2- 172.21.15.105:0/2500100581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2620107f20 0x7f2620108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 -- 172.21.15.105:0/2500100581 >> 172.21.15.105:0/2500100581 conn(0x7f2620076ee0 msgr2=0x7f2620079300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 -- 172.21.15.105:0/2500100581 shutdown_connections 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.439+0000 7f2627452640 1 -- 172.21.15.105:0/2500100581 wait complete. 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f2627452640 1 Processor -- start 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f2627452640 1 -- start start 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f2627452640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2620107f20 0x7f262019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f2627452640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109070 0x7f262019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f2627452640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2620109a30 0x7f26201a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f2627452640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2620113a80 con 0x7f2620109070 2024-03-26T17:40:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f2627452640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2620113900 con 0x7f2620107f20 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f2627452640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2620113c00 con 0x7f2620109a30 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26249ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109070 0x7f262019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26251ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2620107f20 0x7f262019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26249ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109070 0x7f262019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57502/0 (socket says 172.21.15.105:57502) 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26259ef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2620109a30 0x7f26201a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26249ed640 1 -- 172.21.15.105:0/3778881462 learned_addr learned my addr 172.21.15.105:0/3778881462 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26259ef640 1 -- 172.21.15.105:0/3778881462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2620107f20 msgr2=0x7f262019e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26259ef640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2620107f20 0x7f262019e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26259ef640 1 -- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109070 msgr2=0x7f262019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26259ef640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109070 0x7f262019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26259ef640 1 -- 172.21.15.105:0/3778881462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26201a5c90 con 0x7f2620109a30 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26249ed640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109070 0x7f262019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.440+0000 7f26259ef640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2620109a30 0x7f26201a5620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f261802c630 tx=0x7f26180328a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.441+0000 7f26167fc640 1 -- 172.21.15.105:0/3778881462 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618041020 con 0x7f2620109a30 2024-03-26T17:40:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.441+0000 7f2627452640 1 -- 172.21.15.105:0/3778881462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26201a5f20 con 0x7f2620109a30 2024-03-26T17:40:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.441+0000 7f26167fc640 1 -- 172.21.15.105:0/3778881462 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2618003e20 con 0x7f2620109a30 2024-03-26T17:40:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.441+0000 7f26167fc640 1 -- 172.21.15.105:0/3778881462 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618039040 con 0x7f2620109a30 2024-03-26T17:40:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.441+0000 7f2627452640 1 -- 172.21.15.105:0/3778881462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201a63e0 con 0x7f2620109a30 2024-03-26T17:40:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.443+0000 7f262a03e740 1 Processor -- start 2024-03-26T17:40:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26167fc640 1 -- 172.21.15.105:0/3778881462 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2618044020 con 0x7f2620109a30 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f262a03e740 1 -- start start 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f262a03e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06ed5d0 0x5613f06ed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f262a03e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f262a03e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f262a03e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5613f06520b0 con 0x5613f06f3130 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f262a03e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613f05fa960 con 0x5613f06edfc0 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f262a03e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613f064e060 con 0x5613f06ed5d0 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26251ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26249ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06ed5d0 0x5613f06ed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26251ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53232/0 (socket says 172.21.15.105:53232) 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26251ee640 1 -- 172.21.15.105:0/495120910 learned_addr learned my addr 172.21.15.105:0/495120910 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26249ed640 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26167fc640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f260c0a2fc0 0x7f260c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26251ee640 1 -- 172.21.15.105:0/495120910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06ed5d0 msgr2=0x5613f06ed9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26259ef640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f260c0a2fc0 0x7f260c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26251ee640 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06ed5d0 0x5613f06ed9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26251ee640 1 -- 172.21.15.105:0/495120910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 msgr2=0x5613f06f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26251ee640 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26251ee640 1 -- 172.21.15.105:0/495120910 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f060cc60 con 0x5613f06edfc0 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.444+0000 7f26249ed640 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06ed5d0 0x5613f06ed9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.445+0000 7f26167fc640 1 -- 172.21.15.105:0/3778881462 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f26180f15a0 con 0x7f2620109a30 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.445+0000 7f26251ee640 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06f2ac0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f261c009cf0 tx=0x7f261c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26cee2af15cdbec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.445+0000 7f26259ef640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f260c0a2fc0 0x7f260c0a5480 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f26180098d0 tx=0x7f26180095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.445+0000 7f260b7fe640 1 -- 172.21.15.105:0/495120910 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c01e070 con 0x5613f06edfc0 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.445+0000 7f260b7fe640 1 -- 172.21.15.105:0/495120910 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261c019ae0 con 0x5613f06edfc0 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.445+0000 7f260b7fe640 1 -- 172.21.15.105:0/495120910 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c00e8c0 con 0x5613f06edfc0 2024-03-26T17:40:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 -- 172.21.15.105:0/495120910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 msgr2=0x5613f06f2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06f2ac0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f261c009cf0 tx=0x7f261c011810 comp rx=0 tx=0).stop 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 -- 172.21.15.105:0/495120910 shutdown_connections 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06f2ac0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 --2- 172.21.15.105:0/495120910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06ed5d0 0x5613f06ed9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 -- 172.21.15.105:0/495120910 >> 172.21.15.105:0/495120910 conn(0x5613f06e8b40 msgr2=0x5613f06e8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 -- 172.21.15.105:0/495120910 shutdown_connections 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 -- 172.21.15.105:0/495120910 wait complete. 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 -- start start 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.446+0000 7f262a03e740 1 Processor -- start 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f262a03e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f262a03e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06edfc0 0x5613f06f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f262a03e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 0x5613f06fe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f262a03e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5613f06f60e0 con 0x5613f06edfc0 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f262a03e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613f060ddb0 con 0x5613f06ed5d0 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f262a03e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613f06f6260 con 0x5613f06f3130 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f262a03e740 10 client.0 _ll_register_callbacks cb 0x5613f0612110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53244/0 (socket says 172.21.15.105:53244) 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 -- 172.21.15.105:0/4231493544 learned_addr learned my addr 172.21.15.105:0/4231493544 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 0x5613f06fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26251ee640 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06edfc0 0x5613f06f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 -- 172.21.15.105:0/4231493544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 msgr2=0x5613f06fe960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 0x5613f06fe960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 -- 172.21.15.105:0/4231493544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06edfc0 msgr2=0x5613f06f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06edfc0 0x5613f06f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.447+0000 7f26249ed640 1 -- 172.21.15.105:0/4231493544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f0366210 con 0x5613f06ed5d0 2024-03-26T17:40:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f26249ed640 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06f7450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f261000f4a0 tx=0x7f2610004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b61273da137007 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f2608ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f2608ff9640 1 -- 172.21.15.105:0/4231493544 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610004450 con 0x5613f06ed5d0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f2608ff9640 1 -- 172.21.15.105:0/4231493544 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f2610005020 con 0x5613f06ed5d0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f262a03e740 1 -- 172.21.15.105:0/4231493544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f261c019ae0 con 0x5613f06ed5d0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f2608ff9640 1 -- 172.21.15.105:0/4231493544 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2610029050 con 0x5613f06ed5d0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f262a03e740 10 client.26135 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f262a03e740 1 -- 172.21.15.105:0/4231493544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5613f0432620 con 0x5613f06ed5d0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.448+0000 7f2608ff9640 1 -- 172.21.15.105:0/4231493544 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261c019ae0 con 0x5613f06ed5d0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f2608ff9640 1 -- 172.21.15.105:0/4231493544 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2610004bf0 con 0x5613f06ed5d0 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f262a03e740 10 client.26135 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f262a03e740 10 client.26135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f262a03e740 20 client.26135 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f262a03e740 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613f0703190 0x5613f07055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f262a03e740 4 client.26135 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:40:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f262a03e740 1 -- 172.21.15.105:0/4231493544 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613f065f5f0 con 0x5613f0703190 2024-03-26T17:40:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f26251ee640 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613f0703190 0x5613f07055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f26251ee640 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613f0703190 0x5613f07055c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9058bbdf24a8f4 server_cookie=15ff6a6b8f4a1dca in_seq=0 out_seq=0 2024-03-26T17:40:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.449+0000 7f2608ff9640 10 client.26135 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f2608ff9640 1 -- 172.21.15.105:0/4231493544 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5613f065f5f0 con 0x5613f0703190 2024-03-26T17:40:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f2608ff9640 10 client.26135 handle_command_reply: tid=0 2024-03-26T17:40:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 client.26135 shutdown 2024-03-26T17:40:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 -- 172.21.15.105:0/4231493544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 msgr2=0x5613f06f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06f7450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f261000f4a0 tx=0x7f2610004730 comp rx=0 tx=0).stop 2024-03-26T17:40:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 -- 172.21.15.105:0/4231493544 shutdown_connections 2024-03-26T17:40:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613f0703190 0x5613f07055c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 0x5613f06fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06edfc0 0x5613f06f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f2608ff9640 0 client.26135 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 --2- 172.21.15.105:0/4231493544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06f7450 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 -- 172.21.15.105:0/4231493544 >> 172.21.15.105:0/4231493544 conn(0x5613f06e8b40 msgr2=0x5613f06e7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 -- 172.21.15.105:0/4231493544 shutdown_connections 2024-03-26T17:40:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.450+0000 7f262a03e740 1 -- 172.21.15.105:0/4231493544 wait complete. 2024-03-26T17:40:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.451+0000 7f262a03e740 20 client.26135 trim_cache size 0 max 16384 2024-03-26T17:40:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f262a03e740 1 Processor -- start 2024-03-26T17:40:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f262a03e740 1 -- start start 2024-03-26T17:40:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f262a03e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f262a03e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06edfc0 0x5613f06fe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f262a03e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06e7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f262a03e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26201af410 con 0x5613f06f3130 2024-03-26T17:40:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f262a03e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b0f50 con 0x5613f06ed5d0 2024-03-26T17:40:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f262a03e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26201afae0 con 0x5613f06edfc0 2024-03-26T17:40:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06e7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26251ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06edfc0 0x5613f06fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06e7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57512/0 (socket says 172.21.15.105:57512) 2024-03-26T17:40:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26251ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06edfc0 0x5613f06fe920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33826/0 (socket says 172.21.15.105:33826) 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 -- 172.21.15.105:0/2939184562 learned_addr learned my addr 172.21.15.105:0/2939184562 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 -- 172.21.15.105:0/2939184562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06edfc0 msgr2=0x5613f06fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06edfc0 0x5613f06fe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 -- 172.21.15.105:0/2939184562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 msgr2=0x5613f06ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 -- 172.21.15.105:0/2939184562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f06f20f0 con 0x5613f06f3130 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26251ee640 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06edfc0 0x5613f06fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.467+0000 7f26249ed640 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06e7560 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5613f06ee730 tx=0x7f26100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824d48f7b6b00f3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f260affd640 1 -- 172.21.15.105:0/2939184562 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610003c30 con 0x5613f06f3130 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f260affd640 1 -- 172.21.15.105:0/2939184562 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2610003dd0 con 0x5613f06f3130 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f260affd640 1 -- 172.21.15.105:0/2939184562 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610005220 con 0x5613f06f3130 2024-03-26T17:40:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 -- 172.21.15.105:0/2939184562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 msgr2=0x5613f06e7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06e7560 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5613f06ee730 tx=0x7f26100046c0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 -- 172.21.15.105:0/2939184562 shutdown_connections 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06f3130 0x5613f06e7560 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06edfc0 0x5613f06fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 --2- 172.21.15.105:0/2939184562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06ed5d0 0x5613f06ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 -- 172.21.15.105:0/2939184562 >> 172.21.15.105:0/2939184562 conn(0x5613f06e8b40 msgr2=0x5613f06f76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 -- 172.21.15.105:0/2939184562 shutdown_connections 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.468+0000 7f262a03e740 1 -- 172.21.15.105:0/2939184562 wait complete. 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 1 -- start start 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 1 Processor -- start 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06ed5d0 0x5613f06fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 0x5613f06fd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b0880 con 0x5613f06ed5d0 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b01b0 con 0x5613f06edfc0 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26201ad3f0 con 0x5613f06f3130 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f262a03e740 10 client.0 _ll_register_callbacks cb 0x7f2610011970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f26249ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06ed5d0 0x5613f06fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f26251ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f26249ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06ed5d0 0x5613f06fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57526/0 (socket says 172.21.15.105:57526) 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f26249ed640 1 -- 172.21.15.105:0/4162373775 learned_addr learned my addr 172.21.15.105:0/4162373775 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.469+0000 7f26249ed640 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 0x5613f06fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f26251ee640 1 -- 172.21.15.105:0/4162373775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 msgr2=0x5613f06fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f26251ee640 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 0x5613f06fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f26251ee640 1 -- 172.21.15.105:0/4162373775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06ed5d0 msgr2=0x5613f06fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f26251ee640 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06ed5d0 0x5613f06fbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f26251ee640 1 -- 172.21.15.105:0/4162373775 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f06fd7f0 con 0x5613f06edfc0 2024-03-26T17:40:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f26251ee640 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06fc1f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5613f06e9ab0 tx=0x7f261c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5869b587267430f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f2608ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f2608ff9640 1 -- 172.21.15.105:0/4162373775 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c02f060 con 0x5613f06edfc0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f262a03e740 1 -- 172.21.15.105:0/4162373775 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2610003dd0 con 0x5613f06edfc0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f2608ff9640 1 -- 172.21.15.105:0/4162373775 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8293+0+0 (secure 0 0 0) 0x7f261c0054c0 con 0x5613f06edfc0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f2608ff9640 1 -- 172.21.15.105:0/4162373775 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f261c01e040 con 0x5613f06edfc0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f262a03e740 10 client.26141 fetch_fsmap learned FSMap version 29 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.470+0000 7f262a03e740 1 -- 172.21.15.105:0/4162373775 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5613f06feb70 con 0x5613f06edfc0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f2608ff9640 1 -- 172.21.15.105:0/4162373775 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2610003dd0 con 0x5613f06edfc0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f2608ff9640 1 -- 172.21.15.105:0/4162373775 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f261c003780 con 0x5613f06edfc0 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f262a03e740 10 client.26141 fetch_fsmap finished waiting for FSMap version 29 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f262a03e740 10 client.26141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f262a03e740 20 client.26141 populate_metadata read hostname 'smithi105' 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f262a03e740 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613f0703190 0x5613f076b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:40:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f262a03e740 4 client.26141 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:40:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f262a03e740 1 -- 172.21.15.105:0/4162373775 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613f0659940 con 0x5613f0703190 2024-03-26T17:40:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f26249ed640 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613f0703190 0x5613f076b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:40:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f2608ff9640 10 client.26141 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.471+0000 7f26249ed640 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613f0703190 0x5613f076b9d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c2f3955900f42c server_cookie=24d4e63a975b257d in_seq=0 out_seq=0 2024-03-26T17:40:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.513+0000 7f2608ff9640 1 -- 172.21.15.105:0/4162373775 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613f0659940 con 0x5613f0703190 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.513+0000 7f2608ff9640 10 client.26141 handle_command_reply: tid=0 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.513+0000 7f262a03e740 1 client.26141 shutdown 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 -- 172.21.15.105:0/4162373775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 msgr2=0x5613f06fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06fc1f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5613f06e9ab0 tx=0x7f261c005d30 comp rx=0 tx=0).stop 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 -- 172.21.15.105:0/4162373775 shutdown_connections 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613f0703190 0x5613f076b9d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613f06f3130 0x5613f06fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613f06edfc0 0x5613f06fc1f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f2608ff9640 0 client.26141 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 --2- 172.21.15.105:0/4162373775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613f06ed5d0 0x5613f06fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 -- 172.21.15.105:0/4162373775 >> 172.21.15.105:0/4162373775 conn(0x5613f06e8b40 msgr2=0x5613f06ebda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:52.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 -- 172.21.15.105:0/4162373775 shutdown_connections 2024-03-26T17:40:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 1 -- 172.21.15.105:0/4162373775 wait complete. 2024-03-26T17:40:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.514+0000 7f262a03e740 20 client.26141 trim_cache size 0 max 16384 2024-03-26T17:40:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:40:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:40:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:40:52.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 -- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f260c0a2fc0 msgr2=0x7f260c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f260c0a2fc0 0x7f260c0a5480 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f26180098d0 tx=0x7f26180095c0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 -- 172.21.15.105:0/3778881462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2620109a30 msgr2=0x7f26201a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2620109a30 0x7f26201a5620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f261802c630 tx=0x7f26180328a0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 -- 172.21.15.105:0/3778881462 shutdown_connections 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f260c0a2fc0 0x7f260c0a5480 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2620109a30 0x7f26201a5620 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2620109070 0x7f262019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 --2- 172.21.15.105:0/3778881462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2620107f20 0x7f262019e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:40:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 -- 172.21.15.105:0/3778881462 >> 172.21.15.105:0/3778881462 conn(0x7f2620076ee0 msgr2=0x7f262010c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:40:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 -- 172.21.15.105:0/3778881462 shutdown_connections 2024-03-26T17:40:52.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:40:52.515+0000 7f2627452640 1 -- 172.21.15.105:0/3778881462 wait complete. 2024-03-26T17:40:53.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 57195 3.41 MB/sec warmup 96 sec latency 15.363 ms 2024-03-26T17:40:53.415 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setuser ceph since I am not root 2024-03-26T17:40:53.415 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setgroup ceph since I am not root 2024-03-26T17:40:53.416 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/2926543053 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf9ab73a40 con 0x55bf9ab46c00 2024-03-26T17:40:53.416 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T17:40:54.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 57683 3.38 MB/sec warmup 97 sec latency 22.775 ms 2024-03-26T17:40:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:53 smithi144 ceph-mon[36095]: pgmap v423: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:40:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:53 smithi144 ceph-mon[36095]: daemon mds.e restarted 2024-03-26T17:40:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:53 smithi144 ceph-mon[36095]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T17:40:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:53 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/260658746,v1:172.21.15.135:6835/260658746] up:boot 2024-03-26T17:40:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:53 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:40:54.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:40:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:53 smithi105 ceph-mon[32280]: pgmap v423: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:40:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:53 smithi105 ceph-mon[32280]: daemon mds.e restarted 2024-03-26T17:40:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:53 smithi105 ceph-mon[32280]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T17:40:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:53 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/260658746,v1:172.21.15.135:6835/260658746] up:boot 2024-03-26T17:40:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:53 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:40:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:40:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-mon[35636]: pgmap v423: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:40:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-mon[35636]: daemon mds.e restarted 2024-03-26T17:40:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-mon[35636]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T17:40:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/260658746,v1:172.21.15.135:6835/260658746] up:boot 2024-03-26T17:40:54.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:40:54.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:40:55.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 58309 3.39 MB/sec warmup 98 sec latency 16.293 ms 2024-03-26T17:40:56.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 58995 3.36 MB/sec warmup 99 sec latency 16.145 ms 2024-03-26T17:40:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:55 smithi144 ceph-mon[36095]: pgmap v425: 97 pgs: 97 active+clean; 24 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-03-26T17:40:56.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:55 smithi105 ceph-mon[32280]: pgmap v425: 97 pgs: 97 active+clean; 24 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-03-26T17:40:56.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:55 smithi135 ceph-mon[35636]: pgmap v425: 97 pgs: 97 active+clean; 24 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-03-26T17:40:57.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 59702 3.39 MB/sec warmup 100 sec latency 19.330 ms 2024-03-26T17:40:58.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 60268 3.41 MB/sec warmup 101 sec latency 17.077 ms 2024-03-26T17:40:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:57 smithi144 ceph-mon[36095]: pgmap v426: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:40:58.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:57 smithi105 ceph-mon[32280]: pgmap v426: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:40:58.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:57 smithi135 ceph-mon[35636]: pgmap v426: 97 pgs: 97 active+clean; 24 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:40:59.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 60746 3.39 MB/sec warmup 102 sec latency 16.506 ms 2024-03-26T17:41:00.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 61241 3.37 MB/sec warmup 103 sec latency 18.611 ms 2024-03-26T17:41:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:40:59 smithi144 ceph-mon[36095]: pgmap v427: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-03-26T17:41:00.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:40:59 smithi105 ceph-mon[32280]: pgmap v427: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-03-26T17:41:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:40:59 smithi135 ceph-mon[35636]: pgmap v427: 97 pgs: 97 active+clean; 24 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-03-26T17:41:01.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 61859 3.38 MB/sec warmup 104 sec latency 17.466 ms 2024-03-26T17:41:02.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 62485 3.35 MB/sec warmup 105 sec latency 20.309 ms 2024-03-26T17:41:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:01 smithi144 ceph-mon[36095]: pgmap v428: 97 pgs: 97 active+clean; 24 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:41:02.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:01 smithi105 ceph-mon[32280]: pgmap v428: 97 pgs: 97 active+clean; 24 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:41:02.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:01 smithi135 ceph-mon[35636]: pgmap v428: 97 pgs: 97 active+clean; 24 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:41:03.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 63198 3.37 MB/sec warmup 106 sec latency 16.896 ms 2024-03-26T17:41:04.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 63703 3.37 MB/sec warmup 107 sec latency 16.249 ms 2024-03-26T17:41:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:03 smithi144 ceph-mon[36095]: pgmap v429: 97 pgs: 97 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:41:04.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:03 smithi105 ceph-mon[32280]: pgmap v429: 97 pgs: 97 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:41:04.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:03 smithi135 ceph-mon[35636]: pgmap v429: 97 pgs: 97 active+clean; 26 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:41:05.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 64267 3.38 MB/sec warmup 108 sec latency 15.739 ms 2024-03-26T17:41:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 -- 172.21.15.105:0/2410436404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf80fffa0 msgr2=0x7fdaf81003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 --2- 172.21.15.105:0/2410436404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf80fffa0 0x7fdaf81003c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009900 tx=0x7fdae802bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 -- 172.21.15.105:0/2410436404 shutdown_connections 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 --2- 172.21.15.105:0/2410436404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf8101b50 0x7fdaf81083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 --2- 172.21.15.105:0/2410436404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaf8101190 0x7fdaf8101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 --2- 172.21.15.105:0/2410436404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf80fffa0 0x7fdaf81003c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 -- 172.21.15.105:0/2410436404 >> 172.21.15.105:0/2410436404 conn(0x7fdaf80fb510 msgr2=0x7fdaf80fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 -- 172.21.15.105:0/2410436404 shutdown_connections 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.621+0000 7fdafe305640 1 -- 172.21.15.105:0/2410436404 wait complete. 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafe305640 1 Processor -- start 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafe305640 1 -- start start 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafe305640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf80fffa0 0x7fdaf8199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafe305640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8101190 0x7fdaf819a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafe305640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaf8101b50 0x7fdaf81a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafe305640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810c350 con 0x7fdaf8101190 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafe305640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810c1d0 con 0x7fdaf8101b50 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafe305640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf810c4d0 con 0x7fdaf80fffa0 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdaf77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8101190 0x7fdaf819a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdaf77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8101190 0x7fdaf819a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51956/0 (socket says 172.21.15.105:51956) 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdaf77fe640 1 -- 172.21.15.105:0/2927876501 learned_addr learned my addr 172.21.15.105:0/2927876501 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf80fffa0 0x7fdaf8199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.622+0000 7fdafc8a2640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaf8101b50 0x7fdaf81a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf77fe640 1 -- 172.21.15.105:0/2927876501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf80fffa0 msgr2=0x7fdaf8199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf80fffa0 0x7fdaf8199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf77fe640 1 -- 172.21.15.105:0/2927876501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaf8101b50 msgr2=0x7fdaf81a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaf8101b50 0x7fdaf81a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf77fe640 1 -- 172.21.15.105:0/2927876501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81a1900 con 0x7fdaf8101190 2024-03-26T17:41:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8101190 0x7fdaf819a200 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00e9f0 tx=0x7fdaec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf57fa640 1 -- 172.21.15.105:0/2927876501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00cd80 con 0x7fdaf8101190 2024-03-26T17:41:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf57fa640 1 -- 172.21.15.105:0/2927876501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaec0108e0 con 0x7fdaf8101190 2024-03-26T17:41:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdaf57fa640 1 -- 172.21.15.105:0/2927876501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec018920 con 0x7fdaf8101190 2024-03-26T17:41:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdafe305640 1 -- 172.21.15.105:0/2927876501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81a1bf0 con 0x7fdaf8101190 2024-03-26T17:41:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.623+0000 7fdafe305640 1 -- 172.21.15.105:0/2927876501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a2110 con 0x7fdaf8101190 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdb00fc5740 1 Processor -- start 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdb00fc5740 1 -- start start 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a25d0 0x558dff4a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a2fc0 0x558dff4a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdaf57fa640 1 -- 172.21.15.105:0/2927876501 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdaec010a90 con 0x7fdaf8101190 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a8130 0x558dff4aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558dff4070b0 con 0x558dff4a8130 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558dff3af960 con 0x558dff4a2fc0 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558dff403060 con 0x558dff4a25d0 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdafc8a2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a25d0 0x558dff4a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdaf77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a8130 0x558dff4aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a2fc0 0x558dff4a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdafc8a2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a25d0 0x558dff4a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45018/0 (socket says 172.21.15.105:45018) 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdafc8a2640 1 -- 172.21.15.105:0/3703923748 learned_addr learned my addr 172.21.15.105:0/3703923748 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdafc8a2640 1 -- 172.21.15.105:0/3703923748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a2fc0 msgr2=0x558dff4a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdafc8a2640 1 --2- 172.21.15.105:0/3703923748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a2fc0 0x558dff4a7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdafc8a2640 1 -- 172.21.15.105:0/3703923748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a8130 msgr2=0x558dff4aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.626+0000 7fdafc8a2640 1 --2- 172.21.15.105:0/3703923748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a8130 0x558dff4aa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.627+0000 7fdafc8a2640 1 -- 172.21.15.105:0/3703923748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dff3c1c60 con 0x558dff4a25d0 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.627+0000 7fdafc8a2640 1 --2- 172.21.15.105:0/3703923748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a25d0 0x558dff4a29f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009fd0 tx=0x7fdaf00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00d892f91205c21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.627+0000 7fdaf57fa640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdae00a2fc0 0x7fdae00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.627+0000 7fdade7fc640 1 -- 172.21.15.105:0/3703923748 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0004d20 con 0x558dff4a25d0 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.627+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdae00a2fc0 0x7fdae00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.627+0000 7fdaf57fa640 1 -- 172.21.15.105:0/2927876501 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdaec0cf700 con 0x7fdaf8101190 2024-03-26T17:41:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.627+0000 7fdade7fc640 1 -- 172.21.15.105:0/3703923748 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaf0004050 con 0x558dff4a25d0 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.627+0000 7fdade7fc640 1 -- 172.21.15.105:0/3703923748 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0005280 con 0x558dff4a25d0 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3703923748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a25d0 msgr2=0x558dff4a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdae00a2fc0 0x7fdae00a5480 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdae8005ab0 tx=0x7fdae8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/3703923748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a25d0 0x558dff4a29f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009fd0 tx=0x7fdaf00118d0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3703923748 shutdown_connections 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/3703923748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a8130 0x558dff4aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/3703923748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a2fc0 0x558dff4a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/3703923748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a25d0 0x558dff4a29f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3703923748 >> 172.21.15.105:0/3703923748 conn(0x558dff49db40 msgr2=0x558dff49df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3703923748 shutdown_connections 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3703923748 wait complete. 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.628+0000 7fdb00fc5740 1 -- start start 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdb00fc5740 1 Processor -- start 2024-03-26T17:41:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558dff4ab0e0 con 0x558dff4a25d0 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558dff3c2db0 con 0x558dff4a8130 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558dff4ab260 con 0x558dff4a2fc0 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdb00fc5740 10 client.0 _ll_register_callbacks cb 0x558dff3c7110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51970/0 (socket says 172.21.15.105:51970) 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf7fff640 1 -- 172.21.15.105:0/2300485797 learned_addr learned my addr 172.21.15.105:0/2300485797 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdafc8a2640 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf77fe640 1 -- 172.21.15.105:0/2300485797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 msgr2=0x558dff4b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf77fe640 1 -- 172.21.15.105:0/2300485797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 msgr2=0x558dff4ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf77fe640 1 -- 172.21.15.105:0/2300485797 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dff11b210 con 0x558dff4a2fc0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4ac8f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0cd210 tx=0x7fdaec098140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eea869792ac284f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.629+0000 7fdad77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdad77fe640 1 -- 172.21.15.105:0/2300485797 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec09b800 con 0x558dff4a2fc0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdad77fe640 1 -- 172.21.15.105:0/2300485797 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdaec0cfbb0 con 0x558dff4a2fc0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 1 -- 172.21.15.105:0/2300485797 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf0004050 con 0x558dff4a2fc0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdad77fe640 1 -- 172.21.15.105:0/2300485797 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdaec010de0 con 0x558dff4a2fc0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 10 client.26155 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 1 -- 172.21.15.105:0/2300485797 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558dff1e7620 con 0x558dff4a2fc0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdad77fe640 1 -- 172.21.15.105:0/2300485797 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdaf0004050 con 0x558dff4a2fc0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdad77fe640 1 -- 172.21.15.105:0/2300485797 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdaec0bab60 con 0x558dff4a2fc0 2024-03-26T17:41:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 10 client.26155 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 10 client.26155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 20 client.26155 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558dff4b8200 0x558dff4ba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 4 client.26155 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.630+0000 7fdb00fc5740 1 -- 172.21.15.105:0/2300485797 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dff4145f0 con 0x558dff4b8200 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.631+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558dff4b8200 0x558dff4ba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.631+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558dff4b8200 0x558dff4ba650 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82e546ef29bff33 server_cookie=83842935a54c9c69 in_seq=0 out_seq=0 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.631+0000 7fdad77fe640 10 client.26155 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdad77fe640 1 -- 172.21.15.105:0/2300485797 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558dff4145f0 con 0x558dff4b8200 2024-03-26T17:41:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdad77fe640 10 client.26155 handle_command_reply: tid=0 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 client.26155 shutdown 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 -- 172.21.15.105:0/2300485797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 msgr2=0x558dff4ac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4ac8f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0cd210 tx=0x7fdaec098140 comp rx=0 tx=0).stop 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 -- 172.21.15.105:0/2300485797 shutdown_connections 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558dff4b8200 0x558dff4ba650 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4ac8f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/2300485797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdad77fe640 0 client.26155 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 -- 172.21.15.105:0/2300485797 >> 172.21.15.105:0/2300485797 conn(0x558dff49db40 msgr2=0x558dff49cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 -- 172.21.15.105:0/2300485797 shutdown_connections 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 1 -- 172.21.15.105:0/2300485797 wait complete. 2024-03-26T17:41:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.632+0000 7fdb00fc5740 20 client.26155 trim_cache size 0 max 16384 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.656+0000 7fdb00fc5740 1 Processor -- start 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.656+0000 7fdb00fc5740 1 -- start start 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.656+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.656+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4a3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81aaf50 con 0x558dff4a25d0 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81aca90 con 0x558dff4a8130 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81ab620 con 0x558dff4a2fc0 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4a7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51984/0 (socket says 172.21.15.105:51984) 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 -- 172.21.15.105:0/3345877346 learned_addr learned my addr 172.21.15.105:0/3345877346 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdafc8a2640 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 -- 172.21.15.105:0/3345877346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 msgr2=0x558dff4b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 -- 172.21.15.105:0/3345877346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 msgr2=0x558dff4a3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4a3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 -- 172.21.15.105:0/3345877346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dff4a7080 con 0x558dff4a25d0 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4a7b10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdae80096c0 tx=0x7fdae8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08ba5657e0488b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaddffb640 1 -- 172.21.15.105:0/3345877346 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8056510 con 0x558dff4a25d0 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaddffb640 1 -- 172.21.15.105:0/3345877346 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdae80566b0 con 0x558dff4a25d0 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.657+0000 7fdaddffb640 1 -- 172.21.15.105:0/3345877346 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80046c0 con 0x558dff4a25d0 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3345877346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 msgr2=0x558dff4a7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4a7b10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdae80096c0 tx=0x7fdae8041000 comp rx=0 tx=0).stop 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3345877346 shutdown_connections 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/3345877346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4a7b10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3345877346 >> 172.21.15.105:0/3345877346 conn(0x558dff49db40 msgr2=0x558dff4a7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3345877346 shutdown_connections 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 -- 172.21.15.105:0/3345877346 wait complete. 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.658+0000 7fdb00fc5740 1 -- start start 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdb00fc5740 1 Processor -- start 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdb00fc5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81ac3c0 con 0x558dff4a25d0 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81abcf0 con 0x558dff4a8130 2024-03-26T17:41:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdb00fc5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81a8f30 con 0x558dff4a2fc0 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdb00fc5740 10 client.0 _ll_register_callbacks cb 0x7fdaec00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51998/0 (socket says 172.21.15.105:51998) 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf7fff640 1 -- 172.21.15.105:0/1584271982 learned_addr learned my addr 172.21.15.105:0/1584271982 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdafc8a2640 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf77fe640 1 -- 172.21.15.105:0/1584271982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 msgr2=0x558dff4b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf77fe640 1 -- 172.21.15.105:0/1584271982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 msgr2=0x558dff4b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf77fe640 1 -- 172.21.15.105:0/1584271982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dff4b26d0 con 0x558dff4a2fc0 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.659+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdaf77fe640 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b10d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdaec09a150 tx=0x7fdaec009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd54e20d2a73c95 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdad77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdad77fe640 1 -- 172.21.15.105:0/1584271982 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0b7080 con 0x558dff4a2fc0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdad77fe640 1 -- 172.21.15.105:0/1584271982 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdaec0cfa90 con 0x558dff4a2fc0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 1 -- 172.21.15.105:0/1584271982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae80566b0 con 0x558dff4a2fc0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdad77fe640 1 -- 172.21.15.105:0/1584271982 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdaec09bc70 con 0x558dff4a2fc0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 10 client.26161 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 1 -- 172.21.15.105:0/1584271982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558dff4b3a50 con 0x558dff4a2fc0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdad77fe640 1 -- 172.21.15.105:0/1584271982 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdae80566b0 con 0x558dff4a2fc0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdad77fe640 1 -- 172.21.15.105:0/1584271982 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdaec01ab80 con 0x558dff4a2fc0 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 10 client.26161 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 10 client.26161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 20 client.26161 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558dff51ec50 0x558dff521040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 4 client.26161 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:41:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.660+0000 7fdb00fc5740 1 -- 172.21.15.105:0/1584271982 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dff4ba9e0 con 0x558dff51ec50 2024-03-26T17:41:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.661+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558dff51ec50 0x558dff521040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.661+0000 7fdad77fe640 10 client.26161 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.661+0000 7fdaf7fff640 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558dff51ec50 0x558dff521040 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6123c776636ab5fe server_cookie=41d814c59b185bba in_seq=0 out_seq=0 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdad77fe640 1 -- 172.21.15.105:0/1584271982 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dff4ba9e0 con 0x558dff51ec50 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdad77fe640 10 client.26161 handle_command_reply: tid=0 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 client.26161 shutdown 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 -- 172.21.15.105:0/1584271982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 msgr2=0x558dff4b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b10d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdaec09a150 tx=0x7fdaec009740 comp rx=0 tx=0).stop 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 -- 172.21.15.105:0/1584271982 shutdown_connections 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558dff51ec50 0x558dff521040 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558dff4a8130 0x558dff4b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558dff4a2fc0 0x558dff4b10d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdad77fe640 0 client.26161 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 --2- 172.21.15.105:0/1584271982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558dff4a25d0 0x558dff4b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 -- 172.21.15.105:0/1584271982 >> 172.21.15.105:0/1584271982 conn(0x558dff49db40 msgr2=0x558dff4a3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 -- 172.21.15.105:0/1584271982 shutdown_connections 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.706+0000 7fdb00fc5740 1 -- 172.21.15.105:0/1584271982 wait complete. 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdb00fc5740 20 client.26161 trim_cache size 0 max 16384 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:41:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:41:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 -- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdae00a2fc0 msgr2=0x7fdae00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdae00a2fc0 0x7fdae00a5480 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdae8005ab0 tx=0x7fdae8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 -- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8101190 msgr2=0x7fdaf819a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8101190 0x7fdaf819a200 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00e9f0 tx=0x7fdaec00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 -- 172.21.15.105:0/2927876501 shutdown_connections 2024-03-26T17:41:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdae00a2fc0 0x7fdae00a5480 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdaf8101b50 0x7fdaf81a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdaf8101190 0x7fdaf819a200 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 --2- 172.21.15.105:0/2927876501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf80fffa0 0x7fdaf8199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 -- 172.21.15.105:0/2927876501 >> 172.21.15.105:0/2927876501 conn(0x7fdaf80fb510 msgr2=0x7fdaf80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 -- 172.21.15.105:0/2927876501 shutdown_connections 2024-03-26T17:41:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:05.707+0000 7fdafe305640 1 -- 172.21.15.105:0/2927876501 wait complete. 2024-03-26T17:41:06.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 64737 3.36 MB/sec warmup 109 sec latency 28.302 ms 2024-03-26T17:41:06.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:05 smithi144 ceph-mon[36095]: pgmap v430: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:41:06.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:05 smithi105 ceph-mon[32280]: pgmap v430: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:41:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:05 smithi135 ceph-mon[35636]: pgmap v430: 97 pgs: 97 active+clean; 25 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:41:07.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 65366 3.37 MB/sec warmup 110 sec latency 17.507 ms 2024-03-26T17:41:08.019 INFO:tasks.workunit.client.0.smithi105.stdout: 1 66052 3.34 MB/sec warmup 111 sec latency 15.971 ms 2024-03-26T17:41:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:07 smithi144 ceph-mon[36095]: pgmap v431: 97 pgs: 97 active+clean; 26 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:41:08.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:07 smithi105 ceph-mon[32280]: pgmap v431: 97 pgs: 97 active+clean; 26 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:41:08.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:07 smithi135 ceph-mon[35636]: pgmap v431: 97 pgs: 97 active+clean; 26 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:41:09.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 66770 3.37 MB/sec warmup 112 sec latency 16.733 ms 2024-03-26T17:41:09.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:41:09.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.270+0000 7f996f8ac640 1 -- 172.21.15.105:0/1915471894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99680ffb80 msgr2=0x7f99680fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.270+0000 7f996f8ac640 1 --2- 172.21.15.105:0/1915471894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99680ffb80 0x7f99680fffa0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9964009930 tx=0x7f996402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.270+0000 7f996f8ac640 1 -- 172.21.15.105:0/1915471894 shutdown_connections 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.270+0000 7f996f8ac640 1 --2- 172.21.15.105:0/1915471894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9968103510 0x7f9968109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.270+0000 7f996f8ac640 1 --2- 172.21.15.105:0/1915471894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9968102b50 0x7f9968102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.270+0000 7f996f8ac640 1 --2- 172.21.15.105:0/1915471894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99680ffb80 0x7f99680fffa0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.270+0000 7f996f8ac640 1 -- 172.21.15.105:0/1915471894 >> 172.21.15.105:0/1915471894 conn(0x7f99680fb4b0 msgr2=0x7f99680fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.270+0000 7f996f8ac640 1 -- 172.21.15.105:0/1915471894 shutdown_connections 2024-03-26T17:41:09.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 -- 172.21.15.105:0/1915471894 wait complete. 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 Processor -- start 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 -- start start 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99680ffb80 0x7f9968197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9968102b50 0x7f99681980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9968103510 0x7f996819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f996810d430 con 0x7f9968102b50 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f996810d2b0 con 0x7f9968103510 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996f8ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f996810d5b0 con 0x7f99680ffb80 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996ce47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9968102b50 0x7f99681980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.271+0000 7f996ce47640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9968102b50 0x7f99681980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52006/0 (socket says 172.21.15.105:52006) 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996ce47640 1 -- 172.21.15.105:0/2908948212 learned_addr learned my addr 172.21.15.105:0/2908948212 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996d648640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99680ffb80 0x7f9968197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996de49640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9968103510 0x7f996819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996ce47640 1 -- 172.21.15.105:0/2908948212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99680ffb80 msgr2=0x7f9968197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996ce47640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99680ffb80 0x7f9968197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996ce47640 1 -- 172.21.15.105:0/2908948212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9968103510 msgr2=0x7f996819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996ce47640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9968103510 0x7f996819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996ce47640 1 -- 172.21.15.105:0/2908948212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996819f780 con 0x7f9968102b50 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996ce47640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9968102b50 0x7f99681980a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f995800e9f0 tx=0x7f995800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f995e7fc640 1 -- 172.21.15.105:0/2908948212 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995800cd80 con 0x7f9968102b50 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f995e7fc640 1 -- 172.21.15.105:0/2908948212 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99580108e0 con 0x7f9968102b50 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996f8ac640 1 -- 172.21.15.105:0/2908948212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996819fa70 con 0x7f9968102b50 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f995e7fc640 1 -- 172.21.15.105:0/2908948212 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958018920 con 0x7f9968102b50 2024-03-26T17:41:09.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.272+0000 7f996f8ac640 1 -- 172.21.15.105:0/2908948212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996819ffc0 con 0x7f9968102b50 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.274+0000 7f995e7fc640 1 -- 172.21.15.105:0/2908948212 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9958010a90 con 0x7f9968102b50 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.275+0000 7f9972574740 1 Processor -- start 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.275+0000 7f995e7fc640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f99540a2f20 0x7f99540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.275+0000 7f9972574740 1 -- start start 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.275+0000 7f996de49640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f99540a2f20 0x7f99540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f9972574740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f9972574740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcead50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f995e7fc640 1 -- 172.21.15.105:0/2908948212 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f99580cf700 con 0x7f9968102b50 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f9972574740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadcf8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f9972574740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55acadc4c980 con 0x55acadce7870 2024-03-26T17:41:09.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f9972574740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55acadbf4be0 con 0x55acadce8260 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f9972574740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55acadc48930 con 0x55acadceb3c0 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52008/0 (socket says 172.21.15.105:52008) 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 -- 172.21.15.105:0/2305750844 learned_addr learned my addr 172.21.15.105:0/2305750844 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996ce47640 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcead50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996de49640 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadcf8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 -- 172.21.15.105:0/2305750844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 msgr2=0x55acadcf8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadcf8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 -- 172.21.15.105:0/2305750844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 msgr2=0x55acadcead50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcead50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 -- 172.21.15.105:0/2305750844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acada2a7d0 con 0x55acadce7870 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996ce47640 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcead50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996de49640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f99540a2f20 0x7f99540a53e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9960005fd0 tx=0x7f9960011940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f996d648640 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce7c90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9964009900 tx=0x7f99640095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599bfc71b946b178 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.276+0000 7f99537fe640 1 -- 172.21.15.105:0/2305750844 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964045020 con 0x55acadce7870 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.277+0000 7f99537fe640 1 -- 172.21.15.105:0/2305750844 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f996402c9a0 con 0x55acadce7870 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.277+0000 7f99537fe640 1 -- 172.21.15.105:0/2305750844 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964005610 con 0x55acadce7870 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.277+0000 7f9972574740 1 -- 172.21.15.105:0/2305750844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 msgr2=0x55acadce7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.277+0000 7f9972574740 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce7c90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9964009900 tx=0x7f99640095e0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 -- 172.21.15.105:0/2305750844 shutdown_connections 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadcf8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcead50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 --2- 172.21.15.105:0/2305750844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce7c90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 -- 172.21.15.105:0/2305750844 >> 172.21.15.105:0/2305750844 conn(0x55acadce2e60 msgr2=0x55acadce3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 -- 172.21.15.105:0/2305750844 shutdown_connections 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 -- 172.21.15.105:0/2305750844 wait complete. 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 -- start start 2024-03-26T17:41:09.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 Processor -- start 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce7870 0x55acadcfa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadcfa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadceb3c0 0x55acadd01830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55acadcf8bd0 con 0x55acadceb3c0 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55acadc08160 con 0x55acadce7870 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f9972574740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55acadcf8d50 con 0x55acadce8260 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f996d648640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce7870 0x55acadcfa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.278+0000 7f996d648640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce7870 0x55acadcfa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53792/0 (socket says 172.21.15.105:53792) 2024-03-26T17:41:09.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996d648640 1 -- 172.21.15.105:0/984015546 learned_addr learned my addr 172.21.15.105:0/984015546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996de49640 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadceb3c0 0x55acadd01830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f9972574740 10 client.0 _ll_register_callbacks cb 0x55acadc44720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996ce47640 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadcfa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996d648640 1 -- 172.21.15.105:0/984015546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 msgr2=0x55acadcfa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996d648640 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadcfa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996d648640 1 -- 172.21.15.105:0/984015546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadceb3c0 msgr2=0x55acadd01830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996d648640 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadceb3c0 0x55acadd01830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996d648640 1 -- 172.21.15.105:0/984015546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acad9603f0 con 0x55acadce7870 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996de49640 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadceb3c0 0x55acadd01830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f996d648640 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce7870 0x55acadcfa2b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9964006010 tx=0x7f99640058c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=452eee5083856255 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f9950ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f9950ff9640 1 -- 172.21.15.105:0/984015546 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964045070 con 0x55acadce7870 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f9950ff9640 1 -- 172.21.15.105:0/984015546 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f99640037f0 con 0x55acadce7870 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.279+0000 7f9972574740 1 -- 172.21.15.105:0/984015546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f996402c9a0 con 0x55acadce7870 2024-03-26T17:41:09.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9950ff9640 1 -- 172.21.15.105:0/984015546 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9964004490 con 0x55acadce7870 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9972574740 10 client.26177 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9972574740 1 -- 172.21.15.105:0/984015546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55acad9200f0 con 0x55acadce7870 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9950ff9640 1 -- 172.21.15.105:0/984015546 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f996402c9a0 con 0x55acadce7870 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9950ff9640 1 -- 172.21.15.105:0/984015546 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9964004e50 con 0x55acadce7870 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9972574740 10 client.26177 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9972574740 10 client.26177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9972574740 20 client.26177 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9972574740 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55acadd05d20 0x55acadd08110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9972574740 4 client.26177 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f9972574740 1 -- 172.21.15.105:0/984015546 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acadc59ec0 con 0x55acadd05d20 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.280+0000 7f996ce47640 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55acadd05d20 0x55acadd08110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.281+0000 7f9950ff9640 10 client.26177 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.281+0000 7f996ce47640 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55acadd05d20 0x55acadd08110 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df62cbc620e26d8 server_cookie=5a7cb2867767d1cb in_seq=0 out_seq=0 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9950ff9640 1 -- 172.21.15.105:0/984015546 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55acadc59ec0 con 0x55acadd05d20 2024-03-26T17:41:09.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9950ff9640 10 client.26177 handle_command_reply: tid=0 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 client.26177 shutdown 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 -- 172.21.15.105:0/984015546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce7870 msgr2=0x55acadcfa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce7870 0x55acadcfa2b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9964006010 tx=0x7f99640058c0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 -- 172.21.15.105:0/984015546 shutdown_connections 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55acadd05d20 0x55acadd08110 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadceb3c0 0x55acadd01830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadcfa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9950ff9640 0 client.26177 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 --2- 172.21.15.105:0/984015546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce7870 0x55acadcfa2b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 -- 172.21.15.105:0/984015546 >> 172.21.15.105:0/984015546 conn(0x55acadce2e60 msgr2=0x55acadce2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 -- 172.21.15.105:0/984015546 shutdown_connections 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 1 -- 172.21.15.105:0/984015546 wait complete. 2024-03-26T17:41:09.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.282+0000 7f9972574740 20 client.26177 trim_cache size 0 max 16384 2024-03-26T17:41:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.304+0000 7f9972574740 1 Processor -- start 2024-03-26T17:41:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.304+0000 7f9972574740 1 -- start start 2024-03-26T17:41:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.304+0000 7f9972574740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.304+0000 7f9972574740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadd017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.304+0000 7f9972574740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadceb3c0 0x55acadcf98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.304+0000 7f9972574740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f99681a6ae0 con 0x55acadce7870 2024-03-26T17:41:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.304+0000 7f9972574740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99681a8620 con 0x55acadceb3c0 2024-03-26T17:41:09.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.304+0000 7f9972574740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99681a71b0 con 0x55acadce8260 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996d648640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadd017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadd017f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45080/0 (socket says 172.21.15.105:45080) 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 -- 172.21.15.105:0/887846146 learned_addr learned my addr 172.21.15.105:0/887846146 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996d648640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52020/0 (socket says 172.21.15.105:52020) 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996de49640 1 --2- 172.21.15.105:0/887846146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadceb3c0 0x55acadcf98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 -- 172.21.15.105:0/887846146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadceb3c0 msgr2=0x55acadcf98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 --2- 172.21.15.105:0/887846146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadceb3c0 0x55acadcf98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 -- 172.21.15.105:0/887846146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 msgr2=0x55acadce67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 --2- 172.21.15.105:0/887846146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 -- 172.21.15.105:0/887846146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acadd08740 con 0x55acadce8260 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f996ce47640 1 --2- 172.21.15.105:0/887846146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadd017f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f99580cd210 tx=0x7f99580bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa66d2cf7f9cf177 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f9952ffd640 1 -- 172.21.15.105:0/887846146 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99580a4800 con 0x55acadce8260 2024-03-26T17:41:09.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f9952ffd640 1 -- 172.21.15.105:0/887846146 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9958099f40 con 0x55acadce8260 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.305+0000 7f9952ffd640 1 -- 172.21.15.105:0/887846146 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995809b2f0 con 0x55acadce8260 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 -- 172.21.15.105:0/887846146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 msgr2=0x55acadd017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 --2- 172.21.15.105:0/887846146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadd017f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f99580cd210 tx=0x7f99580bc040 comp rx=0 tx=0).stop 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 -- 172.21.15.105:0/887846146 shutdown_connections 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 --2- 172.21.15.105:0/887846146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadceb3c0 0x55acadcf98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 --2- 172.21.15.105:0/887846146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadce8260 0x55acadd017f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 --2- 172.21.15.105:0/887846146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadce67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 -- 172.21.15.105:0/887846146 >> 172.21.15.105:0/887846146 conn(0x55acadce2e60 msgr2=0x55acadce60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 -- 172.21.15.105:0/887846146 shutdown_connections 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.306+0000 7f9972574740 1 -- 172.21.15.105:0/887846146 wait complete. 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 1 -- start start 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 1 Processor -- start 2024-03-26T17:41:09.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadcfe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcfe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadd69f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f99681a7f50 con 0x55acadce7870 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99681a7880 con 0x55acadce8260 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f99681a4ac0 con 0x55acadceb3c0 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f9972574740 10 client.0 _ll_register_callbacks cb 0x7f9964009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f996d648640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadcfe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f996ce47640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcfe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f996d648640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadcfe490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52030/0 (socket says 172.21.15.105:52030) 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f996d648640 1 -- 172.21.15.105:0/3499268780 learned_addr learned my addr 172.21.15.105:0/3499268780 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.307+0000 7f996de49640 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadd69f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f996d648640 1 -- 172.21.15.105:0/3499268780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 msgr2=0x55acadd69f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f996d648640 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadd69f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f996d648640 1 -- 172.21.15.105:0/3499268780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 msgr2=0x55acadcfe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f996d648640 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcfe9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f996d648640 1 -- 172.21.15.105:0/3499268780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acadcfefc0 con 0x55acadce7870 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f996ce47640 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcfe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f996de49640 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadd69f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f996d648640 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadcfe490 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9964005c40 tx=0x7f9964041e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9757fdd646568a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f9950ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f9950ff9640 1 -- 172.21.15.105:0/3499268780 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964045070 con 0x55acadce7870 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f9950ff9640 1 -- 172.21.15.105:0/3499268780 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f996404c850 con 0x55acadce7870 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f9972574740 1 -- 172.21.15.105:0/3499268780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9958099f40 con 0x55acadce7870 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f9950ff9640 1 -- 172.21.15.105:0/3499268780 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f996402c9a0 con 0x55acadce7870 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f9972574740 10 client.16326 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f9972574740 1 -- 172.21.15.105:0/3499268780 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55acadd6b550 con 0x55acadce7870 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.308+0000 7f9950ff9640 1 -- 172.21.15.105:0/3499268780 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9958099f40 con 0x55acadce7870 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f9950ff9640 1 -- 172.21.15.105:0/3499268780 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9964069a20 con 0x55acadce7870 2024-03-26T17:41:09.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f9972574740 10 client.16326 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f9972574740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f9972574740 20 client.16326 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f9972574740 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55acadd6bdf0 0x55acadd6e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f9972574740 4 client.16326 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f9972574740 1 -- 172.21.15.105:0/3499268780 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55acadc54210 con 0x55acadd6bdf0 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f996ce47640 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55acadd6bdf0 0x55acadd6e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f9950ff9640 10 client.16326 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.309+0000 7f996ce47640 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55acadd6bdf0 0x55acadd6e210 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b9323bd43da5cb server_cookie=f99ea12c545b426 in_seq=0 out_seq=0 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9950ff9640 1 -- 172.21.15.105:0/3499268780 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55acadc54210 con 0x55acadd6bdf0 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9950ff9640 10 client.16326 handle_command_reply: tid=0 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 client.16326 shutdown 2024-03-26T17:41:09.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 -- 172.21.15.105:0/3499268780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 msgr2=0x55acadcfe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadcfe490 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9964005c40 tx=0x7f9964041e60 comp rx=0 tx=0).stop 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 -- 172.21.15.105:0/3499268780 shutdown_connections 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55acadd6bdf0 0x55acadd6e210 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55acadceb3c0 0x55acadd69f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55acadce8260 0x55acadcfe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 --2- 172.21.15.105:0/3499268780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55acadce7870 0x55acadcfe490 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9950ff9640 0 client.16326 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 -- 172.21.15.105:0/3499268780 >> 172.21.15.105:0/3499268780 conn(0x55acadce2e60 msgr2=0x55acadce6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 -- 172.21.15.105:0/3499268780 shutdown_connections 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 1 -- 172.21.15.105:0/3499268780 wait complete. 2024-03-26T17:41:09.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.310+0000 7f9972574740 20 client.16326 trim_cache size 0 max 16384 2024-03-26T17:41:09.311 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:41:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:41:09.311 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:41:09.311 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 -- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f99540a2f20 msgr2=0x7f99540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f99540a2f20 0x7f99540a53e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9960005fd0 tx=0x7f9960011940 comp rx=0 tx=0).stop 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 -- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9968102b50 msgr2=0x7f99681980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9968102b50 0x7f99681980a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f995800e9f0 tx=0x7f995800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 -- 172.21.15.105:0/2908948212 shutdown_connections 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f99540a2f20 0x7f99540a53e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9968103510 0x7f996819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9968102b50 0x7f99681980a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 --2- 172.21.15.105:0/2908948212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99680ffb80 0x7f9968197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 -- 172.21.15.105:0/2908948212 >> 172.21.15.105:0/2908948212 conn(0x7f99680fb4b0 msgr2=0x7f9968105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 -- 172.21.15.105:0/2908948212 shutdown_connections 2024-03-26T17:41:09.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.311+0000 7f996f8ac640 1 -- 172.21.15.105:0/2908948212 wait complete. 2024-03-26T17:41:09.316 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:41:09.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:41:09.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 -- 172.21.15.105:0/618277801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 msgr2=0x7f9e68108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 --2- 172.21.15.105:0/618277801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 0x7f9e68108c20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009ca0 tx=0x7f9e6002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 -- 172.21.15.105:0/618277801 shutdown_connections 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 --2- 172.21.15.105:0/618277801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 0x7f9e68108c20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 --2- 172.21.15.105:0/618277801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e681019d0 0x7f9e68101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 --2- 172.21.15.105:0/618277801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e681007e0 0x7f9e68100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 -- 172.21.15.105:0/618277801 >> 172.21.15.105:0/618277801 conn(0x7f9e680fbd50 msgr2=0x7f9e680fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 -- 172.21.15.105:0/618277801 shutdown_connections 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.437+0000 7f9e6f328640 1 -- 172.21.15.105:0/618277801 wait complete. 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6f328640 1 Processor -- start 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6f328640 1 -- start start 2024-03-26T17:41:09.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6f328640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e681007e0 0x7f9e68197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6f328640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e681019d0 0x7f9e68197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6f328640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 0x7f9e6819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6f328640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810c310 con 0x7f9e68102390 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6f328640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810c190 con 0x7f9e681007e0 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6f328640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6810c490 con 0x7f9e681019d0 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6d0c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e681007e0 0x7f9e68197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6d8c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 0x7f9e6819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6d0c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e681007e0 0x7f9e68197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53820/0 (socket says 172.21.15.105:53820) 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6d0c4640 1 -- 172.21.15.105:0/3262953219 learned_addr learned my addr 172.21.15.105:0/3262953219 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6c8c3640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e681019d0 0x7f9e68197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6d8c5640 1 -- 172.21.15.105:0/3262953219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e681019d0 msgr2=0x7f9e68197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6d8c5640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e681019d0 0x7f9e68197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6d8c5640 1 -- 172.21.15.105:0/3262953219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e681007e0 msgr2=0x7f9e68197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.438+0000 7f9e6d8c5640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e681007e0 0x7f9e68197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.439+0000 7f9e6d8c5640 1 -- 172.21.15.105:0/3262953219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6819f6d0 con 0x7f9e68102390 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.439+0000 7f9e6d0c4640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e681007e0 0x7f9e68197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.439+0000 7f9e6d8c5640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 0x7f9e6819f060 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9e6002c630 tx=0x7f9e600328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.439+0000 7f9e5e7fc640 1 -- 172.21.15.105:0/3262953219 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60041020 con 0x7f9e68102390 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.439+0000 7f9e5e7fc640 1 -- 172.21.15.105:0/3262953219 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e60003e20 con 0x7f9e68102390 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.439+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6819f960 con 0x7f9e68102390 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.439+0000 7f9e5e7fc640 1 -- 172.21.15.105:0/3262953219 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e60039040 con 0x7f9e68102390 2024-03-26T17:41:09.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.439+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6819fe50 con 0x7f9e68102390 2024-03-26T17:41:09.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.440+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e681a0020 con 0x7f9e68102390 2024-03-26T17:41:09.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.441+0000 7f9e5e7fc640 1 -- 172.21.15.105:0/3262953219 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9e60044020 con 0x7f9e68102390 2024-03-26T17:41:09.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.442+0000 7f9e5e7fc640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e540a2f20 0x7f9e540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.442+0000 7f9e6d0c4640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e540a2f20 0x7f9e540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.442+0000 7f9e5e7fc640 1 -- 172.21.15.105:0/3262953219 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9e600f1990 con 0x7f9e68102390 2024-03-26T17:41:09.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.442+0000 7f9e6d0c4640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e540a2f20 0x7f9e540a53e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9e640058c0 tx=0x7f9e64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.446+0000 7f9e5e7fc640 1 -- 172.21.15.105:0/3262953219 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9e6004b020 con 0x7f9e68102390 2024-03-26T17:41:09.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.685+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e681a01f0 con 0x7f9e68102390 2024-03-26T17:41:09.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.686+0000 7f9e5e7fc640 1 -- 172.21.15.105:0/3262953219 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f9e600be780 con 0x7f9e68102390 2024-03-26T17:41:09.679 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:41:09.679 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:41:09.679 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e540a2f20 msgr2=0x7f9e540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e540a2f20 0x7f9e540a53e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9e640058c0 tx=0x7f9e64005f90 comp rx=0 tx=0).stop 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 msgr2=0x7f9e6819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 0x7f9e6819f060 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9e6002c630 tx=0x7f9e600328a0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 shutdown_connections 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9e540a2f20 0x7f9e540a53e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e68102390 0x7f9e6819f060 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e681019d0 0x7f9e68197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 --2- 172.21.15.105:0/3262953219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e681007e0 0x7f9e68197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 >> 172.21.15.105:0/3262953219 conn(0x7f9e680fbd50 msgr2=0x7f9e680fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 shutdown_connections 2024-03-26T17:41:09.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.688+0000 7f9e6f328640 1 -- 172.21.15.105:0/3262953219 wait complete. 2024-03-26T17:41:09.693 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 -- 172.21.15.105:0/1520781329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501019f0 msgr2=0x7f9750101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1520781329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501019f0 0x7f9750101e70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f974402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 -- 172.21.15.105:0/1520781329 shutdown_connections 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1520781329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97501023b0 0x7f9750108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1520781329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501019f0 0x7f9750101e70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1520781329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9750100800 0x7f9750100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 -- 172.21.15.105:0/1520781329 >> 172.21.15.105:0/1520781329 conn(0x7f97500fbd70 msgr2=0x7f97500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 -- 172.21.15.105:0/1520781329 shutdown_connections 2024-03-26T17:41:09.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.821+0000 7f9754a4b640 1 -- 172.21.15.105:0/1520781329 wait complete. 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.822+0000 7f9754a4b640 1 Processor -- start 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.822+0000 7f9754a4b640 1 -- start start 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.822+0000 7f9754a4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9750100800 0x7f975006a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.822+0000 7f9754a4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97501019f0 0x7f975019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.822+0000 7f9754a4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501023b0 0x7f975019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.822+0000 7f9754a4b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f975010c330 con 0x7f97501023b0 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.822+0000 7f9754a4b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f975010c1b0 con 0x7f9750100800 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.822+0000 7f9754a4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f975010c4b0 con 0x7f97501019f0 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9750100800 0x7f975006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501023b0 0x7f975019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9750100800 0x7f975006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53834/0 (socket says 172.21.15.105:53834) 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 -- 172.21.15.105:0/1237004677 learned_addr learned my addr 172.21.15.105:0/1237004677 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974dd9b640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97501019f0 0x7f975019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 -- 172.21.15.105:0/1237004677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97501019f0 msgr2=0x7f975019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97501019f0 0x7f975019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 -- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501023b0 msgr2=0x7f975019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501023b0 0x7f975019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 -- 172.21.15.105:0/1237004677 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975019f830 con 0x7f9750100800 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974ed9d640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501023b0 0x7f975019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f974e59c640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9750100800 0x7f975006a770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f974000e9c0 tx=0x7f974000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f973f7fe640 1 -- 172.21.15.105:0/1237004677 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974000cd80 con 0x7f9750100800 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f9754a4b640 1 -- 172.21.15.105:0/1237004677 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975019fac0 con 0x7f9750100800 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f973f7fe640 1 -- 172.21.15.105:0/1237004677 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97400035f0 con 0x7f9750100800 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f973f7fe640 1 -- 172.21.15.105:0/1237004677 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740010690 con 0x7f9750100800 2024-03-26T17:41:09.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.823+0000 7f9754a4b640 1 -- 172.21.15.105:0/1237004677 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975019ffe0 con 0x7f9750100800 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.825+0000 7f973f7fe640 1 -- 172.21.15.105:0/1237004677 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f97400037a0 con 0x7f9750100800 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f975763b740 1 Processor -- start 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f975763b740 1 -- start start 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f973f7fe640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97380a2f20 0x7f97380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f975763b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc391a30 0x5649dc391e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f975763b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc396f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f974ed9d640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97380a2f20 0x7f97380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f973f7fe640 1 -- 172.21.15.105:0/1237004677 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f97400cf7a0 con 0x7f9750100800 2024-03-26T17:41:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f975763b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc397590 0x5649dc399a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f975763b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5649dc2f6970 con 0x5649dc392420 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f975763b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5649dc29ed70 con 0x5649dc391a30 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.826+0000 7f975763b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5649dc2f2920 con 0x5649dc397590 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc391a30 0x5649dc391e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc391a30 0x5649dc391e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53844/0 (socket says 172.21.15.105:53844) 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 -- 172.21.15.105:0/457515257 learned_addr learned my addr 172.21.15.105:0/457515257 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974dd9b640 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc396f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974ed9d640 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc397590 0x5649dc399a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 -- 172.21.15.105:0/457515257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc397590 msgr2=0x5649dc399a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc397590 0x5649dc399a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 -- 172.21.15.105:0/457515257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 msgr2=0x5649dc396f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc396f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 -- 172.21.15.105:0/457515257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649dc00a3f0 con 0x5649dc391a30 2024-03-26T17:41:09.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974ed9d640 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc397590 0x5649dc399a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974ed9d640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97380a2f20 0x7f97380a53e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9748009fd0 tx=0x7f974800fb40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f974e59c640 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc391a30 0x5649dc391e50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9740098860 tx=0x7f974009b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5408b70f403335c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f973cff9640 1 -- 172.21.15.105:0/457515257 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974009c590 con 0x5649dc391a30 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f973cff9640 1 -- 172.21.15.105:0/457515257 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974009be90 con 0x5649dc391a30 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.827+0000 7f973cff9640 1 -- 172.21.15.105:0/457515257 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740058eb0 con 0x5649dc391a30 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 -- 172.21.15.105:0/457515257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc391a30 msgr2=0x5649dc391e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc391a30 0x5649dc391e50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9740098860 tx=0x7f974009b9d0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 -- 172.21.15.105:0/457515257 shutdown_connections 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc397590 0x5649dc399a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc396f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 --2- 172.21.15.105:0/457515257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc391a30 0x5649dc391e50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 -- 172.21.15.105:0/457515257 >> 172.21.15.105:0/457515257 conn(0x5649dc38cfa0 msgr2=0x5649dc38d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 -- 172.21.15.105:0/457515257 shutdown_connections 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 -- 172.21.15.105:0/457515257 wait complete. 2024-03-26T17:41:09.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 -- start start 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.828+0000 7f975763b740 1 Processor -- start 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f975763b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc391a30 0x5649dc39bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f975763b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 0x5649dc39c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f975763b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc3a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f975763b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5649dc39a540 con 0x5649dc391a30 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f975763b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5649dc2b2310 con 0x5649dc397590 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f975763b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5649dc39a6c0 con 0x5649dc392420 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f975763b740 10 client.0 _ll_register_callbacks cb 0x5649dc2edc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc391a30 0x5649dc39bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc391a30 0x5649dc39bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52076/0 (socket says 172.21.15.105:52076) 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 -- 172.21.15.105:0/1853213733 learned_addr learned my addr 172.21.15.105:0/1853213733 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974ed9d640 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc3a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974dd9b640 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 0x5649dc39c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 -- 172.21.15.105:0/1853213733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 msgr2=0x5649dc39c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 0x5649dc39c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 -- 172.21.15.105:0/1853213733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 msgr2=0x5649dc3a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc3a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 -- 172.21.15.105:0/1853213733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649dc0d47d0 con 0x5649dc391a30 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f974e59c640 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc391a30 0x5649dc39bf40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9740098860 tx=0x7f974009ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198354c2145621e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.829+0000 7f9735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f9735ffb640 1 -- 172.21.15.105:0/1853213733 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740056060 con 0x5649dc391a30 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f9735ffb640 1 -- 172.21.15.105:0/1853213733 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f97400590b0 con 0x5649dc391a30 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 1 -- 172.21.15.105:0/1853213733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f974009be90 con 0x5649dc391a30 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f9735ffb640 1 -- 172.21.15.105:0/1853213733 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97400b4070 con 0x5649dc391a30 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 10 client.16344 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 1 -- 172.21.15.105:0/1853213733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649dc3a67a0 con 0x5649dc391a30 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f9735ffb640 1 -- 172.21.15.105:0/1853213733 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974009be90 con 0x5649dc391a30 2024-03-26T17:41:09.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f9735ffb640 1 -- 172.21.15.105:0/1853213733 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9740097d30 con 0x5649dc391a30 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 10 client.16344 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 10 client.16344 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 20 client.16344 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5649dc3a7af0 0x5649dc3a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 4 client.16344 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f975763b740 1 -- 172.21.15.105:0/1853213733 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649dc303eb0 con 0x5649dc3a7af0 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.830+0000 7f974dd9b640 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5649dc3a7af0 0x5649dc3a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.831+0000 7f9735ffb640 10 client.16344 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.831+0000 7f974dd9b640 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5649dc3a7af0 0x5649dc3a9ee0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30426a9defc26079 server_cookie=e2e3f4b6bc21db1f in_seq=0 out_seq=0 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f9735ffb640 1 -- 172.21.15.105:0/1853213733 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5649dc303eb0 con 0x5649dc3a7af0 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f9735ffb640 10 client.16344 handle_command_reply: tid=0 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 client.16344 shutdown 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 -- 172.21.15.105:0/1853213733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc391a30 msgr2=0x5649dc39bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc391a30 0x5649dc39bf40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9740098860 tx=0x7f974009ffe0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 -- 172.21.15.105:0/1853213733 shutdown_connections 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5649dc3a7af0 0x5649dc3a9ee0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc3a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 0x5649dc39c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 --2- 172.21.15.105:0/1853213733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc391a30 0x5649dc39bf40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 -- 172.21.15.105:0/1853213733 >> 172.21.15.105:0/1853213733 conn(0x5649dc38cfa0 msgr2=0x5649dc38c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f9735ffb640 0 client.16344 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 -- 172.21.15.105:0/1853213733 shutdown_connections 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 1 -- 172.21.15.105:0/1853213733 wait complete. 2024-03-26T17:41:09.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.832+0000 7f975763b740 20 client.16344 trim_cache size 0 max 16384 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.846+0000 7f975763b740 1 Processor -- start 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.846+0000 7f975763b740 1 -- start start 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f975763b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc38cfa0 0x5649dc39c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f975763b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 0x5649dc3a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f975763b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc38b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f975763b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97501a8ea0 con 0x5649dc38cfa0 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f975763b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97501aa9e0 con 0x5649dc397590 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f975763b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97501a9570 con 0x5649dc392420 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc38cfa0 0x5649dc39c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc38cfa0 0x5649dc39c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52082/0 (socket says 172.21.15.105:52082) 2024-03-26T17:41:09.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 -- 172.21.15.105:0/4288215819 learned_addr learned my addr 172.21.15.105:0/4288215819 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974ed9d640 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc38b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974dd9b640 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 0x5649dc3a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 -- 172.21.15.105:0/4288215819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 msgr2=0x5649dc3a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 0x5649dc3a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 -- 172.21.15.105:0/4288215819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 msgr2=0x5649dc38b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc38b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 -- 172.21.15.105:0/4288215819 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649dc396bf0 con 0x5649dc38cfa0 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f974e59c640 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc38cfa0 0x5649dc39c260 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f97400bc230 tx=0x7f97400cd2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecf1417bf29058 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f9737fff640 1 -- 172.21.15.105:0/4288215819 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740056050 con 0x5649dc38cfa0 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f9737fff640 1 -- 172.21.15.105:0/4288215819 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974009c590 con 0x5649dc38cfa0 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.847+0000 7f9737fff640 1 -- 172.21.15.105:0/4288215819 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97400a4670 con 0x5649dc38cfa0 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 -- 172.21.15.105:0/4288215819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc38cfa0 msgr2=0x5649dc39c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc38cfa0 0x5649dc39c260 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f97400bc230 tx=0x7f97400cd2b0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 -- 172.21.15.105:0/4288215819 shutdown_connections 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc38b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc392420 0x5649dc3a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 --2- 172.21.15.105:0/4288215819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc38cfa0 0x5649dc39c260 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 -- 172.21.15.105:0/4288215819 >> 172.21.15.105:0/4288215819 conn(0x5649dc393200 msgr2=0x5649dc39bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 -- 172.21.15.105:0/4288215819 shutdown_connections 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 -- 172.21.15.105:0/4288215819 wait complete. 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 -- start start 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.848+0000 7f975763b740 1 Processor -- start 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f975763b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc38cfa0 0x5649dc3a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f975763b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc3a1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f975763b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc3a2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f975763b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97501aa310 con 0x5649dc392420 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f975763b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97501a9c40 con 0x5649dc397590 2024-03-26T17:41:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f975763b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97501a6e80 con 0x5649dc38cfa0 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f975763b740 10 client.0 _ll_register_callbacks cb 0x7f97480093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc3a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc3a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52096/0 (socket says 172.21.15.105:52096) 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 -- 172.21.15.105:0/696394840 learned_addr learned my addr 172.21.15.105:0/696394840 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974ed9d640 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc3a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974e59c640 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc38cfa0 0x5649dc3a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 -- 172.21.15.105:0/696394840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc38cfa0 msgr2=0x5649dc3a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc38cfa0 0x5649dc3a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 -- 172.21.15.105:0/696394840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 msgr2=0x5649dc3a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc3a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 -- 172.21.15.105:0/696394840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649dc3a2910 con 0x5649dc392420 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f974dd9b640 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc3a1310 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f974402cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e9a8251e98e117 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f9735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f9735ffb640 1 -- 172.21.15.105:0/696394840 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744004250 con 0x5649dc392420 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f9735ffb640 1 -- 172.21.15.105:0/696394840 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f974403a040 con 0x5649dc392420 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.849+0000 7f975763b740 1 -- 172.21.15.105:0/696394840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f974009c590 con 0x5649dc392420 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f9735ffb640 1 -- 172.21.15.105:0/696394840 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97440043f0 con 0x5649dc392420 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f975763b740 10 client.16350 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f975763b740 1 -- 172.21.15.105:0/696394840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5649dc412240 con 0x5649dc392420 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f9735ffb640 1 -- 172.21.15.105:0/696394840 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974009c590 con 0x5649dc392420 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f9735ffb640 1 -- 172.21.15.105:0/696394840 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9744005160 con 0x5649dc392420 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f975763b740 10 client.16350 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f975763b740 10 client.16350 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f975763b740 20 client.16350 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f975763b740 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5649dc412ca0 0x5649dc415090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f975763b740 4 client.16350 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f975763b740 1 -- 172.21.15.105:0/696394840 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649dc2fe200 con 0x5649dc412ca0 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f974e59c640 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5649dc412ca0 0x5649dc415090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f974e59c640 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5649dc412ca0 0x5649dc415090 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0ef35832d8ad742 server_cookie=d97d4f2e7dc11e85 in_seq=0 out_seq=0 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.850+0000 7f9735ffb640 10 client.16350 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f9735ffb640 1 -- 172.21.15.105:0/696394840 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649dc2fe200 con 0x5649dc412ca0 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f9735ffb640 10 client.16350 handle_command_reply: tid=0 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 client.16350 shutdown 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 -- 172.21.15.105:0/696394840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 msgr2=0x5649dc3a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc3a1310 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f974402cc10 comp rx=0 tx=0).stop 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 -- 172.21.15.105:0/696394840 shutdown_connections 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5649dc412ca0 0x5649dc415090 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5649dc397590 0x5649dc3a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5649dc392420 0x5649dc3a1310 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f9735ffb640 0 client.16350 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 --2- 172.21.15.105:0/696394840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5649dc38cfa0 0x5649dc3a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 -- 172.21.15.105:0/696394840 >> 172.21.15.105:0/696394840 conn(0x5649dc393200 msgr2=0x5649dc390200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 -- 172.21.15.105:0/696394840 shutdown_connections 2024-03-26T17:41:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 1 -- 172.21.15.105:0/696394840 wait complete. 2024-03-26T17:41:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f975763b740 20 client.16350 trim_cache size 0 max 16384 2024-03-26T17:41:09.851 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f9754a4b640 1 -- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97380a2f20 msgr2=0x7f97380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97380a2f20 0x7f97380a53e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9748009fd0 tx=0x7f974800fb40 comp rx=0 tx=0).stop 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f9754a4b640 1 -- 172.21.15.105:0/1237004677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9750100800 msgr2=0x7f975006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9750100800 0x7f975006a770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f974000e9c0 tx=0x7f974000ee80 comp rx=0 tx=0).stop 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.851+0000 7f9754a4b640 1 -- 172.21.15.105:0/1237004677 shutdown_connections 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.852+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97380a2f20 0x7f97380a53e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.852+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97501023b0 0x7f975019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.852+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97501019f0 0x7f975019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.852+0000 7f9754a4b640 1 --2- 172.21.15.105:0/1237004677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9750100800 0x7f975006a770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.852+0000 7f9754a4b640 1 -- 172.21.15.105:0/1237004677 >> 172.21.15.105:0/1237004677 conn(0x7f97500fbd70 msgr2=0x7f9750104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.852+0000 7f9754a4b640 1 -- 172.21.15.105:0/1237004677 shutdown_connections 2024-03-26T17:41:09.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.852+0000 7f9754a4b640 1 -- 172.21.15.105:0/1237004677 wait complete. 2024-03-26T17:41:09.856 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:41:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.987+0000 7fd91b510640 1 -- 172.21.15.105:0/2805648503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141007c0 msgr2=0x7fd914100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.987+0000 7fd91b510640 1 --2- 172.21.15.105:0/2805648503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141007c0 0x7fd914100be0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd910009900 tx=0x7fd91002bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.987+0000 7fd91b510640 1 -- 172.21.15.105:0/2805648503 shutdown_connections 2024-03-26T17:41:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.987+0000 7fd91b510640 1 --2- 172.21.15.105:0/2805648503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd914102370 0x7fd914108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.987+0000 7fd91b510640 1 --2- 172.21.15.105:0/2805648503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9141019b0 0x7fd914101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 --2- 172.21.15.105:0/2805648503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141007c0 0x7fd914100be0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 -- 172.21.15.105:0/2805648503 >> 172.21.15.105:0/2805648503 conn(0x7fd9140fbd50 msgr2=0x7fd9140fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 -- 172.21.15.105:0/2805648503 shutdown_connections 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 -- 172.21.15.105:0/2805648503 wait complete. 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 Processor -- start 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 -- start start 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9141007c0 0x7fd914197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141019b0 0x7fd914198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd914102370 0x7fd91419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd91410c2c0 con 0x7fd9141019b0 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd91410c140 con 0x7fd914102370 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.988+0000 7fd91b510640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd91410c440 con 0x7fd9141007c0 2024-03-26T17:41:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141019b0 0x7fd914198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141019b0 0x7fd914198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52106/0 (socket says 172.21.15.105:52106) 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 -- 172.21.15.105:0/3209621458 learned_addr learned my addr 172.21.15.105:0/3209621458 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd919aad640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd914102370 0x7fd91419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd9192ac640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9141007c0 0x7fd914197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 -- 172.21.15.105:0/3209621458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9141007c0 msgr2=0x7fd914197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9141007c0 0x7fd914197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 -- 172.21.15.105:0/3209621458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd914102370 msgr2=0x7fd91419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd914102370 0x7fd91419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 -- 172.21.15.105:0/3209621458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91419f770 con 0x7fd9141019b0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd918aab640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141019b0 0x7fd914198090 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e9f0 tx=0x7fd90400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd90a7fc640 1 -- 172.21.15.105:0/3209621458 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90400cd80 con 0x7fd9141019b0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd90a7fc640 1 -- 172.21.15.105:0/3209621458 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9040108e0 con 0x7fd9141019b0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd90a7fc640 1 -- 172.21.15.105:0/3209621458 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904018920 con 0x7fd9141019b0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd91b510640 1 -- 172.21.15.105:0/3209621458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91419fa60 con 0x7fd9141019b0 2024-03-26T17:41:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.989+0000 7fd91b510640 1 -- 172.21.15.105:0/3209621458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9141a0000 con 0x7fd9141019b0 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.992+0000 7fd90a7fc640 1 -- 172.21.15.105:0/3209621458 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd904010a90 con 0x7fd9141019b0 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.992+0000 7fd91e1da740 1 Processor -- start 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.992+0000 7fd91e1da740 1 -- start start 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f972e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f973420 0x55859f977f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd90a7fc640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9000a73b0 0x7fd9000a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f978590 0x55859f97aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd91e1da740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55859f8d7970 con 0x55859f978590 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd91e1da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55859f87fd70 con 0x55859f972a30 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd91e1da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55859f8d3920 con 0x55859f973420 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f972e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd90a7fc640 1 -- 172.21.15.105:0/3209621458 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd9040cf670 con 0x7fd9141019b0 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f972e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53894/0 (socket says 172.21.15.105:53894) 2024-03-26T17:41:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 -- 172.21.15.105:0/204772905 learned_addr learned my addr 172.21.15.105:0/204772905 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd918aab640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9000a73b0 0x7fd9000a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f978590 0x55859f97aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd9192ac640 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f973420 0x55859f977f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 -- 172.21.15.105:0/204772905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f973420 msgr2=0x55859f977f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f973420 0x55859f977f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 -- 172.21.15.105:0/204772905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f978590 msgr2=0x55859f97aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f978590 0x55859f97aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd919aad640 1 -- 172.21.15.105:0/204772905 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55859f5eb3f0 con 0x55859f972a30 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.993+0000 7fd918aab640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9000a73b0 0x7fd9000a9870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd904000c00 tx=0x7fd90400cf20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.994+0000 7fd919aad640 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f972e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd90c011b40 tx=0x7fd90c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac85bed84f6e157 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.994+0000 7fd8ff7fe640 1 -- 172.21.15.105:0/204772905 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011050 con 0x55859f972a30 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.994+0000 7fd8ff7fe640 1 -- 172.21.15.105:0/204772905 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd90c003e20 con 0x55859f972a30 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.994+0000 7fd8ff7fe640 1 -- 172.21.15.105:0/204772905 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c004290 con 0x55859f972a30 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 -- 172.21.15.105:0/204772905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 msgr2=0x55859f972e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f972e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd90c011b40 tx=0x7fd90c009b30 comp rx=0 tx=0).stop 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 -- 172.21.15.105:0/204772905 shutdown_connections 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f978590 0x55859f97aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f973420 0x55859f977f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 --2- 172.21.15.105:0/204772905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f972e50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 -- 172.21.15.105:0/204772905 >> 172.21.15.105:0/204772905 conn(0x55859f96dfa0 msgr2=0x55859f96e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 -- 172.21.15.105:0/204772905 shutdown_connections 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 -- 172.21.15.105:0/204772905 wait complete. 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 -- start start 2024-03-26T17:41:09.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.995+0000 7fd91e1da740 1 Processor -- start 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f97ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f97d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f9843c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd91e1da740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55859f97b4e0 con 0x55859f972a30 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd91e1da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55859f893310 con 0x55859f973420 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd91e1da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55859f97b660 con 0x55859f978590 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd91e1da740 10 client.0 _ll_register_callbacks cb 0x55859f8cec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f97ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f97ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52138/0 (socket says 172.21.15.105:52138) 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd9192ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f97d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 -- 172.21.15.105:0/222764081 learned_addr learned my addr 172.21.15.105:0/222764081 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f9843c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 -- 172.21.15.105:0/222764081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 msgr2=0x55859f9843c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f9843c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 -- 172.21.15.105:0/222764081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 msgr2=0x55859f97d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f97d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 -- 172.21.15.105:0/222764081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55859f6b57d0 con 0x55859f972a30 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd919aad640 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f97ce40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00ffe0 tx=0x7fd90c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd549ad7a5b46be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd8fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd8fcff9640 1 -- 172.21.15.105:0/222764081 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011230 con 0x55859f972a30 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd91e1da740 1 -- 172.21.15.105:0/222764081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd90c003e20 con 0x55859f972a30 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd8fcff9640 1 -- 172.21.15.105:0/222764081 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd90c004a40 con 0x55859f972a30 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.996+0000 7fd8fcff9640 1 -- 172.21.15.105:0/222764081 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd90c004c40 con 0x55859f972a30 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd91e1da740 10 client.16362 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd91e1da740 1 -- 172.21.15.105:0/222764081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55859f987780 con 0x55859f972a30 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd8fcff9640 1 -- 172.21.15.105:0/222764081 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd90c003e20 con 0x55859f972a30 2024-03-26T17:41:09.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd8fcff9640 1 -- 172.21.15.105:0/222764081 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd90c01d020 con 0x55859f972a30 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd91e1da740 10 client.16362 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd91e1da740 10 client.16362 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd91e1da740 20 client.16362 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd91e1da740 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55859f988c00 0x55859f98b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd91e1da740 4 client.16362 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd91e1da740 1 -- 172.21.15.105:0/222764081 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55859f8e4eb0 con 0x55859f988c00 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.997+0000 7fd9192ac640 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55859f988c00 0x55859f98b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.998+0000 7fd9192ac640 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55859f988c00 0x55859f98b030 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c479fce537afa338 server_cookie=5c6f12e38ec78752 in_seq=0 out_seq=0 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.998+0000 7fd8fcff9640 10 client.16362 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.998+0000 7fd8fcff9640 1 -- 172.21.15.105:0/222764081 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55859f8e4eb0 con 0x55859f988c00 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.998+0000 7fd8fcff9640 10 client.16362 handle_command_reply: tid=0 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 client.16362 shutdown 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 -- 172.21.15.105:0/222764081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 msgr2=0x55859f97ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f97ce40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00ffe0 tx=0x7fd90c005f90 comp rx=0 tx=0).stop 2024-03-26T17:41:09.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 -- 172.21.15.105:0/222764081 shutdown_connections 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55859f988c00 0x55859f98b030 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f9843c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f97d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd8fcff9640 0 client.16362 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 --2- 172.21.15.105:0/222764081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f97ce40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 -- 172.21.15.105:0/222764081 >> 172.21.15.105:0/222764081 conn(0x55859f96dfa0 msgr2=0x55859f96d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 -- 172.21.15.105:0/222764081 shutdown_connections 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 1 -- 172.21.15.105:0/222764081 wait complete. 2024-03-26T17:41:10.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:09.999+0000 7fd91e1da740 20 client.16362 trim_cache size 0 max 16384 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd91e1da740 1 Processor -- start 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd91e1da740 1 -- start start 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f97d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f973420 0x55859f984380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f971910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd91e1da740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141a6c70 con 0x55859f973420 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd91e1da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141a87b0 con 0x55859f972a30 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd91e1da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141a7340 con 0x55859f978590 2024-03-26T17:41:10.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd9192ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f973420 0x55859f984380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd9192ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f973420 0x55859f984380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52152/0 (socket says 172.21.15.105:52152) 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd919aad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f97d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd9192ac640 1 -- 172.21.15.105:0/948930414 learned_addr learned my addr 172.21.15.105:0/948930414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd919aad640 1 --2- 172.21.15.105:0/948930414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f971910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd9192ac640 1 -- 172.21.15.105:0/948930414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 msgr2=0x55859f971910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd9192ac640 1 --2- 172.21.15.105:0/948930414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f971910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd9192ac640 1 -- 172.21.15.105:0/948930414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 msgr2=0x55859f97d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd9192ac640 1 --2- 172.21.15.105:0/948930414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f97d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.014+0000 7fd9192ac640 1 -- 172.21.15.105:0/948930414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55859f96ddb0 con 0x55859f973420 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd9192ac640 1 --2- 172.21.15.105:0/948930414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f973420 0x55859f984380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd91002c840 tx=0x7fd910005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a0b1a8236003ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd8feffd640 1 -- 172.21.15.105:0/948930414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91004e020 con 0x55859f973420 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd8feffd640 1 -- 172.21.15.105:0/948930414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91002cac0 con 0x55859f973420 2024-03-26T17:41:10.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd8feffd640 1 -- 172.21.15.105:0/948930414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910048070 con 0x55859f973420 2024-03-26T17:41:10.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd91e1da740 1 -- 172.21.15.105:0/948930414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f973420 msgr2=0x55859f984380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd91e1da740 1 --2- 172.21.15.105:0/948930414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f973420 0x55859f984380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd91002c840 tx=0x7fd910005fd0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd91e1da740 1 -- 172.21.15.105:0/948930414 shutdown_connections 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd91e1da740 1 --2- 172.21.15.105:0/948930414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f971910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd91e1da740 1 --2- 172.21.15.105:0/948930414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f973420 0x55859f984380 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd91e1da740 1 --2- 172.21.15.105:0/948930414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f972a30 0x55859f97d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.015+0000 7fd91e1da740 1 -- 172.21.15.105:0/948930414 >> 172.21.15.105:0/948930414 conn(0x55859f96dfa0 msgr2=0x55859f971200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 -- 172.21.15.105:0/948930414 shutdown_connections 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 -- 172.21.15.105:0/948930414 wait complete. 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 -- start start 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 Processor -- start 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f981170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f9816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f9ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141a80e0 con 0x55859f972a30 2024-03-26T17:41:10.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141a7a10 con 0x55859f973420 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9141a4c50 con 0x55859f978590 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd91e1da740 10 client.0 _ll_register_callbacks cb 0x7fd90c011630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd919aad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f981170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd919aad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f981170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52156/0 (socket says 172.21.15.105:52156) 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd9192ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f9816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.016+0000 7fd919aad640 1 -- 172.21.15.105:0/3650562492 learned_addr learned my addr 172.21.15.105:0/3650562492 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd919aad640 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f9ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd9192ac640 1 -- 172.21.15.105:0/3650562492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 msgr2=0x55859f9ecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd9192ac640 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f9ecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd9192ac640 1 -- 172.21.15.105:0/3650562492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 msgr2=0x55859f981170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd9192ac640 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f981170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd9192ac640 1 -- 172.21.15.105:0/3650562492 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55859f981cf0 con 0x55859f973420 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd919aad640 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f981170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd9192ac640 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f9816b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55859f973360 tx=0x7fd91002cd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0905ce4e947d217 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd8fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd8fcff9640 1 -- 172.21.15.105:0/3650562492 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91004e040 con 0x55859f973420 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd8fcff9640 1 -- 172.21.15.105:0/3650562492 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd91004acb0 con 0x55859f973420 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd91e1da740 1 -- 172.21.15.105:0/3650562492 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91002cac0 con 0x55859f973420 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd8fcff9640 1 -- 172.21.15.105:0/3650562492 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd91004a310 con 0x55859f973420 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd91e1da740 10 client.26225 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.017+0000 7fd91e1da740 1 -- 172.21.15.105:0/3650562492 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55859f9ee530 con 0x55859f973420 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd8fcff9640 1 -- 172.21.15.105:0/3650562492 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91002cac0 con 0x55859f973420 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd8fcff9640 1 -- 172.21.15.105:0/3650562492 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd9100046c0 con 0x55859f973420 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd91e1da740 10 client.26225 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd91e1da740 10 client.26225 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd91e1da740 20 client.26225 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd91e1da740 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55859f9eef60 0x55859f9f1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd91e1da740 4 client.26225 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd91e1da740 1 -- 172.21.15.105:0/3650562492 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55859f8df200 con 0x55859f9eef60 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd919aad640 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55859f9eef60 0x55859f9f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd8fcff9640 10 client.26225 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd919aad640 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55859f9eef60 0x55859f9f1380 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5dc98003b8c273f5 server_cookie=c31b7ebb77cfed52 in_seq=0 out_seq=0 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd8fcff9640 1 -- 172.21.15.105:0/3650562492 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55859f8df200 con 0x55859f9eef60 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd8fcff9640 10 client.26225 handle_command_reply: tid=0 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.018+0000 7fd91e1da740 1 client.26225 shutdown 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 -- 172.21.15.105:0/3650562492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 msgr2=0x55859f9816b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f9816b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55859f973360 tx=0x7fd91002cd00 comp rx=0 tx=0).stop 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 -- 172.21.15.105:0/3650562492 shutdown_connections 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55859f9eef60 0x55859f9f1380 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55859f978590 0x55859f9ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd8fcff9640 0 client.26225 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55859f973420 0x55859f9816b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 --2- 172.21.15.105:0/3650562492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55859f972a30 0x55859f981170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 -- 172.21.15.105:0/3650562492 >> 172.21.15.105:0/3650562492 conn(0x55859f96dfa0 msgr2=0x55859f97d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 -- 172.21.15.105:0/3650562492 shutdown_connections 2024-03-26T17:41:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 1 -- 172.21.15.105:0/3650562492 wait complete. 2024-03-26T17:41:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91e1da740 20 client.26225 trim_cache size 0 max 16384 2024-03-26T17:41:10.018 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 -- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9000a73b0 msgr2=0x7fd9000a9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9000a73b0 0x7fd9000a9870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd904000c00 tx=0x7fd90400cf20 comp rx=0 tx=0).stop 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 -- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141019b0 msgr2=0x7fd914198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141019b0 0x7fd914198090 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e9f0 tx=0x7fd90400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 -- 172.21.15.105:0/3209621458 shutdown_connections 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9000a73b0 0x7fd9000a9870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd914102370 0x7fd91419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9141019b0 0x7fd914198090 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 --2- 172.21.15.105:0/3209621458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9141007c0 0x7fd914197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.019+0000 7fd91b510640 1 -- 172.21.15.105:0/3209621458 >> 172.21.15.105:0/3209621458 conn(0x7fd9140fbd50 msgr2=0x7fd914068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.020+0000 7fd91b510640 1 -- 172.21.15.105:0/3209621458 shutdown_connections 2024-03-26T17:41:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.020+0000 7fd91b510640 1 -- 172.21.15.105:0/3209621458 wait complete. 2024-03-26T17:41:10.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 67263 3.36 MB/sec warmup 113 sec latency 15.490 ms 2024-03-26T17:41:10.022 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:09 smithi105 ceph-mon[32280]: pgmap v432: 97 pgs: 97 active+clean; 27 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:41:10.022 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:09 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3262953219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:10.024 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:41:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:09 smithi144 ceph-mon[36095]: pgmap v432: 97 pgs: 97 active+clean; 27 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:41:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:09 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3262953219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 -- 172.21.15.105:0/3945656487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0324103b00 msgr2=0x7f0324103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 --2- 172.21.15.105:0/3945656487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0324103b00 0x7f0324103f20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0314009900 tx=0x7f031402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 -- 172.21.15.105:0/3945656487 shutdown_connections 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 --2- 172.21.15.105:0/3945656487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03241056b0 0x7f032410bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 --2- 172.21.15.105:0/3945656487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0324104cf0 0x7f0324105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 --2- 172.21.15.105:0/3945656487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0324103b00 0x7f0324103f20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 -- 172.21.15.105:0/3945656487 >> 172.21.15.105:0/3945656487 conn(0x7f03240ff070 msgr2=0x7f03241014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 -- 172.21.15.105:0/3945656487 shutdown_connections 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.146+0000 7f032a3f3640 1 -- 172.21.15.105:0/3945656487 wait complete. 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f032a3f3640 1 Processor -- start 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f032a3f3640 1 -- start start 2024-03-26T17:41:10.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f032a3f3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0324103b00 0x7f0324199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f032a3f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0324104cf0 0x7f032419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f032a3f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03241056b0 0x7f03241a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f032a3f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f032410f600 con 0x7f03241056b0 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f032a3f3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f032410f480 con 0x7f0324103b00 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f032a3f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f032410f780 con 0x7f0324104cf0 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0328990640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03241056b0 0x7f03241a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0328990640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03241056b0 0x7f03241a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52168/0 (socket says 172.21.15.105:52168) 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0323fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0324103b00 0x7f0324199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f03237fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0324104cf0 0x7f032419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0328990640 1 -- 172.21.15.105:0/1854871761 learned_addr learned my addr 172.21.15.105:0/1854871761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0328990640 1 -- 172.21.15.105:0/1854871761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0324104cf0 msgr2=0x7f032419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0328990640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0324104cf0 0x7f032419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0328990640 1 -- 172.21.15.105:0/1854871761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0324103b00 msgr2=0x7f0324199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0328990640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0324103b00 0x7f0324199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.147+0000 7f0328990640 1 -- 172.21.15.105:0/1854871761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03241a19b0 con 0x7f03241056b0 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.148+0000 7f0328990640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03241056b0 0x7f03241a1310 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f031c00eb10 tx=0x7f031c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.148+0000 7f03217fa640 1 -- 172.21.15.105:0/1854871761 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c004b20 con 0x7f03241056b0 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.148+0000 7f03217fa640 1 -- 172.21.15.105:0/1854871761 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f031c004cc0 con 0x7f03241056b0 2024-03-26T17:41:10.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.148+0000 7f032a3f3640 1 -- 172.21.15.105:0/1854871761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03241a1ca0 con 0x7f03241056b0 2024-03-26T17:41:10.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.148+0000 7f03217fa640 1 -- 172.21.15.105:0/1854871761 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c010690 con 0x7f03241056b0 2024-03-26T17:41:10.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.148+0000 7f032a3f3640 1 -- 172.21.15.105:0/1854871761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03241a2170 con 0x7f03241056b0 2024-03-26T17:41:10.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.150+0000 7f032cfd6740 1 Processor -- start 2024-03-26T17:41:10.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f03217fa640 1 -- 172.21.15.105:0/1854871761 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f031c005020 con 0x7f03241056b0 2024-03-26T17:41:10.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f032cfd6740 1 -- start start 2024-03-26T17:41:10.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed3420 0x55a087ed7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed8590 0x55a087edaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f032cfd6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a087e37970 con 0x55a087ed3420 2024-03-26T17:41:10.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f032cfd6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a087ddfd70 con 0x55a087ed2a30 2024-03-26T17:41:10.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f03217fa640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f030c0a2f20 0x7f030c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f032cfd6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a087e33920 con 0x55a087ed8590 2024-03-26T17:41:10.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f03237fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f03237fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53962/0 (socket says 172.21.15.105:53962) 2024-03-26T17:41:10.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03237fe640 1 -- 172.21.15.105:0/3338255712 learned_addr learned my addr 172.21.15.105:0/3338255712 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03237fe640 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed8590 0x55a087edaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.151+0000 7f0323fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed3420 0x55a087ed7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03217fa640 1 -- 172.21.15.105:0/1854871761 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f031c0cfa10 con 0x7f03241056b0 2024-03-26T17:41:10.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f0328990640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f030c0a2f20 0x7f030c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03237fe640 1 -- 172.21.15.105:0/3338255712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed8590 msgr2=0x55a087edaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03237fe640 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed8590 0x55a087edaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03237fe640 1 -- 172.21.15.105:0/3338255712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed3420 msgr2=0x55a087ed7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03237fe640 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed3420 0x55a087ed7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03237fe640 1 -- 172.21.15.105:0/3338255712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a087b4b3f0 con 0x55a087ed2a30 2024-03-26T17:41:10.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f0323fff640 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed3420 0x55a087ed7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:10.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f0328990640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f030c0a2f20 0x7f030c0a53e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f031c098a40 tx=0x7f031c00efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f03237fe640 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed2e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f03180093e0 tx=0x7f0318009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a65a03fa2616f3b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f030a7fc640 1 -- 172.21.15.105:0/3338255712 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318004050 con 0x55a087ed2a30 2024-03-26T17:41:10.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f030a7fc640 1 -- 172.21.15.105:0/3338255712 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f031800fc70 con 0x55a087ed2a30 2024-03-26T17:41:10.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.152+0000 7f030a7fc640 1 -- 172.21.15.105:0/3338255712 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03180038f0 con 0x55a087ed2a30 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 -- 172.21.15.105:0/3338255712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 msgr2=0x55a087ed2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed2e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f03180093e0 tx=0x7f0318009370 comp rx=0 tx=0).stop 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 -- 172.21.15.105:0/3338255712 shutdown_connections 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed8590 0x55a087edaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed3420 0x55a087ed7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 --2- 172.21.15.105:0/3338255712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed2e50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 -- 172.21.15.105:0/3338255712 >> 172.21.15.105:0/3338255712 conn(0x55a087ecdfa0 msgr2=0x55a087ece3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 -- 172.21.15.105:0/3338255712 shutdown_connections 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 -- 172.21.15.105:0/3338255712 wait complete. 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.153+0000 7f032cfd6740 1 -- start start 2024-03-26T17:41:10.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f032cfd6740 1 Processor -- start 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed2a30 0x55a087edcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087edd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed8590 0x55a087ee44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f032cfd6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a087edb540 con 0x55a087ed2a30 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f032cfd6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a087df3310 con 0x55a087ed8590 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f032cfd6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a087edb6c0 con 0x55a087ed3420 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f032cfd6740 10 client.0 _ll_register_callbacks cb 0x55a087e2ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f03237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed2a30 0x55a087edcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f0323fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087edd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f03237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed2a30 0x55a087edcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52186/0 (socket says 172.21.15.105:52186) 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f0323fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087edd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45270/0 (socket says 172.21.15.105:45270) 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f03237fe640 1 -- 172.21.15.105:0/695697525 learned_addr learned my addr 172.21.15.105:0/695697525 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f03237fe640 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed8590 0x55a087ee44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f0323fff640 1 -- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed8590 msgr2=0x55a087ee44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f0323fff640 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed8590 0x55a087ee44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f0323fff640 1 -- 172.21.15.105:0/695697525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed2a30 msgr2=0x55a087edcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f0323fff640 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed2a30 0x55a087edcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f0323fff640 1 -- 172.21.15.105:0/695697525 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a087c157d0 con 0x55a087ed3420 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.154+0000 7f03237fe640 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed8590 0x55a087ee44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.155+0000 7f0323fff640 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087edd460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0314036040 tx=0x7f0314004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eeb4f6479a71a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.155+0000 7f03037fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.155+0000 7f03037fe640 1 -- 172.21.15.105:0/695697525 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314005030 con 0x55a087ed3420 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.155+0000 7f03037fe640 1 -- 172.21.15.105:0/695697525 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f03140051d0 con 0x55a087ed3420 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.155+0000 7f032cfd6740 1 -- 172.21.15.105:0/695697525 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031800fc70 con 0x55a087ed3420 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.155+0000 7f03037fe640 1 -- 172.21.15.105:0/695697525 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0314051050 con 0x55a087ed3420 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.155+0000 7f032cfd6740 10 client.26230 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.155+0000 7f032cfd6740 1 -- 172.21.15.105:0/695697525 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a087ee7880 con 0x55a087ed3420 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f03037fe640 1 -- 172.21.15.105:0/695697525 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f031800fc70 con 0x55a087ed3420 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f03037fe640 1 -- 172.21.15.105:0/695697525 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f03140035f0 con 0x55a087ed3420 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f032cfd6740 10 client.26230 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f032cfd6740 10 client.26230 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f032cfd6740 20 client.26230 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:10.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f032cfd6740 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a087ee8f50 0x55a087eeb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f032cfd6740 4 client.26230 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f032cfd6740 1 -- 172.21.15.105:0/695697525 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a087e44eb0 con 0x55a087ee8f50 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f03237fe640 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a087ee8f50 0x55a087eeb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f03037fe640 10 client.26230 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.156+0000 7f03237fe640 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a087ee8f50 0x55a087eeb3d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff26c44c17abbb79 server_cookie=b0790f8df5f6eb41 in_seq=0 out_seq=0 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f03037fe640 1 -- 172.21.15.105:0/695697525 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a087e44eb0 con 0x55a087ee8f50 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f03037fe640 10 client.26230 handle_command_reply: tid=0 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 client.26230 shutdown 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 -- 172.21.15.105:0/695697525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 msgr2=0x55a087edd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087edd460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0314036040 tx=0x7f0314004270 comp rx=0 tx=0).stop 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 -- 172.21.15.105:0/695697525 shutdown_connections 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a087ee8f50 0x55a087eeb3d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f03037fe640 0 client.26230 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed8590 0x55a087ee44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087edd460 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 --2- 172.21.15.105:0/695697525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed2a30 0x55a087edcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.157+0000 7f032cfd6740 1 -- 172.21.15.105:0/695697525 >> 172.21.15.105:0/695697525 conn(0x55a087ecdfa0 msgr2=0x55a087ecd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.158+0000 7f032cfd6740 1 -- 172.21.15.105:0/695697525 shutdown_connections 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.158+0000 7f032cfd6740 1 -- 172.21.15.105:0/695697525 wait complete. 2024-03-26T17:41:10.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.158+0000 7f032cfd6740 20 client.26230 trim_cache size 0 max 16384 2024-03-26T17:41:10.171 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:09 smithi135 ceph-mon[35636]: pgmap v432: 97 pgs: 97 active+clean; 27 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:41:10.172 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:09 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3262953219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:10.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f032cfd6740 1 Processor -- start 2024-03-26T17:41:10.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f032cfd6740 1 -- start start 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087ee4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ed4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f032cfd6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03241aafe0 con 0x55a087ed8590 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f032cfd6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03241acb20 con 0x55a087ed2a30 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f032cfd6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03241ab6b0 con 0x55a087ed3420 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ed4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f0323fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087ee4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ed4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52198/0 (socket says 172.21.15.105:52198) 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 -- 172.21.15.105:0/35866288 learned_addr learned my addr 172.21.15.105:0/35866288 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 --2- 172.21.15.105:0/35866288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 -- 172.21.15.105:0/35866288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 msgr2=0x55a087ee4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 --2- 172.21.15.105:0/35866288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087ee4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 -- 172.21.15.105:0/35866288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 msgr2=0x55a087ed48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 --2- 172.21.15.105:0/35866288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.172+0000 7f03237fe640 1 -- 172.21.15.105:0/35866288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a087edba60 con 0x55a087ed8590 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f03237fe640 1 --2- 172.21.15.105:0/35866288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ed4e20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0318009d80 tx=0x7f0318005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8593d277419ae34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f0309ffb640 1 -- 172.21.15.105:0/35866288 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800fc30 con 0x55a087ed8590 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f0309ffb640 1 -- 172.21.15.105:0/35866288 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0318029040 con 0x55a087ed8590 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f0309ffb640 1 -- 172.21.15.105:0/35866288 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031802a820 con 0x55a087ed8590 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 -- 172.21.15.105:0/35866288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 msgr2=0x55a087ed4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 --2- 172.21.15.105:0/35866288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ed4e20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0318009d80 tx=0x7f0318005f90 comp rx=0 tx=0).stop 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 -- 172.21.15.105:0/35866288 shutdown_connections 2024-03-26T17:41:10.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 --2- 172.21.15.105:0/35866288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ed4e20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 --2- 172.21.15.105:0/35866288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed3420 0x55a087ee4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 --2- 172.21.15.105:0/35866288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed2a30 0x55a087ed48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 -- 172.21.15.105:0/35866288 >> 172.21.15.105:0/35866288 conn(0x55a087ecdfa0 msgr2=0x55a087edcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 -- 172.21.15.105:0/35866288 shutdown_connections 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.173+0000 7f032cfd6740 1 -- 172.21.15.105:0/35866288 wait complete. 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 1 -- start start 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 1 Processor -- start 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed2a30 0x55a087ee17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed3420 0x55a087ee1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ee2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03241ac450 con 0x55a087ed8590 2024-03-26T17:41:10.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03241abd80 con 0x55a087ed3420 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03241a8fc0 con 0x55a087ed2a30 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f032cfd6740 10 client.0 _ll_register_callbacks cb 0x7f0314009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f03237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ee2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f03237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ee2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52206/0 (socket says 172.21.15.105:52206) 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f03237fe640 1 -- 172.21.15.105:0/1216652409 learned_addr learned my addr 172.21.15.105:0/1216652409 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f0323fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed3420 0x55a087ee1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.174+0000 7f03237fe640 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed2a30 0x55a087ee17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03237fe640 1 -- 172.21.15.105:0/1216652409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed2a30 msgr2=0x55a087ee17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03237fe640 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed2a30 0x55a087ee17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03237fe640 1 -- 172.21.15.105:0/1216652409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed3420 msgr2=0x55a087ee1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03237fe640 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed3420 0x55a087ee1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03237fe640 1 -- 172.21.15.105:0/1216652409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a087ee32a0 con 0x55a087ed8590 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f0323fff640 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed3420 0x55a087ee1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03237fe640 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ee2cb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0318009940 tx=0x7f0318009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3dda53fa9d7b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03037fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03037fe640 1 -- 172.21.15.105:0/1216652409 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031802a5a0 con 0x55a087ed8590 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03037fe640 1 -- 172.21.15.105:0/1216652409 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f031802a740 con 0x55a087ed8590 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 1 -- 172.21.15.105:0/1216652409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0318029040 con 0x55a087ed8590 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03037fe640 1 -- 172.21.15.105:0/1216652409 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f031802aad0 con 0x55a087ed8590 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 10 client.16386 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:10.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 1 -- 172.21.15.105:0/1216652409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a087ee45a0 con 0x55a087ed8590 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03037fe640 1 -- 172.21.15.105:0/1216652409 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0318029040 con 0x55a087ed8590 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f03037fe640 1 -- 172.21.15.105:0/1216652409 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f03180035f0 con 0x55a087ed8590 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 10 client.16386 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 10 client.16386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 20 client.16386 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a087ee8f50 0x55a087f51870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 4 client.16386 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.175+0000 7f032cfd6740 1 -- 172.21.15.105:0/1216652409 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a087e3f200 con 0x55a087ee8f50 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f0323fff640 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a087ee8f50 0x55a087f51870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f03037fe640 10 client.16386 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f0323fff640 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a087ee8f50 0x55a087f51870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c73ff634537d23e6 server_cookie=13ca51d1c05a72a8 in_seq=0 out_seq=0 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f03037fe640 1 -- 172.21.15.105:0/1216652409 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a087e3f200 con 0x55a087ee8f50 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f03037fe640 10 client.16386 handle_command_reply: tid=0 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 client.16386 shutdown 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 -- 172.21.15.105:0/1216652409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 msgr2=0x55a087ee2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ee2cb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0318009940 tx=0x7f0318009bf0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 -- 172.21.15.105:0/1216652409 shutdown_connections 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a087ed8590 0x55a087ee2cb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a087ee8f50 0x55a087f51870 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a087ed3420 0x55a087ee1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 --2- 172.21.15.105:0/1216652409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a087ed2a30 0x55a087ee17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f03037fe640 0 client.16386 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:10.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.176+0000 7f032cfd6740 1 -- 172.21.15.105:0/1216652409 >> 172.21.15.105:0/1216652409 conn(0x55a087ecdfa0 msgr2=0x55a087edcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032cfd6740 1 -- 172.21.15.105:0/1216652409 shutdown_connections 2024-03-26T17:41:10.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032cfd6740 1 -- 172.21.15.105:0/1216652409 wait complete. 2024-03-26T17:41:10.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032cfd6740 20 client.16386 trim_cache size 0 max 16384 2024-03-26T17:41:10.180 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:10.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 -- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f030c0a2f20 msgr2=0x7f030c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f030c0a2f20 0x7f030c0a53e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f031c098a40 tx=0x7f031c00efd0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 -- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03241056b0 msgr2=0x7f03241a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03241056b0 0x7f03241a1310 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f031c00eb10 tx=0x7f031c00c2d0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 -- 172.21.15.105:0/1854871761 shutdown_connections 2024-03-26T17:41:10.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f030c0a2f20 0x7f030c0a53e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03241056b0 0x7f03241a1310 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0324104cf0 0x7f032419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 --2- 172.21.15.105:0/1854871761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0324103b00 0x7f0324199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 -- 172.21.15.105:0/1854871761 >> 172.21.15.105:0/1854871761 conn(0x7f03240ff070 msgr2=0x7f0324108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 -- 172.21.15.105:0/1854871761 shutdown_connections 2024-03-26T17:41:10.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.177+0000 7f032a3f3640 1 -- 172.21.15.105:0/1854871761 wait complete. 2024-03-26T17:41:10.185 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 -- 172.21.15.105:0/2594436718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8881019b0 msgr2=0x7fc888101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/2594436718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8881019b0 0x7fc888101e30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc870009900 tx=0x7fc87002bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 -- 172.21.15.105:0/2594436718 shutdown_connections 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/2594436718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc888102370 0x7fc888108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/2594436718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8881019b0 0x7fc888101e30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/2594436718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8881007c0 0x7fc888100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 -- 172.21.15.105:0/2594436718 >> 172.21.15.105:0/2594436718 conn(0x7fc8880fbd50 msgr2=0x7fc8880fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 -- 172.21.15.105:0/2594436718 shutdown_connections 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.317+0000 7fc88d6c0640 1 -- 172.21.15.105:0/2594436718 wait complete. 2024-03-26T17:41:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc88d6c0640 1 Processor -- start 2024-03-26T17:41:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc88d6c0640 1 -- start start 2024-03-26T17:41:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc88d6c0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8881007c0 0x7fc888196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc88d6c0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc888102370 0x7fc888197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc88d6c0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88819c210 0x7fc88819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc88d6c0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810c2f0 con 0x7fc88819c210 2024-03-26T17:41:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc88d6c0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810c170 con 0x7fc8881007c0 2024-03-26T17:41:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc88d6c0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc88810c470 con 0x7fc888102370 2024-03-26T17:41:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc8867fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc888102370 0x7fc888197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc886ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8881007c0 0x7fc888196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc8867fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc888102370 0x7fc888197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45310/0 (socket says 172.21.15.105:45310) 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc8877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88819c210 0x7fc88819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.318+0000 7fc886ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8881007c0 0x7fc888196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53992/0 (socket says 172.21.15.105:53992) 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc8867fc640 1 -- 172.21.15.105:0/1900382874 learned_addr learned my addr 172.21.15.105:0/1900382874 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc8867fc640 1 -- 172.21.15.105:0/1900382874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8881007c0 msgr2=0x7fc888196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc8867fc640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8881007c0 0x7fc888196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc8867fc640 1 -- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88819c210 msgr2=0x7fc88819e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc8867fc640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88819c210 0x7fc88819e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc8867fc640 1 -- 172.21.15.105:0/1900382874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88819eca0 con 0x7fc888102370 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc886ffd640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8881007c0 0x7fc888196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc8867fc640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc888102370 0x7fc888197210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc870036040 tx=0x7fc870005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc87ffff640 1 -- 172.21.15.105:0/1900382874 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870004e00 con 0x7fc888102370 2024-03-26T17:41:10.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc88d6c0640 1 -- 172.21.15.105:0/1900382874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc888068a20 con 0x7fc888102370 2024-03-26T17:41:10.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc87ffff640 1 -- 172.21.15.105:0/1900382874 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc870005450 con 0x7fc888102370 2024-03-26T17:41:10.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc87ffff640 1 -- 172.21.15.105:0/1900382874 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc870047480 con 0x7fc888102370 2024-03-26T17:41:10.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.319+0000 7fc88d6c0640 1 -- 172.21.15.105:0/1900382874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc888068f70 con 0x7fc888102370 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc87ffff640 1 -- 172.21.15.105:0/1900382874 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc870003680 con 0x7fc888102370 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc890387740 1 Processor -- start 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc890387740 1 -- start start 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc890387740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc890387740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba59420 0x56431ba5df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc890387740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba60a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc890387740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56431b9bd970 con 0x56431ba5e590 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc890387740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56431b965d70 con 0x56431ba59420 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.322+0000 7fc890387740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56431b9b9920 con 0x56431ba58a30 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc87ffff640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba60a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba58e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45312/0 (socket says 172.21.15.105:45312) 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba60a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52246/0 (socket says 172.21.15.105:52246) 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 -- 172.21.15.105:0/3429431001 learned_addr learned my addr 172.21.15.105:0/3429431001 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc886ffd640 1 --2- 172.21.15.105:0/3429431001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba59420 0x56431ba5df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc87ffff640 1 -- 172.21.15.105:0/1900382874 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc8700f3420 con 0x7fc888102370 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 -- 172.21.15.105:0/3429431001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba59420 msgr2=0x56431ba5df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 --2- 172.21.15.105:0/3429431001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba59420 0x56431ba5df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 -- 172.21.15.105:0/3429431001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 msgr2=0x56431ba60a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 --2- 172.21.15.105:0/3429431001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba60a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 -- 172.21.15.105:0/3429431001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431b6d13f0 con 0x56431ba58a30 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 --2- 172.21.15.105:0/3429431001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba58e50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc878005c40 tx=0x7fc87801c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b1cbcf90b036f08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc87d7fa640 1 -- 172.21.15.105:0/3429431001 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878029070 con 0x56431ba58a30 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc87d7fa640 1 -- 172.21.15.105:0/3429431001 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc878003e00 con 0x56431ba58a30 2024-03-26T17:41:10.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc8877fe640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc878026000 tx=0x7fc878023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.323+0000 7fc87d7fa640 1 -- 172.21.15.105:0/3429431001 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878010690 con 0x56431ba58a30 2024-03-26T17:41:10.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 -- 172.21.15.105:0/3429431001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 msgr2=0x56431ba58e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 --2- 172.21.15.105:0/3429431001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba58e50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc878005c40 tx=0x7fc87801c4f0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 -- 172.21.15.105:0/3429431001 shutdown_connections 2024-03-26T17:41:10.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 --2- 172.21.15.105:0/3429431001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba60a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 --2- 172.21.15.105:0/3429431001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba59420 0x56431ba5df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 --2- 172.21.15.105:0/3429431001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba58e50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 -- 172.21.15.105:0/3429431001 >> 172.21.15.105:0/3429431001 conn(0x56431ba53fa0 msgr2=0x56431ba543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 -- 172.21.15.105:0/3429431001 shutdown_connections 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.324+0000 7fc890387740 1 -- 172.21.15.105:0/3429431001 wait complete. 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 1 -- start start 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 1 Processor -- start 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba59420 0x56431ba635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba5e590 0x56431ba6a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56431ba61540 con 0x56431ba59420 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56431b979310 con 0x56431ba5e590 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56431ba616c0 con 0x56431ba58a30 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc890387740 10 client.0 _ll_register_callbacks cb 0x56431b9b4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc8867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba59420 0x56431ba635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc886ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc886ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45322/0 (socket says 172.21.15.105:45322) 2024-03-26T17:41:10.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc8877fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba5e590 0x56431ba6a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc886ffd640 1 -- 172.21.15.105:0/4032471379 learned_addr learned my addr 172.21.15.105:0/4032471379 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.325+0000 7fc886ffd640 1 -- 172.21.15.105:0/4032471379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba5e590 msgr2=0x56431ba6a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc886ffd640 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba5e590 0x56431ba6a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc886ffd640 1 -- 172.21.15.105:0/4032471379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba59420 msgr2=0x56431ba635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc886ffd640 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba59420 0x56431ba635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc886ffd640 1 -- 172.21.15.105:0/4032471379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431b79b7d0 con 0x56431ba58a30 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc886ffd640 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba630a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc880012cf0 tx=0x7fc88000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5045ba12e260db6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc86e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc86e7fc640 1 -- 172.21.15.105:0/4032471379 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880012110 con 0x56431ba58a30 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc86e7fc640 1 -- 172.21.15.105:0/4032471379 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc880004050 con 0x56431ba58a30 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc890387740 1 -- 172.21.15.105:0/4032471379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc878003e00 con 0x56431ba58a30 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc86e7fc640 1 -- 172.21.15.105:0/4032471379 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8800122b0 con 0x56431ba58a30 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc890387740 10 client.26248 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:10.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc890387740 1 -- 172.21.15.105:0/4032471379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56431ba6d9f0 con 0x56431ba58a30 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.326+0000 7fc86e7fc640 1 -- 172.21.15.105:0/4032471379 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc880004250 con 0x56431ba58a30 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc86e7fc640 1 -- 172.21.15.105:0/4032471379 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc880010430 con 0x56431ba58a30 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc890387740 10 client.26248 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc890387740 10 client.26248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc890387740 20 client.26248 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc890387740 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56431ba6f0f0 0x56431ba71550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc890387740 4 client.26248 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc890387740 1 -- 172.21.15.105:0/4032471379 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc878003e00 con 0x56431ba6f0f0 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc8867fc640 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56431ba6f0f0 0x56431ba71550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc8867fc640 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56431ba6f0f0 0x56431ba71550 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4de7e5ea2fda988 server_cookie=66b385890d3b6631 in_seq=0 out_seq=0 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.327+0000 7fc86e7fc640 10 client.26248 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc86e7fc640 1 -- 172.21.15.105:0/4032471379 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fc878003e00 con 0x56431ba6f0f0 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc86e7fc640 10 client.26248 handle_command_reply: tid=0 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 client.26248 shutdown 2024-03-26T17:41:10.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 -- 172.21.15.105:0/4032471379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 msgr2=0x56431ba630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba630a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc880012cf0 tx=0x7fc88000f790 comp rx=0 tx=0).stop 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 -- 172.21.15.105:0/4032471379 shutdown_connections 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56431ba6f0f0 0x56431ba71550 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba5e590 0x56431ba6a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc86e7fc640 0 client.26248 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba59420 0x56431ba635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 --2- 172.21.15.105:0/4032471379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba58a30 0x56431ba630a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 -- 172.21.15.105:0/4032471379 >> 172.21.15.105:0/4032471379 conn(0x56431ba53fa0 msgr2=0x56431ba531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 -- 172.21.15.105:0/4032471379 shutdown_connections 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 1 -- 172.21.15.105:0/4032471379 wait complete. 2024-03-26T17:41:10.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.328+0000 7fc890387740 20 client.26248 trim_cache size 0 max 16384 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.343+0000 7fc890387740 1 Processor -- start 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.343+0000 7fc890387740 1 -- start start 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.343+0000 7fc890387740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba57910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.343+0000 7fc890387740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba6a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.343+0000 7fc890387740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.343+0000 7fc890387740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881a70f0 con 0x56431ba5e590 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.343+0000 7fc890387740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881a8c30 con 0x56431ba58a30 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.343+0000 7fc890387740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881a77c0 con 0x56431ba59420 2024-03-26T17:41:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8867fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba6a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc886ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba57910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52260/0 (socket says 172.21.15.105:52260) 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8867fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba6a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45328/0 (socket says 172.21.15.105:45328) 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 -- 172.21.15.105:0/2167228936 learned_addr learned my addr 172.21.15.105:0/2167228936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc886ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba57910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54018/0 (socket says 172.21.15.105:54018) 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 -- 172.21.15.105:0/2167228936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 msgr2=0x56431ba6a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 --2- 172.21.15.105:0/2167228936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba6a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 -- 172.21.15.105:0/2167228936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 msgr2=0x56431ba57910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 --2- 172.21.15.105:0/2167228936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba57910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 -- 172.21.15.105:0/2167228936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431ba59800 con 0x56431ba5e590 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc8877fe640 1 --2- 172.21.15.105:0/2167228936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba626c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc87801c9f0 tx=0x7fc878009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c852e884585b75 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc87cff9640 1 -- 172.21.15.105:0/2167228936 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878031020 con 0x56431ba5e590 2024-03-26T17:41:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc87cff9640 1 -- 172.21.15.105:0/2167228936 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc878009b30 con 0x56431ba5e590 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.344+0000 7fc87cff9640 1 -- 172.21.15.105:0/2167228936 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878029070 con 0x56431ba5e590 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 -- 172.21.15.105:0/2167228936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 msgr2=0x56431ba626c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 --2- 172.21.15.105:0/2167228936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba626c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc87801c9f0 tx=0x7fc878009500 comp rx=0 tx=0).stop 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 -- 172.21.15.105:0/2167228936 shutdown_connections 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 --2- 172.21.15.105:0/2167228936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba626c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 --2- 172.21.15.105:0/2167228936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba6a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 --2- 172.21.15.105:0/2167228936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba57910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 -- 172.21.15.105:0/2167228936 >> 172.21.15.105:0/2167228936 conn(0x56431ba53fa0 msgr2=0x56431ba57200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 -- 172.21.15.105:0/2167228936 shutdown_connections 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 -- 172.21.15.105:0/2167228936 wait complete. 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 -- start start 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.345+0000 7fc890387740 1 Processor -- start 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc890387740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc890387740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba67f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc890387740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba68f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc890387740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881a8560 con 0x56431ba5e590 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc890387740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881a7e90 con 0x56431ba58a30 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc890387740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881a50d0 con 0x56431ba59420 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc890387740 10 client.0 _ll_register_callbacks cb 0x7fc880009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba68f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba68f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52276/0 (socket says 172.21.15.105:52276) 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 -- 172.21.15.105:0/173787984 learned_addr learned my addr 172.21.15.105:0/173787984 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8867fc640 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba67f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc886ffd640 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 -- 172.21.15.105:0/173787984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 msgr2=0x56431ba67f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba67f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 -- 172.21.15.105:0/173787984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 msgr2=0x56431ba679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba679d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 -- 172.21.15.105:0/173787984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431ba69510 con 0x56431ba5e590 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc8877fe640 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba68f20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc878005d70 tx=0x7fc878010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19377d8caf77311 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc86e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc86e7fc640 1 -- 172.21.15.105:0/173787984 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87803a070 con 0x56431ba5e590 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc890387740 1 -- 172.21.15.105:0/173787984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc878009b30 con 0x56431ba5e590 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc86e7fc640 1 -- 172.21.15.105:0/173787984 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc87801c020 con 0x56431ba5e590 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.346+0000 7fc86e7fc640 1 -- 172.21.15.105:0/173787984 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc878029030 con 0x56431ba5e590 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc890387740 10 client.16404 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc890387740 1 -- 172.21.15.105:0/173787984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56431ba6a810 con 0x56431ba5e590 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc86e7fc640 1 -- 172.21.15.105:0/173787984 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc878009b30 con 0x56431ba5e590 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc86e7fc640 1 -- 172.21.15.105:0/173787984 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc87803d030 con 0x56431ba5e590 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc890387740 10 client.16404 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc890387740 10 client.16404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc890387740 20 client.16404 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc890387740 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56431ba6f0f0 0x56431bad79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc890387740 4 client.16404 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc890387740 1 -- 172.21.15.105:0/173787984 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56431b9caeb0 con 0x56431ba6f0f0 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.347+0000 7fc886ffd640 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56431ba6f0f0 0x56431bad79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc86e7fc640 10 client.16404 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc886ffd640 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56431ba6f0f0 0x56431bad79c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8c81fe5322e2183 server_cookie=3870c0f2035b05 in_seq=0 out_seq=0 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc86e7fc640 1 -- 172.21.15.105:0/173787984 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56431b9caeb0 con 0x56431ba6f0f0 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc86e7fc640 10 client.16404 handle_command_reply: tid=0 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 client.16404 shutdown 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 -- 172.21.15.105:0/173787984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 msgr2=0x56431ba68f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba68f20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc878005d70 tx=0x7fc878010d40 comp rx=0 tx=0).stop 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 -- 172.21.15.105:0/173787984 shutdown_connections 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56431ba6f0f0 0x56431bad79c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ba5e590 0x56431ba68f20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56431ba59420 0x56431ba67f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 --2- 172.21.15.105:0/173787984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56431ba58a30 0x56431ba679d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 -- 172.21.15.105:0/173787984 >> 172.21.15.105:0/173787984 conn(0x56431ba53fa0 msgr2=0x56431ba52930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc86e7fc640 0 client.16404 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 -- 172.21.15.105:0/173787984 shutdown_connections 2024-03-26T17:41:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 1 -- 172.21.15.105:0/173787984 wait complete. 2024-03-26T17:41:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.348+0000 7fc890387740 20 client.16404 trim_cache size 0 max 16384 2024-03-26T17:41:10.348 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:10.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 -- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 msgr2=0x7fc8740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc878026000 tx=0x7fc878023000 comp rx=0 tx=0).stop 2024-03-26T17:41:10.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 -- 172.21.15.105:0/1900382874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc888102370 msgr2=0x7fc888197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc888102370 0x7fc888197210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc870036040 tx=0x7fc870005dc0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 -- 172.21.15.105:0/1900382874 shutdown_connections 2024-03-26T17:41:10.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88819c210 0x7fc88819e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc888102370 0x7fc888197210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 --2- 172.21.15.105:0/1900382874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc8881007c0 0x7fc888196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 -- 172.21.15.105:0/1900382874 >> 172.21.15.105:0/1900382874 conn(0x7fc8880fbd50 msgr2=0x7fc888104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 -- 172.21.15.105:0/1900382874 shutdown_connections 2024-03-26T17:41:10.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.349+0000 7fc88d6c0640 1 -- 172.21.15.105:0/1900382874 wait complete. 2024-03-26T17:41:10.353 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:41:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 -- 172.21.15.105:0/4040723030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1750101dc0 msgr2=0x7f17501021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 --2- 172.21.15.105:0/4040723030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1750101dc0 0x7f17501021e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1740009900 tx=0x7f174002bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:10.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 -- 172.21.15.105:0/4040723030 shutdown_connections 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 --2- 172.21.15.105:0/4040723030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1750100300 0x7f1750100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 --2- 172.21.15.105:0/4040723030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17500ff940 0x7f17500ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 --2- 172.21.15.105:0/4040723030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1750101dc0 0x7f17501021e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 -- 172.21.15.105:0/4040723030 >> 172.21.15.105:0/4040723030 conn(0x7f17500fb4c0 msgr2=0x7f17500fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 -- 172.21.15.105:0/4040723030 shutdown_connections 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.481+0000 7f17553f3640 1 -- 172.21.15.105:0/4040723030 wait complete. 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f17553f3640 1 Processor -- start 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f17553f3640 1 -- start start 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f17553f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17500ff940 0x7f1750199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f17553f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100300 0x7f175019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f17553f3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1750101dc0 0x7f17501a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f17553f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f175010f620 con 0x7f17500ff940 2024-03-26T17:41:10.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f17553f3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f175010f4a0 con 0x7f1750101dc0 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f17553f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f175010f7a0 con 0x7f1750100300 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f174effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17500ff940 0x7f1750199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f174effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17500ff940 0x7f1750199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52302/0 (socket says 172.21.15.105:52302) 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f174effd640 1 -- 172.21.15.105:0/3687712494 learned_addr learned my addr 172.21.15.105:0/3687712494 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f174e7fc640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100300 0x7f175019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.482+0000 7f174f7fe640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1750101dc0 0x7f17501a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f174effd640 1 -- 172.21.15.105:0/3687712494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100300 msgr2=0x7f175019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f174effd640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100300 0x7f175019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f174effd640 1 -- 172.21.15.105:0/3687712494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1750101dc0 msgr2=0x7f17501a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f174effd640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1750101dc0 0x7f17501a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f174effd640 1 -- 172.21.15.105:0/3687712494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17501a1960 con 0x7f17500ff940 2024-03-26T17:41:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f174effd640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17500ff940 0x7f1750199d50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1740006010 tx=0x7f1740005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f1737fff640 1 -- 172.21.15.105:0/3687712494 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740043020 con 0x7f17500ff940 2024-03-26T17:41:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f17553f3640 1 -- 172.21.15.105:0/3687712494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17501a1bf0 con 0x7f17500ff940 2024-03-26T17:41:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f1737fff640 1 -- 172.21.15.105:0/3687712494 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17400043d0 con 0x7f17500ff940 2024-03-26T17:41:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f1737fff640 1 -- 172.21.15.105:0/3687712494 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740003b80 con 0x7f17500ff940 2024-03-26T17:41:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.483+0000 7f17553f3640 1 -- 172.21.15.105:0/3687712494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17501a2140 con 0x7f17500ff940 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1737fff640 1 -- 172.21.15.105:0/3687712494 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1740046020 con 0x7f17500ff940 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1757fd9740 1 Processor -- start 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1757fd9740 1 -- start start 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc1a30 0x55c082bc1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bc6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc7590 0x55c082bc9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1757fd9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c082b26970 con 0x55c082bc7590 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1757fd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c082aced70 con 0x55c082bc1a30 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1757fd9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c082b22920 con 0x55c082bc2420 2024-03-26T17:41:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f174e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc7590 0x55c082bc9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f174e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc7590 0x55c082bc9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52308/0 (socket says 172.21.15.105:52308) 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f174e7fc640 1 -- 172.21.15.105:0/2706626691 learned_addr learned my addr 172.21.15.105:0/2706626691 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f174f7fe640 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc1a30 0x55c082bc1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f174effd640 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bc6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.486+0000 7f1737fff640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17380a2f20 0x7f17380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174e7fc640 1 -- 172.21.15.105:0/2706626691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 msgr2=0x55c082bc6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174e7fc640 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bc6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f1737fff640 1 -- 172.21.15.105:0/3687712494 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f17400f1680 con 0x7f17500ff940 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174e7fc640 1 -- 172.21.15.105:0/2706626691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc1a30 msgr2=0x55c082bc1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174e7fc640 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc1a30 0x55c082bc1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174e7fc640 1 -- 172.21.15.105:0/2706626691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08283a3f0 con 0x55c082bc7590 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174f7fe640 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc1a30 0x55c082bc1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174effd640 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bc6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174f7fe640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17380a2f20 0x7f17380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f174e7fc640 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc7590 0x55c082bc9a20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1744009d20 tx=0x7f1744011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38aa02766068172d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f17357fa640 1 -- 172.21.15.105:0/2706626691 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174401e070 con 0x55c082bc7590 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f17357fa640 1 -- 172.21.15.105:0/2706626691 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1744019ae0 con 0x55c082bc7590 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.487+0000 7f17357fa640 1 -- 172.21.15.105:0/2706626691 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174400eac0 con 0x55c082bc7590 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 -- 172.21.15.105:0/2706626691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc7590 msgr2=0x55c082bc9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc7590 0x55c082bc9a20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1744009d20 tx=0x7f1744011810 comp rx=0 tx=0).stop 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f174f7fe640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17380a2f20 0x7f17380a53e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55c082bc2c80 tx=0x7f1748014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 -- 172.21.15.105:0/2706626691 shutdown_connections 2024-03-26T17:41:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc7590 0x55c082bc9a20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bc6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 --2- 172.21.15.105:0/2706626691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc1a30 0x55c082bc1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 -- 172.21.15.105:0/2706626691 >> 172.21.15.105:0/2706626691 conn(0x55c082bbcfa0 msgr2=0x55c082bbd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 -- 172.21.15.105:0/2706626691 shutdown_connections 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 -- 172.21.15.105:0/2706626691 wait complete. 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 -- start start 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.488+0000 7f1757fd9740 1 Processor -- start 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc1a30 0x55c082bcbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc2420 0x55c082bcc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1757fd9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c082bca540 con 0x55c082bc2420 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1757fd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c082ae2310 con 0x55c082bc7590 2024-03-26T17:41:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1757fd9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c082bca6c0 con 0x55c082bc1a30 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1757fd9740 10 client.0 _ll_register_callbacks cb 0x55c082b1dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc2420 0x55c082bcc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc2420 0x55c082bcc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52312/0 (socket says 172.21.15.105:52312) 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174e7fc640 1 -- 172.21.15.105:0/3442573266 learned_addr learned my addr 172.21.15.105:0/3442573266 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174effd640 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc1a30 0x55c082bcbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174f7fe640 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174effd640 1 -- 172.21.15.105:0/3442573266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 msgr2=0x55c082bd34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174effd640 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174effd640 1 -- 172.21.15.105:0/3442573266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc2420 msgr2=0x55c082bcc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174effd640 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc2420 0x55c082bcc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174effd640 1 -- 172.21.15.105:0/3442573266 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0829047d0 con 0x55c082bc1a30 2024-03-26T17:41:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174e7fc640 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc2420 0x55c082bcc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f174effd640 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc1a30 0x55c082bcbf20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17400ef190 tx=0x7f1740009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=870f5415e277a4cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1732ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.489+0000 7f1732ff5640 1 -- 172.21.15.105:0/3442573266 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17400bbeb0 con 0x55c082bc1a30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1732ff5640 1 -- 172.21.15.105:0/3442573266 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f17400bdfd0 con 0x55c082bc1a30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 1 -- 172.21.15.105:0/3442573266 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1744019ae0 con 0x55c082bc1a30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1732ff5640 1 -- 172.21.15.105:0/3442573266 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17400056b0 con 0x55c082bc1a30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 10 client.26266 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 1 -- 172.21.15.105:0/3442573266 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c082bca5b0 con 0x55c082bc1a30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1732ff5640 1 -- 172.21.15.105:0/3442573266 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1744019ae0 con 0x55c082bc1a30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1732ff5640 1 -- 172.21.15.105:0/3442573266 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f17400bd1c0 con 0x55c082bc1a30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 10 client.26266 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 10 client.26266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 20 client.26266 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c082bd7f10 0x55c082bda380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 4 client.26266 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f1757fd9740 1 -- 172.21.15.105:0/3442573266 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c082b33eb0 con 0x55c082bd7f10 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.490+0000 7f174e7fc640 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c082bd7f10 0x55c082bda380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.491+0000 7f1732ff5640 10 client.26266 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.491+0000 7f174e7fc640 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c082bd7f10 0x55c082bda380 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b820e9b3111cf0cd server_cookie=91d1b5d476acae0b in_seq=0 out_seq=0 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.491+0000 7f1732ff5640 1 -- 172.21.15.105:0/3442573266 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c082b33eb0 con 0x55c082bd7f10 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.491+0000 7f1732ff5640 10 client.26266 handle_command_reply: tid=0 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.491+0000 7f1757fd9740 1 client.26266 shutdown 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 -- 172.21.15.105:0/3442573266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc1a30 msgr2=0x55c082bcbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc1a30 0x55c082bcbf20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17400ef190 tx=0x7f1740009630 comp rx=0 tx=0).stop 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 -- 172.21.15.105:0/3442573266 shutdown_connections 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c082bd7f10 0x55c082bda380 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc2420 0x55c082bcc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3442573266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc1a30 0x55c082bcbf20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1732ff5640 0 client.26266 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 -- 172.21.15.105:0/3442573266 >> 172.21.15.105:0/3442573266 conn(0x55c082bbcfa0 msgr2=0x55c082bbc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 -- 172.21.15.105:0/3442573266 shutdown_connections 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 1 -- 172.21.15.105:0/3442573266 wait complete. 2024-03-26T17:41:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.492+0000 7f1757fd9740 20 client.26266 trim_cache size 0 max 16384 2024-03-26T17:41:10.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f1757fd9740 1 Processor -- start 2024-03-26T17:41:10.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f1757fd9740 1 -- start start 2024-03-26T17:41:10.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bc9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bd3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bcc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f1757fd9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17501aaf30 con 0x55c082bc1a30 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f1757fd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17501aca70 con 0x55c082bc7590 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f1757fd9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17501ab600 con 0x55c082bc2420 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f174effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bc9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.506+0000 7f174effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bc9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52318/0 (socket says 172.21.15.105:52318) 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174effd640 1 -- 172.21.15.105:0/3321514189 learned_addr learned my addr 172.21.15.105:0/3321514189 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174f7fe640 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bcc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174e7fc640 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bd3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174effd640 1 -- 172.21.15.105:0/3321514189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 msgr2=0x55c082bd3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174effd640 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bd3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174effd640 1 -- 172.21.15.105:0/3321514189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 msgr2=0x55c082bcc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174effd640 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bcc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174effd640 1 -- 172.21.15.105:0/3321514189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c082bbc8f0 con 0x55c082bc1a30 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f174effd640 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bc9980 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f17400bdd00 tx=0x7f17400de0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bf937bddda82c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f1734ff9640 1 -- 172.21.15.105:0/3321514189 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17400da080 con 0x55c082bc1a30 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f1734ff9640 1 -- 172.21.15.105:0/3321514189 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17400be470 con 0x55c082bc1a30 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.507+0000 7f1734ff9640 1 -- 172.21.15.105:0/3321514189 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17400bdfd0 con 0x55c082bc1a30 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 -- 172.21.15.105:0/3321514189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 msgr2=0x55c082bc9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bc9980 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f17400bdd00 tx=0x7f17400de0b0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 -- 172.21.15.105:0/3321514189 shutdown_connections 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bcc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bd3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3321514189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bc9980 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 -- 172.21.15.105:0/3321514189 >> 172.21.15.105:0/3321514189 conn(0x55c082bbcfa0 msgr2=0x55c082bcbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 -- 172.21.15.105:0/3321514189 shutdown_connections 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 -- 172.21.15.105:0/3321514189 wait complete. 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 -- start start 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.508+0000 7f1757fd9740 1 Processor -- start 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bd0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bd10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1757fd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1757fd9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17501ac3a0 con 0x55c082bc1a30 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1757fd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17501abcd0 con 0x55c082bc7590 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1757fd9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17501a8f10 con 0x55c082bc2420 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1757fd9740 10 client.0 _ll_register_callbacks cb 0x7f1740009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:10.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bd0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bd0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52330/0 (socket says 172.21.15.105:52330) 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 -- 172.21.15.105:0/3576595866 learned_addr learned my addr 172.21.15.105:0/3576595866 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174f7fe640 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174e7fc640 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bd10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 -- 172.21.15.105:0/3576595866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 msgr2=0x55c082bd10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bd10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 -- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 msgr2=0x55c082bd2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 -- 172.21.15.105:0/3576595866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c082bd2630 con 0x55c082bc1a30 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174f7fe640 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:10.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f174effd640 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bd0b60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f17400ba660 tx=0x7f17400ba080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240fc62b7ef8335d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1732ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1732ff5640 1 -- 172.21.15.105:0/3576595866 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17400da050 con 0x55c082bc1a30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1732ff5640 1 -- 172.21.15.105:0/3576595866 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f174009cd60 con 0x55c082bc1a30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.509+0000 7f1757fd9740 1 -- 172.21.15.105:0/3576595866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17400be470 con 0x55c082bc1a30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1732ff5640 1 -- 172.21.15.105:0/3576595866 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17400bbeb0 con 0x55c082bc1a30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1757fd9740 10 client.16422 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1757fd9740 1 -- 172.21.15.105:0/3576595866 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c082bd2630 con 0x55c082bc1a30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1732ff5640 1 -- 172.21.15.105:0/3576595866 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17400be470 con 0x55c082bc1a30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1732ff5640 1 -- 172.21.15.105:0/3576595866 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f17400bcc30 con 0x55c082bc1a30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1757fd9740 10 client.16422 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1757fd9740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1757fd9740 20 client.16422 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c082c3e7c0 0x55c082c40bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:10.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1757fd9740 4 client.16422 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f1757fd9740 1 -- 172.21.15.105:0/3576595866 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c082b2e200 con 0x55c082c3e7c0 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.510+0000 7f174e7fc640 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c082c3e7c0 0x55c082c40bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1732ff5640 10 client.16422 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f174e7fc640 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c082c3e7c0 0x55c082c40bb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4eb3608a8c95d5 server_cookie=ac86cb3058f34bae in_seq=0 out_seq=0 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1732ff5640 1 -- 172.21.15.105:0/3576595866 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c082b2e200 con 0x55c082c3e7c0 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1732ff5640 10 client.16422 handle_command_reply: tid=0 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 client.16422 shutdown 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 -- 172.21.15.105:0/3576595866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 msgr2=0x55c082bd0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bd0b60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f17400ba660 tx=0x7f17400ba080 comp rx=0 tx=0).stop 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 -- 172.21.15.105:0/3576595866 shutdown_connections 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c082c3e7c0 0x55c082c40bb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c082bc7590 0x55c082bd2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c082bc2420 0x55c082bd10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 --2- 172.21.15.105:0/3576595866 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c082bc1a30 0x55c082bd0b60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 -- 172.21.15.105:0/3576595866 >> 172.21.15.105:0/3576595866 conn(0x55c082bbcfa0 msgr2=0x55c082bbb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1732ff5640 0 client.16422 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:10.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 -- 172.21.15.105:0/3576595866 shutdown_connections 2024-03-26T17:41:10.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 1 -- 172.21.15.105:0/3576595866 wait complete. 2024-03-26T17:41:10.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.511+0000 7f1757fd9740 20 client.16422 trim_cache size 0 max 16384 2024-03-26T17:41:10.513 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:10.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 -- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17380a2f20 msgr2=0x7f17380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17380a2f20 0x7f17380a53e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55c082bc2c80 tx=0x7f1748014040 comp rx=0 tx=0).stop 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 -- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17500ff940 msgr2=0x7f1750199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17500ff940 0x7f1750199d50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1740006010 tx=0x7f1740005e40 comp rx=0 tx=0).stop 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 -- 172.21.15.105:0/3687712494 shutdown_connections 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17380a2f20 0x7f17380a53e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1750101dc0 0x7f17501a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100300 0x7f175019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 --2- 172.21.15.105:0/3687712494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17500ff940 0x7f1750199d50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 -- 172.21.15.105:0/3687712494 >> 172.21.15.105:0/3687712494 conn(0x7f17500fb4c0 msgr2=0x7f17500fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 -- 172.21.15.105:0/3687712494 shutdown_connections 2024-03-26T17:41:10.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:10.512+0000 7f17553f3640 1 -- 172.21.15.105:0/3687712494 wait complete. 2024-03-26T17:41:11.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 67820 3.37 MB/sec warmup 114 sec latency 15.518 ms 2024-03-26T17:41:11.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:41:11.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:41:11.517 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:41:11.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 -- 172.21.15.105:0/2738207530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c28102380 msgr2=0x7f1c28104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 --2- 172.21.15.105:0/2738207530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c28102380 0x7f1c28104770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009900 tx=0x7f1c2402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:11.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 -- 172.21.15.105:0/2738207530 shutdown_connections 2024-03-26T17:41:11.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 --2- 172.21.15.105:0/2738207530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c281075e0 0x7f1c281099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 --2- 172.21.15.105:0/2738207530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c28104cb0 0x7f1c281070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 --2- 172.21.15.105:0/2738207530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c28102380 0x7f1c28104770 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 -- 172.21.15.105:0/2738207530 >> 172.21.15.105:0/2738207530 conn(0x7f1c280fbd50 msgr2=0x7f1c280fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 -- 172.21.15.105:0/2738207530 shutdown_connections 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 -- 172.21.15.105:0/2738207530 wait complete. 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 Processor -- start 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 -- start start 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c28102380 0x7f1c28197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c28104cb0 0x7f1c281980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.640+0000 7f1c302f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c281075e0 0x7f1c2819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c302f1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810d3e0 con 0x7f1c281075e0 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c302f1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810d260 con 0x7f1c28104cb0 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c302f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2810d560 con 0x7f1c28102380 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c28102380 0x7f1c28197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2e88e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c281075e0 0x7f1c2819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2d88c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c28104cb0 0x7f1c281980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c28102380 0x7f1c28197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45412/0 (socket says 172.21.15.105:45412) 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2e08d640 1 -- 172.21.15.105:0/878072100 learned_addr learned my addr 172.21.15.105:0/878072100 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2d88c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c28104cb0 0x7f1c281980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54106/0 (socket says 172.21.15.105:54106) 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2e88e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c281075e0 0x7f1c2819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52346/0 (socket says 172.21.15.105:52346) 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2d88c640 1 -- 172.21.15.105:0/878072100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c28102380 msgr2=0x7f1c28197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c28102380 0x7f1c28197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2d88c640 1 -- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c281075e0 msgr2=0x7f1c2819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c281075e0 0x7f1c2819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2d88c640 1 -- 172.21.15.105:0/878072100 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2819f780 con 0x7f1c28104cb0 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c28104cb0 0x7f1c281980a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800e9c0 tx=0x7f1c1800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c1f7fe640 1 -- 172.21.15.105:0/878072100 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800ccc0 con 0x7f1c28104cb0 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c302f1640 1 -- 172.21.15.105:0/878072100 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2819fa70 con 0x7f1c28104cb0 2024-03-26T17:41:11.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c1f7fe640 1 -- 172.21.15.105:0/878072100 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c180108e0 con 0x7f1c28104cb0 2024-03-26T17:41:11.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.641+0000 7f1c1f7fe640 1 -- 172.21.15.105:0/878072100 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180187a0 con 0x7f1c28104cb0 2024-03-26T17:41:11.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.642+0000 7f1c302f1640 1 -- 172.21.15.105:0/878072100 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c281a0010 con 0x7f1c28104cb0 2024-03-26T17:41:11.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.644+0000 7f1c1f7fe640 1 -- 172.21.15.105:0/878072100 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1c18010a90 con 0x7f1c28104cb0 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.644+0000 7f1c32fb8740 1 Processor -- start 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c32fb8740 1 -- start start 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d3215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321b80 0x55d65d326680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d326cf0 0x55d65d329180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d65d286630 con 0x55d65d321b80 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d65d22ee10 con 0x55d65d321190 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c1f7fe640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c140a2fa0 0x7f1c140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d65d2825e0 con 0x55d65d326cf0 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321b80 0x55d65d326680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e88e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d3215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321b80 0x55d65d326680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52354/0 (socket says 172.21.15.105:52354) 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2d88c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d326cf0 0x55d65d329180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e08d640 1 -- 172.21.15.105:0/2717441202 learned_addr learned my addr 172.21.15.105:0/2717441202 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e88e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d3215b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54120/0 (socket says 172.21.15.105:54120) 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c140a2fa0 0x7f1c140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c1f7fe640 1 -- 172.21.15.105:0/878072100 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1c180cf6f0 con 0x7f1c28104cb0 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e88e640 1 -- 172.21.15.105:0/2717441202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d326cf0 msgr2=0x55d65d329180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d326cf0 0x55d65d329180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e88e640 1 -- 172.21.15.105:0/2717441202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321b80 msgr2=0x55d65d326680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321b80 0x55d65d326680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e88e640 1 -- 172.21.15.105:0/2717441202 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65d245740 con 0x55d65d321190 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321b80 0x55d65d326680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.645+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d326cf0 0x55d65d329180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.646+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d3215b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1c20011d30 tx=0x7f1c2000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0d3b449f278064 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.646+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c140a2fa0 0x7f1c140a5460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000fc90 tx=0x7f1c20018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.646+0000 7f1c1cbfe640 1 -- 172.21.15.105:0/2717441202 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20011570 con 0x55d65d321190 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.646+0000 7f1c1cbfe640 1 -- 172.21.15.105:0/2717441202 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c20009d30 con 0x55d65d321190 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.646+0000 7f1c1cbfe640 1 -- 172.21.15.105:0/2717441202 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20005870 con 0x55d65d321190 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2717441202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 msgr2=0x55d65d3215b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d3215b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1c20011d30 tx=0x7f1c2000f790 comp rx=0 tx=0).stop 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2717441202 shutdown_connections 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d326cf0 0x55d65d329180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321b80 0x55d65d326680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/2717441202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d3215b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2717441202 >> 172.21.15.105:0/2717441202 conn(0x55d65d31c700 msgr2=0x55d65d31cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2717441202 shutdown_connections 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2717441202 wait complete. 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- start start 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 Processor -- start 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d32b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d32bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d332c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d65d329be0 con 0x55d65d326cf0 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d65d2421a0 con 0x55d65d321190 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d65d329d60 con 0x55d65d321b80 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.647+0000 7f1c32fb8740 10 client.0 _ll_register_callbacks cb 0x55d65cf99c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d32b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d32b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54130/0 (socket says 172.21.15.105:54130) 2024-03-26T17:41:11.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 -- 172.21.15.105:0/1013736260 learned_addr learned my addr 172.21.15.105:0/1013736260 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d332c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d32bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 -- 172.21.15.105:0/1013736260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 msgr2=0x55d65d32bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d32bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 -- 172.21.15.105:0/1013736260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 msgr2=0x55d65d332c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d332c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 -- 172.21.15.105:0/1013736260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65d0627e0 con 0x55d65d321190 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d332c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d32bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d32b710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005ab0 tx=0x7f1c24004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=688c166667f6a09d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c11ffb640 1 -- 172.21.15.105:0/1013736260 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24005030 con 0x55d65d321190 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c11ffb640 1 -- 172.21.15.105:0/1013736260 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1c240051d0 con 0x55d65d321190 2024-03-26T17:41:11.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.648+0000 7f1c32fb8740 1 -- 172.21.15.105:0/1013736260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c20009d30 con 0x55d65d321190 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c11ffb640 1 -- 172.21.15.105:0/1013736260 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1c240053d0 con 0x55d65d321190 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c32fb8740 10 client.26288 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c32fb8740 1 -- 172.21.15.105:0/1013736260 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d65d0c2f70 con 0x55d65d321190 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c11ffb640 1 -- 172.21.15.105:0/1013736260 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c20009d30 con 0x55d65d321190 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c11ffb640 1 -- 172.21.15.105:0/1013736260 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1c24003710 con 0x55d65d321190 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c32fb8740 10 client.26288 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c32fb8740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c32fb8740 20 client.26288 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d65d337460 0x55d65d3398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c32fb8740 4 client.26288 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.649+0000 7f1c32fb8740 1 -- 172.21.15.105:0/1013736260 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65d293b70 con 0x55d65d337460 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.650+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d65d337460 0x55d65d3398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.650+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d65d337460 0x55d65d3398d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae7cee405e0aa051 server_cookie=c3adaf92f110110d in_seq=0 out_seq=0 2024-03-26T17:41:11.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.650+0000 7f1c11ffb640 10 client.26288 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c11ffb640 1 -- 172.21.15.105:0/1013736260 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d65d293b70 con 0x55d65d337460 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c11ffb640 10 client.26288 handle_command_reply: tid=0 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 client.26288 shutdown 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 -- 172.21.15.105:0/1013736260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 msgr2=0x55d65d32b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d32b710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005ab0 tx=0x7f1c24004270 comp rx=0 tx=0).stop 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 -- 172.21.15.105:0/1013736260 shutdown_connections 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d65d337460 0x55d65d3398d0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d332c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d32bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c11ffb640 0 client.26288 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/1013736260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d32b710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 -- 172.21.15.105:0/1013736260 >> 172.21.15.105:0/1013736260 conn(0x55d65d31c700 msgr2=0x55d65d31b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 -- 172.21.15.105:0/1013736260 shutdown_connections 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 1 -- 172.21.15.105:0/1013736260 wait complete. 2024-03-26T17:41:11.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.651+0000 7f1c32fb8740 20 client.26288 trim_cache size 0 max 16384 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.686+0000 7f1c32fb8740 1 Processor -- start 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.686+0000 7f1c32fb8740 1 -- start start 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321190 0x55d65d320070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d332be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d326cf0 0x55d65d32ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281a8d70 con 0x55d65d321190 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281aa8b0 con 0x55d65d326cf0 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281a9440 con 0x55d65d321b80 2024-03-26T17:41:11.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321190 0x55d65d320070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321190 0x55d65d320070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52382/0 (socket says 172.21.15.105:52382) 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 -- 172.21.15.105:0/2247190103 learned_addr learned my addr 172.21.15.105:0/2247190103 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d326cf0 0x55d65d32ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d332be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 -- 172.21.15.105:0/2247190103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 msgr2=0x55d65d332be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d332be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 -- 172.21.15.105:0/2247190103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d326cf0 msgr2=0x55d65d32ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d326cf0 0x55d65d32ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 -- 172.21.15.105:0/2247190103 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65cf5a170 con 0x55d65d321190 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321190 0x55d65d320070 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1c24000c00 tx=0x7f1c2402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e12a2d63addad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c13fff640 1 -- 172.21.15.105:0/2247190103 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24000e60 con 0x55d65d321190 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.687+0000 7f1c13fff640 1 -- 172.21.15.105:0/2247190103 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c2405b070 con 0x55d65d321190 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c13fff640 1 -- 172.21.15.105:0/2247190103 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24040970 con 0x55d65d321190 2024-03-26T17:41:11.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2247190103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321190 msgr2=0x55d65d320070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321190 0x55d65d320070 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1c24000c00 tx=0x7f1c2402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2247190103 shutdown_connections 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d326cf0 0x55d65d32ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d332be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/2247190103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d321190 0x55d65d320070 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2247190103 >> 172.21.15.105:0/2247190103 conn(0x55d65d31c700 msgr2=0x55d65d31f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2247190103 shutdown_connections 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.688+0000 7f1c32fb8740 1 -- 172.21.15.105:0/2247190103 wait complete. 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 1 -- start start 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 1 Processor -- start 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d330160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d3306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d3316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281aa1e0 con 0x55d65d326cf0 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281a9b10 con 0x55d65d321190 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281a6d50 con 0x55d65d321b80 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c32fb8740 10 client.0 _ll_register_callbacks cb 0x7f1c24009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e88e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d3316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d330160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e08d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d330160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54152/0 (socket says 172.21.15.105:54152) 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e08d640 1 -- 172.21.15.105:0/498925721 learned_addr learned my addr 172.21.15.105:0/498925721 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d3306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e08d640 1 -- 172.21.15.105:0/498925721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 msgr2=0x55d65d3306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d3306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e08d640 1 -- 172.21.15.105:0/498925721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 msgr2=0x55d65d3316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d3316b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e08d640 1 -- 172.21.15.105:0/498925721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65d331ca0 con 0x55d65d321190 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2e88e640 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d3316b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.689+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d3306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c2e08d640 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d330160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c24036040 tx=0x7f1c2402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeedc1613ab25041 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c11ffb640 1 -- 172.21.15.105:0/498925721 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c24065020 con 0x55d65d321190 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c11ffb640 1 -- 172.21.15.105:0/498925721 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1c24040b00 con 0x55d65d321190 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c32fb8740 1 -- 172.21.15.105:0/498925721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c2405b070 con 0x55d65d321190 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c11ffb640 1 -- 172.21.15.105:0/498925721 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1c24070050 con 0x55d65d321190 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c32fb8740 10 client.26294 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c32fb8740 1 -- 172.21.15.105:0/498925721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d65d39d7d0 con 0x55d65d321190 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c11ffb640 1 -- 172.21.15.105:0/498925721 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c2405b070 con 0x55d65d321190 2024-03-26T17:41:11.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.690+0000 7f1c11ffb640 1 -- 172.21.15.105:0/498925721 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1c240618f0 con 0x55d65d321190 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c32fb8740 10 client.26294 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c32fb8740 10 client.26294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c32fb8740 20 client.26294 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d65d39e060 0x55d65d3a0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c32fb8740 4 client.26294 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c32fb8740 1 -- 172.21.15.105:0/498925721 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d65d28dec0 con 0x55d65d39e060 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d65d39e060 0x55d65d3a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c11ffb640 10 client.26294 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:11.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.691+0000 7f1c2d88c640 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d65d39e060 0x55d65d3a0480 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5026b866fd543b server_cookie=f95b2fa4bdc1069c in_seq=0 out_seq=0 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c11ffb640 1 -- 172.21.15.105:0/498925721 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d65d28dec0 con 0x55d65d39e060 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c11ffb640 10 client.26294 handle_command_reply: tid=0 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 client.26294 shutdown 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 -- 172.21.15.105:0/498925721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 msgr2=0x55d65d330160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d330160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c24036040 tx=0x7f1c2402c5d0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 -- 172.21.15.105:0/498925721 shutdown_connections 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d65d39e060 0x55d65d3a0480 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d65d326cf0 0x55d65d3316b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d65d321b80 0x55d65d3306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c11ffb640 0 client.26294 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:11.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 --2- 172.21.15.105:0/498925721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d65d321190 0x55d65d330160 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 -- 172.21.15.105:0/498925721 >> 172.21.15.105:0/498925721 conn(0x55d65d31c700 msgr2=0x55d65d3201f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 -- 172.21.15.105:0/498925721 shutdown_connections 2024-03-26T17:41:11.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 1 -- 172.21.15.105:0/498925721 wait complete. 2024-03-26T17:41:11.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.694+0000 7f1c32fb8740 20 client.26294 trim_cache size 0 max 16384 2024-03-26T17:41:11.693 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:41:11.694 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:41:11.694 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "c4145073-8e8c-466b-be11-c3c459f17f3c", 2024-03-26T17:41:11.694 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:41:11.694 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 -- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c140a2fa0 msgr2=0x7f1c140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c140a2fa0 0x7f1c140a5460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1c2000fc90 tx=0x7f1c20018000 comp rx=0 tx=0).stop 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 -- 172.21.15.105:0/878072100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c28104cb0 msgr2=0x7f1c281980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c28104cb0 0x7f1c281980a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800e9c0 tx=0x7f1c1800ee80 comp rx=0 tx=0).stop 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 -- 172.21.15.105:0/878072100 shutdown_connections 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1c140a2fa0 0x7f1c140a5460 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c281075e0 0x7f1c2819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c28104cb0 0x7f1c281980a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 --2- 172.21.15.105:0/878072100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c28102380 0x7f1c28197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 -- 172.21.15.105:0/878072100 >> 172.21.15.105:0/878072100 conn(0x7f1c280fbd50 msgr2=0x7f1c28104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 -- 172.21.15.105:0/878072100 shutdown_connections 2024-03-26T17:41:11.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.695+0000 7f1c302f1640 1 -- 172.21.15.105:0/878072100 wait complete. 2024-03-26T17:41:11.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:41:11.699 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:41:11.800 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:11 smithi105 ceph-mon[32280]: pgmap v433: 97 pgs: 97 active+clean; 27 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-03-26T17:41:11.800 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:11 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:41:11.800 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:11 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:11.800 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:11 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:41:11.800 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:11 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:11.800 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:11 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:41:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 -- 172.21.15.105:0/3775363378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c102ae0 msgr2=0x7f5a8c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 --2- 172.21.15.105:0/3775363378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c102ae0 0x7f5a8c102f60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009900 tx=0x7f5a8002bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:11.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 -- 172.21.15.105:0/3775363378 shutdown_connections 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 --2- 172.21.15.105:0/3775363378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c1034a0 0x7f5a8c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 --2- 172.21.15.105:0/3775363378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c102ae0 0x7f5a8c102f60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 --2- 172.21.15.105:0/3775363378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 -- 172.21.15.105:0/3775363378 >> 172.21.15.105:0/3775363378 conn(0x7f5a8c0fce60 msgr2=0x7f5a8c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 -- 172.21.15.105:0/3775363378 shutdown_connections 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.834+0000 7f5a91c85640 1 -- 172.21.15.105:0/3775363378 wait complete. 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a91c85640 1 Processor -- start 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a91c85640 1 -- start start 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a91c85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a91c85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c102ae0 0x7f5a8c073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a91c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c1034a0 0x7f5a8c0741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a91c85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c10d420 con 0x7f5a8c1034a0 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a91c85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c10d2a0 con 0x7f5a8c1018f0 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a91c85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c10d5a0 con 0x7f5a8c102ae0 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a8bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c1034a0 0x7f5a8c0741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a8affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c102ae0 0x7f5a8c073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a8bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c1034a0 0x7f5a8c0741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52416/0 (socket says 172.21.15.105:52416) 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.835+0000 7f5a8bfff640 1 -- 172.21.15.105:0/2369505730 learned_addr learned my addr 172.21.15.105:0/2369505730 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a8affd640 1 -- 172.21.15.105:0/2369505730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a8c1018f0 msgr2=0x7f5a8c075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a8affd640 1 -- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c1034a0 msgr2=0x7f5a8c0741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c1034a0 0x7f5a8c0741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a8affd640 1 -- 172.21.15.105:0/2369505730 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8c074a80 con 0x7f5a8c102ae0 2024-03-26T17:41:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c102ae0 0x7f5a8c073cb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a80000c00 tx=0x7f5a80005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a88ff9640 1 -- 172.21.15.105:0/2369505730 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80043020 con 0x7f5a8c102ae0 2024-03-26T17:41:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a91c85640 1 -- 172.21.15.105:0/2369505730 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8c1a1c60 con 0x7f5a8c102ae0 2024-03-26T17:41:11.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.836+0000 7f5a88ff9640 1 -- 172.21.15.105:0/2369505730 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a80004350 con 0x7f5a8c102ae0 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.837+0000 7f5a91c85640 1 -- 172.21.15.105:0/2369505730 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8c1a21b0 con 0x7f5a8c102ae0 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.837+0000 7f5a88ff9640 1 -- 172.21.15.105:0/2369505730 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80003b50 con 0x7f5a8c102ae0 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a88ff9640 1 -- 172.21.15.105:0/2369505730 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5a80046020 con 0x7f5a8c102ae0 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a9487a740 1 Processor -- start 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a9487a740 1 -- start start 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a88ff9640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a740a2fa0 0x7f5a740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f138ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f138fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f13928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a9487a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5597f12ef980 con 0x5597f1390450 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a9487a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597f1297be0 con 0x5597f138a8f0 2024-03-26T17:41:11.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a8bfff640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a740a2fa0 0x7f5a740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.839+0000 7f5a88ff9640 1 -- 172.21.15.105:0/2369505730 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5a800f1420 con 0x7f5a8c102ae0 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a9487a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5597f12eb930 con 0x5597f138b2e0 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f138ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f138ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54172/0 (socket says 172.21.15.105:54172) 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/586907992 learned_addr learned my addr 172.21.15.105:0/586907992 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8affd640 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f138fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8bfff640 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f13928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/586907992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 msgr2=0x5597f138fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f138fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/586907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 msgr2=0x5597f13928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f13928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/586907992 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597f10cd7d0 con 0x5597f138a8f0 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8affd640 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f138fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8bfff640 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f13928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:11.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f138ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009ca0 tx=0x7f5a7c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd6f8f6e18a87512 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a8bfff640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a740a2fa0 0x7f5a740a5460 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5a840049d0 tx=0x7f5a84009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.840+0000 7f5a71ffb640 1 -- 172.21.15.105:0/586907992 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c004220 con 0x5597f138a8f0 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a71ffb640 1 -- 172.21.15.105:0/586907992 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a7c0043c0 con 0x5597f138a8f0 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a71ffb640 1 -- 172.21.15.105:0/586907992 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c0198b0 con 0x5597f138a8f0 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 -- 172.21.15.105:0/586907992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 msgr2=0x5597f138ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f138ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009ca0 tx=0x7f5a7c011810 comp rx=0 tx=0).stop 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 -- 172.21.15.105:0/586907992 shutdown_connections 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f13928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f138fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 --2- 172.21.15.105:0/586907992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f138ad10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 -- 172.21.15.105:0/586907992 >> 172.21.15.105:0/586907992 conn(0x5597f1385e60 msgr2=0x5597f1386260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 -- 172.21.15.105:0/586907992 shutdown_connections 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.841+0000 7f5a9487a740 1 -- 172.21.15.105:0/586907992 wait complete. 2024-03-26T17:41:11.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 1 -- start start 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 1 Processor -- start 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f138a8f0 0x5597f1394b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1395040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f1390450 0x5597f139c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5597f1393400 con 0x5597f138a8f0 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597f12ab160 con 0x5597f1390450 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5597f1393580 con 0x5597f138b2e0 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f138a8f0 0x5597f1394b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a9487a740 10 client.0 _ll_register_callbacks cb 0x5597f12e7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f138a8f0 0x5597f1394b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52444/0 (socket says 172.21.15.105:52444) 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/2324310556 learned_addr learned my addr 172.21.15.105:0/2324310556 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1395040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.842+0000 7f5a8bfff640 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f1390450 0x5597f139c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a8affd640 1 -- 172.21.15.105:0/2324310556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f1390450 msgr2=0x5597f139c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f1390450 0x5597f139c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a8affd640 1 -- 172.21.15.105:0/2324310556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f138a8f0 msgr2=0x5597f1394b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f138a8f0 0x5597f1394b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a8affd640 1 -- 172.21.15.105:0/2324310556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597f10033f0 con 0x5597f138b2e0 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1395040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005ab0 tx=0x7f5a800bd530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cc304d09bd6b692 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a6affd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a6affd640 1 -- 172.21.15.105:0/2324310556 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800c1c60 con 0x5597f138b2e0 2024-03-26T17:41:11.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a6affd640 1 -- 172.21.15.105:0/2324310556 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5a800bdd70 con 0x5597f138b2e0 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a9487a740 1 -- 172.21.15.105:0/2324310556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a7c0043c0 con 0x5597f138b2e0 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a6affd640 1 -- 172.21.15.105:0/2324310556 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a800c6520 con 0x5597f138b2e0 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a9487a740 10 client.26299 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.843+0000 7f5a9487a740 1 -- 172.21.15.105:0/2324310556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597f0fc30f0 con 0x5597f138b2e0 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a6affd640 1 -- 172.21.15.105:0/2324310556 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a7c0043c0 con 0x5597f138b2e0 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a6affd640 1 -- 172.21.15.105:0/2324310556 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5a800b9860 con 0x5597f138b2e0 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a9487a740 10 client.26299 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a9487a740 10 client.26299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a9487a740 20 client.26299 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597f13a0860 0x5597f13a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a9487a740 4 client.26299 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:11.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a9487a740 1 -- 172.21.15.105:0/2324310556 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597f12fcec0 con 0x5597f13a0860 2024-03-26T17:41:11.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597f13a0860 0x5597f13a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a6affd640 10 client.26299 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:11.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.844+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597f13a0860 0x5597f13a2cf0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1660135b21ad81e8 server_cookie=219daa5e65c1fef3 in_seq=0 out_seq=0 2024-03-26T17:41:11.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.845+0000 7f5a6affd640 1 -- 172.21.15.105:0/2324310556 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5597f12fcec0 con 0x5597f13a0860 2024-03-26T17:41:11.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.845+0000 7f5a6affd640 10 client.26299 handle_command_reply: tid=0 2024-03-26T17:41:11.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.845+0000 7f5a9487a740 1 client.26299 shutdown 2024-03-26T17:41:11.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 -- 172.21.15.105:0/2324310556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 msgr2=0x5597f1395040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1395040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005ab0 tx=0x7f5a800bd530 comp rx=0 tx=0).stop 2024-03-26T17:41:11.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 -- 172.21.15.105:0/2324310556 shutdown_connections 2024-03-26T17:41:11.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597f13a0860 0x5597f13a2cf0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f1390450 0x5597f139c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1395040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a6affd640 0 client.26299 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:11.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2324310556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f138a8f0 0x5597f1394b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 -- 172.21.15.105:0/2324310556 >> 172.21.15.105:0/2324310556 conn(0x5597f1385e60 msgr2=0x5597f1385080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 -- 172.21.15.105:0/2324310556 shutdown_connections 2024-03-26T17:41:11.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 1 -- 172.21.15.105:0/2324310556 wait complete. 2024-03-26T17:41:11.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.846+0000 7f5a9487a740 20 client.26299 trim_cache size 0 max 16384 2024-03-26T17:41:11.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a9487a740 1 Processor -- start 2024-03-26T17:41:11.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a9487a740 1 -- start start 2024-03-26T17:41:11.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138a8f0 0x5597f138c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138b2e0 0x5597f138ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a9487a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1aafa0 con 0x5597f1390450 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a9487a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1acae0 con 0x5597f138b2e0 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a9487a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1ab670 con 0x5597f138a8f0 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138b2e0 0x5597f138ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138a8f0 0x5597f138c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138a8f0 0x5597f138c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45518/0 (socket says 172.21.15.105:45518) 2024-03-26T17:41:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138b2e0 0x5597f138ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54190/0 (socket says 172.21.15.105:54190) 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/4228872720 learned_addr learned my addr 172.21.15.105:0/4228872720 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8bfff640 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52446/0 (socket says 172.21.15.105:52446) 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/4228872720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138b2e0 msgr2=0x5597f138ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138b2e0 0x5597f138ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/4228872720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 msgr2=0x5597f139c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8b7fe640 1 -- 172.21.15.105:0/4228872720 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597f138b6c0 con 0x5597f138a8f0 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.866+0000 7f5a8bfff640 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.867+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138a8f0 0x5597f138c500 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009710 tx=0x7f5a7c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10b82e51152ce95 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.867+0000 7f5a717fa640 1 -- 172.21.15.105:0/4228872720 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c003e10 con 0x5597f138a8f0 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.867+0000 7f5a717fa640 1 -- 172.21.15.105:0/4228872720 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a7c02ba30 con 0x5597f138a8f0 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.867+0000 7f5a717fa640 1 -- 172.21.15.105:0/4228872720 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c02b550 con 0x5597f138a8f0 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- 172.21.15.105:0/4228872720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138a8f0 msgr2=0x5597f138c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138a8f0 0x5597f138c500 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009710 tx=0x7f5a7c011810 comp rx=0 tx=0).stop 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- 172.21.15.105:0/4228872720 shutdown_connections 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138b2e0 0x5597f138ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 --2- 172.21.15.105:0/4228872720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138a8f0 0x5597f138c500 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- 172.21.15.105:0/4228872720 >> 172.21.15.105:0/4228872720 conn(0x5597f1385e60 msgr2=0x5597f1386240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- 172.21.15.105:0/4228872720 shutdown_connections 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- 172.21.15.105:0/4228872720 wait complete. 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- start start 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 Processor -- start 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f1399830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1399d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1ac410 con 0x5597f1390450 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1abd40 con 0x5597f138a8f0 2024-03-26T17:41:11.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c1a8f80 con 0x5597f138b2e0 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.868+0000 7f5a9487a740 10 client.0 _ll_register_callbacks cb 0x7f5a80009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f1399830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1399d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1399d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45522/0 (socket says 172.21.15.105:45522) 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f1399830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54196/0 (socket says 172.21.15.105:54196) 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 -- 172.21.15.105:0/2255121271 learned_addr learned my addr 172.21.15.105:0/2255121271 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 -- 172.21.15.105:0/2255121271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 msgr2=0x5597f1399830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f1399830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 -- 172.21.15.105:0/2255121271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 msgr2=0x5597f139acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 -- 172.21.15.105:0/2255121271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597f139b2b0 con 0x5597f138b2e0 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8bfff640 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a8affd640 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1399d70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5a800ba4e0 tx=0x7f5a80005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc52e5641b5c2f0a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a6affd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a6affd640 1 -- 172.21.15.105:0/2255121271 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800c1c60 con 0x5597f138b2e0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a6affd640 1 -- 172.21.15.105:0/2255121271 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5a800c6090 con 0x5597f138b2e0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a9487a740 1 -- 172.21.15.105:0/2255121271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a7c02ba30 con 0x5597f138b2e0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a6affd640 1 -- 172.21.15.105:0/2255121271 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a800bd650 con 0x5597f138b2e0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a9487a740 10 client.26305 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.869+0000 7f5a9487a740 1 -- 172.21.15.105:0/2255121271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597f1406c60 con 0x5597f138b2e0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a6affd640 1 -- 172.21.15.105:0/2255121271 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a7c02ba30 con 0x5597f138b2e0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a6affd640 1 -- 172.21.15.105:0/2255121271 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5a800a3020 con 0x5597f138b2e0 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a9487a740 10 client.26305 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a9487a740 10 client.26305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a9487a740 20 client.26305 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597f14073e0 0x5597f1409800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a9487a740 4 client.26305 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a9487a740 1 -- 172.21.15.105:0/2255121271 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597f12f7210 con 0x5597f14073e0 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597f14073e0 0x5597f1409800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a6affd640 10 client.26305 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.870+0000 7f5a8b7fe640 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597f14073e0 0x5597f1409800 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af899d14292b6507 server_cookie=f0f23e461be79b85 in_seq=0 out_seq=0 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a6affd640 1 -- 172.21.15.105:0/2255121271 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5597f12f7210 con 0x5597f14073e0 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a6affd640 10 client.26305 handle_command_reply: tid=0 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 client.26305 shutdown 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 -- 172.21.15.105:0/2255121271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 msgr2=0x5597f1399d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1399d70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5a800ba4e0 tx=0x7f5a80005fd0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 -- 172.21.15.105:0/2255121271 shutdown_connections 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597f14073e0 0x5597f1409800 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597f1390450 0x5597f139acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597f138b2e0 0x5597f1399d70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a6affd640 0 client.26305 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:11.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 --2- 172.21.15.105:0/2255121271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597f138a8f0 0x5597f1399830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 -- 172.21.15.105:0/2255121271 >> 172.21.15.105:0/2255121271 conn(0x5597f1385e60 msgr2=0x5597f1394ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 -- 172.21.15.105:0/2255121271 shutdown_connections 2024-03-26T17:41:11.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 1 -- 172.21.15.105:0/2255121271 wait complete. 2024-03-26T17:41:11.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.871+0000 7f5a9487a740 20 client.26305 trim_cache size 0 max 16384 2024-03-26T17:41:11.872 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:41:11.872 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:41:11.872 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:41:11.872 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:41:11.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 -- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a740a2fa0 msgr2=0x7f5a740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a740a2fa0 0x7f5a740a5460 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5a840049d0 tx=0x7f5a84009f90 comp rx=0 tx=0).stop 2024-03-26T17:41:11.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 -- 172.21.15.105:0/2369505730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c102ae0 msgr2=0x7f5a8c073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c102ae0 0x7f5a8c073cb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a80000c00 tx=0x7f5a80005dc0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 -- 172.21.15.105:0/2369505730 shutdown_connections 2024-03-26T17:41:11.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a740a2fa0 0x7f5a740a5460 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a8c1034a0 0x7f5a8c0741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a8c102ae0 0x7f5a8c073cb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 --2- 172.21.15.105:0/2369505730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a8c1018f0 0x7f5a8c075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 -- 172.21.15.105:0/2369505730 >> 172.21.15.105:0/2369505730 conn(0x7f5a8c0fce60 msgr2=0x7f5a8c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 -- 172.21.15.105:0/2369505730 shutdown_connections 2024-03-26T17:41:11.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:11.872+0000 7f5a91c85640 1 -- 172.21.15.105:0/2369505730 wait complete. 2024-03-26T17:41:11.876 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:41:11.877 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:41:11.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.000+0000 7f80f89f4640 1 -- 172.21.15.105:0/1901149582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4102360 msgr2=0x7f80f4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.000+0000 7f80f89f4640 1 --2- 172.21.15.105:0/1901149582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4102360 0x7f80f4104750 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009900 tx=0x7f80e402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:11.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.000+0000 7f80f89f4640 1 -- 172.21.15.105:0/1901149582 shutdown_connections 2024-03-26T17:41:11.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.000+0000 7f80f89f4640 1 --2- 172.21.15.105:0/1901149582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f41075c0 0x7f80f4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.000+0000 7f80f89f4640 1 --2- 172.21.15.105:0/1901149582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4104c90 0x7f80f4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.000+0000 7f80f89f4640 1 --2- 172.21.15.105:0/1901149582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4102360 0x7f80f4104750 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.000+0000 7f80f89f4640 1 -- 172.21.15.105:0/1901149582 >> 172.21.15.105:0/1901149582 conn(0x7f80f40fbd50 msgr2=0x7f80f40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.001+0000 7f80f89f4640 1 -- 172.21.15.105:0/1901149582 shutdown_connections 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.001+0000 7f80f89f4640 1 -- 172.21.15.105:0/1901149582 wait complete. 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.001+0000 7f80f89f4640 1 Processor -- start 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.001+0000 7f80f89f4640 1 -- start start 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.001+0000 7f80f89f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102360 0x7f80f4197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.001+0000 7f80f89f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4104c90 0x7f80f41980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f89f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f41075c0 0x7f80f419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f89f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80f410d440 con 0x7f80f4104c90 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f89f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80f410d2c0 con 0x7f80f41075c0 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f89f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80f410d5c0 con 0x7f80f4102360 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4104c90 0x7f80f41980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4104c90 0x7f80f41980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52458/0 (socket says 172.21.15.105:52458) 2024-03-26T17:41:11.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 -- 172.21.15.105:0/3586375285 learned_addr learned my addr 172.21.15.105:0/3586375285 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f2d9d640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f41075c0 0x7f80f419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f259c640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102360 0x7f80f4197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 -- 172.21.15.105:0/3586375285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102360 msgr2=0x7f80f4197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102360 0x7f80f4197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 -- 172.21.15.105:0/3586375285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f41075c0 msgr2=0x7f80f419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f41075c0 0x7f80f419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 -- 172.21.15.105:0/3586375285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f419f790 con 0x7f80f4104c90 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f1d9b640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4104c90 0x7f80f41980b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f80e800e9f0 tx=0x7f80e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80e37fe640 1 -- 172.21.15.105:0/3586375285 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e800ccc0 con 0x7f80f4104c90 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80e37fe640 1 -- 172.21.15.105:0/3586375285 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80e80054d0 con 0x7f80f4104c90 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f419fa80 con 0x7f80f4104c90 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80e37fe640 1 -- 172.21.15.105:0/3586375285 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8010430 con 0x7f80f4104c90 2024-03-26T17:41:11.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.002+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f419ff80 con 0x7f80f4104c90 2024-03-26T17:41:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.004+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f40fd840 con 0x7f80f4104c90 2024-03-26T17:41:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.004+0000 7f80e37fe640 1 -- 172.21.15.105:0/3586375285 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f80e8005020 con 0x7f80f4104c90 2024-03-26T17:41:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.005+0000 7f80e37fe640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.005+0000 7f80f259c640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.005+0000 7f80e37fe640 1 -- 172.21.15.105:0/3586375285 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f80e80cf7a0 con 0x7f80f4104c90 2024-03-26T17:41:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.006+0000 7f80f259c640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009230 tx=0x7f80e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.010+0000 7f80e37fe640 1 -- 172.21.15.105:0/3586375285 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f80e809c590 con 0x7f80f4104c90 2024-03-26T17:41:12.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 68285 3.35 MB/sec warmup 115 sec latency 27.188 ms 2024-03-26T17:41:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:11 smithi144 ceph-mon[36095]: pgmap v433: 97 pgs: 97 active+clean; 27 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-03-26T17:41:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:11 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:41:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:11 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:11 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:41:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:11 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:12.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:11 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:41:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:11 smithi135 ceph-mon[35636]: pgmap v433: 97 pgs: 97 active+clean; 27 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-03-26T17:41:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:11 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:41:12.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:11 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:12.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:11 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:41:12.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:11 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:12.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:11 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:41:12.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.265+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80f402d240 con 0x7f80f4104c90 2024-03-26T17:41:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.266+0000 7f80e37fe640 1 -- 172.21.15.105:0/3586375285 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f80e809be90 con 0x7f80f4104c90 2024-03-26T17:41:12.259 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:41:12.259 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:41:12.259 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:41:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 msgr2=0x7f80dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009230 tx=0x7f80e4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4104c90 msgr2=0x7f80f41980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4104c90 0x7f80f41980b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f80e800e9f0 tx=0x7f80e800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 shutdown_connections 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f41075c0 0x7f80f419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4104c90 0x7f80f41980b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 --2- 172.21.15.105:0/3586375285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102360 0x7f80f4197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 >> 172.21.15.105:0/3586375285 conn(0x7f80f40fbd50 msgr2=0x7f80f4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 shutdown_connections 2024-03-26T17:41:12.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.268+0000 7f80f89f4640 1 -- 172.21.15.105:0/3586375285 wait complete. 2024-03-26T17:41:12.274 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:41:12.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 -- 172.21.15.105:0/4287794081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 msgr2=0x7f36e4107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/4287794081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 0x7f36e4107100 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009900 tx=0x7f36d402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 -- 172.21.15.105:0/4287794081 shutdown_connections 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/4287794081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e4107640 0x7f36e4109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/4287794081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 0x7f36e4107100 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/4287794081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36e41023e0 0x7f36e41047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 -- 172.21.15.105:0/4287794081 >> 172.21.15.105:0/4287794081 conn(0x7f36e40fbd70 msgr2=0x7f36e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 -- 172.21.15.105:0/4287794081 shutdown_connections 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.400+0000 7f36eb5b9640 1 -- 172.21.15.105:0/4287794081 wait complete. 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36eb5b9640 1 Processor -- start 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36eb5b9640 1 -- start start 2024-03-26T17:41:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36eb5b9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e41023e0 0x7f36e4101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36eb5b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 0x7f36e4100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36eb5b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36e4107640 0x7f36e41008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36eb5b9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410d4a0 con 0x7f36e4104d10 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36eb5b9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410d320 con 0x7f36e41023e0 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36eb5b9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410d620 con 0x7f36e4107640 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 0x7f36e4100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 0x7f36e4100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52476/0 (socket says 172.21.15.105:52476) 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e8b54640 1 -- 172.21.15.105:0/1293526354 learned_addr learned my addr 172.21.15.105:0/1293526354 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e9355640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e41023e0 0x7f36e4101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e9b56640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36e4107640 0x7f36e41008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e8b54640 1 -- 172.21.15.105:0/1293526354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36e4107640 msgr2=0x7f36e41008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36e4107640 0x7f36e41008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e8b54640 1 -- 172.21.15.105:0/1293526354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e41023e0 msgr2=0x7f36e4101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e41023e0 0x7f36e4101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.401+0000 7f36e8b54640 1 -- 172.21.15.105:0/1293526354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e4101160 con 0x7f36e4104d10 2024-03-26T17:41:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.402+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 0x7f36e4100390 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f36d4000c00 tx=0x7f36d4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.402+0000 7f36da7fc640 1 -- 172.21.15.105:0/1293526354 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4043020 con 0x7f36e4104d10 2024-03-26T17:41:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.402+0000 7f36eb5b9640 1 -- 172.21.15.105:0/1293526354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e41a1d60 con 0x7f36e4104d10 2024-03-26T17:41:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.402+0000 7f36da7fc640 1 -- 172.21.15.105:0/1293526354 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d4004350 con 0x7f36e4104d10 2024-03-26T17:41:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.402+0000 7f36da7fc640 1 -- 172.21.15.105:0/1293526354 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4003b80 con 0x7f36e4104d10 2024-03-26T17:41:12.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.402+0000 7f36eb5b9640 1 -- 172.21.15.105:0/1293526354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e41a22a0 con 0x7f36e4104d10 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.404+0000 7f36da7fc640 1 -- 172.21.15.105:0/1293526354 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f36d4046020 con 0x7f36e4104d10 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.404+0000 7f36da7fc640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.405+0000 7f36da7fc640 1 -- 172.21.15.105:0/1293526354 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f36d40f1580 con 0x7f36e4104d10 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.405+0000 7f36e9355640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.405+0000 7f36e9355640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36d00a2f20 0x7f36d00a53e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009bd0 tx=0x7f36e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.405+0000 7f36ee1a6740 1 Processor -- start 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.405+0000 7f36ee1a6740 1 -- start start 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfc9a30 0x55596bfc9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfcef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfcf590 0x55596bfd1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55596bf2e970 con 0x55596bfca420 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55596bed6d70 con 0x55596bfcf590 2024-03-26T17:41:12.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55596bf2a920 con 0x55596bfc9a30 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfcef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e9355640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfc9a30 0x55596bfc9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e9355640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfc9a30 0x55596bfc9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45576/0 (socket says 172.21.15.105:45576) 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e9b56640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfcf590 0x55596bfd1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfcef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52488/0 (socket says 172.21.15.105:52488) 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e9355640 1 -- 172.21.15.105:0/1659195086 learned_addr learned my addr 172.21.15.105:0/1659195086 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e8b54640 1 -- 172.21.15.105:0/1659195086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfc9a30 msgr2=0x55596bfc9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1659195086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfc9a30 0x55596bfc9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e8b54640 1 -- 172.21.15.105:0/1659195086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfcf590 msgr2=0x55596bfd1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1659195086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfcf590 0x55596bfd1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e8b54640 1 -- 172.21.15.105:0/1659195086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596bc423f0 con 0x55596bfca420 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1659195086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfcef20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f36d40ef090 tx=0x7f36d40bd7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537ad2bc7b58bc37 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.406+0000 7f36cf7fe640 1 -- 172.21.15.105:0/1659195086 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40bdc70 con 0x55596bfca420 2024-03-26T17:41:12.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36cf7fe640 1 -- 172.21.15.105:0/1659195086 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d40c1dc0 con 0x55596bfca420 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36cf7fe640 1 -- 172.21.15.105:0/1659195086 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40ba2e0 con 0x55596bfca420 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1659195086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 msgr2=0x55596bfcef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/1659195086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfcef20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f36d40ef090 tx=0x7f36d40bd7b0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1659195086 shutdown_connections 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/1659195086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfcf590 0x55596bfd1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/1659195086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfcef20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/1659195086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfc9a30 0x55596bfc9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1659195086 >> 172.21.15.105:0/1659195086 conn(0x55596bfc4fa0 msgr2=0x55596bfc53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.407+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1659195086 shutdown_connections 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1659195086 wait complete. 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 -- start start 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 Processor -- start 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfc9a30 0x55596bfd3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfca420 0x55596bfd4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfdb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55596bfd2300 con 0x55596bfc9a30 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55596beea310 con 0x55596bfca420 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55596bfd2480 con 0x55596bfcf590 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36ee1a6740 10 client.0 _ll_register_callbacks cb 0x55596bf25c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36e9355640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfc9a30 0x55596bfd3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36e9b56640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfdb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfca420 0x55596bfd4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36e9355640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfc9a30 0x55596bfd3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52498/0 (socket says 172.21.15.105:52498) 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.408+0000 7f36e9355640 1 -- 172.21.15.105:0/4215448892 learned_addr learned my addr 172.21.15.105:0/4215448892 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36e9b56640 1 -- 172.21.15.105:0/4215448892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfca420 msgr2=0x55596bfd4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36e9b56640 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfca420 0x55596bfd4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36e9b56640 1 -- 172.21.15.105:0/4215448892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfc9a30 msgr2=0x55596bfd3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36e9b56640 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfc9a30 0x55596bfd3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36e9b56640 1 -- 172.21.15.105:0/4215448892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596bd0c7d0 con 0x55596bfcf590 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36e8b54640 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfca420 0x55596bfd4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36e9b56640 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfdb1c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00aaa0 tx=0x7f36dc00aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a4626135ddc284 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36ccff9640 1 -- 172.21.15.105:0/4215448892 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc01a070 con 0x55596bfcf590 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36ccff9640 1 -- 172.21.15.105:0/4215448892 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f36dc011c90 con 0x55596bfcf590 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36ee1a6740 1 -- 172.21.15.105:0/4215448892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d40c1dc0 con 0x55596bfcf590 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36ccff9640 1 -- 172.21.15.105:0/4215448892 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36dc026050 con 0x55596bfcf590 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36ee1a6740 10 client.26323 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.409+0000 7f36ee1a6740 1 -- 172.21.15.105:0/4215448892 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55596bfde600 con 0x55596bfcf590 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36ccff9640 1 -- 172.21.15.105:0/4215448892 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d40c1dc0 con 0x55596bfcf590 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36ccff9640 1 -- 172.21.15.105:0/4215448892 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f36dc01e440 con 0x55596bfcf590 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36ee1a6740 10 client.26323 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36ee1a6740 10 client.26323 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:41:12.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36ee1a6740 20 client.26323 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55596bfdfce0 0x55596bfe2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36ee1a6740 4 client.26323 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36ee1a6740 1 -- 172.21.15.105:0/4215448892 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55596bf3beb0 con 0x55596bfdfce0 2024-03-26T17:41:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.410+0000 7f36e9355640 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55596bfdfce0 0x55596bfe2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ccff9640 10 client.26323 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:12.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36e9355640 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55596bfdfce0 0x55596bfe2140 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7bb715aff3071c05 server_cookie=de954ee0be8964d8 in_seq=0 out_seq=0 2024-03-26T17:41:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ccff9640 1 -- 172.21.15.105:0/4215448892 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55596bf3beb0 con 0x55596bfdfce0 2024-03-26T17:41:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ccff9640 10 client.26323 handle_command_reply: tid=0 2024-03-26T17:41:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 client.26323 shutdown 2024-03-26T17:41:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 -- 172.21.15.105:0/4215448892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 msgr2=0x55596bfdb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfdb1c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00aaa0 tx=0x7f36dc00aea0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 -- 172.21.15.105:0/4215448892 shutdown_connections 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55596bfdfce0 0x55596bfe2140 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfdb1c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ccff9640 0 client.26323 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfca420 0x55596bfd4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/4215448892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfc9a30 0x55596bfd3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.411+0000 7f36ee1a6740 1 -- 172.21.15.105:0/4215448892 >> 172.21.15.105:0/4215448892 conn(0x55596bfc4fa0 msgr2=0x55596bfc41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.412+0000 7f36ee1a6740 1 -- 172.21.15.105:0/4215448892 shutdown_connections 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.412+0000 7f36ee1a6740 1 -- 172.21.15.105:0/4215448892 wait complete. 2024-03-26T17:41:12.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.412+0000 7f36ee1a6740 20 client.26323 trim_cache size 0 max 16384 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36ee1a6740 1 Processor -- start 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36ee1a6740 1 -- start start 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfc8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfdb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfd2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41ab110 con 0x55596bfca420 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41acc50 con 0x55596bfc9a30 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41ab7e0 con 0x55596bfcf590 2024-03-26T17:41:12.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e9355640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfc8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfdb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfdb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52508/0 (socket says 172.21.15.105:52508) 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e9b56640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfd2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e9355640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfc8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54260/0 (socket says 172.21.15.105:54260) 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 -- 172.21.15.105:0/1671665697 learned_addr learned my addr 172.21.15.105:0/1671665697 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 -- 172.21.15.105:0/1671665697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 msgr2=0x55596bfd2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1671665697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfd2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 -- 172.21.15.105:0/1671665697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 msgr2=0x55596bfc8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1671665697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfc8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 -- 172.21.15.105:0/1671665697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596bfc5d10 con 0x55596bfca420 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.426+0000 7f36e8b54640 1 --2- 172.21.15.105:0/1671665697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfdb150 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f36d40bd970 tx=0x7f36d40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85ec0eda820070c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36e9355640 1 --2- 172.21.15.105:0/1671665697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfc8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ceffd640 1 -- 172.21.15.105:0/1671665697 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40bdc70 con 0x55596bfca420 2024-03-26T17:41:12.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ceffd640 1 -- 172.21.15.105:0/1671665697 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d40c6680 con 0x55596bfca420 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ceffd640 1 -- 172.21.15.105:0/1671665697 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40c61f0 con 0x55596bfca420 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1671665697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 msgr2=0x55596bfdb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/1671665697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfdb150 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f36d40bd970 tx=0x7f36d40035f0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1671665697 shutdown_connections 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/1671665697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfd2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/1671665697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfdb150 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/1671665697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfc8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1671665697 >> 172.21.15.105:0/1671665697 conn(0x55596bfc4fa0 msgr2=0x55596bfc8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.427+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1671665697 shutdown_connections 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 -- 172.21.15.105:0/1671665697 wait complete. 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 -- start start 2024-03-26T17:41:12.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 Processor -- start 2024-03-26T17:41:12.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfd8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfd8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfd9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41ac580 con 0x55596bfca420 2024-03-26T17:41:12.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41abeb0 con 0x55596bfc9a30 2024-03-26T17:41:12.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41a90f0 con 0x55596bfcf590 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36ee1a6740 10 client.0 _ll_register_callbacks cb 0x7f36dc009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfd8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36e8b54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfd8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52514/0 (socket says 172.21.15.105:52514) 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36e8b54640 1 -- 172.21.15.105:0/584998696 learned_addr learned my addr 172.21.15.105:0/584998696 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36e9355640 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfd8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.428+0000 7f36e9b56640 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfd9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36e8b54640 1 -- 172.21.15.105:0/584998696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 msgr2=0x55596bfd9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36e8b54640 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfd9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36e8b54640 1 -- 172.21.15.105:0/584998696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 msgr2=0x55596bfd8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36e8b54640 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfd8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36e8b54640 1 -- 172.21.15.105:0/584998696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596bfd9ef0 con 0x55596bfca420 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36e9355640 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfd8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36e8b54640 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfd8940 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f36d4004580 tx=0x7f36d40ef090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedb2213edcb7a74 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36ccff9640 1 -- 172.21.15.105:0/584998696 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40a0070 con 0x55596bfca420 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36ccff9640 1 -- 172.21.15.105:0/584998696 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f36d40f10f0 con 0x55596bfca420 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36ee1a6740 1 -- 172.21.15.105:0/584998696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d40c6680 con 0x55596bfca420 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36ccff9640 1 -- 172.21.15.105:0/584998696 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36d40bdc70 con 0x55596bfca420 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36ee1a6740 10 client.16482 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36ee1a6740 1 -- 172.21.15.105:0/584998696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55596bfdb1f0 con 0x55596bfca420 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.429+0000 7f36ccff9640 1 -- 172.21.15.105:0/584998696 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d40c6680 con 0x55596bfca420 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36ccff9640 1 -- 172.21.15.105:0/584998696 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f36d40b99c0 con 0x55596bfca420 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36ee1a6740 10 client.16482 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36ee1a6740 10 client.16482 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36ee1a6740 20 client.16482 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55596c046250 0x55596c048640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36ee1a6740 4 client.16482 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36ee1a6740 1 -- 172.21.15.105:0/584998696 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55596bf36200 con 0x55596c046250 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36e9355640 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55596c046250 0x55596c048640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36ccff9640 10 client.16482 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.430+0000 7f36e9355640 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55596c046250 0x55596c048640 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a1600108464a1708 server_cookie=4c2bca29ed1d42df in_seq=0 out_seq=0 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ccff9640 1 -- 172.21.15.105:0/584998696 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55596bf36200 con 0x55596c046250 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ccff9640 10 client.16482 handle_command_reply: tid=0 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 client.16482 shutdown 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 -- 172.21.15.105:0/584998696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 msgr2=0x55596bfd8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfd8940 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f36d4004580 tx=0x7f36d40ef090 comp rx=0 tx=0).stop 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 -- 172.21.15.105:0/584998696 shutdown_connections 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55596c046250 0x55596c048640 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55596bfcf590 0x55596bfd9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55596bfca420 0x55596bfd8940 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 --2- 172.21.15.105:0/584998696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55596bfc9a30 0x55596bfd8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ccff9640 0 client.16482 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:12.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 -- 172.21.15.105:0/584998696 >> 172.21.15.105:0/584998696 conn(0x55596bfc4fa0 msgr2=0x55596bfc8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 -- 172.21.15.105:0/584998696 shutdown_connections 2024-03-26T17:41:12.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 1 -- 172.21.15.105:0/584998696 wait complete. 2024-03-26T17:41:12.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.431+0000 7f36ee1a6740 20 client.16482 trim_cache size 0 max 16384 2024-03-26T17:41:12.431 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 -- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36d00a2f20 msgr2=0x7f36d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36d00a2f20 0x7f36d00a53e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f36e0009bd0 tx=0x7f36e0005f90 comp rx=0 tx=0).stop 2024-03-26T17:41:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 -- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 msgr2=0x7f36e4100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 0x7f36e4100390 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f36d4000c00 tx=0x7f36d4005e00 comp rx=0 tx=0).stop 2024-03-26T17:41:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 -- 172.21.15.105:0/1293526354 shutdown_connections 2024-03-26T17:41:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36d00a2f20 0x7f36d00a53e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36e4107640 0x7f36e41008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f36e4104d10 0x7f36e4100390 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 --2- 172.21.15.105:0/1293526354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f36e41023e0 0x7f36e4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 -- 172.21.15.105:0/1293526354 >> 172.21.15.105:0/1293526354 conn(0x7f36e40fbd70 msgr2=0x7f36e4102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 -- 172.21.15.105:0/1293526354 shutdown_connections 2024-03-26T17:41:12.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.432+0000 7f36eb5b9640 1 -- 172.21.15.105:0/1293526354 wait complete. 2024-03-26T17:41:12.437 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:41:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 -- 172.21.15.105:0/3365039113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6601006d0 msgr2=0x7fb660100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 --2- 172.21.15.105:0/3365039113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6601006d0 0x7fb660100af0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb650009900 tx=0x7fb65002bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 -- 172.21.15.105:0/3365039113 shutdown_connections 2024-03-26T17:41:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 --2- 172.21.15.105:0/3365039113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb660102280 0x7fb660108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 --2- 172.21.15.105:0/3365039113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6601018c0 0x7fb660101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 --2- 172.21.15.105:0/3365039113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6601006d0 0x7fb660100af0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 -- 172.21.15.105:0/3365039113 >> 172.21.15.105:0/3365039113 conn(0x7fb6600fbc60 msgr2=0x7fb6600fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 -- 172.21.15.105:0/3365039113 shutdown_connections 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.562+0000 7fb6657ff640 1 -- 172.21.15.105:0/3365039113 wait complete. 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb6657ff640 1 Processor -- start 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb6657ff640 1 -- start start 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb6657ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6601006d0 0x7fb660197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb6657ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6601018c0 0x7fb660197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb6657ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb660102280 0x7fb66019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb6657ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010d320 con 0x7fb660102280 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb6657ff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010d1a0 con 0x7fb6601006d0 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb6657ff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010d4a0 con 0x7fb6601018c0 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb65f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb660102280 0x7fb66019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb65f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb660102280 0x7fb66019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52520/0 (socket says 172.21.15.105:52520) 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb65f7fe640 1 -- 172.21.15.105:0/1955150416 learned_addr learned my addr 172.21.15.105:0/1955150416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb65effd640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6601006d0 0x7fb660197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.563+0000 7fb657fff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6601018c0 0x7fb660197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65f7fe640 1 -- 172.21.15.105:0/1955150416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6601018c0 msgr2=0x7fb660197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6601018c0 0x7fb660197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65f7fe640 1 -- 172.21.15.105:0/1955150416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6601006d0 msgr2=0x7fb660197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6601006d0 0x7fb660197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65f7fe640 1 -- 172.21.15.105:0/1955150416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66019f640 con 0x7fb660102280 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65effd640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6601006d0 0x7fb660197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb657fff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6601018c0 0x7fb660197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb660102280 0x7fb66019efd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb65800eb10 tx=0x7fb65800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65cff9640 1 -- 172.21.15.105:0/1955150416 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658004b20 con 0x7fb660102280 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb6657ff640 1 -- 172.21.15.105:0/1955150416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66019f930 con 0x7fb660102280 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65cff9640 1 -- 172.21.15.105:0/1955150416 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb658004cc0 con 0x7fb660102280 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb65cff9640 1 -- 172.21.15.105:0/1955150416 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658010690 con 0x7fb660102280 2024-03-26T17:41:12.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.564+0000 7fb6657ff640 1 -- 172.21.15.105:0/1955150416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66019fe80 con 0x7fb660102280 2024-03-26T17:41:12.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.566+0000 7fb65cff9640 1 -- 172.21.15.105:0/1955150416 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb658005020 con 0x7fb660102280 2024-03-26T17:41:12.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb6683ee740 1 Processor -- start 2024-03-26T17:41:12.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb6683ee740 1 -- start start 2024-03-26T17:41:12.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb65cff9640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb64c0a2f20 0x7fb64c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56217c40 0x55ed56218060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed5621d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 0x55ed5621fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb6683ee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ed5617c970 con 0x55ed56217c40 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb6683ee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed56124d70 con 0x55ed56218630 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb6683ee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed56178920 con 0x55ed5621d7a0 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb65effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56217c40 0x55ed56218060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb657fff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb64c0a2f20 0x7fb64c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb65cff9640 1 -- 172.21.15.105:0/1955150416 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb6580cfae0 con 0x7fb660102280 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb65effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56217c40 0x55ed56218060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52534/0 (socket says 172.21.15.105:52534) 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.567+0000 7fb65effd640 1 -- 172.21.15.105:0/4039518272 learned_addr learned my addr 172.21.15.105:0/4039518272 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb657fff640 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed5621d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 0x55ed5621fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb65effd640 1 -- 172.21.15.105:0/4039518272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 msgr2=0x55ed5621fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb65effd640 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 0x55ed5621fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb65effd640 1 -- 172.21.15.105:0/4039518272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 msgr2=0x55ed5621d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb65effd640 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed5621d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb65effd640 1 -- 172.21.15.105:0/4039518272 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed55e903f0 con 0x55ed56217c40 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb657fff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb64c0a2f20 0x7fb64c0a53e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb648009fd0 tx=0x7fb648009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb65effd640 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56217c40 0x55ed56218060 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb650009900 tx=0x7fb6500095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9df4e95dfc73b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb6557fa640 1 -- 172.21.15.105:0/4039518272 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650045020 con 0x55ed56217c40 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb6557fa640 1 -- 172.21.15.105:0/4039518272 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb65002cc90 con 0x55ed56217c40 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.568+0000 7fb6557fa640 1 -- 172.21.15.105:0/4039518272 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500046c0 con 0x55ed56217c40 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 -- 172.21.15.105:0/4039518272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56217c40 msgr2=0x55ed56218060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56217c40 0x55ed56218060 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb650009900 tx=0x7fb6500095f0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 -- 172.21.15.105:0/4039518272 shutdown_connections 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 0x55ed5621fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed5621d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 --2- 172.21.15.105:0/4039518272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56217c40 0x55ed56218060 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 -- 172.21.15.105:0/4039518272 >> 172.21.15.105:0/4039518272 conn(0x55ed562131b0 msgr2=0x55ed562135b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 -- 172.21.15.105:0/4039518272 shutdown_connections 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.569+0000 7fb6683ee740 1 -- 172.21.15.105:0/4039518272 wait complete. 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 1 -- start start 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 1 Processor -- start 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed562220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56218630 0x55ed56222610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed56229650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ed562206f0 con 0x55ed5621d7a0 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed56138310 con 0x55ed56217c40 2024-03-26T17:41:12.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed56220870 con 0x55ed56218630 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6683ee740 10 client.0 _ll_register_callbacks cb 0x55ed56173c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed56229650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed56229650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52544/0 (socket says 172.21.15.105:52544) 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb657fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56218630 0x55ed56222610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 -- 172.21.15.105:0/296452850 learned_addr learned my addr 172.21.15.105:0/296452850 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65effd640 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed562220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 -- 172.21.15.105:0/296452850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56218630 msgr2=0x55ed56222610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56218630 0x55ed56222610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 -- 172.21.15.105:0/296452850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 msgr2=0x55ed562220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed562220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 -- 172.21.15.105:0/296452850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed55f5a7d0 con 0x55ed5621d7a0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed56229650 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb6580cd5a0 tx=0x7fb65809bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a213998ee1543dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.570+0000 7fb6467fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6467fc640 1 -- 172.21.15.105:0/296452850 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580a0120 con 0x55ed5621d7a0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6467fc640 1 -- 172.21.15.105:0/296452850 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb6580bc200 con 0x55ed5621d7a0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 1 -- 172.21.15.105:0/296452850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb65002cc90 con 0x55ed5621d7a0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6467fc640 1 -- 172.21.15.105:0/296452850 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb65809c1d0 con 0x55ed5621d7a0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 10 client.16494 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 1 -- 172.21.15.105:0/296452850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed5622ca30 con 0x55ed5621d7a0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6467fc640 1 -- 172.21.15.105:0/296452850 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb65002cc90 con 0x55ed5621d7a0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6467fc640 1 -- 172.21.15.105:0/296452850 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb658005310 con 0x55ed5621d7a0 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 10 client.16494 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 10 client.16494 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 20 client.16494 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:12.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ed5622e1c0 0x55ed56230660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 4 client.16494 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.571+0000 7fb6683ee740 1 -- 172.21.15.105:0/296452850 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed56189eb0 con 0x55ed5622e1c0 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.572+0000 7fb65effd640 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ed5622e1c0 0x55ed56230660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.572+0000 7fb65effd640 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ed5622e1c0 0x55ed56230660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2317a5893ec0f2d server_cookie=885cdcd16fd9607a in_seq=0 out_seq=0 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.572+0000 7fb6467fc640 10 client.16494 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6467fc640 1 -- 172.21.15.105:0/296452850 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ed56189eb0 con 0x55ed5622e1c0 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6467fc640 10 client.16494 handle_command_reply: tid=0 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 client.16494 shutdown 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 -- 172.21.15.105:0/296452850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 msgr2=0x55ed56229650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed56229650 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb6580cd5a0 tx=0x7fb65809bb30 comp rx=0 tx=0).stop 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 -- 172.21.15.105:0/296452850 shutdown_connections 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ed5622e1c0 0x55ed56230660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed56229650 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56218630 0x55ed56222610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 --2- 172.21.15.105:0/296452850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed562220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 -- 172.21.15.105:0/296452850 >> 172.21.15.105:0/296452850 conn(0x55ed562131b0 msgr2=0x55ed562123d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6467fc640 0 client.16494 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:12.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 -- 172.21.15.105:0/296452850 shutdown_connections 2024-03-26T17:41:12.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 1 -- 172.21.15.105:0/296452850 wait complete. 2024-03-26T17:41:12.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.573+0000 7fb6683ee740 20 client.16494 trim_cache size 0 max 16384 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.588+0000 7fb6683ee740 1 Processor -- start 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.588+0000 7fb6683ee740 1 -- start start 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56217c40 0x55ed56211b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed56212080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed562125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb6683ee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6601a8ae0 con 0x55ed5621d7a0 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb6683ee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6601aa620 con 0x55ed56218630 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb6683ee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6601a91b0 con 0x55ed56217c40 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed562125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed562125c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52552/0 (socket says 172.21.15.105:52552) 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 -- 172.21.15.105:0/544661998 learned_addr learned my addr 172.21.15.105:0/544661998 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb657fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed56212080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65effd640 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56217c40 0x55ed56211b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 -- 172.21.15.105:0/544661998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56217c40 msgr2=0x55ed56211b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56217c40 0x55ed56211b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 -- 172.21.15.105:0/544661998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 msgr2=0x55ed56212080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed56212080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 -- 172.21.15.105:0/544661998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed56212b00 con 0x55ed5621d7a0 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb657fff640 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed56212080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed562125c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb65809a480 tx=0x7fb65809bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7aa036c771936f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb654ff9640 1 -- 172.21.15.105:0/544661998 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580a0120 con 0x55ed5621d7a0 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb654ff9640 1 -- 172.21.15.105:0/544661998 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6580a0320 con 0x55ed5621d7a0 2024-03-26T17:41:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.589+0000 7fb654ff9640 1 -- 172.21.15.105:0/544661998 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65809bfa0 con 0x55ed5621d7a0 2024-03-26T17:41:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 -- 172.21.15.105:0/544661998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 msgr2=0x55ed562125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed562125c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb65809a480 tx=0x7fb65809bc60 comp rx=0 tx=0).stop 2024-03-26T17:41:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 -- 172.21.15.105:0/544661998 shutdown_connections 2024-03-26T17:41:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed5621d7a0 0x55ed562125c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56218630 0x55ed56212080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 --2- 172.21.15.105:0/544661998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed56217c40 0x55ed56211b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 -- 172.21.15.105:0/544661998 >> 172.21.15.105:0/544661998 conn(0x55ed562131b0 msgr2=0x55ed56219ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 -- 172.21.15.105:0/544661998 shutdown_connections 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.590+0000 7fb6683ee740 1 -- 172.21.15.105:0/544661998 wait complete. 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 1 -- start start 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 1 Processor -- start 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed5628fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56218630 0x55ed562901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 0x55ed56297230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6601a9f50 con 0x55ed56218630 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6601a9880 con 0x55ed56217c40 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6601a6ac0 con 0x55ed5621d7a0 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb657fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56218630 0x55ed562901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb65effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed5628fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb6683ee740 10 client.0 _ll_register_callbacks cb 0x7fb658004ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb65effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed5628fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54316/0 (socket says 172.21.15.105:54316) 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb65effd640 1 -- 172.21.15.105:0/3620083300 learned_addr learned my addr 172.21.15.105:0/3620083300 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb657fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56218630 0x55ed562901f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52564/0 (socket says 172.21.15.105:52564) 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb65f7fe640 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 0x55ed56297230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb657fff640 1 -- 172.21.15.105:0/3620083300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 msgr2=0x55ed56297230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb657fff640 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 0x55ed56297230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb657fff640 1 -- 172.21.15.105:0/3620083300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 msgr2=0x55ed5628fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb657fff640 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed5628fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.591+0000 7fb657fff640 1 -- 172.21.15.105:0/3620083300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed56297820 con 0x55ed56218630 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb657fff640 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56218630 0x55ed562901f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb648011610 tx=0x7fb648005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb8d7e981ecf2e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6467fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6467fc640 1 -- 172.21.15.105:0/3620083300 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648031060 con 0x55ed56218630 2024-03-26T17:41:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6467fc640 1 -- 172.21.15.105:0/3620083300 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb648009b50 con 0x55ed56218630 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 1 -- 172.21.15.105:0/3620083300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6580a0320 con 0x55ed56218630 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6467fc640 1 -- 172.21.15.105:0/3620083300 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb648026040 con 0x55ed56218630 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 10 client.16500 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 1 -- 172.21.15.105:0/3620083300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed56298b90 con 0x55ed56218630 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6467fc640 1 -- 172.21.15.105:0/3620083300 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6580a0320 con 0x55ed56218630 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6467fc640 1 -- 172.21.15.105:0/3620083300 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb64802c4c0 con 0x55ed56218630 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 10 client.16500 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 10 client.16500 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 20 client.16500 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ed56299590 0x55ed5629b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 4 client.16500 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.592+0000 7fb6683ee740 1 -- 172.21.15.105:0/3620083300 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed56184200 con 0x55ed56299590 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb65effd640 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ed56299590 0x55ed5629b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb65effd640 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ed56299590 0x55ed5629b9b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a82d376cc071448f server_cookie=9e629f103fe9da47 in_seq=0 out_seq=0 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6467fc640 10 client.16500 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6467fc640 1 -- 172.21.15.105:0/3620083300 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed56184200 con 0x55ed56299590 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6467fc640 10 client.16500 handle_command_reply: tid=0 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 client.16500 shutdown 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 -- 172.21.15.105:0/3620083300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56218630 msgr2=0x55ed562901f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56218630 0x55ed562901f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb648011610 tx=0x7fb648005f90 comp rx=0 tx=0).stop 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 -- 172.21.15.105:0/3620083300 shutdown_connections 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ed56299590 0x55ed5629b9b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed5621d7a0 0x55ed56297230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed56218630 0x55ed562901f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 --2- 172.21.15.105:0/3620083300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed56217c40 0x55ed5628fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 -- 172.21.15.105:0/3620083300 >> 172.21.15.105:0/3620083300 conn(0x55ed562131b0 msgr2=0x55ed56211b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6467fc640 0 client.16500 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 -- 172.21.15.105:0/3620083300 shutdown_connections 2024-03-26T17:41:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.593+0000 7fb6683ee740 1 -- 172.21.15.105:0/3620083300 wait complete. 2024-03-26T17:41:12.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6683ee740 20 client.16500 trim_cache size 0 max 16384 2024-03-26T17:41:12.594 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 -- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb64c0a2f20 msgr2=0x7fb64c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb64c0a2f20 0x7fb64c0a53e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb648009fd0 tx=0x7fb648009ee0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 -- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb660102280 msgr2=0x7fb66019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb660102280 0x7fb66019efd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb65800eb10 tx=0x7fb65800c2d0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 -- 172.21.15.105:0/1955150416 shutdown_connections 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb64c0a2f20 0x7fb64c0a53e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb660102280 0x7fb66019efd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6601018c0 0x7fb660197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 --2- 172.21.15.105:0/1955150416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6601006d0 0x7fb660197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 -- 172.21.15.105:0/1955150416 >> 172.21.15.105:0/1955150416 conn(0x7fb6600fbc60 msgr2=0x7fb660104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 -- 172.21.15.105:0/1955150416 shutdown_connections 2024-03-26T17:41:12.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.594+0000 7fb6657ff640 1 -- 172.21.15.105:0/1955150416 wait complete. 2024-03-26T17:41:12.600 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:41:12.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 -- 172.21.15.105:0/1822517399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 msgr2=0x7fdf0c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 --2- 172.21.15.105:0/1822517399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 0x7fdf0c109d30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009ca0 tx=0x7fdf0402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 -- 172.21.15.105:0/1822517399 shutdown_connections 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 --2- 172.21.15.105:0/1822517399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 0x7fdf0c109d30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 --2- 172.21.15.105:0/1822517399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf0c102ae0 0x7fdf0c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 --2- 172.21.15.105:0/1822517399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf0c1018f0 0x7fdf0c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 -- 172.21.15.105:0/1822517399 >> 172.21.15.105:0/1822517399 conn(0x7fdf0c0fce60 msgr2=0x7fdf0c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 -- 172.21.15.105:0/1822517399 shutdown_connections 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 -- 172.21.15.105:0/1822517399 wait complete. 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 Processor -- start 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.724+0000 7fdf13777640 1 -- start start 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf13777640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf0c1018f0 0x7fdf0c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf13777640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf0c102ae0 0x7fdf0c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf13777640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 0x7fdf0c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf13777640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10d400 con 0x7fdf0c1034a0 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf13777640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10d280 con 0x7fdf0c1018f0 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf13777640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c10d580 con 0x7fdf0c102ae0 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 0x7fdf0c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 0x7fdf0c19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52584/0 (socket says 172.21.15.105:52584) 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 -- 172.21.15.105:0/134620817 learned_addr learned my addr 172.21.15.105:0/134620817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11513640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf0c1018f0 0x7fdf0c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf10d12640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf0c102ae0 0x7fdf0c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 -- 172.21.15.105:0/134620817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf0c102ae0 msgr2=0x7fdf0c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf0c102ae0 0x7fdf0c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 -- 172.21.15.105:0/134620817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf0c1018f0 msgr2=0x7fdf0c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf0c1018f0 0x7fdf0c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 -- 172.21.15.105:0/134620817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c19f6c0 con 0x7fdf0c1034a0 2024-03-26T17:41:12.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf10d12640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf0c102ae0 0x7fdf0c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.725+0000 7fdf11d14640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 0x7fdf0c19f050 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdf0402c700 tx=0x7fdf040328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.726+0000 7fdf027fc640 1 -- 172.21.15.105:0/134620817 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04041020 con 0x7fdf0c1034a0 2024-03-26T17:41:12.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.726+0000 7fdf13777640 1 -- 172.21.15.105:0/134620817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c19f950 con 0x7fdf0c1034a0 2024-03-26T17:41:12.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.726+0000 7fdf027fc640 1 -- 172.21.15.105:0/134620817 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf04003e20 con 0x7fdf0c1034a0 2024-03-26T17:41:12.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.726+0000 7fdf027fc640 1 -- 172.21.15.105:0/134620817 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04039040 con 0x7fdf0c1034a0 2024-03-26T17:41:12.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.726+0000 7fdf13777640 1 -- 172.21.15.105:0/134620817 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c19fe40 con 0x7fdf0c1034a0 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf16366740 1 Processor -- start 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf027fc640 1 -- 172.21.15.105:0/134620817 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdf04044020 con 0x7fdf0c1034a0 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf16366740 1 -- start start 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf16366740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb68a30 0x55d18eb68e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf16366740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb6df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf16366740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb6e590 0x55d18eb70a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf16366740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d18eacd970 con 0x55d18eb68a30 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf16366740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d18ea75d70 con 0x55d18eb69420 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf16366740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d18eac9920 con 0x55d18eb6e590 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf10d12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb68a30 0x55d18eb68e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf11513640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb6df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf10d12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb68a30 0x55d18eb68e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52590/0 (socket says 172.21.15.105:52590) 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.729+0000 7fdf10d12640 1 -- 172.21.15.105:0/3990867486 learned_addr learned my addr 172.21.15.105:0/3990867486 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf10d12640 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb6e590 0x55d18eb70a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf027fc640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdef80a2f20 0x7fdef80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf11513640 1 -- 172.21.15.105:0/3990867486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb6e590 msgr2=0x55d18eb70a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf11513640 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb6e590 0x55d18eb70a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf11513640 1 -- 172.21.15.105:0/3990867486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb68a30 msgr2=0x55d18eb68e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf11513640 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb68a30 0x55d18eb68e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf11513640 1 -- 172.21.15.105:0/3990867486 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18e7e13f0 con 0x55d18eb69420 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf10d12640 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb6e590 0x55d18eb70a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf027fc640 1 -- 172.21.15.105:0/134620817 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdf040f1600 con 0x7fdf0c1034a0 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf11d14640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdef80a2f20 0x7fdef80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdf11513640 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb6df20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009bf0 tx=0x7fdf08011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d173bec1c615d16a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdef77fe640 1 -- 172.21.15.105:0/3990867486 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08009da0 con 0x55d18eb69420 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdef77fe640 1 -- 172.21.15.105:0/3990867486 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf08005020 con 0x55d18eb69420 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.730+0000 7fdef77fe640 1 -- 172.21.15.105:0/3990867486 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004a00 con 0x55d18eb69420 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 -- 172.21.15.105:0/3990867486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 msgr2=0x55d18eb6df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf11d14640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdef80a2f20 0x7fdef80a53e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdf040bdc80 tx=0x7fdf040095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb6df20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009bf0 tx=0x7fdf08011810 comp rx=0 tx=0).stop 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 -- 172.21.15.105:0/3990867486 shutdown_connections 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb6e590 0x55d18eb70a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb6df20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 --2- 172.21.15.105:0/3990867486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb68a30 0x55d18eb68e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 -- 172.21.15.105:0/3990867486 >> 172.21.15.105:0/3990867486 conn(0x55d18eb63fa0 msgr2=0x55d18eb643a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 -- 172.21.15.105:0/3990867486 shutdown_connections 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 -- 172.21.15.105:0/3990867486 wait complete. 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 -- start start 2024-03-26T17:41:12.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.731+0000 7fdf16366740 1 Processor -- start 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf16366740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb72f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf16366740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf16366740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb7a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf16366740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d18eb71540 con 0x55d18eb6e590 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf16366740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d18ea89310 con 0x55d18eb69420 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf16366740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d18eb716c0 con 0x55d18eb68a30 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf16366740 10 client.0 _ll_register_callbacks cb 0x55d18eac4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf10d12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb7a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf11513640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf10d12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb7a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52602/0 (socket says 172.21.15.105:52602) 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf10d12640 1 -- 172.21.15.105:0/612148201 learned_addr learned my addr 172.21.15.105:0/612148201 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf10d12640 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb72f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf11513640 1 -- 172.21.15.105:0/612148201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 msgr2=0x55d18eb72f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf11513640 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb72f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf11513640 1 -- 172.21.15.105:0/612148201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 msgr2=0x55d18eb7a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf11513640 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb7a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf11513640 1 -- 172.21.15.105:0/612148201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18e8ab7d0 con 0x55d18eb69420 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.732+0000 7fdf10d12640 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb7a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf11513640 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb734b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55d18eb64f10 tx=0x7fdf080048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd690a9a40cd93c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdef4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdef4ff9640 1 -- 172.21.15.105:0/612148201 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08009da0 con 0x55d18eb69420 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 1 -- 172.21.15.105:0/612148201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf08005020 con 0x55d18eb69420 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdef4ff9640 1 -- 172.21.15.105:0/612148201 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdf08004cf0 con 0x55d18eb69420 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdef4ff9640 1 -- 172.21.15.105:0/612148201 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf08025470 con 0x55d18eb69420 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 10 client.26363 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 1 -- 172.21.15.105:0/612148201 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d18eb7d840 con 0x55d18eb69420 2024-03-26T17:41:12.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdef4ff9640 1 -- 172.21.15.105:0/612148201 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf08005020 con 0x55d18eb69420 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdef4ff9640 1 -- 172.21.15.105:0/612148201 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdf080195e0 con 0x55d18eb69420 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 10 client.26363 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 10 client.26363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 20 client.26363 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d18eb7ed50 0x55d18eb81180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 4 client.26363 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.733+0000 7fdf16366740 1 -- 172.21.15.105:0/612148201 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d18eadaeb0 con 0x55d18eb7ed50 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.734+0000 7fdf10d12640 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d18eb7ed50 0x55d18eb81180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.734+0000 7fdf10d12640 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d18eb7ed50 0x55d18eb81180 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf31c1583a54fe64 server_cookie=31e66cc556453813 in_seq=0 out_seq=0 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.734+0000 7fdef4ff9640 10 client.26363 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdef4ff9640 1 -- 172.21.15.105:0/612148201 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d18eadaeb0 con 0x55d18eb7ed50 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdef4ff9640 10 client.26363 handle_command_reply: tid=0 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 client.26363 shutdown 2024-03-26T17:41:12.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 -- 172.21.15.105:0/612148201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 msgr2=0x55d18eb734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb734b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55d18eb64f10 tx=0x7fdf080048f0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 -- 172.21.15.105:0/612148201 shutdown_connections 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb7a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d18eb7ed50 0x55d18eb81180 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdef4ff9640 0 client.26363 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb734b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 --2- 172.21.15.105:0/612148201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb72f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 -- 172.21.15.105:0/612148201 >> 172.21.15.105:0/612148201 conn(0x55d18eb63fa0 msgr2=0x55d18eb631c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 -- 172.21.15.105:0/612148201 shutdown_connections 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 1 -- 172.21.15.105:0/612148201 wait complete. 2024-03-26T17:41:12.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.735+0000 7fdf16366740 20 client.26363 trim_cache size 0 max 16384 2024-03-26T17:41:12.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf16366740 1 Processor -- start 2024-03-26T17:41:12.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf16366740 1 -- start start 2024-03-26T17:41:12.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf16366740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf16366740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf16366740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf16366740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c1a8d50 con 0x55d18eb6e590 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf16366740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c1aa890 con 0x55d18eb69420 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf16366740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c1a9420 con 0x55d18eb68a30 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf11513640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf10d12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf11513640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54372/0 (socket says 172.21.15.105:54372) 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf11513640 1 -- 172.21.15.105:0/3167233362 learned_addr learned my addr 172.21.15.105:0/3167233362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf10d12640 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf11513640 1 -- 172.21.15.105:0/3167233362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 msgr2=0x55d18eb71a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf11513640 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb71a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf11513640 1 -- 172.21.15.105:0/3167233362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 msgr2=0x55d18eb71fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf11513640 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb71fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf11513640 1 -- 172.21.15.105:0/3167233362 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18eb5e5d0 con 0x55d18eb69420 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.751+0000 7fdf10d12640 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.752+0000 7fdf11513640 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a4b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf08019c40 tx=0x7fdf080096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f53f547e4cb1d60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.752+0000 7fdef6ffd640 1 -- 172.21.15.105:0/3167233362 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08009a70 con 0x55d18eb69420 2024-03-26T17:41:12.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.752+0000 7fdef6ffd640 1 -- 172.21.15.105:0/3167233362 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf08009ce0 con 0x55d18eb69420 2024-03-26T17:41:12.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.752+0000 7fdef6ffd640 1 -- 172.21.15.105:0/3167233362 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08025630 con 0x55d18eb69420 2024-03-26T17:41:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 -- 172.21.15.105:0/3167233362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 msgr2=0x55d18eb7a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a4b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdf08019c40 tx=0x7fdf080096c0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 -- 172.21.15.105:0/3167233362 shutdown_connections 2024-03-26T17:41:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a4b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 --2- 172.21.15.105:0/3167233362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb71a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 -- 172.21.15.105:0/3167233362 >> 172.21.15.105:0/3167233362 conn(0x55d18eb63fa0 msgr2=0x55d18eb72f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 -- 172.21.15.105:0/3167233362 shutdown_connections 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 -- 172.21.15.105:0/3167233362 wait complete. 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 -- start start 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.753+0000 7fdf16366740 1 Processor -- start 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf16366740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb79f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf16366740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf16366740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb76c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf16366740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c1aa1c0 con 0x55d18eb6e590 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf16366740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c1a9af0 con 0x55d18eb69420 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf16366740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c1a6d30 con 0x55d18eb68a30 2024-03-26T17:41:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf16366740 10 client.0 _ll_register_callbacks cb 0x7fdefc009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf10d12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb79f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf11513640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf10d12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb79f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45692/0 (socket says 172.21.15.105:45692) 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf11513640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54388/0 (socket says 172.21.15.105:54388) 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf10d12640 1 -- 172.21.15.105:0/1400067618 learned_addr learned my addr 172.21.15.105:0/1400067618 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf10d12640 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb76c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf11513640 1 -- 172.21.15.105:0/1400067618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 msgr2=0x55d18eb79f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf11513640 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb79f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf11513640 1 -- 172.21.15.105:0/1400067618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 msgr2=0x55d18eb76c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf11513640 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb76c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf11513640 1 -- 172.21.15.105:0/1400067618 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18eb77430 con 0x55d18eb69420 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.754+0000 7fdf10d12640 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb79f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf11513640 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55d18eb5f900 tx=0x7fdf0802e440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a0f88a47b920aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdef4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdef4ff9640 1 -- 172.21.15.105:0/1400067618 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08025810 con 0x55d18eb69420 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdef4ff9640 1 -- 172.21.15.105:0/1400067618 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdf080259b0 con 0x55d18eb69420 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 1 -- 172.21.15.105:0/1400067618 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf08009ce0 con 0x55d18eb69420 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdef4ff9640 1 -- 172.21.15.105:0/1400067618 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf080048d0 con 0x55d18eb69420 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 10 client.26369 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 1 -- 172.21.15.105:0/1400067618 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d18ebe3e40 con 0x55d18eb69420 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdef4ff9640 1 -- 172.21.15.105:0/1400067618 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf08009ce0 con 0x55d18eb69420 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdef4ff9640 1 -- 172.21.15.105:0/1400067618 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdf08040060 con 0x55d18eb69420 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 10 client.26369 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 10 client.26369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 20 client.26369 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d18ebe45a0 0x55d18ebe6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 4 client.26369 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.755+0000 7fdf16366740 1 -- 172.21.15.105:0/1400067618 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d18ead5200 con 0x55d18ebe45a0 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf10d12640 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d18ebe45a0 0x55d18ebe6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf10d12640 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d18ebe45a0 0x55d18ebe6990 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44324fd8f5686c47 server_cookie=35017ea8163f4e3b in_seq=0 out_seq=0 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdef4ff9640 10 client.26369 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdef4ff9640 1 -- 172.21.15.105:0/1400067618 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d18ead5200 con 0x55d18ebe45a0 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdef4ff9640 10 client.26369 handle_command_reply: tid=0 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 client.26369 shutdown 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 -- 172.21.15.105:0/1400067618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 msgr2=0x55d18eb7a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55d18eb5f900 tx=0x7fdf0802e440 comp rx=0 tx=0).stop 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 -- 172.21.15.105:0/1400067618 shutdown_connections 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d18eb6e590 0x55d18eb76c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d18ebe45a0 0x55d18ebe6990 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d18eb69420 0x55d18eb7a460 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdef4ff9640 0 client.26369 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 --2- 172.21.15.105:0/1400067618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d18eb68a30 0x55d18eb79f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 -- 172.21.15.105:0/1400067618 >> 172.21.15.105:0/1400067618 conn(0x55d18eb63fa0 msgr2=0x55d18eb6a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 -- 172.21.15.105:0/1400067618 shutdown_connections 2024-03-26T17:41:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.756+0000 7fdf16366740 1 -- 172.21.15.105:0/1400067618 wait complete. 2024-03-26T17:41:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf16366740 20 client.26369 trim_cache size 0 max 16384 2024-03-26T17:41:12.756 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 -- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdef80a2f20 msgr2=0x7fdef80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdef80a2f20 0x7fdef80a53e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdf040bdc80 tx=0x7fdf040095c0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 -- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 msgr2=0x7fdf0c19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 0x7fdf0c19f050 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdf0402c700 tx=0x7fdf040328a0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 -- 172.21.15.105:0/134620817 shutdown_connections 2024-03-26T17:41:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdef80a2f20 0x7fdef80a53e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf0c1034a0 0x7fdf0c19f050 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf0c102ae0 0x7fdf0c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 --2- 172.21.15.105:0/134620817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf0c1018f0 0x7fdf0c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 -- 172.21.15.105:0/134620817 >> 172.21.15.105:0/134620817 conn(0x7fdf0c0fce60 msgr2=0x7fdf0c105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 -- 172.21.15.105:0/134620817 shutdown_connections 2024-03-26T17:41:12.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.757+0000 7fdf13777640 1 -- 172.21.15.105:0/134620817 wait complete. 2024-03-26T17:41:12.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:41:12.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 -- 172.21.15.105:0/3060925677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 msgr2=0x7fcc1c1047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/3060925677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 0x7fcc1c1047d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009900 tx=0x7fcc0c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:12.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 -- 172.21.15.105:0/3060925677 shutdown_connections 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/3060925677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc1c107640 0x7fcc1c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/3060925677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1c104d10 0x7fcc1c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/3060925677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 0x7fcc1c1047d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 -- 172.21.15.105:0/3060925677 >> 172.21.15.105:0/3060925677 conn(0x7fcc1c0fbd50 msgr2=0x7fcc1c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 -- 172.21.15.105:0/3060925677 shutdown_connections 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.886+0000 7fcc21a4b640 1 -- 172.21.15.105:0/3060925677 wait complete. 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc21a4b640 1 Processor -- start 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc21a4b640 1 -- start start 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc21a4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 0x7fcc1c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc21a4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc1c104d10 0x7fcc1c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc21a4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1c107640 0x7fcc1c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc21a4b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c10b250 con 0x7fcc1c1023e0 2024-03-26T17:41:12.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc21a4b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c10b0d0 con 0x7fcc1c104d10 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc21a4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c10b3d0 con 0x7fcc1c107640 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 0x7fcc1c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 0x7fcc1c195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52658/0 (socket says 172.21.15.105:52658) 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 -- 172.21.15.105:0/1779366277 learned_addr learned my addr 172.21.15.105:0/1779366277 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc13fff640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc1c104d10 0x7fcc1c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1c107640 0x7fcc1c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 -- 172.21.15.105:0/1779366277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1c107640 msgr2=0x7fcc1c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1c107640 0x7fcc1c19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 -- 172.21.15.105:0/1779366277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc1c104d10 msgr2=0x7fcc1c195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc1c104d10 0x7fcc1c195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 -- 172.21.15.105:0/1779366277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c19d5a0 con 0x7fcc1c1023e0 2024-03-26T17:41:12.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc13fff640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc1c104d10 0x7fcc1c195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.887+0000 7fcc1affd640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 0x7fcc1c195980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c006010 tx=0x7fcc0c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.888+0000 7fcc20a49640 1 -- 172.21.15.105:0/1779366277 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c043020 con 0x7fcc1c1023e0 2024-03-26T17:41:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.888+0000 7fcc21a4b640 1 -- 172.21.15.105:0/1779366277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c19d830 con 0x7fcc1c1023e0 2024-03-26T17:41:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.888+0000 7fcc20a49640 1 -- 172.21.15.105:0/1779366277 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc0c0043d0 con 0x7fcc1c1023e0 2024-03-26T17:41:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.888+0000 7fcc20a49640 1 -- 172.21.15.105:0/1779366277 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c003b80 con 0x7fcc1c1023e0 2024-03-26T17:41:12.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.888+0000 7fcc21a4b640 1 -- 172.21.15.105:0/1779366277 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c19dd80 con 0x7fcc1c1023e0 2024-03-26T17:41:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.890+0000 7fcc24636740 1 Processor -- start 2024-03-26T17:41:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.890+0000 7fcc24636740 1 -- start start 2024-03-26T17:41:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.890+0000 7fcc24636740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d656e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc24636740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d65bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc24636740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d65ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc24636740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56310d5bb970 con 0x56310d656a30 2024-03-26T17:41:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc24636740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56310d563d70 con 0x56310d65c590 2024-03-26T17:41:12.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc24636740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56310d5b7920 con 0x56310d657420 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc1b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d656e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc1b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d656e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52670/0 (socket says 172.21.15.105:52670) 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc1b7fe640 1 -- 172.21.15.105:0/2291323112 learned_addr learned my addr 172.21.15.105:0/2291323112 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc13fff640 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d65ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc20a49640 1 -- 172.21.15.105:0/1779366277 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcc0c046020 con 0x7fcc1c1023e0 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc1affd640 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d65bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc13fff640 1 -- 172.21.15.105:0/2291323112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 msgr2=0x56310d65bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc13fff640 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d65bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc13fff640 1 -- 172.21.15.105:0/2291323112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 msgr2=0x56310d656e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc13fff640 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d656e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc13fff640 1 -- 172.21.15.105:0/2291323112 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56310d2cf3f0 con 0x56310d65c590 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d656e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc1affd640 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d65bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc13fff640 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d65ea20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0694e0 tx=0x7fcc040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd5a7377853f5d0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc20a49640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc080a2f20 0x7fcc080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.891+0000 7fcc11ffb640 1 -- 172.21.15.105:0/2291323112 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04004310 con 0x56310d65c590 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc11ffb640 1 -- 172.21.15.105:0/2291323112 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc040044b0 con 0x56310d65c590 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc11ffb640 1 -- 172.21.15.105:0/2291323112 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc040035f0 con 0x56310d65c590 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc080a2f20 0x7fcc080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc20a49640 1 -- 172.21.15.105:0/1779366277 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcc0c0f1600 con 0x7fcc1c1023e0 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 -- 172.21.15.105:0/2291323112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 msgr2=0x56310d65ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc080a2f20 0x7fcc080a53e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009fd0 tx=0x7fcc14011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d65ea20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0694e0 tx=0x7fcc040118d0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 -- 172.21.15.105:0/2291323112 shutdown_connections 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d65ea20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d65bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 --2- 172.21.15.105:0/2291323112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d656e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 -- 172.21.15.105:0/2291323112 >> 172.21.15.105:0/2291323112 conn(0x56310d651fa0 msgr2=0x56310d6523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 -- 172.21.15.105:0/2291323112 shutdown_connections 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.892+0000 7fcc24636740 1 -- 172.21.15.105:0/2291323112 wait complete. 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 1 -- start start 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 1 Processor -- start 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d656a30 0x56310d660f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d661440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d65c590 0x56310d668480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56310d65f540 con 0x56310d65c590 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56310d577310 con 0x56310d656a30 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56310d65f6c0 con 0x56310d657420 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc24636740 10 client.0 _ll_register_callbacks cb 0x56310d5b2c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:12.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc1affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d656a30 0x56310d660f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc1affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d656a30 0x56310d660f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54434/0 (socket says 172.21.15.105:54434) 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc1affd640 1 -- 172.21.15.105:0/2161332663 learned_addr learned my addr 172.21.15.105:0/2161332663 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.893+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d65c590 0x56310d668480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc13fff640 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d661440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc1affd640 1 -- 172.21.15.105:0/2161332663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 msgr2=0x56310d661440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc1affd640 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d661440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc1affd640 1 -- 172.21.15.105:0/2161332663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d65c590 msgr2=0x56310d668480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc1affd640 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d65c590 0x56310d668480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc1affd640 1 -- 172.21.15.105:0/2161332663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56310d3997d0 con 0x56310d656a30 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d65c590 0x56310d668480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc1affd640 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d656a30 0x56310d660f00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005ab0 tx=0x7fcc0c0bd650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdebd88695fc7c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcbfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcbfaffd640 1 -- 172.21.15.105:0/2161332663 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c0c1e40 con 0x56310d656a30 2024-03-26T17:41:12.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcbfaffd640 1 -- 172.21.15.105:0/2161332663 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcc0c0054b0 con 0x56310d656a30 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc24636740 1 -- 172.21.15.105:0/2161332663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc040044b0 con 0x56310d656a30 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcbfaffd640 1 -- 172.21.15.105:0/2161332663 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc0c0056b0 con 0x56310d656a30 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc24636740 10 client.26381 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.894+0000 7fcc24636740 1 -- 172.21.15.105:0/2161332663 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56310d66b760 con 0x56310d656a30 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcbfaffd640 1 -- 172.21.15.105:0/2161332663 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc0c0c6700 con 0x56310d656a30 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcbfaffd640 1 -- 172.21.15.105:0/2161332663 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcc0c0b9a40 con 0x56310d656a30 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcc24636740 10 client.26381 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcc24636740 10 client.26381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcc24636740 20 client.26381 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcc24636740 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56310d66cda0 0x56310d66f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcc24636740 4 client.26381 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcc24636740 1 -- 172.21.15.105:0/2161332663 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcc040044b0 con 0x56310d66cda0 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcc13fff640 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56310d66cda0 0x56310d66f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcbfaffd640 10 client.26381 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.895+0000 7fcc13fff640 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56310d66cda0 0x56310d66f210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=803e2f14e4136883 server_cookie=55bae5246b494896 in_seq=0 out_seq=0 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcbfaffd640 1 -- 172.21.15.105:0/2161332663 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fcc040044b0 con 0x56310d66cda0 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcbfaffd640 10 client.26381 handle_command_reply: tid=0 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 client.26381 shutdown 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 -- 172.21.15.105:0/2161332663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d656a30 msgr2=0x56310d660f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d656a30 0x56310d660f00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c005ab0 tx=0x7fcc0c0bd650 comp rx=0 tx=0).stop 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 -- 172.21.15.105:0/2161332663 shutdown_connections 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56310d66cda0 0x56310d66f210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d65c590 0x56310d668480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcbfaffd640 0 client.26381 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d661440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 --2- 172.21.15.105:0/2161332663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d656a30 0x56310d660f00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 -- 172.21.15.105:0/2161332663 >> 172.21.15.105:0/2161332663 conn(0x56310d651fa0 msgr2=0x56310d6511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 -- 172.21.15.105:0/2161332663 shutdown_connections 2024-03-26T17:41:12.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 1 -- 172.21.15.105:0/2161332663 wait complete. 2024-03-26T17:41:12.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.896+0000 7fcc24636740 20 client.26381 trim_cache size 0 max 16384 2024-03-26T17:41:12.898 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:12 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3586375285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.910+0000 7fcc24636740 1 Processor -- start 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.910+0000 7fcc24636740 1 -- start start 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.910+0000 7fcc24636740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d660b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.910+0000 7fcc24636740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d668440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.910+0000 7fcc24636740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d661050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.910+0000 7fcc24636740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1a6b30 con 0x56310d656a30 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.910+0000 7fcc24636740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1a8670 con 0x56310d65c590 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.910+0000 7fcc24636740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1a7200 con 0x56310d657420 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d660b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d661050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc13fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d668440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d661050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54446/0 (socket says 172.21.15.105:54446) 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d660b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52696/0 (socket says 172.21.15.105:52696) 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 -- 172.21.15.105:0/3619698644 learned_addr learned my addr 172.21.15.105:0/3619698644 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc13fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d668440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45740/0 (socket says 172.21.15.105:45740) 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 -- 172.21.15.105:0/3619698644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 msgr2=0x56310d668440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/3619698644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d668440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 -- 172.21.15.105:0/3619698644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 msgr2=0x56310d660b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/3619698644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d660b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 -- 172.21.15.105:0/3619698644 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56310d657800 con 0x56310d65c590 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc13fff640 1 --2- 172.21.15.105:0/3619698644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d668440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/3619698644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d661050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcc14003fd0 tx=0x7fcc1401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec285fa8eca2e63a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc117fa640 1 -- 172.21.15.105:0/3619698644 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14004540 con 0x56310d65c590 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc117fa640 1 -- 172.21.15.105:0/3619698644 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc1402f070 con 0x56310d65c590 2024-03-26T17:41:12.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.911+0000 7fcc117fa640 1 -- 172.21.15.105:0/3619698644 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1401b680 con 0x56310d65c590 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 -- 172.21.15.105:0/3619698644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 msgr2=0x56310d661050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 --2- 172.21.15.105:0/3619698644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d661050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcc14003fd0 tx=0x7fcc1401b000 comp rx=0 tx=0).stop 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 -- 172.21.15.105:0/3619698644 shutdown_connections 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 --2- 172.21.15.105:0/3619698644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d65c590 0x56310d661050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 --2- 172.21.15.105:0/3619698644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d657420 0x56310d668440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 --2- 172.21.15.105:0/3619698644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d660b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 -- 172.21.15.105:0/3619698644 >> 172.21.15.105:0/3619698644 conn(0x56310d651fa0 msgr2=0x56310d658920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 -- 172.21.15.105:0/3619698644 shutdown_connections 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 -- 172.21.15.105:0/3619698644 wait complete. 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.912+0000 7fcc24636740 1 -- start start 2024-03-26T17:41:12.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc24636740 1 Processor -- start 2024-03-26T17:41:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc24636740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d665790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc24636740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d657420 0x56310d665cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc24636740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d65c590 0x56310d666ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc24636740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1a7fa0 con 0x56310d656a30 2024-03-26T17:41:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc24636740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1a78d0 con 0x56310d657420 2024-03-26T17:41:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc24636740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1a4b10 con 0x56310d65c590 2024-03-26T17:41:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc24636740 10 client.0 _ll_register_callbacks cb 0x7fcc0c0bdac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:12.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d665790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d657420 0x56310d665cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d665790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52702/0 (socket says 172.21.15.105:52702) 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1affd640 1 -- 172.21.15.105:0/3944683362 learned_addr learned my addr 172.21.15.105:0/3944683362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1b7fe640 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d65c590 0x56310d666ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1affd640 1 -- 172.21.15.105:0/3944683362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d65c590 msgr2=0x56310d666ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1affd640 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d65c590 0x56310d666ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1affd640 1 -- 172.21.15.105:0/3944683362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d657420 msgr2=0x56310d665cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1affd640 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d657420 0x56310d665cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.913+0000 7fcc1affd640 1 -- 172.21.15.105:0/3944683362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56310d6672d0 con 0x56310d656a30 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc1affd640 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d665790 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0b9e00 tx=0x7fcc0c0bc200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806f7ab82c07f079 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcbfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcbfaffd640 1 -- 172.21.15.105:0/3944683362 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c0c6700 con 0x56310d656a30 2024-03-26T17:41:12.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcbfaffd640 1 -- 172.21.15.105:0/3944683362 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcc0c0c6270 con 0x56310d656a30 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 1 -- 172.21.15.105:0/3944683362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc1402f070 con 0x56310d656a30 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcbfaffd640 1 -- 172.21.15.105:0/3944683362 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc0c0c6470 con 0x56310d656a30 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 10 client.16536 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 1 -- 172.21.15.105:0/3944683362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56310d6685d0 con 0x56310d656a30 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcbfaffd640 1 -- 172.21.15.105:0/3944683362 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc1402f070 con 0x56310d656a30 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcbfaffd640 1 -- 172.21.15.105:0/3944683362 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcc0c0c2fd0 con 0x56310d656a30 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 10 client.16536 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 10 client.16536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 20 client.16536 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56310d6d33f0 0x56310d6d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 4 client.16536 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.914+0000 7fcc24636740 1 -- 172.21.15.105:0/3944683362 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56310d5c8eb0 con 0x56310d6d33f0 2024-03-26T17:41:12.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc13fff640 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56310d6d33f0 0x56310d6d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcbfaffd640 10 client.16536 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc13fff640 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56310d6d33f0 0x56310d6d57e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f79a2fdc79137b5a server_cookie=7caecc747d287f50 in_seq=0 out_seq=0 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcbfaffd640 1 -- 172.21.15.105:0/3944683362 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56310d5c8eb0 con 0x56310d6d33f0 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcbfaffd640 10 client.16536 handle_command_reply: tid=0 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 client.16536 shutdown 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 -- 172.21.15.105:0/3944683362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 msgr2=0x56310d665790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d665790 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c0b9e00 tx=0x7fcc0c0bc200 comp rx=0 tx=0).stop 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 -- 172.21.15.105:0/3944683362 shutdown_connections 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56310d6d33f0 0x56310d6d57e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56310d65c590 0x56310d666ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56310d657420 0x56310d665cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcbfaffd640 0 client.16536 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 --2- 172.21.15.105:0/3944683362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56310d656a30 0x56310d665790 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.915+0000 7fcc24636740 1 -- 172.21.15.105:0/3944683362 >> 172.21.15.105:0/3944683362 conn(0x56310d651fa0 msgr2=0x56310d660b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc24636740 1 -- 172.21.15.105:0/3944683362 shutdown_connections 2024-03-26T17:41:12.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc24636740 1 -- 172.21.15.105:0/3944683362 wait complete. 2024-03-26T17:41:12.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc24636740 20 client.16536 trim_cache size 0 max 16384 2024-03-26T17:41:12.916 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 -- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc080a2f20 msgr2=0x7fcc080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc080a2f20 0x7fcc080a53e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009fd0 tx=0x7fcc14011040 comp rx=0 tx=0).stop 2024-03-26T17:41:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 -- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 msgr2=0x7fcc1c195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 0x7fcc1c195980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c006010 tx=0x7fcc0c005e40 comp rx=0 tx=0).stop 2024-03-26T17:41:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 -- 172.21.15.105:0/1779366277 shutdown_connections 2024-03-26T17:41:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc080a2f20 0x7fcc080a53e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc1c107640 0x7fcc1c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcc1c104d10 0x7fcc1c195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 --2- 172.21.15.105:0/1779366277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc1c1023e0 0x7fcc1c195980 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:12.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 -- 172.21.15.105:0/1779366277 >> 172.21.15.105:0/1779366277 conn(0x7fcc1c0fbd50 msgr2=0x7fcc1c104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:12.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 -- 172.21.15.105:0/1779366277 shutdown_connections 2024-03-26T17:41:12.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:12.916+0000 7fcc21a4b640 1 -- 172.21.15.105:0/1779366277 wait complete. 2024-03-26T17:41:12.921 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:41:13.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 68888 3.36 MB/sec warmup 116 sec latency 18.867 ms 2024-03-26T17:41:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 -- 172.21.15.105:0/2276658770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 msgr2=0x7f3134104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 --2- 172.21.15.105:0/2276658770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 0x7f3134104730 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3124009900 tx=0x7f312402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 -- 172.21.15.105:0/2276658770 shutdown_connections 2024-03-26T17:41:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 --2- 172.21.15.105:0/2276658770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31341075a0 0x7f31341099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 --2- 172.21.15.105:0/2276658770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3134104c70 0x7f3134107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 --2- 172.21.15.105:0/2276658770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 0x7f3134104730 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 -- 172.21.15.105:0/2276658770 >> 172.21.15.105:0/2276658770 conn(0x7f31340fbd50 msgr2=0x7f31340fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 -- 172.21.15.105:0/2276658770 shutdown_connections 2024-03-26T17:41:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.049+0000 7f3139952640 1 -- 172.21.15.105:0/2276658770 wait complete. 2024-03-26T17:41:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.050+0000 7f3139952640 1 Processor -- start 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.050+0000 7f3139952640 1 -- start start 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3139952640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 0x7f3134197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3139952640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3134104c70 0x7f31341980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3139952640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31341075a0 0x7f313419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3139952640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f313410d420 con 0x7f3134102340 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3139952640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f313410d2a0 con 0x7f3134104c70 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3139952640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f313410d5a0 con 0x7f31341075a0 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 0x7f3134197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 0x7f3134197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52720/0 (socket says 172.21.15.105:52720) 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 -- 172.21.15.105:0/2458355090 learned_addr learned my addr 172.21.15.105:0/2458355090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f31327fc640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3134104c70 0x7f31341980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f31337fe640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31341075a0 0x7f313419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 -- 172.21.15.105:0/2458355090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31341075a0 msgr2=0x7f313419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31341075a0 0x7f313419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 -- 172.21.15.105:0/2458355090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3134104c70 msgr2=0x7f31341980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3134104c70 0x7f31341980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 -- 172.21.15.105:0/2458355090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313419f7a0 con 0x7f3134102340 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.051+0000 7f3132ffd640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 0x7f3134197b80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3124006010 tx=0x7f3124005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.052+0000 7f3138950640 1 -- 172.21.15.105:0/2458355090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124043020 con 0x7f3134102340 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.052+0000 7f3138950640 1 -- 172.21.15.105:0/2458355090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31240043d0 con 0x7f3134102340 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.052+0000 7f3139952640 1 -- 172.21.15.105:0/2458355090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313419fa30 con 0x7f3134102340 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.052+0000 7f3138950640 1 -- 172.21.15.105:0/2458355090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124003b80 con 0x7f3134102340 2024-03-26T17:41:13.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.052+0000 7f3139952640 1 -- 172.21.15.105:0/2458355090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313419ff80 con 0x7f3134102340 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.054+0000 7f3138950640 1 -- 172.21.15.105:0/2458355090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3124046020 con 0x7f3134102340 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f3138950640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f311c0a3070 0x7f311c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f313c610740 1 Processor -- start 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f313c610740 1 -- start start 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f31327fc640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f311c0a3070 0x7f311c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f3138950640 1 -- 172.21.15.105:0/2458355090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f31240f1510 con 0x7f3134102340 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f313c610740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc417260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f313c610740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc41c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f313c610740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc41ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f313c610740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5572cc377970 con 0x5572cc41c9a0 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f313c610740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5572cc31fd70 con 0x5572cc416e40 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f313c610740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5572cc373920 con 0x5572cc417830 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f31327fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc41c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f31337fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc41ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f31327fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc41c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45784/0 (socket says 172.21.15.105:45784) 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f31327fc640 1 -- 172.21.15.105:0/1524732852 learned_addr learned my addr 172.21.15.105:0/1524732852 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.055+0000 7f3132ffd640 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc417260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f31337fe640 1 -- 172.21.15.105:0/1524732852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 msgr2=0x5572cc41c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f31337fe640 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc41c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f31337fe640 1 -- 172.21.15.105:0/1524732852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 msgr2=0x5572cc417260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f31337fe640 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc417260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f31337fe640 1 -- 172.21.15.105:0/1524732852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572cc08b3f0 con 0x5572cc41c9a0 2024-03-26T17:41:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f3132ffd640 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc417260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f31327fc640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f311c0a3070 0x7f311c0a5530 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3128009fd0 tx=0x7f3128011920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f31337fe640 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc41ee30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f312c009f00 tx=0x7f312c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e63b87f903e9ee4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f3119ffb640 1 -- 172.21.15.105:0/1524732852 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c004d20 con 0x5572cc41c9a0 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f3119ffb640 1 -- 172.21.15.105:0/1524732852 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f312c003e20 con 0x5572cc41c9a0 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.056+0000 7f3119ffb640 1 -- 172.21.15.105:0/1524732852 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c005020 con 0x5572cc41c9a0 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 -- 172.21.15.105:0/1524732852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 msgr2=0x5572cc41ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc41ee30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f312c009f00 tx=0x7f312c011810 comp rx=0 tx=0).stop 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 -- 172.21.15.105:0/1524732852 shutdown_connections 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc41ee30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc41c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 --2- 172.21.15.105:0/1524732852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc417260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 -- 172.21.15.105:0/1524732852 >> 172.21.15.105:0/1524732852 conn(0x5572cc40dfa0 msgr2=0x5572cc40e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 -- 172.21.15.105:0/1524732852 shutdown_connections 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 -- 172.21.15.105:0/1524732852 wait complete. 2024-03-26T17:41:13.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.057+0000 7f313c610740 1 -- start start 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f313c610740 1 Processor -- start 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f313c610740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc421330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f313c610740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc421870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f313c610740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc4288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f313c610740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5572cc41f950 con 0x5572cc41c9a0 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f313c610740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5572cc333310 con 0x5572cc416e40 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f313c610740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5572cc41fad0 con 0x5572cc417830 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f313c610740 10 client.0 _ll_register_callbacks cb 0x5572cc36ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31337fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc4288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc421330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31327fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc421870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc421330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54470/0 (socket says 172.21.15.105:54470) 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31327fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc421870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45800/0 (socket says 172.21.15.105:45800) 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31337fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc4288b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52736/0 (socket says 172.21.15.105:52736) 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f3132ffd640 1 -- 172.21.15.105:0/3483749370 learned_addr learned my addr 172.21.15.105:0/3483749370 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31327fc640 1 -- 172.21.15.105:0/3483749370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 msgr2=0x5572cc421330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31327fc640 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc421330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31327fc640 1 -- 172.21.15.105:0/3483749370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 msgr2=0x5572cc4288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31327fc640 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc4288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.058+0000 7f31327fc640 1 -- 172.21.15.105:0/3483749370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572cc1557d0 con 0x5572cc417830 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f31327fc640 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc421870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5572cc417c30 tx=0x7f3128004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7e74163df729f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f3112ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f3112ffd640 1 -- 172.21.15.105:0/3483749370 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312801a040 con 0x5572cc417830 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 1 -- 172.21.15.105:0/3483749370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f312c003e20 con 0x5572cc417830 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f3112ffd640 1 -- 172.21.15.105:0/3483749370 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f3128003a00 con 0x5572cc417830 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f3112ffd640 1 -- 172.21.15.105:0/3483749370 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f312800fe20 con 0x5572cc417830 2024-03-26T17:41:13.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 10 client.26392 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 1 -- 172.21.15.105:0/3483749370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572cc42bce0 con 0x5572cc417830 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f3112ffd640 1 -- 172.21.15.105:0/3483749370 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f312c003e20 con 0x5572cc417830 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f3112ffd640 1 -- 172.21.15.105:0/3483749370 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f312801d030 con 0x5572cc417830 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 10 client.26392 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 20 client.26392 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572cc42d350 0x5572cc42f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 4 client.26392 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.059+0000 7f313c610740 1 -- 172.21.15.105:0/3483749370 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572cc384eb0 con 0x5572cc42d350 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.060+0000 7f3132ffd640 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572cc42d350 0x5572cc42f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.060+0000 7f3112ffd640 10 client.26392 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.060+0000 7f3132ffd640 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572cc42d350 0x5572cc42f7a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd0b6aeebd1c26a server_cookie=8d3976683a35ce77 in_seq=0 out_seq=0 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f3112ffd640 1 -- 172.21.15.105:0/3483749370 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5572cc384eb0 con 0x5572cc42d350 2024-03-26T17:41:13.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f3112ffd640 10 client.26392 handle_command_reply: tid=0 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 client.26392 shutdown 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 -- 172.21.15.105:0/3483749370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 msgr2=0x5572cc421870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc421870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5572cc417c30 tx=0x7f3128004730 comp rx=0 tx=0).stop 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 -- 172.21.15.105:0/3483749370 shutdown_connections 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572cc42d350 0x5572cc42f7a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc41c9a0 0x5572cc4288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc421870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 --2- 172.21.15.105:0/3483749370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc416e40 0x5572cc421330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f3112ffd640 0 client.26392 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 -- 172.21.15.105:0/3483749370 >> 172.21.15.105:0/3483749370 conn(0x5572cc40dfa0 msgr2=0x5572cc40d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 -- 172.21.15.105:0/3483749370 shutdown_connections 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 1 -- 172.21.15.105:0/3483749370 wait complete. 2024-03-26T17:41:13.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.061+0000 7f313c610740 20 client.26392 trim_cache size 0 max 16384 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.077+0000 7f313c610740 1 Processor -- start 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.077+0000 7f313c610740 1 -- start start 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.077+0000 7f313c610740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc409050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.077+0000 7f313c610740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc428870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.077+0000 7f313c610740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc411910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.077+0000 7f313c610740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31341a6b20 con 0x5572cc416e40 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.077+0000 7f313c610740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31341a8660 con 0x5572cc41c9a0 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.077+0000 7f313c610740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31341a71f0 con 0x5572cc417830 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc409050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc409050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52750/0 (socket says 172.21.15.105:52750) 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31327fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc428870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f3132ffd640 1 -- 172.21.15.105:0/2406795403 learned_addr learned my addr 172.21.15.105:0/2406795403 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31337fe640 1 --2- 172.21.15.105:0/2406795403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc411910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31327fc640 1 -- 172.21.15.105:0/2406795403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 msgr2=0x5572cc411910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31327fc640 1 --2- 172.21.15.105:0/2406795403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc411910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31327fc640 1 -- 172.21.15.105:0/2406795403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 msgr2=0x5572cc409050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31327fc640 1 --2- 172.21.15.105:0/2406795403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc409050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31327fc640 1 -- 172.21.15.105:0/2406795403 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572cc409820 con 0x5572cc417830 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31327fc640 1 --2- 172.21.15.105:0/2406795403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc428870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3128020f10 tx=0x7f3128009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5605dd51de1d565 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31197fa640 1 -- 172.21.15.105:0/2406795403 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128037030 con 0x5572cc417830 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31197fa640 1 -- 172.21.15.105:0/2406795403 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f312802e030 con 0x5572cc417830 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.078+0000 7f31197fa640 1 -- 172.21.15.105:0/2406795403 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312801a070 con 0x5572cc417830 2024-03-26T17:41:13.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 -- 172.21.15.105:0/2406795403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 msgr2=0x5572cc428870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 --2- 172.21.15.105:0/2406795403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc428870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3128020f10 tx=0x7f3128009e40 comp rx=0 tx=0).stop 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 -- 172.21.15.105:0/2406795403 shutdown_connections 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 --2- 172.21.15.105:0/2406795403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc411910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 --2- 172.21.15.105:0/2406795403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc417830 0x5572cc428870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 --2- 172.21.15.105:0/2406795403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc409050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 -- 172.21.15.105:0/2406795403 >> 172.21.15.105:0/2406795403 conn(0x5572cc40dfa0 msgr2=0x5572cc411200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 -- 172.21.15.105:0/2406795403 shutdown_connections 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 -- 172.21.15.105:0/2406795403 wait complete. 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.079+0000 7f313c610740 1 -- start start 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f313c610740 1 Processor -- start 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f313c610740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc428020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f313c610740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc428560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f313c610740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc426630 0x5572cc426ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f313c610740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31341a7f90 con 0x5572cc416e40 2024-03-26T17:41:13.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f313c610740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31341a78c0 con 0x5572cc41c9a0 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f313c610740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31341a4b00 con 0x5572cc426630 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f313c610740 10 client.0 _ll_register_callbacks cb 0x7f3128009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc428020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc428020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52760/0 (socket says 172.21.15.105:52760) 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f31337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc426630 0x5572cc426ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 -- 172.21.15.105:0/3596012642 learned_addr learned my addr 172.21.15.105:0/3596012642 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f31327fc640 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc428560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 -- 172.21.15.105:0/3596012642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc426630 msgr2=0x5572cc426ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc426630 0x5572cc426ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 -- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 msgr2=0x5572cc428560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:13.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc428560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 -- 172.21.15.105:0/3596012642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572cc427020 con 0x5572cc416e40 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f31327fc640 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc428560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3132ffd640 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc428020 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f31240ef340 tx=0x7f312402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee7c6ed49ccd392 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.080+0000 7f3112ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f3112ffd640 1 -- 172.21.15.105:0/3596012642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31240c6610 con 0x5572cc416e40 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f3112ffd640 1 -- 172.21.15.105:0/3596012642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f31240bbf00 con 0x5572cc416e40 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 1 -- 172.21.15.105:0/3596012642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f312802e030 con 0x5572cc416e40 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f3112ffd640 1 -- 172.21.15.105:0/3596012642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3124003d20 con 0x5572cc416e40 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 10 client.16554 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 1 -- 172.21.15.105:0/3596012642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572cc494450 con 0x5572cc416e40 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f3112ffd640 1 -- 172.21.15.105:0/3596012642 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f312802e030 con 0x5572cc416e40 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f3112ffd640 1 -- 172.21.15.105:0/3596012642 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f31240c6180 con 0x5572cc416e40 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 10 client.16554 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:13.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 20 client.16554 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572cc494dd0 0x5572cc4971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 4 client.16554 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f313c610740 1 -- 172.21.15.105:0/3596012642 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572cc37f200 con 0x5572cc494dd0 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.081+0000 7f31327fc640 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572cc494dd0 0x5572cc4971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f31327fc640 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572cc494dd0 0x5572cc4971f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56800b3c06c452ab server_cookie=32d3dacb9a3418c2 in_seq=0 out_seq=0 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f3112ffd640 10 client.16554 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f3112ffd640 1 -- 172.21.15.105:0/3596012642 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572cc37f200 con 0x5572cc494dd0 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f3112ffd640 10 client.16554 handle_command_reply: tid=0 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 client.16554 shutdown 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 -- 172.21.15.105:0/3596012642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 msgr2=0x5572cc428020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc428020 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f31240ef340 tx=0x7f312402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 -- 172.21.15.105:0/3596012642 shutdown_connections 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572cc494dd0 0x5572cc4971f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5572cc426630 0x5572cc426ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f3112ffd640 0 client.16554 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5572cc41c9a0 0x5572cc428560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 --2- 172.21.15.105:0/3596012642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5572cc416e40 0x5572cc428020 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 -- 172.21.15.105:0/3596012642 >> 172.21.15.105:0/3596012642 conn(0x5572cc40dfa0 msgr2=0x5572cc411a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 -- 172.21.15.105:0/3596012642 shutdown_connections 2024-03-26T17:41:13.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.082+0000 7f313c610740 1 -- 172.21.15.105:0/3596012642 wait complete. 2024-03-26T17:41:13.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f313c610740 20 client.16554 trim_cache size 0 max 16384 2024-03-26T17:41:13.084 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 -- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f311c0a3070 msgr2=0x7f311c0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f311c0a3070 0x7f311c0a5530 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3128009fd0 tx=0x7f3128011920 comp rx=0 tx=0).stop 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 -- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 msgr2=0x7f3134197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 0x7f3134197b80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3124006010 tx=0x7f3124005e40 comp rx=0 tx=0).stop 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 -- 172.21.15.105:0/2458355090 shutdown_connections 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f311c0a3070 0x7f311c0a5530 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31341075a0 0x7f313419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3134104c70 0x7f31341980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 --2- 172.21.15.105:0/2458355090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3134102340 0x7f3134197b80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 -- 172.21.15.105:0/2458355090 >> 172.21.15.105:0/2458355090 conn(0x7f31340fbd50 msgr2=0x7f31341041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:13.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 -- 172.21.15.105:0/2458355090 shutdown_connections 2024-03-26T17:41:13.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:13.083+0000 7f3139952640 1 -- 172.21.15.105:0/2458355090 wait complete. 2024-03-26T17:41:13.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:12 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3586375285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:13.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:12 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3586375285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:14.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 69526 3.33 MB/sec warmup 117 sec latency 15.592 ms 2024-03-26T17:41:14.089 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:41:14.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:41:14.089 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:41:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:13 smithi144 ceph-mon[36095]: pgmap v434: 97 pgs: 97 active+clean; 27 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-26T17:41:14.115 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:13 smithi105 ceph-mon[32280]: pgmap v434: 97 pgs: 97 active+clean; 27 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-26T17:41:14.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:13 smithi135 ceph-mon[35636]: pgmap v434: 97 pgs: 97 active+clean; 27 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-03-26T17:41:14.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 -- 172.21.15.105:0/1867404079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 msgr2=0x7fc6a01099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/1867404079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 0x7fc6a01099f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc698009ca0 tx=0x7fc69802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 -- 172.21.15.105:0/1867404079 shutdown_connections 2024-03-26T17:41:14.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/1867404079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 0x7fc6a01099f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/1867404079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a0104cd0 0x7fc6a01070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/1867404079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a01023a0 0x7fc6a0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 -- 172.21.15.105:0/1867404079 >> 172.21.15.105:0/1867404079 conn(0x7fc6a00fbd50 msgr2=0x7fc6a00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 -- 172.21.15.105:0/1867404079 shutdown_connections 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 -- 172.21.15.105:0/1867404079 wait complete. 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.222+0000 7fc6a5c26640 1 Processor -- start 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc6a5c26640 1 -- start start 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc6a5c26640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a01023a0 0x7fc6a0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc6a5c26640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a0104cd0 0x7fc6a0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc6a5c26640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 0x7fc6a019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc6a5c26640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a010d410 con 0x7fc6a0107600 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc6a5c26640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a010d290 con 0x7fc6a01023a0 2024-03-26T17:41:14.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc6a5c26640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a010d590 con 0x7fc6a0104cd0 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 0x7fc6a019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 0x7fc6a019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52772/0 (socket says 172.21.15.105:52772) 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 -- 172.21.15.105:0/3251930875 learned_addr learned my addr 172.21.15.105:0/3251930875 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a01023a0 0x7fc6a0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69effd640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a0104cd0 0x7fc6a0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 -- 172.21.15.105:0/3251930875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a0104cd0 msgr2=0x7fc6a0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a0104cd0 0x7fc6a0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 -- 172.21.15.105:0/3251930875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a01023a0 msgr2=0x7fc6a0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a01023a0 0x7fc6a0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 -- 172.21.15.105:0/3251930875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a019f760 con 0x7fc6a0107600 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.223+0000 7fc69ffff640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 0x7fc6a019f0f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc69802c700 tx=0x7fc6980328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.224+0000 7fc69cff9640 1 -- 172.21.15.105:0/3251930875 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698041020 con 0x7fc6a0107600 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.224+0000 7fc69cff9640 1 -- 172.21.15.105:0/3251930875 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc698003e20 con 0x7fc6a0107600 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.224+0000 7fc6a5c26640 1 -- 172.21.15.105:0/3251930875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a0100190 con 0x7fc6a0107600 2024-03-26T17:41:14.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.224+0000 7fc69cff9640 1 -- 172.21.15.105:0/3251930875 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698039040 con 0x7fc6a0107600 2024-03-26T17:41:14.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.224+0000 7fc6a5c26640 1 -- 172.21.15.105:0/3251930875 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a01006d0 con 0x7fc6a0107600 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.226+0000 7fc69cff9640 1 -- 172.21.15.105:0/3251930875 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc698044020 con 0x7fc6a0107600 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc6a880b740 1 Processor -- start 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69cff9640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6880a2fa0 0x7fc6880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc6a880b740 1 -- start start 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69effd640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6880a2fa0 0x7fc6880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de1e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69cff9640 1 -- 172.21.15.105:0/3251930875 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc6980f1540 con 0x7fc6a0107600 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de21180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc6a880b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ab6dd7e630 con 0x55ab6de19b80 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc6a880b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ab6dd26e10 con 0x55ab6de1ecf0 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc6a880b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab6dd7a5e0 con 0x55ab6de19190 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de1e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de195b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45842/0 (socket says 172.21.15.105:45842) 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de1e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52778/0 (socket says 172.21.15.105:52778) 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69f7fe640 1 -- 172.21.15.105:0/3445096944 learned_addr learned my addr 172.21.15.105:0/3445096944 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69f7fe640 1 -- 172.21.15.105:0/3445096944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 msgr2=0x55ab6de21180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/3445096944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de21180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69f7fe640 1 -- 172.21.15.105:0/3445096944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 msgr2=0x55ab6de1e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/3445096944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de1e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69f7fe640 1 -- 172.21.15.105:0/3445096944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6dd3d740 con 0x55ab6de19190 2024-03-26T17:41:14.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69ffff640 1 --2- 172.21.15.105:0/3445096944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de21180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.227+0000 7fc69effd640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6880a2fa0 0x7fc6880a5460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0199050 tx=0x7fc694009af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.228+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/3445096944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de195b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc690009cf0 tx=0x7fc690011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a98beb5d9c0acbb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.228+0000 7fc685ffb640 1 -- 172.21.15.105:0/3445096944 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690004230 con 0x55ab6de19190 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.228+0000 7fc685ffb640 1 -- 172.21.15.105:0/3445096944 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6900043d0 con 0x55ab6de19190 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.228+0000 7fc685ffb640 1 -- 172.21.15.105:0/3445096944 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6900197c0 con 0x55ab6de19190 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- 172.21.15.105:0/3445096944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 msgr2=0x55ab6de195b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3445096944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de195b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc690009cf0 tx=0x7fc690011810 comp rx=0 tx=0).stop 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- 172.21.15.105:0/3445096944 shutdown_connections 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3445096944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de21180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3445096944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de1e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3445096944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de195b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- 172.21.15.105:0/3445096944 >> 172.21.15.105:0/3445096944 conn(0x55ab6de14700 msgr2=0x55ab6de14b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- 172.21.15.105:0/3445096944 shutdown_connections 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- 172.21.15.105:0/3445096944 wait complete. 2024-03-26T17:41:14.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- start start 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 Processor -- start 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19190 0x55ab6de23660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19b80 0x55ab6de23ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de2abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ab6de21ca0 con 0x55ab6de1ecf0 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ab6dd3a1a0 con 0x55ab6de19190 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.229+0000 7fc6a880b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ab6de21e20 con 0x55ab6de19b80 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc6a880b740 10 client.0 _ll_register_callbacks cb 0x55ab6da91c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de2abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19b80 0x55ab6de23ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de2abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52786/0 (socket says 172.21.15.105:52786) 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69ffff640 1 -- 172.21.15.105:0/64777793 learned_addr learned my addr 172.21.15.105:0/64777793 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19190 0x55ab6de23660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69effd640 1 -- 172.21.15.105:0/64777793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19190 msgr2=0x55ab6de23660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69effd640 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19190 0x55ab6de23660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69effd640 1 -- 172.21.15.105:0/64777793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 msgr2=0x55ab6de2abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69effd640 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de2abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69effd640 1 -- 172.21.15.105:0/64777793 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6db5a7e0 con 0x55ab6de19b80 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69ffff640 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de2abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc69effd640 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19b80 0x55ab6de23ba0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ab6de1a5f0 tx=0x7fc694004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7309a7b9319fe4b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc67effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc67effd640 1 -- 172.21.15.105:0/64777793 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69401fa50 con 0x55ab6de19b80 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc67effd640 1 -- 172.21.15.105:0/64777793 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc694019070 con 0x55ab6de19b80 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.230+0000 7fc6a880b740 1 -- 172.21.15.105:0/64777793 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6900043d0 con 0x55ab6de19b80 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc67effd640 1 -- 172.21.15.105:0/64777793 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc69401fbf0 con 0x55ab6de19b80 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc6a880b740 10 client.26410 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc6a880b740 1 -- 172.21.15.105:0/64777793 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ab6dbbaf70 con 0x55ab6de19b80 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc67effd640 1 -- 172.21.15.105:0/64777793 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6900043d0 con 0x55ab6de19b80 2024-03-26T17:41:14.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc67effd640 1 -- 172.21.15.105:0/64777793 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc694004830 con 0x55ab6de19b80 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc6a880b740 10 client.26410 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc6a880b740 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc6a880b740 20 client.26410 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc6a880b740 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ab6de2f580 0x55ab6de31a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc6a880b740 4 client.26410 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc6a880b740 1 -- 172.21.15.105:0/64777793 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab6dd8bb70 con 0x55ab6de2f580 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.231+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ab6de2f580 0x55ab6de31a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.232+0000 7fc67effd640 10 client.26410 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:14.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.232+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ab6de2f580 0x55ab6de31a00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df1a909915bbdbb server_cookie=6b2d30f49b1efa5b in_seq=0 out_seq=0 2024-03-26T17:41:14.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc67effd640 1 -- 172.21.15.105:0/64777793 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ab6dd8bb70 con 0x55ab6de2f580 2024-03-26T17:41:14.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc67effd640 10 client.26410 handle_command_reply: tid=0 2024-03-26T17:41:14.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 client.26410 shutdown 2024-03-26T17:41:14.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 -- 172.21.15.105:0/64777793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19b80 msgr2=0x55ab6de23ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19b80 0x55ab6de23ba0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ab6de1a5f0 tx=0x7fc694004730 comp rx=0 tx=0).stop 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 -- 172.21.15.105:0/64777793 shutdown_connections 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ab6de2f580 0x55ab6de31a00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de2abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19b80 0x55ab6de23ba0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 --2- 172.21.15.105:0/64777793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19190 0x55ab6de23660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc67effd640 0 client.26410 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 -- 172.21.15.105:0/64777793 >> 172.21.15.105:0/64777793 conn(0x55ab6de14700 msgr2=0x55ab6de13900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 -- 172.21.15.105:0/64777793 shutdown_connections 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.233+0000 7fc6a880b740 1 -- 172.21.15.105:0/64777793 wait complete. 2024-03-26T17:41:14.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.234+0000 7fc6a880b740 20 client.26410 trim_cache size 0 max 16384 2024-03-26T17:41:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc6a880b740 1 Processor -- start 2024-03-26T17:41:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc6a880b740 1 -- start start 2024-03-26T17:41:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de13070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc6a880b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a01a6c60 con 0x55ab6de19b80 2024-03-26T17:41:14.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc6a880b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a01a87a0 con 0x55ab6de1ecf0 2024-03-26T17:41:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc6a880b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a01a7330 con 0x55ab6de19190 2024-03-26T17:41:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de13070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de2aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52792/0 (socket says 172.21.15.105:52792) 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69effd640 1 -- 172.21.15.105:0/621696770 learned_addr learned my addr 172.21.15.105:0/621696770 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69ffff640 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69effd640 1 -- 172.21.15.105:0/621696770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 msgr2=0x55ab6de13070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69effd640 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de13070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69effd640 1 -- 172.21.15.105:0/621696770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 msgr2=0x55ab6de135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69effd640 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de135b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69effd640 1 -- 172.21.15.105:0/621696770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6da52170 con 0x55ab6de19b80 2024-03-26T17:41:14.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.253+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de13070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.254+0000 7fc69effd640 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de2aba0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc694000c00 tx=0x7fc694004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904bd87805b45a3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.254+0000 7fc6857fa640 1 -- 172.21.15.105:0/621696770 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69401fa50 con 0x55ab6de19b80 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.254+0000 7fc6857fa640 1 -- 172.21.15.105:0/621696770 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc694010c90 con 0x55ab6de19b80 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.254+0000 7fc6857fa640 1 -- 172.21.15.105:0/621696770 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69400b720 con 0x55ab6de19b80 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.254+0000 7fc6a880b740 1 -- 172.21.15.105:0/621696770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 msgr2=0x55ab6de2aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.254+0000 7fc6a880b740 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de2aba0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc694000c00 tx=0x7fc694004f60 comp rx=0 tx=0).stop 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 -- 172.21.15.105:0/621696770 shutdown_connections 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de1ecf0 0x55ab6de135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de19b80 0x55ab6de2aba0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 --2- 172.21.15.105:0/621696770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de13070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 -- 172.21.15.105:0/621696770 >> 172.21.15.105:0/621696770 conn(0x55ab6de14700 msgr2=0x55ab6de23660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 -- 172.21.15.105:0/621696770 shutdown_connections 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 -- 172.21.15.105:0/621696770 wait complete. 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 -- start start 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 Processor -- start 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de27ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19b80 0x55ab6de28420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de29430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a01a80d0 con 0x55ab6de1ecf0 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a01a7a00 con 0x55ab6de19b80 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a01a4c40 con 0x55ab6de19190 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc6a880b740 10 client.0 _ll_register_callbacks cb 0x7fc69400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc69ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de29430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc69ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de29430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52804/0 (socket says 172.21.15.105:52804) 2024-03-26T17:41:14.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.255+0000 7fc69ffff640 1 -- 172.21.15.105:0/3061098055 learned_addr learned my addr 172.21.15.105:0/3061098055 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc69ffff640 1 -- 172.21.15.105:0/3061098055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 msgr2=0x55ab6de27ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc69ffff640 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de27ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc69ffff640 1 -- 172.21.15.105:0/3061098055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19b80 msgr2=0x55ab6de28420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc69ffff640 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19b80 0x55ab6de28420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc69ffff640 1 -- 172.21.15.105:0/3061098055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6de29a20 con 0x55ab6de1ecf0 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc69ffff640 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de29430 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc6980bdbc0 tx=0x7fc698009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c4db597129f338 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc67effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc67effd640 1 -- 172.21.15.105:0/3061098055 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6980f1310 con 0x55ab6de1ecf0 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc67effd640 1 -- 172.21.15.105:0/3061098055 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc6980c61b0 con 0x55ab6de1ecf0 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc6a880b740 1 -- 172.21.15.105:0/3061098055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc694010c90 con 0x55ab6de1ecf0 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc67effd640 1 -- 172.21.15.105:0/3061098055 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc6980c1d80 con 0x55ab6de1ecf0 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc6a880b740 10 client.16572 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc6a880b740 1 -- 172.21.15.105:0/3061098055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ab6de2ae30 con 0x55ab6de1ecf0 2024-03-26T17:41:14.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.256+0000 7fc67effd640 1 -- 172.21.15.105:0/3061098055 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc694010c90 con 0x55ab6de1ecf0 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc67effd640 1 -- 172.21.15.105:0/3061098055 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc69803c030 con 0x55ab6de1ecf0 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc6a880b740 10 client.16572 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc6a880b740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc6a880b740 20 client.16572 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ab6de96020 0x55ab6de98460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc6a880b740 4 client.16572 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc6a880b740 1 -- 172.21.15.105:0/3061098055 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab6dd85ec0 con 0x55ab6de96020 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ab6de96020 0x55ab6de98460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc69f7fe640 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ab6de96020 0x55ab6de98460 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6919cc1a9510c7b server_cookie=67c1256c53fcc7e3 in_seq=0 out_seq=0 2024-03-26T17:41:14.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.257+0000 7fc67effd640 10 client.16572 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc67effd640 1 -- 172.21.15.105:0/3061098055 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ab6dd85ec0 con 0x55ab6de96020 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc67effd640 10 client.16572 handle_command_reply: tid=0 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 client.16572 shutdown 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 -- 172.21.15.105:0/3061098055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 msgr2=0x55ab6de29430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de29430 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc6980bdbc0 tx=0x7fc698009980 comp rx=0 tx=0).stop 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 -- 172.21.15.105:0/3061098055 shutdown_connections 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ab6de96020 0x55ab6de98460 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ab6de1ecf0 0x55ab6de29430 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab6de19b80 0x55ab6de28420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc67effd640 0 client.16572 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 --2- 172.21.15.105:0/3061098055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ab6de19190 0x55ab6de27ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 -- 172.21.15.105:0/3061098055 >> 172.21.15.105:0/3061098055 conn(0x55ab6de14700 msgr2=0x55ab6de13070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 -- 172.21.15.105:0/3061098055 shutdown_connections 2024-03-26T17:41:14.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 1 -- 172.21.15.105:0/3061098055 wait complete. 2024-03-26T17:41:14.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.263+0000 7fc6a880b740 20 client.16572 trim_cache size 0 max 16384 2024-03-26T17:41:14.260 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:41:14.260 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:41:14.260 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "62af83b1-ed46-49fb-a969-908c6fb1ed2b", 2024-03-26T17:41:14.260 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:41:14.260 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:41:14.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 -- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6880a2fa0 msgr2=0x7fc6880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6880a2fa0 0x7fc6880a5460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0199050 tx=0x7fc694009af0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 -- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 msgr2=0x7fc6a019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 0x7fc6a019f0f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc69802c700 tx=0x7fc6980328a0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 -- 172.21.15.105:0/3251930875 shutdown_connections 2024-03-26T17:41:14.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6880a2fa0 0x7fc6880a5460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a0107600 0x7fc6a019f0f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a0104cd0 0x7fc6a0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 --2- 172.21.15.105:0/3251930875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a01023a0 0x7fc6a0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 -- 172.21.15.105:0/3251930875 >> 172.21.15.105:0/3251930875 conn(0x7fc6a00fbd50 msgr2=0x7fc6a0102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 -- 172.21.15.105:0/3251930875 shutdown_connections 2024-03-26T17:41:14.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.264+0000 7fc6a5c26640 1 -- 172.21.15.105:0/3251930875 wait complete. 2024-03-26T17:41:14.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:41:14.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:41:14.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 -- 172.21.15.105:0/1443975250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c102b50 msgr2=0x7faf5c102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 --2- 172.21.15.105:0/1443975250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c102b50 0x7faf5c102fd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009930 tx=0x7faf4c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:14.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 -- 172.21.15.105:0/1443975250 shutdown_connections 2024-03-26T17:41:14.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 --2- 172.21.15.105:0/1443975250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf5c103510 0x7faf5c109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 --2- 172.21.15.105:0/1443975250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c102b50 0x7faf5c102fd0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 --2- 172.21.15.105:0/1443975250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf5c0ffb80 0x7faf5c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 -- 172.21.15.105:0/1443975250 >> 172.21.15.105:0/1443975250 conn(0x7faf5c0fb4b0 msgr2=0x7faf5c0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 -- 172.21.15.105:0/1443975250 shutdown_connections 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 -- 172.21.15.105:0/1443975250 wait complete. 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 Processor -- start 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.434+0000 7faf63cc6640 1 -- start start 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf63cc6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c0ffb80 0x7faf5c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf63cc6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf5c102b50 0x7faf5c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf63cc6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf5c103510 0x7faf5c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf63cc6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c10d410 con 0x7faf5c0ffb80 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf63cc6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c10d290 con 0x7faf5c103510 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf63cc6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c10d590 con 0x7faf5c102b50 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c0ffb80 0x7faf5c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c0ffb80 0x7faf5c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52832/0 (socket says 172.21.15.105:52832) 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 -- 172.21.15.105:0/3091150558 learned_addr learned my addr 172.21.15.105:0/3091150558 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61261640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf5c102b50 0x7faf5c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf62263640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf5c103510 0x7faf5c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 -- 172.21.15.105:0/3091150558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf5c102b50 msgr2=0x7faf5c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf5c102b50 0x7faf5c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 -- 172.21.15.105:0/3091150558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf5c103510 msgr2=0x7faf5c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf5c103510 0x7faf5c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 -- 172.21.15.105:0/3091150558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5c19f7b0 con 0x7faf5c0ffb80 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf61a62640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c0ffb80 0x7faf5c197b90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e9c0 tx=0x7faf5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf52ffd640 1 -- 172.21.15.105:0/3091150558 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5800cd80 con 0x7faf5c0ffb80 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf52ffd640 1 -- 172.21.15.105:0/3091150558 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf580035f0 con 0x7faf5c0ffb80 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf63cc6640 1 -- 172.21.15.105:0/3091150558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5c19faa0 con 0x7faf5c0ffb80 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf52ffd640 1 -- 172.21.15.105:0/3091150558 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58010690 con 0x7faf5c0ffb80 2024-03-26T17:41:14.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.435+0000 7faf63cc6640 1 -- 172.21.15.105:0/3091150558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5c19ff70 con 0x7faf5c0ffb80 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.438+0000 7faf52ffd640 1 -- 172.21.15.105:0/3091150558 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7faf580037a0 con 0x7faf5c0ffb80 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.438+0000 7faf66991740 1 Processor -- start 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf66991740 1 -- start start 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf66991740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c18f0 0x562c649c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf66991740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c22e0 0x562c649c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf66991740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c7450 0x562c649c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf52ffd640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf480a2f20 0x7faf480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf66991740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562c64926980 con 0x562c649c18f0 2024-03-26T17:41:14.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf66991740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562c648cebe0 con 0x562c649c22e0 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf66991740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562c64922930 con 0x562c649c7450 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf61261640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf480a2f20 0x7faf480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf62263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c18f0 0x562c649c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf62263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c18f0 0x562c649c1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52846/0 (socket says 172.21.15.105:52846) 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf62263640 1 -- 172.21.15.105:0/2689593684 learned_addr learned my addr 172.21.15.105:0/2689593684 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf61a62640 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c22e0 0x562c649c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf62263640 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c7450 0x562c649c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf52ffd640 1 -- 172.21.15.105:0/3091150558 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7faf580cf900 con 0x7faf5c0ffb80 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf61a62640 1 -- 172.21.15.105:0/2689593684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c7450 msgr2=0x562c649c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf61a62640 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c7450 0x562c649c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf61a62640 1 -- 172.21.15.105:0/2689593684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c18f0 msgr2=0x562c649c1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.439+0000 7faf61a62640 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c18f0 0x562c649c1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.440+0000 7faf61a62640 1 -- 172.21.15.105:0/2689593684 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c647047d0 con 0x562c649c22e0 2024-03-26T17:41:14.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.440+0000 7faf62263640 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c18f0 0x562c649c1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.440+0000 7faf61261640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf480a2f20 0x7faf480a53e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009900 tx=0x7faf4c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.440+0000 7faf61a62640 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c22e0 0x562c649c6de0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf5800c6a0 tx=0x7faf5809be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7f0342c765bb64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.440+0000 7faf47fff640 1 -- 172.21.15.105:0/2689593684 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf580b5320 con 0x562c649c22e0 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.440+0000 7faf47fff640 1 -- 172.21.15.105:0/2689593684 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf58003b10 con 0x562c649c22e0 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.440+0000 7faf47fff640 1 -- 172.21.15.105:0/2689593684 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf580b7c70 con 0x562c649c22e0 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 -- 172.21.15.105:0/2689593684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c22e0 msgr2=0x562c649c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c22e0 0x562c649c6de0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf5800c6a0 tx=0x7faf5809be80 comp rx=0 tx=0).stop 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 -- 172.21.15.105:0/2689593684 shutdown_connections 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c7450 0x562c649c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c22e0 0x562c649c6de0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 --2- 172.21.15.105:0/2689593684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c18f0 0x562c649c1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 -- 172.21.15.105:0/2689593684 >> 172.21.15.105:0/2689593684 conn(0x562c649bce60 msgr2=0x562c649bd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 -- 172.21.15.105:0/2689593684 shutdown_connections 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 -- 172.21.15.105:0/2689593684 wait complete. 2024-03-26T17:41:14.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 -- start start 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.441+0000 7faf66991740 1 Processor -- start 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf66991740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf66991740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf66991740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c649d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf66991740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562c649ca400 con 0x562c649c7450 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf66991740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562c648e2160 con 0x562c649c18f0 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf66991740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562c649ca580 con 0x562c649c22e0 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf66991740 10 client.0 _ll_register_callbacks cb 0x562c6491e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c649d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c649d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52856/0 (socket says 172.21.15.105:52856) 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf61a62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 -- 172.21.15.105:0/3423243291 learned_addr learned my addr 172.21.15.105:0/3423243291 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf61261640 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 -- 172.21.15.105:0/3423243291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 msgr2=0x562c649cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 -- 172.21.15.105:0/3423243291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 msgr2=0x562c649cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 -- 172.21.15.105:0/3423243291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6463a3f0 con 0x562c649c7450 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf61a62640 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf62263640 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c649d3090 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faf54011e50 tx=0x7faf54009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0f1288e6d7d716 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf457fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf457fa640 1 -- 172.21.15.105:0/3423243291 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54003970 con 0x562c649c7450 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf457fa640 1 -- 172.21.15.105:0/3423243291 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7faf54003b10 con 0x562c649c7450 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.442+0000 7faf66991740 1 -- 172.21.15.105:0/3423243291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562c64933ec0 con 0x562c649c7450 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf457fa640 1 -- 172.21.15.105:0/3423243291 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf54003d10 con 0x562c649c7450 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf66991740 10 client.16584 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf66991740 1 -- 172.21.15.105:0/3423243291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c645fa0f0 con 0x562c649c7450 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf457fa640 1 -- 172.21.15.105:0/3423243291 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562c64933ec0 con 0x562c649c7450 2024-03-26T17:41:14.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf457fa640 1 -- 172.21.15.105:0/3423243291 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7faf5401d070 con 0x562c649c7450 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf66991740 10 client.16584 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf66991740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf66991740 20 client.16584 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf66991740 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562c649d7930 0x562c649d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf66991740 4 client.16584 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf66991740 1 -- 172.21.15.105:0/3423243291 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c6492e210 con 0x562c649d7930 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.443+0000 7faf61a62640 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562c649d7930 0x562c649d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.444+0000 7faf61a62640 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562c649d7930 0x562c649d9d80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830e37b6a8a0d03a server_cookie=f84236f525475441 in_seq=0 out_seq=0 2024-03-26T17:41:14.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.444+0000 7faf457fa640 10 client.16584 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:14.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf457fa640 1 -- 172.21.15.105:0/3423243291 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562c6492e210 con 0x562c649d7930 2024-03-26T17:41:14.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf457fa640 10 client.16584 handle_command_reply: tid=0 2024-03-26T17:41:14.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 client.16584 shutdown 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 -- 172.21.15.105:0/3423243291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 msgr2=0x562c649d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c649d3090 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faf54011e50 tx=0x7faf54009f50 comp rx=0 tx=0).stop 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 -- 172.21.15.105:0/3423243291 shutdown_connections 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562c649d7930 0x562c649d9d80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf457fa640 0 client.16584 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c649d3090 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 --2- 172.21.15.105:0/3423243291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 -- 172.21.15.105:0/3423243291 >> 172.21.15.105:0/3423243291 conn(0x562c649bce60 msgr2=0x562c649bc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 -- 172.21.15.105:0/3423243291 shutdown_connections 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 1 -- 172.21.15.105:0/3423243291 wait complete. 2024-03-26T17:41:14.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.445+0000 7faf66991740 20 client.16584 trim_cache size 0 max 16384 2024-03-26T17:41:14.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.465+0000 7faf66991740 1 Processor -- start 2024-03-26T17:41:14.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.465+0000 7faf66991740 1 -- start start 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.465+0000 7faf66991740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.465+0000 7faf66991740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c22e0 0x562c649d3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.465+0000 7faf66991740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c7450 0x562c649cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.465+0000 7faf66991740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1a8e30 con 0x562c649c22e0 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.465+0000 7faf66991740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1aa970 con 0x562c649c7450 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.465+0000 7faf66991740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1a9500 con 0x562c649c18f0 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf61261640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c22e0 0x562c649d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf61261640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c22e0 0x562c649d3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52858/0 (socket says 172.21.15.105:52858) 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf61261640 1 -- 172.21.15.105:0/3299355976 learned_addr learned my addr 172.21.15.105:0/3299355976 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf62263640 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c7450 0x562c649cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf61a62640 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf62263640 1 -- 172.21.15.105:0/3299355976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c18f0 msgr2=0x562c649cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf62263640 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf62263640 1 -- 172.21.15.105:0/3299355976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c22e0 msgr2=0x562c649d3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf62263640 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c22e0 0x562c649d3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf62263640 1 -- 172.21.15.105:0/3299355976 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c649c26c0 con 0x562c649c7450 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf61261640 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c22e0 0x562c649d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf62263640 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c7450 0x562c649cc050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faf54006010 tx=0x7faf54009de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15d7a88dabcedd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf477fe640 1 -- 172.21.15.105:0/3299355976 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54003970 con 0x562c649c7450 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf477fe640 1 -- 172.21.15.105:0/3299355976 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf5402e070 con 0x562c649c7450 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.466+0000 7faf477fe640 1 -- 172.21.15.105:0/3299355976 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54004b20 con 0x562c649c7450 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 -- 172.21.15.105:0/3299355976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c7450 msgr2=0x562c649cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c7450 0x562c649cc050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faf54006010 tx=0x7faf54009de0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 -- 172.21.15.105:0/3299355976 shutdown_connections 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c7450 0x562c649cc050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c22e0 0x562c649d3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 --2- 172.21.15.105:0/3299355976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c18f0 0x562c649cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 -- 172.21.15.105:0/3299355976 >> 172.21.15.105:0/3299355976 conn(0x562c649bce60 msgr2=0x562c649bd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 -- 172.21.15.105:0/3299355976 shutdown_connections 2024-03-26T17:41:14.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 -- 172.21.15.105:0/3299355976 wait complete. 2024-03-26T17:41:14.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 -- start start 2024-03-26T17:41:14.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.467+0000 7faf66991740 1 Processor -- start 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf66991740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf66991740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf66991740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c64a3bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf66991740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1aa2a0 con 0x562c649c7450 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf66991740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1a9bd0 con 0x562c649c18f0 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf66991740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1a6e10 con 0x562c649c22e0 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf66991740 10 client.0 _ll_register_callbacks cb 0x7faf54011530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c64a3bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf61a62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c64a3bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52870/0 (socket says 172.21.15.105:52870) 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 -- 172.21.15.105:0/412108906 learned_addr learned my addr 172.21.15.105:0/412108906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf61261640 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 -- 172.21.15.105:0/412108906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 msgr2=0x562c649d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 -- 172.21.15.105:0/412108906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 msgr2=0x562c649cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 -- 172.21.15.105:0/412108906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c64a3c1b0 con 0x562c649c7450 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf61a62640 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf61261640 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf62263640 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c64a3bc70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faf54003cc0 tx=0x7faf540116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273606d07b1b554a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.468+0000 7faf457fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf457fa640 1 -- 172.21.15.105:0/412108906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54037030 con 0x562c649c7450 2024-03-26T17:41:14.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 1 -- 172.21.15.105:0/412108906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf5402e070 con 0x562c649c7450 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf457fa640 1 -- 172.21.15.105:0/412108906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7faf54011230 con 0x562c649c7450 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf457fa640 1 -- 172.21.15.105:0/412108906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf54011c20 con 0x562c649c7450 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 10 client.16590 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 1 -- 172.21.15.105:0/412108906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c64a3d3d0 con 0x562c649c7450 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf457fa640 1 -- 172.21.15.105:0/412108906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf5402e070 con 0x562c649c7450 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf457fa640 1 -- 172.21.15.105:0/412108906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7faf5401d040 con 0x562c649c7450 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 10 client.16590 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 20 client.16590 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562c649d7930 0x562c64a3fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 4 client.16590 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf66991740 1 -- 172.21.15.105:0/412108906 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c648ab540 con 0x562c649d7930 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.469+0000 7faf61a62640 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562c649d7930 0x562c64a3fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.470+0000 7faf457fa640 10 client.16590 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:14.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.470+0000 7faf61a62640 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562c649d7930 0x562c64a3fd00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c61a72beccd848 server_cookie=1d3bf2ef20a5275f in_seq=0 out_seq=0 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf457fa640 1 -- 172.21.15.105:0/412108906 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562c648ab540 con 0x562c649d7930 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf457fa640 10 client.16590 handle_command_reply: tid=0 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 client.16590 shutdown 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 -- 172.21.15.105:0/412108906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 msgr2=0x562c64a3bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c64a3bc70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faf54003cc0 tx=0x7faf540116e0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 -- 172.21.15.105:0/412108906 shutdown_connections 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562c649d7930 0x562c64a3fd00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c649c7450 0x562c64a3bc70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf457fa640 0 client.16590 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c649c22e0 0x562c649d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 --2- 172.21.15.105:0/412108906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c649c18f0 0x562c649cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 -- 172.21.15.105:0/412108906 >> 172.21.15.105:0/412108906 conn(0x562c649bce60 msgr2=0x562c649c3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 -- 172.21.15.105:0/412108906 shutdown_connections 2024-03-26T17:41:14.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 1 -- 172.21.15.105:0/412108906 wait complete. 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.471+0000 7faf66991740 20 client.16590 trim_cache size 0 max 16384 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout: "62af83b1-ed46-49fb-a969-908c6fb1ed2b": { 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "62af83b1-ed46-49fb-a969-908c6fb1ed2b", 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:41:14.471 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:41:14.472 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:41:14.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 -- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf480a2f20 msgr2=0x7faf480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf480a2f20 0x7faf480a53e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009900 tx=0x7faf4c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 -- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c0ffb80 msgr2=0x7faf5c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c0ffb80 0x7faf5c197b90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e9c0 tx=0x7faf5800ee80 comp rx=0 tx=0).stop 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 -- 172.21.15.105:0/3091150558 shutdown_connections 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf480a2f20 0x7faf480a53e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf5c103510 0x7faf5c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf5c102b50 0x7faf5c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 --2- 172.21.15.105:0/3091150558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf5c0ffb80 0x7faf5c197b90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 -- 172.21.15.105:0/3091150558 >> 172.21.15.105:0/3091150558 conn(0x7faf5c0fb4b0 msgr2=0x7faf5c105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 -- 172.21.15.105:0/3091150558 shutdown_connections 2024-03-26T17:41:14.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:14.472+0000 7faf63cc6640 1 -- 172.21.15.105:0/3091150558 wait complete. 2024-03-26T17:41:14.476 INFO:tasks.cephfs.filesystem:scrub status for tag:62af83b1-ed46-49fb-a969-908c6fb1ed2b - {'path': '/', 'tag': '62af83b1-ed46-49fb-a969-908c6fb1ed2b', 'options': 'recursive,force'} 2024-03-26T17:41:15.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 70178 3.35 MB/sec warmup 118 sec latency 22.785 ms 2024-03-26T17:41:15.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:14 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:41:15.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:15.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:41:15.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:14 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:41:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:41:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:14 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:41:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:41:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:16.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 70767 3.35 MB/sec warmup 119 sec latency 16.417 ms 2024-03-26T17:41:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:15 smithi144 ceph-mon[36095]: pgmap v435: 97 pgs: 97 active+clean; 29 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:41:16.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:15 smithi105 ceph-mon[32280]: pgmap v435: 97 pgs: 97 active+clean; 29 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:41:16.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:15 smithi135 ceph-mon[35636]: pgmap v435: 97 pgs: 97 active+clean; 29 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:41:17.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:16 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:41:17.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:16 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:41:17.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:16 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:41:18.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 71796 0.54 MB/sec execute 1 sec latency 20.643 ms 2024-03-26T17:41:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:17 smithi144 ceph-mon[36095]: pgmap v436: 97 pgs: 97 active+clean; 30 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:41:18.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:17 smithi105 ceph-mon[32280]: pgmap v436: 97 pgs: 97 active+clean; 30 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:41:18.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:17 smithi135 ceph-mon[35636]: pgmap v436: 97 pgs: 97 active+clean; 30 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:41:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.811+0000 7f2cbf71b640 1 -- 172.21.15.105:0/1020005190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb81023a0 msgr2=0x7f2cb8104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.811+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/1020005190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb81023a0 0x7f2cb8104790 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009900 tx=0x7f2cb402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.811+0000 7f2cbf71b640 1 -- 172.21.15.105:0/1020005190 shutdown_connections 2024-03-26T17:41:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.811+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/1020005190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8107600 0x7f2cb8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.811+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/1020005190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb8104cd0 0x7f2cb81070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.811+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/1020005190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb81023a0 0x7f2cb8104790 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.811+0000 7f2cbf71b640 1 -- 172.21.15.105:0/1020005190 >> 172.21.15.105:0/1020005190 conn(0x7f2cb80fbd50 msgr2=0x7f2cb80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.812+0000 7f2cbf71b640 1 -- 172.21.15.105:0/1020005190 shutdown_connections 2024-03-26T17:41:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.812+0000 7f2cbf71b640 1 -- 172.21.15.105:0/1020005190 wait complete. 2024-03-26T17:41:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.812+0000 7f2cbf71b640 1 Processor -- start 2024-03-26T17:41:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.812+0000 7f2cbf71b640 1 -- start start 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.812+0000 7f2cbf71b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb81023a0 0x7f2cb8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.812+0000 7f2cbf71b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb8104cd0 0x7f2cb8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbf71b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8107600 0x7f2cb819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbf71b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb810d480 con 0x7f2cb8104cd0 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbf71b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb810d300 con 0x7f2cb8107600 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbf71b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb810d600 con 0x7f2cb81023a0 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb8104cd0 0x7f2cb8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb8104cd0 0x7f2cb8198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49106/0 (socket says 172.21.15.105:49106) 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 -- 172.21.15.105:0/3547818169 learned_addr learned my addr 172.21.15.105:0/3547818169 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb81023a0 0x7f2cb8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbdcb8640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8107600 0x7f2cb819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 -- 172.21.15.105:0/3547818169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb81023a0 msgr2=0x7f2cb8197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb81023a0 0x7f2cb8197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 -- 172.21.15.105:0/3547818169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8107600 msgr2=0x7f2cb819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8107600 0x7f2cb819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 -- 172.21.15.105:0/3547818169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb819f6f0 con 0x7f2cb8104cd0 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb8104cd0 0x7f2cb8198010 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9f0 tx=0x7f2ca800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cae7fc640 1 -- 172.21.15.105:0/3547818169 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800ccc0 con 0x7f2cb8104cd0 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cae7fc640 1 -- 172.21.15.105:0/3547818169 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ca80054d0 con 0x7f2cb8104cd0 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbf71b640 1 -- 172.21.15.105:0/3547818169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb81001f0 con 0x7f2cb8104cd0 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cae7fc640 1 -- 172.21.15.105:0/3547818169 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8010430 con 0x7f2cb8104cd0 2024-03-26T17:41:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.813+0000 7f2cbf71b640 1 -- 172.21.15.105:0/3547818169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb8100740 con 0x7f2cb8104cd0 2024-03-26T17:41:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.816+0000 7f2cae7fc640 1 -- 172.21.15.105:0/3547818169 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2ca8005020 con 0x7f2cb8104cd0 2024-03-26T17:41:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cc23e1740 1 Processor -- start 2024-03-26T17:41:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cc23e1740 1 -- start start 2024-03-26T17:41:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c595e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595efc0 0x55d3c5963ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c5964130 0x55d3c59665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d3c58c30b0 con 0x55d3c5964130 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3c586b960 con 0x55d3c595e5d0 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3c58bf060 con 0x55d3c595efc0 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c5964130 0x55d3c59665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbdcb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c595e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c5964130 0x55d3c59665c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49118/0 (socket says 172.21.15.105:49118) 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbccb6640 1 -- 172.21.15.105:0/1362737198 learned_addr learned my addr 172.21.15.105:0/1362737198 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595efc0 0x55d3c5963ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cae7fc640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ca40a3040 0x7f2ca40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbdcb8640 1 -- 172.21.15.105:0/1362737198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595efc0 msgr2=0x55d3c5963ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cae7fc640 1 -- 172.21.15.105:0/3547818169 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f2ca80cf740 con 0x7f2cb8104cd0 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbdcb8640 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595efc0 0x55d3c5963ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbdcb8640 1 -- 172.21.15.105:0/1362737198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c5964130 msgr2=0x55d3c59665c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbdcb8640 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c5964130 0x55d3c59665c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbdcb8640 1 -- 172.21.15.105:0/1362737198 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c587dc60 con 0x55d3c595e5d0 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c5964130 0x55d3c59665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.817+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595efc0 0x55d3c5963ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.818+0000 7f2cbdcb8640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ca40a3040 0x7f2ca40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.818+0000 7f2cbdcb8640 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c595e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009fd0 tx=0x7f2cb0011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a50974cb8ba3b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.818+0000 7f2cbdcb8640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ca40a3040 0x7f2ca40a5500 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0022000 tx=0x7f2cb0023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.818+0000 7f2ca37fe640 1 -- 172.21.15.105:0/1362737198 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0017070 con 0x55d3c595e5d0 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.818+0000 7f2ca37fe640 1 -- 172.21.15.105:0/1362737198 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cb000fcb0 con 0x55d3c595e5d0 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.818+0000 7f2ca37fe640 1 -- 172.21.15.105:0/1362737198 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000eb20 con 0x55d3c595e5d0 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 -- 172.21.15.105:0/1362737198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 msgr2=0x55d3c595e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c595e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009fd0 tx=0x7f2cb0011920 comp rx=0 tx=0).stop 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 -- 172.21.15.105:0/1362737198 shutdown_connections 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c5964130 0x55d3c59665c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595efc0 0x55d3c5963ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/1362737198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c595e9f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 -- 172.21.15.105:0/1362737198 >> 172.21.15.105:0/1362737198 conn(0x55d3c5959b40 msgr2=0x55d3c5959f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 -- 172.21.15.105:0/1362737198 shutdown_connections 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 -- 172.21.15.105:0/1362737198 wait complete. 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 -- start start 2024-03-26T17:41:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.819+0000 7f2cc23e1740 1 Processor -- start 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595e5d0 0x55d3c59683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c59688f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c5964130 0x55d3c596f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d3c59670e0 con 0x55d3c595efc0 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3c587edb0 con 0x55d3c5964130 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3c5967260 con 0x55d3c595e5d0 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cc23e1740 10 client.0 _ll_register_callbacks cb 0x55d3c5883110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c59688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbd4b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595e5d0 0x55d3c59683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c59688f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49122/0 (socket says 172.21.15.105:49122) 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbdcb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c5964130 0x55d3c596f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbd4b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595e5d0 0x55d3c59683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33716/0 (socket says 172.21.15.105:33716) 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbccb6640 1 -- 172.21.15.105:0/3830939414 learned_addr learned my addr 172.21.15.105:0/3830939414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbd4b7640 1 -- 172.21.15.105:0/3830939414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c5964130 msgr2=0x55d3c596f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c5964130 0x55d3c596f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbd4b7640 1 -- 172.21.15.105:0/3830939414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 msgr2=0x55d3c59688f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c59688f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbd4b7640 1 -- 172.21.15.105:0/3830939414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c55d7210 con 0x55d3c595e5d0 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbdcb8640 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c5964130 0x55d3c596f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.820+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c59688f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595e5d0 0x55d3c59683b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2cb402c170 tx=0x7f2cb4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bb4747579c27a98 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2ca0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/3830939414 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb404c060 con 0x55d3c595e5d0 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 1 -- 172.21.15.105:0/3830939414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cb000fcb0 con 0x55d3c595e5d0 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/3830939414 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f2cb4005230 con 0x55d3c595e5d0 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/3830939414 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cb4005430 con 0x55d3c595e5d0 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 10 client.26443 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 1 -- 172.21.15.105:0/3830939414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d3c56a3620 con 0x55d3c595e5d0 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/3830939414 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cb000fcb0 con 0x55d3c595e5d0 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/3830939414 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2cb4046070 con 0x55d3c595e5d0 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 10 client.26443 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 10 client.26443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 20 client.26443 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3c5974260 0x55d3c5976690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 4 client.26443 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.821+0000 7f2cc23e1740 1 -- 172.21.15.105:0/3830939414 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3c58d05f0 con 0x55d3c5974260 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.822+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3c5974260 0x55d3c5976690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.822+0000 7f2ca0ff9640 10 client.26443 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.822+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3c5974260 0x55d3c5976690 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70df8655825c513d server_cookie=ceb116ccdddee2c3 in_seq=0 out_seq=0 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/3830939414 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d3c58d05f0 con 0x55d3c5974260 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2ca0ff9640 10 client.26443 handle_command_reply: tid=0 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 client.26443 shutdown 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 -- 172.21.15.105:0/3830939414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595e5d0 msgr2=0x55d3c59683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595e5d0 0x55d3c59683b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2cb402c170 tx=0x7f2cb4004270 comp rx=0 tx=0).stop 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 -- 172.21.15.105:0/3830939414 shutdown_connections 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3c5974260 0x55d3c5976690 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c5964130 0x55d3c596f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c59688f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2ca0ff9640 0 client.26443 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/3830939414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c595e5d0 0x55d3c59683b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 -- 172.21.15.105:0/3830939414 >> 172.21.15.105:0/3830939414 conn(0x55d3c5959b40 msgr2=0x55d3c5958df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 -- 172.21.15.105:0/3830939414 shutdown_connections 2024-03-26T17:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 1 -- 172.21.15.105:0/3830939414 wait complete. 2024-03-26T17:41:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.823+0000 7f2cc23e1740 20 client.26443 trim_cache size 0 max 16384 2024-03-26T17:41:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.843+0000 7f2cc23e1740 1 Processor -- start 2024-03-26T17:41:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.843+0000 7f2cc23e1740 1 -- start start 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.843+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c59602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.843+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.843+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 0x55d3c5958c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.843+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb81a6ac0 con 0x55d3c595efc0 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.843+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb81a8600 con 0x55d3c595e5d0 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.843+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb81a7190 con 0x55d3c5964130 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbd4b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c59602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbdcb8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 0x55d3c5958c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbd4b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c59602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57380/0 (socket says 172.21.15.105:57380) 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbd4b7640 1 -- 172.21.15.105:0/2057946087 learned_addr learned my addr 172.21.15.105:0/2057946087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbccb6640 1 -- 172.21.15.105:0/2057946087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 msgr2=0x55d3c5958c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/2057946087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 0x55d3c5958c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbccb6640 1 -- 172.21.15.105:0/2057946087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 msgr2=0x55d3c59602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/2057946087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c59602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbccb6640 1 -- 172.21.15.105:0/2057946087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c5972be0 con 0x55d3c595efc0 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/2057946087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c59602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/2057946087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596f8f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80cd250 tx=0x7f2ca80bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ced583c50fc05f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2ca2ffd640 1 -- 172.21.15.105:0/2057946087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800ce60 con 0x55d3c595efc0 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2ca2ffd640 1 -- 172.21.15.105:0/2057946087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ca80046c0 con 0x55d3c595efc0 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.844+0000 7f2ca2ffd640 1 -- 172.21.15.105:0/2057946087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca809ff80 con 0x55d3c595efc0 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 -- 172.21.15.105:0/2057946087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 msgr2=0x55d3c596f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/2057946087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596f8f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80cd250 tx=0x7f2ca80bc040 comp rx=0 tx=0).stop 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 -- 172.21.15.105:0/2057946087 shutdown_connections 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/2057946087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 0x55d3c5958c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/2057946087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596f8f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/2057946087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c59602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 -- 172.21.15.105:0/2057946087 >> 172.21.15.105:0/2057946087 conn(0x55d3c5959b40 msgr2=0x55d3c5958560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 -- 172.21.15.105:0/2057946087 shutdown_connections 2024-03-26T17:41:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 -- 172.21.15.105:0/2057946087 wait complete. 2024-03-26T17:41:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 -- start start 2024-03-26T17:41:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.845+0000 7f2cc23e1740 1 Processor -- start 2024-03-26T17:41:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c596cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cc23e1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 0x55d3c596e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb81a7f30 con 0x55d3c595efc0 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb81a7860 con 0x55d3c595e5d0 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cc23e1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb81a4aa0 con 0x55d3c5964130 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cc23e1740 10 client.0 _ll_register_callbacks cb 0x7f2cb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49148/0 (socket says 172.21.15.105:49148) 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbd4b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c596cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 -- 172.21.15.105:0/654496521 learned_addr learned my addr 172.21.15.105:0/654496521 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbdcb8640 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 0x55d3c596e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 -- 172.21.15.105:0/654496521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 msgr2=0x55d3c596e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 0x55d3c596e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 -- 172.21.15.105:0/654496521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 msgr2=0x55d3c596cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c596cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 -- 172.21.15.105:0/654496521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c596e8a0 con 0x55d3c595efc0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2cbccb6640 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596d2a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8098290 tx=0x7f2ca80b5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af50a3aaf11835c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.846+0000 7f2ca0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/654496521 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800ce60 con 0x55d3c595efc0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/654496521 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f2ca80a43b0 con 0x55d3c595efc0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 1 -- 172.21.15.105:0/654496521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d3c58ca940 con 0x55d3c595efc0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/654496521 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ca8005310 con 0x55d3c595efc0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 10 client.16608 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 1 -- 172.21.15.105:0/654496521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d3c596fc50 con 0x55d3c595efc0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/654496521 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d3c58ca940 con 0x55d3c595efc0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/654496521 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2ca80bacc0 con 0x55d3c595efc0 2024-03-26T17:41:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 10 client.16608 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 20 client.16608 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3c59dad50 0x55d3c59dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 4 client.16608 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cc23e1740 1 -- 172.21.15.105:0/654496521 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3c5848770 con 0x55d3c59dad50 2024-03-26T17:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.847+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3c59dad50 0x55d3c59dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.848+0000 7f2cbd4b7640 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3c59dad50 0x55d3c59dd170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac80378f7993f48 server_cookie=1bbcd96aa7d8f809 in_seq=0 out_seq=0 2024-03-26T17:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.848+0000 7f2ca0ff9640 10 client.16608 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2ca0ff9640 1 -- 172.21.15.105:0/654496521 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3c5848770 con 0x55d3c59dad50 2024-03-26T17:41:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2ca0ff9640 10 client.16608 handle_command_reply: tid=0 2024-03-26T17:41:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 client.16608 shutdown 2024-03-26T17:41:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 -- 172.21.15.105:0/654496521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 msgr2=0x55d3c596d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596d2a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8098290 tx=0x7f2ca80b5040 comp rx=0 tx=0).stop 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 -- 172.21.15.105:0/654496521 shutdown_connections 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3c59dad50 0x55d3c59dd170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3c5964130 0x55d3c596e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3c595efc0 0x55d3c596d2a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2ca0ff9640 0 client.16608 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 --2- 172.21.15.105:0/654496521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3c595e5d0 0x55d3c596cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 -- 172.21.15.105:0/654496521 >> 172.21.15.105:0/654496521 conn(0x55d3c5959b40 msgr2=0x55d3c595d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 -- 172.21.15.105:0/654496521 shutdown_connections 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 1 -- 172.21.15.105:0/654496521 wait complete. 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.903+0000 7f2cc23e1740 20 client.16608 trim_cache size 0 max 16384 2024-03-26T17:41:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:41:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:41:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:41:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 -- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ca40a3040 msgr2=0x7f2ca40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ca40a3040 0x7f2ca40a5500 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0022000 tx=0x7f2cb0023000 comp rx=0 tx=0).stop 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 -- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb8104cd0 msgr2=0x7f2cb8198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb8104cd0 0x7f2cb8198010 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9f0 tx=0x7f2ca800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 -- 172.21.15.105:0/3547818169 shutdown_connections 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ca40a3040 0x7f2ca40a5500 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cb8107600 0x7f2cb819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2cb8104cd0 0x7f2cb8198010 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 --2- 172.21.15.105:0/3547818169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cb81023a0 0x7f2cb8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 -- 172.21.15.105:0/3547818169 >> 172.21.15.105:0/3547818169 conn(0x7f2cb80fbd50 msgr2=0x7f2cb8102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 -- 172.21.15.105:0/3547818169 shutdown_connections 2024-03-26T17:41:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:18.904+0000 7f2cbf71b640 1 -- 172.21.15.105:0/3547818169 wait complete. 2024-03-26T17:41:19.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 72415 2.54 MB/sec execute 2 sec latency 41.526 ms 2024-03-26T17:41:20.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 73089 1.92 MB/sec execute 3 sec latency 16.693 ms 2024-03-26T17:41:20.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:19 smithi144 ceph-mon[36095]: pgmap v437: 97 pgs: 97 active+clean; 30 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:41:20.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:19 smithi105 ceph-mon[32280]: pgmap v437: 97 pgs: 97 active+clean; 30 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:41:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:19 smithi135 ceph-mon[35636]: pgmap v437: 97 pgs: 97 active+clean; 30 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:41:21.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 73714 2.76 MB/sec execute 4 sec latency 25.194 ms 2024-03-26T17:41:22.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 74325 2.90 MB/sec execute 5 sec latency 16.360 ms 2024-03-26T17:41:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:21 smithi144 ceph-mon[36095]: pgmap v438: 97 pgs: 97 active+clean; 30 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:41:22.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:21 smithi105 ceph-mon[32280]: pgmap v438: 97 pgs: 97 active+clean; 30 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:41:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:21 smithi135 ceph-mon[35636]: pgmap v438: 97 pgs: 97 active+clean; 30 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:41:23.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 74880 3.19 MB/sec execute 6 sec latency 18.475 ms 2024-03-26T17:41:23.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:22 smithi105 ceph-mon[32280]: pgmap v439: 97 pgs: 97 active+clean; 27 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-03-26T17:41:23.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:22 smithi135 ceph-mon[35636]: pgmap v439: 97 pgs: 97 active+clean; 27 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-03-26T17:41:23.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:22 smithi144 ceph-mon[36095]: pgmap v439: 97 pgs: 97 active+clean; 27 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-03-26T17:41:24.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 75331 2.80 MB/sec execute 7 sec latency 24.244 ms 2024-03-26T17:41:25.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 75791 2.67 MB/sec execute 8 sec latency 211.794 ms 2024-03-26T17:41:26.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 76464 2.74 MB/sec execute 9 sec latency 17.537 ms 2024-03-26T17:41:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:25 smithi144 ceph-mon[36095]: pgmap v440: 97 pgs: 97 active+clean; 27 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-03-26T17:41:26.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:25 smithi105 ceph-mon[32280]: pgmap v440: 97 pgs: 97 active+clean; 27 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-03-26T17:41:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:25 smithi135 ceph-mon[35636]: pgmap v440: 97 pgs: 97 active+clean; 27 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.8 MiB/s wr, 126 op/s 2024-03-26T17:41:27.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 77037 2.70 MB/sec execute 10 sec latency 90.106 ms 2024-03-26T17:41:28.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 77844 3.05 MB/sec execute 11 sec latency 16.305 ms 2024-03-26T17:41:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:27 smithi144 ceph-mon[36095]: pgmap v441: 97 pgs: 97 active+clean; 28 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:41:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:27 smithi105 ceph-mon[32280]: pgmap v441: 97 pgs: 97 active+clean; 28 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:41:28.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:27 smithi135 ceph-mon[35636]: pgmap v441: 97 pgs: 97 active+clean; 28 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:41:29.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 78335 3.17 MB/sec execute 12 sec latency 15.312 ms 2024-03-26T17:41:30.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 78708 2.95 MB/sec execute 13 sec latency 23.061 ms 2024-03-26T17:41:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:29 smithi144 ceph-mon[36095]: pgmap v442: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-03-26T17:41:30.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:29 smithi105 ceph-mon[32280]: pgmap v442: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-03-26T17:41:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:29 smithi135 ceph-mon[35636]: pgmap v442: 97 pgs: 97 active+clean; 29 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.5 MiB/s wr, 127 op/s 2024-03-26T17:41:31.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 79324 2.89 MB/sec execute 14 sec latency 23.339 ms 2024-03-26T17:41:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 -- 172.21.15.105:0/3328009236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6154102390 msgr2=0x7f6154108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 --2- 172.21.15.105:0/3328009236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6154102390 0x7f6154108c20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f614c009ca0 tx=0x7f614c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 -- 172.21.15.105:0/3328009236 shutdown_connections 2024-03-26T17:41:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 --2- 172.21.15.105:0/3328009236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6154102390 0x7f6154108c20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 --2- 172.21.15.105:0/3328009236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61541019d0 0x7f6154101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 --2- 172.21.15.105:0/3328009236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61541007e0 0x7f6154100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 -- 172.21.15.105:0/3328009236 >> 172.21.15.105:0/3328009236 conn(0x7f61540fbd50 msgr2=0x7f61540fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 -- 172.21.15.105:0/3328009236 shutdown_connections 2024-03-26T17:41:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.008+0000 7f615aeea640 1 -- 172.21.15.105:0/3328009236 wait complete. 2024-03-26T17:41:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f615aeea640 1 Processor -- start 2024-03-26T17:41:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f615aeea640 1 -- start start 2024-03-26T17:41:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f615aeea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61541007e0 0x7f6154069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f615aeea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61541019d0 0x7f615419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f615aeea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6154102390 0x7f615419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f615aeea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f615410c310 con 0x7f61541007e0 2024-03-26T17:41:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f615aeea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f615410c190 con 0x7f6154102390 2024-03-26T17:41:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f615aeea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f615410c490 con 0x7f61541019d0 2024-03-26T17:41:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f6158c86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61541007e0 0x7f6154069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f6158c86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61541007e0 0x7f6154069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55732/0 (socket says 172.21.15.105:55732) 2024-03-26T17:41:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f6158c86640 1 -- 172.21.15.105:0/28787214 learned_addr learned my addr 172.21.15.105:0/28787214 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f614bfff640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61541019d0 0x7f615419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f6159487640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6154102390 0x7f615419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f614bfff640 1 -- 172.21.15.105:0/28787214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6154102390 msgr2=0x7f615419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f614bfff640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6154102390 0x7f615419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f614bfff640 1 -- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61541007e0 msgr2=0x7f6154069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f614bfff640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61541007e0 0x7f6154069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f614bfff640 1 -- 172.21.15.105:0/28787214 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615419f680 con 0x7f61541019d0 2024-03-26T17:41:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f6158c86640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61541007e0 0x7f6154069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f614bfff640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61541019d0 0x7f615419c780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f614400e9f0 tx=0x7f614400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.009+0000 7f6149ffb640 1 -- 172.21.15.105:0/28787214 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614400ccc0 con 0x7f61541019d0 2024-03-26T17:41:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.010+0000 7f615aeea640 1 -- 172.21.15.105:0/28787214 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615419f970 con 0x7f61541019d0 2024-03-26T17:41:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.010+0000 7f6149ffb640 1 -- 172.21.15.105:0/28787214 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6144005020 con 0x7f61541019d0 2024-03-26T17:41:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.010+0000 7f615aeea640 1 -- 172.21.15.105:0/28787214 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615419fec0 con 0x7f61541019d0 2024-03-26T17:41:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.010+0000 7f6149ffb640 1 -- 172.21.15.105:0/28787214 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144004a00 con 0x7f61541019d0 2024-03-26T17:41:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.012+0000 7f6149ffb640 1 -- 172.21.15.105:0/28787214 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6144004ba0 con 0x7f61541019d0 2024-03-26T17:41:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.012+0000 7f615dbb5740 1 Processor -- start 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.012+0000 7f615dbb5740 1 -- start start 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.012+0000 7f6149ffb640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f613c0a3040 0x7f613c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d48549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4859ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6159487640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f613c0a3040 0x7f613c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d485c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6149ffb640 1 -- 172.21.15.105:0/28787214 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f61440cf260 con 0x7f61541019d0 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f615dbb5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0d47b90b0 con 0x55a0d485a130 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f615dbb5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0d4761960 con 0x55a0d4854fc0 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f615dbb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0d47b5060 con 0x55a0d48545d0 2024-03-26T17:41:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6159487640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d485c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f614bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4859ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f614bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4859ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58174/0 (socket says 172.21.15.105:58174) 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f614bfff640 1 -- 172.21.15.105:0/3842786221 learned_addr learned my addr 172.21.15.105:0/3842786221 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6158c86640 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d48549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6158c86640 1 -- 172.21.15.105:0/3842786221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 msgr2=0x55a0d4859ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6158c86640 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4859ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6158c86640 1 -- 172.21.15.105:0/3842786221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 msgr2=0x55a0d485c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6158c86640 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d485c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6158c86640 1 -- 172.21.15.105:0/3842786221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d4773c60 con 0x55a0d48545d0 2024-03-26T17:41:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f614bfff640 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4859ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6159487640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f613c0a3040 0x7f613c0a5500 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f614c006010 tx=0x7f614c009970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f6158c86640 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d48549f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f61500114c0 tx=0x7f6150011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff15b8d0d9de3734 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.013+0000 7f613affd640 1 -- 172.21.15.105:0/3842786221 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150005020 con 0x55a0d48545d0 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f613affd640 1 -- 172.21.15.105:0/3842786221 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61500047a0 con 0x55a0d48545d0 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f613affd640 1 -- 172.21.15.105:0/3842786221 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615000f770 con 0x55a0d48545d0 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 -- 172.21.15.105:0/3842786221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 msgr2=0x55a0d48549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d48549f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f61500114c0 tx=0x7f6150011980 comp rx=0 tx=0).stop 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 -- 172.21.15.105:0/3842786221 shutdown_connections 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d485c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.019 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4859ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3842786221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d48549f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 -- 172.21.15.105:0/3842786221 >> 172.21.15.105:0/3842786221 conn(0x55a0d484fb40 msgr2=0x55a0d484ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 -- 172.21.15.105:0/3842786221 shutdown_connections 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.014+0000 7f615dbb5740 1 -- 172.21.15.105:0/3842786221 wait complete. 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 1 -- start start 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 1 Processor -- start 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d48545d0 0x55a0d485e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d4854fc0 0x55a0d485e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d48658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a0d485d0e0 con 0x55a0d485a130 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a0d4774db0 con 0x55a0d48545d0 2024-03-26T17:41:32.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a0d485d260 con 0x55a0d4854fc0 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f615dbb5740 10 client.0 _ll_register_callbacks cb 0x55a0d4779110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6159487640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d48658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f614bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d4854fc0 0x55a0d485e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6158c86640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d48545d0 0x55a0d485e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6159487640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d48658a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55750/0 (socket says 172.21.15.105:55750) 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f614bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d4854fc0 0x55a0d485e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40376/0 (socket says 172.21.15.105:40376) 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6159487640 1 -- 172.21.15.105:0/1337705944 learned_addr learned my addr 172.21.15.105:0/1337705944 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6159487640 1 -- 172.21.15.105:0/1337705944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d4854fc0 msgr2=0x55a0d485e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:32.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6159487640 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d4854fc0 0x55a0d485e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6159487640 1 -- 172.21.15.105:0/1337705944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d48545d0 msgr2=0x55a0d485e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6159487640 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d48545d0 0x55a0d485e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.015+0000 7f6159487640 1 -- 172.21.15.105:0/1337705944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d44cd210 con 0x55a0d485a130 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f6159487640 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d48658a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f614c036000 tx=0x7f614c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d8eb7549329b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f6133fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f6133fff640 1 -- 172.21.15.105:0/1337705944 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c039040 con 0x55a0d485a130 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 1 -- 172.21.15.105:0/1337705944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61500047a0 con 0x55a0d485a130 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f6133fff640 1 -- 172.21.15.105:0/1337705944 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f614c034440 con 0x55a0d485a130 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f6133fff640 1 -- 172.21.15.105:0/1337705944 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f614c034640 con 0x55a0d485a130 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 10 client.16620 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 1 -- 172.21.15.105:0/1337705944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0d4599620 con 0x55a0d485a130 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f6133fff640 1 -- 172.21.15.105:0/1337705944 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61500047a0 con 0x55a0d485a130 2024-03-26T17:41:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f6133fff640 1 -- 172.21.15.105:0/1337705944 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f614c03f040 con 0x55a0d485a130 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 10 client.16620 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 20 client.16620 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0d486a6b0 0x55a0d486cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 4 client.16620 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.016+0000 7f615dbb5740 1 -- 172.21.15.105:0/1337705944 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d47c65f0 con 0x55a0d486a6b0 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.017+0000 7f6158c86640 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0d486a6b0 0x55a0d486cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.017+0000 7f6158c86640 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0d486a6b0 0x55a0d486cb70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb407a013f152843 server_cookie=b5f6769e37f42d28 in_seq=0 out_seq=0 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.017+0000 7f6133fff640 10 client.16620 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f6133fff640 1 -- 172.21.15.105:0/1337705944 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a0d47c65f0 con 0x55a0d486a6b0 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f6133fff640 10 client.16620 handle_command_reply: tid=0 2024-03-26T17:41:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 client.16620 shutdown 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 -- 172.21.15.105:0/1337705944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 msgr2=0x55a0d48658a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d48658a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f614c036000 tx=0x7f614c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 -- 172.21.15.105:0/1337705944 shutdown_connections 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0d486a6b0 0x55a0d486cb70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d48658a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d4854fc0 0x55a0d485e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 --2- 172.21.15.105:0/1337705944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d48545d0 0x55a0d485e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f6133fff640 0 client.16620 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 -- 172.21.15.105:0/1337705944 >> 172.21.15.105:0/1337705944 conn(0x55a0d484fb40 msgr2=0x55a0d484edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 -- 172.21.15.105:0/1337705944 shutdown_connections 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 1 -- 172.21.15.105:0/1337705944 wait complete. 2024-03-26T17:41:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.018+0000 7f615dbb5740 20 client.16620 trim_cache size 0 max 16384 2024-03-26T17:41:32.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 79961 2.91 MB/sec execute 15 sec latency 50.665 ms 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f615dbb5740 1 Processor -- start 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f615dbb5740 1 -- start start 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4855850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d4854fc0 0x55a0d48657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d485a130 0x55a0d4855d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f615dbb5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f614000e530 con 0x55a0d4854fc0 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f615dbb5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6140010070 con 0x55a0d485a130 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f615dbb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f614000ec00 con 0x55a0d48545d0 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f614bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d4854fc0 0x55a0d48657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f6158c86640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4855850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f614bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d4854fc0 0x55a0d48657c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55764/0 (socket says 172.21.15.105:55764) 2024-03-26T17:41:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f6158c86640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4855850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40386/0 (socket says 172.21.15.105:40386) 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f614bfff640 1 -- 172.21.15.105:0/2807019761 learned_addr learned my addr 172.21.15.105:0/2807019761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f6159487640 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d485a130 0x55a0d4855d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f6158c86640 1 -- 172.21.15.105:0/2807019761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d485a130 msgr2=0x55a0d4855d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f6158c86640 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d485a130 0x55a0d4855d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f6158c86640 1 -- 172.21.15.105:0/2807019761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d4854fc0 msgr2=0x55a0d48657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.035+0000 7f6158c86640 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d4854fc0 0x55a0d48657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.036+0000 7f6158c86640 1 -- 172.21.15.105:0/2807019761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d484b450 con 0x55a0d48545d0 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.036+0000 7f614bfff640 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d4854fc0 0x55a0d48657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.036+0000 7f6158c86640 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4855850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f61500058c0 tx=0x7f6150011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ed6dc797d28d13 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.036+0000 7f613a7fc640 1 -- 172.21.15.105:0/2807019761 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150009c10 con 0x55a0d48545d0 2024-03-26T17:41:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.036+0000 7f613a7fc640 1 -- 172.21.15.105:0/2807019761 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61500044f0 con 0x55a0d48545d0 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.036+0000 7f613a7fc640 1 -- 172.21.15.105:0/2807019761 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150005360 con 0x55a0d48545d0 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 -- 172.21.15.105:0/2807019761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 msgr2=0x55a0d4855850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4855850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f61500058c0 tx=0x7f6150011f90 comp rx=0 tx=0).stop 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 -- 172.21.15.105:0/2807019761 shutdown_connections 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d485a130 0x55a0d4855d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d4854fc0 0x55a0d48657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 --2- 172.21.15.105:0/2807019761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4855850 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 -- 172.21.15.105:0/2807019761 >> 172.21.15.105:0/2807019761 conn(0x55a0d484fb40 msgr2=0x55a0d485e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 -- 172.21.15.105:0/2807019761 shutdown_connections 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 -- 172.21.15.105:0/2807019761 wait complete. 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 -- start start 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.037+0000 7f615dbb5740 1 Processor -- start 2024-03-26T17:41:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4862c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4863170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f615dbb5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d4864180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f615dbb5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f614000f9a0 con 0x55a0d485a130 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f615dbb5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f614000f2d0 con 0x55a0d4854fc0 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f615dbb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f614000c510 con 0x55a0d48545d0 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f615dbb5740 10 client.0 _ll_register_callbacks cb 0x7f6144005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6158c86640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4862c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6158c86640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4862c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40402/0 (socket says 172.21.15.105:40402) 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f614bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4863170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6158c86640 1 -- 172.21.15.105:0/3450086286 learned_addr learned my addr 172.21.15.105:0/3450086286 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6159487640 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d4864180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6159487640 1 -- 172.21.15.105:0/3450086286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 msgr2=0x55a0d4862c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6159487640 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4862c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6159487640 1 -- 172.21.15.105:0/3450086286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 msgr2=0x55a0d4863170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6159487640 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4863170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6159487640 1 -- 172.21.15.105:0/3450086286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d4864770 con 0x55a0d485a130 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.038+0000 7f6158c86640 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4862c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f6159487640 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d4864180 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f614c0436e0 tx=0x7f614c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e61303303db8a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f6133fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f6133fff640 1 -- 172.21.15.105:0/3450086286 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c05d070 con 0x55a0d485a130 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f6133fff640 1 -- 172.21.15.105:0/3450086286 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f614c004750 con 0x55a0d485a130 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 1 -- 172.21.15.105:0/3450086286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61500044f0 con 0x55a0d485a130 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f6133fff640 1 -- 172.21.15.105:0/3450086286 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f614c039040 con 0x55a0d485a130 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 10 client.16626 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 1 -- 172.21.15.105:0/3450086286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0d4865a80 con 0x55a0d485a130 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f6133fff640 1 -- 172.21.15.105:0/3450086286 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61500044f0 con 0x55a0d485a130 2024-03-26T17:41:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f6133fff640 1 -- 172.21.15.105:0/3450086286 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f614c03f040 con 0x55a0d485a130 2024-03-26T17:41:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 10 client.16626 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 20 client.16626 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0d48d1030 0x55a0d48d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 4 client.16626 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:41:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.039+0000 7f615dbb5740 1 -- 172.21.15.105:0/3450086286 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0d47c0940 con 0x55a0d48d1030 2024-03-26T17:41:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.040+0000 7f6158c86640 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0d48d1030 0x55a0d48d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.041+0000 7f6133fff640 10 client.16626 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.041+0000 7f6158c86640 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0d48d1030 0x55a0d48d3450 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97c71262be2e89e server_cookie=5d4732cbe594793f in_seq=0 out_seq=0 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f6133fff640 1 -- 172.21.15.105:0/3450086286 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0d47c0940 con 0x55a0d48d1030 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f6133fff640 10 client.16626 handle_command_reply: tid=0 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 client.16626 shutdown 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 -- 172.21.15.105:0/3450086286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 msgr2=0x55a0d4864180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d4864180 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f614c0436e0 tx=0x7f614c005b70 comp rx=0 tx=0).stop 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 -- 172.21.15.105:0/3450086286 shutdown_connections 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a0d48d1030 0x55a0d48d3450 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a0d485a130 0x55a0d4864180 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a0d4854fc0 0x55a0d4863170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f6133fff640 0 client.16626 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 --2- 172.21.15.105:0/3450086286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a0d48545d0 0x55a0d4862c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 -- 172.21.15.105:0/3450086286 >> 172.21.15.105:0/3450086286 conn(0x55a0d484fb40 msgr2=0x55a0d4852da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 -- 172.21.15.105:0/3450086286 shutdown_connections 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 1 -- 172.21.15.105:0/3450086286 wait complete. 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.076+0000 7f615dbb5740 20 client.16626 trim_cache size 0 max 16384 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:41:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:41:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 -- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f613c0a3040 msgr2=0x7f613c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f613c0a3040 0x7f613c0a5500 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f614c006010 tx=0x7f614c009970 comp rx=0 tx=0).stop 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 -- 172.21.15.105:0/28787214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61541019d0 msgr2=0x7f615419c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61541019d0 0x7f615419c780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f614400e9f0 tx=0x7f614400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 -- 172.21.15.105:0/28787214 shutdown_connections 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f613c0a3040 0x7f613c0a5500 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6154102390 0x7f615419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61541019d0 0x7f615419c780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 --2- 172.21.15.105:0/28787214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61541007e0 0x7f6154069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 -- 172.21.15.105:0/28787214 >> 172.21.15.105:0/28787214 conn(0x7f61540fbd50 msgr2=0x7f6154104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 -- 172.21.15.105:0/28787214 shutdown_connections 2024-03-26T17:41:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:32.077+0000 7f615aeea640 1 -- 172.21.15.105:0/28787214 wait complete. 2024-03-26T17:41:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:31 smithi144 ceph-mon[36095]: pgmap v443: 97 pgs: 97 active+clean; 28 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:41:32.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:31 smithi105 ceph-mon[32280]: pgmap v443: 97 pgs: 97 active+clean; 28 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:41:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:31 smithi135 ceph-mon[35636]: pgmap v443: 97 pgs: 97 active+clean; 28 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:41:33.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 80555 2.87 MB/sec execute 16 sec latency 19.238 ms 2024-03-26T17:41:34.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 81386 3.10 MB/sec execute 17 sec latency 16.804 ms 2024-03-26T17:41:34.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:33 smithi144 ceph-mon[36095]: pgmap v444: 97 pgs: 97 active+clean; 29 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-03-26T17:41:34.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:33 smithi105 ceph-mon[32280]: pgmap v444: 97 pgs: 97 active+clean; 29 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-03-26T17:41:34.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:33 smithi135 ceph-mon[35636]: pgmap v444: 97 pgs: 97 active+clean; 29 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-03-26T17:41:35.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 81874 3.17 MB/sec execute 18 sec latency 20.664 ms 2024-03-26T17:41:35.966 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:35 smithi135 ceph-mon[35636]: pgmap v445: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:41:36.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 82257 3.02 MB/sec execute 19 sec latency 18.561 ms 2024-03-26T17:41:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:35 smithi144 ceph-mon[36095]: pgmap v445: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:41:36.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:35 smithi105 ceph-mon[32280]: pgmap v445: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:41:37.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 82836 2.98 MB/sec execute 20 sec latency 19.323 ms 2024-03-26T17:41:38.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 83539 2.99 MB/sec execute 21 sec latency 15.488 ms 2024-03-26T17:41:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:37 smithi144 ceph-mon[36095]: pgmap v446: 97 pgs: 97 active+clean; 27 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-03-26T17:41:38.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:37 smithi105 ceph-mon[32280]: pgmap v446: 97 pgs: 97 active+clean; 27 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-03-26T17:41:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:37 smithi135 ceph-mon[35636]: pgmap v446: 97 pgs: 97 active+clean; 27 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2024-03-26T17:41:39.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 84138 2.96 MB/sec execute 22 sec latency 17.246 ms 2024-03-26T17:41:40.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 84941 3.12 MB/sec execute 23 sec latency 14.991 ms 2024-03-26T17:41:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:39 smithi144 ceph-mon[36095]: pgmap v447: 97 pgs: 97 active+clean; 28 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:41:40.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:39 smithi105 ceph-mon[32280]: pgmap v447: 97 pgs: 97 active+clean; 28 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:41:40.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:39 smithi135 ceph-mon[35636]: pgmap v447: 97 pgs: 97 active+clean; 28 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:41:41.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 85417 3.18 MB/sec execute 24 sec latency 23.025 ms 2024-03-26T17:41:42.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 85796 3.06 MB/sec execute 25 sec latency 15.618 ms 2024-03-26T17:41:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:41 smithi144 ceph-mon[36095]: pgmap v448: 97 pgs: 97 active+clean; 26 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.5 MiB/s wr, 93 op/s 2024-03-26T17:41:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:41 smithi105 ceph-mon[32280]: pgmap v448: 97 pgs: 97 active+clean; 26 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.5 MiB/s wr, 93 op/s 2024-03-26T17:41:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:41 smithi135 ceph-mon[35636]: pgmap v448: 97 pgs: 97 active+clean; 26 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.5 MiB/s wr, 93 op/s 2024-03-26T17:41:43.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 86364 3.02 MB/sec execute 26 sec latency 21.851 ms 2024-03-26T17:41:44.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 87084 3.04 MB/sec execute 27 sec latency 15.871 ms 2024-03-26T17:41:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:43 smithi144 ceph-mon[36095]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:41:44.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:43 smithi105 ceph-mon[32280]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:41:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:43 smithi135 ceph-mon[35636]: pgmap v449: 97 pgs: 97 active+clean; 28 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:41:44.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:41:44.478 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/1011504152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb41007c0 msgr2=0x7f5fb4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/1011504152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb41007c0 0x7f5fb4100be0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009900 tx=0x7f5fb002bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/1011504152 shutdown_connections 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/1011504152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb4102370 0x7f5fb4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/1011504152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fb41019b0 0x7f5fb4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/1011504152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb41007c0 0x7f5fb4100be0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/1011504152 >> 172.21.15.105:0/1011504152 conn(0x7f5fb40fbd50 msgr2=0x7f5fb40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/1011504152 shutdown_connections 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.604+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/1011504152 wait complete. 2024-03-26T17:41:44.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fbbfb9640 1 Processor -- start 2024-03-26T17:41:44.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fbbfb9640 1 -- start start 2024-03-26T17:41:44.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fbbfb9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb41007c0 0x7f5fb4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fbbfb9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fb41019b0 0x7f5fb41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fbbfb9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb4102370 0x7f5fb419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fbbfb9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410c2f0 con 0x7f5fb4102370 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fbbfb9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410c170 con 0x7f5fb41019b0 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fbbfb9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410c470 con 0x7f5fb41007c0 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fb9554640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fb41019b0 0x7f5fb41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fba556640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb4102370 0x7f5fb419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fb9d55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb41007c0 0x7f5fb4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fb9554640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fb41019b0 0x7f5fb41980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58286/0 (socket says 172.21.15.105:58286) 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.605+0000 7f5fb9554640 1 -- 172.21.15.105:0/252373665 learned_addr learned my addr 172.21.15.105:0/252373665 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fb9d55640 1 -- 172.21.15.105:0/252373665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fb41019b0 msgr2=0x7f5fb41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fb41019b0 0x7f5fb41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fb9d55640 1 -- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb4102370 msgr2=0x7f5fb419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb4102370 0x7f5fb419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fb9d55640 1 -- 172.21.15.105:0/252373665 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb419f7a0 con 0x7f5fb41007c0 2024-03-26T17:41:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fb9554640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fb41019b0 0x7f5fb41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fba556640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb4102370 0x7f5fb419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb41007c0 0x7f5fb4197b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0036040 tx=0x7f5fb0005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5faaffd640 1 -- 172.21.15.105:0/252373665 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0043020 con 0x7f5fb41007c0 2024-03-26T17:41:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5faaffd640 1 -- 172.21.15.105:0/252373665 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fb0004350 con 0x7f5fb41007c0 2024-03-26T17:41:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5faaffd640 1 -- 172.21.15.105:0/252373665 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0003ab0 con 0x7f5fb41007c0 2024-03-26T17:41:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/252373665 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb419fa30 con 0x7f5fb41007c0 2024-03-26T17:41:44.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.606+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/252373665 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb419ffa0 con 0x7f5fb41007c0 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.608+0000 7f5faaffd640 1 -- 172.21.15.105:0/252373665 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5fb0046020 con 0x7f5fb41007c0 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5faaffd640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fa00a2fa0 0x7f5fa00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fbeba7740 1 Processor -- start 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5faaffd640 1 -- 172.21.15.105:0/252373665 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5fb00f14d0 con 0x7f5fb41007c0 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fbeba7740 1 -- start start 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fb9554640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fa00a2fa0 0x7f5fa00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784becf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556478417980 con 0x5564784b76f0 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5564783bfbe0 con 0x5564784b6d00 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556478413930 con 0x5564784bc860 2024-03-26T17:41:44.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fb9d55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fb9d55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784b7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58296/0 (socket says 172.21.15.105:58296) 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fb9d55640 1 -- 172.21.15.105:0/2725243233 learned_addr learned my addr 172.21.15.105:0/2725243233 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fba556640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.609+0000 7f5fb9554640 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5fb9554640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fa00a2fa0 0x7f5fa00a5460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4199090 tx=0x7f5fa4011940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5fb9554640 1 -- 172.21.15.105:0/2725243233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 msgr2=0x5564784becf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5fb9554640 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784becf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5fb9554640 1 -- 172.21.15.105:0/2725243233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 msgr2=0x5564784b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5fb9554640 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5fb9554640 1 -- 172.21.15.105:0/2725243233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564781f57d0 con 0x5564784b76f0 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5fba556640 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784becf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5fb9554640 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784bc1f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400f7d0 tx=0x7f5fa401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a4c3739fa73b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5f9ffff640 1 -- 172.21.15.105:0/2725243233 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4017040 con 0x5564784b76f0 2024-03-26T17:41:44.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5f9ffff640 1 -- 172.21.15.105:0/2725243233 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fa4019050 con 0x5564784b76f0 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.610+0000 7f5f9ffff640 1 -- 172.21.15.105:0/2725243233 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa401d6a0 con 0x5564784b76f0 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 -- 172.21.15.105:0/2725243233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 msgr2=0x5564784bc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784bc1f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400f7d0 tx=0x7f5fa401c000 comp rx=0 tx=0).stop 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 -- 172.21.15.105:0/2725243233 shutdown_connections 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784becf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784bc1f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/2725243233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 -- 172.21.15.105:0/2725243233 >> 172.21.15.105:0/2725243233 conn(0x5564784b2270 msgr2=0x5564784b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 -- 172.21.15.105:0/2725243233 shutdown_connections 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 -- 172.21.15.105:0/2725243233 wait complete. 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 -- start start 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.611+0000 7f5fbeba7740 1 Processor -- start 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5564784bf810 con 0x5564784b6d00 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5564783d3160 con 0x5564784b76f0 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5564784bf990 con 0x5564784bc860 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fbeba7740 10 client.0 _ll_register_callbacks cb 0x55647840f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41702/0 (socket says 172.21.15.105:41702) 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 -- 172.21.15.105:0/1759090762 learned_addr learned my addr 172.21.15.105:0/1759090762 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9554640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fba556640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 -- 172.21.15.105:0/1759090762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 msgr2=0x5564784c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 -- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 msgr2=0x5564784c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 -- 172.21.15.105:0/1759090762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647812b3f0 con 0x5564784b6d00 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fba556640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9554640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c0fa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00bd8c0 tx=0x7f5fb00bdbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1c30c3631ad288 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5f9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.612+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/1759090762 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0003e50 con 0x5564784b6d00 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/1759090762 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5fb00f1110 con 0x5564784b6d00 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 1 -- 172.21.15.105:0/1759090762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa4019050 con 0x5564784b6d00 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/1759090762 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5fb00f1310 con 0x5564784b6d00 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 10 client.16638 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 1 -- 172.21.15.105:0/1759090762 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5564780eb0f0 con 0x5564784b6d00 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/1759090762 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fa4019050 con 0x5564784b6d00 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/1759090762 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5fb00b9910 con 0x5564784b6d00 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 10 client.16638 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:44.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 20 client.16638 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5564784ccd70 0x5564784cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 4 client.16638 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fbeba7740 1 -- 172.21.15.105:0/1759090762 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556478424ec0 con 0x5564784ccd70 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.613+0000 7f5fb9554640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5564784ccd70 0x5564784cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.614+0000 7f5fb9554640 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5564784ccd70 0x5564784cf1c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b59bc0df3666d server_cookie=7f39b875d0157edb in_seq=0 out_seq=0 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.614+0000 7f5f9d7fa640 10 client.16638 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.614+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/1759090762 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556478424ec0 con 0x5564784ccd70 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.614+0000 7f5f9d7fa640 10 client.16638 handle_command_reply: tid=0 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.614+0000 7f5fbeba7740 1 client.16638 shutdown 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 -- 172.21.15.105:0/1759090762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 msgr2=0x5564784c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c0fa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00bd8c0 tx=0x7f5fb00bdbc0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 -- 172.21.15.105:0/1759090762 shutdown_connections 2024-03-26T17:41:44.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5564784ccd70 0x5564784cf1c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784bc860 0x5564784c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/1759090762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c0fa0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5f9d7fa640 0 client.16638 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:44.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 -- 172.21.15.105:0/1759090762 >> 172.21.15.105:0/1759090762 conn(0x5564784b2270 msgr2=0x5564784ad080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:44.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 -- 172.21.15.105:0/1759090762 shutdown_connections 2024-03-26T17:41:44.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 1 -- 172.21.15.105:0/1759090762 wait complete. 2024-03-26T17:41:44.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.615+0000 7f5fbeba7740 20 client.16638 trim_cache size 0 max 16384 2024-03-26T17:41:44.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fbeba7740 1 Processor -- start 2024-03-26T17:41:44.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fbeba7740 1 -- start start 2024-03-26T17:41:44.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41a6bd0 con 0x5564784b76f0 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41a8710 con 0x5564784b6d00 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41a72a0 con 0x5564784b2270 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fba556640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fb9d55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fb9d55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44956/0 (socket says 172.21.15.105:44956) 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fb9d55640 1 -- 172.21.15.105:0/644965134 learned_addr learned my addr 172.21.15.105:0/644965134 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.635+0000 7f5fb9554640 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5fba556640 1 -- 172.21.15.105:0/644965134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 msgr2=0x5564784c0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5fba556640 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5fba556640 1 -- 172.21.15.105:0/644965134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 msgr2=0x5564784c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:44.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5fba556640 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5fba556640 1 -- 172.21.15.105:0/644965134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564784a98d0 con 0x5564784b76f0 2024-03-26T17:41:44.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5fb9554640 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:44.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5fba556640 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784c10f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009fd0 tx=0x7f5fac00fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73957eace937410 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:44.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5f9f7fe640 1 -- 172.21.15.105:0/644965134 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac021070 con 0x5564784b76f0 2024-03-26T17:41:44.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5f9f7fe640 1 -- 172.21.15.105:0/644965134 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fac010d90 con 0x5564784b76f0 2024-03-26T17:41:44.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.636+0000 7f5f9f7fe640 1 -- 172.21.15.105:0/644965134 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac01c3f0 con 0x5564784b76f0 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 -- 172.21.15.105:0/644965134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 msgr2=0x5564784c10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784c10f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009fd0 tx=0x7f5fac00fde0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 -- 172.21.15.105:0/644965134 shutdown_connections 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b76f0 0x5564784c10f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b6d00 0x5564784c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/644965134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 -- 172.21.15.105:0/644965134 >> 172.21.15.105:0/644965134 conn(0x5564784bc330 msgr2=0x5564784b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 -- 172.21.15.105:0/644965134 shutdown_connections 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 -- 172.21.15.105:0/644965134 wait complete. 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.637+0000 7f5fbeba7740 1 -- start start 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fbeba7740 1 Processor -- start 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fbeba7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41a8040 con 0x5564784b6d00 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41a7970 con 0x5564784b76f0 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fbeba7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41a4bb0 con 0x5564784b2270 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fbeba7740 10 client.0 _ll_register_callbacks cb 0x7f5fb00eeed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9554640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9554640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41710/0 (socket says 172.21.15.105:41710) 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9554640 1 -- 172.21.15.105:0/648879951 learned_addr learned my addr 172.21.15.105:0/648879951 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9d55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fba556640 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9d55640 1 -- 172.21.15.105:0/648879951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 msgr2=0x5564784c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9d55640 1 -- 172.21.15.105:0/648879951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 msgr2=0x5564784c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:44.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9d55640 1 -- 172.21.15.105:0/648879951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564784c7880 con 0x5564784b2270 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.638+0000 7f5fb9554640 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fb9d55640 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c5d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00ba020 tx=0x7f5fb0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20232f204b49a383 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5f9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/648879951 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00f1110 con 0x5564784b2270 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/648879951 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5fb00b9910 con 0x5564784b2270 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 1 -- 172.21.15.105:0/648879951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fac010d90 con 0x5564784b2270 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/648879951 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5fb00b9b10 con 0x5564784b2270 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 10 client.26485 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 1 -- 172.21.15.105:0/648879951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556478533280 con 0x5564784b2270 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/648879951 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fac010d90 con 0x5564784b2270 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/648879951 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5fb00a3020 con 0x5564784b2270 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 10 client.26485 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:44.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 20 client.26485 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5564784bc860 0x556478535a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 4 client.26485 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.639+0000 7f5fbeba7740 1 -- 172.21.15.105:0/648879951 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55647841f210 con 0x5564784bc860 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.640+0000 7f5fb9554640 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5564784bc860 0x556478535a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.640+0000 7f5f9d7fa640 10 client.26485 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.640+0000 7f5fb9554640 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5564784bc860 0x556478535a10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c88413898c7d0e server_cookie=ce23aecbd8c64ee8 in_seq=0 out_seq=0 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.641+0000 7f5f9d7fa640 1 -- 172.21.15.105:0/648879951 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55647841f210 con 0x5564784bc860 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.641+0000 7f5f9d7fa640 10 client.26485 handle_command_reply: tid=0 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.641+0000 7f5fbeba7740 1 client.26485 shutdown 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.641+0000 7f5fbeba7740 1 -- 172.21.15.105:0/648879951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 msgr2=0x5564784c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.641+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c5d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00ba020 tx=0x7f5fb0000bc0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 1 -- 172.21.15.105:0/648879951 shutdown_connections 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5564784bc860 0x556478535a10 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5f9d7fa640 0 client.26485 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:44.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5564784b76f0 0x5564784c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5564784b6d00 0x5564784c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 1 --2- 172.21.15.105:0/648879951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5564784b2270 0x5564784c5d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 1 -- 172.21.15.105:0/648879951 >> 172.21.15.105:0/648879951 conn(0x5564784bc330 msgr2=0x5564784b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 1 -- 172.21.15.105:0/648879951 shutdown_connections 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 1 -- 172.21.15.105:0/648879951 wait complete. 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbeba7740 20 client.26485 trim_cache size 0 max 16384 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:41:44.639 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:41:44.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fa00a2fa0 msgr2=0x7f5fa00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fa00a2fa0 0x7f5fa00a5460 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4199090 tx=0x7f5fa4011940 comp rx=0 tx=0).stop 2024-03-26T17:41:44.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/252373665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb41007c0 msgr2=0x7f5fb4197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.642+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb41007c0 0x7f5fb4197b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0036040 tx=0x7f5fb0005dc0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.643+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/252373665 shutdown_connections 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.643+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5fa00a2fa0 0x7f5fa00a5460 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.643+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fb4102370 0x7f5fb419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.643+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fb41019b0 0x7f5fb41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.643+0000 7f5fbbfb9640 1 --2- 172.21.15.105:0/252373665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fb41007c0 0x7f5fb4197b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.643+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/252373665 >> 172.21.15.105:0/252373665 conn(0x7f5fb40fbd50 msgr2=0x7f5fb4068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.643+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/252373665 shutdown_connections 2024-03-26T17:41:44.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.643+0000 7f5fbbfb9640 1 -- 172.21.15.105:0/252373665 wait complete. 2024-03-26T17:41:44.646 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:41:44.647 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3112163121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 msgr2=0x7fc5f4100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3112163121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 0x7fc5f4100c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009860 tx=0x7fc5e402bb70 comp rx=0 tx=0).stop 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3112163121 shutdown_connections 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3112163121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f4102390 0x7fc5f4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3112163121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f41019d0 0x7fc5f4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3112163121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 0x7fc5f4100c00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3112163121 >> 172.21.15.105:0/3112163121 conn(0x7fc5f40fbd50 msgr2=0x7fc5f40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3112163121 shutdown_connections 2024-03-26T17:41:44.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.770+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3112163121 wait complete. 2024-03-26T17:41:44.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f98dc640 1 Processor -- start 2024-03-26T17:41:44.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f98dc640 1 -- start start 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f98dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 0x7fc5f40695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f98dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f41019d0 0x7fc5f419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f98dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4102390 0x7fc5f419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f98dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410c310 con 0x7fc5f4102390 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f98dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410c190 con 0x7fc5f41007e0 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f98dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f410c490 con 0x7fc5f41019d0 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 0x7fc5f40695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4102390 0x7fc5f419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f27fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f41019d0 0x7fc5f419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 0x7fc5f40695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58350/0 (socket says 172.21.15.105:58350) 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.771+0000 7fc5f2ffd640 1 -- 172.21.15.105:0/3697671782 learned_addr learned my addr 172.21.15.105:0/3697671782 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f2ffd640 1 -- 172.21.15.105:0/3697671782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f41019d0 msgr2=0x7fc5f419c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f2ffd640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f41019d0 0x7fc5f419c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f2ffd640 1 -- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4102390 msgr2=0x7fc5f419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f2ffd640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4102390 0x7fc5f419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f2ffd640 1 -- 172.21.15.105:0/3697671782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f419f760 con 0x7fc5f41007e0 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f27fc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f41019d0 0x7fc5f419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f2ffd640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 0x7fc5f40695f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4005940 tx=0x7fc5e40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f88da640 1 -- 172.21.15.105:0/3697671782 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4041020 con 0x7fc5f41007e0 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f88da640 1 -- 172.21.15.105:0/3697671782 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e40035f0 con 0x7fc5f41007e0 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f88da640 1 -- 172.21.15.105:0/3697671782 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4039040 con 0x7fc5f41007e0 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f4196600 con 0x7fc5f41007e0 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.772+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f4196b40 con 0x7fc5f41007e0 2024-03-26T17:41:44.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.773+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f4107b70 con 0x7fc5f41007e0 2024-03-26T17:41:44.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.775+0000 7fc5f88da640 1 -- 172.21.15.105:0/3697671782 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc5e4044020 con 0x7fc5f41007e0 2024-03-26T17:41:44.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.775+0000 7fc5f88da640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5dc0a2fa0 0x7fc5dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:44.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.775+0000 7fc5f27fc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5dc0a2fa0 0x7fc5dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:44.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.775+0000 7fc5f88da640 1 -- 172.21.15.105:0/3697671782 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc5e40f0f80 con 0x7fc5f41007e0 2024-03-26T17:41:44.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.776+0000 7fc5f27fc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5dc0a2fa0 0x7fc5dc0a5460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80096c0 tx=0x7fc5e8012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:44.779+0000 7fc5f88da640 1 -- 172.21.15.105:0/3697671782 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc5e404b020 con 0x7fc5f41007e0 2024-03-26T17:41:45.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 87725 3.02 MB/sec execute 28 sec latency 15.709 ms 2024-03-26T17:41:45.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.060+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5f402d240 con 0x7fc5f41007e0 2024-03-26T17:41:45.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.061+0000 7fc5f88da640 1 -- 172.21.15.105:0/3697671782 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fc5e40bdd90 con 0x7fc5f41007e0 2024-03-26T17:41:45.054 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:41:45.054 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:41:45.054 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:41:45.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5dc0a2fa0 msgr2=0x7fc5dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5dc0a2fa0 0x7fc5dc0a5460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80096c0 tx=0x7fc5e8012040 comp rx=0 tx=0).stop 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 msgr2=0x7fc5f40695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 0x7fc5f40695f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4005940 tx=0x7fc5e40042f0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 shutdown_connections 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5dc0a2fa0 0x7fc5dc0a5460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc5f4102390 0x7fc5f419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5f41019d0 0x7fc5f419c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 --2- 172.21.15.105:0/3697671782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5f41007e0 0x7fc5f40695f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 >> 172.21.15.105:0/3697671782 conn(0x7fc5f40fbd50 msgr2=0x7fc5f4104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 shutdown_connections 2024-03-26T17:41:45.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.063+0000 7fc5f98dc640 1 -- 172.21.15.105:0/3697671782 wait complete. 2024-03-26T17:41:45.067 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:41:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 -- 172.21.15.105:0/3048634431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 msgr2=0x7fc60c103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 --2- 172.21.15.105:0/3048634431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 0x7fc60c103040 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009930 tx=0x7fc5fc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 -- 172.21.15.105:0/3048634431 shutdown_connections 2024-03-26T17:41:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 --2- 172.21.15.105:0/3048634431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c103580 0x7fc60c109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 --2- 172.21.15.105:0/3048634431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 0x7fc60c103040 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 --2- 172.21.15.105:0/3048634431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc60c0ffbf0 0x7fc60c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 -- 172.21.15.105:0/3048634431 >> 172.21.15.105:0/3048634431 conn(0x7fc60c0fb520 msgr2=0x7fc60c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 -- 172.21.15.105:0/3048634431 shutdown_connections 2024-03-26T17:41:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.183+0000 7fc6137a9640 1 -- 172.21.15.105:0/3048634431 wait complete. 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc6137a9640 1 Processor -- start 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc6137a9640 1 -- start start 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc6137a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c0ffbf0 0x7fc60c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc6137a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 0x7fc60c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc6137a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc60c103580 0x7fc60c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc6137a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c10d460 con 0x7fc60c102bc0 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc6137a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c10d2e0 con 0x7fc60c103580 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc6137a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c10d5e0 con 0x7fc60c0ffbf0 2024-03-26T17:41:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc611545640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c0ffbf0 0x7fc60c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc610d44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 0x7fc60c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc611545640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c0ffbf0 0x7fc60c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44994/0 (socket says 172.21.15.105:44994) 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc611545640 1 -- 172.21.15.105:0/432480741 learned_addr learned my addr 172.21.15.105:0/432480741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc611d46640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc60c103580 0x7fc60c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc610d44640 1 -- 172.21.15.105:0/432480741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c0ffbf0 msgr2=0x7fc60c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc610d44640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c0ffbf0 0x7fc60c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc610d44640 1 -- 172.21.15.105:0/432480741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc60c103580 msgr2=0x7fc60c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.184+0000 7fc610d44640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc60c103580 0x7fc60c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.185+0000 7fc610d44640 1 -- 172.21.15.105:0/432480741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c19f6f0 con 0x7fc60c102bc0 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.185+0000 7fc611d46640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc60c103580 0x7fc60c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.185+0000 7fc610d44640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 0x7fc60c198010 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009230 tx=0x7fc5fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.185+0000 7fc6027fc640 1 -- 172.21.15.105:0/432480741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc043020 con 0x7fc60c102bc0 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.185+0000 7fc6137a9640 1 -- 172.21.15.105:0/432480741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c19f980 con 0x7fc60c102bc0 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.185+0000 7fc6027fc640 1 -- 172.21.15.105:0/432480741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5fc004440 con 0x7fc60c102bc0 2024-03-26T17:41:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.185+0000 7fc6027fc640 1 -- 172.21.15.105:0/432480741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc003bf0 con 0x7fc60c102bc0 2024-03-26T17:41:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.185+0000 7fc6137a9640 1 -- 172.21.15.105:0/432480741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c19fed0 con 0x7fc60c102bc0 2024-03-26T17:41:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.187+0000 7fc6027fc640 1 -- 172.21.15.105:0/432480741 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc5fc046020 con 0x7fc60c102bc0 2024-03-26T17:41:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc61638d740 1 Processor -- start 2024-03-26T17:41:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc61638d740 1 -- start start 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc6027fc640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5f80a2fc0 0x7fc5f80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc61638d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb44095d0 0x55cbb44099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc61638d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb440eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc61638d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb440f130 0x55cbb44115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc61638d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbb436e0b0 con 0x55cbb44095d0 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc61638d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cbb4316960 con 0x55cbb440f130 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc61638d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cbb436a060 con 0x55cbb4409fc0 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc611545640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb44095d0 0x55cbb44099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc6027fc640 1 -- 172.21.15.105:0/432480741 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc5fc0f16a0 con 0x7fc60c102bc0 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc611545640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb44095d0 0x55cbb44099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41756/0 (socket says 172.21.15.105:41756) 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc611545640 1 -- 172.21.15.105:0/2242472911 learned_addr learned my addr 172.21.15.105:0/2242472911 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc611d46640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5f80a2fc0 0x7fc5f80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.188+0000 7fc610d44640 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb440eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc611d46640 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb440f130 0x55cbb44115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc611545640 1 -- 172.21.15.105:0/2242472911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 msgr2=0x55cbb440eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc611545640 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb440eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc611545640 1 -- 172.21.15.105:0/2242472911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb440f130 msgr2=0x55cbb44115c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc611545640 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb440f130 0x55cbb44115c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc611545640 1 -- 172.21.15.105:0/2242472911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb4328c60 con 0x55cbb44095d0 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc611d46640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5f80a2fc0 0x7fc5f80a5480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc604009370 tx=0x7fc604009910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc611545640 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb44095d0 0x55cbb44099f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc608009d60 tx=0x7fc608015c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e254f894e7dc16a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc5f77fe640 1 -- 172.21.15.105:0/2242472911 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608004230 con 0x55cbb44095d0 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc5f77fe640 1 -- 172.21.15.105:0/2242472911 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc60800fe50 con 0x55cbb44095d0 2024-03-26T17:41:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.189+0000 7fc5f77fe640 1 -- 172.21.15.105:0/2242472911 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608003e40 con 0x55cbb44095d0 2024-03-26T17:41:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.190+0000 7fc61638d740 1 -- 172.21.15.105:0/2242472911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb44095d0 msgr2=0x55cbb44099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.190+0000 7fc61638d740 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb44095d0 0x55cbb44099f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc608009d60 tx=0x7fc608015c20 comp rx=0 tx=0).stop 2024-03-26T17:41:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.190+0000 7fc61638d740 1 -- 172.21.15.105:0/2242472911 shutdown_connections 2024-03-26T17:41:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.190+0000 7fc61638d740 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb440f130 0x55cbb44115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.190+0000 7fc61638d740 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb440eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.190+0000 7fc61638d740 1 --2- 172.21.15.105:0/2242472911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb44095d0 0x55cbb44099f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.190+0000 7fc61638d740 1 -- 172.21.15.105:0/2242472911 >> 172.21.15.105:0/2242472911 conn(0x55cbb4404b40 msgr2=0x55cbb4404f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 -- 172.21.15.105:0/2242472911 shutdown_connections 2024-03-26T17:41:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 -- 172.21.15.105:0/2242472911 wait complete. 2024-03-26T17:41:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 -- start start 2024-03-26T17:41:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 Processor -- start 2024-03-26T17:41:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb44095d0 0x55cbb44132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb4409fc0 0x55cbb4413830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb441a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cbb44120e0 con 0x55cbb440f130 2024-03-26T17:41:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cbb4329db0 con 0x55cbb4409fc0 2024-03-26T17:41:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cbb4412260 con 0x55cbb44095d0 2024-03-26T17:41:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc61638d740 10 client.0 _ll_register_callbacks cb 0x55cbb432e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc611545640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb44095d0 0x55cbb44132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc610d44640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb4409fc0 0x55cbb4413830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.191+0000 7fc611d46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb441a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc610d44640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb4409fc0 0x55cbb4413830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58374/0 (socket says 172.21.15.105:58374) 2024-03-26T17:41:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc611545640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb44095d0 0x55cbb44132f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45004/0 (socket says 172.21.15.105:45004) 2024-03-26T17:41:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc611d46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb441a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41768/0 (socket says 172.21.15.105:41768) 2024-03-26T17:41:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc610d44640 1 -- 172.21.15.105:0/349982082 learned_addr learned my addr 172.21.15.105:0/349982082 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc610d44640 1 -- 172.21.15.105:0/349982082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb44095d0 msgr2=0x55cbb44132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc610d44640 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb44095d0 0x55cbb44132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc610d44640 1 -- 172.21.15.105:0/349982082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 msgr2=0x55cbb441a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc610d44640 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb441a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc610d44640 1 -- 172.21.15.105:0/349982082 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb4082210 con 0x55cbb4409fc0 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc611545640 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb44095d0 0x55cbb44132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc610d44640 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb4409fc0 0x55cbb4413830 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ef160 tx=0x7fc5fc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c0a90408305069 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc5f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/349982082 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0a4170 con 0x55cbb4409fc0 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/349982082 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc5fc0bc090 con 0x55cbb4409fc0 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.192+0000 7fc61638d740 1 -- 172.21.15.105:0/349982082 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc60800fe50 con 0x55cbb4409fc0 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/349982082 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc5fc0be490 con 0x55cbb4409fc0 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc61638d740 10 client.26510 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc61638d740 1 -- 172.21.15.105:0/349982082 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbb414e620 con 0x55cbb4409fc0 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/349982082 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc60800fe50 con 0x55cbb4409fc0 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/349982082 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc5fc0a0ce0 con 0x55cbb4409fc0 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc61638d740 10 client.26510 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc61638d740 10 client.26510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc61638d740 20 client.26510 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc61638d740 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cbb441f140 0x55cbb44215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc61638d740 4 client.26510 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc61638d740 1 -- 172.21.15.105:0/349982082 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbb437b5f0 con 0x55cbb441f140 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.193+0000 7fc611545640 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cbb441f140 0x55cbb44215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.194+0000 7fc5f4ff9640 10 client.26510 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.194+0000 7fc611545640 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cbb441f140 0x55cbb44215d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c546ddd44a2345e server_cookie=e8dd2ba0fd45c089 in_seq=0 out_seq=0 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.194+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/349982082 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cbb437b5f0 con 0x55cbb441f140 2024-03-26T17:41:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.194+0000 7fc5f4ff9640 10 client.26510 handle_command_reply: tid=0 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.194+0000 7fc61638d740 1 client.26510 shutdown 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.194+0000 7fc61638d740 1 -- 172.21.15.105:0/349982082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb4409fc0 msgr2=0x55cbb4413830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.194+0000 7fc61638d740 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb4409fc0 0x55cbb4413830 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ef160 tx=0x7fc5fc009260 comp rx=0 tx=0).stop 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 1 -- 172.21.15.105:0/349982082 shutdown_connections 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cbb441f140 0x55cbb44215d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb441a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb4409fc0 0x55cbb4413830 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc5f4ff9640 0 client.26510 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 1 --2- 172.21.15.105:0/349982082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb44095d0 0x55cbb44132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 1 -- 172.21.15.105:0/349982082 >> 172.21.15.105:0/349982082 conn(0x55cbb4404b40 msgr2=0x55cbb4403df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 1 -- 172.21.15.105:0/349982082 shutdown_connections 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 1 -- 172.21.15.105:0/349982082 wait complete. 2024-03-26T17:41:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.195+0000 7fc61638d740 20 client.26510 trim_cache size 0 max 16384 2024-03-26T17:41:45.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 -- 172.21.15.105:0/3715830067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81007c0 msgr2=0x7f24d8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 --2- 172.21.15.105:0/3715830067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81007c0 0x7f24d8100be0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009900 tx=0x7f24c802bc40 comp rx=0 tx=0).stop 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 -- 172.21.15.105:0/3715830067 shutdown_connections 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 --2- 172.21.15.105:0/3715830067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d8102370 0x7f24d8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 --2- 172.21.15.105:0/3715830067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d81019b0 0x7f24d8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 --2- 172.21.15.105:0/3715830067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81007c0 0x7f24d8100be0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 -- 172.21.15.105:0/3715830067 >> 172.21.15.105:0/3715830067 conn(0x7f24d80fbd50 msgr2=0x7f24d80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 -- 172.21.15.105:0/3715830067 shutdown_connections 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.200+0000 7f24dddb3640 1 -- 172.21.15.105:0/3715830067 wait complete. 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.201+0000 7f24dddb3640 1 Processor -- start 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.201+0000 7f24dddb3640 1 -- start start 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.201+0000 7f24dddb3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d81007c0 0x7f24d810afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.201+0000 7f24dddb3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81019b0 0x7f24d8109600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.201+0000 7f24dddb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d8102370 0x7f24d8109b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.201+0000 7f24dddb3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d810d410 con 0x7f24d8102370 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.201+0000 7f24dddb3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d810d290 con 0x7f24d81019b0 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.201+0000 7f24dddb3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d810d590 con 0x7f24d81007c0 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d81007c0 0x7f24d810afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d81007c0 0x7f24d810afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45026/0 (socket says 172.21.15.105:45026) 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 -- 172.21.15.105:0/1521040444 learned_addr learned my addr 172.21.15.105:0/1521040444 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d7fff640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d8102370 0x7f24d8109b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81019b0 0x7f24d8109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 -- 172.21.15.105:0/1521040444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81019b0 msgr2=0x7f24d8109600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81019b0 0x7f24d8109600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 -- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d8102370 msgr2=0x7f24d8109b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d8102370 0x7f24d8109b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 -- 172.21.15.105:0/1521040444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d810a3d0 con 0x7f24d81007c0 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d7fff640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d8102370 0x7f24d8109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81019b0 0x7f24d8109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d77fe640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d81007c0 0x7f24d810afc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f24c8000c00 tx=0x7f24c8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.202+0000 7f24d4ff9640 1 -- 172.21.15.105:0/1521040444 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8043020 con 0x7f24d81007c0 2024-03-26T17:41:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.203+0000 7f24d4ff9640 1 -- 172.21.15.105:0/1521040444 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c8004350 con 0x7f24d81007c0 2024-03-26T17:41:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.203+0000 7f24d4ff9640 1 -- 172.21.15.105:0/1521040444 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8003b80 con 0x7f24d81007c0 2024-03-26T17:41:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.203+0000 7f24dddb3640 1 -- 172.21.15.105:0/1521040444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d81a1c80 con 0x7f24d81007c0 2024-03-26T17:41:45.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.203+0000 7f24dddb3640 1 -- 172.21.15.105:0/1521040444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d81a21d0 con 0x7f24d81007c0 2024-03-26T17:41:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24d4ff9640 1 -- 172.21.15.105:0/1521040444 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f24c8046020 con 0x7f24d81007c0 2024-03-26T17:41:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24e0996740 1 Processor -- start 2024-03-26T17:41:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24e0996740 1 -- start start 2024-03-26T17:41:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24e0996740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24e0996740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24e0996740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5bda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24e0996740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f9d51a970 con 0x558f9d5b6420 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24e0996740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558f9d4c2d70 con 0x558f9d5bb590 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24e0996740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558f9d516920 con 0x558f9d5b5a30 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5baf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41794/0 (socket says 172.21.15.105:41794) 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24d77fe640 1 -- 172.21.15.105:0/576989680 learned_addr learned my addr 172.21.15.105:0/576989680 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24d7fff640 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.206+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5bda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d4ff9640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24c00a2f20 0x7f24c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d77fe640 1 -- 172.21.15.105:0/576989680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 msgr2=0x558f9d5b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d77fe640 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d77fe640 1 -- 172.21.15.105:0/576989680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 msgr2=0x558f9d5bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d77fe640 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5bda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d77fe640 1 -- 172.21.15.105:0/576989680 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f9d22e3f0 con 0x558f9d5b6420 2024-03-26T17:41:45.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d7fff640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24c00a2f20 0x7f24c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d77fe640 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5baf20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f24c80d5da0 tx=0x7f24c80b9a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8110539ea984973f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24d4ff9640 1 -- 172.21.15.105:0/1521040444 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f24c80f1630 con 0x7f24d81007c0 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24bdffb640 1 -- 172.21.15.105:0/576989680 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80bdaf0 con 0x558f9d5b6420 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24bdffb640 1 -- 172.21.15.105:0/576989680 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c80be420 con 0x558f9d5b6420 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.207+0000 7f24bdffb640 1 -- 172.21.15.105:0/576989680 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80d2370 con 0x558f9d5b6420 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 -- 172.21.15.105:0/576989680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 msgr2=0x558f9d5baf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5baf20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f24c80d5da0 tx=0x7f24c80b9a70 comp rx=0 tx=0).stop 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24d7fff640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24c00a2f20 0x7f24c00a53e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x558f9d5b6c80 tx=0x7f24d0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 -- 172.21.15.105:0/576989680 shutdown_connections 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5bda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5baf20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 --2- 172.21.15.105:0/576989680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 -- 172.21.15.105:0/576989680 >> 172.21.15.105:0/576989680 conn(0x558f9d5b0fa0 msgr2=0x558f9d5b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 -- 172.21.15.105:0/576989680 shutdown_connections 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 -- 172.21.15.105:0/576989680 wait complete. 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.208+0000 7f24e0996740 1 -- start start 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24e0996740 1 Processor -- start 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24e0996740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24e0996740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24e0996740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24e0996740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558f9d5be300 con 0x558f9d5b6420 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24e0996740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558f9d4d6310 con 0x558f9d5bb590 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24e0996740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558f9d5be480 con 0x558f9d5b5a30 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24e0996740 10 client.0 _ll_register_callbacks cb 0x558f9d511c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41798/0 (socket says 172.21.15.105:41798) 2024-03-26T17:41:45.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24d6ffd640 1 -- 172.21.15.105:0/1606211635 learned_addr learned my addr 172.21.15.105:0/1606211635 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24d77fe640 1 -- 172.21.15.105:0/1606211635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 msgr2=0x558f9d5c7170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24d77fe640 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5c7170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.209+0000 7f24d77fe640 1 -- 172.21.15.105:0/1606211635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 msgr2=0x558f9d5c01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24d77fe640 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24d77fe640 1 -- 172.21.15.105:0/1606211635 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9d2f87d0 con 0x558f9d5b5a30 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24d77fe640 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bfc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f24c80ef0f0 tx=0x7f24c80035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607c638b8ae7dab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24b6ffd640 1 -- 172.21.15.105:0/1606211635 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80d2c70 con 0x558f9d5b5a30 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24e0996740 1 -- 172.21.15.105:0/1606211635 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c80be420 con 0x558f9d5b5a30 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24b6ffd640 1 -- 172.21.15.105:0/1606211635 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f24c80d2030 con 0x558f9d5b5a30 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24b6ffd640 1 -- 172.21.15.105:0/1606211635 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24c80bd680 con 0x558f9d5b5a30 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24e0996740 10 client.26515 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24e0996740 1 -- 172.21.15.105:0/1606211635 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f9d5be300 con 0x558f9d5b5a30 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.210+0000 7f24b6ffd640 1 -- 172.21.15.105:0/1606211635 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c80be420 con 0x558f9d5b5a30 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.211+0000 7f24b6ffd640 1 -- 172.21.15.105:0/1606211635 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f24c80d4070 con 0x558f9d5b5a30 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.211+0000 7f24e0996740 10 client.26515 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.211+0000 7f24e0996740 10 client.26515 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.211+0000 7f24e0996740 20 client.26515 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.211+0000 7f24e0996740 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558f9d5cba40 0x558f9d5cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.211+0000 7f24e0996740 4 client.26515 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.211+0000 7f24e0996740 1 -- 172.21.15.105:0/1606211635 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f9d527eb0 con 0x558f9d5cba40 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.211+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558f9d5cba40 0x558f9d5cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.212+0000 7f24b6ffd640 10 client.26515 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:45.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.212+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558f9d5cba40 0x558f9d5cde90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a6a1fc0f0ee5421 server_cookie=fbf6145cc1914e78 in_seq=0 out_seq=0 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.212+0000 7f24b6ffd640 1 -- 172.21.15.105:0/1606211635 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558f9d527eb0 con 0x558f9d5cba40 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.212+0000 7f24b6ffd640 10 client.26515 handle_command_reply: tid=0 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 client.26515 shutdown 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 -- 172.21.15.105:0/1606211635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 msgr2=0x558f9d5bfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bfc60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f24c80ef0f0 tx=0x7f24c80035f0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 -- 172.21.15.105:0/1606211635 shutdown_connections 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558f9d5cba40 0x558f9d5cde90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 --2- 172.21.15.105:0/1606211635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bfc60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 -- 172.21.15.105:0/1606211635 >> 172.21.15.105:0/1606211635 conn(0x558f9d5b0fa0 msgr2=0x558f9d5b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24b6ffd640 0 client.26515 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:45.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 -- 172.21.15.105:0/1606211635 shutdown_connections 2024-03-26T17:41:45.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 1 -- 172.21.15.105:0/1606211635 wait complete. 2024-03-26T17:41:45.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.213+0000 7f24e0996740 20 client.26515 trim_cache size 0 max 16384 2024-03-26T17:41:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc61638d740 1 Processor -- start 2024-03-26T17:41:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc61638d740 1 -- start start 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc61638d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb440aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc61638d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb441a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc61638d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb440af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc61638d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a6b40 con 0x55cbb440f130 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc61638d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a8680 con 0x55cbb44095d0 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc61638d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a7210 con 0x55cbb4409fc0 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc611d46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb440af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc610d44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb441a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc611545640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb440aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc611d46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb440af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41800/0 (socket says 172.21.15.105:41800) 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc610d44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb441a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45058/0 (socket says 172.21.15.105:45058) 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.215+0000 7fc611d46640 1 -- 172.21.15.105:0/3810271329 learned_addr learned my addr 172.21.15.105:0/3810271329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.216+0000 7fc611545640 1 -- 172.21.15.105:0/3810271329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 msgr2=0x55cbb441a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.216+0000 7fc611545640 1 --2- 172.21.15.105:0/3810271329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb441a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.216+0000 7fc611545640 1 -- 172.21.15.105:0/3810271329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 msgr2=0x55cbb440af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.216+0000 7fc611545640 1 --2- 172.21.15.105:0/3810271329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb440af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.216+0000 7fc611545640 1 -- 172.21.15.105:0/3810271329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb440a3a0 con 0x55cbb44095d0 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.216+0000 7fc611d46640 1 --2- 172.21.15.105:0/3810271329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb440af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.216+0000 7fc611545640 1 --2- 172.21.15.105:0/3810271329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb440aa30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc608003770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4784003b38479202 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.216+0000 7fc5f6ffd640 1 -- 172.21.15.105:0/3810271329 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608034070 con 0x55cbb44095d0 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc5f6ffd640 1 -- 172.21.15.105:0/3810271329 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6080302d0 con 0x55cbb44095d0 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc5f6ffd640 1 -- 172.21.15.105:0/3810271329 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080050c0 con 0x55cbb44095d0 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 -- 172.21.15.105:0/3810271329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 msgr2=0x55cbb440aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 --2- 172.21.15.105:0/3810271329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb440aa30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc608000c00 tx=0x7fc608003770 comp rx=0 tx=0).stop 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 -- 172.21.15.105:0/3810271329 shutdown_connections 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 --2- 172.21.15.105:0/3810271329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb440f130 0x55cbb440af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 --2- 172.21.15.105:0/3810271329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb4409fc0 0x55cbb441a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 --2- 172.21.15.105:0/3810271329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb440aa30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 -- 172.21.15.105:0/3810271329 >> 172.21.15.105:0/3810271329 conn(0x55cbb4404b40 msgr2=0x55cbb44132f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 -- 172.21.15.105:0/3810271329 shutdown_connections 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.217+0000 7fc61638d740 1 -- 172.21.15.105:0/3810271329 wait complete. 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 1 -- start start 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 1 Processor -- start 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb4417c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb4409fc0 0x55cbb4418140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb440f130 0x55cbb4419150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a7fb0 con 0x55cbb4409fc0 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a78e0 con 0x55cbb44095d0 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c1a4b20 con 0x55cbb440f130 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc61638d740 10 client.0 _ll_register_callbacks cb 0x7fc5fc0de0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc611545640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb4417c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc610d44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb4409fc0 0x55cbb4418140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc611545640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb4417c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58420/0 (socket says 172.21.15.105:58420) 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc611545640 1 -- 172.21.15.105:0/4150498532 learned_addr learned my addr 172.21.15.105:0/4150498532 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.218+0000 7fc611d46640 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb440f130 0x55cbb4419150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc610d44640 1 -- 172.21.15.105:0/4150498532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb440f130 msgr2=0x55cbb4419150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc610d44640 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb440f130 0x55cbb4419150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc610d44640 1 -- 172.21.15.105:0/4150498532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 msgr2=0x55cbb4417c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc610d44640 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb4417c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc610d44640 1 -- 172.21.15.105:0/4150498532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb4419740 con 0x55cbb4409fc0 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc611545640 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb4417c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc610d44640 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb4409fc0 0x55cbb4418140 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ba710 tx=0x7fc5fc0bd640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96ebd2b64d3502f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc5f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/4150498532 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0bdb60 con 0x55cbb4409fc0 2024-03-26T17:41:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/4150498532 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc5fc0dcb00 con 0x55cbb4409fc0 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc61638d740 1 -- 172.21.15.105:0/4150498532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6080302d0 con 0x55cbb4409fc0 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/4150498532 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc5fc0c67a0 con 0x55cbb4409fc0 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc61638d740 10 client.16680 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc61638d740 1 -- 172.21.15.105:0/4150498532 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbb441aa50 con 0x55cbb4409fc0 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/4150498532 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6080302d0 con 0x55cbb4409fc0 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.219+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/4150498532 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc5fc0bcc50 con 0x55cbb4409fc0 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc61638d740 10 client.16680 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc61638d740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc61638d740 20 client.16680 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc61638d740 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cbb4485a10 0x55cbb4487e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc61638d740 4 client.16680 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc61638d740 1 -- 172.21.15.105:0/4150498532 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbb4375940 con 0x55cbb4485a10 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc611545640 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cbb4485a10 0x55cbb4487e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc5f4ff9640 10 client.16680 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.220+0000 7fc611545640 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cbb4485a10 0x55cbb4487e30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446ad61f72755062 server_cookie=50c76586629885d6 in_seq=0 out_seq=0 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.229+0000 7f24e0996740 1 Processor -- start 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.229+0000 7f24e0996740 1 -- start start 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24e0996740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24e0996740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24e0996740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24e0996740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81aaff0 con 0x558f9d5b6420 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24e0996740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81acb30 con 0x558f9d5bb590 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24e0996740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81ab6c0 con 0x558f9d5b5a30 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45082/0 (socket says 172.21.15.105:45082) 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d77fe640 1 -- 172.21.15.105:0/3100867994 learned_addr learned my addr 172.21.15.105:0/3100867994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d7fff640 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d77fe640 1 -- 172.21.15.105:0/3100867994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 msgr2=0x558f9d5bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d77fe640 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d77fe640 1 -- 172.21.15.105:0/3100867994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 msgr2=0x558f9d5c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d77fe640 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d77fe640 1 -- 172.21.15.105:0/3100867994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f9d5bb260 con 0x558f9d5b5a30 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.230+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.231+0000 7f24d77fe640 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bf870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f24c80bbfd0 tx=0x7f24c8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc3b117b43347c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.231+0000 7f24bd7fa640 1 -- 172.21.15.105:0/3100867994 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80bd860 con 0x558f9d5b5a30 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.231+0000 7f24bd7fa640 1 -- 172.21.15.105:0/3100867994 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c80be420 con 0x558f9d5b5a30 2024-03-26T17:41:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.231+0000 7f24bd7fa640 1 -- 172.21.15.105:0/3100867994 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80bc020 con 0x558f9d5b5a30 2024-03-26T17:41:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.232+0000 7f24e0996740 1 -- 172.21.15.105:0/3100867994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 msgr2=0x558f9d5bf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.232+0000 7f24e0996740 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bf870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f24c80bbfd0 tx=0x7f24c8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.232+0000 7f24e0996740 1 -- 172.21.15.105:0/3100867994 shutdown_connections 2024-03-26T17:41:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.232+0000 7f24e0996740 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5bb590 0x558f9d5bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.232+0000 7f24e0996740 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.232+0000 7f24e0996740 1 --2- 172.21.15.105:0/3100867994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5b5a30 0x558f9d5bf870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.232+0000 7f24e0996740 1 -- 172.21.15.105:0/3100867994 >> 172.21.15.105:0/3100867994 conn(0x558f9d5b0fa0 msgr2=0x558f9d5b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.232+0000 7f24e0996740 1 -- 172.21.15.105:0/3100867994 shutdown_connections 2024-03-26T17:41:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.233+0000 7f24e0996740 1 -- 172.21.15.105:0/3100867994 wait complete. 2024-03-26T17:41:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.234+0000 7f24e0996740 1 -- start start 2024-03-26T17:41:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.234+0000 7f24e0996740 1 Processor -- start 2024-03-26T17:41:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24e0996740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5b5a30 0x558f9d5c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24e0996740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24e0996740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5bb590 0x558f9d5c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24e0996740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81ac460 con 0x558f9d5b6420 2024-03-26T17:41:45.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41828/0 (socket says 172.21.15.105:41828) 2024-03-26T17:41:45.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24d6ffd640 1 -- 172.21.15.105:0/2913182969 learned_addr learned my addr 172.21.15.105:0/2913182969 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81abd90 con 0x558f9d5b5a30 2024-03-26T17:41:45.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24d7fff640 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5bb590 0x558f9d5c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81a8fd0 con 0x558f9d5bb590 2024-03-26T17:41:45.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.235+0000 7f24e0996740 10 client.0 _ll_register_callbacks cb 0x7f24cc0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24d6ffd640 1 -- 172.21.15.105:0/2913182969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5bb590 msgr2=0x558f9d5c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5bb590 0x558f9d5c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24d6ffd640 1 -- 172.21.15.105:0/2913182969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5b5a30 msgr2=0x558f9d5c41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5b5a30 0x558f9d5c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24d6ffd640 1 -- 172.21.15.105:0/2913182969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f9d5c5d20 con 0x558f9d5b6420 2024-03-26T17:41:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24d6ffd640 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c4720 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f24cc004700 tx=0x7f24cc012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1f3138ad35777c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:45.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24b6ffd640 1 -- 172.21.15.105:0/2913182969 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc01fce0 con 0x558f9d5b6420 2024-03-26T17:41:45.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.236+0000 7f24b6ffd640 1 -- 172.21.15.105:0/2913182969 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f24cc0210b0 con 0x558f9d5b6420 2024-03-26T17:41:45.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.237+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c80be420 con 0x558f9d5b6420 2024-03-26T17:41:45.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.237+0000 7f24b6ffd640 1 -- 172.21.15.105:0/2913182969 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24cc0212b0 con 0x558f9d5b6420 2024-03-26T17:41:45.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.237+0000 7f24e0996740 10 client.16686 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.237+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f9d5c70c0 con 0x558f9d5b6420 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.237+0000 7f24b6ffd640 1 -- 172.21.15.105:0/2913182969 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c80be420 con 0x558f9d5b6420 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.238+0000 7f24b6ffd640 1 -- 172.21.15.105:0/2913182969 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f24cc02aaf0 con 0x558f9d5b6420 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.238+0000 7f24e0996740 10 client.16686 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.238+0000 7f24e0996740 10 client.16686 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.238+0000 7f24e0996740 20 client.16686 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.238+0000 7f24e0996740 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558f9d632250 0x558f9d634670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.238+0000 7f24e0996740 4 client.16686 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.238+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f9d522200 con 0x558f9d632250 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.238+0000 7f24d77fe640 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558f9d632250 0x558f9d634670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.239+0000 7f24d77fe640 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558f9d632250 0x558f9d634670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6859438fd79c212 server_cookie=2a1512d772bc6742 in_seq=0 out_seq=0 2024-03-26T17:41:45.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.239+0000 7f24b6ffd640 10 client.16686 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:45.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.241+0000 7f24b6ffd640 1 -- 172.21.15.105:0/2913182969 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f9d522200 con 0x558f9d632250 2024-03-26T17:41:45.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.241+0000 7f24b6ffd640 10 client.16686 handle_command_reply: tid=0 2024-03-26T17:41:45.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.241+0000 7f24e0996740 1 client.16686 shutdown 2024-03-26T17:41:45.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.241+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 msgr2=0x558f9d5c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.241+0000 7f24e0996740 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c4720 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f24cc004700 tx=0x7f24cc012c40 comp rx=0 tx=0).stop 2024-03-26T17:41:45.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 shutdown_connections 2024-03-26T17:41:45.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558f9d632250 0x558f9d634670 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558f9d5bb590 0x558f9d5c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558f9d5b6420 0x558f9d5c4720 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24b6ffd640 0 client.16686 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:41:45.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 1 --2- 172.21.15.105:0/2913182969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558f9d5b5a30 0x558f9d5c41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 >> 172.21.15.105:0/2913182969 conn(0x558f9d5b0fa0 msgr2=0x558f9d5bf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 shutdown_connections 2024-03-26T17:41:45.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 1 -- 172.21.15.105:0/2913182969 wait complete. 2024-03-26T17:41:45.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.242+0000 7f24e0996740 20 client.16686 trim_cache size 0 max 16384 2024-03-26T17:41:45.235 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 -- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24c00a2f20 msgr2=0x7f24c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24c00a2f20 0x7f24c00a53e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x558f9d5b6c80 tx=0x7f24d0004a40 comp rx=0 tx=0).stop 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 -- 172.21.15.105:0/1521040444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d81007c0 msgr2=0x7f24d810afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d81007c0 0x7f24d810afc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f24c8000c00 tx=0x7f24c8005dc0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 -- 172.21.15.105:0/1521040444 shutdown_connections 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24c00a2f20 0x7f24c00a53e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d8102370 0x7f24d8109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d81019b0 0x7f24d8109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 --2- 172.21.15.105:0/1521040444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d81007c0 0x7f24d810afc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 -- 172.21.15.105:0/1521040444 >> 172.21.15.105:0/1521040444 conn(0x7f24d80fbd50 msgr2=0x7f24d8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 -- 172.21.15.105:0/1521040444 shutdown_connections 2024-03-26T17:41:45.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.243+0000 7f24dddb3640 1 -- 172.21.15.105:0/1521040444 wait complete. 2024-03-26T17:41:45.246 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:41:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.262+0000 7fc5f4ff9640 1 -- 172.21.15.105:0/4150498532 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbb4375940 con 0x55cbb4485a10 2024-03-26T17:41:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.262+0000 7fc5f4ff9640 10 client.16680 handle_command_reply: tid=0 2024-03-26T17:41:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.262+0000 7fc61638d740 1 client.16680 shutdown 2024-03-26T17:41:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.262+0000 7fc61638d740 1 -- 172.21.15.105:0/4150498532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb4409fc0 msgr2=0x55cbb4418140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.262+0000 7fc61638d740 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb4409fc0 0x55cbb4418140 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ba710 tx=0x7fc5fc0bd640 comp rx=0 tx=0).stop 2024-03-26T17:41:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 1 -- 172.21.15.105:0/4150498532 shutdown_connections 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cbb4485a10 0x55cbb4487e30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cbb440f130 0x55cbb4419150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cbb4409fc0 0x55cbb4418140 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 1 --2- 172.21.15.105:0/4150498532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cbb44095d0 0x55cbb4417c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc5f4ff9640 0 client.16680 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 1 -- 172.21.15.105:0/4150498532 >> 172.21.15.105:0/4150498532 conn(0x55cbb4404b40 msgr2=0x55cbb440a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 1 -- 172.21.15.105:0/4150498532 shutdown_connections 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 1 -- 172.21.15.105:0/4150498532 wait complete. 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc61638d740 20 client.16680 trim_cache size 0 max 16384 2024-03-26T17:41:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:41:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:41:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:41:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:41:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc6137a9640 1 -- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5f80a2fc0 msgr2=0x7fc5f80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc6137a9640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5f80a2fc0 0x7fc5f80a5480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc604009370 tx=0x7fc604009910 comp rx=0 tx=0).stop 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc6137a9640 1 -- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 msgr2=0x7fc60c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.263+0000 7fc6137a9640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 0x7fc60c198010 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009230 tx=0x7fc5fc004040 comp rx=0 tx=0).stop 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.264+0000 7fc6137a9640 1 -- 172.21.15.105:0/432480741 shutdown_connections 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.264+0000 7fc6137a9640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc5f80a2fc0 0x7fc5f80a5480 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.264+0000 7fc6137a9640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc60c103580 0x7fc60c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.264+0000 7fc6137a9640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc60c102bc0 0x7fc60c198010 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.264+0000 7fc6137a9640 1 --2- 172.21.15.105:0/432480741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c0ffbf0 0x7fc60c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.264+0000 7fc6137a9640 1 -- 172.21.15.105:0/432480741 >> 172.21.15.105:0/432480741 conn(0x7fc60c0fb520 msgr2=0x7fc60c105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.264+0000 7fc6137a9640 1 -- 172.21.15.105:0/432480741 shutdown_connections 2024-03-26T17:41:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:45.264+0000 7fc6137a9640 1 -- 172.21.15.105:0/432480741 wait complete. 2024-03-26T17:41:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 -- 172.21.15.105:0/2549736761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c100190 msgr2=0x7f7d7c1005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 --2- 172.21.15.105:0/2549736761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c100190 0x7f7d7c1005b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009900 tx=0x7f7d6c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 -- 172.21.15.105:0/2549736761 shutdown_connections 2024-03-26T17:41:45.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 --2- 172.21.15.105:0/2549736761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c101490 0x7f7d7c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 --2- 172.21.15.105:0/2549736761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c100af0 0x7f7d7c100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 --2- 172.21.15.105:0/2549736761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c100190 0x7f7d7c1005b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 -- 172.21.15.105:0/2549736761 >> 172.21.15.105:0/2549736761 conn(0x7f7d7c0fbd50 msgr2=0x7f7d7c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 -- 172.21.15.105:0/2549736761 shutdown_connections 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.383+0000 7f7d82019640 1 -- 172.21.15.105:0/2549736761 wait complete. 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d82019640 1 Processor -- start 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d82019640 1 -- start start 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d82019640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c100190 0x7f7d7c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d82019640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c100af0 0x7f7d7c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d82019640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c101490 0x7f7d7c1a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d82019640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c10f5d0 con 0x7f7d7c101490 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d82019640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c10f450 con 0x7f7d7c100af0 2024-03-26T17:41:45.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d82019640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c10f750 con 0x7f7d7c100190 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c101490 0x7f7d7c1a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c101490 0x7f7d7c1a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41844/0 (socket says 172.21.15.105:41844) 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d7bfff640 1 -- 172.21.15.105:0/3309507379 learned_addr learned my addr 172.21.15.105:0/3309507379 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d7affd640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c100af0 0x7f7d7c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.384+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c100190 0x7f7d7c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d7bfff640 1 -- 172.21.15.105:0/3309507379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c100190 msgr2=0x7f7d7c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d7bfff640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c100190 0x7f7d7c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d7bfff640 1 -- 172.21.15.105:0/3309507379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c100af0 msgr2=0x7f7d7c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d7bfff640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c100af0 0x7f7d7c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d7bfff640 1 -- 172.21.15.105:0/3309507379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c1a1960 con 0x7f7d7c101490 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d7bfff640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c101490 0x7f7d7c1a12c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400eb10 tx=0x7f7d7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d78ff9640 1 -- 172.21.15.105:0/3309507379 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74004b20 con 0x7f7d7c101490 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d82019640 1 -- 172.21.15.105:0/3309507379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c1a1c50 con 0x7f7d7c101490 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d78ff9640 1 -- 172.21.15.105:0/3309507379 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d74004cc0 con 0x7f7d7c101490 2024-03-26T17:41:45.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d78ff9640 1 -- 172.21.15.105:0/3309507379 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740106c0 con 0x7f7d7c101490 2024-03-26T17:41:45.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.385+0000 7f7d82019640 1 -- 172.21.15.105:0/3309507379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1a21a0 con 0x7f7d7c101490 2024-03-26T17:41:45.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.388+0000 7f7d84c08740 1 Processor -- start 2024-03-26T17:41:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.389+0000 7f7d78ff9640 1 -- 172.21.15.105:0/3309507379 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7d74005000 con 0x7f7d7c101490 2024-03-26T17:41:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.389+0000 7f7d84c08740 1 -- start start 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.389+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.389+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f3420 0x55cff49f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.389+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.389+0000 7f7d84c08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cff4957970 con 0x55cff49f8590 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.389+0000 7f7d84c08740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cff48ffd70 con 0x55cff49f3420 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.389+0000 7f7d84c08740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cff4953920 con 0x55cff49f2a30 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f3420 0x55cff49f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f3420 0x55cff49f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58460/0 (socket says 172.21.15.105:58460) 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/4003322228 learned_addr learned my addr 172.21.15.105:0/4003322228 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45114/0 (socket says 172.21.15.105:45114) 2024-03-26T17:41:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d78ff9640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d640a2f20 0x7f7d640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7affd640 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7bfff640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d640a2f20 0x7f7d640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d78ff9640 1 -- 172.21.15.105:0/3309507379 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7d740cfb80 con 0x7f7d7c101490 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/4003322228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 msgr2=0x55cff49f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/4003322228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 msgr2=0x55cff49faa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49faa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/4003322228 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff466b3f0 con 0x55cff49f3420 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7affd640 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7bfff640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d640a2f20 0x7f7d640a53e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7d740049d0 tx=0x7f7d740c6f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f3420 0x55cff49f7f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c02c170 tx=0x7f7d6c0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca35b696eff96c8e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.390+0000 7f7d61ffb640 1 -- 172.21.15.105:0/4003322228 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c045020 con 0x55cff49f3420 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d61ffb640 1 -- 172.21.15.105:0/4003322228 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d6c004040 con 0x55cff49f3420 2024-03-26T17:41:45.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d61ffb640 1 -- 172.21.15.105:0/4003322228 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c005510 con 0x55cff49f3420 2024-03-26T17:41:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d84c08740 1 -- 172.21.15.105:0/4003322228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f3420 msgr2=0x55cff49f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d84c08740 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f3420 0x55cff49f7f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c02c170 tx=0x7f7d6c0095f0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d84c08740 1 -- 172.21.15.105:0/4003322228 shutdown_connections 2024-03-26T17:41:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d84c08740 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d84c08740 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f3420 0x55cff49f7f20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d84c08740 1 --2- 172.21.15.105:0/4003322228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.391+0000 7f7d84c08740 1 -- 172.21.15.105:0/4003322228 >> 172.21.15.105:0/4003322228 conn(0x55cff49edfa0 msgr2=0x55cff49ee3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 -- 172.21.15.105:0/4003322228 shutdown_connections 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 -- 172.21.15.105:0/4003322228 wait complete. 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 -- start start 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 Processor -- start 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff49fd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a044a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cff49fb4e0 con 0x55cff49f3420 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cff4913310 con 0x55cff49f8590 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cff49fb660 con 0x55cff49f2a30 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d84c08740 10 client.0 _ll_register_callbacks cb 0x55cff494ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff49fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d7affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d7affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49fcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45126/0 (socket says 172.21.15.105:45126) 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d7affd640 1 -- 172.21.15.105:0/3493568771 learned_addr learned my addr 172.21.15.105:0/3493568771 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.392+0000 7f7d7affd640 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/3493568771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 msgr2=0x55cff49fcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49fcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/3493568771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 msgr2=0x55cff4a044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a044a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/3493568771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff47357d0 con 0x55cff49f3420 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff49fd460 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55cff49eef10 tx=0x7f7d6c0054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6615f0e8bf7814 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d5affd640 1 -- 172.21.15.105:0/3493568771 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c045040 con 0x55cff49f3420 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d5affd640 1 -- 172.21.15.105:0/3493568771 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7d6c0046c0 con 0x55cff49f3420 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d84c08740 1 -- 172.21.15.105:0/3493568771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6c004040 con 0x55cff49f3420 2024-03-26T17:41:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d5affd640 1 -- 172.21.15.105:0/3493568771 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d6c05b050 con 0x55cff49f3420 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d84c08740 10 client.16698 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.393+0000 7f7d84c08740 1 -- 172.21.15.105:0/3493568771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cff4a07810 con 0x55cff49f3420 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d5affd640 1 -- 172.21.15.105:0/3493568771 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d6c004040 con 0x55cff49f3420 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d5affd640 1 -- 172.21.15.105:0/3493568771 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7d6c003780 con 0x55cff49f3420 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d84c08740 10 client.16698 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d84c08740 10 client.16698 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d84c08740 20 client.16698 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d84c08740 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55cff4a08b60 0x55cff4a0af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d84c08740 4 client.16698 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d84c08740 1 -- 172.21.15.105:0/3493568771 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cff4964eb0 con 0x55cff4a08b60 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d7affd640 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55cff4a08b60 0x55cff4a0af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d5affd640 10 client.16698 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.394+0000 7f7d7affd640 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55cff4a08b60 0x55cff4a0af50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4e977d103d0004c server_cookie=33738622fd096731 in_seq=0 out_seq=0 2024-03-26T17:41:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d5affd640 1 -- 172.21.15.105:0/3493568771 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cff4964eb0 con 0x55cff4a08b60 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d5affd640 10 client.16698 handle_command_reply: tid=0 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 client.16698 shutdown 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 -- 172.21.15.105:0/3493568771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 msgr2=0x55cff49fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff49fd460 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55cff49eef10 tx=0x7f7d6c0054c0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 -- 172.21.15.105:0/3493568771 shutdown_connections 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55cff4a08b60 0x55cff4a0af50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d5affd640 0 client.16698 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff49fd460 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 --2- 172.21.15.105:0/3493568771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff49fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 -- 172.21.15.105:0/3493568771 >> 172.21.15.105:0/3493568771 conn(0x55cff49edfa0 msgr2=0x55cff49ed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 -- 172.21.15.105:0/3493568771 shutdown_connections 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.395+0000 7f7d84c08740 1 -- 172.21.15.105:0/3493568771 wait complete. 2024-03-26T17:41:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.396+0000 7f7d84c08740 20 client.16698 trim_cache size 0 max 16384 2024-03-26T17:41:45.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d84c08740 1 Processor -- start 2024-03-26T17:41:45.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d84c08740 1 -- start start 2024-03-26T17:41:45.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f2a30 0x55cff49ec930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f3420 0x55cff4a04460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49ece70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d84c08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1aaf10 con 0x55cff49f8590 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d84c08740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1aca50 con 0x55cff49f2a30 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d84c08740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ab5e0 con 0x55cff49f3420 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f2a30 0x55cff49ec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f2a30 0x55cff49ec930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58480/0 (socket says 172.21.15.105:58480) 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f3420 0x55cff4a04460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7affd640 1 -- 172.21.15.105:0/522883805 learned_addr learned my addr 172.21.15.105:0/522883805 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7affd640 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49ece70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7affd640 1 -- 172.21.15.105:0/522883805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f3420 msgr2=0x55cff4a04460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7affd640 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f3420 0x55cff4a04460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7affd640 1 -- 172.21.15.105:0/522883805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 msgr2=0x55cff49ece70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.410+0000 7f7d7affd640 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49ece70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.411+0000 7f7d7affd640 1 -- 172.21.15.105:0/522883805 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff49e9820 con 0x55cff49f2a30 2024-03-26T17:41:45.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.411+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f3420 0x55cff4a04460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.411+0000 7f7d7affd640 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f2a30 0x55cff49ec930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55cff4a0b790 tx=0x7f7d70004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc4bcbdfb7583d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.411+0000 7f7d617fa640 1 -- 172.21.15.105:0/522883805 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700117d0 con 0x55cff49f2a30 2024-03-26T17:41:45.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.411+0000 7f7d617fa640 1 -- 172.21.15.105:0/522883805 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d70011e20 con 0x55cff49f2a30 2024-03-26T17:41:45.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.411+0000 7f7d617fa640 1 -- 172.21.15.105:0/522883805 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700035f0 con 0x55cff49f2a30 2024-03-26T17:41:45.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- 172.21.15.105:0/522883805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f2a30 msgr2=0x55cff49ec930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f2a30 0x55cff49ec930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55cff4a0b790 tx=0x7f7d70004610 comp rx=0 tx=0).stop 2024-03-26T17:41:45.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- 172.21.15.105:0/522883805 shutdown_connections 2024-03-26T17:41:45.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f8590 0x55cff49ece70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f3420 0x55cff4a04460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 --2- 172.21.15.105:0/522883805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f2a30 0x55cff49ec930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- 172.21.15.105:0/522883805 >> 172.21.15.105:0/522883805 conn(0x55cff49edfa0 msgr2=0x55cff49fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- 172.21.15.105:0/522883805 shutdown_connections 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- 172.21.15.105:0/522883805 wait complete. 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- start start 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 Processor -- start 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff4a016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff4a01c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a02c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ac380 con 0x55cff49f3420 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1abcb0 con 0x55cff49f8590 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1a8ef0 con 0x55cff49f2a30 2024-03-26T17:41:45.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.412+0000 7f7d84c08740 10 client.0 _ll_register_callbacks cb 0x7f7d70009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff4a01c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff4a01c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41880/0 (socket says 172.21.15.105:41880) 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/2321021741 learned_addr learned my addr 172.21.15.105:0/2321021741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7affd640 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a02c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7affd640 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff4a016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/2321021741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 msgr2=0x55cff4a016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff4a016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/2321021741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 msgr2=0x55cff4a02c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a02c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 -- 172.21.15.105:0/2321021741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff4a03230 con 0x55cff49f3420 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7affd640 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a02c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d7b7fe640 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff4a01c30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009900 tx=0x7f7d6c004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7192aad061cfce83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:45.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d5affd640 1 -- 172.21.15.105:0/2321021741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c045040 con 0x55cff49f3420 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d5affd640 1 -- 172.21.15.105:0/2321021741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7d6c0035f0 con 0x55cff49f3420 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d84c08740 1 -- 172.21.15.105:0/2321021741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d70011e20 con 0x55cff49f3420 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d5affd640 1 -- 172.21.15.105:0/2321021741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7d6c067050 con 0x55cff49f3420 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d84c08740 10 client.16704 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.413+0000 7f7d84c08740 1 -- 172.21.15.105:0/2321021741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cff4a04530 con 0x55cff49f3420 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d5affd640 1 -- 172.21.15.105:0/2321021741 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d70011e20 con 0x55cff49f3420 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d5affd640 1 -- 172.21.15.105:0/2321021741 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7d6c05e070 con 0x55cff49f3420 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d84c08740 10 client.16704 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d84c08740 10 client.16704 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d84c08740 20 client.16704 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d84c08740 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55cff4a6f1e0 0x55cff4a715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d84c08740 4 client.16704 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d84c08740 1 -- 172.21.15.105:0/2321021741 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cff495f200 con 0x55cff4a6f1e0 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d7affd640 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55cff4a6f1e0 0x55cff4a715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d5affd640 10 client.16704 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.414+0000 7f7d7affd640 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55cff4a6f1e0 0x55cff4a715d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=80cac571fda8b89b server_cookie=5a95dc34df02ad3 in_seq=0 out_seq=0 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d5affd640 1 -- 172.21.15.105:0/2321021741 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cff495f200 con 0x55cff4a6f1e0 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d5affd640 10 client.16704 handle_command_reply: tid=0 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 client.16704 shutdown 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 -- 172.21.15.105:0/2321021741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 msgr2=0x55cff4a01c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff4a01c30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009900 tx=0x7f7d6c004070 comp rx=0 tx=0).stop 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 -- 172.21.15.105:0/2321021741 shutdown_connections 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55cff4a6f1e0 0x55cff4a715d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cff49f8590 0x55cff4a02c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cff49f3420 0x55cff4a01c30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 --2- 172.21.15.105:0/2321021741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cff49f2a30 0x55cff4a016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 -- 172.21.15.105:0/2321021741 >> 172.21.15.105:0/2321021741 conn(0x55cff49edfa0 msgr2=0x55cff49f1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d5affd640 0 client.16704 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:41:45.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 -- 172.21.15.105:0/2321021741 shutdown_connections 2024-03-26T17:41:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 1 -- 172.21.15.105:0/2321021741 wait complete. 2024-03-26T17:41:45.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.415+0000 7f7d84c08740 20 client.16704 trim_cache size 0 max 16384 2024-03-26T17:41:45.415 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 -- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d640a2f20 msgr2=0x7f7d640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d640a2f20 0x7f7d640a53e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7d740049d0 tx=0x7f7d740c6f30 comp rx=0 tx=0).stop 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 -- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c101490 msgr2=0x7f7d7c1a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c101490 0x7f7d7c1a12c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400eb10 tx=0x7f7d7400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 -- 172.21.15.105:0/3309507379 shutdown_connections 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d640a2f20 0x7f7d640a53e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d7c101490 0x7f7d7c1a12c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c100af0 0x7f7d7c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 --2- 172.21.15.105:0/3309507379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c100190 0x7f7d7c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 -- 172.21.15.105:0/3309507379 >> 172.21.15.105:0/3309507379 conn(0x7f7d7c0fbd50 msgr2=0x7f7d7c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 -- 172.21.15.105:0/3309507379 shutdown_connections 2024-03-26T17:41:45.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.416+0000 7f7d82019640 1 -- 172.21.15.105:0/3309507379 wait complete. 2024-03-26T17:41:45.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:41:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 -- 172.21.15.105:0/2594703095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c1007c0 msgr2=0x7f1a1c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 --2- 172.21.15.105:0/2594703095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c1007c0 0x7f1a1c100be0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009900 tx=0x7f1a1802bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 -- 172.21.15.105:0/2594703095 shutdown_connections 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 --2- 172.21.15.105:0/2594703095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a1c102370 0x7f1a1c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 --2- 172.21.15.105:0/2594703095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c1019b0 0x7f1a1c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 --2- 172.21.15.105:0/2594703095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c1007c0 0x7f1a1c100be0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 -- 172.21.15.105:0/2594703095 >> 172.21.15.105:0/2594703095 conn(0x7f1a1c0fbd50 msgr2=0x7f1a1c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 -- 172.21.15.105:0/2594703095 shutdown_connections 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.604+0000 7f1a2360b640 1 -- 172.21.15.105:0/2594703095 wait complete. 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a2360b640 1 Processor -- start 2024-03-26T17:41:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a2360b640 1 -- start start 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a2360b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a1c1007c0 0x7f1a1c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a2360b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c1019b0 0x7f1a1c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a2360b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c102370 0x7f1a1c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a2360b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c10c2c0 con 0x7f1a1c102370 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a2360b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c10c140 con 0x7f1a1c1019b0 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a2360b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c10c440 con 0x7f1a1c1007c0 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a21ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c102370 0x7f1a1c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a1c1007c0 0x7f1a1c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a1c1007c0 0x7f1a1c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45164/0 (socket says 172.21.15.105:45164) 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a21ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c102370 0x7f1a1c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41894/0 (socket says 172.21.15.105:41894) 2024-03-26T17:41:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a213a7640 1 -- 172.21.15.105:0/17694740 learned_addr learned my addr 172.21.15.105:0/17694740 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c1019b0 0x7f1a1c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a213a7640 1 -- 172.21.15.105:0/17694740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c1019b0 msgr2=0x7f1a1c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a213a7640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c1019b0 0x7f1a1c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a213a7640 1 -- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c102370 msgr2=0x7f1a1c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a213a7640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c102370 0x7f1a1c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a213a7640 1 -- 172.21.15.105:0/17694740 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c19f770 con 0x7f1a1c1007c0 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.605+0000 7f1a21ba8640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c102370 0x7f1a1c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.606+0000 7f1a213a7640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a1c1007c0 0x7f1a1c197b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a18000c00 tx=0x7f1a18005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.606+0000 7f1a127fc640 1 -- 172.21.15.105:0/17694740 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18043020 con 0x7f1a1c1007c0 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.606+0000 7f1a2360b640 1 -- 172.21.15.105:0/17694740 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1c19fa00 con 0x7f1a1c1007c0 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.606+0000 7f1a127fc640 1 -- 172.21.15.105:0/17694740 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a18004350 con 0x7f1a1c1007c0 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.606+0000 7f1a127fc640 1 -- 172.21.15.105:0/17694740 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18003b80 con 0x7f1a1c1007c0 2024-03-26T17:41:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.606+0000 7f1a2360b640 1 -- 172.21.15.105:0/17694740 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1c19ffa0 con 0x7f1a1c1007c0 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.608+0000 7f1a127fc640 1 -- 172.21.15.105:0/17694740 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1a18046020 con 0x7f1a1c1007c0 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a261fb740 1 Processor -- start 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a127fc640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a080a2f20 0x7f1a080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a261fb740 1 -- start start 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca23f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca26a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a261fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556b4c983970 con 0x556b4ca24590 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a261fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556b4c92bd70 con 0x556b4ca1ea30 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a261fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556b4c97f920 con 0x556b4ca1f420 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a127fc640 1 -- 172.21.15.105:0/17694740 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1a180f1580 con 0x7f1a1c1007c0 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a21ba8640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a080a2f20 0x7f1a080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a20ba6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca23f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca1ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58522/0 (socket says 172.21.15.105:58522) 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a20ba6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca23f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45170/0 (socket says 172.21.15.105:45170) 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a213a7640 1 -- 172.21.15.105:0/683432846 learned_addr learned my addr 172.21.15.105:0/683432846 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a21ba8640 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca26a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a213a7640 1 -- 172.21.15.105:0/683432846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 msgr2=0x556b4ca23f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a213a7640 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca23f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a213a7640 1 -- 172.21.15.105:0/683432846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 msgr2=0x556b4ca26a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a213a7640 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca26a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a213a7640 1 -- 172.21.15.105:0/683432846 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4c6973f0 con 0x556b4ca1ea30 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.609+0000 7f1a21ba8640 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca26a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.610+0000 7f1a213a7640 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca1ee50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a180ba0d0 tx=0x7f1a180ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df7b8e6cd3afe59 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.610+0000 7f1a21ba8640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a080a2f20 0x7f1a080a53e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1a14005fd0 tx=0x7f1a14009ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.610+0000 7f1a077fe640 1 -- 172.21.15.105:0/683432846 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18008b10 con 0x556b4ca1ea30 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.610+0000 7f1a077fe640 1 -- 172.21.15.105:0/683432846 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a180be370 con 0x556b4ca1ea30 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.610+0000 7f1a077fe640 1 -- 172.21.15.105:0/683432846 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18000ed0 con 0x556b4ca1ea30 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.610+0000 7f1a261fb740 1 -- 172.21.15.105:0/683432846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 msgr2=0x556b4ca1ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.610+0000 7f1a261fb740 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca1ee50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1a180ba0d0 tx=0x7f1a180ba040 comp rx=0 tx=0).stop 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 -- 172.21.15.105:0/683432846 shutdown_connections 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca26a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca23f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 --2- 172.21.15.105:0/683432846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca1ee50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 -- 172.21.15.105:0/683432846 >> 172.21.15.105:0/683432846 conn(0x556b4ca19fa0 msgr2=0x556b4ca1a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 -- 172.21.15.105:0/683432846 shutdown_connections 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 -- 172.21.15.105:0/683432846 wait complete. 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 -- start start 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.611+0000 7f1a261fb740 1 Processor -- start 2024-03-26T17:41:45.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca28ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1f420 0x556b4ca291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca24590 0x556b4ca30220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a261fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556b4ca27300 con 0x556b4ca1ea30 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a261fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556b4c93f310 con 0x556b4ca1f420 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a261fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556b4ca27480 con 0x556b4ca24590 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a261fb740 10 client.0 _ll_register_callbacks cb 0x556b4c97ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca28ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca28ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41912/0 (socket says 172.21.15.105:41912) 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 -- 172.21.15.105:0/108720 learned_addr learned my addr 172.21.15.105:0/108720 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a21ba8640 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca24590 0x556b4ca30220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1f420 0x556b4ca291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 -- 172.21.15.105:0/108720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca24590 msgr2=0x556b4ca30220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca24590 0x556b4ca30220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 -- 172.21.15.105:0/108720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1f420 msgr2=0x556b4ca291e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1f420 0x556b4ca291e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 -- 172.21.15.105:0/108720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4c7617d0 con 0x556b4ca1ea30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a213a7640 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca28ca0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1a1802c840 tx=0x7f1a18005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf873d55917d8e83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a04ff9640 1 -- 172.21.15.105:0/108720 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a180010b0 con 0x556b4ca1ea30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a04ff9640 1 -- 172.21.15.105:0/108720 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1a18003d20 con 0x556b4ca1ea30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a261fb740 1 -- 172.21.15.105:0/108720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a180be370 con 0x556b4ca1ea30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.612+0000 7f1a04ff9640 1 -- 172.21.15.105:0/108720 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a180bdc70 con 0x556b4ca1ea30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a261fb740 10 client.16716 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a261fb740 1 -- 172.21.15.105:0/108720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b4ca27300 con 0x556b4ca1ea30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a04ff9640 1 -- 172.21.15.105:0/108720 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a180be370 con 0x556b4ca1ea30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a04ff9640 1 -- 172.21.15.105:0/108720 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1a1809eba0 con 0x556b4ca1ea30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a261fb740 10 client.16716 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a261fb740 10 client.16716 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a261fb740 20 client.16716 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a261fb740 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b4ca34a00 0x556b4ca36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a261fb740 4 client.16716 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a261fb740 1 -- 172.21.15.105:0/108720 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b4c990eb0 con 0x556b4ca34a00 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.613+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b4ca34a00 0x556b4ca36e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.614+0000 7f1a04ff9640 10 client.16716 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.614+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b4ca34a00 0x556b4ca36e50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=22d7334631724b45 server_cookie=ff4ffbb63af0fe47 in_seq=0 out_seq=0 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.614+0000 7f1a04ff9640 1 -- 172.21.15.105:0/108720 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556b4c990eb0 con 0x556b4ca34a00 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.614+0000 7f1a04ff9640 10 client.16716 handle_command_reply: tid=0 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.614+0000 7f1a261fb740 1 client.16716 shutdown 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 -- 172.21.15.105:0/108720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 msgr2=0x556b4ca28ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca28ca0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1a1802c840 tx=0x7f1a18005fd0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 -- 172.21.15.105:0/108720 shutdown_connections 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca24590 0x556b4ca30220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b4ca34a00 0x556b4ca36e50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1f420 0x556b4ca291e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 --2- 172.21.15.105:0/108720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca28ca0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 -- 172.21.15.105:0/108720 >> 172.21.15.105:0/108720 conn(0x556b4ca19fa0 msgr2=0x556b4ca191c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a04ff9640 0 client.16716 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 -- 172.21.15.105:0/108720 shutdown_connections 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 1 -- 172.21.15.105:0/108720 wait complete. 2024-03-26T17:41:45.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.615+0000 7f1a261fb740 20 client.16716 trim_cache size 0 max 16384 2024-03-26T17:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a261fb740 1 Processor -- start 2024-03-26T17:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a261fb740 1 -- start start 2024-03-26T17:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca36a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca28fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a261fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1a8e30 con 0x556b4ca24590 2024-03-26T17:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a261fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1aa970 con 0x556b4ca1ea30 2024-03-26T17:41:45.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a261fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1a9500 con 0x556b4ca1f420 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca36a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a21ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca28fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a20ba6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca36a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58542/0 (socket says 172.21.15.105:58542) 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.629+0000 7f1a213a7640 1 -- 172.21.15.105:0/4078969959 learned_addr learned my addr 172.21.15.105:0/4078969959 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a20ba6640 1 -- 172.21.15.105:0/4078969959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 msgr2=0x556b4ca36a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/4078969959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca36a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a20ba6640 1 -- 172.21.15.105:0/4078969959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 msgr2=0x556b4ca28fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/4078969959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca28fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a20ba6640 1 -- 172.21.15.105:0/4078969959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4ca26ee0 con 0x556b4ca1f420 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/4078969959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca301b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x556b4ca37690 tx=0x7f1a0c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391576517bc0a5e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a06ffd640 1 -- 172.21.15.105:0/4078969959 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c01f630 con 0x556b4ca1f420 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a06ffd640 1 -- 172.21.15.105:0/4078969959 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0c01f7d0 con 0x556b4ca1f420 2024-03-26T17:41:45.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.630+0000 7f1a06ffd640 1 -- 172.21.15.105:0/4078969959 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c01e390 con 0x556b4ca1f420 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 -- 172.21.15.105:0/4078969959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 msgr2=0x556b4ca301b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 --2- 172.21.15.105:0/4078969959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca301b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x556b4ca37690 tx=0x7f1a0c005f90 comp rx=0 tx=0).stop 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 -- 172.21.15.105:0/4078969959 shutdown_connections 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 --2- 172.21.15.105:0/4078969959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca24590 0x556b4ca28fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 --2- 172.21.15.105:0/4078969959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca301b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 --2- 172.21.15.105:0/4078969959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca1ea30 0x556b4ca36a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 -- 172.21.15.105:0/4078969959 >> 172.21.15.105:0/4078969959 conn(0x556b4ca19fa0 msgr2=0x556b4ca288b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 -- 172.21.15.105:0/4078969959 shutdown_connections 2024-03-26T17:41:45.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 -- 172.21.15.105:0/4078969959 wait complete. 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 -- start start 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.631+0000 7f1a261fb740 1 Processor -- start 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca2d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca2d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a261fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca24590 0x556b4ca98d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a261fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1aa2a0 con 0x556b4ca1ea30 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a261fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1a9bd0 con 0x556b4ca24590 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a261fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1a6e10 con 0x556b4ca1f420 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a261fb740 10 client.0 _ll_register_callbacks cb 0x7f1a0c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca2d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca2d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca2d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45192/0 (socket says 172.21.15.105:45192) 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a213a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca2d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41938/0 (socket says 172.21.15.105:41938) 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 -- 172.21.15.105:0/1728336118 learned_addr learned my addr 172.21.15.105:0/1728336118 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a21ba8640 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca24590 0x556b4ca98d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 -- 172.21.15.105:0/1728336118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca24590 msgr2=0x556b4ca98d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca24590 0x556b4ca98d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 -- 172.21.15.105:0/1728336118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 msgr2=0x556b4ca2d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca2d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 -- 172.21.15.105:0/1728336118 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4ca99300 con 0x556b4ca1f420 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a213a7640 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca2d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a20ba6640 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca2d710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009380 tx=0x7f1a0c01f430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37f571e7c63d61c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:45.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a04ff9640 1 -- 172.21.15.105:0/1728336118 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c005030 con 0x556b4ca1f420 2024-03-26T17:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a04ff9640 1 -- 172.21.15.105:0/1728336118 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1a0c0051d0 con 0x556b4ca1f420 2024-03-26T17:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.632+0000 7f1a261fb740 1 -- 172.21.15.105:0/1728336118 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a0c01f7d0 con 0x556b4ca1f420 2024-03-26T17:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a04ff9640 1 -- 172.21.15.105:0/1728336118 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a0c039050 con 0x556b4ca1f420 2024-03-26T17:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a261fb740 10 client.26563 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a261fb740 1 -- 172.21.15.105:0/1728336118 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b4ca9a630 con 0x556b4ca1f420 2024-03-26T17:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a04ff9640 1 -- 172.21.15.105:0/1728336118 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a0c01f7d0 con 0x556b4ca1f420 2024-03-26T17:41:45.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a04ff9640 1 -- 172.21.15.105:0/1728336118 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1a0c003d00 con 0x556b4ca1f420 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a261fb740 10 client.26563 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a261fb740 10 client.26563 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a261fb740 20 client.26563 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a261fb740 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b4ca9afa0 0x556b4ca9d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a261fb740 4 client.26563 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a261fb740 1 -- 172.21.15.105:0/1728336118 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b4ca371e0 con 0x556b4ca9afa0 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.633+0000 7f1a213a7640 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b4ca9afa0 0x556b4ca9d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a04ff9640 10 client.26563 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a213a7640 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b4ca9afa0 0x556b4ca9d3c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d1f24077a71f8d8 server_cookie=1c3d0c47e6eec36e in_seq=0 out_seq=0 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a04ff9640 1 -- 172.21.15.105:0/1728336118 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b4ca371e0 con 0x556b4ca9afa0 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a04ff9640 10 client.26563 handle_command_reply: tid=0 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 client.26563 shutdown 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 -- 172.21.15.105:0/1728336118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 msgr2=0x556b4ca2d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca2d710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009380 tx=0x7f1a0c01f430 comp rx=0 tx=0).stop 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 -- 172.21.15.105:0/1728336118 shutdown_connections 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b4ca9afa0 0x556b4ca9d3c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b4ca24590 0x556b4ca98d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b4ca1f420 0x556b4ca2d710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a04ff9640 0 client.26563 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 --2- 172.21.15.105:0/1728336118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b4ca1ea30 0x556b4ca2d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 -- 172.21.15.105:0/1728336118 >> 172.21.15.105:0/1728336118 conn(0x556b4ca19fa0 msgr2=0x556b4ca1d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 -- 172.21.15.105:0/1728336118 shutdown_connections 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 1 -- 172.21.15.105:0/1728336118 wait complete. 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.634+0000 7f1a261fb740 20 client.26563 trim_cache size 0 max 16384 2024-03-26T17:41:45.635 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 -- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a080a2f20 msgr2=0x7f1a080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a080a2f20 0x7f1a080a53e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1a14005fd0 tx=0x7f1a14009ef0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 -- 172.21.15.105:0/17694740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a1c1007c0 msgr2=0x7f1a1c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a1c1007c0 0x7f1a1c197b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a18000c00 tx=0x7f1a18005dc0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 -- 172.21.15.105:0/17694740 shutdown_connections 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a080a2f20 0x7f1a080a53e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a1c102370 0x7f1a1c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a1c1019b0 0x7f1a1c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 --2- 172.21.15.105:0/17694740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a1c1007c0 0x7f1a1c197b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 -- 172.21.15.105:0/17694740 >> 172.21.15.105:0/17694740 conn(0x7f1a1c0fbd50 msgr2=0x7f1a1c068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 -- 172.21.15.105:0/17694740 shutdown_connections 2024-03-26T17:41:45.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.635+0000 7f1a2360b640 1 -- 172.21.15.105:0/17694740 wait complete. 2024-03-26T17:41:45.638 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:41:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 -- 172.21.15.105:0/2781694164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec540ffba0 msgr2=0x7fec540fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 --2- 172.21.15.105:0/2781694164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec540ffba0 0x7fec540fffc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fec48009930 tx=0x7fec4802bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 -- 172.21.15.105:0/2781694164 shutdown_connections 2024-03-26T17:41:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 --2- 172.21.15.105:0/2781694164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec54103530 0x7fec54109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 --2- 172.21.15.105:0/2781694164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec54102b70 0x7fec54102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 --2- 172.21.15.105:0/2781694164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec540ffba0 0x7fec540fffc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 -- 172.21.15.105:0/2781694164 >> 172.21.15.105:0/2781694164 conn(0x7fec540fb4b0 msgr2=0x7fec540fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 -- 172.21.15.105:0/2781694164 shutdown_connections 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.778+0000 7fec5ab32640 1 -- 172.21.15.105:0/2781694164 wait complete. 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec5ab32640 1 Processor -- start 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec5ab32640 1 -- start start 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec5ab32640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec540ffba0 0x7fec54102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec5ab32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec54102b70 0x7fec54100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec5ab32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec54103530 0x7fec54101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec5ab32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec5410d420 con 0x7fec54102b70 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec5ab32640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fec5410d2a0 con 0x7fec540ffba0 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec5ab32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec5410d5a0 con 0x7fec54103530 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec588ce640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec540ffba0 0x7fec54102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec54102b70 0x7fec54100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec588ce640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec540ffba0 0x7fec54102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58580/0 (socket says 172.21.15.105:58580) 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec588ce640 1 -- 172.21.15.105:0/1558224994 learned_addr learned my addr 172.21.15.105:0/1558224994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.779+0000 7fec590cf640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec54103530 0x7fec54101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec588ce640 1 -- 172.21.15.105:0/1558224994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec54103530 msgr2=0x7fec54101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec588ce640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec54103530 0x7fec54101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec588ce640 1 -- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec54102b70 msgr2=0x7fec54100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec588ce640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec54102b70 0x7fec54100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec588ce640 1 -- 172.21.15.105:0/1558224994 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec54101920 con 0x7fec540ffba0 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec588ce640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec540ffba0 0x7fec54102490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec48005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec51ffb640 1 -- 172.21.15.105:0/1558224994 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48043020 con 0x7fec540ffba0 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec5ab32640 1 -- 172.21.15.105:0/1558224994 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec541a1c80 con 0x7fec540ffba0 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec51ffb640 1 -- 172.21.15.105:0/1558224994 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec480043d0 con 0x7fec540ffba0 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec51ffb640 1 -- 172.21.15.105:0/1558224994 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48003b50 con 0x7fec540ffba0 2024-03-26T17:41:45.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.780+0000 7fec5ab32640 1 -- 172.21.15.105:0/1558224994 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec541a21a0 con 0x7fec540ffba0 2024-03-26T17:41:45.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec51ffb640 1 -- 172.21.15.105:0/1558224994 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fec48046020 con 0x7fec540ffba0 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec5d7f4740 1 Processor -- start 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec51ffb640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec440a2f20 0x7fec440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec5d7f4740 1 -- start start 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec590cf640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec440a2f20 0x7fec440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec51ffb640 1 -- 172.21.15.105:0/1558224994 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fec480f15c0 con 0x7fec540ffba0 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d06a30 0x55b7f2d06e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d0bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d0c590 0x55b7f2d0ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b7f2c6b970 con 0x55b7f2d07420 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7f2c13d70 con 0x55b7f2d06a30 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.783+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7f2c67920 con 0x55b7f2d0c590 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec588ce640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d06a30 0x55b7f2d06e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d0bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec588ce640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d06a30 0x55b7f2d06e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58584/0 (socket says 172.21.15.105:58584) 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec588ce640 1 -- 172.21.15.105:0/1459735373 learned_addr learned my addr 172.21.15.105:0/1459735373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec590cf640 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d0c590 0x55b7f2d0ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec53fff640 1 -- 172.21.15.105:0/1459735373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d0c590 msgr2=0x55b7f2d0ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec53fff640 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d0c590 0x55b7f2d0ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec53fff640 1 -- 172.21.15.105:0/1459735373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d06a30 msgr2=0x55b7f2d06e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec53fff640 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d06a30 0x55b7f2d06e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec53fff640 1 -- 172.21.15.105:0/1459735373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f297f3f0 con 0x55b7f2d07420 2024-03-26T17:41:45.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec588ce640 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d06a30 0x55b7f2d06e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec590cf640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec440a2f20 0x7fec440a53e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005fd0 tx=0x7fec4c011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec53fff640 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d0bf20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fec40009bd0 tx=0x7fec40011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5aaaf2abb337d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec3effd640 1 -- 172.21.15.105:0/1459735373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40004050 con 0x55b7f2d07420 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec3effd640 1 -- 172.21.15.105:0/1459735373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec400054d0 con 0x55b7f2d07420 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.784+0000 7fec3effd640 1 -- 172.21.15.105:0/1459735373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40004c10 con 0x55b7f2d07420 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1459735373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 msgr2=0x55b7f2d0bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d0bf20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fec40009bd0 tx=0x7fec40011810 comp rx=0 tx=0).stop 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1459735373 shutdown_connections 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d0c590 0x55b7f2d0ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d0bf20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1459735373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d06a30 0x55b7f2d06e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1459735373 >> 172.21.15.105:0/1459735373 conn(0x55b7f2d01fa0 msgr2=0x55b7f2d023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1459735373 shutdown_connections 2024-03-26T17:41:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1459735373 wait complete. 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 -- start start 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.785+0000 7fec5d7f4740 1 Processor -- start 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d10f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d11450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 0x55b7f2d18490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b7f2d0f540 con 0x55b7f2d07420 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7f2c27310 con 0x55b7f2d0c590 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7f2d0f6c0 con 0x55b7f2d06a30 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec5d7f4740 10 client.0 _ll_register_callbacks cb 0x55b7f2c62c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d11450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d11450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:41986/0 (socket says 172.21.15.105:41986) 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 -- 172.21.15.105:0/551210735 learned_addr learned my addr 172.21.15.105:0/551210735 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec588ce640 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d10f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec590cf640 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 0x55b7f2d18490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 -- 172.21.15.105:0/551210735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 msgr2=0x55b7f2d10f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d10f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 -- 172.21.15.105:0/551210735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 msgr2=0x55b7f2d18490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 0x55b7f2d18490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 -- 172.21.15.105:0/551210735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f2a497d0 con 0x55b7f2d07420 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec53fff640 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d11450 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fec40012fd0 tx=0x7fec40003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eec7f72b5957416 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.786+0000 7fec33fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec33fff640 1 -- 172.21.15.105:0/551210735 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40029070 con 0x55b7f2d07420 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec33fff640 1 -- 172.21.15.105:0/551210735 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fec40003ba0 con 0x55b7f2d07420 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 1 -- 172.21.15.105:0/551210735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec400054d0 con 0x55b7f2d07420 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec33fff640 1 -- 172.21.15.105:0/551210735 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec40035050 con 0x55b7f2d07420 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 10 client.16734 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 1 -- 172.21.15.105:0/551210735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7f2d1b800 con 0x55b7f2d07420 2024-03-26T17:41:45.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec33fff640 1 -- 172.21.15.105:0/551210735 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec400054d0 con 0x55b7f2d07420 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec33fff640 1 -- 172.21.15.105:0/551210735 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fec40004fd0 con 0x55b7f2d07420 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 10 client.16734 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 10 client.16734 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 20 client.16734 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b7f2d1cd50 0x55b7f2d1f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 4 client.16734 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec5d7f4740 1 -- 172.21.15.105:0/551210735 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7f2c78eb0 con 0x55b7f2d1cd50 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.787+0000 7fec588ce640 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b7f2d1cd50 0x55b7f2d1f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.788+0000 7fec33fff640 10 client.16734 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.788+0000 7fec588ce640 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b7f2d1cd50 0x55b7f2d1f190 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0576d1216197575 server_cookie=c8d2487927e23c75 in_seq=0 out_seq=0 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.788+0000 7fec33fff640 1 -- 172.21.15.105:0/551210735 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b7f2c78eb0 con 0x55b7f2d1cd50 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.788+0000 7fec33fff640 10 client.16734 handle_command_reply: tid=0 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.788+0000 7fec5d7f4740 1 client.16734 shutdown 2024-03-26T17:41:45.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 -- 172.21.15.105:0/551210735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 msgr2=0x55b7f2d11450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d11450 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fec40012fd0 tx=0x7fec40003820 comp rx=0 tx=0).stop 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 -- 172.21.15.105:0/551210735 shutdown_connections 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b7f2d1cd50 0x55b7f2d1f190 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 0x55b7f2d18490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d11450 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec33fff640 0 client.16734 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/551210735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d10f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 -- 172.21.15.105:0/551210735 >> 172.21.15.105:0/551210735 conn(0x55b7f2d01fa0 msgr2=0x55b7f2d011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 -- 172.21.15.105:0/551210735 shutdown_connections 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 1 -- 172.21.15.105:0/551210735 wait complete. 2024-03-26T17:41:45.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.789+0000 7fec5d7f4740 20 client.16734 trim_cache size 0 max 16384 2024-03-26T17:41:45.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec5d7f4740 1 Processor -- start 2024-03-26T17:41:45.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec5d7f4740 1 -- start start 2024-03-26T17:41:45.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2cfd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d07420 0x55b7f2d18450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d0c590 0x55b7f2d00930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec541aafe0 con 0x55b7f2d0c590 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fec541acb20 con 0x55b7f2d07420 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec541ab6b0 con 0x55b7f2d06a30 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec590cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d0c590 0x55b7f2d00930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec588ce640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2cfd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d07420 0x55b7f2d18450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec588ce640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2cfd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45230/0 (socket says 172.21.15.105:45230) 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec588ce640 1 -- 172.21.15.105:0/3137436804 learned_addr learned my addr 172.21.15.105:0/3137436804 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.805+0000 7fec590cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d0c590 0x55b7f2d00930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42000/0 (socket says 172.21.15.105:42000) 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec588ce640 1 -- 172.21.15.105:0/3137436804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d07420 msgr2=0x55b7f2d18450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec588ce640 1 --2- 172.21.15.105:0/3137436804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d07420 0x55b7f2d18450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec588ce640 1 -- 172.21.15.105:0/3137436804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d0c590 msgr2=0x55b7f2d00930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec588ce640 1 --2- 172.21.15.105:0/3137436804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d0c590 0x55b7f2d00930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec588ce640 1 -- 172.21.15.105:0/3137436804 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7f2cfd820 con 0x55b7f2d06a30 2024-03-26T17:41:45.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec588ce640 1 --2- 172.21.15.105:0/3137436804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2cfd050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fec480bd950 tx=0x7fec480eeef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8df60c55f175458 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec3e7fc640 1 -- 172.21.15.105:0/3137436804 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480bdc50 con 0x55b7f2d06a30 2024-03-26T17:41:45.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec3e7fc640 1 -- 172.21.15.105:0/3137436804 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec480c1da0 con 0x55b7f2d06a30 2024-03-26T17:41:45.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.806+0000 7fec3e7fc640 1 -- 172.21.15.105:0/3137436804 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480055d0 con 0x55b7f2d06a30 2024-03-26T17:41:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 -- 172.21.15.105:0/3137436804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 msgr2=0x55b7f2cfd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/3137436804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2cfd050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fec480bd950 tx=0x7fec480eeef0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 -- 172.21.15.105:0/3137436804 shutdown_connections 2024-03-26T17:41:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/3137436804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d0c590 0x55b7f2d00930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/3137436804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d07420 0x55b7f2d18450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/3137436804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2cfd050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 -- 172.21.15.105:0/3137436804 >> 172.21.15.105:0/3137436804 conn(0x55b7f2d01fa0 msgr2=0x55b7f2d088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 -- 172.21.15.105:0/3137436804 shutdown_connections 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 -- 172.21.15.105:0/3137436804 wait complete. 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 -- start start 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.807+0000 7fec5d7f4740 1 Processor -- start 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d15790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d15cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec5d7f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 0x55b7f2d16c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec541ac450 con 0x55b7f2d07420 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fec541abd80 con 0x55b7f2d0c590 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec5d7f4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec541a8fc0 con 0x55b7f2d06a30 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec5d7f4740 10 client.0 _ll_register_callbacks cb 0x7fec48004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d15cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec588ce640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d15790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d15cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42010/0 (socket says 172.21.15.105:42010) 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec53fff640 1 -- 172.21.15.105:0/1806714010 learned_addr learned my addr 172.21.15.105:0/1806714010 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec590cf640 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 0x55b7f2d16c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec588ce640 1 -- 172.21.15.105:0/1806714010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 msgr2=0x55b7f2d16c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec588ce640 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 0x55b7f2d16c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec588ce640 1 -- 172.21.15.105:0/1806714010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 msgr2=0x55b7f2d15cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec588ce640 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d15cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec588ce640 1 -- 172.21.15.105:0/1806714010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7f2d17280 con 0x55b7f2d06a30 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.808+0000 7fec53fff640 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d15cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec588ce640 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d15790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec480bc1a0 tx=0x7fec480bd850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd5d5432ff133f2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec33fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec33fff640 1 -- 172.21.15.105:0/1806714010 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480055b0 con 0x55b7f2d06a30 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec33fff640 1 -- 172.21.15.105:0/1806714010 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fec48003cf0 con 0x55b7f2d06a30 2024-03-26T17:41:45.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1806714010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec480c1da0 con 0x55b7f2d06a30 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec33fff640 1 -- 172.21.15.105:0/1806714010 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fec480bdc50 con 0x55b7f2d06a30 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 10 client.26581 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1806714010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7f2d185f0 con 0x55b7f2d06a30 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec33fff640 1 -- 172.21.15.105:0/1806714010 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec480c1da0 con 0x55b7f2d06a30 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec33fff640 1 -- 172.21.15.105:0/1806714010 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fec480f1200 con 0x55b7f2d06a30 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 10 client.26581 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 10 client.26581 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 20 client.26581 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b7f2d83540 0x55b7f2d85960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 4 client.26581 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.809+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1806714010 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7f2c73200 con 0x55b7f2d83540 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec53fff640 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b7f2d83540 0x55b7f2d85960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec33fff640 10 client.26581 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec53fff640 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b7f2d83540 0x55b7f2d85960 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8126192fd3e8e63f server_cookie=2cc3fcff3b1e4fed in_seq=0 out_seq=0 2024-03-26T17:41:45.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec33fff640 1 -- 172.21.15.105:0/1806714010 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7f2c73200 con 0x55b7f2d83540 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec33fff640 10 client.26581 handle_command_reply: tid=0 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 client.26581 shutdown 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1806714010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 msgr2=0x55b7f2d15790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d15790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec480bc1a0 tx=0x7fec480bd850 comp rx=0 tx=0).stop 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1806714010 shutdown_connections 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b7f2d83540 0x55b7f2d85960 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7f2d0c590 0x55b7f2d16c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7f2d07420 0x55b7f2d15cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 --2- 172.21.15.105:0/1806714010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7f2d06a30 0x55b7f2d15790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1806714010 >> 172.21.15.105:0/1806714010 conn(0x55b7f2d01fa0 msgr2=0x55b7f2d05200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.810+0000 7fec33fff640 0 client.26581 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1806714010 shutdown_connections 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5d7f4740 1 -- 172.21.15.105:0/1806714010 wait complete. 2024-03-26T17:41:45.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5d7f4740 20 client.26581 trim_cache size 0 max 16384 2024-03-26T17:41:45.810 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:45.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 -- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec440a2f20 msgr2=0x7fec440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec440a2f20 0x7fec440a53e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005fd0 tx=0x7fec4c011bc0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 -- 172.21.15.105:0/1558224994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec540ffba0 msgr2=0x7fec54102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec540ffba0 0x7fec54102490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec48000c00 tx=0x7fec48005e70 comp rx=0 tx=0).stop 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 -- 172.21.15.105:0/1558224994 shutdown_connections 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fec440a2f20 0x7fec440a53e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec54103530 0x7fec54101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec54102b70 0x7fec54100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.811+0000 7fec5ab32640 1 --2- 172.21.15.105:0/1558224994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec540ffba0 0x7fec54102490 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.812+0000 7fec5ab32640 1 -- 172.21.15.105:0/1558224994 >> 172.21.15.105:0/1558224994 conn(0x7fec540fb4b0 msgr2=0x7fec54105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.812+0000 7fec5ab32640 1 -- 172.21.15.105:0/1558224994 shutdown_connections 2024-03-26T17:41:45.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.812+0000 7fec5ab32640 1 -- 172.21.15.105:0/1558224994 wait complete. 2024-03-26T17:41:45.816 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:41:45.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 -- 172.21.15.105:0/666860952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 msgr2=0x7fd628100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/666860952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 0x7fd628100be0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd624009900 tx=0x7fd62402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:45.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 -- 172.21.15.105:0/666860952 shutdown_connections 2024-03-26T17:41:45.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/666860952 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd628102370 0x7fd628108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/666860952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6281019b0 0x7fd628101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/666860952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 0x7fd628100be0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 -- 172.21.15.105:0/666860952 >> 172.21.15.105:0/666860952 conn(0x7fd6280fbd50 msgr2=0x7fd6280fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 -- 172.21.15.105:0/666860952 shutdown_connections 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.936+0000 7fd62f9e6640 1 -- 172.21.15.105:0/666860952 wait complete. 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62f9e6640 1 Processor -- start 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62f9e6640 1 -- start start 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62f9e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 0x7fd628104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62f9e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6281019b0 0x7fd628102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62f9e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd628102370 0x7fd6281032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62f9e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810c2c0 con 0x7fd6281007c0 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62f9e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810c140 con 0x7fd6281019b0 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62f9e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd62810c440 con 0x7fd628102370 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 0x7fd628104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 0x7fd628104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42038/0 (socket says 172.21.15.105:42038) 2024-03-26T17:41:45.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62cf81640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6281019b0 0x7fd628102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 -- 172.21.15.105:0/2493586619 learned_addr learned my addr 172.21.15.105:0/2493586619 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62df83640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd628102370 0x7fd6281032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 -- 172.21.15.105:0/2493586619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd628102370 msgr2=0x7fd6281032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd628102370 0x7fd6281032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 -- 172.21.15.105:0/2493586619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6281019b0 msgr2=0x7fd628102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6281019b0 0x7fd628102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 -- 172.21.15.105:0/2493586619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd628103b60 con 0x7fd6281007c0 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62cf81640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6281019b0 0x7fd628102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.937+0000 7fd62d782640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 0x7fd628104750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd624000c00 tx=0x7fd624005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.938+0000 7fd61e7fc640 1 -- 172.21.15.105:0/2493586619 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624043020 con 0x7fd6281007c0 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.938+0000 7fd62f9e6640 1 -- 172.21.15.105:0/2493586619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6281a1c70 con 0x7fd6281007c0 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.938+0000 7fd61e7fc640 1 -- 172.21.15.105:0/2493586619 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd624004350 con 0x7fd6281007c0 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.938+0000 7fd61e7fc640 1 -- 172.21.15.105:0/2493586619 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624003b80 con 0x7fd6281007c0 2024-03-26T17:41:45.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.938+0000 7fd62f9e6640 1 -- 172.21.15.105:0/2493586619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281a21c0 con 0x7fd6281007c0 2024-03-26T17:41:45.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd6325d5740 1 Processor -- start 2024-03-26T17:41:45.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd6325d5740 1 -- start start 2024-03-26T17:41:45.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd61e7fc640 1 -- 172.21.15.105:0/2493586619 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd624046020 con 0x7fd6281007c0 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7618e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad761df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad7620a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd6325d5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560ad757d970 con 0x560ad761e590 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd6325d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560ad7525d70 con 0x560ad7618a30 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd6325d5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ad7579920 con 0x560ad7619420 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd62df83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7618e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd62d782640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad761df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd62df83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7618e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58624/0 (socket says 172.21.15.105:58624) 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd62df83640 1 -- 172.21.15.105:0/2736712407 learned_addr learned my addr 172.21.15.105:0/2736712407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.941+0000 7fd62cf81640 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad7620a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd62df83640 1 -- 172.21.15.105:0/2736712407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 msgr2=0x560ad761df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd62df83640 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad761df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd62df83640 1 -- 172.21.15.105:0/2736712407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 msgr2=0x560ad7620a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd62df83640 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad7620a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd62df83640 1 -- 172.21.15.105:0/2736712407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad72913f0 con 0x560ad7618a30 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd62d782640 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad761df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd61e7fc640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6140a2f20 0x7fd6140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd62df83640 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7618e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd620009fd0 tx=0x7fd6200118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81100b348498ceb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd62d782640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6140a2f20 0x7fd6140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd6137fe640 1 -- 172.21.15.105:0/2736712407 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620004d20 con 0x560ad7618a30 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd61e7fc640 1 -- 172.21.15.105:0/2493586619 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd6240f1580 con 0x7fd6281007c0 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd6137fe640 1 -- 172.21.15.105:0/2736712407 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd620004050 con 0x560ad7618a30 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.942+0000 7fd6137fe640 1 -- 172.21.15.105:0/2736712407 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620005280 con 0x560ad7618a30 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 -- 172.21.15.105:0/2736712407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 msgr2=0x560ad7618e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd62d782640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6140a2f20 0x7fd6140a53e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6240ef3b0 tx=0x7fd6240f70e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7618e50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd620009fd0 tx=0x7fd6200118d0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 -- 172.21.15.105:0/2736712407 shutdown_connections 2024-03-26T17:41:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad7620a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad761df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 --2- 172.21.15.105:0/2736712407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7618e50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 -- 172.21.15.105:0/2736712407 >> 172.21.15.105:0/2736712407 conn(0x560ad7613fa0 msgr2=0x560ad76143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 -- 172.21.15.105:0/2736712407 shutdown_connections 2024-03-26T17:41:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 -- 172.21.15.105:0/2736712407 wait complete. 2024-03-26T17:41:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 -- start start 2024-03-26T17:41:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.943+0000 7fd6325d5740 1 Processor -- start 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad7618a30 0x560ad7622ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7619420 0x560ad7623430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad761e590 0x560ad762a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd6325d5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560ad7621540 con 0x560ad7618a30 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd6325d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560ad7539310 con 0x560ad7619420 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd6325d5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ad76216c0 con 0x560ad761e590 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad7618a30 0x560ad7622ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd6325d5740 10 client.0 _ll_register_callbacks cb 0x560ad7574c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad7618a30 0x560ad7622ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42052/0 (socket says 172.21.15.105:42052) 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 -- 172.21.15.105:0/3296750892 learned_addr learned my addr 172.21.15.105:0/3296750892 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62cf81640 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7619420 0x560ad7623430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad761e590 0x560ad762a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 -- 172.21.15.105:0/3296750892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad761e590 msgr2=0x560ad762a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad761e590 0x560ad762a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 -- 172.21.15.105:0/3296750892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7619420 msgr2=0x560ad7623430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7619420 0x560ad7623430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 -- 172.21.15.105:0/3296750892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad735b7d0 con 0x560ad7618a30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd62df83640 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad7618a30 0x560ad7622ef0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd62001b040 tx=0x7fd620003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad08d2bb086559bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.944+0000 7fd610ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd610ff9640 1 -- 172.21.15.105:0/3296750892 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620004e50 con 0x560ad7618a30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd610ff9640 1 -- 172.21.15.105:0/3296750892 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd62000f4e0 con 0x560ad7618a30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 1 -- 172.21.15.105:0/3296750892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd620004050 con 0x560ad7618a30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd610ff9640 1 -- 172.21.15.105:0/3296750892 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd62000f6e0 con 0x560ad7618a30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 10 client.16752 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 1 -- 172.21.15.105:0/3296750892 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ad762d7e0 con 0x560ad7618a30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd610ff9640 1 -- 172.21.15.105:0/3296750892 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd620004050 con 0x560ad7618a30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd610ff9640 1 -- 172.21.15.105:0/3296750892 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd620005530 con 0x560ad7618a30 2024-03-26T17:41:45.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 10 client.16752 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 20 client.16752 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560ad762ed00 0x560ad7631110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 4 client.16752 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.945+0000 7fd6325d5740 1 -- 172.21.15.105:0/3296750892 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad758aeb0 con 0x560ad762ed00 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.946+0000 7fd62cf81640 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560ad762ed00 0x560ad7631110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.946+0000 7fd610ff9640 10 client.16752 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.946+0000 7fd62cf81640 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560ad762ed00 0x560ad7631110 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1267d6b476ce9c server_cookie=f2d9bcfc1ea96015 in_seq=0 out_seq=0 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.947+0000 7fd610ff9640 1 -- 172.21.15.105:0/3296750892 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560ad758aeb0 con 0x560ad762ed00 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.947+0000 7fd610ff9640 10 client.16752 handle_command_reply: tid=0 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.947+0000 7fd6325d5740 1 client.16752 shutdown 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.947+0000 7fd6325d5740 1 -- 172.21.15.105:0/3296750892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad7618a30 msgr2=0x560ad7622ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.947+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad7618a30 0x560ad7622ef0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd62001b040 tx=0x7fd620003f70 comp rx=0 tx=0).stop 2024-03-26T17:41:45.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 1 -- 172.21.15.105:0/3296750892 shutdown_connections 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560ad762ed00 0x560ad7631110 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad761e590 0x560ad762a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7619420 0x560ad7623430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3296750892 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad7618a30 0x560ad7622ef0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 1 -- 172.21.15.105:0/3296750892 >> 172.21.15.105:0/3296750892 conn(0x560ad7613fa0 msgr2=0x560ad76131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd610ff9640 0 client.16752 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 1 -- 172.21.15.105:0/3296750892 shutdown_connections 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 1 -- 172.21.15.105:0/3296750892 wait complete. 2024-03-26T17:41:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.948+0000 7fd6325d5740 20 client.16752 trim_cache size 0 max 16384 2024-03-26T17:41:45.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd6325d5740 1 Processor -- start 2024-03-26T17:41:45.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd6325d5740 1 -- start start 2024-03-26T17:41:45.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7622b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad762a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad7623040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd6325d5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281ab040 con 0x560ad761e590 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd6325d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281acb80 con 0x560ad7618a30 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd6325d5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281ab710 con 0x560ad7619420 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd62df83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7622b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd62df83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7622b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58648/0 (socket says 172.21.15.105:58648) 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd62df83640 1 -- 172.21.15.105:0/3033946123 learned_addr learned my addr 172.21.15.105:0/3033946123 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd62cf81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad762a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.962+0000 7fd62df83640 1 --2- 172.21.15.105:0/3033946123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad7623040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd62cf81640 1 -- 172.21.15.105:0/3033946123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 msgr2=0x560ad7622b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd62cf81640 1 --2- 172.21.15.105:0/3033946123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7622b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd62cf81640 1 -- 172.21.15.105:0/3033946123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 msgr2=0x560ad7623040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd62cf81640 1 --2- 172.21.15.105:0/3033946123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad7623040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd62cf81640 1 -- 172.21.15.105:0/3033946123 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad760f820 con 0x560ad7619420 2024-03-26T17:41:45.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd62cf81640 1 --2- 172.21.15.105:0/3033946123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad762a430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x560ad7631990 tx=0x7fd618005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ea3e923bbb03815 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd612ffd640 1 -- 172.21.15.105:0/3033946123 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618005020 con 0x560ad7619420 2024-03-26T17:41:45.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd612ffd640 1 -- 172.21.15.105:0/3033946123 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd618024030 con 0x560ad7619420 2024-03-26T17:41:45.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.963+0000 7fd612ffd640 1 -- 172.21.15.105:0/3033946123 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800f790 con 0x560ad7619420 2024-03-26T17:41:45.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 -- 172.21.15.105:0/3033946123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 msgr2=0x560ad762a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3033946123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad762a430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x560ad7631990 tx=0x7fd618005f90 comp rx=0 tx=0).stop 2024-03-26T17:41:45.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 -- 172.21.15.105:0/3033946123 shutdown_connections 2024-03-26T17:41:45.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3033946123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad7623040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3033946123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad762a430 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 --2- 172.21.15.105:0/3033946123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad7622b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 -- 172.21.15.105:0/3033946123 >> 172.21.15.105:0/3033946123 conn(0x560ad7613fa0 msgr2=0x560ad761a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 -- 172.21.15.105:0/3033946123 shutdown_connections 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 -- 172.21.15.105:0/3033946123 wait complete. 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 -- start start 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.964+0000 7fd6325d5740 1 Processor -- start 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad762a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad7626ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd6325d5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad76274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd6325d5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281ac4b0 con 0x560ad761e590 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd6325d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281abde0 con 0x560ad7618a30 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd6325d5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281a9020 con 0x560ad7619420 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd6325d5740 10 client.0 _ll_register_callbacks cb 0x7fd618009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad762a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad76274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62cf81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad7626ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad76274f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42064/0 (socket says 172.21.15.105:42064) 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62cf81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad7626ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45308/0 (socket says 172.21.15.105:45308) 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 -- 172.21.15.105:0/1127366521 learned_addr learned my addr 172.21.15.105:0/1127366521 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 -- 172.21.15.105:0/1127366521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 msgr2=0x560ad7626ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad7626ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 -- 172.21.15.105:0/1127366521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 msgr2=0x560ad762a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad762a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 -- 172.21.15.105:0/1127366521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad76930f0 con 0x560ad761e590 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62cf81640 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad7626ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd62df83640 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad76274f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd62000f3c0 tx=0x7fd6200118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb49b6c6bbda2c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.965+0000 7fd610ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd610ff9640 1 -- 172.21.15.105:0/1127366521 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620004e50 con 0x560ad761e590 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd610ff9640 1 -- 172.21.15.105:0/1127366521 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd62000f880 con 0x560ad761e590 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 1 -- 172.21.15.105:0/1127366521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd618024030 con 0x560ad761e590 2024-03-26T17:41:45.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd610ff9640 1 -- 172.21.15.105:0/1127366521 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd62000fa80 con 0x560ad761e590 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 10 client.16758 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 1 -- 172.21.15.105:0/1127366521 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ad76943f0 con 0x560ad761e590 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd610ff9640 1 -- 172.21.15.105:0/1127366521 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd618024030 con 0x560ad761e590 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd610ff9640 1 -- 172.21.15.105:0/1127366521 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd620040450 con 0x560ad761e590 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 10 client.16758 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 20 client.16758 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560ad7694ca0 0x560ad76970c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 4 client.16758 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd6325d5740 1 -- 172.21.15.105:0/1127366521 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ad7585200 con 0x560ad7694ca0 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.966+0000 7fd62cf81640 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560ad7694ca0 0x560ad76970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd62cf81640 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560ad7694ca0 0x560ad76970c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8356e1d16cd7586 server_cookie=c920813e458b74b7 in_seq=0 out_seq=0 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd610ff9640 10 client.16758 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd610ff9640 1 -- 172.21.15.105:0/1127366521 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ad7585200 con 0x560ad7694ca0 2024-03-26T17:41:45.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd610ff9640 10 client.16758 handle_command_reply: tid=0 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 client.16758 shutdown 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 -- 172.21.15.105:0/1127366521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 msgr2=0x560ad76274f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad76274f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd62000f3c0 tx=0x7fd6200118d0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 -- 172.21.15.105:0/1127366521 shutdown_connections 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560ad7694ca0 0x560ad76970c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560ad761e590 0x560ad76274f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ad7619420 0x560ad7626ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd610ff9640 0 client.16758 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 --2- 172.21.15.105:0/1127366521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560ad7618a30 0x560ad762a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 -- 172.21.15.105:0/1127366521 >> 172.21.15.105:0/1127366521 conn(0x560ad7613fa0 msgr2=0x560ad7622b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.967+0000 7fd6325d5740 1 -- 172.21.15.105:0/1127366521 shutdown_connections 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd6325d5740 1 -- 172.21.15.105:0/1127366521 wait complete. 2024-03-26T17:41:45.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd6325d5740 20 client.16758 trim_cache size 0 max 16384 2024-03-26T17:41:45.966 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:41:45.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 -- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6140a2f20 msgr2=0x7fd6140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6140a2f20 0x7fd6140a53e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6240ef3b0 tx=0x7fd6240f70e0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 -- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 msgr2=0x7fd628104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 0x7fd628104750 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd624000c00 tx=0x7fd624005dc0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 -- 172.21.15.105:0/2493586619 shutdown_connections 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6140a2f20 0x7fd6140a53e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd628102370 0x7fd6281032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6281019b0 0x7fd628102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 --2- 172.21.15.105:0/2493586619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6281007c0 0x7fd628104750 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 -- 172.21.15.105:0/2493586619 >> 172.21.15.105:0/2493586619 conn(0x7fd6280fbd50 msgr2=0x7fd628068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 -- 172.21.15.105:0/2493586619 shutdown_connections 2024-03-26T17:41:45.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:45.968+0000 7fd62f9e6640 1 -- 172.21.15.105:0/2493586619 wait complete. 2024-03-26T17:41:45.971 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:45 smithi105 ceph-mon[32280]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2024-03-26T17:41:45.971 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3697671782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:46.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 88474 3.14 MB/sec execute 29 sec latency 29.635 ms 2024-03-26T17:41:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:45 smithi144 ceph-mon[36095]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2024-03-26T17:41:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3697671782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:45 smithi135 ceph-mon[35636]: pgmap v450: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.7 MiB/s wr, 110 op/s 2024-03-26T17:41:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3697671782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:41:46.972 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:41:46.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:41:46.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:41:47.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 88957 3.18 MB/sec execute 30 sec latency 59.856 ms 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.103+0000 7fad90332640 1 -- 172.21.15.105:0/4058038214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 msgr2=0x7fad88100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.103+0000 7fad90332640 1 --2- 172.21.15.105:0/4058038214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 0x7fad88100be0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fad84009900 tx=0x7fad8402bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 -- 172.21.15.105:0/4058038214 shutdown_connections 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 --2- 172.21.15.105:0/4058038214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad88102370 0x7fad88108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 --2- 172.21.15.105:0/4058038214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad881019b0 0x7fad88101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 --2- 172.21.15.105:0/4058038214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 0x7fad88100be0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 -- 172.21.15.105:0/4058038214 >> 172.21.15.105:0/4058038214 conn(0x7fad880fbd50 msgr2=0x7fad880fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 -- 172.21.15.105:0/4058038214 shutdown_connections 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 -- 172.21.15.105:0/4058038214 wait complete. 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 Processor -- start 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 -- start start 2024-03-26T17:41:47.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 0x7fad88197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad881019b0 0x7fad88198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad88102370 0x7fad8819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad8810c2f0 con 0x7fad881007c0 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad8810c170 con 0x7fad881019b0 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.104+0000 7fad90332640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad8810c470 con 0x7fad88102370 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 0x7fad88197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 0x7fad88197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42080/0 (socket says 172.21.15.105:42080) 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 -- 172.21.15.105:0/1793465313 learned_addr learned my addr 172.21.15.105:0/1793465313 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad881019b0 0x7fad88198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e8cf640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad88102370 0x7fad8819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 -- 172.21.15.105:0/1793465313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad88102370 msgr2=0x7fad8819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad88102370 0x7fad8819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 -- 172.21.15.105:0/1793465313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad881019b0 msgr2=0x7fad88198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad881019b0 0x7fad88198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 -- 172.21.15.105:0/1793465313 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8819f770 con 0x7fad881007c0 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 0x7fad88197b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fad84000c00 tx=0x7fad84005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad7f7fe640 1 -- 172.21.15.105:0/1793465313 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84043020 con 0x7fad881007c0 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad90332640 1 -- 172.21.15.105:0/1793465313 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8819fa00 con 0x7fad881007c0 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad7f7fe640 1 -- 172.21.15.105:0/1793465313 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad84004350 con 0x7fad881007c0 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad7f7fe640 1 -- 172.21.15.105:0/1793465313 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84003b80 con 0x7fad881007c0 2024-03-26T17:41:47.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.105+0000 7fad90332640 1 -- 172.21.15.105:0/1793465313 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8819ffa0 con 0x7fad881007c0 2024-03-26T17:41:47.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad92ff5740 1 Processor -- start 2024-03-26T17:41:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad92ff5740 1 -- start start 2024-03-26T17:41:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db1315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 0x5586db136680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db139180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad92ff5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5586db096630 con 0x5586db131190 2024-03-26T17:41:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad92ff5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5586db03ee10 con 0x5586db136cf0 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad92ff5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5586db0925e0 con 0x5586db131b80 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad7f7fe640 1 -- 172.21.15.105:0/1793465313 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fad84046020 con 0x7fad881007c0 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad8e8cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db1315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.108+0000 7fad8d8cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8e8cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db1315b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42086/0 (socket says 172.21.15.105:42086) 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8e8cf640 1 -- 172.21.15.105:0/1941908404 learned_addr learned my addr 172.21.15.105:0/1941908404 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/1941908404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 0x5586db136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8d8cd640 1 -- 172.21.15.105:0/1941908404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 msgr2=0x5586db136680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/1941908404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 0x5586db136680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8d8cd640 1 -- 172.21.15.105:0/1941908404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 msgr2=0x5586db1315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/1941908404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db1315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8d8cd640 1 -- 172.21.15.105:0/1941908404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586db055740 con 0x5586db136cf0 2024-03-26T17:41:47.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/1941908404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db139180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad78009bf0 tx=0x7fad78011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcd185044baed1a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad7f7fe640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad740a2f20 0x7fad740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad7cbfe640 1 -- 172.21.15.105:0/1941908404 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78004050 con 0x5586db136cf0 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad7cbfe640 1 -- 172.21.15.105:0/1941908404 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad780054d0 con 0x5586db136cf0 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad7cbfe640 1 -- 172.21.15.105:0/1941908404 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78004cf0 con 0x5586db136cf0 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad8e8cf640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad740a2f20 0x7fad740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.109+0000 7fad7f7fe640 1 -- 172.21.15.105:0/1793465313 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad840f15b0 con 0x7fad881007c0 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.110+0000 7fad92ff5740 1 -- 172.21.15.105:0/1941908404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 msgr2=0x5586db139180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.110+0000 7fad8e8cf640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad740a2f20 0x7fad740a53e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5586db1323e0 tx=0x7fad80009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.110+0000 7fad92ff5740 1 --2- 172.21.15.105:0/1941908404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db139180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad78009bf0 tx=0x7fad78011810 comp rx=0 tx=0).stop 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 -- 172.21.15.105:0/1941908404 shutdown_connections 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 --2- 172.21.15.105:0/1941908404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db139180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 --2- 172.21.15.105:0/1941908404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 0x5586db136680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 --2- 172.21.15.105:0/1941908404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db1315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 -- 172.21.15.105:0/1941908404 >> 172.21.15.105:0/1941908404 conn(0x5586db12c700 msgr2=0x5586db12cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 -- 172.21.15.105:0/1941908404 shutdown_connections 2024-03-26T17:41:47.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 -- 172.21.15.105:0/1941908404 wait complete. 2024-03-26T17:41:47.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 -- start start 2024-03-26T17:41:47.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.111+0000 7fad92ff5740 1 Processor -- start 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db13b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db131b80 0x5586db13bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db136cf0 0x5586db142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad92ff5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5586db139ca0 con 0x5586db131190 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad92ff5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5586db0521a0 con 0x5586db131b80 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad92ff5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5586db139e20 con 0x5586db136cf0 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad92ff5740 10 client.0 _ll_register_callbacks cb 0x5586dada9c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db13b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db13b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42096/0 (socket says 172.21.15.105:42096) 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 -- 172.21.15.105:0/2122627603 learned_addr learned my addr 172.21.15.105:0/2122627603 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db131b80 0x5586db13bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e8cf640 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db136cf0 0x5586db142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 -- 172.21.15.105:0/2122627603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db136cf0 msgr2=0x5586db142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db136cf0 0x5586db142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 -- 172.21.15.105:0/2122627603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db131b80 msgr2=0x5586db13bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db131b80 0x5586db13bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 -- 172.21.15.105:0/2122627603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586dae727e0 con 0x5586db131190 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db13b670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fad840ba0a0 tx=0x7fad84005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74fdd575b5d919c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad71ffb640 1 -- 172.21.15.105:0/2122627603 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840bda10 con 0x5586db131190 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad71ffb640 1 -- 172.21.15.105:0/2122627603 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad840054d0 con 0x5586db131190 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.112+0000 7fad92ff5740 1 -- 172.21.15.105:0/2122627603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad780054d0 con 0x5586db131190 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad71ffb640 1 -- 172.21.15.105:0/2122627603 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad840bd5a0 con 0x5586db131190 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad92ff5740 10 client.16770 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad92ff5740 1 -- 172.21.15.105:0/2122627603 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586daed2f70 con 0x5586db131190 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad71ffb640 1 -- 172.21.15.105:0/2122627603 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad780054d0 con 0x5586db131190 2024-03-26T17:41:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad71ffb640 1 -- 172.21.15.105:0/2122627603 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fad840d4070 con 0x5586db131190 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad92ff5740 10 client.16770 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad92ff5740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad92ff5740 20 client.16770 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad92ff5740 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5586db147420 0x5586db1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad92ff5740 4 client.16770 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad92ff5740 1 -- 172.21.15.105:0/2122627603 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586db0a3b70 con 0x5586db147420 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.113+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5586db147420 0x5586db1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad71ffb640 10 client.16770 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5586db147420 0x5586db1498b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550d34ee57e874b server_cookie=724492cc9ef2d211 in_seq=0 out_seq=0 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad71ffb640 1 -- 172.21.15.105:0/2122627603 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5586db0a3b70 con 0x5586db147420 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad71ffb640 10 client.16770 handle_command_reply: tid=0 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad92ff5740 1 client.16770 shutdown 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad92ff5740 1 -- 172.21.15.105:0/2122627603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 msgr2=0x5586db13b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad92ff5740 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db13b670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fad840ba0a0 tx=0x7fad84005bc0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad92ff5740 1 -- 172.21.15.105:0/2122627603 shutdown_connections 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad92ff5740 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5586db147420 0x5586db1498b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad92ff5740 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db136cf0 0x5586db142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad92ff5740 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db131b80 0x5586db13bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad71ffb640 0 client.16770 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.114+0000 7fad92ff5740 1 --2- 172.21.15.105:0/2122627603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db13b670 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.115+0000 7fad92ff5740 1 -- 172.21.15.105:0/2122627603 >> 172.21.15.105:0/2122627603 conn(0x5586db12c700 msgr2=0x5586db12b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.115+0000 7fad92ff5740 1 -- 172.21.15.105:0/2122627603 shutdown_connections 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.115+0000 7fad92ff5740 1 -- 172.21.15.105:0/2122627603 wait complete. 2024-03-26T17:41:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.115+0000 7fad92ff5740 20 client.16770 trim_cache size 0 max 16384 2024-03-26T17:41:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.140+0000 7fad92ff5740 1 Processor -- start 2024-03-26T17:41:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.140+0000 7fad92ff5740 1 -- start start 2024-03-26T17:41:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131190 0x5586db12b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131b80 0x5586db142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db12b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad92ff5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad881a8e50 con 0x5586db131b80 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad92ff5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad881aa990 con 0x5586db136cf0 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad92ff5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad881a9520 con 0x5586db131190 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131b80 0x5586db142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131b80 0x5586db142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42112/0 (socket says 172.21.15.105:42112) 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 -- 172.21.15.105:0/3287381389 learned_addr learned my addr 172.21.15.105:0/3287381389 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131190 0x5586db12b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8e8cf640 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db12b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 -- 172.21.15.105:0/3287381389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131190 msgr2=0x5586db12b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131190 0x5586db12b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 -- 172.21.15.105:0/3287381389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 msgr2=0x5586db12b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db12b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 -- 172.21.15.105:0/3287381389 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586dad6a170 con 0x5586db131b80 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131190 0x5586db12b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131b80 0x5586db142bb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fad78009b40 tx=0x7fad78005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41a52ca47fcf817 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad73fff640 1 -- 172.21.15.105:0/3287381389 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7802ecb0 con 0x5586db131b80 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad73fff640 1 -- 172.21.15.105:0/3287381389 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad7802ee50 con 0x5586db131b80 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.141+0000 7fad73fff640 1 -- 172.21.15.105:0/3287381389 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78003850 con 0x5586db131b80 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 -- 172.21.15.105:0/3287381389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131b80 msgr2=0x5586db142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131b80 0x5586db142bb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fad78009b40 tx=0x7fad78005f10 comp rx=0 tx=0).stop 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 -- 172.21.15.105:0/3287381389 shutdown_connections 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db12b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131b80 0x5586db142bb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3287381389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131190 0x5586db12b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 -- 172.21.15.105:0/3287381389 >> 172.21.15.105:0/3287381389 conn(0x5586db12c700 msgr2=0x5586db1301f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 -- 172.21.15.105:0/3287381389 shutdown_connections 2024-03-26T17:41:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.142+0000 7fad92ff5740 1 -- 172.21.15.105:0/3287381389 wait complete. 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 1 -- start start 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 1 Processor -- start 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 0x5586db1413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db1421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad881aa2c0 con 0x5586db131190 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad881a9bf0 con 0x5586db136cf0 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad881a6e30 con 0x5586db131b80 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad92ff5740 10 client.0 _ll_register_callbacks cb 0x7fad840057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad8e0ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad8e0ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db140ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42116/0 (socket says 172.21.15.105:42116) 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad8e0ce640 1 -- 172.21.15.105:0/3320089175 learned_addr learned my addr 172.21.15.105:0/3320089175 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 0x5586db1413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.143+0000 7fad8e8cf640 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db1421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad8e0ce640 1 -- 172.21.15.105:0/3320089175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 msgr2=0x5586db1413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 0x5586db1413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad8e0ce640 1 -- 172.21.15.105:0/3320089175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 msgr2=0x5586db1421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db1421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad8e0ce640 1 -- 172.21.15.105:0/3320089175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586db142890 con 0x5586db131190 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad8e0ce640 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db140ea0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fad84005ab0 tx=0x7fad840c6230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d187fc852a3ed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad71ffb640 1 -- 172.21.15.105:0/3320089175 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840bd780 con 0x5586db131190 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad71ffb640 1 -- 172.21.15.105:0/3320089175 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad840bd090 con 0x5586db131190 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad92ff5740 1 -- 172.21.15.105:0/3320089175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad7802ee50 con 0x5586db131190 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad71ffb640 1 -- 172.21.15.105:0/3320089175 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad840bda10 con 0x5586db131190 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad92ff5740 10 client.16776 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad92ff5740 1 -- 172.21.15.105:0/3320089175 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586db1ae180 con 0x5586db131190 2024-03-26T17:41:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad71ffb640 1 -- 172.21.15.105:0/3320089175 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad7802ee50 con 0x5586db131190 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.144+0000 7fad71ffb640 1 -- 172.21.15.105:0/3320089175 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fad840d4070 con 0x5586db131190 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad92ff5740 10 client.16776 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad92ff5740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad92ff5740 20 client.16776 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5586db1ae9a0 0x5586db1b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad92ff5740 4 client.16776 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad92ff5740 1 -- 172.21.15.105:0/3320089175 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5586db09dec0 con 0x5586db1ae9a0 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5586db1ae9a0 0x5586db1b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad71ffb640 10 client.16776 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.145+0000 7fad8d8cd640 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5586db1ae9a0 0x5586db1b0dc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06382f668cbc778 server_cookie=5713ebe7dd7b48fb in_seq=0 out_seq=0 2024-03-26T17:41:47.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad71ffb640 1 -- 172.21.15.105:0/3320089175 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5586db09dec0 con 0x5586db1ae9a0 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad71ffb640 10 client.16776 handle_command_reply: tid=0 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 client.16776 shutdown 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 -- 172.21.15.105:0/3320089175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 msgr2=0x5586db140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db140ea0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fad84005ab0 tx=0x7fad840c6230 comp rx=0 tx=0).stop 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 -- 172.21.15.105:0/3320089175 shutdown_connections 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5586db1ae9a0 0x5586db1b0dc0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586db136cf0 0x5586db1421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586db131b80 0x5586db1413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 --2- 172.21.15.105:0/3320089175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586db131190 0x5586db140ea0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 -- 172.21.15.105:0/3320089175 >> 172.21.15.105:0/3320089175 conn(0x5586db12c700 msgr2=0x5586db13a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad71ffb640 0 client.16776 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 -- 172.21.15.105:0/3320089175 shutdown_connections 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 1 -- 172.21.15.105:0/3320089175 wait complete. 2024-03-26T17:41:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.148+0000 7fad92ff5740 20 client.16776 trim_cache size 0 max 16384 2024-03-26T17:41:47.145 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:41:47.145 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:41:47.145 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "e4ae95c9-713e-4012-9010-4fee3585dd5c", 2024-03-26T17:41:47.145 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:41:47.145 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 -- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad740a2f20 msgr2=0x7fad740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad740a2f20 0x7fad740a53e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5586db1323e0 tx=0x7fad80009500 comp rx=0 tx=0).stop 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 -- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 msgr2=0x7fad88197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 0x7fad88197b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fad84000c00 tx=0x7fad84005dc0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 -- 172.21.15.105:0/1793465313 shutdown_connections 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad740a2f20 0x7fad740a53e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad88102370 0x7fad8819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad881019b0 0x7fad88198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 --2- 172.21.15.105:0/1793465313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad881007c0 0x7fad88197b50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 -- 172.21.15.105:0/1793465313 >> 172.21.15.105:0/1793465313 conn(0x7fad880fbd50 msgr2=0x7fad88068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 -- 172.21.15.105:0/1793465313 shutdown_connections 2024-03-26T17:41:47.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.149+0000 7fad90332640 1 -- 172.21.15.105:0/1793465313 wait complete. 2024-03-26T17:41:47.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:41:47.155 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:41:47.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 -- 172.21.15.105:0/4201124148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba41007c0 msgr2=0x7feba4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 --2- 172.21.15.105:0/4201124148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba41007c0 0x7feba4100be0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7feb98009900 tx=0x7feb9802bc30 comp rx=0 tx=0).stop 2024-03-26T17:41:47.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 -- 172.21.15.105:0/4201124148 shutdown_connections 2024-03-26T17:41:47.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 --2- 172.21.15.105:0/4201124148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feba4102370 0x7feba4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 --2- 172.21.15.105:0/4201124148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba41019b0 0x7feba4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 --2- 172.21.15.105:0/4201124148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba41007c0 0x7feba4100be0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 -- 172.21.15.105:0/4201124148 >> 172.21.15.105:0/4201124148 conn(0x7feba40fbd50 msgr2=0x7feba40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 -- 172.21.15.105:0/4201124148 shutdown_connections 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.289+0000 7febaaf5b640 1 -- 172.21.15.105:0/4201124148 wait complete. 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7febaaf5b640 1 Processor -- start 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7febaaf5b640 1 -- start start 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7febaaf5b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba41007c0 0x7feba4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7febaaf5b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feba41019b0 0x7feba419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7febaaf5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba4102370 0x7feba41a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7febaaf5b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feba410d3d0 con 0x7feba4102370 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7febaaf5b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feba410d250 con 0x7feba41019b0 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7febaaf5b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feba410d550 con 0x7feba41007c0 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba8cf7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba41007c0 0x7feba4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba8cf7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba41007c0 0x7feba4199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45362/0 (socket says 172.21.15.105:45362) 2024-03-26T17:41:47.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba8cf7640 1 -- 172.21.15.105:0/2880182107 learned_addr learned my addr 172.21.15.105:0/2880182107 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba94f8640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba4102370 0x7feba41a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba3fff640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feba41019b0 0x7feba419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba8cf7640 1 -- 172.21.15.105:0/2880182107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feba41019b0 msgr2=0x7feba419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba8cf7640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feba41019b0 0x7feba419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba8cf7640 1 -- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba4102370 msgr2=0x7feba41a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba8cf7640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba4102370 0x7feba41a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba8cf7640 1 -- 172.21.15.105:0/2880182107 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba41a19d0 con 0x7feba41007c0 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.290+0000 7feba94f8640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba4102370 0x7feba41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.291+0000 7feba8cf7640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba41007c0 0x7feba4199dc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feb98000c00 tx=0x7feb98005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.291+0000 7feba1ffb640 1 -- 172.21.15.105:0/2880182107 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98043020 con 0x7feba41007c0 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.291+0000 7febaaf5b640 1 -- 172.21.15.105:0/2880182107 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba41a1c60 con 0x7feba41007c0 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.291+0000 7feba1ffb640 1 -- 172.21.15.105:0/2880182107 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb98004350 con 0x7feba41007c0 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.291+0000 7feba1ffb640 1 -- 172.21.15.105:0/2880182107 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98003b80 con 0x7feba41007c0 2024-03-26T17:41:47.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.291+0000 7febaaf5b640 1 -- 172.21.15.105:0/2880182107 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba41a21b0 con 0x7feba41007c0 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.293+0000 7feba1ffb640 1 -- 172.21.15.105:0/2880182107 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7feb98046020 con 0x7feba41007c0 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7febadc29740 1 Processor -- start 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7febadc29740 1 -- start start 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7febadc29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ced9bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7febadc29740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 0x5575ceda0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7febadc29740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ceda38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7febadc29740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5575ced00980 con 0x5575ceda1450 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7febadc29740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5575ceca8be0 con 0x5575ced9c2e0 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7febadc29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5575cecfc930 con 0x5575ced9b8f0 2024-03-26T17:41:47.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ced9bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ceda38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba8cf7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 0x5575ceda0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ced9bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45374/0 (socket says 172.21.15.105:45374) 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 -- 172.21.15.105:0/817127464 learned_addr learned my addr 172.21.15.105:0/817127464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba1ffb640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb940a2f20 0x7feb940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 -- 172.21.15.105:0/817127464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 msgr2=0x5575ceda0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 --2- 172.21.15.105:0/817127464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 0x5575ceda0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 -- 172.21.15.105:0/817127464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 msgr2=0x5575ceda38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 --2- 172.21.15.105:0/817127464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ceda38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 -- 172.21.15.105:0/817127464 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ceade7d0 con 0x5575ced9b8f0 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba94f8640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb940a2f20 0x7feb940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.294+0000 7feba1ffb640 1 -- 172.21.15.105:0/2880182107 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feb980f1610 con 0x7feba41007c0 2024-03-26T17:41:47.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7feba94f8640 1 --2- 172.21.15.105:0/817127464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ced9bd10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb9c011940 tx=0x7feb9c011e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a306a5e186dd9ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7feb8effd640 1 -- 172.21.15.105:0/817127464 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c017070 con 0x5575ced9b8f0 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7feb8effd640 1 -- 172.21.15.105:0/817127464 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9c011250 con 0x5575ced9b8f0 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7feb8effd640 1 -- 172.21.15.105:0/817127464 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c010a90 con 0x5575ced9b8f0 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7feba94f8640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb940a2f20 0x7feb940a53e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feb9c023000 tx=0x7feb9c024000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7febadc29740 1 -- 172.21.15.105:0/817127464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 msgr2=0x5575ced9bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7febadc29740 1 --2- 172.21.15.105:0/817127464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ced9bd10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb9c011940 tx=0x7feb9c011e00 comp rx=0 tx=0).stop 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7febadc29740 1 -- 172.21.15.105:0/817127464 shutdown_connections 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7febadc29740 1 --2- 172.21.15.105:0/817127464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ceda38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7febadc29740 1 --2- 172.21.15.105:0/817127464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 0x5575ceda0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7febadc29740 1 --2- 172.21.15.105:0/817127464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ced9bd10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.295+0000 7febadc29740 1 -- 172.21.15.105:0/817127464 >> 172.21.15.105:0/817127464 conn(0x5575ced96e60 msgr2=0x5575ced97260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 -- 172.21.15.105:0/817127464 shutdown_connections 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 -- 172.21.15.105:0/817127464 wait complete. 2024-03-26T17:41:47.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 -- start start 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 Processor -- start 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ceda5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 0x5575ceda60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575cedad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5575ceda4400 con 0x5575ceda1450 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5575cecbc160 con 0x5575ced9c2e0 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.296+0000 7febadc29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5575ceda4580 con 0x5575ced9b8f0 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7febadc29740 10 client.0 _ll_register_callbacks cb 0x5575cecf8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba94f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575cedad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ceda5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ceda5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:45380/0 (socket says 172.21.15.105:45380) 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 -- 172.21.15.105:0/379027611 learned_addr learned my addr 172.21.15.105:0/379027611 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 0x5575ceda60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba94f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575cedad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42154/0 (socket says 172.21.15.105:42154) 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 -- 172.21.15.105:0/379027611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 msgr2=0x5575ceda60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 0x5575ceda60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 -- 172.21.15.105:0/379027611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 msgr2=0x5575cedad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575cedad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 -- 172.21.15.105:0/379027611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575cea143f0 con 0x5575ced9b8f0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feba8cf7640 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ceda5b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb98005ab0 tx=0x7feb98009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d02c1df8c03733a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feb83fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feb83fff640 1 -- 172.21.15.105:0/379027611 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980bd840 con 0x5575ced9b8f0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7febadc29740 1 -- 172.21.15.105:0/379027611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9c011250 con 0x5575ced9b8f0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feb83fff640 1 -- 172.21.15.105:0/379027611 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feb98003d20 con 0x5575ced9b8f0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7feb83fff640 1 -- 172.21.15.105:0/379027611 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb980bbe40 con 0x5575ced9b8f0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7febadc29740 10 client.26629 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.297+0000 7febadc29740 1 -- 172.21.15.105:0/379027611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575ce9d40f0 con 0x5575ced9b8f0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7feb83fff640 1 -- 172.21.15.105:0/379027611 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9c011250 con 0x5575ced9b8f0 2024-03-26T17:41:47.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7feb83fff640 1 -- 172.21.15.105:0/379027611 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7feb980c6280 con 0x5575ced9b8f0 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7febadc29740 10 client.26629 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7febadc29740 10 client.26629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7febadc29740 20 client.26629 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7febadc29740 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5575cedb1960 0x5575cedb3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7febadc29740 4 client.26629 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7febadc29740 1 -- 172.21.15.105:0/379027611 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575ced0dec0 con 0x5575cedb1960 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7feba3fff640 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5575cedb1960 0x5575cedb3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7feba3fff640 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5575cedb1960 0x5575cedb3dd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c52abec3e18607 server_cookie=98761656ff936214 in_seq=0 out_seq=0 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.298+0000 7feb83fff640 10 client.26629 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.299+0000 7feb83fff640 1 -- 172.21.15.105:0/379027611 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5575ced0dec0 con 0x5575cedb1960 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.299+0000 7feb83fff640 10 client.26629 handle_command_reply: tid=0 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.299+0000 7febadc29740 1 client.26629 shutdown 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.299+0000 7febadc29740 1 -- 172.21.15.105:0/379027611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 msgr2=0x5575ceda5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.299+0000 7febadc29740 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ceda5b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb98005ab0 tx=0x7feb98009260 comp rx=0 tx=0).stop 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 1 -- 172.21.15.105:0/379027611 shutdown_connections 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5575cedb1960 0x5575cedb3dd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575cedad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9c2e0 0x5575ceda60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7feb83fff640 0 client.26629 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 1 --2- 172.21.15.105:0/379027611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9b8f0 0x5575ceda5b80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 1 -- 172.21.15.105:0/379027611 >> 172.21.15.105:0/379027611 conn(0x5575ced96e60 msgr2=0x5575ced96080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 1 -- 172.21.15.105:0/379027611 shutdown_connections 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 1 -- 172.21.15.105:0/379027611 wait complete. 2024-03-26T17:41:47.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.300+0000 7febadc29740 20 client.26629 trim_cache size 0 max 16384 2024-03-26T17:41:47.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.319+0000 7febadc29740 1 Processor -- start 2024-03-26T17:41:47.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7febadc29740 1 -- start start 2024-03-26T17:41:47.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7febadc29740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 0x5575ceda6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7febadc29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9c2e0 0x5575ced957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7febadc29740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ced95d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7febadc29740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feba41aaf40 con 0x5575ceda1450 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7febadc29740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feba41aca80 con 0x5575ced9b8f0 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7febadc29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feba41ab610 con 0x5575ced9c2e0 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ced95d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ced95d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42170/0 (socket says 172.21.15.105:42170) 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 -- 172.21.15.105:0/3050498787 learned_addr learned my addr 172.21.15.105:0/3050498787 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba3fff640 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9c2e0 0x5575ced957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba8cf7640 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 0x5575ceda6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 -- 172.21.15.105:0/3050498787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9c2e0 msgr2=0x5575ced957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9c2e0 0x5575ced957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 -- 172.21.15.105:0/3050498787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 msgr2=0x5575ceda6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 0x5575ceda6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 -- 172.21.15.105:0/3050498787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575cedb4420 con 0x5575ceda1450 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba3fff640 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9c2e0 0x5575ced957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.320+0000 7feba94f8640 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ced95d30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00f790 tx=0x7feb9c00fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5360f45a37124dc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7feb8e7fc640 1 -- 172.21.15.105:0/3050498787 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c017030 con 0x5575ceda1450 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7feb8e7fc640 1 -- 172.21.15.105:0/3050498787 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9c011510 con 0x5575ceda1450 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7feb8e7fc640 1 -- 172.21.15.105:0/3050498787 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c010720 con 0x5575ceda1450 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 -- 172.21.15.105:0/3050498787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 msgr2=0x5575ced95d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ced95d30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00f790 tx=0x7feb9c00fac0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 -- 172.21.15.105:0/3050498787 shutdown_connections 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ceda1450 0x5575ced95d30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ced9c2e0 0x5575ced957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 --2- 172.21.15.105:0/3050498787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 0x5575ceda6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 -- 172.21.15.105:0/3050498787 >> 172.21.15.105:0/3050498787 conn(0x5575ced96e60 msgr2=0x5575ced97240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 -- 172.21.15.105:0/3050498787 shutdown_connections 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.321+0000 7febadc29740 1 -- 172.21.15.105:0/3050498787 wait complete. 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 1 -- start start 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 1 Processor -- start 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 0x5575cedaa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ced9c2e0 0x5575cedaacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ceda1450 0x5575cedabbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feba41ac3b0 con 0x5575ced9c2e0 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feba41abce0 con 0x5575ced9b8f0 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feba41a8f20 con 0x5575ceda1450 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7febadc29740 10 client.0 _ll_register_callbacks cb 0x7feb980057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7feba3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ced9c2e0 0x5575cedaacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7feba3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ced9c2e0 0x5575cedaacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42176/0 (socket says 172.21.15.105:42176) 2024-03-26T17:41:47.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7feba3fff640 1 -- 172.21.15.105:0/4198088798 learned_addr learned my addr 172.21.15.105:0/4198088798 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7feba8cf7640 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 0x5575cedaa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.322+0000 7feba94f8640 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ceda1450 0x5575cedabbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feba3fff640 1 -- 172.21.15.105:0/4198088798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ceda1450 msgr2=0x5575cedabbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feba3fff640 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ceda1450 0x5575cedabbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feba3fff640 1 -- 172.21.15.105:0/4198088798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 msgr2=0x5575cedaa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feba3fff640 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 0x5575cedaa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feba3fff640 1 -- 172.21.15.105:0/4198088798 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575cedac1c0 con 0x5575ced9c2e0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feba3fff640 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ced9c2e0 0x5575cedaacb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feb90004700 tx=0x7feb900108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320d55dbcacb1c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feb83fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feb83fff640 1 -- 172.21.15.105:0/4198088798 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90005020 con 0x5575ced9c2e0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feb83fff640 1 -- 172.21.15.105:0/4198088798 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feb900051c0 con 0x5575ced9c2e0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7febadc29740 1 -- 172.21.15.105:0/4198088798 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9c011510 con 0x5575ced9c2e0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feb83fff640 1 -- 172.21.15.105:0/4198088798 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feb900053c0 con 0x5575ced9c2e0 2024-03-26T17:41:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7febadc29740 10 client.16794 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7febadc29740 1 -- 172.21.15.105:0/4198088798 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575cee17ff0 con 0x5575ced9c2e0 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.323+0000 7feb83fff640 1 -- 172.21.15.105:0/4198088798 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9c011510 con 0x5575ced9c2e0 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7feb83fff640 1 -- 172.21.15.105:0/4198088798 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7feb90027520 con 0x5575ced9c2e0 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7febadc29740 10 client.16794 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7febadc29740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7febadc29740 20 client.16794 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7febadc29740 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5575cee18a20 0x5575cee1ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7febadc29740 4 client.16794 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7febadc29740 1 -- 172.21.15.105:0/4198088798 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5575ced08210 con 0x5575cee18a20 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7feba8cf7640 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5575cee18a20 0x5575cee1ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7feba8cf7640 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5575cee18a20 0x5575cee1ae40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a9ba33ab5986b9 server_cookie=b87e7f188f9d1104 in_seq=0 out_seq=0 2024-03-26T17:41:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.324+0000 7feb83fff640 10 client.16794 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7feb83fff640 1 -- 172.21.15.105:0/4198088798 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5575ced08210 con 0x5575cee18a20 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7feb83fff640 10 client.16794 handle_command_reply: tid=0 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 client.16794 shutdown 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 -- 172.21.15.105:0/4198088798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ced9c2e0 msgr2=0x5575cedaacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ced9c2e0 0x5575cedaacb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feb90004700 tx=0x7feb900108c0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 -- 172.21.15.105:0/4198088798 shutdown_connections 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5575cee18a20 0x5575cee1ae40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5575ceda1450 0x5575cedabbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5575ced9c2e0 0x5575cedaacb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 --2- 172.21.15.105:0/4198088798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5575ced9b8f0 0x5575cedaa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 -- 172.21.15.105:0/4198088798 >> 172.21.15.105:0/4198088798 conn(0x5575ced96e60 msgr2=0x5575ced9a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7feb83fff640 0 client.16794 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 -- 172.21.15.105:0/4198088798 shutdown_connections 2024-03-26T17:41:47.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 1 -- 172.21.15.105:0/4198088798 wait complete. 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.326+0000 7febadc29740 20 client.16794 trim_cache size 0 max 16384 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout: "e4ae95c9-713e-4012-9010-4fee3585dd5c": { 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "e4ae95c9-713e-4012-9010-4fee3585dd5c", 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:41:47.324 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:41:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 -- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb940a2f20 msgr2=0x7feb940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb940a2f20 0x7feb940a53e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feb9c023000 tx=0x7feb9c024000 comp rx=0 tx=0).stop 2024-03-26T17:41:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 -- 172.21.15.105:0/2880182107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba41007c0 msgr2=0x7feba4199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba41007c0 0x7feba4199dc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feb98000c00 tx=0x7feb98005dc0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 -- 172.21.15.105:0/2880182107 shutdown_connections 2024-03-26T17:41:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb940a2f20 0x7feb940a53e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feba4102370 0x7feba41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feba41019b0 0x7feba419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 --2- 172.21.15.105:0/2880182107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feba41007c0 0x7feba4199dc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 -- 172.21.15.105:0/2880182107 >> 172.21.15.105:0/2880182107 conn(0x7feba40fbd50 msgr2=0x7feba41095c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 -- 172.21.15.105:0/2880182107 shutdown_connections 2024-03-26T17:41:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:41:47.327+0000 7febaaf5b640 1 -- 172.21.15.105:0/2880182107 wait complete. 2024-03-26T17:41:47.331 INFO:tasks.cephfs.filesystem:scrub status for tag:e4ae95c9-713e-4012-9010-4fee3585dd5c - {'path': '/', 'tag': 'e4ae95c9-713e-4012-9010-4fee3585dd5c', 'options': 'recursive,force'} 2024-03-26T17:41:48.020 INFO:tasks.workunit.client.0.smithi105.stdout: 1 89317 3.09 MB/sec execute 31 sec latency 16.196 ms 2024-03-26T17:41:48.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:47 smithi144 ceph-mon[36095]: pgmap v451: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:41:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:47 smithi105 ceph-mon[32280]: pgmap v451: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:41:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:47 smithi135 ceph-mon[35636]: pgmap v451: 97 pgs: 97 active+clean; 29 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:41:49.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 89852 3.05 MB/sec execute 32 sec latency 22.042 ms 2024-03-26T17:41:49.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:48 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:41:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:48 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:48 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:41:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:48 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:48 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:41:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:48 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:41:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:48 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:41:49.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:48 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:41:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:48 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:41:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:48 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:48 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:41:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:48 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:48 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:41:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:48 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:41:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:48 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:41:49.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:48 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:41:49.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:48 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:41:49.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:48 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:49.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:48 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:41:49.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:48 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:41:49.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:48 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:41:49.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:48 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:41:49.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:48 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:41:49.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:48 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:41:50.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 90552 3.06 MB/sec execute 33 sec latency 16.177 ms 2024-03-26T17:41:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:49 smithi144 ceph-mon[36095]: pgmap v452: 97 pgs: 97 active+clean; 25 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:41:50.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:49 smithi105 ceph-mon[32280]: pgmap v452: 97 pgs: 97 active+clean; 25 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:41:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:49 smithi135 ceph-mon[35636]: pgmap v452: 97 pgs: 97 active+clean; 25 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:41:51.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 91144 3.02 MB/sec execute 34 sec latency 17.009 ms 2024-03-26T17:41:52.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 92003 3.14 MB/sec execute 35 sec latency 16.956 ms 2024-03-26T17:41:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:51 smithi144 ceph-mon[36095]: pgmap v453: 97 pgs: 97 active+clean; 25 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-03-26T17:41:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:51 smithi105 ceph-mon[32280]: pgmap v453: 97 pgs: 97 active+clean; 25 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-03-26T17:41:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:51 smithi135 ceph-mon[35636]: pgmap v453: 97 pgs: 97 active+clean; 25 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.4 MiB/s wr, 92 op/s 2024-03-26T17:41:53.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 92401 3.15 MB/sec execute 36 sec latency 15.965 ms 2024-03-26T17:41:53.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:52 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:41:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:52 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:41:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:52 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:41:54.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 92782 3.10 MB/sec execute 37 sec latency 50.704 ms 2024-03-26T17:41:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:53 smithi144 ceph-mon[36095]: pgmap v454: 97 pgs: 97 active+clean; 27 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:41:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:53 smithi105 ceph-mon[32280]: pgmap v454: 97 pgs: 97 active+clean; 27 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:41:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:53 smithi135 ceph-mon[35636]: pgmap v454: 97 pgs: 97 active+clean; 27 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:41:55.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 93325 3.07 MB/sec execute 38 sec latency 26.989 ms 2024-03-26T17:41:56.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 93962 3.08 MB/sec execute 39 sec latency 19.035 ms 2024-03-26T17:41:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:55 smithi144 ceph-mon[36095]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-03-26T17:41:56.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:55 smithi105 ceph-mon[32280]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-03-26T17:41:56.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:55 smithi135 ceph-mon[35636]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.7 MiB/s wr, 112 op/s 2024-03-26T17:41:57.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 94638 3.04 MB/sec execute 40 sec latency 20.385 ms 2024-03-26T17:41:58.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 95456 3.12 MB/sec execute 41 sec latency 17.623 ms 2024-03-26T17:41:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:57 smithi144 ceph-mon[36095]: pgmap v456: 97 pgs: 97 active+clean; 28 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:41:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:57 smithi105 ceph-mon[32280]: pgmap v456: 97 pgs: 97 active+clean; 28 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:41:58.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:57 smithi135 ceph-mon[35636]: pgmap v456: 97 pgs: 97 active+clean; 28 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:41:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 -- 172.21.15.105:0/2097779961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c1075d0 msgr2=0x7fe49c109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/2097779961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c1075d0 0x7fe49c109a20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe494009ca0 tx=0x7fe49402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 -- 172.21.15.105:0/2097779961 shutdown_connections 2024-03-26T17:41:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/2097779961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c1075d0 0x7fe49c109a20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/2097779961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe49c104ca0 0x7fe49c107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/2097779961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe49c102370 0x7fe49c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 -- 172.21.15.105:0/2097779961 >> 172.21.15.105:0/2097779961 conn(0x7fe49c0fbd60 msgr2=0x7fe49c0fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 -- 172.21.15.105:0/2097779961 shutdown_connections 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 -- 172.21.15.105:0/2097779961 wait complete. 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 Processor -- start 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.363+0000 7fe4a29f4640 1 -- start start 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe4a29f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c102370 0x7fe49c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe4a29f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe49c104ca0 0x7fe49c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe4a29f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe49c1075d0 0x7fe49c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe4a29f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10d420 con 0x7fe49c102370 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe4a29f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10d2a0 con 0x7fe49c104ca0 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe4a29f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10d5a0 con 0x7fe49c1075d0 2024-03-26T17:41:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c102370 0x7fe49c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c102370 0x7fe49c1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40230/0 (socket says 172.21.15.105:40230) 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe49c104ca0 0x7fe49c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 -- 172.21.15.105:0/126429397 learned_addr learned my addr 172.21.15.105:0/126429397 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe4a0f91640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe49c1075d0 0x7fe49c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 -- 172.21.15.105:0/126429397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe49c1075d0 msgr2=0x7fe49c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe49c1075d0 0x7fe49c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 -- 172.21.15.105:0/126429397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe49c104ca0 msgr2=0x7fe49c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe49c104ca0 0x7fe49c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 -- 172.21.15.105:0/126429397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c19f600 con 0x7fe49c102370 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe49c104ca0 0x7fe49c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe4a0f91640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe49c1075d0 0x7fe49c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.364+0000 7fe49bfff640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c102370 0x7fe49c1979e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00e9f0 tx=0x7fe48c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.365+0000 7fe4997fa640 1 -- 172.21.15.105:0/126429397 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c00cd80 con 0x7fe49c102370 2024-03-26T17:41:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.365+0000 7fe4997fa640 1 -- 172.21.15.105:0/126429397 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe48c0108e0 con 0x7fe49c102370 2024-03-26T17:41:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.365+0000 7fe4a29f4640 1 -- 172.21.15.105:0/126429397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c19f8f0 con 0x7fe49c102370 2024-03-26T17:41:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.365+0000 7fe4a29f4640 1 -- 172.21.15.105:0/126429397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c19fe40 con 0x7fe49c102370 2024-03-26T17:41:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.365+0000 7fe4997fa640 1 -- 172.21.15.105:0/126429397 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c018920 con 0x7fe49c102370 2024-03-26T17:41:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.367+0000 7fe4997fa640 1 -- 172.21.15.105:0/126429397 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe48c010a90 con 0x7fe49c102370 2024-03-26T17:41:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.367+0000 7fe4a55de740 1 Processor -- start 2024-03-26T17:41:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.367+0000 7fe4a55de740 1 -- start start 2024-03-26T17:41:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.367+0000 7fe4997fa640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4840a2fc0 0x7fe4840a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c059ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 0x55598c05c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a55de740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55598bfb90b0 con 0x55598c054fc0 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a55de740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55598bf61960 con 0x55598c05a130 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a55de740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55598bfb5060 con 0x55598c0545d0 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a0f91640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4840a2fc0 0x7fe4840a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4997fa640 1 -- 172.21.15.105:0/126429397 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe48c0cf8c0 con 0x7fe49c102370 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c059ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c059ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40244/0 (socket says 172.21.15.105:40244) 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a0f91640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 0x55598c05c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 -- 172.21.15.105:0/4073231998 learned_addr learned my addr 172.21.15.105:0/4073231998 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49bfff640 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 -- 172.21.15.105:0/4073231998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 msgr2=0x55598c0549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 -- 172.21.15.105:0/4073231998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 msgr2=0x55598c05c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 0x55598c05c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 -- 172.21.15.105:0/4073231998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598bf73c60 con 0x55598c054fc0 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49bfff640 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4a0f91640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4840a2fc0 0x7fe4840a5480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe494006010 tx=0x7fe494009930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c059ac0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe4900114f0 tx=0x7fe4900119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36260e64205e0fbf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4827fc640 1 -- 172.21.15.105:0/4073231998 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490011310 con 0x55598c054fc0 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4827fc640 1 -- 172.21.15.105:0/4073231998 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4900054d0 con 0x55598c054fc0 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.368+0000 7fe4827fc640 1 -- 172.21.15.105:0/4073231998 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49000f580 con 0x55598c054fc0 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 -- 172.21.15.105:0/4073231998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 msgr2=0x55598c059ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c059ac0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe4900114f0 tx=0x7fe4900119b0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 -- 172.21.15.105:0/4073231998 shutdown_connections 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 0x55598c05c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c059ac0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 --2- 172.21.15.105:0/4073231998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 -- 172.21.15.105:0/4073231998 >> 172.21.15.105:0/4073231998 conn(0x55598c04fb40 msgr2=0x55598c04ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 -- 172.21.15.105:0/4073231998 shutdown_connections 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.369+0000 7fe4a55de740 1 -- 172.21.15.105:0/4073231998 wait complete. 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 1 -- start start 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 1 Processor -- start 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c057630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c058640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55598c05cec0 con 0x55598c05a130 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55598bf74db0 con 0x55598c054fc0 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55598c05d040 con 0x55598c0545d0 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a55de740 10 client.0 _ll_register_callbacks cb 0x55598bf79110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a0f91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c058640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe49bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c057630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe49bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34064/0 (socket says 172.21.15.105:34064) 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe4a0f91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c058640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40252/0 (socket says 172.21.15.105:40252) 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.370+0000 7fe49bfff640 1 -- 172.21.15.105:0/1910893462 learned_addr learned my addr 172.21.15.105:0/1910893462 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe49bfff640 1 -- 172.21.15.105:0/1910893462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 msgr2=0x55598c057630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe49bfff640 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c057630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe49bfff640 1 -- 172.21.15.105:0/1910893462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 msgr2=0x55598c058640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe49bfff640 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c058640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe49bfff640 1 -- 172.21.15.105:0/1910893462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598bccd210 con 0x55598c0545d0 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe4a0f91640 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c058640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe49bfff640 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0570f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0cd380 tx=0x7fe48c010de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f51a74ad637250d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe47b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:41:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1910893462 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c09bfb0 con 0x55598c0545d0 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe4a55de740 1 -- 172.21.15.105:0/1910893462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4900054d0 con 0x55598c0545d0 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1910893462 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe48c013cb0 con 0x55598c0545d0 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1910893462 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe48c013eb0 con 0x55598c0545d0 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe4a55de740 10 client.26647 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe4a55de740 1 -- 172.21.15.105:0/1910893462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55598bd99620 con 0x55598c0545d0 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.371+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1910893462 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4900054d0 con 0x55598c0545d0 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1910893462 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe48c0a4530 con 0x55598c0545d0 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe4a55de740 10 client.26647 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe4a55de740 10 client.26647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe4a55de740 20 client.26647 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55598c072a40 0x55598c074e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe4a55de740 4 client.26647 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe4a55de740 1 -- 172.21.15.105:0/1910893462 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55598bfc65f0 con 0x55598c072a40 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55598c072a40 0x55598c074e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55598c072a40 0x55598c074e50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bffcac51cb3a97d3 server_cookie=a5d94868a0df1f04 in_seq=0 out_seq=0 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.372+0000 7fe47b7fe640 10 client.26647 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.373+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1910893462 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55598bfc65f0 con 0x55598c072a40 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.373+0000 7fe47b7fe640 10 client.26647 handle_command_reply: tid=0 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.373+0000 7fe4a55de740 1 client.26647 shutdown 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 -- 172.21.15.105:0/1910893462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 msgr2=0x55598c0570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0570f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0cd380 tx=0x7fe48c010de0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 -- 172.21.15.105:0/1910893462 shutdown_connections 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55598c072a40 0x55598c074e50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c058640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe47b7fe640 0 client.26647 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c057630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1910893462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c0570f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 -- 172.21.15.105:0/1910893462 >> 172.21.15.105:0/1910893462 conn(0x55598c04fb40 msgr2=0x55598c04edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 -- 172.21.15.105:0/1910893462 shutdown_connections 2024-03-26T17:41:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 1 -- 172.21.15.105:0/1910893462 wait complete. 2024-03-26T17:41:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.374+0000 7fe4a55de740 20 client.26647 trim_cache size 0 max 16384 2024-03-26T17:41:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.391+0000 7fe4a55de740 1 Processor -- start 2024-03-26T17:41:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe4a55de740 1 -- start start 2024-03-26T17:41:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c053850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c056d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 0x55598c057240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe4a55de740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c1a6a80 con 0x55598c054fc0 2024-03-26T17:41:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe4a55de740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c1a85c0 con 0x55598c05a130 2024-03-26T17:41:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe4a55de740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c1a7150 con 0x55598c0545d0 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c056d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe4a0f91640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 0x55598c057240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c056d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40254/0 (socket says 172.21.15.105:40254) 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 -- 172.21.15.105:0/1553133067 learned_addr learned my addr 172.21.15.105:0/1553133067 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49bfff640 1 --2- 172.21.15.105:0/1553133067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c053850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 -- 172.21.15.105:0/1553133067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 msgr2=0x55598c053850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/1553133067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c053850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 -- 172.21.15.105:0/1553133067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 msgr2=0x55598c057240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/1553133067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 0x55598c057240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 -- 172.21.15.105:0/1553133067 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598c050590 con 0x55598c054fc0 2024-03-26T17:41:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.392+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/1553133067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c056d00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe490009230 tx=0x7fe490009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44356cdf0e28e206 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe481ffb640 1 -- 172.21.15.105:0/1553133067 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49002eab0 con 0x55598c054fc0 2024-03-26T17:41:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe481ffb640 1 -- 172.21.15.105:0/1553133067 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe49002e7d0 con 0x55598c054fc0 2024-03-26T17:41:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe481ffb640 1 -- 172.21.15.105:0/1553133067 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490004970 con 0x55598c054fc0 2024-03-26T17:41:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 -- 172.21.15.105:0/1553133067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 msgr2=0x55598c056d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1553133067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c056d00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fe490009230 tx=0x7fe490009310 comp rx=0 tx=0).stop 2024-03-26T17:41:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 -- 172.21.15.105:0/1553133067 shutdown_connections 2024-03-26T17:41:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1553133067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c05a130 0x55598c057240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1553133067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c054fc0 0x55598c056d00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1553133067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c053850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 -- 172.21.15.105:0/1553133067 >> 172.21.15.105:0/1553133067 conn(0x55598c04fb40 msgr2=0x55598c052da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 -- 172.21.15.105:0/1553133067 shutdown_connections 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.393+0000 7fe4a55de740 1 -- 172.21.15.105:0/1553133067 wait complete. 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 1 -- start start 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 1 Processor -- start 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c06c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c06cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c06de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c1a7ef0 con 0x55598c05a130 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c1a7820 con 0x55598c054fc0 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c1a4a60 con 0x55598c0545d0 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a55de740 10 client.0 _ll_register_callbacks cb 0x7fe48c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c06cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe4a0f91640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c06cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43398/0 (socket says 172.21.15.105:43398) 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe49b7fe640 1 -- 172.21.15.105:0/1242619765 learned_addr learned my addr 172.21.15.105:0/1242619765 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.394+0000 7fe49bfff640 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c06c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe49b7fe640 1 -- 172.21.15.105:0/1242619765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 msgr2=0x55598c06c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c06c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe49b7fe640 1 -- 172.21.15.105:0/1242619765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 msgr2=0x55598c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c06de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe49b7fe640 1 -- 172.21.15.105:0/1242619765 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598c06e3f0 con 0x55598c054fc0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe49b7fe640 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c06cdf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe490009bd0 tx=0x7fe490005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40aa00adaf7e4394 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe47b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1242619765 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49003d030 con 0x55598c054fc0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe4a55de740 1 -- 172.21.15.105:0/1242619765 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe49002e7d0 con 0x55598c054fc0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1242619765 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe49002b040 con 0x55598c054fc0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1242619765 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe49002eb70 con 0x55598c054fc0 2024-03-26T17:41:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe4a55de740 10 client.26654 fetch_fsmap learned FSMap version 30 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe4a55de740 1 -- 172.21.15.105:0/1242619765 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55598c06f700 con 0x55598c054fc0 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.395+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1242619765 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe49002e7d0 con 0x55598c054fc0 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1242619765 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe4900048b0 con 0x55598c054fc0 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe4a55de740 10 client.26654 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe4a55de740 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe4a55de740 20 client.26654 populate_metadata read hostname 'smithi105' 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55598c0d9150 0x55598c0db570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe4a55de740 4 client.26654 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe4a55de740 1 -- 172.21.15.105:0/1242619765 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55598bfc0940 con 0x55598c0d9150 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe49bfff640 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55598c0d9150 0x55598c0db570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe47b7fe640 10 client.26654 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:58.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.396+0000 7fe49bfff640 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55598c0d9150 0x55598c0db570 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15fd236de44de001 server_cookie=4b7bce562c60faf in_seq=0 out_seq=0 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe47b7fe640 1 -- 172.21.15.105:0/1242619765 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55598bfc0940 con 0x55598c0d9150 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe47b7fe640 10 client.26654 handle_command_reply: tid=0 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 client.26654 shutdown 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 -- 172.21.15.105:0/1242619765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 msgr2=0x55598c06cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c06cdf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe490009bd0 tx=0x7fe490005b40 comp rx=0 tx=0).stop 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 -- 172.21.15.105:0/1242619765 shutdown_connections 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55598c0d9150 0x55598c0db570 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe47b7fe640 0 client.26654 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55598c05a130 0x55598c06de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55598c054fc0 0x55598c06cdf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 --2- 172.21.15.105:0/1242619765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55598c0545d0 0x55598c06c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 -- 172.21.15.105:0/1242619765 >> 172.21.15.105:0/1242619765 conn(0x55598c04fb40 msgr2=0x55598c056d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 -- 172.21.15.105:0/1242619765 shutdown_connections 2024-03-26T17:41:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 1 -- 172.21.15.105:0/1242619765 wait complete. 2024-03-26T17:41:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.436+0000 7fe4a55de740 20 client.26654 trim_cache size 0 max 16384 2024-03-26T17:41:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:41:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:41:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:41:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:41:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 -- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4840a2fc0 msgr2=0x7fe4840a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4840a2fc0 0x7fe4840a5480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe494006010 tx=0x7fe494009930 comp rx=0 tx=0).stop 2024-03-26T17:41:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 -- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c102370 msgr2=0x7fe49c1979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:41:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c102370 0x7fe49c1979e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00e9f0 tx=0x7fe48c00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 -- 172.21.15.105:0/126429397 shutdown_connections 2024-03-26T17:41:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4840a2fc0 0x7fe4840a5480 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe49c1075d0 0x7fe49c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe49c104ca0 0x7fe49c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 --2- 172.21.15.105:0/126429397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe49c102370 0x7fe49c1979e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:41:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 -- 172.21.15.105:0/126429397 >> 172.21.15.105:0/126429397 conn(0x7fe49c0fbd60 msgr2=0x7fe49c102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:41:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 -- 172.21.15.105:0/126429397 shutdown_connections 2024-03-26T17:41:58.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:41:58.437+0000 7fe4a29f4640 1 -- 172.21.15.105:0/126429397 wait complete. 2024-03-26T17:41:59.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 95888 3.15 MB/sec execute 42 sec latency 41.324 ms 2024-03-26T17:42:00.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 96305 3.11 MB/sec execute 43 sec latency 16.354 ms 2024-03-26T17:42:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:41:59 smithi144 ceph-mon[36095]: pgmap v457: 97 pgs: 97 active+clean; 28 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:42:00.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:41:59 smithi105 ceph-mon[32280]: pgmap v457: 97 pgs: 97 active+clean; 28 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:42:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:41:59 smithi135 ceph-mon[35636]: pgmap v457: 97 pgs: 97 active+clean; 28 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:42:01.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 96854 3.09 MB/sec execute 44 sec latency 29.954 ms 2024-03-26T17:42:02.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 97504 3.09 MB/sec execute 45 sec latency 15.251 ms 2024-03-26T17:42:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:01 smithi144 ceph-mon[36095]: pgmap v458: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-26T17:42:02.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:01 smithi105 ceph-mon[32280]: pgmap v458: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-26T17:42:02.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:01 smithi135 ceph-mon[35636]: pgmap v458: 97 pgs: 97 active+clean; 29 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-26T17:42:03.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 98182 3.06 MB/sec execute 46 sec latency 16.564 ms 2024-03-26T17:42:04.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 99039 3.13 MB/sec execute 47 sec latency 17.965 ms 2024-03-26T17:42:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:03 smithi144 ceph-mon[36095]: pgmap v459: 97 pgs: 97 active+clean; 25 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:42:04.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:03 smithi105 ceph-mon[32280]: pgmap v459: 97 pgs: 97 active+clean; 25 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:42:04.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:03 smithi135 ceph-mon[35636]: pgmap v459: 97 pgs: 97 active+clean; 25 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:42:05.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 99492 3.16 MB/sec execute 48 sec latency 17.183 ms 2024-03-26T17:42:06.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 99888 3.12 MB/sec execute 49 sec latency 16.413 ms 2024-03-26T17:42:06.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:05 smithi144 ceph-mon[36095]: pgmap v460: 97 pgs: 97 active+clean; 24 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:42:06.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:05 smithi105 ceph-mon[32280]: pgmap v460: 97 pgs: 97 active+clean; 24 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:42:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:05 smithi135 ceph-mon[35636]: pgmap v460: 97 pgs: 97 active+clean; 24 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:42:07.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 100454 3.10 MB/sec execute 50 sec latency 26.770 ms 2024-03-26T17:42:08.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 101137 3.11 MB/sec execute 51 sec latency 16.439 ms 2024-03-26T17:42:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:07 smithi144 ceph-mon[36095]: pgmap v461: 97 pgs: 97 active+clean; 25 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-03-26T17:42:08.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:07 smithi105 ceph-mon[32280]: pgmap v461: 97 pgs: 97 active+clean; 25 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-03-26T17:42:08.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:07 smithi135 ceph-mon[35636]: pgmap v461: 97 pgs: 97 active+clean; 25 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-03-26T17:42:09.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 101762 3.08 MB/sec execute 52 sec latency 22.546 ms 2024-03-26T17:42:10.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 102625 3.16 MB/sec execute 53 sec latency 16.462 ms 2024-03-26T17:42:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:09 smithi144 ceph-mon[36095]: pgmap v462: 97 pgs: 97 active+clean; 28 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-03-26T17:42:10.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:09 smithi105 ceph-mon[32280]: pgmap v462: 97 pgs: 97 active+clean; 28 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-03-26T17:42:10.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:09 smithi135 ceph-mon[35636]: pgmap v462: 97 pgs: 97 active+clean; 28 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-03-26T17:42:11.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 103026 3.16 MB/sec execute 54 sec latency 16.519 ms 2024-03-26T17:42:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 -- 172.21.15.105:0/3953193181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14104d30 msgr2=0x7fcb14107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 --2- 172.21.15.105:0/3953193181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14104d30 0x7fcb14107120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009900 tx=0x7fcb0802bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 -- 172.21.15.105:0/3953193181 shutdown_connections 2024-03-26T17:42:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 --2- 172.21.15.105:0/3953193181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb14107660 0x7fcb14109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 --2- 172.21.15.105:0/3953193181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14104d30 0x7fcb14107120 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 --2- 172.21.15.105:0/3953193181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb14102400 0x7fcb141047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 -- 172.21.15.105:0/3953193181 >> 172.21.15.105:0/3953193181 conn(0x7fcb140fbd90 msgr2=0x7fcb140fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 -- 172.21.15.105:0/3953193181 shutdown_connections 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 -- 172.21.15.105:0/3953193181 wait complete. 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.532+0000 7fcb198bd640 1 Processor -- start 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb198bd640 1 -- start start 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb198bd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb14102400 0x7fcb14101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb198bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb14104d30 0x7fcb141003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb198bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14107660 0x7fcb141008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb198bd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1410d4a0 con 0x7fcb14107660 2024-03-26T17:42:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb198bd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1410d320 con 0x7fcb14102400 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb198bd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1410d620 con 0x7fcb14104d30 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14107660 0x7fcb141008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb14102400 0x7fcb14101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14107660 0x7fcb141008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38956/0 (socket says 172.21.15.105:38956) 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 -- 172.21.15.105:0/2590492386 learned_addr learned my addr 172.21.15.105:0/2590492386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb127fc640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb14104d30 0x7fcb141003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 -- 172.21.15.105:0/2590492386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb14104d30 msgr2=0x7fcb141003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb14104d30 0x7fcb141003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 -- 172.21.15.105:0/2590492386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb14102400 msgr2=0x7fcb14101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb14102400 0x7fcb14101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 -- 172.21.15.105:0/2590492386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb14101180 con 0x7fcb14107660 2024-03-26T17:42:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb137fe640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14107660 0x7fcb141008f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00eb10 tx=0x7fcb0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.533+0000 7fcb188bb640 1 -- 172.21.15.105:0/2590492386 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c004b20 con 0x7fcb14107660 2024-03-26T17:42:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.534+0000 7fcb198bd640 1 -- 172.21.15.105:0/2590492386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb141a1c00 con 0x7fcb14107660 2024-03-26T17:42:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.534+0000 7fcb188bb640 1 -- 172.21.15.105:0/2590492386 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c004cc0 con 0x7fcb14107660 2024-03-26T17:42:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.534+0000 7fcb198bd640 1 -- 172.21.15.105:0/2590492386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb141a2170 con 0x7fcb14107660 2024-03-26T17:42:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.534+0000 7fcb188bb640 1 -- 172.21.15.105:0/2590492386 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c010690 con 0x7fcb14107660 2024-03-26T17:42:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.536+0000 7fcb188bb640 1 -- 172.21.15.105:0/2590492386 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcb0c005020 con 0x7fcb14107660 2024-03-26T17:42:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.536+0000 7fcb1c583740 1 Processor -- start 2024-03-26T17:42:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.536+0000 7fcb1c583740 1 -- start start 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea698909f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea69895ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69896130 0x55ea698985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb1c583740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ea697f50b0 con 0x55ea69890fc0 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb1c583740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ea6979d960 con 0x55ea69896130 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb1c583740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea697f1060 con 0x55ea698905d0 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb188bb640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2fc0 0x7fcafc0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea698909f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea698909f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42528/0 (socket says 172.21.15.105:42528) 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea69895ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 -- 172.21.15.105:0/122326692 learned_addr learned my addr 172.21.15.105:0/122326692 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69896130 0x55ea698985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb137fe640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2fc0 0x7fcafc0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 -- 172.21.15.105:0/122326692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69896130 msgr2=0x55ea698985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69896130 0x55ea698985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 -- 172.21.15.105:0/122326692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 msgr2=0x55ea69895ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea69895ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 -- 172.21.15.105:0/122326692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea697afc60 con 0x55ea698905d0 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea69895ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb188bb640 1 -- 172.21.15.105:0/2590492386 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcb0c0cfb40 con 0x7fcb14107660 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb127fc640 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea698909f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb08006010 tx=0x7fcb0802c5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a09c4a8c6771947 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.537+0000 7fcb137fe640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2fc0 0x7fcafc0a5480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c096cd0 tx=0x7fcb0c06ae20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcaf9ffb640 1 -- 172.21.15.105:0/122326692 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0804a070 con 0x55ea698905d0 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcaf9ffb640 1 -- 172.21.15.105:0/122326692 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb08005050 con 0x55ea698905d0 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcaf9ffb640 1 -- 172.21.15.105:0/122326692 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080044e0 con 0x55ea698905d0 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 -- 172.21.15.105:0/122326692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 msgr2=0x55ea698909f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea698909f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb08006010 tx=0x7fcb0802c5f0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 -- 172.21.15.105:0/122326692 shutdown_connections 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69896130 0x55ea698985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea69895ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 --2- 172.21.15.105:0/122326692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea698909f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 -- 172.21.15.105:0/122326692 >> 172.21.15.105:0/122326692 conn(0x55ea6988bb40 msgr2=0x55ea6988bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 -- 172.21.15.105:0/122326692 shutdown_connections 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.538+0000 7fcb1c583740 1 -- 172.21.15.105:0/122326692 wait complete. 2024-03-26T17:42:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 1 -- start start 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 1 Processor -- start 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea698905d0 0x55ea6989a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea6989a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea69896130 0x55ea698a1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ea69899080 con 0x55ea69890fc0 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ea697b0db0 con 0x55ea698905d0 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea69899200 con 0x55ea69896130 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb1c583740 10 client.0 _ll_register_callbacks cb 0x55ea697b5110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb127fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea698905d0 0x55ea6989a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb127fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea698905d0 0x55ea6989a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41908/0 (socket says 172.21.15.105:41908) 2024-03-26T17:42:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb127fc640 1 -- 172.21.15.105:0/3648238443 learned_addr learned my addr 172.21.15.105:0/3648238443 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea6989a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.539+0000 7fcb127fc640 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea69896130 0x55ea698a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb127fc640 1 -- 172.21.15.105:0/3648238443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea69896130 msgr2=0x55ea698a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb127fc640 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea69896130 0x55ea698a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb127fc640 1 -- 172.21.15.105:0/3648238443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 msgr2=0x55ea6989a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb127fc640 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea6989a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb127fc640 1 -- 172.21.15.105:0/3648238443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea69509210 con 0x55ea698905d0 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb127fc640 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea698905d0 0x55ea6989a3f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb08036040 tx=0x7fcb0802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a5ae1201f791195 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcaf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/3648238443 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0804a070 con 0x55ea698905d0 2024-03-26T17:42:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/3648238443 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcb08005200 con 0x55ea698905d0 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb1c583740 1 -- 172.21.15.105:0/3648238443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb08005050 con 0x55ea698905d0 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/3648238443 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb08005400 con 0x55ea698905d0 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb1c583740 10 client.26666 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.540+0000 7fcb1c583740 1 -- 172.21.15.105:0/3648238443 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea695d5620 con 0x55ea698905d0 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/3648238443 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb08005050 con 0x55ea698905d0 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/3648238443 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcb08043450 con 0x55ea698905d0 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcb1c583740 10 client.26666 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcb1c583740 10 client.26666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcb1c583740 20 client.26666 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcb1c583740 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea698a5e60 0x55ea698a8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcb1c583740 4 client.26666 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcb1c583740 1 -- 172.21.15.105:0/3648238443 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea698025f0 con 0x55ea698a5e60 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea698a5e60 0x55ea698a8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcaf2ffd640 10 client.26666 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.541+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea698a5e60 0x55ea698a8250 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27128709c6a99f5 server_cookie=4e4863422bb8cdf9 in_seq=0 out_seq=0 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/3648238443 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ea698025f0 con 0x55ea698a5e60 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcaf2ffd640 10 client.26666 handle_command_reply: tid=0 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 client.26666 shutdown 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 -- 172.21.15.105:0/3648238443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea698905d0 msgr2=0x55ea6989a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea698905d0 0x55ea6989a3f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb08036040 tx=0x7fcb0802bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 -- 172.21.15.105:0/3648238443 shutdown_connections 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea698a5e60 0x55ea698a8250 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea69896130 0x55ea698a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69890fc0 0x55ea6989a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 --2- 172.21.15.105:0/3648238443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea698905d0 0x55ea6989a3f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcaf2ffd640 0 client.26666 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 -- 172.21.15.105:0/3648238443 >> 172.21.15.105:0/3648238443 conn(0x55ea6988bb40 msgr2=0x55ea6988adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 -- 172.21.15.105:0/3648238443 shutdown_connections 2024-03-26T17:42:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 1 -- 172.21.15.105:0/3648238443 wait complete. 2024-03-26T17:42:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.542+0000 7fcb1c583740 20 client.26666 trim_cache size 0 max 16384 2024-03-26T17:42:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb1c583740 1 Processor -- start 2024-03-26T17:42:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb1c583740 1 -- start start 2024-03-26T17:42:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea69891a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea698a1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea69891fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb1c583740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb141ab070 con 0x55ea69896130 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb1c583740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb141acbb0 con 0x55ea69890fc0 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb1c583740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb141ab740 con 0x55ea698905d0 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea69891a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea698a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea69891a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42554/0 (socket says 172.21.15.105:42554) 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb127fc640 1 -- 172.21.15.105:0/2362355492 learned_addr learned my addr 172.21.15.105:0/2362355492 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb127fc640 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea69891fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb12ffd640 1 -- 172.21.15.105:0/2362355492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 msgr2=0x55ea69891a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.559+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea69891a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.560+0000 7fcb12ffd640 1 -- 172.21.15.105:0/2362355492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 msgr2=0x55ea69891fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.560+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea69891fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.560+0000 7fcb12ffd640 1 -- 172.21.15.105:0/2362355492 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea6988c8b0 con 0x55ea69890fc0 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.560+0000 7fcb127fc640 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea69891a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.560+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea698a1900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55ea698a8930 tx=0x7fcb04005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4cdbade4a1b4a97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.560+0000 7fcaf97fa640 1 -- 172.21.15.105:0/2362355492 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04005220 con 0x55ea69890fc0 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.560+0000 7fcaf97fa640 1 -- 172.21.15.105:0/2362355492 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb04024030 con 0x55ea69890fc0 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.560+0000 7fcaf97fa640 1 -- 172.21.15.105:0/2362355492 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04004b00 con 0x55ea69890fc0 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- 172.21.15.105:0/2362355492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 msgr2=0x55ea698a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea698a1900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55ea698a8930 tx=0x7fcb04005f90 comp rx=0 tx=0).stop 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- 172.21.15.105:0/2362355492 shutdown_connections 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea69891fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea698a1900 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 --2- 172.21.15.105:0/2362355492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea69891a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- 172.21.15.105:0/2362355492 >> 172.21.15.105:0/2362355492 conn(0x55ea6988bb40 msgr2=0x55ea6988eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- 172.21.15.105:0/2362355492 shutdown_connections 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- 172.21.15.105:0/2362355492 wait complete. 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- start start 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 Processor -- start 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea6989f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea6989f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea698a0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb141ac4e0 con 0x55ea69896130 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb141abe10 con 0x55ea69890fc0 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.561+0000 7fcb1c583740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb141a9050 con 0x55ea698905d0 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb1c583740 10 client.0 _ll_register_callbacks cb 0x7fcb08048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea6989f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea6989f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea698a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea6989f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41930/0 (socket says 172.21.15.105:41930) 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb12ffd640 1 -- 172.21.15.105:0/821689501 learned_addr learned my addr 172.21.15.105:0/821689501 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb127fc640 1 -- 172.21.15.105:0/821689501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 msgr2=0x55ea6989f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb127fc640 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea6989f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb127fc640 1 -- 172.21.15.105:0/821689501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 msgr2=0x55ea6989f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb127fc640 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea6989f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb127fc640 1 -- 172.21.15.105:0/821689501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea698a0b40 con 0x55ea69896130 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea6989f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb127fc640 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea698a0550 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcb08043820 tx=0x7fcb08004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a71117380ef93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcaf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/821689501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08064040 con 0x55ea69896130 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/821689501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcb08043a10 con 0x55ea69896130 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.562+0000 7fcb1c583740 1 -- 172.21.15.105:0/821689501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb04024030 con 0x55ea69896130 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/821689501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcb0804a040 con 0x55ea69896130 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb1c583740 10 client.16830 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb1c583740 1 -- 172.21.15.105:0/821689501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea6990c050 con 0x55ea69896130 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/821689501 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb04024030 con 0x55ea69896130 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/821689501 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcb080035f0 con 0x55ea69896130 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb1c583740 10 client.16830 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb1c583740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb1c583740 20 client.16830 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb1c583740 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea6990c950 0x55ea6990ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb1c583740 4 client.16830 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:42:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb1c583740 1 -- 172.21.15.105:0/821689501 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea697fc940 con 0x55ea6990c950 2024-03-26T17:42:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.563+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea6990c950 0x55ea6990ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.564+0000 7fcb12ffd640 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea6990c950 0x55ea6990ed70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23cd22fcd233fa82 server_cookie=aed0744633c42ebd in_seq=0 out_seq=0 2024-03-26T17:42:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.564+0000 7fcaf2ffd640 10 client.16830 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.608+0000 7fcaf2ffd640 1 -- 172.21.15.105:0/821689501 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea697fc940 con 0x55ea6990c950 2024-03-26T17:42:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.608+0000 7fcaf2ffd640 10 client.16830 handle_command_reply: tid=0 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.608+0000 7fcb1c583740 1 client.16830 shutdown 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.608+0000 7fcb1c583740 1 -- 172.21.15.105:0/821689501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 msgr2=0x55ea698a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.608+0000 7fcb1c583740 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea698a0550 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcb08043820 tx=0x7fcb08004040 comp rx=0 tx=0).stop 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 1 -- 172.21.15.105:0/821689501 shutdown_connections 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea6990c950 0x55ea6990ed70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea69896130 0x55ea698a0550 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea69890fc0 0x55ea6989f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcaf2ffd640 0 client.16830 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 1 --2- 172.21.15.105:0/821689501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea698905d0 0x55ea6989f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 1 -- 172.21.15.105:0/821689501 >> 172.21.15.105:0/821689501 conn(0x55ea6988bb40 msgr2=0x55ea69891a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 1 -- 172.21.15.105:0/821689501 shutdown_connections 2024-03-26T17:42:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 1 -- 172.21.15.105:0/821689501 wait complete. 2024-03-26T17:42:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb1c583740 20 client.16830 trim_cache size 0 max 16384 2024-03-26T17:42:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:42:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:42:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:42:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:42:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb198bd640 1 -- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2fc0 msgr2=0x7fcafc0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb198bd640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2fc0 0x7fcafc0a5480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c096cd0 tx=0x7fcb0c06ae20 comp rx=0 tx=0).stop 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb198bd640 1 -- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14107660 msgr2=0x7fcb141008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.609+0000 7fcb198bd640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14107660 0x7fcb141008f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00eb10 tx=0x7fcb0c00c2d0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.610+0000 7fcb198bd640 1 -- 172.21.15.105:0/2590492386 shutdown_connections 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.610+0000 7fcb198bd640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2fc0 0x7fcafc0a5480 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.610+0000 7fcb198bd640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb14107660 0x7fcb141008f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.610+0000 7fcb198bd640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb14104d30 0x7fcb141003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.610+0000 7fcb198bd640 1 --2- 172.21.15.105:0/2590492386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb14102400 0x7fcb14101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.610+0000 7fcb198bd640 1 -- 172.21.15.105:0/2590492386 >> 172.21.15.105:0/2590492386 conn(0x7fcb140fbd90 msgr2=0x7fcb14102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.610+0000 7fcb198bd640 1 -- 172.21.15.105:0/2590492386 shutdown_connections 2024-03-26T17:42:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:11.610+0000 7fcb198bd640 1 -- 172.21.15.105:0/2590492386 wait complete. 2024-03-26T17:42:12.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 103407 3.13 MB/sec execute 55 sec latency 38.161 ms 2024-03-26T17:42:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:11 smithi144 ceph-mon[36095]: pgmap v463: 97 pgs: 97 active+clean; 28 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:42:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:11 smithi105 ceph-mon[32280]: pgmap v463: 97 pgs: 97 active+clean; 28 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:42:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:11 smithi135 ceph-mon[35636]: pgmap v463: 97 pgs: 97 active+clean; 28 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:42:13.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 103952 3.11 MB/sec execute 56 sec latency 22.432 ms 2024-03-26T17:42:14.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 104602 3.11 MB/sec execute 57 sec latency 16.386 ms 2024-03-26T17:42:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:13 smithi144 ceph-mon[36095]: pgmap v464: 97 pgs: 97 active+clean; 26 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-03-26T17:42:14.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:13 smithi105 ceph-mon[32280]: pgmap v464: 97 pgs: 97 active+clean; 26 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-03-26T17:42:14.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:13 smithi135 ceph-mon[35636]: pgmap v464: 97 pgs: 97 active+clean; 26 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-03-26T17:42:15.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 105274 3.09 MB/sec execute 58 sec latency 17.856 ms 2024-03-26T17:42:16.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 106082 3.14 MB/sec execute 59 sec latency 15.759 ms 2024-03-26T17:42:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:15 smithi144 ceph-mon[36095]: pgmap v465: 97 pgs: 97 active+clean; 25 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2024-03-26T17:42:16.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:15 smithi105 ceph-mon[32280]: pgmap v465: 97 pgs: 97 active+clean; 25 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2024-03-26T17:42:16.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:15 smithi135 ceph-mon[35636]: pgmap v465: 97 pgs: 97 active+clean; 25 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.7 MiB/s wr, 122 op/s 2024-03-26T17:42:17.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 106526 3.16 MB/sec execute 60 sec latency 20.520 ms 2024-03-26T17:42:17.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:42:17.333 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:42:17.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 -- 172.21.15.105:0/3290562566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da0103530 msgr2=0x7f0da0109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3290562566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da0103530 0x7f0da0109dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009cd0 tx=0x7f0d9802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 -- 172.21.15.105:0/3290562566 shutdown_connections 2024-03-26T17:42:17.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3290562566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da0103530 0x7f0da0109dc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3290562566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0da0102b70 0x7f0da0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3290562566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da00ffba0 0x7f0da00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 -- 172.21.15.105:0/3290562566 >> 172.21.15.105:0/3290562566 conn(0x7f0da00fb4b0 msgr2=0x7f0da00fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:17.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 -- 172.21.15.105:0/3290562566 shutdown_connections 2024-03-26T17:42:17.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 -- 172.21.15.105:0/3290562566 wait complete. 2024-03-26T17:42:17.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 Processor -- start 2024-03-26T17:42:17.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.460+0000 7f0da6d37640 1 -- start start 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da6d37640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0da00ffba0 0x7f0da0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da6d37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da0102b70 0x7f0da0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da6d37640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da0103530 0x7f0da019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da6d37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0da010d430 con 0x7f0da0102b70 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da6d37640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0da010d2b0 con 0x7f0da00ffba0 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da6d37640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0da010d5b0 con 0x7f0da0103530 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da0102b70 0x7f0da0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da0102b70 0x7f0da0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59804/0 (socket says 172.21.15.105:59804) 2024-03-26T17:42:17.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 -- 172.21.15.105:0/3952449488 learned_addr learned my addr 172.21.15.105:0/3952449488 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0da00ffba0 0x7f0da0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da52d4640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da0103530 0x7f0da019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 -- 172.21.15.105:0/3952449488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da0103530 msgr2=0x7f0da019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da0103530 0x7f0da019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 -- 172.21.15.105:0/3952449488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0da00ffba0 msgr2=0x7f0da0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0da00ffba0 0x7f0da0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 -- 172.21.15.105:0/3952449488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da019f6f0 con 0x7f0da0102b70 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da0102b70 0x7f0da0198010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e9f0 tx=0x7f0d8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9dffb640 1 -- 172.21.15.105:0/3952449488 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00cd80 con 0x7f0da0102b70 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9dffb640 1 -- 172.21.15.105:0/3952449488 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d8c0054d0 con 0x7f0da0102b70 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0da6d37640 1 -- 172.21.15.105:0/3952449488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da019f9e0 con 0x7f0da0102b70 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.461+0000 7f0d9dffb640 1 -- 172.21.15.105:0/3952449488 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c010490 con 0x7f0da0102b70 2024-03-26T17:42:17.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.462+0000 7f0da6d37640 1 -- 172.21.15.105:0/3952449488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da019fee0 con 0x7f0da0102b70 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.464+0000 7f0da99f7740 1 Processor -- start 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.464+0000 7f0d9dffb640 1 -- 172.21.15.105:0/3952449488 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0d8c005020 con 0x7f0da0102b70 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.464+0000 7f0da99f7740 1 -- start start 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6bede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da99f7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b11a61e980 con 0x55b11a6b98f0 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da99f7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b11a5c6be0 con 0x55b11a6bf450 2024-03-26T17:42:17.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da99f7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b11a61a930 con 0x55b11a6ba2e0 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6b9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59816/0 (socket says 172.21.15.105:59816) 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 -- 172.21.15.105:0/2111867886 learned_addr learned my addr 172.21.15.105:0/2111867886 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6bede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 -- 172.21.15.105:0/2111867886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 msgr2=0x55b11a6bede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6bede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 -- 172.21.15.105:0/2111867886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 msgr2=0x55b11a6c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 -- 172.21.15.105:0/2111867886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b11a3fc7d0 con 0x55b11a6b98f0 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6bede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:17.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0d9dffb640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d900a2f20 0x7f0d900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da52d4640 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6b9d10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0d98005e30 tx=0x7f0d98009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bce502f1cace100 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d900a2f20 0x7f0d900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.465+0000 7f0d9dffb640 1 -- 172.21.15.105:0/3952449488 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f0d8c0cf770 con 0x7f0da0102b70 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.466+0000 7f0d8affd640 1 -- 172.21.15.105:0/2111867886 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98043070 con 0x55b11a6b98f0 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.466+0000 7f0d8affd640 1 -- 172.21.15.105:0/2111867886 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d98032d70 con 0x55b11a6b98f0 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.466+0000 7f0d8affd640 1 -- 172.21.15.105:0/2111867886 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98039040 con 0x55b11a6b98f0 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.466+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d900a2f20 0x7f0d900a53e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55b11a6ba6e0 tx=0x7f0d94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.466+0000 7f0da99f7740 1 -- 172.21.15.105:0/2111867886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 msgr2=0x55b11a6b9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.466+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6b9d10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0d98005e30 tx=0x7f0d98009800 comp rx=0 tx=0).stop 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 -- 172.21.15.105:0/2111867886 shutdown_connections 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6bede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2111867886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6b9d10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 -- 172.21.15.105:0/2111867886 >> 172.21.15.105:0/2111867886 conn(0x55b11a6b4e60 msgr2=0x55b11a6b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 -- 172.21.15.105:0/2111867886 shutdown_connections 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 -- 172.21.15.105:0/2111867886 wait complete. 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 -- start start 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 Processor -- start 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6bf450 0x55b11a6cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b11a6c23a0 con 0x55b11a6bf450 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b11a5da160 con 0x55b11a6ba2e0 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.467+0000 7f0da99f7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b11a6c2520 con 0x55b11a6b98f0 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6bf450 0x55b11a6cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da99f7740 10 client.0 _ll_register_callbacks cb 0x55b11a616720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:17.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da4ad3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6bf450 0x55b11a6cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59830/0 (socket says 172.21.15.105:59830) 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 -- 172.21.15.105:0/215379455 learned_addr learned my addr 172.21.15.105:0/215379455 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 -- 172.21.15.105:0/215379455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 msgr2=0x55b11a6c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 -- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6ba2e0 msgr2=0x55b11a6c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 -- 172.21.15.105:0/215379455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b11a3323f0 con 0x55b11a6bf450 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da52d4640 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6bf450 0x55b11a6cb0b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009bf0 tx=0x7f0d980329e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbd9cd26c126de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0d7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:17.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0d7ffff640 1 -- 172.21.15.105:0/215379455 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9804c030 con 0x55b11a6bf450 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0d7ffff640 1 -- 172.21.15.105:0/215379455 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f0d98032b70 con 0x55b11a6bf450 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.468+0000 7f0da99f7740 1 -- 172.21.15.105:0/215379455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d98032d70 con 0x55b11a6bf450 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0d7ffff640 1 -- 172.21.15.105:0/215379455 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d98039070 con 0x55b11a6bf450 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da99f7740 10 client.16842 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da99f7740 1 -- 172.21.15.105:0/215379455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b11a2f20f0 con 0x55b11a6bf450 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0d7ffff640 1 -- 172.21.15.105:0/215379455 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d98032d70 con 0x55b11a6bf450 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0d7ffff640 1 -- 172.21.15.105:0/215379455 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0d980534d0 con 0x55b11a6bf450 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da99f7740 10 client.16842 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da99f7740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da99f7740 20 client.16842 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da99f7740 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b11a6cf5a0 0x55b11a6d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da99f7740 4 client.16842 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da99f7740 1 -- 172.21.15.105:0/215379455 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b11a62bec0 con 0x55b11a6cf5a0 2024-03-26T17:42:17.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.469+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b11a6cf5a0 0x55b11a6d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.470+0000 7f0d7ffff640 10 client.16842 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:17.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.470+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b11a6cf5a0 0x55b11a6d1990 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5448d27ba42774 server_cookie=104897c75aa74886 in_seq=0 out_seq=0 2024-03-26T17:42:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0d7ffff640 1 -- 172.21.15.105:0/215379455 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b11a62bec0 con 0x55b11a6cf5a0 2024-03-26T17:42:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0d7ffff640 10 client.16842 handle_command_reply: tid=0 2024-03-26T17:42:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 client.16842 shutdown 2024-03-26T17:42:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 -- 172.21.15.105:0/215379455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6bf450 msgr2=0x55b11a6cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6bf450 0x55b11a6cb0b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009bf0 tx=0x7f0d980329e0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 -- 172.21.15.105:0/215379455 shutdown_connections 2024-03-26T17:42:17.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b11a6cf5a0 0x55b11a6d1990 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6bf450 0x55b11a6cb0b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 --2- 172.21.15.105:0/215379455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 -- 172.21.15.105:0/215379455 >> 172.21.15.105:0/215379455 conn(0x55b11a6b4e60 msgr2=0x55b11a6b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0d7ffff640 0 client.16842 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 -- 172.21.15.105:0/215379455 shutdown_connections 2024-03-26T17:42:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 1 -- 172.21.15.105:0/215379455 wait complete. 2024-03-26T17:42:17.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.472+0000 7f0da99f7740 20 client.16842 trim_cache size 0 max 16384 2024-03-26T17:42:17.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da99f7740 1 Processor -- start 2024-03-26T17:42:17.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da99f7740 1 -- start start 2024-03-26T17:42:17.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6ba2e0 0x55b11a6cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da99f7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01a8ca0 con 0x55b11a6ba2e0 2024-03-26T17:42:17.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da99f7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01aa7e0 con 0x55b11a6bf450 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da99f7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01a9370 con 0x55b11a6b98f0 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6ba2e0 0x55b11a6cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6ba2e0 0x55b11a6cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59842/0 (socket says 172.21.15.105:59842) 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0d9ffff640 1 -- 172.21.15.105:0/3578988672 learned_addr learned my addr 172.21.15.105:0/3578988672 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.496+0000 7f0da52d4640 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d9ffff640 1 -- 172.21.15.105:0/3578988672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 msgr2=0x55b11a6c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d9ffff640 1 -- 172.21.15.105:0/3578988672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 msgr2=0x55b11a6c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d9ffff640 1 -- 172.21.15.105:0/3578988672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b11a6bafb0 con 0x55b11a6ba2e0 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6ba2e0 0x55b11a6cb070 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c098830 tx=0x7f0d8c0d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba005d72f3efa839 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d8a7fc640 1 -- 172.21.15.105:0/3578988672 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c098100 con 0x55b11a6ba2e0 2024-03-26T17:42:17.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d8a7fc640 1 -- 172.21.15.105:0/3578988672 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d8c005310 con 0x55b11a6ba2e0 2024-03-26T17:42:17.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.497+0000 7f0d8a7fc640 1 -- 172.21.15.105:0/3578988672 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c09b7c0 con 0x55b11a6ba2e0 2024-03-26T17:42:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 -- 172.21.15.105:0/3578988672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6ba2e0 msgr2=0x55b11a6cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6ba2e0 0x55b11a6cb070 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c098830 tx=0x7f0d8c0d5550 comp rx=0 tx=0).stop 2024-03-26T17:42:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 -- 172.21.15.105:0/3578988672 shutdown_connections 2024-03-26T17:42:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a6c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6ba2e0 0x55b11a6cb070 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 --2- 172.21.15.105:0/3578988672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6b98f0 0x55b11a6c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 -- 172.21.15.105:0/3578988672 >> 172.21.15.105:0/3578988672 conn(0x55b11a6b4e60 msgr2=0x55b11a6b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 -- 172.21.15.105:0/3578988672 shutdown_connections 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 -- 172.21.15.105:0/3578988672 wait complete. 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 -- start start 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 Processor -- start 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.498+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da99f7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a733740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da99f7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01aa110 con 0x55b11a6b98f0 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da99f7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01a9a40 con 0x55b11a6bf450 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da99f7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0da01a6c80 con 0x55b11a6ba2e0 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da99f7740 10 client.0 _ll_register_callbacks cb 0x7f0d8c0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59848/0 (socket says 172.21.15.105:59848) 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 -- 172.21.15.105:0/2946601257 learned_addr learned my addr 172.21.15.105:0/2946601257 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da52d4640 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a733740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 -- 172.21.15.105:0/2946601257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 msgr2=0x55b11a6c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 -- 172.21.15.105:0/2946601257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 msgr2=0x55b11a733740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a733740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 -- 172.21.15.105:0/2946601257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b11a733d00 con 0x55b11a6b98f0 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da4ad3640 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6c80d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55b11a6b5dd0 tx=0x7f0d940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77dbaaa60e0c33 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0d7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0d7ffff640 1 -- 172.21.15.105:0/2946601257 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9401ebb0 con 0x55b11a6b98f0 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0d7ffff640 1 -- 172.21.15.105:0/2946601257 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f0d9401ed50 con 0x55b11a6b98f0 2024-03-26T17:42:17.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.499+0000 7f0da99f7740 1 -- 172.21.15.105:0/2946601257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b11a626210 con 0x55b11a6b98f0 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0d7ffff640 1 -- 172.21.15.105:0/2946601257 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d9402e040 con 0x55b11a6b98f0 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0da99f7740 10 client.16848 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0da99f7740 1 -- 172.21.15.105:0/2946601257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b11a7350e0 con 0x55b11a6b98f0 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0d7ffff640 1 -- 172.21.15.105:0/2946601257 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b11a626210 con 0x55b11a6b98f0 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0d7ffff640 1 -- 172.21.15.105:0/2946601257 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0d940399e0 con 0x55b11a6b98f0 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0da99f7740 10 client.16848 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0da99f7740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0da99f7740 20 client.16848 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b11a735a70 0x55b11a737e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0da99f7740 4 client.16848 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0da99f7740 1 -- 172.21.15.105:0/2946601257 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b11a5a3540 con 0x55b11a735a70 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.500+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b11a735a70 0x55b11a737e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0d7ffff640 10 client.16848 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0d9ffff640 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b11a735a70 0x55b11a737e90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6465541ee25763 server_cookie=dfc8d81d76a90710 in_seq=0 out_seq=0 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0d7ffff640 1 -- 172.21.15.105:0/2946601257 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b11a5a3540 con 0x55b11a735a70 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0d7ffff640 10 client.16848 handle_command_reply: tid=0 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 client.16848 shutdown 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 -- 172.21.15.105:0/2946601257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 msgr2=0x55b11a6c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6c80d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55b11a6b5dd0 tx=0x7f0d940046c0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 -- 172.21.15.105:0/2946601257 shutdown_connections 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b11a735a70 0x55b11a737e90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b11a6bf450 0x55b11a733740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b11a6ba2e0 0x55b11a6c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0d7ffff640 0 client.16848 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 --2- 172.21.15.105:0/2946601257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b11a6b98f0 0x55b11a6c80d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.501+0000 7f0da99f7740 1 -- 172.21.15.105:0/2946601257 >> 172.21.15.105:0/2946601257 conn(0x55b11a6b4e60 msgr2=0x55b11a6b80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da99f7740 1 -- 172.21.15.105:0/2946601257 shutdown_connections 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da99f7740 1 -- 172.21.15.105:0/2946601257 wait complete. 2024-03-26T17:42:17.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da99f7740 20 client.16848 trim_cache size 0 max 16384 2024-03-26T17:42:17.500 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:42:17.500 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:42:17.500 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:42:17.500 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 -- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d900a2f20 msgr2=0x7f0d900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d900a2f20 0x7f0d900a53e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55b11a6ba6e0 tx=0x7f0d94005f90 comp rx=0 tx=0).stop 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 -- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da0102b70 msgr2=0x7f0da0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da0102b70 0x7f0da0198010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e9f0 tx=0x7f0d8c00edf0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 -- 172.21.15.105:0/3952449488 shutdown_connections 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d900a2f20 0x7f0d900a53e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0da0103530 0x7f0da019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0da0102b70 0x7f0da0198010 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 --2- 172.21.15.105:0/3952449488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0da00ffba0 0x7f0da0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 -- 172.21.15.105:0/3952449488 >> 172.21.15.105:0/3952449488 conn(0x7f0da00fb4b0 msgr2=0x7f0da0105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:17.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 -- 172.21.15.105:0/3952449488 shutdown_connections 2024-03-26T17:42:17.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.502+0000 7f0da6d37640 1 -- 172.21.15.105:0/3952449488 wait complete. 2024-03-26T17:42:17.507 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:42:17.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:42:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.633+0000 7fdb93990640 1 -- 172.21.15.105:0/1649694604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 msgr2=0x7fdb8c107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.633+0000 7fdb93990640 1 --2- 172.21.15.105:0/1649694604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 0x7fdb8c107120 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009900 tx=0x7fdb7c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.633+0000 7fdb93990640 1 -- 172.21.15.105:0/1649694604 shutdown_connections 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.633+0000 7fdb93990640 1 --2- 172.21.15.105:0/1649694604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8c107660 0x7fdb8c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.633+0000 7fdb93990640 1 --2- 172.21.15.105:0/1649694604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 0x7fdb8c107120 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.633+0000 7fdb93990640 1 --2- 172.21.15.105:0/1649694604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb8c102400 0x7fdb8c1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 -- 172.21.15.105:0/1649694604 >> 172.21.15.105:0/1649694604 conn(0x7fdb8c0fbd50 msgr2=0x7fdb8c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 -- 172.21.15.105:0/1649694604 shutdown_connections 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 -- 172.21.15.105:0/1649694604 wait complete. 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 Processor -- start 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 -- start start 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb8c102400 0x7fdb8c101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 0x7fdb8c1003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8c107660 0x7fdb8c1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c10d3e0 con 0x7fdb8c104d30 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c10d260 con 0x7fdb8c107660 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb93990640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c10d560 con 0x7fdb8c102400 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb90f2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 0x7fdb8c1003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.634+0000 7fdb90f2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 0x7fdb8c1003b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59862/0 (socket says 172.21.15.105:59862) 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb90f2b640 1 -- 172.21.15.105:0/2325738322 learned_addr learned my addr 172.21.15.105:0/2325738322 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb91f2d640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8c107660 0x7fdb8c1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb9172c640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb8c102400 0x7fdb8c101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb90f2b640 1 -- 172.21.15.105:0/2325738322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb8c102400 msgr2=0x7fdb8c101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb90f2b640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb8c102400 0x7fdb8c101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb90f2b640 1 -- 172.21.15.105:0/2325738322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8c107660 msgr2=0x7fdb8c1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb90f2b640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8c107660 0x7fdb8c1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb90f2b640 1 -- 172.21.15.105:0/2325738322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8c101180 con 0x7fdb8c104d30 2024-03-26T17:42:17.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb90f2b640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 0x7fdb8c1003b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000c00 tx=0x7fdb7c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb827fc640 1 -- 172.21.15.105:0/2325738322 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c043020 con 0x7fdb8c104d30 2024-03-26T17:42:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb827fc640 1 -- 172.21.15.105:0/2325738322 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb7c004350 con 0x7fdb8c104d30 2024-03-26T17:42:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb827fc640 1 -- 172.21.15.105:0/2325738322 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c003b10 con 0x7fdb8c104d30 2024-03-26T17:42:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8c1a1c70 con 0x7fdb8c104d30 2024-03-26T17:42:17.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.635+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8c1a21c0 con 0x7fdb8c104d30 2024-03-26T17:42:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.636+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8c0fd840 con 0x7fdb8c104d30 2024-03-26T17:42:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.638+0000 7fdb827fc640 1 -- 172.21.15.105:0/2325738322 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdb7c046020 con 0x7fdb8c104d30 2024-03-26T17:42:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.638+0000 7fdb827fc640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb780a2fa0 0x7fdb780a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.638+0000 7fdb827fc640 1 -- 172.21.15.105:0/2325738322 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdb7c0f1860 con 0x7fdb8c104d30 2024-03-26T17:42:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.639+0000 7fdb9172c640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb780a2fa0 0x7fdb780a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:17.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.639+0000 7fdb9172c640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb780a2fa0 0x7fdb780a5460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009bd0 tx=0x7fdb88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:17.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.643+0000 7fdb827fc640 1 -- 172.21.15.105:0/2325738322 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdb7c04d020 con 0x7fdb8c104d30 2024-03-26T17:42:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.900+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb8c02d240 con 0x7fdb8c104d30 2024-03-26T17:42:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.901+0000 7fdb827fc640 1 -- 172.21.15.105:0/2325738322 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fdb7c0be650 con 0x7fdb8c104d30 2024-03-26T17:42:17.893 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:42:17.894 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:42:17.894 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:42:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb780a2fa0 msgr2=0x7fdb780a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb780a2fa0 0x7fdb780a5460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009bd0 tx=0x7fdb88005f90 comp rx=0 tx=0).stop 2024-03-26T17:42:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 msgr2=0x7fdb8c1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:17.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 0x7fdb8c1003b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000c00 tx=0x7fdb7c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 shutdown_connections 2024-03-26T17:42:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb780a2fa0 0x7fdb780a5460 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb8c107660 0x7fdb8c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb8c104d30 0x7fdb8c1003b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 --2- 172.21.15.105:0/2325738322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb8c102400 0x7fdb8c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 >> 172.21.15.105:0/2325738322 conn(0x7fdb8c0fbd50 msgr2=0x7fdb8c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 shutdown_connections 2024-03-26T17:42:17.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:17.903+0000 7fdb93990640 1 -- 172.21.15.105:0/2325738322 wait complete. 2024-03-26T17:42:17.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:42:18.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 106930 3.14 MB/sec execute 61 sec latency 15.420 ms 2024-03-26T17:42:18.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 -- 172.21.15.105:0/3818014440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 msgr2=0x7f7de4105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 --2- 172.21.15.105:0/3818014440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 0x7f7de4105170 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009900 tx=0x7f7dd002bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 -- 172.21.15.105:0/3818014440 shutdown_connections 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 --2- 172.21.15.105:0/3818014440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de41056b0 0x7f7de410bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 --2- 172.21.15.105:0/3818014440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 0x7f7de4105170 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 --2- 172.21.15.105:0/3818014440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7de4103b00 0x7f7de4103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 -- 172.21.15.105:0/3818014440 >> 172.21.15.105:0/3818014440 conn(0x7f7de40ff070 msgr2=0x7f7de41014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 -- 172.21.15.105:0/3818014440 shutdown_connections 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 -- 172.21.15.105:0/3818014440 wait complete. 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.035+0000 7f7deaeea640 1 Processor -- start 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7deaeea640 1 -- start start 2024-03-26T17:42:18.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7deaeea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7de4103b00 0x7f7de4199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7deaeea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 0x7f7de419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7deaeea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de41056b0 0x7f7de41a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7deaeea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7de410f630 con 0x7f7de4104cf0 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7deaeea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7de410f4b0 con 0x7f7de4103b00 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7deaeea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7de410f7b0 con 0x7f7de41056b0 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 0x7f7de419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de8c86640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7de4103b00 0x7f7de4199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 0x7f7de419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59870/0 (socket says 172.21.15.105:59870) 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de8c86640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7de4103b00 0x7f7de4199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60436/0 (socket says 172.21.15.105:60436) 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 -- 172.21.15.105:0/159546275 learned_addr learned my addr 172.21.15.105:0/159546275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de9487640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de41056b0 0x7f7de41a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 -- 172.21.15.105:0/159546275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de41056b0 msgr2=0x7f7de41a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de41056b0 0x7f7de41a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 -- 172.21.15.105:0/159546275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7de4103b00 msgr2=0x7f7de4199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7de4103b00 0x7f7de4199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 -- 172.21.15.105:0/159546275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de41a1970 con 0x7f7de4104cf0 2024-03-26T17:42:18.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de3fff640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 0x7f7de419a290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0006010 tx=0x7f7dd0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.036+0000 7f7de1ffb640 1 -- 172.21.15.105:0/159546275 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0043020 con 0x7f7de4104cf0 2024-03-26T17:42:18.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.037+0000 7f7de1ffb640 1 -- 172.21.15.105:0/159546275 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd00043d0 con 0x7f7de4104cf0 2024-03-26T17:42:18.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.037+0000 7f7de1ffb640 1 -- 172.21.15.105:0/159546275 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0003b80 con 0x7f7de4104cf0 2024-03-26T17:42:18.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.037+0000 7f7deaeea640 1 -- 172.21.15.105:0/159546275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de41a1c00 con 0x7f7de4104cf0 2024-03-26T17:42:18.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.037+0000 7f7deaeea640 1 -- 172.21.15.105:0/159546275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de41a2150 con 0x7f7de4104cf0 2024-03-26T17:42:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.039+0000 7f7de1ffb640 1 -- 172.21.15.105:0/159546275 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7dd0046020 con 0x7f7de4104cf0 2024-03-26T17:42:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.040+0000 7f7de1ffb640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7dd40a2f20 0x7f7dd40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.040+0000 7f7dedbb1740 1 Processor -- start 2024-03-26T17:42:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7dedbb1740 1 -- start start 2024-03-26T17:42:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7de1ffb640 1 -- 172.21.15.105:0/159546275 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7dd00f1600 con 0x7f7de4104cf0 2024-03-26T17:42:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c0a30 0x55ef337c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7de8c86640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7dd40a2f20 0x7f7dd40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c6590 0x55ef337c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef33725970 con 0x55ef337c6590 2024-03-26T17:42:18.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef336cdd70 con 0x55ef337c1420 2024-03-26T17:42:18.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef33721920 con 0x55ef337c0a30 2024-03-26T17:42:18.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7de8c86640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c0a30 0x55ef337c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60452/0 (socket says 172.21.15.105:60452) 2024-03-26T17:42:18.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7de8c86640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c0a30 0x55ef337c0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43230/0 (socket says 172.21.15.105:43230) 2024-03-26T17:42:18.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7de3fff640 1 -- 172.21.15.105:0/964322151 learned_addr learned my addr 172.21.15.105:0/964322151 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.041+0000 7f7de9487640 1 --2- 172.21.15.105:0/964322151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c6590 0x55ef337c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7de3fff640 1 -- 172.21.15.105:0/964322151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c0a30 msgr2=0x55ef337c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7de3fff640 1 --2- 172.21.15.105:0/964322151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c0a30 0x55ef337c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7de3fff640 1 -- 172.21.15.105:0/964322151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c6590 msgr2=0x55ef337c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7de3fff640 1 --2- 172.21.15.105:0/964322151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c6590 0x55ef337c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7de8c86640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7dd40a2f20 0x7f7dd40a53e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7de4104b50 tx=0x7f7dd8011920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7de3fff640 1 -- 172.21.15.105:0/964322151 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef334393f0 con 0x55ef337c1420 2024-03-26T17:42:18.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7de3fff640 1 --2- 172.21.15.105:0/964322151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c5f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00c61d0 tx=0x7f7dd00bd710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbe4fb43fcc73b94 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7dceffd640 1 -- 172.21.15.105:0/964322151 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00c1e40 con 0x55ef337c1420 2024-03-26T17:42:18.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7dceffd640 1 -- 172.21.15.105:0/964322151 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd00c6700 con 0x55ef337c1420 2024-03-26T17:42:18.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.042+0000 7f7dceffd640 1 -- 172.21.15.105:0/964322151 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00c6270 con 0x55ef337c1420 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 -- 172.21.15.105:0/964322151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 msgr2=0x55ef337c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/964322151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c5f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00c61d0 tx=0x7f7dd00bd710 comp rx=0 tx=0).stop 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 -- 172.21.15.105:0/964322151 shutdown_connections 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/964322151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c6590 0x55ef337c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/964322151 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c5f20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/964322151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c0a30 0x55ef337c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 -- 172.21.15.105:0/964322151 >> 172.21.15.105:0/964322151 conn(0x55ef337bbfa0 msgr2=0x55ef337bc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 -- 172.21.15.105:0/964322151 shutdown_connections 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 -- 172.21.15.105:0/964322151 wait complete. 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 -- start start 2024-03-26T17:42:18.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.043+0000 7f7dedbb1740 1 Processor -- start 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef337c9300 con 0x55ef337c1420 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef336e1310 con 0x55ef337c0a30 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef337c9480 con 0x55ef337c6590 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7dedbb1740 10 client.0 _ll_register_callbacks cb 0x55ef3371cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de8c86640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de9487640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59874/0 (socket says 172.21.15.105:59874) 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de3fff640 1 -- 172.21.15.105:0/2303241828 learned_addr learned my addr 172.21.15.105:0/2303241828 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de8c86640 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60462/0 (socket says 172.21.15.105:60462) 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de3fff640 1 -- 172.21.15.105:0/2303241828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 msgr2=0x55ef337d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de3fff640 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de3fff640 1 -- 172.21.15.105:0/2303241828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 msgr2=0x55ef337cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de3fff640 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de3fff640 1 -- 172.21.15.105:0/2303241828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef335037d0 con 0x55ef337c1420 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de9487640 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.044+0000 7f7de8c86640 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7de3fff640 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cb220 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00ef110 tx=0x7f7dd00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fce8929421fb2d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2303241828 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00c6470 con 0x55ef337c1420 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2303241828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd00c6700 con 0x55ef337c1420 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2303241828 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7dd009cb60 con 0x55ef337c1420 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2303241828 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7dd00bc0b0 con 0x55ef337c1420 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 10 client.16866 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2303241828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef337c9300 con 0x55ef337c1420 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2303241828 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd00c6700 con 0x55ef337c1420 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2303241828 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7dd00b9a40 con 0x55ef337c1420 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 10 client.16866 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 10 client.16866 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 20 client.16866 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ef337d6a20 0x55ef337d8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 4 client.16866 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.045+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2303241828 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef33732eb0 con 0x55ef337d6a20 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.046+0000 7f7de8c86640 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ef337d6a20 0x55ef337d8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.046+0000 7f7de8c86640 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ef337d6a20 0x55ef337d8e40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=13477649a430232c server_cookie=dd162094bb151d57 in_seq=0 out_seq=0 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.046+0000 7f7dc3fff640 10 client.16866 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2303241828 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ef33732eb0 con 0x55ef337d6a20 2024-03-26T17:42:18.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dc3fff640 10 client.16866 handle_command_reply: tid=0 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 client.16866 shutdown 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2303241828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 msgr2=0x55ef337cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cb220 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00ef110 tx=0x7f7dd00035f0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2303241828 shutdown_connections 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ef337d6a20 0x55ef337d8e40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cb220 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2303241828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2303241828 >> 172.21.15.105:0/2303241828 conn(0x55ef337bbfa0 msgr2=0x55ef337bb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dc3fff640 0 client.16866 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2303241828 shutdown_connections 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2303241828 wait complete. 2024-03-26T17:42:18.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.047+0000 7f7dedbb1740 20 client.16866 trim_cache size 0 max 16384 2024-03-26T17:42:18.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7dedbb1740 1 Processor -- start 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7dedbb1740 1 -- start start 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c0a30 0x55ef337c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41aafd0 con 0x55ef337c0a30 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41acb10 con 0x55ef337c1420 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41ab6a0 con 0x55ef337c6590 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7de8c86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c0a30 0x55ef337c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7de8c86640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c0a30 0x55ef337c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59878/0 (socket says 172.21.15.105:59878) 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7de8c86640 1 -- 172.21.15.105:0/488249230 learned_addr learned my addr 172.21.15.105:0/488249230 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7de3fff640 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.068+0000 7f7de9487640 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7de8c86640 1 -- 172.21.15.105:0/488249230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 msgr2=0x55ef337d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7de8c86640 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7de8c86640 1 -- 172.21.15.105:0/488249230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 msgr2=0x55ef337c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7de8c86640 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7de8c86640 1 -- 172.21.15.105:0/488249230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef337c1800 con 0x55ef337c0a30 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7de8c86640 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c0a30 0x55ef337c97b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x55ef337c1c80 tx=0x7f7dd801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c4e06b51c25b42 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7dce7fc640 1 -- 172.21.15.105:0/488249230 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd801a040 con 0x55ef337c0a30 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7dce7fc640 1 -- 172.21.15.105:0/488249230 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd800fc70 con 0x55ef337c0a30 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.069+0000 7f7dce7fc640 1 -- 172.21.15.105:0/488249230 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8020d10 con 0x55ef337c0a30 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 -- 172.21.15.105:0/488249230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c0a30 msgr2=0x55ef337c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c0a30 0x55ef337c97b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x55ef337c1c80 tx=0x7f7dd801f000 comp rx=0 tx=0).stop 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 -- 172.21.15.105:0/488249230 shutdown_connections 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef337d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c1420 0x55ef337c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/488249230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c0a30 0x55ef337c97b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 -- 172.21.15.105:0/488249230 >> 172.21.15.105:0/488249230 conn(0x55ef337bbfa0 msgr2=0x55ef337cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 -- 172.21.15.105:0/488249230 shutdown_connections 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 -- 172.21.15.105:0/488249230 wait complete. 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 -- start start 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.070+0000 7f7dedbb1740 1 Processor -- start 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dedbb1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef3383ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41ac440 con 0x55ef337c1420 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41abd70 con 0x55ef337c0a30 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dedbb1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7de41a8fb0 con 0x55ef337c6590 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dedbb1740 10 client.0 _ll_register_callbacks cb 0x7f7ddc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de9487640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef3383ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de9487640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef3383ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43252/0 (socket says 172.21.15.105:43252) 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de9487640 1 -- 172.21.15.105:0/2506440635 learned_addr learned my addr 172.21.15.105:0/2506440635 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de8c86640 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de3fff640 1 -- 172.21.15.105:0/2506440635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 msgr2=0x55ef3383ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de3fff640 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef3383ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de3fff640 1 -- 172.21.15.105:0/2506440635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 msgr2=0x55ef337cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de3fff640 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de3fff640 1 -- 172.21.15.105:0/2506440635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3383b270 con 0x55ef337c1420 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de9487640 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef3383ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7de3fff640 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cf660 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00bdc80 tx=0x7f7dd002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5feb66cd2fcf852 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2506440635 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00bcc00 con 0x55ef337c1420 2024-03-26T17:42:18.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2506440635 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7dd00bce10 con 0x55ef337c1420 2024-03-26T17:42:18.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.071+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2506440635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd800fc70 con 0x55ef337c1420 2024-03-26T17:42:18.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2506440635 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7dd00c6700 con 0x55ef337c1420 2024-03-26T17:42:18.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dedbb1740 10 client.16872 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2506440635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ef3383b270 con 0x55ef337c1420 2024-03-26T17:42:18.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2506440635 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dd800fc70 con 0x55ef337c1420 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2506440635 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7dd00c6270 con 0x55ef337c1420 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dedbb1740 10 client.16872 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dedbb1740 10 client.16872 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dedbb1740 20 client.16872 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ef3383caf0 0x55ef3383ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dedbb1740 4 client.16872 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2506440635 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef3372d200 con 0x55ef3383caf0 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7de8c86640 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ef3383caf0 0x55ef3383ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.072+0000 7f7dc3fff640 10 client.16872 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7de8c86640 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ef3383caf0 0x55ef3383ef10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a283cffa6d89c027 server_cookie=baccf3285faed002 in_seq=0 out_seq=0 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dc3fff640 1 -- 172.21.15.105:0/2506440635 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef3372d200 con 0x55ef3383caf0 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dc3fff640 10 client.16872 handle_command_reply: tid=0 2024-03-26T17:42:18.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 client.16872 shutdown 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2506440635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 msgr2=0x55ef337cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cf660 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00bdc80 tx=0x7f7dd002bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2506440635 shutdown_connections 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ef3383caf0 0x55ef3383ef10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef337c6590 0x55ef3383ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef337c1420 0x55ef337cf660 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 --2- 172.21.15.105:0/2506440635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef337c0a30 0x55ef337cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2506440635 >> 172.21.15.105:0/2506440635 conn(0x55ef337bbfa0 msgr2=0x55ef337c29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dc3fff640 0 client.16872 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2506440635 shutdown_connections 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 1 -- 172.21.15.105:0/2506440635 wait complete. 2024-03-26T17:42:18.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.073+0000 7f7dedbb1740 20 client.16872 trim_cache size 0 max 16384 2024-03-26T17:42:18.072 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:18.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 -- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7dd40a2f20 msgr2=0x7f7dd40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7dd40a2f20 0x7f7dd40a53e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7de4104b50 tx=0x7f7dd8011920 comp rx=0 tx=0).stop 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 -- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 msgr2=0x7f7de419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 0x7f7de419a290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0006010 tx=0x7f7dd0005e40 comp rx=0 tx=0).stop 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 -- 172.21.15.105:0/159546275 shutdown_connections 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7dd40a2f20 0x7f7dd40a53e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de41056b0 0x7f7de41a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7de4104cf0 0x7f7de419a290 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 --2- 172.21.15.105:0/159546275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7de4103b00 0x7f7de4199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 -- 172.21.15.105:0/159546275 >> 172.21.15.105:0/159546275 conn(0x7f7de40ff070 msgr2=0x7f7de41080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 -- 172.21.15.105:0/159546275 shutdown_connections 2024-03-26T17:42:18.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.074+0000 7f7deaeea640 1 -- 172.21.15.105:0/159546275 wait complete. 2024-03-26T17:42:18.077 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:17 smithi105 ceph-mon[32280]: pgmap v466: 97 pgs: 97 active+clean; 24 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:42:18.078 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:42:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:17 smithi144 ceph-mon[36095]: pgmap v466: 97 pgs: 97 active+clean; 24 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:42:18.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:17 smithi135 ceph-mon[35636]: pgmap v466: 97 pgs: 97 active+clean; 24 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:42:18.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1482187420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d81007c0 msgr2=0x7fa5d8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1482187420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d81007c0 0x7fa5d8100be0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009900 tx=0x7fa5c802bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:18.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1482187420 shutdown_connections 2024-03-26T17:42:18.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1482187420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5d8102370 0x7fa5d8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1482187420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5d81019b0 0x7fa5d8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1482187420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d81007c0 0x7fa5d8100be0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1482187420 >> 172.21.15.105:0/1482187420 conn(0x7fa5d80fbd30 msgr2=0x7fa5d80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1482187420 shutdown_connections 2024-03-26T17:42:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.198+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1482187420 wait complete. 2024-03-26T17:42:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.199+0000 7fa5ddcd1640 1 Processor -- start 2024-03-26T17:42:18.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.199+0000 7fa5ddcd1640 1 -- start start 2024-03-26T17:42:18.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5ddcd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5d81007c0 0x7fa5d8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5ddcd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5d81019b0 0x7fa5d8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5ddcd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d8102370 0x7fa5d819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5ddcd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810c2c0 con 0x7fa5d8102370 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5ddcd1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810c140 con 0x7fa5d81019b0 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5ddcd1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810c440 con 0x7fa5d81007c0 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5d81007c0 0x7fa5d8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5d81007c0 0x7fa5d8197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43278/0 (socket says 172.21.15.105:43278) 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5d81019b0 0x7fa5d8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 -- 172.21.15.105:0/1521374494 learned_addr learned my addr 172.21.15.105:0/1521374494 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d8102370 0x7fa5d819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 -- 172.21.15.105:0/1521374494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5d81019b0 msgr2=0x7fa5d8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5d81019b0 0x7fa5d8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 -- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d8102370 msgr2=0x7fa5d819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d8102370 0x7fa5d819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 -- 172.21.15.105:0/1521374494 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d819f750 con 0x7fa5d81007c0 2024-03-26T17:42:18.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.200+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5d81007c0 0x7fa5d8197b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8000c00 tx=0x7fa5c8005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.201+0000 7fa5d4ff9640 1 -- 172.21.15.105:0/1521374494 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8043020 con 0x7fa5d81007c0 2024-03-26T17:42:18.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.201+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1521374494 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d819f9e0 con 0x7fa5d81007c0 2024-03-26T17:42:18.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.201+0000 7fa5d4ff9640 1 -- 172.21.15.105:0/1521374494 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5c8004350 con 0x7fa5d81007c0 2024-03-26T17:42:18.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.201+0000 7fa5d4ff9640 1 -- 172.21.15.105:0/1521374494 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8003b80 con 0x7fa5d81007c0 2024-03-26T17:42:18.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.201+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1521374494 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d819ff80 con 0x7fa5d81007c0 2024-03-26T17:42:18.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5e099a740 1 Processor -- start 2024-03-26T17:42:18.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5e099a740 1 -- start start 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092bae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5e099a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55de0921f970 con 0x55de092bb420 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5e099a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55de091c7d70 con 0x55de092c0590 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5e099a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55de0921b920 con 0x55de092baa30 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.203+0000 7fa5d4ff9640 1 -- 172.21.15.105:0/1521374494 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa5c8046020 con 0x7fa5d81007c0 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092bae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092c2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60520/0 (socket says 172.21.15.105:60520) 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 -- 172.21.15.105:0/1524051656 learned_addr learned my addr 172.21.15.105:0/1524051656 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092bae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43292/0 (socket says 172.21.15.105:43292) 2024-03-26T17:42:18.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 -- 172.21.15.105:0/1524051656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 msgr2=0x55de092bae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092bae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 -- 172.21.15.105:0/1524051656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 msgr2=0x55de092bff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092bff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 -- 172.21.15.105:0/1524051656 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de08f333f0 con 0x55de092c0590 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092c2a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009c90 tx=0x7fa5cc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4baf0144ee3e012d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d4ff9640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c00a2f20 0x7fa5c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5bdffb640 1 -- 172.21.15.105:0/1524051656 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc01e070 con 0x55de092c0590 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c00a2f20 0x7fa5c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5bdffb640 1 -- 172.21.15.105:0/1524051656 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5cc019ae0 con 0x55de092c0590 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5d4ff9640 1 -- 172.21.15.105:0/1521374494 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa5c80f1580 con 0x7fa5d81007c0 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.204+0000 7fa5bdffb640 1 -- 172.21.15.105:0/1524051656 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00e9b0 con 0x55de092c0590 2024-03-26T17:42:18.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 -- 172.21.15.105:0/1524051656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 msgr2=0x55de092c2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c00a2f20 0x7fa5c00a53e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55de092bbc80 tx=0x7fa5d0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092c2a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009c90 tx=0x7fa5cc011810 comp rx=0 tx=0).stop 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 -- 172.21.15.105:0/1524051656 shutdown_connections 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092c2a20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 --2- 172.21.15.105:0/1524051656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 -- 172.21.15.105:0/1524051656 >> 172.21.15.105:0/1524051656 conn(0x55de092b5fa0 msgr2=0x55de092b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 -- 172.21.15.105:0/1524051656 shutdown_connections 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 -- 172.21.15.105:0/1524051656 wait complete. 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.205+0000 7fa5e099a740 1 -- start start 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5e099a740 1 Processor -- start 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092baa30 0x55de092c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092c5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092c0590 0x55de092cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5e099a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55de092c3540 con 0x55de092bb420 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5e099a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55de091db310 con 0x55de092baa30 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5e099a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55de092c36c0 con 0x55de092c0590 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5e099a740 10 client.0 _ll_register_callbacks cb 0x55de09216c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59932/0 (socket says 172.21.15.105:59932) 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092c0590 0x55de092cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d6ffd640 1 -- 172.21.15.105:0/3956223117 learned_addr learned my addr 172.21.15.105:0/3956223117 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092baa30 0x55de092c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d77fe640 1 -- 172.21.15.105:0/3956223117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092c0590 msgr2=0x55de092cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092c0590 0x55de092cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d77fe640 1 -- 172.21.15.105:0/3956223117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 msgr2=0x55de092c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d77fe640 1 -- 172.21.15.105:0/3956223117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de08ffd7d0 con 0x55de092baa30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.206+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092baa30 0x55de092c4f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80ef090 tx=0x7fa5c8005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c51d79140f7d9676 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/3956223117 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80bda40 con 0x55de092baa30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/3956223117 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa5c8003d20 con 0x55de092baa30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 1 -- 172.21.15.105:0/3956223117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5cc019ae0 con 0x55de092baa30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/3956223117 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5c80bd5d0 con 0x55de092baa30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 10 client.26726 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 1 -- 172.21.15.105:0/3956223117 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55de092cf840 con 0x55de092baa30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/3956223117 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5cc019ae0 con 0x55de092baa30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/3956223117 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa5c80c61f0 con 0x55de092baa30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 10 client.26726 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 10 client.26726 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:42:18.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 20 client.26726 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55de092d0f00 0x55de092d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 4 client.26726 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.207+0000 7fa5e099a740 1 -- 172.21.15.105:0/3956223117 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de0922ceb0 con 0x55de092d0f00 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.208+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55de092d0f00 0x55de092d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.208+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55de092d0f00 0x55de092d33a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40bcb4fb472b948 server_cookie=72977ad023a3ade9 in_seq=0 out_seq=0 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.208+0000 7fa5b6ffd640 10 client.26726 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/3956223117 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55de0922ceb0 con 0x55de092d0f00 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5b6ffd640 10 client.26726 handle_command_reply: tid=0 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 client.26726 shutdown 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 -- 172.21.15.105:0/3956223117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092baa30 msgr2=0x55de092c4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092baa30 0x55de092c4f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80ef090 tx=0x7fa5c8005b70 comp rx=0 tx=0).stop 2024-03-26T17:42:18.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 -- 172.21.15.105:0/3956223117 shutdown_connections 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55de092d0f00 0x55de092d33a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092c0590 0x55de092cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092bb420 0x55de092c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 --2- 172.21.15.105:0/3956223117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092baa30 0x55de092c4f50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5b6ffd640 0 client.26726 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 -- 172.21.15.105:0/3956223117 >> 172.21.15.105:0/3956223117 conn(0x55de092b5fa0 msgr2=0x55de092b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 -- 172.21.15.105:0/3956223117 shutdown_connections 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 1 -- 172.21.15.105:0/3956223117 wait complete. 2024-03-26T17:42:18.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.209+0000 7fa5e099a740 20 client.26726 trim_cache size 0 max 16384 2024-03-26T17:42:18.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5e099a740 1 Processor -- start 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5e099a740 1 -- start start 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092bb420 0x55de092cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092c0590 0x55de092b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5e099a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81a8dd0 con 0x55de092c0590 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5e099a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81aa910 con 0x55de092bb420 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5e099a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81a94a0 con 0x55de092baa30 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092c0590 0x55de092b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092c0590 0x55de092b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59938/0 (socket says 172.21.15.105:59938) 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 -- 172.21.15.105:0/1946776570 learned_addr learned my addr 172.21.15.105:0/1946776570 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092bb420 0x55de092cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 -- 172.21.15.105:0/1946776570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 msgr2=0x55de092c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 -- 172.21.15.105:0/1946776570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092bb420 msgr2=0x55de092cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092bb420 0x55de092cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 -- 172.21.15.105:0/1946776570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de092c3800 con 0x55de092c0590 2024-03-26T17:42:18.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092c0590 0x55de092b4930 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0003c90 tx=0x7fa5d001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5b9c97dfa06bff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5bd7fa640 1 -- 172.21.15.105:0/1946776570 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d000fe60 con 0x55de092c0590 2024-03-26T17:42:18.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5bd7fa640 1 -- 172.21.15.105:0/1946776570 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5d0003cc0 con 0x55de092c0590 2024-03-26T17:42:18.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.227+0000 7fa5bd7fa640 1 -- 172.21.15.105:0/1946776570 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0029df0 con 0x55de092c0590 2024-03-26T17:42:18.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 -- 172.21.15.105:0/1946776570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092c0590 msgr2=0x55de092b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092c0590 0x55de092b4930 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0003c90 tx=0x7fa5d001a000 comp rx=0 tx=0).stop 2024-03-26T17:42:18.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 -- 172.21.15.105:0/1946776570 shutdown_connections 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092c0590 0x55de092b4930 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092bb420 0x55de092cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 --2- 172.21.15.105:0/1946776570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092baa30 0x55de092c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 -- 172.21.15.105:0/1946776570 >> 172.21.15.105:0/1946776570 conn(0x55de092b5fa0 msgr2=0x55de092c4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 -- 172.21.15.105:0/1946776570 shutdown_connections 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 -- 172.21.15.105:0/1946776570 wait complete. 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.228+0000 7fa5e099a740 1 -- start start 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5e099a740 1 Processor -- start 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092baa30 0x55de092c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092bb420 0x55de092ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5e099a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092cb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5e099a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81aa240 con 0x55de092baa30 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5e099a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81a9b70 con 0x55de092c0590 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092baa30 0x55de092c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5e099a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81a6db0 con 0x55de092bb420 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092baa30 0x55de092c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59954/0 (socket says 172.21.15.105:59954) 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d77fe640 1 -- 172.21.15.105:0/273491501 learned_addr learned my addr 172.21.15.105:0/273491501 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092bb420 0x55de092ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5e099a740 10 client.0 _ll_register_callbacks cb 0x7fa5c80057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d7fff640 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d77fe640 1 -- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092bb420 msgr2=0x55de092ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092bb420 0x55de092ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d77fe640 1 -- 172.21.15.105:0/273491501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 msgr2=0x55de092cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d77fe640 1 -- 172.21.15.105:0/273491501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de092cb620 con 0x55de092baa30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.229+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092bb420 0x55de092ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5d77fe640 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092baa30 0x55de092c9ae0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8005ab0 tx=0x7fa5c80be370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e93949fdbb94474 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/273491501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80bd7b0 con 0x55de092baa30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/273491501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa5c8003d20 con 0x55de092baa30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 1 -- 172.21.15.105:0/273491501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5d0003cc0 con 0x55de092baa30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/273491501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa5c80c6680 con 0x55de092baa30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 10 client.16890 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 1 -- 172.21.15.105:0/273491501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55de093372a0 con 0x55de092baa30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/273491501 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5d0003cc0 con 0x55de092baa30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/273491501 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa5c809cae0 con 0x55de092baa30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 10 client.16890 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 10 client.16890 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 20 client.16890 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55de093379a0 0x55de09339dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 4 client.16890 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5e099a740 1 -- 172.21.15.105:0/273491501 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de09227200 con 0x55de093379a0 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.230+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55de093379a0 0x55de09339dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5b6ffd640 10 client.16890 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5d6ffd640 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55de093379a0 0x55de09339dc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d91982654f1d1105 server_cookie=7bfc15d8d60e2326 in_seq=0 out_seq=0 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5b6ffd640 1 -- 172.21.15.105:0/273491501 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de09227200 con 0x55de093379a0 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5b6ffd640 10 client.16890 handle_command_reply: tid=0 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 client.16890 shutdown 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 -- 172.21.15.105:0/273491501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092baa30 msgr2=0x55de092c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092baa30 0x55de092c9ae0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8005ab0 tx=0x7fa5c80be370 comp rx=0 tx=0).stop 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 -- 172.21.15.105:0/273491501 shutdown_connections 2024-03-26T17:42:18.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55de093379a0 0x55de09339dc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de092c0590 0x55de092cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55de092bb420 0x55de092ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5b6ffd640 0 client.16890 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 --2- 172.21.15.105:0/273491501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55de092baa30 0x55de092c9ae0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 -- 172.21.15.105:0/273491501 >> 172.21.15.105:0/273491501 conn(0x55de092b5fa0 msgr2=0x55de092b9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 -- 172.21.15.105:0/273491501 shutdown_connections 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 1 -- 172.21.15.105:0/273491501 wait complete. 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.231+0000 7fa5e099a740 20 client.16890 trim_cache size 0 max 16384 2024-03-26T17:42:18.230 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:18.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c00a2f20 msgr2=0x7fa5c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c00a2f20 0x7fa5c00a53e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55de092bbc80 tx=0x7fa5d0009500 comp rx=0 tx=0).stop 2024-03-26T17:42:18.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1521374494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5d81007c0 msgr2=0x7fa5d8197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5d81007c0 0x7fa5d8197b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8000c00 tx=0x7fa5c8005dc0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1521374494 shutdown_connections 2024-03-26T17:42:18.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa5c00a2f20 0x7fa5c00a53e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5d8102370 0x7fa5d819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa5d81019b0 0x7fa5d8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 --2- 172.21.15.105:0/1521374494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5d81007c0 0x7fa5d8197b30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1521374494 >> 172.21.15.105:0/1521374494 conn(0x7fa5d80fbd30 msgr2=0x7fa5d8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1521374494 shutdown_connections 2024-03-26T17:42:18.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.232+0000 7fa5ddcd1640 1 -- 172.21.15.105:0/1521374494 wait complete. 2024-03-26T17:42:18.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:42:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 -- 172.21.15.105:0/3548972979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 msgr2=0x7ff014108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 --2- 172.21.15.105:0/3548972979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 0x7ff014108c00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009ca0 tx=0x7ff00c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 -- 172.21.15.105:0/3548972979 shutdown_connections 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 --2- 172.21.15.105:0/3548972979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 0x7ff014108c00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 --2- 172.21.15.105:0/3548972979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0141019b0 0x7ff014101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 --2- 172.21.15.105:0/3548972979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141007c0 0x7ff014100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 -- 172.21.15.105:0/3548972979 >> 172.21.15.105:0/3548972979 conn(0x7ff0140fbd50 msgr2=0x7ff0140fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 -- 172.21.15.105:0/3548972979 shutdown_connections 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 -- 172.21.15.105:0/3548972979 wait complete. 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.378+0000 7ff0198f8640 1 Processor -- start 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0198f8640 1 -- start start 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0198f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0141007c0 0x7ff014197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0198f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141019b0 0x7ff014198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0198f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 0x7ff01419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0198f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff01410d410 con 0x7ff014102370 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0198f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff01410d290 con 0x7ff0141007c0 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0198f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff01410d590 con 0x7ff0141019b0 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0141007c0 0x7ff014197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 0x7ff01419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0141007c0 0x7ff014197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60550/0 (socket says 172.21.15.105:60550) 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff012ffd640 1 -- 172.21.15.105:0/1985739601 learned_addr learned my addr 172.21.15.105:0/1985739601 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141019b0 0x7ff014198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 0x7ff01419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59962/0 (socket says 172.21.15.105:59962) 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff012ffd640 1 -- 172.21.15.105:0/1985739601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141019b0 msgr2=0x7ff014198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff012ffd640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141019b0 0x7ff014198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff012ffd640 1 -- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 msgr2=0x7ff01419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff012ffd640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 0x7ff01419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff012ffd640 1 -- 172.21.15.105:0/1985739601 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01419f710 con 0x7ff0141007c0 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.379+0000 7ff0127fc640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141019b0 0x7ff014198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.380+0000 7ff012ffd640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0141007c0 0x7ff014197af0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff00400e9c0 tx=0x7ff00400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.380+0000 7ff0188f6640 1 -- 172.21.15.105:0/1985739601 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00400cd80 con 0x7ff0141007c0 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.380+0000 7ff0198f8640 1 -- 172.21.15.105:0/1985739601 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01419fa00 con 0x7ff0141007c0 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.380+0000 7ff0188f6640 1 -- 172.21.15.105:0/1985739601 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0040035f0 con 0x7ff0141007c0 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.380+0000 7ff0188f6640 1 -- 172.21.15.105:0/1985739601 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004010690 con 0x7ff0141007c0 2024-03-26T17:42:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.380+0000 7ff0198f8640 1 -- 172.21.15.105:0/1985739601 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01419ff50 con 0x7ff0141007c0 2024-03-26T17:42:18.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.382+0000 7ff0188f6640 1 -- 172.21.15.105:0/1985739601 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff0040037a0 con 0x7ff0141007c0 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff0188f6640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feffc0a2f20 0x7feffc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff0188f6640 1 -- 172.21.15.105:0/1985739601 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff0040cf930 con 0x7ff0141007c0 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff0127fc640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feffc0a2f20 0x7feffc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff01c5bf740 1 Processor -- start 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff01c5bf740 1 -- start start 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb81c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb89290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55727eadd970 con 0x55727eb81830 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55727ea85d70 con 0x55727eb82220 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55727ead9920 con 0x55727eb86e20 2024-03-26T17:42:18.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.383+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb81c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb89290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0127fc640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feffc0a2f20 0x7feffc0a53e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff014199000 tx=0x7ff008005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0127fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb89290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43352/0 (socket says 172.21.15.105:43352) 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 -- 172.21.15.105:0/3328314426 learned_addr learned my addr 172.21.15.105:0/3328314426 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb81c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59970/0 (socket says 172.21.15.105:59970) 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 -- 172.21.15.105:0/3328314426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 msgr2=0x55727eb867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 --2- 172.21.15.105:0/3328314426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb867b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 -- 172.21.15.105:0/3328314426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 msgr2=0x55727eb81c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 --2- 172.21.15.105:0/3328314426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb81c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 -- 172.21.15.105:0/3328314426 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727e7f13f0 con 0x55727eb86e20 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff012ffd640 1 --2- 172.21.15.105:0/3328314426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb81c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7ff0137fe640 1 --2- 172.21.15.105:0/3328314426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb89290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005e30 tx=0x7ff00c009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd064a8e1c04625 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7feff9ffb640 1 -- 172.21.15.105:0/3328314426 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c043070 con 0x55727eb86e20 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7feff9ffb640 1 -- 172.21.15.105:0/3328314426 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff00c032db0 con 0x55727eb86e20 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.384+0000 7feff9ffb640 1 -- 172.21.15.105:0/3328314426 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c039040 con 0x55727eb86e20 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 -- 172.21.15.105:0/3328314426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 msgr2=0x55727eb89290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/3328314426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb89290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005e30 tx=0x7ff00c009800 comp rx=0 tx=0).stop 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 -- 172.21.15.105:0/3328314426 shutdown_connections 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/3328314426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb89290 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/3328314426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/3328314426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb81c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 -- 172.21.15.105:0/3328314426 >> 172.21.15.105:0/3328314426 conn(0x55727eb73f20 msgr2=0x55727eb74320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 -- 172.21.15.105:0/3328314426 shutdown_connections 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 -- 172.21.15.105:0/3328314426 wait complete. 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.385+0000 7ff01c5bf740 1 -- start start 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff01c5bf740 1 Processor -- start 2024-03-26T17:42:18.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb8b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb92c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55727eb89d50 con 0x55727eb81830 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55727ea99310 con 0x55727eb82220 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55727eb89ed0 con 0x55727eb86e20 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff01c5bf740 10 client.0 _ll_register_callbacks cb 0x55727ead4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb8b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb8b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59972/0 (socket says 172.21.15.105:59972) 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 -- 172.21.15.105:0/2120814402 learned_addr learned my addr 172.21.15.105:0/2120814402 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff0127fc640 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff0137fe640 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb92c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 -- 172.21.15.105:0/2120814402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 msgr2=0x55727eb92c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb92c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 -- 172.21.15.105:0/2120814402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 msgr2=0x55727eb8bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb8bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 -- 172.21.15.105:0/2120814402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727e8bb7d0 con 0x55727eb81830 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7ff012ffd640 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb8b690 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff004009fd0 tx=0x7ff004003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb1f115cfcfb97c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.386+0000 7feff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7feff2ffd640 1 -- 172.21.15.105:0/2120814402 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0040a0170 con 0x55727eb81830 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 1 -- 172.21.15.105:0/2120814402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff00c032db0 con 0x55727eb81830 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7feff2ffd640 1 -- 172.21.15.105:0/2120814402 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff0040bc0c0 con 0x55727eb81830 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7feff2ffd640 1 -- 172.21.15.105:0/2120814402 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0040a4a30 con 0x55727eb81830 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 10 client.16902 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 1 -- 172.21.15.105:0/2120814402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55727eb96040 con 0x55727eb81830 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7feff2ffd640 1 -- 172.21.15.105:0/2120814402 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff00c032db0 con 0x55727eb81830 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7feff2ffd640 1 -- 172.21.15.105:0/2120814402 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff00409aee0 con 0x55727eb81830 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 10 client.16902 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 10 client.16902 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 20 client.16902 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55727eb97690 0x55727eb99b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 4 client.16902 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.387+0000 7ff01c5bf740 1 -- 172.21.15.105:0/2120814402 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55727eaeaeb0 con 0x55727eb97690 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.388+0000 7ff0127fc640 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55727eb97690 0x55727eb99b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.388+0000 7ff0127fc640 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55727eb97690 0x55727eb99b00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d975269bcebd9b6 server_cookie=e5b3f6aafd428c2f in_seq=0 out_seq=0 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.388+0000 7feff2ffd640 10 client.16902 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7feff2ffd640 1 -- 172.21.15.105:0/2120814402 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55727eaeaeb0 con 0x55727eb97690 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7feff2ffd640 10 client.16902 handle_command_reply: tid=0 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 client.16902 shutdown 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 -- 172.21.15.105:0/2120814402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 msgr2=0x55727eb8b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb8b690 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff004009fd0 tx=0x7ff004003bd0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 -- 172.21.15.105:0/2120814402 shutdown_connections 2024-03-26T17:42:18.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb86e20 0x55727eb92c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55727eb97690 0x55727eb99b00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/2120814402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb81830 0x55727eb8b690 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7feff2ffd640 0 client.16902 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:42:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 -- 172.21.15.105:0/2120814402 >> 172.21.15.105:0/2120814402 conn(0x55727eb73f20 msgr2=0x55727eb80ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 -- 172.21.15.105:0/2120814402 shutdown_connections 2024-03-26T17:42:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 1 -- 172.21.15.105:0/2120814402 wait complete. 2024-03-26T17:42:18.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.389+0000 7ff01c5bf740 20 client.16902 trim_cache size 0 max 16384 2024-03-26T17:42:18.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff01c5bf740 1 Processor -- start 2024-03-26T17:42:18.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff01c5bf740 1 -- start start 2024-03-26T17:42:18.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb82220 0x55727eb92ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb86e20 0x55727eb8a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141a6b00 con 0x55727eb82220 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141a8640 con 0x55727eb86e20 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141a71d0 con 0x55727eb81830 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff0127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb82220 0x55727eb92ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff012ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff0127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb82220 0x55727eb92ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59986/0 (socket says 172.21.15.105:59986) 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff0127fc640 1 -- 172.21.15.105:0/584675524 learned_addr learned my addr 172.21.15.105:0/584675524 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb86e20 0x55727eb8a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff0127fc640 1 -- 172.21.15.105:0/584675524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 msgr2=0x55727eb8bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff0127fc640 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff0127fc640 1 -- 172.21.15.105:0/584675524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb86e20 msgr2=0x55727eb8a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.403+0000 7ff0127fc640 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb86e20 0x55727eb8a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.404+0000 7ff0127fc640 1 -- 172.21.15.105:0/584675524 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55727eb74970 con 0x55727eb82220 2024-03-26T17:42:18.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.404+0000 7ff012ffd640 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.404+0000 7ff0137fe640 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb86e20 0x55727eb8a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.404+0000 7ff0127fc640 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb82220 0x55727eb92ba0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff008009e80 tx=0x7ff008014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62739d8a9f0b16c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.404+0000 7feff97fa640 1 -- 172.21.15.105:0/584675524 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008031070 con 0x55727eb82220 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.404+0000 7feff97fa640 1 -- 172.21.15.105:0/584675524 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff00802c810 con 0x55727eb82220 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.404+0000 7feff97fa640 1 -- 172.21.15.105:0/584675524 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00800fc80 con 0x55727eb82220 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 -- 172.21.15.105:0/584675524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb82220 msgr2=0x55727eb92ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb82220 0x55727eb92ba0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff008009e80 tx=0x7ff008014000 comp rx=0 tx=0).stop 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 -- 172.21.15.105:0/584675524 shutdown_connections 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb86e20 0x55727eb8a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb82220 0x55727eb92ba0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/584675524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 -- 172.21.15.105:0/584675524 >> 172.21.15.105:0/584675524 conn(0x55727eb73f20 msgr2=0x55727eb8a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 -- 172.21.15.105:0/584675524 shutdown_connections 2024-03-26T17:42:18.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 -- 172.21.15.105:0/584675524 wait complete. 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 -- start start 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.405+0000 7ff01c5bf740 1 Processor -- start 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb90450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff01c5bf740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb86e20 0x55727eb91410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141a7f70 con 0x55727eb86e20 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141a78a0 con 0x55727eb82220 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff01c5bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141a4ae0 con 0x55727eb81830 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff01c5bf740 10 client.0 _ll_register_callbacks cb 0x7ff004003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb86e20 0x55727eb91410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb86e20 0x55727eb91410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59994/0 (socket says 172.21.15.105:59994) 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0127fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb90450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0137fe640 1 -- 172.21.15.105:0/339169178 learned_addr learned my addr 172.21.15.105:0/339169178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff012ffd640 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0127fc640 1 -- 172.21.15.105:0/339169178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 msgr2=0x55727eb8ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0127fc640 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0127fc640 1 -- 172.21.15.105:0/339169178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb86e20 msgr2=0x55727eb91410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0127fc640 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb86e20 0x55727eb91410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0127fc640 1 -- 172.21.15.105:0/339169178 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727eb91a00 con 0x55727eb82220 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0137fe640 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb86e20 0x55727eb91410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff0127fc640 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb90450 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55727eb93bd0 tx=0x7ff0080056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f371f461af13416 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7feff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7feff2ffd640 1 -- 172.21.15.105:0/339169178 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008031070 con 0x55727eb82220 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7feff2ffd640 1 -- 172.21.15.105:0/339169178 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff00802c160 con 0x55727eb82220 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.406+0000 7ff01c5bf740 1 -- 172.21.15.105:0/339169178 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff00802c810 con 0x55727eb82220 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7feff2ffd640 1 -- 172.21.15.105:0/339169178 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff00802c360 con 0x55727eb82220 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff01c5bf740 10 client.26750 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff01c5bf740 1 -- 172.21.15.105:0/339169178 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55727eb92cc0 con 0x55727eb82220 2024-03-26T17:42:18.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7feff2ffd640 1 -- 172.21.15.105:0/339169178 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff00802c810 con 0x55727eb82220 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7feff2ffd640 1 -- 172.21.15.105:0/339169178 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff00801a040 con 0x55727eb82220 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff01c5bf740 10 client.26750 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff01c5bf740 10 client.26750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff01c5bf740 20 client.26750 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55727ebfdf60 0x55727ec00380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff01c5bf740 4 client.26750 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff01c5bf740 1 -- 172.21.15.105:0/339169178 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55727eae5200 con 0x55727ebfdf60 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.407+0000 7ff012ffd640 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55727ebfdf60 0x55727ec00380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff012ffd640 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55727ebfdf60 0x55727ec00380 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30d8a6e1fc0698bb server_cookie=9506c8178c5c41b3 in_seq=0 out_seq=0 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7feff2ffd640 10 client.26750 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7feff2ffd640 1 -- 172.21.15.105:0/339169178 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55727eae5200 con 0x55727ebfdf60 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7feff2ffd640 10 client.26750 handle_command_reply: tid=0 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 client.26750 shutdown 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 -- 172.21.15.105:0/339169178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 msgr2=0x55727eb90450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb90450 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55727eb93bd0 tx=0x7ff0080056d0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 -- 172.21.15.105:0/339169178 shutdown_connections 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55727eb86e20 0x55727eb91410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55727ebfdf60 0x55727ec00380 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55727eb82220 0x55727eb90450 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7feff2ffd640 0 client.26750 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 --2- 172.21.15.105:0/339169178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55727eb81830 0x55727eb8ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 -- 172.21.15.105:0/339169178 >> 172.21.15.105:0/339169178 conn(0x55727eb73f20 msgr2=0x55727eb8aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 -- 172.21.15.105:0/339169178 shutdown_connections 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 1 -- 172.21.15.105:0/339169178 wait complete. 2024-03-26T17:42:18.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.408+0000 7ff01c5bf740 20 client.26750 trim_cache size 0 max 16384 2024-03-26T17:42:18.410 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:18.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 -- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feffc0a2f20 msgr2=0x7feffc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feffc0a2f20 0x7feffc0a53e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff014199000 tx=0x7ff008005f90 comp rx=0 tx=0).stop 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 -- 172.21.15.105:0/1985739601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0141007c0 msgr2=0x7ff014197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0141007c0 0x7ff014197af0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff00400e9c0 tx=0x7ff00400ee80 comp rx=0 tx=0).stop 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 -- 172.21.15.105:0/1985739601 shutdown_connections 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feffc0a2f20 0x7feffc0a53e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff014102370 0x7ff01419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0141019b0 0x7ff014198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 --2- 172.21.15.105:0/1985739601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0141007c0 0x7ff014197af0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 -- 172.21.15.105:0/1985739601 >> 172.21.15.105:0/1985739601 conn(0x7ff0140fbd50 msgr2=0x7ff014104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 -- 172.21.15.105:0/1985739601 shutdown_connections 2024-03-26T17:42:18.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.409+0000 7ff0198f8640 1 -- 172.21.15.105:0/1985739601 wait complete. 2024-03-26T17:42:18.412 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:42:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 -- 172.21.15.105:0/1343873728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf41007c0 msgr2=0x7feaf4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 --2- 172.21.15.105:0/1343873728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf41007c0 0x7feaf4100be0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009900 tx=0x7feaf002bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 -- 172.21.15.105:0/1343873728 shutdown_connections 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 --2- 172.21.15.105:0/1343873728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaf4102370 0x7feaf4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 --2- 172.21.15.105:0/1343873728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf41019b0 0x7feaf4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 --2- 172.21.15.105:0/1343873728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf41007c0 0x7feaf4100be0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 -- 172.21.15.105:0/1343873728 >> 172.21.15.105:0/1343873728 conn(0x7feaf40fbd50 msgr2=0x7feaf40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 -- 172.21.15.105:0/1343873728 shutdown_connections 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.534+0000 7feafbb7e640 1 -- 172.21.15.105:0/1343873728 wait complete. 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafbb7e640 1 Processor -- start 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafbb7e640 1 -- start start 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafbb7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaf41007c0 0x7feaf4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafbb7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf41019b0 0x7feaf41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafbb7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf4102370 0x7feaf419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafbb7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feaf410c2e0 con 0x7feaf4102370 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafbb7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feaf410c160 con 0x7feaf41007c0 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafbb7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feaf410c460 con 0x7feaf41019b0 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafa11b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf4102370 0x7feaf419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feaf991a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaf41007c0 0x7feaf4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafa11b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf4102370 0x7feaf419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60012/0 (socket says 172.21.15.105:60012) 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feafa11b640 1 -- 172.21.15.105:0/3032271088 learned_addr learned my addr 172.21.15.105:0/3032271088 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.535+0000 7feaf9119640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf41019b0 0x7feaf41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaf991a640 1 -- 172.21.15.105:0/3032271088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf41019b0 msgr2=0x7feaf41980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaf991a640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf41019b0 0x7feaf41980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaf991a640 1 -- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf4102370 msgr2=0x7feaf419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaf991a640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf4102370 0x7feaf419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaf991a640 1 -- 172.21.15.105:0/3032271088 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf419f7b0 con 0x7feaf41007c0 2024-03-26T17:42:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feafa11b640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf4102370 0x7feaf419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaf991a640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaf41007c0 0x7feaf4197b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feaf0036040 tx=0x7feaf0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaeaffd640 1 -- 172.21.15.105:0/3032271088 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0043020 con 0x7feaf41007c0 2024-03-26T17:42:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaeaffd640 1 -- 172.21.15.105:0/3032271088 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf0004350 con 0x7feaf41007c0 2024-03-26T17:42:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feafbb7e640 1 -- 172.21.15.105:0/3032271088 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf419fa40 con 0x7feaf41007c0 2024-03-26T17:42:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feaeaffd640 1 -- 172.21.15.105:0/3032271088 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0003ab0 con 0x7feaf41007c0 2024-03-26T17:42:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.536+0000 7feafbb7e640 1 -- 172.21.15.105:0/3032271088 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf419ff60 con 0x7feaf41007c0 2024-03-26T17:42:18.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feaeaffd640 1 -- 172.21.15.105:0/3032271088 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7feaf0046020 con 0x7feaf41007c0 2024-03-26T17:42:18.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafe76f740 1 Processor -- start 2024-03-26T17:42:18.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafe76f740 1 -- start start 2024-03-26T17:42:18.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafe76f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a61e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafe76f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a66f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafe76f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafe76f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cfc99c6970 con 0x55cfc9a61a30 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafe76f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cfc996ed70 con 0x55cfc9a67590 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafe76f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cfc99c2920 con 0x55cfc9a62420 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feaf991a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a66f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feaf991a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a66f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43388/0 (socket says 172.21.15.105:43388) 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feaf991a640 1 -- 172.21.15.105:0/3401004623 learned_addr learned my addr 172.21.15.105:0/3401004623 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feafa11b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a61e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.539+0000 7feaf9119640 1 --2- 172.21.15.105:0/3401004623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feaeaffd640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feae00a2f20 0x7feae00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feaf991a640 1 -- 172.21.15.105:0/3401004623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 msgr2=0x55cfc9a69a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feaf991a640 1 --2- 172.21.15.105:0/3401004623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a69a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feaf991a640 1 -- 172.21.15.105:0/3401004623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 msgr2=0x55cfc9a61e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feaf991a640 1 --2- 172.21.15.105:0/3401004623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a61e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feaf991a640 1 -- 172.21.15.105:0/3401004623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc96da3f0 con 0x55cfc9a62420 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feafa11b640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feae00a2f20 0x7feae00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feaeaffd640 1 -- 172.21.15.105:0/3032271088 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feaf00f1500 con 0x7feaf41007c0 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feaf991a640 1 --2- 172.21.15.105:0/3401004623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a66f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feaf00e01e0 tx=0x7feaf00e88b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d93fe3547de428d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feadffff640 1 -- 172.21.15.105:0/3401004623 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00bd9c0 con 0x55cfc9a62420 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feadffff640 1 -- 172.21.15.105:0/3401004623 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf00a8030 con 0x55cfc9a62420 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.540+0000 7feadffff640 1 -- 172.21.15.105:0/3401004623 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00e0790 con 0x55cfc9a62420 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafa11b640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feae00a2f20 0x7feae00a53e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55cfc9a62c80 tx=0x7feaec004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 -- 172.21.15.105:0/3401004623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 msgr2=0x55cfc9a66f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 --2- 172.21.15.105:0/3401004623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a66f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feaf00e01e0 tx=0x7feaf00e88b0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 -- 172.21.15.105:0/3401004623 shutdown_connections 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 --2- 172.21.15.105:0/3401004623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a69a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 --2- 172.21.15.105:0/3401004623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a66f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 --2- 172.21.15.105:0/3401004623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a61e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 -- 172.21.15.105:0/3401004623 >> 172.21.15.105:0/3401004623 conn(0x55cfc9a5cfa0 msgr2=0x55cfc9a5d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 -- 172.21.15.105:0/3401004623 shutdown_connections 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 -- 172.21.15.105:0/3401004623 wait complete. 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.541+0000 7feafe76f740 1 -- start start 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafe76f740 1 Processor -- start 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafe76f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a61a30 0x55cfc9a6bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafe76f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a62420 0x55cfc9a6c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafe76f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafe76f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9a6a300 con 0x55cfc9a62420 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafe76f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9982310 con 0x55cfc9a61a30 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafe76f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9a6a480 con 0x55cfc9a67590 2024-03-26T17:42:18.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafe76f740 10 client.0 _ll_register_callbacks cb 0x55cfc99bdc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feaf9119640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a62420 0x55cfc9a6c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feaf991a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a61a30 0x55cfc9a6bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feaf9119640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a62420 0x55cfc9a6c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60026/0 (socket says 172.21.15.105:60026) 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feaf9119640 1 -- 172.21.15.105:0/939528114 learned_addr learned my addr 172.21.15.105:0/939528114 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feafa11b640 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feaf9119640 1 -- 172.21.15.105:0/939528114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 msgr2=0x55cfc9a73110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feaf9119640 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a73110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feaf9119640 1 -- 172.21.15.105:0/939528114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a61a30 msgr2=0x55cfc9a6bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.542+0000 7feaf9119640 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a61a30 0x55cfc9a6bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feaf9119640 1 -- 172.21.15.105:0/939528114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc97a47d0 con 0x55cfc9a62420 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feaf9119640 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a62420 0x55cfc9a6c0d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feae40126e0 tx=0x7feae4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad9a1c1e0b1d4ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feadd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:18.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feadd7fa640 1 -- 172.21.15.105:0/939528114 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae400f8c0 con 0x55cfc9a62420 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feadd7fa640 1 -- 172.21.15.105:0/939528114 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feae400fa60 con 0x55cfc9a62420 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feafe76f740 1 -- 172.21.15.105:0/939528114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf00a8030 con 0x55cfc9a62420 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feadd7fa640 1 -- 172.21.15.105:0/939528114 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feae400fc60 con 0x55cfc9a62420 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feafe76f740 10 client.16920 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feafe76f740 1 -- 172.21.15.105:0/939528114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfc9a76550 con 0x55cfc9a62420 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.543+0000 7feadd7fa640 1 -- 172.21.15.105:0/939528114 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf00a8030 con 0x55cfc9a62420 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feadd7fa640 1 -- 172.21.15.105:0/939528114 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7feae4026420 con 0x55cfc9a62420 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feafe76f740 10 client.16920 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feafe76f740 10 client.16920 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feafe76f740 20 client.16920 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feafe76f740 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55cfc9a77b90 0x55cfc9a7a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feafe76f740 4 client.16920 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feafe76f740 1 -- 172.21.15.105:0/939528114 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfc99d3eb0 con 0x55cfc9a77b90 2024-03-26T17:42:18.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feaf991a640 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55cfc9a77b90 0x55cfc9a7a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feadd7fa640 10 client.16920 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.544+0000 7feaf991a640 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55cfc9a77b90 0x55cfc9a7a050 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fde440d2b12a8c8f server_cookie=5cf0af648754867b in_seq=0 out_seq=0 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.545+0000 7feadd7fa640 1 -- 172.21.15.105:0/939528114 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cfc99d3eb0 con 0x55cfc9a77b90 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.545+0000 7feadd7fa640 10 client.16920 handle_command_reply: tid=0 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.545+0000 7feafe76f740 1 client.16920 shutdown 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.545+0000 7feafe76f740 1 -- 172.21.15.105:0/939528114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a62420 msgr2=0x55cfc9a6c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.545+0000 7feafe76f740 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a62420 0x55cfc9a6c0d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feae40126e0 tx=0x7feae4012ba0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.545+0000 7feafe76f740 1 -- 172.21.15.105:0/939528114 shutdown_connections 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feafe76f740 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55cfc9a77b90 0x55cfc9a7a050 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feafe76f740 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feafe76f740 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a62420 0x55cfc9a6c0d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feafe76f740 1 --2- 172.21.15.105:0/939528114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a61a30 0x55cfc9a6bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feadd7fa640 0 client.16920 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feafe76f740 1 -- 172.21.15.105:0/939528114 >> 172.21.15.105:0/939528114 conn(0x55cfc9a5cfa0 msgr2=0x55cfc9a5c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feafe76f740 1 -- 172.21.15.105:0/939528114 shutdown_connections 2024-03-26T17:42:18.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feafe76f740 1 -- 172.21.15.105:0/939528114 wait complete. 2024-03-26T17:42:18.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.546+0000 7feafe76f740 20 client.16920 trim_cache size 0 max 16384 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.560+0000 7feafe76f740 1 Processor -- start 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafe76f740 1 -- start start 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafe76f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a79970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafe76f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafe76f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a79eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafe76f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41a8de0 con 0x55cfc9a61a30 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafe76f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41aa920 con 0x55cfc9a67590 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafe76f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41a94b0 con 0x55cfc9a62420 2024-03-26T17:42:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a79970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafa11b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a79eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf9119640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a79970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60040/0 (socket says 172.21.15.105:60040) 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feafa11b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a79eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60634/0 (socket says 172.21.15.105:60634) 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 -- 172.21.15.105:0/1100016926 learned_addr learned my addr 172.21.15.105:0/1100016926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 -- 172.21.15.105:0/1100016926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 msgr2=0x55cfc9a730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 --2- 172.21.15.105:0/1100016926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 -- 172.21.15.105:0/1100016926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 msgr2=0x55cfc9a79eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 --2- 172.21.15.105:0/1100016926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a79eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 -- 172.21.15.105:0/1100016926 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc9a62800 con 0x55cfc9a61a30 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf9119640 1 --2- 172.21.15.105:0/1100016926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feaf991a640 1 --2- 172.21.15.105:0/1100016926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a79970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009900 tx=0x7feaf00f7060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24358cfb5e9fd9fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feadf7fe640 1 -- 172.21.15.105:0/1100016926 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00c5f60 con 0x55cfc9a61a30 2024-03-26T17:42:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feadf7fe640 1 -- 172.21.15.105:0/1100016926 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf00a8040 con 0x55cfc9a61a30 2024-03-26T17:42:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.561+0000 7feadf7fe640 1 -- 172.21.15.105:0/1100016926 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00bc1b0 con 0x55cfc9a61a30 2024-03-26T17:42:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 -- 172.21.15.105:0/1100016926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 msgr2=0x55cfc9a79970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 --2- 172.21.15.105:0/1100016926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a79970 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009900 tx=0x7feaf00f7060 comp rx=0 tx=0).stop 2024-03-26T17:42:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 -- 172.21.15.105:0/1100016926 shutdown_connections 2024-03-26T17:42:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 --2- 172.21.15.105:0/1100016926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a67590 0x55cfc9a79eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 --2- 172.21.15.105:0/1100016926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a62420 0x55cfc9a730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 --2- 172.21.15.105:0/1100016926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a79970 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 -- 172.21.15.105:0/1100016926 >> 172.21.15.105:0/1100016926 conn(0x55cfc9a5cfa0 msgr2=0x55cfc9a6bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 -- 172.21.15.105:0/1100016926 shutdown_connections 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.562+0000 7feafe76f740 1 -- 172.21.15.105:0/1100016926 wait complete. 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 1 -- start start 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 1 Processor -- start 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a70470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a62420 0x55cfc9a709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a71970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41aa250 con 0x55cfc9a61a30 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41a9b80 con 0x55cfc9a62420 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feaf41a6dc0 con 0x55cfc9a67590 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafe76f740 10 client.0 _ll_register_callbacks cb 0x7feae4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feaf991a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a70470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feaf991a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a70470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60046/0 (socket says 172.21.15.105:60046) 2024-03-26T17:42:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feaf991a640 1 -- 172.21.15.105:0/1516364208 learned_addr learned my addr 172.21.15.105:0/1516364208 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feaf9119640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a62420 0x55cfc9a709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.563+0000 7feafa11b640 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a71970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feaf991a640 1 -- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 msgr2=0x55cfc9a71970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feaf991a640 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a71970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feaf991a640 1 -- 172.21.15.105:0/1516364208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a62420 msgr2=0x55cfc9a709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feaf991a640 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a62420 0x55cfc9a709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feaf991a640 1 -- 172.21.15.105:0/1516364208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc9a71f60 con 0x55cfc9a61a30 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feafa11b640 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a71970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feaf9119640 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a62420 0x55cfc9a709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feadd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feaf991a640 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a70470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feaf0006010 tx=0x7feaf00bda50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ba6417311c130f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feadd7fa640 1 -- 172.21.15.105:0/1516364208 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00db060 con 0x55cfc9a61a30 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feafe76f740 1 -- 172.21.15.105:0/1516364208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf00a8040 con 0x55cfc9a61a30 2024-03-26T17:42:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feadd7fa640 1 -- 172.21.15.105:0/1516364208 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feaf00d1e80 con 0x55cfc9a61a30 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feadd7fa640 1 -- 172.21.15.105:0/1516364208 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feaf00bc3a0 con 0x55cfc9a61a30 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feafe76f740 10 client.16926 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.564+0000 7feafe76f740 1 -- 172.21.15.105:0/1516364208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfc9a73260 con 0x55cfc9a61a30 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feadd7fa640 1 -- 172.21.15.105:0/1516364208 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf00a8040 con 0x55cfc9a61a30 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feadd7fa640 1 -- 172.21.15.105:0/1516364208 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7feaf00ee090 con 0x55cfc9a61a30 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feafe76f740 10 client.16926 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feafe76f740 10 client.16926 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feafe76f740 20 client.16926 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feafe76f740 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55cfc9ade3c0 0x55cfc9ae07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feafe76f740 4 client.16926 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feafe76f740 1 -- 172.21.15.105:0/1516364208 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfc9a63430 con 0x55cfc9ade3c0 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feaf9119640 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55cfc9ade3c0 0x55cfc9ae07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feadd7fa640 10 client.16926 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.565+0000 7feaf9119640 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55cfc9ade3c0 0x55cfc9ae07e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d87e7a9b4d5657f server_cookie=965ced0704297ecd in_seq=0 out_seq=0 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feadd7fa640 1 -- 172.21.15.105:0/1516364208 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfc9a63430 con 0x55cfc9ade3c0 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feadd7fa640 10 client.16926 handle_command_reply: tid=0 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 client.16926 shutdown 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 -- 172.21.15.105:0/1516364208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 msgr2=0x55cfc9a70470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a70470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7feaf0006010 tx=0x7feaf00bda50 comp rx=0 tx=0).stop 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 -- 172.21.15.105:0/1516364208 shutdown_connections 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55cfc9ade3c0 0x55cfc9ae07e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cfc9a67590 0x55cfc9a71970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cfc9a62420 0x55cfc9a709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 --2- 172.21.15.105:0/1516364208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cfc9a61a30 0x55cfc9a70470 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feadd7fa640 0 client.16926 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:42:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 -- 172.21.15.105:0/1516364208 >> 172.21.15.105:0/1516364208 conn(0x55cfc9a5cfa0 msgr2=0x55cfc9a79970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 -- 172.21.15.105:0/1516364208 shutdown_connections 2024-03-26T17:42:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 1 -- 172.21.15.105:0/1516364208 wait complete. 2024-03-26T17:42:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafe76f740 20 client.16926 trim_cache size 0 max 16384 2024-03-26T17:42:18.565 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 -- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feae00a2f20 msgr2=0x7feae00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feae00a2f20 0x7feae00a53e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55cfc9a62c80 tx=0x7feaec004ab0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 -- 172.21.15.105:0/3032271088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaf41007c0 msgr2=0x7feaf4197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaf41007c0 0x7feaf4197b90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7feaf0036040 tx=0x7feaf0005dc0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 -- 172.21.15.105:0/3032271088 shutdown_connections 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feae00a2f20 0x7feae00a53e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feaf4102370 0x7feaf419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf41019b0 0x7feaf41980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 --2- 172.21.15.105:0/3032271088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feaf41007c0 0x7feaf4197b90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.566+0000 7feafbb7e640 1 -- 172.21.15.105:0/3032271088 >> 172.21.15.105:0/3032271088 conn(0x7feaf40fbd50 msgr2=0x7feaf4068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.567+0000 7feafbb7e640 1 -- 172.21.15.105:0/3032271088 shutdown_connections 2024-03-26T17:42:18.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.567+0000 7feafbb7e640 1 -- 172.21.15.105:0/3032271088 wait complete. 2024-03-26T17:42:18.571 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:42:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 -- 172.21.15.105:0/410782033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd81007c0 msgr2=0x7f9cd8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/410782033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd81007c0 0x7f9cd8100be0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009900 tx=0x7f9cc802bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 -- 172.21.15.105:0/410782033 shutdown_connections 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/410782033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cd8102370 0x7f9cd8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/410782033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd81019b0 0x7f9cd8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/410782033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd81007c0 0x7f9cd8100be0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 -- 172.21.15.105:0/410782033 >> 172.21.15.105:0/410782033 conn(0x7f9cd80fbd50 msgr2=0x7f9cd80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 -- 172.21.15.105:0/410782033 shutdown_connections 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.707+0000 7f9cdc93b640 1 -- 172.21.15.105:0/410782033 wait complete. 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cdc93b640 1 Processor -- start 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cdc93b640 1 -- start start 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cdc93b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cd81007c0 0x7f9cd8196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cdc93b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd81019b0 0x7f9cd8196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cdc93b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd8102370 0x7f9cd819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cdc93b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd810c2e0 con 0x7f9cd81019b0 2024-03-26T17:42:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cdc93b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd810c160 con 0x7f9cd81007c0 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cdc93b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd810c460 con 0x7f9cd8102370 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cd659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cd81007c0 0x7f9cd8196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9ccffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd81019b0 0x7f9cd8196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cd659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cd81007c0 0x7f9cd8196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60666/0 (socket says 172.21.15.105:60666) 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.708+0000 7f9cd659c640 1 -- 172.21.15.105:0/1377160762 learned_addr learned my addr 172.21.15.105:0/1377160762 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd8102370 0x7f9cd819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cd659c640 1 -- 172.21.15.105:0/1377160762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd8102370 msgr2=0x7f9cd819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cd659c640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd8102370 0x7f9cd819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cd659c640 1 -- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd81019b0 msgr2=0x7f9cd8196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cd659c640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd81019b0 0x7f9cd8196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cd659c640 1 -- 172.21.15.105:0/1377160762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd819e670 con 0x7f9cd81007c0 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cd659c640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cd81007c0 0x7f9cd8196a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8000c00 tx=0x7f9cc8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9ccf7fe640 1 -- 172.21.15.105:0/1377160762 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8043020 con 0x7f9cd81007c0 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cdc93b640 1 -- 172.21.15.105:0/1377160762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd819e900 con 0x7f9cd81007c0 2024-03-26T17:42:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9ccf7fe640 1 -- 172.21.15.105:0/1377160762 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cc8004350 con 0x7f9cd81007c0 2024-03-26T17:42:18.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9ccf7fe640 1 -- 172.21.15.105:0/1377160762 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8003b80 con 0x7f9cd81007c0 2024-03-26T17:42:18.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.709+0000 7f9cdc93b640 1 -- 172.21.15.105:0/1377160762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd819eea0 con 0x7f9cd81007c0 2024-03-26T17:42:18.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cdf608740 1 Processor -- start 2024-03-26T17:42:18.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cdf608740 1 -- start start 2024-03-26T17:42:18.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9ccf7fe640 1 -- 172.21.15.105:0/1377160762 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9cc8046020 con 0x7f9cd81007c0 2024-03-26T17:42:18.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94ba7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94bac7a0 0x55ac94baec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cdf608740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac94b09970 con 0x55ac94ba7630 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cdf608740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ac94ab1d70 con 0x55ac94bac7a0 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cdf608740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac94b05920 con 0x55ac94ba6c40 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cd659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9ccffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94bac7a0 0x55ac94baec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cd6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94ba7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cd6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94ba7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43444/0 (socket says 172.21.15.105:43444) 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9ccffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94bac7a0 0x55ac94baec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60670/0 (socket says 172.21.15.105:60670) 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cd6d9d640 1 -- 172.21.15.105:0/3463838364 learned_addr learned my addr 172.21.15.105:0/3463838364 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.712+0000 7f9cd659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60080/0 (socket says 172.21.15.105:60080) 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cd6d9d640 1 -- 172.21.15.105:0/3463838364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94bac7a0 msgr2=0x55ac94baec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/3463838364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94bac7a0 0x55ac94baec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cd6d9d640 1 -- 172.21.15.105:0/3463838364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 msgr2=0x55ac94bac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/3463838364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cd6d9d640 1 -- 172.21.15.105:0/3463838364 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9481d3f0 con 0x55ac94ba6c40 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cd659c640 1 --2- 172.21.15.105:0/3463838364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9ccf7fe640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9cc40a2fa0 0x7f9cc40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/3463838364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94ba7060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009fd0 tx=0x7f9cd00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c47deda0bd032bef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cd659c640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9cc40a2fa0 0x7f9cc40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9ccf7fe640 1 -- 172.21.15.105:0/1377160762 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9cc80f14b0 con 0x7f9cd81007c0 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cccff9640 1 -- 172.21.15.105:0/3463838364 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0004d20 con 0x55ac94ba6c40 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cccff9640 1 -- 172.21.15.105:0/3463838364 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd0004050 con 0x55ac94ba6c40 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.713+0000 7f9cccff9640 1 -- 172.21.15.105:0/3463838364 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0005280 con 0x55ac94ba6c40 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 -- 172.21.15.105:0/3463838364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 msgr2=0x55ac94ba7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 --2- 172.21.15.105:0/3463838364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94ba7060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009fd0 tx=0x7f9cd00118d0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cd659c640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9cc40a2fa0 0x7f9cc40a5460 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80ef2e0 tx=0x7f9cc80f4f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 -- 172.21.15.105:0/3463838364 shutdown_connections 2024-03-26T17:42:18.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 --2- 172.21.15.105:0/3463838364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94bac7a0 0x55ac94baec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 --2- 172.21.15.105:0/3463838364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 --2- 172.21.15.105:0/3463838364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94ba7060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 -- 172.21.15.105:0/3463838364 >> 172.21.15.105:0/3463838364 conn(0x55ac94b9ffa0 msgr2=0x55ac94ba03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 -- 172.21.15.105:0/3463838364 shutdown_connections 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 -- 172.21.15.105:0/3463838364 wait complete. 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.714+0000 7f9cdf608740 1 -- start start 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cdf608740 1 Processor -- start 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba6c40 0x55ac94bb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bb1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 0x55ac94bb86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cdf608740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac94baf750 con 0x55ac94ba7630 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cdf608740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ac94ac5310 con 0x55ac94ba6c40 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cdf608740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac94baf8d0 con 0x55ac94bac7a0 2024-03-26T17:42:18.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cdf608740 10 client.0 _ll_register_callbacks cb 0x55ac94b00c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bb1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cd6d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba6c40 0x55ac94bb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bb1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60094/0 (socket says 172.21.15.105:60094) 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 -- 172.21.15.105:0/1037797840 learned_addr learned my addr 172.21.15.105:0/1037797840 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 0x55ac94bb86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 -- 172.21.15.105:0/1037797840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 msgr2=0x55ac94bb86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 0x55ac94bb86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 -- 172.21.15.105:0/1037797840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba6c40 msgr2=0x55ac94bb1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba6c40 0x55ac94bb1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 -- 172.21.15.105:0/1037797840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac948e77d0 con 0x55ac94ba7630 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba6c40 0x55ac94bb1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9ccffff640 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bb1680 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00126e0 tx=0x7f9cc0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d10dc1947816dcf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.715+0000 7f9cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:18.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.716+0000 7f9cbdffb640 1 -- 172.21.15.105:0/1037797840 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000f8c0 con 0x55ac94ba7630 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.716+0000 7f9cbdffb640 1 -- 172.21.15.105:0/1037797840 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9cc000fa60 con 0x55ac94ba7630 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.716+0000 7f9cdf608740 1 -- 172.21.15.105:0/1037797840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd0004050 con 0x55ac94ba7630 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.716+0000 7f9cbdffb640 1 -- 172.21.15.105:0/1037797840 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9cc000fc60 con 0x55ac94ba7630 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.716+0000 7f9cdf608740 10 client.16938 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.716+0000 7f9cdf608740 1 -- 172.21.15.105:0/1037797840 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac94bbbb00 con 0x55ac94ba7630 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.716+0000 7f9cbdffb640 1 -- 172.21.15.105:0/1037797840 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd0004050 con 0x55ac94ba7630 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cbdffb640 1 -- 172.21.15.105:0/1037797840 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9cc0010430 con 0x55ac94ba7630 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cdf608740 10 client.16938 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cdf608740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cdf608740 20 client.16938 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac94bbd140 0x55ac94bbf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cdf608740 4 client.16938 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cdf608740 1 -- 172.21.15.105:0/1037797840 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac94b16eb0 con 0x55ac94bbd140 2024-03-26T17:42:18.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac94bbd140 0x55ac94bbf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cbdffb640 10 client.16938 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:18.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.717+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac94bbd140 0x55ac94bbf5c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e29001772e7d6f2 server_cookie=bc37a4aa615ba70a in_seq=0 out_seq=0 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cbdffb640 1 -- 172.21.15.105:0/1037797840 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ac94b16eb0 con 0x55ac94bbd140 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cbdffb640 10 client.16938 handle_command_reply: tid=0 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 client.16938 shutdown 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 -- 172.21.15.105:0/1037797840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 msgr2=0x55ac94bb1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bb1680 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00126e0 tx=0x7f9cc0012ba0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 -- 172.21.15.105:0/1037797840 shutdown_connections 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac94bbd140 0x55ac94bbf5c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 0x55ac94bb86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba7630 0x55ac94bb1680 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1037797840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba6c40 0x55ac94bb1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cbdffb640 0 client.16938 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 -- 172.21.15.105:0/1037797840 >> 172.21.15.105:0/1037797840 conn(0x55ac94b9ffa0 msgr2=0x55ac94b9f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 -- 172.21.15.105:0/1037797840 shutdown_connections 2024-03-26T17:42:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 1 -- 172.21.15.105:0/1037797840 wait complete. 2024-03-26T17:42:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.718+0000 7f9cdf608740 20 client.16938 trim_cache size 0 max 16384 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.734+0000 7f9cdf608740 1 Processor -- start 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.734+0000 7f9cdf608740 1 -- start start 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba6c40 0x55ac94bbef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 0x55ac94bbf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cdf608740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd81a6b20 con 0x55ac94ba6c40 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cdf608740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd81a8660 con 0x55ac94ba7630 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cdf608740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd81a71f0 con 0x55ac94bac7a0 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba6c40 0x55ac94bbef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba6c40 0x55ac94bbef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60110/0 (socket says 172.21.15.105:60110) 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 -- 172.21.15.105:0/1611418226 learned_addr learned my addr 172.21.15.105:0/1611418226 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9ccffff640 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 0x55ac94bbf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 -- 172.21.15.105:0/1611418226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 msgr2=0x55ac94bbf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 0x55ac94bbf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 -- 172.21.15.105:0/1611418226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 msgr2=0x55ac94bb8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 -- 172.21.15.105:0/1611418226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac94ba7ec0 con 0x55ac94ba6c40 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba6c40 0x55ac94bbef20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0013010 tx=0x7f9cd0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e59f1e16cc8091f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cbffff640 1 -- 172.21.15.105:0/1611418226 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0003d90 con 0x55ac94ba6c40 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cbffff640 1 -- 172.21.15.105:0/1611418226 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd0004840 con 0x55ac94ba6c40 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.735+0000 7f9cbffff640 1 -- 172.21.15.105:0/1611418226 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd00052a0 con 0x55ac94ba6c40 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 -- 172.21.15.105:0/1611418226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba6c40 msgr2=0x55ac94bbef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba6c40 0x55ac94bbef20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0013010 tx=0x7f9cd0004760 comp rx=0 tx=0).stop 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 -- 172.21.15.105:0/1611418226 shutdown_connections 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94bac7a0 0x55ac94bbf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 --2- 172.21.15.105:0/1611418226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94ba6c40 0x55ac94bbef20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 -- 172.21.15.105:0/1611418226 >> 172.21.15.105:0/1611418226 conn(0x55ac94b9ffa0 msgr2=0x55ac94ba0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 -- 172.21.15.105:0/1611418226 shutdown_connections 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 -- 172.21.15.105:0/1611418226 wait complete. 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.736+0000 7f9cdf608740 1 -- start start 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cdf608740 1 Processor -- start 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94bb5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cdf608740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94bac7a0 0x55ac94bb6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cdf608740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd81a7f90 con 0x55ac94bac7a0 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cdf608740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd81a78c0 con 0x55ac94ba7630 2024-03-26T17:42:18.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cdf608740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd81a4b00 con 0x55ac94ba6c40 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cdf608740 10 client.0 _ll_register_callbacks cb 0x7f9cc0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60694/0 (socket says 172.21.15.105:60694) 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 -- 172.21.15.105:0/2974340306 learned_addr learned my addr 172.21.15.105:0/2974340306 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cd6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94bb5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94bac7a0 0x55ac94bb6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 -- 172.21.15.105:0/2974340306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 msgr2=0x55ac94bb5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94bb5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 -- 172.21.15.105:0/2974340306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94bac7a0 msgr2=0x55ac94bb6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94bac7a0 0x55ac94bb6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 -- 172.21.15.105:0/2974340306 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac94bb7430 con 0x55ac94ba7630 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9ccffff640 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb5e80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55ac94bb2620 tx=0x7f9cc00096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e4a7a9b3e7e41c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:18.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.737+0000 7f9cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cbdffb640 1 -- 172.21.15.105:0/2974340306 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0010640 con 0x55ac94ba7630 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cbdffb640 1 -- 172.21.15.105:0/2974340306 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9cc0022030 con 0x55ac94ba7630 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 1 -- 172.21.15.105:0/2974340306 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd0004840 con 0x55ac94ba7630 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cbdffb640 1 -- 172.21.15.105:0/2974340306 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9cc0036050 con 0x55ac94ba7630 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 10 client.26786 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 1 -- 172.21.15.105:0/2974340306 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac94bb8760 con 0x55ac94ba7630 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cbdffb640 1 -- 172.21.15.105:0/2974340306 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd0004840 con 0x55ac94ba7630 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cbdffb640 1 -- 172.21.15.105:0/2974340306 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9cc00035f0 con 0x55ac94ba7630 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 10 client.26786 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 10 client.26786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 20 client.26786 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac94c23a60 0x55ac94c25e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 4 client.26786 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.738+0000 7f9cdf608740 1 -- 172.21.15.105:0/2974340306 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac94b11200 con 0x55ac94c23a60 2024-03-26T17:42:18.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.739+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac94c23a60 0x55ac94c25e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:18.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.739+0000 7f9cd6d9d640 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac94c23a60 0x55ac94c25e80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35540aad6bfac1b server_cookie=1ea788bd1766e060 in_seq=0 out_seq=0 2024-03-26T17:42:18.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.739+0000 7f9cbdffb640 10 client.26786 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cbdffb640 1 -- 172.21.15.105:0/2974340306 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac94b11200 con 0x55ac94c23a60 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cbdffb640 10 client.26786 handle_command_reply: tid=0 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 client.26786 shutdown 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 -- 172.21.15.105:0/2974340306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 msgr2=0x55ac94bb5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb5e80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55ac94bb2620 tx=0x7f9cc00096a0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 -- 172.21.15.105:0/2974340306 shutdown_connections 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac94c23a60 0x55ac94c25e80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac94bac7a0 0x55ac94bb6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac94ba7630 0x55ac94bb5e80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 --2- 172.21.15.105:0/2974340306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac94ba6c40 0x55ac94bb5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cbdffb640 0 client.26786 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:18.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 -- 172.21.15.105:0/2974340306 >> 172.21.15.105:0/2974340306 conn(0x55ac94b9ffa0 msgr2=0x55ac94bb1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 -- 172.21.15.105:0/2974340306 shutdown_connections 2024-03-26T17:42:18.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 1 -- 172.21.15.105:0/2974340306 wait complete. 2024-03-26T17:42:18.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.740+0000 7f9cdf608740 20 client.26786 trim_cache size 0 max 16384 2024-03-26T17:42:18.738 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:18.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 -- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9cc40a2fa0 msgr2=0x7f9cc40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9cc40a2fa0 0x7f9cc40a5460 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80ef2e0 tx=0x7f9cc80f4f90 comp rx=0 tx=0).stop 2024-03-26T17:42:18.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 -- 172.21.15.105:0/1377160762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cd81007c0 msgr2=0x7f9cd8196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:18.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cd81007c0 0x7f9cd8196a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8000c00 tx=0x7f9cc8005dc0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 -- 172.21.15.105:0/1377160762 shutdown_connections 2024-03-26T17:42:18.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9cc40a2fa0 0x7f9cc40a5460 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd8102370 0x7f9cd819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9cd81019b0 0x7f9cd8196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 --2- 172.21.15.105:0/1377160762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9cd81007c0 0x7f9cd8196a50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:18.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 -- 172.21.15.105:0/1377160762 >> 172.21.15.105:0/1377160762 conn(0x7f9cd80fbd50 msgr2=0x7f9cd8104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:18.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 -- 172.21.15.105:0/1377160762 shutdown_connections 2024-03-26T17:42:18.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:18.741+0000 7f9cdc93b640 1 -- 172.21.15.105:0/1377160762 wait complete. 2024-03-26T17:42:19.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 107429 3.10 MB/sec execute 62 sec latency 20.178 ms 2024-03-26T17:42:19.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:18 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2325738322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:42:19.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:18 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2325738322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:42:19.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:18 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2325738322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:42:19.745 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:42:19.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:42:19.746 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:42:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.875+0000 7f13a840b640 1 -- 172.21.15.105:0/1829792628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a0102ac0 msgr2=0x7f13a0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.875+0000 7f13a840b640 1 --2- 172.21.15.105:0/1829792628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a0102ac0 0x7f13a0102f40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1390009900 tx=0x7f139002bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 -- 172.21.15.105:0/1829792628 shutdown_connections 2024-03-26T17:42:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 --2- 172.21.15.105:0/1829792628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0103480 0x7f13a0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 --2- 172.21.15.105:0/1829792628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a0102ac0 0x7f13a0102f40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 --2- 172.21.15.105:0/1829792628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a01018d0 0x7f13a0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 -- 172.21.15.105:0/1829792628 >> 172.21.15.105:0/1829792628 conn(0x7f13a00fce20 msgr2=0x7f13a00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 -- 172.21.15.105:0/1829792628 shutdown_connections 2024-03-26T17:42:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 -- 172.21.15.105:0/1829792628 wait complete. 2024-03-26T17:42:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 Processor -- start 2024-03-26T17:42:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 -- start start 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a01018d0 0x7f13a0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0102ac0 0x7f13a0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a0103480 0x7f13a019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010d3e0 con 0x7f13a01018d0 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010d260 con 0x7f13a0102ac0 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.876+0000 7f13a840b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13a010d560 con 0x7f13a0103480 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a01018d0 0x7f13a0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a01018d0 0x7f13a0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60138/0 (socket says 172.21.15.105:60138) 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 -- 172.21.15.105:0/3691150590 learned_addr learned my addr 172.21.15.105:0/3691150590 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a59a6640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0102ac0 0x7f13a0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a69a8640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a0103480 0x7f13a019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 -- 172.21.15.105:0/3691150590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a0103480 msgr2=0x7f13a019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a0103480 0x7f13a019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 -- 172.21.15.105:0/3691150590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0102ac0 msgr2=0x7f13a0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0102ac0 0x7f13a0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 -- 172.21.15.105:0/3691150590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a019f730 con 0x7f13a01018d0 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a59a6640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0102ac0 0x7f13a0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a69a8640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a0103480 0x7f13a019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a61a7640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a01018d0 0x7f13a0197b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f139c00e9c0 tx=0x7f139c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13977fe640 1 -- 172.21.15.105:0/3691150590 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c00cd80 con 0x7f13a01018d0 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13977fe640 1 -- 172.21.15.105:0/3691150590 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f139c0035f0 con 0x7f13a01018d0 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a840b640 1 -- 172.21.15.105:0/3691150590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a019fa20 con 0x7f13a01018d0 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13977fe640 1 -- 172.21.15.105:0/3691150590 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c010690 con 0x7f13a01018d0 2024-03-26T17:42:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.877+0000 7f13a840b640 1 -- 172.21.15.105:0/3691150590 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a019ff90 con 0x7f13a01018d0 2024-03-26T17:42:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.879+0000 7f13977fe640 1 -- 172.21.15.105:0/3691150590 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f139c0037a0 con 0x7f13a01018d0 2024-03-26T17:42:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.880+0000 7f13977fe640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a2fa0 0x7f138c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.880+0000 7f13a59a6640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a2fa0 0x7f138c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.880+0000 7f13977fe640 1 -- 172.21.15.105:0/3691150590 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f139c0cf720 con 0x7f13a01018d0 2024-03-26T17:42:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.880+0000 7f13aaffa740 1 Processor -- start 2024-03-26T17:42:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13aaffa740 1 -- start start 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a59a6640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a2fa0 0x7f138c0a5460 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1390009230 tx=0x7f1390005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x56139208c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208cb80 0x561392091680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x561392094180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13aaffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561391ff1630 con 0x56139208cb80 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13aaffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561391f99e10 con 0x561392091cf0 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13aaffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561391fed5e0 con 0x56139208c190 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x56139208c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a59a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208cb80 0x561392091680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a69a8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x561392094180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x56139208c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43494/0 (socket says 172.21.15.105:43494) 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a61a7640 1 -- 172.21.15.105:0/2217900715 learned_addr learned my addr 172.21.15.105:0/2217900715 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a61a7640 1 -- 172.21.15.105:0/2217900715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 msgr2=0x561392094180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2217900715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x561392094180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a61a7640 1 -- 172.21.15.105:0/2217900715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208cb80 msgr2=0x561392091680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2217900715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208cb80 0x561392091680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.881+0000 7f13a61a7640 1 -- 172.21.15.105:0/2217900715 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561391fb0740 con 0x56139208c190 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.882+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2217900715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x56139208c5b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f139c098270 tx=0x7f139c09ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0e1e975fbe26d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.882+0000 7f1394bfe640 1 -- 172.21.15.105:0/2217900715 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c09c510 con 0x56139208c190 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.882+0000 7f1394bfe640 1 -- 172.21.15.105:0/2217900715 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f139c09be10 con 0x56139208c190 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.882+0000 7f1394bfe640 1 -- 172.21.15.105:0/2217900715 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c09c070 con 0x56139208c190 2024-03-26T17:42:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.882+0000 7f13aaffa740 1 -- 172.21.15.105:0/2217900715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 msgr2=0x56139208c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.882+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2217900715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x56139208c5b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f139c098270 tx=0x7f139c09ba10 comp rx=0 tx=0).stop 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 -- 172.21.15.105:0/2217900715 shutdown_connections 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2217900715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x561392094180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2217900715 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208cb80 0x561392091680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2217900715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x56139208c5b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 -- 172.21.15.105:0/2217900715 >> 172.21.15.105:0/2217900715 conn(0x561392087700 msgr2=0x561392087b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 -- 172.21.15.105:0/2217900715 shutdown_connections 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 -- 172.21.15.105:0/2217900715 wait complete. 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 -- start start 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 Processor -- start 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x561392096740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x561392096c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x56139209dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561392094ca0 con 0x56139208c190 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561391fad1a0 con 0x561392091cf0 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561392094e20 con 0x56139208cb80 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x561392096740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x561392096740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60158/0 (socket says 172.21.15.105:60158) 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13a61a7640 1 -- 172.21.15.105:0/2190626031 learned_addr learned my addr 172.21.15.105:0/2190626031 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13a59a6640 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x561392096c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13aaffa740 10 client.0 _ll_register_callbacks cb 0x561391d04c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.883+0000 7f13a69a8640 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x56139209dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13a61a7640 1 -- 172.21.15.105:0/2190626031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 msgr2=0x561392096c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x561392096c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13a61a7640 1 -- 172.21.15.105:0/2190626031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 msgr2=0x56139209dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x56139209dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13a61a7640 1 -- 172.21.15.105:0/2190626031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561391dcd7e0 con 0x56139208c190 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13a59a6640 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x561392096c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x561392096740 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f139c0058c0 tx=0x7f139c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a414d565973b5556 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f1389ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f1389ffb640 1 -- 172.21.15.105:0/2190626031 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c07f060 con 0x56139208c190 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f1389ffb640 1 -- 172.21.15.105:0/2190626031 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f139c0a4390 con 0x56139208c190 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13aaffa740 1 -- 172.21.15.105:0/2190626031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f139c09be10 con 0x56139208c190 2024-03-26T17:42:19.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f1389ffb640 1 -- 172.21.15.105:0/2190626031 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f139c09ff60 con 0x56139208c190 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13aaffa740 10 client.16956 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.884+0000 7f13aaffa740 1 -- 172.21.15.105:0/2190626031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561391e2df70 con 0x56139208c190 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f1389ffb640 1 -- 172.21.15.105:0/2190626031 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f139c09be10 con 0x56139208c190 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f1389ffb640 1 -- 172.21.15.105:0/2190626031 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f139c097b60 con 0x56139208c190 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f13aaffa740 10 client.16956 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f13aaffa740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f13aaffa740 20 client.16956 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613920a2660 0x5613920a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f13aaffa740 4 client.16956 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f13aaffa740 1 -- 172.21.15.105:0/2190626031 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561391ffeb70 con 0x5613920a2660 2024-03-26T17:42:19.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.885+0000 7f13a59a6640 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613920a2660 0x5613920a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.886+0000 7f1389ffb640 10 client.16956 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.886+0000 7f13a59a6640 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613920a2660 0x5613920a4ae0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cafd9f80f5f27eff server_cookie=22de3486491c67cc in_seq=0 out_seq=0 2024-03-26T17:42:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f1389ffb640 1 -- 172.21.15.105:0/2190626031 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561391ffeb70 con 0x5613920a2660 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f1389ffb640 10 client.16956 handle_command_reply: tid=0 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 client.16956 shutdown 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 -- 172.21.15.105:0/2190626031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 msgr2=0x561392096740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x561392096740 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f139c0058c0 tx=0x7f139c003d40 comp rx=0 tx=0).stop 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 -- 172.21.15.105:0/2190626031 shutdown_connections 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613920a2660 0x5613920a4ae0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x56139209dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x561392096c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2190626031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x561392096740 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 -- 172.21.15.105:0/2190626031 >> 172.21.15.105:0/2190626031 conn(0x561392087700 msgr2=0x561392086900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f1389ffb640 0 client.16956 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 -- 172.21.15.105:0/2190626031 shutdown_connections 2024-03-26T17:42:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 1 -- 172.21.15.105:0/2190626031 wait complete. 2024-03-26T17:42:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.887+0000 7f13aaffa740 20 client.16956 trim_cache size 0 max 16384 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13aaffa740 1 Processor -- start 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13aaffa740 1 -- start start 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x561392096a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56139208cb80 0x56139209dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561392091cf0 0x561392086070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13aaffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01aafd0 con 0x561392091cf0 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13aaffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01acb10 con 0x56139208cb80 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13aaffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01ab6a0 con 0x56139208c190 2024-03-26T17:42:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.907+0000 7f13a69a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561392091cf0 0x561392086070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x561392096a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a69a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561392091cf0 0x561392086070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60162/0 (socket says 172.21.15.105:60162) 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x561392096a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43510/0 (socket says 172.21.15.105:43510) 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a69a8640 1 -- 172.21.15.105:0/3497181447 learned_addr learned my addr 172.21.15.105:0/3497181447 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a59a6640 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56139208cb80 0x56139209dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a61a7640 1 -- 172.21.15.105:0/3497181447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56139208cb80 msgr2=0x56139209dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a61a7640 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56139208cb80 0x56139209dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a61a7640 1 -- 172.21.15.105:0/3497181447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561392091cf0 msgr2=0x561392086070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a61a7640 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561392091cf0 0x561392086070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a61a7640 1 -- 172.21.15.105:0/3497181447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613920958a0 con 0x56139208c190 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a69a8640 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561392091cf0 0x561392086070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f13a61a7640 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x561392096a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f139c098270 tx=0x7f139c0979d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc68e98265d4b9cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f138bfff640 1 -- 172.21.15.105:0/3497181447 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c07f070 con 0x56139208c190 2024-03-26T17:42:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f138bfff640 1 -- 172.21.15.105:0/3497181447 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f139c099eb0 con 0x56139208c190 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.908+0000 7f138bfff640 1 -- 172.21.15.105:0/3497181447 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c0a4820 con 0x56139208c190 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 -- 172.21.15.105:0/3497181447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 msgr2=0x561392096a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x561392096a60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f139c098270 tx=0x7f139c0979d0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 -- 172.21.15.105:0/3497181447 shutdown_connections 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561392091cf0 0x561392086070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56139208cb80 0x56139209dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 --2- 172.21.15.105:0/3497181447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208c190 0x561392096a60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 -- 172.21.15.105:0/3497181447 >> 172.21.15.105:0/3497181447 conn(0x561392087700 msgr2=0x561392096350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 -- 172.21.15.105:0/3497181447 shutdown_connections 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 -- 172.21.15.105:0/3497181447 wait complete. 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 -- start start 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.909+0000 7f13aaffa740 1 Processor -- start 2024-03-26T17:42:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x56139209ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x56139209b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13aaffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x561392106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13aaffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01ac440 con 0x56139208c190 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13aaffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01abd70 con 0x561392091cf0 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13aaffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13a01a8fb0 con 0x56139208cb80 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13aaffa740 10 client.0 _ll_register_callbacks cb 0x7f139c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x56139209ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x56139209ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60168/0 (socket says 172.21.15.105:60168) 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 -- 172.21.15.105:0/2397050340 learned_addr learned my addr 172.21.15.105:0/2397050340 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a69a8640 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x561392106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a59a6640 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x56139209b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 -- 172.21.15.105:0/2397050340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 msgr2=0x56139209b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x56139209b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 -- 172.21.15.105:0/2397050340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 msgr2=0x561392106a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x561392106a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 -- 172.21.15.105:0/2397050340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561391cc5170 con 0x56139208c190 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13a61a7640 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x56139209ab10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f139c097f40 tx=0x7f139c005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44439595d1160a55 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f1389ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f1389ffb640 1 -- 172.21.15.105:0/2397050340 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c07f070 con 0x56139208c190 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f1389ffb640 1 -- 172.21.15.105:0/2397050340 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f139c097b60 con 0x56139208c190 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.910+0000 7f13aaffa740 1 -- 172.21.15.105:0/2397050340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561391ff8ec0 con 0x56139208c190 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f1389ffb640 1 -- 172.21.15.105:0/2397050340 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f139c0a4820 con 0x56139208c190 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13aaffa740 10 client.16962 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13aaffa740 1 -- 172.21.15.105:0/2397050340 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561391cc5170 con 0x56139208c190 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f1389ffb640 1 -- 172.21.15.105:0/2397050340 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561391ff8ec0 con 0x56139208c190 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f1389ffb640 1 -- 172.21.15.105:0/2397050340 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f139c0a4390 con 0x56139208c190 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13aaffa740 10 client.16962 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13aaffa740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13aaffa740 20 client.16962 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561392108490 0x56139210a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13aaffa740 4 client.16962 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:42:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13aaffa740 1 -- 172.21.15.105:0/2397050340 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561391f76be0 con 0x561392108490 2024-03-26T17:42:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13a59a6640 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561392108490 0x56139210a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f1389ffb640 10 client.16962 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.911+0000 7f13a59a6640 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561392108490 0x56139210a880 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac069ae51b44c0a server_cookie=207eea9856bc3c91 in_seq=0 out_seq=0 2024-03-26T17:42:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.912+0000 7f1389ffb640 1 -- 172.21.15.105:0/2397050340 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561391f76be0 con 0x561392108490 2024-03-26T17:42:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.912+0000 7f1389ffb640 10 client.16962 handle_command_reply: tid=0 2024-03-26T17:42:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.912+0000 7f13aaffa740 1 client.16962 shutdown 2024-03-26T17:42:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 -- 172.21.15.105:0/2397050340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 msgr2=0x56139209ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x56139209ab10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f139c097f40 tx=0x7f139c005ab0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 -- 172.21.15.105:0/2397050340 shutdown_connections 2024-03-26T17:42:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561392108490 0x56139210a880 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561392091cf0 0x561392106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56139208cb80 0x56139209b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 --2- 172.21.15.105:0/2397050340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56139208c190 0x56139209ab10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 -- 172.21.15.105:0/2397050340 >> 172.21.15.105:0/2397050340 conn(0x561392087700 msgr2=0x56139208b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f1389ffb640 0 client.16962 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 -- 172.21.15.105:0/2397050340 shutdown_connections 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 1 -- 172.21.15.105:0/2397050340 wait complete. 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13aaffa740 20 client.16962 trim_cache size 0 max 16384 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:42:19.911 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "2572f0fd-c0d9-4074-8676-290eaa0b6774", 2024-03-26T17:42:19.912 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:42:19.912 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:42:19.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.913+0000 7f13a840b640 1 -- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a2fa0 msgr2=0x7f138c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a2fa0 0x7f138c0a5460 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1390009230 tx=0x7f1390005fd0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 -- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a01018d0 msgr2=0x7f13a0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:19.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a01018d0 0x7f13a0197b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f139c00e9c0 tx=0x7f139c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:42:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 -- 172.21.15.105:0/3691150590 shutdown_connections 2024-03-26T17:42:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f138c0a2fa0 0x7f138c0a5460 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13a0103480 0x7f13a019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13a0102ac0 0x7f13a0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 --2- 172.21.15.105:0/3691150590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13a01018d0 0x7f13a0197b10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 -- 172.21.15.105:0/3691150590 >> 172.21.15.105:0/3691150590 conn(0x7f13a00fce20 msgr2=0x7f13a0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 -- 172.21.15.105:0/3691150590 shutdown_connections 2024-03-26T17:42:19.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:19.914+0000 7f13a840b640 1 -- 172.21.15.105:0/3691150590 wait complete. 2024-03-26T17:42:19.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:42:19.918 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:42:20.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 108066 3.12 MB/sec execute 63 sec latency 16.892 ms 2024-03-26T17:42:20.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 -- 172.21.15.105:0/2014408347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 msgr2=0x7f7350109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 --2- 172.21.15.105:0/2014408347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 0x7f7350109d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7348009be0 tx=0x7f734802bef0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 -- 172.21.15.105:0/2014408347 shutdown_connections 2024-03-26T17:42:20.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 --2- 172.21.15.105:0/2014408347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 0x7f7350109d10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 --2- 172.21.15.105:0/2014408347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7350102ac0 0x7f7350102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 --2- 172.21.15.105:0/2014408347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73501018d0 0x7f7350101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 -- 172.21.15.105:0/2014408347 >> 172.21.15.105:0/2014408347 conn(0x7f73500fce60 msgr2=0x7f73500ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:20.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 -- 172.21.15.105:0/2014408347 shutdown_connections 2024-03-26T17:42:20.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.044+0000 7f7354d7e640 1 -- 172.21.15.105:0/2014408347 wait complete. 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.045+0000 7f7354d7e640 1 Processor -- start 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.045+0000 7f7354d7e640 1 -- start start 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.045+0000 7f7354d7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73501018d0 0x7f735019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.045+0000 7f7354d7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7350102ac0 0x7f735019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.045+0000 7f7354d7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 0x7f73501a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.045+0000 7f7354d7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f735010d400 con 0x7f7350102ac0 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.045+0000 7f7354d7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f735010d280 con 0x7f73501018d0 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.045+0000 7f7354d7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f735010d580 con 0x7f7350103480 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73501018d0 0x7f735019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 0x7f73501a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7350102ac0 0x7f735019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73501018d0 0x7f735019e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60752/0 (socket says 172.21.15.105:60752) 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 0x7f73501a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43544/0 (socket says 172.21.15.105:43544) 2024-03-26T17:42:20.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734e59c640 1 -- 172.21.15.105:0/691132609 learned_addr learned my addr 172.21.15.105:0/691132609 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734ed9d640 1 -- 172.21.15.105:0/691132609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73501018d0 msgr2=0x7f735019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734ed9d640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73501018d0 0x7f735019e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734ed9d640 1 -- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7350102ac0 msgr2=0x7f735019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734ed9d640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7350102ac0 0x7f735019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734ed9d640 1 -- 172.21.15.105:0/691132609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73501a5cd0 con 0x7f7350103480 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734dd9b640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7350102ac0 0x7f735019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f734ed9d640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 0x7f73501a5660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f734802c430 tx=0x7f7348033c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f733f7fe640 1 -- 172.21.15.105:0/691132609 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348043020 con 0x7f7350103480 2024-03-26T17:42:20.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.046+0000 7f733f7fe640 1 -- 172.21.15.105:0/691132609 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7348003b80 con 0x7f7350103480 2024-03-26T17:42:20.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.047+0000 7f733f7fe640 1 -- 172.21.15.105:0/691132609 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348034690 con 0x7f7350103480 2024-03-26T17:42:20.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.047+0000 7f7354d7e640 1 -- 172.21.15.105:0/691132609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73501a5f60 con 0x7f7350103480 2024-03-26T17:42:20.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.047+0000 7f7354d7e640 1 -- 172.21.15.105:0/691132609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73501a6470 con 0x7f7350103480 2024-03-26T17:42:20.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.050+0000 7f733f7fe640 1 -- 172.21.15.105:0/691132609 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7348046020 con 0x7f7350103480 2024-03-26T17:42:20.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.051+0000 7f735796b740 1 Processor -- start 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.051+0000 7f735796b740 1 -- start start 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.051+0000 7f733f7fe640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73380a2fa0 0x7f73380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.051+0000 7f735796b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b980bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.051+0000 7f735796b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b9810de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.051+0000 7f735796b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 0x55a9b98138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.051+0000 7f735796b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a9b9770980 con 0x55a9b980c2e0 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.051+0000 7f735796b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a9b9718be0 con 0x55a9b9811450 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f735796b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a9b976c930 con 0x55a9b980b8f0 2024-03-26T17:42:20.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f733f7fe640 1 -- 172.21.15.105:0/691132609 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f73480f1a60 con 0x7f7350103480 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b9810de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b9810de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60198/0 (socket says 172.21.15.105:60198) 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 -- 172.21.15.105:0/3770443893 learned_addr learned my addr 172.21.15.105:0/3770443893 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734e59c640 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b980bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734ed9d640 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 0x55a9b98138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73380a2fa0 0x7f73380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 -- 172.21.15.105:0/3770443893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 msgr2=0x55a9b980bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b980bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 -- 172.21.15.105:0/3770443893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 msgr2=0x55a9b98138e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 0x55a9b98138e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 -- 172.21.15.105:0/3770443893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b954e7d0 con 0x55a9b980c2e0 2024-03-26T17:42:20.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.052+0000 7f734dd9b640 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b9810de0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7340012b50 tx=0x7f7340012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b473485e275dcb75 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.053+0000 7f733cff9640 1 -- 172.21.15.105:0/3770443893 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340021020 con 0x55a9b980c2e0 2024-03-26T17:42:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.053+0000 7f733cff9640 1 -- 172.21.15.105:0/3770443893 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7340010d70 con 0x55a9b980c2e0 2024-03-26T17:42:20.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.053+0000 7f733cff9640 1 -- 172.21.15.105:0/3770443893 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7340025e00 con 0x55a9b980c2e0 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 -- 172.21.15.105:0/3770443893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 msgr2=0x55a9b9810de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f734dd9b640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73380a2fa0 0x7f73380a5460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7340022000 tx=0x7f7340020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b9810de0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7340012b50 tx=0x7f7340012f50 comp rx=0 tx=0).stop 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 -- 172.21.15.105:0/3770443893 shutdown_connections 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 0x55a9b98138e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b9810de0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 --2- 172.21.15.105:0/3770443893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b980bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 -- 172.21.15.105:0/3770443893 >> 172.21.15.105:0/3770443893 conn(0x55a9b9806e60 msgr2=0x55a9b9807260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:20.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 -- 172.21.15.105:0/3770443893 shutdown_connections 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 -- 172.21.15.105:0/3770443893 wait complete. 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 -- start start 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.054+0000 7f735796b740 1 Processor -- start 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f735796b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b9815b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f735796b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b98160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f735796b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 0x55a9b981d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f735796b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a9b9814400 con 0x55a9b980c2e0 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f735796b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a9b972c160 con 0x55a9b9811450 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f735796b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a9b9814580 con 0x55a9b980b8f0 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f735796b740 10 client.0 _ll_register_callbacks cb 0x55a9b9768720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b9815b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b9815b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43562/0 (socket says 172.21.15.105:43562) 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734e59c640 1 -- 172.21.15.105:0/642791543 learned_addr learned my addr 172.21.15.105:0/642791543 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734dd9b640 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b98160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734ed9d640 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 0x55a9b981d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734e59c640 1 -- 172.21.15.105:0/642791543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 msgr2=0x55a9b981d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734e59c640 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 0x55a9b981d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734e59c640 1 -- 172.21.15.105:0/642791543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 msgr2=0x55a9b98160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734e59c640 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b98160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734e59c640 1 -- 172.21.15.105:0/642791543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b94843f0 con 0x55a9b980b8f0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.055+0000 7f734dd9b640 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b98160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.056+0000 7f734e59c640 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b9815b60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7344012840 tx=0x7f7344012d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb1f9b4523ae2ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.056+0000 7f7335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.056+0000 7f7335ffb640 1 -- 172.21.15.105:0/642791543 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344004030 con 0x55a9b980b8f0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.056+0000 7f7335ffb640 1 -- 172.21.15.105:0/642791543 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f73440041d0 con 0x55a9b980b8f0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.056+0000 7f735796b740 1 -- 172.21.15.105:0/642791543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7340010d70 con 0x55a9b980b8f0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.056+0000 7f7335ffb640 1 -- 172.21.15.105:0/642791543 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7344004410 con 0x55a9b980b8f0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.056+0000 7f735796b740 10 client.26815 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.056+0000 7f735796b740 1 -- 172.21.15.105:0/642791543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a9b94440f0 con 0x55a9b980b8f0 2024-03-26T17:42:20.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f7335ffb640 1 -- 172.21.15.105:0/642791543 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7340010d70 con 0x55a9b980b8f0 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f7335ffb640 1 -- 172.21.15.105:0/642791543 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7344003b10 con 0x55a9b980b8f0 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f735796b740 10 client.26815 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f735796b740 10 client.26815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f735796b740 20 client.26815 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f735796b740 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a9b9821a50 0x55a9b9823f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f735796b740 4 client.26815 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f735796b740 1 -- 172.21.15.105:0/642791543 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9b977dec0 con 0x55a9b9821a50 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.057+0000 7f734dd9b640 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a9b9821a50 0x55a9b9823f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.058+0000 7f734dd9b640 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a9b9821a50 0x55a9b9823f10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95c7a148736619 server_cookie=3d2078f7494a600b in_seq=0 out_seq=0 2024-03-26T17:42:20.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.058+0000 7f7335ffb640 10 client.26815 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f7335ffb640 1 -- 172.21.15.105:0/642791543 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a9b977dec0 con 0x55a9b9821a50 2024-03-26T17:42:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f7335ffb640 10 client.26815 handle_command_reply: tid=0 2024-03-26T17:42:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 client.26815 shutdown 2024-03-26T17:42:20.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 -- 172.21.15.105:0/642791543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 msgr2=0x55a9b9815b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b9815b60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7344012840 tx=0x7f7344012d00 comp rx=0 tx=0).stop 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 -- 172.21.15.105:0/642791543 shutdown_connections 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a9b9821a50 0x55a9b9823f10 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b9811450 0x55a9b981d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b98160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 --2- 172.21.15.105:0/642791543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b980b8f0 0x55a9b9815b60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f7335ffb640 0 client.26815 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 -- 172.21.15.105:0/642791543 >> 172.21.15.105:0/642791543 conn(0x55a9b9806e60 msgr2=0x55a9b9806080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 -- 172.21.15.105:0/642791543 shutdown_connections 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.059+0000 7f735796b740 1 -- 172.21.15.105:0/642791543 wait complete. 2024-03-26T17:42:20.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.060+0000 7f735796b740 20 client.26815 trim_cache size 0 max 16384 2024-03-26T17:42:20.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.079+0000 7f735796b740 1 Processor -- start 2024-03-26T17:42:20.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f735796b740 1 -- start start 2024-03-26T17:42:20.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f735796b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980b8f0 0x55a9b980a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f735796b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980c2e0 0x55a9b981d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f735796b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 0x55a9b9815180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f735796b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73501af260 con 0x55a9b980b8f0 2024-03-26T17:42:20.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f735796b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73501b0da0 con 0x55a9b980c2e0 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f735796b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73501af930 con 0x55a9b9811450 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980b8f0 0x55a9b980a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980b8f0 0x55a9b980a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60212/0 (socket says 172.21.15.105:60212) 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 -- 172.21.15.105:0/2951250371 learned_addr learned my addr 172.21.15.105:0/2951250371 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734dd9b640 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980c2e0 0x55a9b981d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734ed9d640 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 0x55a9b9815180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 -- 172.21.15.105:0/2951250371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 msgr2=0x55a9b9815180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 0x55a9b9815180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 -- 172.21.15.105:0/2951250371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980c2e0 msgr2=0x55a9b981d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980c2e0 0x55a9b981d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 -- 172.21.15.105:0/2951250371 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b9821810 con 0x55a9b980b8f0 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.080+0000 7f734e59c640 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980b8f0 0x55a9b980a7d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7344003f80 tx=0x7f73440124b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab32a5086719cac1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:20.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.081+0000 7f7337fff640 1 -- 172.21.15.105:0/2951250371 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344004000 con 0x55a9b980b8f0 2024-03-26T17:42:20.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.081+0000 7f7337fff640 1 -- 172.21.15.105:0/2951250371 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73440041a0 con 0x55a9b980b8f0 2024-03-26T17:42:20.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.081+0000 7f7337fff640 1 -- 172.21.15.105:0/2951250371 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73440039e0 con 0x55a9b980b8f0 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.081+0000 7f735796b740 1 -- 172.21.15.105:0/2951250371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980b8f0 msgr2=0x55a9b980a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.081+0000 7f735796b740 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980b8f0 0x55a9b980a7d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7344003f80 tx=0x7f73440124b0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 -- 172.21.15.105:0/2951250371 shutdown_connections 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 0x55a9b9815180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980c2e0 0x55a9b981d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 --2- 172.21.15.105:0/2951250371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980b8f0 0x55a9b980a7d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 -- 172.21.15.105:0/2951250371 >> 172.21.15.105:0/2951250371 conn(0x55a9b9806e60 msgr2=0x55a9b980a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 -- 172.21.15.105:0/2951250371 shutdown_connections 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 -- 172.21.15.105:0/2951250371 wait complete. 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 -- start start 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 Processor -- start 2024-03-26T17:42:20.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980b8f0 0x55a9b981a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b981ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 0x55a9b981bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73501b06d0 con 0x55a9b980c2e0 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73501b0000 con 0x55a9b980b8f0 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.082+0000 7f735796b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73501ad240 con 0x55a9b9811450 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f735796b740 10 client.0 _ll_register_callbacks cb 0x7f7344009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b981ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b981ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60218/0 (socket says 172.21.15.105:60218) 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 -- 172.21.15.105:0/361698719 learned_addr learned my addr 172.21.15.105:0/361698719 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734e59c640 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980b8f0 0x55a9b981a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734ed9d640 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 0x55a9b981bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 -- 172.21.15.105:0/361698719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 msgr2=0x55a9b981bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 0x55a9b981bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 -- 172.21.15.105:0/361698719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980b8f0 msgr2=0x55a9b981a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980b8f0 0x55a9b981a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 -- 172.21.15.105:0/361698719 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b981c390 con 0x55a9b980c2e0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f734dd9b640 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b981ade0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7340005c90 tx=0x7f7340028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b2f0def8fe1f7b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f7335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f7335ffb640 1 -- 172.21.15.105:0/361698719 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734002a040 con 0x55a9b980c2e0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f7335ffb640 1 -- 172.21.15.105:0/361698719 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f73400050b0 con 0x55a9b980c2e0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f735796b740 1 -- 172.21.15.105:0/361698719 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73440041a0 con 0x55a9b980c2e0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f7335ffb640 1 -- 172.21.15.105:0/361698719 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7340047050 con 0x55a9b980c2e0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f735796b740 10 client.16980 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.083+0000 7f735796b740 1 -- 172.21.15.105:0/361698719 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a9b9887e80 con 0x55a9b980c2e0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f7335ffb640 1 -- 172.21.15.105:0/361698719 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73440041a0 con 0x55a9b980c2e0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f7335ffb640 1 -- 172.21.15.105:0/361698719 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7340037650 con 0x55a9b980c2e0 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f735796b740 10 client.16980 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:20.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f735796b740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f735796b740 20 client.16980 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f735796b740 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a9b98886c0 0x55a9b988aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f735796b740 4 client.16980 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f735796b740 1 -- 172.21.15.105:0/361698719 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a9b9778210 con 0x55a9b98886c0 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f734e59c640 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a9b98886c0 0x55a9b988aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f734e59c640 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a9b98886c0 0x55a9b988aae0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f740fc19b9e4d00 server_cookie=79a1125414f20d49 in_seq=0 out_seq=0 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.084+0000 7f7335ffb640 10 client.16980 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f7335ffb640 1 -- 172.21.15.105:0/361698719 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a9b9778210 con 0x55a9b98886c0 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f7335ffb640 10 client.16980 handle_command_reply: tid=0 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 client.16980 shutdown 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 -- 172.21.15.105:0/361698719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 msgr2=0x55a9b981ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b981ade0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7340005c90 tx=0x7f7340028000 comp rx=0 tx=0).stop 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 -- 172.21.15.105:0/361698719 shutdown_connections 2024-03-26T17:42:20.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a9b98886c0 0x55a9b988aae0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a9b9811450 0x55a9b981bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a9b980c2e0 0x55a9b981ade0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f7335ffb640 0 client.16980 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 --2- 172.21.15.105:0/361698719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9b980b8f0 0x55a9b981a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 -- 172.21.15.105:0/361698719 >> 172.21.15.105:0/361698719 conn(0x55a9b9806e60 msgr2=0x55a9b980a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 -- 172.21.15.105:0/361698719 shutdown_connections 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 1 -- 172.21.15.105:0/361698719 wait complete. 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.085+0000 7f735796b740 20 client.16980 trim_cache size 0 max 16384 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:42:20.083 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:42:20.084 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:42:20.084 INFO:teuthology.orchestra.run.smithi105.stdout: "2572f0fd-c0d9-4074-8676-290eaa0b6774": { 2024-03-26T17:42:20.084 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:42:20.084 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "2572f0fd-c0d9-4074-8676-290eaa0b6774", 2024-03-26T17:42:20.084 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:42:20.084 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:42:20.084 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:42:20.084 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 -- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73380a2fa0 msgr2=0x7f73380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73380a2fa0 0x7f73380a5460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7340022000 tx=0x7f7340020000 comp rx=0 tx=0).stop 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 -- 172.21.15.105:0/691132609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 msgr2=0x7f73501a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 0x7f73501a5660 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f734802c430 tx=0x7f7348033c90 comp rx=0 tx=0).stop 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 -- 172.21.15.105:0/691132609 shutdown_connections 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73380a2fa0 0x7f73380a5460 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7350103480 0x7f73501a5660 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7350102ac0 0x7f735019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 --2- 172.21.15.105:0/691132609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73501018d0 0x7f735019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 -- 172.21.15.105:0/691132609 >> 172.21.15.105:0/691132609 conn(0x7f73500fce60 msgr2=0x7f7350103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 -- 172.21.15.105:0/691132609 shutdown_connections 2024-03-26T17:42:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:20.086+0000 7f7354d7e640 1 -- 172.21.15.105:0/691132609 wait complete. 2024-03-26T17:42:20.088 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:19 smithi105 ceph-mon[32280]: pgmap v467: 97 pgs: 97 active+clean; 26 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-03-26T17:42:20.089 INFO:tasks.cephfs.filesystem:scrub status for tag:2572f0fd-c0d9-4074-8676-290eaa0b6774 - {'path': '/', 'tag': '2572f0fd-c0d9-4074-8676-290eaa0b6774', 'options': 'recursive,force'} 2024-03-26T17:42:20.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:19 smithi144 ceph-mon[36095]: pgmap v467: 97 pgs: 97 active+clean; 26 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-03-26T17:42:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:19 smithi135 ceph-mon[35636]: pgmap v467: 97 pgs: 97 active+clean; 26 MiB data, 591 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-03-26T17:42:21.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 108755 3.10 MB/sec execute 64 sec latency 16.772 ms 2024-03-26T17:42:22.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 109426 3.14 MB/sec execute 65 sec latency 18.755 ms 2024-03-26T17:42:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:21 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:42:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:21 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:21 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:42:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:21 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:22.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:21 smithi144 ceph-mon[36095]: pgmap v468: 97 pgs: 97 active+clean; 26 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:42:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:21 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:42:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:21 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:22.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:21 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:42:22.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:21 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:22.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:21 smithi105 ceph-mon[32280]: pgmap v468: 97 pgs: 97 active+clean; 26 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:42:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:21 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:42:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:21 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:21 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:42:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:21 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:21 smithi135 ceph-mon[35636]: pgmap v468: 97 pgs: 97 active+clean; 26 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:42:23.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 109998 3.16 MB/sec execute 66 sec latency 15.455 ms 2024-03-26T17:42:23.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:22 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:42:23.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:22 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:42:23.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:22 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:42:24.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 110447 3.14 MB/sec execute 67 sec latency 16.609 ms 2024-03-26T17:42:24.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:23 smithi144 ceph-mon[36095]: pgmap v469: 97 pgs: 97 active+clean; 26 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:42:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:23 smithi105 ceph-mon[32280]: pgmap v469: 97 pgs: 97 active+clean; 26 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:42:24.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:23 smithi135 ceph-mon[35636]: pgmap v469: 97 pgs: 97 active+clean; 26 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 -- 172.21.15.105:0/471328775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb860107600 msgr2=0x7fb860109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 --2- 172.21.15.105:0/471328775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb860107600 0x7fb860109a10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb858009ca0 tx=0x7fb85802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 -- 172.21.15.105:0/471328775 shutdown_connections 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 --2- 172.21.15.105:0/471328775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb860107600 0x7fb860109a10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 --2- 172.21.15.105:0/471328775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb860104cd0 0x7fb8601070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 --2- 172.21.15.105:0/471328775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8601023a0 0x7fb860104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 -- 172.21.15.105:0/471328775 >> 172.21.15.105:0/471328775 conn(0x7fb8600fbd50 msgr2=0x7fb8600fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 -- 172.21.15.105:0/471328775 shutdown_connections 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 -- 172.21.15.105:0/471328775 wait complete. 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 Processor -- start 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 -- start start 2024-03-26T17:42:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.709+0000 7fb867909640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8601023a0 0x7fb860197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb867909640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb860104cd0 0x7fb860197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb867909640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb860107600 0x7fb86019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb867909640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb86010d410 con 0x7fb8601023a0 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb867909640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb86010d290 con 0x7fb860107600 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb867909640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb86010d590 con 0x7fb860104cd0 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8601023a0 0x7fb860197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8601023a0 0x7fb860197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37256/0 (socket says 172.21.15.105:37256) 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 -- 172.21.15.105:0/3349184338 learned_addr learned my addr 172.21.15.105:0/3349184338 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb865ea6640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb860107600 0x7fb86019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb864ea4640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb860104cd0 0x7fb860197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 -- 172.21.15.105:0/3349184338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb860104cd0 msgr2=0x7fb860197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb860104cd0 0x7fb860197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 -- 172.21.15.105:0/3349184338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb860107600 msgr2=0x7fb86019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb860107600 0x7fb86019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 -- 172.21.15.105:0/3349184338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86019f670 con 0x7fb8601023a0 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb865ea6640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb860107600 0x7fb86019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8656a5640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8601023a0 0x7fb860197a50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9f0 tx=0x7fb85c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8567fc640 1 -- 172.21.15.105:0/3349184338 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00cd80 con 0x7fb8601023a0 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb867909640 1 -- 172.21.15.105:0/3349184338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86019f960 con 0x7fb8601023a0 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8567fc640 1 -- 172.21.15.105:0/3349184338 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb85c0108e0 con 0x7fb8601023a0 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.710+0000 7fb8567fc640 1 -- 172.21.15.105:0/3349184338 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c018920 con 0x7fb8601023a0 2024-03-26T17:42:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.711+0000 7fb867909640 1 -- 172.21.15.105:0/3349184338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86019ff00 con 0x7fb8601023a0 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.713+0000 7fb8567fc640 1 -- 172.21.15.105:0/3349184338 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb85c010a90 con 0x7fb8601023a0 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.713+0000 7fb86a5d1740 1 Processor -- start 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.713+0000 7fb86a5d1740 1 -- start start 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.713+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.713+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.713+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc3125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.713+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5653dc26f0b0 con 0x5653dc310130 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.713+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653dc217960 con 0x5653dc30a5d0 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5653dc26b060 con 0x5653dc30afc0 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb865ea6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb8656a5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb865ea6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33246/0 (socket says 172.21.15.105:33246) 2024-03-26T17:42:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb865ea6640 1 -- 172.21.15.105:0/3876845487 learned_addr learned my addr 172.21.15.105:0/3876845487 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb864ea4640 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc3125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb8567fc640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb84c0a3040 0x7fb84c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb865ea6640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb84c0a3040 0x7fb84c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb864ea4640 1 -- 172.21.15.105:0/3876845487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 msgr2=0x5653dc30fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb8567fc640 1 -- 172.21.15.105:0/3349184338 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb85c0cf720 con 0x7fb8601023a0 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb864ea4640 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb864ea4640 1 -- 172.21.15.105:0/3876845487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 msgr2=0x5653dc30a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb864ea4640 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb864ea4640 1 -- 172.21.15.105:0/3876845487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dc229c60 con 0x5653dc310130 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb865ea6640 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb8656a5640 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb864ea4640 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc3125c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb860069a60 tx=0x7fb850011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a62122c6fd939f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb84b7fe640 1 -- 172.21.15.105:0/3876845487 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85001e070 con 0x5653dc310130 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb865ea6640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb84c0a3040 0x7fb84c0a5500 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5653dc30b820 tx=0x7fb85802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.714+0000 7fb84b7fe640 1 -- 172.21.15.105:0/3876845487 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb850009d70 con 0x5653dc310130 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb84b7fe640 1 -- 172.21.15.105:0/3876845487 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850018730 con 0x5653dc310130 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3876845487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 msgr2=0x5653dc3125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc3125c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb860069a60 tx=0x7fb850011810 comp rx=0 tx=0).stop 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3876845487 shutdown_connections 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc3125c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/3876845487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3876845487 >> 172.21.15.105:0/3876845487 conn(0x5653dc305b40 msgr2=0x5653dc305f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3876845487 shutdown_connections 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.715+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3876845487 wait complete. 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 1 -- start start 2024-03-26T17:42:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 1 Processor -- start 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30a5d0 0x5653dc3143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30afc0 0x5653dc3148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 0x5653dc31b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5653dc3130e0 con 0x5653dc30afc0 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5653dc22adb0 con 0x5653dc310130 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5653dc313260 con 0x5653dc30a5d0 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb86a5d1740 10 client.0 _ll_register_callbacks cb 0x5653dc22f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb864ea4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30afc0 0x5653dc3148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb864ea4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30afc0 0x5653dc3148f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37286/0 (socket says 172.21.15.105:37286) 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb864ea4640 1 -- 172.21.15.105:0/2229473574 learned_addr learned my addr 172.21.15.105:0/2229473574 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb8656a5640 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30a5d0 0x5653dc3143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.716+0000 7fb865ea6640 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 0x5653dc31b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb864ea4640 1 -- 172.21.15.105:0/2229473574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30a5d0 msgr2=0x5653dc3143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:24.716 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb864ea4640 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30a5d0 0x5653dc3143b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb864ea4640 1 -- 172.21.15.105:0/2229473574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 msgr2=0x5653dc31b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb864ea4640 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 0x5653dc31b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb864ea4640 1 -- 172.21.15.105:0/2229473574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dbf83210 con 0x5653dc30afc0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb864ea4640 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30afc0 0x5653dc3148f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb850000c00 tx=0x7fb8500056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d914d9746b608cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb848ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb848ff9640 1 -- 172.21.15.105:0/2229473574 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850027020 con 0x5653dc30afc0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb848ff9640 1 -- 172.21.15.105:0/2229473574 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb850019620 con 0x5653dc30afc0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2229473574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb850009d70 con 0x5653dc30afc0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb848ff9640 1 -- 172.21.15.105:0/2229473574 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb850032050 con 0x5653dc30afc0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb86a5d1740 10 client.16992 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2229473574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5653dc04f620 con 0x5653dc30afc0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.717+0000 7fb848ff9640 1 -- 172.21.15.105:0/2229473574 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb850009d70 con 0x5653dc30afc0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb848ff9640 1 -- 172.21.15.105:0/2229473574 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb8500184b0 con 0x5653dc30afc0 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb86a5d1740 10 client.16992 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb86a5d1740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb86a5d1740 20 client.16992 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5653dc31ff90 0x5653dc3223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb86a5d1740 4 client.16992 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2229473574 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653dc27c5f0 con 0x5653dc31ff90 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb8656a5640 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5653dc31ff90 0x5653dc3223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb8656a5640 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5653dc31ff90 0x5653dc3223f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c854b11b3f3b12d server_cookie=39977f992c172d3f in_seq=0 out_seq=0 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.718+0000 7fb848ff9640 10 client.16992 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb848ff9640 1 -- 172.21.15.105:0/2229473574 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5653dc27c5f0 con 0x5653dc31ff90 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb848ff9640 10 client.16992 handle_command_reply: tid=0 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 client.16992 shutdown 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2229473574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30afc0 msgr2=0x5653dc3148f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30afc0 0x5653dc3148f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb850000c00 tx=0x7fb8500056a0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2229473574 shutdown_connections 2024-03-26T17:42:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5653dc31ff90 0x5653dc3223f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 0x5653dc31b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30afc0 0x5653dc3148f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb848ff9640 0 client.16992 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2229473574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30a5d0 0x5653dc3143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2229473574 >> 172.21.15.105:0/2229473574 conn(0x5653dc305b40 msgr2=0x5653dc304df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2229473574 shutdown_connections 2024-03-26T17:42:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.719+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2229473574 wait complete. 2024-03-26T17:42:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.720+0000 7fb86a5d1740 20 client.16992 trim_cache size 0 max 16384 2024-03-26T17:42:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb86a5d1740 1 Processor -- start 2024-03-26T17:42:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb86a5d1740 1 -- start start 2024-03-26T17:42:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc30bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601a8e00 con 0x5653dc310130 2024-03-26T17:42:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601aa940 con 0x5653dc30a5d0 2024-03-26T17:42:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601a94d0 con 0x5653dc30afc0 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb864ea4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb8656a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb865ea6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc30bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb8656a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33274/0 (socket says 172.21.15.105:33274) 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb864ea4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39040/0 (socket says 172.21.15.105:39040) 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb8656a5640 1 -- 172.21.15.105:0/3642930302 learned_addr learned my addr 172.21.15.105:0/3642930302 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.736+0000 7fb865ea6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc30bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37300/0 (socket says 172.21.15.105:37300) 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb8656a5640 1 -- 172.21.15.105:0/3642930302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 msgr2=0x5653dc30ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb8656a5640 1 --2- 172.21.15.105:0/3642930302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb8656a5640 1 -- 172.21.15.105:0/3642930302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 msgr2=0x5653dc30bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb8656a5640 1 --2- 172.21.15.105:0/3642930302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc30bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb8656a5640 1 -- 172.21.15.105:0/3642930302 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dc3068b0 con 0x5653dc30a5d0 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb865ea6640 1 --2- 172.21.15.105:0/3642930302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc30bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb8656a5640 1 --2- 172.21.15.105:0/3642930302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30fac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9c0 tx=0x7fb85c0bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65d0f082d979fde7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb84affd640 1 -- 172.21.15.105:0/3642930302 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0a47c0 con 0x5653dc30a5d0 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb84affd640 1 -- 172.21.15.105:0/3642930302 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb85c09a160 con 0x5653dc30a5d0 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.737+0000 7fb84affd640 1 -- 172.21.15.105:0/3642930302 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0d1e60 con 0x5653dc30a5d0 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3642930302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 msgr2=0x5653dc30fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/3642930302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30fac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9c0 tx=0x7fb85c0bc040 comp rx=0 tx=0).stop 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3642930302 shutdown_connections 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/3642930302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc310130 0x5653dc30bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/3642930302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc30ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/3642930302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc30a5d0 0x5653dc30fac0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3642930302 >> 172.21.15.105:0/3642930302 conn(0x5653dc305b40 msgr2=0x5653dc30f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3642930302 shutdown_connections 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.738+0000 7fb86a5d1740 1 -- 172.21.15.105:0/3642930302 wait complete. 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 1 -- start start 2024-03-26T17:42:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 1 Processor -- start 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30a5d0 0x5653dc3811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc3816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 0x5653dc388730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601aa270 con 0x5653dc30a5d0 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601a9ba0 con 0x5653dc310130 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601a6de0 con 0x5653dc30afc0 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb86a5d1740 10 client.0 _ll_register_callbacks cb 0x7fb85001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb864ea4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc3816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb8656a5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30a5d0 0x5653dc3811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb864ea4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc3816f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39044/0 (socket says 172.21.15.105:39044) 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb864ea4640 1 -- 172.21.15.105:0/2716784041 learned_addr learned my addr 172.21.15.105:0/2716784041 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.739+0000 7fb865ea6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 0x5653dc388730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb864ea4640 1 -- 172.21.15.105:0/2716784041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 msgr2=0x5653dc388730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:24.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb864ea4640 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 0x5653dc388730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb864ea4640 1 -- 172.21.15.105:0/2716784041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30a5d0 msgr2=0x5653dc3811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb864ea4640 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30a5d0 0x5653dc3811b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb864ea4640 1 -- 172.21.15.105:0/2716784041 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dc388d20 con 0x5653dc30afc0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb8656a5640 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30a5d0 0x5653dc3811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb864ea4640 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc3816f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5653dc306a80 tx=0x7fb850009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3264a7fa14d93a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb848ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb848ff9640 1 -- 172.21.15.105:0/2716784041 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850027040 con 0x5653dc30afc0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb848ff9640 1 -- 172.21.15.105:0/2716784041 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb8500185b0 con 0x5653dc30afc0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2716784041 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85c09a160 con 0x5653dc30afc0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb848ff9640 1 -- 172.21.15.105:0/2716784041 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb85001e070 con 0x5653dc30afc0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb86a5d1740 10 client.26839 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2716784041 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5653dc38a030 con 0x5653dc30afc0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb848ff9640 1 -- 172.21.15.105:0/2716784041 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb85c09a160 con 0x5653dc30afc0 2024-03-26T17:42:24.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb848ff9640 1 -- 172.21.15.105:0/2716784041 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb850003680 con 0x5653dc30afc0 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb86a5d1740 10 client.26839 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb86a5d1740 10 client.26839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb86a5d1740 20 client.26839 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.740+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5653dc38a9a0 0x5653dc38cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.741+0000 7fb86a5d1740 4 client.26839 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.741+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2716784041 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653dc3141a0 con 0x5653dc38a9a0 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.741+0000 7fb8656a5640 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5653dc38a9a0 0x5653dc38cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.741+0000 7fb848ff9640 10 client.26839 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.741+0000 7fb8656a5640 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5653dc38a9a0 0x5653dc38cdc0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd765251d056913c server_cookie=ae1cbc030886a1a7 in_seq=0 out_seq=0 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb848ff9640 1 -- 172.21.15.105:0/2716784041 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653dc3141a0 con 0x5653dc38a9a0 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb848ff9640 10 client.26839 handle_command_reply: tid=0 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 client.26839 shutdown 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2716784041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 msgr2=0x5653dc3816f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc3816f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5653dc306a80 tx=0x7fb850009ee0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2716784041 shutdown_connections 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5653dc38a9a0 0x5653dc38cdc0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5653dc310130 0x5653dc388730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5653dc30afc0 0x5653dc3816f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 --2- 172.21.15.105:0/2716784041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5653dc30a5d0 0x5653dc3811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb848ff9640 0 client.26839 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2716784041 >> 172.21.15.105:0/2716784041 conn(0x5653dc305b40 msgr2=0x5653dc30b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2716784041 shutdown_connections 2024-03-26T17:42:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 1 -- 172.21.15.105:0/2716784041 wait complete. 2024-03-26T17:42:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.798+0000 7fb86a5d1740 20 client.26839 trim_cache size 0 max 16384 2024-03-26T17:42:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:42:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:42:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:42:24.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:42:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 -- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb84c0a3040 msgr2=0x7fb84c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb84c0a3040 0x7fb84c0a5500 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x5653dc30b820 tx=0x7fb85802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 -- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8601023a0 msgr2=0x7fb860197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:24.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8601023a0 0x7fb860197a50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00e9f0 tx=0x7fb85c00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 -- 172.21.15.105:0/3349184338 shutdown_connections 2024-03-26T17:42:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb84c0a3040 0x7fb84c0a5500 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb860107600 0x7fb86019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb860104cd0 0x7fb860197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 --2- 172.21.15.105:0/3349184338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8601023a0 0x7fb860197a50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 -- 172.21.15.105:0/3349184338 >> 172.21.15.105:0/3349184338 conn(0x7fb8600fbd50 msgr2=0x7fb860102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 -- 172.21.15.105:0/3349184338 shutdown_connections 2024-03-26T17:42:24.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:24.799+0000 7fb867909640 1 -- 172.21.15.105:0/3349184338 wait complete. 2024-03-26T17:42:25.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 110940 3.11 MB/sec execute 68 sec latency 35.139 ms 2024-03-26T17:42:26.021 INFO:tasks.workunit.client.0.smithi105.stdout: 1 111559 3.12 MB/sec execute 69 sec latency 16.477 ms 2024-03-26T17:42:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:25 smithi144 ceph-mon[36095]: pgmap v470: 97 pgs: 97 active+clean; 26 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-03-26T17:42:26.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:25 smithi105 ceph-mon[32280]: pgmap v470: 97 pgs: 97 active+clean; 26 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-03-26T17:42:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:25 smithi135 ceph-mon[35636]: pgmap v470: 97 pgs: 97 active+clean; 26 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-03-26T17:42:27.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 112223 3.09 MB/sec execute 70 sec latency 16.542 ms 2024-03-26T17:42:28.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 112923 3.14 MB/sec execute 71 sec latency 17.831 ms 2024-03-26T17:42:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:27 smithi144 ceph-mon[36095]: pgmap v471: 97 pgs: 97 active+clean; 27 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-03-26T17:42:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:27 smithi105 ceph-mon[32280]: pgmap v471: 97 pgs: 97 active+clean; 27 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-03-26T17:42:28.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:27 smithi135 ceph-mon[35636]: pgmap v471: 97 pgs: 97 active+clean; 27 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-03-26T17:42:29.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 113374 3.12 MB/sec execute 72 sec latency 21.195 ms 2024-03-26T17:42:30.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 113936 3.14 MB/sec execute 73 sec latency 15.947 ms 2024-03-26T17:42:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:29 smithi144 ceph-mon[36095]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:42:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:29 smithi105 ceph-mon[32280]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:42:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:29 smithi135 ceph-mon[35636]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:42:31.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 114416 3.11 MB/sec execute 74 sec latency 19.852 ms 2024-03-26T17:42:32.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 115049 3.13 MB/sec execute 75 sec latency 19.543 ms 2024-03-26T17:42:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:31 smithi144 ceph-mon[36095]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:42:32.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:31 smithi105 ceph-mon[32280]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:42:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:31 smithi135 ceph-mon[35636]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:42:33.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 115729 3.10 MB/sec execute 76 sec latency 15.777 ms 2024-03-26T17:42:34.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 116443 3.14 MB/sec execute 77 sec latency 16.647 ms 2024-03-26T17:42:34.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:33 smithi144 ceph-mon[36095]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:42:34.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:33 smithi105 ceph-mon[32280]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:42:34.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:33 smithi135 ceph-mon[35636]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:42:35.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 116937 3.13 MB/sec execute 78 sec latency 18.662 ms 2024-03-26T17:42:36.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 117506 3.15 MB/sec execute 79 sec latency 16.013 ms 2024-03-26T17:42:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:35 smithi144 ceph-mon[36095]: pgmap v475: 97 pgs: 97 active+clean; 28 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-03-26T17:42:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:35 smithi105 ceph-mon[32280]: pgmap v475: 97 pgs: 97 active+clean; 28 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-03-26T17:42:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:35 smithi135 ceph-mon[35636]: pgmap v475: 97 pgs: 97 active+clean; 28 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-03-26T17:42:37.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 117992 3.12 MB/sec execute 80 sec latency 26.501 ms 2024-03-26T17:42:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 -- 172.21.15.105:0/121751654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654107680 msgr2=0x7f3654109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 --2- 172.21.15.105:0/121751654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654107680 0x7f3654109a70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f364c009ca0 tx=0x7f364c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 -- 172.21.15.105:0/121751654 shutdown_connections 2024-03-26T17:42:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 --2- 172.21.15.105:0/121751654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654107680 0x7f3654109a70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 --2- 172.21.15.105:0/121751654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3654104d50 0x7f3654107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.887 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 --2- 172.21.15.105:0/121751654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3654102420 0x7f3654104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 -- 172.21.15.105:0/121751654 >> 172.21.15.105:0/121751654 conn(0x7f36540fbd50 msgr2=0x7f36540fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 -- 172.21.15.105:0/121751654 shutdown_connections 2024-03-26T17:42:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.895+0000 7f365b390640 1 -- 172.21.15.105:0/121751654 wait complete. 2024-03-26T17:42:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365b390640 1 Processor -- start 2024-03-26T17:42:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365b390640 1 -- start start 2024-03-26T17:42:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365b390640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3654102420 0x7f3654197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365b390640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654104d50 0x7f3654197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365b390640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3654107680 0x7f365419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365b390640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f365410d440 con 0x7f3654104d50 2024-03-26T17:42:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365b390640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f365410d2c0 con 0x7f3654107680 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365b390640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f365410d5c0 con 0x7f3654102420 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654104d50 0x7f3654197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654104d50 0x7f3654197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54660/0 (socket says 172.21.15.105:54660) 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 -- 172.21.15.105:0/2512499384 learned_addr learned my addr 172.21.15.105:0/2512499384 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365912c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3654102420 0x7f3654197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365992d640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3654107680 0x7f365419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 -- 172.21.15.105:0/2512499384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3654102420 msgr2=0x7f3654197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3654102420 0x7f3654197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 -- 172.21.15.105:0/2512499384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3654107680 msgr2=0x7f365419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3654107680 0x7f365419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 -- 172.21.15.105:0/2512499384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365419f6b0 con 0x7f3654104d50 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365912c640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3654102420 0x7f3654197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365992d640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3654107680 0x7f365419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.896+0000 7f365892b640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654104d50 0x7f3654197fd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f364400e9f0 tx=0x7f364400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.897+0000 7f364a7fc640 1 -- 172.21.15.105:0/2512499384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364400ccc0 con 0x7f3654104d50 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.897+0000 7f365b390640 1 -- 172.21.15.105:0/2512499384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365419f9a0 con 0x7f3654104d50 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.897+0000 7f364a7fc640 1 -- 172.21.15.105:0/2512499384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36440054d0 con 0x7f3654104d50 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.897+0000 7f364a7fc640 1 -- 172.21.15.105:0/2512499384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644010430 con 0x7f3654104d50 2024-03-26T17:42:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.897+0000 7f365b390640 1 -- 172.21.15.105:0/2512499384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365419fef0 con 0x7f3654104d50 2024-03-26T17:42:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.899+0000 7f364a7fc640 1 -- 172.21.15.105:0/2512499384 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3644005020 con 0x7f3654104d50 2024-03-26T17:42:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365df80740 1 Processor -- start 2024-03-26T17:42:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365df80740 1 -- start start 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365df80740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb5385d0 0x55edfb5389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365df80740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb538fc0 0x55edfb53dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365df80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb5405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365df80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55edfb49d0b0 con 0x55edfb53e130 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f364a7fc640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36400a2fc0 0x7f36400a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365df80740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55edfb445960 con 0x55edfb5385d0 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365df80740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55edfb499060 con 0x55edfb538fc0 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365992d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb5385d0 0x55edfb5389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365912c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb538fc0 0x55edfb53dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365912c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb538fc0 0x55edfb53dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52804/0 (socket says 172.21.15.105:52804) 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365912c640 1 -- 172.21.15.105:0/1142400638 learned_addr learned my addr 172.21.15.105:0/1142400638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365992d640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36400a2fc0 0x7f36400a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365892b640 1 --2- 172.21.15.105:0/1142400638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb5405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f364a7fc640 1 -- 172.21.15.105:0/2512499384 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f36440cf7a0 con 0x7f3654104d50 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365912c640 1 -- 172.21.15.105:0/1142400638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb5385d0 msgr2=0x55edfb5389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365912c640 1 --2- 172.21.15.105:0/1142400638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb5385d0 0x55edfb5389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365912c640 1 -- 172.21.15.105:0/1142400638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 msgr2=0x55edfb5405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365912c640 1 --2- 172.21.15.105:0/1142400638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb5405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.900+0000 7f365912c640 1 -- 172.21.15.105:0/1142400638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfb457c60 con 0x55edfb538fc0 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.901+0000 7f365912c640 1 --2- 172.21.15.105:0/1142400638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb538fc0 0x55edfb53dac0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3650011540 tx=0x7f3650011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ea151d12781a532 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.901+0000 7f365992d640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36400a2fc0 0x7f36400a5480 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f364c006010 tx=0x7f364c0324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.901+0000 7f363f7fe640 1 -- 172.21.15.105:0/1142400638 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365000f640 con 0x55edfb538fc0 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.901+0000 7f363f7fe640 1 -- 172.21.15.105:0/1142400638 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f365000fc90 con 0x55edfb538fc0 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.901+0000 7f363f7fe640 1 -- 172.21.15.105:0/1142400638 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650019c10 con 0x55edfb538fc0 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 -- 172.21.15.105:0/1142400638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb538fc0 msgr2=0x55edfb53dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 --2- 172.21.15.105:0/1142400638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb538fc0 0x55edfb53dac0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3650011540 tx=0x7f3650011a00 comp rx=0 tx=0).stop 2024-03-26T17:42:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 -- 172.21.15.105:0/1142400638 shutdown_connections 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 --2- 172.21.15.105:0/1142400638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb5405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 --2- 172.21.15.105:0/1142400638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb538fc0 0x55edfb53dac0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 --2- 172.21.15.105:0/1142400638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb5385d0 0x55edfb5389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 -- 172.21.15.105:0/1142400638 >> 172.21.15.105:0/1142400638 conn(0x55edfb533b40 msgr2=0x55edfb533f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 -- 172.21.15.105:0/1142400638 shutdown_connections 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 -- 172.21.15.105:0/1142400638 wait complete. 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 -- start start 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 Processor -- start 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5423f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.902+0000 7f365df80740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb542930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365df80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb549900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365df80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55edfb5410e0 con 0x55edfb53e130 2024-03-26T17:42:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365df80740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55edfb458db0 con 0x55edfb538fc0 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365df80740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55edfb541260 con 0x55edfb5385d0 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365992d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb549900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365912c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365df80740 10 client.0 _ll_register_callbacks cb 0x55edfb45d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365912c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52814/0 (socket says 172.21.15.105:52814) 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365912c640 1 -- 172.21.15.105:0/2902009806 learned_addr learned my addr 172.21.15.105:0/2902009806 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365892b640 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb542930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365992d640 1 -- 172.21.15.105:0/2902009806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 msgr2=0x55edfb5423f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365992d640 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5423f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365992d640 1 -- 172.21.15.105:0/2902009806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 msgr2=0x55edfb542930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365992d640 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb542930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365992d640 1 -- 172.21.15.105:0/2902009806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfb1b1210 con 0x55edfb53e130 2024-03-26T17:42:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365912c640 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365992d640 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb549900 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f364c02c630 tx=0x7f364c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbc001a78696b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f363cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f363cff9640 1 -- 172.21.15.105:0/2902009806 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c039040 con 0x55edfb53e130 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f363cff9640 1 -- 172.21.15.105:0/2902009806 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f364c042620 con 0x55edfb53e130 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.903+0000 7f365df80740 1 -- 172.21.15.105:0/2902009806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365000fc90 con 0x55edfb53e130 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f363cff9640 1 -- 172.21.15.105:0/2902009806 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f364c042820 con 0x55edfb53e130 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365df80740 10 client.17010 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365df80740 1 -- 172.21.15.105:0/2902009806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edfb27d620 con 0x55edfb53e130 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f363cff9640 1 -- 172.21.15.105:0/2902009806 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f365000fc90 con 0x55edfb53e130 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f363cff9640 1 -- 172.21.15.105:0/2902009806 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f364c044a40 con 0x55edfb53e130 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365df80740 10 client.17010 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365df80740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365df80740 20 client.17010 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365df80740 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55edfb54e2a0 0x55edfb5506d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365df80740 4 client.17010 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365df80740 1 -- 172.21.15.105:0/2902009806 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edfb4aa5f0 con 0x55edfb54e2a0 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.904+0000 7f365912c640 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55edfb54e2a0 0x55edfb5506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.905+0000 7f363cff9640 10 client.17010 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.905+0000 7f365912c640 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55edfb54e2a0 0x55edfb5506d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2733f18b699ee056 server_cookie=b1407b223ba9027b in_seq=0 out_seq=0 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f363cff9640 1 -- 172.21.15.105:0/2902009806 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55edfb4aa5f0 con 0x55edfb54e2a0 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f363cff9640 10 client.17010 handle_command_reply: tid=0 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 client.17010 shutdown 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 -- 172.21.15.105:0/2902009806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 msgr2=0x55edfb549900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb549900 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f364c02c630 tx=0x7f364c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 -- 172.21.15.105:0/2902009806 shutdown_connections 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55edfb54e2a0 0x55edfb5506d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb549900 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f363cff9640 0 client.17010 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb542930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 --2- 172.21.15.105:0/2902009806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 -- 172.21.15.105:0/2902009806 >> 172.21.15.105:0/2902009806 conn(0x55edfb533b40 msgr2=0x55edfb532df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 -- 172.21.15.105:0/2902009806 shutdown_connections 2024-03-26T17:42:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 1 -- 172.21.15.105:0/2902009806 wait complete. 2024-03-26T17:42:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.906+0000 7f365df80740 20 client.17010 trim_cache size 0 max 16384 2024-03-26T17:42:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365df80740 1 Processor -- start 2024-03-26T17:42:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365df80740 1 -- start start 2024-03-26T17:42:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365df80740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb53a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365df80740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb5498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365df80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb532560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365df80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36541a8ce0 con 0x55edfb53e130 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365df80740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36541aa820 con 0x55edfb538fc0 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365df80740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36541a93b0 con 0x55edfb5385d0 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365992d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb532560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365912c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb53a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365992d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb532560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:54692/0 (socket says 172.21.15.105:54692) 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365992d640 1 -- 172.21.15.105:0/2548072285 learned_addr learned my addr 172.21.15.105:0/2548072285 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.922+0000 7f365892b640 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb5498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f365912c640 1 -- 172.21.15.105:0/2548072285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 msgr2=0x55edfb5498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f365912c640 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb5498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f365912c640 1 -- 172.21.15.105:0/2548072285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 msgr2=0x55edfb532560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f365912c640 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb532560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f365912c640 1 -- 172.21.15.105:0/2548072285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfb5393a0 con 0x55edfb5385d0 2024-03-26T17:42:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f365992d640 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb532560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f365892b640 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb5498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f365912c640 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb53a140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f36500097b0 tx=0x7f3650005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec26d104a0317d2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f363effd640 1 -- 172.21.15.105:0/2548072285 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650032070 con 0x55edfb5385d0 2024-03-26T17:42:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f363effd640 1 -- 172.21.15.105:0/2548072285 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3650005040 con 0x55edfb5385d0 2024-03-26T17:42:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.923+0000 7f363effd640 1 -- 172.21.15.105:0/2548072285 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650019720 con 0x55edfb5385d0 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 -- 172.21.15.105:0/2548072285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 msgr2=0x55edfb53a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb53a140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f36500097b0 tx=0x7f3650005850 comp rx=0 tx=0).stop 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 -- 172.21.15.105:0/2548072285 shutdown_connections 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb53e130 0x55edfb532560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb538fc0 0x55edfb5498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 --2- 172.21.15.105:0/2548072285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb53a140 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 -- 172.21.15.105:0/2548072285 >> 172.21.15.105:0/2548072285 conn(0x55edfb533b40 msgr2=0x55edfb539a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 -- 172.21.15.105:0/2548072285 shutdown_connections 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 -- 172.21.15.105:0/2548072285 wait complete. 2024-03-26T17:42:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 -- start start 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.924+0000 7f365df80740 1 Processor -- start 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365df80740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365df80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb538fc0 0x55edfb547700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365df80740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb53e130 0x55edfb548670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365df80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f36541aa150 con 0x55edfb538fc0 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365df80740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f36541a9a80 con 0x55edfb53e130 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365df80740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36541a6cc0 con 0x55edfb5385d0 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365df80740 10 client.0 _ll_register_callbacks cb 0x7f3644005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52836/0 (socket says 172.21.15.105:52836) 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 -- 172.21.15.105:0/2802863490 learned_addr learned my addr 172.21.15.105:0/2802863490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365892b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb538fc0 0x55edfb547700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365992d640 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb53e130 0x55edfb548670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 -- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb53e130 msgr2=0x55edfb548670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb53e130 0x55edfb548670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 -- 172.21.15.105:0/2802863490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb538fc0 msgr2=0x55edfb547700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb538fc0 0x55edfb547700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 -- 172.21.15.105:0/2802863490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfb548c60 con 0x55edfb5385d0 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365992d640 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb53e130 0x55edfb548670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365892b640 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb538fc0 0x55edfb547700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f365912c640 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5471c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3650005c20 tx=0x7f3650009cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e46f6fbd0e8229e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f363cff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.925+0000 7f363cff9640 1 -- 172.21.15.105:0/2802863490 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650032070 con 0x55edfb5385d0 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 1 -- 172.21.15.105:0/2802863490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3650005040 con 0x55edfb5385d0 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f363cff9640 1 -- 172.21.15.105:0/2802863490 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f365002a040 con 0x55edfb5385d0 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f363cff9640 1 -- 172.21.15.105:0/2802863490 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f365000fa70 con 0x55edfb5385d0 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 10 client.26857 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 1 -- 172.21.15.105:0/2802863490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edfb5b4790 con 0x55edfb5385d0 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f363cff9640 1 -- 172.21.15.105:0/2802863490 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3650005040 con 0x55edfb5385d0 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f363cff9640 1 -- 172.21.15.105:0/2802863490 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3650019b60 con 0x55edfb5385d0 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 10 client.26857 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 10 client.26857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 20 client.26857 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55edfb5b4f00 0x55edfb5b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 4 client.26857 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:42:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365df80740 1 -- 172.21.15.105:0/2802863490 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edfb4a4940 con 0x55edfb5b4f00 2024-03-26T17:42:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.926+0000 7f365892b640 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55edfb5b4f00 0x55edfb5b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.927+0000 7f363cff9640 10 client.26857 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.927+0000 7f365892b640 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55edfb5b4f00 0x55edfb5b7320 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290c30411a37e331 server_cookie=d0da77852a53022a in_seq=0 out_seq=0 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f363cff9640 1 -- 172.21.15.105:0/2802863490 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edfb4a4940 con 0x55edfb5b4f00 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f363cff9640 10 client.26857 handle_command_reply: tid=0 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 client.26857 shutdown 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 -- 172.21.15.105:0/2802863490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 msgr2=0x55edfb5471c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5471c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3650005c20 tx=0x7f3650009cc0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 -- 172.21.15.105:0/2802863490 shutdown_connections 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55edfb5b4f00 0x55edfb5b7320 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edfb53e130 0x55edfb548670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edfb538fc0 0x55edfb547700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f363cff9640 0 client.26857 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 --2- 172.21.15.105:0/2802863490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edfb5385d0 0x55edfb5471c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.971+0000 7f365df80740 1 -- 172.21.15.105:0/2802863490 >> 172.21.15.105:0/2802863490 conn(0x55edfb533b40 msgr2=0x55edfb537630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365df80740 1 -- 172.21.15.105:0/2802863490 shutdown_connections 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365df80740 1 -- 172.21.15.105:0/2802863490 wait complete. 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365df80740 20 client.26857 trim_cache size 0 max 16384 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:42:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:42:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 -- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36400a2fc0 msgr2=0x7f36400a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36400a2fc0 0x7f36400a5480 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f364c006010 tx=0x7f364c0324f0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 -- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654104d50 msgr2=0x7f3654197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654104d50 0x7f3654197fd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f364400e9f0 tx=0x7f364400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 -- 172.21.15.105:0/2512499384 shutdown_connections 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36400a2fc0 0x7f36400a5480 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3654107680 0x7f365419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3654104d50 0x7f3654197fd0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 --2- 172.21.15.105:0/2512499384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3654102420 0x7f3654197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 -- 172.21.15.105:0/2512499384 >> 172.21.15.105:0/2512499384 conn(0x7f36540fbd50 msgr2=0x7f3654102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 -- 172.21.15.105:0/2512499384 shutdown_connections 2024-03-26T17:42:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:37.972+0000 7f365b390640 1 -- 172.21.15.105:0/2512499384 wait complete. 2024-03-26T17:42:38.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 118596 3.13 MB/sec execute 81 sec latency 29.519 ms 2024-03-26T17:42:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:37 smithi144 ceph-mon[36095]: pgmap v476: 97 pgs: 97 active+clean; 30 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:42:38.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:37 smithi105 ceph-mon[32280]: pgmap v476: 97 pgs: 97 active+clean; 30 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:42:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:37 smithi135 ceph-mon[35636]: pgmap v476: 97 pgs: 97 active+clean; 30 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:42:39.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 119275 3.10 MB/sec execute 82 sec latency 15.438 ms 2024-03-26T17:42:40.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 119985 3.14 MB/sec execute 83 sec latency 16.857 ms 2024-03-26T17:42:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:39 smithi144 ceph-mon[36095]: pgmap v477: 97 pgs: 97 active+clean; 26 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:42:40.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:39 smithi105 ceph-mon[32280]: pgmap v477: 97 pgs: 97 active+clean; 26 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:42:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:39 smithi135 ceph-mon[35636]: pgmap v477: 97 pgs: 97 active+clean; 26 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:42:41.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 120466 3.13 MB/sec execute 84 sec latency 21.742 ms 2024-03-26T17:42:42.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 121031 3.15 MB/sec execute 85 sec latency 16.797 ms 2024-03-26T17:42:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:41 smithi144 ceph-mon[36095]: pgmap v478: 97 pgs: 97 active+clean; 26 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:42:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:41 smithi105 ceph-mon[32280]: pgmap v478: 97 pgs: 97 active+clean; 26 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:42:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:41 smithi135 ceph-mon[35636]: pgmap v478: 97 pgs: 97 active+clean; 26 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:42:43.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 121524 3.12 MB/sec execute 86 sec latency 20.972 ms 2024-03-26T17:42:44.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 122152 3.14 MB/sec execute 87 sec latency 17.948 ms 2024-03-26T17:42:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:43 smithi144 ceph-mon[36095]: pgmap v479: 97 pgs: 97 active+clean; 27 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:42:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:43 smithi105 ceph-mon[32280]: pgmap v479: 97 pgs: 97 active+clean; 27 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:42:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:43 smithi135 ceph-mon[35636]: pgmap v479: 97 pgs: 97 active+clean; 27 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-03-26T17:42:45.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 122818 3.11 MB/sec execute 88 sec latency 16.130 ms 2024-03-26T17:42:46.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 123547 3.15 MB/sec execute 89 sec latency 16.137 ms 2024-03-26T17:42:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:45 smithi144 ceph-mon[36095]: pgmap v480: 97 pgs: 97 active+clean; 27 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:42:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:45 smithi105 ceph-mon[32280]: pgmap v480: 97 pgs: 97 active+clean; 27 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:42:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:45 smithi135 ceph-mon[35636]: pgmap v480: 97 pgs: 97 active+clean; 27 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:42:47.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 124021 3.14 MB/sec execute 90 sec latency 19.382 ms 2024-03-26T17:42:48.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 124584 3.15 MB/sec execute 91 sec latency 15.491 ms 2024-03-26T17:42:48.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:47 smithi144 ceph-mon[36095]: pgmap v481: 97 pgs: 97 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-03-26T17:42:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:47 smithi105 ceph-mon[32280]: pgmap v481: 97 pgs: 97 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-03-26T17:42:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:47 smithi135 ceph-mon[35636]: pgmap v481: 97 pgs: 97 active+clean; 29 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-03-26T17:42:49.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 125063 3.13 MB/sec execute 92 sec latency 26.375 ms 2024-03-26T17:42:49.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:48 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:42:49.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:48 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:42:49.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:48 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:42:50.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 125675 3.14 MB/sec execute 93 sec latency 19.375 ms 2024-03-26T17:42:50.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:42:50.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:42:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:49 smithi144 ceph-mon[36095]: pgmap v482: 97 pgs: 97 active+clean; 30 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:42:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:42:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:42:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:42:50.116 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:49 smithi105 ceph-mon[32280]: pgmap v482: 97 pgs: 97 active+clean; 30 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:42:50.117 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:42:50.117 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:42:50.117 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:42:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:49 smithi135 ceph-mon[35636]: pgmap v482: 97 pgs: 97 active+clean; 30 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:42:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:42:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:42:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 -- 172.21.15.105:0/3213410211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b41007c0 msgr2=0x7f00b4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 --2- 172.21.15.105:0/3213410211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b41007c0 0x7f00b4100be0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009900 tx=0x7f00b002bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 -- 172.21.15.105:0/3213410211 shutdown_connections 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 --2- 172.21.15.105:0/3213410211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b4102370 0x7f00b4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 --2- 172.21.15.105:0/3213410211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00b41019b0 0x7f00b4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 --2- 172.21.15.105:0/3213410211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b41007c0 0x7f00b4100be0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 -- 172.21.15.105:0/3213410211 >> 172.21.15.105:0/3213410211 conn(0x7f00b40fbd50 msgr2=0x7f00b40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 -- 172.21.15.105:0/3213410211 shutdown_connections 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 -- 172.21.15.105:0/3213410211 wait complete. 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 Processor -- start 2024-03-26T17:42:50.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.216+0000 7f00bc590640 1 -- start start 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bc590640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00b41007c0 0x7f00b4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bc590640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b41019b0 0x7f00b419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bc590640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b4102370 0x7f00b419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bc590640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00b410c2e0 con 0x7f00b4102370 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bc590640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00b410c160 con 0x7f00b41019b0 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bc590640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00b410c460 con 0x7f00b41007c0 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b4102370 0x7f00b419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00b9b2b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b41019b0 0x7f00b419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b4102370 0x7f00b419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46578/0 (socket says 172.21.15.105:46578) 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00ba32c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00b41007c0 0x7f00b4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 -- 172.21.15.105:0/3731853178 learned_addr learned my addr 172.21.15.105:0/3731853178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00b9b2b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b41019b0 0x7f00b419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53280/0 (socket says 172.21.15.105:53280) 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 -- 172.21.15.105:0/3731853178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00b41007c0 msgr2=0x7f00b4069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00b41007c0 0x7f00b4069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 -- 172.21.15.105:0/3731853178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b41019b0 msgr2=0x7f00b419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b41019b0 0x7f00b419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 -- 172.21.15.105:0/3731853178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b419f800 con 0x7f00b4102370 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00ba32c640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00b41007c0 0x7f00b4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bab2d640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b4102370 0x7f00b419eef0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00eb10 tx=0x7f00ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00ab7fe640 1 -- 172.21.15.105:0/3731853178 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac004b20 con 0x7f00b4102370 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00bc590640 1 -- 172.21.15.105:0/3731853178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b419fa90 con 0x7f00b4102370 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00ab7fe640 1 -- 172.21.15.105:0/3731853178 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00ac004cc0 con 0x7f00b4102370 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.217+0000 7f00ab7fe640 1 -- 172.21.15.105:0/3731853178 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac010690 con 0x7f00b4102370 2024-03-26T17:42:50.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.218+0000 7f00bc590640 1 -- 172.21.15.105:0/3731853178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b419ff90 con 0x7f00b4102370 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.220+0000 7f00ab7fe640 1 -- 172.21.15.105:0/3731853178 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f00ac005020 con 0x7f00b4102370 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.220+0000 7f00bf17d740 1 Processor -- start 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.220+0000 7f00bf17d740 1 -- start start 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b83de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00bf17d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565146ae3980 con 0x565146b7f2e0 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00bf17d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565146a8bbe0 con 0x565146b7e8f0 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00bf17d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565146adf930 con 0x565146b84450 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00ba32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b83de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00b9b2b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00ba32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b83de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46582/0 (socket says 172.21.15.105:46582) 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00ba32c640 1 -- 172.21.15.105:0/572678572 learned_addr learned my addr 172.21.15.105:0/572678572 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/572678572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00b9b2b640 1 -- 172.21.15.105:0/572678572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 msgr2=0x565146b868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/572678572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b868e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00b9b2b640 1 -- 172.21.15.105:0/572678572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 msgr2=0x565146b83de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00ab7fe640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00a00a2f20 0x7f00a00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/572678572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b83de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00b9b2b640 1 -- 172.21.15.105:0/572678572 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651468c17d0 con 0x565146b7e8f0 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00bab2d640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00a00a2f20 0x7f00a00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/572678572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7ed10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f00b4069300 tx=0x7f00a4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c7f2f23b211c54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.221+0000 7f00ab7fe640 1 -- 172.21.15.105:0/3731853178 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f00ac0cf980 con 0x7f00b4102370 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.222+0000 7f00a8bfe640 1 -- 172.21.15.105:0/572678572 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4004050 con 0x565146b7e8f0 2024-03-26T17:42:50.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.222+0000 7f00a8bfe640 1 -- 172.21.15.105:0/572678572 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00a400fc70 con 0x565146b7e8f0 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.222+0000 7f00a8bfe640 1 -- 172.21.15.105:0/572678572 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a40038e0 con 0x565146b7e8f0 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.222+0000 7f00bf17d740 1 -- 172.21.15.105:0/572678572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 msgr2=0x565146b7ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.222+0000 7f00bf17d740 1 --2- 172.21.15.105:0/572678572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7ed10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f00b4069300 tx=0x7f00a4004730 comp rx=0 tx=0).stop 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.222+0000 7f00bab2d640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00a00a2f20 0x7f00a00a53e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0984d0 tx=0x7f00ac0bc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 -- 172.21.15.105:0/572678572 shutdown_connections 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 --2- 172.21.15.105:0/572678572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 --2- 172.21.15.105:0/572678572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b83de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 --2- 172.21.15.105:0/572678572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7ed10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 -- 172.21.15.105:0/572678572 >> 172.21.15.105:0/572678572 conn(0x565146b79e60 msgr2=0x565146b7a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 -- 172.21.15.105:0/572678572 shutdown_connections 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 -- 172.21.15.105:0/572678572 wait complete. 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 -- start start 2024-03-26T17:42:50.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 Processor -- start 2024-03-26T17:42:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.223+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b88b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b7f2e0 0x565146b890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b84450 0x565146b900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00bf17d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565146b87400 con 0x565146b84450 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00bf17d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565146a9f160 con 0x565146b7e8f0 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00bf17d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565146b87580 con 0x565146b7f2e0 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00bf17d740 10 client.0 _ll_register_callbacks cb 0x565146adb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b84450 0x565146b900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b88b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b88b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53300/0 (socket says 172.21.15.105:53300) 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00ba32c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b7f2e0 0x565146b890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 -- 172.21.15.105:0/2479583868 learned_addr learned my addr 172.21.15.105:0/2479583868 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 -- 172.21.15.105:0/2479583868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b7f2e0 msgr2=0x565146b890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b7f2e0 0x565146b890a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 -- 172.21.15.105:0/2479583868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b84450 msgr2=0x565146b900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b84450 0x565146b900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 -- 172.21.15.105:0/2479583868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651467f73f0 con 0x565146b7e8f0 2024-03-26T17:42:50.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00ba32c640 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b7f2e0 0x565146b890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b88b60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f00a40097a0 tx=0x7f00a4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e70bf11be5e348d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f009dffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f009dffb640 1 -- 172.21.15.105:0/2479583868 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4023020 con 0x565146b7e8f0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f009dffb640 1 -- 172.21.15.105:0/2479583868 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f00a4003a80 con 0x565146b7e8f0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.224+0000 7f00bf17d740 1 -- 172.21.15.105:0/2479583868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a400fc70 con 0x565146b7e8f0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f009dffb640 1 -- 172.21.15.105:0/2479583868 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00a4036050 con 0x565146b7e8f0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00bf17d740 10 client.26870 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00bf17d740 1 -- 172.21.15.105:0/2479583868 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5651467b70f0 con 0x565146b7e8f0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f009dffb640 1 -- 172.21.15.105:0/2479583868 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00a400fc70 con 0x565146b7e8f0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f009dffb640 1 -- 172.21.15.105:0/2479583868 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f00a401f8c0 con 0x565146b7e8f0 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00bf17d740 10 client.26870 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00bf17d740 10 client.26870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00bf17d740 20 client.26870 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565146b94620 0x565146b96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00bf17d740 4 client.26870 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00bf17d740 1 -- 172.21.15.105:0/2479583868 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565146af0ec0 con 0x565146b94620 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.225+0000 7f00ba32c640 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565146b94620 0x565146b96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.226+0000 7f00ba32c640 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565146b94620 0x565146b96a10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3589def986f4dad6 server_cookie=22f5980047c69728 in_seq=0 out_seq=0 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.226+0000 7f009dffb640 10 client.26870 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f009dffb640 1 -- 172.21.15.105:0/2479583868 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x565146af0ec0 con 0x565146b94620 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f009dffb640 10 client.26870 handle_command_reply: tid=0 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 client.26870 shutdown 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 -- 172.21.15.105:0/2479583868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 msgr2=0x565146b88b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b88b60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f00a40097a0 tx=0x7f00a4005f90 comp rx=0 tx=0).stop 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 -- 172.21.15.105:0/2479583868 shutdown_connections 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565146b94620 0x565146b96a10 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b84450 0x565146b900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b7f2e0 0x565146b890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2479583868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b88b60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 -- 172.21.15.105:0/2479583868 >> 172.21.15.105:0/2479583868 conn(0x565146b79e60 msgr2=0x565146b79080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f009dffb640 0 client.26870 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 -- 172.21.15.105:0/2479583868 shutdown_connections 2024-03-26T17:42:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 1 -- 172.21.15.105:0/2479583868 wait complete. 2024-03-26T17:42:50.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.227+0000 7f00bf17d740 20 client.26870 trim_cache size 0 max 16384 2024-03-26T17:42:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.253+0000 7f00bf17d740 1 Processor -- start 2024-03-26T17:42:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.253+0000 7f00bf17d740 1 -- start start 2024-03-26T17:42:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.253+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.253+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b90070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.253+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b88e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.253+0000 7f00bf17d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00b41a8e00 con 0x565146b7f2e0 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.253+0000 7f00bf17d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00b41aa940 con 0x565146b7e8f0 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.253+0000 7f00bf17d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00b41a94d0 con 0x565146b84450 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b90070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00b9b2b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b90070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46602/0 (socket says 172.21.15.105:46602) 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 -- 172.21.15.105:0/2058295416 learned_addr learned my addr 172.21.15.105:0/2058295416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/2058295416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b88e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 -- 172.21.15.105:0/2058295416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 msgr2=0x565146b88e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 --2- 172.21.15.105:0/2058295416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b88e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 -- 172.21.15.105:0/2058295416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 msgr2=0x565146b7a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 --2- 172.21.15.105:0/2058295416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 -- 172.21.15.105:0/2058295416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565146b80c00 con 0x565146b7f2e0 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f00ba32c640 1 --2- 172.21.15.105:0/2058295416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b90070 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f00b002c170 tx=0x7f00b0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df8c31635bdaadf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f009ffff640 1 -- 172.21.15.105:0/2058295416 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b004f020 con 0x565146b7f2e0 2024-03-26T17:42:50.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f009ffff640 1 -- 172.21.15.105:0/2058295416 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00b0004490 con 0x565146b7f2e0 2024-03-26T17:42:50.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.254+0000 7f009ffff640 1 -- 172.21.15.105:0/2058295416 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00051c0 con 0x565146b7f2e0 2024-03-26T17:42:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 -- 172.21.15.105:0/2058295416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 msgr2=0x565146b90070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2058295416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b90070 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f00b002c170 tx=0x7f00b0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 -- 172.21.15.105:0/2058295416 shutdown_connections 2024-03-26T17:42:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2058295416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b88e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2058295416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b90070 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 --2- 172.21.15.105:0/2058295416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b7a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 -- 172.21.15.105:0/2058295416 >> 172.21.15.105:0/2058295416 conn(0x565146b79e60 msgr2=0x565146b88770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 -- 172.21.15.105:0/2058295416 shutdown_connections 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 -- 172.21.15.105:0/2058295416 wait complete. 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.255+0000 7f00bf17d740 1 -- start start 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00bf17d740 1 Processor -- start 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b8d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b8d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00bf17d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b8e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00bf17d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00b41aa270 con 0x565146b7f2e0 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00bf17d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00b41a9ba0 con 0x565146b7e8f0 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00bf17d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f00b41a6de0 con 0x565146b84450 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00bf17d740 10 client.0 _ll_register_callbacks cb 0x7f00a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b8d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00b9b2b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b8d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b8d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46604/0 (socket says 172.21.15.105:46604) 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 -- 172.21.15.105:0/3677402507 learned_addr learned my addr 172.21.15.105:0/3677402507 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b8e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 -- 172.21.15.105:0/3677402507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 msgr2=0x565146b8e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b8e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 -- 172.21.15.105:0/3677402507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 msgr2=0x565146b8d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b8d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 -- 172.21.15.105:0/3677402507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565146b8eda0 con 0x565146b7f2e0 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b8d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f00ba32c640 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b8d860 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009850 tx=0x7f00b004b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3299299d8092573a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.256+0000 7f009dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f009dffb640 1 -- 172.21.15.105:0/3677402507 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b004bcc0 con 0x565146b7f2e0 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f009dffb640 1 -- 172.21.15.105:0/3677402507 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f00b0005140 con 0x565146b7f2e0 2024-03-26T17:42:50.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 1 -- 172.21.15.105:0/3677402507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b0004490 con 0x565146b7f2e0 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f009dffb640 1 -- 172.21.15.105:0/3677402507 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00b0005340 con 0x565146b7f2e0 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 10 client.17034 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 1 -- 172.21.15.105:0/3677402507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565146b900e0 con 0x565146b7f2e0 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f009dffb640 1 -- 172.21.15.105:0/3677402507 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00b0004490 con 0x565146b7f2e0 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f009dffb640 1 -- 172.21.15.105:0/3677402507 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f00b004a030 con 0x565146b7f2e0 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 10 client.17034 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 20 client.17034 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565146bfacf0 0x565146bfd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 4 client.17034 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00bf17d740 1 -- 172.21.15.105:0/3677402507 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565146aeb210 con 0x565146bfacf0 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.257+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565146bfacf0 0x565146bfd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.258+0000 7f009dffb640 10 client.17034 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.258+0000 7f00b9b2b640 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565146bfacf0 0x565146bfd110 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cc8cc4d72ceed1 server_cookie=c76fe127e691eaa0 in_seq=0 out_seq=0 2024-03-26T17:42:50.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f009dffb640 1 -- 172.21.15.105:0/3677402507 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565146aeb210 con 0x565146bfacf0 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f009dffb640 10 client.17034 handle_command_reply: tid=0 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 client.17034 shutdown 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 -- 172.21.15.105:0/3677402507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 msgr2=0x565146b8d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b8d860 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009850 tx=0x7f00b004b840 comp rx=0 tx=0).stop 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 -- 172.21.15.105:0/3677402507 shutdown_connections 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565146bfacf0 0x565146bfd110 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565146b84450 0x565146b8e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565146b7f2e0 0x565146b8d860 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 --2- 172.21.15.105:0/3677402507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565146b7e8f0 0x565146b8d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 -- 172.21.15.105:0/3677402507 >> 172.21.15.105:0/3677402507 conn(0x565146b79e60 msgr2=0x565146b7d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f009dffb640 0 client.17034 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 -- 172.21.15.105:0/3677402507 shutdown_connections 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 1 -- 172.21.15.105:0/3677402507 wait complete. 2024-03-26T17:42:50.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.259+0000 7f00bf17d740 20 client.17034 trim_cache size 0 max 16384 2024-03-26T17:42:50.257 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:42:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:42:50.257 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:42:50.257 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 -- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00a00a2f20 msgr2=0x7f00a00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00a00a2f20 0x7f00a00a53e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0984d0 tx=0x7f00ac0bc040 comp rx=0 tx=0).stop 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 -- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b4102370 msgr2=0x7f00b419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b4102370 0x7f00b419eef0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00eb10 tx=0x7f00ac00c2d0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 -- 172.21.15.105:0/3731853178 shutdown_connections 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00a00a2f20 0x7f00a00a53e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00b4102370 0x7f00b419eef0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00b41019b0 0x7f00b419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 --2- 172.21.15.105:0/3731853178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00b41007c0 0x7f00b4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 -- 172.21.15.105:0/3731853178 >> 172.21.15.105:0/3731853178 conn(0x7f00b40fbd50 msgr2=0x7f00b4104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 -- 172.21.15.105:0/3731853178 shutdown_connections 2024-03-26T17:42:50.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.260+0000 7f00bc590640 1 -- 172.21.15.105:0/3731853178 wait complete. 2024-03-26T17:42:50.264 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:42:50.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:42:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 -- 172.21.15.105:0/1344637135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 msgr2=0x7f6b14109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 --2- 172.21.15.105:0/1344637135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 0x7f6b14109a50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009ca0 tx=0x7f6b0c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 -- 172.21.15.105:0/1344637135 shutdown_connections 2024-03-26T17:42:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 --2- 172.21.15.105:0/1344637135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 0x7f6b14109a50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 --2- 172.21.15.105:0/1344637135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14104d30 0x7f6b14107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 --2- 172.21.15.105:0/1344637135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b14102400 0x7f6b141047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 -- 172.21.15.105:0/1344637135 >> 172.21.15.105:0/1344637135 conn(0x7f6b140fbd50 msgr2=0x7f6b140fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 -- 172.21.15.105:0/1344637135 shutdown_connections 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.424+0000 7f6b19881640 1 -- 172.21.15.105:0/1344637135 wait complete. 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b19881640 1 Processor -- start 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b19881640 1 -- start start 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b19881640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14102400 0x7f6b14197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b19881640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b14104d30 0x7f6b14198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b19881640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 0x7f6b1419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b19881640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410d410 con 0x7f6b14107660 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b19881640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410d290 con 0x7f6b14102400 2024-03-26T17:42:50.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b19881640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410d590 con 0x7f6b14104d30 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14102400 0x7f6b14197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b14104d30 0x7f6b14198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 0x7f6b1419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14102400 0x7f6b14197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53340/0 (socket says 172.21.15.105:53340) 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b14104d30 0x7f6b14198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36598/0 (socket says 172.21.15.105:36598) 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b12ffd640 1 -- 172.21.15.105:0/152531835 learned_addr learned my addr 172.21.15.105:0/152531835 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b12ffd640 1 -- 172.21.15.105:0/152531835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b14104d30 msgr2=0x7f6b14198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b12ffd640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b14104d30 0x7f6b14198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b12ffd640 1 -- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 msgr2=0x7f6b1419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b12ffd640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 0x7f6b1419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b12ffd640 1 -- 172.21.15.105:0/152531835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1419f730 con 0x7f6b14102400 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.425+0000 7f6b137fe640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 0x7f6b1419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.426+0000 7f6b12ffd640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14102400 0x7f6b14197b10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400ea60 tx=0x7f6b0400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.426+0000 7f6b1887f640 1 -- 172.21.15.105:0/152531835 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400cde0 con 0x7f6b14102400 2024-03-26T17:42:50.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.426+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1419fa20 con 0x7f6b14102400 2024-03-26T17:42:50.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.426+0000 7f6b1887f640 1 -- 172.21.15.105:0/152531835 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b040035f0 con 0x7f6b14102400 2024-03-26T17:42:50.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.426+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1419ffc0 con 0x7f6b14102400 2024-03-26T17:42:50.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.426+0000 7f6b1887f640 1 -- 172.21.15.105:0/152531835 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04010690 con 0x7f6b14102400 2024-03-26T17:42:50.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.427+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b140fd840 con 0x7f6b14102400 2024-03-26T17:42:50.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.428+0000 7f6b1887f640 1 -- 172.21.15.105:0/152531835 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6b040037a0 con 0x7f6b14102400 2024-03-26T17:42:50.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.429+0000 7f6b1887f640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6afc0a2f20 0x7f6afc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.429+0000 7f6b127fc640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6afc0a2f20 0x7f6afc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.429+0000 7f6b1887f640 1 -- 172.21.15.105:0/152531835 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f6b040cfaf0 con 0x7f6b14102400 2024-03-26T17:42:50.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.429+0000 7f6b127fc640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6afc0a2f20 0x7f6afc0a53e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b14199020 tx=0x7f6b08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.432+0000 7f6b1887f640 1 -- 172.21.15.105:0/152531835 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b0409c8e0 con 0x7f6b14102400 2024-03-26T17:42:50.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.674+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1402d240 con 0x7f6b14102400 2024-03-26T17:42:50.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.675+0000 7f6b1887f640 1 -- 172.21.15.105:0/152531835 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f6b0409c1e0 con 0x7f6b14102400 2024-03-26T17:42:50.668 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:42:50.668 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:42:50.668 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6afc0a2f20 msgr2=0x7f6afc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6afc0a2f20 0x7f6afc0a53e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b14199020 tx=0x7f6b08005f90 comp rx=0 tx=0).stop 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14102400 msgr2=0x7f6b14197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14102400 0x7f6b14197b10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400ea60 tx=0x7f6b0400ef20 comp rx=0 tx=0).stop 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 shutdown_connections 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6afc0a2f20 0x7f6afc0a53e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6b14107660 0x7f6b1419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b14104d30 0x7f6b14198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 --2- 172.21.15.105:0/152531835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b14102400 0x7f6b14197b10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 >> 172.21.15.105:0/152531835 conn(0x7f6b140fbd50 msgr2=0x7f6b14102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 shutdown_connections 2024-03-26T17:42:50.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.677+0000 7f6b19881640 1 -- 172.21.15.105:0/152531835 wait complete. 2024-03-26T17:42:50.681 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 -- 172.21.15.105:0/3898315079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc0103550 msgr2=0x7fecc0109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 --2- 172.21.15.105:0/3898315079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc0103550 0x7fecc0109de0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009cd0 tx=0x7fecb802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 -- 172.21.15.105:0/3898315079 shutdown_connections 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 --2- 172.21.15.105:0/3898315079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc0103550 0x7fecc0109de0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 --2- 172.21.15.105:0/3898315079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc0102b90 0x7fecc0103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 --2- 172.21.15.105:0/3898315079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc00ffbc0 0x7fecc00fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 -- 172.21.15.105:0/3898315079 >> 172.21.15.105:0/3898315079 conn(0x7fecc00fb4f0 msgr2=0x7fecc00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 -- 172.21.15.105:0/3898315079 shutdown_connections 2024-03-26T17:42:50.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 -- 172.21.15.105:0/3898315079 wait complete. 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 Processor -- start 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.803+0000 7fecc66d1640 1 -- start start 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc66d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc00ffbc0 0x7fecc0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc66d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc0102b90 0x7fecc0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc66d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc0103550 0x7fecc019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc66d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010d420 con 0x7fecc00ffbc0 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc66d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010d2a0 con 0x7fecc0103550 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc66d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecc010d5a0 con 0x7fecc0102b90 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc00ffbc0 0x7fecc0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc00ffbc0 0x7fecc0197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46644/0 (socket says 172.21.15.105:46644) 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 -- 172.21.15.105:0/2664401939 learned_addr learned my addr 172.21.15.105:0/2664401939 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc0103550 0x7fecc019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc0102b90 0x7fecc0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 -- 172.21.15.105:0/2664401939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc0102b90 msgr2=0x7fecc0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc0102b90 0x7fecc0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 -- 172.21.15.105:0/2664401939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc0103550 msgr2=0x7fecc019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc0103550 0x7fecc019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 -- 172.21.15.105:0/2664401939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc019f710 con 0x7fecc00ffbc0 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc0103550 0x7fecc019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbffff640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc00ffbc0 0x7fecc0197af0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e9c0 tx=0x7fecb000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbd7fa640 1 -- 172.21.15.105:0/2664401939 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb000cd80 con 0x7fecc00ffbc0 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecc66d1640 1 -- 172.21.15.105:0/2664401939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc019fa00 con 0x7fecc00ffbc0 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbd7fa640 1 -- 172.21.15.105:0/2664401939 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb00035f0 con 0x7fecc00ffbc0 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.804+0000 7fecbd7fa640 1 -- 172.21.15.105:0/2664401939 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0010690 con 0x7fecc00ffbc0 2024-03-26T17:42:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.805+0000 7fecc66d1640 1 -- 172.21.15.105:0/2664401939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc019ffa0 con 0x7fecc00ffbc0 2024-03-26T17:42:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.807+0000 7fecbd7fa640 1 -- 172.21.15.105:0/2664401939 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fecb00037a0 con 0x7fecc00ffbc0 2024-03-26T17:42:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc9396740 1 Processor -- start 2024-03-26T17:42:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc9396740 1 -- start start 2024-03-26T17:42:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc9396740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f0ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc9396740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc9396740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f14a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc9396740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5567f2e71970 con 0x5567f2f12590 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc9396740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5567f2e19d70 con 0x5567f2f0ca30 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc9396740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567f2e6d920 con 0x5567f2f0d420 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc4c6e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f0ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f14a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc4c6e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f0ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53350/0 (socket says 172.21.15.105:53350) 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc4c6e640 1 -- 172.21.15.105:0/1668604153 learned_addr learned my addr 172.21.15.105:0/1668604153 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecbffff640 1 --2- 172.21.15.105:0/1668604153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc4c6e640 1 -- 172.21.15.105:0/1668604153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 msgr2=0x5567f2f11f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/1668604153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f11f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc4c6e640 1 -- 172.21.15.105:0/1668604153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 msgr2=0x5567f2f14a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/1668604153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f14a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecc4c6e640 1 -- 172.21.15.105:0/1668604153 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f2b853f0 con 0x5567f2f0ca30 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.808+0000 7fecbd7fa640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feca80a2f20 0x7feca80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.809+0000 7fecbffff640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feca80a2f20 0x7feca80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.809+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/1668604153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f0ce50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fecb8005e30 tx=0x7fecb8009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87954d8417d54ac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.809+0000 7fecbd7fa640 1 -- 172.21.15.105:0/2664401939 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fecb00cf7a0 con 0x7fecc00ffbc0 2024-03-26T17:42:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.809+0000 7fecbffff640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feca80a2f20 0x7feca80a53e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fecb00cd5d0 tx=0x7fecb00d5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.813+0000 7feca67fc640 1 -- 172.21.15.105:0/1668604153 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8043070 con 0x5567f2f0ca30 2024-03-26T17:42:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.813+0000 7feca67fc640 1 -- 172.21.15.105:0/1668604153 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb8032d70 con 0x5567f2f0ca30 2024-03-26T17:42:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.813+0000 7feca67fc640 1 -- 172.21.15.105:0/1668604153 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8039040 con 0x5567f2f0ca30 2024-03-26T17:42:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 -- 172.21.15.105:0/1668604153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 msgr2=0x5567f2f0ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 --2- 172.21.15.105:0/1668604153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f0ce50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fecb8005e30 tx=0x7fecb8009800 comp rx=0 tx=0).stop 2024-03-26T17:42:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 -- 172.21.15.105:0/1668604153 shutdown_connections 2024-03-26T17:42:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 --2- 172.21.15.105:0/1668604153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f14a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 --2- 172.21.15.105:0/1668604153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f11f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 --2- 172.21.15.105:0/1668604153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f0ce50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 -- 172.21.15.105:0/1668604153 >> 172.21.15.105:0/1668604153 conn(0x5567f2f07fa0 msgr2=0x5567f2f083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 -- 172.21.15.105:0/1668604153 shutdown_connections 2024-03-26T17:42:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 -- 172.21.15.105:0/1668604153 wait complete. 2024-03-26T17:42:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.814+0000 7fecc9396740 1 -- start start 2024-03-26T17:42:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc9396740 1 Processor -- start 2024-03-26T17:42:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc9396740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc9396740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc9396740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f1e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc9396740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5567f2f154e0 con 0x5567f2f12590 2024-03-26T17:42:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc9396740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5567f2e2d310 con 0x5567f2f0ca30 2024-03-26T17:42:50.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc9396740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5567f2f15660 con 0x5567f2f0d420 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc9396740 10 client.0 _ll_register_callbacks cb 0x5567f2e68c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc4c6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f1e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36630/0 (socket says 172.21.15.105:36630) 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecbf7fe640 1 -- 172.21.15.105:0/2287201112 learned_addr learned my addr 172.21.15.105:0/2287201112 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecbf7fe640 1 -- 172.21.15.105:0/2287201112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 msgr2=0x5567f2f16ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecbf7fe640 1 -- 172.21.15.105:0/2287201112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 msgr2=0x5567f2f1e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f1e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecbf7fe640 1 -- 172.21.15.105:0/2287201112 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f2c4f7d0 con 0x5567f2f0d420 2024-03-26T17:42:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.815+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f1e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fecb4012720 tx=0x7fecb4012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4c40ebfd43e7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fec9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:42:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fec9f7fe640 1 -- 172.21.15.105:0/2287201112 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb400f8c0 con 0x5567f2f0d420 2024-03-26T17:42:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fec9f7fe640 1 -- 172.21.15.105:0/2287201112 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fecb400fa60 con 0x5567f2f0d420 2024-03-26T17:42:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fecc9396740 1 -- 172.21.15.105:0/2287201112 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb8032d70 con 0x5567f2f0d420 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fec9f7fe640 1 -- 172.21.15.105:0/2287201112 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fecb400fca0 con 0x5567f2f0d420 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fecc9396740 10 client.26893 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fecc9396740 1 -- 172.21.15.105:0/2287201112 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567f2f21890 con 0x5567f2f0d420 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fec9f7fe640 1 -- 172.21.15.105:0/2287201112 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb8032d70 con 0x5567f2f0d420 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fec9f7fe640 1 -- 172.21.15.105:0/2287201112 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fecb40104c0 con 0x5567f2f0d420 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fecc9396740 10 client.26893 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fecc9396740 10 client.26893 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fecc9396740 20 client.26893 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.816+0000 7fecc9396740 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5567f2f23000 0x5567f2f25440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.817+0000 7fecc9396740 4 client.26893 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.817+0000 7fecc9396740 1 -- 172.21.15.105:0/2287201112 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567f2e7eeb0 con 0x5567f2f23000 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.817+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5567f2f23000 0x5567f2f25440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.817+0000 7fec9f7fe640 10 client.26893 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:42:50.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.817+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5567f2f23000 0x5567f2f25440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e07928e0e2fa926 server_cookie=dd114a57c00df8b0 in_seq=0 out_seq=0 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fec9f7fe640 1 -- 172.21.15.105:0/2287201112 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5567f2e7eeb0 con 0x5567f2f23000 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fec9f7fe640 10 client.26893 handle_command_reply: tid=0 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 client.26893 shutdown 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 -- 172.21.15.105:0/2287201112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 msgr2=0x5567f2f17410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fecb4012720 tx=0x7fecb4012be0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 -- 172.21.15.105:0/2287201112 shutdown_connections 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5567f2f23000 0x5567f2f25440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f12590 0x5567f2f1e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 --2- 172.21.15.105:0/2287201112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fecc9396740 1 -- 172.21.15.105:0/2287201112 >> 172.21.15.105:0/2287201112 conn(0x5567f2f07fa0 msgr2=0x5567f2f071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.818+0000 7fec9f7fe640 0 client.26893 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.819+0000 7fecc9396740 1 -- 172.21.15.105:0/2287201112 shutdown_connections 2024-03-26T17:42:50.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.819+0000 7fecc9396740 1 -- 172.21.15.105:0/2287201112 wait complete. 2024-03-26T17:42:50.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.819+0000 7fecc9396740 20 client.26893 trim_cache size 0 max 16384 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc9396740 1 Processor -- start 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc9396740 1 -- start start 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc9396740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc9396740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc9396740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f1e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc9396740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01a8da0 con 0x5567f2f0ca30 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc9396740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01aa8e0 con 0x5567f2f12590 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc9396740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01a9470 con 0x5567f2f0d420 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc4c6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc4c6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46674/0 (socket says 172.21.15.105:46674) 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc4c6e640 1 -- 172.21.15.105:0/3391142483 learned_addr learned my addr 172.21.15.105:0/3391142483 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.837+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f1e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7fecbf7fe640 1 -- 172.21.15.105:0/3391142483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 msgr2=0x5567f2f1e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f1e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7fecbf7fe640 1 -- 172.21.15.105:0/3391142483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 msgr2=0x5567f2f16ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7fecbf7fe640 1 -- 172.21.15.105:0/3391142483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f2f0dcb0 con 0x5567f2f0d420 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f1e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fecb4000c00 tx=0x7fecb40096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcd15d96b76d281d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7feca5ffb640 1 -- 172.21.15.105:0/3391142483 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb402a030 con 0x5567f2f0d420 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7feca5ffb640 1 -- 172.21.15.105:0/3391142483 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb4010430 con 0x5567f2f0d420 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.838+0000 7feca5ffb640 1 -- 172.21.15.105:0/3391142483 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40056e0 con 0x5567f2f0d420 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 -- 172.21.15.105:0/3391142483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 msgr2=0x5567f2f17410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fecb4000c00 tx=0x7fecb40096c0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 -- 172.21.15.105:0/3391142483 shutdown_connections 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f1e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f17410 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 --2- 172.21.15.105:0/3391142483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f16ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 -- 172.21.15.105:0/3391142483 >> 172.21.15.105:0/3391142483 conn(0x5567f2f07fa0 msgr2=0x5567f2f08380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 -- 172.21.15.105:0/3391142483 shutdown_connections 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 -- 172.21.15.105:0/3391142483 wait complete. 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 -- start start 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.839+0000 7fecc9396740 1 Processor -- start 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc9396740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc9396740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f1b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc9396740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f87330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc9396740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01aa210 con 0x5567f2f0ca30 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc9396740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01a9b40 con 0x5567f2f12590 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc9396740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01a6d80 con 0x5567f2f0d420 2024-03-26T17:42:50.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc9396740 10 client.0 _ll_register_callbacks cb 0x7fecb4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc4c6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f1b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecbf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f1b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc4c6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f1b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46684/0 (socket says 172.21.15.105:46684) 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc4c6e640 1 -- 172.21.15.105:0/830635594 learned_addr learned my addr 172.21.15.105:0/830635594 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f87330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecbf7fe640 1 -- 172.21.15.105:0/830635594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 msgr2=0x5567f2f87330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f87330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecbf7fe640 1 -- 172.21.15.105:0/830635594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 msgr2=0x5567f2f1b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f1b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecbf7fe640 1 -- 172.21.15.105:0/830635594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f2f87870 con 0x5567f2f0d420 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecbf7fe640 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f1b960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fecb40126f0 tx=0x7fecb4005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=544ebdac29d334cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fec9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:42:50.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fec9f7fe640 1 -- 172.21.15.105:0/830635594 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4033060 con 0x5567f2f0d420 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fecc9396740 1 -- 172.21.15.105:0/830635594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb4010430 con 0x5567f2f0d420 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.840+0000 7fec9f7fe640 1 -- 172.21.15.105:0/830635594 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fecb40105e0 con 0x5567f2f0d420 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fec9f7fe640 1 -- 172.21.15.105:0/830635594 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fecb402a040 con 0x5567f2f0d420 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc9396740 10 client.26899 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc9396740 1 -- 172.21.15.105:0/830635594 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567f2f88b70 con 0x5567f2f0d420 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fec9f7fe640 1 -- 172.21.15.105:0/830635594 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb4010430 con 0x5567f2f0d420 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fec9f7fe640 1 -- 172.21.15.105:0/830635594 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fecb401c470 con 0x5567f2f0d420 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc9396740 10 client.26899 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc9396740 10 client.26899 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc9396740 20 client.26899 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc9396740 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5567f2f894b0 0x5567f2f8b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc9396740 4 client.26899 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc9396740 1 -- 172.21.15.105:0/830635594 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567f2e79200 con 0x5567f2f894b0 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5567f2f894b0 0x5567f2f8b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.841+0000 7fec9f7fe640 10 client.26899 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc4c6e640 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5567f2f894b0 0x5567f2f8b8d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96522017fa2bf4ac server_cookie=b123b5d4fa615176 in_seq=0 out_seq=0 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fec9f7fe640 1 -- 172.21.15.105:0/830635594 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567f2e79200 con 0x5567f2f894b0 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fec9f7fe640 10 client.26899 handle_command_reply: tid=0 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 client.26899 shutdown 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 -- 172.21.15.105:0/830635594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 msgr2=0x5567f2f1b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f1b960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fecb40126f0 tx=0x7fecb4005970 comp rx=0 tx=0).stop 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 -- 172.21.15.105:0/830635594 shutdown_connections 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5567f2f894b0 0x5567f2f8b8d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567f2f12590 0x5567f2f87330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5567f2f0d420 0x5567f2f1b960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fec9f7fe640 0 client.26899 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 --2- 172.21.15.105:0/830635594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5567f2f0ca30 0x5567f2f1b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 -- 172.21.15.105:0/830635594 >> 172.21.15.105:0/830635594 conn(0x5567f2f07fa0 msgr2=0x5567f2f0e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 -- 172.21.15.105:0/830635594 shutdown_connections 2024-03-26T17:42:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 1 -- 172.21.15.105:0/830635594 wait complete. 2024-03-26T17:42:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.842+0000 7fecc9396740 20 client.26899 trim_cache size 0 max 16384 2024-03-26T17:42:50.842 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 -- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feca80a2f20 msgr2=0x7feca80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feca80a2f20 0x7feca80a53e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fecb00cd5d0 tx=0x7fecb00d5550 comp rx=0 tx=0).stop 2024-03-26T17:42:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 -- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc00ffbc0 msgr2=0x7fecc0197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc00ffbc0 0x7fecc0197af0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e9c0 tx=0x7fecb000ee80 comp rx=0 tx=0).stop 2024-03-26T17:42:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 -- 172.21.15.105:0/2664401939 shutdown_connections 2024-03-26T17:42:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feca80a2f20 0x7feca80a53e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc0103550 0x7fecc019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc0102b90 0x7fecc0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 --2- 172.21.15.105:0/2664401939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc00ffbc0 0x7fecc0197af0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 -- 172.21.15.105:0/2664401939 >> 172.21.15.105:0/2664401939 conn(0x7fecc00fb4f0 msgr2=0x7fecc0105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 -- 172.21.15.105:0/2664401939 shutdown_connections 2024-03-26T17:42:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.843+0000 7fecc66d1640 1 -- 172.21.15.105:0/2664401939 wait complete. 2024-03-26T17:42:50.850 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:42:50.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.970+0000 7fcec7159640 1 -- 172.21.15.105:0/2919579811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 msgr2=0x7fcec0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.970+0000 7fcec7159640 1 --2- 172.21.15.105:0/2919579811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 0x7fcec0108c00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009ca0 tx=0x7fceb802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.970+0000 7fcec7159640 1 -- 172.21.15.105:0/2919579811 shutdown_connections 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.970+0000 7fcec7159640 1 --2- 172.21.15.105:0/2919579811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 0x7fcec0108c00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.970+0000 7fcec7159640 1 --2- 172.21.15.105:0/2919579811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec01019b0 0x7fcec0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.970+0000 7fcec7159640 1 --2- 172.21.15.105:0/2919579811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcec01007c0 0x7fcec0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.970+0000 7fcec7159640 1 -- 172.21.15.105:0/2919579811 >> 172.21.15.105:0/2919579811 conn(0x7fcec00fbd50 msgr2=0x7fcec00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 -- 172.21.15.105:0/2919579811 shutdown_connections 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 -- 172.21.15.105:0/2919579811 wait complete. 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 Processor -- start 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 -- start start 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcec01007c0 0x7fcec0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec01019b0 0x7fcec019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 0x7fcec019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010c2d0 con 0x7fcec0102370 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010c150 con 0x7fcec01007c0 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.971+0000 7fcec7159640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010c450 con 0x7fcec01019b0 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 0x7fcec019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 0x7fcec019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46696/0 (socket says 172.21.15.105:46696) 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 -- 172.21.15.105:0/2472873689 learned_addr learned my addr 172.21.15.105:0/2472873689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcec01007c0 0x7fcec0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcebffff640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec01019b0 0x7fcec019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 -- 172.21.15.105:0/2472873689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec01019b0 msgr2=0x7fcec019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec01019b0 0x7fcec019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 -- 172.21.15.105:0/2472873689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcec01007c0 msgr2=0x7fcec0069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcec01007c0 0x7fcec0069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 -- 172.21.15.105:0/2472873689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec019f780 con 0x7fcec0102370 2024-03-26T17:42:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec56f6640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 0x7fcec019ee70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fceb802c700 tx=0x7fceb80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcebdffb640 1 -- 172.21.15.105:0/2472873689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8041020 con 0x7fcec0102370 2024-03-26T17:42:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcebdffb640 1 -- 172.21.15.105:0/2472873689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceb8003e20 con 0x7fcec0102370 2024-03-26T17:42:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec7159640 1 -- 172.21.15.105:0/2472873689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec019f9b0 con 0x7fcec0102370 2024-03-26T17:42:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcebdffb640 1 -- 172.21.15.105:0/2472873689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8039040 con 0x7fcec0102370 2024-03-26T17:42:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.972+0000 7fcec7159640 1 -- 172.21.15.105:0/2472873689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec019fea0 con 0x7fcec0102370 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.975+0000 7fcec9e15740 1 Processor -- start 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebdffb640 1 -- 172.21.15.105:0/2472873689 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fceb8044020 con 0x7fcec0102370 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec9e15740 1 -- start start 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x56305713e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x563057143330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 0x563057145e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec9e15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56305709e970 con 0x56305713e830 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec9e15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563057046d70 con 0x5630571439a0 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec9e15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56305709a920 con 0x56305713de40 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec4ef5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x563057143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec4ef5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x563057143330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46702/0 (socket says 172.21.15.105:46702) 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec4ef5640 1 -- 172.21.15.105:0/986680456 learned_addr learned my addr 172.21.15.105:0/986680456 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebffff640 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x56305713e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebffff640 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 0x563057145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebdffb640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fceb00a2f20 0x7fceb00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebffff640 1 -- 172.21.15.105:0/986680456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 msgr2=0x563057145e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebffff640 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 0x563057145e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebffff640 1 -- 172.21.15.105:0/986680456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 msgr2=0x563057143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebffff640 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x563057143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcebffff640 1 -- 172.21.15.105:0/986680456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563056db23f0 con 0x56305713de40 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec56f6640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fceb00a2f20 0x7fceb00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.976+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x563057143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.977+0000 7fcebdffb640 1 -- 172.21.15.105:0/2472873689 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fceb80f15c0 con 0x7fcec0102370 2024-03-26T17:42:50.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.977+0000 7fcebffff640 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x56305713e260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fceac009730 tx=0x7fceac0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d4e14332cfb6a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.977+0000 7fceaaffd640 1 -- 172.21.15.105:0/986680456 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac004050 con 0x56305713de40 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.977+0000 7fceaaffd640 1 -- 172.21.15.105:0/986680456 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac0054d0 con 0x56305713de40 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.977+0000 7fceaaffd640 1 -- 172.21.15.105:0/986680456 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac004c60 con 0x56305713de40 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.977+0000 7fcec9e15740 1 -- 172.21.15.105:0/986680456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 msgr2=0x56305713e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.977+0000 7fcec9e15740 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x56305713e260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fceac009730 tx=0x7fceac0096c0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.977+0000 7fcec56f6640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fceb00a2f20 0x7fceb00a53e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fceb80bd9b0 tx=0x7fceb802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 -- 172.21.15.105:0/986680456 shutdown_connections 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 0x563057145e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x563057143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 --2- 172.21.15.105:0/986680456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x56305713e260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 -- 172.21.15.105:0/986680456 >> 172.21.15.105:0/986680456 conn(0x563057134fa0 msgr2=0x5630571353a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 -- 172.21.15.105:0/986680456 shutdown_connections 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 -- 172.21.15.105:0/986680456 wait complete. 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 -- start start 2024-03-26T17:42:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 Processor -- start 2024-03-26T17:42:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x5630571483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x5630571488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 0x56305714f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563057146950 con 0x56305713e830 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56305705a310 con 0x5630571439a0 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563057146ad0 con 0x56305713de40 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.978+0000 7fcec9e15740 10 client.0 _ll_register_callbacks cb 0x563057095c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x5630571488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x5630571488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46706/0 (socket says 172.21.15.105:46706) 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 -- 172.21.15.105:0/1913965794 learned_addr learned my addr 172.21.15.105:0/1913965794 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcebffff640 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 0x56305714f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcebffff640 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x5630571483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 -- 172.21.15.105:0/1913965794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 msgr2=0x5630571483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x5630571483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 -- 172.21.15.105:0/1913965794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 msgr2=0x56305714f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 0x56305714f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 -- 172.21.15.105:0/1913965794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563056e7c7d0 con 0x56305713e830 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x5630571488e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fceb40126e0 tx=0x7fceb4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c9d89d1432a81d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fce9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fce9ffff640 1 -- 172.21.15.105:0/1913965794 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4012190 con 0x56305713e830 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fce9ffff640 1 -- 172.21.15.105:0/1913965794 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fceb4004030 con 0x56305713e830 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec9e15740 1 -- 172.21.15.105:0/1913965794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceac0054d0 con 0x56305713e830 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fce9ffff640 1 -- 172.21.15.105:0/1913965794 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fceb4004230 con 0x56305713e830 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec9e15740 10 client.17070 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.979+0000 7fcec9e15740 1 -- 172.21.15.105:0/1913965794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563057152d40 con 0x56305713e830 2024-03-26T17:42:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fce9ffff640 1 -- 172.21.15.105:0/1913965794 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac0054d0 con 0x56305713e830 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fce9ffff640 1 -- 172.21.15.105:0/1913965794 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fceb4003820 con 0x56305713e830 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fcec9e15740 10 client.17070 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fcec9e15740 10 client.17070 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fcec9e15740 20 client.17070 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fcec9e15740 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5630571544b0 0x563057156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fcec9e15740 4 client.17070 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fcec9e15740 1 -- 172.21.15.105:0/1913965794 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630570abeb0 con 0x5630571544b0 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fcebffff640 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5630571544b0 0x563057156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fce9ffff640 10 client.17070 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.980+0000 7fcebffff640 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5630571544b0 0x563057156910 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2403915cbc6b027a server_cookie=84f57b448d0c033a in_seq=0 out_seq=0 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fce9ffff640 1 -- 172.21.15.105:0/1913965794 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5630570abeb0 con 0x5630571544b0 2024-03-26T17:42:50.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fce9ffff640 10 client.17070 handle_command_reply: tid=0 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 client.17070 shutdown 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 -- 172.21.15.105:0/1913965794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 msgr2=0x5630571488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x5630571488e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fceb40126e0 tx=0x7fceb4012ba0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 -- 172.21.15.105:0/1913965794 shutdown_connections 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5630571544b0 0x563057156910 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630571439a0 0x56305714f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x5630571488e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fce9ffff640 0 client.17070 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 --2- 172.21.15.105:0/1913965794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56305713de40 0x5630571483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 -- 172.21.15.105:0/1913965794 >> 172.21.15.105:0/1913965794 conn(0x563057134fa0 msgr2=0x5630571341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 -- 172.21.15.105:0/1913965794 shutdown_connections 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.981+0000 7fcec9e15740 1 -- 172.21.15.105:0/1913965794 wait complete. 2024-03-26T17:42:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.982+0000 7fcec9e15740 20 client.17070 trim_cache size 0 max 16384 2024-03-26T17:42:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec9e15740 1 Processor -- start 2024-03-26T17:42:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec9e15740 1 -- start start 2024-03-26T17:42:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713de40 0x563057133930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x56305714f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 0x563057133e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec9e15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a6c10 con 0x56305713e830 2024-03-26T17:42:51.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec9e15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a8750 con 0x56305713de40 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec9e15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a72e0 con 0x5630571439a0 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcebffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713de40 0x563057133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcebffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713de40 0x563057133930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53438/0 (socket says 172.21.15.105:53438) 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcebffff640 1 -- 172.21.15.105:0/4154160984 learned_addr learned my addr 172.21.15.105:0/4154160984 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcebffff640 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 0x563057133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.998+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x56305714f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fcebffff640 1 -- 172.21.15.105:0/4154160984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 msgr2=0x563057133e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fcebffff640 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 0x563057133e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fcebffff640 1 -- 172.21.15.105:0/4154160984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 msgr2=0x56305714f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fcebffff640 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x56305714f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fcebffff640 1 -- 172.21.15.105:0/4154160984 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563057135d10 con 0x56305713de40 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fcebffff640 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713de40 0x563057133930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fceac005fd0 tx=0x7fceac0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d687b0ef752d4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fceaa7fc640 1 -- 172.21.15.105:0/4154160984 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac0313d0 con 0x56305713de40 2024-03-26T17:42:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fceaa7fc640 1 -- 172.21.15.105:0/4154160984 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac00fc40 con 0x56305713de40 2024-03-26T17:42:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:50.999+0000 7fceaa7fc640 1 -- 172.21.15.105:0/4154160984 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac004e40 con 0x56305713de40 2024-03-26T17:42:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 -- 172.21.15.105:0/4154160984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713de40 msgr2=0x563057133930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713de40 0x563057133930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fceac005fd0 tx=0x7fceac0096c0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 -- 172.21.15.105:0/4154160984 shutdown_connections 2024-03-26T17:42:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 0x563057133e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713e830 0x56305714f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 --2- 172.21.15.105:0/4154160984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713de40 0x563057133930 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 -- 172.21.15.105:0/4154160984 >> 172.21.15.105:0/4154160984 conn(0x563057134fa0 msgr2=0x5630571483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 -- 172.21.15.105:0/4154160984 shutdown_connections 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.000+0000 7fcec9e15740 1 -- 172.21.15.105:0/4154160984 wait complete. 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 1 -- start start 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 1 Processor -- start 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713de40 0x56305714d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713e830 0x56305714d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 0x56305714e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a8080 con 0x56305713de40 2024-03-26T17:42:51.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a79b0 con 0x56305713e830 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01a4bf0 con 0x5630571439a0 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec9e15740 10 client.0 _ll_register_callbacks cb 0x7fceb4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcebffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713de40 0x56305714d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcec4ef5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713e830 0x56305714d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcebffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713de40 0x56305714d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46724/0 (socket says 172.21.15.105:46724) 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcebffff640 1 -- 172.21.15.105:0/3169941453 learned_addr learned my addr 172.21.15.105:0/3169941453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.001+0000 7fcebffff640 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 0x56305714e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec4ef5640 1 -- 172.21.15.105:0/3169941453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 msgr2=0x56305714e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 0x56305714e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec4ef5640 1 -- 172.21.15.105:0/3169941453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713de40 msgr2=0x56305714d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713de40 0x56305714d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec4ef5640 1 -- 172.21.15.105:0/3169941453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56305714eda0 con 0x56305713e830 2024-03-26T17:42:51.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcebffff640 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713de40 0x56305714d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec4ef5640 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713e830 0x56305714d7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fceb40059f0 tx=0x7fceb4009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a28770584c1b00 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fce9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fce9ffff640 1 -- 172.21.15.105:0/3169941453 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4033020 con 0x56305713e830 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec9e15740 1 -- 172.21.15.105:0/3169941453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceac00fc40 con 0x56305713e830 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fce9ffff640 1 -- 172.21.15.105:0/3169941453 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fceb40055a0 con 0x56305713e830 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fce9ffff640 1 -- 172.21.15.105:0/3169941453 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fceb402b040 con 0x56305713e830 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec9e15740 10 client.26918 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fcec9e15740 1 -- 172.21.15.105:0/3169941453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630571baa40 con 0x56305713e830 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.002+0000 7fce9ffff640 1 -- 172.21.15.105:0/3169941453 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac00fc40 con 0x56305713e830 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fce9ffff640 1 -- 172.21.15.105:0/3169941453 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fceb401c460 con 0x56305713e830 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcec9e15740 10 client.26918 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcec9e15740 10 client.26918 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcec9e15740 20 client.26918 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcec9e15740 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5630571544b0 0x5630571bd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcec9e15740 4 client.26918 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcec9e15740 1 -- 172.21.15.105:0/3169941453 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563057148190 con 0x5630571544b0 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcebffff640 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5630571544b0 0x5630571bd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fce9ffff640 10 client.26918 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcebffff640 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5630571544b0 0x5630571bd3f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab5377023c092981 server_cookie=e7d8c6f8f27eaeee in_seq=0 out_seq=0 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fce9ffff640 1 -- 172.21.15.105:0/3169941453 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563057148190 con 0x5630571544b0 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fce9ffff640 10 client.26918 handle_command_reply: tid=0 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.003+0000 7fcec9e15740 1 client.26918 shutdown 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 -- 172.21.15.105:0/3169941453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713e830 msgr2=0x56305714d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713e830 0x56305714d7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fceb40059f0 tx=0x7fceb4009f30 comp rx=0 tx=0).stop 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 -- 172.21.15.105:0/3169941453 shutdown_connections 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5630571544b0 0x5630571bd3f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630571439a0 0x56305714e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56305713e830 0x56305714d7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 --2- 172.21.15.105:0/3169941453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56305713de40 0x56305714d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 -- 172.21.15.105:0/3169941453 >> 172.21.15.105:0/3169941453 conn(0x563057134fa0 msgr2=0x563057138200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fce9ffff640 0 client.26918 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:42:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 -- 172.21.15.105:0/3169941453 shutdown_connections 2024-03-26T17:42:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 1 -- 172.21.15.105:0/3169941453 wait complete. 2024-03-26T17:42:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.004+0000 7fcec9e15740 20 client.26918 trim_cache size 0 max 16384 2024-03-26T17:42:51.007 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:51.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 -- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fceb00a2f20 msgr2=0x7fceb00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fceb00a2f20 0x7fceb00a53e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fceb80bd9b0 tx=0x7fceb802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 -- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 msgr2=0x7fcec019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 0x7fcec019ee70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fceb802c700 tx=0x7fceb80328a0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 -- 172.21.15.105:0/2472873689 shutdown_connections 2024-03-26T17:42:51.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fceb00a2f20 0x7fceb00a53e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec0102370 0x7fcec019ee70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcec01019b0 0x7fcec019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 --2- 172.21.15.105:0/2472873689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcec01007c0 0x7fcec0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 -- 172.21.15.105:0/2472873689 >> 172.21.15.105:0/2472873689 conn(0x7fcec00fbd50 msgr2=0x7fcec0104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 -- 172.21.15.105:0/2472873689 shutdown_connections 2024-03-26T17:42:51.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.005+0000 7fcec7159640 1 -- 172.21.15.105:0/2472873689 wait complete. 2024-03-26T17:42:51.010 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:50 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/152531835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:42:51.011 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:42:51.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 126286 3.11 MB/sec execute 94 sec latency 15.835 ms 2024-03-26T17:42:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 -- 172.21.15.105:0/3812776663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0101910 msgr2=0x7f57d0101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 --2- 172.21.15.105:0/3812776663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0101910 0x7f57d0101d30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009900 tx=0x7f57c002bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 -- 172.21.15.105:0/3812776663 shutdown_connections 2024-03-26T17:42:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 --2- 172.21.15.105:0/3812776663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d01034c0 0x7f57d0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 --2- 172.21.15.105:0/3812776663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d0102b00 0x7f57d0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 --2- 172.21.15.105:0/3812776663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0101910 0x7f57d0101d30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 -- 172.21.15.105:0/3812776663 >> 172.21.15.105:0/3812776663 conn(0x7f57d00fce80 msgr2=0x7f57d00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 -- 172.21.15.105:0/3812776663 shutdown_connections 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.073+0000 7f57d6a36640 1 -- 172.21.15.105:0/3812776663 wait complete. 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d6a36640 1 Processor -- start 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d6a36640 1 -- start start 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d6a36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d0101910 0x7f57d0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d6a36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0102b00 0x7f57d0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d6a36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01034c0 0x7f57d019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d6a36640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010d410 con 0x7f57d0102b00 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d6a36640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010d290 con 0x7f57d0101910 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d6a36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010d590 con 0x7f57d01034c0 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0102b00 0x7f57d0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0102b00 0x7f57d0197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46748/0 (socket says 172.21.15.105:46748) 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57cf7fe640 1 -- 172.21.15.105:0/1058863608 learned_addr learned my addr 172.21.15.105:0/1058863608 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.074+0000 7f57d4fd3640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01034c0 0x7f57d019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57d4fd3640 1 -- 172.21.15.105:0/1058863608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d0101910 msgr2=0x7f57d0197a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57d4fd3640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d0101910 0x7f57d0197a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57d4fd3640 1 -- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0102b00 msgr2=0x7f57d0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57d4fd3640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0102b00 0x7f57d0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57d4fd3640 1 -- 172.21.15.105:0/1058863608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d019f6b0 con 0x7f57d01034c0 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0102b00 0x7f57d0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57d4fd3640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01034c0 0x7f57d019f040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009fd0 tx=0x7f57c800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57cd7fa640 1 -- 172.21.15.105:0/1058863608 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c800ce20 con 0x7f57d01034c0 2024-03-26T17:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57d6a36640 1 -- 172.21.15.105:0/1058863608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d0073b50 con 0x7f57d01034c0 2024-03-26T17:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57cd7fa640 1 -- 172.21.15.105:0/1058863608 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57c80137d0 con 0x7f57d01034c0 2024-03-26T17:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57d6a36640 1 -- 172.21.15.105:0/1058863608 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d00740a0 con 0x7f57d01034c0 2024-03-26T17:42:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.075+0000 7f57cd7fa640 1 -- 172.21.15.105:0/1058863608 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8005540 con 0x7f57d01034c0 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57d9629740 1 Processor -- start 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57cd7fa640 1 -- 172.21.15.105:0/1058863608 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f57c8013980 con 0x7f57d01034c0 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57d9629740 1 -- start start 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57d9629740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57d9629740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57d9629740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57d9629740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd8190b0 con 0x55e6dd8b45d0 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57d9629740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd7c1960 con 0x55e6dd8b4fc0 2024-03-26T17:42:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57d9629740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd815060 con 0x55e6dd8ba130 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46752/0 (socket says 172.21.15.105:46752) 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53484/0 (socket says 172.21.15.105:53484) 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57cf7fe640 1 -- 172.21.15.105:0/124228047 learned_addr learned my addr 172.21.15.105:0/124228047 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.079+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/124228047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57cd7fa640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57b80a2fc0 0x7f57b80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57cf7fe640 1 -- 172.21.15.105:0/124228047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 msgr2=0x55e6dd8bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/124228047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8bc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57cf7fe640 1 -- 172.21.15.105:0/124228047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 msgr2=0x55e6dd8b9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/124228047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57cf7fe640 1 -- 172.21.15.105:0/124228047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dd7d3c60 con 0x55e6dd8b45d0 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57d4fd3640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57b80a2fc0 0x7f57b80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/124228047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8b49f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f57c4013d10 tx=0x7f57c4009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc4d256fe8b2689 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57cd7fa640 1 -- 172.21.15.105:0/1058863608 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f57c80d3c10 con 0x7f57d01034c0 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57b67fc640 1 -- 172.21.15.105:0/124228047 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4013e00 con 0x55e6dd8b45d0 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57d4fd3640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57b80a2fc0 0x7f57b80a5480 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f57c809ccd0 tx=0x7f57c809bf00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.080+0000 7f57b67fc640 1 -- 172.21.15.105:0/124228047 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57c4011c70 con 0x55e6dd8b45d0 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.081+0000 7f57b67fc640 1 -- 172.21.15.105:0/124228047 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4003d80 con 0x55e6dd8b45d0 2024-03-26T17:42:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.081+0000 7f57d9629740 1 -- 172.21.15.105:0/124228047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b45d0 msgr2=0x55e6dd8b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.081+0000 7f57d9629740 1 --2- 172.21.15.105:0/124228047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8b49f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f57c4013d10 tx=0x7f57c4009590 comp rx=0 tx=0).stop 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.082+0000 7f57d9629740 1 -- 172.21.15.105:0/124228047 shutdown_connections 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.082+0000 7f57d9629740 1 --2- 172.21.15.105:0/124228047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.082+0000 7f57d9629740 1 --2- 172.21.15.105:0/124228047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.082+0000 7f57d9629740 1 --2- 172.21.15.105:0/124228047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8b49f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.082+0000 7f57d9629740 1 -- 172.21.15.105:0/124228047 >> 172.21.15.105:0/124228047 conn(0x55e6dd8afb40 msgr2=0x55e6dd8aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.082+0000 7f57d9629740 1 -- 172.21.15.105:0/124228047 shutdown_connections 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.082+0000 7f57d9629740 1 -- 172.21.15.105:0/124228047 wait complete. 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.083+0000 7f57d9629740 1 -- start start 2024-03-26T17:42:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.083+0000 7f57d9629740 1 Processor -- start 2024-03-26T17:42:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.083+0000 7f57d9629740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57d9629740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57d9629740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57d9629740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd8bd130 con 0x55e6dd8b4fc0 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57d9629740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd7d4db0 con 0x55e6dd8b45d0 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57d9629740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd8bd2b0 con 0x55e6dd8ba130 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57d9629740 10 client.0 _ll_register_callbacks cb 0x55e6dd7d9110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53492/0 (socket says 172.21.15.105:53492) 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 -- 172.21.15.105:0/1505532325 learned_addr learned my addr 172.21.15.105:0/1505532325 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cffff640 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 -- 172.21.15.105:0/1505532325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 msgr2=0x55e6dd8c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 -- 172.21.15.105:0/1505532325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 msgr2=0x55e6dd8be8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8be8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.084+0000 7f57cf7fe640 1 -- 172.21.15.105:0/1505532325 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dd52d210 con 0x55e6dd8b45d0 2024-03-26T17:42:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.085+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57c4005d80 tx=0x7f57c4005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2f61ba52b4c3174 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.085+0000 7f57af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:42:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.086+0000 7f57af7fe640 1 -- 172.21.15.105:0/1505532325 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40046c0 con 0x55e6dd8b45d0 2024-03-26T17:42:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.087+0000 7f57d9629740 1 -- 172.21.15.105:0/1505532325 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57c4011c70 con 0x55e6dd8b45d0 2024-03-26T17:42:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.087+0000 7f57af7fe640 1 -- 172.21.15.105:0/1505532325 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f57c4013e00 con 0x55e6dd8b45d0 2024-03-26T17:42:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57af7fe640 1 -- 172.21.15.105:0/1505532325 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57c4004860 con 0x55e6dd8b45d0 2024-03-26T17:42:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57d9629740 10 client.26927 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57d9629740 1 -- 172.21.15.105:0/1505532325 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6dd5f9620 con 0x55e6dd8b45d0 2024-03-26T17:42:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57af7fe640 1 -- 172.21.15.105:0/1505532325 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57c4017040 con 0x55e6dd8b45d0 2024-03-26T17:42:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57af7fe640 1 -- 172.21.15.105:0/1505532325 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f57c403b020 con 0x55e6dd8b45d0 2024-03-26T17:42:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57d9629740 10 client.26927 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57d9629740 10 client.26927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57d9629740 20 client.26927 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57d9629740 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6dd8c9ef0 0x55e6dd8cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57d9629740 4 client.26927 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.088+0000 7f57d9629740 1 -- 172.21.15.105:0/1505532325 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f57c4011c70 con 0x55e6dd8c9ef0 2024-03-26T17:42:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.089+0000 7f57cffff640 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6dd8c9ef0 0x55e6dd8cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.089+0000 7f57af7fe640 10 client.26927 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.089+0000 7f57cffff640 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6dd8c9ef0 0x55e6dd8cc2e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582547888e62e7f8 server_cookie=9fd045164590eb7a in_seq=0 out_seq=0 2024-03-26T17:42:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.091+0000 7f57af7fe640 1 -- 172.21.15.105:0/1505532325 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f57c4011c70 con 0x55e6dd8c9ef0 2024-03-26T17:42:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.091+0000 7f57af7fe640 10 client.26927 handle_command_reply: tid=0 2024-03-26T17:42:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.091+0000 7f57d9629740 1 client.26927 shutdown 2024-03-26T17:42:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57d9629740 1 -- 172.21.15.105:0/1505532325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 msgr2=0x55e6dd8be380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57d9629740 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57c4005d80 tx=0x7f57c4005c50 comp rx=0 tx=0).stop 2024-03-26T17:42:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57d9629740 1 -- 172.21.15.105:0/1505532325 shutdown_connections 2024-03-26T17:42:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57d9629740 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6dd8c9ef0 0x55e6dd8cc2e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57af7fe640 0 client.26927 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57d9629740 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57d9629740 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8be8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57d9629740 1 --2- 172.21.15.105:0/1505532325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be380 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.092+0000 7f57d9629740 1 -- 172.21.15.105:0/1505532325 >> 172.21.15.105:0/1505532325 conn(0x55e6dd8afb40 msgr2=0x55e6dd8aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.093+0000 7f57d9629740 1 -- 172.21.15.105:0/1505532325 shutdown_connections 2024-03-26T17:42:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.093+0000 7f57d9629740 1 -- 172.21.15.105:0/1505532325 wait complete. 2024-03-26T17:42:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.093+0000 7f57d9629740 20 client.26927 trim_cache size 0 max 16384 2024-03-26T17:42:51.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:50 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/152531835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:42:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.112+0000 7f57d9629740 1 Processor -- start 2024-03-26T17:42:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.112+0000 7f57d9629740 1 -- start start 2024-03-26T17:42:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.112+0000 7f57d9629740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.112+0000 7f57d9629740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.112+0000 7f57d9629740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.112+0000 7f57d9629740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0072740 con 0x55e6dd8b4fc0 2024-03-26T17:42:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.112+0000 7f57d9629740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f57d019ffb0 con 0x55e6dd8b45d0 2024-03-26T17:42:51.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.112+0000 7f57d9629740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0072e10 con 0x55e6dd8ba130 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46762/0 (socket says 172.21.15.105:46762) 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 -- 172.21.15.105:0/4285200188 learned_addr learned my addr 172.21.15.105:0/4285200188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 -- 172.21.15.105:0/4285200188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 msgr2=0x55e6dd8b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 -- 172.21.15.105:0/4285200188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 msgr2=0x55e6dd8be6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 -- 172.21.15.105:0/4285200188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dd8cc650 con 0x55e6dd8b4fc0 2024-03-26T17:42:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57cffff640 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b5910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009230 tx=0x7f57c002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4297b540e19bb10c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.113+0000 7f57b5ffb640 1 -- 172.21.15.105:0/4285200188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c004c070 con 0x55e6dd8b4fc0 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57b5ffb640 1 -- 172.21.15.105:0/4285200188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57c0009520 con 0x55e6dd8b4fc0 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57b5ffb640 1 -- 172.21.15.105:0/4285200188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00462f0 con 0x55e6dd8b4fc0 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 -- 172.21.15.105:0/4285200188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 msgr2=0x55e6dd8b5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b5910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009230 tx=0x7f57c002e040 comp rx=0 tx=0).stop 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 -- 172.21.15.105:0/4285200188 shutdown_connections 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8ba130 0x55e6dd8b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8b5910 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 --2- 172.21.15.105:0/4285200188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8be6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 -- 172.21.15.105:0/4285200188 >> 172.21.15.105:0/4285200188 conn(0x55e6dd8afb40 msgr2=0x55e6dd8bdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 -- 172.21.15.105:0/4285200188 shutdown_connections 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.114+0000 7f57d9629740 1 -- 172.21.15.105:0/4285200188 wait complete. 2024-03-26T17:42:51.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 1 -- start start 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 1 Processor -- start 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8c2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f57d019f900 con 0x55e6dd8ba130 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00734e0 con 0x55e6dd8b4fc0 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0070720 con 0x55e6dd8b45d0 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57d9629740 10 client.0 _ll_register_callbacks cb 0x7f57c80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.115+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46768/0 (socket says 172.21.15.105:46768) 2024-03-26T17:42:51.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cf7fe640 1 -- 172.21.15.105:0/3081037110 learned_addr learned my addr 172.21.15.105:0/3081037110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cffff640 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cf7fe640 1 -- 172.21.15.105:0/3081037110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8b45d0 msgr2=0x55e6dd8c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cf7fe640 1 -- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 msgr2=0x55e6dd8c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8c2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cf7fe640 1 -- 172.21.15.105:0/3081037110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dd8c3e10 con 0x55e6dd8ba130 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cffff640 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57cf7fe640 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c3820 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f57c4004390 tx=0x7f57c4004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c69e222e553a5ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57af7fe640 1 -- 172.21.15.105:0/3081037110 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4005350 con 0x55e6dd8ba130 2024-03-26T17:42:51.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57af7fe640 1 -- 172.21.15.105:0/3081037110 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f57c40045d0 con 0x55e6dd8ba130 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57d9629740 1 -- 172.21.15.105:0/3081037110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57c0009520 con 0x55e6dd8ba130 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57af7fe640 1 -- 172.21.15.105:0/3081037110 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57c4004a00 con 0x55e6dd8ba130 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57d9629740 10 client.17094 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.116+0000 7f57d9629740 1 -- 172.21.15.105:0/3081037110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6dd930190 con 0x55e6dd8ba130 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57af7fe640 1 -- 172.21.15.105:0/3081037110 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57c0009520 con 0x55e6dd8ba130 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57af7fe640 1 -- 172.21.15.105:0/3081037110 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f57c403b040 con 0x55e6dd8ba130 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57d9629740 10 client.17094 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57d9629740 10 client.17094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57d9629740 20 client.17094 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57d9629740 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6dd930a20 0x55e6dd932e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57d9629740 4 client.17094 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:42:51.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57d9629740 1 -- 172.21.15.105:0/3081037110 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6dd8265f0 con 0x55e6dd930a20 2024-03-26T17:42:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.117+0000 7f57cffff640 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6dd930a20 0x55e6dd932e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.118+0000 7f57af7fe640 10 client.17094 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.118+0000 7f57cffff640 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6dd930a20 0x55e6dd932e40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03fd8939e4b18d server_cookie=844f926b76efa8d6 in_seq=0 out_seq=0 2024-03-26T17:42:51.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 -- 172.21.15.105:0/2841763056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf4107600 msgr2=0x7ffbf41099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 --2- 172.21.15.105:0/2841763056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf4107600 0x7ffbf41099f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009ca0 tx=0x7ffbec02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 -- 172.21.15.105:0/2841763056 shutdown_connections 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 --2- 172.21.15.105:0/2841763056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf4107600 0x7ffbf41099f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 --2- 172.21.15.105:0/2841763056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf4104cd0 0x7ffbf41070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 --2- 172.21.15.105:0/2841763056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbf41023a0 0x7ffbf4104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 -- 172.21.15.105:0/2841763056 >> 172.21.15.105:0/2841763056 conn(0x7ffbf40fbd50 msgr2=0x7ffbf40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 -- 172.21.15.105:0/2841763056 shutdown_connections 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.158+0000 7ffbfa868640 1 -- 172.21.15.105:0/2841763056 wait complete. 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbfa868640 1 Processor -- start 2024-03-26T17:42:51.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbfa868640 1 -- start start 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbfa868640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf41023a0 0x7ffbf4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbfa868640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbf4104cd0 0x7ffbf4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbfa868640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf4107600 0x7ffbf419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbfa868640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf410d410 con 0x7ffbf41023a0 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbfa868640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf410d290 con 0x7ffbf4107600 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbfa868640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf410d590 con 0x7ffbf4104cd0 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf41023a0 0x7ffbf4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf41023a0 0x7ffbf4197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46786/0 (socket says 172.21.15.105:46786) 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbf3fff640 1 -- 172.21.15.105:0/4004292347 learned_addr learned my addr 172.21.15.105:0/4004292347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbf8e05640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf4107600 0x7ffbf419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.159+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbf4104cd0 0x7ffbf4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf37fe640 1 -- 172.21.15.105:0/4004292347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf4107600 msgr2=0x7ffbf419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf4107600 0x7ffbf419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf37fe640 1 -- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf41023a0 msgr2=0x7ffbf4197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf41023a0 0x7ffbf4197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf37fe640 1 -- 172.21.15.105:0/4004292347 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf419f710 con 0x7ffbf4104cd0 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf8e05640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf4107600 0x7ffbf419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf41023a0 0x7ffbf4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbf4104cd0 0x7ffbf4198030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffbe800e9c0 tx=0x7ffbe800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf17fa640 1 -- 172.21.15.105:0/4004292347 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe800ccc0 con 0x7ffbf4104cd0 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf17fa640 1 -- 172.21.15.105:0/4004292347 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbe80108e0 con 0x7ffbf4104cd0 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbfa868640 1 -- 172.21.15.105:0/4004292347 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf419fa00 con 0x7ffbf4104cd0 2024-03-26T17:42:51.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbf17fa640 1 -- 172.21.15.105:0/4004292347 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe80187a0 con 0x7ffbf4104cd0 2024-03-26T17:42:51.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.160+0000 7ffbfa868640 1 -- 172.21.15.105:0/4004292347 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf419ff50 con 0x7ffbf4104cd0 2024-03-26T17:42:51.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:50 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/152531835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:42:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbfd44e740 1 Processor -- start 2024-03-26T17:42:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbfd44e740 1 -- start start 2024-03-26T17:42:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf23e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf28f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf2ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5586fbe88970 con 0x5586fbf24420 2024-03-26T17:42:51.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5586fbe30d70 con 0x5586fbf23a30 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5586fbe84920 con 0x5586fbf29590 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf28f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf8e05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf23e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf28f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46794/0 (socket says 172.21.15.105:46794) 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf8e05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf23e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53512/0 (socket says 172.21.15.105:53512) 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 -- 172.21.15.105:0/4005871015 learned_addr learned my addr 172.21.15.105:0/4005871015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 -- 172.21.15.105:0/4005871015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 msgr2=0x5586fbf2ba20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/4005871015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf2ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 -- 172.21.15.105:0/4005871015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 msgr2=0x5586fbf23e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/4005871015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf23e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf17fa640 1 -- 172.21.15.105:0/4004292347 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ffbe8010a90 con 0x7ffbf4104cd0 2024-03-26T17:42:51.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 -- 172.21.15.105:0/4005871015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fbb9c3f0 con 0x5586fbf24420 2024-03-26T17:42:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.163+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/4005871015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf28f20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4011630 tx=0x7ffbe4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab436796ff848aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbda7fc640 1 -- 172.21.15.105:0/4005871015 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe401e070 con 0x5586fbf24420 2024-03-26T17:42:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbda7fc640 1 -- 172.21.15.105:0/4005871015 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbe400fba0 con 0x5586fbf24420 2024-03-26T17:42:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbda7fc640 1 -- 172.21.15.105:0/4005871015 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4019c80 con 0x5586fbf24420 2024-03-26T17:42:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbf17fa640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffbdc0a2f20 0x7ffbdc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbf8e05640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffbdc0a2f20 0x7ffbdc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbf17fa640 1 -- 172.21.15.105:0/4004292347 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ffbe80cf720 con 0x7ffbf4104cd0 2024-03-26T17:42:51.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 -- 172.21.15.105:0/4005871015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 msgr2=0x5586fbf28f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/4005871015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf28f20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4011630 tx=0x7ffbe4011af0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 -- 172.21.15.105:0/4005871015 shutdown_connections 2024-03-26T17:42:51.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/4005871015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf2ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/4005871015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf28f20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/4005871015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf23e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 -- 172.21.15.105:0/4005871015 >> 172.21.15.105:0/4005871015 conn(0x5586fbf1efa0 msgr2=0x5586fbf1f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 -- 172.21.15.105:0/4005871015 shutdown_connections 2024-03-26T17:42:51.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbfd44e740 1 -- 172.21.15.105:0/4005871015 wait complete. 2024-03-26T17:42:51.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.164+0000 7ffbf8e05640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffbdc0a2f20 0x7ffbdc0a53e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffbec006010 tx=0x7ffbec0324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 1 -- start start 2024-03-26T17:42:51.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 1 Processor -- start 2024-03-26T17:42:51.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf2df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf2e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf35500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5586fbf2c540 con 0x5586fbf24420 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5586fbe44310 con 0x5586fbf23a30 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5586fbf2c6c0 con 0x5586fbf29590 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbfd44e740 10 client.0 _ll_register_callbacks cb 0x5586fbe7fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf2e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf2e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46796/0 (socket says 172.21.15.105:46796) 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf2df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbf37fe640 1 -- 172.21.15.105:0/3541546805 learned_addr learned my addr 172.21.15.105:0/3541546805 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.165+0000 7ffbf8e05640 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf35500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbf37fe640 1 -- 172.21.15.105:0/3541546805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 msgr2=0x5586fbf35500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf35500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbf37fe640 1 -- 172.21.15.105:0/3541546805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 msgr2=0x5586fbf2df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf2df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbf37fe640 1 -- 172.21.15.105:0/3541546805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fbc667d0 con 0x5586fbf24420 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf2df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf2e4c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80cd1e0 tx=0x7ffbe809b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dcbc6d1c3dfb66 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbd37fe640 1 -- 172.21.15.105:0/3541546805 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe80bad80 con 0x5586fbf24420 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbd37fe640 1 -- 172.21.15.105:0/3541546805 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ffbe8010430 con 0x5586fbf24420 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3541546805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbe400fba0 con 0x5586fbf24420 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbd37fe640 1 -- 172.21.15.105:0/3541546805 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffbe809ff60 con 0x5586fbf24420 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbfd44e740 10 client.17106 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3541546805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586fbf388e0 con 0x5586fbf24420 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.166+0000 7ffbd37fe640 1 -- 172.21.15.105:0/3541546805 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbe400fba0 con 0x5586fbf24420 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbd37fe640 1 -- 172.21.15.105:0/3541546805 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffbe809ad80 con 0x5586fbf24420 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbfd44e740 10 client.17106 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbfd44e740 10 client.17106 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbfd44e740 20 client.17106 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5586fbf39f90 0x5586fbf3c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbfd44e740 4 client.17106 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3541546805 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586fbe95eb0 con 0x5586fbf39f90 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5586fbf39f90 0x5586fbf3c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbd37fe640 10 client.17106 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:42:51.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.167+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5586fbf39f90 0x5586fbf3c3f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37e34d5078454765 server_cookie=bd19b1e24f772222 in_seq=0 out_seq=0 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.168+0000 7ffbd37fe640 1 -- 172.21.15.105:0/3541546805 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5586fbe95eb0 con 0x5586fbf39f90 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.168+0000 7ffbd37fe640 10 client.17106 handle_command_reply: tid=0 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.168+0000 7ffbfd44e740 1 client.17106 shutdown 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3541546805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 msgr2=0x5586fbf2e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf2e4c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80cd1e0 tx=0x7ffbe809b820 comp rx=0 tx=0).stop 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3541546805 shutdown_connections 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf35500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf2e4c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5586fbf39f90 0x5586fbf3c3f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3541546805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf2df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3541546805 >> 172.21.15.105:0/3541546805 conn(0x5586fbf1efa0 msgr2=0x5586fbf1e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbd37fe640 0 client.17106 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:42:51.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3541546805 shutdown_connections 2024-03-26T17:42:51.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3541546805 wait complete. 2024-03-26T17:42:51.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.169+0000 7ffbfd44e740 20 client.17106 trim_cache size 0 max 16384 2024-03-26T17:42:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.174+0000 7f57af7fe640 1 -- 172.21.15.105:0/3081037110 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6dd8265f0 con 0x55e6dd930a20 2024-03-26T17:42:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.174+0000 7f57af7fe640 10 client.17094 handle_command_reply: tid=0 2024-03-26T17:42:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.174+0000 7f57d9629740 1 client.17094 shutdown 2024-03-26T17:42:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 -- 172.21.15.105:0/3081037110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8ba130 msgr2=0x55e6dd8c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c3820 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f57c4004390 tx=0x7f57c4004130 comp rx=0 tx=0).stop 2024-03-26T17:42:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 -- 172.21.15.105:0/3081037110 shutdown_connections 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6dd930a20 0x55e6dd932e40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6dd8ba130 0x55e6dd8c3820 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6dd8b4fc0 0x55e6dd8c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57af7fe640 0 client.17094 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 --2- 172.21.15.105:0/3081037110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6dd8b45d0 0x55e6dd8c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 -- 172.21.15.105:0/3081037110 >> 172.21.15.105:0/3081037110 conn(0x55e6dd8afb40 msgr2=0x55e6dd8bdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 -- 172.21.15.105:0/3081037110 shutdown_connections 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 1 -- 172.21.15.105:0/3081037110 wait complete. 2024-03-26T17:42:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.175+0000 7f57d9629740 20 client.17094 trim_cache size 0 max 16384 2024-03-26T17:42:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:42:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:42:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:42:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:42:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 -- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57b80a2fc0 msgr2=0x7f57b80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57b80a2fc0 0x7f57b80a5480 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f57c809ccd0 tx=0x7f57c809bf00 comp rx=0 tx=0).stop 2024-03-26T17:42:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 -- 172.21.15.105:0/1058863608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01034c0 msgr2=0x7f57d019f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01034c0 0x7f57d019f040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009fd0 tx=0x7f57c800ef10 comp rx=0 tx=0).stop 2024-03-26T17:42:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 -- 172.21.15.105:0/1058863608 shutdown_connections 2024-03-26T17:42:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57b80a2fc0 0x7f57b80a5480 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01034c0 0x7f57d019f040 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57d0102b00 0x7f57d0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 --2- 172.21.15.105:0/1058863608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d0101910 0x7f57d0197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 -- 172.21.15.105:0/1058863608 >> 172.21.15.105:0/1058863608 conn(0x7f57d00fce80 msgr2=0x7f57d0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 -- 172.21.15.105:0/1058863608 shutdown_connections 2024-03-26T17:42:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:42:51.176+0000 7f57d6a36640 1 -- 172.21.15.105:0/1058863608 wait complete. 2024-03-26T17:42:51.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbfd44e740 1 Processor -- start 2024-03-26T17:42:51.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbfd44e740 1 -- start start 2024-03-26T17:42:51.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf25960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41a8d80 con 0x5586fbf24420 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41aa8c0 con 0x5586fbf23a30 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41a9450 con 0x5586fbf29590 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46804/0 (socket says 172.21.15.105:46804) 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbf37fe640 1 -- 172.21.15.105:0/3373976765 learned_addr learned my addr 172.21.15.105:0/3373976765 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.185+0000 7ffbf8e05640 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf25960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbf37fe640 1 -- 172.21.15.105:0/3373976765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 msgr2=0x5586fbf25960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf25960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbf37fe640 1 -- 172.21.15.105:0/3373976765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 msgr2=0x5586fbf3bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf3bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbf37fe640 1 -- 172.21.15.105:0/3373976765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586fbf1fd10 con 0x5586fbf24420 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf354c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8098320 tx=0x7ffbe809a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6d537855549d58 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbd9ffb640 1 -- 172.21.15.105:0/3373976765 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe80a4820 con 0x5586fbf24420 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbd9ffb640 1 -- 172.21.15.105:0/3373976765 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbe8097c10 con 0x5586fbf24420 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.186+0000 7ffbd9ffb640 1 -- 172.21.15.105:0/3373976765 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe80bade0 con 0x5586fbf24420 2024-03-26T17:42:51.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3373976765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 msgr2=0x5586fbf354c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf354c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8098320 tx=0x7ffbe809a3d0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3373976765 shutdown_connections 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf25960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf24420 0x5586fbf354c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/3373976765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf3bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3373976765 >> 172.21.15.105:0/3373976765 conn(0x5586fbf1efa0 msgr2=0x5586fbf2ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3373976765 shutdown_connections 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.187+0000 7ffbfd44e740 1 -- 172.21.15.105:0/3373976765 wait complete. 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 1 -- start start 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 1 Processor -- start 2024-03-26T17:42:51.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf32690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf32bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf33b70 0x5586fbf33ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41aa1f0 con 0x5586fbf33b70 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41a9b20 con 0x5586fbf23a30 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41a6d60 con 0x5586fbf29590 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbfd44e740 10 client.0 _ll_register_callbacks cb 0x7ffbe8097a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf32690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf32bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbf8e05640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf33b70 0x5586fbf33ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf32690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53538/0 (socket says 172.21.15.105:53538) 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf32bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36826/0 (socket says 172.21.15.105:36826) 2024-03-26T17:42:51.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.188+0000 7ffbf3fff640 1 -- 172.21.15.105:0/2710770925 learned_addr learned my addr 172.21.15.105:0/2710770925 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbf37fe640 1 -- 172.21.15.105:0/2710770925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 msgr2=0x5586fbf32690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf32690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbf37fe640 1 -- 172.21.15.105:0/2710770925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf33b70 msgr2=0x5586fbf33ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf33b70 0x5586fbf33ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbf37fe640 1 -- 172.21.15.105:0/2710770925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586fbf34530 con 0x5586fbf29590 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf32690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbf8e05640 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf33b70 0x5586fbf33ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbf37fe640 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf32bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8098320 tx=0x7ffbe809b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324bd9dbda69447c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbd37fe640 1 -- 172.21.15.105:0/2710770925 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe80a12b0 con 0x5586fbf29590 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbd37fe640 1 -- 172.21.15.105:0/2710770925 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ffbe80bad90 con 0x5586fbf29590 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbfd44e740 1 -- 172.21.15.105:0/2710770925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbe8097c10 con 0x5586fbf29590 2024-03-26T17:42:51.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbd37fe640 1 -- 172.21.15.105:0/2710770925 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffbe80a4820 con 0x5586fbf29590 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbfd44e740 10 client.26950 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbfd44e740 1 -- 172.21.15.105:0/2710770925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586fbf35830 con 0x5586fbf29590 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.189+0000 7ffbd37fe640 1 -- 172.21.15.105:0/2710770925 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbe8097c10 con 0x5586fbf29590 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbd37fe640 1 -- 172.21.15.105:0/2710770925 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffbe809b4a0 con 0x5586fbf29590 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbfd44e740 10 client.26950 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbfd44e740 10 client.26950 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbfd44e740 20 client.26950 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5586fbfa07d0 0x5586fbfa2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbfd44e740 4 client.26950 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbfd44e740 1 -- 172.21.15.105:0/2710770925 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586fbe90200 con 0x5586fbfa07d0 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5586fbfa07d0 0x5586fbfa2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbd37fe640 10 client.26950 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbf3fff640 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5586fbfa07d0 0x5586fbfa2bc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=870298eadb028950 server_cookie=78f6c9e440a80de0 in_seq=0 out_seq=0 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbd37fe640 1 -- 172.21.15.105:0/2710770925 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586fbe90200 con 0x5586fbfa07d0 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.190+0000 7ffbd37fe640 10 client.26950 handle_command_reply: tid=0 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 client.26950 shutdown 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 -- 172.21.15.105:0/2710770925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 msgr2=0x5586fbf32bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf32bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8098320 tx=0x7ffbe809b820 comp rx=0 tx=0).stop 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 -- 172.21.15.105:0/2710770925 shutdown_connections 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5586fbf33b70 0x5586fbf33ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5586fbf29590 0x5586fbf32bd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5586fbfa07d0 0x5586fbfa2bc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 --2- 172.21.15.105:0/2710770925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5586fbf23a30 0x5586fbf32690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbd37fe640 0 client.26950 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 -- 172.21.15.105:0/2710770925 >> 172.21.15.105:0/2710770925 conn(0x5586fbf1efa0 msgr2=0x5586fbf2df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 -- 172.21.15.105:0/2710770925 shutdown_connections 2024-03-26T17:42:51.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 1 -- 172.21.15.105:0/2710770925 wait complete. 2024-03-26T17:42:51.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.191+0000 7ffbfd44e740 20 client.26950 trim_cache size 0 max 16384 2024-03-26T17:42:51.196 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:51.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 -- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffbdc0a2f20 msgr2=0x7ffbdc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffbdc0a2f20 0x7ffbdc0a53e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffbec006010 tx=0x7ffbec0324f0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 -- 172.21.15.105:0/4004292347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbf4104cd0 msgr2=0x7ffbf4198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbf4104cd0 0x7ffbf4198030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffbe800e9c0 tx=0x7ffbe800ee80 comp rx=0 tx=0).stop 2024-03-26T17:42:51.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 -- 172.21.15.105:0/4004292347 shutdown_connections 2024-03-26T17:42:51.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffbdc0a2f20 0x7ffbdc0a53e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbf4107600 0x7ffbf419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbf4104cd0 0x7ffbf4198030 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 --2- 172.21.15.105:0/4004292347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbf41023a0 0x7ffbf4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 -- 172.21.15.105:0/4004292347 >> 172.21.15.105:0/4004292347 conn(0x7ffbf40fbd50 msgr2=0x7ffbf4102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 -- 172.21.15.105:0/4004292347 shutdown_connections 2024-03-26T17:42:51.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.192+0000 7ffbfa868640 1 -- 172.21.15.105:0/4004292347 wait complete. 2024-03-26T17:42:51.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:42:51.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 -- 172.21.15.105:0/663925287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 msgr2=0x7f8760107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 --2- 172.21.15.105:0/663925287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 0x7f8760107060 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8750009900 tx=0x7f875002bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 -- 172.21.15.105:0/663925287 shutdown_connections 2024-03-26T17:42:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 --2- 172.21.15.105:0/663925287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87601075a0 0x7f87601099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 --2- 172.21.15.105:0/663925287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 0x7f8760107060 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 --2- 172.21.15.105:0/663925287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8760102340 0x7f8760104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 -- 172.21.15.105:0/663925287 >> 172.21.15.105:0/663925287 conn(0x7f87600fbd50 msgr2=0x7f87600fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 -- 172.21.15.105:0/663925287 shutdown_connections 2024-03-26T17:42:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.359+0000 7f8767732640 1 -- 172.21.15.105:0/663925287 wait complete. 2024-03-26T17:42:51.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8767732640 1 Processor -- start 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8767732640 1 -- start start 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8767732640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8760102340 0x7f8760101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8767732640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 0x7f87601002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8767732640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87601075a0 0x7f8760100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8767732640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f876010d3f0 con 0x7f8760104c70 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8767732640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f876010d270 con 0x7f87601075a0 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8767732640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f876010d570 con 0x7f8760102340 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8764ccd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 0x7f87601002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8764ccd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 0x7f87601002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46820/0 (socket says 172.21.15.105:46820) 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8764ccd640 1 -- 172.21.15.105:0/1731877197 learned_addr learned my addr 172.21.15.105:0/1731877197 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f8765ccf640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87601075a0 0x7f8760100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.360+0000 7f87654ce640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8760102340 0x7f8760101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8764ccd640 1 -- 172.21.15.105:0/1731877197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8760102340 msgr2=0x7f8760101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8764ccd640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8760102340 0x7f8760101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8764ccd640 1 -- 172.21.15.105:0/1731877197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87601075a0 msgr2=0x7f8760100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8764ccd640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87601075a0 0x7f8760100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8764ccd640 1 -- 172.21.15.105:0/1731877197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87601010c0 con 0x7f8760104c70 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8765ccf640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87601075a0 0x7f8760100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8764ccd640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 0x7f87601002f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8750000c00 tx=0x7f8750005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f87567fc640 1 -- 172.21.15.105:0/1731877197 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750043020 con 0x7f8760104c70 2024-03-26T17:42:51.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f87567fc640 1 -- 172.21.15.105:0/1731877197 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8750004350 con 0x7f8760104c70 2024-03-26T17:42:51.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8767732640 1 -- 172.21.15.105:0/1731877197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87601a1c90 con 0x7f8760104c70 2024-03-26T17:42:51.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f87567fc640 1 -- 172.21.15.105:0/1731877197 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750003b80 con 0x7f8760104c70 2024-03-26T17:42:51.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.361+0000 7f8767732640 1 -- 172.21.15.105:0/1731877197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601a2180 con 0x7f8760104c70 2024-03-26T17:42:51.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.363+0000 7f87567fc640 1 -- 172.21.15.105:0/1731877197 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8750046020 con 0x7f8760104c70 2024-03-26T17:42:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f87567fc640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f874c0a2fa0 0x7f874c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f876a3f5740 1 Processor -- start 2024-03-26T17:42:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f876a3f5740 1 -- start start 2024-03-26T17:42:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f87654ce640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f874c0a2fa0 0x7f874c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f87567fc640 1 -- 172.21.15.105:0/1731877197 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f87500f14d0 con 0x7f8760104c70 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614f3590 0x562f614f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f876a3f5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562f61452970 con 0x562f614eda30 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.364+0000 7f876a3f5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562f613fad70 con 0x562f614f3590 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f876a3f5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f6144e920 con 0x562f614ee420 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f87654ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f87654ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46826/0 (socket says 172.21.15.105:46826) 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f87654ce640 1 -- 172.21.15.105:0/956767203 learned_addr learned my addr 172.21.15.105:0/956767203 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f8765ccf640 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614f3590 0x562f614f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f8764ccd640 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f87654ce640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f874c0a2fa0 0x7f874c0a5460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f875c013520 tx=0x7f875c013920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f8765ccf640 1 -- 172.21.15.105:0/956767203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 msgr2=0x562f614f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f8765ccf640 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f8765ccf640 1 -- 172.21.15.105:0/956767203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 msgr2=0x562f614ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f8765ccf640 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614ede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f8765ccf640 1 -- 172.21.15.105:0/956767203 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f611663f0 con 0x562f614f3590 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f87654ce640 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f8765ccf640 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614f3590 0x562f614f5a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8758005fd0 tx=0x7f8758011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1af367da5883c28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f874b7fe640 1 -- 172.21.15.105:0/956767203 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758004d20 con 0x562f614f3590 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f874b7fe640 1 -- 172.21.15.105:0/956767203 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f875800f9a0 con 0x562f614f3590 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.365+0000 7f874b7fe640 1 -- 172.21.15.105:0/956767203 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87580043d0 con 0x562f614f3590 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 -- 172.21.15.105:0/956767203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614f3590 msgr2=0x562f614f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614f3590 0x562f614f5a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8758005fd0 tx=0x7f8758011bf0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 -- 172.21.15.105:0/956767203 shutdown_connections 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614f3590 0x562f614f5a20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 --2- 172.21.15.105:0/956767203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 -- 172.21.15.105:0/956767203 >> 172.21.15.105:0/956767203 conn(0x562f614e8fa0 msgr2=0x562f614e93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 -- 172.21.15.105:0/956767203 shutdown_connections 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.366+0000 7f876a3f5740 1 -- 172.21.15.105:0/956767203 wait complete. 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 1 -- start start 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 1 Processor -- start 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614ee420 0x562f614f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614ff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562f614f6540 con 0x562f614eda30 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562f6140e310 con 0x562f614ee420 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f614f66c0 con 0x562f614f3590 2024-03-26T17:42:51.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f87654ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f8765ccf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f8765ccf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614ff470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36872/0 (socket says 172.21.15.105:36872) 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f87654ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46828/0 (socket says 172.21.15.105:46828) 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f8765ccf640 1 -- 172.21.15.105:0/560935046 learned_addr learned my addr 172.21.15.105:0/560935046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.367+0000 7f876a3f5740 10 client.0 _ll_register_callbacks cb 0x562f61449c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8764ccd640 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614ee420 0x562f614f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8765ccf640 1 -- 172.21.15.105:0/560935046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614ee420 msgr2=0x562f614f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8765ccf640 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614ee420 0x562f614f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8765ccf640 1 -- 172.21.15.105:0/560935046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 msgr2=0x562f614f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8765ccf640 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8765ccf640 1 -- 172.21.15.105:0/560935046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f612307d0 con 0x562f614f3590 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8764ccd640 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614ee420 0x562f614f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:42:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f87654ce640 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8765ccf640 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614ff470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8758005c10 tx=0x7f8758003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89d3b34b2c1b4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8748ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8748ff9640 1 -- 172.21.15.105:0/560935046 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758005000 con 0x562f614f3590 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8748ff9640 1 -- 172.21.15.105:0/560935046 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f87580051a0 con 0x562f614f3590 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f876a3f5740 1 -- 172.21.15.105:0/560935046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f875800f9a0 con 0x562f614f3590 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f8748ff9640 1 -- 172.21.15.105:0/560935046 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87580053a0 con 0x562f614f3590 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f876a3f5740 10 client.26962 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.368+0000 7f876a3f5740 1 -- 172.21.15.105:0/560935046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f61502810 con 0x562f614f3590 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f8748ff9640 1 -- 172.21.15.105:0/560935046 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8758029070 con 0x562f614f3590 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f8748ff9640 1 -- 172.21.15.105:0/560935046 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8758018070 con 0x562f614f3590 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f876a3f5740 10 client.26962 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f876a3f5740 10 client.26962 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:42:51.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f876a3f5740 20 client.26962 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f876a3f5740 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562f61503d20 0x562f615061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f876a3f5740 4 client.26962 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f876a3f5740 1 -- 172.21.15.105:0/560935046 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f875800f9a0 con 0x562f61503d20 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f87654ce640 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562f61503d20 0x562f615061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.369+0000 7f8748ff9640 10 client.26962 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f87654ce640 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562f61503d20 0x562f615061c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2af5a909a083689c server_cookie=1c125e7583a6d39 in_seq=0 out_seq=0 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f8748ff9640 1 -- 172.21.15.105:0/560935046 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f875800f9a0 con 0x562f61503d20 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f8748ff9640 10 client.26962 handle_command_reply: tid=0 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f876a3f5740 1 client.26962 shutdown 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f876a3f5740 1 -- 172.21.15.105:0/560935046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 msgr2=0x562f614ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f876a3f5740 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614ff470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8758005c10 tx=0x7f8758003ce0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f876a3f5740 1 -- 172.21.15.105:0/560935046 shutdown_connections 2024-03-26T17:42:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f876a3f5740 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562f61503d20 0x562f615061c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.370+0000 7f876a3f5740 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614ff470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.371+0000 7f876a3f5740 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614ee420 0x562f614f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.371+0000 7f8748ff9640 0 client.26962 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:42:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.371+0000 7f876a3f5740 1 --2- 172.21.15.105:0/560935046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614eda30 0x562f614f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.371+0000 7f876a3f5740 1 -- 172.21.15.105:0/560935046 >> 172.21.15.105:0/560935046 conn(0x562f614e8fa0 msgr2=0x562f614e81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.371+0000 7f876a3f5740 1 -- 172.21.15.105:0/560935046 shutdown_connections 2024-03-26T17:42:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.371+0000 7f876a3f5740 1 -- 172.21.15.105:0/560935046 wait complete. 2024-03-26T17:42:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.371+0000 7f876a3f5740 20 client.26962 trim_cache size 0 max 16384 2024-03-26T17:42:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f876a3f5740 1 Processor -- start 2024-03-26T17:42:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f876a3f5740 1 -- start start 2024-03-26T17:42:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614ee420 0x562f614ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f876a3f5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87601aaf40 con 0x562f614ee420 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f876a3f5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87601aca80 con 0x562f614eda30 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f876a3f5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ab610 con 0x562f614f3590 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8764ccd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614ee420 0x562f614ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8764ccd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614ee420 0x562f614ff430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46840/0 (socket says 172.21.15.105:46840) 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f87654ce640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8764ccd640 1 -- 172.21.15.105:0/3403420721 learned_addr learned my addr 172.21.15.105:0/3403420721 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8765ccf640 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8764ccd640 1 -- 172.21.15.105:0/3403420721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 msgr2=0x562f614e7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8764ccd640 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614e7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8764ccd640 1 -- 172.21.15.105:0/3403420721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 msgr2=0x562f614ec910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8764ccd640 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614ec910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f8764ccd640 1 -- 172.21.15.105:0/3403420721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f614e4820 con 0x562f614ee420 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.390+0000 7f87654ce640 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614ec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.391+0000 7f8764ccd640 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614ee420 0x562f614ff430 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f87500ef300 tx=0x7f8750041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911cd2b8b22bb339 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.391+0000 7f874affd640 1 -- 172.21.15.105:0/3403420721 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500bdbc0 con 0x562f614ee420 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.391+0000 7f874affd640 1 -- 172.21.15.105:0/3403420721 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87500c1d10 con 0x562f614ee420 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.391+0000 7f874affd640 1 -- 172.21.15.105:0/3403420721 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500bde20 con 0x562f614ee420 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.391+0000 7f876a3f5740 1 -- 172.21.15.105:0/3403420721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614ee420 msgr2=0x562f614ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.391+0000 7f876a3f5740 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614ee420 0x562f614ff430 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f87500ef300 tx=0x7f8750041040 comp rx=0 tx=0).stop 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 -- 172.21.15.105:0/3403420721 shutdown_connections 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614f3590 0x562f614e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614ee420 0x562f614ff430 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 --2- 172.21.15.105:0/3403420721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614ec910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 -- 172.21.15.105:0/3403420721 >> 172.21.15.105:0/3403420721 conn(0x562f614e8fa0 msgr2=0x562f614ec200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 -- 172.21.15.105:0/3403420721 shutdown_connections 2024-03-26T17:42:51.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 -- 172.21.15.105:0/3403420721 wait complete. 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 -- start start 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 Processor -- start 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614fc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614f3590 0x562f615680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87601ac3b0 con 0x562f614f3590 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87601abce0 con 0x562f614eda30 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87601a8f20 con 0x562f614ee420 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f876a3f5740 10 client.0 _ll_register_callbacks cb 0x7f875c009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.392+0000 7f8765ccf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614f3590 0x562f615680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f87654ce640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8765ccf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614f3590 0x562f615680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46846/0 (socket says 172.21.15.105:46846) 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f87654ce640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614fc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53568/0 (socket says 172.21.15.105:53568) 2024-03-26T17:42:51.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8765ccf640 1 -- 172.21.15.105:0/407251639 learned_addr learned my addr 172.21.15.105:0/407251639 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8764ccd640 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8765ccf640 1 -- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 msgr2=0x562f614fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8765ccf640 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614fcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8765ccf640 1 -- 172.21.15.105:0/407251639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 msgr2=0x562f614fc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8765ccf640 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614fc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8765ccf640 1 -- 172.21.15.105:0/407251639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f61568670 con 0x562f614f3590 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8764ccd640 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f87654ce640 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8765ccf640 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614f3590 0x562f615680b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x562f614e4af0 tx=0x7f8758009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c9ab959d1d2cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8748ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8748ff9640 1 -- 172.21.15.105:0/407251639 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758005540 con 0x562f614f3590 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f876a3f5740 1 -- 172.21.15.105:0/407251639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87500c1d10 con 0x562f614f3590 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8748ff9640 1 -- 172.21.15.105:0/407251639 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f87580056e0 con 0x562f614f3590 2024-03-26T17:42:51.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f8748ff9640 1 -- 172.21.15.105:0/407251639 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87580058e0 con 0x562f614f3590 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f876a3f5740 10 client.17130 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.393+0000 7f876a3f5740 1 -- 172.21.15.105:0/407251639 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f61569970 con 0x562f614f3590 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f8748ff9640 1 -- 172.21.15.105:0/407251639 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87500c1d10 con 0x562f614f3590 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f8748ff9640 1 -- 172.21.15.105:0/407251639 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8758018070 con 0x562f614f3590 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f876a3f5740 10 client.17130 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f876a3f5740 10 client.17130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f876a3f5740 20 client.17130 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f876a3f5740 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562f61503d20 0x562f6156c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f876a3f5740 4 client.17130 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f876a3f5740 1 -- 172.21.15.105:0/407251639 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f6145feb0 con 0x562f61503d20 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f87654ce640 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562f61503d20 0x562f6156c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f87654ce640 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562f61503d20 0x562f6156c360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b17a5708e7e986d7 server_cookie=6c008c287fe2f43c in_seq=0 out_seq=0 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.394+0000 7f8748ff9640 10 client.17130 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f8748ff9640 1 -- 172.21.15.105:0/407251639 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f6145feb0 con 0x562f61503d20 2024-03-26T17:42:51.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f8748ff9640 10 client.17130 handle_command_reply: tid=0 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 client.17130 shutdown 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 -- 172.21.15.105:0/407251639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614f3590 msgr2=0x562f615680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614f3590 0x562f615680b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x562f614e4af0 tx=0x7f8758009fd0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 -- 172.21.15.105:0/407251639 shutdown_connections 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562f61503d20 0x562f6156c360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f614f3590 0x562f615680b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f8748ff9640 0 client.17130 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f614ee420 0x562f614fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 --2- 172.21.15.105:0/407251639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f614eda30 0x562f614fc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 -- 172.21.15.105:0/407251639 >> 172.21.15.105:0/407251639 conn(0x562f614e8fa0 msgr2=0x562f614eca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 -- 172.21.15.105:0/407251639 shutdown_connections 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 1 -- 172.21.15.105:0/407251639 wait complete. 2024-03-26T17:42:51.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.395+0000 7f876a3f5740 20 client.17130 trim_cache size 0 max 16384 2024-03-26T17:42:51.397 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 -- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f874c0a2fa0 msgr2=0x7f874c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f874c0a2fa0 0x7f874c0a5460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f875c013520 tx=0x7f875c013920 comp rx=0 tx=0).stop 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 -- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 msgr2=0x7f87601002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 0x7f87601002f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8750000c00 tx=0x7f8750005dc0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 -- 172.21.15.105:0/1731877197 shutdown_connections 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f874c0a2fa0 0x7f874c0a5460 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87601075a0 0x7f8760100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8760104c70 0x7f87601002f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 --2- 172.21.15.105:0/1731877197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8760102340 0x7f8760101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 -- 172.21.15.105:0/1731877197 >> 172.21.15.105:0/1731877197 conn(0x7f87600fbd50 msgr2=0x7f87601041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 -- 172.21.15.105:0/1731877197 shutdown_connections 2024-03-26T17:42:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.396+0000 7f8767732640 1 -- 172.21.15.105:0/1731877197 wait complete. 2024-03-26T17:42:51.401 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:42:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 -- 172.21.15.105:0/163685110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a0102370 msgr2=0x7f94a0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 --2- 172.21.15.105:0/163685110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a0102370 0x7f94a0108c00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9498009ca0 tx=0x7f949802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 -- 172.21.15.105:0/163685110 shutdown_connections 2024-03-26T17:42:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 --2- 172.21.15.105:0/163685110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a0102370 0x7f94a0108c00 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 --2- 172.21.15.105:0/163685110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a01019b0 0x7f94a0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 --2- 172.21.15.105:0/163685110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a01007c0 0x7f94a0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 -- 172.21.15.105:0/163685110 >> 172.21.15.105:0/163685110 conn(0x7f94a00fbd50 msgr2=0x7f94a00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 -- 172.21.15.105:0/163685110 shutdown_connections 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 -- 172.21.15.105:0/163685110 wait complete. 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 Processor -- start 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.522+0000 7f94a4eac640 1 -- start start 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f94a4eac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a01007c0 0x7f94a0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f94a4eac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a01019b0 0x7f94a019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f94a4eac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a0102370 0x7f94a01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f94a4eac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94a010c2d0 con 0x7f94a01007c0 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f94a4eac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94a010c150 con 0x7f94a01019b0 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f94a4eac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94a010c450 con 0x7f94a0102370 2024-03-26T17:42:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a01007c0 0x7f94a0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a01007c0 0x7f94a0069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46872/0 (socket says 172.21.15.105:46872) 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a0102370 0x7f94a01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 -- 172.21.15.105:0/1078150279 learned_addr learned my addr 172.21.15.105:0/1078150279 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f9497fff640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a01019b0 0x7f94a019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 -- 172.21.15.105:0/1078150279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a0102370 msgr2=0x7f94a01a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a0102370 0x7f94a01a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 -- 172.21.15.105:0/1078150279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a01019b0 msgr2=0x7f94a019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a01019b0 0x7f94a019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 -- 172.21.15.105:0/1078150279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a01a1990 con 0x7f94a01007c0 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949ed9d640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a0102370 0x7f94a01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f949e59c640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a01007c0 0x7f94a0069590 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f949000e9c0 tx=0x7f949000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.523+0000 7f94977fe640 1 -- 172.21.15.105:0/1078150279 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949000cd80 con 0x7f94a01007c0 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.524+0000 7f94977fe640 1 -- 172.21.15.105:0/1078150279 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94900035f0 con 0x7f94a01007c0 2024-03-26T17:42:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.524+0000 7f94a4eac640 1 -- 172.21.15.105:0/1078150279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a01a1c20 con 0x7f94a01007c0 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.524+0000 7f94977fe640 1 -- 172.21.15.105:0/1078150279 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490010690 con 0x7f94a01007c0 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.524+0000 7f94a4eac640 1 -- 172.21.15.105:0/1078150279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a01a2170 con 0x7f94a01007c0 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94977fe640 1 -- 172.21.15.105:0/1078150279 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f94900037a0 con 0x7f94a01007c0 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94a7b75740 1 Processor -- start 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94a7b75740 1 -- start start 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af6fa30 0x55af5af6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af70420 0x55af5af74f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af77a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94a7b75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55af5aed4970 con 0x55af5af75590 2024-03-26T17:42:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94a7b75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55af5ae7cd70 con 0x55af5af6fa30 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f94a7b75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55af5aed0920 con 0x55af5af70420 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.526+0000 7f9497fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af77a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af6fa30 0x55af5af6fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af6fa30 0x55af5af6fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53602/0 (socket says 172.21.15.105:53602) 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9497fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af77a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46888/0 (socket says 172.21.15.105:46888) 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f949ed9d640 1 -- 172.21.15.105:0/665650604 learned_addr learned my addr 172.21.15.105:0/665650604 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f949e59c640 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af70420 0x55af5af74f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9497fff640 1 -- 172.21.15.105:0/665650604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af70420 msgr2=0x55af5af74f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9497fff640 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af70420 0x55af5af74f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9497fff640 1 -- 172.21.15.105:0/665650604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af6fa30 msgr2=0x55af5af6fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9497fff640 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af6fa30 0x55af5af6fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9497fff640 1 -- 172.21.15.105:0/665650604 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af5abe83f0 con 0x55af5af75590 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f949ed9d640 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af6fa30 0x55af5af6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f94977fe640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f948c0a2f50 0x7f948c0a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9497fff640 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af77a20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9488009bf0 tx=0x7f9488011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c10efa947ae910 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f949ed9d640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f948c0a2f50 0x7f948c0a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f94977fe640 1 -- 172.21.15.105:0/1078150279 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f94900cf760 con 0x7f94a01007c0 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9494ff9640 1 -- 172.21.15.105:0/665650604 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948801e070 con 0x55af5af75590 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9494ff9640 1 -- 172.21.15.105:0/665650604 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9488009da0 con 0x55af5af75590 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.527+0000 7f9494ff9640 1 -- 172.21.15.105:0/665650604 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94880187d0 con 0x55af5af75590 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f94a7b75740 1 -- 172.21.15.105:0/665650604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 msgr2=0x55af5af77a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f949ed9d640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f948c0a2f50 0x7f948c0a5410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55af5af70c80 tx=0x7f94980324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f94a7b75740 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af77a20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9488009bf0 tx=0x7f9488011810 comp rx=0 tx=0).stop 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f94a7b75740 1 -- 172.21.15.105:0/665650604 shutdown_connections 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f94a7b75740 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af77a20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f94a7b75740 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af70420 0x55af5af74f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f94a7b75740 1 --2- 172.21.15.105:0/665650604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af6fa30 0x55af5af6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f94a7b75740 1 -- 172.21.15.105:0/665650604 >> 172.21.15.105:0/665650604 conn(0x55af5af6afa0 msgr2=0x55af5af6b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.528+0000 7f94a7b75740 1 -- 172.21.15.105:0/665650604 shutdown_connections 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 -- 172.21.15.105:0/665650604 wait complete. 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 -- start start 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 Processor -- start 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af79f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af70420 0x55af5af7a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af75590 0x55af5af814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55af5af78540 con 0x55af5af70420 2024-03-26T17:42:51.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55af5ae90310 con 0x55af5af75590 2024-03-26T17:42:51.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55af5af786c0 con 0x55af5af6fa30 2024-03-26T17:42:51.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f94a7b75740 10 client.0 _ll_register_callbacks cb 0x55af5aecbc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af75590 0x55af5af814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f949e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af79f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af75590 0x55af5af814d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53612/0 (socket says 172.21.15.105:53612) 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.529+0000 7f949ed9d640 1 -- 172.21.15.105:0/605309690 learned_addr learned my addr 172.21.15.105:0/605309690 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f9497fff640 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af70420 0x55af5af7a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f949ed9d640 1 -- 172.21.15.105:0/605309690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 msgr2=0x55af5af79f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f949ed9d640 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af79f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f949ed9d640 1 -- 172.21.15.105:0/605309690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af70420 msgr2=0x55af5af7a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f949ed9d640 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af70420 0x55af5af7a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f949ed9d640 1 -- 172.21.15.105:0/605309690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5acb27d0 con 0x55af5af75590 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f949ed9d640 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af75590 0x55af5af814d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f949802c700 tx=0x7f94980095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272c8c291e9a2a55 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f9485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f9485ffb640 1 -- 172.21.15.105:0/605309690 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498050020 con 0x55af5af75590 2024-03-26T17:42:51.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f9485ffb640 1 -- 172.21.15.105:0/605309690 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9498004590 con 0x55af5af75590 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f94a7b75740 1 -- 172.21.15.105:0/605309690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9488009da0 con 0x55af5af75590 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f9485ffb640 1 -- 172.21.15.105:0/605309690 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f949805b050 con 0x55af5af75590 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f94a7b75740 10 client.26978 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.530+0000 7f94a7b75740 1 -- 172.21.15.105:0/605309690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af5af848b0 con 0x55af5af75590 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f9485ffb640 1 -- 172.21.15.105:0/605309690 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9488009da0 con 0x55af5af75590 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f9485ffb640 1 -- 172.21.15.105:0/605309690 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f949804ccd0 con 0x55af5af75590 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f94a7b75740 10 client.26978 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f94a7b75740 10 client.26978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f94a7b75740 20 client.26978 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f94a7b75740 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55af5af85fe0 0x55af5af88480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f94a7b75740 4 client.26978 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:51.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f94a7b75740 1 -- 172.21.15.105:0/605309690 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af5aee1eb0 con 0x55af5af85fe0 2024-03-26T17:42:51.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.531+0000 7f949e59c640 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55af5af85fe0 0x55af5af88480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.532+0000 7f949e59c640 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55af5af85fe0 0x55af5af88480 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b3a3ad52b566f server_cookie=f283e58f989e1ed in_seq=0 out_seq=0 2024-03-26T17:42:51.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.532+0000 7f9485ffb640 10 client.26978 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:51.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f9485ffb640 1 -- 172.21.15.105:0/605309690 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55af5aee1eb0 con 0x55af5af85fe0 2024-03-26T17:42:51.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f9485ffb640 10 client.26978 handle_command_reply: tid=0 2024-03-26T17:42:51.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 client.26978 shutdown 2024-03-26T17:42:51.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 -- 172.21.15.105:0/605309690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af75590 msgr2=0x55af5af814d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af75590 0x55af5af814d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f949802c700 tx=0x7f94980095c0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 -- 172.21.15.105:0/605309690 shutdown_connections 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55af5af85fe0 0x55af5af88480 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af75590 0x55af5af814d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af70420 0x55af5af7a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 --2- 172.21.15.105:0/605309690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af79f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 -- 172.21.15.105:0/605309690 >> 172.21.15.105:0/605309690 conn(0x55af5af6afa0 msgr2=0x55af5af6a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f9485ffb640 0 client.26978 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 -- 172.21.15.105:0/605309690 shutdown_connections 2024-03-26T17:42:51.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 1 -- 172.21.15.105:0/605309690 wait complete. 2024-03-26T17:42:51.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.534+0000 7f94a7b75740 20 client.26978 trim_cache size 0 max 16384 2024-03-26T17:42:51.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f94a7b75740 1 Processor -- start 2024-03-26T17:42:51.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f94a7b75740 1 -- start start 2024-03-26T17:42:51.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af81490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af69e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f94a7b75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01aaff0 con 0x55af5af75590 2024-03-26T17:42:51.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f94a7b75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01acb30 con 0x55af5af70420 2024-03-26T17:42:51.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f94a7b75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01ab6c0 con 0x55af5af6fa30 2024-03-26T17:42:51.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af69e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f9497fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af81490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af69e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46916/0 (socket says 172.21.15.105:46916) 2024-03-26T17:42:51.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 -- 172.21.15.105:0/3831481189 learned_addr learned my addr 172.21.15.105:0/3831481189 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949e59c640 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 -- 172.21.15.105:0/3831481189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 msgr2=0x55af5af69930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af69930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 -- 172.21.15.105:0/3831481189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 msgr2=0x55af5af81490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af81490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 -- 172.21.15.105:0/3831481189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af5af6bd10 con 0x55af5af75590 2024-03-26T17:42:51.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f9497fff640 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af81490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.550+0000 7f949ed9d640 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af69e70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9498034590 tx=0x7f9498032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b91c81ad1a40e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f9487fff640 1 -- 172.21.15.105:0/3831481189 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498063030 con 0x55af5af75590 2024-03-26T17:42:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f9487fff640 1 -- 172.21.15.105:0/3831481189 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9498003e80 con 0x55af5af75590 2024-03-26T17:42:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f9487fff640 1 -- 172.21.15.105:0/3831481189 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498050070 con 0x55af5af75590 2024-03-26T17:42:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f94a7b75740 1 -- 172.21.15.105:0/3831481189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 msgr2=0x55af5af69e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f94a7b75740 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af69e70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9498034590 tx=0x7f9498032f60 comp rx=0 tx=0).stop 2024-03-26T17:42:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f94a7b75740 1 -- 172.21.15.105:0/3831481189 shutdown_connections 2024-03-26T17:42:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f94a7b75740 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af69e70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f94a7b75740 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af81490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f94a7b75740 1 --2- 172.21.15.105:0/3831481189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af69930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.551+0000 7f94a7b75740 1 -- 172.21.15.105:0/3831481189 >> 172.21.15.105:0/3831481189 conn(0x55af5af6afa0 msgr2=0x55af5af718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 -- 172.21.15.105:0/3831481189 shutdown_connections 2024-03-26T17:42:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 -- 172.21.15.105:0/3831481189 wait complete. 2024-03-26T17:42:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 -- start start 2024-03-26T17:42:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 Processor -- start 2024-03-26T17:42:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af7e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af7eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af7fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01ac460 con 0x55af5af75590 2024-03-26T17:42:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01abd90 con 0x55af5af70420 2024-03-26T17:42:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94a01a8fd0 con 0x55af5af6fa30 2024-03-26T17:42:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f94a7b75740 10 client.0 _ll_register_callbacks cb 0x7f9498044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af7fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f9497fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af7eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f949ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af7fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46918/0 (socket says 172.21.15.105:46918) 2024-03-26T17:42:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f949ed9d640 1 -- 172.21.15.105:0/2167491204 learned_addr learned my addr 172.21.15.105:0/2167491204 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.552+0000 7f949e59c640 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af7e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f949ed9d640 1 -- 172.21.15.105:0/2167491204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 msgr2=0x55af5af7e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f949ed9d640 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af7e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f949ed9d640 1 -- 172.21.15.105:0/2167491204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 msgr2=0x55af5af7eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f949ed9d640 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af7eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f949ed9d640 1 -- 172.21.15.105:0/2167491204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5af80140 con 0x55af5af75590 2024-03-26T17:42:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f9497fff640 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af7eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f949e59c640 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af7e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f949ed9d640 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af7fb50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f949802c630 tx=0x7f94980349b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf555fd0feb876dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f9485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f9485ffb640 1 -- 172.21.15.105:0/2167491204 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949806c060 con 0x55af5af75590 2024-03-26T17:42:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f9485ffb640 1 -- 172.21.15.105:0/2167491204 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9498004620 con 0x55af5af75590 2024-03-26T17:42:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f94a7b75740 1 -- 172.21.15.105:0/2167491204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af5aedc200 con 0x55af5af75590 2024-03-26T17:42:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f9485ffb640 1 -- 172.21.15.105:0/2167491204 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9498063030 con 0x55af5af75590 2024-03-26T17:42:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f94a7b75740 10 client.17148 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f94a7b75740 1 -- 172.21.15.105:0/2167491204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af5af81440 con 0x55af5af75590 2024-03-26T17:42:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.553+0000 7f9485ffb640 1 -- 172.21.15.105:0/2167491204 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55af5aedc200 con 0x55af5af75590 2024-03-26T17:42:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f9485ffb640 1 -- 172.21.15.105:0/2167491204 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f949805f430 con 0x55af5af75590 2024-03-26T17:42:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f94a7b75740 10 client.17148 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f94a7b75740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f94a7b75740 20 client.17148 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f94a7b75740 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55af5afec6c0 0x55af5afeeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f94a7b75740 4 client.17148 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:42:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f94a7b75740 1 -- 172.21.15.105:0/2167491204 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af5ae596a0 con 0x55af5afec6c0 2024-03-26T17:42:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f949e59c640 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55af5afec6c0 0x55af5afeeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f9485ffb640 10 client.17148 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.554+0000 7f949e59c640 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55af5afec6c0 0x55af5afeeab0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1adc5c6986fd1033 server_cookie=906f6a9cc3c58815 in_seq=0 out_seq=0 2024-03-26T17:42:51.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f9485ffb640 1 -- 172.21.15.105:0/2167491204 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af5ae596a0 con 0x55af5afec6c0 2024-03-26T17:42:51.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f9485ffb640 10 client.17148 handle_command_reply: tid=0 2024-03-26T17:42:51.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 client.17148 shutdown 2024-03-26T17:42:51.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 -- 172.21.15.105:0/2167491204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 msgr2=0x55af5af7fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af7fb50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f949802c630 tx=0x7f94980349b0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 -- 172.21.15.105:0/2167491204 shutdown_connections 2024-03-26T17:42:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55af5afec6c0 0x55af5afeeab0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f9485ffb640 0 client.17148 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55af5af75590 0x55af5af7fb50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af5af70420 0x55af5af7eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 --2- 172.21.15.105:0/2167491204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55af5af6fa30 0x55af5af7e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 -- 172.21.15.105:0/2167491204 >> 172.21.15.105:0/2167491204 conn(0x55af5af6afa0 msgr2=0x55af5af6e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 -- 172.21.15.105:0/2167491204 shutdown_connections 2024-03-26T17:42:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 1 -- 172.21.15.105:0/2167491204 wait complete. 2024-03-26T17:42:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.555+0000 7f94a7b75740 20 client.17148 trim_cache size 0 max 16384 2024-03-26T17:42:51.566 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:42:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 -- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f948c0a2f50 msgr2=0x7f948c0a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f948c0a2f50 0x7f948c0a5410 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55af5af70c80 tx=0x7f94980324f0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 -- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a01007c0 msgr2=0x7f94a0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a01007c0 0x7f94a0069590 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f949000e9c0 tx=0x7f949000ee80 comp rx=0 tx=0).stop 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 -- 172.21.15.105:0/1078150279 shutdown_connections 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f948c0a2f50 0x7f948c0a5410 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a0102370 0x7f94a01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a01019b0 0x7f94a019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 --2- 172.21.15.105:0/1078150279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94a01007c0 0x7f94a0069590 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 -- 172.21.15.105:0/1078150279 >> 172.21.15.105:0/1078150279 conn(0x7f94a00fbd50 msgr2=0x7f94a0104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 -- 172.21.15.105:0/1078150279 shutdown_connections 2024-03-26T17:42:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:51.556+0000 7f94a4eac640 1 -- 172.21.15.105:0/1078150279 wait complete. 2024-03-26T17:42:51.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:51 smithi105 ceph-mon[32280]: pgmap v483: 97 pgs: 97 active+clean; 30 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:42:52.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 126841 3.12 MB/sec execute 95 sec latency 31.399 ms 2024-03-26T17:42:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:51 smithi144 ceph-mon[36095]: pgmap v483: 97 pgs: 97 active+clean; 30 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:42:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:51 smithi135 ceph-mon[35636]: pgmap v483: 97 pgs: 97 active+clean; 30 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:42:52.571 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:42:52.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:42:52.571 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:42:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 -- 172.21.15.105:0/1571830926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac1007c0 msgr2=0x7f01ac100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/1571830926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac1007c0 0x7f01ac100be0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009900 tx=0x7f01a802bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 -- 172.21.15.105:0/1571830926 shutdown_connections 2024-03-26T17:42:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/1571830926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ac102370 0x7f01ac108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/1571830926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac1019b0 0x7f01ac101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/1571830926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac1007c0 0x7f01ac100be0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 -- 172.21.15.105:0/1571830926 >> 172.21.15.105:0/1571830926 conn(0x7f01ac0fbd50 msgr2=0x7f01ac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 -- 172.21.15.105:0/1571830926 shutdown_connections 2024-03-26T17:42:52.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 -- 172.21.15.105:0/1571830926 wait complete. 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 Processor -- start 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.709+0000 7f01b3ecc640 1 -- start start 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b3ecc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac1007c0 0x7f01ac104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b3ecc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ac1019b0 0x7f01ac102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b3ecc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac102370 0x7f01ac1032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b3ecc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10c2c0 con 0x7f01ac102370 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b3ecc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10c140 con 0x7f01ac1007c0 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b3ecc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10c440 con 0x7f01ac1019b0 2024-03-26T17:42:52.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac102370 0x7f01ac1032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac102370 0x7f01ac1032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46936/0 (socket says 172.21.15.105:46936) 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 -- 172.21.15.105:0/3742453042 learned_addr learned my addr 172.21.15.105:0/3742453042 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b1467640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ac1019b0 0x7f01ac102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b1c68640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac1007c0 0x7f01ac104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 -- 172.21.15.105:0/3742453042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ac1019b0 msgr2=0x7f01ac102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ac1019b0 0x7f01ac102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 -- 172.21.15.105:0/3742453042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac1007c0 msgr2=0x7f01ac104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac1007c0 0x7f01ac104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 -- 172.21.15.105:0/3742453042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac103b60 con 0x7f01ac102370 2024-03-26T17:42:52.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b2469640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac102370 0x7f01ac1032d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f01a400efd0 tx=0x7f01a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01a2ffd640 1 -- 172.21.15.105:0/3742453042 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4003b70 con 0x7f01ac102370 2024-03-26T17:42:52.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b3ecc640 1 -- 172.21.15.105:0/3742453042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac1a1cc0 con 0x7f01ac102370 2024-03-26T17:42:52.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01a2ffd640 1 -- 172.21.15.105:0/3742453042 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01a4003d10 con 0x7f01ac102370 2024-03-26T17:42:52.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01a2ffd640 1 -- 172.21.15.105:0/3742453042 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4010980 con 0x7f01ac102370 2024-03-26T17:42:52.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.710+0000 7f01b3ecc640 1 -- 172.21.15.105:0/3742453042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac1a21c0 con 0x7f01ac102370 2024-03-26T17:42:52.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.713+0000 7f01a2ffd640 1 -- 172.21.15.105:0/3742453042 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f01a4004020 con 0x7f01ac102370 2024-03-26T17:42:52.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.713+0000 7f01b6b8b740 1 Processor -- start 2024-03-26T17:42:52.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b6b8b740 1 -- start start 2024-03-26T17:42:52.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbf15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dbf6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dbf9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558a8db56630 con 0x558a8dbf1190 2024-03-26T17:42:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a8dafee10 con 0x558a8dbf6cf0 2024-03-26T17:42:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558a8db525e0 con 0x558a8dbf1b80 2024-03-26T17:42:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01a2ffd640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01980a2f20 0x7f01980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1467640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbf15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1c68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dbf6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1467640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbf15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46942/0 (socket says 172.21.15.105:46942) 2024-03-26T17:42:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1467640 1 -- 172.21.15.105:0/635935263 learned_addr learned my addr 172.21.15.105:0/635935263 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b2469640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01980a2f20 0x7f01980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1467640 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dbf9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01a2ffd640 1 -- 172.21.15.105:0/3742453042 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f01a40cf870 con 0x7f01ac102370 2024-03-26T17:42:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1c68640 1 -- 172.21.15.105:0/635935263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 msgr2=0x558a8dbf9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1c68640 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dbf9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1c68640 1 -- 172.21.15.105:0/635935263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 msgr2=0x558a8dbf15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1c68640 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbf15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1c68640 1 -- 172.21.15.105:0/635935263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8db15740 con 0x558a8dbf1b80 2024-03-26T17:42:52.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.714+0000 7f01b1467640 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dbf9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.715+0000 7f01b1c68640 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dbf6680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01a8036040 tx=0x7f01a8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac9aa9ab898a8c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.715+0000 7f01b2469640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01980a2f20 0x7f01980a53e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f01a401b000 tx=0x7f01a4066f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.715+0000 7f0197fff640 1 -- 172.21.15.105:0/635935263 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8045020 con 0x558a8dbf1b80 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.715+0000 7f0197fff640 1 -- 172.21.15.105:0/635935263 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01a8004040 con 0x558a8dbf1b80 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.715+0000 7f0197fff640 1 -- 172.21.15.105:0/635935263 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a80046c0 con 0x558a8dbf1b80 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 -- 172.21.15.105:0/635935263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 msgr2=0x558a8dbf6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dbf6680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01a8036040 tx=0x7f01a8009560 comp rx=0 tx=0).stop 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 -- 172.21.15.105:0/635935263 shutdown_connections 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dbf9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dbf6680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/635935263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbf15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 -- 172.21.15.105:0/635935263 >> 172.21.15.105:0/635935263 conn(0x558a8dbec700 msgr2=0x558a8dbecb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 -- 172.21.15.105:0/635935263 shutdown_connections 2024-03-26T17:42:52.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 -- 172.21.15.105:0/635935263 wait complete. 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 -- start start 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.716+0000 7f01b6b8b740 1 Processor -- start 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbfb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1b80 0x558a8dbfbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf6cf0 0x558a8dc02b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558a8dbf9c40 con 0x558a8dbf1190 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a8db121a0 con 0x558a8dbf1b80 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558a8dbf9dc0 con 0x558a8dbf6cf0 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b6b8b740 10 client.0 _ll_register_callbacks cb 0x558a8d869c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1467640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbfb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1c68640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1b80 0x558a8dbfbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1c68640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1b80 0x558a8dbfbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53660/0 (socket says 172.21.15.105:53660) 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1c68640 1 -- 172.21.15.105:0/1240504863 learned_addr learned my addr 172.21.15.105:0/1240504863 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1467640 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf6cf0 0x558a8dc02b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1467640 1 -- 172.21.15.105:0/1240504863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf6cf0 msgr2=0x558a8dc02b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1467640 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf6cf0 0x558a8dc02b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1467640 1 -- 172.21.15.105:0/1240504863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1b80 msgr2=0x558a8dbfbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1467640 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1b80 0x558a8dbfbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1467640 1 -- 172.21.15.105:0/1240504863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8d9327e0 con 0x558a8dbf1190 2024-03-26T17:42:52.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1c68640 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1b80 0x558a8dbfbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01b1467640 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbfb600 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x558a8dbf23e0 tx=0x7f019c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ba4b45345fcdd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.717+0000 7f01957fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01957fa640 1 -- 172.21.15.105:0/1240504863 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c014030 con 0x558a8dbf1190 2024-03-26T17:42:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01957fa640 1 -- 172.21.15.105:0/1240504863 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f019c00fe30 con 0x558a8dbf1190 2024-03-26T17:42:52.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1240504863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a8004040 con 0x558a8dbf1190 2024-03-26T17:42:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01957fa640 1 -- 172.21.15.105:0/1240504863 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f019c029050 con 0x558a8dbf1190 2024-03-26T17:42:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 10 client.17160 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1240504863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a8d992f70 con 0x558a8dbf1190 2024-03-26T17:42:52.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01957fa640 1 -- 172.21.15.105:0/1240504863 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01a8004040 con 0x558a8dbf1190 2024-03-26T17:42:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01957fa640 1 -- 172.21.15.105:0/1240504863 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f019c020370 con 0x558a8dbf1190 2024-03-26T17:42:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 10 client.17160 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 20 client.17160 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:52.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558a8dc07390 0x558a8dc09830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 4 client.17160 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.718+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1240504863 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a8db63b70 con 0x558a8dc07390 2024-03-26T17:42:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.719+0000 7f01b1c68640 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558a8dc07390 0x558a8dc09830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.719+0000 7f01b1c68640 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558a8dc07390 0x558a8dc09830 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3c1778c7652973 server_cookie=d578ab58f4937e2f in_seq=0 out_seq=0 2024-03-26T17:42:52.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.719+0000 7f01957fa640 10 client.17160 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01957fa640 1 -- 172.21.15.105:0/1240504863 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558a8db63b70 con 0x558a8dc07390 2024-03-26T17:42:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01957fa640 10 client.17160 handle_command_reply: tid=0 2024-03-26T17:42:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 client.17160 shutdown 2024-03-26T17:42:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1240504863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 msgr2=0x558a8dbfb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbfb600 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x558a8dbf23e0 tx=0x7f019c005f90 comp rx=0 tx=0).stop 2024-03-26T17:42:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1240504863 shutdown_connections 2024-03-26T17:42:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558a8dc07390 0x558a8dc09830 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf6cf0 0x558a8dc02b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1b80 0x558a8dbfbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1240504863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dbfb600 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01957fa640 0 client.17160 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:52.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1240504863 >> 172.21.15.105:0/1240504863 conn(0x558a8dbec700 msgr2=0x558a8dbeb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1240504863 shutdown_connections 2024-03-26T17:42:52.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1240504863 wait complete. 2024-03-26T17:42:52.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.720+0000 7f01b6b8b740 20 client.17160 trim_cache size 0 max 16384 2024-03-26T17:42:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.739+0000 7f01b6b8b740 1 Processor -- start 2024-03-26T17:42:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.739+0000 7f01b6b8b740 1 -- start start 2024-03-26T17:42:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.739+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1190 0x558a8dbfbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc02b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf6cf0 0x558a8dbeb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1aaf30 con 0x558a8dbf6cf0 2024-03-26T17:42:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1aca70 con 0x558a8dbf1190 2024-03-26T17:42:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1ab600 con 0x558a8dbf1b80 2024-03-26T17:42:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf6cf0 0x558a8dbeb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1c68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc02b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf6cf0 0x558a8dbeb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46950/0 (socket says 172.21.15.105:46950) 2024-03-26T17:42:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 -- 172.21.15.105:0/4013113010 learned_addr learned my addr 172.21.15.105:0/4013113010 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 --2- 172.21.15.105:0/4013113010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1190 0x558a8dbfbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 -- 172.21.15.105:0/4013113010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 msgr2=0x558a8dc02b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 --2- 172.21.15.105:0/4013113010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc02b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 -- 172.21.15.105:0/4013113010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1190 msgr2=0x558a8dbfbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 --2- 172.21.15.105:0/4013113010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1190 0x558a8dbfbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 -- 172.21.15.105:0/4013113010 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8d82a170 con 0x558a8dbf6cf0 2024-03-26T17:42:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01b1467640 1 --2- 172.21.15.105:0/4013113010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf6cf0 0x558a8dbeb780 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f019c0092e0 tx=0x7f019c014bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351fb94b16fdaa78 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01977fe640 1 -- 172.21.15.105:0/4013113010 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c014040 con 0x558a8dbf6cf0 2024-03-26T17:42:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01977fe640 1 -- 172.21.15.105:0/4013113010 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f019c0141e0 con 0x558a8dbf6cf0 2024-03-26T17:42:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.740+0000 7f01977fe640 1 -- 172.21.15.105:0/4013113010 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c010a00 con 0x558a8dbf6cf0 2024-03-26T17:42:52.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 -- 172.21.15.105:0/4013113010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf6cf0 msgr2=0x558a8dbeb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/4013113010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf6cf0 0x558a8dbeb780 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f019c0092e0 tx=0x7f019c014bb0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 -- 172.21.15.105:0/4013113010 shutdown_connections 2024-03-26T17:42:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/4013113010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf6cf0 0x558a8dbeb780 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/4013113010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc02b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/4013113010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf1190 0x558a8dbfbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 -- 172.21.15.105:0/4013113010 >> 172.21.15.105:0/4013113010 conn(0x558a8dbec700 msgr2=0x558a8dbeb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 -- 172.21.15.105:0/4013113010 shutdown_connections 2024-03-26T17:42:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 -- 172.21.15.105:0/4013113010 wait complete. 2024-03-26T17:42:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 -- start start 2024-03-26T17:42:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.741+0000 7f01b6b8b740 1 Processor -- start 2024-03-26T17:42:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dc00590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc00ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b6b8b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dc01a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1ac3a0 con 0x558a8dbf1190 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1abcd0 con 0x558a8dbf6cf0 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b6b8b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1a8f10 con 0x558a8dbf1b80 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b6b8b740 10 client.0 _ll_register_callbacks cb 0x7f019c009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dc00590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dc00590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46954/0 (socket says 172.21.15.105:46954) 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 -- 172.21.15.105:0/1271417202 learned_addr learned my addr 172.21.15.105:0/1271417202 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1c68640 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc00ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dc01a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 -- 172.21.15.105:0/1271417202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 msgr2=0x558a8dc00ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc00ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 -- 172.21.15.105:0/1271417202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 msgr2=0x558a8dc01a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dc01a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1467640 1 -- 172.21.15.105:0/1271417202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8dc02080 con 0x558a8dbf1190 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.742+0000 7f01b1c68640 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc00ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b1467640 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dc00590 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f019c009e90 tx=0x7f019c00ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cb13a415c3436 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01957fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:52.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01957fa640 1 -- 172.21.15.105:0/1271417202 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0050d0 con 0x558a8dbf1190 2024-03-26T17:42:52.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01957fa640 1 -- 172.21.15.105:0/1271417202 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f019c005270 con 0x558a8dbf1190 2024-03-26T17:42:52.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1271417202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f019c0141e0 con 0x558a8dbf1190 2024-03-26T17:42:52.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01957fa640 1 -- 172.21.15.105:0/1271417202 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f019c005470 con 0x558a8dbf1190 2024-03-26T17:42:52.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 10 client.17166 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:52.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1271417202 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a8dc6d9a0 con 0x558a8dbf1190 2024-03-26T17:42:52.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01957fa640 1 -- 172.21.15.105:0/1271417202 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f019c0141e0 con 0x558a8dbf1190 2024-03-26T17:42:52.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01957fa640 1 -- 172.21.15.105:0/1271417202 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f019c003b30 con 0x558a8dbf1190 2024-03-26T17:42:52.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 10 client.17166 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:52.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:52.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 20 client.17166 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:52.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558a8dc6e2e0 0x558a8dc70700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 4 client.17166 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:42:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.743+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1271417202 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558a8db5dec0 con 0x558a8dc6e2e0 2024-03-26T17:42:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.744+0000 7f01b1c68640 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558a8dc6e2e0 0x558a8dc70700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.744+0000 7f01b1c68640 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558a8dc6e2e0 0x558a8dc70700 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32112c41c29d47bc server_cookie=27eb3dfd6ab95006 in_seq=0 out_seq=0 2024-03-26T17:42:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.744+0000 7f01957fa640 10 client.17166 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01957fa640 1 -- 172.21.15.105:0/1271417202 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558a8db5dec0 con 0x558a8dc6e2e0 2024-03-26T17:42:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01957fa640 10 client.17166 handle_command_reply: tid=0 2024-03-26T17:42:52.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 client.17166 shutdown 2024-03-26T17:42:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1271417202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 msgr2=0x558a8dc00590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dc00590 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f019c009e90 tx=0x7f019c00ffa0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1271417202 shutdown_connections 2024-03-26T17:42:52.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558a8dc6e2e0 0x558a8dc70700 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a8dbf6cf0 0x558a8dc01a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01957fa640 0 client.17166 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558a8dbf1b80 0x558a8dc00ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 --2- 172.21.15.105:0/1271417202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558a8dbf1190 0x558a8dc00590 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1271417202 >> 172.21.15.105:0/1271417202 conn(0x558a8dbec700 msgr2=0x558a8dbeb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1271417202 shutdown_connections 2024-03-26T17:42:52.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 1 -- 172.21.15.105:0/1271417202 wait complete. 2024-03-26T17:42:52.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.746+0000 7f01b6b8b740 20 client.17166 trim_cache size 0 max 16384 2024-03-26T17:42:52.762 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:42:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:42:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "7df6dce6-b2fd-40fe-b3b7-fe265110ac32", 2024-03-26T17:42:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:42:52.763 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:42:52.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 -- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01980a2f20 msgr2=0x7f01980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01980a2f20 0x7f01980a53e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f01a401b000 tx=0x7f01a4066f50 comp rx=0 tx=0).stop 2024-03-26T17:42:52.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 -- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac102370 msgr2=0x7f01ac1032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac102370 0x7f01ac1032d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f01a400efd0 tx=0x7f01a400c6a0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 -- 172.21.15.105:0/3742453042 shutdown_connections 2024-03-26T17:42:52.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01980a2f20 0x7f01980a53e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01ac102370 0x7f01ac1032d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ac1019b0 0x7f01ac102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 --2- 172.21.15.105:0/3742453042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f01ac1007c0 0x7f01ac104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 -- 172.21.15.105:0/3742453042 >> 172.21.15.105:0/3742453042 conn(0x7f01ac0fbd50 msgr2=0x7f01ac068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 -- 172.21.15.105:0/3742453042 shutdown_connections 2024-03-26T17:42:52.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.747+0000 7f01b3ecc640 1 -- 172.21.15.105:0/3742453042 wait complete. 2024-03-26T17:42:52.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:42:52.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:42:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.889+0000 7f8f05a81640 1 -- 172.21.15.105:0/2560278742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f00104c80 msgr2=0x7f8f00107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.889+0000 7f8f05a81640 1 --2- 172.21.15.105:0/2560278742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f00104c80 0x7f8f00107070 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009900 tx=0x7f8ef402bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:52.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.889+0000 7f8f05a81640 1 -- 172.21.15.105:0/2560278742 shutdown_connections 2024-03-26T17:42:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 --2- 172.21.15.105:0/2560278742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f001075b0 0x7f8f00109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 --2- 172.21.15.105:0/2560278742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f00104c80 0x7f8f00107070 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 --2- 172.21.15.105:0/2560278742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f00102350 0x7f8f00104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 -- 172.21.15.105:0/2560278742 >> 172.21.15.105:0/2560278742 conn(0x7f8f000fbd60 msgr2=0x7f8f000fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 -- 172.21.15.105:0/2560278742 shutdown_connections 2024-03-26T17:42:52.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 -- 172.21.15.105:0/2560278742 wait complete. 2024-03-26T17:42:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 Processor -- start 2024-03-26T17:42:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 -- start start 2024-03-26T17:42:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f00102350 0x7f8f00197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f00104c80 0x7f8f001980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f001075b0 0x7f8f0019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010d410 con 0x7f8f001075b0 2024-03-26T17:42:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010d290 con 0x7f8f00102350 2024-03-26T17:42:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.890+0000 7f8f05a81640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0010d590 con 0x7f8f00104c80 2024-03-26T17:42:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f001075b0 0x7f8f0019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f001075b0 0x7f8f0019f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46966/0 (socket says 172.21.15.105:46966) 2024-03-26T17:42:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f00102350 0x7f8f00197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 -- 172.21.15.105:0/3202302670 learned_addr learned my addr 172.21.15.105:0/3202302670 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efeffd640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f00104c80 0x7f8f001980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 -- 172.21.15.105:0/3202302670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f00104c80 msgr2=0x7f8f001980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f00104c80 0x7f8f001980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 -- 172.21.15.105:0/3202302670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f00102350 msgr2=0x7f8f00197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f00102350 0x7f8f00197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 -- 172.21.15.105:0/3202302670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0019f7b0 con 0x7f8f001075b0 2024-03-26T17:42:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f00102350 0x7f8f00197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:52.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efffff640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f001075b0 0x7f8f0019f140 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800eb10 tx=0x7f8ef800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efcff9640 1 -- 172.21.15.105:0/3202302670 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8004b20 con 0x7f8f001075b0 2024-03-26T17:42:52.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8f05a81640 1 -- 172.21.15.105:0/3202302670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f001001a0 con 0x7f8f001075b0 2024-03-26T17:42:52.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efcff9640 1 -- 172.21.15.105:0/3202302670 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ef8004cc0 con 0x7f8f001075b0 2024-03-26T17:42:52.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8efcff9640 1 -- 172.21.15.105:0/3202302670 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8010690 con 0x7f8f001075b0 2024-03-26T17:42:52.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.891+0000 7f8f05a81640 1 -- 172.21.15.105:0/3202302670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f001006c0 con 0x7f8f001075b0 2024-03-26T17:42:52.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8efcff9640 1 -- 172.21.15.105:0/3202302670 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8ef8005020 con 0x7f8f001075b0 2024-03-26T17:42:52.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8f0866c740 1 Processor -- start 2024-03-26T17:42:52.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8f0866c740 1 -- start start 2024-03-26T17:42:52.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e68520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e6d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 0x55e1c9e700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8f0866c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e1c9dc4980 con 0x55e1c9e68100 2024-03-26T17:42:52.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8f0866c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e1c9d6cbe0 con 0x55e1c9e6dc60 2024-03-26T17:42:52.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8f0866c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e1c9dc0930 con 0x55e1c9e68af0 2024-03-26T17:42:52.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e68520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e6d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e68520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46972/0 (socket says 172.21.15.105:46972) 2024-03-26T17:42:52.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8efeffd640 1 -- 172.21.15.105:0/1355585862 learned_addr learned my addr 172.21.15.105:0/1355585862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.894+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 0x55e1c9e700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efcff9640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8ee80a2f20 0x7f8ee80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efeffd640 1 -- 172.21.15.105:0/1355585862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 msgr2=0x55e1c9e6d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e6d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efeffd640 1 -- 172.21.15.105:0/1355585862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 msgr2=0x55e1c9e700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 0x55e1c9e700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efeffd640 1 -- 172.21.15.105:0/1355585862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c9ba27d0 con 0x55e1c9e68100 2024-03-26T17:42:52.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efffff640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8ee80a2f20 0x7f8ee80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e6d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:52.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efcff9640 1 -- 172.21.15.105:0/3202302670 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8ef80cf970 con 0x7f8f001075b0 2024-03-26T17:42:52.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e68520 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8ef402c840 tx=0x7f8ef402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9f41f2dacb890 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8ee5ffb640 1 -- 172.21.15.105:0/1355585862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef404a070 con 0x55e1c9e68100 2024-03-26T17:42:52.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8ee5ffb640 1 -- 172.21.15.105:0/1355585862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ef4005050 con 0x55e1c9e68100 2024-03-26T17:42:52.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.895+0000 7f8ee5ffb640 1 -- 172.21.15.105:0/1355585862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4004920 con 0x55e1c9e68100 2024-03-26T17:42:52.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8efffff640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8ee80a2f20 0x7f8ee80a53e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8098a30 tx=0x7f8ef8091540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 -- 172.21.15.105:0/1355585862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 msgr2=0x55e1c9e68520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e68520 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8ef402c840 tx=0x7f8ef402bc30 comp rx=0 tx=0).stop 2024-03-26T17:42:52.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 -- 172.21.15.105:0/1355585862 shutdown_connections 2024-03-26T17:42:52.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 0x55e1c9e700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e6d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1355585862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e68520 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 -- 172.21.15.105:0/1355585862 >> 172.21.15.105:0/1355585862 conn(0x55e1c9e63670 msgr2=0x55e1c9e63a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 -- 172.21.15.105:0/1355585862 shutdown_connections 2024-03-26T17:42:52.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 -- 172.21.15.105:0/1355585862 wait complete. 2024-03-26T17:42:52.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.896+0000 7f8f0866c740 1 -- start start 2024-03-26T17:42:52.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8f0866c740 1 Processor -- start 2024-03-26T17:42:52.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68af0 0x55e1c9e728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 0x55e1c9e79920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8f0866c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e1c9e70bb0 con 0x55e1c9e68af0 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8f0866c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e1c9d80160 con 0x55e1c9e6dc60 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8f0866c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e1c9e70d30 con 0x55e1c9e68100 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8f0866c740 10 client.0 _ll_register_callbacks cb 0x55e1c9dbc720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68af0 0x55e1c9e728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68af0 0x55e1c9e728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46984/0 (socket says 172.21.15.105:46984) 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37028/0 (socket says 172.21.15.105:37028) 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 -- 172.21.15.105:0/1520167832 learned_addr learned my addr 172.21.15.105:0/1520167832 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 0x55e1c9e79920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 -- 172.21.15.105:0/1520167832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 msgr2=0x55e1c9e723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 -- 172.21.15.105:0/1520167832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 msgr2=0x55e1c9e79920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 0x55e1c9e79920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 -- 172.21.15.105:0/1520167832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c9ad83f0 con 0x55e1c9e68af0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68af0 0x55e1c9e728e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009e70 tx=0x7f8ef000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2306b93c90f9f58 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.897+0000 7f8edeffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8edeffd640 1 -- 172.21.15.105:0/1520167832 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0004050 con 0x55e1c9e68af0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8edeffd640 1 -- 172.21.15.105:0/1520167832 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8ef00041f0 con 0x55e1c9e68af0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 1 -- 172.21.15.105:0/1520167832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef4005050 con 0x55e1c9e68af0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8edeffd640 1 -- 172.21.15.105:0/1520167832 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ef00043f0 con 0x55e1c9e68af0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 10 client.17178 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 1 -- 172.21.15.105:0/1520167832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e1c9a980f0 con 0x55e1c9e68af0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8edeffd640 1 -- 172.21.15.105:0/1520167832 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ef4005050 con 0x55e1c9e68af0 2024-03-26T17:42:52.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8edeffd640 1 -- 172.21.15.105:0/1520167832 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef001a5b0 con 0x55e1c9e68af0 2024-03-26T17:42:52.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 10 client.17178 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:52.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:52.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 20 client.17178 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:52.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e1c9e7e320 0x55e1c9e80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 4 client.17178 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:42:52.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.898+0000 7f8f0866c740 1 -- 172.21.15.105:0/1520167832 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1c9dd1ec0 con 0x55e1c9e7e320 2024-03-26T17:42:52.910 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:52 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:42:52.910 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:52 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:52.911 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:52 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:42:52.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.899+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e1c9e7e320 0x55e1c9e80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.899+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e1c9e7e320 0x55e1c9e80760 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce9608247da1320 server_cookie=8325c16fbae84acc in_seq=0 out_seq=0 2024-03-26T17:42:52.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.899+0000 7f8edeffd640 10 client.17178 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:52.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8edeffd640 1 -- 172.21.15.105:0/1520167832 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e1c9dd1ec0 con 0x55e1c9e7e320 2024-03-26T17:42:52.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8edeffd640 10 client.17178 handle_command_reply: tid=0 2024-03-26T17:42:52.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 client.17178 shutdown 2024-03-26T17:42:52.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 -- 172.21.15.105:0/1520167832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68af0 msgr2=0x55e1c9e728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68af0 0x55e1c9e728e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009e70 tx=0x7f8ef000e720 comp rx=0 tx=0).stop 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 -- 172.21.15.105:0/1520167832 shutdown_connections 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e1c9e7e320 0x55e1c9e80760 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8edeffd640 0 client.17178 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e6dc60 0x55e1c9e79920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68af0 0x55e1c9e728e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1520167832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 -- 172.21.15.105:0/1520167832 >> 172.21.15.105:0/1520167832 conn(0x55e1c9e63670 msgr2=0x55e1c9e5a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 -- 172.21.15.105:0/1520167832 shutdown_connections 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 1 -- 172.21.15.105:0/1520167832 wait complete. 2024-03-26T17:42:52.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.900+0000 7f8f0866c740 20 client.17178 trim_cache size 0 max 16384 2024-03-26T17:42:52.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.920+0000 7f8f0866c740 1 Processor -- start 2024-03-26T17:42:52.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.920+0000 7f8f0866c740 1 -- start start 2024-03-26T17:42:52.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e63670 0x55e1c9e723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8f0866c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a6c30 con 0x55e1c9e68100 2024-03-26T17:42:52.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8f0866c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a8770 con 0x55e1c9e63670 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8f0866c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a7300 con 0x55e1c9e68af0 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e63670 0x55e1c9e723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46998/0 (socket says 172.21.15.105:46998) 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 -- 172.21.15.105:0/4144244452 learned_addr learned my addr 172.21.15.105:0/4144244452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8efeffd640 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 -- 172.21.15.105:0/4144244452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 msgr2=0x55e1c9e798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 -- 172.21.15.105:0/4144244452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e63670 msgr2=0x55e1c9e723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e63670 0x55e1c9e723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 -- 172.21.15.105:0/4144244452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c9e566b0 con 0x55e1c9e68100 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8efeffd640 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e63670 0x55e1c9e723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:52.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e728e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0000c00 tx=0x7f8ef0005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58c8d9a671acc2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8ee57fa640 1 -- 172.21.15.105:0/4144244452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0033070 con 0x55e1c9e68100 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8ee57fa640 1 -- 172.21.15.105:0/4144244452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ef001ac40 con 0x55e1c9e68100 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.921+0000 7f8ee57fa640 1 -- 172.21.15.105:0/4144244452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0004590 con 0x55e1c9e68100 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 -- 172.21.15.105:0/4144244452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 msgr2=0x55e1c9e728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e728e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0000c00 tx=0x7f8ef0005990 comp rx=0 tx=0).stop 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 -- 172.21.15.105:0/4144244452 shutdown_connections 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68af0 0x55e1c9e798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e68100 0x55e1c9e728e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 --2- 172.21.15.105:0/4144244452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e63670 0x55e1c9e723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 -- 172.21.15.105:0/4144244452 >> 172.21.15.105:0/4144244452 conn(0x55e1c9e6d7a0 msgr2=0x55e1c9e63a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 -- 172.21.15.105:0/4144244452 shutdown_connections 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.922+0000 7f8f0866c740 1 -- 172.21.15.105:0/4144244452 wait complete. 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 1 -- start start 2024-03-26T17:42:52.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 1 Processor -- start 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e63670 0x55e1c9e76a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e76fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e68af0 0x55e1c9e77f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a80a0 con 0x55e1c9e63670 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a79d0 con 0x55e1c9e68af0 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001a4c10 con 0x55e1c9e68100 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8f0866c740 10 client.0 _ll_register_callbacks cb 0x7f8ef00103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e63670 0x55e1c9e76a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e63670 0x55e1c9e76a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47004/0 (socket says 172.21.15.105:47004) 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8efeffd640 1 -- 172.21.15.105:0/1849500047 learned_addr learned my addr 172.21.15.105:0/1849500047 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e76fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.923+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e68af0 0x55e1c9e77f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8efeffd640 1 -- 172.21.15.105:0/1849500047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 msgr2=0x55e1c9e76fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e76fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8efeffd640 1 -- 172.21.15.105:0/1849500047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e68af0 msgr2=0x55e1c9e77f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e68af0 0x55e1c9e77f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8efeffd640 1 -- 172.21.15.105:0/1849500047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c9e78550 con 0x55e1c9e63670 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e76fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8efeffd640 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e63670 0x55e1c9e76a80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009850 tx=0x7f8ef4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b4ddfa71e32b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8edeffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8edeffd640 1 -- 172.21.15.105:0/1849500047 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef404a040 con 0x55e1c9e63670 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8edeffd640 1 -- 172.21.15.105:0/1849500047 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8ef40047a0 con 0x55e1c9e63670 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8f0866c740 1 -- 172.21.15.105:0/1849500047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef001ac40 con 0x55e1c9e63670 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8edeffd640 1 -- 172.21.15.105:0/1849500047 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ef4068050 con 0x55e1c9e63670 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8f0866c740 10 client.17184 fetch_fsmap learned FSMap version 30 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8f0866c740 1 -- 172.21.15.105:0/1849500047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e1c9e79860 con 0x55e1c9e63670 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8edeffd640 1 -- 172.21.15.105:0/1849500047 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ef001ac40 con 0x55e1c9e63670 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.924+0000 7f8edeffd640 1 -- 172.21.15.105:0/1849500047 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef4050040 con 0x55e1c9e63670 2024-03-26T17:42:52.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8f0866c740 10 client.17184 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8f0866c740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8f0866c740 20 client.17184 populate_metadata read hostname 'smithi105' 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e1c9e6dc60 0x55e1c9ee6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8f0866c740 4 client.17184 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8f0866c740 1 -- 172.21.15.105:0/1849500047 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1c9dcc210 con 0x55e1c9e6dc60 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e1c9e6dc60 0x55e1c9ee6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8edeffd640 10 client.17184 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.925+0000 7f8eff7fe640 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e1c9e6dc60 0x55e1c9ee6720 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=decca44b1f3d53c server_cookie=af57709f6deaeda7 in_seq=0 out_seq=0 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.926+0000 7f8edeffd640 1 -- 172.21.15.105:0/1849500047 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e1c9dcc210 con 0x55e1c9e6dc60 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.926+0000 7f8edeffd640 10 client.17184 handle_command_reply: tid=0 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.926+0000 7f8f0866c740 1 client.17184 shutdown 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 -- 172.21.15.105:0/1849500047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e63670 msgr2=0x55e1c9e76a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e63670 0x55e1c9e76a80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009850 tx=0x7f8ef4005d50 comp rx=0 tx=0).stop 2024-03-26T17:42:52.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 -- 172.21.15.105:0/1849500047 shutdown_connections 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e1c9e6dc60 0x55e1c9ee6720 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e1c9e68af0 0x55e1c9e77f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e1c9e68100 0x55e1c9e76fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8edeffd640 0 client.17184 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 --2- 172.21.15.105:0/1849500047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e1c9e63670 0x55e1c9e76a80 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 -- 172.21.15.105:0/1849500047 >> 172.21.15.105:0/1849500047 conn(0x55e1c9e6d7a0 msgr2=0x55e1c9e69e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 -- 172.21.15.105:0/1849500047 shutdown_connections 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 1 -- 172.21.15.105:0/1849500047 wait complete. 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f0866c740 20 client.17184 trim_cache size 0 max 16384 2024-03-26T17:42:52.927 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "7df6dce6-b2fd-40fe-b3b7-fe265110ac32": { 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "7df6dce6-b2fd-40fe-b3b7-fe265110ac32", 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:42:52.928 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:42:52.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 -- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8ee80a2f20 msgr2=0x7f8ee80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8ee80a2f20 0x7f8ee80a53e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8098a30 tx=0x7f8ef8091540 comp rx=0 tx=0).stop 2024-03-26T17:42:52.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 -- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f001075b0 msgr2=0x7f8f0019f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:42:52.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f001075b0 0x7f8f0019f140 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800eb10 tx=0x7f8ef800c2d0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 -- 172.21.15.105:0/3202302670 shutdown_connections 2024-03-26T17:42:52.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8ee80a2f20 0x7f8ee80a53e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8f001075b0 0x7f8f0019f140 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f00104c80 0x7f8f001980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 --2- 172.21.15.105:0/3202302670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f00102350 0x7f8f00197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:42:52.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.927+0000 7f8f05a81640 1 -- 172.21.15.105:0/3202302670 >> 172.21.15.105:0/3202302670 conn(0x7f8f000fbd60 msgr2=0x7f8f001041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:42:52.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.928+0000 7f8f05a81640 1 -- 172.21.15.105:0/3202302670 shutdown_connections 2024-03-26T17:42:52.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:42:52.928+0000 7f8f05a81640 1 -- 172.21.15.105:0/3202302670 wait complete. 2024-03-26T17:42:52.932 INFO:tasks.cephfs.filesystem:scrub status for tag:7df6dce6-b2fd-40fe-b3b7-fe265110ac32 - {'path': '/', 'tag': '7df6dce6-b2fd-40fe-b3b7-fe265110ac32', 'options': 'recursive,force'} 2024-03-26T17:42:53.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 127506 3.14 MB/sec execute 96 sec latency 20.145 ms 2024-03-26T17:42:53.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:52 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:42:53.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:52 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:53.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:52 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:42:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:52 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:42:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:52 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:52 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:42:54.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 128038 3.15 MB/sec execute 97 sec latency 80.458 ms 2024-03-26T17:42:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:53 smithi144 ceph-mon[36095]: pgmap v484: 97 pgs: 97 active+clean; 31 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-03-26T17:42:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:53 smithi105 ceph-mon[32280]: pgmap v484: 97 pgs: 97 active+clean; 31 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-03-26T17:42:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:53 smithi135 ceph-mon[35636]: pgmap v484: 97 pgs: 97 active+clean; 31 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-03-26T17:42:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:42:55.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 128443 3.13 MB/sec execute 98 sec latency 24.525 ms 2024-03-26T17:42:56.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 129075 3.13 MB/sec execute 99 sec latency 16.303 ms 2024-03-26T17:42:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:55 smithi144 ceph-mon[36095]: pgmap v485: 97 pgs: 97 active+clean; 30 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-03-26T17:42:56.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:55 smithi105 ceph-mon[32280]: pgmap v485: 97 pgs: 97 active+clean; 30 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-03-26T17:42:56.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:55 smithi135 ceph-mon[35636]: pgmap v485: 97 pgs: 97 active+clean; 30 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-03-26T17:42:57.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 129764 3.12 MB/sec execute 100 sec latency 15.554 ms 2024-03-26T17:42:57.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:56 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:42:57.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:56 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:42:57.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:56 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:42:58.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 130344 3.11 MB/sec execute 101 sec latency 19.143 ms 2024-03-26T17:42:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:57 smithi144 ceph-mon[36095]: pgmap v486: 97 pgs: 97 active+clean; 27 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:42:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:57 smithi105 ceph-mon[32280]: pgmap v486: 97 pgs: 97 active+clean; 27 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:42:58.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:57 smithi135 ceph-mon[35636]: pgmap v486: 97 pgs: 97 active+clean; 27 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:42:59.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 131057 3.14 MB/sec execute 102 sec latency 15.991 ms 2024-03-26T17:43:00.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 131619 3.16 MB/sec execute 103 sec latency 15.374 ms 2024-03-26T17:43:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:42:59 smithi144 ceph-mon[36095]: pgmap v487: 97 pgs: 97 active+clean; 27 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:43:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:42:59 smithi105 ceph-mon[32280]: pgmap v487: 97 pgs: 97 active+clean; 27 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:43:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:42:59 smithi135 ceph-mon[35636]: pgmap v487: 97 pgs: 97 active+clean; 27 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:43:01.022 INFO:tasks.workunit.client.0.smithi105.stdout: 1 132082 3.13 MB/sec execute 104 sec latency 27.115 ms 2024-03-26T17:43:02.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 132731 3.14 MB/sec execute 105 sec latency 16.890 ms 2024-03-26T17:43:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:01 smithi144 ceph-mon[36095]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:43:02.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:01 smithi105 ceph-mon[32280]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:43:02.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:01 smithi135 ceph-mon[35636]: pgmap v488: 97 pgs: 97 active+clean; 28 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:43:03.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 133419 3.12 MB/sec execute 106 sec latency 15.552 ms 2024-03-26T17:43:03.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:02 smithi105 ceph-mon[32280]: pgmap v489: 97 pgs: 97 active+clean; 29 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:43:03.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:02 smithi135 ceph-mon[35636]: pgmap v489: 97 pgs: 97 active+clean; 29 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:43:03.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:02 smithi144 ceph-mon[36095]: pgmap v489: 97 pgs: 97 active+clean; 29 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:43:04.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 134045 3.14 MB/sec execute 107 sec latency 16.725 ms 2024-03-26T17:43:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 -- 172.21.15.105:0/4051012729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 msgr2=0x7fd8e4108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/4051012729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 0x7fd8e4108c20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009ca0 tx=0x7fd8dc02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 -- 172.21.15.105:0/4051012729 shutdown_connections 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/4051012729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 0x7fd8e4108c20 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/4051012729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8e41019d0 0x7fd8e4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/4051012729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8e41007e0 0x7fd8e4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 -- 172.21.15.105:0/4051012729 >> 172.21.15.105:0/4051012729 conn(0x7fd8e40fbd50 msgr2=0x7fd8e40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 -- 172.21.15.105:0/4051012729 shutdown_connections 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.277+0000 7fd8ebeff640 1 -- 172.21.15.105:0/4051012729 wait complete. 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ebeff640 1 Processor -- start 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ebeff640 1 -- start start 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ebeff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8e41007e0 0x7fd8e4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ebeff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8e41019d0 0x7fd8e4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ebeff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 0x7fd8e419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ebeff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e410c310 con 0x7fd8e4102390 2024-03-26T17:43:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ebeff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e410c190 con 0x7fd8e41019d0 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ebeff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e410c490 con 0x7fd8e41007e0 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8e9c9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8e41007e0 0x7fd8e4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ea49c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 0x7fd8e419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8e949a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8e41019d0 0x7fd8e4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ea49c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 0x7fd8e419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48156/0 (socket says 172.21.15.105:48156) 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8e9c9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8e41007e0 0x7fd8e4197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59148/0 (socket says 172.21.15.105:59148) 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ea49c640 1 -- 172.21.15.105:0/2625386101 learned_addr learned my addr 172.21.15.105:0/2625386101 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ea49c640 1 -- 172.21.15.105:0/2625386101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8e41007e0 msgr2=0x7fd8e4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ea49c640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8e41007e0 0x7fd8e4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ea49c640 1 -- 172.21.15.105:0/2625386101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8e41019d0 msgr2=0x7fd8e4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ea49c640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8e41019d0 0x7fd8e4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8ea49c640 1 -- 172.21.15.105:0/2625386101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e419f670 con 0x7fd8e4102390 2024-03-26T17:43:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.278+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8e41007e0 0x7fd8e4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.279+0000 7fd8ea49c640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 0x7fd8e419f000 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc02c700 tx=0x7fd8dc0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.279+0000 7fd8daffd640 1 -- 172.21.15.105:0/2625386101 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc041020 con 0x7fd8e4102390 2024-03-26T17:43:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.279+0000 7fd8daffd640 1 -- 172.21.15.105:0/2625386101 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8dc003e20 con 0x7fd8e4102390 2024-03-26T17:43:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.279+0000 7fd8daffd640 1 -- 172.21.15.105:0/2625386101 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc039040 con 0x7fd8e4102390 2024-03-26T17:43:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.279+0000 7fd8ebeff640 1 -- 172.21.15.105:0/2625386101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e419f900 con 0x7fd8e4102390 2024-03-26T17:43:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.279+0000 7fd8ebeff640 1 -- 172.21.15.105:0/2625386101 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e419fdf0 con 0x7fd8e4102390 2024-03-26T17:43:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.281+0000 7fd8daffd640 1 -- 172.21.15.105:0/2625386101 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd8dc044020 con 0x7fd8e4102390 2024-03-26T17:43:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.281+0000 7fd8eebca740 1 Processor -- start 2024-03-26T17:43:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8eebca740 1 -- start start 2024-03-26T17:43:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c64ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c653ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 0x56239c6569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8eebca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56239c5af0b0 con 0x56239c64e9e0 2024-03-26T17:43:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8eebca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56239c557960 con 0x56239c654540 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8eebca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56239c5ab060 con 0x56239c64f3d0 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c64ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c64ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48168/0 (socket says 172.21.15.105:48168) 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 -- 172.21.15.105:0/3142974675 learned_addr learned my addr 172.21.15.105:0/3142974675 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8daffd640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8d00a2fc0 0x7fd8d00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e9c9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c653ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 0x56239c6569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8ea49c640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8d00a2fc0 0x7fd8d00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 -- 172.21.15.105:0/3142974675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 msgr2=0x56239c653ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8daffd640 1 -- 172.21.15.105:0/2625386101 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd8dc0f15c0 con 0x7fd8e4102390 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c653ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 -- 172.21.15.105:0/3142974675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 msgr2=0x56239c6569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 0x56239c6569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 -- 172.21.15.105:0/3142974675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239c569c60 con 0x56239c64e9e0 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c653ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.282+0000 7fd8e949a640 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c64ee00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4011680 tx=0x7fd8d4009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d750dff0a01413b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.283+0000 7fd8ea49c640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8d00a2fc0 0x7fd8d00a5480 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0ba110 tx=0x7fd8dc088800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.283+0000 7fd8cffff640 1 -- 172.21.15.105:0/3142974675 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d401a070 con 0x56239c64e9e0 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.283+0000 7fd8cffff640 1 -- 172.21.15.105:0/3142974675 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8d4004250 con 0x56239c64e9e0 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.283+0000 7fd8cffff640 1 -- 172.21.15.105:0/3142974675 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d400f870 con 0x56239c64e9e0 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.283+0000 7fd8eebca740 1 -- 172.21.15.105:0/3142974675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 msgr2=0x56239c64ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.283+0000 7fd8eebca740 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c64ee00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4011680 tx=0x7fd8d4009950 comp rx=0 tx=0).stop 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 -- 172.21.15.105:0/3142974675 shutdown_connections 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 0x56239c6569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c653ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 --2- 172.21.15.105:0/3142974675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c64ee00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 -- 172.21.15.105:0/3142974675 >> 172.21.15.105:0/3142974675 conn(0x56239c645b40 msgr2=0x56239c645f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 -- 172.21.15.105:0/3142974675 shutdown_connections 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 -- 172.21.15.105:0/3142974675 wait complete. 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 -- start start 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.284+0000 7fd8eebca740 1 Processor -- start 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64e9e0 0x56239c658850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64f3d0 0x56239c658d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c654540 0x56239c65fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8eebca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56239c6574f0 con 0x56239c64f3d0 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8eebca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56239c56adb0 con 0x56239c64e9e0 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8eebca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56239c657670 con 0x56239c654540 2024-03-26T17:43:04.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e9c9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64f3d0 0x56239c658d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e949a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64e9e0 0x56239c658850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e9c9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64f3d0 0x56239c658d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48180/0 (socket says 172.21.15.105:48180) 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8eebca740 10 client.0 _ll_register_callbacks cb 0x56239c56f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e9c9b640 1 -- 172.21.15.105:0/1331708334 learned_addr learned my addr 172.21.15.105:0/1331708334 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e949a640 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c654540 0x56239c65fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e949a640 1 -- 172.21.15.105:0/1331708334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c654540 msgr2=0x56239c65fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e949a640 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c654540 0x56239c65fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e949a640 1 -- 172.21.15.105:0/1331708334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64f3d0 msgr2=0x56239c658d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e949a640 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64f3d0 0x56239c658d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e949a640 1 -- 172.21.15.105:0/1331708334 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239c2c3210 con 0x56239c64e9e0 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64f3d0 0x56239c658d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:04.285 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8e949a640 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64e9e0 0x56239c658850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009d40 tx=0x7fd8d4009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fc0aef3c747682 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.285+0000 7fd8cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1331708334 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d401a070 con 0x56239c64e9e0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1331708334 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd8d400f5f0 con 0x56239c64e9e0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 1 -- 172.21.15.105:0/1331708334 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d4004250 con 0x56239c64e9e0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1331708334 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8d4028040 con 0x56239c64e9e0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 10 client.27029 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 1 -- 172.21.15.105:0/1331708334 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56239c38f620 con 0x56239c64e9e0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1331708334 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8d4004250 con 0x56239c64e9e0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1331708334 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd8d401f720 con 0x56239c64e9e0 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 10 client.27029 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 10 client.27029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 20 client.27029 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56239c6642f0 0x56239c666720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 4 client.27029 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8eebca740 1 -- 172.21.15.105:0/1331708334 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239c5bc5f0 con 0x56239c6642f0 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.286+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56239c6642f0 0x56239c666720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.287+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56239c6642f0 0x56239c666720 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d53d4d46024a2a9 server_cookie=7a2a04d950a278ed in_seq=0 out_seq=0 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.287+0000 7fd8cd7fa640 10 client.27029 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1331708334 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56239c5bc5f0 con 0x56239c6642f0 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8cd7fa640 10 client.27029 handle_command_reply: tid=0 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 client.27029 shutdown 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 -- 172.21.15.105:0/1331708334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64e9e0 msgr2=0x56239c658850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64e9e0 0x56239c658850 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009d40 tx=0x7fd8d4009c90 comp rx=0 tx=0).stop 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 -- 172.21.15.105:0/1331708334 shutdown_connections 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56239c6642f0 0x56239c666720 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c654540 0x56239c65fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64f3d0 0x56239c658d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1331708334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64e9e0 0x56239c658850 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8cd7fa640 0 client.27029 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 -- 172.21.15.105:0/1331708334 >> 172.21.15.105:0/1331708334 conn(0x56239c645b40 msgr2=0x56239c644df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 -- 172.21.15.105:0/1331708334 shutdown_connections 2024-03-26T17:43:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 1 -- 172.21.15.105:0/1331708334 wait complete. 2024-03-26T17:43:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.288+0000 7fd8eebca740 20 client.27029 trim_cache size 0 max 16384 2024-03-26T17:43:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8eebca740 1 Processor -- start 2024-03-26T17:43:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8eebca740 1 -- start start 2024-03-26T17:43:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c658cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c65fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 0x56239c650550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8eebca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41a6a90 con 0x56239c64e9e0 2024-03-26T17:43:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8eebca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41a85d0 con 0x56239c654540 2024-03-26T17:43:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8eebca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41a7160 con 0x56239c64f3d0 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8e949a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c658cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8e949a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c658cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48192/0 (socket says 172.21.15.105:48192) 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8e949a640 1 -- 172.21.15.105:0/2584295600 learned_addr learned my addr 172.21.15.105:0/2584295600 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.305+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c65fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8e949a640 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 0x56239c650550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8e9c9b640 1 -- 172.21.15.105:0/2584295600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 msgr2=0x56239c650550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 0x56239c650550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8e9c9b640 1 -- 172.21.15.105:0/2584295600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 msgr2=0x56239c658cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c658cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8e9c9b640 1 -- 172.21.15.105:0/2584295600 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239c6464c0 con 0x56239c64f3d0 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8e949a640 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c658cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c65fd20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x56239c666d20 tx=0x7fd8e0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2590fe5c11975f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8cf7fe640 1 -- 172.21.15.105:0/2584295600 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000fc80 con 0x56239c64f3d0 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8cf7fe640 1 -- 172.21.15.105:0/2584295600 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8e000fe20 con 0x56239c64f3d0 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.306+0000 7fd8cf7fe640 1 -- 172.21.15.105:0/2584295600 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00226d0 con 0x56239c64f3d0 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 -- 172.21.15.105:0/2584295600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 msgr2=0x56239c65fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c65fd20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x56239c666d20 tx=0x7fd8e0005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 -- 172.21.15.105:0/2584295600 shutdown_connections 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c654540 0x56239c650550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64f3d0 0x56239c65fd20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 --2- 172.21.15.105:0/2584295600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c64e9e0 0x56239c658cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 -- 172.21.15.105:0/2584295600 >> 172.21.15.105:0/2584295600 conn(0x56239c645b40 msgr2=0x56239c64fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 -- 172.21.15.105:0/2584295600 shutdown_connections 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 -- 172.21.15.105:0/2584295600 wait complete. 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 -- start start 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.307+0000 7fd8eebca740 1 Processor -- start 2024-03-26T17:43:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64e9e0 0x56239c65ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64f3d0 0x56239c65d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8eebca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c654540 0x56239c65e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8eebca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41a7f00 con 0x56239c654540 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8eebca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41a7830 con 0x56239c64f3d0 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8eebca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41a4a70 con 0x56239c64e9e0 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8eebca740 10 client.0 _ll_register_callbacks cb 0x7fd8e0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64f3d0 0x56239c65d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64f3d0 0x56239c65d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36586/0 (socket says 172.21.15.105:36586) 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 -- 172.21.15.105:0/1305637184 learned_addr learned my addr 172.21.15.105:0/1305637184 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e949a640 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c654540 0x56239c65e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e949a640 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64e9e0 0x56239c65ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 -- 172.21.15.105:0/1305637184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64e9e0 msgr2=0x56239c65ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64e9e0 0x56239c65ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 -- 172.21.15.105:0/1305637184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c654540 msgr2=0x56239c65e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c654540 0x56239c65e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 -- 172.21.15.105:0/1305637184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239c65ea10 con 0x56239c64f3d0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8e9c9b640 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64f3d0 0x56239c65d410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00129f0 tx=0x7fd8e0009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=331f477e48a11dce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.308+0000 7fd8cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1305637184 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e002f070 con 0x56239c64f3d0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1305637184 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd8e0022610 con 0x56239c64f3d0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 1 -- 172.21.15.105:0/1305637184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e000fe20 con 0x56239c64f3d0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1305637184 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8e003b050 con 0x56239c64f3d0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 10 client.27035 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 1 -- 172.21.15.105:0/1305637184 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56239c65fd50 con 0x56239c64f3d0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1305637184 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8e000fe20 con 0x56239c64f3d0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1305637184 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd8e00039c0 con 0x56239c64f3d0 2024-03-26T17:43:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 10 client.27035 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 10 client.27035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 20 client.27035 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56239c6caa70 0x56239c6cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 4 client.27035 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.309+0000 7fd8eebca740 1 -- 172.21.15.105:0/1305637184 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56239c5b6940 con 0x56239c6caa70 2024-03-26T17:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.310+0000 7fd8e949a640 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56239c6caa70 0x56239c6cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.310+0000 7fd8cd7fa640 10 client.27035 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.310+0000 7fd8e949a640 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56239c6caa70 0x56239c6cce90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f731a0c3d9bb817 server_cookie=de74bf250b92e604 in_seq=0 out_seq=0 2024-03-26T17:43:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8cd7fa640 1 -- 172.21.15.105:0/1305637184 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56239c5b6940 con 0x56239c6caa70 2024-03-26T17:43:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8cd7fa640 10 client.27035 handle_command_reply: tid=0 2024-03-26T17:43:04.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 client.27035 shutdown 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 -- 172.21.15.105:0/1305637184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64f3d0 msgr2=0x56239c65d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64f3d0 0x56239c65d410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00129f0 tx=0x7fd8e0009d90 comp rx=0 tx=0).stop 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 -- 172.21.15.105:0/1305637184 shutdown_connections 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56239c6caa70 0x56239c6cce90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56239c654540 0x56239c65e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56239c64f3d0 0x56239c65d410 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8cd7fa640 0 client.27035 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 --2- 172.21.15.105:0/1305637184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56239c64e9e0 0x56239c65ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.352+0000 7fd8eebca740 1 -- 172.21.15.105:0/1305637184 >> 172.21.15.105:0/1305637184 conn(0x56239c645b40 msgr2=0x56239c658b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8eebca740 1 -- 172.21.15.105:0/1305637184 shutdown_connections 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8eebca740 1 -- 172.21.15.105:0/1305637184 wait complete. 2024-03-26T17:43:04.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8eebca740 20 client.27035 trim_cache size 0 max 16384 2024-03-26T17:43:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:43:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:43:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:43:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:43:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 -- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8d00a2fc0 msgr2=0x7fd8d00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8d00a2fc0 0x7fd8d00a5480 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0ba110 tx=0x7fd8dc088800 comp rx=0 tx=0).stop 2024-03-26T17:43:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 -- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 msgr2=0x7fd8e419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 0x7fd8e419f000 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc02c700 tx=0x7fd8dc0328a0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 -- 172.21.15.105:0/2625386101 shutdown_connections 2024-03-26T17:43:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8d00a2fc0 0x7fd8d00a5480 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8e4102390 0x7fd8e419f000 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8e41019d0 0x7fd8e4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 --2- 172.21.15.105:0/2625386101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8e41007e0 0x7fd8e4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 -- 172.21.15.105:0/2625386101 >> 172.21.15.105:0/2625386101 conn(0x7fd8e40fbd50 msgr2=0x7fd8e40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 -- 172.21.15.105:0/2625386101 shutdown_connections 2024-03-26T17:43:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:04.353+0000 7fd8ebeff640 1 -- 172.21.15.105:0/2625386101 wait complete. 2024-03-26T17:43:05.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 134607 3.14 MB/sec execute 108 sec latency 19.177 ms 2024-03-26T17:43:06.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 135192 3.16 MB/sec execute 109 sec latency 17.195 ms 2024-03-26T17:43:06.051 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:05 smithi144 ceph-mon[36095]: pgmap v490: 97 pgs: 97 active+clean; 28 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:43:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:05 smithi105 ceph-mon[32280]: pgmap v490: 97 pgs: 97 active+clean; 28 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:43:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:05 smithi135 ceph-mon[35636]: pgmap v490: 97 pgs: 97 active+clean; 28 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:43:07.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 135644 3.13 MB/sec execute 110 sec latency 25.398 ms 2024-03-26T17:43:08.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 136268 3.15 MB/sec execute 111 sec latency 46.392 ms 2024-03-26T17:43:08.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:07 smithi144 ceph-mon[36095]: pgmap v491: 97 pgs: 97 active+clean; 26 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:43:08.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:07 smithi105 ceph-mon[32280]: pgmap v491: 97 pgs: 97 active+clean; 26 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:43:08.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:07 smithi135 ceph-mon[35636]: pgmap v491: 97 pgs: 97 active+clean; 26 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:43:09.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 136964 3.12 MB/sec execute 112 sec latency 15.117 ms 2024-03-26T17:43:10.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 137693 3.15 MB/sec execute 113 sec latency 19.475 ms 2024-03-26T17:43:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:09 smithi144 ceph-mon[36095]: pgmap v492: 97 pgs: 97 active+clean; 26 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:43:10.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:09 smithi105 ceph-mon[32280]: pgmap v492: 97 pgs: 97 active+clean; 26 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:43:10.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:09 smithi135 ceph-mon[35636]: pgmap v492: 97 pgs: 97 active+clean; 26 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:43:11.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 138215 3.15 MB/sec execute 114 sec latency 16.113 ms 2024-03-26T17:43:12.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 138780 3.16 MB/sec execute 115 sec latency 24.367 ms 2024-03-26T17:43:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:11 smithi144 ceph-mon[36095]: pgmap v493: 97 pgs: 97 active+clean; 27 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:43:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:11 smithi105 ceph-mon[32280]: pgmap v493: 97 pgs: 97 active+clean; 27 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:43:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:11 smithi135 ceph-mon[35636]: pgmap v493: 97 pgs: 97 active+clean; 27 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:43:13.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 139270 3.14 MB/sec execute 116 sec latency 24.325 ms 2024-03-26T17:43:14.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 139860 3.15 MB/sec execute 117 sec latency 97.268 ms 2024-03-26T17:43:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:13 smithi144 ceph-mon[36095]: pgmap v494: 97 pgs: 97 active+clean; 28 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:43:14.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:13 smithi105 ceph-mon[32280]: pgmap v494: 97 pgs: 97 active+clean; 28 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:43:14.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:13 smithi135 ceph-mon[35636]: pgmap v494: 97 pgs: 97 active+clean; 28 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:43:15.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 140511 3.13 MB/sec execute 118 sec latency 16.806 ms 2024-03-26T17:43:16.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 141267 3.15 MB/sec execute 119 sec latency 16.489 ms 2024-03-26T17:43:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:15 smithi144 ceph-mon[36095]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-03-26T17:43:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:15 smithi105 ceph-mon[32280]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-03-26T17:43:16.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:15 smithi135 ceph-mon[35636]: pgmap v495: 97 pgs: 97 active+clean; 28 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-03-26T17:43:17.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 141856 3.17 MB/sec execute 120 sec latency 17.101 ms 2024-03-26T17:43:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 -- 172.21.15.105:0/3198337817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab781019d0 msgr2=0x7fab78101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3198337817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab781019d0 0x7fab78101e50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009900 tx=0x7fab6c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 -- 172.21.15.105:0/3198337817 shutdown_connections 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3198337817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab78102390 0x7fab78108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3198337817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab781019d0 0x7fab78101e50 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3198337817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab781007e0 0x7fab78100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 -- 172.21.15.105:0/3198337817 >> 172.21.15.105:0/3198337817 conn(0x7fab780fbd50 msgr2=0x7fab780fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 -- 172.21.15.105:0/3198337817 shutdown_connections 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.455+0000 7fab7dc19640 1 -- 172.21.15.105:0/3198337817 wait complete. 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab7dc19640 1 Processor -- start 2024-03-26T17:43:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab7dc19640 1 -- start start 2024-03-26T17:43:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab7dc19640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab781007e0 0x7fab78197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab7dc19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab781019d0 0x7fab78197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab7dc19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab78102390 0x7fab7819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab7dc19640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810d430 con 0x7fab78102390 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab7dc19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810d2b0 con 0x7fab781019d0 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab7dc19640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab7810d5b0 con 0x7fab781007e0 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab78102390 0x7fab7819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab781007e0 0x7fab78197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab76ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab781019d0 0x7fab78197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab781007e0 0x7fab78197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60442/0 (socket says 172.21.15.105:60442) 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab777fe640 1 -- 172.21.15.105:0/3890854123 learned_addr learned my addr 172.21.15.105:0/3890854123 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab76ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab781019d0 0x7fab78197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50016/0 (socket says 172.21.15.105:50016) 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.456+0000 7fab77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab78102390 0x7fab7819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43680/0 (socket says 172.21.15.105:43680) 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab777fe640 1 -- 172.21.15.105:0/3890854123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab781019d0 msgr2=0x7fab78197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab777fe640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab781019d0 0x7fab78197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab777fe640 1 -- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab78102390 msgr2=0x7fab7819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab777fe640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab78102390 0x7fab7819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab777fe640 1 -- 172.21.15.105:0/3890854123 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7819f6a0 con 0x7fab781007e0 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab76ffd640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab781019d0 0x7fab78197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab777fe640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab781007e0 0x7fab78197a80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab6800e9c0 tx=0x7fab6800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab74ff9640 1 -- 172.21.15.105:0/3890854123 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6800cde0 con 0x7fab781007e0 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab74ff9640 1 -- 172.21.15.105:0/3890854123 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab680108e0 con 0x7fab781007e0 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab74ff9640 1 -- 172.21.15.105:0/3890854123 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680188e0 con 0x7fab781007e0 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab7dc19640 1 -- 172.21.15.105:0/3890854123 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7819f990 con 0x7fab781007e0 2024-03-26T17:43:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.457+0000 7fab7dc19640 1 -- 172.21.15.105:0/3890854123 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7819feb0 con 0x7fab781007e0 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.459+0000 7fab74ff9640 1 -- 172.21.15.105:0/3890854123 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fab6801f020 con 0x7fab781007e0 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab8080a740 1 Processor -- start 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab8080a740 1 -- start start 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab8080a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab8080a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda681ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab8080a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 0x560dda6849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab77fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43696/0 (socket says 172.21.15.105:43696) 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab77fff640 1 -- 172.21.15.105:0/2655499937 learned_addr learned my addr 172.21.15.105:0/2655499937 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab8080a740 1 -- 172.21.15.105:0/2655499937 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560dda5dd0b0 con 0x560dda67c9e0 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab8080a740 1 -- 172.21.15.105:0/2655499937 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560dda585960 con 0x560dda682540 2024-03-26T17:43:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab8080a740 1 -- 172.21.15.105:0/2655499937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560dda5d9060 con 0x560dda67d3d0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab777fe640 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda681ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab76ffd640 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 0x560dda6849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab74ff9640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab600a2fc0 0x7fab600a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab777fe640 1 -- 172.21.15.105:0/2655499937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 msgr2=0x560dda6849d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab777fe640 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 0x560dda6849d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab777fe640 1 -- 172.21.15.105:0/2655499937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 msgr2=0x560dda67ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.460+0000 7fab777fe640 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.461+0000 7fab777fe640 1 -- 172.21.15.105:0/2655499937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dda597c60 con 0x560dda67d3d0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.461+0000 7fab77fff640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab600a2fc0 0x7fab600a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.461+0000 7fab74ff9640 1 -- 172.21.15.105:0/3890854123 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fab680cf5b0 con 0x7fab781007e0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.461+0000 7fab777fe640 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda681ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fab680cd040 tx=0x7fab680667f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41605261b000fe57 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.461+0000 7fab5dffb640 1 -- 172.21.15.105:0/2655499937 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6809b600 con 0x560dda67d3d0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.461+0000 7fab77fff640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab600a2fc0 0x7fab600a5480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fab70009fd0 tx=0x7fab70004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.461+0000 7fab5dffb640 1 -- 172.21.15.105:0/2655499937 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab68010e20 con 0x560dda67d3d0 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.461+0000 7fab5dffb640 1 -- 172.21.15.105:0/2655499937 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680c69d0 con 0x560dda67d3d0 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 -- 172.21.15.105:0/2655499937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 msgr2=0x560dda681ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda681ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fab680cd040 tx=0x7fab680667f0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 -- 172.21.15.105:0/2655499937 shutdown_connections 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 0x560dda6849d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda681ed0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 --2- 172.21.15.105:0/2655499937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 -- 172.21.15.105:0/2655499937 >> 172.21.15.105:0/2655499937 conn(0x560dda673b40 msgr2=0x560dda673f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 -- 172.21.15.105:0/2655499937 shutdown_connections 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 -- 172.21.15.105:0/2655499937 wait complete. 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 -- start start 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 Processor -- start 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67c9e0 0x560dda6866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.462+0000 7fab8080a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda686c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab8080a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda682540 0x560dda68dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab8080a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560dda685490 con 0x560dda682540 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab8080a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560dda598db0 con 0x560dda67d3d0 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab8080a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560dda685610 con 0x560dda67c9e0 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab8080a740 10 client.0 _ll_register_callbacks cb 0x560dda59d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67c9e0 0x560dda6866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67c9e0 0x560dda6866c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60460/0 (socket says 172.21.15.105:60460) 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 -- 172.21.15.105:0/422062544 learned_addr learned my addr 172.21.15.105:0/422062544 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab77fff640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda682540 0x560dda68dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab76ffd640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda686c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 -- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 msgr2=0x560dda686c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda686c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 -- 172.21.15.105:0/422062544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda682540 msgr2=0x560dda68dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda682540 0x560dda68dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 -- 172.21.15.105:0/422062544 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dda2f1210 con 0x560dda67c9e0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab76ffd640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda686c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab77fff640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda682540 0x560dda68dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab777fe640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67c9e0 0x560dda6866c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fab680cd3e0 tx=0x7fab6806a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cda71017ef8bdc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab56ffd640 1 -- 172.21.15.105:0/422062544 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680d2e60 con 0x560dda67c9e0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab56ffd640 1 -- 172.21.15.105:0/422062544 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fab680affb0 con 0x560dda67c9e0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.463+0000 7fab8080a740 1 -- 172.21.15.105:0/422062544 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560dda5ea5f0 con 0x560dda67c9e0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab56ffd640 1 -- 172.21.15.105:0/422062544 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab6809b600 con 0x560dda67c9e0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab8080a740 10 client.27052 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab8080a740 1 -- 172.21.15.105:0/422062544 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560dda3bd620 con 0x560dda67c9e0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab56ffd640 1 -- 172.21.15.105:0/422062544 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab680c7e50 con 0x560dda67c9e0 2024-03-26T17:43:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab56ffd640 1 -- 172.21.15.105:0/422062544 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fab680b1030 con 0x560dda67c9e0 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab8080a740 10 client.27052 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab8080a740 10 client.27052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab8080a740 20 client.27052 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab8080a740 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560dda67ac60 0x560dda67b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab8080a740 4 client.27052 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab8080a740 1 -- 172.21.15.105:0/422062544 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dda5e4940 con 0x560dda67ac60 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.464+0000 7fab76ffd640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560dda67ac60 0x560dda67b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.465+0000 7fab76ffd640 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560dda67ac60 0x560dda67b0c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f49565ab2668a0 server_cookie=c0dedfb0cce750d8 in_seq=0 out_seq=0 2024-03-26T17:43:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.465+0000 7fab56ffd640 10 client.27052 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.467+0000 7fab56ffd640 1 -- 172.21.15.105:0/422062544 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560dda5e4940 con 0x560dda67ac60 2024-03-26T17:43:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.467+0000 7fab56ffd640 10 client.27052 handle_command_reply: tid=0 2024-03-26T17:43:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 client.27052 shutdown 2024-03-26T17:43:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 -- 172.21.15.105:0/422062544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67c9e0 msgr2=0x560dda6866c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67c9e0 0x560dda6866c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fab680cd3e0 tx=0x7fab6806a710 comp rx=0 tx=0).stop 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 -- 172.21.15.105:0/422062544 shutdown_connections 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560dda67ac60 0x560dda67b0c0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda682540 0x560dda68dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda686c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 --2- 172.21.15.105:0/422062544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67c9e0 0x560dda6866c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 -- 172.21.15.105:0/422062544 >> 172.21.15.105:0/422062544 conn(0x560dda673b40 msgr2=0x560dda672df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab56ffd640 0 client.27052 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 -- 172.21.15.105:0/422062544 shutdown_connections 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 1 -- 172.21.15.105:0/422062544 wait complete. 2024-03-26T17:43:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.468+0000 7fab8080a740 20 client.27052 trim_cache size 0 max 16384 2024-03-26T17:43:17.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab8080a740 1 Processor -- start 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab8080a740 1 -- start start 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab8080a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab8080a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda68dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab8080a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 0x560dda67ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab8080a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab781a6b50 con 0x560dda67c9e0 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab8080a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab781a8690 con 0x560dda682540 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab8080a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab781a7220 con 0x560dda67d3d0 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.486+0000 7fab777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43716/0 (socket says 172.21.15.105:43716) 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab777fe640 1 -- 172.21.15.105:0/1594216779 learned_addr learned my addr 172.21.15.105:0/1594216779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab76ffd640 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda68dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab77fff640 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 0x560dda67ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab76ffd640 1 -- 172.21.15.105:0/1594216779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 msgr2=0x560dda67ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:17.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab76ffd640 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 0x560dda67ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab76ffd640 1 -- 172.21.15.105:0/1594216779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 msgr2=0x560dda67e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab76ffd640 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab76ffd640 1 -- 172.21.15.105:0/1594216779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dda67b430 con 0x560dda67d3d0 2024-03-26T17:43:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab777fe640 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab76ffd640 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda68dc00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab6c000c00 tx=0x7fab6c02cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ce397af706a5f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab5d7fa640 1 -- 172.21.15.105:0/1594216779 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0041a0 con 0x560dda67d3d0 2024-03-26T17:43:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab5d7fa640 1 -- 172.21.15.105:0/1594216779 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab6c04c070 con 0x560dda67d3d0 2024-03-26T17:43:17.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.487+0000 7fab5d7fa640 1 -- 172.21.15.105:0/1594216779 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c004a80 con 0x560dda67d3d0 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 -- 172.21.15.105:0/1594216779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 msgr2=0x560dda68dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda68dc00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab6c000c00 tx=0x7fab6c02cd90 comp rx=0 tx=0).stop 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 -- 172.21.15.105:0/1594216779 shutdown_connections 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda682540 0x560dda67ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda67d3d0 0x560dda68dc00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 --2- 172.21.15.105:0/1594216779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda67e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 -- 172.21.15.105:0/1594216779 >> 172.21.15.105:0/1594216779 conn(0x560dda673b40 msgr2=0x560dda681620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 -- 172.21.15.105:0/1594216779 shutdown_connections 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 -- 172.21.15.105:0/1594216779 wait complete. 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.488+0000 7fab8080a740 1 -- start start 2024-03-26T17:43:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab8080a740 1 Processor -- start 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab8080a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda68d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab8080a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda68d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab8080a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda682540 0x560dda689fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab8080a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab781a7fc0 con 0x560dda67c9e0 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab8080a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab781a78f0 con 0x560dda67d3d0 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab8080a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab781a4b30 con 0x560dda682540 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab8080a740 10 client.0 _ll_register_callbacks cb 0x7fab70009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda68d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda68d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43718/0 (socket says 172.21.15.105:43718) 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab777fe640 1 -- 172.21.15.105:0/3079088763 learned_addr learned my addr 172.21.15.105:0/3079088763 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab76ffd640 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda68d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab77fff640 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda682540 0x560dda689fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab777fe640 1 -- 172.21.15.105:0/3079088763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda682540 msgr2=0x560dda689fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab777fe640 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda682540 0x560dda689fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab777fe640 1 -- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 msgr2=0x560dda68d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab777fe640 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda68d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab777fe640 1 -- 172.21.15.105:0/3079088763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dda68a6c0 con 0x560dda67c9e0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.489+0000 7fab76ffd640 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda68d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab777fe640 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda68d490 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fab68099f50 tx=0x7fab680a43e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e62d49b5b97e02 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab56ffd640 1 -- 172.21.15.105:0/3079088763 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6809c3a0 con 0x560dda67c9e0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab56ffd640 1 -- 172.21.15.105:0/3079088763 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fab68010a90 con 0x560dda67c9e0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 1 -- 172.21.15.105:0/3079088763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6c04c070 con 0x560dda67c9e0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab56ffd640 1 -- 172.21.15.105:0/3079088763 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab6809fdf0 con 0x560dda67c9e0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 10 client.17220 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 1 -- 172.21.15.105:0/3079088763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560dda6fc200 con 0x560dda67c9e0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab56ffd640 1 -- 172.21.15.105:0/3079088763 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab6c04c070 con 0x560dda67c9e0 2024-03-26T17:43:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab56ffd640 1 -- 172.21.15.105:0/3079088763 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fab680b1070 con 0x560dda67c9e0 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 10 client.17220 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 10 client.17220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 20 client.17220 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560dda6fca20 0x560dda6fee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 4 client.17220 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.490+0000 7fab8080a740 1 -- 172.21.15.105:0/3079088763 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dda562770 con 0x560dda6fca20 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.491+0000 7fab76ffd640 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560dda6fca20 0x560dda6fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.491+0000 7fab56ffd640 10 client.17220 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.491+0000 7fab76ffd640 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560dda6fca20 0x560dda6fee40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3effb5060f8d6b4d server_cookie=7b3fdb8d4dd0a62 in_seq=0 out_seq=0 2024-03-26T17:43:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.547+0000 7fab56ffd640 1 -- 172.21.15.105:0/3079088763 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560dda562770 con 0x560dda6fca20 2024-03-26T17:43:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.547+0000 7fab56ffd640 10 client.17220 handle_command_reply: tid=0 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.547+0000 7fab8080a740 1 client.17220 shutdown 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 -- 172.21.15.105:0/3079088763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 msgr2=0x560dda68d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda68d490 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fab68099f50 tx=0x7fab680a43e0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 -- 172.21.15.105:0/3079088763 shutdown_connections 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560dda6fca20 0x560dda6fee40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560dda682540 0x560dda689fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560dda67d3d0 0x560dda68d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 --2- 172.21.15.105:0/3079088763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560dda67c9e0 0x560dda68d490 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 -- 172.21.15.105:0/3079088763 >> 172.21.15.105:0/3079088763 conn(0x560dda673b40 msgr2=0x560dda67e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab56ffd640 0 client.17220 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 -- 172.21.15.105:0/3079088763 shutdown_connections 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 1 -- 172.21.15.105:0/3079088763 wait complete. 2024-03-26T17:43:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.548+0000 7fab8080a740 20 client.17220 trim_cache size 0 max 16384 2024-03-26T17:43:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:43:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:43:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:43:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:43:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 -- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab600a2fc0 msgr2=0x7fab600a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab600a2fc0 0x7fab600a5480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fab70009fd0 tx=0x7fab70004a40 comp rx=0 tx=0).stop 2024-03-26T17:43:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 -- 172.21.15.105:0/3890854123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab781007e0 msgr2=0x7fab78197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab781007e0 0x7fab78197a80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab6800e9c0 tx=0x7fab6800ee80 comp rx=0 tx=0).stop 2024-03-26T17:43:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 -- 172.21.15.105:0/3890854123 shutdown_connections 2024-03-26T17:43:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab600a2fc0 0x7fab600a5480 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab78102390 0x7fab7819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab781019d0 0x7fab78197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 --2- 172.21.15.105:0/3890854123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab781007e0 0x7fab78197a80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 -- 172.21.15.105:0/3890854123 >> 172.21.15.105:0/3890854123 conn(0x7fab780fbd50 msgr2=0x7fab78104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 -- 172.21.15.105:0/3890854123 shutdown_connections 2024-03-26T17:43:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:17.549+0000 7fab7dc19640 1 -- 172.21.15.105:0/3890854123 wait complete. 2024-03-26T17:43:18.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 142316 3.16 MB/sec execute 121 sec latency 18.482 ms 2024-03-26T17:43:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:17 smithi144 ceph-mon[36095]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:43:18.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:17 smithi105 ceph-mon[32280]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:43:18.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:17 smithi135 ceph-mon[35636]: pgmap v496: 97 pgs: 97 active+clean; 28 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:43:19.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 142792 3.14 MB/sec execute 122 sec latency 20.841 ms 2024-03-26T17:43:20.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 143430 3.15 MB/sec execute 123 sec latency 17.112 ms 2024-03-26T17:43:20.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:19 smithi144 ceph-mon[36095]: pgmap v497: 97 pgs: 97 active+clean; 28 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:43:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:19 smithi105 ceph-mon[32280]: pgmap v497: 97 pgs: 97 active+clean; 28 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:43:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:19 smithi135 ceph-mon[35636]: pgmap v497: 97 pgs: 97 active+clean; 28 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:43:21.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 144087 3.13 MB/sec execute 124 sec latency 18.048 ms 2024-03-26T17:43:22.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 144782 3.15 MB/sec execute 125 sec latency 17.921 ms 2024-03-26T17:43:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:21 smithi144 ceph-mon[36095]: pgmap v498: 97 pgs: 97 active+clean; 29 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:43:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:21 smithi105 ceph-mon[32280]: pgmap v498: 97 pgs: 97 active+clean; 29 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:43:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:21 smithi135 ceph-mon[35636]: pgmap v498: 97 pgs: 97 active+clean; 29 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:43:22.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:43:22.933 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:43:23.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 145281 3.15 MB/sec execute 126 sec latency 18.804 ms 2024-03-26T17:43:23.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 -- 172.21.15.105:0/2035995202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8418102370 msgr2=0x7f8418108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 --2- 172.21.15.105:0/2035995202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8418102370 0x7f8418108c00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8410009ca0 tx=0x7f841002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 -- 172.21.15.105:0/2035995202 shutdown_connections 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 --2- 172.21.15.105:0/2035995202 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8418102370 0x7f8418108c00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 --2- 172.21.15.105:0/2035995202 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84181019b0 0x7f8418101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 --2- 172.21.15.105:0/2035995202 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84181007c0 0x7f8418100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 -- 172.21.15.105:0/2035995202 >> 172.21.15.105:0/2035995202 conn(0x7f84180fbd50 msgr2=0x7f84180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 -- 172.21.15.105:0/2035995202 shutdown_connections 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.061+0000 7f841f952640 1 -- 172.21.15.105:0/2035995202 wait complete. 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841f952640 1 Processor -- start 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841f952640 1 -- start start 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841f952640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84181007c0 0x7f8418104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841f952640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84181019b0 0x7f8418102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841f952640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8418102370 0x7f84181032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841f952640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f841810c2f0 con 0x7f84181019b0 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841f952640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f841810c170 con 0x7f84181007c0 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841f952640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f841810c470 con 0x7f8418102370 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841d6ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84181007c0 0x7f8418104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841ceed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84181019b0 0x7f8418102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841d6ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84181007c0 0x7f8418104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50078/0 (socket says 172.21.15.105:50078) 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841d6ee640 1 -- 172.21.15.105:0/2496370007 learned_addr learned my addr 172.21.15.105:0/2496370007 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.062+0000 7f841deef640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8418102370 0x7f84181032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841ceed640 1 -- 172.21.15.105:0/2496370007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8418102370 msgr2=0x7f84181032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841ceed640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8418102370 0x7f84181032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841ceed640 1 -- 172.21.15.105:0/2496370007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84181007c0 msgr2=0x7f8418104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841ceed640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84181007c0 0x7f8418104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841ceed640 1 -- 172.21.15.105:0/2496370007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8418103b60 con 0x7f84181019b0 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841d6ee640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84181007c0 0x7f8418104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841ceed640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84181019b0 0x7f8418102d90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f840800e9f0 tx=0x7f840800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f840e7fc640 1 -- 172.21.15.105:0/2496370007 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840800ccc0 con 0x7f84181019b0 2024-03-26T17:43:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f840e7fc640 1 -- 172.21.15.105:0/2496370007 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84080108e0 con 0x7f84181019b0 2024-03-26T17:43:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841f952640 1 -- 172.21.15.105:0/2496370007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84181a1c30 con 0x7f84181019b0 2024-03-26T17:43:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f840e7fc640 1 -- 172.21.15.105:0/2496370007 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408018880 con 0x7f84181019b0 2024-03-26T17:43:23.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.063+0000 7f841f952640 1 -- 172.21.15.105:0/2496370007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84181a2180 con 0x7f84181019b0 2024-03-26T17:43:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.065+0000 7f840e7fc640 1 -- 172.21.15.105:0/2496370007 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8408010a90 con 0x7f84181019b0 2024-03-26T17:43:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.066+0000 7f840e7fc640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84040a2f20 0x7f84040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.066+0000 7f841d6ee640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84040a2f20 0x7f84040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.066+0000 7f840e7fc640 1 -- 172.21.15.105:0/2496370007 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f84080cf720 con 0x7f84181019b0 2024-03-26T17:43:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f842260f740 1 Processor -- start 2024-03-26T17:43:23.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f842260f740 1 -- start start 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841d6ee640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84040a2f20 0x7f84040a53e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f84140058c0 tx=0x7f8414005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f842260f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f842260f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f92e0 0x55bfa42fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f842260f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa43008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f842260f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bfa425d980 con 0x55bfa42fe450 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f842260f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bfa4205be0 con 0x55bfa42f88f0 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f842260f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bfa4259930 con 0x55bfa42f92e0 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841d6ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841deef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa43008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841d6ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50090/0 (socket says 172.21.15.105:50090) 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841deef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa43008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43738/0 (socket says 172.21.15.105:43738) 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841d6ee640 1 -- 172.21.15.105:0/1566043906 learned_addr learned my addr 172.21.15.105:0/1566043906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841ceed640 1 --2- 172.21.15.105:0/1566043906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f92e0 0x55bfa42fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841d6ee640 1 -- 172.21.15.105:0/1566043906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f92e0 msgr2=0x55bfa42fdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841d6ee640 1 --2- 172.21.15.105:0/1566043906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f92e0 0x55bfa42fdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841d6ee640 1 -- 172.21.15.105:0/1566043906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 msgr2=0x55bfa43008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.067+0000 7f841d6ee640 1 --2- 172.21.15.105:0/1566043906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa43008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.068+0000 7f841d6ee640 1 -- 172.21.15.105:0/1566043906 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa403b7d0 con 0x55bfa42f88f0 2024-03-26T17:43:23.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.068+0000 7f841d6ee640 1 --2- 172.21.15.105:0/1566043906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f8d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f841400f6f0 tx=0x7f8414014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36697fd5b727395 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.068+0000 7f84037fe640 1 -- 172.21.15.105:0/1566043906 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841402a070 con 0x55bfa42f88f0 2024-03-26T17:43:23.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.068+0000 7f84037fe640 1 -- 172.21.15.105:0/1566043906 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f841401b8e0 con 0x55bfa42f88f0 2024-03-26T17:43:23.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.068+0000 7f84037fe640 1 -- 172.21.15.105:0/1566043906 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841401c9e0 con 0x55bfa42f88f0 2024-03-26T17:43:23.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 -- 172.21.15.105:0/1566043906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 msgr2=0x55bfa42f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 --2- 172.21.15.105:0/1566043906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f8d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f841400f6f0 tx=0x7f8414014000 comp rx=0 tx=0).stop 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 -- 172.21.15.105:0/1566043906 shutdown_connections 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 --2- 172.21.15.105:0/1566043906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa43008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 --2- 172.21.15.105:0/1566043906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f92e0 0x55bfa42fdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 --2- 172.21.15.105:0/1566043906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f8d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 -- 172.21.15.105:0/1566043906 >> 172.21.15.105:0/1566043906 conn(0x55bfa42f3e60 msgr2=0x55bfa42f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 -- 172.21.15.105:0/1566043906 shutdown_connections 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 -- 172.21.15.105:0/1566043906 wait complete. 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 -- start start 2024-03-26T17:43:23.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.069+0000 7f842260f740 1 Processor -- start 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f842260f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa4302b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f842260f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa4303080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f842260f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42fe450 0x55bfa430a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f842260f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bfa4301400 con 0x55bfa42f92e0 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f842260f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bfa4219160 con 0x55bfa42fe450 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f842260f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bfa4301580 con 0x55bfa42f88f0 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa4303080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f842260f740 10 client.0 _ll_register_callbacks cb 0x55bfa4255720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa4303080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43744/0 (socket says 172.21.15.105:43744) 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 -- 172.21.15.105:0/1831888998 learned_addr learned my addr 172.21.15.105:0/1831888998 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841deef640 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42fe450 0x55bfa430a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841d6ee640 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa4302b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 -- 172.21.15.105:0/1831888998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 msgr2=0x55bfa4302b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa4302b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 -- 172.21.15.105:0/1831888998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42fe450 msgr2=0x55bfa430a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42fe450 0x55bfa430a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 -- 172.21.15.105:0/1831888998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa3f713f0 con 0x55bfa42f92e0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f841ceed640 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa4303080 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f84080cd230 tx=0x7f8408010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13a2a155e5ca23b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f8400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f8400ff9640 1 -- 172.21.15.105:0/1831888998 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080b4020 con 0x55bfa42f92e0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f8400ff9640 1 -- 172.21.15.105:0/1831888998 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f84080a4590 con 0x55bfa42f92e0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.070+0000 7f842260f740 1 -- 172.21.15.105:0/1831888998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f841401b8e0 con 0x55bfa42f92e0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f8400ff9640 1 -- 172.21.15.105:0/1831888998 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f84080a4820 con 0x55bfa42f92e0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f842260f740 10 client.17232 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f842260f740 1 -- 172.21.15.105:0/1831888998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfa3f310f0 con 0x55bfa42f92e0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f8400ff9640 1 -- 172.21.15.105:0/1831888998 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f841401b8e0 con 0x55bfa42f92e0 2024-03-26T17:43:23.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f8400ff9640 1 -- 172.21.15.105:0/1831888998 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8408097c10 con 0x55bfa42f92e0 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f842260f740 10 client.17232 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f842260f740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f842260f740 20 client.17232 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f842260f740 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfa430e730 0x55bfa4310b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f842260f740 4 client.17232 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f842260f740 1 -- 172.21.15.105:0/1831888998 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfa426aec0 con 0x55bfa430e730 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.071+0000 7f841d6ee640 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfa430e730 0x55bfa4310b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.072+0000 7f8400ff9640 10 client.17232 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.072+0000 7f841d6ee640 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfa430e730 0x55bfa4310b90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef6ca6c3ced70df server_cookie=fb1b69cf9d15d9d6 in_seq=0 out_seq=0 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f8400ff9640 1 -- 172.21.15.105:0/1831888998 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bfa426aec0 con 0x55bfa430e730 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f8400ff9640 10 client.17232 handle_command_reply: tid=0 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 client.17232 shutdown 2024-03-26T17:43:23.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 -- 172.21.15.105:0/1831888998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 msgr2=0x55bfa4303080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa4303080 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f84080cd230 tx=0x7f8408010ec0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 -- 172.21.15.105:0/1831888998 shutdown_connections 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfa430e730 0x55bfa4310b90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42fe450 0x55bfa430a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa4303080 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 --2- 172.21.15.105:0/1831888998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa4302b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 -- 172.21.15.105:0/1831888998 >> 172.21.15.105:0/1831888998 conn(0x55bfa42f3e60 msgr2=0x55bfa42f3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f8400ff9640 0 client.17232 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 -- 172.21.15.105:0/1831888998 shutdown_connections 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.073+0000 7f842260f740 1 -- 172.21.15.105:0/1831888998 wait complete. 2024-03-26T17:43:23.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.074+0000 7f842260f740 20 client.17232 trim_cache size 0 max 16384 2024-03-26T17:43:23.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.092+0000 7f842260f740 1 Processor -- start 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.092+0000 7f842260f740 1 -- start start 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f842260f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f842260f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa430a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f842260f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42fe450 0x55bfa42fa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f842260f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84181ab0a0 con 0x55bfa42f92e0 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f842260f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84181acbe0 con 0x55bfa42f88f0 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f842260f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84181ab770 con 0x55bfa42fe450 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa430a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa430a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43754/0 (socket says 172.21.15.105:43754) 2024-03-26T17:43:23.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 -- 172.21.15.105:0/2154513131 learned_addr learned my addr 172.21.15.105:0/2154513131 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841d6ee640 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841deef640 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42fe450 0x55bfa42fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 -- 172.21.15.105:0/2154513131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42fe450 msgr2=0x55bfa42fa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42fe450 0x55bfa42fa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 -- 172.21.15.105:0/2154513131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 msgr2=0x55bfa42f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 -- 172.21.15.105:0/2154513131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa42f86e0 con 0x55bfa42f92e0 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f841ceed640 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa430a010 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8408097fb0 tx=0x7f840809a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113b3e9a4c297b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f8402ffd640 1 -- 172.21.15.105:0/2154513131 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080b4070 con 0x55bfa42f92e0 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f8402ffd640 1 -- 172.21.15.105:0/2154513131 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f840809aec0 con 0x55bfa42f92e0 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.093+0000 7f8402ffd640 1 -- 172.21.15.105:0/2154513131 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080a45f0 con 0x55bfa42f92e0 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 -- 172.21.15.105:0/2154513131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 msgr2=0x55bfa430a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa430a010 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8408097fb0 tx=0x7f840809a3d0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 -- 172.21.15.105:0/2154513131 shutdown_connections 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42fe450 0x55bfa42fa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42f92e0 0x55bfa430a010 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 --2- 172.21.15.105:0/2154513131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f88f0 0x55bfa42f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 -- 172.21.15.105:0/2154513131 >> 172.21.15.105:0/2154513131 conn(0x55bfa42f3e60 msgr2=0x55bfa4302b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 -- 172.21.15.105:0/2154513131 shutdown_connections 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 -- 172.21.15.105:0/2154513131 wait complete. 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 -- start start 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.094+0000 7f842260f740 1 Processor -- start 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f842260f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa43072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f842260f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f92e0 0x55bfa4307830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f842260f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa4308840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f842260f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84181ac510 con 0x55bfa42fe450 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f842260f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84181abe40 con 0x55bfa42f92e0 2024-03-26T17:43:23.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f842260f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84181a9060 con 0x55bfa42f88f0 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f842260f740 10 client.0 _ll_register_callbacks cb 0x7f840809bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841d6ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa43072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841d6ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa43072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60526/0 (socket says 172.21.15.105:60526) 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841d6ee640 1 -- 172.21.15.105:0/2696957852 learned_addr learned my addr 172.21.15.105:0/2696957852 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841deef640 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa4308840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841d6ee640 1 -- 172.21.15.105:0/2696957852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f92e0 msgr2=0x55bfa4307830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841d6ee640 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f92e0 0x55bfa4307830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841d6ee640 1 -- 172.21.15.105:0/2696957852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 msgr2=0x55bfa4308840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841d6ee640 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa4308840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.095+0000 7f841d6ee640 1 -- 172.21.15.105:0/2696957852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa4308e30 con 0x55bfa42f88f0 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f841d6ee640 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa43072f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f841400fbb0 tx=0x7f8414005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a340661ea4e1d4f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f8400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f8400ff9640 1 -- 172.21.15.105:0/2696957852 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841402a040 con 0x55bfa42f88f0 2024-03-26T17:43:23.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 1 -- 172.21.15.105:0/2696957852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f840809aec0 con 0x55bfa42f88f0 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f8400ff9640 1 -- 172.21.15.105:0/2696957852 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8414032500 con 0x55bfa42f88f0 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f8400ff9640 1 -- 172.21.15.105:0/2696957852 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8414032700 con 0x55bfa42f88f0 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 10 client.27076 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 1 -- 172.21.15.105:0/2696957852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfa430a140 con 0x55bfa42f88f0 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f8400ff9640 1 -- 172.21.15.105:0/2696957852 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f840809aec0 con 0x55bfa42f88f0 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f8400ff9640 1 -- 172.21.15.105:0/2696957852 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8414024030 con 0x55bfa42f88f0 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 10 client.27076 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 10 client.27076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 20 client.27076 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfa4374c30 0x55bfa4377020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 4 client.27076 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.096+0000 7f842260f740 1 -- 172.21.15.105:0/2696957852 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bfa4265210 con 0x55bfa4374c30 2024-03-26T17:43:23.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f841ceed640 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfa4374c30 0x55bfa4377020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f8400ff9640 10 client.27076 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f841ceed640 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfa4374c30 0x55bfa4377020 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71a5e48abe2093c server_cookie=3ab38ba1585ac0b2 in_seq=0 out_seq=0 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f8400ff9640 1 -- 172.21.15.105:0/2696957852 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bfa4265210 con 0x55bfa4374c30 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f8400ff9640 10 client.27076 handle_command_reply: tid=0 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 client.27076 shutdown 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 -- 172.21.15.105:0/2696957852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 msgr2=0x55bfa43072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa43072f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f841400fbb0 tx=0x7f8414005040 comp rx=0 tx=0).stop 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 -- 172.21.15.105:0/2696957852 shutdown_connections 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfa4374c30 0x55bfa4377020 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfa42fe450 0x55bfa4308840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfa42f92e0 0x55bfa4307830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 --2- 172.21.15.105:0/2696957852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfa42f88f0 0x55bfa43072f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f8400ff9640 0 client.27076 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 -- 172.21.15.105:0/2696957852 >> 172.21.15.105:0/2696957852 conn(0x55bfa42f3e60 msgr2=0x55bfa42fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 -- 172.21.15.105:0/2696957852 shutdown_connections 2024-03-26T17:43:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 1 -- 172.21.15.105:0/2696957852 wait complete. 2024-03-26T17:43:23.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.097+0000 7f842260f740 20 client.27076 trim_cache size 0 max 16384 2024-03-26T17:43:23.096 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:43:23.096 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:43:23.096 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:43:23.096 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:43:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 -- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84040a2f20 msgr2=0x7f84040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84040a2f20 0x7f84040a53e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f84140058c0 tx=0x7f8414005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 -- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84181019b0 msgr2=0x7f8418102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84181019b0 0x7f8418102d90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f840800e9f0 tx=0x7f840800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 -- 172.21.15.105:0/2496370007 shutdown_connections 2024-03-26T17:43:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84040a2f20 0x7f84040a53e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8418102370 0x7f84181032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84181019b0 0x7f8418102d90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 --2- 172.21.15.105:0/2496370007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84181007c0 0x7f8418104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 -- 172.21.15.105:0/2496370007 >> 172.21.15.105:0/2496370007 conn(0x7f84180fbd50 msgr2=0x7f8418068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 -- 172.21.15.105:0/2496370007 shutdown_connections 2024-03-26T17:43:23.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.098+0000 7f841f952640 1 -- 172.21.15.105:0/2496370007 wait complete. 2024-03-26T17:43:23.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:43:23.102 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:43:23.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.226+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1480998911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac1019d0 msgr2=0x7ff0ac101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.226+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1480998911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac1019d0 0x7ff0ac101e50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009900 tx=0x7ff0a002bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.227+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1480998911 shutdown_connections 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.227+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1480998911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ac102390 0x7ff0ac108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.227+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1480998911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac1019d0 0x7ff0ac101e50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.227+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1480998911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0ac1007e0 0x7ff0ac100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.227+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1480998911 >> 172.21.15.105:0/1480998911 conn(0x7ff0ac0fbd50 msgr2=0x7ff0ac0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.227+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1480998911 shutdown_connections 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.227+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1480998911 wait complete. 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.227+0000 7ff0b11e6640 1 Processor -- start 2024-03-26T17:43:23.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0b11e6640 1 -- start start 2024-03-26T17:43:23.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0b11e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0ac1007e0 0x7ff0ac069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0b11e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ac1019d0 0x7ff0ac19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0b11e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac102390 0x7ff0ac19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0b11e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac10c2f0 con 0x7ff0ac102390 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0b11e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac10c170 con 0x7ff0ac1007e0 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0b11e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac10c470 con 0x7ff0ac1019d0 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0ac1007e0 0x7ff0ac069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0ab59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac102390 0x7ff0ac19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aa59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ac1019d0 0x7ff0ac19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0ac1007e0 0x7ff0ac069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50142/0 (socket says 172.21.15.105:50142) 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 -- 172.21.15.105:0/1729664281 learned_addr learned my addr 172.21.15.105:0/1729664281 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 -- 172.21.15.105:0/1729664281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ac1019d0 msgr2=0x7ff0ac19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ac1019d0 0x7ff0ac19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 -- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac102390 msgr2=0x7ff0ac19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac102390 0x7ff0ac19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 -- 172.21.15.105:0/1729664281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac19f7a0 con 0x7ff0ac1007e0 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0ab59e640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac102390 0x7ff0ac19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.228+0000 7ff0aad9d640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0ac1007e0 0x7ff0ac069590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00e9c0 tx=0x7ff09c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.229+0000 7ff093fff640 1 -- 172.21.15.105:0/1729664281 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00cd80 con 0x7ff0ac1007e0 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.229+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac19fa30 con 0x7ff0ac1007e0 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.229+0000 7ff093fff640 1 -- 172.21.15.105:0/1729664281 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff09c0035f0 con 0x7ff0ac1007e0 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.229+0000 7ff093fff640 1 -- 172.21.15.105:0/1729664281 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c010690 con 0x7ff0ac1007e0 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.229+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac19ffa0 con 0x7ff0ac1007e0 2024-03-26T17:43:23.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.230+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac0686b0 con 0x7ff0ac1007e0 2024-03-26T17:43:23.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.231+0000 7ff093fff640 1 -- 172.21.15.105:0/1729664281 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff09c0037a0 con 0x7ff0ac1007e0 2024-03-26T17:43:23.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.232+0000 7ff093fff640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0940a2f20 0x7ff0940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.232+0000 7ff0aa59c640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0940a2f20 0x7ff0940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.232+0000 7ff093fff640 1 -- 172.21.15.105:0/1729664281 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff09c0cf7a0 con 0x7ff0ac1007e0 2024-03-26T17:43:23.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.232+0000 7ff0aa59c640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0940a2f20 0x7ff0940a53e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009230 tx=0x7ff0a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.238+0000 7ff093fff640 1 -- 172.21.15.105:0/1729664281 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff09c09c590 con 0x7ff0ac1007e0 2024-03-26T17:43:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.519+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0ac107b80 con 0x7ff0ac1007e0 2024-03-26T17:43:23.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.519+0000 7ff093fff640 1 -- 172.21.15.105:0/1729664281 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7ff09c09be90 con 0x7ff0ac1007e0 2024-03-26T17:43:23.722 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:43:23.722 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:43:23.724 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:43:23.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0940a2f20 msgr2=0x7ff0940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0940a2f20 0x7ff0940a53e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009230 tx=0x7ff0a0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0ac1007e0 msgr2=0x7ff0ac069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0ac1007e0 0x7ff0ac069590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00e9c0 tx=0x7ff09c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:43:23.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 shutdown_connections 2024-03-26T17:43:23.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff0940a2f20 0x7ff0940a53e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0ac102390 0x7ff0ac19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0ac1019d0 0x7ff0ac19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 --2- 172.21.15.105:0/1729664281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0ac1007e0 0x7ff0ac069590 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 >> 172.21.15.105:0/1729664281 conn(0x7ff0ac0fbd50 msgr2=0x7ff0ac104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 shutdown_connections 2024-03-26T17:43:23.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.522+0000 7ff0b11e6640 1 -- 172.21.15.105:0/1729664281 wait complete. 2024-03-26T17:43:23.726 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:43:23.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 -- 172.21.15.105:0/3603842385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 msgr2=0x7f9344101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 --2- 172.21.15.105:0/3603842385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 0x7f9344101e30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9334009900 tx=0x7f933402bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 -- 172.21.15.105:0/3603842385 shutdown_connections 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 --2- 172.21.15.105:0/3603842385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9344102370 0x7f9344108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 --2- 172.21.15.105:0/3603842385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 0x7f9344101e30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 --2- 172.21.15.105:0/3603842385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93441007c0 0x7f9344100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 -- 172.21.15.105:0/3603842385 >> 172.21.15.105:0/3603842385 conn(0x7f93440fbd50 msgr2=0x7f93440fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 -- 172.21.15.105:0/3603842385 shutdown_connections 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.850+0000 7f934c6c6640 1 -- 172.21.15.105:0/3603842385 wait complete. 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934c6c6640 1 Processor -- start 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934c6c6640 1 -- start start 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934c6c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93441007c0 0x7f9344197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934c6c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 0x7f9344198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934c6c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9344102370 0x7f934419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934c6c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f934410c2d0 con 0x7f93441019b0 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934c6c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f934410c150 con 0x7f9344102370 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934c6c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f934410c450 con 0x7f93441007c0 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f9349c61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 0x7f9344198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f9349c61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 0x7f9344198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43784/0 (socket says 172.21.15.105:43784) 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f9349c61640 1 -- 172.21.15.105:0/2509989037 learned_addr learned my addr 172.21.15.105:0/2509989037 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934a462640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93441007c0 0x7f9344197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934ac63640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9344102370 0x7f934419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f9349c61640 1 -- 172.21.15.105:0/2509989037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93441007c0 msgr2=0x7f9344197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f9349c61640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93441007c0 0x7f9344197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f9349c61640 1 -- 172.21.15.105:0/2509989037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9344102370 msgr2=0x7f934419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f9349c61640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9344102370 0x7f934419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f9349c61640 1 -- 172.21.15.105:0/2509989037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934419f770 con 0x7f93441019b0 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934a462640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93441007c0 0x7f9344197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.851+0000 7f934ac63640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9344102370 0x7f934419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.852+0000 7f9349c61640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 0x7f9344198090 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9334006010 tx=0x7f9334005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.852+0000 7f933b7fe640 1 -- 172.21.15.105:0/2509989037 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334043020 con 0x7f93441019b0 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.852+0000 7f933b7fe640 1 -- 172.21.15.105:0/2509989037 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93340043d0 con 0x7f93441019b0 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.852+0000 7f934c6c6640 1 -- 172.21.15.105:0/2509989037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9344102bd0 con 0x7f93441019b0 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.852+0000 7f933b7fe640 1 -- 172.21.15.105:0/2509989037 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334003b80 con 0x7f93441019b0 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.852+0000 7f934c6c6640 1 -- 172.21.15.105:0/2509989037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9344103120 con 0x7f93441019b0 2024-03-26T17:43:23.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.855+0000 7f933b7fe640 1 -- 172.21.15.105:0/2509989037 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9334046020 con 0x7f93441019b0 2024-03-26T17:43:23.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.855+0000 7f934f394740 1 Processor -- start 2024-03-26T17:43:23.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.855+0000 7f934f394740 1 -- start start 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934f394740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafaa30 0x56518dafae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934f394740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518dafff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934f394740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518db00590 0x56518db02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934f394740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56518da5f970 con 0x56518db00590 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934f394740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56518da07d70 con 0x56518dafb420 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934f394740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56518da5b920 con 0x56518dafaa30 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f933b7fe640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93300a2f20 0x7f93300a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934a462640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518dafff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934a462640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518dafff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50168/0 (socket says 172.21.15.105:50168) 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934a462640 1 -- 172.21.15.105:0/2391362940 learned_addr learned my addr 172.21.15.105:0/2391362940 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f9349c61640 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518db00590 0x56518db02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934ac63640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93300a2f20 0x7f93300a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934ac63640 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafaa30 0x56518dafae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934a462640 1 -- 172.21.15.105:0/2391362940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafaa30 msgr2=0x56518dafae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f933b7fe640 1 -- 172.21.15.105:0/2509989037 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f93340f1600 con 0x7f93441019b0 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934a462640 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafaa30 0x56518dafae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934a462640 1 -- 172.21.15.105:0/2391362940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518db00590 msgr2=0x56518db02a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934a462640 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518db00590 0x56518db02a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934a462640 1 -- 172.21.15.105:0/2391362940 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56518d7733f0 con 0x56518dafb420 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.856+0000 7f934ac63640 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafaa30 0x56518dafae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.857+0000 7f934a462640 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518dafff20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f93400114c0 tx=0x7f9340011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10050c4a36bfbd79 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.857+0000 7f934ac63640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93300a2f20 0x7f93300a53e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x56518dafbc80 tx=0x7f933c0106e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.857+0000 7f9338ff9640 1 -- 172.21.15.105:0/2391362940 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340004020 con 0x56518dafb420 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.857+0000 7f9338ff9640 1 -- 172.21.15.105:0/2391362940 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93400041c0 con 0x56518dafb420 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.857+0000 7f9338ff9640 1 -- 172.21.15.105:0/2391362940 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934000f860 con 0x56518dafb420 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- 172.21.15.105:0/2391362940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 msgr2=0x56518dafff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518dafff20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f93400114c0 tx=0x7f9340011980 comp rx=0 tx=0).stop 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- 172.21.15.105:0/2391362940 shutdown_connections 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518db00590 0x56518db02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518dafff20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 --2- 172.21.15.105:0/2391362940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafaa30 0x56518dafae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- 172.21.15.105:0/2391362940 >> 172.21.15.105:0/2391362940 conn(0x56518daf5fa0 msgr2=0x56518daf63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- 172.21.15.105:0/2391362940 shutdown_connections 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- 172.21.15.105:0/2391362940 wait complete. 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- start start 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 Processor -- start 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafaa30 0x56518db04f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafb420 0x56518db05490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 0x56518db0c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56518db03540 con 0x56518dafb420 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56518da1b310 con 0x56518dafaa30 2024-03-26T17:43:23.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56518db036c0 con 0x56518db00590 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f934f394740 10 client.0 _ll_register_callbacks cb 0x56518da56c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.858+0000 7f9349c61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafb420 0x56518db05490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f9349c61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafb420 0x56518db05490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43814/0 (socket says 172.21.15.105:43814) 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f9349c61640 1 -- 172.21.15.105:0/494629863 learned_addr learned my addr 172.21.15.105:0/494629863 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f934a462640 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafaa30 0x56518db04f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f934ac63640 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 0x56518db0c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f9349c61640 1 -- 172.21.15.105:0/494629863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 msgr2=0x56518db0c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f9349c61640 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 0x56518db0c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f9349c61640 1 -- 172.21.15.105:0/494629863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafaa30 msgr2=0x56518db04f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f9349c61640 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafaa30 0x56518db04f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f9349c61640 1 -- 172.21.15.105:0/494629863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56518d83d7d0 con 0x56518dafb420 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f934a462640 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafaa30 0x56518db04f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:23.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f9349c61640 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafb420 0x56518db05490 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9334000c00 tx=0x7f93340095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab002d648580650 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f932dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f932dffb640 1 -- 172.21.15.105:0/494629863 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93340bd650 con 0x56518dafb420 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f932dffb640 1 -- 172.21.15.105:0/494629863 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f93340054b0 con 0x56518dafb420 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f934f394740 1 -- 172.21.15.105:0/494629863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93400041c0 con 0x56518dafb420 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f932dffb640 1 -- 172.21.15.105:0/494629863 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f93340be3f0 con 0x56518dafb420 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f934f394740 10 client.17256 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.859+0000 7f934f394740 1 -- 172.21.15.105:0/494629863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56518db0f840 con 0x56518dafb420 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f932dffb640 1 -- 172.21.15.105:0/494629863 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93400041c0 con 0x56518dafb420 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f932dffb640 1 -- 172.21.15.105:0/494629863 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f93340cdc90 con 0x56518dafb420 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f934f394740 10 client.17256 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f934f394740 10 client.17256 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f934f394740 20 client.17256 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f934f394740 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56518db10f60 0x56518db133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f934f394740 4 client.17256 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:23.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f934f394740 1 -- 172.21.15.105:0/494629863 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56518da6ceb0 con 0x56518db10f60 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f934a462640 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56518db10f60 0x56518db133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f934a462640 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56518db10f60 0x56518db133b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3aa017c0dea3b09 server_cookie=96ae9966e6fd451 in_seq=0 out_seq=0 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.860+0000 7f932dffb640 10 client.17256 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f932dffb640 1 -- 172.21.15.105:0/494629863 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56518da6ceb0 con 0x56518db10f60 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f932dffb640 10 client.17256 handle_command_reply: tid=0 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 client.17256 shutdown 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 -- 172.21.15.105:0/494629863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafb420 msgr2=0x56518db05490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafb420 0x56518db05490 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9334000c00 tx=0x7f93340095e0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 -- 172.21.15.105:0/494629863 shutdown_connections 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56518db10f60 0x56518db133b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 0x56518db0c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafb420 0x56518db05490 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f932dffb640 0 client.17256 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 --2- 172.21.15.105:0/494629863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafaa30 0x56518db04f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 -- 172.21.15.105:0/494629863 >> 172.21.15.105:0/494629863 conn(0x56518daf5fa0 msgr2=0x56518daf51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 -- 172.21.15.105:0/494629863 shutdown_connections 2024-03-26T17:43:23.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 1 -- 172.21.15.105:0/494629863 wait complete. 2024-03-26T17:43:23.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.861+0000 7f934f394740 20 client.17256 trim_cache size 0 max 16384 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.877+0000 7f934f394740 1 Processor -- start 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.877+0000 7f934f394740 1 -- start start 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.877+0000 7f934f394740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518daf9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.877+0000 7f934f394740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafb420 0x56518db0c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934f394740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518db00590 0x56518db17cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934f394740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f93441a6b00 con 0x56518dafaa30 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934f394740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93441a8640 con 0x56518db00590 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934f394740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93441a71d0 con 0x56518dafb420 2024-03-26T17:43:23.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518daf9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f9349c61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafb420 0x56518db0c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f9349c61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafb420 0x56518db0c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60578/0 (socket says 172.21.15.105:60578) 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518daf9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43820/0 (socket says 172.21.15.105:43820) 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 -- 172.21.15.105:0/256200178 learned_addr learned my addr 172.21.15.105:0/256200178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934ac63640 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518db00590 0x56518db17cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 -- 172.21.15.105:0/256200178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafb420 msgr2=0x56518db0c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafb420 0x56518db0c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 -- 172.21.15.105:0/256200178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518db00590 msgr2=0x56518db17cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518db00590 0x56518db17cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 -- 172.21.15.105:0/256200178 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56518daf5dd0 con 0x56518dafaa30 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f9349c61640 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafb420 0x56518db0c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f934a462640 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518daf9910 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9340009730 tx=0x7f9340009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f6b180ae25cdc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f932ffff640 1 -- 172.21.15.105:0/256200178 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340030070 con 0x56518dafaa30 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f932ffff640 1 -- 172.21.15.105:0/256200178 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f934002b030 con 0x56518dafaa30 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.878+0000 7f932ffff640 1 -- 172.21.15.105:0/256200178 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934001d040 con 0x56518dafaa30 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 -- 172.21.15.105:0/256200178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 msgr2=0x56518daf9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518daf9910 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9340009730 tx=0x7f9340009310 comp rx=0 tx=0).stop 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 -- 172.21.15.105:0/256200178 shutdown_connections 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518db00590 0x56518db17cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518dafb420 0x56518db0c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 --2- 172.21.15.105:0/256200178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518daf9910 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 -- 172.21.15.105:0/256200178 >> 172.21.15.105:0/256200178 conn(0x56518daf5fa0 msgr2=0x56518daf9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 -- 172.21.15.105:0/256200178 shutdown_connections 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.879+0000 7f934f394740 1 -- 172.21.15.105:0/256200178 wait complete. 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 1 -- start start 2024-03-26T17:43:23.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 1 Processor -- start 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518db09fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518db0a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 0x56518db0b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f93441a7f70 con 0x56518dafaa30 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93441a78a0 con 0x56518dafb420 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93441a4ae0 con 0x56518db00590 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934f394740 10 client.0 _ll_register_callbacks cb 0x7f93340035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934a462640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518db09fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934a462640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518db09fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43836/0 (socket says 172.21.15.105:43836) 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934a462640 1 -- 172.21.15.105:0/475121478 learned_addr learned my addr 172.21.15.105:0/475121478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f9349c61640 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518db0a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.880+0000 7f934ac63640 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 0x56518db0b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934a462640 1 -- 172.21.15.105:0/475121478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 msgr2=0x56518db0b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934a462640 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 0x56518db0b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934a462640 1 -- 172.21.15.105:0/475121478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 msgr2=0x56518db0a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:23.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934a462640 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518db0a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934a462640 1 -- 172.21.15.105:0/475121478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56518db0bae0 con 0x56518dafaa30 2024-03-26T17:43:23.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934a462640 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518db09fa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f934002e040 tx=0x7f9340011450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588e667af79bf1fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f932dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f932dffb640 1 -- 172.21.15.105:0/475121478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934001d070 con 0x56518dafaa30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f932dffb640 1 -- 172.21.15.105:0/475121478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f93400111d0 con 0x56518dafaa30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934f394740 1 -- 172.21.15.105:0/475121478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934002b030 con 0x56518dafaa30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f932dffb640 1 -- 172.21.15.105:0/475121478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9340030040 con 0x56518dafaa30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934f394740 10 client.17262 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f934f394740 1 -- 172.21.15.105:0/475121478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56518db77610 con 0x56518dafaa30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f932dffb640 1 -- 172.21.15.105:0/475121478 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f934002b030 con 0x56518dafaa30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.881+0000 7f932dffb640 1 -- 172.21.15.105:0/475121478 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f934003c030 con 0x56518dafaa30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 10 client.17262 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:23.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 10 client.17262 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:43:23.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 20 client.17262 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:23.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56518db77de0 0x56518db7a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:23.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 4 client.17262 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:23.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 -- 172.21.15.105:0/475121478 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56518da67200 con 0x56518db77de0 2024-03-26T17:43:23.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f9349c61640 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56518db77de0 0x56518db7a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:23.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f932dffb640 10 client.17262 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:43:23.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f9349c61640 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56518db77de0 0x56518db7a200 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f06f025a4e4ec66c server_cookie=32a027c096a3b65f in_seq=0 out_seq=0 2024-03-26T17:43:23.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f932dffb640 1 -- 172.21.15.105:0/475121478 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56518da67200 con 0x56518db77de0 2024-03-26T17:43:23.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f932dffb640 10 client.17262 handle_command_reply: tid=0 2024-03-26T17:43:23.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 client.17262 shutdown 2024-03-26T17:43:23.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 -- 172.21.15.105:0/475121478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 msgr2=0x56518db09fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518db09fa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f934002e040 tx=0x7f9340011450 comp rx=0 tx=0).stop 2024-03-26T17:43:23.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 -- 172.21.15.105:0/475121478 shutdown_connections 2024-03-26T17:43:23.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56518db77de0 0x56518db7a200 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56518db00590 0x56518db0b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f932dffb640 0 client.17262 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:43:23.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56518dafb420 0x56518db0a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.882+0000 7f934f394740 1 --2- 172.21.15.105:0/475121478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56518dafaa30 0x56518db09fa0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934f394740 1 -- 172.21.15.105:0/475121478 >> 172.21.15.105:0/475121478 conn(0x56518daf5fa0 msgr2=0x56518daf9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934f394740 1 -- 172.21.15.105:0/475121478 shutdown_connections 2024-03-26T17:43:23.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934f394740 1 -- 172.21.15.105:0/475121478 wait complete. 2024-03-26T17:43:23.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934f394740 20 client.17262 trim_cache size 0 max 16384 2024-03-26T17:43:23.891 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:23.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 -- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93300a2f20 msgr2=0x7f93300a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93300a2f20 0x7f93300a53e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x56518dafbc80 tx=0x7f933c0106e0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 -- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 msgr2=0x7f9344198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 0x7f9344198090 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9334006010 tx=0x7f9334005e40 comp rx=0 tx=0).stop 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 -- 172.21.15.105:0/2509989037 shutdown_connections 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93300a2f20 0x7f93300a53e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9344102370 0x7f934419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93441019b0 0x7f9344198090 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 --2- 172.21.15.105:0/2509989037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93441007c0 0x7f9344197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 -- 172.21.15.105:0/2509989037 >> 172.21.15.105:0/2509989037 conn(0x7f93440fbd50 msgr2=0x7f9344068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 -- 172.21.15.105:0/2509989037 shutdown_connections 2024-03-26T17:43:23.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:23.883+0000 7f934c6c6640 1 -- 172.21.15.105:0/2509989037 wait complete. 2024-03-26T17:43:23.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:23 smithi105 ceph-mon[32280]: pgmap v499: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:43:23.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:23 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1729664281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:43:23.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:43:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 -- 172.21.15.105:0/2321163419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c1019b0 msgr2=0x7f425c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 --2- 172.21.15.105:0/2321163419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c1019b0 0x7f425c101e30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4244009900 tx=0x7f424402bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 -- 172.21.15.105:0/2321163419 shutdown_connections 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 --2- 172.21.15.105:0/2321163419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f425c102370 0x7f425c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 --2- 172.21.15.105:0/2321163419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c1019b0 0x7f425c101e30 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 --2- 172.21.15.105:0/2321163419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c1007c0 0x7f425c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 -- 172.21.15.105:0/2321163419 >> 172.21.15.105:0/2321163419 conn(0x7f425c0fbd50 msgr2=0x7f425c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 -- 172.21.15.105:0/2321163419 shutdown_connections 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.019+0000 7f4260ec0640 1 -- 172.21.15.105:0/2321163419 wait complete. 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4260ec0640 1 Processor -- start 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4260ec0640 1 -- start start 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4260ec0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f425c1007c0 0x7f425c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4260ec0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c1019b0 0x7f425c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4260ec0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c102370 0x7f425c1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4260ec0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f425c10c2d0 con 0x7f425c102370 2024-03-26T17:43:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4260ec0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f425c10c150 con 0x7f425c1007c0 2024-03-26T17:43:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4260ec0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f425c10c450 con 0x7f425c1019b0 2024-03-26T17:43:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f425ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c102370 0x7f425c1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4251d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c1019b0 0x7f425c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f425ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c102370 0x7f425c1a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43864/0 (socket says 172.21.15.105:43864) 2024-03-26T17:43:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f425a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f425c1007c0 0x7f425c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f425ad9d640 1 -- 172.21.15.105:0/3370710245 learned_addr learned my addr 172.21.15.105:0/3370710245 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4251d9b640 1 -- 172.21.15.105:0/3370710245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f425c1007c0 msgr2=0x7f425c199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4251d9b640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f425c1007c0 0x7f425c199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4251d9b640 1 -- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c102370 msgr2=0x7f425c1a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4251d9b640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c102370 0x7f425c1a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.020+0000 7f4251d9b640 1 -- 172.21.15.105:0/3370710245 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c1a1960 con 0x7f425c1019b0 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.021+0000 7f4251d9b640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c1019b0 0x7f425c19a280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4244000c00 tx=0x7f4244005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.021+0000 7f4253fff640 1 -- 172.21.15.105:0/3370710245 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244043020 con 0x7f425c1019b0 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.021+0000 7f4260ec0640 1 -- 172.21.15.105:0/3370710245 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425c1a1bf0 con 0x7f425c1019b0 2024-03-26T17:43:24.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.021+0000 7f4253fff640 1 -- 172.21.15.105:0/3370710245 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4244004350 con 0x7f425c1019b0 2024-03-26T17:43:24.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.021+0000 7f4253fff640 1 -- 172.21.15.105:0/3370710245 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244003b50 con 0x7f425c1019b0 2024-03-26T17:43:24.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.021+0000 7f4260ec0640 1 -- 172.21.15.105:0/3370710245 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425c1a2190 con 0x7f425c1019b0 2024-03-26T17:43:24.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4263b85740 1 Processor -- start 2024-03-26T17:43:24.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4263b85740 1 -- start start 2024-03-26T17:43:24.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4263b85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc56ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4263b85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc573f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4263b85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 0x560efc576a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4263b85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560efc4d3970 con 0x560efc56f420 2024-03-26T17:43:24.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4263b85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560efc47bd70 con 0x560efc56ea30 2024-03-26T17:43:24.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4263b85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560efc4cf920 con 0x560efc574590 2024-03-26T17:43:24.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4253fff640 1 -- 172.21.15.105:0/3370710245 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4244046020 con 0x7f425c1019b0 2024-03-26T17:43:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc573f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc56ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc573f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43868/0 (socket says 172.21.15.105:43868) 2024-03-26T17:43:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425a59c640 1 -- 172.21.15.105:0/65192133 learned_addr learned my addr 172.21.15.105:0/65192133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f4251d9b640 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 0x560efc576a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425ad9d640 1 -- 172.21.15.105:0/65192133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 msgr2=0x560efc576a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425ad9d640 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 0x560efc576a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425ad9d640 1 -- 172.21.15.105:0/65192133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 msgr2=0x560efc573f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425ad9d640 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc573f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425ad9d640 1 -- 172.21.15.105:0/65192133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560efc1e73f0 con 0x560efc56ea30 2024-03-26T17:43:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.024+0000 7f425a59c640 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc573f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.025+0000 7f4253fff640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42480a2f20 0x7f42480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.025+0000 7f425ad9d640 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc56ee50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4254009fb0 tx=0x7f42540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc8be059545c901 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.025+0000 7f425a59c640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42480a2f20 0x7f42480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.025+0000 7f4253fff640 1 -- 172.21.15.105:0/3370710245 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f42440f14e0 con 0x7f425c1019b0 2024-03-26T17:43:24.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.025+0000 7f4250d99640 1 -- 172.21.15.105:0/65192133 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254004b20 con 0x560efc56ea30 2024-03-26T17:43:24.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.025+0000 7f4250d99640 1 -- 172.21.15.105:0/65192133 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254004cc0 con 0x560efc56ea30 2024-03-26T17:43:24.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.025+0000 7f4250d99640 1 -- 172.21.15.105:0/65192133 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254005310 con 0x560efc56ea30 2024-03-26T17:43:24.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f425a59c640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42480a2f20 0x7f42480a53e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x560efc56f820 tx=0x7f424c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 -- 172.21.15.105:0/65192133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 msgr2=0x560efc56ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc56ee50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4254009fb0 tx=0x7f42540118d0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 -- 172.21.15.105:0/65192133 shutdown_connections 2024-03-26T17:43:24.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 0x560efc576a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc573f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 --2- 172.21.15.105:0/65192133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc56ee50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 -- 172.21.15.105:0/65192133 >> 172.21.15.105:0/65192133 conn(0x560efc569fa0 msgr2=0x560efc56a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 -- 172.21.15.105:0/65192133 shutdown_connections 2024-03-26T17:43:24.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 -- 172.21.15.105:0/65192133 wait complete. 2024-03-26T17:43:24.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 -- start start 2024-03-26T17:43:24.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.026+0000 7f4263b85740 1 Processor -- start 2024-03-26T17:43:24.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4263b85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56ea30 0x560efc578eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4263b85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56f420 0x560efc5793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4263b85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 0x560efc580430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4263b85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560efc577590 con 0x560efc56ea30 2024-03-26T17:43:24.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4263b85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560efc48f310 con 0x560efc574590 2024-03-26T17:43:24.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4263b85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560efc577710 con 0x560efc56f420 2024-03-26T17:43:24.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f425a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56ea30 0x560efc578eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f425a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56ea30 0x560efc578eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43876/0 (socket says 172.21.15.105:43876) 2024-03-26T17:43:24.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4251d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56f420 0x560efc5793f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f425a59c640 1 -- 172.21.15.105:0/2467106129 learned_addr learned my addr 172.21.15.105:0/2467106129 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4263b85740 10 client.0 _ll_register_callbacks cb 0x560efc4cac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:24.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f425ad9d640 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 0x560efc580430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4251d9b640 1 -- 172.21.15.105:0/2467106129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 msgr2=0x560efc580430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4251d9b640 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 0x560efc580430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4251d9b640 1 -- 172.21.15.105:0/2467106129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56ea30 msgr2=0x560efc578eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4251d9b640 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56ea30 0x560efc578eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4251d9b640 1 -- 172.21.15.105:0/2467106129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560efc2b17d0 con 0x560efc56f420 2024-03-26T17:43:24.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f425a59c640 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56ea30 0x560efc578eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4251d9b640 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56f420 0x560efc5793f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42440eeff0 tx=0x7f4244005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=710b541f79542bce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.027+0000 7f4241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:24.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4241ffb640 1 -- 172.21.15.105:0/2467106129 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42440bd9a0 con 0x560efc56f420 2024-03-26T17:43:24.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4241ffb640 1 -- 172.21.15.105:0/2467106129 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f42440f10d0 con 0x560efc56f420 2024-03-26T17:43:24.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 1 -- 172.21.15.105:0/2467106129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4254004cc0 con 0x560efc56f420 2024-03-26T17:43:24.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4241ffb640 1 -- 172.21.15.105:0/2467106129 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42440bd530 con 0x560efc56f420 2024-03-26T17:43:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 10 client.27112 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 1 -- 172.21.15.105:0/2467106129 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560efc577650 con 0x560efc56f420 2024-03-26T17:43:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4241ffb640 1 -- 172.21.15.105:0/2467106129 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4254004cc0 con 0x560efc56f420 2024-03-26T17:43:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4241ffb640 1 -- 172.21.15.105:0/2467106129 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f42440c6150 con 0x560efc56f420 2024-03-26T17:43:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 10 client.27112 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 10 client.27112 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:43:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 20 client.27112 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560efc584e60 0x560efc587290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 4 client.27112 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f4263b85740 1 -- 172.21.15.105:0/2467106129 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560efc4e0eb0 con 0x560efc584e60 2024-03-26T17:43:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.028+0000 7f425a59c640 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560efc584e60 0x560efc587290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.029+0000 7f4241ffb640 10 client.27112 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:43:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.029+0000 7f425a59c640 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560efc584e60 0x560efc587290 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=691f90e00a4434d3 server_cookie=c229cc3d039b9849 in_seq=0 out_seq=0 2024-03-26T17:43:24.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.029+0000 7f4241ffb640 1 -- 172.21.15.105:0/2467106129 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560efc4e0eb0 con 0x560efc584e60 2024-03-26T17:43:24.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.029+0000 7f4241ffb640 10 client.27112 handle_command_reply: tid=0 2024-03-26T17:43:24.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.029+0000 7f4263b85740 1 client.27112 shutdown 2024-03-26T17:43:24.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 -- 172.21.15.105:0/2467106129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56f420 msgr2=0x560efc5793f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56f420 0x560efc5793f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f42440eeff0 tx=0x7f4244005c90 comp rx=0 tx=0).stop 2024-03-26T17:43:24.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 -- 172.21.15.105:0/2467106129 shutdown_connections 2024-03-26T17:43:24.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560efc584e60 0x560efc587290 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 0x560efc580430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4241ffb640 0 client.27112 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:43:24.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56f420 0x560efc5793f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 --2- 172.21.15.105:0/2467106129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56ea30 0x560efc578eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 -- 172.21.15.105:0/2467106129 >> 172.21.15.105:0/2467106129 conn(0x560efc569fa0 msgr2=0x560efc5691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 -- 172.21.15.105:0/2467106129 shutdown_connections 2024-03-26T17:43:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 1 -- 172.21.15.105:0/2467106129 wait complete. 2024-03-26T17:43:24.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.030+0000 7f4263b85740 20 client.27112 trim_cache size 0 max 16384 2024-03-26T17:43:24.042 INFO:tasks.workunit.client.0.smithi105.stdout: 1 145846 3.16 MB/sec execute 127 sec latency 14.982 ms 2024-03-26T17:43:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.044+0000 7f4263b85740 1 Processor -- start 2024-03-26T17:43:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.044+0000 7f4263b85740 1 -- start start 2024-03-26T17:43:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4263b85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56ea30 0x560efc56fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4263b85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc5803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4263b85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 0x560efc56a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4263b85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1aaf40 con 0x560efc56f420 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4263b85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1aca80 con 0x560efc574590 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4263b85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1ab610 con 0x560efc56ea30 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc5803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f425a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56ea30 0x560efc56fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f425ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 0x560efc56a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc5803f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43882/0 (socket says 172.21.15.105:43882) 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 -- 172.21.15.105:0/4206392789 learned_addr learned my addr 172.21.15.105:0/4206392789 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 -- 172.21.15.105:0/4206392789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56ea30 msgr2=0x560efc56fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 --2- 172.21.15.105:0/4206392789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56ea30 0x560efc56fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 -- 172.21.15.105:0/4206392789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 msgr2=0x560efc56a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 --2- 172.21.15.105:0/4206392789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 0x560efc56a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 -- 172.21.15.105:0/4206392789 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560efc5878c0 con 0x560efc56f420 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4251d9b640 1 --2- 172.21.15.105:0/4206392789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc5803f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4244005ab0 tx=0x7f42440c1d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800b36b3403db7a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4243fff640 1 -- 172.21.15.105:0/4206392789 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42440bd710 con 0x560efc56f420 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.045+0000 7f4243fff640 1 -- 172.21.15.105:0/4206392789 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42440bdbd0 con 0x560efc56f420 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4243fff640 1 -- 172.21.15.105:0/4206392789 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42440b9920 con 0x560efc56f420 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4263b85740 1 -- 172.21.15.105:0/4206392789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 msgr2=0x560efc5803f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4263b85740 1 --2- 172.21.15.105:0/4206392789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc5803f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4244005ab0 tx=0x7f42440c1d20 comp rx=0 tx=0).stop 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4263b85740 1 -- 172.21.15.105:0/4206392789 shutdown_connections 2024-03-26T17:43:24.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4263b85740 1 --2- 172.21.15.105:0/4206392789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc574590 0x560efc56a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4263b85740 1 --2- 172.21.15.105:0/4206392789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc5803f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4263b85740 1 --2- 172.21.15.105:0/4206392789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc56ea30 0x560efc56fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4263b85740 1 -- 172.21.15.105:0/4206392789 >> 172.21.15.105:0/4206392789 conn(0x560efc569fa0 msgr2=0x560efc565290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.046+0000 7f4263b85740 1 -- 172.21.15.105:0/4206392789 shutdown_connections 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 -- 172.21.15.105:0/4206392789 wait complete. 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 -- start start 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 Processor -- start 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc57d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc57ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 0x560efc57ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1ac3b0 con 0x560efc56f420 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1abce0 con 0x560efc56ea30 2024-03-26T17:43:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1a8f20 con 0x560efc574590 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4263b85740 10 client.0 _ll_register_callbacks cb 0x7f42440035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4251d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc57ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f425a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc57d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f425a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc57d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50264/0 (socket says 172.21.15.105:50264) 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f4251d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc57ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43890/0 (socket says 172.21.15.105:43890) 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f425a59c640 1 -- 172.21.15.105:0/3595942195 learned_addr learned my addr 172.21.15.105:0/3595942195 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.047+0000 7f425ad9d640 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 0x560efc57ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4251d9b640 1 -- 172.21.15.105:0/3595942195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 msgr2=0x560efc57ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4251d9b640 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 0x560efc57ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4251d9b640 1 -- 172.21.15.105:0/3595942195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 msgr2=0x560efc57d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4251d9b640 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc57d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4251d9b640 1 -- 172.21.15.105:0/3595942195 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560efc57f3d0 con 0x560efc56f420 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f425a59c640 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc57d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4251d9b640 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc57ddd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4244005ab0 tx=0x7f42440ef030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2faa76ec82221a0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4241ffb640 1 -- 172.21.15.105:0/3595942195 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42440da070 con 0x560efc56f420 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4241ffb640 1 -- 172.21.15.105:0/3595942195 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f42440b9920 con 0x560efc56f420 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4263b85740 1 -- 172.21.15.105:0/3595942195 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560efc4db200 con 0x560efc56f420 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4241ffb640 1 -- 172.21.15.105:0/3595942195 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42440a3050 con 0x560efc56f420 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4263b85740 10 client.17280 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4263b85740 1 -- 172.21.15.105:0/3595942195 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560efc57f3d0 con 0x560efc56f420 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4241ffb640 1 -- 172.21.15.105:0/3595942195 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560efc4db200 con 0x560efc56f420 2024-03-26T17:43:24.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4241ffb640 1 -- 172.21.15.105:0/3595942195 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f42440c2eb0 con 0x560efc56f420 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4263b85740 10 client.17280 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.048+0000 7f4263b85740 10 client.17280 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 20 client.17280 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560efc5eb510 0x560efc5ed900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 4 client.17280 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 -- 172.21.15.105:0/3595942195 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560efc4586a0 con 0x560efc5eb510 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f425a59c640 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560efc5eb510 0x560efc5ed900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f425a59c640 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560efc5eb510 0x560efc5ed900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b444eed862289a2f server_cookie=2508664e00c8a93 in_seq=0 out_seq=0 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4241ffb640 10 client.17280 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4241ffb640 1 -- 172.21.15.105:0/3595942195 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560efc4586a0 con 0x560efc5eb510 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4241ffb640 10 client.17280 handle_command_reply: tid=0 2024-03-26T17:43:24.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 client.17280 shutdown 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 -- 172.21.15.105:0/3595942195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 msgr2=0x560efc57ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc57ddd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4244005ab0 tx=0x7f42440ef030 comp rx=0 tx=0).stop 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 -- 172.21.15.105:0/3595942195 shutdown_connections 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560efc5eb510 0x560efc5ed900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4241ffb640 0 client.17280 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560efc574590 0x560efc57ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560efc56f420 0x560efc57ddd0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 --2- 172.21.15.105:0/3595942195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560efc56ea30 0x560efc57d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.049+0000 7f4263b85740 1 -- 172.21.15.105:0/3595942195 >> 172.21.15.105:0/3595942195 conn(0x560efc569fa0 msgr2=0x560efc573a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4263b85740 1 -- 172.21.15.105:0/3595942195 shutdown_connections 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4263b85740 1 -- 172.21.15.105:0/3595942195 wait complete. 2024-03-26T17:43:24.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4263b85740 20 client.17280 trim_cache size 0 max 16384 2024-03-26T17:43:24.058 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 -- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42480a2f20 msgr2=0x7f42480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42480a2f20 0x7f42480a53e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x560efc56f820 tx=0x7f424c005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 -- 172.21.15.105:0/3370710245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c1019b0 msgr2=0x7f425c19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c1019b0 0x7f425c19a280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4244000c00 tx=0x7f4244005dc0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 -- 172.21.15.105:0/3370710245 shutdown_connections 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42480a2f20 0x7f42480a53e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f425c102370 0x7f425c1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f425c1019b0 0x7f425c19a280 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 --2- 172.21.15.105:0/3370710245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f425c1007c0 0x7f425c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 -- 172.21.15.105:0/3370710245 >> 172.21.15.105:0/3370710245 conn(0x7f425c0fbd50 msgr2=0x7f425c068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 -- 172.21.15.105:0/3370710245 shutdown_connections 2024-03-26T17:43:24.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.050+0000 7f4260ec0640 1 -- 172.21.15.105:0/3370710245 wait complete. 2024-03-26T17:43:24.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:43:24.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:23 smithi144 ceph-mon[36095]: pgmap v499: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:43:24.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:23 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1729664281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:43:24.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:23 smithi135 ceph-mon[35636]: pgmap v499: 97 pgs: 97 active+clean; 25 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:43:24.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:23 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1729664281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:43:24.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/4166157216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 msgr2=0x7f5b58109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/4166157216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 0x7f5b58109a50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009ca0 tx=0x7f5b5002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/4166157216 shutdown_connections 2024-03-26T17:43:24.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/4166157216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 0x7f5b58109a50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/4166157216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b58104d30 0x7f5b58107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/4166157216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58102400 0x7f5b581047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/4166157216 >> 172.21.15.105:0/4166157216 conn(0x7f5b580fbd50 msgr2=0x7f5b580fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/4166157216 shutdown_connections 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.190+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/4166157216 wait complete. 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5fe0b640 1 Processor -- start 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5fe0b640 1 -- start start 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5fe0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58102400 0x7f5b58197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5fe0b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b58104d30 0x7f5b58198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5fe0b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 0x7f5b5819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5fe0b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5810d420 con 0x7f5b58107660 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5fe0b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5810d2a0 con 0x7f5b58104d30 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5fe0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5810d5a0 con 0x7f5b58102400 2024-03-26T17:43:24.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5e3a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 0x7f5b5819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5d3a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b58104d30 0x7f5b58198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58102400 0x7f5b58197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5e3a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 0x7f5b5819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43908/0 (socket says 172.21.15.105:43908) 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.191+0000 7f5b5e3a8640 1 -- 172.21.15.105:0/3051912459 learned_addr learned my addr 172.21.15.105:0/3051912459 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5e3a8640 1 -- 172.21.15.105:0/3051912459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58102400 msgr2=0x7f5b58197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5e3a8640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58102400 0x7f5b58197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5e3a8640 1 -- 172.21.15.105:0/3051912459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b58104d30 msgr2=0x7f5b58198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5e3a8640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b58104d30 0x7f5b58198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5e3a8640 1 -- 172.21.15.105:0/3051912459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5819f750 con 0x7f5b58107660 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b58104d30 0x7f5b58198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58102400 0x7f5b58197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5e3a8640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 0x7f5b5819f0e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5b50032380 tx=0x7f5b50032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b4effd640 1 -- 172.21.15.105:0/3051912459 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50049070 con 0x7f5b58107660 2024-03-26T17:43:24.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/3051912459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b581001f0 con 0x7f5b58107660 2024-03-26T17:43:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b4effd640 1 -- 172.21.15.105:0/3051912459 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b50003e20 con 0x7f5b58107660 2024-03-26T17:43:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b4effd640 1 -- 172.21.15.105:0/3051912459 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50004680 con 0x7f5b58107660 2024-03-26T17:43:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.192+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/3051912459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581006e0 con 0x7f5b58107660 2024-03-26T17:43:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b4effd640 1 -- 172.21.15.105:0/3051912459 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5b5004c070 con 0x7f5b58107660 2024-03-26T17:43:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b629fb740 1 Processor -- start 2024-03-26T17:43:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b629fb740 1 -- start start 2024-03-26T17:43:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431dde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 0x55f9431e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b629fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f943142970 con 0x55f9431de420 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b629fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f9430ead70 con 0x55f9431dda30 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b629fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f94313e920 con 0x55f9431e3590 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b5d3a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431dde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b5d3a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431dde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50288/0 (socket says 172.21.15.105:50288) 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43922/0 (socket says 172.21.15.105:43922) 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b5d3a6640 1 -- 172.21.15.105:0/2162872824 learned_addr learned my addr 172.21.15.105:0/2162872824 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.195+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/2162872824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 0x55f9431e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b4effd640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5b440a2f20 0x7f5b440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b5d3a6640 1 -- 172.21.15.105:0/2162872824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 msgr2=0x55f9431e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/2162872824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 0x55f9431e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b5d3a6640 1 -- 172.21.15.105:0/2162872824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 msgr2=0x55f9431e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/2162872824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b5d3a6640 1 -- 172.21.15.105:0/2162872824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f942e563f0 con 0x55f9431dda30 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b5e3a8640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5b440a2f20 0x7f5b440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b4effd640 1 -- 172.21.15.105:0/3051912459 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5b500f5090 con 0x7f5b58107660 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/2162872824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431dde50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b48011bb0 tx=0x7f5b48004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a429b8f4cdb749c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b43fff640 1 -- 172.21.15.105:0/2162872824 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800fc50 con 0x55f9431dda30 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b43fff640 1 -- 172.21.15.105:0/2162872824 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b4800fdf0 con 0x55f9431dda30 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.196+0000 7f5b43fff640 1 -- 172.21.15.105:0/2162872824 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b480039b0 con 0x55f9431dda30 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 -- 172.21.15.105:0/2162872824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 msgr2=0x55f9431dde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 --2- 172.21.15.105:0/2162872824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431dde50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5b48011bb0 tx=0x7f5b48004730 comp rx=0 tx=0).stop 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b5e3a8640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5b440a2f20 0x7f5b440a53e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5b500c9c00 tx=0x7f5b500b6cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 -- 172.21.15.105:0/2162872824 shutdown_connections 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 --2- 172.21.15.105:0/2162872824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 0x55f9431e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 --2- 172.21.15.105:0/2162872824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 --2- 172.21.15.105:0/2162872824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431dde50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 -- 172.21.15.105:0/2162872824 >> 172.21.15.105:0/2162872824 conn(0x55f9431d8fa0 msgr2=0x55f9431d93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 -- 172.21.15.105:0/2162872824 shutdown_connections 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 -- 172.21.15.105:0/2162872824 wait complete. 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.197+0000 7f5b629fb740 1 -- start start 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b629fb740 1 Processor -- start 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 0x55f9431ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b629fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f9431e6540 con 0x55f9431de420 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b629fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f9430fe310 con 0x55f9431e3590 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b629fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9431e66c0 con 0x55f9431dda30 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b629fb740 10 client.0 _ll_register_callbacks cb 0x55f943139c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5d3a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43934/0 (socket says 172.21.15.105:43934) 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5d3a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60678/0 (socket says 172.21.15.105:60678) 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5dba7640 1 -- 172.21.15.105:0/3150201498 learned_addr learned my addr 172.21.15.105:0/3150201498 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 0x55f9431ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5dba7640 1 -- 172.21.15.105:0/3150201498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 msgr2=0x55f9431e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5dba7640 1 -- 172.21.15.105:0/3150201498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 msgr2=0x55f9431ef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 0x55f9431ef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5dba7640 1 -- 172.21.15.105:0/3150201498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f942f207d0 con 0x55f9431de420 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.198+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e8480 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5b54012d20 tx=0x7f5b5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b580d53b645ddd11 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b417fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b417fa640 1 -- 172.21.15.105:0/3150201498 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54012110 con 0x55f9431de420 2024-03-26T17:43:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b417fa640 1 -- 172.21.15.105:0/3150201498 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5b54004050 con 0x55f9431de420 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b629fb740 1 -- 172.21.15.105:0/3150201498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4800fdf0 con 0x55f9431de420 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b417fa640 1 -- 172.21.15.105:0/3150201498 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b540122b0 con 0x55f9431de420 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b629fb740 10 client.17292 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b629fb740 1 -- 172.21.15.105:0/3150201498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9431f28e0 con 0x55f9431de420 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b417fa640 1 -- 172.21.15.105:0/3150201498 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b4800fdf0 con 0x55f9431de420 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.199+0000 7f5b417fa640 1 -- 172.21.15.105:0/3150201498 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5b54010430 con 0x55f9431de420 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b629fb740 10 client.17292 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b629fb740 10 client.17292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b629fb740 20 client.17292 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b629fb740 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55f9431f4050 0x55f9431f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b629fb740 4 client.17292 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b629fb740 1 -- 172.21.15.105:0/3150201498 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f94314feb0 con 0x55f9431f4050 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55f9431f4050 0x55f9431f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b417fa640 10 client.17292 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:43:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.200+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55f9431f4050 0x55f9431f64b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b36953364209b67a server_cookie=686004674e03ae14 in_seq=0 out_seq=0 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b417fa640 1 -- 172.21.15.105:0/3150201498 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f94314feb0 con 0x55f9431f4050 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b417fa640 10 client.17292 handle_command_reply: tid=0 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 client.17292 shutdown 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 -- 172.21.15.105:0/3150201498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 msgr2=0x55f9431e8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e8480 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5b54012d20 tx=0x7f5b5400f790 comp rx=0 tx=0).stop 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 -- 172.21.15.105:0/3150201498 shutdown_connections 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55f9431f4050 0x55f9431f64b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 0x55f9431ef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431e8480 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b417fa640 0 client.17292 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 --2- 172.21.15.105:0/3150201498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 -- 172.21.15.105:0/3150201498 >> 172.21.15.105:0/3150201498 conn(0x55f9431d8fa0 msgr2=0x55f9431d81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 -- 172.21.15.105:0/3150201498 shutdown_connections 2024-03-26T17:43:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.201+0000 7f5b629fb740 1 -- 172.21.15.105:0/3150201498 wait complete. 2024-03-26T17:43:24.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.202+0000 7f5b629fb740 20 client.17292 trim_cache size 0 max 16384 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b629fb740 1 Processor -- start 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b629fb740 1 -- start start 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 0x55f9431d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b629fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a6b50 con 0x55f9431de420 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b629fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a8690 con 0x55f9431dda30 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b629fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a7220 con 0x55f9431e3590 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5d3a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43938/0 (socket says 172.21.15.105:43938) 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5d3a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431d7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50306/0 (socket says 172.21.15.105:50306) 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5dba7640 1 -- 172.21.15.105:0/1307281436 learned_addr learned my addr 172.21.15.105:0/1307281436 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/1307281436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 0x55f9431d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5dba7640 1 -- 172.21.15.105:0/1307281436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 msgr2=0x55f9431d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/1307281436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 0x55f9431d7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5dba7640 1 -- 172.21.15.105:0/1307281436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 msgr2=0x55f9431d7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/1307281436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431d7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.218+0000 7f5b5dba7640 1 -- 172.21.15.105:0/1307281436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9431d9d10 con 0x55f9431de420 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.219+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/1307281436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ef480 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5b54012cf0 tx=0x7f5b54012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b79938427a2d63b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.219+0000 7f5b437fe640 1 -- 172.21.15.105:0/1307281436 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400fe80 con 0x55f9431de420 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.219+0000 7f5b437fe640 1 -- 172.21.15.105:0/1307281436 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b540055c0 con 0x55f9431de420 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.219+0000 7f5b437fe640 1 -- 172.21.15.105:0/1307281436 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54003850 con 0x55f9431de420 2024-03-26T17:43:24.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 -- 172.21.15.105:0/1307281436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 msgr2=0x55f9431ef480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 --2- 172.21.15.105:0/1307281436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ef480 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5b54012cf0 tx=0x7f5b54012990 comp rx=0 tx=0).stop 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 -- 172.21.15.105:0/1307281436 shutdown_connections 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 --2- 172.21.15.105:0/1307281436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431e3590 0x55f9431d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 --2- 172.21.15.105:0/1307281436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ef480 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 --2- 172.21.15.105:0/1307281436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431dda30 0x55f9431d7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 -- 172.21.15.105:0/1307281436 >> 172.21.15.105:0/1307281436 conn(0x55f9431d8fa0 msgr2=0x55f9431e7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 -- 172.21.15.105:0/1307281436 shutdown_connections 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 -- 172.21.15.105:0/1307281436 wait complete. 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 -- start start 2024-03-26T17:43:24.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 Processor -- start 2024-03-26T17:43:24.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 0x55f9431ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a7fc0 con 0x55f9431de420 2024-03-26T17:43:24.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.220+0000 7f5b629fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a78f0 con 0x55f9431e3590 2024-03-26T17:43:24.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b629fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581a4b30 con 0x55f9431dda30 2024-03-26T17:43:24.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b629fb740 10 client.0 _ll_register_callbacks cb 0x7f5b54009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5dba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ed240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43954/0 (socket says 172.21.15.105:43954) 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5d3a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5dba7640 1 -- 172.21.15.105:0/246388921 learned_addr learned my addr 172.21.15.105:0/246388921 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 0x55f9431ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5d3a6640 1 -- 172.21.15.105:0/246388921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 msgr2=0x55f9431ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 0x55f9431ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5d3a6640 1 -- 172.21.15.105:0/246388921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 msgr2=0x55f9431ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5d3a6640 1 -- 172.21.15.105:0/246388921 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9431ee840 con 0x55f9431dda30 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b5d3a6640 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431ecd00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55f9431f6d20 tx=0x7f5b48009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65907fdfcd360587 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b417fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b417fa640 1 -- 172.21.15.105:0/246388921 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4802f460 con 0x55f9431dda30 2024-03-26T17:43:24.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b417fa640 1 -- 172.21.15.105:0/246388921 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5b48019040 con 0x55f9431dda30 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.221+0000 7f5b629fb740 1 -- 172.21.15.105:0/246388921 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b540055c0 con 0x55f9431dda30 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b417fa640 1 -- 172.21.15.105:0/246388921 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b48040050 con 0x55f9431dda30 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b629fb740 10 client.27136 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b629fb740 1 -- 172.21.15.105:0/246388921 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f94325a470 con 0x55f9431dda30 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b417fa640 1 -- 172.21.15.105:0/246388921 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b540055c0 con 0x55f9431dda30 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b417fa640 1 -- 172.21.15.105:0/246388921 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5b48010430 con 0x55f9431dda30 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b629fb740 10 client.27136 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b629fb740 10 client.27136 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b629fb740 20 client.27136 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b629fb740 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55f94325ab70 0x55f94325cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b629fb740 4 client.27136 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b629fb740 1 -- 172.21.15.105:0/246388921 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f94314a200 con 0x55f94325ab70 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55f94325ab70 0x55f94325cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.222+0000 7f5b417fa640 10 client.27136 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b5dba7640 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55f94325ab70 0x55f94325cf90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3dca4864df855ee0 server_cookie=42a2e8f3204cfa31 in_seq=0 out_seq=0 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b417fa640 1 -- 172.21.15.105:0/246388921 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f94314a200 con 0x55f94325ab70 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b417fa640 10 client.27136 handle_command_reply: tid=0 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 client.27136 shutdown 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 -- 172.21.15.105:0/246388921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 msgr2=0x55f9431ecd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431ecd00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55f9431f6d20 tx=0x7f5b48009f10 comp rx=0 tx=0).stop 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 -- 172.21.15.105:0/246388921 shutdown_connections 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55f94325ab70 0x55f94325cf90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f9431e3590 0x55f9431ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f9431de420 0x55f9431ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b417fa640 0 client.27136 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 --2- 172.21.15.105:0/246388921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9431dda30 0x55f9431ecd00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 -- 172.21.15.105:0/246388921 >> 172.21.15.105:0/246388921 conn(0x55f9431d8fa0 msgr2=0x55f9431dc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 -- 172.21.15.105:0/246388921 shutdown_connections 2024-03-26T17:43:24.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 1 -- 172.21.15.105:0/246388921 wait complete. 2024-03-26T17:43:24.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.223+0000 7f5b629fb740 20 client.27136 trim_cache size 0 max 16384 2024-03-26T17:43:24.222 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5b440a2f20 msgr2=0x7f5b440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5b440a2f20 0x7f5b440a53e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5b500c9c00 tx=0x7f5b500b6cb0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 msgr2=0x7f5b5819f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 0x7f5b5819f0e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5b50032380 tx=0x7f5b50032820 comp rx=0 tx=0).stop 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/3051912459 shutdown_connections 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5b440a2f20 0x7f5b440a53e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58107660 0x7f5b5819f0e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b58104d30 0x7f5b58198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 --2- 172.21.15.105:0/3051912459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58102400 0x7f5b58197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/3051912459 >> 172.21.15.105:0/3051912459 conn(0x7f5b580fbd50 msgr2=0x7f5b58102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/3051912459 shutdown_connections 2024-03-26T17:43:24.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.224+0000 7f5b5fe0b640 1 -- 172.21.15.105:0/3051912459 wait complete. 2024-03-26T17:43:24.227 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:43:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 -- 172.21.15.105:0/3812997954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 msgr2=0x7f5330100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 --2- 172.21.15.105:0/3812997954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 0x7f5330100be0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f532c009900 tx=0x7f532c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 -- 172.21.15.105:0/3812997954 shutdown_connections 2024-03-26T17:43:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 --2- 172.21.15.105:0/3812997954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5330102370 0x7f5330108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 --2- 172.21.15.105:0/3812997954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53301019b0 0x7f5330101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 --2- 172.21.15.105:0/3812997954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 0x7f5330100be0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 -- 172.21.15.105:0/3812997954 >> 172.21.15.105:0/3812997954 conn(0x7f53300fbd50 msgr2=0x7f53300fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 -- 172.21.15.105:0/3812997954 shutdown_connections 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.354+0000 7f5337177640 1 -- 172.21.15.105:0/3812997954 wait complete. 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5337177640 1 Processor -- start 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5337177640 1 -- start start 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5337177640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 0x7f53300695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5337177640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53301019b0 0x7f533019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5337177640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5330102370 0x7f533019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5337177640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f533010c2f0 con 0x7f53301007c0 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5337177640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f533010c170 con 0x7f5330102370 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5337177640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f533010c470 con 0x7f53301019b0 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5334f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 0x7f53300695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5334f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 0x7f53300695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43966/0 (socket says 172.21.15.105:43966) 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5334f13640 1 -- 172.21.15.105:0/2683310767 learned_addr learned my addr 172.21.15.105:0/2683310767 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5327fff640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53301019b0 0x7f533019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5335714640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5330102370 0x7f533019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5334f13640 1 -- 172.21.15.105:0/2683310767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53301019b0 msgr2=0x7f533019c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5334f13640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53301019b0 0x7f533019c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5334f13640 1 -- 172.21.15.105:0/2683310767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5330102370 msgr2=0x7f533019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5334f13640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5330102370 0x7f533019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5334f13640 1 -- 172.21.15.105:0/2683310767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533019f780 con 0x7f53301007c0 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.355+0000 7f5327fff640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53301019b0 0x7f533019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.356+0000 7f5334f13640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 0x7f53300695b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f532c000c00 tx=0x7f532c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.356+0000 7f5325ffb640 1 -- 172.21.15.105:0/2683310767 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c043020 con 0x7f53301007c0 2024-03-26T17:43:24.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.356+0000 7f5325ffb640 1 -- 172.21.15.105:0/2683310767 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f532c004350 con 0x7f53301007c0 2024-03-26T17:43:24.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.356+0000 7f5325ffb640 1 -- 172.21.15.105:0/2683310767 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c003b80 con 0x7f53301007c0 2024-03-26T17:43:24.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.356+0000 7f5337177640 1 -- 172.21.15.105:0/2683310767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533019fa10 con 0x7f53301007c0 2024-03-26T17:43:24.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.356+0000 7f5337177640 1 -- 172.21.15.105:0/2683310767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533019ff60 con 0x7f53301007c0 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f53399fd740 1 Processor -- start 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f53399fd740 1 -- start start 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f5325ffb640 1 -- 172.21.15.105:0/2683310767 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f532c046020 con 0x7f53301007c0 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f53399fd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b6fa30 0x55f292b6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f53399fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b70420 0x55f292b74f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f53399fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b75590 0x55f292b77a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f53399fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f292ad4970 con 0x55f292b70420 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f53399fd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f292a7cd70 con 0x55f292b6fa30 2024-03-26T17:43:24.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f53399fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f292ad0920 con 0x55f292b75590 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f5334f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b70420 0x55f292b74f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f5335714640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b6fa30 0x55f292b6fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f5334f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b70420 0x55f292b74f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43968/0 (socket says 172.21.15.105:43968) 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f5334f13640 1 -- 172.21.15.105:0/2952450605 learned_addr learned my addr 172.21.15.105:0/2952450605 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.359+0000 7f5327fff640 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b75590 0x55f292b77a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5334f13640 1 -- 172.21.15.105:0/2952450605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b75590 msgr2=0x55f292b77a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5334f13640 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b75590 0x55f292b77a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5334f13640 1 -- 172.21.15.105:0/2952450605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b6fa30 msgr2=0x55f292b6fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5334f13640 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b6fa30 0x55f292b6fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5334f13640 1 -- 172.21.15.105:0/2952450605 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2927e83f0 con 0x55f292b70420 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5327fff640 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b75590 0x55f292b77a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5335714640 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b6fa30 0x55f292b6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5325ffb640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53180a2f20 0x7f53180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5335714640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53180a2f20 0x7f53180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5334f13640 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b70420 0x55f292b74f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f532c0f7560 tx=0x7f532c0f7650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580d6d77d2e76d9b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5325ffb640 1 -- 172.21.15.105:0/2683310767 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f532c0f1580 con 0x7f53301007c0 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5316ffd640 1 -- 172.21.15.105:0/2952450605 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0b9f10 con 0x55f292b70420 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5316ffd640 1 -- 172.21.15.105:0/2952450605 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f532c0be370 con 0x55f292b70420 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.360+0000 7f5316ffd640 1 -- 172.21.15.105:0/2952450605 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0bd5d0 con 0x55f292b70420 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 -- 172.21.15.105:0/2952450605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b70420 msgr2=0x55f292b74f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f5335714640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53180a2f20 0x7f53180a53e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55f292b70c80 tx=0x7f5328009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b70420 0x55f292b74f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f532c0f7560 tx=0x7f532c0f7650 comp rx=0 tx=0).stop 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 -- 172.21.15.105:0/2952450605 shutdown_connections 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b75590 0x55f292b77a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b70420 0x55f292b74f20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 --2- 172.21.15.105:0/2952450605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b6fa30 0x55f292b6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 -- 172.21.15.105:0/2952450605 >> 172.21.15.105:0/2952450605 conn(0x55f292b6afa0 msgr2=0x55f292b6b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 -- 172.21.15.105:0/2952450605 shutdown_connections 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 -- 172.21.15.105:0/2952450605 wait complete. 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.361+0000 7f53399fd740 1 -- start start 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f53399fd740 1 Processor -- start 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f53399fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b6fa30 0x55f292b79c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f53399fd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b70420 0x55f292b7a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f53399fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b75590 0x55f292b811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f53399fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f292b78300 con 0x55f292b75590 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f53399fd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f292a90310 con 0x55f292b70420 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f53399fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f292b78480 con 0x55f292b6fa30 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5335714640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b75590 0x55f292b811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5334f13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b6fa30 0x55f292b79c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5327fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b70420 0x55f292b7a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f53399fd740 10 client.0 _ll_register_callbacks cb 0x55f292acbc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:24.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5334f13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b6fa30 0x55f292b79c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60730/0 (socket says 172.21.15.105:60730) 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5335714640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b75590 0x55f292b811c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43976/0 (socket says 172.21.15.105:43976) 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5334f13640 1 -- 172.21.15.105:0/2886834869 learned_addr learned my addr 172.21.15.105:0/2886834869 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5334f13640 1 -- 172.21.15.105:0/2886834869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b70420 msgr2=0x55f292b7a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5334f13640 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b70420 0x55f292b7a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5334f13640 1 -- 172.21.15.105:0/2886834869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b75590 msgr2=0x55f292b811c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5334f13640 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b75590 0x55f292b811c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.362+0000 7f5334f13640 1 -- 172.21.15.105:0/2886834869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2928b27d0 con 0x55f292b6fa30 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f5334f13640 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b6fa30 0x55f292b79c40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f532c0f7560 tx=0x7f532c0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5db125eabbcb431 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f530ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f530ffff640 1 -- 172.21.15.105:0/2886834869 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0bd7b0 con 0x55f292b6fa30 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f530ffff640 1 -- 172.21.15.105:0/2886834869 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f532c003d20 con 0x55f292b6fa30 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 1 -- 172.21.15.105:0/2886834869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532c0be370 con 0x55f292b6fa30 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f530ffff640 1 -- 172.21.15.105:0/2886834869 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f532c0bda40 con 0x55f292b6fa30 2024-03-26T17:43:24.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 10 client.27148 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 1 -- 172.21.15.105:0/2886834869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f292b78300 con 0x55f292b6fa30 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f530ffff640 1 -- 172.21.15.105:0/2886834869 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f532c0be370 con 0x55f292b6fa30 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f530ffff640 1 -- 172.21.15.105:0/2886834869 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f532c0bded0 con 0x55f292b6fa30 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 10 client.27148 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 10 client.27148 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 20 client.27148 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f292b859d0 0x55f292b87de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 4 client.27148 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.363+0000 7f53399fd740 1 -- 172.21.15.105:0/2886834869 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f292ae1eb0 con 0x55f292b859d0 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.364+0000 7f5327fff640 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f292b859d0 0x55f292b87de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.364+0000 7f5327fff640 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f292b859d0 0x55f292b87de0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26e6b2614bf7de3d server_cookie=363f2857ffe76c61 in_seq=0 out_seq=0 2024-03-26T17:43:24.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.364+0000 7f530ffff640 10 client.27148 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f530ffff640 1 -- 172.21.15.105:0/2886834869 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f292ae1eb0 con 0x55f292b859d0 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f530ffff640 10 client.27148 handle_command_reply: tid=0 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 client.27148 shutdown 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 -- 172.21.15.105:0/2886834869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b6fa30 msgr2=0x55f292b79c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b6fa30 0x55f292b79c40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f532c0f7560 tx=0x7f532c0035f0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 -- 172.21.15.105:0/2886834869 shutdown_connections 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f292b859d0 0x55f292b87de0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b75590 0x55f292b811c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f530ffff640 0 client.27148 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b70420 0x55f292b7a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 --2- 172.21.15.105:0/2886834869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b6fa30 0x55f292b79c40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 -- 172.21.15.105:0/2886834869 >> 172.21.15.105:0/2886834869 conn(0x55f292b6afa0 msgr2=0x55f292b6a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 -- 172.21.15.105:0/2886834869 shutdown_connections 2024-03-26T17:43:24.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 1 -- 172.21.15.105:0/2886834869 wait complete. 2024-03-26T17:43:24.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.365+0000 7f53399fd740 20 client.27148 trim_cache size 0 max 16384 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f53399fd740 1 Processor -- start 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f53399fd740 1 -- start start 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f53399fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f53399fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 0x55f292b81180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f53399fd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 0x55f292b69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f53399fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53301a8dd0 con 0x55f292b6fa30 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f53399fd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53301aa910 con 0x55f292b75590 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f53399fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53301a94a0 con 0x55f292b70420 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5334f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5334f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b79f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43984/0 (socket says 172.21.15.105:43984) 2024-03-26T17:43:24.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5334f13640 1 -- 172.21.15.105:0/3728304416 learned_addr learned my addr 172.21.15.105:0/3728304416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5327fff640 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 0x55f292b81180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5335714640 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 0x55f292b69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5334f13640 1 -- 172.21.15.105:0/3728304416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 msgr2=0x55f292b81180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5334f13640 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 0x55f292b81180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5334f13640 1 -- 172.21.15.105:0/3728304416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 msgr2=0x55f292b69930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5334f13640 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 0x55f292b69930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.390+0000 7f5334f13640 1 -- 172.21.15.105:0/3728304416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f292b6add0 con 0x55f292b6fa30 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.391+0000 7f5334f13640 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b79f60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f532c005ab0 tx=0x7f532c0bc180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30551afb85a15d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.391+0000 7f53167fc640 1 -- 172.21.15.105:0/3728304416 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0bd7b0 con 0x55f292b6fa30 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.391+0000 7f53167fc640 1 -- 172.21.15.105:0/3728304416 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f532c0be370 con 0x55f292b6fa30 2024-03-26T17:43:24.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.391+0000 7f53167fc640 1 -- 172.21.15.105:0/3728304416 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0bd5d0 con 0x55f292b6fa30 2024-03-26T17:43:24.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 -- 172.21.15.105:0/3728304416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 msgr2=0x55f292b79f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b79f60 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f532c005ab0 tx=0x7f532c0bc180 comp rx=0 tx=0).stop 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 -- 172.21.15.105:0/3728304416 shutdown_connections 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 0x55f292b69930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 0x55f292b81180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 --2- 172.21.15.105:0/3728304416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b79f60 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 -- 172.21.15.105:0/3728304416 >> 172.21.15.105:0/3728304416 conn(0x55f292b6afa0 msgr2=0x55f292b79850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 -- 172.21.15.105:0/3728304416 shutdown_connections 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 -- 172.21.15.105:0/3728304416 wait complete. 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 -- start start 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.392+0000 7f53399fd740 1 Processor -- start 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f53399fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f53399fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 0x55f292b7cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f53399fd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 0x55f292b7fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f53399fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f53301aa240 con 0x55f292b6fa30 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f53399fd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53301a9b70 con 0x55f292b75590 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f53399fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f53301a6db0 con 0x55f292b70420 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f53399fd740 10 client.0 _ll_register_callbacks cb 0x7f5328009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b7c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43990/0 (socket says 172.21.15.105:43990) 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5327fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 0x55f292b7cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 -- 172.21.15.105:0/4111079741 learned_addr learned my addr 172.21.15.105:0/4111079741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5335714640 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 0x55f292b7fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 -- 172.21.15.105:0/4111079741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 msgr2=0x55f292b7cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 0x55f292b7cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 -- 172.21.15.105:0/4111079741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 msgr2=0x55f292b7fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 0x55f292b7fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 -- 172.21.15.105:0/4111079741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f292b80220 con 0x55f292b6fa30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f5334f13640 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b7c740 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f532c0ba5f0 tx=0x7f532c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882e6e321fd1c7af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f530ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f530ffff640 1 -- 172.21.15.105:0/4111079741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c0b9f10 con 0x55f292b6fa30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f530ffff640 1 -- 172.21.15.105:0/4111079741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f532c0c63f0 con 0x55f292b6fa30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.393+0000 7f53399fd740 1 -- 172.21.15.105:0/4111079741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532c0be370 con 0x55f292b6fa30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f530ffff640 1 -- 172.21.15.105:0/4111079741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f532c0bd5d0 con 0x55f292b6fa30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f53399fd740 10 client.17316 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f53399fd740 1 -- 172.21.15.105:0/4111079741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f292b80220 con 0x55f292b6fa30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f530ffff640 1 -- 172.21.15.105:0/4111079741 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f532c0be370 con 0x55f292b6fa30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f530ffff640 1 -- 172.21.15.105:0/4111079741 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f532c0bd0c0 con 0x55f292b6fa30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f53399fd740 10 client.17316 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f53399fd740 10 client.17316 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:43:24.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f53399fd740 20 client.17316 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f53399fd740 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f292bf04a0 0x55f292bf2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f53399fd740 4 client.17316 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f53399fd740 1 -- 172.21.15.105:0/4111079741 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f292adc200 con 0x55f292bf04a0 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.394+0000 7f5327fff640 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f292bf04a0 0x55f292bf2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f530ffff640 10 client.17316 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f5327fff640 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f292bf04a0 0x55f292bf2890 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=629395d08454ad21 server_cookie=cb41353a1f1971be in_seq=0 out_seq=0 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f530ffff640 1 -- 172.21.15.105:0/4111079741 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f292adc200 con 0x55f292bf04a0 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f530ffff640 10 client.17316 handle_command_reply: tid=0 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 client.17316 shutdown 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 -- 172.21.15.105:0/4111079741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 msgr2=0x55f292b7c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b7c740 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f532c0ba5f0 tx=0x7f532c0057a0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 -- 172.21.15.105:0/4111079741 shutdown_connections 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f292bf04a0 0x55f292bf2890 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f292b75590 0x55f292b7fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f530ffff640 0 client.17316 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:43:24.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f292b70420 0x55f292b7cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 --2- 172.21.15.105:0/4111079741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f292b6fa30 0x55f292b7c740 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 -- 172.21.15.105:0/4111079741 >> 172.21.15.105:0/4111079741 conn(0x55f292b6afa0 msgr2=0x55f292b6ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 -- 172.21.15.105:0/4111079741 shutdown_connections 2024-03-26T17:43:24.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 1 -- 172.21.15.105:0/4111079741 wait complete. 2024-03-26T17:43:24.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.395+0000 7f53399fd740 20 client.17316 trim_cache size 0 max 16384 2024-03-26T17:43:24.393 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 -- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53180a2f20 msgr2=0x7f53180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53180a2f20 0x7f53180a53e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55f292b70c80 tx=0x7f5328009500 comp rx=0 tx=0).stop 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 -- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 msgr2=0x7f53300695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 0x7f53300695b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f532c000c00 tx=0x7f532c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 -- 172.21.15.105:0/2683310767 shutdown_connections 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f53180a2f20 0x7f53180a53e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5330102370 0x7f533019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53301019b0 0x7f533019c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 --2- 172.21.15.105:0/2683310767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f53301007c0 0x7f53300695b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 -- 172.21.15.105:0/2683310767 >> 172.21.15.105:0/2683310767 conn(0x7f53300fbd50 msgr2=0x7f5330104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 -- 172.21.15.105:0/2683310767 shutdown_connections 2024-03-26T17:43:24.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.396+0000 7f5337177640 1 -- 172.21.15.105:0/2683310767 wait complete. 2024-03-26T17:43:24.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 -- 172.21.15.105:0/2157238059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2084102370 msgr2=0x7f2084108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 --2- 172.21.15.105:0/2157238059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2084102370 0x7f2084108c00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f207c009be0 tx=0x7f207c02bef0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 -- 172.21.15.105:0/2157238059 shutdown_connections 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 --2- 172.21.15.105:0/2157238059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2084102370 0x7f2084108c00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 --2- 172.21.15.105:0/2157238059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20841019b0 0x7f2084101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 --2- 172.21.15.105:0/2157238059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20841007c0 0x7f2084100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 -- 172.21.15.105:0/2157238059 >> 172.21.15.105:0/2157238059 conn(0x7f20840fbd50 msgr2=0x7f20840fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 -- 172.21.15.105:0/2157238059 shutdown_connections 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.522+0000 7f2089d89640 1 -- 172.21.15.105:0/2157238059 wait complete. 2024-03-26T17:43:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.523+0000 7f2089d89640 1 Processor -- start 2024-03-26T17:43:24.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.523+0000 7f2089d89640 1 -- start start 2024-03-26T17:43:24.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.523+0000 7f2089d89640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20841007c0 0x7f2084069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.523+0000 7f2089d89640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20841019b0 0x7f208419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.523+0000 7f2089d89640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2084102370 0x7f208419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.523+0000 7f2089d89640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f208410c2f0 con 0x7f20841019b0 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.523+0000 7f2089d89640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f208410c170 con 0x7f2084102370 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.523+0000 7f2089d89640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f208410c470 con 0x7f20841007c0 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20841019b0 0x7f208419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20841019b0 0x7f208419c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36184/0 (socket says 172.21.15.105:36184) 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 -- 172.21.15.105:0/141798095 learned_addr learned my addr 172.21.15.105:0/141798095 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2083fff640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2084102370 0x7f208419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f20837fe640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20841007c0 0x7f2084069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 -- 172.21.15.105:0/141798095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20841007c0 msgr2=0x7f2084069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20841007c0 0x7f2084069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 -- 172.21.15.105:0/141798095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2084102370 msgr2=0x7f208419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2084102370 0x7f208419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 -- 172.21.15.105:0/141798095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208419f790 con 0x7f20841019b0 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2082ffd640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20841019b0 0x7f208419c860 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f207800e9f0 tx=0x7f207800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2080ff9640 1 -- 172.21.15.105:0/141798095 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207800cd80 con 0x7f20841019b0 2024-03-26T17:43:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2080ff9640 1 -- 172.21.15.105:0/141798095 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20780054d0 con 0x7f20841019b0 2024-03-26T17:43:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2089d89640 1 -- 172.21.15.105:0/141798095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208419fa20 con 0x7f20841019b0 2024-03-26T17:43:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2080ff9640 1 -- 172.21.15.105:0/141798095 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078010490 con 0x7f20841019b0 2024-03-26T17:43:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.524+0000 7f2089d89640 1 -- 172.21.15.105:0/141798095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208419ff20 con 0x7f20841019b0 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.526+0000 7f2080ff9640 1 -- 172.21.15.105:0/141798095 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2078005020 con 0x7f20841019b0 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f208c97a740 1 Processor -- start 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f208c97a740 1 -- start start 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f208c97a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87273e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f208c97a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87278f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f208c97a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e8727ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f208c97a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562e871d8970 con 0x562e87273a30 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f208c97a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562e87180d70 con 0x562e87274420 2024-03-26T17:43:24.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f208c97a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562e871d4920 con 0x562e87279590 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f2080ff9640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f206c0a2f20 0x7f206c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f2083fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87273e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f2083fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87273e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36194/0 (socket says 172.21.15.105:36194) 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f2083fff640 1 -- 172.21.15.105:0/3310619540 learned_addr learned my addr 172.21.15.105:0/3310619540 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f2082ffd640 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e8727ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f20837fe640 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87278f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f2083fff640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f206c0a2f20 0x7f206c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f20837fe640 1 -- 172.21.15.105:0/3310619540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 msgr2=0x562e8727ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.527+0000 7f20837fe640 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e8727ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f20837fe640 1 -- 172.21.15.105:0/3310619540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 msgr2=0x562e87273e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f20837fe640 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87273e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f2080ff9640 1 -- 172.21.15.105:0/141798095 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f20780cf790 con 0x7f20841019b0 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f20837fe640 1 -- 172.21.15.105:0/3310619540 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e86eec3f0 con 0x562e87274420 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f2083fff640 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87273e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f20837fe640 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87278f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2084101810 tx=0x7f2074011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8fa605797d3933 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f2083fff640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f206c0a2f20 0x7f206c0a53e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x562e87274c80 tx=0x7f207c02bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f2069ffb640 1 -- 172.21.15.105:0/3310619540 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207401e070 con 0x562e87274420 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f2069ffb640 1 -- 172.21.15.105:0/3310619540 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20740198e0 con 0x562e87274420 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.528+0000 7f2069ffb640 1 -- 172.21.15.105:0/3310619540 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207400e940 con 0x562e87274420 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 -- 172.21.15.105:0/3310619540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 msgr2=0x562e87278f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87278f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2084101810 tx=0x7f2074011810 comp rx=0 tx=0).stop 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 -- 172.21.15.105:0/3310619540 shutdown_connections 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e8727ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87278f20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 --2- 172.21.15.105:0/3310619540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87273e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 -- 172.21.15.105:0/3310619540 >> 172.21.15.105:0/3310619540 conn(0x562e8726efa0 msgr2=0x562e8726f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 -- 172.21.15.105:0/3310619540 shutdown_connections 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 -- 172.21.15.105:0/3310619540 wait complete. 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.529+0000 7f208c97a740 1 -- start start 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f208c97a740 1 Processor -- start 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f208c97a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87273a30 0x562e8727df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f208c97a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e8727e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f208c97a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87279590 0x562e872854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f208c97a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562e8727c540 con 0x562e87279590 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f208c97a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562e87194310 con 0x562e87274420 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f208c97a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562e8727c6c0 con 0x562e87273a30 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f208c97a740 10 client.0 _ll_register_callbacks cb 0x562e871cfc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f2083fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87279590 0x562e872854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f2082ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e8727e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f20837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87273a30 0x562e8727df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f2082ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e8727e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37424/0 (socket says 172.21.15.105:37424) 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f2082ffd640 1 -- 172.21.15.105:0/3978842375 learned_addr learned my addr 172.21.15.105:0/3978842375 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f2083fff640 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87279590 0x562e872854f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36196/0 (socket says 172.21.15.105:36196) 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f20837fe640 1 -- 172.21.15.105:0/3978842375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 msgr2=0x562e8727e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f20837fe640 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e8727e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f20837fe640 1 -- 172.21.15.105:0/3978842375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87279590 msgr2=0x562e872854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f20837fe640 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87279590 0x562e872854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f20837fe640 1 -- 172.21.15.105:0/3978842375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e86fb67d0 con 0x562e87273a30 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.530+0000 7f2082ffd640 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e8727e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f20837fe640 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87273a30 0x562e8727df70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x562e8726ff10 tx=0x7f2074009870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde321f2de0a9d55 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f2062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f2062ffd640 1 -- 172.21.15.105:0/3978842375 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207401e050 con 0x562e87273a30 2024-03-26T17:43:24.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f2062ffd640 1 -- 172.21.15.105:0/3978842375 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f2074005040 con 0x562e87273a30 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 1 -- 172.21.15.105:0/3978842375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20740198e0 con 0x562e87273a30 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f2062ffd640 1 -- 172.21.15.105:0/3978842375 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2074005240 con 0x562e87273a30 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 10 client.27166 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 1 -- 172.21.15.105:0/3978842375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e87288860 con 0x562e87273a30 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f2062ffd640 1 -- 172.21.15.105:0/3978842375 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20740198e0 con 0x562e87273a30 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f2062ffd640 1 -- 172.21.15.105:0/3978842375 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f207400e680 con 0x562e87273a30 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 10 client.27166 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 10 client.27166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 20 client.27166 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e87289d20 0x562e8728c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 4 client.27166 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.531+0000 7f208c97a740 1 -- 172.21.15.105:0/3978842375 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e871e5eb0 con 0x562e87289d20 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.532+0000 7f2082ffd640 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e87289d20 0x562e8728c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.532+0000 7f2062ffd640 10 client.27166 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:24.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.532+0000 7f2082ffd640 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e87289d20 0x562e8728c190 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5841e6e13c6e3c5 server_cookie=9186ef6a9d87151a in_seq=0 out_seq=0 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.535+0000 7f2062ffd640 1 -- 172.21.15.105:0/3978842375 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562e871e5eb0 con 0x562e87289d20 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.535+0000 7f2062ffd640 10 client.27166 handle_command_reply: tid=0 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.535+0000 7f208c97a740 1 client.27166 shutdown 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.535+0000 7f208c97a740 1 -- 172.21.15.105:0/3978842375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87273a30 msgr2=0x562e8727df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.535+0000 7f208c97a740 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87273a30 0x562e8727df70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x562e8726ff10 tx=0x7f2074009870 comp rx=0 tx=0).stop 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 1 -- 172.21.15.105:0/3978842375 shutdown_connections 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e87289d20 0x562e8728c190 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87279590 0x562e872854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e8727e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f2062ffd640 0 client.27166 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 1 --2- 172.21.15.105:0/3978842375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87273a30 0x562e8727df70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 1 -- 172.21.15.105:0/3978842375 >> 172.21.15.105:0/3978842375 conn(0x562e8726efa0 msgr2=0x562e8726e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 1 -- 172.21.15.105:0/3978842375 shutdown_connections 2024-03-26T17:43:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 1 -- 172.21.15.105:0/3978842375 wait complete. 2024-03-26T17:43:24.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.536+0000 7f208c97a740 20 client.27166 trim_cache size 0 max 16384 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f208c97a740 1 Processor -- start 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f208c97a740 1 -- start start 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f208c97a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e8726e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f208c97a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e872854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f208c97a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e8727d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f208c97a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20841a6c40 con 0x562e87273a30 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f208c97a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20841a8780 con 0x562e87274420 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f208c97a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20841a7310 con 0x562e87279590 2024-03-26T17:43:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e8726e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e8726e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36206/0 (socket says 172.21.15.105:36206) 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 -- 172.21.15.105:0/936064423 learned_addr learned my addr 172.21.15.105:0/936064423 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f2082ffd640 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e872854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f2083fff640 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e8727d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 -- 172.21.15.105:0/936064423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 msgr2=0x562e8727d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e8727d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 -- 172.21.15.105:0/936064423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 msgr2=0x562e872854b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e872854b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 -- 172.21.15.105:0/936064423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e8726e8f0 con 0x562e87273a30 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f2082ffd640 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e872854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.560+0000 7f20837fe640 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e8726e040 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2074004fd0 tx=0x7f2074005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c54b10ee73d65e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.561+0000 7f20697fa640 1 -- 172.21.15.105:0/936064423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207401e070 con 0x562e87273a30 2024-03-26T17:43:24.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.561+0000 7f20697fa640 1 -- 172.21.15.105:0/936064423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f207402a070 con 0x562e87273a30 2024-03-26T17:43:24.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.561+0000 7f20697fa640 1 -- 172.21.15.105:0/936064423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074004640 con 0x562e87273a30 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 -- 172.21.15.105:0/936064423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 msgr2=0x562e8726e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e8726e040 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2074004fd0 tx=0x7f2074005e70 comp rx=0 tx=0).stop 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 -- 172.21.15.105:0/936064423 shutdown_connections 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e8727d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e872854b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 --2- 172.21.15.105:0/936064423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e8726e040 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 -- 172.21.15.105:0/936064423 >> 172.21.15.105:0/936064423 conn(0x562e8726efa0 msgr2=0x562e8726d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 -- 172.21.15.105:0/936064423 shutdown_connections 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 -- 172.21.15.105:0/936064423 wait complete. 2024-03-26T17:43:24.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 -- start start 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 Processor -- start 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.562+0000 7f208c97a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87285540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f208c97a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87281fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f208c97a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e872825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f208c97a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20841a80b0 con 0x562e87273a30 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f208c97a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20841a79e0 con 0x562e87274420 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f208c97a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20841a4c20 con 0x562e87279590 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f208c97a740 10 client.0 _ll_register_callbacks cb 0x7f207401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87285540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87285540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36208/0 (socket says 172.21.15.105:36208) 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 -- 172.21.15.105:0/940555253 learned_addr learned my addr 172.21.15.105:0/940555253 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f2082ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87281fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f2083fff640 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e872825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 -- 172.21.15.105:0/940555253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 msgr2=0x562e872825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e872825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 -- 172.21.15.105:0/940555253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 msgr2=0x562e87281fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87281fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 -- 172.21.15.105:0/940555253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e8727c860 con 0x562e87273a30 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f2082ffd640 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87281fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f20837fe640 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87285540 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f20740059d0 tx=0x7f2074019a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc0e177a1235ffa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f2062ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f2062ffd640 1 -- 172.21.15.105:0/940555253 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207401e070 con 0x562e87273a30 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f2062ffd640 1 -- 172.21.15.105:0/940555253 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f207400e5b0 con 0x562e87273a30 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.563+0000 7f208c97a740 1 -- 172.21.15.105:0/940555253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207402a070 con 0x562e87273a30 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f2062ffd640 1 -- 172.21.15.105:0/940555253 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f207400e7b0 con 0x562e87273a30 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f208c97a740 10 client.17334 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f208c97a740 1 -- 172.21.15.105:0/940555253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e872ef200 con 0x562e87273a30 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f2062ffd640 1 -- 172.21.15.105:0/940555253 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f207402a070 con 0x562e87273a30 2024-03-26T17:43:24.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f2062ffd640 1 -- 172.21.15.105:0/940555253 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f20740254a0 con 0x562e87273a30 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f208c97a740 10 client.17334 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f208c97a740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f208c97a740 20 client.17334 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f208c97a740 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e872efac0 0x562e872f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f208c97a740 4 client.17334 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f208c97a740 1 -- 172.21.15.105:0/940555253 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e871e0200 con 0x562e872efac0 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f2082ffd640 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e872efac0 0x562e872f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f2082ffd640 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e872efac0 0x562e872f1eb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de94e4a3080d3691 server_cookie=d9d6e338ee67496a in_seq=0 out_seq=0 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.564+0000 7f2062ffd640 10 client.17334 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f2062ffd640 1 -- 172.21.15.105:0/940555253 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e871e0200 con 0x562e872efac0 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f2062ffd640 10 client.17334 handle_command_reply: tid=0 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 client.17334 shutdown 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 -- 172.21.15.105:0/940555253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 msgr2=0x562e87285540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87285540 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f20740059d0 tx=0x7f2074019a40 comp rx=0 tx=0).stop 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 -- 172.21.15.105:0/940555253 shutdown_connections 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e872efac0 0x562e872f1eb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e87279590 0x562e872825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e87274420 0x562e87281fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f2062ffd640 0 client.17334 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 --2- 172.21.15.105:0/940555253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e87273a30 0x562e87285540 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 -- 172.21.15.105:0/940555253 >> 172.21.15.105:0/940555253 conn(0x562e8726efa0 msgr2=0x562e8726e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 -- 172.21.15.105:0/940555253 shutdown_connections 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 1 -- 172.21.15.105:0/940555253 wait complete. 2024-03-26T17:43:24.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.565+0000 7f208c97a740 20 client.17334 trim_cache size 0 max 16384 2024-03-26T17:43:24.564 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:24.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 -- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f206c0a2f20 msgr2=0x7f206c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f206c0a2f20 0x7f206c0a53e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x562e87274c80 tx=0x7f207c02bef0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 -- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20841019b0 msgr2=0x7f208419c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20841019b0 0x7f208419c860 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f207800e9f0 tx=0x7f207800edf0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 -- 172.21.15.105:0/141798095 shutdown_connections 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f206c0a2f20 0x7f206c0a53e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2084102370 0x7f208419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20841019b0 0x7f208419c860 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 --2- 172.21.15.105:0/141798095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20841007c0 0x7f2084069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 -- 172.21.15.105:0/141798095 >> 172.21.15.105:0/141798095 conn(0x7f20840fbd50 msgr2=0x7f2084104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 -- 172.21.15.105:0/141798095 shutdown_connections 2024-03-26T17:43:24.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:24.566+0000 7f2089d89640 1 -- 172.21.15.105:0/141798095 wait complete. 2024-03-26T17:43:25.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 146293 3.14 MB/sec execute 128 sec latency 18.604 ms 2024-03-26T17:43:25.570 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:43:25.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:43:25.570 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:43:25.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.697+0000 7f36625d4640 1 -- 172.21.15.105:0/3073953193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 msgr2=0x7f365c108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.697+0000 7f36625d4640 1 --2- 172.21.15.105:0/3073953193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 0x7f365c108c40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ca0 tx=0x7f365402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.697+0000 7f36625d4640 1 -- 172.21.15.105:0/3073953193 shutdown_connections 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.697+0000 7f36625d4640 1 --2- 172.21.15.105:0/3073953193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 0x7f365c108c40 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.697+0000 7f36625d4640 1 --2- 172.21.15.105:0/3073953193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f365c1019f0 0x7f365c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.697+0000 7f36625d4640 1 --2- 172.21.15.105:0/3073953193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365c100800 0x7f365c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.697+0000 7f36625d4640 1 -- 172.21.15.105:0/3073953193 >> 172.21.15.105:0/3073953193 conn(0x7f365c0fbd70 msgr2=0x7f365c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.697+0000 7f36625d4640 1 -- 172.21.15.105:0/3073953193 shutdown_connections 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 -- 172.21.15.105:0/3073953193 wait complete. 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 Processor -- start 2024-03-26T17:43:25.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 -- start start 2024-03-26T17:43:25.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f365c100800 0x7f365c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365c1019f0 0x7f365c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 0x7f365c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f365c10c310 con 0x7f365c1023b0 2024-03-26T17:43:25.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f365c10c190 con 0x7f365c100800 2024-03-26T17:43:25.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.698+0000 7f36625d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f365c10c490 con 0x7f365c1019f0 2024-03-26T17:43:25.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f365bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f365c100800 0x7f365c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f3660b71640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 0x7f365c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f365bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f365c100800 0x7f365c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37466/0 (socket says 172.21.15.105:37466) 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f365bfff640 1 -- 172.21.15.105:0/1641479766 learned_addr learned my addr 172.21.15.105:0/1641479766 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365c1019f0 0x7f365c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f3660b71640 1 -- 172.21.15.105:0/1641479766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365c1019f0 msgr2=0x7f365c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f3660b71640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365c1019f0 0x7f365c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f3660b71640 1 -- 172.21.15.105:0/1641479766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f365c100800 msgr2=0x7f365c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f3660b71640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f365c100800 0x7f365c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f3660b71640 1 -- 172.21.15.105:0/1641479766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c19f790 con 0x7f365c1023b0 2024-03-26T17:43:25.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f365bfff640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f365c100800 0x7f365c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:25.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f3660b71640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 0x7f365c19f120 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f365402c700 tx=0x7f36540328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f36597fa640 1 -- 172.21.15.105:0/1641479766 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654041020 con 0x7f365c1023b0 2024-03-26T17:43:25.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f36625d4640 1 -- 172.21.15.105:0/1641479766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c19fa20 con 0x7f365c1023b0 2024-03-26T17:43:25.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f36597fa640 1 -- 172.21.15.105:0/1641479766 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3654003e20 con 0x7f365c1023b0 2024-03-26T17:43:25.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f36597fa640 1 -- 172.21.15.105:0/1641479766 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654039040 con 0x7f365c1023b0 2024-03-26T17:43:25.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.699+0000 7f36625d4640 1 -- 172.21.15.105:0/1641479766 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c19ff60 con 0x7f365c1023b0 2024-03-26T17:43:25.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36651c2740 1 Processor -- start 2024-03-26T17:43:25.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36597fa640 1 -- 172.21.15.105:0/1641479766 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3654044020 con 0x7f365c1023b0 2024-03-26T17:43:25.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36651c2740 1 -- start start 2024-03-26T17:43:25.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36651c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36651c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36651c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 0x55c3d36b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36651c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c3d3614630 con 0x55c3d36afb80 2024-03-26T17:43:25.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36651c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c3d35bce10 con 0x55c3d36af190 2024-03-26T17:43:25.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.702+0000 7f36651c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c3d36105e0 con 0x55c3d36b4cf0 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36232/0 (socket says 172.21.15.105:36232) 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365bfff640 1 -- 172.21.15.105:0/1605666132 learned_addr learned my addr 172.21.15.105:0/1605666132 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 0x55c3d36b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f36597fa640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36440a2fa0 0x7f36440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365b7fe640 1 -- 172.21.15.105:0/1605666132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 msgr2=0x55c3d36b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 0x55c3d36b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365b7fe640 1 -- 172.21.15.105:0/1605666132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 msgr2=0x55c3d36b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f3660b71640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36440a2fa0 0x7f36440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365b7fe640 1 -- 172.21.15.105:0/1605666132 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d35d3740 con 0x55c3d36af190 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365bfff640 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f36597fa640 1 -- 172.21.15.105:0/1641479766 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f36540f1480 con 0x7f365c1023b0 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36af5b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3650009730 tx=0x7f36500096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cefefafa504c566 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.703+0000 7f36427fc640 1 -- 172.21.15.105:0/1605666132 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650004050 con 0x55c3d36af190 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36427fc640 1 -- 172.21.15.105:0/1605666132 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36500054d0 con 0x55c3d36af190 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36427fc640 1 -- 172.21.15.105:0/1605666132 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650004c00 con 0x55c3d36af190 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 -- 172.21.15.105:0/1605666132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 msgr2=0x55c3d36af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f3660b71640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36440a2fa0 0x7f36440a5460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3654006010 tx=0x7f365402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36af5b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3650009730 tx=0x7f36500096c0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 -- 172.21.15.105:0/1605666132 shutdown_connections 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 0x55c3d36b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 --2- 172.21.15.105:0/1605666132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36af5b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 -- 172.21.15.105:0/1605666132 >> 172.21.15.105:0/1605666132 conn(0x55c3d36aa700 msgr2=0x55c3d36aab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 -- 172.21.15.105:0/1605666132 shutdown_connections 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.704+0000 7f36651c2740 1 -- 172.21.15.105:0/1605666132 wait complete. 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 1 -- start start 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 1 Processor -- start 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 0x55c3d36c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c3d36b7ca0 con 0x55c3d36afb80 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c3d35d01a0 con 0x55c3d36af190 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c3d36b7e20 con 0x55c3d36b4cf0 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f36651c2740 10 client.0 _ll_register_callbacks cb 0x55c3d3327c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f365bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f365bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36248/0 (socket says 172.21.15.105:36248) 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f365bfff640 1 -- 172.21.15.105:0/2592152040 learned_addr learned my addr 172.21.15.105:0/2592152040 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 0x55c3d36c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.705+0000 7f365b7fe640 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f365bfff640 1 -- 172.21.15.105:0/2592152040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 msgr2=0x55c3d36c0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f365bfff640 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 0x55c3d36c0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f365bfff640 1 -- 172.21.15.105:0/2592152040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 msgr2=0x55c3d36b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f365bfff640 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f365bfff640 1 -- 172.21.15.105:0/2592152040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d33f07e0 con 0x55c3d36afb80 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f365bfff640 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b9c40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f364c012d20 tx=0x7f364c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80017ee1c4974e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f363b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f363b7fe640 1 -- 172.21.15.105:0/2592152040 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364c010430 con 0x55c3d36afb80 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f363b7fe640 1 -- 172.21.15.105:0/2592152040 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f364c0105d0 con 0x55c3d36afb80 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f36651c2740 1 -- 172.21.15.105:0/2592152040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36500054d0 con 0x55c3d36afb80 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f363b7fe640 1 -- 172.21.15.105:0/2592152040 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f364c005020 con 0x55c3d36afb80 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f36651c2740 10 client.17346 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.706+0000 7f36651c2740 1 -- 172.21.15.105:0/2592152040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3d3450f70 con 0x55c3d36afb80 2024-03-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f363b7fe640 1 -- 172.21.15.105:0/2592152040 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36500054d0 con 0x55c3d36afb80 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f363b7fe640 1 -- 172.21.15.105:0/2592152040 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f364c010b20 con 0x55c3d36afb80 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f36651c2740 10 client.17346 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f36651c2740 10 client.17346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f36651c2740 20 client.17346 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f36651c2740 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c3d36c5610 0x55c3d36c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f36651c2740 4 client.17346 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f36651c2740 1 -- 172.21.15.105:0/2592152040 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3d3621b70 con 0x55c3d36c5610 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f365b7fe640 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c3d36c5610 0x55c3d36c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f363b7fe640 10 client.17346 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.707+0000 7f365b7fe640 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c3d36c5610 0x55c3d36c7a90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ecb8efa1101e527 server_cookie=e67d7cc057200e0f in_seq=0 out_seq=0 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f363b7fe640 1 -- 172.21.15.105:0/2592152040 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c3d3621b70 con 0x55c3d36c5610 2024-03-26T17:43:25.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f363b7fe640 10 client.17346 handle_command_reply: tid=0 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 client.17346 shutdown 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 -- 172.21.15.105:0/2592152040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 msgr2=0x55c3d36b9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b9c40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f364c012d20 tx=0x7f364c00f790 comp rx=0 tx=0).stop 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 -- 172.21.15.105:0/2592152040 shutdown_connections 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c3d36c5610 0x55c3d36c7a90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36b4cf0 0x55c3d36c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f363b7fe640 0 client.17346 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36afb80 0x55c3d36b9c40 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 --2- 172.21.15.105:0/2592152040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 -- 172.21.15.105:0/2592152040 >> 172.21.15.105:0/2592152040 conn(0x55c3d36aa700 msgr2=0x55c3d36a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 -- 172.21.15.105:0/2592152040 shutdown_connections 2024-03-26T17:43:25.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.708+0000 7f36651c2740 1 -- 172.21.15.105:0/2592152040 wait complete. 2024-03-26T17:43:25.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.709+0000 7f36651c2740 20 client.17346 trim_cache size 0 max 16384 2024-03-26T17:43:25.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f36651c2740 1 Processor -- start 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f36651c2740 1 -- start start 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f36651c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36aa700 0x55c3d36ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f36651c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36af190 0x55c3d36c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f36651c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 0x55c3d36cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f36651c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1a8e70 con 0x55c3d36af190 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f36651c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1aa9b0 con 0x55c3d36aa700 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f36651c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1a9540 con 0x55c3d36afb80 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36aa700 0x55c3d36ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.727+0000 7f365bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36af190 0x55c3d36c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36aa700 0x55c3d36ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37496/0 (socket says 172.21.15.105:37496) 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 -- 172.21.15.105:0/1732287962 learned_addr learned my addr 172.21.15.105:0/1732287962 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1732287962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 0x55c3d36cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 -- 172.21.15.105:0/1732287962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 msgr2=0x55c3d36cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1732287962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 0x55c3d36cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 -- 172.21.15.105:0/1732287962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36af190 msgr2=0x55c3d36c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1732287962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36af190 0x55c3d36c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 -- 172.21.15.105:0/1732287962 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d32e8170 con 0x55c3d36aa700 2024-03-26T17:43:25.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f365b7fe640 1 --2- 172.21.15.105:0/1732287962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36aa700 0x55c3d36ae070 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55c3d36b03e0 tx=0x7f36500096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db2c691714a2a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f3641ffb640 1 -- 172.21.15.105:0/1732287962 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650038070 con 0x55c3d36aa700 2024-03-26T17:43:25.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f3641ffb640 1 -- 172.21.15.105:0/1732287962 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f365000f790 con 0x55c3d36aa700 2024-03-26T17:43:25.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.728+0000 7f3641ffb640 1 -- 172.21.15.105:0/1732287962 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650004e60 con 0x55c3d36aa700 2024-03-26T17:43:25.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.729+0000 7f36651c2740 1 -- 172.21.15.105:0/1732287962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36aa700 msgr2=0x55c3d36ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.729+0000 7f36651c2740 1 --2- 172.21.15.105:0/1732287962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36aa700 0x55c3d36ae070 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55c3d36b03e0 tx=0x7f36500096c0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.729+0000 7f36651c2740 1 -- 172.21.15.105:0/1732287962 shutdown_connections 2024-03-26T17:43:25.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.729+0000 7f36651c2740 1 --2- 172.21.15.105:0/1732287962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 0x55c3d36cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.729+0000 7f36651c2740 1 --2- 172.21.15.105:0/1732287962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36af190 0x55c3d36c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.729+0000 7f36651c2740 1 --2- 172.21.15.105:0/1732287962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36aa700 0x55c3d36ae070 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.730+0000 7f36651c2740 1 -- 172.21.15.105:0/1732287962 >> 172.21.15.105:0/1732287962 conn(0x55c3d36aae50 msgr2=0x55c3d36ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.730+0000 7f36651c2740 1 -- 172.21.15.105:0/1732287962 shutdown_connections 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.730+0000 7f36651c2740 1 -- 172.21.15.105:0/1732287962 wait complete. 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.730+0000 7f36651c2740 1 -- start start 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.730+0000 7f36651c2740 1 Processor -- start 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f36651c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36aa700 0x55c3d36bead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f36651c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36bf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f36651c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 0x55c3d36c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f36651c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1aa2e0 con 0x55c3d36aa700 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f36651c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1a9c10 con 0x55c3d36af190 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f36651c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1a6e50 con 0x55c3d36afb80 2024-03-26T17:43:25.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f36651c2740 10 client.0 _ll_register_callbacks cb 0x7f364c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36aa700 0x55c3d36bead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36aa700 0x55c3d36bead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36256/0 (socket says 172.21.15.105:36256) 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 -- 172.21.15.105:0/2850800966 learned_addr learned my addr 172.21.15.105:0/2850800966 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 0x55c3d36c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 -- 172.21.15.105:0/2850800966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 msgr2=0x55c3d36c0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 0x55c3d36c0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 -- 172.21.15.105:0/2850800966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 msgr2=0x55c3d36bf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36bf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365b7fe640 1 -- 172.21.15.105:0/2850800966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d36c0610 con 0x55c3d36aa700 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.731+0000 7f365bfff640 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36bf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:25.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f365b7fe640 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36aa700 0x55c3d36bead0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f36500097c0 tx=0x7f3650005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a22ea102bdfdfcc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f363b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f363b7fe640 1 -- 172.21.15.105:0/2850800966 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650038040 con 0x55c3d36aa700 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 1 -- 172.21.15.105:0/2850800966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365000f790 con 0x55c3d36aa700 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f363b7fe640 1 -- 172.21.15.105:0/2850800966 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f365000fb40 con 0x55c3d36aa700 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f363b7fe640 1 -- 172.21.15.105:0/2850800966 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f365000fd40 con 0x55c3d36aa700 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 10 client.17352 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 1 -- 172.21.15.105:0/2850800966 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3d372c190 con 0x55c3d36aa700 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f363b7fe640 1 -- 172.21.15.105:0/2850800966 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f365000f790 con 0x55c3d36aa700 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f363b7fe640 1 -- 172.21.15.105:0/2850800966 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3650048610 con 0x55c3d36aa700 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 10 client.17352 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 20 client.17352 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:25.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c3d36b4cf0 0x55c3d372e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 4 client.17352 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:43:25.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.732+0000 7f36651c2740 1 -- 172.21.15.105:0/2850800966 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c3d361bec0 con 0x55c3d36b4cf0 2024-03-26T17:43:25.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.733+0000 7f365bfff640 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c3d36b4cf0 0x55c3d372e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.733+0000 7f363b7fe640 10 client.17352 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:25.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.733+0000 7f365bfff640 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c3d36b4cf0 0x55c3d372e8b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23274f2946df8435 server_cookie=df8ad19fa768ee0a in_seq=0 out_seq=0 2024-03-26T17:43:25.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f363b7fe640 1 -- 172.21.15.105:0/2850800966 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c3d361bec0 con 0x55c3d36b4cf0 2024-03-26T17:43:25.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f363b7fe640 10 client.17352 handle_command_reply: tid=0 2024-03-26T17:43:25.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 client.17352 shutdown 2024-03-26T17:43:25.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 -- 172.21.15.105:0/2850800966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36aa700 msgr2=0x55c3d36bead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36aa700 0x55c3d36bead0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f36500097c0 tx=0x7f3650005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:25.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 -- 172.21.15.105:0/2850800966 shutdown_connections 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c3d36b4cf0 0x55c3d372e8b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c3d36afb80 0x55c3d36c0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c3d36af190 0x55c3d36bf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 --2- 172.21.15.105:0/2850800966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c3d36aa700 0x55c3d36bead0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f363b7fe640 0 client.17352 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 -- 172.21.15.105:0/2850800966 >> 172.21.15.105:0/2850800966 conn(0x55c3d36aae50 msgr2=0x55c3d36ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 -- 172.21.15.105:0/2850800966 shutdown_connections 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 1 -- 172.21.15.105:0/2850800966 wait complete. 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.736+0000 7f36651c2740 20 client.17352 trim_cache size 0 max 16384 2024-03-26T17:43:25.732 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:43:25.733 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:43:25.733 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "d7063f70-79fb-4ed1-80e0-4deb1f4d9c2b", 2024-03-26T17:43:25.733 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:43:25.733 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 -- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36440a2fa0 msgr2=0x7f36440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36440a2fa0 0x7f36440a5460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3654006010 tx=0x7f365402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 -- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 msgr2=0x7f365c19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 0x7f365c19f120 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f365402c700 tx=0x7f36540328a0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 -- 172.21.15.105:0/1641479766 shutdown_connections 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36440a2fa0 0x7f36440a5460 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f365c1023b0 0x7f365c19f120 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f365c1019f0 0x7f365c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 --2- 172.21.15.105:0/1641479766 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f365c100800 0x7f365c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 -- 172.21.15.105:0/1641479766 >> 172.21.15.105:0/1641479766 conn(0x7f365c0fbd70 msgr2=0x7f365c0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 -- 172.21.15.105:0/1641479766 shutdown_connections 2024-03-26T17:43:25.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.737+0000 7f36625d4640 1 -- 172.21.15.105:0/1641479766 wait complete. 2024-03-26T17:43:25.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:43:25.742 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:43:25.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/1874869183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff0102370 msgr2=0x7f7ff0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/1874869183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff0102370 0x7f7ff0108c00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009ca0 tx=0x7f7fe802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/1874869183 shutdown_connections 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/1874869183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff0102370 0x7f7ff0108c00 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/1874869183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff01019b0 0x7f7ff0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/1874869183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff01007c0 0x7f7ff0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/1874869183 >> 172.21.15.105:0/1874869183 conn(0x7f7ff00fbd50 msgr2=0x7f7ff00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/1874869183 shutdown_connections 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.866+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/1874869183 wait complete. 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7ff5e7e640 1 Processor -- start 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7ff5e7e640 1 -- start start 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7ff5e7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff01007c0 0x7f7ff01969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7ff5e7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff01019b0 0x7f7ff0196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7ff5e7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff0102370 0x7f7ff019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7ff5e7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff010c2f0 con 0x7f7ff01019b0 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7ff5e7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff010c170 con 0x7f7ff01007c0 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7ff5e7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff010c470 con 0x7f7ff0102370 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff01019b0 0x7f7ff0196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff01007c0 0x7f7ff01969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff01019b0 0x7f7ff0196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36272/0 (socket says 172.21.15.105:36272) 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff01007c0 0x7f7ff01969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37534/0 (socket says 172.21.15.105:37534) 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7feeffd640 1 -- 172.21.15.105:0/3779944627 learned_addr learned my addr 172.21.15.105:0/3779944627 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.867+0000 7f7feffff640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff0102370 0x7f7ff019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7feffff640 1 -- 172.21.15.105:0/3779944627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff01007c0 msgr2=0x7f7ff01969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7feffff640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff01007c0 0x7f7ff01969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7feffff640 1 -- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff01019b0 msgr2=0x7f7ff0196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7feffff640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff01019b0 0x7f7ff0196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7feffff640 1 -- 172.21.15.105:0/3779944627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff019e5e0 con 0x7f7ff0102370 2024-03-26T17:43:25.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7feeffd640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff01019b0 0x7f7ff0196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:25.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff01007c0 0x7f7ff01969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:25.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7feffff640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff0102370 0x7f7ff019df70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7fe802c630 tx=0x7f7fe80328a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7fecff9640 1 -- 172.21.15.105:0/3779944627 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8041020 con 0x7f7ff0102370 2024-03-26T17:43:25.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7fecff9640 1 -- 172.21.15.105:0/3779944627 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fe8003e20 con 0x7f7ff0102370 2024-03-26T17:43:25.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/3779944627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff0068a20 con 0x7f7ff0102370 2024-03-26T17:43:25.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7fecff9640 1 -- 172.21.15.105:0/3779944627 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8039040 con 0x7f7ff0102370 2024-03-26T17:43:25.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.868+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/3779944627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff0068f10 con 0x7f7ff0102370 2024-03-26T17:43:25.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.871+0000 7f7fecff9640 1 -- 172.21.15.105:0/3779944627 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7fe8044020 con 0x7f7ff0102370 2024-03-26T17:43:25.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.871+0000 7f7ff8a68740 1 Processor -- start 2024-03-26T17:43:25.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.871+0000 7f7ff8a68740 1 -- start start 2024-03-26T17:43:25.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b3e2340980 con 0x55b3e23dc2e0 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b3e22e8be0 con 0x55b3e23e1450 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3e233c930 con 0x55b3e23db8f0 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fecff9640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7fd80a2f20 0x7f7fd80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36282/0 (socket says 172.21.15.105:36282) 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 -- 172.21.15.105:0/1028351138 learned_addr learned my addr 172.21.15.105:0/1028351138 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7feeffd640 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7feeffd640 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fecff9640 1 -- 172.21.15.105:0/3779944627 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7fe80f1640 con 0x7f7ff0102370 2024-03-26T17:43:25.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7feffff640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7fd80a2f20 0x7f7fd80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 -- 172.21.15.105:0/1028351138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 msgr2=0x55b3e23dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 -- 172.21.15.105:0/1028351138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 msgr2=0x55b3e23e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 -- 172.21.15.105:0/1028351138 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e211e7d0 con 0x55b3e23dc2e0 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e0de0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009730 tx=0x7f7fe4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfd41a8b9d24b7e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7fd5ffb640 1 -- 172.21.15.105:0/1028351138 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4008a10 con 0x55b3e23dc2e0 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.872+0000 7f7feffff640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7fd80a2f20 0x7f7fd80a53e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80ba190 tx=0x7f7fe802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7fd5ffb640 1 -- 172.21.15.105:0/1028351138 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fe401f070 con 0x55b3e23dc2e0 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7fd5ffb640 1 -- 172.21.15.105:0/1028351138 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe401ad10 con 0x55b3e23dc2e0 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 -- 172.21.15.105:0/1028351138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 msgr2=0x55b3e23e0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e0de0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009730 tx=0x7f7fe4009bf0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 -- 172.21.15.105:0/1028351138 shutdown_connections 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e0de0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/1028351138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 -- 172.21.15.105:0/1028351138 >> 172.21.15.105:0/1028351138 conn(0x55b3e23d6e60 msgr2=0x55b3e23d7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 -- 172.21.15.105:0/1028351138 shutdown_connections 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.873+0000 7f7ff8a68740 1 -- 172.21.15.105:0/1028351138 wait complete. 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 1 -- start start 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 1 Processor -- start 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23e5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b3e23e4400 con 0x55b3e23dc2e0 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b3e22fc160 con 0x55b3e23e1450 2024-03-26T17:43:25.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3e23e4580 con 0x55b3e23db8f0 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7ff8a68740 10 client.0 _ll_register_callbacks cb 0x55b3e2338720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36284/0 (socket says 172.21.15.105:36284) 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23e5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2962764147 learned_addr learned my addr 172.21.15.105:0/2962764147 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.874+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2962764147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 msgr2=0x55b3e23e5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23e5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2962764147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 msgr2=0x55b3e23ed080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23ed080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fef7fe640 1 -- 172.21.15.105:0/2962764147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e20543f0 con 0x55b3e23dc2e0 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:25.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e6040 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4000c00 tx=0x7f7fe401bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66da69757f4769c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fceffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fceffd640 1 -- 172.21.15.105:0/2962764147 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4004230 con 0x55b3e23dc2e0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fceffd640 1 -- 172.21.15.105:0/2962764147 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7fe40043d0 con 0x55b3e23dc2e0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7ff8a68740 1 -- 172.21.15.105:0/2962764147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe401f070 con 0x55b3e23dc2e0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fceffd640 1 -- 172.21.15.105:0/2962764147 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fe400f050 con 0x55b3e23dc2e0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7ff8a68740 10 client.17364 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7ff8a68740 1 -- 172.21.15.105:0/2962764147 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3e20140f0 con 0x55b3e23dc2e0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fceffd640 1 -- 172.21.15.105:0/2962764147 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fe401f070 con 0x55b3e23dc2e0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.875+0000 7f7fceffd640 1 -- 172.21.15.105:0/2962764147 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7fe4005040 con 0x55b3e23dc2e0 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7ff8a68740 10 client.17364 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7ff8a68740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7ff8a68740 20 client.17364 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b3e23f1620 0x55b3e23f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7ff8a68740 4 client.17364 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7ff8a68740 1 -- 172.21.15.105:0/2962764147 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3e234dec0 con 0x55b3e23f1620 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b3e23f1620 0x55b3e23f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7feeffd640 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b3e23f1620 0x55b3e23f3a30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d20956097f75f5 server_cookie=6762746c440a9d82 in_seq=0 out_seq=0 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.876+0000 7f7fceffd640 10 client.17364 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7fceffd640 1 -- 172.21.15.105:0/2962764147 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b3e234dec0 con 0x55b3e23f1620 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7fceffd640 10 client.17364 handle_command_reply: tid=0 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 client.17364 shutdown 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 -- 172.21.15.105:0/2962764147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 msgr2=0x55b3e23e6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e6040 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4000c00 tx=0x7f7fe401bcf0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 -- 172.21.15.105:0/2962764147 shutdown_connections 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b3e23f1620 0x55b3e23f3a30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23e6040 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7fceffd640 0 client.17364 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:25.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/2962764147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23e5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 -- 172.21.15.105:0/2962764147 >> 172.21.15.105:0/2962764147 conn(0x55b3e23d6e60 msgr2=0x55b3e23d6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 -- 172.21.15.105:0/2962764147 shutdown_connections 2024-03-26T17:43:25.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 1 -- 172.21.15.105:0/2962764147 wait complete. 2024-03-26T17:43:25.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.877+0000 7f7ff8a68740 20 client.17364 trim_cache size 0 max 16384 2024-03-26T17:43:25.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:25 smithi105 ceph-mon[32280]: pgmap v500: 97 pgs: 97 active+clean; 25 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:43:25.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7ff8a68740 1 Processor -- start 2024-03-26T17:43:25.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7ff8a68740 1 -- start start 2024-03-26T17:43:25.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23dc2e0 0x55b3e23dda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23e1450 0x55b3e23ed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff01a6bb0 con 0x55b3e23e1450 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff01a86f0 con 0x55b3e23dc2e0 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff01a7280 con 0x55b3e23db8f0 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.902+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23e1450 0x55b3e23ed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23dc2e0 0x55b3e23dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53738/0 (socket says 172.21.15.105:53738) 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 -- 172.21.15.105:0/795894267 learned_addr learned my addr 172.21.15.105:0/795894267 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 -- 172.21.15.105:0/795894267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23dc2e0 msgr2=0x55b3e23dda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 --2- 172.21.15.105:0/795894267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23dc2e0 0x55b3e23dda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 -- 172.21.15.105:0/795894267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23e1450 msgr2=0x55b3e23ed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 --2- 172.21.15.105:0/795894267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23e1450 0x55b3e23ed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 -- 172.21.15.105:0/795894267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3e23dc6c0 con 0x55b3e23db8f0 2024-03-26T17:43:25.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7feeffd640 1 --2- 172.21.15.105:0/795894267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dd500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0004700 tx=0x7f7fe0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfba28294e603e10 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7fd57fa640 1 -- 172.21.15.105:0/795894267 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0005020 con 0x55b3e23db8f0 2024-03-26T17:43:25.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7fd57fa640 1 -- 172.21.15.105:0/795894267 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fe00271c0 con 0x55b3e23db8f0 2024-03-26T17:43:25.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.903+0000 7f7fd57fa640 1 -- 172.21.15.105:0/795894267 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00049a0 con 0x55b3e23db8f0 2024-03-26T17:43:25.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 -- 172.21.15.105:0/795894267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 msgr2=0x55b3e23dd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/795894267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dd500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0004700 tx=0x7f7fe0004730 comp rx=0 tx=0).stop 2024-03-26T17:43:25.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 -- 172.21.15.105:0/795894267 shutdown_connections 2024-03-26T17:43:25.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/795894267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23e1450 0x55b3e23ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/795894267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23dc2e0 0x55b3e23dda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/795894267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23dd500 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 -- 172.21.15.105:0/795894267 >> 172.21.15.105:0/795894267 conn(0x55b3e23d6e60 msgr2=0x55b3e23e5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 -- 172.21.15.105:0/795894267 shutdown_connections 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 -- 172.21.15.105:0/795894267 wait complete. 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.904+0000 7f7ff8a68740 1 -- start start 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7ff8a68740 1 Processor -- start 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23ea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23eab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7ff8a68740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff01a8020 con 0x55b3e23dc2e0 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff01a7950 con 0x55b3e23e1450 2024-03-26T17:43:25.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7ff8a68740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff01a4b90 con 0x55b3e23db8f0 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7ff8a68740 10 client.0 _ll_register_callbacks cb 0x7f7fe000e5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23eab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36306/0 (socket says 172.21.15.105:36306) 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 -- 172.21.15.105:0/951404142 learned_addr learned my addr 172.21.15.105:0/951404142 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7feeffd640 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 -- 172.21.15.105:0/951404142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 msgr2=0x55b3e23ea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23ea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 -- 172.21.15.105:0/951404142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 msgr2=0x55b3e23ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23ebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 -- 172.21.15.105:0/951404142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3e23ec100 con 0x55b3e23dc2e0 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7feeffd640 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:25.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fef7fe640 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23eab50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009510 tx=0x7f7fe400f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becfeaa40d2ed1f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.905+0000 7f7fceffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7fceffd640 1 -- 172.21.15.105:0/951404142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe401a9f0 con 0x55b3e23dc2e0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7ff8a68740 1 -- 172.21.15.105:0/951404142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fe00271c0 con 0x55b3e23dc2e0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7fceffd640 1 -- 172.21.15.105:0/951404142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7fe401ab90 con 0x55b3e23dc2e0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7fceffd640 1 -- 172.21.15.105:0/951404142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fe401ad90 con 0x55b3e23dc2e0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7ff8a68740 10 client.17370 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7ff8a68740 1 -- 172.21.15.105:0/951404142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3e2457930 con 0x55b3e23dc2e0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7fceffd640 1 -- 172.21.15.105:0/951404142 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fe00271c0 con 0x55b3e23dc2e0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7fceffd640 1 -- 172.21.15.105:0/951404142 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7fe4005020 con 0x55b3e23dc2e0 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7ff8a68740 10 client.17370 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7ff8a68740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.906+0000 7f7ff8a68740 20 client.17370 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.907+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b3e24580a0 0x55b3e245a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.907+0000 7f7ff8a68740 4 client.17370 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:43:25.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.907+0000 7f7ff8a68740 1 -- 172.21.15.105:0/951404142 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3e2348210 con 0x55b3e24580a0 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.907+0000 7f7feeffd640 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b3e24580a0 0x55b3e245a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.907+0000 7f7fceffd640 10 client.17370 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.907+0000 7f7feeffd640 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b3e24580a0 0x55b3e245a4c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b5848c614c2fa95 server_cookie=c1caa155b524ba7c in_seq=0 out_seq=0 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.908+0000 7f7fceffd640 1 -- 172.21.15.105:0/951404142 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b3e2348210 con 0x55b3e24580a0 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.908+0000 7f7fceffd640 10 client.17370 handle_command_reply: tid=0 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.908+0000 7f7ff8a68740 1 client.17370 shutdown 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.908+0000 7f7ff8a68740 1 -- 172.21.15.105:0/951404142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 msgr2=0x55b3e23eab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.908+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23eab50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009510 tx=0x7f7fe400f680 comp rx=0 tx=0).stop 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.908+0000 7f7ff8a68740 1 -- 172.21.15.105:0/951404142 shutdown_connections 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.908+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b3e24580a0 0x55b3e245a4c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b3e23e1450 0x55b3e23ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b3e23dc2e0 0x55b3e23eab50 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff8a68740 1 --2- 172.21.15.105:0/951404142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3e23db8f0 0x55b3e23ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff8a68740 1 -- 172.21.15.105:0/951404142 >> 172.21.15.105:0/951404142 conn(0x55b3e23d6e60 msgr2=0x55b3e23da950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7fceffd640 0 client.17370 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff8a68740 1 -- 172.21.15.105:0/951404142 shutdown_connections 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff8a68740 1 -- 172.21.15.105:0/951404142 wait complete. 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff8a68740 20 client.17370 trim_cache size 0 max 16384 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stdout: "d7063f70-79fb-4ed1-80e0-4deb1f4d9c2b": { 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "d7063f70-79fb-4ed1-80e0-4deb1f4d9c2b", 2024-03-26T17:43:25.906 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:43:25.907 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:43:25.907 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:43:25.907 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:43:25.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7fd80a2f20 msgr2=0x7f7fd80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7fd80a2f20 0x7f7fd80a53e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80ba190 tx=0x7f7fe802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/3779944627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff0102370 msgr2=0x7f7ff019df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff0102370 0x7f7ff019df70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7fe802c630 tx=0x7f7fe80328a0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/3779944627 shutdown_connections 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7fd80a2f20 0x7f7fd80a53e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ff0102370 0x7f7ff019df70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ff01019b0 0x7f7ff0196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 --2- 172.21.15.105:0/3779944627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ff01007c0 0x7f7ff01969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/3779944627 >> 172.21.15.105:0/3779944627 conn(0x7f7ff00fbd50 msgr2=0x7f7ff0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.909+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/3779944627 shutdown_connections 2024-03-26T17:43:25.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:25.910+0000 7f7ff5e7e640 1 -- 172.21.15.105:0/3779944627 wait complete. 2024-03-26T17:43:25.914 INFO:tasks.cephfs.filesystem:scrub status for tag:d7063f70-79fb-4ed1-80e0-4deb1f4d9c2b - {'path': '/', 'tag': 'd7063f70-79fb-4ed1-80e0-4deb1f4d9c2b', 'options': 'recursive,force'} 2024-03-26T17:43:26.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 146932 3.15 MB/sec execute 129 sec latency 24.630 ms 2024-03-26T17:43:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:25 smithi144 ceph-mon[36095]: pgmap v500: 97 pgs: 97 active+clean; 25 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:43:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:25 smithi135 ceph-mon[35636]: pgmap v500: 97 pgs: 97 active+clean; 25 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:43:27.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 147604 3.13 MB/sec execute 130 sec latency 21.347 ms 2024-03-26T17:43:27.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:26 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:43:27.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:43:27.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:26 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:43:27.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:43:27.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:26 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:43:27.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:43:27.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:26 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:43:27.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:43:27.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:26 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:43:27.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:43:27.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:26 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:43:27.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:43:28.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 148220 3.15 MB/sec execute 131 sec latency 18.842 ms 2024-03-26T17:43:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:27 smithi144 ceph-mon[36095]: pgmap v501: 97 pgs: 97 active+clean; 27 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:43:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:43:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:27 smithi105 ceph-mon[32280]: pgmap v501: 97 pgs: 97 active+clean; 27 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:43:28.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:43:28.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:27 smithi135 ceph-mon[35636]: pgmap v501: 97 pgs: 97 active+clean; 27 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:43:28.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:43:29.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 148795 3.15 MB/sec execute 132 sec latency 21.479 ms 2024-03-26T17:43:30.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 149387 3.16 MB/sec execute 133 sec latency 15.922 ms 2024-03-26T17:43:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:29 smithi144 ceph-mon[36095]: pgmap v502: 97 pgs: 97 active+clean; 28 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-03-26T17:43:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:29 smithi105 ceph-mon[32280]: pgmap v502: 97 pgs: 97 active+clean; 28 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-03-26T17:43:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:29 smithi135 ceph-mon[35636]: pgmap v502: 97 pgs: 97 active+clean; 28 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-03-26T17:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 -- 172.21.15.105:0/2667350362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c1007e0 msgr2=0x7fba2c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 --2- 172.21.15.105:0/2667350362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c1007e0 0x7fba2c100c00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 -- 172.21.15.105:0/2667350362 shutdown_connections 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 --2- 172.21.15.105:0/2667350362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c102390 0x7fba2c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 --2- 172.21.15.105:0/2667350362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c1019d0 0x7fba2c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 --2- 172.21.15.105:0/2667350362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c1007e0 0x7fba2c100c00 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 -- 172.21.15.105:0/2667350362 >> 172.21.15.105:0/2667350362 conn(0x7fba2c0fbd50 msgr2=0x7fba2c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 -- 172.21.15.105:0/2667350362 shutdown_connections 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.650+0000 7fba30bd4640 1 -- 172.21.15.105:0/2667350362 wait complete. 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba30bd4640 1 Processor -- start 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba30bd4640 1 -- start start 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba30bd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c1007e0 0x7fba2c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba30bd4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c1019d0 0x7fba2c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba30bd4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c102390 0x7fba2c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba30bd4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c10c300 con 0x7fba2c102390 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba30bd4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c10c180 con 0x7fba2c1019d0 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba30bd4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c10c480 con 0x7fba2c1007e0 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba21d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c1019d0 0x7fba2c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba2a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c1007e0 0x7fba2c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba2ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c102390 0x7fba2c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba21d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c1019d0 0x7fba2c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37594/0 (socket says 172.21.15.105:37594) 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba2a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c1007e0 0x7fba2c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53774/0 (socket says 172.21.15.105:53774) 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba21d9b640 1 -- 172.21.15.105:0/900295048 learned_addr learned my addr 172.21.15.105:0/900295048 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba2ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c102390 0x7fba2c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36330/0 (socket says 172.21.15.105:36330) 2024-03-26T17:43:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba21d9b640 1 -- 172.21.15.105:0/900295048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c1007e0 msgr2=0x7fba2c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba21d9b640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c1007e0 0x7fba2c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba21d9b640 1 -- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c102390 msgr2=0x7fba2c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba21d9b640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c102390 0x7fba2c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba21d9b640 1 -- 172.21.15.105:0/900295048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c19f6f0 con 0x7fba2c1019d0 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.651+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c102390 0x7fba2c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.652+0000 7fba21d9b640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c1019d0 0x7fba2c198010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba1400e9f0 tx=0x7fba1400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.652+0000 7fba23fff640 1 -- 172.21.15.105:0/900295048 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1400cd80 con 0x7fba2c1019d0 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.652+0000 7fba23fff640 1 -- 172.21.15.105:0/900295048 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba140054d0 con 0x7fba2c1019d0 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.652+0000 7fba30bd4640 1 -- 172.21.15.105:0/900295048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2c19f9e0 con 0x7fba2c1019d0 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.652+0000 7fba23fff640 1 -- 172.21.15.105:0/900295048 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14010490 con 0x7fba2c1019d0 2024-03-26T17:43:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.652+0000 7fba30bd4640 1 -- 172.21.15.105:0/900295048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2c19fee0 con 0x7fba2c1019d0 2024-03-26T17:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.654+0000 7fba337c1740 1 Processor -- start 2024-03-26T17:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba337c1740 1 -- start start 2024-03-26T17:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba337c1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c8708769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba337c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c87087bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba337c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c87087e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba337c1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8707db0b0 con 0x55c8708765d0 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba337c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c870783960 con 0x55c870876fc0 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba337c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8707d7060 con 0x55c87087c130 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c8708769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c8708769f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36346/0 (socket says 172.21.15.105:36346) 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2ad9d640 1 -- 172.21.15.105:0/1972453986 learned_addr learned my addr 172.21.15.105:0/1972453986 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2a59c640 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c87087bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba23fff640 1 -- 172.21.15.105:0/900295048 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fba14005020 con 0x7fba2c1019d0 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2ad9d640 1 -- 172.21.15.105:0/1972453986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 msgr2=0x55c87087e5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c87087e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2ad9d640 1 -- 172.21.15.105:0/1972453986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 msgr2=0x55c87087bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c87087bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2ad9d640 1 -- 172.21.15.105:0/1972453986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c870795c60 con 0x55c8708765d0 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.655+0000 7fba2a59c640 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c87087bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.656+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c8708769f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fba240117e0 tx=0x7fba24011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04fe791ba9c1cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.656+0000 7fba20d99640 1 -- 172.21.15.105:0/1972453986 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24011310 con 0x55c8708765d0 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.656+0000 7fba20d99640 1 -- 172.21.15.105:0/1972453986 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba24003e20 con 0x55c8708765d0 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.656+0000 7fba20d99640 1 -- 172.21.15.105:0/1972453986 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240044f0 con 0x55c8708765d0 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.656+0000 7fba23fff640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2fc0 0x7fba180a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.656+0000 7fba23fff640 1 -- 172.21.15.105:0/900295048 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fba140cf790 con 0x7fba2c1019d0 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.656+0000 7fba2a59c640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2fc0 0x7fba180a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 -- 172.21.15.105:0/1972453986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 msgr2=0x55c8708769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba2a59c640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2fc0 0x7fba180a5480 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005ab0 tx=0x7fba1c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c8708769f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fba240117e0 tx=0x7fba24011ca0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 -- 172.21.15.105:0/1972453986 shutdown_connections 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c87087e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c87087bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 --2- 172.21.15.105:0/1972453986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c8708769f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 -- 172.21.15.105:0/1972453986 >> 172.21.15.105:0/1972453986 conn(0x55c870871b40 msgr2=0x55c870871f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 -- 172.21.15.105:0/1972453986 shutdown_connections 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 -- 172.21.15.105:0/1972453986 wait complete. 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 -- start start 2024-03-26T17:43:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.657+0000 7fba337c1740 1 Processor -- start 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba337c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8708765d0 0x55c870880450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba337c1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870880990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba337c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c87087c130 0x55c8708879d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba337c1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c87087f0e0 con 0x55c870876fc0 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba337c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c870796db0 con 0x55c87087c130 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba337c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c87087f260 con 0x55c8708765d0 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba337c1740 10 client.0 _ll_register_callbacks cb 0x55c87079b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba21d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870880990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba21d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870880990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36350/0 (socket says 172.21.15.105:36350) 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba21d9b640 1 -- 172.21.15.105:0/204433822 learned_addr learned my addr 172.21.15.105:0/204433822 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c87087c130 0x55c8708879d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba2a59c640 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8708765d0 0x55c870880450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba2ad9d640 1 -- 172.21.15.105:0/204433822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8708765d0 msgr2=0x55c870880450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8708765d0 0x55c870880450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba2ad9d640 1 -- 172.21.15.105:0/204433822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 msgr2=0x55c870880990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870880990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba2ad9d640 1 -- 172.21.15.105:0/204433822 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8704ef210 con 0x55c87087c130 2024-03-26T17:43:30.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba21d9b640 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870880990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.658+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c87087c130 0x55c8708879d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba24000f80 tx=0x7fba240113b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4b01bfaf97445e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba11ffb640 1 -- 172.21.15.105:0/204433822 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2402a060 con 0x55c87087c130 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba11ffb640 1 -- 172.21.15.105:0/204433822 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fba2400fdb0 con 0x55c87087c130 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 1 -- 172.21.15.105:0/204433822 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba24003e20 con 0x55c87087c130 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba11ffb640 1 -- 172.21.15.105:0/204433822 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fba24004270 con 0x55c87087c130 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 10 client.27209 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 1 -- 172.21.15.105:0/204433822 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8705bb620 con 0x55c87087c130 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba11ffb640 1 -- 172.21.15.105:0/204433822 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba24003e20 con 0x55c87087c130 2024-03-26T17:43:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba11ffb640 1 -- 172.21.15.105:0/204433822 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fba24018020 con 0x55c87087c130 2024-03-26T17:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 10 client.27209 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 10 client.27209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 20 client.27209 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c87088c100 0x55c87088e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 4 client.27209 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.659+0000 7fba337c1740 1 -- 172.21.15.105:0/204433822 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8707e85f0 con 0x55c87088c100 2024-03-26T17:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.660+0000 7fba2a59c640 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c87088c100 0x55c87088e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.660+0000 7fba2a59c640 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c87088c100 0x55c87088e4f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9107901ecb2d231 server_cookie=375ac118f10c584e in_seq=0 out_seq=0 2024-03-26T17:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.660+0000 7fba11ffb640 10 client.27209 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba11ffb640 1 -- 172.21.15.105:0/204433822 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c8707e85f0 con 0x55c87088c100 2024-03-26T17:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba11ffb640 10 client.27209 handle_command_reply: tid=0 2024-03-26T17:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 client.27209 shutdown 2024-03-26T17:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 -- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c87087c130 msgr2=0x55c8708879d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c87087c130 0x55c8708879d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba24000f80 tx=0x7fba240113b0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 -- 172.21.15.105:0/204433822 shutdown_connections 2024-03-26T17:43:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c87088c100 0x55c87088e4f0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c87087c130 0x55c8708879d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba11ffb640 0 client.27209 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870880990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 --2- 172.21.15.105:0/204433822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8708765d0 0x55c870880450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 -- 172.21.15.105:0/204433822 >> 172.21.15.105:0/204433822 conn(0x55c870871b40 msgr2=0x55c870870df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 -- 172.21.15.105:0/204433822 shutdown_connections 2024-03-26T17:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.663+0000 7fba337c1740 1 -- 172.21.15.105:0/204433822 wait complete. 2024-03-26T17:43:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.664+0000 7fba337c1740 20 client.27209 trim_cache size 0 max 16384 2024-03-26T17:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba337c1740 1 Processor -- start 2024-03-26T17:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba337c1740 1 -- start start 2024-03-26T17:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba337c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8708765d0 0x55c870870560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba337c1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870887990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.683 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba337c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c870870aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba337c1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1a8dd0 con 0x55c870876fc0 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba337c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1aa910 con 0x55c8708765d0 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba337c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1a94a0 con 0x55c87087c130 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba21d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870887990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba2a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8708765d0 0x55c870870560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba21d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870887990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36360/0 (socket says 172.21.15.105:36360) 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba21d9b640 1 -- 172.21.15.105:0/2972005108 learned_addr learned my addr 172.21.15.105:0/2972005108 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/2972005108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c870870aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba21d9b640 1 -- 172.21.15.105:0/2972005108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 msgr2=0x55c870870aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba21d9b640 1 --2- 172.21.15.105:0/2972005108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c870870aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba21d9b640 1 -- 172.21.15.105:0/2972005108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8708765d0 msgr2=0x55c870870560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba21d9b640 1 --2- 172.21.15.105:0/2972005108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8708765d0 0x55c870870560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.685+0000 7fba21d9b640 1 -- 172.21.15.105:0/2972005108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c87086d450 con 0x55c870876fc0 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.686+0000 7fba21d9b640 1 --2- 172.21.15.105:0/2972005108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870887990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fba14098800 tx=0x7fba1409ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a71f79cb6a7a9a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.686+0000 7fba13fff640 1 -- 172.21.15.105:0/2972005108 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14005680 con 0x55c870876fc0 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.686+0000 7fba13fff640 1 -- 172.21.15.105:0/2972005108 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba1409c580 con 0x55c870876fc0 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.686+0000 7fba13fff640 1 -- 172.21.15.105:0/2972005108 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140a4400 con 0x55c870876fc0 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 -- 172.21.15.105:0/2972005108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 msgr2=0x55c870887990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 --2- 172.21.15.105:0/2972005108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870887990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fba14098800 tx=0x7fba1409ba80 comp rx=0 tx=0).stop 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 -- 172.21.15.105:0/2972005108 shutdown_connections 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 --2- 172.21.15.105:0/2972005108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c870870aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 --2- 172.21.15.105:0/2972005108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c870876fc0 0x55c870887990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 --2- 172.21.15.105:0/2972005108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8708765d0 0x55c870870560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 -- 172.21.15.105:0/2972005108 >> 172.21.15.105:0/2972005108 conn(0x55c870871b40 msgr2=0x55c87087b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 -- 172.21.15.105:0/2972005108 shutdown_connections 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 -- 172.21.15.105:0/2972005108 wait complete. 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 -- start start 2024-03-26T17:43:30.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.687+0000 7fba337c1740 1 Processor -- start 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba337c1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c870884970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba337c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c870884eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba337c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c8708f03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c870884970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba337c1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1aa240 con 0x55c8708765d0 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba337c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1a9b70 con 0x55c870876fc0 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c870884970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36364/0 (socket says 172.21.15.105:36364) 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 -- 172.21.15.105:0/1085068314 learned_addr learned my addr 172.21.15.105:0/1085068314 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba21d9b640 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c870884eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c1a6db0 con 0x55c87087c130 2024-03-26T17:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba337c1740 10 client.0 _ll_register_callbacks cb 0x7fba14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2ad9d640 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c8708f03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 -- 172.21.15.105:0/1085068314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 msgr2=0x55c8708f03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c8708f03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 -- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 msgr2=0x55c870884eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c870884eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 -- 172.21.15.105:0/1085068314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c87088e860 con 0x55c8708765d0 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba21d9b640 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c870884eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba2a59c640 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c870884970 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009230 tx=0x7fba1c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4d9684c0760065 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba11ffb640 1 -- 172.21.15.105:0/1085068314 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0053d0 con 0x55c8708765d0 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba11ffb640 1 -- 172.21.15.105:0/1085068314 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fba1c005570 con 0x55c8708765d0 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.688+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1409c580 con 0x55c8708765d0 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba11ffb640 1 -- 172.21.15.105:0/1085068314 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fba1c0046c0 con 0x55c8708765d0 2024-03-26T17:43:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba337c1740 10 client.17388 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8708f1af0 con 0x55c8708765d0 2024-03-26T17:43:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba11ffb640 1 -- 172.21.15.105:0/1085068314 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba1409c580 con 0x55c8708765d0 2024-03-26T17:43:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba11ffb640 1 -- 172.21.15.105:0/1085068314 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fba1c003c10 con 0x55c8708765d0 2024-03-26T17:43:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba337c1740 10 client.17388 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba337c1740 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba337c1740 20 client.17388 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba337c1740 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c8708f24f0 0x55c8708f4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba337c1740 4 client.17388 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:43:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8707e2940 con 0x55c8708f24f0 2024-03-26T17:43:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.689+0000 7fba21d9b640 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c8708f24f0 0x55c8708f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.690+0000 7fba21d9b640 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c8708f24f0 0x55c8708f4910 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7acb639412c14afb server_cookie=b7975bde85c63353 in_seq=0 out_seq=0 2024-03-26T17:43:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.690+0000 7fba11ffb640 10 client.17388 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.731+0000 7fba11ffb640 1 -- 172.21.15.105:0/1085068314 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8707e2940 con 0x55c8708f24f0 2024-03-26T17:43:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.731+0000 7fba11ffb640 10 client.17388 handle_command_reply: tid=0 2024-03-26T17:43:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.731+0000 7fba337c1740 1 client.17388 shutdown 2024-03-26T17:43:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.731+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 msgr2=0x55c870884970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.731+0000 7fba337c1740 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c870884970 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009230 tx=0x7fba1c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 shutdown_connections 2024-03-26T17:43:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c8708f24f0 0x55c8708f4910 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c87087c130 0x55c8708f03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c870876fc0 0x55c870884eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 1 --2- 172.21.15.105:0/1085068314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8708765d0 0x55c870884970 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 >> 172.21.15.105:0/1085068314 conn(0x55c870871b40 msgr2=0x55c870874da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba11ffb640 0 client.17388 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 shutdown_connections 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 1 -- 172.21.15.105:0/1085068314 wait complete. 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba337c1740 20 client.17388 trim_cache size 0 max 16384 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:43:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:43:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:43:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba30bd4640 1 -- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2fc0 msgr2=0x7fba180a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba30bd4640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2fc0 0x7fba180a5480 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fba1c005ab0 tx=0x7fba1c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba30bd4640 1 -- 172.21.15.105:0/900295048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c1019d0 msgr2=0x7fba2c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.732+0000 7fba30bd4640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c1019d0 0x7fba2c198010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba1400e9f0 tx=0x7fba1400edf0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.733+0000 7fba30bd4640 1 -- 172.21.15.105:0/900295048 shutdown_connections 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.733+0000 7fba30bd4640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2fc0 0x7fba180a5480 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.733+0000 7fba30bd4640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba2c102390 0x7fba2c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.733+0000 7fba30bd4640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c1019d0 0x7fba2c198010 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.733+0000 7fba30bd4640 1 --2- 172.21.15.105:0/900295048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba2c1007e0 0x7fba2c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.733+0000 7fba30bd4640 1 -- 172.21.15.105:0/900295048 >> 172.21.15.105:0/900295048 conn(0x7fba2c0fbd50 msgr2=0x7fba2c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.733+0000 7fba30bd4640 1 -- 172.21.15.105:0/900295048 shutdown_connections 2024-03-26T17:43:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:30.733+0000 7fba30bd4640 1 -- 172.21.15.105:0/900295048 wait complete. 2024-03-26T17:43:31.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 149836 3.14 MB/sec execute 134 sec latency 40.371 ms 2024-03-26T17:43:32.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 150469 3.15 MB/sec execute 135 sec latency 16.208 ms 2024-03-26T17:43:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:31 smithi144 ceph-mon[36095]: pgmap v503: 97 pgs: 97 active+clean; 28 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:43:32.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:31 smithi105 ceph-mon[32280]: pgmap v503: 97 pgs: 97 active+clean; 28 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:43:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:31 smithi135 ceph-mon[35636]: pgmap v503: 97 pgs: 97 active+clean; 28 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:43:33.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 151140 3.14 MB/sec execute 136 sec latency 16.035 ms 2024-03-26T17:43:34.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 151764 3.15 MB/sec execute 137 sec latency 17.201 ms 2024-03-26T17:43:34.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:33 smithi144 ceph-mon[36095]: pgmap v504: 97 pgs: 97 active+clean; 28 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:43:34.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:33 smithi105 ceph-mon[32280]: pgmap v504: 97 pgs: 97 active+clean; 28 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:43:34.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:33 smithi135 ceph-mon[35636]: pgmap v504: 97 pgs: 97 active+clean; 28 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:43:35.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 152352 3.15 MB/sec execute 138 sec latency 23.277 ms 2024-03-26T17:43:36.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 152922 3.16 MB/sec execute 139 sec latency 17.497 ms 2024-03-26T17:43:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:35 smithi144 ceph-mon[36095]: pgmap v505: 97 pgs: 97 active+clean; 29 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:43:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:35 smithi105 ceph-mon[32280]: pgmap v505: 97 pgs: 97 active+clean; 29 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:43:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:35 smithi135 ceph-mon[35636]: pgmap v505: 97 pgs: 97 active+clean; 29 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:43:37.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 153355 3.15 MB/sec execute 140 sec latency 24.835 ms 2024-03-26T17:43:38.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 153986 3.16 MB/sec execute 141 sec latency 16.000 ms 2024-03-26T17:43:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:37 smithi144 ceph-mon[36095]: pgmap v506: 97 pgs: 97 active+clean; 25 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-26T17:43:38.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:37 smithi105 ceph-mon[32280]: pgmap v506: 97 pgs: 97 active+clean; 25 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-26T17:43:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:37 smithi135 ceph-mon[35636]: pgmap v506: 97 pgs: 97 active+clean; 25 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-26T17:43:39.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 154622 3.14 MB/sec execute 142 sec latency 19.845 ms 2024-03-26T17:43:40.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 155170 3.13 MB/sec execute 143 sec latency 17.925 ms 2024-03-26T17:43:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:39 smithi144 ceph-mon[36095]: pgmap v507: 97 pgs: 97 active+clean; 26 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-03-26T17:43:40.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:39 smithi105 ceph-mon[32280]: pgmap v507: 97 pgs: 97 active+clean; 26 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-03-26T17:43:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:39 smithi135 ceph-mon[35636]: pgmap v507: 97 pgs: 97 active+clean; 26 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-03-26T17:43:41.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 155877 3.15 MB/sec execute 144 sec latency 19.559 ms 2024-03-26T17:43:42.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 156448 3.17 MB/sec execute 145 sec latency 15.905 ms 2024-03-26T17:43:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:41 smithi144 ceph-mon[36095]: pgmap v508: 97 pgs: 97 active+clean; 27 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:43:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:41 smithi105 ceph-mon[32280]: pgmap v508: 97 pgs: 97 active+clean; 27 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:43:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:41 smithi135 ceph-mon[35636]: pgmap v508: 97 pgs: 97 active+clean; 27 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:43:43.023 INFO:tasks.workunit.client.0.smithi105.stdout: 1 156883 3.15 MB/sec execute 146 sec latency 27.576 ms 2024-03-26T17:43:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.833+0000 7f77a6b3b640 1 -- 172.21.15.105:0/1531083085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a01007e0 msgr2=0x7f77a0100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.833+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/1531083085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a01007e0 0x7f77a0100c00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7794009900 tx=0x7f779402bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 -- 172.21.15.105:0/1531083085 shutdown_connections 2024-03-26T17:43:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/1531083085 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a0102390 0x7f77a0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/1531083085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a01019d0 0x7f77a0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/1531083085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a01007e0 0x7f77a0100c00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 -- 172.21.15.105:0/1531083085 >> 172.21.15.105:0/1531083085 conn(0x7f77a00fbd50 msgr2=0x7f77a00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 -- 172.21.15.105:0/1531083085 shutdown_connections 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 -- 172.21.15.105:0/1531083085 wait complete. 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 Processor -- start 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.834+0000 7f77a6b3b640 1 -- start start 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a6b3b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a01007e0 0x7f77a0069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a6b3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a01019d0 0x7f77a019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a6b3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a0102390 0x7f77a019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a6b3b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77a010c300 con 0x7f77a0102390 2024-03-26T17:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a6b3b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77a010c180 con 0x7f77a01007e0 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a6b3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77a010c480 con 0x7f77a01019d0 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a50d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a0102390 0x7f77a019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a01007e0 0x7f77a0069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a01007e0 0x7f77a0069ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60316/0 (socket says 172.21.15.105:60316) 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 -- 172.21.15.105:0/413900041 learned_addr learned my addr 172.21.15.105:0/413900041 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f779ffff640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a01019d0 0x7f77a019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 -- 172.21.15.105:0/413900041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a01019d0 msgr2=0x7f77a019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a01019d0 0x7f77a019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 -- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a0102390 msgr2=0x7f77a019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a0102390 0x7f77a019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 -- 172.21.15.105:0/413900041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a019f780 con 0x7f77a01007e0 2024-03-26T17:43:43.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a50d8640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a0102390 0x7f77a019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.835+0000 7f77a48d7640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a01007e0 0x7f77a0069ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7794000c00 tx=0x7f7794005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.836+0000 7f779dffb640 1 -- 172.21.15.105:0/413900041 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794043020 con 0x7f77a01007e0 2024-03-26T17:43:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.836+0000 7f779dffb640 1 -- 172.21.15.105:0/413900041 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7794004350 con 0x7f77a01007e0 2024-03-26T17:43:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.836+0000 7f77a6b3b640 1 -- 172.21.15.105:0/413900041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a019f9b0 con 0x7f77a01007e0 2024-03-26T17:43:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.836+0000 7f779dffb640 1 -- 172.21.15.105:0/413900041 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794003b10 con 0x7f77a01007e0 2024-03-26T17:43:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.836+0000 7f77a6b3b640 1 -- 172.21.15.105:0/413900041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a019ff00 con 0x7f77a01007e0 2024-03-26T17:43:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.838+0000 7f779dffb640 1 -- 172.21.15.105:0/413900041 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7794046020 con 0x7f77a01007e0 2024-03-26T17:43:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a9807740 1 Processor -- start 2024-03-26T17:43:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a9807740 1 -- start start 2024-03-26T17:43:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f779dffb640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77900a2fc0 0x7f77900a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a9807740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652aad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a9807740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 0x55f652ab2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a9807740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652ab55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a9807740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f652a120b0 con 0x55f652ab3130 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a9807740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f6529ba960 con 0x55f652aad5d0 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a9807740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f652a0e060 con 0x55f652aadfc0 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a50d8640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77900a2fc0 0x7f77900a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652aad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f779dffb640 1 -- 172.21.15.105:0/413900041 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f77940f1530 con 0x7f77a01007e0 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652aad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60320/0 (socket says 172.21.15.105:60320) 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a48d7640 1 -- 172.21.15.105:0/2446391503 learned_addr learned my addr 172.21.15.105:0/2446391503 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f779ffff640 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 0x55f652ab2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.839+0000 7f77a50d8640 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652ab55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f77a48d7640 1 -- 172.21.15.105:0/2446391503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 msgr2=0x55f652ab2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f77a48d7640 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 0x55f652ab2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f77a48d7640 1 -- 172.21.15.105:0/2446391503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 msgr2=0x55f652ab55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f77a48d7640 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652ab55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f77a48d7640 1 -- 172.21.15.105:0/2446391503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6529ccc60 con 0x55f652aad5d0 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f77a48d7640 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652aad9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77940eeff0 tx=0x7f779402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44b314b77f5f309c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f77a50d8640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77900a2fc0 0x7f77900a5480 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f77980099a0 tx=0x7f7798013660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f778affd640 1 -- 172.21.15.105:0/2446391503 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77940be320 con 0x55f652aad5d0 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f778affd640 1 -- 172.21.15.105:0/2446391503 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77940bdc20 con 0x55f652aad5d0 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.840+0000 7f778affd640 1 -- 172.21.15.105:0/2446391503 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77940bbfb0 con 0x55f652aad5d0 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 -- 172.21.15.105:0/2446391503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 msgr2=0x55f652aad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652aad9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77940eeff0 tx=0x7f779402bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 -- 172.21.15.105:0/2446391503 shutdown_connections 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652ab55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 0x55f652ab2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 --2- 172.21.15.105:0/2446391503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652aad9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 -- 172.21.15.105:0/2446391503 >> 172.21.15.105:0/2446391503 conn(0x55f652aa8b40 msgr2=0x55f652aa8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 -- 172.21.15.105:0/2446391503 shutdown_connections 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 -- 172.21.15.105:0/2446391503 wait complete. 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.841+0000 7f77a9807740 1 -- start start 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a9807740 1 Processor -- start 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a9807740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652ab7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a9807740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652ab7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a9807740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652ab3130 0x55f652abe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a9807740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f652ab5ea0 con 0x55f652aadfc0 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a9807740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f6529cddb0 con 0x55f652aad5d0 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a9807740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f652ab6020 con 0x55f652ab3130 2024-03-26T17:43:43.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a9807740 10 client.0 _ll_register_callbacks cb 0x55f6529d2110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f779ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652ab7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652ab7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f779ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652ab7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52624/0 (socket says 172.21.15.105:52624) 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652ab7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60326/0 (socket says 172.21.15.105:60326) 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f779ffff640 1 -- 172.21.15.105:0/3673035122 learned_addr learned my addr 172.21.15.105:0/3673035122 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a50d8640 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652ab3130 0x55f652abe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a48d7640 1 -- 172.21.15.105:0/3673035122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652ab3130 msgr2=0x55f652abe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a48d7640 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652ab3130 0x55f652abe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a48d7640 1 -- 172.21.15.105:0/3673035122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 msgr2=0x55f652ab7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.842+0000 7f77a48d7640 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652ab7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a48d7640 1 -- 172.21.15.105:0/3673035122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f652726210 con 0x55f652aad5d0 2024-03-26T17:43:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f779ffff640 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652ab7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a50d8640 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652ab3130 0x55f652abe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a48d7640 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652ab7030 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55f652aae820 tx=0x7f77940c1df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37bca6d025e5a63e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f777ffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f777ffff640 1 -- 172.21.15.105:0/3673035122 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77940be320 con 0x55f652aad5d0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a9807740 1 -- 172.21.15.105:0/3673035122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77940bdc20 con 0x55f652aad5d0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f777ffff640 1 -- 172.21.15.105:0/3673035122 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f77940eb880 con 0x55f652aad5d0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f777ffff640 1 -- 172.21.15.105:0/3673035122 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77940eba80 con 0x55f652aad5d0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a9807740 10 client.27227 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a9807740 1 -- 172.21.15.105:0/3673035122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f6527f2620 con 0x55f652aad5d0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f777ffff640 1 -- 172.21.15.105:0/3673035122 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77940bdc20 con 0x55f652aad5d0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f777ffff640 1 -- 172.21.15.105:0/3673035122 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f77940f11d0 con 0x55f652aad5d0 2024-03-26T17:43:43.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a9807740 10 client.27227 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a9807740 10 client.27227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a9807740 20 client.27227 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.843+0000 7f77a9807740 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f652ac2c40 0x55f652ac5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.844+0000 7f77a9807740 4 client.27227 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.844+0000 7f77a9807740 1 -- 172.21.15.105:0/3673035122 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f652a1f5f0 con 0x55f652ac2c40 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.844+0000 7f779ffff640 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f652ac2c40 0x55f652ac5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.844+0000 7f779ffff640 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f652ac2c40 0x55f652ac5030 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa1cd4f3ed7d41c server_cookie=faab8831bc4ac7b7 in_seq=0 out_seq=0 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.844+0000 7f777ffff640 10 client.27227 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f777ffff640 1 -- 172.21.15.105:0/3673035122 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f652a1f5f0 con 0x55f652ac2c40 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f777ffff640 10 client.27227 handle_command_reply: tid=0 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 client.27227 shutdown 2024-03-26T17:43:43.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 -- 172.21.15.105:0/3673035122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 msgr2=0x55f652ab7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652ab7030 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55f652aae820 tx=0x7f77940c1df0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 -- 172.21.15.105:0/3673035122 shutdown_connections 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f652ac2c40 0x55f652ac5030 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652ab3130 0x55f652abe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652ab7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 --2- 172.21.15.105:0/3673035122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652ab7030 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 -- 172.21.15.105:0/3673035122 >> 172.21.15.105:0/3673035122 conn(0x55f652aa8b40 msgr2=0x55f652aa7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f777ffff640 0 client.27227 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 -- 172.21.15.105:0/3673035122 shutdown_connections 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.845+0000 7f77a9807740 1 -- 172.21.15.105:0/3673035122 wait complete. 2024-03-26T17:43:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.846+0000 7f77a9807740 20 client.27227 trim_cache size 0 max 16384 2024-03-26T17:43:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a9807740 1 Processor -- start 2024-03-26T17:43:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a9807740 1 -- start start 2024-03-26T17:43:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a9807740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aad5d0 0x55f652aaf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.857 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a9807740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652abe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a9807740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652ab3130 0x55f652ab6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a9807740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01a6ae0 con 0x55f652aadfc0 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a9807740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01a8620 con 0x55f652ab3130 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a9807740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01a71b0 con 0x55f652aad5d0 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f779ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652abe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aad5d0 0x55f652aaf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f779ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652abe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52640/0 (socket says 172.21.15.105:52640) 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aad5d0 0x55f652aaf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:50462/0 (socket says 172.21.15.105:50462) 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f779ffff640 1 -- 172.21.15.105:0/984413697 learned_addr learned my addr 172.21.15.105:0/984413697 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.863+0000 7f77a50d8640 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652ab3130 0x55f652ab6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.858 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f779ffff640 1 -- 172.21.15.105:0/984413697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aad5d0 msgr2=0x55f652aaf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f779ffff640 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aad5d0 0x55f652aaf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f779ffff640 1 -- 172.21.15.105:0/984413697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652ab3130 msgr2=0x55f652ab6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f779ffff640 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652ab3130 0x55f652ab6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f779ffff640 1 -- 172.21.15.105:0/984413697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f652aa98b0 con 0x55f652aadfc0 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f77a48d7640 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aad5d0 0x55f652aaf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f779ffff640 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652abe570 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f778c0129b0 tx=0x7f778c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7213e7b82955e072 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f778a7fc640 1 -- 172.21.15.105:0/984413697 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0122d0 con 0x55f652aadfc0 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f778a7fc640 1 -- 172.21.15.105:0/984413697 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f778c00fd70 con 0x55f652aadfc0 2024-03-26T17:43:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.864+0000 7f778a7fc640 1 -- 172.21.15.105:0/984413697 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c004a00 con 0x55f652aadfc0 2024-03-26T17:43:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 -- 172.21.15.105:0/984413697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 msgr2=0x55f652abe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652abe570 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f778c0129b0 tx=0x7f778c005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 -- 172.21.15.105:0/984413697 shutdown_connections 2024-03-26T17:43:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652ab3130 0x55f652ab6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652aadfc0 0x55f652abe570 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 --2- 172.21.15.105:0/984413697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aad5d0 0x55f652aaf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 -- 172.21.15.105:0/984413697 >> 172.21.15.105:0/984413697 conn(0x55f652aa8b40 msgr2=0x55f652aaea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 -- 172.21.15.105:0/984413697 shutdown_connections 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 -- 172.21.15.105:0/984413697 wait complete. 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.865+0000 7f77a9807740 1 -- start start 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a9807740 1 Processor -- start 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a9807740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652abb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a9807740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 0x55f652abbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a9807740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652abcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a9807740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01a7f50 con 0x55f652ab3130 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a9807740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01a7880 con 0x55f652aad5d0 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a9807740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01a4ac0 con 0x55f652aadfc0 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a9807740 10 client.0 _ll_register_callbacks cb 0x7f7794009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a50d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652abcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a48d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652abb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a50d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652abcbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52646/0 (socket says 172.21.15.105:52646) 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f779ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 0x55f652abbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a50d8640 1 -- 172.21.15.105:0/2970464396 learned_addr learned my addr 172.21.15.105:0/2970464396 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a50d8640 1 -- 172.21.15.105:0/2970464396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 msgr2=0x55f652abbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a50d8640 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 0x55f652abbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a50d8640 1 -- 172.21.15.105:0/2970464396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 msgr2=0x55f652abb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a50d8640 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652abb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a50d8640 1 -- 172.21.15.105:0/2970464396 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f652abd1d0 con 0x55f652ab3130 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.866+0000 7f77a48d7640 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652abb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a50d8640 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652abcbe0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7798010b40 tx=0x7f7798004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c060abe713f576f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f777ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f777ffff640 1 -- 172.21.15.105:0/2970464396 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798004b20 con 0x55f652ab3130 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f777ffff640 1 -- 172.21.15.105:0/2970464396 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f779801c040 con 0x55f652ab3130 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 1 -- 172.21.15.105:0/2970464396 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778c00fd70 con 0x55f652ab3130 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f777ffff640 1 -- 172.21.15.105:0/2970464396 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7798004cc0 con 0x55f652ab3130 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 10 client.17406 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 1 -- 172.21.15.105:0/2970464396 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f652abe550 con 0x55f652ab3130 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f777ffff640 1 -- 172.21.15.105:0/2970464396 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f778c00fd70 con 0x55f652ab3130 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f777ffff640 1 -- 172.21.15.105:0/2970464396 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7798020050 con 0x55f652ab3130 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 10 client.17406 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 20 client.17406 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f652b293d0 0x55f652b2b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 4 client.17406 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:43:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a9807740 1 -- 172.21.15.105:0/2970464396 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f652a19940 con 0x55f652b293d0 2024-03-26T17:43:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.867+0000 7f77a48d7640 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f652b293d0 0x55f652b2b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.868+0000 7f777ffff640 10 client.17406 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.868+0000 7f77a48d7640 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f652b293d0 0x55f652b2b7f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b9441bf36b72e3 server_cookie=68009aefdf770aa7 in_seq=0 out_seq=0 2024-03-26T17:43:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f777ffff640 1 -- 172.21.15.105:0/2970464396 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f652a19940 con 0x55f652b293d0 2024-03-26T17:43:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f777ffff640 10 client.17406 handle_command_reply: tid=0 2024-03-26T17:43:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 client.17406 shutdown 2024-03-26T17:43:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 -- 172.21.15.105:0/2970464396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 msgr2=0x55f652abcbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652abcbe0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7798010b40 tx=0x7f7798004ab0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 -- 172.21.15.105:0/2970464396 shutdown_connections 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f652b293d0 0x55f652b2b7f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f652ab3130 0x55f652abcbe0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f777ffff640 0 client.17406 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f652aadfc0 0x55f652abbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 --2- 172.21.15.105:0/2970464396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f652aad5d0 0x55f652abb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 -- 172.21.15.105:0/2970464396 >> 172.21.15.105:0/2970464396 conn(0x55f652aa8b40 msgr2=0x55f652ab6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 -- 172.21.15.105:0/2970464396 shutdown_connections 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.917+0000 7f77a9807740 1 -- 172.21.15.105:0/2970464396 wait complete. 2024-03-26T17:43:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a9807740 20 client.17406 trim_cache size 0 max 16384 2024-03-26T17:43:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:43:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:43:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:43:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 -- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77900a2fc0 msgr2=0x7f77900a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77900a2fc0 0x7f77900a5480 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f77980099a0 tx=0x7f7798013660 comp rx=0 tx=0).stop 2024-03-26T17:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 -- 172.21.15.105:0/413900041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a01007e0 msgr2=0x7f77a0069ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a01007e0 0x7f77a0069ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7794000c00 tx=0x7f7794005dc0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 -- 172.21.15.105:0/413900041 shutdown_connections 2024-03-26T17:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77900a2fc0 0x7f77900a5480 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a0102390 0x7f77a019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a01019d0 0x7f77a019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 --2- 172.21.15.105:0/413900041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a01007e0 0x7f77a0069ab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 -- 172.21.15.105:0/413900041 >> 172.21.15.105:0/413900041 conn(0x7f77a00fbd50 msgr2=0x7f77a0104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.918+0000 7f77a6b3b640 1 -- 172.21.15.105:0/413900041 shutdown_connections 2024-03-26T17:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:43.919+0000 7f77a6b3b640 1 -- 172.21.15.105:0/413900041 wait complete. 2024-03-26T17:43:44.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 157485 3.16 MB/sec execute 147 sec latency 42.912 ms 2024-03-26T17:43:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:43 smithi144 ceph-mon[36095]: pgmap v509: 97 pgs: 97 active+clean; 27 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:43:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:43 smithi105 ceph-mon[32280]: pgmap v509: 97 pgs: 97 active+clean; 27 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:43:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:43 smithi135 ceph-mon[35636]: pgmap v509: 97 pgs: 97 active+clean; 27 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:43:45.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 158132 3.14 MB/sec execute 148 sec latency 37.778 ms 2024-03-26T17:43:46.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 158689 3.13 MB/sec execute 149 sec latency 16.332 ms 2024-03-26T17:43:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:45 smithi144 ceph-mon[36095]: pgmap v510: 97 pgs: 97 active+clean; 28 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-03-26T17:43:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:45 smithi105 ceph-mon[32280]: pgmap v510: 97 pgs: 97 active+clean; 28 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-03-26T17:43:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:45 smithi135 ceph-mon[35636]: pgmap v510: 97 pgs: 97 active+clean; 28 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-03-26T17:43:47.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 159412 3.16 MB/sec execute 150 sec latency 15.546 ms 2024-03-26T17:43:48.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 159933 3.16 MB/sec execute 151 sec latency 16.292 ms 2024-03-26T17:43:48.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:47 smithi144 ceph-mon[36095]: pgmap v511: 97 pgs: 97 active+clean; 28 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-03-26T17:43:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:47 smithi105 ceph-mon[32280]: pgmap v511: 97 pgs: 97 active+clean; 28 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-03-26T17:43:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:47 smithi135 ceph-mon[35636]: pgmap v511: 97 pgs: 97 active+clean; 28 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-03-26T17:43:49.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 160332 3.15 MB/sec execute 152 sec latency 24.430 ms 2024-03-26T17:43:50.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 160919 3.14 MB/sec execute 153 sec latency 17.004 ms 2024-03-26T17:43:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:49 smithi144 ceph-mon[36095]: pgmap v512: 97 pgs: 97 active+clean; 23 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:43:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:49 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:43:50.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:49 smithi105 ceph-mon[32280]: pgmap v512: 97 pgs: 97 active+clean; 23 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:43:50.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:49 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:43:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:49 smithi135 ceph-mon[35636]: pgmap v512: 97 pgs: 97 active+clean; 23 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:43:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:49 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:43:51.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 161606 3.14 MB/sec execute 154 sec latency 21.554 ms 2024-03-26T17:43:51.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:43:51.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:43:51.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:43:51.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:43:51.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:43:51.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:43:51.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:43:51.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:43:51.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:43:52.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 162199 3.14 MB/sec execute 155 sec latency 23.688 ms 2024-03-26T17:43:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:51 smithi144 ceph-mon[36095]: pgmap v513: 97 pgs: 97 active+clean; 25 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:43:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:51 smithi105 ceph-mon[32280]: pgmap v513: 97 pgs: 97 active+clean; 25 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:43:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:51 smithi135 ceph-mon[35636]: pgmap v513: 97 pgs: 97 active+clean; 25 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:43:53.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 162949 3.16 MB/sec execute 156 sec latency 15.940 ms 2024-03-26T17:43:54.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 163446 3.17 MB/sec execute 157 sec latency 15.025 ms 2024-03-26T17:43:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:53 smithi144 ceph-mon[36095]: pgmap v514: 97 pgs: 97 active+clean; 26 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:43:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:53 smithi105 ceph-mon[32280]: pgmap v514: 97 pgs: 97 active+clean; 26 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:43:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:53 smithi135 ceph-mon[35636]: pgmap v514: 97 pgs: 97 active+clean; 26 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:43:55.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 163810 3.15 MB/sec execute 158 sec latency 24.771 ms 2024-03-26T17:43:55.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:43:55.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:43:56.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 164394 3.14 MB/sec execute 159 sec latency 14.679 ms 2024-03-26T17:43:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2909794064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 msgr2=0x7f68f41047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2909794064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 0x7f68f41047d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009900 tx=0x7f68f002bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2909794064 shutdown_connections 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2909794064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f4107640 0x7f68f4109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2909794064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68f4104d10 0x7f68f4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2909794064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 0x7f68f41047d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2909794064 >> 172.21.15.105:0/2909794064 conn(0x7f68f40fbd50 msgr2=0x7f68f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2909794064 shutdown_connections 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.044+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2909794064 wait complete. 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fc7b9640 1 Processor -- start 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fc7b9640 1 -- start start 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fc7b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 0x7f68f4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fc7b9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68f4104d10 0x7f68f419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fc7b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f4107640 0x7f68f41a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fc7b9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68f410d450 con 0x7f68f41023e0 2024-03-26T17:43:56.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fc7b9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68f410d2d0 con 0x7f68f4104d10 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fc7b9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68f410d5d0 con 0x7f68f4107640 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fa555640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 0x7f68f4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fa555640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 0x7f68f4199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50894/0 (socket says 172.21.15.105:50894) 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fa555640 1 -- 172.21.15.105:0/2491574915 learned_addr learned my addr 172.21.15.105:0/2491574915 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68f9d54640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68f4104d10 0x7f68f419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fad56640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f4107640 0x7f68f41a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fa555640 1 -- 172.21.15.105:0/2491574915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f4107640 msgr2=0x7f68f41a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fa555640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f4107640 0x7f68f41a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fa555640 1 -- 172.21.15.105:0/2491574915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68f4104d10 msgr2=0x7f68f419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fa555640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68f4104d10 0x7f68f419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68fa555640 1 -- 172.21.15.105:0/2491574915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f41a19d0 con 0x7f68f41023e0 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.045+0000 7f68f9d54640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68f4104d10 0x7f68f419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.046+0000 7f68fa555640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 0x7f68f4199dc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f68f0000c00 tx=0x7f68f00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.046+0000 7f68eb7fe640 1 -- 172.21.15.105:0/2491574915 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0043020 con 0x7f68f41023e0 2024-03-26T17:43:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.046+0000 7f68eb7fe640 1 -- 172.21.15.105:0/2491574915 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68f00044c0 con 0x7f68f41023e0 2024-03-26T17:43:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.046+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2491574915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f41a1c60 con 0x7f68f41023e0 2024-03-26T17:43:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.046+0000 7f68eb7fe640 1 -- 172.21.15.105:0/2491574915 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0003c60 con 0x7f68f41023e0 2024-03-26T17:43:56.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.046+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2491574915 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f41a21b0 con 0x7f68f41023e0 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.048+0000 7f68eb7fe640 1 -- 172.21.15.105:0/2491574915 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f68f0046020 con 0x7f68f41023e0 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68eb7fe640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f68e00a2f20 0x7f68e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68f9d54640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f68e00a2f20 0x7f68e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68eb7fe640 1 -- 172.21.15.105:0/2491574915 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f68f00f16d0 con 0x7f68f41023e0 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68ff3a7740 1 Processor -- start 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68ff3a7740 1 -- start start 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad777d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad77cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad77f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5610ad6dc980 con 0x5610ad7778f0 2024-03-26T17:43:56.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610ad684be0 con 0x5610ad77d450 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610ad6d8930 con 0x5610ad7782e0 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68f9d54640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f68e00a2f20 0x7f68e00a53e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68f40694e0 tx=0x7f68e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68fa555640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad777d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68fa555640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad777d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50898/0 (socket says 172.21.15.105:50898) 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.049+0000 7f68fa555640 1 -- 172.21.15.105:0/1650622032 learned_addr learned my addr 172.21.15.105:0/1650622032 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68f9d54640 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad77cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68fad56640 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad77f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68fa555640 1 -- 172.21.15.105:0/1650622032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 msgr2=0x5610ad77cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68fa555640 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad77cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68fa555640 1 -- 172.21.15.105:0/1650622032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 msgr2=0x5610ad77f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68fa555640 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad77f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68fa555640 1 -- 172.21.15.105:0/1650622032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ad4ba7d0 con 0x5610ad7778f0 2024-03-26T17:43:56.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68f9d54640 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad77cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68fa555640 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad777d10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f68f00ef190 tx=0x7f68f002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadd89dfe19b4e36 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68e8ff9640 1 -- 172.21.15.105:0/1650622032 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f00035f0 con 0x5610ad7778f0 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68e8ff9640 1 -- 172.21.15.105:0/1650622032 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68f00be4c0 con 0x5610ad7778f0 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.050+0000 7f68e8ff9640 1 -- 172.21.15.105:0/1650622032 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f00bc170 con 0x5610ad7778f0 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 -- 172.21.15.105:0/1650622032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 msgr2=0x5610ad777d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad777d10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f68f00ef190 tx=0x7f68f002bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 -- 172.21.15.105:0/1650622032 shutdown_connections 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad77f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad77cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/1650622032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad777d10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 -- 172.21.15.105:0/1650622032 >> 172.21.15.105:0/1650622032 conn(0x5610ad772e60 msgr2=0x5610ad773260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 -- 172.21.15.105:0/1650622032 shutdown_connections 2024-03-26T17:43:56.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.051+0000 7f68ff3a7740 1 -- 172.21.15.105:0/1650622032 wait complete. 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 1 -- start start 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 1 Processor -- start 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7778f0 0x5610ad781860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7782e0 0x5610ad781da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad788de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5610ad7801c0 con 0x5610ad7782e0 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610ad698160 con 0x5610ad77d450 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610ad780340 con 0x5610ad7778f0 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68ff3a7740 10 client.0 _ll_register_callbacks cb 0x5610ad6d4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68f9d54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7782e0 0x5610ad781da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68fa555640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7778f0 0x5610ad781860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68f9d54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7782e0 0x5610ad781da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50900/0 (socket says 172.21.15.105:50900) 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68fa555640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7778f0 0x5610ad781860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33718/0 (socket says 172.21.15.105:33718) 2024-03-26T17:43:56.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68fad56640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad788de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68f9d54640 1 -- 172.21.15.105:0/4154879876 learned_addr learned my addr 172.21.15.105:0/4154879876 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68f9d54640 1 -- 172.21.15.105:0/4154879876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7778f0 msgr2=0x5610ad781860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68f9d54640 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7778f0 0x5610ad781860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68f9d54640 1 -- 172.21.15.105:0/4154879876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 msgr2=0x5610ad788de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68f9d54640 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad788de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68f9d54640 1 -- 172.21.15.105:0/4154879876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ad3f03f0 con 0x5610ad7782e0 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68fa555640 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7778f0 0x5610ad781860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.052+0000 7f68fad56640 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad788de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68f9d54640 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7782e0 0x5610ad781da0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f68e4003760 tx=0x7f68e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc18fc03c863004 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ddffb640 1 -- 172.21.15.105:0/4154879876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e402b060 con 0x5610ad7782e0 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 1 -- 172.21.15.105:0/4154879876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f00be4c0 con 0x5610ad7782e0 2024-03-26T17:43:56.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ddffb640 1 -- 172.21.15.105:0/4154879876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f68e400f910 con 0x5610ad7782e0 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ddffb640 1 -- 172.21.15.105:0/4154879876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68e400fb10 con 0x5610ad7782e0 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 10 client.17418 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 1 -- 172.21.15.105:0/4154879876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5610ad3b00f0 con 0x5610ad7782e0 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ddffb640 1 -- 172.21.15.105:0/4154879876 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68f00be4c0 con 0x5610ad7782e0 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ddffb640 1 -- 172.21.15.105:0/4154879876 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f68e401a050 con 0x5610ad7782e0 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 10 client.17418 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 20 client.17418 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610ad78d760 0x5610ad78fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 4 client.17418 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.053+0000 7f68ff3a7740 1 -- 172.21.15.105:0/4154879876 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610ad6e9ec0 con 0x5610ad78d760 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.054+0000 7f68fa555640 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610ad78d760 0x5610ad78fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.054+0000 7f68ddffb640 10 client.17418 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:56.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.054+0000 7f68fa555640 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610ad78d760 0x5610ad78fc20 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d00f9d14249cf0 server_cookie=e6cc6483ce881763 in_seq=0 out_seq=0 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ddffb640 1 -- 172.21.15.105:0/4154879876 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5610ad6e9ec0 con 0x5610ad78d760 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ddffb640 10 client.17418 handle_command_reply: tid=0 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 client.17418 shutdown 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 -- 172.21.15.105:0/4154879876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7782e0 msgr2=0x5610ad781da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7782e0 0x5610ad781da0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f68e4003760 tx=0x7f68e4004730 comp rx=0 tx=0).stop 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 -- 172.21.15.105:0/4154879876 shutdown_connections 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610ad78d760 0x5610ad78fc20 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad788de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7782e0 0x5610ad781da0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/4154879876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7778f0 0x5610ad781860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 -- 172.21.15.105:0/4154879876 >> 172.21.15.105:0/4154879876 conn(0x5610ad772e60 msgr2=0x5610ad772080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ddffb640 0 client.17418 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 -- 172.21.15.105:0/4154879876 shutdown_connections 2024-03-26T17:43:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 1 -- 172.21.15.105:0/4154879876 wait complete. 2024-03-26T17:43:56.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.055+0000 7f68ff3a7740 20 client.17418 trim_cache size 0 max 16384 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68ff3a7740 1 Processor -- start 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68ff3a7740 1 -- start start 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad772e60 0x5610ad7717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad788da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad7782e0 0x5610ad771d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41aaf80 con 0x5610ad7778f0 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41acac0 con 0x5610ad7782e0 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41ab650 con 0x5610ad772e60 2024-03-26T17:43:56.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68f9d54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad788da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68f9d54640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad788da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50916/0 (socket says 172.21.15.105:50916) 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68f9d54640 1 -- 172.21.15.105:0/2260355110 learned_addr learned my addr 172.21.15.105:0/2260355110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68fad56640 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad7782e0 0x5610ad771d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.075+0000 7f68fa555640 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad772e60 0x5610ad7717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68f9d54640 1 -- 172.21.15.105:0/2260355110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad772e60 msgr2=0x5610ad7717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68f9d54640 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad772e60 0x5610ad7717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68f9d54640 1 -- 172.21.15.105:0/2260355110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad7782e0 msgr2=0x5610ad771d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68f9d54640 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad7782e0 0x5610ad771d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68f9d54640 1 -- 172.21.15.105:0/2260355110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ad7786c0 con 0x5610ad7778f0 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68f9d54640 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad788da0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009730 tx=0x7f68e400faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a540153f8f7b11f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68dffff640 1 -- 172.21.15.105:0/2260355110 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e402b040 con 0x5610ad7778f0 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68dffff640 1 -- 172.21.15.105:0/2260355110 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68e400fcb0 con 0x5610ad7778f0 2024-03-26T17:43:56.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.076+0000 7f68dffff640 1 -- 172.21.15.105:0/2260355110 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4026ba0 con 0x5610ad7778f0 2024-03-26T17:43:56.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- 172.21.15.105:0/2260355110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 msgr2=0x5610ad788da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad788da0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009730 tx=0x7f68e400faa0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- 172.21.15.105:0/2260355110 shutdown_connections 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad7782e0 0x5610ad771d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad7778f0 0x5610ad788da0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/2260355110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad772e60 0x5610ad7717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- 172.21.15.105:0/2260355110 >> 172.21.15.105:0/2260355110 conn(0x5610ad77ce80 msgr2=0x5610ad781860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- 172.21.15.105:0/2260355110 shutdown_connections 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- 172.21.15.105:0/2260355110 wait complete. 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- start start 2024-03-26T17:43:56.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 Processor -- start 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad772e60 0x5610ad786830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad786d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad787ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41ac3f0 con 0x5610ad772e60 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41abd20 con 0x5610ad77d450 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41a8f60 con 0x5610ad7782e0 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68ff3a7740 10 client.0 _ll_register_callbacks cb 0x7f68e40040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68fa555640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad772e60 0x5610ad786830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68fa555640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad772e60 0x5610ad786830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50922/0 (socket says 172.21.15.105:50922) 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68fa555640 1 -- 172.21.15.105:0/987080980 learned_addr learned my addr 172.21.15.105:0/987080980 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68fad56640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad787ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.077+0000 7f68f9d54640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad786d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68fa555640 1 -- 172.21.15.105:0/987080980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 msgr2=0x5610ad786d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68fa555640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad786d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68fa555640 1 -- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 msgr2=0x5610ad787ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68fa555640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad787ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68fa555640 1 -- 172.21.15.105:0/987080980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ad7882d0 con 0x5610ad772e60 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68f9d54640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad786d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68fad56640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad787ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68fa555640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad772e60 0x5610ad786830 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f68f00c62a0 tx=0x7f68f00bd720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d415ce9e770f94d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68ddffb640 1 -- 172.21.15.105:0/987080980 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f00a0020 con 0x5610ad772e60 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68ddffb640 1 -- 172.21.15.105:0/987080980 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f68f00bc0c0 con 0x5610ad772e60 2024-03-26T17:43:56.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68ff3a7740 1 -- 172.21.15.105:0/987080980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68e400fcb0 con 0x5610ad772e60 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68ddffb640 1 -- 172.21.15.105:0/987080980 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68f00cb050 con 0x5610ad772e60 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68ff3a7740 10 client.17424 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.078+0000 7f68ff3a7740 1 -- 172.21.15.105:0/987080980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5610ad7f3e10 con 0x5610ad772e60 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ddffb640 1 -- 172.21.15.105:0/987080980 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68e400fcb0 con 0x5610ad772e60 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ddffb640 1 -- 172.21.15.105:0/987080980 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f68f00c6340 con 0x5610ad772e60 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ff3a7740 10 client.17424 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ff3a7740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ff3a7740 20 client.17424 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610ad7f4670 0x5610ad7f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ff3a7740 4 client.17424 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ff3a7740 1 -- 172.21.15.105:0/987080980 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610ad6e4210 con 0x5610ad7f4670 2024-03-26T17:43:56.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68f9d54640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610ad7f4670 0x5610ad7f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68ddffb640 10 client.17424 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.079+0000 7f68f9d54640 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610ad7f4670 0x5610ad7f6a90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b0ea2754f5d73df server_cookie=6320bf67e96af857 in_seq=0 out_seq=0 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ddffb640 1 -- 172.21.15.105:0/987080980 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5610ad6e4210 con 0x5610ad7f4670 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ddffb640 10 client.17424 handle_command_reply: tid=0 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 client.17424 shutdown 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 -- 172.21.15.105:0/987080980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad772e60 msgr2=0x5610ad786830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad772e60 0x5610ad786830 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f68f00c62a0 tx=0x7f68f00bd720 comp rx=0 tx=0).stop 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 -- 172.21.15.105:0/987080980 shutdown_connections 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610ad7f4670 0x5610ad7f6a90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610ad77d450 0x5610ad787ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610ad7782e0 0x5610ad786d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ddffb640 0 client.17424 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 --2- 172.21.15.105:0/987080980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610ad772e60 0x5610ad786830 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 -- 172.21.15.105:0/987080980 >> 172.21.15.105:0/987080980 conn(0x5610ad77ce80 msgr2=0x5610ad7760c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 -- 172.21.15.105:0/987080980 shutdown_connections 2024-03-26T17:43:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 1 -- 172.21.15.105:0/987080980 wait complete. 2024-03-26T17:43:56.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68ff3a7740 20 client.17424 trim_cache size 0 max 16384 2024-03-26T17:43:56.080 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:43:56.080 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:43:56.080 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:43:56.081 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:43:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f68e00a2f20 msgr2=0x7f68e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f68e00a2f20 0x7f68e00a53e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68f40694e0 tx=0x7f68e4005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 msgr2=0x7f68f4199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.080+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 0x7f68f4199dc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f68f0000c00 tx=0x7f68f00040c0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.081+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2491574915 shutdown_connections 2024-03-26T17:43:56.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.081+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f68e00a2f20 0x7f68e00a53e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.081+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68f4107640 0x7f68f41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.081+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f68f4104d10 0x7f68f419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.081+0000 7f68fc7b9640 1 --2- 172.21.15.105:0/2491574915 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68f41023e0 0x7f68f4199dc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.081+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2491574915 >> 172.21.15.105:0/2491574915 conn(0x7f68f40fbd50 msgr2=0x7f68f4101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.081+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2491574915 shutdown_connections 2024-03-26T17:43:56.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.081+0000 7f68fc7b9640 1 -- 172.21.15.105:0/2491574915 wait complete. 2024-03-26T17:43:56.084 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:55 smithi105 ceph-mon[32280]: pgmap v515: 97 pgs: 97 active+clean; 26 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:43:56.085 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:43:56.086 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:43:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:55 smithi144 ceph-mon[36095]: pgmap v515: 97 pgs: 97 active+clean; 26 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:43:56.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:55 smithi135 ceph-mon[35636]: pgmap v515: 97 pgs: 97 active+clean; 26 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:43:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 -- 172.21.15.105:0/856742178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 msgr2=0x7f052c1047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 --2- 172.21.15.105:0/856742178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 0x7f052c1047f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f051c009900 tx=0x7f051c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 -- 172.21.15.105:0/856742178 shutdown_connections 2024-03-26T17:43:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 --2- 172.21.15.105:0/856742178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052c107660 0x7f052c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 --2- 172.21.15.105:0/856742178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052c104d30 0x7f052c107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 --2- 172.21.15.105:0/856742178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 0x7f052c1047f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 -- 172.21.15.105:0/856742178 >> 172.21.15.105:0/856742178 conn(0x7f052c0fbd50 msgr2=0x7f052c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 -- 172.21.15.105:0/856742178 shutdown_connections 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.207+0000 7f05324c6640 1 -- 172.21.15.105:0/856742178 wait complete. 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.208+0000 7f05324c6640 1 Processor -- start 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.208+0000 7f05324c6640 1 -- start start 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.208+0000 7f05324c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 0x7f052c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.208+0000 7f05324c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052c104d30 0x7f052c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.208+0000 7f05324c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052c107660 0x7f052c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.208+0000 7f05324c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f052c10d420 con 0x7f052c102400 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.208+0000 7f05324c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f052c10d2a0 con 0x7f052c104d30 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.208+0000 7f05324c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f052c10d5a0 con 0x7f052c107660 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 0x7f052c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 0x7f052c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50930/0 (socket says 172.21.15.105:50930) 2024-03-26T17:43:56.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052bfff640 1 -- 172.21.15.105:0/1910640440 learned_addr learned my addr 172.21.15.105:0/1910640440 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052b7fe640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052c104d30 0x7f052c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f0530a63640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052c107660 0x7f052c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052b7fe640 1 -- 172.21.15.105:0/1910640440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052c107660 msgr2=0x7f052c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052b7fe640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052c107660 0x7f052c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052b7fe640 1 -- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 msgr2=0x7f052c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052b7fe640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 0x7f052c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052b7fe640 1 -- 172.21.15.105:0/1910640440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052c19f770 con 0x7f052c104d30 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f052b7fe640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052c104d30 0x7f052c198090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f052000e9c0 tx=0x7f052000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f05297fa640 1 -- 172.21.15.105:0/1910640440 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052000ccc0 con 0x7f052c104d30 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052c19fa60 con 0x7f052c104d30 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052c1a0000 con 0x7f052c104d30 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f05297fa640 1 -- 172.21.15.105:0/1910640440 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05200108e0 con 0x7f052c104d30 2024-03-26T17:43:56.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.209+0000 7f05297fa640 1 -- 172.21.15.105:0/1910640440 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520018660 con 0x7f052c104d30 2024-03-26T17:43:56.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.210+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05180052c0 con 0x7f052c104d30 2024-03-26T17:43:56.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.211+0000 7f05297fa640 1 -- 172.21.15.105:0/1910640440 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0520010430 con 0x7f052c104d30 2024-03-26T17:43:56.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.212+0000 7f05297fa640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05140a2fa0 0x7f05140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.212+0000 7f052bfff640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05140a2fa0 0x7f05140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.212+0000 7f05297fa640 1 -- 172.21.15.105:0/1910640440 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f05200cf5b0 con 0x7f052c104d30 2024-03-26T17:43:56.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.213+0000 7f052bfff640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05140a2fa0 0x7f05140a5460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f051c0096c0 tx=0x7f051c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.217+0000 7f05297fa640 1 -- 172.21.15.105:0/1910640440 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f052009bd50 con 0x7f052c104d30 2024-03-26T17:43:56.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.513+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05180055b0 con 0x7f052c104d30 2024-03-26T17:43:56.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.514+0000 7f05297fa640 1 -- 172.21.15.105:0/1910640440 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f052009fdf0 con 0x7f052c104d30 2024-03-26T17:43:56.506 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:43:56.506 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:43:56.506 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:43:56.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05140a2fa0 msgr2=0x7f05140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05140a2fa0 0x7f05140a5460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f051c0096c0 tx=0x7f051c02c7e0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052c104d30 msgr2=0x7f052c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052c104d30 0x7f052c198090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f052000e9c0 tx=0x7f052000ee80 comp rx=0 tx=0).stop 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 shutdown_connections 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05140a2fa0 0x7f05140a5460 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052c107660 0x7f052c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f052c104d30 0x7f052c198090 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 --2- 172.21.15.105:0/1910640440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f052c102400 0x7f052c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 >> 172.21.15.105:0/1910640440 conn(0x7f052c0fbd50 msgr2=0x7f052c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 shutdown_connections 2024-03-26T17:43:56.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.517+0000 7f05324c6640 1 -- 172.21.15.105:0/1910640440 wait complete. 2024-03-26T17:43:56.520 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:43:56.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 -- 172.21.15.105:0/2425659555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f01023d0 msgr2=0x7fb7f01047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/2425659555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f01023d0 0x7fb7f01047c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009900 tx=0x7fb7ec02bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 -- 172.21.15.105:0/2425659555 shutdown_connections 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/2425659555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f0107630 0x7fb7f0109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/2425659555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7f0104d00 0x7fb7f01070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/2425659555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f01023d0 0x7fb7f01047c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 -- 172.21.15.105:0/2425659555 >> 172.21.15.105:0/2425659555 conn(0x7fb7f00fbd60 msgr2=0x7fb7f00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 -- 172.21.15.105:0/2425659555 shutdown_connections 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.640+0000 7fb7f83b3640 1 -- 172.21.15.105:0/2425659555 wait complete. 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f83b3640 1 Processor -- start 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f83b3640 1 -- start start 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f83b3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f01023d0 0x7fb7f0101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f83b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f0104d00 0x7fb7f0100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f83b3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7f0107630 0x7fb7f01008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f83b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010d3f0 con 0x7fb7f0104d00 2024-03-26T17:43:56.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f83b3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010d270 con 0x7fb7f0107630 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f83b3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f010d570 con 0x7fb7f01023d0 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f594e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f0104d00 0x7fb7f0100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f594e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f0104d00 0x7fb7f0100380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50940/0 (socket says 172.21.15.105:50940) 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f594e640 1 -- 172.21.15.105:0/3272705074 learned_addr learned my addr 172.21.15.105:0/3272705074 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f6950640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7f0107630 0x7fb7f01008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.641+0000 7fb7f614f640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f01023d0 0x7fb7f0101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7f594e640 1 -- 172.21.15.105:0/3272705074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f01023d0 msgr2=0x7fb7f0101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7f594e640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f01023d0 0x7fb7f0101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7f594e640 1 -- 172.21.15.105:0/3272705074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7f0107630 msgr2=0x7fb7f01008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7f594e640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7f0107630 0x7fb7f01008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7f594e640 1 -- 172.21.15.105:0/3272705074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f0101150 con 0x7fb7f0104d00 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7f594e640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f0104d00 0x7fb7f0100380 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e9c0 tx=0x7fb7e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7e77fe640 1 -- 172.21.15.105:0/3272705074 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000cd80 con 0x7fb7f0104d00 2024-03-26T17:43:56.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7f83b3640 1 -- 172.21.15.105:0/3272705074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f01a1c70 con 0x7fb7f0104d00 2024-03-26T17:43:56.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7e77fe640 1 -- 172.21.15.105:0/3272705074 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7e00054d0 con 0x7fb7f0104d00 2024-03-26T17:43:56.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7e77fe640 1 -- 172.21.15.105:0/3272705074 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0010430 con 0x7fb7f0104d00 2024-03-26T17:43:56.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.642+0000 7fb7f83b3640 1 -- 172.21.15.105:0/3272705074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f01a21c0 con 0x7fb7f0104d00 2024-03-26T17:43:56.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.644+0000 7fb7e77fe640 1 -- 172.21.15.105:0/3272705074 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb7e0004fb0 con 0x7fb7f0104d00 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.645+0000 7fb7e77fe640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb7dc0a2f20 0x7fb7dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.645+0000 7fb7f614f640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb7dc0a2f20 0x7fb7dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.645+0000 7fb7e77fe640 1 -- 172.21.15.105:0/3272705074 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb7e00cf8b0 con 0x7fb7f0104d00 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7faf9a740 1 Processor -- start 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7faf9a740 1 -- start start 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640336a30 0x559640336e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f614f640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb7dc0a2f20 0x7fb7dc0a53e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009230 tx=0x7fb7ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x55964033bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55964033c590 0x55964033ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55964029b970 con 0x55964033c590 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559640243d70 con 0x559640336a30 2024-03-26T17:43:56.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559640297920 con 0x559640337420 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f594e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x55964033bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f614f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640336a30 0x559640336e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f594e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x55964033bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33806/0 (socket says 172.21.15.105:33806) 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f594e640 1 -- 172.21.15.105:0/1633823920 learned_addr learned my addr 172.21.15.105:0/1633823920 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f6950640 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55964033c590 0x55964033ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f594e640 1 -- 172.21.15.105:0/1633823920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640336a30 msgr2=0x559640336e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f594e640 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640336a30 0x559640336e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f594e640 1 -- 172.21.15.105:0/1633823920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55964033c590 msgr2=0x55964033ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f594e640 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55964033c590 0x55964033ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.646+0000 7fb7f594e640 1 -- 172.21.15.105:0/1633823920 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55963ffaf3f0 con 0x559640337420 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.647+0000 7fb7f6950640 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55964033c590 0x55964033ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.647+0000 7fb7f594e640 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x55964033bf20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00a4480 tx=0x7fb7e009be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6738ecd1f921ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.647+0000 7fb7e4bfe640 1 -- 172.21.15.105:0/1633823920 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e009bfa0 con 0x559640337420 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.647+0000 7fb7e4bfe640 1 -- 172.21.15.105:0/1633823920 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7e00a00f0 con 0x559640337420 2024-03-26T17:43:56.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.647+0000 7fb7e4bfe640 1 -- 172.21.15.105:0/1633823920 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e009a380 con 0x559640337420 2024-03-26T17:43:56.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1633823920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 msgr2=0x55964033bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x55964033bf20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00a4480 tx=0x7fb7e009be30 comp rx=0 tx=0).stop 2024-03-26T17:43:56.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1633823920 shutdown_connections 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55964033c590 0x55964033ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x55964033bf20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1633823920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640336a30 0x559640336e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1633823920 >> 172.21.15.105:0/1633823920 conn(0x559640331fa0 msgr2=0x5596403323a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1633823920 shutdown_connections 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1633823920 wait complete. 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 -- start start 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.648+0000 7fb7faf9a740 1 Processor -- start 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640336a30 0x559640341000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640337420 0x559640341540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640348580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55964033f540 con 0x559640337420 2024-03-26T17:43:56.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559640257310 con 0x55964033c590 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55964033f6c0 con 0x559640336a30 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7faf9a740 10 client.0 _ll_register_callbacks cb 0x559640292c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f594e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640337420 0x559640341540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f614f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640336a30 0x559640341000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f594e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640337420 0x559640341540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50956/0 (socket says 172.21.15.105:50956) 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f594e640 1 -- 172.21.15.105:0/461142546 learned_addr learned my addr 172.21.15.105:0/461142546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f614f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640336a30 0x559640341000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33818/0 (socket says 172.21.15.105:33818) 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f6950640 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640348580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f614f640 1 -- 172.21.15.105:0/461142546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 msgr2=0x559640348580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f614f640 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640348580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f614f640 1 -- 172.21.15.105:0/461142546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640337420 msgr2=0x559640341540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f614f640 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640337420 0x559640341540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f614f640 1 -- 172.21.15.105:0/461142546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596400797d0 con 0x559640336a30 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f594e640 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640337420 0x559640341540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7f614f640 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640336a30 0x559640341000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000c00 tx=0x7fb7ec009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f661fbd35c43bac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.649+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/461142546 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec050030 con 0x559640336a30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/461142546 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb7ec03a040 con 0x559640336a30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 1 -- 172.21.15.105:0/461142546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e00a00f0 con 0x559640336a30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/461142546 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec0048d0 con 0x559640336a30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 10 client.27277 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 1 -- 172.21.15.105:0/461142546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55964034b960 con 0x559640336a30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/461142546 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7e00a00f0 con 0x559640336a30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/461142546 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb7ec003c60 con 0x559640336a30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 10 client.27277 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 10 client.27277 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 20 client.27277 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55964034d080 0x55964034f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 4 client.27277 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:56.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7faf9a740 1 -- 172.21.15.105:0/461142546 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596402a8eb0 con 0x55964034d080 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.650+0000 7fb7f594e640 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55964034d080 0x55964034f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7f594e640 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55964034d080 0x55964034f4d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d2d95a762c2aa8f server_cookie=260af82b9c2aa965 in_seq=0 out_seq=0 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7d9ffb640 10 client.27277 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/461142546 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5596402a8eb0 con 0x55964034d080 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7d9ffb640 10 client.27277 handle_command_reply: tid=0 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 client.27277 shutdown 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 -- 172.21.15.105:0/461142546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640336a30 msgr2=0x559640341000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640336a30 0x559640341000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000c00 tx=0x7fb7ec009260 comp rx=0 tx=0).stop 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 -- 172.21.15.105:0/461142546 shutdown_connections 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55964034d080 0x55964034f4d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640348580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640337420 0x559640341540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7d9ffb640 0 client.27277 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:43:56.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/461142546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640336a30 0x559640341000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.651+0000 7fb7faf9a740 1 -- 172.21.15.105:0/461142546 >> 172.21.15.105:0/461142546 conn(0x559640331fa0 msgr2=0x5596403311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.652+0000 7fb7faf9a740 1 -- 172.21.15.105:0/461142546 shutdown_connections 2024-03-26T17:43:56.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.652+0000 7fb7faf9a740 1 -- 172.21.15.105:0/461142546 wait complete. 2024-03-26T17:43:56.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.652+0000 7fb7faf9a740 20 client.27277 trim_cache size 0 max 16384 2024-03-26T17:43:56.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7faf9a740 1 Processor -- start 2024-03-26T17:43:56.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7faf9a740 1 -- start start 2024-03-26T17:43:56.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x5596403414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x559640348540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640335910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7f614f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x5596403414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7f594e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x559640348540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7f614f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x5596403414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50968/0 (socket says 172.21.15.105:50968) 2024-03-26T17:43:56.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7f6950640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640335910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7f6950640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640335910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52450/0 (socket says 172.21.15.105:52450) 2024-03-26T17:43:56.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7f614f640 1 -- 172.21.15.105:0/1008658950 learned_addr learned my addr 172.21.15.105:0/1008658950 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1008658950 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f01ab100 con 0x559640336a30 2024-03-26T17:43:56.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1008658950 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f01acc40 con 0x55964033c590 2024-03-26T17:43:56.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.666+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1008658950 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f01ab7d0 con 0x559640337420 2024-03-26T17:43:56.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.667+0000 7fb7f6950640 1 -- 172.21.15.105:0/1008658950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 msgr2=0x559640348540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.667+0000 7fb7f6950640 1 --2- 172.21.15.105:0/1008658950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x559640348540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.667+0000 7fb7f6950640 1 -- 172.21.15.105:0/1008658950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 msgr2=0x5596403414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.667+0000 7fb7f6950640 1 --2- 172.21.15.105:0/1008658950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x5596403414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.667+0000 7fb7f6950640 1 -- 172.21.15.105:0/1008658950 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964032da10 con 0x55964033c590 2024-03-26T17:43:56.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.667+0000 7fb7f6950640 1 --2- 172.21.15.105:0/1008658950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640335910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8005fd0 tx=0x7fb7e8003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f84205ce0c11e855 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.667+0000 7fb7dbfff640 1 -- 172.21.15.105:0/1008658950 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8004840 con 0x55964033c590 2024-03-26T17:43:56.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7dbfff640 1 -- 172.21.15.105:0/1008658950 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7e8005020 con 0x55964033c590 2024-03-26T17:43:56.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7dbfff640 1 -- 172.21.15.105:0/1008658950 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8010670 con 0x55964033c590 2024-03-26T17:43:56.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1008658950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 msgr2=0x559640335910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1008658950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640335910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8005fd0 tx=0x7fb7e8003f90 comp rx=0 tx=0).stop 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1008658950 shutdown_connections 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1008658950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55964033c590 0x559640335910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1008658950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559640337420 0x559640348540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1008658950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x5596403414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1008658950 >> 172.21.15.105:0/1008658950 conn(0x559640331fa0 msgr2=0x559640335200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1008658950 shutdown_connections 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.668+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1008658950 wait complete. 2024-03-26T17:43:56.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 1 -- start start 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 1 Processor -- start 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x559640345a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640337420 0x559640345f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55964033c590 0x559640346f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f01ac570 con 0x559640336a30 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f01abea0 con 0x559640337420 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f01a90e0 con 0x55964033c590 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7faf9a740 10 client.0 _ll_register_callbacks cb 0x7fb7ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f614f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x559640345a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f594e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640337420 0x559640345f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f614f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x559640345a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50972/0 (socket says 172.21.15.105:50972) 2024-03-26T17:43:56.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f614f640 1 -- 172.21.15.105:0/1262710444 learned_addr learned my addr 172.21.15.105:0/1262710444 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f6950640 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55964033c590 0x559640346f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f614f640 1 -- 172.21.15.105:0/1262710444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55964033c590 msgr2=0x559640346f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f614f640 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55964033c590 0x559640346f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f614f640 1 -- 172.21.15.105:0/1262710444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640337420 msgr2=0x559640345f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f614f640 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640337420 0x559640345f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.669+0000 7fb7f614f640 1 -- 172.21.15.105:0/1262710444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559640347520 con 0x559640336a30 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7f614f640 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x559640345a50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000c00 tx=0x7fb7ec0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cad67ea06714f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/1262710444 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec050070 con 0x559640336a30 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/1262710444 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb7ec041ac0 con 0x559640336a30 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1262710444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e8005020 con 0x559640336a30 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/1262710444 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec041cc0 con 0x559640336a30 2024-03-26T17:43:56.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7faf9a740 10 client.17448 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1262710444 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559640348890 con 0x559640336a30 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/1262710444 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7e8005020 con 0x559640336a30 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/1262710444 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb7ec003a40 con 0x559640336a30 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7faf9a740 10 client.17448 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.670+0000 7fb7faf9a740 10 client.17448 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 20 client.17448 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5596403b3ba0 0x5596403b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 4 client.17448 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1262710444 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596402a3200 con 0x5596403b3ba0 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7f594e640 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5596403b3ba0 0x5596403b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7f594e640 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5596403b3ba0 0x5596403b5fc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a0a105ded499401 server_cookie=529894e5744dd629 in_seq=0 out_seq=0 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7d9ffb640 10 client.17448 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7d9ffb640 1 -- 172.21.15.105:0/1262710444 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596402a3200 con 0x5596403b3ba0 2024-03-26T17:43:56.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7d9ffb640 10 client.17448 handle_command_reply: tid=0 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 client.17448 shutdown 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1262710444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 msgr2=0x559640345a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x559640345a50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000c00 tx=0x7fb7ec0092d0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1262710444 shutdown_connections 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5596403b3ba0 0x5596403b5fc0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55964033c590 0x559640346f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7d9ffb640 0 client.17448 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559640337420 0x559640345f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 --2- 172.21.15.105:0/1262710444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559640336a30 0x559640345a50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.671+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1262710444 >> 172.21.15.105:0/1262710444 conn(0x559640331fa0 msgr2=0x559640335a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1262710444 shutdown_connections 2024-03-26T17:43:56.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7faf9a740 1 -- 172.21.15.105:0/1262710444 wait complete. 2024-03-26T17:43:56.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7faf9a740 20 client.17448 trim_cache size 0 max 16384 2024-03-26T17:43:56.671 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:56.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 -- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb7dc0a2f20 msgr2=0x7fb7dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb7dc0a2f20 0x7fb7dc0a53e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009230 tx=0x7fb7ec005fd0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 -- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f0104d00 msgr2=0x7fb7f0100380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f0104d00 0x7fb7f0100380 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e9c0 tx=0x7fb7e000ee80 comp rx=0 tx=0).stop 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 -- 172.21.15.105:0/3272705074 shutdown_connections 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb7dc0a2f20 0x7fb7dc0a53e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7f0107630 0x7fb7f01008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7f0104d00 0x7fb7f0100380 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 --2- 172.21.15.105:0/3272705074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f01023d0 0x7fb7f0101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 -- 172.21.15.105:0/3272705074 >> 172.21.15.105:0/3272705074 conn(0x7fb7f00fbd60 msgr2=0x7fb7f0102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 -- 172.21.15.105:0/3272705074 shutdown_connections 2024-03-26T17:43:56.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.672+0000 7fb7f83b3640 1 -- 172.21.15.105:0/3272705074 wait complete. 2024-03-26T17:43:56.676 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:43:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.803+0000 7f1da6c63640 1 -- 172.21.15.105:0/1636093978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 msgr2=0x7f1da0101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.803+0000 7f1da6c63640 1 --2- 172.21.15.105:0/1636093978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 0x7f1da0101e30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009900 tx=0x7f1d8c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 -- 172.21.15.105:0/1636093978 shutdown_connections 2024-03-26T17:43:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 --2- 172.21.15.105:0/1636093978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da0102370 0x7f1da0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 --2- 172.21.15.105:0/1636093978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 0x7f1da0101e30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 --2- 172.21.15.105:0/1636093978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1da01007c0 0x7f1da0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 -- 172.21.15.105:0/1636093978 >> 172.21.15.105:0/1636093978 conn(0x7f1da00fbd50 msgr2=0x7f1da00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 -- 172.21.15.105:0/1636093978 shutdown_connections 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 -- 172.21.15.105:0/1636093978 wait complete. 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 Processor -- start 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 -- start start 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.804+0000 7f1da6c63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1da01007c0 0x7f1da0199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da6c63640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 0x7f1da019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da6c63640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da0102370 0x7f1da01a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da6c63640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010c2f0 con 0x7f1da01019b0 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da6c63640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010c170 con 0x7f1da01007c0 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da6c63640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1da010c470 con 0x7f1da0102370 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1d9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 0x7f1da019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1da01007c0 0x7f1da0199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1da01007c0 0x7f1da0199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52492/0 (socket says 172.21.15.105:52492) 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1d9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 0x7f1da019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50996/0 (socket says 172.21.15.105:50996) 2024-03-26T17:43:56.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 -- 172.21.15.105:0/3886020040 learned_addr learned my addr 172.21.15.105:0/3886020040 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da5200640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da0102370 0x7f1da01a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 -- 172.21.15.105:0/3886020040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da0102370 msgr2=0x7f1da01a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da0102370 0x7f1da01a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 -- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 msgr2=0x7f1da019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 0x7f1da019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 -- 172.21.15.105:0/3886020040 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da01a1960 con 0x7f1da01007c0 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 0x7f1da019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da5200640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da0102370 0x7f1da01a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1da01007c0 0x7f1da0199d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400e9c0 tx=0x7f1d9400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.805+0000 7f1d9dffb640 1 -- 172.21.15.105:0/3886020040 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9400cd80 con 0x7f1da01007c0 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.806+0000 7f1da6c63640 1 -- 172.21.15.105:0/3886020040 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da01a1c50 con 0x7f1da01007c0 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.806+0000 7f1d9dffb640 1 -- 172.21.15.105:0/3886020040 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d940035f0 con 0x7f1da01007c0 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.806+0000 7f1d9dffb640 1 -- 172.21.15.105:0/3886020040 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94010690 con 0x7f1da01007c0 2024-03-26T17:43:56.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.806+0000 7f1da6c63640 1 -- 172.21.15.105:0/3886020040 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a2170 con 0x7f1da01007c0 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.808+0000 7f1d9dffb640 1 -- 172.21.15.105:0/3886020040 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1d940037a0 con 0x7f1da01007c0 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.808+0000 7f1da9847740 1 Processor -- start 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da9847740 1 -- start start 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1d9dffb640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d900a2f20 0x7f1d900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da9847740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe509de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da9847740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da9847740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da9847740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ffe5002970 con 0x55ffe50a3590 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da9847740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ffe4faad70 con 0x55ffe509e420 2024-03-26T17:43:56.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da9847740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ffe4ffe920 con 0x55ffe509da30 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da5200640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d900a2f20 0x7f1d900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe509de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1d9dffb640 1 -- 172.21.15.105:0/3886020040 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1d940cf790 con 0x7f1da01007c0 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe509de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33872/0 (socket says 172.21.15.105:33872) 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 -- 172.21.15.105:0/785479812 learned_addr learned my addr 172.21.15.105:0/785479812 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da5200640 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 -- 172.21.15.105:0/785479812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 msgr2=0x55ffe50a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 -- 172.21.15.105:0/785479812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 msgr2=0x55ffe50a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 -- 172.21.15.105:0/785479812 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe4d163f0 con 0x55ffe509da30 2024-03-26T17:43:56.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.809+0000 7f1da49ff640 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe509de50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d940a4360 tx=0x7f1d9409bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a58497dcb4e78e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da5200640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d900a2f20 0x7f1d900a53e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005fd0 tx=0x7f1d98011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1d8affd640 1 -- 172.21.15.105:0/785479812 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b3820 con 0x55ffe509da30 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1d8affd640 1 -- 172.21.15.105:0/785479812 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d94003b30 con 0x55ffe509da30 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1d8affd640 1 -- 172.21.15.105:0/785479812 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b51b0 con 0x55ffe509da30 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 -- 172.21.15.105:0/785479812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 msgr2=0x55ffe509de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe509de50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d940a4360 tx=0x7f1d9409bd10 comp rx=0 tx=0).stop 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 -- 172.21.15.105:0/785479812 shutdown_connections 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 --2- 172.21.15.105:0/785479812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe509de50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 -- 172.21.15.105:0/785479812 >> 172.21.15.105:0/785479812 conn(0x55ffe5098fa0 msgr2=0x55ffe50993a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 -- 172.21.15.105:0/785479812 shutdown_connections 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.810+0000 7f1da9847740 1 -- 172.21.15.105:0/785479812 wait complete. 2024-03-26T17:43:56.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 1 -- start start 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 1 Processor -- start 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509e420 0x55ffe50a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 0x55ffe50af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ffe50a6540 con 0x55ffe509e420 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ffe4fbe310 con 0x55ffe50a3590 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ffe50a66c0 con 0x55ffe509da30 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da9847740 10 client.0 _ll_register_callbacks cb 0x55ffe4ff9c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1d9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509e420 0x55ffe50a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1d9ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509e420 0x55ffe50a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51012/0 (socket says 172.21.15.105:51012) 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50a7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33888/0 (socket says 172.21.15.105:33888) 2024-03-26T17:43:56.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1da5200640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 0x55ffe50af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.811+0000 7f1d9ffff640 1 -- 172.21.15.105:0/3486453080 learned_addr learned my addr 172.21.15.105:0/3486453080 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da49ff640 1 -- 172.21.15.105:0/3486453080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 msgr2=0x55ffe50af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 0x55ffe50af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da49ff640 1 -- 172.21.15.105:0/3486453080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509e420 msgr2=0x55ffe50a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509e420 0x55ffe50a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da49ff640 1 -- 172.21.15.105:0/3486453080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe4de07d0 con 0x55ffe509da30 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50a7f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55ffe50a6210 tx=0x7f1d940bc110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30720060572987e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1d7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1d7ffff640 1 -- 172.21.15.105:0/3486453080 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b5390 con 0x55ffe509da30 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1d7ffff640 1 -- 172.21.15.105:0/3486453080 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1d940a4600 con 0x55ffe509da30 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da9847740 1 -- 172.21.15.105:0/3486453080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffe500feb0 con 0x55ffe509da30 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1d7ffff640 1 -- 172.21.15.105:0/3486453080 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d940b51b0 con 0x55ffe509da30 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da9847740 10 client.27295 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:56.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da9847740 1 -- 172.21.15.105:0/3486453080 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ffe50a6540 con 0x55ffe509da30 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1d7ffff640 1 -- 172.21.15.105:0/3486453080 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ffe500feb0 con 0x55ffe509da30 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1d7ffff640 1 -- 172.21.15.105:0/3486453080 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1d94097bd0 con 0x55ffe509da30 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da9847740 10 client.27295 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da9847740 10 client.27295 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da9847740 20 client.27295 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.812+0000 7f1da9847740 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ffe50b3990 0x55ffe50b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.813+0000 7f1da9847740 4 client.27295 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.813+0000 7f1da9847740 1 -- 172.21.15.105:0/3486453080 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe500a200 con 0x55ffe50b3990 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.813+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ffe50b3990 0x55ffe50b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.813+0000 7f1d7ffff640 10 client.27295 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:43:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.813+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ffe50b3990 0x55ffe50b5da0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f31ed4230d91561 server_cookie=44807ea68df51c7e in_seq=0 out_seq=0 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1d7ffff640 1 -- 172.21.15.105:0/3486453080 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ffe500a200 con 0x55ffe50b3990 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1d7ffff640 10 client.27295 handle_command_reply: tid=0 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 client.27295 shutdown 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 -- 172.21.15.105:0/3486453080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 msgr2=0x55ffe50a7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50a7f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55ffe50a6210 tx=0x7f1d940bc110 comp rx=0 tx=0).stop 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 -- 172.21.15.105:0/3486453080 shutdown_connections 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ffe50b3990 0x55ffe50b5da0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 0x55ffe50af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509e420 0x55ffe50a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1d7ffff640 0 client.27295 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 --2- 172.21.15.105:0/3486453080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50a7f20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 -- 172.21.15.105:0/3486453080 >> 172.21.15.105:0/3486453080 conn(0x55ffe5098fa0 msgr2=0x55ffe50981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 -- 172.21.15.105:0/3486453080 shutdown_connections 2024-03-26T17:43:56.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 1 -- 172.21.15.105:0/3486453080 wait complete. 2024-03-26T17:43:56.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.814+0000 7f1da9847740 20 client.27295 trim_cache size 0 max 16384 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.828+0000 7f1da9847740 1 Processor -- start 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.828+0000 7f1da9847740 1 -- start start 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da9847740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509da30 0x55ffe50a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da9847740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509e420 0x55ffe50af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da9847740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 0x55ffe509c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da9847740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01aaf80 con 0x55ffe509da30 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da9847740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01acac0 con 0x55ffe50a3590 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da9847740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01ab650 con 0x55ffe509e420 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509da30 0x55ffe50a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509da30 0x55ffe50a83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51020/0 (socket says 172.21.15.105:51020) 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 -- 172.21.15.105:0/3311481428 learned_addr learned my addr 172.21.15.105:0/3311481428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509e420 0x55ffe50af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da5200640 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 0x55ffe509c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 -- 172.21.15.105:0/3311481428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509e420 msgr2=0x55ffe50af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509e420 0x55ffe50af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 -- 172.21.15.105:0/3311481428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 msgr2=0x55ffe509c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 0x55ffe509c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 -- 172.21.15.105:0/3311481428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe50a2bd0 con 0x55ffe509da30 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1da49ff640 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509da30 0x55ffe50a83c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400c6a0 tx=0x7f1d9409a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1810b57281ae64ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1d8a7fc640 1 -- 172.21.15.105:0/3311481428 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b3820 con 0x55ffe509da30 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1d8a7fc640 1 -- 172.21.15.105:0/3311481428 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d940b9070 con 0x55ffe509da30 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.829+0000 7f1d8a7fc640 1 -- 172.21.15.105:0/3311481428 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940a4660 con 0x55ffe509da30 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 -- 172.21.15.105:0/3311481428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509da30 msgr2=0x55ffe50a83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509da30 0x55ffe50a83c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400c6a0 tx=0x7f1d9409a390 comp rx=0 tx=0).stop 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 -- 172.21.15.105:0/3311481428 shutdown_connections 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe50a3590 0x55ffe509c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509e420 0x55ffe50af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 --2- 172.21.15.105:0/3311481428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe509da30 0x55ffe50a83c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 -- 172.21.15.105:0/3311481428 >> 172.21.15.105:0/3311481428 conn(0x55ffe5098fa0 msgr2=0x55ffe509c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 -- 172.21.15.105:0/3311481428 shutdown_connections 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.830+0000 7f1da9847740 1 -- 172.21.15.105:0/3311481428 wait complete. 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 1 -- start start 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 1 Processor -- start 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50ad810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01ac3f0 con 0x55ffe50a3590 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01abd20 con 0x55ffe509e420 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01a8f60 con 0x55ffe509da30 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da9847740 10 client.0 _ll_register_callbacks cb 0x7f1d98011690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:56.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da5200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da5200640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51030/0 (socket says 172.21.15.105:51030) 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1d9ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da5200640 1 -- 172.21.15.105:0/75484453 learned_addr learned my addr 172.21.15.105:0/75484453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.831+0000 7f1da49ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33906/0 (socket says 172.21.15.105:33906) 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da49ff640 1 -- 172.21.15.105:0/75484453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 msgr2=0x55ffe50ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da49ff640 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da49ff640 1 -- 172.21.15.105:0/75484453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 msgr2=0x55ffe50ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da49ff640 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da49ff640 1 -- 172.21.15.105:0/75484453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe50aee10 con 0x55ffe509da30 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da5200640 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da49ff640 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50ad2d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1d94098800 tx=0x7f1d9409b500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=792a5cc533b9a61f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1d7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:56.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1d7ffff640 1 -- 172.21.15.105:0/75484453 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d940b3820 con 0x55ffe509da30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da9847740 1 -- 172.21.15.105:0/75484453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d940b9070 con 0x55ffe509da30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1d7ffff640 1 -- 172.21.15.105:0/75484453 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1d940a4cf0 con 0x55ffe509da30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1d7ffff640 1 -- 172.21.15.105:0/75484453 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d940a4890 con 0x55ffe509da30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da9847740 10 client.27301 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da9847740 1 -- 172.21.15.105:0/75484453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ffe50aee10 con 0x55ffe509da30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1d7ffff640 1 -- 172.21.15.105:0/75484453 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d940b9070 con 0x55ffe509da30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1d7ffff640 1 -- 172.21.15.105:0/75484453 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1d940a1160 con 0x55ffe509da30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da9847740 10 client.27301 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da9847740 10 client.27301 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da9847740 20 client.27301 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.832+0000 7f1da9847740 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ffe511a960 0x55ffe511cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1da9847740 4 client.27301 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1da9847740 1 -- 172.21.15.105:0/75484453 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffe4f876a0 con 0x55ffe511a960 2024-03-26T17:43:56.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ffe511a960 0x55ffe511cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1d7ffff640 10 client.27301 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ffe511a960 0x55ffe511cd50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd45454ea8814bc4 server_cookie=143abe0868399e5c in_seq=0 out_seq=0 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1d7ffff640 1 -- 172.21.15.105:0/75484453 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffe4f876a0 con 0x55ffe511a960 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1d7ffff640 10 client.27301 handle_command_reply: tid=0 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1da9847740 1 client.27301 shutdown 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1da9847740 1 -- 172.21.15.105:0/75484453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 msgr2=0x55ffe50ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.833+0000 7f1da9847740 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50ad2d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1d94098800 tx=0x7f1d9409b500 comp rx=0 tx=0).stop 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 1 -- 172.21.15.105:0/75484453 shutdown_connections 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ffe511a960 0x55ffe511cd50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffe50a3590 0x55ffe50ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1d7ffff640 0 client.27301 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffe509e420 0x55ffe50ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 1 --2- 172.21.15.105:0/75484453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffe509da30 0x55ffe50ad2d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 1 -- 172.21.15.105:0/75484453 >> 172.21.15.105:0/75484453 conn(0x55ffe5098fa0 msgr2=0x55ffe509ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 1 -- 172.21.15.105:0/75484453 shutdown_connections 2024-03-26T17:43:56.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 1 -- 172.21.15.105:0/75484453 wait complete. 2024-03-26T17:43:56.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.834+0000 7f1da9847740 20 client.27301 trim_cache size 0 max 16384 2024-03-26T17:43:56.833 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 -- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d900a2f20 msgr2=0x7f1d900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d900a2f20 0x7f1d900a53e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005fd0 tx=0x7f1d98011440 comp rx=0 tx=0).stop 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 -- 172.21.15.105:0/3886020040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1da01007c0 msgr2=0x7f1da0199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1da01007c0 0x7f1da0199d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400e9c0 tx=0x7f1d9400ee80 comp rx=0 tx=0).stop 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 -- 172.21.15.105:0/3886020040 shutdown_connections 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d900a2f20 0x7f1d900a53e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1da0102370 0x7f1da01a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1da01019b0 0x7f1da019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 --2- 172.21.15.105:0/3886020040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1da01007c0 0x7f1da0199d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 -- 172.21.15.105:0/3886020040 >> 172.21.15.105:0/3886020040 conn(0x7f1da00fbd50 msgr2=0x7f1da0068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 -- 172.21.15.105:0/3886020040 shutdown_connections 2024-03-26T17:43:56.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.835+0000 7f1da6c63640 1 -- 172.21.15.105:0/3886020040 wait complete. 2024-03-26T17:43:56.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:43:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 -- 172.21.15.105:0/2826841250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 msgr2=0x7f8d90100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2826841250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 0x7f8d90100c00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009860 tx=0x7f8d8002bb70 comp rx=0 tx=0).stop 2024-03-26T17:43:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 -- 172.21.15.105:0/2826841250 shutdown_connections 2024-03-26T17:43:56.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2826841250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d90102390 0x7f8d90108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2826841250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d901019d0 0x7f8d90101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2826841250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 0x7f8d90100c00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 -- 172.21.15.105:0/2826841250 >> 172.21.15.105:0/2826841250 conn(0x7f8d900fbd50 msgr2=0x7f8d900fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 -- 172.21.15.105:0/2826841250 shutdown_connections 2024-03-26T17:43:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 -- 172.21.15.105:0/2826841250 wait complete. 2024-03-26T17:43:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 Processor -- start 2024-03-26T17:43:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.969+0000 7f8d9595b640 1 -- start start 2024-03-26T17:43:56.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d9595b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 0x7f8d901959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d9595b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d901019d0 0x7f8d90195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d9595b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d90102390 0x7f8d9019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d9595b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9010b220 con 0x7f8d901019d0 2024-03-26T17:43:56.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d9595b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9010b0a0 con 0x7f8d90102390 2024-03-26T17:43:56.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d9595b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9010b3a0 con 0x7f8d901007e0 2024-03-26T17:43:56.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d901019d0 0x7f8d90195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d901019d0 0x7f8d90195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51056/0 (socket says 172.21.15.105:51056) 2024-03-26T17:43:56.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8e7fc640 1 -- 172.21.15.105:0/2552932706 learned_addr learned my addr 172.21.15.105:0/2552932706 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8effd640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 0x7f8d901959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8f7fe640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d90102390 0x7f8d9019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8effd640 1 -- 172.21.15.105:0/2552932706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d90102390 msgr2=0x7f8d9019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8effd640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d90102390 0x7f8d9019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8effd640 1 -- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d901019d0 msgr2=0x7f8d90195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8effd640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d901019d0 0x7f8d90195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8effd640 1 -- 172.21.15.105:0/2552932706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9019d5c0 con 0x7f8d901007e0 2024-03-26T17:43:56.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d901019d0 0x7f8d90195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8f7fe640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d90102390 0x7f8d9019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.970+0000 7f8d8effd640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 0x7f8d901959a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d80037040 tx=0x7f8d800040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.971+0000 7f8d94959640 1 -- 172.21.15.105:0/2552932706 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80041020 con 0x7f8d901007e0 2024-03-26T17:43:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.971+0000 7f8d94959640 1 -- 172.21.15.105:0/2552932706 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d800044d0 con 0x7f8d901007e0 2024-03-26T17:43:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.971+0000 7f8d94959640 1 -- 172.21.15.105:0/2552932706 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80039040 con 0x7f8d901007e0 2024-03-26T17:43:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.971+0000 7f8d9595b640 1 -- 172.21.15.105:0/2552932706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9019d850 con 0x7f8d901007e0 2024-03-26T17:43:56.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.971+0000 7f8d9595b640 1 -- 172.21.15.105:0/2552932706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9019dd40 con 0x7f8d901007e0 2024-03-26T17:43:56.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.973+0000 7f8d94959640 1 -- 172.21.15.105:0/2552932706 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8d80044020 con 0x7f8d901007e0 2024-03-26T17:43:56.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d94959640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d780a2f20 0x7f8d780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d98628740 1 Processor -- start 2024-03-26T17:43:56.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d98628740 1 -- start start 2024-03-26T17:43:56.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d780a2f20 0x7f8d780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d94959640 1 -- 172.21.15.105:0/2552932706 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d800f0c40 con 0x7f8d901007e0 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d98628740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c7be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d98628740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c80f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d98628740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4c83a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d98628740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562bd4be0970 con 0x562bd4c7c420 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d98628740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562bd4b88d70 con 0x562bd4c7ba30 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d98628740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562bd4bdc920 con 0x562bd4c81590 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c7be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c7be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52556/0 (socket says 172.21.15.105:52556) 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d8effd640 1 -- 172.21.15.105:0/1404319218 learned_addr learned my addr 172.21.15.105:0/1404319218 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.974+0000 7f8d8f7fe640 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4c83a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c80f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d8effd640 1 -- 172.21.15.105:0/1404319218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 msgr2=0x562bd4c83a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d8effd640 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4c83a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d8effd640 1 -- 172.21.15.105:0/1404319218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 msgr2=0x562bd4c80f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d8effd640 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c80f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d8effd640 1 -- 172.21.15.105:0/1404319218 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd48f43f0 con 0x562bd4c7ba30 2024-03-26T17:43:56.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d780a2f20 0x7f8d780a53e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8d90196eb0 tx=0x7f8d84011920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d8effd640 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c7be50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d800eea70 tx=0x7f8d8002bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aed1fdabfd9ba95 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d75ffb640 1 -- 172.21.15.105:0/1404319218 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800bd330 con 0x562bd4c7ba30 2024-03-26T17:43:56.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d75ffb640 1 -- 172.21.15.105:0/1404319218 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d800c1480 con 0x562bd4c7ba30 2024-03-26T17:43:56.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.975+0000 7f8d75ffb640 1 -- 172.21.15.105:0/1404319218 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80003680 con 0x562bd4c7ba30 2024-03-26T17:43:56.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 -- 172.21.15.105:0/1404319218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 msgr2=0x562bd4c7be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c7be50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d800eea70 tx=0x7f8d8002bb70 comp rx=0 tx=0).stop 2024-03-26T17:43:56.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 -- 172.21.15.105:0/1404319218 shutdown_connections 2024-03-26T17:43:56.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4c83a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c80f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 --2- 172.21.15.105:0/1404319218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c7be50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 -- 172.21.15.105:0/1404319218 >> 172.21.15.105:0/1404319218 conn(0x562bd4c76fa0 msgr2=0x562bd4c773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 -- 172.21.15.105:0/1404319218 shutdown_connections 2024-03-26T17:43:56.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 -- 172.21.15.105:0/1404319218 wait complete. 2024-03-26T17:43:56.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.976+0000 7f8d98628740 1 -- start start 2024-03-26T17:43:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d98628740 1 Processor -- start 2024-03-26T17:43:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d98628740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d98628740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7c420 0x562bd4c86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d98628740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c81590 0x562bd4c8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d98628740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562bd4c84300 con 0x562bd4c7ba30 2024-03-26T17:43:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d98628740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562bd4b9c310 con 0x562bd4c81590 2024-03-26T17:43:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d98628740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562bd4c84480 con 0x562bd4c7c420 2024-03-26T17:43:56.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d98628740 10 client.0 _ll_register_callbacks cb 0x562bd4bd7c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c81590 0x562bd4c8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c81590 0x562bd4c8d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52564/0 (socket says 172.21.15.105:52564) 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8f7fe640 1 -- 172.21.15.105:0/487842279 learned_addr learned my addr 172.21.15.105:0/487842279 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7c420 0x562bd4c86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8f7fe640 1 -- 172.21.15.105:0/487842279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7c420 msgr2=0x562bd4c86090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8f7fe640 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7c420 0x562bd4c86090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8f7fe640 1 -- 172.21.15.105:0/487842279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7ba30 msgr2=0x562bd4c85b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8f7fe640 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.977+0000 7f8d8f7fe640 1 -- 172.21.15.105:0/487842279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd49be7d0 con 0x562bd4c81590 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d8f7fe640 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c81590 0x562bd4c8d0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800efd0 tx=0x7f8d88018990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958f9c5f388a7710 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d6effd640 1 -- 172.21.15.105:0/487842279 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88003e40 con 0x562bd4c81590 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d98628740 1 -- 172.21.15.105:0/487842279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d800c1480 con 0x562bd4c81590 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d6effd640 1 -- 172.21.15.105:0/487842279 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d88004b20 con 0x562bd4c81590 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d6effd640 1 -- 172.21.15.105:0/487842279 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d88004d60 con 0x562bd4c81590 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d98628740 10 client.27305 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d98628740 1 -- 172.21.15.105:0/487842279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bd4c904b0 con 0x562bd4c81590 2024-03-26T17:43:56.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d6effd640 1 -- 172.21.15.105:0/487842279 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d800c1480 con 0x562bd4c81590 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.978+0000 7f8d6effd640 1 -- 172.21.15.105:0/487842279 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8d8801a790 con 0x562bd4c81590 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d98628740 10 client.27305 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d98628740 10 client.27305 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d98628740 20 client.27305 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d98628740 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562bd4c91bf0 0x562bd4c940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d98628740 4 client.27305 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d98628740 1 -- 172.21.15.105:0/487842279 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bd4bedeb0 con 0x562bd4c91bf0 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d8effd640 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562bd4c91bf0 0x562bd4c940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d8effd640 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562bd4c91bf0 0x562bd4c940b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78faaf7d4045b71e server_cookie=d9a6dd0b4d5c3792 in_seq=0 out_seq=0 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.979+0000 7f8d6effd640 10 client.27305 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d6effd640 1 -- 172.21.15.105:0/487842279 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562bd4bedeb0 con 0x562bd4c91bf0 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d6effd640 10 client.27305 handle_command_reply: tid=0 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 client.27305 shutdown 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 -- 172.21.15.105:0/487842279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c81590 msgr2=0x562bd4c8d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c81590 0x562bd4c8d0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800efd0 tx=0x7f8d88018990 comp rx=0 tx=0).stop 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 -- 172.21.15.105:0/487842279 shutdown_connections 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562bd4c91bf0 0x562bd4c940b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c81590 0x562bd4c8d0d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7c420 0x562bd4c86090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 --2- 172.21.15.105:0/487842279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d98628740 1 -- 172.21.15.105:0/487842279 >> 172.21.15.105:0/487842279 conn(0x562bd4c76fa0 msgr2=0x562bd4c761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.980+0000 7f8d6effd640 0 client.27305 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.981+0000 7f8d98628740 1 -- 172.21.15.105:0/487842279 shutdown_connections 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.981+0000 7f8d98628740 1 -- 172.21.15.105:0/487842279 wait complete. 2024-03-26T17:43:56.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.981+0000 7f8d98628740 20 client.27305 trim_cache size 0 max 16384 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.997+0000 7f8d98628740 1 Processor -- start 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.997+0000 7f8d98628740 1 -- start start 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.997+0000 7f8d98628740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.997+0000 7f8d98628740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7c420 0x562bd4c8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.997+0000 7f8d98628740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c81590 0x562bd4c86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.997+0000 7f8d98628740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901a6c40 con 0x562bd4c81590 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.997+0000 7f8d98628740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901a8780 con 0x562bd4c7c420 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.997+0000 7f8d98628740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901a7310 con 0x562bd4c7ba30 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c81590 0x562bd4c86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c81590 0x562bd4c86090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51088/0 (socket says 172.21.15.105:51088) 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33938/0 (socket says 172.21.15.105:33938) 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8f7fe640 1 -- 172.21.15.105:0/2442415461 learned_addr learned my addr 172.21.15.105:0/2442415461 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/2442415461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7c420 0x562bd4c8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8effd640 1 -- 172.21.15.105:0/2442415461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7c420 msgr2=0x562bd4c8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8effd640 1 --2- 172.21.15.105:0/2442415461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7c420 0x562bd4c8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8effd640 1 -- 172.21.15.105:0/2442415461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c81590 msgr2=0x562bd4c86090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8effd640 1 --2- 172.21.15.105:0/2442415461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c81590 0x562bd4c86090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8effd640 1 -- 172.21.15.105:0/2442415461 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd4c768f0 con 0x562bd4c7ba30 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d8effd640 1 --2- 172.21.15.105:0/2442415461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d800c5810 tx=0x7f8d800035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=688688d70eeb469c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d757fa640 1 -- 172.21.15.105:0/2442415461 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800db070 con 0x562bd4c7ba30 2024-03-26T17:43:56.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d757fa640 1 -- 172.21.15.105:0/2442415461 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d800c1480 con 0x562bd4c7ba30 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.998+0000 7f8d757fa640 1 -- 172.21.15.105:0/2442415461 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800a6070 con 0x562bd4c7ba30 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 -- 172.21.15.105:0/2442415461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7ba30 msgr2=0x562bd4c85b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 --2- 172.21.15.105:0/2442415461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8d800c5810 tx=0x7f8d800035f0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 -- 172.21.15.105:0/2442415461 shutdown_connections 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 --2- 172.21.15.105:0/2442415461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c81590 0x562bd4c86090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 --2- 172.21.15.105:0/2442415461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7c420 0x562bd4c8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 --2- 172.21.15.105:0/2442415461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c7ba30 0x562bd4c85b50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 -- 172.21.15.105:0/2442415461 >> 172.21.15.105:0/2442415461 conn(0x562bd4c76fa0 msgr2=0x562bd4c7d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 -- 172.21.15.105:0/2442415461 shutdown_connections 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:56.999+0000 7f8d98628740 1 -- 172.21.15.105:0/2442415461 wait complete. 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d98628740 1 -- start start 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d98628740 1 Processor -- start 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d98628740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c8a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:56.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d98628740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c8a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d98628740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4cf5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d98628740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901a80b0 con 0x562bd4c7c420 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d98628740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901a79e0 con 0x562bd4c7ba30 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c8a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.000+0000 7f8d8e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c8a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d98628740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901a4c20 con 0x562bd4c81590 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c8a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51104/0 (socket says 172.21.15.105:51104) 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8e7fc640 1 -- 172.21.15.105:0/1738181166 learned_addr learned my addr 172.21.15.105:0/1738181166 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d98628740 10 client.0 _ll_register_callbacks cb 0x7f8d8800ef00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8f7fe640 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4cf5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8e7fc640 1 -- 172.21.15.105:0/1738181166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 msgr2=0x562bd4cf5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4cf5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8e7fc640 1 -- 172.21.15.105:0/1738181166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 msgr2=0x562bd4c8a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c8a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8e7fc640 1 -- 172.21.15.105:0/1738181166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd4c76b40 con 0x562bd4c7c420 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8f7fe640 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4cf5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d8e7fc640 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c8a5f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8d84011e20 tx=0x7f8d84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ad8e67280d8e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d6effd640 1 -- 172.21.15.105:0/1738181166 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8402e060 con 0x562bd4c7c420 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d6effd640 1 -- 172.21.15.105:0/1738181166 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d84029440 con 0x562bd4c7c420 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.001+0000 7f8d98628740 1 -- 172.21.15.105:0/1738181166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d800c1480 con 0x562bd4c7c420 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d6effd640 1 -- 172.21.15.105:0/1738181166 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d84017040 con 0x562bd4c7c420 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d98628740 10 client.17484 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d98628740 1 -- 172.21.15.105:0/1738181166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562bd4cf7780 con 0x562bd4c7c420 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d6effd640 1 -- 172.21.15.105:0/1738181166 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d800c1480 con 0x562bd4c7c420 2024-03-26T17:43:57.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d6effd640 1 -- 172.21.15.105:0/1738181166 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8d84034420 con 0x562bd4c7c420 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d98628740 10 client.17484 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d98628740 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d98628740 20 client.17484 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d98628740 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562bd4cf81e0 0x562bd4cfa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d98628740 4 client.17484 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d98628740 1 -- 172.21.15.105:0/1738181166 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bd4be8200 con 0x562bd4cf81e0 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.002+0000 7f8d8effd640 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562bd4cf81e0 0x562bd4cfa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d8effd640 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562bd4cf81e0 0x562bd4cfa600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfa027cbf8ac4a6e server_cookie=b639713e11903e07 in_seq=0 out_seq=0 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d6effd640 10 client.17484 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d6effd640 1 -- 172.21.15.105:0/1738181166 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bd4be8200 con 0x562bd4cf81e0 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d6effd640 10 client.17484 handle_command_reply: tid=0 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 client.17484 shutdown 2024-03-26T17:43:57.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 -- 172.21.15.105:0/1738181166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 msgr2=0x562bd4c8a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c8a5f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8d84011e20 tx=0x7f8d84004730 comp rx=0 tx=0).stop 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 -- 172.21.15.105:0/1738181166 shutdown_connections 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562bd4c81590 0x562bd4cf5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562bd4c7c420 0x562bd4c8a5f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562bd4cf81e0 0x562bd4cfa600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 --2- 172.21.15.105:0/1738181166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562bd4c7ba30 0x562bd4c8a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d6effd640 0 client.17484 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 -- 172.21.15.105:0/1738181166 >> 172.21.15.105:0/1738181166 conn(0x562bd4c76fa0 msgr2=0x562bd4c85b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 -- 172.21.15.105:0/1738181166 shutdown_connections 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 1 -- 172.21.15.105:0/1738181166 wait complete. 2024-03-26T17:43:57.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.003+0000 7f8d98628740 20 client.17484 trim_cache size 0 max 16384 2024-03-26T17:43:57.004 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 -- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d780a2f20 msgr2=0x7f8d780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d780a2f20 0x7f8d780a53e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8d90196eb0 tx=0x7f8d84011920 comp rx=0 tx=0).stop 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 -- 172.21.15.105:0/2552932706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 msgr2=0x7f8d901959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 0x7f8d901959a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8d80037040 tx=0x7f8d800040e0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 -- 172.21.15.105:0/2552932706 shutdown_connections 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d780a2f20 0x7f8d780a53e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d90102390 0x7f8d9019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d901019d0 0x7f8d90195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 --2- 172.21.15.105:0/2552932706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d901007e0 0x7f8d901959a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 -- 172.21.15.105:0/2552932706 >> 172.21.15.105:0/2552932706 conn(0x7f8d900fbd50 msgr2=0x7f8d90104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 -- 172.21.15.105:0/2552932706 shutdown_connections 2024-03-26T17:43:57.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.004+0000 7f8d9595b640 1 -- 172.21.15.105:0/2552932706 wait complete. 2024-03-26T17:43:57.008 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 -- 172.21.15.105:0/1485999864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d0102b10 msgr2=0x7f90d0102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 --2- 172.21.15.105:0/1485999864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d0102b10 0x7f90d0102f90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009900 tx=0x7f90c002bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 -- 172.21.15.105:0/1485999864 shutdown_connections 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 --2- 172.21.15.105:0/1485999864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90d01034d0 0x7f90d0109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 --2- 172.21.15.105:0/1485999864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d0102b10 0x7f90d0102f90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 --2- 172.21.15.105:0/1485999864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90d0101920 0x7f90d0101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 -- 172.21.15.105:0/1485999864 >> 172.21.15.105:0/1485999864 conn(0x7f90d00fce90 msgr2=0x7f90d00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 -- 172.21.15.105:0/1485999864 shutdown_connections 2024-03-26T17:43:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.026+0000 7f90d7559640 1 -- 172.21.15.105:0/1485999864 wait complete. 2024-03-26T17:43:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d7559640 1 Processor -- start 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d7559640 1 -- start start 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d7559640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90d0101920 0x7f90d0197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d7559640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90d0102b10 0x7f90d0197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d7559640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d01034d0 0x7f90d019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d7559640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010d450 con 0x7f90d0102b10 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d7559640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010d2d0 con 0x7f90d0101920 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d7559640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90d010d5d0 con 0x7f90d01034d0 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d4af4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90d0102b10 0x7f90d0197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d52f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90d0101920 0x7f90d0197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d4af4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90d0102b10 0x7f90d0197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51128/0 (socket says 172.21.15.105:51128) 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d4af4640 1 -- 172.21.15.105:0/1448054640 learned_addr learned my addr 172.21.15.105:0/1448054640 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d5af6640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d01034d0 0x7f90d019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d4af4640 1 -- 172.21.15.105:0/1448054640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d01034d0 msgr2=0x7f90d019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d4af4640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d01034d0 0x7f90d019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d4af4640 1 -- 172.21.15.105:0/1448054640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90d0101920 msgr2=0x7f90d0197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d4af4640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90d0101920 0x7f90d0197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d4af4640 1 -- 172.21.15.105:0/1448054640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d019f630 con 0x7f90d0102b10 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d52f5640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90d0101920 0x7f90d0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.027+0000 7f90d5af6640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d01034d0 0x7f90d019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.028+0000 7f90d4af4640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90d0102b10 0x7f90d0197fa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006010 tx=0x7f90c0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.028+0000 7f90c67fc640 1 -- 172.21.15.105:0/1448054640 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0043020 con 0x7f90d0102b10 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.028+0000 7f90d7559640 1 -- 172.21.15.105:0/1448054640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d019f8c0 con 0x7f90d0102b10 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.028+0000 7f90c67fc640 1 -- 172.21.15.105:0/1448054640 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90c00043d0 con 0x7f90d0102b10 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.028+0000 7f90c67fc640 1 -- 172.21.15.105:0/1448054640 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0003b80 con 0x7f90d0102b10 2024-03-26T17:43:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.028+0000 7f90d7559640 1 -- 172.21.15.105:0/1448054640 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d019fe60 con 0x7f90d0102b10 2024-03-26T17:43:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.031+0000 7f90c67fc640 1 -- 172.21.15.105:0/1448054640 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f90c0046020 con 0x7f90d0102b10 2024-03-26T17:43:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.031+0000 7f90da214740 1 Processor -- start 2024-03-26T17:43:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.031+0000 7f90da214740 1 -- start start 2024-03-26T17:43:57.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.031+0000 7f90da214740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d77595d0 0x5613d77599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.031+0000 7f90da214740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d7759fc0 0x5613d775eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.031+0000 7f90da214740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 0x5613d77615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90da214740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5613d76be0b0 con 0x5613d775f130 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90da214740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613d7666960 con 0x5613d7759fc0 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90da214740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613d76ba060 con 0x5613d77595d0 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d77595d0 0x5613d77599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d52f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d7759fc0 0x5613d775eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d77595d0 0x5613d77599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33966/0 (socket says 172.21.15.105:33966) 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 -- 172.21.15.105:0/3593885785 learned_addr learned my addr 172.21.15.105:0/3593885785 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d4af4640 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 0x5613d77615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 -- 172.21.15.105:0/3593885785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d7759fc0 msgr2=0x5613d775eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d7759fc0 0x5613d775eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 -- 172.21.15.105:0/3593885785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 msgr2=0x5613d77615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 0x5613d77615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 -- 172.21.15.105:0/3593885785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d7678c60 con 0x5613d77595d0 2024-03-26T17:43:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90c67fc640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90bc0a3040 0x7f90bc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d52f5640 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d7759fc0 0x5613d775eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 165047 3.14 MB/sec execute 160 sec latency 20.218 ms 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90c67fc640 1 -- 172.21.15.105:0/1448054640 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f90c00f1590 con 0x7f90d0102b10 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d52f5640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90bc0a3040 0x7f90bc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.032+0000 7f90d5af6640 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d77595d0 0x5613d77599f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90c80117e0 tx=0x7f90c8011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c73b65defc73d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.033+0000 7f90bb7fe640 1 -- 172.21.15.105:0/3593885785 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8011310 con 0x5613d77595d0 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.033+0000 7f90bb7fe640 1 -- 172.21.15.105:0/3593885785 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90c8003e20 con 0x5613d77595d0 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.033+0000 7f90bb7fe640 1 -- 172.21.15.105:0/3593885785 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c80044f0 con 0x5613d77595d0 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.033+0000 7f90da214740 1 -- 172.21.15.105:0/3593885785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d77595d0 msgr2=0x5613d77599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.033+0000 7f90d52f5640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90bc0a3040 0x7f90bc0a5500 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009fd0 tx=0x7f90cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.033+0000 7f90da214740 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d77595d0 0x5613d77599f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f90c80117e0 tx=0x7f90c8011ca0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 -- 172.21.15.105:0/3593885785 shutdown_connections 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 0x5613d77615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d7759fc0 0x5613d775eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 --2- 172.21.15.105:0/3593885785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d77595d0 0x5613d77599f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 -- 172.21.15.105:0/3593885785 >> 172.21.15.105:0/3593885785 conn(0x5613d7754b40 msgr2=0x5613d7754f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 -- 172.21.15.105:0/3593885785 shutdown_connections 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 -- 172.21.15.105:0/3593885785 wait complete. 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 -- start start 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.034+0000 7f90da214740 1 Processor -- start 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90da214740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d77595d0 0x5613d7763410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90da214740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7763950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90da214740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 0x5613d776a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90da214740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5613d77620e0 con 0x5613d775f130 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90da214740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613d7679db0 con 0x5613d77595d0 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90da214740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5613d7762260 con 0x5613d7759fc0 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7763950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7763950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33982/0 (socket says 172.21.15.105:33982) 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 -- 172.21.15.105:0/467802816 learned_addr learned my addr 172.21.15.105:0/467802816 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d5af6640 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 0x5613d776a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90da214740 10 client.0 _ll_register_callbacks cb 0x5613d767e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d52f5640 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d77595d0 0x5613d7763410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 -- 172.21.15.105:0/467802816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d77595d0 msgr2=0x5613d7763410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d77595d0 0x5613d7763410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 -- 172.21.15.105:0/467802816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 msgr2=0x5613d776a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 0x5613d776a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 -- 172.21.15.105:0/467802816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d73d2210 con 0x5613d7759fc0 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90d4af4640 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7763950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f90c00ba0e0 tx=0x7f90c0009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30932252d5191a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.035+0000 7f90b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90b8ff9640 1 -- 172.21.15.105:0/467802816 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00bd7c0 con 0x5613d7759fc0 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 1 -- 172.21.15.105:0/467802816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c8003e20 con 0x5613d7759fc0 2024-03-26T17:43:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90b8ff9640 1 -- 172.21.15.105:0/467802816 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f90c00c6200 con 0x5613d7759fc0 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90b8ff9640 1 -- 172.21.15.105:0/467802816 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90c00054b0 con 0x5613d7759fc0 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 10 client.27331 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 1 -- 172.21.15.105:0/467802816 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613d749e620 con 0x5613d7759fc0 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90b8ff9640 1 -- 172.21.15.105:0/467802816 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90c8003e20 con 0x5613d7759fc0 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90b8ff9640 1 -- 172.21.15.105:0/467802816 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f90c00f1110 con 0x5613d7759fc0 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 10 client.27331 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 10 client.27331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 20 client.27331 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613d776f2b0 0x5613d77716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 4 client.27331 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90da214740 1 -- 172.21.15.105:0/467802816 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d76cb5f0 con 0x5613d776f2b0 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.036+0000 7f90d52f5640 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613d776f2b0 0x5613d77716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.037+0000 7f90d52f5640 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613d776f2b0 0x5613d77716f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fecf898bdb7f2b1 server_cookie=e70a47108b59bb65 in_seq=0 out_seq=0 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.037+0000 7f90b8ff9640 10 client.27331 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90b8ff9640 1 -- 172.21.15.105:0/467802816 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5613d76cb5f0 con 0x5613d776f2b0 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90b8ff9640 10 client.27331 handle_command_reply: tid=0 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 client.27331 shutdown 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 -- 172.21.15.105:0/467802816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 msgr2=0x5613d7763950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7763950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f90c00ba0e0 tx=0x7f90c0009560 comp rx=0 tx=0).stop 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 -- 172.21.15.105:0/467802816 shutdown_connections 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613d776f2b0 0x5613d77716f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d775f130 0x5613d776a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7763950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 --2- 172.21.15.105:0/467802816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d77595d0 0x5613d7763410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 -- 172.21.15.105:0/467802816 >> 172.21.15.105:0/467802816 conn(0x5613d7754b40 msgr2=0x5613d7753df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90b8ff9640 0 client.27331 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 -- 172.21.15.105:0/467802816 shutdown_connections 2024-03-26T17:43:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 1 -- 172.21.15.105:0/467802816 wait complete. 2024-03-26T17:43:57.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.038+0000 7f90da214740 20 client.27331 trim_cache size 0 max 16384 2024-03-26T17:43:57.036 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:56 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1910640440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:43:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.056+0000 7f90da214740 1 Processor -- start 2024-03-26T17:43:57.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.056+0000 7f90da214740 1 -- start start 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90da214740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d775aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90da214740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d776a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90da214740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d775afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90da214740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01a8ba0 con 0x5613d77595d0 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90da214740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01aa6e0 con 0x5613d775f130 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90da214740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01a9270 con 0x5613d7759fc0 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d52f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d775aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d776a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d776a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33990/0 (socket says 172.21.15.105:33990) 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 -- 172.21.15.105:0/3665702944 learned_addr learned my addr 172.21.15.105:0/3665702944 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d5af6640 1 --2- 172.21.15.105:0/3665702944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d775afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 -- 172.21.15.105:0/3665702944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 msgr2=0x5613d775afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 --2- 172.21.15.105:0/3665702944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d775afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 -- 172.21.15.105:0/3665702944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 msgr2=0x5613d775aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 --2- 172.21.15.105:0/3665702944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d775aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 -- 172.21.15.105:0/3665702944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d77555b0 con 0x5613d7759fc0 2024-03-26T17:43:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.057+0000 7f90d4af4640 1 --2- 172.21.15.105:0/3665702944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d776a950 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90c0005ab0 tx=0x7f90c00f4bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c9230c631494ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90baffd640 1 -- 172.21.15.105:0/3665702944 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00bd5e0 con 0x5613d7759fc0 2024-03-26T17:43:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90baffd640 1 -- 172.21.15.105:0/3665702944 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90c00be380 con 0x5613d7759fc0 2024-03-26T17:43:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90baffd640 1 -- 172.21.15.105:0/3665702944 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c00c6460 con 0x5613d7759fc0 2024-03-26T17:43:57.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90da214740 1 -- 172.21.15.105:0/3665702944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 msgr2=0x5613d776a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90da214740 1 --2- 172.21.15.105:0/3665702944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d776a950 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90c0005ab0 tx=0x7f90c00f4bd0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90da214740 1 -- 172.21.15.105:0/3665702944 shutdown_connections 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90da214740 1 --2- 172.21.15.105:0/3665702944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d775afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90da214740 1 --2- 172.21.15.105:0/3665702944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d776a950 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90da214740 1 --2- 172.21.15.105:0/3665702944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d775aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.058+0000 7f90da214740 1 -- 172.21.15.105:0/3665702944 >> 172.21.15.105:0/3665702944 conn(0x5613d7754b40 msgr2=0x5613d7763410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 -- 172.21.15.105:0/3665702944 shutdown_connections 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 -- 172.21.15.105:0/3665702944 wait complete. 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 -- start start 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 Processor -- start 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d77680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7768600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d7769610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01aa010 con 0x5613d77595d0 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01a9940 con 0x5613d775f130 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01a6b80 con 0x5613d7759fc0 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90da214740 10 client.0 _ll_register_callbacks cb 0x7f90c0004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.059+0000 7f90d52f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d77680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d52f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d77680c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51156/0 (socket says 172.21.15.105:51156) 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d52f5640 1 -- 172.21.15.105:0/2110368024 learned_addr learned my addr 172.21.15.105:0/2110368024 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d5af6640 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d7769610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d4af4640 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7768600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d52f5640 1 -- 172.21.15.105:0/2110368024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 msgr2=0x5613d7768600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d52f5640 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7768600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d52f5640 1 -- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 msgr2=0x5613d7769610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d52f5640 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d7769610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d52f5640 1 -- 172.21.15.105:0/2110368024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d7769c00 con 0x5613d77595d0 2024-03-26T17:43:57.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d5af6640 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d7769610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90d52f5640 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d77680c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x5613d775aa30 tx=0x7f90cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3c3cb4287dbc7e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90b8ff9640 1 -- 172.21.15.105:0/2110368024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0044d0 con 0x5613d77595d0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90b8ff9640 1 -- 172.21.15.105:0/2110368024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f90cc0122c0 con 0x5613d77595d0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90da214740 1 -- 172.21.15.105:0/2110368024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c00be380 con 0x5613d77595d0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90b8ff9640 1 -- 172.21.15.105:0/2110368024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90cc02ab20 con 0x5613d77595d0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90da214740 10 client.17502 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.060+0000 7f90da214740 1 -- 172.21.15.105:0/2110368024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613d77d9b70 con 0x5613d77595d0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90b8ff9640 1 -- 172.21.15.105:0/2110368024 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90c00be380 con 0x5613d77595d0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90b8ff9640 1 -- 172.21.15.105:0/2110368024 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f90cc0052b0 con 0x5613d77595d0 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90da214740 10 client.17502 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90da214740 10 client.17502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90da214740 20 client.17502 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90da214740 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613d77da5a0 0x5613d77dc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90da214740 4 client.17502 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:43:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90da214740 1 -- 172.21.15.105:0/2110368024 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613d7763200 con 0x5613d77da5a0 2024-03-26T17:43:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.061+0000 7f90d4af4640 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613d77da5a0 0x5613d77dc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.062+0000 7f90b8ff9640 10 client.17502 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:57.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.062+0000 7f90d4af4640 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613d77da5a0 0x5613d77dc9c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ed11902283f4c8d server_cookie=1a56061a21fdafd4 in_seq=0 out_seq=0 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90b8ff9640 1 -- 172.21.15.105:0/2110368024 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613d7763200 con 0x5613d77da5a0 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90b8ff9640 10 client.17502 handle_command_reply: tid=0 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 client.17502 shutdown 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 -- 172.21.15.105:0/2110368024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 msgr2=0x5613d77680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d77680c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x5613d775aa30 tx=0x7f90cc004730 comp rx=0 tx=0).stop 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 -- 172.21.15.105:0/2110368024 shutdown_connections 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5613d77da5a0 0x5613d77dc9c0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90b8ff9640 0 client.17502 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613d775f130 0x5613d7769610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5613d7759fc0 0x5613d7768600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 --2- 172.21.15.105:0/2110368024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5613d77595d0 0x5613d77680c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 -- 172.21.15.105:0/2110368024 >> 172.21.15.105:0/2110368024 conn(0x5613d7754b40 msgr2=0x5613d775aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 -- 172.21.15.105:0/2110368024 shutdown_connections 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 1 -- 172.21.15.105:0/2110368024 wait complete. 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.101+0000 7f90da214740 20 client.17502 trim_cache size 0 max 16384 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:43:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:43:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 -- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90bc0a3040 msgr2=0x7f90bc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90bc0a3040 0x7f90bc0a5500 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009fd0 tx=0x7f90cc005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 -- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90d0102b10 msgr2=0x7f90d0197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90d0102b10 0x7f90d0197fa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006010 tx=0x7f90c0005e40 comp rx=0 tx=0).stop 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 -- 172.21.15.105:0/1448054640 shutdown_connections 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90bc0a3040 0x7f90bc0a5500 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90d01034d0 0x7f90d019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90d0102b10 0x7f90d0197fa0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 --2- 172.21.15.105:0/1448054640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90d0101920 0x7f90d0197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 -- 172.21.15.105:0/1448054640 >> 172.21.15.105:0/1448054640 conn(0x7f90d00fce90 msgr2=0x7f90d0105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 -- 172.21.15.105:0/1448054640 shutdown_connections 2024-03-26T17:43:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:43:57.102+0000 7f90d7559640 1 -- 172.21.15.105:0/1448054640 wait complete. 2024-03-26T17:43:57.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:56 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1910640440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:43:57.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.151+0000 7efd2224b640 1 -- 172.21.15.105:0/3554560585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c102370 msgr2=0x7efd1c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.151+0000 7efd2224b640 1 --2- 172.21.15.105:0/3554560585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c102370 0x7efd1c108c00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7efd14009ca0 tx=0x7efd1402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 -- 172.21.15.105:0/3554560585 shutdown_connections 2024-03-26T17:43:57.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 --2- 172.21.15.105:0/3554560585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c102370 0x7efd1c108c00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 --2- 172.21.15.105:0/3554560585 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1c1019b0 0x7efd1c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 --2- 172.21.15.105:0/3554560585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd1c1007c0 0x7efd1c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 -- 172.21.15.105:0/3554560585 >> 172.21.15.105:0/3554560585 conn(0x7efd1c0fbd50 msgr2=0x7efd1c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 -- 172.21.15.105:0/3554560585 shutdown_connections 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 -- 172.21.15.105:0/3554560585 wait complete. 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 Processor -- start 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 -- start start 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c1007c0 0x7efd1c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.152+0000 7efd2224b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd1c1019b0 0x7efd1c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd2224b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1c102370 0x7efd1c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd2224b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10c2f0 con 0x7efd1c1007c0 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd2224b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10c170 con 0x7efd1c102370 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd2224b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c10c470 con 0x7efd1c1019b0 2024-03-26T17:43:57.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c1007c0 0x7efd1c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c1007c0 0x7efd1c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51172/0 (socket says 172.21.15.105:51172) 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 -- 172.21.15.105:0/3772293312 learned_addr learned my addr 172.21.15.105:0/3772293312 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1bfff640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1c102370 0x7efd1c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1affd640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd1c1019b0 0x7efd1c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 -- 172.21.15.105:0/3772293312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd1c1019b0 msgr2=0x7efd1c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd1c1019b0 0x7efd1c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 -- 172.21.15.105:0/3772293312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1c102370 msgr2=0x7efd1c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1c102370 0x7efd1c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 -- 172.21.15.105:0/3772293312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c19f750 con 0x7efd1c1007c0 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c1007c0 0x7efd1c197b80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00e9c0 tx=0x7efd0c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd18ff9640 1 -- 172.21.15.105:0/3772293312 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c00cd80 con 0x7efd1c1007c0 2024-03-26T17:43:57.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd18ff9640 1 -- 172.21.15.105:0/3772293312 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd0c0035f0 con 0x7efd1c1007c0 2024-03-26T17:43:57.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd2224b640 1 -- 172.21.15.105:0/3772293312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c19fa40 con 0x7efd1c1007c0 2024-03-26T17:43:57.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd18ff9640 1 -- 172.21.15.105:0/3772293312 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c010690 con 0x7efd1c1007c0 2024-03-26T17:43:57.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.153+0000 7efd2224b640 1 -- 172.21.15.105:0/3772293312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c19ff40 con 0x7efd1c1007c0 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd24e32740 1 Processor -- start 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd18ff9640 1 -- 172.21.15.105:0/3772293312 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7efd0c0037a0 con 0x7efd1c1007c0 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd24e32740 1 -- start start 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd24e32740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd24e32740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd24e32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd24e32740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bf5d82f970 con 0x55bf5d8caa30 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd24e32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bf5d7d7d70 con 0x55bf5d8d0590 2024-03-26T17:43:57.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd24e32740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf5d82b920 con 0x55bf5d8cb420 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd1bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.156+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8d2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52662/0 (socket says 172.21.15.105:52662) 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1affd640 1 -- 172.21.15.105:0/107656218 learned_addr learned my addr 172.21.15.105:0/107656218 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd18ff9640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd040a2f20 0x7efd040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1b7fe640 1 -- 172.21.15.105:0/107656218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 msgr2=0x55bf5d8d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/107656218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1b7fe640 1 -- 172.21.15.105:0/107656218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 msgr2=0x55bf5d8cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/107656218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1b7fe640 1 -- 172.21.15.105:0/107656218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5d5433f0 con 0x55bf5d8cb420 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1bfff640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd040a2f20 0x7efd040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd18ff9640 1 -- 172.21.15.105:0/3772293312 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7efd0c0cf930 con 0x7efd1c1007c0 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/107656218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8cff20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0cd3c0 tx=0x7efd0c0b02b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40257521b5c54894 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd01ffb640 1 -- 172.21.15.105:0/107656218 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c09bdf0 con 0x55bf5d8cb420 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd01ffb640 1 -- 172.21.15.105:0/107656218 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd0c003b10 con 0x55bf5d8cb420 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.157+0000 7efd01ffb640 1 -- 172.21.15.105:0/107656218 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c09b980 con 0x55bf5d8cb420 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd24e32740 1 -- 172.21.15.105:0/107656218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 msgr2=0x55bf5d8cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd1bfff640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd040a2f20 0x7efd040a53e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efd14006010 tx=0x7efd1402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd24e32740 1 --2- 172.21.15.105:0/107656218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8cff20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0cd3c0 tx=0x7efd0c0b02b0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd24e32740 1 -- 172.21.15.105:0/107656218 shutdown_connections 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd24e32740 1 --2- 172.21.15.105:0/107656218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd24e32740 1 --2- 172.21.15.105:0/107656218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8cff20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd24e32740 1 --2- 172.21.15.105:0/107656218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd24e32740 1 -- 172.21.15.105:0/107656218 >> 172.21.15.105:0/107656218 conn(0x55bf5d8c5fa0 msgr2=0x55bf5d8c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.158+0000 7efd24e32740 1 -- 172.21.15.105:0/107656218 shutdown_connections 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 -- 172.21.15.105:0/107656218 wait complete. 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 -- start start 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 Processor -- start 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8d4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bf5d8d34e0 con 0x55bf5d8cb420 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bf5d7eb310 con 0x55bf5d8d0590 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.159+0000 7efd24e32740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf5d8d3660 con 0x55bf5d8caa30 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd24e32740 10 client.0 _ll_register_callbacks cb 0x55bf5d826c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8d53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51186/0 (socket says 172.21.15.105:51186) 2024-03-26T17:43:57.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 -- 172.21.15.105:0/1572249053 learned_addr learned my addr 172.21.15.105:0/1572249053 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1bfff640 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8d4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 -- 172.21.15.105:0/1572249053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 msgr2=0x55bf5d8d4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8d4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 -- 172.21.15.105:0/1572249053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 msgr2=0x55bf5d8dc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8dc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 -- 172.21.15.105:0/1572249053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5d60d7d0 con 0x55bf5d8cb420 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd1affd640 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8d53e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efd10012740 tx=0x7efd10012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aab4f07bf0c091 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efcfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efcfaffd640 1 -- 172.21.15.105:0/1572249053 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10004050 con 0x55bf5d8cb420 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efcfaffd640 1 -- 172.21.15.105:0/1572249053 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7efd100041f0 con 0x55bf5d8cb420 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd24e32740 1 -- 172.21.15.105:0/1572249053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf5d83ceb0 con 0x55bf5d8cb420 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efcfaffd640 1 -- 172.21.15.105:0/1572249053 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd100043f0 con 0x55bf5d8cb420 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd24e32740 10 client.17514 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:57.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.160+0000 7efd24e32740 1 -- 172.21.15.105:0/1572249053 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf5d8df860 con 0x55bf5d8cb420 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efcfaffd640 1 -- 172.21.15.105:0/1572249053 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bf5d83ceb0 con 0x55bf5d8cb420 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efcfaffd640 1 -- 172.21.15.105:0/1572249053 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efd10003b90 con 0x55bf5d8cb420 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efd24e32740 10 client.17514 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efd24e32740 10 client.17514 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efd24e32740 20 client.17514 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efd24e32740 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bf5d8e0f20 0x55bf5d8e3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efd24e32740 4 client.17514 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efd24e32740 1 -- 172.21.15.105:0/1572249053 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf5d837200 con 0x55bf5d8e0f20 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bf5d8e0f20 0x55bf5d8e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efcfaffd640 10 client.17514 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:43:57.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.161+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bf5d8e0f20 0x55bf5d8e3350 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71e45fbdc903a3fe server_cookie=e3aa0802d14353e2 in_seq=0 out_seq=0 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.162+0000 7efcfaffd640 1 -- 172.21.15.105:0/1572249053 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bf5d837200 con 0x55bf5d8e0f20 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.162+0000 7efcfaffd640 10 client.17514 handle_command_reply: tid=0 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.162+0000 7efd24e32740 1 client.17514 shutdown 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.162+0000 7efd24e32740 1 -- 172.21.15.105:0/1572249053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 msgr2=0x55bf5d8d53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.162+0000 7efd24e32740 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8d53e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efd10012740 tx=0x7efd10012c00 comp rx=0 tx=0).stop 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.162+0000 7efd24e32740 1 -- 172.21.15.105:0/1572249053 shutdown_connections 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.162+0000 7efd24e32740 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bf5d8e0f20 0x55bf5d8e3350 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.163+0000 7efcfaffd640 0 client.17514 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.163+0000 7efd24e32740 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8dc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.163+0000 7efd24e32740 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8d53e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.163+0000 7efd24e32740 1 --2- 172.21.15.105:0/1572249053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8d4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.163+0000 7efd24e32740 1 -- 172.21.15.105:0/1572249053 >> 172.21.15.105:0/1572249053 conn(0x55bf5d8c5fa0 msgr2=0x55bf5d8c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.163+0000 7efd24e32740 1 -- 172.21.15.105:0/1572249053 shutdown_connections 2024-03-26T17:43:57.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.163+0000 7efd24e32740 1 -- 172.21.15.105:0/1572249053 wait complete. 2024-03-26T17:43:57.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.163+0000 7efd24e32740 20 client.17514 trim_cache size 0 max 16384 2024-03-26T17:43:57.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:56 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1910640440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:43:57.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd24e32740 1 Processor -- start 2024-03-26T17:43:57.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd24e32740 1 -- start start 2024-03-26T17:43:57.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd24e32740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd24e32740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd24e32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd24e32740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1a6c50 con 0x55bf5d8caa30 2024-03-26T17:43:57.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd24e32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1a8790 con 0x55bf5d8d0590 2024-03-26T17:43:57.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd24e32740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1a7320 con 0x55bf5d8cb420 2024-03-26T17:43:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51188/0 (socket says 172.21.15.105:51188) 2024-03-26T17:43:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd1b7fe640 1 -- 172.21.15.105:0/1517943786 learned_addr learned my addr 172.21.15.105:0/1517943786 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd1affd640 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd1bfff640 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd1affd640 1 -- 172.21.15.105:0/1517943786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 msgr2=0x55bf5d8e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.185+0000 7efd1affd640 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.186+0000 7efd1affd640 1 -- 172.21.15.105:0/1517943786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 msgr2=0x55bf5d8c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.186+0000 7efd1affd640 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.186+0000 7efd1affd640 1 -- 172.21.15.105:0/1517943786 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf5d8c6d10 con 0x55bf5d8cb420 2024-03-26T17:43:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.186+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.186+0000 7efd1affd640 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8dc3e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efd10009e60 tx=0x7efd10010880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8cf5958e6b3d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.186+0000 7efd017fa640 1 -- 172.21.15.105:0/1517943786 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10010af0 con 0x55bf5d8cb420 2024-03-26T17:43:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.186+0000 7efd017fa640 1 -- 172.21.15.105:0/1517943786 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd10022030 con 0x55bf5d8cb420 2024-03-26T17:43:57.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.186+0000 7efd017fa640 1 -- 172.21.15.105:0/1517943786 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10005290 con 0x55bf5d8cb420 2024-03-26T17:43:57.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 -- 172.21.15.105:0/1517943786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 msgr2=0x55bf5d8dc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8dc3e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efd10009e60 tx=0x7efd10010880 comp rx=0 tx=0).stop 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 -- 172.21.15.105:0/1517943786 shutdown_connections 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8cb420 0x55bf5d8dc3e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 --2- 172.21.15.105:0/1517943786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8caa30 0x55bf5d8c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 -- 172.21.15.105:0/1517943786 >> 172.21.15.105:0/1517943786 conn(0x55bf5d8c5fa0 msgr2=0x55bf5d8c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 -- 172.21.15.105:0/1517943786 shutdown_connections 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 -- 172.21.15.105:0/1517943786 wait complete. 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 -- start start 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.187+0000 7efd24e32740 1 Processor -- start 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd24e32740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd24e32740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8da780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd24e32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd24e32740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1a80c0 con 0x55bf5d8cb420 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd24e32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1a79f0 con 0x55bf5d8d0590 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd24e32740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c1a4c30 con 0x55bf5d8caa30 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd24e32740 10 client.0 _ll_register_callbacks cb 0x7efd10009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8da780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8da240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34050/0 (socket says 172.21.15.105:34050) 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1b7fe640 1 -- 172.21.15.105:0/344453666 learned_addr learned my addr 172.21.15.105:0/344453666 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1bfff640 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1affd640 1 -- 172.21.15.105:0/344453666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 msgr2=0x55bf5d8da240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1affd640 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8da240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1affd640 1 -- 172.21.15.105:0/344453666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 msgr2=0x55bf5d8db790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1affd640 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8db790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1affd640 1 -- 172.21.15.105:0/344453666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf5d8dbd80 con 0x55bf5d8cb420 2024-03-26T17:43:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd1affd640 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8da780 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7efd10010010 tx=0x7efd10011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9afef7ab48b4c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efcfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efcfaffd640 1 -- 172.21.15.105:0/344453666 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1000f9a0 con 0x55bf5d8cb420 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efcfaffd640 1 -- 172.21.15.105:0/344453666 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7efd1000fb40 con 0x55bf5d8cb420 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.188+0000 7efd24e32740 1 -- 172.21.15.105:0/344453666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd10022030 con 0x55bf5d8cb420 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efcfaffd640 1 -- 172.21.15.105:0/344453666 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd10005070 con 0x55bf5d8cb420 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd24e32740 10 client.17520 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd24e32740 1 -- 172.21.15.105:0/344453666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf5d947970 con 0x55bf5d8cb420 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efcfaffd640 1 -- 172.21.15.105:0/344453666 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd10022030 con 0x55bf5d8cb420 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efcfaffd640 1 -- 172.21.15.105:0/344453666 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efd100049e0 con 0x55bf5d8cb420 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd24e32740 10 client.17520 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd24e32740 10 client.17520 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:43:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd24e32740 20 client.17520 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd24e32740 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bf5d8e0f20 0x55bf5d94a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd24e32740 4 client.17520 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd24e32740 1 -- 172.21.15.105:0/344453666 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf5d7b46a0 con 0x55bf5d8e0f20 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.189+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bf5d8e0f20 0x55bf5d94a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd1b7fe640 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bf5d8e0f20 0x55bf5d94a210 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d870b8cbbf293d79 server_cookie=e244654e679fecc6 in_seq=0 out_seq=0 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efcfaffd640 10 client.17520 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efcfaffd640 1 -- 172.21.15.105:0/344453666 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf5d7b46a0 con 0x55bf5d8e0f20 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efcfaffd640 10 client.17520 handle_command_reply: tid=0 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 client.17520 shutdown 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 -- 172.21.15.105:0/344453666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 msgr2=0x55bf5d8da780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8da780 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7efd10010010 tx=0x7efd10011d10 comp rx=0 tx=0).stop 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 -- 172.21.15.105:0/344453666 shutdown_connections 2024-03-26T17:43:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bf5d8e0f20 0x55bf5d94a210 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf5d8d0590 0x55bf5d8db790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf5d8cb420 0x55bf5d8da780 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efcfaffd640 0 client.17520 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 --2- 172.21.15.105:0/344453666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf5d8caa30 0x55bf5d8da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 -- 172.21.15.105:0/344453666 >> 172.21.15.105:0/344453666 conn(0x55bf5d8c5fa0 msgr2=0x55bf5d8c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 -- 172.21.15.105:0/344453666 shutdown_connections 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 1 -- 172.21.15.105:0/344453666 wait complete. 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.190+0000 7efd24e32740 20 client.17520 trim_cache size 0 max 16384 2024-03-26T17:43:57.191 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 -- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd040a2f20 msgr2=0x7efd040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd040a2f20 0x7efd040a53e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efd14006010 tx=0x7efd1402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 -- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c1007c0 msgr2=0x7efd1c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c1007c0 0x7efd1c197b80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00e9c0 tx=0x7efd0c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:43:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 -- 172.21.15.105:0/3772293312 shutdown_connections 2024-03-26T17:43:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efd040a2f20 0x7efd040a53e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efd1c102370 0x7efd1c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd1c1019b0 0x7efd1c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 --2- 172.21.15.105:0/3772293312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd1c1007c0 0x7efd1c197b80 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 -- 172.21.15.105:0/3772293312 >> 172.21.15.105:0/3772293312 conn(0x7efd1c0fbd50 msgr2=0x7efd1c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 -- 172.21.15.105:0/3772293312 shutdown_connections 2024-03-26T17:43:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.191+0000 7efd2224b640 1 -- 172.21.15.105:0/3772293312 wait complete. 2024-03-26T17:43:57.194 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:43:57.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 -- 172.21.15.105:0/3953513142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 msgr2=0x7f72f4108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 --2- 172.21.15.105:0/3953513142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 0x7f72f4108c00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009ca0 tx=0x7f72ec02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 -- 172.21.15.105:0/3953513142 shutdown_connections 2024-03-26T17:43:57.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 --2- 172.21.15.105:0/3953513142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 0x7f72f4108c00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 --2- 172.21.15.105:0/3953513142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f41019b0 0x7f72f4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 --2- 172.21.15.105:0/3953513142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72f41007c0 0x7f72f4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 -- 172.21.15.105:0/3953513142 >> 172.21.15.105:0/3953513142 conn(0x7f72f40fbd50 msgr2=0x7f72f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 -- 172.21.15.105:0/3953513142 shutdown_connections 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.369+0000 7f72fbd32640 1 -- 172.21.15.105:0/3953513142 wait complete. 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fbd32640 1 Processor -- start 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fbd32640 1 -- start start 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fbd32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72f41007c0 0x7f72f4104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fbd32640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f41019b0 0x7f72f4102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fbd32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 0x7f72f41032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fbd32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72f410c2f0 con 0x7f72f4102370 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fbd32640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72f410c170 con 0x7f72f41019b0 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fbd32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72f410c470 con 0x7f72f41007c0 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72f41007c0 0x7f72f4104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fa2cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 0x7f72f41032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72f92cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f41019b0 0x7f72f4102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72f41007c0 0x7f72f4104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34066/0 (socket says 172.21.15.105:34066) 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72f9ace640 1 -- 172.21.15.105:0/373599662 learned_addr learned my addr 172.21.15.105:0/373599662 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fa2cf640 1 -- 172.21.15.105:0/373599662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72f41007c0 msgr2=0x7f72f4104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fa2cf640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72f41007c0 0x7f72f4104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fa2cf640 1 -- 172.21.15.105:0/373599662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f41019b0 msgr2=0x7f72f4102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fa2cf640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f41019b0 0x7f72f4102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72fa2cf640 1 -- 172.21.15.105:0/373599662 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f4103b60 con 0x7f72f4102370 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.370+0000 7f72f92cd640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f41019b0 0x7f72f4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.371+0000 7f72fa2cf640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 0x7f72f41032d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f72ec02c630 tx=0x7f72ec0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.371+0000 7f72eaffd640 1 -- 172.21.15.105:0/373599662 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec041020 con 0x7f72f4102370 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.371+0000 7f72eaffd640 1 -- 172.21.15.105:0/373599662 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72ec003e20 con 0x7f72f4102370 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.371+0000 7f72fbd32640 1 -- 172.21.15.105:0/373599662 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f41a1bb0 con 0x7f72f4102370 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.371+0000 7f72eaffd640 1 -- 172.21.15.105:0/373599662 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec039040 con 0x7f72f4102370 2024-03-26T17:43:57.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.371+0000 7f72fbd32640 1 -- 172.21.15.105:0/373599662 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f41a2100 con 0x7f72f4102370 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.374+0000 7f72eaffd640 1 -- 172.21.15.105:0/373599662 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f72ec044020 con 0x7f72f4102370 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.374+0000 7f72fe9f3740 1 Processor -- start 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.374+0000 7f72fe9f3740 1 -- start start 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac9329be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac932a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac93200970 con 0x55ac932a1590 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ac931a8d70 con 0x55ac9329c420 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac931fc920 con 0x55ac9329ba30 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f92cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac9329be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72eaffd640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72e00a2f20 0x7f72e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52728/0 (socket says 172.21.15.105:52728) 2024-03-26T17:43:57.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 -- 172.21.15.105:0/226953038 learned_addr learned my addr 172.21.15.105:0/226953038 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f92cd640 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac932a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72fa2cf640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72e00a2f20 0x7f72e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 -- 172.21.15.105:0/226953038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 msgr2=0x55ac9329be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac9329be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 -- 172.21.15.105:0/226953038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 msgr2=0x55ac932a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac932a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72eaffd640 1 -- 172.21.15.105:0/373599662 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f72ec0f1590 con 0x7f72f4102370 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 -- 172.21.15.105:0/226953038 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac92f143f0 con 0x55ac9329c420 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f92cd640 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac9329be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.375+0000 7f72f9ace640 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a0f20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009d70 tx=0x7f72f0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c812efc3062d7515 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72fa2cf640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72e00a2f20 0x7f72e00a53e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0ef3c0 tx=0x7f72ec0887d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72dffff640 1 -- 172.21.15.105:0/226953038 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0004220 con 0x55ac9329c420 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72dffff640 1 -- 172.21.15.105:0/226953038 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72f00043c0 con 0x55ac9329c420 2024-03-26T17:43:57.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72dffff640 1 -- 172.21.15.105:0/226953038 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0019860 con 0x55ac9329c420 2024-03-26T17:43:57.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72fe9f3740 1 -- 172.21.15.105:0/226953038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 msgr2=0x55ac932a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a0f20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009d70 tx=0x7f72f0011810 comp rx=0 tx=0).stop 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72fe9f3740 1 -- 172.21.15.105:0/226953038 shutdown_connections 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac932a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a0f20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/226953038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac9329be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.376+0000 7f72fe9f3740 1 -- 172.21.15.105:0/226953038 >> 172.21.15.105:0/226953038 conn(0x55ac93296fa0 msgr2=0x55ac932973a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 -- 172.21.15.105:0/226953038 shutdown_connections 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 -- 172.21.15.105:0/226953038 wait complete. 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 -- start start 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 Processor -- start 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329ba30 0x55ac932a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac932ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac932a4540 con 0x55ac9329ba30 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ac931bc310 con 0x55ac9329c420 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac932a46c0 con 0x55ac932a1590 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72fe9f3740 10 client.0 _ll_register_callbacks cb 0x55ac931f7c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72f92cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329ba30 0x55ac932a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72f92cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329ba30 0x55ac932a5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51228/0 (socket says 172.21.15.105:51228) 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52734/0 (socket says 172.21.15.105:52734) 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.377+0000 7f72f92cd640 1 -- 172.21.15.105:0/2173306259 learned_addr learned my addr 172.21.15.105:0/2173306259 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72f92cd640 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac932ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72f92cd640 1 -- 172.21.15.105:0/2173306259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 msgr2=0x55ac932ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72f92cd640 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac932ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72f92cd640 1 -- 172.21.15.105:0/2173306259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 msgr2=0x55ac932a6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72f92cd640 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72f92cd640 1 -- 172.21.15.105:0/2173306259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac92fde7d0 con 0x55ac9329ba30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72f9ace640 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72f92cd640 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329ba30 0x55ac932a5f50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009230 tx=0x7f72e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17fb9daeb0a31b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2173306259 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4003db0 con 0x55ac9329ba30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2173306259 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f72e401e3c0 con 0x55ac9329ba30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2173306259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72f00043c0 con 0x55ac9329ba30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2173306259 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72e401e5c0 con 0x55ac9329ba30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72fe9f3740 10 client.17532 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.378+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2173306259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac932b0840 con 0x55ac9329ba30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2173306259 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72f00043c0 con 0x55ac9329ba30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2173306259 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f72e402b6b0 con 0x55ac9329ba30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72fe9f3740 10 client.17532 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72fe9f3740 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72fe9f3740 20 client.17532 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:57.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac932b1e80 0x55ac932b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72fe9f3740 4 client.17532 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2173306259 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac9320deb0 con 0x55ac932b1e80 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72f9ace640 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac932b1e80 0x55ac932b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72dd7fa640 10 client.17532 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.379+0000 7f72f9ace640 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac932b1e80 0x55ac932b4320 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597a7f8d8cafa9c8 server_cookie=7038a000ea407ea5 in_seq=0 out_seq=0 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.380+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2173306259 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ac9320deb0 con 0x55ac932b1e80 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.380+0000 7f72dd7fa640 10 client.17532 handle_command_reply: tid=0 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.380+0000 7f72fe9f3740 1 client.17532 shutdown 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.380+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2173306259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329ba30 msgr2=0x55ac932a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.380+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329ba30 0x55ac932a5f50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009230 tx=0x7f72e4004730 comp rx=0 tx=0).stop 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2173306259 shutdown_connections 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac932b1e80 0x55ac932b4320 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac932ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932a6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72dd7fa640 0 client.17532 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2173306259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329ba30 0x55ac932a5f50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2173306259 >> 172.21.15.105:0/2173306259 conn(0x55ac93296fa0 msgr2=0x55ac932961c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2173306259 shutdown_connections 2024-03-26T17:43:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2173306259 wait complete. 2024-03-26T17:43:57.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.381+0000 7f72fe9f3740 20 client.17532 trim_cache size 0 max 16384 2024-03-26T17:43:57.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72fe9f3740 1 Processor -- start 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72fe9f3740 1 -- start start 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac932a6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932ad420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac93295930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41aaf80 con 0x55ac932a1590 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41acac0 con 0x55ac9329c420 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41ab650 con 0x55ac9329ba30 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac93295930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac93295930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51236/0 (socket says 172.21.15.105:51236) 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 -- 172.21.15.105:0/3278231290 learned_addr learned my addr 172.21.15.105:0/3278231290 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac932a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 -- 172.21.15.105:0/3278231290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 msgr2=0x55ac932a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac932a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 -- 172.21.15.105:0/3278231290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 msgr2=0x55ac932ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 -- 172.21.15.105:0/3278231290 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac93292820 con 0x55ac932a1590 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f9ace640 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72f92cd640 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac93295930 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f72e40092e0 tx=0x7f72e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88fb4fa4329fef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.396+0000 7f72df7fe640 1 -- 172.21.15.105:0/3278231290 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4003bb0 con 0x55ac932a1590 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72df7fe640 1 -- 172.21.15.105:0/3278231290 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e4032070 con 0x55ac932a1590 2024-03-26T17:43:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72df7fe640 1 -- 172.21.15.105:0/3278231290 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e402b7a0 con 0x55ac932a1590 2024-03-26T17:43:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 -- 172.21.15.105:0/3278231290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 msgr2=0x55ac93295930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac93295930 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f72e40092e0 tx=0x7f72e40046c0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 -- 172.21.15.105:0/3278231290 shutdown_connections 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac932a1590 0x55ac93295930 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329c420 0x55ac932ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/3278231290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac9329ba30 0x55ac932a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 -- 172.21.15.105:0/3278231290 >> 172.21.15.105:0/3278231290 conn(0x55ac93296fa0 msgr2=0x55ac932a5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 -- 172.21.15.105:0/3278231290 shutdown_connections 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.397+0000 7f72fe9f3740 1 -- 172.21.15.105:0/3278231290 wait complete. 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 1 -- start start 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 1 Processor -- start 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329ba30 0x55ac932aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329c420 0x55ac932aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac93316210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41ac3f0 con 0x55ac9329c420 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41abd20 con 0x55ac9329ba30 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72f41a8f60 con 0x55ac932a1590 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72fe9f3740 10 client.0 _ll_register_callbacks cb 0x7f72e4004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329c420 0x55ac932aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72f92cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329ba30 0x55ac932aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72f9ace640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329c420 0x55ac932aa990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51246/0 (socket says 172.21.15.105:51246) 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72f9ace640 1 -- 172.21.15.105:0/2385837487 learned_addr learned my addr 172.21.15.105:0/2385837487 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.398+0000 7f72f92cd640 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac93316210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72f9ace640 1 -- 172.21.15.105:0/2385837487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 msgr2=0x55ac93316210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72f9ace640 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac93316210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72f9ace640 1 -- 172.21.15.105:0/2385837487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329ba30 msgr2=0x55ac932aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72f9ace640 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329ba30 0x55ac932aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72f9ace640 1 -- 172.21.15.105:0/2385837487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac93316750 con 0x55ac9329c420 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72f92cd640 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac93316210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72f9ace640 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329c420 0x55ac932aa990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009250 tx=0x7f72f0009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9ea1bf92a17784 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2385837487 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0030030 con 0x55ac9329c420 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2385837487 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f72f001f040 con 0x55ac9329c420 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2385837487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e4032070 con 0x55ac9329c420 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2385837487 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72f002ca30 con 0x55ac9329c420 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72fe9f3740 10 client.17538 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.399+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2385837487 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac93317a20 con 0x55ac9329c420 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2385837487 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72e4032070 con 0x55ac9329c420 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2385837487 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f72f002c240 con 0x55ac9329c420 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72fe9f3740 10 client.17538 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:57.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72fe9f3740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72fe9f3740 20 client.17538 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac933181d0 0x55ac9331a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72fe9f3740 4 client.17538 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2385837487 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac93208200 con 0x55ac933181d0 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.400+0000 7f72f92cd640 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac933181d0 0x55ac9331a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72dd7fa640 10 client.17538 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72f92cd640 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac933181d0 0x55ac9331a5c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1582ba501d56382 server_cookie=32cdfd789840a5be in_seq=0 out_seq=0 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72dd7fa640 1 -- 172.21.15.105:0/2385837487 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac93208200 con 0x55ac933181d0 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72dd7fa640 10 client.17538 handle_command_reply: tid=0 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 client.17538 shutdown 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2385837487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329c420 msgr2=0x55ac932aa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329c420 0x55ac932aa990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009250 tx=0x7f72f0009960 comp rx=0 tx=0).stop 2024-03-26T17:43:57.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2385837487 shutdown_connections 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac933181d0 0x55ac9331a5c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac932a1590 0x55ac93316210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72dd7fa640 0 client.17538 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac9329c420 0x55ac932aa990 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 --2- 172.21.15.105:0/2385837487 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac9329ba30 0x55ac932aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2385837487 >> 172.21.15.105:0/2385837487 conn(0x55ac93296fa0 msgr2=0x55ac9329a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2385837487 shutdown_connections 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 1 -- 172.21.15.105:0/2385837487 wait complete. 2024-03-26T17:43:57.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.401+0000 7f72fe9f3740 20 client.17538 trim_cache size 0 max 16384 2024-03-26T17:43:57.400 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 -- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72e00a2f20 msgr2=0x7f72e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72e00a2f20 0x7f72e00a53e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0ef3c0 tx=0x7f72ec0887d0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 -- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 msgr2=0x7f72f41032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 0x7f72f41032d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f72ec02c630 tx=0x7f72ec0328a0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 -- 172.21.15.105:0/373599662 shutdown_connections 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72e00a2f20 0x7f72e00a53e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72f4102370 0x7f72f41032d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72f41019b0 0x7f72f4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 --2- 172.21.15.105:0/373599662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72f41007c0 0x7f72f4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:57.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 -- 172.21.15.105:0/373599662 >> 172.21.15.105:0/373599662 conn(0x7f72f40fbd50 msgr2=0x7f72f40fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:57.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 -- 172.21.15.105:0/373599662 shutdown_connections 2024-03-26T17:43:57.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:57.402+0000 7f72fbd32640 1 -- 172.21.15.105:0/373599662 wait complete. 2024-03-26T17:43:58.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 165616 3.13 MB/sec execute 161 sec latency 19.735 ms 2024-03-26T17:43:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:57 smithi144 ceph-mon[36095]: pgmap v516: 97 pgs: 97 active+clean; 27 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:43:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:57 smithi105 ceph-mon[32280]: pgmap v516: 97 pgs: 97 active+clean; 27 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:43:58.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:57 smithi135 ceph-mon[35636]: pgmap v516: 97 pgs: 97 active+clean; 27 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:43:58.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:43:58.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:43:58.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:43:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 -- 172.21.15.105:0/79821436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c104c70 msgr2=0x7f747c107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 --2- 172.21.15.105:0/79821436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c104c70 0x7f747c107060 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7468009900 tx=0x7f746802bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 -- 172.21.15.105:0/79821436 shutdown_connections 2024-03-26T17:43:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 --2- 172.21.15.105:0/79821436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c1075a0 0x7f747c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 --2- 172.21.15.105:0/79821436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c104c70 0x7f747c107060 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 --2- 172.21.15.105:0/79821436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f747c102340 0x7f747c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 -- 172.21.15.105:0/79821436 >> 172.21.15.105:0/79821436 conn(0x7f747c0fbd50 msgr2=0x7f747c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 -- 172.21.15.105:0/79821436 shutdown_connections 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 -- 172.21.15.105:0/79821436 wait complete. 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 Processor -- start 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 -- start start 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c102340 0x7f747c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f747c104c70 0x7f747c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.555+0000 7f7482bb3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c1075a0 0x7f747c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7482bb3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f747c10d3f0 con 0x7f747c102340 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7482bb3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f747c10d270 con 0x7f747c104c70 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7482bb3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f747c10d570 con 0x7f747c1075a0 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f748094f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c102340 0x7f747c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f747bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f747c104c70 0x7f747c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f748094f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c102340 0x7f747c199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51262/0 (socket says 172.21.15.105:51262) 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7481150640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c1075a0 0x7f747c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f748094f640 1 -- 172.21.15.105:0/715580833 learned_addr learned my addr 172.21.15.105:0/715580833 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7481150640 1 -- 172.21.15.105:0/715580833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f747c104c70 msgr2=0x7f747c19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7481150640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f747c104c70 0x7f747c19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7481150640 1 -- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c102340 msgr2=0x7f747c199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7481150640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c102340 0x7f747c199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7481150640 1 -- 172.21.15.105:0/715580833 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c1a1990 con 0x7f747c1075a0 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f748094f640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c102340 0x7f747c199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f747bfff640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f747c104c70 0x7f747c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7481150640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c1075a0 0x7f747c1a1320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7474009fd0 tx=0x7f747400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7479ffb640 1 -- 172.21.15.105:0/715580833 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747400ce20 con 0x7f747c1075a0 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.556+0000 7f7482bb3640 1 -- 172.21.15.105:0/715580833 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747c1a1c80 con 0x7f747c1075a0 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.557+0000 7f7479ffb640 1 -- 172.21.15.105:0/715580833 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7474005020 con 0x7f747c1075a0 2024-03-26T17:43:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.557+0000 7f7479ffb640 1 -- 172.21.15.105:0/715580833 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474010690 con 0x7f747c1075a0 2024-03-26T17:43:58.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.557+0000 7f7482bb3640 1 -- 172.21.15.105:0/715580833 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747c1a21a0 con 0x7f747c1075a0 2024-03-26T17:43:58.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.559+0000 7f7479ffb640 1 -- 172.21.15.105:0/715580833 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f74740051d0 con 0x7f747c1075a0 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.559+0000 7f7479ffb640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f746c0a2f20 0x7f746c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f748094f640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f746c0a2f20 0x7f746c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7479ffb640 1 -- 172.21.15.105:0/715580833 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f74740cfae0 con 0x7f747c1075a0 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7485799740 1 Processor -- start 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7485799740 1 -- start start 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7485799740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7485799740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf801680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f748094f640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f746c0a2f20 0x7f746c0a53e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7470011550 tx=0x7f7470005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7485799740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf801cf0 0x55ebdf804180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7485799740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ebdf761630 con 0x55ebdf7fcb80 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7485799740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ebdf709e10 con 0x55ebdf7fc190 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7485799740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ebdf75d5e0 con 0x55ebdf801cf0 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f747bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf801680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f748094f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f747bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf801680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51272/0 (socket says 172.21.15.105:51272) 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f747bfff640 1 -- 172.21.15.105:0/2111083007 learned_addr learned my addr 172.21.15.105:0/2111083007 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f7481150640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf801cf0 0x55ebdf804180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.560+0000 7f747bfff640 1 -- 172.21.15.105:0/2111083007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf801cf0 msgr2=0x55ebdf804180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f747bfff640 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf801cf0 0x55ebdf804180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f747bfff640 1 -- 172.21.15.105:0/2111083007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 msgr2=0x55ebdf7fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f747bfff640 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f747bfff640 1 -- 172.21.15.105:0/2111083007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebdf720740 con 0x55ebdf7fcb80 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f748094f640 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f7481150640 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf801cf0 0x55ebdf804180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f747bfff640 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf801680 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f747c069580 tx=0x7f74680095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0245d3d863f83a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f7466ffd640 1 -- 172.21.15.105:0/2111083007 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468045020 con 0x55ebdf7fcb80 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f7466ffd640 1 -- 172.21.15.105:0/2111083007 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f746802cc90 con 0x55ebdf7fcb80 2024-03-26T17:43:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.561+0000 7f7466ffd640 1 -- 172.21.15.105:0/2111083007 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468004920 con 0x55ebdf7fcb80 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 -- 172.21.15.105:0/2111083007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 msgr2=0x55ebdf801680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf801680 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f747c069580 tx=0x7f74680095f0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 -- 172.21.15.105:0/2111083007 shutdown_connections 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf801cf0 0x55ebdf804180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf801680 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 --2- 172.21.15.105:0/2111083007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 -- 172.21.15.105:0/2111083007 >> 172.21.15.105:0/2111083007 conn(0x55ebdf7f7700 msgr2=0x55ebdf7f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 -- 172.21.15.105:0/2111083007 shutdown_connections 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 -- 172.21.15.105:0/2111083007 wait complete. 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 -- start start 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.562+0000 7f7485799740 1 Processor -- start 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f7485799740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf806620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f7485799740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf806b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f7485799740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf801cf0 0x55ebdf80dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f7485799740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ebdf804c40 con 0x55ebdf801cf0 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f7485799740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ebdf71d1a0 con 0x55ebdf7fc190 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f7485799740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ebdf804dc0 con 0x55ebdf7fcb80 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f7485799740 10 client.0 _ll_register_callbacks cb 0x55ebdf474c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f748094f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf806620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf806b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf806b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34140/0 (socket says 172.21.15.105:34140) 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 -- 172.21.15.105:0/2471888229 learned_addr learned my addr 172.21.15.105:0/2471888229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f7481150640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf801cf0 0x55ebdf80dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 -- 172.21.15.105:0/2471888229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 msgr2=0x55ebdf806620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf806620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 -- 172.21.15.105:0/2471888229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf801cf0 msgr2=0x55ebdf80dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf801cf0 0x55ebdf80dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 -- 172.21.15.105:0/2471888229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebdf53d7e0 con 0x55ebdf7fcb80 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f747bfff640 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf806b60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7468005ab0 tx=0x7f746802c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca19935981d5b0b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.563+0000 7f745bfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f745bfff640 1 -- 172.21.15.105:0/2471888229 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468050020 con 0x55ebdf7fcb80 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f745bfff640 1 -- 172.21.15.105:0/2471888229 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7468003b80 con 0x55ebdf7fcb80 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 1 -- 172.21.15.105:0/2471888229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f746802cc90 con 0x55ebdf7fcb80 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f745bfff640 1 -- 172.21.15.105:0/2471888229 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f746805b050 con 0x55ebdf7fcb80 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 10 client.27385 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 1 -- 172.21.15.105:0/2471888229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ebdf59df70 con 0x55ebdf7fcb80 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f745bfff640 1 -- 172.21.15.105:0/2471888229 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f746802cc90 con 0x55ebdf7fcb80 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f745bfff640 1 -- 172.21.15.105:0/2471888229 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f746804c460 con 0x55ebdf7fcb80 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 10 client.27385 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 10 client.27385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 20 client.27385 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:58.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ebdf812090 0x55ebdf814480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 4 client.27385 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f7485799740 1 -- 172.21.15.105:0/2471888229 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebdf76eb70 con 0x55ebdf812090 2024-03-26T17:43:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.564+0000 7f748094f640 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ebdf812090 0x55ebdf814480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.565+0000 7f745bfff640 10 client.27385 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.565+0000 7f748094f640 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ebdf812090 0x55ebdf814480 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297a0c2e7183fec9 server_cookie=1f276333292668ff in_seq=0 out_seq=0 2024-03-26T17:43:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f745bfff640 1 -- 172.21.15.105:0/2471888229 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ebdf76eb70 con 0x55ebdf812090 2024-03-26T17:43:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f745bfff640 10 client.27385 handle_command_reply: tid=0 2024-03-26T17:43:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 client.27385 shutdown 2024-03-26T17:43:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 -- 172.21.15.105:0/2471888229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 msgr2=0x55ebdf806b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf806b60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7468005ab0 tx=0x7f746802c3b0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 -- 172.21.15.105:0/2471888229 shutdown_connections 2024-03-26T17:43:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ebdf812090 0x55ebdf814480 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf801cf0 0x55ebdf80dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf806b60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f745bfff640 0 client.27385 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:58.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 --2- 172.21.15.105:0/2471888229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf7fc190 0x55ebdf806620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 -- 172.21.15.105:0/2471888229 >> 172.21.15.105:0/2471888229 conn(0x55ebdf7f7700 msgr2=0x55ebdf7f6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 -- 172.21.15.105:0/2471888229 shutdown_connections 2024-03-26T17:43:58.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 1 -- 172.21.15.105:0/2471888229 wait complete. 2024-03-26T17:43:58.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.566+0000 7f7485799740 20 client.27385 trim_cache size 0 max 16384 2024-03-26T17:43:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7485799740 1 Processor -- start 2024-03-26T17:43:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7485799740 1 -- start start 2024-03-26T17:43:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7485799740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7485799740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7485799740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf7fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7485799740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1aaf60 con 0x55ebdf7fcb80 2024-03-26T17:43:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7485799740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1acaa0 con 0x55ebdf801cf0 2024-03-26T17:43:58.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7485799740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1ab630 con 0x55ebdf7fc190 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f747bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f748094f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f747bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51282/0 (socket says 172.21.15.105:51282) 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f748094f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34148/0 (socket says 172.21.15.105:34148) 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f747bfff640 1 -- 172.21.15.105:0/1630769230 learned_addr learned my addr 172.21.15.105:0/1630769230 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.586+0000 7f7481150640 1 --2- 172.21.15.105:0/1630769230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf7fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f747bfff640 1 -- 172.21.15.105:0/1630769230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fc190 msgr2=0x55ebdf7fd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f747bfff640 1 --2- 172.21.15.105:0/1630769230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f747bfff640 1 -- 172.21.15.105:0/1630769230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 msgr2=0x55ebdf7fb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f747bfff640 1 --2- 172.21.15.105:0/1630769230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf7fb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f747bfff640 1 -- 172.21.15.105:0/1630769230 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebdf8147f0 con 0x55ebdf7fcb80 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f747bfff640 1 --2- 172.21.15.105:0/1630769230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80db60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7468005bc0 tx=0x7f7468005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c4d8f15deba4c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f74667fc640 1 -- 172.21.15.105:0/1630769230 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468050070 con 0x55ebdf7fcb80 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f74667fc640 1 -- 172.21.15.105:0/1630769230 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f746802cc90 con 0x55ebdf7fcb80 2024-03-26T17:43:58.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.587+0000 7f74667fc640 1 -- 172.21.15.105:0/1630769230 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746804ce60 con 0x55ebdf7fcb80 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 -- 172.21.15.105:0/1630769230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 msgr2=0x55ebdf80db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 --2- 172.21.15.105:0/1630769230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80db60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7468005bc0 tx=0x7f7468005fd0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 -- 172.21.15.105:0/1630769230 shutdown_connections 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 --2- 172.21.15.105:0/1630769230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf7fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 --2- 172.21.15.105:0/1630769230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80db60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 --2- 172.21.15.105:0/1630769230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fc190 0x55ebdf7fd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 -- 172.21.15.105:0/1630769230 >> 172.21.15.105:0/1630769230 conn(0x55ebdf7f7700 msgr2=0x55ebdf7fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 -- 172.21.15.105:0/1630769230 shutdown_connections 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 -- 172.21.15.105:0/1630769230 wait complete. 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 -- start start 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.588+0000 7f7485799740 1 Processor -- start 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7485799740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fc190 0x55ebdf80d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7485799740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7485799740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf80bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7485799740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1ac3d0 con 0x55ebdf7fc190 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7485799740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1abd00 con 0x55ebdf801cf0 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7485799740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f747c1a8f40 con 0x55ebdf7fcb80 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7485799740 10 client.0 _ll_register_callbacks cb 0x7f7470009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f748094f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fc190 0x55ebdf80d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f747bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7481150640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf80bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f748094f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fc190 0x55ebdf80d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51286/0 (socket says 172.21.15.105:51286) 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f747bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34158/0 (socket says 172.21.15.105:34158) 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f748094f640 1 -- 172.21.15.105:0/2036164638 learned_addr learned my addr 172.21.15.105:0/2036164638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f747bfff640 1 -- 172.21.15.105:0/2036164638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 msgr2=0x55ebdf80bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f747bfff640 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf80bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f747bfff640 1 -- 172.21.15.105:0/2036164638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fc190 msgr2=0x55ebdf80d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f747bfff640 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fc190 0x55ebdf80d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f747bfff640 1 -- 172.21.15.105:0/2036164638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebdf435170 con 0x55ebdf7fcb80 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f748094f640 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fc190 0x55ebdf80d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f7481150640 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf80bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f747bfff640 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80daa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f746804ca10 tx=0x7f746804cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c2549aa177ba470 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f745bfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f745bfff640 1 -- 172.21.15.105:0/2036164638 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468063020 con 0x55ebdf7fcb80 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.589+0000 7f745bfff640 1 -- 172.21.15.105:0/2036164638 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7468003b80 con 0x55ebdf7fcb80 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 1 -- 172.21.15.105:0/2036164638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f746802cc90 con 0x55ebdf7fcb80 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f745bfff640 1 -- 172.21.15.105:0/2036164638 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7468050070 con 0x55ebdf7fcb80 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 10 client.27391 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 1 -- 172.21.15.105:0/2036164638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ebdf878d90 con 0x55ebdf7fcb80 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f745bfff640 1 -- 172.21.15.105:0/2036164638 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f746802cc90 con 0x55ebdf7fcb80 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f745bfff640 1 -- 172.21.15.105:0/2036164638 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7468034440 con 0x55ebdf7fcb80 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 10 client.27391 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:58.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 10 client.27391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 20 client.27391 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ebdf879740 0x55ebdf87bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 4 client.27391 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f7485799740 1 -- 172.21.15.105:0/2036164638 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ebdf768ec0 con 0x55ebdf879740 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.590+0000 7f748094f640 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ebdf879740 0x55ebdf87bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.591+0000 7f745bfff640 10 client.27391 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.591+0000 7f748094f640 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ebdf879740 0x55ebdf87bb60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e571361b416d324a server_cookie=e0c045d5db5428e3 in_seq=0 out_seq=0 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f745bfff640 1 -- 172.21.15.105:0/2036164638 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ebdf768ec0 con 0x55ebdf879740 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f745bfff640 10 client.27391 handle_command_reply: tid=0 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 client.27391 shutdown 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 -- 172.21.15.105:0/2036164638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 msgr2=0x55ebdf80daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80daa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f746804ca10 tx=0x7f746804cd20 comp rx=0 tx=0).stop 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 -- 172.21.15.105:0/2036164638 shutdown_connections 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ebdf879740 0x55ebdf87bb60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ebdf801cf0 0x55ebdf80bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ebdf7fcb80 0x55ebdf80daa0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f745bfff640 0 client.27391 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 --2- 172.21.15.105:0/2036164638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ebdf7fc190 0x55ebdf80d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 -- 172.21.15.105:0/2036164638 >> 172.21.15.105:0/2036164638 conn(0x55ebdf7f7700 msgr2=0x55ebdf7fb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 -- 172.21.15.105:0/2036164638 shutdown_connections 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 1 -- 172.21.15.105:0/2036164638 wait complete. 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.592+0000 7f7485799740 20 client.27391 trim_cache size 0 max 16384 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:43:58.589 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "417dc48b-39c2-432c-8fa2-a14974ad32ba", 2024-03-26T17:43:58.590 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:43:58.590 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:43:58.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 -- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f746c0a2f20 msgr2=0x7f746c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f746c0a2f20 0x7f746c0a53e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7470011550 tx=0x7f7470005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:58.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 -- 172.21.15.105:0/715580833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c1075a0 msgr2=0x7f747c1a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c1075a0 0x7f747c1a1320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7474009fd0 tx=0x7f747400ef10 comp rx=0 tx=0).stop 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 -- 172.21.15.105:0/715580833 shutdown_connections 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f746c0a2f20 0x7f746c0a53e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c1075a0 0x7f747c1a1320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f747c104c70 0x7f747c19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 --2- 172.21.15.105:0/715580833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f747c102340 0x7f747c199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 -- 172.21.15.105:0/715580833 >> 172.21.15.105:0/715580833 conn(0x7f747c0fbd50 msgr2=0x7f747c101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 -- 172.21.15.105:0/715580833 shutdown_connections 2024-03-26T17:43:58.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.593+0000 7f7482bb3640 1 -- 172.21.15.105:0/715580833 wait complete. 2024-03-26T17:43:58.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:43:58.598 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:43:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 -- 172.21.15.105:0/2746965105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c1019b0 msgr2=0x7f472c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 --2- 172.21.15.105:0/2746965105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c1019b0 0x7f472c101e30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4720009900 tx=0x7f472002bc30 comp rx=0 tx=0).stop 2024-03-26T17:43:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 -- 172.21.15.105:0/2746965105 shutdown_connections 2024-03-26T17:43:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 --2- 172.21.15.105:0/2746965105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f472c102370 0x7f472c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 --2- 172.21.15.105:0/2746965105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c1019b0 0x7f472c101e30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 --2- 172.21.15.105:0/2746965105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f472c1007c0 0x7f472c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 -- 172.21.15.105:0/2746965105 >> 172.21.15.105:0/2746965105 conn(0x7f472c0fbd50 msgr2=0x7f472c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 -- 172.21.15.105:0/2746965105 shutdown_connections 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.732+0000 7f47317f3640 1 -- 172.21.15.105:0/2746965105 wait complete. 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f47317f3640 1 Processor -- start 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f47317f3640 1 -- start start 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f47317f3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f472c1007c0 0x7f472c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f47317f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f472c1019b0 0x7f472c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f47317f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c102370 0x7f472c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f47317f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10c2f0 con 0x7f472c102370 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f47317f3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10c170 con 0x7f472c1007c0 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f47317f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10c470 con 0x7f472c1019b0 2024-03-26T17:43:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c102370 0x7f472c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f472affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f472c1007c0 0x7f472c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c102370 0x7f472c1a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51308/0 (socket says 172.21.15.105:51308) 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f472affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f472c1007c0 0x7f472c199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52820/0 (socket says 172.21.15.105:52820) 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.733+0000 7f472b7fe640 1 -- 172.21.15.105:0/4102509249 learned_addr learned my addr 172.21.15.105:0/4102509249 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f472a7fc640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f472c1019b0 0x7f472c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f472b7fe640 1 -- 172.21.15.105:0/4102509249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f472c1019b0 msgr2=0x7f472c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f472b7fe640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f472c1019b0 0x7f472c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f472b7fe640 1 -- 172.21.15.105:0/4102509249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f472c1007c0 msgr2=0x7f472c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f472b7fe640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f472c1007c0 0x7f472c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f472b7fe640 1 -- 172.21.15.105:0/4102509249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c1a1930 con 0x7f472c102370 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f472a7fc640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f472c1019b0 0x7f472c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f472b7fe640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c102370 0x7f472c1a1290 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f472400eb10 tx=0x7f472400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f4713fff640 1 -- 172.21.15.105:0/4102509249 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724004b20 con 0x7f472c102370 2024-03-26T17:43:58.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f47317f3640 1 -- 172.21.15.105:0/4102509249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c1a1c20 con 0x7f472c102370 2024-03-26T17:43:58.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f4713fff640 1 -- 172.21.15.105:0/4102509249 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4724004cc0 con 0x7f472c102370 2024-03-26T17:43:58.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f4713fff640 1 -- 172.21.15.105:0/4102509249 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724010690 con 0x7f472c102370 2024-03-26T17:43:58.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.734+0000 7f47317f3640 1 -- 172.21.15.105:0/4102509249 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c1a21c0 con 0x7f472c102370 2024-03-26T17:43:58.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.736+0000 7f4713fff640 1 -- 172.21.15.105:0/4102509249 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4724005020 con 0x7f472c102370 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f47343db740 1 Processor -- start 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f47343db740 1 -- start start 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f47343db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f47343db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f47343db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17ea8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f47343db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5598d1747980 con 0x5598d17e8450 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f47343db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5598d16efbe0 con 0x5598d17e28f0 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f47343db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5598d1743930 con 0x5598d17e32e0 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f4713fff640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47140a2f20 0x7f47140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f472a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f472a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17e2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52828/0 (socket says 172.21.15.105:52828) 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f472a7fc640 1 -- 172.21.15.105:0/2938436679 learned_addr learned my addr 172.21.15.105:0/2938436679 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17ea8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f472affd640 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.737+0000 7f472b7fe640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47140a2f20 0x7f47140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f472a7fc640 1 -- 172.21.15.105:0/2938436679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 msgr2=0x5598d17e7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17e7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f472a7fc640 1 -- 172.21.15.105:0/2938436679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 msgr2=0x5598d17ea8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17ea8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f472a7fc640 1 -- 172.21.15.105:0/2938436679 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598d15257d0 con 0x5598d17e28f0 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f472affd640 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f4713fff640 1 -- 172.21.15.105:0/4102509249 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f47240cfb70 con 0x7f472c102370 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17e2d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4720005c40 tx=0x7f4720005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec2cf7594795cf1e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f472b7fe640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47140a2f20 0x7f47140a53e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f47240cd9a0 tx=0x7f4724096bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f47117fa640 1 -- 172.21.15.105:0/2938436679 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720047020 con 0x5598d17e28f0 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f47117fa640 1 -- 172.21.15.105:0/2938436679 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4720005040 con 0x5598d17e28f0 2024-03-26T17:43:58.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.738+0000 7f47117fa640 1 -- 172.21.15.105:0/2938436679 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720004c10 con 0x5598d17e28f0 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 -- 172.21.15.105:0/2938436679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 msgr2=0x5598d17e2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17e2d10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4720005c40 tx=0x7f4720005b50 comp rx=0 tx=0).stop 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 -- 172.21.15.105:0/2938436679 shutdown_connections 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17ea8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 --2- 172.21.15.105:0/2938436679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17e2d10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 -- 172.21.15.105:0/2938436679 >> 172.21.15.105:0/2938436679 conn(0x5598d17dde60 msgr2=0x5598d17de260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 -- 172.21.15.105:0/2938436679 shutdown_connections 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 -- 172.21.15.105:0/2938436679 wait complete. 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.739+0000 7f47343db740 1 -- start start 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f47343db740 1 Processor -- start 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f47343db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17ecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f47343db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17ed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f47343db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17f4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f47343db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5598d17eb3a0 con 0x5598d17e8450 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f47343db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5598d1703160 con 0x5598d17e28f0 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f47343db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5598d17eb520 con 0x5598d17e32e0 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f47343db740 10 client.0 _ll_register_callbacks cb 0x5598d173f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17ecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51328/0 (socket says 172.21.15.105:51328) 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17ed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 -- 172.21.15.105:0/2210716749 learned_addr learned my addr 172.21.15.105:0/2210716749 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 -- 172.21.15.105:0/2210716749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 msgr2=0x5598d17ed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17ed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 -- 172.21.15.105:0/2210716749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 msgr2=0x5598d17f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 -- 172.21.15.105:0/2210716749 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598d145b3f0 con 0x5598d17e28f0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472affd640 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f470eff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.740+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17ecb00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4720045000 tx=0x7f4720005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5181fe031699e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f470eff5640 1 -- 172.21.15.105:0/2210716749 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4720047070 con 0x5598d17e28f0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f470eff5640 1 -- 172.21.15.105:0/2210716749 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f4720004e20 con 0x5598d17e28f0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 1 -- 172.21.15.105:0/2210716749 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4720005040 con 0x5598d17e28f0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f470eff5640 1 -- 172.21.15.105:0/2210716749 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47200035f0 con 0x5598d17e28f0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 10 client.27395 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 1 -- 172.21.15.105:0/2210716749 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598d141b0f0 con 0x5598d17e28f0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f470eff5640 1 -- 172.21.15.105:0/2210716749 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4720005040 con 0x5598d17e28f0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f470eff5640 1 -- 172.21.15.105:0/2210716749 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4720040810 con 0x5598d17e28f0 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 10 client.27395 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 10 client.27395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 20 client.27395 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5598d17f8550 0x5598d17fa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 4 client.27395 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:43:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.741+0000 7f47343db740 1 -- 172.21.15.105:0/2210716749 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598d1754ec0 con 0x5598d17f8550 2024-03-26T17:43:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.742+0000 7f472affd640 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5598d17f8550 0x5598d17fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.742+0000 7f472affd640 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5598d17f8550 0x5598d17fa940 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0feadd9bc9835e2 server_cookie=6b4a11686a533801 in_seq=0 out_seq=0 2024-03-26T17:43:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.742+0000 7f470eff5640 10 client.27395 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.744+0000 7f470eff5640 1 -- 172.21.15.105:0/2210716749 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5598d1754ec0 con 0x5598d17f8550 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.744+0000 7f470eff5640 10 client.27395 handle_command_reply: tid=0 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.744+0000 7f47343db740 1 client.27395 shutdown 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 -- 172.21.15.105:0/2210716749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 msgr2=0x5598d17ecb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17ecb00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4720045000 tx=0x7f4720005fd0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 -- 172.21.15.105:0/2210716749 shutdown_connections 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5598d17f8550 0x5598d17fa940 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e8450 0x5598d17f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f470eff5640 0 client.27395 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 --2- 172.21.15.105:0/2210716749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17e28f0 0x5598d17ecb00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 -- 172.21.15.105:0/2210716749 >> 172.21.15.105:0/2210716749 conn(0x5598d17dde60 msgr2=0x5598d17dd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 -- 172.21.15.105:0/2210716749 shutdown_connections 2024-03-26T17:43:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 1 -- 172.21.15.105:0/2210716749 wait complete. 2024-03-26T17:43:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.745+0000 7f47343db740 20 client.27395 trim_cache size 0 max 16384 2024-03-26T17:43:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f47343db740 1 Processor -- start 2024-03-26T17:43:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f47343db740 1 -- start start 2024-03-26T17:43:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f47343db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 0x5598d17dc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f47343db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f47343db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17dcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f47343db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1aaf30 con 0x5598d17e28f0 2024-03-26T17:43:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f47343db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1aca70 con 0x5598d17dde60 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f47343db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1ab600 con 0x5598d17e32e0 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51342/0 (socket says 172.21.15.105:51342) 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472affd640 1 -- 172.21.15.105:0/2407037429 learned_addr learned my addr 172.21.15.105:0/2407037429 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17dcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 0x5598d17dc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472affd640 1 -- 172.21.15.105:0/2407037429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 msgr2=0x5598d17dcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472affd640 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17dcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472affd640 1 -- 172.21.15.105:0/2407037429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 msgr2=0x5598d17dc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472affd640 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 0x5598d17dc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.774+0000 7f472affd640 1 -- 172.21.15.105:0/2407037429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598d17e3fb0 con 0x5598d17e28f0 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.775+0000 7f472affd640 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f4040 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f471c009fd0 tx=0x7f471c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c04a364d8f6455d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.775+0000 7f4710ff9640 1 -- 172.21.15.105:0/2407037429 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c024020 con 0x5598d17e28f0 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.775+0000 7f4710ff9640 1 -- 172.21.15.105:0/2407037429 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f471c0207a0 con 0x5598d17e28f0 2024-03-26T17:43:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.775+0000 7f4710ff9640 1 -- 172.21.15.105:0/2407037429 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c01e3f0 con 0x5598d17e28f0 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- 172.21.15.105:0/2407037429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 msgr2=0x5598d17f4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f4040 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f471c009fd0 tx=0x7f471c005f90 comp rx=0 tx=0).stop 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- 172.21.15.105:0/2407037429 shutdown_connections 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17dcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f4040 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 --2- 172.21.15.105:0/2407037429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 0x5598d17dc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- 172.21.15.105:0/2407037429 >> 172.21.15.105:0/2407037429 conn(0x5598d17de5b0 msgr2=0x5598d17e7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- 172.21.15.105:0/2407037429 shutdown_connections 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- 172.21.15.105:0/2407037429 wait complete. 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- start start 2024-03-26T17:43:58.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 Processor -- start 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 0x5598d17f1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1ac3a0 con 0x5598d17e28f0 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1abcd0 con 0x5598d17dde60 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1a8f10 con 0x5598d17e32e0 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f47343db740 10 client.0 _ll_register_callbacks cb 0x7f4720009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.776+0000 7f472affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 0x5598d17f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51344/0 (socket says 172.21.15.105:51344) 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472affd640 1 -- 172.21.15.105:0/2212352348 learned_addr learned my addr 172.21.15.105:0/2212352348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472affd640 1 -- 172.21.15.105:0/2212352348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 msgr2=0x5598d17f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472affd640 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472affd640 1 -- 172.21.15.105:0/2212352348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 msgr2=0x5598d17f1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472affd640 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 0x5598d17f1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472affd640 1 -- 172.21.15.105:0/2212352348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598d17f3250 con 0x5598d17e28f0 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f472affd640 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f1c50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f471c012600 tx=0x7f471c010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce0f72eb7874f85 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f470eff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f470eff5640 1 -- 172.21.15.105:0/2212352348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c010840 con 0x5598d17e28f0 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f470eff5640 1 -- 172.21.15.105:0/2212352348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f471c0200f0 con 0x5598d17e28f0 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f47343db740 1 -- 172.21.15.105:0/2212352348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f471c0207a0 con 0x5598d17e28f0 2024-03-26T17:43:58.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f470eff5640 1 -- 172.21.15.105:0/2212352348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f471c010e10 con 0x5598d17e28f0 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f47343db740 10 client.17574 fetch_fsmap learned FSMap version 30 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f47343db740 1 -- 172.21.15.105:0/2212352348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598d185e780 con 0x5598d17e28f0 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.777+0000 7f470eff5640 1 -- 172.21.15.105:0/2212352348 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f471c0207a0 con 0x5598d17e28f0 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f470eff5640 1 -- 172.21.15.105:0/2212352348 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f471c004770 con 0x5598d17e28f0 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f47343db740 10 client.17574 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f47343db740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f47343db740 20 client.17574 populate_metadata read hostname 'smithi105' 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f47343db740 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5598d17e8450 0x5598d1861130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f47343db740 4 client.17574 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f47343db740 1 -- 172.21.15.105:0/2212352348 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5598d174f210 con 0x5598d17e8450 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5598d17e8450 0x5598d1861130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f470eff5640 10 client.17574 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.778+0000 7f472a7fc640 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5598d17e8450 0x5598d1861130 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d260055df287a69 server_cookie=5a0b29873d7bbaa1 in_seq=0 out_seq=0 2024-03-26T17:43:58.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.779+0000 7f470eff5640 1 -- 172.21.15.105:0/2212352348 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5598d174f210 con 0x5598d17e8450 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.779+0000 7f470eff5640 10 client.17574 handle_command_reply: tid=0 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.779+0000 7f47343db740 1 client.17574 shutdown 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.779+0000 7f47343db740 1 -- 172.21.15.105:0/2212352348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 msgr2=0x5598d17f1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.779+0000 7f47343db740 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f1c50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f471c012600 tx=0x7f471c010ca0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 1 -- 172.21.15.105:0/2212352348 shutdown_connections 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5598d17e8450 0x5598d1861130 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f470eff5640 0 client.17574 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5598d17e32e0 0x5598d17f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5598d17e28f0 0x5598d17f1c50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 1 --2- 172.21.15.105:0/2212352348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5598d17dde60 0x5598d17f1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 1 -- 172.21.15.105:0/2212352348 >> 172.21.15.105:0/2212352348 conn(0x5598d17de5b0 msgr2=0x5598d17e10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 1 -- 172.21.15.105:0/2212352348 shutdown_connections 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 1 -- 172.21.15.105:0/2212352348 wait complete. 2024-03-26T17:43:58.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47343db740 20 client.17574 trim_cache size 0 max 16384 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout: "417dc48b-39c2-432c-8fa2-a14974ad32ba": { 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "417dc48b-39c2-432c-8fa2-a14974ad32ba", 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:43:58.778 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:43:58.779 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:43:58.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 -- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47140a2f20 msgr2=0x7f47140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47140a2f20 0x7f47140a53e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f47240cd9a0 tx=0x7f4724096bf0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 -- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c102370 msgr2=0x7f472c1a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c102370 0x7f472c1a1290 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f472400eb10 tx=0x7f472400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 -- 172.21.15.105:0/4102509249 shutdown_connections 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47140a2f20 0x7f47140a53e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f472c102370 0x7f472c1a1290 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f472c1019b0 0x7f472c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 --2- 172.21.15.105:0/4102509249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f472c1007c0 0x7f472c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.780+0000 7f47317f3640 1 -- 172.21.15.105:0/4102509249 >> 172.21.15.105:0/4102509249 conn(0x7f472c0fbd50 msgr2=0x7f472c068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.781+0000 7f47317f3640 1 -- 172.21.15.105:0/4102509249 shutdown_connections 2024-03-26T17:43:58.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:43:58.781+0000 7f47317f3640 1 -- 172.21.15.105:0/4102509249 wait complete. 2024-03-26T17:43:58.784 INFO:tasks.cephfs.filesystem:scrub status for tag:417dc48b-39c2-432c-8fa2-a14974ad32ba - {'path': '/', 'tag': '417dc48b-39c2-432c-8fa2-a14974ad32ba', 'options': 'recursive,force'} 2024-03-26T17:43:59.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 166461 3.16 MB/sec execute 162 sec latency 16.564 ms 2024-03-26T17:44:00.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 166867 3.16 MB/sec execute 163 sec latency 19.427 ms 2024-03-26T17:44:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:59 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:44:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:59 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:59 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:44:00.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:59 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:00.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:43:59 smithi144 ceph-mon[36095]: pgmap v517: 97 pgs: 97 active+clean; 27 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:44:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:59 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:44:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:59 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:59 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:44:00.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:59 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:00.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:43:59 smithi105 ceph-mon[32280]: pgmap v517: 97 pgs: 97 active+clean; 27 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:44:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:59 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:44:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:59 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:59 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:44:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:59 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:43:59 smithi135 ceph-mon[35636]: pgmap v517: 97 pgs: 97 active+clean; 27 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:44:01.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 167272 3.15 MB/sec execute 164 sec latency 16.433 ms 2024-03-26T17:44:02.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 167841 3.14 MB/sec execute 165 sec latency 22.415 ms 2024-03-26T17:44:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:01 smithi144 ceph-mon[36095]: pgmap v518: 97 pgs: 97 active+clean; 27 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:44:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:01 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:44:02.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:01 smithi105 ceph-mon[32280]: pgmap v518: 97 pgs: 97 active+clean; 27 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:44:02.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:01 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:44:02.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:01 smithi135 ceph-mon[35636]: pgmap v518: 97 pgs: 97 active+clean; 27 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:44:02.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:01 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:44:03.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 168524 3.14 MB/sec execute 166 sec latency 17.390 ms 2024-03-26T17:44:04.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 169162 3.13 MB/sec execute 167 sec latency 15.961 ms 2024-03-26T17:44:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:03 smithi144 ceph-mon[36095]: pgmap v519: 97 pgs: 97 active+clean; 27 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-03-26T17:44:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:03 smithi105 ceph-mon[32280]: pgmap v519: 97 pgs: 97 active+clean; 27 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-03-26T17:44:04.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:03 smithi135 ceph-mon[35636]: pgmap v519: 97 pgs: 97 active+clean; 27 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-03-26T17:44:05.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 170013 3.16 MB/sec execute 168 sec latency 17.541 ms 2024-03-26T17:44:06.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 170473 3.17 MB/sec execute 169 sec latency 16.760 ms 2024-03-26T17:44:06.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:05 smithi144 ceph-mon[36095]: pgmap v520: 97 pgs: 97 active+clean; 28 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:44:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:05 smithi105 ceph-mon[32280]: pgmap v520: 97 pgs: 97 active+clean; 28 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:44:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:05 smithi135 ceph-mon[35636]: pgmap v520: 97 pgs: 97 active+clean; 28 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:44:07.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 170836 3.15 MB/sec execute 170 sec latency 22.244 ms 2024-03-26T17:44:08.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 171394 3.14 MB/sec execute 171 sec latency 24.764 ms 2024-03-26T17:44:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:07 smithi144 ceph-mon[36095]: pgmap v521: 97 pgs: 97 active+clean; 28 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-03-26T17:44:08.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:07 smithi105 ceph-mon[32280]: pgmap v521: 97 pgs: 97 active+clean; 28 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-03-26T17:44:08.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:07 smithi135 ceph-mon[35636]: pgmap v521: 97 pgs: 97 active+clean; 28 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-03-26T17:44:09.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 172080 3.14 MB/sec execute 172 sec latency 37.408 ms 2024-03-26T17:44:10.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 172685 3.14 MB/sec execute 173 sec latency 17.783 ms 2024-03-26T17:44:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:09 smithi144 ceph-mon[36095]: pgmap v522: 97 pgs: 97 active+clean; 30 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:44:10.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:09 smithi105 ceph-mon[32280]: pgmap v522: 97 pgs: 97 active+clean; 30 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:44:10.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:09 smithi135 ceph-mon[35636]: pgmap v522: 97 pgs: 97 active+clean; 30 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:44:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 -- 172.21.15.105:0/3670484684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 msgr2=0x7f9e04109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 --2- 172.21.15.105:0/3670484684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 0x7f9e04109a10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009ca0 tx=0x7f9dfc02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 -- 172.21.15.105:0/3670484684 shutdown_connections 2024-03-26T17:44:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 --2- 172.21.15.105:0/3670484684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 0x7f9e04109a10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 --2- 172.21.15.105:0/3670484684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04104c90 0x7f9e04107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 --2- 172.21.15.105:0/3670484684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e04102360 0x7f9e04104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 -- 172.21.15.105:0/3670484684 >> 172.21.15.105:0/3670484684 conn(0x7f9e040fbd50 msgr2=0x7f9e040fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 -- 172.21.15.105:0/3670484684 shutdown_connections 2024-03-26T17:44:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.204+0000 7f9e0984b640 1 -- 172.21.15.105:0/3670484684 wait complete. 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e0984b640 1 Processor -- start 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e0984b640 1 -- start start 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e0984b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e04102360 0x7f9e04195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e0984b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04104c90 0x7f9e04195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e0984b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 0x7f9e0419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e0984b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0410b250 con 0x7f9e041075c0 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e0984b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0410b0d0 con 0x7f9e04102360 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e0984b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0410b3d0 con 0x7f9e04104c90 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.205+0000 7f9e037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 0x7f9e0419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e02ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e04102360 0x7f9e04195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 0x7f9e0419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50316/0 (socket says 172.21.15.105:50316) 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e037fe640 1 -- 172.21.15.105:0/2291043704 learned_addr learned my addr 172.21.15.105:0/2291043704 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e027fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04104c90 0x7f9e04195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e037fe640 1 -- 172.21.15.105:0/2291043704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04104c90 msgr2=0x7f9e04195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e037fe640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04104c90 0x7f9e04195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e037fe640 1 -- 172.21.15.105:0/2291043704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e04102360 msgr2=0x7f9e04195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e037fe640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e04102360 0x7f9e04195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e037fe640 1 -- 172.21.15.105:0/2291043704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0419d440 con 0x7f9e041075c0 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e037fe640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 0x7f9e0419cdd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc02c700 tx=0x7f9dfc0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e08849640 1 -- 172.21.15.105:0/2291043704 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc041020 con 0x7f9e041075c0 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e08849640 1 -- 172.21.15.105:0/2291043704 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dfc003e20 con 0x7f9e041075c0 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e0984b640 1 -- 172.21.15.105:0/2291043704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0419d6d0 con 0x7f9e041075c0 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e08849640 1 -- 172.21.15.105:0/2291043704 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc039040 con 0x7f9e041075c0 2024-03-26T17:44:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.206+0000 7f9e0984b640 1 -- 172.21.15.105:0/2291043704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0419dc10 con 0x7f9e041075c0 2024-03-26T17:44:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.209+0000 7f9e08849640 1 -- 172.21.15.105:0/2291043704 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9dfc044020 con 0x7f9e041075c0 2024-03-26T17:44:10.203 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.209+0000 7f9e08849640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dec0a3040 0x7f9dec0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.209+0000 7f9e08849640 1 -- 172.21.15.105:0/2291043704 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9dfc0f15f0 con 0x7f9e041075c0 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.209+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dec0a3040 0x7f9dec0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e0c439740 1 Processor -- start 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e0c439740 1 -- start start 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dec0a3040 0x7f9dec0a5500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009bd0 tx=0x7f9df4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a05045d0 0x5604a05049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a0504fc0 0x5604a0509ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a050a130 0x5604a050c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e0c439740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5604a04690b0 con 0x5604a05045d0 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e0c439740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5604a0411960 con 0x5604a050a130 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e0c439740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604a0465060 con 0x5604a0504fc0 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a05045d0 0x5604a05049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a05045d0 0x5604a05049f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50328/0 (socket says 172.21.15.105:50328) 2024-03-26T17:44:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 -- 172.21.15.105:0/71847727 learned_addr learned my addr 172.21.15.105:0/71847727 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e027fc640 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a0504fc0 0x5604a0509ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e037fe640 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a050a130 0x5604a050c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 -- 172.21.15.105:0/71847727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a0504fc0 msgr2=0x5604a0509ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a0504fc0 0x5604a0509ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 -- 172.21.15.105:0/71847727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a050a130 msgr2=0x5604a050c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a050a130 0x5604a050c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 -- 172.21.15.105:0/71847727 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a0423c60 con 0x5604a05045d0 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.210+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a05045d0 0x5604a05049f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9df400e630 tx=0x7f9df4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68db1d456f52ba64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9de9ffb640 1 -- 172.21.15.105:0/71847727 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df4004020 con 0x5604a05045d0 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9de9ffb640 1 -- 172.21.15.105:0/71847727 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df40041c0 con 0x5604a05045d0 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9de9ffb640 1 -- 172.21.15.105:0/71847727 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df401c690 con 0x5604a05045d0 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9e0c439740 1 -- 172.21.15.105:0/71847727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a05045d0 msgr2=0x5604a05049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9e0c439740 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a05045d0 0x5604a05049f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9df400e630 tx=0x7f9df4014000 comp rx=0 tx=0).stop 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9e0c439740 1 -- 172.21.15.105:0/71847727 shutdown_connections 2024-03-26T17:44:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9e0c439740 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a050a130 0x5604a050c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9e0c439740 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a0504fc0 0x5604a0509ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9e0c439740 1 --2- 172.21.15.105:0/71847727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a05045d0 0x5604a05049f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.211+0000 7f9e0c439740 1 -- 172.21.15.105:0/71847727 >> 172.21.15.105:0/71847727 conn(0x5604a04ffb40 msgr2=0x5604a04fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.212+0000 7f9e0c439740 1 -- 172.21.15.105:0/71847727 shutdown_connections 2024-03-26T17:44:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.212+0000 7f9e0c439740 1 -- 172.21.15.105:0/71847727 wait complete. 2024-03-26T17:44:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.212+0000 7f9e0c439740 1 -- start start 2024-03-26T17:44:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.212+0000 7f9e0c439740 1 Processor -- start 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a050e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a050e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a050e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a0515990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a050e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54434/0 (socket says 172.21.15.105:54434) 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 -- 172.21.15.105:0/280867621 learned_addr learned my addr 172.21.15.105:0/280867621 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e027fc640 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a050e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e0c439740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5604a050d0e0 con 0x5604a0504fc0 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5604a0424db0 con 0x5604a05045d0 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604a050d260 con 0x5604a050a130 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e0c439740 10 client.0 _ll_register_callbacks cb 0x5604a0429110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e037fe640 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a0515990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 -- 172.21.15.105:0/280867621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 msgr2=0x5604a0515990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a0515990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 -- 172.21.15.105:0/280867621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 msgr2=0x5604a050e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a050e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 -- 172.21.15.105:0/280867621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604a017d210 con 0x5604a05045d0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e027fc640 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a050e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a050e410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9df400e600 tx=0x7f9df4004020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c3bc22f1dbfed3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.213+0000 7f9de2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9de2ffd640 1 -- 172.21.15.105:0/280867621 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df4035060 con 0x5604a05045d0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9df40041c0 con 0x5604a05045d0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9de2ffd640 1 -- 172.21.15.105:0/280867621 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9df4004370 con 0x5604a05045d0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9de2ffd640 1 -- 172.21.15.105:0/280867621 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9df4003690 con 0x5604a05045d0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 10 client.27413 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604a0249620 con 0x5604a05045d0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9de2ffd640 1 -- 172.21.15.105:0/280867621 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9df40041c0 con 0x5604a05045d0 2024-03-26T17:44:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9de2ffd640 1 -- 172.21.15.105:0/280867621 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9df403b420 con 0x5604a05045d0 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 10 client.27413 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 10 client.27413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 20 client.27413 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604a0519ef0 0x5604a051c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 4 client.27413 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.214+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604a04765f0 con 0x5604a0519ef0 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.215+0000 7f9e027fc640 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604a0519ef0 0x5604a051c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.215+0000 7f9de2ffd640 10 client.27413 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.215+0000 7f9e027fc640 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604a0519ef0 0x5604a051c2e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf1d1579c0b888 server_cookie=d8b2ac3b91cbaa77 in_seq=0 out_seq=0 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9de2ffd640 1 -- 172.21.15.105:0/280867621 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5604a04765f0 con 0x5604a0519ef0 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9de2ffd640 10 client.27413 handle_command_reply: tid=0 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 client.27413 shutdown 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 msgr2=0x5604a050e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a050e410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9df400e600 tx=0x7f9df4004020 comp rx=0 tx=0).stop 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 shutdown_connections 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604a0519ef0 0x5604a051c2e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a0515990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a050e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 --2- 172.21.15.105:0/280867621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a050e410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9de2ffd640 0 client.27413 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 >> 172.21.15.105:0/280867621 conn(0x5604a04ffb40 msgr2=0x5604a04fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 shutdown_connections 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 1 -- 172.21.15.105:0/280867621 wait complete. 2024-03-26T17:44:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.216+0000 7f9e0c439740 20 client.27413 trim_cache size 0 max 16384 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e0c439740 1 Processor -- start 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e0c439740 1 -- start start 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a05045d0 0x5604a0505a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a0504fc0 0x5604a0505fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a050a130 0x5604a04fe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e0c439740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041a6b30 con 0x5604a050a130 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e0c439740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041a8670 con 0x5604a0504fc0 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e0c439740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041a7200 con 0x5604a05045d0 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a050a130 0x5604a04fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e027fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a0504fc0 0x5604a0505fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a050a130 0x5604a04fe560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50350/0 (socket says 172.21.15.105:50350) 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 -- 172.21.15.105:0/3755190489 learned_addr learned my addr 172.21.15.105:0/3755190489 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e02ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a05045d0 0x5604a0505a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 -- 172.21.15.105:0/3755190489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a05045d0 msgr2=0x5604a0505a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a05045d0 0x5604a0505a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 -- 172.21.15.105:0/3755190489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a0504fc0 msgr2=0x5604a0505fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a0504fc0 0x5604a0505fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 -- 172.21.15.105:0/3755190489 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a0516e10 con 0x5604a050a130 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e027fc640 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a0504fc0 0x5604a0505fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a05045d0 0x5604a0505a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.233+0000 7f9e037fe640 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a050a130 0x5604a04fe560 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0ba6b0 tx=0x7f9dfc0f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f05aef6a0645b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9de97fa640 1 -- 172.21.15.105:0/3755190489 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0bdce0 con 0x5604a050a130 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9de97fa640 1 -- 172.21.15.105:0/3755190489 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dfc0c1e30 con 0x5604a050a130 2024-03-26T17:44:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9de97fa640 1 -- 172.21.15.105:0/3755190489 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0bdf40 con 0x5604a050a130 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 -- 172.21.15.105:0/3755190489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a050a130 msgr2=0x5604a04fe560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a050a130 0x5604a04fe560 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0ba6b0 tx=0x7f9dfc0f1040 comp rx=0 tx=0).stop 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 -- 172.21.15.105:0/3755190489 shutdown_connections 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a050a130 0x5604a04fe560 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a0504fc0 0x5604a0505fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 --2- 172.21.15.105:0/3755190489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a05045d0 0x5604a0505a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 -- 172.21.15.105:0/3755190489 >> 172.21.15.105:0/3755190489 conn(0x5604a04ffb40 msgr2=0x5604a050e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 -- 172.21.15.105:0/3755190489 shutdown_connections 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.234+0000 7f9e0c439740 1 -- 172.21.15.105:0/3755190489 wait complete. 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 1 -- start start 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 1 Processor -- start 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a05124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a0512a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a057e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041a7fa0 con 0x5604a0504fc0 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041a78d0 con 0x5604a05045d0 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e041a4b10 con 0x5604a050a130 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e0c439740 10 client.0 _ll_register_callbacks cb 0x7f9dfc0347b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e02ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a05124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e027fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a0512a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a057e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e02ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a05124c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54460/0 (socket says 172.21.15.105:54460) 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.235+0000 7f9e02ffd640 1 -- 172.21.15.105:0/2839291403 learned_addr learned my addr 172.21.15.105:0/2839291403 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e037fe640 1 -- 172.21.15.105:0/2839291403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 msgr2=0x5604a05124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e037fe640 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a05124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e037fe640 1 -- 172.21.15.105:0/2839291403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 msgr2=0x5604a0512a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e037fe640 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a0512a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e037fe640 1 -- 172.21.15.105:0/2839291403 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604a057e6e0 con 0x5604a050a130 2024-03-26T17:44:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a05124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e027fc640 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a0512a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e037fe640 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a057e1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5604a050f6c0 tx=0x7f9dfc009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=303e6843616562cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9de2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9de2ffd640 1 -- 172.21.15.105:0/2839291403 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0c6460 con 0x5604a050a130 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e0c439740 1 -- 172.21.15.105:0/2839291403 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dfc0c1e30 con 0x5604a050a130 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9de2ffd640 1 -- 172.21.15.105:0/2839291403 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9dfc09cb50 con 0x5604a050a130 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9de2ffd640 1 -- 172.21.15.105:0/2839291403 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9dfc0ba350 con 0x5604a050a130 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e0c439740 10 client.27427 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e0c439740 1 -- 172.21.15.105:0/2839291403 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604a057f950 con 0x5604a050a130 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9de2ffd640 1 -- 172.21.15.105:0/2839291403 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9dfc0c1e30 con 0x5604a050a130 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9de2ffd640 1 -- 172.21.15.105:0/2839291403 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9dfc03c030 con 0x5604a050a130 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e0c439740 10 client.27427 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.236+0000 7f9e0c439740 10 client.27427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.237+0000 7f9e0c439740 20 client.27427 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.237+0000 7f9e0c439740 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604a0580300 0x5604a0582720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.237+0000 7f9e0c439740 4 client.27427 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:44:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.237+0000 7f9e0c439740 1 -- 172.21.15.105:0/2839291403 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604a0470940 con 0x5604a0580300 2024-03-26T17:44:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.237+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604a0580300 0x5604a0582720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.237+0000 7f9de2ffd640 10 client.27427 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.237+0000 7f9e02ffd640 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604a0580300 0x5604a0582720 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12882cff8a641504 server_cookie=b107cdeaaaa34810 in_seq=0 out_seq=0 2024-03-26T17:44:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9de2ffd640 1 -- 172.21.15.105:0/2839291403 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604a0470940 con 0x5604a0580300 2024-03-26T17:44:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9de2ffd640 10 client.27427 handle_command_reply: tid=0 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 client.27427 shutdown 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 -- 172.21.15.105:0/2839291403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 msgr2=0x5604a057e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a057e1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5604a050f6c0 tx=0x7f9dfc009360 comp rx=0 tx=0).stop 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 -- 172.21.15.105:0/2839291403 shutdown_connections 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604a0580300 0x5604a0582720 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604a050a130 0x5604a057e1a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604a0504fc0 0x5604a0512a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 --2- 172.21.15.105:0/2839291403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604a05045d0 0x5604a05124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 -- 172.21.15.105:0/2839291403 >> 172.21.15.105:0/2839291403 conn(0x5604a04ffb40 msgr2=0x5604a0505a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9de2ffd640 0 client.27427 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 -- 172.21.15.105:0/2839291403 shutdown_connections 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 1 -- 172.21.15.105:0/2839291403 wait complete. 2024-03-26T17:44:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.275+0000 7f9e0c439740 20 client.27427 trim_cache size 0 max 16384 2024-03-26T17:44:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:44:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:44:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:44:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 -- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dec0a3040 msgr2=0x7f9dec0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dec0a3040 0x7f9dec0a5500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009bd0 tx=0x7f9df4005f90 comp rx=0 tx=0).stop 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 -- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 msgr2=0x7f9e0419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 0x7f9e0419cdd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc02c700 tx=0x7f9dfc0328a0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 -- 172.21.15.105:0/2291043704 shutdown_connections 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9dec0a3040 0x7f9dec0a5500 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9e041075c0 0x7f9e0419cdd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e04104c90 0x7f9e04195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 --2- 172.21.15.105:0/2291043704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e04102360 0x7f9e04195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 -- 172.21.15.105:0/2291043704 >> 172.21.15.105:0/2291043704 conn(0x7f9e040fbd50 msgr2=0x7f9e04102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 -- 172.21.15.105:0/2291043704 shutdown_connections 2024-03-26T17:44:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:10.276+0000 7f9e0984b640 1 -- 172.21.15.105:0/2291043704 wait complete. 2024-03-26T17:44:11.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 173543 3.16 MB/sec execute 174 sec latency 29.659 ms 2024-03-26T17:44:12.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 173940 3.16 MB/sec execute 175 sec latency 25.608 ms 2024-03-26T17:44:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:11 smithi144 ceph-mon[36095]: pgmap v523: 97 pgs: 97 active+clean; 31 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:44:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:11 smithi105 ceph-mon[32280]: pgmap v523: 97 pgs: 97 active+clean; 31 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:44:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:11 smithi135 ceph-mon[35636]: pgmap v523: 97 pgs: 97 active+clean; 31 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:44:13.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 174345 3.15 MB/sec execute 176 sec latency 14.212 ms 2024-03-26T17:44:14.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 174879 3.14 MB/sec execute 177 sec latency 17.346 ms 2024-03-26T17:44:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:13 smithi144 ceph-mon[36095]: pgmap v524: 97 pgs: 97 active+clean; 31 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:44:14.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:13 smithi105 ceph-mon[32280]: pgmap v524: 97 pgs: 97 active+clean; 31 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:44:14.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:13 smithi135 ceph-mon[35636]: pgmap v524: 97 pgs: 97 active+clean; 31 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:44:15.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 175523 3.15 MB/sec execute 178 sec latency 16.825 ms 2024-03-26T17:44:16.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 176195 3.14 MB/sec execute 179 sec latency 19.414 ms 2024-03-26T17:44:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:15 smithi144 ceph-mon[36095]: pgmap v525: 97 pgs: 97 active+clean; 31 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-03-26T17:44:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:15 smithi105 ceph-mon[32280]: pgmap v525: 97 pgs: 97 active+clean; 31 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-03-26T17:44:16.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:15 smithi135 ceph-mon[35636]: pgmap v525: 97 pgs: 97 active+clean; 31 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-03-26T17:44:17.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 177008 3.15 MB/sec execute 180 sec latency 17.048 ms 2024-03-26T17:44:18.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 177493 3.16 MB/sec execute 181 sec latency 23.913 ms 2024-03-26T17:44:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:17 smithi144 ceph-mon[36095]: pgmap v526: 97 pgs: 97 active+clean; 27 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:44:18.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:17 smithi105 ceph-mon[32280]: pgmap v526: 97 pgs: 97 active+clean; 27 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:44:18.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:17 smithi135 ceph-mon[35636]: pgmap v526: 97 pgs: 97 active+clean; 27 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:44:19.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 177892 3.15 MB/sec execute 182 sec latency 15.062 ms 2024-03-26T17:44:20.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 178458 3.15 MB/sec execute 183 sec latency 23.546 ms 2024-03-26T17:44:20.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:19 smithi144 ceph-mon[36095]: pgmap v527: 97 pgs: 97 active+clean; 28 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:44:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:19 smithi105 ceph-mon[32280]: pgmap v527: 97 pgs: 97 active+clean; 28 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:44:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:19 smithi135 ceph-mon[35636]: pgmap v527: 97 pgs: 97 active+clean; 28 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:44:21.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 179144 3.15 MB/sec execute 184 sec latency 16.672 ms 2024-03-26T17:44:22.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 179763 3.14 MB/sec execute 185 sec latency 17.556 ms 2024-03-26T17:44:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:21 smithi144 ceph-mon[36095]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-03-26T17:44:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:21 smithi105 ceph-mon[32280]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-03-26T17:44:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:21 smithi135 ceph-mon[35636]: pgmap v528: 97 pgs: 97 active+clean; 28 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-03-26T17:44:23.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 180628 3.16 MB/sec execute 186 sec latency 16.044 ms 2024-03-26T17:44:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 -- 172.21.15.105:0/3942480152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 msgr2=0x7fad24108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/3942480152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 0x7fad24108a50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009ca0 tx=0x7fad1c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 -- 172.21.15.105:0/3942480152 shutdown_connections 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/3942480152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 0x7fad24108a50 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/3942480152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad24101800 0x7fad24101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/3942480152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24100610 0x7fad24100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 -- 172.21.15.105:0/3942480152 >> 172.21.15.105:0/3942480152 conn(0x7fad240fbba0 msgr2=0x7fad240fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 -- 172.21.15.105:0/3942480152 shutdown_connections 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.367+0000 7fad2ab9d640 1 -- 172.21.15.105:0/3942480152 wait complete. 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2ab9d640 1 Processor -- start 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2ab9d640 1 -- start start 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2ab9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad24100610 0x7fad24069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2ab9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24101800 0x7fad2419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2ab9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 0x7fad2419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2ab9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410c120 con 0x7fad241021c0 2024-03-26T17:44:23.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2ab9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410bfa0 con 0x7fad24100610 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2ab9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410c2a0 con 0x7fad24101800 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad29b9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad24100610 0x7fad24069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2a39c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 0x7fad2419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad29b9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad24100610 0x7fad24069ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34428/0 (socket says 172.21.15.105:34428) 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad29b9b640 1 -- 172.21.15.105:0/2550834832 learned_addr learned my addr 172.21.15.105:0/2550834832 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.368+0000 7fad2939a640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24101800 0x7fad2419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad2a39c640 1 -- 172.21.15.105:0/2550834832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24101800 msgr2=0x7fad2419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad2a39c640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24101800 0x7fad2419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad2a39c640 1 -- 172.21.15.105:0/2550834832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad24100610 msgr2=0x7fad24069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad2a39c640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad24100610 0x7fad24069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad2a39c640 1 -- 172.21.15.105:0/2550834832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2419f570 con 0x7fad241021c0 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad2a39c640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 0x7fad2419ec60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fad1c02c630 tx=0x7fad1c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad1affd640 1 -- 172.21.15.105:0/2550834832 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c041020 con 0x7fad241021c0 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad1affd640 1 -- 172.21.15.105:0/2550834832 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad1c003e20 con 0x7fad241021c0 2024-03-26T17:44:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad1affd640 1 -- 172.21.15.105:0/2550834832 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c039040 con 0x7fad241021c0 2024-03-26T17:44:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad2ab9d640 1 -- 172.21.15.105:0/2550834832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2419f7a0 con 0x7fad241021c0 2024-03-26T17:44:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.369+0000 7fad2ab9d640 1 -- 172.21.15.105:0/2550834832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2419fc90 con 0x7fad241021c0 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad1affd640 1 -- 172.21.15.105:0/2550834832 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fad1c044020 con 0x7fad241021c0 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2e9eb740 1 Processor -- start 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2e9eb740 1 -- start start 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed6004afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004b590 0x55ed60050090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed60052b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ed5ffab0b0 con 0x55ed60050700 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed5ff53960 con 0x55ed6004b590 2024-03-26T17:44:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed5ffa7060 con 0x55ed6004aba0 2024-03-26T17:44:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad29b9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004b590 0x55ed60050090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2939a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed6004afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad29b9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004b590 0x55ed60050090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34430/0 (socket says 172.21.15.105:34430) 2024-03-26T17:44:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad29b9b640 1 -- 172.21.15.105:0/4001708232 learned_addr learned my addr 172.21.15.105:0/4001708232 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad2939a640 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed60052b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.372+0000 7fad1affd640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad100a2f10 0x7fad100a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad2939a640 1 -- 172.21.15.105:0/4001708232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004b590 msgr2=0x55ed60050090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad2939a640 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004b590 0x55ed60050090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad2939a640 1 -- 172.21.15.105:0/4001708232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 msgr2=0x55ed60052b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad2939a640 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed60052b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad2939a640 1 -- 172.21.15.105:0/4001708232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5ff65c60 con 0x55ed6004aba0 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad2a39c640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad100a2f10 0x7fad100a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad29b9b640 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004b590 0x55ed60050090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad1affd640 1 -- 172.21.15.105:0/2550834832 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad1c0f1680 con 0x7fad241021c0 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad2939a640 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed6004afc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad14005d00 tx=0x7fad140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dadf9c5631a6e581 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad0ffff640 1 -- 172.21.15.105:0/4001708232 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14012c90 con 0x55ed6004aba0 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad0ffff640 1 -- 172.21.15.105:0/4001708232 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad14012e30 con 0x55ed6004aba0 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.373+0000 7fad0ffff640 1 -- 172.21.15.105:0/4001708232 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad140047a0 con 0x55ed6004aba0 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4001708232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 msgr2=0x55ed6004afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2a39c640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad100a2f10 0x7fad100a53d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0ba1d0 tx=0x7fad1c040fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed6004afc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad14005d00 tx=0x7fad140046c0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4001708232 shutdown_connections 2024-03-26T17:44:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed60052b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004b590 0x55ed60050090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4001708232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed6004afc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4001708232 >> 172.21.15.105:0/4001708232 conn(0x55ed60046110 msgr2=0x55ed60046510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4001708232 shutdown_connections 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4001708232 wait complete. 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 -- start start 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.374+0000 7fad2e9eb740 1 Processor -- start 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004aba0 0x55ed60054990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004b590 0x55ed60054ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed6005bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ed600536b0 con 0x55ed60050700 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed5ff66db0 con 0x55ed6004aba0 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed60053830 con 0x55ed6004b590 2024-03-26T17:44:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2e9eb740 10 client.0 _ll_register_callbacks cb 0x55ed5ff6b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad29b9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004b590 0x55ed60054ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2939a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed6005bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad29b9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004b590 0x55ed60054ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54260/0 (socket says 172.21.15.105:54260) 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad29b9b640 1 -- 172.21.15.105:0/4039828246 learned_addr learned my addr 172.21.15.105:0/4039828246 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad2939a640 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004aba0 0x55ed60054990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad29b9b640 1 -- 172.21.15.105:0/4039828246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004aba0 msgr2=0x55ed60054990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad29b9b640 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004aba0 0x55ed60054990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad29b9b640 1 -- 172.21.15.105:0/4039828246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 msgr2=0x55ed6005bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad29b9b640 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed6005bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.375+0000 7fad29b9b640 1 -- 172.21.15.105:0/4039828246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5fcbf210 con 0x55ed6004b590 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad29b9b640 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004b590 0x55ed60054ed0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad20012770 tx=0x7fad20012c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbda9bd4aaf1d874 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad0d7fa640 1 -- 172.21.15.105:0/4039828246 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2000f8c0 con 0x55ed6004b590 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4039828246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad14012e30 con 0x55ed6004b590 2024-03-26T17:44:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad0d7fa640 1 -- 172.21.15.105:0/4039828246 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad2000fa60 con 0x55ed6004b590 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad0d7fa640 1 -- 172.21.15.105:0/4039828246 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad2000fc60 con 0x55ed6004b590 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 10 client.27439 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4039828246 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed5fd8b620 con 0x55ed6004b590 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad0d7fa640 1 -- 172.21.15.105:0/4039828246 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad14012e30 con 0x55ed6004b590 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad0d7fa640 1 -- 172.21.15.105:0/4039828246 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fad20010430 con 0x55ed6004b590 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 10 client.27439 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 10 client.27439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 20 client.27439 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed600608c0 0x55ed60062cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 4 client.27439 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4039828246 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed5ffb85f0 con 0x55ed600608c0 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.376+0000 7fad2939a640 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed600608c0 0x55ed60062cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.377+0000 7fad0d7fa640 10 client.27439 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.377+0000 7fad2939a640 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed600608c0 0x55ed60062cd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e86789b02a936c server_cookie=793b2fd1d4960d4d in_seq=0 out_seq=0 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad0d7fa640 1 -- 172.21.15.105:0/4039828246 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ed5ffb85f0 con 0x55ed600608c0 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad0d7fa640 10 client.27439 handle_command_reply: tid=0 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 client.27439 shutdown 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4039828246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004b590 msgr2=0x55ed60054ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004b590 0x55ed60054ed0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad20012770 tx=0x7fad20012c30 comp rx=0 tx=0).stop 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4039828246 shutdown_connections 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed600608c0 0x55ed60062cd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed6005bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004b590 0x55ed60054ed0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4039828246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed6004aba0 0x55ed60054990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad0d7fa640 0 client.27439 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4039828246 >> 172.21.15.105:0/4039828246 conn(0x55ed60046110 msgr2=0x55ed60047360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4039828246 shutdown_connections 2024-03-26T17:44:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4039828246 wait complete. 2024-03-26T17:44:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.378+0000 7fad2e9eb740 20 client.27439 trim_cache size 0 max 16384 2024-03-26T17:44:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2e9eb740 1 Processor -- start 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2e9eb740 1 -- start start 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed6004aba0 0x55ed60040810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60050700 0x55ed6005bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed60040d50 0x55ed600411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad241a6910 con 0x55ed6004aba0 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad241a8450 con 0x55ed60050700 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad241a6fe0 con 0x55ed60040d50 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed6004aba0 0x55ed60040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed6004aba0 0x55ed60040810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57644/0 (socket says 172.21.15.105:57644) 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad29b9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60050700 0x55ed6005bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 -- 172.21.15.105:0/4075481624 learned_addr learned my addr 172.21.15.105:0/4075481624 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 --2- 172.21.15.105:0/4075481624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed60040d50 0x55ed600411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 -- 172.21.15.105:0/4075481624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed60040d50 msgr2=0x55ed600411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 --2- 172.21.15.105:0/4075481624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed60040d50 0x55ed600411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 -- 172.21.15.105:0/4075481624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60050700 msgr2=0x55ed6005bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 --2- 172.21.15.105:0/4075481624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60050700 0x55ed6005bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 -- 172.21.15.105:0/4075481624 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed60063350 con 0x55ed6004aba0 2024-03-26T17:44:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.394+0000 7fad2939a640 1 --2- 172.21.15.105:0/4075481624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed6004aba0 0x55ed60040810 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x55ed6004bdf0 tx=0x7fad14017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b39007409b7e84b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad0f7fe640 1 -- 172.21.15.105:0/4075481624 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14012df0 con 0x55ed6004aba0 2024-03-26T17:44:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad0f7fe640 1 -- 172.21.15.105:0/4075481624 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad14005370 con 0x55ed6004aba0 2024-03-26T17:44:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad0f7fe640 1 -- 172.21.15.105:0/4075481624 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14010940 con 0x55ed6004aba0 2024-03-26T17:44:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4075481624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed6004aba0 msgr2=0x55ed60040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4075481624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed6004aba0 0x55ed60040810 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x55ed6004bdf0 tx=0x7fad14017040 comp rx=0 tx=0).stop 2024-03-26T17:44:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4075481624 shutdown_connections 2024-03-26T17:44:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4075481624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed60040d50 0x55ed600411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4075481624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60050700 0x55ed6005bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/4075481624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed6004aba0 0x55ed60040810 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4075481624 >> 172.21.15.105:0/4075481624 conn(0x55ed60046110 msgr2=0x55ed60054990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4075481624 shutdown_connections 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.395+0000 7fad2e9eb740 1 -- 172.21.15.105:0/4075481624 wait complete. 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 1 -- start start 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 1 Processor -- start 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60040d50 0x55ed60059a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed60059f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed6005aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad241a7d80 con 0x55ed60050700 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad241a76b0 con 0x55ed60040d50 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad241a48f0 con 0x55ed6004aba0 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2e9eb740 10 client.0 _ll_register_callbacks cb 0x7fad200099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2939a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60040d50 0x55ed60059a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2939a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60040d50 0x55ed60059a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34446/0 (socket says 172.21.15.105:34446) 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2939a640 1 -- 172.21.15.105:0/720784206 learned_addr learned my addr 172.21.15.105:0/720784206 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad2939a640 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed6005aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.396+0000 7fad29b9b640 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed60059f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2939a640 1 -- 172.21.15.105:0/720784206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 msgr2=0x55ed60059f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2939a640 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed60059f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2939a640 1 -- 172.21.15.105:0/720784206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 msgr2=0x55ed6005aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2939a640 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed6005aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2939a640 1 -- 172.21.15.105:0/720784206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed6005b4c0 con 0x55ed60040d50 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad29b9b640 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed60059f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2939a640 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60040d50 0x55ed60059a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fad1400b690 tx=0x7fad14032040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e7313cad093b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad0d7fa640 1 -- 172.21.15.105:0/720784206 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14043060 con 0x55ed60040d50 2024-03-26T17:44:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2e9eb740 1 -- 172.21.15.105:0/720784206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad14005370 con 0x55ed60040d50 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad0d7fa640 1 -- 172.21.15.105:0/720784206 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad1402f430 con 0x55ed60040d50 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad0d7fa640 1 -- 172.21.15.105:0/720784206 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad1402f630 con 0x55ed60040d50 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2e9eb740 10 client.27437 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad2e9eb740 1 -- 172.21.15.105:0/720784206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed600c6f80 con 0x55ed60040d50 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.397+0000 7fad0d7fa640 1 -- 172.21.15.105:0/720784206 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad14005370 con 0x55ed60040d50 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad0d7fa640 1 -- 172.21.15.105:0/720784206 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fad14003780 con 0x55ed60040d50 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad2e9eb740 10 client.27437 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad2e9eb740 10 client.27437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad2e9eb740 20 client.27437 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed600c7770 0x55ed600c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad2e9eb740 4 client.27437 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad2e9eb740 1 -- 172.21.15.105:0/720784206 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed6004b350 con 0x55ed600c7770 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad29b9b640 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed600c7770 0x55ed600c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad0d7fa640 10 client.27437 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.398+0000 7fad29b9b640 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed600c7770 0x55ed600c9b90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16a67dd1beacfa5 server_cookie=83d02b4fa058863b in_seq=0 out_seq=0 2024-03-26T17:44:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.445+0000 7fad0d7fa640 1 -- 172.21.15.105:0/720784206 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed6004b350 con 0x55ed600c7770 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad0d7fa640 10 client.27437 handle_command_reply: tid=0 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 client.27437 shutdown 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 -- 172.21.15.105:0/720784206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60040d50 msgr2=0x55ed60059a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60040d50 0x55ed60059a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fad1400b690 tx=0x7fad14032040 comp rx=0 tx=0).stop 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 -- 172.21.15.105:0/720784206 shutdown_connections 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed600c7770 0x55ed600c9b90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed60050700 0x55ed6005aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad0d7fa640 0 client.27437 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed6004aba0 0x55ed60059f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 --2- 172.21.15.105:0/720784206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed60040d50 0x55ed60059a40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 -- 172.21.15.105:0/720784206 >> 172.21.15.105:0/720784206 conn(0x55ed60046110 msgr2=0x55ed60049370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 -- 172.21.15.105:0/720784206 shutdown_connections 2024-03-26T17:44:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 1 -- 172.21.15.105:0/720784206 wait complete. 2024-03-26T17:44:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.446+0000 7fad2e9eb740 20 client.27437 trim_cache size 0 max 16384 2024-03-26T17:44:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:44:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:44:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:44:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:44:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 -- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad100a2f10 msgr2=0x7fad100a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad100a2f10 0x7fad100a53d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0ba1d0 tx=0x7fad1c040fb0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 -- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 msgr2=0x7fad2419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 0x7fad2419ec60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fad1c02c630 tx=0x7fad1c0328a0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 -- 172.21.15.105:0/2550834832 shutdown_connections 2024-03-26T17:44:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad100a2f10 0x7fad100a53d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad241021c0 0x7fad2419ec60 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad24101800 0x7fad2419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 --2- 172.21.15.105:0/2550834832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad24100610 0x7fad24069ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 -- 172.21.15.105:0/2550834832 >> 172.21.15.105:0/2550834832 conn(0x7fad240fbba0 msgr2=0x7fad24104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 -- 172.21.15.105:0/2550834832 shutdown_connections 2024-03-26T17:44:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:23.447+0000 7fad2ab9d640 1 -- 172.21.15.105:0/2550834832 wait complete. 2024-03-26T17:44:24.024 INFO:tasks.workunit.client.0.smithi105.stdout: 1 180982 3.16 MB/sec execute 187 sec latency 33.325 ms 2024-03-26T17:44:24.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:23 smithi144 ceph-mon[36095]: pgmap v529: 97 pgs: 97 active+clean; 29 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:44:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:23 smithi105 ceph-mon[32280]: pgmap v529: 97 pgs: 97 active+clean; 29 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:44:24.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:23 smithi135 ceph-mon[35636]: pgmap v529: 97 pgs: 97 active+clean; 29 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:44:25.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 181401 3.15 MB/sec execute 188 sec latency 14.996 ms 2024-03-26T17:44:26.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 181967 3.15 MB/sec execute 189 sec latency 19.713 ms 2024-03-26T17:44:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:25 smithi144 ceph-mon[36095]: pgmap v530: 97 pgs: 97 active+clean; 29 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:44:26.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:25 smithi105 ceph-mon[32280]: pgmap v530: 97 pgs: 97 active+clean; 29 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:44:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:25 smithi135 ceph-mon[35636]: pgmap v530: 97 pgs: 97 active+clean; 29 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:44:27.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 182616 3.15 MB/sec execute 190 sec latency 17.023 ms 2024-03-26T17:44:28.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 183275 3.14 MB/sec execute 191 sec latency 17.969 ms 2024-03-26T17:44:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:27 smithi144 ceph-mon[36095]: pgmap v531: 97 pgs: 97 active+clean; 31 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:44:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:27 smithi105 ceph-mon[32280]: pgmap v531: 97 pgs: 97 active+clean; 31 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:44:28.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:27 smithi135 ceph-mon[35636]: pgmap v531: 97 pgs: 97 active+clean; 31 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:44:28.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:44:28.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:44:28.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- 172.21.15.105:0/3828243509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 msgr2=0x7f82b0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 --2- 172.21.15.105:0/3828243509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 0x7f82b0108c00 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009ca0 tx=0x7f82a802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- 172.21.15.105:0/3828243509 shutdown_connections 2024-03-26T17:44:28.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 --2- 172.21.15.105:0/3828243509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 0x7f82b0108c00 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 --2- 172.21.15.105:0/3828243509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b01019b0 0x7f82b0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 --2- 172.21.15.105:0/3828243509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82b01007c0 0x7f82b0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- 172.21.15.105:0/3828243509 >> 172.21.15.105:0/3828243509 conn(0x7f82b00fbd50 msgr2=0x7f82b00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- 172.21.15.105:0/3828243509 shutdown_connections 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- 172.21.15.105:0/3828243509 wait complete. 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 Processor -- start 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- start start 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b01007c0 0x7f82b0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82b01019b0 0x7f82b019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 0x7f82b019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82b010c2d0 con 0x7f82b0102370 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82b010c150 con 0x7f82b01007c0 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.907+0000 7f82b85c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82b010c450 con 0x7f82b01019b0 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6360640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b01007c0 0x7f82b0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6b61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 0x7f82b019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6360640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b01007c0 0x7f82b0069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38728/0 (socket says 172.21.15.105:38728) 2024-03-26T17:44:28.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6360640 1 -- 172.21.15.105:0/2226013478 learned_addr learned my addr 172.21.15.105:0/2226013478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82b01019b0 0x7f82b019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6b61640 1 -- 172.21.15.105:0/2226013478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82b01019b0 msgr2=0x7f82b019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6b61640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82b01019b0 0x7f82b019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6b61640 1 -- 172.21.15.105:0/2226013478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b01007c0 msgr2=0x7f82b0069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6b61640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b01007c0 0x7f82b0069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6b61640 1 -- 172.21.15.105:0/2226013478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b019f750 con 0x7f82b0102370 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b6b61640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 0x7f82b019ee40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f82a802c630 tx=0x7f82a80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82a77fe640 1 -- 172.21.15.105:0/2226013478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8041020 con 0x7f82b0102370 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b85c4640 1 -- 172.21.15.105:0/2226013478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b019f980 con 0x7f82b0102370 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82a77fe640 1 -- 172.21.15.105:0/2226013478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82a8003e20 con 0x7f82b0102370 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82a77fe640 1 -- 172.21.15.105:0/2226013478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8039040 con 0x7f82b0102370 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.908+0000 7f82b85c4640 1 -- 172.21.15.105:0/2226013478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b019fec0 con 0x7f82b0102370 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.911+0000 7f82bb1a9740 1 Processor -- start 2024-03-26T17:44:28.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.911+0000 7f82a77fe640 1 -- 172.21.15.105:0/2226013478 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f82a8044020 con 0x7f82b0102370 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82bb1a9740 1 -- start start 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb59d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb5ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5f450 0x55dd9cb618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dd9cabe980 con 0x55dd9cb598f0 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dd9ca66be0 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dd9caba930 con 0x55dd9cb5f450 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b5b5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb59d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6360640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb5ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6360640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb5ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38736/0 (socket says 172.21.15.105:38736) 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b5b5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb59d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56006/0 (socket says 172.21.15.105:56006) 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6360640 1 -- 172.21.15.105:0/3456891073 learned_addr learned my addr 172.21.15.105:0/3456891073 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82a77fe640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f829c0a7330 0x7f829c0a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/3456891073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5f450 0x55dd9cb618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6b61640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f829c0a7330 0x7f829c0a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6360640 1 -- 172.21.15.105:0/3456891073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5f450 msgr2=0x55dd9cb618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6360640 1 --2- 172.21.15.105:0/3456891073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5f450 0x55dd9cb618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6360640 1 -- 172.21.15.105:0/3456891073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 msgr2=0x55dd9cb59d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6360640 1 --2- 172.21.15.105:0/3456891073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb59d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.912+0000 7f82b6360640 1 -- 172.21.15.105:0/3456891073 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9c89c7d0 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.913+0000 7f82a77fe640 1 -- 172.21.15.105:0/2226013478 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f82a80f15a0 con 0x7f82b0102370 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.913+0000 7f82b6360640 1 --2- 172.21.15.105:0/3456891073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb5ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009d60 tx=0x7f82ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab915f16fbee08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.913+0000 7f82b6b61640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f829c0a7330 0x7f829c0a97f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f82a80bd990 tx=0x7f82a808f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:28.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.913+0000 7f82a4bfe640 1 -- 172.21.15.105:0/3456891073 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac01e070 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.913+0000 7f82a4bfe640 1 -- 172.21.15.105:0/3456891073 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82ac009e50 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.913+0000 7f82a4bfe640 1 -- 172.21.15.105:0/3456891073 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac003e60 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 -- 172.21.15.105:0/3456891073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 msgr2=0x55dd9cb5ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/3456891073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb5ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009d60 tx=0x7f82ac011810 comp rx=0 tx=0).stop 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 -- 172.21.15.105:0/3456891073 shutdown_connections 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/3456891073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5f450 0x55dd9cb618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/3456891073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb5ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/3456891073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb59d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 -- 172.21.15.105:0/3456891073 >> 172.21.15.105:0/3456891073 conn(0x55dd9cb54e60 msgr2=0x55dd9cb55260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 -- 172.21.15.105:0/3456891073 shutdown_connections 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 -- 172.21.15.105:0/3456891073 wait complete. 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 -- start start 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 Processor -- start 2024-03-26T17:44:28.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.914+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb64070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb6b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dd9cb62400 con 0x55dd9cb598f0 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dd9ca7a160 con 0x55dd9cb5f450 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dd9cb62580 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82bb1a9740 10 client.0 _ll_register_callbacks cb 0x55dd9cab6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56010/0 (socket says 172.21.15.105:56010) 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 -- 172.21.15.105:0/1493876787 learned_addr learned my addr 172.21.15.105:0/1493876787 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b6360640 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb64070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb6b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 -- 172.21.15.105:0/1493876787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5a2e0 msgr2=0x55dd9cb64070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb64070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 -- 172.21.15.105:0/1493876787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 msgr2=0x55dd9cb6b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb6b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 -- 172.21.15.105:0/1493876787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd9c7d23f0 con 0x55dd9cb598f0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63b30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f82a0004630 tx=0x7f82a0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203132c87b35aa21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f8299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f8299ffb640 1 -- 172.21.15.105:0/1493876787 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0005050 con 0x55dd9cb598f0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1493876787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82ac009e50 con 0x55dd9cb598f0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f8299ffb640 1 -- 172.21.15.105:0/1493876787 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f82a00051f0 con 0x55dd9cb598f0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.915+0000 7f8299ffb640 1 -- 172.21.15.105:0/1493876787 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82a00053f0 con 0x55dd9cb598f0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82bb1a9740 10 client.17622 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1493876787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd9c7920f0 con 0x55dd9cb598f0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f8299ffb640 1 -- 172.21.15.105:0/1493876787 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82ac009e50 con 0x55dd9cb598f0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f8299ffb640 1 -- 172.21.15.105:0/1493876787 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f82a0010460 con 0x55dd9cb598f0 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82bb1a9740 10 client.17622 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82bb1a9740 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:28.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82bb1a9740 20 client.17622 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9cb6f8e0 0x55dd9cb71da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82bb1a9740 4 client.17622 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1493876787 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd9cacbec0 con 0x55dd9cb6f8e0 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.916+0000 7f82b6360640 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9cb6f8e0 0x55dd9cb71da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.917+0000 7f8299ffb640 10 client.17622 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.917+0000 7f82b6360640 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9cb6f8e0 0x55dd9cb71da0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c654ab9a284cc471 server_cookie=5bcb1ed1f97dc375 in_seq=0 out_seq=0 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f8299ffb640 1 -- 172.21.15.105:0/1493876787 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dd9cacbec0 con 0x55dd9cb6f8e0 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f8299ffb640 10 client.17622 handle_command_reply: tid=0 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 client.17622 shutdown 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1493876787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 msgr2=0x55dd9cb63b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63b30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f82a0004630 tx=0x7f82a0015040 comp rx=0 tx=0).stop 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1493876787 shutdown_connections 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9cb6f8e0 0x55dd9cb71da0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb6b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f8299ffb640 0 client.17622 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:28.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb64070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1493876787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63b30 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1493876787 >> 172.21.15.105:0/1493876787 conn(0x55dd9cb54e60 msgr2=0x55dd9cb54080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:28.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1493876787 shutdown_connections 2024-03-26T17:44:28.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1493876787 wait complete. 2024-03-26T17:44:28.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.918+0000 7f82bb1a9740 20 client.17622 trim_cache size 0 max 16384 2024-03-26T17:44:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.938+0000 7f82bb1a9740 1 Processor -- start 2024-03-26T17:44:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.938+0000 7f82bb1a9740 1 -- start start 2024-03-26T17:44:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb6b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5f450 0x55dd9cb5eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01a6c00 con 0x55dd9cb5f450 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01a8740 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01a72d0 con 0x55dd9cb598f0 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b5b5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b6360640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb6b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b6360640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb6b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38768/0 (socket says 172.21.15.105:38768) 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b5b5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5f450 0x55dd9cb5eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b6360640 1 -- 172.21.15.105:0/2491871583 learned_addr learned my addr 172.21.15.105:0/2491871583 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b5b5f640 1 -- 172.21.15.105:0/2491871583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 msgr2=0x55dd9cb63fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/2491871583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b5b5f640 1 -- 172.21.15.105:0/2491871583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 msgr2=0x55dd9cb6b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/2491871583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb6b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b5b5f640 1 -- 172.21.15.105:0/2491871583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9cb5e7b0 con 0x55dd9cb5f450 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b6360640 1 --2- 172.21.15.105:0/2491871583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb6b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:28.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.939+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/2491871583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5f450 0x55dd9cb5eb30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f82a0004050 tx=0x7f82a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fc039d10b31d0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f829bfff640 1 -- 172.21.15.105:0/2491871583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0005590 con 0x55dd9cb5f450 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f829bfff640 1 -- 172.21.15.105:0/2491871583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82a0033070 con 0x55dd9cb5f450 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f829bfff640 1 -- 172.21.15.105:0/2491871583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0022650 con 0x55dd9cb5f450 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 -- 172.21.15.105:0/2491871583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5f450 msgr2=0x55dd9cb5eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/2491871583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5f450 0x55dd9cb5eb30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f82a0004050 tx=0x7f82a0000bc0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 -- 172.21.15.105:0/2491871583 shutdown_connections 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/2491871583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5f450 0x55dd9cb5eb30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/2491871583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb6b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/2491871583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 0x55dd9cb63fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 -- 172.21.15.105:0/2491871583 >> 172.21.15.105:0/2491871583 conn(0x55dd9cb54e60 msgr2=0x55dd9cb580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 -- 172.21.15.105:0/2491871583 shutdown_connections 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.940+0000 7f82bb1a9740 1 -- 172.21.15.105:0/2491871583 wait complete. 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 1 -- start start 2024-03-26T17:44:28.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 1 Processor -- start 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 0x55dd9cb68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb68fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb69fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01a8070 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01a79a0 con 0x55dd9cb5f450 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01a4be0 con 0x55dd9cb598f0 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82bb1a9740 10 client.0 _ll_register_callbacks cb 0x7f82a00036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82b6360640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb68fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82b6360640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb68fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56036/0 (socket says 172.21.15.105:56036) 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82b5b5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb69fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82b6360640 1 -- 172.21.15.105:0/1686438797 learned_addr learned my addr 172.21.15.105:0/1686438797 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.941+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 0x55dd9cb68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82b6360640 1 -- 172.21.15.105:0/1686438797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 msgr2=0x55dd9cb68a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82b6360640 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 0x55dd9cb68a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82b6360640 1 -- 172.21.15.105:0/1686438797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 msgr2=0x55dd9cb69fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82b6360640 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb69fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82b6360640 1 -- 172.21.15.105:0/1686438797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd9cb6a5a0 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb69fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82b6360640 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb68fa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f82ac012fd0 tx=0x7f82ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e5bb16e525e338 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f8299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f8299ffb640 1 -- 172.21.15.105:0/1686438797 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac02b030 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f8299ffb640 1 -- 172.21.15.105:0/1686438797 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f82ac016040 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1686438797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a0033070 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f8299ffb640 1 -- 172.21.15.105:0/1686438797 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82ac01e040 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82bb1a9740 10 client.17628 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1686438797 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd9cbd6000 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.942+0000 7f8299ffb640 1 -- 172.21.15.105:0/1686438797 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82a0033070 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f8299ffb640 1 -- 172.21.15.105:0/1686438797 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f82ac027280 con 0x55dd9cb5a2e0 2024-03-26T17:44:28.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 10 client.17628 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 20 client.17628 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9cb6f8e0 0x55dd9cbd8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 4 client.17628 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1686438797 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd9cac6210 con 0x55dd9cb6f8e0 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9cb6f8e0 0x55dd9cbd8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82b5b5f640 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9cb6f8e0 0x55dd9cbd8880 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d44d7218ffeb0a2 server_cookie=16cba12d4a5738b0 in_seq=0 out_seq=0 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f8299ffb640 10 client.17628 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f8299ffb640 1 -- 172.21.15.105:0/1686438797 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dd9cac6210 con 0x55dd9cb6f8e0 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f8299ffb640 10 client.17628 handle_command_reply: tid=0 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 client.17628 shutdown 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1686438797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5a2e0 msgr2=0x55dd9cb68fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb68fa0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f82ac012fd0 tx=0x7f82ac009260 comp rx=0 tx=0).stop 2024-03-26T17:44:28.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1686438797 shutdown_connections 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9cb6f8e0 0x55dd9cbd8880 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9cb5f450 0x55dd9cb69fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9cb5a2e0 0x55dd9cb68fa0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.943+0000 7f82bb1a9740 1 --2- 172.21.15.105:0/1686438797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9cb598f0 0x55dd9cb68a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1686438797 >> 172.21.15.105:0/1686438797 conn(0x55dd9cb54e60 msgr2=0x55dd9cb580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f8299ffb640 0 client.17628 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1686438797 shutdown_connections 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82bb1a9740 1 -- 172.21.15.105:0/1686438797 wait complete. 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82bb1a9740 20 client.17628 trim_cache size 0 max 16384 2024-03-26T17:44:28.948 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:44:28.949 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:44:28.949 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:44:28.949 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:44:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 -- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f829c0a7330 msgr2=0x7f829c0a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f829c0a7330 0x7f829c0a97f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f82a80bd990 tx=0x7f82a808f040 comp rx=0 tx=0).stop 2024-03-26T17:44:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 -- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 msgr2=0x7f82b019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 0x7f82b019ee40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f82a802c630 tx=0x7f82a80328a0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 -- 172.21.15.105:0/2226013478 shutdown_connections 2024-03-26T17:44:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f829c0a7330 0x7f829c0a97f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82b0102370 0x7f82b019ee40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82b01019b0 0x7f82b019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 --2- 172.21.15.105:0/2226013478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82b01007c0 0x7f82b0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 -- 172.21.15.105:0/2226013478 >> 172.21.15.105:0/2226013478 conn(0x7f82b00fbd50 msgr2=0x7f82b0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 -- 172.21.15.105:0/2226013478 shutdown_connections 2024-03-26T17:44:28.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:28.944+0000 7f82b85c4640 1 -- 172.21.15.105:0/2226013478 wait complete. 2024-03-26T17:44:28.951 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:44:28.951 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:44:29.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 184082 3.16 MB/sec execute 192 sec latency 16.362 ms 2024-03-26T17:44:29.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 -- 172.21.15.105:0/3867309949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 msgr2=0x7fd87c107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 --2- 172.21.15.105:0/3867309949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 0x7fd87c107100 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd868009900 tx=0x7fd86802bc30 comp rx=0 tx=0).stop 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 -- 172.21.15.105:0/3867309949 shutdown_connections 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 --2- 172.21.15.105:0/3867309949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd87c107640 0x7fd87c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 --2- 172.21.15.105:0/3867309949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 0x7fd87c107100 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 --2- 172.21.15.105:0/3867309949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c1023e0 0x7fd87c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 -- 172.21.15.105:0/3867309949 >> 172.21.15.105:0/3867309949 conn(0x7fd87c0fbd50 msgr2=0x7fd87c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 -- 172.21.15.105:0/3867309949 shutdown_connections 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.073+0000 7fd882ca7640 1 -- 172.21.15.105:0/3867309949 wait complete. 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd882ca7640 1 Processor -- start 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd882ca7640 1 -- start start 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd882ca7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd87c1023e0 0x7fd87c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd882ca7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 0x7fd87c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd882ca7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c107640 0x7fd87c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd882ca7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c10d410 con 0x7fd87c104d10 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd882ca7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c10d290 con 0x7fd87c1023e0 2024-03-26T17:44:29.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd882ca7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c10d590 con 0x7fd87c107640 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 0x7fd87c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 0x7fd87c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56060/0 (socket says 172.21.15.105:56060) 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd87bfff640 1 -- 172.21.15.105:0/4241148462 learned_addr learned my addr 172.21.15.105:0/4241148462 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd880a43640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd87c1023e0 0x7fd87c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.074+0000 7fd881244640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c107640 0x7fd87c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd87bfff640 1 -- 172.21.15.105:0/4241148462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c107640 msgr2=0x7fd87c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd87bfff640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c107640 0x7fd87c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd87bfff640 1 -- 172.21.15.105:0/4241148462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd87c1023e0 msgr2=0x7fd87c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd87bfff640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd87c1023e0 0x7fd87c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd87bfff640 1 -- 172.21.15.105:0/4241148462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c19f760 con 0x7fd87c104d10 2024-03-26T17:44:29.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd87bfff640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 0x7fd87c198080 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd868000c00 tx=0x7fd868005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd879ffb640 1 -- 172.21.15.105:0/4241148462 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868043020 con 0x7fd87c104d10 2024-03-26T17:44:29.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd879ffb640 1 -- 172.21.15.105:0/4241148462 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd868004350 con 0x7fd87c104d10 2024-03-26T17:44:29.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c19f9f0 con 0x7fd87c104d10 2024-03-26T17:44:29.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd879ffb640 1 -- 172.21.15.105:0/4241148462 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868003b80 con 0x7fd87c104d10 2024-03-26T17:44:29.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.075+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c19ff40 con 0x7fd87c104d10 2024-03-26T17:44:29.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.076+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87c0686b0 con 0x7fd87c104d10 2024-03-26T17:44:29.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.077+0000 7fd879ffb640 1 -- 172.21.15.105:0/4241148462 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd868046020 con 0x7fd87c104d10 2024-03-26T17:44:29.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.078+0000 7fd879ffb640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd86c0a2f20 0x7fd86c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.078+0000 7fd879ffb640 1 -- 172.21.15.105:0/4241148462 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd8680f1580 con 0x7fd87c104d10 2024-03-26T17:44:29.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.078+0000 7fd880a43640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd86c0a2f20 0x7fd86c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.078+0000 7fd880a43640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd86c0a2f20 0x7fd86c0a53e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd870009bd0 tx=0x7fd870005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.084+0000 7fd879ffb640 1 -- 172.21.15.105:0/4241148462 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd86804d020 con 0x7fd87c104d10 2024-03-26T17:44:29.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.372+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd87c0fd840 con 0x7fd87c104d10 2024-03-26T17:44:29.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.373+0000 7fd879ffb640 1 -- 172.21.15.105:0/4241148462 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fd8680be370 con 0x7fd87c104d10 2024-03-26T17:44:29.366 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:44:29.366 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:44:29.367 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:44:29.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd86c0a2f20 msgr2=0x7fd86c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd86c0a2f20 0x7fd86c0a53e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd870009bd0 tx=0x7fd870005f90 comp rx=0 tx=0).stop 2024-03-26T17:44:29.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 msgr2=0x7fd87c198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 0x7fd87c198080 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd868000c00 tx=0x7fd868005dc0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 shutdown_connections 2024-03-26T17:44:29.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd86c0a2f20 0x7fd86c0a53e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c107640 0x7fd87c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd87c104d10 0x7fd87c198080 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 --2- 172.21.15.105:0/4241148462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd87c1023e0 0x7fd87c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 >> 172.21.15.105:0/4241148462 conn(0x7fd87c0fbd50 msgr2=0x7fd87c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 shutdown_connections 2024-03-26T17:44:29.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.375+0000 7fd882ca7640 1 -- 172.21.15.105:0/4241148462 wait complete. 2024-03-26T17:44:29.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:44:29.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.499+0000 7fba35af1640 1 -- 172.21.15.105:0/3466362193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba301097b0 msgr2=0x7fba3010bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.499+0000 7fba35af1640 1 --2- 172.21.15.105:0/3466362193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba301097b0 0x7fba3010bc40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fba28009ca0 tx=0x7fba2802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 -- 172.21.15.105:0/3466362193 shutdown_connections 2024-03-26T17:44:29.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 --2- 172.21.15.105:0/3466362193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba301097b0 0x7fba3010bc40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 --2- 172.21.15.105:0/3466362193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba30074320 0x7fba300747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 --2- 172.21.15.105:0/3466362193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba30075900 0x7fba30073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 -- 172.21.15.105:0/3466362193 >> 172.21.15.105:0/3466362193 conn(0x7fba300fce60 msgr2=0x7fba300ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 -- 172.21.15.105:0/3466362193 shutdown_connections 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 -- 172.21.15.105:0/3466362193 wait complete. 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 Processor -- start 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.500+0000 7fba35af1640 1 -- start start 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba35af1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba30074320 0x7fba30199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba35af1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba30075900 0x7fba3019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba35af1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba301097b0 0x7fba301a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba35af1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba3010f5e0 con 0x7fba30075900 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba35af1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba3010f460 con 0x7fba30074320 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba35af1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba3010f760 con 0x7fba301097b0 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba30075900 0x7fba3019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba30074320 0x7fba30199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba30075900 0x7fba3019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56070/0 (socket says 172.21.15.105:56070) 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba30074320 0x7fba30199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38818/0 (socket says 172.21.15.105:38818) 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2effd640 1 -- 172.21.15.105:0/1795933774 learned_addr learned my addr 172.21.15.105:0/1795933774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2ffff640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba301097b0 0x7fba301a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2f7fe640 1 -- 172.21.15.105:0/1795933774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba301097b0 msgr2=0x7fba301a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba301097b0 0x7fba301a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2f7fe640 1 -- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba30075900 msgr2=0x7fba3019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba30075900 0x7fba3019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2f7fe640 1 -- 172.21.15.105:0/1795933774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba301a1910 con 0x7fba30074320 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2effd640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba30075900 0x7fba3019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba30074320 0x7fba30199d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba2000e9f0 tx=0x7fba2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2cff9640 1 -- 172.21.15.105:0/1795933774 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2000ccc0 con 0x7fba30074320 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba2cff9640 1 -- 172.21.15.105:0/1795933774 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba200108e0 con 0x7fba30074320 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.501+0000 7fba35af1640 1 -- 172.21.15.105:0/1795933774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba301a1c00 con 0x7fba30074320 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.502+0000 7fba2cff9640 1 -- 172.21.15.105:0/1795933774 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20018880 con 0x7fba30074320 2024-03-26T17:44:29.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.502+0000 7fba35af1640 1 -- 172.21.15.105:0/1795933774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba301a2150 con 0x7fba30074320 2024-03-26T17:44:29.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.504+0000 7fba2cff9640 1 -- 172.21.15.105:0/1795933774 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fba20010a90 con 0x7fba30074320 2024-03-26T17:44:29.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.504+0000 7fba387b0740 1 Processor -- start 2024-03-26T17:44:29.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.504+0000 7fba387b0740 1 -- start start 2024-03-26T17:44:29.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.504+0000 7fba387b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6ca30 0x560058b6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.504+0000 7fba387b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6d420 0x560058b71f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba387b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058b74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba387b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560058ad1970 con 0x560058b6ca30 2024-03-26T17:44:29.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba387b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560058a79d70 con 0x560058b6d420 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba387b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560058acd920 con 0x560058b72590 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6ca30 0x560058b6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6d420 0x560058b71f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2cff9640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2f20 0x7fba180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6ca30 0x560058b6ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56078/0 (socket says 172.21.15.105:56078) 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 -- 172.21.15.105:0/408131083 learned_addr learned my addr 172.21.15.105:0/408131083 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2effd640 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058b74a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2f20 0x7fba180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 -- 172.21.15.105:0/408131083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 msgr2=0x560058b74a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058b74a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 -- 172.21.15.105:0/408131083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6d420 msgr2=0x560058b71f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6d420 0x560058b71f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2cff9640 1 -- 172.21.15.105:0/1795933774 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fba200cf850 con 0x7fba30074320 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 -- 172.21.15.105:0/408131083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600587e53f0 con 0x560058b6ca30 2024-03-26T17:44:29.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2effd640 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058b74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6d420 0x560058b71f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba2ffff640 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6ca30 0x560058b6ce50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fba28032380 tx=0x7fba28032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b2df85b8179ead server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.505+0000 7fba15ffb640 1 -- 172.21.15.105:0/408131083 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28050070 con 0x560058b6ca30 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba15ffb640 1 -- 172.21.15.105:0/408131083 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba280044f0 con 0x560058b6ca30 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba15ffb640 1 -- 172.21.15.105:0/408131083 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28039040 con 0x560058b6ca30 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba2ffff640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2f20 0x7fba180a53e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fba28009a60 tx=0x7fba28048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 -- 172.21.15.105:0/408131083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6ca30 msgr2=0x560058b6ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6ca30 0x560058b6ce50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fba28032380 tx=0x7fba28032f40 comp rx=0 tx=0).stop 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 -- 172.21.15.105:0/408131083 shutdown_connections 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058b74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6d420 0x560058b71f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 --2- 172.21.15.105:0/408131083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6ca30 0x560058b6ce50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 -- 172.21.15.105:0/408131083 >> 172.21.15.105:0/408131083 conn(0x560058b67fa0 msgr2=0x560058b683a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 -- 172.21.15.105:0/408131083 shutdown_connections 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.506+0000 7fba387b0740 1 -- 172.21.15.105:0/408131083 wait complete. 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 1 -- start start 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 1 Processor -- start 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6ca30 0x560058b76f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b77460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b72590 0x560058b7e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560058b75540 con 0x560058b6d420 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560058a8d310 con 0x560058b72590 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560058b756c0 con 0x560058b6ca30 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba387b0740 10 client.0 _ll_register_callbacks cb 0x560058ac8c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b77460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b77460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56088/0 (socket says 172.21.15.105:56088) 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba2effd640 1 -- 172.21.15.105:0/1430165393 learned_addr learned my addr 172.21.15.105:0/1430165393 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6ca30 0x560058b76f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.507+0000 7fba2ffff640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b72590 0x560058b7e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba2effd640 1 -- 172.21.15.105:0/1430165393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6ca30 msgr2=0x560058b76f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba2effd640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6ca30 0x560058b76f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba2effd640 1 -- 172.21.15.105:0/1430165393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b72590 msgr2=0x560058b7e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba2effd640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b72590 0x560058b7e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba2effd640 1 -- 172.21.15.105:0/1430165393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600588af7d0 con 0x560058b6d420 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6ca30 0x560058b76f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba2ffff640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b72590 0x560058b7e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba2effd640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b77460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fba24012d20 tx=0x7fba2400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe35dc030b30deb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba0effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba0effd640 1 -- 172.21.15.105:0/1430165393 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24004030 con 0x560058b6d420 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba0effd640 1 -- 172.21.15.105:0/1430165393 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fba240041d0 con 0x560058b6d420 2024-03-26T17:44:29.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba387b0740 1 -- 172.21.15.105:0/1430165393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba280044f0 con 0x560058b6d420 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba0effd640 1 -- 172.21.15.105:0/1430165393 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fba240043d0 con 0x560058b6d420 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba387b0740 10 client.17646 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.508+0000 7fba387b0740 1 -- 172.21.15.105:0/1430165393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560058b81930 con 0x560058b6d420 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba0effd640 1 -- 172.21.15.105:0/1430165393 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba280044f0 con 0x560058b6d420 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba0effd640 1 -- 172.21.15.105:0/1430165393 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fba2401c460 con 0x560058b6d420 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba387b0740 10 client.17646 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba387b0740 10 client.17646 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba387b0740 20 client.17646 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba387b0740 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x560058b83010 0x560058b85450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba387b0740 4 client.17646 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba387b0740 1 -- 172.21.15.105:0/1430165393 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560058adeeb0 con 0x560058b83010 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x560058b83010 0x560058b85450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x560058b83010 0x560058b85450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=37c4ac3c42fd7b7e server_cookie=474737950c089fc8 in_seq=0 out_seq=0 2024-03-26T17:44:29.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.509+0000 7fba0effd640 10 client.17646 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba0effd640 1 -- 172.21.15.105:0/1430165393 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560058adeeb0 con 0x560058b83010 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba0effd640 10 client.17646 handle_command_reply: tid=0 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 client.17646 shutdown 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 -- 172.21.15.105:0/1430165393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 msgr2=0x560058b77460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b77460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fba24012d20 tx=0x7fba2400f790 comp rx=0 tx=0).stop 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 -- 172.21.15.105:0/1430165393 shutdown_connections 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x560058b83010 0x560058b85450 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b72590 0x560058b7e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b77460 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 --2- 172.21.15.105:0/1430165393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6ca30 0x560058b76f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba0effd640 0 client.17646 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:44:29.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 -- 172.21.15.105:0/1430165393 >> 172.21.15.105:0/1430165393 conn(0x560058b67fa0 msgr2=0x560058b671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 -- 172.21.15.105:0/1430165393 shutdown_connections 2024-03-26T17:44:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 1 -- 172.21.15.105:0/1430165393 wait complete. 2024-03-26T17:44:29.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.510+0000 7fba387b0740 20 client.17646 trim_cache size 0 max 16384 2024-03-26T17:44:29.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba387b0740 1 Processor -- start 2024-03-26T17:44:29.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba387b0740 1 -- start start 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba387b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 0x560058b63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba387b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6d420 0x560058b7e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba387b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b72590 0x560058b76f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba387b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba301aaf30 con 0x560058b72590 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba387b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba301aca70 con 0x560058b6ca30 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba387b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba301ab600 con 0x560058b6d420 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b72590 0x560058b76f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b72590 0x560058b76f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56090/0 (socket says 172.21.15.105:56090) 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 -- 172.21.15.105:0/1519586087 learned_addr learned my addr 172.21.15.105:0/1519586087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 0x560058b63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2effd640 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6d420 0x560058b7e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 -- 172.21.15.105:0/1519586087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6d420 msgr2=0x560058b7e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6d420 0x560058b7e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 -- 172.21.15.105:0/1519586087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 msgr2=0x560058b63050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 0x560058b63050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 -- 172.21.15.105:0/1519586087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560058b67dd0 con 0x560058b72590 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.525+0000 7fba2ffff640 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b72590 0x560058b76f20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fba28006010 tx=0x7fba2802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb5c44305fb6fa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba157fa640 1 -- 172.21.15.105:0/1519586087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28058070 con 0x560058b72590 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba157fa640 1 -- 172.21.15.105:0/1519586087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba280044f0 con 0x560058b72590 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba157fa640 1 -- 172.21.15.105:0/1519586087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28039040 con 0x560058b72590 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 -- 172.21.15.105:0/1519586087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b72590 msgr2=0x560058b76f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b72590 0x560058b76f20 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fba28006010 tx=0x7fba2802e040 comp rx=0 tx=0).stop 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 -- 172.21.15.105:0/1519586087 shutdown_connections 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b72590 0x560058b76f20 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b6d420 0x560058b7e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 --2- 172.21.15.105:0/1519586087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 0x560058b63050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 -- 172.21.15.105:0/1519586087 >> 172.21.15.105:0/1519586087 conn(0x560058b67fa0 msgr2=0x560058b68380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 -- 172.21.15.105:0/1519586087 shutdown_connections 2024-03-26T17:44:29.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.526+0000 7fba387b0740 1 -- 172.21.15.105:0/1519586087 wait complete. 2024-03-26T17:44:29.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 1 -- start start 2024-03-26T17:44:29.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 1 Processor -- start 2024-03-26T17:44:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 0x560058b7b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b7b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058be7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba301ac3a0 con 0x560058b6d420 2024-03-26T17:44:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba301abcd0 con 0x560058b6ca30 2024-03-26T17:44:29.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba301a8f10 con 0x560058b72590 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba387b0740 10 client.0 _ll_register_callbacks cb 0x7fba24009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b7b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b7b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56102/0 (socket says 172.21.15.105:56102) 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba2effd640 1 -- 172.21.15.105:0/675752061 learned_addr learned my addr 172.21.15.105:0/675752061 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 0x560058b7b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.527+0000 7fba2ffff640 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058be7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba2effd640 1 -- 172.21.15.105:0/675752061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 msgr2=0x560058be7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba2effd640 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058be7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba2effd640 1 -- 172.21.15.105:0/675752061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 msgr2=0x560058b7b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba2effd640 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 0x560058b7b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba2effd640 1 -- 172.21.15.105:0/675752061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560058b7bce0 con 0x560058b6d420 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba2effd640 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b7b6a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fba24000c00 tx=0x7fba240128a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b72b6cfe2b70eff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba0effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba0effd640 1 -- 172.21.15.105:0/675752061 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2400fe80 con 0x560058b6d420 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba387b0740 1 -- 172.21.15.105:0/675752061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba280044f0 con 0x560058b6d420 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba0effd640 1 -- 172.21.15.105:0/675752061 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fba24018040 con 0x560058b6d420 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba0effd640 1 -- 172.21.15.105:0/675752061 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fba2401cd00 con 0x560058b6d420 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba387b0740 10 client.17652 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba387b0740 1 -- 172.21.15.105:0/675752061 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560058be89c0 con 0x560058b6d420 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.528+0000 7fba0effd640 1 -- 172.21.15.105:0/675752061 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba280044f0 con 0x560058b6d420 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba0effd640 1 -- 172.21.15.105:0/675752061 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fba24004570 con 0x560058b6d420 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba387b0740 10 client.17652 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:29.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba387b0740 10 client.17652 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba387b0740 20 client.17652 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba387b0740 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x560058be93a0 0x560058beb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba387b0740 4 client.17652 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba387b0740 1 -- 172.21.15.105:0/675752061 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560058ad9200 con 0x560058be93a0 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x560058be93a0 0x560058beb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba0effd640 10 client.17652 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba2f7fe640 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x560058be93a0 0x560058beb7c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=815af64470524be2 server_cookie=d78444bbfb6b7fe6 in_seq=0 out_seq=0 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba0effd640 1 -- 172.21.15.105:0/675752061 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560058ad9200 con 0x560058be93a0 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba0effd640 10 client.17652 handle_command_reply: tid=0 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.529+0000 7fba387b0740 1 client.17652 shutdown 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 -- 172.21.15.105:0/675752061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 msgr2=0x560058b7b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b7b6a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fba24000c00 tx=0x7fba240128a0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 -- 172.21.15.105:0/675752061 shutdown_connections 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x560058be93a0 0x560058beb7c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560058b72590 0x560058be7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560058b6d420 0x560058b7b6a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba0effd640 0 client.17652 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 --2- 172.21.15.105:0/675752061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560058b6ca30 0x560058b7b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 -- 172.21.15.105:0/675752061 >> 172.21.15.105:0/675752061 conn(0x560058b67fa0 msgr2=0x560058b6e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 -- 172.21.15.105:0/675752061 shutdown_connections 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 1 -- 172.21.15.105:0/675752061 wait complete. 2024-03-26T17:44:29.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba387b0740 20 client.17652 trim_cache size 0 max 16384 2024-03-26T17:44:29.531 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba35af1640 1 -- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2f20 msgr2=0x7fba180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba35af1640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2f20 0x7fba180a53e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fba28009a60 tx=0x7fba28048000 comp rx=0 tx=0).stop 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba35af1640 1 -- 172.21.15.105:0/1795933774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba30074320 msgr2=0x7fba30199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.530+0000 7fba35af1640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba30074320 0x7fba30199d50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba2000e9f0 tx=0x7fba2000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.531+0000 7fba35af1640 1 -- 172.21.15.105:0/1795933774 shutdown_connections 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.531+0000 7fba35af1640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba180a2f20 0x7fba180a53e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.531+0000 7fba35af1640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba301097b0 0x7fba301a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.531+0000 7fba35af1640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba30075900 0x7fba3019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.531+0000 7fba35af1640 1 --2- 172.21.15.105:0/1795933774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba30074320 0x7fba30199d50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.531+0000 7fba35af1640 1 -- 172.21.15.105:0/1795933774 >> 172.21.15.105:0/1795933774 conn(0x7fba300fce60 msgr2=0x7fba301056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.531+0000 7fba35af1640 1 -- 172.21.15.105:0/1795933774 shutdown_connections 2024-03-26T17:44:29.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.531+0000 7fba35af1640 1 -- 172.21.15.105:0/1795933774 wait complete. 2024-03-26T17:44:29.535 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:44:29.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 -- 172.21.15.105:0/603478338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65381019b0 msgr2=0x7f6538101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 --2- 172.21.15.105:0/603478338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65381019b0 0x7f6538101e30 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6528009900 tx=0x7f652802bc30 comp rx=0 tx=0).stop 2024-03-26T17:44:29.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 -- 172.21.15.105:0/603478338 shutdown_connections 2024-03-26T17:44:29.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 --2- 172.21.15.105:0/603478338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6538102370 0x7f6538108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 --2- 172.21.15.105:0/603478338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65381019b0 0x7f6538101e30 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 --2- 172.21.15.105:0/603478338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65381007c0 0x7f6538100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 -- 172.21.15.105:0/603478338 >> 172.21.15.105:0/603478338 conn(0x7f65380fbd50 msgr2=0x7f65380fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 -- 172.21.15.105:0/603478338 shutdown_connections 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.667+0000 7f65400a1640 1 -- 172.21.15.105:0/603478338 wait complete. 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f65400a1640 1 Processor -- start 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f65400a1640 1 -- start start 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f65400a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65381007c0 0x7f6538199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f65400a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65381019b0 0x7f653819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f65400a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6538102370 0x7f65381a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f65400a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6538068ef0 con 0x7f6538102370 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f65400a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6538068d70 con 0x7f65381019b0 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f65400a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6538069070 con 0x7f65381007c0 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f653de3d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65381007c0 0x7f6538199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f653d63c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65381019b0 0x7f653819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f653de3d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65381007c0 0x7f6538199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34854/0 (socket says 172.21.15.105:34854) 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f653e63e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6538102370 0x7f65381a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.668+0000 7f653de3d640 1 -- 172.21.15.105:0/1526596109 learned_addr learned my addr 172.21.15.105:0/1526596109 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f653e63e640 1 -- 172.21.15.105:0/1526596109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65381007c0 msgr2=0x7f6538199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f653e63e640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65381007c0 0x7f6538199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f653e63e640 1 -- 172.21.15.105:0/1526596109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65381019b0 msgr2=0x7f653819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f653e63e640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65381019b0 0x7f653819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f653e63e640 1 -- 172.21.15.105:0/1526596109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65381a1960 con 0x7f6538102370 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f653de3d640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65381007c0 0x7f6538199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f653e63e640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6538102370 0x7f65381a12f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6530009fd0 tx=0x7f653000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f652effd640 1 -- 172.21.15.105:0/1526596109 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653000ce20 con 0x7f6538102370 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f65400a1640 1 -- 172.21.15.105:0/1526596109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65381a1c50 con 0x7f6538102370 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f652effd640 1 -- 172.21.15.105:0/1526596109 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6530005020 con 0x7f6538102370 2024-03-26T17:44:29.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f652effd640 1 -- 172.21.15.105:0/1526596109 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530010690 con 0x7f6538102370 2024-03-26T17:44:29.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.669+0000 7f65400a1640 1 -- 172.21.15.105:0/1526596109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65381a2170 con 0x7f6538102370 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.672+0000 7f652effd640 1 -- 172.21.15.105:0/1526596109 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f65300051d0 con 0x7f6538102370 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.672+0000 7f652effd640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65240a2f20 0x7f65240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.672+0000 7f6542d6b740 1 Processor -- start 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.672+0000 7f652effd640 1 -- 172.21.15.105:0/1526596109 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f65300cfae0 con 0x7f6538102370 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.672+0000 7f653de3d640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65240a2f20 0x7f65240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f6542d6b740 1 -- start start 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e24a30 0x5578b9e24e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e29f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e2a590 0x5578b9e2ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f6542d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d89970 con 0x5578b9e24a30 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f6542d6b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d31d70 con 0x5578b9e25420 2024-03-26T17:44:29.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f6542d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d85920 con 0x5578b9e2a590 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653de3d640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65240a2f20 0x7f65240a53e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f65340058c0 tx=0x7f6534005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653de3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e24a30 0x5578b9e24e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653d63c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e29f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653de3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e24a30 0x5578b9e24e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56146/0 (socket says 172.21.15.105:56146) 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653d63c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e29f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38872/0 (socket says 172.21.15.105:38872) 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653e63e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e2a590 0x5578b9e2ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653de3d640 1 -- 172.21.15.105:0/326976897 learned_addr learned my addr 172.21.15.105:0/326976897 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653d63c640 1 -- 172.21.15.105:0/326976897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e2a590 msgr2=0x5578b9e2ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653d63c640 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e2a590 0x5578b9e2ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653d63c640 1 -- 172.21.15.105:0/326976897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e24a30 msgr2=0x5578b9e24e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653d63c640 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e24a30 0x5578b9e24e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653d63c640 1 -- 172.21.15.105:0/326976897 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b9a9d3f0 con 0x5578b9e25420 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653de3d640 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e24a30 0x5578b9e24e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.673+0000 7f653e63e640 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e2a590 0x5578b9e2ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.674+0000 7f653d63c640 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e29f20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6528000c00 tx=0x7f6528009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc90829af49ee702 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.674+0000 7f6523fff640 1 -- 172.21.15.105:0/326976897 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528045020 con 0x5578b9e25420 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.674+0000 7f6523fff640 1 -- 172.21.15.105:0/326976897 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6528004040 con 0x5578b9e25420 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.674+0000 7f6523fff640 1 -- 172.21.15.105:0/326976897 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6528005580 con 0x5578b9e25420 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.674+0000 7f6542d6b740 1 -- 172.21.15.105:0/326976897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 msgr2=0x5578b9e29f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e29f20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6528000c00 tx=0x7f6528009560 comp rx=0 tx=0).stop 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 -- 172.21.15.105:0/326976897 shutdown_connections 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e2a590 0x5578b9e2ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e29f20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 --2- 172.21.15.105:0/326976897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e24a30 0x5578b9e24e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 -- 172.21.15.105:0/326976897 >> 172.21.15.105:0/326976897 conn(0x5578b9e1ffa0 msgr2=0x5578b9e203a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 -- 172.21.15.105:0/326976897 shutdown_connections 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 -- 172.21.15.105:0/326976897 wait complete. 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 -- start start 2024-03-26T17:44:29.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.675+0000 7f6542d6b740 1 Processor -- start 2024-03-26T17:44:29.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e2eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e2f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e36440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f6542d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5578b9e2d4e0 con 0x5578b9e2a590 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f6542d6b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5578b9d45310 con 0x5578b9e25420 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f6542d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5578b9e2d660 con 0x5578b9e24a30 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f6542d6b740 10 client.0 _ll_register_callbacks cb 0x5578b9d80c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653e63e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e36440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653d63c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e2f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653de3d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e2eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653d63c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e2f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38880/0 (socket says 172.21.15.105:38880) 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653de3d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e2eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34858/0 (socket says 172.21.15.105:34858) 2024-03-26T17:44:29.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653d63c640 1 -- 172.21.15.105:0/2355374139 learned_addr learned my addr 172.21.15.105:0/2355374139 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653e63e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e36440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56160/0 (socket says 172.21.15.105:56160) 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653de3d640 1 -- 172.21.15.105:0/2355374139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 msgr2=0x5578b9e2f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653de3d640 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e2f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653de3d640 1 -- 172.21.15.105:0/2355374139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 msgr2=0x5578b9e36440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653de3d640 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e36440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653de3d640 1 -- 172.21.15.105:0/2355374139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b9b677d0 con 0x5578b9e24a30 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653e63e640 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e36440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f653de3d640 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e2eec0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6534013010 tx=0x7f6534004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6111e46a370ff074 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f65217fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.676+0000 7f65217fa640 1 -- 172.21.15.105:0/2355374139 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400fe90 con 0x5578b9e24a30 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 1 -- 172.21.15.105:0/2355374139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6528004040 con 0x5578b9e24a30 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f65217fa640 1 -- 172.21.15.105:0/2355374139 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f653401c8e0 con 0x5578b9e24a30 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f65217fa640 1 -- 172.21.15.105:0/2355374139 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f653401cae0 con 0x5578b9e24a30 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 10 client.27499 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:29.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 1 -- 172.21.15.105:0/2355374139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578b9e39820 con 0x5578b9e24a30 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f65217fa640 1 -- 172.21.15.105:0/2355374139 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6528004040 con 0x5578b9e24a30 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f65217fa640 1 -- 172.21.15.105:0/2355374139 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6534004a40 con 0x5578b9e24a30 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 10 client.27499 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 10 client.27499 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 20 client.27499 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5578b9e3af70 0x5578b9e3d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 4 client.27499 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f6542d6b740 1 -- 172.21.15.105:0/2355374139 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578b9d96eb0 con 0x5578b9e3af70 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.677+0000 7f653d63c640 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5578b9e3af70 0x5578b9e3d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.678+0000 7f65217fa640 10 client.27499 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.678+0000 7f653d63c640 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5578b9e3af70 0x5578b9e3d430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c00c5c07d6d16b5 server_cookie=253137736f9acb56 in_seq=0 out_seq=0 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f65217fa640 1 -- 172.21.15.105:0/2355374139 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5578b9d96eb0 con 0x5578b9e3af70 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f65217fa640 10 client.27499 handle_command_reply: tid=0 2024-03-26T17:44:29.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 client.27499 shutdown 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 -- 172.21.15.105:0/2355374139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 msgr2=0x5578b9e2eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e2eec0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6534013010 tx=0x7f6534004730 comp rx=0 tx=0).stop 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 -- 172.21.15.105:0/2355374139 shutdown_connections 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5578b9e3af70 0x5578b9e3d430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e36440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e25420 0x5578b9e2f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 --2- 172.21.15.105:0/2355374139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e2eec0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 -- 172.21.15.105:0/2355374139 >> 172.21.15.105:0/2355374139 conn(0x5578b9e1ffa0 msgr2=0x5578b9e1f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f65217fa640 0 client.27499 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 -- 172.21.15.105:0/2355374139 shutdown_connections 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 1 -- 172.21.15.105:0/2355374139 wait complete. 2024-03-26T17:44:29.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.679+0000 7f6542d6b740 20 client.27499 trim_cache size 0 max 16384 2024-03-26T17:44:29.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f6542d6b740 1 Processor -- start 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f6542d6b740 1 -- start start 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e24a30 0x5578b9e26920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e25420 0x5578b9e26e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f6542d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65381aaf20 con 0x5578b9e2a590 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f6542d6b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65381aca60 con 0x5578b9e24a30 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f6542d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65381ab5f0 con 0x5578b9e25420 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e24a30 0x5578b9e26920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653e63e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e24a30 0x5578b9e26920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38890/0 (socket says 172.21.15.105:38890) 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 -- 172.21.15.105:0/1883956129 learned_addr learned my addr 172.21.15.105:0/1883956129 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653d63c640 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e25420 0x5578b9e26e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 -- 172.21.15.105:0/1883956129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e25420 msgr2=0x5578b9e26e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e25420 0x5578b9e26e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 -- 172.21.15.105:0/1883956129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 msgr2=0x5578b9e363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 -- 172.21.15.105:0/1883956129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b9e1f8f0 con 0x5578b9e24a30 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653d63c640 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e25420 0x5578b9e26e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653e63e640 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.700+0000 7f653de3d640 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e24a30 0x5578b9e26920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65340097f0 tx=0x7f6534005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abcab3386269d87 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f65237fe640 1 -- 172.21.15.105:0/1883956129 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653401c8a0 con 0x5578b9e24a30 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f65237fe640 1 -- 172.21.15.105:0/1883956129 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6534028420 con 0x5578b9e24a30 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f65237fe640 1 -- 172.21.15.105:0/1883956129 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534028d80 con 0x5578b9e24a30 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 -- 172.21.15.105:0/1883956129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e24a30 msgr2=0x5578b9e26920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e24a30 0x5578b9e26920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65340097f0 tx=0x7f6534005e30 comp rx=0 tx=0).stop 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 -- 172.21.15.105:0/1883956129 shutdown_connections 2024-03-26T17:44:29.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e2a590 0x5578b9e363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e25420 0x5578b9e26e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 --2- 172.21.15.105:0/1883956129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e24a30 0x5578b9e26920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 -- 172.21.15.105:0/1883956129 >> 172.21.15.105:0/1883956129 conn(0x5578b9e1ffa0 msgr2=0x5578b9e25b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 -- 172.21.15.105:0/1883956129 shutdown_connections 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.701+0000 7f6542d6b740 1 -- 172.21.15.105:0/1883956129 wait complete. 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 1 -- start start 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 1 Processor -- start 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e32f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e25420 0x5578b9e33450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e2a590 0x5578b9e9f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65381ac390 con 0x5578b9e25420 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65381abcc0 con 0x5578b9e2a590 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65381a8f00 con 0x5578b9e24a30 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f6542d6b740 10 client.0 _ll_register_callbacks cb 0x7f653400f4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:29.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f653d63c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e25420 0x5578b9e33450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f653de3d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e32f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f653e63e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e2a590 0x5578b9e9f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f653d63c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e25420 0x5578b9e33450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56172/0 (socket says 172.21.15.105:56172) 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.702+0000 7f653d63c640 1 -- 172.21.15.105:0/3361059579 learned_addr learned my addr 172.21.15.105:0/3361059579 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f653de3d640 1 -- 172.21.15.105:0/3361059579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e2a590 msgr2=0x5578b9e9f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f653de3d640 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e2a590 0x5578b9e9f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f653de3d640 1 -- 172.21.15.105:0/3361059579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e25420 msgr2=0x5578b9e33450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f653de3d640 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e25420 0x5578b9e33450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f653de3d640 1 -- 172.21.15.105:0/3361059579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b9e36520 con 0x5578b9e24a30 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f653d63c640 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e25420 0x5578b9e33450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f653e63e640 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e2a590 0x5578b9e9f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f653de3d640 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e32f10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6534004e10 tx=0x7f6534004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5355964262a2d2cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f65217fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:44:29.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f65217fa640 1 -- 172.21.15.105:0/3361059579 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400fe90 con 0x5578b9e24a30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f65217fa640 1 -- 172.21.15.105:0/3361059579 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f6534028870 con 0x5578b9e24a30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f6542d6b740 1 -- 172.21.15.105:0/3361059579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6534028420 con 0x5578b9e24a30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f65217fa640 1 -- 172.21.15.105:0/3361059579 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6534009040 con 0x5578b9e24a30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f6542d6b740 10 client.27505 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f6542d6b740 1 -- 172.21.15.105:0/3361059579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5578b9ea0770 con 0x5578b9e24a30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f65217fa640 1 -- 172.21.15.105:0/3361059579 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6534028420 con 0x5578b9e24a30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.703+0000 7f65217fa640 1 -- 172.21.15.105:0/3361059579 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f653403a760 con 0x5578b9e24a30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 10 client.27505 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 10 client.27505 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 20 client.27505 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5578b9ea0f80 0x5578b9ea33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 4 client.27505 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 1 -- 172.21.15.105:0/3361059579 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578b9d91200 con 0x5578b9ea0f80 2024-03-26T17:44:29.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f653d63c640 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5578b9ea0f80 0x5578b9ea33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f653d63c640 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5578b9ea0f80 0x5578b9ea33a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=548da22eeaf04bea server_cookie=485c0fec06c3af0a in_seq=0 out_seq=0 2024-03-26T17:44:29.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f65217fa640 10 client.27505 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:44:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f65217fa640 1 -- 172.21.15.105:0/3361059579 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578b9d91200 con 0x5578b9ea0f80 2024-03-26T17:44:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f65217fa640 10 client.27505 handle_command_reply: tid=0 2024-03-26T17:44:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 1 client.27505 shutdown 2024-03-26T17:44:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 1 -- 172.21.15.105:0/3361059579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 msgr2=0x5578b9e32f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.704+0000 7f6542d6b740 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e32f10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6534004e10 tx=0x7f6534004b60 comp rx=0 tx=0).stop 2024-03-26T17:44:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 1 -- 172.21.15.105:0/3361059579 shutdown_connections 2024-03-26T17:44:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5578b9ea0f80 0x5578b9ea33a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578b9e2a590 0x5578b9e9f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5578b9e25420 0x5578b9e33450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65217fa640 0 client.27505 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:44:29.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 1 --2- 172.21.15.105:0/3361059579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5578b9e24a30 0x5578b9e32f10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 1 -- 172.21.15.105:0/3361059579 >> 172.21.15.105:0/3361059579 conn(0x5578b9e1ffa0 msgr2=0x5578b9e25b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 1 -- 172.21.15.105:0/3361059579 shutdown_connections 2024-03-26T17:44:29.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 1 -- 172.21.15.105:0/3361059579 wait complete. 2024-03-26T17:44:29.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f6542d6b740 20 client.27505 trim_cache size 0 max 16384 2024-03-26T17:44:29.704 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:44:29.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 -- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65240a2f20 msgr2=0x7f65240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65240a2f20 0x7f65240a53e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f65340058c0 tx=0x7f6534005f90 comp rx=0 tx=0).stop 2024-03-26T17:44:29.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 -- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6538102370 msgr2=0x7f65381a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6538102370 0x7f65381a12f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6530009fd0 tx=0x7f653000ef10 comp rx=0 tx=0).stop 2024-03-26T17:44:29.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 -- 172.21.15.105:0/1526596109 shutdown_connections 2024-03-26T17:44:29.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65240a2f20 0x7f65240a53e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6538102370 0x7f65381a12f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65381019b0 0x7f653819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 --2- 172.21.15.105:0/1526596109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65381007c0 0x7f6538199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.705+0000 7f65400a1640 1 -- 172.21.15.105:0/1526596109 >> 172.21.15.105:0/1526596109 conn(0x7f65380fbd50 msgr2=0x7f653810be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.706+0000 7f65400a1640 1 -- 172.21.15.105:0/1526596109 shutdown_connections 2024-03-26T17:44:29.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.706+0000 7f65400a1640 1 -- 172.21.15.105:0/1526596109 wait complete. 2024-03-26T17:44:29.709 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:44:29.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 -- 172.21.15.105:0/2343320614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1019b0 msgr2=0x7f260c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 --2- 172.21.15.105:0/2343320614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1019b0 0x7f260c101e30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2600009900 tx=0x7f260002bc30 comp rx=0 tx=0).stop 2024-03-26T17:44:29.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 -- 172.21.15.105:0/2343320614 shutdown_connections 2024-03-26T17:44:29.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 --2- 172.21.15.105:0/2343320614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f260c102370 0x7f260c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 --2- 172.21.15.105:0/2343320614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1019b0 0x7f260c101e30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 --2- 172.21.15.105:0/2343320614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c1007c0 0x7f260c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 -- 172.21.15.105:0/2343320614 >> 172.21.15.105:0/2343320614 conn(0x7f260c0fbd50 msgr2=0x7f260c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 -- 172.21.15.105:0/2343320614 shutdown_connections 2024-03-26T17:44:29.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.847+0000 7f2610b3b640 1 -- 172.21.15.105:0/2343320614 wait complete. 2024-03-26T17:44:29.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.848+0000 7f2610b3b640 1 Processor -- start 2024-03-26T17:44:29.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.848+0000 7f2610b3b640 1 -- start start 2024-03-26T17:44:29.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.848+0000 7f2610b3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1007c0 0x7f260c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.848+0000 7f2610b3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f260c1019b0 0x7f260c196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.848+0000 7f2610b3b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c102370 0x7f260c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.848+0000 7f2610b3b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f260c10c2d0 con 0x7f260c1007c0 2024-03-26T17:44:29.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.848+0000 7f2610b3b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f260c10c150 con 0x7f260c102370 2024-03-26T17:44:29.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.848+0000 7f2610b3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f260c10c450 con 0x7f260c1019b0 2024-03-26T17:44:29.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1007c0 0x7f260c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1007c0 0x7f260c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56202/0 (socket says 172.21.15.105:56202) 2024-03-26T17:44:29.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 -- 172.21.15.105:0/929918028 learned_addr learned my addr 172.21.15.105:0/929918028 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260ad9d640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c102370 0x7f260c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f2609d9b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f260c1019b0 0x7f260c196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 -- 172.21.15.105:0/929918028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f260c1019b0 msgr2=0x7f260c196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f260c1019b0 0x7f260c196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 -- 172.21.15.105:0/929918028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c102370 msgr2=0x7f260c19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c102370 0x7f260c19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 -- 172.21.15.105:0/929918028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c19e630 con 0x7f260c1007c0 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f260a59c640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1007c0 0x7f260c196a10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9c0 tx=0x7f25fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f25fb7fe640 1 -- 172.21.15.105:0/929918028 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00cd80 con 0x7f260c1007c0 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f25fb7fe640 1 -- 172.21.15.105:0/929918028 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc0035f0 con 0x7f260c1007c0 2024-03-26T17:44:29.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f2610b3b640 1 -- 172.21.15.105:0/929918028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c19e920 con 0x7f260c1007c0 2024-03-26T17:44:29.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f25fb7fe640 1 -- 172.21.15.105:0/929918028 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc010690 con 0x7f260c1007c0 2024-03-26T17:44:29.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.849+0000 7f2610b3b640 1 -- 172.21.15.105:0/929918028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c19ee30 con 0x7f260c1007c0 2024-03-26T17:44:29.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.851+0000 7f25fb7fe640 1 -- 172.21.15.105:0/929918028 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f25fc0037a0 con 0x7f260c1007c0 2024-03-26T17:44:29.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.852+0000 7f25fb7fe640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25f40a2fa0 0x7f25f40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.852+0000 7f2609d9b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25f40a2fa0 0x7f25f40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.852+0000 7f25fb7fe640 1 -- 172.21.15.105:0/929918028 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f25fc0cf720 con 0x7f260c1007c0 2024-03-26T17:44:29.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.853+0000 7f2609d9b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25f40a2fa0 0x7f25f40a5460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2600009230 tx=0x7f2600005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2613801740 1 Processor -- start 2024-03-26T17:44:29.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2613801740 1 -- start start 2024-03-26T17:44:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2613801740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2613801740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 0x564a530f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2613801740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530f4590 0x564a530f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2613801740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564a53053970 con 0x564a530eea30 2024-03-26T17:44:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2613801740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564a52ffbd70 con 0x564a530f4590 2024-03-26T17:44:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2613801740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564a5304f920 con 0x564a530ef420 2024-03-26T17:44:29.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f260a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f260a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530eee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56204/0 (socket says 172.21.15.105:56204) 2024-03-26T17:44:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f260a59c640 1 -- 172.21.15.105:0/709167003 learned_addr learned my addr 172.21.15.105:0/709167003 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f260ad9d640 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530f4590 0x564a530f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f2609d9b640 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 0x564a530f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f260a59c640 1 -- 172.21.15.105:0/709167003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 msgr2=0x564a530f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f260a59c640 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 0x564a530f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f260a59c640 1 -- 172.21.15.105:0/709167003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530f4590 msgr2=0x564a530f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.854+0000 7f260a59c640 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530f4590 0x564a530f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.855+0000 7f260a59c640 1 -- 172.21.15.105:0/709167003 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a52d673f0 con 0x564a530eea30 2024-03-26T17:44:29.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.855+0000 7f260ad9d640 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530f4590 0x564a530f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.855+0000 7f260a59c640 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530eee50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f25fc098270 tx=0x7f25fc09ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ab068cafcdddfe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.855+0000 7f25f8ff9640 1 -- 172.21.15.105:0/709167003 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc09c510 con 0x564a530eea30 2024-03-26T17:44:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.855+0000 7f25f8ff9640 1 -- 172.21.15.105:0/709167003 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc09be10 con 0x564a530eea30 2024-03-26T17:44:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.855+0000 7f25f8ff9640 1 -- 172.21.15.105:0/709167003 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc09c2d0 con 0x564a530eea30 2024-03-26T17:44:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- 172.21.15.105:0/709167003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 msgr2=0x564a530eee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530eee50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f25fc098270 tx=0x7f25fc09ba10 comp rx=0 tx=0).stop 2024-03-26T17:44:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- 172.21.15.105:0/709167003 shutdown_connections 2024-03-26T17:44:29.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530f4590 0x564a530f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 0x564a530f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 --2- 172.21.15.105:0/709167003 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530eee50 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- 172.21.15.105:0/709167003 >> 172.21.15.105:0/709167003 conn(0x564a530e9fa0 msgr2=0x564a530ea3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- 172.21.15.105:0/709167003 shutdown_connections 2024-03-26T17:44:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- 172.21.15.105:0/709167003 wait complete. 2024-03-26T17:44:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- start start 2024-03-26T17:44:29.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 Processor -- start 2024-03-26T17:44:29.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530f9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a530f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a53100580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564a530f7540 con 0x564a530eea30 2024-03-26T17:44:29.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564a5300f310 con 0x564a530ef420 2024-03-26T17:44:29.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.856+0000 7f2613801740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564a530f76c0 con 0x564a530f4590 2024-03-26T17:44:29.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f2613801740 10 client.0 _ll_register_callbacks cb 0x564a5304ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:29.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56210/0 (socket says 172.21.15.105:56210) 2024-03-26T17:44:29.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 -- 172.21.15.105:0/1079462385 learned_addr learned my addr 172.21.15.105:0/1079462385 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f2609d9b640 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a530f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260ad9d640 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a53100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 -- 172.21.15.105:0/1079462385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 msgr2=0x564a53100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a53100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 -- 172.21.15.105:0/1079462385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 msgr2=0x564a530f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a530f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 -- 172.21.15.105:0/1079462385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a52e317d0 con 0x564a530eea30 2024-03-26T17:44:29.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f260a59c640 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530f9000 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0cd550 tx=0x7f25fc09c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc75d3b845696b56 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f25f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f25f1ffb640 1 -- 172.21.15.105:0/1079462385 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc07e070 con 0x564a530eea30 2024-03-26T17:44:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f25f1ffb640 1 -- 172.21.15.105:0/1079462385 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f25fc0a4590 con 0x564a530eea30 2024-03-26T17:44:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f2613801740 1 -- 172.21.15.105:0/1079462385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc09be10 con 0x564a530eea30 2024-03-26T17:44:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f25f1ffb640 1 -- 172.21.15.105:0/1079462385 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25fc0a4820 con 0x564a530eea30 2024-03-26T17:44:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f2613801740 10 client.17682 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.857+0000 7f2613801740 1 -- 172.21.15.105:0/1079462385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a531038f0 con 0x564a530eea30 2024-03-26T17:44:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f25f1ffb640 1 -- 172.21.15.105:0/1079462385 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc09be10 con 0x564a530eea30 2024-03-26T17:44:29.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f25f1ffb640 1 -- 172.21.15.105:0/1079462385 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f25fc081070 con 0x564a530eea30 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f2613801740 10 client.17682 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f2613801740 10 client.17682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f2613801740 20 client.17682 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f2613801740 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a53104c40 0x564a53107030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f2613801740 4 client.17682 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f2613801740 1 -- 172.21.15.105:0/1079462385 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a53060eb0 con 0x564a53104c40 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f2609d9b640 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a53104c40 0x564a53107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f25f1ffb640 10 client.17682 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:44:29.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.858+0000 7f2609d9b640 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a53104c40 0x564a53107030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=efd2544373eeae25 server_cookie=efd8fce4de26115b in_seq=0 out_seq=0 2024-03-26T17:44:29.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f25f1ffb640 1 -- 172.21.15.105:0/1079462385 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564a53060eb0 con 0x564a53104c40 2024-03-26T17:44:29.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f25f1ffb640 10 client.17682 handle_command_reply: tid=0 2024-03-26T17:44:29.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 client.17682 shutdown 2024-03-26T17:44:29.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 -- 172.21.15.105:0/1079462385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 msgr2=0x564a530f9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530f9000 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0cd550 tx=0x7f25fc09c400 comp rx=0 tx=0).stop 2024-03-26T17:44:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 -- 172.21.15.105:0/1079462385 shutdown_connections 2024-03-26T17:44:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a53100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a53104c40 0x564a53107030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a530f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 --2- 172.21.15.105:0/1079462385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a530f9000 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 -- 172.21.15.105:0/1079462385 >> 172.21.15.105:0/1079462385 conn(0x564a530e9fa0 msgr2=0x564a530e91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f25f1ffb640 0 client.17682 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:44:29.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 -- 172.21.15.105:0/1079462385 shutdown_connections 2024-03-26T17:44:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 1 -- 172.21.15.105:0/1079462385 wait complete. 2024-03-26T17:44:29.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.859+0000 7f2613801740 20 client.17682 trim_cache size 0 max 16384 2024-03-26T17:44:29.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2613801740 1 Processor -- start 2024-03-26T17:44:29.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2613801740 1 -- start start 2024-03-26T17:44:29.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2613801740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530eea30 0x564a530f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2613801740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 0x564a530ed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2613801740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530f4590 0x564a530f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2613801740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1a6bc0 con 0x564a530f4590 2024-03-26T17:44:29.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2613801740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1a8700 con 0x564a530eea30 2024-03-26T17:44:29.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2613801740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1a7290 con 0x564a530ef420 2024-03-26T17:44:29.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530f4590 0x564a530f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530eea30 0x564a530f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530f4590 0x564a530f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56212/0 (socket says 172.21.15.105:56212) 2024-03-26T17:44:29.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260ad9d640 1 -- 172.21.15.105:0/890982603 learned_addr learned my addr 172.21.15.105:0/890982603 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f2609d9b640 1 --2- 172.21.15.105:0/890982603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 0x564a530ed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260a59c640 1 -- 172.21.15.105:0/890982603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 msgr2=0x564a530ed910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260a59c640 1 --2- 172.21.15.105:0/890982603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 0x564a530ed910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260a59c640 1 -- 172.21.15.105:0/890982603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530f4590 msgr2=0x564a530f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260a59c640 1 --2- 172.21.15.105:0/890982603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530f4590 0x564a530f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.874+0000 7f260a59c640 1 -- 172.21.15.105:0/890982603 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a530efc40 con 0x564a530eea30 2024-03-26T17:44:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.875+0000 7f260a59c640 1 --2- 172.21.15.105:0/890982603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530eea30 0x564a530f94a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c6a0 tx=0x7f25fc0bc090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b28a9d7e8ab606b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.875+0000 7f25f3fff640 1 -- 172.21.15.105:0/890982603 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc07e070 con 0x564a530eea30 2024-03-26T17:44:29.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.875+0000 7f25f3fff640 1 -- 172.21.15.105:0/890982603 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc09c510 con 0x564a530eea30 2024-03-26T17:44:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.875+0000 7f25f3fff640 1 -- 172.21.15.105:0/890982603 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc097b60 con 0x564a530eea30 2024-03-26T17:44:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- 172.21.15.105:0/890982603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530eea30 msgr2=0x564a530f94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 --2- 172.21.15.105:0/890982603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530eea30 0x564a530f94a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c6a0 tx=0x7f25fc0bc090 comp rx=0 tx=0).stop 2024-03-26T17:44:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- 172.21.15.105:0/890982603 shutdown_connections 2024-03-26T17:44:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 --2- 172.21.15.105:0/890982603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530f4590 0x564a530f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 --2- 172.21.15.105:0/890982603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530ef420 0x564a530ed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 --2- 172.21.15.105:0/890982603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530eea30 0x564a530f94a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- 172.21.15.105:0/890982603 >> 172.21.15.105:0/890982603 conn(0x564a530e9fa0 msgr2=0x564a530ed200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- 172.21.15.105:0/890982603 shutdown_connections 2024-03-26T17:44:29.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- 172.21.15.105:0/890982603 wait complete. 2024-03-26T17:44:29.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- start start 2024-03-26T17:44:29.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 Processor -- start 2024-03-26T17:44:29.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a53165890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a53165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a5316ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1a8030 con 0x564a530eea30 2024-03-26T17:44:29.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1a7960 con 0x564a530ef420 2024-03-26T17:44:29.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1a4ba0 con 0x564a530f4590 2024-03-26T17:44:29.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2613801740 10 client.0 _ll_register_callbacks cb 0x7f2604009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:29.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f260a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a53165890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f2609d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a53165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f260a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a53165890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56220/0 (socket says 172.21.15.105:56220) 2024-03-26T17:44:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.876+0000 7f260a59c640 1 -- 172.21.15.105:0/3265779593 learned_addr learned my addr 172.21.15.105:0/3265779593 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f260ad9d640 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a5316ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f260a59c640 1 -- 172.21.15.105:0/3265779593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 msgr2=0x564a5316ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f260a59c640 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a5316ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f260a59c640 1 -- 172.21.15.105:0/3265779593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 msgr2=0x564a53165dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:29.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f260a59c640 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a53165dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f260a59c640 1 -- 172.21.15.105:0/3265779593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a5316d400 con 0x564a530eea30 2024-03-26T17:44:29.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f260ad9d640 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a5316ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:44:29.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f2609d9b640 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a53165dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:29.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f260a59c640 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a53165890 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f25fc09bb10 tx=0x7f25fc003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dfdbc2b82a61b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:29.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f25f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:29.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f25f1ffb640 1 -- 172.21.15.105:0/3265779593 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc07e070 con 0x564a530eea30 2024-03-26T17:44:29.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f25f1ffb640 1 -- 172.21.15.105:0/3265779593 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f25fc097da0 con 0x564a530eea30 2024-03-26T17:44:29.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f2613801740 1 -- 172.21.15.105:0/3265779593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc09c510 con 0x564a530eea30 2024-03-26T17:44:29.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f25f1ffb640 1 -- 172.21.15.105:0/3265779593 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25fc097b60 con 0x564a530eea30 2024-03-26T17:44:29.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f2613801740 10 client.17688 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:29.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f2613801740 1 -- 172.21.15.105:0/3265779593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a5316d400 con 0x564a530eea30 2024-03-26T17:44:29.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.877+0000 7f25f1ffb640 1 -- 172.21.15.105:0/3265779593 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc09c510 con 0x564a530eea30 2024-03-26T17:44:29.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f25f1ffb640 1 -- 172.21.15.105:0/3265779593 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f25fc081070 con 0x564a530eea30 2024-03-26T17:44:29.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 10 client.17688 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:29.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 10 client.17688 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:44:29.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 20 client.17688 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:29.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a5316ece0 0x564a531710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:29.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 4 client.17688 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:44:29.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 -- 172.21.15.105:0/3265779593 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a53107560 con 0x564a5316ece0 2024-03-26T17:44:29.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2609d9b640 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a5316ece0 0x564a531710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2609d9b640 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a5316ece0 0x564a531710d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bb96891c41d096c server_cookie=c9b85a8d25a3b329 in_seq=0 out_seq=0 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f25f1ffb640 10 client.17688 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f25f1ffb640 1 -- 172.21.15.105:0/3265779593 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a53107560 con 0x564a5316ece0 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f25f1ffb640 10 client.17688 handle_command_reply: tid=0 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 client.17688 shutdown 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 -- 172.21.15.105:0/3265779593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 msgr2=0x564a53165890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a53165890 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f25fc09bb10 tx=0x7f25fc003d60 comp rx=0 tx=0).stop 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 -- 172.21.15.105:0/3265779593 shutdown_connections 2024-03-26T17:44:29.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a530f4590 0x564a5316ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a5316ece0 0x564a531710d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a530ef420 0x564a53165dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f25f1ffb640 0 client.17688 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:44:29.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 --2- 172.21.15.105:0/3265779593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a530eea30 0x564a53165890 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.878+0000 7f2613801740 1 -- 172.21.15.105:0/3265779593 >> 172.21.15.105:0/3265779593 conn(0x564a530e9fa0 msgr2=0x564a530e8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2613801740 1 -- 172.21.15.105:0/3265779593 shutdown_connections 2024-03-26T17:44:29.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2613801740 1 -- 172.21.15.105:0/3265779593 wait complete. 2024-03-26T17:44:29.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2613801740 20 client.17688 trim_cache size 0 max 16384 2024-03-26T17:44:29.891 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:44:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 -- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25f40a2fa0 msgr2=0x7f25f40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25f40a2fa0 0x7f25f40a5460 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2600009230 tx=0x7f2600005fd0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 -- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1007c0 msgr2=0x7f260c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:29.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1007c0 0x7f260c196a10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e9c0 tx=0x7f25fc00ee80 comp rx=0 tx=0).stop 2024-03-26T17:44:29.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 -- 172.21.15.105:0/929918028 shutdown_connections 2024-03-26T17:44:29.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25f40a2fa0 0x7f25f40a5460 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f260c102370 0x7f260c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f260c1019b0 0x7f260c196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 --2- 172.21.15.105:0/929918028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f260c1007c0 0x7f260c196a10 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 -- 172.21.15.105:0/929918028 >> 172.21.15.105:0/929918028 conn(0x7f260c0fbd50 msgr2=0x7f260c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 -- 172.21.15.105:0/929918028 shutdown_connections 2024-03-26T17:44:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:29.879+0000 7f2610b3b640 1 -- 172.21.15.105:0/929918028 wait complete. 2024-03-26T17:44:29.895 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:44:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.028+0000 7f030f234640 1 -- 172.21.15.105:0/2157456961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081007c0 msgr2=0x7f0308100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.028+0000 7f030f234640 1 --2- 172.21.15.105:0/2157456961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081007c0 0x7f0308100be0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009900 tx=0x7f02fc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:44:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.028+0000 7f030f234640 1 -- 172.21.15.105:0/2157456961 shutdown_connections 2024-03-26T17:44:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.028+0000 7f030f234640 1 --2- 172.21.15.105:0/2157456961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0308102370 0x7f0308108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.028+0000 7f030f234640 1 --2- 172.21.15.105:0/2157456961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03081019b0 0x7f0308101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.028+0000 7f030f234640 1 --2- 172.21.15.105:0/2157456961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081007c0 0x7f0308100be0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.028+0000 7f030f234640 1 -- 172.21.15.105:0/2157456961 >> 172.21.15.105:0/2157456961 conn(0x7f03080fbd50 msgr2=0x7f03080fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 -- 172.21.15.105:0/2157456961 shutdown_connections 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 -- 172.21.15.105:0/2157456961 wait complete. 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 Processor -- start 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 -- start start 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03081007c0 0x7f0308069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081019b0 0x7f030819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0308102370 0x7f030819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f030810c2c0 con 0x7f03081019b0 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f030810c140 con 0x7f0308102370 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030f234640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f030810c440 con 0x7f03081007c0 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f0307fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081019b0 0x7f030819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f0307fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081019b0 0x7f030819c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56238/0 (socket says 172.21.15.105:56238) 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f0307fff640 1 -- 172.21.15.105:0/1250413187 learned_addr learned my addr 172.21.15.105:0/1250413187 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030cfd0640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03081007c0 0x7f0308069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.029+0000 7f030d7d1640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0308102370 0x7f030819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0307fff640 1 -- 172.21.15.105:0/1250413187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03081007c0 msgr2=0x7f0308069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0307fff640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03081007c0 0x7f0308069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0307fff640 1 -- 172.21.15.105:0/1250413187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0308102370 msgr2=0x7f030819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0307fff640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0308102370 0x7f030819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0307fff640 1 -- 172.21.15.105:0/1250413187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030819f7b0 con 0x7f03081019b0 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0307fff640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081019b0 0x7f030819c880 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f02f400e9f0 tx=0x7f02f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0305ffb640 1 -- 172.21.15.105:0/1250413187 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f400cd80 con 0x7f03081019b0 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0305ffb640 1 -- 172.21.15.105:0/1250413187 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02f40108e0 con 0x7f03081019b0 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f030f234640 1 -- 172.21.15.105:0/1250413187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030819fa40 con 0x7f03081019b0 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f0305ffb640 1 -- 172.21.15.105:0/1250413187 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4018920 con 0x7f03081019b0 2024-03-26T17:44:30.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.030+0000 7f030f234640 1 -- 172.21.15.105:0/1250413187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030819ffe0 con 0x7f03081019b0 2024-03-26T17:44:30.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 184487 3.16 MB/sec execute 193 sec latency 15.810 ms 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0305ffb640 1 -- 172.21.15.105:0/1250413187 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f02f4010a90 con 0x7f03081019b0 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0311e19740 1 Processor -- start 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0311e19740 1 -- start start 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0311e19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b3ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0311e19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b43f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0311e19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b46a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0311e19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55edf0aa3970 con 0x55edf0b44590 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0311e19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55edf0a4bd70 con 0x55edf0b3ea30 2024-03-26T17:44:30.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0311e19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55edf0a9f920 con 0x55edf0b3f420 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.033+0000 7f0305ffb640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02f80a2fa0 0x7f02f80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f030d7d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b3ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f030d7d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b3ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38980/0 (socket says 172.21.15.105:38980) 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f030d7d1640 1 -- 172.21.15.105:0/2001729394 learned_addr learned my addr 172.21.15.105:0/2001729394 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f030d7d1640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02f80a2fa0 0x7f02f80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f0305ffb640 1 -- 172.21.15.105:0/1250413187 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f02f40cf6a0 con 0x7f03081019b0 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f0307fff640 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b46a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f030cfd0640 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b43f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f0307fff640 1 -- 172.21.15.105:0/2001729394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 msgr2=0x55edf0b43f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f0307fff640 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b43f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f0307fff640 1 -- 172.21.15.105:0/2001729394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 msgr2=0x55edf0b3ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f0307fff640 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b3ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f0307fff640 1 -- 172.21.15.105:0/2001729394 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf07b73f0 con 0x55edf0b44590 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f0307fff640 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b46a20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f02f409bb40 tx=0x7f02f409ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfce551d47362916 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f030d7d1640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02f80a2fa0 0x7f02f80a5460 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f03000099e0 tx=0x7f0300009c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f02f2ffd640 1 -- 172.21.15.105:0/2001729394 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f409be40 con 0x55edf0b44590 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f02f2ffd640 1 -- 172.21.15.105:0/2001729394 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02f409fee0 con 0x55edf0b44590 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.034+0000 7f02f2ffd640 1 -- 172.21.15.105:0/2001729394 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f409b290 con 0x55edf0b44590 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 -- 172.21.15.105:0/2001729394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 msgr2=0x55edf0b46a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b46a20 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f02f409bb40 tx=0x7f02f409ba40 comp rx=0 tx=0).stop 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 -- 172.21.15.105:0/2001729394 shutdown_connections 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b46a20 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b43f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 --2- 172.21.15.105:0/2001729394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b3ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 -- 172.21.15.105:0/2001729394 >> 172.21.15.105:0/2001729394 conn(0x55edf0b39fa0 msgr2=0x55edf0b3a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 -- 172.21.15.105:0/2001729394 shutdown_connections 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 -- 172.21.15.105:0/2001729394 wait complete. 2024-03-26T17:44:30.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 -- start start 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.035+0000 7f0311e19740 1 Processor -- start 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0311e19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b48e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0311e19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3f420 0x55edf0b493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0311e19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b44590 0x55edf0b50410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0311e19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55edf0b474e0 con 0x55edf0b3f420 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0311e19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55edf0a5f310 con 0x55edf0b3ea30 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0311e19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55edf0b47660 con 0x55edf0b44590 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0311e19740 10 client.0 _ll_register_callbacks cb 0x55edf0a9ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f030cfd0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b48e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0307fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3f420 0x55edf0b493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f030d7d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b44590 0x55edf0b50410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0307fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3f420 0x55edf0b493d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56254/0 (socket says 172.21.15.105:56254) 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f030cfd0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b48e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38982/0 (socket says 172.21.15.105:38982) 2024-03-26T17:44:30.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0307fff640 1 -- 172.21.15.105:0/406793467 learned_addr learned my addr 172.21.15.105:0/406793467 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0307fff640 1 -- 172.21.15.105:0/406793467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b44590 msgr2=0x55edf0b50410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0307fff640 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b44590 0x55edf0b50410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0307fff640 1 -- 172.21.15.105:0/406793467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 msgr2=0x55edf0b48e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0307fff640 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b48e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.036+0000 7f0307fff640 1 -- 172.21.15.105:0/406793467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf08817d0 con 0x55edf0b3f420 2024-03-26T17:44:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f0307fff640 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3f420 0x55edf0b493d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f02f40cd1b0 tx=0x7f02f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438202c68b01acb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f02e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f02e7fff640 1 -- 172.21.15.105:0/406793467 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4010630 con 0x55edf0b3f420 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f02e7fff640 1 -- 172.21.15.105:0/406793467 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f02f409b290 con 0x55edf0b3f420 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f0311e19740 1 -- 172.21.15.105:0/406793467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f409fee0 con 0x55edf0b3f420 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f02e7fff640 1 -- 172.21.15.105:0/406793467 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02f4083050 con 0x55edf0b3f420 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f0311e19740 10 client.17700 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f0311e19740 1 -- 172.21.15.105:0/406793467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edf0b53780 con 0x55edf0b3f420 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f02e7fff640 1 -- 172.21.15.105:0/406793467 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02f409fee0 con 0x55edf0b3f420 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.037+0000 7f02e7fff640 1 -- 172.21.15.105:0/406793467 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f02f4097b90 con 0x55edf0b3f420 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f0311e19740 10 client.17700 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:30.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f0311e19740 10 client.17700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f0311e19740 20 client.17700 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f0311e19740 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55edf0b54ad0 0x55edf0b56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f0311e19740 4 client.17700 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f0311e19740 1 -- 172.21.15.105:0/406793467 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edf0ab0eb0 con 0x55edf0b54ad0 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f030cfd0640 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55edf0b54ad0 0x55edf0b56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f02e7fff640 10 client.17700 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.038+0000 7f030cfd0640 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55edf0b54ad0 0x55edf0b56ec0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25d8bbf5d8d59f95 server_cookie=29b6791a6eb83f3f in_seq=0 out_seq=0 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f02e7fff640 1 -- 172.21.15.105:0/406793467 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55edf0ab0eb0 con 0x55edf0b54ad0 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f02e7fff640 10 client.17700 handle_command_reply: tid=0 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 client.17700 shutdown 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 -- 172.21.15.105:0/406793467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3f420 msgr2=0x55edf0b493d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3f420 0x55edf0b493d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f02f40cd1b0 tx=0x7f02f400cf90 comp rx=0 tx=0).stop 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 -- 172.21.15.105:0/406793467 shutdown_connections 2024-03-26T17:44:30.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55edf0b54ad0 0x55edf0b56ec0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b44590 0x55edf0b50410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3f420 0x55edf0b493d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f02e7fff640 0 client.17700 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:44:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 --2- 172.21.15.105:0/406793467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3ea30 0x55edf0b48e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 -- 172.21.15.105:0/406793467 >> 172.21.15.105:0/406793467 conn(0x55edf0b39fa0 msgr2=0x55edf0b391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 -- 172.21.15.105:0/406793467 shutdown_connections 2024-03-26T17:44:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 1 -- 172.21.15.105:0/406793467 wait complete. 2024-03-26T17:44:30.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.039+0000 7f0311e19740 20 client.17700 trim_cache size 0 max 16384 2024-03-26T17:44:30.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.055+0000 7f0311e19740 1 Processor -- start 2024-03-26T17:44:30.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0311e19740 1 -- start start 2024-03-26T17:44:30.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0311e19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3ea30 0x55edf0b38930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0311e19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0311e19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b44590 0x55edf0b38e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0311e19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03081a8ef0 con 0x55edf0b3ea30 2024-03-26T17:44:30.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0311e19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03081aaa30 con 0x55edf0b44590 2024-03-26T17:44:30.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0311e19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03081a95c0 con 0x55edf0b3f420 2024-03-26T17:44:30.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3ea30 0x55edf0b38930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3ea30 0x55edf0b38930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56258/0 (socket says 172.21.15.105:56258) 2024-03-26T17:44:30.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 -- 172.21.15.105:0/4126052700 learned_addr learned my addr 172.21.15.105:0/4126052700 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030d7d1640 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b44590 0x55edf0b38e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0307fff640 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 -- 172.21.15.105:0/4126052700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 msgr2=0x55edf0b503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 -- 172.21.15.105:0/4126052700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b44590 msgr2=0x55edf0b38e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b44590 0x55edf0b38e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 -- 172.21.15.105:0/4126052700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf0b398f0 con 0x55edf0b3ea30 2024-03-26T17:44:30.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030d7d1640 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b44590 0x55edf0b38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:30.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f0307fff640 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:30.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.056+0000 7f030cfd0640 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3ea30 0x55edf0b38930 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f02fc005ab0 tx=0x7f02fc005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c773318499a67ae0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f02f27fc640 1 -- 172.21.15.105:0/4126052700 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc02cc00 con 0x55edf0b3ea30 2024-03-26T17:44:30.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f02f27fc640 1 -- 172.21.15.105:0/4126052700 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02fc04b410 con 0x55edf0b3ea30 2024-03-26T17:44:30.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f02f27fc640 1 -- 172.21.15.105:0/4126052700 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0044d0 con 0x55edf0b3ea30 2024-03-26T17:44:30.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 -- 172.21.15.105:0/4126052700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3ea30 msgr2=0x55edf0b38930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3ea30 0x55edf0b38930 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f02fc005ab0 tx=0x7f02fc005e70 comp rx=0 tx=0).stop 2024-03-26T17:44:30.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 -- 172.21.15.105:0/4126052700 shutdown_connections 2024-03-26T17:44:30.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b44590 0x55edf0b38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3f420 0x55edf0b503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 --2- 172.21.15.105:0/4126052700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b3ea30 0x55edf0b38930 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 -- 172.21.15.105:0/4126052700 >> 172.21.15.105:0/4126052700 conn(0x55edf0b39fa0 msgr2=0x55edf0b40920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 -- 172.21.15.105:0/4126052700 shutdown_connections 2024-03-26T17:44:30.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.057+0000 7f0311e19740 1 -- 172.21.15.105:0/4126052700 wait complete. 2024-03-26T17:44:30.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 1 -- start start 2024-03-26T17:44:30.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 1 Processor -- start 2024-03-26T17:44:30.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3ea30 0x55edf0b4b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3f420 0x55edf0b4bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b4eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03081aa360 con 0x55edf0b44590 2024-03-26T17:44:30.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03081a9c90 con 0x55edf0b3f420 2024-03-26T17:44:30.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03081a6ed0 con 0x55edf0b3ea30 2024-03-26T17:44:30.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0311e19740 10 client.0 _ll_register_callbacks cb 0x7f0300011890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:30.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0307fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3f420 0x55edf0b4bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0307fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3f420 0x55edf0b4bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39004/0 (socket says 172.21.15.105:39004) 2024-03-26T17:44:30.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f030d7d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b4eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f0307fff640 1 -- 172.21.15.105:0/951564923 learned_addr learned my addr 172.21.15.105:0/951564923 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.058+0000 7f030cfd0640 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3ea30 0x55edf0b4b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0307fff640 1 -- 172.21.15.105:0/951564923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3ea30 msgr2=0x55edf0b4b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0307fff640 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3ea30 0x55edf0b4b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0307fff640 1 -- 172.21.15.105:0/951564923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 msgr2=0x55edf0b4eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0307fff640 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b4eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0307fff640 1 -- 172.21.15.105:0/951564923 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf0b39dd0 con 0x55edf0b3f420 2024-03-26T17:44:30.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0307fff640 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3f420 0x55edf0b4bed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c6a0 tx=0x7f02f409bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b6a59347881927 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f02e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f02e7fff640 1 -- 172.21.15.105:0/951564923 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4018e60 con 0x55edf0b3f420 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0311e19740 1 -- 172.21.15.105:0/951564923 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02fc04b410 con 0x55edf0b3f420 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f02e7fff640 1 -- 172.21.15.105:0/951564923 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f02f4010630 con 0x55edf0b3f420 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f02e7fff640 1 -- 172.21.15.105:0/951564923 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f02f409fee0 con 0x55edf0b3f420 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0311e19740 10 client.27533 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f0311e19740 1 -- 172.21.15.105:0/951564923 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edf0b50610 con 0x55edf0b3f420 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.059+0000 7f02e7fff640 1 -- 172.21.15.105:0/951564923 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02fc04b410 con 0x55edf0b3f420 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f02e7fff640 1 -- 172.21.15.105:0/951564923 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f02f409ad00 con 0x55edf0b3f420 2024-03-26T17:44:30.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f0311e19740 10 client.27533 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f0311e19740 10 client.27533 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f0311e19740 20 client.27533 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f0311e19740 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55edf0bbf560 0x55edf0bc1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f0311e19740 4 client.27533 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f0311e19740 1 -- 172.21.15.105:0/951564923 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edf0aab200 con 0x55edf0bbf560 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f030cfd0640 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55edf0bbf560 0x55edf0bc1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f02e7fff640 10 client.27533 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f030cfd0640 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55edf0bbf560 0x55edf0bc1980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f96ec4d748c2fec9 server_cookie=77385855cb7f4d65 in_seq=0 out_seq=0 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f02e7fff640 1 -- 172.21.15.105:0/951564923 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edf0aab200 con 0x55edf0bbf560 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f02e7fff640 10 client.27533 handle_command_reply: tid=0 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.060+0000 7f0311e19740 1 client.27533 shutdown 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 -- 172.21.15.105:0/951564923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3f420 msgr2=0x55edf0b4bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3f420 0x55edf0b4bed0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c6a0 tx=0x7f02f409bab0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 -- 172.21.15.105:0/951564923 shutdown_connections 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55edf0bbf560 0x55edf0bc1980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55edf0b44590 0x55edf0b4eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55edf0b3f420 0x55edf0b4bed0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f02e7fff640 0 client.27533 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 --2- 172.21.15.105:0/951564923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edf0b3ea30 0x55edf0b4b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 -- 172.21.15.105:0/951564923 >> 172.21.15.105:0/951564923 conn(0x55edf0b39fa0 msgr2=0x55edf0b38930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 -- 172.21.15.105:0/951564923 shutdown_connections 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 1 -- 172.21.15.105:0/951564923 wait complete. 2024-03-26T17:44:30.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f0311e19740 20 client.27533 trim_cache size 0 max 16384 2024-03-26T17:44:30.073 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:44:30.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f030f234640 1 -- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02f80a2fa0 msgr2=0x7f02f80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f030f234640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02f80a2fa0 0x7f02f80a5460 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f03000099e0 tx=0x7f0300009c80 comp rx=0 tx=0).stop 2024-03-26T17:44:30.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f030f234640 1 -- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081019b0 msgr2=0x7f030819c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.061+0000 7f030f234640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081019b0 0x7f030819c880 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f02f400e9f0 tx=0x7f02f400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.062+0000 7f030f234640 1 -- 172.21.15.105:0/1250413187 shutdown_connections 2024-03-26T17:44:30.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.062+0000 7f030f234640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f02f80a2fa0 0x7f02f80a5460 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.062+0000 7f030f234640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0308102370 0x7f030819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.062+0000 7f030f234640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03081019b0 0x7f030819c880 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.062+0000 7f030f234640 1 --2- 172.21.15.105:0/1250413187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03081007c0 0x7f0308069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.062+0000 7f030f234640 1 -- 172.21.15.105:0/1250413187 >> 172.21.15.105:0/1250413187 conn(0x7f03080fbd50 msgr2=0x7f0308104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.062+0000 7f030f234640 1 -- 172.21.15.105:0/1250413187 shutdown_connections 2024-03-26T17:44:30.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.062+0000 7f030f234640 1 -- 172.21.15.105:0/1250413187 wait complete. 2024-03-26T17:44:30.077 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:29 smithi105 ceph-mon[32280]: pgmap v532: 97 pgs: 97 active+clean; 27 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:44:30.077 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:29 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4241148462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:44:30.078 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:44:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:29 smithi144 ceph-mon[36095]: pgmap v532: 97 pgs: 97 active+clean; 27 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:44:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:29 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4241148462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:44:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:29 smithi135 ceph-mon[35636]: pgmap v532: 97 pgs: 97 active+clean; 27 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:44:30.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:29 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4241148462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:44:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 -- 172.21.15.105:0/2745095188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4660102ac0 msgr2=0x7f4660102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 --2- 172.21.15.105:0/2745095188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4660102ac0 0x7f4660102f40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f464c009900 tx=0x7f464c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:44:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 -- 172.21.15.105:0/2745095188 shutdown_connections 2024-03-26T17:44:30.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 --2- 172.21.15.105:0/2745095188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4660103480 0x7f4660109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 --2- 172.21.15.105:0/2745095188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4660102ac0 0x7f4660102f40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 --2- 172.21.15.105:0/2745095188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46601018d0 0x7f4660101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 -- 172.21.15.105:0/2745095188 >> 172.21.15.105:0/2745095188 conn(0x7f46600fce20 msgr2=0x7f46600ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 -- 172.21.15.105:0/2745095188 shutdown_connections 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.198+0000 7f4667019640 1 -- 172.21.15.105:0/2745095188 wait complete. 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4667019640 1 Processor -- start 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4667019640 1 -- start start 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4667019640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46601018d0 0x7f4660075620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4667019640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4660102ac0 0x7f4660073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4667019640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4660103480 0x7f46600741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4667019640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f466010d400 con 0x7f46601018d0 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4667019640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f466010d280 con 0x7f4660103480 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4667019640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f466010d580 con 0x7f4660102ac0 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4664db5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46601018d0 0x7f4660075620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4664db5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46601018d0 0x7f4660075620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56290/0 (socket says 172.21.15.105:56290) 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4664db5640 1 -- 172.21.15.105:0/3641140247 learned_addr learned my addr 172.21.15.105:0/3641140247 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f465ffff640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4660102ac0 0x7f4660073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f46655b6640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4660103480 0x7f46600741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4664db5640 1 -- 172.21.15.105:0/3641140247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4660102ac0 msgr2=0x7f4660073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.199+0000 7f4664db5640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4660102ac0 0x7f4660073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f4664db5640 1 -- 172.21.15.105:0/3641140247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4660103480 msgr2=0x7f46600741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f4664db5640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4660103480 0x7f46600741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f4664db5640 1 -- 172.21.15.105:0/3641140247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4660074ab0 con 0x7f46601018d0 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f46655b6640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4660103480 0x7f46600741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:44:30.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f4664db5640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46601018d0 0x7f4660075620 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f465400e9c0 tx=0x7f465400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f465dffb640 1 -- 172.21.15.105:0/3641140247 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465400cd80 con 0x7f46601018d0 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f465dffb640 1 -- 172.21.15.105:0/3641140247 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46540035f0 con 0x7f46601018d0 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f4667019640 1 -- 172.21.15.105:0/3641140247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46601a1cb0 con 0x7f46601018d0 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f465dffb640 1 -- 172.21.15.105:0/3641140247 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654010690 con 0x7f46601018d0 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.200+0000 7f4667019640 1 -- 172.21.15.105:0/3641140247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46601a21d0 con 0x7f46601018d0 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.203+0000 7f465dffb640 1 -- 172.21.15.105:0/3641140247 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f46540037a0 con 0x7f46601018d0 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.203+0000 7f4669c06740 1 Processor -- start 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.203+0000 7f4669c06740 1 -- start start 2024-03-26T17:44:30.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465dffb640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f46500a2fa0 0x7f46500a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4669c06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d358ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4669c06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358f420 0x5555d3593f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4669c06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d3596a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4669c06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555d34f3970 con 0x5555d358f420 2024-03-26T17:44:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4669c06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5555d349bd70 con 0x5555d358ea30 2024-03-26T17:44:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4669c06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5555d34ef920 con 0x5555d3594590 2024-03-26T17:44:30.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4664db5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d358ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358f420 0x5555d3593f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f46655b6640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f46500a2fa0 0x7f46500a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4664db5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d358ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39026/0 (socket says 172.21.15.105:39026) 2024-03-26T17:44:30.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358f420 0x5555d3593f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56298/0 (socket says 172.21.15.105:56298) 2024-03-26T17:44:30.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4664db5640 1 -- 172.21.15.105:0/1648629687 learned_addr learned my addr 172.21.15.105:0/1648629687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465dffb640 1 -- 172.21.15.105:0/3641140247 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f46540cf9e0 con 0x7f46601018d0 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f46655b6640 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d3596a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465ffff640 1 -- 172.21.15.105:0/1648629687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 msgr2=0x5555d3596a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465ffff640 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d3596a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465ffff640 1 -- 172.21.15.105:0/1648629687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 msgr2=0x5555d358ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465ffff640 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d358ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f465ffff640 1 -- 172.21.15.105:0/1648629687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d32073f0 con 0x5555d358f420 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f4664db5640 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d358ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.204+0000 7f46655b6640 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d3596a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:44:30.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.205+0000 7f465ffff640 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358f420 0x5555d3593f20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f464c000c00 tx=0x7f464c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e51dd93a80e3021 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.205+0000 7f46655b6640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f46500a2fa0 0x7f46500a5460 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4658009fd0 tx=0x7f4658009b30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.205+0000 7f464affd640 1 -- 172.21.15.105:0/1648629687 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c045020 con 0x5555d358f420 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.205+0000 7f464affd640 1 -- 172.21.15.105:0/1648629687 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f464c004040 con 0x5555d358f420 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.205+0000 7f464affd640 1 -- 172.21.15.105:0/1648629687 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c0055a0 con 0x5555d358f420 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.206+0000 7f4669c06740 1 -- 172.21.15.105:0/1648629687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358f420 msgr2=0x5555d3593f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.206+0000 7f4669c06740 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358f420 0x5555d3593f20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f464c000c00 tx=0x7f464c009560 comp rx=0 tx=0).stop 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.206+0000 7f4669c06740 1 -- 172.21.15.105:0/1648629687 shutdown_connections 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.206+0000 7f4669c06740 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d3596a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.206+0000 7f4669c06740 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358f420 0x5555d3593f20 secure :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f464c000c00 tx=0x7f464c009560 comp rx=0 tx=0).stop 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.206+0000 7f4669c06740 1 --2- 172.21.15.105:0/1648629687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d358ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.206+0000 7f4669c06740 1 -- 172.21.15.105:0/1648629687 >> 172.21.15.105:0/1648629687 conn(0x5555d3589fa0 msgr2=0x5555d358a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.206+0000 7f4669c06740 1 -- 172.21.15.105:0/1648629687 shutdown_connections 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.207+0000 7f4669c06740 1 -- 172.21.15.105:0/1648629687 wait complete. 2024-03-26T17:44:30.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.207+0000 7f4669c06740 1 -- start start 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.207+0000 7f4669c06740 1 Processor -- start 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4669c06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d3599070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4669c06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d3594590 0x5555d35995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4669c06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d359e5e0 0x5555d35a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4669c06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5555d35974e0 con 0x5555d3594590 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4669c06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5555d34af310 con 0x5555d358ea30 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4669c06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5555d3597660 con 0x5555d359e5e0 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4669c06740 10 client.0 _ll_register_callbacks cb 0x5555d34eac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f465ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d3594590 0x5555d35995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4664db5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d3599070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f46655b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d359e5e0 0x5555d35a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f465ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d3594590 0x5555d35995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56300/0 (socket says 172.21.15.105:56300) 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f465ffff640 1 -- 172.21.15.105:0/1140328555 learned_addr learned my addr 172.21.15.105:0/1140328555 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f4664db5640 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d3599070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39028/0 (socket says 172.21.15.105:39028) 2024-03-26T17:44:30.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f465ffff640 1 -- 172.21.15.105:0/1140328555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d359e5e0 msgr2=0x5555d35a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.208+0000 7f465ffff640 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d359e5e0 0x5555d35a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f465ffff640 1 -- 172.21.15.105:0/1140328555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 msgr2=0x5555d3599070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f465ffff640 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d3599070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f465ffff640 1 -- 172.21.15.105:0/1140328555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d32d17d0 con 0x5555d3594590 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f465ffff640 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d3594590 0x5555d35995b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f464c02c840 tx=0x7f464c005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9139e7841c37d8aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f463ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f463ffff640 1 -- 172.21.15.105:0/1140328555 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c04e070 con 0x5555d3594590 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f463ffff640 1 -- 172.21.15.105:0/1140328555 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f464c0035f0 con 0x5555d3594590 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f4669c06740 1 -- 172.21.15.105:0/1140328555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f464c004040 con 0x5555d3594590 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.209+0000 7f463ffff640 1 -- 172.21.15.105:0/1140328555 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f464c045070 con 0x5555d3594590 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4669c06740 10 client.17718 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4669c06740 1 -- 172.21.15.105:0/1140328555 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555d35a3cb0 con 0x5555d3594590 2024-03-26T17:44:30.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f463ffff640 1 -- 172.21.15.105:0/1140328555 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f464c003ca0 con 0x5555d3594590 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f463ffff640 1 -- 172.21.15.105:0/1140328555 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f464c051030 con 0x5555d3594590 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4669c06740 10 client.17718 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4669c06740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4669c06740 20 client.17718 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4669c06740 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5555d35a5000 0x5555d35a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4669c06740 4 client.17718 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4669c06740 1 -- 172.21.15.105:0/1140328555 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f464c004040 con 0x5555d35a5000 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.210+0000 7f4664db5640 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5555d35a5000 0x5555d35a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.211+0000 7f463ffff640 10 client.17718 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:30.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.211+0000 7f4664db5640 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5555d35a5000 0x5555d35a73f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c28dc52dc8e49fb4 server_cookie=2a300bc1c7e81901 in_seq=0 out_seq=0 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f463ffff640 1 -- 172.21.15.105:0/1140328555 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f464c004040 con 0x5555d35a5000 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f463ffff640 10 client.17718 handle_command_reply: tid=0 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 client.17718 shutdown 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 -- 172.21.15.105:0/1140328555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d3594590 msgr2=0x5555d35995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d3594590 0x5555d35995b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f464c02c840 tx=0x7f464c005690 comp rx=0 tx=0).stop 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 -- 172.21.15.105:0/1140328555 shutdown_connections 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5555d35a5000 0x5555d35a73f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d359e5e0 0x5555d35a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d3594590 0x5555d35995b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f463ffff640 0 client.17718 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 --2- 172.21.15.105:0/1140328555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d358ea30 0x5555d3599070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.212+0000 7f4669c06740 1 -- 172.21.15.105:0/1140328555 >> 172.21.15.105:0/1140328555 conn(0x5555d3589fa0 msgr2=0x5555d35891c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.213+0000 7f4669c06740 1 -- 172.21.15.105:0/1140328555 shutdown_connections 2024-03-26T17:44:30.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.213+0000 7f4669c06740 1 -- 172.21.15.105:0/1140328555 wait complete. 2024-03-26T17:44:30.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.213+0000 7f4669c06740 20 client.17718 trim_cache size 0 max 16384 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.227+0000 7f4669c06740 1 Processor -- start 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.227+0000 7f4669c06740 1 -- start start 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.227+0000 7f4669c06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d358ea30 0x5555d3598690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.227+0000 7f4669c06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d3594590 0x5555d3590b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.227+0000 7f4669c06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d359e5e0 0x5555d3591040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f4669c06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f46601aaea0 con 0x5555d359e5e0 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f4669c06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46601ac9e0 con 0x5555d3594590 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f4669c06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46601ab570 con 0x5555d358ea30 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d359e5e0 0x5555d3591040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d359e5e0 0x5555d3591040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56312/0 (socket says 172.21.15.105:56312) 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 -- 172.21.15.105:0/1926965275 learned_addr learned my addr 172.21.15.105:0/1926965275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f4664db5640 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d358ea30 0x5555d3598690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f465ffff640 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d3594590 0x5555d3590b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 -- 172.21.15.105:0/1926965275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d358ea30 msgr2=0x5555d3598690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d358ea30 0x5555d3598690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 -- 172.21.15.105:0/1926965275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d3594590 msgr2=0x5555d3590b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d3594590 0x5555d3590b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 -- 172.21.15.105:0/1926965275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d35a7760 con 0x5555d359e5e0 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f46655b6640 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d359e5e0 0x5555d3591040 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4658009ab0 tx=0x7f4658014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9270a2f377f416a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f464a7fc640 1 -- 172.21.15.105:0/1926965275 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658018070 con 0x5555d359e5e0 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f464a7fc640 1 -- 172.21.15.105:0/1926965275 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f465800ebb0 con 0x5555d359e5e0 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.228+0000 7f464a7fc640 1 -- 172.21.15.105:0/1926965275 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465801e690 con 0x5555d359e5e0 2024-03-26T17:44:30.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 -- 172.21.15.105:0/1926965275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d359e5e0 msgr2=0x5555d3591040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d359e5e0 0x5555d3591040 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4658009ab0 tx=0x7f4658014070 comp rx=0 tx=0).stop 2024-03-26T17:44:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 -- 172.21.15.105:0/1926965275 shutdown_connections 2024-03-26T17:44:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d359e5e0 0x5555d3591040 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d3594590 0x5555d3590b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 --2- 172.21.15.105:0/1926965275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d358ea30 0x5555d3598690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 -- 172.21.15.105:0/1926965275 >> 172.21.15.105:0/1926965275 conn(0x5555d3589fa0 msgr2=0x5555d358a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 -- 172.21.15.105:0/1926965275 shutdown_connections 2024-03-26T17:44:30.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 -- 172.21.15.105:0/1926965275 wait complete. 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.229+0000 7f4669c06740 1 -- start start 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4669c06740 1 Processor -- start 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4669c06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358ea30 0x5555d35b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4669c06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d35b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4669c06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d359e5e0 0x5555d35b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4669c06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f46601ac310 con 0x5555d358ea30 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4669c06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f46601abc40 con 0x5555d359e5e0 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4669c06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46601a8e80 con 0x5555d3594590 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4669c06740 10 client.0 _ll_register_callbacks cb 0x7f4654003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4664db5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358ea30 0x5555d35b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f465ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d35b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f465ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d35b66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35040/0 (socket says 172.21.15.105:35040) 2024-03-26T17:44:30.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f465ffff640 1 -- 172.21.15.105:0/302657704 learned_addr learned my addr 172.21.15.105:0/302657704 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f46655b6640 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d359e5e0 0x5555d35b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f465ffff640 1 -- 172.21.15.105:0/302657704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d359e5e0 msgr2=0x5555d35b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f465ffff640 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d359e5e0 0x5555d35b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f465ffff640 1 -- 172.21.15.105:0/302657704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358ea30 msgr2=0x5555d35b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f465ffff640 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358ea30 0x5555d35b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f465ffff640 1 -- 172.21.15.105:0/302657704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d35b7cd0 con 0x5555d3594590 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.230+0000 7f4664db5640 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358ea30 0x5555d35b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f465ffff640 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d35b66d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f464c004d40 tx=0x7f464c005160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90eeca0e966e33e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f463ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f463ffff640 1 -- 172.21.15.105:0/302657704 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c04e030 con 0x5555d3594590 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f463ffff640 1 -- 172.21.15.105:0/302657704 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f464c041480 con 0x5555d3594590 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 1 -- 172.21.15.105:0/302657704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465800ebb0 con 0x5555d3594590 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f463ffff640 1 -- 172.21.15.105:0/302657704 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f464c004ad0 con 0x5555d3594590 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 10 client.27556 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:30.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 1 -- 172.21.15.105:0/302657704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555d360b300 con 0x5555d3594590 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f463ffff640 1 -- 172.21.15.105:0/302657704 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f465800ebb0 con 0x5555d3594590 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f463ffff640 1 -- 172.21.15.105:0/302657704 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f464c051070 con 0x5555d3594590 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 10 client.27556 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 10 client.27556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 20 client.27556 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5555d360bcd0 0x5555d360e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 4 client.27556 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.231+0000 7f4669c06740 1 -- 172.21.15.105:0/302657704 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555d3500eb0 con 0x5555d360bcd0 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4664db5640 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5555d360bcd0 0x5555d360e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f463ffff640 10 client.27556 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4664db5640 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5555d360bcd0 0x5555d360e0f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68c31e68214fac server_cookie=828ab855386f1904 in_seq=0 out_seq=0 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f463ffff640 1 -- 172.21.15.105:0/302657704 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555d3500eb0 con 0x5555d360bcd0 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f463ffff640 10 client.27556 handle_command_reply: tid=0 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 client.27556 shutdown 2024-03-26T17:44:30.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 -- 172.21.15.105:0/302657704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 msgr2=0x5555d35b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d35b66d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f464c004d40 tx=0x7f464c005160 comp rx=0 tx=0).stop 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 -- 172.21.15.105:0/302657704 shutdown_connections 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5555d360bcd0 0x5555d360e0f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555d359e5e0 0x5555d35b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5555d3594590 0x5555d35b66d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f463ffff640 0 client.27556 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 --2- 172.21.15.105:0/302657704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5555d358ea30 0x5555d35b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.232+0000 7f4669c06740 1 -- 172.21.15.105:0/302657704 >> 172.21.15.105:0/302657704 conn(0x5555d3589fa0 msgr2=0x5555d3590b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4669c06740 1 -- 172.21.15.105:0/302657704 shutdown_connections 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4669c06740 1 -- 172.21.15.105:0/302657704 wait complete. 2024-03-26T17:44:30.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4669c06740 20 client.27556 trim_cache size 0 max 16384 2024-03-26T17:44:30.234 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:44:30.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 -- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f46500a2fa0 msgr2=0x7f46500a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f46500a2fa0 0x7f46500a5460 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4658009fd0 tx=0x7f4658009b30 comp rx=0 tx=0).stop 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 -- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46601018d0 msgr2=0x7f4660075620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46601018d0 0x7f4660075620 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f465400e9c0 tx=0x7f465400ee80 comp rx=0 tx=0).stop 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 -- 172.21.15.105:0/3641140247 shutdown_connections 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f46500a2fa0 0x7f46500a5460 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4660103480 0x7f46600741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4660102ac0 0x7f4660073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 --2- 172.21.15.105:0/3641140247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46601018d0 0x7f4660075620 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.233+0000 7f4667019640 1 -- 172.21.15.105:0/3641140247 >> 172.21.15.105:0/3641140247 conn(0x7f46600fce20 msgr2=0x7f4660105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.234+0000 7f4667019640 1 -- 172.21.15.105:0/3641140247 shutdown_connections 2024-03-26T17:44:30.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:30.234+0000 7f4667019640 1 -- 172.21.15.105:0/3641140247 wait complete. 2024-03-26T17:44:31.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 184926 3.15 MB/sec execute 194 sec latency 16.003 ms 2024-03-26T17:44:31.239 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:44:31.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:44:31.239 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:44:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 -- 172.21.15.105:0/2656508918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 msgr2=0x7fe878108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/2656508918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 0x7fe878108c20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe870009ca0 tx=0x7fe87002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 -- 172.21.15.105:0/2656508918 shutdown_connections 2024-03-26T17:44:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/2656508918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 0x7fe878108c20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/2656508918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8781019d0 0x7fe878101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/2656508918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8781007e0 0x7fe878100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 -- 172.21.15.105:0/2656508918 >> 172.21.15.105:0/2656508918 conn(0x7fe8780fbd70 msgr2=0x7fe8780fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 -- 172.21.15.105:0/2656508918 shutdown_connections 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.362+0000 7fe87d6dc640 1 -- 172.21.15.105:0/2656508918 wait complete. 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe87d6dc640 1 Processor -- start 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe87d6dc640 1 -- start start 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe87d6dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8781007e0 0x7fe878104770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe87d6dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8781019d0 0x7fe878102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe87d6dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 0x7fe8781032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe87d6dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe87810c2f0 con 0x7fe878102390 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe87d6dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe87810c170 con 0x7fe8781007e0 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe87d6dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe87810c470 con 0x7fe8781019d0 2024-03-26T17:44:31.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 0x7fe8781032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe876ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8781007e0 0x7fe878104770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 0x7fe8781032f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56334/0 (socket says 172.21.15.105:56334) 2024-03-26T17:44:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe8777fe640 1 -- 172.21.15.105:0/1358397530 learned_addr learned my addr 172.21.15.105:0/1358397530 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.363+0000 7fe8767fc640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8781019d0 0x7fe878102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe8777fe640 1 -- 172.21.15.105:0/1358397530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8781019d0 msgr2=0x7fe878102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe8777fe640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8781019d0 0x7fe878102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe8777fe640 1 -- 172.21.15.105:0/1358397530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8781007e0 msgr2=0x7fe878104770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe8777fe640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8781007e0 0x7fe878104770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe8777fe640 1 -- 172.21.15.105:0/1358397530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe878103b80 con 0x7fe878102390 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe8767fc640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8781019d0 0x7fe878102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe8777fe640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 0x7fe8781032f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe870009dc0 tx=0x7fe8700328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe85ffff640 1 -- 172.21.15.105:0/1358397530 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870041020 con 0x7fe878102390 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe85ffff640 1 -- 172.21.15.105:0/1358397530 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe870003e20 con 0x7fe878102390 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe87d6dc640 1 -- 172.21.15.105:0/1358397530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8781a1be0 con 0x7fe878102390 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe85ffff640 1 -- 172.21.15.105:0/1358397530 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870039040 con 0x7fe878102390 2024-03-26T17:44:31.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.364+0000 7fe87d6dc640 1 -- 172.21.15.105:0/1358397530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781a2130 con 0x7fe878102390 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.366+0000 7fe85ffff640 1 -- 172.21.15.105:0/1358397530 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe870044020 con 0x7fe878102390 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8803a6740 1 Processor -- start 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8803a6740 1 -- start start 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7be5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8803a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bb1a723630 con 0x55bb1a7beb80 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8803a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bb1a6cbe10 con 0x55bb1a7c3cf0 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8803a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bb1a71f5e0 con 0x55bb1a7be190 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe85ffff640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8600a2f20 0x7fe8600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8767fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7be5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8767fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7be5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35064/0 (socket says 172.21.15.105:35064) 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe876ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8767fc640 1 -- 172.21.15.105:0/2154739731 learned_addr learned my addr 172.21.15.105:0/2154739731 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8767fc640 1 --2- 172.21.15.105:0/2154739731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe8777fe640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8600a2f20 0x7fe8600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.367+0000 7fe85ffff640 1 -- 172.21.15.105:0/1358397530 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe8700f15c0 con 0x7fe878102390 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe8767fc640 1 -- 172.21.15.105:0/2154739731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 msgr2=0x55bb1a7c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe8767fc640 1 --2- 172.21.15.105:0/2154739731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe8767fc640 1 -- 172.21.15.105:0/2154739731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 msgr2=0x55bb1a7c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe8767fc640 1 --2- 172.21.15.105:0/2154739731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe8767fc640 1 -- 172.21.15.105:0/2154739731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1a6e2740 con 0x55bb1a7be190 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe8767fc640 1 --2- 172.21.15.105:0/2154739731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7be5b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe86c011be0 tx=0x7fe86c009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64b96595fcc0a8d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe8777fe640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8600a2f20 0x7fe8600a53e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe87003bfd0 tx=0x7fe870088800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe85d7fa640 1 -- 172.21.15.105:0/2154739731 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00fc50 con 0x55bb1a7be190 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe85d7fa640 1 -- 172.21.15.105:0/2154739731 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe86c00fdf0 con 0x55bb1a7be190 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.368+0000 7fe85d7fa640 1 -- 172.21.15.105:0/2154739731 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00e610 con 0x55bb1a7be190 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- 172.21.15.105:0/2154739731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 msgr2=0x55bb1a7be5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 --2- 172.21.15.105:0/2154739731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7be5b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe86c011be0 tx=0x7fe86c009ba0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- 172.21.15.105:0/2154739731 shutdown_connections 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 --2- 172.21.15.105:0/2154739731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 --2- 172.21.15.105:0/2154739731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 --2- 172.21.15.105:0/2154739731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7be5b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- 172.21.15.105:0/2154739731 >> 172.21.15.105:0/2154739731 conn(0x55bb1a7b9700 msgr2=0x55bb1a7b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- 172.21.15.105:0/2154739731 shutdown_connections 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- 172.21.15.105:0/2154739731 wait complete. 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- start start 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 Processor -- start 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bb1a7c6ca0 con 0x55bb1a7be190 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bb1a6df1a0 con 0x55bb1a7c3cf0 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bb1a7c6e20 con 0x55bb1a7beb80 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8803a6740 10 client.0 _ll_register_callbacks cb 0x55bb1a436c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56352/0 (socket says 172.21.15.105:56352) 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8767fc640 1 -- 172.21.15.105:0/857827325 learned_addr learned my addr 172.21.15.105:0/857827325 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe876ffd640 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.369+0000 7fe8767fc640 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8767fc640 1 -- 172.21.15.105:0/857827325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 msgr2=0x55bb1a7c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8767fc640 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8767fc640 1 -- 172.21.15.105:0/857827325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 msgr2=0x55bb1a7cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8767fc640 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8767fc640 1 -- 172.21.15.105:0/857827325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1a4ff7e0 con 0x55bb1a7be190 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe876ffd640 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8767fc640 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7c8710 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe86c005760 tx=0x7fe86c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f924c797da869c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe85aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe85aff5640 1 -- 172.21.15.105:0/857827325 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c004020 con 0x55bb1a7be190 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe85aff5640 1 -- 172.21.15.105:0/857827325 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe86c019040 con 0x55bb1a7be190 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8803a6740 1 -- 172.21.15.105:0/857827325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe86c00fdf0 con 0x55bb1a7be190 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe85aff5640 1 -- 172.21.15.105:0/857827325 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe86c035050 con 0x55bb1a7be190 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8803a6740 10 client.17736 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.370+0000 7fe8803a6740 1 -- 172.21.15.105:0/857827325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb1a55ff70 con 0x55bb1a7be190 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe85aff5640 1 -- 172.21.15.105:0/857827325 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe86c00fdf0 con 0x55bb1a7be190 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe85aff5640 1 -- 172.21.15.105:0/857827325 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe86c01d020 con 0x55bb1a7be190 2024-03-26T17:44:31.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe8803a6740 10 client.17736 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe8803a6740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe8803a6740 20 client.17736 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe8803a6740 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb1a7d4260 0x55bb1a7d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe8803a6740 4 client.17736 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe8803a6740 1 -- 172.21.15.105:0/857827325 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb1a730b70 con 0x55bb1a7d4260 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe876ffd640 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb1a7d4260 0x55bb1a7d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe85aff5640 10 client.17736 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.371+0000 7fe876ffd640 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb1a7d4260 0x55bb1a7d6670 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e57c98f318512c2 server_cookie=355895e1404e71b4 in_seq=0 out_seq=0 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe85aff5640 1 -- 172.21.15.105:0/857827325 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bb1a730b70 con 0x55bb1a7d4260 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe85aff5640 10 client.17736 handle_command_reply: tid=0 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 client.17736 shutdown 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 -- 172.21.15.105:0/857827325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 msgr2=0x55bb1a7c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7c8710 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe86c005760 tx=0x7fe86c0096c0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 -- 172.21.15.105:0/857827325 shutdown_connections 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb1a7d4260 0x55bb1a7d6670 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe85aff5640 0 client.17736 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 --2- 172.21.15.105:0/857827325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7c8710 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 -- 172.21.15.105:0/857827325 >> 172.21.15.105:0/857827325 conn(0x55bb1a7b9700 msgr2=0x55bb1a7b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 -- 172.21.15.105:0/857827325 shutdown_connections 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 1 -- 172.21.15.105:0/857827325 wait complete. 2024-03-26T17:44:31.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.373+0000 7fe8803a6740 20 client.17736 trim_cache size 0 max 16384 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8803a6740 1 Processor -- start 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8803a6740 1 -- start start 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8803a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781aaff0 con 0x55bb1a7beb80 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8803a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781acb30 con 0x55bb1a7c3cf0 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8803a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781ab6c0 con 0x55bb1a7be190 2024-03-26T17:44:31.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe876ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe876ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56356/0 (socket says 172.21.15.105:56356) 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe876ffd640 1 -- 172.21.15.105:0/1537284698 learned_addr learned my addr 172.21.15.105:0/1537284698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8767fc640 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe8767fc640 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe876ffd640 1 -- 172.21.15.105:0/1537284698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 msgr2=0x55bb1a7c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe876ffd640 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.394+0000 7fe876ffd640 1 -- 172.21.15.105:0/1537284698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 msgr2=0x55bb1a7dafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.395+0000 7fe876ffd640 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7dafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.395+0000 7fe876ffd640 1 -- 172.21.15.105:0/1537284698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb1a3f7170 con 0x55bb1a7beb80 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.395+0000 7fe876ffd640 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cfc50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x55bb1a7bef80 tx=0x7fe868005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39953b4602a5517 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.395+0000 7fe85cff9640 1 -- 172.21.15.105:0/1537284698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868023020 con 0x55bb1a7beb80 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.395+0000 7fe85cff9640 1 -- 172.21.15.105:0/1537284698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe86801f700 con 0x55bb1a7beb80 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.395+0000 7fe85cff9640 1 -- 172.21.15.105:0/1537284698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868010d20 con 0x55bb1a7beb80 2024-03-26T17:44:31.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 -- 172.21.15.105:0/1537284698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 msgr2=0x55bb1a7cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cfc50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x55bb1a7bef80 tx=0x7fe868005f90 comp rx=0 tx=0).stop 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 -- 172.21.15.105:0/1537284698 shutdown_connections 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cfc50 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 --2- 172.21.15.105:0/1537284698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7be190 0x55bb1a7c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 -- 172.21.15.105:0/1537284698 >> 172.21.15.105:0/1537284698 conn(0x55bb1a7b9700 msgr2=0x55bb1a7c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 -- 172.21.15.105:0/1537284698 shutdown_connections 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 -- 172.21.15.105:0/1537284698 wait complete. 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 -- start start 2024-03-26T17:44:31.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.396+0000 7fe8803a6740 1 Processor -- start 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8803a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8803a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781ac460 con 0x55bb1a7be190 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8803a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781abd90 con 0x55bb1a7c3cf0 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8803a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8781a8fd0 con 0x55bb1a7beb80 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8803a6740 10 client.0 _ll_register_callbacks cb 0x7fe8680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56372/0 (socket says 172.21.15.105:56372) 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 -- 172.21.15.105:0/57066607 learned_addr learned my addr 172.21.15.105:0/57066607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe876ffd640 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 -- 172.21.15.105:0/57066607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 msgr2=0x55bb1a7cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 -- 172.21.15.105:0/57066607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 msgr2=0x55bb1a7cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 -- 172.21.15.105:0/57066607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb1a7cf3d0 con 0x55bb1a7be190 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8767fc640 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7cd900 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00fc20 tx=0x7fe86c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a271a71f1d61dfc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe85aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe85aff5640 1 -- 172.21.15.105:0/57066607 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c009e70 con 0x55bb1a7be190 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe85aff5640 1 -- 172.21.15.105:0/57066607 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe86c01f650 con 0x55bb1a7be190 2024-03-26T17:44:31.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.397+0000 7fe8803a6740 1 -- 172.21.15.105:0/57066607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe86801f700 con 0x55bb1a7be190 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe85aff5640 1 -- 172.21.15.105:0/57066607 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe86c01f850 con 0x55bb1a7be190 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe8803a6740 10 client.17742 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe8803a6740 1 -- 172.21.15.105:0/57066607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb1a83ab00 con 0x55bb1a7be190 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe85aff5640 1 -- 172.21.15.105:0/57066607 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe86801f700 con 0x55bb1a7be190 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe85aff5640 1 -- 172.21.15.105:0/57066607 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe86c01d040 con 0x55bb1a7be190 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe8803a6740 10 client.17742 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe8803a6740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe8803a6740 20 client.17742 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe8803a6740 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb1a83b390 0x55bb1a83d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe8803a6740 4 client.17742 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe8803a6740 1 -- 172.21.15.105:0/57066607 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bb1a72aec0 con 0x55bb1a83b390 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe876ffd640 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb1a83b390 0x55bb1a83d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe85aff5640 10 client.17742 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.398+0000 7fe876ffd640 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb1a83b390 0x55bb1a83d7b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f0970216d1d4853 server_cookie=3a03b816ac47ba75 in_seq=0 out_seq=0 2024-03-26T17:44:31.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe85aff5640 1 -- 172.21.15.105:0/57066607 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb1a72aec0 con 0x55bb1a83b390 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe85aff5640 10 client.17742 handle_command_reply: tid=0 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 client.17742 shutdown 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 -- 172.21.15.105:0/57066607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 msgr2=0x55bb1a7cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7cd900 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00fc20 tx=0x7fe86c0096c0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 -- 172.21.15.105:0/57066607 shutdown_connections 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb1a83b390 0x55bb1a83d7b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb1a7c3cf0 0x55bb1a7cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb1a7beb80 0x55bb1a7cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 --2- 172.21.15.105:0/57066607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb1a7be190 0x55bb1a7cd900 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 -- 172.21.15.105:0/57066607 >> 172.21.15.105:0/57066607 conn(0x55bb1a7b9700 msgr2=0x55bb1a7c7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe85aff5640 0 client.17742 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 -- 172.21.15.105:0/57066607 shutdown_connections 2024-03-26T17:44:31.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 1 -- 172.21.15.105:0/57066607 wait complete. 2024-03-26T17:44:31.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.401+0000 7fe8803a6740 20 client.17742 trim_cache size 0 max 16384 2024-03-26T17:44:31.399 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:44:31.399 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:44:31.399 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "989d2cc5-e31a-4f39-a856-f8dc2c1be48a", 2024-03-26T17:44:31.399 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:44:31.399 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:44:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 -- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8600a2f20 msgr2=0x7fe8600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8600a2f20 0x7fe8600a53e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe87003bfd0 tx=0x7fe870088800 comp rx=0 tx=0).stop 2024-03-26T17:44:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 -- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 msgr2=0x7fe8781032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 0x7fe8781032f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe870009dc0 tx=0x7fe8700328a0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 -- 172.21.15.105:0/1358397530 shutdown_connections 2024-03-26T17:44:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8600a2f20 0x7fe8600a53e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe878102390 0x7fe8781032f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8781019d0 0x7fe878102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 --2- 172.21.15.105:0/1358397530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8781007e0 0x7fe878104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 -- 172.21.15.105:0/1358397530 >> 172.21.15.105:0/1358397530 conn(0x7fe8780fbd70 msgr2=0x7fe8780fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 -- 172.21.15.105:0/1358397530 shutdown_connections 2024-03-26T17:44:31.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.402+0000 7fe87d6dc640 1 -- 172.21.15.105:0/1358397530 wait complete. 2024-03-26T17:44:31.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:44:31.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:44:31.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 -- 172.21.15.105:0/2716908670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe674101470 msgr2=0x7fe6741018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 --2- 172.21.15.105:0/2716908670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe674101470 0x7fe6741018f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009ca0 tx=0x7fe66c02c140 comp rx=0 tx=0).stop 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 -- 172.21.15.105:0/2716908670 shutdown_connections 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 --2- 172.21.15.105:0/2716908670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe674101470 0x7fe6741018f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 --2- 172.21.15.105:0/2716908670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe674100ad0 0x7fe674100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 --2- 172.21.15.105:0/2716908670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe674100170 0x7fe674100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 -- 172.21.15.105:0/2716908670 >> 172.21.15.105:0/2716908670 conn(0x7fe6740fbd50 msgr2=0x7fe6740fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 -- 172.21.15.105:0/2716908670 shutdown_connections 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.534+0000 7fe67a587640 1 -- 172.21.15.105:0/2716908670 wait complete. 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.535+0000 7fe67a587640 1 Processor -- start 2024-03-26T17:44:31.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.535+0000 7fe67a587640 1 -- start start 2024-03-26T17:44:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.535+0000 7fe67a587640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe674100170 0x7fe674199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.535+0000 7fe67a587640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe674100ad0 0x7fe67419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.535+0000 7fe67a587640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe674101470 0x7fe6741a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.535+0000 7fe67a587640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe67410f5f0 con 0x7fe674100ad0 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.535+0000 7fe67a587640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe67410f470 con 0x7fe674101470 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.535+0000 7fe67a587640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe67410f770 con 0x7fe674100170 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe674100170 0x7fe674199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe674100170 0x7fe674199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35106/0 (socket says 172.21.15.105:35106) 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 -- 172.21.15.105:0/4064746572 learned_addr learned my addr 172.21.15.105:0/4064746572 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe678b24640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe674101470 0x7fe6741a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe6737fe640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe674100ad0 0x7fe67419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 -- 172.21.15.105:0/4064746572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe674101470 msgr2=0x7fe6741a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe674101470 0x7fe6741a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 -- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe674100ad0 msgr2=0x7fe67419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe674100ad0 0x7fe67419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 -- 172.21.15.105:0/4064746572 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6741a1920 con 0x7fe674100170 2024-03-26T17:44:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe678b24640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe674101470 0x7fe6741a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe673fff640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe674100170 0x7fe674199d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9f0 tx=0x7fe66400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe6717fa640 1 -- 172.21.15.105:0/4064746572 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400cd80 con 0x7fe674100170 2024-03-26T17:44:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe6717fa640 1 -- 172.21.15.105:0/4064746572 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6640054d0 con 0x7fe674100170 2024-03-26T17:44:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe67a587640 1 -- 172.21.15.105:0/4064746572 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6741a1c10 con 0x7fe674100170 2024-03-26T17:44:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.536+0000 7fe6717fa640 1 -- 172.21.15.105:0/4064746572 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664010490 con 0x7fe674100170 2024-03-26T17:44:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.537+0000 7fe67a587640 1 -- 172.21.15.105:0/4064746572 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6741a2160 con 0x7fe674100170 2024-03-26T17:44:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.539+0000 7fe6717fa640 1 -- 172.21.15.105:0/4064746572 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe664005020 con 0x7fe674100170 2024-03-26T17:44:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe6717fa640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe65c0a2fa0 0x7fe65c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe6737fe640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe65c0a2fa0 0x7fe65c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe6717fa640 1 -- 172.21.15.105:0/4064746572 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe6640cf790 con 0x7fe674100170 2024-03-26T17:44:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe67d170740 1 Processor -- start 2024-03-26T17:44:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe67d170740 1 -- start start 2024-03-26T17:44:31.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe6737fe640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe65c0a2fa0 0x7fe65c0a5460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe668009bd0 tx=0x7fe668005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe67d170740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e918f0 0x55f466e91d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe67d170740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e922e0 0x55f466e96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe67d170740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466e998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe67d170740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f466df6980 con 0x55f466e97450 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe67d170740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f466d9ebe0 con 0x55f466e922e0 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe67d170740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f466df2930 con 0x55f466e918f0 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.540+0000 7fe678b24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466e998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e918f0 0x55f466e91d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e922e0 0x55f466e96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e918f0 0x55f466e91d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35116/0 (socket says 172.21.15.105:35116) 2024-03-26T17:44:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe678b24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466e998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56396/0 (socket says 172.21.15.105:56396) 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 -- 172.21.15.105:0/2332719082 learned_addr learned my addr 172.21.15.105:0/2332719082 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 -- 172.21.15.105:0/2332719082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e922e0 msgr2=0x55f466e96de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 --2- 172.21.15.105:0/2332719082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e922e0 0x55f466e96de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 -- 172.21.15.105:0/2332719082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 msgr2=0x55f466e998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 --2- 172.21.15.105:0/2332719082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466e998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 -- 172.21.15.105:0/2332719082 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f466bd47d0 con 0x55f466e918f0 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe673fff640 1 --2- 172.21.15.105:0/2332719082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e918f0 0x55f466e91d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe66409be10 tx=0x7fe66409b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f58eae5700feb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe65a7fc640 1 -- 172.21.15.105:0/2332719082 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664005680 con 0x55f466e918f0 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe65a7fc640 1 -- 172.21.15.105:0/2332719082 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe66409c580 con 0x55f466e918f0 2024-03-26T17:44:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.541+0000 7fe65a7fc640 1 -- 172.21.15.105:0/2332719082 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664098460 con 0x55f466e918f0 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 -- 172.21.15.105:0/2332719082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e918f0 msgr2=0x55f466e91d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 --2- 172.21.15.105:0/2332719082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e918f0 0x55f466e91d10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe66409be10 tx=0x7fe66409b8a0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 -- 172.21.15.105:0/2332719082 shutdown_connections 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 --2- 172.21.15.105:0/2332719082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466e998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 --2- 172.21.15.105:0/2332719082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e922e0 0x55f466e96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 --2- 172.21.15.105:0/2332719082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e918f0 0x55f466e91d10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 -- 172.21.15.105:0/2332719082 >> 172.21.15.105:0/2332719082 conn(0x55f466e8ce60 msgr2=0x55f466e8d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 -- 172.21.15.105:0/2332719082 shutdown_connections 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 -- 172.21.15.105:0/2332719082 wait complete. 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.542+0000 7fe67d170740 1 -- start start 2024-03-26T17:44:31.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe67d170740 1 Processor -- start 2024-03-26T17:44:31.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe67d170740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe67d170740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe67d170740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466ea3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe67d170740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f466e9a400 con 0x55f466e97450 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe67d170740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f466db2160 con 0x55f466e918f0 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe673fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe673fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39102/0 (socket says 172.21.15.105:39102) 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe673fff640 1 -- 172.21.15.105:0/453188427 learned_addr learned my addr 172.21.15.105:0/453188427 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe6737fe640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe678b24640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466ea3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f466e9a580 con 0x55f466e922e0 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe67d170740 10 client.0 _ll_register_callbacks cb 0x55f466dee720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe673fff640 1 -- 172.21.15.105:0/453188427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 msgr2=0x55f466e9c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe673fff640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe673fff640 1 -- 172.21.15.105:0/453188427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 msgr2=0x55f466ea3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe673fff640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466ea3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.543+0000 7fe673fff640 1 -- 172.21.15.105:0/453188427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f466b0a3f0 con 0x55f466e918f0 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe673fff640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9bae0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe66409bb80 tx=0x7fe6640048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeddfea9257ed869 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe6537fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe6537fe640 1 -- 172.21.15.105:0/453188427 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66409be80 con 0x55f466e918f0 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe678b24640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466ea3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:44:31.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe66409c580 con 0x55f466e918f0 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe6537fe640 1 -- 172.21.15.105:0/453188427 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe6640bc050 con 0x55f466e918f0 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe6737fe640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe6537fe640 1 -- 172.21.15.105:0/453188427 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe6640a4610 con 0x55f466e918f0 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe67d170740 10 client.27581 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.544+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f466aca0f0 con 0x55f466e918f0 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe6537fe640 1 -- 172.21.15.105:0/453188427 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe66409c580 con 0x55f466e918f0 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe6537fe640 1 -- 172.21.15.105:0/453188427 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe6640b8070 con 0x55f466e918f0 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe67d170740 10 client.27581 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe67d170740 10 client.27581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe67d170740 20 client.27581 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe67d170740 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f466ea7530 0x55f466ea9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe67d170740 4 client.27581 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:31.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f466e03ec0 con 0x55f466ea7530 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.545+0000 7fe6737fe640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f466ea7530 0x55f466ea9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe6537fe640 10 client.27581 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe6737fe640 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f466ea7530 0x55f466ea9920 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56dcc84a16512316 server_cookie=b1d5360d824d4c01 in_seq=0 out_seq=0 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe6537fe640 1 -- 172.21.15.105:0/453188427 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f466e03ec0 con 0x55f466ea7530 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe6537fe640 10 client.27581 handle_command_reply: tid=0 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 client.27581 shutdown 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 msgr2=0x55f466e9bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9bae0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe66409bb80 tx=0x7fe6640048f0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 shutdown_connections 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f466ea7530 0x55f466ea9920 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e97450 0x55f466ea3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe6537fe640 0 client.27581 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:31.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 --2- 172.21.15.105:0/453188427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9bae0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.546+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 >> 172.21.15.105:0/453188427 conn(0x55f466e8ce60 msgr2=0x55f466e8c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.547+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 shutdown_connections 2024-03-26T17:44:31.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.547+0000 7fe67d170740 1 -- 172.21.15.105:0/453188427 wait complete. 2024-03-26T17:44:31.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.547+0000 7fe67d170740 20 client.27581 trim_cache size 0 max 16384 2024-03-26T17:44:31.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe67d170740 1 Processor -- start 2024-03-26T17:44:31.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe67d170740 1 -- start start 2024-03-26T17:44:31.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe67d170740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e918f0 0x55f466e9b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe67d170740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe67d170740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e97450 0x55f466e8b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe67d170740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741ab020 con 0x55f466e918f0 2024-03-26T17:44:31.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe67d170740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741acb60 con 0x55f466e97450 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe67d170740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741ab6f0 con 0x55f466e922e0 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe673fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e918f0 0x55f466e9b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe673fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e918f0 0x55f466e9b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56408/0 (socket says 172.21.15.105:56408) 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe673fff640 1 -- 172.21.15.105:0/1808915842 learned_addr learned my addr 172.21.15.105:0/1808915842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe678b24640 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e97450 0x55f466e8b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe6737fe640 1 -- 172.21.15.105:0/1808915842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e97450 msgr2=0x55f466e8b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe6737fe640 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e97450 0x55f466e8b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe6737fe640 1 -- 172.21.15.105:0/1808915842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e918f0 msgr2=0x55f466e9b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe6737fe640 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e918f0 0x55f466e9b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe6737fe640 1 -- 172.21.15.105:0/1808915842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f466e92080 con 0x55f466e922e0 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe678b24640 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e97450 0x55f466e8b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.566+0000 7fe673fff640 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e918f0 0x55f466e9b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:31.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.567+0000 7fe6737fe640 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9bc30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55f466eaa030 tx=0x7fe668014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3da25fc9e81f629 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.567+0000 7fe659ffb640 1 -- 172.21.15.105:0/1808915842 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66802e070 con 0x55f466e922e0 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.567+0000 7fe659ffb640 1 -- 172.21.15.105:0/1808915842 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe668029d90 con 0x55f466e922e0 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.567+0000 7fe659ffb640 1 -- 172.21.15.105:0/1808915842 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66801d740 con 0x55f466e922e0 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- 172.21.15.105:0/1808915842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 msgr2=0x55f466e9bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9bc30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55f466eaa030 tx=0x7fe668014000 comp rx=0 tx=0).stop 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- 172.21.15.105:0/1808915842 shutdown_connections 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e97450 0x55f466e8b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e922e0 0x55f466e9bc30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 --2- 172.21.15.105:0/1808915842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e918f0 0x55f466e9b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- 172.21.15.105:0/1808915842 >> 172.21.15.105:0/1808915842 conn(0x55f466e8ce60 msgr2=0x55f466ea9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- 172.21.15.105:0/1808915842 shutdown_connections 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- 172.21.15.105:0/1808915842 wait complete. 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- start start 2024-03-26T17:44:31.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 Processor -- start 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e922e0 0x55f466ea04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e97450 0x55f466ea1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741ac490 con 0x55f466e922e0 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741abdc0 con 0x55f466e918f0 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741a9000 con 0x55f466e97450 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.568+0000 7fe67d170740 10 client.0 _ll_register_callbacks cb 0x7fe668005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e922e0 0x55f466ea04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e922e0 0x55f466ea04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56420/0 (socket says 172.21.15.105:56420) 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe678b24640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e97450 0x55f466ea1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 -- 172.21.15.105:0/2232599833 learned_addr learned my addr 172.21.15.105:0/2232599833 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe673fff640 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 -- 172.21.15.105:0/2232599833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e97450 msgr2=0x55f466ea1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e97450 0x55f466ea1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 -- 172.21.15.105:0/2232599833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 msgr2=0x55f466e9ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 -- 172.21.15.105:0/2232599833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f466e92fb0 con 0x55f466e922e0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6737fe640 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e922e0 0x55f466ea04c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe668004650 tx=0x7fe668003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602e7753947fe38a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6537fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6537fe640 1 -- 172.21.15.105:0/2232599833 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668036020 con 0x55f466e922e0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6537fe640 1 -- 172.21.15.105:0/2232599833 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe668029430 con 0x55f466e922e0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe67d170740 1 -- 172.21.15.105:0/2232599833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe668029d90 con 0x55f466e922e0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe6537fe640 1 -- 172.21.15.105:0/2232599833 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe668041050 con 0x55f466e922e0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe67d170740 10 client.17757 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.569+0000 7fe67d170740 1 -- 172.21.15.105:0/2232599833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f466ea2be0 con 0x55f466e922e0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe6537fe640 1 -- 172.21.15.105:0/2232599833 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe668029d90 con 0x55f466e922e0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe6537fe640 1 -- 172.21.15.105:0/2232599833 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe668031050 con 0x55f466e922e0 2024-03-26T17:44:31.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe67d170740 10 client.17757 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe67d170740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe67d170740 20 client.17757 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe67d170740 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f466f0dad0 0x55f466f0fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe67d170740 4 client.17757 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe67d170740 1 -- 172.21.15.105:0/2232599833 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f466dfe210 con 0x55f466f0dad0 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe673fff640 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f466f0dad0 0x55f466f0fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe6537fe640 10 client.17757 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.570+0000 7fe673fff640 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f466f0dad0 0x55f466f0fef0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5776e1e91e33e3d server_cookie=baf02e557125f228 in_seq=0 out_seq=0 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe6537fe640 1 -- 172.21.15.105:0/2232599833 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f466dfe210 con 0x55f466f0dad0 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe6537fe640 10 client.17757 handle_command_reply: tid=0 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 client.17757 shutdown 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 -- 172.21.15.105:0/2232599833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e922e0 msgr2=0x55f466ea04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e922e0 0x55f466ea04c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe668004650 tx=0x7fe668003b80 comp rx=0 tx=0).stop 2024-03-26T17:44:31.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 -- 172.21.15.105:0/2232599833 shutdown_connections 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f466f0dad0 0x55f466f0fef0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f466e97450 0x55f466ea1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe6537fe640 0 client.17757 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f466e922e0 0x55f466ea04c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 --2- 172.21.15.105:0/2232599833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f466e918f0 0x55f466e9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 -- 172.21.15.105:0/2232599833 >> 172.21.15.105:0/2232599833 conn(0x55f466e8ce60 msgr2=0x55f466e9b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 -- 172.21.15.105:0/2232599833 shutdown_connections 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 1 -- 172.21.15.105:0/2232599833 wait complete. 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.571+0000 7fe67d170740 20 client.17757 trim_cache size 0 max 16384 2024-03-26T17:44:31.576 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout: "989d2cc5-e31a-4f39-a856-f8dc2c1be48a": { 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "989d2cc5-e31a-4f39-a856-f8dc2c1be48a", 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:44:31.577 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:44:31.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 -- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe65c0a2fa0 msgr2=0x7fe65c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe65c0a2fa0 0x7fe65c0a5460 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe668009bd0 tx=0x7fe668005f90 comp rx=0 tx=0).stop 2024-03-26T17:44:31.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 -- 172.21.15.105:0/4064746572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe674100170 msgr2=0x7fe674199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe674100170 0x7fe674199d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9f0 tx=0x7fe66400edf0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 -- 172.21.15.105:0/4064746572 shutdown_connections 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe65c0a2fa0 0x7fe65c0a5460 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe674101470 0x7fe6741a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe674100ad0 0x7fe67419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 --2- 172.21.15.105:0/4064746572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe674100170 0x7fe674199d10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 -- 172.21.15.105:0/4064746572 >> 172.21.15.105:0/4064746572 conn(0x7fe6740fbd50 msgr2=0x7fe6740fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 -- 172.21.15.105:0/4064746572 shutdown_connections 2024-03-26T17:44:31.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:44:31.572+0000 7fe67a587640 1 -- 172.21.15.105:0/4064746572 wait complete. 2024-03-26T17:44:31.580 INFO:tasks.cephfs.filesystem:scrub status for tag:989d2cc5-e31a-4f39-a856-f8dc2c1be48a - {'path': '/', 'tag': '989d2cc5-e31a-4f39-a856-f8dc2c1be48a', 'options': 'recursive,force'} 2024-03-26T17:44:31.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:31 smithi105 ceph-mon[32280]: pgmap v533: 97 pgs: 97 active+clean; 27 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-03-26T17:44:31.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:31 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:44:31.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:31 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:31.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:31 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:44:31.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:31 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:31.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:31 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:44:32.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 185400 3.14 MB/sec execute 195 sec latency 26.097 ms 2024-03-26T17:44:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:31 smithi144 ceph-mon[36095]: pgmap v533: 97 pgs: 97 active+clean; 27 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-03-26T17:44:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:31 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:44:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:31 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:31 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:44:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:31 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:32.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:31 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:44:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:31 smithi135 ceph-mon[35636]: pgmap v533: 97 pgs: 97 active+clean; 27 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-03-26T17:44:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:31 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:44:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:31 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:31 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:44:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:31 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:44:32.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:31 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:44:33.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 186068 3.15 MB/sec execute 196 sec latency 16.884 ms 2024-03-26T17:44:34.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 186763 3.14 MB/sec execute 197 sec latency 18.735 ms 2024-03-26T17:44:34.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:33 smithi144 ceph-mon[36095]: pgmap v534: 97 pgs: 97 active+clean; 27 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-26T17:44:34.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:33 smithi105 ceph-mon[32280]: pgmap v534: 97 pgs: 97 active+clean; 27 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-26T17:44:34.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:33 smithi135 ceph-mon[35636]: pgmap v534: 97 pgs: 97 active+clean; 27 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-03-26T17:44:35.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 187435 3.15 MB/sec execute 198 sec latency 17.268 ms 2024-03-26T17:44:36.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 188023 3.16 MB/sec execute 199 sec latency 15.684 ms 2024-03-26T17:44:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:35 smithi144 ceph-mon[36095]: pgmap v535: 97 pgs: 97 active+clean; 28 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:44:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:35 smithi105 ceph-mon[32280]: pgmap v535: 97 pgs: 97 active+clean; 28 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:44:36.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:35 smithi135 ceph-mon[35636]: pgmap v535: 97 pgs: 97 active+clean; 28 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:44:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 -- 172.21.15.105:0/1101811045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41014b0 msgr2=0x7f90c4101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 --2- 172.21.15.105:0/1101811045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41014b0 0x7f90c4101930 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009ca0 tx=0x7f90bc02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 -- 172.21.15.105:0/1101811045 shutdown_connections 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 --2- 172.21.15.105:0/1101811045 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41014b0 0x7f90c4101930 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 --2- 172.21.15.105:0/1101811045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c4100b10 0x7f90c4100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 --2- 172.21.15.105:0/1101811045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c41001b0 0x7f90c41005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 -- 172.21.15.105:0/1101811045 >> 172.21.15.105:0/1101811045 conn(0x7f90c40fbd50 msgr2=0x7f90c40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 -- 172.21.15.105:0/1101811045 shutdown_connections 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.546+0000 7f90c8e85640 1 -- 172.21.15.105:0/1101811045 wait complete. 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c8e85640 1 Processor -- start 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c8e85640 1 -- start start 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c8e85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41001b0 0x7f90c4199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c8e85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c4100b10 0x7f90c419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c8e85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c41014b0 0x7f90c41a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c8e85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90c410f5e0 con 0x7f90c41001b0 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c8e85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90c410f460 con 0x7f90c41014b0 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c8e85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90c410f760 con 0x7f90c4100b10 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41001b0 0x7f90c4199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c41014b0 0x7f90c41a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c4100b10 0x7f90c419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c41014b0 0x7f90c41a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37956/0 (socket says 172.21.15.105:37956) 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c2d9d640 1 -- 172.21.15.105:0/261218465 learned_addr learned my addr 172.21.15.105:0/261218465 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41001b0 0x7f90c4199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43036/0 (socket says 172.21.15.105:43036) 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c259c640 1 -- 172.21.15.105:0/261218465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c4100b10 msgr2=0x7f90c419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c259c640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c4100b10 0x7f90c419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c259c640 1 -- 172.21.15.105:0/261218465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c41014b0 msgr2=0x7f90c41a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c259c640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c41014b0 0x7f90c41a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.547+0000 7f90c259c640 1 -- 172.21.15.105:0/261218465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c41a1880 con 0x7f90c41001b0 2024-03-26T17:44:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.548+0000 7f90c259c640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41001b0 0x7f90c4199c70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f90b400e9f0 tx=0x7f90b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.548+0000 7f90b37fe640 1 -- 172.21.15.105:0/261218465 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b400ccc0 con 0x7f90c41001b0 2024-03-26T17:44:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.548+0000 7f90b37fe640 1 -- 172.21.15.105:0/261218465 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90b40108e0 con 0x7f90c41001b0 2024-03-26T17:44:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.548+0000 7f90c8e85640 1 -- 172.21.15.105:0/261218465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c41a1b70 con 0x7f90c41001b0 2024-03-26T17:44:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.548+0000 7f90b37fe640 1 -- 172.21.15.105:0/261218465 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4018880 con 0x7f90c41001b0 2024-03-26T17:44:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.548+0000 7f90c8e85640 1 -- 172.21.15.105:0/261218465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c41a2110 con 0x7f90c41001b0 2024-03-26T17:44:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90b37fe640 1 -- 172.21.15.105:0/261218465 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f90b4010a90 con 0x7f90c41001b0 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90cba75740 1 Processor -- start 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90cba75740 1 -- start start 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90cba75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853ea99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90cba75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eaeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90cba75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90cba75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561853e0e0b0 con 0x561853ea9fc0 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90cba75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561853db6960 con 0x561853ea95d0 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90cba75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561853e0a060 con 0x561853eaf130 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853ea99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eaeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853ea99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37970/0 (socket says 172.21.15.105:37970) 2024-03-26T17:44:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.551+0000 7f90c1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c2d9d640 1 -- 172.21.15.105:0/3732971453 learned_addr learned my addr 172.21.15.105:0/3732971453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90b37fe640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a3040 0x7f90ac0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a3040 0x7f90ac0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c1d9b640 1 -- 172.21.15.105:0/3732971453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 msgr2=0x561853ea99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3732971453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853ea99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c1d9b640 1 -- 172.21.15.105:0/3732971453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 msgr2=0x561853eaeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3732971453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eaeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c1d9b640 1 -- 172.21.15.105:0/3732971453 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561853dc8c60 con 0x561853eaf130 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/3732971453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853ea99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90b37fe640 1 -- 172.21.15.105:0/261218465 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f90b40cf730 con 0x7f90c41001b0 2024-03-26T17:44:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a3040 0x7f90ac0a5500 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x561853eaa820 tx=0x7f90bc02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.552+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3732971453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb15c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f90b8017fd0 tx=0x7f90b8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4fa426640576f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.553+0000 7f90b0ff9640 1 -- 172.21.15.105:0/3732971453 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8004050 con 0x561853eaf130 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.553+0000 7f90b0ff9640 1 -- 172.21.15.105:0/3732971453 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90b800ee40 con 0x561853eaf130 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.553+0000 7f90b0ff9640 1 -- 172.21.15.105:0/3732971453 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8003c50 con 0x561853eaf130 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.553+0000 7f90cba75740 1 -- 172.21.15.105:0/3732971453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 msgr2=0x561853eb15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.553+0000 7f90cba75740 1 --2- 172.21.15.105:0/3732971453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb15c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f90b8017fd0 tx=0x7f90b8009c30 comp rx=0 tx=0).stop 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.553+0000 7f90cba75740 1 -- 172.21.15.105:0/3732971453 shutdown_connections 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.553+0000 7f90cba75740 1 --2- 172.21.15.105:0/3732971453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb15c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 --2- 172.21.15.105:0/3732971453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eaeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 --2- 172.21.15.105:0/3732971453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853ea99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 -- 172.21.15.105:0/3732971453 >> 172.21.15.105:0/3732971453 conn(0x561853ea4b40 msgr2=0x561853ea4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 -- 172.21.15.105:0/3732971453 shutdown_connections 2024-03-26T17:44:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 -- 172.21.15.105:0/3732971453 wait complete. 2024-03-26T17:44:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 -- start start 2024-03-26T17:44:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 Processor -- start 2024-03-26T17:44:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea9fc0 0x561853eb38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853eaf130 0x561853eba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561853eb20e0 con 0x561853eaf130 2024-03-26T17:44:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561853dc9db0 con 0x561853ea95d0 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561853eb2260 con 0x561853ea9fc0 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90cba75740 10 client.0 _ll_register_callbacks cb 0x561853dce110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90c259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853eaf130 0x561853eba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90c259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37980/0 (socket says 172.21.15.105:37980) 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90c259c640 1 -- 172.21.15.105:0/3052996907 learned_addr learned my addr 172.21.15.105:0/3052996907 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.554+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea9fc0 0x561853eb38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90c259c640 1 -- 172.21.15.105:0/3052996907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea9fc0 msgr2=0x561853eb38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90c259c640 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea9fc0 0x561853eb38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90c259c640 1 -- 172.21.15.105:0/3052996907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853eaf130 msgr2=0x561853eba890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90c259c640 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853eaf130 0x561853eba890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90c259c640 1 -- 172.21.15.105:0/3052996907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561853b22210 con 0x561853ea95d0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90c259c640 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb3380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f90b40982a0 tx=0x7f90b4096870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf305b8a82d526d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3052996907 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b40cfde0 con 0x561853ea95d0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3052996907 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f90b40bab90 con 0x561853ea95d0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90cba75740 1 -- 172.21.15.105:0/3052996907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561853e1b5f0 con 0x561853ea95d0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3052996907 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90b409bed0 con 0x561853ea95d0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90cba75740 10 client.27599 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.555+0000 7f90cba75740 1 -- 172.21.15.105:0/3052996907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561853bee620 con 0x561853ea95d0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3052996907 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561853e1b5f0 con 0x561853ea95d0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3052996907 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f90b409b320 con 0x561853ea95d0 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90cba75740 10 client.27599 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90cba75740 10 client.27599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90cba75740 20 client.27599 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90cba75740 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561853ebf130 0x561853ec1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90cba75740 4 client.27599 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90cba75740 1 -- 172.21.15.105:0/3052996907 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561853e15940 con 0x561853ebf130 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.556+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561853ebf130 0x561853ec1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.557+0000 7f90a9ffb640 10 client.27599 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.557+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561853ebf130 0x561853ec1560 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200da4913f286917 server_cookie=53f64e70cf57dc63 in_seq=0 out_seq=0 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3052996907 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561853e15940 con 0x561853ebf130 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90a9ffb640 10 client.27599 handle_command_reply: tid=0 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 client.27599 shutdown 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 -- 172.21.15.105:0/3052996907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 msgr2=0x561853eb3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb3380 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f90b40982a0 tx=0x7f90b4096870 comp rx=0 tx=0).stop 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 -- 172.21.15.105:0/3052996907 shutdown_connections 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561853ebf130 0x561853ec1560 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853eaf130 0x561853eba890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90a9ffb640 0 client.27599 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea9fc0 0x561853eb38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 --2- 172.21.15.105:0/3052996907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb3380 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 -- 172.21.15.105:0/3052996907 >> 172.21.15.105:0/3052996907 conn(0x561853ea4b40 msgr2=0x561853ea3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 -- 172.21.15.105:0/3052996907 shutdown_connections 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 1 -- 172.21.15.105:0/3052996907 wait complete. 2024-03-26T17:44:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.558+0000 7f90cba75740 20 client.27599 trim_cache size 0 max 16384 2024-03-26T17:44:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90cba75740 1 Processor -- start 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90cba75740 1 -- start start 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90cba75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea95d0 0x561853eaaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90cba75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90cba75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853eaf130 0x561853eaaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90cba75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41a6b80 con 0x561853ea9fc0 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90cba75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41a86c0 con 0x561853eaf130 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90cba75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41a7250 con 0x561853ea95d0 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90c259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea95d0 0x561853eaaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eba850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43068/0 (socket says 172.21.15.105:43068) 2024-03-26T17:44:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90c1d9b640 1 -- 172.21.15.105:0/660048275 learned_addr learned my addr 172.21.15.105:0/660048275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.585+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853eaf130 0x561853eaaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90c259c640 1 -- 172.21.15.105:0/660048275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853eaf130 msgr2=0x561853eaaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90c259c640 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853eaf130 0x561853eaaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90c259c640 1 -- 172.21.15.105:0/660048275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 msgr2=0x561853eba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90c259c640 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90c259c640 1 -- 172.21.15.105:0/660048275 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561853ea5590 con 0x561853ea95d0 2024-03-26T17:44:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90c259c640 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea95d0 0x561853eaaa10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f90b40968e0 tx=0x7f90b40cd280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9d84c7687aa1d49 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90abfff640 1 -- 172.21.15.105:0/660048275 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b409a010 con 0x561853ea95d0 2024-03-26T17:44:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90abfff640 1 -- 172.21.15.105:0/660048275 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90b40a4830 con 0x561853ea95d0 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.586+0000 7f90abfff640 1 -- 172.21.15.105:0/660048275 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4010690 con 0x561853ea95d0 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 -- 172.21.15.105:0/660048275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea95d0 msgr2=0x561853eaaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea95d0 0x561853eaaa10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f90b40968e0 tx=0x7f90b40cd280 comp rx=0 tx=0).stop 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 -- 172.21.15.105:0/660048275 shutdown_connections 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853eaf130 0x561853eaaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 --2- 172.21.15.105:0/660048275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853ea95d0 0x561853eaaa10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 -- 172.21.15.105:0/660048275 >> 172.21.15.105:0/660048275 conn(0x561853ea4b40 msgr2=0x561853ea7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 -- 172.21.15.105:0/660048275 shutdown_connections 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 -- 172.21.15.105:0/660048275 wait complete. 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 -- start start 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.587+0000 7f90cba75740 1 Processor -- start 2024-03-26T17:44:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90cba75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90cba75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eb8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90cba75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90cba75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41a7ff0 con 0x561853ea9fc0 2024-03-26T17:44:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90cba75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41a7920 con 0x561853ea95d0 2024-03-26T17:44:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90cba75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41a4b60 con 0x561853eaf130 2024-03-26T17:44:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90cba75740 10 client.0 _ll_register_callbacks cb 0x7f90b400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eb8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38012/0 (socket says 172.21.15.105:38012) 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c259c640 1 -- 172.21.15.105:0/1790602656 learned_addr learned my addr 172.21.15.105:0/1790602656 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eb8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43076/0 (socket says 172.21.15.105:43076) 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c1d9b640 1 -- 172.21.15.105:0/1790602656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 msgr2=0x561853eb9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c1d9b640 1 -- 172.21.15.105:0/1790602656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 msgr2=0x561853eb7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c1d9b640 1 -- 172.21.15.105:0/1790602656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561853eb9a00 con 0x561853ea9fc0 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c259c640 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eb8400 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f90b800e780 tx=0x7f90b8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7483df252e259 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1790602656 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8039060 con 0x561853ea9fc0 2024-03-26T17:44:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1790602656 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f90b801d040 con 0x561853ea9fc0 2024-03-26T17:44:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90cba75740 1 -- 172.21.15.105:0/1790602656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b40a4830 con 0x561853ea9fc0 2024-03-26T17:44:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.588+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1790602656 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90b800ea90 con 0x561853ea9fc0 2024-03-26T17:44:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90cba75740 10 client.17775 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90cba75740 1 -- 172.21.15.105:0/1790602656 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561853f254a0 con 0x561853ea9fc0 2024-03-26T17:44:36.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1790602656 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90b40a4830 con 0x561853ea9fc0 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1790602656 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f90b8005070 con 0x561853ea9fc0 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90cba75740 10 client.17775 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90cba75740 10 client.17775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90cba75740 20 client.17775 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90cba75740 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561853f25b10 0x561853f27f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90cba75740 4 client.17775 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90cba75740 1 -- 172.21.15.105:0/1790602656 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561853d93770 con 0x561853f25b10 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.589+0000 7f90c259c640 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561853f25b10 0x561853f27f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.590+0000 7f90c259c640 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561853f25b10 0x561853f27f30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb88aa242495b033 server_cookie=dc23f9d08b3f5a11 in_seq=0 out_seq=0 2024-03-26T17:44:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.590+0000 7f90a9ffb640 10 client.17775 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1790602656 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561853d93770 con 0x561853f25b10 2024-03-26T17:44:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90a9ffb640 10 client.17775 handle_command_reply: tid=0 2024-03-26T17:44:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 client.17775 shutdown 2024-03-26T17:44:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 -- 172.21.15.105:0/1790602656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 msgr2=0x561853eb8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eb8400 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f90b800e780 tx=0x7f90b8009680 comp rx=0 tx=0).stop 2024-03-26T17:44:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 -- 172.21.15.105:0/1790602656 shutdown_connections 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561853f25b10 0x561853f27f30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561853eaf130 0x561853eb9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561853ea9fc0 0x561853eb8400 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 --2- 172.21.15.105:0/1790602656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561853ea95d0 0x561853eb7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90a9ffb640 0 client.17775 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.631+0000 7f90cba75740 1 -- 172.21.15.105:0/1790602656 >> 172.21.15.105:0/1790602656 conn(0x561853ea4b40 msgr2=0x561853eb2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90cba75740 1 -- 172.21.15.105:0/1790602656 shutdown_connections 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90cba75740 1 -- 172.21.15.105:0/1790602656 wait complete. 2024-03-26T17:44:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90cba75740 20 client.17775 trim_cache size 0 max 16384 2024-03-26T17:44:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:44:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:44:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:44:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:44:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 -- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a3040 msgr2=0x7f90ac0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a3040 0x7f90ac0a5500 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x561853eaa820 tx=0x7f90bc02e040 comp rx=0 tx=0).stop 2024-03-26T17:44:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 -- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41001b0 msgr2=0x7f90c4199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41001b0 0x7f90c4199c70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f90b400e9f0 tx=0x7f90b400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 -- 172.21.15.105:0/261218465 shutdown_connections 2024-03-26T17:44:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a3040 0x7f90ac0a5500 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c41014b0 0x7f90c41a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c4100b10 0x7f90c419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 --2- 172.21.15.105:0/261218465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41001b0 0x7f90c4199c70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 -- 172.21.15.105:0/261218465 >> 172.21.15.105:0/261218465 conn(0x7f90c40fbd50 msgr2=0x7f90c40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 -- 172.21.15.105:0/261218465 shutdown_connections 2024-03-26T17:44:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:36.632+0000 7f90c8e85640 1 -- 172.21.15.105:0/261218465 wait complete. 2024-03-26T17:44:37.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 188451 3.16 MB/sec execute 200 sec latency 24.028 ms 2024-03-26T17:44:38.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 188942 3.14 MB/sec execute 201 sec latency 23.229 ms 2024-03-26T17:44:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:37 smithi144 ceph-mon[36095]: pgmap v536: 97 pgs: 97 active+clean; 29 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-26T17:44:38.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:37 smithi105 ceph-mon[32280]: pgmap v536: 97 pgs: 97 active+clean; 29 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-26T17:44:38.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:37 smithi135 ceph-mon[35636]: pgmap v536: 97 pgs: 97 active+clean; 29 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-26T17:44:39.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 189564 3.15 MB/sec execute 202 sec latency 19.099 ms 2024-03-26T17:44:40.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 190230 3.14 MB/sec execute 203 sec latency 15.792 ms 2024-03-26T17:44:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:39 smithi144 ceph-mon[36095]: pgmap v537: 97 pgs: 97 active+clean; 29 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:44:40.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:39 smithi105 ceph-mon[32280]: pgmap v537: 97 pgs: 97 active+clean; 29 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:44:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:39 smithi135 ceph-mon[35636]: pgmap v537: 97 pgs: 97 active+clean; 29 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:44:41.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 190938 3.15 MB/sec execute 204 sec latency 16.653 ms 2024-03-26T17:44:42.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 191399 3.15 MB/sec execute 205 sec latency 17.259 ms 2024-03-26T17:44:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:41 smithi144 ceph-mon[36095]: pgmap v538: 97 pgs: 97 active+clean; 29 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-03-26T17:44:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:41 smithi105 ceph-mon[32280]: pgmap v538: 97 pgs: 97 active+clean; 29 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-03-26T17:44:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:41 smithi135 ceph-mon[35636]: pgmap v538: 97 pgs: 97 active+clean; 29 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-03-26T17:44:43.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 191962 3.16 MB/sec execute 206 sec latency 16.273 ms 2024-03-26T17:44:44.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 192483 3.14 MB/sec execute 207 sec latency 18.528 ms 2024-03-26T17:44:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:43 smithi144 ceph-mon[36095]: pgmap v539: 97 pgs: 97 active+clean; 26 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-03-26T17:44:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:43 smithi105 ceph-mon[32280]: pgmap v539: 97 pgs: 97 active+clean; 26 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-03-26T17:44:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:43 smithi135 ceph-mon[35636]: pgmap v539: 97 pgs: 97 active+clean; 26 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-03-26T17:44:45.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 193084 3.15 MB/sec execute 208 sec latency 16.596 ms 2024-03-26T17:44:46.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 193730 3.14 MB/sec execute 209 sec latency 17.430 ms 2024-03-26T17:44:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:45 smithi144 ceph-mon[36095]: pgmap v540: 97 pgs: 97 active+clean; 27 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:44:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:45 smithi105 ceph-mon[32280]: pgmap v540: 97 pgs: 97 active+clean; 27 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:44:46.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:45 smithi135 ceph-mon[35636]: pgmap v540: 97 pgs: 97 active+clean; 27 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:44:47.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 194448 3.15 MB/sec execute 210 sec latency 17.786 ms 2024-03-26T17:44:48.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 194943 3.15 MB/sec execute 211 sec latency 16.064 ms 2024-03-26T17:44:48.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:47 smithi144 ceph-mon[36095]: pgmap v541: 97 pgs: 97 active+clean; 27 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:44:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:47 smithi105 ceph-mon[32280]: pgmap v541: 97 pgs: 97 active+clean; 27 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:44:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:47 smithi135 ceph-mon[35636]: pgmap v541: 97 pgs: 97 active+clean; 27 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:44:49.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 195512 3.16 MB/sec execute 212 sec latency 14.952 ms 2024-03-26T17:44:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 -- 172.21.15.105:0/4071975004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc107660 msgr2=0x7f12cc109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/4071975004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc107660 0x7f12cc109a50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009ca0 tx=0x7f12c402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 -- 172.21.15.105:0/4071975004 shutdown_connections 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/4071975004 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc107660 0x7f12cc109a50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/4071975004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12cc104d30 0x7f12cc107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/4071975004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12cc102400 0x7f12cc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 -- 172.21.15.105:0/4071975004 >> 172.21.15.105:0/4071975004 conn(0x7f12cc0fbd50 msgr2=0x7f12cc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 -- 172.21.15.105:0/4071975004 shutdown_connections 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.733+0000 7f12d1a7e640 1 -- 172.21.15.105:0/4071975004 wait complete. 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12d1a7e640 1 Processor -- start 2024-03-26T17:44:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12d1a7e640 1 -- start start 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12d1a7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc102400 0x7f12cc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12d1a7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12cc104d30 0x7f12cc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12d1a7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12cc107660 0x7f12cc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12d1a7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10d410 con 0x7f12cc102400 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12d1a7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10d290 con 0x7f12cc104d30 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12d1a7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10d590 con 0x7f12cc107660 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc102400 0x7f12cc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc102400 0x7f12cc197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48438/0 (socket says 172.21.15.105:48438) 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12cb7fe640 1 -- 172.21.15.105:0/3389383018 learned_addr learned my addr 172.21.15.105:0/3389383018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12caffd640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12cc104d30 0x7f12cc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.734+0000 7f12cbfff640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12cc107660 0x7f12cc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12caffd640 1 -- 172.21.15.105:0/3389383018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12cc107660 msgr2=0x7f12cc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12caffd640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12cc107660 0x7f12cc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12caffd640 1 -- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc102400 msgr2=0x7f12cc197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12caffd640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc102400 0x7f12cc197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12caffd640 1 -- 172.21.15.105:0/3389383018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc19f680 con 0x7f12cc104d30 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc102400 0x7f12cc197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12cbfff640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12cc107660 0x7f12cc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12caffd640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12cc104d30 0x7f12cc197fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f12c000e9c0 tx=0x7f12c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.735+0000 7f12c8ff9640 1 -- 172.21.15.105:0/3389383018 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c000ccc0 con 0x7f12cc104d30 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.736+0000 7f12d1a7e640 1 -- 172.21.15.105:0/3389383018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc19f970 con 0x7f12cc104d30 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.736+0000 7f12c8ff9640 1 -- 172.21.15.105:0/3389383018 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12c00108e0 con 0x7f12cc104d30 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.736+0000 7f12c8ff9640 1 -- 172.21.15.105:0/3389383018 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c00187a0 con 0x7f12cc104d30 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.736+0000 7f12d1a7e640 1 -- 172.21.15.105:0/3389383018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc19ff10 con 0x7f12cc104d30 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.738+0000 7f12c8ff9640 1 -- 172.21.15.105:0/3389383018 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f12c0010a90 con 0x7f12cc104d30 2024-03-26T17:44:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12c8ff9640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12b40a2fc0 0x7f12b40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12d4661740 1 Processor -- start 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12d4661740 1 -- start start 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12c8ff9640 1 -- 172.21.15.105:0/3389383018 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f12c00cf720 con 0x7f12cc104d30 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12b40a2fc0 0x7f12b40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12d4661740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d267c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12d4661740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 0x55d8d2681ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12d4661740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12d4661740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d8d25e10b0 con 0x55d8d267c5d0 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12d4661740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8d2589960 con 0x55d8d267cfc0 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12d4661740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d8d25dd060 con 0x55d8d2682130 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12caffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 0x55d8d2681ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26845c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40020/0 (socket says 172.21.15.105:40020) 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cbfff640 1 -- 172.21.15.105:0/1506994530 learned_addr learned my addr 172.21.15.105:0/1506994530 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d267c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cbfff640 1 -- 172.21.15.105:0/1506994530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 msgr2=0x55d8d2681ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cbfff640 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 0x55d8d2681ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cbfff640 1 -- 172.21.15.105:0/1506994530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 msgr2=0x55d8d267c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cbfff640 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d267c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.739+0000 7f12cbfff640 1 -- 172.21.15.105:0/1506994530 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8d259bc60 con 0x55d8d2682130 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.740+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d267c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.740+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12b40a2fc0 0x7f12b40a5480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12bc013920 tx=0x7f12bc013d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.740+0000 7f12cbfff640 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26845c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12c4032380 tx=0x7f12c4005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1387eff2354fc47 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.740+0000 7f12b1ffb640 1 -- 172.21.15.105:0/1506994530 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4043070 con 0x55d8d2682130 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.740+0000 7f12b1ffb640 1 -- 172.21.15.105:0/1506994530 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12c4003e20 con 0x55d8d2682130 2024-03-26T17:44:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.740+0000 7f12b1ffb640 1 -- 172.21.15.105:0/1506994530 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4039040 con 0x55d8d2682130 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 -- 172.21.15.105:0/1506994530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 msgr2=0x55d8d26845c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26845c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12c4032380 tx=0x7f12c4005f70 comp rx=0 tx=0).stop 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 -- 172.21.15.105:0/1506994530 shutdown_connections 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26845c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 0x55d8d2681ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 --2- 172.21.15.105:0/1506994530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d267c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 -- 172.21.15.105:0/1506994530 >> 172.21.15.105:0/1506994530 conn(0x55d8d2677b40 msgr2=0x55d8d2677f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 -- 172.21.15.105:0/1506994530 shutdown_connections 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 -- 172.21.15.105:0/1506994530 wait complete. 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 -- start start 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.741+0000 7f12d4661740 1 Processor -- start 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12d4661740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d267c5d0 0x55d8d268a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12d4661740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12d4661740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d2682130 0x55d8d2691c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12d4661740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d8d2689490 con 0x55d8d267cfc0 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12d4661740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8d259cdb0 con 0x55d8d2682130 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12d4661740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d8d2689610 con 0x55d8d267c5d0 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12d4661740 10 client.0 _ll_register_callbacks cb 0x55d8d25a1110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48464/0 (socket says 172.21.15.105:48464) 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d267c5d0 0x55d8d268a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 -- 172.21.15.105:0/3979653849 learned_addr learned my addr 172.21.15.105:0/3979653849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d2682130 0x55d8d2691c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 -- 172.21.15.105:0/3979653849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d267c5d0 msgr2=0x55d8d268a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d267c5d0 0x55d8d268a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 -- 172.21.15.105:0/3979653849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d2682130 msgr2=0x55d8d2691c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d2682130 0x55d8d2691c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 -- 172.21.15.105:0/3979653849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8d22f5210 con 0x55d8d267cfc0 2024-03-26T17:44:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12cbfff640 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d2682130 0x55d8d2691c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:44:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12caffd640 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268abd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f12c009be50 tx=0x7f12c0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9e8cc471df0766 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.742+0000 7f12aaffd640 1 -- 172.21.15.105:0/3979653849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c00bab80 con 0x55d8d267cfc0 2024-03-26T17:44:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 1 -- 172.21.15.105:0/3979653849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12c4003e20 con 0x55d8d267cfc0 2024-03-26T17:44:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12aaffd640 1 -- 172.21.15.105:0/3979653849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f12c00bad20 con 0x55d8d267cfc0 2024-03-26T17:44:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12aaffd640 1 -- 172.21.15.105:0/3979653849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12c007f050 con 0x55d8d267cfc0 2024-03-26T17:44:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 10 client.17787 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 1 -- 172.21.15.105:0/3979653849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8d23c1620 con 0x55d8d267cfc0 2024-03-26T17:44:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12aaffd640 1 -- 172.21.15.105:0/3979653849 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12c4003e20 con 0x55d8d267cfc0 2024-03-26T17:44:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12aaffd640 1 -- 172.21.15.105:0/3979653849 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f12c0097ca0 con 0x55d8d267cfc0 2024-03-26T17:44:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 10 client.17787 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 10 client.17787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 20 client.17787 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8d2696580 0x55d8d26989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 4 client.17787 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:44:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12d4661740 1 -- 172.21.15.105:0/3979653849 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8d25ee5f0 con 0x55d8d2696580 2024-03-26T17:44:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.743+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8d2696580 0x55d8d26989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.744+0000 7f12aaffd640 10 client.17787 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.744+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8d2696580 0x55d8d26989d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b74055aae86c1e server_cookie=d44f980919d1f7de in_seq=0 out_seq=0 2024-03-26T17:44:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.745+0000 7f12aaffd640 1 -- 172.21.15.105:0/3979653849 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d8d25ee5f0 con 0x55d8d2696580 2024-03-26T17:44:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.745+0000 7f12aaffd640 10 client.17787 handle_command_reply: tid=0 2024-03-26T17:44:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.745+0000 7f12d4661740 1 client.17787 shutdown 2024-03-26T17:44:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.745+0000 7f12d4661740 1 -- 172.21.15.105:0/3979653849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 msgr2=0x55d8d268abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.745+0000 7f12d4661740 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268abd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f12c009be50 tx=0x7f12c0010f40 comp rx=0 tx=0).stop 2024-03-26T17:44:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 1 -- 172.21.15.105:0/3979653849 shutdown_connections 2024-03-26T17:44:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8d2696580 0x55d8d26989d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d2682130 0x55d8d2691c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12aaffd640 0 client.17787 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268abd0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 1 --2- 172.21.15.105:0/3979653849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d267c5d0 0x55d8d268a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 1 -- 172.21.15.105:0/3979653849 >> 172.21.15.105:0/3979653849 conn(0x55d8d2677b40 msgr2=0x55d8d2676df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 1 -- 172.21.15.105:0/3979653849 shutdown_connections 2024-03-26T17:44:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 1 -- 172.21.15.105:0/3979653849 wait complete. 2024-03-26T17:44:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.746+0000 7f12d4661740 20 client.17787 trim_cache size 0 max 16384 2024-03-26T17:44:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.761+0000 7f12d4661740 1 Processor -- start 2024-03-26T17:44:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12d4661740 1 -- start start 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12d4661740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d2689cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12d4661740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 0x55d8d2691bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12d4661740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d267d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12d4661740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a6a70 con 0x55d8d267c5d0 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12d4661740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a85b0 con 0x55d8d267cfc0 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12d4661740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a7140 con 0x55d8d2682130 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d2689cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d2689cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48474/0 (socket says 172.21.15.105:48474) 2024-03-26T17:44:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 -- 172.21.15.105:0/3813775960 learned_addr learned my addr 172.21.15.105:0/3813775960 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12caffd640 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 0x55d8d2691bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cbfff640 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d267d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 -- 172.21.15.105:0/3813775960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 msgr2=0x55d8d267d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d267d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 -- 172.21.15.105:0/3813775960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 msgr2=0x55d8d2691bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 0x55d8d2691bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 -- 172.21.15.105:0/3813775960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8d2699050 con 0x55d8d267c5d0 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d2689cb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00f790 tx=0x7f12bc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6929a3d8b517add server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12b17fa640 1 -- 172.21.15.105:0/3813775960 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00fd20 con 0x55d8d267c5d0 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12b17fa640 1 -- 172.21.15.105:0/3813775960 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12bc02a780 con 0x55d8d267c5d0 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.762+0000 7f12b17fa640 1 -- 172.21.15.105:0/3813775960 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc005560 con 0x55d8d267c5d0 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 -- 172.21.15.105:0/3813775960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 msgr2=0x55d8d2689cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d2689cb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00f790 tx=0x7f12bc01f000 comp rx=0 tx=0).stop 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 -- 172.21.15.105:0/3813775960 shutdown_connections 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d267d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267cfc0 0x55d8d2691bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 --2- 172.21.15.105:0/3813775960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267c5d0 0x55d8d2689cb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 -- 172.21.15.105:0/3813775960 >> 172.21.15.105:0/3813775960 conn(0x55d8d2677b40 msgr2=0x55d8d2681300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 -- 172.21.15.105:0/3813775960 shutdown_connections 2024-03-26T17:44:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 -- 172.21.15.105:0/3813775960 wait complete. 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.763+0000 7f12d4661740 1 -- start start 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12d4661740 1 Processor -- start 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12d4661740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267c5d0 0x55d8d268f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12d4661740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12d4661740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12d4661740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a7ee0 con 0x55d8d267cfc0 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12d4661740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a7810 con 0x55d8d267c5d0 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12d4661740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1a4a50 con 0x55d8d2682130 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12d4661740 10 client.0 _ll_register_callbacks cb 0x7f12c009bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267c5d0 0x55d8d268f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12caffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48488/0 (socket says 172.21.15.105:48488) 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12caffd640 1 -- 172.21.15.105:0/3798160241 learned_addr learned my addr 172.21.15.105:0/3798160241 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12caffd640 1 -- 172.21.15.105:0/3798160241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 msgr2=0x55d8d26907f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12caffd640 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26907f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12caffd640 1 -- 172.21.15.105:0/3798160241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267c5d0 msgr2=0x55d8d268f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12caffd640 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267c5d0 0x55d8d268f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.764+0000 7f12caffd640 1 -- 172.21.15.105:0/3798160241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8d2690de0 con 0x55d8d267cfc0 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12caffd640 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268f7e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f12c009bbc0 tx=0x7f12c009b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c91910a1f4d6007 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12aaffd640 1 -- 172.21.15.105:0/3798160241 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0097c10 con 0x55d8d267cfc0 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12aaffd640 1 -- 172.21.15.105:0/3798160241 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f12c0097db0 con 0x55d8d267cfc0 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 1 -- 172.21.15.105:0/3798160241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12bc02a780 con 0x55d8d267cfc0 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12aaffd640 1 -- 172.21.15.105:0/3798160241 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12c009add0 con 0x55d8d267cfc0 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 10 client.17793 fetch_fsmap learned FSMap version 30 2024-03-26T17:44:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 1 -- 172.21.15.105:0/3798160241 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d8d26fc910 con 0x55d8d267cfc0 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12aaffd640 1 -- 172.21.15.105:0/3798160241 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12bc02a780 con 0x55d8d267cfc0 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12aaffd640 1 -- 172.21.15.105:0/3798160241 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f12c009b310 con 0x55d8d267cfc0 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 10 client.17793 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 10 client.17793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 20 client.17793 populate_metadata read hostname 'smithi105' 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8d26fd1e0 0x55d8d26ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 4 client.17793 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.765+0000 7f12d4661740 1 -- 172.21.15.105:0/3798160241 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8d268a480 con 0x55d8d26fd1e0 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.766+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8d26fd1e0 0x55d8d26ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:44:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.766+0000 7f12aaffd640 10 client.17793 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:49.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.766+0000 7f12cb7fe640 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8d26fd1e0 0x55d8d26ff600 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6fb44ec190a5c9 server_cookie=b9e7f8b871571ab7 in_seq=0 out_seq=0 2024-03-26T17:44:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.842+0000 7f12aaffd640 1 -- 172.21.15.105:0/3798160241 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8d268a480 con 0x55d8d26fd1e0 2024-03-26T17:44:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.842+0000 7f12aaffd640 10 client.17793 handle_command_reply: tid=0 2024-03-26T17:44:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.842+0000 7f12d4661740 1 client.17793 shutdown 2024-03-26T17:44:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 -- 172.21.15.105:0/3798160241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 msgr2=0x55d8d268f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268f7e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f12c009bbc0 tx=0x7f12c009b250 comp rx=0 tx=0).stop 2024-03-26T17:44:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 -- 172.21.15.105:0/3798160241 shutdown_connections 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8d26fd1e0 0x55d8d26ff600 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12aaffd640 0 client.17793 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8d2682130 0x55d8d26907f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8d267cfc0 0x55d8d268f7e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 --2- 172.21.15.105:0/3798160241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8d267c5d0 0x55d8d268f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 -- 172.21.15.105:0/3798160241 >> 172.21.15.105:0/3798160241 conn(0x55d8d2677b40 msgr2=0x55d8d267d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 -- 172.21.15.105:0/3798160241 shutdown_connections 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 1 -- 172.21.15.105:0/3798160241 wait complete. 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d4661740 20 client.17793 trim_cache size 0 max 16384 2024-03-26T17:44:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:44:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:44:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:44:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:44:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 -- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12b40a2fc0 msgr2=0x7f12b40a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12b40a2fc0 0x7f12b40a5480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12bc013920 tx=0x7f12bc013d20 comp rx=0 tx=0).stop 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 -- 172.21.15.105:0/3389383018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12cc104d30 msgr2=0x7f12cc197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12cc104d30 0x7f12cc197fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f12c000e9c0 tx=0x7f12c000ee80 comp rx=0 tx=0).stop 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 -- 172.21.15.105:0/3389383018 shutdown_connections 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12b40a2fc0 0x7f12b40a5480 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12cc107660 0x7f12cc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12cc104d30 0x7f12cc197fa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 --2- 172.21.15.105:0/3389383018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12cc102400 0x7f12cc197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.843+0000 7f12d1a7e640 1 -- 172.21.15.105:0/3389383018 >> 172.21.15.105:0/3389383018 conn(0x7f12cc0fbd50 msgr2=0x7f12cc102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.844+0000 7f12d1a7e640 1 -- 172.21.15.105:0/3389383018 shutdown_connections 2024-03-26T17:44:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:44:49.844+0000 7f12d1a7e640 1 -- 172.21.15.105:0/3389383018 wait complete. 2024-03-26T17:44:50.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 195987 3.15 MB/sec execute 213 sec latency 27.771 ms 2024-03-26T17:44:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:49 smithi144 ceph-mon[36095]: pgmap v542: 97 pgs: 97 active+clean; 27 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:44:50.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:49 smithi105 ceph-mon[32280]: pgmap v542: 97 pgs: 97 active+clean; 27 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:44:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:49 smithi135 ceph-mon[35636]: pgmap v542: 97 pgs: 97 active+clean; 27 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:44:51.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 196593 3.15 MB/sec execute 214 sec latency 41.478 ms 2024-03-26T17:44:51.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:50 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:44:51.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:50 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:44:51.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:50 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:44:52.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 197237 3.14 MB/sec execute 215 sec latency 17.659 ms 2024-03-26T17:44:52.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:51 smithi144 ceph-mon[36095]: pgmap v543: 97 pgs: 97 active+clean; 27 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:44:52.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:44:52.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:44:52.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:44:52.151 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:51 smithi105 ceph-mon[32280]: pgmap v543: 97 pgs: 97 active+clean; 27 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:44:52.151 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.151 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.151 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.151 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.151 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:44:52.151 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:44:52.152 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:44:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:51 smithi135 ceph-mon[35636]: pgmap v543: 97 pgs: 97 active+clean; 27 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:44:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:44:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:44:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:44:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:44:53.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 197873 3.15 MB/sec execute 216 sec latency 16.724 ms 2024-03-26T17:44:53.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:52 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-03-26T17:44:53.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:52 smithi144 ceph-mon[36095]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-03-26T17:44:53.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:52 smithi144 ceph-mon[36095]: pgmap v544: 97 pgs: 97 active+clean; 25 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:44:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:52 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-03-26T17:44:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:52 smithi105 ceph-mon[32280]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-03-26T17:44:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:52 smithi105 ceph-mon[32280]: pgmap v544: 97 pgs: 97 active+clean; 25 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:44:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:52 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-03-26T17:44:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:52 smithi135 ceph-mon[35636]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-03-26T17:44:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:52 smithi135 ceph-mon[35636]: pgmap v544: 97 pgs: 97 active+clean; 25 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:44:54.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 198455 3.15 MB/sec execute 217 sec latency 18.423 ms 2024-03-26T17:44:55.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 199020 3.16 MB/sec execute 218 sec latency 17.186 ms 2024-03-26T17:44:56.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 199474 3.15 MB/sec execute 219 sec latency 17.653 ms 2024-03-26T17:44:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:55 smithi144 ceph-mon[36095]: pgmap v545: 97 pgs: 97 active+clean; 25 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:44:56.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:55 smithi105 ceph-mon[32280]: pgmap v545: 97 pgs: 97 active+clean; 25 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:44:56.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:55 smithi135 ceph-mon[35636]: pgmap v545: 97 pgs: 97 active+clean; 25 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:44:57.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 200127 3.15 MB/sec execute 220 sec latency 21.202 ms 2024-03-26T17:44:58.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 200783 3.14 MB/sec execute 221 sec latency 17.182 ms 2024-03-26T17:44:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:57 smithi144 ceph-mon[36095]: pgmap v546: 97 pgs: 97 active+clean; 26 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-03-26T17:44:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:57 smithi105 ceph-mon[32280]: pgmap v546: 97 pgs: 97 active+clean; 26 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-03-26T17:44:58.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:57 smithi135 ceph-mon[35636]: pgmap v546: 97 pgs: 97 active+clean; 26 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 1.3 MiB/s wr, 126 op/s 2024-03-26T17:44:59.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 201410 3.15 MB/sec execute 222 sec latency 17.280 ms 2024-03-26T17:45:00.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 201973 3.15 MB/sec execute 223 sec latency 16.156 ms 2024-03-26T17:45:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:44:59 smithi144 ceph-mon[36095]: pgmap v547: 97 pgs: 97 active+clean; 27 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:45:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:44:59 smithi105 ceph-mon[32280]: pgmap v547: 97 pgs: 97 active+clean; 27 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:45:00.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:44:59 smithi135 ceph-mon[35636]: pgmap v547: 97 pgs: 97 active+clean; 27 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:45:01.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 202559 3.16 MB/sec execute 224 sec latency 15.520 ms 2024-03-26T17:45:01.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:45:01.581 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.706+0000 7ff1b6d40640 1 -- 172.21.15.105:0/2813644172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b0104c80 msgr2=0x7ff1b0107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.706+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/2813644172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b0104c80 0x7ff1b0107070 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009900 tx=0x7ff19c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.706+0000 7ff1b6d40640 1 -- 172.21.15.105:0/2813644172 shutdown_connections 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.706+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/2813644172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b01075b0 0x7ff1b0109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.706+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/2813644172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b0104c80 0x7ff1b0107070 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.706+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/2813644172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b0102350 0x7ff1b0104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.706+0000 7ff1b6d40640 1 -- 172.21.15.105:0/2813644172 >> 172.21.15.105:0/2813644172 conn(0x7ff1b00fbd60 msgr2=0x7ff1b00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.707+0000 7ff1b6d40640 1 -- 172.21.15.105:0/2813644172 shutdown_connections 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.707+0000 7ff1b6d40640 1 -- 172.21.15.105:0/2813644172 wait complete. 2024-03-26T17:45:01.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.707+0000 7ff1b6d40640 1 Processor -- start 2024-03-26T17:45:01.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.707+0000 7ff1b6d40640 1 -- start start 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b6d40640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b0102350 0x7ff1b0197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b6d40640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b0104c80 0x7ff1b01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b6d40640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b01075b0 0x7ff1b019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b6d40640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b010d3f0 con 0x7ff1b01075b0 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b6d40640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b010d270 con 0x7ff1b0102350 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b6d40640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b010d570 con 0x7ff1b0104c80 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b52dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b01075b0 0x7ff1b019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b4adc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b0102350 0x7ff1b0197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b4adc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b0102350 0x7ff1b0197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58918/0 (socket says 172.21.15.105:58918) 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b52dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b01075b0 0x7ff1b019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45120/0 (socket says 172.21.15.105:45120) 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b4adc640 1 -- 172.21.15.105:0/3371945048 learned_addr learned my addr 172.21.15.105:0/3371945048 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:01.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1affff640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b0104c80 0x7ff1b01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b52dd640 1 -- 172.21.15.105:0/3371945048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b0104c80 msgr2=0x7ff1b01980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b0104c80 0x7ff1b01980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b52dd640 1 -- 172.21.15.105:0/3371945048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b0102350 msgr2=0x7ff1b0197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b0102350 0x7ff1b0197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b52dd640 1 -- 172.21.15.105:0/3371945048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b019f780 con 0x7ff1b01075b0 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b0102350 0x7ff1b0197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b01075b0 0x7ff1b019f110 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009fd0 tx=0x7ff1a800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.708+0000 7ff1adffb640 1 -- 172.21.15.105:0/3371945048 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8004b20 con 0x7ff1b01075b0 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.709+0000 7ff1adffb640 1 -- 172.21.15.105:0/3371945048 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a8004cc0 con 0x7ff1b01075b0 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.709+0000 7ff1b6d40640 1 -- 172.21.15.105:0/3371945048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b01001a0 con 0x7ff1b01075b0 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.709+0000 7ff1adffb640 1 -- 172.21.15.105:0/3371945048 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8005410 con 0x7ff1b01075b0 2024-03-26T17:45:01.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.709+0000 7ff1b6d40640 1 -- 172.21.15.105:0/3371945048 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b01006c0 con 0x7ff1b01075b0 2024-03-26T17:45:01.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.711+0000 7ff1adffb640 1 -- 172.21.15.105:0/3371945048 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff1a8005900 con 0x7ff1b01075b0 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b99fd740 1 Processor -- start 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1adffb640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1a00a2fa0 0x7ff1a00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b99fd740 1 -- start start 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b74d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b752e0 0x563ba7b79de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1affff640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1a00a2fa0 0x7ff1a00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563ba7ad9980 con 0x563ba7b752e0 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563ba7a81be0 con 0x563ba7b748f0 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563ba7ad5930 con 0x563ba7b7a450 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1adffb640 1 -- 172.21.15.105:0/3371945048 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff1a80d34c0 con 0x7ff1b01075b0 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b752e0 0x563ba7b79de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b752e0 0x563ba7b79de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45132/0 (socket says 172.21.15.105:45132) 2024-03-26T17:45:01.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1affff640 1 -- 172.21.15.105:0/46213996 learned_addr learned my addr 172.21.15.105:0/46213996 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b74d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b7c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1affff640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1a00a2fa0 0x7ff1a00a5460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009230 tx=0x7ff19c02cb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b4adc640 1 -- 172.21.15.105:0/46213996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 msgr2=0x563ba7b7c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b7c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b4adc640 1 -- 172.21.15.105:0/46213996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b752e0 msgr2=0x563ba7b79de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b752e0 0x563ba7b79de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b4adc640 1 -- 172.21.15.105:0/46213996 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ba78b77d0 con 0x563ba7b748f0 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1affff640 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b752e0 0x563ba7b79de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.712+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.713+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b74d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40114f0 tx=0x7ff1a40119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a5839d1e32b172 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.713+0000 7ff19affd640 1 -- 172.21.15.105:0/46213996 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401e070 con 0x563ba7b748f0 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.713+0000 7ff19affd640 1 -- 172.21.15.105:0/46213996 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a40035f0 con 0x563ba7b748f0 2024-03-26T17:45:01.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.713+0000 7ff19affd640 1 -- 172.21.15.105:0/46213996 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400f760 con 0x563ba7b748f0 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 -- 172.21.15.105:0/46213996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 msgr2=0x563ba7b74d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b74d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40114f0 tx=0x7ff1a40119b0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 -- 172.21.15.105:0/46213996 shutdown_connections 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b752e0 0x563ba7b79de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/46213996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b74d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 -- 172.21.15.105:0/46213996 >> 172.21.15.105:0/46213996 conn(0x563ba7b6fe60 msgr2=0x563ba7b70260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 -- 172.21.15.105:0/46213996 shutdown_connections 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 -- 172.21.15.105:0/46213996 wait complete. 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 -- start start 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 Processor -- start 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b748f0 0x563ba7b7eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b7f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.714+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563ba7b7d400 con 0x563ba7b748f0 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563ba7a95160 con 0x563ba7b752e0 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563ba7b7d580 con 0x563ba7b7a450 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b99fd740 10 client.0 _ll_register_callbacks cb 0x563ba7ad1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b748f0 0x563ba7b7eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b748f0 0x563ba7b7eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45142/0 (socket says 172.21.15.105:45142) 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 -- 172.21.15.105:0/3348943561 learned_addr learned my addr 172.21.15.105:0/3348943561 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1affff640 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b7f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 -- 172.21.15.105:0/3348943561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 msgr2=0x563ba7b86080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b86080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 -- 172.21.15.105:0/3348943561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 msgr2=0x563ba7b7f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b7f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 -- 172.21.15.105:0/3348943561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ba77ed3f0 con 0x563ba7b748f0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b748f0 0x563ba7b7eb00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4005e00 tx=0x7ff1a4003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c96378441d54930 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff18ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff18ffff640 1 -- 172.21.15.105:0/3348943561 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401e030 con 0x563ba7b748f0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff18ffff640 1 -- 172.21.15.105:0/3348943561 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff1a4005260 con 0x563ba7b748f0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b99fd740 1 -- 172.21.15.105:0/3348943561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a40035f0 con 0x563ba7b748f0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff18ffff640 1 -- 172.21.15.105:0/3348943561 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff1a4005460 con 0x563ba7b748f0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b99fd740 10 client.17805 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.715+0000 7ff1b99fd740 1 -- 172.21.15.105:0/3348943561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563ba77ad0f0 con 0x563ba7b748f0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff18ffff640 1 -- 172.21.15.105:0/3348943561 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a40035f0 con 0x563ba7b748f0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff18ffff640 1 -- 172.21.15.105:0/3348943561 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff1a400f560 con 0x563ba7b748f0 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff1b99fd740 10 client.17805 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff1b99fd740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff1b99fd740 20 client.17805 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563ba7b8a730 0x563ba7b8cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff1b99fd740 4 client.17805 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:01.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff1b99fd740 1 -- 172.21.15.105:0/3348943561 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ba7ae6ec0 con 0x563ba7b8a730 2024-03-26T17:45:01.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff1affff640 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563ba7b8a730 0x563ba7b8cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff18ffff640 10 client.17805 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:01.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.716+0000 7ff1affff640 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563ba7b8a730 0x563ba7b8cb80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b9e908a8001457 server_cookie=bbf8bd019ea17695 in_seq=0 out_seq=0 2024-03-26T17:45:01.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.718+0000 7ff18ffff640 1 -- 172.21.15.105:0/3348943561 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563ba7ae6ec0 con 0x563ba7b8a730 2024-03-26T17:45:01.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.718+0000 7ff18ffff640 10 client.17805 handle_command_reply: tid=0 2024-03-26T17:45:01.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.718+0000 7ff1b99fd740 1 client.17805 shutdown 2024-03-26T17:45:01.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 -- 172.21.15.105:0/3348943561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b748f0 msgr2=0x563ba7b7eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b748f0 0x563ba7b7eb00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4005e00 tx=0x7ff1a4003a50 comp rx=0 tx=0).stop 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 -- 172.21.15.105:0/3348943561 shutdown_connections 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563ba7b8a730 0x563ba7b8cb80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b7a450 0x563ba7b86080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b7f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/3348943561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b748f0 0x563ba7b7eb00 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 -- 172.21.15.105:0/3348943561 >> 172.21.15.105:0/3348943561 conn(0x563ba7b6fe60 msgr2=0x563ba7b6f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff18ffff640 0 client.17805 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 -- 172.21.15.105:0/3348943561 shutdown_connections 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 1 -- 172.21.15.105:0/3348943561 wait complete. 2024-03-26T17:45:01.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.719+0000 7ff1b99fd740 20 client.17805 trim_cache size 0 max 16384 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b99fd740 1 Processor -- start 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b99fd740 1 -- start start 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b8c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b752e0 0x563ba7b86040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7b91440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a6be0 con 0x563ba7b7a450 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a8720 con 0x563ba7b748f0 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a72b0 con 0x563ba7b752e0 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b52dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7b91440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b752e0 0x563ba7b86040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b4adc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b8c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b752e0 0x563ba7b86040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42186/0 (socket says 172.21.15.105:42186) 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b52dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7b91440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45144/0 (socket says 172.21.15.105:45144) 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1affff640 1 -- 172.21.15.105:0/1966171702 learned_addr learned my addr 172.21.15.105:0/1966171702 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b52dd640 1 -- 172.21.15.105:0/1966171702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b752e0 msgr2=0x563ba7b86040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/1966171702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b752e0 0x563ba7b86040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b52dd640 1 -- 172.21.15.105:0/1966171702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 msgr2=0x563ba7b8c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/1966171702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b8c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1b52dd640 1 -- 172.21.15.105:0/1966171702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ba7b75fb0 con 0x563ba7b7a450 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.741+0000 7ff1affff640 1 --2- 172.21.15.105:0/1966171702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b752e0 0x563ba7b86040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.742+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/1966171702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7b91440 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80d12f0 tx=0x7ff1a80d3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b51864b9d4c20d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.742+0000 7ff19a7fc640 1 -- 172.21.15.105:0/1966171702 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80a3d00 con 0x563ba7b7a450 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.742+0000 7ff19a7fc640 1 -- 172.21.15.105:0/1966171702 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a80a85c0 con 0x563ba7b7a450 2024-03-26T17:45:01.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.742+0000 7ff19a7fc640 1 -- 172.21.15.105:0/1966171702 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80a83c0 con 0x563ba7b7a450 2024-03-26T17:45:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 -- 172.21.15.105:0/1966171702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 msgr2=0x563ba7b91440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/1966171702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7b91440 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80d12f0 tx=0x7ff1a80d3040 comp rx=0 tx=0).stop 2024-03-26T17:45:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 -- 172.21.15.105:0/1966171702 shutdown_connections 2024-03-26T17:45:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/1966171702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7b91440 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/1966171702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b752e0 0x563ba7b86040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/1966171702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b748f0 0x563ba7b8c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 -- 172.21.15.105:0/1966171702 >> 172.21.15.105:0/1966171702 conn(0x563ba7b6fe60 msgr2=0x563ba7b73950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:01.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 -- 172.21.15.105:0/1966171702 shutdown_connections 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 -- 172.21.15.105:0/1966171702 wait complete. 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 -- start start 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.743+0000 7ff1b99fd740 1 Processor -- start 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b748f0 0x563ba7b82ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b83420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b99fd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7beea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a8050 con 0x563ba7b7a450 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a7980 con 0x563ba7b752e0 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b99fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b01a4bc0 con 0x563ba7b748f0 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b99fd740 10 client.0 _ll_register_callbacks cb 0x7ff1a401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b4adc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b748f0 0x563ba7b82ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b83420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b83420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58964/0 (socket says 172.21.15.105:58964) 2024-03-26T17:45:01.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 -- 172.21.15.105:0/57012999 learned_addr learned my addr 172.21.15.105:0/57012999 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b4adc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b748f0 0x563ba7b82ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42192/0 (socket says 172.21.15.105:42192) 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1b52dd640 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7beea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 -- 172.21.15.105:0/57012999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b748f0 msgr2=0x563ba7b82ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b748f0 0x563ba7b82ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 -- 172.21.15.105:0/57012999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 msgr2=0x563ba7beea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7beea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 -- 172.21.15.105:0/57012999 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ba7beefb0 con 0x563ba7b752e0 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff1affff640 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b83420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006a990 tx=0x7ff19c045070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a65fe7792b47ad6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.744+0000 7ff18ffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff18ffff640 1 -- 172.21.15.105:0/57012999 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c05a070 con 0x563ba7b752e0 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 1 -- 172.21.15.105:0/57012999 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a80a85c0 con 0x563ba7b752e0 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff18ffff640 1 -- 172.21.15.105:0/57012999 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff19c005250 con 0x563ba7b752e0 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff18ffff640 1 -- 172.21.15.105:0/57012999 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff19c049070 con 0x563ba7b752e0 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 10 client.27641 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:01.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 1 -- 172.21.15.105:0/57012999 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563ba7bf0320 con 0x563ba7b752e0 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff18ffff640 1 -- 172.21.15.105:0/57012999 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1a80a85c0 con 0x563ba7b752e0 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff18ffff640 1 -- 172.21.15.105:0/57012999 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff19c040040 con 0x563ba7b752e0 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 10 client.27641 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 10 client.27641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 20 client.27641 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563ba7bf0cb0 0x563ba7bf30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 4 client.27641 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b99fd740 1 -- 172.21.15.105:0/57012999 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ba7b8cf10 con 0x563ba7bf0cb0 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.745+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563ba7bf0cb0 0x563ba7bf30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.746+0000 7ff18ffff640 10 client.27641 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.746+0000 7ff1b4adc640 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563ba7bf0cb0 0x563ba7bf30d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75af4f2e3a336fe7 server_cookie=222a8b4da595933f in_seq=0 out_seq=0 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff18ffff640 1 -- 172.21.15.105:0/57012999 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563ba7b8cf10 con 0x563ba7bf0cb0 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff18ffff640 10 client.27641 handle_command_reply: tid=0 2024-03-26T17:45:01.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 client.27641 shutdown 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 -- 172.21.15.105:0/57012999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 msgr2=0x563ba7b83420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b83420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006a990 tx=0x7ff19c045070 comp rx=0 tx=0).stop 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 -- 172.21.15.105:0/57012999 shutdown_connections 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563ba7bf0cb0 0x563ba7bf30d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563ba7b7a450 0x563ba7beea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff18ffff640 0 client.27641 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ba7b752e0 0x563ba7b83420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 --2- 172.21.15.105:0/57012999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ba7b748f0 0x563ba7b82ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 -- 172.21.15.105:0/57012999 >> 172.21.15.105:0/57012999 conn(0x563ba7b6fe60 msgr2=0x563ba7b6e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 -- 172.21.15.105:0/57012999 shutdown_connections 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 1 -- 172.21.15.105:0/57012999 wait complete. 2024-03-26T17:45:01.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.747+0000 7ff1b99fd740 20 client.27641 trim_cache size 0 max 16384 2024-03-26T17:45:01.745 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:45:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:45:01.745 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:45:01.745 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 -- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1a00a2fa0 msgr2=0x7ff1a00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1a00a2fa0 0x7ff1a00a5460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009230 tx=0x7ff19c02cb70 comp rx=0 tx=0).stop 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 -- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b01075b0 msgr2=0x7ff1b019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b01075b0 0x7ff1b019f110 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009fd0 tx=0x7ff1a800b690 comp rx=0 tx=0).stop 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 -- 172.21.15.105:0/3371945048 shutdown_connections 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1a00a2fa0 0x7ff1a00a5460 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff1b01075b0 0x7ff1b019f110 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b0104c80 0x7ff1b01980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 --2- 172.21.15.105:0/3371945048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff1b0102350 0x7ff1b0197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 -- 172.21.15.105:0/3371945048 >> 172.21.15.105:0/3371945048 conn(0x7ff1b00fbd60 msgr2=0x7ff1b01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 -- 172.21.15.105:0/3371945048 shutdown_connections 2024-03-26T17:45:01.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.748+0000 7ff1b6d40640 1 -- 172.21.15.105:0/3371945048 wait complete. 2024-03-26T17:45:01.752 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:45:01.752 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:45:01.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 -- 172.21.15.105:0/3120089737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d81001b0 msgr2=0x7fd0d81005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 --2- 172.21.15.105:0/3120089737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d81001b0 0x7fd0d81005d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009900 tx=0x7fd0d402bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:01.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 -- 172.21.15.105:0/3120089737 shutdown_connections 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 --2- 172.21.15.105:0/3120089737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d81014b0 0x7fd0d8101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 --2- 172.21.15.105:0/3120089737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d8100b10 0x7fd0d8100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 --2- 172.21.15.105:0/3120089737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d81001b0 0x7fd0d81005d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 -- 172.21.15.105:0/3120089737 >> 172.21.15.105:0/3120089737 conn(0x7fd0d80fbd50 msgr2=0x7fd0d80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 -- 172.21.15.105:0/3120089737 shutdown_connections 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.877+0000 7fd0e020b640 1 -- 172.21.15.105:0/3120089737 wait complete. 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0e020b640 1 Processor -- start 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0e020b640 1 -- start start 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0e020b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d81001b0 0x7fd0d8199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0e020b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d8100b10 0x7fd0d819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0e020b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d81014b0 0x7fd0d81a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0e020b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d810f5d0 con 0x7fd0d8100b10 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0e020b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d810f450 con 0x7fd0d81014b0 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0e020b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d810f750 con 0x7fd0d81001b0 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0dd7a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d8100b10 0x7fd0d819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0dd7a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d8100b10 0x7fd0d819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45172/0 (socket says 172.21.15.105:45172) 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0dd7a6640 1 -- 172.21.15.105:0/2010125480 learned_addr learned my addr 172.21.15.105:0/2010125480 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0ddfa7640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d81001b0 0x7fd0d8199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.878+0000 7fd0de7a8640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d81014b0 0x7fd0d81a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0de7a8640 1 -- 172.21.15.105:0/2010125480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d81001b0 msgr2=0x7fd0d8199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0de7a8640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d81001b0 0x7fd0d8199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0de7a8640 1 -- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d8100b10 msgr2=0x7fd0d819a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0de7a8640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d8100b10 0x7fd0d819a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0de7a8640 1 -- 172.21.15.105:0/2010125480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d81a1900 con 0x7fd0d81014b0 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0dd7a6640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d8100b10 0x7fd0d819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:01.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0de7a8640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d81014b0 0x7fd0d81a1260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009fd0 tx=0x7fd0d000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:01.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0ceffd640 1 -- 172.21.15.105:0/2010125480 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000ce20 con 0x7fd0d81014b0 2024-03-26T17:45:01.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d81a1bf0 con 0x7fd0d81014b0 2024-03-26T17:45:01.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0ceffd640 1 -- 172.21.15.105:0/2010125480 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0d0005020 con 0x7fd0d81014b0 2024-03-26T17:45:01.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0ceffd640 1 -- 172.21.15.105:0/2010125480 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0010690 con 0x7fd0d81014b0 2024-03-26T17:45:01.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.879+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d81a2140 con 0x7fd0d81014b0 2024-03-26T17:45:01.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.880+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d810b130 con 0x7fd0d81014b0 2024-03-26T17:45:01.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.881+0000 7fd0ceffd640 1 -- 172.21.15.105:0/2010125480 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd0d00051d0 con 0x7fd0d81014b0 2024-03-26T17:45:01.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.882+0000 7fd0ceffd640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c40a2fa0 0x7fd0c40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:01.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.882+0000 7fd0ddfa7640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c40a2fa0 0x7fd0c40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:01.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.882+0000 7fd0ceffd640 1 -- 172.21.15.105:0/2010125480 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd0d00cfb50 con 0x7fd0d81014b0 2024-03-26T17:45:01.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.882+0000 7fd0ddfa7640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c40a2fa0 0x7fd0c40a5460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd0d819b1c0 tx=0x7fd0d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:01.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:01.888+0000 7fd0ceffd640 1 -- 172.21.15.105:0/2010125480 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd0d009c8e0 con 0x7fd0d81014b0 2024-03-26T17:45:01.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:01 smithi105 ceph-mon[32280]: pgmap v548: 97 pgs: 97 active+clean; 26 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:45:02.025 INFO:tasks.workunit.client.0.smithi105.stdout: 1 203010 3.15 MB/sec execute 225 sec latency 20.195 ms 2024-03-26T17:45:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:01 smithi144 ceph-mon[36095]: pgmap v548: 97 pgs: 97 active+clean; 26 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:45:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.157+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0d80624c0 con 0x7fd0d81014b0 2024-03-26T17:45:02.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.158+0000 7fd0ceffd640 1 -- 172.21.15.105:0/2010125480 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fd0d009c1e0 con 0x7fd0d81014b0 2024-03-26T17:45:02.149 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:45:02.150 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:45:02.150 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c40a2fa0 msgr2=0x7fd0c40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c40a2fa0 0x7fd0c40a5460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd0d819b1c0 tx=0x7fd0d4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d81014b0 msgr2=0x7fd0d81a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d81014b0 0x7fd0d81a1260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009fd0 tx=0x7fd0d000ef10 comp rx=0 tx=0).stop 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 shutdown_connections 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c40a2fa0 0x7fd0c40a5460 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d81014b0 0x7fd0d81a1260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d8100b10 0x7fd0d819a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 --2- 172.21.15.105:0/2010125480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d81001b0 0x7fd0d8199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 >> 172.21.15.105:0/2010125480 conn(0x7fd0d80fbd50 msgr2=0x7fd0d80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 shutdown_connections 2024-03-26T17:45:02.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.160+0000 7fd0e020b640 1 -- 172.21.15.105:0/2010125480 wait complete. 2024-03-26T17:45:02.163 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:45:02.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:01 smithi135 ceph-mon[35636]: pgmap v548: 97 pgs: 97 active+clean; 26 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 -- 172.21.15.105:0/3512944638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 msgr2=0x7fe220108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3512944638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 0x7fe220108c00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe218009be0 tx=0x7fe21802bef0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 -- 172.21.15.105:0/3512944638 shutdown_connections 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3512944638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 0x7fe220108c00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3512944638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201019b0 0x7fe220101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3512944638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2201007c0 0x7fe220100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 -- 172.21.15.105:0/3512944638 >> 172.21.15.105:0/3512944638 conn(0x7fe2200fbd30 msgr2=0x7fe2200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 -- 172.21.15.105:0/3512944638 shutdown_connections 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.287+0000 7fe2262c6640 1 -- 172.21.15.105:0/3512944638 wait complete. 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.288+0000 7fe2262c6640 1 Processor -- start 2024-03-26T17:45:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.288+0000 7fe2262c6640 1 -- start start 2024-03-26T17:45:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.288+0000 7fe2262c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2201007c0 0x7fe220197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.288+0000 7fe2262c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201019b0 0x7fe2201980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.288+0000 7fe2262c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 0x7fe22019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.288+0000 7fe2262c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010c2f0 con 0x7fe220102370 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.288+0000 7fe2262c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010c170 con 0x7fe2201007c0 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.288+0000 7fe2262c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010c470 con 0x7fe2201019b0 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 0x7fe22019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 0x7fe22019f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45200/0 (socket says 172.21.15.105:45200) 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 -- 172.21.15.105:0/3384318298 learned_addr learned my addr 172.21.15.105:0/3384318298 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201019b0 0x7fe2201980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe21ffff640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2201007c0 0x7fe220197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 -- 172.21.15.105:0/3384318298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201019b0 msgr2=0x7fe2201980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201019b0 0x7fe2201980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 -- 172.21.15.105:0/3384318298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2201007c0 msgr2=0x7fe220197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2201007c0 0x7fe220197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 -- 172.21.15.105:0/3384318298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22019f730 con 0x7fe220102370 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe224863640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 0x7fe22019f0c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe218004af0 tx=0x7fe218033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe21d7fa640 1 -- 172.21.15.105:0/3384318298 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218043020 con 0x7fe220102370 2024-03-26T17:45:02.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe21d7fa640 1 -- 172.21.15.105:0/3384318298 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe21802ca60 con 0x7fe220102370 2024-03-26T17:45:02.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe2262c6640 1 -- 172.21.15.105:0/3384318298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22019f9c0 con 0x7fe220102370 2024-03-26T17:45:02.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe21d7fa640 1 -- 172.21.15.105:0/3384318298 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218034430 con 0x7fe220102370 2024-03-26T17:45:02.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.289+0000 7fe2262c6640 1 -- 172.21.15.105:0/3384318298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22019fec0 con 0x7fe220102370 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.292+0000 7fe21d7fa640 1 -- 172.21.15.105:0/3384318298 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe218046020 con 0x7fe220102370 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.292+0000 7fe228f8c740 1 Processor -- start 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.292+0000 7fe228f8c740 1 -- start start 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21d7fa640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2080a2f20 0x7fe2080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491992a30 0x561491992e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 0x561491997f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491998590 0x56149199aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe228f8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5614918f7970 con 0x561491992a30 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe228f8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56149189fd70 con 0x561491998590 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe228f8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614918f3920 con 0x561491993420 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491992a30 0x561491992e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491992a30 0x561491992e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45214/0 (socket says 172.21.15.105:45214) 2024-03-26T17:45:02.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 -- 172.21.15.105:0/8808601 learned_addr learned my addr 172.21.15.105:0/8808601 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21d7fa640 1 -- 172.21.15.105:0/3384318298 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe2180f1b40 con 0x7fe220102370 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe224863640 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491998590 0x56149199aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2080a2f20 0x7fe2080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 0x561491997f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 -- 172.21.15.105:0/8808601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 msgr2=0x561491997f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 0x561491997f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 -- 172.21.15.105:0/8808601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491998590 msgr2=0x56149199aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491998590 0x56149199aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 -- 172.21.15.105:0/8808601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56149160b3f0 con 0x561491992a30 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe224863640 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491998590 0x56149199aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.293+0000 7fe21ffff640 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491992a30 0x561491992e50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe210009c20 tx=0x7fe2100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0b86a4e141d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.294+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2080a2f20 0x7fe2080a53e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x561491993820 tx=0x7fe214009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.294+0000 7fe2067fc640 1 -- 172.21.15.105:0/8808601 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210009e50 con 0x561491992a30 2024-03-26T17:45:02.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.294+0000 7fe2067fc640 1 -- 172.21.15.105:0/8808601 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe210012900 con 0x561491992a30 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.294+0000 7fe2067fc640 1 -- 172.21.15.105:0/8808601 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000f900 con 0x561491992a30 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.294+0000 7fe228f8c740 1 -- 172.21.15.105:0/8808601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491992a30 msgr2=0x561491992e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.294+0000 7fe228f8c740 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491992a30 0x561491992e50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe210009c20 tx=0x7fe2100117b0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 -- 172.21.15.105:0/8808601 shutdown_connections 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491998590 0x56149199aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 0x561491997f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 --2- 172.21.15.105:0/8808601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491992a30 0x561491992e50 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 -- 172.21.15.105:0/8808601 >> 172.21.15.105:0/8808601 conn(0x56149198dfa0 msgr2=0x56149198e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 -- 172.21.15.105:0/8808601 shutdown_connections 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 -- 172.21.15.105:0/8808601 wait complete. 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 -- start start 2024-03-26T17:45:02.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 Processor -- start 2024-03-26T17:45:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.295+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x56149199d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x5614919a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe228f8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56149199b540 con 0x561491993420 2024-03-26T17:45:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe228f8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5614918b3310 con 0x561491992a30 2024-03-26T17:45:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe228f8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56149199b6c0 con 0x561491998590 2024-03-26T17:45:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe228f8c740 10 client.0 _ll_register_callbacks cb 0x5614918eec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59028/0 (socket says 172.21.15.105:59028) 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 -- 172.21.15.105:0/765362926 learned_addr learned my addr 172.21.15.105:0/765362926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe224863640 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x5614919a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x56149199d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 -- 172.21.15.105:0/765362926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 msgr2=0x5614919a44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x5614919a44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 -- 172.21.15.105:0/765362926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 msgr2=0x56149199d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x56149199d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 -- 172.21.15.105:0/765362926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614916d57d0 con 0x561491992a30 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe21ffff640 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199cf20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe21001c040 tx=0x7fe210012840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20120ad980c51391 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe1ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:02.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/765362926 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210028020 con 0x561491992a30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe228f8c740 1 -- 172.21.15.105:0/765362926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe210012900 con 0x561491992a30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.296+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/765362926 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe210005250 con 0x561491992a30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/765362926 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe210033050 con 0x561491992a30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe228f8c740 10 client.27659 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe228f8c740 1 -- 172.21.15.105:0/765362926 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5614919a7810 con 0x561491992a30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/765362926 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe210012900 con 0x561491992a30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/765362926 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe21000f680 con 0x561491992a30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe228f8c740 10 client.27659 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe228f8c740 10 client.27659 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe228f8c740 20 client.27659 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe228f8c740 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5614919a8c70 0x5614919ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe228f8c740 4 client.27659 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe228f8c740 1 -- 172.21.15.105:0/765362926 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561491904eb0 con 0x5614919a8c70 2024-03-26T17:45:02.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.297+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5614919a8c70 0x5614919ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.298+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5614919a8c70 0x5614919ab0c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b74cdf29529bf3 server_cookie=361f694ab7f68b7f in_seq=0 out_seq=0 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.298+0000 7fe1ff7fe640 10 client.27659 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.298+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/765362926 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561491904eb0 con 0x5614919a8c70 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.298+0000 7fe1ff7fe640 10 client.27659 handle_command_reply: tid=0 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.298+0000 7fe228f8c740 1 client.27659 shutdown 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.298+0000 7fe228f8c740 1 -- 172.21.15.105:0/765362926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 msgr2=0x56149199cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199cf20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe21001c040 tx=0x7fe210012840 comp rx=0 tx=0).stop 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 -- 172.21.15.105:0/765362926 shutdown_connections 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5614919a8c70 0x5614919ab0c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x5614919a44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x56149199d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 --2- 172.21.15.105:0/765362926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199cf20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 -- 172.21.15.105:0/765362926 >> 172.21.15.105:0/765362926 conn(0x56149198dfa0 msgr2=0x56149198d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe1ff7fe640 0 client.27659 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:45:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 -- 172.21.15.105:0/765362926 shutdown_connections 2024-03-26T17:45:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 1 -- 172.21.15.105:0/765362926 wait complete. 2024-03-26T17:45:02.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.299+0000 7fe228f8c740 20 client.27659 trim_cache size 0 max 16384 2024-03-26T17:45:02.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.316+0000 7fe228f8c740 1 Processor -- start 2024-03-26T17:45:02.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.316+0000 7fe228f8c740 1 -- start start 2024-03-26T17:45:02.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x5614919a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x56149199c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe228f8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201a8e20 con 0x561491993420 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe228f8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201aa960 con 0x561491992a30 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe228f8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201a94f0 con 0x561491998590 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x5614919a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x5614919a43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45222/0 (socket says 172.21.15.105:45222) 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 -- 172.21.15.105:0/63452574 learned_addr learned my addr 172.21.15.105:0/63452574 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe224863640 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x56149199c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 -- 172.21.15.105:0/63452574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 msgr2=0x56149199c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x56149199c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 -- 172.21.15.105:0/63452574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 msgr2=0x56149199bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 -- 172.21.15.105:0/63452574 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56149198ed10 con 0x561491993420 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe224863640 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x56149199c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x5614919a43c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe214005fd0 tx=0x7fe214013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb3392f7814a3f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.317+0000 7fe205ffb640 1 -- 172.21.15.105:0/63452574 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21402d190 con 0x561491993420 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe205ffb640 1 -- 172.21.15.105:0/63452574 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe214031030 con 0x561491993420 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe205ffb640 1 -- 172.21.15.105:0/63452574 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21401dd20 con 0x561491993420 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 -- 172.21.15.105:0/63452574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 msgr2=0x5614919a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x5614919a43c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe214005fd0 tx=0x7fe214013070 comp rx=0 tx=0).stop 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 -- 172.21.15.105:0/63452574 shutdown_connections 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491998590 0x56149199c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491993420 0x5614919a43c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 --2- 172.21.15.105:0/63452574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x56149199bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 -- 172.21.15.105:0/63452574 >> 172.21.15.105:0/63452574 conn(0x56149198dfa0 msgr2=0x561491991a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 -- 172.21.15.105:0/63452574 shutdown_connections 2024-03-26T17:45:02.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.318+0000 7fe228f8c740 1 -- 172.21.15.105:0/63452574 wait complete. 2024-03-26T17:45:02.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 1 -- start start 2024-03-26T17:45:02.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 1 Processor -- start 2024-03-26T17:45:02.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x5614919a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 0x5614919a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491998590 0x5614919a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201aa290 con 0x561491998590 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201a9bc0 con 0x561491992a30 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201a6e00 con 0x561491993420 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe228f8c740 10 client.0 _ll_register_callbacks cb 0x7fe210009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe224863640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491998590 0x5614919a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe21ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x5614919a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe224863640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491998590 0x5614919a2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45234/0 (socket says 172.21.15.105:45234) 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe224863640 1 -- 172.21.15.105:0/1859224484 learned_addr learned my addr 172.21.15.105:0/1859224484 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe21ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x5614919a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59046/0 (socket says 172.21.15.105:59046) 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe21f7fe640 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 0x5614919a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe224863640 1 -- 172.21.15.105:0/1859224484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 msgr2=0x5614919a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe224863640 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 0x5614919a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe224863640 1 -- 172.21.15.105:0/1859224484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 msgr2=0x5614919a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe224863640 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x5614919a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.319+0000 7fe224863640 1 -- 172.21.15.105:0/1859224484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614919a3500 con 0x561491998590 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe224863640 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491998590 0x5614919a2f10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe2180be1c0 tx=0x7fe2180be110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d82b102a85d7cdb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe1ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/1859224484 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180be930 con 0x561491998590 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/1859224484 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe218003ca0 con 0x561491998590 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe228f8c740 1 -- 172.21.15.105:0/1859224484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe214031030 con 0x561491998590 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/1859224484 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe218093230 con 0x561491998590 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe228f8c740 10 client.17835 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe228f8c740 1 -- 172.21.15.105:0/1859224484 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5614919a4870 con 0x561491998590 2024-03-26T17:45:02.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/1859224484 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe214031030 con 0x561491998590 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/1859224484 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe2180bd680 con 0x561491998590 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe228f8c740 10 client.17835 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe228f8c740 10 client.17835 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe228f8c740 20 client.17835 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.320+0000 7fe228f8c740 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561491a0f7c0 0x561491a11be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe228f8c740 4 client.17835 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe228f8c740 1 -- 172.21.15.105:0/1859224484 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614918ff200 con 0x561491a0f7c0 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe21ffff640 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561491a0f7c0 0x561491a11be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe1ff7fe640 10 client.17835 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe21ffff640 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561491a0f7c0 0x561491a11be0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fee7820aa67b0fc9 server_cookie=a9988a5e94e915f8 in_seq=0 out_seq=0 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe1ff7fe640 1 -- 172.21.15.105:0/1859224484 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614918ff200 con 0x561491a0f7c0 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe1ff7fe640 10 client.17835 handle_command_reply: tid=0 2024-03-26T17:45:02.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe228f8c740 1 client.17835 shutdown 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe228f8c740 1 -- 172.21.15.105:0/1859224484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491998590 msgr2=0x5614919a2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe228f8c740 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491998590 0x5614919a2f10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe2180be1c0 tx=0x7fe2180be110 comp rx=0 tx=0).stop 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe228f8c740 1 -- 172.21.15.105:0/1859224484 shutdown_connections 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.321+0000 7fe228f8c740 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561491a0f7c0 0x561491a11be0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe228f8c740 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561491998590 0x5614919a2f10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe228f8c740 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561491993420 0x5614919a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe228f8c740 1 --2- 172.21.15.105:0/1859224484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561491992a30 0x5614919a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe228f8c740 1 -- 172.21.15.105:0/1859224484 >> 172.21.15.105:0/1859224484 conn(0x56149198dfa0 msgr2=0x56149198c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe1ff7fe640 0 client.17835 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe228f8c740 1 -- 172.21.15.105:0/1859224484 shutdown_connections 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe228f8c740 1 -- 172.21.15.105:0/1859224484 wait complete. 2024-03-26T17:45:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe228f8c740 20 client.17835 trim_cache size 0 max 16384 2024-03-26T17:45:02.321 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:02.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 -- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2080a2f20 msgr2=0x7fe2080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2080a2f20 0x7fe2080a53e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x561491993820 tx=0x7fe214009600 comp rx=0 tx=0).stop 2024-03-26T17:45:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 -- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 msgr2=0x7fe22019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 0x7fe22019f0c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe218004af0 tx=0x7fe218033c90 comp rx=0 tx=0).stop 2024-03-26T17:45:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 -- 172.21.15.105:0/3384318298 shutdown_connections 2024-03-26T17:45:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2080a2f20 0x7fe2080a53e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe220102370 0x7fe22019f0c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2201019b0 0x7fe2201980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 --2- 172.21.15.105:0/3384318298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2201007c0 0x7fe220197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.322+0000 7fe2262c6640 1 -- 172.21.15.105:0/3384318298 >> 172.21.15.105:0/3384318298 conn(0x7fe2200fbd30 msgr2=0x7fe220069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.323+0000 7fe2262c6640 1 -- 172.21.15.105:0/3384318298 shutdown_connections 2024-03-26T17:45:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.323+0000 7fe2262c6640 1 -- 172.21.15.105:0/3384318298 wait complete. 2024-03-26T17:45:02.326 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:45:02.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 -- 172.21.15.105:0/2969507942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f649c1075a0 msgr2=0x7f649c1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 --2- 172.21.15.105:0/2969507942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f649c1075a0 0x7f649c1099f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6494009ca0 tx=0x7f649402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 -- 172.21.15.105:0/2969507942 shutdown_connections 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 --2- 172.21.15.105:0/2969507942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f649c1075a0 0x7f649c1099f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 --2- 172.21.15.105:0/2969507942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f649c104c70 0x7f649c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 --2- 172.21.15.105:0/2969507942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649c102340 0x7f649c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 -- 172.21.15.105:0/2969507942 >> 172.21.15.105:0/2969507942 conn(0x7f649c0fbd50 msgr2=0x7f649c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 -- 172.21.15.105:0/2969507942 shutdown_connections 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.455+0000 7f64a45a9640 1 -- 172.21.15.105:0/2969507942 wait complete. 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a45a9640 1 Processor -- start 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a45a9640 1 -- start start 2024-03-26T17:45:02.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a45a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f649c102340 0x7f649c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a45a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f649c104c70 0x7f649c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a45a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649c1075a0 0x7f649c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a45a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f649c10d3f0 con 0x7f649c104c70 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a45a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f649c10d270 con 0x7f649c102340 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a45a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f649c10d570 con 0x7f649c1075a0 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a2345640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f649c102340 0x7f649c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a1b44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f649c104c70 0x7f649c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a2345640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f649c102340 0x7f649c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59060/0 (socket says 172.21.15.105:59060) 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a2345640 1 -- 172.21.15.105:0/729348121 learned_addr learned my addr 172.21.15.105:0/729348121 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.456+0000 7f64a2b46640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649c1075a0 0x7f649c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64a2345640 1 -- 172.21.15.105:0/729348121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649c1075a0 msgr2=0x7f649c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64a2345640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649c1075a0 0x7f649c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64a2345640 1 -- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f649c104c70 msgr2=0x7f649c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64a2345640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f649c104c70 0x7f649c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64a2345640 1 -- 172.21.15.105:0/729348121 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c19f710 con 0x7f649c102340 2024-03-26T17:45:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64a2345640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f649c102340 0x7f649c197af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f649800e9f0 tx=0x7f649800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64937fe640 1 -- 172.21.15.105:0/729348121 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649800cd80 con 0x7f649c102340 2024-03-26T17:45:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64a45a9640 1 -- 172.21.15.105:0/729348121 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c19fa00 con 0x7f649c102340 2024-03-26T17:45:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64937fe640 1 -- 172.21.15.105:0/729348121 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64980108e0 con 0x7f649c102340 2024-03-26T17:45:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64937fe640 1 -- 172.21.15.105:0/729348121 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498018920 con 0x7f649c102340 2024-03-26T17:45:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.457+0000 7f64a45a9640 1 -- 172.21.15.105:0/729348121 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c19ffa0 con 0x7f649c102340 2024-03-26T17:45:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64937fe640 1 -- 172.21.15.105:0/729348121 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6498010a90 con 0x7f649c102340 2024-03-26T17:45:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a7191740 1 Processor -- start 2024-03-26T17:45:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a7191740 1 -- start start 2024-03-26T17:45:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a7191740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006076a30 0x556006076e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a7191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x55600607bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a7191740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x55600607ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a7191740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556005fdb970 con 0x55600607c590 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a7191740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556005f83d70 con 0x556006076a30 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a7191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556005fd7920 con 0x556006077420 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a2b46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006076a30 0x556006076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a2345640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x55600607bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a2b46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006076a30 0x556006076e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59074/0 (socket says 172.21.15.105:59074) 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a2b46640 1 -- 172.21.15.105:0/230315544 learned_addr learned my addr 172.21.15.105:0/230315544 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64a1b44640 1 --2- 172.21.15.105:0/230315544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x55600607ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.460+0000 7f64937fe640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64880a2f20 0x7f64880a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64a2345640 1 -- 172.21.15.105:0/230315544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006076a30 msgr2=0x556006076e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64a2345640 1 --2- 172.21.15.105:0/230315544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006076a30 0x556006076e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64a2345640 1 -- 172.21.15.105:0/230315544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 msgr2=0x55600607ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64a2345640 1 --2- 172.21.15.105:0/230315544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x55600607ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64a2345640 1 -- 172.21.15.105:0/230315544 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556005cef3f0 con 0x556006077420 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64a2b46640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64880a2f20 0x7f64880a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64937fe640 1 -- 172.21.15.105:0/729348121 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f64980cf700 con 0x7f649c102340 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64a2345640 1 --2- 172.21.15.105:0/230315544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x55600607bf20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f64980cd190 tx=0x7f6498066ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff24e0ec9fc6235c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f64a2b46640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64880a2f20 0x7f64880a53e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x556006077c80 tx=0x7f649402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f6490bfe640 1 -- 172.21.15.105:0/230315544 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649809b9e0 con 0x556006077420 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f6490bfe640 1 -- 172.21.15.105:0/230315544 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6498010de0 con 0x556006077420 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.461+0000 7f6490bfe640 1 -- 172.21.15.105:0/230315544 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649806e080 con 0x556006077420 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 -- 172.21.15.105:0/230315544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 msgr2=0x55600607bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 --2- 172.21.15.105:0/230315544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x55600607bf20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f64980cd190 tx=0x7f6498066ab0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 -- 172.21.15.105:0/230315544 shutdown_connections 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 --2- 172.21.15.105:0/230315544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x55600607ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 --2- 172.21.15.105:0/230315544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x55600607bf20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 --2- 172.21.15.105:0/230315544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006076a30 0x556006076e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 -- 172.21.15.105:0/230315544 >> 172.21.15.105:0/230315544 conn(0x556006071fa0 msgr2=0x5560060723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 -- 172.21.15.105:0/230315544 shutdown_connections 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.462+0000 7f64a7191740 1 -- 172.21.15.105:0/230315544 wait complete. 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 1 -- start start 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 1 Processor -- start 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006080e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55600607c590 0x5560060883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55600607f480 con 0x556006076a30 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556005f97310 con 0x55600607c590 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55600607f600 con 0x556006077420 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a7191740 10 client.0 _ll_register_callbacks cb 0x556005fd2c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a1b44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a2345640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006080e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a1b44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42298/0 (socket says 172.21.15.105:42298) 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a1b44640 1 -- 172.21.15.105:0/2737163620 learned_addr learned my addr 172.21.15.105:0/2737163620 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a2b46640 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55600607c590 0x5560060883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a1b44640 1 -- 172.21.15.105:0/2737163620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55600607c590 msgr2=0x5560060883b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a1b44640 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55600607c590 0x5560060883b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a1b44640 1 -- 172.21.15.105:0/2737163620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 msgr2=0x556006080e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a1b44640 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006080e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.463+0000 7f64a1b44640 1 -- 172.21.15.105:0/2737163620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556005db97d0 con 0x556006077420 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a1b44640 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x556006081370 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f648c0126e0 tx=0x7f648c012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56b1c24b05cfea34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f6485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:45:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f6485ffb640 1 -- 172.21.15.105:0/2737163620 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c012190 con 0x556006077420 2024-03-26T17:45:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f6485ffb640 1 -- 172.21.15.105:0/2737163620 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f648c00f8c0 con 0x556006077420 2024-03-26T17:45:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 1 -- 172.21.15.105:0/2737163620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556005fe8eb0 con 0x556006077420 2024-03-26T17:45:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f6485ffb640 1 -- 172.21.15.105:0/2737163620 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f648c00fac0 con 0x556006077420 2024-03-26T17:45:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 10 client.27679 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 1 -- 172.21.15.105:0/2737163620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55600608b840 con 0x556006077420 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f6485ffb640 1 -- 172.21.15.105:0/2737163620 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556005fe8eb0 con 0x556006077420 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f6485ffb640 1 -- 172.21.15.105:0/2737163620 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f648c026c90 con 0x556006077420 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 10 client.27679 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 10 client.27679 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 20 client.27679 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55600608cf00 0x55600608f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 4 client.27679 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.464+0000 7f64a7191740 1 -- 172.21.15.105:0/2737163620 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556005fe3200 con 0x55600608cf00 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.465+0000 7f64a2345640 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55600608cf00 0x55600608f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.465+0000 7f64a2345640 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55600608cf00 0x55600608f3a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ffeaf975b79756c1 server_cookie=393e3c4c64d951a1 in_seq=0 out_seq=0 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.465+0000 7f6485ffb640 10 client.27679 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.465+0000 7f6485ffb640 1 -- 172.21.15.105:0/2737163620 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556005fe3200 con 0x55600608cf00 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.465+0000 7f6485ffb640 10 client.27679 handle_command_reply: tid=0 2024-03-26T17:45:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 client.27679 shutdown 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 -- 172.21.15.105:0/2737163620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 msgr2=0x556006081370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x556006081370 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f648c0126e0 tx=0x7f648c012ba0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 -- 172.21.15.105:0/2737163620 shutdown_connections 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55600608cf00 0x55600608f3a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55600607c590 0x5560060883b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f6485ffb640 0 client.27679 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 --2- 172.21.15.105:0/2737163620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006080e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 -- 172.21.15.105:0/2737163620 >> 172.21.15.105:0/2737163620 conn(0x556006071fa0 msgr2=0x5560060711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 -- 172.21.15.105:0/2737163620 shutdown_connections 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 1 -- 172.21.15.105:0/2737163620 wait complete. 2024-03-26T17:45:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.466+0000 7f64a7191740 20 client.27679 trim_cache size 0 max 16384 2024-03-26T17:45:02.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a7191740 1 Processor -- start 2024-03-26T17:45:02.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a7191740 1 -- start start 2024-03-26T17:45:02.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a7191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006076a30 0x556006080e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a7191740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a7191740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x556006088370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a7191740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1a8d60 con 0x55600607c590 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a7191740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1aa8a0 con 0x556006077420 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a7191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1a9430 con 0x556006076a30 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a2b46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x556006088370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a2b46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x556006088370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45282/0 (socket says 172.21.15.105:45282) 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a1b44640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a2b46640 1 -- 172.21.15.105:0/4130383355 learned_addr learned my addr 172.21.15.105:0/4130383355 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.482+0000 7f64a2345640 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006076a30 0x556006080e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f64a2b46640 1 -- 172.21.15.105:0/4130383355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006076a30 msgr2=0x556006080e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f64a2b46640 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006076a30 0x556006080e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f64a2b46640 1 -- 172.21.15.105:0/4130383355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 msgr2=0x556006081370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f64a2b46640 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f64a2b46640 1 -- 172.21.15.105:0/4130383355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600608b630 con 0x55600607c590 2024-03-26T17:45:02.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f64a1b44640 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f64a2b46640 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x556006088370 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6494032420 tx=0x7f6494041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b46eaa82cd0f0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f6487fff640 1 -- 172.21.15.105:0/4130383355 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494039040 con 0x55600607c590 2024-03-26T17:45:02.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f6487fff640 1 -- 172.21.15.105:0/4130383355 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6494053070 con 0x55600607c590 2024-03-26T17:45:02.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.483+0000 7f6487fff640 1 -- 172.21.15.105:0/4130383355 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940346a0 con 0x55600607c590 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 -- 172.21.15.105:0/4130383355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 msgr2=0x556006088370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x556006088370 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6494032420 tx=0x7f6494041000 comp rx=0 tx=0).stop 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 -- 172.21.15.105:0/4130383355 shutdown_connections 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55600607c590 0x556006088370 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006081370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 --2- 172.21.15.105:0/4130383355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556006076a30 0x556006080e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 -- 172.21.15.105:0/4130383355 >> 172.21.15.105:0/4130383355 conn(0x556006071fa0 msgr2=0x556006072380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 -- 172.21.15.105:0/4130383355 shutdown_connections 2024-03-26T17:45:02.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 -- 172.21.15.105:0/4130383355 wait complete. 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 -- start start 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 Processor -- start 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006085910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006085e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.484+0000 7f64a7191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55600607c590 0x556006086df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a7191740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1aa1d0 con 0x556006076a30 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a7191740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1a9b00 con 0x556006077420 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a7191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1a6d40 con 0x55600607c590 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a7191740 10 client.0 _ll_register_callbacks cb 0x7f648c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006085910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006085910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45286/0 (socket says 172.21.15.105:45286) 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 -- 172.21.15.105:0/3123600986 learned_addr learned my addr 172.21.15.105:0/3123600986 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a1b44640 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006085e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2b46640 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55600607c590 0x556006086df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 -- 172.21.15.105:0/3123600986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55600607c590 msgr2=0x556006086df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55600607c590 0x556006086df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 -- 172.21.15.105:0/3123600986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 msgr2=0x556006085e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006085e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 -- 172.21.15.105:0/3123600986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560060873e0 con 0x556006076a30 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a2345640 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006085910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6498009380 tx=0x7f6498009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195f8e05bb4c7b6a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f6485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f6485ffb640 1 -- 172.21.15.105:0/3123600986 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649806eaf0 con 0x556006076a30 2024-03-26T17:45:02.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f6485ffb640 1 -- 172.21.15.105:0/3123600986 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f6498099f40 con 0x556006076a30 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.485+0000 7f64a7191740 1 -- 172.21.15.105:0/3123600986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6494053070 con 0x556006076a30 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f6485ffb640 1 -- 172.21.15.105:0/3123600986 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6498065070 con 0x556006076a30 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a7191740 10 client.17853 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a7191740 1 -- 172.21.15.105:0/3123600986 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556006088750 con 0x556006076a30 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f6485ffb640 1 -- 172.21.15.105:0/3123600986 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6494053070 con 0x556006076a30 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f6485ffb640 1 -- 172.21.15.105:0/3123600986 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6498098510 con 0x556006076a30 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a7191740 10 client.17853 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a7191740 10 client.17853 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a7191740 20 client.17853 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a7191740 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55600608cf00 0x5560060f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a7191740 4 client.17853 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a7191740 1 -- 172.21.15.105:0/3123600986 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556005f606a0 con 0x55600608cf00 2024-03-26T17:45:02.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.486+0000 7f64a1b44640 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55600608cf00 0x5560060f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a1b44640 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55600608cf00 0x5560060f5680 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db2d2125f305784b server_cookie=2cec8131827a5e30 in_seq=0 out_seq=0 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f6485ffb640 10 client.17853 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f6485ffb640 1 -- 172.21.15.105:0/3123600986 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556005f606a0 con 0x55600608cf00 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f6485ffb640 10 client.17853 handle_command_reply: tid=0 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 client.17853 shutdown 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 -- 172.21.15.105:0/3123600986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 msgr2=0x556006085910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006085910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6498009380 tx=0x7f6498009700 comp rx=0 tx=0).stop 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 -- 172.21.15.105:0/3123600986 shutdown_connections 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55600608cf00 0x5560060f5680 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55600607c590 0x556006086df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556006077420 0x556006085e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 --2- 172.21.15.105:0/3123600986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556006076a30 0x556006085910 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 -- 172.21.15.105:0/3123600986 >> 172.21.15.105:0/3123600986 conn(0x556006071fa0 msgr2=0x556006078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f6485ffb640 0 client.17853 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:45:02.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 -- 172.21.15.105:0/3123600986 shutdown_connections 2024-03-26T17:45:02.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 1 -- 172.21.15.105:0/3123600986 wait complete. 2024-03-26T17:45:02.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.487+0000 7f64a7191740 20 client.17853 trim_cache size 0 max 16384 2024-03-26T17:45:02.486 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 -- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64880a2f20 msgr2=0x7f64880a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64880a2f20 0x7f64880a53e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x556006077c80 tx=0x7f649402e040 comp rx=0 tx=0).stop 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 -- 172.21.15.105:0/729348121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f649c102340 msgr2=0x7f649c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f649c102340 0x7f649c197af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f649800e9f0 tx=0x7f649800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 -- 172.21.15.105:0/729348121 shutdown_connections 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64880a2f20 0x7f64880a53e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649c1075a0 0x7f649c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f649c104c70 0x7f649c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 --2- 172.21.15.105:0/729348121 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f649c102340 0x7f649c197af0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 -- 172.21.15.105:0/729348121 >> 172.21.15.105:0/729348121 conn(0x7f649c0fbd50 msgr2=0x7f649c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 -- 172.21.15.105:0/729348121 shutdown_connections 2024-03-26T17:45:02.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.488+0000 7f64a45a9640 1 -- 172.21.15.105:0/729348121 wait complete. 2024-03-26T17:45:02.491 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:45:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.617+0000 7fc4561c6640 1 -- 172.21.15.105:0/2859440999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 msgr2=0x7fc450101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.617+0000 7fc4561c6640 1 --2- 172.21.15.105:0/2859440999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 0x7fc450101e30 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc444009900 tx=0x7fc44402bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.617+0000 7fc4561c6640 1 -- 172.21.15.105:0/2859440999 shutdown_connections 2024-03-26T17:45:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.617+0000 7fc4561c6640 1 --2- 172.21.15.105:0/2859440999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc450102370 0x7fc450108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.617+0000 7fc4561c6640 1 --2- 172.21.15.105:0/2859440999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 0x7fc450101e30 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.617+0000 7fc4561c6640 1 --2- 172.21.15.105:0/2859440999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4501007c0 0x7fc450100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.617+0000 7fc4561c6640 1 -- 172.21.15.105:0/2859440999 >> 172.21.15.105:0/2859440999 conn(0x7fc4500fbd50 msgr2=0x7fc4500fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 -- 172.21.15.105:0/2859440999 shutdown_connections 2024-03-26T17:45:02.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 -- 172.21.15.105:0/2859440999 wait complete. 2024-03-26T17:45:02.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 Processor -- start 2024-03-26T17:45:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 -- start start 2024-03-26T17:45:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4501007c0 0x7fc450197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 0x7fc4501980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc450102370 0x7fc45019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc45010c340 con 0x7fc4501019b0 2024-03-26T17:45:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc45010c1c0 con 0x7fc450102370 2024-03-26T17:45:02.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.618+0000 7fc4561c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc45010c4c0 con 0x7fc4501007c0 2024-03-26T17:45:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 0x7fc4501980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4501007c0 0x7fc450197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 0x7fc4501980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45296/0 (socket says 172.21.15.105:45296) 2024-03-26T17:45:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4501007c0 0x7fc450197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42338/0 (socket says 172.21.15.105:42338) 2024-03-26T17:45:02.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 -- 172.21.15.105:0/172371476 learned_addr learned my addr 172.21.15.105:0/172371476 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44ffff640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc450102370 0x7fc45019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 -- 172.21.15.105:0/172371476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4501007c0 msgr2=0x7fc450197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4501007c0 0x7fc450197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 -- 172.21.15.105:0/172371476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc450102370 msgr2=0x7fc45019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc450102370 0x7fc45019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 -- 172.21.15.105:0/172371476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45019f780 con 0x7fc4501019b0 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44effd640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 0x7fc4501980a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc444000c00 tx=0x7fc4440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44cff9640 1 -- 172.21.15.105:0/172371476 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444043020 con 0x7fc4501019b0 2024-03-26T17:45:02.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44cff9640 1 -- 172.21.15.105:0/172371476 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4440044c0 con 0x7fc4501019b0 2024-03-26T17:45:02.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc4561c6640 1 -- 172.21.15.105:0/172371476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45019fa10 con 0x7fc4501019b0 2024-03-26T17:45:02.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc44cff9640 1 -- 172.21.15.105:0/172371476 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc444003c30 con 0x7fc4501019b0 2024-03-26T17:45:02.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.619+0000 7fc4561c6640 1 -- 172.21.15.105:0/172371476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45019ff60 con 0x7fc4501019b0 2024-03-26T17:45:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc44cff9640 1 -- 172.21.15.105:0/172371476 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc444046020 con 0x7fc4501019b0 2024-03-26T17:45:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc458db3740 1 Processor -- start 2024-03-26T17:45:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc458db3740 1 -- start start 2024-03-26T17:45:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc458db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f820e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc458db3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f825f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc458db3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f826590 0x55960f828a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc458db3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55960f785970 con 0x55960f826590 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc458db3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55960f72dd70 con 0x55960f821420 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc458db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55960f781920 con 0x55960f820a30 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.622+0000 7fc44cff9640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4380a2f20 0x7fc4380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f826590 0x55960f828a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f825f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44ffff640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4380a2f20 0x7fc4380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f820e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f825f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59144/0 (socket says 172.21.15.105:59144) 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 -- 172.21.15.105:0/3686841215 learned_addr learned my addr 172.21.15.105:0/3686841215 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f826590 0x55960f828a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45298/0 (socket says 172.21.15.105:45298) 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44cff9640 1 -- 172.21.15.105:0/172371476 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc4440f15e0 con 0x7fc4501019b0 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 -- 172.21.15.105:0/3686841215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 msgr2=0x55960f820e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/3686841215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f820e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 -- 172.21.15.105:0/3686841215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f826590 msgr2=0x55960f828a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/3686841215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f826590 0x55960f828a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 -- 172.21.15.105:0/3686841215 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960f4993f0 con 0x55960f821420 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44effd640 1 --2- 172.21.15.105:0/3686841215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f826590 0x55960f828a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44ffff640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4380a2f20 0x7fc4380a53e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc448005c40 tx=0x7fc448005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/3686841215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f825f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc440009ca0 tx=0x7fc440011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaed9c21ff984af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc435ffb640 1 -- 172.21.15.105:0/3686841215 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440009d90 con 0x55960f821420 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc435ffb640 1 -- 172.21.15.105:0/3686841215 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4400035f0 con 0x55960f821420 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.623+0000 7fc435ffb640 1 -- 172.21.15.105:0/3686841215 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440019650 con 0x55960f821420 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 -- 172.21.15.105:0/3686841215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 msgr2=0x55960f825f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 --2- 172.21.15.105:0/3686841215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f825f20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc440009ca0 tx=0x7fc440011810 comp rx=0 tx=0).stop 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 -- 172.21.15.105:0/3686841215 shutdown_connections 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 --2- 172.21.15.105:0/3686841215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f826590 0x55960f828a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 --2- 172.21.15.105:0/3686841215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f825f20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 --2- 172.21.15.105:0/3686841215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f820e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 -- 172.21.15.105:0/3686841215 >> 172.21.15.105:0/3686841215 conn(0x55960f81bfa0 msgr2=0x55960f81c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 -- 172.21.15.105:0/3686841215 shutdown_connections 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 -- 172.21.15.105:0/3686841215 wait complete. 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.624+0000 7fc458db3740 1 -- start start 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc458db3740 1 Processor -- start 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc458db3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc458db3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f82b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc458db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f826590 0x55960f832520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc458db3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55960f829540 con 0x55960f820a30 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc458db3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55960f741310 con 0x55960f821420 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc458db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55960f8296c0 con 0x55960f826590 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc458db3740 10 client.0 _ll_register_callbacks cb 0x55960f77cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45304/0 (socket says 172.21.15.105:45304) 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44f7fe640 1 -- 172.21.15.105:0/1635966961 learned_addr learned my addr 172.21.15.105:0/1635966961 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44effd640 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f82b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44ffff640 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f826590 0x55960f832520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44f7fe640 1 -- 172.21.15.105:0/1635966961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f826590 msgr2=0x55960f832520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f826590 0x55960f832520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44f7fe640 1 -- 172.21.15.105:0/1635966961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 msgr2=0x55960f82b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f82b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44f7fe640 1 -- 172.21.15.105:0/1635966961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960f5637d0 con 0x55960f820a30 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.625+0000 7fc44ffff640 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f826590 0x55960f832520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82afa0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc440011d10 tx=0x7fc440003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef930b357f43f66 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc42effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc42effd640 1 -- 172.21.15.105:0/1635966961 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440028020 con 0x55960f820a30 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 1 -- 172.21.15.105:0/1635966961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4400035f0 con 0x55960f820a30 2024-03-26T17:45:02.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc42effd640 1 -- 172.21.15.105:0/1635966961 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc440016040 con 0x55960f820a30 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc42effd640 1 -- 172.21.15.105:0/1635966961 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc440033050 con 0x55960f820a30 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 10 client.17865 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 1 -- 172.21.15.105:0/1635966961 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55960f8358e0 con 0x55960f820a30 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc42effd640 1 -- 172.21.15.105:0/1635966961 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4400035f0 con 0x55960f820a30 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc42effd640 1 -- 172.21.15.105:0/1635966961 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc440004830 con 0x55960f820a30 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 10 client.17865 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 10 client.17865 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 20 client.17865 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55960f836dc0 0x55960f839210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 4 client.17865 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.626+0000 7fc458db3740 1 -- 172.21.15.105:0/1635966961 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55960f792eb0 con 0x55960f836dc0 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.627+0000 7fc44effd640 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55960f836dc0 0x55960f839210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.627+0000 7fc42effd640 10 client.17865 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:45:02.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.627+0000 7fc44effd640 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55960f836dc0 0x55960f839210 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ac921e06a7e032 server_cookie=b0e897a2a8d6bf20 in_seq=0 out_seq=0 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc42effd640 1 -- 172.21.15.105:0/1635966961 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55960f792eb0 con 0x55960f836dc0 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc42effd640 10 client.17865 handle_command_reply: tid=0 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 client.17865 shutdown 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 -- 172.21.15.105:0/1635966961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 msgr2=0x55960f82afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82afa0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fc440011d10 tx=0x7fc440003bc0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 -- 172.21.15.105:0/1635966961 shutdown_connections 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f826590 0x55960f832520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55960f836dc0 0x55960f839210 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f821420 0x55960f82b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc42effd640 0 client.17865 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 --2- 172.21.15.105:0/1635966961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82afa0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 -- 172.21.15.105:0/1635966961 >> 172.21.15.105:0/1635966961 conn(0x55960f81bfa0 msgr2=0x55960f81b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.628+0000 7fc458db3740 1 -- 172.21.15.105:0/1635966961 shutdown_connections 2024-03-26T17:45:02.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.629+0000 7fc458db3740 1 -- 172.21.15.105:0/1635966961 wait complete. 2024-03-26T17:45:02.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.629+0000 7fc458db3740 20 client.17865 trim_cache size 0 max 16384 2024-03-26T17:45:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.643+0000 7fc458db3740 1 Processor -- start 2024-03-26T17:45:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.643+0000 7fc458db3740 1 -- start start 2024-03-26T17:45:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.643+0000 7fc458db3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f83dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.643+0000 7fc458db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f8324e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.643+0000 7fc458db3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f821420 0x55960f83e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.643+0000 7fc458db3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501a8dc0 con 0x55960f821420 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.643+0000 7fc458db3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501aa900 con 0x55960f81bfa0 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.643+0000 7fc458db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501a9490 con 0x55960f820a30 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f83dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f8324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f83dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59154/0 (socket says 172.21.15.105:59154) 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44f7fe640 1 -- 172.21.15.105:0/3959885059 learned_addr learned my addr 172.21.15.105:0/3959885059 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44ffff640 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f821420 0x55960f83e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44effd640 1 -- 172.21.15.105:0/3959885059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 msgr2=0x55960f83dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44effd640 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f83dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44effd640 1 -- 172.21.15.105:0/3959885059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f821420 msgr2=0x55960f83e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44effd640 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f821420 0x55960f83e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44effd640 1 -- 172.21.15.105:0/3959885059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960f817a10 con 0x55960f820a30 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f83dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44ffff640 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f821420 0x55960f83e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc44effd640 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f8324e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc4440ef410 tx=0x7fc444090a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77a549fbaaa519d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc4357fa640 1 -- 172.21.15.105:0/3959885059 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4440035f0 con 0x55960f820a30 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc4357fa640 1 -- 172.21.15.105:0/3959885059 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4440be3d0 con 0x55960f820a30 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.644+0000 7fc4357fa640 1 -- 172.21.15.105:0/3959885059 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4440bbd70 con 0x55960f820a30 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 -- 172.21.15.105:0/3959885059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 msgr2=0x55960f8324e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f8324e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc4440ef410 tx=0x7fc444090a00 comp rx=0 tx=0).stop 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 -- 172.21.15.105:0/3959885059 shutdown_connections 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f821420 0x55960f83e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f820a30 0x55960f8324e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 --2- 172.21.15.105:0/3959885059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f83dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 -- 172.21.15.105:0/3959885059 >> 172.21.15.105:0/3959885059 conn(0x55960f822200 msgr2=0x55960f81fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 -- 172.21.15.105:0/3959885059 shutdown_connections 2024-03-26T17:45:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 -- 172.21.15.105:0/3959885059 wait complete. 2024-03-26T17:45:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.645+0000 7fc458db3740 1 -- start start 2024-03-26T17:45:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc458db3740 1 Processor -- start 2024-03-26T17:45:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc458db3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f82f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc458db3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc458db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f821420 0x55960f830dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc458db3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501aa230 con 0x55960f820a30 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc458db3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501a9b60 con 0x55960f81bfa0 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc458db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501a6da0 con 0x55960f821420 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc458db3740 10 client.0 _ll_register_callbacks cb 0x7fc4480137c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f82f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45314/0 (socket says 172.21.15.105:45314) 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 -- 172.21.15.105:0/1704520685 learned_addr learned my addr 172.21.15.105:0/1704520685 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f821420 0x55960f830dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 -- 172.21.15.105:0/1704520685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f821420 msgr2=0x55960f830dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f821420 0x55960f830dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 -- 172.21.15.105:0/1704520685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 msgr2=0x55960f82f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f82f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 -- 172.21.15.105:0/1704520685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960f8313c0 con 0x55960f820a30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f82f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc44effd640 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82fdc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc4440ba130 tx=0x7fc4440bdbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64e183c88fdb55c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.646+0000 7fc42effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc42effd640 1 -- 172.21.15.105:0/1704520685 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4440bdcd0 con 0x55960f820a30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc42effd640 1 -- 172.21.15.105:0/1704520685 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc4440bbfd0 con 0x55960f820a30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 1 -- 172.21.15.105:0/1704520685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4440be3d0 con 0x55960f820a30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc42effd640 1 -- 172.21.15.105:0/1704520685 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4440ba340 con 0x55960f820a30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 10 client.17871 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 1 -- 172.21.15.105:0/1704520685 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55960f832730 con 0x55960f820a30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc42effd640 1 -- 172.21.15.105:0/1704520685 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4440c6250 con 0x55960f820a30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc42effd640 1 -- 172.21.15.105:0/1704520685 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc4440b9a20 con 0x55960f820a30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 10 client.17871 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 10 client.17871 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 20 client.17871 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55960f826590 0x55960f89f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 4 client.17871 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.647+0000 7fc458db3740 1 -- 172.21.15.105:0/1704520685 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc4440be3d0 con 0x55960f826590 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.648+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55960f826590 0x55960f89f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.648+0000 7fc42effd640 10 client.17871 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.648+0000 7fc44f7fe640 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55960f826590 0x55960f89f6e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e830aa678547f8b9 server_cookie=18be374cfd48be4a in_seq=0 out_seq=0 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.648+0000 7fc42effd640 1 -- 172.21.15.105:0/1704520685 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc4440be3d0 con 0x55960f826590 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.648+0000 7fc42effd640 10 client.17871 handle_command_reply: tid=0 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.648+0000 7fc458db3740 1 client.17871 shutdown 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.648+0000 7fc458db3740 1 -- 172.21.15.105:0/1704520685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 msgr2=0x55960f82fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.648+0000 7fc458db3740 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82fdc0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc4440ba130 tx=0x7fc4440bdbd0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 1 -- 172.21.15.105:0/1704520685 shutdown_connections 2024-03-26T17:45:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55960f821420 0x55960f830dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55960f820a30 0x55960f82fdc0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55960f826590 0x55960f89f6e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 1 --2- 172.21.15.105:0/1704520685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55960f81bfa0 0x55960f82f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 1 -- 172.21.15.105:0/1704520685 >> 172.21.15.105:0/1704520685 conn(0x55960f822200 msgr2=0x55960f81a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc42effd640 0 client.17871 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 1 -- 172.21.15.105:0/1704520685 shutdown_connections 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 1 -- 172.21.15.105:0/1704520685 wait complete. 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc458db3740 20 client.17871 trim_cache size 0 max 16384 2024-03-26T17:45:02.648 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 -- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4380a2f20 msgr2=0x7fc4380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4380a2f20 0x7fc4380a53e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc448005c40 tx=0x7fc448005bd0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 -- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 msgr2=0x7fc4501980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 0x7fc4501980a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc444000c00 tx=0x7fc4440040c0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 -- 172.21.15.105:0/172371476 shutdown_connections 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4380a2f20 0x7fc4380a53e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc450102370 0x7fc45019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4501019b0 0x7fc4501980a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 --2- 172.21.15.105:0/172371476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4501007c0 0x7fc450197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.649+0000 7fc4561c6640 1 -- 172.21.15.105:0/172371476 >> 172.21.15.105:0/172371476 conn(0x7fc4500fbd50 msgr2=0x7fc4500fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.650+0000 7fc4561c6640 1 -- 172.21.15.105:0/172371476 shutdown_connections 2024-03-26T17:45:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.650+0000 7fc4561c6640 1 -- 172.21.15.105:0/172371476 wait complete. 2024-03-26T17:45:02.654 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:45:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 -- 172.21.15.105:0/551141829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c100190 msgr2=0x7fe42c1005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 --2- 172.21.15.105:0/551141829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c100190 0x7fe42c1005b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009900 tx=0x7fe41c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 -- 172.21.15.105:0/551141829 shutdown_connections 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 --2- 172.21.15.105:0/551141829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe42c101490 0x7fe42c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 --2- 172.21.15.105:0/551141829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe42c100af0 0x7fe42c100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 --2- 172.21.15.105:0/551141829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c100190 0x7fe42c1005b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 -- 172.21.15.105:0/551141829 >> 172.21.15.105:0/551141829 conn(0x7fe42c0fbd50 msgr2=0x7fe42c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 -- 172.21.15.105:0/551141829 shutdown_connections 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.794+0000 7fe4318a1640 1 -- 172.21.15.105:0/551141829 wait complete. 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe4318a1640 1 Processor -- start 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe4318a1640 1 -- start start 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe4318a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe42c100190 0x7fe42c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe4318a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe42c100af0 0x7fe42c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe4318a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c101490 0x7fe42c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe4318a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c10f5b0 con 0x7fe42c101490 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe4318a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c10f430 con 0x7fe42c100190 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe4318a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c10f730 con 0x7fe42c100af0 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe42b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c101490 0x7fe42c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe423fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe42c100af0 0x7fe42c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe42affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe42c100190 0x7fe42c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe42affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe42c100190 0x7fe42c199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59188/0 (socket says 172.21.15.105:59188) 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe42affd640 1 -- 172.21.15.105:0/4002297566 learned_addr learned my addr 172.21.15.105:0/4002297566 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe423fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe42c100af0 0x7fe42c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42396/0 (socket says 172.21.15.105:42396) 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe423fff640 1 -- 172.21.15.105:0/4002297566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe42c100190 msgr2=0x7fe42c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe423fff640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe42c100190 0x7fe42c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe423fff640 1 -- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c101490 msgr2=0x7fe42c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe423fff640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c101490 0x7fe42c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe423fff640 1 -- 172.21.15.105:0/4002297566 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c1a1940 con 0x7fe42c100af0 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.795+0000 7fe42affd640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe42c100190 0x7fe42c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.796+0000 7fe423fff640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe42c100af0 0x7fe42c19a270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe41400e9f0 tx=0x7fe41400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.796+0000 7fe43089f640 1 -- 172.21.15.105:0/4002297566 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41400ccc0 con 0x7fe42c100af0 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.796+0000 7fe43089f640 1 -- 172.21.15.105:0/4002297566 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4140108e0 con 0x7fe42c100af0 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.796+0000 7fe4318a1640 1 -- 172.21.15.105:0/4002297566 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c1a1c30 con 0x7fe42c100af0 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.796+0000 7fe43089f640 1 -- 172.21.15.105:0/4002297566 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414018880 con 0x7fe42c100af0 2024-03-26T17:45:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.796+0000 7fe4318a1640 1 -- 172.21.15.105:0/4002297566 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c1a21d0 con 0x7fe42c100af0 2024-03-26T17:45:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.798+0000 7fe43456a740 1 Processor -- start 2024-03-26T17:45:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe43456a740 1 -- start start 2024-03-26T17:45:02.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe43456a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b305471e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe43089f640 1 -- 172.21.15.105:0/4002297566 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe414010a90 con 0x7fe42c100af0 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe43456a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 0x55b305476f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe43456a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 0x55b305479a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe43456a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b3053d6970 con 0x55b305471a30 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe43456a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b30537ed70 con 0x55b305477590 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe43456a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3053d2920 con 0x55b305472420 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b305471e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe423fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 0x55b305479a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 0x55b305476f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b305471e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45330/0 (socket says 172.21.15.105:45330) 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 -- 172.21.15.105:0/1505445418 learned_addr learned my addr 172.21.15.105:0/1505445418 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 -- 172.21.15.105:0/1505445418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 msgr2=0x55b305476f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 --2- 172.21.15.105:0/1505445418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 0x55b305476f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 -- 172.21.15.105:0/1505445418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 msgr2=0x55b305479a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 --2- 172.21.15.105:0/1505445418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 0x55b305479a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 -- 172.21.15.105:0/1505445418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3050ea3f0 con 0x55b305471a30 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.799+0000 7fe42b7fe640 1 --2- 172.21.15.105:0/1505445418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b305471e50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe424009fd0 tx=0x7fe4240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb0460a7ecd0435 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe43089f640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4180a2fa0 0x7fe4180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe421ffb640 1 -- 172.21.15.105:0/1505445418 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424004d20 con 0x55b305471a30 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe42affd640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4180a2fa0 0x7fe4180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe43089f640 1 -- 172.21.15.105:0/4002297566 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe4140cf730 con 0x7fe42c100af0 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe421ffb640 1 -- 172.21.15.105:0/1505445418 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe424004050 con 0x55b305471a30 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe421ffb640 1 -- 172.21.15.105:0/1505445418 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424005260 con 0x55b305471a30 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe43456a740 1 -- 172.21.15.105:0/1505445418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 msgr2=0x55b305471e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe43456a740 1 --2- 172.21.15.105:0/1505445418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b305471e50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe424009fd0 tx=0x7fe4240118d0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.800+0000 7fe42affd640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4180a2fa0 0x7fe4180a5460 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe41c000c00 tx=0x7fe41c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 -- 172.21.15.105:0/1505445418 shutdown_connections 2024-03-26T17:45:02.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 --2- 172.21.15.105:0/1505445418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 0x55b305479a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 --2- 172.21.15.105:0/1505445418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 0x55b305476f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 --2- 172.21.15.105:0/1505445418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b305471e50 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 -- 172.21.15.105:0/1505445418 >> 172.21.15.105:0/1505445418 conn(0x55b30546cfa0 msgr2=0x55b30546d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 -- 172.21.15.105:0/1505445418 shutdown_connections 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 -- 172.21.15.105:0/1505445418 wait complete. 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 -- start start 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 Processor -- start 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305471a30 0x55b30547bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305477590 0x55b3054834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b30547a540 con 0x55b305477590 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b305392310 con 0x55b305472420 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.801+0000 7fe43456a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b30547a6c0 con 0x55b305471a30 2024-03-26T17:45:02.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe43456a740 10 client.0 _ll_register_callbacks cb 0x55b3053cdc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305477590 0x55b3054834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305477590 0x55b3054834e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45344/0 (socket says 172.21.15.105:45344) 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 -- 172.21.15.105:0/1773154106 learned_addr learned my addr 172.21.15.105:0/1773154106 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305471a30 0x55b30547bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe423fff640 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 -- 172.21.15.105:0/1773154106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305471a30 msgr2=0x55b30547bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305471a30 0x55b30547bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 -- 172.21.15.105:0/1773154106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 msgr2=0x55b30547c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 -- 172.21.15.105:0/1773154106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3051b47d0 con 0x55b305477590 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe42b7fe640 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305477590 0x55b3054834e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe424011dd0 tx=0x7fe424004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c476ba8b0bca6731 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe40affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe40affd640 1 -- 172.21.15.105:0/1773154106 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424028020 con 0x55b305477590 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe40affd640 1 -- 172.21.15.105:0/1773154106 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe424003c40 con 0x55b305477590 2024-03-26T17:45:02.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe43456a740 1 -- 172.21.15.105:0/1773154106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe424004050 con 0x55b305477590 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe40affd640 1 -- 172.21.15.105:0/1773154106 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe424033050 con 0x55b305477590 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe43456a740 10 client.17883 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.802+0000 7fe43456a740 1 -- 172.21.15.105:0/1773154106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3054868a0 con 0x55b305477590 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe40affd640 1 -- 172.21.15.105:0/1773154106 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe424004050 con 0x55b305477590 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe40affd640 1 -- 172.21.15.105:0/1773154106 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe42400f4e0 con 0x55b305477590 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe43456a740 10 client.17883 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe43456a740 10 client.17883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe43456a740 20 client.17883 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe43456a740 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b3054883a0 0x55b30548a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe43456a740 4 client.17883 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe43456a740 1 -- 172.21.15.105:0/1773154106 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3053e3eb0 con 0x55b3054883a0 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe42affd640 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b3054883a0 0x55b30548a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.803+0000 7fe40affd640 10 client.17883 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.804+0000 7fe42affd640 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b3054883a0 0x55b30548a7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=591da4ae6dd847f3 server_cookie=db772ecced69dfdd in_seq=0 out_seq=0 2024-03-26T17:45:02.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.804+0000 7fe40affd640 1 -- 172.21.15.105:0/1773154106 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b3053e3eb0 con 0x55b3054883a0 2024-03-26T17:45:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.804+0000 7fe40affd640 10 client.17883 handle_command_reply: tid=0 2024-03-26T17:45:02.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.804+0000 7fe43456a740 1 client.17883 shutdown 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 -- 172.21.15.105:0/1773154106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305477590 msgr2=0x55b3054834e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305477590 0x55b3054834e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe424011dd0 tx=0x7fe424004cf0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 -- 172.21.15.105:0/1773154106 shutdown_connections 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b3054883a0 0x55b30548a7e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305477590 0x55b3054834e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe40affd640 0 client.17883 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 --2- 172.21.15.105:0/1773154106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305471a30 0x55b30547bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 -- 172.21.15.105:0/1773154106 >> 172.21.15.105:0/1773154106 conn(0x55b30546cfa0 msgr2=0x55b30546c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 -- 172.21.15.105:0/1773154106 shutdown_connections 2024-03-26T17:45:02.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 1 -- 172.21.15.105:0/1773154106 wait complete. 2024-03-26T17:45:02.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.805+0000 7fe43456a740 20 client.17883 trim_cache size 0 max 16384 2024-03-26T17:45:02.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe43456a740 1 Processor -- start 2024-03-26T17:45:02.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe43456a740 1 -- start start 2024-03-26T17:45:02.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe43456a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe43456a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 0x55b305473ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe43456a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 0x55b3054834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe43456a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c1aafb0 con 0x55b305471a30 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe43456a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c1acaf0 con 0x55b305477590 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe43456a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c1ab680 con 0x55b305472420 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe42affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe423fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 0x55b305473ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe42affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45350/0 (socket says 172.21.15.105:45350) 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe42affd640 1 -- 172.21.15.105:0/2692767456 learned_addr learned my addr 172.21.15.105:0/2692767456 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.820+0000 7fe42b7fe640 1 --2- 172.21.15.105:0/2692767456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 0x55b3054834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe42affd640 1 -- 172.21.15.105:0/2692767456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 msgr2=0x55b305473ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe42affd640 1 --2- 172.21.15.105:0/2692767456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 0x55b305473ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe42affd640 1 -- 172.21.15.105:0/2692767456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 msgr2=0x55b3054834a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe42affd640 1 --2- 172.21.15.105:0/2692767456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 0x55b3054834a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe42affd640 1 -- 172.21.15.105:0/2692767456 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b30546dd10 con 0x55b305471a30 2024-03-26T17:45:02.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe42affd640 1 --2- 172.21.15.105:0/2692767456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054739b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe41c006010 tx=0x7fe41c045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0d389f15cc5a02 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe4217fa640 1 -- 172.21.15.105:0/2692767456 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c005510 con 0x55b305471a30 2024-03-26T17:45:02.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe4217fa640 1 -- 172.21.15.105:0/2692767456 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe41c05b070 con 0x55b305471a30 2024-03-26T17:45:02.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.821+0000 7fe4217fa640 1 -- 172.21.15.105:0/2692767456 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c004920 con 0x55b305471a30 2024-03-26T17:45:02.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 -- 172.21.15.105:0/2692767456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 msgr2=0x55b3054739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 --2- 172.21.15.105:0/2692767456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054739b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe41c006010 tx=0x7fe41c045000 comp rx=0 tx=0).stop 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 -- 172.21.15.105:0/2692767456 shutdown_connections 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 --2- 172.21.15.105:0/2692767456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305477590 0x55b3054834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 --2- 172.21.15.105:0/2692767456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305472420 0x55b305473ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 --2- 172.21.15.105:0/2692767456 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054739b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 -- 172.21.15.105:0/2692767456 >> 172.21.15.105:0/2692767456 conn(0x55b30546cfa0 msgr2=0x55b30547b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 -- 172.21.15.105:0/2692767456 shutdown_connections 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 -- 172.21.15.105:0/2692767456 wait complete. 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 -- start start 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.822+0000 7fe43456a740 1 Processor -- start 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe43456a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe43456a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe43456a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305477590 0x55b3054803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe43456a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c1ac420 con 0x55b305471a30 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe43456a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c1abd50 con 0x55b305472420 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe43456a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c1a8f90 con 0x55b305477590 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe43456a740 10 client.0 _ll_register_callbacks cb 0x7fe4140d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe423fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054833f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45356/0 (socket says 172.21.15.105:45356) 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 -- 172.21.15.105:0/4027651575 learned_addr learned my addr 172.21.15.105:0/4027651575 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42b7fe640 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305477590 0x55b3054803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 -- 172.21.15.105:0/4027651575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305477590 msgr2=0x55b3054803d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305477590 0x55b3054803d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 -- 172.21.15.105:0/4027651575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 msgr2=0x55b30547fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 -- 172.21.15.105:0/4027651575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3054ec710 con 0x55b305471a30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe423fff640 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe42affd640 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054833f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005ab0 tx=0x7fe41c0056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e38d43a910953c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe40affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe40affd640 1 -- 172.21.15.105:0/4027651575 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c063020 con 0x55b305471a30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe43456a740 1 -- 172.21.15.105:0/4027651575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41c05b070 con 0x55b305471a30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.823+0000 7fe40affd640 1 -- 172.21.15.105:0/4027651575 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe41c056d10 con 0x55b305471a30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe40affd640 1 -- 172.21.15.105:0/4027651575 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe41c0046c0 con 0x55b305471a30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe43456a740 10 client.17889 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe43456a740 1 -- 172.21.15.105:0/4027651575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3054ed9a0 con 0x55b305471a30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe40affd640 1 -- 172.21.15.105:0/4027651575 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe41c05b070 con 0x55b305471a30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe40affd640 1 -- 172.21.15.105:0/4027651575 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe41c003a40 con 0x55b305471a30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe43456a740 10 client.17889 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe43456a740 10 client.17889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:45:02.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe43456a740 20 client.17889 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe43456a740 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b3054ee380 0x55b3054f07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe43456a740 4 client.17889 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe43456a740 1 -- 172.21.15.105:0/4027651575 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3053de200 con 0x55b3054ee380 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.824+0000 7fe423fff640 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b3054ee380 0x55b3054f07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe40affd640 10 client.17889 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe423fff640 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b3054ee380 0x55b3054f07a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4998163619555fb2 server_cookie=d81c73c10778a6f4 in_seq=0 out_seq=0 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe40affd640 1 -- 172.21.15.105:0/4027651575 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3053de200 con 0x55b3054ee380 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe40affd640 10 client.17889 handle_command_reply: tid=0 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 client.17889 shutdown 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 -- 172.21.15.105:0/4027651575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 msgr2=0x55b3054833f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054833f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe41c005ab0 tx=0x7fe41c0056c0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 -- 172.21.15.105:0/4027651575 shutdown_connections 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b3054ee380 0x55b3054f07a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b305477590 0x55b3054803d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b305472420 0x55b30547fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe40affd640 0 client.17889 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:45:02.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 --2- 172.21.15.105:0/4027651575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b305471a30 0x55b3054833f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 -- 172.21.15.105:0/4027651575 >> 172.21.15.105:0/4027651575 conn(0x55b30546cfa0 msgr2=0x55b305470200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 -- 172.21.15.105:0/4027651575 shutdown_connections 2024-03-26T17:45:02.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 1 -- 172.21.15.105:0/4027651575 wait complete. 2024-03-26T17:45:02.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.825+0000 7fe43456a740 20 client.17889 trim_cache size 0 max 16384 2024-03-26T17:45:02.824 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 -- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4180a2fa0 msgr2=0x7fe4180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4180a2fa0 0x7fe4180a5460 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe41c000c00 tx=0x7fe41c02c7e0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 -- 172.21.15.105:0/4002297566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe42c100af0 msgr2=0x7fe42c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe42c100af0 0x7fe42c19a270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe41400e9f0 tx=0x7fe41400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 -- 172.21.15.105:0/4002297566 shutdown_connections 2024-03-26T17:45:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4180a2fa0 0x7fe4180a5460 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe42c101490 0x7fe42c1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe42c100af0 0x7fe42c19a270 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 --2- 172.21.15.105:0/4002297566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe42c100190 0x7fe42c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 -- 172.21.15.105:0/4002297566 >> 172.21.15.105:0/4002297566 conn(0x7fe42c0fbd50 msgr2=0x7fe42c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 -- 172.21.15.105:0/4002297566 shutdown_connections 2024-03-26T17:45:02.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.826+0000 7fe4318a1640 1 -- 172.21.15.105:0/4002297566 wait complete. 2024-03-26T17:45:02.831 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:45:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 -- 172.21.15.105:0/534730276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b01019d0 msgr2=0x7f75b0101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 --2- 172.21.15.105:0/534730276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b01019d0 0x7f75b0101e50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009900 tx=0x7f75a402bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 -- 172.21.15.105:0/534730276 shutdown_connections 2024-03-26T17:45:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 --2- 172.21.15.105:0/534730276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75b0102390 0x7f75b0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 --2- 172.21.15.105:0/534730276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b01019d0 0x7f75b0101e50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 --2- 172.21.15.105:0/534730276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b01007e0 0x7f75b0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 -- 172.21.15.105:0/534730276 >> 172.21.15.105:0/534730276 conn(0x7f75b00fbd50 msgr2=0x7f75b00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 -- 172.21.15.105:0/534730276 shutdown_connections 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.946+0000 7f75b6497640 1 -- 172.21.15.105:0/534730276 wait complete. 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b6497640 1 Processor -- start 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b6497640 1 -- start start 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b6497640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75b01007e0 0x7f75b0104770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b6497640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b01019d0 0x7f75b0102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b6497640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b0102390 0x7f75b01032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b6497640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010c2f0 con 0x7f75b0102390 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b6497640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010c170 con 0x7f75b01007e0 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b6497640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75b010c470 con 0x7f75b01019d0 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b4a34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b0102390 0x7f75b01032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b4a34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b0102390 0x7f75b01032f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45376/0 (socket says 172.21.15.105:45376) 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75b4a34640 1 -- 172.21.15.105:0/419149708 learned_addr learned my addr 172.21.15.105:0/419149708 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75affff640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75b01007e0 0x7f75b0104770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.947+0000 7f75af7fe640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b01019d0 0x7f75b0102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75affff640 1 -- 172.21.15.105:0/419149708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b01019d0 msgr2=0x7f75b0102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75affff640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b01019d0 0x7f75b0102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75affff640 1 -- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b0102390 msgr2=0x7f75b01032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75affff640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b0102390 0x7f75b01032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75affff640 1 -- 172.21.15.105:0/419149708 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b0103b80 con 0x7f75b01007e0 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75b4a34640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b0102390 0x7f75b01032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75affff640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75b01007e0 0x7f75b0104770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f75a000e9c0 tx=0x7f75a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75ad7fa640 1 -- 172.21.15.105:0/419149708 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a000cd80 con 0x7f75b01007e0 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75ad7fa640 1 -- 172.21.15.105:0/419149708 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75a00108e0 con 0x7f75b01007e0 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75b6497640 1 -- 172.21.15.105:0/419149708 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b01a1bb0 con 0x7f75b01007e0 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75ad7fa640 1 -- 172.21.15.105:0/419149708 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a00187f0 con 0x7f75b01007e0 2024-03-26T17:45:02.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.948+0000 7f75b6497640 1 -- 172.21.15.105:0/419149708 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b01a20d0 con 0x7f75b01007e0 2024-03-26T17:45:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.950+0000 7f75ad7fa640 1 -- 172.21.15.105:0/419149708 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f75a001f020 con 0x7f75b01007e0 2024-03-26T17:45:02.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b9083740 1 Processor -- start 2024-03-26T17:45:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b9083740 1 -- start start 2024-03-26T17:45:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75ad7fa640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75980a2fc0 0x7f75980a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b9083740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e05d0 0x564ea98e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b9083740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b9083740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b9083740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564ea98450b0 con 0x564ea98e05d0 2024-03-26T17:45:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b9083740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ea97ed960 con 0x564ea98e0fc0 2024-03-26T17:45:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b9083740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ea9841060 con 0x564ea98e6130 2024-03-26T17:45:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75ad7fa640 1 -- 172.21.15.105:0/419149708 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f75a00cf3f0 con 0x7f75b01007e0 2024-03-26T17:45:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.951+0000 7f75b4a34640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75980a2fc0 0x7f75980a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75af7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75af7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59250/0 (socket says 172.21.15.105:59250) 2024-03-26T17:45:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75af7fe640 1 -- 172.21.15.105:0/765756251 learned_addr learned my addr 172.21.15.105:0/765756251 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75affff640 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e05d0 0x564ea98e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75b4a34640 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75affff640 1 -- 172.21.15.105:0/765756251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 msgr2=0x564ea98e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75affff640 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75affff640 1 -- 172.21.15.105:0/765756251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 msgr2=0x564ea98e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75affff640 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75affff640 1 -- 172.21.15.105:0/765756251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea97ffc60 con 0x564ea98e05d0 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75b4a34640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75980a2fc0 0x7f75980a5480 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009fd0 tx=0x7f75a800fc00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75affff640 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e05d0 0x564ea98e09f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f75a00cce50 tx=0x7f75a009b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c0e8f88cfc2fe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75967fc640 1 -- 172.21.15.105:0/765756251 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a009c180 con 0x564ea98e05d0 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75967fc640 1 -- 172.21.15.105:0/765756251 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75a009ba80 con 0x564ea98e05d0 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.952+0000 7f75967fc640 1 -- 172.21.15.105:0/765756251 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0010a90 con 0x564ea98e05d0 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.953+0000 7f75b9083740 1 -- 172.21.15.105:0/765756251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e05d0 msgr2=0x564ea98e09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.953+0000 7f75b9083740 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e05d0 0x564ea98e09f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f75a00cce50 tx=0x7f75a009b3e0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.953+0000 7f75b9083740 1 -- 172.21.15.105:0/765756251 shutdown_connections 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.953+0000 7f75b9083740 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.953+0000 7f75b9083740 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.953+0000 7f75b9083740 1 --2- 172.21.15.105:0/765756251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e05d0 0x564ea98e09f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.953+0000 7f75b9083740 1 -- 172.21.15.105:0/765756251 >> 172.21.15.105:0/765756251 conn(0x564ea98dbb40 msgr2=0x564ea98dbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.953+0000 7f75b9083740 1 -- 172.21.15.105:0/765756251 shutdown_connections 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 -- 172.21.15.105:0/765756251 wait complete. 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 -- start start 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 Processor -- start 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e05d0 0x564ea98ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e6130 0x564ea98f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564ea98e9080 con 0x564ea98e6130 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ea9800db0 con 0x564ea98e0fc0 2024-03-26T17:45:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ea98e9200 con 0x564ea98e05d0 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b9083740 10 client.0 _ll_register_callbacks cb 0x564ea9805110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b4a34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e6130 0x564ea98f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e05d0 0x564ea98ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75af7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e05d0 0x564ea98ea3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42464/0 (socket says 172.21.15.105:42464) 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75af7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98ea930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59258/0 (socket says 172.21.15.105:59258) 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75b4a34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e6130 0x564ea98f1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45392/0 (socket says 172.21.15.105:45392) 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.954+0000 7f75affff640 1 -- 172.21.15.105:0/389212567 learned_addr learned my addr 172.21.15.105:0/389212567 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75affff640 1 -- 172.21.15.105:0/389212567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 msgr2=0x564ea98ea930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75affff640 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98ea930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75affff640 1 -- 172.21.15.105:0/389212567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e6130 msgr2=0x564ea98f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75affff640 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e6130 0x564ea98f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75affff640 1 -- 172.21.15.105:0/389212567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea9559210 con 0x564ea98e05d0 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75b4a34640 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e6130 0x564ea98f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75affff640 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e05d0 0x564ea98ea3f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f75a009b780 tx=0x7f75a000efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d54c98ebdb28b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f758f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:45:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f758f7fe640 1 -- 172.21.15.105:0/389212567 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a009c180 con 0x564ea98e05d0 2024-03-26T17:45:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f758f7fe640 1 -- 172.21.15.105:0/389212567 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f75a00a4000 con 0x564ea98e05d0 2024-03-26T17:45:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75b9083740 1 -- 172.21.15.105:0/389212567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a009ba80 con 0x564ea98e05d0 2024-03-26T17:45:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f758f7fe640 1 -- 172.21.15.105:0/389212567 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f75a00a4200 con 0x564ea98e05d0 2024-03-26T17:45:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75b9083740 10 client.27733 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.955+0000 7f75b9083740 1 -- 172.21.15.105:0/389212567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ea9625620 con 0x564ea98e05d0 2024-03-26T17:45:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f758f7fe640 1 -- 172.21.15.105:0/389212567 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75a009ba80 con 0x564ea98e05d0 2024-03-26T17:45:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f758f7fe640 1 -- 172.21.15.105:0/389212567 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f75a009af60 con 0x564ea98e05d0 2024-03-26T17:45:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f75b9083740 10 client.27733 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f75b9083740 10 client.27733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f75b9083740 20 client.27733 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f75b9083740 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ea98f5e60 0x564ea98f8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f75b9083740 4 client.27733 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:02.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f75b9083740 1 -- 172.21.15.105:0/389212567 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ea98525f0 con 0x564ea98f5e60 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.956+0000 7f75af7fe640 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ea98f5e60 0x564ea98f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.957+0000 7f758f7fe640 10 client.27733 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.957+0000 7f75af7fe640 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ea98f5e60 0x564ea98f8250 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae66920ae69b791 server_cookie=ea9742f4b7003af1 in_seq=0 out_seq=0 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.957+0000 7f758f7fe640 1 -- 172.21.15.105:0/389212567 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564ea98525f0 con 0x564ea98f5e60 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.957+0000 7f758f7fe640 10 client.27733 handle_command_reply: tid=0 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 client.27733 shutdown 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 -- 172.21.15.105:0/389212567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e05d0 msgr2=0x564ea98ea3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e05d0 0x564ea98ea3f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f75a009b780 tx=0x7f75a000efb0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 -- 172.21.15.105:0/389212567 shutdown_connections 2024-03-26T17:45:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ea98f5e60 0x564ea98f8250 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e6130 0x564ea98f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e0fc0 0x564ea98ea930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f758f7fe640 0 client.27733 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 --2- 172.21.15.105:0/389212567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e05d0 0x564ea98ea3f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 -- 172.21.15.105:0/389212567 >> 172.21.15.105:0/389212567 conn(0x564ea98dbb40 msgr2=0x564ea98dadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 -- 172.21.15.105:0/389212567 shutdown_connections 2024-03-26T17:45:02.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 1 -- 172.21.15.105:0/389212567 wait complete. 2024-03-26T17:45:02.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.958+0000 7f75b9083740 20 client.27733 trim_cache size 0 max 16384 2024-03-26T17:45:02.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 -- 172.21.15.105:0/2724917557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c101db0 msgr2=0x7f2f4c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 --2- 172.21.15.105:0/2724917557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c101db0 0x7f2f4c1021d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009900 tx=0x7f2f4802bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 -- 172.21.15.105:0/2724917557 shutdown_connections 2024-03-26T17:45:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 --2- 172.21.15.105:0/2724917557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f4c1013c0 0x7f2f4c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 --2- 172.21.15.105:0/2724917557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f4c100a00 0x7f2f4c100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 --2- 172.21.15.105:0/2724917557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c101db0 0x7f2f4c1021d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 -- 172.21.15.105:0/2724917557 >> 172.21.15.105:0/2724917557 conn(0x7f2f4c0fb4d0 msgr2=0x7f2f4c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 -- 172.21.15.105:0/2724917557 shutdown_connections 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.966+0000 7f2f544a1640 1 -- 172.21.15.105:0/2724917557 wait complete. 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f544a1640 1 Processor -- start 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f544a1640 1 -- start start 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f544a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c100a00 0x7f2f4c100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f544a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f4c1013c0 0x7f2f4c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f544a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f4c101db0 0x7f2f4c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f544a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c10c2f0 con 0x7f2f4c100a00 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f544a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c10c170 con 0x7f2f4c1013c0 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f544a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c10c470 con 0x7f2f4c101db0 2024-03-26T17:45:02.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.967+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c100a00 0x7f2f4c100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c100a00 0x7f2f4c100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45408/0 (socket says 172.21.15.105:45408) 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f5223d640 1 -- 172.21.15.105:0/903028571 learned_addr learned my addr 172.21.15.105:0/903028571 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f4c101db0 0x7f2f4c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f4c1013c0 0x7f2f4c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f5223d640 1 -- 172.21.15.105:0/903028571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f4c101db0 msgr2=0x7f2f4c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f5223d640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f4c101db0 0x7f2f4c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f5223d640 1 -- 172.21.15.105:0/903028571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f4c1013c0 msgr2=0x7f2f4c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f5223d640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f4c1013c0 0x7f2f4c19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f5223d640 1 -- 172.21.15.105:0/903028571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c196660 con 0x7f2f4c100a00 2024-03-26T17:45:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f4c101db0 0x7f2f4c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f5223d640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c100a00 0x7f2f4c100460 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000c00 tx=0x7f2f48005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f437fe640 1 -- 172.21.15.105:0/903028571 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48043020 con 0x7f2f4c100a00 2024-03-26T17:45:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f437fe640 1 -- 172.21.15.105:0/903028571 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f48004350 con 0x7f2f4c100a00 2024-03-26T17:45:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f544a1640 1 -- 172.21.15.105:0/903028571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4c196890 con 0x7f2f4c100a00 2024-03-26T17:45:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f437fe640 1 -- 172.21.15.105:0/903028571 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48003b10 con 0x7f2f4c100a00 2024-03-26T17:45:02.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.968+0000 7f2f544a1640 1 -- 172.21.15.105:0/903028571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4c196de0 con 0x7f2f4c100a00 2024-03-26T17:45:02.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.971+0000 7f2f57169740 1 Processor -- start 2024-03-26T17:45:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.971+0000 7f2f57169740 1 -- start start 2024-03-26T17:45:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f57169740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f6a30 0x55eb849f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f437fe640 1 -- 172.21.15.105:0/903028571 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2f48046020 con 0x7f2f4c100a00 2024-03-26T17:45:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f57169740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f7420 0x55eb849fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f57169740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb849fea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f57169740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55eb8495b970 con 0x55eb849fc590 2024-03-26T17:45:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f57169740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55eb84903d70 con 0x55eb849f7420 2024-03-26T17:45:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f57169740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb84957920 con 0x55eb849f6a30 2024-03-26T17:45:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f51a3c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb849fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f7420 0x55eb849fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f52a3e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f6a30 0x55eb849f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f7420 0x55eb849fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59278/0 (socket says 172.21.15.105:59278) 2024-03-26T17:45:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 -- 172.21.15.105:0/3425488726 learned_addr learned my addr 172.21.15.105:0/3425488726 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 -- 172.21.15.105:0/3425488726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f6a30 msgr2=0x55eb849f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 --2- 172.21.15.105:0/3425488726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f6a30 0x55eb849f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 -- 172.21.15.105:0/3425488726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 msgr2=0x55eb849fea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 --2- 172.21.15.105:0/3425488726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb849fea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 -- 172.21.15.105:0/3425488726 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb8466f3f0 con 0x55eb849f7420 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f5223d640 1 --2- 172.21.15.105:0/3425488726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f7420 0x55eb849fbf20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f480f7510 tx=0x7f2f480f7600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d8c3a2f4c5fe9c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.972+0000 7f2f437fe640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f380a2f20 0x7f2f380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.973+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f380a2f20 0x7f2f380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.973+0000 7f2f40bfe640 1 -- 172.21.15.105:0/3425488726 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480b9ec0 con 0x55eb849f7420 2024-03-26T17:45:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.973+0000 7f2f40bfe640 1 -- 172.21.15.105:0/3425488726 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f480fe030 con 0x55eb849f7420 2024-03-26T17:45:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.973+0000 7f2f40bfe640 1 -- 172.21.15.105:0/3425488726 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480bc750 con 0x55eb849f7420 2024-03-26T17:45:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.973+0000 7f2f437fe640 1 -- 172.21.15.105:0/903028571 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f2f480f1530 con 0x7f2f4c100a00 2024-03-26T17:45:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.973+0000 7f2f57169740 1 -- 172.21.15.105:0/3425488726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f7420 msgr2=0x55eb849fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.973+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f380a2f20 0x7f2f380a53e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55eb849f7c80 tx=0x7f2f44009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.973+0000 7f2f57169740 1 --2- 172.21.15.105:0/3425488726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f7420 0x55eb849fbf20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f480f7510 tx=0x7f2f480f7600 comp rx=0 tx=0).stop 2024-03-26T17:45:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 -- 172.21.15.105:0/3425488726 shutdown_connections 2024-03-26T17:45:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 --2- 172.21.15.105:0/3425488726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb849fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 --2- 172.21.15.105:0/3425488726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f7420 0x55eb849fbf20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 --2- 172.21.15.105:0/3425488726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f6a30 0x55eb849f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 -- 172.21.15.105:0/3425488726 >> 172.21.15.105:0/3425488726 conn(0x55eb849f1fa0 msgr2=0x55eb849f23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 -- 172.21.15.105:0/3425488726 shutdown_connections 2024-03-26T17:45:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 -- 172.21.15.105:0/3425488726 wait complete. 2024-03-26T17:45:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 -- start start 2024-03-26T17:45:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 Processor -- start 2024-03-26T17:45:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb84a011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb84a08190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55eb849ff300 con 0x55eb849fc590 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55eb84917310 con 0x55eb849f6a30 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.974+0000 7f2f57169740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb849ff480 con 0x55eb849f7420 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f57169740 10 client.0 _ll_register_callbacks cb 0x55eb84952c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a00c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59284/0 (socket says 172.21.15.105:59284) 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f51a3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb84a011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 -- 172.21.15.105:0/419718460 learned_addr learned my addr 172.21.15.105:0/419718460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb84a08190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 -- 172.21.15.105:0/419718460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 msgr2=0x55eb84a011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb84a011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 -- 172.21.15.105:0/419718460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 msgr2=0x55eb84a08190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb84a08190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 -- 172.21.15.105:0/419718460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb847397d0 con 0x55eb849f6a30 2024-03-26T17:45:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb84a08190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:45:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb84a011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f5223d640 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a00c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f480ba080 tx=0x7f2f480f1830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbaf504776c59f78 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f35ffb640 1 -- 172.21.15.105:0/419718460 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480bc920 con 0x55eb849f6a30 2024-03-26T17:45:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f35ffb640 1 -- 172.21.15.105:0/419718460 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f2f4809fc80 con 0x55eb849f6a30 2024-03-26T17:45:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f57169740 1 -- 172.21.15.105:0/419718460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f480fe030 con 0x55eb849f6a30 2024-03-26T17:45:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.975+0000 7f2f35ffb640 1 -- 172.21.15.105:0/419718460 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2f480bd070 con 0x55eb849f6a30 2024-03-26T17:45:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f57169740 10 client.27743 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f57169740 1 -- 172.21.15.105:0/419718460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eb849ff300 con 0x55eb849f6a30 2024-03-26T17:45:02.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f35ffb640 1 -- 172.21.15.105:0/419718460 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f480fe030 con 0x55eb849f6a30 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f35ffb640 1 -- 172.21.15.105:0/419718460 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2f480ba290 con 0x55eb849f6a30 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f57169740 10 client.27743 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f57169740 10 client.27743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f57169740 20 client.27743 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f57169740 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb84a0c950 0x55eb84a0edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f57169740 4 client.27743 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f57169740 1 -- 172.21.15.105:0/419718460 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb84968eb0 con 0x55eb84a0c950 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.976+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb84a0c950 0x55eb84a0edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.977+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb84a0c950 0x55eb84a0edc0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc9ab79c50cc783 server_cookie=4d2fd12f47deda5b in_seq=0 out_seq=0 2024-03-26T17:45:02.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.977+0000 7f2f35ffb640 10 client.27743 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b9083740 1 Processor -- start 2024-03-26T17:45:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b9083740 1 -- start start 2024-03-26T17:45:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b9083740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98ea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b9083740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98e2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b9083740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98da560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b9083740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01aaf50 con 0x564ea98e0fc0 2024-03-26T17:45:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b9083740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01aca90 con 0x564ea98e05d0 2024-03-26T17:45:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b9083740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01ab620 con 0x564ea98e6130 2024-03-26T17:45:02.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f35ffb640 1 -- 172.21.15.105:0/419718460 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55eb84968eb0 con 0x55eb84a0c950 2024-03-26T17:45:02.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f35ffb640 10 client.27743 handle_command_reply: tid=0 2024-03-26T17:45:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b4a34640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98e2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45452/0 (socket says 172.21.15.105:45452) 2024-03-26T17:45:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75af7fe640 1 -- 172.21.15.105:0/3750109329 learned_addr learned my addr 172.21.15.105:0/3750109329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75affff640 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:02.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 client.27743 shutdown 2024-03-26T17:45:02.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 -- 172.21.15.105:0/419718460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 msgr2=0x55eb84a00c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a00c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f480ba080 tx=0x7f2f480f1830 comp rx=0 tx=0).stop 2024-03-26T17:45:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b4a34640 1 -- 172.21.15.105:0/3750109329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 msgr2=0x564ea98ea890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b4a34640 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98ea890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 -- 172.21.15.105:0/419718460 shutdown_connections 2024-03-26T17:45:02.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb84a0c950 0x55eb84a0edc0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b4a34640 1 -- 172.21.15.105:0/3750109329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 msgr2=0x564ea98e2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b4a34640 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98e2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849fc590 0x55eb84a08190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb84a011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 --2- 172.21.15.105:0/419718460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a00c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b4a34640 1 -- 172.21.15.105:0/3750109329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea98e5100 con 0x564ea98e6130 2024-03-26T17:45:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75af7fe640 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:02.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 -- 172.21.15.105:0/419718460 >> 172.21.15.105:0/419718460 conn(0x55eb849f1fa0 msgr2=0x55eb849f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:02.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f35ffb640 0 client.27743 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:02.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 -- 172.21.15.105:0/419718460 shutdown_connections 2024-03-26T17:45:02.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 1 -- 172.21.15.105:0/419718460 wait complete. 2024-03-26T17:45:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f75b4a34640 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98da560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f75a801d680 tx=0x7f75a801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbfb2de41f1aa3b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:02.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.978+0000 7f2f57169740 20 client.27743 trim_cache size 0 max 16384 2024-03-26T17:45:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f7595ffb640 1 -- 172.21.15.105:0/3750109329 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a802d070 con 0x564ea98e6130 2024-03-26T17:45:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f7595ffb640 1 -- 172.21.15.105:0/3750109329 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75a8019050 con 0x564ea98e6130 2024-03-26T17:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f7595ffb640 1 -- 172.21.15.105:0/3750109329 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8017040 con 0x564ea98e6130 2024-03-26T17:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 -- 172.21.15.105:0/3750109329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 msgr2=0x564ea98da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98da560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f75a801d680 tx=0x7f75a801d000 comp rx=0 tx=0).stop 2024-03-26T17:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 -- 172.21.15.105:0/3750109329 shutdown_connections 2024-03-26T17:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98da560 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 --2- 172.21.15.105:0/3750109329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98ea890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 -- 172.21.15.105:0/3750109329 >> 172.21.15.105:0/3750109329 conn(0x564ea98dbb40 msgr2=0x564ea98e1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 -- 172.21.15.105:0/3750109329 shutdown_connections 2024-03-26T17:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.979+0000 7f75b9083740 1 -- 172.21.15.105:0/3750109329 wait complete. 2024-03-26T17:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 1 -- start start 2024-03-26T17:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 1 Processor -- start 2024-03-26T17:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01ac3c0 con 0x564ea98e0fc0 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01abcf0 con 0x564ea98e05d0 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75b01a8f30 con 0x564ea98e6130 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b9083740 10 client.0 _ll_register_callbacks cb 0x7f75a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75affff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75affff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98eee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59302/0 (socket says 172.21.15.105:59302) 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75affff640 1 -- 172.21.15.105:0/1108976899 learned_addr learned my addr 172.21.15.105:0/1108976899 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75b4a34640 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75af7fe640 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75affff640 1 -- 172.21.15.105:0/1108976899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 msgr2=0x564ea98f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75affff640 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98f0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75affff640 1 -- 172.21.15.105:0/1108976899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 msgr2=0x564ea98ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.980+0000 7f75affff640 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f75affff640 1 -- 172.21.15.105:0/1108976899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea98f0940 con 0x564ea98e05d0 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f75affff640 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98eee00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f75a0098450 tx=0x7f75a000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=679fe665471ffabb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f758f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f758f7fe640 1 -- 172.21.15.105:0/1108976899 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0082040 con 0x564ea98e05d0 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f75b9083740 1 -- 172.21.15.105:0/1108976899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a8019050 con 0x564ea98e05d0 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f758f7fe640 1 -- 172.21.15.105:0/1108976899 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f75a00862a0 con 0x564ea98e05d0 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f758f7fe640 1 -- 172.21.15.105:0/1108976899 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f75a009c180 con 0x564ea98e05d0 2024-03-26T17:45:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f75b9083740 10 client.27749 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f75b9083740 1 -- 172.21.15.105:0/1108976899 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ea98f0940 con 0x564ea98e05d0 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f758f7fe640 1 -- 172.21.15.105:0/1108976899 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75a8019050 con 0x564ea98e05d0 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.981+0000 7f758f7fe640 1 -- 172.21.15.105:0/1108976899 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f75a009aed0 con 0x564ea98e05d0 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f75b9083740 10 client.27749 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f75b9083740 10 client.27749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f75b9083740 20 client.27749 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f75b9083740 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ea995c6b0 0x564ea995ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f75b9083740 4 client.27749 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f75b9083740 1 -- 172.21.15.105:0/1108976899 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ea984c940 con 0x564ea995c6b0 2024-03-26T17:45:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f75af7fe640 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ea995c6b0 0x564ea995ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f758f7fe640 10 client.27749 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:02.982+0000 7f75af7fe640 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ea995c6b0 0x564ea995ead0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff602fcf8a491a37 server_cookie=1852a3fa51ccf083 in_seq=0 out_seq=0 2024-03-26T17:45:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.994+0000 7f2f57169740 1 Processor -- start 2024-03-26T17:45:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.994+0000 7f2f57169740 1 -- start start 2024-03-26T17:45:03.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.994+0000 7f2f57169740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f1fa0 0x55eb849ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.994+0000 7f2f57169740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f6a30 0x55eb84a08150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.994+0000 7f2f57169740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb849f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.994+0000 7f2f57169740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c1aaf80 con 0x55eb849f6a30 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.994+0000 7f2f57169740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c1acac0 con 0x55eb849f1fa0 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.994+0000 7f2f57169740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c1ab650 con 0x55eb849f7420 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f1fa0 0x55eb849ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f1fa0 0x55eb849ed050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59316/0 (socket says 172.21.15.105:59316) 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 -- 172.21.15.105:0/3625580593 learned_addr learned my addr 172.21.15.105:0/3625580593 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:03.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f6a30 0x55eb84a08150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb849f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 -- 172.21.15.105:0/3625580593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 msgr2=0x55eb849f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb849f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 -- 172.21.15.105:0/3625580593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f6a30 msgr2=0x55eb84a08150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f6a30 0x55eb84a08150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 -- 172.21.15.105:0/3625580593 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb849ed820 con 0x55eb849f1fa0 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f6a30 0x55eb84a08150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f5223d640 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f1fa0 0x55eb849ed050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f480fc040 tx=0x7f2f480f7220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8186a4a3581f23 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f37fff640 1 -- 172.21.15.105:0/3625580593 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480db070 con 0x55eb849f1fa0 2024-03-26T17:45:03.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f37fff640 1 -- 172.21.15.105:0/3625580593 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f480fe070 con 0x55eb849f1fa0 2024-03-26T17:45:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.995+0000 7f2f37fff640 1 -- 172.21.15.105:0/3625580593 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480bd3a0 con 0x55eb849f1fa0 2024-03-26T17:45:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 -- 172.21.15.105:0/3625580593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f1fa0 msgr2=0x55eb849ed050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f1fa0 0x55eb849ed050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f480fc040 tx=0x7f2f480f7220 comp rx=0 tx=0).stop 2024-03-26T17:45:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 -- 172.21.15.105:0/3625580593 shutdown_connections 2024-03-26T17:45:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f7420 0x55eb849f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f6a30 0x55eb84a08150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 --2- 172.21.15.105:0/3625580593 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f1fa0 0x55eb849ed050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 -- 172.21.15.105:0/3625580593 >> 172.21.15.105:0/3625580593 conn(0x55eb849ffe90 msgr2=0x55eb849f5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 -- 172.21.15.105:0/3625580593 shutdown_connections 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.996+0000 7f2f57169740 1 -- 172.21.15.105:0/3625580593 wait complete. 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f57169740 1 -- start start 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f57169740 1 Processor -- start 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f57169740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f1fa0 0x55eb84a04e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f57169740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a05360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f57169740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f7420 0x55eb84a70cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f57169740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c1ac3f0 con 0x55eb849f7420 2024-03-26T17:45:03.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f57169740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c1abd20 con 0x55eb849f6a30 2024-03-26T17:45:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f1fa0 0x55eb84a04e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f5223d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f1fa0 0x55eb84a04e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42550/0 (socket says 172.21.15.105:42550) 2024-03-26T17:45:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f5223d640 1 -- 172.21.15.105:0/2256062069 learned_addr learned my addr 172.21.15.105:0/2256062069 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c1a8f40 con 0x55eb849f1fa0 2024-03-26T17:45:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a05360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.997+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f7420 0x55eb84a70cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f5223d640 1 -- 172.21.15.105:0/2256062069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 msgr2=0x55eb84a05360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f5223d640 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a05360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f5223d640 1 -- 172.21.15.105:0/2256062069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f7420 msgr2=0x55eb84a70cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f5223d640 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f7420 0x55eb84a70cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f5223d640 1 -- 172.21.15.105:0/2256062069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb84a083c0 con 0x55eb849f1fa0 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f52a3e640 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f7420 0x55eb84a70cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f57169740 10 client.0 _ll_register_callbacks cb 0x7f2f3c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f480fe070 con 0x55eb849f1fa0 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f5223d640 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f1fa0 0x55eb84a04e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f480ba080 tx=0x7f2f480045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47a649c04dcfe197 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:03.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f35ffb640 10 client.27757 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:45:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f35ffb640 1 -- 172.21.15.105:0/2256062069 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480a2050 con 0x55eb849f1fa0 2024-03-26T17:45:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f35ffb640 1 -- 172.21.15.105:0/2256062069 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f2f480ba290 con 0x55eb849f1fa0 2024-03-26T17:45:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f35ffb640 1 -- 172.21.15.105:0/2256062069 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2f480db070 con 0x55eb849f1fa0 2024-03-26T17:45:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f57169740 10 client.27757 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.998+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55eb84a083c0 con 0x55eb849f1fa0 2024-03-26T17:45:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f35ffb640 1 -- 172.21.15.105:0/2256062069 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2f48003cb0 con 0x55eb849f1fa0 2024-03-26T17:45:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f57169740 10 client.27757 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:03.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f57169740 10 client.27757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f57169740 20 client.27757 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f57169740 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb849fc590 0x55eb84a74630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f57169740 4 client.27757 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb84963200 con 0x55eb849fc590 2024-03-26T17:45:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb849fc590 0x55eb84a74630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f35ffb640 10 client.27757 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:03.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:02.999+0000 7f2f51a3c640 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb849fc590 0x55eb84a74630 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969df8fa60148df3 server_cookie=d24602639b790370 in_seq=0 out_seq=0 2024-03-26T17:45:03.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 203596 3.15 MB/sec execute 226 sec latency 45.320 ms 2024-03-26T17:45:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.041+0000 7f758f7fe640 1 -- 172.21.15.105:0/1108976899 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ea984c940 con 0x564ea995c6b0 2024-03-26T17:45:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.041+0000 7f758f7fe640 10 client.27749 handle_command_reply: tid=0 2024-03-26T17:45:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.041+0000 7f75b9083740 1 client.27749 shutdown 2024-03-26T17:45:03.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.041+0000 7f2f35ffb640 1 -- 172.21.15.105:0/2256062069 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb84963200 con 0x55eb849fc590 2024-03-26T17:45:03.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.041+0000 7f2f35ffb640 10 client.27757 handle_command_reply: tid=0 2024-03-26T17:45:03.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.041+0000 7f2f57169740 1 client.27757 shutdown 2024-03-26T17:45:03.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f1fa0 msgr2=0x55eb84a04e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:03.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f1fa0 0x55eb84a04e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f480ba080 tx=0x7f2f480045e0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 -- 172.21.15.105:0/1108976899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 msgr2=0x564ea98eee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98eee00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f75a0098450 tx=0x7f75a000cf20 comp rx=0 tx=0).stop 2024-03-26T17:45:03.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 shutdown_connections 2024-03-26T17:45:03.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb849fc590 0x55eb84a74630 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb849f7420 0x55eb84a70cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f35ffb640 0 client.27757 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:03.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb849f6a30 0x55eb84a05360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 --2- 172.21.15.105:0/2256062069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb849f1fa0 0x55eb84a04e20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 >> 172.21.15.105:0/2256062069 conn(0x55eb849ffe90 msgr2=0x55eb849f5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 -- 172.21.15.105:0/1108976899 shutdown_connections 2024-03-26T17:45:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ea995c6b0 0x564ea995ead0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ea98e6130 0x564ea98f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f758f7fe640 0 client.27749 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ea98e0fc0 0x564ea98ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 --2- 172.21.15.105:0/1108976899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ea98e05d0 0x564ea98eee00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 -- 172.21.15.105:0/1108976899 >> 172.21.15.105:0/1108976899 conn(0x564ea98dbb40 msgr2=0x564ea98e2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:03.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 shutdown_connections 2024-03-26T17:45:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 1 -- 172.21.15.105:0/2256062069 wait complete. 2024-03-26T17:45:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 -- 172.21.15.105:0/1108976899 shutdown_connections 2024-03-26T17:45:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 1 -- 172.21.15.105:0/1108976899 wait complete. 2024-03-26T17:45:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f57169740 20 client.27757 trim_cache size 0 max 16384 2024-03-26T17:45:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f75b9083740 20 client.27749 trim_cache size 0 max 16384 2024-03-26T17:45:03.039 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:45:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:45:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:45:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:45:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.042+0000 7f2f544a1640 1 -- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f380a2f20 msgr2=0x7f2f380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f380a2f20 0x7f2f380a53e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55eb849f7c80 tx=0x7f2f44009500 comp rx=0 tx=0).stop 2024-03-26T17:45:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 -- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c100a00 msgr2=0x7f2f4c100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c100a00 0x7f2f4c100460 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2f48000c00 tx=0x7f2f48005dc0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 -- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75980a2fc0 msgr2=0x7f75980a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75980a2fc0 0x7f75980a5480 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009fd0 tx=0x7f75a800fc00 comp rx=0 tx=0).stop 2024-03-26T17:45:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 -- 172.21.15.105:0/419149708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75b01007e0 msgr2=0x7f75b0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75b01007e0 0x7f75b0104770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f75a000e9c0 tx=0x7f75a000ee80 comp rx=0 tx=0).stop 2024-03-26T17:45:03.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 -- 172.21.15.105:0/903028571 shutdown_connections 2024-03-26T17:45:03.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f380a2f20 0x7f2f380a53e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f4c101db0 0x7f2f4c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f4c1013c0 0x7f2f4c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 --2- 172.21.15.105:0/903028571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f4c100a00 0x7f2f4c100460 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 -- 172.21.15.105:0/903028571 >> 172.21.15.105:0/903028571 conn(0x7f2f4c0fb4d0 msgr2=0x7f2f4c104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 -- 172.21.15.105:0/419149708 shutdown_connections 2024-03-26T17:45:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75980a2fc0 0x7f75980a5480 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75b0102390 0x7f75b01032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75b01019d0 0x7f75b0102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 -- 172.21.15.105:0/903028571 shutdown_connections 2024-03-26T17:45:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 --2- 172.21.15.105:0/419149708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75b01007e0 0x7f75b0104770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 -- 172.21.15.105:0/419149708 >> 172.21.15.105:0/419149708 conn(0x7f75b00fbd50 msgr2=0x7f75b00fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:03.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f2f544a1640 1 -- 172.21.15.105:0/903028571 wait complete. 2024-03-26T17:45:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 -- 172.21.15.105:0/419149708 shutdown_connections 2024-03-26T17:45:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:03.043+0000 7f75b6497640 1 -- 172.21.15.105:0/419149708 wait complete. 2024-03-26T17:45:03.046 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:02 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2010125480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:45:03.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:02 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2010125480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:45:03.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:02 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2010125480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:45:04.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 204250 3.14 MB/sec execute 227 sec latency 17.108 ms 2024-03-26T17:45:04.049 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:45:04.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:45:04.049 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:45:04.075 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:03 smithi105 ceph-mon[32280]: pgmap v549: 97 pgs: 97 active+clean; 28 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:03 smithi144 ceph-mon[36095]: pgmap v549: 97 pgs: 97 active+clean; 28 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:04.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:03 smithi135 ceph-mon[35636]: pgmap v549: 97 pgs: 97 active+clean; 28 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/3135218213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102370 msgr2=0x7fdf74108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/3135218213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102370 0x7fdf74108c00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009ca0 tx=0x7fdf6c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/3135218213 shutdown_connections 2024-03-26T17:45:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/3135218213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102370 0x7fdf74108c00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/3135218213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf741019b0 0x7fdf74101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/3135218213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf741007c0 0x7fdf74100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/3135218213 >> 172.21.15.105:0/3135218213 conn(0x7fdf740fbd50 msgr2=0x7fdf740fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/3135218213 shutdown_connections 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/3135218213 wait complete. 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 Processor -- start 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.176+0000 7fdf7b6ea640 1 -- start start 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf7b6ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf741007c0 0x7fdf74197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf7b6ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741019b0 0x7fdf74198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf7b6ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf74102370 0x7fdf7419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf7b6ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410c2d0 con 0x7fdf741019b0 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf7b6ea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410c150 con 0x7fdf74102370 2024-03-26T17:45:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf7b6ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410c450 con 0x7fdf741007c0 2024-03-26T17:45:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741019b0 0x7fdf74198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf79486640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf741007c0 0x7fdf74197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf79c87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf74102370 0x7fdf7419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741019b0 0x7fdf74198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45498/0 (socket says 172.21.15.105:45498) 2024-03-26T17:45:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 -- 172.21.15.105:0/874298157 learned_addr learned my addr 172.21.15.105:0/874298157 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 -- 172.21.15.105:0/874298157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf741007c0 msgr2=0x7fdf74197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf741007c0 0x7fdf74197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 -- 172.21.15.105:0/874298157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf74102370 msgr2=0x7fdf7419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf74102370 0x7fdf7419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 -- 172.21.15.105:0/874298157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7419f740 con 0x7fdf741019b0 2024-03-26T17:45:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf79c87640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf74102370 0x7fdf7419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf79486640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf741007c0 0x7fdf74197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf78c85640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741019b0 0x7fdf74198060 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400d950 tx=0x7fdf6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf6a7fc640 1 -- 172.21.15.105:0/874298157 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64017030 con 0x7fdf741019b0 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/874298157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7419fa30 con 0x7fdf741019b0 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.177+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/874298157 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7419ffd0 con 0x7fdf741019b0 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.178+0000 7fdf6a7fc640 1 -- 172.21.15.105:0/874298157 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf64004020 con 0x7fdf741019b0 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.178+0000 7fdf6a7fc640 1 -- 172.21.15.105:0/874298157 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf640047a0 con 0x7fdf741019b0 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf6a7fc640 1 -- 172.21.15.105:0/874298157 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdf640041d0 con 0x7fdf741019b0 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf7e3b2740 1 Processor -- start 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf7e3b2740 1 -- start start 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030efb80 0x5641030f4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564103054630 con 0x5641030ef190 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564102ffce10 con 0x5641030f4cf0 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5641030505e0 con 0x5641030efb80 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf79c87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf79c87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030ef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45510/0 (socket says 172.21.15.105:45510) 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf79c87640 1 -- 172.21.15.105:0/4145656727 learned_addr learned my addr 172.21.15.105:0/4145656727 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf78c85640 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.180+0000 7fdf79486640 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030efb80 0x5641030f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf6a7fc640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf79c87640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf6a7fc640 1 -- 172.21.15.105:0/874298157 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdf640ced50 con 0x7fdf741019b0 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf78c85640 1 -- 172.21.15.105:0/4145656727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030efb80 msgr2=0x5641030f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf78c85640 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030efb80 0x5641030f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf78c85640 1 -- 172.21.15.105:0/4145656727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 msgr2=0x5641030ef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf78c85640 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030ef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf78c85640 1 -- 172.21.15.105:0/4145656727 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564103013740 con 0x5641030f4cf0 2024-03-26T17:45:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf79486640 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030efb80 0x5641030f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf79c87640 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf78c85640 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f7180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf640cc860 tx=0x7fdf6409b2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7597507c6beeba5a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf5f7fe640 1 -- 172.21.15.105:0/4145656727 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6409ada0 con 0x5641030f4cf0 2024-03-26T17:45:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf79c87640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005fe0 tx=0x7fdf6c02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf5f7fe640 1 -- 172.21.15.105:0/4145656727 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6409bb40 con 0x5641030f4cf0 2024-03-26T17:45:04.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.181+0000 7fdf5f7fe640 1 -- 172.21.15.105:0/4145656727 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64097ab0 con 0x5641030f4cf0 2024-03-26T17:45:04.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/4145656727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 msgr2=0x5641030f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f7180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf640cc860 tx=0x7fdf6409b2a0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/4145656727 shutdown_connections 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f7180 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030efb80 0x5641030f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/4145656727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/4145656727 >> 172.21.15.105:0/4145656727 conn(0x5641030ea700 msgr2=0x5641030eab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/4145656727 shutdown_connections 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.182+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/4145656727 wait complete. 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 1 -- start start 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 1 Processor -- start 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 0x5641030f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x5641030f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x564103100c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5641030f7c40 con 0x5641030efb80 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5641030101a0 con 0x5641030f4cf0 2024-03-26T17:45:04.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5641030f7dc0 con 0x5641030ef190 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf78c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x5641030f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf79c87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x564103100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf7e3b2740 10 client.0 _ll_register_callbacks cb 0x564102d67c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf78c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x5641030f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45524/0 (socket says 172.21.15.105:45524) 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf78c85640 1 -- 172.21.15.105:0/3490959416 learned_addr learned my addr 172.21.15.105:0/3490959416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.183+0000 7fdf79486640 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 0x5641030f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf79c87640 1 -- 172.21.15.105:0/3490959416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 msgr2=0x5641030f96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf79c87640 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 0x5641030f96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf79c87640 1 -- 172.21.15.105:0/3490959416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 msgr2=0x5641030f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf79c87640 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x5641030f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf79c87640 1 -- 172.21.15.105:0/3490959416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564102e307e0 con 0x5641030f4cf0 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf78c85640 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x5641030f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf79c87640 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x564103100c00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c032380 tx=0x7fdf6c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=156bbc10526125e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3490959416 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c039040 con 0x5641030f4cf0 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3490959416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf6409bb40 con 0x5641030f4cf0 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3490959416 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdf6c0045d0 con 0x5641030f4cf0 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3490959416 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf6c0047d0 con 0x5641030f4cf0 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf7e3b2740 10 client.27767 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3490959416 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564102e90f70 con 0x5641030f4cf0 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.184+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3490959416 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6409bb40 con 0x5641030f4cf0 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3490959416 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdf6c0574e0 con 0x5641030f4cf0 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf7e3b2740 10 client.27767 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf7e3b2740 10 client.27767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf7e3b2740 20 client.27767 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564103105490 0x5641031078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf7e3b2740 4 client.27767 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:04.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3490959416 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564103061b70 con 0x564103105490 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf79486640 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564103105490 0x5641031078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf5cff9640 10 client.27767 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.185+0000 7fdf79486640 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564103105490 0x5641031078f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a081eced61ce7556 server_cookie=8f0785b9b19cf260 in_seq=0 out_seq=0 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3490959416 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564103061b70 con 0x564103105490 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf5cff9640 10 client.27767 handle_command_reply: tid=0 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 client.27767 shutdown 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3490959416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 msgr2=0x564103100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x564103100c00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c032380 tx=0x7fdf6c0095c0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3490959416 shutdown_connections 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564103105490 0x5641031078f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x564103100c00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf5cff9640 0 client.27767 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x5641030f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3490959416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 0x5641030f96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3490959416 >> 172.21.15.105:0/3490959416 conn(0x5641030ea700 msgr2=0x5641030e9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3490959416 shutdown_connections 2024-03-26T17:45:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3490959416 wait complete. 2024-03-26T17:45:04.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.186+0000 7fdf7e3b2740 20 client.27767 trim_cache size 0 max 16384 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf7e3b2740 1 Processor -- start 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf7e3b2740 1 -- start start 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 0x5641030ee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x564103100bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5800e500 con 0x5641030efb80 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58010040 con 0x5641030f4cf0 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5800ebd0 con 0x5641030ef190 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf78c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x564103100bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf79c87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf78c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x564103100bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45530/0 (socket says 172.21.15.105:45530) 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf78c85640 1 -- 172.21.15.105:0/444056419 learned_addr learned my addr 172.21.15.105:0/444056419 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.206+0000 7fdf79486640 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 0x5641030ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf78c85640 1 -- 172.21.15.105:0/444056419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 msgr2=0x5641030ee070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf78c85640 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 0x5641030ee070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf78c85640 1 -- 172.21.15.105:0/444056419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 msgr2=0x5641030f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf78c85640 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf78c85640 1 -- 172.21.15.105:0/444056419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564102d28170 con 0x5641030efb80 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf79c87640 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf78c85640 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x564103100bc0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009610 tx=0x7fdf6409b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a5e269b2cfc1b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf5effd640 1 -- 172.21.15.105:0/444056419 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf640a3e50 con 0x5641030efb80 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf5effd640 1 -- 172.21.15.105:0/444056419 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6409b440 con 0x5641030efb80 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.207+0000 7fdf5effd640 1 -- 172.21.15.105:0/444056419 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64097190 con 0x5641030efb80 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/444056419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 msgr2=0x564103100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x564103100bc0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009610 tx=0x7fdf6409b0a0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/444056419 shutdown_connections 2024-03-26T17:45:04.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030f4cf0 0x5641030f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030efb80 0x564103100bc0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/444056419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030ef190 0x5641030ee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/444056419 >> 172.21.15.105:0/444056419 conn(0x5641030ea700 msgr2=0x5641030ed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/444056419 shutdown_connections 2024-03-26T17:45:04.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/444056419 wait complete. 2024-03-26T17:45:04.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 -- start start 2024-03-26T17:45:04.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.208+0000 7fdf7e3b2740 1 Processor -- start 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030efb80 0x5641030feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf7e3b2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030f4cf0 0x5641030ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5800f970 con 0x5641030ef190 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5800f2a0 con 0x5641030efb80 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf7e3b2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5800c4e0 con 0x5641030f4cf0 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf7e3b2740 10 client.0 _ll_register_callbacks cb 0x7fdf6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030fe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45546/0 (socket says 172.21.15.105:45546) 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 -- 172.21.15.105:0/3711707496 learned_addr learned my addr 172.21.15.105:0/3711707496 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf78c85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030efb80 0x5641030feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79c87640 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030f4cf0 0x5641030ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 -- 172.21.15.105:0/3711707496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030f4cf0 msgr2=0x5641030ffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030f4cf0 0x5641030ffb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 -- 172.21.15.105:0/3711707496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030efb80 msgr2=0x5641030feb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030efb80 0x5641030feb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 -- 172.21.15.105:0/3711707496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564103100100 con 0x5641030ef190 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf78c85640 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030efb80 0x5641030feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf79486640 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030fe610 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009fd0 tx=0x7fdf70012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84c7f2a4b33008e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3711707496 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70011c20 con 0x5641030ef190 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3711707496 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fdf70011dc0 con 0x5641030ef190 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.209+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3711707496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf6409b440 con 0x5641030ef190 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3711707496 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf70005020 con 0x5641030ef190 2024-03-26T17:45:04.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf7e3b2740 10 client.17943 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3711707496 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56410316bbc0 con 0x5641030ef190 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3711707496 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6409b440 con 0x5641030ef190 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3711707496 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdf70004c60 con 0x5641030ef190 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf7e3b2740 10 client.17943 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf7e3b2740 10 client.17943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf7e3b2740 20 client.17943 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56410316c640 0x56410316ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf7e3b2740 4 client.17943 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3711707496 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56410305bec0 con 0x56410316c640 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf78c85640 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56410316c640 0x56410316ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf78c85640 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56410316c640 0x56410316ea60 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e88548f36af57f5 server_cookie=31693b6821d21a0b in_seq=0 out_seq=0 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.210+0000 7fdf5cff9640 10 client.17943 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf5cff9640 1 -- 172.21.15.105:0/3711707496 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56410305bec0 con 0x56410316c640 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf5cff9640 10 client.17943 handle_command_reply: tid=0 2024-03-26T17:45:04.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 client.17943 shutdown 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3711707496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 msgr2=0x5641030fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030fe610 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009fd0 tx=0x7fdf70012bc0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3711707496 shutdown_connections 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56410316c640 0x56410316ea60 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641030f4cf0 0x5641030ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641030efb80 0x5641030feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf5cff9640 0 client.17943 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 --2- 172.21.15.105:0/3711707496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641030ef190 0x5641030fe610 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3711707496 >> 172.21.15.105:0/3711707496 conn(0x5641030ea700 msgr2=0x5641030ee1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3711707496 shutdown_connections 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 1 -- 172.21.15.105:0/3711707496 wait complete. 2024-03-26T17:45:04.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.213+0000 7fdf7e3b2740 20 client.17943 trim_cache size 0 max 16384 2024-03-26T17:45:04.210 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:45:04.210 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:45:04.210 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "cd636fc3-76e0-44e7-9029-94b48ab9310c", 2024-03-26T17:45:04.210 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:45:04.210 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 msgr2=0x7fdf600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005fe0 tx=0x7fdf6c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741019b0 msgr2=0x7fdf74198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741019b0 0x7fdf74198060 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400d950 tx=0x7fdf6400de10 comp rx=0 tx=0).stop 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/874298157 shutdown_connections 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf74102370 0x7fdf7419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf741019b0 0x7fdf74198060 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 --2- 172.21.15.105:0/874298157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf741007c0 0x7fdf74197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/874298157 >> 172.21.15.105:0/874298157 conn(0x7fdf740fbd50 msgr2=0x7fdf74068b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/874298157 shutdown_connections 2024-03-26T17:45:04.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.214+0000 7fdf7b6ea640 1 -- 172.21.15.105:0/874298157 wait complete. 2024-03-26T17:45:04.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:45:04.217 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:45:04.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 -- 172.21.15.105:0/108155432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb00102370 msgr2=0x7feb00108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 --2- 172.21.15.105:0/108155432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb00102370 0x7feb00108c00 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009ca0 tx=0x7feaf802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 -- 172.21.15.105:0/108155432 shutdown_connections 2024-03-26T17:45:04.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 --2- 172.21.15.105:0/108155432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb00102370 0x7feb00108c00 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 --2- 172.21.15.105:0/108155432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb001019b0 0x7feb00101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 --2- 172.21.15.105:0/108155432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb001007c0 0x7feb00100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 -- 172.21.15.105:0/108155432 >> 172.21.15.105:0/108155432 conn(0x7feb000fbd50 msgr2=0x7feb000fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 -- 172.21.15.105:0/108155432 shutdown_connections 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.387+0000 7feb07c8c640 1 -- 172.21.15.105:0/108155432 wait complete. 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb07c8c640 1 Processor -- start 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb07c8c640 1 -- start start 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb07c8c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb001007c0 0x7feb0010b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb07c8c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb001019b0 0x7feb00109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb07c8c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb00102370 0x7feb00109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb07c8c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb0010d410 con 0x7feb001007c0 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb07c8c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb0010d290 con 0x7feb00102370 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb07c8c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb0010d590 con 0x7feb001019b0 2024-03-26T17:45:04.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb05a28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb001007c0 0x7feb0010b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb05227640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb001019b0 0x7feb00109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb05a28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb001007c0 0x7feb0010b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45556/0 (socket says 172.21.15.105:45556) 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb05a28640 1 -- 172.21.15.105:0/1143089153 learned_addr learned my addr 172.21.15.105:0/1143089153 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.388+0000 7feb06229640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb00102370 0x7feb00109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb05a28640 1 -- 172.21.15.105:0/1143089153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb001019b0 msgr2=0x7feb00109780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb05a28640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb001019b0 0x7feb00109780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb05a28640 1 -- 172.21.15.105:0/1143089153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb00102370 msgr2=0x7feb00109cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb05a28640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb00102370 0x7feb00109cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb05a28640 1 -- 172.21.15.105:0/1143089153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0010a200 con 0x7feb001007c0 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb05227640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb001019b0 0x7feb00109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb06229640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb00102370 0x7feb00109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb05a28640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb001007c0 0x7feb0010b140 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e9c0 tx=0x7feafc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feaf6ffd640 1 -- 172.21.15.105:0/1143089153 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc00cd80 con 0x7feb001007c0 2024-03-26T17:45:04.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feaf6ffd640 1 -- 172.21.15.105:0/1143089153 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feafc0035f0 con 0x7feb001007c0 2024-03-26T17:45:04.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feaf6ffd640 1 -- 172.21.15.105:0/1143089153 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc010690 con 0x7feb001007c0 2024-03-26T17:45:04.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb07c8c640 1 -- 172.21.15.105:0/1143089153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0010a4f0 con 0x7feb001007c0 2024-03-26T17:45:04.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.389+0000 7feb07c8c640 1 -- 172.21.15.105:0/1143089153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb001a65d0 con 0x7feb001007c0 2024-03-26T17:45:04.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.392+0000 7feaf6ffd640 1 -- 172.21.15.105:0/1143089153 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7feafc0037a0 con 0x7feb001007c0 2024-03-26T17:45:04.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.392+0000 7feb0a87e740 1 Processor -- start 2024-03-26T17:45:04.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb0a87e740 1 -- start start 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087958f0 0x55f408795d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f40879ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f40879d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb0a87e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f4086fa980 con 0x55f4087958f0 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb0a87e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f4086a2be0 con 0x55f4087962e0 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb0a87e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f4086f6930 con 0x55f40879b450 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb05a28640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f40879ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb06229640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087958f0 0x55f408795d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb06229640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087958f0 0x55f408795d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45570/0 (socket says 172.21.15.105:45570) 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb06229640 1 -- 172.21.15.105:0/2471001503 learned_addr learned my addr 172.21.15.105:0/2471001503 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb05227640 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f40879d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feaf6ffd640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feaec0a2f20 0x7feaec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb05a28640 1 -- 172.21.15.105:0/2471001503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 msgr2=0x55f40879d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb05a28640 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f40879d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb05a28640 1 -- 172.21.15.105:0/2471001503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087958f0 msgr2=0x55f408795d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb05a28640 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087958f0 0x55f408795d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb05a28640 1 -- 172.21.15.105:0/2471001503 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4084d87d0 con 0x55f4087962e0 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb05227640 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f40879d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb06229640 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087958f0 0x55f408795d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.393+0000 7feb06229640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feaec0a2f20 0x7feaec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.394+0000 7feaf6ffd640 1 -- 172.21.15.105:0/1143089153 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feafc0cf790 con 0x7feb001007c0 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.394+0000 7feb05a28640 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f40879ade0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feafc0cd220 tx=0x7feafc01bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ec3ff539079648 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.394+0000 7feaebfff640 1 -- 172.21.15.105:0/2471001503 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc09b9c0 con 0x55f4087962e0 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.394+0000 7feaebfff640 1 -- 172.21.15.105:0/2471001503 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feafc003b30 con 0x55f4087962e0 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.394+0000 7feaebfff640 1 -- 172.21.15.105:0/2471001503 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc09b7e0 con 0x55f4087962e0 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 -- 172.21.15.105:0/2471001503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 msgr2=0x55f40879ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb06229640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feaec0a2f20 0x7feaec0a53e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feaf8005e30 tx=0x7feaf802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f40879ade0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feafc0cd220 tx=0x7feafc01bb30 comp rx=0 tx=0).stop 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 -- 172.21.15.105:0/2471001503 shutdown_connections 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f40879d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f40879ade0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 --2- 172.21.15.105:0/2471001503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087958f0 0x55f408795d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 -- 172.21.15.105:0/2471001503 >> 172.21.15.105:0/2471001503 conn(0x55f408790e60 msgr2=0x55f408791260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 -- 172.21.15.105:0/2471001503 shutdown_connections 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 -- 172.21.15.105:0/2471001503 wait complete. 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 -- start start 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.395+0000 7feb0a87e740 1 Processor -- start 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087958f0 0x55f40879fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f40879ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f4087a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb0a87e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f40879e3a0 con 0x55f4087962e0 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb0a87e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f4086b6160 con 0x55f4087958f0 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb0a87e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f40879e520 con 0x55f40879b450 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05227640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f40879ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087958f0 0x55f40879fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087958f0 0x55f40879fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59404/0 (socket says 172.21.15.105:59404) 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 -- 172.21.15.105:0/515936939 learned_addr learned my addr 172.21.15.105:0/515936939 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb0a87e740 10 client.0 _ll_register_callbacks cb 0x55f4086f2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb06229640 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f4087a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 -- 172.21.15.105:0/515936939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 msgr2=0x55f4087a6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f4087a6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 -- 172.21.15.105:0/515936939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 msgr2=0x55f40879ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f40879ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 -- 172.21.15.105:0/515936939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f40840e3f0 con 0x55f4087958f0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05227640 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f40879ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feb05a28640 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087958f0 0x55f40879fa40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feafc0cd2a0 tx=0x7feafc0cfb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87debe4fc1c80864 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feae97fa640 1 -- 172.21.15.105:0/515936939 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc09b7e0 con 0x55f4087958f0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 1 -- 172.21.15.105:0/515936939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f408707ec0 con 0x55f4087958f0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.396+0000 7feae97fa640 1 -- 172.21.15.105:0/515936939 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feafc0b0190 con 0x55f4087958f0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feae97fa640 1 -- 172.21.15.105:0/515936939 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feafc09b9c0 con 0x55f4087958f0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 10 client.27785 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 1 -- 172.21.15.105:0/515936939 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4083ce0f0 con 0x55f4087958f0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feae97fa640 1 -- 172.21.15.105:0/515936939 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f408707ec0 con 0x55f4087958f0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feae97fa640 1 -- 172.21.15.105:0/515936939 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7feafc09c0e0 con 0x55f4087958f0 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 10 client.27785 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 10 client.27785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 20 client.27785 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:04.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f4087ab490 0x55f4087ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 4 client.27785 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:04.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb0a87e740 1 -- 172.21.15.105:0/515936939 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f408702210 con 0x55f4087ab490 2024-03-26T17:45:04.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.397+0000 7feb05227640 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f4087ab490 0x55f4087ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.398+0000 7feae97fa640 10 client.27785 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:04.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.398+0000 7feb05227640 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f4087ab490 0x55f4087ad880 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8314093b24bd508 server_cookie=9382bbb789db8c65 in_seq=0 out_seq=0 2024-03-26T17:45:04.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feae97fa640 1 -- 172.21.15.105:0/515936939 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f408702210 con 0x55f4087ab490 2024-03-26T17:45:04.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feae97fa640 10 client.27785 handle_command_reply: tid=0 2024-03-26T17:45:04.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 client.27785 shutdown 2024-03-26T17:45:04.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 -- 172.21.15.105:0/515936939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087958f0 msgr2=0x55f40879fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087958f0 0x55f40879fa40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feafc0cd2a0 tx=0x7feafc0cfb00 comp rx=0 tx=0).stop 2024-03-26T17:45:04.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 -- 172.21.15.105:0/515936939 shutdown_connections 2024-03-26T17:45:04.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f4087ab490 0x55f4087ad880 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f40879b450 0x55f4087a6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f40879ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 --2- 172.21.15.105:0/515936939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087958f0 0x55f40879fa40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feae97fa640 0 client.27785 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:04.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 -- 172.21.15.105:0/515936939 >> 172.21.15.105:0/515936939 conn(0x55f408790e60 msgr2=0x55f408790080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 -- 172.21.15.105:0/515936939 shutdown_connections 2024-03-26T17:45:04.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 1 -- 172.21.15.105:0/515936939 wait complete. 2024-03-26T17:45:04.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.399+0000 7feb0a87e740 20 client.27785 trim_cache size 0 max 16384 2024-03-26T17:45:04.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.417+0000 7feb0a87e740 1 Processor -- start 2024-03-26T17:45:04.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.417+0000 7feb0a87e740 1 -- start start 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f40879fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f4087a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f40879b450 0x55f408791240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb0a87e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb001af4a0 con 0x55f40879b450 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb0a87e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb001b0fe0 con 0x55f4087962e0 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb0a87e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb001afb70 con 0x55f4087958f0 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f4087a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb06229640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f40879b450 0x55f408791240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f4087a6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59420/0 (socket says 172.21.15.105:59420) 2024-03-26T17:45:04.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 -- 172.21.15.105:0/959795697 learned_addr learned my addr 172.21.15.105:0/959795697 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05a28640 1 --2- 172.21.15.105:0/959795697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f40879fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 -- 172.21.15.105:0/959795697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 msgr2=0x55f40879fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 --2- 172.21.15.105:0/959795697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f40879fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 -- 172.21.15.105:0/959795697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f40879b450 msgr2=0x55f408791240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 --2- 172.21.15.105:0/959795697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f40879b450 0x55f408791240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 -- 172.21.15.105:0/959795697 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4087ade60 con 0x55f4087962e0 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feb05227640 1 --2- 172.21.15.105:0/959795697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f4087a6f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f4087a0f20 tx=0x7feaf0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6691b782373b4b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feaeb7fe640 1 -- 172.21.15.105:0/959795697 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00120d0 con 0x55f4087962e0 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feaeb7fe640 1 -- 172.21.15.105:0/959795697 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf0012270 con 0x55f4087962e0 2024-03-26T17:45:04.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.418+0000 7feaeb7fe640 1 -- 172.21.15.105:0/959795697 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0005580 con 0x55f4087962e0 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 -- 172.21.15.105:0/959795697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 msgr2=0x55f4087a6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 --2- 172.21.15.105:0/959795697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f4087a6f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f4087a0f20 tx=0x7feaf0005f90 comp rx=0 tx=0).stop 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 -- 172.21.15.105:0/959795697 shutdown_connections 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 --2- 172.21.15.105:0/959795697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f40879b450 0x55f408791240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 --2- 172.21.15.105:0/959795697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4087962e0 0x55f4087a6f80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 --2- 172.21.15.105:0/959795697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f40879fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 -- 172.21.15.105:0/959795697 >> 172.21.15.105:0/959795697 conn(0x55f408790e60 msgr2=0x55f40878f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 -- 172.21.15.105:0/959795697 shutdown_connections 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 -- 172.21.15.105:0/959795697 wait complete. 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.419+0000 7feb0a87e740 1 -- start start 2024-03-26T17:45:04.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb0a87e740 1 Processor -- start 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f4087a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f4087a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb0a87e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f40879b450 0x55f4087a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb0a87e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb001b0910 con 0x55f4087962e0 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb0a87e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb001b0240 con 0x55f40879b450 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb0a87e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb001ad480 con 0x55f4087958f0 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb0a87e740 10 client.0 _ll_register_callbacks cb 0x7feafc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05227640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f4087a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05a28640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f4087a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb06229640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f40879b450 0x55f4087a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05a28640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f4087a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42674/0 (socket says 172.21.15.105:42674) 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05a28640 1 -- 172.21.15.105:0/632685139 learned_addr learned my addr 172.21.15.105:0/632685139 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05a28640 1 -- 172.21.15.105:0/632685139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f40879b450 msgr2=0x55f4087a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05a28640 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f40879b450 0x55f4087a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05a28640 1 -- 172.21.15.105:0/632685139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 msgr2=0x55f4087a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05a28640 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f4087a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.420+0000 7feb05a28640 1 -- 172.21.15.105:0/632685139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4087907b0 con 0x55f4087958f0 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb05a28640 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f4087a4690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feafc0cd5c0 tx=0x7feafc0cd2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f7f8a61deb0957a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feae97fa640 1 -- 172.21.15.105:0/632685139 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc06b070 con 0x55f4087958f0 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feae97fa640 1 -- 172.21.15.105:0/632685139 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7feafc09b2d0 con 0x55f4087958f0 2024-03-26T17:45:04.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 1 -- 172.21.15.105:0/632685139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf0012270 con 0x55f4087958f0 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feae97fa640 1 -- 172.21.15.105:0/632685139 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feafc09b7e0 con 0x55f4087958f0 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 10 client.27793 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 1 -- 172.21.15.105:0/632685139 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4087a7270 con 0x55f4087958f0 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feae97fa640 1 -- 172.21.15.105:0/632685139 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf0012270 con 0x55f4087958f0 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feae97fa640 1 -- 172.21.15.105:0/632685139 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7feafc0a1160 con 0x55f4087958f0 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 10 client.27793 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 10 client.27793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 20 client.27793 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f408811d00 0x55f408814120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 4 client.27793 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:45:04.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.421+0000 7feb0a87e740 1 -- 172.21.15.105:0/632685139 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f40867f540 con 0x55f408811d00 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.422+0000 7feb05227640 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f408811d00 0x55f408814120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.422+0000 7feb05227640 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f408811d00 0x55f408814120 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42932d7c0ed2e4bb server_cookie=e2157e6359e6eba8 in_seq=0 out_seq=0 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.422+0000 7feae97fa640 10 client.27793 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.422+0000 7feae97fa640 1 -- 172.21.15.105:0/632685139 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f40867f540 con 0x55f408811d00 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.422+0000 7feae97fa640 10 client.27793 handle_command_reply: tid=0 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.422+0000 7feb0a87e740 1 client.27793 shutdown 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.422+0000 7feb0a87e740 1 -- 172.21.15.105:0/632685139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 msgr2=0x55f4087a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.422+0000 7feb0a87e740 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f4087a4690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feafc0cd5c0 tx=0x7feafc0cd2d0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 1 -- 172.21.15.105:0/632685139 shutdown_connections 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f408811d00 0x55f408814120 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f40879b450 0x55f4087a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f4087962e0 0x55f4087a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feae97fa640 0 client.27793 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:04.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 1 --2- 172.21.15.105:0/632685139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f4087958f0 0x55f4087a4690 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 1 -- 172.21.15.105:0/632685139 >> 172.21.15.105:0/632685139 conn(0x55f408790e60 msgr2=0x55f408790080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 1 -- 172.21.15.105:0/632685139 shutdown_connections 2024-03-26T17:45:04.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 1 -- 172.21.15.105:0/632685139 wait complete. 2024-03-26T17:45:04.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb0a87e740 20 client.27793 trim_cache size 0 max 16384 2024-03-26T17:45:04.423 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout: "cd636fc3-76e0-44e7-9029-94b48ab9310c": { 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "cd636fc3-76e0-44e7-9029-94b48ab9310c", 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:45:04.424 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:45:04.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 -- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feaec0a2f20 msgr2=0x7feaec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feaec0a2f20 0x7feaec0a53e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7feaf8005e30 tx=0x7feaf802e040 comp rx=0 tx=0).stop 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 -- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb001007c0 msgr2=0x7feb0010b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb001007c0 0x7feb0010b140 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e9c0 tx=0x7feafc00ee80 comp rx=0 tx=0).stop 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 -- 172.21.15.105:0/1143089153 shutdown_connections 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feaec0a2f20 0x7feaec0a53e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb00102370 0x7feb00109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb001019b0 0x7feb00109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 --2- 172.21.15.105:0/1143089153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb001007c0 0x7feb0010b140 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 -- 172.21.15.105:0/1143089153 >> 172.21.15.105:0/1143089153 conn(0x7feb000fbd50 msgr2=0x7feb00104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 -- 172.21.15.105:0/1143089153 shutdown_connections 2024-03-26T17:45:04.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:04.423+0000 7feb07c8c640 1 -- 172.21.15.105:0/1143089153 wait complete. 2024-03-26T17:45:04.428 INFO:tasks.cephfs.filesystem:scrub status for tag:cd636fc3-76e0-44e7-9029-94b48ab9310c - {'path': '/', 'tag': 'cd636fc3-76e0-44e7-9029-94b48ab9310c', 'options': 'recursive,force'} 2024-03-26T17:45:05.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 204791 3.14 MB/sec execute 228 sec latency 17.979 ms 2024-03-26T17:45:06.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 205506 3.15 MB/sec execute 229 sec latency 23.234 ms 2024-03-26T17:45:06.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:05 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:45:06.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:05 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:06.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:05 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:45:06.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:05 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:06.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:05 smithi144 ceph-mon[36095]: pgmap v550: 97 pgs: 97 active+clean; 28 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:45:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:05 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:45:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:05 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:05 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:45:06.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:05 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:06.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:05 smithi105 ceph-mon[32280]: pgmap v550: 97 pgs: 97 active+clean; 28 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:45:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:05 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:45:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:05 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:05 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:45:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:05 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:05 smithi135 ceph-mon[35636]: pgmap v550: 97 pgs: 97 active+clean; 28 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:45:07.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 206024 3.16 MB/sec execute 230 sec latency 17.876 ms 2024-03-26T17:45:08.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 206424 3.15 MB/sec execute 231 sec latency 25.425 ms 2024-03-26T17:45:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:07 smithi144 ceph-mon[36095]: pgmap v551: 97 pgs: 97 active+clean; 24 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:07 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:45:08.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:07 smithi105 ceph-mon[32280]: pgmap v551: 97 pgs: 97 active+clean; 24 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:08.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:07 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:45:08.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:07 smithi135 ceph-mon[35636]: pgmap v551: 97 pgs: 97 active+clean; 24 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:08.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:07 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:45:09.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 207020 3.14 MB/sec execute 232 sec latency 16.541 ms 2024-03-26T17:45:10.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 207747 3.14 MB/sec execute 233 sec latency 16.425 ms 2024-03-26T17:45:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:09 smithi144 ceph-mon[36095]: pgmap v552: 97 pgs: 97 active+clean; 25 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:45:10.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:09 smithi105 ceph-mon[32280]: pgmap v552: 97 pgs: 97 active+clean; 25 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:45:10.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:09 smithi135 ceph-mon[35636]: pgmap v552: 97 pgs: 97 active+clean; 25 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:45:11.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 208299 3.14 MB/sec execute 234 sec latency 72.119 ms 2024-03-26T17:45:12.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 209042 3.15 MB/sec execute 235 sec latency 21.732 ms 2024-03-26T17:45:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:11 smithi144 ceph-mon[36095]: pgmap v553: 97 pgs: 97 active+clean; 26 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:45:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:11 smithi105 ceph-mon[32280]: pgmap v553: 97 pgs: 97 active+clean; 26 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:45:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:11 smithi135 ceph-mon[35636]: pgmap v553: 97 pgs: 97 active+clean; 26 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:45:13.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 209582 3.16 MB/sec execute 236 sec latency 15.506 ms 2024-03-26T17:45:14.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 210011 3.15 MB/sec execute 237 sec latency 27.071 ms 2024-03-26T17:45:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:13 smithi144 ceph-mon[36095]: pgmap v554: 97 pgs: 97 active+clean; 27 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T17:45:14.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:13 smithi105 ceph-mon[32280]: pgmap v554: 97 pgs: 97 active+clean; 27 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T17:45:14.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:13 smithi135 ceph-mon[35636]: pgmap v554: 97 pgs: 97 active+clean; 27 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-03-26T17:45:15.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 210662 3.15 MB/sec execute 238 sec latency 16.470 ms 2024-03-26T17:45:16.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 211197 3.14 MB/sec execute 239 sec latency 105.674 ms 2024-03-26T17:45:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:15 smithi144 ceph-mon[36095]: pgmap v555: 97 pgs: 97 active+clean; 25 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:45:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 -- 172.21.15.105:0/2157677067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701007e0 msgr2=0x7fcd70100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 --2- 172.21.15.105:0/2157677067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701007e0 0x7fcd70100c00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009900 tx=0x7fcd6c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 -- 172.21.15.105:0/2157677067 shutdown_connections 2024-03-26T17:45:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 --2- 172.21.15.105:0/2157677067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd70102390 0x7fcd70108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 --2- 172.21.15.105:0/2157677067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd701019d0 0x7fcd70101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 --2- 172.21.15.105:0/2157677067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701007e0 0x7fcd70100c00 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 -- 172.21.15.105:0/2157677067 >> 172.21.15.105:0/2157677067 conn(0x7fcd700fbd50 msgr2=0x7fcd700fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:16.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 -- 172.21.15.105:0/2157677067 shutdown_connections 2024-03-26T17:45:16.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.148+0000 7fcd78063640 1 -- 172.21.15.105:0/2157677067 wait complete. 2024-03-26T17:45:16.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:15 smithi105 ceph-mon[32280]: pgmap v555: 97 pgs: 97 active+clean; 25 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:45:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd78063640 1 Processor -- start 2024-03-26T17:45:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd78063640 1 -- start start 2024-03-26T17:45:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd78063640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd701007e0 0x7fcd70196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd78063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701019d0 0x7fcd70196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd78063640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd70102390 0x7fcd7019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd78063640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7010c2e0 con 0x7fcd701019d0 2024-03-26T17:45:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd78063640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7010c160 con 0x7fcd701007e0 2024-03-26T17:45:16.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd78063640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7010c460 con 0x7fcd70102390 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd755fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701019d0 0x7fcd70196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd755fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701019d0 0x7fcd70196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60296/0 (socket says 172.21.15.105:60296) 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd755fe640 1 -- 172.21.15.105:0/1901617356 learned_addr learned my addr 172.21.15.105:0/1901617356 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd75dff640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd701007e0 0x7fcd70196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd76600640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd70102390 0x7fcd7019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd755fe640 1 -- 172.21.15.105:0/1901617356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd70102390 msgr2=0x7fcd7019df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd755fe640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd70102390 0x7fcd7019df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd755fe640 1 -- 172.21.15.105:0/1901617356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd701007e0 msgr2=0x7fcd70196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd755fe640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd701007e0 0x7fcd70196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd755fe640 1 -- 172.21.15.105:0/1901617356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd70068f40 con 0x7fcd701019d0 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.149+0000 7fcd75dff640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd701007e0 0x7fcd70196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.150+0000 7fcd755fe640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701019d0 0x7fcd70196ec0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000e9f0 tx=0x7fcd6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.150+0000 7fcd66ffd640 1 -- 172.21.15.105:0/1901617356 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000cd80 con 0x7fcd701019d0 2024-03-26T17:45:16.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.150+0000 7fcd78063640 1 -- 172.21.15.105:0/1901617356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd70069230 con 0x7fcd701019d0 2024-03-26T17:45:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.150+0000 7fcd66ffd640 1 -- 172.21.15.105:0/1901617356 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd600108e0 con 0x7fcd701019d0 2024-03-26T17:45:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.150+0000 7fcd66ffd640 1 -- 172.21.15.105:0/1901617356 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60018920 con 0x7fcd701019d0 2024-03-26T17:45:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.150+0000 7fcd78063640 1 -- 172.21.15.105:0/1901617356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd700697d0 con 0x7fcd701019d0 2024-03-26T17:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd7ac48740 1 Processor -- start 2024-03-26T17:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd66ffd640 1 -- 172.21.15.105:0/1901617356 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcd60010a90 con 0x7fcd701019d0 2024-03-26T17:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd7ac48740 1 -- start start 2024-03-26T17:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53c5d0 0x55575b53c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b541ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b542130 0x55575b5445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55575b4a10b0 con 0x55575b53c5d0 2024-03-26T17:45:16.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55575b449960 con 0x55575b53cfc0 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55575b49d060 con 0x55575b542130 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd76600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53c5d0 0x55575b53c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd76600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53c5d0 0x55575b53c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60300/0 (socket says 172.21.15.105:60300) 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd76600640 1 -- 172.21.15.105:0/642936269 learned_addr learned my addr 172.21.15.105:0/642936269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.153+0000 7fcd75dff640 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b541ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd755fe640 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b542130 0x55575b5445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd75dff640 1 -- 172.21.15.105:0/642936269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b542130 msgr2=0x55575b5445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd75dff640 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b542130 0x55575b5445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd75dff640 1 -- 172.21.15.105:0/642936269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53c5d0 msgr2=0x55575b53c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd75dff640 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53c5d0 0x55575b53c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd75dff640 1 -- 172.21.15.105:0/642936269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575b45bc60 con 0x55575b53cfc0 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd76600640 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53c5d0 0x55575b53c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:16.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd66ffd640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd5c0a2fc0 0x7fcd5c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd75dff640 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b541ac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c036040 tx=0x7fcd6c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aadd0fc082f8cb99 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd76600640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd5c0a2fc0 0x7fcd5c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd5bfff640 1 -- 172.21.15.105:0/642936269 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c04c070 con 0x55575b53cfc0 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd66ffd640 1 -- 172.21.15.105:0/1901617356 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcd600cf700 con 0x7fcd701019d0 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd5bfff640 1 -- 172.21.15.105:0/642936269 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd6c02cc90 con 0x55575b53cfc0 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.154+0000 7fcd5bfff640 1 -- 172.21.15.105:0/642936269 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0046c0 con 0x55575b53cfc0 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 -- 172.21.15.105:0/642936269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 msgr2=0x55575b541ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd76600640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd5c0a2fc0 0x7fcd5c0a5480 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55575b53d820 tx=0x7fcd68009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b541ac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c036040 tx=0x7fcd6c009580 comp rx=0 tx=0).stop 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 -- 172.21.15.105:0/642936269 shutdown_connections 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b542130 0x55575b5445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b541ac0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/642936269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53c5d0 0x55575b53c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 -- 172.21.15.105:0/642936269 >> 172.21.15.105:0/642936269 conn(0x55575b537b40 msgr2=0x55575b537f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 -- 172.21.15.105:0/642936269 shutdown_connections 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.155+0000 7fcd7ac48740 1 -- 172.21.15.105:0/642936269 wait complete. 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 1 -- start start 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 1 Processor -- start 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b546390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b5468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b54d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55575b545080 con 0x55575b53cfc0 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55575b45cdb0 con 0x55575b542130 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55575b545200 con 0x55575b53c5d0 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd7ac48740 10 client.0 _ll_register_callbacks cb 0x55575b461110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd755fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b5468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd76600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b54d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd755fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b5468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60308/0 (socket says 172.21.15.105:60308) 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd75dff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b546390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.156+0000 7fcd755fe640 1 -- 172.21.15.105:0/284679749 learned_addr learned my addr 172.21.15.105:0/284679749 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd755fe640 1 -- 172.21.15.105:0/284679749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 msgr2=0x55575b546390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd755fe640 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b546390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd755fe640 1 -- 172.21.15.105:0/284679749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 msgr2=0x55575b54d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd755fe640 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b54d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd755fe640 1 -- 172.21.15.105:0/284679749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575b1b5210 con 0x55575b53cfc0 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd75dff640 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b546390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd76600640 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b54d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd755fe640 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b5468d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000e9c0 tx=0x7fcd60009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4749b829d48dc837 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd597fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd597fa640 1 -- 172.21.15.105:0/284679749 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6009b800 con 0x55575b53cfc0 2024-03-26T17:45:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd597fa640 1 -- 172.21.15.105:0/284679749 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcd600cfbb0 con 0x55575b53cfc0 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd7ac48740 1 -- 172.21.15.105:0/284679749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6c02cc90 con 0x55575b53cfc0 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd597fa640 1 -- 172.21.15.105:0/284679749 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd60010de0 con 0x55575b53cfc0 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd7ac48740 10 client.17973 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd7ac48740 1 -- 172.21.15.105:0/284679749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55575b281620 con 0x55575b53cfc0 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.157+0000 7fcd597fa640 1 -- 172.21.15.105:0/284679749 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd6c02cc90 con 0x55575b53cfc0 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd597fa640 1 -- 172.21.15.105:0/284679749 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcd600bab60 con 0x55575b53cfc0 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd7ac48740 10 client.17973 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd7ac48740 10 client.17973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd7ac48740 20 client.17973 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55575b5522f0 0x55575b554700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd7ac48740 4 client.17973 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd7ac48740 1 -- 172.21.15.105:0/284679749 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55575b4ae5f0 con 0x55575b5522f0 2024-03-26T17:45:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd75dff640 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55575b5522f0 0x55575b554700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd597fa640 10 client.17973 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.158+0000 7fcd75dff640 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55575b5522f0 0x55575b554700 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0061d0982a40996 server_cookie=cae676a6955b2f2 in_seq=0 out_seq=0 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd597fa640 1 -- 172.21.15.105:0/284679749 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55575b4ae5f0 con 0x55575b5522f0 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd597fa640 10 client.17973 handle_command_reply: tid=0 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 client.17973 shutdown 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 -- 172.21.15.105:0/284679749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 msgr2=0x55575b5468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b5468d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000e9c0 tx=0x7fcd60009310 comp rx=0 tx=0).stop 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 -- 172.21.15.105:0/284679749 shutdown_connections 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55575b5522f0 0x55575b554700 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b54d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b5468d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/284679749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b546390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd597fa640 0 client.17973 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 -- 172.21.15.105:0/284679749 >> 172.21.15.105:0/284679749 conn(0x55575b537b40 msgr2=0x55575b536df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 -- 172.21.15.105:0/284679749 shutdown_connections 2024-03-26T17:45:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 1 -- 172.21.15.105:0/284679749 wait complete. 2024-03-26T17:45:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.159+0000 7fcd7ac48740 20 client.17973 trim_cache size 0 max 16384 2024-03-26T17:45:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:15 smithi135 ceph-mon[35636]: pgmap v555: 97 pgs: 97 active+clean; 25 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:45:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd7ac48740 1 Processor -- start 2024-03-26T17:45:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd7ac48740 1 -- start start 2024-03-26T17:45:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b53e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b54d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b536560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701a6a90 con 0x55575b53cfc0 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701a85d0 con 0x55575b542130 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701a7160 con 0x55575b53c5d0 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd755fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b54d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd755fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b54d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60324/0 (socket says 172.21.15.105:60324) 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd76600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b536560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd755fe640 1 -- 172.21.15.105:0/282321850 learned_addr learned my addr 172.21.15.105:0/282321850 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd75dff640 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b53e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd755fe640 1 -- 172.21.15.105:0/282321850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 msgr2=0x55575b53e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd755fe640 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b53e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd755fe640 1 -- 172.21.15.105:0/282321850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 msgr2=0x55575b536560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd755fe640 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b536560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd755fe640 1 -- 172.21.15.105:0/282321850 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575b550bc0 con 0x55575b53cfc0 2024-03-26T17:45:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.176+0000 7fcd76600640 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b536560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.177+0000 7fcd755fe640 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b54d8d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c6a0 tx=0x7fcd6009a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0dde8e6ae7ab48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.177+0000 7fcd5b7fe640 1 -- 172.21.15.105:0/282321850 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6009b800 con 0x55575b53cfc0 2024-03-26T17:45:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.177+0000 7fcd5b7fe640 1 -- 172.21.15.105:0/282321850 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd6009ff40 con 0x55575b53cfc0 2024-03-26T17:45:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.177+0000 7fcd5b7fe640 1 -- 172.21.15.105:0/282321850 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6009bc70 con 0x55575b53cfc0 2024-03-26T17:45:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 -- 172.21.15.105:0/282321850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 msgr2=0x55575b54d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b54d8d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c6a0 tx=0x7fcd6009a3b0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 -- 172.21.15.105:0/282321850 shutdown_connections 2024-03-26T17:45:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b542130 0x55575b536560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b53cfc0 0x55575b54d8d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/282321850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b53e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 -- 172.21.15.105:0/282321850 >> 172.21.15.105:0/282321850 conn(0x55575b537b40 msgr2=0x55575b53da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 -- 172.21.15.105:0/282321850 shutdown_connections 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 -- 172.21.15.105:0/282321850 wait complete. 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 -- start start 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 Processor -- start 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b54afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.178+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b54b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd7ac48740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b542130 0x55575b54c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701a7f00 con 0x55575b542130 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701a7830 con 0x55575b53cfc0 2024-03-26T17:45:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd7ac48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701a4a70 con 0x55575b53c5d0 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd7ac48740 10 client.0 _ll_register_callbacks cb 0x7fcd600d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b54afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd755fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b54b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b54afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46868/0 (socket says 172.21.15.105:46868) 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 -- 172.21.15.105:0/4124751724 learned_addr learned my addr 172.21.15.105:0/4124751724 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd76600640 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b542130 0x55575b54c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 -- 172.21.15.105:0/4124751724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 msgr2=0x55575b54b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b54b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 -- 172.21.15.105:0/4124751724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b542130 msgr2=0x55575b54c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b542130 0x55575b54c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 -- 172.21.15.105:0/4124751724 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575b54cb10 con 0x55575b53c5d0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd75dff640 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b54afd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c005ab0 tx=0x7fcd6c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33387badb59e8223 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd597fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd597fa640 1 -- 172.21.15.105:0/4124751724 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c04c040 con 0x55575b53c5d0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd7ac48740 1 -- 172.21.15.105:0/4124751724 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6009ff40 con 0x55575b53c5d0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.179+0000 7fcd597fa640 1 -- 172.21.15.105:0/4124751724 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcd6c004470 con 0x55575b53c5d0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd597fa640 1 -- 172.21.15.105:0/4124751724 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd6c005d50 con 0x55575b53c5d0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd7ac48740 10 client.27811 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd7ac48740 1 -- 172.21.15.105:0/4124751724 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55575b5b8710 con 0x55575b53c5d0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd597fa640 1 -- 172.21.15.105:0/4124751724 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd6009ff40 con 0x55575b53c5d0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd597fa640 1 -- 172.21.15.105:0/4124751724 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcd6c005330 con 0x55575b53c5d0 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd7ac48740 10 client.27811 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd7ac48740 10 client.27811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd7ac48740 20 client.27811 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55575b5b8c80 0x55575b5bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd7ac48740 4 client.27811 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:45:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd7ac48740 1 -- 172.21.15.105:0/4124751724 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55575b4a8940 con 0x55575b5b8c80 2024-03-26T17:45:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd755fe640 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55575b5b8c80 0x55575b5bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.180+0000 7fcd597fa640 10 client.27811 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.181+0000 7fcd755fe640 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55575b5b8c80 0x55575b5bb070 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83bd8c6813a620bd server_cookie=44638db8b64dad94 in_seq=0 out_seq=0 2024-03-26T17:45:16.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd597fa640 1 -- 172.21.15.105:0/4124751724 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55575b4a8940 con 0x55575b5b8c80 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd597fa640 10 client.27811 handle_command_reply: tid=0 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 client.27811 shutdown 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 -- 172.21.15.105:0/4124751724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 msgr2=0x55575b54afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b54afd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c005ab0 tx=0x7fcd6c009580 comp rx=0 tx=0).stop 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 -- 172.21.15.105:0/4124751724 shutdown_connections 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55575b5b8c80 0x55575b5bb070 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55575b542130 0x55575b54c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55575b53cfc0 0x55575b54b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd597fa640 0 client.27811 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 --2- 172.21.15.105:0/4124751724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55575b53c5d0 0x55575b54afd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 -- 172.21.15.105:0/4124751724 >> 172.21.15.105:0/4124751724 conn(0x55575b537b40 msgr2=0x55575b53b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 -- 172.21.15.105:0/4124751724 shutdown_connections 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 1 -- 172.21.15.105:0/4124751724 wait complete. 2024-03-26T17:45:16.219 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.225+0000 7fcd7ac48740 20 client.27811 trim_cache size 0 max 16384 2024-03-26T17:45:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:45:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:45:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:45:16.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 -- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd5c0a2fc0 msgr2=0x7fcd5c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd5c0a2fc0 0x7fcd5c0a5480 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55575b53d820 tx=0x7fcd68009500 comp rx=0 tx=0).stop 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 -- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701019d0 msgr2=0x7fcd70196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701019d0 0x7fcd70196ec0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000e9f0 tx=0x7fcd6000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 -- 172.21.15.105:0/1901617356 shutdown_connections 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcd5c0a2fc0 0x7fcd5c0a5480 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd70102390 0x7fcd7019df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd701019d0 0x7fcd70196ec0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 --2- 172.21.15.105:0/1901617356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd701007e0 0x7fcd70196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 -- 172.21.15.105:0/1901617356 >> 172.21.15.105:0/1901617356 conn(0x7fcd700fbd50 msgr2=0x7fcd70104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 -- 172.21.15.105:0/1901617356 shutdown_connections 2024-03-26T17:45:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:16.226+0000 7fcd78063640 1 -- 172.21.15.105:0/1901617356 wait complete. 2024-03-26T17:45:17.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 211772 3.14 MB/sec execute 240 sec latency 31.869 ms 2024-03-26T17:45:18.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 212573 3.15 MB/sec execute 241 sec latency 16.890 ms 2024-03-26T17:45:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:17 smithi144 ceph-mon[36095]: pgmap v556: 97 pgs: 97 active+clean; 28 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T17:45:18.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:17 smithi105 ceph-mon[32280]: pgmap v556: 97 pgs: 97 active+clean; 28 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T17:45:18.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:17 smithi135 ceph-mon[35636]: pgmap v556: 97 pgs: 97 active+clean; 28 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-03-26T17:45:19.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 213055 3.16 MB/sec execute 242 sec latency 16.823 ms 2024-03-26T17:45:20.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 213435 3.15 MB/sec execute 243 sec latency 14.930 ms 2024-03-26T17:45:20.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:19 smithi144 ceph-mon[36095]: pgmap v557: 97 pgs: 97 active+clean; 29 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-03-26T17:45:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:19 smithi105 ceph-mon[32280]: pgmap v557: 97 pgs: 97 active+clean; 29 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-03-26T17:45:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:19 smithi135 ceph-mon[35636]: pgmap v557: 97 pgs: 97 active+clean; 29 MiB data, 809 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-03-26T17:45:21.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 213998 3.14 MB/sec execute 244 sec latency 21.368 ms 2024-03-26T17:45:22.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 214691 3.14 MB/sec execute 245 sec latency 16.490 ms 2024-03-26T17:45:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:21 smithi144 ceph-mon[36095]: pgmap v558: 97 pgs: 97 active+clean; 29 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-03-26T17:45:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:21 smithi105 ceph-mon[32280]: pgmap v558: 97 pgs: 97 active+clean; 29 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-03-26T17:45:22.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:21 smithi135 ceph-mon[35636]: pgmap v558: 97 pgs: 97 active+clean; 29 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-03-26T17:45:23.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 215323 3.14 MB/sec execute 246 sec latency 18.474 ms 2024-03-26T17:45:24.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 216165 3.16 MB/sec execute 247 sec latency 17.313 ms 2024-03-26T17:45:24.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:23 smithi144 ceph-mon[36095]: pgmap v559: 97 pgs: 97 active+clean; 30 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-03-26T17:45:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:23 smithi105 ceph-mon[32280]: pgmap v559: 97 pgs: 97 active+clean; 30 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-03-26T17:45:24.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:23 smithi135 ceph-mon[35636]: pgmap v559: 97 pgs: 97 active+clean; 30 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.5 MiB/s wr, 123 op/s 2024-03-26T17:45:25.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 216654 3.16 MB/sec execute 248 sec latency 15.936 ms 2024-03-26T17:45:26.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 217042 3.15 MB/sec execute 249 sec latency 15.030 ms 2024-03-26T17:45:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:25 smithi144 ceph-mon[36095]: pgmap v560: 97 pgs: 97 active+clean; 29 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:45:26.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:25 smithi105 ceph-mon[32280]: pgmap v560: 97 pgs: 97 active+clean; 29 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:45:26.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:25 smithi135 ceph-mon[35636]: pgmap v560: 97 pgs: 97 active+clean; 29 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:45:27.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 217601 3.14 MB/sec execute 250 sec latency 29.275 ms 2024-03-26T17:45:28.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 218283 3.15 MB/sec execute 251 sec latency 17.075 ms 2024-03-26T17:45:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:27 smithi144 ceph-mon[36095]: pgmap v561: 97 pgs: 97 active+clean; 31 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:45:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:27 smithi105 ceph-mon[32280]: pgmap v561: 97 pgs: 97 active+clean; 31 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:45:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:27 smithi135 ceph-mon[35636]: pgmap v561: 97 pgs: 97 active+clean; 31 MiB data, 830 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:45:29.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 218894 3.14 MB/sec execute 252 sec latency 16.530 ms 2024-03-26T17:45:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 -- 172.21.15.105:0/2954689903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 msgr2=0x7f5d30100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 --2- 172.21.15.105:0/2954689903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 0x7f5d30100c00 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000fb10 tx=0x7f5d2002fdc0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 -- 172.21.15.105:0/2954689903 shutdown_connections 2024-03-26T17:45:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 --2- 172.21.15.105:0/2954689903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d30102390 0x7f5d30108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 --2- 172.21.15.105:0/2954689903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d301019d0 0x7f5d30101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 --2- 172.21.15.105:0/2954689903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 0x7f5d30100c00 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 -- 172.21.15.105:0/2954689903 >> 172.21.15.105:0/2954689903 conn(0x7f5d300fbd50 msgr2=0x7f5d300fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 -- 172.21.15.105:0/2954689903 shutdown_connections 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.327+0000 7f5d349b9640 1 -- 172.21.15.105:0/2954689903 wait complete. 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.328+0000 7f5d349b9640 1 Processor -- start 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.328+0000 7f5d349b9640 1 -- start start 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d349b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 0x7f5d30195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d349b9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d301019d0 0x7f5d30195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d349b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d30102390 0x7f5d3019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d349b9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3010b1c0 con 0x7f5d301007e0 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d349b9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3010b040 con 0x7f5d301019d0 2024-03-26T17:45:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d349b9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3010b340 con 0x7f5d30102390 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 0x7f5d30195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 0x7f5d30195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52290/0 (socket says 172.21.15.105:52290) 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 -- 172.21.15.105:0/1356169929 learned_addr learned my addr 172.21.15.105:0/1356169929 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d27fff640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d301019d0 0x7f5d30195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d30102390 0x7f5d3019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 -- 172.21.15.105:0/1356169929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d30102390 msgr2=0x7f5d3019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d30102390 0x7f5d3019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 -- 172.21.15.105:0/1356169929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d301019d0 msgr2=0x7f5d30195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d301019d0 0x7f5d30195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 -- 172.21.15.105:0/1356169929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3019d450 con 0x7f5d301007e0 2024-03-26T17:45:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d27fff640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d301019d0 0x7f5d30195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d2e59c640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 0x7f5d30195880 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000f370 tx=0x7f5d2000f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.329+0000 7f5d277fe640 1 -- 172.21.15.105:0/1356169929 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20045030 con 0x7f5d301007e0 2024-03-26T17:45:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.330+0000 7f5d349b9640 1 -- 172.21.15.105:0/1356169929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3019d6e0 con 0x7f5d301007e0 2024-03-26T17:45:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.330+0000 7f5d277fe640 1 -- 172.21.15.105:0/1356169929 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d20004020 con 0x7f5d301007e0 2024-03-26T17:45:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.330+0000 7f5d277fe640 1 -- 172.21.15.105:0/1356169929 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d200309c0 con 0x7f5d301007e0 2024-03-26T17:45:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.330+0000 7f5d349b9640 1 -- 172.21.15.105:0/1356169929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3019dc20 con 0x7f5d301007e0 2024-03-26T17:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d375a6740 1 Processor -- start 2024-03-26T17:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d277fe640 1 -- 172.21.15.105:0/1356169929 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5d20048070 con 0x7f5d301007e0 2024-03-26T17:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d375a6740 1 -- start start 2024-03-26T17:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa111025d0 0x55fa111029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa11107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11108130 0x55fa1110a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d375a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa110670b0 con 0x55fa11108130 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d375a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fa1100f960 con 0x55fa111025d0 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.332+0000 7f5d375a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa11063060 con 0x55fa11102fc0 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa111025d0 0x55fa111029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11108130 0x55fa1110a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa111025d0 0x55fa111029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50190/0 (socket says 172.21.15.105:50190) 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa11107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/3823952938 learned_addr learned my addr 172.21.15.105:0/3823952938 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/3823952938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 msgr2=0x55fa11107ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/3823952938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa11107ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/3823952938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11108130 msgr2=0x55fa1110a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/3823952938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11108130 0x55fa1110a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/3823952938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa11021c60 con 0x55fa111025d0 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d27fff640 1 --2- 172.21.15.105:0/3823952938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11108130 0x55fa1110a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d277fe640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5d1c0a2fc0 0x7f5d1c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2e59c640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5d1c0a2fc0 0x7f5d1c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/3823952938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa111025d0 0x55fa111029f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009fd0 tx=0x7f5d280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cccf3e3dc0c4d8d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d277fe640 1 -- 172.21.15.105:0/1356169929 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5d200f47e0 con 0x7f5d301007e0 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d24ff9640 1 -- 172.21.15.105:0/3823952938 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28004d20 con 0x55fa111025d0 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d24ff9640 1 -- 172.21.15.105:0/3823952938 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d28004050 con 0x55fa111025d0 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.333+0000 7f5d24ff9640 1 -- 172.21.15.105:0/3823952938 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28005280 con 0x55fa111025d0 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 -- 172.21.15.105:0/3823952938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa111025d0 msgr2=0x55fa111029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 --2- 172.21.15.105:0/3823952938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa111025d0 0x55fa111029f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009fd0 tx=0x7f5d280118d0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d2e59c640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5d1c0a2fc0 0x7f5d1c0a5480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5d200f2610 tx=0x7f5d200fa550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 -- 172.21.15.105:0/3823952938 shutdown_connections 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 --2- 172.21.15.105:0/3823952938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11108130 0x55fa1110a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 --2- 172.21.15.105:0/3823952938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa11107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 --2- 172.21.15.105:0/3823952938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa111025d0 0x55fa111029f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 -- 172.21.15.105:0/3823952938 >> 172.21.15.105:0/3823952938 conn(0x55fa110fdb40 msgr2=0x55fa110fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 -- 172.21.15.105:0/3823952938 shutdown_connections 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.334+0000 7f5d375a6740 1 -- 172.21.15.105:0/3823952938 wait complete. 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 1 -- start start 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 1 Processor -- start 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa1110c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa1110c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 0x55fa11113990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa1110b0e0 con 0x55fa111025d0 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fa11022db0 con 0x55fa11108130 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa1110b260 con 0x55fa11102fc0 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d375a6740 10 client.0 _ll_register_callbacks cb 0x55fa11027110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa1110c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa1110c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52308/0 (socket says 172.21.15.105:52308) 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/92422002 learned_addr learned my addr 172.21.15.105:0/92422002 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa1110c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 0x55fa11113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d27fff640 1 -- 172.21.15.105:0/92422002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 msgr2=0x55fa11113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d27fff640 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 0x55fa11113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d27fff640 1 -- 172.21.15.105:0/92422002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 msgr2=0x55fa1110c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d27fff640 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa1110c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.335+0000 7f5d27fff640 1 -- 172.21.15.105:0/92422002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa10d7b210 con 0x55fa11102fc0 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d27fff640 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa1110c950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5d30196d40 tx=0x7f5d1800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972c2c6ccd83051e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d15ffb640 1 -- 172.21.15.105:0/92422002 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d180120d0 con 0x55fa11102fc0 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d15ffb640 1 -- 172.21.15.105:0/92422002 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5d18004030 con 0x55fa11102fc0 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 1 -- 172.21.15.105:0/92422002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d28004050 con 0x55fa11102fc0 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d15ffb640 1 -- 172.21.15.105:0/92422002 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d18012270 con 0x55fa11102fc0 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 10 client.27823 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 1 -- 172.21.15.105:0/92422002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa10e47620 con 0x55fa11102fc0 2024-03-26T17:45:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d15ffb640 1 -- 172.21.15.105:0/92422002 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d28004050 con 0x55fa11102fc0 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d15ffb640 1 -- 172.21.15.105:0/92422002 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5d1801c460 con 0x55fa11102fc0 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 10 client.27823 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 10 client.27823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 20 client.27823 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa111182d0 0x55fa1111a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 4 client.27823 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.336+0000 7f5d375a6740 1 -- 172.21.15.105:0/92422002 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa110745f0 con 0x55fa111182d0 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.337+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa111182d0 0x55fa1111a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.337+0000 7f5d15ffb640 10 client.27823 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.337+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa111182d0 0x55fa1111a720 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61eb5bc9be02516e server_cookie=8f9abe0cb1fdb3f5 in_seq=0 out_seq=0 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d15ffb640 1 -- 172.21.15.105:0/92422002 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fa110745f0 con 0x55fa111182d0 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d15ffb640 10 client.27823 handle_command_reply: tid=0 2024-03-26T17:45:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 client.27823 shutdown 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 -- 172.21.15.105:0/92422002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 msgr2=0x55fa1110c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa1110c950 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5d30196d40 tx=0x7f5d1800f790 comp rx=0 tx=0).stop 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 -- 172.21.15.105:0/92422002 shutdown_connections 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa111182d0 0x55fa1111a720 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 0x55fa11113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d15ffb640 0 client.27823 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11102fc0 0x55fa1110c950 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 --2- 172.21.15.105:0/92422002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa1110c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 -- 172.21.15.105:0/92422002 >> 172.21.15.105:0/92422002 conn(0x55fa110fdb40 msgr2=0x55fa110fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 -- 172.21.15.105:0/92422002 shutdown_connections 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 1 -- 172.21.15.105:0/92422002 wait complete. 2024-03-26T17:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.338+0000 7f5d375a6740 20 client.27823 trim_cache size 0 max 16384 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.354+0000 7f5d375a6740 1 Processor -- start 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.354+0000 7f5d375a6740 1 -- start start 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.354+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa11103a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.354+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11102fc0 0x55fa11113950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.354+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11108130 0x55fa11103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.354+0000 7f5d375a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a6aa0 con 0x55fa111025d0 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.354+0000 7f5d375a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a85e0 con 0x55fa11102fc0 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.354+0000 7f5d375a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a7170 con 0x55fa11108130 2024-03-26T17:45:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa11103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa11103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52324/0 (socket says 172.21.15.105:52324) 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/1763284646 learned_addr learned my addr 172.21.15.105:0/1763284646 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d27fff640 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11102fc0 0x55fa11113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11108130 0x55fa11103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/1763284646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11108130 msgr2=0x55fa11103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11108130 0x55fa11103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/1763284646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11102fc0 msgr2=0x55fa11113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11102fc0 0x55fa11113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/1763284646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa1110b370 con 0x55fa111025d0 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa11103a60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009f70 tx=0x7f5d28003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84429d3251f226f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d17fff640 1 -- 172.21.15.105:0/1763284646 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28004520 con 0x55fa111025d0 2024-03-26T17:45:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d17fff640 1 -- 172.21.15.105:0/1763284646 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d28004cf0 con 0x55fa111025d0 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.355+0000 7f5d17fff640 1 -- 172.21.15.105:0/1763284646 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28005550 con 0x55fa111025d0 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- 172.21.15.105:0/1763284646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 msgr2=0x55fa11103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa11103a60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009f70 tx=0x7f5d28003f50 comp rx=0 tx=0).stop 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- 172.21.15.105:0/1763284646 shutdown_connections 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa11108130 0x55fa11103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11102fc0 0x55fa11113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1763284646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa111025d0 0x55fa11103a60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- 172.21.15.105:0/1763284646 >> 172.21.15.105:0/1763284646 conn(0x55fa110fdb40 msgr2=0x55fa11107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- 172.21.15.105:0/1763284646 shutdown_connections 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- 172.21.15.105:0/1763284646 wait complete. 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- start start 2024-03-26T17:45:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 Processor -- start 2024-03-26T17:45:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa111025d0 0x55fa11110c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11102fc0 0x55fa11111160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 0x55fa11112170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a7f10 con 0x55fa11102fc0 2024-03-26T17:45:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a7840 con 0x55fa11108130 2024-03-26T17:45:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d301a4a80 con 0x55fa111025d0 2024-03-26T17:45:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.356+0000 7f5d375a6740 10 client.0 _ll_register_callbacks cb 0x7f5d18009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d27fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11102fc0 0x55fa11111160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa111025d0 0x55fa11110c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa111025d0 0x55fa11110c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55582/0 (socket says 172.21.15.105:55582) 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d2ed9d640 1 -- 172.21.15.105:0/1773412672 learned_addr learned my addr 172.21.15.105:0/1773412672 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 0x55fa11112170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d27fff640 1 -- 172.21.15.105:0/1773412672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa111025d0 msgr2=0x55fa11110c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d27fff640 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa111025d0 0x55fa11110c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d27fff640 1 -- 172.21.15.105:0/1773412672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 msgr2=0x55fa11112170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d27fff640 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 0x55fa11112170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d27fff640 1 -- 172.21.15.105:0/1773412672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa11112760 con 0x55fa11102fc0 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d27fff640 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11102fc0 0x55fa11111160 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009730 tx=0x7f5d18009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec5374e1051f911 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d15ffb640 1 -- 172.21.15.105:0/1773412672 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d18034060 con 0x55fa11102fc0 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d15ffb640 1 -- 172.21.15.105:0/1773412672 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5d1801cd20 con 0x55fa11102fc0 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d375a6740 1 -- 172.21.15.105:0/1773412672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d28004cf0 con 0x55fa11102fc0 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d15ffb640 1 -- 172.21.15.105:0/1773412672 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d1802b070 con 0x55fa11102fc0 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d375a6740 10 client.17997 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.357+0000 7f5d375a6740 1 -- 172.21.15.105:0/1773412672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fa11113ae0 con 0x55fa11102fc0 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d15ffb640 1 -- 172.21.15.105:0/1773412672 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d28004cf0 con 0x55fa11102fc0 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d15ffb640 1 -- 172.21.15.105:0/1773412672 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5d180036c0 con 0x55fa11102fc0 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d375a6740 10 client.17997 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d375a6740 10 client.17997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d375a6740 20 client.17997 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa1117ec10 0x55fa11181030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d375a6740 4 client.17997 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d375a6740 1 -- 172.21.15.105:0/1773412672 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa1106e940 con 0x55fa1117ec10 2024-03-26T17:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.358+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa1117ec10 0x55fa11181030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.359+0000 7f5d15ffb640 10 client.17997 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.359+0000 7f5d2ed9d640 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa1117ec10 0x55fa11181030 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5a1ccf6ecd4076 server_cookie=7803f033350f8cbd in_seq=0 out_seq=0 2024-03-26T17:45:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d15ffb640 1 -- 172.21.15.105:0/1773412672 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa1106e940 con 0x55fa1117ec10 2024-03-26T17:45:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d15ffb640 10 client.17997 handle_command_reply: tid=0 2024-03-26T17:45:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 client.17997 shutdown 2024-03-26T17:45:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 -- 172.21.15.105:0/1773412672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11102fc0 msgr2=0x55fa11111160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11102fc0 0x55fa11111160 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009730 tx=0x7f5d18009420 comp rx=0 tx=0).stop 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 -- 172.21.15.105:0/1773412672 shutdown_connections 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa1117ec10 0x55fa11181030 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa11108130 0x55fa11112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa11102fc0 0x55fa11111160 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 --2- 172.21.15.105:0/1773412672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa111025d0 0x55fa11110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d15ffb640 0 client.17997 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 -- 172.21.15.105:0/1773412672 >> 172.21.15.105:0/1773412672 conn(0x55fa110fdb40 msgr2=0x55fa11103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 -- 172.21.15.105:0/1773412672 shutdown_connections 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.397+0000 7f5d375a6740 1 -- 172.21.15.105:0/1773412672 wait complete. 2024-03-26T17:45:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d375a6740 20 client.17997 trim_cache size 0 max 16384 2024-03-26T17:45:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:45:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:45:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:45:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:45:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 -- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5d1c0a2fc0 msgr2=0x7f5d1c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5d1c0a2fc0 0x7f5d1c0a5480 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5d200f2610 tx=0x7f5d200fa550 comp rx=0 tx=0).stop 2024-03-26T17:45:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 -- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 msgr2=0x7f5d30195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 0x7f5d30195880 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000f370 tx=0x7f5d2000f3a0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 -- 172.21.15.105:0/1356169929 shutdown_connections 2024-03-26T17:45:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5d1c0a2fc0 0x7f5d1c0a5480 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d30102390 0x7f5d3019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d301019d0 0x7f5d30195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 --2- 172.21.15.105:0/1356169929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301007e0 0x7f5d30195880 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 -- 172.21.15.105:0/1356169929 >> 172.21.15.105:0/1356169929 conn(0x7f5d300fbd50 msgr2=0x7f5d30104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 -- 172.21.15.105:0/1356169929 shutdown_connections 2024-03-26T17:45:29.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:29.398+0000 7f5d349b9640 1 -- 172.21.15.105:0/1356169929 wait complete. 2024-03-26T17:45:30.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 219709 3.16 MB/sec execute 253 sec latency 38.521 ms 2024-03-26T17:45:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:29 smithi144 ceph-mon[36095]: pgmap v562: 97 pgs: 97 active+clean; 31 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:45:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:29 smithi105 ceph-mon[32280]: pgmap v562: 97 pgs: 97 active+clean; 31 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:45:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:29 smithi135 ceph-mon[35636]: pgmap v562: 97 pgs: 97 active+clean; 31 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:45:31.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 220166 3.16 MB/sec execute 254 sec latency 16.108 ms 2024-03-26T17:45:32.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 220526 3.15 MB/sec execute 255 sec latency 15.883 ms 2024-03-26T17:45:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:31 smithi144 ceph-mon[36095]: pgmap v563: 97 pgs: 97 active+clean; 30 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:45:32.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:31 smithi105 ceph-mon[32280]: pgmap v563: 97 pgs: 97 active+clean; 30 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:45:32.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:31 smithi135 ceph-mon[35636]: pgmap v563: 97 pgs: 97 active+clean; 30 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:45:33.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 221076 3.15 MB/sec execute 256 sec latency 24.537 ms 2024-03-26T17:45:34.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 221710 3.15 MB/sec execute 257 sec latency 15.913 ms 2024-03-26T17:45:34.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:33 smithi144 ceph-mon[36095]: pgmap v564: 97 pgs: 97 active+clean; 26 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:45:34.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:33 smithi105 ceph-mon[32280]: pgmap v564: 97 pgs: 97 active+clean; 26 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:45:34.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:33 smithi135 ceph-mon[35636]: pgmap v564: 97 pgs: 97 active+clean; 26 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:45:34.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:45:34.429 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:45:34.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.556+0000 7fd260153640 1 -- 172.21.15.105:0/152467666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2581019b0 msgr2=0x7fd258101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.556+0000 7fd260153640 1 --2- 172.21.15.105:0/152467666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2581019b0 0x7fd258101e30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd24802bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:34.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.556+0000 7fd260153640 1 -- 172.21.15.105:0/152467666 shutdown_connections 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.556+0000 7fd260153640 1 --2- 172.21.15.105:0/152467666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd258102370 0x7fd258108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.556+0000 7fd260153640 1 --2- 172.21.15.105:0/152467666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2581019b0 0x7fd258101e30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.556+0000 7fd260153640 1 --2- 172.21.15.105:0/152467666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2581007c0 0x7fd258100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.556+0000 7fd260153640 1 -- 172.21.15.105:0/152467666 >> 172.21.15.105:0/152467666 conn(0x7fd2580fbd50 msgr2=0x7fd2580fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.556+0000 7fd260153640 1 -- 172.21.15.105:0/152467666 shutdown_connections 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 -- 172.21.15.105:0/152467666 wait complete. 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 Processor -- start 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 -- start start 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2581007c0 0x7fd258104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2581019b0 0x7fd258102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd258102370 0x7fd2581032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810c2f0 con 0x7fd258102370 2024-03-26T17:45:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810c170 con 0x7fd2581007c0 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd260153640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd25810c470 con 0x7fd2581019b0 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd25e6f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd258102370 0x7fd2581032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd25deef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2581007c0 0x7fd258104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd25e6f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd258102370 0x7fd2581032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51210/0 (socket says 172.21.15.105:51210) 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd25e6f0640 1 -- 172.21.15.105:0/2440935994 learned_addr learned my addr 172.21.15.105:0/2440935994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.557+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2581019b0 0x7fd258102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd25e6f0640 1 -- 172.21.15.105:0/2440935994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2581019b0 msgr2=0x7fd258102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd25e6f0640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2581019b0 0x7fd258102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd25e6f0640 1 -- 172.21.15.105:0/2440935994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2581007c0 msgr2=0x7fd258104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd25e6f0640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2581007c0 0x7fd258104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd25e6f0640 1 -- 172.21.15.105:0/2440935994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd258103b60 con 0x7fd258102370 2024-03-26T17:45:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd25deef640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2581007c0 0x7fd258104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:34.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd25e6f0640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd258102370 0x7fd2581032d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd25000eb10 tx=0x7fd25000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:34.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd24effd640 1 -- 172.21.15.105:0/2440935994 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250004b20 con 0x7fd258102370 2024-03-26T17:45:34.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd260153640 1 -- 172.21.15.105:0/2440935994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2581a1c60 con 0x7fd258102370 2024-03-26T17:45:34.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd24effd640 1 -- 172.21.15.105:0/2440935994 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd250004cc0 con 0x7fd258102370 2024-03-26T17:45:34.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd24effd640 1 -- 172.21.15.105:0/2440935994 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250010690 con 0x7fd258102370 2024-03-26T17:45:34.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.558+0000 7fd260153640 1 -- 172.21.15.105:0/2440935994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2581a2180 con 0x7fd258102370 2024-03-26T17:45:34.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.560+0000 7fd24effd640 1 -- 172.21.15.105:0/2440935994 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd250005020 con 0x7fd258102370 2024-03-26T17:45:34.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.561+0000 7fd24effd640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2440a2f20 0x7fd2440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.561+0000 7fd262e19740 1 Processor -- start 2024-03-26T17:45:34.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.561+0000 7fd25deef640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2440a2f20 0x7fd2440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.561+0000 7fd24effd640 1 -- 172.21.15.105:0/2440935994 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd2500cfa00 con 0x7fd258102370 2024-03-26T17:45:34.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd262e19740 1 -- start start 2024-03-26T17:45:34.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd262e19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd262e19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd262e19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd262e19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5627d7953980 con 0x5627d79f4450 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd262e19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5627d78fbbe0 con 0x5627d79ef2e0 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd262e19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5627d794f930 con 0x5627d79ee8f0 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55970/0 (socket says 172.21.15.105:55970) 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 -- 172.21.15.105:0/763752927 learned_addr learned my addr 172.21.15.105:0/763752927 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25e6f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25deef640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2440a2f20 0x7fd2440a53e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd258101810 tx=0x7fd254005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25deef640 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 -- 172.21.15.105:0/763752927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 msgr2=0x5627d79eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 -- 172.21.15.105:0/763752927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 msgr2=0x5627d79f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 -- 172.21.15.105:0/763752927 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d77317d0 con 0x5627d79ef2e0 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25deef640 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.562+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f3de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd248009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f492747ba40fd9f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd243fff640 1 -- 172.21.15.105:0/763752927 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248045020 con 0x5627d79ef2e0 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd243fff640 1 -- 172.21.15.105:0/763752927 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd248004040 con 0x5627d79ef2e0 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd243fff640 1 -- 172.21.15.105:0/763752927 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480055c0 con 0x5627d79ef2e0 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd262e19740 1 -- 172.21.15.105:0/763752927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 msgr2=0x5627d79f3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd262e19740 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f3de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd248009900 tx=0x7fd248009560 comp rx=0 tx=0).stop 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd262e19740 1 -- 172.21.15.105:0/763752927 shutdown_connections 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd262e19740 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd262e19740 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f3de0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.563+0000 7fd262e19740 1 --2- 172.21.15.105:0/763752927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.564+0000 7fd262e19740 1 -- 172.21.15.105:0/763752927 >> 172.21.15.105:0/763752927 conn(0x5627d79e9e60 msgr2=0x5627d79ea260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.564+0000 7fd262e19740 1 -- 172.21.15.105:0/763752927 shutdown_connections 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.564+0000 7fd262e19740 1 -- 172.21.15.105:0/763752927 wait complete. 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.564+0000 7fd262e19740 1 -- start start 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.564+0000 7fd262e19740 1 Processor -- start 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd262e19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd262e19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd262e19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79f4450 0x5627d7a00050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd262e19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5627d79f73a0 con 0x5627d79ee8f0 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd262e19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5627d790f160 con 0x5627d79ef2e0 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd262e19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5627d79f7520 con 0x5627d79f4450 2024-03-26T17:45:34.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51232/0 (socket says 172.21.15.105:51232) 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 -- 172.21.15.105:0/2515940629 learned_addr learned my addr 172.21.15.105:0/2515940629 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd262e19740 10 client.0 _ll_register_callbacks cb 0x5627d794b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25e6f0640 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79f4450 0x5627d7a00050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 -- 172.21.15.105:0/2515940629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79f4450 msgr2=0x5627d7a00050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79f4450 0x5627d7a00050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 -- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 msgr2=0x5627d79f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 -- 172.21.15.105:0/2515940629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d76673f0 con 0x5627d79ee8f0 2024-03-26T17:45:34.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd25deef640 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8ad0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd25400fad0 tx=0x7fd254004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d8d807113b66b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd2417fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd2417fa640 1 -- 172.21.15.105:0/2515940629 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25402a060 con 0x5627d79ee8f0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd262e19740 1 -- 172.21.15.105:0/2515940629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248004040 con 0x5627d79ee8f0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.565+0000 7fd2417fa640 1 -- 172.21.15.105:0/2515940629 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd254009040 con 0x5627d79ee8f0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd2417fa640 1 -- 172.21.15.105:0/2515940629 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd25401cd20 con 0x5627d79ee8f0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd262e19740 10 client.18009 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd262e19740 1 -- 172.21.15.105:0/2515940629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627d76270f0 con 0x5627d79ee8f0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd2417fa640 1 -- 172.21.15.105:0/2515940629 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd248004040 con 0x5627d79ee8f0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd2417fa640 1 -- 172.21.15.105:0/2515940629 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd254024070 con 0x5627d79ee8f0 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd262e19740 10 client.18009 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd262e19740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd262e19740 20 client.18009 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:34.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd262e19740 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5627d7a04a20 0x5627d7a06ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd262e19740 4 client.18009 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd262e19740 1 -- 172.21.15.105:0/2515940629 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627d7960ec0 con 0x5627d7a04a20 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.566+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5627d7a04a20 0x5627d7a06ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.567+0000 7fd2417fa640 10 client.18009 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.567+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5627d7a04a20 0x5627d7a06ec0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200403a0cf04d208 server_cookie=b204119c0a32aee2 in_seq=0 out_seq=0 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.567+0000 7fd2417fa640 1 -- 172.21.15.105:0/2515940629 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5627d7960ec0 con 0x5627d7a04a20 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.567+0000 7fd2417fa640 10 client.18009 handle_command_reply: tid=0 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.567+0000 7fd262e19740 1 client.18009 shutdown 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.567+0000 7fd262e19740 1 -- 172.21.15.105:0/2515940629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 msgr2=0x5627d79f8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.567+0000 7fd262e19740 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8ad0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd25400fad0 tx=0x7fd254004730 comp rx=0 tx=0).stop 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 1 -- 172.21.15.105:0/2515940629 shutdown_connections 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5627d7a04a20 0x5627d7a06ec0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79f4450 0x5627d7a00050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd2417fa640 0 client.18009 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:34.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 1 --2- 172.21.15.105:0/2515940629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8ad0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 1 -- 172.21.15.105:0/2515940629 >> 172.21.15.105:0/2515940629 conn(0x5627d79e9e60 msgr2=0x5627d79e9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:34.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 1 -- 172.21.15.105:0/2515940629 shutdown_connections 2024-03-26T17:45:34.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 1 -- 172.21.15.105:0/2515940629 wait complete. 2024-03-26T17:45:34.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.568+0000 7fd262e19740 20 client.18009 trim_cache size 0 max 16384 2024-03-26T17:45:34.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.586+0000 7fd262e19740 1 Processor -- start 2024-03-26T17:45:34.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.586+0000 7fd262e19740 1 -- start start 2024-03-26T17:45:34.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd262e19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd262e19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ef2e0 0x5627d7a00010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd262e19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79f4450 0x5627d79f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd262e19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581aae70 con 0x5627d79ee8f0 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd262e19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581ac9b0 con 0x5627d79f4450 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd262e19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581ab540 con 0x5627d79ef2e0 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51240/0 (socket says 172.21.15.105:51240) 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 -- 172.21.15.105:0/2239967914 learned_addr learned my addr 172.21.15.105:0/2239967914 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ef2e0 0x5627d7a00010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25e6f0640 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79f4450 0x5627d79f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 -- 172.21.15.105:0/2239967914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ef2e0 msgr2=0x5627d7a00010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ef2e0 0x5627d7a00010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 -- 172.21.15.105:0/2239967914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79f4450 msgr2=0x5627d79f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79f4450 0x5627d79f04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 -- 172.21.15.105:0/2239967914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d79effb0 con 0x5627d79ee8f0 2024-03-26T17:45:34.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.587+0000 7fd25deef640 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8f70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd25401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209a86e3ffb24ea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.588+0000 7fd2437fe640 1 -- 172.21.15.105:0/2239967914 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25402a070 con 0x5627d79ee8f0 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.588+0000 7fd2437fe640 1 -- 172.21.15.105:0/2239967914 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd25400f8a0 con 0x5627d79ee8f0 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.588+0000 7fd2437fe640 1 -- 172.21.15.105:0/2239967914 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254030650 con 0x5627d79ee8f0 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.588+0000 7fd262e19740 1 -- 172.21.15.105:0/2239967914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 msgr2=0x5627d79f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.588+0000 7fd262e19740 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8f70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd25401c890 comp rx=0 tx=0).stop 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 -- 172.21.15.105:0/2239967914 shutdown_connections 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79f4450 0x5627d79f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ef2e0 0x5627d7a00010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 --2- 172.21.15.105:0/2239967914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79ee8f0 0x5627d79f8f70 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 -- 172.21.15.105:0/2239967914 >> 172.21.15.105:0/2239967914 conn(0x5627d79e9e60 msgr2=0x5627d79ed0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 -- 172.21.15.105:0/2239967914 shutdown_connections 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 -- 172.21.15.105:0/2239967914 wait complete. 2024-03-26T17:45:34.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 -- start start 2024-03-26T17:45:34.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 Processor -- start 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79fdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581ac2e0 con 0x5627d79f4450 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581abc10 con 0x5627d79ef2e0 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2581a8e50 con 0x5627d79ee8f0 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.589+0000 7fd262e19740 10 client.0 _ll_register_callbacks cb 0x7fd254009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25d6ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79fdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51242/0 (socket says 172.21.15.105:51242) 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 -- 172.21.15.105:0/3835474455 learned_addr learned my addr 172.21.15.105:0/3835474455 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25deef640 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 -- 172.21.15.105:0/3835474455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 msgr2=0x5627d79fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:34.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 -- 172.21.15.105:0/3835474455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 msgr2=0x5627d79fdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:34.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79fdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 -- 172.21.15.105:0/3835474455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d79ff390 con 0x5627d79f4450 2024-03-26T17:45:34.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25d6ee640 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79fdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:34.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd25e6f0640 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79feda0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd2500049d0 tx=0x7fd25009c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e91709bc2accf4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd2417fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd2417fa640 1 -- 172.21.15.105:0/3835474455 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2500bae60 con 0x5627d79f4450 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd2417fa640 1 -- 172.21.15.105:0/3835474455 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd25009c550 con 0x5627d79f4450 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd262e19740 1 -- 172.21.15.105:0/3835474455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd25400f8a0 con 0x5627d79f4450 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd2417fa640 1 -- 172.21.15.105:0/3835474455 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd2500a4670 con 0x5627d79f4450 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd262e19740 10 client.18015 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.590+0000 7fd262e19740 1 -- 172.21.15.105:0/3835474455 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627d7a6aff0 con 0x5627d79f4450 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd2417fa640 1 -- 172.21.15.105:0/3835474455 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd25400f8a0 con 0x5627d79f4450 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd2417fa640 1 -- 172.21.15.105:0/3835474455 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd250097e40 con 0x5627d79f4450 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd262e19740 10 client.18015 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:34.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd262e19740 10 client.18015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd262e19740 20 client.18015 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd262e19740 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5627d7a6b9a0 0x5627d7a6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd262e19740 4 client.18015 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd262e19740 1 -- 172.21.15.105:0/3835474455 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627d795b210 con 0x5627d7a6b9a0 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.591+0000 7fd25deef640 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5627d7a6b9a0 0x5627d7a6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd2417fa640 10 client.18015 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd25deef640 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5627d7a6b9a0 0x5627d7a6ddc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc7c8a44d6df54a server_cookie=412cf00b55d44da1 in_seq=0 out_seq=0 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd2417fa640 1 -- 172.21.15.105:0/3835474455 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5627d795b210 con 0x5627d7a6b9a0 2024-03-26T17:45:34.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd2417fa640 10 client.18015 handle_command_reply: tid=0 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 client.18015 shutdown 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 -- 172.21.15.105:0/3835474455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 msgr2=0x5627d79feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79feda0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd2500049d0 tx=0x7fd25009c1b0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 -- 172.21.15.105:0/3835474455 shutdown_connections 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5627d7a6b9a0 0x5627d7a6ddc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5627d79f4450 0x5627d79feda0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5627d79ef2e0 0x5627d79fdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd2417fa640 0 client.18015 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 --2- 172.21.15.105:0/3835474455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5627d79ee8f0 0x5627d79fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 -- 172.21.15.105:0/3835474455 >> 172.21.15.105:0/3835474455 conn(0x5627d79e9e60 msgr2=0x5627d79e87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:34.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 -- 172.21.15.105:0/3835474455 shutdown_connections 2024-03-26T17:45:34.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 1 -- 172.21.15.105:0/3835474455 wait complete. 2024-03-26T17:45:34.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.592+0000 7fd262e19740 20 client.18015 trim_cache size 0 max 16384 2024-03-26T17:45:34.592 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:45:34.592 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:45:34.592 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:45:34.592 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:45:34.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 -- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2440a2f20 msgr2=0x7fd2440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2440a2f20 0x7fd2440a53e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd258101810 tx=0x7fd254005f90 comp rx=0 tx=0).stop 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 -- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd258102370 msgr2=0x7fd2581032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd258102370 0x7fd2581032d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd25000eb10 tx=0x7fd25000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 -- 172.21.15.105:0/2440935994 shutdown_connections 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2440a2f20 0x7fd2440a53e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd258102370 0x7fd2581032d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2581019b0 0x7fd258102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 --2- 172.21.15.105:0/2440935994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2581007c0 0x7fd258104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 -- 172.21.15.105:0/2440935994 >> 172.21.15.105:0/2440935994 conn(0x7fd2580fbd50 msgr2=0x7fd258068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 -- 172.21.15.105:0/2440935994 shutdown_connections 2024-03-26T17:45:34.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.593+0000 7fd260153640 1 -- 172.21.15.105:0/2440935994 wait complete. 2024-03-26T17:45:34.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:45:34.597 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 -- 172.21.15.105:0/3201326454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0084100800 msgr2=0x7f0084100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 --2- 172.21.15.105:0/3201326454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0084100800 0x7f0084100c20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0080009900 tx=0x7f008002bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 -- 172.21.15.105:0/3201326454 shutdown_connections 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 --2- 172.21.15.105:0/3201326454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00841023b0 0x7f0084108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 --2- 172.21.15.105:0/3201326454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00841019f0 0x7f0084101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 --2- 172.21.15.105:0/3201326454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0084100800 0x7f0084100c20 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 -- 172.21.15.105:0/3201326454 >> 172.21.15.105:0/3201326454 conn(0x7f00840fbd70 msgr2=0x7f00840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 -- 172.21.15.105:0/3201326454 shutdown_connections 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.731+0000 7f008b7f3640 1 -- 172.21.15.105:0/3201326454 wait complete. 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008b7f3640 1 Processor -- start 2024-03-26T17:45:34.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008b7f3640 1 -- start start 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008b7f3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0084100800 0x7f008406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008b7f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00841019f0 0x7f008419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008b7f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00841023b0 0x7f008419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008b7f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f008410c300 con 0x7f00841019f0 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008b7f3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f008410c180 con 0x7f0084100800 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008b7f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f008410c480 con 0x7f00841023b0 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0088d8e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00841019f0 0x7f008419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f008958f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0084100800 0x7f008406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0088d8e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00841019f0 0x7f008419c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51256/0 (socket says 172.21.15.105:51256) 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0088d8e640 1 -- 172.21.15.105:0/986419245 learned_addr learned my addr 172.21.15.105:0/986419245 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0089d90640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00841023b0 0x7f008419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0088d8e640 1 -- 172.21.15.105:0/986419245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00841023b0 msgr2=0x7f008419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0088d8e640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00841023b0 0x7f008419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0088d8e640 1 -- 172.21.15.105:0/986419245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0084100800 msgr2=0x7f008406a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0088d8e640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0084100800 0x7f008406a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.732+0000 7f0088d8e640 1 -- 172.21.15.105:0/986419245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f008419f7f0 con 0x7f00841019f0 2024-03-26T17:45:34.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.733+0000 7f0088d8e640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00841019f0 0x7f008419c8f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f007400e9f0 tx=0x7f007400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:34.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.733+0000 7f007a7fc640 1 -- 172.21.15.105:0/986419245 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007400ccc0 con 0x7f00841019f0 2024-03-26T17:45:34.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.733+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f008419fae0 con 0x7f00841019f0 2024-03-26T17:45:34.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.733+0000 7f007a7fc640 1 -- 172.21.15.105:0/986419245 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00740108e0 con 0x7f00841019f0 2024-03-26T17:45:34.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.733+0000 7f007a7fc640 1 -- 172.21.15.105:0/986419245 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00740187c0 con 0x7f00841019f0 2024-03-26T17:45:34.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.733+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00841a0080 con 0x7f00841019f0 2024-03-26T17:45:34.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.734+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0084107c60 con 0x7f00841019f0 2024-03-26T17:45:34.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.735+0000 7f007a7fc640 1 -- 172.21.15.105:0/986419245 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0074010a90 con 0x7f00841019f0 2024-03-26T17:45:34.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.736+0000 7f007a7fc640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00700a2f20 0x7f00700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:34.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.736+0000 7f007a7fc640 1 -- 172.21.15.105:0/986419245 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f00740cfc60 con 0x7f00841019f0 2024-03-26T17:45:34.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.736+0000 7f008958f640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00700a2f20 0x7f00700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:34.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.736+0000 7f008958f640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00700a2f20 0x7f00700a53e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0080005ab0 tx=0x7f0080005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:34.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.741+0000 7f007a7fc640 1 -- 172.21.15.105:0/986419245 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f007409ca50 con 0x7f00841019f0 2024-03-26T17:45:34.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.988+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f008402d3f0 con 0x7f00841019f0 2024-03-26T17:45:34.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.988+0000 7f007a7fc640 1 -- 172.21.15.105:0/986419245 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f007409c350 con 0x7f00841019f0 2024-03-26T17:45:34.980 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:45:34.980 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:45:34.981 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00700a2f20 msgr2=0x7f00700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00700a2f20 0x7f00700a53e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0080005ab0 tx=0x7f0080005fd0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00841019f0 msgr2=0x7f008419c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00841019f0 0x7f008419c8f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f007400e9f0 tx=0x7f007400edf0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 shutdown_connections 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00700a2f20 0x7f00700a53e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00841023b0 0x7f008419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00841019f0 0x7f008419c8f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 --2- 172.21.15.105:0/986419245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0084100800 0x7f008406a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:34.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 >> 172.21.15.105:0/986419245 conn(0x7f00840fbd70 msgr2=0x7f0084104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 shutdown_connections 2024-03-26T17:45:34.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:34.991+0000 7f008b7f3640 1 -- 172.21.15.105:0/986419245 wait complete. 2024-03-26T17:45:34.994 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:45:35.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 222338 3.14 MB/sec execute 258 sec latency 20.076 ms 2024-03-26T17:45:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 -- 172.21.15.105:0/3747132326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c104cd0 msgr2=0x7f691c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 --2- 172.21.15.105:0/3747132326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c104cd0 0x7f691c1070c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6910009900 tx=0x7f691002bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 -- 172.21.15.105:0/3747132326 shutdown_connections 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 --2- 172.21.15.105:0/3747132326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c107600 0x7f691c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 --2- 172.21.15.105:0/3747132326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c104cd0 0x7f691c1070c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 --2- 172.21.15.105:0/3747132326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f691c1023a0 0x7f691c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 -- 172.21.15.105:0/3747132326 >> 172.21.15.105:0/3747132326 conn(0x7f691c0fbd50 msgr2=0x7f691c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 -- 172.21.15.105:0/3747132326 shutdown_connections 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.117+0000 7f6922026640 1 -- 172.21.15.105:0/3747132326 wait complete. 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f6922026640 1 Processor -- start 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f6922026640 1 -- start start 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f6922026640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c1023a0 0x7f691c101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f6922026640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f691c104cd0 0x7f691c100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f6922026640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c107600 0x7f691c100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f6922026640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10d3f0 con 0x7f691c107600 2024-03-26T17:45:35.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f6922026640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10d270 con 0x7f691c104cd0 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f6922026640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f691c10d570 con 0x7f691c1023a0 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f691bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c107600 0x7f691c100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f691bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c107600 0x7f691c100890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51274/0 (socket says 172.21.15.105:51274) 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f691bfff640 1 -- 172.21.15.105:0/3169461386 learned_addr learned my addr 172.21.15.105:0/3169461386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f691affd640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f691c104cd0 0x7f691c100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.118+0000 7f691b7fe640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c1023a0 0x7f691c101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f691bfff640 1 -- 172.21.15.105:0/3169461386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c1023a0 msgr2=0x7f691c101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f691bfff640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c1023a0 0x7f691c101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f691bfff640 1 -- 172.21.15.105:0/3169461386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f691c104cd0 msgr2=0x7f691c100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f691bfff640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f691c104cd0 0x7f691c100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f691bfff640 1 -- 172.21.15.105:0/3169461386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c101120 con 0x7f691c107600 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f691bfff640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c107600 0x7f691c100890 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f691400eb10 tx=0x7f691400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f6918ff9640 1 -- 172.21.15.105:0/3169461386 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914004b20 con 0x7f691c107600 2024-03-26T17:45:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f6918ff9640 1 -- 172.21.15.105:0/3169461386 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6914004cc0 con 0x7f691c107600 2024-03-26T17:45:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f6922026640 1 -- 172.21.15.105:0/3169461386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c1a1cd0 con 0x7f691c107600 2024-03-26T17:45:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f6918ff9640 1 -- 172.21.15.105:0/3169461386 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914010690 con 0x7f691c107600 2024-03-26T17:45:35.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.119+0000 7f6922026640 1 -- 172.21.15.105:0/3169461386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1a21a0 con 0x7f691c107600 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.121+0000 7f6918ff9640 1 -- 172.21.15.105:0/3169461386 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6914005020 con 0x7f691c107600 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6924c0b740 1 Processor -- start 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6924c0b740 1 -- start start 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d668249a30 0x55d668249e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66824a420 0x55d66824ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d668251a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6924c0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d6681ae970 con 0x55d66824a420 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6924c0b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d668156d70 con 0x55d668249a30 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6924c0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d6681aa920 con 0x55d66824f590 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6918ff9640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69040a2f20 0x7f69040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d668249a30 0x55d668249e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66824a420 0x55d66824ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d668249a30 0x55d668249e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56026/0 (socket says 172.21.15.105:56026) 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 -- 172.21.15.105:0/770215815 learned_addr learned my addr 172.21.15.105:0/770215815 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d668251a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691bfff640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69040a2f20 0x7f69040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 -- 172.21.15.105:0/770215815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 msgr2=0x55d668251a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d668251a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 -- 172.21.15.105:0/770215815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66824a420 msgr2=0x55d66824ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66824a420 0x55d66824ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691affd640 1 -- 172.21.15.105:0/770215815 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d667ec23f0 con 0x55d668249a30 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f691b7fe640 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66824a420 0x55d66824ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.122+0000 7f6918ff9640 1 -- 172.21.15.105:0/3169461386 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f69140cfac0 con 0x7f691c107600 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.123+0000 7f691affd640 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d668249a30 0x55d668249e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f691002c840 tx=0x7f691002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8afecaf20f82da12 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.123+0000 7f691bfff640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69040a2f20 0x7f69040a53e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6914098b80 tx=0x7f691400efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.123+0000 7f6901ffb640 1 -- 172.21.15.105:0/770215815 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910004030 con 0x55d668249a30 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.123+0000 7f6901ffb640 1 -- 172.21.15.105:0/770215815 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6910005050 con 0x55d668249a30 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.123+0000 7f6901ffb640 1 -- 172.21.15.105:0/770215815 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69100049b0 con 0x55d668249a30 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 -- 172.21.15.105:0/770215815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d668249a30 msgr2=0x55d668249e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d668249a30 0x55d668249e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f691002c840 tx=0x7f691002bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 -- 172.21.15.105:0/770215815 shutdown_connections 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d668251a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66824a420 0x55d66824ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 --2- 172.21.15.105:0/770215815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d668249a30 0x55d668249e50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 -- 172.21.15.105:0/770215815 >> 172.21.15.105:0/770215815 conn(0x55d668244fa0 msgr2=0x55d6682453a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 -- 172.21.15.105:0/770215815 shutdown_connections 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 -- 172.21.15.105:0/770215815 wait complete. 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.124+0000 7f6924c0b740 1 -- start start 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f6924c0b740 1 Processor -- start 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668253f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824a420 0x55d6682544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824f590 0x55d66825b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f6924c0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d6682524e0 con 0x55d668249a30 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f6924c0b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d66816a310 con 0x55d66824f590 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f6924c0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d668252660 con 0x55d66824a420 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668253f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668253f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51304/0 (socket says 172.21.15.105:51304) 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 -- 172.21.15.105:0/1687855876 learned_addr learned my addr 172.21.15.105:0/1687855876 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f6924c0b740 10 client.0 _ll_register_callbacks cb 0x55d6681a5c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824f590 0x55d66825b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691b7fe640 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824a420 0x55d6682544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 -- 172.21.15.105:0/1687855876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824a420 msgr2=0x55d6682544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824a420 0x55d6682544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 -- 172.21.15.105:0/1687855876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824f590 msgr2=0x55d66825b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824f590 0x55d66825b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 -- 172.21.15.105:0/1687855876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d667f8c7d0 con 0x55d668249a30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f691affd640 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668253f80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f691002ccd0 tx=0x7f6910005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2665f424c567eabc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.125+0000 7f68faffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f68faffd640 1 -- 172.21.15.105:0/1687855876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910004d70 con 0x55d668249a30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f68faffd640 1 -- 172.21.15.105:0/1687855876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f69100035f0 con 0x55d668249a30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 1 -- 172.21.15.105:0/1687855876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6910005050 con 0x55d668249a30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f68faffd640 1 -- 172.21.15.105:0/1687855876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69100055e0 con 0x55d668249a30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 10 client.18033 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 1 -- 172.21.15.105:0/1687855876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d66825e850 con 0x55d668249a30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f68faffd640 1 -- 172.21.15.105:0/1687855876 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6910065030 con 0x55d668249a30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f68faffd640 1 -- 172.21.15.105:0/1687855876 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f691005e030 con 0x55d668249a30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 10 client.18033 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 10 client.18033 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 20 client.18033 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d66825fba0 0x55d668261f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 4 client.18033 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.126+0000 7f6924c0b740 1 -- 172.21.15.105:0/1687855876 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6910005050 con 0x55d66825fba0 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.127+0000 7f691b7fe640 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d66825fba0 0x55d668261f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.127+0000 7f68faffd640 10 client.18033 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.127+0000 7f691b7fe640 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d66825fba0 0x55d668261f90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7834ba763aaa4d4d server_cookie=9bd6c05ef086a3db in_seq=0 out_seq=0 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.127+0000 7f68faffd640 1 -- 172.21.15.105:0/1687855876 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f6910005050 con 0x55d66825fba0 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.127+0000 7f68faffd640 10 client.18033 handle_command_reply: tid=0 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.127+0000 7f6924c0b740 1 client.18033 shutdown 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 -- 172.21.15.105:0/1687855876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 msgr2=0x55d668253f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668253f80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f691002ccd0 tx=0x7f6910005fd0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 -- 172.21.15.105:0/1687855876 shutdown_connections 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d66825fba0 0x55d668261f90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824f590 0x55d66825b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824a420 0x55d6682544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f68faffd640 0 client.18033 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:45:35.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1687855876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668253f80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 -- 172.21.15.105:0/1687855876 >> 172.21.15.105:0/1687855876 conn(0x55d668244fa0 msgr2=0x55d6682441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 -- 172.21.15.105:0/1687855876 shutdown_connections 2024-03-26T17:45:35.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 1 -- 172.21.15.105:0/1687855876 wait complete. 2024-03-26T17:45:35.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.128+0000 7f6924c0b740 20 client.18033 trim_cache size 0 max 16384 2024-03-26T17:45:35.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f6924c0b740 1 Processor -- start 2024-03-26T17:45:35.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f6924c0b740 1 -- start start 2024-03-26T17:45:35.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d6682542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d66825b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d668243930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f6924c0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1a6c00 con 0x55d668249a30 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f6924c0b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1a8740 con 0x55d66824a420 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f6924c0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1a72d0 con 0x55d66824f590 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d6682542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d6682542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51316/0 (socket says 172.21.15.105:51316) 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 -- 172.21.15.105:0/1057741800 learned_addr learned my addr 172.21.15.105:0/1057741800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d66825b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d668243930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 -- 172.21.15.105:0/1057741800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 msgr2=0x55d668243930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d668243930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 -- 172.21.15.105:0/1057741800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 msgr2=0x55d66825b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d66825b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 -- 172.21.15.105:0/1057741800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d66823f5d0 con 0x55d668249a30 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691b7fe640 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d66825b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.143+0000 7f691affd640 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d6682542a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f69100347c0 tx=0x7f6910058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875a442050f667a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f69017fa640 1 -- 172.21.15.105:0/1057741800 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910009780 con 0x55d668249a30 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f69017fa640 1 -- 172.21.15.105:0/1057741800 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6910065070 con 0x55d668249a30 2024-03-26T17:45:35.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f69017fa640 1 -- 172.21.15.105:0/1057741800 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910003d00 con 0x55d668249a30 2024-03-26T17:45:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 -- 172.21.15.105:0/1057741800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 msgr2=0x55d6682542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d6682542a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f69100347c0 tx=0x7f6910058040 comp rx=0 tx=0).stop 2024-03-26T17:45:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 -- 172.21.15.105:0/1057741800 shutdown_connections 2024-03-26T17:45:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d668243930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d66825b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1057741800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d6682542a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 -- 172.21.15.105:0/1057741800 >> 172.21.15.105:0/1057741800 conn(0x55d668244fa0 msgr2=0x55d668253b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 -- 172.21.15.105:0/1057741800 shutdown_connections 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.144+0000 7f6924c0b740 1 -- 172.21.15.105:0/1057741800 wait complete. 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 1 -- start start 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 1 Processor -- start 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668258e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d6682593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d66825a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1a8070 con 0x55d668249a30 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1a79a0 con 0x55d66824a420 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1a4be0 con 0x55d66824f590 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f6924c0b740 10 client.0 _ll_register_callbacks cb 0x7f691002c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f691affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668258e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f691b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d6682593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f691affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668258e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51320/0 (socket says 172.21.15.105:51320) 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f691affd640 1 -- 172.21.15.105:0/1739736179 learned_addr learned my addr 172.21.15.105:0/1739736179 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.145+0000 7f691affd640 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d66825a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f691b7fe640 1 -- 172.21.15.105:0/1739736179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 msgr2=0x55d66825a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f691b7fe640 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d66825a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f691b7fe640 1 -- 172.21.15.105:0/1739736179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 msgr2=0x55d668258e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f691b7fe640 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668258e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f691b7fe640 1 -- 172.21.15.105:0/1739736179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66825a9b0 con 0x55d66824a420 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f691affd640 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d66825a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f691b7fe640 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d6682593b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55d668245f10 tx=0x7f690c013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c8bd26c4d1ac4fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f68faffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f68faffd640 1 -- 172.21.15.105:0/1739736179 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c004220 con 0x55d66824a420 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f6924c0b740 1 -- 172.21.15.105:0/1739736179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6910065070 con 0x55d66824a420 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f68faffd640 1 -- 172.21.15.105:0/1739736179 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f690c0043c0 con 0x55d66824a420 2024-03-26T17:45:35.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f68faffd640 1 -- 172.21.15.105:0/1739736179 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f690c01d380 con 0x55d66824a420 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f6924c0b740 10 client.27869 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f6924c0b740 1 -- 172.21.15.105:0/1739736179 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6682c5f20 con 0x55d66824a420 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.146+0000 7f68faffd640 1 -- 172.21.15.105:0/1739736179 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6910065070 con 0x55d66824a420 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f68faffd640 1 -- 172.21.15.105:0/1739736179 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f690c003c60 con 0x55d66824a420 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f6924c0b740 10 client.27869 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f6924c0b740 10 client.27869 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f6924c0b740 20 client.27869 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d6682c6890 0x55d6682c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f6924c0b740 4 client.27869 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f6924c0b740 1 -- 172.21.15.105:0/1739736179 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6681bbeb0 con 0x55d6682c6890 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f691affd640 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d6682c6890 0x55d6682c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f68faffd640 10 client.27869 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.147+0000 7f691affd640 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d6682c6890 0x55d6682c8cb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3503753717fbc093 server_cookie=ce75713a46e70ead in_seq=0 out_seq=0 2024-03-26T17:45:35.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f68faffd640 1 -- 172.21.15.105:0/1739736179 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6681bbeb0 con 0x55d6682c6890 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f68faffd640 10 client.27869 handle_command_reply: tid=0 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 client.27869 shutdown 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 -- 172.21.15.105:0/1739736179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 msgr2=0x55d6682593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d6682593b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55d668245f10 tx=0x7f690c013830 comp rx=0 tx=0).stop 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 -- 172.21.15.105:0/1739736179 shutdown_connections 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d6682c6890 0x55d6682c8cb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66824f590 0x55d66825a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66824a420 0x55d6682593b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f68faffd640 0 client.27869 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 --2- 172.21.15.105:0/1739736179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d668249a30 0x55d668258e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 -- 172.21.15.105:0/1739736179 >> 172.21.15.105:0/1739736179 conn(0x55d668244fa0 msgr2=0x55d668248200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 -- 172.21.15.105:0/1739736179 shutdown_connections 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 1 -- 172.21.15.105:0/1739736179 wait complete. 2024-03-26T17:45:35.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.148+0000 7f6924c0b740 20 client.27869 trim_cache size 0 max 16384 2024-03-26T17:45:35.147 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 -- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69040a2f20 msgr2=0x7f69040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69040a2f20 0x7f69040a53e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6914098b80 tx=0x7f691400efd0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 -- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c107600 msgr2=0x7f691c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c107600 0x7f691c100890 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f691400eb10 tx=0x7f691400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 -- 172.21.15.105:0/3169461386 shutdown_connections 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69040a2f20 0x7f69040a53e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f691c107600 0x7f691c100890 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f691c104cd0 0x7f691c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 --2- 172.21.15.105:0/3169461386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f691c1023a0 0x7f691c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 -- 172.21.15.105:0/3169461386 >> 172.21.15.105:0/3169461386 conn(0x7f691c0fbd50 msgr2=0x7f691c104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 -- 172.21.15.105:0/3169461386 shutdown_connections 2024-03-26T17:45:35.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.149+0000 7f6922026640 1 -- 172.21.15.105:0/3169461386 wait complete. 2024-03-26T17:45:35.152 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:45:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 -- 172.21.15.105:0/569552165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1007c0 msgr2=0x7fad6c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 --2- 172.21.15.105:0/569552165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1007c0 0x7fad6c100be0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009900 tx=0x7fad5c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 -- 172.21.15.105:0/569552165 shutdown_connections 2024-03-26T17:45:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 --2- 172.21.15.105:0/569552165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c102370 0x7fad6c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 --2- 172.21.15.105:0/569552165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c1019b0 0x7fad6c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 --2- 172.21.15.105:0/569552165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1007c0 0x7fad6c100be0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 -- 172.21.15.105:0/569552165 >> 172.21.15.105:0/569552165 conn(0x7fad6c0fbd50 msgr2=0x7fad6c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 -- 172.21.15.105:0/569552165 shutdown_connections 2024-03-26T17:45:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 -- 172.21.15.105:0/569552165 wait complete. 2024-03-26T17:45:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 Processor -- start 2024-03-26T17:45:35.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.324+0000 7fad72732640 1 -- start start 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad72732640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c1007c0 0x7fad6c104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad72732640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1019b0 0x7fad6c102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad72732640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c102370 0x7fad6c1032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad72732640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c10c2e0 con 0x7fad6c1019b0 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad72732640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c10c160 con 0x7fad6c1007c0 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad72732640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c10c460 con 0x7fad6c102370 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1019b0 0x7fad6c102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c1007c0 0x7fad6c104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1019b0 0x7fad6c102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51336/0 (socket says 172.21.15.105:51336) 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 -- 172.21.15.105:0/1257674895 learned_addr learned my addr 172.21.15.105:0/1257674895 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad70ccf640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c102370 0x7fad6c1032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 -- 172.21.15.105:0/1257674895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c102370 msgr2=0x7fad6c1032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c102370 0x7fad6c1032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 -- 172.21.15.105:0/1257674895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c1007c0 msgr2=0x7fad6c104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c1007c0 0x7fad6c104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 -- 172.21.15.105:0/1257674895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c103b60 con 0x7fad6c1019b0 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad70ccf640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c102370 0x7fad6c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6bfff640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c1007c0 0x7fad6c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.325+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1019b0 0x7fad6c102d90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fad6000e9f0 tx=0x7fad6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.326+0000 7fad697fa640 1 -- 172.21.15.105:0/1257674895 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6000cd80 con 0x7fad6c1019b0 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.326+0000 7fad697fa640 1 -- 172.21.15.105:0/1257674895 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad600054d0 con 0x7fad6c1019b0 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.326+0000 7fad697fa640 1 -- 172.21.15.105:0/1257674895 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60010490 con 0x7fad6c1019b0 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.326+0000 7fad72732640 1 -- 172.21.15.105:0/1257674895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c1a1ce0 con 0x7fad6c1019b0 2024-03-26T17:45:35.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.326+0000 7fad72732640 1 -- 172.21.15.105:0/1257674895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c1a2180 con 0x7fad6c1019b0 2024-03-26T17:45:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.328+0000 7fad697fa640 1 -- 172.21.15.105:0/1257674895 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fad60005020 con 0x7fad6c1019b0 2024-03-26T17:45:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad753f4740 1 Processor -- start 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad697fa640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad540a2f20 0x7fad540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad753f4740 1 -- start start 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad70ccf640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad540a2f20 0x7fad540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad753f4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0718e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad753f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f071df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad753f4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0720a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad753f4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5591f067d970 con 0x5591f071e590 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad753f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5591f0625d70 con 0x5591f0719420 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad753f4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5591f0679920 con 0x5591f0718a30 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad697fa640 1 -- 172.21.15.105:0/1257674895 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad600cf7c0 con 0x7fad6c1019b0 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad6bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0718e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad6bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0718e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47702/0 (socket says 172.21.15.105:47702) 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad6bfff640 1 -- 172.21.15.105:0/170643407 learned_addr learned my addr 172.21.15.105:0/170643407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f071df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.329+0000 7fad70ccf640 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0720a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad6b7fe640 1 -- 172.21.15.105:0/170643407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 msgr2=0x5591f0718e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0718e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad6b7fe640 1 -- 172.21.15.105:0/170643407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 msgr2=0x5591f0720a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0720a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad6b7fe640 1 -- 172.21.15.105:0/170643407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f03913f0 con 0x5591f0719420 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad6bfff640 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0718e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad70ccf640 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0720a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad70ccf640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad540a2f20 0x7fad540a53e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad64005fd0 tx=0x7fad64011bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f071df20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fad600cd5f0 tx=0x7fad6009bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75aec8d7f54e5765 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad527fc640 1 -- 172.21.15.105:0/170643407 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6009c5b0 con 0x5591f0719420 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad527fc640 1 -- 172.21.15.105:0/170643407 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad6009beb0 con 0x5591f0719420 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.330+0000 7fad527fc640 1 -- 172.21.15.105:0/170643407 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60098520 con 0x5591f0719420 2024-03-26T17:45:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 -- 172.21.15.105:0/170643407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 msgr2=0x5591f071df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f071df20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fad600cd5f0 tx=0x7fad6009bdb0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 -- 172.21.15.105:0/170643407 shutdown_connections 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0720a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f071df20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 --2- 172.21.15.105:0/170643407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0718e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 -- 172.21.15.105:0/170643407 >> 172.21.15.105:0/170643407 conn(0x5591f0713fa0 msgr2=0x5591f07143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 -- 172.21.15.105:0/170643407 shutdown_connections 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 -- 172.21.15.105:0/170643407 wait complete. 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.331+0000 7fad753f4740 1 -- start start 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad753f4740 1 Processor -- start 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad753f4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0722f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad753f4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f07234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad753f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f071e590 0x5591f072a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad753f4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5591f0721540 con 0x5591f0719420 2024-03-26T17:45:35.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad753f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5591f0639310 con 0x5591f071e590 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad753f4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5591f07216c0 con 0x5591f0718a30 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad753f4740 10 client.0 _ll_register_callbacks cb 0x5591f0674c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f07234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f07234b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51354/0 (socket says 172.21.15.105:51354) 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 -- 172.21.15.105:0/3803382238 learned_addr learned my addr 172.21.15.105:0/3803382238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6bfff640 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0722f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad70ccf640 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f071e590 0x5591f072a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 -- 172.21.15.105:0/3803382238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 msgr2=0x5591f0722f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0722f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 -- 172.21.15.105:0/3803382238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f071e590 msgr2=0x5591f072a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f071e590 0x5591f072a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 -- 172.21.15.105:0/3803382238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f045b7d0 con 0x5591f0719420 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad70ccf640 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f071e590 0x5591f072a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.332+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f07234b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fad6009bbb0 tx=0x7fad6009c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7a84f32d76de01 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad4b7fe640 1 -- 172.21.15.105:0/3803382238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad600a48c0 con 0x5591f0719420 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad4b7fe640 1 -- 172.21.15.105:0/3803382238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad600046c0 con 0x5591f0719420 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 1 -- 172.21.15.105:0/3803382238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad6009beb0 con 0x5591f0719420 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad4b7fe640 1 -- 172.21.15.105:0/3803382238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad600a4430 con 0x5591f0719420 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 10 client.18051 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 1 -- 172.21.15.105:0/3803382238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591f072d860 con 0x5591f0719420 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad4b7fe640 1 -- 172.21.15.105:0/3803382238 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad6009beb0 con 0x5591f0719420 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad4b7fe640 1 -- 172.21.15.105:0/3803382238 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fad6009af00 con 0x5591f0719420 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 10 client.18051 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 10 client.18051 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 20 client.18051 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5591f072ebb0 0x5591f0730fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 4 client.18051 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.333+0000 7fad753f4740 1 -- 172.21.15.105:0/3803382238 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591f068aeb0 con 0x5591f072ebb0 2024-03-26T17:45:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.334+0000 7fad6bfff640 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5591f072ebb0 0x5591f0730fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.334+0000 7fad4b7fe640 10 client.18051 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:45:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.334+0000 7fad6bfff640 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5591f072ebb0 0x5591f0730fa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=854b9e963a04eda1 server_cookie=6852c97c050a6519 in_seq=0 out_seq=0 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad4b7fe640 1 -- 172.21.15.105:0/3803382238 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5591f068aeb0 con 0x5591f072ebb0 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad4b7fe640 10 client.18051 handle_command_reply: tid=0 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 client.18051 shutdown 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 -- 172.21.15.105:0/3803382238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 msgr2=0x5591f07234b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f07234b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fad6009bbb0 tx=0x7fad6009c5b0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 -- 172.21.15.105:0/3803382238 shutdown_connections 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5591f072ebb0 0x5591f0730fa0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f071e590 0x5591f072a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f07234b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 --2- 172.21.15.105:0/3803382238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0722f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 -- 172.21.15.105:0/3803382238 >> 172.21.15.105:0/3803382238 conn(0x5591f0713fa0 msgr2=0x5591f07131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad4b7fe640 0 client.18051 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:45:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 -- 172.21.15.105:0/3803382238 shutdown_connections 2024-03-26T17:45:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 1 -- 172.21.15.105:0/3803382238 wait complete. 2024-03-26T17:45:35.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.335+0000 7fad753f4740 20 client.18051 trim_cache size 0 max 16384 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad753f4740 1 Processor -- start 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad753f4740 1 -- start start 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad753f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0718a30 0x5591f070f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad753f4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f072a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad753f4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f071e590 0x5591f071a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad753f4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1aaff0 con 0x5591f0719420 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad753f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1acb30 con 0x5591f0718a30 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad753f4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1ab6c0 con 0x5591f071e590 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f072a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f072a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51364/0 (socket says 172.21.15.105:51364) 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6b7fe640 1 -- 172.21.15.105:0/708699607 learned_addr learned my addr 172.21.15.105:0/708699607 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6bfff640 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0718a30 0x5591f070f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad70ccf640 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f071e590 0x5591f071a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6b7fe640 1 -- 172.21.15.105:0/708699607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f071e590 msgr2=0x5591f071a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f071e590 0x5591f071a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6b7fe640 1 -- 172.21.15.105:0/708699607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0718a30 msgr2=0x5591f070f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0718a30 0x5591f070f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.357+0000 7fad6b7fe640 1 -- 172.21.15.105:0/708699607 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f070f820 con 0x5591f0719420 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.358+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f072a4b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fad600cd5f0 tx=0x7fad6009bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6123ee157770e2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.358+0000 7fad51ffb640 1 -- 172.21.15.105:0/708699607 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60098150 con 0x5591f0719420 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.358+0000 7fad51ffb640 1 -- 172.21.15.105:0/708699607 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad600bc0c0 con 0x5591f0719420 2024-03-26T17:45:35.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.358+0000 7fad51ffb640 1 -- 172.21.15.105:0/708699607 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad600b00f0 con 0x5591f0719420 2024-03-26T17:45:35.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 -- 172.21.15.105:0/708699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 msgr2=0x5591f072a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f072a4b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fad600cd5f0 tx=0x7fad6009bdb0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 -- 172.21.15.105:0/708699607 shutdown_connections 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f071e590 0x5591f071a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f0719420 0x5591f072a4b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 --2- 172.21.15.105:0/708699607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0718a30 0x5591f070f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 -- 172.21.15.105:0/708699607 >> 172.21.15.105:0/708699607 conn(0x5591f0713fa0 msgr2=0x5591f0730bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 -- 172.21.15.105:0/708699607 shutdown_connections 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 -- 172.21.15.105:0/708699607 wait complete. 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 -- start start 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.359+0000 7fad753f4740 1 Processor -- start 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad753f4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0727380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad753f4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f07278c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad753f4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0792d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad753f4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1ac460 con 0x5591f071e590 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad753f4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1abd90 con 0x5591f0719420 2024-03-26T17:45:35.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad753f4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c1a8fd0 con 0x5591f0718a30 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad753f4740 10 client.0 _ll_register_callbacks cb 0x7fad60005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0792d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0792d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51366/0 (socket says 172.21.15.105:51366) 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 -- 172.21.15.105:0/2953901957 learned_addr learned my addr 172.21.15.105:0/2953901957 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad6b7fe640 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f07278c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad6bfff640 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0727380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 -- 172.21.15.105:0/2953901957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 msgr2=0x5591f0727380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0727380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 -- 172.21.15.105:0/2953901957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 msgr2=0x5591f07278c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f07278c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 -- 172.21.15.105:0/2953901957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f0793290 con 0x5591f071e590 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad70ccf640 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0792d50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fad640099e0 tx=0x7fad64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7de36dc1c51dd82 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:35.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2953901957 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64017040 con 0x5591f071e590 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad753f4740 1 -- 172.21.15.105:0/2953901957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591f0685200 con 0x5591f071e590 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.360+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2953901957 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad64027650 con 0x5591f071e590 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2953901957 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad64004860 con 0x5591f071e590 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad753f4740 10 client.18057 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad753f4740 1 -- 172.21.15.105:0/2953901957 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591f0794500 con 0x5591f071e590 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2953901957 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5591f0685200 con 0x5591f071e590 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2953901957 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fad6401d980 con 0x5591f071e590 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad753f4740 10 client.18057 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad753f4740 10 client.18057 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad753f4740 20 client.18057 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad753f4740 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5591f0794f60 0x5591f0797380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad753f4740 4 client.18057 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:35.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad753f4740 1 -- 172.21.15.105:0/2953901957 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591f06026a0 con 0x5591f0794f60 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.361+0000 7fad6bfff640 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5591f0794f60 0x5591f0797380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad4b7fe640 10 client.18057 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad6bfff640 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5591f0794f60 0x5591f0797380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=920892871977d15c server_cookie=95cf14914564cbc5 in_seq=0 out_seq=0 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2953901957 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591f06026a0 con 0x5591f0794f60 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad4b7fe640 10 client.18057 handle_command_reply: tid=0 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 client.18057 shutdown 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 -- 172.21.15.105:0/2953901957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 msgr2=0x5591f0792d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0792d50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fad640099e0 tx=0x7fad64004ab0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 -- 172.21.15.105:0/2953901957 shutdown_connections 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5591f0794f60 0x5591f0797380 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5591f071e590 0x5591f0792d50 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5591f0719420 0x5591f07278c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 --2- 172.21.15.105:0/2953901957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591f0718a30 0x5591f0727380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad4b7fe640 0 client.18057 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:45:35.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 -- 172.21.15.105:0/2953901957 >> 172.21.15.105:0/2953901957 conn(0x5591f0713fa0 msgr2=0x5591f071a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 -- 172.21.15.105:0/2953901957 shutdown_connections 2024-03-26T17:45:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 1 -- 172.21.15.105:0/2953901957 wait complete. 2024-03-26T17:45:35.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.362+0000 7fad753f4740 20 client.18057 trim_cache size 0 max 16384 2024-03-26T17:45:35.360 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 -- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad540a2f20 msgr2=0x7fad540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad540a2f20 0x7fad540a53e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad64005fd0 tx=0x7fad64011bd0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 -- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1019b0 msgr2=0x7fad6c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1019b0 0x7fad6c102d90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fad6000e9f0 tx=0x7fad6000edf0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 -- 172.21.15.105:0/1257674895 shutdown_connections 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad540a2f20 0x7fad540a53e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad6c102370 0x7fad6c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad6c1019b0 0x7fad6c102d90 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 --2- 172.21.15.105:0/1257674895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad6c1007c0 0x7fad6c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 -- 172.21.15.105:0/1257674895 >> 172.21.15.105:0/1257674895 conn(0x7fad6c0fbd50 msgr2=0x7fad6c068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 -- 172.21.15.105:0/1257674895 shutdown_connections 2024-03-26T17:45:35.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.363+0000 7fad72732640 1 -- 172.21.15.105:0/1257674895 wait complete. 2024-03-26T17:45:35.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.512+0000 7f8ca1201640 1 -- 172.21.15.105:0/3004668705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 msgr2=0x7f8c9c101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.512+0000 7f8ca1201640 1 --2- 172.21.15.105:0/3004668705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 0x7f8c9c101d40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009900 tx=0x7f8c9002bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.512+0000 7f8ca1201640 1 -- 172.21.15.105:0/3004668705 shutdown_connections 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.512+0000 7f8ca1201640 1 --2- 172.21.15.105:0/3004668705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c102280 0x7f8c9c108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.512+0000 7f8ca1201640 1 --2- 172.21.15.105:0/3004668705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 0x7f8c9c101d40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.512+0000 7f8ca1201640 1 --2- 172.21.15.105:0/3004668705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c9c1006d0 0x7f8c9c100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.513+0000 7f8ca1201640 1 -- 172.21.15.105:0/3004668705 >> 172.21.15.105:0/3004668705 conn(0x7f8c9c0fbc60 msgr2=0x7f8c9c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.513+0000 7f8ca1201640 1 -- 172.21.15.105:0/3004668705 shutdown_connections 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.513+0000 7f8ca1201640 1 -- 172.21.15.105:0/3004668705 wait complete. 2024-03-26T17:45:35.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.513+0000 7f8ca1201640 1 Processor -- start 2024-03-26T17:45:35.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8ca1201640 1 -- start start 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8ca1201640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c9c1006d0 0x7f8c9c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8ca1201640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 0x7f8c9c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8ca1201640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c102280 0x7f8c9c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8ca1201640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c10c1e0 con 0x7f8c9c1018c0 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8ca1201640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c10c060 con 0x7f8c9c1006d0 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8ca1201640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c10c360 con 0x7f8c9c102280 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 0x7f8c9c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c9c1006d0 0x7f8c9c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c9c1006d0 0x7f8c9c1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56124/0 (socket says 172.21.15.105:56124) 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 0x7f8c9c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51382/0 (socket says 172.21.15.105:51382) 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9ad9d640 1 -- 172.21.15.105:0/1669398493 learned_addr learned my addr 172.21.15.105:0/1669398493 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c102280 0x7f8c9c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9ad9d640 1 -- 172.21.15.105:0/1669398493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c102280 msgr2=0x7f8c9c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9ad9d640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c102280 0x7f8c9c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9ad9d640 1 -- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 msgr2=0x7f8c9c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9ad9d640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 0x7f8c9c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9ad9d640 1 -- 172.21.15.105:0/1669398493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c19f600 con 0x7f8c9c1006d0 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.514+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c102280 0x7f8c9c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.515+0000 7f8c9ad9d640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c9c1006d0 0x7f8c9c1979e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e9c0 tx=0x7f8c8c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.515+0000 7f8c83fff640 1 -- 172.21.15.105:0/1669398493 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00cd80 con 0x7f8c9c1006d0 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.515+0000 7f8c83fff640 1 -- 172.21.15.105:0/1669398493 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c8c0035f0 con 0x7f8c9c1006d0 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.515+0000 7f8c83fff640 1 -- 172.21.15.105:0/1669398493 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c010690 con 0x7f8c9c1006d0 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.515+0000 7f8ca1201640 1 -- 172.21.15.105:0/1669398493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9c19f8f0 con 0x7f8c9c1006d0 2024-03-26T17:45:35.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.515+0000 7f8ca1201640 1 -- 172.21.15.105:0/1669398493 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9c19fe10 con 0x7f8c9c1006d0 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.517+0000 7f8ca3dea740 1 Processor -- start 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.517+0000 7f8c83fff640 1 -- 172.21.15.105:0/1669398493 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8c8c0108f0 con 0x7f8c9c1006d0 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.517+0000 7f8ca3dea740 1 -- start start 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d2630 0x55c4b22d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 0x55c4b22d9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2236970 con 0x55c4b22d1c40 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c4b21ded70 con 0x55c4b22d2630 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c4b2232920 con 0x55c4b22d77a0 2024-03-26T17:45:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51384/0 (socket says 172.21.15.105:51384) 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 -- 172.21.15.105:0/4100050179 learned_addr learned my addr 172.21.15.105:0/4100050179 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d2630 0x55c4b22d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/4100050179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 0x55c4b22d9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 -- 172.21.15.105:0/4100050179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 msgr2=0x55c4b22d9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/4100050179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 0x55c4b22d9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 -- 172.21.15.105:0/4100050179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d2630 msgr2=0x55c4b22d7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/4100050179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d2630 0x55c4b22d7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 -- 172.21.15.105:0/4100050179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b1f4a3f0 con 0x55c4b22d1c40 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c83fff640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c840a2f20 0x7f8c840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9ad9d640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c840a2f20 0x7f8c840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/4100050179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22d2060 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8c940117e0 tx=0x7f8c94011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbb21926e481b3c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.518+0000 7f8c83fff640 1 -- 172.21.15.105:0/1669398493 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8c8c0cf7e0 con 0x7f8c9c1006d0 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8c817fa640 1 -- 172.21.15.105:0/4100050179 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94011310 con 0x55c4b22d1c40 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8c817fa640 1 -- 172.21.15.105:0/4100050179 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c94003e20 con 0x55c4b22d1c40 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8c817fa640 1 -- 172.21.15.105:0/4100050179 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c940044f0 con 0x55c4b22d1c40 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8ca3dea740 1 -- 172.21.15.105:0/4100050179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 msgr2=0x55c4b22d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8c9ad9d640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c840a2f20 0x7f8c840a53e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0cd610 tx=0x7f8c8c0d5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/4100050179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22d2060 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8c940117e0 tx=0x7f8c94011ca0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8ca3dea740 1 -- 172.21.15.105:0/4100050179 shutdown_connections 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/4100050179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 0x55c4b22d9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/4100050179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d2630 0x55c4b22d7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/4100050179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22d2060 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.519+0000 7f8ca3dea740 1 -- 172.21.15.105:0/4100050179 >> 172.21.15.105:0/4100050179 conn(0x55c4b22cd1b0 msgr2=0x55c4b22cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 -- 172.21.15.105:0/4100050179 shutdown_connections 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 -- 172.21.15.105:0/4100050179 wait complete. 2024-03-26T17:45:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 -- start start 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 Processor -- start 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 0x55c4b22dc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22dc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d77a0 0x55c4b22e36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c4b22da750 con 0x55c4b22d77a0 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c4b21f2310 con 0x55c4b22d1c40 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c4b22da8d0 con 0x55c4b22d2630 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8ca3dea740 10 client.0 _ll_register_callbacks cb 0x55c4b222dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d77a0 0x55c4b22e36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d77a0 0x55c4b22e36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51386/0 (socket says 172.21.15.105:51386) 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 -- 172.21.15.105:0/397324906 learned_addr learned my addr 172.21.15.105:0/397324906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22dc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 0x55c4b22dc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 -- 172.21.15.105:0/397324906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 msgr2=0x55c4b22dc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22dc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 -- 172.21.15.105:0/397324906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 msgr2=0x55c4b22dc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 0x55c4b22dc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.520+0000 7f8c9b59e640 1 -- 172.21.15.105:0/397324906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b20147d0 con 0x55c4b22d77a0 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d77a0 0x55c4b22e36b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8c94011610 tx=0x7f8c94005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932d6812c764f60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8c7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8c7eff5640 1 -- 172.21.15.105:0/397324906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94025040 con 0x55c4b22d77a0 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8c7eff5640 1 -- 172.21.15.105:0/397324906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8c94004a40 con 0x55c4b22d77a0 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 1 -- 172.21.15.105:0/397324906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c94003e20 con 0x55c4b22d77a0 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8c7eff5640 1 -- 172.21.15.105:0/397324906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8c94031050 con 0x55c4b22d77a0 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 10 client.18069 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 1 -- 172.21.15.105:0/397324906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4b22e6a20 con 0x55c4b22d77a0 2024-03-26T17:45:35.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8c7eff5640 1 -- 172.21.15.105:0/397324906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c94003e20 con 0x55c4b22d77a0 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8c7eff5640 1 -- 172.21.15.105:0/397324906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8c94018020 con 0x55c4b22d77a0 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 10 client.18069 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 10 client.18069 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 20 client.18069 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c4b22e7ef0 0x55c4b22ea300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 4 client.18069 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.521+0000 7f8ca3dea740 1 -- 172.21.15.105:0/397324906 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4b2243eb0 con 0x55c4b22e7ef0 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.522+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c4b22e7ef0 0x55c4b22ea300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.522+0000 7f8c7eff5640 10 client.18069 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.522+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c4b22e7ef0 0x55c4b22ea300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26fb88effbc910a9 server_cookie=c324a580294bafa8 in_seq=0 out_seq=0 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8c7eff5640 1 -- 172.21.15.105:0/397324906 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c4b2243eb0 con 0x55c4b22e7ef0 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8c7eff5640 10 client.18069 handle_command_reply: tid=0 2024-03-26T17:45:35.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 client.18069 shutdown 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 -- 172.21.15.105:0/397324906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d77a0 msgr2=0x55c4b22e36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d77a0 0x55c4b22e36b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8c94011610 tx=0x7f8c94005bd0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 -- 172.21.15.105:0/397324906 shutdown_connections 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d77a0 0x55c4b22e36b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22dc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c4b22e7ef0 0x55c4b22ea300 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/397324906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 0x55c4b22dc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8c7eff5640 0 client.18069 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 -- 172.21.15.105:0/397324906 >> 172.21.15.105:0/397324906 conn(0x55c4b22cd1b0 msgr2=0x55c4b22cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 -- 172.21.15.105:0/397324906 shutdown_connections 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 1 -- 172.21.15.105:0/397324906 wait complete. 2024-03-26T17:45:35.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.523+0000 7f8ca3dea740 20 client.18069 trim_cache size 0 max 16384 2024-03-26T17:45:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.537+0000 7f8ca3dea740 1 Processor -- start 2024-03-26T17:45:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.537+0000 7f8ca3dea740 1 -- start start 2024-03-26T17:45:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.537+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 0x55c4b22d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.537+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d2630 0x55c4b22d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.537+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 0x55c4b22e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.537+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c1a8c60 con 0x55c4b22d2630 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c1aa7a0 con 0x55c4b22d1c40 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c1a9330 con 0x55c4b22d77a0 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d2630 0x55c4b22d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d2630 0x55c4b22d4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51394/0 (socket says 172.21.15.105:51394) 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 -- 172.21.15.105:0/3895091649 learned_addr learned my addr 172.21.15.105:0/3895091649 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 0x55c4b22d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 0x55c4b22e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 -- 172.21.15.105:0/3895091649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 msgr2=0x55c4b22e35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 0x55c4b22e35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 -- 172.21.15.105:0/3895091649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 msgr2=0x55c4b22d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 0x55c4b22d3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 -- 172.21.15.105:0/3895091649 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4b22cdc00 con 0x55c4b22d2630 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d2630 0x55c4b22d4070 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009900 tx=0x7f8c90005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9fc15db02420ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c80ff9640 1 -- 172.21.15.105:0/3895091649 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9002cba0 con 0x55c4b22d2630 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c80ff9640 1 -- 172.21.15.105:0/3895091649 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c9004b420 con 0x55c4b22d2630 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.538+0000 7f8c80ff9640 1 -- 172.21.15.105:0/3895091649 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90005040 con 0x55c4b22d2630 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3895091649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d2630 msgr2=0x55c4b22d4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d2630 0x55c4b22d4070 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009900 tx=0x7f8c90005850 comp rx=0 tx=0).stop 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3895091649 shutdown_connections 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d77a0 0x55c4b22e35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d2630 0x55c4b22d4070 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3895091649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d1c40 0x55c4b22d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3895091649 >> 172.21.15.105:0/3895091649 conn(0x55c4b22cd1b0 msgr2=0x55c4b22dc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3895091649 shutdown_connections 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3895091649 wait complete. 2024-03-26T17:45:35.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 -- start start 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 Processor -- start 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.539+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8ca3dea740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d77a0 0x55c4b234c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c1aa0d0 con 0x55c4b22d1c40 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c1a9a00 con 0x55c4b22d77a0 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8ca3dea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c1a6c40 con 0x55c4b22d2630 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8ca3dea740 10 client.0 _ll_register_callbacks cb 0x7f8c8c0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22e0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51400/0 (socket says 172.21.15.105:51400) 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 -- 172.21.15.105:0/3664979328 learned_addr learned my addr 172.21.15.105:0/3664979328 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d77a0 0x55c4b234c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 -- 172.21.15.105:0/3664979328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 msgr2=0x55c4b22e06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22e06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 -- 172.21.15.105:0/3664979328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d77a0 msgr2=0x55c4b234c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d77a0 0x55c4b234c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 -- 172.21.15.105:0/3664979328 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4b22e3770 con 0x55c4b22d1c40 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c9b59e640 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22e0160 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8c9402c920 tx=0x7f8c94000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edfc363adab6ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c7eff5640 1 -- 172.21.15.105:0/3664979328 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9402c470 con 0x55c4b22d1c40 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8c7eff5640 1 -- 172.21.15.105:0/3664979328 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8c9402c610 con 0x55c4b22d1c40 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.540+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3664979328 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9004b420 con 0x55c4b22d1c40 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8c7eff5640 1 -- 172.21.15.105:0/3664979328 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8c94005110 con 0x55c4b22d1c40 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8ca3dea740 10 client.18075 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3664979328 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c4b234d610 con 0x55c4b22d1c40 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8c7eff5640 1 -- 172.21.15.105:0/3664979328 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c9004b420 con 0x55c4b22d1c40 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8c7eff5640 1 -- 172.21.15.105:0/3664979328 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8c94034030 con 0x55c4b22d1c40 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8ca3dea740 10 client.18075 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8ca3dea740 10 client.18075 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8ca3dea740 20 client.18075 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c4b234e030 0x55c4b2350450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8ca3dea740 4 client.18075 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3664979328 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4b223e200 con 0x55c4b234e030 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.541+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c4b234e030 0x55c4b2350450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8c7eff5640 10 client.18075 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8c9a59c640 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c4b234e030 0x55c4b2350450 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44deb601df84ae2c server_cookie=ecc2bca416d5738 in_seq=0 out_seq=0 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8c7eff5640 1 -- 172.21.15.105:0/3664979328 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4b223e200 con 0x55c4b234e030 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8c7eff5640 10 client.18075 handle_command_reply: tid=0 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 client.18075 shutdown 2024-03-26T17:45:35.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3664979328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 msgr2=0x55c4b22e0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22e0160 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8c9402c920 tx=0x7f8c94000f40 comp rx=0 tx=0).stop 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3664979328 shutdown_connections 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c4b234e030 0x55c4b2350450 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c4b22d77a0 0x55c4b234c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8c7eff5640 0 client.18075 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c4b22d2630 0x55c4b22e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 --2- 172.21.15.105:0/3664979328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c4b22d1c40 0x55c4b22e0160 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3664979328 >> 172.21.15.105:0/3664979328 conn(0x55c4b22cd1b0 msgr2=0x55c4b22dc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3664979328 shutdown_connections 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 1 -- 172.21.15.105:0/3664979328 wait complete. 2024-03-26T17:45:35.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.542+0000 7f8ca3dea740 20 client.18075 trim_cache size 0 max 16384 2024-03-26T17:45:35.541 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:35.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 -- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c840a2f20 msgr2=0x7f8c840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c840a2f20 0x7f8c840a53e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0cd610 tx=0x7f8c8c0d5550 comp rx=0 tx=0).stop 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 -- 172.21.15.105:0/1669398493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c9c1006d0 msgr2=0x7f8c9c1979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c9c1006d0 0x7f8c9c1979e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e9c0 tx=0x7f8c8c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 -- 172.21.15.105:0/1669398493 shutdown_connections 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c840a2f20 0x7f8c840a53e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c102280 0x7f8c9c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c9c1018c0 0x7f8c9c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 --2- 172.21.15.105:0/1669398493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c9c1006d0 0x7f8c9c1979e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 -- 172.21.15.105:0/1669398493 >> 172.21.15.105:0/1669398493 conn(0x7f8c9c0fbc60 msgr2=0x7f8c9c068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 -- 172.21.15.105:0/1669398493 shutdown_connections 2024-03-26T17:45:35.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.543+0000 7f8ca1201640 1 -- 172.21.15.105:0/1669398493 wait complete. 2024-03-26T17:45:35.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:45:35.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 -- 172.21.15.105:0/3830101048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 msgr2=0x7fd6e4103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/3830101048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 0x7fd6e4103330 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009900 tx=0x7fd6d402bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:35.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 -- 172.21.15.105:0/3830101048 shutdown_connections 2024-03-26T17:45:35.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/3830101048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6e4100340 0x7fd6e4109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/3830101048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6e40ff980 0x7fd6e40ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/3830101048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 0x7fd6e4103330 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 -- 172.21.15.105:0/3830101048 >> 172.21.15.105:0/3830101048 conn(0x7fd6e40fb4e0 msgr2=0x7fd6e40fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 -- 172.21.15.105:0/3830101048 shutdown_connections 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.668+0000 7fd6ea28c640 1 -- 172.21.15.105:0/3830101048 wait complete. 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6ea28c640 1 Processor -- start 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6ea28c640 1 -- start start 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6ea28c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6e40ff980 0x7fd6e419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6ea28c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6e4100340 0x7fd6e419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6ea28c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 0x7fd6e41a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6ea28c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41024a0 con 0x7fd6e4102f10 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6ea28c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4102320 con 0x7fd6e40ff980 2024-03-26T17:45:35.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6ea28c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4102620 con 0x7fd6e4100340 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6e8829640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 0x7fd6e41a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6e40ff980 0x7fd6e419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6e40ff980 0x7fd6e419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56178/0 (socket says 172.21.15.105:56178) 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6e8829640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 0x7fd6e41a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51424/0 (socket says 172.21.15.105:51424) 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6e3fff640 1 -- 172.21.15.105:0/170713348 learned_addr learned my addr 172.21.15.105:0/170713348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.669+0000 7fd6e37fe640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6e4100340 0x7fd6e419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e3fff640 1 -- 172.21.15.105:0/170713348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6e4100340 msgr2=0x7fd6e419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6e4100340 0x7fd6e419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e3fff640 1 -- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 msgr2=0x7fd6e41a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 0x7fd6e41a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e3fff640 1 -- 172.21.15.105:0/170713348 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e41a5d90 con 0x7fd6e40ff980 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e8829640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 0x7fd6e41a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6e40ff980 0x7fd6e419e170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000c00 tx=0x7fd6d4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e17fa640 1 -- 172.21.15.105:0/170713348 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4043020 con 0x7fd6e40ff980 2024-03-26T17:45:35.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6ea28c640 1 -- 172.21.15.105:0/170713348 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e41a6020 con 0x7fd6e40ff980 2024-03-26T17:45:35.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e17fa640 1 -- 172.21.15.105:0/170713348 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d4004350 con 0x7fd6e40ff980 2024-03-26T17:45:35.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6e17fa640 1 -- 172.21.15.105:0/170713348 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4003b80 con 0x7fd6e40ff980 2024-03-26T17:45:35.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.670+0000 7fd6ea28c640 1 -- 172.21.15.105:0/170713348 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e41a65c0 con 0x7fd6e40ff980 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6e17fa640 1 -- 172.21.15.105:0/170713348 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd6d4046020 con 0x7fd6e40ff980 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6ece7f740 1 Processor -- start 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6ece7f740 1 -- start start 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd5a30 0x560611dd5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd6420 0x560611ddaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611ddda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560611d3a970 con 0x560611dd5a30 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6e17fa640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6cc0a2f20 0x7fd6cc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560611ce2d70 con 0x560611dd6420 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560611d36920 con 0x560611ddb590 2024-03-26T17:45:35.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6e8829640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd5a30 0x560611dd5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6e8829640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd5a30 0x560611dd5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51438/0 (socket says 172.21.15.105:51438) 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.673+0000 7fd6e8829640 1 -- 172.21.15.105:0/1727233477 learned_addr learned my addr 172.21.15.105:0/1727233477 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd6420 0x560611ddaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e37fe640 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611ddda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e17fa640 1 -- 172.21.15.105:0/170713348 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd6d40f1580 con 0x7fd6e40ff980 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e8829640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6cc0a2f20 0x7fd6cc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e3fff640 1 -- 172.21.15.105:0/1727233477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 msgr2=0x560611ddda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611ddda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e3fff640 1 -- 172.21.15.105:0/1727233477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd5a30 msgr2=0x560611dd5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd5a30 0x560611dd5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e3fff640 1 -- 172.21.15.105:0/1727233477 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560611a4e3f0 con 0x560611dd6420 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e8829640 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd5a30 0x560611dd5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd6420 0x560611ddaf20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40bd970 tx=0x7fd6d403bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09a56e6a718682e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6e8829640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6cc0a2f20 0x7fd6cc0a53e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x560611dd6c80 tx=0x7fd6dc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6ca7fc640 1 -- 172.21.15.105:0/1727233477 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40b9f10 con 0x560611dd6420 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6ca7fc640 1 -- 172.21.15.105:0/1727233477 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d40be370 con 0x560611dd6420 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.674+0000 7fd6ca7fc640 1 -- 172.21.15.105:0/1727233477 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40452c0 con 0x560611dd6420 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1727233477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd6420 msgr2=0x560611ddaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd6420 0x560611ddaf20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40bd970 tx=0x7fd6d403bbb0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1727233477 shutdown_connections 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611ddda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd6420 0x560611ddaf20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1727233477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd5a30 0x560611dd5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1727233477 >> 172.21.15.105:0/1727233477 conn(0x560611dd0fa0 msgr2=0x560611dd13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1727233477 shutdown_connections 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1727233477 wait complete. 2024-03-26T17:45:35.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.675+0000 7fd6ece7f740 1 -- start start 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6ece7f740 1 Processor -- start 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611ddfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611de7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560611dde300 con 0x560611dd6420 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560611cf6310 con 0x560611dd5a30 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560611dde480 con 0x560611ddb590 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6ece7f740 10 client.0 _ll_register_callbacks cb 0x560611d31c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611ddfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e8829640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611de7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51448/0 (socket says 172.21.15.105:51448) 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611ddfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56202/0 (socket says 172.21.15.105:56202) 2024-03-26T17:45:35.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e37fe640 1 -- 172.21.15.105:0/1251927075 learned_addr learned my addr 172.21.15.105:0/1251927075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e37fe640 1 -- 172.21.15.105:0/1251927075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 msgr2=0x560611de7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e37fe640 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611de7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e37fe640 1 -- 172.21.15.105:0/1251927075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 msgr2=0x560611ddfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e37fe640 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611ddfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.676+0000 7fd6e37fe640 1 -- 172.21.15.105:0/1251927075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560611b187d0 con 0x560611dd6420 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6e37fe640 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de00c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8012700 tx=0x7fd6d8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaca1962a9988c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1251927075 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800f8c0 con 0x560611dd6420 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1251927075 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd6d800fa60 con 0x560611dd6420 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1251927075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d40be370 con 0x560611dd6420 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1251927075 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd6d800fc60 con 0x560611dd6420 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 10 client.18087 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1251927075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560611dea540 con 0x560611dd6420 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1251927075 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d40be370 con 0x560611dd6420 2024-03-26T17:45:35.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1251927075 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd6d8004a60 con 0x560611dd6420 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 10 client.18087 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 10 client.18087 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 20 client.18087 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560611debb60 0x560611dee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 4 client.18087 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.677+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1251927075 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560611d47eb0 con 0x560611debb60 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.678+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560611debb60 0x560611dee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.678+0000 7fd6c37fe640 10 client.18087 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.678+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560611debb60 0x560611dee020 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a69a5ab3a50aca78 server_cookie=f3bc9e26797736c7 in_seq=0 out_seq=0 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1251927075 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560611d47eb0 con 0x560611debb60 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6c37fe640 10 client.18087 handle_command_reply: tid=0 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6ece7f740 1 client.18087 shutdown 2024-03-26T17:45:35.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1251927075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 msgr2=0x560611de00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de00c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8012700 tx=0x7fd6d8012bc0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1251927075 shutdown_connections 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560611debb60 0x560611dee020 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611de7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6c37fe640 0 client.18087 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de00c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.679+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1251927075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611ddfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.680+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1251927075 >> 172.21.15.105:0/1251927075 conn(0x560611dd0fa0 msgr2=0x560611dd01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.680+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1251927075 shutdown_connections 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.680+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1251927075 wait complete. 2024-03-26T17:45:35.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.680+0000 7fd6ece7f740 20 client.18087 trim_cache size 0 max 16384 2024-03-26T17:45:35.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6ece7f740 1 Processor -- start 2024-03-26T17:45:35.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6ece7f740 1 -- start start 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611dcf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611dcfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41af430 con 0x560611dd6420 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41b0f70 con 0x560611dd5a30 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41afb00 con 0x560611ddb590 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611dcf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51450/0 (socket says 172.21.15.105:51450) 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611dcf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56214/0 (socket says 172.21.15.105:56214) 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e37fe640 1 -- 172.21.15.105:0/674325567 learned_addr learned my addr 172.21.15.105:0/674325567 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e8829640 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611dcfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e3fff640 1 -- 172.21.15.105:0/674325567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 msgr2=0x560611dcfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611dcfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e3fff640 1 -- 172.21.15.105:0/674325567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 msgr2=0x560611de70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e3fff640 1 -- 172.21.15.105:0/674325567 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560611de0a20 con 0x560611dd5a30 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e37fe640 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.704+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611dcf930 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40be520 tx=0x7fd6d403ae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09ed22e53794f4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6c9ffb640 1 -- 172.21.15.105:0/674325567 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a1070 con 0x560611dd5a30 2024-03-26T17:45:35.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6c9ffb640 1 -- 172.21.15.105:0/674325567 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d40be370 con 0x560611dd5a30 2024-03-26T17:45:35.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6c9ffb640 1 -- 172.21.15.105:0/674325567 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40c2f50 con 0x560611dd5a30 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 -- 172.21.15.105:0/674325567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 msgr2=0x560611dcf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611dcf930 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40be520 tx=0x7fd6d403ae30 comp rx=0 tx=0).stop 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 -- 172.21.15.105:0/674325567 shutdown_connections 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611dcfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/674325567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611dcf930 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 -- 172.21.15.105:0/674325567 >> 172.21.15.105:0/674325567 conn(0x560611dd0fa0 msgr2=0x560611dd7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 -- 172.21.15.105:0/674325567 shutdown_connections 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.705+0000 7fd6ece7f740 1 -- 172.21.15.105:0/674325567 wait complete. 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 1 -- start start 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 1 Processor -- start 2024-03-26T17:45:35.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611de41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611de5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41b08a0 con 0x560611dd6420 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41b01d0 con 0x560611dd5a30 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41ad410 con 0x560611ddb590 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6ece7f740 10 client.0 _ll_register_callbacks cb 0x7fd6d8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611de41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6e37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51452/0 (socket says 172.21.15.105:51452) 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6e37fe640 1 -- 172.21.15.105:0/1061961329 learned_addr learned my addr 172.21.15.105:0/1061961329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6e8829640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611de5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.706+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611de41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56222/0 (socket says 172.21.15.105:56222) 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6e3fff640 1 -- 172.21.15.105:0/1061961329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 msgr2=0x560611de5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611de5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6e3fff640 1 -- 172.21.15.105:0/1061961329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 msgr2=0x560611de46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6e3fff640 1 -- 172.21.15.105:0/1061961329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560611de5cf0 con 0x560611dd5a30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6e3fff640 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611de41b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4006010 tx=0x7fd6d40ba010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c90b93bd09f5ca58 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1061961329 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a1070 con 0x560611dd5a30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1061961329 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd6d40c3190 con 0x560611dd5a30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1061961329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d40be370 con 0x560611dd5a30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1061961329 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd6d40bd5d0 con 0x560611dd5a30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6ece7f740 10 client.27923 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1061961329 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560611de6ff0 con 0x560611dd5a30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.707+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1061961329 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d40be370 con 0x560611dd5a30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1061961329 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd6d40bdf60 con 0x560611dd5a30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6ece7f740 10 client.27923 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6ece7f740 10 client.27923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6ece7f740 20 client.27923 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560611debb60 0x560611e53f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6ece7f740 4 client.27923 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1061961329 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560611d42200 con 0x560611debb60 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6e37fe640 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560611debb60 0x560611e53f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6e37fe640 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560611debb60 0x560611e53f80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cf84162e938eefc server_cookie=1ec6daba27690aa2 in_seq=0 out_seq=0 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6c37fe640 10 client.27923 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6c37fe640 1 -- 172.21.15.105:0/1061961329 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560611d42200 con 0x560611debb60 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.708+0000 7fd6c37fe640 10 client.27923 handle_command_reply: tid=0 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 client.27923 shutdown 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1061961329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 msgr2=0x560611de41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611de41b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4006010 tx=0x7fd6d40ba010 comp rx=0 tx=0).stop 2024-03-26T17:45:35.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1061961329 shutdown_connections 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560611debb60 0x560611e53f80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560611ddb590 0x560611de5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560611dd6420 0x560611de46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6c37fe640 0 client.27923 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 --2- 172.21.15.105:0/1061961329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560611dd5a30 0x560611de41b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1061961329 >> 172.21.15.105:0/1061961329 conn(0x560611dd0fa0 msgr2=0x560611dd4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1061961329 shutdown_connections 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 1 -- 172.21.15.105:0/1061961329 wait complete. 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ece7f740 20 client.27923 trim_cache size 0 max 16384 2024-03-26T17:45:35.707 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ea28c640 1 -- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6cc0a2f20 msgr2=0x7fd6cc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.709+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6cc0a2f20 0x7fd6cc0a53e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x560611dd6c80 tx=0x7fd6dc004ab0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 -- 172.21.15.105:0/170713348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6e40ff980 msgr2=0x7fd6e419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6e40ff980 0x7fd6e419e170 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4000c00 tx=0x7fd6d4005dc0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 -- 172.21.15.105:0/170713348 shutdown_connections 2024-03-26T17:45:35.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6cc0a2f20 0x7fd6cc0a53e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e4102f10 0x7fd6e41a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6e4100340 0x7fd6e419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 --2- 172.21.15.105:0/170713348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6e40ff980 0x7fd6e419e170 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 -- 172.21.15.105:0/170713348 >> 172.21.15.105:0/170713348 conn(0x7fd6e40fb4e0 msgr2=0x7fd6e410ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 -- 172.21.15.105:0/170713348 shutdown_connections 2024-03-26T17:45:35.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.710+0000 7fd6ea28c640 1 -- 172.21.15.105:0/170713348 wait complete. 2024-03-26T17:45:35.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:45:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 -- 172.21.15.105:0/2334435010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fd01007e0 msgr2=0x7f8fd0100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/2334435010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fd01007e0 0x7f8fd0100c00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009900 tx=0x7f8fc002bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 -- 172.21.15.105:0/2334435010 shutdown_connections 2024-03-26T17:45:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/2334435010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fd0102390 0x7f8fd0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/2334435010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd01019d0 0x7f8fd0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/2334435010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fd01007e0 0x7f8fd0100c00 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 -- 172.21.15.105:0/2334435010 >> 172.21.15.105:0/2334435010 conn(0x7f8fd00fbd50 msgr2=0x7f8fd00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 -- 172.21.15.105:0/2334435010 shutdown_connections 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.837+0000 7f8fd5e85640 1 -- 172.21.15.105:0/2334435010 wait complete. 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fd5e85640 1 Processor -- start 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fd5e85640 1 -- start start 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fd5e85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd01007e0 0x7f8fd01959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fd5e85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fd01019d0 0x7f8fd0195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fd5e85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fd0102390 0x7f8fd019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fd5e85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010b210 con 0x7f8fd0102390 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fd5e85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010b090 con 0x7f8fd01019d0 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fd5e85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd010b390 con 0x7f8fd01007e0 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd01007e0 0x7f8fd01959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fd01019d0 0x7f8fd0195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd01007e0 0x7f8fd01959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47872/0 (socket says 172.21.15.105:47872) 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fcf7fe640 1 -- 172.21.15.105:0/3307845907 learned_addr learned my addr 172.21.15.105:0/3307845907 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fcffff640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fd0102390 0x7f8fd019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fceffd640 1 -- 172.21.15.105:0/3307845907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd01007e0 msgr2=0x7f8fd01959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fceffd640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd01007e0 0x7f8fd01959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fceffd640 1 -- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fd0102390 msgr2=0x7f8fd019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fceffd640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fd0102390 0x7f8fd019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.838+0000 7f8fceffd640 1 -- 172.21.15.105:0/3307845907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019d5c0 con 0x7f8fd01019d0 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.839+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd01007e0 0x7f8fd01959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.839+0000 7f8fceffd640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fd01019d0 0x7f8fd0195f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400e9f0 tx=0x7f8fc400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.839+0000 7f8fccff9640 1 -- 172.21.15.105:0/3307845907 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400ccc0 con 0x7f8fd01019d0 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.839+0000 7f8fd5e85640 1 -- 172.21.15.105:0/3307845907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019d8b0 con 0x7f8fd01019d0 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.839+0000 7f8fccff9640 1 -- 172.21.15.105:0/3307845907 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fc4005020 con 0x7f8fd01019d0 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.839+0000 7f8fccff9640 1 -- 172.21.15.105:0/3307845907 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004a00 con 0x7f8fd01019d0 2024-03-26T17:45:35.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.839+0000 7f8fd5e85640 1 -- 172.21.15.105:0/3307845907 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd019de00 con 0x7f8fd01019d0 2024-03-26T17:45:35.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.841+0000 7f8fccff9640 1 -- 172.21.15.105:0/3307845907 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8fc4004ba0 con 0x7f8fd01019d0 2024-03-26T17:45:35.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fd8a78740 1 Processor -- start 2024-03-26T17:45:35.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fd8a78740 1 -- start start 2024-03-26T17:45:35.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81eabf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81eaea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ed81e0b970 con 0x55ed81ea6a30 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed81db3d70 con 0x55ed81ea7420 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed81e07920 con 0x55ed81eac590 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51480/0 (socket says 172.21.15.105:51480) 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81eabf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 -- 172.21.15.105:0/8320260 learned_addr learned my addr 172.21.15.105:0/8320260 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fceffd640 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81eaea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fccff9640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fb80a2f20 0x7f8fb80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 -- 172.21.15.105:0/8320260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 msgr2=0x55ed81eaea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81eaea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 -- 172.21.15.105:0/8320260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 msgr2=0x55ed81eabf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81eabf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 -- 172.21.15.105:0/8320260 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed81b1f3f0 con 0x55ed81ea6a30 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81eabf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fccff9640 1 -- 172.21.15.105:0/3307845907 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8fc40cf450 con 0x7f8fd01019d0 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.842+0000 7f8fcffff640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fb80a2f20 0x7f8fb80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.843+0000 7f8fcffff640 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea6e50 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8011590 tx=0x7f8fc8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44002232398ac4cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.843+0000 7f8fcffff640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fb80a2f20 0x7f8fb80a53e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8022000 tx=0x7f8fc8023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.843+0000 7f8fb5ffb640 1 -- 172.21.15.105:0/8320260 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8017050 con 0x55ed81ea6a30 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.843+0000 7f8fb5ffb640 1 -- 172.21.15.105:0/8320260 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fc800fcf0 con 0x55ed81ea6a30 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.843+0000 7f8fb5ffb640 1 -- 172.21.15.105:0/8320260 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc801bb40 con 0x55ed81ea6a30 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 -- 172.21.15.105:0/8320260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 msgr2=0x55ed81ea6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea6e50 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8011590 tx=0x7f8fc8011a50 comp rx=0 tx=0).stop 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 -- 172.21.15.105:0/8320260 shutdown_connections 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81eaea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81eabf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/8320260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea6e50 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 -- 172.21.15.105:0/8320260 >> 172.21.15.105:0/8320260 conn(0x55ed81ea1fa0 msgr2=0x55ed81ea23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 -- 172.21.15.105:0/8320260 shutdown_connections 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 -- 172.21.15.105:0/8320260 wait complete. 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 -- start start 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.844+0000 7f8fd8a78740 1 Processor -- start 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea7420 0x55ed81eb58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81ebc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ed81eb3950 con 0x55ed81ea7420 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ed81dc7310 con 0x55ed81ea6a30 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ed81eb3ad0 con 0x55ed81eac590 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fd8a78740 10 client.0 _ll_register_callbacks cb 0x55ed81e02c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea7420 0x55ed81eb58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea7420 0x55ed81eb58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51484/0 (socket says 172.21.15.105:51484) 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 -- 172.21.15.105:0/1965328203 learned_addr learned my addr 172.21.15.105:0/1965328203 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fcffff640 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81ebc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 -- 172.21.15.105:0/1965328203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 msgr2=0x55ed81ebc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81ebc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 -- 172.21.15.105:0/1965328203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 msgr2=0x55ed81eb5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 -- 172.21.15.105:0/1965328203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed81be97d0 con 0x55ed81ea7420 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fceffd640 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea7420 0x55ed81eb58d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400c6a0 tx=0x7f8fc4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233be9b2d7e88c89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8faeffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8faeffd640 1 -- 172.21.15.105:0/1965328203 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401c040 con 0x55ed81ea7420 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8faeffd640 1 -- 172.21.15.105:0/1965328203 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8fc40051d0 con 0x55ed81ea7420 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.845+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1965328203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc800fcf0 con 0x55ed81ea7420 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8faeffd640 1 -- 172.21.15.105:0/1965328203 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8fc409c1e0 con 0x55ed81ea7420 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fd8a78740 10 client.18105 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1965328203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed81ebfcd0 con 0x55ed81ea7420 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8faeffd640 1 -- 172.21.15.105:0/1965328203 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fc800fcf0 con 0x55ed81ea7420 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8faeffd640 1 -- 172.21.15.105:0/1965328203 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8fc40a4060 con 0x55ed81ea7420 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fd8a78740 10 client.18105 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fd8a78740 10 client.18105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fd8a78740 20 client.18105 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed81ec1340 0x55ed81ec3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fd8a78740 4 client.18105 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1965328203 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed81e18eb0 con 0x55ed81ec1340 2024-03-26T17:45:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.846+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed81ec1340 0x55ed81ec3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.847+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed81ec1340 0x55ed81ec3790 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c73ad1eab07a945 server_cookie=5aea714e3fc1500c in_seq=0 out_seq=0 2024-03-26T17:45:35.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.847+0000 7f8faeffd640 10 client.18105 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8faeffd640 1 -- 172.21.15.105:0/1965328203 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ed81e18eb0 con 0x55ed81ec1340 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8faeffd640 10 client.18105 handle_command_reply: tid=0 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 client.18105 shutdown 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1965328203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea7420 msgr2=0x55ed81eb58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea7420 0x55ed81eb58d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400c6a0 tx=0x7f8fc4009f60 comp rx=0 tx=0).stop 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1965328203 shutdown_connections 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed81ec1340 0x55ed81ec3790 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81ebc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea7420 0x55ed81eb58d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1965328203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1965328203 >> 172.21.15.105:0/1965328203 conn(0x55ed81ea1fa0 msgr2=0x55ed81ea11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8faeffd640 0 client.18105 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:35.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1965328203 shutdown_connections 2024-03-26T17:45:35.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.848+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1965328203 wait complete. 2024-03-26T17:45:35.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.849+0000 7f8fd8a78740 20 client.18105 trim_cache size 0 max 16384 2024-03-26T17:45:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fd8a78740 1 Processor -- start 2024-03-26T17:45:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fd8a78740 1 -- start start 2024-03-26T17:45:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81ebc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81ea0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01a6b60 con 0x55ed81ea6a30 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01a86a0 con 0x55ed81ea7420 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01a7230 con 0x55ed81eac590 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fcffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81ea0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51490/0 (socket says 172.21.15.105:51490) 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81ebc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.864+0000 7f8fcf7fe640 1 -- 172.21.15.105:0/1102021480 learned_addr learned my addr 172.21.15.105:0/1102021480 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fcf7fe640 1 -- 172.21.15.105:0/1102021480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 msgr2=0x55ed81ea0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/1102021480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81ea0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fcf7fe640 1 -- 172.21.15.105:0/1102021480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 msgr2=0x55ed81ebc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/1102021480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81ebc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fcf7fe640 1 -- 172.21.15.105:0/1102021480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed81ec10e0 con 0x55ed81ea6a30 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/1102021480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea0930 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0000c00 tx=0x7f8fc0005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c90b97c10f8fc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fb57fa640 1 -- 172.21.15.105:0/1102021480 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004180 con 0x55ed81ea6a30 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fb57fa640 1 -- 172.21.15.105:0/1102021480 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fc00054f0 con 0x55ed81ea6a30 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.865+0000 7f8fb57fa640 1 -- 172.21.15.105:0/1102021480 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004d90 con 0x55ed81ea6a30 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1102021480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 msgr2=0x55ed81ea0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1102021480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea0930 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0000c00 tx=0x7f8fc0005e70 comp rx=0 tx=0).stop 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1102021480 shutdown_connections 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1102021480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81eac590 0x55ed81ea0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1102021480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea7420 0x55ed81ebc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/1102021480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81ea6a30 0x55ed81ea0930 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1102021480 >> 172.21.15.105:0/1102021480 conn(0x55ed81ea1fa0 msgr2=0x55ed81eb5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1102021480 shutdown_connections 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 -- 172.21.15.105:0/1102021480 wait complete. 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 -- start start 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 Processor -- start 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81ea7420 0x55ed81eba440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.866+0000 7f8fd8a78740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81eac590 0x55ed81ebb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01a7fd0 con 0x55ed81eac590 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01a7900 con 0x55ed81ea6a30 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fd8a78740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01a4b40 con 0x55ed81ea7420 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fd8a78740 10 client.0 _ll_register_callbacks cb 0x7f8fc400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81eac590 0x55ed81ebb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56276/0 (socket says 172.21.15.105:56276) 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81ea7420 0x55ed81eba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81eac590 0x55ed81ebb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51506/0 (socket says 172.21.15.105:51506) 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcf7fe640 1 -- 172.21.15.105:0/4043223925 learned_addr learned my addr 172.21.15.105:0/4043223925 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcffff640 1 -- 172.21.15.105:0/4043223925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81ea7420 msgr2=0x55ed81eba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcffff640 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81ea7420 0x55ed81eba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcffff640 1 -- 172.21.15.105:0/4043223925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 msgr2=0x55ed81eb9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcffff640 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcffff640 1 -- 172.21.15.105:0/4043223925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed81ebba40 con 0x55ed81eac590 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fceffd640 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81ea7420 0x55ed81eba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fcffff640 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81eac590 0x55ed81ebb450 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009560 tx=0x7f8fc8004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aa83e1a4c93ace server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8faeffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8faeffd640 1 -- 172.21.15.105:0/4043223925 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8038060 con 0x55ed81eac590 2024-03-26T17:45:35.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8faeffd640 1 -- 172.21.15.105:0/4043223925 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8fc800fa50 con 0x55ed81eac590 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.867+0000 7f8fd8a78740 1 -- 172.21.15.105:0/4043223925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc00054f0 con 0x55ed81eac590 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8faeffd640 1 -- 172.21.15.105:0/4043223925 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8fc8017020 con 0x55ed81eac590 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fd8a78740 10 client.18111 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fd8a78740 1 -- 172.21.15.105:0/4043223925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed81f27690 con 0x55ed81eac590 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8faeffd640 1 -- 172.21.15.105:0/4043223925 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fc00054f0 con 0x55ed81eac590 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8faeffd640 1 -- 172.21.15.105:0/4043223925 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8fc801a070 con 0x55ed81eac590 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fd8a78740 10 client.18111 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fd8a78740 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fd8a78740 20 client.18111 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed81f27dc0 0x55ed81f2a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fd8a78740 4 client.18111 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fd8a78740 1 -- 172.21.15.105:0/4043223925 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed81e13200 con 0x55ed81f27dc0 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.868+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed81f27dc0 0x55ed81f2a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:35.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8faeffd640 10 client.18111 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fcf7fe640 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed81f27dc0 0x55ed81f2a1e0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9e24d6b690c259e server_cookie=a19ee2c305fa8bb in_seq=0 out_seq=0 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8faeffd640 1 -- 172.21.15.105:0/4043223925 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed81e13200 con 0x55ed81f27dc0 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8faeffd640 10 client.18111 handle_command_reply: tid=0 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 client.18111 shutdown 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 -- 172.21.15.105:0/4043223925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81eac590 msgr2=0x55ed81ebb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81eac590 0x55ed81ebb450 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009560 tx=0x7f8fc8004f50 comp rx=0 tx=0).stop 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 -- 172.21.15.105:0/4043223925 shutdown_connections 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ed81f27dc0 0x55ed81f2a1e0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ed81eac590 0x55ed81ebb450 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ed81ea7420 0x55ed81eba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 --2- 172.21.15.105:0/4043223925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ed81ea6a30 0x55ed81eb9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 -- 172.21.15.105:0/4043223925 >> 172.21.15.105:0/4043223925 conn(0x55ed81ea1fa0 msgr2=0x55ed81ea5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8faeffd640 0 client.18111 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:35.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 -- 172.21.15.105:0/4043223925 shutdown_connections 2024-03-26T17:45:35.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 1 -- 172.21.15.105:0/4043223925 wait complete. 2024-03-26T17:45:35.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.869+0000 7f8fd8a78740 20 client.18111 trim_cache size 0 max 16384 2024-03-26T17:45:35.871 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:45:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 -- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fb80a2f20 msgr2=0x7f8fb80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fb80a2f20 0x7f8fb80a53e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8022000 tx=0x7f8fc8023000 comp rx=0 tx=0).stop 2024-03-26T17:45:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 -- 172.21.15.105:0/3307845907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fd01019d0 msgr2=0x7f8fd0195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:35.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fd01019d0 0x7f8fd0195f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400e9f0 tx=0x7f8fc400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 -- 172.21.15.105:0/3307845907 shutdown_connections 2024-03-26T17:45:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8fb80a2f20 0x7f8fb80a53e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fd0102390 0x7f8fd019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fd01019d0 0x7f8fd0195f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 --2- 172.21.15.105:0/3307845907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd01007e0 0x7f8fd01959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 -- 172.21.15.105:0/3307845907 >> 172.21.15.105:0/3307845907 conn(0x7f8fd00fbd50 msgr2=0x7f8fd0104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 -- 172.21.15.105:0/3307845907 shutdown_connections 2024-03-26T17:45:35.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:35.870+0000 7f8fd5e85640 1 -- 172.21.15.105:0/3307845907 wait complete. 2024-03-26T17:45:36.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 223151 3.15 MB/sec execute 259 sec latency 17.423 ms 2024-03-26T17:45:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:35 smithi144 ceph-mon[36095]: pgmap v565: 97 pgs: 97 active+clean; 28 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:45:36.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:35 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/986419245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:45:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:35 smithi105 ceph-mon[32280]: pgmap v565: 97 pgs: 97 active+clean; 28 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:45:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:35 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/986419245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:45:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:35 smithi135 ceph-mon[35636]: pgmap v565: 97 pgs: 97 active+clean; 28 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.5 MiB/s wr, 97 op/s 2024-03-26T17:45:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:35 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/986419245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:45:36.874 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:45:36.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:45:36.875 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:45:36.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 -- 172.21.15.105:0/3298538897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 msgr2=0x7f480c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 --2- 172.21.15.105:0/3298538897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 0x7f480c108c00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f4804009ca0 tx=0x7f480402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 -- 172.21.15.105:0/3298538897 shutdown_connections 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 --2- 172.21.15.105:0/3298538897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 0x7f480c108c00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 --2- 172.21.15.105:0/3298538897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f480c1019b0 0x7f480c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 --2- 172.21.15.105:0/3298538897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f480c1007c0 0x7f480c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 -- 172.21.15.105:0/3298538897 >> 172.21.15.105:0/3298538897 conn(0x7f480c0fbd50 msgr2=0x7f480c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 -- 172.21.15.105:0/3298538897 shutdown_connections 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 -- 172.21.15.105:0/3298538897 wait complete. 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 Processor -- start 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.002+0000 7f4813d60640 1 -- start start 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f4813d60640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f480c1007c0 0x7f480c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f4813d60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f480c1019b0 0x7f480c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:36.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f4813d60640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 0x7f480c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f4813d60640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f480c10c2d0 con 0x7f480c102370 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f4813d60640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f480c10c150 con 0x7f480c1007c0 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f4813d60640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f480c10c450 con 0x7f480c1019b0 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 0x7f480c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f4811afc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f480c1007c0 0x7f480c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 0x7f480c1a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51528/0 (socket says 172.21.15.105:51528) 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 -- 172.21.15.105:0/2560943987 learned_addr learned my addr 172.21.15.105:0/2560943987 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48112fb640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f480c1019b0 0x7f480c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 -- 172.21.15.105:0/2560943987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f480c1019b0 msgr2=0x7f480c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f480c1019b0 0x7f480c19a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 -- 172.21.15.105:0/2560943987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f480c1007c0 msgr2=0x7f480c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f480c1007c0 0x7f480c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 -- 172.21.15.105:0/2560943987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1a1930 con 0x7f480c102370 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f48122fd640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 0x7f480c1a1290 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f480402c700 tx=0x7f48040328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.003+0000 7f4802ffd640 1 -- 172.21.15.105:0/2560943987 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804041020 con 0x7f480c102370 2024-03-26T17:45:36.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.004+0000 7f4802ffd640 1 -- 172.21.15.105:0/2560943987 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4804003e20 con 0x7f480c102370 2024-03-26T17:45:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.004+0000 7f4813d60640 1 -- 172.21.15.105:0/2560943987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c1a1bc0 con 0x7f480c102370 2024-03-26T17:45:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.004+0000 7f4802ffd640 1 -- 172.21.15.105:0/2560943987 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804039040 con 0x7f480c102370 2024-03-26T17:45:36.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.004+0000 7f4813d60640 1 -- 172.21.15.105:0/2560943987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c1a20b0 con 0x7f480c102370 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4816a1f740 1 Processor -- start 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4802ffd640 1 -- 172.21.15.105:0/2560943987 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4804044020 con 0x7f480c102370 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4816a1f740 1 -- start start 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd5b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd5e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4816a1f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca8dcbb630 con 0x55ca8dd56b80 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4816a1f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca8dc63e10 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4816a1f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca8dcb75e0 con 0x55ca8dd56190 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4811afc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd5b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f48112fb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4811afc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd5b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51544/0 (socket says 172.21.15.105:51544) 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.007+0000 7f4811afc640 1 -- 172.21.15.105:0/2290024035 learned_addr learned my addr 172.21.15.105:0/2290024035 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48112fb640 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd5e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f4802ffd640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47f80a2f20 0x7f47f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48122fd640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47f80a2f20 0x7f47f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48112fb640 1 -- 172.21.15.105:0/2290024035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 msgr2=0x55ca8dd5e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48112fb640 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd5e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48112fb640 1 -- 172.21.15.105:0/2290024035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 msgr2=0x55ca8dd5b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48112fb640 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd5b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48112fb640 1 -- 172.21.15.105:0/2290024035 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8dc7a740 con 0x55ca8dd56190 2024-03-26T17:45:37.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f4811afc640 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd5b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f4802ffd640 1 -- 172.21.15.105:0/2560943987 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f48040f1650 con 0x7f480c102370 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48112fb640 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd565b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009730 tx=0x7f47fc0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183f5bd76e502d51 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f48122fd640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47f80a2f20 0x7f47f80a53e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f48040ba680 tx=0x7f48040909e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f47f7fff640 1 -- 172.21.15.105:0/2290024035 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc004050 con 0x55ca8dd56190 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f47f7fff640 1 -- 172.21.15.105:0/2290024035 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47fc0054d0 con 0x55ca8dd56190 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.008+0000 7f47f7fff640 1 -- 172.21.15.105:0/2290024035 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc004c40 con 0x55ca8dd56190 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 -- 172.21.15.105:0/2290024035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 msgr2=0x55ca8dd565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd565b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009730 tx=0x7f47fc0096c0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 -- 172.21.15.105:0/2290024035 shutdown_connections 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd5e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd5b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2290024035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd565b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 -- 172.21.15.105:0/2290024035 >> 172.21.15.105:0/2290024035 conn(0x55ca8dd51700 msgr2=0x55ca8dd51b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 -- 172.21.15.105:0/2290024035 shutdown_connections 2024-03-26T17:45:37.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 -- 172.21.15.105:0/2290024035 wait complete. 2024-03-26T17:45:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 -- start start 2024-03-26T17:45:37.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.009+0000 7f4816a1f740 1 Processor -- start 2024-03-26T17:45:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56190 0x55ca8dd606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56b80 0x55ca8dd60c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd67c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4816a1f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca8dd5eca0 con 0x55ca8dd56190 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4816a1f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca8dc771a0 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4816a1f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca8dd5ee20 con 0x55ca8dd56b80 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4816a1f740 10 client.0 _ll_register_callbacks cb 0x55ca8d9cec60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f48112fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56190 0x55ca8dd606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f48112fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56190 0x55ca8dd606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51558/0 (socket says 172.21.15.105:51558) 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4811afc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56b80 0x55ca8dd60c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f48112fb640 1 -- 172.21.15.105:0/2672525857 learned_addr learned my addr 172.21.15.105:0/2672525857 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f48112fb640 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd67c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4811afc640 1 -- 172.21.15.105:0/2672525857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 msgr2=0x55ca8dd67c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4811afc640 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd67c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4811afc640 1 -- 172.21.15.105:0/2672525857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56190 msgr2=0x55ca8dd606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4811afc640 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56190 0x55ca8dd606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f4811afc640 1 -- 172.21.15.105:0/2672525857 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8da977e0 con 0x55ca8dd56b80 2024-03-26T17:45:37.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.010+0000 7f48112fb640 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56190 0x55ca8dd606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4811afc640 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56b80 0x55ca8dd60c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55ca8dd56f80 tx=0x7f4808012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb6903f2e98fe98 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f47f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f47f57fa640 1 -- 172.21.15.105:0/2672525857 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808004030 con 0x55ca8dd56b80 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 1 -- 172.21.15.105:0/2672525857 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47fc0054d0 con 0x55ca8dd56b80 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f47f57fa640 1 -- 172.21.15.105:0/2672525857 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f48080041d0 con 0x55ca8dd56b80 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f47f57fa640 1 -- 172.21.15.105:0/2672525857 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48080043d0 con 0x55ca8dd56b80 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 10 client.27955 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 1 -- 172.21.15.105:0/2672525857 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ca8daf7f70 con 0x55ca8dd56b80 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f47f57fa640 1 -- 172.21.15.105:0/2672525857 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47fc0054d0 con 0x55ca8dd56b80 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f47f57fa640 1 -- 172.21.15.105:0/2672525857 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4808003ab0 con 0x55ca8dd56b80 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 10 client.27955 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 10 client.27955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 20 client.27955 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ca8dd6c4f0 0x55ca8dd6e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 4 client.27955 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:37.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.011+0000 7f4816a1f740 1 -- 172.21.15.105:0/2672525857 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca8dcc8b70 con 0x55ca8dd6c4f0 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.012+0000 7f48112fb640 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ca8dd6c4f0 0x55ca8dd6e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.012+0000 7f48112fb640 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ca8dd6c4f0 0x55ca8dd6e970 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98b983b69408de74 server_cookie=35723b0ee73e7975 in_seq=0 out_seq=0 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.012+0000 7f47f57fa640 10 client.27955 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.013+0000 7f47f57fa640 1 -- 172.21.15.105:0/2672525857 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ca8dcc8b70 con 0x55ca8dd6c4f0 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.013+0000 7f47f57fa640 10 client.27955 handle_command_reply: tid=0 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.013+0000 7f4816a1f740 1 client.27955 shutdown 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.013+0000 7f4816a1f740 1 -- 172.21.15.105:0/2672525857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56b80 msgr2=0x55ca8dd60c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.013+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56b80 0x55ca8dd60c20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55ca8dd56f80 tx=0x7f4808012b60 comp rx=0 tx=0).stop 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.013+0000 7f4816a1f740 1 -- 172.21.15.105:0/2672525857 shutdown_connections 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.013+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ca8dd6c4f0 0x55ca8dd6e970 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.014+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd67c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.014+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56b80 0x55ca8dd60c20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.014+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2672525857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56190 0x55ca8dd606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.014+0000 7f47f57fa640 0 client.27955 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.014+0000 7f4816a1f740 1 -- 172.21.15.105:0/2672525857 >> 172.21.15.105:0/2672525857 conn(0x55ca8dd51700 msgr2=0x55ca8dd50900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.014+0000 7f4816a1f740 1 -- 172.21.15.105:0/2672525857 shutdown_connections 2024-03-26T17:45:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.014+0000 7f4816a1f740 1 -- 172.21.15.105:0/2672525857 wait complete. 2024-03-26T17:45:37.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.014+0000 7f4816a1f740 20 client.27955 trim_cache size 0 max 16384 2024-03-26T17:45:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.033+0000 7f4816a1f740 1 Processor -- start 2024-03-26T17:45:37.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.033+0000 7f4816a1f740 1 -- start start 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56190 0x55ca8dd50070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd67c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4816a1f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1aafa0 con 0x55ca8dd56b80 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4816a1f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1acae0 con 0x55ca8dd56190 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4816a1f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1ab670 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd67c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd67c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51562/0 (socket says 172.21.15.105:51562) 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f48112fb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56190 0x55ca8dd50070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 -- 172.21.15.105:0/2546439794 learned_addr learned my addr 172.21.15.105:0/2546439794 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f48112fb640 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 -- 172.21.15.105:0/2546439794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd5bcf0 msgr2=0x55ca8dd505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 -- 172.21.15.105:0/2546439794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56190 msgr2=0x55ca8dd50070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56190 0x55ca8dd50070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 -- 172.21.15.105:0/2546439794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8dd58720 con 0x55ca8dd56b80 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f48112fb640 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56190 0x55ca8dd50070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f4811afc640 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd67c20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f480801c4b0 tx=0x7f48080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cb4a4b93cb25b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.034+0000 7f47f77fe640 1 -- 172.21.15.105:0/2546439794 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808000e60 con 0x55ca8dd56b80 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f47f77fe640 1 -- 172.21.15.105:0/2546439794 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4808022030 con 0x55ca8dd56b80 2024-03-26T17:45:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f47f77fe640 1 -- 172.21.15.105:0/2546439794 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808003850 con 0x55ca8dd56b80 2024-03-26T17:45:37.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 223627 3.16 MB/sec execute 260 sec latency 15.742 ms 2024-03-26T17:45:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 -- 172.21.15.105:0/2546439794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 msgr2=0x55ca8dd67c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd67c20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f480801c4b0 tx=0x7f48080057b0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 -- 172.21.15.105:0/2546439794 shutdown_connections 2024-03-26T17:45:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd56b80 0x55ca8dd67c20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 --2- 172.21.15.105:0/2546439794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56190 0x55ca8dd50070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 -- 172.21.15.105:0/2546439794 >> 172.21.15.105:0/2546439794 conn(0x55ca8dd51700 msgr2=0x55ca8dd6e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 -- 172.21.15.105:0/2546439794 shutdown_connections 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.035+0000 7f4816a1f740 1 -- 172.21.15.105:0/2546439794 wait complete. 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 1 -- start start 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 1 Processor -- start 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56b80 0x55ca8dd65cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd66cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1ac410 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1abd40 con 0x55ca8dd56b80 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1a8f80 con 0x55ca8dd56190 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4816a1f740 10 client.0 _ll_register_callbacks cb 0x7f4808009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd66cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd66cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51572/0 (socket says 172.21.15.105:51572) 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 -- 172.21.15.105:0/992120502 learned_addr learned my addr 172.21.15.105:0/992120502 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f4811afc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56b80 0x55ca8dd65cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 -- 172.21.15.105:0/992120502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 msgr2=0x55ca8dd65770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd65770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 -- 172.21.15.105:0/992120502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56b80 msgr2=0x55ca8dd65cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56b80 0x55ca8dd65cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.036+0000 7f48112fb640 1 -- 172.21.15.105:0/992120502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8d98f170 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f48112fb640 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd66cc0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f47fc011750 tx=0x7f47fc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00f858695d3d249 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f47f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f47f57fa640 1 -- 172.21.15.105:0/992120502 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc009ba0 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f47f57fa640 1 -- 172.21.15.105:0/992120502 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f47fc004c40 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 1 -- 172.21.15.105:0/992120502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4808022030 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f47f57fa640 1 -- 172.21.15.105:0/992120502 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47fc004e40 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 10 client.18129 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 1 -- 172.21.15.105:0/992120502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ca8ddd2ba0 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f47f57fa640 1 -- 172.21.15.105:0/992120502 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4808022030 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f47f57fa640 1 -- 172.21.15.105:0/992120502 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f47fc00f800 con 0x55ca8dd5bcf0 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 10 client.18129 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 10 client.18129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 20 client.18129 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ca8ddd3410 0x55ca8ddd5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 4 client.18129 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4816a1f740 1 -- 172.21.15.105:0/992120502 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ca8dcc2ec0 con 0x55ca8ddd3410 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.037+0000 7f4811afc640 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ca8ddd3410 0x55ca8ddd5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.038+0000 7f4811afc640 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ca8ddd3410 0x55ca8ddd5830 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a4a6cc82f6483f server_cookie=6e552e557b56dea2 in_seq=0 out_seq=0 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.038+0000 7f47f57fa640 10 client.18129 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f47f57fa640 1 -- 172.21.15.105:0/992120502 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ca8dcc2ec0 con 0x55ca8ddd3410 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f47f57fa640 10 client.18129 handle_command_reply: tid=0 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 client.18129 shutdown 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 -- 172.21.15.105:0/992120502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd5bcf0 msgr2=0x55ca8dd66cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd66cc0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f47fc011750 tx=0x7f47fc0096c0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 -- 172.21.15.105:0/992120502 shutdown_connections 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ca8ddd3410 0x55ca8ddd5830 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca8dd5bcf0 0x55ca8dd66cc0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca8dd56b80 0x55ca8dd65cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f47f57fa640 0 client.18129 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 --2- 172.21.15.105:0/992120502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca8dd56190 0x55ca8dd65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 -- 172.21.15.105:0/992120502 >> 172.21.15.105:0/992120502 conn(0x55ca8dd51700 msgr2=0x55ca8dd54960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 -- 172.21.15.105:0/992120502 shutdown_connections 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 1 -- 172.21.15.105:0/992120502 wait complete. 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.040+0000 7f4816a1f740 20 client.18129 trim_cache size 0 max 16384 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:45:37.037 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:45:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "9066724e-dd71-4e2f-9be9-ba686454d65c", 2024-03-26T17:45:37.038 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:45:37.038 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 -- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47f80a2f20 msgr2=0x7f47f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47f80a2f20 0x7f47f80a53e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f48040ba680 tx=0x7f48040909e0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 -- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 msgr2=0x7f480c1a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 0x7f480c1a1290 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f480402c700 tx=0x7f48040328a0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 -- 172.21.15.105:0/2560943987 shutdown_connections 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47f80a2f20 0x7f47f80a53e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f480c102370 0x7f480c1a1290 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f480c1019b0 0x7f480c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 --2- 172.21.15.105:0/2560943987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f480c1007c0 0x7f480c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 -- 172.21.15.105:0/2560943987 >> 172.21.15.105:0/2560943987 conn(0x7f480c0fbd50 msgr2=0x7f480c0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 -- 172.21.15.105:0/2560943987 shutdown_connections 2024-03-26T17:45:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.041+0000 7f4813d60640 1 -- 172.21.15.105:0/2560943987 wait complete. 2024-03-26T17:45:37.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:45:37.044 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:45:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 -- 172.21.15.105:0/1283798849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1341007e0 msgr2=0x7fe134100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 --2- 172.21.15.105:0/1283798849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1341007e0 0x7fe134100c00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe124009900 tx=0x7fe12402bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 -- 172.21.15.105:0/1283798849 shutdown_connections 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 --2- 172.21.15.105:0/1283798849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe134102390 0x7fe134108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 --2- 172.21.15.105:0/1283798849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1341019d0 0x7fe134101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 --2- 172.21.15.105:0/1283798849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe1341007e0 0x7fe134100c00 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 -- 172.21.15.105:0/1283798849 >> 172.21.15.105:0/1283798849 conn(0x7fe1340fbd70 msgr2=0x7fe1340fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 -- 172.21.15.105:0/1283798849 shutdown_connections 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.174+0000 7fe139c97640 1 -- 172.21.15.105:0/1283798849 wait complete. 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.175+0000 7fe139c97640 1 Processor -- start 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.175+0000 7fe139c97640 1 -- start start 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.175+0000 7fe139c97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1341007e0 0x7fe134197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.175+0000 7fe139c97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1341019d0 0x7fe134198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.175+0000 7fe139c97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe134102390 0x7fe13419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.175+0000 7fe139c97640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe13410c300 con 0x7fe134102390 2024-03-26T17:45:37.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.175+0000 7fe139c97640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe13410c180 con 0x7fe1341019d0 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.175+0000 7fe139c97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe13410c480 con 0x7fe1341007e0 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe134102390 0x7fe13419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe134102390 0x7fe13419f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51584/0 (socket says 172.21.15.105:51584) 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 -- 172.21.15.105:0/331208988 learned_addr learned my addr 172.21.15.105:0/331208988 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1341007e0 0x7fe134197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe132ffd640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1341019d0 0x7fe134198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 -- 172.21.15.105:0/331208988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1341007e0 msgr2=0x7fe134197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1341007e0 0x7fe134197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 -- 172.21.15.105:0/331208988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1341019d0 msgr2=0x7fe134198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1341019d0 0x7fe134198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 -- 172.21.15.105:0/331208988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13419f770 con 0x7fe134102390 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe1337fe640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1341007e0 0x7fe134197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe132ffd640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1341019d0 0x7fe134198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:37.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe133fff640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe134102390 0x7fe13419f100 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00eb10 tx=0x7fe12c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe130ff9640 1 -- 172.21.15.105:0/331208988 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c004b20 con 0x7fe134102390 2024-03-26T17:45:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe130ff9640 1 -- 172.21.15.105:0/331208988 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe12c004cc0 con 0x7fe134102390 2024-03-26T17:45:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe130ff9640 1 -- 172.21.15.105:0/331208988 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c010690 con 0x7fe134102390 2024-03-26T17:45:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe139c97640 1 -- 172.21.15.105:0/331208988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13419fa60 con 0x7fe134102390 2024-03-26T17:45:37.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.176+0000 7fe139c97640 1 -- 172.21.15.105:0/331208988 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13419ff60 con 0x7fe134102390 2024-03-26T17:45:37.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.179+0000 7fe130ff9640 1 -- 172.21.15.105:0/331208988 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe12c005020 con 0x7fe134102390 2024-03-26T17:45:37.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.179+0000 7fe130ff9640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe11c0a2f20 0x7fe11c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.179+0000 7fe1337fe640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe11c0a2f20 0x7fe11c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.179+0000 7fe130ff9640 1 -- 172.21.15.105:0/331208988 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe12c0cf980 con 0x7fe134102390 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe13c884740 1 Processor -- start 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe13c884740 1 -- start start 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe13c884740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158db8f0 0x5632158dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe13c884740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe13c884740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158e1450 0x5632158e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe13c884740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563215840980 con 0x5632158db8f0 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe13c884740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5632157e8be0 con 0x5632158dc2e0 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe13c884740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56321583c930 con 0x5632158e1450 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe11c0a2f20 0x7fe11c0a53e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe124005ab0 tx=0x7fe124005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158db8f0 0x5632158dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe132ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158db8f0 0x5632158dbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51600/0 (socket says 172.21.15.105:51600) 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 -- 172.21.15.105:0/3085241425 learned_addr learned my addr 172.21.15.105:0/3085241425 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe133fff640 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158e1450 0x5632158e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 -- 172.21.15.105:0/3085241425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158e1450 msgr2=0x5632158e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158e1450 0x5632158e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 -- 172.21.15.105:0/3085241425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 msgr2=0x5632158e0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158e0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe1337fe640 1 -- 172.21.15.105:0/3085241425 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321561e7d0 con 0x5632158db8f0 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.180+0000 7fe132ffd640 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.181+0000 7fe1337fe640 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158db8f0 0x5632158dbd10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe124009230 tx=0x7fe12403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737007b7b25721d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.181+0000 7fe119ffb640 1 -- 172.21.15.105:0/3085241425 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124005180 con 0x5632158db8f0 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.181+0000 7fe119ffb640 1 -- 172.21.15.105:0/3085241425 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe124050070 con 0x5632158db8f0 2024-03-26T17:45:37.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.181+0000 7fe119ffb640 1 -- 172.21.15.105:0/3085241425 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124003850 con 0x5632158db8f0 2024-03-26T17:45:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.182+0000 7fe13c884740 1 -- 172.21.15.105:0/3085241425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158db8f0 msgr2=0x5632158dbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.182+0000 7fe13c884740 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158db8f0 0x5632158dbd10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe124009230 tx=0x7fe12403e000 comp rx=0 tx=0).stop 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.183+0000 7fe13c884740 1 -- 172.21.15.105:0/3085241425 shutdown_connections 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.183+0000 7fe13c884740 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158e1450 0x5632158e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.183+0000 7fe13c884740 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.183+0000 7fe13c884740 1 --2- 172.21.15.105:0/3085241425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158db8f0 0x5632158dbd10 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.183+0000 7fe13c884740 1 -- 172.21.15.105:0/3085241425 >> 172.21.15.105:0/3085241425 conn(0x5632158d6e60 msgr2=0x5632158d7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.183+0000 7fe13c884740 1 -- 172.21.15.105:0/3085241425 shutdown_connections 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.183+0000 7fe13c884740 1 -- 172.21.15.105:0/3085241425 wait complete. 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.183+0000 7fe13c884740 1 -- start start 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe13c884740 1 Processor -- start 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe13c884740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x5632158e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe13c884740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 0x5632158e6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe13c884740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158f1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe13c884740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5632158e43a0 con 0x5632158e1450 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe13c884740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5632157fc160 con 0x5632158db8f0 2024-03-26T17:45:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe13c884740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5632158e4520 con 0x5632158dc2e0 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe13c884740 10 client.0 _ll_register_callbacks cb 0x563215838720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x5632158e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x5632158e5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56336/0 (socket says 172.21.15.105:56336) 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe1337fe640 1 -- 172.21.15.105:0/2070533200 learned_addr learned my addr 172.21.15.105:0/2070533200 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe133fff640 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe132ffd640 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 0x5632158e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe133fff640 1 -- 172.21.15.105:0/2070533200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 msgr2=0x5632158e6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe133fff640 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 0x5632158e6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe133fff640 1 -- 172.21.15.105:0/2070533200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 msgr2=0x5632158e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe133fff640 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x5632158e5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.184+0000 7fe133fff640 1 -- 172.21.15.105:0/2070533200 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632155543f0 con 0x5632158e1450 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe133fff640 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158f1460 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe12c09bd70 tx=0x7fe12c09ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc13ff03396f95b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe112ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe112ffd640 1 -- 172.21.15.105:0/2070533200 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0b4070 con 0x5632158e1450 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe112ffd640 1 -- 172.21.15.105:0/2070533200 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe12c096de0 con 0x5632158e1450 2024-03-26T17:45:37.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe13c884740 1 -- 172.21.15.105:0/2070533200 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe124050070 con 0x5632158e1450 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe112ffd640 1 -- 172.21.15.105:0/2070533200 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe12c0a4a80 con 0x5632158e1450 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe13c884740 10 client.18141 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe13c884740 1 -- 172.21.15.105:0/2070533200 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5632155140f0 con 0x5632158e1450 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe112ffd640 1 -- 172.21.15.105:0/2070533200 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe124050070 con 0x5632158e1450 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe112ffd640 1 -- 172.21.15.105:0/2070533200 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe12c0a45f0 con 0x5632158e1450 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe13c884740 10 client.18141 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe13c884740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.185+0000 7fe13c884740 20 client.18141 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.186+0000 7fe13c884740 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5632158f5d90 0x5632158f81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.186+0000 7fe13c884740 4 client.18141 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.186+0000 7fe13c884740 1 -- 172.21.15.105:0/2070533200 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321584dec0 con 0x5632158f5d90 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.186+0000 7fe1337fe640 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5632158f5d90 0x5632158f81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.186+0000 7fe112ffd640 10 client.18141 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:37.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.186+0000 7fe1337fe640 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5632158f5d90 0x5632158f81e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589f8e2ac3e07aef server_cookie=147cb2dfbb94ab6b in_seq=0 out_seq=0 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe112ffd640 1 -- 172.21.15.105:0/2070533200 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56321584dec0 con 0x5632158f5d90 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe112ffd640 10 client.18141 handle_command_reply: tid=0 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 client.18141 shutdown 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 -- 172.21.15.105:0/2070533200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 msgr2=0x5632158f1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158f1460 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe12c09bd70 tx=0x7fe12c09ba90 comp rx=0 tx=0).stop 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 -- 172.21.15.105:0/2070533200 shutdown_connections 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5632158f5d90 0x5632158f81e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158f1460 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 0x5632158e6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 --2- 172.21.15.105:0/2070533200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x5632158e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe13c884740 1 -- 172.21.15.105:0/2070533200 >> 172.21.15.105:0/2070533200 conn(0x5632158d6e60 msgr2=0x5632158d6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.187+0000 7fe112ffd640 0 client.18141 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.188+0000 7fe13c884740 1 -- 172.21.15.105:0/2070533200 shutdown_connections 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.188+0000 7fe13c884740 1 -- 172.21.15.105:0/2070533200 wait complete. 2024-03-26T17:45:37.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.188+0000 7fe13c884740 20 client.18141 trim_cache size 0 max 16384 2024-03-26T17:45:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.206+0000 7fe13c884740 1 Processor -- start 2024-03-26T17:45:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.206+0000 7fe13c884740 1 -- start start 2024-03-26T17:45:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe13c884740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158db8f0 0x5632158dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe13c884740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe13c884740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe13c884740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341a6b40 con 0x5632158e1450 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe13c884740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341a8680 con 0x5632158dc2e0 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe13c884740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341a7210 con 0x5632158db8f0 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158db8f0 0x5632158dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe133fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe132ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158db8f0 0x5632158dd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47992/0 (socket says 172.21.15.105:47992) 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe1337fe640 1 -- 172.21.15.105:0/360522220 learned_addr learned my addr 172.21.15.105:0/360522220 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe133fff640 1 -- 172.21.15.105:0/360522220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158db8f0 msgr2=0x5632158dd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe133fff640 1 --2- 172.21.15.105:0/360522220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158db8f0 0x5632158dd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe133fff640 1 -- 172.21.15.105:0/360522220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 msgr2=0x5632158dda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe133fff640 1 --2- 172.21.15.105:0/360522220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158dda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe133fff640 1 -- 172.21.15.105:0/360522220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632158d78b0 con 0x5632158e1450 2024-03-26T17:45:37.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe133fff640 1 --2- 172.21.15.105:0/360522220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158d57f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe12c0cd7b0 tx=0x7fe12c09a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997a6996016cb055 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe1197fa640 1 -- 172.21.15.105:0/360522220 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0b4050 con 0x5632158e1450 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe1197fa640 1 -- 172.21.15.105:0/360522220 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe12c0a45f0 con 0x5632158e1450 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.207+0000 7fe1197fa640 1 -- 172.21.15.105:0/360522220 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c096e60 con 0x5632158e1450 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 -- 172.21.15.105:0/360522220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 msgr2=0x5632158d57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 --2- 172.21.15.105:0/360522220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158d57f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe12c0cd7b0 tx=0x7fe12c09a580 comp rx=0 tx=0).stop 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 -- 172.21.15.105:0/360522220 shutdown_connections 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 --2- 172.21.15.105:0/360522220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x5632158d57f0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 --2- 172.21.15.105:0/360522220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158dc2e0 0x5632158dda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 --2- 172.21.15.105:0/360522220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158db8f0 0x5632158dd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 -- 172.21.15.105:0/360522220 >> 172.21.15.105:0/360522220 conn(0x5632158d6e60 msgr2=0x5632158e0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 -- 172.21.15.105:0/360522220 shutdown_connections 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.208+0000 7fe13c884740 1 -- 172.21.15.105:0/360522220 wait complete. 2024-03-26T17:45:37.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 1 -- start start 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 1 Processor -- start 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x563215907010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 0x563215907550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x563215908560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341a7fb0 con 0x5632158e1450 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341a78e0 con 0x5632158db8f0 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341a4b20 con 0x5632158dc2e0 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe13c884740 10 client.0 _ll_register_callbacks cb 0x7fe12c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe133fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x563215908560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x563215907010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x563215907010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56358/0 (socket says 172.21.15.105:56358) 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe133fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x563215908560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51636/0 (socket says 172.21.15.105:51636) 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe1337fe640 1 -- 172.21.15.105:0/909549617 learned_addr learned my addr 172.21.15.105:0/909549617 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:37.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.209+0000 7fe132ffd640 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 0x563215907550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe1337fe640 1 -- 172.21.15.105:0/909549617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 msgr2=0x563215907550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe1337fe640 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 0x563215907550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe1337fe640 1 -- 172.21.15.105:0/909549617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 msgr2=0x563215908560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe1337fe640 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x563215908560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe1337fe640 1 -- 172.21.15.105:0/909549617 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563215908b50 con 0x5632158db8f0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe133fff640 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x563215908560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe1337fe640 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x563215907010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe124009900 tx=0x7fe12402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc70bec42a48a8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe112ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe112ffd640 1 -- 172.21.15.105:0/909549617 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124005180 con 0x5632158db8f0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe13c884740 1 -- 172.21.15.105:0/909549617 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe12c0a45f0 con 0x5632158db8f0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe112ffd640 1 -- 172.21.15.105:0/909549617 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe124005320 con 0x5632158db8f0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe112ffd640 1 -- 172.21.15.105:0/909549617 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe12406d050 con 0x5632158db8f0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe13c884740 10 client.27977 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe13c884740 1 -- 172.21.15.105:0/909549617 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56321595c190 con 0x5632158db8f0 2024-03-26T17:45:37.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.210+0000 7fe112ffd640 1 -- 172.21.15.105:0/909549617 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe12c0a45f0 con 0x5632158db8f0 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe112ffd640 1 -- 172.21.15.105:0/909549617 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe124004a70 con 0x5632158db8f0 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe13c884740 10 client.27977 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe13c884740 10 client.27977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe13c884740 20 client.27977 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe13c884740 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56321595c6f0 0x56321595eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe13c884740 4 client.27977 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe13c884740 1 -- 172.21.15.105:0/909549617 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563215848210 con 0x56321595c6f0 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe132ffd640 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56321595c6f0 0x56321595eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe112ffd640 10 client.27977 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe132ffd640 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56321595c6f0 0x56321595eb10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeef553f09d3326 server_cookie=83d8e262988c2ac2 in_seq=0 out_seq=0 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe112ffd640 1 -- 172.21.15.105:0/909549617 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563215848210 con 0x56321595c6f0 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe112ffd640 10 client.27977 handle_command_reply: tid=0 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.211+0000 7fe13c884740 1 client.27977 shutdown 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 -- 172.21.15.105:0/909549617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 msgr2=0x563215907010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x563215907010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe124009900 tx=0x7fe12402bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 -- 172.21.15.105:0/909549617 shutdown_connections 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56321595c6f0 0x56321595eb10 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5632158e1450 0x563215908560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5632158dc2e0 0x563215907550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe112ffd640 0 client.27977 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 --2- 172.21.15.105:0/909549617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5632158db8f0 0x563215907010 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 -- 172.21.15.105:0/909549617 >> 172.21.15.105:0/909549617 conn(0x5632158d6e60 msgr2=0x5632158e5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 -- 172.21.15.105:0/909549617 shutdown_connections 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 1 -- 172.21.15.105:0/909549617 wait complete. 2024-03-26T17:45:37.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe13c884740 20 client.27977 trim_cache size 0 max 16384 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "9066724e-dd71-4e2f-9be9-ba686454d65c": { 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "9066724e-dd71-4e2f-9be9-ba686454d65c", 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:45:37.211 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:45:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 -- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe11c0a2f20 msgr2=0x7fe11c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe11c0a2f20 0x7fe11c0a53e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe124005ab0 tx=0x7fe124005fd0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 -- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe134102390 msgr2=0x7fe13419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:37.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe134102390 0x7fe13419f100 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00eb10 tx=0x7fe12c00c2d0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 -- 172.21.15.105:0/331208988 shutdown_connections 2024-03-26T17:45:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe11c0a2f20 0x7fe11c0a53e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe134102390 0x7fe13419f100 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1341019d0 0x7fe134198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.212+0000 7fe139c97640 1 --2- 172.21.15.105:0/331208988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe1341007e0 0x7fe134197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.213+0000 7fe139c97640 1 -- 172.21.15.105:0/331208988 >> 172.21.15.105:0/331208988 conn(0x7fe1340fbd70 msgr2=0x7fe134069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.213+0000 7fe139c97640 1 -- 172.21.15.105:0/331208988 shutdown_connections 2024-03-26T17:45:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:45:37.213+0000 7fe139c97640 1 -- 172.21.15.105:0/331208988 wait complete. 2024-03-26T17:45:37.216 INFO:tasks.cephfs.filesystem:scrub status for tag:9066724e-dd71-4e2f-9be9-ba686454d65c - {'path': '/', 'tag': '9066724e-dd71-4e2f-9be9-ba686454d65c', 'options': 'recursive,force'} 2024-03-26T17:45:38.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 224016 3.15 MB/sec execute 261 sec latency 16.105 ms 2024-03-26T17:45:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:37 smithi144 ceph-mon[36095]: pgmap v566: 97 pgs: 97 active+clean; 28 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:45:38.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:37 smithi105 ceph-mon[32280]: pgmap v566: 97 pgs: 97 active+clean; 28 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:45:38.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:37 smithi135 ceph-mon[35636]: pgmap v566: 97 pgs: 97 active+clean; 28 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:45:39.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 224556 3.15 MB/sec execute 262 sec latency 18.736 ms 2024-03-26T17:45:39.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:38 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:45:39.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:38 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:39.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:38 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:45:39.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:38 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:39.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:38 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:45:39.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:38 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:39.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:38 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:45:39.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:38 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:39.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:38 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:45:39.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:38 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:39.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:38 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:45:39.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:38 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:45:40.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 225172 3.15 MB/sec execute 263 sec latency 16.435 ms 2024-03-26T17:45:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:39 smithi144 ceph-mon[36095]: pgmap v567: 97 pgs: 97 active+clean; 28 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-03-26T17:45:40.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:39 smithi105 ceph-mon[32280]: pgmap v567: 97 pgs: 97 active+clean; 28 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-03-26T17:45:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:39 smithi135 ceph-mon[35636]: pgmap v567: 97 pgs: 97 active+clean; 28 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-03-26T17:45:41.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 225879 3.14 MB/sec execute 264 sec latency 18.580 ms 2024-03-26T17:45:42.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 226680 3.15 MB/sec execute 265 sec latency 18.329 ms 2024-03-26T17:45:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:41 smithi144 ceph-mon[36095]: pgmap v568: 97 pgs: 97 active+clean; 29 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-03-26T17:45:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:41 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:45:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:41 smithi105 ceph-mon[32280]: pgmap v568: 97 pgs: 97 active+clean; 29 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-03-26T17:45:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:41 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:45:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:41 smithi135 ceph-mon[35636]: pgmap v568: 97 pgs: 97 active+clean; 29 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-03-26T17:45:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:41 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:45:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 -- 172.21.15.105:0/3341543663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48102390 msgr2=0x7f1a48108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/3341543663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48102390 0x7f1a48108c20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ca0 tx=0x7f1a4002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 -- 172.21.15.105:0/3341543663 shutdown_connections 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/3341543663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a48102390 0x7f1a48108c20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/3341543663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a481019d0 0x7f1a48101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/3341543663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a481007e0 0x7f1a48100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 -- 172.21.15.105:0/3341543663 >> 172.21.15.105:0/3341543663 conn(0x7f1a480fbd50 msgr2=0x7f1a480fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 -- 172.21.15.105:0/3341543663 shutdown_connections 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.493+0000 7f1a4e91d640 1 -- 172.21.15.105:0/3341543663 wait complete. 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4e91d640 1 Processor -- start 2024-03-26T17:45:42.486 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4e91d640 1 -- start start 2024-03-26T17:45:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4e91d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a481007e0 0x7f1a48104770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4e91d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a481019d0 0x7f1a48102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4e91d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a48102390 0x7f1a481032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4e91d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4810c2f0 con 0x7f1a481019d0 2024-03-26T17:45:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4e91d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4810c170 con 0x7f1a481007e0 2024-03-26T17:45:42.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4e91d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4810c470 con 0x7f1a48102390 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a481019d0 0x7f1a48102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a481019d0 0x7f1a48102db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51646/0 (socket says 172.21.15.105:51646) 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a477fe640 1 -- 172.21.15.105:0/1762330901 learned_addr learned my addr 172.21.15.105:0/1762330901 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a47fff640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a481007e0 0x7f1a48104770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.494+0000 7f1a4ceba640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a48102390 0x7f1a481032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a477fe640 1 -- 172.21.15.105:0/1762330901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a48102390 msgr2=0x7f1a481032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a477fe640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a48102390 0x7f1a481032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a477fe640 1 -- 172.21.15.105:0/1762330901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a481007e0 msgr2=0x7f1a48104770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a477fe640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a481007e0 0x7f1a48104770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a477fe640 1 -- 172.21.15.105:0/1762330901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a48103b80 con 0x7f1a481019d0 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a47fff640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a481007e0 0x7f1a48104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a477fe640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a481019d0 0x7f1a48102db0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00e9f0 tx=0x7f1a3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a457fa640 1 -- 172.21.15.105:0/1762330901 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00cd80 con 0x7f1a481019d0 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a457fa640 1 -- 172.21.15.105:0/1762330901 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a3c0108e0 con 0x7f1a481019d0 2024-03-26T17:45:42.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a4e91d640 1 -- 172.21.15.105:0/1762330901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a481a1b20 con 0x7f1a481019d0 2024-03-26T17:45:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a457fa640 1 -- 172.21.15.105:0/1762330901 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c018920 con 0x7f1a481019d0 2024-03-26T17:45:42.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.495+0000 7f1a4e91d640 1 -- 172.21.15.105:0/1762330901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a481a2070 con 0x7f1a481019d0 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.498+0000 7f1a457fa640 1 -- 172.21.15.105:0/1762330901 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1a3c010a90 con 0x7f1a481019d0 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.498+0000 7f1a457fa640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a300a2fc0 0x7f1a300a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.498+0000 7f1a47fff640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a300a2fc0 0x7f1a300a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.498+0000 7f1a457fa640 1 -- 172.21.15.105:0/1762330901 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1a3c0cf760 con 0x7f1a481019d0 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.498+0000 7f1a515df740 1 Processor -- start 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a515df740 1 -- start start 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a300a2fc0 0x7f1a300a5480 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1a48101830 tx=0x7f1a38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a515df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b7f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a515df740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b84ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a515df740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a515df740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bef0ae40b0 con 0x55bef0b7ffc0 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a515df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bef0a8c960 con 0x55bef0b7f5d0 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a515df740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bef0ae0060 con 0x55bef0b85130 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b7f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b84ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b7f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56376/0 (socket says 172.21.15.105:56376) 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 -- 172.21.15.105:0/36654805 learned_addr learned my addr 172.21.15.105:0/36654805 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a4ceba640 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 -- 172.21.15.105:0/36654805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 msgr2=0x55bef0b875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 -- 172.21.15.105:0/36654805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 msgr2=0x55bef0b84ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b84ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 -- 172.21.15.105:0/36654805 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef0a9ec60 con 0x55bef0b7f5d0 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a477fe640 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b84ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.499+0000 7f1a47fff640 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b7f9f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009ba0 tx=0x7f1a38014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2029a29d9a403e51 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.500+0000 7f1a2e7fc640 1 -- 172.21.15.105:0/36654805 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800f480 con 0x55bef0b7f5d0 2024-03-26T17:45:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.500+0000 7f1a2e7fc640 1 -- 172.21.15.105:0/36654805 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a3800fad0 con 0x55bef0b7f5d0 2024-03-26T17:45:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.500+0000 7f1a2e7fc640 1 -- 172.21.15.105:0/36654805 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3801cb10 con 0x55bef0b7f5d0 2024-03-26T17:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- 172.21.15.105:0/36654805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 msgr2=0x55bef0b7f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b7f9f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009ba0 tx=0x7f1a38014000 comp rx=0 tx=0).stop 2024-03-26T17:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- 172.21.15.105:0/36654805 shutdown_connections 2024-03-26T17:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b84ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 --2- 172.21.15.105:0/36654805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b7f9f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- 172.21.15.105:0/36654805 >> 172.21.15.105:0/36654805 conn(0x55bef0b7ab40 msgr2=0x55bef0b7af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- 172.21.15.105:0/36654805 shutdown_connections 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- 172.21.15.105:0/36654805 wait complete. 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- start start 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 Processor -- start 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b89310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b89850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b90890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bef0b880e0 con 0x55bef0b7ffc0 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bef0a9fdb0 con 0x55bef0b7f5d0 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.501+0000 7f1a515df740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bef0b88260 con 0x55bef0b85130 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a515df740 10 client.0 _ll_register_callbacks cb 0x55bef0aa4110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a4ceba640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b90890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b89310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b89850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b89310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56392/0 (socket says 172.21.15.105:56392) 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a4ceba640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b90890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:48036/0 (socket says 172.21.15.105:48036) 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 -- 172.21.15.105:0/4265795446 learned_addr learned my addr 172.21.15.105:0/4265795446 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b89850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51662/0 (socket says 172.21.15.105:51662) 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 -- 172.21.15.105:0/4265795446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 msgr2=0x55bef0b90890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b90890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 -- 172.21.15.105:0/4265795446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 msgr2=0x55bef0b89850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b89850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 -- 172.21.15.105:0/4265795446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef07f8210 con 0x55bef0b7f5d0 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a477fe640 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b89850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a47fff640 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b89310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009b70 tx=0x7f1a38004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c3d94f3ad28e7c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a277fe640 1 -- 172.21.15.105:0/4265795446 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800fdc0 con 0x55bef0b7f5d0 2024-03-26T17:45:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a277fe640 1 -- 172.21.15.105:0/4265795446 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1a3802b040 con 0x55bef0b7f5d0 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.502+0000 7f1a515df740 1 -- 172.21.15.105:0/4265795446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3800fad0 con 0x55bef0b7f5d0 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a277fe640 1 -- 172.21.15.105:0/4265795446 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a3800f480 con 0x55bef0b7f5d0 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a515df740 10 client.27989 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a515df740 1 -- 172.21.15.105:0/4265795446 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bef08c4620 con 0x55bef0b7f5d0 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a277fe640 1 -- 172.21.15.105:0/4265795446 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a3800fad0 con 0x55bef0b7f5d0 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a277fe640 1 -- 172.21.15.105:0/4265795446 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1a3801c850 con 0x55bef0b7f5d0 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a515df740 10 client.27989 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a515df740 10 client.27989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a515df740 20 client.27989 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a515df740 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bef0b94e30 0x55bef0b97220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a515df740 4 client.27989 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a515df740 1 -- 172.21.15.105:0/4265795446 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bef0af15f0 con 0x55bef0b94e30 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.503+0000 7f1a477fe640 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bef0b94e30 0x55bef0b97220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.504+0000 7f1a477fe640 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bef0b94e30 0x55bef0b97220 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b36812301d6fd27 server_cookie=22c57e479cb5be22 in_seq=0 out_seq=0 2024-03-26T17:45:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.504+0000 7f1a277fe640 10 client.27989 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.504+0000 7f1a277fe640 1 -- 172.21.15.105:0/4265795446 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bef0af15f0 con 0x55bef0b94e30 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.504+0000 7f1a277fe640 10 client.27989 handle_command_reply: tid=0 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.504+0000 7f1a515df740 1 client.27989 shutdown 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 -- 172.21.15.105:0/4265795446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 msgr2=0x55bef0b89310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b89310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009b70 tx=0x7f1a38004420 comp rx=0 tx=0).stop 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 -- 172.21.15.105:0/4265795446 shutdown_connections 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bef0b94e30 0x55bef0b97220 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b85130 0x55bef0b90890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b7ffc0 0x55bef0b89850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 --2- 172.21.15.105:0/4265795446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b89310 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a277fe640 0 client.27989 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 -- 172.21.15.105:0/4265795446 >> 172.21.15.105:0/4265795446 conn(0x55bef0b7ab40 msgr2=0x55bef0b79df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 -- 172.21.15.105:0/4265795446 shutdown_connections 2024-03-26T17:45:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 1 -- 172.21.15.105:0/4265795446 wait complete. 2024-03-26T17:45:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.505+0000 7f1a515df740 20 client.27989 trim_cache size 0 max 16384 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a515df740 1 Processor -- start 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a515df740 1 -- start start 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a515df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b88f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a515df740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7ffc0 0x55bef0b90850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a515df740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b89460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a515df740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481aaf70 con 0x55bef0b85130 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a515df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481acab0 con 0x55bef0b7f5d0 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a515df740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481ab640 con 0x55bef0b7ffc0 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a4ceba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b89460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a4ceba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b89460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51668/0 (socket says 172.21.15.105:51668) 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b88f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a4ceba640 1 -- 172.21.15.105:0/2446679075 learned_addr learned my addr 172.21.15.105:0/2446679075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.521+0000 7f1a477fe640 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7ffc0 0x55bef0b90850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a4ceba640 1 -- 172.21.15.105:0/2446679075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7ffc0 msgr2=0x55bef0b90850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a4ceba640 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7ffc0 0x55bef0b90850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a4ceba640 1 -- 172.21.15.105:0/2446679075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 msgr2=0x55bef0b88f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a4ceba640 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b88f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a4ceba640 1 -- 172.21.15.105:0/2446679075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bef0b7b590 con 0x55bef0b85130 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a477fe640 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7ffc0 0x55bef0b90850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a47fff640 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b88f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a4ceba640 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b89460 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ca0 tx=0x7f1a400324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649497741c75d2f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a2dffb640 1 -- 172.21.15.105:0/2446679075 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40039040 con 0x55bef0b85130 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a2dffb640 1 -- 172.21.15.105:0/2446679075 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a40032e70 con 0x55bef0b85130 2024-03-26T17:45:42.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.522+0000 7f1a2dffb640 1 -- 172.21.15.105:0/2446679075 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40004460 con 0x55bef0b85130 2024-03-26T17:45:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 -- 172.21.15.105:0/2446679075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 msgr2=0x55bef0b89460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b89460 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ca0 tx=0x7f1a400324f0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 -- 172.21.15.105:0/2446679075 shutdown_connections 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b89460 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7ffc0 0x55bef0b90850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 --2- 172.21.15.105:0/2446679075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7f5d0 0x55bef0b88f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 -- 172.21.15.105:0/2446679075 >> 172.21.15.105:0/2446679075 conn(0x55bef0b7ab40 msgr2=0x55bef0b7dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 -- 172.21.15.105:0/2446679075 shutdown_connections 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 -- 172.21.15.105:0/2446679075 wait complete. 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 -- start start 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.523+0000 7f1a515df740 1 Processor -- start 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a515df740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7f5d0 0x55bef0b8dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a515df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7ffc0 0x55bef0b8e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a515df740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b8f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a515df740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481ac3e0 con 0x55bef0b85130 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a515df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481abd10 con 0x55bef0b7ffc0 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a515df740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481a8f50 con 0x55bef0b7f5d0 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a515df740 10 client.0 _ll_register_callbacks cb 0x7f1a3c097b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b8f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a477fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7ffc0 0x55bef0b8e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b8f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51670/0 (socket says 172.21.15.105:51670) 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 -- 172.21.15.105:0/1039346890 learned_addr learned my addr 172.21.15.105:0/1039346890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a47fff640 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7f5d0 0x55bef0b8dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 -- 172.21.15.105:0/1039346890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7f5d0 msgr2=0x55bef0b8dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7f5d0 0x55bef0b8dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 -- 172.21.15.105:0/1039346890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7ffc0 msgr2=0x55bef0b8e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7ffc0 0x55bef0b8e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 -- 172.21.15.105:0/1039346890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bef0b8fae0 con 0x55bef0b85130 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a4ceba640 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b8f4f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1a40037040 tx=0x7f1a40032c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1509c18792da9ee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a277fe640 1 -- 172.21.15.105:0/1039346890 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40059060 con 0x55bef0b85130 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a515df740 1 -- 172.21.15.105:0/1039346890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bef0aeb940 con 0x55bef0b85130 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.524+0000 7f1a277fe640 1 -- 172.21.15.105:0/1039346890 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1a40003970 con 0x55bef0b85130 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a277fe640 1 -- 172.21.15.105:0/1039346890 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a40039050 con 0x55bef0b85130 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a515df740 10 client.18165 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a515df740 1 -- 172.21.15.105:0/1039346890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bef0bfb180 con 0x55bef0b85130 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a277fe640 1 -- 172.21.15.105:0/1039346890 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55bef0aeb940 con 0x55bef0b85130 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a277fe640 1 -- 172.21.15.105:0/1039346890 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1a40034440 con 0x55bef0b85130 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a515df740 10 client.18165 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a515df740 10 client.18165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a515df740 20 client.18165 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a515df740 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bef0bfbb60 0x55bef0bfdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a515df740 4 client.18165 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a515df740 1 -- 172.21.15.105:0/1039346890 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bef0a69770 con 0x55bef0bfbb60 2024-03-26T17:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.525+0000 7f1a47fff640 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bef0bfbb60 0x55bef0bfdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.526+0000 7f1a277fe640 10 client.18165 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.526+0000 7f1a47fff640 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bef0bfbb60 0x55bef0bfdf80 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439da5d1b76d6f03 server_cookie=7a5a45768fa57d80 in_seq=0 out_seq=0 2024-03-26T17:45:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a277fe640 1 -- 172.21.15.105:0/1039346890 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bef0a69770 con 0x55bef0bfbb60 2024-03-26T17:45:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a277fe640 10 client.18165 handle_command_reply: tid=0 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 client.18165 shutdown 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 -- 172.21.15.105:0/1039346890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 msgr2=0x55bef0b8f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b8f4f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1a40037040 tx=0x7f1a40032c70 comp rx=0 tx=0).stop 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 -- 172.21.15.105:0/1039346890 shutdown_connections 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bef0bfbb60 0x55bef0bfdf80 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bef0b85130 0x55bef0b8f4f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a277fe640 0 client.18165 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bef0b7ffc0 0x55bef0b8e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 --2- 172.21.15.105:0/1039346890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bef0b7f5d0 0x55bef0b8dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 -- 172.21.15.105:0/1039346890 >> 172.21.15.105:0/1039346890 conn(0x55bef0b7ab40 msgr2=0x55bef0b88f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 -- 172.21.15.105:0/1039346890 shutdown_connections 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.570+0000 7f1a515df740 1 -- 172.21.15.105:0/1039346890 wait complete. 2024-03-26T17:45:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a515df740 20 client.18165 trim_cache size 0 max 16384 2024-03-26T17:45:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:45:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:45:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:45:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 -- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a300a2fc0 msgr2=0x7f1a300a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a300a2fc0 0x7f1a300a5480 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1a48101830 tx=0x7f1a38005f90 comp rx=0 tx=0).stop 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 -- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a481019d0 msgr2=0x7f1a48102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a481019d0 0x7f1a48102db0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00e9f0 tx=0x7f1a3c00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 -- 172.21.15.105:0/1762330901 shutdown_connections 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1a300a2fc0 0x7f1a300a5480 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a48102390 0x7f1a481032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a481019d0 0x7f1a48102db0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 --2- 172.21.15.105:0/1762330901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a481007e0 0x7f1a48104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 -- 172.21.15.105:0/1762330901 >> 172.21.15.105:0/1762330901 conn(0x7f1a480fbd50 msgr2=0x7f1a480fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 -- 172.21.15.105:0/1762330901 shutdown_connections 2024-03-26T17:45:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:42.571+0000 7f1a4e91d640 1 -- 172.21.15.105:0/1762330901 wait complete. 2024-03-26T17:45:43.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 227065 3.16 MB/sec execute 266 sec latency 41.115 ms 2024-03-26T17:45:44.026 INFO:tasks.workunit.client.0.smithi105.stdout: 1 227517 3.15 MB/sec execute 267 sec latency 15.040 ms 2024-03-26T17:45:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:43 smithi144 ceph-mon[36095]: pgmap v569: 97 pgs: 97 active+clean; 29 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:45:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:43 smithi105 ceph-mon[32280]: pgmap v569: 97 pgs: 97 active+clean; 29 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:45:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:43 smithi135 ceph-mon[35636]: pgmap v569: 97 pgs: 97 active+clean; 29 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:45:45.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 228000 3.14 MB/sec execute 268 sec latency 22.742 ms 2024-03-26T17:45:46.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 228667 3.15 MB/sec execute 269 sec latency 17.228 ms 2024-03-26T17:45:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:45 smithi144 ceph-mon[36095]: pgmap v570: 97 pgs: 97 active+clean; 30 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:45:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:45 smithi105 ceph-mon[32280]: pgmap v570: 97 pgs: 97 active+clean; 30 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:45:46.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:45 smithi135 ceph-mon[35636]: pgmap v570: 97 pgs: 97 active+clean; 30 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:45:47.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 229381 3.14 MB/sec execute 270 sec latency 16.328 ms 2024-03-26T17:45:48.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 230060 3.15 MB/sec execute 271 sec latency 15.801 ms 2024-03-26T17:45:48.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:47 smithi144 ceph-mon[36095]: pgmap v571: 97 pgs: 97 active+clean; 28 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:47 smithi105 ceph-mon[32280]: pgmap v571: 97 pgs: 97 active+clean; 28 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:47 smithi135 ceph-mon[35636]: pgmap v571: 97 pgs: 97 active+clean; 28 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-03-26T17:45:49.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 230619 3.16 MB/sec execute 272 sec latency 15.351 ms 2024-03-26T17:45:50.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 231065 3.15 MB/sec execute 273 sec latency 16.870 ms 2024-03-26T17:45:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:49 smithi144 ceph-mon[36095]: pgmap v572: 97 pgs: 97 active+clean; 26 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:45:50.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:49 smithi105 ceph-mon[32280]: pgmap v572: 97 pgs: 97 active+clean; 26 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:45:50.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:49 smithi135 ceph-mon[35636]: pgmap v572: 97 pgs: 97 active+clean; 26 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:45:51.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 231518 3.14 MB/sec execute 274 sec latency 102.711 ms 2024-03-26T17:45:52.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 232111 3.15 MB/sec execute 275 sec latency 32.172 ms 2024-03-26T17:45:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:51 smithi144 ceph-mon[36095]: pgmap v573: 97 pgs: 97 active+clean; 27 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:45:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:51 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:45:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:51 smithi105 ceph-mon[32280]: pgmap v573: 97 pgs: 97 active+clean; 27 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:45:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:51 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:45:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:51 smithi135 ceph-mon[35636]: pgmap v573: 97 pgs: 97 active+clean; 27 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:45:52.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:51 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:45:53.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 232781 3.14 MB/sec execute 276 sec latency 19.100 ms 2024-03-26T17:45:53.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:45:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:45:53.100 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:45:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:45:53.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:45:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:45:53.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:45:53.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:45:53.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:45:54.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 233398 3.15 MB/sec execute 277 sec latency 19.059 ms 2024-03-26T17:45:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:53 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-03-26T17:45:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:53 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-03-26T17:45:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:53 smithi144 ceph-mon[36095]: Unable to set osd_memory_target on smithi144 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:45:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:53 smithi144 ceph-mon[36095]: Unable to set osd_memory_target on smithi135 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:45:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:53 smithi144 ceph-mon[36095]: pgmap v574: 97 pgs: 97 active+clean; 29 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-03-26T17:45:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:53 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-03-26T17:45:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:53 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-03-26T17:45:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:53 smithi105 ceph-mon[32280]: Unable to set osd_memory_target on smithi144 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:45:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:53 smithi105 ceph-mon[32280]: Unable to set osd_memory_target on smithi135 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:45:54.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:53 smithi105 ceph-mon[32280]: pgmap v574: 97 pgs: 97 active+clean; 29 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-03-26T17:45:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:53 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-03-26T17:45:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:53 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-03-26T17:45:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:53 smithi135 ceph-mon[35636]: Unable to set osd_memory_target on smithi144 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:45:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:53 smithi135 ceph-mon[35636]: Unable to set osd_memory_target on smithi135 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:45:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:53 smithi135 ceph-mon[35636]: pgmap v574: 97 pgs: 97 active+clean; 29 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-03-26T17:45:55.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 233980 3.15 MB/sec execute 278 sec latency 15.083 ms 2024-03-26T17:45:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 -- 172.21.15.105:0/1938499800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c102360 msgr2=0x7f369c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 --2- 172.21.15.105:0/1938499800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c102360 0x7f369c104750 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f368c009900 tx=0x7f368c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:45:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 -- 172.21.15.105:0/1938499800 shutdown_connections 2024-03-26T17:45:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 --2- 172.21.15.105:0/1938499800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f369c1075c0 0x7f369c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.659 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 --2- 172.21.15.105:0/1938499800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c104c90 0x7f369c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 --2- 172.21.15.105:0/1938499800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c102360 0x7f369c104750 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 -- 172.21.15.105:0/1938499800 >> 172.21.15.105:0/1938499800 conn(0x7f369c0fbd50 msgr2=0x7f369c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 -- 172.21.15.105:0/1938499800 shutdown_connections 2024-03-26T17:45:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.667+0000 7f36a0f59640 1 -- 172.21.15.105:0/1938499800 wait complete. 2024-03-26T17:45:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f36a0f59640 1 Processor -- start 2024-03-26T17:45:55.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f36a0f59640 1 -- start start 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f36a0f59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f369c102360 0x7f369c101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f36a0f59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c104c90 0x7f369c100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f36a0f59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1075c0 0x7f369c100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f36a0f59640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f369c10d440 con 0x7f369c104c90 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f36a0f59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f369c10d2c0 con 0x7f369c102360 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f36a0f59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f369c10d5c0 con 0x7f369c1075c0 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f369c102360 0x7f369c101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f3699d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c104c90 0x7f369c100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f369c102360 0x7f369c101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35740/0 (socket says 172.21.15.105:35740) 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369a59c640 1 -- 172.21.15.105:0/950898959 learned_addr learned my addr 172.21.15.105:0/950898959 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:55.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369ad9d640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1075c0 0x7f369c100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369a59c640 1 -- 172.21.15.105:0/950898959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1075c0 msgr2=0x7f369c100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369a59c640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1075c0 0x7f369c100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369a59c640 1 -- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c104c90 msgr2=0x7f369c100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369a59c640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c104c90 0x7f369c100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.668+0000 7f369a59c640 1 -- 172.21.15.105:0/950898959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c1010e0 con 0x7f369c102360 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.669+0000 7f369a59c640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f369c102360 0x7f369c101cd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f368c006010 tx=0x7f368c005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.669+0000 7f368b7fe640 1 -- 172.21.15.105:0/950898959 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c043020 con 0x7f369c102360 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.669+0000 7f368b7fe640 1 -- 172.21.15.105:0/950898959 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f368c0043d0 con 0x7f369c102360 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.669+0000 7f36a0f59640 1 -- 172.21.15.105:0/950898959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c1a1bd0 con 0x7f369c102360 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.669+0000 7f368b7fe640 1 -- 172.21.15.105:0/950898959 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c003b80 con 0x7f369c102360 2024-03-26T17:45:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.669+0000 7f36a0f59640 1 -- 172.21.15.105:0/950898959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c1a2120 con 0x7f369c102360 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.671+0000 7f368b7fe640 1 -- 172.21.15.105:0/950898959 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f368c046020 con 0x7f369c102360 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f368b7fe640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36840a2fc0 0x7f36840a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f3699d9b640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36840a2fc0 0x7f36840a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f368b7fe640 1 -- 172.21.15.105:0/950898959 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f368c0f1600 con 0x7f369c102360 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f36a3c12740 1 Processor -- start 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f36a3c12740 1 -- start start 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f3699d9b640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36840a2fc0 0x7f36840a5480 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f369c101a40 tx=0x7f3690005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e501ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e5045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f36a3c12740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560d6e4610b0 con 0x560d6e4fc5d0 2024-03-26T17:45:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f36a3c12740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560d6e409960 con 0x560d6e502130 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.672+0000 7f36a3c12740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560d6e45d060 con 0x560d6e4fcfc0 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38076/0 (socket says 172.21.15.105:38076) 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 -- 172.21.15.105:0/1473181416 learned_addr learned my addr 172.21.15.105:0/1473181416 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f3699d9b640 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e501ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369ad9d640 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e5045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 -- 172.21.15.105:0/1473181416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 msgr2=0x560d6e501ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e501ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 -- 172.21.15.105:0/1473181416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 msgr2=0x560d6e5045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e5045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 -- 172.21.15.105:0/1473181416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6e41bc60 con 0x560d6e4fc5d0 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f3699d9b640 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e501ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369ad9d640 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e5045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f369a59c640 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fc9f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f368c0c61d0 tx=0x7f368c0bd710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e6d690548703ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f3688ff9640 1 -- 172.21.15.105:0/1473181416 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0c1e40 con 0x560d6e4fc5d0 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f3688ff9640 1 -- 172.21.15.105:0/1473181416 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f368c0c6700 con 0x560d6e4fc5d0 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.673+0000 7f3688ff9640 1 -- 172.21.15.105:0/1473181416 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0c6270 con 0x560d6e4fc5d0 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 -- 172.21.15.105:0/1473181416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 msgr2=0x560d6e4fc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fc9f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f368c0c61d0 tx=0x7f368c0bd710 comp rx=0 tx=0).stop 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 -- 172.21.15.105:0/1473181416 shutdown_connections 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e5045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e501ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 --2- 172.21.15.105:0/1473181416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fc9f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 -- 172.21.15.105:0/1473181416 >> 172.21.15.105:0/1473181416 conn(0x560d6e4f7b40 msgr2=0x560d6e4f7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 -- 172.21.15.105:0/1473181416 shutdown_connections 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.674+0000 7f36a3c12740 1 -- 172.21.15.105:0/1473181416 wait complete. 2024-03-26T17:45:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 1 -- start start 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 1 Processor -- start 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fc5d0 0x560d6e506180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e5066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e50d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560d6e504ea0 con 0x560d6e502130 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560d6e41cdb0 con 0x560d6e4fc5d0 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560d6e505020 con 0x560d6e4fcfc0 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e50d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f36a3c12740 10 client.0 _ll_register_callbacks cb 0x560d6e421110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e50d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38078/0 (socket says 172.21.15.105:38078) 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369ad9d640 1 -- 172.21.15.105:0/379582210 learned_addr learned my addr 172.21.15.105:0/379582210 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369a59c640 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fc5d0 0x560d6e506180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f3699d9b640 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e5066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369ad9d640 1 -- 172.21.15.105:0/379582210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 msgr2=0x560d6e5066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369ad9d640 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e5066c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369ad9d640 1 -- 172.21.15.105:0/379582210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fc5d0 msgr2=0x560d6e506180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369ad9d640 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fc5d0 0x560d6e506180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f369ad9d640 1 -- 172.21.15.105:0/379582210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6e175210 con 0x560d6e502130 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.675+0000 7f3699d9b640 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e5066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f369ad9d640 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e50d690 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3694011d10 tx=0x7f369400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffffd583ea5f4bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f3681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f3681ffb640 1 -- 172.21.15.105:0/379582210 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369401e060 con 0x560d6e502130 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f3681ffb640 1 -- 172.21.15.105:0/379582210 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f3694003970 con 0x560d6e502130 2024-03-26T17:45:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 1 -- 172.21.15.105:0/379582210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f368c0c6700 con 0x560d6e502130 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f3681ffb640 1 -- 172.21.15.105:0/379582210 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3694009e30 con 0x560d6e502130 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 10 client.18177 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 1 -- 172.21.15.105:0/379582210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d6e241620 con 0x560d6e502130 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f3681ffb640 1 -- 172.21.15.105:0/379582210 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f368c0c6700 con 0x560d6e502130 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f3681ffb640 1 -- 172.21.15.105:0/379582210 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3694019070 con 0x560d6e502130 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 10 client.18177 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 10 client.18177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 20 client.18177 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560d6e5120b0 0x560d6e5144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 4 client.18177 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f36a3c12740 1 -- 172.21.15.105:0/379582210 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d6e46e5f0 con 0x560d6e5120b0 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.676+0000 7f369a59c640 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560d6e5120b0 0x560d6e5144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.677+0000 7f369a59c640 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560d6e5120b0 0x560d6e5144a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dca334ad4b2f75c server_cookie=432475f210e6396c in_seq=0 out_seq=0 2024-03-26T17:45:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.677+0000 7f3681ffb640 10 client.18177 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.677+0000 7f3681ffb640 1 -- 172.21.15.105:0/379582210 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560d6e46e5f0 con 0x560d6e5120b0 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.677+0000 7f3681ffb640 10 client.18177 handle_command_reply: tid=0 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.677+0000 7f36a3c12740 1 client.18177 shutdown 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 -- 172.21.15.105:0/379582210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 msgr2=0x560d6e50d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e50d690 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3694011d10 tx=0x7f369400f790 comp rx=0 tx=0).stop 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 -- 172.21.15.105:0/379582210 shutdown_connections 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560d6e5120b0 0x560d6e5144a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f3681ffb640 0 client.18177 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e50d690 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e5066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 --2- 172.21.15.105:0/379582210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fc5d0 0x560d6e506180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 -- 172.21.15.105:0/379582210 >> 172.21.15.105:0/379582210 conn(0x560d6e4f7b40 msgr2=0x560d6e4f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 -- 172.21.15.105:0/379582210 shutdown_connections 2024-03-26T17:45:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 1 -- 172.21.15.105:0/379582210 wait complete. 2024-03-26T17:45:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.678+0000 7f36a3c12740 20 client.18177 trim_cache size 0 max 16384 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f36a3c12740 1 Processor -- start 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f36a3c12740 1 -- start start 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fcfc0 0x560d6e50d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e4fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f36a3c12740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1aaf80 con 0x560d6e502130 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f36a3c12740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1acac0 con 0x560d6e4fcfc0 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f36a3c12740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1ab650 con 0x560d6e4fc5d0 2024-03-26T17:45:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e4fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f3699d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fcfc0 0x560d6e50d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e4fdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38084/0 (socket says 172.21.15.105:38084) 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 -- 172.21.15.105:0/3763406963 learned_addr learned my addr 172.21.15.105:0/3763406963 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369a59c640 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 -- 172.21.15.105:0/3763406963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fc5d0 msgr2=0x560d6e4fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 -- 172.21.15.105:0/3763406963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fcfc0 msgr2=0x560d6e50d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fcfc0 0x560d6e50d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 -- 172.21.15.105:0/3763406963 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6e4f3450 con 0x560d6e502130 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f3699d9b640 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fcfc0 0x560d6e50d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.696+0000 7f369ad9d640 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e4fdfa0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3694002230 tx=0x7f3694009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cfe7f605ceb8d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f3683fff640 1 -- 172.21.15.105:0/3763406963 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694031070 con 0x560d6e502130 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f3683fff640 1 -- 172.21.15.105:0/3763406963 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3694004df0 con 0x560d6e502130 2024-03-26T17:45:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f3683fff640 1 -- 172.21.15.105:0/3763406963 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694028040 con 0x560d6e502130 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 -- 172.21.15.105:0/3763406963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 msgr2=0x560d6e4fdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e4fdfa0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3694002230 tx=0x7f3694009c00 comp rx=0 tx=0).stop 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 -- 172.21.15.105:0/3763406963 shutdown_connections 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e502130 0x560d6e4fdfa0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e4fcfc0 0x560d6e50d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3763406963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fc5d0 0x560d6e4fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 -- 172.21.15.105:0/3763406963 >> 172.21.15.105:0/3763406963 conn(0x560d6e4f7b40 msgr2=0x560d6e501460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 -- 172.21.15.105:0/3763406963 shutdown_connections 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.697+0000 7f36a3c12740 1 -- 172.21.15.105:0/3763406963 wait complete. 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 1 -- start start 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 1 Processor -- start 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e50ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e50b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e50c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1ac3f0 con 0x560d6e4fc5d0 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1abd20 con 0x560d6e502130 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1a8f60 con 0x560d6e4fcfc0 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f36a3c12740 10 client.0 _ll_register_callbacks cb 0x7f3694009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e50ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e50ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38086/0 (socket says 172.21.15.105:38086) 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 -- 172.21.15.105:0/3354005514 learned_addr learned my addr 172.21.15.105:0/3354005514 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369ad9d640 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e50c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f3699d9b640 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e50b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 -- 172.21.15.105:0/3354005514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 msgr2=0x560d6e50b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e50b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 -- 172.21.15.105:0/3354005514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 msgr2=0x560d6e50c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:45:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e50c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 -- 172.21.15.105:0/3354005514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6e514810 con 0x560d6e4fc5d0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f369a59c640 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e50ac90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f368c0ba150 tx=0x7f368c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabde527472a92b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.698+0000 7f3681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f3681ffb640 1 -- 172.21.15.105:0/3354005514 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0dc030 con 0x560d6e4fc5d0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f3681ffb640 1 -- 172.21.15.105:0/3354005514 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f368c003d20 con 0x560d6e4fc5d0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 1 -- 172.21.15.105:0/3354005514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3694004df0 con 0x560d6e4fc5d0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f3681ffb640 1 -- 172.21.15.105:0/3354005514 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f368c0c6700 con 0x560d6e4fc5d0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 10 client.18183 fetch_fsmap learned FSMap version 30 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 1 -- 172.21.15.105:0/3354005514 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d6e50d8a0 con 0x560d6e4fc5d0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f3681ffb640 1 -- 172.21.15.105:0/3354005514 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3694004df0 con 0x560d6e4fc5d0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f3681ffb640 1 -- 172.21.15.105:0/3354005514 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f368c0c6270 con 0x560d6e4fc5d0 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 10 client.18183 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 10 client.18183 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:45:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 20 client.18183 populate_metadata read hostname 'smithi105' 2024-03-26T17:45:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560d6e5120b0 0x560d6e57a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:45:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 4 client.18183 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:45:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.699+0000 7f36a3c12740 1 -- 172.21.15.105:0/3354005514 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d6e468940 con 0x560d6e5120b0 2024-03-26T17:45:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.712+0000 7f3699d9b640 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560d6e5120b0 0x560d6e57a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:45:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.713+0000 7f3681ffb640 10 client.18183 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.713+0000 7f3699d9b640 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560d6e5120b0 0x560d6e57a880 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b75b98bae02d91 server_cookie=ad4f9d1156661a86 in_seq=0 out_seq=0 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.757+0000 7f3681ffb640 1 -- 172.21.15.105:0/3354005514 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d6e468940 con 0x560d6e5120b0 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.757+0000 7f3681ffb640 10 client.18183 handle_command_reply: tid=0 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.757+0000 7f36a3c12740 1 client.18183 shutdown 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.757+0000 7f36a3c12740 1 -- 172.21.15.105:0/3354005514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 msgr2=0x560d6e50ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.757+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e50ac90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f368c0ba150 tx=0x7f368c005be0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 1 -- 172.21.15.105:0/3354005514 shutdown_connections 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560d6e5120b0 0x560d6e57a880 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560d6e502130 0x560d6e50c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560d6e4fcfc0 0x560d6e50b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f3681ffb640 0 client.18183 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 1 --2- 172.21.15.105:0/3354005514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560d6e4fc5d0 0x560d6e50ac90 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 1 -- 172.21.15.105:0/3354005514 >> 172.21.15.105:0/3354005514 conn(0x560d6e4f7b40 msgr2=0x560d6e4fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 1 -- 172.21.15.105:0/3354005514 shutdown_connections 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 1 -- 172.21.15.105:0/3354005514 wait complete. 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a3c12740 20 client.18183 trim_cache size 0 max 16384 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:45:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:45:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 -- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36840a2fc0 msgr2=0x7f36840a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36840a2fc0 0x7f36840a5480 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f369c101a40 tx=0x7f3690005f90 comp rx=0 tx=0).stop 2024-03-26T17:45:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 -- 172.21.15.105:0/950898959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f369c102360 msgr2=0x7f369c101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f369c102360 0x7f369c101cd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f368c006010 tx=0x7f368c005e40 comp rx=0 tx=0).stop 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 -- 172.21.15.105:0/950898959 shutdown_connections 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f36840a2fc0 0x7f36840a5480 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f369c1075c0 0x7f369c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f369c104c90 0x7f369c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 --2- 172.21.15.105:0/950898959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f369c102360 0x7f369c101cd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.758+0000 7f36a0f59640 1 -- 172.21.15.105:0/950898959 >> 172.21.15.105:0/950898959 conn(0x7f369c0fbd50 msgr2=0x7f369c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.759+0000 7f36a0f59640 1 -- 172.21.15.105:0/950898959 shutdown_connections 2024-03-26T17:45:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:45:55.759+0000 7f36a0f59640 1 -- 172.21.15.105:0/950898959 wait complete. 2024-03-26T17:45:56.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 234526 3.15 MB/sec execute 279 sec latency 23.888 ms 2024-03-26T17:45:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:55 smithi144 ceph-mon[36095]: pgmap v575: 97 pgs: 97 active+clean; 28 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-03-26T17:45:56.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:55 smithi105 ceph-mon[32280]: pgmap v575: 97 pgs: 97 active+clean; 28 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-03-26T17:45:56.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:55 smithi135 ceph-mon[35636]: pgmap v575: 97 pgs: 97 active+clean; 28 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-03-26T17:45:57.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 234972 3.14 MB/sec execute 280 sec latency 30.593 ms 2024-03-26T17:45:58.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 235599 3.15 MB/sec execute 281 sec latency 15.481 ms 2024-03-26T17:45:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:57 smithi144 ceph-mon[36095]: pgmap v576: 97 pgs: 97 active+clean; 25 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:45:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:57 smithi105 ceph-mon[32280]: pgmap v576: 97 pgs: 97 active+clean; 25 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:45:58.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:57 smithi135 ceph-mon[35636]: pgmap v576: 97 pgs: 97 active+clean; 25 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:45:59.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 236246 3.14 MB/sec execute 282 sec latency 18.238 ms 2024-03-26T17:46:00.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 236907 3.15 MB/sec execute 283 sec latency 16.728 ms 2024-03-26T17:46:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:45:59 smithi144 ceph-mon[36095]: pgmap v577: 97 pgs: 97 active+clean; 26 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:46:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:45:59 smithi105 ceph-mon[32280]: pgmap v577: 97 pgs: 97 active+clean; 26 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:46:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:45:59 smithi135 ceph-mon[35636]: pgmap v577: 97 pgs: 97 active+clean; 26 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:46:01.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 237476 3.15 MB/sec execute 284 sec latency 45.355 ms 2024-03-26T17:46:02.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 238040 3.15 MB/sec execute 285 sec latency 16.579 ms 2024-03-26T17:46:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:01 smithi144 ceph-mon[36095]: pgmap v578: 97 pgs: 97 active+clean; 26 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-03-26T17:46:02.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:01 smithi105 ceph-mon[32280]: pgmap v578: 97 pgs: 97 active+clean; 26 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-03-26T17:46:02.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:01 smithi135 ceph-mon[35636]: pgmap v578: 97 pgs: 97 active+clean; 26 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-03-26T17:46:03.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 238487 3.14 MB/sec execute 286 sec latency 23.365 ms 2024-03-26T17:46:04.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 239120 3.15 MB/sec execute 287 sec latency 16.341 ms 2024-03-26T17:46:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:03 smithi144 ceph-mon[36095]: pgmap v579: 97 pgs: 97 active+clean; 26 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-03-26T17:46:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:03 smithi105 ceph-mon[32280]: pgmap v579: 97 pgs: 97 active+clean; 26 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-03-26T17:46:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:03 smithi135 ceph-mon[35636]: pgmap v579: 97 pgs: 97 active+clean; 26 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-03-26T17:46:05.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 239792 3.14 MB/sec execute 288 sec latency 15.285 ms 2024-03-26T17:46:06.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 240454 3.15 MB/sec execute 289 sec latency 20.078 ms 2024-03-26T17:46:06.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:05 smithi144 ceph-mon[36095]: pgmap v580: 97 pgs: 97 active+clean; 27 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:46:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:05 smithi105 ceph-mon[32280]: pgmap v580: 97 pgs: 97 active+clean; 27 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:46:06.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:05 smithi135 ceph-mon[35636]: pgmap v580: 97 pgs: 97 active+clean; 27 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-03-26T17:46:07.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 241057 3.15 MB/sec execute 290 sec latency 14.876 ms 2024-03-26T17:46:07.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:46:07.217 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:46:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 -- 172.21.15.105:0/4260593272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c1019b0 msgr2=0x7fad4c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 --2- 172.21.15.105:0/4260593272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c1019b0 0x7fad4c101e30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fad40009900 tx=0x7fad4002bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:07.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 -- 172.21.15.105:0/4260593272 shutdown_connections 2024-03-26T17:46:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 --2- 172.21.15.105:0/4260593272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad4c102370 0x7fad4c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 --2- 172.21.15.105:0/4260593272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c1019b0 0x7fad4c101e30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 --2- 172.21.15.105:0/4260593272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad4c1007c0 0x7fad4c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 -- 172.21.15.105:0/4260593272 >> 172.21.15.105:0/4260593272 conn(0x7fad4c0fbd50 msgr2=0x7fad4c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 -- 172.21.15.105:0/4260593272 shutdown_connections 2024-03-26T17:46:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.346+0000 7fad520ea640 1 -- 172.21.15.105:0/4260593272 wait complete. 2024-03-26T17:46:07.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.347+0000 7fad520ea640 1 Processor -- start 2024-03-26T17:46:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.347+0000 7fad520ea640 1 -- start start 2024-03-26T17:46:07.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.347+0000 7fad520ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad4c1007c0 0x7fad4c0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.347+0000 7fad520ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad4c1019b0 0x7fad4c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.347+0000 7fad520ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c102370 0x7fad4c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.347+0000 7fad520ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10c2d0 con 0x7fad4c102370 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.347+0000 7fad520ea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10c150 con 0x7fad4c1007c0 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.347+0000 7fad520ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c10c450 con 0x7fad4c1019b0 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad4c1019b0 0x7fad4c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad4c1007c0 0x7fad4c0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad4c1019b0 0x7fad4c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40668/0 (socket says 172.21.15.105:40668) 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad4c1007c0 0x7fad4c0695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53044/0 (socket says 172.21.15.105:53044) 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 -- 172.21.15.105:0/1421662449 learned_addr learned my addr 172.21.15.105:0/1421662449 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4bfff640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c102370 0x7fad4c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 -- 172.21.15.105:0/1421662449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad4c1007c0 msgr2=0x7fad4c0695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad4c1007c0 0x7fad4c0695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 -- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c102370 msgr2=0x7fad4c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c102370 0x7fad4c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 -- 172.21.15.105:0/1421662449 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c19f790 con 0x7fad4c1019b0 2024-03-26T17:46:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4bfff640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c102370 0x7fad4c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad4affd640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad4c1019b0 0x7fad4c19c860 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad40006010 tx=0x7fad40004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad48ff9640 1 -- 172.21.15.105:0/1421662449 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40043020 con 0x7fad4c1019b0 2024-03-26T17:46:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad520ea640 1 -- 172.21.15.105:0/1421662449 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c196630 con 0x7fad4c1019b0 2024-03-26T17:46:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad48ff9640 1 -- 172.21.15.105:0/1421662449 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad40004440 con 0x7fad4c1019b0 2024-03-26T17:46:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad48ff9640 1 -- 172.21.15.105:0/1421662449 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40003bf0 con 0x7fad4c1019b0 2024-03-26T17:46:07.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.348+0000 7fad520ea640 1 -- 172.21.15.105:0/1421662449 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c196bd0 con 0x7fad4c1019b0 2024-03-26T17:46:07.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.351+0000 7fad48ff9640 1 -- 172.21.15.105:0/1421662449 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fad40046020 con 0x7fad4c1019b0 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.351+0000 7fad54db3740 1 Processor -- start 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.351+0000 7fad54db3740 1 -- start start 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad54db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c7f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad54db3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f32e0 0x55b34c7f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad54db3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c7fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad54db3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b34c757980 con 0x55b34c7f32e0 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad54db3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b34c6ffbe0 con 0x55b34c7f8450 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad54db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b34c753930 con 0x55b34c7f28f0 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f32e0 0x55b34c7f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c7f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c7fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c7f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40680/0 (socket says 172.21.15.105:40680) 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4bfff640 1 -- 172.21.15.105:0/1213658922 learned_addr learned my addr 172.21.15.105:0/1213658922 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad48ff9640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad340a2f20 0x7fad340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4bfff640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad340a2f20 0x7fad340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4affd640 1 -- 172.21.15.105:0/1213658922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 msgr2=0x55b34c7f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4affd640 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c7f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4affd640 1 -- 172.21.15.105:0/1213658922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f32e0 msgr2=0x55b34c7f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4affd640 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f32e0 0x55b34c7f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4affd640 1 -- 172.21.15.105:0/1213658922 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34c5357d0 con 0x55b34c7f8450 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4bfff640 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c7f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4b7fe640 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f32e0 0x55b34c7f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad48ff9640 1 -- 172.21.15.105:0/1421662449 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad400f1730 con 0x7fad4c1019b0 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.352+0000 7fad4affd640 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c7fa8e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fad400ef240 tx=0x7fad400b3300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=819f22a966bed05a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad4bfff640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad340a2f20 0x7fad340a53e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55b34c7f3b40 tx=0x7fad44004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad31ffb640 1 -- 172.21.15.105:0/1213658922 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40005560 con 0x55b34c7f8450 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad31ffb640 1 -- 172.21.15.105:0/1213658922 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad400be520 con 0x55b34c7f8450 2024-03-26T17:46:07.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad31ffb640 1 -- 172.21.15.105:0/1213658922 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400c1f70 con 0x55b34c7f8450 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad54db3740 1 -- 172.21.15.105:0/1213658922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 msgr2=0x55b34c7fa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad54db3740 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c7fa8e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fad400ef240 tx=0x7fad400b3300 comp rx=0 tx=0).stop 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad54db3740 1 -- 172.21.15.105:0/1213658922 shutdown_connections 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad54db3740 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c7fa8e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad54db3740 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f32e0 0x55b34c7f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad54db3740 1 --2- 172.21.15.105:0/1213658922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c7f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.353+0000 7fad54db3740 1 -- 172.21.15.105:0/1213658922 >> 172.21.15.105:0/1213658922 conn(0x55b34c7ede60 msgr2=0x55b34c7ee260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 -- 172.21.15.105:0/1213658922 shutdown_connections 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 -- 172.21.15.105:0/1213658922 wait complete. 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 -- start start 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 Processor -- start 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c7fc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 0x55b34c7fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c803db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b34c7fb1c0 con 0x55b34c7f28f0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad54db3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b34c713160 con 0x55b34c7f8450 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.354+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c7fc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad54db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b34c7fb340 con 0x55b34c7f32e0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c7fc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37852/0 (socket says 172.21.15.105:37852) 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2607500832 learned_addr learned my addr 172.21.15.105:0/2607500832 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4bfff640 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c803db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4affd640 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 0x55b34c7fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad54db3740 10 client.0 _ll_register_callbacks cb 0x55b34c74f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2607500832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 msgr2=0x55b34c7fcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4b7fe640 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 0x55b34c7fcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2607500832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 msgr2=0x55b34c803db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4b7fe640 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c803db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2607500832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34c46b3f0 con 0x55b34c7f28f0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad4b7fe640 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c7fc830 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009920 tx=0x7fad3c009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8ded54357ba894 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad2affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:07.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad2affd640 1 -- 172.21.15.105:0/2607500832 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00bc90 con 0x55b34c7f28f0 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad2affd640 1 -- 172.21.15.105:0/2607500832 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad3c004030 con 0x55b34c7f28f0 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.355+0000 7fad54db3740 1 -- 172.21.15.105:0/2607500832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad400be520 con 0x55b34c7f28f0 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad2affd640 1 -- 172.21.15.105:0/2607500832 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad3c02a050 con 0x55b34c7f28f0 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad54db3740 10 client.18195 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad54db3740 1 -- 172.21.15.105:0/2607500832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b34c42b0f0 con 0x55b34c7f28f0 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad2affd640 1 -- 172.21.15.105:0/2607500832 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad400be520 con 0x55b34c7f28f0 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad2affd640 1 -- 172.21.15.105:0/2607500832 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fad3c01b440 con 0x55b34c7f28f0 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad54db3740 10 client.18195 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad54db3740 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad54db3740 20 client.18195 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad54db3740 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b34c808700 0x55b34c80abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad54db3740 4 client.18195 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad54db3740 1 -- 172.21.15.105:0/2607500832 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b34c764ec0 con 0x55b34c808700 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.356+0000 7fad4affd640 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b34c808700 0x55b34c80abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.357+0000 7fad2affd640 10 client.18195 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.357+0000 7fad4affd640 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b34c808700 0x55b34c80abc0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5246bc5a7ad61ca7 server_cookie=c35980dfc4d4f7f9 in_seq=0 out_seq=0 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.358+0000 7fad2affd640 1 -- 172.21.15.105:0/2607500832 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b34c764ec0 con 0x55b34c808700 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.358+0000 7fad2affd640 10 client.18195 handle_command_reply: tid=0 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.358+0000 7fad54db3740 1 client.18195 shutdown 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 -- 172.21.15.105:0/2607500832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 msgr2=0x55b34c7fc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c7fc830 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009920 tx=0x7fad3c009de0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 -- 172.21.15.105:0/2607500832 shutdown_connections 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b34c808700 0x55b34c80abc0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f8450 0x55b34c803db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 0x55b34c7fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 --2- 172.21.15.105:0/2607500832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c7fc830 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 -- 172.21.15.105:0/2607500832 >> 172.21.15.105:0/2607500832 conn(0x55b34c7ede60 msgr2=0x55b34c7ed080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad2affd640 0 client.18195 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:07.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 -- 172.21.15.105:0/2607500832 shutdown_connections 2024-03-26T17:46:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 1 -- 172.21.15.105:0/2607500832 wait complete. 2024-03-26T17:46:07.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.359+0000 7fad54db3740 20 client.18195 trim_cache size 0 max 16384 2024-03-26T17:46:07.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad54db3740 1 Processor -- start 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad54db3740 1 -- start start 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad54db3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7ede60 0x55b34c7ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad54db3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c803d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad54db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 0x55b34c7fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad54db3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c1a6bc0 con 0x55b34c7f28f0 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad54db3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c1a8700 con 0x55b34c7ede60 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad54db3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c1a7290 con 0x55b34c7f32e0 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c803d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7ede60 0x55b34c7ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c803d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37868/0 (socket says 172.21.15.105:37868) 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4affd640 1 -- 172.21.15.105:0/4242394033 learned_addr learned my addr 172.21.15.105:0/4242394033 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4bfff640 1 --2- 172.21.15.105:0/4242394033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 0x55b34c7fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4affd640 1 -- 172.21.15.105:0/4242394033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 msgr2=0x55b34c7fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4affd640 1 --2- 172.21.15.105:0/4242394033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 0x55b34c7fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4affd640 1 -- 172.21.15.105:0/4242394033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7ede60 msgr2=0x55b34c7ee240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4affd640 1 --2- 172.21.15.105:0/4242394033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7ede60 0x55b34c7ee240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.378+0000 7fad4affd640 1 -- 172.21.15.105:0/4242394033 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34c7eebd0 con 0x55b34c7f28f0 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.379+0000 7fad4affd640 1 --2- 172.21.15.105:0/4242394033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c803d70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fad40003710 tx=0x7fad400ba1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd8ce65124c074c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.379+0000 7fad317fa640 1 -- 172.21.15.105:0/4242394033 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40005560 con 0x55b34c7f28f0 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.379+0000 7fad317fa640 1 -- 172.21.15.105:0/4242394033 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad400bde20 con 0x55b34c7f28f0 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.379+0000 7fad317fa640 1 -- 172.21.15.105:0/4242394033 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad400c1f70 con 0x55b34c7f28f0 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 -- 172.21.15.105:0/4242394033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 msgr2=0x55b34c803d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 --2- 172.21.15.105:0/4242394033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c803d70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fad40003710 tx=0x7fad400ba1f0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 -- 172.21.15.105:0/4242394033 shutdown_connections 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 --2- 172.21.15.105:0/4242394033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f32e0 0x55b34c7fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 --2- 172.21.15.105:0/4242394033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7f28f0 0x55b34c803d70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 --2- 172.21.15.105:0/4242394033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7ede60 0x55b34c7ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 -- 172.21.15.105:0/4242394033 >> 172.21.15.105:0/4242394033 conn(0x55b34c7f7f20 msgr2=0x55b34c7f1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 -- 172.21.15.105:0/4242394033 shutdown_connections 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 -- 172.21.15.105:0/4242394033 wait complete. 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 -- start start 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 Processor -- start 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.380+0000 7fad54db3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7ede60 0x55b34c800c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad54db3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c8011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7ede60 0x55b34c800c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7ede60 0x55b34c800c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37870/0 (socket says 172.21.15.105:37870) 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4b7fe640 1 -- 172.21.15.105:0/2396565047 learned_addr learned my addr 172.21.15.105:0/2396565047 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4affd640 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c8011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad54db3740 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f32e0 0x55b34c86cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c1a8030 con 0x55b34c7ede60 2024-03-26T17:46:07.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c1a7960 con 0x55b34c7f32e0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c1a4ba0 con 0x55b34c7f28f0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad54db3740 10 client.0 _ll_register_callbacks cb 0x7fad3c010ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4bfff640 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f32e0 0x55b34c86cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4affd640 1 -- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f32e0 msgr2=0x55b34c86cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4affd640 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f32e0 0x55b34c86cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4affd640 1 -- 172.21.15.105:0/2396565047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7ede60 msgr2=0x55b34c800c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4affd640 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7ede60 0x55b34c800c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4affd640 1 -- 172.21.15.105:0/2396565047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34c86cff0 con 0x55b34c7f28f0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4b7fe640 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7ede60 0x55b34c800c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad4affd640 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c8011b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad40005ab0 tx=0x7fad40005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a62c8d1fd51aa13f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.381+0000 7fad2affd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad2affd640 1 -- 172.21.15.105:0/2396565047 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40005560 con 0x55b34c7f28f0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad2affd640 1 -- 172.21.15.105:0/2396565047 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fad400bc120 con 0x55b34c7f28f0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b34c75f210 con 0x55b34c7f28f0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad2affd640 1 -- 172.21.15.105:0/2396565047 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fad400c1f70 con 0x55b34c7f28f0 2024-03-26T17:46:07.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 10 client.28033 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b34c86e2b0 con 0x55b34c7f28f0 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad2affd640 1 -- 172.21.15.105:0/2396565047 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b34c75f210 con 0x55b34c7f28f0 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad2affd640 1 -- 172.21.15.105:0/2396565047 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fad400c63a0 con 0x55b34c7f28f0 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 10 client.28033 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 10 client.28033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 20 client.28033 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b34c7f8450 0x55b34c870890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 4 client.28033 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b34c6dc540 con 0x55b34c7f8450 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.382+0000 7fad4b7fe640 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b34c7f8450 0x55b34c870890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad2affd640 10 client.28033 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad4b7fe640 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b34c7f8450 0x55b34c870890 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52b40e900bac560 server_cookie=606aba1bfc178997 in_seq=0 out_seq=0 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad2affd640 1 -- 172.21.15.105:0/2396565047 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b34c6dc540 con 0x55b34c7f8450 2024-03-26T17:46:07.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad2affd640 10 client.28033 handle_command_reply: tid=0 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad54db3740 1 client.28033 shutdown 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 msgr2=0x55b34c8011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad54db3740 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c8011b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fad40005ab0 tx=0x7fad40005bd0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 shutdown_connections 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad54db3740 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b34c7f8450 0x55b34c870890 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad54db3740 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b34c7f32e0 0x55b34c86cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad2affd640 0 client.28033 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.383+0000 7fad54db3740 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b34c7f28f0 0x55b34c8011b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad54db3740 1 --2- 172.21.15.105:0/2396565047 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b34c7ede60 0x55b34c800c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 >> 172.21.15.105:0/2396565047 conn(0x55b34c7f7f20 msgr2=0x55b34c7f1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 shutdown_connections 2024-03-26T17:46:07.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad54db3740 1 -- 172.21.15.105:0/2396565047 wait complete. 2024-03-26T17:46:07.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad54db3740 20 client.28033 trim_cache size 0 max 16384 2024-03-26T17:46:07.384 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:46:07.385 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:46:07.385 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:46:07.385 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:46:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad520ea640 1 -- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad340a2f20 msgr2=0x7fad340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad520ea640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad340a2f20 0x7fad340a53e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55b34c7f3b40 tx=0x7fad44004ab0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad520ea640 1 -- 172.21.15.105:0/1421662449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad4c1019b0 msgr2=0x7fad4c19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.384+0000 7fad520ea640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad4c1019b0 0x7fad4c19c860 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad40006010 tx=0x7fad40004040 comp rx=0 tx=0).stop 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.385+0000 7fad520ea640 1 -- 172.21.15.105:0/1421662449 shutdown_connections 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.385+0000 7fad520ea640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fad340a2f20 0x7fad340a53e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.385+0000 7fad520ea640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad4c102370 0x7fad4c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.385+0000 7fad520ea640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad4c1019b0 0x7fad4c19c860 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.385+0000 7fad520ea640 1 --2- 172.21.15.105:0/1421662449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad4c1007c0 0x7fad4c0695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.385+0000 7fad520ea640 1 -- 172.21.15.105:0/1421662449 >> 172.21.15.105:0/1421662449 conn(0x7fad4c0fbd50 msgr2=0x7fad4c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.385+0000 7fad520ea640 1 -- 172.21.15.105:0/1421662449 shutdown_connections 2024-03-26T17:46:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.385+0000 7fad520ea640 1 -- 172.21.15.105:0/1421662449 wait complete. 2024-03-26T17:46:07.389 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:46:07.389 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:46:07.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.514+0000 7effec24b640 1 -- 172.21.15.105:0/890160191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41019d0 msgr2=0x7effe4101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.514+0000 7effec24b640 1 --2- 172.21.15.105:0/890160191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41019d0 0x7effe4101e50 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7effd4009900 tx=0x7effd402bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 -- 172.21.15.105:0/890160191 shutdown_connections 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 --2- 172.21.15.105:0/890160191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe4102390 0x7effe4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 --2- 172.21.15.105:0/890160191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41019d0 0x7effe4101e50 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 --2- 172.21.15.105:0/890160191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe41007e0 0x7effe4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 -- 172.21.15.105:0/890160191 >> 172.21.15.105:0/890160191 conn(0x7effe40fbd50 msgr2=0x7effe40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 -- 172.21.15.105:0/890160191 shutdown_connections 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 -- 172.21.15.105:0/890160191 wait complete. 2024-03-26T17:46:07.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 Processor -- start 2024-03-26T17:46:07.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.515+0000 7effec24b640 1 -- start start 2024-03-26T17:46:07.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effec24b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41007e0 0x7effe410e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effec24b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe41019d0 0x7effe410cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effec24b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe4102390 0x7effe410d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effec24b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7effe410c310 con 0x7effe41007e0 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effec24b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effe410c190 con 0x7effe41019d0 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effec24b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7effe410c490 con 0x7effe4102390 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41007e0 0x7effe410e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41007e0 0x7effe410e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37882/0 (socket says 172.21.15.105:37882) 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 -- 172.21.15.105:0/1270596247 learned_addr learned my addr 172.21.15.105:0/1270596247 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effea7e8640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe4102390 0x7effe410d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe97e6640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe41019d0 0x7effe410cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 -- 172.21.15.105:0/1270596247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe4102390 msgr2=0x7effe410d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe4102390 0x7effe410d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 -- 172.21.15.105:0/1270596247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe41019d0 msgr2=0x7effe410cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe41019d0 0x7effe410cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 -- 172.21.15.105:0/1270596247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe410dc70 con 0x7effe41007e0 2024-03-26T17:46:07.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effe9fe7640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41007e0 0x7effe410e860 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7effe000e9c0 tx=0x7effe000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effdaffd640 1 -- 172.21.15.105:0/1270596247 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe000cde0 con 0x7effe41007e0 2024-03-26T17:46:07.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe41a1c60 con 0x7effe41007e0 2024-03-26T17:46:07.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.516+0000 7effdaffd640 1 -- 172.21.15.105:0/1270596247 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effe00108e0 con 0x7effe41007e0 2024-03-26T17:46:07.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.517+0000 7effdaffd640 1 -- 172.21.15.105:0/1270596247 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe00188a0 con 0x7effe41007e0 2024-03-26T17:46:07.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.517+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe41a21d0 con 0x7effe41007e0 2024-03-26T17:46:07.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.518+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe41a23a0 con 0x7effe41007e0 2024-03-26T17:46:07.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.519+0000 7effdaffd640 1 -- 172.21.15.105:0/1270596247 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7effe0010a90 con 0x7effe41007e0 2024-03-26T17:46:07.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.519+0000 7effdaffd640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effd00a2f20 0x7effd00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.519+0000 7effe97e6640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effd00a2f20 0x7effd00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.519+0000 7effdaffd640 1 -- 172.21.15.105:0/1270596247 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7effe00d0030 con 0x7effe41007e0 2024-03-26T17:46:07.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.520+0000 7effe97e6640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effd00a2f20 0x7effd00a53e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7effd4009230 tx=0x7effd4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.527+0000 7effdaffd640 1 -- 172.21.15.105:0/1270596247 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7effe0026070 con 0x7effe41007e0 2024-03-26T17:46:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.811+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effe41a2570 con 0x7effe41007e0 2024-03-26T17:46:07.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.812+0000 7effdaffd640 1 -- 172.21.15.105:0/1270596247 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7effe009ce20 con 0x7effe41007e0 2024-03-26T17:46:07.805 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:46:07.805 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:46:07.805 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effd00a2f20 msgr2=0x7effd00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effd00a2f20 0x7effd00a53e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7effd4009230 tx=0x7effd4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41007e0 msgr2=0x7effe410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41007e0 0x7effe410e860 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7effe000e9c0 tx=0x7effe000ee80 comp rx=0 tx=0).stop 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 shutdown_connections 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7effd00a2f20 0x7effd00a53e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effe4102390 0x7effe410d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effe41019d0 0x7effe410cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 --2- 172.21.15.105:0/1270596247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7effe41007e0 0x7effe410e860 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 >> 172.21.15.105:0/1270596247 conn(0x7effe40fbd50 msgr2=0x7effe40fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 shutdown_connections 2024-03-26T17:46:07.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.814+0000 7effec24b640 1 -- 172.21.15.105:0/1270596247 wait complete. 2024-03-26T17:46:07.819 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:46:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 -- 172.21.15.105:0/3920990232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c102340 msgr2=0x7f8d0c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/3920990232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c102340 0x7f8d0c104730 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009900 tx=0x7f8d0002bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 -- 172.21.15.105:0/3920990232 shutdown_connections 2024-03-26T17:46:07.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/3920990232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d0c1075a0 0x7f8d0c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/3920990232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c104c70 0x7f8d0c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/3920990232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c102340 0x7f8d0c104730 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 -- 172.21.15.105:0/3920990232 >> 172.21.15.105:0/3920990232 conn(0x7f8d0c0fbd50 msgr2=0x7f8d0c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 -- 172.21.15.105:0/3920990232 shutdown_connections 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.944+0000 7f8d12b3b640 1 -- 172.21.15.105:0/3920990232 wait complete. 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d12b3b640 1 Processor -- start 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d12b3b640 1 -- start start 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d12b3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d0c102340 0x7f8d0c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d12b3b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c104c70 0x7f8d0c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d12b3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c1075a0 0x7f8d0c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d12b3b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10d410 con 0x7f8d0c1075a0 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d12b3b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10d290 con 0x7f8d0c104c70 2024-03-26T17:46:07.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d12b3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c10d590 con 0x7f8d0c102340 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d110d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c1075a0 0x7f8d0c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d110d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c1075a0 0x7f8d0c19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37910/0 (socket says 172.21.15.105:37910) 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d110d8640 1 -- 172.21.15.105:0/515091679 learned_addr learned my addr 172.21.15.105:0/515091679 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d108d7640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d0c102340 0x7f8d0c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.945+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c104c70 0x7f8d0c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d110d8640 1 -- 172.21.15.105:0/515091679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d0c102340 msgr2=0x7f8d0c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d110d8640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d0c102340 0x7f8d0c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d110d8640 1 -- 172.21.15.105:0/515091679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c104c70 msgr2=0x7f8d0c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d110d8640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c104c70 0x7f8d0c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d110d8640 1 -- 172.21.15.105:0/515091679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c19f790 con 0x7f8d0c1075a0 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d108d7640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d0c102340 0x7f8d0c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d110d8640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c1075a0 0x7f8d0c19f120 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400eb10 tx=0x7f8d0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d09ffb640 1 -- 172.21.15.105:0/515091679 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004b20 con 0x7f8d0c1075a0 2024-03-26T17:46:07.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d09ffb640 1 -- 172.21.15.105:0/515091679 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d04004cc0 con 0x7f8d0c1075a0 2024-03-26T17:46:07.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d12b3b640 1 -- 172.21.15.105:0/515091679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c19fa80 con 0x7f8d0c1075a0 2024-03-26T17:46:07.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d09ffb640 1 -- 172.21.15.105:0/515091679 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04010690 con 0x7f8d0c1075a0 2024-03-26T17:46:07.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.946+0000 7f8d12b3b640 1 -- 172.21.15.105:0/515091679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c19ffd0 con 0x7f8d0c1075a0 2024-03-26T17:46:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.949+0000 7f8d09ffb640 1 -- 172.21.15.105:0/515091679 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8d04005020 con 0x7f8d0c1075a0 2024-03-26T17:46:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.949+0000 7f8d15803740 1 Processor -- start 2024-03-26T17:46:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.949+0000 7f8d15803740 1 -- start start 2024-03-26T17:46:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.949+0000 7f8d09ffb640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cfc0a2f20 0x7f8cfc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.949+0000 7f8d15803740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765dfa30 0x5577765dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d15803740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d15803740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765e5590 0x5577765e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d15803740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557776544970 con 0x5577765e0420 2024-03-26T17:46:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d15803740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5577764ecd70 con 0x5577765dfa30 2024-03-26T17:46:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d15803740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557776540920 con 0x5577765e5590 2024-03-26T17:46:07.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cfc0a2f20 0x7f8cfc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765dfa30 0x5577765dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d09ffb640 1 -- 172.21.15.105:0/515091679 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d040cfae0 con 0x7f8d0c1075a0 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765dfa30 0x5577765dfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53130/0 (socket says 172.21.15.105:53130) 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 -- 172.21.15.105:0/3458545436 learned_addr learned my addr 172.21.15.105:0/3458545436 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d110d8640 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765e5590 0x5577765e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 -- 172.21.15.105:0/3458545436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765e5590 msgr2=0x5577765e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765e5590 0x5577765e7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 -- 172.21.15.105:0/3458545436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 msgr2=0x5577765e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 -- 172.21.15.105:0/3458545436 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577762583f0 con 0x5577765dfa30 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d110d8640 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765e5590 0x5577765e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:46:07.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cfc0a2f20 0x7f8cfc0a53e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c199080 tx=0x7f8cf8011860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.950+0000 7f8d108d7640 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765dfa30 0x5577765dfe50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009900 tx=0x7f8d000095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc0813c79c782ff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8cf6ffd640 1 -- 172.21.15.105:0/3458545436 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0002cc90 con 0x5577765dfa30 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8cf6ffd640 1 -- 172.21.15.105:0/3458545436 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d00005040 con 0x5577765dfa30 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8cf6ffd640 1 -- 172.21.15.105:0/3458545436 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00040a50 con 0x5577765dfa30 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8d15803740 1 -- 172.21.15.105:0/3458545436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765dfa30 msgr2=0x5577765dfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8d15803740 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765dfa30 0x5577765dfe50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009900 tx=0x7f8d000095f0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8d15803740 1 -- 172.21.15.105:0/3458545436 shutdown_connections 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8d15803740 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765e5590 0x5577765e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8d15803740 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8d15803740 1 --2- 172.21.15.105:0/3458545436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765dfa30 0x5577765dfe50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8d15803740 1 -- 172.21.15.105:0/3458545436 >> 172.21.15.105:0/3458545436 conn(0x5577765dafa0 msgr2=0x5577765db3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.951+0000 7f8d15803740 1 -- 172.21.15.105:0/3458545436 shutdown_connections 2024-03-26T17:46:07.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 -- 172.21.15.105:0/3458545436 wait complete. 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 -- start start 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 Processor -- start 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 0x5577765f1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5577765e8530 con 0x5577765e0420 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557776500310 con 0x5577765e5590 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5577765e86b0 con 0x5577765dfa30 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d0bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d108d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d108d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765e9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40740/0 (socket says 172.21.15.105:40740) 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d108d7640 1 -- 172.21.15.105:0/2492931826 learned_addr learned my addr 172.21.15.105:0/2492931826 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d15803740 10 client.0 _ll_register_callbacks cb 0x55777653bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.952+0000 7f8d110d8640 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 0x5577765f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d108d7640 1 -- 172.21.15.105:0/2492931826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 msgr2=0x5577765f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d108d7640 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 0x5577765f1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d108d7640 1 -- 172.21.15.105:0/2492931826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 msgr2=0x5577765ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d108d7640 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765ea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d108d7640 1 -- 172.21.15.105:0/2492931826 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577763227d0 con 0x5577765dfa30 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d108d7640 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765e9ef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d00004650 tx=0x7f8d00040a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a34e30103a882b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8cebfff640 1 -- 172.21.15.105:0/2492931826 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00059070 con 0x5577765dfa30 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8cebfff640 1 -- 172.21.15.105:0/2492931826 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d000558d0 con 0x5577765dfa30 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d15803740 1 -- 172.21.15.105:0/2492931826 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d00005040 con 0x5577765dfa30 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8cebfff640 1 -- 172.21.15.105:0/2492931826 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d00065050 con 0x5577765dfa30 2024-03-26T17:46:07.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d15803740 10 client.28051 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.953+0000 7f8d15803740 1 -- 172.21.15.105:0/2492931826 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577765e85a0 con 0x5577765dfa30 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8cebfff640 1 -- 172.21.15.105:0/2492931826 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d00005040 con 0x5577765dfa30 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8cebfff640 1 -- 172.21.15.105:0/2492931826 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8d00004850 con 0x5577765dfa30 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8d15803740 10 client.28051 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8d15803740 10 client.28051 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8d15803740 20 client.28051 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8d15803740 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5577765f5a20 0x5577765f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8d15803740 4 client.28051 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8d15803740 1 -- 172.21.15.105:0/2492931826 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557776551eb0 con 0x5577765f5a20 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5577765f5a20 0x5577765f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8cebfff640 10 client.28051 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.954+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5577765f5a20 0x5577765f7e90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c58fe582be0d7a5c server_cookie=54dcbedd7d7be22c in_seq=0 out_seq=0 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.955+0000 7f8cebfff640 1 -- 172.21.15.105:0/2492931826 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557776551eb0 con 0x5577765f5a20 2024-03-26T17:46:07.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.955+0000 7f8cebfff640 10 client.28051 handle_command_reply: tid=0 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.955+0000 7f8d15803740 1 client.28051 shutdown 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.955+0000 7f8d15803740 1 -- 172.21.15.105:0/2492931826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 msgr2=0x5577765e9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.955+0000 7f8d15803740 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765e9ef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d00004650 tx=0x7f8d00040a60 comp rx=0 tx=0).stop 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 1 -- 172.21.15.105:0/2492931826 shutdown_connections 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5577765f5a20 0x5577765f7e90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 0x5577765f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8cebfff640 0 client.28051 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 1 --2- 172.21.15.105:0/2492931826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765e9ef0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 1 -- 172.21.15.105:0/2492931826 >> 172.21.15.105:0/2492931826 conn(0x5577765dafa0 msgr2=0x5577765da1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 1 -- 172.21.15.105:0/2492931826 shutdown_connections 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 1 -- 172.21.15.105:0/2492931826 wait complete. 2024-03-26T17:46:07.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.956+0000 7f8d15803740 20 client.28051 trim_cache size 0 max 16384 2024-03-26T17:46:07.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d15803740 1 Processor -- start 2024-03-26T17:46:07.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d15803740 1 -- start start 2024-03-26T17:46:07.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d15803740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765de910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d15803740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765f1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d15803740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 0x5577765e9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d15803740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1a6ba0 con 0x5577765e0420 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d15803740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1a86e0 con 0x5577765e5590 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d15803740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1a7270 con 0x5577765dfa30 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765f1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765f1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37952/0 (socket says 172.21.15.105:37952) 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 -- 172.21.15.105:0/2707767898 learned_addr learned my addr 172.21.15.105:0/2707767898 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d110d8640 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 0x5577765e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d108d7640 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765de910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 -- 172.21.15.105:0/2707767898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 msgr2=0x5577765de910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765de910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 -- 172.21.15.105:0/2707767898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 msgr2=0x5577765e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 0x5577765e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 -- 172.21.15.105:0/2707767898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577765e52e0 con 0x5577765e0420 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d108d7640 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.974+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765f1320 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800f730 tx=0x7f8cf801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc37e68183e64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8cf67fc640 1 -- 172.21.15.105:0/2707767898 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800fcf0 con 0x5577765e0420 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8cf67fc640 1 -- 172.21.15.105:0/2707767898 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cf8032070 con 0x5577765e0420 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8cf67fc640 1 -- 172.21.15.105:0/2707767898 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf802b6d0 con 0x5577765e0420 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 -- 172.21.15.105:0/2707767898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 msgr2=0x5577765f1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765f1320 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800f730 tx=0x7f8cf801f000 comp rx=0 tx=0).stop 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 -- 172.21.15.105:0/2707767898 shutdown_connections 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e5590 0x5577765e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e0420 0x5577765f1320 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 --2- 172.21.15.105:0/2707767898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 -- 172.21.15.105:0/2707767898 >> 172.21.15.105:0/2707767898 conn(0x5577765dafa0 msgr2=0x5577765de200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 -- 172.21.15.105:0/2707767898 shutdown_connections 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.975+0000 7f8d15803740 1 -- 172.21.15.105:0/2707767898 wait complete. 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 1 -- start start 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 1 Processor -- start 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e0420 0x5577765ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e5590 0x557776659d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1a8010 con 0x5577765e5590 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1a7940 con 0x5577765e0420 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1a4b80 con 0x5577765dfa30 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d15803740 10 client.0 _ll_register_callbacks cb 0x7f8d00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d110d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e5590 0x557776659d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d0bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e0420 0x5577765ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d0bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e0420 0x5577765ee8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53148/0 (socket says 172.21.15.105:53148) 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d0bfff640 1 -- 172.21.15.105:0/1949856558 learned_addr learned my addr 172.21.15.105:0/1949856558 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.976+0000 7f8d108d7640 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d110d8640 1 -- 172.21.15.105:0/1949856558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 msgr2=0x5577765ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d110d8640 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765ee3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d110d8640 1 -- 172.21.15.105:0/1949856558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e0420 msgr2=0x5577765ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d110d8640 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e0420 0x5577765ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d110d8640 1 -- 172.21.15.105:0/1949856558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777665a2c0 con 0x5577765e5590 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d0bfff640 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e0420 0x5577765ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d110d8640 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e5590 0x557776659d80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8d040cd910 tx=0x7f8d04009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555bdbeb2e1872bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8cebfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8cebfff640 1 -- 172.21.15.105:0/1949856558 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040b4070 con 0x5577765e5590 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8cebfff640 1 -- 172.21.15.105:0/1949856558 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d04005310 con 0x5577765e5590 2024-03-26T17:46:07.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d15803740 1 -- 172.21.15.105:0/1949856558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf8032070 con 0x5577765e5590 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8cebfff640 1 -- 172.21.15.105:0/1949856558 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d040a0320 con 0x5577765e5590 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d15803740 10 client.18225 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8d15803740 1 -- 172.21.15.105:0/1949856558 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55777665b640 con 0x5577765e5590 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.977+0000 7f8cebfff640 1 -- 172.21.15.105:0/1949856558 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cf8032070 con 0x5577765e5590 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8cebfff640 1 -- 172.21.15.105:0/1949856558 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8d04097f20 con 0x5577765e5590 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8d15803740 10 client.18225 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8d15803740 10 client.18225 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8d15803740 20 client.18225 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8d15803740 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55777665c040 0x55777665e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8d15803740 4 client.18225 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8d15803740 1 -- 172.21.15.105:0/1949856558 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55777654c200 con 0x55777665c040 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8d108d7640 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55777665c040 0x55777665e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8cebfff640 10 client.18225 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:46:07.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8d108d7640 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55777665c040 0x55777665e460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=90b31b827adc1b31 server_cookie=38886d8f2780d745 in_seq=0 out_seq=0 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8cebfff640 1 -- 172.21.15.105:0/1949856558 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55777654c200 con 0x55777665c040 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.978+0000 7f8cebfff640 10 client.18225 handle_command_reply: tid=0 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 client.18225 shutdown 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 -- 172.21.15.105:0/1949856558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e5590 msgr2=0x557776659d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e5590 0x557776659d80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f8d040cd910 tx=0x7f8d04009a70 comp rx=0 tx=0).stop 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 -- 172.21.15.105:0/1949856558 shutdown_connections 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55777665c040 0x55777665e460 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577765e5590 0x557776659d80 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577765e0420 0x5577765ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8cebfff640 0 client.18225 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 --2- 172.21.15.105:0/1949856558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577765dfa30 0x5577765ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 -- 172.21.15.105:0/1949856558 >> 172.21.15.105:0/1949856558 conn(0x5577765dafa0 msgr2=0x5577765dea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 -- 172.21.15.105:0/1949856558 shutdown_connections 2024-03-26T17:46:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 1 -- 172.21.15.105:0/1949856558 wait complete. 2024-03-26T17:46:07.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.979+0000 7f8d15803740 20 client.18225 trim_cache size 0 max 16384 2024-03-26T17:46:07.979 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 -- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cfc0a2f20 msgr2=0x7f8cfc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cfc0a2f20 0x7f8cfc0a53e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c199080 tx=0x7f8cf8011860 comp rx=0 tx=0).stop 2024-03-26T17:46:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 -- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c1075a0 msgr2=0x7f8d0c19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c1075a0 0x7f8d0c19f120 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400eb10 tx=0x7f8d0400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 -- 172.21.15.105:0/515091679 shutdown_connections 2024-03-26T17:46:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cfc0a2f20 0x7f8cfc0a53e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d0c1075a0 0x7f8d0c19f120 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d0c104c70 0x7f8d0c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 --2- 172.21.15.105:0/515091679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d0c102340 0x7f8d0c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:07.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 -- 172.21.15.105:0/515091679 >> 172.21.15.105:0/515091679 conn(0x7f8d0c0fbd50 msgr2=0x7f8d0c1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:07.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 -- 172.21.15.105:0/515091679 shutdown_connections 2024-03-26T17:46:07.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:07.980+0000 7f8d12b3b640 1 -- 172.21.15.105:0/515091679 wait complete. 2024-03-26T17:46:07.983 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:46:08.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 241617 3.15 MB/sec execute 291 sec latency 16.569 ms 2024-03-26T17:46:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:07 smithi144 ceph-mon[36095]: pgmap v581: 97 pgs: 97 active+clean; 28 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:46:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:07 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1270596247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:46:08.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 -- 172.21.15.105:0/1577632843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01007c0 msgr2=0x7fd8d0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 --2- 172.21.15.105:0/1577632843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01007c0 0x7fd8d0100be0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009860 tx=0x7fd8c002bb70 comp rx=0 tx=0).stop 2024-03-26T17:46:08.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 -- 172.21.15.105:0/1577632843 shutdown_connections 2024-03-26T17:46:08.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 --2- 172.21.15.105:0/1577632843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8d0102370 0x7fd8d0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 --2- 172.21.15.105:0/1577632843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8d01019b0 0x7fd8d0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 --2- 172.21.15.105:0/1577632843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01007c0 0x7fd8d0100be0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 -- 172.21.15.105:0/1577632843 >> 172.21.15.105:0/1577632843 conn(0x7fd8d00fbd50 msgr2=0x7fd8d00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 -- 172.21.15.105:0/1577632843 shutdown_connections 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.110+0000 7fd8d6863640 1 -- 172.21.15.105:0/1577632843 wait complete. 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d6863640 1 Processor -- start 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d6863640 1 -- start start 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d6863640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8d01007c0 0x7fd8d00695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d6863640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01019b0 0x7fd8d019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d6863640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8d0102370 0x7fd8d019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d6863640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d010c2c0 con 0x7fd8d0102370 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d6863640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d010c140 con 0x7fd8d01007c0 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d6863640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d010c440 con 0x7fd8d01019b0 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8d0102370 0x7fd8d019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8d01007c0 0x7fd8d00695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01019b0 0x7fd8d019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8d01007c0 0x7fd8d00695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53176/0 (socket says 172.21.15.105:53176) 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8d4e00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8d0102370 0x7fd8d019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37984/0 (socket says 172.21.15.105:37984) 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01019b0 0x7fd8d019c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40772/0 (socket says 172.21.15.105:40772) 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.111+0000 7fd8cffff640 1 -- 172.21.15.105:0/2280642156 learned_addr learned my addr 172.21.15.105:0/2280642156 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8d4e00640 1 -- 172.21.15.105:0/2280642156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01019b0 msgr2=0x7fd8d019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8d4e00640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01019b0 0x7fd8d019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8d4e00640 1 -- 172.21.15.105:0/2280642156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8d01007c0 msgr2=0x7fd8d00695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8d4e00640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8d01007c0 0x7fd8d00695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8d4e00640 1 -- 172.21.15.105:0/2280642156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d019f760 con 0x7fd8d0102370 2024-03-26T17:46:08.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8cffff640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8d01007c0 0x7fd8d00695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8d4e00640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8d0102370 0x7fd8d019eea0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800ef00 tx=0x7fd8c800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8cdffb640 1 -- 172.21.15.105:0/2280642156 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8004b20 con 0x7fd8d0102370 2024-03-26T17:46:08.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8d6863640 1 -- 172.21.15.105:0/2280642156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d019fa50 con 0x7fd8d0102370 2024-03-26T17:46:08.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8cdffb640 1 -- 172.21.15.105:0/2280642156 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8c8003e20 con 0x7fd8d0102370 2024-03-26T17:46:08.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8cdffb640 1 -- 172.21.15.105:0/2280642156 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800b570 con 0x7fd8d0102370 2024-03-26T17:46:08.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.112+0000 7fd8d6863640 1 -- 172.21.15.105:0/2280642156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d019ffa0 con 0x7fd8d0102370 2024-03-26T17:46:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.115+0000 7fd8cdffb640 1 -- 172.21.15.105:0/2280642156 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd8c801d020 con 0x7fd8d0102370 2024-03-26T17:46:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.115+0000 7fd8d944a740 1 Processor -- start 2024-03-26T17:46:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.115+0000 7fd8d944a740 1 -- start start 2024-03-26T17:46:08.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef2420 0x559974ef6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974ef9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8d944a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559974e56970 con 0x559974ef1a30 2024-03-26T17:46:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8d944a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559974dfed70 con 0x559974ef2420 2024-03-26T17:46:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8d944a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559974e52920 con 0x559974ef7590 2024-03-26T17:46:08.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37986/0 (socket says 172.21.15.105:37986) 2024-03-26T17:46:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1115330600 learned_addr learned my addr 172.21.15.105:0/1115330600 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974ef9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8cffff640 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef2420 0x559974ef6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8cdffb640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8bc0a2f20 0x7fd8bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1115330600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 msgr2=0x559974ef9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974ef9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1115330600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef2420 msgr2=0x559974ef6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef2420 0x559974ef6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8cdffb640 1 -- 172.21.15.105:0/2280642156 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd8c80cf290 con 0x7fd8d0102370 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8d4e00640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8bc0a2f20 0x7fd8bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1115330600 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559974b6a3f0 con 0x559974ef1a30 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8cffff640 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef2420 0x559974ef6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef1e50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8013630 tx=0x7fd8b8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4479fe7638ac798 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.116+0000 7fd8c67fc640 1 -- 172.21.15.105:0/1115330600 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800f500 con 0x559974ef1a30 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8c67fc640 1 -- 172.21.15.105:0/1115330600 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b800fb50 con 0x559974ef1a30 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8c67fc640 1 -- 172.21.15.105:0/1115330600 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8010a30 con 0x559974ef1a30 2024-03-26T17:46:08.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 -- 172.21.15.105:0/1115330600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 msgr2=0x559974ef1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d4e00640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8bc0a2f20 0x7fd8bc0a53e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800cc40 tx=0x7fd8c800cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef1e50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8013630 tx=0x7fd8b8005f90 comp rx=0 tx=0).stop 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 -- 172.21.15.105:0/1115330600 shutdown_connections 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974ef9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef2420 0x559974ef6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1115330600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef1e50 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 -- 172.21.15.105:0/1115330600 >> 172.21.15.105:0/1115330600 conn(0x559974eecfa0 msgr2=0x559974eed3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 -- 172.21.15.105:0/1115330600 shutdown_connections 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.117+0000 7fd8d944a740 1 -- 172.21.15.105:0/1115330600 wait complete. 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 1 -- start start 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 1 Processor -- start 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef1a30 0x559974efbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974efc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559974efa540 con 0x559974ef2420 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559974e12310 con 0x559974ef7590 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559974efa6c0 con 0x559974ef1a30 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8d944a740 10 client.0 _ll_register_callbacks cb 0x559974e4dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974efc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53186/0 (socket says 172.21.15.105:53186) 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974efc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37988/0 (socket says 172.21.15.105:37988) 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1837750112 learned_addr learned my addr 172.21.15.105:0/1837750112 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.118+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef1a30 0x559974efbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8cffff640 1 -- 172.21.15.105:0/1837750112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef1a30 msgr2=0x559974efbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8cffff640 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef1a30 0x559974efbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8cffff640 1 -- 172.21.15.105:0/1837750112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 msgr2=0x559974f03460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8cffff640 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8cffff640 1 -- 172.21.15.105:0/1837750112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559974c347d0 con 0x559974ef2420 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8cffff640 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974efc420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009830 tx=0x7fd8c0034f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06c315fbc8ddc71 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1837750112 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0039040 con 0x559974ef2420 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1837750112 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd8c0004450 con 0x559974ef2420 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8d944a740 1 -- 172.21.15.105:0/1837750112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b800fb50 con 0x559974ef2420 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1837750112 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8c004d050 con 0x559974ef2420 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8d944a740 10 client.18237 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8d944a740 1 -- 172.21.15.105:0/1837750112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559974f06840 con 0x559974ef2420 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.119+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1837750112 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b800fb50 con 0x559974ef2420 2024-03-26T17:46:08.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1837750112 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd8c0003a70 con 0x559974ef2420 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8d944a740 10 client.18237 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8d944a740 10 client.18237 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8d944a740 20 client.18237 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559974f07fb0 0x559974f0a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8d944a740 4 client.18237 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8d944a740 1 -- 172.21.15.105:0/1837750112 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559974e63eb0 con 0x559974f07fb0 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559974f07fb0 0x559974f0a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8af7fe640 10 client.18237 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.120+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559974f07fb0 0x559974f0a3f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f7436a50f90eb72 server_cookie=2611d61821b6d549 in_seq=0 out_seq=0 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.121+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1837750112 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559974e63eb0 con 0x559974f07fb0 2024-03-26T17:46:08.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.121+0000 7fd8af7fe640 10 client.18237 handle_command_reply: tid=0 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.121+0000 7fd8d944a740 1 client.18237 shutdown 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.121+0000 7fd8d944a740 1 -- 172.21.15.105:0/1837750112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 msgr2=0x559974efc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.121+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974efc420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009830 tx=0x7fd8c0034f70 comp rx=0 tx=0).stop 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 1 -- 172.21.15.105:0/1837750112 shutdown_connections 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559974f07fb0 0x559974f0a3f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8af7fe640 0 client.18237 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974efc420 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1837750112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef1a30 0x559974efbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 1 -- 172.21.15.105:0/1837750112 >> 172.21.15.105:0/1837750112 conn(0x559974eecfa0 msgr2=0x559974eec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 1 -- 172.21.15.105:0/1837750112 shutdown_connections 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 1 -- 172.21.15.105:0/1837750112 wait complete. 2024-03-26T17:46:08.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.122+0000 7fd8d944a740 20 client.18237 trim_cache size 0 max 16384 2024-03-26T17:46:08.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.138+0000 7fd8d944a740 1 Processor -- start 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8d944a740 1 -- start start 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef2420 0x559974ef3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8d944a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01a8dc0 con 0x559974ef1a30 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8d944a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01aa900 con 0x559974ef7590 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8d944a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01a9490 con 0x559974ef2420 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38004/0 (socket says 172.21.15.105:38004) 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 -- 172.21.15.105:0/482974306 learned_addr learned my addr 172.21.15.105:0/482974306 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8cffff640 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef2420 0x559974ef3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 -- 172.21.15.105:0/482974306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef2420 msgr2=0x559974ef3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef2420 0x559974ef3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 -- 172.21.15.105:0/482974306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 msgr2=0x559974f03420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 -- 172.21.15.105:0/482974306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559974efa7c0 con 0x559974ef1a30 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.139+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef3920 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8010010 tx=0x7fd8b8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6216f4e41a326cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.140+0000 7fd8c5ffb640 1 -- 172.21.15.105:0/482974306 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80304c0 con 0x559974ef1a30 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.140+0000 7fd8c5ffb640 1 -- 172.21.15.105:0/482974306 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b80054f0 con 0x559974ef1a30 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.140+0000 7fd8c5ffb640 1 -- 172.21.15.105:0/482974306 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800fa70 con 0x559974ef1a30 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.140+0000 7fd8d944a740 1 -- 172.21.15.105:0/482974306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 msgr2=0x559974ef3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.140+0000 7fd8d944a740 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef3920 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8010010 tx=0x7fd8b8015040 comp rx=0 tx=0).stop 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 -- 172.21.15.105:0/482974306 shutdown_connections 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974ef7590 0x559974f03420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef2420 0x559974ef3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 --2- 172.21.15.105:0/482974306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef1a30 0x559974ef3920 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 -- 172.21.15.105:0/482974306 >> 172.21.15.105:0/482974306 conn(0x559974eecfa0 msgr2=0x559974eed380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 -- 172.21.15.105:0/482974306 shutdown_connections 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 -- 172.21.15.105:0/482974306 wait complete. 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 -- start start 2024-03-26T17:46:08.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 Processor -- start 2024-03-26T17:46:08.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974f692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974f69810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974f07fb0 0x559974f70850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01aa230 con 0x559974ef2420 2024-03-26T17:46:08.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01a9b60 con 0x559974f07fb0 2024-03-26T17:46:08.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01a6da0 con 0x559974ef7590 2024-03-26T17:46:08.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.141+0000 7fd8d944a740 10 client.0 _ll_register_callbacks cb 0x7fd8c8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974f692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8cffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974f69810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974f692d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38014/0 (socket says 172.21.15.105:38014) 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1161787254 learned_addr learned my addr 172.21.15.105:0/1161787254 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1161787254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 msgr2=0x559974f69810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974f69810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1161787254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974f07fb0 msgr2=0x559974f70850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974f07fb0 0x559974f70850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 -- 172.21.15.105:0/1161787254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559974f70e40 con 0x559974ef2420 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8cffff640 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974f69810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8c7fff640 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974f692d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x559974f04100 tx=0x7fd8b8009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7514b26d6f81ba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1161787254 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80052d0 con 0x559974ef2420 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1161787254 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fd8b80304c0 con 0x559974ef2420 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8d944a740 1 -- 172.21.15.105:0/1161787254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b80054f0 con 0x559974ef2420 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1161787254 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8b8042050 con 0x559974ef2420 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8d944a740 10 client.18243 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.142+0000 7fd8d944a740 1 -- 172.21.15.105:0/1161787254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559974f72140 con 0x559974ef2420 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1161787254 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b80054f0 con 0x559974ef2420 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1161787254 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd8b8003b20 con 0x559974ef2420 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8d944a740 10 client.18243 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8d944a740 10 client.18243 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8d944a740 20 client.18243 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559974f72850 0x559974f74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8d944a740 4 client.18243 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:08.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8d944a740 1 -- 172.21.15.105:0/1161787254 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559974e5e200 con 0x559974f72850 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8cffff640 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559974f72850 0x559974f74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8af7fe640 10 client.18243 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.143+0000 7fd8cffff640 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559974f72850 0x559974f74c70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c2b489dcaabf0772 server_cookie=c99a48a639c95144 in_seq=0 out_seq=0 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8af7fe640 1 -- 172.21.15.105:0/1161787254 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559974e5e200 con 0x559974f72850 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8af7fe640 10 client.18243 handle_command_reply: tid=0 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 client.18243 shutdown 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 -- 172.21.15.105:0/1161787254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 msgr2=0x559974f692d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974f692d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x559974f04100 tx=0x7fd8b8009b00 comp rx=0 tx=0).stop 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 -- 172.21.15.105:0/1161787254 shutdown_connections 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559974f72850 0x559974f74c70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559974f07fb0 0x559974f70850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559974ef7590 0x559974f69810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8af7fe640 0 client.18243 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 --2- 172.21.15.105:0/1161787254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559974ef2420 0x559974f692d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 -- 172.21.15.105:0/1161787254 >> 172.21.15.105:0/1161787254 conn(0x559974eecfa0 msgr2=0x559974ef3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 -- 172.21.15.105:0/1161787254 shutdown_connections 2024-03-26T17:46:08.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 1 -- 172.21.15.105:0/1161787254 wait complete. 2024-03-26T17:46:08.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.144+0000 7fd8d944a740 20 client.18243 trim_cache size 0 max 16384 2024-03-26T17:46:08.145 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 -- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8bc0a2f20 msgr2=0x7fd8bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8bc0a2f20 0x7fd8bc0a53e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800cc40 tx=0x7fd8c800cf70 comp rx=0 tx=0).stop 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 -- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8d0102370 msgr2=0x7fd8d019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8d0102370 0x7fd8d019eea0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800ef00 tx=0x7fd8c800c740 comp rx=0 tx=0).stop 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 -- 172.21.15.105:0/2280642156 shutdown_connections 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd8bc0a2f20 0x7fd8bc0a53e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd8d0102370 0x7fd8d019eea0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8d01019b0 0x7fd8d019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 --2- 172.21.15.105:0/2280642156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd8d01007c0 0x7fd8d00695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 -- 172.21.15.105:0/2280642156 >> 172.21.15.105:0/2280642156 conn(0x7fd8d00fbd50 msgr2=0x7fd8d0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 -- 172.21.15.105:0/2280642156 shutdown_connections 2024-03-26T17:46:08.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.145+0000 7fd8d6863640 1 -- 172.21.15.105:0/2280642156 wait complete. 2024-03-26T17:46:08.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:07 smithi105 ceph-mon[32280]: pgmap v581: 97 pgs: 97 active+clean; 28 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:46:08.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:07 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1270596247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:46:08.151 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:46:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:07 smithi135 ceph-mon[35636]: pgmap v581: 97 pgs: 97 active+clean; 28 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-03-26T17:46:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:07 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1270596247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:46:08.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 -- 172.21.15.105:0/4144687539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 msgr2=0x7f87f0100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/4144687539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 0x7f87f0100c20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009900 tx=0x7f87e002bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:08.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 -- 172.21.15.105:0/4144687539 shutdown_connections 2024-03-26T17:46:08.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/4144687539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87f01023b0 0x7f87f0108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/4144687539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f01019f0 0x7f87f0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/4144687539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 0x7f87f0100c20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 -- 172.21.15.105:0/4144687539 >> 172.21.15.105:0/4144687539 conn(0x7f87f00fbd70 msgr2=0x7f87f00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 -- 172.21.15.105:0/4144687539 shutdown_connections 2024-03-26T17:46:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.279+0000 7f87f5cd1640 1 -- 172.21.15.105:0/4144687539 wait complete. 2024-03-26T17:46:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87f5cd1640 1 Processor -- start 2024-03-26T17:46:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87f5cd1640 1 -- start start 2024-03-26T17:46:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87f5cd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 0x7f87f0104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87f5cd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f01019f0 0x7f87f0102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87f5cd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87f01023b0 0x7f87f0103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87f5cd1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87f010c300 con 0x7f87f0100800 2024-03-26T17:46:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87f5cd1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87f010c180 con 0x7f87f01023b0 2024-03-26T17:46:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87f5cd1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87f010c480 con 0x7f87f01019f0 2024-03-26T17:46:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 0x7f87f0104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f01019f0 0x7f87f0102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87ef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 0x7f87f0104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38036/0 (socket says 172.21.15.105:38036) 2024-03-26T17:46:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87ef7fe640 1 -- 172.21.15.105:0/2607365540 learned_addr learned my addr 172.21.15.105:0/2607365540 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.280+0000 7f87effff640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87f01023b0 0x7f87f0103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ef7fe640 1 -- 172.21.15.105:0/2607365540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f01019f0 msgr2=0x7f87f0102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ef7fe640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f01019f0 0x7f87f0102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ef7fe640 1 -- 172.21.15.105:0/2607365540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87f01023b0 msgr2=0x7f87f0103310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ef7fe640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87f01023b0 0x7f87f0103310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ef7fe640 1 -- 172.21.15.105:0/2607365540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f0103ba0 con 0x7f87f0100800 2024-03-26T17:46:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87eeffd640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f01019f0 0x7f87f0102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ef7fe640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 0x7f87f0104790 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f87e0000c00 tx=0x7f87e0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ecff9640 1 -- 172.21.15.105:0/2607365540 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0043020 con 0x7f87f0100800 2024-03-26T17:46:08.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ecff9640 1 -- 172.21.15.105:0/2607365540 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87e00042d0 con 0x7f87f0100800 2024-03-26T17:46:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87f5cd1640 1 -- 172.21.15.105:0/2607365540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f01a1d00 con 0x7f87f0100800 2024-03-26T17:46:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87ecff9640 1 -- 172.21.15.105:0/2607365540 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0003b10 con 0x7f87f0100800 2024-03-26T17:46:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.281+0000 7f87f5cd1640 1 -- 172.21.15.105:0/2607365540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f01a21a0 con 0x7f87f0100800 2024-03-26T17:46:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.283+0000 7f87ecff9640 1 -- 172.21.15.105:0/2607365540 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f87e0046020 con 0x7f87f0100800 2024-03-26T17:46:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87f8997740 1 Processor -- start 2024-03-26T17:46:08.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87f8997740 1 -- start start 2024-03-26T17:46:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87f8997740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf6a30 0x5599ccdf6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87ecff9640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f87d80a2f20 0x7f87d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87f8997740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599ccdfbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87f8997740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599ccdfea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87f8997740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5599ccd5b970 con 0x5599ccdfc590 2024-03-26T17:46:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87f8997740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5599ccd03d70 con 0x5599ccdf7420 2024-03-26T17:46:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87f8997740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599ccd57920 con 0x5599ccdf6a30 2024-03-26T17:46:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599ccdfea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599ccdfbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87ef7fe640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f87d80a2f20 0x7f87d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599ccdfea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38042/0 (socket says 172.21.15.105:38042) 2024-03-26T17:46:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599ccdfbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53234/0 (socket says 172.21.15.105:53234) 2024-03-26T17:46:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87effff640 1 -- 172.21.15.105:0/3304422326 learned_addr learned my addr 172.21.15.105:0/3304422326 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.284+0000 7f87effff640 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf6a30 0x5599ccdf6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87ecff9640 1 -- 172.21.15.105:0/2607365540 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f87e00f1500 con 0x7f87f0100800 2024-03-26T17:46:08.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87effff640 1 -- 172.21.15.105:0/3304422326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf6a30 msgr2=0x5599ccdf6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87effff640 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf6a30 0x5599ccdf6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87effff640 1 -- 172.21.15.105:0/3304422326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 msgr2=0x5599ccdfbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87effff640 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599ccdfbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87effff640 1 -- 172.21.15.105:0/3304422326 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599cca6f3f0 con 0x5599ccdfc590 2024-03-26T17:46:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87eeffd640 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599ccdfbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87ef7fe640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f87d80a2f20 0x7f87d80a53e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f87e00ba5c0 tx=0x7f87e009e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87effff640 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599ccdfea20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009fd0 tx=0x7f87e8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38ad6ac11ff6ac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87d5ffb640 1 -- 172.21.15.105:0/3304422326 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8011070 con 0x5599ccdfc590 2024-03-26T17:46:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87d5ffb640 1 -- 172.21.15.105:0/3304422326 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87e8003e20 con 0x5599ccdfc590 2024-03-26T17:46:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.285+0000 7f87d5ffb640 1 -- 172.21.15.105:0/3304422326 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8004270 con 0x5599ccdfc590 2024-03-26T17:46:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 -- 172.21.15.105:0/3304422326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 msgr2=0x5599ccdfea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599ccdfea20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009fd0 tx=0x7f87e8009b80 comp rx=0 tx=0).stop 2024-03-26T17:46:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 -- 172.21.15.105:0/3304422326 shutdown_connections 2024-03-26T17:46:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599ccdfea20 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599ccdfbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 --2- 172.21.15.105:0/3304422326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf6a30 0x5599ccdf6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 -- 172.21.15.105:0/3304422326 >> 172.21.15.105:0/3304422326 conn(0x5599ccdf1fa0 msgr2=0x5599ccdf23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 -- 172.21.15.105:0/3304422326 shutdown_connections 2024-03-26T17:46:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 -- 172.21.15.105:0/3304422326 wait complete. 2024-03-26T17:46:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 -- start start 2024-03-26T17:46:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.286+0000 7f87f8997740 1 Processor -- start 2024-03-26T17:46:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87f8997740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 0x5599cce00f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87f8997740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce01490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87f8997740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87f8997740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5599ccdff540 con 0x5599ccdfc590 2024-03-26T17:46:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87f8997740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5599ccd17310 con 0x5599ccdf6a30 2024-03-26T17:46:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87f8997740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5599ccdff6c0 con 0x5599ccdf7420 2024-03-26T17:46:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87f8997740 10 client.0 _ll_register_callbacks cb 0x5599ccd52c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce084d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38046/0 (socket says 172.21.15.105:38046) 2024-03-26T17:46:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 -- 172.21.15.105:0/4234003006 learned_addr learned my addr 172.21.15.105:0/4234003006 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 0x5599cce00f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87eeffd640 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce01490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 -- 172.21.15.105:0/4234003006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 msgr2=0x5599cce01490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce01490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 -- 172.21.15.105:0/4234003006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 msgr2=0x5599cce00f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 0x5599cce00f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 -- 172.21.15.105:0/4234003006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ccb397d0 con 0x5599ccdfc590 2024-03-26T17:46:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87effff640 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce084d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f87e8011500 tx=0x7f87e8011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef987952482c1140 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87ceffd640 1 -- 172.21.15.105:0/4234003006 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e800f900 con 0x5599ccdfc590 2024-03-26T17:46:08.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87ceffd640 1 -- 172.21.15.105:0/4234003006 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f87e8011050 con 0x5599ccdfc590 2024-03-26T17:46:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.287+0000 7f87f8997740 1 -- 172.21.15.105:0/4234003006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e8003e20 con 0x5599ccdfc590 2024-03-26T17:46:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87ceffd640 1 -- 172.21.15.105:0/4234003006 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87e8034050 con 0x5599ccdfc590 2024-03-26T17:46:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87f8997740 10 client.18255 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87f8997740 1 -- 172.21.15.105:0/4234003006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5599cce0b840 con 0x5599ccdfc590 2024-03-26T17:46:08.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87ceffd640 1 -- 172.21.15.105:0/4234003006 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87e8003e20 con 0x5599ccdfc590 2024-03-26T17:46:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87ceffd640 1 -- 172.21.15.105:0/4234003006 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f87e802c020 con 0x5599ccdfc590 2024-03-26T17:46:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87f8997740 10 client.18255 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87f8997740 10 client.18255 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:46:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87f8997740 20 client.18255 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87f8997740 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5599cce0cdc0 0x5599cce0f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87f8997740 4 client.18255 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87f8997740 1 -- 172.21.15.105:0/4234003006 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599ccd68eb0 con 0x5599cce0cdc0 2024-03-26T17:46:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.288+0000 7f87eeffd640 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5599cce0cdc0 0x5599cce0f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.289+0000 7f87ceffd640 10 client.18255 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:46:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.289+0000 7f87eeffd640 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5599cce0cdc0 0x5599cce0f1b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b333a8773c39567 server_cookie=aaa0b4f30f0f6374 in_seq=0 out_seq=0 2024-03-26T17:46:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.289+0000 7f87ceffd640 1 -- 172.21.15.105:0/4234003006 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5599ccd68eb0 con 0x5599cce0cdc0 2024-03-26T17:46:08.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.289+0000 7f87ceffd640 10 client.18255 handle_command_reply: tid=0 2024-03-26T17:46:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.289+0000 7f87f8997740 1 client.18255 shutdown 2024-03-26T17:46:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 -- 172.21.15.105:0/4234003006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 msgr2=0x5599cce084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce084d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f87e8011500 tx=0x7f87e8011930 comp rx=0 tx=0).stop 2024-03-26T17:46:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 -- 172.21.15.105:0/4234003006 shutdown_connections 2024-03-26T17:46:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce084d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce01490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5599cce0cdc0 0x5599cce0f1b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 --2- 172.21.15.105:0/4234003006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 0x5599cce00f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 -- 172.21.15.105:0/4234003006 >> 172.21.15.105:0/4234003006 conn(0x5599ccdf1fa0 msgr2=0x5599ccdf11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87ceffd640 0 client.18255 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:46:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 -- 172.21.15.105:0/4234003006 shutdown_connections 2024-03-26T17:46:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 1 -- 172.21.15.105:0/4234003006 wait complete. 2024-03-26T17:46:08.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.290+0000 7f87f8997740 20 client.18255 trim_cache size 0 max 16384 2024-03-26T17:46:08.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.305+0000 7f87f8997740 1 Processor -- start 2024-03-26T17:46:08.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.305+0000 7f87f8997740 1 -- start start 2024-03-26T17:46:08.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87f8997740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdf6a30 0x5599cce0f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87f8997740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599cce08490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87f8997740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdfc590 0x5599ccdf0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87f8997740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01ab050 con 0x5599ccdf6a30 2024-03-26T17:46:08.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87f8997740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01acb90 con 0x5599ccdf7420 2024-03-26T17:46:08.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87f8997740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01ab720 con 0x5599ccdfc590 2024-03-26T17:46:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdf6a30 0x5599cce0f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599cce08490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdf6a30 0x5599cce0f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38052/0 (socket says 172.21.15.105:38052) 2024-03-26T17:46:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 -- 172.21.15.105:0/3223261114 learned_addr learned my addr 172.21.15.105:0/3223261114 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdfc590 0x5599ccdf0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 -- 172.21.15.105:0/3223261114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdfc590 msgr2=0x5599ccdf0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdfc590 0x5599ccdf0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 -- 172.21.15.105:0/3223261114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 msgr2=0x5599cce08490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599cce08490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 -- 172.21.15.105:0/3223261114 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ccdfbbd0 con 0x5599ccdf6a30 2024-03-26T17:46:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87eeffd640 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599cce08490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87effff640 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdf6a30 0x5599cce0f2b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f87e800a010 tx=0x7f87e8011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54788f7b9c470971 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87d57fa640 1 -- 172.21.15.105:0/3223261114 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e800f8e0 con 0x5599ccdf6a30 2024-03-26T17:46:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87d57fa640 1 -- 172.21.15.105:0/3223261114 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87e800fb90 con 0x5599ccdf6a30 2024-03-26T17:46:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.306+0000 7f87d57fa640 1 -- 172.21.15.105:0/3223261114 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e8005900 con 0x5599ccdf6a30 2024-03-26T17:46:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 -- 172.21.15.105:0/3223261114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdf6a30 msgr2=0x5599cce0f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdf6a30 0x5599cce0f2b0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f87e800a010 tx=0x7f87e8011440 comp rx=0 tx=0).stop 2024-03-26T17:46:08.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 -- 172.21.15.105:0/3223261114 shutdown_connections 2024-03-26T17:46:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdfc590 0x5599ccdf0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf7420 0x5599cce08490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 --2- 172.21.15.105:0/3223261114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdf6a30 0x5599cce0f2b0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 -- 172.21.15.105:0/3223261114 >> 172.21.15.105:0/3223261114 conn(0x5599ccdf1fa0 msgr2=0x5599cce0eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 -- 172.21.15.105:0/3223261114 shutdown_connections 2024-03-26T17:46:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.307+0000 7f87f8997740 1 -- 172.21.15.105:0/3223261114 wait complete. 2024-03-26T17:46:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 1 -- start start 2024-03-26T17:46:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 1 Processor -- start 2024-03-26T17:46:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 0x5599cce05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce06290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce07250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01ac4c0 con 0x5599ccdfc590 2024-03-26T17:46:08.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01abdf0 con 0x5599ccdf6a30 2024-03-26T17:46:08.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87f01a9030 con 0x5599ccdf7420 2024-03-26T17:46:08.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87f8997740 10 client.0 _ll_register_callbacks cb 0x7f87e4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce06290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87effff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 0x5599cce05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87eeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce06290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40878/0 (socket says 172.21.15.105:40878) 2024-03-26T17:46:08.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87eeffd640 1 -- 172.21.15.105:0/2638423962 learned_addr learned my addr 172.21.15.105:0/2638423962 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.308+0000 7f87effff640 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce07250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87eeffd640 1 -- 172.21.15.105:0/2638423962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 msgr2=0x5599cce05d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87eeffd640 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 0x5599cce05d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87eeffd640 1 -- 172.21.15.105:0/2638423962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 msgr2=0x5599cce07250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87eeffd640 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce07250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87eeffd640 1 -- 172.21.15.105:0/2638423962 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599cce07840 con 0x5599ccdf7420 2024-03-26T17:46:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87effff640 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce07250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87eeffd640 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce06290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f87e40093e0 tx=0x7f87e401f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb9248c2f3889524 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:46:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87ceffd640 1 -- 172.21.15.105:0/2638423962 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4005220 con 0x5599ccdf7420 2024-03-26T17:46:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87ceffd640 1 -- 172.21.15.105:0/2638423962 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f87e40053c0 con 0x5599ccdf7420 2024-03-26T17:46:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87f8997740 1 -- 172.21.15.105:0/2638423962 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e800fb90 con 0x5599ccdf7420 2024-03-26T17:46:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87ceffd640 1 -- 172.21.15.105:0/2638423962 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87e4012e10 con 0x5599ccdf7420 2024-03-26T17:46:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87f8997740 10 client.28093 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87f8997740 1 -- 172.21.15.105:0/2638423962 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5599cce73120 con 0x5599ccdf7420 2024-03-26T17:46:08.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.309+0000 7f87ceffd640 1 -- 172.21.15.105:0/2638423962 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87e800fb90 con 0x5599ccdf7420 2024-03-26T17:46:08.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87ceffd640 1 -- 172.21.15.105:0/2638423962 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f87e40105f0 con 0x5599ccdf7420 2024-03-26T17:46:08.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87f8997740 10 client.28093 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87f8997740 10 client.28093 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:46:08.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87f8997740 20 client.28093 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87f8997740 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5599cce73bd0 0x5599cce75ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87f8997740 4 client.28093 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:08.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87f8997740 1 -- 172.21.15.105:0/2638423962 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599ccd63200 con 0x5599cce73bd0 2024-03-26T17:46:08.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87effff640 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5599cce73bd0 0x5599cce75ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87ceffd640 10 client.28093 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:46:08.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87effff640 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5599cce73bd0 0x5599cce75ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=146794d167188fa4 server_cookie=594db9429645f6ec in_seq=0 out_seq=0 2024-03-26T17:46:08.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87ceffd640 1 -- 172.21.15.105:0/2638423962 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599ccd63200 con 0x5599cce73bd0 2024-03-26T17:46:08.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87ceffd640 10 client.28093 handle_command_reply: tid=0 2024-03-26T17:46:08.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.310+0000 7f87f8997740 1 client.28093 shutdown 2024-03-26T17:46:08.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 -- 172.21.15.105:0/2638423962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 msgr2=0x5599cce06290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce06290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f87e40093e0 tx=0x7f87e401f770 comp rx=0 tx=0).stop 2024-03-26T17:46:08.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 -- 172.21.15.105:0/2638423962 shutdown_connections 2024-03-26T17:46:08.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5599ccdfc590 0x5599cce07250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5599ccdf7420 0x5599cce06290 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5599cce73bd0 0x5599cce75ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 --2- 172.21.15.105:0/2638423962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599ccdf6a30 0x5599cce05d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87ceffd640 0 client.28093 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:46:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 -- 172.21.15.105:0/2638423962 >> 172.21.15.105:0/2638423962 conn(0x5599ccdf1fa0 msgr2=0x5599ccdf5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 -- 172.21.15.105:0/2638423962 shutdown_connections 2024-03-26T17:46:08.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 1 -- 172.21.15.105:0/2638423962 wait complete. 2024-03-26T17:46:08.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f8997740 20 client.28093 trim_cache size 0 max 16384 2024-03-26T17:46:08.339 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 -- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f87d80a2f20 msgr2=0x7f87d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f87d80a2f20 0x7f87d80a53e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f87e00ba5c0 tx=0x7f87e009e040 comp rx=0 tx=0).stop 2024-03-26T17:46:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 -- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 msgr2=0x7f87f0104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 0x7f87f0104790 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f87e0000c00 tx=0x7f87e0005d50 comp rx=0 tx=0).stop 2024-03-26T17:46:08.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 -- 172.21.15.105:0/2607365540 shutdown_connections 2024-03-26T17:46:08.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f87d80a2f20 0x7f87d80a53e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87f01023b0 0x7f87f0103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87f01019f0 0x7f87f0102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 --2- 172.21.15.105:0/2607365540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f0100800 0x7f87f0104790 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.311+0000 7f87f5cd1640 1 -- 172.21.15.105:0/2607365540 >> 172.21.15.105:0/2607365540 conn(0x7f87f00fbd70 msgr2=0x7f87f0069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.312+0000 7f87f5cd1640 1 -- 172.21.15.105:0/2607365540 shutdown_connections 2024-03-26T17:46:08.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.312+0000 7f87f5cd1640 1 -- 172.21.15.105:0/2607365540 wait complete. 2024-03-26T17:46:08.344 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:46:08.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 -- 172.21.15.105:0/1089566359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 msgr2=0x7f4870107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 --2- 172.21.15.105:0/1089566359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 0x7f4870107060 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4864009860 tx=0x7f486402bb70 comp rx=0 tx=0).stop 2024-03-26T17:46:08.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 -- 172.21.15.105:0/1089566359 shutdown_connections 2024-03-26T17:46:08.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 --2- 172.21.15.105:0/1089566359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48701075a0 0x7f48701099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 --2- 172.21.15.105:0/1089566359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 0x7f4870107060 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 --2- 172.21.15.105:0/1089566359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870102340 0x7f4870104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 -- 172.21.15.105:0/1089566359 >> 172.21.15.105:0/1089566359 conn(0x7f48700fbd50 msgr2=0x7f48700fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 -- 172.21.15.105:0/1089566359 shutdown_connections 2024-03-26T17:46:08.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.469+0000 7f4874e44640 1 -- 172.21.15.105:0/1089566359 wait complete. 2024-03-26T17:46:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.470+0000 7f4874e44640 1 Processor -- start 2024-03-26T17:46:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.470+0000 7f4874e44640 1 -- start start 2024-03-26T17:46:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f4874e44640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870102340 0x7f4870197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f4874e44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 0x7f48701980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f4874e44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48701075a0 0x7f487019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f4874e44640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f487010d400 con 0x7f48701075a0 2024-03-26T17:46:08.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f4874e44640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f487010d280 con 0x7f4870102340 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f4874e44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f487010d580 con 0x7f4870104c70 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48701075a0 0x7f487019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 0x7f48701980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870102340 0x7f4870197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 0x7f48701980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40896/0 (socket says 172.21.15.105:40896) 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 -- 172.21.15.105:0/454047438 learned_addr learned my addr 172.21.15.105:0/454047438 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48701075a0 0x7f487019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38088/0 (socket says 172.21.15.105:38088) 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 -- 172.21.15.105:0/454047438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870102340 msgr2=0x7f4870197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870102340 0x7f4870197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 -- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48701075a0 msgr2=0x7f487019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48701075a0 0x7f487019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 -- 172.21.15.105:0/454047438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487019f790 con 0x7f4870104c70 2024-03-26T17:46:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f486dd9b640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 0x7f48701980b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4864005940 tx=0x7f48640042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.471+0000 7f485f7fe640 1 -- 172.21.15.105:0/454047438 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864041020 con 0x7f4870104c70 2024-03-26T17:46:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.472+0000 7f485f7fe640 1 -- 172.21.15.105:0/454047438 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48640035f0 con 0x7f4870104c70 2024-03-26T17:46:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.472+0000 7f4874e44640 1 -- 172.21.15.105:0/454047438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487019fa20 con 0x7f4870104c70 2024-03-26T17:46:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.472+0000 7f485f7fe640 1 -- 172.21.15.105:0/454047438 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864039040 con 0x7f4870104c70 2024-03-26T17:46:08.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.472+0000 7f4874e44640 1 -- 172.21.15.105:0/454047438 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487019ff10 con 0x7f4870104c70 2024-03-26T17:46:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.474+0000 7f485f7fe640 1 -- 172.21.15.105:0/454047438 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4864044020 con 0x7f4870104c70 2024-03-26T17:46:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.474+0000 7f4877a2b740 1 Processor -- start 2024-03-26T17:46:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f4877a2b740 1 -- start start 2024-03-26T17:46:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f485f7fe640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb1de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1e420 0x556edfb22f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb25a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f4877a2b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556edfa82970 con 0x556edfb23590 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f4877a2b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556edfa2ad70 con 0x556edfb1da30 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f4877a2b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556edfa7e920 con 0x556edfb1e420 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f486dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1e420 0x556edfb22f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f486ed9d640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f486e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb1de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f486dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1e420 0x556edfb22f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40898/0 (socket says 172.21.15.105:40898) 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f486e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb1de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53278/0 (socket says 172.21.15.105:53278) 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f486dd9b640 1 -- 172.21.15.105:0/3217546254 learned_addr learned my addr 172.21.15.105:0/3217546254 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.475+0000 7f486ed9d640 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb25a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f485f7fe640 1 -- 172.21.15.105:0/454047438 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f48640f0e30 con 0x7f4870104c70 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f486dd9b640 1 -- 172.21.15.105:0/3217546254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 msgr2=0x556edfb1de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f486dd9b640 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb1de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f486dd9b640 1 -- 172.21.15.105:0/3217546254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 msgr2=0x556edfb25a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f486dd9b640 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb25a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f486dd9b640 1 -- 172.21.15.105:0/3217546254 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556edf7963f0 con 0x556edfb1e420 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f486ed9d640 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb25a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f486dd9b640 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1e420 0x556edfb22f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f48640bd1c0 tx=0x7f48640884b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c8f722e48e53c4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f485cff9640 1 -- 172.21.15.105:0/3217546254 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640bd000 con 0x556edfb1e420 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f485cff9640 1 -- 172.21.15.105:0/3217546254 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486404ceb0 con 0x556edfb1e420 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.476+0000 7f485cff9640 1 -- 172.21.15.105:0/3217546254 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640bdbc0 con 0x556edfb1e420 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 -- 172.21.15.105:0/3217546254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1e420 msgr2=0x556edfb22f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1e420 0x556edfb22f20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f48640bd1c0 tx=0x7f48640884b0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f486ed9d640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4868009c50 tx=0x7f4868009be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 -- 172.21.15.105:0/3217546254 shutdown_connections 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb25a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1e420 0x556edfb22f20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 --2- 172.21.15.105:0/3217546254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb1de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 -- 172.21.15.105:0/3217546254 >> 172.21.15.105:0/3217546254 conn(0x556edfb18fa0 msgr2=0x556edfb193a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 -- 172.21.15.105:0/3217546254 shutdown_connections 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 -- 172.21.15.105:0/3217546254 wait complete. 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 -- start start 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.477+0000 7f4877a2b740 1 Processor -- start 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb1da30 0x556edfb27ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1e420 0x556edfb281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 0x556edfb2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f4877a2b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556edfb26300 con 0x556edfb1da30 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f4877a2b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556edfa3e310 con 0x556edfb1e420 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f4877a2b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556edfb26480 con 0x556edfb23590 2024-03-26T17:46:08.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f4877a2b740 10 client.0 _ll_register_callbacks cb 0x556edfa79c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1e420 0x556edfb281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1e420 0x556edfb281e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53286/0 (socket says 172.21.15.105:53286) 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486dd9b640 1 -- 172.21.15.105:0/839992576 learned_addr learned my addr 172.21.15.105:0/839992576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb1da30 0x556edfb27ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486ed9d640 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 0x556edfb2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486dd9b640 1 -- 172.21.15.105:0/839992576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 msgr2=0x556edfb2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486dd9b640 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 0x556edfb2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486dd9b640 1 -- 172.21.15.105:0/839992576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb1da30 msgr2=0x556edfb27ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486dd9b640 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb1da30 0x556edfb27ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486dd9b640 1 -- 172.21.15.105:0/839992576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556edf8607d0 con 0x556edfb1e420 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.478+0000 7f486e59c640 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb1da30 0x556edfb27ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f486dd9b640 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1e420 0x556edfb281e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f48640349e0 tx=0x7f48640bdec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9228fe90ba202a62 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4855ffb640 1 -- 172.21.15.105:0/839992576 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640be5f0 con 0x556edfb1e420 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4877a2b740 1 -- 172.21.15.105:0/839992576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556edfa8feb0 con 0x556edfb1e420 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4855ffb640 1 -- 172.21.15.105:0/839992576 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f48640b9b30 con 0x556edfb1e420 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4855ffb640 1 -- 172.21.15.105:0/839992576 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48640bce20 con 0x556edfb1e420 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4877a2b740 10 client.28100 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4877a2b740 1 -- 172.21.15.105:0/839992576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556edfb26300 con 0x556edfb1e420 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4855ffb640 1 -- 172.21.15.105:0/839992576 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556edfa8feb0 con 0x556edfb1e420 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4855ffb640 1 -- 172.21.15.105:0/839992576 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f486403c030 con 0x556edfb1e420 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4877a2b740 10 client.28100 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4877a2b740 10 client.28100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4877a2b740 20 client.28100 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4877a2b740 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x556edfb33900 0x556edfb35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.479+0000 7f4877a2b740 4 client.28100 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:08.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.480+0000 7f4877a2b740 1 -- 172.21.15.105:0/839992576 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556edfa8a200 con 0x556edfb33900 2024-03-26T17:46:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.480+0000 7f486e59c640 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x556edfb33900 0x556edfb35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.481+0000 7f4855ffb640 10 client.28100 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:46:08.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.481+0000 7f486e59c640 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x556edfb33900 0x556edfb35d30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53186bb6042ea071 server_cookie=44735a69509d2a29 in_seq=0 out_seq=0 2024-03-26T17:46:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4855ffb640 1 -- 172.21.15.105:0/839992576 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556edfa8a200 con 0x556edfb33900 2024-03-26T17:46:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4855ffb640 10 client.28100 handle_command_reply: tid=0 2024-03-26T17:46:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 client.28100 shutdown 2024-03-26T17:46:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 -- 172.21.15.105:0/839992576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1e420 msgr2=0x556edfb281e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1e420 0x556edfb281e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f48640349e0 tx=0x7f48640bdec0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 -- 172.21.15.105:0/839992576 shutdown_connections 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x556edfb33900 0x556edfb35d30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 0x556edfb2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4855ffb640 0 client.28100 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1e420 0x556edfb281e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 --2- 172.21.15.105:0/839992576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb1da30 0x556edfb27ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 -- 172.21.15.105:0/839992576 >> 172.21.15.105:0/839992576 conn(0x556edfb18fa0 msgr2=0x556edfb181c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 -- 172.21.15.105:0/839992576 shutdown_connections 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 1 -- 172.21.15.105:0/839992576 wait complete. 2024-03-26T17:46:08.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.482+0000 7f4877a2b740 20 client.28100 trim_cache size 0 max 16384 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.498+0000 7f4877a2b740 1 Processor -- start 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.498+0000 7f4877a2b740 1 -- start start 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb26700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 0x556edfb2f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb20100 0x556edfb26c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4877a2b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f48701aafe0 con 0x556edfb20100 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4877a2b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48701acb20 con 0x556edfb1da30 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4877a2b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48701ab6b0 con 0x556edfb23590 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb20100 0x556edfb26c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb20100 0x556edfb26c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38128/0 (socket says 172.21.15.105:38128) 2024-03-26T17:46:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb26700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 -- 172.21.15.105:0/366667879 learned_addr learned my addr 172.21.15.105:0/366667879 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486dd9b640 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 0x556edfb2f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 -- 172.21.15.105:0/366667879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 msgr2=0x556edfb2f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 0x556edfb2f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 -- 172.21.15.105:0/366667879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 msgr2=0x556edfb26700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb26700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 -- 172.21.15.105:0/366667879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556edfb14820 con 0x556edfb20100 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486e59c640 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb26700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f486ed9d640 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb20100 0x556edfb26c40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f486800da00 tx=0x7f4868019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c884a55fd70e45f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4857fff640 1 -- 172.21.15.105:0/366667879 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486801db40 con 0x556edfb20100 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4857fff640 1 -- 172.21.15.105:0/366667879 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486802c070 con 0x556edfb20100 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.499+0000 7f4857fff640 1 -- 172.21.15.105:0/366667879 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868027400 con 0x556edfb20100 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 -- 172.21.15.105:0/366667879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb20100 msgr2=0x556edfb26c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb20100 0x556edfb26c40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f486800da00 tx=0x7f4868019000 comp rx=0 tx=0).stop 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 -- 172.21.15.105:0/366667879 shutdown_connections 2024-03-26T17:46:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb20100 0x556edfb26c40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb23590 0x556edfb2f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 --2- 172.21.15.105:0/366667879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb1da30 0x556edfb26700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 -- 172.21.15.105:0/366667879 >> 172.21.15.105:0/366667879 conn(0x556edfb18fa0 msgr2=0x556edfb1f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 -- 172.21.15.105:0/366667879 shutdown_connections 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 -- 172.21.15.105:0/366667879 wait complete. 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.500+0000 7f4877a2b740 1 -- start start 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4877a2b740 1 Processor -- start 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1da30 0x556edfb2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb20100 0x556edfb2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4877a2b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb2db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4877a2b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f48701ac450 con 0x556edfb23590 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4877a2b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48701abd80 con 0x556edfb20100 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4877a2b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48701a8fc0 con 0x556edfb1da30 2024-03-26T17:46:08.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4877a2b740 10 client.0 _ll_register_callbacks cb 0x7f4860009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb20100 0x556edfb2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb2db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb20100 0x556edfb2cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53306/0 (socket says 172.21.15.105:53306) 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486dd9b640 1 -- 172.21.15.105:0/2350977347 learned_addr learned my addr 172.21.15.105:0/2350977347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486ed9d640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb2db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38140/0 (socket says 172.21.15.105:38140) 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486e59c640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1da30 0x556edfb2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486ed9d640 1 -- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1da30 msgr2=0x556edfb2c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486ed9d640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1da30 0x556edfb2c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486ed9d640 1 -- 172.21.15.105:0/2350977347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb20100 msgr2=0x556edfb2cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486ed9d640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb20100 0x556edfb2cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486ed9d640 1 -- 172.21.15.105:0/2350977347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556edfb2e110 con 0x556edfb23590 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486e59c640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1da30 0x556edfb2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486dd9b640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb20100 0x556edfb2cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f486ed9d640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb2db20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4868006010 tx=0x7f4868009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9505034d2425bfa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.501+0000 7f4855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4855ffb640 1 -- 172.21.15.105:0/2350977347 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486801d940 con 0x556edfb23590 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4855ffb640 1 -- 172.21.15.105:0/2350977347 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f486801dae0 con 0x556edfb23590 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 1 -- 172.21.15.105:0/2350977347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486802c070 con 0x556edfb23590 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4855ffb640 1 -- 172.21.15.105:0/2350977347 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f486801dce0 con 0x556edfb23590 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 10 client.18276 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 1 -- 172.21.15.105:0/2350977347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556edfb2f410 con 0x556edfb23590 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4855ffb640 1 -- 172.21.15.105:0/2350977347 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486802c070 con 0x556edfb23590 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4855ffb640 1 -- 172.21.15.105:0/2350977347 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f486800b040 con 0x556edfb23590 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 10 client.18276 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 10 client.18276 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 20 client.18276 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x556edfb9a280 0x556edfb9c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 4 client.18276 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f4877a2b740 1 -- 172.21.15.105:0/2350977347 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556edfa076a0 con 0x556edfb9a280 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.502+0000 7f486e59c640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x556edfb9a280 0x556edfb9c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f486e59c640 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x556edfb9a280 0x556edfb9c6a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f50a2797074de0c server_cookie=bcbd45d86976c8df in_seq=0 out_seq=0 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4855ffb640 10 client.18276 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4855ffb640 1 -- 172.21.15.105:0/2350977347 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556edfa076a0 con 0x556edfb9a280 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4855ffb640 10 client.18276 handle_command_reply: tid=0 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 client.18276 shutdown 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 -- 172.21.15.105:0/2350977347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 msgr2=0x556edfb2db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb2db20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4868006010 tx=0x7f4868009840 comp rx=0 tx=0).stop 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 -- 172.21.15.105:0/2350977347 shutdown_connections 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x556edfb9a280 0x556edfb9c6a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edfb23590 0x556edfb2db20 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edfb20100 0x556edfb2cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 --2- 172.21.15.105:0/2350977347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edfb1da30 0x556edfb2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 -- 172.21.15.105:0/2350977347 >> 172.21.15.105:0/2350977347 conn(0x556edfb18fa0 msgr2=0x556edfb27ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4855ffb640 0 client.18276 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:46:08.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 -- 172.21.15.105:0/2350977347 shutdown_connections 2024-03-26T17:46:08.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 1 -- 172.21.15.105:0/2350977347 wait complete. 2024-03-26T17:46:08.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.503+0000 7f4877a2b740 20 client.18276 trim_cache size 0 max 16384 2024-03-26T17:46:08.504 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 -- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 msgr2=0x7f48580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4868009c50 tx=0x7f4868009be0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 -- 172.21.15.105:0/454047438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 msgr2=0x7f48701980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 0x7f48701980b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4864005940 tx=0x7f48640042f0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 -- 172.21.15.105:0/454047438 shutdown_connections 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48701075a0 0x7f487019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4870104c70 0x7f48701980b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 --2- 172.21.15.105:0/454047438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4870102340 0x7f4870197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 -- 172.21.15.105:0/454047438 >> 172.21.15.105:0/454047438 conn(0x7f48700fbd50 msgr2=0x7f4870102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 -- 172.21.15.105:0/454047438 shutdown_connections 2024-03-26T17:46:08.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.504+0000 7f4874e44640 1 -- 172.21.15.105:0/454047438 wait complete. 2024-03-26T17:46:08.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:46:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 -- 172.21.15.105:0/1107620195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 msgr2=0x7fe2b8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1107620195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 0x7fe2b8100be0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009900 tx=0x7fe2a802bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 -- 172.21.15.105:0/1107620195 shutdown_connections 2024-03-26T17:46:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1107620195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2b8102370 0x7fe2b8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1107620195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b81019b0 0x7fe2b8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1107620195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 0x7fe2b8100be0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 -- 172.21.15.105:0/1107620195 >> 172.21.15.105:0/1107620195 conn(0x7fe2b80fbd50 msgr2=0x7fe2b80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 -- 172.21.15.105:0/1107620195 shutdown_connections 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 -- 172.21.15.105:0/1107620195 wait complete. 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.630+0000 7fe2bccff640 1 Processor -- start 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2bccff640 1 -- start start 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2bccff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 0x7fe2b8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2bccff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2b81019b0 0x7fe2b819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2bccff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b8102370 0x7fe2b819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2bccff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810c2c0 con 0x7fe2b81007c0 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2bccff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810c140 con 0x7fe2b81019b0 2024-03-26T17:46:08.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2bccff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b810c440 con 0x7fe2b8102370 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 0x7fe2b8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 0x7fe2b8069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38148/0 (socket says 172.21.15.105:38148) 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 -- 172.21.15.105:0/1721186428 learned_addr learned my addr 172.21.15.105:0/1721186428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2b81019b0 0x7fe2b819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b6d9d640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b8102370 0x7fe2b819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 -- 172.21.15.105:0/1721186428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b8102370 msgr2=0x7fe2b819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b8102370 0x7fe2b819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 -- 172.21.15.105:0/1721186428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2b81019b0 msgr2=0x7fe2b819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2b81019b0 0x7fe2b819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 -- 172.21.15.105:0/1721186428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b819f7b0 con 0x7fe2b81007c0 2024-03-26T17:46:08.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2b659c640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 0x7fe2b8069590 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.631+0000 7fe2a77fe640 1 -- 172.21.15.105:0/1721186428 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8043020 con 0x7fe2b81007c0 2024-03-26T17:46:08.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.632+0000 7fe2a77fe640 1 -- 172.21.15.105:0/1721186428 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a8004350 con 0x7fe2b81007c0 2024-03-26T17:46:08.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.632+0000 7fe2bccff640 1 -- 172.21.15.105:0/1721186428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b819f9e0 con 0x7fe2b81007c0 2024-03-26T17:46:08.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.632+0000 7fe2a77fe640 1 -- 172.21.15.105:0/1721186428 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8003b80 con 0x7fe2b81007c0 2024-03-26T17:46:08.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.632+0000 7fe2bccff640 1 -- 172.21.15.105:0/1721186428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b819ff30 con 0x7fe2b81007c0 2024-03-26T17:46:08.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.634+0000 7fe2a77fe640 1 -- 172.21.15.105:0/1721186428 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe2a8046020 con 0x7fe2b81007c0 2024-03-26T17:46:08.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.634+0000 7fe2a77fe640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2a00a2f20 0x7fe2a00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.634+0000 7fe2bf9c7740 1 Processor -- start 2024-03-26T17:46:08.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.634+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2a00a2f20 0x7fe2a00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.634+0000 7fe2a77fe640 1 -- 172.21.15.105:0/1721186428 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe2a80f1580 con 0x7fe2b81007c0 2024-03-26T17:46:08.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.634+0000 7fe2bf9c7740 1 -- start start 2024-03-26T17:46:08.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x56284445ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 0x562844461f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x562844464a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5628443c1970 con 0x56284445ca30 2024-03-26T17:46:08.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562844369d70 con 0x562844462590 2024-03-26T17:46:08.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628443bd920 con 0x56284445d420 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x56284445ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x56284445ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38156/0 (socket says 172.21.15.105:38156) 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 -- 172.21.15.105:0/1627160009 learned_addr learned my addr 172.21.15.105:0/1627160009 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b6d9d640 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x562844464a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 0x562844461f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2a00a2f20 0x7fe2a00a53e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8069300 tx=0x7fe2ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 -- 172.21.15.105:0/1627160009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 msgr2=0x562844461f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 0x562844461f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 -- 172.21.15.105:0/1627160009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 msgr2=0x562844464a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x562844464a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 -- 172.21.15.105:0/1627160009 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628440d53f0 con 0x56284445ca30 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2b659c640 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x56284445ce50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80bdc00 tx=0x7fe2a80bd7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e06a3df33c72df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.635+0000 7fe2a4ff9640 1 -- 172.21.15.105:0/1627160009 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80bdc70 con 0x56284445ca30 2024-03-26T17:46:08.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.636+0000 7fe2a4ff9640 1 -- 172.21.15.105:0/1627160009 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a80c1dc0 con 0x56284445ca30 2024-03-26T17:46:08.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.636+0000 7fe2a4ff9640 1 -- 172.21.15.105:0/1627160009 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80ba2e0 con 0x56284445ca30 2024-03-26T17:46:08.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.636+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/1627160009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 msgr2=0x56284445ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.636+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x56284445ce50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80bdc00 tx=0x7fe2a80bd7b0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/1627160009 shutdown_connections 2024-03-26T17:46:08.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x562844464a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 0x562844461f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/1627160009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x56284445ce50 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/1627160009 >> 172.21.15.105:0/1627160009 conn(0x562844457fa0 msgr2=0x5628444583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/1627160009 shutdown_connections 2024-03-26T17:46:08.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/1627160009 wait complete. 2024-03-26T17:46:08.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 -- start start 2024-03-26T17:46:08.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 Processor -- start 2024-03-26T17:46:08.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x562844466c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 0x5628444671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x56284446e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562844465300 con 0x56284445ca30 2024-03-26T17:46:08.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56284437d310 con 0x562844462590 2024-03-26T17:46:08.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562844465480 con 0x56284445d420 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2bf9c7740 10 client.0 _ll_register_callbacks cb 0x5628443b8c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2b659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x562844466c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2b659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x562844466c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38164/0 (socket says 172.21.15.105:38164) 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2b659c640 1 -- 172.21.15.105:0/2903628794 learned_addr learned my addr 172.21.15.105:0/2903628794 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2b6d9d640 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x56284446e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.637+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 0x5628444671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2b659c640 1 -- 172.21.15.105:0/2903628794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 msgr2=0x5628444671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2b659c640 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 0x5628444671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2b659c640 1 -- 172.21.15.105:0/2903628794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 msgr2=0x56284446e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2b659c640 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x56284446e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2b659c640 1 -- 172.21.15.105:0/2903628794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56284419f7d0 con 0x56284445ca30 2024-03-26T17:46:08.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2b659c640 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x562844466c80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80ef090 tx=0x7fe2a8003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e10b1d07bd2e153 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe29dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:08.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe29dffb640 1 -- 172.21.15.105:0/2903628794 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80bdc70 con 0x56284445ca30 2024-03-26T17:46:08.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe29dffb640 1 -- 172.21.15.105:0/2903628794 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe2a80c61f0 con 0x56284445ca30 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2903628794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a80c1dc0 con 0x56284445ca30 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe29dffb640 1 -- 172.21.15.105:0/2903628794 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe2a80c6410 con 0x56284445ca30 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2bf9c7740 10 client.18288 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2903628794 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562844465300 con 0x56284445ca30 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe29dffb640 1 -- 172.21.15.105:0/2903628794 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2a80c1dc0 con 0x56284445ca30 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe29dffb640 1 -- 172.21.15.105:0/2903628794 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe2a80b9bc0 con 0x56284445ca30 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2bf9c7740 10 client.18288 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2bf9c7740 10 client.18288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.638+0000 7fe2bf9c7740 20 client.18288 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.639+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562844472a60 0x562844474ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.639+0000 7fe2bf9c7740 4 client.18288 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.639+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2903628794 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628443ceeb0 con 0x562844472a60 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.639+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562844472a60 0x562844474ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.639+0000 7fe29dffb640 10 client.18288 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.639+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562844472a60 0x562844474ea0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fdd1058b5466231 server_cookie=8c5a1b2dadf67f29 in_seq=0 out_seq=0 2024-03-26T17:46:08.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.640+0000 7fe29dffb640 1 -- 172.21.15.105:0/2903628794 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5628443ceeb0 con 0x562844472a60 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.640+0000 7fe29dffb640 10 client.18288 handle_command_reply: tid=0 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 client.18288 shutdown 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2903628794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 msgr2=0x562844466c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x562844466c80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80ef090 tx=0x7fe2a8003660 comp rx=0 tx=0).stop 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2903628794 shutdown_connections 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562844472a60 0x562844474ea0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x56284446e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445d420 0x5628444671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe29dffb640 0 client.18288 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2903628794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445ca30 0x562844466c80 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2903628794 >> 172.21.15.105:0/2903628794 conn(0x562844457fa0 msgr2=0x5628444571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2903628794 shutdown_connections 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2903628794 wait complete. 2024-03-26T17:46:08.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.641+0000 7fe2bf9c7740 20 client.18288 trim_cache size 0 max 16384 2024-03-26T17:46:08.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2bf9c7740 1 Processor -- start 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2bf9c7740 1 -- start start 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 0x562844466fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56284445d420 0x56284446e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562844462590 0x562844456930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81a8d20 con 0x562844462590 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81aa860 con 0x56284445d420 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81a93f0 con 0x56284445ca30 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562844462590 0x562844456930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562844462590 0x562844456930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38168/0 (socket says 172.21.15.105:38168) 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b6d9d640 1 -- 172.21.15.105:0/2487494300 learned_addr learned my addr 172.21.15.105:0/2487494300 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b659c640 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 0x562844466fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56284445d420 0x56284446e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b6d9d640 1 -- 172.21.15.105:0/2487494300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 msgr2=0x562844466fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b6d9d640 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 0x562844466fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b6d9d640 1 -- 172.21.15.105:0/2487494300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56284445d420 msgr2=0x56284446e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b6d9d640 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56284445d420 0x56284446e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.667+0000 7fe2b6d9d640 1 -- 172.21.15.105:0/2487494300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56284445dc60 con 0x562844462590 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe2b6d9d640 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562844462590 0x562844456930 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0003cf0 tx=0x7fe2b0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b813f4826be63f3b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe29ffff640 1 -- 172.21.15.105:0/2487494300 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0020070 con 0x562844462590 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe29ffff640 1 -- 172.21.15.105:0/2487494300 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b00108e0 con 0x562844462590 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe29ffff640 1 -- 172.21.15.105:0/2487494300 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b001ba00 con 0x562844462590 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2487494300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562844462590 msgr2=0x562844456930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562844462590 0x562844456930 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0003cf0 tx=0x7fe2b0004410 comp rx=0 tx=0).stop 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2487494300 shutdown_connections 2024-03-26T17:46:08.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562844462590 0x562844456930 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56284445d420 0x56284446e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/2487494300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 0x562844466fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.668+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2487494300 >> 172.21.15.105:0/2487494300 conn(0x562844457fa0 msgr2=0x562844466890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2487494300 shutdown_connections 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/2487494300 wait complete. 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 -- start start 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 Processor -- start 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 0x56284446b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445d420 0x56284446bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x56284446cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81aa190 con 0x56284445d420 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81a9ac0 con 0x562844462590 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b81a6d00 con 0x56284445ca30 2024-03-26T17:46:08.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2bf9c7740 10 client.0 _ll_register_callbacks cb 0x7fe2b0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2b5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445d420 0x56284446bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2b5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445d420 0x56284446bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38182/0 (socket says 172.21.15.105:38182) 2024-03-26T17:46:08.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2b5d9b640 1 -- 172.21.15.105:0/3170065343 learned_addr learned my addr 172.21.15.105:0/3170065343 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2b6d9d640 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x56284446cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.669+0000 7fe2b659c640 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 0x56284446b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2b5d9b640 1 -- 172.21.15.105:0/3170065343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 msgr2=0x56284446b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 0x56284446b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2b5d9b640 1 -- 172.21.15.105:0/3170065343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 msgr2=0x56284446cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x56284446cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2b5d9b640 1 -- 172.21.15.105:0/3170065343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56284446d3b0 con 0x56284445d420 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2b5d9b640 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445d420 0x56284446bdb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac004650 tx=0x7fe2ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390317520fd4717b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe29dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe29dffb640 1 -- 172.21.15.105:0/3170065343 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0055d0 con 0x56284445d420 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe29dffb640 1 -- 172.21.15.105:0/3170065343 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe2ac02b0e0 con 0x56284445d420 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/3170065343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b00108e0 con 0x56284445d420 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe29dffb640 1 -- 172.21.15.105:0/3170065343 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe2ac03b050 con 0x56284445d420 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2bf9c7740 10 client.18294 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/3170065343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5628444d8ed0 con 0x56284445d420 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe29dffb640 1 -- 172.21.15.105:0/3170065343 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2b00108e0 con 0x56284445d420 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.670+0000 7fe29dffb640 1 -- 172.21.15.105:0/3170065343 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe2ac01c960 con 0x56284445d420 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe2bf9c7740 10 client.18294 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe2bf9c7740 10 client.18294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe2bf9c7740 20 client.18294 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628444d97d0 0x5628444dbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe2bf9c7740 4 client.18294 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/3170065343 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628443c9200 con 0x5628444d97d0 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe2b659c640 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628444d97d0 0x5628444dbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe29dffb640 10 client.18294 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.671+0000 7fe2b659c640 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628444d97d0 0x5628444dbbf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef9827e21e8e2ae server_cookie=fcb9f340b53e6929 in_seq=0 out_seq=0 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe29dffb640 1 -- 172.21.15.105:0/3170065343 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628443c9200 con 0x5628444d97d0 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe29dffb640 10 client.18294 handle_command_reply: tid=0 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 client.18294 shutdown 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/3170065343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445d420 msgr2=0x56284446bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445d420 0x56284446bdb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac004650 tx=0x7fe2ac004730 comp rx=0 tx=0).stop 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/3170065343 shutdown_connections 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628444d97d0 0x5628444dbbf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562844462590 0x56284446cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56284445d420 0x56284446bdb0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe29dffb640 0 client.18294 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 --2- 172.21.15.105:0/3170065343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56284445ca30 0x56284446b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.672+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/3170065343 >> 172.21.15.105:0/3170065343 conn(0x562844457fa0 msgr2=0x56284445b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/3170065343 shutdown_connections 2024-03-26T17:46:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bf9c7740 1 -- 172.21.15.105:0/3170065343 wait complete. 2024-03-26T17:46:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bf9c7740 20 client.18294 trim_cache size 0 max 16384 2024-03-26T17:46:08.673 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 -- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2a00a2f20 msgr2=0x7fe2a00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2a00a2f20 0x7fe2a00a53e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8069300 tx=0x7fe2ac005f90 comp rx=0 tx=0).stop 2024-03-26T17:46:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 -- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 msgr2=0x7fe2b8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 0x7fe2b8069590 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8000c00 tx=0x7fe2a8005dc0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 -- 172.21.15.105:0/1721186428 shutdown_connections 2024-03-26T17:46:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2a00a2f20 0x7fe2a00a53e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2b8102370 0x7fe2b819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe2b81019b0 0x7fe2b819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 --2- 172.21.15.105:0/1721186428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2b81007c0 0x7fe2b8069590 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 -- 172.21.15.105:0/1721186428 >> 172.21.15.105:0/1721186428 conn(0x7fe2b80fbd50 msgr2=0x7fe2b8104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 -- 172.21.15.105:0/1721186428 shutdown_connections 2024-03-26T17:46:08.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:08.673+0000 7fe2bccff640 1 -- 172.21.15.105:0/1721186428 wait complete. 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 -- 172.21.15.105:0/2891865893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 msgr2=0x7f1ff4101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 --2- 172.21.15.105:0/2891865893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 0x7f1ff4101e90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009900 tx=0x7f1fe802bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 -- 172.21.15.105:0/2891865893 shutdown_connections 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 --2- 172.21.15.105:0/2891865893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff41023d0 0x7f1ff4108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 --2- 172.21.15.105:0/2891865893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 0x7f1ff4101e90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 --2- 172.21.15.105:0/2891865893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ff4100820 0x7f1ff4100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 -- 172.21.15.105:0/2891865893 >> 172.21.15.105:0/2891865893 conn(0x7f1ff40fbd90 msgr2=0x7f1ff40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 -- 172.21.15.105:0/2891865893 shutdown_connections 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.864+0000 7f1ff9777640 1 -- 172.21.15.105:0/2891865893 wait complete. 2024-03-26T17:46:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff9777640 1 Processor -- start 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff9777640 1 -- start start 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff9777640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ff4100820 0x7f1ff4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff9777640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 0x7f1ff4197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff9777640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff41023d0 0x7f1ff419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff9777640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410c3a0 con 0x7f1ff4101a10 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff9777640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410c220 con 0x7f1ff4100820 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff9777640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410c520 con 0x7f1ff41023d0 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 0x7f1ff4197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 0x7f1ff4197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38198/0 (socket says 172.21.15.105:38198) 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff27fc640 1 -- 172.21.15.105:0/1198889696 learned_addr learned my addr 172.21.15.105:0/1198889696 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.865+0000 7f1ff2ffd640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ff4100820 0x7f1ff4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff41023d0 0x7f1ff419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff27fc640 1 -- 172.21.15.105:0/1198889696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff41023d0 msgr2=0x7f1ff419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff41023d0 0x7f1ff419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff27fc640 1 -- 172.21.15.105:0/1198889696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ff4100820 msgr2=0x7f1ff4197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ff4100820 0x7f1ff4197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff27fc640 1 -- 172.21.15.105:0/1198889696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff419f6a0 con 0x7f1ff4101a10 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff2ffd640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ff4100820 0x7f1ff4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 0x7f1ff4197fc0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8000c00 tx=0x7f1fe80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1fdbfff640 1 -- 172.21.15.105:0/1198889696 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8043020 con 0x7f1ff4101a10 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff9777640 1 -- 172.21.15.105:0/1198889696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff419f930 con 0x7f1ff4101a10 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1fdbfff640 1 -- 172.21.15.105:0/1198889696 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fe80044c0 con 0x7f1ff4101a10 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1fdbfff640 1 -- 172.21.15.105:0/1198889696 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8003c60 con 0x7f1ff4101a10 2024-03-26T17:46:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.866+0000 7f1ff9777640 1 -- 172.21.15.105:0/1198889696 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff419fed0 con 0x7f1ff4101a10 2024-03-26T17:46:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1fdbfff640 1 -- 172.21.15.105:0/1198889696 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1fe8046020 con 0x7f1ff4101a10 2024-03-26T17:46:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ffbffd740 1 Processor -- start 2024-03-26T17:46:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ffbffd740 1 -- start start 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c3a40a0b0 con 0x558c3a4a5fc0 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c3a3b2960 con 0x558c3a4ab130 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c3a406060 con 0x558c3a4a55d0 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ff2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ff37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ff37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41014/0 (socket says 172.21.15.105:41014) 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ff37fe640 1 -- 172.21.15.105:0/810405986 learned_addr learned my addr 172.21.15.105:0/810405986 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.869+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/810405986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1fdbfff640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1fdc0a2fc0 0x7f1fdc0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1ff37fe640 1 -- 172.21.15.105:0/810405986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 msgr2=0x558c3a4ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/810405986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1ff37fe640 1 -- 172.21.15.105:0/810405986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 msgr2=0x558c3a4aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/810405986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4aaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1ff37fe640 1 -- 172.21.15.105:0/810405986 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c3a3c4c60 con 0x558c3a4a55d0 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1fdbfff640 1 -- 172.21.15.105:0/1198889696 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1fe80f1640 con 0x7f1ff4101a10 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1fdc0a2fc0 0x7f1fdc0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/810405986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4a59f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009fd0 tx=0x7f1fec0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e64d4e9ea19793af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1fd97fa640 1 -- 172.21.15.105:0/810405986 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec004050 con 0x558c3a4a55d0 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1fd97fa640 1 -- 172.21.15.105:0/810405986 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fec003e70 con 0x558c3a4a55d0 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.870+0000 7f1fd97fa640 1 -- 172.21.15.105:0/810405986 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00fe80 con 0x558c3a4a55d0 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 -- 172.21.15.105:0/810405986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 msgr2=0x558c3a4a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/810405986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4a59f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009fd0 tx=0x7f1fec0118d0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1fdc0a2fc0 0x7f1fdc0a5480 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1fec01d000 tx=0x7f1fec019640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 -- 172.21.15.105:0/810405986 shutdown_connections 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/810405986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/810405986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/810405986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4a59f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 -- 172.21.15.105:0/810405986 >> 172.21.15.105:0/810405986 conn(0x558c3a4a0b40 msgr2=0x558c3a4a0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 -- 172.21.15.105:0/810405986 shutdown_connections 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 -- 172.21.15.105:0/810405986 wait complete. 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 -- start start 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.871+0000 7f1ffbffd740 1 Processor -- start 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4b6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c3a4ae0e0 con 0x558c3a4a5fc0 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c3a3c5db0 con 0x558c3a4ab130 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c3a4ae260 con 0x558c3a4a55d0 2024-03-26T17:46:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ffbffd740 10 client.0 _ll_register_callbacks cb 0x558c3a3ca110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41016/0 (socket says 172.21.15.105:41016) 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff2ffd640 1 -- 172.21.15.105:0/3396973551 learned_addr learned my addr 172.21.15.105:0/3396973551 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38220/0 (socket says 172.21.15.105:38220) 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff27fc640 1 -- 172.21.15.105:0/3396973551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 msgr2=0x558c3a4af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff27fc640 1 -- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 msgr2=0x558c3a4b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4b6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff27fc640 1 -- 172.21.15.105:0/3396973551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3a11e210 con 0x558c3a4a5fc0 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff2ffd640 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4af830 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80ef150 tx=0x7f1fe8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a071238845e58f13 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.872+0000 7f1fd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/3396973551 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80bd690 con 0x558c3a4a5fc0 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/3396973551 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1fe80bc030 con 0x558c3a4a5fc0 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1ffbffd740 1 -- 172.21.15.105:0/3396973551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fec003e70 con 0x558c3a4a5fc0 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/3396973551 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1fe80035f0 con 0x558c3a4a5fc0 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1ffbffd740 10 client.18306 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1ffbffd740 1 -- 172.21.15.105:0/3396973551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c3a1ea620 con 0x558c3a4a5fc0 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/3396973551 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fec003e70 con 0x558c3a4a5fc0 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/3396973551 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1fe80c62b0 con 0x558c3a4a5fc0 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1ffbffd740 10 client.18306 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.873+0000 7f1ffbffd740 10 client.18306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.874+0000 7f1ffbffd740 20 client.18306 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.874+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c3a4bb120 0x558c3a4bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.874+0000 7f1ffbffd740 4 client.18306 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.874+0000 7f1ffbffd740 1 -- 172.21.15.105:0/3396973551 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c3a4175f0 con 0x558c3a4bb120 2024-03-26T17:46:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.874+0000 7f1ff2ffd640 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c3a4bb120 0x558c3a4bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.874+0000 7f1fd6ff5640 10 client.18306 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.874+0000 7f1ff2ffd640 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c3a4bb120 0x558c3a4bd5a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=befa7475de00dd32 server_cookie=ff6e3251399e1834 in_seq=0 out_seq=0 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/3396973551 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558c3a4175f0 con 0x558c3a4bb120 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1fd6ff5640 10 client.18306 handle_command_reply: tid=0 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 client.18306 shutdown 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 -- 172.21.15.105:0/3396973551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 msgr2=0x558c3a4af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4af830 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80ef150 tx=0x7f1fe8005b50 comp rx=0 tx=0).stop 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 -- 172.21.15.105:0/3396973551 shutdown_connections 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c3a4bb120 0x558c3a4bd5a0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4ab130 0x558c3a4b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4af830 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/3396973551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a55d0 0x558c3a4af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 -- 172.21.15.105:0/3396973551 >> 172.21.15.105:0/3396973551 conn(0x558c3a4a0b40 msgr2=0x558c3a49fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1fd6ff5640 0 client.18306 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 -- 172.21.15.105:0/3396973551 shutdown_connections 2024-03-26T17:46:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 1 -- 172.21.15.105:0/3396973551 wait complete. 2024-03-26T17:46:08.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.875+0000 7f1ffbffd740 20 client.18306 trim_cache size 0 max 16384 2024-03-26T17:46:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ffbffd740 1 Processor -- start 2024-03-26T17:46:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ffbffd740 1 -- start start 2024-03-26T17:46:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 0x558c3a4a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4ab130 0x558c3a49f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41a6a90 con 0x558c3a4ab130 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41a85d0 con 0x558c3a4a55d0 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41a7160 con 0x558c3a4a5fc0 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff27fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4ab130 0x558c3a49f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 0x558c3a4a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff27fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41018/0 (socket says 172.21.15.105:41018) 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff27fc640 1 -- 172.21.15.105:0/2629690738 learned_addr learned my addr 172.21.15.105:0/2629690738 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff27fc640 1 -- 172.21.15.105:0/2629690738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 msgr2=0x558c3a4a70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/2629690738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 0x558c3a4a70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff27fc640 1 -- 172.21.15.105:0/2629690738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4ab130 msgr2=0x558c3a49f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/2629690738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4ab130 0x558c3a49f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.895+0000 7f1ff27fc640 1 -- 172.21.15.105:0/2629690738 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c3a4a53c0 con 0x558c3a4a5fc0 2024-03-26T17:46:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.896+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/2629690738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b6830 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8005ec0 tx=0x7f1fe8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aba84bb5bd66803b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.896+0000 7f1fd8ff9640 1 -- 172.21.15.105:0/2629690738 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80bd690 con 0x558c3a4a5fc0 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.896+0000 7f1fd8ff9640 1 -- 172.21.15.105:0/2629690738 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fe80be430 con 0x558c3a4a5fc0 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.896+0000 7f1fd8ff9640 1 -- 172.21.15.105:0/2629690738 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80b9a80 con 0x558c3a4a5fc0 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2629690738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a5fc0 msgr2=0x558c3a4b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2629690738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b6830 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8005ec0 tx=0x7f1fe8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2629690738 shutdown_connections 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2629690738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4ab130 0x558c3a49f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2629690738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b6830 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2629690738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 0x558c3a4a70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2629690738 >> 172.21.15.105:0/2629690738 conn(0x558c3a4a0b40 msgr2=0x558c3a4a69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2629690738 shutdown_connections 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2629690738 wait complete. 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- start start 2024-03-26T17:46:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 Processor -- start 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 0x558c3a4b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4ab130 0x558c3a51f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41a7f00 con 0x558c3a4a5fc0 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41a7830 con 0x558c3a4a55d0 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41a4a70 con 0x558c3a4ab130 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ffbffd740 10 client.0 _ll_register_callbacks cb 0x7f1fe80de050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ff27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.897+0000 7f1ff27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38240/0 (socket says 172.21.15.105:38240) 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff27fc640 1 -- 172.21.15.105:0/2250833809 learned_addr learned my addr 172.21.15.105:0/2250833809 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff2ffd640 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 0x558c3a4b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff37fe640 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4ab130 0x558c3a51f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff27fc640 1 -- 172.21.15.105:0/2250833809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4ab130 msgr2=0x558c3a51f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4ab130 0x558c3a51f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff27fc640 1 -- 172.21.15.105:0/2250833809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 msgr2=0x558c3a4b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 0x558c3a4b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff27fc640 1 -- 172.21.15.105:0/2250833809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c3a4b6a30 con 0x558c3a4a5fc0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ff27fc640 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b3960 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80f1ac0 tx=0x7f1fe80f1e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1c62eafad39bf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1fd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/2250833809 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80bd690 con 0x558c3a4a5fc0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/2250833809 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1fe80bd390 con 0x558c3a4a5fc0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2250833809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe80be430 con 0x558c3a4a5fc0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/2250833809 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1fe80bdd30 con 0x558c3a4a5fc0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ffbffd740 10 client.18312 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2250833809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c3a4b6a30 con 0x558c3a4a5fc0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.898+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/2250833809 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fe80be430 con 0x558c3a4a5fc0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/2250833809 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1fe809ec60 con 0x558c3a4a5fc0 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1ffbffd740 10 client.18312 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1ffbffd740 10 client.18312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1ffbffd740 20 client.18312 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c3a520fb0 0x558c3a5233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1ffbffd740 4 client.18312 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:46:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2250833809 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c3a411940 con 0x558c3a520fb0 2024-03-26T17:46:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1ff2ffd640 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c3a520fb0 0x558c3a5233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1fd6ff5640 10 client.18312 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.899+0000 7f1ff2ffd640 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c3a520fb0 0x558c3a5233d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf972fb9bf22cb7b server_cookie=6b6d4df77c7fe67b in_seq=0 out_seq=0 2024-03-26T17:46:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1fd6ff5640 1 -- 172.21.15.105:0/2250833809 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c3a411940 con 0x558c3a520fb0 2024-03-26T17:46:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1fd6ff5640 10 client.18312 handle_command_reply: tid=0 2024-03-26T17:46:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 client.18312 shutdown 2024-03-26T17:46:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2250833809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 msgr2=0x558c3a4b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b3960 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80f1ac0 tx=0x7f1fe80f1e50 comp rx=0 tx=0).stop 2024-03-26T17:46:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2250833809 shutdown_connections 2024-03-26T17:46:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c3a520fb0 0x558c3a5233d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c3a4ab130 0x558c3a51f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1fd6ff5640 0 client.18312 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c3a4a5fc0 0x558c3a4b3960 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 --2- 172.21.15.105:0/2250833809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c3a4a55d0 0x558c3a4b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2250833809 >> 172.21.15.105:0/2250833809 conn(0x558c3a4a0b40 msgr2=0x558c3a4a3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2250833809 shutdown_connections 2024-03-26T17:46:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 1 -- 172.21.15.105:0/2250833809 wait complete. 2024-03-26T17:46:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.945+0000 7f1ffbffd740 20 client.18312 trim_cache size 0 max 16384 2024-03-26T17:46:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:46:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:46:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:46:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:46:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 -- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1fdc0a2fc0 msgr2=0x7f1fdc0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1fdc0a2fc0 0x7f1fdc0a5480 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1fec01d000 tx=0x7f1fec019640 comp rx=0 tx=0).stop 2024-03-26T17:46:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 -- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 msgr2=0x7f1ff4197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 0x7f1ff4197fc0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8000c00 tx=0x7f1fe80040c0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 -- 172.21.15.105:0/1198889696 shutdown_connections 2024-03-26T17:46:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1fdc0a2fc0 0x7f1fdc0a5480 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff41023d0 0x7f1ff419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ff4101a10 0x7f1ff4197fc0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 --2- 172.21.15.105:0/1198889696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ff4100820 0x7f1ff4197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 -- 172.21.15.105:0/1198889696 >> 172.21.15.105:0/1198889696 conn(0x7f1ff40fbd90 msgr2=0x7f1ff40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 -- 172.21.15.105:0/1198889696 shutdown_connections 2024-03-26T17:46:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:08.946+0000 7f1ff9777640 1 -- 172.21.15.105:0/1198889696 wait complete. 2024-03-26T17:46:09.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 242046 3.15 MB/sec execute 292 sec latency 28.203 ms 2024-03-26T17:46:09.678 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:46:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:46:09.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:46:09.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 -- 172.21.15.105:0/3934320994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 msgr2=0x7f5e38100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 --2- 172.21.15.105:0/3934320994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 0x7f5e38100be0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009900 tx=0x7f5e2802bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 -- 172.21.15.105:0/3934320994 shutdown_connections 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 --2- 172.21.15.105:0/3934320994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e38102370 0x7f5e38108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 --2- 172.21.15.105:0/3934320994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e381019b0 0x7f5e38101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 --2- 172.21.15.105:0/3934320994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 0x7f5e38100be0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 -- 172.21.15.105:0/3934320994 >> 172.21.15.105:0/3934320994 conn(0x7f5e380fbd50 msgr2=0x7f5e380fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 -- 172.21.15.105:0/3934320994 shutdown_connections 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.816+0000 7f5e3e368640 1 -- 172.21.15.105:0/3934320994 wait complete. 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e3e368640 1 Processor -- start 2024-03-26T17:46:09.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e3e368640 1 -- start start 2024-03-26T17:46:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e3e368640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 0x7f5e38069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e3e368640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e381019b0 0x7f5e3819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e3e368640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e38102370 0x7f5e3819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e3e368640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3810c2f0 con 0x7f5e381007c0 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e3e368640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3810c170 con 0x7f5e38102370 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e3e368640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3810c470 con 0x7f5e381019b0 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 0x7f5e38069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.817+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 0x7f5e38069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38258/0 (socket says 172.21.15.105:38258) 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e37fff640 1 -- 172.21.15.105:0/4075952447 learned_addr learned my addr 172.21.15.105:0/4075952447 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e3c905640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e38102370 0x7f5e3819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e377fe640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e381019b0 0x7f5e3819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e37fff640 1 -- 172.21.15.105:0/4075952447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e381019b0 msgr2=0x7f5e3819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e37fff640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e381019b0 0x7f5e3819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e37fff640 1 -- 172.21.15.105:0/4075952447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e38102370 msgr2=0x7f5e3819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e37fff640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e38102370 0x7f5e3819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e37fff640 1 -- 172.21.15.105:0/4075952447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3819f7d0 con 0x7f5e381007c0 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e37fff640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 0x7f5e38069590 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5e28000c00 tx=0x7f5e28005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e357fa640 1 -- 172.21.15.105:0/4075952447 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28043020 con 0x7f5e381007c0 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e357fa640 1 -- 172.21.15.105:0/4075952447 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e28004350 con 0x7f5e381007c0 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e3e368640 1 -- 172.21.15.105:0/4075952447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e38196670 con 0x7f5e381007c0 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e357fa640 1 -- 172.21.15.105:0/4075952447 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28003b80 con 0x7f5e381007c0 2024-03-26T17:46:09.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.818+0000 7f5e3e368640 1 -- 172.21.15.105:0/4075952447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e38196bc0 con 0x7f5e381007c0 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.821+0000 7f5e357fa640 1 -- 172.21.15.105:0/4075952447 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5e28046020 con 0x7f5e381007c0 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.821+0000 7f5e357fa640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e200a2f20 0x7f5e200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.821+0000 7f5e357fa640 1 -- 172.21.15.105:0/4075952447 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5e280f1550 con 0x7f5e381007c0 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.821+0000 7f5e377fe640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e200a2f20 0x7f5e200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.821+0000 7f5e4102a740 1 Processor -- start 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e4102a740 1 -- start start 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa487190 0x5626aa4875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e377fe640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e200a2f20 0x7f5e200a53e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e38069300 tx=0x7f5e2c012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa48c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa48ccf0 0x5626aa48f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa487190 0x5626aa4875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa48c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e4102a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5626aa3ec630 con 0x5626aa487b80 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e4102a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5626aa394e10 con 0x5626aa487190 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e4102a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626aa3e85e0 con 0x5626aa48ccf0 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa487190 0x5626aa4875b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53436/0 (socket says 172.21.15.105:53436) 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e37fff640 1 -- 172.21.15.105:0/3945984128 learned_addr learned my addr 172.21.15.105:0/3945984128 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e3c905640 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa48ccf0 0x5626aa48f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e377fe640 1 -- 172.21.15.105:0/3945984128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa48ccf0 msgr2=0x5626aa48f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e377fe640 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa48ccf0 0x5626aa48f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e377fe640 1 -- 172.21.15.105:0/3945984128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa487190 msgr2=0x5626aa4875b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e377fe640 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa487190 0x5626aa4875b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.822+0000 7f5e377fe640 1 -- 172.21.15.105:0/3945984128 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626aa3ab740 con 0x5626aa487b80 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.823+0000 7f5e37fff640 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa487190 0x5626aa4875b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:09.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.823+0000 7f5e377fe640 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa48c680 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c01c580 tx=0x7f5e2c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292c9d063b402344 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.823+0000 7f5e1e7fc640 1 -- 172.21.15.105:0/3945984128 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c02b070 con 0x5626aa487b80 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.823+0000 7f5e1e7fc640 1 -- 172.21.15.105:0/3945984128 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e2c004230 con 0x5626aa487b80 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.823+0000 7f5e1e7fc640 1 -- 172.21.15.105:0/3945984128 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c022040 con 0x5626aa487b80 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 -- 172.21.15.105:0/3945984128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 msgr2=0x5626aa48c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa48c680 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c01c580 tx=0x7f5e2c018000 comp rx=0 tx=0).stop 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 -- 172.21.15.105:0/3945984128 shutdown_connections 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa48ccf0 0x5626aa48f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa48c680 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 --2- 172.21.15.105:0/3945984128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa487190 0x5626aa4875b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 -- 172.21.15.105:0/3945984128 >> 172.21.15.105:0/3945984128 conn(0x5626aa482700 msgr2=0x5626aa482b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 -- 172.21.15.105:0/3945984128 shutdown_connections 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 -- 172.21.15.105:0/3945984128 wait complete. 2024-03-26T17:46:09.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 -- start start 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.824+0000 7f5e4102a740 1 Processor -- start 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa4916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa491c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa4916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e4102a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5626aa48fca0 con 0x5626aa487190 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa4916d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38262/0 (socket says 172.21.15.105:38262) 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 -- 172.21.15.105:0/696304570 learned_addr learned my addr 172.21.15.105:0/696304570 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5626aa3a81a0 con 0x5626aa48ccf0 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626aa48fe20 con 0x5626aa487b80 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e377fe640 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa491c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e4102a740 10 client.0 _ll_register_callbacks cb 0x5626aa0ffc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e3c905640 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 -- 172.21.15.105:0/696304570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 msgr2=0x5626aa491c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa491c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 -- 172.21.15.105:0/696304570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 msgr2=0x5626aa498c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 -- 172.21.15.105:0/696304570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626aa1c87e0 con 0x5626aa487190 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e377fe640 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa491c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e37fff640 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa4916d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5e280ef060 tx=0x7f5e280bd660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a75e5785a9cfcc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e177fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e177fe640 1 -- 172.21.15.105:0/696304570 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280c1d90 con 0x5626aa487190 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e177fe640 1 -- 172.21.15.105:0/696304570 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5e280054d0 con 0x5626aa487190 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.825+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e2c004230 con 0x5626aa487190 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e177fe640 1 -- 172.21.15.105:0/696304570 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e280c6650 con 0x5626aa487190 2024-03-26T17:46:09.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e4102a740 10 client.18324 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626aa228f70 con 0x5626aa487190 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e177fe640 1 -- 172.21.15.105:0/696304570 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e2c004230 con 0x5626aa487190 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e177fe640 1 -- 172.21.15.105:0/696304570 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5e280d4070 con 0x5626aa487190 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e4102a740 10 client.18324 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e4102a740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e4102a740 20 client.18324 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e4102a740 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626aa4a5ce0 0x5626aa4a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e4102a740 4 client.18324 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626aa3f9b70 con 0x5626aa4a5ce0 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.826+0000 7f5e377fe640 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626aa4a5ce0 0x5626aa4a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.827+0000 7f5e177fe640 10 client.18324 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.827+0000 7f5e377fe640 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626aa4a5ce0 0x5626aa4a8150 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c09a4664a018cc server_cookie=eed32bc31872f4e in_seq=0 out_seq=0 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.827+0000 7f5e177fe640 1 -- 172.21.15.105:0/696304570 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5626aa3f9b70 con 0x5626aa4a5ce0 2024-03-26T17:46:09.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.827+0000 7f5e177fe640 10 client.18324 handle_command_reply: tid=0 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.827+0000 7f5e4102a740 1 client.18324 shutdown 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.827+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 msgr2=0x5626aa4916d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.827+0000 7f5e4102a740 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa4916d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5e280ef060 tx=0x7f5e280bd660 comp rx=0 tx=0).stop 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 shutdown_connections 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626aa4a5ce0 0x5626aa4a8150 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e177fe640 0 client.18324 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa491c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 1 --2- 172.21.15.105:0/696304570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa4916d0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 >> 172.21.15.105:0/696304570 conn(0x5626aa482700 msgr2=0x5626aa481900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 shutdown_connections 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 1 -- 172.21.15.105:0/696304570 wait complete. 2024-03-26T17:46:09.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.828+0000 7f5e4102a740 20 client.18324 trim_cache size 0 max 16384 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.848+0000 7f5e4102a740 1 Processor -- start 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e4102a740 1 -- start start 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa491b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa498c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa486070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e4102a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381a6ca0 con 0x5626aa487190 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e4102a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381a87e0 con 0x5626aa48ccf0 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e4102a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381a7370 con 0x5626aa487b80 2024-03-26T17:46:09.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa491b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa491b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38264/0 (socket says 172.21.15.105:38264) 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 -- 172.21.15.105:0/3064657036 learned_addr learned my addr 172.21.15.105:0/3064657036 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e377fe640 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa498c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e3c905640 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa486070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 -- 172.21.15.105:0/3064657036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 msgr2=0x5626aa498c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa498c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 -- 172.21.15.105:0/3064657036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 msgr2=0x5626aa486070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa486070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 -- 172.21.15.105:0/3064657036 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626aa0c0170 con 0x5626aa487190 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.849+0000 7f5e37fff640 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa491b70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5e280ba610 tx=0x7f5e280045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a16d007373870a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e1dffb640 1 -- 172.21.15.105:0/3064657036 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280c1d90 con 0x5626aa487190 2024-03-26T17:46:09.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e1dffb640 1 -- 172.21.15.105:0/3064657036 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e280054d0 con 0x5626aa487190 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e1dffb640 1 -- 172.21.15.105:0/3064657036 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e280f1100 con 0x5626aa487190 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 -- 172.21.15.105:0/3064657036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 msgr2=0x5626aa491b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa491b70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5e280ba610 tx=0x7f5e280045b0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 -- 172.21.15.105:0/3064657036 shutdown_connections 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa486070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487b80 0x5626aa498c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 --2- 172.21.15.105:0/3064657036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487190 0x5626aa491b70 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 -- 172.21.15.105:0/3064657036 >> 172.21.15.105:0/3064657036 conn(0x5626aa482700 msgr2=0x5626aa485960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 -- 172.21.15.105:0/3064657036 shutdown_connections 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.850+0000 7f5e4102a740 1 -- 172.21.15.105:0/3064657036 wait complete. 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 1 -- start start 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 1 Processor -- start 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487190 0x5626aa496b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa4970d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381a8110 con 0x5626aa487b80 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381a7a40 con 0x5626aa48ccf0 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e381a4c80 con 0x5626aa487190 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e4102a740 10 client.0 _ll_register_callbacks cb 0x7f5e280bda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa4970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa4970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38270/0 (socket says 172.21.15.105:38270) 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e377fe640 1 -- 172.21.15.105:0/1490306999 learned_addr learned my addr 172.21.15.105:0/1490306999 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e37fff640 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487190 0x5626aa496b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.851+0000 7f5e3c905640 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e377fe640 1 -- 172.21.15.105:0/1490306999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487190 msgr2=0x5626aa496b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e377fe640 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487190 0x5626aa496b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e377fe640 1 -- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 msgr2=0x5626aa498040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e377fe640 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e377fe640 1 -- 172.21.15.105:0/1490306999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626aa498630 con 0x5626aa487b80 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e3c905640 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e377fe640 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa4970d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009d90 tx=0x7f5e2c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89501c982fcc6454 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e177fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e177fe640 1 -- 172.21.15.105:0/1490306999 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c035070 con 0x5626aa487b80 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e177fe640 1 -- 172.21.15.105:0/1490306999 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5e2c031bd0 con 0x5626aa487b80 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e4102a740 1 -- 172.21.15.105:0/1490306999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e280054d0 con 0x5626aa487b80 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e177fe640 1 -- 172.21.15.105:0/1490306999 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e2c022040 con 0x5626aa487b80 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e4102a740 10 client.18330 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e4102a740 1 -- 172.21.15.105:0/1490306999 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626aa50c900 con 0x5626aa487b80 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.852+0000 7f5e177fe640 1 -- 172.21.15.105:0/1490306999 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e2c041070 con 0x5626aa487b80 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e177fe640 1 -- 172.21.15.105:0/1490306999 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5e2c01a890 con 0x5626aa487b80 2024-03-26T17:46:09.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e4102a740 10 client.18330 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e4102a740 10 client.18330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e4102a740 20 client.18330 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e4102a740 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626aa50cfd0 0x5626aa50f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e4102a740 4 client.18330 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:46:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e4102a740 1 -- 172.21.15.105:0/1490306999 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5626aa3f3ec0 con 0x5626aa50cfd0 2024-03-26T17:46:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e37fff640 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626aa50cfd0 0x5626aa50f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e177fe640 10 client.18330 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.853+0000 7f5e37fff640 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626aa50cfd0 0x5626aa50f3f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff2d3899e7136ba server_cookie=f3c0d259a1646689 in_seq=0 out_seq=0 2024-03-26T17:46:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.855+0000 7f5e177fe640 1 -- 172.21.15.105:0/1490306999 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5626aa3f3ec0 con 0x5626aa50cfd0 2024-03-26T17:46:09.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.855+0000 7f5e177fe640 10 client.18330 handle_command_reply: tid=0 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.855+0000 7f5e4102a740 1 client.18330 shutdown 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.855+0000 7f5e4102a740 1 -- 172.21.15.105:0/1490306999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 msgr2=0x5626aa4970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.855+0000 7f5e4102a740 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa4970d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009d90 tx=0x7f5e2c010430 comp rx=0 tx=0).stop 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 1 -- 172.21.15.105:0/1490306999 shutdown_connections 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626aa50cfd0 0x5626aa50f3f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626aa48ccf0 0x5626aa498040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626aa487b80 0x5626aa4970d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 1 --2- 172.21.15.105:0/1490306999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626aa487190 0x5626aa496b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 1 -- 172.21.15.105:0/1490306999 >> 172.21.15.105:0/1490306999 conn(0x5626aa482700 msgr2=0x5626aa491960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e177fe640 0 client.18330 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 1 -- 172.21.15.105:0/1490306999 shutdown_connections 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 1 -- 172.21.15.105:0/1490306999 wait complete. 2024-03-26T17:46:09.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e4102a740 20 client.18330 trim_cache size 0 max 16384 2024-03-26T17:46:09.856 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:46:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:46:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "2ef97015-b3c4-4697-8f32-1c795c9670b2", 2024-03-26T17:46:09.856 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:46:09.856 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 -- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e200a2f20 msgr2=0x7f5e200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e200a2f20 0x7f5e200a53e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e38069300 tx=0x7f5e2c012040 comp rx=0 tx=0).stop 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 -- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 msgr2=0x7f5e38069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 0x7f5e38069590 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5e28000c00 tx=0x7f5e28005dc0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 -- 172.21.15.105:0/4075952447 shutdown_connections 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e200a2f20 0x7f5e200a53e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e38102370 0x7f5e3819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e381019b0 0x7f5e3819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 --2- 172.21.15.105:0/4075952447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e381007c0 0x7f5e38069590 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.856+0000 7f5e3e368640 1 -- 172.21.15.105:0/4075952447 >> 172.21.15.105:0/4075952447 conn(0x7f5e380fbd50 msgr2=0x7f5e38104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.857+0000 7f5e3e368640 1 -- 172.21.15.105:0/4075952447 shutdown_connections 2024-03-26T17:46:09.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.857+0000 7f5e3e368640 1 -- 172.21.15.105:0/4075952447 wait complete. 2024-03-26T17:46:09.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:46:09.863 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 -- 172.21.15.105:0/571516124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 msgr2=0x7f4b08108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 --2- 172.21.15.105:0/571516124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 0x7f4b08108c00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009ca0 tx=0x7f4b0002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 -- 172.21.15.105:0/571516124 shutdown_connections 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 --2- 172.21.15.105:0/571516124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 0x7f4b08108c00 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 --2- 172.21.15.105:0/571516124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b081019b0 0x7f4b08101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 --2- 172.21.15.105:0/571516124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b081007c0 0x7f4b08100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 -- 172.21.15.105:0/571516124 >> 172.21.15.105:0/571516124 conn(0x7f4b080fbd50 msgr2=0x7f4b080fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 -- 172.21.15.105:0/571516124 shutdown_connections 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.985+0000 7f4b0db87640 1 -- 172.21.15.105:0/571516124 wait complete. 2024-03-26T17:46:09.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b0db87640 1 Processor -- start 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b0db87640 1 -- start start 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b0db87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b081007c0 0x7f4b080695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b0db87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b081019b0 0x7f4b0819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b0db87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 0x7f4b0819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b0db87640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810c2f0 con 0x7f4b08102370 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b0db87640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810c170 con 0x7f4b081019b0 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b0db87640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810c470 con 0x7f4b081007c0 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 0x7f4b0819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b06ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b081019b0 0x7f4b0819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 0x7f4b0819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38276/0 (socket says 172.21.15.105:38276) 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b077fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b081007c0 0x7f4b080695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b07fff640 1 -- 172.21.15.105:0/984886130 learned_addr learned my addr 172.21.15.105:0/984886130 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b07fff640 1 -- 172.21.15.105:0/984886130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b081007c0 msgr2=0x7f4b080695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b07fff640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b081007c0 0x7f4b080695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b07fff640 1 -- 172.21.15.105:0/984886130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b081019b0 msgr2=0x7f4b0819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b07fff640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b081019b0 0x7f4b0819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.986+0000 7f4b07fff640 1 -- 172.21.15.105:0/984886130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0819f740 con 0x7f4b08102370 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.987+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b081019b0 0x7f4b0819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.987+0000 7f4b07fff640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 0x7f4b0819ee80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4b0002c700 tx=0x7f4b000328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.987+0000 7f4b04ff9640 1 -- 172.21.15.105:0/984886130 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00041020 con 0x7f4b08102370 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.987+0000 7f4b0db87640 1 -- 172.21.15.105:0/984886130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0819f9d0 con 0x7f4b08102370 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.987+0000 7f4b04ff9640 1 -- 172.21.15.105:0/984886130 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b00003e20 con 0x7f4b08102370 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.987+0000 7f4b04ff9640 1 -- 172.21.15.105:0/984886130 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00039040 con 0x7f4b08102370 2024-03-26T17:46:09.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.987+0000 7f4b0db87640 1 -- 172.21.15.105:0/984886130 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0819ff10 con 0x7f4b08102370 2024-03-26T17:46:09.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.989+0000 7f4b04ff9640 1 -- 172.21.15.105:0/984886130 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4b00044020 con 0x7f4b08102370 2024-03-26T17:46:09.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b04ff9640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4af00a2f20 0x7f4af00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b10770740 1 Processor -- start 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b077fe640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4af00a2f20 0x7f4af00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b10770740 1 -- start start 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b04ff9640 1 -- 172.21.15.105:0/984886130 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f4b000f15c0 con 0x7f4b08102370 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b10770740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c0f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b10770740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b10770740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0fd450 0x56341c0ff8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b10770740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56341c05c980 con 0x56341c0fd450 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b10770740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56341c004be0 con 0x56341c0f78f0 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b10770740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56341c058930 con 0x56341c0f82e0 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.990+0000 7f4b06ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b077fe640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4af00a2f20 0x7f4af00a53e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4b08101810 tx=0x7f4af8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b06ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41120/0 (socket says 172.21.15.105:41120) 2024-03-26T17:46:09.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b06ffd640 1 -- 172.21.15.105:0/36183837 learned_addr learned my addr 172.21.15.105:0/36183837 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b07fff640 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0fd450 0x56341c0ff8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b077fe640 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c0f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b06ffd640 1 -- 172.21.15.105:0/36183837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 msgr2=0x56341c0f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c0f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b06ffd640 1 -- 172.21.15.105:0/36183837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0fd450 msgr2=0x56341c0ff8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0fd450 0x56341c0ff8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b06ffd640 1 -- 172.21.15.105:0/36183837 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56341be3a7d0 con 0x56341c0f82e0 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b07fff640 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0fd450 0x56341c0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0fcde0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0114f0 tx=0x7f4afc0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dfe621c77beaacf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4aedffb640 1 -- 172.21.15.105:0/36183837 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc01e070 con 0x56341c0f82e0 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4aedffb640 1 -- 172.21.15.105:0/36183837 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4afc0035f0 con 0x56341c0f82e0 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.991+0000 7f4aedffb640 1 -- 172.21.15.105:0/36183837 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00f820 con 0x56341c0f82e0 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 -- 172.21.15.105:0/36183837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 msgr2=0x56341c0fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0fcde0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0114f0 tx=0x7f4afc0119b0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 -- 172.21.15.105:0/36183837 shutdown_connections 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0fd450 0x56341c0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0fcde0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 --2- 172.21.15.105:0/36183837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c0f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 -- 172.21.15.105:0/36183837 >> 172.21.15.105:0/36183837 conn(0x56341c0f2e60 msgr2=0x56341c0f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 -- 172.21.15.105:0/36183837 shutdown_connections 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 -- 172.21.15.105:0/36183837 wait complete. 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.992+0000 7f4b10770740 1 -- start start 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b10770740 1 Processor -- start 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b10770740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c101c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b10770740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f82e0 0x56341c102140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b10770740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 0x56341c109110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b10770740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56341c100400 con 0x56341c0f82e0 2024-03-26T17:46:09.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b10770740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56341c018160 con 0x56341c0f78f0 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b10770740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56341c100580 con 0x56341c0fd450 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b10770740 10 client.0 _ll_register_callbacks cb 0x56341c054720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b077fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c101c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b07fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 0x56341c109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b077fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c101c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53478/0 (socket says 172.21.15.105:53478) 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b077fe640 1 -- 172.21.15.105:0/126770785 learned_addr learned my addr 172.21.15.105:0/126770785 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f82e0 0x56341c102140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b077fe640 1 -- 172.21.15.105:0/126770785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 msgr2=0x56341c109110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b077fe640 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 0x56341c109110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b077fe640 1 -- 172.21.15.105:0/126770785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f82e0 msgr2=0x56341c102140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b077fe640 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f82e0 0x56341c102140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.993+0000 7f4b077fe640 1 -- 172.21.15.105:0/126770785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56341bd703f0 con 0x56341c0f78f0 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.994+0000 7f4b077fe640 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c101c00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ed70 tx=0x7f4af8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e1c0276c9722a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.994+0000 7f4ae6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:46:09.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.994+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/126770785 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af801db20 con 0x56341c0f78f0 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.994+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/126770785 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f4af801dcc0 con 0x56341c0f78f0 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.994+0000 7f4b10770740 1 -- 172.21.15.105:0/126770785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4afc0035f0 con 0x56341c0f78f0 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.994+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/126770785 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4af8005020 con 0x56341c0f78f0 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.994+0000 7f4b10770740 10 client.28172 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.994+0000 7f4b10770740 1 -- 172.21.15.105:0/126770785 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56341bd300f0 con 0x56341c0f78f0 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/126770785 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4afc0035f0 con 0x56341c0f78f0 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/126770785 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4af8026410 con 0x56341c0f78f0 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4b10770740 10 client.28172 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4b10770740 10 client.28172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4b10770740 20 client.28172 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4b10770740 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56341c10d9f0 0x56341c10fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4b10770740 4 client.28172 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4b10770740 1 -- 172.21.15.105:0/126770785 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56341c069ec0 con 0x56341c10d9f0 2024-03-26T17:46:09.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56341c10d9f0 0x56341c10fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56341c10d9f0 0x56341c10fe90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71a9795c945007e server_cookie=65e18b21dd660cee in_seq=0 out_seq=0 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.995+0000 7f4ae6ffd640 10 client.28172 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.996+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/126770785 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56341c069ec0 con 0x56341c10d9f0 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.996+0000 7f4ae6ffd640 10 client.28172 handle_command_reply: tid=0 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.996+0000 7f4b10770740 1 client.28172 shutdown 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 -- 172.21.15.105:0/126770785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 msgr2=0x56341c101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c101c00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ed70 tx=0x7f4af8004730 comp rx=0 tx=0).stop 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 -- 172.21.15.105:0/126770785 shutdown_connections 2024-03-26T17:46:09.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56341c10d9f0 0x56341c10fe90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 0x56341c109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f82e0 0x56341c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 --2- 172.21.15.105:0/126770785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f78f0 0x56341c101c00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4ae6ffd640 0 client.28172 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 -- 172.21.15.105:0/126770785 >> 172.21.15.105:0/126770785 conn(0x56341c0f2e60 msgr2=0x56341c0f2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 -- 172.21.15.105:0/126770785 shutdown_connections 2024-03-26T17:46:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 1 -- 172.21.15.105:0/126770785 wait complete. 2024-03-26T17:46:09.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:09.997+0000 7f4b10770740 20 client.28172 trim_cache size 0 max 16384 2024-03-26T17:46:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.018+0000 7f4b10770740 1 Processor -- start 2024-03-26T17:46:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.018+0000 7f4b10770740 1 -- start start 2024-03-26T17:46:10.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.018+0000 7f4b10770740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c1020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.018+0000 7f4b10770740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.018+0000 7f4b10770740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0fd450 0x56341c0f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.018+0000 7f4b10770740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081a6c90 con 0x56341c0f78f0 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.018+0000 7f4b10770740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081a87d0 con 0x56341c0fd450 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.018+0000 7f4b10770740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081a7360 con 0x56341c0f82e0 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c1020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0fd450 0x56341c0f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41142/0 (socket says 172.21.15.105:41142) 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 -- 172.21.15.105:0/620169806 learned_addr learned my addr 172.21.15.105:0/620169806 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c1020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38304/0 (socket says 172.21.15.105:38304) 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 -- 172.21.15.105:0/620169806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0fd450 msgr2=0x56341c0f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:10.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/620169806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0fd450 0x56341c0f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 -- 172.21.15.105:0/620169806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 msgr2=0x56341c1020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/620169806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c1020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 -- 172.21.15.105:0/620169806 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56341c0fc850 con 0x56341c0f82e0 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b077fe640 1 --2- 172.21.15.105:0/620169806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/620169806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0f17f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4afc014040 tx=0x7f4afc003780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a29bc525b6ebe153 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4aed7fa640 1 -- 172.21.15.105:0/620169806 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc029070 con 0x56341c0f82e0 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4aed7fa640 1 -- 172.21.15.105:0/620169806 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4afc005560 con 0x56341c0f82e0 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.019+0000 7f4aed7fa640 1 -- 172.21.15.105:0/620169806 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc01e040 con 0x56341c0f82e0 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 -- 172.21.15.105:0/620169806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 msgr2=0x56341c0f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 --2- 172.21.15.105:0/620169806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0f17f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4afc014040 tx=0x7f4afc003780 comp rx=0 tx=0).stop 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 -- 172.21.15.105:0/620169806 shutdown_connections 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 --2- 172.21.15.105:0/620169806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0fd450 0x56341c0f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 --2- 172.21.15.105:0/620169806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0f82e0 0x56341c0f17f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 --2- 172.21.15.105:0/620169806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 -- 172.21.15.105:0/620169806 >> 172.21.15.105:0/620169806 conn(0x56341c0f2e60 msgr2=0x56341c0f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 -- 172.21.15.105:0/620169806 shutdown_connections 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 -- 172.21.15.105:0/620169806 wait complete. 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.020+0000 7f4b10770740 1 -- start start 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b10770740 1 Processor -- start 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b10770740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c106b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b10770740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f82e0 0x56341c107060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b10770740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 0x56341c108070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b10770740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081a8100 con 0x56341c0f78f0 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b10770740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081a7a30 con 0x56341c0f82e0 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b10770740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b081a4c70 con 0x56341c0fd450 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b10770740 10 client.0 _ll_register_callbacks cb 0x7f4af800e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b06ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f82e0 0x56341c107060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:10.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c106b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38314/0 (socket says 172.21.15.105:38314) 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 -- 172.21.15.105:0/596039173 learned_addr learned my addr 172.21.15.105:0/596039173 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b07fff640 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 0x56341c108070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 -- 172.21.15.105:0/596039173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 msgr2=0x56341c108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 0x56341c108070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 -- 172.21.15.105:0/596039173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f82e0 msgr2=0x56341c107060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f82e0 0x56341c107060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 -- 172.21.15.105:0/596039173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56341c108660 con 0x56341c0f78f0 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f82e0 0x56341c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4b077fe640 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c106b20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4af8026a70 tx=0x7f4af8026da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee7abbedbddc6ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.021+0000 7f4ae6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/596039173 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8035070 con 0x56341c0f78f0 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/596039173 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f4af8009040 con 0x56341c0f78f0 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 1 -- 172.21.15.105:0/596039173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4afc005560 con 0x56341c0f78f0 2024-03-26T17:46:10.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/596039173 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4af80265f0 con 0x56341c0f78f0 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 10 client.18348 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 1 -- 172.21.15.105:0/596039173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56341c174400 con 0x56341c0f78f0 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/596039173 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4afc005560 con 0x56341c0f78f0 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/596039173 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4af8004930 con 0x56341c0f78f0 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 10 client.18348 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 10 client.18348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 20 client.18348 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56341c174d80 0x56341c1771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 4 client.18348 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b10770740 1 -- 172.21.15.105:0/596039173 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56341c064210 con 0x56341c174d80 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.022+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56341c174d80 0x56341c1771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.023+0000 7f4b06ffd640 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56341c174d80 0x56341c1771a0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2b27c879d5da79 server_cookie=a9627be2dbdfb38e in_seq=0 out_seq=0 2024-03-26T17:46:10.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.023+0000 7f4ae6ffd640 10 client.18348 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.023+0000 7f4ae6ffd640 1 -- 172.21.15.105:0/596039173 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56341c064210 con 0x56341c174d80 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.023+0000 7f4ae6ffd640 10 client.18348 handle_command_reply: tid=0 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.023+0000 7f4b10770740 1 client.18348 shutdown 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 -- 172.21.15.105:0/596039173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 msgr2=0x56341c106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c106b20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4af8026a70 tx=0x7f4af8026da0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 -- 172.21.15.105:0/596039173 shutdown_connections 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56341c174d80 0x56341c1771a0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56341c0fd450 0x56341c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56341c0f82e0 0x56341c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4ae6ffd640 0 client.18348 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 --2- 172.21.15.105:0/596039173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56341c0f78f0 0x56341c106b20 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 -- 172.21.15.105:0/596039173 >> 172.21.15.105:0/596039173 conn(0x56341c0f2e60 msgr2=0x56341c0f60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 -- 172.21.15.105:0/596039173 shutdown_connections 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 1 -- 172.21.15.105:0/596039173 wait complete. 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b10770740 20 client.18348 trim_cache size 0 max 16384 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stdout: "2ef97015-b3c4-4697-8f32-1c795c9670b2": { 2024-03-26T17:46:10.023 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:46:10.024 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "2ef97015-b3c4-4697-8f32-1c795c9670b2", 2024-03-26T17:46:10.024 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:46:10.024 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:46:10.024 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:46:10.024 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 -- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4af00a2f20 msgr2=0x7f4af00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4af00a2f20 0x7f4af00a53e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4b08101810 tx=0x7f4af8005f90 comp rx=0 tx=0).stop 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 -- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 msgr2=0x7f4b0819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 0x7f4b0819ee80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f4b0002c700 tx=0x7f4b000328a0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 -- 172.21.15.105:0/984886130 shutdown_connections 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4af00a2f20 0x7f4af00a53e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b08102370 0x7f4b0819ee80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b081019b0 0x7f4b0819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 --2- 172.21.15.105:0/984886130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b081007c0 0x7f4b080695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.024+0000 7f4b0db87640 1 -- 172.21.15.105:0/984886130 >> 172.21.15.105:0/984886130 conn(0x7f4b080fbd50 msgr2=0x7f4b08104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.025+0000 7f4b0db87640 1 -- 172.21.15.105:0/984886130 shutdown_connections 2024-03-26T17:46:10.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:10.025+0000 7f4b0db87640 1 -- 172.21.15.105:0/984886130 wait complete. 2024-03-26T17:46:10.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 242663 3.15 MB/sec execute 293 sec latency 18.682 ms 2024-03-26T17:46:10.028 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:09 smithi105 ceph-mon[32280]: pgmap v582: 97 pgs: 97 active+clean; 30 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-03-26T17:46:10.029 INFO:tasks.cephfs.filesystem:scrub status for tag:2ef97015-b3c4-4697-8f32-1c795c9670b2 - {'path': '/', 'tag': '2ef97015-b3c4-4697-8f32-1c795c9670b2', 'options': 'recursive,force'} 2024-03-26T17:46:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:09 smithi144 ceph-mon[36095]: pgmap v582: 97 pgs: 97 active+clean; 30 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-03-26T17:46:10.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:09 smithi135 ceph-mon[35636]: pgmap v582: 97 pgs: 97 active+clean; 30 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-03-26T17:46:11.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 243323 3.14 MB/sec execute 294 sec latency 34.814 ms 2024-03-26T17:46:11.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:10 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:46:11.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:10 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:11.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:10 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:46:11.099 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:10 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:11.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:10 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:46:11.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:10 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:11.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:10 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:46:11.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:10 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:11.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:10 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:46:11.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:10 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:11.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:10 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:46:11.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:10 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:12.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 243956 3.15 MB/sec execute 295 sec latency 16.899 ms 2024-03-26T17:46:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:11 smithi144 ceph-mon[36095]: pgmap v583: 97 pgs: 97 active+clean; 29 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:46:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:11 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:46:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:11 smithi105 ceph-mon[32280]: pgmap v583: 97 pgs: 97 active+clean; 29 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:46:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:11 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:46:12.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:11 smithi135 ceph-mon[35636]: pgmap v583: 97 pgs: 97 active+clean; 29 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:46:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:11 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:46:13.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 244568 3.15 MB/sec execute 296 sec latency 22.049 ms 2024-03-26T17:46:14.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 245124 3.15 MB/sec execute 297 sec latency 16.897 ms 2024-03-26T17:46:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:13 smithi144 ceph-mon[36095]: pgmap v584: 97 pgs: 97 active+clean; 25 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:46:14.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:13 smithi105 ceph-mon[32280]: pgmap v584: 97 pgs: 97 active+clean; 25 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:46:14.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:13 smithi135 ceph-mon[35636]: pgmap v584: 97 pgs: 97 active+clean; 25 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:46:15.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 245569 3.15 MB/sec execute 298 sec latency 17.605 ms 2024-03-26T17:46:16.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 246190 3.15 MB/sec execute 299 sec latency 16.470 ms 2024-03-26T17:46:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:15 smithi144 ceph-mon[36095]: pgmap v585: 97 pgs: 97 active+clean; 26 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-03-26T17:46:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:15 smithi105 ceph-mon[32280]: pgmap v585: 97 pgs: 97 active+clean; 26 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-03-26T17:46:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:15 smithi135 ceph-mon[35636]: pgmap v585: 97 pgs: 97 active+clean; 26 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-03-26T17:46:17.027 INFO:tasks.workunit.client.0.smithi105.stdout: 1 246850 3.14 MB/sec execute 300 sec latency 20.897 ms 2024-03-26T17:46:18.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 247395 3.14 MB/sec execute 301 sec latency 18.578 ms 2024-03-26T17:46:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:17 smithi144 ceph-mon[36095]: pgmap v586: 97 pgs: 97 active+clean; 27 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:46:18.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:17 smithi105 ceph-mon[32280]: pgmap v586: 97 pgs: 97 active+clean; 27 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:46:18.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:17 smithi135 ceph-mon[35636]: pgmap v586: 97 pgs: 97 active+clean; 27 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:46:19.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 248116 3.15 MB/sec execute 302 sec latency 15.285 ms 2024-03-26T17:46:20.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 248674 3.16 MB/sec execute 303 sec latency 15.953 ms 2024-03-26T17:46:20.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:19 smithi144 ceph-mon[36095]: pgmap v587: 97 pgs: 97 active+clean; 28 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-03-26T17:46:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:19 smithi105 ceph-mon[32280]: pgmap v587: 97 pgs: 97 active+clean; 28 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-03-26T17:46:20.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:19 smithi135 ceph-mon[35636]: pgmap v587: 97 pgs: 97 active+clean; 28 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-03-26T17:46:21.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 249120 3.15 MB/sec execute 304 sec latency 22.925 ms 2024-03-26T17:46:22.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 249753 3.15 MB/sec execute 305 sec latency 17.385 ms 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 -- 172.21.15.105:0/3398613465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 msgr2=0x7fa878109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 --2- 172.21.15.105:0/3398613465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 0x7fa878109a10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa868009ca0 tx=0x7fa86802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 -- 172.21.15.105:0/3398613465 shutdown_connections 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 --2- 172.21.15.105:0/3398613465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 0x7fa878109a10 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 --2- 172.21.15.105:0/3398613465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878104c90 0x7fa878107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 --2- 172.21.15.105:0/3398613465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa878102360 0x7fa878104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 -- 172.21.15.105:0/3398613465 >> 172.21.15.105:0/3398613465 conn(0x7fa8780fbd50 msgr2=0x7fa8780fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 -- 172.21.15.105:0/3398613465 shutdown_connections 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.044+0000 7fa87d227640 1 -- 172.21.15.105:0/3398613465 wait complete. 2024-03-26T17:46:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87d227640 1 Processor -- start 2024-03-26T17:46:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87d227640 1 -- start start 2024-03-26T17:46:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87d227640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa878102360 0x7fa878197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87d227640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878104c90 0x7fa878197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87d227640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 0x7fa87819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87d227640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810d410 con 0x7fa8781075c0 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87d227640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810d290 con 0x7fa878102360 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87d227640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810d590 con 0x7fa878104c90 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa876d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa878102360 0x7fa878197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 0x7fa87819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa876d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa878102360 0x7fa878197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57840/0 (socket says 172.21.15.105:57840) 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878104c90 0x7fa878197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa876d9d640 1 -- 172.21.15.105:0/317866509 learned_addr learned my addr 172.21.15.105:0/317866509 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa876d9d640 1 -- 172.21.15.105:0/317866509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878104c90 msgr2=0x7fa878197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa876d9d640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878104c90 0x7fa878197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa876d9d640 1 -- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 msgr2=0x7fa87819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa876d9d640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 0x7fa87819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa876d9d640 1 -- 172.21.15.105:0/317866509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87819f630 con 0x7fa878102360 2024-03-26T17:46:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.045+0000 7fa87759e640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 0x7fa87819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.046+0000 7fa876d9d640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa878102360 0x7fa878197a10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa87000e9f0 tx=0x7fa87000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.046+0000 7fa86ffff640 1 -- 172.21.15.105:0/317866509 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000cd80 con 0x7fa878102360 2024-03-26T17:46:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.046+0000 7fa87d227640 1 -- 172.21.15.105:0/317866509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87819f920 con 0x7fa878102360 2024-03-26T17:46:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.046+0000 7fa86ffff640 1 -- 172.21.15.105:0/317866509 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8700108e0 con 0x7fa878102360 2024-03-26T17:46:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.046+0000 7fa86ffff640 1 -- 172.21.15.105:0/317866509 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870018920 con 0x7fa878102360 2024-03-26T17:46:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.046+0000 7fa87d227640 1 -- 172.21.15.105:0/317866509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87819fec0 con 0x7fa878102360 2024-03-26T17:46:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.048+0000 7fa86ffff640 1 -- 172.21.15.105:0/317866509 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa870010a90 con 0x7fa878102360 2024-03-26T17:46:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.048+0000 7fa87fe0f740 1 Processor -- start 2024-03-26T17:46:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87fe0f740 1 -- start start 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621bada39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 0x5621badab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5621bad080b0 con 0x5621bada3fc0 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5621bacb0960 con 0x5621bada35d0 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5621bad04060 con 0x5621bada9130 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa876d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621bada39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621bada39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57852/0 (socket says 172.21.15.105:57852) 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 0x5621badab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87759e640 1 -- 172.21.15.105:0/2622085415 learned_addr learned my addr 172.21.15.105:0/2622085415 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa876d9d640 1 -- 172.21.15.105:0/2622085415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 msgr2=0x5621badab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa876d9d640 1 --2- 172.21.15.105:0/2622085415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 0x5621badab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa876d9d640 1 -- 172.21.15.105:0/2622085415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 msgr2=0x5621bada39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa876d9d640 1 --2- 172.21.15.105:0/2622085415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621bada39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa876d9d640 1 -- 172.21.15.105:0/2622085415 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621bacc2c60 con 0x5621bada3fc0 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa87759e640 1 --2- 172.21.15.105:0/2622085415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621bada39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.049+0000 7fa86ffff640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8640a2fc0 0x7fa8640a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.050+0000 7fa87759e640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8640a2fc0 0x7fa8640a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.050+0000 7fa876d9d640 1 --2- 172.21.15.105:0/2622085415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada8ac0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa8700cd190 tx=0x7fa8700d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f202dd5d6d96a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.050+0000 7fa86ffff640 1 -- 172.21.15.105:0/317866509 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa8700cf700 con 0x7fa878102360 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.050+0000 7fa86d7fa640 1 -- 172.21.15.105:0/2622085415 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870026030 con 0x5621bada3fc0 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.050+0000 7fa86d7fa640 1 -- 172.21.15.105:0/2622085415 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa870010de0 con 0x5621bada3fc0 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.050+0000 7fa86d7fa640 1 -- 172.21.15.105:0/2622085415 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87009b9e0 con 0x5621bada3fc0 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- 172.21.15.105:0/2622085415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 msgr2=0x5621bada8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/2622085415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada8ac0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa8700cd190 tx=0x7fa8700d7690 comp rx=0 tx=0).stop 2024-03-26T17:46:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87759e640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8640a2fc0 0x7fa8640a5480 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa868032380 tx=0x7fa86802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- 172.21.15.105:0/2622085415 shutdown_connections 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/2622085415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 0x5621badab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/2622085415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada8ac0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/2622085415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621bada39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- 172.21.15.105:0/2622085415 >> 172.21.15.105:0/2622085415 conn(0x5621bad9eb40 msgr2=0x5621bad9ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- 172.21.15.105:0/2622085415 shutdown_connections 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- 172.21.15.105:0/2622085415 wait complete. 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- start start 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 Processor -- start 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada35d0 0x5621badad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada3fc0 0x5621badad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 0x5621badb4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5621badac0e0 con 0x5621bada35d0 2024-03-26T17:46:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5621bacc3db0 con 0x5621bada3fc0 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5621badac260 con 0x5621bada9130 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87fe0f740 10 client.0 _ll_register_callbacks cb 0x5621bacc8110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa876d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada35d0 0x5621badad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa876d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada35d0 0x5621badad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55454/0 (socket says 172.21.15.105:55454) 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa876d9d640 1 -- 172.21.15.105:0/1313589631 learned_addr learned my addr 172.21.15.105:0/1313589631 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.051+0000 7fa87659c640 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada3fc0 0x5621badad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa87759e640 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 0x5621badb4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa876d9d640 1 -- 172.21.15.105:0/1313589631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 msgr2=0x5621badb4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa876d9d640 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 0x5621badb4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa876d9d640 1 -- 172.21.15.105:0/1313589631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada3fc0 msgr2=0x5621badad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa876d9d640 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada3fc0 0x5621badad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa876d9d640 1 -- 172.21.15.105:0/1313589631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621baa1c210 con 0x5621bada35d0 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa876d9d640 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada35d0 0x5621badad350 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa8700cd210 tx=0x7fa87000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51af6478dd1e505d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa85e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa85e7fc640 1 -- 172.21.15.105:0/1313589631 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87009b800 con 0x5621bada35d0 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa85e7fc640 1 -- 172.21.15.105:0/1313589631 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa870098510 con 0x5621bada35d0 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1313589631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621bad155f0 con 0x5621bada35d0 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa85e7fc640 1 -- 172.21.15.105:0/1313589631 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8700b7030 con 0x5621bada35d0 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa87fe0f740 10 client.18360 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1313589631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621baae8620 con 0x5621bada35d0 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.052+0000 7fa85e7fc640 1 -- 172.21.15.105:0/1313589631 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5621bad155f0 con 0x5621bada35d0 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa85e7fc640 1 -- 172.21.15.105:0/1313589631 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa8700d5cd0 con 0x5621bada35d0 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa87fe0f740 10 client.18360 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa87fe0f740 10 client.18360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa87fe0f740 20 client.18360 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621badb8d50 0x5621badbb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa87fe0f740 4 client.18360 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1313589631 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621bad0f940 con 0x5621badb8d50 2024-03-26T17:46:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa87659c640 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621badb8d50 0x5621badbb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa85e7fc640 10 client.18360 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.053+0000 7fa87659c640 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621badb8d50 0x5621badbb140 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb3fa7b4536bc server_cookie=b1d8274084941dc2 in_seq=0 out_seq=0 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa85e7fc640 1 -- 172.21.15.105:0/1313589631 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5621bad0f940 con 0x5621badb8d50 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa85e7fc640 10 client.18360 handle_command_reply: tid=0 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 client.18360 shutdown 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1313589631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada35d0 msgr2=0x5621badad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada35d0 0x5621badad350 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fa8700cd210 tx=0x7fa87000c7d0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1313589631 shutdown_connections 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621badb8d50 0x5621badbb140 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa85e7fc640 0 client.18360 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bada9130 0x5621badb4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada3fc0 0x5621badad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1313589631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada35d0 0x5621badad350 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1313589631 >> 172.21.15.105:0/1313589631 conn(0x5621bad9eb40 msgr2=0x5621bad9ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1313589631 shutdown_connections 2024-03-26T17:46:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1313589631 wait complete. 2024-03-26T17:46:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.055+0000 7fa87fe0f740 20 client.18360 trim_cache size 0 max 16384 2024-03-26T17:46:22.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.070+0000 7fa87fe0f740 1 Processor -- start 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87fe0f740 1 -- start start 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781a8d00 con 0x5621bada3fc0 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781aa840 con 0x5621bada35d0 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781a93d0 con 0x5621bad9eb40 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57862/0 (socket says 172.21.15.105:57862) 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87659c640 1 -- 172.21.15.105:0/1686494712 learned_addr learned my addr 172.21.15.105:0/1686494712 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa876d9d640 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87659c640 1 -- 172.21.15.105:0/1686494712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 msgr2=0x5621badad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87659c640 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87659c640 1 -- 172.21.15.105:0/1686494712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 msgr2=0x5621bada4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87659c640 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87659c640 1 -- 172.21.15.105:0/1686494712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621badac3c0 con 0x5621bada35d0 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa876d9d640 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.071+0000 7fa87759e640 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.072+0000 7fa87659c640 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb47f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa860000c00 tx=0x7fa860005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f1b61f7a10caba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.072+0000 7fa86cff9640 1 -- 172.21.15.105:0/1686494712 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86001f430 con 0x5621bada35d0 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.072+0000 7fa86cff9640 1 -- 172.21.15.105:0/1686494712 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa86001f5d0 con 0x5621bada35d0 2024-03-26T17:46:22.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.072+0000 7fa86cff9640 1 -- 172.21.15.105:0/1686494712 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86000fde0 con 0x5621bada35d0 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1686494712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 msgr2=0x5621badb47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb47f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa860000c00 tx=0x7fa860005f90 comp rx=0 tx=0).stop 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1686494712 shutdown_connections 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621bada4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb47f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/1686494712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1686494712 >> 172.21.15.105:0/1686494712 conn(0x5621badbb4b0 msgr2=0x5621badacf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1686494712 shutdown_connections 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- 172.21.15.105:0/1686494712 wait complete. 2024-03-26T17:46:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- start start 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 Processor -- start 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badb0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621badb3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781aa170 con 0x5621bada3fc0 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781a9aa0 con 0x5621bada35d0 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781a6ce0 con 0x5621bad9eb40 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87fe0f740 10 client.0 _ll_register_callbacks cb 0x7fa8600097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621badb3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa876d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badb0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa87659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa876d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badb0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34080/0 (socket says 172.21.15.105:34080) 2024-03-26T17:46:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.073+0000 7fa876d9d640 1 -- 172.21.15.105:0/3666501576 learned_addr learned my addr 172.21.15.105:0/3666501576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87759e640 1 -- 172.21.15.105:0/3666501576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 msgr2=0x5621badb0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87759e640 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badb0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87759e640 1 -- 172.21.15.105:0/3666501576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 msgr2=0x5621badb0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87759e640 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87759e640 1 -- 172.21.15.105:0/3666501576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621badb4040 con 0x5621bada3fc0 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87759e640 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621badb3a50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x5621bada4820 tx=0x7fa86802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9658acb0d9af41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa85e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa85e7fc640 1 -- 172.21.15.105:0/3666501576 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868052030 con 0x5621bada3fc0 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa85e7fc640 1 -- 172.21.15.105:0/3666501576 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa8680043d0 con 0x5621bada3fc0 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87fe0f740 1 -- 172.21.15.105:0/3666501576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86001f5d0 con 0x5621bada3fc0 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa85e7fc640 1 -- 172.21.15.105:0/3666501576 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa868039040 con 0x5621bada3fc0 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87fe0f740 10 client.18366 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa87fe0f740 1 -- 172.21.15.105:0/3666501576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621bae239a0 con 0x5621bada3fc0 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa85e7fc640 1 -- 172.21.15.105:0/3666501576 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa86001f5d0 con 0x5621bada3fc0 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.074+0000 7fa85e7fc640 1 -- 172.21.15.105:0/3666501576 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa868040c80 con 0x5621bada3fc0 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa87fe0f740 10 client.18366 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa87fe0f740 10 client.18366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa87fe0f740 20 client.18366 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621bada9130 0x5621bae26460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa87fe0f740 4 client.18366 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa87fe0f740 1 -- 172.21.15.105:0/3666501576 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621bac8d770 con 0x5621bada9130 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa876d9d640 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621bada9130 0x5621bae26460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa876d9d640 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621bada9130 0x5621bae26460 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be97e9d054921876 server_cookie=42f41fb2c6cacc5f in_seq=0 out_seq=0 2024-03-26T17:46:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.075+0000 7fa85e7fc640 10 client.18366 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:21 smithi144 ceph-mon[36095]: pgmap v588: 97 pgs: 97 active+clean; 27 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-03-26T17:46:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa85e7fc640 1 -- 172.21.15.105:0/3666501576 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621bac8d770 con 0x5621bada9130 2024-03-26T17:46:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa85e7fc640 10 client.18366 handle_command_reply: tid=0 2024-03-26T17:46:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 client.18366 shutdown 2024-03-26T17:46:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 -- 172.21.15.105:0/3666501576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 msgr2=0x5621badb3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621badb3a50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x5621bada4820 tx=0x7fa86802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 -- 172.21.15.105:0/3666501576 shutdown_connections 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5621bada9130 0x5621bae26460 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5621bada3fc0 0x5621badb3a50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621bada35d0 0x5621badb0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 --2- 172.21.15.105:0/3666501576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5621bad9eb40 0x5621badb0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 -- 172.21.15.105:0/3666501576 >> 172.21.15.105:0/3666501576 conn(0x5621badbb4b0 msgr2=0x5621bada4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa85e7fc640 0 client.18366 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 -- 172.21.15.105:0/3666501576 shutdown_connections 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 1 -- 172.21.15.105:0/3666501576 wait complete. 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.127+0000 7fa87fe0f740 20 client.18366 trim_cache size 0 max 16384 2024-03-26T17:46:22.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:46:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:46:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:46:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 -- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8640a2fc0 msgr2=0x7fa8640a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8640a2fc0 0x7fa8640a5480 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa868032380 tx=0x7fa86802e040 comp rx=0 tx=0).stop 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 -- 172.21.15.105:0/317866509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa878102360 msgr2=0x7fa878197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa878102360 0x7fa878197a10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa87000e9f0 tx=0x7fa87000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 -- 172.21.15.105:0/317866509 shutdown_connections 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa8640a2fc0 0x7fa8640a5480 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8781075c0 0x7fa87819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa878104c90 0x7fa878197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 --2- 172.21.15.105:0/317866509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa878102360 0x7fa878197a10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 -- 172.21.15.105:0/317866509 >> 172.21.15.105:0/317866509 conn(0x7fa8780fbd50 msgr2=0x7fa878102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.128+0000 7fa87d227640 1 -- 172.21.15.105:0/317866509 shutdown_connections 2024-03-26T17:46:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:22.129+0000 7fa87d227640 1 -- 172.21.15.105:0/317866509 wait complete. 2024-03-26T17:46:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:21 smithi105 ceph-mon[32280]: pgmap v588: 97 pgs: 97 active+clean; 27 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-03-26T17:46:22.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:21 smithi135 ceph-mon[35636]: pgmap v588: 97 pgs: 97 active+clean; 27 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-03-26T17:46:23.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 250383 3.14 MB/sec execute 306 sec latency 27.033 ms 2024-03-26T17:46:24.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 250943 3.14 MB/sec execute 307 sec latency 18.339 ms 2024-03-26T17:46:24.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:23 smithi144 ceph-mon[36095]: pgmap v589: 97 pgs: 97 active+clean; 27 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:46:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:23 smithi105 ceph-mon[32280]: pgmap v589: 97 pgs: 97 active+clean; 27 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:46:24.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:23 smithi135 ceph-mon[35636]: pgmap v589: 97 pgs: 97 active+clean; 27 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-03-26T17:46:25.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 251652 3.15 MB/sec execute 308 sec latency 20.269 ms 2024-03-26T17:46:26.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 252195 3.16 MB/sec execute 309 sec latency 16.044 ms 2024-03-26T17:46:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:25 smithi144 ceph-mon[36095]: pgmap v590: 97 pgs: 97 active+clean; 25 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-03-26T17:46:26.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:25 smithi105 ceph-mon[32280]: pgmap v590: 97 pgs: 97 active+clean; 25 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-03-26T17:46:26.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:25 smithi135 ceph-mon[35636]: pgmap v590: 97 pgs: 97 active+clean; 25 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-03-26T17:46:27.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 252624 3.15 MB/sec execute 310 sec latency 19.990 ms 2024-03-26T17:46:28.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 253283 3.15 MB/sec execute 311 sec latency 15.871 ms 2024-03-26T17:46:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:27 smithi144 ceph-mon[36095]: pgmap v591: 97 pgs: 97 active+clean; 24 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:46:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:27 smithi105 ceph-mon[32280]: pgmap v591: 97 pgs: 97 active+clean; 24 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:46:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:27 smithi135 ceph-mon[35636]: pgmap v591: 97 pgs: 97 active+clean; 24 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:46:29.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 253929 3.14 MB/sec execute 312 sec latency 20.739 ms 2024-03-26T17:46:30.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 254486 3.14 MB/sec execute 313 sec latency 21.804 ms 2024-03-26T17:46:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:29 smithi144 ceph-mon[36095]: pgmap v592: 97 pgs: 97 active+clean; 26 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-03-26T17:46:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:29 smithi105 ceph-mon[32280]: pgmap v592: 97 pgs: 97 active+clean; 26 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-03-26T17:46:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:29 smithi135 ceph-mon[35636]: pgmap v592: 97 pgs: 97 active+clean; 26 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.5 MiB/s wr, 131 op/s 2024-03-26T17:46:31.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 255195 3.15 MB/sec execute 314 sec latency 68.298 ms 2024-03-26T17:46:31.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:30 smithi135 ceph-mon[35636]: pgmap v593: 97 pgs: 97 active+clean; 27 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:46:31.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:30 smithi144 ceph-mon[36095]: pgmap v593: 97 pgs: 97 active+clean; 27 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:46:31.391 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:30 smithi105 ceph-mon[32280]: pgmap v593: 97 pgs: 97 active+clean; 27 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.5 MiB/s wr, 109 op/s 2024-03-26T17:46:32.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 255728 3.16 MB/sec execute 315 sec latency 15.906 ms 2024-03-26T17:46:33.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 256118 3.15 MB/sec execute 316 sec latency 25.220 ms 2024-03-26T17:46:34.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 256707 3.14 MB/sec execute 317 sec latency 17.191 ms 2024-03-26T17:46:34.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:33 smithi144 ceph-mon[36095]: pgmap v594: 97 pgs: 97 active+clean; 29 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-03-26T17:46:34.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:33 smithi105 ceph-mon[32280]: pgmap v594: 97 pgs: 97 active+clean; 29 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-03-26T17:46:34.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:33 smithi135 ceph-mon[35636]: pgmap v594: 97 pgs: 97 active+clean; 29 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-03-26T17:46:35.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 257400 3.14 MB/sec execute 318 sec latency 60.468 ms 2024-03-26T17:46:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.226+0000 7fcd0f36f640 1 -- 172.21.15.105:0/994125610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081019e0 msgr2=0x7fcd08101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.226+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/994125610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081019e0 0x7fcd08101e60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009900 tx=0x7fccf802bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.226+0000 7fcd0f36f640 1 -- 172.21.15.105:0/994125610 shutdown_connections 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.226+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/994125610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd081023a0 0x7fcd08108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.226+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/994125610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081019e0 0x7fcd08101e60 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.226+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/994125610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd081007f0 0x7fcd08100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 -- 172.21.15.105:0/994125610 >> 172.21.15.105:0/994125610 conn(0x7fcd080fbd80 msgr2=0x7fcd080fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 -- 172.21.15.105:0/994125610 shutdown_connections 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 -- 172.21.15.105:0/994125610 wait complete. 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 Processor -- start 2024-03-26T17:46:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 -- start start 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081007f0 0x7fcd0806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd081019e0 0x7fcd0819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd081023a0 0x7fcd0819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0810c320 con 0x7fcd081007f0 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0810c1a0 con 0x7fcd081023a0 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.227+0000 7fcd0f36f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0810c4a0 con 0x7fcd081019e0 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081007f0 0x7fcd0806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081007f0 0x7fcd0806abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43980/0 (socket says 172.21.15.105:43980) 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 -- 172.21.15.105:0/3392096420 learned_addr learned my addr 172.21.15.105:0/3392096420 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd081019e0 0x7fcd0819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd081023a0 0x7fcd0819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 -- 172.21.15.105:0/3392096420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd081019e0 msgr2=0x7fcd0819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd081019e0 0x7fcd0819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 -- 172.21.15.105:0/3392096420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd081023a0 msgr2=0x7fcd0819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd081023a0 0x7fcd0819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 -- 172.21.15.105:0/3392096420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0819f6c0 con 0x7fcd081007f0 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081007f0 0x7fcd0806abd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400e9c0 tx=0x7fcd0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fccfe7fc640 1 -- 172.21.15.105:0/3392096420 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400cd80 con 0x7fcd081007f0 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fccfe7fc640 1 -- 172.21.15.105:0/3392096420 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd040035f0 con 0x7fcd081007f0 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0f36f640 1 -- 172.21.15.105:0/3392096420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0819f950 con 0x7fcd081007f0 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fccfe7fc640 1 -- 172.21.15.105:0/3392096420 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04010690 con 0x7fcd081007f0 2024-03-26T17:46:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.228+0000 7fcd0f36f640 1 -- 172.21.15.105:0/3392096420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0819fe70 con 0x7fcd081007f0 2024-03-26T17:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.231+0000 7fccfe7fc640 1 -- 172.21.15.105:0/3392096420 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcd040037a0 con 0x7fcd081007f0 2024-03-26T17:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.231+0000 7fcd12031740 1 Processor -- start 2024-03-26T17:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.231+0000 7fcd12031740 1 -- start start 2024-03-26T17:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd12031740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 0x5625bc06a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd12031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06afc0 0x5625bc06fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd12031740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc0725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd12031740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5625bbfcf0b0 con 0x5625bc070130 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd12031740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5625bbf77960 con 0x5625bc06a5d0 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd12031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625bbfcb060 con 0x5625bc06afc0 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fccfe7fc640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fccf40a2fc0 0x7fccf40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d10b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06afc0 0x5625bc06fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0c90a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc0725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d90c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 0x5625bc06a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0c90a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc0725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43986/0 (socket says 172.21.15.105:43986) 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0c90a640 1 -- 172.21.15.105:0/1637380466 learned_addr learned my addr 172.21.15.105:0/1637380466 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fccf40a2fc0 0x7fccf40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fccfe7fc640 1 -- 172.21.15.105:0/3392096420 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcd040cf7a0 con 0x7fcd081007f0 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d10b640 1 -- 172.21.15.105:0/1637380466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 msgr2=0x5625bc06a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/1637380466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 0x5625bc06a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d10b640 1 -- 172.21.15.105:0/1637380466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 msgr2=0x5625bc0725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/1637380466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc0725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d10b640 1 -- 172.21.15.105:0/1637380466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625bbf89c60 con 0x5625bc06afc0 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/1637380466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc0725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/1637380466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06afc0 0x5625bc06fac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcd04098860 tx=0x7fcd04003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5214be24574cfed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.232+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fccf40a2fc0 0x7fccf40a5480 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x5625bc06b820 tx=0x7fcd00004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.233+0000 7fccf37fe640 1 -- 172.21.15.105:0/1637380466 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0409b7f0 con 0x5625bc06afc0 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.233+0000 7fccf37fe640 1 -- 172.21.15.105:0/1637380466 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd04003b30 con 0x5625bc06afc0 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.233+0000 7fccf37fe640 1 -- 172.21.15.105:0/1637380466 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04098500 con 0x5625bc06afc0 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.233+0000 7fcd12031740 1 -- 172.21.15.105:0/1637380466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06afc0 msgr2=0x5625bc06fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.233+0000 7fcd12031740 1 --2- 172.21.15.105:0/1637380466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06afc0 0x5625bc06fac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcd04098860 tx=0x7fcd04003e30 comp rx=0 tx=0).stop 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 -- 172.21.15.105:0/1637380466 shutdown_connections 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 --2- 172.21.15.105:0/1637380466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc0725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 --2- 172.21.15.105:0/1637380466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06afc0 0x5625bc06fac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 --2- 172.21.15.105:0/1637380466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 0x5625bc06a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 -- 172.21.15.105:0/1637380466 >> 172.21.15.105:0/1637380466 conn(0x5625bc065b40 msgr2=0x5625bc065f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 -- 172.21.15.105:0/1637380466 shutdown_connections 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 -- 172.21.15.105:0/1637380466 wait complete. 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 -- start start 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 Processor -- start 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06a5d0 0x5625bc0743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc074900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc07b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5625bc0730e0 con 0x5625bc070130 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5625bbf8adb0 con 0x5625bc06afc0 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625bc073260 con 0x5625bc06a5d0 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.234+0000 7fcd12031740 10 client.0 _ll_register_callbacks cb 0x5625bbf8f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0d10b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06a5d0 0x5625bc0743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0c90a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0d90c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0d10b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06a5d0 0x5625bc0743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56590/0 (socket says 172.21.15.105:56590) 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0d10b640 1 -- 172.21.15.105:0/2745657485 learned_addr learned my addr 172.21.15.105:0/2745657485 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0c90a640 1 -- 172.21.15.105:0/2745657485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06a5d0 msgr2=0x5625bc0743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06a5d0 0x5625bc0743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0c90a640 1 -- 172.21.15.105:0/2745657485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 msgr2=0x5625bc07b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc07b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0c90a640 1 -- 172.21.15.105:0/2745657485 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625bbce3210 con 0x5625bc06afc0 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06a5d0 0x5625bc0743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc07b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc074900 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5625bc066ab0 tx=0x7fccf8004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c243b6dfce2a92dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fccf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2745657485 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8005d50 con 0x5625bc06afc0 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fcd12031740 1 -- 172.21.15.105:0/2745657485 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5625bbfdc5f0 con 0x5625bc06afc0 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.235+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2745657485 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fccf8005030 con 0x5625bc06afc0 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2745657485 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccf8051050 con 0x5625bc06afc0 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd12031740 10 client.28208 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd12031740 1 -- 172.21.15.105:0/2745657485 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625bbdaf620 con 0x5625bc06afc0 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2745657485 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5625bbfdc5f0 con 0x5625bc06afc0 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2745657485 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fccf80035f0 con 0x5625bc06afc0 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd12031740 10 client.28208 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd12031740 10 client.28208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd12031740 20 client.28208 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd12031740 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5625bc080280 0x5625bc082670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd12031740 4 client.28208 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd12031740 1 -- 172.21.15.105:0/2745657485 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625bbfd6940 con 0x5625bc080280 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.236+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5625bc080280 0x5625bc082670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.237+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5625bc080280 0x5625bc082670 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=470145eead2891df server_cookie=d2dad1306c4fb24b in_seq=0 out_seq=0 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.237+0000 7fccf0ff9640 10 client.28208 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fccf0ff9640 1 -- 172.21.15.105:0/2745657485 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5625bbfd6940 con 0x5625bc080280 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fccf0ff9640 10 client.28208 handle_command_reply: tid=0 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 client.28208 shutdown 2024-03-26T17:46:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 -- 172.21.15.105:0/2745657485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 msgr2=0x5625bc074900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc074900 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5625bc066ab0 tx=0x7fccf8004250 comp rx=0 tx=0).stop 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 -- 172.21.15.105:0/2745657485 shutdown_connections 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5625bc080280 0x5625bc082670 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc070130 0x5625bc07b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fccf0ff9640 0 client.28208 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc074900 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 --2- 172.21.15.105:0/2745657485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc06a5d0 0x5625bc0743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 -- 172.21.15.105:0/2745657485 >> 172.21.15.105:0/2745657485 conn(0x5625bc065b40 msgr2=0x5625bc064df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 -- 172.21.15.105:0/2745657485 shutdown_connections 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 1 -- 172.21.15.105:0/2745657485 wait complete. 2024-03-26T17:46:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.238+0000 7fcd12031740 20 client.28208 trim_cache size 0 max 16384 2024-03-26T17:46:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.255+0000 7fcd12031740 1 Processor -- start 2024-03-26T17:46:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.255+0000 7fcd12031740 1 -- start start 2024-03-26T17:46:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd12031740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 0x5625bc06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd12031740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06afc0 0x5625bc06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd12031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc06bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd12031740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081a8d10 con 0x5625bc06afc0 2024-03-26T17:46:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd12031740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081aa850 con 0x5625bc06a5d0 2024-03-26T17:46:35.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd12031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081a93e0 con 0x5625bc070130 2024-03-26T17:46:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d90c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc06bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d90c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc06bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56594/0 (socket says 172.21.15.105:56594) 2024-03-26T17:46:35.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d90c640 1 -- 172.21.15.105:0/2173586530 learned_addr learned my addr 172.21.15.105:0/2173586530 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 0x5625bc06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06afc0 0x5625bc06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d90c640 1 -- 172.21.15.105:0/2173586530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 msgr2=0x5625bc06fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 0x5625bc06fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d90c640 1 -- 172.21.15.105:0/2173586530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06afc0 msgr2=0x5625bc06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06afc0 0x5625bc06b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0d90c640 1 -- 172.21.15.105:0/2173586530 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625bc061450 con 0x5625bc070130 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.256+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06afc0 0x5625bc06b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.257+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc06bf00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcd00013ef0 tx=0x7fcd00005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb5ece4facff6dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.257+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2173586530 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00004030 con 0x5625bc070130 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.257+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2173586530 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd00004680 con 0x5625bc070130 2024-03-26T17:46:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.257+0000 7fccf2ffd640 1 -- 172.21.15.105:0/2173586530 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00010d30 con 0x5625bc070130 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 -- 172.21.15.105:0/2173586530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 msgr2=0x5625bc06bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc06bf00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcd00013ef0 tx=0x7fcd00005bd0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 -- 172.21.15.105:0/2173586530 shutdown_connections 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc06bf00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06afc0 0x5625bc06b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 --2- 172.21.15.105:0/2173586530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06a5d0 0x5625bc06fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 -- 172.21.15.105:0/2173586530 >> 172.21.15.105:0/2173586530 conn(0x5625bc065b40 msgr2=0x5625bc06f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 -- 172.21.15.105:0/2173586530 shutdown_connections 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 -- 172.21.15.105:0/2173586530 wait complete. 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 -- start start 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.258+0000 7fcd12031740 1 Processor -- start 2024-03-26T17:46:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd12031740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06a5d0 0x5625bc0e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd12031740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc0e1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd12031740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc0e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd12031740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081aa180 con 0x5625bc06a5d0 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd12031740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081a9ab0 con 0x5625bc06afc0 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd12031740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd081a6cf0 con 0x5625bc070130 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0d10b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06a5d0 0x5625bc0e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd12031740 10 client.0 _ll_register_callbacks cb 0x7fccf8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0c90a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc0e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0d10b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06a5d0 0x5625bc0e0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44010/0 (socket says 172.21.15.105:44010) 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0d10b640 1 -- 172.21.15.105:0/1687497643 learned_addr learned my addr 172.21.15.105:0/1687497643 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc0e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0c90a640 1 -- 172.21.15.105:0/1687497643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 msgr2=0x5625bc0e83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc0e83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0c90a640 1 -- 172.21.15.105:0/1687497643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06a5d0 msgr2=0x5625bc0e0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06a5d0 0x5625bc0e0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0c90a640 1 -- 172.21.15.105:0/1687497643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625bc0e89c0 con 0x5625bc06afc0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0d90c640 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc0e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fcd0c90a640 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc0e1390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fccf8004650 tx=0x7fccf8003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4af7606c4211dad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fccf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.259+0000 7fccf0ff9640 1 -- 172.21.15.105:0/1687497643 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8005d50 con 0x5625bc06afc0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 1 -- 172.21.15.105:0/1687497643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd00004680 con 0x5625bc06afc0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fccf0ff9640 1 -- 172.21.15.105:0/1687497643 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fccf8005220 con 0x5625bc06afc0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fccf0ff9640 1 -- 172.21.15.105:0/1687497643 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccf8005440 con 0x5625bc06afc0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 10 client.28214 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 1 -- 172.21.15.105:0/1687497643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625bc0e9cb0 con 0x5625bc06afc0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fccf0ff9640 1 -- 172.21.15.105:0/1687497643 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd00004680 con 0x5625bc06afc0 2024-03-26T17:46:35.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fccf0ff9640 1 -- 172.21.15.105:0/1687497643 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fccf8004c00 con 0x5625bc06afc0 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 10 client.28214 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 10 client.28214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 20 client.28214 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5625bc0ea600 0x5625bc0eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 4 client.28214 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd12031740 1 -- 172.21.15.105:0/1687497643 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625bbf54770 con 0x5625bc0ea600 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.260+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5625bc0ea600 0x5625bc0eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.261+0000 7fcd0d10b640 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5625bc0ea600 0x5625bc0eca20 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb1b710ba0d107d server_cookie=fc9c76cd960996cd in_seq=0 out_seq=0 2024-03-26T17:46:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.261+0000 7fccf0ff9640 10 client.28214 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fccf0ff9640 1 -- 172.21.15.105:0/1687497643 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625bbf54770 con 0x5625bc0ea600 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fccf0ff9640 10 client.28214 handle_command_reply: tid=0 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 client.28214 shutdown 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 -- 172.21.15.105:0/1687497643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 msgr2=0x5625bc0e1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc0e1390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fccf8004650 tx=0x7fccf8003e30 comp rx=0 tx=0).stop 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 -- 172.21.15.105:0/1687497643 shutdown_connections 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5625bc0ea600 0x5625bc0eca20 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625bc070130 0x5625bc0e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fccf0ff9640 0 client.28214 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625bc06afc0 0x5625bc0e1390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 --2- 172.21.15.105:0/1687497643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625bc06a5d0 0x5625bc0e0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 -- 172.21.15.105:0/1687497643 >> 172.21.15.105:0/1687497643 conn(0x5625bc065b40 msgr2=0x5625bc068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 -- 172.21.15.105:0/1687497643 shutdown_connections 2024-03-26T17:46:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.313+0000 7fcd12031740 1 -- 172.21.15.105:0/1687497643 wait complete. 2024-03-26T17:46:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd12031740 20 client.28214 trim_cache size 0 max 16384 2024-03-26T17:46:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:46:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:46:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:46:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:46:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 -- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fccf40a2fc0 msgr2=0x7fccf40a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fccf40a2fc0 0x7fccf40a5480 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x5625bc06b820 tx=0x7fcd00004ab0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 -- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081007f0 msgr2=0x7fcd0806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081007f0 0x7fcd0806abd0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400e9c0 tx=0x7fcd0400ee80 comp rx=0 tx=0).stop 2024-03-26T17:46:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 -- 172.21.15.105:0/3392096420 shutdown_connections 2024-03-26T17:46:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fccf40a2fc0 0x7fccf40a5480 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd081023a0 0x7fcd0819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd081019e0 0x7fcd0819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 --2- 172.21.15.105:0/3392096420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd081007f0 0x7fcd0806abd0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 -- 172.21.15.105:0/3392096420 >> 172.21.15.105:0/3392096420 conn(0x7fcd080fbd80 msgr2=0x7fcd08104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 -- 172.21.15.105:0/3392096420 shutdown_connections 2024-03-26T17:46:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:35.314+0000 7fcd0f36f640 1 -- 172.21.15.105:0/3392096420 wait complete. 2024-03-26T17:46:36.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 257981 3.14 MB/sec execute 319 sec latency 40.469 ms 2024-03-26T17:46:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:35 smithi144 ceph-mon[36095]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-03-26T17:46:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:35 smithi105 ceph-mon[32280]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-03-26T17:46:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:35 smithi135 ceph-mon[35636]: pgmap v595: 97 pgs: 97 active+clean; 28 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-03-26T17:46:37.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 258728 3.15 MB/sec execute 320 sec latency 16.406 ms 2024-03-26T17:46:38.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 259238 3.16 MB/sec execute 321 sec latency 14.564 ms 2024-03-26T17:46:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:37 smithi144 ceph-mon[36095]: pgmap v596: 97 pgs: 97 active+clean; 27 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:46:38.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:37 smithi105 ceph-mon[32280]: pgmap v596: 97 pgs: 97 active+clean; 27 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:46:38.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:37 smithi135 ceph-mon[35636]: pgmap v596: 97 pgs: 97 active+clean; 27 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:46:39.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 259630 3.15 MB/sec execute 322 sec latency 20.411 ms 2024-03-26T17:46:40.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 260224 3.14 MB/sec execute 323 sec latency 16.704 ms 2024-03-26T17:46:40.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:46:40.031 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:46:40.056 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:39 smithi105 ceph-mon[32280]: pgmap v597: 97 pgs: 97 active+clean; 27 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:46:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:39 smithi144 ceph-mon[36095]: pgmap v597: 97 pgs: 97 active+clean; 27 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:46:40.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 -- 172.21.15.105:0/4031699083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 msgr2=0x7fbc10101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 --2- 172.21.15.105:0/4031699083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 0x7fbc10101e30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009900 tx=0x7fbc0002bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:40.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 -- 172.21.15.105:0/4031699083 shutdown_connections 2024-03-26T17:46:40.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 --2- 172.21.15.105:0/4031699083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc10102370 0x7fbc10108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 --2- 172.21.15.105:0/4031699083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 0x7fbc10101e30 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 --2- 172.21.15.105:0/4031699083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc101007c0 0x7fbc10100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 -- 172.21.15.105:0/4031699083 >> 172.21.15.105:0/4031699083 conn(0x7fbc100fbd50 msgr2=0x7fbc100fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 -- 172.21.15.105:0/4031699083 shutdown_connections 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 -- 172.21.15.105:0/4031699083 wait complete. 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 Processor -- start 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.161+0000 7fbc17109640 1 -- start start 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc17109640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc101007c0 0x7fbc10197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc17109640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 0x7fbc10198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc17109640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc10102370 0x7fbc1019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc17109640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010c2f0 con 0x7fbc101019b0 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc17109640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010c170 con 0x7fbc101007c0 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc17109640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1010c470 con 0x7fbc10102370 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 0x7fbc10198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 0x7fbc10198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44018/0 (socket says 172.21.15.105:44018) 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 -- 172.21.15.105:0/6034737 learned_addr learned my addr 172.21.15.105:0/6034737 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc101007c0 0x7fbc10197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc156a6640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc10102370 0x7fbc1019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 -- 172.21.15.105:0/6034737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc10102370 msgr2=0x7fbc1019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc10102370 0x7fbc1019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 -- 172.21.15.105:0/6034737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc101007c0 msgr2=0x7fbc10197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc101007c0 0x7fbc10197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 -- 172.21.15.105:0/6034737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1019f760 con 0x7fbc101019b0 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc101007c0 0x7fbc10197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc07fff640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 0x7fbc10198080 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006010 tx=0x7fbc00005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.162+0000 7fbc05ffb640 1 -- 172.21.15.105:0/6034737 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00043020 con 0x7fbc101019b0 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.163+0000 7fbc05ffb640 1 -- 172.21.15.105:0/6034737 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc000043d0 con 0x7fbc101019b0 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.163+0000 7fbc05ffb640 1 -- 172.21.15.105:0/6034737 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00003b80 con 0x7fbc101019b0 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.163+0000 7fbc17109640 1 -- 172.21.15.105:0/6034737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc10102bd0 con 0x7fbc101019b0 2024-03-26T17:46:40.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.163+0000 7fbc17109640 1 -- 172.21.15.105:0/6034737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc101030d0 con 0x7fbc101019b0 2024-03-26T17:46:40.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:39 smithi135 ceph-mon[35636]: pgmap v597: 97 pgs: 97 active+clean; 27 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-03-26T17:46:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.166+0000 7fbc05ffb640 1 -- 172.21.15.105:0/6034737 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbc00046020 con 0x7fbc101019b0 2024-03-26T17:46:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.166+0000 7fbc05ffb640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbf80a2fa0 0x7fbbf80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.166+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbf80a2fa0 0x7fbbf80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.166+0000 7fbc05ffb640 1 -- 172.21.15.105:0/6034737 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fbc000f0fc0 con 0x7fbc101019b0 2024-03-26T17:46:40.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.166+0000 7fbc19dd3740 1 Processor -- start 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc19dd3740 1 -- start start 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6bdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a622980 con 0x55dd9a6c3450 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a5cabe0 con 0x55dd9a6be2e0 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a61e930 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbf80a2fa0 0x7fbbf80a5460 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc10101810 tx=0x7fbc0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6bdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6bdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56630/0 (socket says 172.21.15.105:56630) 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36098/0 (socket says 172.21.15.105:36098) 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 -- 172.21.15.105:0/4191388289 learned_addr learned my addr 172.21.15.105:0/4191388289 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc156a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 -- 172.21.15.105:0/4191388289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 msgr2=0x55dd9a6c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/4191388289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 -- 172.21.15.105:0/4191388289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 msgr2=0x55dd9a6c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/4191388289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 -- 172.21.15.105:0/4191388289 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9a4007d0 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc07fff640 1 --2- 172.21.15.105:0/4191388289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.167+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/4191388289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6bdd10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00f650 tx=0x7fbc0c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78f92340753a882 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.168+0000 7fbbf6ffd640 1 -- 172.21.15.105:0/4191388289 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c01c810 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.168+0000 7fbbf6ffd640 1 -- 172.21.15.105:0/4191388289 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc0c0263f0 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.168+0000 7fbbf6ffd640 1 -- 172.21.15.105:0/4191388289 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c01ce60 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.168+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4191388289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 msgr2=0x55dd9a6bdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.168+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4191388289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6bdd10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00f650 tx=0x7fbc0c014000 comp rx=0 tx=0).stop 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4191388289 shutdown_connections 2024-03-26T17:46:40.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4191388289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4191388289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4191388289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6bdd10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4191388289 >> 172.21.15.105:0/4191388289 conn(0x55dd9a6b8e60 msgr2=0x55dd9a6b9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4191388289 shutdown_connections 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4191388289 wait complete. 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 -- start start 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 Processor -- start 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a6c6400 con 0x55dd9a6be2e0 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a5de160 con 0x55dd9a6c3450 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.169+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a6c6580 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc19dd3740 10 client.0 _ll_register_callbacks cb 0x55dd9a61a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44026/0 (socket says 172.21.15.105:44026) 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc07fff640 1 -- 172.21.15.105:0/3266708733 learned_addr learned my addr 172.21.15.105:0/3266708733 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc156a6640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc14ea5640 1 -- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 msgr2=0x55dd9a6cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc14ea5640 1 -- 172.21.15.105:0/3266708733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 msgr2=0x55dd9a6c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc14ea5640 1 -- 172.21.15.105:0/3266708733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd9a3363f0 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc156a6640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc07fff640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c7ad0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c011ca0 tx=0x7fbc0c026a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602f2833ef35a44a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbbeffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbbeffff640 1 -- 172.21.15.105:0/3266708733 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c035060 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbbeffff640 1 -- 172.21.15.105:0/3266708733 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fbc0c005250 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbc19dd3740 1 -- 172.21.15.105:0/3266708733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc0c0263f0 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.170+0000 7fbbeffff640 1 -- 172.21.15.105:0/3266708733 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc0c005450 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc19dd3740 10 client.28231 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc19dd3740 1 -- 172.21.15.105:0/3266708733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd9a2f60f0 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbbeffff640 1 -- 172.21.15.105:0/3266708733 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc0c0263f0 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbbeffff640 1 -- 172.21.15.105:0/3266708733 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbc0c02f070 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc19dd3740 10 client.28231 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:40.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc19dd3740 10 client.28231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc19dd3740 20 client.28231 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9a6d3590 0x55dd9a6d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc19dd3740 4 client.28231 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc19dd3740 1 -- 172.21.15.105:0/3266708733 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd9a62fec0 con 0x55dd9a6d3590 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.171+0000 7fbc07fff640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9a6d3590 0x55dd9a6d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc07fff640 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9a6d3590 0x55dd9a6d59c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb46585dc1cc015 server_cookie=c0872931d12992c8 in_seq=0 out_seq=0 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbbeffff640 10 client.28231 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbbeffff640 1 -- 172.21.15.105:0/3266708733 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dd9a62fec0 con 0x55dd9a6d3590 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbbeffff640 10 client.28231 handle_command_reply: tid=0 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 client.28231 shutdown 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 -- 172.21.15.105:0/3266708733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 msgr2=0x55dd9a6c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c7ad0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c011ca0 tx=0x7fbc0c026a40 comp rx=0 tx=0).stop 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 -- 172.21.15.105:0/3266708733 shutdown_connections 2024-03-26T17:46:40.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9a6d3590 0x55dd9a6d59c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbbeffff640 0 client.28231 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:40.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/3266708733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c7ad0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.172+0000 7fbc19dd3740 1 -- 172.21.15.105:0/3266708733 >> 172.21.15.105:0/3266708733 conn(0x55dd9a6b8e60 msgr2=0x55dd9a6b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.173+0000 7fbc19dd3740 1 -- 172.21.15.105:0/3266708733 shutdown_connections 2024-03-26T17:46:40.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.173+0000 7fbc19dd3740 1 -- 172.21.15.105:0/3266708733 wait complete. 2024-03-26T17:46:40.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.173+0000 7fbc19dd3740 20 client.28231 trim_cache size 0 max 16384 2024-03-26T17:46:40.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc19dd3740 1 Processor -- start 2024-03-26T17:46:40.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc19dd3740 1 -- start start 2024-03-26T17:46:40.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101a6bd0 con 0x55dd9a6be2e0 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101a8710 con 0x55dd9a6c3450 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101a72a0 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44038/0 (socket says 172.21.15.105:44038) 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 -- 172.21.15.105:0/2967208093 learned_addr learned my addr 172.21.15.105:0/2967208093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc156a6640 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 -- 172.21.15.105:0/2967208093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 msgr2=0x55dd9a6c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 -- 172.21.15.105:0/2967208093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 msgr2=0x55dd9a6c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 -- 172.21.15.105:0/2967208093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9a6c74f0 con 0x55dd9a6be2e0 2024-03-26T17:46:40.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc156a6640 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.191+0000 7fbc07fff640 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cf010 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbc00004650 tx=0x7fbc000f0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26021e9492437eb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.192+0000 7fbbf67fc640 1 -- 172.21.15.105:0/2967208093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc000db070 con 0x55dd9a6be2e0 2024-03-26T17:46:40.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.192+0000 7fbbf67fc640 1 -- 172.21.15.105:0/2967208093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc000c1800 con 0x55dd9a6be2e0 2024-03-26T17:46:40.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.192+0000 7fbbf67fc640 1 -- 172.21.15.105:0/2967208093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc000d5070 con 0x55dd9a6be2e0 2024-03-26T17:46:40.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.192+0000 7fbc19dd3740 1 -- 172.21.15.105:0/2967208093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 msgr2=0x55dd9a6cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.192+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cf010 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbc00004650 tx=0x7fbc000f0040 comp rx=0 tx=0).stop 2024-03-26T17:46:40.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 -- 172.21.15.105:0/2967208093 shutdown_connections 2024-03-26T17:46:40.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6c3450 0x55dd9a6c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cf010 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/2967208093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 -- 172.21.15.105:0/2967208093 >> 172.21.15.105:0/2967208093 conn(0x55dd9a6b8e60 msgr2=0x55dd9a6c2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 -- 172.21.15.105:0/2967208093 shutdown_connections 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 -- 172.21.15.105:0/2967208093 wait complete. 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 -- start start 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 Processor -- start 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 0x55dd9a7378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101a8040 con 0x55dd9a6c3450 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101a7970 con 0x55dd9a6be2e0 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.193+0000 7fbc19dd3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc101a4bb0 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc19dd3740 10 client.0 _ll_register_callbacks cb 0x7fbc08009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56658/0 (socket says 172.21.15.105:56658) 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc156a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 0x55dd9a7378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36130/0 (socket says 172.21.15.105:36130) 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 -- 172.21.15.105:0/4003251292 learned_addr learned my addr 172.21.15.105:0/4003251292 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 -- 172.21.15.105:0/4003251292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 msgr2=0x55dd9a6cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 -- 172.21.15.105:0/4003251292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 msgr2=0x55dd9a7378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 0x55dd9a7378f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 -- 172.21.15.105:0/4003251292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd9a737e30 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc14ea5640 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6cbf10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c016040 tx=0x7fbc0c00f550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b66f8357cc97a80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbbeffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbbeffff640 1 -- 172.21.15.105:0/4003251292 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c03f070 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4003251292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc000c1800 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbbeffff640 1 -- 172.21.15.105:0/4003251292 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fbc0c01c610 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.194+0000 7fbbeffff640 1 -- 172.21.15.105:0/4003251292 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbc0c04b050 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc19dd3740 10 client.28237 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4003251292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd9a739110 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbbeffff640 1 -- 172.21.15.105:0/4003251292 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc000c1800 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbbeffff640 1 -- 172.21.15.105:0/4003251292 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbc0c02f040 con 0x55dd9a6bd8f0 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc19dd3740 10 client.28237 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc19dd3740 10 client.28237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc19dd3740 20 client.28237 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9a739a90 0x55dd9a73beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc19dd3740 4 client.28237 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4003251292 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd9a62a210 con 0x55dd9a739a90 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbc07fff640 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9a739a90 0x55dd9a73beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.195+0000 7fbbeffff640 10 client.28237 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc07fff640 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9a739a90 0x55dd9a73beb0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b6891e82f107f3 server_cookie=79b1b1519b1bfcc7 in_seq=0 out_seq=0 2024-03-26T17:46:40.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbbeffff640 1 -- 172.21.15.105:0/4003251292 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dd9a62a210 con 0x55dd9a739a90 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbbeffff640 10 client.28237 handle_command_reply: tid=0 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 client.28237 shutdown 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4003251292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 msgr2=0x55dd9a6cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6cbf10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c016040 tx=0x7fbc0c00f550 comp rx=0 tx=0).stop 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4003251292 shutdown_connections 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dd9a739a90 0x55dd9a73beb0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dd9a6c3450 0x55dd9a7378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dd9a6be2e0 0x55dd9a6cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 --2- 172.21.15.105:0/4003251292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dd9a6bd8f0 0x55dd9a6cbf10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbbeffff640 0 client.28237 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4003251292 >> 172.21.15.105:0/4003251292 conn(0x55dd9a6b8e60 msgr2=0x55dd9a6c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4003251292 shutdown_connections 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.196+0000 7fbc19dd3740 1 -- 172.21.15.105:0/4003251292 wait complete. 2024-03-26T17:46:40.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc19dd3740 20 client.28237 trim_cache size 0 max 16384 2024-03-26T17:46:40.194 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:46:40.194 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:46:40.194 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:46:40.194 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 -- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbf80a2fa0 msgr2=0x7fbbf80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbf80a2fa0 0x7fbbf80a5460 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc10101810 tx=0x7fbc0c005f90 comp rx=0 tx=0).stop 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 -- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 msgr2=0x7fbc10198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 0x7fbc10198080 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbc00006010 tx=0x7fbc00005e40 comp rx=0 tx=0).stop 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 -- 172.21.15.105:0/6034737 shutdown_connections 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbf80a2fa0 0x7fbbf80a5460 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc10102370 0x7fbc1019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc101019b0 0x7fbc10198080 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 --2- 172.21.15.105:0/6034737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc101007c0 0x7fbc10197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 -- 172.21.15.105:0/6034737 >> 172.21.15.105:0/6034737 conn(0x7fbc100fbd50 msgr2=0x7fbc10068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 -- 172.21.15.105:0/6034737 shutdown_connections 2024-03-26T17:46:40.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.197+0000 7fbc17109640 1 -- 172.21.15.105:0/6034737 wait complete. 2024-03-26T17:46:40.200 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:46:40.200 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 -- 172.21.15.105:0/1209225258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 msgr2=0x7fa6d4109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 --2- 172.21.15.105:0/1209225258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 0x7fa6d4109dd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009860 tx=0x7fa6c402bb70 comp rx=0 tx=0).stop 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 -- 172.21.15.105:0/1209225258 shutdown_connections 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 --2- 172.21.15.105:0/1209225258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 0x7fa6d4109dd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 --2- 172.21.15.105:0/1209225258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d4102b80 0x7fa6d4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 --2- 172.21.15.105:0/1209225258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d40ffbb0 0x7fa6d40fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 -- 172.21.15.105:0/1209225258 >> 172.21.15.105:0/1209225258 conn(0x7fa6d40fb4e0 msgr2=0x7fa6d40fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 -- 172.21.15.105:0/1209225258 shutdown_connections 2024-03-26T17:46:40.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.383+0000 7fa6d9177640 1 -- 172.21.15.105:0/1209225258 wait complete. 2024-03-26T17:46:40.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d9177640 1 Processor -- start 2024-03-26T17:46:40.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d9177640 1 -- start start 2024-03-26T17:46:40.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d9177640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d40ffbb0 0x7fa6d4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d9177640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4102b80 0x7fa6d4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d9177640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 0x7fa6d419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d9177640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410d430 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d9177640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410d2b0 con 0x7fa6d4103540 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d9177640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410d5b0 con 0x7fa6d4102b80 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d40ffbb0 0x7fa6d4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d40ffbb0 0x7fa6d4197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44066/0 (socket says 172.21.15.105:44066) 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 0x7fa6d419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.384+0000 7fa6d2d9d640 1 -- 172.21.15.105:0/260723758 learned_addr learned my addr 172.21.15.105:0/260723758 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d259c640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4102b80 0x7fa6d4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d2d9d640 1 -- 172.21.15.105:0/260723758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4102b80 msgr2=0x7fa6d4197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d2d9d640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4102b80 0x7fa6d4197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d2d9d640 1 -- 172.21.15.105:0/260723758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 msgr2=0x7fa6d419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d2d9d640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 0x7fa6d419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d2d9d640 1 -- 172.21.15.105:0/260723758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d419f6c0 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d359e640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 0x7fa6d419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d259c640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4102b80 0x7fa6d4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d2d9d640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d40ffbb0 0x7fa6d4197aa0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c3a0 tx=0x7fa6cc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6bbfff640 1 -- 172.21.15.105:0/260723758 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01b070 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6bbfff640 1 -- 172.21.15.105:0/260723758 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6cc003e20 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d419f9b0 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6bbfff640 1 -- 172.21.15.105:0/260723758 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00b890 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.385+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d419feb0 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.387+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d4108cd0 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.387+0000 7fa6bbfff640 1 -- 172.21.15.105:0/260723758 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa6cc01e070 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.388+0000 7fa6bbfff640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.388+0000 7fa6d259c640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.388+0000 7fa6bbfff640 1 -- 172.21.15.105:0/260723758 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa6cc0cf490 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.389+0000 7fa6d259c640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80095f0 tx=0x7fa6c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.394+0000 7fa6bbfff640 1 -- 172.21.15.105:0/260723758 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa6cc09bc30 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.685+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6d402d240 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.686+0000 7fa6bbfff640 1 -- 172.21.15.105:0/260723758 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fa6d402d240 con 0x7fa6d40ffbb0 2024-03-26T17:46:40.678 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:46:40.678 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:46:40.678 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:46:40.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6bc0a2f20 msgr2=0x7fa6bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80095f0 tx=0x7fa6c8005f90 comp rx=0 tx=0).stop 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d40ffbb0 msgr2=0x7fa6d4197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d40ffbb0 0x7fa6d4197aa0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c3a0 tx=0x7fa6cc00c860 comp rx=0 tx=0).stop 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 shutdown_connections 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6bc0a2f20 0x7fa6bc0a53e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6d4103540 0x7fa6d419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6d4102b80 0x7fa6d4197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 --2- 172.21.15.105:0/260723758 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6d40ffbb0 0x7fa6d4197aa0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 >> 172.21.15.105:0/260723758 conn(0x7fa6d40fb4e0 msgr2=0x7fa6d4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 shutdown_connections 2024-03-26T17:46:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.689+0000 7fa6d9177640 1 -- 172.21.15.105:0/260723758 wait complete. 2024-03-26T17:46:40.693 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:46:40.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 -- 172.21.15.105:0/2767193612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c102ac0 msgr2=0x7fe74c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 --2- 172.21.15.105:0/2767193612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c102ac0 0x7fe74c102f40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009900 tx=0x7fe73c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:40.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 -- 172.21.15.105:0/2767193612 shutdown_connections 2024-03-26T17:46:40.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 --2- 172.21.15.105:0/2767193612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe74c103480 0x7fe74c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 --2- 172.21.15.105:0/2767193612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c102ac0 0x7fe74c102f40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 --2- 172.21.15.105:0/2767193612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c1018d0 0x7fe74c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 -- 172.21.15.105:0/2767193612 >> 172.21.15.105:0/2767193612 conn(0x7fe74c0fce20 msgr2=0x7fe74c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 -- 172.21.15.105:0/2767193612 shutdown_connections 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.823+0000 7fe753c61640 1 -- 172.21.15.105:0/2767193612 wait complete. 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe753c61640 1 Processor -- start 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe753c61640 1 -- start start 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe753c61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c1018d0 0x7fe74c075620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe753c61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe74c102ac0 0x7fe74c073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe753c61640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c103480 0x7fe74c0741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe753c61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c10d400 con 0x7fe74c1018d0 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe753c61640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c10d280 con 0x7fe74c103480 2024-03-26T17:46:40.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe753c61640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c10d580 con 0x7fe74c102ac0 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.824+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c1018d0 0x7fe74c075620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c1018d0 0x7fe74c075620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44080/0 (socket says 172.21.15.105:44080) 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7519fd640 1 -- 172.21.15.105:0/938918364 learned_addr learned my addr 172.21.15.105:0/938918364 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7511fc640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe74c102ac0 0x7fe74c073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7521fe640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c103480 0x7fe74c0741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7519fd640 1 -- 172.21.15.105:0/938918364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe74c102ac0 msgr2=0x7fe74c073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7519fd640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe74c102ac0 0x7fe74c073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7519fd640 1 -- 172.21.15.105:0/938918364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c103480 msgr2=0x7fe74c0741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7519fd640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c103480 0x7fe74c0741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7519fd640 1 -- 172.21.15.105:0/938918364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c074ab0 con 0x7fe74c1018d0 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe7519fd640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c1018d0 0x7fe74c075620 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe74800e9c0 tx=0x7fe74800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe742ffd640 1 -- 172.21.15.105:0/938918364 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74800cd80 con 0x7fe74c1018d0 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe753c61640 1 -- 172.21.15.105:0/938918364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74c1a1cb0 con 0x7fe74c1018d0 2024-03-26T17:46:40.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe742ffd640 1 -- 172.21.15.105:0/938918364 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7480035f0 con 0x7fe74c1018d0 2024-03-26T17:46:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe742ffd640 1 -- 172.21.15.105:0/938918364 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748010690 con 0x7fe74c1018d0 2024-03-26T17:46:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.825+0000 7fe753c61640 1 -- 172.21.15.105:0/938918364 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74c1a21d0 con 0x7fe74c1018d0 2024-03-26T17:46:40.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.828+0000 7fe742ffd640 1 -- 172.21.15.105:0/938918364 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe7480037a0 con 0x7fe74c1018d0 2024-03-26T17:46:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.828+0000 7fe756853740 1 Processor -- start 2024-03-26T17:46:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe756853740 1 -- start start 2024-03-26T17:46:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe756853740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdaa30 0x555f36fdae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe756853740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fdff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe756853740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fe2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe756853740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555f36f3f970 con 0x555f36fe0590 2024-03-26T17:46:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe756853740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555f36ee7d70 con 0x555f36fdb420 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe756853740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555f36f3b920 con 0x555f36fdaa30 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fdff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7511fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fe2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdaa30 0x555f36fdae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe742ffd640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7380a2f20 0x7fe7380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdaa30 0x555f36fdae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56684/0 (socket says 172.21.15.105:56684) 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fdff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36174/0 (socket says 172.21.15.105:36174) 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 -- 172.21.15.105:0/1292111283 learned_addr learned my addr 172.21.15.105:0/1292111283 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7511fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fe2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44082/0 (socket says 172.21.15.105:44082) 2024-03-26T17:46:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7380a2f20 0x7fe7380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 -- 172.21.15.105:0/1292111283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 msgr2=0x555f36fdff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 --2- 172.21.15.105:0/1292111283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fdff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 -- 172.21.15.105:0/1292111283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 msgr2=0x555f36fe2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 --2- 172.21.15.105:0/1292111283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fe2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7521fe640 1 -- 172.21.15.105:0/1292111283 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f36c533f0 con 0x555f36fdaa30 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe742ffd640 1 -- 172.21.15.105:0/938918364 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe7480cf7f0 con 0x7fe74c1018d0 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.829+0000 7fe7511fc640 1 --2- 172.21.15.105:0/1292111283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fe2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.830+0000 7fe7521fe640 1 --2- 172.21.15.105:0/1292111283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdaa30 0x555f36fdae50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe744005c40 tx=0x7fe74400f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4fde45cbd670c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.830+0000 7fe737fff640 1 -- 172.21.15.105:0/1292111283 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744022020 con 0x555f36fdaa30 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.830+0000 7fe7521fe640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7380a2f20 0x7fe7380a53e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe744023000 tx=0x7fe744021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.830+0000 7fe737fff640 1 -- 172.21.15.105:0/1292111283 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe744013170 con 0x555f36fdaa30 2024-03-26T17:46:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.830+0000 7fe737fff640 1 -- 172.21.15.105:0/1292111283 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744010690 con 0x555f36fdaa30 2024-03-26T17:46:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 -- 172.21.15.105:0/1292111283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdaa30 msgr2=0x555f36fdae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 --2- 172.21.15.105:0/1292111283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdaa30 0x555f36fdae50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe744005c40 tx=0x7fe74400f3c0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 -- 172.21.15.105:0/1292111283 shutdown_connections 2024-03-26T17:46:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 --2- 172.21.15.105:0/1292111283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fe2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 --2- 172.21.15.105:0/1292111283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fdff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 --2- 172.21.15.105:0/1292111283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdaa30 0x555f36fdae50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 -- 172.21.15.105:0/1292111283 >> 172.21.15.105:0/1292111283 conn(0x555f36fd5fa0 msgr2=0x555f36fd63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 -- 172.21.15.105:0/1292111283 shutdown_connections 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 -- 172.21.15.105:0/1292111283 wait complete. 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 -- start start 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.831+0000 7fe756853740 1 Processor -- start 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe756853740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe756853740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fe54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe756853740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fe0590 0x555f36fec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe756853740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555f36fe3540 con 0x555f36fdaa30 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe756853740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555f36efb310 con 0x555f36fe0590 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe756853740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555f36fe36c0 con 0x555f36fdb420 2024-03-26T17:46:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe756853740 10 client.0 _ll_register_callbacks cb 0x555f36f36c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44094/0 (socket says 172.21.15.105:44094) 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 -- 172.21.15.105:0/1556019947 learned_addr learned my addr 172.21.15.105:0/1556019947 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7511fc640 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fe54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7521fe640 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fe0590 0x555f36fec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 -- 172.21.15.105:0/1556019947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 msgr2=0x555f36fe54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fe54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 -- 172.21.15.105:0/1556019947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fe0590 msgr2=0x555f36fec520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fe0590 0x555f36fec520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 -- 172.21.15.105:0/1556019947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f36d1d7d0 con 0x555f36fdaa30 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7521fe640 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fe0590 0x555f36fec520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7519fd640 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe4fa0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe7480babf0 tx=0x7fe74809ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed55a0ed6c27916 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7357fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.832+0000 7fe7357fa640 1 -- 172.21.15.105:0/1556019947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7480cfde0 con 0x555f36fdaa30 2024-03-26T17:46:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe7357fa640 1 -- 172.21.15.105:0/1556019947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe7480a4400 con 0x555f36fdaa30 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 1 -- 172.21.15.105:0/1556019947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe744013170 con 0x555f36fdaa30 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe7357fa640 1 -- 172.21.15.105:0/1556019947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe748003b30 con 0x555f36fdaa30 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 10 client.18417 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 1 -- 172.21.15.105:0/1556019947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f36fef900 con 0x555f36fdaa30 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe7357fa640 1 -- 172.21.15.105:0/1556019947 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe744013170 con 0x555f36fdaa30 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe7357fa640 1 -- 172.21.15.105:0/1556019947 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe748097bd0 con 0x555f36fdaa30 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 10 client.18417 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 10 client.18417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 20 client.18417 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x555f36ff0ff0 0x555f36ff3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 4 client.18417 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe756853740 1 -- 172.21.15.105:0/1556019947 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f36f4ceb0 con 0x555f36ff0ff0 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.833+0000 7fe7511fc640 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x555f36ff0ff0 0x555f36ff3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.834+0000 7fe7357fa640 10 client.18417 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.834+0000 7fe7511fc640 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x555f36ff0ff0 0x555f36ff3420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=978cf139ee30cfc3 server_cookie=244e70cea267317 in_seq=0 out_seq=0 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.834+0000 7fe7357fa640 1 -- 172.21.15.105:0/1556019947 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555f36f4ceb0 con 0x555f36ff0ff0 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.834+0000 7fe7357fa640 10 client.18417 handle_command_reply: tid=0 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.834+0000 7fe756853740 1 client.18417 shutdown 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 -- 172.21.15.105:0/1556019947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 msgr2=0x555f36fe4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe4fa0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe7480babf0 tx=0x7fe74809ba80 comp rx=0 tx=0).stop 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 -- 172.21.15.105:0/1556019947 shutdown_connections 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x555f36ff0ff0 0x555f36ff3420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fe0590 0x555f36fec520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fe54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe7357fa640 0 client.18417 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:46:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 --2- 172.21.15.105:0/1556019947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe4fa0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 -- 172.21.15.105:0/1556019947 >> 172.21.15.105:0/1556019947 conn(0x555f36fd5fa0 msgr2=0x555f36fd51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 -- 172.21.15.105:0/1556019947 shutdown_connections 2024-03-26T17:46:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 1 -- 172.21.15.105:0/1556019947 wait complete. 2024-03-26T17:46:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.835+0000 7fe756853740 20 client.18417 trim_cache size 0 max 16384 2024-03-26T17:46:40.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe756853740 1 Processor -- start 2024-03-26T17:46:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe756853740 1 -- start start 2024-03-26T17:46:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe756853740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdaa30 0x555f36fe5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe756853740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe756853740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fd5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe756853740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c1ab010 con 0x555f36fe0590 2024-03-26T17:46:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe756853740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c1acb50 con 0x555f36fdaa30 2024-03-26T17:46:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe756853740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c1ab6e0 con 0x555f36fdb420 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7521fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fd5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7511fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdaa30 0x555f36fe5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7511fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fec4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56694/0 (socket says 172.21.15.105:56694) 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7511fc640 1 -- 172.21.15.105:0/2374249921 learned_addr learned my addr 172.21.15.105:0/2374249921 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7511fc640 1 -- 172.21.15.105:0/2374249921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdaa30 msgr2=0x555f36fe5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7511fc640 1 --2- 172.21.15.105:0/2374249921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdaa30 0x555f36fe5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7511fc640 1 -- 172.21.15.105:0/2374249921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 msgr2=0x555f36fd5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7511fc640 1 --2- 172.21.15.105:0/2374249921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fd5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.850+0000 7fe7511fc640 1 -- 172.21.15.105:0/2374249921 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f36fd58f0 con 0x555f36fdb420 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe7511fc640 1 --2- 172.21.15.105:0/2374249921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fec4e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe73c035000 tx=0x7fe73c02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2ac974bd758bd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe7377fe640 1 -- 172.21.15.105:0/2374249921 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c02cc20 con 0x555f36fdb420 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe7377fe640 1 -- 172.21.15.105:0/2374249921 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe73c005050 con 0x555f36fdb420 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe7377fe640 1 -- 172.21.15.105:0/2374249921 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c004a40 con 0x555f36fdb420 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe756853740 1 -- 172.21.15.105:0/2374249921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 msgr2=0x555f36fec4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe756853740 1 --2- 172.21.15.105:0/2374249921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fec4e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe73c035000 tx=0x7fe73c02c5d0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe756853740 1 -- 172.21.15.105:0/2374249921 shutdown_connections 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe756853740 1 --2- 172.21.15.105:0/2374249921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fe0590 0x555f36fd5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.851+0000 7fe756853740 1 --2- 172.21.15.105:0/2374249921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fdb420 0x555f36fec4e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 --2- 172.21.15.105:0/2374249921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdaa30 0x555f36fe5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 -- 172.21.15.105:0/2374249921 >> 172.21.15.105:0/2374249921 conn(0x555f36fd5fa0 msgr2=0x555f36fd4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 -- 172.21.15.105:0/2374249921 shutdown_connections 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 -- 172.21.15.105:0/2374249921 wait complete. 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 -- start start 2024-03-26T17:46:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 Processor -- start 2024-03-26T17:46:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fe0590 0x555f36feb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c1ac480 con 0x555f36fdaa30 2024-03-26T17:46:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c1abdb0 con 0x555f36fdb420 2024-03-26T17:46:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c1a8ff0 con 0x555f36fe0590 2024-03-26T17:46:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.852+0000 7fe756853740 10 client.0 _ll_register_callbacks cb 0x7fe748003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7519fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44116/0 (socket says 172.21.15.105:44116) 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7511fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7519fd640 1 -- 172.21.15.105:0/2168010686 learned_addr learned my addr 172.21.15.105:0/2168010686 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7521fe640 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fe0590 0x555f36feb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7511fc640 1 -- 172.21.15.105:0/2168010686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fe0590 msgr2=0x555f36feb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7511fc640 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fe0590 0x555f36feb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7511fc640 1 -- 172.21.15.105:0/2168010686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 msgr2=0x555f36fe9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7511fc640 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7511fc640 1 -- 172.21.15.105:0/2168010686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f36feb990 con 0x555f36fdb420 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7519fd640 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7511fc640 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fea390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009430 tx=0x7fe73c02cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dd847c34d403677 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7357fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7357fa640 1 -- 172.21.15.105:0/2168010686 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c057060 con 0x555f36fdb420 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7357fa640 1 -- 172.21.15.105:0/2168010686 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe73c005200 con 0x555f36fdb420 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe756853740 1 -- 172.21.15.105:0/2168010686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe73c005050 con 0x555f36fdb420 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.853+0000 7fe7357fa640 1 -- 172.21.15.105:0/2168010686 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe73c005400 con 0x555f36fdb420 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe756853740 10 client.28256 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe756853740 1 -- 172.21.15.105:0/2168010686 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f370574d0 con 0x555f36fdb420 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe7357fa640 1 -- 172.21.15.105:0/2168010686 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe73c005050 con 0x555f36fdb420 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe7357fa640 1 -- 172.21.15.105:0/2168010686 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe73c051070 con 0x555f36fdb420 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe756853740 10 client.28256 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe756853740 10 client.28256 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe756853740 20 client.28256 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe756853740 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x555f37057e10 0x555f3705a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe756853740 4 client.28256 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe756853740 1 -- 172.21.15.105:0/2168010686 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f36f47200 con 0x555f37057e10 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe7519fd640 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x555f37057e10 0x555f3705a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe7357fa640 10 client.28256 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.854+0000 7fe7519fd640 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x555f37057e10 0x555f3705a230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e53e3ae81c257c58 server_cookie=d6da1bf3375d1bc3 in_seq=0 out_seq=0 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe7357fa640 1 -- 172.21.15.105:0/2168010686 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f36f47200 con 0x555f37057e10 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe7357fa640 10 client.28256 handle_command_reply: tid=0 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 client.28256 shutdown 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 -- 172.21.15.105:0/2168010686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 msgr2=0x555f36fea390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fea390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009430 tx=0x7fe73c02cd80 comp rx=0 tx=0).stop 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 -- 172.21.15.105:0/2168010686 shutdown_connections 2024-03-26T17:46:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x555f37057e10 0x555f3705a230 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555f36fe0590 0x555f36feb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f36fdb420 0x555f36fea390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 --2- 172.21.15.105:0/2168010686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555f36fdaa30 0x555f36fe9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe7357fa640 0 client.28256 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 -- 172.21.15.105:0/2168010686 >> 172.21.15.105:0/2168010686 conn(0x555f36fd5fa0 msgr2=0x555f36fe52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 -- 172.21.15.105:0/2168010686 shutdown_connections 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 1 -- 172.21.15.105:0/2168010686 wait complete. 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.855+0000 7fe756853740 20 client.28256 trim_cache size 0 max 16384 2024-03-26T17:46:40.854 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 -- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7380a2f20 msgr2=0x7fe7380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7380a2f20 0x7fe7380a53e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe744023000 tx=0x7fe744021000 comp rx=0 tx=0).stop 2024-03-26T17:46:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 -- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c1018d0 msgr2=0x7fe74c075620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c1018d0 0x7fe74c075620 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe74800e9c0 tx=0x7fe74800ee80 comp rx=0 tx=0).stop 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 -- 172.21.15.105:0/938918364 shutdown_connections 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7380a2f20 0x7fe7380a53e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe74c103480 0x7fe74c0741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe74c102ac0 0x7fe74c073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 --2- 172.21.15.105:0/938918364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe74c1018d0 0x7fe74c075620 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 -- 172.21.15.105:0/938918364 >> 172.21.15.105:0/938918364 conn(0x7fe74c0fce20 msgr2=0x7fe74c105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 -- 172.21.15.105:0/938918364 shutdown_connections 2024-03-26T17:46:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.856+0000 7fe753c61640 1 -- 172.21.15.105:0/938918364 wait complete. 2024-03-26T17:46:40.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:46:40.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 -- 172.21.15.105:0/2072451526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 msgr2=0x7fa320102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 --2- 172.21.15.105:0/2072451526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 0x7fa320102f50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa314009900 tx=0x7fa31402bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:40.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 -- 172.21.15.105:0/2072451526 shutdown_connections 2024-03-26T17:46:40.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 --2- 172.21.15.105:0/2072451526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa320103490 0x7fa320109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 --2- 172.21.15.105:0/2072451526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 0x7fa320102f50 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 --2- 172.21.15.105:0/2072451526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3201018e0 0x7fa320101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 -- 172.21.15.105:0/2072451526 >> 172.21.15.105:0/2072451526 conn(0x7fa3200fce50 msgr2=0x7fa3200ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 -- 172.21.15.105:0/2072451526 shutdown_connections 2024-03-26T17:46:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 -- 172.21.15.105:0/2072451526 wait complete. 2024-03-26T17:46:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 Processor -- start 2024-03-26T17:46:40.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.980+0000 7fa32671b640 1 -- start start 2024-03-26T17:46:40.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa32671b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3201018e0 0x7fa320197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa32671b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 0x7fa320198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa32671b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa320103490 0x7fa32019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa32671b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa32010d3f0 con 0x7fa320102ad0 2024-03-26T17:46:40.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa32671b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa32010d270 con 0x7fa320103490 2024-03-26T17:46:40.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa32671b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa32010d570 con 0x7fa3201018e0 2024-03-26T17:46:40.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 0x7fa320198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 0x7fa320198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44134/0 (socket says 172.21.15.105:44134) 2024-03-26T17:46:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31f7fe640 1 -- 172.21.15.105:0/3276217468 learned_addr learned my addr 172.21.15.105:0/3276217468 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa324cb8640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa320103490 0x7fa32019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31ffff640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3201018e0 0x7fa320197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31f7fe640 1 -- 172.21.15.105:0/3276217468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3201018e0 msgr2=0x7fa320197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3201018e0 0x7fa320197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31f7fe640 1 -- 172.21.15.105:0/3276217468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa320103490 msgr2=0x7fa32019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa320103490 0x7fa32019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa31f7fe640 1 -- 172.21.15.105:0/3276217468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32019f770 con 0x7fa320102ad0 2024-03-26T17:46:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.981+0000 7fa324cb8640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa320103490 0x7fa32019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.982+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 0x7fa320198090 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa314006010 tx=0x7fa314005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.982+0000 7fa31d7fa640 1 -- 172.21.15.105:0/3276217468 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140043d0 con 0x7fa320102ad0 2024-03-26T17:46:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.982+0000 7fa31d7fa640 1 -- 172.21.15.105:0/3276217468 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3140052b0 con 0x7fa320102ad0 2024-03-26T17:46:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.982+0000 7fa32671b640 1 -- 172.21.15.105:0/3276217468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa320073af0 con 0x7fa320102ad0 2024-03-26T17:46:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.982+0000 7fa31d7fa640 1 -- 172.21.15.105:0/3276217468 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31403ec70 con 0x7fa320102ad0 2024-03-26T17:46:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.982+0000 7fa32671b640 1 -- 172.21.15.105:0/3276217468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa320074040 con 0x7fa320102ad0 2024-03-26T17:46:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.984+0000 7fa31d7fa640 1 -- 172.21.15.105:0/3276217468 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa314005460 con 0x7fa320102ad0 2024-03-26T17:46:40.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa3293e7740 1 Processor -- start 2024-03-26T17:46:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa3293e7740 1 -- start start 2024-03-26T17:46:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 0x555c88b7de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b85a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa3293e7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555c88ae2970 con 0x555c88b7e420 2024-03-26T17:46:40.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa3293e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555c88a8ad70 con 0x555c88b83590 2024-03-26T17:46:40.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa3293e7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555c88ade920 con 0x555c88b7da30 2024-03-26T17:46:40.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31d7fa640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3080a2f20 0x7fa3080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:40.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b85a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b82f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44150/0 (socket says 172.21.15.105:44150) 2024-03-26T17:46:40.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31ffff640 1 -- 172.21.15.105:0/350637935 learned_addr learned my addr 172.21.15.105:0/350637935 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:40.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa324cb8640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3080a2f20 0x7fa3080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa324cb8640 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 0x555c88b7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:40.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31d7fa640 1 -- 172.21.15.105:0/3276217468 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa3140f5280 con 0x7fa320102ad0 2024-03-26T17:46:40.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31ffff640 1 -- 172.21.15.105:0/350637935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 msgr2=0x555c88b7de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31ffff640 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 0x555c88b7de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31ffff640 1 -- 172.21.15.105:0/350637935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 msgr2=0x555c88b85a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:40.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31ffff640 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b85a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31ffff640 1 -- 172.21.15.105:0/350637935 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c887f63f0 con 0x555c88b7e420 2024-03-26T17:46:40.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.985+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b85a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:40.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.986+0000 7fa31ffff640 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b82f20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa310009ca0 tx=0x7fa310011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632d6fe76b673bbf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.986+0000 7fa324cb8640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3080a2f20 0x7fa3080a53e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x555c88b7ec80 tx=0x7fa318009b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:40.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.986+0000 7fa3067fc640 1 -- 172.21.15.105:0/350637935 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310004220 con 0x555c88b7e420 2024-03-26T17:46:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.986+0000 7fa3067fc640 1 -- 172.21.15.105:0/350637935 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3100043c0 con 0x555c88b7e420 2024-03-26T17:46:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.986+0000 7fa3067fc640 1 -- 172.21.15.105:0/350637935 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100197f0 con 0x555c88b7e420 2024-03-26T17:46:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.986+0000 7fa3293e7740 1 -- 172.21.15.105:0/350637935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 msgr2=0x555c88b82f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.986+0000 7fa3293e7740 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b82f20 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa310009ca0 tx=0x7fa310011810 comp rx=0 tx=0).stop 2024-03-26T17:46:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.986+0000 7fa3293e7740 1 -- 172.21.15.105:0/350637935 shutdown_connections 2024-03-26T17:46:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.987+0000 7fa3293e7740 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b85a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.987+0000 7fa3293e7740 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b82f20 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.987+0000 7fa3293e7740 1 --2- 172.21.15.105:0/350637935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 0x555c88b7de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.987+0000 7fa3293e7740 1 -- 172.21.15.105:0/350637935 >> 172.21.15.105:0/350637935 conn(0x555c88b78fa0 msgr2=0x555c88b793a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.987+0000 7fa3293e7740 1 -- 172.21.15.105:0/350637935 shutdown_connections 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.987+0000 7fa3293e7740 1 -- 172.21.15.105:0/350637935 wait complete. 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.987+0000 7fa3293e7740 1 -- start start 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.987+0000 7fa3293e7740 1 Processor -- start 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b87f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 0x555c88b8f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa3293e7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555c88b86540 con 0x555c88b7e420 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa3293e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555c88a9e310 con 0x555c88b7da30 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa3293e7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555c88b866c0 con 0x555c88b83590 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b87f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa3293e7740 10 client.0 _ll_register_callbacks cb 0x555c88ad9c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa324cb8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 0x555c88b8f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b87f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36218/0 (socket says 172.21.15.105:36218) 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31ffff640 1 -- 172.21.15.105:0/2825753883 learned_addr learned my addr 172.21.15.105:0/2825753883 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31f7fe640 1 -- 172.21.15.105:0/2825753883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 msgr2=0x555c88b8f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 0x555c88b8f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31f7fe640 1 -- 172.21.15.105:0/2825753883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 msgr2=0x555c88b87f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b87f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31f7fe640 1 -- 172.21.15.105:0/2825753883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c888c07d0 con 0x555c88b7e420 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31ffff640 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b87f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b884a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa3140f2d90 tx=0x7fa3140c17c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ddb34bf8d87159 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa2ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/2825753883 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140c2070 con 0x555c88b7e420 2024-03-26T17:46:41.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/2825753883 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa3140f00b0 con 0x555c88b7e420 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.988+0000 7fa3293e7740 1 -- 172.21.15.105:0/2825753883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3100043c0 con 0x555c88b7e420 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/2825753883 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa3140c1970 con 0x555c88b7e420 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa3293e7740 10 client.18435 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa3293e7740 1 -- 172.21.15.105:0/2825753883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c88b92850 con 0x555c88b7e420 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/2825753883 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3100043c0 con 0x555c88b7e420 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/2825753883 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa3140c9ef0 con 0x555c88b7e420 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa3293e7740 10 client.18435 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa3293e7740 10 client.18435 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa3293e7740 20 client.18435 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa3293e7740 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555c88b93e70 0x555c88b962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa3293e7740 4 client.18435 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa3293e7740 1 -- 172.21.15.105:0/2825753883 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c88aefeb0 con 0x555c88b93e70 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.989+0000 7fa31ffff640 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555c88b93e70 0x555c88b962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.990+0000 7fa31ffff640 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555c88b93e70 0x555c88b962c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5037bd64360843c9 server_cookie=372c7e43b77a078d in_seq=0 out_seq=0 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.990+0000 7fa2ff7fe640 10 client.18435 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.990+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/2825753883 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555c88aefeb0 con 0x555c88b93e70 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.990+0000 7fa2ff7fe640 10 client.18435 handle_command_reply: tid=0 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.990+0000 7fa3293e7740 1 client.18435 shutdown 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.990+0000 7fa3293e7740 1 -- 172.21.15.105:0/2825753883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 msgr2=0x555c88b884a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.990+0000 7fa3293e7740 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b884a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa3140f2d90 tx=0x7fa3140c17c0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 1 -- 172.21.15.105:0/2825753883 shutdown_connections 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555c88b93e70 0x555c88b962c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 0x555c88b8f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa2ff7fe640 0 client.18435 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b884a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 1 --2- 172.21.15.105:0/2825753883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b87f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 1 -- 172.21.15.105:0/2825753883 >> 172.21.15.105:0/2825753883 conn(0x555c88b78fa0 msgr2=0x555c88b781c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 1 -- 172.21.15.105:0/2825753883 shutdown_connections 2024-03-26T17:46:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 1 -- 172.21.15.105:0/2825753883 wait complete. 2024-03-26T17:46:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:40.991+0000 7fa3293e7740 20 client.18435 trim_cache size 0 max 16384 2024-03-26T17:46:41.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa3293e7740 1 Processor -- start 2024-03-26T17:46:41.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa3293e7740 1 -- start start 2024-03-26T17:46:41.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 0x555c88b77930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b77e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa3293e7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201a6b30 con 0x555c88b7e420 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa3293e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201a8670 con 0x555c88b83590 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa3293e7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201a7200 con 0x555c88b7da30 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44172/0 (socket says 172.21.15.105:44172) 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa31f7fe640 1 -- 172.21.15.105:0/3499553774 learned_addr learned my addr 172.21.15.105:0/3499553774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa31ffff640 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 0x555c88b77930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.006+0000 7fa324cb8640 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b77e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa31f7fe640 1 -- 172.21.15.105:0/3499553774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 msgr2=0x555c88b77930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 0x555c88b77930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa31f7fe640 1 -- 172.21.15.105:0/3499553774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 msgr2=0x555c88b77e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b77e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa31f7fe640 1 -- 172.21.15.105:0/3499553774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c88b788f0 con 0x555c88b7e420 2024-03-26T17:46:41.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa324cb8640 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b77e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8f4a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa31402c680 tx=0x7fa3140f2dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cb0d978ca6c832 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa305ffb640 1 -- 172.21.15.105:0/3499553774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140bdc10 con 0x555c88b7e420 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa305ffb640 1 -- 172.21.15.105:0/3499553774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3140ca380 con 0x555c88b7e420 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.007+0000 7fa305ffb640 1 -- 172.21.15.105:0/3499553774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140c9ef0 con 0x555c88b7e420 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 -- 172.21.15.105:0/3499553774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 msgr2=0x555c88b8f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8f4a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa31402c680 tx=0x7fa3140f2dc0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 -- 172.21.15.105:0/3499553774 shutdown_connections 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b83590 0x555c88b77e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8f4a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 --2- 172.21.15.105:0/3499553774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b7da30 0x555c88b77930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 -- 172.21.15.105:0/3499553774 >> 172.21.15.105:0/3499553774 conn(0x555c88b78fa0 msgr2=0x555c88b87f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 -- 172.21.15.105:0/3499553774 shutdown_connections 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 -- 172.21.15.105:0/3499553774 wait complete. 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 -- start start 2024-03-26T17:46:41.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.008+0000 7fa3293e7740 1 Processor -- start 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b8cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa3293e7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 0x555c88b8e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44184/0 (socket says 172.21.15.105:44184) 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31f7fe640 1 -- 172.21.15.105:0/4122509777 learned_addr learned my addr 172.21.15.105:0/4122509777 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201a7fa0 con 0x555c88b7e420 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201a78d0 con 0x555c88b7da30 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201a4b10 con 0x555c88b83590 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b8cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa324cb8640 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 0x555c88b8e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa3293e7740 10 client.0 _ll_register_callbacks cb 0x7fa31402c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31f7fe640 1 -- 172.21.15.105:0/4122509777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 msgr2=0x555c88b8e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 0x555c88b8e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31f7fe640 1 -- 172.21.15.105:0/4122509777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 msgr2=0x555c88b8cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b8cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.009+0000 7fa31f7fe640 1 -- 172.21.15.105:0/4122509777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c88b8e8a0 con 0x555c88b7e420 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa31f7fe640 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8d2a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa314004620 tx=0x7fa31403e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44b0a1683de3ab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa2ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/4122509777 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140bdc10 con 0x555c88b7e420 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/4122509777 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa3140e0270 con 0x555c88b7e420 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c88aea200 con 0x555c88b7e420 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/4122509777 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa3140f0190 con 0x555c88b7e420 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 10 client.18441 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c88b8e8a0 con 0x555c88b7e420 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/4122509777 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555c88aea200 con 0x555c88b7e420 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/4122509777 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa3140c0dc0 con 0x555c88b7e420 2024-03-26T17:46:41.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 10 client.18441 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 10 client.18441 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 20 client.18441 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555c88b93e70 0x555c88bfc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 4 client.18441 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.010+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c88a676a0 con 0x555c88b93e70 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa31ffff640 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555c88b93e70 0x555c88bfc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa2ff7fe640 10 client.18441 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa31ffff640 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555c88b93e70 0x555c88bfc990 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25148446379eb791 server_cookie=f690ef47705a9038 in_seq=0 out_seq=0 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa2ff7fe640 1 -- 172.21.15.105:0/4122509777 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c88a676a0 con 0x555c88b93e70 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa2ff7fe640 10 client.18441 handle_command_reply: tid=0 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 client.18441 shutdown 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 msgr2=0x555c88b8d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8d2a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa314004620 tx=0x7fa31403e710 comp rx=0 tx=0).stop 2024-03-26T17:46:41.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 shutdown_connections 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555c88b93e70 0x555c88bfc990 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c88b83590 0x555c88b8e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa2ff7fe640 0 client.18441 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c88b7e420 0x555c88b8d2a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 --2- 172.21.15.105:0/4122509777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c88b7da30 0x555c88b8cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 >> 172.21.15.105:0/4122509777 conn(0x555c88b78fa0 msgr2=0x555c88b77930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.011+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 shutdown_connections 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa3293e7740 1 -- 172.21.15.105:0/4122509777 wait complete. 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa3293e7740 20 client.18441 trim_cache size 0 max 16384 2024-03-26T17:46:41.015 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:41.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 -- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3080a2f20 msgr2=0x7fa3080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3080a2f20 0x7fa3080a53e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x555c88b7ec80 tx=0x7fa318009b10 comp rx=0 tx=0).stop 2024-03-26T17:46:41.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 -- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 msgr2=0x7fa320198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 0x7fa320198090 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa314006010 tx=0x7fa314005e40 comp rx=0 tx=0).stop 2024-03-26T17:46:41.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 -- 172.21.15.105:0/3276217468 shutdown_connections 2024-03-26T17:46:41.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3080a2f20 0x7fa3080a53e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa320103490 0x7fa32019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa320102ad0 0x7fa320198090 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 --2- 172.21.15.105:0/3276217468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3201018e0 0x7fa320197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 -- 172.21.15.105:0/3276217468 >> 172.21.15.105:0/3276217468 conn(0x7fa3200fce50 msgr2=0x7fa320105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 -- 172.21.15.105:0/3276217468 shutdown_connections 2024-03-26T17:46:41.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.012+0000 7fa32671b640 1 -- 172.21.15.105:0/3276217468 wait complete. 2024-03-26T17:46:41.017 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:40 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/260723758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:46:41.018 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:46:41.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 260873 3.14 MB/sec execute 324 sec latency 20.223 ms 2024-03-26T17:46:41.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:40 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/260723758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:46:41.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 -- 172.21.15.105:0/3058603343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81007c0 msgr2=0x7fcca8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 --2- 172.21.15.105:0/3058603343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81007c0 0x7fcca8100be0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009900 tx=0x7fcc9c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 -- 172.21.15.105:0/3058603343 shutdown_connections 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 --2- 172.21.15.105:0/3058603343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8102370 0x7fcca8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 --2- 172.21.15.105:0/3058603343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcca81019b0 0x7fcca8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 --2- 172.21.15.105:0/3058603343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81007c0 0x7fcca8100be0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 -- 172.21.15.105:0/3058603343 >> 172.21.15.105:0/3058603343 conn(0x7fcca80fbd50 msgr2=0x7fcca80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 -- 172.21.15.105:0/3058603343 shutdown_connections 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.145+0000 7fccaeb76640 1 -- 172.21.15.105:0/3058603343 wait complete. 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccaeb76640 1 Processor -- start 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccaeb76640 1 -- start start 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccaeb76640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcca81007c0 0x7fcca8104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccaeb76640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81019b0 0x7fcca8102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccaeb76640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8102370 0x7fcca81032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccaeb76640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810c2c0 con 0x7fcca81019b0 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccaeb76640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810c140 con 0x7fcca81007c0 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccaeb76640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810c440 con 0x7fcca8102370 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81019b0 0x7fcca8102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccac912640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcca81007c0 0x7fcca8104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81019b0 0x7fcca8102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44206/0 (socket says 172.21.15.105:44206) 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 -- 172.21.15.105:0/2405699008 learned_addr learned my addr 172.21.15.105:0/2405699008 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fccad113640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8102370 0x7fcca81032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 -- 172.21.15.105:0/2405699008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8102370 msgr2=0x7fcca81032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8102370 0x7fcca81032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 -- 172.21.15.105:0/2405699008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcca81007c0 msgr2=0x7fcca8104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcca81007c0 0x7fcca8104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 -- 172.21.15.105:0/2405699008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca8103b60 con 0x7fcca81019b0 2024-03-26T17:46:41.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca7fff640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81019b0 0x7fcca8102d90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400e9f0 tx=0x7fcc9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.146+0000 7fcca5ffb640 1 -- 172.21.15.105:0/2405699008 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400cd80 con 0x7fcca81019b0 2024-03-26T17:46:41.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.147+0000 7fcca5ffb640 1 -- 172.21.15.105:0/2405699008 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc940108e0 con 0x7fcca81019b0 2024-03-26T17:46:41.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.147+0000 7fccaeb76640 1 -- 172.21.15.105:0/2405699008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca81a1c70 con 0x7fcca81019b0 2024-03-26T17:46:41.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.147+0000 7fcca5ffb640 1 -- 172.21.15.105:0/2405699008 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94018920 con 0x7fcca81019b0 2024-03-26T17:46:41.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.147+0000 7fccaeb76640 1 -- 172.21.15.105:0/2405699008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81a21c0 con 0x7fcca81019b0 2024-03-26T17:46:41.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.149+0000 7fcca5ffb640 1 -- 172.21.15.105:0/2405699008 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcc94010a90 con 0x7fcca81019b0 2024-03-26T17:46:41.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccb1835740 1 Processor -- start 2024-03-26T17:46:41.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccb1835740 1 -- start start 2024-03-26T17:46:41.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca5ffb640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc980a2f20 0x7fcc980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccb1835740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999246e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccb1835740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c99924bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccb1835740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c99924ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccb1835740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c9991ab970 con 0x55c999246a30 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccb1835740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c999153d70 con 0x55c99924c590 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccb1835740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c9991a7920 con 0x55c999247420 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccad113640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc980a2f20 0x7fcc980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca5ffb640 1 -- 172.21.15.105:0/2405699008 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcc940cf700 con 0x7fcca81019b0 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c99924bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccad113640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c99924ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c99924bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56774/0 (socket says 172.21.15.105:56774) 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccad113640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c99924ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36266/0 (socket says 172.21.15.105:36266) 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca7fff640 1 -- 172.21.15.105:0/1233429937 learned_addr learned my addr 172.21.15.105:0/1233429937 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccac912640 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999246e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca7fff640 1 -- 172.21.15.105:0/1233429937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 msgr2=0x55c99924ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca7fff640 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c99924ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca7fff640 1 -- 172.21.15.105:0/1233429937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 msgr2=0x55c999246e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca7fff640 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999246e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fcca7fff640 1 -- 172.21.15.105:0/1233429937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c998ebf3f0 con 0x55c999247420 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccac912640 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999246e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.150+0000 7fccad113640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc980a2f20 0x7fcc980a53e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcca0005fd0 tx=0x7fcca0009e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fcca7fff640 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c99924bf20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc9409be30 tx=0x7fcc94010de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa68ff8cfd42118 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fcc92ffd640 1 -- 172.21.15.105:0/1233429937 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9409f070 con 0x55c999247420 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fcc92ffd640 1 -- 172.21.15.105:0/1233429937 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc940a4800 con 0x55c999247420 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fcc92ffd640 1 -- 172.21.15.105:0/1233429937 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94098680 con 0x55c999247420 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fccb1835740 1 -- 172.21.15.105:0/1233429937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 msgr2=0x55c99924bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fccb1835740 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c99924bf20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc9409be30 tx=0x7fcc94010de0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fccb1835740 1 -- 172.21.15.105:0/1233429937 shutdown_connections 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fccb1835740 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c99924ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fccb1835740 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c99924bf20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fccb1835740 1 --2- 172.21.15.105:0/1233429937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999246e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.151+0000 7fccb1835740 1 -- 172.21.15.105:0/1233429937 >> 172.21.15.105:0/1233429937 conn(0x55c999241fa0 msgr2=0x55c9992423a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 -- 172.21.15.105:0/1233429937 shutdown_connections 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 -- 172.21.15.105:0/1233429937 wait complete. 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 -- start start 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 Processor -- start 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999250df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c999251330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c999258370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c99924f4e0 con 0x55c999246a30 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c999167310 con 0x55c99924c590 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c99924f660 con 0x55c999247420 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccb1835740 10 client.0 _ll_register_callbacks cb 0x55c9991a2c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccac912640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999250df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccac912640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999250df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44228/0 (socket says 172.21.15.105:44228) 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccac912640 1 -- 172.21.15.105:0/1890542609 learned_addr learned my addr 172.21.15.105:0/1890542609 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fcca7fff640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c999251330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.152+0000 7fccad113640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c999258370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcca7fff640 1 -- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 msgr2=0x55c999258370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcca7fff640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c999258370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcca7fff640 1 -- 172.21.15.105:0/1890542609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 msgr2=0x55c999250df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcca7fff640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999250df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcca7fff640 1 -- 172.21.15.105:0/1890542609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c998f897d0 con 0x55c999247420 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fccac912640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999250df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fccad113640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c999258370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcca7fff640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c999251330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55c999247e90 tx=0x7fcc94095d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3700f128e1ae6c92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcc87fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcc87fff640 1 -- 172.21.15.105:0/1890542609 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9409f070 con 0x55c999247420 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcc87fff640 1 -- 172.21.15.105:0/1890542609 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcc94010430 con 0x55c999247420 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fccb1835740 1 -- 172.21.15.105:0/1890542609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc940a4800 con 0x55c999247420 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fcc87fff640 1 -- 172.21.15.105:0/1890542609 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc94099f40 con 0x55c999247420 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fccb1835740 10 client.28291 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.153+0000 7fccb1835740 1 -- 172.21.15.105:0/1890542609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c99924f550 con 0x55c999247420 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fcc87fff640 1 -- 172.21.15.105:0/1890542609 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc940a4800 con 0x55c999247420 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fcc87fff640 1 -- 172.21.15.105:0/1890542609 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcc94097bf0 con 0x55c999247420 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fccb1835740 10 client.28291 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:41.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fccb1835740 10 client.28291 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fccb1835740 20 client.28291 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fccb1835740 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c99925c970 0x55c99925eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fccb1835740 4 client.28291 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fccb1835740 1 -- 172.21.15.105:0/1890542609 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9991b8eb0 con 0x55c99925c970 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fccac912640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c99925c970 0x55c99925eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fcc87fff640 10 client.28291 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.154+0000 7fccac912640 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c99925c970 0x55c99925eda0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=808cbb4e2c895df3 server_cookie=8ece01b6b26f4047 in_seq=0 out_seq=0 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fcc87fff640 1 -- 172.21.15.105:0/1890542609 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c9991b8eb0 con 0x55c99925c970 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fcc87fff640 10 client.28291 handle_command_reply: tid=0 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 client.28291 shutdown 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 -- 172.21.15.105:0/1890542609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 msgr2=0x55c999251330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c999251330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55c999247e90 tx=0x7fcc94095d30 comp rx=0 tx=0).stop 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 -- 172.21.15.105:0/1890542609 shutdown_connections 2024-03-26T17:46:41.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c99925c970 0x55c99925eda0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c99924c590 0x55c999258370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999247420 0x55c999251330 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fcc87fff640 0 client.28291 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:46:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 --2- 172.21.15.105:0/1890542609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c999246a30 0x55c999250df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 -- 172.21.15.105:0/1890542609 >> 172.21.15.105:0/1890542609 conn(0x55c999241fa0 msgr2=0x55c9992411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 -- 172.21.15.105:0/1890542609 shutdown_connections 2024-03-26T17:46:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.155+0000 7fccb1835740 1 -- 172.21.15.105:0/1890542609 wait complete. 2024-03-26T17:46:41.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.156+0000 7fccb1835740 20 client.28291 trim_cache size 0 max 16384 2024-03-26T17:46:41.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccb1835740 1 Processor -- start 2024-03-26T17:46:41.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccb1835740 1 -- start start 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccb1835740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999240930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccb1835740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999258330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccb1835740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c999240e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccb1835740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81ab0c0 con 0x55c99924c590 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccb1835740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81acc00 con 0x55c999247420 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccb1835740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81ab790 con 0x55c999246a30 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccac912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999240930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccad113640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c999240e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fcca7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999258330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fcca7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999258330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36270/0 (socket says 172.21.15.105:36270) 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccad113640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c999240e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44230/0 (socket says 172.21.15.105:44230) 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fcca7fff640 1 -- 172.21.15.105:0/4048490261 learned_addr learned my addr 172.21.15.105:0/4048490261 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fccac912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999240930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56786/0 (socket says 172.21.15.105:56786) 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fcca7fff640 1 -- 172.21.15.105:0/4048490261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 msgr2=0x55c999240930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fcca7fff640 1 --2- 172.21.15.105:0/4048490261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999240930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fcca7fff640 1 -- 172.21.15.105:0/4048490261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 msgr2=0x55c999240e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fcca7fff640 1 --2- 172.21.15.105:0/4048490261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c999240e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.169+0000 7fcca7fff640 1 -- 172.21.15.105:0/4048490261 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c999246820 con 0x55c999247420 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.170+0000 7fcca7fff640 1 --2- 172.21.15.105:0/4048490261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999258330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc9409bba0 tx=0x7fcc94000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9752cfaa9e36ad8c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.170+0000 7fcc927fc640 1 -- 172.21.15.105:0/4048490261 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9409f040 con 0x55c999247420 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.170+0000 7fcc927fc640 1 -- 172.21.15.105:0/4048490261 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc9409a1a0 con 0x55c999247420 2024-03-26T17:46:41.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.170+0000 7fcc927fc640 1 -- 172.21.15.105:0/4048490261 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94018ac0 con 0x55c999247420 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- 172.21.15.105:0/4048490261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 msgr2=0x55c999258330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 --2- 172.21.15.105:0/4048490261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999258330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc9409bba0 tx=0x7fcc94000bc0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- 172.21.15.105:0/4048490261 shutdown_connections 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 --2- 172.21.15.105:0/4048490261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c999240e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 --2- 172.21.15.105:0/4048490261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999258330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 --2- 172.21.15.105:0/4048490261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999240930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- 172.21.15.105:0/4048490261 >> 172.21.15.105:0/4048490261 conn(0x55c999241fa0 msgr2=0x55c999248880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- 172.21.15.105:0/4048490261 shutdown_connections 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- 172.21.15.105:0/4048490261 wait complete. 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- start start 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 Processor -- start 2024-03-26T17:46:41.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999255020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999255560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c9992c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81ac530 con 0x55c99924c590 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81abe60 con 0x55c999247420 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.171+0000 7fccb1835740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81a90a0 con 0x55c999246a30 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccb1835740 10 client.0 _ll_register_callbacks cb 0x7fcc9409bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c9992c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c9992c0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44232/0 (socket says 172.21.15.105:44232) 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 -- 172.21.15.105:0/2097391474 learned_addr learned my addr 172.21.15.105:0/2097391474 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fcca7fff640 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999255560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccac912640 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999255020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 -- 172.21.15.105:0/2097391474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 msgr2=0x55c999255020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999255020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 -- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 msgr2=0x55c999255560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999255560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 -- 172.21.15.105:0/2097391474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c999255ba0 con 0x55c99924c590 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fcca7fff640 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999255560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccad113640 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c9992c0c90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcca001fb50 tx=0x7fcca001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1e8f04a2054805 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fcc87fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fcc87fff640 1 -- 172.21.15.105:0/2097391474 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0004b20 con 0x55c99924c590 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccb1835740 1 -- 172.21.15.105:0/2097391474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9409a1a0 con 0x55c99924c590 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fcc87fff640 1 -- 172.21.15.105:0/2097391474 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcca001c040 con 0x55c99924c590 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fcc87fff640 1 -- 172.21.15.105:0/2097391474 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcca0004cc0 con 0x55c99924c590 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccb1835740 10 client.18459 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.172+0000 7fccb1835740 1 -- 172.21.15.105:0/2097391474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9992c2360 con 0x55c99924c590 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fcc87fff640 1 -- 172.21.15.105:0/2097391474 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc9409a1a0 con 0x55c99924c590 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fcc87fff640 1 -- 172.21.15.105:0/2097391474 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcca0020050 con 0x55c99924c590 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fccb1835740 10 client.18459 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:41.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fccb1835740 10 client.18459 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fccb1835740 20 client.18459 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fccb1835740 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c9992c2d70 0x55c9992c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fccb1835740 4 client.18459 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fccb1835740 1 -- 172.21.15.105:0/2097391474 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9991b3200 con 0x55c9992c2d70 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fccac912640 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c9992c2d70 0x55c9992c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fcc87fff640 10 client.18459 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.173+0000 7fccac912640 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c9992c2d70 0x55c9992c5190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e84552689d434f server_cookie=1da5db52f12d3f0e in_seq=0 out_seq=0 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fcc87fff640 1 -- 172.21.15.105:0/2097391474 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9991b3200 con 0x55c9992c2d70 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fcc87fff640 10 client.18459 handle_command_reply: tid=0 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 client.18459 shutdown 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 -- 172.21.15.105:0/2097391474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 msgr2=0x55c9992c0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c9992c0c90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcca001fb50 tx=0x7fcca001d000 comp rx=0 tx=0).stop 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 -- 172.21.15.105:0/2097391474 shutdown_connections 2024-03-26T17:46:41.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c99924c590 0x55c9992c0c90 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c9992c2d70 0x55c9992c5190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fcc87fff640 0 client.18459 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c999247420 0x55c999255560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 --2- 172.21.15.105:0/2097391474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c999246a30 0x55c999255020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 -- 172.21.15.105:0/2097391474 >> 172.21.15.105:0/2097391474 conn(0x55c999241fa0 msgr2=0x55c999245200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 -- 172.21.15.105:0/2097391474 shutdown_connections 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 1 -- 172.21.15.105:0/2097391474 wait complete. 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.174+0000 7fccb1835740 20 client.18459 trim_cache size 0 max 16384 2024-03-26T17:46:41.172 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:41.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 -- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc980a2f20 msgr2=0x7fcc980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc980a2f20 0x7fcc980a53e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcca0005fd0 tx=0x7fcca0009e00 comp rx=0 tx=0).stop 2024-03-26T17:46:41.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 -- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81019b0 msgr2=0x7fcca8102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81019b0 0x7fcca8102d90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400e9f0 tx=0x7fcc9400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 -- 172.21.15.105:0/2405699008 shutdown_connections 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcc980a2f20 0x7fcc980a53e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca8102370 0x7fcca81032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcca81019b0 0x7fcca8102d90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 --2- 172.21.15.105:0/2405699008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcca81007c0 0x7fcca8104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 -- 172.21.15.105:0/2405699008 >> 172.21.15.105:0/2405699008 conn(0x7fcca80fbd50 msgr2=0x7fcca8068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 -- 172.21.15.105:0/2405699008 shutdown_connections 2024-03-26T17:46:41.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.175+0000 7fccaeb76640 1 -- 172.21.15.105:0/2405699008 wait complete. 2024-03-26T17:46:41.174 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:40 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/260723758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:46:41.178 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 -- 172.21.15.105:0/4163662482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901007d0 msgr2=0x7f8890100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 --2- 172.21.15.105:0/4163662482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901007d0 0x7f8890100bf0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f888c009900 tx=0x7f888c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 -- 172.21.15.105:0/4163662482 shutdown_connections 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 --2- 172.21.15.105:0/4163662482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8890102380 0x7f8890108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 --2- 172.21.15.105:0/4163662482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88901019c0 0x7f8890101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 --2- 172.21.15.105:0/4163662482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901007d0 0x7f8890100bf0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 -- 172.21.15.105:0/4163662482 >> 172.21.15.105:0/4163662482 conn(0x7f88900fbd60 msgr2=0x7f88900fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 -- 172.21.15.105:0/4163662482 shutdown_connections 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.344+0000 7f889729a640 1 -- 172.21.15.105:0/4163662482 wait complete. 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f889729a640 1 Processor -- start 2024-03-26T17:46:41.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f889729a640 1 -- start start 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f889729a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88901007d0 0x7f8890197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f889729a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901019c0 0x7f88901980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f889729a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8890102380 0x7f889019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f889729a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f889010c300 con 0x7f88901019c0 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f889729a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f889010c180 con 0x7f88901007d0 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f889729a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f889010c480 con 0x7f8890102380 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8894835640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901019c0 0x7f88901980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8894835640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901019c0 0x7f88901980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44244/0 (socket says 172.21.15.105:44244) 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8894835640 1 -- 172.21.15.105:0/334401589 learned_addr learned my addr 172.21.15.105:0/334401589 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8895036640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88901007d0 0x7f8890197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8895837640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8890102380 0x7f889019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8894835640 1 -- 172.21.15.105:0/334401589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8890102380 msgr2=0x7f889019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8894835640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8890102380 0x7f889019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8894835640 1 -- 172.21.15.105:0/334401589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88901007d0 msgr2=0x7f8890197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8894835640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88901007d0 0x7f8890197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8894835640 1 -- 172.21.15.105:0/334401589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889019f7a0 con 0x7f88901019c0 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.345+0000 7f8895036640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88901007d0 0x7f8890197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.346+0000 7f8894835640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901019c0 0x7f88901980c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f888000e9f0 tx=0x7f888000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.346+0000 7f88867fc640 1 -- 172.21.15.105:0/334401589 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888000cd80 con 0x7f88901019c0 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.346+0000 7f88867fc640 1 -- 172.21.15.105:0/334401589 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88800108e0 con 0x7f88901019c0 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.346+0000 7f889729a640 1 -- 172.21.15.105:0/334401589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889019fa90 con 0x7f88901019c0 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.346+0000 7f88867fc640 1 -- 172.21.15.105:0/334401589 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880018920 con 0x7f88901019c0 2024-03-26T17:46:41.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.346+0000 7f889729a640 1 -- 172.21.15.105:0/334401589 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889019ffe0 con 0x7f88901019c0 2024-03-26T17:46:41.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f88867fc640 1 -- 172.21.15.105:0/334401589 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8880010a90 con 0x7f88901019c0 2024-03-26T17:46:41.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f8899f65740 1 Processor -- start 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f8899f65740 1 -- start start 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f8899f65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d232a30 0x55f24d232e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f8899f65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d237f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f8899f65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d238590 0x55f24d23aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f8899f65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f24d197970 con 0x55f24d232a30 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f8899f65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f24d13fd70 con 0x55f24d238590 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f8899f65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f24d193920 con 0x55f24d233420 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.349+0000 7f88867fc640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f887c0a2f20 0x7f887c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d232a30 0x55f24d232e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8894835640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d238590 0x55f24d23aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895036640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d237f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d232a30 0x55f24d232e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44258/0 (socket says 172.21.15.105:44258) 2024-03-26T17:46:41.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 -- 172.21.15.105:0/2362928876 learned_addr learned my addr 172.21.15.105:0/2362928876 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f887c0a2f20 0x7f887c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f88867fc640 1 -- 172.21.15.105:0/334401589 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f88800cf700 con 0x7f88901019c0 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 -- 172.21.15.105:0/2362928876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 msgr2=0x55f24d237f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d237f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 -- 172.21.15.105:0/2362928876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d238590 msgr2=0x55f24d23aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d238590 0x55f24d23aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 -- 172.21.15.105:0/2362928876 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f24ceab3f0 con 0x55f24d232a30 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895036640 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d237f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8894835640 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d238590 0x55f24d23aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f8895837640 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d232a30 0x55f24d232e50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f88880099e0 tx=0x7f8888011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5d3fbcaa92e4da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f887b7fe640 1 -- 172.21.15.105:0/2362928876 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888022020 con 0x55f24d232a30 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f887b7fe640 1 -- 172.21.15.105:0/2362928876 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f888801edc0 con 0x55f24d232a30 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.350+0000 7f887b7fe640 1 -- 172.21.15.105:0/2362928876 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888801daf0 con 0x55f24d232a30 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 -- 172.21.15.105:0/2362928876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d232a30 msgr2=0x55f24d232e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8895837640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f887c0a2f20 0x7f887c0a53e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8888023000 tx=0x7f8888021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d232a30 0x55f24d232e50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f88880099e0 tx=0x7f8888011e20 comp rx=0 tx=0).stop 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 -- 172.21.15.105:0/2362928876 shutdown_connections 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d238590 0x55f24d23aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d237f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 --2- 172.21.15.105:0/2362928876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d232a30 0x55f24d232e50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 -- 172.21.15.105:0/2362928876 >> 172.21.15.105:0/2362928876 conn(0x55f24d22dfa0 msgr2=0x55f24d22e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 -- 172.21.15.105:0/2362928876 shutdown_connections 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.351+0000 7f8899f65740 1 -- 172.21.15.105:0/2362928876 wait complete. 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 1 -- start start 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 1 Processor -- start 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d23d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 0x55f24d2444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f24d23b540 con 0x55f24d233420 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f24d153310 con 0x55f24d232a30 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f24d23b6c0 con 0x55f24d238590 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8899f65740 10 client.0 _ll_register_callbacks cb 0x55f24d18ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8894835640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d23d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8895036640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8894835640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d23d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44262/0 (socket says 172.21.15.105:44262) 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8894835640 1 -- 172.21.15.105:0/349432265 learned_addr learned my addr 172.21.15.105:0/349432265 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8895036640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36314/0 (socket says 172.21.15.105:36314) 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.352+0000 7f8895837640 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 0x55f24d2444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8894835640 1 -- 172.21.15.105:0/349432265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 msgr2=0x55f24d2444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8894835640 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 0x55f24d2444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8894835640 1 -- 172.21.15.105:0/349432265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 msgr2=0x55f24d23cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8894835640 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8894835640 1 -- 172.21.15.105:0/349432265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f24cf757d0 con 0x55f24d233420 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8895036640 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8894835640 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d23d480 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f88800cd1c0 tx=0x7f888006ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0c1eb8cd3a4009 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8878ff9640 1 -- 172.21.15.105:0/349432265 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888009f060 con 0x55f24d233420 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8878ff9640 1 -- 172.21.15.105:0/349432265 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f88800bab60 con 0x55f24d233420 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8899f65740 1 -- 172.21.15.105:0/349432265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f888801edc0 con 0x55f24d233420 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8878ff9640 1 -- 172.21.15.105:0/349432265 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8880026040 con 0x55f24d233420 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8899f65740 10 client.18471 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8899f65740 1 -- 172.21.15.105:0/349432265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f24d247900 con 0x55f24d233420 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8878ff9640 1 -- 172.21.15.105:0/349432265 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f888801edc0 con 0x55f24d233420 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.353+0000 7f8878ff9640 1 -- 172.21.15.105:0/349432265 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8880086bb0 con 0x55f24d233420 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8899f65740 10 client.18471 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8899f65740 10 client.18471 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:46:41.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8899f65740 20 client.18471 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8899f65740 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f24d248fb0 0x55f24d24b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8899f65740 4 client.18471 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8899f65740 1 -- 172.21.15.105:0/349432265 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f24d1a4eb0 con 0x55f24d248fb0 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8895036640 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f24d248fb0 0x55f24d24b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8895036640 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f24d248fb0 0x55f24d24b400 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7e672fcf5776d8c server_cookie=569ca7c433236bbe in_seq=0 out_seq=0 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.354+0000 7f8878ff9640 10 client.18471 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8878ff9640 1 -- 172.21.15.105:0/349432265 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f24d1a4eb0 con 0x55f24d248fb0 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8878ff9640 10 client.18471 handle_command_reply: tid=0 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 client.18471 shutdown 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 -- 172.21.15.105:0/349432265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 msgr2=0x55f24d23d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d23d480 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f88800cd1c0 tx=0x7f888006ba80 comp rx=0 tx=0).stop 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 -- 172.21.15.105:0/349432265 shutdown_connections 2024-03-26T17:46:41.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f24d248fb0 0x55f24d24b400 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 0x55f24d2444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d23d480 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8878ff9640 0 client.18471 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:46:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 --2- 172.21.15.105:0/349432265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 -- 172.21.15.105:0/349432265 >> 172.21.15.105:0/349432265 conn(0x55f24d22dfa0 msgr2=0x55f24d22d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 -- 172.21.15.105:0/349432265 shutdown_connections 2024-03-26T17:46:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.355+0000 7f8899f65740 1 -- 172.21.15.105:0/349432265 wait complete. 2024-03-26T17:46:41.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.356+0000 7f8899f65740 20 client.18471 trim_cache size 0 max 16384 2024-03-26T17:46:41.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.375+0000 7f8899f65740 1 Processor -- start 2024-03-26T17:46:41.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.375+0000 7f8899f65740 1 -- start start 2024-03-26T17:46:41.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.375+0000 7f8899f65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.375+0000 7f8899f65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d23d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8899f65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d238590 0x55f24d244480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8894835640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d23d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8899f65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88901a8e30 con 0x55f24d238590 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8895837640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d238590 0x55f24d244480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8894835640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d23d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56856/0 (socket says 172.21.15.105:56856) 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8895036640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8894835640 1 -- 172.21.15.105:0/3440678385 learned_addr learned my addr 172.21.15.105:0/3440678385 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8899f65740 1 -- 172.21.15.105:0/3440678385 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88901aa970 con 0x55f24d232a30 2024-03-26T17:46:41.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8899f65740 1 -- 172.21.15.105:0/3440678385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88901a9500 con 0x55f24d233420 2024-03-26T17:46:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8894835640 1 -- 172.21.15.105:0/3440678385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 msgr2=0x55f24d23cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8894835640 1 --2- 172.21.15.105:0/3440678385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8894835640 1 -- 172.21.15.105:0/3440678385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d238590 msgr2=0x55f24d244480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8894835640 1 --2- 172.21.15.105:0/3440678385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d238590 0x55f24d244480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.376+0000 7f8894835640 1 -- 172.21.15.105:0/3440678385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f24d23c2f0 con 0x55f24d233420 2024-03-26T17:46:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.377+0000 7f8894835640 1 --2- 172.21.15.105:0/3440678385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d23d480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8880098870 tx=0x7f888009a3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee4b90fd0d0d26b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.377+0000 7f887affd640 1 -- 172.21.15.105:0/3440678385 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888006b030 con 0x55f24d233420 2024-03-26T17:46:41.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.377+0000 7f887affd640 1 -- 172.21.15.105:0/3440678385 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8880010de0 con 0x55f24d233420 2024-03-26T17:46:41.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.377+0000 7f887affd640 1 -- 172.21.15.105:0/3440678385 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888009b2f0 con 0x55f24d233420 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 -- 172.21.15.105:0/3440678385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 msgr2=0x55f24d23d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 --2- 172.21.15.105:0/3440678385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d23d480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8880098870 tx=0x7f888009a3b0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 -- 172.21.15.105:0/3440678385 shutdown_connections 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 --2- 172.21.15.105:0/3440678385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d238590 0x55f24d244480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 --2- 172.21.15.105:0/3440678385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d233420 0x55f24d23d480 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 --2- 172.21.15.105:0/3440678385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d23cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 -- 172.21.15.105:0/3440678385 >> 172.21.15.105:0/3440678385 conn(0x55f24d22dfa0 msgr2=0x55f24d22e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 -- 172.21.15.105:0/3440678385 shutdown_connections 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.378+0000 7f8899f65740 1 -- 172.21.15.105:0/3440678385 wait complete. 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 1 -- start start 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 1 Processor -- start 2024-03-26T17:46:41.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d2444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d240fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 0x55f24d2415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88901aa2a0 con 0x55f24d233420 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88901a9bd0 con 0x55f24d232a30 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88901a6e10 con 0x55f24d238590 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8899f65740 10 client.0 _ll_register_callbacks cb 0x7f888000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8895036640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d2444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8895036640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d2444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36330/0 (socket says 172.21.15.105:36330) 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.379+0000 7f8895036640 1 -- 172.21.15.105:0/763973836 learned_addr learned my addr 172.21.15.105:0/763973836 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8895837640 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 0x55f24d2415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8894835640 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d240fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8895036640 1 -- 172.21.15.105:0/763973836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 msgr2=0x55f24d2415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8895036640 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 0x55f24d2415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8895036640 1 -- 172.21.15.105:0/763973836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 msgr2=0x55f24d240fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8895036640 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d240fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8895036640 1 -- 172.21.15.105:0/763973836 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f24d2b5ae0 con 0x55f24d232a30 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8895036640 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d2444f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f888c005ab0 tx=0x7f888c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc219b3a344f65f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.380+0000 7f8878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8878ff9640 1 -- 172.21.15.105:0/763973836 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c04c030 con 0x55f24d232a30 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8878ff9640 1 -- 172.21.15.105:0/763973836 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f888c03a040 con 0x55f24d232a30 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8899f65740 1 -- 172.21.15.105:0/763973836 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f24d19f200 con 0x55f24d232a30 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8878ff9640 1 -- 172.21.15.105:0/763973836 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f888c056050 con 0x55f24d232a30 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8899f65740 10 client.28310 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:41.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8899f65740 1 -- 172.21.15.105:0/763973836 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f24d2b6e50 con 0x55f24d232a30 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8878ff9640 1 -- 172.21.15.105:0/763973836 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f24d19f200 con 0x55f24d232a30 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8878ff9640 1 -- 172.21.15.105:0/763973836 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f888c005470 con 0x55f24d232a30 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8899f65740 10 client.28310 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8899f65740 10 client.28310 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.381+0000 7f8899f65740 20 client.28310 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.382+0000 7f8899f65740 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f24d248fb0 0x55f24d2b9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.382+0000 7f8899f65740 4 client.28310 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.382+0000 7f8899f65740 1 -- 172.21.15.105:0/763973836 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f24d11c6a0 con 0x55f24d248fb0 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.382+0000 7f8894835640 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f24d248fb0 0x55f24d2b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.382+0000 7f8878ff9640 10 client.28310 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.382+0000 7f8894835640 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f24d248fb0 0x55f24d2b9840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f177a05783a4e99d server_cookie=7c9f758f1f42320f in_seq=0 out_seq=0 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8878ff9640 1 -- 172.21.15.105:0/763973836 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f24d11c6a0 con 0x55f24d248fb0 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8878ff9640 10 client.28310 handle_command_reply: tid=0 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 client.28310 shutdown 2024-03-26T17:46:41.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 -- 172.21.15.105:0/763973836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 msgr2=0x55f24d2444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d2444f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f888c005ab0 tx=0x7f888c004610 comp rx=0 tx=0).stop 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 -- 172.21.15.105:0/763973836 shutdown_connections 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f24d248fb0 0x55f24d2b9840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f24d238590 0x55f24d2415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f24d233420 0x55f24d240fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8878ff9640 0 client.28310 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 --2- 172.21.15.105:0/763973836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f24d232a30 0x55f24d2444f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 -- 172.21.15.105:0/763973836 >> 172.21.15.105:0/763973836 conn(0x55f24d22dfa0 msgr2=0x55f24d234920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 -- 172.21.15.105:0/763973836 shutdown_connections 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 1 -- 172.21.15.105:0/763973836 wait complete. 2024-03-26T17:46:41.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.383+0000 7f8899f65740 20 client.28310 trim_cache size 0 max 16384 2024-03-26T17:46:41.378 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:41.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 -- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f887c0a2f20 msgr2=0x7f887c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f887c0a2f20 0x7f887c0a53e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8888023000 tx=0x7f8888021000 comp rx=0 tx=0).stop 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 -- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901019c0 msgr2=0x7f88901980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901019c0 0x7f88901980c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f888000e9f0 tx=0x7f888000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 -- 172.21.15.105:0/334401589 shutdown_connections 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f887c0a2f20 0x7f887c0a53e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8890102380 0x7f889019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88901019c0 0x7f88901980c0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 --2- 172.21.15.105:0/334401589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88901007d0 0x7f8890197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 -- 172.21.15.105:0/334401589 >> 172.21.15.105:0/334401589 conn(0x7f88900fbd60 msgr2=0x7f8890069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 -- 172.21.15.105:0/334401589 shutdown_connections 2024-03-26T17:46:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.384+0000 7f889729a640 1 -- 172.21.15.105:0/334401589 wait complete. 2024-03-26T17:46:41.388 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:46:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 -- 172.21.15.105:0/1401611186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee81007c0 msgr2=0x7f7ee8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 --2- 172.21.15.105:0/1401611186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee81007c0 0x7f7ee8100be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009860 tx=0x7f7ed802bb70 comp rx=0 tx=0).stop 2024-03-26T17:46:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 -- 172.21.15.105:0/1401611186 shutdown_connections 2024-03-26T17:46:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 --2- 172.21.15.105:0/1401611186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee8102370 0x7f7ee8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 --2- 172.21.15.105:0/1401611186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ee81019b0 0x7f7ee8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 --2- 172.21.15.105:0/1401611186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee81007c0 0x7f7ee8100be0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 -- 172.21.15.105:0/1401611186 >> 172.21.15.105:0/1401611186 conn(0x7f7ee80fbd50 msgr2=0x7f7ee80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 -- 172.21.15.105:0/1401611186 shutdown_connections 2024-03-26T17:46:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 -- 172.21.15.105:0/1401611186 wait complete. 2024-03-26T17:46:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.516+0000 7f7eedd59640 1 Processor -- start 2024-03-26T17:46:41.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7eedd59640 1 -- start start 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7eedd59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ee81007c0 0x7f7ee8196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7eedd59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee81019b0 0x7f7ee8196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7eedd59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee8102370 0x7f7ee819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7eedd59640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee810c2c0 con 0x7f7ee81019b0 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7eedd59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee810c140 con 0x7f7ee8102370 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7eedd59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee810c440 con 0x7f7ee81007c0 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee81019b0 0x7f7ee8196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ee81007c0 0x7f7ee8196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee8102370 0x7f7ee819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ee81007c0 0x7f7ee8196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56890/0 (socket says 172.21.15.105:56890) 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee81019b0 0x7f7ee8196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44294/0 (socket says 172.21.15.105:44294) 2024-03-26T17:46:41.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee77fe640 1 -- 172.21.15.105:0/549886193 learned_addr learned my addr 172.21.15.105:0/549886193 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee7fff640 1 -- 172.21.15.105:0/549886193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ee81007c0 msgr2=0x7f7ee8196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee7fff640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ee81007c0 0x7f7ee8196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee7fff640 1 -- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee81019b0 msgr2=0x7f7ee8196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee7fff640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee81019b0 0x7f7ee8196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee7fff640 1 -- 172.21.15.105:0/549886193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee8068a20 con 0x7f7ee8102370 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.517+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ee81007c0 0x7f7ee8196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.518+0000 7f7ee7fff640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee8102370 0x7f7ee819dfe0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009fd0 tx=0x7f7ee000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.518+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/549886193 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee000cd60 con 0x7f7ee8102370 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.518+0000 7f7eedd59640 1 -- 172.21.15.105:0/549886193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee8068d10 con 0x7f7ee8102370 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.518+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/549886193 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee0005020 con 0x7f7ee8102370 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.518+0000 7f7eedd59640 1 -- 172.21.15.105:0/549886193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee8069210 con 0x7f7ee8102370 2024-03-26T17:46:41.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.518+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/549886193 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0018670 con 0x7f7ee8102370 2024-03-26T17:46:41.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.520+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/549886193 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7ee0018810 con 0x7f7ee8102370 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.520+0000 7f7ef0a15740 1 Processor -- start 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.520+0000 7f7ef0a15740 1 -- start start 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0da30 0x55c063a0de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a12f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 0x55c063a15a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c063972970 con 0x55c063a0da30 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c06391ad70 con 0x55c063a0e420 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c06396e920 con 0x55c063a13590 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0da30 0x55c063a0de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a12f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee4ff9640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ed00a2f20 0x7f7ed00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a12f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36368/0 (socket says 172.21.15.105:36368) 2024-03-26T17:46:41.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2840745864 learned_addr learned my addr 172.21.15.105:0/2840745864 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/2840745864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 0x55c063a15a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2840745864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 msgr2=0x55c063a15a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/2840745864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 0x55c063a15a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee7fff640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ed00a2f20 0x7f7ed00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2840745864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0da30 msgr2=0x55c063a0de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/2840745864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0da30 0x55c063a0de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.521+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2840745864 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0636863f0 con 0x55c063a0e420 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.522+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/549886193 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7ee00cfae0 con 0x7f7ee8102370 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.522+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/2840745864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a12f20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8000c00 tx=0x7f7ed8005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d89fddd86cbf58c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.522+0000 7f7ecdffb640 1 -- 172.21.15.105:0/2840745864 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8039040 con 0x55c063a0e420 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.522+0000 7f7ee7fff640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ed00a2f20 0x7f7ed00a53e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0098b40 tx=0x7f7ee0091650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.522+0000 7f7ecdffb640 1 -- 172.21.15.105:0/2840745864 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ed8034e40 con 0x55c063a0e420 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.522+0000 7f7ecdffb640 1 -- 172.21.15.105:0/2840745864 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8003850 con 0x55c063a0e420 2024-03-26T17:46:41.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2840745864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 msgr2=0x55c063a12f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/2840745864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a12f20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8000c00 tx=0x7f7ed8005dc0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2840745864 shutdown_connections 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/2840745864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 0x55c063a15a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/2840745864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a12f20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/2840745864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0da30 0x55c063a0de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2840745864 >> 172.21.15.105:0/2840745864 conn(0x55c063a08fa0 msgr2=0x55c063a093a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2840745864 shutdown_connections 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2840745864 wait complete. 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- start start 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 Processor -- start 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a13590 0x55c063a1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c063a164e0 con 0x55c063a0e420 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c06392e310 con 0x55c063a13590 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.523+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c063a16660 con 0x55c063a0da30 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ef0a15740 10 client.0 _ll_register_callbacks cb 0x55c063969c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44316/0 (socket says 172.21.15.105:44316) 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56898/0 (socket says 172.21.15.105:56898) 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 -- 172.21.15.105:0/3382527434 learned_addr learned my addr 172.21.15.105:0/3382527434 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a13590 0x55c063a1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 -- 172.21.15.105:0/3382527434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 msgr2=0x55c063a17f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 -- 172.21.15.105:0/3382527434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a13590 msgr2=0x55c063a1f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a13590 0x55c063a1f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 -- 172.21.15.105:0/3382527434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0637507d0 con 0x55c063a0e420 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009830 tx=0x7f7ed802c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7386541722d33c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ec6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/3382527434 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8039040 con 0x55c063a0e420 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/3382527434 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7ed802c510 con 0x55c063a0e420 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.524+0000 7f7ef0a15740 1 -- 172.21.15.105:0/3382527434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c06397feb0 con 0x55c063a0e420 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/3382527434 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ed80035f0 con 0x55c063a0e420 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ef0a15740 10 client.18489 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ef0a15740 1 -- 172.21.15.105:0/3382527434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c063a22810 con 0x55c063a0e420 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/3382527434 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c06397feb0 con 0x55c063a0e420 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/3382527434 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed805a560 con 0x55c063a0e420 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ef0a15740 10 client.18489 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ef0a15740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ef0a15740 20 client.18489 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:41.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c063a23b60 0x55c063a25f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ef0a15740 4 client.18489 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ef0a15740 1 -- 172.21.15.105:0/3382527434 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c06397a200 con 0x55c063a23b60 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.525+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c063a23b60 0x55c063a25f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ec6ffd640 10 client.18489 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c063a23b60 0x55c063a25f50 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=453a777eee0dd552 server_cookie=da1f7bfa298e7624 in_seq=0 out_seq=0 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/3382527434 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c06397a200 con 0x55c063a23b60 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ec6ffd640 10 client.18489 handle_command_reply: tid=0 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ef0a15740 1 client.18489 shutdown 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ef0a15740 1 -- 172.21.15.105:0/3382527434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 msgr2=0x55c063a18460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009830 tx=0x7f7ed802c770 comp rx=0 tx=0).stop 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ef0a15740 1 -- 172.21.15.105:0/3382527434 shutdown_connections 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.526+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c063a23b60 0x55c063a25f50 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.527+0000 7f7ec6ffd640 0 client.18489 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:41.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.527+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a13590 0x55c063a1f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.527+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.527+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/3382527434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.527+0000 7f7ef0a15740 1 -- 172.21.15.105:0/3382527434 >> 172.21.15.105:0/3382527434 conn(0x55c063a08fa0 msgr2=0x55c063a081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.527+0000 7f7ef0a15740 1 -- 172.21.15.105:0/3382527434 shutdown_connections 2024-03-26T17:46:41.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.527+0000 7f7ef0a15740 1 -- 172.21.15.105:0/3382527434 wait complete. 2024-03-26T17:46:41.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.527+0000 7f7ef0a15740 20 client.18489 trim_cache size 0 max 16384 2024-03-26T17:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ef0a15740 1 Processor -- start 2024-03-26T17:46:41.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ef0a15740 1 -- start start 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 0x55c063a1f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81a6b60 con 0x55c063a0e420 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81a86a0 con 0x55c063a0da30 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81a7230 con 0x55c063a13590 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44330/0 (socket says 172.21.15.105:44330) 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2769073753 learned_addr learned my addr 172.21.15.105:0/2769073753 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 0x55c063a1f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2769073753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 msgr2=0x55c063a1f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 0x55c063a1f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2769073753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0da30 msgr2=0x55c063a17f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2769073753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c063a262c0 con 0x55c063a0e420 2024-03-26T17:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.541+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.542+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005b60 tx=0x7f7ed8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e343e953738f28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.542+0000 7f7ecd7fa640 1 -- 172.21.15.105:0/2769073753 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed805f070 con 0x55c063a0e420 2024-03-26T17:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.542+0000 7f7ecd7fa640 1 -- 172.21.15.105:0/2769073753 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ed802c510 con 0x55c063a0e420 2024-03-26T17:46:41.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.542+0000 7f7ecd7fa640 1 -- 172.21.15.105:0/2769073753 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8039040 con 0x55c063a0e420 2024-03-26T17:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2769073753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 msgr2=0x55c063a18460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005b60 tx=0x7f7ed8009510 comp rx=0 tx=0).stop 2024-03-26T17:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2769073753 shutdown_connections 2024-03-26T17:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a13590 0x55c063a1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a0e420 0x55c063a18460 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/2769073753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0da30 0x55c063a17f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2769073753 >> 172.21.15.105:0/2769073753 conn(0x55c063a08fa0 msgr2=0x55c063a09380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2769073753 shutdown_connections 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- 172.21.15.105:0/2769073753 wait complete. 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- start start 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 Processor -- start 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a1cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a13590 0x55c063a1e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81a7fd0 con 0x55c063a13590 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81a7900 con 0x55c063a0e420 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.543+0000 7f7ef0a15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81a4b40 con 0x55c063a0da30 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ef0a15740 10 client.0 _ll_register_callbacks cb 0x7f7edc0108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a13590 0x55c063a1e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a13590 0x55c063a1e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44340/0 (socket says 172.21.15.105:44340) 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 -- 172.21.15.105:0/585258617 learned_addr learned my addr 172.21.15.105:0/585258617 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a1cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 -- 172.21.15.105:0/585258617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 msgr2=0x55c063a1cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a1cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 -- 172.21.15.105:0/585258617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 msgr2=0x55c063a1d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a1d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 -- 172.21.15.105:0/585258617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c063a1e620 con 0x55c063a13590 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ee6ffd640 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a13590 0x55c063a1e030 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x55c063a266d0 tx=0x7f7edc0299c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed3707bdbaebb96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ec6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/585258617 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0040f0 con 0x55c063a13590 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/585258617 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7edc019040 con 0x55c063a13590 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ef0a15740 1 -- 172.21.15.105:0/585258617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed802c510 con 0x55c063a13590 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/585258617 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7edc004290 con 0x55c063a13590 2024-03-26T17:46:41.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.544+0000 7f7ef0a15740 10 client.18495 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ef0a15740 1 -- 172.21.15.105:0/585258617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c063a89d60 con 0x55c063a13590 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/585258617 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ed802c510 con 0x55c063a13590 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/585258617 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7edc01f040 con 0x55c063a13590 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ef0a15740 10 client.18495 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ef0a15740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ef0a15740 20 client.18495 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c063a8a700 0x55c063a8cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ef0a15740 4 client.18495 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ef0a15740 1 -- 172.21.15.105:0/585258617 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0638f76a0 con 0x55c063a8a700 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c063a8a700 0x55c063a8cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ec6ffd640 10 client.18495 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.545+0000 7f7ee77fe640 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c063a8a700 0x55c063a8cb20 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7610c17bb3dc2332 server_cookie=3498cb086410c335 in_seq=0 out_seq=0 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ec6ffd640 1 -- 172.21.15.105:0/585258617 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0638f76a0 con 0x55c063a8a700 2024-03-26T17:46:41.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ec6ffd640 10 client.18495 handle_command_reply: tid=0 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 client.18495 shutdown 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 -- 172.21.15.105:0/585258617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a13590 msgr2=0x55c063a1e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a13590 0x55c063a1e030 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x55c063a266d0 tx=0x7f7edc0299c0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 -- 172.21.15.105:0/585258617 shutdown_connections 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c063a8a700 0x55c063a8cb20 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c063a13590 0x55c063a1e030 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c063a0e420 0x55c063a1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 --2- 172.21.15.105:0/585258617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c063a0da30 0x55c063a1cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 -- 172.21.15.105:0/585258617 >> 172.21.15.105:0/585258617 conn(0x55c063a08fa0 msgr2=0x55c063a17ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ec6ffd640 0 client.18495 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 -- 172.21.15.105:0/585258617 shutdown_connections 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 1 -- 172.21.15.105:0/585258617 wait complete. 2024-03-26T17:46:41.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.546+0000 7f7ef0a15740 20 client.18495 trim_cache size 0 max 16384 2024-03-26T17:46:41.544 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 -- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ed00a2f20 msgr2=0x7f7ed00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ed00a2f20 0x7f7ed00a53e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0098b40 tx=0x7f7ee0091650 comp rx=0 tx=0).stop 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 -- 172.21.15.105:0/549886193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee8102370 msgr2=0x7f7ee819dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee8102370 0x7f7ee819dfe0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009fd0 tx=0x7f7ee000ee50 comp rx=0 tx=0).stop 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 -- 172.21.15.105:0/549886193 shutdown_connections 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ed00a2f20 0x7f7ed00a53e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ee8102370 0x7f7ee819dfe0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ee81019b0 0x7f7ee8196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 --2- 172.21.15.105:0/549886193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ee81007c0 0x7f7ee8196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 -- 172.21.15.105:0/549886193 >> 172.21.15.105:0/549886193 conn(0x7f7ee80fbd50 msgr2=0x7f7ee8104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 -- 172.21.15.105:0/549886193 shutdown_connections 2024-03-26T17:46:41.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:41.547+0000 7f7eedd59640 1 -- 172.21.15.105:0/549886193 wait complete. 2024-03-26T17:46:42.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 261430 3.14 MB/sec execute 325 sec latency 18.834 ms 2024-03-26T17:46:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:41 smithi144 ceph-mon[36095]: pgmap v598: 97 pgs: 97 active+clean; 29 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:46:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:41 smithi105 ceph-mon[32280]: pgmap v598: 97 pgs: 97 active+clean; 29 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:46:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:41 smithi135 ceph-mon[35636]: pgmap v598: 97 pgs: 97 active+clean; 29 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:46:42.550 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:46:42.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:46:42.550 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 -- 172.21.15.105:0/3686534837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301007c0 msgr2=0x7fea30100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 --2- 172.21.15.105:0/3686534837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301007c0 0x7fea30100be0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fea20009900 tx=0x7fea2002bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 -- 172.21.15.105:0/3686534837 shutdown_connections 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 --2- 172.21.15.105:0/3686534837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea30102370 0x7fea30108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 --2- 172.21.15.105:0/3686534837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea301019b0 0x7fea30101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 --2- 172.21.15.105:0/3686534837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301007c0 0x7fea30100be0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 -- 172.21.15.105:0/3686534837 >> 172.21.15.105:0/3686534837 conn(0x7fea300fbd50 msgr2=0x7fea300fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 -- 172.21.15.105:0/3686534837 shutdown_connections 2024-03-26T17:46:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.678+0000 7fea35b7e640 1 -- 172.21.15.105:0/3686534837 wait complete. 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea35b7e640 1 Processor -- start 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea35b7e640 1 -- start start 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea35b7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea301007c0 0x7fea30069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea35b7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301019b0 0x7fea3019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea35b7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea30102370 0x7fea301a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea35b7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea3010c2f0 con 0x7fea301019b0 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea35b7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea3010c170 con 0x7fea301007c0 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea35b7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea3010c470 con 0x7fea30102370 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301019b0 0x7fea3019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea301007c0 0x7fea30069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea2effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301019b0 0x7fea3019eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44360/0 (socket says 172.21.15.105:44360) 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea301007c0 0x7fea30069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36418/0 (socket says 172.21.15.105:36418) 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea30102370 0x7fea301a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea2effd640 1 -- 172.21.15.105:0/2062699864 learned_addr learned my addr 172.21.15.105:0/2062699864 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea2ffff640 1 -- 172.21.15.105:0/2062699864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea301007c0 msgr2=0x7fea30069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.679+0000 7fea2ffff640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea301007c0 0x7fea30069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea2ffff640 1 -- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301019b0 msgr2=0x7fea3019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea2ffff640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301019b0 0x7fea3019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea2ffff640 1 -- 172.21.15.105:0/2062699864 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea301a19e0 con 0x7fea30102370 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea2effd640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301019b0 0x7fea3019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea2ffff640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea30102370 0x7fea301a11f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fea28009fd0 tx=0x7fea2800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea2cff9640 1 -- 172.21.15.105:0/2062699864 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2800ce20 con 0x7fea30102370 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea2cff9640 1 -- 172.21.15.105:0/2062699864 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea28005020 con 0x7fea30102370 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea35b7e640 1 -- 172.21.15.105:0/2062699864 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea301a1c70 con 0x7fea30102370 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea2cff9640 1 -- 172.21.15.105:0/2062699864 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28010690 con 0x7fea30102370 2024-03-26T17:46:42.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.680+0000 7fea35b7e640 1 -- 172.21.15.105:0/2062699864 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea301a21c0 con 0x7fea30102370 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.682+0000 7fea2cff9640 1 -- 172.21.15.105:0/2062699864 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fea280051d0 con 0x7fea30102370 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea3876a740 1 Processor -- start 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea3876a740 1 -- start start 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2cff9640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea180a2f50 0x7fea180a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea3876a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea3876a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 0x559f11b60e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea3876a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b63990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea3876a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f11ab8630 con 0x559f11b5b9a0 2024-03-26T17:46:42.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea3876a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559f11a60e10 con 0x559f11b61500 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea3876a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f11ab45e0 con 0x559f11b5c390 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2effd640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea180a2f50 0x7fea180a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b5bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2cff9640 1 -- 172.21.15.105:0/2062699864 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fea280cfa40 con 0x7fea30102370 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b63990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b5bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44362/0 (socket says 172.21.15.105:44362) 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b63990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36428/0 (socket says 172.21.15.105:36428) 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2f7fe640 1 -- 172.21.15.105:0/56389390 learned_addr learned my addr 172.21.15.105:0/56389390 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.683+0000 7fea2effd640 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 0x559f11b60e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea2ffff640 1 -- 172.21.15.105:0/56389390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 msgr2=0x559f11b60e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea2ffff640 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 0x559f11b60e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea2ffff640 1 -- 172.21.15.105:0/56389390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 msgr2=0x559f11b5bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea2ffff640 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b5bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea2ffff640 1 -- 172.21.15.105:0/56389390 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f11a77740 con 0x559f11b61500 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b5bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea2effd640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea180a2f50 0x7fea180a5410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fea240097f0 tx=0x7fea24013d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea2ffff640 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b63990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fea280cd500 tx=0x7fea280175c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b325ce5821897fa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea15ffb640 1 -- 172.21.15.105:0/56389390 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28007b00 con 0x559f11b61500 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea15ffb640 1 -- 172.21.15.105:0/56389390 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea2809c830 con 0x559f11b61500 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.684+0000 7fea15ffb640 1 -- 172.21.15.105:0/56389390 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea280a0280 con 0x559f11b61500 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 -- 172.21.15.105:0/56389390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 msgr2=0x559f11b63990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b63990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fea280cd500 tx=0x7fea280175c0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 -- 172.21.15.105:0/56389390 shutdown_connections 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b63990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 0x559f11b60e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 --2- 172.21.15.105:0/56389390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b5bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 -- 172.21.15.105:0/56389390 >> 172.21.15.105:0/56389390 conn(0x559f11b4e700 msgr2=0x559f11b4eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 -- 172.21.15.105:0/56389390 shutdown_connections 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 -- 172.21.15.105:0/56389390 wait complete. 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.685+0000 7fea3876a740 1 -- start start 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea3876a740 1 Processor -- start 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea3876a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea3876a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b66420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea3876a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b61500 0x559f11b6d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea3876a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f11b644b0 con 0x559f11b5b9a0 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea3876a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559f11a741a0 con 0x559f11b5c390 2024-03-26T17:46:42.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea3876a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f11b64630 con 0x559f11b61500 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea3876a740 10 client.0 _ll_register_callbacks cb 0x559f117cbc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44370/0 (socket says 172.21.15.105:44370) 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b66420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2f7fe640 1 -- 172.21.15.105:0/3625211700 learned_addr learned my addr 172.21.15.105:0/3625211700 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2ffff640 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b61500 0x559f11b6d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2f7fe640 1 -- 172.21.15.105:0/3625211700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b61500 msgr2=0x559f11b6d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b61500 0x559f11b6d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2f7fe640 1 -- 172.21.15.105:0/3625211700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 msgr2=0x559f11b66420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b66420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2f7fe640 1 -- 172.21.15.105:0/3625211700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f118947e0 con 0x559f11b5b9a0 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.686+0000 7fea2ffff640 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b61500 0x559f11b6d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65ee0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fea20036040 tx=0x7fea20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce384646e83e098b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea0effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:42.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea0effd640 1 -- 172.21.15.105:0/3625211700 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20005d50 con 0x559f11b5b9a0 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea0effd640 1 -- 172.21.15.105:0/3625211700 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fea20005030 con 0x559f11b5b9a0 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 1 -- 172.21.15.105:0/3625211700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2809c830 con 0x559f11b5b9a0 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea0effd640 1 -- 172.21.15.105:0/3625211700 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fea20051050 con 0x559f11b5b9a0 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 10 client.18507 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 1 -- 172.21.15.105:0/3625211700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f118f4f70 con 0x559f11b5b9a0 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea0effd640 1 -- 172.21.15.105:0/3625211700 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea200046c0 con 0x559f11b5b9a0 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea0effd640 1 -- 172.21.15.105:0/3625211700 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fea200035f0 con 0x559f11b5b9a0 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 10 client.18507 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 20 client.18507 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f11b71d90 0x559f11b74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 4 client.18507 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:42.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.687+0000 7fea3876a740 1 -- 172.21.15.105:0/3625211700 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fea2809c830 con 0x559f11b71d90 2024-03-26T17:46:42.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.688+0000 7fea2effd640 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f11b71d90 0x559f11b74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.688+0000 7fea0effd640 10 client.18507 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:42.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.688+0000 7fea2effd640 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f11b71d90 0x559f11b74210 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6dac0ea0c2dcf5 server_cookie=6d91ff1261bbeb7c in_seq=0 out_seq=0 2024-03-26T17:46:42.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea0effd640 1 -- 172.21.15.105:0/3625211700 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fea2809c830 con 0x559f11b71d90 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea0effd640 10 client.18507 handle_command_reply: tid=0 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 client.18507 shutdown 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 -- 172.21.15.105:0/3625211700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 msgr2=0x559f11b65ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65ee0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fea20036040 tx=0x7fea20004270 comp rx=0 tx=0).stop 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 -- 172.21.15.105:0/3625211700 shutdown_connections 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f11b71d90 0x559f11b74210 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b61500 0x559f11b6d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b66420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 --2- 172.21.15.105:0/3625211700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65ee0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 -- 172.21.15.105:0/3625211700 >> 172.21.15.105:0/3625211700 conn(0x559f11b4e700 msgr2=0x559f11b4d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea0effd640 0 client.18507 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 -- 172.21.15.105:0/3625211700 shutdown_connections 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 1 -- 172.21.15.105:0/3625211700 wait complete. 2024-03-26T17:46:42.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.689+0000 7fea3876a740 20 client.18507 trim_cache size 0 max 16384 2024-03-26T17:46:42.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea3876a740 1 Processor -- start 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea3876a740 1 -- start start 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea3876a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea3876a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 0x559f11b6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea3876a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b65a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea3876a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea301a6c20 con 0x559f11b5b9a0 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea3876a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea301a8760 con 0x559f11b61500 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea3876a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea301a72f0 con 0x559f11b5c390 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44374/0 (socket says 172.21.15.105:44374) 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2f7fe640 1 -- 172.21.15.105:0/3251331019 learned_addr learned my addr 172.21.15.105:0/3251331019 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2ffff640 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b65a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2effd640 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 0x559f11b6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2f7fe640 1 -- 172.21.15.105:0/3251331019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 msgr2=0x559f11b6d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 0x559f11b6d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2f7fe640 1 -- 172.21.15.105:0/3251331019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 msgr2=0x559f11b65a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b65a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.709+0000 7fea2f7fe640 1 -- 172.21.15.105:0/3251331019 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f1178c170 con 0x559f11b5b9a0 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.710+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65500 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fea20003ba0 tx=0x7fea2002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac2fddd2446280b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.710+0000 7fea157fa640 1 -- 172.21.15.105:0/3251331019 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20005d50 con 0x559f11b5b9a0 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.710+0000 7fea157fa640 1 -- 172.21.15.105:0/3251331019 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea20003c20 con 0x559f11b5b9a0 2024-03-26T17:46:42.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.710+0000 7fea157fa640 1 -- 172.21.15.105:0/3251331019 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20034440 con 0x559f11b5b9a0 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 -- 172.21.15.105:0/3251331019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 msgr2=0x559f11b65500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65500 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fea20003ba0 tx=0x7fea2002bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 -- 172.21.15.105:0/3251331019 shutdown_connections 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b61500 0x559f11b65a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5c390 0x559f11b6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 --2- 172.21.15.105:0/3251331019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b5b9a0 0x559f11b65500 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 -- 172.21.15.105:0/3251331019 >> 172.21.15.105:0/3251331019 conn(0x559f11b4e700 msgr2=0x559f11b521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 -- 172.21.15.105:0/3251331019 shutdown_connections 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 -- 172.21.15.105:0/3251331019 wait complete. 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.711+0000 7fea3876a740 1 -- start start 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea3876a740 1 Processor -- start 2024-03-26T17:46:42.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea3876a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5b9a0 0x559f11b6af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea3876a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b6b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea3876a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b61500 0x559f11b6c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea3876a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea301a8090 con 0x559f11b61500 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea3876a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea301a79c0 con 0x559f11b5c390 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea3876a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea301a4c20 con 0x559f11b5b9a0 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b61500 0x559f11b6c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b6b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5b9a0 0x559f11b6af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b61500 0x559f11b6c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44384/0 (socket says 172.21.15.105:44384) 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b6b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36448/0 (socket says 172.21.15.105:36448) 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2ffff640 1 -- 172.21.15.105:0/1279312254 learned_addr learned my addr 172.21.15.105:0/1279312254 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5b9a0 0x559f11b6af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56976/0 (socket says 172.21.15.105:56976) 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea3876a740 10 client.0 _ll_register_callbacks cb 0x7fea20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2f7fe640 1 -- 172.21.15.105:0/1279312254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 msgr2=0x559f11b6b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b6b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2f7fe640 1 -- 172.21.15.105:0/1279312254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b61500 msgr2=0x559f11b6c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b61500 0x559f11b6c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2f7fe640 1 -- 172.21.15.105:0/1279312254 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f11b6ca60 con 0x559f11b5b9a0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.712+0000 7fea2ffff640 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b61500 0x559f11b6c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea2f7fe640 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5b9a0 0x559f11b6af20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x559f11b40080 tx=0x7fea2002cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39165331ed07604f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea0effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea0effd640 1 -- 172.21.15.105:0/1279312254 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2005a030 con 0x559f11b5b9a0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea0effd640 1 -- 172.21.15.105:0/1279312254 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fea20040450 con 0x559f11b5b9a0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 1 -- 172.21.15.105:0/1279312254 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea20003c20 con 0x559f11b5b9a0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea0effd640 1 -- 172.21.15.105:0/1279312254 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fea20066050 con 0x559f11b5b9a0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 10 client.28351 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 1 -- 172.21.15.105:0/1279312254 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f11bd8440 con 0x559f11b5b9a0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea0effd640 1 -- 172.21.15.105:0/1279312254 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea20003c20 con 0x559f11b5b9a0 2024-03-26T17:46:42.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea0effd640 1 -- 172.21.15.105:0/1279312254 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fea200046c0 con 0x559f11b5b9a0 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 10 client.28351 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 10 client.28351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 20 client.28351 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f11bd8d50 0x559f11bdb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 4 client.28351 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea3876a740 1 -- 172.21.15.105:0/1279312254 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f11ac5b70 con 0x559f11bd8d50 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.713+0000 7fea2effd640 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f11bd8d50 0x559f11bdb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.714+0000 7fea0effd640 10 client.28351 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.714+0000 7fea2effd640 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f11bd8d50 0x559f11bdb170 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb214cfe22134a8 server_cookie=42f10ecbb441b496 in_seq=0 out_seq=0 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea0effd640 1 -- 172.21.15.105:0/1279312254 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f11ac5b70 con 0x559f11bd8d50 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea0effd640 10 client.28351 handle_command_reply: tid=0 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 client.28351 shutdown 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 -- 172.21.15.105:0/1279312254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5b9a0 msgr2=0x559f11b6af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5b9a0 0x559f11b6af20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x559f11b40080 tx=0x7fea2002cd30 comp rx=0 tx=0).stop 2024-03-26T17:46:42.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 -- 172.21.15.105:0/1279312254 shutdown_connections 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f11bd8d50 0x559f11bdb170 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f11b61500 0x559f11b6c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f11b5c390 0x559f11b6b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea0effd640 0 client.28351 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 --2- 172.21.15.105:0/1279312254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f11b5b9a0 0x559f11b6af20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 -- 172.21.15.105:0/1279312254 >> 172.21.15.105:0/1279312254 conn(0x559f11b4e700 msgr2=0x559f11b4d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 -- 172.21.15.105:0/1279312254 shutdown_connections 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 1 -- 172.21.15.105:0/1279312254 wait complete. 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.715+0000 7fea3876a740 20 client.28351 trim_cache size 0 max 16384 2024-03-26T17:46:42.712 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:46:42.713 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:46:42.713 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "970d5216-fb8c-454c-a2a0-1d5c0a2a4790", 2024-03-26T17:46:42.713 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:46:42.713 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:46:42.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 -- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea180a2f50 msgr2=0x7fea180a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea180a2f50 0x7fea180a5410 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fea240097f0 tx=0x7fea24013d10 comp rx=0 tx=0).stop 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 -- 172.21.15.105:0/2062699864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea30102370 msgr2=0x7fea301a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea30102370 0x7fea301a11f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fea28009fd0 tx=0x7fea2800ef10 comp rx=0 tx=0).stop 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 -- 172.21.15.105:0/2062699864 shutdown_connections 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea180a2f50 0x7fea180a5410 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea30102370 0x7fea301a11f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea301019b0 0x7fea3019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 --2- 172.21.15.105:0/2062699864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea301007c0 0x7fea30069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 -- 172.21.15.105:0/2062699864 >> 172.21.15.105:0/2062699864 conn(0x7fea300fbd50 msgr2=0x7fea30104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 -- 172.21.15.105:0/2062699864 shutdown_connections 2024-03-26T17:46:42.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.716+0000 7fea35b7e640 1 -- 172.21.15.105:0/2062699864 wait complete. 2024-03-26T17:46:42.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:46:42.719 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:46:42.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 -- 172.21.15.105:0/1619382120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 msgr2=0x7f20901099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 --2- 172.21.15.105:0/1619382120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 0x7f20901099f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2088009ca0 tx=0x7f208802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 -- 172.21.15.105:0/1619382120 shutdown_connections 2024-03-26T17:46:42.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 --2- 172.21.15.105:0/1619382120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 0x7f20901099f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 --2- 172.21.15.105:0/1619382120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2090104c70 0x7f2090107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 --2- 172.21.15.105:0/1619382120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090102340 0x7f2090104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 -- 172.21.15.105:0/1619382120 >> 172.21.15.105:0/1619382120 conn(0x7f20900fbd50 msgr2=0x7f20900fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 -- 172.21.15.105:0/1619382120 shutdown_connections 2024-03-26T17:46:42.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.845+0000 7f209651b640 1 -- 172.21.15.105:0/1619382120 wait complete. 2024-03-26T17:46:42.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f209651b640 1 Processor -- start 2024-03-26T17:46:42.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f209651b640 1 -- start start 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f209651b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2090102340 0x7f2090199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f209651b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090104c70 0x7f209019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f209651b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 0x7f20901a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f209651b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f209010d3f0 con 0x7f20901075a0 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f209651b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f209010d270 con 0x7f2090104c70 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f209651b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f209010d570 con 0x7f2090102340 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f2094ab8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 0x7f20901a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f2094ab8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 0x7f20901a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44410/0 (socket says 172.21.15.105:44410) 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f2094ab8640 1 -- 172.21.15.105:0/537383534 learned_addr learned my addr 172.21.15.105:0/537383534 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f208ffff640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2090102340 0x7f2090199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f208f7fe640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090104c70 0x7f209019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f208ffff640 1 -- 172.21.15.105:0/537383534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090104c70 msgr2=0x7f209019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f208ffff640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090104c70 0x7f209019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f208ffff640 1 -- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 msgr2=0x7f20901a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f208ffff640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 0x7f20901a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f208ffff640 1 -- 172.21.15.105:0/537383534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901a1910 con 0x7f2090102340 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.846+0000 7f2094ab8640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 0x7f20901a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.847+0000 7f208ffff640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2090102340 0x7f2090199d00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f208000e9f0 tx=0x7f208000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.847+0000 7f208d7fa640 1 -- 172.21.15.105:0/537383534 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208000cd80 con 0x7f2090102340 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.847+0000 7f209651b640 1 -- 172.21.15.105:0/537383534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20901a1c00 con 0x7f2090102340 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.847+0000 7f208d7fa640 1 -- 172.21.15.105:0/537383534 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20800035f0 con 0x7f2090102340 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.847+0000 7f208d7fa640 1 -- 172.21.15.105:0/537383534 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080010690 con 0x7f2090102340 2024-03-26T17:46:42.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.847+0000 7f209651b640 1 -- 172.21.15.105:0/537383534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a21a0 con 0x7f2090102340 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.849+0000 7f208d7fa640 1 -- 172.21.15.105:0/537383534 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f20800037a0 con 0x7f2090102340 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f20991e5740 1 Processor -- start 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f20991e5740 1 -- start start 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208d7fa640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f20780a2f20 0x7f20780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f20991e5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db31bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f20991e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31c2e0 0x5574db320de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f20991e5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 0x5574db3238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f20991e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5574db280980 con 0x5574db31c2e0 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f20991e5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5574db228be0 con 0x5574db31b8f0 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f2094ab8640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f20780a2f20 0x7f20780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f20991e5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574db27c930 con 0x5574db321450 2024-03-26T17:46:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208d7fa640 1 -- 172.21.15.105:0/537383534 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f20800cf760 con 0x7f2090102340 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31c2e0 0x5574db320de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db31bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31c2e0 0x5574db320de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44426/0 (socket says 172.21.15.105:44426) 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208f7fe640 1 -- 172.21.15.105:0/478004236 learned_addr learned my addr 172.21.15.105:0/478004236 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f2094ab8640 1 --2- 172.21.15.105:0/478004236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 0x5574db3238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208ffff640 1 -- 172.21.15.105:0/478004236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 msgr2=0x5574db3238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208ffff640 1 --2- 172.21.15.105:0/478004236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 0x5574db3238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208ffff640 1 -- 172.21.15.105:0/478004236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31c2e0 msgr2=0x5574db320de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208ffff640 1 --2- 172.21.15.105:0/478004236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31c2e0 0x5574db320de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.850+0000 7f208ffff640 1 -- 172.21.15.105:0/478004236 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574db05e7d0 con 0x5574db31b8f0 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f2094ab8640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f20780a2f20 0x7f20780a53e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2088032380 tx=0x7f2088032880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f208ffff640 1 --2- 172.21.15.105:0/478004236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db31bd10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f20800cd270 tx=0x7f2080098190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bada3713ad565e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20767fc640 1 -- 172.21.15.105:0/478004236 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208009c550 con 0x5574db31b8f0 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20767fc640 1 -- 172.21.15.105:0/478004236 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f208009e070 con 0x5574db31b8f0 2024-03-26T17:46:42.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20767fc640 1 -- 172.21.15.105:0/478004236 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080098430 con 0x5574db31b8f0 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20991e5740 1 -- 172.21.15.105:0/478004236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 msgr2=0x5574db31bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20991e5740 1 --2- 172.21.15.105:0/478004236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db31bd10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f20800cd270 tx=0x7f2080098190 comp rx=0 tx=0).stop 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20991e5740 1 -- 172.21.15.105:0/478004236 shutdown_connections 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20991e5740 1 --2- 172.21.15.105:0/478004236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 0x5574db3238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20991e5740 1 --2- 172.21.15.105:0/478004236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31c2e0 0x5574db320de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20991e5740 1 --2- 172.21.15.105:0/478004236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db31bd10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.851+0000 7f20991e5740 1 -- 172.21.15.105:0/478004236 >> 172.21.15.105:0/478004236 conn(0x5574db316e60 msgr2=0x5574db317260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 -- 172.21.15.105:0/478004236 shutdown_connections 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 -- 172.21.15.105:0/478004236 wait complete. 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 -- start start 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 Processor -- start 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db325ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db326020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db321450 0x5574db32d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5574db3243a0 con 0x5574db31b8f0 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5574db23c160 con 0x5574db321450 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574db324520 con 0x5574db31c2e0 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f20991e5740 10 client.0 _ll_register_callbacks cb 0x5574db278720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f208ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db325ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f208ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db325ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44434/0 (socket says 172.21.15.105:44434) 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f208ffff640 1 -- 172.21.15.105:0/520034774 learned_addr learned my addr 172.21.15.105:0/520034774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.852+0000 7f208f7fe640 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db326020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f2094ab8640 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db321450 0x5574db32d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f208ffff640 1 -- 172.21.15.105:0/520034774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 msgr2=0x5574db326020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f208ffff640 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db326020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f208ffff640 1 -- 172.21.15.105:0/520034774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db321450 msgr2=0x5574db32d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f208ffff640 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db321450 0x5574db32d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f208ffff640 1 -- 172.21.15.105:0/520034774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574daf943f0 con 0x5574db31b8f0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f208ffff640 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db325ae0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2080003dd0 tx=0x7f2080009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361ba9154b859b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f206f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f206f7fe640 1 -- 172.21.15.105:0/520034774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208009be50 con 0x5574db31b8f0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f206f7fe640 1 -- 172.21.15.105:0/520034774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f20800babc0 con 0x5574db31b8f0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f20991e5740 1 -- 172.21.15.105:0/520034774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208009e070 con 0x5574db31b8f0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f206f7fe640 1 -- 172.21.15.105:0/520034774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f208009c550 con 0x5574db31b8f0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f20991e5740 10 client.18525 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.853+0000 7f20991e5740 1 -- 172.21.15.105:0/520034774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574daf540f0 con 0x5574db31b8f0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f206f7fe640 1 -- 172.21.15.105:0/520034774 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f208009e070 con 0x5574db31b8f0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f206f7fe640 1 -- 172.21.15.105:0/520034774 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f20800b9070 con 0x5574db31b8f0 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f20991e5740 10 client.18525 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f20991e5740 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:42.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f20991e5740 20 client.18525 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:42.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f20991e5740 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5574db331550 0x5574db333940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f20991e5740 4 client.18525 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:42.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f20991e5740 1 -- 172.21.15.105:0/520034774 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574db28dec0 con 0x5574db331550 2024-03-26T17:46:42.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f208f7fe640 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5574db331550 0x5574db333940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f206f7fe640 10 client.18525 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:42.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.854+0000 7f208f7fe640 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5574db331550 0x5574db333940 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aba3c1b9697aa95f server_cookie=8bc2f8502b6f1207 in_seq=0 out_seq=0 2024-03-26T17:46:42.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f206f7fe640 1 -- 172.21.15.105:0/520034774 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5574db28dec0 con 0x5574db331550 2024-03-26T17:46:42.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f206f7fe640 10 client.18525 handle_command_reply: tid=0 2024-03-26T17:46:42.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 client.18525 shutdown 2024-03-26T17:46:42.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 -- 172.21.15.105:0/520034774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 msgr2=0x5574db325ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db325ae0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2080003dd0 tx=0x7f2080009540 comp rx=0 tx=0).stop 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 -- 172.21.15.105:0/520034774 shutdown_connections 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5574db331550 0x5574db333940 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db321450 0x5574db32d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db326020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f206f7fe640 0 client.18525 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 --2- 172.21.15.105:0/520034774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db325ae0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 -- 172.21.15.105:0/520034774 >> 172.21.15.105:0/520034774 conn(0x5574db316e60 msgr2=0x5574db316080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 -- 172.21.15.105:0/520034774 shutdown_connections 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 1 -- 172.21.15.105:0/520034774 wait complete. 2024-03-26T17:46:42.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.857+0000 7f20991e5740 20 client.18525 trim_cache size 0 max 16384 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.878+0000 7f20991e5740 1 Processor -- start 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.878+0000 7f20991e5740 1 -- start start 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f20991e5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db316800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f20991e5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db32cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f20991e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db321450 0x5574db31a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f20991e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20901aaf20 con 0x5574db321450 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f20991e5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20901aca60 con 0x5574db31b8f0 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f20991e5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20901ab5f0 con 0x5574db31c2e0 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f2094ab8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db321450 0x5574db31a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db316800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db32cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db316800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36490/0 (socket says 172.21.15.105:36490) 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db32cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57002/0 (socket says 172.21.15.105:57002) 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208ffff640 1 -- 172.21.15.105:0/296400216 learned_addr learned my addr 172.21.15.105:0/296400216 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f2094ab8640 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db321450 0x5574db31a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44440/0 (socket says 172.21.15.105:44440) 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208f7fe640 1 -- 172.21.15.105:0/296400216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 msgr2=0x5574db316800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208f7fe640 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db316800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208f7fe640 1 -- 172.21.15.105:0/296400216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db321450 msgr2=0x5574db31a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208f7fe640 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db321450 0x5574db31a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208f7fe640 1 -- 172.21.15.105:0/296400216 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574db3178d0 con 0x5574db31c2e0 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208ffff640 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db316800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f208f7fe640 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db32cff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2084009530 tx=0x7f2084005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa528a08188212dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f2075ffb640 1 -- 172.21.15.105:0/296400216 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20840200f0 con 0x5574db31c2e0 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.879+0000 7f2075ffb640 1 -- 172.21.15.105:0/296400216 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2084020740 con 0x5574db31c2e0 2024-03-26T17:46:42.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f2075ffb640 1 -- 172.21.15.105:0/296400216 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084010d50 con 0x5574db31c2e0 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 -- 172.21.15.105:0/296400216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 msgr2=0x5574db32cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db32cff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2084009530 tx=0x7f2084005f90 comp rx=0 tx=0).stop 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 -- 172.21.15.105:0/296400216 shutdown_connections 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db321450 0x5574db31a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db31c2e0 0x5574db32cff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 --2- 172.21.15.105:0/296400216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31b8f0 0x5574db316800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 -- 172.21.15.105:0/296400216 >> 172.21.15.105:0/296400216 conn(0x5574db316e60 msgr2=0x5574db31a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 -- 172.21.15.105:0/296400216 shutdown_connections 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.880+0000 7f20991e5740 1 -- 172.21.15.105:0/296400216 wait complete. 2024-03-26T17:46:42.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 1 -- start start 2024-03-26T17:46:42.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 1 Processor -- start 2024-03-26T17:46:42.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db32af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31c2e0 0x5574db32b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 0x5574db32c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20901ac390 con 0x5574db31b8f0 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20901abcc0 con 0x5574db31c2e0 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20901a8f00 con 0x5574db321450 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f20991e5740 10 client.0 _ll_register_callbacks cb 0x7f20840097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f208ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db32af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f208ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db32af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44448/0 (socket says 172.21.15.105:44448) 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31c2e0 0x5574db32b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f208ffff640 1 -- 172.21.15.105:0/2428552610 learned_addr learned my addr 172.21.15.105:0/2428552610 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f2094ab8640 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 0x5574db32c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f208ffff640 1 -- 172.21.15.105:0/2428552610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 msgr2=0x5574db32c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f208ffff640 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 0x5574db32c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f208ffff640 1 -- 172.21.15.105:0/2428552610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31c2e0 msgr2=0x5574db32b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.881+0000 7f208ffff640 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31c2e0 0x5574db32b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f208ffff640 1 -- 172.21.15.105:0/2428552610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574db32ca20 con 0x5574db31b8f0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f208f7fe640 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31c2e0 0x5574db32b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f208ffff640 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db32af50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f208009bb50 tx=0x7f20800bc130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f800e551259ece96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f206f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f206f7fe640 1 -- 172.21.15.105:0/2428552610 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208009b4a0 con 0x5574db31b8f0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f206f7fe640 1 -- 172.21.15.105:0/2428552610 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f208009ad60 con 0x5574db31b8f0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f20991e5740 1 -- 172.21.15.105:0/2428552610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2084020740 con 0x5574db31b8f0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f206f7fe640 1 -- 172.21.15.105:0/2428552610 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f208009be50 con 0x5574db31b8f0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f20991e5740 10 client.18531 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f20991e5740 1 -- 172.21.15.105:0/2428552610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574db397e30 con 0x5574db31b8f0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f206f7fe640 1 -- 172.21.15.105:0/2428552610 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2084020740 con 0x5574db31b8f0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f206f7fe640 1 -- 172.21.15.105:0/2428552610 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f20800b9040 con 0x5574db31b8f0 2024-03-26T17:46:42.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.882+0000 7f20991e5740 10 client.18531 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f20991e5740 10 client.18531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f20991e5740 20 client.18531 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f20991e5740 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5574db398600 0x5574db39aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f20991e5740 4 client.18531 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f20991e5740 1 -- 172.21.15.105:0/2428552610 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574db288210 con 0x5574db398600 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f208f7fe640 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5574db398600 0x5574db39aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f206f7fe640 10 client.18531 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f208f7fe640 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5574db398600 0x5574db39aa20 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152f3cdccbcd0644 server_cookie=b467e07b4de8dc71 in_seq=0 out_seq=0 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f206f7fe640 1 -- 172.21.15.105:0/2428552610 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5574db288210 con 0x5574db398600 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f206f7fe640 10 client.18531 handle_command_reply: tid=0 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.883+0000 7f20991e5740 1 client.18531 shutdown 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 -- 172.21.15.105:0/2428552610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 msgr2=0x5574db32af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db32af50 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f208009bb50 tx=0x7f20800bc130 comp rx=0 tx=0).stop 2024-03-26T17:46:42.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 -- 172.21.15.105:0/2428552610 shutdown_connections 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5574db398600 0x5574db39aa20 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574db321450 0x5574db32c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574db31c2e0 0x5574db32b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 --2- 172.21.15.105:0/2428552610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574db31b8f0 0x5574db32af50 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 -- 172.21.15.105:0/2428552610 >> 172.21.15.105:0/2428552610 conn(0x5574db316e60 msgr2=0x5574db31a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f206f7fe640 0 client.18531 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 -- 172.21.15.105:0/2428552610 shutdown_connections 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 1 -- 172.21.15.105:0/2428552610 wait complete. 2024-03-26T17:46:42.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f20991e5740 20 client.18531 trim_cache size 0 max 16384 2024-03-26T17:46:42.886 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:46:42.886 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:46:42.886 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:46:42.886 INFO:teuthology.orchestra.run.smithi105.stdout: "970d5216-fb8c-454c-a2a0-1d5c0a2a4790": { 2024-03-26T17:46:42.887 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:46:42.887 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "970d5216-fb8c-454c-a2a0-1d5c0a2a4790", 2024-03-26T17:46:42.887 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:46:42.887 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:46:42.887 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:46:42.887 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:46:42.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f209651b640 1 -- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f20780a2f20 msgr2=0x7f20780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f209651b640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f20780a2f20 0x7f20780a53e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2088032380 tx=0x7f2088032880 comp rx=0 tx=0).stop 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f209651b640 1 -- 172.21.15.105:0/537383534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2090102340 msgr2=0x7f2090199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.884+0000 7f209651b640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2090102340 0x7f2090199d00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f208000e9f0 tx=0x7f208000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.885+0000 7f209651b640 1 -- 172.21.15.105:0/537383534 shutdown_connections 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.885+0000 7f209651b640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f20780a2f20 0x7f20780a53e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.885+0000 7f209651b640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20901075a0 0x7f20901a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.885+0000 7f209651b640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090104c70 0x7f209019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.885+0000 7f209651b640 1 --2- 172.21.15.105:0/537383534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2090102340 0x7f2090199d00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.885+0000 7f209651b640 1 -- 172.21.15.105:0/537383534 >> 172.21.15.105:0/537383534 conn(0x7f20900fbd50 msgr2=0x7f2090101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.885+0000 7f209651b640 1 -- 172.21.15.105:0/537383534 shutdown_connections 2024-03-26T17:46:42.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:46:42.885+0000 7f209651b640 1 -- 172.21.15.105:0/537383534 wait complete. 2024-03-26T17:46:42.891 INFO:tasks.cephfs.filesystem:scrub status for tag:970d5216-fb8c-454c-a2a0-1d5c0a2a4790 - {'path': '/', 'tag': '970d5216-fb8c-454c-a2a0-1d5c0a2a4790', 'options': 'recursive,force'} 2024-03-26T17:46:43.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 262256 3.15 MB/sec execute 326 sec latency 29.220 ms 2024-03-26T17:46:44.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 262736 3.16 MB/sec execute 327 sec latency 15.339 ms 2024-03-26T17:46:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:43 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:46:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:43 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:43 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:46:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:43 smithi144 ceph-mon[36095]: pgmap v599: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:46:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:43 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:43 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:46:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:43 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:43 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:46:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:43 smithi105 ceph-mon[32280]: pgmap v599: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:46:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:43 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:44.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:43 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:46:44.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:43 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:44.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:43 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:46:44.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:43 smithi135 ceph-mon[35636]: pgmap v599: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:46:44.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:43 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:46:45.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 263110 3.15 MB/sec execute 328 sec latency 15.665 ms 2024-03-26T17:46:46.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 263687 3.15 MB/sec execute 329 sec latency 23.526 ms 2024-03-26T17:46:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:45 smithi144 ceph-mon[36095]: pgmap v600: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-03-26T17:46:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:45 smithi105 ceph-mon[32280]: pgmap v600: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-03-26T17:46:46.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:45 smithi135 ceph-mon[35636]: pgmap v600: 97 pgs: 97 active+clean; 29 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 124 op/s 2024-03-26T17:46:47.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 264367 3.15 MB/sec execute 330 sec latency 15.438 ms 2024-03-26T17:46:48.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 265006 3.14 MB/sec execute 331 sec latency 17.665 ms 2024-03-26T17:46:48.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:47 smithi144 ceph-mon[36095]: pgmap v601: 97 pgs: 97 active+clean; 30 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:46:48.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:47 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:46:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:47 smithi105 ceph-mon[32280]: pgmap v601: 97 pgs: 97 active+clean; 30 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:46:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:47 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:46:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:47 smithi135 ceph-mon[35636]: pgmap v601: 97 pgs: 97 active+clean; 30 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:46:48.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:47 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:46:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 -- 172.21.15.105:0/3935082601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc104c90 msgr2=0x7ff6fc107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 --2- 172.21.15.105:0/3935082601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc104c90 0x7ff6fc107080 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009900 tx=0x7ff6f002bc30 comp rx=0 tx=0).stop 2024-03-26T17:46:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 -- 172.21.15.105:0/3935082601 shutdown_connections 2024-03-26T17:46:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 --2- 172.21.15.105:0/3935082601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6fc1075c0 0x7ff6fc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 --2- 172.21.15.105:0/3935082601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc104c90 0x7ff6fc107080 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 --2- 172.21.15.105:0/3935082601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6fc102360 0x7ff6fc104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 -- 172.21.15.105:0/3935082601 >> 172.21.15.105:0/3935082601 conn(0x7ff6fc0fbd50 msgr2=0x7ff6fc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 -- 172.21.15.105:0/3935082601 shutdown_connections 2024-03-26T17:46:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.410+0000 7ff702836640 1 -- 172.21.15.105:0/3935082601 wait complete. 2024-03-26T17:46:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff702836640 1 Processor -- start 2024-03-26T17:46:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff702836640 1 -- start start 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff702836640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6fc102360 0x7ff6fc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff702836640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6fc104c90 0x7ff6fc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff702836640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc1075c0 0x7ff6fc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff702836640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc10d400 con 0x7ff6fc1075c0 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff702836640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc10d280 con 0x7ff6fc104c90 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff702836640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc10d580 con 0x7ff6fc102360 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff6fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6fc102360 0x7ff6fc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff6fb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6fc104c90 0x7ff6fc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff700dd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc1075c0 0x7ff6fc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff6fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6fc102360 0x7ff6fc197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41846/0 (socket says 172.21.15.105:41846) 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff6fb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6fc104c90 0x7ff6fc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33306/0 (socket says 172.21.15.105:33306) 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.411+0000 7ff6fb7fe640 1 -- 172.21.15.105:0/555327573 learned_addr learned my addr 172.21.15.105:0/555327573 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6fbfff640 1 -- 172.21.15.105:0/555327573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6fc104c90 msgr2=0x7ff6fc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6fc104c90 0x7ff6fc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6fbfff640 1 -- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc1075c0 msgr2=0x7ff6fc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc1075c0 0x7ff6fc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6fbfff640 1 -- 172.21.15.105:0/555327573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc19f6c0 con 0x7ff6fc102360 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6fb7fe640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6fc104c90 0x7ff6fc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff700dd3640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc1075c0 0x7ff6fc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6fc102360 0x7ff6fc197aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009bc0 tx=0x7ff6ec00b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6f97fa640 1 -- 172.21.15.105:0/555327573 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec01e070 con 0x7ff6fc102360 2024-03-26T17:46:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6f97fa640 1 -- 172.21.15.105:0/555327573 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6ec0198c0 con 0x7ff6fc102360 2024-03-26T17:46:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff702836640 1 -- 172.21.15.105:0/555327573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6fc1001b0 con 0x7ff6fc102360 2024-03-26T17:46:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff6f97fa640 1 -- 172.21.15.105:0/555327573 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec018710 con 0x7ff6fc102360 2024-03-26T17:46:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.412+0000 7ff702836640 1 -- 172.21.15.105:0/555327573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6fc100700 con 0x7ff6fc102360 2024-03-26T17:46:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.414+0000 7ff6f97fa640 1 -- 172.21.15.105:0/555327573 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff6ec021070 con 0x7ff6fc102360 2024-03-26T17:46:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff705424740 1 Processor -- start 2024-03-26T17:46:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff705424740 1 -- start start 2024-03-26T17:46:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff705424740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff705424740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff705424740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff705424740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b2d50470b0 con 0x55b2d50e2fc0 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff705424740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2d4fef960 con 0x55b2d50e25d0 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff705424740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2d5043060 con 0x55b2d50e8130 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff6fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff700dd3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff6fb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff700dd3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50e29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33320/0 (socket says 172.21.15.105:33320) 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff700dd3640 1 -- 172.21.15.105:0/1018151755 learned_addr learned my addr 172.21.15.105:0/1018151755 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff6f97fa640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6e40a2fc0 0x7ff6e40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff700dd3640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6e40a2fc0 0x7ff6e40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff6f97fa640 1 -- 172.21.15.105:0/555327573 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff6ec0d2c40 con 0x7ff6fc102360 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff6fbfff640 1 -- 172.21.15.105:0/1018151755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 msgr2=0x55b2d50ea5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/1018151755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50ea5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff6fbfff640 1 -- 172.21.15.105:0/1018151755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 msgr2=0x55b2d50e29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.415+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/1018151755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50e29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.416+0000 7ff6fbfff640 1 -- 172.21.15.105:0/1018151755 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d5001c60 con 0x55b2d50e2fc0 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.416+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/1018151755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50e7ac0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec09f080 tx=0x7ff6ec00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b4c35224350ef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.416+0000 7ff6e27fc640 1 -- 172.21.15.105:0/1018151755 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec09f380 con 0x55b2d50e2fc0 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.416+0000 7ff700dd3640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6e40a2fc0 0x7ff6e40a5480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x55b2d50e3820 tx=0x7ff6f4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.416+0000 7ff6e27fc640 1 -- 172.21.15.105:0/1018151755 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6ec0a3420 con 0x55b2d50e2fc0 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.416+0000 7ff6e27fc640 1 -- 172.21.15.105:0/1018151755 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec019a70 con 0x55b2d50e2fc0 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 -- 172.21.15.105:0/1018151755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 msgr2=0x55b2d50e7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 --2- 172.21.15.105:0/1018151755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50e7ac0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec09f080 tx=0x7ff6ec00be60 comp rx=0 tx=0).stop 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 -- 172.21.15.105:0/1018151755 shutdown_connections 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 --2- 172.21.15.105:0/1018151755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50ea5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 --2- 172.21.15.105:0/1018151755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50e7ac0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 --2- 172.21.15.105:0/1018151755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50e29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 -- 172.21.15.105:0/1018151755 >> 172.21.15.105:0/1018151755 conn(0x55b2d50ddb40 msgr2=0x55b2d50ddf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 -- 172.21.15.105:0/1018151755 shutdown_connections 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 -- 172.21.15.105:0/1018151755 wait complete. 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 -- start start 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.417+0000 7ff705424740 1 Processor -- start 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff705424740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff705424740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff705424740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50f3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff705424740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b2d50eb080 con 0x55b2d50e2fc0 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff705424740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2d5002db0 con 0x55b2d50e25d0 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff705424740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2d50eb200 con 0x55b2d50e8130 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff705424740 10 client.0 _ll_register_callbacks cb 0x55b2d5007110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50ec8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39732/0 (socket says 172.21.15.105:39732) 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 -- 172.21.15.105:0/2939911326 learned_addr learned my addr 172.21.15.105:0/2939911326 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff700dd3640 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 -- 172.21.15.105:0/2939911326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 msgr2=0x55b2d50f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 -- 172.21.15.105:0/2939911326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 msgr2=0x55b2d50ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50ec360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 -- 172.21.15.105:0/2939911326 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d4d5b210 con 0x55b2d50e2fc0 2024-03-26T17:46:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6fb7fe640 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50ec8a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0000c00 tx=0x7ff6f0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db19ea8198886bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6db7fe640 1 -- 172.21.15.105:0/2939911326 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0005d50 con 0x55b2d50e2fc0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff6db7fe640 1 -- 172.21.15.105:0/2939911326 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff6f00046c0 con 0x55b2d50e2fc0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.418+0000 7ff705424740 1 -- 172.21.15.105:0/2939911326 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ec0a3420 con 0x55b2d50e2fc0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff6db7fe640 1 -- 172.21.15.105:0/2939911326 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6f002cc90 con 0x55b2d50e2fc0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff705424740 10 client.18543 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff705424740 1 -- 172.21.15.105:0/2939911326 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2d4e27620 con 0x55b2d50e2fc0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff6db7fe640 1 -- 172.21.15.105:0/2939911326 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6ec0a3420 con 0x55b2d50e2fc0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff6db7fe640 1 -- 172.21.15.105:0/2939911326 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff6f0003b10 con 0x55b2d50e2fc0 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff705424740 10 client.18543 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff705424740 10 client.18543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff705424740 20 client.18543 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff705424740 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b2d50f8590 0x55b2d50faa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff705424740 4 client.18543 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:46:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff705424740 1 -- 172.21.15.105:0/2939911326 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2d50545f0 con 0x55b2d50f8590 2024-03-26T17:46:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.419+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b2d50f8590 0x55b2d50faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.420+0000 7ff6db7fe640 10 client.18543 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.420+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b2d50f8590 0x55b2d50faa30 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20f5d9d555899bad server_cookie=4a583e9225881f0f in_seq=0 out_seq=0 2024-03-26T17:46:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff6db7fe640 1 -- 172.21.15.105:0/2939911326 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b2d50545f0 con 0x55b2d50f8590 2024-03-26T17:46:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff6db7fe640 10 client.18543 handle_command_reply: tid=0 2024-03-26T17:46:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 client.18543 shutdown 2024-03-26T17:46:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 -- 172.21.15.105:0/2939911326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 msgr2=0x55b2d50ec8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50ec8a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0000c00 tx=0x7ff6f0005250 comp rx=0 tx=0).stop 2024-03-26T17:46:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 -- 172.21.15.105:0/2939911326 shutdown_connections 2024-03-26T17:46:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b2d50f8590 0x55b2d50faa30 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff6db7fe640 0 client.18543 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50ec8a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 --2- 172.21.15.105:0/2939911326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e25d0 0x55b2d50ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 -- 172.21.15.105:0/2939911326 >> 172.21.15.105:0/2939911326 conn(0x55b2d50ddb40 msgr2=0x55b2d50dcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.422+0000 7ff705424740 1 -- 172.21.15.105:0/2939911326 shutdown_connections 2024-03-26T17:46:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.423+0000 7ff705424740 1 -- 172.21.15.105:0/2939911326 wait complete. 2024-03-26T17:46:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.423+0000 7ff705424740 20 client.18543 trim_cache size 0 max 16384 2024-03-26T17:46:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.442+0000 7ff705424740 1 Processor -- start 2024-03-26T17:46:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.442+0000 7ff705424740 1 -- start start 2024-03-26T17:46:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff705424740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e25d0 0x55b2d50e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff705424740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e2fc0 0x55b2d50f3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff705424740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff705424740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1a6ae0 con 0x55b2d50e25d0 2024-03-26T17:46:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff705424740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1a8620 con 0x55b2d50e2fc0 2024-03-26T17:46:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff705424740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1a71b0 con 0x55b2d50e8130 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e25d0 0x55b2d50e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e25d0 0x55b2d50e73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39748/0 (socket says 172.21.15.105:39748) 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 -- 172.21.15.105:0/849112508 learned_addr learned my addr 172.21.15.105:0/849112508 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff700dd3640 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fb7fe640 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e2fc0 0x55b2d50f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 -- 172.21.15.105:0/849112508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 msgr2=0x55b2d50e7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50e7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 -- 172.21.15.105:0/849112508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e2fc0 msgr2=0x55b2d50f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e2fc0 0x55b2d50f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 -- 172.21.15.105:0/849112508 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d50e7e70 con 0x55b2d50e25d0 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e25d0 0x55b2d50e73f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec09f080 tx=0x7ff6ec0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2740a9fc6fae040d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6e1ffb640 1 -- 172.21.15.105:0/849112508 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec09f380 con 0x55b2d50e25d0 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6e1ffb640 1 -- 172.21.15.105:0/849112508 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6ec0a3420 con 0x55b2d50e25d0 2024-03-26T17:46:48.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.443+0000 7ff6e1ffb640 1 -- 172.21.15.105:0/849112508 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec084be0 con 0x55b2d50e25d0 2024-03-26T17:46:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 -- 172.21.15.105:0/849112508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e25d0 msgr2=0x55b2d50e73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e25d0 0x55b2d50e73f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec09f080 tx=0x7ff6ec0108b0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 -- 172.21.15.105:0/849112508 shutdown_connections 2024-03-26T17:46:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e8130 0x55b2d50e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e2fc0 0x55b2d50f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 --2- 172.21.15.105:0/849112508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e25d0 0x55b2d50e73f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 -- 172.21.15.105:0/849112508 >> 172.21.15.105:0/849112508 conn(0x55b2d50ddb40 msgr2=0x55b2d50ec360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 -- 172.21.15.105:0/849112508 shutdown_connections 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 -- 172.21.15.105:0/849112508 wait complete. 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.444+0000 7ff705424740 1 -- start start 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff705424740 1 Processor -- start 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff705424740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e25d0 0x55b2d50f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff705424740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff705424740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e8130 0x55b2d515c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff705424740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1a7f50 con 0x55b2d50e2fc0 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff705424740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1a7880 con 0x55b2d50e8130 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff705424740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc1a4ac0 con 0x55b2d50e25d0 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff705424740 10 client.0 _ll_register_callbacks cb 0x7ff6f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e25d0 0x55b2d50f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e25d0 0x55b2d50f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41880/0 (socket says 172.21.15.105:41880) 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fbfff640 1 -- 172.21.15.105:0/878634610 learned_addr learned my addr 172.21.15.105:0/878634610 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:46:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff700dd3640 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e8130 0x55b2d515c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fbfff640 1 -- 172.21.15.105:0/878634610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e8130 msgr2=0x55b2d515c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e8130 0x55b2d515c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fbfff640 1 -- 172.21.15.105:0/878634610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 msgr2=0x55b2d50f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50f0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.445+0000 7ff6fbfff640 1 -- 172.21.15.105:0/878634610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d515cea0 con 0x55b2d50e25d0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff6fbfff640 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e25d0 0x55b2d50f0720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec005eb0 tx=0x7ff6ec09b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9cfec551c0cdd01 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff6db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff6db7fe640 1 -- 172.21.15.105:0/878634610 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec0bc020 con 0x55b2d50e25d0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff6db7fe640 1 -- 172.21.15.105:0/878634610 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff6ec019ae0 con 0x55b2d50e25d0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 1 -- 172.21.15.105:0/878634610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ec0a3420 con 0x55b2d50e25d0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff6db7fe640 1 -- 172.21.15.105:0/878634610 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6ec09f380 con 0x55b2d50e25d0 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 10 client.28387 fetch_fsmap learned FSMap version 30 2024-03-26T17:46:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 1 -- 172.21.15.105:0/878634610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2d515e160 con 0x55b2d50e25d0 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff6db7fe640 1 -- 172.21.15.105:0/878634610 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6ec0a3420 con 0x55b2d50e25d0 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff6db7fe640 1 -- 172.21.15.105:0/878634610 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff6ec09e7d0 con 0x55b2d50e25d0 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 10 client.28387 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 10 client.28387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 20 client.28387 populate_metadata read hostname 'smithi105' 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b2d515e6e0 0x55b2d5160b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 4 client.28387 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.446+0000 7ff705424740 1 -- 172.21.15.105:0/878634610 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2d504e940 con 0x55b2d515e6e0 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.447+0000 7ff6fb7fe640 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b2d515e6e0 0x55b2d5160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.447+0000 7ff6db7fe640 10 client.28387 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.447+0000 7ff6fb7fe640 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b2d515e6e0 0x55b2d5160b00 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b69c100ea802c5 server_cookie=42d158ef71074d38 in_seq=0 out_seq=0 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.484+0000 7ff6db7fe640 1 -- 172.21.15.105:0/878634610 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2d504e940 con 0x55b2d515e6e0 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.484+0000 7ff6db7fe640 10 client.28387 handle_command_reply: tid=0 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.484+0000 7ff705424740 1 client.28387 shutdown 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.484+0000 7ff705424740 1 -- 172.21.15.105:0/878634610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e25d0 msgr2=0x55b2d50f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.484+0000 7ff705424740 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e25d0 0x55b2d50f0720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec005eb0 tx=0x7ff6ec09b720 comp rx=0 tx=0).stop 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 1 -- 172.21.15.105:0/878634610 shutdown_connections 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b2d515e6e0 0x55b2d5160b00 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2d50e8130 0x55b2d515c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2d50e2fc0 0x55b2d50f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff6db7fe640 0 client.28387 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 1 --2- 172.21.15.105:0/878634610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2d50e25d0 0x55b2d50f0720 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 1 -- 172.21.15.105:0/878634610 >> 172.21.15.105:0/878634610 conn(0x55b2d50ddb40 msgr2=0x55b2d50e6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 1 -- 172.21.15.105:0/878634610 shutdown_connections 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 1 -- 172.21.15.105:0/878634610 wait complete. 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff705424740 20 client.28387 trim_cache size 0 max 16384 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:46:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:46:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 -- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6e40a2fc0 msgr2=0x7ff6e40a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6e40a2fc0 0x7ff6e40a5480 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x55b2d50e3820 tx=0x7ff6f4004ab0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 -- 172.21.15.105:0/555327573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6fc102360 msgr2=0x7ff6fc197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6fc102360 0x7ff6fc197aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009bc0 tx=0x7ff6ec00b690 comp rx=0 tx=0).stop 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 -- 172.21.15.105:0/555327573 shutdown_connections 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6e40a2fc0 0x7ff6e40a5480 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6fc1075c0 0x7ff6fc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6fc104c90 0x7ff6fc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 --2- 172.21.15.105:0/555327573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6fc102360 0x7ff6fc197aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.485+0000 7ff702836640 1 -- 172.21.15.105:0/555327573 >> 172.21.15.105:0/555327573 conn(0x7ff6fc0fbd50 msgr2=0x7ff6fc102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.486+0000 7ff702836640 1 -- 172.21.15.105:0/555327573 shutdown_connections 2024-03-26T17:46:48.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:46:48.486+0000 7ff702836640 1 -- 172.21.15.105:0/555327573 wait complete. 2024-03-26T17:46:49.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 265804 3.15 MB/sec execute 332 sec latency 33.692 ms 2024-03-26T17:46:50.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 266306 3.16 MB/sec execute 333 sec latency 15.497 ms 2024-03-26T17:46:50.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:49 smithi144 ceph-mon[36095]: pgmap v602: 97 pgs: 97 active+clean; 30 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:46:50.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:49 smithi105 ceph-mon[32280]: pgmap v602: 97 pgs: 97 active+clean; 30 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:46:50.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:49 smithi135 ceph-mon[35636]: pgmap v602: 97 pgs: 97 active+clean; 30 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:46:51.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 266686 3.15 MB/sec execute 334 sec latency 21.369 ms 2024-03-26T17:46:52.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 267271 3.15 MB/sec execute 335 sec latency 16.908 ms 2024-03-26T17:46:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:51 smithi144 ceph-mon[36095]: pgmap v603: 97 pgs: 97 active+clean; 32 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:46:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:51 smithi105 ceph-mon[32280]: pgmap v603: 97 pgs: 97 active+clean; 32 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:46:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:51 smithi135 ceph-mon[35636]: pgmap v603: 97 pgs: 97 active+clean; 32 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:46:53.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 267932 3.15 MB/sec execute 336 sec latency 16.093 ms 2024-03-26T17:46:53.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:52 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:46:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:52 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:46:53.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:52 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:46:54.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 268510 3.14 MB/sec execute 337 sec latency 18.832 ms 2024-03-26T17:46:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:53 smithi144 ceph-mon[36095]: pgmap v604: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:46:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:46:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:46:54.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:46:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:53 smithi105 ceph-mon[32280]: pgmap v604: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:46:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:46:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:46:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:46:54.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:53 smithi135 ceph-mon[35636]: pgmap v604: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:46:54.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:46:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:46:54.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:46:55.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 269351 3.15 MB/sec execute 338 sec latency 15.416 ms 2024-03-26T17:46:56.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 269835 3.16 MB/sec execute 339 sec latency 16.955 ms 2024-03-26T17:46:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:55 smithi144 ceph-mon[36095]: pgmap v605: 97 pgs: 97 active+clean; 28 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-03-26T17:46:56.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:55 smithi105 ceph-mon[32280]: pgmap v605: 97 pgs: 97 active+clean; 28 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-03-26T17:46:56.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:55 smithi135 ceph-mon[35636]: pgmap v605: 97 pgs: 97 active+clean; 28 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-03-26T17:46:57.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 270219 3.15 MB/sec execute 340 sec latency 21.566 ms 2024-03-26T17:46:58.028 INFO:tasks.workunit.client.0.smithi105.stdout: 1 270809 3.15 MB/sec execute 341 sec latency 18.839 ms 2024-03-26T17:46:58.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:57 smithi144 ceph-mon[36095]: pgmap v606: 97 pgs: 97 active+clean; 29 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:46:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:57 smithi105 ceph-mon[32280]: pgmap v606: 97 pgs: 97 active+clean; 29 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:46:58.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:57 smithi135 ceph-mon[35636]: pgmap v606: 97 pgs: 97 active+clean; 29 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:46:59.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 271473 3.15 MB/sec execute 342 sec latency 15.556 ms 2024-03-26T17:47:00.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 272083 3.14 MB/sec execute 343 sec latency 17.789 ms 2024-03-26T17:47:00.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:46:59 smithi144 ceph-mon[36095]: pgmap v607: 97 pgs: 97 active+clean; 30 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:47:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:46:59 smithi105 ceph-mon[32280]: pgmap v607: 97 pgs: 97 active+clean; 30 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:47:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:46:59 smithi135 ceph-mon[35636]: pgmap v607: 97 pgs: 97 active+clean; 30 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:47:01.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 272893 3.15 MB/sec execute 344 sec latency 16.406 ms 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.581+0000 7f0c5b71d640 1 -- 172.21.15.105:0/1352633798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54107660 msgr2=0x7f0c54109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.581+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/1352633798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54107660 0x7f0c54109a50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009ca0 tx=0x7f0c4c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 -- 172.21.15.105:0/1352633798 shutdown_connections 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/1352633798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54107660 0x7f0c54109a50 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/1352633798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c54104d30 0x7f0c54107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/1352633798 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c54102400 0x7f0c541047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 -- 172.21.15.105:0/1352633798 >> 172.21.15.105:0/1352633798 conn(0x7f0c540fbd50 msgr2=0x7f0c540fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 -- 172.21.15.105:0/1352633798 shutdown_connections 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 -- 172.21.15.105:0/1352633798 wait complete. 2024-03-26T17:47:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 Processor -- start 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 -- start start 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54102400 0x7f0c54199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c54104d30 0x7f0c5419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c54107660 0x7f0c541a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5410d430 con 0x7f0c54102400 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5410d2b0 con 0x7f0c54104d30 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c5b71d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5410d5b0 con 0x7f0c54107660 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c594b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54102400 0x7f0c54199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c594b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54102400 0x7f0c54199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47162/0 (socket says 172.21.15.105:47162) 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c594b9640 1 -- 172.21.15.105:0/415813543 learned_addr learned my addr 172.21.15.105:0/415813543 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c54104d30 0x7f0c5419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.582+0000 7f0c59cba640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c54107660 0x7f0c541a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c594b9640 1 -- 172.21.15.105:0/415813543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c54107660 msgr2=0x7f0c541a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c594b9640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c54107660 0x7f0c541a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c594b9640 1 -- 172.21.15.105:0/415813543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c54104d30 msgr2=0x7f0c5419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c594b9640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c54104d30 0x7f0c5419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c594b9640 1 -- 172.21.15.105:0/415813543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c541a18e0 con 0x7f0c54102400 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c54104d30 0x7f0c5419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c594b9640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54102400 0x7f0c54199ca0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000e9f0 tx=0x7f0c5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c4a7fc640 1 -- 172.21.15.105:0/415813543 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000cd80 con 0x7f0c54102400 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c4a7fc640 1 -- 172.21.15.105:0/415813543 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c500108e0 con 0x7f0c54102400 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c4a7fc640 1 -- 172.21.15.105:0/415813543 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50018920 con 0x7f0c54102400 2024-03-26T17:47:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c5b71d640 1 -- 172.21.15.105:0/415813543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c541a1bd0 con 0x7f0c54102400 2024-03-26T17:47:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.583+0000 7f0c5b71d640 1 -- 172.21.15.105:0/415813543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c541a20d0 con 0x7f0c54102400 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.586+0000 7f0c4a7fc640 1 -- 172.21.15.105:0/415813543 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0c50010a90 con 0x7f0c54102400 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.586+0000 7f0c5e3e0740 1 Processor -- start 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.586+0000 7f0c5e3e0740 1 -- start start 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.586+0000 7f0c4a7fc640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c400a2fc0 0x7f0c400a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8b5d0 0x5638bdc8b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8bfc0 0x5638bdc90ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5638bdbf00b0 con 0x5638bdc8b5d0 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5638bdb98960 con 0x5638bdc8bfc0 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638bdbec060 con 0x5638bdc91130 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c59cba640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c400a2fc0 0x7f0c400a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c4a7fc640 1 -- 172.21.15.105:0/415813543 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f0c500cf700 con 0x7f0c54102400 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8bfc0 0x5638bdc90ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8bfc0 0x5638bdc90ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53936/0 (socket says 172.21.15.105:53936) 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 -- 172.21.15.105:0/1943942386 learned_addr learned my addr 172.21.15.105:0/1943942386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c594b9640 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8b5d0 0x5638bdc8b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c59cba640 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 -- 172.21.15.105:0/1943942386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 msgr2=0x5638bdc935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 -- 172.21.15.105:0/1943942386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8b5d0 msgr2=0x5638bdc8b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8b5d0 0x5638bdc8b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 -- 172.21.15.105:0/1943942386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638bdbaac60 con 0x5638bdc8bfc0 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c59cba640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c400a2fc0 0x7f0c400a5480 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006010 tx=0x7f0c4c032ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.587+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8bfc0 0x5638bdc90ac0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0c440114f0 tx=0x7f0c440118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bdae16c4ea6d8d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c3f7fe640 1 -- 172.21.15.105:0/1943942386 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44005020 con 0x5638bdc8bfc0 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c3f7fe640 1 -- 172.21.15.105:0/1943942386 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c440047a0 con 0x5638bdc8bfc0 2024-03-26T17:47:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c3f7fe640 1 -- 172.21.15.105:0/1943942386 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400f7a0 con 0x5638bdc8bfc0 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/1943942386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8bfc0 msgr2=0x5638bdc90ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8bfc0 0x5638bdc90ac0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0c440114f0 tx=0x7f0c440118f0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/1943942386 shutdown_connections 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8bfc0 0x5638bdc90ac0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/1943942386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8b5d0 0x5638bdc8b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/1943942386 >> 172.21.15.105:0/1943942386 conn(0x5638bdc86b40 msgr2=0x5638bdc86f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/1943942386 shutdown_connections 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.588+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/1943942386 wait complete. 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 1 -- start start 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 1 Processor -- start 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc95310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc95850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc9c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5638bdc940e0 con 0x5638bdc8bfc0 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5638bdbabdb0 con 0x5638bdc8b5d0 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638bdc94260 con 0x5638bdc91130 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c594b9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc95310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c58cb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc95850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c594b9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc95310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53946/0 (socket says 172.21.15.105:53946) 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c594b9640 1 -- 172.21.15.105:0/3367322690 learned_addr learned my addr 172.21.15.105:0/3367322690 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c59cba640 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc9c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.589+0000 7f0c5e3e0740 10 client.0 _ll_register_callbacks cb 0x5638bdbb0110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c594b9640 1 -- 172.21.15.105:0/3367322690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 msgr2=0x5638bdc9c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c594b9640 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc9c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c594b9640 1 -- 172.21.15.105:0/3367322690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 msgr2=0x5638bdc95850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c594b9640 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc95850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c594b9640 1 -- 172.21.15.105:0/3367322690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638bd904210 con 0x5638bdc8b5d0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c594b9640 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc95310 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c5009be30 tx=0x7f0c5009bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c7f2b99d013cdfe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3367322690 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c500793f0 con 0x5638bdc8b5d0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3367322690 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f0c50078480 con 0x5638bdc8b5d0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3367322690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c440047a0 con 0x5638bdc8b5d0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3367322690 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c50010de0 con 0x5638bdc8b5d0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c5e3e0740 10 client.28394 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3367322690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638bd9d0620 con 0x5638bdc8b5d0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.590+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3367322690 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c440047a0 con 0x5638bdc8b5d0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3367322690 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0c500ba020 con 0x5638bdc8b5d0 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c5e3e0740 10 client.28394 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c5e3e0740 10 client.28394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c5e3e0740 20 client.28394 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638bdca1110 0x5638bdca3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c5e3e0740 4 client.28394 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3367322690 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638bdbfd5f0 con 0x5638bdca1110 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638bdca1110 0x5638bdca3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638bdca1110 0x5638bdca3560 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebdd2a9a5ad5b55 server_cookie=31fa34847b33b25d in_seq=0 out_seq=0 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.591+0000 7f0c3cff9640 10 client.28394 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3367322690 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5638bdbfd5f0 con 0x5638bdca1110 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c3cff9640 10 client.28394 handle_command_reply: tid=0 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 client.28394 shutdown 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3367322690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 msgr2=0x5638bdc95310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc95310 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c5009be30 tx=0x7f0c5009bb10 comp rx=0 tx=0).stop 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3367322690 shutdown_connections 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638bdca1110 0x5638bdca3560 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc9c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc95850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3367322690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc95310 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c3cff9640 0 client.28394 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3367322690 >> 172.21.15.105:0/3367322690 conn(0x5638bdc86b40 msgr2=0x5638bdc85df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3367322690 shutdown_connections 2024-03-26T17:47:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3367322690 wait complete. 2024-03-26T17:47:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.592+0000 7f0c5e3e0740 20 client.28394 trim_cache size 0 max 16384 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.614+0000 7f0c5e3e0740 1 Processor -- start 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c5e3e0740 1 -- start start 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc8d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc94f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc95460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541aaf30 con 0x5638bdc8bfc0 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541aca70 con 0x5638bdc8b5d0 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541ab600 con 0x5638bdc91130 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc94f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c594b9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc8d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc94f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47200/0 (socket says 172.21.15.105:47200) 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 -- 172.21.15.105:0/3914559333 learned_addr learned my addr 172.21.15.105:0/3914559333 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c59cba640 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc95460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 -- 172.21.15.105:0/3914559333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 msgr2=0x5638bdc95460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc95460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 -- 172.21.15.105:0/3914559333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 msgr2=0x5638bdc8d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc8d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 -- 172.21.15.105:0/3914559333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638bdc8ffb0 con 0x5638bdc8bfc0 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c59cba640 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc95460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.615+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc94f20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f0c4401b040 tx=0x7f0c44005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fa829dc343b17d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c3effd640 1 -- 172.21.15.105:0/3914559333 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4402f070 con 0x5638bdc8bfc0 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c3effd640 1 -- 172.21.15.105:0/3914559333 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c44009bf0 con 0x5638bdc8bfc0 2024-03-26T17:47:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c3effd640 1 -- 172.21.15.105:0/3914559333 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44004280 con 0x5638bdc8bfc0 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3914559333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 msgr2=0x5638bdc94f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc94f20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f0c4401b040 tx=0x7f0c44005d30 comp rx=0 tx=0).stop 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3914559333 shutdown_connections 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc91130 0x5638bdc95460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc8bfc0 0x5638bdc94f20 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3914559333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc8b5d0 0x5638bdc8d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3914559333 >> 172.21.15.105:0/3914559333 conn(0x5638bdc86b40 msgr2=0x5638bdc8ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.616+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3914559333 shutdown_connections 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3914559333 wait complete. 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 -- start start 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 Processor -- start 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc8b5d0 0x5638bdc9a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc91130 0x5638bdc9a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc9b7d0 0x5638bdc9bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541ac3a0 con 0x5638bdc9b7d0 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541abcd0 con 0x5638bdc91130 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541a8f10 con 0x5638bdc8b5d0 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.617+0000 7f0c5e3e0740 10 client.0 _ll_register_callbacks cb 0x7f0c5000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc91130 0x5638bdc9a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c59cba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc9b7d0 0x5638bdc9bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc91130 0x5638bdc9a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:53968/0 (socket says 172.21.15.105:53968) 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c594b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc8b5d0 0x5638bdc9a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 -- 172.21.15.105:0/3809211041 learned_addr learned my addr 172.21.15.105:0/3809211041 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 -- 172.21.15.105:0/3809211041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc8b5d0 msgr2=0x5638bdc9a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc8b5d0 0x5638bdc9a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 -- 172.21.15.105:0/3809211041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc9b7d0 msgr2=0x5638bdc9bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc9b7d0 0x5638bdc9bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 -- 172.21.15.105:0/3809211041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638bdc9c190 con 0x5638bdc91130 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c594b9640 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc8b5d0 0x5638bdc9a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c59cba640 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc9b7d0 0x5638bdc9bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c58cb8640 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc91130 0x5638bdc9a7c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0c440114c0 tx=0x7f0c44009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3c9fcc8d4f8dbc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3809211041 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4402f040 con 0x5638bdc91130 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3809211041 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f0c44005000 con 0x5638bdc91130 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.618+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3809211041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c44009bf0 con 0x5638bdc91130 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3809211041 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c44005200 con 0x5638bdc91130 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c5e3e0740 10 client.28400 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3809211041 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638bdd07e90 con 0x5638bdc91130 2024-03-26T17:47:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3809211041 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c44039030 con 0x5638bdc91130 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3809211041 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0c440035f0 con 0x5638bdc91130 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c5e3e0740 10 client.28400 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c5e3e0740 10 client.28400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c5e3e0740 20 client.28400 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638bdd08560 0x5638bdd0a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c5e3e0740 4 client.28400 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3809211041 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0c44009bf0 con 0x5638bdd08560 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.619+0000 7f0c594b9640 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638bdd08560 0x5638bdd0a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.620+0000 7f0c3cff9640 10 client.28400 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.620+0000 7f0c594b9640 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638bdd08560 0x5638bdd0a980 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da333aae49d9b240 server_cookie=399215c0655c897f in_seq=0 out_seq=0 2024-03-26T17:47:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.669+0000 7f0c3cff9640 1 -- 172.21.15.105:0/3809211041 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0c44009bf0 con 0x5638bdd08560 2024-03-26T17:47:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.669+0000 7f0c3cff9640 10 client.28400 handle_command_reply: tid=0 2024-03-26T17:47:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 client.28400 shutdown 2024-03-26T17:47:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3809211041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc91130 msgr2=0x5638bdc9a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc91130 0x5638bdc9a7c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0c440114c0 tx=0x7f0c44009da0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3809211041 shutdown_connections 2024-03-26T17:47:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5638bdd08560 0x5638bdd0a980 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638bdc9b7d0 0x5638bdc9bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638bdc91130 0x5638bdc9a7c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c3cff9640 0 client.28400 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 --2- 172.21.15.105:0/3809211041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638bdc8b5d0 0x5638bdc9a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3809211041 >> 172.21.15.105:0/3809211041 conn(0x5638bdc86b40 msgr2=0x5638bdc94e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3809211041 shutdown_connections 2024-03-26T17:47:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 1 -- 172.21.15.105:0/3809211041 wait complete. 2024-03-26T17:47:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.670+0000 7f0c5e3e0740 20 client.28400 trim_cache size 0 max 16384 2024-03-26T17:47:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:47:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:47:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:47:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:47:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 -- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c400a2fc0 msgr2=0x7f0c400a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c400a2fc0 0x7f0c400a5480 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006010 tx=0x7f0c4c032ea0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 -- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54102400 msgr2=0x7f0c54199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54102400 0x7f0c54199ca0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000e9f0 tx=0x7f0c5000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 -- 172.21.15.105:0/415813543 shutdown_connections 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c400a2fc0 0x7f0c400a5480 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c54107660 0x7f0c541a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c54104d30 0x7f0c5419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 --2- 172.21.15.105:0/415813543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c54102400 0x7f0c54199ca0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 -- 172.21.15.105:0/415813543 >> 172.21.15.105:0/415813543 conn(0x7f0c540fbd50 msgr2=0x7f0c54101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 -- 172.21.15.105:0/415813543 shutdown_connections 2024-03-26T17:47:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:01.671+0000 7f0c5b71d640 1 -- 172.21.15.105:0/415813543 wait complete. 2024-03-26T17:47:02.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 273382 3.16 MB/sec execute 345 sec latency 38.055 ms 2024-03-26T17:47:02.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:01 smithi144 ceph-mon[36095]: pgmap v608: 97 pgs: 97 active+clean; 30 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:47:02.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:01 smithi105 ceph-mon[32280]: pgmap v608: 97 pgs: 97 active+clean; 30 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:47:02.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:01 smithi135 ceph-mon[35636]: pgmap v608: 97 pgs: 97 active+clean; 30 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.6 MiB/s wr, 105 op/s 2024-03-26T17:47:03.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 273760 3.15 MB/sec execute 346 sec latency 18.829 ms 2024-03-26T17:47:04.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 274327 3.15 MB/sec execute 347 sec latency 17.825 ms 2024-03-26T17:47:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:03 smithi144 ceph-mon[36095]: pgmap v609: 97 pgs: 97 active+clean; 32 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:47:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:03 smithi105 ceph-mon[32280]: pgmap v609: 97 pgs: 97 active+clean; 32 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:47:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:03 smithi135 ceph-mon[35636]: pgmap v609: 97 pgs: 97 active+clean; 32 MiB data, 974 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:47:05.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 274999 3.15 MB/sec execute 348 sec latency 19.536 ms 2024-03-26T17:47:06.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 275623 3.15 MB/sec execute 349 sec latency 18.555 ms 2024-03-26T17:47:06.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:05 smithi144 ceph-mon[36095]: pgmap v610: 97 pgs: 97 active+clean; 27 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-03-26T17:47:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:05 smithi105 ceph-mon[32280]: pgmap v610: 97 pgs: 97 active+clean; 27 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-03-26T17:47:06.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:05 smithi135 ceph-mon[35636]: pgmap v610: 97 pgs: 97 active+clean; 27 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-03-26T17:47:07.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 276441 3.16 MB/sec execute 350 sec latency 19.864 ms 2024-03-26T17:47:08.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 276938 3.16 MB/sec execute 351 sec latency 15.570 ms 2024-03-26T17:47:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:07 smithi144 ceph-mon[36095]: pgmap v611: 97 pgs: 97 active+clean; 28 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:47:08.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:07 smithi105 ceph-mon[32280]: pgmap v611: 97 pgs: 97 active+clean; 28 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:47:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:07 smithi135 ceph-mon[35636]: pgmap v611: 97 pgs: 97 active+clean; 28 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:47:09.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 277310 3.15 MB/sec execute 352 sec latency 30.452 ms 2024-03-26T17:47:10.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 277876 3.15 MB/sec execute 353 sec latency 27.118 ms 2024-03-26T17:47:10.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:09 smithi144 ceph-mon[36095]: pgmap v612: 97 pgs: 97 active+clean; 28 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:47:10.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:09 smithi105 ceph-mon[32280]: pgmap v612: 97 pgs: 97 active+clean; 28 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:47:10.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:09 smithi135 ceph-mon[35636]: pgmap v612: 97 pgs: 97 active+clean; 28 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:47:11.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 278567 3.15 MB/sec execute 354 sec latency 16.139 ms 2024-03-26T17:47:12.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 279192 3.15 MB/sec execute 355 sec latency 16.082 ms 2024-03-26T17:47:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:11 smithi144 ceph-mon[36095]: pgmap v613: 97 pgs: 97 active+clean; 28 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-03-26T17:47:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:11 smithi105 ceph-mon[32280]: pgmap v613: 97 pgs: 97 active+clean; 28 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-03-26T17:47:12.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:11 smithi135 ceph-mon[35636]: pgmap v613: 97 pgs: 97 active+clean; 28 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-03-26T17:47:12.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:47:12.892 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:47:13.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 -- 172.21.15.105:0/123957107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6004102370 msgr2=0x7f6004108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 --2- 172.21.15.105:0/123957107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6004102370 0x7f6004108c00 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009ca0 tx=0x7f5ffc02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 -- 172.21.15.105:0/123957107 shutdown_connections 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 --2- 172.21.15.105:0/123957107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6004102370 0x7f6004108c00 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 --2- 172.21.15.105:0/123957107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60041019b0 0x7f6004101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 --2- 172.21.15.105:0/123957107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f60041007c0 0x7f6004100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 -- 172.21.15.105:0/123957107 >> 172.21.15.105:0/123957107 conn(0x7f60040fbd50 msgr2=0x7f60040fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 -- 172.21.15.105:0/123957107 shutdown_connections 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 -- 172.21.15.105:0/123957107 wait complete. 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 Processor -- start 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.016+0000 7f600bad1640 1 -- start start 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600bad1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60041007c0 0x7f60040695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600bad1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60041019b0 0x7f600419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600bad1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6004102370 0x7f600419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600bad1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f600410c2f0 con 0x7f60041007c0 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600bad1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f600410c170 con 0x7f6004102370 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600bad1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f600410c470 con 0x7f60041019b0 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60041007c0 0x7f60040695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60041007c0 0x7f60040695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56906/0 (socket says 172.21.15.105:56906) 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 -- 172.21.15.105:0/2565437014 learned_addr learned my addr 172.21.15.105:0/2565437014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600906c640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60041019b0 0x7f600419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600a06e640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6004102370 0x7f600419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 -- 172.21.15.105:0/2565437014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60041019b0 msgr2=0x7f600419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60041019b0 0x7f600419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 -- 172.21.15.105:0/2565437014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6004102370 msgr2=0x7f600419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6004102370 0x7f600419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 -- 172.21.15.105:0/2565437014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600419f6f0 con 0x7f60041007c0 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600906c640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60041019b0 0x7f600419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600a06e640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6004102370 0x7f600419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f600986d640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60041007c0 0x7f60040695b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f600000e9c0 tx=0x7f600000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.017+0000 7f5ffaffd640 1 -- 172.21.15.105:0/2565437014 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000cd80 con 0x7f60041007c0 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.018+0000 7f5ffaffd640 1 -- 172.21.15.105:0/2565437014 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60000035f0 con 0x7f60041007c0 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.018+0000 7f600bad1640 1 -- 172.21.15.105:0/2565437014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600419f9e0 con 0x7f60041007c0 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.018+0000 7f5ffaffd640 1 -- 172.21.15.105:0/2565437014 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000010690 con 0x7f60041007c0 2024-03-26T17:47:13.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.018+0000 7f600bad1640 1 -- 172.21.15.105:0/2565437014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f600419ff30 con 0x7f60041007c0 2024-03-26T17:47:13.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.020+0000 7f5ffaffd640 1 -- 172.21.15.105:0/2565437014 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f60000037a0 con 0x7f60041007c0 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.020+0000 7f5ffaffd640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ff00a2e00 0x7f5ff00a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600906c640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ff00a2e00 0x7f5ff00a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f5ffaffd640 1 -- 172.21.15.105:0/2565437014 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f60000cf790 con 0x7f60041007c0 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600e796740 1 Processor -- start 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600906c640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ff00a2e00 0x7f5ff00a52c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6004069320 tx=0x7f5ff4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600e796740 1 -- start start 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600e796740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600e796740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600e796740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77d88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600e796740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c9b7735980 con 0x55c9b77d6450 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600e796740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9b76ddbe0 con 0x55c9b77d12e0 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.021+0000 7f600e796740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c9b7731930 con 0x55c9b77d08f0 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77d88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600906c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77d88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56916/0 (socket says 172.21.15.105:56916) 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 -- 172.21.15.105:0/1811468904 learned_addr learned my addr 172.21.15.105:0/1811468904 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600986d640 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 -- 172.21.15.105:0/1811468904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 msgr2=0x55c9b77d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 -- 172.21.15.105:0/1811468904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 msgr2=0x55c9b77d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 -- 172.21.15.105:0/1811468904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b75137d0 con 0x55c9b77d6450 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600906c640 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f600a06e640 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77d88e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc033000 tx=0x7f5ffc0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47eefeae17a0563 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f5feffff640 1 -- 172.21.15.105:0/1811468904 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc043070 con 0x55c9b77d6450 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f5feffff640 1 -- 172.21.15.105:0/1811468904 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ffc003e20 con 0x55c9b77d6450 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.022+0000 7f5feffff640 1 -- 172.21.15.105:0/1811468904 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc039040 con 0x55c9b77d6450 2024-03-26T17:47:13.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 -- 172.21.15.105:0/1811468904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 msgr2=0x55c9b77d88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77d88e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc033000 tx=0x7f5ffc0093e0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 -- 172.21.15.105:0/1811468904 shutdown_connections 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77d88e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 --2- 172.21.15.105:0/1811468904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 -- 172.21.15.105:0/1811468904 >> 172.21.15.105:0/1811468904 conn(0x55c9b77cbe60 msgr2=0x55c9b77cc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 -- 172.21.15.105:0/1811468904 shutdown_connections 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.023+0000 7f600e796740 1 -- 172.21.15.105:0/1811468904 wait complete. 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 1 -- start start 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 1 Processor -- start 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d08f0 0x55c9b77daab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77daff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d6450 0x55c9b77e2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c9b77d93a0 con 0x55c9b77d08f0 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9b76f1160 con 0x55c9b77d12e0 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c9b77d9520 con 0x55c9b77d6450 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600986d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d08f0 0x55c9b77daab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600e796740 10 client.0 _ll_register_callbacks cb 0x55c9b772d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600906c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77daff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600906c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77daff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44742/0 (socket says 172.21.15.105:44742) 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600906c640 1 -- 172.21.15.105:0/113292250 learned_addr learned my addr 172.21.15.105:0/113292250 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.024+0000 7f600a06e640 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d6450 0x55c9b77e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600906c640 1 -- 172.21.15.105:0/113292250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d6450 msgr2=0x55c9b77e2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600906c640 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d6450 0x55c9b77e2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600906c640 1 -- 172.21.15.105:0/113292250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d08f0 msgr2=0x55c9b77daab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600906c640 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d08f0 0x55c9b77daab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600906c640 1 -- 172.21.15.105:0/113292250 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b74493f0 con 0x55c9b77d12e0 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600a06e640 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d6450 0x55c9b77e2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600906c640 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77daff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ff401d820 tx=0x7f5ff4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8202aea0f424eccb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f5fed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f5fed7fa640 1 -- 172.21.15.105:0/113292250 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff401de60 con 0x55c9b77d12e0 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600e796740 1 -- 172.21.15.105:0/113292250 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ffc003e20 con 0x55c9b77d12e0 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f5fed7fa640 1 -- 172.21.15.105:0/113292250 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f5ff4029460 con 0x55c9b77d12e0 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f5fed7fa640 1 -- 172.21.15.105:0/113292250 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ff4029660 con 0x55c9b77d12e0 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600e796740 10 client.28412 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:13.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.025+0000 7f600e796740 1 -- 172.21.15.105:0/113292250 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9b74090f0 con 0x55c9b77d12e0 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f5fed7fa640 1 -- 172.21.15.105:0/113292250 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ffc003e20 con 0x55c9b77d12e0 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f5fed7fa640 1 -- 172.21.15.105:0/113292250 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5ff400f8c0 con 0x55c9b77d12e0 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f600e796740 10 client.28412 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f600e796740 10 client.28412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f600e796740 20 client.28412 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f600e796740 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c9b77e6930 0x55c9b77e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f600e796740 4 client.28412 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f600e796740 1 -- 172.21.15.105:0/113292250 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9b7742ec0 con 0x55c9b77e6930 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.026+0000 7f600986d640 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c9b77e6930 0x55c9b77e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.027+0000 7f5fed7fa640 10 client.28412 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.027+0000 7f600986d640 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c9b77e6930 0x55c9b77e8d90 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca2e0cb4020a08b server_cookie=bc451fea4b530798 in_seq=0 out_seq=0 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f5fed7fa640 1 -- 172.21.15.105:0/113292250 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c9b7742ec0 con 0x55c9b77e6930 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f5fed7fa640 10 client.28412 handle_command_reply: tid=0 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 client.28412 shutdown 2024-03-26T17:47:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 -- 172.21.15.105:0/113292250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 msgr2=0x55c9b77daff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77daff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ff401d820 tx=0x7f5ff4004730 comp rx=0 tx=0).stop 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 -- 172.21.15.105:0/113292250 shutdown_connections 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c9b77e6930 0x55c9b77e8d90 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d6450 0x55c9b77e2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77daff0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 --2- 172.21.15.105:0/113292250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d08f0 0x55c9b77daab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 -- 172.21.15.105:0/113292250 >> 172.21.15.105:0/113292250 conn(0x55c9b77cbe60 msgr2=0x55c9b77cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f5fed7fa640 0 client.28412 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 -- 172.21.15.105:0/113292250 shutdown_connections 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 1 -- 172.21.15.105:0/113292250 wait complete. 2024-03-26T17:47:13.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.028+0000 7f600e796740 20 client.28412 trim_cache size 0 max 16384 2024-03-26T17:47:13.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 279982 3.16 MB/sec execute 356 sec latency 98.853 ms 2024-03-26T17:47:13.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600e796740 1 Processor -- start 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600e796740 1 -- start start 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600e796740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600e796740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77e1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600e796740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600e796740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f60041a6a70 con 0x55c9b77d6450 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600e796740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60041a85b0 con 0x55c9b77d12e0 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600e796740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60041a7140 con 0x55c9b77d08f0 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600a06e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600a06e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77da2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56948/0 (socket says 172.21.15.105:56948) 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600906c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77e1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600a06e640 1 -- 172.21.15.105:0/3264049725 learned_addr learned my addr 172.21.15.105:0/3264049725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600986d640 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600986d640 1 -- 172.21.15.105:0/3264049725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 msgr2=0x55c9b77e1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600986d640 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77e1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600986d640 1 -- 172.21.15.105:0/3264049725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 msgr2=0x55c9b77da2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600986d640 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77da2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600986d640 1 -- 172.21.15.105:0/3264049725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b77d16c0 con 0x55c9b77d08f0 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.050+0000 7f600a06e640 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77da2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.051+0000 7f600986d640 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d9d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60000cd2a0 tx=0x7f60000bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19647d2ac3198b21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.051+0000 7f5fef7fe640 1 -- 172.21.15.105:0/3264049725 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600009ffd0 con 0x55c9b77d08f0 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.051+0000 7f5fef7fe640 1 -- 172.21.15.105:0/3264049725 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60000a4890 con 0x55c9b77d08f0 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.051+0000 7f5fef7fe640 1 -- 172.21.15.105:0/3264049725 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000097bd0 con 0x55c9b77d08f0 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 -- 172.21.15.105:0/3264049725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 msgr2=0x55c9b77d9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d9d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f60000cd2a0 tx=0x7f60000bc040 comp rx=0 tx=0).stop 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 -- 172.21.15.105:0/3264049725 shutdown_connections 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77da2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d12e0 0x55c9b77e1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 --2- 172.21.15.105:0/3264049725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d08f0 0x55c9b77d9d80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 -- 172.21.15.105:0/3264049725 >> 172.21.15.105:0/3264049725 conn(0x55c9b77cbe60 msgr2=0x55c9b77d25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 -- 172.21.15.105:0/3264049725 shutdown_connections 2024-03-26T17:47:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 -- 172.21.15.105:0/3264049725 wait complete. 2024-03-26T17:47:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 -- start start 2024-03-26T17:47:13.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 Processor -- start 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d08f0 0x55c9b77df260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.052+0000 7f600e796740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d12e0 0x55c9b77df7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600e796740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600e796740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f60041a7ee0 con 0x55c9b77d6450 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600e796740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60041a7810 con 0x55c9b77d08f0 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600e796740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60041a4a50 con 0x55c9b77d12e0 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600e796740 10 client.0 _ll_register_callbacks cb 0x7f5ff40040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d08f0 0x55c9b77df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600a06e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d08f0 0x55c9b77df260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44752/0 (socket says 172.21.15.105:44752) 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 -- 172.21.15.105:0/3308757110 learned_addr learned my addr 172.21.15.105:0/3308757110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600a06e640 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77e07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56960/0 (socket says 172.21.15.105:56960) 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600906c640 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d12e0 0x55c9b77df7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 -- 172.21.15.105:0/3308757110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d12e0 msgr2=0x55c9b77df7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d12e0 0x55c9b77df7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 -- 172.21.15.105:0/3308757110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 msgr2=0x55c9b77e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77e07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 -- 172.21.15.105:0/3308757110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b77e0da0 con 0x55c9b77d08f0 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600a06e640 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600986d640 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d08f0 0x55c9b77df260 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f600000c6a0 tx=0x7f600009bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fad2c7945d169c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f5fed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f5fed7fa640 1 -- 172.21.15.105:0/3308757110 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600007e070 con 0x55c9b77d08f0 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f5fed7fa640 1 -- 172.21.15.105:0/3308757110 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f60000984f0 con 0x55c9b77d08f0 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.053+0000 7f600e796740 1 -- 172.21.15.105:0/3308757110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60000a4890 con 0x55c9b77d08f0 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f5fed7fa640 1 -- 172.21.15.105:0/3308757110 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f600009ffd0 con 0x55c9b77d08f0 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600e796740 10 client.28418 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600e796740 1 -- 172.21.15.105:0/3308757110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9b77e20b0 con 0x55c9b77d08f0 2024-03-26T17:47:13.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f5fed7fa640 1 -- 172.21.15.105:0/3308757110 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60000a4890 con 0x55c9b77d08f0 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f5fed7fa640 1 -- 172.21.15.105:0/3308757110 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f60000b8030 con 0x55c9b77d08f0 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600e796740 10 client.28418 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600e796740 10 client.28418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600e796740 20 client.28418 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600e796740 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c9b784d120 0x55c9b784f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600e796740 4 client.28418 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600e796740 1 -- 172.21.15.105:0/3308757110 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9b773d210 con 0x55c9b784d120 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.054+0000 7f600906c640 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c9b784d120 0x55c9b784f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f5fed7fa640 10 client.28418 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600906c640 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c9b784d120 0x55c9b784f540 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5bcb1369771353 server_cookie=59cbd02b4071d1c8 in_seq=0 out_seq=0 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f5fed7fa640 1 -- 172.21.15.105:0/3308757110 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c9b773d210 con 0x55c9b784d120 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f5fed7fa640 10 client.28418 handle_command_reply: tid=0 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 client.28418 shutdown 2024-03-26T17:47:13.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 -- 172.21.15.105:0/3308757110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d08f0 msgr2=0x55c9b77df260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d08f0 0x55c9b77df260 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f600000c6a0 tx=0x7f600009bd80 comp rx=0 tx=0).stop 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 -- 172.21.15.105:0/3308757110 shutdown_connections 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c9b784d120 0x55c9b784f540 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b77d6450 0x55c9b77e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f5fed7fa640 0 client.28418 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b77d12e0 0x55c9b77df7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 --2- 172.21.15.105:0/3308757110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b77d08f0 0x55c9b77df260 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 -- 172.21.15.105:0/3308757110 >> 172.21.15.105:0/3308757110 conn(0x55c9b77cbe60 msgr2=0x55c9b77d9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 -- 172.21.15.105:0/3308757110 shutdown_connections 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 1 -- 172.21.15.105:0/3308757110 wait complete. 2024-03-26T17:47:13.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.055+0000 7f600e796740 20 client.28418 trim_cache size 0 max 16384 2024-03-26T17:47:13.054 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:47:13.054 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:47:13.054 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:47:13.054 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 -- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ff00a2e00 msgr2=0x7f5ff00a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ff00a2e00 0x7f5ff00a52c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6004069320 tx=0x7f5ff4005f90 comp rx=0 tx=0).stop 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 -- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60041007c0 msgr2=0x7f60040695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60041007c0 0x7f60040695b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f600000e9c0 tx=0x7f600000ee80 comp rx=0 tx=0).stop 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 -- 172.21.15.105:0/2565437014 shutdown_connections 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5ff00a2e00 0x7f5ff00a52c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6004102370 0x7f600419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60041019b0 0x7f600419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 --2- 172.21.15.105:0/2565437014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f60041007c0 0x7f60040695b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 -- 172.21.15.105:0/2565437014 >> 172.21.15.105:0/2565437014 conn(0x7f60040fbd50 msgr2=0x7f6004104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 -- 172.21.15.105:0/2565437014 shutdown_connections 2024-03-26T17:47:13.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.056+0000 7f600bad1640 1 -- 172.21.15.105:0/2565437014 wait complete. 2024-03-26T17:47:13.058 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:47:13.059 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:47:13.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 -- 172.21.15.105:0/3302600790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc1019d0 msgr2=0x7f13bc101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 --2- 172.21.15.105:0/3302600790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc1019d0 0x7f13bc101e50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009900 tx=0x7f13a402bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 -- 172.21.15.105:0/3302600790 shutdown_connections 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 --2- 172.21.15.105:0/3302600790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13bc102390 0x7f13bc108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 --2- 172.21.15.105:0/3302600790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc1019d0 0x7f13bc101e50 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 --2- 172.21.15.105:0/3302600790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc1007e0 0x7f13bc100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 -- 172.21.15.105:0/3302600790 >> 172.21.15.105:0/3302600790 conn(0x7f13bc0fbd50 msgr2=0x7f13bc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 -- 172.21.15.105:0/3302600790 shutdown_connections 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.188+0000 7f13c0827640 1 -- 172.21.15.105:0/3302600790 wait complete. 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13c0827640 1 Processor -- start 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13c0827640 1 -- start start 2024-03-26T17:47:13.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13c0827640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc1007e0 0x7f13bc199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13c0827640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13bc1019d0 0x7f13bc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13c0827640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc102390 0x7f13bc1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13c0827640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc10c2f0 con 0x7f13bc102390 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13c0827640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc10c170 con 0x7f13bc1019d0 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13c0827640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc10c470 con 0x7f13bc1007e0 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13bad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc102390 0x7f13bc1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13ba59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc1007e0 0x7f13bc199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13bad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc102390 0x7f13bc1a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56976/0 (socket says 172.21.15.105:56976) 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13ba59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc1007e0 0x7f13bc199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54406/0 (socket says 172.21.15.105:54406) 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13b9d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13bc1019d0 0x7f13bc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13bad9d640 1 -- 172.21.15.105:0/1546798150 learned_addr learned my addr 172.21.15.105:0/1546798150 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13ba59c640 1 -- 172.21.15.105:0/1546798150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13bc1019d0 msgr2=0x7f13bc19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13ba59c640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13bc1019d0 0x7f13bc19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13ba59c640 1 -- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc102390 msgr2=0x7f13bc1a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13ba59c640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc102390 0x7f13bc1a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13ba59c640 1 -- 172.21.15.105:0/1546798150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc1a1960 con 0x7f13bc1007e0 2024-03-26T17:47:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.189+0000 7f13bad9d640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc102390 0x7f13bc1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.190+0000 7f13ba59c640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc1007e0 0x7f13bc199d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f13ac005fd0 tx=0x7f13ac00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.190+0000 7f13b37fe640 1 -- 172.21.15.105:0/1546798150 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac003970 con 0x7f13bc1007e0 2024-03-26T17:47:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.190+0000 7f13b37fe640 1 -- 172.21.15.105:0/1546798150 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13ac004030 con 0x7f13bc1007e0 2024-03-26T17:47:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.190+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc1a1c50 con 0x7f13bc1007e0 2024-03-26T17:47:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.190+0000 7f13b37fe640 1 -- 172.21.15.105:0/1546798150 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac010720 con 0x7f13bc1007e0 2024-03-26T17:47:13.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.190+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc1a2170 con 0x7f13bc1007e0 2024-03-26T17:47:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.191+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13bc1a2340 con 0x7f13bc1007e0 2024-03-26T17:47:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.192+0000 7f13b37fe640 1 -- 172.21.15.105:0/1546798150 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f13ac0041e0 con 0x7f13bc1007e0 2024-03-26T17:47:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.192+0000 7f13b37fe640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13a80a2f20 0x7f13a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.193+0000 7f13b9d9b640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13a80a2f20 0x7f13a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.193+0000 7f13b37fe640 1 -- 172.21.15.105:0/1546798150 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f13ac0cf880 con 0x7f13bc1007e0 2024-03-26T17:47:13.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.193+0000 7f13b9d9b640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13a80a2f20 0x7f13a80a53e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009230 tx=0x7f13a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.197+0000 7f13b37fe640 1 -- 172.21.15.105:0/1546798150 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f13ac09c670 con 0x7f13bc1007e0 2024-03-26T17:47:13.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.510+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13bc1a2510 con 0x7f13bc1007e0 2024-03-26T17:47:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.511+0000 7f13b37fe640 1 -- 172.21.15.105:0/1546798150 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f13ac09bf70 con 0x7f13bc1007e0 2024-03-26T17:47:13.502 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:47:13.502 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:47:13.502 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:47:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.512+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13a80a2f20 msgr2=0x7f13a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.512+0000 7f13c0827640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13a80a2f20 0x7f13a80a53e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009230 tx=0x7f13a4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.512+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc1007e0 msgr2=0x7f13bc199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.512+0000 7f13c0827640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc1007e0 0x7f13bc199d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f13ac005fd0 tx=0x7f13ac00ef90 comp rx=0 tx=0).stop 2024-03-26T17:47:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.513+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 shutdown_connections 2024-03-26T17:47:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.513+0000 7f13c0827640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13a80a2f20 0x7f13a80a53e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.513+0000 7f13c0827640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13bc102390 0x7f13bc1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.513+0000 7f13c0827640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13bc1019d0 0x7f13bc19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.513+0000 7f13c0827640 1 --2- 172.21.15.105:0/1546798150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc1007e0 0x7f13bc199d40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.513+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 >> 172.21.15.105:0/1546798150 conn(0x7f13bc0fbd50 msgr2=0x7f13bc068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.513+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 shutdown_connections 2024-03-26T17:47:13.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.513+0000 7f13c0827640 1 -- 172.21.15.105:0/1546798150 wait complete. 2024-03-26T17:47:13.516 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:47:13.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 -- 172.21.15.105:0/3011838363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c102340 msgr2=0x7f868c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 --2- 172.21.15.105:0/3011838363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c102340 0x7f868c104730 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f867c009900 tx=0x7f867c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 -- 172.21.15.105:0/3011838363 shutdown_connections 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 --2- 172.21.15.105:0/3011838363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f868c1075a0 0x7f868c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 --2- 172.21.15.105:0/3011838363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f868c104c70 0x7f868c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 --2- 172.21.15.105:0/3011838363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c102340 0x7f868c104730 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 -- 172.21.15.105:0/3011838363 >> 172.21.15.105:0/3011838363 conn(0x7f868c0fbd50 msgr2=0x7f868c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 -- 172.21.15.105:0/3011838363 shutdown_connections 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.650+0000 7f86926dc640 1 -- 172.21.15.105:0/3011838363 wait complete. 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f86926dc640 1 Processor -- start 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f86926dc640 1 -- start start 2024-03-26T17:47:13.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f86926dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f868c102340 0x7f868c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f86926dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f868c104c70 0x7f868c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f86926dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c1075a0 0x7f868c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f86926dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10d410 con 0x7f868c1075a0 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f86926dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10d290 con 0x7f868c104c70 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f86926dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10d590 con 0x7f868c102340 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f8690c79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c1075a0 0x7f868c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f868bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f868c102340 0x7f868c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f8690c79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c1075a0 0x7f868c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56998/0 (socket says 172.21.15.105:56998) 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f8690c79640 1 -- 172.21.15.105:0/2877406067 learned_addr learned my addr 172.21.15.105:0/2877406067 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f868c104c70 0x7f868c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.651+0000 7f868bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f868c102340 0x7f868c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54424/0 (socket says 172.21.15.105:54424) 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f868bfff640 1 -- 172.21.15.105:0/2877406067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f868c104c70 msgr2=0x7f868c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f868bfff640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f868c104c70 0x7f868c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f868bfff640 1 -- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c1075a0 msgr2=0x7f868c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f868bfff640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c1075a0 0x7f868c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f868bfff640 1 -- 172.21.15.105:0/2877406067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c19f7a0 con 0x7f868c102340 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f868b7fe640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f868c104c70 0x7f868c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f8690c79640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c1075a0 0x7f868c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f868bfff640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f868c102340 0x7f868c197b80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f867c000c00 tx=0x7f867c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f86897fa640 1 -- 172.21.15.105:0/2877406067 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c043020 con 0x7f868c102340 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f86926dc640 1 -- 172.21.15.105:0/2877406067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c19fa30 con 0x7f868c102340 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f86926dc640 1 -- 172.21.15.105:0/2877406067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c19ffd0 con 0x7f868c102340 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f86897fa640 1 -- 172.21.15.105:0/2877406067 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f867c004350 con 0x7f868c102340 2024-03-26T17:47:13.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.652+0000 7f86897fa640 1 -- 172.21.15.105:0/2877406067 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c003b50 con 0x7f868c102340 2024-03-26T17:47:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86897fa640 1 -- 172.21.15.105:0/2877406067 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f867c046020 con 0x7f868c102340 2024-03-26T17:47:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86953a3740 1 Processor -- start 2024-03-26T17:47:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86953a3740 1 -- start start 2024-03-26T17:47:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86953a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa607260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86953a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa60c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86953a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 0x55abfa60ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86953a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55abfa567970 con 0x55abfa607830 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86953a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55abfa50fd70 con 0x55abfa606e40 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f86953a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55abfa563920 con 0x55abfa60c9a0 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f868bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa60c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f8690c79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa607260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f8690c79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa607260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44786/0 (socket says 172.21.15.105:44786) 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f8690c79640 1 -- 172.21.15.105:0/3800602936 learned_addr learned my addr 172.21.15.105:0/3800602936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f868b7fe640 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 0x55abfa60ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f8690c79640 1 -- 172.21.15.105:0/3800602936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 msgr2=0x55abfa60ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f8690c79640 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 0x55abfa60ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f8690c79640 1 -- 172.21.15.105:0/3800602936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 msgr2=0x55abfa60c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.655+0000 7f8690c79640 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa60c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f8690c79640 1 -- 172.21.15.105:0/3800602936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfa27b3f0 con 0x55abfa606e40 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f868bfff640 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa60c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f86897fa640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86740a2fa0 0x7f86740a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f868bfff640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86740a2fa0 0x7f86740a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f8690c79640 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa607260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f86840118b0 tx=0x7f8684011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8197de4baca934cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f86897fa640 1 -- 172.21.15.105:0/2877406067 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f867c0f1420 con 0x7f868c102340 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f86727fc640 1 -- 172.21.15.105:0/3800602936 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868401e070 con 0x55abfa606e40 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f86727fc640 1 -- 172.21.15.105:0/3800602936 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f868400fe70 con 0x55abfa606e40 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.656+0000 7f86727fc640 1 -- 172.21.15.105:0/3800602936 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684019d30 con 0x55abfa606e40 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- 172.21.15.105:0/3800602936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 msgr2=0x55abfa607260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa607260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f86840118b0 tx=0x7f8684011d70 comp rx=0 tx=0).stop 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f868bfff640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86740a2fa0 0x7f86740a5460 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f867c0ef250 tx=0x7f867c0f6040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- 172.21.15.105:0/3800602936 shutdown_connections 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 0x55abfa60ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa60c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 --2- 172.21.15.105:0/3800602936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa607260 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- 172.21.15.105:0/3800602936 >> 172.21.15.105:0/3800602936 conn(0x55abfa6023b0 msgr2=0x55abfa6027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- 172.21.15.105:0/3800602936 shutdown_connections 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- 172.21.15.105:0/3800602936 wait complete. 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- start start 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 Processor -- start 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa6112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa611800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 0x55abfa618840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55abfa60f950 con 0x55abfa607830 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55abfa523310 con 0x55abfa606e40 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.657+0000 7f86953a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55abfa60fad0 con 0x55abfa60c9a0 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa611800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa6112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f86953a3740 10 client.0 _ll_register_callbacks cb 0x55abfa55ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa6112c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44792/0 (socket says 172.21.15.105:44792) 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa611800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57010/0 (socket says 172.21.15.105:57010) 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 -- 172.21.15.105:0/3704094219 learned_addr learned my addr 172.21.15.105:0/3704094219 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 0x55abfa618840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 -- 172.21.15.105:0/3704094219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 msgr2=0x55abfa618840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 0x55abfa618840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 -- 172.21.15.105:0/3704094219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 msgr2=0x55abfa611800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa611800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 -- 172.21.15.105:0/3704094219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfa3457d0 con 0x55abfa606e40 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f8690c79640 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa6112c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8684012fd0 tx=0x7f8684019770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f112206575d51741 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f866b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f866b7fe640 1 -- 172.21.15.105:0/3704094219 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868401e070 con 0x55abfa606e40 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f86953a3740 1 -- 172.21.15.105:0/3704094219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f868400fe70 con 0x55abfa606e40 2024-03-26T17:47:13.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f866b7fe640 1 -- 172.21.15.105:0/3704094219 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8684019a70 con 0x55abfa606e40 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f866b7fe640 1 -- 172.21.15.105:0/3704094219 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8684011520 con 0x55abfa606e40 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f86953a3740 10 client.28436 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.658+0000 7f86953a3740 1 -- 172.21.15.105:0/3704094219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abfa61bbe0 con 0x55abfa606e40 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f866b7fe640 1 -- 172.21.15.105:0/3704094219 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f868400fe70 con 0x55abfa606e40 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f866b7fe640 1 -- 172.21.15.105:0/3704094219 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f868401a600 con 0x55abfa606e40 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f86953a3740 10 client.28436 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f86953a3740 10 client.28436 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f86953a3740 20 client.28436 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f86953a3740 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55abfa61d100 0x55abfa61f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f86953a3740 4 client.28436 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f86953a3740 1 -- 172.21.15.105:0/3704094219 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abfa574eb0 con 0x55abfa61d100 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f868b7fe640 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55abfa61d100 0x55abfa61f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f866b7fe640 10 client.28436 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.659+0000 7f868b7fe640 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55abfa61d100 0x55abfa61f560 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3bf62b784d67fb23 server_cookie=d3c8aa0be708b532 in_seq=0 out_seq=0 2024-03-26T17:47:13.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f866b7fe640 1 -- 172.21.15.105:0/3704094219 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55abfa574eb0 con 0x55abfa61d100 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f866b7fe640 10 client.28436 handle_command_reply: tid=0 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 client.28436 shutdown 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 -- 172.21.15.105:0/3704094219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 msgr2=0x55abfa6112c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa6112c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8684012fd0 tx=0x7f8684019770 comp rx=0 tx=0).stop 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 -- 172.21.15.105:0/3704094219 shutdown_connections 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55abfa61d100 0x55abfa61f560 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa60c9a0 0x55abfa618840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa611800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 --2- 172.21.15.105:0/3704094219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa606e40 0x55abfa6112c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f86953a3740 1 -- 172.21.15.105:0/3704094219 >> 172.21.15.105:0/3704094219 conn(0x55abfa6023b0 msgr2=0x55abfa5fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.660+0000 7f866b7fe640 0 client.28436 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:47:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.661+0000 7f86953a3740 1 -- 172.21.15.105:0/3704094219 shutdown_connections 2024-03-26T17:47:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.661+0000 7f86953a3740 1 -- 172.21.15.105:0/3704094219 wait complete. 2024-03-26T17:47:13.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.661+0000 7f86953a3740 20 client.28436 trim_cache size 0 max 16384 2024-03-26T17:47:13.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f86953a3740 1 Processor -- start 2024-03-26T17:47:13.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f86953a3740 1 -- start start 2024-03-26T17:47:13.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f86953a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 0x55abfa5f9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f86953a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa618800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f86953a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa6115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f86953a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a6b40 con 0x55abfa607830 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f86953a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a8680 con 0x55abfa60c9a0 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f86953a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a7210 con 0x55abfa606e40 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa618800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa618800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57020/0 (socket says 172.21.15.105:57020) 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 -- 172.21.15.105:0/1644615308 learned_addr learned my addr 172.21.15.105:0/1644615308 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f8690c79640 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa6115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f8690c79640 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 0x55abfa5f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 -- 172.21.15.105:0/1644615308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 msgr2=0x55abfa5f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 0x55abfa5f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 -- 172.21.15.105:0/1644615308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 msgr2=0x55abfa6115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa6115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 -- 172.21.15.105:0/1644615308 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfa5f9820 con 0x55abfa607830 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.678+0000 7f868b7fe640 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa618800 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x55abfa603320 tx=0x7f8680005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95517d70dcc17a5f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f8671ffb640 1 -- 172.21.15.105:0/1644615308 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680024070 con 0x55abfa607830 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f8671ffb640 1 -- 172.21.15.105:0/1644615308 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86800047a0 con 0x55abfa607830 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f8671ffb640 1 -- 172.21.15.105:0/1644615308 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86800202d0 con 0x55abfa607830 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f86953a3740 1 -- 172.21.15.105:0/1644615308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 msgr2=0x55abfa618800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f86953a3740 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa618800 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x55abfa603320 tx=0x7f8680005f90 comp rx=0 tx=0).stop 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f86953a3740 1 -- 172.21.15.105:0/1644615308 shutdown_connections 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f86953a3740 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa6115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f86953a3740 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa618800 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f86953a3740 1 --2- 172.21.15.105:0/1644615308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 0x55abfa5f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f86953a3740 1 -- 172.21.15.105:0/1644615308 >> 172.21.15.105:0/1644615308 conn(0x55abfa6023b0 msgr2=0x55abfa610ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.679+0000 7f86953a3740 1 -- 172.21.15.105:0/1644615308 shutdown_connections 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 -- 172.21.15.105:0/1644615308 wait complete. 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 -- start start 2024-03-26T17:47:13.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 Processor -- start 2024-03-26T17:47:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 0x55abfa615590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa615ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa681450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a7fb0 con 0x55abfa607830 2024-03-26T17:47:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a78e0 con 0x55abfa60c9a0 2024-03-26T17:47:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a4b20 con 0x55abfa606e40 2024-03-26T17:47:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f86953a3740 10 client.0 _ll_register_callbacks cb 0x7f8684011780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:13.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.680+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa615ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa615ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57034/0 (socket says 172.21.15.105:57034) 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f868b7fe640 1 -- 172.21.15.105:0/1115731881 learned_addr learned my addr 172.21.15.105:0/1115731881 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f8690c79640 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa681450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f8690c79640 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 0x55abfa615590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f868b7fe640 1 -- 172.21.15.105:0/1115731881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 msgr2=0x55abfa615590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f868b7fe640 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 0x55abfa615590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f868b7fe640 1 -- 172.21.15.105:0/1115731881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 msgr2=0x55abfa681450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f868b7fe640 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa681450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f868b7fe640 1 -- 172.21.15.105:0/1115731881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfa681990 con 0x55abfa607830 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f8690c79640 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa681450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f868b7fe640 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa615ad0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8680000c00 tx=0x7f8680020410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bec0a7cdb9a6405 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f866b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f866b7fe640 1 -- 172.21.15.105:0/1115731881 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680024040 con 0x55abfa607830 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f866b7fe640 1 -- 172.21.15.105:0/1115731881 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8680004b50 con 0x55abfa607830 2024-03-26T17:47:13.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f86953a3740 1 -- 172.21.15.105:0/1115731881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86800047a0 con 0x55abfa607830 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f866b7fe640 1 -- 172.21.15.105:0/1115731881 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8680039050 con 0x55abfa607830 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f86953a3740 10 client.18609 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.681+0000 7f86953a3740 1 -- 172.21.15.105:0/1115731881 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abfa682b10 con 0x55abfa607830 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f866b7fe640 1 -- 172.21.15.105:0/1115731881 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86800047a0 con 0x55abfa607830 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f866b7fe640 1 -- 172.21.15.105:0/1115731881 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f86800107d0 con 0x55abfa607830 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f86953a3740 10 client.18609 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f86953a3740 10 client.18609 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f86953a3740 20 client.18609 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f86953a3740 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55abfa683440 0x55abfa685860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f86953a3740 4 client.18609 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f86953a3740 1 -- 172.21.15.105:0/1115731881 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abfa56f200 con 0x55abfa683440 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f8690c79640 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55abfa683440 0x55abfa685860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f866b7fe640 10 client.18609 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:47:13.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.682+0000 7f8690c79640 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55abfa683440 0x55abfa685860 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=30d3ea010e00dcac server_cookie=e988facf99f4702a in_seq=0 out_seq=0 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f866b7fe640 1 -- 172.21.15.105:0/1115731881 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abfa56f200 con 0x55abfa683440 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f866b7fe640 10 client.18609 handle_command_reply: tid=0 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 client.18609 shutdown 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 -- 172.21.15.105:0/1115731881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 msgr2=0x55abfa615ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa615ad0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8680000c00 tx=0x7f8680020410 comp rx=0 tx=0).stop 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 -- 172.21.15.105:0/1115731881 shutdown_connections 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55abfa683440 0x55abfa685860 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55abfa60c9a0 0x55abfa681450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55abfa607830 0x55abfa615ad0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 --2- 172.21.15.105:0/1115731881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55abfa606e40 0x55abfa615590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 -- 172.21.15.105:0/1115731881 >> 172.21.15.105:0/1115731881 conn(0x55abfa6023b0 msgr2=0x55abfa605610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f866b7fe640 0 client.18609 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 -- 172.21.15.105:0/1115731881 shutdown_connections 2024-03-26T17:47:13.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 1 -- 172.21.15.105:0/1115731881 wait complete. 2024-03-26T17:47:13.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.683+0000 7f86953a3740 20 client.18609 trim_cache size 0 max 16384 2024-03-26T17:47:13.682 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 -- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86740a2fa0 msgr2=0x7f86740a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86740a2fa0 0x7f86740a5460 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f867c0ef250 tx=0x7f867c0f6040 comp rx=0 tx=0).stop 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 -- 172.21.15.105:0/2877406067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f868c102340 msgr2=0x7f868c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f868c102340 0x7f868c197b80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f867c000c00 tx=0x7f867c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 -- 172.21.15.105:0/2877406067 shutdown_connections 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86740a2fa0 0x7f86740a5460 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f868c1075a0 0x7f868c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f868c104c70 0x7f868c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 --2- 172.21.15.105:0/2877406067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f868c102340 0x7f868c197b80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 -- 172.21.15.105:0/2877406067 >> 172.21.15.105:0/2877406067 conn(0x7f868c0fbd50 msgr2=0x7f868c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 -- 172.21.15.105:0/2877406067 shutdown_connections 2024-03-26T17:47:13.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.684+0000 7f86926dc640 1 -- 172.21.15.105:0/2877406067 wait complete. 2024-03-26T17:47:13.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 -- 172.21.15.105:0/1393958963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd0102ee0 msgr2=0x7efcd0103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 --2- 172.21.15.105:0/1393958963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd0102ee0 0x7efcd0103300 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009900 tx=0x7efcc002bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 -- 172.21.15.105:0/1393958963 shutdown_connections 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 --2- 172.21.15.105:0/1393958963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd0100310 0x7efcd0109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 --2- 172.21.15.105:0/1393958963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd00ff950 0x7efcd00ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 --2- 172.21.15.105:0/1393958963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd0102ee0 0x7efcd0103300 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 -- 172.21.15.105:0/1393958963 >> 172.21.15.105:0/1393958963 conn(0x7efcd00fb4f0 msgr2=0x7efcd00fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 -- 172.21.15.105:0/1393958963 shutdown_connections 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.810+0000 7efcd67df640 1 -- 172.21.15.105:0/1393958963 wait complete. 2024-03-26T17:47:13.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd67df640 1 Processor -- start 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd67df640 1 -- start start 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd67df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd00ff950 0x7efcd0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd67df640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd0100310 0x7efcd0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd67df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd0102ee0 0x7efcd019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd67df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010d410 con 0x7efcd00ff950 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd67df640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010d290 con 0x7efcd0100310 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd67df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010d590 con 0x7efcd0102ee0 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efccffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd00ff950 0x7efcd0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efccffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd00ff950 0x7efcd0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57050/0 (socket says 172.21.15.105:57050) 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efccffff640 1 -- 172.21.15.105:0/1735631043 learned_addr learned my addr 172.21.15.105:0/1735631043 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efccf7fe640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd0100310 0x7efcd0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.811+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd0102ee0 0x7efcd019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccffff640 1 -- 172.21.15.105:0/1735631043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd0102ee0 msgr2=0x7efcd019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccffff640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd0102ee0 0x7efcd019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccffff640 1 -- 172.21.15.105:0/1735631043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd0100310 msgr2=0x7efcd0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccffff640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd0100310 0x7efcd0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccffff640 1 -- 172.21.15.105:0/1735631043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd019f760 con 0x7efcd00ff950 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccffff640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd00ff950 0x7efcd0197b40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efcc0000c00 tx=0x7efcc0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccd7fa640 1 -- 172.21.15.105:0/1735631043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0043020 con 0x7efcd00ff950 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccd7fa640 1 -- 172.21.15.105:0/1735631043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcc0004350 con 0x7efcd00ff950 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efccd7fa640 1 -- 172.21.15.105:0/1735631043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0003b10 con 0x7efcd00ff950 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efcd67df640 1 -- 172.21.15.105:0/1735631043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd019f9f0 con 0x7efcd00ff950 2024-03-26T17:47:13.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.812+0000 7efcd67df640 1 -- 172.21.15.105:0/1735631043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd019ff40 con 0x7efcd00ff950 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd93c7740 1 Processor -- start 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd93c7740 1 -- start start 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9aa30 0x55c9b6e9ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9b420 0x55c9b6e9ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efccd7fa640 1 -- 172.21.15.105:0/1735631043 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7efcc0046020 con 0x7efcd00ff950 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd93c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c9b6dff970 con 0x55c9b6e9b420 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd93c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9b6da7d70 con 0x55c9b6ea0590 2024-03-26T17:47:13.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd93c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c9b6dfb920 con 0x55c9b6e9aa30 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd4d7c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9aa30 0x55c9b6e9ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efccffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9b420 0x55c9b6e9ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44836/0 (socket says 172.21.15.105:44836) 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efccf7fe640 1 -- 172.21.15.105:0/1811435692 learned_addr learned my addr 172.21.15.105:0/1811435692 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd4d7c640 1 -- 172.21.15.105:0/1811435692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 msgr2=0x55c9b6ea2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/1811435692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd4d7c640 1 -- 172.21.15.105:0/1811435692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9b420 msgr2=0x55c9b6e9ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/1811435692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9b420 0x55c9b6e9ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.815+0000 7efcd4d7c640 1 -- 172.21.15.105:0/1811435692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b6b133f0 con 0x55c9b6e9aa30 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.816+0000 7efccd7fa640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efcb80a2f20 0x7efcb80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.816+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/1811435692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9aa30 0x55c9b6e9ae50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009f10 tx=0x7efcc8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e850db8b07bb579d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.816+0000 7efccffff640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efcb80a2f20 0x7efcb80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.816+0000 7efcb67fc640 1 -- 172.21.15.105:0/1811435692 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8004d20 con 0x55c9b6e9aa30 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.816+0000 7efccd7fa640 1 -- 172.21.15.105:0/1735631043 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7efcc00f1530 con 0x7efcd00ff950 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.816+0000 7efcb67fc640 1 -- 172.21.15.105:0/1811435692 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcc8004030 con 0x55c9b6e9aa30 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.816+0000 7efcb67fc640 1 -- 172.21.15.105:0/1811435692 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8005260 con 0x55c9b6e9aa30 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 -- 172.21.15.105:0/1811435692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9aa30 msgr2=0x55c9b6e9ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efccffff640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efcb80a2f20 0x7efcb80a53e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efcc00ef360 tx=0x7efcc00f4f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 --2- 172.21.15.105:0/1811435692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9aa30 0x55c9b6e9ae50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009f10 tx=0x7efcc8011810 comp rx=0 tx=0).stop 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 -- 172.21.15.105:0/1811435692 shutdown_connections 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 --2- 172.21.15.105:0/1811435692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 --2- 172.21.15.105:0/1811435692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9b420 0x55c9b6e9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 --2- 172.21.15.105:0/1811435692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9aa30 0x55c9b6e9ae50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 -- 172.21.15.105:0/1811435692 >> 172.21.15.105:0/1811435692 conn(0x55c9b6e95fa0 msgr2=0x55c9b6e963a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 -- 172.21.15.105:0/1811435692 shutdown_connections 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 -- 172.21.15.105:0/1811435692 wait complete. 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 -- start start 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.817+0000 7efcd93c7740 1 Processor -- start 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57076/0 (socket says 172.21.15.105:57076) 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 -- 172.21.15.105:0/3054638746 learned_addr learned my addr 172.21.15.105:0/3054638746 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd93c7740 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6eac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c9b6ea3540 con 0x55c9b6e9aa30 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9b6dbb310 con 0x55c9b6ea0590 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c9b6ea36c0 con 0x55c9b6e9b420 2024-03-26T17:47:13.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd93c7740 10 client.0 _ll_register_callbacks cb 0x55c9b6df6c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efccf7fe640 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6eac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 -- 172.21.15.105:0/3054638746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 msgr2=0x55c9b6ea54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 -- 172.21.15.105:0/3054638746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 msgr2=0x55c9b6eac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6eac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.818+0000 7efcd4d7c640 1 -- 172.21.15.105:0/3054638746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b6bdd7d0 con 0x55c9b6e9aa30 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.819+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4fa0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7efcc8004e50 tx=0x7efcc8009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ea742dcd3a9183 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.819+0000 7efcaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcaf7fe640 1 -- 172.21.15.105:0/3054638746 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc802d060 con 0x55c9b6e9aa30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcaf7fe640 1 -- 172.21.15.105:0/3054638746 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7efcc8005040 con 0x55c9b6e9aa30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc8004030 con 0x55c9b6e9aa30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcaf7fe640 1 -- 172.21.15.105:0/3054638746 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcc801e040 con 0x55c9b6e9aa30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 10 client.18621 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9b6eaf890 con 0x55c9b6e9aa30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcaf7fe640 1 -- 172.21.15.105:0/3054638746 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcc8004030 con 0x55c9b6e9aa30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcaf7fe640 1 -- 172.21.15.105:0/3054638746 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efcc8018070 con 0x55c9b6e9aa30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 10 client.18621 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 10 client.18621 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:47:13.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 20 client.18621 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c9b6eb0fe0 0x55c9b6eb33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 4 client.18621 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.822+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9b6e0ceb0 con 0x55c9b6eb0fe0 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.823+0000 7efccf7fe640 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c9b6eb0fe0 0x55c9b6eb33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.823+0000 7efcaf7fe640 10 client.18621 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.823+0000 7efccf7fe640 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c9b6eb0fe0 0x55c9b6eb33d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=41f0890625a6f7f8 server_cookie=cbfc17e4771c3747 in_seq=0 out_seq=0 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcaf7fe640 1 -- 172.21.15.105:0/3054638746 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c9b6e0ceb0 con 0x55c9b6eb0fe0 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcaf7fe640 10 client.18621 handle_command_reply: tid=0 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 client.18621 shutdown 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 msgr2=0x55c9b6ea4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4fa0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7efcc8004e50 tx=0x7efcc8009cb0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 shutdown_connections 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c9b6eb0fe0 0x55c9b6eb33d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6eac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 --2- 172.21.15.105:0/3054638746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4fa0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcaf7fe640 0 client.18621 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:47:13.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 >> 172.21.15.105:0/3054638746 conn(0x55c9b6e95fa0 msgr2=0x55c9b6e951c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 shutdown_connections 2024-03-26T17:47:13.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 1 -- 172.21.15.105:0/3054638746 wait complete. 2024-03-26T17:47:13.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.824+0000 7efcd93c7740 20 client.18621 trim_cache size 0 max 16384 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd93c7740 1 Processor -- start 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd93c7740 1 -- start start 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6eac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd93c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a8db0 con 0x55c9b6e9aa30 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd93c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01aa8f0 con 0x55c9b6ea0590 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd93c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a9480 con 0x55c9b6e9b420 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd4d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd4d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57092/0 (socket says 172.21.15.105:57092) 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd4d7c640 1 -- 172.21.15.105:0/1179191266 learned_addr learned my addr 172.21.15.105:0/1179191266 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efccf7fe640 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6eac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.839+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcd4d7c640 1 -- 172.21.15.105:0/1179191266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 msgr2=0x55c9b6eac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6eac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcd4d7c640 1 -- 172.21.15.105:0/1179191266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 msgr2=0x55c9b6ea50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcd4d7c640 1 -- 172.21.15.105:0/1179191266 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b6e9fc50 con 0x55c9b6e9aa30 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4bb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7efcc8005d70 tx=0x7efcc8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e348d383f763379 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcb5ffb640 1 -- 172.21.15.105:0/1179191266 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8037040 con 0x55c9b6e9aa30 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcb5ffb640 1 -- 172.21.15.105:0/1179191266 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcc8038040 con 0x55c9b6e9aa30 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.840+0000 7efcb5ffb640 1 -- 172.21.15.105:0/1179191266 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc802d040 con 0x55c9b6e9aa30 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 -- 172.21.15.105:0/1179191266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 msgr2=0x55c9b6ea4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4bb0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7efcc8005d70 tx=0x7efcc8011810 comp rx=0 tx=0).stop 2024-03-26T17:47:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 -- 172.21.15.105:0/1179191266 shutdown_connections 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6ea0590 0x55c9b6ea50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6eac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 --2- 172.21.15.105:0/1179191266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea4bb0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 -- 172.21.15.105:0/1179191266 >> 172.21.15.105:0/1179191266 conn(0x55c9b6e95fa0 msgr2=0x55c9b6ea3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 -- 172.21.15.105:0/1179191266 shutdown_connections 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 -- 172.21.15.105:0/1179191266 wait complete. 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 -- start start 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 Processor -- start 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.841+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd93c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6ea0590 0x55c9b6eaaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd93c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01aa220 con 0x55c9b6ea0590 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd93c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a9b50 con 0x55c9b6e9aa30 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd93c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a6d90 con 0x55c9b6e9b420 2024-03-26T17:47:13.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd93c7740 10 client.0 _ll_register_callbacks cb 0x7efcc00f4ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd4d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6ea0590 0x55c9b6eaaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd4d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6ea0590 0x55c9b6eaaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57094/0 (socket says 172.21.15.105:57094) 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54528/0 (socket says 172.21.15.105:54528) 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd4d7c640 1 -- 172.21.15.105:0/2022644405 learned_addr learned my addr 172.21.15.105:0/2022644405 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efccf7fe640 1 -- 172.21.15.105:0/2022644405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6e9aa30 msgr2=0x55c9b6ea9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efccf7fe640 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efccf7fe640 1 -- 172.21.15.105:0/2022644405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6ea0590 msgr2=0x55c9b6eaaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efccf7fe640 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6ea0590 0x55c9b6eaaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efccf7fe640 1 -- 172.21.15.105:0/2022644405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b6eab080 con 0x55c9b6e9b420 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6ea0590 0x55c9b6eaaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efccf7fe640 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea9a80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55c9b6dbb460 tx=0x7efcc40108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7f3823c28209d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcaf7fe640 1 -- 172.21.15.105:0/2022644405 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4020510 con 0x55c9b6e9b420 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcaf7fe640 1 -- 172.21.15.105:0/2022644405 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7efcc40206b0 con 0x55c9b6e9b420 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.842+0000 7efcd93c7740 1 -- 172.21.15.105:0/2022644405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc8038040 con 0x55c9b6e9b420 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcaf7fe640 1 -- 172.21.15.105:0/2022644405 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcc402f050 con 0x55c9b6e9b420 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd93c7740 10 client.28465 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd93c7740 1 -- 172.21.15.105:0/2022644405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9b6eac450 con 0x55c9b6e9b420 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcaf7fe640 1 -- 172.21.15.105:0/2022644405 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efcc8038040 con 0x55c9b6e9b420 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcaf7fe640 1 -- 172.21.15.105:0/2022644405 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efcc4004c20 con 0x55c9b6e9b420 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd93c7740 10 client.28465 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd93c7740 10 client.28465 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd93c7740 20 client.28465 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd93c7740 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c9b6f17440 0x55c9b6f19860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd93c7740 4 client.28465 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:13.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd93c7740 1 -- 172.21.15.105:0/2022644405 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9b6e07200 con 0x55c9b6f17440 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.843+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c9b6f17440 0x55c9b6f19860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcaf7fe640 10 client.28465 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd4d7c640 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c9b6f17440 0x55c9b6f19860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22d9293532dcaf5d server_cookie=4acd6eba3f0c8317 in_seq=0 out_seq=0 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcaf7fe640 1 -- 172.21.15.105:0/2022644405 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9b6e07200 con 0x55c9b6f17440 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcaf7fe640 10 client.28465 handle_command_reply: tid=0 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 client.28465 shutdown 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 -- 172.21.15.105:0/2022644405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 msgr2=0x55c9b6ea9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea9a80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55c9b6dbb460 tx=0x7efcc40108c0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 -- 172.21.15.105:0/2022644405 shutdown_connections 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c9b6f17440 0x55c9b6f19860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c9b6ea0590 0x55c9b6eaaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcaf7fe640 0 client.28465 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:47:13.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9b6e9b420 0x55c9b6ea9a80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 --2- 172.21.15.105:0/2022644405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9b6e9aa30 0x55c9b6ea9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.844+0000 7efcd93c7740 1 -- 172.21.15.105:0/2022644405 >> 172.21.15.105:0/2022644405 conn(0x55c9b6e95fa0 msgr2=0x55c9b6ea4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.845+0000 7efcd93c7740 1 -- 172.21.15.105:0/2022644405 shutdown_connections 2024-03-26T17:47:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.845+0000 7efcd93c7740 1 -- 172.21.15.105:0/2022644405 wait complete. 2024-03-26T17:47:13.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.845+0000 7efcd93c7740 20 client.28465 trim_cache size 0 max 16384 2024-03-26T17:47:13.843 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.845+0000 7efcd67df640 1 -- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efcb80a2f20 msgr2=0x7efcb80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.845+0000 7efcd67df640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efcb80a2f20 0x7efcb80a53e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efcc00ef360 tx=0x7efcc00f4f90 comp rx=0 tx=0).stop 2024-03-26T17:47:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.845+0000 7efcd67df640 1 -- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd00ff950 msgr2=0x7efcd0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.845+0000 7efcd67df640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd00ff950 0x7efcd0197b40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7efcc0000c00 tx=0x7efcc0005dc0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.846+0000 7efcd67df640 1 -- 172.21.15.105:0/1735631043 shutdown_connections 2024-03-26T17:47:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.846+0000 7efcd67df640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efcb80a2f20 0x7efcb80a53e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.846+0000 7efcd67df640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcd0102ee0 0x7efcd019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.846+0000 7efcd67df640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd0100310 0x7efcd0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.846+0000 7efcd67df640 1 --2- 172.21.15.105:0/1735631043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efcd00ff950 0x7efcd0197b40 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.846+0000 7efcd67df640 1 -- 172.21.15.105:0/1735631043 >> 172.21.15.105:0/1735631043 conn(0x7efcd00fb4f0 msgr2=0x7efcd0106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.846+0000 7efcd67df640 1 -- 172.21.15.105:0/1735631043 shutdown_connections 2024-03-26T17:47:13.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.846+0000 7efcd67df640 1 -- 172.21.15.105:0/1735631043 wait complete. 2024-03-26T17:47:13.849 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:47:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.973+0000 7f3270a0b640 1 -- 172.21.15.105:0/1787061365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c102370 msgr2=0x7f326c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.973+0000 7f3270a0b640 1 --2- 172.21.15.105:0/1787061365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c102370 0x7f326c108c00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f3264009ca0 tx=0x7f326402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.973+0000 7f3270a0b640 1 -- 172.21.15.105:0/1787061365 shutdown_connections 2024-03-26T17:47:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.973+0000 7f3270a0b640 1 --2- 172.21.15.105:0/1787061365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c102370 0x7f326c108c00 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.973+0000 7f3270a0b640 1 --2- 172.21.15.105:0/1787061365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f326c1019b0 0x7f326c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.973+0000 7f3270a0b640 1 --2- 172.21.15.105:0/1787061365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f326c1007c0 0x7f326c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.973+0000 7f3270a0b640 1 -- 172.21.15.105:0/1787061365 >> 172.21.15.105:0/1787061365 conn(0x7f326c0fbd50 msgr2=0x7f326c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 -- 172.21.15.105:0/1787061365 shutdown_connections 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 -- 172.21.15.105:0/1787061365 wait complete. 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 Processor -- start 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 -- start start 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f326c1007c0 0x7f326c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c1019b0 0x7f326c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f326c102370 0x7f326c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f326c10c2d0 con 0x7f326c1019b0 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f326c10c150 con 0x7f326c102370 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.974+0000 7f3270a0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f326c10c450 con 0x7f326c1007c0 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f3263fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c1019b0 0x7f326c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f326a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f326c1007c0 0x7f326c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f326ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f326c102370 0x7f326c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f3263fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c1019b0 0x7f326c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57108/0 (socket says 172.21.15.105:57108) 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f3263fff640 1 -- 172.21.15.105:0/4120523945 learned_addr learned my addr 172.21.15.105:0/4120523945 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f326ad9d640 1 -- 172.21.15.105:0/4120523945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f326c1007c0 msgr2=0x7f326c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f326ad9d640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f326c1007c0 0x7f326c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f326ad9d640 1 -- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c1019b0 msgr2=0x7f326c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f326ad9d640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c1019b0 0x7f326c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f326ad9d640 1 -- 172.21.15.105:0/4120523945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326c19f740 con 0x7f326c102370 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f326ad9d640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f326c102370 0x7f326c19f0d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f326402c630 tx=0x7f32640328a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f32637fe640 1 -- 172.21.15.105:0/4120523945 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264041020 con 0x7f326c102370 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f3270a0b640 1 -- 172.21.15.105:0/4120523945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326c19f9d0 con 0x7f326c102370 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f32637fe640 1 -- 172.21.15.105:0/4120523945 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3264003e20 con 0x7f326c102370 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f32637fe640 1 -- 172.21.15.105:0/4120523945 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264039040 con 0x7f326c102370 2024-03-26T17:47:13.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.975+0000 7f3270a0b640 1 -- 172.21.15.105:0/4120523945 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f326c19ff10 con 0x7f326c102370 2024-03-26T17:47:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32637fe640 1 -- 172.21.15.105:0/4120523945 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3264044020 con 0x7f326c102370 2024-03-26T17:47:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32735fb740 1 Processor -- start 2024-03-26T17:47:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32735fb740 1 -- start start 2024-03-26T17:47:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32735fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8102e40 0x5633c8103260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32735fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c8108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32735fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 0x5633c810ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32735fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5633c8063970 con 0x5633c8102e40 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32735fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5633c800bd70 con 0x5633c81089a0 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32735fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5633c805f920 con 0x5633c8103830 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.978+0000 7f32637fe640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32580a2f20 0x7f32580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c8108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f3263fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8102e40 0x5633c8103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c8108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54554/0 (socket says 172.21.15.105:54554) 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 -- 172.21.15.105:0/48561908 learned_addr learned my addr 172.21.15.105:0/48561908 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f3263fff640 1 --2- 172.21.15.105:0/48561908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 0x5633c810ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326ad9d640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32580a2f20 0x7f32580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f32637fe640 1 -- 172.21.15.105:0/4120523945 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f32640f15a0 con 0x7f326c102370 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 -- 172.21.15.105:0/48561908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 msgr2=0x5633c810ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 --2- 172.21.15.105:0/48561908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 0x5633c810ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 -- 172.21.15.105:0/48561908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8102e40 msgr2=0x5633c8103260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 --2- 172.21.15.105:0/48561908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8102e40 0x5633c8103260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 -- 172.21.15.105:0/48561908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c7d773f0 con 0x5633c8103830 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326a59c640 1 --2- 172.21.15.105:0/48561908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c8108330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f325c009c80 tx=0x7f325c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667e63414d54a52a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f326ad9d640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32580a2f20 0x7f32580a53e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3264032420 tx=0x7f326402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f3260ff9640 1 -- 172.21.15.105:0/48561908 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c01e070 con 0x5633c8103830 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f3260ff9640 1 -- 172.21.15.105:0/48561908 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f325c009e30 con 0x5633c8103830 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.979+0000 7f3260ff9640 1 -- 172.21.15.105:0/48561908 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c00ea10 con 0x5633c8103830 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 -- 172.21.15.105:0/48561908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 msgr2=0x5633c8108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 --2- 172.21.15.105:0/48561908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c8108330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f325c009c80 tx=0x7f325c011810 comp rx=0 tx=0).stop 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 -- 172.21.15.105:0/48561908 shutdown_connections 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 --2- 172.21.15.105:0/48561908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 0x5633c810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 --2- 172.21.15.105:0/48561908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c8108330 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 --2- 172.21.15.105:0/48561908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8102e40 0x5633c8103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 -- 172.21.15.105:0/48561908 >> 172.21.15.105:0/48561908 conn(0x5633c80f9fa0 msgr2=0x5633c80fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 -- 172.21.15.105:0/48561908 shutdown_connections 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.980+0000 7f32735fb740 1 -- 172.21.15.105:0/48561908 wait complete. 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 1 -- start start 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 1 Processor -- start 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c810d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8103830 0x5633c810d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 0x5633c8114930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5633c810b950 con 0x5633c8103830 2024-03-26T17:47:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5633c801f310 con 0x5633c81089a0 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5633c810bad0 con 0x5633c8102e40 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f32735fb740 10 client.0 _ll_register_callbacks cb 0x5633c805ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c810d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f326a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8103830 0x5633c810d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c810d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54560/0 (socket says 172.21.15.105:54560) 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 -- 172.21.15.105:0/3403100889 learned_addr learned my addr 172.21.15.105:0/3403100889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f326a59c640 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8103830 0x5633c810d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57134/0 (socket says 172.21.15.105:57134) 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 0x5633c8114930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 -- 172.21.15.105:0/3403100889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 msgr2=0x5633c8114930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 0x5633c8114930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 -- 172.21.15.105:0/3403100889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8103830 msgr2=0x5633c810d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8103830 0x5633c810d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 -- 172.21.15.105:0/3403100889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c7e417d0 con 0x5633c8102e40 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f326a59c640 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8103830 0x5633c810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3263fff640 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c810d3b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3254010f70 tx=0x7f3254015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e1acd60ead9682a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.981+0000 7f3251ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f3251ffb640 1 -- 172.21.15.105:0/3403100889 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254014020 con 0x5633c8102e40 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f3251ffb640 1 -- 172.21.15.105:0/3403100889 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f3254022420 con 0x5633c8102e40 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 1 -- 172.21.15.105:0/3403100889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f325c009e30 con 0x5633c8102e40 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f3251ffb640 1 -- 172.21.15.105:0/3403100889 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3254022620 con 0x5633c8102e40 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 10 client.28477 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 1 -- 172.21.15.105:0/3403100889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5633c8117d10 con 0x5633c8102e40 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f3251ffb640 1 -- 172.21.15.105:0/3403100889 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f325c009e30 con 0x5633c8102e40 2024-03-26T17:47:13.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f3251ffb640 1 -- 172.21.15.105:0/3403100889 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3254024880 con 0x5633c8102e40 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 10 client.28477 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 10 client.28477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 20 client.28477 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5633c8119330 0x5633c811b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 4 client.28477 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f32735fb740 1 -- 172.21.15.105:0/3403100889 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633c8070eb0 con 0x5633c8119330 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.982+0000 7f326a59c640 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5633c8119330 0x5633c811b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.983+0000 7f3251ffb640 10 client.28477 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.983+0000 7f326a59c640 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5633c8119330 0x5633c811b7a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f4b2a8d428179a0 server_cookie=400a736cdc172b26 in_seq=0 out_seq=0 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.983+0000 7f3251ffb640 1 -- 172.21.15.105:0/3403100889 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5633c8070eb0 con 0x5633c8119330 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.983+0000 7f3251ffb640 10 client.28477 handle_command_reply: tid=0 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.983+0000 7f32735fb740 1 client.28477 shutdown 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 -- 172.21.15.105:0/3403100889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 msgr2=0x5633c810d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c810d3b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3254010f70 tx=0x7f3254015040 comp rx=0 tx=0).stop 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 -- 172.21.15.105:0/3403100889 shutdown_connections 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5633c8119330 0x5633c811b7a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c81089a0 0x5633c8114930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c8103830 0x5633c810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f3251ffb640 0 client.28477 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 --2- 172.21.15.105:0/3403100889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c810d3b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 -- 172.21.15.105:0/3403100889 >> 172.21.15.105:0/3403100889 conn(0x5633c80f9fa0 msgr2=0x5633c80f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 -- 172.21.15.105:0/3403100889 shutdown_connections 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 1 -- 172.21.15.105:0/3403100889 wait complete. 2024-03-26T17:47:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.984+0000 7f32735fb740 20 client.28477 trim_cache size 0 max 16384 2024-03-26T17:47:13.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f32735fb740 1 Processor -- start 2024-03-26T17:47:13.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f32735fb740 1 -- start start 2024-03-26T17:47:13.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f32735fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8102e40 0x5633c810d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f32735fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c81148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f32735fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c80f8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f32735fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1a6c30 con 0x5633c81089a0 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f32735fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1a8770 con 0x5633c8102e40 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f32735fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1a7300 con 0x5633c8103830 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c81148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f3263fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8102e40 0x5633c810d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c81148f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54562/0 (socket says 172.21.15.105:54562) 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f3263fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8102e40 0x5633c810d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44886/0 (socket says 172.21.15.105:44886) 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 -- 172.21.15.105:0/3591940680 learned_addr learned my addr 172.21.15.105:0/3591940680 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f3263fff640 1 --2- 172.21.15.105:0/3591940680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c80f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 -- 172.21.15.105:0/3591940680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8102e40 msgr2=0x5633c810d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 --2- 172.21.15.105:0/3591940680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8102e40 0x5633c810d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 -- 172.21.15.105:0/3591940680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 msgr2=0x5633c80f8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 --2- 172.21.15.105:0/3591940680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c80f8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 -- 172.21.15.105:0/3591940680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633c80f45d0 con 0x5633c8103830 2024-03-26T17:47:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:13.999+0000 7f326a59c640 1 --2- 172.21.15.105:0/3591940680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c81148f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f325c00e420 tx=0x7f325c009fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18296d572eedbc3b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f3253fff640 1 -- 172.21.15.105:0/3591940680 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c030070 con 0x5633c8103830 2024-03-26T17:47:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f3253fff640 1 -- 172.21.15.105:0/3591940680 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f325c02a070 con 0x5633c8103830 2024-03-26T17:47:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f3253fff640 1 -- 172.21.15.105:0/3591940680 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c01e040 con 0x5633c8103830 2024-03-26T17:47:13.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f32735fb740 1 -- 172.21.15.105:0/3591940680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 msgr2=0x5633c81148f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:13.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f32735fb740 1 --2- 172.21.15.105:0/3591940680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c81148f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f325c00e420 tx=0x7f325c009fc0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f32735fb740 1 -- 172.21.15.105:0/3591940680 shutdown_connections 2024-03-26T17:47:13.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f32735fb740 1 --2- 172.21.15.105:0/3591940680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c80f8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f32735fb740 1 --2- 172.21.15.105:0/3591940680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8103830 0x5633c81148f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f32735fb740 1 --2- 172.21.15.105:0/3591940680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8102e40 0x5633c810d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.000+0000 7f32735fb740 1 -- 172.21.15.105:0/3591940680 >> 172.21.15.105:0/3591940680 conn(0x5633c80f9fa0 msgr2=0x5633c810cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 -- 172.21.15.105:0/3591940680 shutdown_connections 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 -- 172.21.15.105:0/3591940680 wait complete. 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 -- start start 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 Processor -- start 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c8112150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8103830 0x5633c8112690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c81136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1a80a0 con 0x5633c81089a0 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1a79d0 con 0x5633c8103830 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f326c1a4c10 con 0x5633c8102e40 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f32735fb740 10 client.0 _ll_register_callbacks cb 0x7f3254010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f3263fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c81136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f3263fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c81136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57144/0 (socket says 172.21.15.105:57144) 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f3263fff640 1 -- 172.21.15.105:0/1657866334 learned_addr learned my addr 172.21.15.105:0/1657866334 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f326a59c640 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8103830 0x5633c8112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.001+0000 7f3263fff640 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c8112150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3263fff640 1 -- 172.21.15.105:0/1657866334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 msgr2=0x5633c8112150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3263fff640 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c8112150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3263fff640 1 -- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8103830 msgr2=0x5633c8112690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3263fff640 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8103830 0x5633c8112690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3263fff640 1 -- 172.21.15.105:0/1657866334 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633c8113c90 con 0x5633c81089a0 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f326a59c640 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8103830 0x5633c8112690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:13.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3263fff640 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c81136a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3254009fd0 tx=0x7f3254024f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0405cf48265f10 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3251ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3251ffb640 1 -- 172.21.15.105:0/1657866334 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3254031070 con 0x5633c81089a0 2024-03-26T17:47:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3251ffb640 1 -- 172.21.15.105:0/1657866334 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f3254005260 con 0x5633c81089a0 2024-03-26T17:47:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f32735fb740 1 -- 172.21.15.105:0/1657866334 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f325c02a070 con 0x5633c81089a0 2024-03-26T17:47:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3251ffb640 1 -- 172.21.15.105:0/1657866334 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3254014040 con 0x5633c81089a0 2024-03-26T17:47:14.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f32735fb740 10 client.18645 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f32735fb740 1 -- 172.21.15.105:0/1657866334 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5633c817f7b0 con 0x5633c81089a0 2024-03-26T17:47:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.002+0000 7f3251ffb640 1 -- 172.21.15.105:0/1657866334 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f325c02a070 con 0x5633c81089a0 2024-03-26T17:47:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f3251ffb640 1 -- 172.21.15.105:0/1657866334 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f32540227c0 con 0x5633c81089a0 2024-03-26T17:47:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 10 client.18645 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 10 client.18645 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:47:14.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 20 client.18645 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:14.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5633c8180040 0x5633c8182430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 4 client.18645 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:14.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 1 -- 172.21.15.105:0/1657866334 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633c806b200 con 0x5633c8180040 2024-03-26T17:47:14.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f326a59c640 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5633c8180040 0x5633c8182430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f326a59c640 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5633c8180040 0x5633c8182430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e11a6c4b9726b09 server_cookie=72cb52a5ba1293fb in_seq=0 out_seq=0 2024-03-26T17:47:14.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f3251ffb640 10 client.18645 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:47:14.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f3251ffb640 1 -- 172.21.15.105:0/1657866334 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633c806b200 con 0x5633c8180040 2024-03-26T17:47:14.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f3251ffb640 10 client.18645 handle_command_reply: tid=0 2024-03-26T17:47:14.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 1 client.18645 shutdown 2024-03-26T17:47:14.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 1 -- 172.21.15.105:0/1657866334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 msgr2=0x5633c81136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c81136a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3254009fd0 tx=0x7f3254024f10 comp rx=0 tx=0).stop 2024-03-26T17:47:14.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 1 -- 172.21.15.105:0/1657866334 shutdown_connections 2024-03-26T17:47:14.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.003+0000 7f32735fb740 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5633c81089a0 0x5633c81136a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f32735fb740 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5633c8180040 0x5633c8182430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f32735fb740 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5633c8103830 0x5633c8112690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f32735fb740 1 --2- 172.21.15.105:0/1657866334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5633c8102e40 0x5633c8112150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f32735fb740 1 -- 172.21.15.105:0/1657866334 >> 172.21.15.105:0/1657866334 conn(0x5633c80f9fa0 msgr2=0x5633c80fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3251ffb640 0 client.18645 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:47:14.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f32735fb740 1 -- 172.21.15.105:0/1657866334 shutdown_connections 2024-03-26T17:47:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f32735fb740 1 -- 172.21.15.105:0/1657866334 wait complete. 2024-03-26T17:47:14.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f32735fb740 20 client.18645 trim_cache size 0 max 16384 2024-03-26T17:47:14.005 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 -- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32580a2f20 msgr2=0x7f32580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32580a2f20 0x7f32580a53e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3264032420 tx=0x7f326402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 -- 172.21.15.105:0/4120523945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f326c102370 msgr2=0x7f326c19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f326c102370 0x7f326c19f0d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f326402c630 tx=0x7f32640328a0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 -- 172.21.15.105:0/4120523945 shutdown_connections 2024-03-26T17:47:14.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32580a2f20 0x7f32580a53e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f326c102370 0x7f326c19f0d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f326c1019b0 0x7f326c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 --2- 172.21.15.105:0/4120523945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f326c1007c0 0x7f326c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 -- 172.21.15.105:0/4120523945 >> 172.21.15.105:0/4120523945 conn(0x7f326c0fbd50 msgr2=0x7f326c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 -- 172.21.15.105:0/4120523945 shutdown_connections 2024-03-26T17:47:14.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.004+0000 7f3270a0b640 1 -- 172.21.15.105:0/4120523945 wait complete. 2024-03-26T17:47:14.010 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:47:14.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 280389 3.16 MB/sec execute 357 sec latency 15.549 ms 2024-03-26T17:47:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:13 smithi144 ceph-mon[36095]: pgmap v614: 97 pgs: 97 active+clean; 29 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:47:14.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:13 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1546798150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 -- 172.21.15.105:0/3903082519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1007c0 msgr2=0x7fcf5c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3903082519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1007c0 0x7fcf5c100be0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009900 tx=0x7fcf4c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 -- 172.21.15.105:0/3903082519 shutdown_connections 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3903082519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf5c102370 0x7fcf5c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3903082519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf5c1019b0 0x7fcf5c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3903082519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1007c0 0x7fcf5c100be0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 -- 172.21.15.105:0/3903082519 >> 172.21.15.105:0/3903082519 conn(0x7fcf5c0fbd50 msgr2=0x7fcf5c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 -- 172.21.15.105:0/3903082519 shutdown_connections 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.132+0000 7fcf6148a640 1 -- 172.21.15.105:0/3903082519 wait complete. 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf6148a640 1 Processor -- start 2024-03-26T17:47:14.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf6148a640 1 -- start start 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf6148a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf5c1007c0 0x7fcf5c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf6148a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1019b0 0x7fcf5c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf6148a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf5c102370 0x7fcf5c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf6148a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c10d3e0 con 0x7fcf5c1019b0 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf6148a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c10d260 con 0x7fcf5c102370 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf6148a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c10d560 con 0x7fcf5c1007c0 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1019b0 0x7fcf5c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf5affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf5c1007c0 0x7fcf5c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1019b0 0x7fcf5c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57172/0 (socket says 172.21.15.105:57172) 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf53fff640 1 -- 172.21.15.105:0/3535088020 learned_addr learned my addr 172.21.15.105:0/3535088020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf5b7fe640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf5c102370 0x7fcf5c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf5b7fe640 1 -- 172.21.15.105:0/3535088020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf5c1007c0 msgr2=0x7fcf5c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf5b7fe640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf5c1007c0 0x7fcf5c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf5b7fe640 1 -- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1019b0 msgr2=0x7fcf5c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf5b7fe640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1019b0 0x7fcf5c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf5b7fe640 1 -- 172.21.15.105:0/3535088020 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c19f7a0 con 0x7fcf5c102370 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf53fff640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1019b0 0x7fcf5c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.133+0000 7fcf5affd640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf5c1007c0 0x7fcf5c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.134+0000 7fcf5b7fe640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf5c102370 0x7fcf5c19f130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009fd0 tx=0x7fcf5400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.134+0000 7fcf58ff9640 1 -- 172.21.15.105:0/3535088020 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400ce20 con 0x7fcf5c102370 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.134+0000 7fcf6148a640 1 -- 172.21.15.105:0/3535088020 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5c19fa90 con 0x7fcf5c102370 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.134+0000 7fcf58ff9640 1 -- 172.21.15.105:0/3535088020 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf54005020 con 0x7fcf5c102370 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.134+0000 7fcf58ff9640 1 -- 172.21.15.105:0/3535088020 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54010690 con 0x7fcf5c102370 2024-03-26T17:47:14.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.134+0000 7fcf6148a640 1 -- 172.21.15.105:0/3535088020 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5c19ff90 con 0x7fcf5c102370 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.136+0000 7fcf58ff9640 1 -- 172.21.15.105:0/3535088020 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcf540051d0 con 0x7fcf5c102370 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.136+0000 7fcf64071740 1 Processor -- start 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.136+0000 7fcf64071740 1 -- start start 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf64071740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d09e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf64071740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d0ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf64071740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 0x560083d11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf64071740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560083c6e970 con 0x560083d09a30 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf64071740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560083c16d70 con 0x560083d0f590 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf64071740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560083c6a920 con 0x560083d0a420 2024-03-26T17:47:14.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d09e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d09e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57176/0 (socket says 172.21.15.105:57176) 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 -- 172.21.15.105:0/2858559646 learned_addr learned my addr 172.21.15.105:0/2858559646 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf5affd640 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d0ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 0x560083d11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf58ff9640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcf480a2f20 0x7fcf480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf5b7fe640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcf480a2f20 0x7fcf480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 -- 172.21.15.105:0/2858559646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 msgr2=0x560083d0ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d0ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 -- 172.21.15.105:0/2858559646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 msgr2=0x560083d11a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 0x560083d11a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 -- 172.21.15.105:0/2858559646 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600839823f0 con 0x560083d09a30 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf5affd640 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d0ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf58ff9640 1 -- 172.21.15.105:0/3535088020 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcf540cfae0 con 0x7fcf5c102370 2024-03-26T17:47:14.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.137+0000 7fcf53fff640 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d09e50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcf44013d20 tx=0x7fcf44009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364f4ac9c631b766 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf517fa640 1 -- 172.21.15.105:0/2858559646 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44013170 con 0x560083d09a30 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf517fa640 1 -- 172.21.15.105:0/2858559646 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf44013310 con 0x560083d09a30 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf5b7fe640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcf480a2f20 0x7fcf480a53e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcf5409c160 tx=0x7fcf5409f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf517fa640 1 -- 172.21.15.105:0/2858559646 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44010c50 con 0x560083d09a30 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf64071740 1 -- 172.21.15.105:0/2858559646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 msgr2=0x560083d09e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf64071740 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d09e50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcf44013d20 tx=0x7fcf44009c30 comp rx=0 tx=0).stop 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf64071740 1 -- 172.21.15.105:0/2858559646 shutdown_connections 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf64071740 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 0x560083d11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf64071740 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d0ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf64071740 1 --2- 172.21.15.105:0/2858559646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d09e50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.138+0000 7fcf64071740 1 -- 172.21.15.105:0/2858559646 >> 172.21.15.105:0/2858559646 conn(0x560083d04fa0 msgr2=0x560083d053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 -- 172.21.15.105:0/2858559646 shutdown_connections 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 -- 172.21.15.105:0/2858559646 wait complete. 2024-03-26T17:47:14.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 -- start start 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 Processor -- start 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d13f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 0x560083d1b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560083d12540 con 0x560083d09a30 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560083c2a310 con 0x560083d0f590 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560083d126c0 con 0x560083d0a420 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf64071740 10 client.0 _ll_register_callbacks cb 0x560083c65c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d13f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d13f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57180/0 (socket says 172.21.15.105:57180) 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf53fff640 1 -- 172.21.15.105:0/4285239831 learned_addr learned my addr 172.21.15.105:0/4285239831 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf5affd640 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.139+0000 7fcf53fff640 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 0x560083d1b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf53fff640 1 -- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 msgr2=0x560083d14450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf53fff640 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf53fff640 1 -- 172.21.15.105:0/4285239831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 msgr2=0x560083d1b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf53fff640 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 0x560083d1b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf53fff640 1 -- 172.21.15.105:0/4285239831 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560083a4c7d0 con 0x560083d09a30 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf5affd640 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf53fff640 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d13f10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcf44013b60 tx=0x7fcf44005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4498d7a40ab5b08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf427fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf427fc640 1 -- 172.21.15.105:0/4285239831 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44013170 con 0x560083d09a30 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf64071740 1 -- 172.21.15.105:0/4285239831 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf44013310 con 0x560083d09a30 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf427fc640 1 -- 172.21.15.105:0/4285239831 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcf44020410 con 0x560083d09a30 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf427fc640 1 -- 172.21.15.105:0/4285239831 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf44020610 con 0x560083d09a30 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf64071740 10 client.18657 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf64071740 1 -- 172.21.15.105:0/4285239831 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560083d1e800 con 0x560083d09a30 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.140+0000 7fcf427fc640 1 -- 172.21.15.105:0/4285239831 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf44013310 con 0x560083d09a30 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf427fc640 1 -- 172.21.15.105:0/4285239831 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcf44030830 con 0x560083d09a30 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf64071740 10 client.18657 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf64071740 10 client.18657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf64071740 20 client.18657 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf64071740 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560083d1fbf0 0x560083d21fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf64071740 4 client.18657 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf64071740 1 -- 172.21.15.105:0/4285239831 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560083c7beb0 con 0x560083d1fbf0 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf5affd640 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560083d1fbf0 0x560083d21fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf427fc640 10 client.18657 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.141+0000 7fcf5affd640 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560083d1fbf0 0x560083d21fe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba52616b83acf7e2 server_cookie=efb12d2c73589c5d in_seq=0 out_seq=0 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf427fc640 1 -- 172.21.15.105:0/4285239831 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560083c7beb0 con 0x560083d1fbf0 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf427fc640 10 client.18657 handle_command_reply: tid=0 2024-03-26T17:47:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 client.18657 shutdown 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 -- 172.21.15.105:0/4285239831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 msgr2=0x560083d13f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d13f10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcf44013b60 tx=0x7fcf44005fd0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 -- 172.21.15.105:0/4285239831 shutdown_connections 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560083d1fbf0 0x560083d21fe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0f590 0x560083d1b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf427fc640 0 client.18657 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 --2- 172.21.15.105:0/4285239831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d13f10 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 -- 172.21.15.105:0/4285239831 >> 172.21.15.105:0/4285239831 conn(0x560083d04fa0 msgr2=0x560083d041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 -- 172.21.15.105:0/4285239831 shutdown_connections 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 1 -- 172.21.15.105:0/4285239831 wait complete. 2024-03-26T17:47:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.142+0000 7fcf64071740 20 client.18657 trim_cache size 0 max 16384 2024-03-26T17:47:14.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:13 smithi105 ceph-mon[32280]: pgmap v614: 97 pgs: 97 active+clean; 29 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:47:14.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:13 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1546798150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf64071740 1 Processor -- start 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf64071740 1 -- start start 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf64071740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d04fa0 0x560083d13b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf64071740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf64071740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf64071740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1a8da0 con 0x560083d09a30 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf64071740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1aa8e0 con 0x560083d04fa0 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf64071740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1a9470 con 0x560083d0a420 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d1b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d04fa0 0x560083d13b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d1b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57196/0 (socket says 172.21.15.105:57196) 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d04fa0 0x560083d13b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44936/0 (socket says 172.21.15.105:44936) 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf5affd640 1 -- 172.21.15.105:0/1877085805 learned_addr learned my addr 172.21.15.105:0/1877085805 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.156+0000 7fcf53fff640 1 --2- 172.21.15.105:0/1877085805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf5affd640 1 -- 172.21.15.105:0/1877085805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 msgr2=0x560083d14060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf5affd640 1 --2- 172.21.15.105:0/1877085805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf5affd640 1 -- 172.21.15.105:0/1877085805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d04fa0 msgr2=0x560083d13b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf5affd640 1 --2- 172.21.15.105:0/1877085805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d04fa0 0x560083d13b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf5affd640 1 -- 172.21.15.105:0/1877085805 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560083d04dd0 con 0x560083d09a30 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf5affd640 1 --2- 172.21.15.105:0/1877085805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d1b420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009900 tx=0x7fcf4c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabe11b9fe06bea1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf50ff9640 1 -- 172.21.15.105:0/1877085805 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004490 con 0x560083d09a30 2024-03-26T17:47:14.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf50ff9640 1 -- 172.21.15.105:0/1877085805 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf4c04d070 con 0x560083d09a30 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.157+0000 7fcf50ff9640 1 -- 172.21.15.105:0/1877085805 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004e10 con 0x560083d09a30 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 -- 172.21.15.105:0/1877085805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 msgr2=0x560083d1b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 --2- 172.21.15.105:0/1877085805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d1b420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009900 tx=0x7fcf4c005850 comp rx=0 tx=0).stop 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 -- 172.21.15.105:0/1877085805 shutdown_connections 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 --2- 172.21.15.105:0/1877085805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d0a420 0x560083d14060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 --2- 172.21.15.105:0/1877085805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d09a30 0x560083d1b420 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 --2- 172.21.15.105:0/1877085805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d04fa0 0x560083d13b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 -- 172.21.15.105:0/1877085805 >> 172.21.15.105:0/1877085805 conn(0x560083d0b200 msgr2=0x560083d05380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 -- 172.21.15.105:0/1877085805 shutdown_connections 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 -- 172.21.15.105:0/1877085805 wait complete. 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 -- start start 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.158+0000 7fcf64071740 1 Processor -- start 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf64071740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d04fa0 0x560083d1b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf64071740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d09a30 0x560083d17da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf64071740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0a420 0x560083d182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf64071740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1aa210 con 0x560083d04fa0 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf64071740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1a9b40 con 0x560083d0a420 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf64071740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1a6d80 con 0x560083d09a30 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf64071740 10 client.0 _ll_register_callbacks cb 0x7fcf440137b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d04fa0 0x560083d1b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d04fa0 0x560083d1b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57210/0 (socket says 172.21.15.105:57210) 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 -- 172.21.15.105:0/336619987 learned_addr learned my addr 172.21.15.105:0/336619987 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0a420 0x560083d182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf5affd640 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d09a30 0x560083d17da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 -- 172.21.15.105:0/336619987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d09a30 msgr2=0x560083d17da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d09a30 0x560083d17da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 -- 172.21.15.105:0/336619987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0a420 msgr2=0x560083d182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0a420 0x560083d182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 -- 172.21.15.105:0/336619987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560083d83ed0 con 0x560083d04fa0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf53fff640 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d04fa0 0x560083d1b300 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcf44030c30 tx=0x7fcf44010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ba336744261c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.159+0000 7fcf427fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf427fc640 1 -- 172.21.15.105:0/336619987 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44009d00 con 0x560083d04fa0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf427fc640 1 -- 172.21.15.105:0/336619987 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fcf44010c40 con 0x560083d04fa0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 1 -- 172.21.15.105:0/336619987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf4c04d070 con 0x560083d04fa0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf427fc640 1 -- 172.21.15.105:0/336619987 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf44005500 con 0x560083d04fa0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 10 client.18663 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 1 -- 172.21.15.105:0/336619987 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560083d85150 con 0x560083d04fa0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf427fc640 1 -- 172.21.15.105:0/336619987 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf4c04d070 con 0x560083d04fa0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf427fc640 1 -- 172.21.15.105:0/336619987 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fcf44020ae0 con 0x560083d04fa0 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 10 client.18663 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 10 client.18663 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:47:14.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 20 client.18663 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560083d0f590 0x560083d87ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 4 client.18663 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf64071740 1 -- 172.21.15.105:0/336619987 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560083c76200 con 0x560083d0f590 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.160+0000 7fcf5affd640 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560083d0f590 0x560083d87ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf427fc640 10 client.18663 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf5affd640 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560083d0f590 0x560083d87ac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93dacf523559888e server_cookie=7b67bc40415332f9 in_seq=0 out_seq=0 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf427fc640 1 -- 172.21.15.105:0/336619987 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560083c76200 con 0x560083d0f590 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf427fc640 10 client.18663 handle_command_reply: tid=0 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 client.18663 shutdown 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 -- 172.21.15.105:0/336619987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d04fa0 msgr2=0x560083d1b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d04fa0 0x560083d1b300 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcf44030c30 tx=0x7fcf44010740 comp rx=0 tx=0).stop 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 -- 172.21.15.105:0/336619987 shutdown_connections 2024-03-26T17:47:14.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x560083d0f590 0x560083d87ac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560083d0a420 0x560083d182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf427fc640 0 client.18663 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560083d09a30 0x560083d17da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 --2- 172.21.15.105:0/336619987 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560083d04fa0 0x560083d1b300 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 -- 172.21.15.105:0/336619987 >> 172.21.15.105:0/336619987 conn(0x560083d0b200 msgr2=0x560083d13a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 -- 172.21.15.105:0/336619987 shutdown_connections 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 1 -- 172.21.15.105:0/336619987 wait complete. 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.161+0000 7fcf64071740 20 client.18663 trim_cache size 0 max 16384 2024-03-26T17:47:14.159 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:14.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 -- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcf480a2f20 msgr2=0x7fcf480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcf480a2f20 0x7fcf480a53e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcf5409c160 tx=0x7fcf5409f040 comp rx=0 tx=0).stop 2024-03-26T17:47:14.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 -- 172.21.15.105:0/3535088020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf5c102370 msgr2=0x7fcf5c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf5c102370 0x7fcf5c19f130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009fd0 tx=0x7fcf5400ef10 comp rx=0 tx=0).stop 2024-03-26T17:47:14.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 -- 172.21.15.105:0/3535088020 shutdown_connections 2024-03-26T17:47:14.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcf480a2f20 0x7fcf480a53e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf5c102370 0x7fcf5c19f130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf5c1019b0 0x7fcf5c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 --2- 172.21.15.105:0/3535088020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf5c1007c0 0x7fcf5c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 -- 172.21.15.105:0/3535088020 >> 172.21.15.105:0/3535088020 conn(0x7fcf5c0fbd50 msgr2=0x7fcf5c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 -- 172.21.15.105:0/3535088020 shutdown_connections 2024-03-26T17:47:14.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.162+0000 7fcf6148a640 1 -- 172.21.15.105:0/3535088020 wait complete. 2024-03-26T17:47:14.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:13 smithi135 ceph-mon[35636]: pgmap v614: 97 pgs: 97 active+clean; 29 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:47:14.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:13 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1546798150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:47:14.165 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:47:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 -- 172.21.15.105:0/819707983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070104d50 msgr2=0x7f7070107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 --2- 172.21.15.105:0/819707983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070104d50 0x7f7070107140 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f705c009900 tx=0x7f705c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 -- 172.21.15.105:0/819707983 shutdown_connections 2024-03-26T17:47:14.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 --2- 172.21.15.105:0/819707983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7070107680 0x7f7070109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 --2- 172.21.15.105:0/819707983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070104d50 0x7f7070107140 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 --2- 172.21.15.105:0/819707983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7070102420 0x7f7070104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 -- 172.21.15.105:0/819707983 >> 172.21.15.105:0/819707983 conn(0x7f70700fbd70 msgr2=0x7f70700fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 -- 172.21.15.105:0/819707983 shutdown_connections 2024-03-26T17:47:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 -- 172.21.15.105:0/819707983 wait complete. 2024-03-26T17:47:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 Processor -- start 2024-03-26T17:47:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 -- start start 2024-03-26T17:47:14.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7070102420 0x7f7070197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.331+0000 7f7076cde640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7070104d50 0x7f70701980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f7076cde640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070107680 0x7f707019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f7076cde640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f707010d470 con 0x7f7070107680 2024-03-26T17:47:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f7076cde640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f707010d2f0 con 0x7f7070104d50 2024-03-26T17:47:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f7076cde640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f707010d5f0 con 0x7f7070102420 2024-03-26T17:47:14.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070107680 0x7f707019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070107680 0x7f707019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57222/0 (socket says 172.21.15.105:57222) 2024-03-26T17:47:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 -- 172.21.15.105:0/1983792104 learned_addr learned my addr 172.21.15.105:0/1983792104 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f7074a7a640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7070102420 0x7f7070197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f706ffff640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7070104d50 0x7f70701980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 -- 172.21.15.105:0/1983792104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7070102420 msgr2=0x7f7070197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7070102420 0x7f7070197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 -- 172.21.15.105:0/1983792104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7070104d50 msgr2=0x7f70701980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7070104d50 0x7f70701980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 -- 172.21.15.105:0/1983792104 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707019f790 con 0x7f7070107680 2024-03-26T17:47:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f707527b640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070107680 0x7f707019f120 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f706800eb10 tx=0x7f706800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f706dffb640 1 -- 172.21.15.105:0/1983792104 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068004b20 con 0x7f7070107680 2024-03-26T17:47:14.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f706dffb640 1 -- 172.21.15.105:0/1983792104 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7068004cc0 con 0x7f7070107680 2024-03-26T17:47:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f7076cde640 1 -- 172.21.15.105:0/1983792104 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707019fa80 con 0x7f7070107680 2024-03-26T17:47:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.332+0000 7f706dffb640 1 -- 172.21.15.105:0/1983792104 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068010690 con 0x7f7070107680 2024-03-26T17:47:14.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.333+0000 7f7076cde640 1 -- 172.21.15.105:0/1983792104 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f707019fff0 con 0x7f7070107680 2024-03-26T17:47:14.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.334+0000 7f706dffb640 1 -- 172.21.15.105:0/1983792104 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7068005020 con 0x7f7070107680 2024-03-26T17:47:14.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f706dffb640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70600a2fa0 0x7f70600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f707999b740 1 Processor -- start 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f7074a7a640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70600a2fa0 0x7f70600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f706dffb640 1 -- 172.21.15.105:0/1983792104 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f70680cfa60 con 0x7f7070107680 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f707999b740 1 -- start start 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f707999b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e38a30 0x55fd64e38e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f707999b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e39420 0x55fd64e3df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f707999b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e40a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f707999b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fd64d9d970 con 0x55fd64e39420 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f707999b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fd64d45d70 con 0x55fd64e38a30 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.335+0000 7f707999b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fd64d99920 con 0x55fd64e3e590 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e39420 0x55fd64e3df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e39420 0x55fd64e3df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57224/0 (socket says 172.21.15.105:57224) 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 -- 172.21.15.105:0/1844290788 learned_addr learned my addr 172.21.15.105:0/1844290788 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f707527b640 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e40a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f7074a7a640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70600a2fa0 0x7f70600a5460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7064009bd0 tx=0x7f7064005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f7074a7a640 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e38a30 0x55fd64e38e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 -- 172.21.15.105:0/1844290788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 msgr2=0x55fd64e40a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e40a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 -- 172.21.15.105:0/1844290788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e38a30 msgr2=0x55fd64e38e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e38a30 0x55fd64e38e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 -- 172.21.15.105:0/1844290788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd64ab13f0 con 0x55fd64e39420 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f707527b640 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e40a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f706ffff640 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e39420 0x55fd64e3df20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f705c000c00 tx=0x7f705c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a92fad964da2c48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f705affd640 1 -- 172.21.15.105:0/1844290788 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c045020 con 0x55fd64e39420 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f705affd640 1 -- 172.21.15.105:0/1844290788 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f705c02c9a0 con 0x55fd64e39420 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.336+0000 7f705affd640 1 -- 172.21.15.105:0/1844290788 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c0055d0 con 0x55fd64e39420 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 -- 172.21.15.105:0/1844290788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e39420 msgr2=0x55fd64e3df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e39420 0x55fd64e3df20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f705c000c00 tx=0x7f705c0095f0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 -- 172.21.15.105:0/1844290788 shutdown_connections 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e40a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e39420 0x55fd64e3df20 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 --2- 172.21.15.105:0/1844290788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e38a30 0x55fd64e38e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 -- 172.21.15.105:0/1844290788 >> 172.21.15.105:0/1844290788 conn(0x55fd64e33fa0 msgr2=0x55fd64e343a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 -- 172.21.15.105:0/1844290788 shutdown_connections 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 -- 172.21.15.105:0/1844290788 wait complete. 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.337+0000 7f707999b740 1 -- start start 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707999b740 1 Processor -- start 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707999b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e42ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707999b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 0x55fd64e43400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707999b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e4a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707999b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fd64e414e0 con 0x55fd64e38a30 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707999b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fd64d59310 con 0x55fd64e39420 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707999b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fd64e41660 con 0x55fd64e3e590 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707999b740 10 client.0 _ll_register_callbacks cb 0x55fd64d94c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e42ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e42ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57228/0 (socket says 172.21.15.105:57228) 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 -- 172.21.15.105:0/3911116964 learned_addr learned my addr 172.21.15.105:0/3911116964 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f706ffff640 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 0x55fd64e43400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707527b640 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e4a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 -- 172.21.15.105:0/3911116964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 msgr2=0x55fd64e4a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e4a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 -- 172.21.15.105:0/3911116964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 msgr2=0x55fd64e43400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 0x55fd64e43400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 -- 172.21.15.105:0/3911116964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd64b7b7d0 con 0x55fd64e38a30 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f707527b640 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f7074a7a640 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e42ec0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f706400ed70 tx=0x7f7064004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7505afd1bc9f08fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.338+0000 7f704ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f704ffff640 1 -- 172.21.15.105:0/3911116964 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706401db20 con 0x55fd64e38a30 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f704ffff640 1 -- 172.21.15.105:0/3911116964 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f706401dcc0 con 0x55fd64e38a30 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 1 -- 172.21.15.105:0/3911116964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f705c02c9a0 con 0x55fd64e38a30 2024-03-26T17:47:14.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f704ffff640 1 -- 172.21.15.105:0/3911116964 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70640047a0 con 0x55fd64e38a30 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 10 client.18675 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 1 -- 172.21.15.105:0/3911116964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd64e4d820 con 0x55fd64e38a30 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f704ffff640 1 -- 172.21.15.105:0/3911116964 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f705c02c9a0 con 0x55fd64e38a30 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f704ffff640 1 -- 172.21.15.105:0/3911116964 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7064026410 con 0x55fd64e38a30 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 10 client.18675 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 10 client.18675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 20 client.18675 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fd64e4efb0 0x55fd64e51470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 4 client.18675 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f707999b740 1 -- 172.21.15.105:0/3911116964 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd64daaeb0 con 0x55fd64e4efb0 2024-03-26T17:47:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.339+0000 7f706ffff640 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fd64e4efb0 0x55fd64e51470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.340+0000 7f704ffff640 10 client.18675 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.340+0000 7f706ffff640 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fd64e4efb0 0x55fd64e51470 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5f317e3fd87835 server_cookie=b2023f549b114fb5 in_seq=0 out_seq=0 2024-03-26T17:47:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.341+0000 7f704ffff640 1 -- 172.21.15.105:0/3911116964 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fd64daaeb0 con 0x55fd64e4efb0 2024-03-26T17:47:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.341+0000 7f704ffff640 10 client.18675 handle_command_reply: tid=0 2024-03-26T17:47:14.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 client.18675 shutdown 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 -- 172.21.15.105:0/3911116964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 msgr2=0x55fd64e42ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e42ec0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f706400ed70 tx=0x7f7064004730 comp rx=0 tx=0).stop 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 -- 172.21.15.105:0/3911116964 shutdown_connections 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fd64e4efb0 0x55fd64e51470 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e3e590 0x55fd64e4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f704ffff640 0 client.18675 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 0x55fd64e43400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 --2- 172.21.15.105:0/3911116964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e42ec0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 -- 172.21.15.105:0/3911116964 >> 172.21.15.105:0/3911116964 conn(0x55fd64e33fa0 msgr2=0x55fd64e331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 -- 172.21.15.105:0/3911116964 shutdown_connections 2024-03-26T17:47:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 1 -- 172.21.15.105:0/3911116964 wait complete. 2024-03-26T17:47:14.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.342+0000 7f707999b740 20 client.18675 trim_cache size 0 max 16384 2024-03-26T17:47:14.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707999b740 1 Processor -- start 2024-03-26T17:47:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707999b740 1 -- start start 2024-03-26T17:47:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707999b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e38a30 0x55fd64e43360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707999b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 0x55fd64e4a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707999b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e3e590 0x55fd64e33040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707999b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70701a8d90 con 0x55fd64e3e590 2024-03-26T17:47:14.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707999b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70701aa8d0 con 0x55fd64e39420 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707999b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70701a9460 con 0x55fd64e38a30 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707527b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e3e590 0x55fd64e33040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707527b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e3e590 0x55fd64e33040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57244/0 (socket says 172.21.15.105:57244) 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f707527b640 1 -- 172.21.15.105:0/3513272551 learned_addr learned my addr 172.21.15.105:0/3513272551 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f706ffff640 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 0x55fd64e4a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.365+0000 7f7074a7a640 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e38a30 0x55fd64e43360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f707527b640 1 -- 172.21.15.105:0/3513272551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e38a30 msgr2=0x55fd64e43360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f707527b640 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e38a30 0x55fd64e43360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f707527b640 1 -- 172.21.15.105:0/3513272551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 msgr2=0x55fd64e4a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f707527b640 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 0x55fd64e4a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f707527b640 1 -- 172.21.15.105:0/3513272551 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd64e338f0 con 0x55fd64e3e590 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f707527b640 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e3e590 0x55fd64e33040 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f70680985b0 tx=0x7f7068009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4662eb3ef7368dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f705a7fc640 1 -- 172.21.15.105:0/3513272551 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706805ff40 con 0x55fd64e3e590 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f705a7fc640 1 -- 172.21.15.105:0/3513272551 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f706809c150 con 0x55fd64e3e590 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.366+0000 7f705a7fc640 1 -- 172.21.15.105:0/3513272551 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068005310 con 0x55fd64e3e590 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 -- 172.21.15.105:0/3513272551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e3e590 msgr2=0x55fd64e33040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e3e590 0x55fd64e33040 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f70680985b0 tx=0x7f7068009a70 comp rx=0 tx=0).stop 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 -- 172.21.15.105:0/3513272551 shutdown_connections 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e3e590 0x55fd64e33040 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e39420 0x55fd64e4a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 --2- 172.21.15.105:0/3513272551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e38a30 0x55fd64e43360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 -- 172.21.15.105:0/3513272551 >> 172.21.15.105:0/3513272551 conn(0x55fd64e33fa0 msgr2=0x55fd64e32930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 -- 172.21.15.105:0/3513272551 shutdown_connections 2024-03-26T17:47:14.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 -- 172.21.15.105:0/3513272551 wait complete. 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 -- start start 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.367+0000 7f707999b740 1 Processor -- start 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f707999b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e47ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f707999b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e39420 0x55fd64e48410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f707999b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e3e590 0x55fd64e49420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f707999b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70701aa200 con 0x55fd64e38a30 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f707999b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70701a9b30 con 0x55fd64e3e590 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f707999b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70701a6d70 con 0x55fd64e39420 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f707999b740 10 client.0 _ll_register_callbacks cb 0x7f706400e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:14.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e47ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e47ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57258/0 (socket says 172.21.15.105:57258) 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 -- 172.21.15.105:0/1219951215 learned_addr learned my addr 172.21.15.105:0/1219951215 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f707527b640 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e3e590 0x55fd64e49420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f706ffff640 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e39420 0x55fd64e48410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 -- 172.21.15.105:0/1219951215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e39420 msgr2=0x55fd64e48410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e39420 0x55fd64e48410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 -- 172.21.15.105:0/1219951215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e3e590 msgr2=0x55fd64e49420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e3e590 0x55fd64e49420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 -- 172.21.15.105:0/1219951215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd64e49a10 con 0x55fd64e38a30 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f7074a7a640 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e47ed0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7064016040 tx=0x7f7064026df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8ae41b10e2a2f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f704ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f704ffff640 1 -- 172.21.15.105:0/1219951215 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064035070 con 0x55fd64e38a30 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.368+0000 7f704ffff640 1 -- 172.21.15.105:0/1219951215 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7064009040 con 0x55fd64e38a30 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 1 -- 172.21.15.105:0/1219951215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f706809c150 con 0x55fd64e38a30 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f704ffff640 1 -- 172.21.15.105:0/1219951215 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70640266b0 con 0x55fd64e38a30 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 10 client.18681 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 1 -- 172.21.15.105:0/1219951215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd64eb5710 con 0x55fd64e38a30 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f704ffff640 1 -- 172.21.15.105:0/1219951215 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f706809c150 con 0x55fd64e38a30 2024-03-26T17:47:14.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f704ffff640 1 -- 172.21.15.105:0/1219951215 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7064003760 con 0x55fd64e38a30 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 10 client.18681 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 10 client.18681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 20 client.18681 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fd64eb6080 0x55fd64eb84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 4 client.18681 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f707999b740 1 -- 172.21.15.105:0/1219951215 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd64da5200 con 0x55fd64eb6080 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.369+0000 7f706ffff640 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fd64eb6080 0x55fd64eb84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f706ffff640 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fd64eb6080 0x55fd64eb84a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e674e4646ead3114 server_cookie=e53bc7185a2c9235 in_seq=0 out_seq=0 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f704ffff640 10 client.18681 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f704ffff640 1 -- 172.21.15.105:0/1219951215 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd64da5200 con 0x55fd64eb6080 2024-03-26T17:47:14.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f704ffff640 10 client.18681 handle_command_reply: tid=0 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 client.18681 shutdown 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 -- 172.21.15.105:0/1219951215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 msgr2=0x55fd64e47ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e47ed0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7064016040 tx=0x7f7064026df0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 -- 172.21.15.105:0/1219951215 shutdown_connections 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fd64eb6080 0x55fd64eb84a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd64e3e590 0x55fd64e49420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fd64e39420 0x55fd64e48410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f704ffff640 0 client.18681 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 --2- 172.21.15.105:0/1219951215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fd64e38a30 0x55fd64e47ed0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.370+0000 7f707999b740 1 -- 172.21.15.105:0/1219951215 >> 172.21.15.105:0/1219951215 conn(0x55fd64e33fa0 msgr2=0x55fd64e331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f707999b740 1 -- 172.21.15.105:0/1219951215 shutdown_connections 2024-03-26T17:47:14.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f707999b740 1 -- 172.21.15.105:0/1219951215 wait complete. 2024-03-26T17:47:14.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f707999b740 20 client.18681 trim_cache size 0 max 16384 2024-03-26T17:47:14.372 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 -- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70600a2fa0 msgr2=0x7f70600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70600a2fa0 0x7f70600a5460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7064009bd0 tx=0x7f7064005f90 comp rx=0 tx=0).stop 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 -- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070107680 msgr2=0x7f707019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070107680 0x7f707019f120 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f706800eb10 tx=0x7f706800c2d0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 -- 172.21.15.105:0/1983792104 shutdown_connections 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70600a2fa0 0x7f70600a5460 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070107680 0x7f707019f120 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7070104d50 0x7f70701980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 --2- 172.21.15.105:0/1983792104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7070102420 0x7f7070197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 -- 172.21.15.105:0/1983792104 >> 172.21.15.105:0/1983792104 conn(0x7f70700fbd70 msgr2=0x7f7070102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 -- 172.21.15.105:0/1983792104 shutdown_connections 2024-03-26T17:47:14.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:14.371+0000 7f7076cde640 1 -- 172.21.15.105:0/1983792104 wait complete. 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 -- 172.21.15.105:0/2221675521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c1019d0 msgr2=0x7f405c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 --2- 172.21.15.105:0/2221675521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c1019d0 0x7f405c101e50 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4050009900 tx=0x7f405002bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 -- 172.21.15.105:0/2221675521 shutdown_connections 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 --2- 172.21.15.105:0/2221675521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f405c102390 0x7f405c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 --2- 172.21.15.105:0/2221675521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c1019d0 0x7f405c101e50 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 --2- 172.21.15.105:0/2221675521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f405c1007e0 0x7f405c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 -- 172.21.15.105:0/2221675521 >> 172.21.15.105:0/2221675521 conn(0x7f405c0fbd50 msgr2=0x7f405c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 -- 172.21.15.105:0/2221675521 shutdown_connections 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 -- 172.21.15.105:0/2221675521 wait complete. 2024-03-26T17:47:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.771+0000 7f4061830640 1 Processor -- start 2024-03-26T17:47:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f4061830640 1 -- start start 2024-03-26T17:47:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f4061830640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f405c1007e0 0x7f405c197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f4061830640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c102390 0x7f405c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f4061830640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f405c19d200 0x7f405c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f4061830640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f405c10d420 con 0x7f405c102390 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f4061830640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f405c10d2a0 con 0x7f405c1007e0 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f4061830640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f405c10d5a0 con 0x7f405c19d200 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c102390 0x7f405c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f405c1007e0 0x7f405c197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c102390 0x7f405c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60822/0 (socket says 172.21.15.105:60822) 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405a7fc640 1 -- 172.21.15.105:0/1125160052 learned_addr learned my addr 172.21.15.105:0/1125160052 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f405c1007e0 0x7f405c197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59774/0 (socket says 172.21.15.105:59774) 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405b7fe640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f405c19d200 0x7f405c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405a7fc640 1 -- 172.21.15.105:0/1125160052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f405c19d200 msgr2=0x7f405c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405a7fc640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f405c19d200 0x7f405c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405a7fc640 1 -- 172.21.15.105:0/1125160052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f405c1007e0 msgr2=0x7f405c197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405a7fc640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f405c1007e0 0x7f405c197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405a7fc640 1 -- 172.21.15.105:0/1125160052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c19fc90 con 0x7f405c102390 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.772+0000 7f405affd640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f405c1007e0 0x7f405c197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.773+0000 7f405a7fc640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c102390 0x7f405c198200 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4050036040 tx=0x7f4050005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.773+0000 7f406082e640 1 -- 172.21.15.105:0/1125160052 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050043020 con 0x7f405c102390 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.773+0000 7f4061830640 1 -- 172.21.15.105:0/1125160052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405c19ff20 con 0x7f405c102390 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.773+0000 7f406082e640 1 -- 172.21.15.105:0/1125160052 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4050004350 con 0x7f405c102390 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.773+0000 7f406082e640 1 -- 172.21.15.105:0/1125160052 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050003b00 con 0x7f405c102390 2024-03-26T17:47:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.773+0000 7f4061830640 1 -- 172.21.15.105:0/1125160052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405c1a03e0 con 0x7f405c102390 2024-03-26T17:47:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f406082e640 1 -- 172.21.15.105:0/1125160052 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4050046020 con 0x7f405c102390 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f4064417740 1 Processor -- start 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f4064417740 1 -- start start 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f4064417740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe01b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f4064417740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe020ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f4064417740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe0235c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f4064417740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5630fdf800b0 con 0x5630fe021130 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f4064417740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5630fdf28960 con 0x5630fe01b5d0 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.775+0000 7f4064417740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5630fdf7c060 con 0x5630fe01bfc0 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe01b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe01b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59778/0 (socket says 172.21.15.105:59778) 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 -- 172.21.15.105:0/371484784 learned_addr learned my addr 172.21.15.105:0/371484784 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405a7fc640 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe0235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405affd640 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe020ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f406082e640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40440a2fc0 0x7f40440a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 -- 172.21.15.105:0/371484784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 msgr2=0x5630fe020ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe020ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 -- 172.21.15.105:0/371484784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 msgr2=0x5630fe0235c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe0235c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 -- 172.21.15.105:0/371484784 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630fdf3ac60 con 0x5630fe01b5d0 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405a7fc640 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe0235c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f406082e640 1 -- 172.21.15.105:0/1125160052 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f40500f1240 con 0x7f405c102390 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40440a2fc0 0x7f40440a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.776+0000 7f405b7fe640 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe01b9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4054009fd0 tx=0x7f40540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2036fc31731cabe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.777+0000 7f4041ffb640 1 -- 172.21.15.105:0/371484784 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054004250 con 0x5630fe01b5d0 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.777+0000 7f4041ffb640 1 -- 172.21.15.105:0/371484784 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4054003db0 con 0x5630fe01b5d0 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.777+0000 7f4041ffb640 1 -- 172.21.15.105:0/371484784 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405401bb80 con 0x5630fe01b5d0 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.777+0000 7f4064417740 1 -- 172.21.15.105:0/371484784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 msgr2=0x5630fe01b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.777+0000 7f405b7fe640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40440a2fc0 0x7f40440a5480 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4054020000 tx=0x7f4054021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.777+0000 7f4064417740 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe01b9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4054009fd0 tx=0x7f40540118d0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 -- 172.21.15.105:0/371484784 shutdown_connections 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe0235c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe020ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 --2- 172.21.15.105:0/371484784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe01b9f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 -- 172.21.15.105:0/371484784 >> 172.21.15.105:0/371484784 conn(0x5630fe016b40 msgr2=0x5630fe016f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 -- 172.21.15.105:0/371484784 shutdown_connections 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 -- 172.21.15.105:0/371484784 wait complete. 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 -- start start 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 Processor -- start 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe025400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe025940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe02c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5630fe0240e0 con 0x5630fe021130 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5630fdf3bdb0 con 0x5630fe01b5d0 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5630fe024260 con 0x5630fe01bfc0 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f4064417740 10 client.0 _ll_register_callbacks cb 0x5630fdf40110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.778+0000 7f405b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe02c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe025400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe02c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60842/0 (socket says 172.21.15.105:60842) 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe025400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59788/0 (socket says 172.21.15.105:59788) 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405b7fe640 1 -- 172.21.15.105:0/100506379 learned_addr learned my addr 172.21.15.105:0/100506379 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405a7fc640 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe025940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405b7fe640 1 -- 172.21.15.105:0/100506379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 msgr2=0x5630fe025940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405b7fe640 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe025940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405b7fe640 1 -- 172.21.15.105:0/100506379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 msgr2=0x5630fe025400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405b7fe640 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe025400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405b7fe640 1 -- 172.21.15.105:0/100506379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630fdc94210 con 0x5630fe021130 2024-03-26T17:47:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405a7fc640 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe025940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405affd640 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe025400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f405b7fe640 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe02c910 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4054004430 tx=0x7f405401b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460236fddec8ec2f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f403affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f403affd640 1 -- 172.21.15.105:0/100506379 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054034020 con 0x5630fe021130 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f403affd640 1 -- 172.21.15.105:0/100506379 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f405401bb00 con 0x5630fe021130 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f4064417740 1 -- 172.21.15.105:0/100506379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4054003db0 con 0x5630fe021130 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f403affd640 1 -- 172.21.15.105:0/100506379 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f405403f050 con 0x5630fe021130 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f4064417740 10 client.18693 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.779+0000 7f4064417740 1 -- 172.21.15.105:0/100506379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630fdd60620 con 0x5630fe021130 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f403affd640 1 -- 172.21.15.105:0/100506379 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4054003db0 con 0x5630fe021130 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f403affd640 1 -- 172.21.15.105:0/100506379 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f405402f9a0 con 0x5630fe021130 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f4064417740 10 client.18693 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f4064417740 10 client.18693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:14.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f4064417740 20 client.18693 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f4064417740 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630fe0310a0 0x5630fe033490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f4064417740 4 client.18693 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f4064417740 1 -- 172.21.15.105:0/100506379 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630fdf8d5f0 con 0x5630fe0310a0 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f405affd640 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630fe0310a0 0x5630fe033490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f405affd640 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630fe0310a0 0x5630fe033490 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4355bb7f55bb7b60 server_cookie=cbfe05bdf463d53 in_seq=0 out_seq=0 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.780+0000 7f403affd640 10 client.18693 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.781+0000 7f403affd640 1 -- 172.21.15.105:0/100506379 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5630fdf8d5f0 con 0x5630fe0310a0 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.781+0000 7f403affd640 10 client.18693 handle_command_reply: tid=0 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.781+0000 7f4064417740 1 client.18693 shutdown 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.781+0000 7f4064417740 1 -- 172.21.15.105:0/100506379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 msgr2=0x5630fe02c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.781+0000 7f4064417740 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe02c910 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4054004430 tx=0x7f405401b4d0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.781+0000 7f4064417740 1 -- 172.21.15.105:0/100506379 shutdown_connections 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.781+0000 7f4064417740 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630fe0310a0 0x5630fe033490 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.782+0000 7f403affd640 0 client.18693 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.782+0000 7f4064417740 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe021130 0x5630fe02c910 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.782+0000 7f4064417740 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01bfc0 0x5630fe025940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.782+0000 7f4064417740 1 --2- 172.21.15.105:0/100506379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe025400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.782+0000 7f4064417740 1 -- 172.21.15.105:0/100506379 >> 172.21.15.105:0/100506379 conn(0x5630fe016b40 msgr2=0x5630fe015df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.782+0000 7f4064417740 1 -- 172.21.15.105:0/100506379 shutdown_connections 2024-03-26T17:47:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.782+0000 7f4064417740 1 -- 172.21.15.105:0/100506379 wait complete. 2024-03-26T17:47:14.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.782+0000 7f4064417740 20 client.18693 trim_cache size 0 max 16384 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f4064417740 1 Processor -- start 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f4064417740 1 -- start start 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f4064417740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe020b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f4064417740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe02c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f4064417740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe021130 0x5630fe01ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f4064417740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1a92a0 con 0x5630fe01bfc0 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f4064417740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1aade0 con 0x5630fe01b5d0 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f4064417740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1a9970 con 0x5630fe021130 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f405a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe02c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f405a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe02c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60848/0 (socket says 172.21.15.105:60848) 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f405a7fc640 1 -- 172.21.15.105:0/2906230833 learned_addr learned my addr 172.21.15.105:0/2906230833 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.800+0000 7f405affd640 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe020b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f405b7fe640 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe021130 0x5630fe01ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f405a7fc640 1 -- 172.21.15.105:0/2906230833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe021130 msgr2=0x5630fe01ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f405a7fc640 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe021130 0x5630fe01ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f405a7fc640 1 -- 172.21.15.105:0/2906230833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 msgr2=0x5630fe020b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f405a7fc640 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe020b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f405a7fc640 1 -- 172.21.15.105:0/2906230833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630fe0178b0 con 0x5630fe01bfc0 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f405a7fc640 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe02c8d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f40500ba300 tx=0x7f40500a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8445eb300391df19 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f40417fa640 1 -- 172.21.15.105:0/2906230833 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500b9bd0 con 0x5630fe01bfc0 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f40417fa640 1 -- 172.21.15.105:0/2906230833 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40500be030 con 0x5630fe01bfc0 2024-03-26T17:47:14.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.801+0000 7f40417fa640 1 -- 172.21.15.105:0/2906230833 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500bbd10 con 0x5630fe01bfc0 2024-03-26T17:47:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 -- 172.21.15.105:0/2906230833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 msgr2=0x5630fe02c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe02c8d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f40500ba300 tx=0x7f40500a8040 comp rx=0 tx=0).stop 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 -- 172.21.15.105:0/2906230833 shutdown_connections 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe021130 0x5630fe01ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe02c8d0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 --2- 172.21.15.105:0/2906230833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe01b5d0 0x5630fe020b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 -- 172.21.15.105:0/2906230833 >> 172.21.15.105:0/2906230833 conn(0x5630fe016b40 msgr2=0x5630fe020400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 -- 172.21.15.105:0/2906230833 shutdown_connections 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 -- 172.21.15.105:0/2906230833 wait complete. 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 -- start start 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.802+0000 7f4064417740 1 Processor -- start 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01b5d0 0x5630fe0294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe0299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe021130 0x5630fe095350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1aa710 con 0x5630fe01bfc0 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1aa040 con 0x5630fe021130 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1a7280 con 0x5630fe01b5d0 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 10 client.0 _ll_register_callbacks cb 0x7f404c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe0299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01b5d0 0x5630fe0294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe0299e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60854/0 (socket says 172.21.15.105:60854) 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 -- 172.21.15.105:0/3549506043 learned_addr learned my addr 172.21.15.105:0/3549506043 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405b7fe640 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe021130 0x5630fe095350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 -- 172.21.15.105:0/3549506043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01b5d0 msgr2=0x5630fe0294a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01b5d0 0x5630fe0294a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 -- 172.21.15.105:0/3549506043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe021130 msgr2=0x5630fe095350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe021130 0x5630fe095350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 -- 172.21.15.105:0/3549506043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630fe02ca10 con 0x5630fe01bfc0 2024-03-26T17:47:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f405a7fc640 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe0299e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f40500c6500 tx=0x7f4050034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a314fc07610b237e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f403affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f403affd640 1 -- 172.21.15.105:0/3549506043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500d5020 con 0x5630fe01bfc0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f403affd640 1 -- 172.21.15.105:0/3549506043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f40500c5eb0 con 0x5630fe01bfc0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 1 -- 172.21.15.105:0/3549506043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40500be030 con 0x5630fe01bfc0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f403affd640 1 -- 172.21.15.105:0/3549506043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40500b9bd0 con 0x5630fe01bfc0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 10 client.18699 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.803+0000 7f4064417740 1 -- 172.21.15.105:0/3549506043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630fe0967b0 con 0x5630fe01bfc0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f403affd640 1 -- 172.21.15.105:0/3549506043 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40500be030 con 0x5630fe01bfc0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f403affd640 1 -- 172.21.15.105:0/3549506043 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f40500d8ce0 con 0x5630fe01bfc0 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f4064417740 10 client.18699 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f4064417740 10 client.18699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f4064417740 20 client.18699 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f4064417740 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630fe0310a0 0x5630fe0990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f4064417740 4 client.18699 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:47:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f4064417740 1 -- 172.21.15.105:0/3549506043 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630fdf87940 con 0x5630fe0310a0 2024-03-26T17:47:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f405affd640 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630fe0310a0 0x5630fe0990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f405affd640 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630fe0310a0 0x5630fe0990b0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0d4959e861a709 server_cookie=552508358fe91de6 in_seq=0 out_seq=0 2024-03-26T17:47:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.804+0000 7f403affd640 10 client.18699 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f403affd640 1 -- 172.21.15.105:0/3549506043 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630fdf87940 con 0x5630fe0310a0 2024-03-26T17:47:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f403affd640 10 client.18699 handle_command_reply: tid=0 2024-03-26T17:47:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 client.18699 shutdown 2024-03-26T17:47:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 -- 172.21.15.105:0/3549506043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 msgr2=0x5630fe0299e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe0299e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f40500c6500 tx=0x7f4050034000 comp rx=0 tx=0).stop 2024-03-26T17:47:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 -- 172.21.15.105:0/3549506043 shutdown_connections 2024-03-26T17:47:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630fe0310a0 0x5630fe0990b0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630fe021130 0x5630fe095350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630fe01bfc0 0x5630fe0299e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f403affd640 0 client.18699 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 --2- 172.21.15.105:0/3549506043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630fe01b5d0 0x5630fe0294a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 -- 172.21.15.105:0/3549506043 >> 172.21.15.105:0/3549506043 conn(0x5630fe016b40 msgr2=0x5630fe020040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 -- 172.21.15.105:0/3549506043 shutdown_connections 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 1 -- 172.21.15.105:0/3549506043 wait complete. 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.857+0000 7f4064417740 20 client.18699 trim_cache size 0 max 16384 2024-03-26T17:47:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:47:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:47:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:47:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:47:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 -- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40440a2fc0 msgr2=0x7f40440a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40440a2fc0 0x7f40440a5480 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4054020000 tx=0x7f4054021000 comp rx=0 tx=0).stop 2024-03-26T17:47:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 -- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c102390 msgr2=0x7f405c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c102390 0x7f405c198200 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4050036040 tx=0x7f4050005dc0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 -- 172.21.15.105:0/1125160052 shutdown_connections 2024-03-26T17:47:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40440a2fc0 0x7f40440a5480 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f405c19d200 0x7f405c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f405c102390 0x7f405c198200 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 --2- 172.21.15.105:0/1125160052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f405c1007e0 0x7f405c197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 -- 172.21.15.105:0/1125160052 >> 172.21.15.105:0/1125160052 conn(0x7f405c0fbd50 msgr2=0x7f405c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 -- 172.21.15.105:0/1125160052 shutdown_connections 2024-03-26T17:47:14.852 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:14.858+0000 7f4061830640 1 -- 172.21.15.105:0/1125160052 wait complete. 2024-03-26T17:47:15.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 280761 3.15 MB/sec execute 358 sec latency 16.762 ms 2024-03-26T17:47:15.375 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:47:15.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:47:15.375 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:47:15.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 -- 172.21.15.105:0/4078167986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d8103480 msgr2=0x7f95d8109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 --2- 172.21.15.105:0/4078167986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d8103480 0x7f95d8109d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009be0 tx=0x7f95d002bef0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 -- 172.21.15.105:0/4078167986 shutdown_connections 2024-03-26T17:47:15.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 --2- 172.21.15.105:0/4078167986 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d8103480 0x7f95d8109d10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 --2- 172.21.15.105:0/4078167986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d8102ac0 0x7f95d8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 --2- 172.21.15.105:0/4078167986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d81018d0 0x7f95d8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 -- 172.21.15.105:0/4078167986 >> 172.21.15.105:0/4078167986 conn(0x7f95d80fce40 msgr2=0x7f95d80ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 -- 172.21.15.105:0/4078167986 shutdown_connections 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.505+0000 7f95de4dc640 1 -- 172.21.15.105:0/4078167986 wait complete. 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95de4dc640 1 Processor -- start 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95de4dc640 1 -- start start 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95de4dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d81018d0 0x7f95d8075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95de4dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d8102ac0 0x7f95d8073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95de4dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d8103480 0x7f95d80741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95de4dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95d810d3e0 con 0x7f95d8102ac0 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95de4dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95d810d260 con 0x7f95d81018d0 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95de4dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95d810d560 con 0x7f95d8103480 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d8102ac0 0x7f95d8073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d81018d0 0x7f95d8075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d8102ac0 0x7f95d8073cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60878/0 (socket says 172.21.15.105:60878) 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95d77fe640 1 -- 172.21.15.105:0/2109166481 learned_addr learned my addr 172.21.15.105:0/2109166481 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.506+0000 7f95dca79640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d8103480 0x7f95d80741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d77fe640 1 -- 172.21.15.105:0/2109166481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d8103480 msgr2=0x7f95d80741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d77fe640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d8103480 0x7f95d80741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d77fe640 1 -- 172.21.15.105:0/2109166481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d81018d0 msgr2=0x7f95d8075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d77fe640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d81018d0 0x7f95d8075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d77fe640 1 -- 172.21.15.105:0/2109166481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d8074a80 con 0x7f95d8102ac0 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d7fff640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d81018d0 0x7f95d8075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d77fe640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d8102ac0 0x7f95d8073cb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00ea60 tx=0x7f95cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d57fa640 1 -- 172.21.15.105:0/2109166481 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc00cde0 con 0x7f95d8102ac0 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d57fa640 1 -- 172.21.15.105:0/2109166481 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95cc0035f0 con 0x7f95d8102ac0 2024-03-26T17:47:15.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95de4dc640 1 -- 172.21.15.105:0/2109166481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d81a1bd0 con 0x7f95d8102ac0 2024-03-26T17:47:15.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95d57fa640 1 -- 172.21.15.105:0/2109166481 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc010690 con 0x7f95d8102ac0 2024-03-26T17:47:15.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.507+0000 7f95de4dc640 1 -- 172.21.15.105:0/2109166481 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81a2120 con 0x7f95d8102ac0 2024-03-26T17:47:15.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95d57fa640 1 -- 172.21.15.105:0/2109166481 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f95cc0037a0 con 0x7f95d8102ac0 2024-03-26T17:47:15.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95e11a2740 1 Processor -- start 2024-03-26T17:47:15.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95e11a2740 1 -- start start 2024-03-26T17:47:15.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95d57fa640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c00a2f20 0x7f95c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x5576491075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557649107b80 0x55764910c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55764910ccf0 0x55764910f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95e11a2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55764906c630 con 0x557649107190 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95e11a2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557649014e10 con 0x557649107b80 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.510+0000 7f95e11a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576490685e0 con 0x55764910ccf0 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d57fa640 1 -- 172.21.15.105:0/2109166481 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f95cc0cf690 con 0x7f95d8102ac0 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95dca79640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c00a2f20 0x7f95c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x5576491075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557649107b80 0x55764910c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557649107b80 0x55764910c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59826/0 (socket says 172.21.15.105:59826) 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 -- 172.21.15.105:0/4206958729 learned_addr learned my addr 172.21.15.105:0/4206958729 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95dca79640 1 --2- 172.21.15.105:0/4206958729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55764910ccf0 0x55764910f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 -- 172.21.15.105:0/4206958729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55764910ccf0 msgr2=0x55764910f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 --2- 172.21.15.105:0/4206958729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55764910ccf0 0x55764910f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 -- 172.21.15.105:0/4206958729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 msgr2=0x5576491075b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 --2- 172.21.15.105:0/4206958729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x5576491075b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 -- 172.21.15.105:0/4206958729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55764902b740 con 0x557649107b80 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95dca79640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c00a2f20 0x7f95c00a53e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f95d0004af0 tx=0x7f95d0033780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95d77fe640 1 --2- 172.21.15.105:0/4206958729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557649107b80 0x55764910c680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f95cc09ba80 tx=0x7f95cc09bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd59d523d40c6b89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95be7fc640 1 -- 172.21.15.105:0/4206958729 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc003b30 con 0x557649107b80 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95be7fc640 1 -- 172.21.15.105:0/4206958729 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95cc0489f0 con 0x557649107b80 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.511+0000 7f95be7fc640 1 -- 172.21.15.105:0/4206958729 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0a4790 con 0x557649107b80 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 -- 172.21.15.105:0/4206958729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557649107b80 msgr2=0x55764910c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 --2- 172.21.15.105:0/4206958729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557649107b80 0x55764910c680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f95cc09ba80 tx=0x7f95cc09bc10 comp rx=0 tx=0).stop 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 -- 172.21.15.105:0/4206958729 shutdown_connections 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 --2- 172.21.15.105:0/4206958729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55764910ccf0 0x55764910f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 --2- 172.21.15.105:0/4206958729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557649107b80 0x55764910c680 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 --2- 172.21.15.105:0/4206958729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x5576491075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 -- 172.21.15.105:0/4206958729 >> 172.21.15.105:0/4206958729 conn(0x557649102700 msgr2=0x557649102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 -- 172.21.15.105:0/4206958729 shutdown_connections 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 -- 172.21.15.105:0/4206958729 wait complete. 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.512+0000 7f95e11a2740 1 -- start start 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95e11a2740 1 Processor -- start 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107190 0x557649111680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107b80 0x557649111bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649118c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95e11a2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55764910fca0 con 0x557649107b80 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95e11a2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5576490281a0 con 0x55764910ccf0 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95e11a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55764910fe20 con 0x557649107190 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95e11a2740 10 client.0 _ll_register_callbacks cb 0x557648d7fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107b80 0x557649111bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107190 0x557649111680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95dca79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649118c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107190 0x557649111680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53770/0 (socket says 172.21.15.105:53770) 2024-03-26T17:47:15.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107b80 0x557649111bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60898/0 (socket says 172.21.15.105:60898) 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d7fff640 1 -- 172.21.15.105:0/1490789120 learned_addr learned my addr 172.21.15.105:0/1490789120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d77fe640 1 -- 172.21.15.105:0/1490789120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107190 msgr2=0x557649111680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d77fe640 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107190 0x557649111680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d77fe640 1 -- 172.21.15.105:0/1490789120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 msgr2=0x557649118c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d77fe640 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649118c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.513+0000 7f95d77fe640 1 -- 172.21.15.105:0/1490789120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557648e487e0 con 0x557649107b80 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95d77fe640 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107b80 0x557649111bc0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f95cc09bd10 tx=0x7f95cc009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a017d82c7d131b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95b77fe640 1 -- 172.21.15.105:0/1490789120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc09c480 con 0x557649107b80 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95b77fe640 1 -- 172.21.15.105:0/1490789120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f95cc048f90 con 0x557649107b80 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 1 -- 172.21.15.105:0/1490789120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95cc0489f0 con 0x557649107b80 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95b77fe640 1 -- 172.21.15.105:0/1490789120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95cc0a4790 con 0x557649107b80 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 10 client.18711 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 1 -- 172.21.15.105:0/1490789120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557648ea8f70 con 0x557649107b80 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95b77fe640 1 -- 172.21.15.105:0/1490789120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95cc0489f0 con 0x557649107b80 2024-03-26T17:47:15.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95b77fe640 1 -- 172.21.15.105:0/1490789120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f95cc0baaf0 con 0x557649107b80 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 10 client.18711 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 10 client.18711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 20 client.18711 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55764911d0f0 0x55764911f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 4 client.18711 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.514+0000 7f95e11a2740 1 -- 172.21.15.105:0/1490789120 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557649079b70 con 0x55764911d0f0 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.515+0000 7f95d7fff640 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55764911d0f0 0x55764911f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.515+0000 7f95d7fff640 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55764911d0f0 0x55764911f4e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95aee73ec4e67a4b server_cookie=fdebb51a03d27e1c in_seq=0 out_seq=0 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.515+0000 7f95b77fe640 10 client.18711 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95b77fe640 1 -- 172.21.15.105:0/1490789120 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557649079b70 con 0x55764911d0f0 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95b77fe640 10 client.18711 handle_command_reply: tid=0 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95e11a2740 1 client.18711 shutdown 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95e11a2740 1 -- 172.21.15.105:0/1490789120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107b80 msgr2=0x557649111bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95e11a2740 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107b80 0x557649111bc0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f95cc09bd10 tx=0x7f95cc009490 comp rx=0 tx=0).stop 2024-03-26T17:47:15.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95e11a2740 1 -- 172.21.15.105:0/1490789120 shutdown_connections 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95e11a2740 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55764911d0f0 0x55764911f4e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95e11a2740 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649118c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95e11a2740 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107b80 0x557649111bc0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.516+0000 7f95e11a2740 1 --2- 172.21.15.105:0/1490789120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107190 0x557649111680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.517+0000 7f95e11a2740 1 -- 172.21.15.105:0/1490789120 >> 172.21.15.105:0/1490789120 conn(0x557649102700 msgr2=0x557649101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.517+0000 7f95b77fe640 0 client.18711 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.517+0000 7f95e11a2740 1 -- 172.21.15.105:0/1490789120 shutdown_connections 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.517+0000 7f95e11a2740 1 -- 172.21.15.105:0/1490789120 wait complete. 2024-03-26T17:47:15.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.517+0000 7f95e11a2740 20 client.18711 trim_cache size 0 max 16384 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.539+0000 7f95e11a2740 1 Processor -- start 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.539+0000 7f95e11a2740 1 -- start start 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.539+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649106070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.539+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649118bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.539+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649123bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.539+0000 7f95e11a2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ab040 con 0x557649107190 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95e11a2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81acb80 con 0x55764910ccf0 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95e11a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ab710 con 0x557649107b80 2024-03-26T17:47:15.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649106070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649106070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60900/0 (socket says 172.21.15.105:60900) 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 -- 172.21.15.105:0/2165458573 learned_addr learned my addr 172.21.15.105:0/2165458573 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95dca79640 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649123bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d77fe640 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649118bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 -- 172.21.15.105:0/2165458573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 msgr2=0x557649118bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649118bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 -- 172.21.15.105:0/2165458573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 msgr2=0x557649123bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649123bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 -- 172.21.15.105:0/2165458573 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557649109720 con 0x557649107190 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95d7fff640 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649106070 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000c00 tx=0x7f95c800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e8939404c01d53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95bdffb640 1 -- 172.21.15.105:0/2165458573 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8026070 con 0x557649107190 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95bdffb640 1 -- 172.21.15.105:0/2165458573 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95c8011440 con 0x557649107190 2024-03-26T17:47:15.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.540+0000 7f95bdffb640 1 -- 172.21.15.105:0/2165458573 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c801d650 con 0x557649107190 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 -- 172.21.15.105:0/2165458573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 msgr2=0x557649106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649106070 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000c00 tx=0x7f95c800b410 comp rx=0 tx=0).stop 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 -- 172.21.15.105:0/2165458573 shutdown_connections 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649123bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649118bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 --2- 172.21.15.105:0/2165458573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649106070 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 -- 172.21.15.105:0/2165458573 >> 172.21.15.105:0/2165458573 conn(0x557649102700 msgr2=0x557649105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 -- 172.21.15.105:0/2165458573 shutdown_connections 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 -- 172.21.15.105:0/2165458573 wait complete. 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.541+0000 7f95e11a2740 1 -- start start 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95e11a2740 1 Processor -- start 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649118b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649117230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95e11a2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649117860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95e11a2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ac4b0 con 0x557649107190 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95e11a2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81abde0 con 0x55764910ccf0 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95e11a2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81a9020 con 0x557649107b80 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95e11a2740 10 client.0 _ll_register_callbacks cb 0x7f95c800bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649117230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649118b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95dca79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649117860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95d77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649117230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53798/0 (socket says 172.21.15.105:53798) 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95d77fe640 1 -- 172.21.15.105:0/3044234274 learned_addr learned my addr 172.21.15.105:0/3044234274 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95d77fe640 1 -- 172.21.15.105:0/3044234274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 msgr2=0x557649117860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95d77fe640 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649117860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95d77fe640 1 -- 172.21.15.105:0/3044234274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 msgr2=0x557649118b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.542+0000 7f95d77fe640 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649118b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95d77fe640 1 -- 172.21.15.105:0/3044234274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557648d40170 con 0x557649107b80 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95dca79640 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649117860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95d77fe640 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649117230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0cd4c0 tx=0x7f95cc048cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87827da1f2048d5f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95b77fe640 1 -- 172.21.15.105:0/3044234274 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc097e40 con 0x557649107b80 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95b77fe640 1 -- 172.21.15.105:0/3044234274 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f95cc0983f0 con 0x557649107b80 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95e11a2740 1 -- 172.21.15.105:0/3044234274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c8011440 con 0x557649107b80 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95b77fe640 1 -- 172.21.15.105:0/3044234274 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95cc09bd80 con 0x557649107b80 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95e11a2740 10 client.28552 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95e11a2740 1 -- 172.21.15.105:0/3044234274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557648d40170 con 0x557649107b80 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95b77fe640 1 -- 172.21.15.105:0/3044234274 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95c8011440 con 0x557649107b80 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.543+0000 7f95b77fe640 1 -- 172.21.15.105:0/3044234274 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f95cc09b2b0 con 0x557649107b80 2024-03-26T17:47:15.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95e11a2740 10 client.28552 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95e11a2740 10 client.28552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95e11a2740 20 client.28552 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95e11a2740 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557649184770 0x557649186b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95e11a2740 4 client.28552 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95e11a2740 1 -- 172.21.15.105:0/3044234274 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557649073ec0 con 0x557649184770 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95d7fff640 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557649184770 0x557649186b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95b77fe640 10 client.28552 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.544+0000 7f95d7fff640 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557649184770 0x557649186b90 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283bd64fbff74e0c server_cookie=807ad6f87f859caa in_seq=0 out_seq=0 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.545+0000 7f95b77fe640 1 -- 172.21.15.105:0/3044234274 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557649073ec0 con 0x557649184770 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.545+0000 7f95b77fe640 10 client.28552 handle_command_reply: tid=0 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.545+0000 7f95e11a2740 1 client.28552 shutdown 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 -- 172.21.15.105:0/3044234274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 msgr2=0x557649117230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649117230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0cd4c0 tx=0x7f95cc048cd0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 -- 172.21.15.105:0/3044234274 shutdown_connections 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557649184770 0x557649186b90 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55764910ccf0 0x557649117860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557649107b80 0x557649117230 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 --2- 172.21.15.105:0/3044234274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557649107190 0x557649118b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 -- 172.21.15.105:0/3044234274 >> 172.21.15.105:0/3044234274 conn(0x557649102700 msgr2=0x5576491061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95b77fe640 0 client.28552 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 -- 172.21.15.105:0/3044234274 shutdown_connections 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 1 -- 172.21.15.105:0/3044234274 wait complete. 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95e11a2740 20 client.28552 trim_cache size 0 max 16384 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:47:15.543 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:47:15.544 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "ffd03405-5895-4ab3-8ef4-12f9afc27b0c", 2024-03-26T17:47:15.544 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:47:15.544 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:47:15.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95de4dc640 1 -- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c00a2f20 msgr2=0x7f95c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95de4dc640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c00a2f20 0x7f95c00a53e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f95d0004af0 tx=0x7f95d0033780 comp rx=0 tx=0).stop 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95de4dc640 1 -- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d8102ac0 msgr2=0x7f95d8073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.546+0000 7f95de4dc640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d8102ac0 0x7f95d8073cb0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00ea60 tx=0x7f95cc00ef20 comp rx=0 tx=0).stop 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.547+0000 7f95de4dc640 1 -- 172.21.15.105:0/2109166481 shutdown_connections 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.547+0000 7f95de4dc640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c00a2f20 0x7f95c00a53e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.547+0000 7f95de4dc640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95d8103480 0x7f95d80741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.547+0000 7f95de4dc640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95d8102ac0 0x7f95d8073cb0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.547+0000 7f95de4dc640 1 --2- 172.21.15.105:0/2109166481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95d81018d0 0x7f95d8075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.547+0000 7f95de4dc640 1 -- 172.21.15.105:0/2109166481 >> 172.21.15.105:0/2109166481 conn(0x7f95d80fce40 msgr2=0x7f95d8105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.547+0000 7f95de4dc640 1 -- 172.21.15.105:0/2109166481 shutdown_connections 2024-03-26T17:47:15.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.547+0000 7f95de4dc640 1 -- 172.21.15.105:0/2109166481 wait complete. 2024-03-26T17:47:15.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:47:15.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 -- 172.21.15.105:0/692187497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1007c0 msgr2=0x7f58bc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 --2- 172.21.15.105:0/692187497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1007c0 0x7f58bc100be0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009900 tx=0x7f58ac02bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 -- 172.21.15.105:0/692187497 shutdown_connections 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 --2- 172.21.15.105:0/692187497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58bc102370 0x7f58bc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 --2- 172.21.15.105:0/692187497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58bc1019b0 0x7f58bc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 --2- 172.21.15.105:0/692187497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1007c0 0x7f58bc100be0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 -- 172.21.15.105:0/692187497 >> 172.21.15.105:0/692187497 conn(0x7f58bc0fbd50 msgr2=0x7f58bc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 -- 172.21.15.105:0/692187497 shutdown_connections 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 -- 172.21.15.105:0/692187497 wait complete. 2024-03-26T17:47:15.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 Processor -- start 2024-03-26T17:47:15.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.676+0000 7f58c24ac640 1 -- start start 2024-03-26T17:47:15.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58c24ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58bc1007c0 0x7f58bc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58c24ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1019b0 0x7f58bc1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58c24ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58bc102370 0x7f58bc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58c24ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc10d410 con 0x7f58bc1019b0 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58c24ac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc10d290 con 0x7f58bc1007c0 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58c24ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc10d590 con 0x7f58bc102370 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1019b0 0x7f58bc1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1019b0 0x7f58bc1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60920/0 (socket says 172.21.15.105:60920) 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 -- 172.21.15.105:0/4057908084 learned_addr learned my addr 172.21.15.105:0/4057908084 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58c0a49640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58bc102370 0x7f58bc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bbfff640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58bc1007c0 0x7f58bc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 -- 172.21.15.105:0/4057908084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58bc102370 msgr2=0x7f58bc19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58bc102370 0x7f58bc19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 -- 172.21.15.105:0/4057908084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58bc1007c0 msgr2=0x7f58bc197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58bc1007c0 0x7f58bc197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 -- 172.21.15.105:0/4057908084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc19f780 con 0x7f58bc1019b0 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.677+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1019b0 0x7f58bc1980a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e9f0 tx=0x7f58b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.678+0000 7f58b97fa640 1 -- 172.21.15.105:0/4057908084 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b000cd80 con 0x7f58bc1019b0 2024-03-26T17:47:15.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.678+0000 7f58c24ac640 1 -- 172.21.15.105:0/4057908084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58bc19fa70 con 0x7f58bc1019b0 2024-03-26T17:47:15.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.678+0000 7f58c24ac640 1 -- 172.21.15.105:0/4057908084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58bc1a0010 con 0x7f58bc1019b0 2024-03-26T17:47:15.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.678+0000 7f58b97fa640 1 -- 172.21.15.105:0/4057908084 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58b00108e0 con 0x7f58bc1019b0 2024-03-26T17:47:15.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.678+0000 7f58b97fa640 1 -- 172.21.15.105:0/4057908084 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0018920 con 0x7f58bc1019b0 2024-03-26T17:47:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.680+0000 7f58b97fa640 1 -- 172.21.15.105:0/4057908084 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f58b0010430 con 0x7f58bc1019b0 2024-03-26T17:47:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.680+0000 7f58c5171740 1 Processor -- start 2024-03-26T17:47:15.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.680+0000 7f58c5171740 1 -- start start 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c5171740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479cd00 0x55817479d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c5171740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479d6f0 0x5581747a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c5171740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 0x5581747a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c5171740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5581746fd980 con 0x55817479d6f0 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c5171740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5581746a5be0 con 0x5581747a2860 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c5171740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581746f9930 con 0x55817479cd00 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479d6f0 0x5581747a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58b97fa640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58a40a2f20 0x7f58a40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479d6f0 0x5581747a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60922/0 (socket says 172.21.15.105:60922) 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bbfff640 1 -- 172.21.15.105:0/2458376204 learned_addr learned my addr 172.21.15.105:0/2458376204 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c0a49640 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479cd00 0x55817479d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 0x5581747a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c0a49640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58a40a2f20 0x7f58a40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bbfff640 1 -- 172.21.15.105:0/2458376204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479cd00 msgr2=0x55817479d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bbfff640 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479cd00 0x55817479d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bbfff640 1 -- 172.21.15.105:0/2458376204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 msgr2=0x5581747a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bbfff640 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 0x5581747a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58bbfff640 1 -- 172.21.15.105:0/2458376204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581744db7d0 con 0x55817479d6f0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58c0a49640 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479cd00 0x55817479d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.681+0000 7f58b97fa640 1 -- 172.21.15.105:0/4057908084 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f58b00cf650 con 0x7f58bc1019b0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.682+0000 7f58bbfff640 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479d6f0 0x5581747a21f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009900 tx=0x7f58ac009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953ad55fbe34325e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.682+0000 7f58c0a49640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58a40a2f20 0x7f58a40a53e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55817479df50 tx=0x7f58b4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.682+0000 7f58a27fc640 1 -- 172.21.15.105:0/2458376204 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac045020 con 0x55817479d6f0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.682+0000 7f58a27fc640 1 -- 172.21.15.105:0/2458376204 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58ac004040 con 0x55817479d6f0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.682+0000 7f58a27fc640 1 -- 172.21.15.105:0/2458376204 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0055d0 con 0x55817479d6f0 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.682+0000 7f58c5171740 1 -- 172.21.15.105:0/2458376204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479d6f0 msgr2=0x5581747a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.682+0000 7f58c5171740 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479d6f0 0x5581747a21f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009900 tx=0x7f58ac009560 comp rx=0 tx=0).stop 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 -- 172.21.15.105:0/2458376204 shutdown_connections 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 0x5581747a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479d6f0 0x5581747a21f0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 --2- 172.21.15.105:0/2458376204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479cd00 0x55817479d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 -- 172.21.15.105:0/2458376204 >> 172.21.15.105:0/2458376204 conn(0x558174793e60 msgr2=0x558174794260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 -- 172.21.15.105:0/2458376204 shutdown_connections 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 -- 172.21.15.105:0/2458376204 wait complete. 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 -- start start 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 Processor -- start 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479cd00 0x5581747a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 0x5581747a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581747a2860 0x5581747ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5581747a57b0 con 0x5581747a2860 2024-03-26T17:47:15.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5581746b9160 con 0x55817479cd00 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5581747a5930 con 0x55817479d6f0 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.683+0000 7f58c5171740 10 client.0 _ll_register_callbacks cb 0x5581746f5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479cd00 0x5581747a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58c0a49640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581747a2860 0x5581747ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479cd00 0x5581747a6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59884/0 (socket says 172.21.15.105:59884) 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58c0a49640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581747a2860 0x5581747ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60928/0 (socket says 172.21.15.105:60928) 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 -- 172.21.15.105:0/785856407 learned_addr learned my addr 172.21.15.105:0/785856407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 0x5581747a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 -- 172.21.15.105:0/785856407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 msgr2=0x5581747a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 0x5581747a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 -- 172.21.15.105:0/785856407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581747a2860 msgr2=0x5581747ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581747a2860 0x5581747ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 -- 172.21.15.105:0/785856407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581744113f0 con 0x55817479cd00 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58c0a49640 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581747a2860 0x5581747ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:15.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58bbfff640 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479cd00 0x5581747a6ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f58ac02c170 tx=0x7f58ac005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5630467e1e08396d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f589b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f589b7fe640 1 -- 172.21.15.105:0/785856407 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac04e070 con 0x55817479cd00 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f589b7fe640 1 -- 172.21.15.105:0/785856407 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f58ac0035f0 con 0x55817479cd00 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.684+0000 7f58c5171740 1 -- 172.21.15.105:0/785856407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58ac004040 con 0x55817479cd00 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f589b7fe640 1 -- 172.21.15.105:0/785856407 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58ac003810 con 0x55817479cd00 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58c5171740 10 client.28562 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58c5171740 1 -- 172.21.15.105:0/785856407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5581743d10f0 con 0x55817479cd00 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f589b7fe640 1 -- 172.21.15.105:0/785856407 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58ac058030 con 0x55817479cd00 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f589b7fe640 1 -- 172.21.15.105:0/785856407 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f58ac004c20 con 0x55817479cd00 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58c5171740 10 client.28562 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58c5171740 10 client.28562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58c5171740 20 client.28562 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58c5171740 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5581747b2950 0x5581747b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58c5171740 4 client.28562 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:15.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58c5171740 1 -- 172.21.15.105:0/785856407 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f58ac004040 con 0x5581747b2950 2024-03-26T17:47:15.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.685+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5581747b2950 0x5581747b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.686+0000 7f589b7fe640 10 client.28562 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:15.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.686+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5581747b2950 0x5581747b4d80 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848b214502404072 server_cookie=7fca09f45380c980 in_seq=0 out_seq=0 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f589b7fe640 1 -- 172.21.15.105:0/785856407 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f58ac004040 con 0x5581747b2950 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f589b7fe640 10 client.28562 handle_command_reply: tid=0 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 client.28562 shutdown 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 -- 172.21.15.105:0/785856407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479cd00 msgr2=0x5581747a6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479cd00 0x5581747a6ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f58ac02c170 tx=0x7f58ac005730 comp rx=0 tx=0).stop 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 -- 172.21.15.105:0/785856407 shutdown_connections 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5581747b2950 0x5581747b4d80 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5581747a2860 0x5581747ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 0x5581747a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f589b7fe640 0 client.28562 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:15.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 --2- 172.21.15.105:0/785856407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479cd00 0x5581747a6ee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 -- 172.21.15.105:0/785856407 >> 172.21.15.105:0/785856407 conn(0x558174793e60 msgr2=0x558174793080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 -- 172.21.15.105:0/785856407 shutdown_connections 2024-03-26T17:47:15.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 1 -- 172.21.15.105:0/785856407 wait complete. 2024-03-26T17:47:15.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.687+0000 7f58c5171740 20 client.28562 trim_cache size 0 max 16384 2024-03-26T17:47:15.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.720+0000 7f58c5171740 1 Processor -- start 2024-03-26T17:47:15.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58c5171740 1 -- start start 2024-03-26T17:47:15.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58c5171740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747b9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58c5171740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479d6f0 0x5581747ae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58c5171740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581747a2860 0x5581747b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58c5171740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58a800e530 con 0x55817479cd00 2024-03-26T17:47:15.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58c5171740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8010070 con 0x55817479d6f0 2024-03-26T17:47:15.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58c5171740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58a800ec00 con 0x5581747a2860 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479d6f0 0x5581747ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479d6f0 0x5581747ae420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59900/0 (socket says 172.21.15.105:59900) 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747b9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60942/0 (socket says 172.21.15.105:60942) 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 -- 172.21.15.105:0/1076537762 learned_addr learned my addr 172.21.15.105:0/1076537762 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58c0a49640 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581747a2860 0x5581747b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 -- 172.21.15.105:0/1076537762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581747a2860 msgr2=0x5581747b9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581747a2860 0x5581747b9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 -- 172.21.15.105:0/1076537762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 msgr2=0x5581747b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 -- 172.21.15.105:0/1076537762 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558174794dd0 con 0x55817479d6f0 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bbfff640 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.721+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479d6f0 0x5581747ae420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c6a0 tx=0x7f58b00ccfe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3354774b0f9d90 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.722+0000 7f58a1ffb640 1 -- 172.21.15.105:0/1076537762 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b00a4750 con 0x55817479d6f0 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.722+0000 7f58a1ffb640 1 -- 172.21.15.105:0/1076537762 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58b0099e90 con 0x55817479d6f0 2024-03-26T17:47:15.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.722+0000 7f58a1ffb640 1 -- 172.21.15.105:0/1076537762 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b0097b40 con 0x55817479d6f0 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.722+0000 7f58c5171740 1 -- 172.21.15.105:0/1076537762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479d6f0 msgr2=0x5581747ae420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.722+0000 7f58c5171740 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479d6f0 0x5581747ae420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f58b000c6a0 tx=0x7f58b00ccfe0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 -- 172.21.15.105:0/1076537762 shutdown_connections 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5581747a2860 0x5581747b9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55817479d6f0 0x5581747ae420 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 --2- 172.21.15.105:0/1076537762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 -- 172.21.15.105:0/1076537762 >> 172.21.15.105:0/1076537762 conn(0x558174793e60 msgr2=0x558174797950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 -- 172.21.15.105:0/1076537762 shutdown_connections 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 -- 172.21.15.105:0/1076537762 wait complete. 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 -- start start 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 Processor -- start 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 0x5581747ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 0x5581747ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58a800f9a0 con 0x55817479cd00 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58a800f2d0 con 0x5581747a2860 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58a800c510 con 0x55817479d6f0 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.723+0000 7f58c5171740 10 client.0 _ll_register_callbacks cb 0x7f58b009bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747abc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60944/0 (socket says 172.21.15.105:60944) 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 -- 172.21.15.105:0/3721811193 learned_addr learned my addr 172.21.15.105:0/3721811193 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:15.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58c0a49640 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 0x5581747ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 0x5581747ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 -- 172.21.15.105:0/3721811193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 msgr2=0x5581747ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 0x5581747ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 -- 172.21.15.105:0/3721811193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 msgr2=0x5581747ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 0x5581747ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 -- 172.21.15.105:0/3721811193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581747ad730 con 0x55817479cd00 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58bbfff640 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747abc60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009900 tx=0x7f58ac0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6938e026855b067 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f589b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f589b7fe640 1 -- 172.21.15.105:0/3721811193 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac04e040 con 0x55817479cd00 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f589b7fe640 1 -- 172.21.15.105:0/3721811193 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f58ac0419d0 con 0x55817479cd00 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58c5171740 1 -- 172.21.15.105:0/3721811193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58b0099e90 con 0x55817479cd00 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f589b7fe640 1 -- 172.21.15.105:0/3721811193 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58ac045020 con 0x55817479cd00 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58c5171740 10 client.18735 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.724+0000 7f58c5171740 1 -- 172.21.15.105:0/3721811193 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558174818e20 con 0x55817479cd00 2024-03-26T17:47:15.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f589b7fe640 1 -- 172.21.15.105:0/3721811193 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58b0099e90 con 0x55817479cd00 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f589b7fe640 1 -- 172.21.15.105:0/3721811193 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f58ac0046c0 con 0x55817479cd00 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f58c5171740 10 client.18735 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f58c5171740 10 client.18735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f58c5171740 20 client.18735 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f58c5171740 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558174819870 0x55817481bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f58c5171740 4 client.18735 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f58c5171740 1 -- 172.21.15.105:0/3721811193 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55817470aec0 con 0x558174819870 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558174819870 0x55817481bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f589b7fe640 10 client.18735 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.725+0000 7f58bb7fe640 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558174819870 0x55817481bc90 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d80a2163a217274 server_cookie=f1144d3c1367d0c4 in_seq=0 out_seq=0 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.726+0000 7f589b7fe640 1 -- 172.21.15.105:0/3721811193 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55817470aec0 con 0x558174819870 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.726+0000 7f589b7fe640 10 client.18735 handle_command_reply: tid=0 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.726+0000 7f58c5171740 1 client.18735 shutdown 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 -- 172.21.15.105:0/3721811193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 msgr2=0x5581747abc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747abc60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009900 tx=0x7f58ac0045b0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 -- 172.21.15.105:0/3721811193 shutdown_connections 2024-03-26T17:47:15.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558174819870 0x55817481bc90 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5581747a2860 0x5581747ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55817479d6f0 0x5581747ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 --2- 172.21.15.105:0/3721811193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55817479cd00 0x5581747abc60 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f589b7fe640 0 client.18735 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 -- 172.21.15.105:0/3721811193 >> 172.21.15.105:0/3721811193 conn(0x558174793e60 msgr2=0x5581747927f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 -- 172.21.15.105:0/3721811193 shutdown_connections 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 1 -- 172.21.15.105:0/3721811193 wait complete. 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c5171740 20 client.18735 trim_cache size 0 max 16384 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:47:15.725 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:47:15.726 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:47:15.726 INFO:teuthology.orchestra.run.smithi105.stdout: "ffd03405-5895-4ab3-8ef4-12f9afc27b0c": { 2024-03-26T17:47:15.726 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:47:15.726 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "ffd03405-5895-4ab3-8ef4-12f9afc27b0c", 2024-03-26T17:47:15.726 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:47:15.726 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:47:15.726 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:47:15.726 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:47:15.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c24ac640 1 -- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58a40a2f20 msgr2=0x7f58a40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c24ac640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58a40a2f20 0x7f58a40a53e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55817479df50 tx=0x7f58b4004ab0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c24ac640 1 -- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1019b0 msgr2=0x7f58bc1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:15.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.727+0000 7f58c24ac640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1019b0 0x7f58bc1980a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e9f0 tx=0x7f58b000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.728+0000 7f58c24ac640 1 -- 172.21.15.105:0/4057908084 shutdown_connections 2024-03-26T17:47:15.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.728+0000 7f58c24ac640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58a40a2f20 0x7f58a40a53e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.728+0000 7f58c24ac640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58bc102370 0x7f58bc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.728+0000 7f58c24ac640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58bc1019b0 0x7f58bc1980a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.728+0000 7f58c24ac640 1 --2- 172.21.15.105:0/4057908084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58bc1007c0 0x7f58bc197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:15.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.728+0000 7f58c24ac640 1 -- 172.21.15.105:0/4057908084 >> 172.21.15.105:0/4057908084 conn(0x7f58bc0fbd50 msgr2=0x7f58bc104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:15.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.728+0000 7f58c24ac640 1 -- 172.21.15.105:0/4057908084 shutdown_connections 2024-03-26T17:47:15.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:15.728+0000 7f58c24ac640 1 -- 172.21.15.105:0/4057908084 wait complete. 2024-03-26T17:47:15.731 INFO:tasks.cephfs.filesystem:scrub status for tag:ffd03405-5895-4ab3-8ef4-12f9afc27b0c - {'path': '/', 'tag': 'ffd03405-5895-4ab3-8ef4-12f9afc27b0c', 'options': 'recursive,force'} 2024-03-26T17:47:16.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 281289 3.15 MB/sec execute 359 sec latency 24.482 ms 2024-03-26T17:47:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:15 smithi144 ceph-mon[36095]: pgmap v615: 97 pgs: 97 active+clean; 30 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:47:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:15 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:47:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:15 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:15 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:47:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:15 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:15 smithi105 ceph-mon[32280]: pgmap v615: 97 pgs: 97 active+clean; 30 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:47:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:15 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:47:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:15 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:15 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:47:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:15 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:15 smithi135 ceph-mon[35636]: pgmap v615: 97 pgs: 97 active+clean; 30 MiB data, 943 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-03-26T17:47:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:15 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:47:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:15 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:16.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:15 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:47:16.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:15 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:17.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 281928 3.15 MB/sec execute 360 sec latency 15.975 ms 2024-03-26T17:47:17.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:16 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:47:17.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:16 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:47:17.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:16 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:47:18.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 282567 3.14 MB/sec execute 361 sec latency 54.781 ms 2024-03-26T17:47:18.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:17 smithi144 ceph-mon[36095]: pgmap v616: 97 pgs: 97 active+clean; 27 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-03-26T17:47:18.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:17 smithi105 ceph-mon[32280]: pgmap v616: 97 pgs: 97 active+clean; 27 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-03-26T17:47:18.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:17 smithi135 ceph-mon[35636]: pgmap v616: 97 pgs: 97 active+clean; 27 MiB data, 939 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-03-26T17:47:19.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 283242 3.15 MB/sec execute 362 sec latency 16.988 ms 2024-03-26T17:47:20.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 283817 3.16 MB/sec execute 363 sec latency 16.470 ms 2024-03-26T17:47:20.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:19 smithi144 ceph-mon[36095]: pgmap v617: 97 pgs: 97 active+clean; 26 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-03-26T17:47:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:19 smithi105 ceph-mon[32280]: pgmap v617: 97 pgs: 97 active+clean; 26 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-03-26T17:47:20.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:19 smithi135 ceph-mon[35636]: pgmap v617: 97 pgs: 97 active+clean; 26 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-03-26T17:47:21.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 284255 3.15 MB/sec execute 364 sec latency 14.976 ms 2024-03-26T17:47:22.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 284731 3.15 MB/sec execute 365 sec latency 29.929 ms 2024-03-26T17:47:22.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:21 smithi144 ceph-mon[36095]: pgmap v618: 97 pgs: 97 active+clean; 26 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:47:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:21 smithi105 ceph-mon[32280]: pgmap v618: 97 pgs: 97 active+clean; 26 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:47:22.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:21 smithi135 ceph-mon[35636]: pgmap v618: 97 pgs: 97 active+clean; 26 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.4 MiB/s wr, 112 op/s 2024-03-26T17:47:23.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 285396 3.15 MB/sec execute 366 sec latency 15.790 ms 2024-03-26T17:47:24.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 286090 3.14 MB/sec execute 367 sec latency 16.520 ms 2024-03-26T17:47:24.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:23 smithi144 ceph-mon[36095]: pgmap v619: 97 pgs: 97 active+clean; 27 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-03-26T17:47:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:23 smithi105 ceph-mon[32280]: pgmap v619: 97 pgs: 97 active+clean; 27 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-03-26T17:47:24.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:23 smithi135 ceph-mon[35636]: pgmap v619: 97 pgs: 97 active+clean; 27 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-03-26T17:47:25.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 286720 3.15 MB/sec execute 368 sec latency 17.653 ms 2024-03-26T17:47:26.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 287316 3.16 MB/sec execute 369 sec latency 17.058 ms 2024-03-26T17:47:26.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:25 smithi144 ceph-mon[36095]: pgmap v620: 97 pgs: 97 active+clean; 28 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:47:26.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:25 smithi105 ceph-mon[32280]: pgmap v620: 97 pgs: 97 active+clean; 28 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:47:26.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:25 smithi135 ceph-mon[35636]: pgmap v620: 97 pgs: 97 active+clean; 28 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:47:27.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 287783 3.15 MB/sec execute 370 sec latency 13.777 ms 2024-03-26T17:47:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 -- 172.21.15.105:0/2213946460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b8104c90 msgr2=0x7f47b8107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/2213946460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b8104c90 0x7f47b8107080 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009900 tx=0x7f47a802bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 -- 172.21.15.105:0/2213946460 shutdown_connections 2024-03-26T17:47:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/2213946460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b81075c0 0x7f47b8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/2213946460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b8104c90 0x7f47b8107080 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/2213946460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47b8102360 0x7f47b8104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 -- 172.21.15.105:0/2213946460 >> 172.21.15.105:0/2213946460 conn(0x7f47b80fbd50 msgr2=0x7f47b80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 -- 172.21.15.105:0/2213946460 shutdown_connections 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.958+0000 7f47bf8ff640 1 -- 172.21.15.105:0/2213946460 wait complete. 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bf8ff640 1 Processor -- start 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bf8ff640 1 -- start start 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bf8ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47b8102360 0x7f47b8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bf8ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b8104c90 0x7f47b8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bf8ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81075c0 0x7f47b819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bf8ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47b810d410 con 0x7f47b81075c0 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bf8ff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47b810d290 con 0x7f47b8102360 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bf8ff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47b810d590 con 0x7f47b8104c90 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bd69b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47b8102360 0x7f47b8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bde9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81075c0 0x7f47b819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bce9a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b8104c90 0x7f47b8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bd69b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47b8102360 0x7f47b8197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34038/0 (socket says 172.21.15.105:34038) 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.959+0000 7f47bd69b640 1 -- 172.21.15.105:0/1545420714 learned_addr learned my addr 172.21.15.105:0/1545420714 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bde9c640 1 -- 172.21.15.105:0/1545420714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b8104c90 msgr2=0x7f47b8197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bde9c640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b8104c90 0x7f47b8197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bde9c640 1 -- 172.21.15.105:0/1545420714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47b8102360 msgr2=0x7f47b8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bde9c640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47b8102360 0x7f47b8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bde9c640 1 -- 172.21.15.105:0/1545420714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b819f6b0 con 0x7f47b81075c0 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bce9a640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b8104c90 0x7f47b8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bd69b640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47b8102360 0x7f47b8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bde9c640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81075c0 0x7f47b819f040 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009fd0 tx=0x7f47b000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47ae7fc640 1 -- 172.21.15.105:0/1545420714 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b000ce20 con 0x7f47b81075c0 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47ae7fc640 1 -- 172.21.15.105:0/1545420714 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47b0005020 con 0x7f47b81075c0 2024-03-26T17:47:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bf8ff640 1 -- 172.21.15.105:0/1545420714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b819f9a0 con 0x7f47b81075c0 2024-03-26T17:47:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47ae7fc640 1 -- 172.21.15.105:0/1545420714 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0010690 con 0x7f47b81075c0 2024-03-26T17:47:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.960+0000 7f47bf8ff640 1 -- 172.21.15.105:0/1545420714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b819fec0 con 0x7f47b81075c0 2024-03-26T17:47:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.963+0000 7f47ae7fc640 1 -- 172.21.15.105:0/1545420714 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f47b00051d0 con 0x7f47b81075c0 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.963+0000 7f47ae7fc640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47a40a2fc0 0x7f47a40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.964+0000 7f47bd69b640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47a40a2fc0 0x7f47a40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.964+0000 7f47ae7fc640 1 -- 172.21.15.105:0/1545420714 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f47b00cfb60 con 0x7f47b81075c0 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.964+0000 7f47bd69b640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47a40a2fc0 0x7f47a40a5480 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47b4011550 tx=0x7f47b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.964+0000 7f47c25c9740 1 Processor -- start 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.964+0000 7f47c25c9740 1 -- start start 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800d5d0 0x55b69800d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800dfc0 0x55b698012ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47c25c9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b697f720b0 con 0x55b69800dfc0 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47c25c9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b697f1a960 con 0x55b698013130 2024-03-26T17:47:27.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47c25c9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b697f6e060 con 0x55b69800d5d0 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800dfc0 0x55b698012ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800dfc0 0x55b698012ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40658/0 (socket says 172.21.15.105:40658) 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 -- 172.21.15.105:0/4122836974 learned_addr learned my addr 172.21.15.105:0/4122836974 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bd69b640 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800d5d0 0x55b69800d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bde9c640 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 -- 172.21.15.105:0/4122836974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800d5d0 msgr2=0x55b69800d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800d5d0 0x55b69800d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 -- 172.21.15.105:0/4122836974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 msgr2=0x55b6980155c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980155c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 -- 172.21.15.105:0/4122836974 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b697f2cc60 con 0x55b69800dfc0 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bd69b640 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800d5d0 0x55b69800d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bde9c640 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47bce9a640 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800dfc0 0x55b698012ac0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f47b8069aa0 tx=0x7f47a80095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e16de42465a625e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47a37fe640 1 -- 172.21.15.105:0/4122836974 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8045020 con 0x55b69800dfc0 2024-03-26T17:47:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47a37fe640 1 -- 172.21.15.105:0/4122836974 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a802cc90 con 0x55b69800dfc0 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.965+0000 7f47a37fe640 1 -- 172.21.15.105:0/4122836974 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8004920 con 0x55b69800dfc0 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 -- 172.21.15.105:0/4122836974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800dfc0 msgr2=0x55b698012ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800dfc0 0x55b698012ac0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f47b8069aa0 tx=0x7f47a80095f0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 -- 172.21.15.105:0/4122836974 shutdown_connections 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800dfc0 0x55b698012ac0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 --2- 172.21.15.105:0/4122836974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800d5d0 0x55b69800d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 -- 172.21.15.105:0/4122836974 >> 172.21.15.105:0/4122836974 conn(0x55b698008b40 msgr2=0x55b698008f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 -- 172.21.15.105:0/4122836974 shutdown_connections 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.966+0000 7f47c25c9740 1 -- 172.21.15.105:0/4122836974 wait complete. 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 1 -- start start 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 1 Processor -- start 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b698017300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b698017840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b69801e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b698016080 con 0x55b69800d5d0 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b697f2ddb0 con 0x55b698013130 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b698016200 con 0x55b69800dfc0 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47c25c9740 10 client.0 _ll_register_callbacks cb 0x55b697f32110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47bd69b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b698017300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47bde9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b69801e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47bd69b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b698017300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40660/0 (socket says 172.21.15.105:40660) 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47bd69b640 1 -- 172.21.15.105:0/2259202734 learned_addr learned my addr 172.21.15.105:0/2259202734 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.967+0000 7f47bce9a640 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b698017840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47bce9a640 1 -- 172.21.15.105:0/2259202734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 msgr2=0x55b69801e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47bce9a640 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b69801e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47bce9a640 1 -- 172.21.15.105:0/2259202734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 msgr2=0x55b698017300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47bce9a640 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b698017300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47bce9a640 1 -- 172.21.15.105:0/2259202734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b697c86210 con 0x55b69800dfc0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47bde9c640 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b69801e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47bce9a640 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b698017840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b698009ab0 tx=0x7f47a802c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e271d77bb7e3a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47a0ff9640 1 -- 172.21.15.105:0/2259202734 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8050020 con 0x55b69800dfc0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47c25c9740 1 -- 172.21.15.105:0/2259202734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a802cc90 con 0x55b69800dfc0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47a0ff9640 1 -- 172.21.15.105:0/2259202734 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f47a8003bc0 con 0x55b69800dfc0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47a0ff9640 1 -- 172.21.15.105:0/2259202734 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47a805b050 con 0x55b69800dfc0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47c25c9740 10 client.28582 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.968+0000 7f47c25c9740 1 -- 172.21.15.105:0/2259202734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b697d52620 con 0x55b69800dfc0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47a0ff9640 1 -- 172.21.15.105:0/2259202734 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a802cc90 con 0x55b69800dfc0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47a0ff9640 1 -- 172.21.15.105:0/2259202734 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f47a804c460 con 0x55b69800dfc0 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47c25c9740 10 client.28582 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47c25c9740 10 client.28582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47c25c9740 20 client.28582 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47c25c9740 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b698022e10 0x55b698025200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47c25c9740 4 client.28582 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47c25c9740 1 -- 172.21.15.105:0/2259202734 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b697f7f5f0 con 0x55b698022e10 2024-03-26T17:47:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.969+0000 7f47bd69b640 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b698022e10 0x55b698025200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.970+0000 7f47bd69b640 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b698022e10 0x55b698025200 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3b5448394bc05 server_cookie=be192904d3463aa1 in_seq=0 out_seq=0 2024-03-26T17:47:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.970+0000 7f47a0ff9640 10 client.28582 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47a0ff9640 1 -- 172.21.15.105:0/2259202734 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b697f7f5f0 con 0x55b698022e10 2024-03-26T17:47:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47a0ff9640 10 client.28582 handle_command_reply: tid=0 2024-03-26T17:47:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 client.28582 shutdown 2024-03-26T17:47:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 -- 172.21.15.105:0/2259202734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 msgr2=0x55b698017840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b698017840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b698009ab0 tx=0x7f47a802c330 comp rx=0 tx=0).stop 2024-03-26T17:47:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 -- 172.21.15.105:0/2259202734 shutdown_connections 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b698022e10 0x55b698025200 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b69801e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b698017840 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47a0ff9640 0 client.28582 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 --2- 172.21.15.105:0/2259202734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b698017300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 -- 172.21.15.105:0/2259202734 >> 172.21.15.105:0/2259202734 conn(0x55b698008b40 msgr2=0x55b698007df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 -- 172.21.15.105:0/2259202734 shutdown_connections 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 1 -- 172.21.15.105:0/2259202734 wait complete. 2024-03-26T17:47:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.972+0000 7f47c25c9740 20 client.28582 trim_cache size 0 max 16384 2024-03-26T17:47:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.988+0000 7f47c25c9740 1 Processor -- start 2024-03-26T17:47:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.988+0000 7f47c25c9740 1 -- start start 2024-03-26T17:47:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.988+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69800e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.988+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b69800dfc0 0x55b69801e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.988+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b698013130 0x55b69800ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.988+0000 7f47c25c9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a8c80 con 0x55b69800d5d0 2024-03-26T17:47:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.988+0000 7f47c25c9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81aa7c0 con 0x55b69800dfc0 2024-03-26T17:47:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.988+0000 7f47c25c9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a9350 con 0x55b698013130 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69800e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bce9a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b69800dfc0 0x55b69801e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69800e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40666/0 (socket says 172.21.15.105:40666) 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 -- 172.21.15.105:0/3474261599 learned_addr learned my addr 172.21.15.105:0/3474261599 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bde9c640 1 --2- 172.21.15.105:0/3474261599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b698013130 0x55b69800ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 -- 172.21.15.105:0/3474261599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b698013130 msgr2=0x55b69800ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 --2- 172.21.15.105:0/3474261599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b698013130 0x55b69800ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 -- 172.21.15.105:0/3474261599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b69800dfc0 msgr2=0x55b69801e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 --2- 172.21.15.105:0/3474261599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b69800dfc0 0x55b69801e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 -- 172.21.15.105:0/3474261599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b698004450 con 0x55b69800d5d0 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47bd69b640 1 --2- 172.21.15.105:0/3474261599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69800e910 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f47b4011de0 tx=0x7f47b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57628dff31282a97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47a2ffd640 1 -- 172.21.15.105:0/3474261599 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b401cba0 con 0x55b69800d5d0 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47a2ffd640 1 -- 172.21.15.105:0/3474261599 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47b401cd40 con 0x55b69800d5d0 2024-03-26T17:47:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.989+0000 7f47a2ffd640 1 -- 172.21.15.105:0/3474261599 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4004e50 con 0x55b69800d5d0 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 -- 172.21.15.105:0/3474261599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 msgr2=0x55b69800e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 --2- 172.21.15.105:0/3474261599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69800e910 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f47b4011de0 tx=0x7f47b4014000 comp rx=0 tx=0).stop 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 -- 172.21.15.105:0/3474261599 shutdown_connections 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 --2- 172.21.15.105:0/3474261599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b698013130 0x55b69800ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 --2- 172.21.15.105:0/3474261599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b69800dfc0 0x55b69801e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 --2- 172.21.15.105:0/3474261599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69800e910 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 -- 172.21.15.105:0/3474261599 >> 172.21.15.105:0/3474261599 conn(0x55b698008b40 msgr2=0x55b698017300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 -- 172.21.15.105:0/3474261599 shutdown_connections 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 -- 172.21.15.105:0/3474261599 wait complete. 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.990+0000 7f47c25c9740 1 -- start start 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47c25c9740 1 Processor -- start 2024-03-26T17:47:27.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69801b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b69801bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47c25c9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47c25c9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81aa0f0 con 0x55b69800d5d0 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47c25c9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a9a20 con 0x55b698013130 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47c25c9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47b81a6c60 con 0x55b69800dfc0 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47c25c9740 10 client.0 _ll_register_callbacks cb 0x7f47b0098490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bd69b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69801b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bd69b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69801b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40676/0 (socket says 172.21.15.105:40676) 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bde9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bd69b640 1 -- 172.21.15.105:0/1316384020 learned_addr learned my addr 172.21.15.105:0/1316384020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bce9a640 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b69801bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bd69b640 1 -- 172.21.15.105:0/1316384020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 msgr2=0x55b69801bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:27.987 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bd69b640 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b69801bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bd69b640 1 -- 172.21.15.105:0/1316384020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 msgr2=0x55b6980870f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bd69b640 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980870f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bd69b640 1 -- 172.21.15.105:0/1316384020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b698087630 con 0x55b69800d5d0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.991+0000 7f47bde9c640 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47bd69b640 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69801b760 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f47b400f7a0 tx=0x7f47b401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4317753565c34bbc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47a0ff9640 1 -- 172.21.15.105:0/1316384020 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4038070 con 0x55b69800d5d0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47a0ff9640 1 -- 172.21.15.105:0/1316384020 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f47b40046c0 con 0x55b69800d5d0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47c25c9740 1 -- 172.21.15.105:0/1316384020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47b401cd40 con 0x55b69800d5d0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47a0ff9640 1 -- 172.21.15.105:0/1316384020 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f47b40048c0 con 0x55b69800d5d0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47c25c9740 10 client.18753 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47c25c9740 1 -- 172.21.15.105:0/1316384020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b698088940 con 0x55b69800d5d0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47a0ff9640 1 -- 172.21.15.105:0/1316384020 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47b401cd40 con 0x55b69800d5d0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47a0ff9640 1 -- 172.21.15.105:0/1316384020 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f47b40348d0 con 0x55b69800d5d0 2024-03-26T17:47:27.988 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47c25c9740 10 client.18753 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47c25c9740 10 client.18753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47c25c9740 20 client.18753 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.992+0000 7f47c25c9740 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b698089320 0x55b69808b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.993+0000 7f47c25c9740 4 client.18753 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:47:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.993+0000 7f47c25c9740 1 -- 172.21.15.105:0/1316384020 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b697f79940 con 0x55b698089320 2024-03-26T17:47:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.993+0000 7f47bce9a640 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b698089320 0x55b69808b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.993+0000 7f47a0ff9640 10 client.18753 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:27.993+0000 7f47bce9a640 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b698089320 0x55b69808b740 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f0621849aef2e1c server_cookie=8ff87c10c552ef23 in_seq=0 out_seq=0 2024-03-26T17:47:28.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 288272 3.15 MB/sec execute 371 sec latency 43.216 ms 2024-03-26T17:47:28.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.044+0000 7f47a0ff9640 1 -- 172.21.15.105:0/1316384020 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b697f79940 con 0x55b698089320 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.044+0000 7f47a0ff9640 10 client.18753 handle_command_reply: tid=0 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.044+0000 7f47c25c9740 1 client.18753 shutdown 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.044+0000 7f47c25c9740 1 -- 172.21.15.105:0/1316384020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 msgr2=0x55b69801b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.044+0000 7f47c25c9740 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69801b760 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f47b400f7a0 tx=0x7f47b401c9a0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 1 -- 172.21.15.105:0/1316384020 shutdown_connections 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b698089320 0x55b69808b740 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47a0ff9640 0 client.18753 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b698013130 0x55b6980870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b69800dfc0 0x55b69801bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 1 --2- 172.21.15.105:0/1316384020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b69800d5d0 0x55b69801b760 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 1 -- 172.21.15.105:0/1316384020 >> 172.21.15.105:0/1316384020 conn(0x55b698008b40 msgr2=0x55b69800e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 1 -- 172.21.15.105:0/1316384020 shutdown_connections 2024-03-26T17:47:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 1 -- 172.21.15.105:0/1316384020 wait complete. 2024-03-26T17:47:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47c25c9740 20 client.18753 trim_cache size 0 max 16384 2024-03-26T17:47:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:47:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:47:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:47:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:47:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47bf8ff640 1 -- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47a40a2fc0 msgr2=0x7f47a40a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47a40a2fc0 0x7f47a40a5480 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f47b4011550 tx=0x7f47b4005f90 comp rx=0 tx=0).stop 2024-03-26T17:47:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47bf8ff640 1 -- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81075c0 msgr2=0x7f47b819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.045+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81075c0 0x7f47b819f040 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009fd0 tx=0x7f47b000ef10 comp rx=0 tx=0).stop 2024-03-26T17:47:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.046+0000 7f47bf8ff640 1 -- 172.21.15.105:0/1545420714 shutdown_connections 2024-03-26T17:47:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.046+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f47a40a2fc0 0x7f47a40a5480 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.046+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47b81075c0 0x7f47b819f040 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.046+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b8104c90 0x7f47b8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.046+0000 7f47bf8ff640 1 --2- 172.21.15.105:0/1545420714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47b8102360 0x7f47b8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.046+0000 7f47bf8ff640 1 -- 172.21.15.105:0/1545420714 >> 172.21.15.105:0/1545420714 conn(0x7f47b80fbd50 msgr2=0x7f47b8102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.046+0000 7f47bf8ff640 1 -- 172.21.15.105:0/1545420714 shutdown_connections 2024-03-26T17:47:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:28.046+0000 7f47bf8ff640 1 -- 172.21.15.105:0/1545420714 wait complete. 2024-03-26T17:47:28.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:27 smithi144 ceph-mon[36095]: pgmap v621: 97 pgs: 97 active+clean; 29 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-03-26T17:47:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:27 smithi105 ceph-mon[32280]: pgmap v621: 97 pgs: 97 active+clean; 29 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-03-26T17:47:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:27 smithi135 ceph-mon[35636]: pgmap v621: 97 pgs: 97 active+clean; 29 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-03-26T17:47:29.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 288922 3.15 MB/sec execute 372 sec latency 17.830 ms 2024-03-26T17:47:30.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 289627 3.15 MB/sec execute 373 sec latency 16.945 ms 2024-03-26T17:47:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:29 smithi144 ceph-mon[36095]: pgmap v622: 97 pgs: 97 active+clean; 28 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-03-26T17:47:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:29 smithi105 ceph-mon[32280]: pgmap v622: 97 pgs: 97 active+clean; 28 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-03-26T17:47:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:29 smithi135 ceph-mon[35636]: pgmap v622: 97 pgs: 97 active+clean; 28 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-03-26T17:47:31.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 290305 3.15 MB/sec execute 374 sec latency 16.570 ms 2024-03-26T17:47:32.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 290883 3.16 MB/sec execute 375 sec latency 16.098 ms 2024-03-26T17:47:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:31 smithi144 ceph-mon[36095]: pgmap v623: 97 pgs: 97 active+clean; 30 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-03-26T17:47:32.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:31 smithi105 ceph-mon[32280]: pgmap v623: 97 pgs: 97 active+clean; 30 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-03-26T17:47:32.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:31 smithi135 ceph-mon[35636]: pgmap v623: 97 pgs: 97 active+clean; 30 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-03-26T17:47:33.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 291334 3.15 MB/sec execute 376 sec latency 15.196 ms 2024-03-26T17:47:34.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 291854 3.15 MB/sec execute 377 sec latency 22.924 ms 2024-03-26T17:47:34.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:33 smithi144 ceph-mon[36095]: pgmap v624: 97 pgs: 97 active+clean; 27 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:47:34.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:33 smithi105 ceph-mon[32280]: pgmap v624: 97 pgs: 97 active+clean; 27 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:47:34.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:33 smithi135 ceph-mon[35636]: pgmap v624: 97 pgs: 97 active+clean; 27 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:47:35.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 292495 3.15 MB/sec execute 378 sec latency 19.814 ms 2024-03-26T17:47:36.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 293216 3.15 MB/sec execute 379 sec latency 19.118 ms 2024-03-26T17:47:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:35 smithi144 ceph-mon[36095]: pgmap v625: 97 pgs: 97 active+clean; 27 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-03-26T17:47:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:35 smithi105 ceph-mon[32280]: pgmap v625: 97 pgs: 97 active+clean; 27 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-03-26T17:47:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:35 smithi135 ceph-mon[35636]: pgmap v625: 97 pgs: 97 active+clean; 27 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.7 MiB/s wr, 121 op/s 2024-03-26T17:47:37.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 293888 3.15 MB/sec execute 380 sec latency 18.490 ms 2024-03-26T17:47:38.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 294452 3.16 MB/sec execute 381 sec latency 16.410 ms 2024-03-26T17:47:38.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:37 smithi144 ceph-mon[36095]: pgmap v626: 97 pgs: 97 active+clean; 26 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:47:38.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:37 smithi105 ceph-mon[32280]: pgmap v626: 97 pgs: 97 active+clean; 26 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:47:38.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:37 smithi135 ceph-mon[35636]: pgmap v626: 97 pgs: 97 active+clean; 26 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-03-26T17:47:39.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 294894 3.15 MB/sec execute 382 sec latency 16.465 ms 2024-03-26T17:47:40.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 295417 3.15 MB/sec execute 383 sec latency 28.058 ms 2024-03-26T17:47:40.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:39 smithi144 ceph-mon[36095]: pgmap v627: 97 pgs: 97 active+clean; 26 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-03-26T17:47:40.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:39 smithi105 ceph-mon[32280]: pgmap v627: 97 pgs: 97 active+clean; 26 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-03-26T17:47:40.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:39 smithi135 ceph-mon[35636]: pgmap v627: 97 pgs: 97 active+clean; 26 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-03-26T17:47:41.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 296072 3.15 MB/sec execute 384 sec latency 17.507 ms 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 -- 172.21.15.105:0/2382338211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541056d0 msgr2=0x7f4c5410bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/2382338211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541056d0 0x7f4c5410bf60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009ca0 tx=0x7f4c4c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 -- 172.21.15.105:0/2382338211 shutdown_connections 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/2382338211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541056d0 0x7f4c5410bf60 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/2382338211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54104d10 0x7f4c54105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/2382338211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c54103b20 0x7f4c54103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 -- 172.21.15.105:0/2382338211 >> 172.21.15.105:0/2382338211 conn(0x7f4c540ff0b0 msgr2=0x7f4c541014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 -- 172.21.15.105:0/2382338211 shutdown_connections 2024-03-26T17:47:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.139+0000 7f4c5bd62640 1 -- 172.21.15.105:0/2382338211 wait complete. 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5bd62640 1 Processor -- start 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5bd62640 1 -- start start 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5bd62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54103b20 0x7f4c54199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5bd62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c54104d10 0x7f4c5419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5bd62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541056d0 0x7f4c541a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5bd62640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410f630 con 0x7f4c54104d10 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5bd62640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410f4b0 con 0x7f4c54103b20 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5bd62640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410f7b0 con 0x7f4c541056d0 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c592fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c54104d10 0x7f4c5419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c59afe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54103b20 0x7f4c54199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c59afe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54103b20 0x7f4c54199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52418/0 (socket says 172.21.15.105:52418) 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c592fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c54104d10 0x7f4c5419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45426/0 (socket says 172.21.15.105:45426) 2024-03-26T17:47:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c59afe640 1 -- 172.21.15.105:0/1412223428 learned_addr learned my addr 172.21.15.105:0/1412223428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541056d0 0x7f4c541a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c59afe640 1 -- 172.21.15.105:0/1412223428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541056d0 msgr2=0x7f4c541a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c59afe640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541056d0 0x7f4c541a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c59afe640 1 -- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c54104d10 msgr2=0x7f4c5419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.140+0000 7f4c59afe640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c54104d10 0x7f4c5419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.141+0000 7f4c59afe640 1 -- 172.21.15.105:0/1412223428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c541a1840 con 0x7f4c54103b20 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.141+0000 7f4c592fd640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c54104d10 0x7f4c5419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.141+0000 7f4c59afe640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54103b20 0x7f4c54199c30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e9c0 tx=0x7f4c5000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.141+0000 7f4c4affd640 1 -- 172.21.15.105:0/1412223428 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000cde0 con 0x7f4c54103b20 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.141+0000 7f4c5bd62640 1 -- 172.21.15.105:0/1412223428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c541a1b30 con 0x7f4c54103b20 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.141+0000 7f4c4affd640 1 -- 172.21.15.105:0/1412223428 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c500108e0 con 0x7f4c54103b20 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.141+0000 7f4c5bd62640 1 -- 172.21.15.105:0/1412223428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c541a20d0 con 0x7f4c54103b20 2024-03-26T17:47:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.141+0000 7f4c4affd640 1 -- 172.21.15.105:0/1412223428 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500188e0 con 0x7f4c54103b20 2024-03-26T17:47:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5ea2f740 1 Processor -- start 2024-03-26T17:47:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5ea2f740 1 -- start start 2024-03-26T17:47:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 0x559a9f4b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559a9f4190b0 con 0x559a9f4b45d0 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559a9f3c1960 con 0x559a9f4b4fc0 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559a9f415060 con 0x559a9f4ba130 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5a2ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c592fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5a2ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45428/0 (socket says 172.21.15.105:45428) 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c5a2ff640 1 -- 172.21.15.105:0/4251715174 learned_addr learned my addr 172.21.15.105:0/4251715174 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c4affd640 1 -- 172.21.15.105:0/1412223428 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4c5001f020 con 0x7f4c54103b20 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.144+0000 7f4c592fd640 1 -- 172.21.15.105:0/4251715174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 msgr2=0x559a9f4b9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c592fd640 1 --2- 172.21.15.105:0/4251715174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 0x559a9f4b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c592fd640 1 -- 172.21.15.105:0/4251715174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 msgr2=0x559a9f4b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c592fd640 1 --2- 172.21.15.105:0/4251715174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c592fd640 1 -- 172.21.15.105:0/4251715174 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9f3d3c60 con 0x559a9f4ba130 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/4251715174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c592fd640 1 --2- 172.21.15.105:0/4251715174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4bc5c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009fd0 tx=0x7f4c440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694b25cf6413d0bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c3ffff640 1 -- 172.21.15.105:0/4251715174 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44004030 con 0x559a9f4ba130 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c3ffff640 1 -- 172.21.15.105:0/4251715174 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c44005020 con 0x559a9f4ba130 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c3ffff640 1 -- 172.21.15.105:0/4251715174 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44004c80 con 0x559a9f4ba130 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c4affd640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c400a2fc0 0x7f4c400a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c400a2fc0 0x7f4c400a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.145+0000 7f4c4affd640 1 -- 172.21.15.105:0/1412223428 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f4c500cf6e0 con 0x7f4c54103b20 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/4251715174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 msgr2=0x559a9f4bc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c400a2fc0 0x7f4c400a5480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x559a9f4b5820 tx=0x7f4c4c0324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/4251715174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4bc5c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009fd0 tx=0x7f4c440118d0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/4251715174 shutdown_connections 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/4251715174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4bc5c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/4251715174 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 0x559a9f4b9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/4251715174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/4251715174 >> 172.21.15.105:0/4251715174 conn(0x559a9f4afb40 msgr2=0x559a9f4aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/4251715174 shutdown_connections 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/4251715174 wait complete. 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 -- start start 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.146+0000 7f4c5ea2f740 1 Processor -- start 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 0x559a9f4be8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559a9f4bd0e0 con 0x559a9f4b45d0 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559a9f3d4db0 con 0x559a9f4b4fc0 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559a9f4bd260 con 0x559a9f4ba130 2024-03-26T17:47:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c5ea2f740 10 client.0 _ll_register_callbacks cb 0x559a9f3d9110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c59afe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c59afe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4be3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45444/0 (socket says 172.21.15.105:45444) 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c59afe640 1 -- 172.21.15.105:0/3955872349 learned_addr learned my addr 172.21.15.105:0/3955872349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c5a2ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c592fd640 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 0x559a9f4be8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c59afe640 1 -- 172.21.15.105:0/3955872349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 msgr2=0x559a9f4c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c59afe640 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c59afe640 1 -- 172.21.15.105:0/3955872349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 msgr2=0x559a9f4be8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c59afe640 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 0x559a9f4be8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.147+0000 7f4c59afe640 1 -- 172.21.15.105:0/3955872349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9f12d210 con 0x559a9f4b45d0 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c59afe640 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4be3a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f4c500986b0 tx=0x7f4c50099f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf9cdf1fe9bd222 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/3955872349 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500a44f0 con 0x559a9f4b45d0 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/3955872349 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f4c50018b80 con 0x559a9f4b45d0 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/3955872349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c44005020 con 0x559a9f4b45d0 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/3955872349 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4c5009bd70 con 0x559a9f4b45d0 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 10 client.18765 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/3955872349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a9f1f9620 con 0x559a9f4b45d0 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/3955872349 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c44005020 con 0x559a9f4b45d0 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/3955872349 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4c5009b1c0 con 0x559a9f4b45d0 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 10 client.18765 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 10 client.18765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 20 client.18765 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559a9f4ca1b0 0x559a9f4cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 4 client.18765 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/3955872349 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a9f4265f0 con 0x559a9f4ca1b0 2024-03-26T17:47:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.148+0000 7f4c592fd640 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559a9f4ca1b0 0x559a9f4cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.149+0000 7f4c3d7fa640 10 client.18765 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.149+0000 7f4c592fd640 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559a9f4ca1b0 0x559a9f4cc5c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e473cd1d4f2b8a server_cookie=cc6f68159a7a78e7 in_seq=0 out_seq=0 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/3955872349 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559a9f4265f0 con 0x559a9f4ca1b0 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c3d7fa640 10 client.18765 handle_command_reply: tid=0 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 client.18765 shutdown 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/3955872349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 msgr2=0x559a9f4be3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4be3a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f4c500986b0 tx=0x7f4c50099f70 comp rx=0 tx=0).stop 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/3955872349 shutdown_connections 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559a9f4ca1b0 0x559a9f4cc5c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4ba130 0x559a9f4c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c3d7fa640 0 client.18765 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b4fc0 0x559a9f4be8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/3955872349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4b45d0 0x559a9f4be3a0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/3955872349 >> 172.21.15.105:0/3955872349 conn(0x559a9f4afb40 msgr2=0x559a9f4aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/3955872349 shutdown_connections 2024-03-26T17:47:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/3955872349 wait complete. 2024-03-26T17:47:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.150+0000 7f4c5ea2f740 20 client.18765 trim_cache size 0 max 16384 2024-03-26T17:47:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5ea2f740 1 Processor -- start 2024-03-26T17:47:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5ea2f740 1 -- start start 2024-03-26T17:47:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4be6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 0x559a9f4c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4ae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541aaf90 con 0x559a9f4ba130 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541acad0 con 0x559a9f4b45d0 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541ab660 con 0x559a9f4b4fc0 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c59afe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4be6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c5a2ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c59afe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4be6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52448/0 (socket says 172.21.15.105:52448) 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c59afe640 1 -- 172.21.15.105:0/1603168434 learned_addr learned my addr 172.21.15.105:0/1603168434 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.167+0000 7f4c592fd640 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 0x559a9f4c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c5a2ff640 1 -- 172.21.15.105:0/1603168434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 msgr2=0x559a9f4c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 0x559a9f4c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c5a2ff640 1 -- 172.21.15.105:0/1603168434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 msgr2=0x559a9f4be6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4be6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c5a2ff640 1 -- 172.21.15.105:0/1603168434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9f4b9a80 con 0x559a9f4ba130 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c59afe640 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4be6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4ae560 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c032380 tx=0x7f4c4c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07da9cd323ab4aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c3f7fe640 1 -- 172.21.15.105:0/1603168434 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c039040 con 0x559a9f4ba130 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c3f7fe640 1 -- 172.21.15.105:0/1603168434 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c4c004040 con 0x559a9f4ba130 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.168+0000 7f4c3f7fe640 1 -- 172.21.15.105:0/1603168434 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c04d6b0 con 0x559a9f4ba130 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1603168434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 msgr2=0x559a9f4ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4ae560 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c032380 tx=0x7f4c4c041000 comp rx=0 tx=0).stop 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1603168434 shutdown_connections 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4ae560 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 0x559a9f4c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1603168434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4be6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1603168434 >> 172.21.15.105:0/1603168434 conn(0x559a9f4afb40 msgr2=0x559a9f4bdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1603168434 shutdown_connections 2024-03-26T17:47:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1603168434 wait complete. 2024-03-26T17:47:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 -- start start 2024-03-26T17:47:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.169+0000 7f4c5ea2f740 1 Processor -- start 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4c3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 0x559a9f4c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5ea2f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541ac400 con 0x559a9f4ba130 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541abd30 con 0x559a9f4b45d0 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5ea2f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a8f70 con 0x559a9f4b4fc0 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5ea2f740 10 client.0 _ll_register_callbacks cb 0x7f4c5009b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c59afe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45458/0 (socket says 172.21.15.105:45458) 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 -- 172.21.15.105:0/1801010572 learned_addr learned my addr 172.21.15.105:0/1801010572 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c592fd640 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 0x559a9f4c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 -- 172.21.15.105:0/1801010572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 msgr2=0x559a9f4c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 0x559a9f4c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 -- 172.21.15.105:0/1801010572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 msgr2=0x559a9f4c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 -- 172.21.15.105:0/1801010572 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9f4c51d0 con 0x559a9f4ba130 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5a2ff640 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4c4be0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c033000 tx=0x7f4c4c02c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05985c6a958f724 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/1801010572 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c059070 con 0x559a9f4ba130 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/1801010572 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f4c4c004600 con 0x559a9f4ba130 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.170+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1801010572 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4c004040 con 0x559a9f4ba130 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/1801010572 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4c4c065050 con 0x559a9f4ba130 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c5ea2f740 10 client.18771 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1801010572 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a9f530b30 con 0x559a9f4ba130 2024-03-26T17:47:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/1801010572 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c4c004040 con 0x559a9f4ba130 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/1801010572 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4c4c04d450 con 0x559a9f4ba130 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c5ea2f740 10 client.18771 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c5ea2f740 10 client.18771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c5ea2f740 20 client.18771 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559a9f531510 0x559a9f533930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c5ea2f740 4 client.18771 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1801010572 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a9f420940 con 0x559a9f531510 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.171+0000 7f4c59afe640 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559a9f531510 0x559a9f533930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.172+0000 7f4c3d7fa640 10 client.18771 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.172+0000 7f4c59afe640 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559a9f531510 0x559a9f533930 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476fd138de63b3b4 server_cookie=7ae122b839da391f in_seq=0 out_seq=0 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.210+0000 7f4c3d7fa640 1 -- 172.21.15.105:0/1801010572 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a9f420940 con 0x559a9f531510 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.210+0000 7f4c3d7fa640 10 client.18771 handle_command_reply: tid=0 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.210+0000 7f4c5ea2f740 1 client.18771 shutdown 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1801010572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 msgr2=0x559a9f4c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4c4be0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c033000 tx=0x7f4c4c02c790 comp rx=0 tx=0).stop 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1801010572 shutdown_connections 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559a9f531510 0x559a9f533930 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559a9f4ba130 0x559a9f4c4be0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c3d7fa640 0 client.18771 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559a9f4b4fc0 0x559a9f4c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 --2- 172.21.15.105:0/1801010572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a9f4b45d0 0x559a9f4c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1801010572 >> 172.21.15.105:0/1801010572 conn(0x559a9f4afb40 msgr2=0x559a9f4b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1801010572 shutdown_connections 2024-03-26T17:47:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 1 -- 172.21.15.105:0/1801010572 wait complete. 2024-03-26T17:47:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.211+0000 7f4c5ea2f740 20 client.18771 trim_cache size 0 max 16384 2024-03-26T17:47:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:47:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:47:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:47:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:47:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 -- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c400a2fc0 msgr2=0x7f4c400a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c400a2fc0 0x7f4c400a5480 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x559a9f4b5820 tx=0x7f4c4c0324f0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 -- 172.21.15.105:0/1412223428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54103b20 msgr2=0x7f4c54199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54103b20 0x7f4c54199c30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000e9c0 tx=0x7f4c5000ee80 comp rx=0 tx=0).stop 2024-03-26T17:47:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 -- 172.21.15.105:0/1412223428 shutdown_connections 2024-03-26T17:47:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c400a2fc0 0x7f4c400a5480 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541056d0 0x7f4c541a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c54104d10 0x7f4c5419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 --2- 172.21.15.105:0/1412223428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54103b20 0x7f4c54199c30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 -- 172.21.15.105:0/1412223428 >> 172.21.15.105:0/1412223428 conn(0x7f4c540ff0b0 msgr2=0x7f4c54107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 -- 172.21.15.105:0/1412223428 shutdown_connections 2024-03-26T17:47:41.207 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:41.212+0000 7f4c5bd62640 1 -- 172.21.15.105:0/1412223428 wait complete. 2024-03-26T17:47:42.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 296727 3.15 MB/sec execute 385 sec latency 25.020 ms 2024-03-26T17:47:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:41 smithi144 ceph-mon[36095]: pgmap v628: 97 pgs: 97 active+clean; 27 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-03-26T17:47:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:41 smithi105 ceph-mon[32280]: pgmap v628: 97 pgs: 97 active+clean; 27 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-03-26T17:47:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:41 smithi135 ceph-mon[35636]: pgmap v628: 97 pgs: 97 active+clean; 27 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-03-26T17:47:43.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 297403 3.15 MB/sec execute 386 sec latency 16.470 ms 2024-03-26T17:47:44.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 297980 3.16 MB/sec execute 387 sec latency 16.047 ms 2024-03-26T17:47:44.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:43 smithi144 ceph-mon[36095]: pgmap v629: 97 pgs: 97 active+clean; 29 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:47:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:43 smithi105 ceph-mon[32280]: pgmap v629: 97 pgs: 97 active+clean; 29 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:47:44.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:43 smithi135 ceph-mon[35636]: pgmap v629: 97 pgs: 97 active+clean; 29 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:47:45.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 298412 3.15 MB/sec execute 388 sec latency 46.332 ms 2024-03-26T17:47:45.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:47:45.732 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:47:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.858+0000 7fcec2459640 1 -- 172.21.15.105:0/1341888299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc102370 msgr2=0x7fcebc108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.858+0000 7fcec2459640 1 --2- 172.21.15.105:0/1341888299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc102370 0x7fcebc108c00 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009ca0 tx=0x7fceb402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.858+0000 7fcec2459640 1 -- 172.21.15.105:0/1341888299 shutdown_connections 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.858+0000 7fcec2459640 1 --2- 172.21.15.105:0/1341888299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc102370 0x7fcebc108c00 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.858+0000 7fcec2459640 1 --2- 172.21.15.105:0/1341888299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcebc1019b0 0x7fcebc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.858+0000 7fcec2459640 1 --2- 172.21.15.105:0/1341888299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcebc1007c0 0x7fcebc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.858+0000 7fcec2459640 1 -- 172.21.15.105:0/1341888299 >> 172.21.15.105:0/1341888299 conn(0x7fcebc0fbd50 msgr2=0x7fcebc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.858+0000 7fcec2459640 1 -- 172.21.15.105:0/1341888299 shutdown_connections 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.859+0000 7fcec2459640 1 -- 172.21.15.105:0/1341888299 wait complete. 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.859+0000 7fcec2459640 1 Processor -- start 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.859+0000 7fcec2459640 1 -- start start 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.859+0000 7fcec2459640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcebc1007c0 0x7fcebc196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.859+0000 7fcec2459640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc1019b0 0x7fcebc196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.859+0000 7fcec2459640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcebc102370 0x7fcebc19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.859+0000 7fcec2459640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc10c2f0 con 0x7fcebc1019b0 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcec2459640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc10c170 con 0x7fcebc1007c0 2024-03-26T17:47:45.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcec2459640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc10c470 con 0x7fcebc102370 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc1019b0 0x7fcebc196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcebc1007c0 0x7fcebc196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc1019b0 0x7fcebc196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59386/0 (socket says 172.21.15.105:59386) 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 -- 172.21.15.105:0/2585530673 learned_addr learned my addr 172.21.15.105:0/2585530673 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcec09f6640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcebc102370 0x7fcebc19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 -- 172.21.15.105:0/2585530673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcebc102370 msgr2=0x7fcebc19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcebc102370 0x7fcebc19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 -- 172.21.15.105:0/2585530673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcebc1007c0 msgr2=0x7fcebc196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcebc1007c0 0x7fcebc196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 -- 172.21.15.105:0/2585530673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcebc068b20 con 0x7fcebc1019b0 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebbfff640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcebc1007c0 0x7fcebc196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc1019b0 0x7fcebc196f50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e9f0 tx=0x7fceb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fceb97fa640 1 -- 172.21.15.105:0/2585530673 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb000cd80 con 0x7fcebc1019b0 2024-03-26T17:47:45.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fceb97fa640 1 -- 172.21.15.105:0/2585530673 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceb00108e0 con 0x7fcebc1019b0 2024-03-26T17:47:45.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fceb97fa640 1 -- 172.21.15.105:0/2585530673 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0018920 con 0x7fcebc1019b0 2024-03-26T17:47:45.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcec2459640 1 -- 172.21.15.105:0/2585530673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcebc068e10 con 0x7fcebc1019b0 2024-03-26T17:47:45.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.860+0000 7fcec2459640 1 -- 172.21.15.105:0/2585530673 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcebc069360 con 0x7fcebc1019b0 2024-03-26T17:47:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fceb97fa640 1 -- 172.21.15.105:0/2585530673 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fceb0010a90 con 0x7fcebc1019b0 2024-03-26T17:47:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec5040740 1 Processor -- start 2024-03-26T17:47:45.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec5040740 1 -- start start 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec5040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6718f0 0x56333e671d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec5040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e676de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec5040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 0x56333e6798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec5040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56333e5d6980 con 0x56333e6722e0 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fceb97fa640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcea40a2f20 0x7fcea40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec5040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56333e57ebe0 con 0x56333e677450 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec5040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56333e5d2930 con 0x56333e6718f0 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e676de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcec09f6640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcea40a2f20 0x7fcea40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e676de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59400/0 (socket says 172.21.15.105:59400) 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcebb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 0x56333e6798e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.863+0000 7fcebbfff640 1 -- 172.21.15.105:0/761942430 learned_addr learned my addr 172.21.15.105:0/761942430 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcec09f6640 1 --2- 172.21.15.105:0/761942430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6718f0 0x56333e671d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fceb97fa640 1 -- 172.21.15.105:0/2585530673 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fceb00cf700 con 0x7fcebc1019b0 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcebbfff640 1 -- 172.21.15.105:0/761942430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6718f0 msgr2=0x56333e671d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcebbfff640 1 --2- 172.21.15.105:0/761942430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6718f0 0x56333e671d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcebbfff640 1 -- 172.21.15.105:0/761942430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 msgr2=0x56333e6798e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcebbfff640 1 --2- 172.21.15.105:0/761942430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 0x56333e6798e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcebbfff640 1 -- 172.21.15.105:0/761942430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333e3b47d0 con 0x56333e6722e0 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcec09f6640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcea40a2f20 0x7fcea40a53e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006010 tx=0x7fceb4009930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcebbfff640 1 --2- 172.21.15.105:0/761942430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e676de0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fceac0114f0 tx=0x7fceac0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9663008e23963f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.864+0000 7fcea27fc640 1 -- 172.21.15.105:0/761942430 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac01e070 con 0x56333e6722e0 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcea27fc640 1 -- 172.21.15.105:0/761942430 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac0035f0 con 0x56333e6722e0 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcea27fc640 1 -- 172.21.15.105:0/761942430 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac00f740 con 0x56333e6722e0 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcec5040740 1 -- 172.21.15.105:0/761942430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 msgr2=0x56333e676de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcec5040740 1 --2- 172.21.15.105:0/761942430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e676de0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fceac0114f0 tx=0x7fceac0119b0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcec5040740 1 -- 172.21.15.105:0/761942430 shutdown_connections 2024-03-26T17:47:45.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcec5040740 1 --2- 172.21.15.105:0/761942430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 0x56333e6798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcec5040740 1 --2- 172.21.15.105:0/761942430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e676de0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcec5040740 1 --2- 172.21.15.105:0/761942430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6718f0 0x56333e671d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcec5040740 1 -- 172.21.15.105:0/761942430 >> 172.21.15.105:0/761942430 conn(0x56333e66ce60 msgr2=0x56333e66d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.865+0000 7fcec5040740 1 -- 172.21.15.105:0/761942430 shutdown_connections 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 -- 172.21.15.105:0/761942430 wait complete. 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 -- start start 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 Processor -- start 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e67bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e67c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e677450 0x56333e6830c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56333e67a400 con 0x56333e677450 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56333e592160 con 0x56333e6718f0 2024-03-26T17:47:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec5040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56333e67a580 con 0x56333e6722e0 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.866+0000 7fcec09f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e677450 0x56333e6830c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e67bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec09f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e677450 0x56333e6830c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59416/0 (socket says 172.21.15.105:59416) 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec09f6640 1 -- 172.21.15.105:0/1055160433 learned_addr learned my addr 172.21.15.105:0/1055160433 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e67c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec5040740 10 client.0 _ll_register_callbacks cb 0x56333e5ce720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec09f6640 1 -- 172.21.15.105:0/1055160433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 msgr2=0x56333e67c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec09f6640 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e67c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec09f6640 1 -- 172.21.15.105:0/1055160433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 msgr2=0x56333e67bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec09f6640 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e67bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec09f6640 1 -- 172.21.15.105:0/1055160433 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333e2ea3f0 con 0x56333e677450 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e67c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:45.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcebbfff640 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e67bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec09f6640 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e677450 0x56333e6830c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009ca0 tx=0x7fceb40324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edef72b34eecc896 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fce9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1055160433 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4051060 con 0x56333e677450 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1055160433 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fceb4004d40 con 0x56333e677450 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec5040740 1 -- 172.21.15.105:0/1055160433 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceac0035f0 con 0x56333e677450 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1055160433 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fceb4039040 con 0x56333e677450 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec5040740 10 client.18783 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.867+0000 7fcec5040740 1 -- 172.21.15.105:0/1055160433 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56333e2aa0f0 con 0x56333e677450 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1055160433 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceac0035f0 con 0x56333e677450 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1055160433 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fceb403f030 con 0x56333e677450 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fcec5040740 10 client.18783 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fcec5040740 10 client.18783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fcec5040740 20 client.18783 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:45.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fcec5040740 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56333e687a40 0x56333e689ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fcec5040740 4 client.18783 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fcec5040740 1 -- 172.21.15.105:0/1055160433 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56333e5e3ec0 con 0x56333e687a40 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fcebbfff640 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56333e687a40 0x56333e689ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fce9b7fe640 10 client.18783 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.868+0000 7fcebbfff640 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56333e687a40 0x56333e689ea0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6e90ed0221967c server_cookie=a969a4be87d9d588 in_seq=0 out_seq=0 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.869+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1055160433 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56333e5e3ec0 con 0x56333e687a40 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.869+0000 7fce9b7fe640 10 client.18783 handle_command_reply: tid=0 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.869+0000 7fcec5040740 1 client.18783 shutdown 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 -- 172.21.15.105:0/1055160433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e677450 msgr2=0x56333e6830c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e677450 0x56333e6830c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009ca0 tx=0x7fceb40324f0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 -- 172.21.15.105:0/1055160433 shutdown_connections 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56333e687a40 0x56333e689ea0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e677450 0x56333e6830c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e67c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fce9b7fe640 0 client.18783 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:45.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 --2- 172.21.15.105:0/1055160433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e67bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 -- 172.21.15.105:0/1055160433 >> 172.21.15.105:0/1055160433 conn(0x56333e66ce60 msgr2=0x56333e66c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:45.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 -- 172.21.15.105:0/1055160433 shutdown_connections 2024-03-26T17:47:45.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 1 -- 172.21.15.105:0/1055160433 wait complete. 2024-03-26T17:47:45.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.870+0000 7fcec5040740 20 client.18783 trim_cache size 0 max 16384 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec5040740 1 Processor -- start 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec5040740 1 -- start start 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec5040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e6707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec5040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e683080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec5040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e677450 0x56333e68e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec5040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc1aaf60 con 0x56333e6722e0 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec5040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc1acaa0 con 0x56333e6718f0 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec5040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc1ab630 con 0x56333e677450 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcebb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e683080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e6707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcebb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e683080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59422/0 (socket says 172.21.15.105:59422) 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcebb7fe640 1 -- 172.21.15.105:0/400865650 learned_addr learned my addr 172.21.15.105:0/400865650 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.892+0000 7fcec09f6640 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e677450 0x56333e68e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcebb7fe640 1 -- 172.21.15.105:0/400865650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e677450 msgr2=0x56333e68e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e677450 0x56333e68e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcebb7fe640 1 -- 172.21.15.105:0/400865650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 msgr2=0x56333e6707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e6707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcebb7fe640 1 -- 172.21.15.105:0/400865650 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333e67a6b0 con 0x56333e6722e0 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcebbfff640 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e6707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcec09f6640 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e677450 0x56333e68e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e683080 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fceb00cd1c0 tx=0x7fceb00ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e527eee1b04501 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcea1ffb640 1 -- 172.21.15.105:0/400865650 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb009ff40 con 0x56333e6722e0 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcea1ffb640 1 -- 172.21.15.105:0/400865650 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceb00a4800 con 0x56333e6722e0 2024-03-26T17:47:45.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.893+0000 7fcea1ffb640 1 -- 172.21.15.105:0/400865650 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb009b2f0 con 0x56333e6722e0 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 -- 172.21.15.105:0/400865650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 msgr2=0x56333e683080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e683080 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fceb00cd1c0 tx=0x7fceb00ab040 comp rx=0 tx=0).stop 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 -- 172.21.15.105:0/400865650 shutdown_connections 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e677450 0x56333e68e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6722e0 0x56333e683080 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 --2- 172.21.15.105:0/400865650 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e6718f0 0x56333e6707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 -- 172.21.15.105:0/400865650 >> 172.21.15.105:0/400865650 conn(0x56333e66ce60 msgr2=0x56333e6700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 -- 172.21.15.105:0/400865650 shutdown_connections 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 -- 172.21.15.105:0/400865650 wait complete. 2024-03-26T17:47:45.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.894+0000 7fcec5040740 1 -- start start 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec5040740 1 Processor -- start 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec5040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6718f0 0x56333e6809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec5040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e680f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec5040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 0x56333e681f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec5040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc1ac3d0 con 0x56333e6718f0 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec5040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc1abd00 con 0x56333e677450 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec5040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc1a8f40 con 0x56333e6722e0 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec5040740 10 client.0 _ll_register_callbacks cb 0x7fceac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6718f0 0x56333e6809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6718f0 0x56333e6809f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59426/0 (socket says 172.21.15.105:59426) 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 -- 172.21.15.105:0/1819229637 learned_addr learned my addr 172.21.15.105:0/1819229637 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcec09f6640 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 0x56333e681f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e680f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 -- 172.21.15.105:0/1819229637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 msgr2=0x56333e680f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e680f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 -- 172.21.15.105:0/1819229637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 msgr2=0x56333e681f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 0x56333e681f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 -- 172.21.15.105:0/1819229637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333e682530 con 0x56333e6718f0 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fcebbfff640 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6718f0 0x56333e6809f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fceac011eb0 tx=0x7fceac003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87b9e8df5e23fa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.895+0000 7fce9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1819229637 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac02e070 con 0x56333e6718f0 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 1 -- 172.21.15.105:0/1819229637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb00a4800 con 0x56333e6718f0 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1819229637 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fceac011050 con 0x56333e6718f0 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1819229637 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fceac03a050 con 0x56333e6718f0 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 10 client.18789 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 1 -- 172.21.15.105:0/1819229637 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56333e6ee050 con 0x56333e6718f0 2024-03-26T17:47:45.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1819229637 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fceb00a4800 con 0x56333e6718f0 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1819229637 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fceac02a3a0 con 0x56333e6718f0 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 10 client.18789 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 10 client.18789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 20 client.18789 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56333e6ee840 0x56333e6f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 4 client.18789 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcec5040740 1 -- 172.21.15.105:0/1819229637 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56333e5de210 con 0x56333e6ee840 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.896+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56333e6ee840 0x56333e6f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.897+0000 7fce9b7fe640 10 client.18789 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.897+0000 7fcebb7fe640 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56333e6ee840 0x56333e6f0c60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b93c5da76f9206 server_cookie=8c4b2c3b855f9726 in_seq=0 out_seq=0 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fce9b7fe640 1 -- 172.21.15.105:0/1819229637 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56333e5de210 con 0x56333e6ee840 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fce9b7fe640 10 client.18789 handle_command_reply: tid=0 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 client.18789 shutdown 2024-03-26T17:47:45.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 -- 172.21.15.105:0/1819229637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6718f0 msgr2=0x56333e6809f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6718f0 0x56333e6809f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fceac011eb0 tx=0x7fceac003ca0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 -- 172.21.15.105:0/1819229637 shutdown_connections 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56333e6ee840 0x56333e6f0c60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56333e677450 0x56333e681f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56333e6722e0 0x56333e680f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 --2- 172.21.15.105:0/1819229637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56333e6718f0 0x56333e6809f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fce9b7fe640 0 client.18789 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 -- 172.21.15.105:0/1819229637 >> 172.21.15.105:0/1819229637 conn(0x56333e66ce60 msgr2=0x56333e6700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 -- 172.21.15.105:0/1819229637 shutdown_connections 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 1 -- 172.21.15.105:0/1819229637 wait complete. 2024-03-26T17:47:45.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec5040740 20 client.18789 trim_cache size 0 max 16384 2024-03-26T17:47:45.897 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:47:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:47:45.897 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:47:45.897 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec2459640 1 -- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcea40a2f20 msgr2=0x7fcea40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec2459640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcea40a2f20 0x7fcea40a53e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006010 tx=0x7fceb4009930 comp rx=0 tx=0).stop 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec2459640 1 -- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc1019b0 msgr2=0x7fcebc196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.898+0000 7fcec2459640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc1019b0 0x7fcebc196f50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e9f0 tx=0x7fceb000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.899+0000 7fcec2459640 1 -- 172.21.15.105:0/2585530673 shutdown_connections 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.899+0000 7fcec2459640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcea40a2f20 0x7fcea40a53e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.899+0000 7fcec2459640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcebc102370 0x7fcebc19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.899+0000 7fcec2459640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcebc1019b0 0x7fcebc196f50 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.899+0000 7fcec2459640 1 --2- 172.21.15.105:0/2585530673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcebc1007c0 0x7fcebc196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:45.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.899+0000 7fcec2459640 1 -- 172.21.15.105:0/2585530673 >> 172.21.15.105:0/2585530673 conn(0x7fcebc0fbd50 msgr2=0x7fcebc104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:45.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.899+0000 7fcec2459640 1 -- 172.21.15.105:0/2585530673 shutdown_connections 2024-03-26T17:47:45.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:45.899+0000 7fcec2459640 1 -- 172.21.15.105:0/2585530673 wait complete. 2024-03-26T17:47:45.902 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:47:45.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:47:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 -- 172.21.15.105:0/1638182418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2730102400 msgr2=0x7f27301047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 --2- 172.21.15.105:0/1638182418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2730102400 0x7f27301047f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f2720009900 tx=0x7f272002bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 -- 172.21.15.105:0/1638182418 shutdown_connections 2024-03-26T17:47:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 --2- 172.21.15.105:0/1638182418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730107660 0x7f2730109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 --2- 172.21.15.105:0/1638182418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2730104d30 0x7f2730107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 --2- 172.21.15.105:0/1638182418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2730102400 0x7f27301047f0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 -- 172.21.15.105:0/1638182418 >> 172.21.15.105:0/1638182418 conn(0x7f27300fbd50 msgr2=0x7f27300fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 -- 172.21.15.105:0/1638182418 shutdown_connections 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.026+0000 7f2734e7e640 1 -- 172.21.15.105:0/1638182418 wait complete. 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2734e7e640 1 Processor -- start 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2734e7e640 1 -- start start 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2734e7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730102400 0x7f2730197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2734e7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2730104d30 0x7f27301980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2734e7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2730107660 0x7f273019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2734e7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d470 con 0x7f2730107660 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2734e7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d2f0 con 0x7f2730104d30 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2734e7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d5f0 con 0x7f2730102400 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f272e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730102400 0x7f2730197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2727fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2730104d30 0x7f27301980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f272ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2730107660 0x7f273019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f272e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730102400 0x7f2730197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33080/0 (socket says 172.21.15.105:33080) 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f272e59c640 1 -- 172.21.15.105:0/3125598159 learned_addr learned my addr 172.21.15.105:0/3125598159 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.027+0000 7f2727fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2730104d30 0x7f27301980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40596/0 (socket says 172.21.15.105:40596) 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f272e59c640 1 -- 172.21.15.105:0/3125598159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2730104d30 msgr2=0x7f27301980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f272e59c640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2730104d30 0x7f27301980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f272e59c640 1 -- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2730107660 msgr2=0x7f273019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f272e59c640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2730107660 0x7f273019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f272e59c640 1 -- 172.21.15.105:0/3125598159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273019f790 con 0x7f2730102400 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f272e59c640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730102400 0x7f2730197b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2720000c00 tx=0x7f27200040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f27277fe640 1 -- 172.21.15.105:0/3125598159 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720043020 con 0x7f2730102400 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273019fa20 con 0x7f2730102400 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f27277fe640 1 -- 172.21.15.105:0/3125598159 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27200044c0 con 0x7f2730102400 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f273019ffc0 con 0x7f2730102400 2024-03-26T17:47:46.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.028+0000 7f27277fe640 1 -- 172.21.15.105:0/3125598159 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720003c30 con 0x7f2730102400 2024-03-26T17:47:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.029+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27300fd840 con 0x7f2730102400 2024-03-26T17:47:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.031+0000 7f27277fe640 1 -- 172.21.15.105:0/3125598159 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2720046020 con 0x7f2730102400 2024-03-26T17:47:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.031+0000 7f27277fe640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f271c0a2fa0 0x7f271c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.032+0000 7f2727fff640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f271c0a2fa0 0x7f271c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.032+0000 7f27277fe640 1 -- 172.21.15.105:0/3125598159 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f27200f15c0 con 0x7f2730102400 2024-03-26T17:47:46.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.032+0000 7f2727fff640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f271c0a2fa0 0x7f271c0a5460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2730199080 tx=0x7f2718005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.036+0000 7f27277fe640 1 -- 172.21.15.105:0/3125598159 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f272004d020 con 0x7f2730102400 2024-03-26T17:47:46.029 INFO:tasks.workunit.client.0.smithi105.stdout: 1 298911 3.15 MB/sec execute 389 sec latency 25.620 ms 2024-03-26T17:47:46.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:45 smithi144 ceph-mon[36095]: pgmap v630: 97 pgs: 97 active+clean; 25 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:47:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:45 smithi105 ceph-mon[32280]: pgmap v630: 97 pgs: 97 active+clean; 25 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:47:46.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:45 smithi135 ceph-mon[35636]: pgmap v630: 97 pgs: 97 active+clean; 25 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:47:46.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.298+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27300624c0 con 0x7f2730102400 2024-03-26T17:47:46.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.299+0000 7f27277fe640 1 -- 172.21.15.105:0/3125598159 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f27200be350 con 0x7f2730102400 2024-03-26T17:47:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:47:46.292 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:47:46.292 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:47:46.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f271c0a2fa0 msgr2=0x7f271c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f271c0a2fa0 0x7f271c0a5460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2730199080 tx=0x7f2718005f90 comp rx=0 tx=0).stop 2024-03-26T17:47:46.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730102400 msgr2=0x7f2730197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730102400 0x7f2730197b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2720000c00 tx=0x7f27200040c0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 shutdown_connections 2024-03-26T17:47:46.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f271c0a2fa0 0x7f271c0a5460 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2730107660 0x7f273019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2730104d30 0x7f27301980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 --2- 172.21.15.105:0/3125598159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730102400 0x7f2730197b70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 >> 172.21.15.105:0/3125598159 conn(0x7f27300fbd50 msgr2=0x7f2730102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 shutdown_connections 2024-03-26T17:47:46.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.301+0000 7f2734e7e640 1 -- 172.21.15.105:0/3125598159 wait complete. 2024-03-26T17:47:46.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 -- 172.21.15.105:0/1587849640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301019b0 msgr2=0x7fe930101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 --2- 172.21.15.105:0/1587849640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301019b0 0x7fe930101e30 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe920009900 tx=0x7fe92002bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 -- 172.21.15.105:0/1587849640 shutdown_connections 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 --2- 172.21.15.105:0/1587849640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe930102370 0x7fe930108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 --2- 172.21.15.105:0/1587849640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301019b0 0x7fe930101e30 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 --2- 172.21.15.105:0/1587849640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9301007c0 0x7fe930100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 -- 172.21.15.105:0/1587849640 >> 172.21.15.105:0/1587849640 conn(0x7fe9300fbd50 msgr2=0x7fe9300fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 -- 172.21.15.105:0/1587849640 shutdown_connections 2024-03-26T17:47:46.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.433+0000 7fe936cbf640 1 -- 172.21.15.105:0/1587849640 wait complete. 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.434+0000 7fe936cbf640 1 Processor -- start 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.434+0000 7fe936cbf640 1 -- start start 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.434+0000 7fe936cbf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301007c0 0x7fe930069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.434+0000 7fe936cbf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9301019b0 0x7fe93019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.434+0000 7fe936cbf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe930102370 0x7fe93019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.434+0000 7fe936cbf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe93010c2f0 con 0x7fe9301007c0 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.434+0000 7fe936cbf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe93010c170 con 0x7fe9301019b0 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.434+0000 7fe936cbf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe93010c470 con 0x7fe930102370 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301007c0 0x7fe930069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301007c0 0x7fe930069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59456/0 (socket says 172.21.15.105:59456) 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 -- 172.21.15.105:0/2782785738 learned_addr learned my addr 172.21.15.105:0/2782785738 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe927fff640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9301019b0 0x7fe93019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe93525c640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe930102370 0x7fe93019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 -- 172.21.15.105:0/2782785738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe930102370 msgr2=0x7fe93019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe930102370 0x7fe93019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 -- 172.21.15.105:0/2782785738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9301019b0 msgr2=0x7fe93019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9301019b0 0x7fe93019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 -- 172.21.15.105:0/2782785738 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93019f7a0 con 0x7fe9301007c0 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe934a5b640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301007c0 0x7fe930069590 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00e9c0 tx=0x7fe92c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe925ffb640 1 -- 172.21.15.105:0/2782785738 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00cd80 con 0x7fe9301007c0 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe925ffb640 1 -- 172.21.15.105:0/2782785738 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe92c0035f0 con 0x7fe9301007c0 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe936cbf640 1 -- 172.21.15.105:0/2782785738 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93019fa30 con 0x7fe9301007c0 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.435+0000 7fe925ffb640 1 -- 172.21.15.105:0/2782785738 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c010690 con 0x7fe9301007c0 2024-03-26T17:47:46.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.436+0000 7fe936cbf640 1 -- 172.21.15.105:0/2782785738 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93019ff50 con 0x7fe9301007c0 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.438+0000 7fe925ffb640 1 -- 172.21.15.105:0/2782785738 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe92c0037a0 con 0x7fe9301007c0 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.438+0000 7fe925ffb640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9180a2f20 0x7fe9180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.438+0000 7fe927fff640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9180a2f20 0x7fe9180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.438+0000 7fe925ffb640 1 -- 172.21.15.105:0/2782785738 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe92c0cf7a0 con 0x7fe9301007c0 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe939988740 1 Processor -- start 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe939988740 1 -- start start 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe939988740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c349e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe939988740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c34ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe939988740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c351a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe939988740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559e3c2ae970 con 0x559e3c349a30 2024-03-26T17:47:46.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe939988740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559e3c256d70 con 0x559e3c34f590 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe939988740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e3c2aa920 con 0x559e3c34a420 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe927fff640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9180a2f20 0x7fe9180a53e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe920009230 tx=0x7fe920005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c349e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c349e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59472/0 (socket says 172.21.15.105:59472) 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 -- 172.21.15.105:0/2967693320 learned_addr learned my addr 172.21.15.105:0/2967693320 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe93525c640 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c351a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe927fff640 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c34ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 -- 172.21.15.105:0/2967693320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 msgr2=0x559e3c34ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c34ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 -- 172.21.15.105:0/2967693320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 msgr2=0x559e3c351a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c351a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 -- 172.21.15.105:0/2967693320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3bfc23f0 con 0x559e3c349a30 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.439+0000 7fe934a5b640 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c349e50 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0a4370 tx=0x7fe92c09ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0efb0132c66575b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe916ffd640 1 -- 172.21.15.105:0/2967693320 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c09be90 con 0x559e3c349a30 2024-03-26T17:47:46.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe916ffd640 1 -- 172.21.15.105:0/2967693320 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe92c09ffe0 con 0x559e3c349a30 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe916ffd640 1 -- 172.21.15.105:0/2967693320 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0a4410 con 0x559e3c349a30 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 -- 172.21.15.105:0/2967693320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 msgr2=0x559e3c349e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c349e50 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0a4370 tx=0x7fe92c09ba90 comp rx=0 tx=0).stop 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 -- 172.21.15.105:0/2967693320 shutdown_connections 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c351a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c34ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 --2- 172.21.15.105:0/2967693320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c349e50 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 -- 172.21.15.105:0/2967693320 >> 172.21.15.105:0/2967693320 conn(0x559e3c344fa0 msgr2=0x559e3c3453a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 -- 172.21.15.105:0/2967693320 shutdown_connections 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.440+0000 7fe939988740 1 -- 172.21.15.105:0/2967693320 wait complete. 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 1 -- start start 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 1 Processor -- start 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34a420 0x559e3c3544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34f590 0x559e3c35b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559e3c3524e0 con 0x559e3c349a30 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559e3c26a310 con 0x559e3c34a420 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e3c352660 con 0x559e3c34f590 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe939988740 10 client.0 _ll_register_callbacks cb 0x559e3c2a5c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59484/0 (socket says 172.21.15.105:59484) 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe934a5b640 1 -- 172.21.15.105:0/3556605169 learned_addr learned my addr 172.21.15.105:0/3556605169 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.441+0000 7fe927fff640 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34a420 0x559e3c3544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe93525c640 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34f590 0x559e3c35b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe934a5b640 1 -- 172.21.15.105:0/3556605169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34f590 msgr2=0x559e3c35b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe934a5b640 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34f590 0x559e3c35b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe934a5b640 1 -- 172.21.15.105:0/3556605169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34a420 msgr2=0x559e3c3544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe934a5b640 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34a420 0x559e3c3544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe934a5b640 1 -- 172.21.15.105:0/3556605169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3c08c7d0 con 0x559e3c349a30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe934a5b640 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353fb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0cd5d0 tx=0x7fe92c0cd1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349f9d7f63bc7ea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe90ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe90ffff640 1 -- 172.21.15.105:0/3556605169 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c09b4e0 con 0x559e3c349a30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe90ffff640 1 -- 172.21.15.105:0/3556605169 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe92c097be0 con 0x559e3c349a30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe939988740 1 -- 172.21.15.105:0/3556605169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92c09ffe0 con 0x559e3c349a30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe90ffff640 1 -- 172.21.15.105:0/3556605169 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe92c0bc050 con 0x559e3c349a30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe939988740 10 client.18807 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.442+0000 7fe939988740 1 -- 172.21.15.105:0/3556605169 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e3c35e830 con 0x559e3c349a30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe90ffff640 1 -- 172.21.15.105:0/3556605169 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe92c09ffe0 con 0x559e3c349a30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe90ffff640 1 -- 172.21.15.105:0/3556605169 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe92c09ade0 con 0x559e3c349a30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe939988740 10 client.18807 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe939988740 10 client.18807 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:47:46.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe939988740 20 client.18807 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe939988740 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559e3c35fb80 0x559e3c361f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe939988740 4 client.18807 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe939988740 1 -- 172.21.15.105:0/3556605169 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e3c2bbeb0 con 0x559e3c35fb80 2024-03-26T17:47:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe927fff640 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559e3c35fb80 0x559e3c361f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe927fff640 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559e3c35fb80 0x559e3c361f70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4cc3b68a5736288 server_cookie=160e1daa99432b0f in_seq=0 out_seq=0 2024-03-26T17:47:46.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.443+0000 7fe90ffff640 10 client.18807 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:47:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.444+0000 7fe90ffff640 1 -- 172.21.15.105:0/3556605169 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559e3c2bbeb0 con 0x559e3c35fb80 2024-03-26T17:47:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.444+0000 7fe90ffff640 10 client.18807 handle_command_reply: tid=0 2024-03-26T17:47:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.444+0000 7fe939988740 1 client.18807 shutdown 2024-03-26T17:47:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 -- 172.21.15.105:0/3556605169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 msgr2=0x559e3c353fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353fb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0cd5d0 tx=0x7fe92c0cd1d0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 -- 172.21.15.105:0/3556605169 shutdown_connections 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559e3c35fb80 0x559e3c361f70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe90ffff640 0 client.18807 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34f590 0x559e3c35b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34a420 0x559e3c3544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 --2- 172.21.15.105:0/3556605169 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353fb0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 -- 172.21.15.105:0/3556605169 >> 172.21.15.105:0/3556605169 conn(0x559e3c344fa0 msgr2=0x559e3c3441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 -- 172.21.15.105:0/3556605169 shutdown_connections 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 1 -- 172.21.15.105:0/3556605169 wait complete. 2024-03-26T17:47:46.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.445+0000 7fe939988740 20 client.18807 trim_cache size 0 max 16384 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.467+0000 7fe939988740 1 Processor -- start 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.467+0000 7fe939988740 1 -- start start 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.467+0000 7fe939988740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe939988740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c35b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe939988740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c354100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe939988740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301a8d50 con 0x559e3c349a30 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe939988740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301aa890 con 0x559e3c34f590 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe939988740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301a9420 con 0x559e3c34a420 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59490/0 (socket says 172.21.15.105:59490) 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 -- 172.21.15.105:0/4143989397 learned_addr learned my addr 172.21.15.105:0/4143989397 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe93525c640 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c354100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe927fff640 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c35b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 -- 172.21.15.105:0/4143989397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 msgr2=0x559e3c35b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c35b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 -- 172.21.15.105:0/4143989397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 msgr2=0x559e3c354100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c354100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 -- 172.21.15.105:0/4143989397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3c345d10 con 0x559e3c349a30 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe934a5b640 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353bc0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0058c0 tx=0x7fe92c098500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593b17cb0844adc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe9167fc640 1 -- 172.21.15.105:0/4143989397 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c09a250 con 0x559e3c349a30 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe9167fc640 1 -- 172.21.15.105:0/4143989397 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe92c0a9020 con 0x559e3c349a30 2024-03-26T17:47:46.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.468+0000 7fe9167fc640 1 -- 172.21.15.105:0/4143989397 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c09ffe0 con 0x559e3c349a30 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 -- 172.21.15.105:0/4143989397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 msgr2=0x559e3c353bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353bc0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0058c0 tx=0x7fe92c098500 comp rx=0 tx=0).stop 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 -- 172.21.15.105:0/4143989397 shutdown_connections 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c354100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c35b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 --2- 172.21.15.105:0/4143989397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c353bc0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 -- 172.21.15.105:0/4143989397 >> 172.21.15.105:0/4143989397 conn(0x559e3c344fa0 msgr2=0x559e3c352a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 -- 172.21.15.105:0/4143989397 shutdown_connections 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.469+0000 7fe939988740 1 -- 172.21.15.105:0/4143989397 wait complete. 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 1 -- start start 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 1 Processor -- start 2024-03-26T17:47:46.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c3585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c358af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c359b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301aa1c0 con 0x559e3c349a30 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301a9af0 con 0x559e3c34f590 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301a6d30 con 0x559e3c34a420 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe939988740 10 client.0 _ll_register_callbacks cb 0x7fe928009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c3585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe934a5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c3585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59504/0 (socket says 172.21.15.105:59504) 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe934a5b640 1 -- 172.21.15.105:0/437548090 learned_addr learned my addr 172.21.15.105:0/437548090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe93525c640 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c359b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe927fff640 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c358af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe934a5b640 1 -- 172.21.15.105:0/437548090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 msgr2=0x559e3c358af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe934a5b640 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c358af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe934a5b640 1 -- 172.21.15.105:0/437548090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 msgr2=0x559e3c359b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe934a5b640 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c359b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.470+0000 7fe934a5b640 1 -- 172.21.15.105:0/437548090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3c35a0f0 con 0x559e3c349a30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe934a5b640 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c3585b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe92c09bb90 tx=0x7fe92c0bc1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76adb6ec32d3a7d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe90ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe90ffff640 1 -- 172.21.15.105:0/437548090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c09a250 con 0x559e3c349a30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe90ffff640 1 -- 172.21.15.105:0/437548090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fe92c09b4f0 con 0x559e3c349a30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 1 -- 172.21.15.105:0/437548090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92c0a9020 con 0x559e3c349a30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe90ffff640 1 -- 172.21.15.105:0/437548090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe92c09be90 con 0x559e3c349a30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 10 client.18813 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 1 -- 172.21.15.105:0/437548090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e3c35a0f0 con 0x559e3c349a30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe90ffff640 1 -- 172.21.15.105:0/437548090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe92c0a9020 con 0x559e3c349a30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe90ffff640 1 -- 172.21.15.105:0/437548090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe92c097be0 con 0x559e3c349a30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 10 client.18813 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 10 client.18813 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:47:46.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 20 client.18813 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559e3c3c5cc0 0x559e3c3c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 4 client.18813 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.471+0000 7fe939988740 1 -- 172.21.15.105:0/437548090 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e3c2b6200 con 0x559e3c3c5cc0 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe927fff640 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559e3c3c5cc0 0x559e3c3c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe90ffff640 10 client.18813 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe927fff640 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559e3c3c5cc0 0x559e3c3c80b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d932a7e5b492a9c4 server_cookie=2fdb5e9d96d4a66a in_seq=0 out_seq=0 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe90ffff640 1 -- 172.21.15.105:0/437548090 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e3c2b6200 con 0x559e3c3c5cc0 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe90ffff640 10 client.18813 handle_command_reply: tid=0 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 client.18813 shutdown 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 -- 172.21.15.105:0/437548090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 msgr2=0x559e3c3585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c3585b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fe92c09bb90 tx=0x7fe92c0bc1f0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 -- 172.21.15.105:0/437548090 shutdown_connections 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559e3c3c5cc0 0x559e3c3c80b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e3c34f590 0x559e3c359b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e3c34a420 0x559e3c358af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 --2- 172.21.15.105:0/437548090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e3c349a30 0x559e3c3585b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe90ffff640 0 client.18813 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:47:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.472+0000 7fe939988740 1 -- 172.21.15.105:0/437548090 >> 172.21.15.105:0/437548090 conn(0x559e3c344fa0 msgr2=0x559e3c353b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe939988740 1 -- 172.21.15.105:0/437548090 shutdown_connections 2024-03-26T17:47:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe939988740 1 -- 172.21.15.105:0/437548090 wait complete. 2024-03-26T17:47:46.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe939988740 20 client.18813 trim_cache size 0 max 16384 2024-03-26T17:47:46.471 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 -- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9180a2f20 msgr2=0x7fe9180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9180a2f20 0x7fe9180a53e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe920009230 tx=0x7fe920005fd0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 -- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301007c0 msgr2=0x7fe930069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301007c0 0x7fe930069590 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00e9c0 tx=0x7fe92c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:47:46.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 -- 172.21.15.105:0/2782785738 shutdown_connections 2024-03-26T17:47:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9180a2f20 0x7fe9180a53e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe930102370 0x7fe93019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9301019b0 0x7fe93019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 --2- 172.21.15.105:0/2782785738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9301007c0 0x7fe930069590 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 -- 172.21.15.105:0/2782785738 >> 172.21.15.105:0/2782785738 conn(0x7fe9300fbd50 msgr2=0x7fe930104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 -- 172.21.15.105:0/2782785738 shutdown_connections 2024-03-26T17:47:46.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.473+0000 7fe936cbf640 1 -- 172.21.15.105:0/2782785738 wait complete. 2024-03-26T17:47:46.477 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:47:46.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 -- 172.21.15.105:0/586359950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 msgr2=0x7f23a4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/586359950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 0x7f23a4109d30 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f239c009ca0 tx=0x7f239c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 -- 172.21.15.105:0/586359950 shutdown_connections 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/586359950 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 0x7f23a4109d30 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/586359950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a4102ae0 0x7f23a4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/586359950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a41018f0 0x7f23a4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 -- 172.21.15.105:0/586359950 >> 172.21.15.105:0/586359950 conn(0x7f23a40fce60 msgr2=0x7f23a40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 -- 172.21.15.105:0/586359950 shutdown_connections 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.602+0000 7f23ac7b2640 1 -- 172.21.15.105:0/586359950 wait complete. 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23ac7b2640 1 Processor -- start 2024-03-26T17:47:46.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23ac7b2640 1 -- start start 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23ac7b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a41018f0 0x7f23a4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23ac7b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a4102ae0 0x7f23a4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23ac7b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 0x7f23a419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23ac7b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23a410d400 con 0x7f23a41034a0 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23ac7b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23a410d280 con 0x7f23a4102ae0 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23aa54e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a41018f0 0x7f23a4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23ac7b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23a410d580 con 0x7f23a41018f0 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23aad4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 0x7f23a419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23aa54e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a41018f0 0x7f23a4197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33174/0 (socket says 172.21.15.105:33174) 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23aad4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 0x7f23a419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59526/0 (socket says 172.21.15.105:59526) 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.603+0000 7f23aa54e640 1 -- 172.21.15.105:0/2976327196 learned_addr learned my addr 172.21.15.105:0/2976327196 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f23aad4f640 1 -- 172.21.15.105:0/2976327196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a41018f0 msgr2=0x7f23a4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f23aad4f640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a41018f0 0x7f23a4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f23aad4f640 1 -- 172.21.15.105:0/2976327196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a4102ae0 msgr2=0x7f23a4198000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T17:47:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f23aad4f640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a4102ae0 0x7f23a4198000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f23aad4f640 1 -- 172.21.15.105:0/2976327196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a419f6e0 con 0x7f23a41034a0 2024-03-26T17:47:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f23aad4f640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 0x7f23a419f070 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f239c02c630 tx=0x7f239c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f239b7fe640 1 -- 172.21.15.105:0/2976327196 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c041020 con 0x7f23a41034a0 2024-03-26T17:47:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f239b7fe640 1 -- 172.21.15.105:0/2976327196 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f239c003e20 con 0x7f23a41034a0 2024-03-26T17:47:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f23ac7b2640 1 -- 172.21.15.105:0/2976327196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a419f970 con 0x7f23a41034a0 2024-03-26T17:47:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f23ac7b2640 1 -- 172.21.15.105:0/2976327196 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a419feb0 con 0x7f23a41034a0 2024-03-26T17:47:46.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.604+0000 7f239b7fe640 1 -- 172.21.15.105:0/2976327196 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c039040 con 0x7f23a41034a0 2024-03-26T17:47:46.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.607+0000 7f239b7fe640 1 -- 172.21.15.105:0/2976327196 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f239c044020 con 0x7f23a41034a0 2024-03-26T17:47:46.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23af39e740 1 Processor -- start 2024-03-26T17:47:46.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23af39e740 1 -- start start 2024-03-26T17:47:46.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f239b7fe640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23900a2fa0 0x7f23900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23af39e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23af39e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c96330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23af39e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089c98e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23af39e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565089bf1970 con 0x565089c91830 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23af39e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565089b99d70 con 0x565089c969a0 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23af39e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565089bed920 con 0x565089c90e40 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23900a2fa0 0x7f23900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23aa54e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23aad4f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089c98e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23aa54e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c91260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33178/0 (socket says 172.21.15.105:33178) 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23aa54e640 1 -- 172.21.15.105:0/1542815863 learned_addr learned my addr 172.21.15.105:0/1542815863 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/1542815863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c96330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.608+0000 7f239b7fe640 1 -- 172.21.15.105:0/2976327196 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f239c0f1550 con 0x7f23a41034a0 2024-03-26T17:47:46.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f23aad4f640 1 -- 172.21.15.105:0/1542815863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 msgr2=0x565089c91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f23aad4f640 1 --2- 172.21.15.105:0/1542815863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f23aad4f640 1 -- 172.21.15.105:0/1542815863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 msgr2=0x565089c96330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f23aad4f640 1 --2- 172.21.15.105:0/1542815863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c96330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f23aad4f640 1 -- 172.21.15.105:0/1542815863 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650899053f0 con 0x565089c969a0 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23900a2fa0 0x7f23900a5460 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2394009460 tx=0x7f23940099a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f23aad4f640 1 --2- 172.21.15.105:0/1542815863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089c98e30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f239c0ef380 tx=0x7f239c02c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de00327645e4adf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f2398ff9640 1 -- 172.21.15.105:0/1542815863 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c09b490 con 0x565089c969a0 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f2398ff9640 1 -- 172.21.15.105:0/1542815863 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f239c0be340 con 0x565089c969a0 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.609+0000 7f2398ff9640 1 -- 172.21.15.105:0/1542815863 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c09b0d0 con 0x565089c969a0 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 -- 172.21.15.105:0/1542815863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 msgr2=0x565089c98e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 --2- 172.21.15.105:0/1542815863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089c98e30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f239c0ef380 tx=0x7f239c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 -- 172.21.15.105:0/1542815863 shutdown_connections 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 --2- 172.21.15.105:0/1542815863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089c98e30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 --2- 172.21.15.105:0/1542815863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c96330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 --2- 172.21.15.105:0/1542815863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 -- 172.21.15.105:0/1542815863 >> 172.21.15.105:0/1542815863 conn(0x565089c87fa0 msgr2=0x565089c883a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 -- 172.21.15.105:0/1542815863 shutdown_connections 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 -- 172.21.15.105:0/1542815863 wait complete. 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 -- start start 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.610+0000 7f23af39e740 1 Processor -- start 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23af39e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23af39e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c9b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23af39e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089ca2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23af39e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565089c99710 con 0x565089c91830 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23af39e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565089bad310 con 0x565089c969a0 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23af39e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565089c99890 con 0x565089c90e40 2024-03-26T17:47:46.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23af39e740 10 client.0 _ll_register_callbacks cb 0x565089be8c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aa54e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23a9d4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c9b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aad4f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089ca2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aa54e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33180/0 (socket says 172.21.15.105:33180) 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23a9d4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c9b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59532/0 (socket says 172.21.15.105:59532) 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aa54e640 1 -- 172.21.15.105:0/1012960817 learned_addr learned my addr 172.21.15.105:0/1012960817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aad4f640 1 -- 172.21.15.105:0/1012960817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 msgr2=0x565089c9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aad4f640 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aad4f640 1 -- 172.21.15.105:0/1012960817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 msgr2=0x565089c9b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aad4f640 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c9b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aad4f640 1 -- 172.21.15.105:0/1012960817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650899cf7d0 con 0x565089c969a0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aa54e640 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c9b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f23aad4f640 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089ca2500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f239c032420 tx=0x7f239c0bdc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca589f93d7656b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.611+0000 7f238dffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f238dffb640 1 -- 172.21.15.105:0/1012960817 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c09b2b0 con 0x565089c969a0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 1 -- 172.21.15.105:0/1012960817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f239c0be340 con 0x565089c969a0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f238dffb640 1 -- 172.21.15.105:0/1012960817 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f239c0bbf40 con 0x565089c969a0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f238dffb640 1 -- 172.21.15.105:0/1012960817 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f239c09ac40 con 0x565089c969a0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 10 client.28652 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 1 -- 172.21.15.105:0/1012960817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565089c99710 con 0x565089c969a0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f238dffb640 1 -- 172.21.15.105:0/1012960817 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f239c0be340 con 0x565089c969a0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f238dffb640 1 -- 172.21.15.105:0/1012960817 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f239c03c030 con 0x565089c969a0 2024-03-26T17:47:46.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 10 client.28652 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:46.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 10 client.28652 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:47:46.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 20 client.28652 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:46.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x565089ca6e00 0x565089ca9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 4 client.28652 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:46.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.612+0000 7f23af39e740 1 -- 172.21.15.105:0/1012960817 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565089bfeeb0 con 0x565089ca6e00 2024-03-26T17:47:46.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.613+0000 7f23aa54e640 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x565089ca6e00 0x565089ca9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.613+0000 7f23aa54e640 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x565089ca6e00 0x565089ca9240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2b6c4188f30f345 server_cookie=4f72971e1a7560c3 in_seq=0 out_seq=0 2024-03-26T17:47:46.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.613+0000 7f238dffb640 10 client.28652 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:47:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f238dffb640 1 -- 172.21.15.105:0/1012960817 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x565089bfeeb0 con 0x565089ca6e00 2024-03-26T17:47:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f238dffb640 10 client.28652 handle_command_reply: tid=0 2024-03-26T17:47:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 client.28652 shutdown 2024-03-26T17:47:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 -- 172.21.15.105:0/1012960817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 msgr2=0x565089ca2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089ca2500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f239c032420 tx=0x7f239c0bdc40 comp rx=0 tx=0).stop 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 -- 172.21.15.105:0/1012960817 shutdown_connections 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x565089ca6e00 0x565089ca9240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c969a0 0x565089ca2500 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f238dffb640 0 client.28652 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c91830 0x565089c9b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 --2- 172.21.15.105:0/1012960817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 -- 172.21.15.105:0/1012960817 >> 172.21.15.105:0/1012960817 conn(0x565089c87fa0 msgr2=0x565089c871c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 -- 172.21.15.105:0/1012960817 shutdown_connections 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 1 -- 172.21.15.105:0/1012960817 wait complete. 2024-03-26T17:47:46.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.614+0000 7f23af39e740 20 client.28652 trim_cache size 0 max 16384 2024-03-26T17:47:46.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23af39e740 1 Processor -- start 2024-03-26T17:47:46.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23af39e740 1 -- start start 2024-03-26T17:47:46.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23af39e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23af39e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089ca24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23af39e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c969a0 0x565089c9a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23af39e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f238800e4e0 con 0x565089c969a0 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23af39e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2388010020 con 0x565089c91830 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23af39e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f238800ebb0 con 0x565089c90e40 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23aad4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c969a0 0x565089c9a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23a9d4d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089ca24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23aad4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c969a0 0x565089c9a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59536/0 (socket says 172.21.15.105:59536) 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.635+0000 7f23aad4f640 1 -- 172.21.15.105:0/4126936546 learned_addr learned my addr 172.21.15.105:0/4126936546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23aa54e640 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23a9d4d640 1 -- 172.21.15.105:0/4126936546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 msgr2=0x565089c9a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23a9d4d640 1 -- 172.21.15.105:0/4126936546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c969a0 msgr2=0x565089c9a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c969a0 0x565089c9a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23a9d4d640 1 -- 172.21.15.105:0/4126936546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565089c87ae0 con 0x565089c91830 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23aad4f640 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c969a0 0x565089c9a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23aa54e640 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089ca24c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2394014010 tx=0x7f239401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=886f3b24db6d1d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f238ffff640 1 -- 172.21.15.105:0/4126936546 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394003b40 con 0x565089c91830 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f238ffff640 1 -- 172.21.15.105:0/4126936546 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2394003ce0 con 0x565089c91830 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.636+0000 7f238ffff640 1 -- 172.21.15.105:0/4126936546 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394004d50 con 0x565089c91830 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 -- 172.21.15.105:0/4126936546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 msgr2=0x565089ca24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089ca24c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2394014010 tx=0x7f239401b000 comp rx=0 tx=0).stop 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 -- 172.21.15.105:0/4126936546 shutdown_connections 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c969a0 0x565089c9a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089ca24c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 --2- 172.21.15.105:0/4126936546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c90e40 0x565089c9a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 -- 172.21.15.105:0/4126936546 >> 172.21.15.105:0/4126936546 conn(0x565089c87fa0 msgr2=0x565089c88380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 -- 172.21.15.105:0/4126936546 shutdown_connections 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 -- 172.21.15.105:0/4126936546 wait complete. 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 -- start start 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.637+0000 7f23af39e740 1 Processor -- start 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23af39e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c90e40 0x565089c9f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23af39e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089c9fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23af39e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c969a0 0x565089ca0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23af39e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f238800f950 con 0x565089c90e40 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23af39e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f238800f280 con 0x565089c91830 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23af39e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f238800c4c0 con 0x565089c969a0 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23af39e740 10 client.0 _ll_register_callbacks cb 0x7f239c0bd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c90e40 0x565089c9f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c90e40 0x565089c9f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59540/0 (socket says 172.21.15.105:59540) 2024-03-26T17:47:46.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 -- 172.21.15.105:0/3709348540 learned_addr learned my addr 172.21.15.105:0/3709348540 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089c9fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aad4f640 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c969a0 0x565089ca0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 -- 172.21.15.105:0/3709348540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c969a0 msgr2=0x565089ca0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c969a0 0x565089ca0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 -- 172.21.15.105:0/3709348540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 msgr2=0x565089c9fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089c9fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 -- 172.21.15.105:0/3709348540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565089ca1480 con 0x565089c90e40 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23aa54e640 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c90e40 0x565089c9f940 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x565089ca9a90 tx=0x7f23a0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d079073031c62bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f238dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f238dffb640 1 -- 172.21.15.105:0/3709348540 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0012dc0 con 0x565089c90e40 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f238dffb640 1 -- 172.21.15.105:0/3709348540 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f23a0005020 con 0x565089c90e40 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.638+0000 7f23af39e740 1 -- 172.21.15.105:0/3709348540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2394003ce0 con 0x565089c90e40 2024-03-26T17:47:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f238dffb640 1 -- 172.21.15.105:0/3709348540 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f23a0005220 con 0x565089c90e40 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23af39e740 10 client.18831 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23af39e740 1 -- 172.21.15.105:0/3709348540 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565089d0d180 con 0x565089c90e40 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f238dffb640 1 -- 172.21.15.105:0/3709348540 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2394003ce0 con 0x565089c90e40 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f238dffb640 1 -- 172.21.15.105:0/3709348540 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f23a00104c0 con 0x565089c90e40 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23af39e740 10 client.18831 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23af39e740 10 client.18831 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23af39e740 20 client.18831 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23af39e740 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x565089ca6e00 0x565089d0f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23af39e740 4 client.18831 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23af39e740 1 -- 172.21.15.105:0/3709348540 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565089bf9200 con 0x565089ca6e00 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.639+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x565089ca6e00 0x565089d0f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23a9d4d640 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x565089ca6e00 0x565089d0f9d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3015ea991d62cfd9 server_cookie=99938c906596fc58 in_seq=0 out_seq=0 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f238dffb640 10 client.18831 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:47:46.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f238dffb640 1 -- 172.21.15.105:0/3709348540 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565089bf9200 con 0x565089ca6e00 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f238dffb640 10 client.18831 handle_command_reply: tid=0 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 client.18831 shutdown 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 -- 172.21.15.105:0/3709348540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c90e40 msgr2=0x565089c9f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c90e40 0x565089c9f940 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x565089ca9a90 tx=0x7f23a0004600 comp rx=0 tx=0).stop 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 -- 172.21.15.105:0/3709348540 shutdown_connections 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x565089ca6e00 0x565089d0f9d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565089c969a0 0x565089ca0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565089c91830 0x565089c9fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 --2- 172.21.15.105:0/3709348540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565089c90e40 0x565089c9f940 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 -- 172.21.15.105:0/3709348540 >> 172.21.15.105:0/3709348540 conn(0x565089c87fa0 msgr2=0x565089c9a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f238dffb640 0 client.18831 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 -- 172.21.15.105:0/3709348540 shutdown_connections 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 1 -- 172.21.15.105:0/3709348540 wait complete. 2024-03-26T17:47:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.640+0000 7f23af39e740 20 client.18831 trim_cache size 0 max 16384 2024-03-26T17:47:46.640 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 -- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23900a2fa0 msgr2=0x7f23900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23900a2fa0 0x7f23900a5460 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2394009460 tx=0x7f23940099a0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 -- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 msgr2=0x7f23a419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 0x7f23a419f070 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f239c02c630 tx=0x7f239c0328a0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 -- 172.21.15.105:0/2976327196 shutdown_connections 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23900a2fa0 0x7f23900a5460 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23a41034a0 0x7f23a419f070 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23a4102ae0 0x7f23a4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 --2- 172.21.15.105:0/2976327196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a41018f0 0x7f23a4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 -- 172.21.15.105:0/2976327196 >> 172.21.15.105:0/2976327196 conn(0x7f23a40fce60 msgr2=0x7f23a4105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 -- 172.21.15.105:0/2976327196 shutdown_connections 2024-03-26T17:47:46.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.641+0000 7f23ac7b2640 1 -- 172.21.15.105:0/2976327196 wait complete. 2024-03-26T17:47:46.645 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:47:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 -- 172.21.15.105:0/2426815822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 msgr2=0x7fb6b0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 --2- 172.21.15.105:0/2426815822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 0x7fb6b0108c00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009ca0 tx=0x7fb6a802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 -- 172.21.15.105:0/2426815822 shutdown_connections 2024-03-26T17:47:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 --2- 172.21.15.105:0/2426815822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 0x7fb6b0108c00 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 --2- 172.21.15.105:0/2426815822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b01019b0 0x7fb6b0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 --2- 172.21.15.105:0/2426815822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6b01007c0 0x7fb6b0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 -- 172.21.15.105:0/2426815822 >> 172.21.15.105:0/2426815822 conn(0x7fb6b00fbd50 msgr2=0x7fb6b00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 -- 172.21.15.105:0/2426815822 shutdown_connections 2024-03-26T17:47:46.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.770+0000 7fb6b6762640 1 -- 172.21.15.105:0/2426815822 wait complete. 2024-03-26T17:47:46.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.771+0000 7fb6b6762640 1 Processor -- start 2024-03-26T17:47:46.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.771+0000 7fb6b6762640 1 -- start start 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.771+0000 7fb6b6762640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6b01007c0 0x7fb6b0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.771+0000 7fb6b6762640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b01019b0 0x7fb6b0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.771+0000 7fb6b6762640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 0x7fb6b019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.771+0000 7fb6b6762640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b010c2d0 con 0x7fb6b0102370 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.771+0000 7fb6b6762640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b010c150 con 0x7fb6b01019b0 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.771+0000 7fb6b6762640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b010c450 con 0x7fb6b01007c0 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6b01007c0 0x7fb6b0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6af7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b01019b0 0x7fb6b0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6b01007c0 0x7fb6b0197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33224/0 (socket says 172.21.15.105:33224) 2024-03-26T17:47:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 -- 172.21.15.105:0/3819634383 learned_addr learned my addr 172.21.15.105:0/3819634383 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 0x7fb6b019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 -- 172.21.15.105:0/3819634383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b01019b0 msgr2=0x7fb6b0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b01019b0 0x7fb6b0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 -- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 msgr2=0x7fb6b019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 0x7fb6b019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 -- 172.21.15.105:0/3819634383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b019f710 con 0x7fb6b01007c0 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 0x7fb6b019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b01019b0 0x7fb6b0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6affff640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6b01007c0 0x7fb6b0197af0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000e9c0 tx=0x7fb6a000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6ad7fa640 1 -- 172.21.15.105:0/3819634383 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000cd80 con 0x7fb6b01007c0 2024-03-26T17:47:46.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6ad7fa640 1 -- 172.21.15.105:0/3819634383 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6a00035f0 con 0x7fb6b01007c0 2024-03-26T17:47:46.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6b6762640 1 -- 172.21.15.105:0/3819634383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b019fa00 con 0x7fb6b01007c0 2024-03-26T17:47:46.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6ad7fa640 1 -- 172.21.15.105:0/3819634383 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0010690 con 0x7fb6b01007c0 2024-03-26T17:47:46.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.772+0000 7fb6b6762640 1 -- 172.21.15.105:0/3819634383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b019ff50 con 0x7fb6b01007c0 2024-03-26T17:47:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.775+0000 7fb6ad7fa640 1 -- 172.21.15.105:0/3819634383 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb6a00037a0 con 0x7fb6b01007c0 2024-03-26T17:47:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.775+0000 7fb6b942b740 1 Processor -- start 2024-03-26T17:47:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.775+0000 7fb6b942b740 1 -- start start 2024-03-26T17:47:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.775+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0097a30 0x55c8c0097e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0098420 0x55c8c009cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c009fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b942b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8bfffc970 con 0x55c8c0097a30 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b942b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c8bffa4d70 con 0x55c8c009d590 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b942b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8bfff8920 con 0x55c8c0098420 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b4cff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0097a30 0x55c8c0097e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b4cff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0097a30 0x55c8c0097e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59556/0 (socket says 172.21.15.105:59556) 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b4cff640 1 -- 172.21.15.105:0/2899373980 learned_addr learned my addr 172.21.15.105:0/2899373980 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c009fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6affff640 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0098420 0x55c8c009cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6ad7fa640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6980a2f20 0x7fb6980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6af7fe640 1 -- 172.21.15.105:0/2899373980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0098420 msgr2=0x55c8c009cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0098420 0x55c8c009cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6af7fe640 1 -- 172.21.15.105:0/2899373980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0097a30 msgr2=0x55c8c0097e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0097a30 0x55c8c0097e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6af7fe640 1 -- 172.21.15.105:0/2899373980 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8bfd103f0 con 0x55c8c009d590 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0097a30 0x55c8c0097e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6980a2f20 0x7fb6980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6ad7fa640 1 -- 172.21.15.105:0/3819634383 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb6a00cf7a0 con 0x7fb6b01007c0 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c009fa20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009fa0 tx=0x7fb6a40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=947ba7c6329b76bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.776+0000 7fb6967fc640 1 -- 172.21.15.105:0/2899373980 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4004030 con 0x55c8c009d590 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6967fc640 1 -- 172.21.15.105:0/2899373980 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6a40054d0 con 0x55c8c009d590 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6980a2f20 0x7fb6980a53e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55c8c0098c80 tx=0x7fb6a802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6967fc640 1 -- 172.21.15.105:0/2899373980 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4004c40 con 0x55c8c009d590 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 -- 172.21.15.105:0/2899373980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 msgr2=0x55c8c009fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c009fa20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009fa0 tx=0x7fb6a40118d0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 -- 172.21.15.105:0/2899373980 shutdown_connections 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c009fa20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0098420 0x55c8c009cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2899373980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0097a30 0x55c8c0097e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 -- 172.21.15.105:0/2899373980 >> 172.21.15.105:0/2899373980 conn(0x55c8c0092fa0 msgr2=0x55c8c00933a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 -- 172.21.15.105:0/2899373980 shutdown_connections 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.777+0000 7fb6b942b740 1 -- 172.21.15.105:0/2899373980 wait complete. 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 1 -- start start 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 1 Processor -- start 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0097a30 0x55c8c00a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c009d590 0x55c8c00a9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c8c00a0540 con 0x55c8c0098420 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c8bffb8310 con 0x55c8c0097a30 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c8c00a06c0 con 0x55c8c009d590 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b942b740 10 client.0 _ll_register_callbacks cb 0x55c8bfff3c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6affff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0097a30 0x55c8c00a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6affff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0097a30 0x55c8c00a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40770/0 (socket says 172.21.15.105:40770) 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6affff640 1 -- 172.21.15.105:0/2121103942 learned_addr learned my addr 172.21.15.105:0/2121103942 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.778+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c009d590 0x55c8c00a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b4cff640 1 -- 172.21.15.105:0/2121103942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0097a30 msgr2=0x55c8c00a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0097a30 0x55c8c00a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b4cff640 1 -- 172.21.15.105:0/2121103942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 msgr2=0x55c8c00a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b4cff640 1 -- 172.21.15.105:0/2121103942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8bfdda7d0 con 0x55c8c009d590 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c009d590 0x55c8c00a9490 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8032380 tx=0x7fb6a80324f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dd02af064263db9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb68f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb68f7fe640 1 -- 172.21.15.105:0/2121103942 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8052060 con 0x55c8c009d590 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb68f7fe640 1 -- 172.21.15.105:0/2121103942 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb6a8003b70 con 0x55c8c009d590 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b942b740 1 -- 172.21.15.105:0/2121103942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a40054d0 con 0x55c8c009d590 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb68f7fe640 1 -- 172.21.15.105:0/2121103942 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6a8046070 con 0x55c8c009d590 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b942b740 10 client.28678 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.779+0000 7fb6b942b740 1 -- 172.21.15.105:0/2121103942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8c00ac870 con 0x55c8c009d590 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.780+0000 7fb68f7fe640 1 -- 172.21.15.105:0/2121103942 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6a40054d0 con 0x55c8c009d590 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.780+0000 7fb68f7fe640 1 -- 172.21.15.105:0/2121103942 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb6a803f040 con 0x55c8c009d590 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.780+0000 7fb6b942b740 10 client.28678 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.780+0000 7fb6b942b740 10 client.28678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.780+0000 7fb6b942b740 20 client.28678 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.780+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c8c00adf90 0x55c8c00b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.780+0000 7fb6b942b740 4 client.28678 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.780+0000 7fb6b942b740 1 -- 172.21.15.105:0/2121103942 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8c0009eb0 con 0x55c8c00adf90 2024-03-26T17:47:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.781+0000 7fb6affff640 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c8c00adf90 0x55c8c00b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.781+0000 7fb6affff640 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c8c00adf90 0x55c8c00b0450 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b0b23c004860014 server_cookie=4712fe8543bcf0f2 in_seq=0 out_seq=0 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.781+0000 7fb68f7fe640 10 client.28678 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb68f7fe640 1 -- 172.21.15.105:0/2121103942 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c8c0009eb0 con 0x55c8c00adf90 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb68f7fe640 10 client.28678 handle_command_reply: tid=0 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 client.28678 shutdown 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 -- 172.21.15.105:0/2121103942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c009d590 msgr2=0x55c8c00a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c009d590 0x55c8c00a9490 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8032380 tx=0x7fb6a80324f0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 -- 172.21.15.105:0/2121103942 shutdown_connections 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c009d590 0x55c8c00a9490 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c8c00adf90 0x55c8c00b0450 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 --2- 172.21.15.105:0/2121103942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0097a30 0x55c8c00a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb68f7fe640 0 client.28678 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 -- 172.21.15.105:0/2121103942 >> 172.21.15.105:0/2121103942 conn(0x55c8c0092fa0 msgr2=0x55c8c00921c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 -- 172.21.15.105:0/2121103942 shutdown_connections 2024-03-26T17:47:46.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 1 -- 172.21.15.105:0/2121103942 wait complete. 2024-03-26T17:47:46.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.782+0000 7fb6b942b740 20 client.28678 trim_cache size 0 max 16384 2024-03-26T17:47:46.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.799+0000 7fb6b942b740 1 Processor -- start 2024-03-26T17:47:46.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b942b740 1 -- start start 2024-03-26T17:47:46.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c0096910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0098420 0x55c8c00a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c009d590 0x55c8c00b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b942b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01a8d40 con 0x55c8c009d590 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b942b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01aa880 con 0x55c8c0098420 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b942b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01a9410 con 0x55c8c0097a30 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c0096910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b4cff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c009d590 0x55c8c00b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c0096910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33256/0 (socket says 172.21.15.105:33256) 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6affff640 1 -- 172.21.15.105:0/509488582 learned_addr learned my addr 172.21.15.105:0/509488582 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0098420 0x55c8c00a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b4cff640 1 -- 172.21.15.105:0/509488582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 msgr2=0x55c8c0096910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c0096910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b4cff640 1 -- 172.21.15.105:0/509488582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0098420 msgr2=0x55c8c00a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0098420 0x55c8c00a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b4cff640 1 -- 172.21.15.105:0/509488582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c0098800 con 0x55c8c009d590 2024-03-26T17:47:46.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6affff640 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c0096910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.800+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c009d590 0x55c8c00b4d10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb6a804d4c0 tx=0x7fb6a8041e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8650c73a29811ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb695ffb640 1 -- 172.21.15.105:0/509488582 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8052070 con 0x55c8c009d590 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb695ffb640 1 -- 172.21.15.105:0/509488582 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6a8009b20 con 0x55c8c009d590 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb695ffb640 1 -- 172.21.15.105:0/509488582 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8039040 con 0x55c8c009d590 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 -- 172.21.15.105:0/509488582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c009d590 msgr2=0x55c8c00b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c009d590 0x55c8c00b4d10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb6a804d4c0 tx=0x7fb6a8041e20 comp rx=0 tx=0).stop 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 -- 172.21.15.105:0/509488582 shutdown_connections 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c009d590 0x55c8c00b4d10 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c0098420 0x55c8c00a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 --2- 172.21.15.105:0/509488582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c0096910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 -- 172.21.15.105:0/509488582 >> 172.21.15.105:0/509488582 conn(0x55c8c0092fa0 msgr2=0x55c8c0096200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 -- 172.21.15.105:0/509488582 shutdown_connections 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.801+0000 7fb6b942b740 1 -- 172.21.15.105:0/509488582 wait complete. 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 1 -- start start 2024-03-26T17:47:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 1 Processor -- start 2024-03-26T17:47:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c00a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c00a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01aa1b0 con 0x55c8c0098420 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01a9ae0 con 0x55c8c009d590 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01a6d20 con 0x55c8c0097a30 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b942b740 10 client.0 _ll_register_callbacks cb 0x7fb6a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c00a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59582/0 (socket says 172.21.15.105:59582) 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6af7fe640 1 -- 172.21.15.105:0/3437578846 learned_addr learned my addr 172.21.15.105:0/3437578846 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.802+0000 7fb6b4cff640 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c00a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6af7fe640 1 -- 172.21.15.105:0/3437578846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 msgr2=0x55c8c00a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c00a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6af7fe640 1 -- 172.21.15.105:0/3437578846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 msgr2=0x55c8c00a7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c00a7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6af7fe640 1 -- 172.21.15.105:0/3437578846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c00a8500 con 0x55c8c0098420 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6af7fe640 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a6f00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009fd0 tx=0x7fb6a40055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af59ee58b4f8f023 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb68f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb68f7fe640 1 -- 172.21.15.105:0/3437578846 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a401d040 con 0x55c8c0098420 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb68f7fe640 1 -- 172.21.15.105:0/3437578846 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fb6a40035f0 con 0x55c8c0098420 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6b942b740 1 -- 172.21.15.105:0/3437578846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a8009b20 con 0x55c8c0098420 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb68f7fe640 1 -- 172.21.15.105:0/3437578846 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb6a40037f0 con 0x55c8c0098420 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6b942b740 10 client.18849 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6b942b740 1 -- 172.21.15.105:0/3437578846 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8c0114350 con 0x55c8c0098420 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb68f7fe640 1 -- 172.21.15.105:0/3437578846 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6a8009b20 con 0x55c8c0098420 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb68f7fe640 1 -- 172.21.15.105:0/3437578846 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb6a400fc20 con 0x55c8c0098420 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6b942b740 10 client.18849 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6b942b740 10 client.18849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6b942b740 20 client.18849 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.803+0000 7fb6b942b740 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c8c0114a00 0x55c8c0116e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb6b942b740 4 client.18849 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb6b942b740 1 -- 172.21.15.105:0/3437578846 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8c0004200 con 0x55c8c0114a00 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb6affff640 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c8c0114a00 0x55c8c0116e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb68f7fe640 10 client.18849 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb6affff640 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c8c0114a00 0x55c8c0116e20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=685fcd14b1a478bb server_cookie=fb20319e10b60566 in_seq=0 out_seq=0 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb68f7fe640 1 -- 172.21.15.105:0/3437578846 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8c0004200 con 0x55c8c0114a00 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb68f7fe640 10 client.18849 handle_command_reply: tid=0 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb6b942b740 1 client.18849 shutdown 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb6b942b740 1 -- 172.21.15.105:0/3437578846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 msgr2=0x55c8c00a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.804+0000 7fb6b942b740 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a6f00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009fd0 tx=0x7fb6a40055f0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 1 -- 172.21.15.105:0/3437578846 shutdown_connections 2024-03-26T17:47:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55c8c0114a00 0x55c8c0116e20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c8c009d590 0x55c8c00a7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c8c0098420 0x55c8c00a6f00 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb68f7fe640 0 client.18849 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 1 --2- 172.21.15.105:0/3437578846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c8c0097a30 0x55c8c00a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 1 -- 172.21.15.105:0/3437578846 >> 172.21.15.105:0/3437578846 conn(0x55c8c0092fa0 msgr2=0x55c8c0096a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 1 -- 172.21.15.105:0/3437578846 shutdown_connections 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 1 -- 172.21.15.105:0/3437578846 wait complete. 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b942b740 20 client.18849 trim_cache size 0 max 16384 2024-03-26T17:47:46.804 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:46.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 -- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6980a2f20 msgr2=0x7fb6980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6980a2f20 0x7fb6980a53e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55c8c0098c80 tx=0x7fb6a802e040 comp rx=0 tx=0).stop 2024-03-26T17:47:46.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 -- 172.21.15.105:0/3819634383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6b01007c0 msgr2=0x7fb6b0197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6b01007c0 0x7fb6b0197af0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000e9c0 tx=0x7fb6a000ee80 comp rx=0 tx=0).stop 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 -- 172.21.15.105:0/3819634383 shutdown_connections 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6980a2f20 0x7fb6980a53e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6b0102370 0x7fb6b019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6b01019b0 0x7fb6b0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 --2- 172.21.15.105:0/3819634383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6b01007c0 0x7fb6b0197af0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 -- 172.21.15.105:0/3819634383 >> 172.21.15.105:0/3819634383 conn(0x7fb6b00fbd50 msgr2=0x7fb6b00fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 -- 172.21.15.105:0/3819634383 shutdown_connections 2024-03-26T17:47:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.805+0000 7fb6b6762640 1 -- 172.21.15.105:0/3819634383 wait complete. 2024-03-26T17:47:46.812 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 -- 172.21.15.105:0/3081733133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d4103480 msgr2=0x7f24d4109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 --2- 172.21.15.105:0/3081733133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d4103480 0x7f24d4109d10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009ca0 tx=0x7f24cc02c100 comp rx=0 tx=0).stop 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 -- 172.21.15.105:0/3081733133 shutdown_connections 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 --2- 172.21.15.105:0/3081733133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d4103480 0x7f24d4109d10 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 --2- 172.21.15.105:0/3081733133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d4102ac0 0x7f24d4102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 --2- 172.21.15.105:0/3081733133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d41018d0 0x7f24d4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 -- 172.21.15.105:0/3081733133 >> 172.21.15.105:0/3081733133 conn(0x7f24d40fce40 msgr2=0x7f24d40ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 -- 172.21.15.105:0/3081733133 shutdown_connections 2024-03-26T17:47:46.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.938+0000 7f24d90dc640 1 -- 172.21.15.105:0/3081733133 wait complete. 2024-03-26T17:47:46.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d90dc640 1 Processor -- start 2024-03-26T17:47:46.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d90dc640 1 -- start start 2024-03-26T17:47:46.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d90dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d41018d0 0x7f24d4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d90dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d4102ac0 0x7f24d4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d90dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d4103480 0x7f24d419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d90dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d3e0 con 0x7f24d41018d0 2024-03-26T17:47:46.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d90dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d260 con 0x7f24d4103480 2024-03-26T17:47:46.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d90dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d410d560 con 0x7f24d4102ac0 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d41018d0 0x7f24d4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d4102ac0 0x7f24d4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d4103480 0x7f24d419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d41018d0 0x7f24d4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59604/0 (socket says 172.21.15.105:59604) 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d2d9d640 1 -- 172.21.15.105:0/2890388982 learned_addr learned my addr 172.21.15.105:0/2890388982 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d259c640 1 -- 172.21.15.105:0/2890388982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d4103480 msgr2=0x7f24d419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d259c640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d4103480 0x7f24d419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d259c640 1 -- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d41018d0 msgr2=0x7f24d4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d259c640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d41018d0 0x7f24d4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d259c640 1 -- 172.21.15.105:0/2890388982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d419f6d0 con 0x7f24d4102ac0 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.939+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d41018d0 0x7f24d4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.940+0000 7f24d259c640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d4102ac0 0x7f24d4197ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e9f0 tx=0x7f24c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.940+0000 7f24bbfff640 1 -- 172.21.15.105:0/2890388982 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c800ccc0 con 0x7f24d4102ac0 2024-03-26T17:47:46.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.940+0000 7f24d90dc640 1 -- 172.21.15.105:0/2890388982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d419f9c0 con 0x7f24d4102ac0 2024-03-26T17:47:46.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.940+0000 7f24bbfff640 1 -- 172.21.15.105:0/2890388982 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c80108e0 con 0x7f24d4102ac0 2024-03-26T17:47:46.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.940+0000 7f24bbfff640 1 -- 172.21.15.105:0/2890388982 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8018880 con 0x7f24d4102ac0 2024-03-26T17:47:46.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.940+0000 7f24d90dc640 1 -- 172.21.15.105:0/2890388982 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d419ff10 con 0x7f24d4102ac0 2024-03-26T17:47:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24bbfff640 1 -- 172.21.15.105:0/2890388982 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f24c8018a20 con 0x7f24d4102ac0 2024-03-26T17:47:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24dbd9e740 1 Processor -- start 2024-03-26T17:47:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24dbd9e740 1 -- start start 2024-03-26T17:47:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8aca30 0x55d66c8ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8b2590 0x55d66c8b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d66c811970 con 0x55d66c8ad420 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d66c7b9d70 con 0x55d66c8b2590 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d66c80d920 con 0x55d66c8aca30 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24bbfff640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24bc0a2f20 0x7f24bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8b1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59618/0 (socket says 172.21.15.105:59618) 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24d2d9d640 1 -- 172.21.15.105:0/842587295 learned_addr learned my addr 172.21.15.105:0/842587295 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24d259c640 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8b2590 0x55d66c8b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24d359e640 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8aca30 0x55d66c8ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24d359e640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24bc0a2f20 0x7f24bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.943+0000 7f24bbfff640 1 -- 172.21.15.105:0/2890388982 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f24c80cf750 con 0x7f24d4102ac0 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24d2d9d640 1 -- 172.21.15.105:0/842587295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8aca30 msgr2=0x55d66c8ace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8aca30 0x55d66c8ace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24d2d9d640 1 -- 172.21.15.105:0/842587295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8b2590 msgr2=0x55d66c8b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8b2590 0x55d66c8b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24d2d9d640 1 -- 172.21.15.105:0/842587295 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d66c5253f0 con 0x55d66c8ad420 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24d359e640 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8aca30 0x55d66c8ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8b1f20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f24c40115c0 tx=0x7f24c4011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99c9ee34c9723d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24d359e640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24bc0a2f20 0x7f24bc0a53e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f24cc032380 tx=0x7f24cc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24b97fa640 1 -- 172.21.15.105:0/842587295 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4005020 con 0x55d66c8ad420 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24b97fa640 1 -- 172.21.15.105:0/842587295 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c40047a0 con 0x55d66c8ad420 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.944+0000 7f24b97fa640 1 -- 172.21.15.105:0/842587295 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c400f8a0 con 0x55d66c8ad420 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 -- 172.21.15.105:0/842587295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 msgr2=0x55d66c8b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8b1f20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f24c40115c0 tx=0x7f24c4011a80 comp rx=0 tx=0).stop 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 -- 172.21.15.105:0/842587295 shutdown_connections 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8b2590 0x55d66c8b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8b1f20 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/842587295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8aca30 0x55d66c8ace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 -- 172.21.15.105:0/842587295 >> 172.21.15.105:0/842587295 conn(0x55d66c8a7fa0 msgr2=0x55d66c8a83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 -- 172.21.15.105:0/842587295 shutdown_connections 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.945+0000 7f24dbd9e740 1 -- 172.21.15.105:0/842587295 wait complete. 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 1 -- start start 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 1 Processor -- start 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8aca30 0x55d66c8b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8ad420 0x55d66c8b7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c8be4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d66c8b5540 con 0x55d66c8aca30 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d66c7cd310 con 0x55d66c8ad420 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d66c8b56c0 con 0x55d66c8b2590 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24dbd9e740 10 client.0 _ll_register_callbacks cb 0x55d66c808c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:46.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8aca30 0x55d66c8b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24d359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c8be4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8aca30 0x55d66c8b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59624/0 (socket says 172.21.15.105:59624) 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24d2d9d640 1 -- 172.21.15.105:0/3101468691 learned_addr learned my addr 172.21.15.105:0/3101468691 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.946+0000 7f24d259c640 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8ad420 0x55d66c8b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24d2d9d640 1 -- 172.21.15.105:0/3101468691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 msgr2=0x55d66c8be4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c8be4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24d2d9d640 1 -- 172.21.15.105:0/3101468691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8ad420 msgr2=0x55d66c8b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8ad420 0x55d66c8b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24d2d9d640 1 -- 172.21.15.105:0/3101468691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66c5ef7d0 con 0x55d66c8aca30 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24d359e640 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c8be4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8aca30 0x55d66c8b6f40 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f24c40058c0 tx=0x7f24c4004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15be7b36421cd936 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24b6ff5640 1 -- 172.21.15.105:0/3101468691 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c401d050 con 0x55d66c8aca30 2024-03-26T17:47:46.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24b6ff5640 1 -- 172.21.15.105:0/3101468691 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f24c4005000 con 0x55d66c8aca30 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24dbd9e740 1 -- 172.21.15.105:0/3101468691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c40047a0 con 0x55d66c8aca30 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24b6ff5640 1 -- 172.21.15.105:0/3101468691 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24c4005220 con 0x55d66c8aca30 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24dbd9e740 10 client.18861 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24dbd9e740 1 -- 172.21.15.105:0/3101468691 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d66c8c1830 con 0x55d66c8aca30 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.947+0000 7f24b6ff5640 1 -- 172.21.15.105:0/3101468691 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c40047a0 con 0x55d66c8aca30 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24b6ff5640 1 -- 172.21.15.105:0/3101468691 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f24c40035f0 con 0x55d66c8aca30 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24dbd9e740 10 client.18861 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24dbd9e740 10 client.18861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24dbd9e740 20 client.18861 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d66c8c2d50 0x55d66c8c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24dbd9e740 4 client.18861 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24dbd9e740 1 -- 172.21.15.105:0/3101468691 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d66c81eeb0 con 0x55d66c8c2d50 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24d259c640 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d66c8c2d50 0x55d66c8c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24b6ff5640 10 client.18861 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:47:46.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.948+0000 7f24d259c640 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d66c8c2d50 0x55d66c8c51c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=686d350e11f264 server_cookie=5e8f7aeec1633dae in_seq=0 out_seq=0 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24b6ff5640 1 -- 172.21.15.105:0/3101468691 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d66c81eeb0 con 0x55d66c8c2d50 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24b6ff5640 10 client.18861 handle_command_reply: tid=0 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 client.18861 shutdown 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 -- 172.21.15.105:0/3101468691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8aca30 msgr2=0x55d66c8b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8aca30 0x55d66c8b6f40 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f24c40058c0 tx=0x7f24c4004250 comp rx=0 tx=0).stop 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 -- 172.21.15.105:0/3101468691 shutdown_connections 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d66c8c2d50 0x55d66c8c51c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c8be4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8ad420 0x55d66c8b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24b6ff5640 0 client.18861 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/3101468691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8aca30 0x55d66c8b6f40 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 -- 172.21.15.105:0/3101468691 >> 172.21.15.105:0/3101468691 conn(0x55d66c8a7fa0 msgr2=0x55d66c8a71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 -- 172.21.15.105:0/3101468691 shutdown_connections 2024-03-26T17:47:46.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 1 -- 172.21.15.105:0/3101468691 wait complete. 2024-03-26T17:47:46.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.949+0000 7f24dbd9e740 20 client.18861 trim_cache size 0 max 16384 2024-03-26T17:47:46.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24dbd9e740 1 Processor -- start 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24dbd9e740 1 -- start start 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8ad420 0x55d66c8be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8b2590 0x55d66c8a7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41a6c20 con 0x55d66c8b2590 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41a8760 con 0x55d66c8aca30 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41a72f0 con 0x55d66c8ad420 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24d359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8b2590 0x55d66c8a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24d2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24d359e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8b2590 0x55d66c8a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59640/0 (socket says 172.21.15.105:59640) 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24d259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8ad420 0x55d66c8be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.968+0000 7f24d359e640 1 -- 172.21.15.105:0/588511429 learned_addr learned my addr 172.21.15.105:0/588511429 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24d259c640 1 -- 172.21.15.105:0/588511429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 msgr2=0x55d66c8caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24d259c640 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24d259c640 1 -- 172.21.15.105:0/588511429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8b2590 msgr2=0x55d66c8a7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24d259c640 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8b2590 0x55d66c8a7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24d259c640 1 -- 172.21.15.105:0/588511429 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d66c8b5420 con 0x55d66c8ad420 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24d359e640 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8b2590 0x55d66c8a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24d259c640 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8ad420 0x55d66c8be480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f24c80cd1e0 tx=0x7f24c80bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd95a0c8de8b2b5a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24b8ff9640 1 -- 172.21.15.105:0/588511429 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80a4850 con 0x55d66c8ad420 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24b8ff9640 1 -- 172.21.15.105:0/588511429 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c8099f90 con 0x55d66c8ad420 2024-03-26T17:47:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.969+0000 7f24b8ff9640 1 -- 172.21.15.105:0/588511429 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80babb0 con 0x55d66c8ad420 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- 172.21.15.105:0/588511429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8ad420 msgr2=0x55d66c8be480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8ad420 0x55d66c8be480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f24c80cd1e0 tx=0x7f24c80bc040 comp rx=0 tx=0).stop 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- 172.21.15.105:0/588511429 shutdown_connections 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8b2590 0x55d66c8a7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8ad420 0x55d66c8be480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/588511429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- 172.21.15.105:0/588511429 >> 172.21.15.105:0/588511429 conn(0x55d66c8a7fa0 msgr2=0x55d66c8a6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- 172.21.15.105:0/588511429 shutdown_connections 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- 172.21.15.105:0/588511429 wait complete. 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- start start 2024-03-26T17:47:46.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 Processor -- start 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8bb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8bba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c9270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41a8090 con 0x55d66c8ad420 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41a79c0 con 0x55d66c8aca30 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41a4c00 con 0x55d66c8b2590 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24dbd9e740 10 client.0 _ll_register_callbacks cb 0x7f24c800ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24d259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24d259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8bba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59642/0 (socket says 172.21.15.105:59642) 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24d259c640 1 -- 172.21.15.105:0/2891650970 learned_addr learned my addr 172.21.15.105:0/2891650970 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8bb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.970+0000 7f24d359e640 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c9270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24d259c640 1 -- 172.21.15.105:0/2891650970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 msgr2=0x55d66c9270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24d259c640 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c9270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24d259c640 1 -- 172.21.15.105:0/2891650970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 msgr2=0x55d66c8bb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24d259c640 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8bb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24d259c640 1 -- 172.21.15.105:0/2891650970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66c927670 con 0x55d66c8ad420 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24d259c640 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8bba00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f24c80988c0 tx=0x7f24c8005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a32c9ef90065771 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24b6ff5640 1 -- 172.21.15.105:0/2891650970 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80b6020 con 0x55d66c8ad420 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24b6ff5640 1 -- 172.21.15.105:0/2891650970 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f24c809a1f0 con 0x55d66c8ad420 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24dbd9e740 1 -- 172.21.15.105:0/2891650970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c8099f90 con 0x55d66c8ad420 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24b6ff5640 1 -- 172.21.15.105:0/2891650970 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24c809e070 con 0x55d66c8ad420 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24dbd9e740 10 client.18867 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24dbd9e740 1 -- 172.21.15.105:0/2891650970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d66c928a10 con 0x55d66c8ad420 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24b6ff5640 1 -- 172.21.15.105:0/2891650970 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24c8099f90 con 0x55d66c8ad420 2024-03-26T17:47:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.971+0000 7f24b6ff5640 1 -- 172.21.15.105:0/2891650970 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f24c80b9020 con 0x55d66c8ad420 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24dbd9e740 10 client.18867 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24dbd9e740 10 client.18867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24dbd9e740 20 client.18867 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d66c929410 0x55d66c92b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24dbd9e740 4 client.18867 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24dbd9e740 1 -- 172.21.15.105:0/2891650970 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d66c8ae430 con 0x55d66c929410 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d66c929410 0x55d66c92b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24b6ff5640 10 client.18867 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24d2d9d640 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d66c929410 0x55d66c92b830 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2773a2d213bf3278 server_cookie=d7f36f4a7e54b909 in_seq=0 out_seq=0 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24b6ff5640 1 -- 172.21.15.105:0/2891650970 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d66c8ae430 con 0x55d66c929410 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24b6ff5640 10 client.18867 handle_command_reply: tid=0 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.972+0000 7f24dbd9e740 1 client.18867 shutdown 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 -- 172.21.15.105:0/2891650970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 msgr2=0x55d66c8bba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8bba00 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f24c80988c0 tx=0x7f24c8005bf0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 -- 172.21.15.105:0/2891650970 shutdown_connections 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d66c929410 0x55d66c92b830 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24b6ff5640 0 client.18867 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d66c8b2590 0x55d66c9270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d66c8ad420 0x55d66c8bba00 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 --2- 172.21.15.105:0/2891650970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d66c8aca30 0x55d66c8bb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 -- 172.21.15.105:0/2891650970 >> 172.21.15.105:0/2891650970 conn(0x55d66c8a7fa0 msgr2=0x55d66c8a71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 -- 172.21.15.105:0/2891650970 shutdown_connections 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 1 -- 172.21.15.105:0/2891650970 wait complete. 2024-03-26T17:47:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24dbd9e740 20 client.18867 trim_cache size 0 max 16384 2024-03-26T17:47:46.973 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:46.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 -- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24bc0a2f20 msgr2=0x7f24bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24bc0a2f20 0x7f24bc0a53e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f24cc032380 tx=0x7f24cc004a40 comp rx=0 tx=0).stop 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 -- 172.21.15.105:0/2890388982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d4102ac0 msgr2=0x7f24d4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d4102ac0 0x7f24d4197ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e9f0 tx=0x7f24c800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 -- 172.21.15.105:0/2890388982 shutdown_connections 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24bc0a2f20 0x7f24bc0a53e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24d4103480 0x7f24d419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24d4102ac0 0x7f24d4197ff0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 --2- 172.21.15.105:0/2890388982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24d41018d0 0x7f24d4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.973+0000 7f24d90dc640 1 -- 172.21.15.105:0/2890388982 >> 172.21.15.105:0/2890388982 conn(0x7f24d40fce40 msgr2=0x7f24d4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.974+0000 7f24d90dc640 1 -- 172.21.15.105:0/2890388982 shutdown_connections 2024-03-26T17:47:46.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:46.974+0000 7f24d90dc640 1 -- 172.21.15.105:0/2890388982 wait complete. 2024-03-26T17:47:46.976 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:46 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3125598159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:47:46.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:47:47.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 299505 3.15 MB/sec execute 390 sec latency 18.600 ms 2024-03-26T17:47:47.102 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:46 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3125598159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 -- 172.21.15.105:0/1244555482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1019b0 msgr2=0x7f6d4c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1244555482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1019b0 0x7f6d4c101e30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009900 tx=0x7f6d4002bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 -- 172.21.15.105:0/1244555482 shutdown_connections 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1244555482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d4c102370 0x7f6d4c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1244555482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1019b0 0x7f6d4c101e30 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1244555482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d4c1007c0 0x7f6d4c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 -- 172.21.15.105:0/1244555482 >> 172.21.15.105:0/1244555482 conn(0x7f6d4c0fbd50 msgr2=0x7f6d4c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 -- 172.21.15.105:0/1244555482 shutdown_connections 2024-03-26T17:47:47.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.104+0000 7f6d51a85640 1 -- 172.21.15.105:0/1244555482 wait complete. 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d51a85640 1 Processor -- start 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d51a85640 1 -- start start 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d51a85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1007c0 0x7f6d4c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d51a85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d4c1019b0 0x7f6d4c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d51a85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d4c102370 0x7f6d4c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d51a85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c10c340 con 0x7f6d4c1007c0 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d51a85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c10c1c0 con 0x7f6d4c1019b0 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d51a85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c10c4c0 con 0x7f6d4c102370 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1007c0 0x7f6d4c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d4c102370 0x7f6d4c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1007c0 0x7f6d4c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59672/0 (socket says 172.21.15.105:59672) 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/1453776905 learned_addr learned my addr 172.21.15.105:0/1453776905 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:47.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.105+0000 7f6d4affd640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d4c1019b0 0x7f6d4c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d4bfff640 1 -- 172.21.15.105:0/1453776905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d4c1019b0 msgr2=0x7f6d4c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d4bfff640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d4c1019b0 0x7f6d4c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d4bfff640 1 -- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1007c0 msgr2=0x7f6d4c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d4bfff640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1007c0 0x7f6d4c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d4bfff640 1 -- 172.21.15.105:0/1453776905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4c19f780 con 0x7f6d4c102370 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1007c0 0x7f6d4c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d4affd640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d4c1019b0 0x7f6d4c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d4bfff640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d4c102370 0x7f6d4c19f110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009fd0 tx=0x7f6d4400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d48ff9640 1 -- 172.21.15.105:0/1453776905 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400ce20 con 0x7f6d4c102370 2024-03-26T17:47:47.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d48ff9640 1 -- 172.21.15.105:0/1453776905 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d44005020 con 0x7f6d4c102370 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d51a85640 1 -- 172.21.15.105:0/1453776905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4c19fa70 con 0x7f6d4c102370 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d48ff9640 1 -- 172.21.15.105:0/1453776905 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44010690 con 0x7f6d4c102370 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.106+0000 7f6d51a85640 1 -- 172.21.15.105:0/1453776905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4c19ff90 con 0x7f6d4c102370 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d48ff9640 1 -- 172.21.15.105:0/1453776905 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6d440051d0 con 0x7f6d4c102370 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d5467a740 1 Processor -- start 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d5467a740 1 -- start start 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32ae40 0x55a57d32b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d330330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d3309a0 0x55a57d332e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d5467a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a57d28b970 con 0x55a57d32ae40 2024-03-26T17:47:47.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d5467a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a57d233d70 con 0x55a57d32b830 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d5467a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a57d287920 con 0x55a57d3309a0 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d330330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32ae40 0x55a57d32b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d330330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40852/0 (socket says 172.21.15.105:40852) 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32ae40 0x55a57d32b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59680/0 (socket says 172.21.15.105:59680) 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2386351171 learned_addr learned my addr 172.21.15.105:0/2386351171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2386351171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d3309a0 0x55a57d332e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2386351171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d3309a0 msgr2=0x55a57d332e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2386351171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d3309a0 0x55a57d332e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2386351171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32ae40 msgr2=0x55a57d32b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2386351171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32ae40 0x55a57d32b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.109+0000 7f6d48ff9640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6d340a2f20 0x7f6d340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.110+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2386351171 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57cf9f3f0 con 0x55a57d32b830 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.110+0000 7f6d4affd640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6d340a2f20 0x7f6d340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.110+0000 7f6d48ff9640 1 -- 172.21.15.105:0/1453776905 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f6d440cfb00 con 0x7f6d4c102370 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.110+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2386351171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d330330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0115c0 tx=0x7f6d3c011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76af685211510f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.110+0000 7f6d31ffb640 1 -- 172.21.15.105:0/2386351171 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c005020 con 0x55a57d32b830 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.110+0000 7f6d31ffb640 1 -- 172.21.15.105:0/2386351171 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d3c0047a0 con 0x55a57d32b830 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.110+0000 7f6d31ffb640 1 -- 172.21.15.105:0/2386351171 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00f900 con 0x55a57d32b830 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d4affd640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6d340a2f20 0x7f6d340a53e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6d4002c240 tx=0x7f6d40005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 -- 172.21.15.105:0/2386351171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 msgr2=0x55a57d330330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2386351171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d330330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0115c0 tx=0x7f6d3c011a80 comp rx=0 tx=0).stop 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 -- 172.21.15.105:0/2386351171 shutdown_connections 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2386351171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d3309a0 0x55a57d332e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2386351171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d330330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2386351171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32ae40 0x55a57d32b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 -- 172.21.15.105:0/2386351171 >> 172.21.15.105:0/2386351171 conn(0x55a57d3263b0 msgr2=0x55a57d3267b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 -- 172.21.15.105:0/2386351171 shutdown_connections 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 -- 172.21.15.105:0/2386351171 wait complete. 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 -- start start 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.111+0000 7f6d5467a740 1 Processor -- start 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d335380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d3358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3309a0 0x55a57d33c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d5467a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a57d333950 con 0x55a57d3309a0 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d5467a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a57d247310 con 0x55a57d32b830 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d5467a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a57d333ad0 con 0x55a57d32ae40 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d5467a740 10 client.0 _ll_register_callbacks cb 0x55a57d282c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d335380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d335380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33320/0 (socket says 172.21.15.105:33320) 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2983085124 learned_addr learned my addr 172.21.15.105:0/2983085124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3309a0 0x55a57d33c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d3358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2983085124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 msgr2=0x55a57d3358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d3358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2983085124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3309a0 msgr2=0x55a57d33c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3309a0 0x55a57d33c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2983085124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57d0697d0 con 0x55a57d32ae40 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d335380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c01b040 tx=0x7f6d3c005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc9f0f888b233e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d2affd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.112+0000 7f6d2affd640 1 -- 172.21.15.105:0/2983085124 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c027070 con 0x55a57d32ae40 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d2affd640 1 -- 172.21.15.105:0/2983085124 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f6d3c005230 con 0x55a57d32ae40 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 1 -- 172.21.15.105:0/2983085124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3c0047a0 con 0x55a57d32ae40 2024-03-26T17:47:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d2affd640 1 -- 172.21.15.105:0/2983085124 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c032050 con 0x55a57d32ae40 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 10 client.28714 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 1 -- 172.21.15.105:0/2983085124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a57d33fc70 con 0x55a57d32ae40 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d2affd640 1 -- 172.21.15.105:0/2983085124 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d3c0047a0 con 0x55a57d32ae40 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d2affd640 1 -- 172.21.15.105:0/2983085124 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6d3c022560 con 0x55a57d32ae40 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 10 client.28714 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 10 client.28714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 20 client.28714 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a57d341090 0x55a57d343500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 4 client.28714 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d5467a740 1 -- 172.21.15.105:0/2983085124 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a57d298eb0 con 0x55a57d341090 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.113+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a57d341090 0x55a57d343500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.114+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a57d341090 0x55a57d343500 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3864a75dcef5192 server_cookie=987d877e50a14e07 in_seq=0 out_seq=0 2024-03-26T17:47:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.114+0000 7f6d2affd640 10 client.28714 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.114+0000 7f6d2affd640 1 -- 172.21.15.105:0/2983085124 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a57d298eb0 con 0x55a57d341090 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.114+0000 7f6d2affd640 10 client.28714 handle_command_reply: tid=0 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.114+0000 7f6d5467a740 1 client.28714 shutdown 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 -- 172.21.15.105:0/2983085124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 msgr2=0x55a57d335380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d335380 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c01b040 tx=0x7f6d3c005db0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 -- 172.21.15.105:0/2983085124 shutdown_connections 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a57d341090 0x55a57d343500 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3309a0 0x55a57d33c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d3358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d2affd640 0 client.28714 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2983085124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d335380 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 -- 172.21.15.105:0/2983085124 >> 172.21.15.105:0/2983085124 conn(0x55a57d3263b0 msgr2=0x55a57d3211c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 -- 172.21.15.105:0/2983085124 shutdown_connections 2024-03-26T17:47:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 1 -- 172.21.15.105:0/2983085124 wait complete. 2024-03-26T17:47:47.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.115+0000 7f6d5467a740 20 client.28714 trim_cache size 0 max 16384 2024-03-26T17:47:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.130+0000 7f6d5467a740 1 Processor -- start 2024-03-26T17:47:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.130+0000 7f6d5467a740 1 -- start start 2024-03-26T17:47:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d3263b0 0x55a57d329d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32b830 0x55a57d347dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d5467a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1a6ca0 con 0x55a57d32b830 2024-03-26T17:47:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d5467a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1a87e0 con 0x55a57d3263b0 2024-03-26T17:47:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d5467a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1a7370 con 0x55a57d32ae40 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32b830 0x55a57d347dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d3263b0 0x55a57d329d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d3263b0 0x55a57d329d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40858/0 (socket says 172.21.15.105:40858) 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32b830 0x55a57d347dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59700/0 (socket says 172.21.15.105:59700) 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2026305675 learned_addr learned my addr 172.21.15.105:0/2026305675 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2026305675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 msgr2=0x55a57d33c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2026305675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32b830 msgr2=0x55a57d347dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32b830 0x55a57d347dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2026305675 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57d333830 con 0x55a57d3263b0 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4bfff640 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32b830 0x55a57d347dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d3263b0 0x55a57d329d20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c012fd0 tx=0x7f6d3c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d199e52e59b79c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d317fa640 1 -- 172.21.15.105:0/2026305675 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c027070 con 0x55a57d3263b0 2024-03-26T17:47:47.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.131+0000 7f6d317fa640 1 -- 172.21.15.105:0/2026305675 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d3c004780 con 0x55a57d3263b0 2024-03-26T17:47:47.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d317fa640 1 -- 172.21.15.105:0/2026305675 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c01d040 con 0x55a57d3263b0 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 -- 172.21.15.105:0/2026305675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d3263b0 msgr2=0x55a57d329d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d3263b0 0x55a57d329d20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c012fd0 tx=0x7f6d3c005970 comp rx=0 tx=0).stop 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 -- 172.21.15.105:0/2026305675 shutdown_connections 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d32b830 0x55a57d347dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2026305675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d3263b0 0x55a57d329d20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 -- 172.21.15.105:0/2026305675 >> 172.21.15.105:0/2026305675 conn(0x55a57d32c610 msgr2=0x55a57d329610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 -- 172.21.15.105:0/2026305675 shutdown_connections 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 -- 172.21.15.105:0/2026305675 wait complete. 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.132+0000 7f6d5467a740 1 -- start start 2024-03-26T17:47:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d5467a740 1 Processor -- start 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3263b0 0x55a57d3258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d5467a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d3a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d5467a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1a8110 con 0x55a57d3263b0 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d5467a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1a7a40 con 0x55a57d32b830 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d5467a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4c1a4ca0 con 0x55a57d32ae40 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d5467a740 10 client.0 _ll_register_callbacks cb 0x7f6d440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3263b0 0x55a57d3258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3263b0 0x55a57d3258d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59716/0 (socket says 172.21.15.105:59716) 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4b7fe640 1 -- 172.21.15.105:0/2379285218 learned_addr learned my addr 172.21.15.105:0/2379285218 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4bfff640 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d3a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4affd640 1 -- 172.21.15.105:0/2379285218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 msgr2=0x55a57d3a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d3a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4affd640 1 -- 172.21.15.105:0/2379285218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3263b0 msgr2=0x55a57d3258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3263b0 0x55a57d3258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4affd640 1 -- 172.21.15.105:0/2379285218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57d3a1bb0 con 0x55a57d32ae40 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.133+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3263b0 0x55a57d3258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d4affd640 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33a810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009850 tx=0x7f6d40000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58432c456313bdb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d2affd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d2affd640 1 -- 172.21.15.105:0/2379285218 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40060be0 con 0x55a57d32ae40 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 1 -- 172.21.15.105:0/2379285218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d3c004780 con 0x55a57d32ae40 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d2affd640 1 -- 172.21.15.105:0/2379285218 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f6d40060d80 con 0x55a57d32ae40 2024-03-26T17:47:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d2affd640 1 -- 172.21.15.105:0/2379285218 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d400046c0 con 0x55a57d32ae40 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 10 client.28720 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 1 -- 172.21.15.105:0/2379285218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a57d3a2f80 con 0x55a57d32ae40 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d2affd640 1 -- 172.21.15.105:0/2379285218 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d3c004780 con 0x55a57d32ae40 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d2affd640 1 -- 172.21.15.105:0/2379285218 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6d40040040 con 0x55a57d32ae40 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 10 client.28720 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 10 client.28720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 20 client.28720 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a57d3309a0 0x55a57d3a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 4 client.28720 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.134+0000 7f6d5467a740 1 -- 172.21.15.105:0/2379285218 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a57d293200 con 0x55a57d3309a0 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a57d3309a0 0x55a57d3a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d2affd640 10 client.28720 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d4b7fe640 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a57d3309a0 0x55a57d3a5970 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec1d4cd33ee241c server_cookie=f4bca8a648d9c0ce in_seq=0 out_seq=0 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d2affd640 1 -- 172.21.15.105:0/2379285218 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a57d293200 con 0x55a57d3309a0 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d2affd640 10 client.28720 handle_command_reply: tid=0 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 client.28720 shutdown 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 -- 172.21.15.105:0/2379285218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 msgr2=0x55a57d33a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33a810 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009850 tx=0x7f6d40000bc0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 -- 172.21.15.105:0/2379285218 shutdown_connections 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a57d3309a0 0x55a57d3a5970 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a57d32b830 0x55a57d3a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a57d32ae40 0x55a57d33a810 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 --2- 172.21.15.105:0/2379285218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a57d3263b0 0x55a57d3258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d2affd640 0 client.28720 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 -- 172.21.15.105:0/2379285218 >> 172.21.15.105:0/2379285218 conn(0x55a57d32c610 msgr2=0x55a57d329ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 -- 172.21.15.105:0/2379285218 shutdown_connections 2024-03-26T17:47:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 1 -- 172.21.15.105:0/2379285218 wait complete. 2024-03-26T17:47:47.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.135+0000 7f6d5467a740 20 client.28720 trim_cache size 0 max 16384 2024-03-26T17:47:47.136 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 -- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6d340a2f20 msgr2=0x7f6d340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6d340a2f20 0x7f6d340a53e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6d4002c240 tx=0x7f6d40005fd0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 -- 172.21.15.105:0/1453776905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d4c102370 msgr2=0x7f6d4c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d4c102370 0x7f6d4c19f110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009fd0 tx=0x7f6d4400ef10 comp rx=0 tx=0).stop 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 -- 172.21.15.105:0/1453776905 shutdown_connections 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6d340a2f20 0x7f6d340a53e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d4c102370 0x7f6d4c19f110 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d4c1019b0 0x7f6d4c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 --2- 172.21.15.105:0/1453776905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d4c1007c0 0x7f6d4c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 -- 172.21.15.105:0/1453776905 >> 172.21.15.105:0/1453776905 conn(0x7f6d4c0fbd50 msgr2=0x7f6d4c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 -- 172.21.15.105:0/1453776905 shutdown_connections 2024-03-26T17:47:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:47.136+0000 7f6d51a85640 1 -- 172.21.15.105:0/1453776905 wait complete. 2024-03-26T17:47:47.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:46 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3125598159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:47:48.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 300167 3.14 MB/sec execute 391 sec latency 18.545 ms 2024-03-26T17:47:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:47 smithi105 ceph-mon[32280]: pgmap v631: 97 pgs: 97 active+clean; 26 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:47:48.143 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:47:48.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:47:48.144 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:47:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:47 smithi135 ceph-mon[35636]: pgmap v631: 97 pgs: 97 active+clean; 26 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:47:48.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 -- 172.21.15.105:0/1185145655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 msgr2=0x7ff254104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 --2- 172.21.15.105:0/1185145655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 0x7ff254104730 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff250009900 tx=0x7ff25002bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 -- 172.21.15.105:0/1185145655 shutdown_connections 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 --2- 172.21.15.105:0/1185145655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2541075a0 0x7ff2541099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 --2- 172.21.15.105:0/1185145655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff254104c70 0x7ff254107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 --2- 172.21.15.105:0/1185145655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 0x7ff254104730 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 -- 172.21.15.105:0/1185145655 >> 172.21.15.105:0/1185145655 conn(0x7ff2540fbd50 msgr2=0x7ff2540fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 -- 172.21.15.105:0/1185145655 shutdown_connections 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.273+0000 7ff25be97640 1 -- 172.21.15.105:0/1185145655 wait complete. 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25be97640 1 Processor -- start 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25be97640 1 -- start start 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25be97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 0x7ff254197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25be97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff254104c70 0x7ff2541980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25be97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2541075a0 0x7ff25419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25be97640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410d3f0 con 0x7ff254102340 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25be97640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410d270 con 0x7ff2541075a0 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25be97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410d570 con 0x7ff254104c70 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff259c33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 0x7ff254197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff259c33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 0x7ff254197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59736/0 (socket says 172.21.15.105:59736) 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff259c33640 1 -- 172.21.15.105:0/4210542759 learned_addr learned my addr 172.21.15.105:0/4210542759 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.274+0000 7ff25a434640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2541075a0 0x7ff25419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff259432640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff254104c70 0x7ff2541980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff259c33640 1 -- 172.21.15.105:0/4210542759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff254104c70 msgr2=0x7ff2541980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff259c33640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff254104c70 0x7ff2541980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff259c33640 1 -- 172.21.15.105:0/4210542759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2541075a0 msgr2=0x7ff25419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff259c33640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2541075a0 0x7ff25419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff259c33640 1 -- 172.21.15.105:0/4210542759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25419f7b0 con 0x7ff254102340 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff259c33640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 0x7ff254197b90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff250006010 tx=0x7ff250005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff24affd640 1 -- 172.21.15.105:0/4210542759 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250043020 con 0x7ff254102340 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff24affd640 1 -- 172.21.15.105:0/4210542759 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2500043d0 con 0x7ff254102340 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff25be97640 1 -- 172.21.15.105:0/4210542759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25419fa40 con 0x7ff254102340 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff24affd640 1 -- 172.21.15.105:0/4210542759 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250003b80 con 0x7ff254102340 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.275+0000 7ff25be97640 1 -- 172.21.15.105:0/4210542759 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25419ff90 con 0x7ff254102340 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25ea81740 1 Processor -- start 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25ea81740 1 -- start start 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff24affd640 1 -- 172.21.15.105:0/4210542759 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff250046020 con 0x7ff254102340 2024-03-26T17:47:48.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d5e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d63680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d66180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25ea81740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560331cc3630 con 0x560331d5e190 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25ea81740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560331c6be10 con 0x560331d63cf0 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25ea81740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560331cbf5e0 con 0x560331d5eb80 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25a434640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d5e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff259c33640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d63680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25a434640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d5e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59742/0 (socket says 172.21.15.105:59742) 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff25a434640 1 -- 172.21.15.105:0/1610675509 learned_addr learned my addr 172.21.15.105:0/1610675509 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.278+0000 7ff259432640 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d66180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff259432640 1 -- 172.21.15.105:0/1610675509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 msgr2=0x560331d63680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff259432640 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d63680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff259432640 1 -- 172.21.15.105:0/1610675509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 msgr2=0x560331d5e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff259432640 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d5e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff259432640 1 -- 172.21.15.105:0/1610675509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560331c82740 con 0x560331d63cf0 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff259c33640 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d63680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff24affd640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2400a2f20 0x7ff2400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff259432640 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d66180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff254069580 tx=0x7ff244011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135208208fdf505d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff24affd640 1 -- 172.21.15.105:0/4210542759 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff2500f1600 con 0x7ff254102340 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff25a434640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2400a2f20 0x7ff2400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff23ffff640 1 -- 172.21.15.105:0/1610675509 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244004050 con 0x560331d63cf0 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff23ffff640 1 -- 172.21.15.105:0/1610675509 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2440054d0 con 0x560331d63cf0 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.279+0000 7ff23ffff640 1 -- 172.21.15.105:0/1610675509 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244004c30 con 0x560331d63cf0 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 -- 172.21.15.105:0/1610675509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 msgr2=0x560331d66180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25a434640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2400a2f20 0x7ff2400a53e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x560331d5f3e0 tx=0x7ff24c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d66180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff254069580 tx=0x7ff244011810 comp rx=0 tx=0).stop 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 -- 172.21.15.105:0/1610675509 shutdown_connections 2024-03-26T17:47:48.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d66180 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d63680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 --2- 172.21.15.105:0/1610675509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d5e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 -- 172.21.15.105:0/1610675509 >> 172.21.15.105:0/1610675509 conn(0x560331d59700 msgr2=0x560331d59b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 -- 172.21.15.105:0/1610675509 shutdown_connections 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 -- 172.21.15.105:0/1610675509 wait complete. 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.280+0000 7ff25ea81740 1 -- start start 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25ea81740 1 Processor -- start 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5e190 0x560331d68640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5eb80 0x560331d68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d6fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25ea81740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560331d66ca0 con 0x560331d5eb80 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25ea81740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560331c7f1a0 con 0x560331d63cf0 2024-03-26T17:47:48.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25ea81740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560331d66e20 con 0x560331d5e190 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25ea81740 10 client.0 _ll_register_callbacks cb 0x5603319d6c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff259432640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5eb80 0x560331d68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff259c33640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5e190 0x560331d68640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff25a434640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d6fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff259432640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5eb80 0x560331d68b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59754/0 (socket says 172.21.15.105:59754) 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff259432640 1 -- 172.21.15.105:0/754023664 learned_addr learned my addr 172.21.15.105:0/754023664 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.281+0000 7ff259432640 1 -- 172.21.15.105:0/754023664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5e190 msgr2=0x560331d68640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff259432640 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5e190 0x560331d68640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff259432640 1 -- 172.21.15.105:0/754023664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 msgr2=0x560331d6fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff259432640 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d6fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff259432640 1 -- 172.21.15.105:0/754023664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560331a9f7e0 con 0x560331d5eb80 2024-03-26T17:47:48.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff259432640 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5eb80 0x560331d68b80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff24401b040 tx=0x7ff244004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44036e6b6a7ff84a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff23d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff23d7fa640 1 -- 172.21.15.105:0/754023664 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244029070 con 0x560331d5eb80 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff23d7fa640 1 -- 172.21.15.105:0/754023664 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff244003a00 con 0x560331d5eb80 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff25ea81740 1 -- 172.21.15.105:0/754023664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2440054d0 con 0x560331d5eb80 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff23d7fa640 1 -- 172.21.15.105:0/754023664 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff244035050 con 0x560331d5eb80 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff25ea81740 10 client.18897 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff25ea81740 1 -- 172.21.15.105:0/754023664 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560331afff70 con 0x560331d5eb80 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff23d7fa640 1 -- 172.21.15.105:0/754023664 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2440054d0 con 0x560331d5eb80 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.282+0000 7ff23d7fa640 1 -- 172.21.15.105:0/754023664 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff244005190 con 0x560331d5eb80 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff25ea81740 10 client.18897 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff25ea81740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff25ea81740 20 client.18897 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff25ea81740 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560331d741f0 0x560331d76640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff25ea81740 4 client.18897 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff25ea81740 1 -- 172.21.15.105:0/754023664 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560331cd0b70 con 0x560331d741f0 2024-03-26T17:47:48.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff259c33640 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560331d741f0 0x560331d76640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff259c33640 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560331d741f0 0x560331d76640 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c700daf8f6563b3 server_cookie=11f3883ccf1e350 in_seq=0 out_seq=0 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.283+0000 7ff23d7fa640 10 client.18897 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.284+0000 7ff23d7fa640 1 -- 172.21.15.105:0/754023664 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560331cd0b70 con 0x560331d741f0 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.284+0000 7ff23d7fa640 10 client.18897 handle_command_reply: tid=0 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.284+0000 7ff25ea81740 1 client.18897 shutdown 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.284+0000 7ff25ea81740 1 -- 172.21.15.105:0/754023664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5eb80 msgr2=0x560331d68b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.284+0000 7ff25ea81740 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5eb80 0x560331d68b80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff24401b040 tx=0x7ff244004480 comp rx=0 tx=0).stop 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.284+0000 7ff25ea81740 1 -- 172.21.15.105:0/754023664 shutdown_connections 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.284+0000 7ff25ea81740 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560331d741f0 0x560331d76640 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.285+0000 7ff25ea81740 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d6fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.285+0000 7ff25ea81740 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5eb80 0x560331d68b80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.285+0000 7ff23d7fa640 0 client.18897 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.285+0000 7ff25ea81740 1 --2- 172.21.15.105:0/754023664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5e190 0x560331d68640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.285+0000 7ff25ea81740 1 -- 172.21.15.105:0/754023664 >> 172.21.15.105:0/754023664 conn(0x560331d59700 msgr2=0x560331d58900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.285+0000 7ff25ea81740 1 -- 172.21.15.105:0/754023664 shutdown_connections 2024-03-26T17:47:48.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.285+0000 7ff25ea81740 1 -- 172.21.15.105:0/754023664 wait complete. 2024-03-26T17:47:48.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.285+0000 7ff25ea81740 20 client.18897 trim_cache size 0 max 16384 2024-03-26T17:47:48.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25ea81740 1 Processor -- start 2024-03-26T17:47:48.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25ea81740 1 -- start start 2024-03-26T17:47:48.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d5e190 0x560331d58070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d63cf0 0x560331d585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25ea81740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541a6c40 con 0x560331d63cf0 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25ea81740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541a8780 con 0x560331d5e190 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25ea81740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541a7310 con 0x560331d5eb80 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259c33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d5e190 0x560331d58070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25a434640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d63cf0 0x560331d585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259432640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259c33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d5e190 0x560331d58070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40912/0 (socket says 172.21.15.105:40912) 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25a434640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d63cf0 0x560331d585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59764/0 (socket says 172.21.15.105:59764) 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259c33640 1 -- 172.21.15.105:0/3916568137 learned_addr learned my addr 172.21.15.105:0/3916568137 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259c33640 1 -- 172.21.15.105:0/3916568137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 msgr2=0x560331d6fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259c33640 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259c33640 1 -- 172.21.15.105:0/3916568137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d63cf0 msgr2=0x560331d585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259c33640 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d63cf0 0x560331d585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259c33640 1 -- 172.21.15.105:0/3916568137 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560331997170 con 0x560331d5e190 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff25a434640 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d63cf0 0x560331d585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.306+0000 7ff259432640 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff259c33640 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d5e190 0x560331d58070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff2500ba6c0 tx=0x7ff2500f7550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899007b9f798c26a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff23f7fe640 1 -- 172.21.15.105:0/3916568137 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2500bdac0 con 0x560331d5e190 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff23f7fe640 1 -- 172.21.15.105:0/3916568137 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2500be3f0 con 0x560331d5e190 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff23f7fe640 1 -- 172.21.15.105:0/3916568137 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2500bd650 con 0x560331d5e190 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 -- 172.21.15.105:0/3916568137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d5e190 msgr2=0x560331d58070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d5e190 0x560331d58070 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff2500ba6c0 tx=0x7ff2500f7550 comp rx=0 tx=0).stop 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 -- 172.21.15.105:0/3916568137 shutdown_connections 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d63cf0 0x560331d585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3916568137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d5e190 0x560331d58070 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 -- 172.21.15.105:0/3916568137 >> 172.21.15.105:0/3916568137 conn(0x560331d59700 msgr2=0x560331d68640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 -- 172.21.15.105:0/3916568137 shutdown_connections 2024-03-26T17:47:48.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.307+0000 7ff25ea81740 1 -- 172.21.15.105:0/3916568137 wait complete. 2024-03-26T17:47:48.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 1 -- start start 2024-03-26T17:47:48.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 1 Processor -- start 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d6d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d6e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541a80b0 con 0x560331d5e190 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541a79e0 con 0x560331d63cf0 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541a4c20 con 0x560331d5eb80 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25ea81740 10 client.0 _ll_register_callbacks cb 0x7ff24c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff259c33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d6d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff259c33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d6d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59772/0 (socket says 172.21.15.105:59772) 2024-03-26T17:47:48.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff259c33640 1 -- 172.21.15.105:0/3699268666 learned_addr learned my addr 172.21.15.105:0/3699268666 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff25a434640 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d6e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.308+0000 7ff259432640 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff259c33640 1 -- 172.21.15.105:0/3699268666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 msgr2=0x560331d6d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff259c33640 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff259c33640 1 -- 172.21.15.105:0/3699268666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 msgr2=0x560331d6e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff259c33640 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d6e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff259c33640 1 -- 172.21.15.105:0/3699268666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560331d6ede0 con 0x560331d5e190 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff259c33640 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d6d2a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff2500ef090 tx=0x7ff2500bd6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ad4a9711469260 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff23d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff23d7fa640 1 -- 172.21.15.105:0/3699268666 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2500bd830 con 0x560331d5e190 2024-03-26T17:47:48.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff23d7fa640 1 -- 172.21.15.105:0/3699268666 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ff2500ba360 con 0x560331d5e190 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff25ea81740 1 -- 172.21.15.105:0/3699268666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2500be3f0 con 0x560331d5e190 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff23d7fa640 1 -- 172.21.15.105:0/3699268666 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2500c1e40 con 0x560331d5e190 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff25ea81740 10 client.18903 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff25ea81740 1 -- 172.21.15.105:0/3699268666 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560331dda5f0 con 0x560331d5e190 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff23d7fa640 1 -- 172.21.15.105:0/3699268666 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2500be3f0 con 0x560331d5e190 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.309+0000 7ff23d7fa640 1 -- 172.21.15.105:0/3699268666 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff2500c6270 con 0x560331d5e190 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff25ea81740 10 client.18903 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff25ea81740 10 client.18903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff25ea81740 20 client.18903 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560331ddaf40 0x560331ddd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff25ea81740 4 client.18903 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:47:48.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff25ea81740 1 -- 172.21.15.105:0/3699268666 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560331ccaec0 con 0x560331ddaf40 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff259432640 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560331ddaf40 0x560331ddd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff23d7fa640 10 client.18903 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.310+0000 7ff259432640 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560331ddaf40 0x560331ddd360 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7c159626ea3f73d server_cookie=94a49e68b4a7e95e in_seq=0 out_seq=0 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff23d7fa640 1 -- 172.21.15.105:0/3699268666 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560331ccaec0 con 0x560331ddaf40 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff23d7fa640 10 client.18903 handle_command_reply: tid=0 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 client.18903 shutdown 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 -- 172.21.15.105:0/3699268666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 msgr2=0x560331d6d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d6d2a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff2500ef090 tx=0x7ff2500bd6e0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 -- 172.21.15.105:0/3699268666 shutdown_connections 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560331ddaf40 0x560331ddd360 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560331d63cf0 0x560331d6e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560331d5eb80 0x560331d6d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff23d7fa640 0 client.18903 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 --2- 172.21.15.105:0/3699268666 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560331d5e190 0x560331d6d2a0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.311+0000 7ff25ea81740 1 -- 172.21.15.105:0/3699268666 >> 172.21.15.105:0/3699268666 conn(0x560331d59700 msgr2=0x560331d5c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25ea81740 1 -- 172.21.15.105:0/3699268666 shutdown_connections 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25ea81740 1 -- 172.21.15.105:0/3699268666 wait complete. 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25ea81740 20 client.18903 trim_cache size 0 max 16384 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "fe03d8c3-45b4-4648-9693-2c12e5195118", 2024-03-26T17:47:48.311 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:47:48.312 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:47:48.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 -- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2400a2f20 msgr2=0x7ff2400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2400a2f20 0x7ff2400a53e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x560331d5f3e0 tx=0x7ff24c009500 comp rx=0 tx=0).stop 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 -- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 msgr2=0x7ff254197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 0x7ff254197b90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff250006010 tx=0x7ff250005e40 comp rx=0 tx=0).stop 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 -- 172.21.15.105:0/4210542759 shutdown_connections 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2400a2f20 0x7ff2400a53e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2541075a0 0x7ff25419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff254104c70 0x7ff2541980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 --2- 172.21.15.105:0/4210542759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff254102340 0x7ff254197b90 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 -- 172.21.15.105:0/4210542759 >> 172.21.15.105:0/4210542759 conn(0x7ff2540fbd50 msgr2=0x7ff2541041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 -- 172.21.15.105:0/4210542759 shutdown_connections 2024-03-26T17:47:48.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.312+0000 7ff25be97640 1 -- 172.21.15.105:0/4210542759 wait complete. 2024-03-26T17:47:48.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:47:48.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:47:48.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:47 smithi144 ceph-mon[36095]: pgmap v631: 97 pgs: 97 active+clean; 26 MiB data, 944 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:47:48.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.439+0000 7f3037d97640 1 -- 172.21.15.105:0/254282789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30301019f0 msgr2=0x7f3030101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.439+0000 7f3037d97640 1 --2- 172.21.15.105:0/254282789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30301019f0 0x7f3030101e70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3020009900 tx=0x7f302002bc30 comp rx=0 tx=0).stop 2024-03-26T17:47:48.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 -- 172.21.15.105:0/254282789 shutdown_connections 2024-03-26T17:47:48.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 --2- 172.21.15.105:0/254282789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30301023b0 0x7f3030108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 --2- 172.21.15.105:0/254282789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30301019f0 0x7f3030101e70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 --2- 172.21.15.105:0/254282789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3030100800 0x7f3030100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 -- 172.21.15.105:0/254282789 >> 172.21.15.105:0/254282789 conn(0x7f30300fbd70 msgr2=0x7f30300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 -- 172.21.15.105:0/254282789 shutdown_connections 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 -- 172.21.15.105:0/254282789 wait complete. 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 Processor -- start 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 -- start start 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3030100800 0x7f303006a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30301019f0 0x7f303019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.440+0000 7f3037d97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30301023b0 0x7f303019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3037d97640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f303010c310 con 0x7f3030100800 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3037d97640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f303010c190 con 0x7f30301023b0 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3037d97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f303010c490 con 0x7f30301019f0 2024-03-26T17:47:48.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3035b33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3030100800 0x7f303006a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3035b33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3030100800 0x7f303006a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59790/0 (socket says 172.21.15.105:59790) 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3035b33640 1 -- 172.21.15.105:0/380887841 learned_addr learned my addr 172.21.15.105:0/380887841 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3036334640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30301023b0 0x7f303019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3035332640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30301019f0 0x7f303019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3036334640 1 -- 172.21.15.105:0/380887841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30301019f0 msgr2=0x7f303019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3036334640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30301019f0 0x7f303019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3036334640 1 -- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3030100800 msgr2=0x7f303006a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3036334640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3030100800 0x7f303006a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3036334640 1 -- 172.21.15.105:0/380887841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303019f7f0 con 0x7f30301023b0 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3035332640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30301019f0 0x7f303019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.441+0000 7f3036334640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30301023b0 0x7f303019ef30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f302800ec50 tx=0x7f302800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.442+0000 7f3026ffd640 1 -- 172.21.15.105:0/380887841 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028003b70 con 0x7f30301023b0 2024-03-26T17:47:48.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.442+0000 7f3037d97640 1 -- 172.21.15.105:0/380887841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30301967b0 con 0x7f30301023b0 2024-03-26T17:47:48.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.442+0000 7f3026ffd640 1 -- 172.21.15.105:0/380887841 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3028003d10 con 0x7f30301023b0 2024-03-26T17:47:48.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.442+0000 7f3026ffd640 1 -- 172.21.15.105:0/380887841 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028010760 con 0x7f30301023b0 2024-03-26T17:47:48.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.442+0000 7f3037d97640 1 -- 172.21.15.105:0/380887841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3030196cd0 con 0x7f30301023b0 2024-03-26T17:47:48.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.444+0000 7f3026ffd640 1 -- 172.21.15.105:0/380887841 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3028004020 con 0x7f30301023b0 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f303a987740 1 Processor -- start 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f303a987740 1 -- start start 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f3026ffd640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f301c0a2f20 0x7f301c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f303a987740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f303a987740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f303a987740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f303a987740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cc3ff32980 con 0x55cc3ffce2e0 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f303a987740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cc3fedabe0 con 0x55cc3ffd3450 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f303a987740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc3ff2e930 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.445+0000 7f3035332640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f301c0a2f20 0x7f301c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3026ffd640 1 -- 172.21.15.105:0/380887841 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f30280cf9c0 con 0x7f30301023b0 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035332640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33404/0 (socket says 172.21.15.105:33404) 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035332640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59794/0 (socket says 172.21.15.105:59794) 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 -- 172.21.15.105:0/239485443 learned_addr learned my addr 172.21.15.105:0/239485443 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3036334640 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 -- 172.21.15.105:0/239485443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 msgr2=0x55cc3ffd58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 -- 172.21.15.105:0/239485443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 msgr2=0x55cc3ffd2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 -- 172.21.15.105:0/239485443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc3fd107d0 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3036334640 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:47:48.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035332640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f301c0a2f20 0x7f301c0a53e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3020009230 tx=0x7f3020005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f3035b33640 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcdd10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f302c009d60 tx=0x7f302c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3afe4bf92e6634a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f301bfff640 1 -- 172.21.15.105:0/239485443 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c004230 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f301bfff640 1 -- 172.21.15.105:0/239485443 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f302c0043d0 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.446+0000 7f301bfff640 1 -- 172.21.15.105:0/239485443 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c019710 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 -- 172.21.15.105:0/239485443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 msgr2=0x55cc3ffcdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcdd10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f302c009d60 tx=0x7f302c011810 comp rx=0 tx=0).stop 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 -- 172.21.15.105:0/239485443 shutdown_connections 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 --2- 172.21.15.105:0/239485443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcdd10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 -- 172.21.15.105:0/239485443 >> 172.21.15.105:0/239485443 conn(0x55cc3ffc8e60 msgr2=0x55cc3ffc9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 -- 172.21.15.105:0/239485443 shutdown_connections 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.447+0000 7f303a987740 1 -- 172.21.15.105:0/239485443 wait complete. 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 1 -- start start 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 1 Processor -- start 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffd7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffdeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cc3ffd6400 con 0x55cc3ffce2e0 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cc3feee160 con 0x55cc3ffd3450 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc3ffd6580 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f303a987740 10 client.0 _ll_register_callbacks cb 0x55cc3ff2a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f3035332640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f3035b33640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffd7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f3036334640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffdeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f3035b33640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffd7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33414/0 (socket says 172.21.15.105:33414) 2024-03-26T17:47:48.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.448+0000 7f3035b33640 1 -- 172.21.15.105:0/1209956595 learned_addr learned my addr 172.21.15.105:0/1209956595 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f3035b33640 1 -- 172.21.15.105:0/1209956595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 msgr2=0x55cc3ffdeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f3035b33640 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffdeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f3035b33640 1 -- 172.21.15.105:0/1209956595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 msgr2=0x55cc3ffd7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f3035b33640 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f3035b33640 1 -- 172.21.15.105:0/1209956595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc3fc463f0 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f3035332640 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f3035b33640 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffd7a70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f302c01b040 tx=0x7f302c0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c784e28fac5c4920 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f30197fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f30197fa640 1 -- 172.21.15.105:0/1209956595 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c028020 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f30197fa640 1 -- 172.21.15.105:0/1209956595 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f302c005050 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f303a987740 1 -- 172.21.15.105:0/1209956595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302c0043d0 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f30197fa640 1 -- 172.21.15.105:0/1209956595 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f302c033050 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f303a987740 10 client.28750 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:48.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f303a987740 1 -- 172.21.15.105:0/1209956595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc3fc060f0 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.449+0000 7f30197fa640 1 -- 172.21.15.105:0/1209956595 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f302c0043d0 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f30197fa640 1 -- 172.21.15.105:0/1209956595 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f302c004c80 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f303a987740 10 client.28750 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f303a987740 10 client.28750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f303a987740 20 client.28750 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f303a987740 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cc3ffe36c0 0x55cc3ffe5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f303a987740 4 client.28750 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f303a987740 1 -- 172.21.15.105:0/1209956595 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc3ff3fec0 con 0x55cc3ffe36c0 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f3035332640 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cc3ffe36c0 0x55cc3ffe5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f30197fa640 10 client.28750 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:48.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.450+0000 7f3035332640 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cc3ffe36c0 0x55cc3ffe5b40 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c884889fc84fb67 server_cookie=39db72dd05370987 in_seq=0 out_seq=0 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.454+0000 7f30197fa640 1 -- 172.21.15.105:0/1209956595 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cc3ff3fec0 con 0x55cc3ffe36c0 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.454+0000 7f30197fa640 10 client.28750 handle_command_reply: tid=0 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.454+0000 7f303a987740 1 client.28750 shutdown 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 -- 172.21.15.105:0/1209956595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 msgr2=0x55cc3ffd7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffd7a70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f302c01b040 tx=0x7f302c0038a0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 -- 172.21.15.105:0/1209956595 shutdown_connections 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cc3ffe36c0 0x55cc3ffe5b40 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffd3450 0x55cc3ffdeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffd7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 --2- 172.21.15.105:0/1209956595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffd7a70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f30197fa640 0 client.28750 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:48.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 -- 172.21.15.105:0/1209956595 >> 172.21.15.105:0/1209956595 conn(0x55cc3ffc8e60 msgr2=0x55cc3ffc8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 -- 172.21.15.105:0/1209956595 shutdown_connections 2024-03-26T17:47:48.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 1 -- 172.21.15.105:0/1209956595 wait complete. 2024-03-26T17:47:48.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.455+0000 7f303a987740 20 client.28750 trim_cache size 0 max 16384 2024-03-26T17:47:48.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f303a987740 1 Processor -- start 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f303a987740 1 -- start start 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f303a987740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f303a987740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f303a987740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f303a987740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f30301a6c30 con 0x55cc3ffd3450 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f303a987740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30301a8770 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f303a987740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30301a7300 con 0x55cc3ffce2e0 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f3036334640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f3035332640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f3036334640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59812/0 (socket says 172.21.15.105:59812) 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f3035332640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33428/0 (socket says 172.21.15.105:33428) 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f3035b33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.483+0000 7f3035332640 1 -- 172.21.15.105:0/1111844799 learned_addr learned my addr 172.21.15.105:0/1111844799 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f3036334640 1 -- 172.21.15.105:0/1111844799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 msgr2=0x55cc3ffdefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f3036334640 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f3036334640 1 -- 172.21.15.105:0/1111844799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 msgr2=0x55cc3ffcc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f3036334640 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f3036334640 1 -- 172.21.15.105:0/1111844799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc3ffc3e20 con 0x55cc3ffd3450 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f3035332640 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f3035b33640 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f3036334640 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd7090 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f302809bdb0 tx=0x7f30280045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230c221dd6f79c46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f301b7fe640 1 -- 172.21.15.105:0/1111844799 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302809c7b0 con 0x55cc3ffd3450 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f301b7fe640 1 -- 172.21.15.105:0/1111844799 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f302809c0b0 con 0x55cc3ffd3450 2024-03-26T17:47:48.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.484+0000 7f301b7fe640 1 -- 172.21.15.105:0/1111844799 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30280a4630 con 0x55cc3ffd3450 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 -- 172.21.15.105:0/1111844799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 msgr2=0x55cc3ffd7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd7090 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f302809bdb0 tx=0x7f30280045e0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 -- 172.21.15.105:0/1111844799 shutdown_connections 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc3ffd7090 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 --2- 172.21.15.105:0/1111844799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffcc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 -- 172.21.15.105:0/1111844799 >> 172.21.15.105:0/1111844799 conn(0x55cc3ffc8e60 msgr2=0x55cc3ffcc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 -- 172.21.15.105:0/1111844799 shutdown_connections 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 -- 172.21.15.105:0/1111844799 wait complete. 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 -- start start 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.485+0000 7f303a987740 1 Processor -- start 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f303a987740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffdbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f303a987740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f303a987740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc40047a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f303a987740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f30301a80a0 con 0x55cc3ffd3450 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f303a987740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30301a79d0 con 0x55cc3ffcd8f0 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f303a987740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30301a4c10 con 0x55cc3ffce2e0 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f303a987740 10 client.0 _ll_register_callbacks cb 0x7f30280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc40047a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3035b33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffdbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc40047a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59818/0 (socket says 172.21.15.105:59818) 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 -- 172.21.15.105:0/1701771 learned_addr learned my addr 172.21.15.105:0/1701771 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3035b33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffdbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40998/0 (socket says 172.21.15.105:40998) 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3035332640 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 -- 172.21.15.105:0/1701771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 msgr2=0x55cc3ffdc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 -- 172.21.15.105:0/1701771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 msgr2=0x55cc3ffdbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffdbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 -- 172.21.15.105:0/1701771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc40047fa0 con 0x55cc3ffd3450 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3035b33640 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffdbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f3036334640 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc40047a30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f30280cd4d0 tx=0x7f3028003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ec28bcdd1ff4d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f30197fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f30197fa640 1 -- 172.21.15.105:0/1701771 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302809c7b0 con 0x55cc3ffd3450 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f30197fa640 1 -- 172.21.15.105:0/1701771 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f30280a4630 con 0x55cc3ffd3450 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.486+0000 7f303a987740 1 -- 172.21.15.105:0/1701771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f302809c0b0 con 0x55cc3ffd3450 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f30197fa640 1 -- 172.21.15.105:0/1701771 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30280a0200 con 0x55cc3ffd3450 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f303a987740 10 client.18921 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f303a987740 1 -- 172.21.15.105:0/1701771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cc400492b0 con 0x55cc3ffd3450 2024-03-26T17:47:48.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f30197fa640 1 -- 172.21.15.105:0/1701771 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f302809c0b0 con 0x55cc3ffd3450 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f30197fa640 1 -- 172.21.15.105:0/1701771 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f3028097e00 con 0x55cc3ffd3450 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f303a987740 10 client.18921 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f303a987740 10 client.18921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f303a987740 20 client.18921 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f303a987740 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cc40049b30 0x55cc4004bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f303a987740 4 client.18921 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f303a987740 1 -- 172.21.15.105:0/1701771 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc3ff3a210 con 0x55cc40049b30 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.487+0000 7f3035b33640 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cc40049b30 0x55cc4004bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f30197fa640 10 client.18921 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f3035b33640 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cc40049b30 0x55cc4004bf50 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6757f94c6401e7dd server_cookie=849b222fb75883cb in_seq=0 out_seq=0 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f30197fa640 1 -- 172.21.15.105:0/1701771 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cc3ff3a210 con 0x55cc40049b30 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f30197fa640 10 client.18921 handle_command_reply: tid=0 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 client.18921 shutdown 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 -- 172.21.15.105:0/1701771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 msgr2=0x55cc40047a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc40047a30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f30280cd4d0 tx=0x7f3028003ec0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 -- 172.21.15.105:0/1701771 shutdown_connections 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cc40049b30 0x55cc4004bf50 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cc3ffd3450 0x55cc40047a30 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f30197fa640 0 client.18921 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc3ffce2e0 0x55cc3ffdc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 --2- 172.21.15.105:0/1701771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cc3ffcd8f0 0x55cc3ffdbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 -- 172.21.15.105:0/1701771 >> 172.21.15.105:0/1701771 conn(0x55cc3ffc8e60 msgr2=0x55cc3ffcc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 -- 172.21.15.105:0/1701771 shutdown_connections 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 1 -- 172.21.15.105:0/1701771 wait complete. 2024-03-26T17:47:48.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.488+0000 7f303a987740 20 client.18921 trim_cache size 0 max 16384 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout: "fe03d8c3-45b4-4648-9693-2c12e5195118": { 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "fe03d8c3-45b4-4648-9693-2c12e5195118", 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:47:48.489 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:47:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 -- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f301c0a2f20 msgr2=0x7f301c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f301c0a2f20 0x7f301c0a53e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3020009230 tx=0x7f3020005e20 comp rx=0 tx=0).stop 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 -- 172.21.15.105:0/380887841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30301023b0 msgr2=0x7f303019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30301023b0 0x7f303019ef30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f302800ec50 tx=0x7f302800c6a0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 -- 172.21.15.105:0/380887841 shutdown_connections 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f301c0a2f20 0x7f301c0a53e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30301023b0 0x7f303019ef30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30301019f0 0x7f303019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 --2- 172.21.15.105:0/380887841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3030100800 0x7f303006a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 -- 172.21.15.105:0/380887841 >> 172.21.15.105:0/380887841 conn(0x7f30300fbd70 msgr2=0x7f3030104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 -- 172.21.15.105:0/380887841 shutdown_connections 2024-03-26T17:47:48.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:47:48.489+0000 7f3037d97640 1 -- 172.21.15.105:0/380887841 wait complete. 2024-03-26T17:47:48.496 INFO:tasks.cephfs.filesystem:scrub status for tag:fe03d8c3-45b4-4648-9693-2c12e5195118 - {'path': '/', 'tag': 'fe03d8c3-45b4-4648-9693-2c12e5195118', 'options': 'recursive,force'} 2024-03-26T17:47:49.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 300873 3.15 MB/sec execute 392 sec latency 16.328 ms 2024-03-26T17:47:49.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:48 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:47:49.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:48 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:49.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:48 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:47:49.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:48 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:49.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:48 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:47:49.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:48 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:49.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:48 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:47:49.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:48 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:49.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:48 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:47:49.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:48 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:49.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:48 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:47:49.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:48 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:47:50.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 301366 3.15 MB/sec execute 393 sec latency 16.381 ms 2024-03-26T17:47:50.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:49 smithi105 ceph-mon[32280]: pgmap v632: 97 pgs: 97 active+clean; 25 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:47:50.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:49 smithi135 ceph-mon[35636]: pgmap v632: 97 pgs: 97 active+clean; 25 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:47:50.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:49 smithi144 ceph-mon[36095]: pgmap v632: 97 pgs: 97 active+clean; 25 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:47:51.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 301917 3.15 MB/sec execute 394 sec latency 16.528 ms 2024-03-26T17:47:52.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 302397 3.15 MB/sec execute 395 sec latency 23.960 ms 2024-03-26T17:47:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:51 smithi105 ceph-mon[32280]: pgmap v633: 97 pgs: 97 active+clean; 27 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:47:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:51 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:47:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:51 smithi135 ceph-mon[35636]: pgmap v633: 97 pgs: 97 active+clean; 27 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:47:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:51 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:47:52.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:51 smithi144 ceph-mon[36095]: pgmap v633: 97 pgs: 97 active+clean; 27 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:47:52.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:51 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:47:53.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 303015 3.15 MB/sec execute 396 sec latency 17.391 ms 2024-03-26T17:47:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:52 smithi105 ceph-mon[32280]: pgmap v634: 97 pgs: 97 active+clean; 26 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-03-26T17:47:53.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:52 smithi135 ceph-mon[35636]: pgmap v634: 97 pgs: 97 active+clean; 26 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-03-26T17:47:53.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:52 smithi144 ceph-mon[36095]: pgmap v634: 97 pgs: 97 active+clean; 26 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-03-26T17:47:54.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 303663 3.15 MB/sec execute 397 sec latency 16.924 ms 2024-03-26T17:47:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:53 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:47:54.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:53 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:47:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.319+0000 7f966b2dc640 1 -- 172.21.15.105:0/391267302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664101450 msgr2=0x7f96641018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.319+0000 7f966b2dc640 1 --2- 172.21.15.105:0/391267302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664101450 0x7f96641018d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f965c009ca0 tx=0x7f965c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.319+0000 7f966b2dc640 1 -- 172.21.15.105:0/391267302 shutdown_connections 2024-03-26T17:47:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.319+0000 7f966b2dc640 1 --2- 172.21.15.105:0/391267302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664101450 0x7f96641018d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.319+0000 7f966b2dc640 1 --2- 172.21.15.105:0/391267302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9664100ab0 0x7f9664100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.319+0000 7f966b2dc640 1 --2- 172.21.15.105:0/391267302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9664100150 0x7f9664100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.319+0000 7f966b2dc640 1 -- 172.21.15.105:0/391267302 >> 172.21.15.105:0/391267302 conn(0x7f96640fbd50 msgr2=0x7f96640fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 -- 172.21.15.105:0/391267302 shutdown_connections 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 -- 172.21.15.105:0/391267302 wait complete. 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 Processor -- start 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 -- start start 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9664100150 0x7f9664199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ab0 0x7f966419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9664101450 0x7f96641a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f966410f600 con 0x7f9664100ab0 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f966410f480 con 0x7f9664100150 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.320+0000 7f966b2dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f966410f780 con 0x7f9664101450 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9664100150 0x7f9664199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9664100150 0x7f9664199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41016/0 (socket says 172.21.15.105:41016) 2024-03-26T17:47:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669078640 1 -- 172.21.15.105:0/458738719 learned_addr learned my addr 172.21.15.105:0/458738719 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669879640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9664101450 0x7f96641a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9668877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ab0 0x7f966419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669879640 1 -- 172.21.15.105:0/458738719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9664100150 msgr2=0x7f9664199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669879640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9664100150 0x7f9664199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669879640 1 -- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ab0 msgr2=0x7f966419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669879640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ab0 0x7f966419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669879640 1 -- 172.21.15.105:0/458738719 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96641a1800 con 0x7f9664101450 2024-03-26T17:47:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9668877640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ab0 0x7f966419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:47:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f9669879640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9664101450 0x7f96641a1190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f965c02c630 tx=0x7f965c0328a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f965a7fc640 1 -- 172.21.15.105:0/458738719 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c041020 con 0x7f9664101450 2024-03-26T17:47:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f965a7fc640 1 -- 172.21.15.105:0/458738719 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f965c003e20 con 0x7f9664101450 2024-03-26T17:47:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.321+0000 7f966b2dc640 1 -- 172.21.15.105:0/458738719 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96641a1a90 con 0x7f9664101450 2024-03-26T17:47:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.322+0000 7f965a7fc640 1 -- 172.21.15.105:0/458738719 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c039040 con 0x7f9664101450 2024-03-26T17:47:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.322+0000 7f966b2dc640 1 -- 172.21.15.105:0/458738719 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96641a1fd0 con 0x7f9664101450 2024-03-26T17:47:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.324+0000 7f965a7fc640 1 -- 172.21.15.105:0/458738719 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f965c044020 con 0x7f9664101450 2024-03-26T17:47:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.324+0000 7f965a7fc640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96500a3040 0x7f96500a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f965a7fc640 1 -- 172.21.15.105:0/458738719 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f965c0f1670 con 0x7f9664101450 2024-03-26T17:47:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f9669078640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96500a3040 0x7f96500a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f966dfa3740 1 Processor -- start 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f966dfa3740 1 -- start start 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e56e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e573ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e5765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f966dfa3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563b1e4d30b0 con 0x563b1e56efc0 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f966dfa3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b1e47b960 con 0x563b1e56e5d0 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f966dfa3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563b1e4cf060 con 0x563b1e574130 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.325+0000 7f9668877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e573ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9668877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e573ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59856/0 (socket says 172.21.15.105:59856) 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9668877640 1 -- 172.21.15.105:0/1457883604 learned_addr learned my addr 172.21.15.105:0/1457883604 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9669078640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96500a3040 0x7f96500a5500 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9660005fd0 tx=0x7f966000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9669879640 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e5765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9669078640 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e56e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9668877640 1 -- 172.21.15.105:0/1457883604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 msgr2=0x563b1e5765c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9668877640 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e5765c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9668877640 1 -- 172.21.15.105:0/1457883604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 msgr2=0x563b1e56e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9668877640 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e56e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9668877640 1 -- 172.21.15.105:0/1457883604 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1e48dc60 con 0x563b1e56efc0 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9669879640 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e5765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f9668877640 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e573ac0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9654009ce0 tx=0x7f96540117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe69bed4273bbe07 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f964f7fe640 1 -- 172.21.15.105:0/1457883604 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654012450 con 0x563b1e56efc0 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f964f7fe640 1 -- 172.21.15.105:0/1457883604 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9654012aa0 con 0x563b1e56efc0 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.326+0000 7f964f7fe640 1 -- 172.21.15.105:0/1457883604 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965400fc30 con 0x563b1e56efc0 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.327+0000 7f966dfa3740 1 -- 172.21.15.105:0/1457883604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 msgr2=0x563b1e573ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.327+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e573ac0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f9654009ce0 tx=0x7f96540117b0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.327+0000 7f966dfa3740 1 -- 172.21.15.105:0/1457883604 shutdown_connections 2024-03-26T17:47:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.327+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e5765c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.327+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e573ac0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.327+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1457883604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e56e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.327+0000 7f966dfa3740 1 -- 172.21.15.105:0/1457883604 >> 172.21.15.105:0/1457883604 conn(0x563b1e569b40 msgr2=0x563b1e569f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.328+0000 7f966dfa3740 1 -- 172.21.15.105:0/1457883604 shutdown_connections 2024-03-26T17:47:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.328+0000 7f966dfa3740 1 -- 172.21.15.105:0/1457883604 wait complete. 2024-03-26T17:47:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.328+0000 7f966dfa3740 1 -- start start 2024-03-26T17:47:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.328+0000 7f966dfa3740 1 Processor -- start 2024-03-26T17:47:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e5783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e5788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9668877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e5788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669879640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9668877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e5788e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59872/0 (socket says 172.21.15.105:59872) 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669879640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33466/0 (socket says 172.21.15.105:33466) 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e5783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9668877640 1 -- 172.21.15.105:0/1074615949 learned_addr learned my addr 172.21.15.105:0/1074615949 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563b1e5770e0 con 0x563b1e56efc0 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b1e48edb0 con 0x563b1e56e5d0 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563b1e577260 con 0x563b1e574130 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f966dfa3740 10 client.0 _ll_register_callbacks cb 0x563b1e493110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669879640 1 -- 172.21.15.105:0/1074615949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 msgr2=0x563b1e5783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669879640 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e5783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669879640 1 -- 172.21.15.105:0/1074615949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 msgr2=0x563b1e5788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669879640 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e5788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669879640 1 -- 172.21.15.105:0/1074615949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1e1e7210 con 0x563b1e574130 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f9669879640 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57f920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f965c0ef130 tx=0x7f965c004900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa54e8c8afb5991 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.329+0000 7f964cff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f964cff9640 1 -- 172.21.15.105:0/1074615949 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c0098b0 con 0x563b1e574130 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f964cff9640 1 -- 172.21.15.105:0/1074615949 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f965c003970 con 0x563b1e574130 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9654012aa0 con 0x563b1e574130 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f964cff9640 1 -- 172.21.15.105:0/1074615949 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f965c0bdd60 con 0x563b1e574130 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 10 client.28768 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b1e2b3620 con 0x563b1e574130 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f964cff9640 1 -- 172.21.15.105:0/1074615949 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9654012aa0 con 0x563b1e574130 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f964cff9640 1 -- 172.21.15.105:0/1074615949 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f965c03c030 con 0x563b1e574130 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 10 client.28768 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 10 client.28768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 20 client.28768 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b1e5842f0 0x563b1e586720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 4 client.28768 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.330+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b1e4e05f0 con 0x563b1e5842f0 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.331+0000 7f9669078640 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b1e5842f0 0x563b1e586720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.331+0000 7f964cff9640 10 client.28768 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.331+0000 7f9669078640 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b1e5842f0 0x563b1e586720 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6057e32ad2dffd server_cookie=5921e3bb9b5628b9 in_seq=0 out_seq=0 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f964cff9640 1 -- 172.21.15.105:0/1074615949 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563b1e4e05f0 con 0x563b1e5842f0 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f964cff9640 10 client.28768 handle_command_reply: tid=0 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 client.28768 shutdown 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 msgr2=0x563b1e57f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57f920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f965c0ef130 tx=0x7f965c004900 comp rx=0 tx=0).stop 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 shutdown_connections 2024-03-26T17:47:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b1e5842f0 0x563b1e586720 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f964cff9640 0 client.28768 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57f920 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e5788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1074615949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e5783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.332+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 >> 172.21.15.105:0/1074615949 conn(0x563b1e569b40 msgr2=0x563b1e568df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.333+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 shutdown_connections 2024-03-26T17:47:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.333+0000 7f966dfa3740 1 -- 172.21.15.105:0/1074615949 wait complete. 2024-03-26T17:47:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.333+0000 7f966dfa3740 20 client.28768 trim_cache size 0 max 16384 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f966dfa3740 1 Processor -- start 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f966dfa3740 1 -- start start 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56e5d0 0x563b1e56f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e56efc0 0x563b1e57f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e574130 0x563b1e56fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f966dfa3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96641aad90 con 0x563b1e56e5d0 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f966dfa3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96641ac8d0 con 0x563b1e574130 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f966dfa3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96641ab460 con 0x563b1e56efc0 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f9669078640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56e5d0 0x563b1e56f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f9668877640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e56efc0 0x563b1e57f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f9668877640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e56efc0 0x563b1e57f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33472/0 (socket says 172.21.15.105:33472) 2024-03-26T17:47:54.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f9668877640 1 -- 172.21.15.105:0/1817704178 learned_addr learned my addr 172.21.15.105:0/1817704178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.351+0000 7f9669879640 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e574130 0x563b1e56fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f9668877640 1 -- 172.21.15.105:0/1817704178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e574130 msgr2=0x563b1e56fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f9668877640 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e574130 0x563b1e56fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f9668877640 1 -- 172.21.15.105:0/1817704178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56e5d0 msgr2=0x563b1e56f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f9668877640 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56e5d0 0x563b1e56f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f9668877640 1 -- 172.21.15.105:0/1817704178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1e565450 con 0x563b1e56efc0 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f9669078640 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56e5d0 0x563b1e56f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f9668877640 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e56efc0 0x563b1e57f8e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9654009e70 tx=0x7f9654009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11775ff9a29ebe30 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f964effd640 1 -- 172.21.15.105:0/1817704178 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965402b070 con 0x563b1e56efc0 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f964effd640 1 -- 172.21.15.105:0/1817704178 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96540054f0 con 0x563b1e56efc0 2024-03-26T17:47:54.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.352+0000 7f964effd640 1 -- 172.21.15.105:0/1817704178 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654004b20 con 0x563b1e56efc0 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 -- 172.21.15.105:0/1817704178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e56efc0 msgr2=0x563b1e57f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e56efc0 0x563b1e57f8e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9654009e70 tx=0x7f9654009ea0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 -- 172.21.15.105:0/1817704178 shutdown_connections 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e574130 0x563b1e56fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e56efc0 0x563b1e57f8e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1817704178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56e5d0 0x563b1e56f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 -- 172.21.15.105:0/1817704178 >> 172.21.15.105:0/1817704178 conn(0x563b1e569b40 msgr2=0x563b1e5783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 -- 172.21.15.105:0/1817704178 shutdown_connections 2024-03-26T17:47:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 -- 172.21.15.105:0/1817704178 wait complete. 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 -- start start 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.353+0000 7f966dfa3740 1 Processor -- start 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e57cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e57d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f966dfa3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f966dfa3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96641ac200 con 0x563b1e56efc0 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f966dfa3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96641abb30 con 0x563b1e56e5d0 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f966dfa3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96641a8d70 con 0x563b1e574130 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f966dfa3740 10 client.0 _ll_register_callbacks cb 0x7f965c0bdb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9669078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e57cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9668877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e57d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9669078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e57cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41050/0 (socket says 172.21.15.105:41050) 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9669078640 1 -- 172.21.15.105:0/1061873806 learned_addr learned my addr 172.21.15.105:0/1061873806 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9669879640 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9668877640 1 -- 172.21.15.105:0/1061873806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 msgr2=0x563b1e57e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9668877640 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9668877640 1 -- 172.21.15.105:0/1061873806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 msgr2=0x563b1e57cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9668877640 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e57cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9668877640 1 -- 172.21.15.105:0/1061873806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1e57e6e0 con 0x563b1e56efc0 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9669879640 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9669078640 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e57cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f9668877640 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e57d0e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9654004770 tx=0x7f9654012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28acebc0bea0542a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.354+0000 7f964cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f964cff9640 1 -- 172.21.15.105:0/1061873806 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654034070 con 0x563b1e56efc0 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f964cff9640 1 -- 172.21.15.105:0/1061873806 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9654012d00 con 0x563b1e56efc0 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 1 -- 172.21.15.105:0/1061873806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96540054f0 con 0x563b1e56efc0 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f964cff9640 1 -- 172.21.15.105:0/1061873806 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9654040050 con 0x563b1e56efc0 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 10 client.18939 fetch_fsmap learned FSMap version 30 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 1 -- 172.21.15.105:0/1061873806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b1e57f9f0 con 0x563b1e56efc0 2024-03-26T17:47:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f964cff9640 1 -- 172.21.15.105:0/1061873806 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96540054f0 con 0x563b1e56efc0 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f964cff9640 1 -- 172.21.15.105:0/1061873806 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9654025440 con 0x563b1e56efc0 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 10 client.18939 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 10 client.18939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 20 client.18939 populate_metadata read hostname 'smithi105' 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b1e5f31b0 0x563b1e5f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 4 client.18939 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.355+0000 7f966dfa3740 1 -- 172.21.15.105:0/1061873806 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b1e4da940 con 0x563b1e5f31b0 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.356+0000 7f9669078640 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b1e5f31b0 0x563b1e5f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.356+0000 7f964cff9640 10 client.18939 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.356+0000 7f9669078640 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b1e5f31b0 0x563b1e5f55d0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83918135fd304e1c server_cookie=b28d2f3adbfd9c65 in_seq=0 out_seq=0 2024-03-26T17:47:54.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:53 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:47:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f964cff9640 1 -- 172.21.15.105:0/1061873806 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b1e4da940 con 0x563b1e5f31b0 2024-03-26T17:47:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f964cff9640 10 client.18939 handle_command_reply: tid=0 2024-03-26T17:47:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 client.18939 shutdown 2024-03-26T17:47:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 -- 172.21.15.105:0/1061873806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 msgr2=0x563b1e57d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e57d0e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9654004770 tx=0x7f9654012700 comp rx=0 tx=0).stop 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 -- 172.21.15.105:0/1061873806 shutdown_connections 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b1e5f31b0 0x563b1e5f55d0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b1e574130 0x563b1e57e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b1e56efc0 0x563b1e57d0e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 --2- 172.21.15.105:0/1061873806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b1e56e5d0 0x563b1e57cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f964cff9640 0 client.18939 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 -- 172.21.15.105:0/1061873806 >> 172.21.15.105:0/1061873806 conn(0x563b1e569b40 msgr2=0x563b1e56cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 -- 172.21.15.105:0/1061873806 shutdown_connections 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 1 -- 172.21.15.105:0/1061873806 wait complete. 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.399+0000 7f966dfa3740 20 client.18939 trim_cache size 0 max 16384 2024-03-26T17:47:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:47:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:47:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:47:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 -- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96500a3040 msgr2=0x7f96500a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96500a3040 0x7f96500a5500 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9660005fd0 tx=0x7f966000b480 comp rx=0 tx=0).stop 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 -- 172.21.15.105:0/458738719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9664101450 msgr2=0x7f96641a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9664101450 0x7f96641a1190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f965c02c630 tx=0x7f965c0328a0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 -- 172.21.15.105:0/458738719 shutdown_connections 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96500a3040 0x7f96500a5500 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9664101450 0x7f96641a1190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9664100ab0 0x7f966419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 --2- 172.21.15.105:0/458738719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9664100150 0x7f9664199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 -- 172.21.15.105:0/458738719 >> 172.21.15.105:0/458738719 conn(0x7f96640fbd50 msgr2=0x7f96640fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 -- 172.21.15.105:0/458738719 shutdown_connections 2024-03-26T17:47:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:47:54.400+0000 7f966b2dc640 1 -- 172.21.15.105:0/458738719 wait complete. 2024-03-26T17:47:55.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 304239 3.15 MB/sec execute 398 sec latency 37.013 ms 2024-03-26T17:47:55.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:47:55.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:47:55.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:54 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:47:55.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:54 smithi105 ceph-mon[32280]: pgmap v635: 97 pgs: 97 active+clean; 26 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:47:55.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:47:55.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:47:55.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:54 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:47:55.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:54 smithi135 ceph-mon[35636]: pgmap v635: 97 pgs: 97 active+clean; 26 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:47:55.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:47:55.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:47:55.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:54 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:47:55.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:54 smithi144 ceph-mon[36095]: pgmap v635: 97 pgs: 97 active+clean; 26 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:47:56.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 304854 3.15 MB/sec execute 399 sec latency 17.365 ms 2024-03-26T17:47:57.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 305428 3.16 MB/sec execute 400 sec latency 16.552 ms 2024-03-26T17:47:58.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 305889 3.15 MB/sec execute 401 sec latency 26.697 ms 2024-03-26T17:47:58.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:57 smithi144 ceph-mon[36095]: pgmap v636: 97 pgs: 97 active+clean; 27 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:47:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:57 smithi105 ceph-mon[32280]: pgmap v636: 97 pgs: 97 active+clean; 27 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:47:58.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:57 smithi135 ceph-mon[35636]: pgmap v636: 97 pgs: 97 active+clean; 27 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:47:59.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 306529 3.15 MB/sec execute 402 sec latency 16.983 ms 2024-03-26T17:48:00.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 307165 3.15 MB/sec execute 403 sec latency 16.470 ms 2024-03-26T17:48:00.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:47:59 smithi144 ceph-mon[36095]: pgmap v637: 97 pgs: 97 active+clean; 27 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2024-03-26T17:48:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:47:59 smithi105 ceph-mon[32280]: pgmap v637: 97 pgs: 97 active+clean; 27 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2024-03-26T17:48:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:47:59 smithi135 ceph-mon[35636]: pgmap v637: 97 pgs: 97 active+clean; 27 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2024-03-26T17:48:01.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 307835 3.15 MB/sec execute 404 sec latency 16.992 ms 2024-03-26T17:48:02.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 308423 3.15 MB/sec execute 405 sec latency 23.757 ms 2024-03-26T17:48:02.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:01 smithi144 ceph-mon[36095]: pgmap v638: 97 pgs: 97 active+clean; 30 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:48:02.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:01 smithi105 ceph-mon[32280]: pgmap v638: 97 pgs: 97 active+clean; 30 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:48:02.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:01 smithi135 ceph-mon[35636]: pgmap v638: 97 pgs: 97 active+clean; 30 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:48:03.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 308973 3.16 MB/sec execute 406 sec latency 57.435 ms 2024-03-26T17:48:04.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 309421 3.15 MB/sec execute 407 sec latency 22.960 ms 2024-03-26T17:48:04.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:03 smithi144 ceph-mon[36095]: pgmap v639: 97 pgs: 97 active+clean; 29 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:48:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:03 smithi105 ceph-mon[32280]: pgmap v639: 97 pgs: 97 active+clean; 29 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:48:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:03 smithi135 ceph-mon[35636]: pgmap v639: 97 pgs: 97 active+clean; 29 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:48:05.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 310051 3.15 MB/sec execute 408 sec latency 16.154 ms 2024-03-26T17:48:06.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 310695 3.15 MB/sec execute 409 sec latency 17.069 ms 2024-03-26T17:48:06.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:05 smithi144 ceph-mon[36095]: pgmap v640: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-03-26T17:48:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:05 smithi105 ceph-mon[32280]: pgmap v640: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-03-26T17:48:06.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:05 smithi135 ceph-mon[35636]: pgmap v640: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-03-26T17:48:07.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 311333 3.15 MB/sec execute 410 sec latency 16.456 ms 2024-03-26T17:48:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 -- 172.21.15.105:0/2462693938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2981023d0 msgr2=0x7fc298108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 --2- 172.21.15.105:0/2462693938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2981023d0 0x7fc298108c60 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc288009ca0 tx=0x7fc28802c120 comp rx=0 tx=0).stop 2024-03-26T17:48:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 -- 172.21.15.105:0/2462693938 shutdown_connections 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 --2- 172.21.15.105:0/2462693938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2981023d0 0x7fc298108c60 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 --2- 172.21.15.105:0/2462693938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc298101a10 0x7fc298101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 --2- 172.21.15.105:0/2462693938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc298100820 0x7fc298100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 -- 172.21.15.105:0/2462693938 >> 172.21.15.105:0/2462693938 conn(0x7fc2980fbd90 msgr2=0x7fc2980fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 -- 172.21.15.105:0/2462693938 shutdown_connections 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.498+0000 7fc29c88c640 1 -- 172.21.15.105:0/2462693938 wait complete. 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29c88c640 1 Processor -- start 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29c88c640 1 -- start start 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29c88c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc298100820 0x7fc2981047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29c88c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc298101a10 0x7fc298102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29c88c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2981023d0 0x7fc298103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29c88c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc29810c3f0 con 0x7fc298100820 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29c88c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc29810c270 con 0x7fc298101a10 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29c88c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc29810c570 con 0x7fc2981023d0 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc298100820 0x7fc2981047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc298100820 0x7fc2981047b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52852/0 (socket says 172.21.15.105:52852) 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc29659c640 1 -- 172.21.15.105:0/4025289466 learned_addr learned my addr 172.21.15.105:0/4025289466 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc295d9b640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc298101a10 0x7fc298102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.499+0000 7fc296d9d640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2981023d0 0x7fc298103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc29659c640 1 -- 172.21.15.105:0/4025289466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2981023d0 msgr2=0x7fc298103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc29659c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2981023d0 0x7fc298103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc29659c640 1 -- 172.21.15.105:0/4025289466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc298101a10 msgr2=0x7fc298102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc29659c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc298101a10 0x7fc298102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc29659c640 1 -- 172.21.15.105:0/4025289466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc298103bc0 con 0x7fc298100820 2024-03-26T17:48:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc29659c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc298100820 0x7fc2981047b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc29000b470 tx=0x7fc29000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc2877fe640 1 -- 172.21.15.105:0/4025289466 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290015020 con 0x7fc298100820 2024-03-26T17:48:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc29c88c640 1 -- 172.21.15.105:0/4025289466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2981a1c40 con 0x7fc298100820 2024-03-26T17:48:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc2877fe640 1 -- 172.21.15.105:0/4025289466 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc290011d40 con 0x7fc298100820 2024-03-26T17:48:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc2877fe640 1 -- 172.21.15.105:0/4025289466 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29000f340 con 0x7fc298100820 2024-03-26T17:48:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.500+0000 7fc29c88c640 1 -- 172.21.15.105:0/4025289466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2981a2190 con 0x7fc298100820 2024-03-26T17:48:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.503+0000 7fc2877fe640 1 -- 172.21.15.105:0/4025289466 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc290018020 con 0x7fc298100820 2024-03-26T17:48:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.503+0000 7fc29f472740 1 Processor -- start 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.503+0000 7fc29f472740 1 -- start start 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.503+0000 7fc2877fe640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2800a2fc0 0x7fc2800a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.503+0000 7fc29f472740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d0607439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.503+0000 7fc29f472740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060743fc0 0x55d060748ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc29f472740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d06074b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc29f472740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d0606a80b0 con 0x55d060743fc0 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc29f472740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d060650960 con 0x55d0607435d0 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc29f472740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0606a4060 con 0x55d060749130 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc296d9d640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2800a2fc0 0x7fc2800a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060743fc0 0x55d060748ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060743fc0 0x55d060748ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52854/0 (socket says 172.21.15.105:52854) 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc29659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d0607439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 -- 172.21.15.105:0/1393879479 learned_addr learned my addr 172.21.15.105:0/1393879479 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc2877fe640 1 -- 172.21.15.105:0/4025289466 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc2900cee30 con 0x7fc298100820 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 -- 172.21.15.105:0/1393879479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 msgr2=0x55d06074b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 --2- 172.21.15.105:0/1393879479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d06074b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 -- 172.21.15.105:0/1393879479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 msgr2=0x55d0607439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 --2- 172.21.15.105:0/1393879479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d0607439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 -- 172.21.15.105:0/1393879479 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d060662c60 con 0x55d060743fc0 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc295d9b640 1 --2- 172.21.15.105:0/1393879479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060743fc0 0x55d060748ac0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009ca0 tx=0x7fc28c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520a047dd733212c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc296d9d640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2800a2fc0 0x7fc2800a5480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc28802c660 tx=0x7fc28802ce00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc284ff9640 1 -- 172.21.15.105:0/1393879479 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c019620 con 0x55d060743fc0 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc284ff9640 1 -- 172.21.15.105:0/1393879479 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc28c019c70 con 0x55d060743fc0 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.504+0000 7fc284ff9640 1 -- 172.21.15.105:0/1393879479 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c018a80 con 0x55d060743fc0 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 -- 172.21.15.105:0/1393879479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060743fc0 msgr2=0x55d060748ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 --2- 172.21.15.105:0/1393879479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060743fc0 0x55d060748ac0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009ca0 tx=0x7fc28c011810 comp rx=0 tx=0).stop 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 -- 172.21.15.105:0/1393879479 shutdown_connections 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 --2- 172.21.15.105:0/1393879479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d06074b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 --2- 172.21.15.105:0/1393879479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060743fc0 0x55d060748ac0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 --2- 172.21.15.105:0/1393879479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d0607439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 -- 172.21.15.105:0/1393879479 >> 172.21.15.105:0/1393879479 conn(0x55d06073eb40 msgr2=0x55d06073ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 -- 172.21.15.105:0/1393879479 shutdown_connections 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.505+0000 7fc29f472740 1 -- 172.21.15.105:0/1393879479 wait complete. 2024-03-26T17:48:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 1 -- start start 2024-03-26T17:48:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 1 Processor -- start 2024-03-26T17:48:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d06074d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 0x55d06074d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d060754850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d06074c0e0 con 0x55d0607435d0 2024-03-26T17:48:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d060663db0 con 0x55d060743fc0 2024-03-26T17:48:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d06074c260 con 0x55d060749130 2024-03-26T17:48:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d06074d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d06074d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52870/0 (socket says 172.21.15.105:52870) 2024-03-26T17:48:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29659c640 1 -- 172.21.15.105:0/3214268098 learned_addr learned my addr 172.21.15.105:0/3214268098 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc295d9b640 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 0x55d06074d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc29f472740 10 client.0 _ll_register_callbacks cb 0x55d060668110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.506+0000 7fc296d9d640 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d060754850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29659c640 1 -- 172.21.15.105:0/3214268098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 msgr2=0x55d060754850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29659c640 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d060754850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29659c640 1 -- 172.21.15.105:0/3214268098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 msgr2=0x55d06074d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29659c640 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 0x55d06074d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29659c640 1 -- 172.21.15.105:0/3214268098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0603bc210 con 0x55d0607435d0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29659c640 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d06074d340 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc290097ef0 tx=0x7fc29001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a571bc0907d857 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc27dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc27dffb640 1 -- 172.21.15.105:0/3214268098 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29009ae80 con 0x55d0607435d0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc27dffb640 1 -- 172.21.15.105:0/3214268098 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc2900649b0 con 0x55d0607435d0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29f472740 1 -- 172.21.15.105:0/3214268098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc28c019c70 con 0x55d0607435d0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc27dffb640 1 -- 172.21.15.105:0/3214268098 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc290064220 con 0x55d0607435d0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29f472740 10 client.18951 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc29f472740 1 -- 172.21.15.105:0/3214268098 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d060488620 con 0x55d0607435d0 2024-03-26T17:48:07.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc27dffb640 1 -- 172.21.15.105:0/3214268098 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc28c019c70 con 0x55d0607435d0 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.507+0000 7fc27dffb640 1 -- 172.21.15.105:0/3214268098 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc2900a3aa0 con 0x55d0607435d0 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc29f472740 10 client.18951 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc29f472740 10 client.18951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc29f472740 20 client.18951 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc29f472740 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0607591a0 0x55d06075b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc29f472740 4 client.18951 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc29f472740 1 -- 172.21.15.105:0/3214268098 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0606b55f0 con 0x55d0607591a0 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc295d9b640 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0607591a0 0x55d06075b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc295d9b640 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0607591a0 0x55d06075b5d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9f19a9102141a3 server_cookie=a25e7ac84826898d in_seq=0 out_seq=0 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.508+0000 7fc27dffb640 10 client.18951 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc27dffb640 1 -- 172.21.15.105:0/3214268098 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d0606b55f0 con 0x55d0607591a0 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc27dffb640 10 client.18951 handle_command_reply: tid=0 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 client.18951 shutdown 2024-03-26T17:48:07.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 -- 172.21.15.105:0/3214268098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 msgr2=0x55d06074d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d06074d340 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc290097ef0 tx=0x7fc29001f3f0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 -- 172.21.15.105:0/3214268098 shutdown_connections 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0607591a0 0x55d06075b5d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d060754850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc27dffb640 0 client.18951 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 0x55d06074d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 --2- 172.21.15.105:0/3214268098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d06074d340 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 -- 172.21.15.105:0/3214268098 >> 172.21.15.105:0/3214268098 conn(0x55d06073eb40 msgr2=0x55d06073ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 -- 172.21.15.105:0/3214268098 shutdown_connections 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 1 -- 172.21.15.105:0/3214268098 wait complete. 2024-03-26T17:48:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.509+0000 7fc29f472740 20 client.18951 trim_cache size 0 max 16384 2024-03-26T17:48:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.526+0000 7fc29f472740 1 Processor -- start 2024-03-26T17:48:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29f472740 1 -- start start 2024-03-26T17:48:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29f472740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d060745170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29f472740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 0x55d060754810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29f472740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d060760ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29f472740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc2981aaf30 con 0x55d0607435d0 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29f472740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2981aca70 con 0x55d060743fc0 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29f472740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2981ab600 con 0x55d060749130 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d060745170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d060745170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52886/0 (socket says 172.21.15.105:52886) 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 -- 172.21.15.105:0/594583907 learned_addr learned my addr 172.21.15.105:0/594583907 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc295d9b640 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 0x55d060754810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc296d9d640 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d060760ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 -- 172.21.15.105:0/594583907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 msgr2=0x55d060760ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d060760ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 -- 172.21.15.105:0/594583907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 msgr2=0x55d060754810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 0x55d060754810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 -- 172.21.15.105:0/594583907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06073f8b0 con 0x55d0607435d0 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.527+0000 7fc29659c640 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d060745170 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fc2900997d0 tx=0x7fc290064870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefebca5902f51af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.528+0000 7fc27ffff640 1 -- 172.21.15.105:0/594583907 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290080070 con 0x55d0607435d0 2024-03-26T17:48:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.528+0000 7fc27ffff640 1 -- 172.21.15.105:0/594583907 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc290010570 con 0x55d0607435d0 2024-03-26T17:48:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.528+0000 7fc27ffff640 1 -- 172.21.15.105:0/594583907 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290064600 con 0x55d0607435d0 2024-03-26T17:48:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.528+0000 7fc29f472740 1 -- 172.21.15.105:0/594583907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 msgr2=0x55d060745170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.528+0000 7fc29f472740 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d060745170 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fc2900997d0 tx=0x7fc290064870 comp rx=0 tx=0).stop 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.528+0000 7fc29f472740 1 -- 172.21.15.105:0/594583907 shutdown_connections 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.528+0000 7fc29f472740 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060749130 0x55d060760ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.528+0000 7fc29f472740 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d060743fc0 0x55d060754810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 --2- 172.21.15.105:0/594583907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0607435d0 0x55d060745170 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 -- 172.21.15.105:0/594583907 >> 172.21.15.105:0/594583907 conn(0x55d06073eb40 msgr2=0x55d060744a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 -- 172.21.15.105:0/594583907 shutdown_connections 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 -- 172.21.15.105:0/594583907 wait complete. 2024-03-26T17:48:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 -- start start 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 Processor -- start 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d060754160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060743fc0 0x55d0607546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060749130 0x55d060752770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc2981ac3a0 con 0x55d060749130 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2981abcd0 con 0x55d0607435d0 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2981a8f10 con 0x55d060743fc0 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29f472740 10 client.0 _ll_register_callbacks cb 0x7fc290097100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc296d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060749130 0x55d060752770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc296d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060749130 0x55d060752770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52892/0 (socket says 172.21.15.105:52892) 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc29659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d060754160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc296d9d640 1 -- 172.21.15.105:0/460531671 learned_addr learned my addr 172.21.15.105:0/460531671 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.529+0000 7fc295d9b640 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060743fc0 0x55d0607546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29659c640 1 -- 172.21.15.105:0/460531671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060743fc0 msgr2=0x55d0607546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29659c640 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060743fc0 0x55d0607546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29659c640 1 -- 172.21.15.105:0/460531671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060749130 msgr2=0x55d060752770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29659c640 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060749130 0x55d060752770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29659c640 1 -- 172.21.15.105:0/460531671 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d060752eb0 con 0x55d0607435d0 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc296d9d640 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060749130 0x55d060752770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29659c640 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d060754160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc290097980 tx=0x7fc290010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987d20ac1be61f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc27dffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc27dffb640 1 -- 172.21.15.105:0/460531671 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290080070 con 0x55d0607435d0 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc27dffb640 1 -- 172.21.15.105:0/460531671 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc290011890 con 0x55d0607435d0 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29f472740 1 -- 172.21.15.105:0/460531671 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d0606af940 con 0x55d0607435d0 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc27dffb640 1 -- 172.21.15.105:0/460531671 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc29009bc20 con 0x55d0607435d0 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29f472740 10 client.28781 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc29f472740 1 -- 172.21.15.105:0/460531671 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d060752eb0 con 0x55d0607435d0 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.530+0000 7fc27dffb640 1 -- 172.21.15.105:0/460531671 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2900a5070 con 0x55d0607435d0 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc27dffb640 1 -- 172.21.15.105:0/460531671 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc2900a8020 con 0x55d0607435d0 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc29f472740 10 client.28781 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc29f472740 10 client.28781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc29f472740 20 client.28781 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc29f472740 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0607591a0 0x55d0607c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc29f472740 4 client.28781 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:48:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc29f472740 1 -- 172.21.15.105:0/460531671 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0606af940 con 0x55d0607591a0 2024-03-26T17:48:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc295d9b640 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0607591a0 0x55d0607c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc295d9b640 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0607591a0 0x55d0607c24e0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7079e766e7717d server_cookie=854f71c3b030414c in_seq=0 out_seq=0 2024-03-26T17:48:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.531+0000 7fc27dffb640 10 client.28781 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.575+0000 7fc27dffb640 1 -- 172.21.15.105:0/460531671 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0606af940 con 0x55d0607591a0 2024-03-26T17:48:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.575+0000 7fc27dffb640 10 client.28781 handle_command_reply: tid=0 2024-03-26T17:48:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.575+0000 7fc29f472740 1 client.28781 shutdown 2024-03-26T17:48:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.575+0000 7fc29f472740 1 -- 172.21.15.105:0/460531671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 msgr2=0x55d060754160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.575+0000 7fc29f472740 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d060754160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc290097980 tx=0x7fc290010890 comp rx=0 tx=0).stop 2024-03-26T17:48:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 1 -- 172.21.15.105:0/460531671 shutdown_connections 2024-03-26T17:48:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0607591a0 0x55d0607c24e0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d060749130 0x55d060752770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d060743fc0 0x55d0607546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 1 --2- 172.21.15.105:0/460531671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0607435d0 0x55d060754160 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc27dffb640 0 client.28781 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 1 -- 172.21.15.105:0/460531671 >> 172.21.15.105:0/460531671 conn(0x55d06073eb40 msgr2=0x55d06074cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 1 -- 172.21.15.105:0/460531671 shutdown_connections 2024-03-26T17:48:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 1 -- 172.21.15.105:0/460531671 wait complete. 2024-03-26T17:48:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29f472740 20 client.28781 trim_cache size 0 max 16384 2024-03-26T17:48:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:48:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:48:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:48:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:48:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 -- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2800a2fc0 msgr2=0x7fc2800a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2800a2fc0 0x7fc2800a5480 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc28802c660 tx=0x7fc28802ce00 comp rx=0 tx=0).stop 2024-03-26T17:48:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 -- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc298100820 msgr2=0x7fc2981047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc298100820 0x7fc2981047b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc29000b470 tx=0x7fc29000b930 comp rx=0 tx=0).stop 2024-03-26T17:48:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 -- 172.21.15.105:0/4025289466 shutdown_connections 2024-03-26T17:48:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2800a2fc0 0x7fc2800a5480 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2981023d0 0x7fc298103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc298101a10 0x7fc298102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 --2- 172.21.15.105:0/4025289466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc298100820 0x7fc2981047b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.576+0000 7fc29c88c640 1 -- 172.21.15.105:0/4025289466 >> 172.21.15.105:0/4025289466 conn(0x7fc2980fbd90 msgr2=0x7fc2980fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.577+0000 7fc29c88c640 1 -- 172.21.15.105:0/4025289466 shutdown_connections 2024-03-26T17:48:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:07.577+0000 7fc29c88c640 1 -- 172.21.15.105:0/4025289466 wait complete. 2024-03-26T17:48:08.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 311936 3.15 MB/sec execute 411 sec latency 35.406 ms 2024-03-26T17:48:08.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:07 smithi144 ceph-mon[36095]: pgmap v641: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:08.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:07 smithi105 ceph-mon[32280]: pgmap v641: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:07 smithi135 ceph-mon[35636]: pgmap v641: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:09.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 312467 3.16 MB/sec execute 412 sec latency 17.063 ms 2024-03-26T17:48:09.847 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:09 smithi144 ceph-mon[36095]: pgmap v642: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-03-26T17:48:09.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:09 smithi105 ceph-mon[32280]: pgmap v642: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-03-26T17:48:09.914 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:09 smithi135 ceph-mon[35636]: pgmap v642: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-03-26T17:48:10.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 312859 3.15 MB/sec execute 413 sec latency 20.699 ms 2024-03-26T17:48:11.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 313443 3.15 MB/sec execute 414 sec latency 17.114 ms 2024-03-26T17:48:12.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 314158 3.15 MB/sec execute 415 sec latency 15.287 ms 2024-03-26T17:48:12.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:11 smithi144 ceph-mon[36095]: pgmap v643: 97 pgs: 97 active+clean; 26 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:48:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:11 smithi105 ceph-mon[32280]: pgmap v643: 97 pgs: 97 active+clean; 26 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:48:12.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:11 smithi135 ceph-mon[35636]: pgmap v643: 97 pgs: 97 active+clean; 26 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-03-26T17:48:13.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 314726 3.15 MB/sec execute 416 sec latency 59.007 ms 2024-03-26T17:48:14.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 315472 3.15 MB/sec execute 417 sec latency 15.415 ms 2024-03-26T17:48:14.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:13 smithi144 ceph-mon[36095]: pgmap v644: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:48:14.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:13 smithi105 ceph-mon[32280]: pgmap v644: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:48:14.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:13 smithi135 ceph-mon[35636]: pgmap v644: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:48:15.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 316013 3.16 MB/sec execute 418 sec latency 16.109 ms 2024-03-26T17:48:16.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 316415 3.15 MB/sec execute 419 sec latency 27.789 ms 2024-03-26T17:48:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:15 smithi144 ceph-mon[36095]: pgmap v645: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:48:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:15 smithi105 ceph-mon[32280]: pgmap v645: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:48:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:15 smithi135 ceph-mon[35636]: pgmap v645: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:48:17.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 316989 3.15 MB/sec execute 420 sec latency 16.527 ms 2024-03-26T17:48:18.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 317684 3.15 MB/sec execute 421 sec latency 16.235 ms 2024-03-26T17:48:18.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:17 smithi144 ceph-mon[36095]: pgmap v646: 97 pgs: 97 active+clean; 30 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-03-26T17:48:18.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:17 smithi105 ceph-mon[32280]: pgmap v646: 97 pgs: 97 active+clean; 30 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-03-26T17:48:18.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:17 smithi135 ceph-mon[35636]: pgmap v646: 97 pgs: 97 active+clean; 30 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-03-26T17:48:18.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:48:18.497 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 -- 172.21.15.105:0/716767524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4090102340 msgr2=0x7f4090104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 --2- 172.21.15.105:0/716767524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4090102340 0x7f4090104730 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f408c009900 tx=0x7f408c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 -- 172.21.15.105:0/716767524 shutdown_connections 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 --2- 172.21.15.105:0/716767524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40901075a0 0x7f40901099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 --2- 172.21.15.105:0/716767524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4090104c70 0x7f4090107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 --2- 172.21.15.105:0/716767524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4090102340 0x7f4090104730 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 -- 172.21.15.105:0/716767524 >> 172.21.15.105:0/716767524 conn(0x7f40900fbd50 msgr2=0x7f40900fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 -- 172.21.15.105:0/716767524 shutdown_connections 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.619+0000 7f4097deb640 1 -- 172.21.15.105:0/716767524 wait complete. 2024-03-26T17:48:18.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4097deb640 1 Processor -- start 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4097deb640 1 -- start start 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4097deb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4090102340 0x7f4090197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4097deb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4090104c70 0x7f40901980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4097deb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40901075a0 0x7f409019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4097deb640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f409010d420 con 0x7f40901075a0 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4097deb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f409010d2a0 con 0x7f4090102340 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4097deb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f409010d5a0 con 0x7f4090104c70 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4096388640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40901075a0 0x7f409019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4096388640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40901075a0 0x7f409019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47698/0 (socket says 172.21.15.105:47698) 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4096388640 1 -- 172.21.15.105:0/2336649090 learned_addr learned my addr 172.21.15.105:0/2336649090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4095b87640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4090102340 0x7f4090197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4095386640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4090104c70 0x7f40901980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4096388640 1 -- 172.21.15.105:0/2336649090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4090104c70 msgr2=0x7f40901980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4096388640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4090104c70 0x7f40901980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4096388640 1 -- 172.21.15.105:0/2336649090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4090102340 msgr2=0x7f4090197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4096388640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4090102340 0x7f4090197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4096388640 1 -- 172.21.15.105:0/2336649090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409019f7a0 con 0x7f40901075a0 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.620+0000 7f4095b87640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4090102340 0x7f4090197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.621+0000 7f4096388640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40901075a0 0x7f409019f130 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f408800eb10 tx=0x7f408800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.621+0000 7f4086ffd640 1 -- 172.21.15.105:0/2336649090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088004b20 con 0x7f40901075a0 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.621+0000 7f4097deb640 1 -- 172.21.15.105:0/2336649090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409019fa90 con 0x7f40901075a0 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.621+0000 7f4086ffd640 1 -- 172.21.15.105:0/2336649090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4088004cc0 con 0x7f40901075a0 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.621+0000 7f4086ffd640 1 -- 172.21.15.105:0/2336649090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088010690 con 0x7f40901075a0 2024-03-26T17:48:18.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.621+0000 7f4097deb640 1 -- 172.21.15.105:0/2336649090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409019ff90 con 0x7f40901075a0 2024-03-26T17:48:18.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.623+0000 7f4086ffd640 1 -- 172.21.15.105:0/2336649090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4088005020 con 0x7f40901075a0 2024-03-26T17:48:18.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f4086ffd640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f407c0a2f20 0x7f407c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f409a9cd740 1 Processor -- start 2024-03-26T17:48:18.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f4095b87640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f407c0a2f20 0x7f407c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f4086ffd640 1 -- 172.21.15.105:0/2336649090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f40880cf980 con 0x7f40901075a0 2024-03-26T17:48:18.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f409a9cd740 1 -- start start 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f409a9cd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e6ef40e980 con 0x55e6ef4ae6f0 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f409a9cd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e6ef3b6be0 con 0x55e6ef4b3860 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.624+0000 7f409a9cd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e6ef40a930 con 0x55e6ef4add00 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47710/0 (socket says 172.21.15.105:47710) 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 -- 172.21.15.105:0/3535633078 learned_addr learned my addr 172.21.15.105:0/3535633078 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095b87640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f407c0a2f20 0x7f407c0a53e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f408c009230 tx=0x7f408c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:18.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095b87640 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4096388640 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 -- 172.21.15.105:0/3535633078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 msgr2=0x55e6ef4ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 -- 172.21.15.105:0/3535633078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4b3860 msgr2=0x55e6ef4b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 -- 172.21.15.105:0/3535633078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ef1ec7d0 con 0x55e6ef4ae6f0 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095b87640 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4096388640 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.625+0000 7f4095386640 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b31f0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4080009c60 tx=0x7f4080011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42021d3c29cad5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.626+0000 7f407bfff640 1 -- 172.21.15.105:0/3535633078 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080004230 con 0x55e6ef4ae6f0 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.626+0000 7f407bfff640 1 -- 172.21.15.105:0/3535633078 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40800043d0 con 0x55e6ef4ae6f0 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.626+0000 7f407bfff640 1 -- 172.21.15.105:0/3535633078 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800196d0 con 0x55e6ef4ae6f0 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.626+0000 7f409a9cd740 1 -- 172.21.15.105:0/3535633078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4ae6f0 msgr2=0x55e6ef4b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.626+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b31f0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4080009c60 tx=0x7f4080011810 comp rx=0 tx=0).stop 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 -- 172.21.15.105:0/3535633078 shutdown_connections 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b31f0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3535633078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 -- 172.21.15.105:0/3535633078 >> 172.21.15.105:0/3535633078 conn(0x55e6ef4a9270 msgr2=0x55e6ef4a9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 -- 172.21.15.105:0/3535633078 shutdown_connections 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 -- 172.21.15.105:0/3535633078 wait complete. 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 -- start start 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 Processor -- start 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4add00 0x55e6ef4b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e6ef4a4ad0 con 0x55e6ef4add00 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e6ef3ca160 con 0x55e6ef4ae6f0 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.627+0000 7f409a9cd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e6ef4a4c50 con 0x55e6ef4b3860 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f409a9cd740 10 client.0 _ll_register_callbacks cb 0x55e6ef406720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4add00 0x55e6ef4b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095386640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4add00 0x55e6ef4b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47724/0 (socket says 172.21.15.105:47724) 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 -- 172.21.15.105:0/3241742153 learned_addr learned my addr 172.21.15.105:0/3241742153 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095386640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60876/0 (socket says 172.21.15.105:60876) 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4096388640 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 -- 172.21.15.105:0/3241742153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4b3860 msgr2=0x55e6ef4b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 -- 172.21.15.105:0/3241742153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 msgr2=0x55e6ef4b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:18.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 -- 172.21.15.105:0/3241742153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ef1223f0 con 0x55e6ef4add00 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095386640 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f4095b87640 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4add00 0x55e6ef4b0d10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f408c02c170 tx=0x7f408c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af706e193ed95f5f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f40797fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f40797fa640 1 -- 172.21.15.105:0/3241742153 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c04f070 con 0x55e6ef4add00 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f40797fa640 1 -- 172.21.15.105:0/3241742153 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f408c004990 con 0x55e6ef4add00 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.628+0000 7f409a9cd740 1 -- 172.21.15.105:0/3241742153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40800043d0 con 0x55e6ef4add00 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f40797fa640 1 -- 172.21.15.105:0/3241742153 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f408c05b050 con 0x55e6ef4add00 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f409a9cd740 10 client.18969 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f409a9cd740 1 -- 172.21.15.105:0/3241742153 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6ef0e20f0 con 0x55e6ef4add00 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f40797fa640 1 -- 172.21.15.105:0/3241742153 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40800043d0 con 0x55e6ef4add00 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f40797fa640 1 -- 172.21.15.105:0/3241742153 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f408c04a070 con 0x55e6ef4add00 2024-03-26T17:48:18.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f409a9cd740 10 client.18969 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:18.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f409a9cd740 10 client.18969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:18.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f409a9cd740 20 client.18969 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:18.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6ef4c8150 0x55e6ef4ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f409a9cd740 4 client.18969 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:18.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f409a9cd740 1 -- 172.21.15.105:0/3241742153 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6ef41bec0 con 0x55e6ef4c8150 2024-03-26T17:48:18.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.629+0000 7f4095386640 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6ef4c8150 0x55e6ef4ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.630+0000 7f40797fa640 10 client.18969 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:18.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.630+0000 7f4095386640 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6ef4c8150 0x55e6ef4ca5e0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b759f03005ecd46 server_cookie=bac50698cf56ffff in_seq=0 out_seq=0 2024-03-26T17:48:18.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.630+0000 7f40797fa640 1 -- 172.21.15.105:0/3241742153 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e6ef41bec0 con 0x55e6ef4c8150 2024-03-26T17:48:18.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.630+0000 7f40797fa640 10 client.18969 handle_command_reply: tid=0 2024-03-26T17:48:18.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.630+0000 7f409a9cd740 1 client.18969 shutdown 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 -- 172.21.15.105:0/3241742153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4add00 msgr2=0x55e6ef4b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4add00 0x55e6ef4b0d10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f408c02c170 tx=0x7f408c009260 comp rx=0 tx=0).stop 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 -- 172.21.15.105:0/3241742153 shutdown_connections 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6ef4c8150 0x55e6ef4ca5e0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4b3860 0x55e6ef4b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f40797fa640 0 client.18969 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3241742153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4add00 0x55e6ef4b0d10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 -- 172.21.15.105:0/3241742153 >> 172.21.15.105:0/3241742153 conn(0x55e6ef4a9270 msgr2=0x55e6ef4a4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 -- 172.21.15.105:0/3241742153 shutdown_connections 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 1 -- 172.21.15.105:0/3241742153 wait complete. 2024-03-26T17:48:18.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.631+0000 7f409a9cd740 20 client.18969 trim_cache size 0 max 16384 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.651+0000 7f409a9cd740 1 Processor -- start 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.651+0000 7f409a9cd740 1 -- start start 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f409a9cd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f40901a6ac0 con 0x55e6ef4b3860 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f409a9cd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40901a8600 con 0x55e6ef4ae6f0 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f409a9cd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40901a7190 con 0x55e6ef4add00 2024-03-26T17:48:18.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095b87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095386640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4096388640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095386640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4a37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60888/0 (socket says 172.21.15.105:60888) 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095b87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57418/0 (socket says 172.21.15.105:57418) 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095386640 1 -- 172.21.15.105:0/3702196818 learned_addr learned my addr 172.21.15.105:0/3702196818 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095b87640 1 -- 172.21.15.105:0/3702196818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 msgr2=0x55e6ef4a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095b87640 1 --2- 172.21.15.105:0/3702196818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095b87640 1 -- 172.21.15.105:0/3702196818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 msgr2=0x55e6ef4a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095b87640 1 --2- 172.21.15.105:0/3702196818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095b87640 1 -- 172.21.15.105:0/3702196818 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ef4a4da0 con 0x55e6ef4add00 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4096388640 1 --2- 172.21.15.105:0/3702196818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.652+0000 7f4095b87640 1 --2- 172.21.15.105:0/3702196818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4b1030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f408c0096c0 tx=0x7f408c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b2b5dbaebaec23 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f407b7fe640 1 -- 172.21.15.105:0/3702196818 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c058020 con 0x55e6ef4add00 2024-03-26T17:48:18.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f407b7fe640 1 -- 172.21.15.105:0/3702196818 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f408c045070 con 0x55e6ef4add00 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f407b7fe640 1 -- 172.21.15.105:0/3702196818 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c04f070 con 0x55e6ef4add00 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 -- 172.21.15.105:0/3702196818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 msgr2=0x55e6ef4b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3702196818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4b1030 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f408c0096c0 tx=0x7f408c009780 comp rx=0 tx=0).stop 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 -- 172.21.15.105:0/3702196818 shutdown_connections 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3702196818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3702196818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 --2- 172.21.15.105:0/3702196818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4b1030 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 -- 172.21.15.105:0/3702196818 >> 172.21.15.105:0/3702196818 conn(0x55e6ef4a9270 msgr2=0x55e6ef4b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 -- 172.21.15.105:0/3702196818 shutdown_connections 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.653+0000 7f409a9cd740 1 -- 172.21.15.105:0/3702196818 wait complete. 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 1 -- start start 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 1 Processor -- start 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f40901a7f30 con 0x55e6ef4b3860 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40901a7860 con 0x55e6ef4ae6f0 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40901a4aa0 con 0x55e6ef4add00 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f409a9cd740 10 client.0 _ll_register_callbacks cb 0x7f4080005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f4095386640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f4095b87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f4096388640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f4095386640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4c25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60896/0 (socket says 172.21.15.105:60896) 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.654+0000 7f4095386640 1 -- 172.21.15.105:0/1760604905 learned_addr learned my addr 172.21.15.105:0/1760604905 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f4095b87640 1 -- 172.21.15.105:0/1760604905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 msgr2=0x55e6ef4c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f4095b87640 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f4095b87640 1 -- 172.21.15.105:0/1760604905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 msgr2=0x55e6ef4c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f4095b87640 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4c35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f4095b87640 1 -- 172.21.15.105:0/1760604905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ef4c3be0 con 0x55e6ef4add00 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f4095b87640 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4c20a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f408c009900 tx=0x7f408c003bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc0bc407a0f4291 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f40797fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f40797fa640 1 -- 172.21.15.105:0/1760604905 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c058030 con 0x55e6ef4add00 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f40797fa640 1 -- 172.21.15.105:0/1760604905 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f408c004ba0 con 0x55e6ef4add00 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f409a9cd740 1 -- 172.21.15.105:0/1760604905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408c045070 con 0x55e6ef4add00 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f40797fa640 1 -- 172.21.15.105:0/1760604905 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f408c06f050 con 0x55e6ef4add00 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f409a9cd740 10 client.28804 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f409a9cd740 1 -- 172.21.15.105:0/1760604905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e6ef52e4e0 con 0x55e6ef4add00 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.655+0000 7f40797fa640 1 -- 172.21.15.105:0/1760604905 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f408c045070 con 0x55e6ef4add00 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f40797fa640 1 -- 172.21.15.105:0/1760604905 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f408c04a040 con 0x55e6ef4add00 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f409a9cd740 10 client.28804 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f409a9cd740 10 client.28804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:18.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f409a9cd740 20 client.28804 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f409a9cd740 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6ef52ed70 0x55e6ef531190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f409a9cd740 4 client.28804 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f409a9cd740 1 -- 172.21.15.105:0/1760604905 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6ef416210 con 0x55e6ef52ed70 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f4095386640 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6ef52ed70 0x55e6ef531190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f40797fa640 10 client.28804 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f4095386640 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6ef52ed70 0x55e6ef531190 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e728ad02a269bb8 server_cookie=d8ceddea075e5285 in_seq=0 out_seq=0 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f40797fa640 1 -- 172.21.15.105:0/1760604905 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e6ef416210 con 0x55e6ef52ed70 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f40797fa640 10 client.28804 handle_command_reply: tid=0 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.656+0000 7f409a9cd740 1 client.28804 shutdown 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 -- 172.21.15.105:0/1760604905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 msgr2=0x55e6ef4c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4c20a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f408c009900 tx=0x7f408c003bf0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 -- 172.21.15.105:0/1760604905 shutdown_connections 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e6ef52ed70 0x55e6ef531190 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6ef4b3860 0x55e6ef4c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6ef4ae6f0 0x55e6ef4c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f40797fa640 0 client.28804 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 --2- 172.21.15.105:0/1760604905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6ef4add00 0x55e6ef4c20a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 -- 172.21.15.105:0/1760604905 >> 172.21.15.105:0/1760604905 conn(0x55e6ef4a9270 msgr2=0x55e6ef4acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 -- 172.21.15.105:0/1760604905 shutdown_connections 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 1 -- 172.21.15.105:0/1760604905 wait complete. 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f409a9cd740 20 client.28804 trim_cache size 0 max 16384 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:48:18.655 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:48:18.656 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:48:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f4097deb640 1 -- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f407c0a2f20 msgr2=0x7f407c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f4097deb640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f407c0a2f20 0x7f407c0a53e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f408c009230 tx=0x7f408c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f4097deb640 1 -- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40901075a0 msgr2=0x7f409019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.657+0000 7f4097deb640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40901075a0 0x7f409019f130 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f408800eb10 tx=0x7f408800c2d0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.658+0000 7f4097deb640 1 -- 172.21.15.105:0/2336649090 shutdown_connections 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.658+0000 7f4097deb640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f407c0a2f20 0x7f407c0a53e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.658+0000 7f4097deb640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40901075a0 0x7f409019f130 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.658+0000 7f4097deb640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4090104c70 0x7f40901980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.658+0000 7f4097deb640 1 --2- 172.21.15.105:0/2336649090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4090102340 0x7f4090197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.658+0000 7f4097deb640 1 -- 172.21.15.105:0/2336649090 >> 172.21.15.105:0/2336649090 conn(0x7f40900fbd50 msgr2=0x7f40901041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.658+0000 7f4097deb640 1 -- 172.21.15.105:0/2336649090 shutdown_connections 2024-03-26T17:48:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.658+0000 7f4097deb640 1 -- 172.21.15.105:0/2336649090 wait complete. 2024-03-26T17:48:18.661 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:48:18.661 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:48:18.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.784+0000 7fbc090ff640 1 -- 172.21.15.105:0/2240328495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc04102390 msgr2=0x7fbc04108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.784+0000 7fbc090ff640 1 --2- 172.21.15.105:0/2240328495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc04102390 0x7fbc04108c20 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009be0 tx=0x7fbbfc02bef0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.785+0000 7fbc090ff640 1 -- 172.21.15.105:0/2240328495 shutdown_connections 2024-03-26T17:48:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.785+0000 7fbc090ff640 1 --2- 172.21.15.105:0/2240328495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc04102390 0x7fbc04108c20 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.785+0000 7fbc090ff640 1 --2- 172.21.15.105:0/2240328495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc041019d0 0x7fbc04101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.785+0000 7fbc090ff640 1 --2- 172.21.15.105:0/2240328495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc041007e0 0x7fbc04100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.785+0000 7fbc090ff640 1 -- 172.21.15.105:0/2240328495 >> 172.21.15.105:0/2240328495 conn(0x7fbc040fbd50 msgr2=0x7fbc040fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.785+0000 7fbc090ff640 1 -- 172.21.15.105:0/2240328495 shutdown_connections 2024-03-26T17:48:18.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.785+0000 7fbc090ff640 1 -- 172.21.15.105:0/2240328495 wait complete. 2024-03-26T17:48:18.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.785+0000 7fbc090ff640 1 Processor -- start 2024-03-26T17:48:18.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc090ff640 1 -- start start 2024-03-26T17:48:18.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc090ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc041007e0 0x7fbc0410afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc090ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc041019d0 0x7fbc04109620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc090ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc04102390 0x7fbc04109b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc090ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0410d430 con 0x7fbc041019d0 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc090ff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0410d2b0 con 0x7fbc04102390 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc090ff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0410d5b0 con 0x7fbc041007e0 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc02d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc041007e0 0x7fbc0410afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc0259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc041019d0 0x7fbc04109620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc02d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc041007e0 0x7fbc0410afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57444/0 (socket says 172.21.15.105:57444) 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc02d9d640 1 -- 172.21.15.105:0/1242557778 learned_addr learned my addr 172.21.15.105:0/1242557778 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc0259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc041019d0 0x7fbc04109620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47764/0 (socket says 172.21.15.105:47764) 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc0359e640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc04102390 0x7fbc04109b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc02d9d640 1 -- 172.21.15.105:0/1242557778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc04102390 msgr2=0x7fbc04109b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc02d9d640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc04102390 0x7fbc04109b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc02d9d640 1 -- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc041019d0 msgr2=0x7fbc04109620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc02d9d640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc041019d0 0x7fbc04109620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc02d9d640 1 -- 172.21.15.105:0/1242557778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0410a3f0 con 0x7fbc041007e0 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc0359e640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc04102390 0x7fbc04109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.786+0000 7fbc0259c640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc041019d0 0x7fbc04109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.787+0000 7fbc02d9d640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc041007e0 0x7fbc0410afe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e9f0 tx=0x7fbbf400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.787+0000 7fbbebfff640 1 -- 172.21.15.105:0/1242557778 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400ccc0 con 0x7fbc041007e0 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.787+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc041a1c00 con 0x7fbc041007e0 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.787+0000 7fbbebfff640 1 -- 172.21.15.105:0/1242557778 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbf40035f0 con 0x7fbc041007e0 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.787+0000 7fbbebfff640 1 -- 172.21.15.105:0/1242557778 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf400ce60 con 0x7fbc041007e0 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.787+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc041a2150 con 0x7fbc041007e0 2024-03-26T17:48:18.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.788+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc04107b50 con 0x7fbc041007e0 2024-03-26T17:48:18.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.789+0000 7fbbebfff640 1 -- 172.21.15.105:0/1242557778 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbbf40037a0 con 0x7fbc041007e0 2024-03-26T17:48:18.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.790+0000 7fbbebfff640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbec0a2f20 0x7fbbec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:18.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.790+0000 7fbc0259c640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbec0a2f20 0x7fbbec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:18.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.790+0000 7fbbebfff640 1 -- 172.21.15.105:0/1242557778 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fbbf40cf730 con 0x7fbc041007e0 2024-03-26T17:48:18.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.790+0000 7fbc0259c640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbec0a2f20 0x7fbbec0a53e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80095f0 tx=0x7fbbf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:18.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:18.793+0000 7fbbebfff640 1 -- 172.21.15.105:0/1242557778 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbbf409c520 con 0x7fbc041007e0 2024-03-26T17:48:19.030 INFO:tasks.workunit.client.0.smithi105.stdout: 1 318253 3.15 MB/sec execute 422 sec latency 57.344 ms 2024-03-26T17:48:19.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.081+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc0402d240 con 0x7fbc041007e0 2024-03-26T17:48:19.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.082+0000 7fbbebfff640 1 -- 172.21.15.105:0/1242557778 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7fbbf409be20 con 0x7fbc041007e0 2024-03-26T17:48:19.073 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:48:19.073 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:48:19.074 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:48:19.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbec0a2f20 msgr2=0x7fbbec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbec0a2f20 0x7fbbec0a53e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80095f0 tx=0x7fbbf8005f90 comp rx=0 tx=0).stop 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc041007e0 msgr2=0x7fbc0410afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc041007e0 0x7fbc0410afe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400e9f0 tx=0x7fbbf400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 shutdown_connections 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbbec0a2f20 0x7fbbec0a53e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc04102390 0x7fbc04109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc041019d0 0x7fbc04109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 --2- 172.21.15.105:0/1242557778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc041007e0 0x7fbc0410afe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 >> 172.21.15.105:0/1242557778 conn(0x7fbc040fbd50 msgr2=0x7fbc04104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 shutdown_connections 2024-03-26T17:48:19.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.084+0000 7fbc090ff640 1 -- 172.21.15.105:0/1242557778 wait complete. 2024-03-26T17:48:19.087 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 -- 172.21.15.105:0/3963350547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1018f0 msgr2=0x7fc88c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 --2- 172.21.15.105:0/3963350547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1018f0 0x7fc88c101d10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009900 tx=0x7fc87c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 -- 172.21.15.105:0/3963350547 shutdown_connections 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 --2- 172.21.15.105:0/3963350547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88c1034a0 0x7fc88c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 --2- 172.21.15.105:0/3963350547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc88c102ae0 0x7fc88c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 --2- 172.21.15.105:0/3963350547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1018f0 0x7fc88c101d10 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 -- 172.21.15.105:0/3963350547 >> 172.21.15.105:0/3963350547 conn(0x7fc88c0fce60 msgr2=0x7fc88c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 -- 172.21.15.105:0/3963350547 shutdown_connections 2024-03-26T17:48:19.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.225+0000 7fc891b48640 1 -- 172.21.15.105:0/3963350547 wait complete. 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc891b48640 1 Processor -- start 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc891b48640 1 -- start start 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc891b48640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88c1018f0 0x7fc88c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc891b48640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc88c102ae0 0x7fc88c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc891b48640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1034a0 0x7fc88c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc891b48640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10d420 con 0x7fc88c1034a0 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc891b48640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10d2a0 con 0x7fc88c102ae0 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc891b48640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c10d5a0 con 0x7fc88c1018f0 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc88b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88c1018f0 0x7fc88c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1034a0 0x7fc88c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1034a0 0x7fc88c1a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47778/0 (socket says 172.21.15.105:47778) 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc88b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88c1018f0 0x7fc88c199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57458/0 (socket says 172.21.15.105:57458) 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc88bfff640 1 -- 172.21.15.105:0/3929231900 learned_addr learned my addr 172.21.15.105:0/3929231900 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.226+0000 7fc88affd640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc88c102ae0 0x7fc88c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc88b7fe640 1 -- 172.21.15.105:0/3929231900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc88c102ae0 msgr2=0x7fc88c19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc88b7fe640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc88c102ae0 0x7fc88c19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc88b7fe640 1 -- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1034a0 msgr2=0x7fc88c1a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc88b7fe640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1034a0 0x7fc88c1a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc88b7fe640 1 -- 172.21.15.105:0/3929231900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c1a1970 con 0x7fc88c1018f0 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc88b7fe640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88c1018f0 0x7fc88c199da0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc87c036040 tx=0x7fc87c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc888ff9640 1 -- 172.21.15.105:0/3929231900 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c043020 con 0x7fc88c1018f0 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc891b48640 1 -- 172.21.15.105:0/3929231900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c1a1c00 con 0x7fc88c1018f0 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc888ff9640 1 -- 172.21.15.105:0/3929231900 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc87c004350 con 0x7fc88c1018f0 2024-03-26T17:48:19.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc888ff9640 1 -- 172.21.15.105:0/3929231900 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c003ab0 con 0x7fc88c1018f0 2024-03-26T17:48:19.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.227+0000 7fc891b48640 1 -- 172.21.15.105:0/3929231900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c1a2170 con 0x7fc88c1018f0 2024-03-26T17:48:19.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.229+0000 7fc89480c740 1 Processor -- start 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc89480c740 1 -- start start 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc888ff9640 1 -- 172.21.15.105:0/3929231900 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc87c046020 con 0x7fc88c1018f0 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc89480c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d07e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc89480c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d0cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc89480c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d0fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc89480c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ea16c6c970 con 0x55ea16d08420 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc89480c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ea16c14d70 con 0x55ea16d07a30 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc89480c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea16c68920 con 0x55ea16d0d590 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d07e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d0cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d07e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60942/0 (socket says 172.21.15.105:60942) 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 -- 172.21.15.105:0/126471727 learned_addr learned my addr 172.21.15.105:0/126471727 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88affd640 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d0fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 -- 172.21.15.105:0/126471727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 msgr2=0x55ea16d0fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d0fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 -- 172.21.15.105:0/126471727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 msgr2=0x55ea16d0cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d0cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 -- 172.21.15.105:0/126471727 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea169803f0 con 0x55ea16d07a30 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc888ff9640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88b7fe640 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d0cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.230+0000 7fc88bfff640 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d07e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc884011570 tx=0x7fc884011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29493e746a2258d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.231+0000 7fc88b7fe640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.231+0000 7fc888ff9640 1 -- 172.21.15.105:0/3929231900 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc87c0f1500 con 0x7fc88c1018f0 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.231+0000 7fc871ffb640 1 -- 172.21.15.105:0/126471727 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884004b20 con 0x55ea16d07a30 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.231+0000 7fc871ffb640 1 -- 172.21.15.105:0/126471727 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc884004cc0 con 0x55ea16d07a30 2024-03-26T17:48:19.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.231+0000 7fc871ffb640 1 -- 172.21.15.105:0/126471727 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884005420 con 0x55ea16d07a30 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 -- 172.21.15.105:0/126471727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 msgr2=0x55ea16d07e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d07e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc884011570 tx=0x7fc884011a30 comp rx=0 tx=0).stop 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc88b7fe640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0ef330 tx=0x7fc87c0f7060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 -- 172.21.15.105:0/126471727 shutdown_connections 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d0fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d0cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 --2- 172.21.15.105:0/126471727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d07e50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 -- 172.21.15.105:0/126471727 >> 172.21.15.105:0/126471727 conn(0x55ea16d02fa0 msgr2=0x55ea16d033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 -- 172.21.15.105:0/126471727 shutdown_connections 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 -- 172.21.15.105:0/126471727 wait complete. 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 -- start start 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 Processor -- start 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d11e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.232+0000 7fc89480c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d19400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc89480c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ea16d10540 con 0x55ea16d08420 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc89480c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ea16c28310 con 0x55ea16d07a30 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc89480c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea16d106c0 con 0x55ea16d0d590 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc89480c740 10 client.0 _ll_register_callbacks cb 0x55ea16c63c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d11e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d11e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60944/0 (socket says 172.21.15.105:60944) 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 -- 172.21.15.105:0/2733299932 learned_addr learned my addr 172.21.15.105:0/2733299932 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d19400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 -- 172.21.15.105:0/2733299932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 msgr2=0x55ea16d19400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d19400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 -- 172.21.15.105:0/2733299932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 msgr2=0x55ea16d123c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d123c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 -- 172.21.15.105:0/2733299932 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea16a4a7d0 con 0x55ea16d07a30 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc88bfff640 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d11e80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc884004f90 tx=0x7fc884009cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c012d78529d855d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc86affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc86affd640 1 -- 172.21.15.105:0/2733299932 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88402c020 con 0x55ea16d07a30 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc86affd640 1 -- 172.21.15.105:0/2733299932 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc884016040 con 0x55ea16d07a30 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.233+0000 7fc89480c740 1 -- 172.21.15.105:0/2733299932 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc884004cc0 con 0x55ea16d07a30 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc86affd640 1 -- 172.21.15.105:0/2733299932 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc884037050 con 0x55ea16d07a30 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc89480c740 10 client.28817 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc89480c740 1 -- 172.21.15.105:0/2733299932 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea16d1c750 con 0x55ea16d07a30 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc86affd640 1 -- 172.21.15.105:0/2733299932 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc884004cc0 con 0x55ea16d07a30 2024-03-26T17:48:19.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc86affd640 1 -- 172.21.15.105:0/2733299932 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc884028570 con 0x55ea16d07a30 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc89480c740 10 client.28817 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc89480c740 10 client.28817 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc89480c740 20 client.28817 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc89480c740 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ea16d1dc10 0x55ea16d20040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc89480c740 4 client.28817 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc89480c740 1 -- 172.21.15.105:0/2733299932 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea16c79eb0 con 0x55ea16d1dc10 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc88affd640 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ea16d1dc10 0x55ea16d20040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.234+0000 7fc86affd640 10 client.28817 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.235+0000 7fc88affd640 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ea16d1dc10 0x55ea16d20040 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a600375218731b5 server_cookie=2bfc65915eef80fa in_seq=0 out_seq=0 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.235+0000 7fc86affd640 1 -- 172.21.15.105:0/2733299932 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ea16c79eb0 con 0x55ea16d1dc10 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.235+0000 7fc86affd640 10 client.28817 handle_command_reply: tid=0 2024-03-26T17:48:19.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.235+0000 7fc89480c740 1 client.28817 shutdown 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.235+0000 7fc89480c740 1 -- 172.21.15.105:0/2733299932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 msgr2=0x55ea16d11e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.235+0000 7fc89480c740 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d11e80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc884004f90 tx=0x7fc884009cb0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 1 -- 172.21.15.105:0/2733299932 shutdown_connections 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ea16d1dc10 0x55ea16d20040 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d19400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc86affd640 0 client.28817 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 1 --2- 172.21.15.105:0/2733299932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d11e80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 1 -- 172.21.15.105:0/2733299932 >> 172.21.15.105:0/2733299932 conn(0x55ea16d02fa0 msgr2=0x55ea16d021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 1 -- 172.21.15.105:0/2733299932 shutdown_connections 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 1 -- 172.21.15.105:0/2733299932 wait complete. 2024-03-26T17:48:19.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.236+0000 7fc89480c740 20 client.28817 trim_cache size 0 max 16384 2024-03-26T17:48:19.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc89480c740 1 Processor -- start 2024-03-26T17:48:19.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc89480c740 1 -- start start 2024-03-26T17:48:19.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc89480c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc89480c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d19390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc89480c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d01930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc89480c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1aaf30 con 0x55ea16d08420 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc89480c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1aca70 con 0x55ea16d07a30 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc89480c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1ab600 con 0x55ea16d0d590 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d19390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d19390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47822/0 (socket says 172.21.15.105:47822) 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88affd640 1 -- 172.21.15.105:0/2616642659 learned_addr learned my addr 172.21.15.105:0/2616642659 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88bfff640 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d01930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88bfff640 1 -- 172.21.15.105:0/2616642659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 msgr2=0x55ea16d01930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88bfff640 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d01930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88bfff640 1 -- 172.21.15.105:0/2616642659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 msgr2=0x55ea16d19390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88bfff640 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d19390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88bfff640 1 -- 172.21.15.105:0/2616642659 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea16d10420 con 0x55ea16d07a30 2024-03-26T17:48:19.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.251+0000 7fc88affd640 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d19390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc88bfff640 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d121a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc884009560 tx=0x7fc884009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944a9865898e617e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc8717fa640 1 -- 172.21.15.105:0/2616642659 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88402c070 con 0x55ea16d07a30 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc8717fa640 1 -- 172.21.15.105:0/2616642659 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc884011020 con 0x55ea16d07a30 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc8717fa640 1 -- 172.21.15.105:0/2616642659 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884010890 con 0x55ea16d07a30 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 -- 172.21.15.105:0/2616642659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 msgr2=0x55ea16d121a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d121a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc884009560 tx=0x7fc884009230 comp rx=0 tx=0).stop 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 -- 172.21.15.105:0/2616642659 shutdown_connections 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d08420 0x55ea16d19390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 --2- 172.21.15.105:0/2616642659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d07a30 0x55ea16d121a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 -- 172.21.15.105:0/2616642659 >> 172.21.15.105:0/2616642659 conn(0x55ea16d02fa0 msgr2=0x55ea16d11a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 -- 172.21.15.105:0/2616642659 shutdown_connections 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.252+0000 7fc89480c740 1 -- 172.21.15.105:0/2616642659 wait complete. 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 1 -- start start 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 1 Processor -- start 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d07a30 0x55ea16d166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d08420 0x55ea16d16bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d17c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1ac3a0 con 0x55ea16d07a30 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1abcd0 con 0x55ea16d08420 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c1a8f10 con 0x55ea16d0d590 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc89480c740 10 client.0 _ll_register_callbacks cb 0x7fc88400f890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d07a30 0x55ea16d166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc88bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d07a30 0x55ea16d166b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47826/0 (socket says 172.21.15.105:47826) 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc88bfff640 1 -- 172.21.15.105:0/1957974273 learned_addr learned my addr 172.21.15.105:0/1957974273 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.253+0000 7fc88affd640 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d08420 0x55ea16d16bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc88bfff640 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d17c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc88affd640 1 -- 172.21.15.105:0/1957974273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 msgr2=0x55ea16d17c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc88affd640 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d17c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc88affd640 1 -- 172.21.15.105:0/1957974273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d07a30 msgr2=0x55ea16d166b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc88affd640 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d07a30 0x55ea16d166b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc88affd640 1 -- 172.21.15.105:0/1957974273 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea16d181f0 con 0x55ea16d08420 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc88bfff640 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d07a30 0x55ea16d166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc88affd640 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d08420 0x55ea16d16bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ea16d0fcc0 tx=0x7fc880010a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3e912fa264aa46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc86affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc86affd640 1 -- 172.21.15.105:0/1957974273 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880005020 con 0x55ea16d08420 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc86affd640 1 -- 172.21.15.105:0/1957974273 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc8800051c0 con 0x55ea16d08420 2024-03-26T17:48:19.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc89480c740 1 -- 172.21.15.105:0/1957974273 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc884011020 con 0x55ea16d08420 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc86affd640 1 -- 172.21.15.105:0/1957974273 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8800053c0 con 0x55ea16d08420 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc89480c740 10 client.28823 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.254+0000 7fc89480c740 1 -- 172.21.15.105:0/1957974273 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea16d19560 con 0x55ea16d08420 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc86affd640 1 -- 172.21.15.105:0/1957974273 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc884011020 con 0x55ea16d08420 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc86affd640 1 -- 172.21.15.105:0/1957974273 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc880027690 con 0x55ea16d08420 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc89480c740 10 client.28823 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc89480c740 10 client.28823 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc89480c740 20 client.28823 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc89480c740 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ea16d84540 0x55ea16d86960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc89480c740 4 client.28823 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc89480c740 1 -- 172.21.15.105:0/1957974273 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea16c74200 con 0x55ea16d84540 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc88bfff640 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ea16d84540 0x55ea16d86960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc86affd640 10 client.28823 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:48:19.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc88bfff640 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ea16d84540 0x55ea16d86960 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=41b0d9d3fc97bd43 server_cookie=ca2671951b3f73ea in_seq=0 out_seq=0 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc86affd640 1 -- 172.21.15.105:0/1957974273 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea16c74200 con 0x55ea16d84540 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc86affd640 10 client.28823 handle_command_reply: tid=0 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.255+0000 7fc89480c740 1 client.28823 shutdown 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 -- 172.21.15.105:0/1957974273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d08420 msgr2=0x55ea16d16bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d08420 0x55ea16d16bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55ea16d0fcc0 tx=0x7fc880010a60 comp rx=0 tx=0).stop 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 -- 172.21.15.105:0/1957974273 shutdown_connections 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ea16d84540 0x55ea16d86960 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea16d0d590 0x55ea16d17c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea16d08420 0x55ea16d16bf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc86affd640 0 client.28823 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 --2- 172.21.15.105:0/1957974273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea16d07a30 0x55ea16d166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 -- 172.21.15.105:0/1957974273 >> 172.21.15.105:0/1957974273 conn(0x55ea16d02fa0 msgr2=0x55ea16d06200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 -- 172.21.15.105:0/1957974273 shutdown_connections 2024-03-26T17:48:19.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 1 -- 172.21.15.105:0/1957974273 wait complete. 2024-03-26T17:48:19.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc89480c740 20 client.28823 trim_cache size 0 max 16384 2024-03-26T17:48:19.255 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 -- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 msgr2=0x7fc8740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0ef330 tx=0x7fc87c0f7060 comp rx=0 tx=0).stop 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 -- 172.21.15.105:0/3929231900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88c1018f0 msgr2=0x7fc88c199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88c1018f0 0x7fc88c199da0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc87c036040 tx=0x7fc87c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 -- 172.21.15.105:0/3929231900 shutdown_connections 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8740a2f20 0x7fc8740a53e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc88c1034a0 0x7fc88c1a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc88c102ae0 0x7fc88c19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 --2- 172.21.15.105:0/3929231900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc88c1018f0 0x7fc88c199da0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.256+0000 7fc891b48640 1 -- 172.21.15.105:0/3929231900 >> 172.21.15.105:0/3929231900 conn(0x7fc88c0fce60 msgr2=0x7fc88c073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.257+0000 7fc891b48640 1 -- 172.21.15.105:0/3929231900 shutdown_connections 2024-03-26T17:48:19.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.257+0000 7fc891b48640 1 -- 172.21.15.105:0/3929231900 wait complete. 2024-03-26T17:48:19.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:48:19.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 -- 172.21.15.105:0/726106754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd41019b0 msgr2=0x7febd4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 --2- 172.21.15.105:0/726106754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd41019b0 0x7febd4101e30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7febbc009900 tx=0x7febbc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:19.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 -- 172.21.15.105:0/726106754 shutdown_connections 2024-03-26T17:48:19.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 --2- 172.21.15.105:0/726106754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd4102370 0x7febd4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 --2- 172.21.15.105:0/726106754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd41019b0 0x7febd4101e30 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 --2- 172.21.15.105:0/726106754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd41007c0 0x7febd4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 -- 172.21.15.105:0/726106754 >> 172.21.15.105:0/726106754 conn(0x7febd40fbd50 msgr2=0x7febd40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 -- 172.21.15.105:0/726106754 shutdown_connections 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 -- 172.21.15.105:0/726106754 wait complete. 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 Processor -- start 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.399+0000 7febd9377640 1 -- start start 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd9377640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd41007c0 0x7febd4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd9377640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd41019b0 0x7febd4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd9377640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd4102370 0x7febd419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd9377640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febd410c2f0 con 0x7febd4102370 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd9377640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febd410c170 con 0x7febd41019b0 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd9377640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febd410c470 con 0x7febd41007c0 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd4102370 0x7febd419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd41007c0 0x7febd4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febcbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd41019b0 0x7febd4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd41007c0 0x7febd4197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57524/0 (socket says 172.21.15.105:57524) 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febcbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd41019b0 0x7febd4198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60968/0 (socket says 172.21.15.105:60968) 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd2ffd640 1 -- 172.21.15.105:0/7971107 learned_addr learned my addr 172.21.15.105:0/7971107 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febcbfff640 1 -- 172.21.15.105:0/7971107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd41007c0 msgr2=0x7febd4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febcbfff640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd41007c0 0x7febd4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febcbfff640 1 -- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd4102370 msgr2=0x7febd419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febcbfff640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd4102370 0x7febd419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febcbfff640 1 -- 172.21.15.105:0/7971107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd419f740 con 0x7febd41019b0 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febd2ffd640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd41007c0 0x7febd4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.400+0000 7febcbfff640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd41019b0 0x7febd4198060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7febbc000c00 tx=0x7febbc005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.401+0000 7febd0ff9640 1 -- 172.21.15.105:0/7971107 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc043020 con 0x7febd41019b0 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.401+0000 7febd9377640 1 -- 172.21.15.105:0/7971107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd419f9d0 con 0x7febd41019b0 2024-03-26T17:48:19.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.401+0000 7febd0ff9640 1 -- 172.21.15.105:0/7971107 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febbc004350 con 0x7febd41019b0 2024-03-26T17:48:19.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.401+0000 7febd0ff9640 1 -- 172.21.15.105:0/7971107 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc003b50 con 0x7febd41019b0 2024-03-26T17:48:19.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.401+0000 7febd9377640 1 -- 172.21.15.105:0/7971107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd419ff20 con 0x7febd41019b0 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.403+0000 7febd0ff9640 1 -- 172.21.15.105:0/7971107 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7febbc046020 con 0x7febd41019b0 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.403+0000 7febdbf64740 1 Processor -- start 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.403+0000 7febdbf64740 1 -- start start 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febdbf64740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fafe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febdbf64740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb0420 0x5644e7fb4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febdbf64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb5590 0x5644e7fb7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febdbf64740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5644e7f14970 con 0x5644e7fafa30 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febdbf64740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5644e7ebcd70 con 0x5644e7fb0420 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febdbf64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5644e7f10920 con 0x5644e7fb5590 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fafe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fafe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47844/0 (socket says 172.21.15.105:47844) 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 -- 172.21.15.105:0/2203739926 learned_addr learned my addr 172.21.15.105:0/2203739926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd2ffd640 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb0420 0x5644e7fb4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febcbfff640 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb5590 0x5644e7fb7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd0ff9640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febc00a2f20 0x7febc00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 -- 172.21.15.105:0/2203739926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb5590 msgr2=0x5644e7fb7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb5590 0x5644e7fb7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 -- 172.21.15.105:0/2203739926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb0420 msgr2=0x5644e7fb4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb0420 0x5644e7fb4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 -- 172.21.15.105:0/2203739926 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e7c283f0 con 0x5644e7fafa30 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd2ffd640 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb0420 0x5644e7fb4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd0ff9640 1 -- 172.21.15.105:0/7971107 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7febbc0f1540 con 0x7febd41019b0 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.404+0000 7febd37fe640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febc00a2f20 0x7febc00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.405+0000 7febd37fe640 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fafe50 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7febcc011590 tx=0x7febcc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400cbb5c878fc689 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.405+0000 7febc97fa640 1 -- 172.21.15.105:0/2203739926 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc017050 con 0x5644e7fafa30 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.405+0000 7febd37fe640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febc00a2f20 0x7febc00a53e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7febcc022000 tx=0x7febcc023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.405+0000 7febc97fa640 1 -- 172.21.15.105:0/2203739926 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febcc00fdb0 con 0x5644e7fafa30 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.405+0000 7febc97fa640 1 -- 172.21.15.105:0/2203739926 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc01ba60 con 0x5644e7fafa30 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 -- 172.21.15.105:0/2203739926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 msgr2=0x5644e7fafe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fafe50 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7febcc011590 tx=0x7febcc011a50 comp rx=0 tx=0).stop 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 -- 172.21.15.105:0/2203739926 shutdown_connections 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb5590 0x5644e7fb7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb0420 0x5644e7fb4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 --2- 172.21.15.105:0/2203739926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fafe50 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 -- 172.21.15.105:0/2203739926 >> 172.21.15.105:0/2203739926 conn(0x5644e7faafa0 msgr2=0x5644e7fab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 -- 172.21.15.105:0/2203739926 shutdown_connections 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 -- 172.21.15.105:0/2203739926 wait complete. 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 -- start start 2024-03-26T17:48:19.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.406+0000 7febdbf64740 1 Processor -- start 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febdbf64740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febdbf64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febdbf64740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 0x5644e7fc1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febdbf64740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5644e7fb8540 con 0x5644e7fafa30 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febdbf64740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5644e7ed0310 con 0x5644e7fb5590 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febdbf64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5644e7fb86c0 con 0x5644e7fb0420 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febdbf64740 10 client.0 _ll_register_callbacks cb 0x5644e7f0bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47856/0 (socket says 172.21.15.105:47856) 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 -- 172.21.15.105:0/661592561 learned_addr learned my addr 172.21.15.105:0/661592561 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febcbfff640 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd37fe640 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 0x5644e7fc1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 -- 172.21.15.105:0/661592561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 msgr2=0x5644e7fba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 -- 172.21.15.105:0/661592561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 msgr2=0x5644e7fc1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 0x5644e7fc1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 -- 172.21.15.105:0/661592561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e7cf27d0 con 0x5644e7fafa30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febd2ffd640 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb9f90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7febc4012cf0 tx=0x7febc400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92df8828d77685c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febba7fc640 1 -- 172.21.15.105:0/661592561 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc40120d0 con 0x5644e7fafa30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febba7fc640 1 -- 172.21.15.105:0/661592561 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7febc4004020 con 0x5644e7fafa30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.407+0000 7febdbf64740 1 -- 172.21.15.105:0/661592561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febcc00fdb0 con 0x5644e7fafa30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febba7fc640 1 -- 172.21.15.105:0/661592561 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febc4012270 con 0x5644e7fafa30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febdbf64740 10 client.19011 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febdbf64740 1 -- 172.21.15.105:0/661592561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644e7fc4950 con 0x5644e7fafa30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febba7fc640 1 -- 172.21.15.105:0/661592561 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febcc00fdb0 con 0x5644e7fafa30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febba7fc640 1 -- 172.21.15.105:0/661592561 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7febc4003d00 con 0x5644e7fafa30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febdbf64740 10 client.19011 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febdbf64740 10 client.19011 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:48:19.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febdbf64740 20 client.19011 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febdbf64740 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5644e7fc5f90 0x5644e7fc8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febdbf64740 4 client.19011 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:19.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febdbf64740 1 -- 172.21.15.105:0/661592561 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644e7f21eb0 con 0x5644e7fc5f90 2024-03-26T17:48:19.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.408+0000 7febcbfff640 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5644e7fc5f90 0x5644e7fc8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.409+0000 7febba7fc640 10 client.19011 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:48:19.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.409+0000 7febcbfff640 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5644e7fc5f90 0x5644e7fc8430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e7e62e69d253d1e server_cookie=825ca335164d9d95 in_seq=0 out_seq=0 2024-03-26T17:48:19.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.409+0000 7febba7fc640 1 -- 172.21.15.105:0/661592561 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5644e7f21eb0 con 0x5644e7fc5f90 2024-03-26T17:48:19.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.409+0000 7febba7fc640 10 client.19011 handle_command_reply: tid=0 2024-03-26T17:48:19.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.409+0000 7febdbf64740 1 client.19011 shutdown 2024-03-26T17:48:19.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 -- 172.21.15.105:0/661592561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 msgr2=0x5644e7fb9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb9f90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7febc4012cf0 tx=0x7febc400f790 comp rx=0 tx=0).stop 2024-03-26T17:48:19.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 -- 172.21.15.105:0/661592561 shutdown_connections 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5644e7fc5f90 0x5644e7fc8430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 0x5644e7fc1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febba7fc640 0 client.19011 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 --2- 172.21.15.105:0/661592561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb9f90 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 -- 172.21.15.105:0/661592561 >> 172.21.15.105:0/661592561 conn(0x5644e7faafa0 msgr2=0x5644e7faa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 -- 172.21.15.105:0/661592561 shutdown_connections 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 1 -- 172.21.15.105:0/661592561 wait complete. 2024-03-26T17:48:19.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.410+0000 7febdbf64740 20 client.19011 trim_cache size 0 max 16384 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.425+0000 7febdbf64740 1 Processor -- start 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.425+0000 7febdbf64740 1 -- start start 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.425+0000 7febdbf64740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febdbf64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fc14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febdbf64740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 0x5644e7fab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febdbf64740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febd41a8d30 con 0x5644e7fafa30 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febdbf64740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febd41aa870 con 0x5644e7fb5590 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febdbf64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febd41a9400 con 0x5644e7fb0420 2024-03-26T17:48:19.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febcbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fc14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47862/0 (socket says 172.21.15.105:47862) 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 -- 172.21.15.105:0/1369595638 learned_addr learned my addr 172.21.15.105:0/1369595638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd37fe640 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 0x5644e7fab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 -- 172.21.15.105:0/1369595638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 msgr2=0x5644e7fc14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fc14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 -- 172.21.15.105:0/1369595638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 msgr2=0x5644e7fab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 0x5644e7fab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 -- 172.21.15.105:0/1369595638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e7fab8c0 con 0x5644e7fafa30 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febcbfff640 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fc14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febd2ffd640 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb4f50 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7febc401c5c0 tx=0x7febc4005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16600da11ebdac24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febc8ff9640 1 -- 172.21.15.105:0/1369595638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc400fe80 con 0x5644e7fafa30 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febc8ff9640 1 -- 172.21.15.105:0/1369595638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc4022030 con 0x5644e7fafa30 2024-03-26T17:48:19.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.426+0000 7febc8ff9640 1 -- 172.21.15.105:0/1369595638 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc401cc10 con 0x5644e7fafa30 2024-03-26T17:48:19.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.427+0000 7febdbf64740 1 -- 172.21.15.105:0/1369595638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 msgr2=0x5644e7fb4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.427+0000 7febdbf64740 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb4f50 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7febc401c5c0 tx=0x7febc4005990 comp rx=0 tx=0).stop 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.427+0000 7febdbf64740 1 -- 172.21.15.105:0/1369595638 shutdown_connections 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.427+0000 7febdbf64740 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fb5590 0x5644e7fab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.427+0000 7febdbf64740 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fc14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.427+0000 7febdbf64740 1 --2- 172.21.15.105:0/1369595638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fafa30 0x5644e7fb4f50 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.427+0000 7febdbf64740 1 -- 172.21.15.105:0/1369595638 >> 172.21.15.105:0/1369595638 conn(0x5644e7faafa0 msgr2=0x5644e7faea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 -- 172.21.15.105:0/1369595638 shutdown_connections 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 -- 172.21.15.105:0/1369595638 wait complete. 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 -- start start 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 Processor -- start 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fafa30 0x5644e7fbead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fbf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fb5590 0x5644e7fc0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febd41aa1a0 con 0x5644e7fb5590 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febd41a9ad0 con 0x5644e7fafa30 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febd41a6d10 con 0x5644e7fb0420 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febdbf64740 10 client.0 _ll_register_callbacks cb 0x7febc4009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febd37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fb5590 0x5644e7fc0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fafa30 0x5644e7fbead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febd37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fb5590 0x5644e7fc0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47878/0 (socket says 172.21.15.105:47878) 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febd2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fafa30 0x5644e7fbead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32770/0 (socket says 172.21.15.105:32770) 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febd37fe640 1 -- 172.21.15.105:0/1657532832 learned_addr learned my addr 172.21.15.105:0/1657532832 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.428+0000 7febcbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fbf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febd37fe640 1 -- 172.21.15.105:0/1657532832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 msgr2=0x5644e7fbf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febd37fe640 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fbf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febd37fe640 1 -- 172.21.15.105:0/1657532832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fafa30 msgr2=0x5644e7fbead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febd37fe640 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fafa30 0x5644e7fbead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febd37fe640 1 -- 172.21.15.105:0/1657532832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e7fc0610 con 0x5644e7fb5590 2024-03-26T17:48:19.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febd2ffd640 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fafa30 0x5644e7fbead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febcbfff640 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fbf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febd37fe640 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fb5590 0x5644e7fc0020 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7febcc011530 tx=0x7febcc009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e31ecf406e94d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febba7fc640 1 -- 172.21.15.105:0/1657532832 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc00fb10 con 0x5644e7fb5590 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febba7fc640 1 -- 172.21.15.105:0/1657532832 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7febcc00fcb0 con 0x5644e7fb5590 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febdbf64740 1 -- 172.21.15.105:0/1657532832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febc4022030 con 0x5644e7fb5590 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febba7fc640 1 -- 172.21.15.105:0/1657532832 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febcc017020 con 0x5644e7fb5590 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febdbf64740 10 client.19017 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febdbf64740 1 -- 172.21.15.105:0/1657532832 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644e802c370 con 0x5644e7fb5590 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febba7fc640 1 -- 172.21.15.105:0/1657532832 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febc4022030 con 0x5644e7fb5590 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febba7fc640 1 -- 172.21.15.105:0/1657532832 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7febcc039030 con 0x5644e7fb5590 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febdbf64740 10 client.19017 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febdbf64740 10 client.19017 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:48:19.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.429+0000 7febdbf64740 20 client.19017 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5644e7fc5f90 0x5644e802ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 4 client.19017 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 -- 172.21.15.105:0/1657532832 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644e7f1c200 con 0x5644e7fc5f90 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febd2ffd640 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5644e7fc5f90 0x5644e802ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febba7fc640 10 client.19017 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febd2ffd640 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5644e7fc5f90 0x5644e802ec00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=475562e4504a91ac server_cookie=3e145bc8dcdb2614 in_seq=0 out_seq=0 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febba7fc640 1 -- 172.21.15.105:0/1657532832 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644e7f1c200 con 0x5644e7fc5f90 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febba7fc640 10 client.19017 handle_command_reply: tid=0 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 client.19017 shutdown 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 -- 172.21.15.105:0/1657532832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fb5590 msgr2=0x5644e7fc0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fb5590 0x5644e7fc0020 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7febcc011530 tx=0x7febcc009de0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 -- 172.21.15.105:0/1657532832 shutdown_connections 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5644e7fc5f90 0x5644e802ec00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5644e7fb5590 0x5644e7fc0020 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febba7fc640 0 client.19017 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:48:19.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5644e7fb0420 0x5644e7fbf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.430+0000 7febdbf64740 1 --2- 172.21.15.105:0/1657532832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5644e7fafa30 0x5644e7fbead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febdbf64740 1 -- 172.21.15.105:0/1657532832 >> 172.21.15.105:0/1657532832 conn(0x5644e7faafa0 msgr2=0x5644e7faea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febdbf64740 1 -- 172.21.15.105:0/1657532832 shutdown_connections 2024-03-26T17:48:19.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febdbf64740 1 -- 172.21.15.105:0/1657532832 wait complete. 2024-03-26T17:48:19.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febdbf64740 20 client.19017 trim_cache size 0 max 16384 2024-03-26T17:48:19.428 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:19.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 -- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febc00a2f20 msgr2=0x7febc00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febc00a2f20 0x7febc00a53e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7febcc022000 tx=0x7febcc023000 comp rx=0 tx=0).stop 2024-03-26T17:48:19.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 -- 172.21.15.105:0/7971107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd41019b0 msgr2=0x7febd4198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd41019b0 0x7febd4198060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7febbc000c00 tx=0x7febbc005dc0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 -- 172.21.15.105:0/7971107 shutdown_connections 2024-03-26T17:48:19.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febc00a2f20 0x7febc00a53e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd4102370 0x7febd419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febd41019b0 0x7febd4198060 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 --2- 172.21.15.105:0/7971107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febd41007c0 0x7febd4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 -- 172.21.15.105:0/7971107 >> 172.21.15.105:0/7971107 conn(0x7febd40fbd50 msgr2=0x7febd4068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 -- 172.21.15.105:0/7971107 shutdown_connections 2024-03-26T17:48:19.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.431+0000 7febd9377640 1 -- 172.21.15.105:0/7971107 wait complete. 2024-03-26T17:48:19.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:48:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 -- 172.21.15.105:0/300708385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa01075a0 msgr2=0x7f9aa01099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/300708385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa01075a0 0x7f9aa01099f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009ca0 tx=0x7f9a9802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 -- 172.21.15.105:0/300708385 shutdown_connections 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/300708385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa01075a0 0x7f9aa01099f0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/300708385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa0104c70 0x7f9aa0107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/300708385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa0102340 0x7f9aa0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 -- 172.21.15.105:0/300708385 >> 172.21.15.105:0/300708385 conn(0x7f9aa00fbd50 msgr2=0x7f9aa00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 -- 172.21.15.105:0/300708385 shutdown_connections 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 -- 172.21.15.105:0/300708385 wait complete. 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 Processor -- start 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 -- start start 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa0102340 0x7f9aa0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.563+0000 7f9aa54b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa0104c70 0x7f9aa0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9aa54b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa01075a0 0x7f9aa019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9aa54b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa010d3f0 con 0x7f9aa0104c70 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9aa54b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa010d270 con 0x7f9aa01075a0 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9aa54b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa010d570 con 0x7f9aa0102340 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa0104c70 0x7f9aa0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa0104c70 0x7f9aa0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47886/0 (socket says 172.21.15.105:47886) 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 -- 172.21.15.105:0/1242576732 learned_addr learned my addr 172.21.15.105:0/1242576732 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9effd640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa0102340 0x7f9aa0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 -- 172.21.15.105:0/1242576732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa0102340 msgr2=0x7f9aa0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa0102340 0x7f9aa0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 -- 172.21.15.105:0/1242576732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa01075a0 msgr2=0x7f9aa019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa01075a0 0x7f9aa019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa01075a0 0x7f9aa019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 -- 172.21.15.105:0/1242576732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa019f710 con 0x7f9aa0104c70 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9effd640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa0102340 0x7f9aa0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa01075a0 0x7f9aa019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:48:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa0104c70 0x7f9aa0198030 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400e9c0 tx=0x7f9a9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.564+0000 7f9a87fff640 1 -- 172.21.15.105:0/1242576732 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9400cd80 con 0x7f9aa0104c70 2024-03-26T17:48:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.565+0000 7f9a87fff640 1 -- 172.21.15.105:0/1242576732 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a940054d0 con 0x7f9aa0104c70 2024-03-26T17:48:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.565+0000 7f9aa54b2640 1 -- 172.21.15.105:0/1242576732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa019fa00 con 0x7f9aa0104c70 2024-03-26T17:48:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.565+0000 7f9a87fff640 1 -- 172.21.15.105:0/1242576732 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a94010430 con 0x7f9aa0104c70 2024-03-26T17:48:19.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.565+0000 7f9aa54b2640 1 -- 172.21.15.105:0/1242576732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa019ffa0 con 0x7f9aa0104c70 2024-03-26T17:48:19.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9a87fff640 1 -- 172.21.15.105:0/1242576732 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9a94004fb0 con 0x7f9aa0104c70 2024-03-26T17:48:19.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9aa817c740 1 Processor -- start 2024-03-26T17:48:19.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9aa817c740 1 -- start start 2024-03-26T17:48:19.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dbae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dbff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9aa817c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b792d1f970 con 0x55b792dc0590 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9aa817c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b792cc7d70 con 0x55b792dbaa30 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.567+0000 7f9aa817c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b792d1b920 con 0x55b792dbb420 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dbae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dbae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32788/0 (socket says 172.21.15.105:32788) 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9effd640 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dbff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/561800342 learned_addr learned my addr 172.21.15.105:0/561800342 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a87fff640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9a880a2fa0 0x7f9a880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9effd640 1 -- 172.21.15.105:0/561800342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 msgr2=0x55b792dbae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9effd640 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dbae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9a880a2fa0 0x7f9a880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9effd640 1 -- 172.21.15.105:0/561800342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 msgr2=0x55b792dc2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9effd640 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9effd640 1 -- 172.21.15.105:0/561800342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b792a333f0 con 0x55b792dbb420 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a87fff640 1 -- 172.21.15.105:0/1242576732 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9a940cf870 con 0x7f9aa0104c70 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9effd640 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dbff20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009fd0 tx=0x7f9a900118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae76be8bb1d736f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9a880a2fa0 0x7f9a880a5460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009430 tx=0x7f9a9802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a857fa640 1 -- 172.21.15.105:0/561800342 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9001e070 con 0x55b792dbb420 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a857fa640 1 -- 172.21.15.105:0/561800342 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a900035f0 con 0x55b792dbb420 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.568+0000 7f9a857fa640 1 -- 172.21.15.105:0/561800342 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a900044e0 con 0x55b792dbb420 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 -- 172.21.15.105:0/561800342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 msgr2=0x55b792dbff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dbff20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009fd0 tx=0x7f9a900118d0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 -- 172.21.15.105:0/561800342 shutdown_connections 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dbff20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 --2- 172.21.15.105:0/561800342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dbae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 -- 172.21.15.105:0/561800342 >> 172.21.15.105:0/561800342 conn(0x55b792db5fa0 msgr2=0x55b792db63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 -- 172.21.15.105:0/561800342 shutdown_connections 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.569+0000 7f9aa817c740 1 -- 172.21.15.105:0/561800342 wait complete. 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 1 -- start start 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 1 Processor -- start 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dc4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbb420 0x55b792dc54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dc0590 0x55b792dcc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b792dc3540 con 0x55b792dbb420 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b792cdb310 con 0x55b792dbaa30 2024-03-26T17:48:19.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b792dc36c0 con 0x55b792dc0590 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9aa817c740 10 client.0 _ll_register_callbacks cb 0x55b792d16c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dc4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dc0590 0x55b792dcc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dc4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32790/0 (socket says 172.21.15.105:32790) 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9effd640 1 -- 172.21.15.105:0/2345288426 learned_addr learned my addr 172.21.15.105:0/2345288426 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbb420 0x55b792dc54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9effd640 1 -- 172.21.15.105:0/2345288426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dc0590 msgr2=0x55b792dcc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9effd640 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dc0590 0x55b792dcc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9effd640 1 -- 172.21.15.105:0/2345288426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbb420 msgr2=0x55b792dc54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9effd640 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbb420 0x55b792dc54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9effd640 1 -- 172.21.15.105:0/2345288426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b792afd7d0 con 0x55b792dbaa30 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dc0590 0x55b792dcc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.570+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbb420 0x55b792dc54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9a9effd640 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dc4fa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009fa0 tx=0x7f9a90004210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93777254c04210d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9a82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2345288426 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9001e040 con 0x55b792dbaa30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 1 -- 172.21.15.105:0/2345288426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a900035f0 con 0x55b792dbaa30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2345288426 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9a90004400 con 0x55b792dbaa30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2345288426 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a90005040 con 0x55b792dbaa30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 10 client.28850 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 1 -- 172.21.15.105:0/2345288426 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b792dcf8e0 con 0x55b792dbaa30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2345288426 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a900035f0 con 0x55b792dbaa30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2345288426 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9a90021030 con 0x55b792dbaa30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 10 client.28850 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 10 client.28850 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 20 client.28850 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b792dd0cf0 0x55b792dd3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 4 client.28850 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.571+0000 7f9aa817c740 1 -- 172.21.15.105:0/2345288426 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b792d2ceb0 con 0x55b792dd0cf0 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.572+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b792dd0cf0 0x55b792dd3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.572+0000 7f9a82ff5640 10 client.28850 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.572+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b792dd0cf0 0x55b792dd3160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b8347ffe1cc96f0 server_cookie=944edf6f3b2798f in_seq=0 out_seq=0 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2345288426 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b792d2ceb0 con 0x55b792dd0cf0 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9a82ff5640 10 client.28850 handle_command_reply: tid=0 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 client.28850 shutdown 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 -- 172.21.15.105:0/2345288426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 msgr2=0x55b792dc4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dc4fa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009fa0 tx=0x7f9a90004210 comp rx=0 tx=0).stop 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 -- 172.21.15.105:0/2345288426 shutdown_connections 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dc0590 0x55b792dcc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbb420 0x55b792dc54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b792dd0cf0 0x55b792dd3160 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2345288426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbaa30 0x55b792dc4fa0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 -- 172.21.15.105:0/2345288426 >> 172.21.15.105:0/2345288426 conn(0x55b792db5fa0 msgr2=0x55b792db51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9a82ff5640 0 client.28850 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:48:19.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 -- 172.21.15.105:0/2345288426 shutdown_connections 2024-03-26T17:48:19.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 1 -- 172.21.15.105:0/2345288426 wait complete. 2024-03-26T17:48:19.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.573+0000 7f9aa817c740 20 client.28850 trim_cache size 0 max 16384 2024-03-26T17:48:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.589+0000 7f9aa817c740 1 Processor -- start 2024-03-26T17:48:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.589+0000 7f9aa817c740 1 -- start start 2024-03-26T17:48:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.589+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbaa30 0x55b792db4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.589+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dcc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.589+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dc0590 0x55b792db4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.589+0000 7f9aa817c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01a8eb0 con 0x55b792dbaa30 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.589+0000 7f9aa817c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01aa9f0 con 0x55b792dc0590 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.589+0000 7f9aa817c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01a9580 con 0x55b792dbb420 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dc0590 0x55b792db4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dcc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dc0590 0x55b792db4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32792/0 (socket says 172.21.15.105:32792) 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/1987247014 learned_addr learned my addr 172.21.15.105:0/1987247014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9effd640 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbaa30 0x55b792db4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/1987247014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 msgr2=0x55b792dcc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dcc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/1987247014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbaa30 msgr2=0x55b792db4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbaa30 0x55b792db4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/1987247014 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b792db69f0 con 0x55b792dc0590 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9e7fc640 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dcc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9effd640 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbaa30 0x55b792db4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dc0590 0x55b792db4e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a98005e30 tx=0x7f9a98043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b908fae895004bb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a84ff9640 1 -- 172.21.15.105:0/1987247014 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98054070 con 0x55b792dc0590 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a84ff9640 1 -- 172.21.15.105:0/1987247014 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a980044f0 con 0x55b792dc0590 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.590+0000 7f9a84ff9640 1 -- 172.21.15.105:0/1987247014 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98039030 con 0x55b792dc0590 2024-03-26T17:48:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 -- 172.21.15.105:0/1987247014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dc0590 msgr2=0x55b792db4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dc0590 0x55b792db4e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a98005e30 tx=0x7f9a98043000 comp rx=0 tx=0).stop 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 -- 172.21.15.105:0/1987247014 shutdown_connections 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dc0590 0x55b792db4e70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbb420 0x55b792dcc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 --2- 172.21.15.105:0/1987247014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dbaa30 0x55b792db4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 -- 172.21.15.105:0/1987247014 >> 172.21.15.105:0/1987247014 conn(0x55b792db5fa0 msgr2=0x55b792dbc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 -- 172.21.15.105:0/1987247014 shutdown_connections 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 -- 172.21.15.105:0/1987247014 wait complete. 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 -- start start 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.591+0000 7f9aa817c740 1 Processor -- start 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbaa30 0x55b792dcc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbb420 0x55b792dc8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9aa817c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9aa817c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01aa320 con 0x55b792dc0590 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9aa817c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01a9c50 con 0x55b792dbb420 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9aa817c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa01a6e90 con 0x55b792dbaa30 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbaa30 0x55b792dcc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbb420 0x55b792dc8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47932/0 (socket says 172.21.15.105:47932) 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbaa30 0x55b792dcc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57616/0 (socket says 172.21.15.105:57616) 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/2515981397 learned_addr learned my addr 172.21.15.105:0/2515981397 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9aa817c740 10 client.0 _ll_register_callbacks cb 0x7f9a9001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/2515981397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbaa30 msgr2=0x55b792dcc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbaa30 0x55b792dcc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/2515981397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbb420 msgr2=0x55b792dc8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbb420 0x55b792dc8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 -- 172.21.15.105:0/2515981397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b792e35050 con 0x55b792dc0590 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9effd640 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbaa30 0x55b792dcc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a9f7fe640 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc9350 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f9a9802c630 tx=0x7f9a980042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc04d53372a80f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.592+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2515981397 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a98054070 con 0x55b792dc0590 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2515981397 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9a98034b70 con 0x55b792dc0590 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 1 -- 172.21.15.105:0/2515981397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a980044f0 con 0x55b792dc0590 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2515981397 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a98034d70 con 0x55b792dc0590 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 10 client.19035 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 1 -- 172.21.15.105:0/2515981397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b792e3a650 con 0x55b792dc0590 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2515981397 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a980044f0 con 0x55b792dc0590 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2515981397 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9a98059bf0 con 0x55b792dc0590 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 10 client.19035 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 10 client.19035 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:48:19.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 20 client.19035 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b792e3aed0 0x55b792e3d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 4 client.19035 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9aa817c740 1 -- 172.21.15.105:0/2515981397 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b792dbc430 con 0x55b792e3aed0 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.593+0000 7f9a9effd640 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b792e3aed0 0x55b792e3d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.594+0000 7f9a82ff5640 10 client.19035 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.594+0000 7f9a9effd640 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b792e3aed0 0x55b792e3d330 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2dee97785eb0bd5 server_cookie=4a6c7c4506f48de2 in_seq=0 out_seq=0 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.594+0000 7f9a82ff5640 1 -- 172.21.15.105:0/2515981397 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b792dbc430 con 0x55b792e3aed0 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.594+0000 7f9a82ff5640 10 client.19035 handle_command_reply: tid=0 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.594+0000 7f9aa817c740 1 client.19035 shutdown 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 -- 172.21.15.105:0/2515981397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 msgr2=0x55b792dc9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc9350 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f9a9802c630 tx=0x7f9a980042d0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 -- 172.21.15.105:0/2515981397 shutdown_connections 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b792dc0590 0x55b792dc9350 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b792e3aed0 0x55b792e3d330 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b792dbb420 0x55b792dc8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 --2- 172.21.15.105:0/2515981397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b792dbaa30 0x55b792dcc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 -- 172.21.15.105:0/2515981397 >> 172.21.15.105:0/2515981397 conn(0x55b792db5fa0 msgr2=0x55b792db9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9a82ff5640 0 client.19035 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:48:19.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 -- 172.21.15.105:0/2515981397 shutdown_connections 2024-03-26T17:48:19.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 1 -- 172.21.15.105:0/2515981397 wait complete. 2024-03-26T17:48:19.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa817c740 20 client.19035 trim_cache size 0 max 16384 2024-03-26T17:48:19.593 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:19.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa54b2640 1 -- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9a880a2fa0 msgr2=0x7f9a880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9a880a2fa0 0x7f9a880a5460 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009430 tx=0x7f9a9802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa54b2640 1 -- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa0104c70 msgr2=0x7f9aa0198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.595+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa0104c70 0x7f9aa0198030 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9a9400e9c0 tx=0x7f9a9400ee80 comp rx=0 tx=0).stop 2024-03-26T17:48:19.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.596+0000 7f9aa54b2640 1 -- 172.21.15.105:0/1242576732 shutdown_connections 2024-03-26T17:48:19.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.596+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9a880a2fa0 0x7f9a880a5460 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.596+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9aa01075a0 0x7f9aa019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.596+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9aa0104c70 0x7f9aa0198030 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.596+0000 7f9aa54b2640 1 --2- 172.21.15.105:0/1242576732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa0102340 0x7f9aa0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.596+0000 7f9aa54b2640 1 -- 172.21.15.105:0/1242576732 >> 172.21.15.105:0/1242576732 conn(0x7f9aa00fbd50 msgr2=0x7f9aa0102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.596+0000 7f9aa54b2640 1 -- 172.21.15.105:0/1242576732 shutdown_connections 2024-03-26T17:48:19.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.596+0000 7f9aa54b2640 1 -- 172.21.15.105:0/1242576732 wait complete. 2024-03-26T17:48:19.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:48:19.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 -- 172.21.15.105:0/2637909557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc44010a310 msgr2=0x7fc440110bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 --2- 172.21.15.105:0/2637909557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc44010a310 0x7fc440110bc0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fc438009ca0 tx=0x7fc43802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 -- 172.21.15.105:0/2637909557 shutdown_connections 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 --2- 172.21.15.105:0/2637909557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc44010a310 0x7fc440110bc0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 --2- 172.21.15.105:0/2637909557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc440109970 0x7fc440109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 --2- 172.21.15.105:0/2637909557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc440108c60 0x7fc440109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 -- 172.21.15.105:0/2637909557 >> 172.21.15.105:0/2637909557 conn(0x7fc44006e720 msgr2=0x7fc440070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 -- 172.21.15.105:0/2637909557 shutdown_connections 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.732+0000 7fc445bcd640 1 -- 172.21.15.105:0/2637909557 wait complete. 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc445bcd640 1 Processor -- start 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc445bcd640 1 -- start start 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc445bcd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc440108c60 0x7fc4401a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc445bcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc440109970 0x7fc4401a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc445bcd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc44010a310 0x7fc4401a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc445bcd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc440114b30 con 0x7fc440109970 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc445bcd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401149b0 con 0x7fc440108c60 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc445bcd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc440114cb0 con 0x7fc44010a310 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc43effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc440109970 0x7fc4401a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc43f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc440108c60 0x7fc4401a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc43ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc44010a310 0x7fc4401a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc43f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc440108c60 0x7fc4401a24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32820/0 (socket says 172.21.15.105:32820) 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.733+0000 7fc43f7fe640 1 -- 172.21.15.105:0/439612326 learned_addr learned my addr 172.21.15.105:0/439612326 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43ffff640 1 -- 172.21.15.105:0/439612326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc440108c60 msgr2=0x7fc4401a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43ffff640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc440108c60 0x7fc4401a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43ffff640 1 -- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc440109970 msgr2=0x7fc4401a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43ffff640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc440109970 0x7fc4401a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43ffff640 1 -- 172.21.15.105:0/439612326 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4401aa0d0 con 0x7fc44010a310 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43ffff640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc44010a310 0x7fc4401a9a60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc438037040 tx=0x7fc4380328a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43cff9640 1 -- 172.21.15.105:0/439612326 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438041020 con 0x7fc44010a310 2024-03-26T17:48:19.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc445bcd640 1 -- 172.21.15.105:0/439612326 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4401aa360 con 0x7fc44010a310 2024-03-26T17:48:19.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43cff9640 1 -- 172.21.15.105:0/439612326 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc438003e20 con 0x7fc44010a310 2024-03-26T17:48:19.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc43cff9640 1 -- 172.21.15.105:0/439612326 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438039040 con 0x7fc44010a310 2024-03-26T17:48:19.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.734+0000 7fc445bcd640 1 -- 172.21.15.105:0/439612326 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4401aa820 con 0x7fc44010a310 2024-03-26T17:48:19.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc4487ba740 1 Processor -- start 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc43cff9640 1 -- 172.21.15.105:0/439612326 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc438044020 con 0x7fc44010a310 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc4487ba740 1 -- start start 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae6f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae6fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae6fea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc4487ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae65b970 con 0x55e8ae6f6a30 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc4487ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae603d70 con 0x55e8ae6f7420 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc4487ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae657920 con 0x55e8ae6fc590 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc43effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae6f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc43f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae6fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc43effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae6f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47970/0 (socket says 172.21.15.105:47970) 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc43effd640 1 -- 172.21.15.105:0/3759275721 learned_addr learned my addr 172.21.15.105:0/3759275721 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.737+0000 7fc43effd640 1 --2- 172.21.15.105:0/3759275721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae6fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43f7fe640 1 -- 172.21.15.105:0/3759275721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 msgr2=0x55e8ae6fea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/3759275721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae6fea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43f7fe640 1 -- 172.21.15.105:0/3759275721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 msgr2=0x55e8ae6f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/3759275721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae6f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43f7fe640 1 -- 172.21.15.105:0/3759275721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ae36f3f0 con 0x55e8ae6f7420 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43cff9640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4280a2f20 0x7fc4280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/3759275721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae6fbf20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc430009c80 tx=0x7fc430011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90911b400ae60055 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43effd640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4280a2f20 0x7fc4280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc43cff9640 1 -- 172.21.15.105:0/439612326 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc4380f15a0 con 0x7fc44010a310 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc425ffb640 1 -- 172.21.15.105:0/3759275721 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430004250 con 0x55e8ae6f7420 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc425ffb640 1 -- 172.21.15.105:0/3759275721 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc430009e30 con 0x55e8ae6f7420 2024-03-26T17:48:19.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.738+0000 7fc425ffb640 1 -- 172.21.15.105:0/3759275721 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430004cf0 con 0x55e8ae6f7420 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 -- 172.21.15.105:0/3759275721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 msgr2=0x55e8ae6fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc43effd640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4280a2f20 0x7fc4280a53e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55e8ae6f7c80 tx=0x7fc434015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3759275721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae6fbf20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc430009c80 tx=0x7fc430011810 comp rx=0 tx=0).stop 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 -- 172.21.15.105:0/3759275721 shutdown_connections 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3759275721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae6fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3759275721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae6fbf20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3759275721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae6f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 -- 172.21.15.105:0/3759275721 >> 172.21.15.105:0/3759275721 conn(0x55e8ae6f1fa0 msgr2=0x55e8ae6f23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 -- 172.21.15.105:0/3759275721 shutdown_connections 2024-03-26T17:48:19.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 -- 172.21.15.105:0/3759275721 wait complete. 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.739+0000 7fc4487ba740 1 -- start start 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc4487ba740 1 Processor -- start 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae700f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae701460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae7084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc4487ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae6ff540 con 0x55e8ae6f6a30 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc4487ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae617310 con 0x55e8ae6f7420 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc4487ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8ae6ff6c0 con 0x55e8ae6fc590 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae700f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae701460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae700f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47986/0 (socket says 172.21.15.105:47986) 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43f7fe640 1 -- 172.21.15.105:0/2403796291 learned_addr learned my addr 172.21.15.105:0/2403796291 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43ffff640 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae7084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc4487ba740 10 client.0 _ll_register_callbacks cb 0x55e8ae652c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43f7fe640 1 -- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 msgr2=0x55e8ae7084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae7084a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43f7fe640 1 -- 172.21.15.105:0/2403796291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 msgr2=0x55e8ae701460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.740+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae701460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.741+0000 7fc43f7fe640 1 -- 172.21.15.105:0/2403796291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ae4397d0 con 0x55e8ae6f6a30 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.741+0000 7fc43ffff640 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae7084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.741+0000 7fc43effd640 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae701460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.741+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae700f20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc430005700 tx=0x7fc4300038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d143961814fe18 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.741+0000 7fc41effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.741+0000 7fc41effd640 1 -- 172.21.15.105:0/2403796291 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430029070 con 0x55e8ae6f6a30 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.741+0000 7fc4487ba740 1 -- 172.21.15.105:0/2403796291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc430009e30 con 0x55e8ae6f6a30 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.741+0000 7fc41effd640 1 -- 172.21.15.105:0/2403796291 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc430023430 con 0x55e8ae6f6a30 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc41effd640 1 -- 172.21.15.105:0/2403796291 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc430035050 con 0x55e8ae6f6a30 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc4487ba740 10 client.19047 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc4487ba740 1 -- 172.21.15.105:0/2403796291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8ae70b840 con 0x55e8ae6f6a30 2024-03-26T17:48:19.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc41effd640 1 -- 172.21.15.105:0/2403796291 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc430009e30 con 0x55e8ae6f6a30 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc41effd640 1 -- 172.21.15.105:0/2403796291 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc430019430 con 0x55e8ae6f6a30 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc4487ba740 10 client.19047 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc4487ba740 10 client.19047 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc4487ba740 20 client.19047 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc4487ba740 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e8ae70ccb0 0x55e8ae70f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc4487ba740 4 client.19047 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc4487ba740 1 -- 172.21.15.105:0/2403796291 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8ae668eb0 con 0x55e8ae70ccb0 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.742+0000 7fc43effd640 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e8ae70ccb0 0x55e8ae70f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.743+0000 7fc41effd640 10 client.19047 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.743+0000 7fc43effd640 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e8ae70ccb0 0x55e8ae70f170 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee95451bd98714f5 server_cookie=6123b983dc439665 in_seq=0 out_seq=0 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.743+0000 7fc41effd640 1 -- 172.21.15.105:0/2403796291 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e8ae668eb0 con 0x55e8ae70ccb0 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.743+0000 7fc41effd640 10 client.19047 handle_command_reply: tid=0 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 client.19047 shutdown 2024-03-26T17:48:19.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 -- 172.21.15.105:0/2403796291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 msgr2=0x55e8ae700f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae700f20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fc430005700 tx=0x7fc4300038a0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 -- 172.21.15.105:0/2403796291 shutdown_connections 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e8ae70ccb0 0x55e8ae70f170 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6fc590 0x55e8ae7084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae701460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc41effd640 0 client.19047 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 --2- 172.21.15.105:0/2403796291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae700f20 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 -- 172.21.15.105:0/2403796291 >> 172.21.15.105:0/2403796291 conn(0x55e8ae6f1fa0 msgr2=0x55e8ae6f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 -- 172.21.15.105:0/2403796291 shutdown_connections 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 1 -- 172.21.15.105:0/2403796291 wait complete. 2024-03-26T17:48:19.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.744+0000 7fc4487ba740 20 client.19047 trim_cache size 0 max 16384 2024-03-26T17:48:19.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc4487ba740 1 Processor -- start 2024-03-26T17:48:19.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc4487ba740 1 -- start start 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae6f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f6a30 0x55e8ae7083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f7420 0x55e8ae700540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc4487ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401b3780 con 0x55e8ae6f7420 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc4487ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401b52c0 con 0x55e8ae6f6a30 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc4487ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401b3e50 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f7420 0x55e8ae700540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f6a30 0x55e8ae7083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f6a30 0x55e8ae7083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32852/0 (socket says 172.21.15.105:32852) 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43effd640 1 -- 172.21.15.105:0/850892366 learned_addr learned my addr 172.21.15.105:0/850892366 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/850892366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae6f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43effd640 1 -- 172.21.15.105:0/850892366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 msgr2=0x55e8ae6f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43effd640 1 --2- 172.21.15.105:0/850892366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae6f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43effd640 1 -- 172.21.15.105:0/850892366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f7420 msgr2=0x55e8ae700540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.767+0000 7fc43effd640 1 --2- 172.21.15.105:0/850892366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f7420 0x55e8ae700540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.768+0000 7fc43effd640 1 -- 172.21.15.105:0/850892366 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ae6f2d10 con 0x55e8ae6f6a30 2024-03-26T17:48:19.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.768+0000 7fc43effd640 1 --2- 172.21.15.105:0/850892366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f6a30 0x55e8ae7083c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4340095f0 tx=0x7fc4340046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=317778dcef62bcec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.768+0000 7fc4257fa640 1 -- 172.21.15.105:0/850892366 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434003d00 con 0x55e8ae6f6a30 2024-03-26T17:48:19.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.768+0000 7fc4257fa640 1 -- 172.21.15.105:0/850892366 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc43402d3a0 con 0x55e8ae6f6a30 2024-03-26T17:48:19.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.768+0000 7fc4257fa640 1 -- 172.21.15.105:0/850892366 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434005020 con 0x55e8ae6f6a30 2024-03-26T17:48:19.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 -- 172.21.15.105:0/850892366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f6a30 msgr2=0x55e8ae7083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 --2- 172.21.15.105:0/850892366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f6a30 0x55e8ae7083c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4340095f0 tx=0x7fc4340046c0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 -- 172.21.15.105:0/850892366 shutdown_connections 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 --2- 172.21.15.105:0/850892366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f7420 0x55e8ae700540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 --2- 172.21.15.105:0/850892366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f6a30 0x55e8ae7083c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 --2- 172.21.15.105:0/850892366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae6f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 -- 172.21.15.105:0/850892366 >> 172.21.15.105:0/850892366 conn(0x55e8ae6f8200 msgr2=0x55e8ae6f5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 -- 172.21.15.105:0/850892366 shutdown_connections 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 -- 172.21.15.105:0/850892366 wait complete. 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 -- start start 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.769+0000 7fc4487ba740 1 Processor -- start 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae7055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae705ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc4487ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae706aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc4487ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401b4bf0 con 0x55e8ae6f6a30 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc4487ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401b4520 con 0x55e8ae6f7420 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc4487ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4401b1760 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc4487ba740 10 client.0 _ll_register_callbacks cb 0x7fc434009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae705ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae7055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae705ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48000/0 (socket says 172.21.15.105:48000) 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43effd640 1 -- 172.21.15.105:0/3835941872 learned_addr learned my addr 172.21.15.105:0/3835941872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43ffff640 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae706aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43f7fe640 1 -- 172.21.15.105:0/3835941872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 msgr2=0x55e8ae706aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae706aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43f7fe640 1 -- 172.21.15.105:0/3835941872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 msgr2=0x55e8ae705ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae705ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43f7fe640 1 -- 172.21.15.105:0/3835941872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ae707090 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43effd640 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae705ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc43f7fe640 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae7055a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc430012fd0 tx=0x7fc430009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bce1e2563033bb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc41effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc41effd640 1 -- 172.21.15.105:0/3835941872 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430029040 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc4487ba740 1 -- 172.21.15.105:0/3835941872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc43402d3a0 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.770+0000 7fc41effd640 1 -- 172.21.15.105:0/3835941872 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc430019a70 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc41effd640 1 -- 172.21.15.105:0/3835941872 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc430003ea0 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc4487ba740 10 client.28882 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc4487ba740 1 -- 172.21.15.105:0/3835941872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8ae708390 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc41effd640 1 -- 172.21.15.105:0/3835941872 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc43402d3a0 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc41effd640 1 -- 172.21.15.105:0/3835941872 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc43003a470 con 0x55e8ae6f1fa0 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc4487ba740 10 client.28882 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc4487ba740 10 client.28882 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc4487ba740 20 client.28882 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e8ae6fc590 0x55e8ae775370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc4487ba740 4 client.28882 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc4487ba740 1 -- 172.21.15.105:0/3835941872 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8ae663200 con 0x55e8ae6fc590 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.771+0000 7fc43effd640 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e8ae6fc590 0x55e8ae775370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc43effd640 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e8ae6fc590 0x55e8ae775370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bf4831133c13c7b server_cookie=401e79f8a13ab3b3 in_seq=0 out_seq=0 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc41effd640 10 client.28882 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc41effd640 1 -- 172.21.15.105:0/3835941872 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8ae663200 con 0x55e8ae6fc590 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc41effd640 10 client.28882 handle_command_reply: tid=0 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 client.28882 shutdown 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 -- 172.21.15.105:0/3835941872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 msgr2=0x55e8ae7055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae7055a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc430012fd0 tx=0x7fc430009fa0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 -- 172.21.15.105:0/3835941872 shutdown_connections 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e8ae6fc590 0x55e8ae775370 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e8ae6f7420 0x55e8ae706aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e8ae6f6a30 0x55e8ae705ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc41effd640 0 client.28882 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:48:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 --2- 172.21.15.105:0/3835941872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8ae6f1fa0 0x55e8ae7055a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 -- 172.21.15.105:0/3835941872 >> 172.21.15.105:0/3835941872 conn(0x55e8ae6f8200 msgr2=0x55e8ae6f5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 -- 172.21.15.105:0/3835941872 shutdown_connections 2024-03-26T17:48:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 1 -- 172.21.15.105:0/3835941872 wait complete. 2024-03-26T17:48:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.772+0000 7fc4487ba740 20 client.28882 trim_cache size 0 max 16384 2024-03-26T17:48:19.770 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 -- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4280a2f20 msgr2=0x7fc4280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4280a2f20 0x7fc4280a53e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55e8ae6f7c80 tx=0x7fc434015040 comp rx=0 tx=0).stop 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 -- 172.21.15.105:0/439612326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc44010a310 msgr2=0x7fc4401a9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc44010a310 0x7fc4401a9a60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc438037040 tx=0x7fc4380328a0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 -- 172.21.15.105:0/439612326 shutdown_connections 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4280a2f20 0x7fc4280a53e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc44010a310 0x7fc4401a9a60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc440109970 0x7fc4401a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 --2- 172.21.15.105:0/439612326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc440108c60 0x7fc4401a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 -- 172.21.15.105:0/439612326 >> 172.21.15.105:0/439612326 conn(0x7fc44006e720 msgr2=0x7fc440111280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 -- 172.21.15.105:0/439612326 shutdown_connections 2024-03-26T17:48:19.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.773+0000 7fc445bcd640 1 -- 172.21.15.105:0/439612326 wait complete. 2024-03-26T17:48:19.777 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:48:19.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.901+0000 7f718bac6640 1 -- 172.21.15.105:0/455649989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841034a0 msgr2=0x7f7184109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.901+0000 7f718bac6640 1 --2- 172.21.15.105:0/455649989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841034a0 0x7f7184109d30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f717c009ca0 tx=0x7f717c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.901+0000 7f718bac6640 1 -- 172.21.15.105:0/455649989 shutdown_connections 2024-03-26T17:48:19.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.901+0000 7f718bac6640 1 --2- 172.21.15.105:0/455649989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841034a0 0x7f7184109d30 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.901+0000 7f718bac6640 1 --2- 172.21.15.105:0/455649989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7184102ae0 0x7f7184102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.901+0000 7f718bac6640 1 --2- 172.21.15.105:0/455649989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71841018f0 0x7f7184101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.901+0000 7f718bac6640 1 -- 172.21.15.105:0/455649989 >> 172.21.15.105:0/455649989 conn(0x7f71840fce60 msgr2=0x7f71840ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.901+0000 7f718bac6640 1 -- 172.21.15.105:0/455649989 shutdown_connections 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 -- 172.21.15.105:0/455649989 wait complete. 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 Processor -- start 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 -- start start 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841018f0 0x7f7184197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7184102ae0 0x7f7184197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71841034a0 0x7f718419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f718410d400 con 0x7f71841018f0 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f718410d280 con 0x7f7184102ae0 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.902+0000 7f718bac6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f718410d580 con 0x7f71841034a0 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841018f0 0x7f7184197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841018f0 0x7f7184197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48028/0 (socket says 172.21.15.105:48028) 2024-03-26T17:48:19.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 -- 172.21.15.105:0/2582518976 learned_addr learned my addr 172.21.15.105:0/2582518976 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189061640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7184102ae0 0x7f7184197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f718a063640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71841034a0 0x7f718419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 -- 172.21.15.105:0/2582518976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71841034a0 msgr2=0x7f718419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71841034a0 0x7f718419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 -- 172.21.15.105:0/2582518976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7184102ae0 msgr2=0x7f7184197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7184102ae0 0x7f7184197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 -- 172.21.15.105:0/2582518976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718419f6d0 con 0x7f71841018f0 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f7189862640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841018f0 0x7f7184197ab0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f718000e9c0 tx=0x7f718000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f717affd640 1 -- 172.21.15.105:0/2582518976 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718000cd80 con 0x7f71841018f0 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f718bac6640 1 -- 172.21.15.105:0/2582518976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718419f9c0 con 0x7f71841018f0 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f717affd640 1 -- 172.21.15.105:0/2582518976 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71800035f0 con 0x7f71841018f0 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f717affd640 1 -- 172.21.15.105:0/2582518976 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180010690 con 0x7f71841018f0 2024-03-26T17:48:19.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.903+0000 7f718bac6640 1 -- 172.21.15.105:0/2582518976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718419ff10 con 0x7f71841018f0 2024-03-26T17:48:19.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.905+0000 7f717affd640 1 -- 172.21.15.105:0/2582518976 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f71800037a0 con 0x7f71841018f0 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.906+0000 7f717affd640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f71700a2f20 0x7f71700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.906+0000 7f7189061640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f71700a2f20 0x7f71700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.906+0000 7f717affd640 1 -- 172.21.15.105:0/2582518976 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f71800cf7a0 con 0x7f71841018f0 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718e790740 1 Processor -- start 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189061640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f71700a2f20 0x7f71700a53e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7184198fc0 tx=0x7f7174005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718e790740 1 -- start start 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718e790740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x56551956ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718e790740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x56551956ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718e790740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x565519572a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718e790740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5655194cf970 con 0x56551956aa30 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718e790740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565519477d70 con 0x565519570590 2024-03-26T17:48:19.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718e790740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5655194cb920 con 0x56551956b420 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x56551956ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x56551956ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48034/0 (socket says 172.21.15.105:48034) 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189862640 1 -- 172.21.15.105:0/3781314782 learned_addr learned my addr 172.21.15.105:0/3781314782 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718a063640 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x565519572a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189061640 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x56551956ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189862640 1 -- 172.21.15.105:0/3781314782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 msgr2=0x56551956ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189862640 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x56551956ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189862640 1 -- 172.21.15.105:0/3781314782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 msgr2=0x565519572a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189862640 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x565519572a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f7189862640 1 -- 172.21.15.105:0/3781314782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655191e33f0 con 0x56551956aa30 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.907+0000 7f718a063640 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x565519572a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.908+0000 7f7189862640 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x56551956ae50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f71800a4370 tx=0x7f718009ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7884e59ad88dce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.908+0000 7f716ffff640 1 -- 172.21.15.105:0/3781314782 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718009be90 con 0x56551956aa30 2024-03-26T17:48:19.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.908+0000 7f716ffff640 1 -- 172.21.15.105:0/3781314782 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f718009ffe0 con 0x56551956aa30 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.908+0000 7f716ffff640 1 -- 172.21.15.105:0/3781314782 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800a4410 con 0x56551956aa30 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.908+0000 7f718e790740 1 -- 172.21.15.105:0/3781314782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 msgr2=0x56551956ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.908+0000 7f718e790740 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x56551956ae50 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f71800a4370 tx=0x7f718009ba90 comp rx=0 tx=0).stop 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 -- 172.21.15.105:0/3781314782 shutdown_connections 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x565519572a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x56551956ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 --2- 172.21.15.105:0/3781314782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x56551956ae50 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 -- 172.21.15.105:0/3781314782 >> 172.21.15.105:0/3781314782 conn(0x565519565fa0 msgr2=0x5655195663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 -- 172.21.15.105:0/3781314782 shutdown_connections 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 -- 172.21.15.105:0/3781314782 wait complete. 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 -- start start 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 Processor -- start 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551957c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5655195734e0 con 0x565519570590 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56551948b310 con 0x56551956aa30 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.909+0000 7f718e790740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x565519573660 con 0x56551956b420 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f718e790740 10 client.0 _ll_register_callbacks cb 0x5655194c6c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f718a063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551957c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189862640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57712/0 (socket says 172.21.15.105:57712) 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 -- 172.21.15.105:0/1948059437 learned_addr learned my addr 172.21.15.105:0/1948059437 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 -- 172.21.15.105:0/1948059437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 msgr2=0x565519574ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 -- 172.21.15.105:0/1948059437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 msgr2=0x56551957c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551957c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 -- 172.21.15.105:0/1948059437 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655192ad7d0 con 0x56551956b420 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f718a063640 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551957c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f7189061640 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7174003970 tx=0x7f7174004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=714c9087ee191bc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f716d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f716d7fa640 1 -- 172.21.15.105:0/1948059437 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717400e760 con 0x56551956b420 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f718e790740 1 -- 172.21.15.105:0/1948059437 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718009ffe0 con 0x56551956b420 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f716d7fa640 1 -- 172.21.15.105:0/1948059437 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f717400e900 con 0x56551956b420 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.910+0000 7f716d7fa640 1 -- 172.21.15.105:0/1948059437 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f717400eb00 con 0x56551956b420 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f718e790740 10 client.28894 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f718e790740 1 -- 172.21.15.105:0/1948059437 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56551957f880 con 0x56551956b420 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f716d7fa640 1 -- 172.21.15.105:0/1948059437 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f718009ffe0 con 0x56551956b420 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f716d7fa640 1 -- 172.21.15.105:0/1948059437 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7174031cb0 con 0x56551956b420 2024-03-26T17:48:19.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f718e790740 10 client.28894 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f718e790740 10 client.28894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f718e790740 20 client.28894 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f718e790740 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565519580f90 0x5655195833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f718e790740 4 client.28894 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f718e790740 1 -- 172.21.15.105:0/1948059437 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655194dceb0 con 0x565519580f90 2024-03-26T17:48:19.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.911+0000 7f7189862640 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565519580f90 0x5655195833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.912+0000 7f716d7fa640 10 client.28894 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.912+0000 7f7189862640 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565519580f90 0x5655195833d0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7f739ee9073a8 server_cookie=2e21f69136fe2b88 in_seq=0 out_seq=0 2024-03-26T17:48:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.912+0000 7f716d7fa640 1 -- 172.21.15.105:0/1948059437 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5655194dceb0 con 0x565519580f90 2024-03-26T17:48:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f716d7fa640 10 client.28894 handle_command_reply: tid=0 2024-03-26T17:48:19.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 client.28894 shutdown 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 -- 172.21.15.105:0/1948059437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 msgr2=0x565519575410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7174003970 tx=0x7f7174004730 comp rx=0 tx=0).stop 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 -- 172.21.15.105:0/1948059437 shutdown_connections 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x565519580f90 0x5655195833d0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551957c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f716d7fa640 0 client.28894 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 --2- 172.21.15.105:0/1948059437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 -- 172.21.15.105:0/1948059437 >> 172.21.15.105:0/1948059437 conn(0x565519565fa0 msgr2=0x5655195651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 -- 172.21.15.105:0/1948059437 shutdown_connections 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 1 -- 172.21.15.105:0/1948059437 wait complete. 2024-03-26T17:48:19.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.913+0000 7f718e790740 20 client.28894 trim_cache size 0 max 16384 2024-03-26T17:48:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718e790740 1 Processor -- start 2024-03-26T17:48:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718e790740 1 -- start start 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718e790740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718e790740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718e790740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551956c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718e790740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f71841a8de0 con 0x565519570590 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718e790740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f71841aa920 con 0x56551956aa30 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718e790740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71841a94b0 con 0x56551956b420 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718a063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551956c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f7189862640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f7189061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718a063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551956c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48050/0 (socket says 172.21.15.105:48050) 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f7189061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57722/0 (socket says 172.21.15.105:57722) 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.928+0000 7f718a063640 1 -- 172.21.15.105:0/558824161 learned_addr learned my addr 172.21.15.105:0/558824161 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f7189862640 1 -- 172.21.15.105:0/558824161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 msgr2=0x565519575410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f7189862640 1 --2- 172.21.15.105:0/558824161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f7189862640 1 -- 172.21.15.105:0/558824161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 msgr2=0x56551956c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f7189862640 1 --2- 172.21.15.105:0/558824161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551956c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f7189862640 1 -- 172.21.15.105:0/558824161 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565519565dd0 con 0x56551956aa30 2024-03-26T17:48:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f7189862640 1 --2- 172.21.15.105:0/558824161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x56551956bc80 tx=0x7f7180097aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f365b6bd685ff7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f716f7fe640 1 -- 172.21.15.105:0/558824161 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800bc0c0 con 0x56551956aa30 2024-03-26T17:48:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f716f7fe640 1 -- 172.21.15.105:0/558824161 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f718009ffe0 con 0x56551956aa30 2024-03-26T17:48:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.929+0000 7f716f7fe640 1 -- 172.21.15.105:0/558824161 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800a4670 con 0x56551956aa30 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 -- 172.21.15.105:0/558824161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 msgr2=0x565519574ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 --2- 172.21.15.105:0/558824161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x56551956bc80 tx=0x7f7180097aa0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 -- 172.21.15.105:0/558824161 shutdown_connections 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 --2- 172.21.15.105:0/558824161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565519570590 0x56551956c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 --2- 172.21.15.105:0/558824161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x565519575410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 --2- 172.21.15.105:0/558824161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56551956aa30 0x565519574ed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 -- 172.21.15.105:0/558824161 >> 172.21.15.105:0/558824161 conn(0x565519565fa0 msgr2=0x565519566380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 -- 172.21.15.105:0/558824161 shutdown_connections 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 -- 172.21.15.105:0/558824161 wait complete. 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 -- start start 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.930+0000 7f718e790740 1 Processor -- start 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718e790740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x5655195e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718e790740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x5655195e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718e790740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x5655195e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718e790740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f71841aa250 con 0x56551956aa30 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718e790740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f71841a9b80 con 0x565519570590 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718e790740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71841a6dc0 con 0x56551956b420 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718e790740 10 client.0 _ll_register_callbacks cb 0x7f7174004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x5655195e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x5655195e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x5655195e26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57726/0 (socket says 172.21.15.105:57726) 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 -- 172.21.15.105:0/1528070942 learned_addr learned my addr 172.21.15.105:0/1528070942 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189862640 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x5655195e2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48060/0 (socket says 172.21.15.105:48060) 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718a063640 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x5655195e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 -- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 msgr2=0x5655195e9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x5655195e9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 -- 172.21.15.105:0/1528070942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 msgr2=0x5655195e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x5655195e2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 -- 172.21.15.105:0/1528070942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655195e9cf0 con 0x56551956b420 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f718a063640 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x5655195e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f7189061640 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x5655195e26c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7174003eb0 tx=0x7f7174005860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5225dc4b1ee550f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.931+0000 7f716d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f716d7fa640 1 -- 172.21.15.105:0/1528070942 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71740277b0 con 0x56551956b420 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f716d7fa640 1 -- 172.21.15.105:0/1528070942 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7174009040 con 0x56551956b420 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 1 -- 172.21.15.105:0/1528070942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718009ffe0 con 0x56551956b420 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f716d7fa640 1 -- 172.21.15.105:0/1528070942 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7174027950 con 0x56551956b420 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 10 client.28900 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 1 -- 172.21.15.105:0/1528070942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5655195eb090 con 0x56551956b420 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f716d7fa640 1 -- 172.21.15.105:0/1528070942 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f718009ffe0 con 0x56551956b420 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f716d7fa640 1 -- 172.21.15.105:0/1528070942 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7174031440 con 0x56551956b420 2024-03-26T17:48:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 10 client.28900 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 10 client.28900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 20 client.28900 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5655195eba00 0x5655195ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 4 client.28900 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f718e790740 1 -- 172.21.15.105:0/1528070942 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5655194d7200 con 0x5655195eba00 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.932+0000 7f7189862640 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5655195eba00 0x5655195ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f716d7fa640 10 client.28900 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f7189862640 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5655195eba00 0x5655195ede20 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8300eeaf586add81 server_cookie=91adc5858850ca10 in_seq=0 out_seq=0 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f716d7fa640 1 -- 172.21.15.105:0/1528070942 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5655194d7200 con 0x5655195eba00 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f716d7fa640 10 client.28900 handle_command_reply: tid=0 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 client.28900 shutdown 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 -- 172.21.15.105:0/1528070942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 msgr2=0x5655195e26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x5655195e26c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7174003eb0 tx=0x7f7174005860 comp rx=0 tx=0).stop 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 -- 172.21.15.105:0/1528070942 shutdown_connections 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5655195eba00 0x5655195ede20 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565519570590 0x5655195e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56551956b420 0x5655195e26c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f716d7fa640 0 client.28900 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 --2- 172.21.15.105:0/1528070942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56551956aa30 0x5655195e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 -- 172.21.15.105:0/1528070942 >> 172.21.15.105:0/1528070942 conn(0x565519565fa0 msgr2=0x565519569200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 -- 172.21.15.105:0/1528070942 shutdown_connections 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 1 -- 172.21.15.105:0/1528070942 wait complete. 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.933+0000 7f718e790740 20 client.28900 trim_cache size 0 max 16384 2024-03-26T17:48:19.931 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 -- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f71700a2f20 msgr2=0x7f71700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f71700a2f20 0x7f71700a53e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7184198fc0 tx=0x7f7174005f90 comp rx=0 tx=0).stop 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 -- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841018f0 msgr2=0x7f7184197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841018f0 0x7f7184197ab0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f718000e9c0 tx=0x7f718000ee80 comp rx=0 tx=0).stop 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 -- 172.21.15.105:0/2582518976 shutdown_connections 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f71700a2f20 0x7f71700a53e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71841034a0 0x7f718419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7184102ae0 0x7f7184197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 --2- 172.21.15.105:0/2582518976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71841018f0 0x7f7184197ab0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 -- 172.21.15.105:0/2582518976 >> 172.21.15.105:0/2582518976 conn(0x7f71840fce60 msgr2=0x7f7184105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 -- 172.21.15.105:0/2582518976 shutdown_connections 2024-03-26T17:48:19.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:19.934+0000 7f718bac6640 1 -- 172.21.15.105:0/2582518976 wait complete. 2024-03-26T17:48:20.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 319006 3.15 MB/sec execute 423 sec latency 20.758 ms 2024-03-26T17:48:20.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:19 smithi144 ceph-mon[36095]: pgmap v647: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:48:20.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:19 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1242557778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:48:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:19 smithi105 ceph-mon[32280]: pgmap v647: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:48:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:19 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1242557778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:48:20.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:19 smithi135 ceph-mon[35636]: pgmap v647: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:48:20.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:19 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1242557778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:48:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 -- 172.21.15.105:0/1983926188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981007e0 msgr2=0x7f9098100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 --2- 172.21.15.105:0/1983926188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981007e0 0x7f9098100c00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9088009900 tx=0x7f908802bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 -- 172.21.15.105:0/1983926188 shutdown_connections 2024-03-26T17:48:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 --2- 172.21.15.105:0/1983926188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9098102390 0x7f9098108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 --2- 172.21.15.105:0/1983926188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90981019d0 0x7f9098101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 --2- 172.21.15.105:0/1983926188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981007e0 0x7f9098100c00 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 -- 172.21.15.105:0/1983926188 >> 172.21.15.105:0/1983926188 conn(0x7f90980fbd50 msgr2=0x7f90980fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 -- 172.21.15.105:0/1983926188 shutdown_connections 2024-03-26T17:48:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.678+0000 7f909e71b640 1 -- 172.21.15.105:0/1983926188 wait complete. 2024-03-26T17:48:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.679+0000 7f909e71b640 1 Processor -- start 2024-03-26T17:48:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.679+0000 7f909e71b640 1 -- start start 2024-03-26T17:48:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f909e71b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90981007e0 0x7f9098197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f909e71b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981019d0 0x7f9098197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981019d0 0x7f9098197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981019d0 0x7f9098197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48076/0 (socket says 172.21.15.105:48076) 2024-03-26T17:48:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f909e71b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9098102390 0x7f909819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f909e71b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f909810d430 con 0x7f90981019d0 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f909e71b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f909810d2b0 con 0x7f9098102390 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f909810d5b0 con 0x7f90981007e0 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 -- 172.21.15.105:0/2589149757 learned_addr learned my addr 172.21.15.105:0/2589149757 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f9097fff640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90981007e0 0x7f9098197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f909ccb8640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9098102390 0x7f909819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 -- 172.21.15.105:0/2589149757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90981007e0 msgr2=0x7f9098197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90981007e0 0x7f9098197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 -- 172.21.15.105:0/2589149757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9098102390 msgr2=0x7f909819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9098102390 0x7f909819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 -- 172.21.15.105:0/2589149757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909819f6c0 con 0x7f90981019d0 2024-03-26T17:48:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f909ccb8640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9098102390 0x7f909819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:48:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.680+0000 7f90977fe640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981019d0 0x7f9098197fe0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e9f0 tx=0x7f908c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.681+0000 7f90957fa640 1 -- 172.21.15.105:0/2589149757 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c00cd80 con 0x7f90981019d0 2024-03-26T17:48:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.681+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909819f9b0 con 0x7f90981019d0 2024-03-26T17:48:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.681+0000 7f90957fa640 1 -- 172.21.15.105:0/2589149757 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f908c0108e0 con 0x7f90981019d0 2024-03-26T17:48:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.681+0000 7f90957fa640 1 -- 172.21.15.105:0/2589149757 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c018920 con 0x7f90981019d0 2024-03-26T17:48:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.681+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909819ff50 con 0x7f90981019d0 2024-03-26T17:48:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90957fa640 1 -- 172.21.15.105:0/2589149757 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f908c010a90 con 0x7f90981019d0 2024-03-26T17:48:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90a13e4740 1 Processor -- start 2024-03-26T17:48:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90a13e4740 1 -- start start 2024-03-26T17:48:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7bf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90a13e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5587ed7240b0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90a13e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5587ed6cc960 con 0x5587ed7bffc0 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90a13e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5587ed720060 con 0x5587ed7c5130 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f9097fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f909ccb8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7bf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f9097fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32918/0 (socket says 172.21.15.105:32918) 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f9097fff640 1 -- 172.21.15.105:0/3058702161 learned_addr learned my addr 172.21.15.105:0/3058702161 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90977fe640 1 --2- 172.21.15.105:0/3058702161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f90957fa640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90800a2fc0 0x7f90800a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f9097fff640 1 -- 172.21.15.105:0/3058702161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 msgr2=0x5587ed7c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f9097fff640 1 --2- 172.21.15.105:0/3058702161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f9097fff640 1 -- 172.21.15.105:0/3058702161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 msgr2=0x5587ed7bf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f9097fff640 1 --2- 172.21.15.105:0/3058702161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7bf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f9097fff640 1 -- 172.21.15.105:0/3058702161 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587ed6dec60 con 0x5587ed7bffc0 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.684+0000 7f909ccb8640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90800a2fc0 0x7f90800a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.685+0000 7f90957fa640 1 -- 172.21.15.105:0/2589149757 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f908c0cf7b0 con 0x7f90981019d0 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.685+0000 7f9097fff640 1 --2- 172.21.15.105:0/3058702161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7c4ac0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9088009900 tx=0x7f9088009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287e0b423432b3e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.685+0000 7f907e7fc640 1 -- 172.21.15.105:0/3058702161 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088045020 con 0x5587ed7bffc0 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.685+0000 7f907e7fc640 1 -- 172.21.15.105:0/3058702161 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9088004040 con 0x5587ed7bffc0 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.685+0000 7f907e7fc640 1 -- 172.21.15.105:0/3058702161 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90880046c0 con 0x5587ed7bffc0 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 -- 172.21.15.105:0/3058702161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 msgr2=0x5587ed7c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f909ccb8640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90800a2fc0 0x7f90800a5480 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x5587ed7c0820 tx=0x7f9090004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 --2- 172.21.15.105:0/3058702161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7c4ac0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9088009900 tx=0x7f9088009560 comp rx=0 tx=0).stop 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 -- 172.21.15.105:0/3058702161 shutdown_connections 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 --2- 172.21.15.105:0/3058702161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 --2- 172.21.15.105:0/3058702161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7c4ac0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 --2- 172.21.15.105:0/3058702161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7bf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 -- 172.21.15.105:0/3058702161 >> 172.21.15.105:0/3058702161 conn(0x5587ed7bab40 msgr2=0x5587ed7baf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 -- 172.21.15.105:0/3058702161 shutdown_connections 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.686+0000 7f90a13e4740 1 -- 172.21.15.105:0/3058702161 wait complete. 2024-03-26T17:48:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 1 -- start start 2024-03-26T17:48:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 1 Processor -- start 2024-03-26T17:48:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bffc0 0x5587ed7c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7c5130 0x5587ed7d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5587ed7c8080 con 0x5587ed7bffc0 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5587ed6dfdb0 con 0x5587ed7c5130 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5587ed7c8200 con 0x5587ed7bf5d0 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90a13e4740 10 client.0 _ll_register_callbacks cb 0x5587ed6e4110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bffc0 0x5587ed7c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bffc0 0x5587ed7c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48108/0 (socket says 172.21.15.105:48108) 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f90977fe640 1 -- 172.21.15.105:0/2850607303 learned_addr learned my addr 172.21.15.105:0/2850607303 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f909ccb8640 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7c5130 0x5587ed7d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.687+0000 7f9097fff640 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f9097fff640 1 -- 172.21.15.105:0/2850607303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7c5130 msgr2=0x5587ed7d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f9097fff640 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7c5130 0x5587ed7d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f9097fff640 1 -- 172.21.15.105:0/2850607303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bffc0 msgr2=0x5587ed7c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f9097fff640 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bffc0 0x5587ed7c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f9097fff640 1 -- 172.21.15.105:0/2850607303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ed438210 con 0x5587ed7bf5d0 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f90977fe640 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bffc0 0x5587ed7c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f9097fff640 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c9370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x5587ed7bbab0 tx=0x7f9088004400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be0abafe6cf0be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f90777fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f90777fe640 1 -- 172.21.15.105:0/2850607303 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908802cba0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f90777fe640 1 -- 172.21.15.105:0/2850607303 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f908804d7c0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f90a13e4740 1 -- 172.21.15.105:0/2850607303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9088004040 con 0x5587ed7bf5d0 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f90777fe640 1 -- 172.21.15.105:0/2850607303 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f908804d9c0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f90a13e4740 10 client.28912 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.688+0000 7f90a13e4740 1 -- 172.21.15.105:0/2850607303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587ed504620 con 0x5587ed7bf5d0 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90777fe640 1 -- 172.21.15.105:0/2850607303 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9088004040 con 0x5587ed7bf5d0 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90777fe640 1 -- 172.21.15.105:0/2850607303 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f90880046c0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90a13e4740 10 client.28912 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90a13e4740 10 client.28912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90a13e4740 20 client.28912 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90a13e4740 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5587ed7d4de0 0x5587ed7d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90a13e4740 4 client.28912 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90a13e4740 1 -- 172.21.15.105:0/2850607303 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587ed7315f0 con 0x5587ed7d4de0 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.689+0000 7f90977fe640 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5587ed7d4de0 0x5587ed7d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.690+0000 7f90777fe640 10 client.28912 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.690+0000 7f90977fe640 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5587ed7d4de0 0x5587ed7d71d0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea2f3109d7e121f server_cookie=6dc7104c9cd58 in_seq=0 out_seq=0 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90777fe640 1 -- 172.21.15.105:0/2850607303 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5587ed7315f0 con 0x5587ed7d4de0 2024-03-26T17:48:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90777fe640 10 client.28912 handle_command_reply: tid=0 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90a13e4740 1 client.28912 shutdown 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90a13e4740 1 -- 172.21.15.105:0/2850607303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7bf5d0 msgr2=0x5587ed7c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90a13e4740 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c9370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x5587ed7bbab0 tx=0x7f9088004400 comp rx=0 tx=0).stop 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90a13e4740 1 -- 172.21.15.105:0/2850607303 shutdown_connections 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90a13e4740 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5587ed7d4de0 0x5587ed7d71d0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90a13e4740 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7c5130 0x5587ed7d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90777fe640 0 client.28912 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.691+0000 7f90a13e4740 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bffc0 0x5587ed7c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.692+0000 7f90a13e4740 1 --2- 172.21.15.105:0/2850607303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c9370 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.692+0000 7f90a13e4740 1 -- 172.21.15.105:0/2850607303 >> 172.21.15.105:0/2850607303 conn(0x5587ed7bab40 msgr2=0x5587ed7b9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.692+0000 7f90a13e4740 1 -- 172.21.15.105:0/2850607303 shutdown_connections 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.692+0000 7f90a13e4740 1 -- 172.21.15.105:0/2850607303 wait complete. 2024-03-26T17:48:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.692+0000 7f90a13e4740 20 client.28912 trim_cache size 0 max 16384 2024-03-26T17:48:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90a13e4740 1 Processor -- start 2024-03-26T17:48:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90a13e4740 1 -- start start 2024-03-26T17:48:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90a13e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90981a6a00 con 0x5587ed7bf5d0 2024-03-26T17:48:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90a13e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90981a8540 con 0x5587ed7bffc0 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90a13e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90981a70d0 con 0x5587ed7c5130 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f9097fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f90977fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f9097fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48110/0 (socket says 172.21.15.105:48110) 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f9097fff640 1 -- 172.21.15.105:0/169171216 learned_addr learned my addr 172.21.15.105:0/169171216 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f909ccb8640 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.708+0000 7f9097fff640 1 -- 172.21.15.105:0/169171216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 msgr2=0x5587ed7c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f9097fff640 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f9097fff640 1 -- 172.21.15.105:0/169171216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 msgr2=0x5587ed7be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f9097fff640 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7be4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f9097fff640 1 -- 172.21.15.105:0/169171216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587ed7ba8b0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f909ccb8640 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f9097fff640 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c12c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f9088004ac0 tx=0x7f9088005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e8f2dedda36a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f907dffb640 1 -- 172.21.15.105:0/169171216 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088045070 con 0x5587ed7bf5d0 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f907dffb640 1 -- 172.21.15.105:0/169171216 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9088065020 con 0x5587ed7bf5d0 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.709+0000 7f907dffb640 1 -- 172.21.15.105:0/169171216 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908804ddc0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 -- 172.21.15.105:0/169171216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 msgr2=0x5587ed7c12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c12c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f9088004ac0 tx=0x7f9088005700 comp rx=0 tx=0).stop 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 -- 172.21.15.105:0/169171216 shutdown_connections 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7be4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 --2- 172.21.15.105:0/169171216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7c12c0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 -- 172.21.15.105:0/169171216 >> 172.21.15.105:0/169171216 conn(0x5587ed7bab40 msgr2=0x5587ed7bdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 -- 172.21.15.105:0/169171216 shutdown_connections 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 -- 172.21.15.105:0/169171216 wait complete. 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.710+0000 7f90a13e4740 1 -- start start 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90a13e4740 1 Processor -- start 2024-03-26T17:48:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90a13e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90a13e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90981a7e70 con 0x5587ed7bf5d0 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90a13e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90981a77a0 con 0x5587ed7bffc0 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90a13e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90981a49e0 con 0x5587ed7c5130 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90a13e4740 10 client.0 _ll_register_callbacks cb 0x7f90900099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48112/0 (socket says 172.21.15.105:48112) 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 -- 172.21.15.105:0/893783207 learned_addr learned my addr 172.21.15.105:0/893783207 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90977fe640 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f909ccb8640 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 -- 172.21.15.105:0/893783207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 msgr2=0x5587ed7cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 -- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 msgr2=0x5587ed7cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 -- 172.21.15.105:0/893783207 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587ed7cf3c0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90977fe640 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f9097fff640 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7cd880 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f908802c170 tx=0x7f908802cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13888b6384cc8af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.711+0000 7f90777fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90777fe640 1 -- 172.21.15.105:0/893783207 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088045040 con 0x5587ed7bf5d0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90777fe640 1 -- 172.21.15.105:0/893783207 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9088004400 con 0x5587ed7bf5d0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 1 -- 172.21.15.105:0/893783207 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9088065020 con 0x5587ed7bf5d0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90777fe640 1 -- 172.21.15.105:0/893783207 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9088071050 con 0x5587ed7bf5d0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 10 client.19089 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 1 -- 172.21.15.105:0/893783207 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587ed7d06d0 con 0x5587ed7bf5d0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90777fe640 1 -- 172.21.15.105:0/893783207 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9088065020 con 0x5587ed7bf5d0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90777fe640 1 -- 172.21.15.105:0/893783207 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9088041780 con 0x5587ed7bf5d0 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 10 client.19089 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 10 client.19089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 20 client.19089 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5587ed7d4de0 0x5587ed83d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 4 client.19089 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:48:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.712+0000 7f90a13e4740 1 -- 172.21.15.105:0/893783207 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587ed72b940 con 0x5587ed7d4de0 2024-03-26T17:48:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.713+0000 7f90977fe640 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5587ed7d4de0 0x5587ed83d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.713+0000 7f90777fe640 10 client.19089 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.713+0000 7f90977fe640 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5587ed7d4de0 0x5587ed83d660 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf1b0904b38c3ad server_cookie=c2ce3be35845d0ed in_seq=0 out_seq=0 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.769+0000 7f90777fe640 1 -- 172.21.15.105:0/893783207 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587ed72b940 con 0x5587ed7d4de0 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.769+0000 7f90777fe640 10 client.19089 handle_command_reply: tid=0 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.769+0000 7f90a13e4740 1 client.19089 shutdown 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 -- 172.21.15.105:0/893783207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 msgr2=0x5587ed7cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7cd880 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f908802c170 tx=0x7f908802cc50 comp rx=0 tx=0).stop 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 -- 172.21.15.105:0/893783207 shutdown_connections 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5587ed7d4de0 0x5587ed83d660 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90777fe640 0 client.19089 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587ed7c5130 0x5587ed7cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5587ed7bffc0 0x5587ed7cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 --2- 172.21.15.105:0/893783207 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5587ed7bf5d0 0x5587ed7cd880 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 -- 172.21.15.105:0/893783207 >> 172.21.15.105:0/893783207 conn(0x5587ed7bab40 msgr2=0x5587ed7c10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 -- 172.21.15.105:0/893783207 shutdown_connections 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 1 -- 172.21.15.105:0/893783207 wait complete. 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.770+0000 7f90a13e4740 20 client.19089 trim_cache size 0 max 16384 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:48:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:48:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90800a2fc0 msgr2=0x7f90800a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90800a2fc0 0x7f90800a5480 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x5587ed7c0820 tx=0x7f9090004ab0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981019d0 msgr2=0x7f9098197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981019d0 0x7f9098197fe0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f908c00e9f0 tx=0x7f908c00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 shutdown_connections 2024-03-26T17:48:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90800a2fc0 0x7f90800a5480 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9098102390 0x7f909819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90981019d0 0x7f9098197fe0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 --2- 172.21.15.105:0/2589149757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90981007e0 0x7f9098197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 >> 172.21.15.105:0/2589149757 conn(0x7f90980fbd50 msgr2=0x7f9098104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 shutdown_connections 2024-03-26T17:48:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:20.771+0000 7f909e71b640 1 -- 172.21.15.105:0/2589149757 wait complete. 2024-03-26T17:48:20.937 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:48:20.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:48:20.938 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:48:21.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 319475 3.16 MB/sec execute 424 sec latency 31.114 ms 2024-03-26T17:48:21.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 -- 172.21.15.105:0/4165178760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f964c1007c0 msgr2=0x7f964c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 --2- 172.21.15.105:0/4165178760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f964c1007c0 0x7f964c100be0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f963c009900 tx=0x7f963c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:21.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 -- 172.21.15.105:0/4165178760 shutdown_connections 2024-03-26T17:48:21.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 --2- 172.21.15.105:0/4165178760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f964c102370 0x7f964c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 --2- 172.21.15.105:0/4165178760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964c1019b0 0x7f964c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 --2- 172.21.15.105:0/4165178760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f964c1007c0 0x7f964c100be0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 -- 172.21.15.105:0/4165178760 >> 172.21.15.105:0/4165178760 conn(0x7f964c0fbd50 msgr2=0x7f964c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 -- 172.21.15.105:0/4165178760 shutdown_connections 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 -- 172.21.15.105:0/4165178760 wait complete. 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.072+0000 7f9650c85640 1 Processor -- start 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f9650c85640 1 -- start start 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f9650c85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f964c1007c0 0x7f964c104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f9650c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f964c1019b0 0x7f964c102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f9650c85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964c102370 0x7f964c1032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f9650c85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f964c10c2c0 con 0x7f964c1019b0 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f9650c85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f964c10c140 con 0x7f964c102370 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f9650c85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f964c10c440 con 0x7f964c1007c0 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f964c1019b0 0x7f964c102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f964c1007c0 0x7f964c104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f964c1007c0 0x7f964c104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57792/0 (socket says 172.21.15.105:57792) 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964a59c640 1 -- 172.21.15.105:0/1466501575 learned_addr learned my addr 172.21.15.105:0/1466501575 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964ad9d640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964c102370 0x7f964c1032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964a59c640 1 -- 172.21.15.105:0/1466501575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964c102370 msgr2=0x7f964c1032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964a59c640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964c102370 0x7f964c1032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964a59c640 1 -- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f964c1019b0 msgr2=0x7f964c102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964a59c640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f964c1019b0 0x7f964c102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.073+0000 7f964a59c640 1 -- 172.21.15.105:0/1466501575 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c103b60 con 0x7f964c1007c0 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.074+0000 7f964a59c640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f964c1007c0 0x7f964c104750 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f963c000c00 tx=0x7f963c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.074+0000 7f9643fff640 1 -- 172.21.15.105:0/1466501575 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c043020 con 0x7f964c1007c0 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.074+0000 7f9650c85640 1 -- 172.21.15.105:0/1466501575 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964c1a1c70 con 0x7f964c1007c0 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.074+0000 7f9643fff640 1 -- 172.21.15.105:0/1466501575 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963c004350 con 0x7f964c1007c0 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.074+0000 7f9643fff640 1 -- 172.21.15.105:0/1466501575 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c003b80 con 0x7f964c1007c0 2024-03-26T17:48:21.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.074+0000 7f9650c85640 1 -- 172.21.15.105:0/1466501575 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f964c1a21c0 con 0x7f964c1007c0 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.076+0000 7f9643fff640 1 -- 172.21.15.105:0/1466501575 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f963c046020 con 0x7f964c1007c0 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.077+0000 7f9643fff640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96380a2f20 0x7f96380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.077+0000 7f9641d9b640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96380a2f20 0x7f96380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.077+0000 7f9643fff640 1 -- 172.21.15.105:0/1466501575 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f963c0f1580 con 0x7f964c1007c0 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.077+0000 7f9653875740 1 Processor -- start 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.077+0000 7f9653875740 1 -- start start 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.077+0000 7f9641d9b640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96380a2f20 0x7f96380a53e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f96340095f0 tx=0x7f9634005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9653875740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9653875740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbab80 0x5596d2fbf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9653875740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbfcf0 0x5596d2fc2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9653875740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5596d2f1f630 con 0x5596d2fbfcf0 2024-03-26T17:48:21.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9653875740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596d2ec7e10 con 0x5596d2fba190 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9653875740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596d2f1b5e0 con 0x5596d2fbab80 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbfcf0 0x5596d2fc2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbfcf0 0x5596d2fc2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48152/0 (socket says 172.21.15.105:48152) 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964ad9d640 1 -- 172.21.15.105:0/2030854216 learned_addr learned my addr 172.21.15.105:0/2030854216 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9641d9b640 1 --2- 172.21.15.105:0/2030854216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbab80 0x5596d2fbf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964a59c640 1 -- 172.21.15.105:0/2030854216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbab80 msgr2=0x5596d2fbf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964a59c640 1 --2- 172.21.15.105:0/2030854216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbab80 0x5596d2fbf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964a59c640 1 -- 172.21.15.105:0/2030854216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbfcf0 msgr2=0x5596d2fc2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964a59c640 1 --2- 172.21.15.105:0/2030854216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbfcf0 0x5596d2fc2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964a59c640 1 -- 172.21.15.105:0/2030854216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d2ede740 con 0x5596d2fba190 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f964a59c640 1 --2- 172.21.15.105:0/2030854216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fba5b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f963c006010 tx=0x7f963c0bd7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b971450292bc82 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9640d99640 1 -- 172.21.15.105:0/2030854216 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c0bdc70 con 0x5596d2fba190 2024-03-26T17:48:21.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9640d99640 1 -- 172.21.15.105:0/2030854216 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963c0c1dc0 con 0x5596d2fba190 2024-03-26T17:48:21.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.078+0000 7f9640d99640 1 -- 172.21.15.105:0/2030854216 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c0ba2e0 con 0x5596d2fba190 2024-03-26T17:48:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 -- 172.21.15.105:0/2030854216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 msgr2=0x5596d2fba5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 --2- 172.21.15.105:0/2030854216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fba5b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f963c006010 tx=0x7f963c0bd7b0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 -- 172.21.15.105:0/2030854216 shutdown_connections 2024-03-26T17:48:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 --2- 172.21.15.105:0/2030854216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbfcf0 0x5596d2fc2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 --2- 172.21.15.105:0/2030854216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbab80 0x5596d2fbf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 --2- 172.21.15.105:0/2030854216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fba5b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 -- 172.21.15.105:0/2030854216 >> 172.21.15.105:0/2030854216 conn(0x5596d2fb5700 msgr2=0x5596d2fb5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 -- 172.21.15.105:0/2030854216 shutdown_connections 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.079+0000 7f9653875740 1 -- 172.21.15.105:0/2030854216 wait complete. 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 1 -- start start 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 1 Processor -- start 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fcb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5596d2fc2a60 con 0x5596d2fbab80 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596d2edb1a0 con 0x5596d2fba190 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596d2fc2be0 con 0x5596d2fbfcf0 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9653875740 10 client.0 _ll_register_callbacks cb 0x5596d2c32c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f964a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48160/0 (socket says 172.21.15.105:48160) 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9641d9b640 1 -- 172.21.15.105:0/1933085460 learned_addr learned my addr 172.21.15.105:0/1933085460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f964ad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fcb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9641d9b640 1 -- 172.21.15.105:0/1933085460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 msgr2=0x5596d2fcb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9641d9b640 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fcb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9641d9b640 1 -- 172.21.15.105:0/1933085460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 msgr2=0x5596d2fc43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9641d9b640 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f9641d9b640 1 -- 172.21.15.105:0/1933085460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d2cfb7e0 con 0x5596d2fbab80 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f964a59c640 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.080+0000 7f964ad9d640 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fcb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9641d9b640 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc48e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f963400ed50 tx=0x7f9634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fa4e5892734acb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9631ffb640 1 -- 172.21.15.105:0/1933085460 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963402b060 con 0x5596d2fbab80 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9631ffb640 1 -- 172.21.15.105:0/1933085460 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f963401daa0 con 0x5596d2fbab80 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 1 -- 172.21.15.105:0/1933085460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f963c0c1dc0 con 0x5596d2fbab80 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9631ffb640 1 -- 172.21.15.105:0/1933085460 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f963401dca0 con 0x5596d2fbab80 2024-03-26T17:48:21.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 10 client.19101 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 1 -- 172.21.15.105:0/1933085460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596d2d5bf70 con 0x5596d2fbab80 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9631ffb640 1 -- 172.21.15.105:0/1933085460 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963c0c1dc0 con 0x5596d2fbab80 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9631ffb640 1 -- 172.21.15.105:0/1933085460 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9634026410 con 0x5596d2fbab80 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 10 client.19101 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 10 client.19101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 20 client.19101 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596d2fd01c0 0x5596d2fd2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 4 client.19101 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.081+0000 7f9653875740 1 -- 172.21.15.105:0/1933085460 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596d2f2cb70 con 0x5596d2fd01c0 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.082+0000 7f964a59c640 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596d2fd01c0 0x5596d2fd2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.082+0000 7f9631ffb640 10 client.19101 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.082+0000 7f964a59c640 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596d2fd01c0 0x5596d2fd2640 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b12587ad6963aa1 server_cookie=171ade7f6d9fe8b5 in_seq=0 out_seq=0 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.083+0000 7f9631ffb640 1 -- 172.21.15.105:0/1933085460 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5596d2f2cb70 con 0x5596d2fd01c0 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.083+0000 7f9631ffb640 10 client.19101 handle_command_reply: tid=0 2024-03-26T17:48:21.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.083+0000 7f9653875740 1 client.19101 shutdown 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 -- 172.21.15.105:0/1933085460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 msgr2=0x5596d2fc48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc48e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f963400ed50 tx=0x7f9634004730 comp rx=0 tx=0).stop 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 -- 172.21.15.105:0/1933085460 shutdown_connections 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596d2fd01c0 0x5596d2fd2640 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fcb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc48e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 --2- 172.21.15.105:0/1933085460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9631ffb640 0 client.19101 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 -- 172.21.15.105:0/1933085460 >> 172.21.15.105:0/1933085460 conn(0x5596d2fb5700 msgr2=0x5596d2fb4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 -- 172.21.15.105:0/1933085460 shutdown_connections 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 1 -- 172.21.15.105:0/1933085460 wait complete. 2024-03-26T17:48:21.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.084+0000 7f9653875740 20 client.19101 trim_cache size 0 max 16384 2024-03-26T17:48:21.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.104+0000 7f9653875740 1 Processor -- start 2024-03-26T17:48:21.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.104+0000 7f9653875740 1 -- start start 2024-03-26T17:48:21.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.104+0000 7f9653875740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fba190 0x5596d2fbb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.104+0000 7f9653875740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fcb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.104+0000 7f9653875740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fbfcf0 0x5596d2fbb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.104+0000 7f9653875740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f964c1aafc0 con 0x5596d2fbab80 2024-03-26T17:48:21.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.104+0000 7f9653875740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f964c1acb00 con 0x5596d2fbfcf0 2024-03-26T17:48:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.104+0000 7f9653875740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f964c1ab690 con 0x5596d2fba190 2024-03-26T17:48:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fcb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fcb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48168/0 (socket says 172.21.15.105:48168) 2024-03-26T17:48:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 -- 172.21.15.105:0/596529714 learned_addr learned my addr 172.21.15.105:0/596529714 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f964ad9d640 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fbfcf0 0x5596d2fbb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f964a59c640 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fba190 0x5596d2fbb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 -- 172.21.15.105:0/596529714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fba190 msgr2=0x5596d2fbb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fba190 0x5596d2fbb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 -- 172.21.15.105:0/596529714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fbfcf0 msgr2=0x5596d2fbb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fbfcf0 0x5596d2fbb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 -- 172.21.15.105:0/596529714 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d2bf3170 con 0x5596d2fbab80 2024-03-26T17:48:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f964a59c640 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fba190 0x5596d2fbb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:48:21.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9641d9b640 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fcb8e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f96340118f0 tx=0x7f9634009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8907a66e8666136 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9633fff640 1 -- 172.21.15.105:0/596529714 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963402b040 con 0x5596d2fbab80 2024-03-26T17:48:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9633fff640 1 -- 172.21.15.105:0/596529714 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963401daa0 con 0x5596d2fbab80 2024-03-26T17:48:21.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.105+0000 7f9633fff640 1 -- 172.21.15.105:0/596529714 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96340265c0 con 0x5596d2fbab80 2024-03-26T17:48:21.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 -- 172.21.15.105:0/596529714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 msgr2=0x5596d2fcb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fcb8e0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f96340118f0 tx=0x7f9634009240 comp rx=0 tx=0).stop 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 -- 172.21.15.105:0/596529714 shutdown_connections 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fbfcf0 0x5596d2fbb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fcb8e0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 --2- 172.21.15.105:0/596529714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fba190 0x5596d2fbb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 -- 172.21.15.105:0/596529714 >> 172.21.15.105:0/596529714 conn(0x5596d2fb5700 msgr2=0x5596d2fbc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 -- 172.21.15.105:0/596529714 shutdown_connections 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.106+0000 7f9653875740 1 -- 172.21.15.105:0/596529714 wait complete. 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 1 -- start start 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 1 Processor -- start 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f964c1ac430 con 0x5596d2fbab80 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f964c1abd60 con 0x5596d2fba190 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f964c1a8fa0 con 0x5596d2fbfcf0 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9653875740 10 client.0 _ll_register_callbacks cb 0x7f963400e4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48170/0 (socket says 172.21.15.105:48170) 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f964a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 -- 172.21.15.105:0/1495758725 learned_addr learned my addr 172.21.15.105:0/1495758725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f964ad9d640 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 -- 172.21.15.105:0/1495758725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 msgr2=0x5596d2fca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 -- 172.21.15.105:0/1495758725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 msgr2=0x5596d2fc8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 -- 172.21.15.105:0/1495758725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d2fca740 con 0x5596d2fbab80 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9641d9b640 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc9190 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9634026950 tx=0x7f9634026c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b68d408b0cf5a60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.107+0000 7f9631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9631ffb640 1 -- 172.21.15.105:0/1495758725 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963403d070 con 0x5596d2fbab80 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 1 -- 172.21.15.105:0/1495758725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f963401daa0 con 0x5596d2fbab80 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9631ffb640 1 -- 172.21.15.105:0/1495758725 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9634004230 con 0x5596d2fbab80 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9631ffb640 1 -- 172.21.15.105:0/1495758725 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9634049050 con 0x5596d2fbab80 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 10 client.19107 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 1 -- 172.21.15.105:0/1495758725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596d2fcba80 con 0x5596d2fbab80 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9631ffb640 1 -- 172.21.15.105:0/1495758725 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f963401daa0 con 0x5596d2fbab80 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9631ffb640 1 -- 172.21.15.105:0/1495758725 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9634038030 con 0x5596d2fbab80 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 10 client.19107 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 10 client.19107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 20 client.19107 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:21.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596d3036a50 0x5596d3038e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 4 client.19107 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.108+0000 7f9653875740 1 -- 172.21.15.105:0/1495758725 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5596d2f26ec0 con 0x5596d3036a50 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.109+0000 7f964a59c640 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596d3036a50 0x5596d3038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.109+0000 7f9631ffb640 10 client.19107 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.109+0000 7f964a59c640 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596d3036a50 0x5596d3038e40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e0d51bde0ca1eb server_cookie=4239e8fb46f71415 in_seq=0 out_seq=0 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9631ffb640 1 -- 172.21.15.105:0/1495758725 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5596d2f26ec0 con 0x5596d3036a50 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9631ffb640 10 client.19107 handle_command_reply: tid=0 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 client.19107 shutdown 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 -- 172.21.15.105:0/1495758725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 msgr2=0x5596d2fc9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc9190 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9634026950 tx=0x7f9634026c50 comp rx=0 tx=0).stop 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 -- 172.21.15.105:0/1495758725 shutdown_connections 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596d3036a50 0x5596d3038e40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596d2fbfcf0 0x5596d2fca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596d2fbab80 0x5596d2fc9190 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 --2- 172.21.15.105:0/1495758725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596d2fba190 0x5596d2fc8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9631ffb640 0 client.19107 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.111+0000 7f9653875740 1 -- 172.21.15.105:0/1495758725 >> 172.21.15.105:0/1495758725 conn(0x5596d2fb5700 msgr2=0x5596d2fbb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9653875740 1 -- 172.21.15.105:0/1495758725 shutdown_connections 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9653875740 1 -- 172.21.15.105:0/1495758725 wait complete. 2024-03-26T17:48:21.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9653875740 20 client.19107 trim_cache size 0 max 16384 2024-03-26T17:48:21.111 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:48:21.111 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:48:21.111 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "81da2136-6a09-4612-9a64-f29a092b5bb5", 2024-03-26T17:48:21.111 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:48:21.111 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 -- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96380a2f20 msgr2=0x7f96380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96380a2f20 0x7f96380a53e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f96340095f0 tx=0x7f9634005f90 comp rx=0 tx=0).stop 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 -- 172.21.15.105:0/1466501575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f964c1007c0 msgr2=0x7f964c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f964c1007c0 0x7f964c104750 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f963c000c00 tx=0x7f963c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 -- 172.21.15.105:0/1466501575 shutdown_connections 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96380a2f20 0x7f96380a53e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f964c102370 0x7f964c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f964c1019b0 0x7f964c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 --2- 172.21.15.105:0/1466501575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f964c1007c0 0x7f964c104750 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 -- 172.21.15.105:0/1466501575 >> 172.21.15.105:0/1466501575 conn(0x7f964c0fbd50 msgr2=0x7f964c068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 -- 172.21.15.105:0/1466501575 shutdown_connections 2024-03-26T17:48:21.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.112+0000 7f9650c85640 1 -- 172.21.15.105:0/1466501575 wait complete. 2024-03-26T17:48:21.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:48:21.116 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:48:21.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 -- 172.21.15.105:0/3071442466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc104d10 msgr2=0x7f92dc107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 --2- 172.21.15.105:0/3071442466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc104d10 0x7f92dc107100 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009900 tx=0x7f92cc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:21.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 -- 172.21.15.105:0/3071442466 shutdown_connections 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 --2- 172.21.15.105:0/3071442466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92dc107640 0x7f92dc109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 --2- 172.21.15.105:0/3071442466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc104d10 0x7f92dc107100 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 --2- 172.21.15.105:0/3071442466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92dc1023e0 0x7f92dc1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 -- 172.21.15.105:0/3071442466 >> 172.21.15.105:0/3071442466 conn(0x7f92dc0fbd50 msgr2=0x7f92dc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 -- 172.21.15.105:0/3071442466 shutdown_connections 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.241+0000 7f92e44f8640 1 -- 172.21.15.105:0/3071442466 wait complete. 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e44f8640 1 Processor -- start 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e44f8640 1 -- start start 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e44f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92dc1023e0 0x7f92dc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e44f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92dc104d10 0x7f92dc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e44f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc107640 0x7f92dc19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e44f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc10d3f0 con 0x7f92dc107640 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e44f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc10d270 con 0x7f92dc104d10 2024-03-26T17:48:21.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e44f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc10d570 con 0x7f92dc1023e0 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2a95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc107640 0x7f92dc19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e1a93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92dc104d10 0x7f92dc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2a95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc107640 0x7f92dc19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48184/0 (socket says 172.21.15.105:48184) 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2a95640 1 -- 172.21.15.105:0/76777345 learned_addr learned my addr 172.21.15.105:0/76777345 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2294640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92dc1023e0 0x7f92dc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2a95640 1 -- 172.21.15.105:0/76777345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92dc1023e0 msgr2=0x7f92dc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2a95640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92dc1023e0 0x7f92dc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2a95640 1 -- 172.21.15.105:0/76777345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92dc104d10 msgr2=0x7f92dc1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2a95640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92dc104d10 0x7f92dc1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.242+0000 7f92e2a95640 1 -- 172.21.15.105:0/76777345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc19f7a0 con 0x7f92dc107640 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.243+0000 7f92e2a95640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc107640 0x7f92dc19f130 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f92d400eb10 tx=0x7f92d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.243+0000 7f92d37fe640 1 -- 172.21.15.105:0/76777345 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4004b20 con 0x7f92dc107640 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.243+0000 7f92d37fe640 1 -- 172.21.15.105:0/76777345 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92d4004cc0 con 0x7f92dc107640 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.243+0000 7f92e44f8640 1 -- 172.21.15.105:0/76777345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc100230 con 0x7f92dc107640 2024-03-26T17:48:21.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.243+0000 7f92d37fe640 1 -- 172.21.15.105:0/76777345 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4010690 con 0x7f92dc107640 2024-03-26T17:48:21.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.243+0000 7f92e44f8640 1 -- 172.21.15.105:0/76777345 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1007a0 con 0x7f92dc107640 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.245+0000 7f92d37fe640 1 -- 172.21.15.105:0/76777345 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f92d4005020 con 0x7f92dc107640 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e71c3740 1 Processor -- start 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e71c3740 1 -- start start 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364e98f0 0x55d0364e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364eede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 0x55d0364f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e71c3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d03644e980 con 0x55d0364ea2e0 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e71c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0363f6be0 con 0x55d0364ef450 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e71c3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d03644a930 con 0x55d0364e98f0 2024-03-26T17:48:21.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e2294640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92d37fe640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92c80a2fa0 0x7f92c80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e2294640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364eede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48196/0 (socket says 172.21.15.105:48196) 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e2294640 1 -- 172.21.15.105:0/2271738477 learned_addr learned my addr 172.21.15.105:0/2271738477 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e1a93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364e98f0 0x55d0364e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.246+0000 7f92e1a93640 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 0x55d0364f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e2a95640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92c80a2fa0 0x7f92c80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e1a93640 1 -- 172.21.15.105:0/2271738477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 msgr2=0x55d0364f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e1a93640 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 0x55d0364f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e1a93640 1 -- 172.21.15.105:0/2271738477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 msgr2=0x55d0364eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e1a93640 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92d37fe640 1 -- 172.21.15.105:0/76777345 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f92d40cfa60 con 0x7f92dc107640 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e1a93640 1 -- 172.21.15.105:0/2271738477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03622c7d0 con 0x55d0364e98f0 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e2294640 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e1a93640 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364e98f0 0x55d0364e9d10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f92cc036040 tx=0x7f92cc02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82cd9db907279431 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92e2a95640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92c80a2fa0 0x7f92c80a5460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92d40a4630 tx=0x7f92d406ddd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92d0bfe640 1 -- 172.21.15.105:0/2271738477 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004030 con 0x55d0364e98f0 2024-03-26T17:48:21.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92d0bfe640 1 -- 172.21.15.105:0/2271738477 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc005050 con 0x55d0364e98f0 2024-03-26T17:48:21.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.247+0000 7f92d0bfe640 1 -- 172.21.15.105:0/2271738477 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004900 con 0x55d0364e98f0 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 -- 172.21.15.105:0/2271738477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364e98f0 msgr2=0x55d0364e9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364e98f0 0x55d0364e9d10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f92cc036040 tx=0x7f92cc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 -- 172.21.15.105:0/2271738477 shutdown_connections 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 0x55d0364f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 --2- 172.21.15.105:0/2271738477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364e98f0 0x55d0364e9d10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 -- 172.21.15.105:0/2271738477 >> 172.21.15.105:0/2271738477 conn(0x55d0364e4e60 msgr2=0x55d0364e5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 -- 172.21.15.105:0/2271738477 shutdown_connections 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.248+0000 7f92e71c3740 1 -- 172.21.15.105:0/2271738477 wait complete. 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 1 -- start start 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 1 Processor -- start 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ef450 0x55d0364fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d0364f23a0 con 0x55d0364ef450 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d03640a160 con 0x55d0364e98f0 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0364f2520 con 0x55d0364ea2e0 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e71c3740 10 client.0 _ll_register_callbacks cb 0x55d036446720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e1a93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ef450 0x55d0364fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e2294640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e1a93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ef450 0x55d0364fb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48206/0 (socket says 172.21.15.105:48206) 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e2294640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364f40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:57860/0 (socket says 172.21.15.105:57860) 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e1a93640 1 -- 172.21.15.105:0/596924374 learned_addr learned my addr 172.21.15.105:0/596924374 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e1a93640 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e1a93640 1 -- 172.21.15.105:0/596924374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 msgr2=0x55d0364f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e1a93640 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.249+0000 7f92e1a93640 1 -- 172.21.15.105:0/596924374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 msgr2=0x55d0364f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92e1a93640 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92e1a93640 1 -- 172.21.15.105:0/596924374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0361623f0 con 0x55d0364ef450 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92e2294640 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92e1a93640 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ef450 0x55d0364fb0f0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f92cc02c170 tx=0x7f92cc03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218e0c7563701e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92c5ffb640 1 -- 172.21.15.105:0/596924374 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004900 con 0x55d0364ef450 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92e71c3740 1 -- 172.21.15.105:0/596924374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92cc005050 con 0x55d0364ef450 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92c5ffb640 1 -- 172.21.15.105:0/596924374 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f92cc03c5d0 con 0x55d0364ef450 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92c5ffb640 1 -- 172.21.15.105:0/596924374 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92cc004020 con 0x55d0364ef450 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92e71c3740 10 client.19119 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92e71c3740 1 -- 172.21.15.105:0/596924374 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0361220f0 con 0x55d0364ef450 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.250+0000 7f92c5ffb640 1 -- 172.21.15.105:0/596924374 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc005050 con 0x55d0364ef450 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92c5ffb640 1 -- 172.21.15.105:0/596924374 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f92cc0035f0 con 0x55d0364ef450 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92e71c3740 10 client.19119 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:21.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92e71c3740 10 client.19119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92e71c3740 20 client.19119 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92e71c3740 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0364ff5e0 0x55d0365019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92e71c3740 4 client.19119 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92e71c3740 1 -- 172.21.15.105:0/596924374 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03645bec0 con 0x55d0364ff5e0 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92e2294640 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0364ff5e0 0x55d0365019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92c5ffb640 10 client.19119 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.251+0000 7f92e2294640 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0364ff5e0 0x55d0365019d0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba8d25f827771f8 server_cookie=6a85f169554238dc in_seq=0 out_seq=0 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92c5ffb640 1 -- 172.21.15.105:0/596924374 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d03645bec0 con 0x55d0364ff5e0 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92c5ffb640 10 client.19119 handle_command_reply: tid=0 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 client.19119 shutdown 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 -- 172.21.15.105:0/596924374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ef450 msgr2=0x55d0364fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ef450 0x55d0364fb0f0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f92cc02c170 tx=0x7f92cc03f040 comp rx=0 tx=0).stop 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 -- 172.21.15.105:0/596924374 shutdown_connections 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0364ff5e0 0x55d0365019d0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92c5ffb640 0 client.19119 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ef450 0x55d0364fb0f0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 --2- 172.21.15.105:0/596924374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.252+0000 7f92e71c3740 1 -- 172.21.15.105:0/596924374 >> 172.21.15.105:0/596924374 conn(0x55d0364e4e60 msgr2=0x55d0364e4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.253+0000 7f92e71c3740 1 -- 172.21.15.105:0/596924374 shutdown_connections 2024-03-26T17:48:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.253+0000 7f92e71c3740 1 -- 172.21.15.105:0/596924374 wait complete. 2024-03-26T17:48:21.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.253+0000 7f92e71c3740 20 client.19119 trim_cache size 0 max 16384 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e71c3740 1 Processor -- start 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e71c3740 1 -- start start 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364e98f0 0x55d0364e37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364fb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 0x55d0364e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e71c3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a6c60 con 0x55d0364e98f0 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e71c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a87a0 con 0x55d0364ef450 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e71c3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a7330 con 0x55d0364ea2e0 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e1a93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364e98f0 0x55d0364e37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e1a93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364e98f0 0x55d0364e37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48216/0 (socket says 172.21.15.105:48216) 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e1a93640 1 -- 172.21.15.105:0/3025174351 learned_addr learned my addr 172.21.15.105:0/3025174351 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e2294640 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364fb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.275+0000 7f92e1a93640 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 0x55d0364e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92e1a93640 1 -- 172.21.15.105:0/3025174351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 msgr2=0x55d0364fb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92e1a93640 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364fb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92e1a93640 1 -- 172.21.15.105:0/3025174351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 msgr2=0x55d0364e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92e1a93640 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 0x55d0364e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92e1a93640 1 -- 172.21.15.105:0/3025174351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0364e5bd0 con 0x55d0364e98f0 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92e1a93640 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364e98f0 0x55d0364e37f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f92cc004eb0 tx=0x7f92cc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb441d495cb0d735 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92c7fff640 1 -- 172.21.15.105:0/3025174351 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004f50 con 0x55d0364e98f0 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92c7fff640 1 -- 172.21.15.105:0/3025174351 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc0035f0 con 0x55d0364e98f0 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.276+0000 7f92c7fff640 1 -- 172.21.15.105:0/3025174351 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc005410 con 0x55d0364e98f0 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- 172.21.15.105:0/3025174351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364e98f0 msgr2=0x55d0364e37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364e98f0 0x55d0364e37f0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f92cc004eb0 tx=0x7f92cc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- 172.21.15.105:0/3025174351 shutdown_connections 2024-03-26T17:48:21.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364ef450 0x55d0364e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ea2e0 0x55d0364fb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 --2- 172.21.15.105:0/3025174351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364e98f0 0x55d0364e37f0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- 172.21.15.105:0/3025174351 >> 172.21.15.105:0/3025174351 conn(0x55d0364e4e60 msgr2=0x55d0364eb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- 172.21.15.105:0/3025174351 shutdown_connections 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- 172.21.15.105:0/3025174351 wait complete. 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- start start 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 Processor -- start 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ef450 0x55d0365636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a80d0 con 0x55d0364ea2e0 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a7a00 con 0x55d0364e98f0 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1a4c40 con 0x55d0364ef450 2024-03-26T17:48:21.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e71c3740 10 client.0 _ll_register_callbacks cb 0x7f92d80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e2294640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e1a93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e2294640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48226/0 (socket says 172.21.15.105:48226) 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e2294640 1 -- 172.21.15.105:0/4116059519 learned_addr learned my addr 172.21.15.105:0/4116059519 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.277+0000 7f92e1a93640 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ef450 0x55d0365636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e2294640 1 -- 172.21.15.105:0/4116059519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ef450 msgr2=0x55d0365636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e2294640 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ef450 0x55d0365636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e2294640 1 -- 172.21.15.105:0/4116059519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 msgr2=0x55d0364f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e2294640 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e2294640 1 -- 172.21.15.105:0/4116059519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d036563c20 con 0x55d0364ea2e0 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e1a93640 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e2294640 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364f84a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f92d8004700 tx=0x7f92d8012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc3c24d7b048732 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:21.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92c5ffb640 1 -- 172.21.15.105:0/4116059519 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8011970 con 0x55d0364ea2e0 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92c5ffb640 1 -- 172.21.15.105:0/4116059519 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f92d8011b10 con 0x55d0364ea2e0 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e71c3740 1 -- 172.21.15.105:0/4116059519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92cc0035f0 con 0x55d0364ea2e0 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92c5ffb640 1 -- 172.21.15.105:0/4116059519 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92d8011d10 con 0x55d0364ea2e0 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e71c3740 10 client.19125 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92e71c3740 1 -- 172.21.15.105:0/4116059519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d036564f50 con 0x55d0364ea2e0 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92c5ffb640 1 -- 172.21.15.105:0/4116059519 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92cc0035f0 con 0x55d0364ea2e0 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.278+0000 7f92c5ffb640 1 -- 172.21.15.105:0/4116059519 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f92d8004b70 con 0x55d0364ea2e0 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 10 client.19125 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 10 client.19125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 20 client.19125 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:21.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0364ff5e0 0x55d036567970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 4 client.19125 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 -- 172.21.15.105:0/4116059519 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d036456210 con 0x55d0364ff5e0 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e1a93640 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0364ff5e0 0x55d036567970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92c5ffb640 10 client.19125 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e1a93640 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0364ff5e0 0x55d036567970 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfdbea473e77c548 server_cookie=2e09bb1f4991e821 in_seq=0 out_seq=0 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92c5ffb640 1 -- 172.21.15.105:0/4116059519 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d036456210 con 0x55d0364ff5e0 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92c5ffb640 10 client.19125 handle_command_reply: tid=0 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 client.19125 shutdown 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 -- 172.21.15.105:0/4116059519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 msgr2=0x55d0364f84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364f84a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f92d8004700 tx=0x7f92d8012b20 comp rx=0 tx=0).stop 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 -- 172.21.15.105:0/4116059519 shutdown_connections 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0364ff5e0 0x55d036567970 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0364ef450 0x55d0365636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0364ea2e0 0x55d0364f84a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92c5ffb640 0 client.19125 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 --2- 172.21.15.105:0/4116059519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0364e98f0 0x55d0364f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.279+0000 7f92e71c3740 1 -- 172.21.15.105:0/4116059519 >> 172.21.15.105:0/4116059519 conn(0x55d0364e4e60 msgr2=0x55d0364eb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e71c3740 1 -- 172.21.15.105:0/4116059519 shutdown_connections 2024-03-26T17:48:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e71c3740 1 -- 172.21.15.105:0/4116059519 wait complete. 2024-03-26T17:48:21.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e71c3740 20 client.19125 trim_cache size 0 max 16384 2024-03-26T17:48:21.278 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:48:21.278 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:48:21.278 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:48:21.279 INFO:teuthology.orchestra.run.smithi105.stdout: "81da2136-6a09-4612-9a64-f29a092b5bb5": { 2024-03-26T17:48:21.279 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:48:21.279 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "81da2136-6a09-4612-9a64-f29a092b5bb5", 2024-03-26T17:48:21.279 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:48:21.279 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:48:21.279 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:48:21.279 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 -- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92c80a2fa0 msgr2=0x7f92c80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92c80a2fa0 0x7f92c80a5460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92d40a4630 tx=0x7f92d406ddd0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 -- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc107640 msgr2=0x7f92dc19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc107640 0x7f92dc19f130 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f92d400eb10 tx=0x7f92d400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 -- 172.21.15.105:0/76777345 shutdown_connections 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92c80a2fa0 0x7f92c80a5460 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92dc107640 0x7f92dc19f130 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f92dc104d10 0x7f92dc1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 --2- 172.21.15.105:0/76777345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92dc1023e0 0x7f92dc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 -- 172.21.15.105:0/76777345 >> 172.21.15.105:0/76777345 conn(0x7f92dc0fbd50 msgr2=0x7f92dc102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 -- 172.21.15.105:0/76777345 shutdown_connections 2024-03-26T17:48:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:21.280+0000 7f92e44f8640 1 -- 172.21.15.105:0/76777345 wait complete. 2024-03-26T17:48:21.283 INFO:tasks.cephfs.filesystem:scrub status for tag:81da2136-6a09-4612-9a64-f29a092b5bb5 - {'path': '/', 'tag': '81da2136-6a09-4612-9a64-f29a092b5bb5', 'options': 'recursive,force'} 2024-03-26T17:48:22.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 319836 3.15 MB/sec execute 425 sec latency 15.407 ms 2024-03-26T17:48:22.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:21 smithi144 ceph-mon[36095]: pgmap v648: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:48:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:21 smithi105 ceph-mon[32280]: pgmap v648: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:48:22.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:21 smithi135 ceph-mon[35636]: pgmap v648: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 1.6 MiB/s wr, 112 op/s 2024-03-26T17:48:23.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 320411 3.15 MB/sec execute 426 sec latency 26.148 ms 2024-03-26T17:48:23.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:22 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:48:23.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:22 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:23.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:22 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:48:23.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:22 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:23.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:22 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:48:23.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:22 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:48:23.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:22 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:23.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:22 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:48:23.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:22 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:23.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:22 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:48:23.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:22 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:48:23.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:22 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:23.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:22 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:48:23.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:22 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:23.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:22 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:48:24.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 321058 3.15 MB/sec execute 427 sec latency 17.641 ms 2024-03-26T17:48:24.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:23 smithi144 ceph-mon[36095]: pgmap v649: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:48:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:23 smithi105 ceph-mon[32280]: pgmap v649: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:48:24.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:23 smithi135 ceph-mon[35636]: pgmap v649: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:48:25.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 321683 3.14 MB/sec execute 428 sec latency 18.253 ms 2024-03-26T17:48:26.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 322530 3.15 MB/sec execute 429 sec latency 16.961 ms 2024-03-26T17:48:26.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:25 smithi144 ceph-mon[36095]: pgmap v650: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:48:26.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:25 smithi105 ceph-mon[32280]: pgmap v650: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:48:26.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:25 smithi135 ceph-mon[35636]: pgmap v650: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:48:27.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 323020 3.16 MB/sec execute 430 sec latency 15.356 ms 2024-03-26T17:48:28.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 323407 3.15 MB/sec execute 431 sec latency 17.083 ms 2024-03-26T17:48:28.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:27 smithi144 ceph-mon[36095]: pgmap v651: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:48:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:27 smithi105 ceph-mon[32280]: pgmap v651: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:48:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:27 smithi135 ceph-mon[35636]: pgmap v651: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-03-26T17:48:29.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 323970 3.15 MB/sec execute 432 sec latency 22.556 ms 2024-03-26T17:48:30.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 324656 3.15 MB/sec execute 433 sec latency 16.719 ms 2024-03-26T17:48:30.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:29 smithi144 ceph-mon[36095]: pgmap v652: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:48:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:29 smithi105 ceph-mon[32280]: pgmap v652: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:48:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:29 smithi135 ceph-mon[35636]: pgmap v652: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.5 MiB/s wr, 111 op/s 2024-03-26T17:48:31.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 325230 3.14 MB/sec execute 434 sec latency 22.713 ms 2024-03-26T17:48:32.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 326073 3.15 MB/sec execute 435 sec latency 19.147 ms 2024-03-26T17:48:32.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:31 smithi144 ceph-mon[36095]: pgmap v653: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-03-26T17:48:32.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:31 smithi105 ceph-mon[32280]: pgmap v653: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-03-26T17:48:32.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:31 smithi135 ceph-mon[35636]: pgmap v653: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-03-26T17:48:33.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 326447 3.15 MB/sec execute 436 sec latency 71.923 ms 2024-03-26T17:48:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 -- 172.21.15.105:0/1118869648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 msgr2=0x7fc408109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 --2- 172.21.15.105:0/1118869648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 0x7fc408109d30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc400009ca0 tx=0x7fc40002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 -- 172.21.15.105:0/1118869648 shutdown_connections 2024-03-26T17:48:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 --2- 172.21.15.105:0/1118869648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 0x7fc408109d30 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 --2- 172.21.15.105:0/1118869648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408102ae0 0x7fc408102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 --2- 172.21.15.105:0/1118869648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4081018f0 0x7fc408101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 -- 172.21.15.105:0/1118869648 >> 172.21.15.105:0/1118869648 conn(0x7fc4080fce80 msgr2=0x7fc4080ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 -- 172.21.15.105:0/1118869648 shutdown_connections 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.867+0000 7fc40f685640 1 -- 172.21.15.105:0/1118869648 wait complete. 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40f685640 1 Processor -- start 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40f685640 1 -- start start 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40f685640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4081018f0 0x7fc408197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40f685640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408102ae0 0x7fc408197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40f685640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 0x7fc40819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40f685640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d400 con 0x7fc4081034a0 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40f685640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d280 con 0x7fc4081018f0 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40f685640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc40810d580 con 0x7fc408102ae0 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40dc22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 0x7fc40819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40dc22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 0x7fc40819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38782/0 (socket says 172.21.15.105:38782) 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40dc22640 1 -- 172.21.15.105:0/2561266782 learned_addr learned my addr 172.21.15.105:0/2561266782 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40cc20640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408102ae0 0x7fc408197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.868+0000 7fc40d421640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4081018f0 0x7fc408197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc40dc22640 1 -- 172.21.15.105:0/2561266782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408102ae0 msgr2=0x7fc408197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc40dc22640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408102ae0 0x7fc408197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc40dc22640 1 -- 172.21.15.105:0/2561266782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4081018f0 msgr2=0x7fc408197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc40dc22640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4081018f0 0x7fc408197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc40dc22640 1 -- 172.21.15.105:0/2561266782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40819f630 con 0x7fc4081034a0 2024-03-26T17:48:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc40dc22640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 0x7fc40819efc0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc40002c700 tx=0x7fc4000328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc3fe7fc640 1 -- 172.21.15.105:0/2561266782 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400041020 con 0x7fc4081034a0 2024-03-26T17:48:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc3fe7fc640 1 -- 172.21.15.105:0/2561266782 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc400003e20 con 0x7fc4081034a0 2024-03-26T17:48:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc40f685640 1 -- 172.21.15.105:0/2561266782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40819f8c0 con 0x7fc4081034a0 2024-03-26T17:48:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc3fe7fc640 1 -- 172.21.15.105:0/2561266782 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400039040 con 0x7fc4081034a0 2024-03-26T17:48:33.865 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.869+0000 7fc40f685640 1 -- 172.21.15.105:0/2561266782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40819fdb0 con 0x7fc4081034a0 2024-03-26T17:48:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc412278740 1 Processor -- start 2024-03-26T17:48:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc3fe7fc640 1 -- 172.21.15.105:0/2561266782 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc400044020 con 0x7fc4081034a0 2024-03-26T17:48:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc412278740 1 -- start start 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc412278740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55854f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc412278740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a55854ffc0 0x55a558554ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc412278740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558555130 0x55a5585575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc412278740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a5584b40b0 con 0x55a558555130 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc412278740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a55845c960 con 0x55a55854f5d0 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc412278740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a5584b0060 con 0x55a55854ffc0 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc40cc20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55854f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc40cc20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55854f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43710/0 (socket says 172.21.15.105:43710) 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc40cc20640 1 -- 172.21.15.105:0/536096032 learned_addr learned my addr 172.21.15.105:0/536096032 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc40cc20640 1 --2- 172.21.15.105:0/536096032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558555130 0x55a5585575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.872+0000 7fc3fe7fc640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc3f40a2fc0 0x7fc3f40a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc40cc20640 1 -- 172.21.15.105:0/536096032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a55854ffc0 msgr2=0x55a558554ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc40cc20640 1 --2- 172.21.15.105:0/536096032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a55854ffc0 0x55a558554ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc40cc20640 1 -- 172.21.15.105:0/536096032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558555130 msgr2=0x55a5585575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc40cc20640 1 --2- 172.21.15.105:0/536096032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558555130 0x55a5585575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc40cc20640 1 -- 172.21.15.105:0/536096032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55846ec60 con 0x55a55854f5d0 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc40dc22640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc3f40a2fc0 0x7fc3f40a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc3fe7fc640 1 -- 172.21.15.105:0/2561266782 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc4000f16b0 con 0x7fc4081034a0 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc40cc20640 1 --2- 172.21.15.105:0/536096032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55854f9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80058c0 tx=0x7fc3f8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98f9b04f7d680df0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc3f37fe640 1 -- 172.21.15.105:0/536096032 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8004030 con 0x55a55854f5d0 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc40dc22640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc3f40a2fc0 0x7fc3f40a5480 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc4000c6280 tx=0x7fc400088840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc3f37fe640 1 -- 172.21.15.105:0/536096032 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3f8005020 con 0x55a55854f5d0 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.873+0000 7fc3f37fe640 1 -- 172.21.15.105:0/536096032 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8004b60 con 0x55a55854f5d0 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 -- 172.21.15.105:0/536096032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 msgr2=0x55a55854f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 --2- 172.21.15.105:0/536096032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55854f9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80058c0 tx=0x7fc3f8005850 comp rx=0 tx=0).stop 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 -- 172.21.15.105:0/536096032 shutdown_connections 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 --2- 172.21.15.105:0/536096032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558555130 0x55a5585575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 --2- 172.21.15.105:0/536096032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a55854ffc0 0x55a558554ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 --2- 172.21.15.105:0/536096032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55854f9f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 -- 172.21.15.105:0/536096032 >> 172.21.15.105:0/536096032 conn(0x55a55854ab40 msgr2=0x55a55854af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 -- 172.21.15.105:0/536096032 shutdown_connections 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 -- 172.21.15.105:0/536096032 wait complete. 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 -- start start 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.874+0000 7fc412278740 1 Processor -- start 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc412278740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a55854f5d0 0x55a558559440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc412278740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a55854ffc0 0x55a558559980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc412278740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a558555130 0x55a5585609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc412278740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a5585580e0 con 0x55a55854ffc0 2024-03-26T17:48:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc412278740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a55846fdb0 con 0x55a558555130 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc412278740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a558558260 con 0x55a55854f5d0 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc412278740 10 client.0 _ll_register_callbacks cb 0x55a558474110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a55854ffc0 0x55a558559980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a55854ffc0 0x55a558559980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38796/0 (socket says 172.21.15.105:38796) 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 -- 172.21.15.105:0/496570212 learned_addr learned my addr 172.21.15.105:0/496570212 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 -- 172.21.15.105:0/496570212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a55854f5d0 msgr2=0x55a558559440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a55854f5d0 0x55a558559440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 -- 172.21.15.105:0/496570212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a558555130 msgr2=0x55a5585609c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a558555130 0x55a5585609c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 -- 172.21.15.105:0/496570212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5581c8210 con 0x55a55854ffc0 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc40d421640 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a55854ffc0 0x55a558559980 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc404011430 tx=0x7fc4040118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b66b3c3f9e711b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc3f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/496570212 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404005220 con 0x55a55854ffc0 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/496570212 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc4040053c0 con 0x55a55854ffc0 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.875+0000 7fc412278740 1 -- 172.21.15.105:0/496570212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f8005020 con 0x55a55854ffc0 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/496570212 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4040055c0 con 0x55a55854ffc0 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc412278740 10 client.19137 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc412278740 1 -- 172.21.15.105:0/496570212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a558294620 con 0x55a55854ffc0 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/496570212 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3f8005020 con 0x55a55854ffc0 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/496570212 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc404012c60 con 0x55a55854ffc0 2024-03-26T17:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc412278740 10 client.19137 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc412278740 10 client.19137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc412278740 20 client.19137 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc412278740 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a558565370 0x55a558567810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc412278740 4 client.19137 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc412278740 1 -- 172.21.15.105:0/496570212 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5584c15f0 con 0x55a558565370 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.876+0000 7fc40cc20640 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a558565370 0x55a558567810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.877+0000 7fc40cc20640 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a558565370 0x55a558567810 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=939efb8506abe4d server_cookie=26fe50f868f33ca1 in_seq=0 out_seq=0 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.877+0000 7fc3f0ff9640 10 client.19137 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/496570212 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a5584c15f0 con 0x55a558565370 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc3f0ff9640 10 client.19137 handle_command_reply: tid=0 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 client.19137 shutdown 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 -- 172.21.15.105:0/496570212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a55854ffc0 msgr2=0x55a558559980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a55854ffc0 0x55a558559980 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc404011430 tx=0x7fc4040118f0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 -- 172.21.15.105:0/496570212 shutdown_connections 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a558565370 0x55a558567810 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a558555130 0x55a5585609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a55854ffc0 0x55a558559980 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 --2- 172.21.15.105:0/496570212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a55854f5d0 0x55a558559440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc412278740 1 -- 172.21.15.105:0/496570212 >> 172.21.15.105:0/496570212 conn(0x55a55854ab40 msgr2=0x55a558549df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.878+0000 7fc3f0ff9640 0 client.19137 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.879+0000 7fc412278740 1 -- 172.21.15.105:0/496570212 shutdown_connections 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.879+0000 7fc412278740 1 -- 172.21.15.105:0/496570212 wait complete. 2024-03-26T17:48:33.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.879+0000 7fc412278740 20 client.19137 trim_cache size 0 max 16384 2024-03-26T17:48:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc412278740 1 Processor -- start 2024-03-26T17:48:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc412278740 1 -- start start 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc412278740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a558554430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc412278740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a558560980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc412278740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 0x55a558554970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc412278740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081a8cc0 con 0x55a558559b50 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc412278740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081aa800 con 0x55a55854f5d0 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc412278740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081a9390 con 0x55a558555130 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc40cc20640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 0x55a558554970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc40cc20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a558554430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc40cc20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a558554430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43730/0 (socket says 172.21.15.105:43730) 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc40cc20640 1 -- 172.21.15.105:0/3739797035 learned_addr learned my addr 172.21.15.105:0/3739797035 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.896+0000 7fc40d421640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a558560980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc40cc20640 1 -- 172.21.15.105:0/3739797035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 msgr2=0x55a558560980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc40cc20640 1 --2- 172.21.15.105:0/3739797035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a558560980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc40cc20640 1 -- 172.21.15.105:0/3739797035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 msgr2=0x55a558554970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc40cc20640 1 --2- 172.21.15.105:0/3739797035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 0x55a558554970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc40cc20640 1 -- 172.21.15.105:0/3739797035 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55854b590 con 0x55a55854f5d0 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc40d421640 1 --2- 172.21.15.105:0/3739797035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a558560980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc40cc20640 1 --2- 172.21.15.105:0/3739797035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a558554430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a558550820 tx=0x7fc3f8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea0b044f4a6939a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc3f2ffd640 1 -- 172.21.15.105:0/3739797035 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f80054a0 con 0x55a55854f5d0 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc3f2ffd640 1 -- 172.21.15.105:0/3739797035 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3f8005640 con 0x55a55854f5d0 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.897+0000 7fc3f2ffd640 1 -- 172.21.15.105:0/3739797035 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f800f9f0 con 0x55a55854f5d0 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 -- 172.21.15.105:0/3739797035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 msgr2=0x55a558554430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 --2- 172.21.15.105:0/3739797035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a558554430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a558550820 tx=0x7fc3f8005f90 comp rx=0 tx=0).stop 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 -- 172.21.15.105:0/3739797035 shutdown_connections 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 --2- 172.21.15.105:0/3739797035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 0x55a558554970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 --2- 172.21.15.105:0/3739797035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a558560980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 --2- 172.21.15.105:0/3739797035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a558554430 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 -- 172.21.15.105:0/3739797035 >> 172.21.15.105:0/3739797035 conn(0x55a55854ab40 msgr2=0x55a558559440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 -- 172.21.15.105:0/3739797035 shutdown_connections 2024-03-26T17:48:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 -- 172.21.15.105:0/3739797035 wait complete. 2024-03-26T17:48:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 -- start start 2024-03-26T17:48:33.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.898+0000 7fc412278740 1 Processor -- start 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc412278740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55855dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc412278740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a55855e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc412278740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 0x55a55855f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc412278740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081aa130 con 0x55a558559b50 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc412278740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081a9a60 con 0x55a55854f5d0 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc412278740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081a6ca0 con 0x55a558555130 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc412278740 10 client.0 _ll_register_callbacks cb 0x7fc4040094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55855dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 0x55a55855f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55855dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43742/0 (socket says 172.21.15.105:43742) 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 -- 172.21.15.105:0/2236761033 learned_addr learned my addr 172.21.15.105:0/2236761033 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40d421640 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a55855e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 -- 172.21.15.105:0/2236761033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 msgr2=0x55a55855e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a55855e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 -- 172.21.15.105:0/2236761033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 msgr2=0x55a55855f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 0x55a55855f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 -- 172.21.15.105:0/2236761033 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55855f7c0 con 0x55a55854f5d0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40d421640 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a55855e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc40cc20640 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55855dc80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8029990 tx=0x7fc3f8022040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12705c66d3fa5e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc3f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.899+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/2236761033 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8029b20 con 0x55a55854f5d0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/2236761033 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fc3f8029cc0 con 0x55a55854f5d0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 1 -- 172.21.15.105:0/2236761033 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f8005640 con 0x55a55854f5d0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/2236761033 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc3f800f790 con 0x55a55854f5d0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 10 client.28961 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 1 -- 172.21.15.105:0/2236761033 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a558560ad0 con 0x55a55854f5d0 2024-03-26T17:48:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/2236761033 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3f8005640 con 0x55a55854f5d0 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/2236761033 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc3f8004cd0 con 0x55a55854f5d0 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 10 client.28961 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 10 client.28961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 20 client.28961 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5585cbb10 0x55a5585cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 4 client.28961 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc412278740 1 -- 172.21.15.105:0/2236761033 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5584bb940 con 0x55a5585cbb10 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.900+0000 7fc40d421640 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5585cbb10 0x55a5585cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.901+0000 7fc40d421640 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5585cbb10 0x55a5585cdf30 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd37f520f4a02188 server_cookie=bbcb7672e66fe410 in_seq=0 out_seq=0 2024-03-26T17:48:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.901+0000 7fc3f0ff9640 10 client.28961 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc3f0ff9640 1 -- 172.21.15.105:0/2236761033 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5584bb940 con 0x55a5585cbb10 2024-03-26T17:48:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc3f0ff9640 10 client.28961 handle_command_reply: tid=0 2024-03-26T17:48:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 client.28961 shutdown 2024-03-26T17:48:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 -- 172.21.15.105:0/2236761033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 msgr2=0x55a55855dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55855dc80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8029990 tx=0x7fc3f8022040 comp rx=0 tx=0).stop 2024-03-26T17:48:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 -- 172.21.15.105:0/2236761033 shutdown_connections 2024-03-26T17:48:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5585cbb10 0x55a5585cdf30 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a558559b50 0x55a55855f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc3f0ff9640 0 client.28961 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a558555130 0x55a55855e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 --2- 172.21.15.105:0/2236761033 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a55854f5d0 0x55a55855dc80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 -- 172.21.15.105:0/2236761033 >> 172.21.15.105:0/2236761033 conn(0x55a55854ab40 msgr2=0x55a5585543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.948+0000 7fc412278740 1 -- 172.21.15.105:0/2236761033 shutdown_connections 2024-03-26T17:48:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc412278740 1 -- 172.21.15.105:0/2236761033 wait complete. 2024-03-26T17:48:33.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc412278740 20 client.28961 trim_cache size 0 max 16384 2024-03-26T17:48:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:48:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:48:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:48:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:48:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 -- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc3f40a2fc0 msgr2=0x7fc3f40a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc3f40a2fc0 0x7fc3f40a5480 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc4000c6280 tx=0x7fc400088840 comp rx=0 tx=0).stop 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 -- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 msgr2=0x7fc40819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 0x7fc40819efc0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc40002c700 tx=0x7fc4000328a0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 -- 172.21.15.105:0/2561266782 shutdown_connections 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc3f40a2fc0 0x7fc3f40a5480 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4081034a0 0x7fc40819efc0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc408102ae0 0x7fc408197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 --2- 172.21.15.105:0/2561266782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4081018f0 0x7fc408197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 -- 172.21.15.105:0/2561266782 >> 172.21.15.105:0/2561266782 conn(0x7fc4080fce80 msgr2=0x7fc408105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.949+0000 7fc40f685640 1 -- 172.21.15.105:0/2561266782 shutdown_connections 2024-03-26T17:48:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:33.950+0000 7fc40f685640 1 -- 172.21.15.105:0/2561266782 wait complete. 2024-03-26T17:48:34.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 326832 3.15 MB/sec execute 437 sec latency 32.661 ms 2024-03-26T17:48:34.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:33 smithi144 ceph-mon[36095]: pgmap v654: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:34.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:33 smithi105 ceph-mon[32280]: pgmap v654: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:34.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:33 smithi135 ceph-mon[35636]: pgmap v654: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:35.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 327392 3.15 MB/sec execute 438 sec latency 23.218 ms 2024-03-26T17:48:36.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 327987 3.15 MB/sec execute 439 sec latency 20.547 ms 2024-03-26T17:48:36.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:35 smithi144 ceph-mon[36095]: pgmap v655: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-03-26T17:48:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:35 smithi105 ceph-mon[32280]: pgmap v655: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-03-26T17:48:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:35 smithi135 ceph-mon[35636]: pgmap v655: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-03-26T17:48:37.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 328663 3.14 MB/sec execute 440 sec latency 26.292 ms 2024-03-26T17:48:38.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 329347 3.15 MB/sec execute 441 sec latency 17.215 ms 2024-03-26T17:48:38.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:37 smithi144 ceph-mon[36095]: pgmap v656: 97 pgs: 97 active+clean; 31 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:48:38.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:37 smithi105 ceph-mon[32280]: pgmap v656: 97 pgs: 97 active+clean; 31 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:48:38.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:37 smithi135 ceph-mon[35636]: pgmap v656: 97 pgs: 97 active+clean; 31 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.4 MiB/s wr, 103 op/s 2024-03-26T17:48:39.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 329904 3.15 MB/sec execute 442 sec latency 17.327 ms 2024-03-26T17:48:40.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 330358 3.15 MB/sec execute 443 sec latency 14.781 ms 2024-03-26T17:48:40.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:39 smithi144 ceph-mon[36095]: pgmap v657: 97 pgs: 97 active+clean; 30 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:48:40.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:39 smithi105 ceph-mon[32280]: pgmap v657: 97 pgs: 97 active+clean; 30 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:48:40.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:39 smithi135 ceph-mon[35636]: pgmap v657: 97 pgs: 97 active+clean; 30 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:48:41.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 330866 3.15 MB/sec execute 444 sec latency 26.030 ms 2024-03-26T17:48:42.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 331510 3.15 MB/sec execute 445 sec latency 21.136 ms 2024-03-26T17:48:42.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:41 smithi144 ceph-mon[36095]: pgmap v658: 97 pgs: 97 active+clean; 28 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T17:48:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:41 smithi105 ceph-mon[32280]: pgmap v658: 97 pgs: 97 active+clean; 28 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T17:48:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:41 smithi135 ceph-mon[35636]: pgmap v658: 97 pgs: 97 active+clean; 28 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.6 MiB/s wr, 126 op/s 2024-03-26T17:48:43.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 332222 3.15 MB/sec execute 446 sec latency 17.332 ms 2024-03-26T17:48:44.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 332914 3.15 MB/sec execute 447 sec latency 16.187 ms 2024-03-26T17:48:44.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:43 smithi144 ceph-mon[36095]: pgmap v659: 97 pgs: 97 active+clean; 28 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:48:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:43 smithi105 ceph-mon[32280]: pgmap v659: 97 pgs: 97 active+clean; 28 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:48:44.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:43 smithi135 ceph-mon[35636]: pgmap v659: 97 pgs: 97 active+clean; 28 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-03-26T17:48:45.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 333513 3.16 MB/sec execute 448 sec latency 15.905 ms 2024-03-26T17:48:46.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 333919 3.15 MB/sec execute 449 sec latency 15.542 ms 2024-03-26T17:48:46.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:45 smithi144 ceph-mon[36095]: pgmap v660: 97 pgs: 97 active+clean; 28 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:48:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:45 smithi105 ceph-mon[32280]: pgmap v660: 97 pgs: 97 active+clean; 28 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:48:46.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:45 smithi135 ceph-mon[35636]: pgmap v660: 97 pgs: 97 active+clean; 28 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:48:47.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 334471 3.15 MB/sec execute 450 sec latency 23.986 ms 2024-03-26T17:48:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 -- 172.21.15.105:0/2654785143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102390 msgr2=0x7f9564108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 --2- 172.21.15.105:0/2654785143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102390 0x7f9564108c20 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f955c009ca0 tx=0x7f955c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 -- 172.21.15.105:0/2654785143 shutdown_connections 2024-03-26T17:48:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 --2- 172.21.15.105:0/2654785143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9564102390 0x7f9564108c20 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 --2- 172.21.15.105:0/2654785143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95641019d0 0x7f9564101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 --2- 172.21.15.105:0/2654785143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95641007e0 0x7f9564100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 -- 172.21.15.105:0/2654785143 >> 172.21.15.105:0/2654785143 conn(0x7f95640fbd50 msgr2=0x7f95640fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 -- 172.21.15.105:0/2654785143 shutdown_connections 2024-03-26T17:48:47.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 -- 172.21.15.105:0/2654785143 wait complete. 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 Processor -- start 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 -- start start 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95641007e0 0x7f9564069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95641019d0 0x7f956419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.047+0000 7f956b40b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564102390 0x7f95641a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f956b40b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f956410c310 con 0x7f95641019d0 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f956b40b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f956410c190 con 0x7f95641007e0 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f956b40b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f956410c490 con 0x7f9564102390 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95641019d0 0x7f956419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95691a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95641007e0 0x7f9564069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95641019d0 0x7f956419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60090/0 (socket says 172.21.15.105:60090) 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 -- 172.21.15.105:0/2149679513 learned_addr learned my addr 172.21.15.105:0/2149679513 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95699a8640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564102390 0x7f95641a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 -- 172.21.15.105:0/2149679513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564102390 msgr2=0x7f95641a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564102390 0x7f95641a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 -- 172.21.15.105:0/2149679513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95641007e0 msgr2=0x7f9564069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95641007e0 0x7f9564069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 -- 172.21.15.105:0/2149679513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95641a18e0 con 0x7f95641019d0 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f95689a6640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95641019d0 0x7f956419eba0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f955400e9f0 tx=0x7f955400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f955a7fc640 1 -- 172.21.15.105:0/2149679513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955400cd80 con 0x7f95641019d0 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f955a7fc640 1 -- 172.21.15.105:0/2149679513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95540054d0 con 0x7f95641019d0 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f955a7fc640 1 -- 172.21.15.105:0/2149679513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554010490 con 0x7f95641019d0 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f956b40b640 1 -- 172.21.15.105:0/2149679513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95641a1b70 con 0x7f95641019d0 2024-03-26T17:48:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.048+0000 7f956b40b640 1 -- 172.21.15.105:0/2149679513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95641a20c0 con 0x7f95641019d0 2024-03-26T17:48:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.051+0000 7f955a7fc640 1 -- 172.21.15.105:0/2149679513 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9554005020 con 0x7f95641019d0 2024-03-26T17:48:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.051+0000 7f956dffa740 1 Processor -- start 2024-03-26T17:48:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.051+0000 7f956dffa740 1 -- start start 2024-03-26T17:48:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f956dffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f956dffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c71ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f956dffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f956dffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dba7bd10b0 con 0x55dba7c6c5d0 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f956dffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dba7b79960 con 0x55dba7c72130 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f956dffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dba7bcd060 con 0x55dba7c6cfc0 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f955a7fc640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95500a3070 0x7f95500a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95699a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c71ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c71ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35220/0 (socket says 172.21.15.105:35220) 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95699a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60100/0 (socket says 172.21.15.105:60100) 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 -- 172.21.15.105:0/293189671 learned_addr learned my addr 172.21.15.105:0/293189671 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95689a6640 1 --2- 172.21.15.105:0/293189671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95699a8640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95500a3070 0x7f95500a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 -- 172.21.15.105:0/293189671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 msgr2=0x55dba7c745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 --2- 172.21.15.105:0/293189671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c745c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 -- 172.21.15.105:0/293189671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 msgr2=0x55dba7c6c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 --2- 172.21.15.105:0/293189671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 -- 172.21.15.105:0/293189671 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba7b8bc60 con 0x55dba7c6cfc0 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f955a7fc640 1 -- 172.21.15.105:0/2149679513 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f95540cf6b0 con 0x7f95641019d0 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95691a7640 1 --2- 172.21.15.105:0/293189671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c71ac0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9560009ca0 tx=0x7f9560011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d8d863816982636 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.052+0000 7f95699a8640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95500a3070 0x7f95500a5530 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f955c006010 tx=0x7f955c0324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f954f7fe640 1 -- 172.21.15.105:0/293189671 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560004220 con 0x55dba7c6cfc0 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f954f7fe640 1 -- 172.21.15.105:0/293189671 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95600043c0 con 0x55dba7c6cfc0 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f954f7fe640 1 -- 172.21.15.105:0/293189671 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95600197f0 con 0x55dba7c6cfc0 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f956dffa740 1 -- 172.21.15.105:0/293189671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 msgr2=0x55dba7c71ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f956dffa740 1 --2- 172.21.15.105:0/293189671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c71ac0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9560009ca0 tx=0x7f9560011810 comp rx=0 tx=0).stop 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f956dffa740 1 -- 172.21.15.105:0/293189671 shutdown_connections 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f956dffa740 1 --2- 172.21.15.105:0/293189671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f956dffa740 1 --2- 172.21.15.105:0/293189671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c71ac0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f956dffa740 1 --2- 172.21.15.105:0/293189671 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.053+0000 7f956dffa740 1 -- 172.21.15.105:0/293189671 >> 172.21.15.105:0/293189671 conn(0x55dba7c67b40 msgr2=0x55dba7c67f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 -- 172.21.15.105:0/293189671 shutdown_connections 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 -- 172.21.15.105:0/293189671 wait complete. 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 -- start start 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 Processor -- start 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c6c5d0 0x55dba7c762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c76820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c72130 0x55dba7c7d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dba7c750e0 con 0x55dba7c72130 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dba7b8cdb0 con 0x55dba7c6c5d0 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.054+0000 7f956dffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dba7c75260 con 0x55dba7c6cfc0 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f956dffa740 10 client.0 _ll_register_callbacks cb 0x55dba7b91110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95699a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c72130 0x55dba7c7d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c76820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95691a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c6c5d0 0x55dba7c762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c76820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35226/0 (socket says 172.21.15.105:35226) 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 -- 172.21.15.105:0/42440496 learned_addr learned my addr 172.21.15.105:0/42440496 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 -- 172.21.15.105:0/42440496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c6c5d0 msgr2=0x55dba7c762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c6c5d0 0x55dba7c762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 -- 172.21.15.105:0/42440496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c72130 msgr2=0x55dba7c7d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c72130 0x55dba7c7d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 -- 172.21.15.105:0/42440496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba78e5210 con 0x55dba7c6cfc0 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95699a8640 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c72130 0x55dba7c7d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f95689a6640 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c76820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f95540cd170 tx=0x7f95540097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d58dc47746875c96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f954cff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f954cff9640 1 -- 172.21.15.105:0/42440496 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554005310 con 0x55dba7c6cfc0 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f954cff9640 1 -- 172.21.15.105:0/42440496 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f95540a4320 con 0x55dba7c6cfc0 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f956dffa740 1 -- 172.21.15.105:0/42440496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95600043c0 con 0x55dba7c6cfc0 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f954cff9640 1 -- 172.21.15.105:0/42440496 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9554005680 con 0x55dba7c6cfc0 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f956dffa740 10 client.28978 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.055+0000 7f956dffa740 1 -- 172.21.15.105:0/42440496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dba79b1620 con 0x55dba7c6cfc0 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f954cff9640 1 -- 172.21.15.105:0/42440496 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95600043c0 con 0x55dba7c6cfc0 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f954cff9640 1 -- 172.21.15.105:0/42440496 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f95540c9070 con 0x55dba7c6cfc0 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f956dffa740 10 client.28978 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f956dffa740 10 client.28978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f956dffa740 20 client.28978 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f956dffa740 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dba7c82210 0x55dba7c84600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f956dffa740 4 client.28978 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f956dffa740 1 -- 172.21.15.105:0/42440496 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dba7bde5f0 con 0x55dba7c82210 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f95691a7640 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dba7c82210 0x55dba7c84600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f954cff9640 10 client.28978 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.056+0000 7f95691a7640 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dba7c82210 0x55dba7c84600 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c20cc676a143594a server_cookie=fbd541756c27e0eb in_seq=0 out_seq=0 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f954cff9640 1 -- 172.21.15.105:0/42440496 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dba7bde5f0 con 0x55dba7c82210 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f954cff9640 10 client.28978 handle_command_reply: tid=0 2024-03-26T17:48:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 client.28978 shutdown 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 -- 172.21.15.105:0/42440496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 msgr2=0x55dba7c76820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c76820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f95540cd170 tx=0x7f95540097a0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 -- 172.21.15.105:0/42440496 shutdown_connections 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dba7c82210 0x55dba7c84600 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c72130 0x55dba7c7d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c76820 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f954cff9640 0 client.28978 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 --2- 172.21.15.105:0/42440496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c6c5d0 0x55dba7c762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 -- 172.21.15.105:0/42440496 >> 172.21.15.105:0/42440496 conn(0x55dba7c67b40 msgr2=0x55dba7c66df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 -- 172.21.15.105:0/42440496 shutdown_connections 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 1 -- 172.21.15.105:0/42440496 wait complete. 2024-03-26T17:48:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.057+0000 7f956dffa740 20 client.28978 trim_cache size 0 max 16384 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.075+0000 7f956dffa740 1 Processor -- start 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.075+0000 7f956dffa740 1 -- start start 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f956dffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f956dffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c7d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f956dffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c6dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f956dffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95641aaf40 con 0x55dba7c6c5d0 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f956dffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95641aca80 con 0x55dba7c72130 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f956dffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95641ab610 con 0x55dba7c6cfc0 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60108/0 (socket says 172.21.15.105:60108) 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 -- 172.21.15.105:0/3932746579 learned_addr learned my addr 172.21.15.105:0/3932746579 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95689a6640 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c7d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95699a8640 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c6dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 -- 172.21.15.105:0/3932746579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 msgr2=0x55dba7c7d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c7d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 -- 172.21.15.105:0/3932746579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 msgr2=0x55dba7c6dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c6dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 -- 172.21.15.105:0/3932746579 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba7c753b0 con 0x55dba7c6c5d0 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f95691a7640 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6da60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9560011d10 tx=0x7f9560011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e626b29054b5272 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f954effd640 1 -- 172.21.15.105:0/3932746579 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560009e50 con 0x55dba7c6c5d0 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f954effd640 1 -- 172.21.15.105:0/3932746579 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9560005a70 con 0x55dba7c6c5d0 2024-03-26T17:48:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.076+0000 7f954effd640 1 -- 172.21.15.105:0/3932746579 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560005380 con 0x55dba7c6c5d0 2024-03-26T17:48:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 -- 172.21.15.105:0/3932746579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 msgr2=0x55dba7c6da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6da60 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9560011d10 tx=0x7f9560011810 comp rx=0 tx=0).stop 2024-03-26T17:48:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 -- 172.21.15.105:0/3932746579 shutdown_connections 2024-03-26T17:48:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c6dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c7d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 --2- 172.21.15.105:0/3932746579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c6da60 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 -- 172.21.15.105:0/3932746579 >> 172.21.15.105:0/3932746579 conn(0x55dba7c67b40 msgr2=0x55dba7c762e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 -- 172.21.15.105:0/3932746579 shutdown_connections 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.077+0000 7f956dffa740 1 -- 172.21.15.105:0/3932746579 wait complete. 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 1 -- start start 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 1 Processor -- start 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c7a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c7ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c7be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95641ac3b0 con 0x55dba7c6c5d0 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95641abce0 con 0x55dba7c72130 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95641a8f20 con 0x55dba7c6cfc0 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f956dffa740 10 client.0 _ll_register_callbacks cb 0x7f955400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:47.074 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f95691a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c7a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f95691a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c7a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60118/0 (socket says 172.21.15.105:60118) 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f95691a7640 1 -- 172.21.15.105:0/3489770775 learned_addr learned my addr 172.21.15.105:0/3489770775 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f95699a8640 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c7be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.078+0000 7f95689a6640 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c7ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f95691a7640 1 -- 172.21.15.105:0/3489770775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 msgr2=0x55dba7c7ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f95691a7640 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c7ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f95691a7640 1 -- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 msgr2=0x55dba7c7be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f95691a7640 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c7be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f95691a7640 1 -- 172.21.15.105:0/3489770775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba7c7c440 con 0x55dba7c6c5d0 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f95699a8640 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c7be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f95691a7640 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c7a900 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9560009230 tx=0x7f9560003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fc730969db3dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f954cff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f954cff9640 1 -- 172.21.15.105:0/3489770775 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560005320 con 0x55dba7c6c5d0 2024-03-26T17:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f954cff9640 1 -- 172.21.15.105:0/3489770775 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f95600054c0 con 0x55dba7c6c5d0 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f956dffa740 1 -- 172.21.15.105:0/3489770775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9560005a70 con 0x55dba7c6c5d0 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f954cff9640 1 -- 172.21.15.105:0/3489770775 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9560044050 con 0x55dba7c6c5d0 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f956dffa740 10 client.19161 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f956dffa740 1 -- 172.21.15.105:0/3489770775 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dba7c7d750 con 0x55dba7c6c5d0 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.079+0000 7f954cff9640 1 -- 172.21.15.105:0/3489770775 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9560005a70 con 0x55dba7c6c5d0 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f954cff9640 1 -- 172.21.15.105:0/3489770775 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f956002bca0 con 0x55dba7c6c5d0 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f956dffa740 10 client.19161 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f956dffa740 10 client.19161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f956dffa740 20 client.19161 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f956dffa740 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dba7ce8690 0x55dba7ceaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f956dffa740 4 client.19161 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f956dffa740 1 -- 172.21.15.105:0/3489770775 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dba7bd8940 con 0x55dba7ce8690 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f95689a6640 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dba7ce8690 0x55dba7ceaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f954cff9640 10 client.19161 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:47.077 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.080+0000 7f95689a6640 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dba7ce8690 0x55dba7ceaa80 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116bc216ad77f186 server_cookie=ecf9754ea726d967 in_seq=0 out_seq=0 2024-03-26T17:48:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.131+0000 7f954cff9640 1 -- 172.21.15.105:0/3489770775 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dba7bd8940 con 0x55dba7ce8690 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f954cff9640 10 client.19161 handle_command_reply: tid=0 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 client.19161 shutdown 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 -- 172.21.15.105:0/3489770775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 msgr2=0x55dba7c7a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c7a900 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9560009230 tx=0x7f9560003e30 comp rx=0 tx=0).stop 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 -- 172.21.15.105:0/3489770775 shutdown_connections 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55dba7ce8690 0x55dba7ceaa80 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba7c72130 0x55dba7c7be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba7c6cfc0 0x55dba7c7ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f954cff9640 0 client.19161 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 --2- 172.21.15.105:0/3489770775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba7c6c5d0 0x55dba7c7a900 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 -- 172.21.15.105:0/3489770775 >> 172.21.15.105:0/3489770775 conn(0x55dba7c67b40 msgr2=0x55dba7c6da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 -- 172.21.15.105:0/3489770775 shutdown_connections 2024-03-26T17:48:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 1 -- 172.21.15.105:0/3489770775 wait complete. 2024-03-26T17:48:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.132+0000 7f956dffa740 20 client.19161 trim_cache size 0 max 16384 2024-03-26T17:48:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:48:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:48:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:48:47.125 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:48:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 -- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95500a3070 msgr2=0x7f95500a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95500a3070 0x7f95500a5530 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f955c006010 tx=0x7f955c0324f0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 -- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95641019d0 msgr2=0x7f956419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95641019d0 0x7f956419eba0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f955400e9f0 tx=0x7f955400edf0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 -- 172.21.15.105:0/2149679513 shutdown_connections 2024-03-26T17:48:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95500a3070 0x7f95500a5530 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9564102390 0x7f95641a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95641019d0 0x7f956419eba0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 --2- 172.21.15.105:0/2149679513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95641007e0 0x7f9564069ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 -- 172.21.15.105:0/2149679513 >> 172.21.15.105:0/2149679513 conn(0x7f95640fbd50 msgr2=0x7f9564104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 -- 172.21.15.105:0/2149679513 shutdown_connections 2024-03-26T17:48:47.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:48:47.133+0000 7f956b40b640 1 -- 172.21.15.105:0/2149679513 wait complete. 2024-03-26T17:48:48.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 335059 3.15 MB/sec execute 451 sec latency 46.905 ms 2024-03-26T17:48:48.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:47 smithi144 ceph-mon[36095]: pgmap v661: 97 pgs: 97 active+clean; 28 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-03-26T17:48:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:47 smithi105 ceph-mon[32280]: pgmap v661: 97 pgs: 97 active+clean; 28 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-03-26T17:48:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:47 smithi135 ceph-mon[35636]: pgmap v661: 97 pgs: 97 active+clean; 28 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-03-26T17:48:49.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 335736 3.15 MB/sec execute 452 sec latency 21.920 ms 2024-03-26T17:48:50.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 336403 3.15 MB/sec execute 453 sec latency 17.624 ms 2024-03-26T17:48:50.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:49 smithi144 ceph-mon[36095]: pgmap v662: 97 pgs: 97 active+clean; 30 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:48:50.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:49 smithi105 ceph-mon[32280]: pgmap v662: 97 pgs: 97 active+clean; 30 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:48:50.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:49 smithi135 ceph-mon[35636]: pgmap v662: 97 pgs: 97 active+clean; 30 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:48:51.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 336933 3.16 MB/sec execute 454 sec latency 17.464 ms 2024-03-26T17:48:51.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:48:51.284 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 -- 172.21.15.105:0/3805434501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c107620 msgr2=0x7f163c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 --2- 172.21.15.105:0/3805434501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c107620 0x7f163c109a10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1634009ca0 tx=0x7f163402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 -- 172.21.15.105:0/3805434501 shutdown_connections 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 --2- 172.21.15.105:0/3805434501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c107620 0x7f163c109a10 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 --2- 172.21.15.105:0/3805434501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f163c104cf0 0x7f163c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 --2- 172.21.15.105:0/3805434501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f163c1023c0 0x7f163c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 -- 172.21.15.105:0/3805434501 >> 172.21.15.105:0/3805434501 conn(0x7f163c0fbd50 msgr2=0x7f163c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 -- 172.21.15.105:0/3805434501 shutdown_connections 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.409+0000 7f1641c52640 1 -- 172.21.15.105:0/3805434501 wait complete. 2024-03-26T17:48:51.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f1641c52640 1 Processor -- start 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f1641c52640 1 -- start start 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f1641c52640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f163c1023c0 0x7f163c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f1641c52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c104cf0 0x7f163c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f1641c52640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f163c107620 0x7f163c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f1641c52640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10d410 con 0x7f163c104cf0 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f1641c52640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10d290 con 0x7f163c107620 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f1641c52640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10d590 con 0x7f163c1023c0 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c104cf0 0x7f163c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f163c107620 0x7f163c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c104cf0 0x7f163c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60130/0 (socket says 172.21.15.105:60130) 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163affd640 1 -- 172.21.15.105:0/3093946383 learned_addr learned my addr 172.21.15.105:0/3093946383 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f163c1023c0 0x7f163c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163affd640 1 -- 172.21.15.105:0/3093946383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f163c1023c0 msgr2=0x7f163c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163affd640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f163c1023c0 0x7f163c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163affd640 1 -- 172.21.15.105:0/3093946383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f163c107620 msgr2=0x7f163c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163affd640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f163c107620 0x7f163c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.410+0000 7f163affd640 1 -- 172.21.15.105:0/3093946383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c19f740 con 0x7f163c104cf0 2024-03-26T17:48:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.411+0000 7f163affd640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c104cf0 0x7f163c198060 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f163000e9c0 tx=0x7f163000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.411+0000 7f1638ff9640 1 -- 172.21.15.105:0/3093946383 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163000cd80 con 0x7f163c104cf0 2024-03-26T17:48:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.411+0000 7f1638ff9640 1 -- 172.21.15.105:0/3093946383 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16300054d0 con 0x7f163c104cf0 2024-03-26T17:48:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.411+0000 7f1641c52640 1 -- 172.21.15.105:0/3093946383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c19fa30 con 0x7f163c104cf0 2024-03-26T17:48:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.411+0000 7f1638ff9640 1 -- 172.21.15.105:0/3093946383 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630010430 con 0x7f163c104cf0 2024-03-26T17:48:51.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.411+0000 7f1641c52640 1 -- 172.21.15.105:0/3093946383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c19ff80 con 0x7f163c104cf0 2024-03-26T17:48:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.413+0000 7f1638ff9640 1 -- 172.21.15.105:0/3093946383 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1630004fd0 con 0x7f163c104cf0 2024-03-26T17:48:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f164483e740 1 Processor -- start 2024-03-26T17:48:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f164483e740 1 -- start start 2024-03-26T17:48:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f164483e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c57120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f164483e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c5c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f164483e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c5ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f164483e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637a4bb7980 con 0x5637a4c576f0 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f164483e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5637a4b5fbe0 con 0x5637a4c5c860 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f164483e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637a4bb3930 con 0x5637a4c56d00 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f1638ff9640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16240a2f20 0x7f16240a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c5c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c5c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60138/0 (socket says 172.21.15.105:60138) 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163b7fe640 1 -- 172.21.15.105:0/1086248090 learned_addr learned my addr 172.21.15.105:0/1086248090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163bfff640 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c57120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163affd640 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c5ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163bfff640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16240a2f20 0x7f16240a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163b7fe640 1 -- 172.21.15.105:0/1086248090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 msgr2=0x5637a4c57120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163b7fe640 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c57120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163b7fe640 1 -- 172.21.15.105:0/1086248090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 msgr2=0x5637a4c5ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f1638ff9640 1 -- 172.21.15.105:0/3093946383 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f16300cf840 con 0x7f163c104cf0 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163b7fe640 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c5ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163b7fe640 1 -- 172.21.15.105:0/1086248090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a49957d0 con 0x5637a4c576f0 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.414+0000 7f163bfff640 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c57120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.415+0000 7f163b7fe640 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c5c1f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f162c009d20 tx=0x7f162c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9802ae4ecffb374e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.415+0000 7f163bfff640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16240a2f20 0x7f16240a53e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1634006010 tx=0x7f16340324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.415+0000 7f1621ffb640 1 -- 172.21.15.105:0/1086248090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c01e070 con 0x5637a4c576f0 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.415+0000 7f1621ffb640 1 -- 172.21.15.105:0/1086248090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f162c019ae0 con 0x5637a4c576f0 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.415+0000 7f1621ffb640 1 -- 172.21.15.105:0/1086248090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c00ea80 con 0x5637a4c576f0 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 -- 172.21.15.105:0/1086248090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 msgr2=0x5637a4c5c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c5c1f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f162c009d20 tx=0x7f162c011810 comp rx=0 tx=0).stop 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 -- 172.21.15.105:0/1086248090 shutdown_connections 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c5ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c5c1f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 --2- 172.21.15.105:0/1086248090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c57120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 -- 172.21.15.105:0/1086248090 >> 172.21.15.105:0/1086248090 conn(0x5637a4c4de60 msgr2=0x5637a4c4e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 -- 172.21.15.105:0/1086248090 shutdown_connections 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 -- 172.21.15.105:0/1086248090 wait complete. 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 -- start start 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.416+0000 7f164483e740 1 Processor -- start 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c60f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c576f0 0x5637a4c61450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c5c860 0x5637a4c68490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637a4c5f810 con 0x5637a4c5c860 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5637a4b73160 con 0x5637a4c576f0 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637a4c5f990 con 0x5637a4c56d00 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c5c860 0x5637a4c68490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 10 client.0 _ll_register_callbacks cb 0x5637a4baf720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c5c860 0x5637a4c68490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60146/0 (socket says 172.21.15.105:60146) 2024-03-26T17:48:51.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 -- 172.21.15.105:0/2636554910 learned_addr learned my addr 172.21.15.105:0/2636554910 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163b7fe640 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c60f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163affd640 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c576f0 0x5637a4c61450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 -- 172.21.15.105:0/2636554910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 msgr2=0x5637a4c60f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c60f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 -- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c576f0 msgr2=0x5637a4c61450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c576f0 0x5637a4c61450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 -- 172.21.15.105:0/2636554910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a48cb3f0 con 0x5637a4c5c860 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163affd640 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c576f0 0x5637a4c61450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f163bfff640 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c5c860 0x5637a4c68490 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f1634036000 tx=0x7f16340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6439948a6fa97e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f161affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f161affd640 1 -- 172.21.15.105:0/2636554910 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634053060 con 0x5637a4c5c860 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f161affd640 1 -- 172.21.15.105:0/2636554910 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1634003d30 con 0x5637a4c5c860 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 1 -- 172.21.15.105:0/2636554910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f162c019ae0 con 0x5637a4c5c860 2024-03-26T17:48:51.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f161affd640 1 -- 172.21.15.105:0/2636554910 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1634039040 con 0x5637a4c5c860 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 10 client.19173 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.417+0000 7f164483e740 1 -- 172.21.15.105:0/2636554910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637a488b0f0 con 0x5637a4c5c860 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f161affd640 1 -- 172.21.15.105:0/2636554910 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f162c019ae0 con 0x5637a4c5c860 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f161affd640 1 -- 172.21.15.105:0/2636554910 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f163403f070 con 0x5637a4c5c860 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f164483e740 10 client.19173 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f164483e740 10 client.19173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f164483e740 20 client.19173 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f164483e740 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a4c6cde0 0x5637a4c6f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f164483e740 4 client.19173 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f164483e740 1 -- 172.21.15.105:0/2636554910 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637a4bc4ec0 con 0x5637a4c6cde0 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f163b7fe640 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a4c6cde0 0x5637a4c6f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f161affd640 10 client.19173 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.418+0000 7f163b7fe640 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a4c6cde0 0x5637a4c6f240 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c82a1f576d1656 server_cookie=4c6408d8c572ad09 in_seq=0 out_seq=0 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f161affd640 1 -- 172.21.15.105:0/2636554910 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5637a4bc4ec0 con 0x5637a4c6cde0 2024-03-26T17:48:51.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f161affd640 10 client.19173 handle_command_reply: tid=0 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 client.19173 shutdown 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 -- 172.21.15.105:0/2636554910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c5c860 msgr2=0x5637a4c68490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c5c860 0x5637a4c68490 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f1634036000 tx=0x7f16340095c0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 -- 172.21.15.105:0/2636554910 shutdown_connections 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a4c6cde0 0x5637a4c6f240 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c5c860 0x5637a4c68490 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f161affd640 0 client.19173 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c576f0 0x5637a4c61450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 --2- 172.21.15.105:0/2636554910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c60f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.419+0000 7f164483e740 1 -- 172.21.15.105:0/2636554910 >> 172.21.15.105:0/2636554910 conn(0x5637a4c4de60 msgr2=0x5637a4c4d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.420+0000 7f164483e740 1 -- 172.21.15.105:0/2636554910 shutdown_connections 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.420+0000 7f164483e740 1 -- 172.21.15.105:0/2636554910 wait complete. 2024-03-26T17:48:51.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.420+0000 7f164483e740 20 client.19173 trim_cache size 0 max 16384 2024-03-26T17:48:51.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f164483e740 1 Processor -- start 2024-03-26T17:48:51.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f164483e740 1 -- start start 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f164483e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c4c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f164483e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c68450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f164483e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c4cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f164483e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f163c1a6bd0 con 0x5637a4c576f0 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f164483e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f163c1a8710 con 0x5637a4c5c860 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f164483e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f163c1a72a0 con 0x5637a4c56d00 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f163affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c68450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f163bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c4cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f163affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c68450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60148/0 (socket says 172.21.15.105:60148) 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f163affd640 1 -- 172.21.15.105:0/4202238512 learned_addr learned my addr 172.21.15.105:0/4202238512 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.438+0000 7f163b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c4c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f163affd640 1 -- 172.21.15.105:0/4202238512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 msgr2=0x5637a4c4c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f163affd640 1 --2- 172.21.15.105:0/4202238512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c4c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f163affd640 1 -- 172.21.15.105:0/4202238512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 msgr2=0x5637a4c4cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f163affd640 1 --2- 172.21.15.105:0/4202238512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c4cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f163affd640 1 -- 172.21.15.105:0/4202238512 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a4c583c0 con 0x5637a4c576f0 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f163bfff640 1 --2- 172.21.15.105:0/4202238512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c4cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f163affd640 1 --2- 172.21.15.105:0/4202238512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c68450 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1630098900 tx=0x7f163006dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e484905be80d87 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f16217fa640 1 -- 172.21.15.105:0/4202238512 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163009ba70 con 0x5637a4c576f0 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f16217fa640 1 -- 172.21.15.105:0/4202238512 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f163009c630 con 0x5637a4c576f0 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.439+0000 7f16217fa640 1 -- 172.21.15.105:0/4202238512 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163009b890 con 0x5637a4c576f0 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 -- 172.21.15.105:0/4202238512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 msgr2=0x5637a4c68450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 --2- 172.21.15.105:0/4202238512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c68450 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1630098900 tx=0x7f163006dbb0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 -- 172.21.15.105:0/4202238512 shutdown_connections 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 --2- 172.21.15.105:0/4202238512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c4cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 --2- 172.21.15.105:0/4202238512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c68450 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 --2- 172.21.15.105:0/4202238512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c4c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 -- 172.21.15.105:0/4202238512 >> 172.21.15.105:0/4202238512 conn(0x5637a4c4de60 msgr2=0x5637a4c60f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 -- 172.21.15.105:0/4202238512 shutdown_connections 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 -- 172.21.15.105:0/4202238512 wait complete. 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 -- start start 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.440+0000 7f164483e740 1 Processor -- start 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f164483e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c66120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f164483e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c66660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f164483e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c67620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f164483e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f163c1a8040 con 0x5637a4c576f0 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f164483e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f163c1a7970 con 0x5637a4c5c860 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f164483e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f163c1a4bb0 con 0x5637a4c56d00 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f164483e740 10 client.0 _ll_register_callbacks cb 0x7f162c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c66660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c66660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60152/0 (socket says 172.21.15.105:60152) 2024-03-26T17:48:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 -- 172.21.15.105:0/3418918239 learned_addr learned my addr 172.21.15.105:0/3418918239 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163b7fe640 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c66120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163bfff640 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c67620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 -- 172.21.15.105:0/3418918239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 msgr2=0x5637a4c66120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c66120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 -- 172.21.15.105:0/3418918239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 msgr2=0x5637a4c67620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c67620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 -- 172.21.15.105:0/3418918239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a4c67c10 con 0x5637a4c576f0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f163affd640 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c66660 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f1630004a00 tx=0x7f163009b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbea4253d1ab89f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f161affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f161affd640 1 -- 172.21.15.105:0/3418918239 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300a4940 con 0x5637a4c576f0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.441+0000 7f161affd640 1 -- 172.21.15.105:0/3418918239 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f16300985a0 con 0x5637a4c576f0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 1 -- 172.21.15.105:0/3418918239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f163009c630 con 0x5637a4c576f0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f161affd640 1 -- 172.21.15.105:0/3418918239 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16300a0080 con 0x5637a4c576f0 2024-03-26T17:48:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 10 client.19179 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 1 -- 172.21.15.105:0/3418918239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637a4cd3750 con 0x5637a4c576f0 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f161affd640 1 -- 172.21.15.105:0/3418918239 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f163009c630 con 0x5637a4c576f0 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f161affd640 1 -- 172.21.15.105:0/3418918239 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f1630097da0 con 0x5637a4c576f0 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 10 client.19179 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 10 client.19179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 20 client.19179 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a4cd3fe0 0x5637a4cd6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 4 client.19179 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f164483e740 1 -- 172.21.15.105:0/3418918239 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5637a4bbf210 con 0x5637a4cd3fe0 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.442+0000 7f163b7fe640 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a4cd3fe0 0x5637a4cd6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f161affd640 10 client.19179 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f163b7fe640 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a4cd3fe0 0x5637a4cd6400 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=814857e263e45d7b server_cookie=f4a873ffec115518 in_seq=0 out_seq=0 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f161affd640 1 -- 172.21.15.105:0/3418918239 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5637a4bbf210 con 0x5637a4cd3fe0 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f161affd640 10 client.19179 handle_command_reply: tid=0 2024-03-26T17:48:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 client.19179 shutdown 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 -- 172.21.15.105:0/3418918239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 msgr2=0x5637a4c66660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c66660 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f1630004a00 tx=0x7f163009b990 comp rx=0 tx=0).stop 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 -- 172.21.15.105:0/3418918239 shutdown_connections 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a4cd3fe0 0x5637a4cd6400 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a4c5c860 0x5637a4c67620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a4c576f0 0x5637a4c66660 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f161affd640 0 client.19179 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 --2- 172.21.15.105:0/3418918239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a4c56d00 0x5637a4c66120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 -- 172.21.15.105:0/3418918239 >> 172.21.15.105:0/3418918239 conn(0x5637a4c4de60 msgr2=0x5637a4c510c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 -- 172.21.15.105:0/3418918239 shutdown_connections 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 1 -- 172.21.15.105:0/3418918239 wait complete. 2024-03-26T17:48:51.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.443+0000 7f164483e740 20 client.19179 trim_cache size 0 max 16384 2024-03-26T17:48:51.442 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:48:51.442 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:48:51.442 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:48:51.442 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 -- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16240a2f20 msgr2=0x7f16240a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16240a2f20 0x7f16240a53e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1634006010 tx=0x7f16340324f0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 -- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c104cf0 msgr2=0x7f163c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c104cf0 0x7f163c198060 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f163000e9c0 tx=0x7f163000ee80 comp rx=0 tx=0).stop 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 -- 172.21.15.105:0/3093946383 shutdown_connections 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16240a2f20 0x7f16240a53e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f163c107620 0x7f163c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f163c104cf0 0x7f163c198060 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 --2- 172.21.15.105:0/3093946383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f163c1023c0 0x7f163c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 -- 172.21.15.105:0/3093946383 >> 172.21.15.105:0/3093946383 conn(0x7f163c0fbd50 msgr2=0x7f163c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 -- 172.21.15.105:0/3093946383 shutdown_connections 2024-03-26T17:48:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.444+0000 7f1641c52640 1 -- 172.21.15.105:0/3093946383 wait complete. 2024-03-26T17:48:51.445 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:48:51.446 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.571+0000 7f1d7035b640 1 -- 172.21.15.105:0/4258964359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68102af0 msgr2=0x7f1d68102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.571+0000 7f1d7035b640 1 --2- 172.21.15.105:0/4258964359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68102af0 0x7f1d68102f70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009900 tx=0x7f1d5c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.571+0000 7f1d7035b640 1 -- 172.21.15.105:0/4258964359 shutdown_connections 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.571+0000 7f1d7035b640 1 --2- 172.21.15.105:0/4258964359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d681034b0 0x7f1d68109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.571+0000 7f1d7035b640 1 --2- 172.21.15.105:0/4258964359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68102af0 0x7f1d68102f70 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.571+0000 7f1d7035b640 1 --2- 172.21.15.105:0/4258964359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d68101900 0x7f1d68101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.571+0000 7f1d7035b640 1 -- 172.21.15.105:0/4258964359 >> 172.21.15.105:0/4258964359 conn(0x7f1d680fce70 msgr2=0x7f1d680ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 -- 172.21.15.105:0/4258964359 shutdown_connections 2024-03-26T17:48:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 -- 172.21.15.105:0/4258964359 wait complete. 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 Processor -- start 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 -- start start 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68101900 0x7f1d68197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d68102af0 0x7f1d681980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d681034b0 0x7f1d6819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810d430 con 0x7f1d68101900 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810d2b0 con 0x7f1d681034b0 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d7035b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6810d5b0 con 0x7f1d68102af0 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.572+0000 7f1d6e0f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68101900 0x7f1d68197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e0f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68101900 0x7f1d68197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60178/0 (socket says 172.21.15.105:60178) 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e0f7640 1 -- 172.21.15.105:0/1977575806 learned_addr learned my addr 172.21.15.105:0/1977575806 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6d8f6640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d68102af0 0x7f1d681980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e8f8640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d681034b0 0x7f1d6819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e8f8640 1 -- 172.21.15.105:0/1977575806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d68102af0 msgr2=0x7f1d681980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e8f8640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d68102af0 0x7f1d681980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e8f8640 1 -- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68101900 msgr2=0x7f1d68197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e8f8640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68101900 0x7f1d68197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e8f8640 1 -- 172.21.15.105:0/1977575806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6819f7a0 con 0x7f1d681034b0 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e0f7640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68101900 0x7f1d68197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d6e8f8640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d681034b0 0x7f1d6819f130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000e970 tx=0x7f1d6000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d677fe640 1 -- 172.21.15.105:0/1977575806 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000ccc0 con 0x7f1d681034b0 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6819fa90 con 0x7f1d681034b0 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d677fe640 1 -- 172.21.15.105:0/1977575806 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d60005020 con 0x7f1d681034b0 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d677fe640 1 -- 172.21.15.105:0/1977575806 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000ce60 con 0x7f1d681034b0 2024-03-26T17:48:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.573+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d681a0000 con 0x7f1d681034b0 2024-03-26T17:48:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.574+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d68108cc0 con 0x7f1d681034b0 2024-03-26T17:48:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.576+0000 7f1d677fe640 1 -- 172.21.15.105:0/1977575806 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1d600051d0 con 0x7f1d681034b0 2024-03-26T17:48:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.576+0000 7f1d677fe640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d500a2f20 0x7f1d500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.577+0000 7f1d677fe640 1 -- 172.21.15.105:0/1977575806 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f1d600cf400 con 0x7f1d681034b0 2024-03-26T17:48:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.577+0000 7f1d6e0f7640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d500a2f20 0x7f1d500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.577+0000 7f1d6e0f7640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d500a2f20 0x7f1d500a53e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800f7b0 tx=0x7f1d58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.582+0000 7f1d677fe640 1 -- 172.21.15.105:0/1977575806 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1d6009c280 con 0x7f1d681034b0 2024-03-26T17:48:51.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.841+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d6802d240 con 0x7f1d681034b0 2024-03-26T17:48:51.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.842+0000 7f1d677fe640 1 -- 172.21.15.105:0/1977575806 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11938 (secure 0 0 0) 0x7f1d6009bb80 con 0x7f1d681034b0 2024-03-26T17:48:51.833 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:48:51.834 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:40:53.347265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26147":{"gid":26147,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/260658746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":260658746},{"type":"v1","addr":"172.21.15.135:6835","nonce":260658746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:48:51.834 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 30 2024-03-26T17:48:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.844+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d500a2f20 msgr2=0x7f1d500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.844+0000 7f1d7035b640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d500a2f20 0x7f1d500a53e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800f7b0 tx=0x7f1d58009500 comp rx=0 tx=0).stop 2024-03-26T17:48:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.844+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d681034b0 msgr2=0x7f1d6819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.844+0000 7f1d7035b640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d681034b0 0x7f1d6819f130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000e970 tx=0x7f1d6000ee30 comp rx=0 tx=0).stop 2024-03-26T17:48:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.845+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 shutdown_connections 2024-03-26T17:48:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.845+0000 7f1d7035b640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1d500a2f20 0x7f1d500a53e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.845+0000 7f1d7035b640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1d681034b0 0x7f1d6819f130 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.845+0000 7f1d7035b640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d68102af0 0x7f1d681980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.845+0000 7f1d7035b640 1 --2- 172.21.15.105:0/1977575806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d68101900 0x7f1d68197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.845+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 >> 172.21.15.105:0/1977575806 conn(0x7f1d680fce70 msgr2=0x7f1d68105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.845+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 shutdown_connections 2024-03-26T17:48:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.845+0000 7f1d7035b640 1 -- 172.21.15.105:0/1977575806 wait complete. 2024-03-26T17:48:51.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 -- 172.21.15.105:0/2729448244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb180fff80 msgr2=0x7ffb181003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/2729448244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb180fff80 0x7ffb181003a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800aa10 tx=0x7ffb0802cf70 comp rx=0 tx=0).stop 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 -- 172.21.15.105:0/2729448244 shutdown_connections 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/2729448244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb18101b30 0x7ffb181083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/2729448244 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb18101170 0x7ffb181015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/2729448244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb180fff80 0x7ffb181003a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 -- 172.21.15.105:0/2729448244 >> 172.21.15.105:0/2729448244 conn(0x7ffb180fb4f0 msgr2=0x7ffb180fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 -- 172.21.15.105:0/2729448244 shutdown_connections 2024-03-26T17:48:51.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.979+0000 7ffb1dc12640 1 -- 172.21.15.105:0/2729448244 wait complete. 2024-03-26T17:48:51.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb1dc12640 1 Processor -- start 2024-03-26T17:48:51.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb1dc12640 1 -- start start 2024-03-26T17:48:51.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb1dc12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb180fff80 0x7ffb181959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb1dc12640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb18101170 0x7ffb18195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb1dc12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb18101b30 0x7ffb1819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb1dc12640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1810b280 con 0x7ffb18101b30 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb1dc12640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1810b100 con 0x7ffb18101170 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb1dc12640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1810b400 con 0x7ffb180fff80 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb18101b30 0x7ffb1819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb180fff80 0x7ffb181959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb16ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb18101170 0x7ffb18195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb18101b30 0x7ffb1819cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60196/0 (socket says 172.21.15.105:60196) 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb180fff80 0x7ffb181959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35328/0 (socket says 172.21.15.105:35328) 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.980+0000 7ffb17fff640 1 -- 172.21.15.105:0/1192866404 learned_addr learned my addr 172.21.15.105:0/1192866404 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb16ffd640 1 -- 172.21.15.105:0/1192866404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb180fff80 msgr2=0x7ffb181959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb16ffd640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb180fff80 0x7ffb181959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb16ffd640 1 -- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb18101b30 msgr2=0x7ffb1819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb16ffd640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb18101b30 0x7ffb1819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb16ffd640 1 -- 172.21.15.105:0/1192866404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1819d5d0 con 0x7ffb18101170 2024-03-26T17:48:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb16ffd640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb18101170 0x7ffb18195ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00d990 tx=0x7ffb0c00de50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb14ff9640 1 -- 172.21.15.105:0/1192866404 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c01b070 con 0x7ffb18101170 2024-03-26T17:48:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb1dc12640 1 -- 172.21.15.105:0/1192866404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1819d8c0 con 0x7ffb18101170 2024-03-26T17:48:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb14ff9640 1 -- 172.21.15.105:0/1192866404 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb0c00ed60 con 0x7ffb18101170 2024-03-26T17:48:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb14ff9640 1 -- 172.21.15.105:0/1192866404 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c014720 con 0x7ffb18101170 2024-03-26T17:48:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.981+0000 7ffb1dc12640 1 -- 172.21.15.105:0/1192866404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1819de90 con 0x7ffb18101170 2024-03-26T17:48:51.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb207fe740 1 Processor -- start 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb207fe740 1 -- start start 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e576260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e57b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e57de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb207fe740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca0e4d6970 con 0x55ca0e575e40 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb207fe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca0e47ed70 con 0x55ca0e576830 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb207fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca0e4d2920 con 0x55ca0e57b9a0 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e576260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e576260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60210/0 (socket says 172.21.15.105:60210) 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.984+0000 7ffb17fff640 1 -- 172.21.15.105:0/2096601262 learned_addr learned my addr 172.21.15.105:0/2096601262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb17fff640 1 -- 172.21.15.105:0/2096601262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 msgr2=0x55ca0e57de30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:48:51.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb17fff640 1 --2- 172.21.15.105:0/2096601262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e57de30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb17fff640 1 -- 172.21.15.105:0/2096601262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 msgr2=0x55ca0e57b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb17fff640 1 --2- 172.21.15.105:0/2096601262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e57b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb17fff640 1 -- 172.21.15.105:0/2096601262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca0e1ea3f0 con 0x55ca0e575e40 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb14ff9640 1 -- 172.21.15.105:0/1192866404 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ffb0c01e070 con 0x7ffb18101170 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb17fff640 1 --2- 172.21.15.105:0/2096601262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e576260 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ffb10011570 tx=0x7ffb10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f400b430646cb3f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffafdffb640 1 -- 172.21.15.105:0/2096601262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10004b20 con 0x55ca0e575e40 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffafdffb640 1 -- 172.21.15.105:0/2096601262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb10004cc0 con 0x55ca0e575e40 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffafdffb640 1 -- 172.21.15.105:0/2096601262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10005390 con 0x55ca0e575e40 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb14ff9640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb000a2f20 0x7ffb000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb177fe640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb000a2f20 0x7ffb000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.985+0000 7ffb14ff9640 1 -- 172.21.15.105:0/1192866404 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ffb0c0ce990 con 0x7ffb18101170 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb177fe640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb000a2f20 0x7ffb000a53e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffb08005ab0 tx=0x7ffb08005740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 -- 172.21.15.105:0/2096601262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 msgr2=0x55ca0e576260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 --2- 172.21.15.105:0/2096601262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e576260 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ffb10011570 tx=0x7ffb10011a30 comp rx=0 tx=0).stop 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 -- 172.21.15.105:0/2096601262 shutdown_connections 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 --2- 172.21.15.105:0/2096601262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e57de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 --2- 172.21.15.105:0/2096601262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e57b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 --2- 172.21.15.105:0/2096601262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e576260 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 -- 172.21.15.105:0/2096601262 >> 172.21.15.105:0/2096601262 conn(0x55ca0e56cfa0 msgr2=0x55ca0e56d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 -- 172.21.15.105:0/2096601262 shutdown_connections 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 -- 172.21.15.105:0/2096601262 wait complete. 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.986+0000 7ffb207fe740 1 -- start start 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb207fe740 1 Processor -- start 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e575e40 0x55ca0e580260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb207fe740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ca0e57e950 con 0x55ca0e57b9a0 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb207fe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ca0e492310 con 0x55ca0e576830 2024-03-26T17:48:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb207fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca0e57ead0 con 0x55ca0e575e40 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb207fe740 10 client.0 _ll_register_callbacks cb 0x55ca0e4cdc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e575e40 0x55ca0e580260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5877e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60212/0 (socket says 172.21.15.105:60212) 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb17fff640 1 -- 172.21.15.105:0/1190546352 learned_addr learned my addr 172.21.15.105:0/1190546352 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e575e40 0x55ca0e580260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35336/0 (socket says 172.21.15.105:35336) 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb16ffd640 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb17fff640 1 -- 172.21.15.105:0/1190546352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e575e40 msgr2=0x55ca0e580260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb17fff640 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e575e40 0x55ca0e580260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb17fff640 1 -- 172.21.15.105:0/1190546352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 msgr2=0x55ca0e5807a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb17fff640 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5807a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.987+0000 7ffb17fff640 1 -- 172.21.15.105:0/1190546352 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca0e2b47d0 con 0x55ca0e57b9a0 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffb17fff640 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5877e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ffb10005c40 tx=0x7ffb10004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b3feb26f5d49e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffaf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1190546352 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10028070 con 0x55ca0e57b9a0 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1190546352 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ffb100050b0 con 0x55ca0e57b9a0 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffb207fe740 1 -- 172.21.15.105:0/1190546352 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb10004cc0 con 0x55ca0e57b9a0 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1190546352 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb10034050 con 0x55ca0e57b9a0 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffb207fe740 10 client.19197 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffb207fe740 1 -- 172.21.15.105:0/1190546352 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ca0e58ac10 con 0x55ca0e57b9a0 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1190546352 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb10004cc0 con 0x55ca0e57b9a0 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1190546352 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffb100248c0 con 0x55ca0e57b9a0 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.988+0000 7ffb207fe740 10 client.19197 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.989+0000 7ffb207fe740 10 client.19197 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.989+0000 7ffb207fe740 20 client.19197 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.989+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ca0e58c160 0x55ca0e58e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.989+0000 7ffb207fe740 4 client.19197 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.989+0000 7ffb207fe740 1 -- 172.21.15.105:0/1190546352 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca0e4e3eb0 con 0x55ca0e58c160 2024-03-26T17:48:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.989+0000 7ffb177fe640 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ca0e58c160 0x55ca0e58e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.989+0000 7ffaf6ffd640 10 client.19197 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:48:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.989+0000 7ffb177fe640 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ca0e58c160 0x55ca0e58e580 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f93d77367fd15fd server_cookie=34e7b714ddab64a3 in_seq=0 out_seq=0 2024-03-26T17:48:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1190546352 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ca0e4e3eb0 con 0x55ca0e58c160 2024-03-26T17:48:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffaf6ffd640 10 client.19197 handle_command_reply: tid=0 2024-03-26T17:48:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 client.19197 shutdown 2024-03-26T17:48:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 -- 172.21.15.105:0/1190546352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e57b9a0 msgr2=0x55ca0e5877e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5877e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ffb10005c40 tx=0x7ffb10004f50 comp rx=0 tx=0).stop 2024-03-26T17:48:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 -- 172.21.15.105:0/1190546352 shutdown_connections 2024-03-26T17:48:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ca0e58c160 0x55ca0e58e580 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5877e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5807a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1190546352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e575e40 0x55ca0e580260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 -- 172.21.15.105:0/1190546352 >> 172.21.15.105:0/1190546352 conn(0x55ca0e56cfa0 msgr2=0x55ca0e56c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffaf6ffd640 0 client.19197 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:48:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 -- 172.21.15.105:0/1190546352 shutdown_connections 2024-03-26T17:48:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.990+0000 7ffb207fe740 1 -- 172.21.15.105:0/1190546352 wait complete. 2024-03-26T17:48:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:51.991+0000 7ffb207fe740 20 client.19197 trim_cache size 0 max 16384 2024-03-26T17:48:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb207fe740 1 Processor -- start 2024-03-26T17:48:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb207fe740 1 -- start start 2024-03-26T17:48:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e57f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e580260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb207fe740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181a6c20 con 0x55ca0e575e40 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb207fe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181a8760 con 0x55ca0e576830 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb207fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181a72f0 con 0x55ca0e57b9a0 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e57f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e57f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60226/0 (socket says 172.21.15.105:60226) 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb177fe640 1 -- 172.21.15.105:0/146484698 learned_addr learned my addr 172.21.15.105:0/146484698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb16ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.006+0000 7ffb17fff640 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e580260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffb177fe640 1 -- 172.21.15.105:0/146484698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 msgr2=0x55ca0e580260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffb177fe640 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e580260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffb177fe640 1 -- 172.21.15.105:0/146484698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 msgr2=0x55ca0e5877a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffb177fe640 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5877a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffb177fe640 1 -- 172.21.15.105:0/146484698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca0e56cb40 con 0x55ca0e575e40 2024-03-26T17:48:52.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffb17fff640 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e580260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffb16ffd640 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffb177fe640 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e57f880 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800a7d0 tx=0x7ffb0803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61f6b60990a22fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffafd7fa640 1 -- 172.21.15.105:0/146484698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080359f0 con 0x55ca0e575e40 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffafd7fa640 1 -- 172.21.15.105:0/146484698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb08035b90 con 0x55ca0e575e40 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.007+0000 7ffafd7fa640 1 -- 172.21.15.105:0/146484698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080516b0 con 0x55ca0e575e40 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 -- 172.21.15.105:0/146484698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 msgr2=0x55ca0e57f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e57f880 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800a7d0 tx=0x7ffb0803f000 comp rx=0 tx=0).stop 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 -- 172.21.15.105:0/146484698 shutdown_connections 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e580260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e5877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 --2- 172.21.15.105:0/146484698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e57f880 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 -- 172.21.15.105:0/146484698 >> 172.21.15.105:0/146484698 conn(0x55ca0e56cfa0 msgr2=0x55ca0e577c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 -- 172.21.15.105:0/146484698 shutdown_connections 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 -- 172.21.15.105:0/146484698 wait complete. 2024-03-26T17:48:52.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 -- start start 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.008+0000 7ffb207fe740 1 Processor -- start 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e584770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e584cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb207fe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5f0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb207fe740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181a8090 con 0x55ca0e575e40 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb207fe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181a79c0 con 0x55ca0e576830 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb207fe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181a4c00 con 0x55ca0e57b9a0 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb207fe740 10 client.0 _ll_register_callbacks cb 0x7ffb1000fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e584770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e584770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60236/0 (socket says 172.21.15.105:60236) 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb16ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e584cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 -- 172.21.15.105:0/1445958331 learned_addr learned my addr 172.21.15.105:0/1445958331 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb17fff640 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 -- 172.21.15.105:0/1445958331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 msgr2=0x55ca0e5f0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5f0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 -- 172.21.15.105:0/1445958331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 msgr2=0x55ca0e584cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e584cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 -- 172.21.15.105:0/1445958331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca0e5f09b0 con 0x55ca0e575e40 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffaf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffb177fe640 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e584770 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ffb08037040 tx=0x7ffb08005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ba4064e4ce1542 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.009+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1445958331 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08061060 con 0x55ca0e575e40 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1445958331 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7ffb080513f0 con 0x55ca0e575e40 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 1 -- 172.21.15.105:0/1445958331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb08035b90 con 0x55ca0e575e40 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1445958331 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffb080515f0 con 0x55ca0e575e40 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 10 client.19203 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 1 -- 172.21.15.105:0/1445958331 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ca0e5f1cb0 con 0x55ca0e575e40 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1445958331 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb08035b90 con 0x55ca0e575e40 2024-03-26T17:48:52.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1445958331 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ffb0800b040 con 0x55ca0e575e40 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 10 client.19203 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 10 client.19203 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 20 client.19203 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ca0e5f2660 0x55ca0e5f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 4 client.19203 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb207fe740 1 -- 172.21.15.105:0/1445958331 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca0e580050 con 0x55ca0e5f2660 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.010+0000 7ffb16ffd640 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ca0e5f2660 0x55ca0e5f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffaf6ffd640 10 client.19203 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb16ffd640 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ca0e5f2660 0x55ca0e5f4a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7d080ab5420c468 server_cookie=1020adaee25313c8 in_seq=0 out_seq=0 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffaf6ffd640 1 -- 172.21.15.105:0/1445958331 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca0e580050 con 0x55ca0e5f2660 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffaf6ffd640 10 client.19203 handle_command_reply: tid=0 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 client.19203 shutdown 2024-03-26T17:48:52.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 -- 172.21.15.105:0/1445958331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 msgr2=0x55ca0e584770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e584770 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ffb08037040 tx=0x7ffb08005fd0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 -- 172.21.15.105:0/1445958331 shutdown_connections 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ca0e5f2660 0x55ca0e5f4a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca0e57b9a0 0x55ca0e5f0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffaf6ffd640 0 client.19203 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ca0e576830 0x55ca0e584cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 --2- 172.21.15.105:0/1445958331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ca0e575e40 0x55ca0e584770 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 -- 172.21.15.105:0/1445958331 >> 172.21.15.105:0/1445958331 conn(0x55ca0e56cfa0 msgr2=0x55ca0e580200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.011+0000 7ffb207fe740 1 -- 172.21.15.105:0/1445958331 shutdown_connections 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb207fe740 1 -- 172.21.15.105:0/1445958331 wait complete. 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb207fe740 20 client.19203 trim_cache size 0 max 16384 2024-03-26T17:48:52.009 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:52.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 -- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb000a2f20 msgr2=0x7ffb000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb000a2f20 0x7ffb000a53e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffb08005ab0 tx=0x7ffb08005740 comp rx=0 tx=0).stop 2024-03-26T17:48:52.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 -- 172.21.15.105:0/1192866404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb18101170 msgr2=0x7ffb18195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb18101170 0x7ffb18195ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00d990 tx=0x7ffb0c00de50 comp rx=0 tx=0).stop 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 -- 172.21.15.105:0/1192866404 shutdown_connections 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb000a2f20 0x7ffb000a53e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb18101b30 0x7ffb1819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb18101170 0x7ffb18195ef0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 --2- 172.21.15.105:0/1192866404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb180fff80 0x7ffb181959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 -- 172.21.15.105:0/1192866404 >> 172.21.15.105:0/1192866404 conn(0x7ffb180fb4f0 msgr2=0x7ffb18104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 -- 172.21.15.105:0/1192866404 shutdown_connections 2024-03-26T17:48:52.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.012+0000 7ffb1dc12640 1 -- 172.21.15.105:0/1192866404 wait complete. 2024-03-26T17:48:52.015 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:48:52.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 337351 3.15 MB/sec execute 455 sec latency 65.402 ms 2024-03-26T17:48:52.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:51 smithi144 ceph-mon[36095]: pgmap v663: 97 pgs: 97 active+clean; 25 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 -- 172.21.15.105:0/2263641963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 msgr2=0x7f8190104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 --2- 172.21.15.105:0/2263641963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 0x7f8190104790 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8180009900 tx=0x7f818002bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 -- 172.21.15.105:0/2263641963 shutdown_connections 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 --2- 172.21.15.105:0/2263641963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8190107600 0x7f81901099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 --2- 172.21.15.105:0/2263641963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8190104cd0 0x7f81901070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 --2- 172.21.15.105:0/2263641963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 0x7f8190104790 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 -- 172.21.15.105:0/2263641963 >> 172.21.15.105:0/2263641963 conn(0x7f81900fbd50 msgr2=0x7f81900fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 -- 172.21.15.105:0/2263641963 shutdown_connections 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 -- 172.21.15.105:0/2263641963 wait complete. 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.139+0000 7f8195bf3640 1 Processor -- start 2024-03-26T17:48:52.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f8195bf3640 1 -- start start 2024-03-26T17:48:52.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f8195bf3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 0x7f8190101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f8195bf3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8190104cd0 0x7f8190100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f8195bf3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8190107600 0x7f8190100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f8195bf3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f819010d410 con 0x7f81901023a0 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f8195bf3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f819010d290 con 0x7f8190107600 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f8195bf3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f819010d590 con 0x7f8190104cd0 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 0x7f8190101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 0x7f8190101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60246/0 (socket says 172.21.15.105:60246) 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 -- 172.21.15.105:0/591850452 learned_addr learned my addr 172.21.15.105:0/591850452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818ffff640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8190107600 0x7f8190100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 -- 172.21.15.105:0/591850452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8190104cd0 msgr2=0x7f8190100350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8190104cd0 0x7f8190100350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 -- 172.21.15.105:0/591850452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8190107600 msgr2=0x7f8190100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8190107600 0x7f8190100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 -- 172.21.15.105:0/591850452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8190101120 con 0x7f81901023a0 2024-03-26T17:48:52.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818f7fe640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 0x7f8190101d10 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8180006010 tx=0x7f8180005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.140+0000 7f818cff9640 1 -- 172.21.15.105:0/591850452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180043020 con 0x7f81901023a0 2024-03-26T17:48:52.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.141+0000 7f818cff9640 1 -- 172.21.15.105:0/591850452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81800043d0 con 0x7f81901023a0 2024-03-26T17:48:52.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.141+0000 7f8195bf3640 1 -- 172.21.15.105:0/591850452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81901a1cd0 con 0x7f81901023a0 2024-03-26T17:48:52.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.141+0000 7f818cff9640 1 -- 172.21.15.105:0/591850452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180003b80 con 0x7f81901023a0 2024-03-26T17:48:52.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.141+0000 7f8195bf3640 1 -- 172.21.15.105:0/591850452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81901a21d0 con 0x7f81901023a0 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f818cff9640 1 -- 172.21.15.105:0/591850452 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8180046020 con 0x7f81901023a0 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f81987db740 1 Processor -- start 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f81987db740 1 -- start start 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f81987db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59ea30 0x56188e59ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f81987db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59f420 0x56188e5a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f81987db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f81987db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56188e503970 con 0x56188e59ea30 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f81987db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56188e4abd70 con 0x56188e5a4590 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.143+0000 7f81987db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56188e4ff920 con 0x56188e59f420 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59ea30 0x56188e59ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59f420 0x56188e5a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59ea30 0x56188e59ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60254/0 (socket says 172.21.15.105:60254) 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818ffff640 1 -- 172.21.15.105:0/1119953886 learned_addr learned my addr 172.21.15.105:0/1119953886 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818effd640 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818f7fe640 1 -- 172.21.15.105:0/1119953886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 msgr2=0x56188e5a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818f7fe640 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818f7fe640 1 -- 172.21.15.105:0/1119953886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59ea30 msgr2=0x56188e59ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818f7fe640 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59ea30 0x56188e59ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818f7fe640 1 -- 172.21.15.105:0/1119953886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56188e2173f0 con 0x56188e59f420 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818ffff640 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59ea30 0x56188e59ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818cff9640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81780a2f20 0x7f81780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818ffff640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81780a2f20 0x7f81780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818f7fe640 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59f420 0x56188e5a3f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f81800ef200 tx=0x7f81800f7b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5fdbc1ef2a00c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f818cff9640 1 -- 172.21.15.105:0/591850452 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f81800f1770 con 0x7f81901023a0 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.144+0000 7f8175ffb640 1 -- 172.21.15.105:0/1119953886 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800ba100 con 0x56188e59f420 2024-03-26T17:48:52.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f8175ffb640 1 -- 172.21.15.105:0/1119953886 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81800be560 con 0x56188e59f420 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f8175ffb640 1 -- 172.21.15.105:0/1119953886 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800bd7c0 con 0x56188e59f420 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 -- 172.21.15.105:0/1119953886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59f420 msgr2=0x56188e5a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59f420 0x56188e5a3f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f81800ef200 tx=0x7f81800f7b40 comp rx=0 tx=0).stop 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f818ffff640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81780a2f20 0x7f81780a53e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x56188e59fc80 tx=0x7f8188009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 -- 172.21.15.105:0/1119953886 shutdown_connections 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59f420 0x56188e5a3f20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 --2- 172.21.15.105:0/1119953886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59ea30 0x56188e59ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 -- 172.21.15.105:0/1119953886 >> 172.21.15.105:0/1119953886 conn(0x56188e599fa0 msgr2=0x56188e59a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 -- 172.21.15.105:0/1119953886 shutdown_connections 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.145+0000 7f81987db740 1 -- 172.21.15.105:0/1119953886 wait complete. 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 1 -- start start 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 1 Processor -- start 2024-03-26T17:48:52.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5a9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56188e5a7300 con 0x56188e59f420 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56188e4bf310 con 0x56188e5a4590 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56188e5a7480 con 0x56188e59ea30 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f81987db740 10 client.0 _ll_register_callbacks cb 0x56188e4fac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.146+0000 7f818ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35388/0 (socket says 172.21.15.105:35388) 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818f7fe640 1 -- 172.21.15.105:0/3308262297 learned_addr learned my addr 172.21.15.105:0/3308262297 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818effd640 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35000/0 (socket says 172.21.15.105:35000) 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818f7fe640 1 -- 172.21.15.105:0/3308262297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 msgr2=0x56188e5b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818f7fe640 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818f7fe640 1 -- 172.21.15.105:0/3308262297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 msgr2=0x56188e5a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818f7fe640 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818f7fe640 1 -- 172.21.15.105:0/3308262297 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56188e2e17d0 con 0x56188e59ea30 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818ffff640 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818effd640 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f818f7fe640 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8180000c00 tx=0x7f8180004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3da743dc6a41999 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f816effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f816effd640 1 -- 172.21.15.105:0/3308262297 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800bd9a0 con 0x56188e59ea30 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f81987db740 1 -- 172.21.15.105:0/3308262297 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81800be560 con 0x56188e59ea30 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f816effd640 1 -- 172.21.15.105:0/3308262297 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8180003d20 con 0x56188e59ea30 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f816effd640 1 -- 172.21.15.105:0/3308262297 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81800bdc30 con 0x56188e59ea30 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f81987db740 10 client.29029 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.147+0000 7f81987db740 1 -- 172.21.15.105:0/3308262297 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56188e5a7300 con 0x56188e59ea30 2024-03-26T17:48:52.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f816effd640 1 -- 172.21.15.105:0/3308262297 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81800be560 con 0x56188e59ea30 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f816effd640 1 -- 172.21.15.105:0/3308262297 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f81800054b0 con 0x56188e59ea30 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f81987db740 10 client.29029 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f81987db740 10 client.29029 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f81987db740 20 client.29029 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f81987db740 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56188e5b4970 0x56188e5b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f81987db740 4 client.29029 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f81987db740 1 -- 172.21.15.105:0/3308262297 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56188e510eb0 con 0x56188e5b4970 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f818effd640 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56188e5b4970 0x56188e5b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f816effd640 10 client.29029 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.148+0000 7f818effd640 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56188e5b4970 0x56188e5b6dc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62df8076735931c5 server_cookie=d93bdffaeeef9edf in_seq=0 out_seq=0 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f816effd640 1 -- 172.21.15.105:0/3308262297 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56188e510eb0 con 0x56188e5b4970 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f816effd640 10 client.29029 handle_command_reply: tid=0 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 client.29029 shutdown 2024-03-26T17:48:52.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 -- 172.21.15.105:0/3308262297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 msgr2=0x56188e5a8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8180000c00 tx=0x7f8180004610 comp rx=0 tx=0).stop 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 -- 172.21.15.105:0/3308262297 shutdown_connections 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56188e5b4970 0x56188e5b6dc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f816effd640 0 client.29029 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 --2- 172.21.15.105:0/3308262297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8c10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 -- 172.21.15.105:0/3308262297 >> 172.21.15.105:0/3308262297 conn(0x56188e599fa0 msgr2=0x56188e5991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 -- 172.21.15.105:0/3308262297 shutdown_connections 2024-03-26T17:48:52.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.149+0000 7f81987db740 1 -- 172.21.15.105:0/3308262297 wait complete. 2024-03-26T17:48:52.148 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:51 smithi105 ceph-mon[32280]: pgmap v663: 97 pgs: 97 active+clean; 25 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:52.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.150+0000 7f81987db740 20 client.29029 trim_cache size 0 max 16384 2024-03-26T17:48:52.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f81987db740 1 Processor -- start 2024-03-26T17:48:52.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f81987db740 1 -- start start 2024-03-26T17:48:52.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f81987db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f81987db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e59f420 0x56188e5b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f81987db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e5a4590 0x56188e598930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f81987db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f81901aafc0 con 0x56188e5a4590 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f81987db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81901acb00 con 0x56188e59f420 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f81987db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ab690 con 0x56188e59ea30 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f818ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e5a4590 0x56188e598930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f818ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e5a4590 0x56188e598930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60268/0 (socket says 172.21.15.105:60268) 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.164+0000 7f818ffff640 1 -- 172.21.15.105:0/2832863929 learned_addr learned my addr 172.21.15.105:0/2832863929 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818ffff640 1 -- 172.21.15.105:0/2832863929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 msgr2=0x56188e5a8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818effd640 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e59f420 0x56188e5b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818f7fe640 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818ffff640 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818ffff640 1 -- 172.21.15.105:0/2832863929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e59f420 msgr2=0x56188e5b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818ffff640 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e59f420 0x56188e5b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818ffff640 1 -- 172.21.15.105:0/2832863929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56188e595a10 con 0x56188e5a4590 2024-03-26T17:48:52.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818effd640 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e59f420 0x56188e5b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:48:52.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818f7fe640 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:48:52.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f818ffff640 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e5a4590 0x56188e598930 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8188009d20 tx=0x7f8188019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4e11175f5255d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f81757fa640 1 -- 172.21.15.105:0/2832863929 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188004b20 con 0x56188e5a4590 2024-03-26T17:48:52.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f81757fa640 1 -- 172.21.15.105:0/2832863929 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f818801ee50 con 0x56188e5a4590 2024-03-26T17:48:52.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.165+0000 7f81757fa640 1 -- 172.21.15.105:0/2832863929 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188005760 con 0x56188e5a4590 2024-03-26T17:48:52.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 -- 172.21.15.105:0/2832863929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e5a4590 msgr2=0x56188e598930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e5a4590 0x56188e598930 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8188009d20 tx=0x7f8188019000 comp rx=0 tx=0).stop 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 -- 172.21.15.105:0/2832863929 shutdown_connections 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e5a4590 0x56188e598930 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e59f420 0x56188e5b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 --2- 172.21.15.105:0/2832863929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5a8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 -- 172.21.15.105:0/2832863929 >> 172.21.15.105:0/2832863929 conn(0x56188e599fa0 msgr2=0x56188e5a8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 -- 172.21.15.105:0/2832863929 shutdown_connections 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 -- 172.21.15.105:0/2832863929 wait complete. 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.166+0000 7f81987db740 1 -- start start 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f81987db740 1 Processor -- start 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f81987db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f81987db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f81987db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f81987db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ac430 con 0x56188e59f420 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f81987db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81901abd60 con 0x56188e5a4590 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f81987db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81901a8fa0 con 0x56188e59ea30 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f81987db740 10 client.0 _ll_register_callbacks cb 0x7f8188009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60274/0 (socket says 172.21.15.105:60274) 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 -- 172.21.15.105:0/475259754 learned_addr learned my addr 172.21.15.105:0/475259754 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818f7fe640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818ffff640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 -- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 msgr2=0x56188e5ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 -- 172.21.15.105:0/475259754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 msgr2=0x56188e5aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 -- 172.21.15.105:0/475259754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56188e5af340 con 0x56188e59f420 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818ffff640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:48:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818f7fe640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f818effd640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5ade30 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8184000c00 tx=0x7f8184010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187c2ab71dc058aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.167+0000 7f816effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f816effd640 1 -- 172.21.15.105:0/475259754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818401de60 con 0x56188e59f420 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f816effd640 1 -- 172.21.15.105:0/475259754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f818401d630 con 0x56188e59f420 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 1 -- 172.21.15.105:0/475259754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818801ee50 con 0x56188e59f420 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f816effd640 1 -- 172.21.15.105:0/475259754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8184004050 con 0x56188e59f420 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 10 client.19221 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 1 -- 172.21.15.105:0/475259754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56188e61ad20 con 0x56188e59f420 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f816effd640 1 -- 172.21.15.105:0/475259754 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f818801ee50 con 0x56188e59f420 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f816effd640 1 -- 172.21.15.105:0/475259754 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f81840253c0 con 0x56188e59f420 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 10 client.19221 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 10 client.19221 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:48:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 20 client.19221 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56188e61b780 0x56188e61dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 4 client.19221 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.168+0000 7f81987db740 1 -- 172.21.15.105:0/475259754 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56188e50b200 con 0x56188e61b780 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f818f7fe640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56188e61b780 0x56188e61dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f818f7fe640 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56188e61b780 0x56188e61dba0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d2e98dd9fb1cf7d9 server_cookie=e26580ed625445de in_seq=0 out_seq=0 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f816effd640 10 client.19221 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f816effd640 1 -- 172.21.15.105:0/475259754 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56188e50b200 con 0x56188e61b780 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f816effd640 10 client.19221 handle_command_reply: tid=0 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 client.19221 shutdown 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 -- 172.21.15.105:0/475259754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 msgr2=0x56188e5ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5ade30 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8184000c00 tx=0x7f8184010800 comp rx=0 tx=0).stop 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 -- 172.21.15.105:0/475259754 shutdown_connections 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x56188e61b780 0x56188e61dba0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56188e5a4590 0x56188e5aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56188e59f420 0x56188e5ade30 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f816effd640 0 client.19221 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:48:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 --2- 172.21.15.105:0/475259754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56188e59ea30 0x56188e5ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 -- 172.21.15.105:0/475259754 >> 172.21.15.105:0/475259754 conn(0x56188e599fa0 msgr2=0x56188e59d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 -- 172.21.15.105:0/475259754 shutdown_connections 2024-03-26T17:48:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.169+0000 7f81987db740 1 -- 172.21.15.105:0/475259754 wait complete. 2024-03-26T17:48:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f81987db740 20 client.19221 trim_cache size 0 max 16384 2024-03-26T17:48:52.167 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 -- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81780a2f20 msgr2=0x7f81780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81780a2f20 0x7f81780a53e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x56188e59fc80 tx=0x7f8188009500 comp rx=0 tx=0).stop 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 -- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 msgr2=0x7f8190101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 0x7f8190101d10 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8180006010 tx=0x7f8180005e40 comp rx=0 tx=0).stop 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 -- 172.21.15.105:0/591850452 shutdown_connections 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81780a2f20 0x7f81780a53e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8190107600 0x7f8190100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8190104cd0 0x7f8190100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 --2- 172.21.15.105:0/591850452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f81901023a0 0x7f8190101d10 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 -- 172.21.15.105:0/591850452 >> 172.21.15.105:0/591850452 conn(0x7f81900fbd50 msgr2=0x7f8190104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 -- 172.21.15.105:0/591850452 shutdown_connections 2024-03-26T17:48:52.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.170+0000 7f8195bf3640 1 -- 172.21.15.105:0/591850452 wait complete. 2024-03-26T17:48:52.169 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:51 smithi135 ceph-mon[35636]: pgmap v663: 97 pgs: 97 active+clean; 25 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-03-26T17:48:52.173 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:48:52.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/1586491756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f4102ae0 msgr2=0x7fa7f4102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/1586491756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f4102ae0 0x7fa7f4102f60 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009900 tx=0x7fa7e402bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/1586491756 shutdown_connections 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/1586491756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f41034a0 0x7fa7f4109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/1586491756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f4102ae0 0x7fa7f4102f60 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/1586491756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f41018f0 0x7fa7f4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/1586491756 >> 172.21.15.105:0/1586491756 conn(0x7fa7f40fce60 msgr2=0x7fa7f40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/1586491756 shutdown_connections 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.333+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/1586491756 wait complete. 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fc0d3640 1 Processor -- start 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fc0d3640 1 -- start start 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fc0d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f41018f0 0x7fa7f4197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fc0d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f4102ae0 0x7fa7f4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fc0d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f41034a0 0x7fa7f419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fc0d3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f410d420 con 0x7fa7f41018f0 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fc0d3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f410d2a0 con 0x7fa7f41034a0 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fc0d3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f410d5a0 con 0x7fa7f4102ae0 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f9e6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f41018f0 0x7fa7f4197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f9e6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f41018f0 0x7fa7f4197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60288/0 (socket says 172.21.15.105:60288) 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/4119973386 learned_addr learned my addr 172.21.15.105:0/4119973386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f966e640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f4102ae0 0x7fa7f4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7fa670640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f41034a0 0x7fa7f419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/4119973386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f4102ae0 msgr2=0x7fa7f4198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f4102ae0 0x7fa7f4198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/4119973386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f41034a0 msgr2=0x7fa7f419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f41034a0 0x7fa7f419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.334+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/4119973386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f419f760 con 0x7fa7f41018f0 2024-03-26T17:48:52.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.335+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f41018f0 0x7fa7f4197b40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d950 tx=0x7fa7f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.335+0000 7fa7eaffd640 1 -- 172.21.15.105:0/4119973386 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0017070 con 0x7fa7f41018f0 2024-03-26T17:48:52.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.335+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/4119973386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f419fa50 con 0x7fa7f41018f0 2024-03-26T17:48:52.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.335+0000 7fa7eaffd640 1 -- 172.21.15.105:0/4119973386 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f0009c30 con 0x7fa7f41018f0 2024-03-26T17:48:52.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.335+0000 7fa7eaffd640 1 -- 172.21.15.105:0/4119973386 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000f620 con 0x7fa7f41018f0 2024-03-26T17:48:52.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.335+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/4119973386 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f419ff70 con 0x7fa7f41018f0 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.337+0000 7fa7eaffd640 1 -- 172.21.15.105:0/4119973386 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa7f001a070 con 0x7fa7f41018f0 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.338+0000 7fa7eaffd640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7e00a2f20 0x7fa7e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.338+0000 7fa7f966e640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7e00a2f20 0x7fa7e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.338+0000 7fa7eaffd640 1 -- 172.21.15.105:0/4119973386 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa7f00cedd0 con 0x7fa7f41018f0 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.338+0000 7fa7fed95740 1 Processor -- start 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.338+0000 7fa7fed95740 1 -- start start 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7f966e640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7e00a2f20 0x7fa7e00a53e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009230 tx=0x7fa7e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a83248e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a83249420 0x564a8324df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a8324e590 0x564a83250a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fed95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564a831ad970 con 0x564a83248a30 2024-03-26T17:48:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fed95740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564a83155d70 con 0x564a83249420 2024-03-26T17:48:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fed95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564a831a9920 con 0x564a8324e590 2024-03-26T17:48:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7f9e6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a83248e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7f966e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a83249420 0x564a8324df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fa670640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a8324e590 0x564a83250a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7f9e6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a83248e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60302/0 (socket says 172.21.15.105:60302) 2024-03-26T17:48:52.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/3664727176 learned_addr learned my addr 172.21.15.105:0/3664727176 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7f966e640 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a83249420 0x564a8324df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35034/0 (socket says 172.21.15.105:35034) 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fa670640 1 -- 172.21.15.105:0/3664727176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a83249420 msgr2=0x564a8324df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fa670640 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a83249420 0x564a8324df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fa670640 1 -- 172.21.15.105:0/3664727176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 msgr2=0x564a83248e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fa670640 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a83248e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fa670640 1 -- 172.21.15.105:0/3664727176 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a82ec13f0 con 0x564a8324e590 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a83248e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7f966e640 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a83249420 0x564a8324df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.339+0000 7fa7fa670640 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a8324e590 0x564a83250a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009f30 tx=0x7fa7ec010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74db432e7ac3f70d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7dffff640 1 -- 172.21.15.105:0/3664727176 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec016070 con 0x564a8324e590 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7dffff640 1 -- 172.21.15.105:0/3664727176 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7ec004d20 con 0x564a8324e590 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7dffff640 1 -- 172.21.15.105:0/3664727176 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec012740 con 0x564a8324e590 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7fed95740 1 -- 172.21.15.105:0/3664727176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a8324e590 msgr2=0x564a83250a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7fed95740 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a8324e590 0x564a83250a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009f30 tx=0x7fa7ec010800 comp rx=0 tx=0).stop 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7fed95740 1 -- 172.21.15.105:0/3664727176 shutdown_connections 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7fed95740 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a8324e590 0x564a83250a20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7fed95740 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a83249420 0x564a8324df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7fed95740 1 --2- 172.21.15.105:0/3664727176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a83248e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7fed95740 1 -- 172.21.15.105:0/3664727176 >> 172.21.15.105:0/3664727176 conn(0x564a83243fa0 msgr2=0x564a832443a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.340+0000 7fa7fed95740 1 -- 172.21.15.105:0/3664727176 shutdown_connections 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 -- 172.21.15.105:0/3664727176 wait complete. 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 -- start start 2024-03-26T17:48:52.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 Processor -- start 2024-03-26T17:48:52.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a8325b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83249420 0x564a8325bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83262ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564a83251540 con 0x564a83248a30 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564a83169310 con 0x564a8324e590 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564a832516c0 con 0x564a83249420 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fed95740 10 client.0 _ll_register_callbacks cb 0x564a831a4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7f9e6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a8325b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7f9e6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a8325b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60306/0 (socket says 172.21.15.105:60306) 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/1193899886 learned_addr learned my addr 172.21.15.105:0/1193899886 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7fa670640 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83262ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.341+0000 7fa7f966e640 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83249420 0x564a8325bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/1193899886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83249420 msgr2=0x564a8325bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83249420 0x564a8325bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/1193899886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 msgr2=0x564a83262ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83262ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7f9e6f640 1 -- 172.21.15.105:0/1193899886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a82f8b7d0 con 0x564a83248a30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a8325b720 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00cc8e0 tx=0x7fa7f0009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0dcc9c41b24cad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1193899886 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00b4060 con 0x564a83248a30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1193899886 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa7f009f170 con 0x564a83248a30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7fed95740 1 -- 172.21.15.105:0/1193899886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ec004d20 con 0x564a83248a30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1193899886 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7f0021040 con 0x564a83248a30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7fed95740 10 client.19233 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7fed95740 1 -- 172.21.15.105:0/1193899886 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a832660d0 con 0x564a83248a30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.342+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1193899886 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7ec004d20 con 0x564a83248a30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1193899886 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa7f0084b00 con 0x564a83248a30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7fed95740 10 client.19233 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7fed95740 10 client.19233 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:48:52.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7fed95740 20 client.19233 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a83267740 0x564a83269c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7fed95740 4 client.19233 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7fed95740 1 -- 172.21.15.105:0/1193899886 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a831baeb0 con 0x564a83267740 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7f966e640 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a83267740 0x564a83269c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7dd7fa640 10 client.19233 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.343+0000 7fa7f966e640 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a83267740 0x564a83269c00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb68a8db3aabef60 server_cookie=287234fd222174eb in_seq=0 out_seq=0 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.344+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/1193899886 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564a831baeb0 con 0x564a83267740 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.344+0000 7fa7dd7fa640 10 client.19233 handle_command_reply: tid=0 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.344+0000 7fa7fed95740 1 client.19233 shutdown 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.344+0000 7fa7fed95740 1 -- 172.21.15.105:0/1193899886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 msgr2=0x564a8325b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.344+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a8325b720 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00cc8e0 tx=0x7fa7f0009f20 comp rx=0 tx=0).stop 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 1 -- 172.21.15.105:0/1193899886 shutdown_connections 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a83267740 0x564a83269c00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83262ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83249420 0x564a8325bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1193899886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83248a30 0x564a8325b720 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7dd7fa640 0 client.19233 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:48:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 1 -- 172.21.15.105:0/1193899886 >> 172.21.15.105:0/1193899886 conn(0x564a83243fa0 msgr2=0x564a832431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 1 -- 172.21.15.105:0/1193899886 shutdown_connections 2024-03-26T17:48:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 1 -- 172.21.15.105:0/1193899886 wait complete. 2024-03-26T17:48:52.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.345+0000 7fa7fed95740 20 client.19233 trim_cache size 0 max 16384 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fed95740 1 Processor -- start 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fed95740 1 -- start start 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 0x564a8325ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83262c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83242930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fed95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41a6b20 con 0x564a83249420 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fed95740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41a8660 con 0x564a8324e590 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fed95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41a71f0 con 0x564a83248a30 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7f966e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83262c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7f966e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83262c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60308/0 (socket says 172.21.15.105:60308) 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7f966e640 1 -- 172.21.15.105:0/1493711478 learned_addr learned my addr 172.21.15.105:0/1493711478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7fa670640 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83242930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.358+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 0x564a8325ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7f966e640 1 -- 172.21.15.105:0/1493711478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 msgr2=0x564a8325ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7f966e640 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 0x564a8325ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7f966e640 1 -- 172.21.15.105:0/1493711478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 msgr2=0x564a83242930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7f966e640 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83242930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7f966e640 1 -- 172.21.15.105:0/1493711478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a83249800 con 0x564a83249420 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7fa670640 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83242930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7f966e640 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83262c60 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4004150 tx=0x7fa7e4042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d82fc518f14259 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7df7fe640 1 -- 172.21.15.105:0/1493711478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40044c0 con 0x564a83249420 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7df7fe640 1 -- 172.21.15.105:0/1493711478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e4059070 con 0x564a83249420 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.359+0000 7fa7df7fe640 1 -- 172.21.15.105:0/1493711478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40547b0 con 0x564a83249420 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- 172.21.15.105:0/1493711478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 msgr2=0x564a83262c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83262c60 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4004150 tx=0x7fa7e4042000 comp rx=0 tx=0).stop 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- 172.21.15.105:0/1493711478 shutdown_connections 2024-03-26T17:48:52.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a83242930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83262c60 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 --2- 172.21.15.105:0/1493711478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 0x564a8325ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- 172.21.15.105:0/1493711478 >> 172.21.15.105:0/1493711478 conn(0x564a83243fa0 msgr2=0x564a8325b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- 172.21.15.105:0/1493711478 shutdown_connections 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- 172.21.15.105:0/1493711478 wait complete. 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- start start 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 Processor -- start 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 0x564a83260330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83260870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a832617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41a7f90 con 0x564a83249420 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41a78c0 con 0x564a8324e590 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.360+0000 7fa7fed95740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f41a4b00 con 0x564a83248a30 2024-03-26T17:48:52.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7fed95740 10 client.0 _ll_register_callbacks cb 0x7fa7f00860b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83260870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f9e6f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 0x564a83260330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83260870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60314/0 (socket says 172.21.15.105:60314) 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 -- 172.21.15.105:0/2816149993 learned_addr learned my addr 172.21.15.105:0/2816149993 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7fa670640 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a832617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 -- 172.21.15.105:0/2816149993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 msgr2=0x564a83260330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 0x564a83260330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 -- 172.21.15.105:0/2816149993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 msgr2=0x564a832617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a832617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 -- 172.21.15.105:0/2816149993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a83261dd0 con 0x564a83249420 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7f966e640 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83260870 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4000c00 tx=0x7fa7e402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da0ef50aafd6986 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:52.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/2816149993 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4054310 con 0x564a83249420 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/2816149993 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa7e403a040 con 0x564a83249420 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7fed95740 1 -- 172.21.15.105:0/2816149993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e4059070 con 0x564a83249420 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/2816149993 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7e406c050 con 0x564a83249420 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7fed95740 10 client.19239 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.361+0000 7fa7fed95740 1 -- 172.21.15.105:0/2816149993 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564a832cdb00 con 0x564a83249420 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/2816149993 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e4059070 con 0x564a83249420 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/2816149993 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa7e4004a60 con 0x564a83249420 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7fed95740 10 client.19239 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7fed95740 10 client.19239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7fed95740 20 client.19239 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7fed95740 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a832ce4c0 0x564a832d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7fed95740 4 client.19239 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7fed95740 1 -- 172.21.15.105:0/2816149993 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a831b5200 con 0x564a832ce4c0 2024-03-26T17:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a832ce4c0 0x564a832d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7f9e6f640 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a832ce4c0 0x564a832d08e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=720bd5fa05ec857e server_cookie=848b44ee148a0a4c in_seq=0 out_seq=0 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7dd7fa640 10 client.19239 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7dd7fa640 1 -- 172.21.15.105:0/2816149993 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a831b5200 con 0x564a832ce4c0 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7dd7fa640 10 client.19239 handle_command_reply: tid=0 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.362+0000 7fa7fed95740 1 client.19239 shutdown 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 -- 172.21.15.105:0/2816149993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 msgr2=0x564a83260870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83260870 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4000c00 tx=0x7fa7e402c940 comp rx=0 tx=0).stop 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 -- 172.21.15.105:0/2816149993 shutdown_connections 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x564a832ce4c0 0x564a832d08e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a8324e590 0x564a832617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564a83249420 0x564a83260870 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 --2- 172.21.15.105:0/2816149993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564a83248a30 0x564a83260330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7dd7fa640 0 client.19239 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 -- 172.21.15.105:0/2816149993 >> 172.21.15.105:0/2816149993 conn(0x564a83243fa0 msgr2=0x564a83247200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 -- 172.21.15.105:0/2816149993 shutdown_connections 2024-03-26T17:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 1 -- 172.21.15.105:0/2816149993 wait complete. 2024-03-26T17:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fed95740 20 client.19239 trim_cache size 0 max 16384 2024-03-26T17:48:52.361 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7e00a2f20 msgr2=0x7fa7e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7e00a2f20 0x7fa7e00a53e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009230 tx=0x7fa7e4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f41018f0 msgr2=0x7fa7f4197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f41018f0 0x7fa7f4197b40 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d950 tx=0x7fa7f000de10 comp rx=0 tx=0).stop 2024-03-26T17:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/4119973386 shutdown_connections 2024-03-26T17:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7e00a2f20 0x7fa7e00a53e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7f41034a0 0x7fa7f419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f4102ae0 0x7fa7f4198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.363+0000 7fa7fc0d3640 1 --2- 172.21.15.105:0/4119973386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7f41018f0 0x7fa7f4197b40 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.364+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/4119973386 >> 172.21.15.105:0/4119973386 conn(0x7fa7f40fce60 msgr2=0x7fa7f4105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.364+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/4119973386 shutdown_connections 2024-03-26T17:48:52.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.364+0000 7fa7fc0d3640 1 -- 172.21.15.105:0/4119973386 wait complete. 2024-03-26T17:48:52.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:48:52.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 -- 172.21.15.105:0/4258443783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 msgr2=0x7fa7d0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 --2- 172.21.15.105:0/4258443783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 0x7fa7d0108c00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009ca0 tx=0x7fa7c802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 -- 172.21.15.105:0/4258443783 shutdown_connections 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 --2- 172.21.15.105:0/4258443783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 0x7fa7d0108c00 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 --2- 172.21.15.105:0/4258443783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7d01019b0 0x7fa7d0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 --2- 172.21.15.105:0/4258443783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d01007c0 0x7fa7d0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 -- 172.21.15.105:0/4258443783 >> 172.21.15.105:0/4258443783 conn(0x7fa7d00fbd50 msgr2=0x7fa7d00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 -- 172.21.15.105:0/4258443783 shutdown_connections 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 -- 172.21.15.105:0/4258443783 wait complete. 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.502+0000 7fa7d603d640 1 Processor -- start 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7d603d640 1 -- start start 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7d603d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7d01007c0 0x7fa7d01969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7d603d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d01019b0 0x7fa7d0196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7d603d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 0x7fa7d019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7d603d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d010c2d0 con 0x7fa7d0102370 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7d603d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d010c150 con 0x7fa7d01007c0 2024-03-26T17:48:52.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7d603d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d010c450 con 0x7fa7d01019b0 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 0x7fa7d019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 0x7fa7d019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60326/0 (socket says 172.21.15.105:60326) 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 -- 172.21.15.105:0/1413706917 learned_addr learned my addr 172.21.15.105:0/1413706917 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7d01007c0 0x7fa7d01969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d01019b0 0x7fa7d0196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 -- 172.21.15.105:0/1413706917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d01019b0 msgr2=0x7fa7d0196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d01019b0 0x7fa7d0196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 -- 172.21.15.105:0/1413706917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7d01007c0 msgr2=0x7fa7d01969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7d01007c0 0x7fa7d01969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 -- 172.21.15.105:0/1413706917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d019e5f0 con 0x7fa7d0102370 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.503+0000 7fa7cffff640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 0x7fa7d019df80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa7c802c700 tx=0x7fa7c80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.504+0000 7fa7ccff9640 1 -- 172.21.15.105:0/1413706917 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8041020 con 0x7fa7d0102370 2024-03-26T17:48:52.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.504+0000 7fa7d603d640 1 -- 172.21.15.105:0/1413706917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d019e880 con 0x7fa7d0102370 2024-03-26T17:48:52.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.504+0000 7fa7ccff9640 1 -- 172.21.15.105:0/1413706917 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c8003e20 con 0x7fa7d0102370 2024-03-26T17:48:52.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.504+0000 7fa7ccff9640 1 -- 172.21.15.105:0/1413706917 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8039040 con 0x7fa7d0102370 2024-03-26T17:48:52.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.504+0000 7fa7d603d640 1 -- 172.21.15.105:0/1413706917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d019edc0 con 0x7fa7d0102370 2024-03-26T17:48:52.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.506+0000 7fa7ccff9640 1 -- 172.21.15.105:0/1413706917 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa7c8044020 con 0x7fa7d0102370 2024-03-26T17:48:52.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7d8c21740 1 Processor -- start 2024-03-26T17:48:52.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7d8c21740 1 -- start start 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc6c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc6c630 0x557a7dc71130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc717a0 0x557a7dc73c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557a7dbce970 con 0x557a7dc6c630 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a7db76d70 con 0x557a7dc717a0 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a7dbca920 con 0x557a7dc6bc40 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7ccff9640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7b80a2f20 0x7fa7b80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc6c630 0x557a7dc71130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc6c630 0x557a7dc71130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60336/0 (socket says 172.21.15.105:60336) 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/3215605842 learned_addr learned my addr 172.21.15.105:0/3215605842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc717a0 0x557a7dc73c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc6c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.507+0000 7fa7cffff640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7b80a2f20 0x7fa7b80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7ccff9640 1 -- 172.21.15.105:0/1413706917 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa7c80f1620 con 0x7fa7d0102370 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/3215605842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 msgr2=0x557a7dc6c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc6c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/3215605842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc717a0 msgr2=0x557a7dc73c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc717a0 0x557a7dc73c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/3215605842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7d8e23f0 con 0x557a7dc6c630 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc6c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc6c630 0x557a7dc71130 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009ca0 tx=0x7fa7c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5070270ea6cb21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7cffff640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7b80a2f20 0x7fa7b80a53e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80ba170 tx=0x7fa7c80e89d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7b5ffb640 1 -- 172.21.15.105:0/3215605842 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0004220 con 0x557a7dc6c630 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7b5ffb640 1 -- 172.21.15.105:0/3215605842 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c0009e50 con 0x557a7dc6c630 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.508+0000 7fa7b5ffb640 1 -- 172.21.15.105:0/3215605842 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00166b0 con 0x557a7dc6c630 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3215605842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc6c630 msgr2=0x557a7dc71130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc6c630 0x557a7dc71130 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009ca0 tx=0x7fa7c0011810 comp rx=0 tx=0).stop 2024-03-26T17:48:52.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3215605842 shutdown_connections 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc717a0 0x557a7dc73c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc6c630 0x557a7dc71130 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/3215605842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc6c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3215605842 >> 172.21.15.105:0/3215605842 conn(0x557a7dc671b0 msgr2=0x557a7dc675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3215605842 shutdown_connections 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3215605842 wait complete. 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 -- start start 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.509+0000 7fa7d8c21740 1 Processor -- start 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc76710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6c630 0x557a7dc78c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc7b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557a7dc64c70 con 0x557a7dc717a0 2024-03-26T17:48:52.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a7db8a310 con 0x557a7dc6c630 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a7dc64df0 con 0x557a7dc6bc40 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7d8c21740 10 client.0 _ll_register_callbacks cb 0x557a7dbc5c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc76710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc76710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35478/0 (socket says 172.21.15.105:35478) 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 -- 172.21.15.105:0/1548550316 learned_addr learned my addr 172.21.15.105:0/1548550316 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc7b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6c630 0x557a7dc78c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 -- 172.21.15.105:0/1548550316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6c630 msgr2=0x557a7dc78c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6c630 0x557a7dc78c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 -- 172.21.15.105:0/1548550316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 msgr2=0x557a7dc7b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc7b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 -- 172.21.15.105:0/1548550316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7d9ac7d0 con 0x557a7dc6bc40 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc76710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4010ed0 tx=0x7fa7c4009b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bae4bd83931bdf68 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.510+0000 7fa7aeffd640 1 -- 172.21.15.105:0/1548550316 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4027060 con 0x557a7dc6bc40 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7aeffd640 1 -- 172.21.15.105:0/1548550316 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa7c40207e0 con 0x557a7dc6bc40 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 1 -- 172.21.15.105:0/1548550316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c0009e50 con 0x557a7dc6bc40 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7aeffd640 1 -- 172.21.15.105:0/1548550316 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7c4019040 con 0x557a7dc6bc40 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 10 client.29062 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 1 -- 172.21.15.105:0/1548550316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a7dc7e920 con 0x557a7dc6bc40 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7aeffd640 1 -- 172.21.15.105:0/1548550316 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c0009e50 con 0x557a7dc6bc40 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7aeffd640 1 -- 172.21.15.105:0/1548550316 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa7c401f040 con 0x557a7dc6bc40 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 10 client.29062 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 10 client.29062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 20 client.29062 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557a7dc7ff80 0x557a7dc82400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 4 client.29062 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.511+0000 7fa7d8c21740 1 -- 172.21.15.105:0/1548550316 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a7dbdbeb0 con 0x557a7dc7ff80 2024-03-26T17:48:52.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.512+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557a7dc7ff80 0x557a7dc82400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.512+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557a7dc7ff80 0x557a7dc82400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4353e0b562697dd server_cookie=e64e10f385413f42 in_seq=0 out_seq=0 2024-03-26T17:48:52.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.512+0000 7fa7aeffd640 10 client.29062 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7aeffd640 1 -- 172.21.15.105:0/1548550316 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557a7dbdbeb0 con 0x557a7dc7ff80 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7aeffd640 10 client.29062 handle_command_reply: tid=0 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 client.29062 shutdown 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 -- 172.21.15.105:0/1548550316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 msgr2=0x557a7dc76710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc76710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4010ed0 tx=0x7fa7c4009b40 comp rx=0 tx=0).stop 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 -- 172.21.15.105:0/1548550316 shutdown_connections 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557a7dc7ff80 0x557a7dc82400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc7b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6c630 0x557a7dc78c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/1548550316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6bc40 0x557a7dc76710 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 -- 172.21.15.105:0/1548550316 >> 172.21.15.105:0/1548550316 conn(0x557a7dc671b0 msgr2=0x557a7dc641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7aeffd640 0 client.29062 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:48:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 -- 172.21.15.105:0/1548550316 shutdown_connections 2024-03-26T17:48:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 1 -- 172.21.15.105:0/1548550316 wait complete. 2024-03-26T17:48:52.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.513+0000 7fa7d8c21740 20 client.29062 trim_cache size 0 max 16384 2024-03-26T17:48:52.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.529+0000 7fa7d8c21740 1 Processor -- start 2024-03-26T17:48:52.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.529+0000 7fa7d8c21740 1 -- start start 2024-03-26T17:48:52.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc78c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc651f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc65730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01a6bb0 con 0x557a7dc717a0 2024-03-26T17:48:52.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01a86f0 con 0x557a7dc6bc40 2024-03-26T17:48:52.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01a7280 con 0x557a7dc6c630 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7ceffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc65730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7ceffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc78c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7cf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7ceffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc78c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35080/0 (socket says 172.21.15.105:35080) 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7cf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35484/0 (socket says 172.21.15.105:35484) 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7ceffd640 1 -- 172.21.15.105:0/3652429877 learned_addr learned my addr 172.21.15.105:0/3652429877 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/3652429877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 msgr2=0x557a7dc78c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/3652429877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc78c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/3652429877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 msgr2=0x557a7dc65730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/3652429877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc65730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/3652429877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7dc67f20 con 0x557a7dc6c630 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/3652429877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc78c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.530+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/3652429877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc651f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x557a7dc68120 tx=0x7fa7c0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee43fb2ba931a39c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7b57fa640 1 -- 172.21.15.105:0/3652429877 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0009e50 con 0x557a7dc6c630 2024-03-26T17:48:52.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7b57fa640 1 -- 172.21.15.105:0/3652429877 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c00043c0 con 0x557a7dc6c630 2024-03-26T17:48:52.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7b57fa640 1 -- 172.21.15.105:0/3652429877 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0004a00 con 0x557a7dc6c630 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3652429877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 msgr2=0x557a7dc651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/3652429877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc651f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x557a7dc68120 tx=0x7fa7c0005f90 comp rx=0 tx=0).stop 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3652429877 shutdown_connections 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/3652429877 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc65730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/3652429877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc651f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/3652429877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc78c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3652429877 >> 172.21.15.105:0/3652429877 conn(0x557a7dc671b0 msgr2=0x557a7dc78530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3652429877 shutdown_connections 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.531+0000 7fa7d8c21740 1 -- 172.21.15.105:0/3652429877 wait complete. 2024-03-26T17:48:52.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 1 -- start start 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 1 Processor -- start 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc92060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc925a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc92ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01a8020 con 0x557a7dc717a0 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01a7950 con 0x557a7dc6bc40 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01a4b90 con 0x557a7dc6c630 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7d8c21740 10 client.0 _ll_register_callbacks cb 0x7fa7c4010d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7cf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.532+0000 7fa7ceffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc92060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7cf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc925a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35500/0 (socket says 172.21.15.105:35500) 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/2062675238 learned_addr learned my addr 172.21.15.105:0/2062675238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc92ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/2062675238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 msgr2=0x557a7dc92060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc92060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/2062675238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 msgr2=0x557a7dc92ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc92ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7cf7fe640 1 -- 172.21.15.105:0/2062675238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7dc93020 con 0x557a7dc6c630 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7cf7fe640 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc925a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0011850 tx=0x7fa7c0009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc5599a33e60e8bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7aeffd640 1 -- 172.21.15.105:0/2062675238 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c003b020 con 0x557a7dc6c630 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7aeffd640 1 -- 172.21.15.105:0/2062675238 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fa7c0005220 con 0x557a7dc6c630 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7d8c21740 1 -- 172.21.15.105:0/2062675238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c00043c0 con 0x557a7dc6c630 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7aeffd640 1 -- 172.21.15.105:0/2062675238 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7c0046050 con 0x557a7dc6c630 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7d8c21740 10 client.29068 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.533+0000 7fa7d8c21740 1 -- 172.21.15.105:0/2062675238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557a7dce6540 con 0x557a7dc6c630 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7aeffd640 1 -- 172.21.15.105:0/2062675238 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c00043c0 con 0x557a7dc6c630 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7aeffd640 1 -- 172.21.15.105:0/2062675238 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fa7c0032070 con 0x557a7dc6c630 2024-03-26T17:48:52.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7d8c21740 10 client.29068 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7d8c21740 10 client.29068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7d8c21740 20 client.29068 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557a7dce6da0 0x557a7dce91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7d8c21740 4 client.29068 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7d8c21740 1 -- 172.21.15.105:0/2062675238 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a7dbd6200 con 0x557a7dce6da0 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557a7dce6da0 0x557a7dce91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7aeffd640 10 client.29068 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7ceffd640 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557a7dce6da0 0x557a7dce91c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3eaf1765b796b915 server_cookie=9b97b6639c32500d in_seq=0 out_seq=0 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7aeffd640 1 -- 172.21.15.105:0/2062675238 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a7dbd6200 con 0x557a7dce6da0 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7aeffd640 10 client.29068 handle_command_reply: tid=0 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.534+0000 7fa7d8c21740 1 client.29068 shutdown 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 -- 172.21.15.105:0/2062675238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 msgr2=0x557a7dc925a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc925a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0011850 tx=0x7fa7c0009ec0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 -- 172.21.15.105:0/2062675238 shutdown_connections 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557a7dce6da0 0x557a7dce91c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557a7dc717a0 0x557a7dc92ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7dc6c630 0x557a7dc925a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7aeffd640 0 client.29068 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 --2- 172.21.15.105:0/2062675238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a7dc6bc40 0x557a7dc92060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 -- 172.21.15.105:0/2062675238 >> 172.21.15.105:0/2062675238 conn(0x557a7dc671b0 msgr2=0x557a7dc63930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 -- 172.21.15.105:0/2062675238 shutdown_connections 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 1 -- 172.21.15.105:0/2062675238 wait complete. 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d8c21740 20 client.29068 trim_cache size 0 max 16384 2024-03-26T17:48:52.532 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:52.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d603d640 1 -- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7b80a2f20 msgr2=0x7fa7b80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d603d640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7b80a2f20 0x7fa7b80a53e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80ba170 tx=0x7fa7c80e89d0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d603d640 1 -- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 msgr2=0x7fa7d019df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.535+0000 7fa7d603d640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 0x7fa7d019df80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa7c802c700 tx=0x7fa7c80328a0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.536+0000 7fa7d603d640 1 -- 172.21.15.105:0/1413706917 shutdown_connections 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.536+0000 7fa7d603d640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa7b80a2f20 0x7fa7b80a53e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.536+0000 7fa7d603d640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7d0102370 0x7fa7d019df80 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.536+0000 7fa7d603d640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d01019b0 0x7fa7d0196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.536+0000 7fa7d603d640 1 --2- 172.21.15.105:0/1413706917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7d01007c0 0x7fa7d01969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.536+0000 7fa7d603d640 1 -- 172.21.15.105:0/1413706917 >> 172.21.15.105:0/1413706917 conn(0x7fa7d00fbd50 msgr2=0x7fa7d0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.536+0000 7fa7d603d640 1 -- 172.21.15.105:0/1413706917 shutdown_connections 2024-03-26T17:48:52.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.536+0000 7fa7d603d640 1 -- 172.21.15.105:0/1413706917 wait complete. 2024-03-26T17:48:52.540 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 -- 172.21.15.105:0/1903372209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76141007e0 msgr2=0x7f7614100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 --2- 172.21.15.105:0/1903372209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76141007e0 0x7f7614100c00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f7610009900 tx=0x7f761002bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 -- 172.21.15.105:0/1903372209 shutdown_connections 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 --2- 172.21.15.105:0/1903372209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7614102390 0x7f7614108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 --2- 172.21.15.105:0/1903372209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76141019d0 0x7f7614101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 --2- 172.21.15.105:0/1903372209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76141007e0 0x7f7614100c00 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 -- 172.21.15.105:0/1903372209 >> 172.21.15.105:0/1903372209 conn(0x7f76140fbd70 msgr2=0x7f76140fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 -- 172.21.15.105:0/1903372209 shutdown_connections 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.666+0000 7f761af62640 1 -- 172.21.15.105:0/1903372209 wait complete. 2024-03-26T17:48:52.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f761af62640 1 Processor -- start 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f761af62640 1 -- start start 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f761af62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76141007e0 0x7f761406a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f761af62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76141019d0 0x7f761419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f761af62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7614102390 0x7f761419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f761af62640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f761410c2e0 con 0x7f7614102390 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f761af62640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f761410c160 con 0x7f76141019d0 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f761af62640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f761410c460 con 0x7f76141007e0 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f76194ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7614102390 0x7f761419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f76194ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7614102390 0x7f761419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60376/0 (socket says 172.21.15.105:60376) 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f76194ff640 1 -- 172.21.15.105:0/3661213068 learned_addr learned my addr 172.21.15.105:0/3661213068 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f760bfff640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76141019d0 0x7f761419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.667+0000 7f7618cfe640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76141007e0 0x7f761406a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f76194ff640 1 -- 172.21.15.105:0/3661213068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76141007e0 msgr2=0x7f761406a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f76194ff640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76141007e0 0x7f761406a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f76194ff640 1 -- 172.21.15.105:0/3661213068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76141019d0 msgr2=0x7f761419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f76194ff640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76141019d0 0x7f761419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f76194ff640 1 -- 172.21.15.105:0/3661213068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761419f820 con 0x7f7614102390 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f760bfff640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76141019d0 0x7f761419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f76194ff640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7614102390 0x7f761419ef10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f760c00efd0 tx=0x7f760c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f7609ffb640 1 -- 172.21.15.105:0/3661213068 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c003b70 con 0x7f7614102390 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f7609ffb640 1 -- 172.21.15.105:0/3661213068 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f760c003d10 con 0x7f7614102390 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f7609ffb640 1 -- 172.21.15.105:0/3661213068 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c010980 con 0x7f7614102390 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f761af62640 1 -- 172.21.15.105:0/3661213068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761419fab0 con 0x7f7614102390 2024-03-26T17:48:52.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.668+0000 7f761af62640 1 -- 172.21.15.105:0/3661213068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76141a0000 con 0x7f7614102390 2024-03-26T17:48:52.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.671+0000 7f761dc2e740 1 Processor -- start 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.671+0000 7f7609ffb640 1 -- 172.21.15.105:0/3661213068 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f760c004020 con 0x7f7614102390 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.671+0000 7f761dc2e740 1 -- start start 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c2995ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c29963f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c29964590 0x557c29966a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f761dc2e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557c298c3970 con 0x557c29964590 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f761dc2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557c2986bd70 con 0x557c2995ea30 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f761dc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557c298bf920 con 0x557c2995f420 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c2995ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c2995ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35116/0 (socket says 172.21.15.105:35116) 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 -- 172.21.15.105:0/4246016733 learned_addr learned my addr 172.21.15.105:0/4246016733 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c29964590 0x557c29966a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f7618cfe640 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c29963f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 -- 172.21.15.105:0/4246016733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 msgr2=0x557c29963f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c29963f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 -- 172.21.15.105:0/4246016733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c29964590 msgr2=0x557c29966a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c29964590 0x557c29966a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 -- 172.21.15.105:0/4246016733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c295d73f0 con 0x557c2995ea30 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f7609ffb640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75fc0a2fa0 0x7f75fc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f7618cfe640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75fc0a2fa0 0x7f75fc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f7609ffb640 1 -- 172.21.15.105:0/3661213068 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f760c0cf800 con 0x7f7614102390 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f760bfff640 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c2995ee50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7604011be0 tx=0x7f7604009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b61a3fe013c674 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.672+0000 7f75faffd640 1 -- 172.21.15.105:0/4246016733 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760400fc50 con 0x557c2995ea30 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.673+0000 7f75faffd640 1 -- 172.21.15.105:0/4246016733 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f760400fdf0 con 0x557c2995ea30 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.673+0000 7f75faffd640 1 -- 172.21.15.105:0/4246016733 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760400e640 con 0x557c2995ea30 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.673+0000 7f7618cfe640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75fc0a2fa0 0x7f75fc0a5460 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7610005ab0 tx=0x7f7610005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.673+0000 7f761dc2e740 1 -- 172.21.15.105:0/4246016733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 msgr2=0x557c2995ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.673+0000 7f761dc2e740 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c2995ee50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7604011be0 tx=0x7f7604009ba0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.673+0000 7f761dc2e740 1 -- 172.21.15.105:0/4246016733 shutdown_connections 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.673+0000 7f761dc2e740 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c29964590 0x557c29966a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c29963f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 --2- 172.21.15.105:0/4246016733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c2995ee50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 -- 172.21.15.105:0/4246016733 >> 172.21.15.105:0/4246016733 conn(0x557c29959fa0 msgr2=0x557c2995a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 -- 172.21.15.105:0/4246016733 shutdown_connections 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 -- 172.21.15.105:0/4246016733 wait complete. 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 -- start start 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 Processor -- start 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c29968fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c29969510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c29970550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557c29967540 con 0x557c2995ea30 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557c2987f310 con 0x557c29964590 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557c299676c0 con 0x557c2995f420 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f761dc2e740 10 client.0 _ll_register_callbacks cb 0x557c298bac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f7618cfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c29968fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f76194ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c29970550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f7618cfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c29968fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60394/0 (socket says 172.21.15.105:60394) 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.674+0000 7f7618cfe640 1 -- 172.21.15.105:0/584778761 learned_addr learned my addr 172.21.15.105:0/584778761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f76194ff640 1 -- 172.21.15.105:0/584778761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 msgr2=0x557c29969510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f76194ff640 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c29969510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f76194ff640 1 -- 172.21.15.105:0/584778761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 msgr2=0x557c29968fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f76194ff640 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c29968fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f76194ff640 1 -- 172.21.15.105:0/584778761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c296a17d0 con 0x557c29964590 2024-03-26T17:48:52.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f7618cfe640 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c29968fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f76194ff640 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c29970550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f760c005c40 tx=0x7f760c09bb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b88979f5b14ddb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f75f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f75f3fff640 1 -- 172.21.15.105:0/584778761 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c09c5f0 con 0x557c29964590 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f75f3fff640 1 -- 172.21.15.105:0/584778761 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f760c004310 con 0x557c29964590 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f761dc2e740 1 -- 172.21.15.105:0/584778761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760400fdf0 con 0x557c29964590 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f75f3fff640 1 -- 172.21.15.105:0/584778761 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f760c09bef0 con 0x557c29964590 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f761dc2e740 10 client.29081 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.675+0000 7f761dc2e740 1 -- 172.21.15.105:0/584778761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557c29973930 con 0x557c29964590 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f75f3fff640 1 -- 172.21.15.105:0/584778761 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f760400fdf0 con 0x557c29964590 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f75f3fff640 1 -- 172.21.15.105:0/584778761 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f760c01f020 con 0x557c29964590 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f761dc2e740 10 client.29081 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f761dc2e740 10 client.29081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:52.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f761dc2e740 20 client.29081 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f761dc2e740 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557c29975100 0x557c299774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f761dc2e740 4 client.29081 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f761dc2e740 1 -- 172.21.15.105:0/584778761 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c298d0eb0 con 0x557c29975100 2024-03-26T17:48:52.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.676+0000 7f7618cfe640 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557c29975100 0x557c299774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.677+0000 7f75f3fff640 10 client.29081 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.677+0000 7f7618cfe640 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557c29975100 0x557c299774f0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54368b2540f9c652 server_cookie=c989f4df5a328653 in_seq=0 out_seq=0 2024-03-26T17:48:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f75f3fff640 1 -- 172.21.15.105:0/584778761 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557c298d0eb0 con 0x557c29975100 2024-03-26T17:48:52.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f75f3fff640 10 client.29081 handle_command_reply: tid=0 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 client.29081 shutdown 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 -- 172.21.15.105:0/584778761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 msgr2=0x557c29970550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c29970550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f760c005c40 tx=0x7f760c09bb60 comp rx=0 tx=0).stop 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 -- 172.21.15.105:0/584778761 shutdown_connections 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557c29975100 0x557c299774f0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c29970550 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c29969510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f75f3fff640 0 client.29081 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 --2- 172.21.15.105:0/584778761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c29968fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 -- 172.21.15.105:0/584778761 >> 172.21.15.105:0/584778761 conn(0x557c29959fa0 msgr2=0x557c299591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 -- 172.21.15.105:0/584778761 shutdown_connections 2024-03-26T17:48:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 1 -- 172.21.15.105:0/584778761 wait complete. 2024-03-26T17:48:52.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.678+0000 7f761dc2e740 20 client.29081 trim_cache size 0 max 16384 2024-03-26T17:48:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f761dc2e740 1 Processor -- start 2024-03-26T17:48:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f761dc2e740 1 -- start start 2024-03-26T17:48:52.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c299609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995f420 0x557c29970510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c29964590 0x557c29960ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f761dc2e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76141a8d80 con 0x557c2995f420 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f761dc2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76141aa8c0 con 0x557c2995ea30 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f761dc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76141a9450 con 0x557c29964590 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f760bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995f420 0x557c29970510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f7618cfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c299609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f760bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995f420 0x557c29970510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60406/0 (socket says 172.21.15.105:60406) 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f760bfff640 1 -- 172.21.15.105:0/3402871239 learned_addr learned my addr 172.21.15.105:0/3402871239 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f7618cfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c299609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35140/0 (socket says 172.21.15.105:35140) 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f76194ff640 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c29964590 0x557c29960ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f760bfff640 1 -- 172.21.15.105:0/3402871239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c29964590 msgr2=0x557c29960ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f760bfff640 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c29964590 0x557c29960ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f760bfff640 1 -- 172.21.15.105:0/3402871239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 msgr2=0x557c299609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.694+0000 7f760bfff640 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c299609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.695+0000 7f760bfff640 1 -- 172.21.15.105:0/3402871239 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c29969860 con 0x557c2995f420 2024-03-26T17:48:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.695+0000 7f76194ff640 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c29964590 0x557c29960ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:48:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.695+0000 7f7618cfe640 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c299609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.695+0000 7f760bfff640 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995f420 0x557c29970510 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7604009b70 tx=0x7f7604005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaa0d608e034fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.695+0000 7f75fa7fc640 1 -- 172.21.15.105:0/3402871239 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760400fe20 con 0x557c2995f420 2024-03-26T17:48:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.695+0000 7f75fa7fc640 1 -- 172.21.15.105:0/3402871239 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7604023070 con 0x557c2995f420 2024-03-26T17:48:52.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.695+0000 7f75fa7fc640 1 -- 172.21.15.105:0/3402871239 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604003c30 con 0x557c2995f420 2024-03-26T17:48:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- 172.21.15.105:0/3402871239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995f420 msgr2=0x557c29970510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995f420 0x557c29970510 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7604009b70 tx=0x7f7604005f90 comp rx=0 tx=0).stop 2024-03-26T17:48:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- 172.21.15.105:0/3402871239 shutdown_connections 2024-03-26T17:48:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c29964590 0x557c29960ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995f420 0x557c29970510 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 --2- 172.21.15.105:0/3402871239 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c2995ea30 0x557c299609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- 172.21.15.105:0/3402871239 >> 172.21.15.105:0/3402871239 conn(0x557c29959fa0 msgr2=0x557c2995a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- 172.21.15.105:0/3402871239 shutdown_connections 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- 172.21.15.105:0/3402871239 wait complete. 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- start start 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 Processor -- start 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c2996d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c2996dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c2996ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76141aa1f0 con 0x557c2995ea30 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76141a9b20 con 0x557c29964590 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76141a6d60 con 0x557c2995f420 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.696+0000 7f761dc2e740 10 client.0 _ll_register_callbacks cb 0x7f760c0047e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c2996d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c2996d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60410/0 (socket says 172.21.15.105:60410) 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 -- 172.21.15.105:0/2956556059 learned_addr learned my addr 172.21.15.105:0/2956556059 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f76194ff640 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c2996ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f760bfff640 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c2996dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 -- 172.21.15.105:0/2956556059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 msgr2=0x557c2996dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c2996dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 -- 172.21.15.105:0/2956556059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 msgr2=0x557c2996ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c2996ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 -- 172.21.15.105:0/2956556059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c2996f230 con 0x557c2995ea30 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f7618cfe640 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c2996d6f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f76100096c0 tx=0x7f7610009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f0c7ab65c6ce21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f75f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f75f3fff640 1 -- 172.21.15.105:0/2956556059 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761005a070 con 0x557c2995ea30 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f75f3fff640 1 -- 172.21.15.105:0/2956556059 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f7610056b10 con 0x557c2995ea30 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f761dc2e740 1 -- 172.21.15.105:0/2956556059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7604023070 con 0x557c2995ea30 2024-03-26T17:48:52.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f75f3fff640 1 -- 172.21.15.105:0/2956556059 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7610066050 con 0x557c2995ea30 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f761dc2e740 10 client.19275 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.697+0000 7f761dc2e740 1 -- 172.21.15.105:0/2956556059 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557c29970510 con 0x557c2995ea30 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f75f3fff640 1 -- 172.21.15.105:0/2956556059 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7604023070 con 0x557c2995ea30 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f75f3fff640 1 -- 172.21.15.105:0/2956556059 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f76100048f0 con 0x557c2995ea30 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f761dc2e740 10 client.19275 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f761dc2e740 10 client.19275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f761dc2e740 20 client.19275 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f761dc2e740 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557c29975100 0x557c299dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f761dc2e740 4 client.19275 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f761dc2e740 1 -- 172.21.15.105:0/2956556059 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c298cb200 con 0x557c29975100 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.698+0000 7f760bfff640 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557c29975100 0x557c299dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f760bfff640 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557c29975100 0x557c299dd890 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49eaf26a982bf36 server_cookie=5e1a46bd25ccd267 in_seq=0 out_seq=0 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f75f3fff640 10 client.19275 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:52.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f75f3fff640 1 -- 172.21.15.105:0/2956556059 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c298cb200 con 0x557c29975100 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f75f3fff640 10 client.19275 handle_command_reply: tid=0 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 client.19275 shutdown 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 -- 172.21.15.105:0/2956556059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 msgr2=0x557c2996d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c2996d6f0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f76100096c0 tx=0x7f7610009260 comp rx=0 tx=0).stop 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 -- 172.21.15.105:0/2956556059 shutdown_connections 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557c29975100 0x557c299dd890 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557c29964590 0x557c2996ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557c2995f420 0x557c2996dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f75f3fff640 0 client.19275 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 --2- 172.21.15.105:0/2956556059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557c2995ea30 0x557c2996d6f0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 -- 172.21.15.105:0/2956556059 >> 172.21.15.105:0/2956556059 conn(0x557c29959fa0 msgr2=0x557c299609b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 -- 172.21.15.105:0/2956556059 shutdown_connections 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 1 -- 172.21.15.105:0/2956556059 wait complete. 2024-03-26T17:48:52.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.699+0000 7f761dc2e740 20 client.19275 trim_cache size 0 max 16384 2024-03-26T17:48:52.697 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:48:52.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 -- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75fc0a2fa0 msgr2=0x7f75fc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75fc0a2fa0 0x7f75fc0a5460 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7610005ab0 tx=0x7f7610005fd0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 -- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7614102390 msgr2=0x7f761419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7614102390 0x7f761419ef10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f760c00efd0 tx=0x7f760c00c6a0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 -- 172.21.15.105:0/3661213068 shutdown_connections 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75fc0a2fa0 0x7f75fc0a5460 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7614102390 0x7f761419ef10 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76141019d0 0x7f761419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 --2- 172.21.15.105:0/3661213068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76141007e0 0x7f761406a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 -- 172.21.15.105:0/3661213068 >> 172.21.15.105:0/3661213068 conn(0x7f76140fbd70 msgr2=0x7f7614104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 -- 172.21.15.105:0/3661213068 shutdown_connections 2024-03-26T17:48:52.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:52.700+0000 7f761af62640 1 -- 172.21.15.105:0/3661213068 wait complete. 2024-03-26T17:48:53.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 337796 3.15 MB/sec execute 456 sec latency 27.637 ms 2024-03-26T17:48:53.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:52 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1977575806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:48:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:52 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1977575806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:48:53.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:52 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1977575806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:48:53.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:48:53.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:48:53.703 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:48:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 -- 172.21.15.105:0/1537142850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c104c70 msgr2=0x7f8d1c107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 --2- 172.21.15.105:0/1537142850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c104c70 0x7f8d1c107060 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009900 tx=0x7f8d0402bc30 comp rx=0 tx=0).stop 2024-03-26T17:48:53.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 -- 172.21.15.105:0/1537142850 shutdown_connections 2024-03-26T17:48:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 --2- 172.21.15.105:0/1537142850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c1075a0 0x7f8d1c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 --2- 172.21.15.105:0/1537142850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c104c70 0x7f8d1c107060 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 --2- 172.21.15.105:0/1537142850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c102340 0x7f8d1c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 -- 172.21.15.105:0/1537142850 >> 172.21.15.105:0/1537142850 conn(0x7f8d1c0fbd50 msgr2=0x7f8d1c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 -- 172.21.15.105:0/1537142850 shutdown_connections 2024-03-26T17:48:53.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.829+0000 7f8d20f22640 1 -- 172.21.15.105:0/1537142850 wait complete. 2024-03-26T17:48:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d20f22640 1 Processor -- start 2024-03-26T17:48:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d20f22640 1 -- start start 2024-03-26T17:48:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d20f22640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c102340 0x7f8d1c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d20f22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c104c70 0x7f8d1c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d20f22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c1075a0 0x7f8d1c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d20f22640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10d410 con 0x7f8d1c1075a0 2024-03-26T17:48:53.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d20f22640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10d290 con 0x7f8d1c104c70 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d20f22640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10d590 con 0x7f8d1c102340 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c1075a0 0x7f8d1c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c1075a0 0x7f8d1c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60442/0 (socket says 172.21.15.105:60442) 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 -- 172.21.15.105:0/4081098763 learned_addr learned my addr 172.21.15.105:0/4081098763 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d13fff640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c104c70 0x7f8d1c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c102340 0x7f8d1c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 -- 172.21.15.105:0/4081098763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c102340 msgr2=0x7f8d1c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c102340 0x7f8d1c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 -- 172.21.15.105:0/4081098763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c104c70 msgr2=0x7f8d1c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c104c70 0x7f8d1c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 -- 172.21.15.105:0/4081098763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c19f7a0 con 0x7f8d1c1075a0 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.830+0000 7f8d1ad9d640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c1075a0 0x7f8d1c19f130 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400eb10 tx=0x7f8d1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.831+0000 7f8d137fe640 1 -- 172.21.15.105:0/4081098763 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14004b20 con 0x7f8d1c1075a0 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.831+0000 7f8d137fe640 1 -- 172.21.15.105:0/4081098763 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d14004cc0 con 0x7f8d1c1075a0 2024-03-26T17:48:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.831+0000 7f8d20f22640 1 -- 172.21.15.105:0/4081098763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c19fa90 con 0x7f8d1c1075a0 2024-03-26T17:48:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.831+0000 7f8d137fe640 1 -- 172.21.15.105:0/4081098763 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14010690 con 0x7f8d1c1075a0 2024-03-26T17:48:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.831+0000 7f8d20f22640 1 -- 172.21.15.105:0/4081098763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c19ffb0 con 0x7f8d1c1075a0 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d23bee740 1 Processor -- start 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d137fe640 1 -- 172.21.15.105:0/4081098763 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8d14005020 con 0x7f8d1c1075a0 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d23bee740 1 -- start start 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3190 0x55d453bf35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3b80 0x55d453bf8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 0x55d453bfb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d23bee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d453b58630 con 0x55d453bf8cf0 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d23bee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d453b00e10 con 0x55d453bf3190 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d23bee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d453b545e0 con 0x55d453bf3b80 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3190 0x55d453bf35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3190 0x55d453bf35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35170/0 (socket says 172.21.15.105:35170) 2024-03-26T17:48:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d13fff640 1 -- 172.21.15.105:0/588093137 learned_addr learned my addr 172.21.15.105:0/588093137 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3b80 0x55d453bf8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.834+0000 7f8d13fff640 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 0x55d453bfb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d137fe640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d080a2f20 0x7f8d080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d13fff640 1 -- 172.21.15.105:0/588093137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3b80 msgr2=0x55d453bf8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d13fff640 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3b80 0x55d453bf8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d13fff640 1 -- 172.21.15.105:0/588093137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 msgr2=0x55d453bfb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d13fff640 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 0x55d453bfb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d13fff640 1 -- 172.21.15.105:0/588093137 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d453b17740 con 0x55d453bf3190 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3b80 0x55d453bf8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d1ad9d640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d080a2f20 0x7f8d080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d137fe640 1 -- 172.21.15.105:0/4081098763 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d140cfaf0 con 0x7f8d1c1075a0 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d13fff640 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3190 0x55d453bf35b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d0402c840 tx=0x7f8d04009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=392fb493f13436b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d10ff9640 1 -- 172.21.15.105:0/588093137 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0404a070 con 0x55d453bf3190 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d10ff9640 1 -- 172.21.15.105:0/588093137 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d04005050 con 0x55d453bf3190 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d10ff9640 1 -- 172.21.15.105:0/588093137 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004920 con 0x55d453bf3190 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.835+0000 7f8d1ad9d640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d080a2f20 0x7f8d080a53e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8d14098640 tx=0x7f8d140916c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 -- 172.21.15.105:0/588093137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3190 msgr2=0x55d453bf35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3190 0x55d453bf35b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d0402c840 tx=0x7f8d04009260 comp rx=0 tx=0).stop 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 -- 172.21.15.105:0/588093137 shutdown_connections 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 0x55d453bfb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3b80 0x55d453bf8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 --2- 172.21.15.105:0/588093137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3190 0x55d453bf35b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 -- 172.21.15.105:0/588093137 >> 172.21.15.105:0/588093137 conn(0x55d453bee700 msgr2=0x55d453beeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 -- 172.21.15.105:0/588093137 shutdown_connections 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 -- 172.21.15.105:0/588093137 wait complete. 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.836+0000 7f8d23bee740 1 -- start start 2024-03-26T17:48:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d23bee740 1 Processor -- start 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453bfd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453bfdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c04c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d23bee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d453bfbc40 con 0x55d453bf3b80 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d23bee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d453b141a0 con 0x55d453bf8cf0 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d23bee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d453bfbdc0 con 0x55d453bf3190 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d23bee740 10 client.0 _ll_register_callbacks cb 0x55d45386bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453bfdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453bfdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60462/0 (socket says 172.21.15.105:60462) 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 -- 172.21.15.105:0/4239853347 learned_addr learned my addr 172.21.15.105:0/4239853347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d13fff640 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c04c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d13fff640 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453bfd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 -- 172.21.15.105:0/4239853347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 msgr2=0x55d453bfd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:53.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453bfd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 -- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 msgr2=0x55d453c04c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c04c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 -- 172.21.15.105:0/4239853347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4539347e0 con 0x55d453bf3b80 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d13fff640 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453bfdcc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0126e0 tx=0x7f8d0c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b59b097cd55b54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.837+0000 7f8d01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d01ffb640 1 -- 172.21.15.105:0/4239853347 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c0121d0 con 0x55d453bf3b80 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 1 -- 172.21.15.105:0/4239853347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d04005050 con 0x55d453bf3b80 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d01ffb640 1 -- 172.21.15.105:0/4239853347 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d0c004050 con 0x55d453bf3b80 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d01ffb640 1 -- 172.21.15.105:0/4239853347 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d0c004250 con 0x55d453bf3b80 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 10 client.19287 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 1 -- 172.21.15.105:0/4239853347 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d453994f70 con 0x55d453bf3b80 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d01ffb640 1 -- 172.21.15.105:0/4239853347 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d04005050 con 0x55d453bf3b80 2024-03-26T17:48:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d01ffb640 1 -- 172.21.15.105:0/4239853347 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8d0c003960 con 0x55d453bf3b80 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 10 client.19287 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 10 client.19287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 20 client.19287 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d453c09a50 0x55d453c0be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 4 client.19287 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d23bee740 1 -- 172.21.15.105:0/4239853347 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d453b65b70 con 0x55d453c09a50 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.838+0000 7f8d13fff640 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d453c09a50 0x55d453c0be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.839+0000 7f8d13fff640 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d453c09a50 0x55d453c0be90 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5cc4de578eeb2b1 server_cookie=5c94bbecdf29ff12 in_seq=0 out_seq=0 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.839+0000 7f8d01ffb640 10 client.19287 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.839+0000 7f8d01ffb640 1 -- 172.21.15.105:0/4239853347 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d453b65b70 con 0x55d453c09a50 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.839+0000 7f8d01ffb640 10 client.19287 handle_command_reply: tid=0 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.839+0000 7f8d23bee740 1 client.19287 shutdown 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 -- 172.21.15.105:0/4239853347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 msgr2=0x55d453bfdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453bfdcc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0126e0 tx=0x7f8d0c012ba0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 -- 172.21.15.105:0/4239853347 shutdown_connections 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d453c09a50 0x55d453c0be90 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453bfdcc0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 --2- 172.21.15.105:0/4239853347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453bfd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 -- 172.21.15.105:0/4239853347 >> 172.21.15.105:0/4239853347 conn(0x55d453bee700 msgr2=0x55d453bed900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d01ffb640 0 client.19287 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 -- 172.21.15.105:0/4239853347 shutdown_connections 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 1 -- 172.21.15.105:0/4239853347 wait complete. 2024-03-26T17:48:53.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.840+0000 7f8d23bee740 20 client.19287 trim_cache size 0 max 16384 2024-03-26T17:48:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d23bee740 1 Processor -- start 2024-03-26T17:48:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d23bee740 1 -- start start 2024-03-26T17:48:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453bf5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3b80 0x55d453bf5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 0x55d453c04c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d23bee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1a8e20 con 0x55d453bf8cf0 2024-03-26T17:48:53.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d23bee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1aa960 con 0x55d453bf3b80 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d23bee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1a94f0 con 0x55d453bf3190 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d1a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3b80 0x55d453bf5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 0x55d453c04c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d1a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3b80 0x55d453bf5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35186/0 (socket says 172.21.15.105:35186) 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d1a59c640 1 -- 172.21.15.105:0/493792095 learned_addr learned my addr 172.21.15.105:0/493792095 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d13fff640 1 --2- 172.21.15.105:0/493792095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453bf5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d1a59c640 1 -- 172.21.15.105:0/493792095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 msgr2=0x55d453bf5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/493792095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453bf5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d1a59c640 1 -- 172.21.15.105:0/493792095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 msgr2=0x55d453c04c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/493792095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 0x55d453c04c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.859+0000 7f8d1a59c640 1 -- 172.21.15.105:0/493792095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45382c170 con 0x55d453bf3b80 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/493792095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3b80 0x55d453bf5770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005eb0 tx=0x7f8d0c0123e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4906e9b5a6d1de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d03fff640 1 -- 172.21.15.105:0/493792095 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c02b070 con 0x55d453bf3b80 2024-03-26T17:48:53.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d03fff640 1 -- 172.21.15.105:0/493792095 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d0c003aa0 con 0x55d453bf3b80 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d03fff640 1 -- 172.21.15.105:0/493792095 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00f790 con 0x55d453bf3b80 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d23bee740 1 -- 172.21.15.105:0/493792095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3b80 msgr2=0x55d453bf5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d23bee740 1 --2- 172.21.15.105:0/493792095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3b80 0x55d453bf5770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005eb0 tx=0x7f8d0c0123e0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d23bee740 1 -- 172.21.15.105:0/493792095 shutdown_connections 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d23bee740 1 --2- 172.21.15.105:0/493792095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf8cf0 0x55d453c04c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d23bee740 1 --2- 172.21.15.105:0/493792095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf3b80 0x55d453bf5770 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d23bee740 1 --2- 172.21.15.105:0/493792095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453bf5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.860+0000 7f8d23bee740 1 -- 172.21.15.105:0/493792095 >> 172.21.15.105:0/493792095 conn(0x55d453bee700 msgr2=0x55d453bfcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 -- 172.21.15.105:0/493792095 shutdown_connections 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 -- 172.21.15.105:0/493792095 wait complete. 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 -- start start 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 Processor -- start 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453c01b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453c020a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1aa290 con 0x55d453bf3b80 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1a9bc0 con 0x55d453bf8cf0 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1a6e00 con 0x55d453bf3190 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d23bee740 10 client.0 _ll_register_callbacks cb 0x7f8d0c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d1a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453c020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d13fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453c01b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d1a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453c020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60490/0 (socket says 172.21.15.105:60490) 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d1a59c640 1 -- 172.21.15.105:0/3234547442 learned_addr learned my addr 172.21.15.105:0/3234547442 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.861+0000 7f8d13fff640 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d13fff640 1 -- 172.21.15.105:0/3234547442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 msgr2=0x55d453c6ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d13fff640 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c6ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d13fff640 1 -- 172.21.15.105:0/3234547442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 msgr2=0x55d453c020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:53.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d13fff640 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453c020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d13fff640 1 -- 172.21.15.105:0/3234547442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d453c6e300 con 0x55d453bf3190 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453c020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d13fff640 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453c01b60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8d04036040 tx=0x7f8d04045040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20e6e7a6ab49cead server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d01ffb640 1 -- 172.21.15.105:0/3234547442 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0404a040 con 0x55d453bf3190 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d23bee740 1 -- 172.21.15.105:0/3234547442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0c003aa0 con 0x55d453bf3190 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d01ffb640 1 -- 172.21.15.105:0/3234547442 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f8d040583c0 con 0x55d453bf3190 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d01ffb640 1 -- 172.21.15.105:0/3234547442 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d040585c0 con 0x55d453bf3190 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d23bee740 10 client.29098 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d23bee740 1 -- 172.21.15.105:0/3234547442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d453c6f570 con 0x55d453bf3190 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.862+0000 7f8d01ffb640 1 -- 172.21.15.105:0/3234547442 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d0c003aa0 con 0x55d453bf3190 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d01ffb640 1 -- 172.21.15.105:0/3234547442 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f8d04050040 con 0x55d453bf3190 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d23bee740 10 client.29098 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:53.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d23bee740 10 client.29098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d23bee740 20 client.29098 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d23bee740 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d453c6fae0 0x55d453c71ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d23bee740 4 client.29098 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:48:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d23bee740 1 -- 172.21.15.105:0/3234547442 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d453b5fec0 con 0x55d453c6fae0 2024-03-26T17:48:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d453c6fae0 0x55d453c71ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d1a59c640 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d453c6fae0 0x55d453c71ed0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d4007a0388e1fb server_cookie=e625482d1ab9a237 in_seq=0 out_seq=0 2024-03-26T17:48:53.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.863+0000 7f8d01ffb640 10 client.29098 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:53.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d01ffb640 1 -- 172.21.15.105:0/3234547442 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d453b5fec0 con 0x55d453c6fae0 2024-03-26T17:48:53.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d01ffb640 10 client.29098 handle_command_reply: tid=0 2024-03-26T17:48:53.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 client.29098 shutdown 2024-03-26T17:48:53.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 -- 172.21.15.105:0/3234547442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 msgr2=0x55d453c01b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453c01b60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8d04036040 tx=0x7f8d04045040 comp rx=0 tx=0).stop 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 -- 172.21.15.105:0/3234547442 shutdown_connections 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d453c6fae0 0x55d453c71ed0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d453bf8cf0 0x55d453c6ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d453bf3b80 0x55d453c020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 --2- 172.21.15.105:0/3234547442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d453bf3190 0x55d453c01b60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 -- 172.21.15.105:0/3234547442 >> 172.21.15.105:0/3234547442 conn(0x55d453bee700 msgr2=0x55d453bf1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d01ffb640 0 client.29098 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.865+0000 7f8d23bee740 1 -- 172.21.15.105:0/3234547442 shutdown_connections 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.866+0000 7f8d23bee740 1 -- 172.21.15.105:0/3234547442 wait complete. 2024-03-26T17:48:53.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.866+0000 7f8d23bee740 20 client.29098 trim_cache size 0 max 16384 2024-03-26T17:48:53.863 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:48:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:48:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "2b3da624-887f-467e-b7b7-c79e7712874a", 2024-03-26T17:48:53.863 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:48:53.863 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 -- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d080a2f20 msgr2=0x7f8d080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d080a2f20 0x7f8d080a53e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8d14098640 tx=0x7f8d140916c0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 -- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c1075a0 msgr2=0x7f8d1c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c1075a0 0x7f8d1c19f130 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400eb10 tx=0x7f8d1400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 -- 172.21.15.105:0/4081098763 shutdown_connections 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8d080a2f20 0x7f8d080a53e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d1c1075a0 0x7f8d1c19f130 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d1c104c70 0x7f8d1c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 --2- 172.21.15.105:0/4081098763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c102340 0x7f8d1c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 -- 172.21.15.105:0/4081098763 >> 172.21.15.105:0/4081098763 conn(0x7f8d1c0fbd50 msgr2=0x7f8d1c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:53.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 -- 172.21.15.105:0/4081098763 shutdown_connections 2024-03-26T17:48:53.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:53.867+0000 7f8d20f22640 1 -- 172.21.15.105:0/4081098763 wait complete. 2024-03-26T17:48:53.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:48:53.869 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:48:53.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 -- 172.21.15.105:0/469493655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e0102370 msgr2=0x7f61e0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 --2- 172.21.15.105:0/469493655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e0102370 0x7f61e0108c00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009ca0 tx=0x7f61d802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 -- 172.21.15.105:0/469493655 shutdown_connections 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 --2- 172.21.15.105:0/469493655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e0102370 0x7f61e0108c00 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 --2- 172.21.15.105:0/469493655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e01019b0 0x7f61e0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 --2- 172.21.15.105:0/469493655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e01007c0 0x7f61e0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 -- 172.21.15.105:0/469493655 >> 172.21.15.105:0/469493655 conn(0x7f61e00fbd50 msgr2=0x7f61e00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 -- 172.21.15.105:0/469493655 shutdown_connections 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 -- 172.21.15.105:0/469493655 wait complete. 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 Processor -- start 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.000+0000 7f61e5612640 1 -- start start 2024-03-26T17:48:53.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61e5612640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e01007c0 0x7f61e0104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61e5612640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e01019b0 0x7f61e0102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61e5612640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e0102370 0x7f61e01032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61e5612640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010c2f0 con 0x7f61e01007c0 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61e5612640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010c170 con 0x7f61e01019b0 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61e5612640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61e010c470 con 0x7f61e0102370 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e01007c0 0x7f61e0104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e01007c0 0x7f61e0104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60522/0 (socket says 172.21.15.105:60522) 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61de7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e01019b0 0x7f61e0102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 -- 172.21.15.105:0/3290682963 learned_addr learned my addr 172.21.15.105:0/3290682963 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61df7fe640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e0102370 0x7f61e01032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 -- 172.21.15.105:0/3290682963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e0102370 msgr2=0x7f61e01032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e0102370 0x7f61e01032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 -- 172.21.15.105:0/3290682963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e01019b0 msgr2=0x7f61e0102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:53.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e01019b0 0x7f61e0102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:53.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 -- 172.21.15.105:0/3290682963 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e0103b60 con 0x7f61e01007c0 2024-03-26T17:48:53.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61de7fc640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e01019b0 0x7f61e0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:53.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61deffd640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e01007c0 0x7f61e0104750 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f61d000e9c0 tx=0x7f61d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:53.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61c7fff640 1 -- 172.21.15.105:0/3290682963 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d000cd80 con 0x7f61e01007c0 2024-03-26T17:48:53.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61c7fff640 1 -- 172.21.15.105:0/3290682963 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d00035f0 con 0x7f61e01007c0 2024-03-26T17:48:53.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61e5612640 1 -- 172.21.15.105:0/3290682963 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e01a1c10 con 0x7f61e01007c0 2024-03-26T17:48:53.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61c7fff640 1 -- 172.21.15.105:0/3290682963 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0010690 con 0x7f61e01007c0 2024-03-26T17:48:53.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.001+0000 7f61e5612640 1 -- 172.21.15.105:0/3290682963 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01a2110 con 0x7f61e01007c0 2024-03-26T17:48:54.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61c7fff640 1 -- 172.21.15.105:0/3290682963 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f61d00037a0 con 0x7f61e01007c0 2024-03-26T17:48:54.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61e8202740 1 Processor -- start 2024-03-26T17:48:54.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61e8202740 1 -- start start 2024-03-26T17:48:54.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61e8202740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc08e8f0 0x55d9fc08ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61e8202740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc093de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61e8202740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc094450 0x55d9fc0968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61e8202740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d9fbff3980 con 0x55d9fc08f2e0 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61e8202740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9fbf9bbe0 con 0x55d9fc094450 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61e8202740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d9fbfef930 con 0x55d9fc08e8f0 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc093de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61de7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc094450 0x55d9fc0968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61de7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc094450 0x55d9fc0968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35214/0 (socket says 172.21.15.105:35214) 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61de7fc640 1 -- 172.21.15.105:0/4203669741 learned_addr learned my addr 172.21.15.105:0/4203669741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc093de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60536/0 (socket says 172.21.15.105:60536) 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61df7fe640 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc08e8f0 0x55d9fc08ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.004+0000 7f61c7fff640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f61c80a2f20 0x7f61c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61de7fc640 1 -- 172.21.15.105:0/4203669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc08e8f0 msgr2=0x55d9fc08ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61de7fc640 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc08e8f0 0x55d9fc08ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61de7fc640 1 -- 172.21.15.105:0/4203669741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 msgr2=0x55d9fc093de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61de7fc640 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc093de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61de7fc640 1 -- 172.21.15.105:0/4203669741 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9fbdd17d0 con 0x55d9fc094450 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61df7fe640 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc08e8f0 0x55d9fc08ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61c7fff640 1 -- 172.21.15.105:0/3290682963 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f61d00cf970 con 0x7f61e01007c0 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61df7fe640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f61c80a2f20 0x7f61c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61de7fc640 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc094450 0x55d9fc0968e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f61d40114f0 tx=0x7f61d40119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5609ea95f1e9f036 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61c57fa640 1 -- 172.21.15.105:0/4203669741 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d401e070 con 0x55d9fc094450 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61c57fa640 1 -- 172.21.15.105:0/4203669741 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d40035f0 con 0x55d9fc094450 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61df7fe640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f61c80a2f20 0x7f61c80a53e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f61d8006010 tx=0x7f61d802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.005+0000 7f61c57fa640 1 -- 172.21.15.105:0/4203669741 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d400f800 con 0x55d9fc094450 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 -- 172.21.15.105:0/4203669741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc094450 msgr2=0x55d9fc0968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc094450 0x55d9fc0968e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f61d40114f0 tx=0x7f61d40119b0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 -- 172.21.15.105:0/4203669741 shutdown_connections 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc094450 0x55d9fc0968e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc093de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 --2- 172.21.15.105:0/4203669741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc08e8f0 0x55d9fc08ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 -- 172.21.15.105:0/4203669741 >> 172.21.15.105:0/4203669741 conn(0x55d9fc089e60 msgr2=0x55d9fc08a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 -- 172.21.15.105:0/4203669741 shutdown_connections 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 -- 172.21.15.105:0/4203669741 wait complete. 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 -- start start 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.006+0000 7f61e8202740 1 Processor -- start 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc099070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc0a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d9fc097400 con 0x55d9fc08e8f0 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9fbfaf160 con 0x55d9fc08f2e0 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d9fc097580 con 0x55d9fc094450 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 10 client.0 _ll_register_callbacks cb 0x55d9fbfeb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:54.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60546/0 (socket says 172.21.15.105:60546) 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 -- 172.21.15.105:0/2966583530 learned_addr learned my addr 172.21.15.105:0/2966583530 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61de7fc640 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc099070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61df7fe640 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc0a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 -- 172.21.15.105:0/2966583530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 msgr2=0x55d9fc0a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc0a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 -- 172.21.15.105:0/2966583530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 msgr2=0x55d9fc099070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc099070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 -- 172.21.15.105:0/2966583530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9fbd073f0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61deffd640 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098b30 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f61d00058c0 tx=0x7f61d009be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fd9a9e97719ac0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61c2ff5640 1 -- 172.21.15.105:0/2966583530 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0003b10 con 0x55d9fc08e8f0 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 1 -- 172.21.15.105:0/2966583530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d40035f0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61c2ff5640 1 -- 172.21.15.105:0/2966583530 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f61d0096bd0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61c2ff5640 1 -- 172.21.15.105:0/2966583530 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61d009c760 con 0x55d9fc08e8f0 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 10 client.19305 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.007+0000 7f61e8202740 1 -- 172.21.15.105:0/2966583530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9fbcc70f0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61c2ff5640 1 -- 172.21.15.105:0/2966583530 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d40035f0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61c2ff5640 1 -- 172.21.15.105:0/2966583530 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f61d00a45e0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61e8202740 10 client.19305 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61e8202740 10 client.19305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61e8202740 20 client.19305 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61e8202740 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d9fc0a49c0 0x55d9fc0a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61e8202740 4 client.19305 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61e8202740 1 -- 172.21.15.105:0/2966583530 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9fc000ec0 con 0x55d9fc0a49c0 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61de7fc640 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d9fc0a49c0 0x55d9fc0a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61c2ff5640 10 client.19305 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.008+0000 7f61de7fc640 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d9fc0a49c0 0x55d9fc0a6e50 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b76b8e2b0edf871 server_cookie=fbd275b1fd13efe5 in_seq=0 out_seq=0 2024-03-26T17:48:54.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61c2ff5640 1 -- 172.21.15.105:0/2966583530 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d9fc000ec0 con 0x55d9fc0a49c0 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61c2ff5640 10 client.19305 handle_command_reply: tid=0 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 client.19305 shutdown 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 -- 172.21.15.105:0/2966583530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 msgr2=0x55d9fc098b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098b30 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f61d00058c0 tx=0x7f61d009be30 comp rx=0 tx=0).stop 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 -- 172.21.15.105:0/2966583530 shutdown_connections 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d9fc0a49c0 0x55d9fc0a6e50 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc0a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc099070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61c2ff5640 0 client.19305 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 --2- 172.21.15.105:0/2966583530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098b30 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.009+0000 7f61e8202740 1 -- 172.21.15.105:0/2966583530 >> 172.21.15.105:0/2966583530 conn(0x55d9fc089e60 msgr2=0x55d9fc089080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.010+0000 7f61e8202740 1 -- 172.21.15.105:0/2966583530 shutdown_connections 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.010+0000 7f61e8202740 1 -- 172.21.15.105:0/2966583530 wait complete. 2024-03-26T17:48:54.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.010+0000 7f61e8202740 20 client.19305 trim_cache size 0 max 16384 2024-03-26T17:48:54.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61e8202740 1 Processor -- start 2024-03-26T17:48:54.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61e8202740 1 -- start start 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61e8202740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61e8202740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc0a0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61e8202740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc08a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61e8202740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01aaf90 con 0x55d9fc08f2e0 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61e8202740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01acad0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61e8202740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ab660 con 0x55d9fc094450 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61de7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc0a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61de7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc0a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60556/0 (socket says 172.21.15.105:60556) 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.031+0000 7f61de7fc640 1 -- 172.21.15.105:0/4147641864 learned_addr learned my addr 172.21.15.105:0/4147641864 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61deffd640 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61df7fe640 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc08a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61de7fc640 1 -- 172.21.15.105:0/4147641864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 msgr2=0x55d9fc08a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61de7fc640 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc08a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61de7fc640 1 -- 172.21.15.105:0/4147641864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08e8f0 msgr2=0x55d9fc098fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:54.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61de7fc640 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61de7fc640 1 -- 172.21.15.105:0/4147641864 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9fc08f6c0 con 0x55d9fc08f2e0 2024-03-26T17:48:54.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61de7fc640 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc0a0000 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009e20 tx=0x7f61d4009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8a9da944c26f1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:54.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61c4ff9640 1 -- 172.21.15.105:0/4147641864 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4030070 con 0x55d9fc08f2e0 2024-03-26T17:48:54.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61c4ff9640 1 -- 172.21.15.105:0/4147641864 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d402a030 con 0x55d9fc08f2e0 2024-03-26T17:48:54.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.032+0000 7f61c4ff9640 1 -- 172.21.15.105:0/4147641864 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d401e040 con 0x55d9fc08f2e0 2024-03-26T17:48:54.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- 172.21.15.105:0/4147641864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 msgr2=0x55d9fc0a0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:54.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc0a0000 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009e20 tx=0x7f61d4009fb0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- 172.21.15.105:0/4147641864 shutdown_connections 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc08a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08f2e0 0x55d9fc0a0000 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 --2- 172.21.15.105:0/4147641864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08e8f0 0x55d9fc098fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- 172.21.15.105:0/4147641864 >> 172.21.15.105:0/4147641864 conn(0x55d9fc089e60 msgr2=0x55d9fc093b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- 172.21.15.105:0/4147641864 shutdown_connections 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- 172.21.15.105:0/4147641864 wait complete. 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- start start 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 Processor -- start 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc09d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc09dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc09ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01ac400 con 0x55d9fc08e8f0 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01abd30 con 0x55d9fc08f2e0 2024-03-26T17:48:54.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61e01a8f70 con 0x55d9fc094450 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61e8202740 10 client.0 _ll_register_callbacks cb 0x7f61d0003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.033+0000 7f61deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc09d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61de7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc09dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc09d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60570/0 (socket says 172.21.15.105:60570) 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61deffd640 1 -- 172.21.15.105:0/1713478223 learned_addr learned my addr 172.21.15.105:0/1713478223 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61de7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc09dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35238/0 (socket says 172.21.15.105:35238) 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61df7fe640 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc09ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61deffd640 1 -- 172.21.15.105:0/1713478223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 msgr2=0x55d9fc09ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61deffd640 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc09ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61deffd640 1 -- 172.21.15.105:0/1713478223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 msgr2=0x55d9fc09dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61deffd640 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc09dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61deffd640 1 -- 172.21.15.105:0/1713478223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9fc09f300 con 0x55d9fc08e8f0 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61de7fc640 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc09dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:48:54.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61deffd640 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc09d7c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f61d00989e0 tx=0x7f61d0097fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56718637948091cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61c2ff5640 1 -- 172.21.15.105:0/1713478223 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d007fca0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61c2ff5640 1 -- 172.21.15.105:0/1713478223 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f61d0096bd0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61e8202740 1 -- 172.21.15.105:0/1713478223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d402a030 con 0x55d9fc08e8f0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61c2ff5640 1 -- 172.21.15.105:0/1713478223 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61d009bba0 con 0x55d9fc08e8f0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61e8202740 10 client.19311 fetch_fsmap learned FSMap version 30 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.034+0000 7f61e8202740 1 -- 172.21.15.105:0/1713478223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9fc10ae80 con 0x55d9fc08e8f0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61c2ff5640 1 -- 172.21.15.105:0/1713478223 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61d402a030 con 0x55d9fc08e8f0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61c2ff5640 1 -- 172.21.15.105:0/1713478223 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f61d009b130 con 0x55d9fc08e8f0 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61e8202740 10 client.19311 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61e8202740 10 client.19311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61e8202740 20 client.19311 populate_metadata read hostname 'smithi105' 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61e8202740 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d9fc10b740 0x55d9fc10db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61e8202740 4 client.19311 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61e8202740 1 -- 172.21.15.105:0/1713478223 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9fbffb210 con 0x55d9fc10b740 2024-03-26T17:48:54.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61de7fc640 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d9fc10b740 0x55d9fc10db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61c2ff5640 10 client.19311 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61de7fc640 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d9fc10b740 0x55d9fc10db60 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353694d24fef763b server_cookie=10f0c5c81aced6d7 in_seq=0 out_seq=0 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61c2ff5640 1 -- 172.21.15.105:0/1713478223 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d9fbffb210 con 0x55d9fc10b740 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61c2ff5640 10 client.19311 handle_command_reply: tid=0 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.035+0000 7f61e8202740 1 client.19311 shutdown 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 -- 172.21.15.105:0/1713478223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 msgr2=0x55d9fc09d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc09d7c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f61d00989e0 tx=0x7f61d0097fe0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 -- 172.21.15.105:0/1713478223 shutdown_connections 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d9fc10b740 0x55d9fc10db60 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d9fc094450 0x55d9fc09ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9fc08f2e0 0x55d9fc09dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 --2- 172.21.15.105:0/1713478223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d9fc08e8f0 0x55d9fc09d7c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61c2ff5640 0 client.19311 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 -- 172.21.15.105:0/1713478223 >> 172.21.15.105:0/1713478223 conn(0x55d9fc089e60 msgr2=0x55d9fc08a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:54.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 -- 172.21.15.105:0/1713478223 shutdown_connections 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 1 -- 172.21.15.105:0/1713478223 wait complete. 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e8202740 20 client.19311 trim_cache size 0 max 16384 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "2b3da624-887f-467e-b7b7-c79e7712874a": { 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "2b3da624-887f-467e-b7b7-c79e7712874a", 2024-03-26T17:48:54.034 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:48:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:48:54.035 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:48:54.035 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:48:54.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 -- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f61c80a2f20 msgr2=0x7f61c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f61c80a2f20 0x7f61c80a53e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f61d8006010 tx=0x7f61d802e040 comp rx=0 tx=0).stop 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 -- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e01007c0 msgr2=0x7f61e0104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e01007c0 0x7f61e0104750 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f61d000e9c0 tx=0x7f61d000ee80 comp rx=0 tx=0).stop 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 -- 172.21.15.105:0/3290682963 shutdown_connections 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f61c80a2f20 0x7f61c80a53e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e0102370 0x7f61e01032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f61e01019b0 0x7f61e0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 --2- 172.21.15.105:0/3290682963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f61e01007c0 0x7f61e0104750 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.036+0000 7f61e5612640 1 -- 172.21.15.105:0/3290682963 >> 172.21.15.105:0/3290682963 conn(0x7f61e00fbd50 msgr2=0x7f61e00fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.037+0000 7f61e5612640 1 -- 172.21.15.105:0/3290682963 shutdown_connections 2024-03-26T17:48:54.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:48:54.037+0000 7f61e5612640 1 -- 172.21.15.105:0/3290682963 wait complete. 2024-03-26T17:48:54.037 INFO:tasks.workunit.client.0.smithi105.stdout: 1 338387 3.15 MB/sec execute 457 sec latency 20.750 ms 2024-03-26T17:48:54.038 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:53 smithi105 ceph-mon[32280]: pgmap v664: 97 pgs: 97 active+clean; 24 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-03-26T17:48:54.039 INFO:tasks.cephfs.filesystem:scrub status for tag:2b3da624-887f-467e-b7b7-c79e7712874a - {'path': '/', 'tag': '2b3da624-887f-467e-b7b7-c79e7712874a', 'options': 'recursive,force'} 2024-03-26T17:48:54.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:53 smithi144 ceph-mon[36095]: pgmap v664: 97 pgs: 97 active+clean; 24 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-03-26T17:48:54.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:53 smithi135 ceph-mon[35636]: pgmap v664: 97 pgs: 97 active+clean; 24 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-03-26T17:48:55.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 339017 3.14 MB/sec execute 458 sec latency 21.822 ms 2024-03-26T17:48:55.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:55 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:48:55.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:55 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:55.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:55 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:48:55.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:55 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:55.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:55 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:48:55.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:55 smithi105 ceph-mon[32280]: pgmap v665: 97 pgs: 97 active+clean; 27 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:48:55.914 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:55 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:48:55.914 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:55 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:55.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:55 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:48:55.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:55 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:55.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:55 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:48:55.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:55 smithi135 ceph-mon[35636]: pgmap v665: 97 pgs: 97 active+clean; 27 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:48:56.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 339539 3.14 MB/sec execute 459 sec latency 18.087 ms 2024-03-26T17:48:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:55 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:48:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:55 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:55 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:48:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:55 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:48:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:55 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:48:56.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:55 smithi144 ceph-mon[36095]: pgmap v665: 97 pgs: 97 active+clean; 27 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-03-26T17:48:57.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 340265 3.15 MB/sec execute 460 sec latency 16.345 ms 2024-03-26T17:48:57.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.848 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: pgmap v666: 97 pgs: 97 active+clean; 27 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:48:57.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:48:57.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:48:57.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:48:57.849 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: pgmap v666: 97 pgs: 97 active+clean; 27 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:48:57.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:48:57.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:48:57.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:48:57.892 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.914 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.914 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.914 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.914 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:57.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: pgmap v666: 97 pgs: 97 active+clean; 27 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 115 op/s 2024-03-26T17:48:57.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:48:57.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:48:57.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:48:57.915 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:48:58.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 340670 3.15 MB/sec execute 461 sec latency 16.522 ms 2024-03-26T17:48:59.031 INFO:tasks.workunit.client.0.smithi105.stdout: 1 341059 3.15 MB/sec execute 462 sec latency 16.361 ms 2024-03-26T17:49:00.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 341591 3.14 MB/sec execute 463 sec latency 23.617 ms 2024-03-26T17:49:00.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:48:59 smithi144 ceph-mon[36095]: pgmap v667: 97 pgs: 97 active+clean; 28 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-03-26T17:49:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:48:59 smithi105 ceph-mon[32280]: pgmap v667: 97 pgs: 97 active+clean; 28 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-03-26T17:49:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:48:59 smithi135 ceph-mon[35636]: pgmap v667: 97 pgs: 97 active+clean; 28 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-03-26T17:49:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 -- 172.21.15.105:0/1768282685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 msgr2=0x7f9478101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 --2- 172.21.15.105:0/1768282685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 0x7f9478101d40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f9468009900 tx=0x7f946802bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 -- 172.21.15.105:0/1768282685 shutdown_connections 2024-03-26T17:49:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 --2- 172.21.15.105:0/1768282685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94781034d0 0x7f9478109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 --2- 172.21.15.105:0/1768282685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9478102b10 0x7f9478102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 --2- 172.21.15.105:0/1768282685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 0x7f9478101d40 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 -- 172.21.15.105:0/1768282685 >> 172.21.15.105:0/1768282685 conn(0x7f94780fce90 msgr2=0x7f94780ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 -- 172.21.15.105:0/1768282685 shutdown_connections 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.235+0000 7f947ca85640 1 -- 172.21.15.105:0/1768282685 wait complete. 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947ca85640 1 Processor -- start 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947ca85640 1 -- start start 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947ca85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 0x7f9478197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947ca85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478102b10 0x7f9478198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947ca85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94781034d0 0x7f947819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947ca85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f947810d450 con 0x7f9478101920 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947ca85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f947810d2d0 con 0x7f94781034d0 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947ca85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f947810d5d0 con 0x7f9478102b10 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 0x7f9478197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 0x7f9478197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55436/0 (socket says 172.21.15.105:55436) 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f947659c640 1 -- 172.21.15.105:0/3478203902 learned_addr learned my addr 172.21.15.105:0/3478203902 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f946ffff640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478102b10 0x7f9478198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.236+0000 7f9476d9d640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94781034d0 0x7f947819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f947659c640 1 -- 172.21.15.105:0/3478203902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478102b10 msgr2=0x7f9478198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f947659c640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478102b10 0x7f9478198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f947659c640 1 -- 172.21.15.105:0/3478203902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94781034d0 msgr2=0x7f947819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f947659c640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94781034d0 0x7f947819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f947659c640 1 -- 172.21.15.105:0/3478203902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947819f6e0 con 0x7f9478101920 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f947659c640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 0x7f9478197ac0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9468000c00 tx=0x7f9468005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f946f7fe640 1 -- 172.21.15.105:0/3478203902 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468043020 con 0x7f9478101920 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f946f7fe640 1 -- 172.21.15.105:0/3478203902 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9468004350 con 0x7f9478101920 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f946f7fe640 1 -- 172.21.15.105:0/3478203902 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468003b80 con 0x7f9478101920 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f947ca85640 1 -- 172.21.15.105:0/3478203902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947819f970 con 0x7f9478101920 2024-03-26T17:49:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.237+0000 7f947ca85640 1 -- 172.21.15.105:0/3478203902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947819ff10 con 0x7f9478101920 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.240+0000 7f946f7fe640 1 -- 172.21.15.105:0/3478203902 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9468046020 con 0x7f9478101920 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.240+0000 7f947f676740 1 Processor -- start 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f947f676740 1 -- start start 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f947f676740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c3785d0 0x56287c3789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f947f676740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c378fc0 0x56287c37dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f947f676740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c37e130 0x56287c3805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f947f676740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56287c2dd0b0 con 0x56287c378fc0 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f947f676740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56287c285960 con 0x56287c3785d0 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f947f676740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56287c2d9060 con 0x56287c37e130 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f947659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c378fc0 0x56287c37dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c3785d0 0x56287c3789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f946f7fe640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f94640a3040 0x7f94640a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c3785d0 0x56287c3789f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36796/0 (socket says 172.21.15.105:36796) 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 -- 172.21.15.105:0/3035558173 learned_addr learned my addr 172.21.15.105:0/3035558173 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f946ffff640 1 --2- 172.21.15.105:0/3035558173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c37e130 0x56287c3805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f94640a3040 0x7f94640a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 -- 172.21.15.105:0/3035558173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c37e130 msgr2=0x56287c3805c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 --2- 172.21.15.105:0/3035558173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c37e130 0x56287c3805c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 -- 172.21.15.105:0/3035558173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c378fc0 msgr2=0x56287c37dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 --2- 172.21.15.105:0/3035558173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c378fc0 0x56287c37dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f9476d9d640 1 -- 172.21.15.105:0/3035558173 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287c297c60 con 0x56287c3785d0 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.241+0000 7f946f7fe640 1 -- 172.21.15.105:0/3478203902 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f94680f14f0 con 0x7f9478101920 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.242+0000 7f9476d9d640 1 --2- 172.21.15.105:0/3035558173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c3785d0 0x56287c3789f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9470005c40 tx=0x7f947000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198da30add626cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.242+0000 7f9476d9d640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f94640a3040 0x7f94640a5500 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9470008000 tx=0x7f947001f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.242+0000 7f946cff9640 1 -- 172.21.15.105:0/3035558173 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947000fd30 con 0x56287c3785d0 2024-03-26T17:49:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.242+0000 7f946cff9640 1 -- 172.21.15.105:0/3035558173 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9470022030 con 0x56287c3785d0 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.242+0000 7f946cff9640 1 -- 172.21.15.105:0/3035558173 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700108a0 con 0x56287c3785d0 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 -- 172.21.15.105:0/3035558173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c3785d0 msgr2=0x56287c3789f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 --2- 172.21.15.105:0/3035558173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c3785d0 0x56287c3789f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9470005c40 tx=0x7f947000f3c0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 -- 172.21.15.105:0/3035558173 shutdown_connections 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 --2- 172.21.15.105:0/3035558173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c37e130 0x56287c3805c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 --2- 172.21.15.105:0/3035558173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c378fc0 0x56287c37dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 --2- 172.21.15.105:0/3035558173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c3785d0 0x56287c3789f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 -- 172.21.15.105:0/3035558173 >> 172.21.15.105:0/3035558173 conn(0x56287c373b40 msgr2=0x56287c373f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 -- 172.21.15.105:0/3035558173 shutdown_connections 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 -- 172.21.15.105:0/3035558173 wait complete. 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 -- start start 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.243+0000 7f947f676740 1 Processor -- start 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947f676740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c382390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947f676740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947f676740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c389910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947f676740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56287c381080 con 0x56287c37e130 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947f676740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56287c298db0 con 0x56287c378fc0 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947f676740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56287c381200 con 0x56287c3785d0 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947f676740 10 client.0 _ll_register_callbacks cb 0x56287c29d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c382390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c382390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58460/0 (socket says 172.21.15.105:58460) 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947659c640 1 -- 172.21.15.105:0/658394523 learned_addr learned my addr 172.21.15.105:0/658394523 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f946ffff640 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f9476d9d640 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c389910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947659c640 1 -- 172.21.15.105:0/658394523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 msgr2=0x56287c3828d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947659c640 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3828d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947659c640 1 -- 172.21.15.105:0/658394523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 msgr2=0x56287c389910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947659c640 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c389910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.244+0000 7f947659c640 1 -- 172.21.15.105:0/658394523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56287bff1210 con 0x56287c3785d0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947659c640 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c382390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f94680ba5b0 tx=0x7f94680bd720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf8fe5b202258ec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f945dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f945dffb640 1 -- 172.21.15.105:0/658394523 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94680a8040 con 0x56287c3785d0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f945dffb640 1 -- 172.21.15.105:0/658394523 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9468003d20 con 0x56287c3785d0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 1 -- 172.21.15.105:0/658394523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9470022030 con 0x56287c3785d0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f945dffb640 1 -- 172.21.15.105:0/658394523 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94680bdbe0 con 0x56287c3785d0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 10 client.29128 fetch_fsmap learned FSMap version 30 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 1 -- 172.21.15.105:0/658394523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56287c0bd620 con 0x56287c3785d0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f945dffb640 1 -- 172.21.15.105:0/658394523 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9470022030 con 0x56287c3785d0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f945dffb640 1 -- 172.21.15.105:0/658394523 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f94680bde40 con 0x56287c3785d0 2024-03-26T17:49:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 10 client.29128 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 10 client.29128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 20 client.29128 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56287c38e1e0 0x56287c390630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 4 client.29128 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.245+0000 7f947f676740 1 -- 172.21.15.105:0/658394523 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56287c2ea5f0 con 0x56287c38e1e0 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.246+0000 7f946ffff640 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56287c38e1e0 0x56287c390630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.246+0000 7f945dffb640 10 client.29128 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.246+0000 7f946ffff640 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56287c38e1e0 0x56287c390630 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d10a02d28ddada server_cookie=801ac061b82fcfa1 in_seq=0 out_seq=0 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f945dffb640 1 -- 172.21.15.105:0/658394523 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56287c2ea5f0 con 0x56287c38e1e0 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f945dffb640 10 client.29128 handle_command_reply: tid=0 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 client.29128 shutdown 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 -- 172.21.15.105:0/658394523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 msgr2=0x56287c382390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c382390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f94680ba5b0 tx=0x7f94680bd720 comp rx=0 tx=0).stop 2024-03-26T17:49:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 -- 172.21.15.105:0/658394523 shutdown_connections 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56287c38e1e0 0x56287c390630 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c389910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f945dffb640 0 client.29128 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3828d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 --2- 172.21.15.105:0/658394523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c382390 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 -- 172.21.15.105:0/658394523 >> 172.21.15.105:0/658394523 conn(0x56287c373b40 msgr2=0x56287c372df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 -- 172.21.15.105:0/658394523 shutdown_connections 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 1 -- 172.21.15.105:0/658394523 wait complete. 2024-03-26T17:49:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.247+0000 7f947f676740 20 client.29128 trim_cache size 0 max 16384 2024-03-26T17:49:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.263+0000 7f947f676740 1 Processor -- start 2024-03-26T17:49:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.263+0000 7f947f676740 1 -- start start 2024-03-26T17:49:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.263+0000 7f947f676740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c381fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.263+0000 7f947f676740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.263+0000 7f947f676740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c3824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.263+0000 7f947f676740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94781a6af0 con 0x56287c37e130 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.263+0000 7f947f676740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94781a8630 con 0x56287c378fc0 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.263+0000 7f947f676740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94781a71c0 con 0x56287c3785d0 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c3824e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c3824e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55470/0 (socket says 172.21.15.105:55470) 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f9476d9d640 1 -- 172.21.15.105:0/472529786 learned_addr learned my addr 172.21.15.105:0/472529786 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f946ffff640 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f947659c640 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c381fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f946ffff640 1 -- 172.21.15.105:0/472529786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 msgr2=0x56287c381fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f946ffff640 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c381fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f946ffff640 1 -- 172.21.15.105:0/472529786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 msgr2=0x56287c3824e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f946ffff640 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c3824e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f946ffff640 1 -- 172.21.15.105:0/472529786 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56287c390c60 con 0x56287c378fc0 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f947659c640 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c381fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f946ffff640 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3898a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x56287c390e30 tx=0x7f9460013b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd1287e9230167e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f945ffff640 1 -- 172.21.15.105:0/472529786 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460013e60 con 0x56287c378fc0 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f945ffff640 1 -- 172.21.15.105:0/472529786 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9460009df0 con 0x56287c378fc0 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.264+0000 7f945ffff640 1 -- 172.21.15.105:0/472529786 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94600203a0 con 0x56287c378fc0 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 -- 172.21.15.105:0/472529786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 msgr2=0x56287c3898a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3898a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x56287c390e30 tx=0x7f9460013b60 comp rx=0 tx=0).stop 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 -- 172.21.15.105:0/472529786 shutdown_connections 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c3824e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c3898a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 --2- 172.21.15.105:0/472529786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c381fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 -- 172.21.15.105:0/472529786 >> 172.21.15.105:0/472529786 conn(0x56287c373b40 msgr2=0x56287c376da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 -- 172.21.15.105:0/472529786 shutdown_connections 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 -- 172.21.15.105:0/472529786 wait complete. 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.265+0000 7f947f676740 1 -- start start 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947f676740 1 Processor -- start 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947f676740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c386d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947f676740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c387240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947f676740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c388250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947f676740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94781a7f60 con 0x56287c37e130 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947f676740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94781a7890 con 0x56287c378fc0 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947f676740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94781a4ad0 con 0x56287c3785d0 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947f676740 10 client.0 _ll_register_callbacks cb 0x7f94680057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c388250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c386d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c388250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55484/0 (socket says 172.21.15.105:55484) 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 -- 172.21.15.105:0/1328874897 learned_addr learned my addr 172.21.15.105:0/1328874897 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f946ffff640 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c387240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 -- 172.21.15.105:0/1328874897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 msgr2=0x56287c386d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c386d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 -- 172.21.15.105:0/1328874897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 msgr2=0x56287c387240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c387240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 -- 172.21.15.105:0/1328874897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56287c388840 con 0x56287c37e130 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f947659c640 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c386d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f9476d9d640 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c388250 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9470006010 tx=0x7f9470003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea8d52ca1e5c23d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.266+0000 7f945dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f945dffb640 1 -- 172.21.15.105:0/1328874897 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470036060 con 0x56287c37e130 2024-03-26T17:49:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f945dffb640 1 -- 172.21.15.105:0/1328874897 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7f9470013120 con 0x56287c37e130 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 1 -- 172.21.15.105:0/1328874897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9460009df0 con 0x56287c37e130 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f945dffb640 1 -- 172.21.15.105:0/1328874897 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9470005030 con 0x56287c37e130 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 10 client.19329 fetch_fsmap learned FSMap version 30 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 1 -- 172.21.15.105:0/1328874897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56287c389bc0 con 0x56287c37e130 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f945dffb640 1 -- 172.21.15.105:0/1328874897 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9460009df0 con 0x56287c37e130 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f945dffb640 1 -- 172.21.15.105:0/1328874897 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9470027570 con 0x56287c37e130 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 10 client.19329 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 10 client.19329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 20 client.19329 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56287c3f4a20 0x56287c3f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 4 client.19329 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947f676740 1 -- 172.21.15.105:0/1328874897 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56287c2e4940 con 0x56287c3f4a20 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.267+0000 7f947659c640 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56287c3f4a20 0x56287c3f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.268+0000 7f945dffb640 10 client.19329 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.268+0000 7f947659c640 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56287c3f4a20 0x56287c3f6e40 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45369586f05032c server_cookie=98efd520ef65e6f5 in_seq=0 out_seq=0 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.320+0000 7f945dffb640 1 -- 172.21.15.105:0/1328874897 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56287c2e4940 con 0x56287c3f4a20 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f945dffb640 10 client.19329 handle_command_reply: tid=0 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 client.19329 shutdown 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 -- 172.21.15.105:0/1328874897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 msgr2=0x56287c388250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c388250 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f9470006010 tx=0x7f9470003e00 comp rx=0 tx=0).stop 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 -- 172.21.15.105:0/1328874897 shutdown_connections 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56287c3f4a20 0x56287c3f6e40 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56287c37e130 0x56287c388250 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56287c378fc0 0x56287c387240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f945dffb640 0 client.19329 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 --2- 172.21.15.105:0/1328874897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56287c3785d0 0x56287c386d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 -- 172.21.15.105:0/1328874897 >> 172.21.15.105:0/1328874897 conn(0x56287c373b40 msgr2=0x56287c381f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 -- 172.21.15.105:0/1328874897 shutdown_connections 2024-03-26T17:49:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 1 -- 172.21.15.105:0/1328874897 wait complete. 2024-03-26T17:49:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947f676740 20 client.19329 trim_cache size 0 max 16384 2024-03-26T17:49:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:49:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:49:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:49:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:49:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947ca85640 1 -- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f94640a3040 msgr2=0x7f94640a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947ca85640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f94640a3040 0x7f94640a5500 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9470008000 tx=0x7f947001f000 comp rx=0 tx=0).stop 2024-03-26T17:49:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947ca85640 1 -- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 msgr2=0x7f9478197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.321+0000 7f947ca85640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 0x7f9478197ac0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9468000c00 tx=0x7f9468005dc0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.322+0000 7f947ca85640 1 -- 172.21.15.105:0/3478203902 shutdown_connections 2024-03-26T17:49:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.322+0000 7f947ca85640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f94640a3040 0x7f94640a5500 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.322+0000 7f947ca85640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94781034d0 0x7f947819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.322+0000 7f947ca85640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9478102b10 0x7f9478198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.322+0000 7f947ca85640 1 --2- 172.21.15.105:0/3478203902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9478101920 0x7f9478197ac0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.322+0000 7f947ca85640 1 -- 172.21.15.105:0/3478203902 >> 172.21.15.105:0/3478203902 conn(0x7f94780fce90 msgr2=0x7f9478105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.322+0000 7f947ca85640 1 -- 172.21.15.105:0/3478203902 shutdown_connections 2024-03-26T17:49:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:00.322+0000 7f947ca85640 1 -- 172.21.15.105:0/3478203902 wait complete. 2024-03-26T17:49:01.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 342225 3.14 MB/sec execute 464 sec latency 39.658 ms 2024-03-26T17:49:02.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 342886 3.14 MB/sec execute 465 sec latency 18.146 ms 2024-03-26T17:49:02.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:01 smithi144 ceph-mon[36095]: pgmap v668: 97 pgs: 97 active+clean; 28 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:49:02.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:01 smithi105 ceph-mon[32280]: pgmap v668: 97 pgs: 97 active+clean; 28 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:49:02.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:01 smithi135 ceph-mon[35636]: pgmap v668: 97 pgs: 97 active+clean; 28 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:49:03.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 343510 3.15 MB/sec execute 466 sec latency 110.618 ms 2024-03-26T17:49:04.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 344087 3.15 MB/sec execute 467 sec latency 14.984 ms 2024-03-26T17:49:04.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:03 smithi144 ceph-mon[36095]: pgmap v669: 97 pgs: 97 active+clean; 28 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-26T17:49:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:03 smithi105 ceph-mon[32280]: pgmap v669: 97 pgs: 97 active+clean; 28 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-26T17:49:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:03 smithi135 ceph-mon[35636]: pgmap v669: 97 pgs: 97 active+clean; 28 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-26T17:49:05.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 344529 3.15 MB/sec execute 468 sec latency 17.236 ms 2024-03-26T17:49:06.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 345019 3.14 MB/sec execute 469 sec latency 22.456 ms 2024-03-26T17:49:06.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:05 smithi144 ceph-mon[36095]: pgmap v670: 97 pgs: 97 active+clean; 25 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:49:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:05 smithi105 ceph-mon[32280]: pgmap v670: 97 pgs: 97 active+clean; 25 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:49:06.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:05 smithi135 ceph-mon[35636]: pgmap v670: 97 pgs: 97 active+clean; 25 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-03-26T17:49:07.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 345641 3.14 MB/sec execute 470 sec latency 24.398 ms 2024-03-26T17:49:08.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 346252 3.14 MB/sec execute 471 sec latency 18.401 ms 2024-03-26T17:49:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:07 smithi144 ceph-mon[36095]: pgmap v671: 97 pgs: 97 active+clean; 25 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-03-26T17:49:08.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:07 smithi105 ceph-mon[32280]: pgmap v671: 97 pgs: 97 active+clean; 25 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-03-26T17:49:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:07 smithi135 ceph-mon[35636]: pgmap v671: 97 pgs: 97 active+clean; 25 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-03-26T17:49:09.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 346936 3.14 MB/sec execute 472 sec latency 19.670 ms 2024-03-26T17:49:10.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 347437 3.14 MB/sec execute 473 sec latency 24.999 ms 2024-03-26T17:49:10.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:09 smithi144 ceph-mon[36095]: pgmap v672: 97 pgs: 97 active+clean; 26 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:49:10.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:09 smithi105 ceph-mon[32280]: pgmap v672: 97 pgs: 97 active+clean; 26 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:49:10.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:09 smithi135 ceph-mon[35636]: pgmap v672: 97 pgs: 97 active+clean; 26 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:49:11.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 348004 3.15 MB/sec execute 474 sec latency 15.160 ms 2024-03-26T17:49:12.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 348483 3.14 MB/sec execute 475 sec latency 23.261 ms 2024-03-26T17:49:12.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:11 smithi144 ceph-mon[36095]: pgmap v673: 97 pgs: 97 active+clean; 26 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:49:12.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:11 smithi105 ceph-mon[32280]: pgmap v673: 97 pgs: 97 active+clean; 26 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:49:12.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:11 smithi135 ceph-mon[35636]: pgmap v673: 97 pgs: 97 active+clean; 26 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-03-26T17:49:13.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 349107 3.14 MB/sec execute 476 sec latency 19.027 ms 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 -- 172.21.15.105:0/4008226799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 msgr2=0x7fbe50101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/4008226799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 0x7fbe50101e50 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009900 tx=0x7fbe4402bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 -- 172.21.15.105:0/4008226799 shutdown_connections 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/4008226799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe50102390 0x7fbe50108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/4008226799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 0x7fbe50101e50 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/4008226799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe501007e0 0x7fbe50100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 -- 172.21.15.105:0/4008226799 >> 172.21.15.105:0/4008226799 conn(0x7fbe500fbd50 msgr2=0x7fbe500fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 -- 172.21.15.105:0/4008226799 shutdown_connections 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 -- 172.21.15.105:0/4008226799 wait complete. 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.420+0000 7fbe55e7e640 1 Processor -- start 2024-03-26T17:49:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe55e7e640 1 -- start start 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe55e7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe501007e0 0x7fbe50069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe55e7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 0x7fbe5019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe55e7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe50102390 0x7fbe5019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe55e7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010c310 con 0x7fbe501019d0 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe55e7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010c190 con 0x7fbe50102390 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe55e7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010c490 con 0x7fbe501007e0 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 0x7fbe5019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 0x7fbe5019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59096/0 (socket says 172.21.15.105:59096) 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 -- 172.21.15.105:0/497024851 learned_addr learned my addr 172.21.15.105:0/497024851 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4ffff640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe50102390 0x7fbe5019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe501007e0 0x7fbe50069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 -- 172.21.15.105:0/497024851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe501007e0 msgr2=0x7fbe50069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe501007e0 0x7fbe50069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 -- 172.21.15.105:0/497024851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe50102390 msgr2=0x7fbe5019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe50102390 0x7fbe5019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 -- 172.21.15.105:0/497024851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5019f700 con 0x7fbe501019d0 2024-03-26T17:49:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4effd640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 0x7fbe5019c7d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fbe44006010 tx=0x7fbe44005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.421+0000 7fbe4cff9640 1 -- 172.21.15.105:0/497024851 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44043020 con 0x7fbe501019d0 2024-03-26T17:49:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.422+0000 7fbe4cff9640 1 -- 172.21.15.105:0/497024851 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe440043d0 con 0x7fbe501019d0 2024-03-26T17:49:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.422+0000 7fbe55e7e640 1 -- 172.21.15.105:0/497024851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5019f930 con 0x7fbe501019d0 2024-03-26T17:49:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.422+0000 7fbe4cff9640 1 -- 172.21.15.105:0/497024851 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44003b80 con 0x7fbe501019d0 2024-03-26T17:49:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.422+0000 7fbe55e7e640 1 -- 172.21.15.105:0/497024851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5019fed0 con 0x7fbe501019d0 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.424+0000 7fbe4cff9640 1 -- 172.21.15.105:0/497024851 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbe44046020 con 0x7fbe501019d0 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe58a69740 1 Processor -- start 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe58a69740 1 -- start start 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4cff9640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe380a3040 0x7fbe380a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb445d0 0x562fdbb449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb49ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb4c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe58a69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562fdbaa90b0 con 0x562fdbb445d0 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe58a69740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562fdba51960 con 0x562fdbb44fc0 2024-03-26T17:49:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe58a69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562fdbaa5060 con 0x562fdbb4a130 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4cff9640 1 -- 172.21.15.105:0/497024851 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fbe440f15a0 con 0x7fbe501019d0 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4ffff640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe380a3040 0x7fbe380a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb445d0 0x562fdbb449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb445d0 0x562fdbb449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59108/0 (socket says 172.21.15.105:59108) 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/3468919567 learned_addr learned my addr 172.21.15.105:0/3468919567 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4effd640 1 --2- 172.21.15.105:0/3468919567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb49ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/3468919567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 msgr2=0x562fdbb4c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/3468919567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb4c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/3468919567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 msgr2=0x562fdbb49ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/3468919567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb49ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.425+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/3468919567 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdba63c60 con 0x562fdbb445d0 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.426+0000 7fbe4ffff640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe380a3040 0x7fbe380a5500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbe48005fd0 tx=0x7fbe48009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.426+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/3468919567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb445d0 0x562fdbb449f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009ca0 tx=0x7fbe40011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94fd17249429bd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.426+0000 7fbe35ffb640 1 -- 172.21.15.105:0/3468919567 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40004220 con 0x562fdbb445d0 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.426+0000 7fbe35ffb640 1 -- 172.21.15.105:0/3468919567 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe400043c0 con 0x562fdbb445d0 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.426+0000 7fbe35ffb640 1 -- 172.21.15.105:0/3468919567 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40018790 con 0x562fdbb445d0 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 -- 172.21.15.105:0/3468919567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb445d0 msgr2=0x562fdbb449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3468919567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb445d0 0x562fdbb449f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009ca0 tx=0x7fbe40011810 comp rx=0 tx=0).stop 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 -- 172.21.15.105:0/3468919567 shutdown_connections 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3468919567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb4c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3468919567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb49ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3468919567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb445d0 0x562fdbb449f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 -- 172.21.15.105:0/3468919567 >> 172.21.15.105:0/3468919567 conn(0x562fdbb3fb40 msgr2=0x562fdbb3ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 -- 172.21.15.105:0/3468919567 shutdown_connections 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 -- 172.21.15.105:0/3468919567 wait complete. 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.427+0000 7fbe58a69740 1 -- start start 2024-03-26T17:49:13.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe58a69740 1 Processor -- start 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb4e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb4e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe58a69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562fdbb4d0e0 con 0x562fdbb44fc0 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe58a69740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562fdba64db0 con 0x562fdbb445d0 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe58a69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562fdbb4d260 con 0x562fdbb4a130 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe58a69740 10 client.0 _ll_register_callbacks cb 0x562fdba69110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb4e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb4e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb4e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59118/0 (socket says 172.21.15.105:59118) 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4effd640 1 -- 172.21.15.105:0/3249605852 learned_addr learned my addr 172.21.15.105:0/3249605852 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4ffff640 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/3249605852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 msgr2=0x562fdbb558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/3249605852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 msgr2=0x562fdbb4e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb4e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.428+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/3249605852 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdb7bd210 con 0x562fdbb445d0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb4e350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009c70 tx=0x7fbe40005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7490a720b757f25a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe2effd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe2effd640 1 -- 172.21.15.105:0/3249605852 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40027020 con 0x562fdbb445d0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe58a69740 1 -- 172.21.15.105:0/3249605852 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe400043c0 con 0x562fdbb445d0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe2effd640 1 -- 172.21.15.105:0/3249605852 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fbe400035f0 con 0x562fdbb445d0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe2effd640 1 -- 172.21.15.105:0/3249605852 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe40032050 con 0x562fdbb445d0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe58a69740 10 client.29153 fetch_fsmap learned FSMap version 30 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe58a69740 1 -- 172.21.15.105:0/3249605852 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562fdb889620 con 0x562fdbb445d0 2024-03-26T17:49:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe2effd640 1 -- 172.21.15.105:0/3249605852 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe400043c0 con 0x562fdbb445d0 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe2effd640 1 -- 172.21.15.105:0/3249605852 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbe40004b80 con 0x562fdbb445d0 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.429+0000 7fbe58a69740 10 client.29153 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.430+0000 7fbe58a69740 10 client.29153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.430+0000 7fbe58a69740 20 client.29153 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.430+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562fdbb5e340 0x562fdbb60730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.430+0000 7fbe58a69740 4 client.29153 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.430+0000 7fbe58a69740 1 -- 172.21.15.105:0/3249605852 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fdbab65f0 con 0x562fdbb5e340 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.430+0000 7fbe4effd640 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562fdbb5e340 0x562fdbb60730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.430+0000 7fbe2effd640 10 client.29153 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.430+0000 7fbe4effd640 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562fdbb5e340 0x562fdbb60730 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71779419fd5bda26 server_cookie=4727073289f83bae in_seq=0 out_seq=0 2024-03-26T17:49:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe2effd640 1 -- 172.21.15.105:0/3249605852 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562fdbab65f0 con 0x562fdbb5e340 2024-03-26T17:49:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe2effd640 10 client.29153 handle_command_reply: tid=0 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 client.29153 shutdown 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 -- 172.21.15.105:0/3249605852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 msgr2=0x562fdbb4e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb4e350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009c70 tx=0x7fbe40005270 comp rx=0 tx=0).stop 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 -- 172.21.15.105:0/3249605852 shutdown_connections 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562fdbb5e340 0x562fdbb60730 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb4e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 --2- 172.21.15.105:0/3249605852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb4e350 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 -- 172.21.15.105:0/3249605852 >> 172.21.15.105:0/3249605852 conn(0x562fdbb3fb40 msgr2=0x562fdbb3edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe2effd640 0 client.29153 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 -- 172.21.15.105:0/3249605852 shutdown_connections 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 1 -- 172.21.15.105:0/3249605852 wait complete. 2024-03-26T17:49:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.433+0000 7fbe58a69740 20 client.29153 trim_cache size 0 max 16384 2024-03-26T17:49:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe58a69740 1 Processor -- start 2024-03-26T17:49:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe58a69740 1 -- start start 2024-03-26T17:49:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb445d0 0x562fdbb3ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb55890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb4a130 0x562fdbb3e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe58a69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501a6ba0 con 0x562fdbb4a130 2024-03-26T17:49:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe58a69740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501a86e0 con 0x562fdbb44fc0 2024-03-26T17:49:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe58a69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501a7270 con 0x562fdbb445d0 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe4effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb55890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe4f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb445d0 0x562fdbb3ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe4ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb4a130 0x562fdbb3e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe4effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb55890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45790/0 (socket says 172.21.15.105:45790) 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe4f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb445d0 0x562fdbb3ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54574/0 (socket says 172.21.15.105:54574) 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.449+0000 7fbe4effd640 1 -- 172.21.15.105:0/2547603958 learned_addr learned my addr 172.21.15.105:0/2547603958 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe4ffff640 1 -- 172.21.15.105:0/2547603958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb445d0 msgr2=0x562fdbb3ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe4ffff640 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb445d0 0x562fdbb3ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe4ffff640 1 -- 172.21.15.105:0/2547603958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 msgr2=0x562fdbb55890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe4ffff640 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb55890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe4ffff640 1 -- 172.21.15.105:0/2547603958 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdbb405b0 con 0x562fdbb4a130 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb445d0 0x562fdbb3ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe4effd640 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb55890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe4ffff640 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb4a130 0x562fdbb3e560 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbe48011ca0 tx=0x7fbe4801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9325641b2b0b2348 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe357fa640 1 -- 172.21.15.105:0/2547603958 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48017030 con 0x562fdbb4a130 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe357fa640 1 -- 172.21.15.105:0/2547603958 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe48019050 con 0x562fdbb4a130 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.450+0000 7fbe357fa640 1 -- 172.21.15.105:0/2547603958 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe480059e0 con 0x562fdbb4a130 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- 172.21.15.105:0/2547603958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb4a130 msgr2=0x562fdbb3e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb4a130 0x562fdbb3e560 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbe48011ca0 tx=0x7fbe4801a000 comp rx=0 tx=0).stop 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- 172.21.15.105:0/2547603958 shutdown_connections 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb4a130 0x562fdbb3e560 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb44fc0 0x562fdbb55890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2547603958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb445d0 0x562fdbb3ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- 172.21.15.105:0/2547603958 >> 172.21.15.105:0/2547603958 conn(0x562fdbb3fb40 msgr2=0x562fdbb43630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- 172.21.15.105:0/2547603958 shutdown_connections 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- 172.21.15.105:0/2547603958 wait complete. 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- start start 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 Processor -- start 2024-03-26T17:49:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb52a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb52f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb53f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501a8010 con 0x562fdbb44fc0 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501a7940 con 0x562fdbb445d0 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.451+0000 7fbe58a69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501a4b80 con 0x562fdbb4a130 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe58a69740 10 client.0 _ll_register_callbacks cb 0x7fbe44005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb52a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb52a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45800/0 (socket says 172.21.15.105:45800) 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/2135881486 learned_addr learned my addr 172.21.15.105:0/2135881486 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4effd640 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb52f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4ffff640 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb53f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/2135881486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 msgr2=0x562fdbb53f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb53f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/2135881486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 msgr2=0x562fdbb52f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:13.449 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb52f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 -- 172.21.15.105:0/2135881486 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdbb54500 con 0x562fdbb445d0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4effd640 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb52f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe4f7fe640 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb52a30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000be0 tx=0x7fbe400183d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6144dfae5e0092 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe2effd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe2effd640 1 -- 172.21.15.105:0/2135881486 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40027040 con 0x562fdbb445d0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe2effd640 1 -- 172.21.15.105:0/2135881486 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8293+0+0 (secure 0 0 0) 0x7fbe40018a70 con 0x562fdbb445d0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.452+0000 7fbe58a69740 1 -- 172.21.15.105:0/2135881486 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe48019050 con 0x562fdbb445d0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe2effd640 1 -- 172.21.15.105:0/2135881486 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe4001e040 con 0x562fdbb445d0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe58a69740 10 client.29159 fetch_fsmap learned FSMap version 30 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe58a69740 1 -- 172.21.15.105:0/2135881486 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562fdbb55810 con 0x562fdbb445d0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe2effd640 1 -- 172.21.15.105:0/2135881486 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe48019050 con 0x562fdbb445d0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe2effd640 1 -- 172.21.15.105:0/2135881486 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbe4003e030 con 0x562fdbb445d0 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe58a69740 10 client.29159 fetch_fsmap finished waiting for FSMap version 30 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe58a69740 10 client.29159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:13.450 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe58a69740 20 client.29159 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562fdbbc4930 0x562fdbbc6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe58a69740 4 client.29159 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:49:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe58a69740 1 -- 172.21.15.105:0/2135881486 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fdbab0940 con 0x562fdbbc4930 2024-03-26T17:49:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe4effd640 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562fdbbc4930 0x562fdbbc6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe4effd640 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562fdbbc4930 0x562fdbbc6d20 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d9e695c9fb7791b server_cookie=d53740c2e81fec90 in_seq=0 out_seq=0 2024-03-26T17:49:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.453+0000 7fbe2effd640 10 client.29159 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe2effd640 1 -- 172.21.15.105:0/2135881486 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fdbab0940 con 0x562fdbbc4930 2024-03-26T17:49:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe2effd640 10 client.29159 handle_command_reply: tid=0 2024-03-26T17:49:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 client.29159 shutdown 2024-03-26T17:49:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 -- 172.21.15.105:0/2135881486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 msgr2=0x562fdbb52a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb52a30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbe40000be0 tx=0x7fbe400183d0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 -- 172.21.15.105:0/2135881486 shutdown_connections 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562fdbbc4930 0x562fdbbc6d20 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562fdbb4a130 0x562fdbb53f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562fdbb44fc0 0x562fdbb52f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe2effd640 0 client.29159 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 --2- 172.21.15.105:0/2135881486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fdbb445d0 0x562fdbb52a30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 -- 172.21.15.105:0/2135881486 >> 172.21.15.105:0/2135881486 conn(0x562fdbb3fb40 msgr2=0x562fdbb3e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.495+0000 7fbe58a69740 1 -- 172.21.15.105:0/2135881486 shutdown_connections 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe58a69740 1 -- 172.21.15.105:0/2135881486 wait complete. 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe58a69740 20 client.29159 trim_cache size 0 max 16384 2024-03-26T17:49:13.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:49:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:49:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:49:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:49:13.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 -- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe380a3040 msgr2=0x7fbe380a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe380a3040 0x7fbe380a5500 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fbe48005fd0 tx=0x7fbe48009f90 comp rx=0 tx=0).stop 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 -- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 msgr2=0x7fbe5019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 0x7fbe5019c7d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fbe44006010 tx=0x7fbe44005e40 comp rx=0 tx=0).stop 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 -- 172.21.15.105:0/497024851 shutdown_connections 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe380a3040 0x7fbe380a5500 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe50102390 0x7fbe5019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe501019d0 0x7fbe5019c7d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 --2- 172.21.15.105:0/497024851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe501007e0 0x7fbe50069ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.496+0000 7fbe55e7e640 1 -- 172.21.15.105:0/497024851 >> 172.21.15.105:0/497024851 conn(0x7fbe500fbd50 msgr2=0x7fbe50104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.497+0000 7fbe55e7e640 1 -- 172.21.15.105:0/497024851 shutdown_connections 2024-03-26T17:49:13.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:13.497+0000 7fbe55e7e640 1 -- 172.21.15.105:0/497024851 wait complete. 2024-03-26T17:49:14.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 349760 3.14 MB/sec execute 477 sec latency 17.241 ms 2024-03-26T17:49:14.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:13 smithi144 ceph-mon[36095]: pgmap v674: 97 pgs: 97 active+clean; 28 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:49:14.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:13 smithi105 ceph-mon[32280]: pgmap v674: 97 pgs: 97 active+clean; 28 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:49:14.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:13 smithi135 ceph-mon[35636]: pgmap v674: 97 pgs: 97 active+clean; 28 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-03-26T17:49:14.199 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:49:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setuser ceph since I am not root 2024-03-26T17:49:14.199 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:49:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setgroup ceph since I am not root 2024-03-26T17:49:14.199 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:49:13 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/3154884972 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55988f095a40 con 0x55988f069000 2024-03-26T17:49:14.199 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:49:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T17:49:15.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 350388 3.14 MB/sec execute 478 sec latency 20.043 ms 2024-03-26T17:49:15.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:14 smithi144 ceph-mon[36095]: daemon mds.e restarted 2024-03-26T17:49:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:14 smithi105 ceph-mon[32280]: daemon mds.e restarted 2024-03-26T17:49:15.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:14 smithi135 ceph-mon[35636]: daemon mds.e restarted 2024-03-26T17:49:16.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 350968 3.14 MB/sec execute 479 sec latency 14.886 ms 2024-03-26T17:49:16.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:15 smithi144 ceph-mon[36095]: pgmap v675: 97 pgs: 97 active+clean; 28 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:49:16.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:15 smithi144 ceph-mon[36095]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T17:49:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:15 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/3150554209,v1:172.21.15.135:6835/3150554209] up:boot 2024-03-26T17:49:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:15 smithi144 ceph-mon[36095]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:49:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:15 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:49:16.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:15 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:49:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:15 smithi105 ceph-mon[32280]: pgmap v675: 97 pgs: 97 active+clean; 28 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:49:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:15 smithi105 ceph-mon[32280]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T17:49:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:15 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/3150554209,v1:172.21.15.135:6835/3150554209] up:boot 2024-03-26T17:49:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:15 smithi105 ceph-mon[32280]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:49:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:15 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:49:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:15 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:49:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:15 smithi135 ceph-mon[35636]: pgmap v675: 97 pgs: 97 active+clean; 28 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:49:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:15 smithi135 ceph-mon[35636]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T17:49:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:15 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/3150554209,v1:172.21.15.135:6835/3150554209] up:boot 2024-03-26T17:49:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:15 smithi135 ceph-mon[35636]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:49:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:15 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:49:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:15 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:49:17.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 351542 3.15 MB/sec execute 480 sec latency 15.966 ms 2024-03-26T17:49:18.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 352029 3.14 MB/sec execute 481 sec latency 18.449 ms 2024-03-26T17:49:18.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:17 smithi144 ceph-mon[36095]: pgmap v677: 97 pgs: 97 active+clean; 28 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:49:18.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:17 smithi105 ceph-mon[32280]: pgmap v677: 97 pgs: 97 active+clean; 28 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:49:18.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:17 smithi135 ceph-mon[35636]: pgmap v677: 97 pgs: 97 active+clean; 28 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:49:19.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 352651 3.14 MB/sec execute 482 sec latency 17.550 ms 2024-03-26T17:49:20.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 353336 3.14 MB/sec execute 483 sec latency 16.617 ms 2024-03-26T17:49:20.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:19 smithi144 ceph-mon[36095]: pgmap v678: 97 pgs: 97 active+clean; 25 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:49:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:19 smithi105 ceph-mon[32280]: pgmap v678: 97 pgs: 97 active+clean; 25 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:49:20.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:19 smithi135 ceph-mon[35636]: pgmap v678: 97 pgs: 97 active+clean; 25 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:49:21.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 354028 3.14 MB/sec execute 484 sec latency 18.077 ms 2024-03-26T17:49:22.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 354496 3.14 MB/sec execute 485 sec latency 23.424 ms 2024-03-26T17:49:22.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:21 smithi144 ceph-mon[36095]: pgmap v679: 97 pgs: 97 active+clean; 26 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:49:22.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:21 smithi105 ceph-mon[32280]: pgmap v679: 97 pgs: 97 active+clean; 26 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:49:22.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:21 smithi135 ceph-mon[35636]: pgmap v679: 97 pgs: 97 active+clean; 26 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-03-26T17:49:23.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 355075 3.15 MB/sec execute 486 sec latency 15.953 ms 2024-03-26T17:49:24.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 355518 3.14 MB/sec execute 487 sec latency 21.749 ms 2024-03-26T17:49:24.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:49:24.041 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:49:24.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:23 smithi144 ceph-mon[36095]: pgmap v680: 97 pgs: 97 active+clean; 25 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-26T17:49:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:23 smithi105 ceph-mon[32280]: pgmap v680: 97 pgs: 97 active+clean; 25 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-26T17:49:24.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 -- 172.21.15.105:0/1810850098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 msgr2=0x7f6f18101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 --2- 172.21.15.105:0/1810850098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 0x7f6f18101d10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009900 tx=0x7f6f1402bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 -- 172.21.15.105:0/1810850098 shutdown_connections 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 --2- 172.21.15.105:0/1810850098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f181034a0 0x7f6f18109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 --2- 172.21.15.105:0/1810850098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f18102ae0 0x7f6f18102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 --2- 172.21.15.105:0/1810850098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 0x7f6f18101d10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 -- 172.21.15.105:0/1810850098 >> 172.21.15.105:0/1810850098 conn(0x7f6f180fce60 msgr2=0x7f6f180ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 -- 172.21.15.105:0/1810850098 shutdown_connections 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.163+0000 7f6f2009a640 1 -- 172.21.15.105:0/1810850098 wait complete. 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f2009a640 1 Processor -- start 2024-03-26T17:49:24.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f2009a640 1 -- start start 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f2009a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 0x7f6f18197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f2009a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f18102ae0 0x7f6f18198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f2009a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f181034a0 0x7f6f1819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f2009a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1810d3f0 con 0x7f6f181018f0 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f2009a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1810d270 con 0x7f6f181034a0 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f2009a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1810d570 con 0x7f6f18102ae0 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f1de36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 0x7f6f18197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f1de36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 0x7f6f18197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60964/0 (socket says 172.21.15.105:60964) 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f1de36640 1 -- 172.21.15.105:0/2507666023 learned_addr learned my addr 172.21.15.105:0/2507666023 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f1d635640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f18102ae0 0x7f6f18198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.164+0000 7f6f1e637640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f181034a0 0x7f6f1819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f1de36640 1 -- 172.21.15.105:0/2507666023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f18102ae0 msgr2=0x7f6f18198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f1de36640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f18102ae0 0x7f6f18198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f1de36640 1 -- 172.21.15.105:0/2507666023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f181034a0 msgr2=0x7f6f1819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f1de36640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f181034a0 0x7f6f1819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f1de36640 1 -- 172.21.15.105:0/2507666023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f18073af0 con 0x7f6f181018f0 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f1de36640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 0x7f6f18197ad0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000c00 tx=0x7f6f14004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f0effd640 1 -- 172.21.15.105:0/2507666023 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14043020 con 0x7f6f181018f0 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f0effd640 1 -- 172.21.15.105:0/2507666023 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f14004440 con 0x7f6f181018f0 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f2009a640 1 -- 172.21.15.105:0/2507666023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f18073d80 con 0x7f6f181018f0 2024-03-26T17:49:24.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f2009a640 1 -- 172.21.15.105:0/2507666023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f18074320 con 0x7f6f181018f0 2024-03-26T17:49:24.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.165+0000 7f6f0effd640 1 -- 172.21.15.105:0/2507666023 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14003c20 con 0x7f6f181018f0 2024-03-26T17:49:24.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.167+0000 7f6f0effd640 1 -- 172.21.15.105:0/2507666023 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6f14046020 con 0x7f6f181018f0 2024-03-26T17:49:24.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f22d66740 1 Processor -- start 2024-03-26T17:49:24.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f22d66740 1 -- start start 2024-03-26T17:49:24.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786329d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632a2e0 0x56178632ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f22d66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56178628e980 con 0x56178632f450 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f22d66740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561786236be0 con 0x5617863298f0 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f22d66740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56178628a930 con 0x56178632a2e0 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f1e637640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786329d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f1d635640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f1e637640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786329d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41832/0 (socket says 172.21.15.105:41832) 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.168+0000 7f6f1d635640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863318e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60972/0 (socket says 172.21.15.105:60972) 2024-03-26T17:49:24.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1d635640 1 -- 172.21.15.105:0/254921567 learned_addr learned my addr 172.21.15.105:0/254921567 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1de36640 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632a2e0 0x56178632ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f0effd640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f040a2fa0 0x7f6f040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1d635640 1 -- 172.21.15.105:0/254921567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632a2e0 msgr2=0x56178632ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1d635640 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632a2e0 0x56178632ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1d635640 1 -- 172.21.15.105:0/254921567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 msgr2=0x561786329d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1d635640 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786329d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1d635640 1 -- 172.21.15.105:0/254921567 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56178606c7d0 con 0x56178632f450 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1e637640 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786329d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1d635640 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863318e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800aca0 tx=0x7f6f0800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec569bc4a2e63fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f0effd640 1 -- 172.21.15.105:0/2507666023 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6f140f11d0 con 0x7f6f181018f0 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f1e637640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f040a2fa0 0x7f6f040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f03fff640 1 -- 172.21.15.105:0/254921567 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0801d070 con 0x56178632f450 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f03fff640 1 -- 172.21.15.105:0/254921567 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f08018af0 con 0x56178632f450 2024-03-26T17:49:24.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.169+0000 7f6f03fff640 1 -- 172.21.15.105:0/254921567 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08010b50 con 0x56178632f450 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 -- 172.21.15.105:0/254921567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 msgr2=0x5617863318e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f1e637640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f040a2fa0 0x7f6f040a5460 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x56178632ab40 tx=0x7f6f10004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863318e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800aca0 tx=0x7f6f0800f790 comp rx=0 tx=0).stop 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 -- 172.21.15.105:0/254921567 shutdown_connections 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863318e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632a2e0 0x56178632ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 --2- 172.21.15.105:0/254921567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786329d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 -- 172.21.15.105:0/254921567 >> 172.21.15.105:0/254921567 conn(0x561786324e60 msgr2=0x561786325260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 -- 172.21.15.105:0/254921567 shutdown_connections 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.170+0000 7f6f22d66740 1 -- 172.21.15.105:0/254921567 wait complete. 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 1 -- start start 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 1 Processor -- start 2024-03-26T17:49:24.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786333b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632a2e0 0x561786334040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632f450 0x56178633b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561786332400 con 0x56178632a2e0 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f1d635640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632a2e0 0x561786334040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f1d635640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632a2e0 0x561786334040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60978/0 (socket says 172.21.15.105:60978) 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56178624a160 con 0x5617863298f0 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f1d635640 1 -- 172.21.15.105:0/2173022451 learned_addr learned my addr 172.21.15.105:0/2173022451 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f1de36640 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786333b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561786332580 con 0x56178632f450 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.171+0000 7f6f22d66740 10 client.0 _ll_register_callbacks cb 0x561786286720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f1e637640 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632f450 0x56178633b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f1de36640 1 -- 172.21.15.105:0/2173022451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632f450 msgr2=0x56178633b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f1de36640 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632f450 0x56178633b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f1de36640 1 -- 172.21.15.105:0/2173022451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632a2e0 msgr2=0x561786334040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f1de36640 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632a2e0 0x561786334040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f1de36640 1 -- 172.21.15.105:0/2173022451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561785fa23f0 con 0x5617863298f0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f1d635640 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632a2e0 0x561786334040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f1de36640 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786333b00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f140eece0 tx=0x7f6f14005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82aa9a371266d73 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f017fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f017fa640 1 -- 172.21.15.105:0/2173022451 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f140bd400 con 0x5617863298f0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f08018af0 con 0x5617863298f0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f017fa640 1 -- 172.21.15.105:0/2173022451 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6f140bbbc0 con 0x5617863298f0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f017fa640 1 -- 172.21.15.105:0/2173022451 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f140bd220 con 0x5617863298f0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f22d66740 10 client.29177 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.172+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561785f620f0 con 0x5617863298f0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f017fa640 1 -- 172.21.15.105:0/2173022451 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f08018af0 con 0x5617863298f0 2024-03-26T17:49:24.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f017fa640 1 -- 172.21.15.105:0/2173022451 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6f140a80b0 con 0x5617863298f0 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f22d66740 10 client.29177 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f22d66740 10 client.29177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f22d66740 20 client.29177 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f22d66740 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56178633f870 0x561786341d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f22d66740 4 client.29177 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56178629bec0 con 0x56178633f870 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f1d635640 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56178633f870 0x561786341d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f017fa640 10 client.29177 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:24.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.173+0000 7f6f1d635640 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56178633f870 0x561786341d00 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00af2641d369262 server_cookie=b6faf08f4da80fc in_seq=0 out_seq=0 2024-03-26T17:49:24.173 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:23 smithi135 ceph-mon[35636]: pgmap v680: 97 pgs: 97 active+clean; 25 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-26T17:49:24.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f017fa640 1 -- 172.21.15.105:0/2173022451 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56178629bec0 con 0x56178633f870 2024-03-26T17:49:24.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f017fa640 10 client.29177 handle_command_reply: tid=0 2024-03-26T17:49:24.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 client.29177 shutdown 2024-03-26T17:49:24.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 msgr2=0x561786333b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786333b00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f140eece0 tx=0x7f6f14005bf0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 shutdown_connections 2024-03-26T17:49:24.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56178633f870 0x561786341d00 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56178632f450 0x56178633b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632a2e0 0x561786334040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f017fa640 0 client.29177 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:24.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 --2- 172.21.15.105:0/2173022451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617863298f0 0x561786333b00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 >> 172.21.15.105:0/2173022451 conn(0x561786324e60 msgr2=0x561786324080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 shutdown_connections 2024-03-26T17:49:24.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 1 -- 172.21.15.105:0/2173022451 wait complete. 2024-03-26T17:49:24.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.175+0000 7f6f22d66740 20 client.29177 trim_cache size 0 max 16384 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f22d66740 1 Processor -- start 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f22d66740 1 -- start start 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x561786333fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 0x56178633b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x56178632eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f22d66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181aafc0 con 0x56178632f450 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f22d66740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181acb00 con 0x56178632a2e0 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f22d66740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181ab690 con 0x5617863298f0 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x56178632eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x56178632eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60994/0 (socket says 172.21.15.105:60994) 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 -- 172.21.15.105:0/1559542323 learned_addr learned my addr 172.21.15.105:0/1559542323 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1de36640 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x561786333fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1d635640 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 0x56178633b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 -- 172.21.15.105:0/1559542323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 msgr2=0x561786333fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x561786333fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 -- 172.21.15.105:0/1559542323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 msgr2=0x56178633b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 0x56178633b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 -- 172.21.15.105:0/1559542323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617863326c0 con 0x56178632f450 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.193+0000 7f6f1e637640 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x56178632eb30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000fda0 tx=0x7f6f10013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202f9f13ac2b9a73 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f037fe640 1 -- 172.21.15.105:0/1559542323 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10029480 con 0x56178632f450 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f037fe640 1 -- 172.21.15.105:0/1559542323 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f10029ad0 con 0x56178632f450 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f037fe640 1 -- 172.21.15.105:0/1559542323 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1001cbc0 con 0x56178632f450 2024-03-26T17:49:24.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 -- 172.21.15.105:0/1559542323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 msgr2=0x56178632eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x56178632eb30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000fda0 tx=0x7f6f10013030 comp rx=0 tx=0).stop 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 -- 172.21.15.105:0/1559542323 shutdown_connections 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x56178632eb30 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 0x56178633b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 --2- 172.21.15.105:0/1559542323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x561786333fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 -- 172.21.15.105:0/1559542323 >> 172.21.15.105:0/1559542323 conn(0x561786324e60 msgr2=0x561786332920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 -- 172.21.15.105:0/1559542323 shutdown_connections 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.194+0000 7f6f22d66740 1 -- 172.21.15.105:0/1559542323 wait complete. 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 1 -- start start 2024-03-26T17:49:24.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 1 Processor -- start 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x5617863383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 0x561786338900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181ac430 con 0x56178632f450 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181abd60 con 0x56178632a2e0 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6f181a8fa0 con 0x5617863298f0 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f22d66740 10 client.0 _ll_register_callbacks cb 0x561786286720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f1de36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x5617863383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f1de36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x5617863383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59644/0 (socket says 172.21.15.105:59644) 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f1de36640 1 -- 172.21.15.105:0/339234401 learned_addr learned my addr 172.21.15.105:0/339234401 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.195+0000 7f6f1d635640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 0x561786338900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f1e637640 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f1de36640 1 -- 172.21.15.105:0/339234401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 msgr2=0x561786338900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f1de36640 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 0x561786338900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f1de36640 1 -- 172.21.15.105:0/339234401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 msgr2=0x5617863398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f1de36640 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f1de36640 1 -- 172.21.15.105:0/339234401 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561786339eb0 con 0x5617863298f0 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f1de36640 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x5617863383c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6f140bd850 tx=0x7f6f140c62d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d273342f4904cc97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f017fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f017fa640 1 -- 172.21.15.105:0/339234401 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f140bd400 con 0x5617863298f0 2024-03-26T17:49:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f22d66740 1 -- 172.21.15.105:0/339234401 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f10029ad0 con 0x5617863298f0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f017fa640 1 -- 172.21.15.105:0/339234401 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6f140bcd10 con 0x5617863298f0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f017fa640 1 -- 172.21.15.105:0/339234401 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f140c1a10 con 0x5617863298f0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f22d66740 10 client.29170 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.196+0000 7f6f22d66740 1 -- 172.21.15.105:0/339234401 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56178633b1c0 con 0x5617863298f0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f017fa640 1 -- 172.21.15.105:0/339234401 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f10029ad0 con 0x5617863298f0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f017fa640 1 -- 172.21.15.105:0/339234401 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6f140dc1e0 con 0x5617863298f0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f22d66740 10 client.29170 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f22d66740 10 client.29170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f22d66740 20 client.29170 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f22d66740 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5617863a60e0 0x5617863a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f22d66740 4 client.29170 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f22d66740 1 -- 172.21.15.105:0/339234401 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561786296210 con 0x5617863a60e0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f1d635640 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5617863a60e0 0x5617863a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f1d635640 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5617863a60e0 0x5617863a84d0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=677def085bb9e2b6 server_cookie=31bff3751b11f526 in_seq=0 out_seq=0 2024-03-26T17:49:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.197+0000 7f6f017fa640 10 client.29170 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f017fa640 1 -- 172.21.15.105:0/339234401 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561786296210 con 0x5617863a60e0 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f017fa640 10 client.29170 handle_command_reply: tid=0 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 client.29170 shutdown 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 -- 172.21.15.105:0/339234401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 msgr2=0x5617863383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x5617863383c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6f140bd850 tx=0x7f6f140c62d0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 -- 172.21.15.105:0/339234401 shutdown_connections 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5617863a60e0 0x5617863a84d0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56178632f450 0x5617863398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f017fa640 0 client.29170 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56178632a2e0 0x561786338900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 --2- 172.21.15.105:0/339234401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5617863298f0 0x5617863383c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 -- 172.21.15.105:0/339234401 >> 172.21.15.105:0/339234401 conn(0x561786324e60 msgr2=0x56178632eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 -- 172.21.15.105:0/339234401 shutdown_connections 2024-03-26T17:49:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 1 -- 172.21.15.105:0/339234401 wait complete. 2024-03-26T17:49:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.198+0000 7f6f22d66740 20 client.29170 trim_cache size 0 max 16384 2024-03-26T17:49:24.197 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:49:24.197 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:49:24.197 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:49:24.197 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:49:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 -- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f040a2fa0 msgr2=0x7f6f040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f040a2fa0 0x7f6f040a5460 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x56178632ab40 tx=0x7f6f10004a40 comp rx=0 tx=0).stop 2024-03-26T17:49:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 -- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 msgr2=0x7f6f18197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 0x7f6f18197ad0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6f14000c00 tx=0x7f6f14004040 comp rx=0 tx=0).stop 2024-03-26T17:49:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 -- 172.21.15.105:0/2507666023 shutdown_connections 2024-03-26T17:49:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f040a2fa0 0x7f6f040a5460 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f181034a0 0x7f6f1819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f18102ae0 0x7f6f18198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 --2- 172.21.15.105:0/2507666023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f181018f0 0x7f6f18197ad0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 -- 172.21.15.105:0/2507666023 >> 172.21.15.105:0/2507666023 conn(0x7f6f180fce60 msgr2=0x7f6f18105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 -- 172.21.15.105:0/2507666023 shutdown_connections 2024-03-26T17:49:24.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.199+0000 7f6f2009a640 1 -- 172.21.15.105:0/2507666023 wait complete. 2024-03-26T17:49:24.204 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:49:24.204 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:49:24.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.327+0000 7fe8f9e85640 1 -- 172.21.15.105:0/1299159065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 msgr2=0x7fe8f4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.327+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/1299159065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 0x7fe8f4104750 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4009900 tx=0x7fe8e402bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.327+0000 7fe8f9e85640 1 -- 172.21.15.105:0/1299159065 shutdown_connections 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.327+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/1299159065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8f41075c0 0x7fe8f4109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.327+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/1299159065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f4104c90 0x7fe8f4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.327+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/1299159065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 0x7fe8f4104750 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.327+0000 7fe8f9e85640 1 -- 172.21.15.105:0/1299159065 >> 172.21.15.105:0/1299159065 conn(0x7fe8f40fbd50 msgr2=0x7fe8f40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 -- 172.21.15.105:0/1299159065 shutdown_connections 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 -- 172.21.15.105:0/1299159065 wait complete. 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 Processor -- start 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 -- start start 2024-03-26T17:49:24.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 0x7fe8f4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8f4104c90 0x7fe8f41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41075c0 0x7fe8f419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f410d430 con 0x7fe8f4102360 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f410d2b0 con 0x7fe8f41075c0 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.328+0000 7fe8f9e85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f410d5b0 con 0x7fe8f4104c90 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 0x7fe8f4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8f4104c90 0x7fe8f41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 0x7fe8f4197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:32778/0 (socket says 172.21.15.105:32778) 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 -- 172.21.15.105:0/2868758756 learned_addr learned my addr 172.21.15.105:0/2868758756 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f3fff640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41075c0 0x7fe8f419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 -- 172.21.15.105:0/2868758756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8f4104c90 msgr2=0x7fe8f41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8f4104c90 0x7fe8f41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 -- 172.21.15.105:0/2868758756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41075c0 msgr2=0x7fe8f419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41075c0 0x7fe8f419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 -- 172.21.15.105:0/2868758756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f419f780 con 0x7fe8f4102360 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f2ffd640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8f4104c90 0x7fe8f41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f3fff640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41075c0 0x7fe8f419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f37fe640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 0x7fe8f4197b60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006010 tx=0x7fe8e4005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f0ff9640 1 -- 172.21.15.105:0/2868758756 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4043020 con 0x7fe8f4102360 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f419fa10 con 0x7fe8f4102360 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f0ff9640 1 -- 172.21.15.105:0/2868758756 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8e40043d0 con 0x7fe8f4102360 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f0ff9640 1 -- 172.21.15.105:0/2868758756 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4003b80 con 0x7fe8f4102360 2024-03-26T17:49:24.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.329+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f419ff60 con 0x7fe8f4102360 2024-03-26T17:49:24.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.330+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f40fd840 con 0x7fe8f4102360 2024-03-26T17:49:24.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.332+0000 7fe8f0ff9640 1 -- 172.21.15.105:0/2868758756 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe8e4046020 con 0x7fe8f4102360 2024-03-26T17:49:24.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.332+0000 7fe8f0ff9640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8dc0a2f20 0x7fe8dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.332+0000 7fe8f2ffd640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8dc0a2f20 0x7fe8dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.333+0000 7fe8f0ff9640 1 -- 172.21.15.105:0/2868758756 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe8e40f1600 con 0x7fe8f4102360 2024-03-26T17:49:24.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.333+0000 7fe8f2ffd640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8dc0a2f20 0x7fe8dc0a53e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4199070 tx=0x7fe8e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.337+0000 7fe8f0ff9640 1 -- 172.21.15.105:0/2868758756 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe8e404d020 con 0x7fe8f4102360 2024-03-26T17:49:24.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.608+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8f40624c0 con 0x7fe8f4102360 2024-03-26T17:49:24.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.609+0000 7fe8f0ff9640 1 -- 172.21.15.105:0/2868758756 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11941 (secure 0 0 0) 0x7fe8e40be3f0 con 0x7fe8f4102360 2024-03-26T17:49:24.601 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:49:24.601 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:49:14.826136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24482,14628,24487,24466]},"id":1}]} 2024-03-26T17:49:24.602 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 32 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8dc0a2f20 msgr2=0x7fe8dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8dc0a2f20 0x7fe8dc0a53e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4199070 tx=0x7fe8e8005f90 comp rx=0 tx=0).stop 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 msgr2=0x7fe8f4197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 0x7fe8f4197b60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006010 tx=0x7fe8e4005e40 comp rx=0 tx=0).stop 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 shutdown_connections 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8dc0a2f20 0x7fe8dc0a53e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f41075c0 0x7fe8f419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8f4104c90 0x7fe8f41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 --2- 172.21.15.105:0/2868758756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8f4102360 0x7fe8f4197b60 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.611+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 >> 172.21.15.105:0/2868758756 conn(0x7fe8f40fbd50 msgr2=0x7fe8f4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.612+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 shutdown_connections 2024-03-26T17:49:24.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.612+0000 7fe8f9e85640 1 -- 172.21.15.105:0/2868758756 wait complete. 2024-03-26T17:49:24.616 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 -- 172.21.15.105:0/1667167379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec100920 msgr2=0x7febec100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 --2- 172.21.15.105:0/1667167379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec100920 0x7febec100da0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7febd8009860 tx=0x7febd802bb70 comp rx=0 tx=0).stop 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 -- 172.21.15.105:0/1667167379 shutdown_connections 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 --2- 172.21.15.105:0/1667167379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febec1012e0 0x7febec105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 --2- 172.21.15.105:0/1667167379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec100920 0x7febec100da0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 --2- 172.21.15.105:0/1667167379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febec0ffbc0 0x7febec0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 -- 172.21.15.105:0/1667167379 >> 172.21.15.105:0/1667167379 conn(0x7febec0fb4f0 msgr2=0x7febec0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 -- 172.21.15.105:0/1667167379 shutdown_connections 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.742+0000 7febf2b22640 1 -- 172.21.15.105:0/1667167379 wait complete. 2024-03-26T17:49:24.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf2b22640 1 Processor -- start 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf2b22640 1 -- start start 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf2b22640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febec0ffbc0 0x7febec19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf2b22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febec100920 0x7febec19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf2b22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec1012e0 0x7febec1a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf2b22640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febec113aa0 con 0x7febec100920 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf2b22640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febec113920 con 0x7febec1012e0 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf2b22640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febec113c20 con 0x7febec0ffbc0 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febec100920 0x7febec19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febec100920 0x7febec19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45542/0 (socket says 172.21.15.105:45542) 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf08be640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febec0ffbc0 0x7febec19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febebfff640 1 -- 172.21.15.105:0/2733976074 learned_addr learned my addr 172.21.15.105:0/2733976074 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febf10bf640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec1012e0 0x7febec1a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.743+0000 7febebfff640 1 -- 172.21.15.105:0/2733976074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febec0ffbc0 msgr2=0x7febec19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febebfff640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febec0ffbc0 0x7febec19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febebfff640 1 -- 172.21.15.105:0/2733976074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec1012e0 msgr2=0x7febec1a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febebfff640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec1012e0 0x7febec1a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febebfff640 1 -- 172.21.15.105:0/2733976074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febec1a5d80 con 0x7febec100920 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febebfff640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febec100920 0x7febec19e6f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7febd8005940 tx=0x7febd80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febe9ffb640 1 -- 172.21.15.105:0/2733976074 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8041020 con 0x7febec100920 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febf2b22640 1 -- 172.21.15.105:0/2733976074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febec1a6010 con 0x7febec100920 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febe9ffb640 1 -- 172.21.15.105:0/2733976074 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febd80035f0 con 0x7febec100920 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febe9ffb640 1 -- 172.21.15.105:0/2733976074 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8039040 con 0x7febec100920 2024-03-26T17:49:24.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.744+0000 7febf2b22640 1 -- 172.21.15.105:0/2733976074 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febec1a6550 con 0x7febec100920 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febe9ffb640 1 -- 172.21.15.105:0/2733976074 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7febd8044020 con 0x7febec100920 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf57ed740 1 Processor -- start 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf57ed740 1 -- start start 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf57ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf57ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775aff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf57ed740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf57ed740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e67750f970 con 0x55e6775b0590 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf57ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e6774b7d70 con 0x55e6775ab420 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf57ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e67750b920 con 0x55e6775aaa30 2024-03-26T17:49:24.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf08be640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45546/0 (socket says 172.21.15.105:45546) 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf10bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febebfff640 1 -- 172.21.15.105:0/2009346372 learned_addr learned my addr 172.21.15.105:0/2009346372 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf08be640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49188/0 (socket says 172.21.15.105:49188) 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febe9ffb640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febdc0a2f20 0x7febdc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.747+0000 7febf10bf640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febdc0a2f20 0x7febdc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febf08be640 1 -- 172.21.15.105:0/2009346372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 msgr2=0x55e6775aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febf08be640 1 --2- 172.21.15.105:0/2009346372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febf08be640 1 -- 172.21.15.105:0/2009346372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 msgr2=0x55e6775b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febf08be640 1 --2- 172.21.15.105:0/2009346372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febf08be640 1 -- 172.21.15.105:0/2009346372 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6772233f0 con 0x55e6775ab420 2024-03-26T17:49:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febf10bf640 1 --2- 172.21.15.105:0/2009346372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febe9ffb640 1 -- 172.21.15.105:0/2733976074 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7febd80f0e30 con 0x7febec100920 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febf08be640 1 --2- 172.21.15.105:0/2009346372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775aff20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7febe0005fd0 tx=0x7febe0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be48d3b6456938d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febf10bf640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febdc0a2f20 0x7febdc0a53e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55e6775abc80 tx=0x7febe4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febd6ffd640 1 -- 172.21.15.105:0/2009346372 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0004020 con 0x55e6775ab420 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febd6ffd640 1 -- 172.21.15.105:0/2009346372 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febe00041c0 con 0x55e6775ab420 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.748+0000 7febd6ffd640 1 -- 172.21.15.105:0/2009346372 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe00197a0 con 0x55e6775ab420 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 -- 172.21.15.105:0/2009346372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 msgr2=0x55e6775aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 --2- 172.21.15.105:0/2009346372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775aff20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7febe0005fd0 tx=0x7febe0011810 comp rx=0 tx=0).stop 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 -- 172.21.15.105:0/2009346372 shutdown_connections 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 --2- 172.21.15.105:0/2009346372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 --2- 172.21.15.105:0/2009346372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775aff20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 --2- 172.21.15.105:0/2009346372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 -- 172.21.15.105:0/2009346372 >> 172.21.15.105:0/2009346372 conn(0x55e6775a5fa0 msgr2=0x55e6775a63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 -- 172.21.15.105:0/2009346372 shutdown_connections 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 -- 172.21.15.105:0/2009346372 wait complete. 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 -- start start 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.749+0000 7febf57ed740 1 Processor -- start 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf57ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf57ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf57ed740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf57ed740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e6775b3540 con 0x55e6775b0590 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf57ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e6774cb310 con 0x55e6775ab420 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf57ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e6775b36c0 con 0x55e6775aaa30 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf57ed740 10 client.0 _ll_register_callbacks cb 0x55e677506c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf08be640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febebfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf10bf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf08be640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37474/0 (socket says 172.21.15.105:37474) 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf08be640 1 -- 172.21.15.105:0/1454892238 learned_addr learned my addr 172.21.15.105:0/1454892238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febebfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49194/0 (socket says 172.21.15.105:49194) 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf10bf640 1 -- 172.21.15.105:0/1454892238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 msgr2=0x55e6775b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf10bf640 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf10bf640 1 -- 172.21.15.105:0/1454892238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 msgr2=0x55e6775b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf10bf640 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf10bf640 1 -- 172.21.15.105:0/1454892238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6772ed7d0 con 0x55e6775b0590 2024-03-26T17:49:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf10bf640 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775bc550 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7febe4005fd0 tx=0x7febe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d61b2695c8ae4f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febcbfff640 1 -- 172.21.15.105:0/1454892238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4025070 con 0x55e6775b0590 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febcbfff640 1 -- 172.21.15.105:0/1454892238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7febe40047d0 con 0x55e6775b0590 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.750+0000 7febf57ed740 1 -- 172.21.15.105:0/1454892238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe00041c0 con 0x55e6775b0590 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febcbfff640 1 -- 172.21.15.105:0/1454892238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7febe4031050 con 0x55e6775b0590 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf57ed740 10 client.19389 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf57ed740 1 -- 172.21.15.105:0/1454892238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6775bf990 con 0x55e6775b0590 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febcbfff640 1 -- 172.21.15.105:0/1454892238 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febe00041c0 con 0x55e6775b0590 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febcbfff640 1 -- 172.21.15.105:0/1454892238 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7febe401f040 con 0x55e6775b0590 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf57ed740 10 client.19389 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf57ed740 10 client.19389 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf57ed740 20 client.19389 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf57ed740 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e6775c1070 0x55e6775c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf57ed740 4 client.19389 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf57ed740 1 -- 172.21.15.105:0/1454892238 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e67751ceb0 con 0x55e6775c1070 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.751+0000 7febf08be640 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e6775c1070 0x55e6775c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febcbfff640 10 client.19389 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf08be640 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e6775c1070 0x55e6775c34b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ccd610c9407b17b3 server_cookie=25094a8575d28461 in_seq=0 out_seq=0 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febcbfff640 1 -- 172.21.15.105:0/1454892238 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e67751ceb0 con 0x55e6775c1070 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febcbfff640 10 client.19389 handle_command_reply: tid=0 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 client.19389 shutdown 2024-03-26T17:49:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 -- 172.21.15.105:0/1454892238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 msgr2=0x55e6775bc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775bc550 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7febe4005fd0 tx=0x7febe4004a40 comp rx=0 tx=0).stop 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 -- 172.21.15.105:0/1454892238 shutdown_connections 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e6775c1070 0x55e6775c34b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775b0590 0x55e6775bc550 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febcbfff640 0 client.19389 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 --2- 172.21.15.105:0/1454892238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.752+0000 7febf57ed740 1 -- 172.21.15.105:0/1454892238 >> 172.21.15.105:0/1454892238 conn(0x55e6775a5fa0 msgr2=0x55e6775a51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.753+0000 7febf57ed740 1 -- 172.21.15.105:0/1454892238 shutdown_connections 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.753+0000 7febf57ed740 1 -- 172.21.15.105:0/1454892238 wait complete. 2024-03-26T17:49:24.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.753+0000 7febf57ed740 20 client.19389 trim_cache size 0 max 16384 2024-03-26T17:49:24.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf57ed740 1 Processor -- start 2024-03-26T17:49:24.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf57ed740 1 -- start start 2024-03-26T17:49:24.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf57ed740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775aaa30 0x55e6775b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf57ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf57ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775b0590 0x55e6775a4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf57ed740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febec1af430 con 0x55e6775aaa30 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf57ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b0f70 con 0x55e6775ab420 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf57ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febec1afb00 con 0x55e6775b0590 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf08be640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775aaa30 0x55e6775b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febebfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf08be640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775aaa30 0x55e6775b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45570/0 (socket says 172.21.15.105:45570) 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf08be640 1 -- 172.21.15.105:0/1544923427 learned_addr learned my addr 172.21.15.105:0/1544923427 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.772+0000 7febf10bf640 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775b0590 0x55e6775a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febf08be640 1 -- 172.21.15.105:0/1544923427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775b0590 msgr2=0x55e6775a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febf08be640 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775b0590 0x55e6775a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febf08be640 1 -- 172.21.15.105:0/1544923427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 msgr2=0x55e6775bc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febf08be640 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775bc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febf08be640 1 -- 172.21.15.105:0/1544923427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6775ab800 con 0x55e6775aaa30 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febebfff640 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febf08be640 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775aaa30 0x55e6775b45f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7febe0009230 tx=0x7febe0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3a58bb5a495c1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febd67fc640 1 -- 172.21.15.105:0/1544923427 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0003d40 con 0x55e6775aaa30 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febd67fc640 1 -- 172.21.15.105:0/1544923427 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febe0003ee0 con 0x55e6775aaa30 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.773+0000 7febd67fc640 1 -- 172.21.15.105:0/1544923427 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0003970 con 0x55e6775aaa30 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- 172.21.15.105:0/1544923427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775aaa30 msgr2=0x55e6775b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775aaa30 0x55e6775b45f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7febe0009230 tx=0x7febe0003cd0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- 172.21.15.105:0/1544923427 shutdown_connections 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775b0590 0x55e6775a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775ab420 0x55e6775bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 --2- 172.21.15.105:0/1544923427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775aaa30 0x55e6775b45f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- 172.21.15.105:0/1544923427 >> 172.21.15.105:0/1544923427 conn(0x55e6775a5fa0 msgr2=0x55e6775b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- 172.21.15.105:0/1544923427 shutdown_connections 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- 172.21.15.105:0/1544923427 wait complete. 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- start start 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 Processor -- start 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775ab420 0x55e6775b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775b0590 0x55e6775baef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b08a0 con 0x55e6775ab420 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febec1b01d0 con 0x55e6775b0590 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febec1ad410 con 0x55e6775aaa30 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.774+0000 7febf57ed740 10 client.0 _ll_register_callbacks cb 0x7febe40136b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:24.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775ab420 0x55e6775b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775ab420 0x55e6775b9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45578/0 (socket says 172.21.15.105:45578) 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febf10bf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775b0590 0x55e6775baef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 -- 172.21.15.105:0/2470887413 learned_addr learned my addr 172.21.15.105:0/2470887413 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febf08be640 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 -- 172.21.15.105:0/2470887413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 msgr2=0x55e6775b99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 -- 172.21.15.105:0/2470887413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775b0590 msgr2=0x55e6775baef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775b0590 0x55e6775baef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 -- 172.21.15.105:0/2470887413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6775bb4e0 con 0x55e6775ab420 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febf10bf640 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775b0590 0x55e6775baef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febebfff640 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775ab420 0x55e6775b9f30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7febd80bd1c0 tx=0x7febd80bd350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c7314b8dd0ac06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febcbfff640 1 -- 172.21.15.105:0/2470887413 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd80044e0 con 0x55e6775ab420 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febf57ed740 1 -- 172.21.15.105:0/2470887413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febe0003ee0 con 0x55e6775ab420 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febcbfff640 1 -- 172.21.15.105:0/2470887413 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7febd80037a0 con 0x55e6775ab420 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febcbfff640 1 -- 172.21.15.105:0/2470887413 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7febd80bdbc0 con 0x55e6775ab420 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febf57ed740 10 client.19395 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.775+0000 7febf57ed740 1 -- 172.21.15.105:0/2470887413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6775bc880 con 0x55e6775ab420 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febcbfff640 1 -- 172.21.15.105:0/2470887413 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febe0003ee0 con 0x55e6775ab420 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febcbfff640 1 -- 172.21.15.105:0/2470887413 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7febd803c030 con 0x55e6775ab420 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febf57ed740 10 client.19395 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:24.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febf57ed740 10 client.19395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febf57ed740 20 client.19395 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febf57ed740 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e677627c10 0x55e67762a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febf57ed740 4 client.19395 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febf57ed740 1 -- 172.21.15.105:0/2470887413 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6775b4dc0 con 0x55e677627c10 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febf08be640 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e677627c10 0x55e67762a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febcbfff640 10 client.19395 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.776+0000 7febf08be640 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e677627c10 0x55e67762a030 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9310cf6e1d393ac server_cookie=f514272c27e2a43e in_seq=0 out_seq=0 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febcbfff640 1 -- 172.21.15.105:0/2470887413 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6775b4dc0 con 0x55e677627c10 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febcbfff640 10 client.19395 handle_command_reply: tid=0 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 client.19395 shutdown 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 -- 172.21.15.105:0/2470887413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775ab420 msgr2=0x55e6775b9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775ab420 0x55e6775b9f30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7febd80bd1c0 tx=0x7febd80bd350 comp rx=0 tx=0).stop 2024-03-26T17:49:24.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 -- 172.21.15.105:0/2470887413 shutdown_connections 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e677627c10 0x55e67762a030 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6775b0590 0x55e6775baef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e6775ab420 0x55e6775b9f30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 --2- 172.21.15.105:0/2470887413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e6775aaa30 0x55e6775b99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 -- 172.21.15.105:0/2470887413 >> 172.21.15.105:0/2470887413 conn(0x55e6775a5fa0 msgr2=0x55e6775a9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febcbfff640 0 client.19395 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 -- 172.21.15.105:0/2470887413 shutdown_connections 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 1 -- 172.21.15.105:0/2470887413 wait complete. 2024-03-26T17:49:24.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf57ed740 20 client.19395 trim_cache size 0 max 16384 2024-03-26T17:49:24.778 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:24.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf2b22640 1 -- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febdc0a2f20 msgr2=0x7febdc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.777+0000 7febf2b22640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febdc0a2f20 0x7febdc0a53e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55e6775abc80 tx=0x7febe4004ab0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 -- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febec100920 msgr2=0x7febec19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febec100920 0x7febec19e6f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7febd8005940 tx=0x7febd80042f0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 -- 172.21.15.105:0/2733976074 shutdown_connections 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7febdc0a2f20 0x7febdc0a53e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febec1012e0 0x7febec1a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febec100920 0x7febec19e6f0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 --2- 172.21.15.105:0/2733976074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febec0ffbc0 0x7febec19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 -- 172.21.15.105:0/2733976074 >> 172.21.15.105:0/2733976074 conn(0x7febec0fb4f0 msgr2=0x7febec0fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 -- 172.21.15.105:0/2733976074 shutdown_connections 2024-03-26T17:49:24.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.778+0000 7febf2b22640 1 -- 172.21.15.105:0/2733976074 wait complete. 2024-03-26T17:49:24.781 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:49:24.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:24 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2868758756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:49:24.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/3835583132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb41019f0 msgr2=0x7f9fb4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/3835583132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb41019f0 0x7f9fb4101e70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009900 tx=0x7f9fa802bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/3835583132 shutdown_connections 2024-03-26T17:49:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/3835583132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb41023b0 0x7f9fb4108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/3835583132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb41019f0 0x7f9fb4101e70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/3835583132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb4100800 0x7f9fb4100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/3835583132 >> 172.21.15.105:0/3835583132 conn(0x7f9fb40fbd70 msgr2=0x7f9fb40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/3835583132 shutdown_connections 2024-03-26T17:49:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.919+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/3835583132 wait complete. 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb9ccc640 1 Processor -- start 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb9ccc640 1 -- start start 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb9ccc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4100800 0x7f9fb4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb9ccc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb41019f0 0x7f9fb4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb9ccc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb41023b0 0x7f9fb419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb9ccc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb410c330 con 0x7f9fb4100800 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb9ccc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb410c1b0 con 0x7f9fb41019f0 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb9ccc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb410c4b0 con 0x7f9fb41023b0 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4100800 0x7f9fb4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4100800 0x7f9fb4197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45596/0 (socket says 172.21.15.105:45596) 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb37fe640 1 -- 172.21.15.105:0/415389473 learned_addr learned my addr 172.21.15.105:0/415389473 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb41019f0 0x7f9fb4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.920+0000 7f9fb3fff640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb41023b0 0x7f9fb419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb37fe640 1 -- 172.21.15.105:0/415389473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb41023b0 msgr2=0x7f9fb419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb41023b0 0x7f9fb419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb37fe640 1 -- 172.21.15.105:0/415389473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb41019f0 msgr2=0x7f9fb4198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb41019f0 0x7f9fb4198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb37fe640 1 -- 172.21.15.105:0/415389473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb419f7e0 con 0x7f9fb4100800 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb41019f0 0x7f9fb4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4100800 0x7f9fb4197bc0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400e9c0 tx=0x7f9fa400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb0ff9640 1 -- 172.21.15.105:0/415389473 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa400cd80 con 0x7f9fb4100800 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/415389473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb419fad0 con 0x7f9fb4100800 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb0ff9640 1 -- 172.21.15.105:0/415389473 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa40035f0 con 0x7f9fb4100800 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb0ff9640 1 -- 172.21.15.105:0/415389473 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4010690 con 0x7f9fb4100800 2024-03-26T17:49:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.921+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/415389473 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb419fff0 con 0x7f9fb4100800 2024-03-26T17:49:24.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.923+0000 7f9fb0ff9640 1 -- 172.21.15.105:0/415389473 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9fa40037a0 con 0x7f9fb4100800 2024-03-26T17:49:24.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.924+0000 7f9fb0ff9640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.924+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.924+0000 7f9fb0ff9640 1 -- 172.21.15.105:0/415389473 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9fa40cf7a0 con 0x7f9fb4100800 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.924+0000 7f9fbc98b740 1 Processor -- start 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.924+0000 7f9fbc98b740 1 -- start start 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 0x5576d1c27c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c2c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c2ce00 0x5576d1c2f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5576d1b83970 con 0x5576d1c28240 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5576d1b2bd70 con 0x5576d1c27850 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576d1b7f920 con 0x5576d1c2ce00 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009230 tx=0x7f9fa8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c2c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 0x5576d1c27c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c2c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45610/0 (socket says 172.21.15.105:45610) 2024-03-26T17:49:24.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 -- 172.21.15.105:0/3223265767 learned_addr learned my addr 172.21.15.105:0/3223265767 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb3fff640 1 --2- 172.21.15.105:0/3223265767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c2ce00 0x5576d1c2f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 -- 172.21.15.105:0/3223265767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c2ce00 msgr2=0x5576d1c2f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/3223265767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c2ce00 0x5576d1c2f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 -- 172.21.15.105:0/3223265767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 msgr2=0x5576d1c27c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/3223265767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 0x5576d1c27c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 -- 172.21.15.105:0/3223265767 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d18973f0 con 0x5576d1c28240 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/3223265767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c2c790 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80096c0 tx=0x7f9fa803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f14bb60b7e13efc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9f99ffb640 1 -- 172.21.15.105:0/3223265767 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8005200 con 0x5576d1c28240 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9f99ffb640 1 -- 172.21.15.105:0/3223265767 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa8050070 con 0x5576d1c28240 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.925+0000 7f9f99ffb640 1 -- 172.21.15.105:0/3223265767 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8003850 con 0x5576d1c28240 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 -- 172.21.15.105:0/3223265767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 msgr2=0x5576d1c2c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/3223265767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c2c790 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80096c0 tx=0x7f9fa803e000 comp rx=0 tx=0).stop 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 -- 172.21.15.105:0/3223265767 shutdown_connections 2024-03-26T17:49:24.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/3223265767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c2ce00 0x5576d1c2f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/3223265767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c2c790 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/3223265767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 0x5576d1c27c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 -- 172.21.15.105:0/3223265767 >> 172.21.15.105:0/3223265767 conn(0x5576d1c19f20 msgr2=0x5576d1c1a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 -- 172.21.15.105:0/3223265767 shutdown_connections 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.926+0000 7f9fbc98b740 1 -- 172.21.15.105:0/3223265767 wait complete. 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 1 -- start start 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 1 Processor -- start 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 0x5576d1c316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c28240 0x5576d1c31c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c38c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5576d1c2fd50 con 0x5576d1c2ce00 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5576d1b3f310 con 0x5576d1c27850 2024-03-26T17:49:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5576d1c2fed0 con 0x5576d1c28240 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fbc98b740 10 client.0 _ll_register_callbacks cb 0x5576d1b7ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c38c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c38c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45626/0 (socket says 172.21.15.105:45626) 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c28240 0x5576d1c31c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb3fff640 1 -- 172.21.15.105:0/1751877201 learned_addr learned my addr 172.21.15.105:0/1751877201 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 0x5576d1c316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb3fff640 1 -- 172.21.15.105:0/1751877201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c28240 msgr2=0x5576d1c31c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb3fff640 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c28240 0x5576d1c31c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb3fff640 1 -- 172.21.15.105:0/1751877201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 msgr2=0x5576d1c316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb3fff640 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 0x5576d1c316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.927+0000 7f9fb3fff640 1 -- 172.21.15.105:0/1751877201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d19617d0 con 0x5576d1c2ce00 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.928+0000 7f9fb3fff640 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c38c40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9fac011d10 tx=0x7f9fac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25af5e17a202b69 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.928+0000 7f9f92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.928+0000 7f9f92ffd640 1 -- 172.21.15.105:0/1751877201 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac009e30 con 0x5576d1c2ce00 2024-03-26T17:49:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.928+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1751877201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa8050070 con 0x5576d1c2ce00 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.928+0000 7f9f92ffd640 1 -- 172.21.15.105:0/1751877201 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9fac004b20 con 0x5576d1c2ce00 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.928+0000 7f9f92ffd640 1 -- 172.21.15.105:0/1751877201 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9fac004d40 con 0x5576d1c2ce00 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.928+0000 7f9fbc98b740 10 client.19407 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.928+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1751877201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576d1c3c080 con 0x5576d1c2ce00 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9f92ffd640 1 -- 172.21.15.105:0/1751877201 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa8050070 con 0x5576d1c2ce00 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9f92ffd640 1 -- 172.21.15.105:0/1751877201 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9fac019030 con 0x5576d1c2ce00 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9fbc98b740 10 client.19407 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9fbc98b740 10 client.19407 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9fbc98b740 20 client.19407 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5576d1c3d7a0 0x5576d1c3fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9fbc98b740 4 client.19407 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1751877201 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576d1b90eb0 con 0x5576d1c3d7a0 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.929+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5576d1c3d7a0 0x5576d1c3fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.930+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5576d1c3d7a0 0x5576d1c3fc40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7cd77510d7315237 server_cookie=e317bb15499b8e25 in_seq=0 out_seq=0 2024-03-26T17:49:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.930+0000 7f9f92ffd640 10 client.19407 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.930+0000 7f9f92ffd640 1 -- 172.21.15.105:0/1751877201 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5576d1b90eb0 con 0x5576d1c3d7a0 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.930+0000 7f9f92ffd640 10 client.19407 handle_command_reply: tid=0 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.930+0000 7f9fbc98b740 1 client.19407 shutdown 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1751877201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 msgr2=0x5576d1c38c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c38c40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9fac011d10 tx=0x7f9fac00f790 comp rx=0 tx=0).stop 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1751877201 shutdown_connections 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5576d1c3d7a0 0x5576d1c3fc40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c38c40 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c28240 0x5576d1c31c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1751877201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c27850 0x5576d1c316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9f92ffd640 0 client.19407 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1751877201 >> 172.21.15.105:0/1751877201 conn(0x5576d1c19f20 msgr2=0x5576d1c26ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1751877201 shutdown_connections 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1751877201 wait complete. 2024-03-26T17:49:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.931+0000 7f9fbc98b740 20 client.19407 trim_cache size 0 max 16384 2024-03-26T17:49:24.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fbc98b740 1 Processor -- start 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fbc98b740 1 -- start start 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c26650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c28240 0x5576d1c38c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41a6c40 con 0x5576d1c2ce00 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41a8780 con 0x5576d1c28240 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41a7310 con 0x5576d1c27850 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c26650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c26650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37542/0 (socket says 172.21.15.105:37542) 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c28240 0x5576d1c38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c26b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45638/0 (socket says 172.21.15.105:45638) 2024-03-26T17:49:24.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 -- 172.21.15.105:0/1280070855 learned_addr learned my addr 172.21.15.105:0/1280070855 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 -- 172.21.15.105:0/1280070855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c28240 msgr2=0x5576d1c38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/1280070855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c28240 0x5576d1c38c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 -- 172.21.15.105:0/1280070855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 msgr2=0x5576d1c26b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/1280070855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c26b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 -- 172.21.15.105:0/1280070855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d1c1ac90 con 0x5576d1c27850 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.945+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/1280070855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c26650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4098860 tx=0x7f9fa40bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3044b5681cb15cdf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9f997fa640 1 -- 172.21.15.105:0/1280070855 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa409c590 con 0x5576d1c27850 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9f997fa640 1 -- 172.21.15.105:0/1280070855 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa409be90 con 0x5576d1c27850 2024-03-26T17:49:24.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9f997fa640 1 -- 172.21.15.105:0/1280070855 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4098500 con 0x5576d1c27850 2024-03-26T17:49:24.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1280070855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 msgr2=0x5576d1c26650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1280070855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c26650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4098860 tx=0x7f9fa40bc040 comp rx=0 tx=0).stop 2024-03-26T17:49:24.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1280070855 shutdown_connections 2024-03-26T17:49:24.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1280070855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c2ce00 0x5576d1c26b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1280070855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c28240 0x5576d1c38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/1280070855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c26650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1280070855 >> 172.21.15.105:0/1280070855 conn(0x5576d1c19f20 msgr2=0x5576d1c316c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1280070855 shutdown_connections 2024-03-26T17:49:24.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.946+0000 7f9fbc98b740 1 -- 172.21.15.105:0/1280070855 wait complete. 2024-03-26T17:49:24.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 1 -- start start 2024-03-26T17:49:24.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 1 Processor -- start 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c36060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c2ce00 0x5576d1c375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41a80b0 con 0x5576d1c28240 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41a79e0 con 0x5576d1c2ce00 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb41a4c20 con 0x5576d1c27850 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fbc98b740 10 client.0 _ll_register_callbacks cb 0x7f9fac009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:24.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fb2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fb2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c365a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45650/0 (socket says 172.21.15.105:45650) 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fb2ffd640 1 -- 172.21.15.105:0/2295963550 learned_addr learned my addr 172.21.15.105:0/2295963550 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fb3fff640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c2ce00 0x5576d1c375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.947+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c36060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fb2ffd640 1 -- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 msgr2=0x5576d1c36060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c36060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fb2ffd640 1 -- 172.21.15.105:0/2295963550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c2ce00 msgr2=0x5576d1c375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c2ce00 0x5576d1c375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fb2ffd640 1 -- 172.21.15.105:0/2295963550 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d1c37ba0 con 0x5576d1c28240 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fb3fff640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c2ce00 0x5576d1c375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c36060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fb2ffd640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c365a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009900 tx=0x7f9fa802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7b7440a82ea5b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9f92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9f92ffd640 1 -- 172.21.15.105:0/2295963550 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8004e00 con 0x5576d1c28240 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9f92ffd640 1 -- 172.21.15.105:0/2295963550 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9fa80035f0 con 0x5576d1c28240 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fbc98b740 1 -- 172.21.15.105:0/2295963550 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fa409be90 con 0x5576d1c28240 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9f92ffd640 1 -- 172.21.15.105:0/2295963550 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9fa803a080 con 0x5576d1c28240 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fbc98b740 10 client.19413 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fbc98b740 1 -- 172.21.15.105:0/2295963550 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576d1c38ed0 con 0x5576d1c28240 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9f92ffd640 1 -- 172.21.15.105:0/2295963550 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fa409be90 con 0x5576d1c28240 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9f92ffd640 1 -- 172.21.15.105:0/2295963550 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9fa80418f0 con 0x5576d1c28240 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fbc98b740 10 client.19413 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:24.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.948+0000 7f9fbc98b740 10 client.19413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 20 client.19413 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5576d1ca3ed0 0x5576d1ca62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 4 client.19413 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 -- 172.21.15.105:0/2295963550 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576d1b8b200 con 0x5576d1ca3ed0 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5576d1ca3ed0 0x5576d1ca62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9f92ffd640 10 client.19413 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fb37fe640 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5576d1ca3ed0 0x5576d1ca62c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc3e37bc40846045 server_cookie=4c5c925f71aa8308 in_seq=0 out_seq=0 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9f92ffd640 1 -- 172.21.15.105:0/2295963550 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576d1b8b200 con 0x5576d1ca3ed0 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9f92ffd640 10 client.19413 handle_command_reply: tid=0 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 client.19413 shutdown 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 -- 172.21.15.105:0/2295963550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 msgr2=0x5576d1c365a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c365a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009900 tx=0x7f9fa802bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 -- 172.21.15.105:0/2295963550 shutdown_connections 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5576d1ca3ed0 0x5576d1ca62c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5576d1c2ce00 0x5576d1c375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5576d1c28240 0x5576d1c365a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 --2- 172.21.15.105:0/2295963550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5576d1c27850 0x5576d1c36060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9fbc98b740 1 -- 172.21.15.105:0/2295963550 >> 172.21.15.105:0/2295963550 conn(0x5576d1c19f20 msgr2=0x5576d1c18930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.949+0000 7f9f92ffd640 0 client.19413 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fbc98b740 1 -- 172.21.15.105:0/2295963550 shutdown_connections 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fbc98b740 1 -- 172.21.15.105:0/2295963550 wait complete. 2024-03-26T17:49:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fbc98b740 20 client.19413 trim_cache size 0 max 16384 2024-03-26T17:49:24.951 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f9c0a2f20 msgr2=0x7f9f9c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009230 tx=0x7f9fa8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4100800 msgr2=0x7f9fb4197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4100800 0x7f9fb4197bc0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400e9c0 tx=0x7f9fa400ee80 comp rx=0 tx=0).stop 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/415389473 shutdown_connections 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f9c0a2f20 0x7f9f9c0a53e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fb41023b0 0x7f9fb419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9fb41019f0 0x7f9fb4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 --2- 172.21.15.105:0/415389473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb4100800 0x7f9fb4197bc0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:24.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/415389473 >> 172.21.15.105:0/415389473 conn(0x7f9fb40fbd70 msgr2=0x7f9fb4069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:24.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/415389473 shutdown_connections 2024-03-26T17:49:24.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:24.950+0000 7f9fb9ccc640 1 -- 172.21.15.105:0/415389473 wait complete. 2024-03-26T17:49:24.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:49:25.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 356085 3.15 MB/sec execute 488 sec latency 19.149 ms 2024-03-26T17:49:25.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.078+0000 7f899cdcd640 1 -- 172.21.15.105:0/860227492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8998102370 msgr2=0x7f8998108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.078+0000 7f899cdcd640 1 --2- 172.21.15.105:0/860227492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8998102370 0x7f8998108c00 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f8990009ca0 tx=0x7f899002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.079+0000 7f899cdcd640 1 -- 172.21.15.105:0/860227492 shutdown_connections 2024-03-26T17:49:25.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.079+0000 7f899cdcd640 1 --2- 172.21.15.105:0/860227492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8998102370 0x7f8998108c00 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.079+0000 7f899cdcd640 1 --2- 172.21.15.105:0/860227492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89981019b0 0x7f8998101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.079+0000 7f899cdcd640 1 --2- 172.21.15.105:0/860227492 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89981007c0 0x7f8998100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.079+0000 7f899cdcd640 1 -- 172.21.15.105:0/860227492 >> 172.21.15.105:0/860227492 conn(0x7f89980fbd50 msgr2=0x7f89980fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.079+0000 7f899cdcd640 1 -- 172.21.15.105:0/860227492 shutdown_connections 2024-03-26T17:49:25.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.079+0000 7f899cdcd640 1 -- 172.21.15.105:0/860227492 wait complete. 2024-03-26T17:49:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.079+0000 7f899cdcd640 1 Processor -- start 2024-03-26T17:49:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899cdcd640 1 -- start start 2024-03-26T17:49:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899cdcd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89981007c0 0x7f8998196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899cdcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981019b0 0x7f8998196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899cdcd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8998102370 0x7f899819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899cdcd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f899810c2d0 con 0x7f89981019b0 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899cdcd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f899810c150 con 0x7f89981007c0 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899cdcd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f899810c450 con 0x7f8998102370 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f898ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981019b0 0x7f8998196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89981007c0 0x7f8998196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f898ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981019b0 0x7f8998196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45674/0 (socket says 172.21.15.105:45674) 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f898ffff640 1 -- 172.21.15.105:0/844614825 learned_addr learned my addr 172.21.15.105:0/844614825 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f8996d9d640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8998102370 0x7f899819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899659c640 1 -- 172.21.15.105:0/844614825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8998102370 msgr2=0x7f899819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899659c640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8998102370 0x7f899819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899659c640 1 -- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981019b0 msgr2=0x7f8998196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899659c640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981019b0 0x7f8998196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f899659c640 1 -- 172.21.15.105:0/844614825 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899819e620 con 0x7f89981007c0 2024-03-26T17:49:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.080+0000 7f898ffff640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981019b0 0x7f8998196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.081+0000 7f899659c640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89981007c0 0x7f8998196a00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f898800e9c0 tx=0x7f898800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.081+0000 7f898f7fe640 1 -- 172.21.15.105:0/844614825 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898800ccc0 con 0x7f89981007c0 2024-03-26T17:49:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.081+0000 7f899cdcd640 1 -- 172.21.15.105:0/844614825 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899819e910 con 0x7f89981007c0 2024-03-26T17:49:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.081+0000 7f898f7fe640 1 -- 172.21.15.105:0/844614825 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89880035f0 con 0x7f89981007c0 2024-03-26T17:49:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.081+0000 7f898f7fe640 1 -- 172.21.15.105:0/844614825 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898800ce60 con 0x7f89981007c0 2024-03-26T17:49:25.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.081+0000 7f899cdcd640 1 -- 172.21.15.105:0/844614825 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899819eeb0 con 0x7f89981007c0 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899f9b0740 1 Processor -- start 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f898f7fe640 1 -- 172.21.15.105:0/844614825 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f89880037a0 con 0x7f89981007c0 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899f9b0740 1 -- start start 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0d0b0 0x563e8bc0b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0bd40 0x563e8bc0c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc18140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899f9b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563e8bb70970 con 0x563e8bc0bd40 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899f9b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e8bb18d70 con 0x563e8bc0d0b0 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899f9b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e8bb6c920 con 0x563e8bc15d50 2024-03-26T17:49:25.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f8996d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0d0b0 0x563e8bc0b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f8996d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0d0b0 0x563e8bc0b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49306/0 (socket says 172.21.15.105:49306) 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f8996d9d640 1 -- 172.21.15.105:0/3766905052 learned_addr learned my addr 172.21.15.105:0/3766905052 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f899659c640 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0bd40 0x563e8bc0c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.084+0000 7f898ffff640 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc18140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898f7fe640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89840a2f20 0x7f89840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898ffff640 1 -- 172.21.15.105:0/3766905052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0d0b0 msgr2=0x563e8bc0b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898ffff640 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0d0b0 0x563e8bc0b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898ffff640 1 -- 172.21.15.105:0/3766905052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0bd40 msgr2=0x563e8bc0c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f8996d9d640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89840a2f20 0x7f89840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898ffff640 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0bd40 0x563e8bc0c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898ffff640 1 -- 172.21.15.105:0/3766905052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e8b8843f0 con 0x563e8bc15d50 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f899659c640 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0bd40 0x563e8bc0c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898f7fe640 1 -- 172.21.15.105:0/844614825 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f89880cf880 con 0x7f89981007c0 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898ffff640 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc18140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8980009fd0 tx=0x7f89800119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eb941b2ce2d5090 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f8996d9d640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89840a2f20 0x7f89840a53e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x563e8bc0cc40 tx=0x7f899002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898cff9640 1 -- 172.21.15.105:0/3766905052 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980005020 con 0x563e8bc15d50 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898cff9640 1 -- 172.21.15.105:0/3766905052 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89800047a0 con 0x563e8bc15d50 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.085+0000 7f898cff9640 1 -- 172.21.15.105:0/3766905052 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898000f8a0 con 0x563e8bc15d50 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 -- 172.21.15.105:0/3766905052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 msgr2=0x563e8bc18140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc18140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8980009fd0 tx=0x7f89800119a0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 -- 172.21.15.105:0/3766905052 shutdown_connections 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc18140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0bd40 0x563e8bc0c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 --2- 172.21.15.105:0/3766905052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0d0b0 0x563e8bc0b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 -- 172.21.15.105:0/3766905052 >> 172.21.15.105:0/3766905052 conn(0x563e8bc06fa0 msgr2=0x563e8bc073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 -- 172.21.15.105:0/3766905052 shutdown_connections 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 -- 172.21.15.105:0/3766905052 wait complete. 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.086+0000 7f899f9b0740 1 -- start start 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899f9b0740 1 Processor -- start 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0bd40 0x563e8bc1a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc15d50 0x563e8bc218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899f9b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563e8bc18980 con 0x563e8bc0d0b0 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899f9b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e8bb2c310 con 0x563e8bc15d50 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f898ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899f9b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e8bc18b00 con 0x563e8bc0bd40 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0bd40 0x563e8bc1a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0bd40 0x563e8bc1a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37586/0 (socket says 172.21.15.105:37586) 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899659c640 1 -- 172.21.15.105:0/2588407443 learned_addr learned my addr 172.21.15.105:0/2588407443 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f898ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45694/0 (socket says 172.21.15.105:45694) 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899f9b0740 10 client.0 _ll_register_callbacks cb 0x563e8bb67c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f8996d9d640 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc15d50 0x563e8bc218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899659c640 1 -- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc15d50 msgr2=0x563e8bc218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899659c640 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc15d50 0x563e8bc218a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899659c640 1 -- 172.21.15.105:0/2588407443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 msgr2=0x563e8bc1a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899659c640 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f899659c640 1 -- 172.21.15.105:0/2588407443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8b94e7d0 con 0x563e8bc0bd40 2024-03-26T17:49:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f898ffff640 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.087+0000 7f8996d9d640 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc15d50 0x563e8bc218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899659c640 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0bd40 0x563e8bc1a320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f898800f010 tx=0x7f898809b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632d5c7c6171ddd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f897dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f897dffb640 1 -- 172.21.15.105:0/2588407443 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880a0060 con 0x563e8bc0bd40 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f897dffb640 1 -- 172.21.15.105:0/2588407443 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f89880cd3f0 con 0x563e8bc0bd40 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 1 -- 172.21.15.105:0/2588407443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89800047a0 con 0x563e8bc0bd40 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f897dffb640 1 -- 172.21.15.105:0/2588407443 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f89880a4920 con 0x563e8bc0bd40 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 10 client.29224 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 1 -- 172.21.15.105:0/2588407443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e8bc24c90 con 0x563e8bc0bd40 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f897dffb640 1 -- 172.21.15.105:0/2588407443 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89800047a0 con 0x563e8bc0bd40 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f897dffb640 1 -- 172.21.15.105:0/2588407443 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f89880a11f0 con 0x563e8bc0bd40 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 10 client.29224 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 10 client.29224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 20 client.29224 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563e8bc26390 0x563e8bc28800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 4 client.29224 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.088+0000 7f899f9b0740 1 -- 172.21.15.105:0/2588407443 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e8bb7deb0 con 0x563e8bc26390 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.089+0000 7f898ffff640 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563e8bc26390 0x563e8bc28800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.089+0000 7f898ffff640 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563e8bc26390 0x563e8bc28800 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff2f5913fe48d3ac server_cookie=a928d87ecc9a0ac5 in_seq=0 out_seq=0 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.089+0000 7f897dffb640 10 client.29224 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f897dffb640 1 -- 172.21.15.105:0/2588407443 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563e8bb7deb0 con 0x563e8bc26390 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f897dffb640 10 client.29224 handle_command_reply: tid=0 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 client.29224 shutdown 2024-03-26T17:49:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 -- 172.21.15.105:0/2588407443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0bd40 msgr2=0x563e8bc1a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0bd40 0x563e8bc1a320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f898800f010 tx=0x7f898809b930 comp rx=0 tx=0).stop 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 -- 172.21.15.105:0/2588407443 shutdown_connections 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563e8bc26390 0x563e8bc28800 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc15d50 0x563e8bc218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f897dffb640 0 client.29224 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2588407443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0bd40 0x563e8bc1a320 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 -- 172.21.15.105:0/2588407443 >> 172.21.15.105:0/2588407443 conn(0x563e8bc06fa0 msgr2=0x563e8bc061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 -- 172.21.15.105:0/2588407443 shutdown_connections 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 1 -- 172.21.15.105:0/2588407443 wait complete. 2024-03-26T17:49:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.090+0000 7f899f9b0740 20 client.29224 trim_cache size 0 max 16384 2024-03-26T17:49:25.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:24 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2868758756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.106+0000 7f899f9b0740 1 Processor -- start 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.106+0000 7f899f9b0740 1 -- start start 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc07380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc21860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc05930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f899f9b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89981a6c20 con 0x563e8bc0d0b0 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f899f9b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89981a8760 con 0x563e8bc0bd40 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f899f9b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f89981a72f0 con 0x563e8bc15d50 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc21860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f899659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc07380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc21860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45704/0 (socket says 172.21.15.105:45704) 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 -- 172.21.15.105:0/1349432694 learned_addr learned my addr 172.21.15.105:0/1349432694 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f8996d9d640 1 --2- 172.21.15.105:0/1349432694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc05930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 -- 172.21.15.105:0/1349432694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 msgr2=0x563e8bc05930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 --2- 172.21.15.105:0/1349432694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc05930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 -- 172.21.15.105:0/1349432694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 msgr2=0x563e8bc07380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 --2- 172.21.15.105:0/1349432694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc07380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 -- 172.21.15.105:0/1349432694 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e8bc184b0 con 0x563e8bc0d0b0 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f898ffff640 1 --2- 172.21.15.105:0/1349432694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc21860 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8980000c00 tx=0x7f8980004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75658cd4e66db6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f897ffff640 1 -- 172.21.15.105:0/1349432694 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980011040 con 0x563e8bc0d0b0 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f897ffff640 1 -- 172.21.15.105:0/1349432694 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89800111e0 con 0x563e8bc0d0b0 2024-03-26T17:49:25.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.107+0000 7f897ffff640 1 -- 172.21.15.105:0/1349432694 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980004570 con 0x563e8bc0d0b0 2024-03-26T17:49:25.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 -- 172.21.15.105:0/1349432694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 msgr2=0x563e8bc21860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 --2- 172.21.15.105:0/1349432694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc21860 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8980000c00 tx=0x7f8980004e10 comp rx=0 tx=0).stop 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 -- 172.21.15.105:0/1349432694 shutdown_connections 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 --2- 172.21.15.105:0/1349432694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc15d50 0x563e8bc05930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 --2- 172.21.15.105:0/1349432694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc0d0b0 0x563e8bc21860 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 --2- 172.21.15.105:0/1349432694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc07380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 -- 172.21.15.105:0/1349432694 >> 172.21.15.105:0/1349432694 conn(0x563e8bc06fa0 msgr2=0x563e8bc0b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 -- 172.21.15.105:0/1349432694 shutdown_connections 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 -- 172.21.15.105:0/1349432694 wait complete. 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.108+0000 7f899f9b0740 1 -- start start 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899f9b0740 1 Processor -- start 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc1ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899f9b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc15d50 0x563e8bc201d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899f9b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89981a8090 con 0x563e8bc15d50 2024-03-26T17:49:25.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899f9b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f89981a79c0 con 0x563e8bc0bd40 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899f9b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f89981a4c00 con 0x563e8bc0d0b0 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899f9b0740 10 client.0 _ll_register_callbacks cb 0x7f89880046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc1ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f8996d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc15d50 0x563e8bc201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc1ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49328/0 (socket says 172.21.15.105:49328) 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f8996d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc15d50 0x563e8bc201d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45718/0 (socket says 172.21.15.105:45718) 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f899659c640 1 -- 172.21.15.105:0/2569038663 learned_addr learned my addr 172.21.15.105:0/2569038663 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f898ffff640 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f8996d9d640 1 -- 172.21.15.105:0/2569038663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0d0b0 msgr2=0x563e8bc1f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f8996d9d640 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f8996d9d640 1 -- 172.21.15.105:0/2569038663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 msgr2=0x563e8bc1ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f8996d9d640 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc1ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f8996d9d640 1 -- 172.21.15.105:0/2569038663 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e8bc207c0 con 0x563e8bc15d50 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f8996d9d640 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc15d50 0x563e8bc201d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8990032380 tx=0x7f89900324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9a85a84d109ca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.109+0000 7f897dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f897dffb640 1 -- 172.21.15.105:0/2569038663 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990056060 con 0x563e8bc15d50 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f897dffb640 1 -- 172.21.15.105:0/2569038663 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8990004450 con 0x563e8bc15d50 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 1 -- 172.21.15.105:0/2569038663 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89800111e0 con 0x563e8bc15d50 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f897dffb640 1 -- 172.21.15.105:0/2569038663 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8990039040 con 0x563e8bc15d50 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 10 client.19431 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 1 -- 172.21.15.105:0/2569038663 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e8bc21b30 con 0x563e8bc15d50 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f897dffb640 1 -- 172.21.15.105:0/2569038663 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89800111e0 con 0x563e8bc15d50 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f897dffb640 1 -- 172.21.15.105:0/2569038663 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8990034440 con 0x563e8bc15d50 2024-03-26T17:49:25.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 10 client.19431 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 10 client.19431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 20 client.19431 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563e8bc8ced0 0x563e8bc8f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 4 client.19431 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.110+0000 7f899f9b0740 1 -- 172.21.15.105:0/2569038663 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e8bb78200 con 0x563e8bc8ced0 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899659c640 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563e8bc8ced0 0x563e8bc8f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899659c640 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563e8bc8ced0 0x563e8bc8f2f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=faada813d338a50d server_cookie=8e9d22eda24dfefb in_seq=0 out_seq=0 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f897dffb640 10 client.19431 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f897dffb640 1 -- 172.21.15.105:0/2569038663 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e8bb78200 con 0x563e8bc8ced0 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f897dffb640 10 client.19431 handle_command_reply: tid=0 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 client.19431 shutdown 2024-03-26T17:49:25.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 -- 172.21.15.105:0/2569038663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc15d50 msgr2=0x563e8bc201d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc15d50 0x563e8bc201d0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8990032380 tx=0x7f89900324f0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 -- 172.21.15.105:0/2569038663 shutdown_connections 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e8bc15d50 0x563e8bc201d0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e8bc0d0b0 0x563e8bc1f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563e8bc8ced0 0x563e8bc8f2f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 --2- 172.21.15.105:0/2569038663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e8bc0bd40 0x563e8bc1ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 -- 172.21.15.105:0/2569038663 >> 172.21.15.105:0/2569038663 conn(0x563e8bc06fa0 msgr2=0x563e8bc07380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f897dffb640 0 client.19431 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 -- 172.21.15.105:0/2569038663 shutdown_connections 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 1 -- 172.21.15.105:0/2569038663 wait complete. 2024-03-26T17:49:25.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.111+0000 7f899f9b0740 20 client.19431 trim_cache size 0 max 16384 2024-03-26T17:49:25.112 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 -- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89840a2f20 msgr2=0x7f89840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89840a2f20 0x7f89840a53e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x563e8bc0cc40 tx=0x7f899002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 -- 172.21.15.105:0/844614825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89981007c0 msgr2=0x7f8998196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89981007c0 0x7f8998196a00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f898800e9c0 tx=0x7f898800ee80 comp rx=0 tx=0).stop 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 -- 172.21.15.105:0/844614825 shutdown_connections 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89840a2f20 0x7f89840a53e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8998102370 0x7f899819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981019b0 0x7f8998196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 --2- 172.21.15.105:0/844614825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f89981007c0 0x7f8998196a00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 -- 172.21.15.105:0/844614825 >> 172.21.15.105:0/844614825 conn(0x7f89980fbd50 msgr2=0x7f8998104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 -- 172.21.15.105:0/844614825 shutdown_connections 2024-03-26T17:49:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.112+0000 7f899cdcd640 1 -- 172.21.15.105:0/844614825 wait complete. 2024-03-26T17:49:25.115 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:49:25.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:24 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2868758756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:49:25.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 -- 172.21.15.105:0/2100131596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1019b0 msgr2=0x7f4e7c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 --2- 172.21.15.105:0/2100131596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1019b0 0x7f4e7c101e30 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009900 tx=0x7f4e6802bc50 comp rx=0 tx=0).stop 2024-03-26T17:49:25.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 -- 172.21.15.105:0/2100131596 shutdown_connections 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 --2- 172.21.15.105:0/2100131596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e7c102370 0x7f4e7c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 --2- 172.21.15.105:0/2100131596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1019b0 0x7f4e7c101e30 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 --2- 172.21.15.105:0/2100131596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c1007c0 0x7f4e7c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 -- 172.21.15.105:0/2100131596 >> 172.21.15.105:0/2100131596 conn(0x7f4e7c0fbd50 msgr2=0x7f4e7c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 -- 172.21.15.105:0/2100131596 shutdown_connections 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 -- 172.21.15.105:0/2100131596 wait complete. 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.243+0000 7f4e8319d640 1 Processor -- start 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e8319d640 1 -- start start 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e8319d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1007c0 0x7f4e7c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e8319d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e7c1019b0 0x7f4e7c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e8319d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c102370 0x7f4e7c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e8319d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10c390 con 0x7f4e7c1007c0 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e8319d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10c210 con 0x7f4e7c1019b0 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e8319d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c10c510 con 0x7f4e7c102370 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1007c0 0x7f4e7c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1007c0 0x7f4e7c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45750/0 (socket says 172.21.15.105:45750) 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 -- 172.21.15.105:0/4157293862 learned_addr learned my addr 172.21.15.105:0/4157293862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e8173a640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c102370 0x7f4e7c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e7c1019b0 0x7f4e7c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 -- 172.21.15.105:0/4157293862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c102370 msgr2=0x7f4e7c19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c102370 0x7f4e7c19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 -- 172.21.15.105:0/4157293862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e7c1019b0 msgr2=0x7f4e7c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e7c1019b0 0x7f4e7c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 -- 172.21.15.105:0/4157293862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c19f840 con 0x7f4e7c1007c0 2024-03-26T17:49:25.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.244+0000 7f4e80f39640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1007c0 0x7f4e7c197c30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000e9c0 tx=0x7f4e7000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.245+0000 7f4e79ffb640 1 -- 172.21.15.105:0/4157293862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7000cd80 con 0x7f4e7c1007c0 2024-03-26T17:49:25.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.245+0000 7f4e79ffb640 1 -- 172.21.15.105:0/4157293862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e700035f0 con 0x7f4e7c1007c0 2024-03-26T17:49:25.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.245+0000 7f4e79ffb640 1 -- 172.21.15.105:0/4157293862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70010690 con 0x7f4e7c1007c0 2024-03-26T17:49:25.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.245+0000 7f4e8319d640 1 -- 172.21.15.105:0/4157293862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c19fb30 con 0x7f4e7c1007c0 2024-03-26T17:49:25.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.245+0000 7f4e8319d640 1 -- 172.21.15.105:0/4157293862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1a0030 con 0x7f4e7c1007c0 2024-03-26T17:49:25.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.247+0000 7f4e79ffb640 1 -- 172.21.15.105:0/4157293862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4e700037a0 con 0x7f4e7c1007c0 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.248+0000 7f4e79ffb640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4e6c0a2f20 0x7f4e6c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.248+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4e6c0a2f20 0x7f4e6c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.248+0000 7f4e79ffb640 1 -- 172.21.15.105:0/4157293862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4e700cf990 con 0x7f4e7c1007c0 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.248+0000 7f4e85d82740 1 Processor -- start 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.248+0000 7f4e85d82740 1 -- start start 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.248+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4e6c0a2f20 0x7f4e6c0a53e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005ab0 tx=0x7f4e68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.248+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa7e40 0x563e5ffa8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.248+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa8830 0x563e5ffad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffafe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e85d82740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563e5ff08970 con 0x563e5ffa7e40 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e85d82740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e5feb0d70 con 0x563e5ffa8830 2024-03-26T17:49:25.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e85d82740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e5ff04920 con 0x563e5ffad9a0 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa7e40 0x563e5ffa8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa7e40 0x563e5ffa8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45758/0 (socket says 172.21.15.105:45758) 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 -- 172.21.15.105:0/2693438647 learned_addr learned my addr 172.21.15.105:0/2693438647 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa8830 0x563e5ffad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e8173a640 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffafe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 -- 172.21.15.105:0/2693438647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 msgr2=0x563e5ffafe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffafe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 -- 172.21.15.105:0/2693438647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa8830 msgr2=0x563e5ffad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa8830 0x563e5ffad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 -- 172.21.15.105:0/2693438647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5fc1c3f0 con 0x563e5ffa7e40 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa8830 0x563e5ffad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e80f39640 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa7e40 0x563e5ffa8260 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f4e700a4560 tx=0x7f4e7009b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a02fd650cc617d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e66ffd640 1 -- 172.21.15.105:0/2693438647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7009c780 con 0x563e5ffa7e40 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e66ffd640 1 -- 172.21.15.105:0/2693438647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e7009c080 con 0x563e5ffa7e40 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.249+0000 7f4e66ffd640 1 -- 172.21.15.105:0/2693438647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e700bc0d0 con 0x563e5ffa7e40 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 -- 172.21.15.105:0/2693438647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa7e40 msgr2=0x563e5ffa8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa7e40 0x563e5ffa8260 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f4e700a4560 tx=0x7f4e7009b9e0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 -- 172.21.15.105:0/2693438647 shutdown_connections 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffafe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa8830 0x563e5ffad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2693438647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa7e40 0x563e5ffa8260 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 -- 172.21.15.105:0/2693438647 >> 172.21.15.105:0/2693438647 conn(0x563e5ff9efa0 msgr2=0x563e5ff9f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 -- 172.21.15.105:0/2693438647 shutdown_connections 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.250+0000 7f4e85d82740 1 -- 172.21.15.105:0/2693438647 wait complete. 2024-03-26T17:49:25.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 1 -- start start 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 1 Processor -- start 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa8830 0x563e5ffb2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffb98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563e5ffb0950 con 0x563e5ffa8830 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e5fec4310 con 0x563e5ffa7e40 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e5ffb0ad0 con 0x563e5ffad9a0 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e85d82740 10 client.0 _ll_register_callbacks cb 0x563e5feffc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa8830 0x563e5ffb2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa8830 0x563e5ffb2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45766/0 (socket says 172.21.15.105:45766) 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e7bfff640 1 -- 172.21.15.105:0/1689543813 learned_addr learned my addr 172.21.15.105:0/1689543813 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e80f39640 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.251+0000 7f4e8173a640 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffb98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e7bfff640 1 -- 172.21.15.105:0/1689543813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 msgr2=0x563e5ffb98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffb98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e7bfff640 1 -- 172.21.15.105:0/1689543813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 msgr2=0x563e5ffb2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e7bfff640 1 -- 172.21.15.105:0/1689543813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5fce67d0 con 0x563e5ffa8830 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa8830 0x563e5ffb2870 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009900 tx=0x7f4e68009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621910f72ef24ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e5bfff640 1 -- 172.21.15.105:0/1689543813 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68050070 con 0x563e5ffa8830 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e5bfff640 1 -- 172.21.15.105:0/1689543813 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4e680048d0 con 0x563e5ffa8830 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e85d82740 1 -- 172.21.15.105:0/1689543813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e7009c080 con 0x563e5ffa8830 2024-03-26T17:49:25.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e5bfff640 1 -- 172.21.15.105:0/1689543813 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4e6805c050 con 0x563e5ffa8830 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e85d82740 10 client.19443 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e85d82740 1 -- 172.21.15.105:0/1689543813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e5ffbcc90 con 0x563e5ffa8830 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.252+0000 7f4e5bfff640 1 -- 172.21.15.105:0/1689543813 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e7009c080 con 0x563e5ffa8830 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e5bfff640 1 -- 172.21.15.105:0/1689543813 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4e68041460 con 0x563e5ffa8830 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e85d82740 10 client.19443 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e85d82740 10 client.19443 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e85d82740 20 client.19443 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e85d82740 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e5ffbe2d0 0x563e5ffc0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e85d82740 4 client.19443 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e85d82740 1 -- 172.21.15.105:0/1689543813 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5ff15eb0 con 0x563e5ffbe2d0 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e80f39640 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e5ffbe2d0 0x563e5ffc0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e5bfff640 10 client.19443 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.253+0000 7f4e80f39640 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e5ffbe2d0 0x563e5ffc0770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51f6b6d670e1f82d server_cookie=bbb4b586bb235bc1 in_seq=0 out_seq=0 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e5bfff640 1 -- 172.21.15.105:0/1689543813 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563e5ff15eb0 con 0x563e5ffbe2d0 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e5bfff640 10 client.19443 handle_command_reply: tid=0 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 client.19443 shutdown 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 -- 172.21.15.105:0/1689543813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa8830 msgr2=0x563e5ffb2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa8830 0x563e5ffb2870 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009900 tx=0x7f4e68009260 comp rx=0 tx=0).stop 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 -- 172.21.15.105:0/1689543813 shutdown_connections 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e5ffbe2d0 0x563e5ffc0770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffb98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffa8830 0x563e5ffb2870 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 --2- 172.21.15.105:0/1689543813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 -- 172.21.15.105:0/1689543813 >> 172.21.15.105:0/1689543813 conn(0x563e5ff9efa0 msgr2=0x563e5ff9e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e5bfff640 0 client.19443 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:49:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 -- 172.21.15.105:0/1689543813 shutdown_connections 2024-03-26T17:49:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 1 -- 172.21.15.105:0/1689543813 wait complete. 2024-03-26T17:49:25.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.254+0000 7f4e85d82740 20 client.19443 trim_cache size 0 max 16384 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e85d82740 1 Processor -- start 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e85d82740 1 -- start start 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ff9d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffc07c0 0x563e5ff9de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e85d82740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1a6ca0 con 0x563e5ffad9a0 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e85d82740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1a87e0 con 0x563e5ffa7e40 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e85d82740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1a7370 con 0x563e5ffc07c0 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45768/0 (socket says 172.21.15.105:45768) 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.270+0000 7f4e80f39640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ff9d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e7bfff640 1 -- 172.21.15.105:0/2403057109 learned_addr learned my addr 172.21.15.105:0/2403057109 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e8173a640 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffc07c0 0x563e5ff9de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e7bfff640 1 -- 172.21.15.105:0/2403057109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffc07c0 msgr2=0x563e5ff9de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffc07c0 0x563e5ff9de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e7bfff640 1 -- 172.21.15.105:0/2403057109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 msgr2=0x563e5ff9d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ff9d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e7bfff640 1 -- 172.21.15.105:0/2403057109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5ff9fd10 con 0x563e5ffad9a0 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e80f39640 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ff9d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9870 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4e68041880 tx=0x7f4e68005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8778a04462e3b7ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e667fc640 1 -- 172.21.15.105:0/2403057109 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68059030 con 0x563e5ffad9a0 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e667fc640 1 -- 172.21.15.105:0/2403057109 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e68005c90 con 0x563e5ffad9a0 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.271+0000 7f4e667fc640 1 -- 172.21.15.105:0/2403057109 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68050040 con 0x563e5ffad9a0 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 -- 172.21.15.105:0/2403057109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffad9a0 msgr2=0x563e5ffb9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9870 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4e68041880 tx=0x7f4e68005410 comp rx=0 tx=0).stop 2024-03-26T17:49:25.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 -- 172.21.15.105:0/2403057109 shutdown_connections 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffc07c0 0x563e5ff9de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9870 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2403057109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ff9d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 -- 172.21.15.105:0/2403057109 >> 172.21.15.105:0/2403057109 conn(0x563e5ff9efa0 msgr2=0x563e5ffc00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 -- 172.21.15.105:0/2403057109 shutdown_connections 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 -- 172.21.15.105:0/2403057109 wait complete. 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 -- start start 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.272+0000 7f4e85d82740 1 Processor -- start 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e85d82740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffc07c0 0x563e5ffb7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e85d82740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1a8110 con 0x563e5ffc07c0 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e85d82740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1a7a40 con 0x563e5ffa7e40 2024-03-26T17:49:25.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e85d82740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1a4c80 con 0x563e5ffad9a0 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e85d82740 10 client.0 _ll_register_callbacks cb 0x7f4e6802cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49396/0 (socket says 172.21.15.105:49396) 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 -- 172.21.15.105:0/2663029055 learned_addr learned my addr 172.21.15.105:0/2663029055 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e8173a640 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffc07c0 0x563e5ffb7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 -- 172.21.15.105:0/2663029055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 msgr2=0x563e5ffb9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 -- 172.21.15.105:0/2663029055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffc07c0 msgr2=0x563e5ffb7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffc07c0 0x563e5ffb7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 -- 172.21.15.105:0/2663029055 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5ffb7da0 con 0x563e5ffa7e40 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e8173a640 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffc07c0 0x563e5ffb7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e80f39640 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb9100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000cfd0 tx=0x7f4e7009a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41899ff0dea196d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.273+0000 7f4e5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:25.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e5bfff640 1 -- 172.21.15.105:0/2663029055 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7007ae60 con 0x563e5ffa7e40 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e5bfff640 1 -- 172.21.15.105:0/2663029055 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4e700bc0b0 con 0x563e5ffa7e40 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 1 -- 172.21.15.105:0/2663029055 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e68005c90 con 0x563e5ffa7e40 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e5bfff640 1 -- 172.21.15.105:0/2663029055 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4e700bc2b0 con 0x563e5ffa7e40 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 10 client.29258 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 1 -- 172.21.15.105:0/2663029055 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e60025020 con 0x563e5ffa7e40 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e5bfff640 1 -- 172.21.15.105:0/2663029055 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e68005c90 con 0x563e5ffa7e40 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e5bfff640 1 -- 172.21.15.105:0/2663029055 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4e70096b60 con 0x563e5ffa7e40 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 10 client.29258 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 10 client.29258 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 20 client.29258 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e60025590 0x563e60027980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 4 client.29258 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e85d82740 1 -- 172.21.15.105:0/2663029055 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e5ff10200 con 0x563e60025590 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.274+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e60025590 0x563e60027980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e5bfff640 10 client.29258 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:49:25.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e7bfff640 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e60025590 0x563e60027980 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e142c7fe49cba48 server_cookie=fa5928c97da8b9ad in_seq=0 out_seq=0 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e5bfff640 1 -- 172.21.15.105:0/2663029055 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e5ff10200 con 0x563e60025590 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e5bfff640 10 client.29258 handle_command_reply: tid=0 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 client.29258 shutdown 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 -- 172.21.15.105:0/2663029055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 msgr2=0x563e5ffb9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb9100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000cfd0 tx=0x7f4e7009a380 comp rx=0 tx=0).stop 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 -- 172.21.15.105:0/2663029055 shutdown_connections 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e60025590 0x563e60027980 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e5ffc07c0 0x563e5ffb7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e5bfff640 0 client.29258 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e5ffad9a0 0x563e5ffb9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 --2- 172.21.15.105:0/2663029055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e5ffa7e40 0x563e5ffb9100 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 -- 172.21.15.105:0/2663029055 >> 172.21.15.105:0/2663029055 conn(0x563e5ff9efa0 msgr2=0x563e5ffa2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 -- 172.21.15.105:0/2663029055 shutdown_connections 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 1 -- 172.21.15.105:0/2663029055 wait complete. 2024-03-26T17:49:25.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.275+0000 7f4e85d82740 20 client.29258 trim_cache size 0 max 16384 2024-03-26T17:49:25.278 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:25.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 -- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4e6c0a2f20 msgr2=0x7f4e6c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4e6c0a2f20 0x7f4e6c0a53e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4e68005ab0 tx=0x7f4e68005fd0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 -- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1007c0 msgr2=0x7f4e7c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1007c0 0x7f4e7c197c30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000e9c0 tx=0x7f4e7000ee80 comp rx=0 tx=0).stop 2024-03-26T17:49:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 -- 172.21.15.105:0/4157293862 shutdown_connections 2024-03-26T17:49:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4e6c0a2f20 0x7f4e6c0a53e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e7c102370 0x7f4e7c19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4e7c1019b0 0x7f4e7c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 --2- 172.21.15.105:0/4157293862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e7c1007c0 0x7f4e7c197c30 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 -- 172.21.15.105:0/4157293862 >> 172.21.15.105:0/4157293862 conn(0x7f4e7c0fbd50 msgr2=0x7f4e7c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 -- 172.21.15.105:0/4157293862 shutdown_connections 2024-03-26T17:49:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.276+0000 7f4e8319d640 1 -- 172.21.15.105:0/4157293862 wait complete. 2024-03-26T17:49:25.280 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:49:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.404+0000 7f285c97e640 1 -- 172.21.15.105:0/2740097127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28581007c0 msgr2=0x7f2858100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.404+0000 7f285c97e640 1 --2- 172.21.15.105:0/2740097127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28581007c0 0x7f2858100be0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2848009900 tx=0x7f284802bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 -- 172.21.15.105:0/2740097127 shutdown_connections 2024-03-26T17:49:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 --2- 172.21.15.105:0/2740097127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2858102370 0x7f2858108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 --2- 172.21.15.105:0/2740097127 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28581019b0 0x7f2858101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 --2- 172.21.15.105:0/2740097127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28581007c0 0x7f2858100be0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 -- 172.21.15.105:0/2740097127 >> 172.21.15.105:0/2740097127 conn(0x7f28580fbd50 msgr2=0x7f28580fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 -- 172.21.15.105:0/2740097127 shutdown_connections 2024-03-26T17:49:25.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 -- 172.21.15.105:0/2740097127 wait complete. 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 Processor -- start 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.405+0000 7f285c97e640 1 -- start start 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f285c97e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28581007c0 0x7f2858196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f285c97e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28581019b0 0x7f2858196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f285c97e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2858102370 0x7f285819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f285c97e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f285810c2f0 con 0x7f2858102370 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f285c97e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f285810c170 con 0x7f28581007c0 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f285c97e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f285810c470 con 0x7f28581019b0 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2858102370 0x7f285819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f285659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28581007c0 0x7f2858196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2858102370 0x7f285819e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45788/0 (socket says 172.21.15.105:45788) 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 -- 172.21.15.105:0/1854225722 learned_addr learned my addr 172.21.15.105:0/1854225722 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2855d9b640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28581019b0 0x7f2858196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 -- 172.21.15.105:0/1854225722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28581019b0 msgr2=0x7f2858196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28581019b0 0x7f2858196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 -- 172.21.15.105:0/1854225722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28581007c0 msgr2=0x7f2858196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28581007c0 0x7f2858196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 -- 172.21.15.105:0/1854225722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285819e670 con 0x7f2858102370 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f2856d9d640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2858102370 0x7f285819e000 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f285000eb10 tx=0x7f285000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.406+0000 7f28477fe640 1 -- 172.21.15.105:0/1854225722 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850004b20 con 0x7f2858102370 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.407+0000 7f28477fe640 1 -- 172.21.15.105:0/1854225722 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2850004cc0 con 0x7f2858102370 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.407+0000 7f285c97e640 1 -- 172.21.15.105:0/1854225722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2858068a80 con 0x7f2858102370 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.407+0000 7f28477fe640 1 -- 172.21.15.105:0/1854225722 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850010690 con 0x7f2858102370 2024-03-26T17:49:25.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.407+0000 7f285c97e640 1 -- 172.21.15.105:0/1854225722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2858069020 con 0x7f2858102370 2024-03-26T17:49:25.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.409+0000 7f28477fe640 1 -- 172.21.15.105:0/1854225722 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2850005020 con 0x7f2858102370 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f28477fe640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28400a2fa0 0x7f28400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f28477fe640 1 -- 172.21.15.105:0/1854225722 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f28500cfa50 con 0x7f2858102370 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285659c640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28400a2fa0 0x7f28400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285f56a740 1 Processor -- start 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285f56a740 1 -- start start 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285f56a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a09a30 0x5604b6a09e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285f56a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a0ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285f56a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285f56a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5604b696e970 con 0x5604b6a09a30 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285f56a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5604b6916d70 con 0x5604b6a0f590 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285f56a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604b696a920 con 0x5604b6a0a420 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.410+0000 7f285659c640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28400a2fa0 0x7f28400a5460 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2848005ab0 tx=0x7f2848005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a09a30 0x5604b6a09e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a09a30 0x5604b6a09e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45792/0 (socket says 172.21.15.105:45792) 2024-03-26T17:49:25.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 -- 172.21.15.105:0/761618918 learned_addr learned my addr 172.21.15.105:0/761618918 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f2855d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a0ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f2856d9d640 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 -- 172.21.15.105:0/761618918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 msgr2=0x5604b6a0ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a0ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 -- 172.21.15.105:0/761618918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 msgr2=0x5604b6a11a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a11a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 -- 172.21.15.105:0/761618918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b66823f0 con 0x5604b6a09a30 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f2856d9d640 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f285659c640 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a09a30 0x5604b6a09e50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2858101810 tx=0x7f284803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925b1dbcee90b14d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f2844ff9640 1 -- 172.21.15.105:0/761618918 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284802cc40 con 0x5604b6a09a30 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f2844ff9640 1 -- 172.21.15.105:0/761618918 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2848050070 con 0x5604b6a09a30 2024-03-26T17:49:25.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.411+0000 7f2844ff9640 1 -- 172.21.15.105:0/761618918 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28480035f0 con 0x5604b6a09a30 2024-03-26T17:49:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 -- 172.21.15.105:0/761618918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a09a30 msgr2=0x5604b6a09e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a09a30 0x5604b6a09e50 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2858101810 tx=0x7f284803e000 comp rx=0 tx=0).stop 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 -- 172.21.15.105:0/761618918 shutdown_connections 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a0ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 --2- 172.21.15.105:0/761618918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a09a30 0x5604b6a09e50 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 -- 172.21.15.105:0/761618918 >> 172.21.15.105:0/761618918 conn(0x5604b6a04fa0 msgr2=0x5604b6a053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 -- 172.21.15.105:0/761618918 shutdown_connections 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.412+0000 7f285f56a740 1 -- 172.21.15.105:0/761618918 wait complete. 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 1 -- start start 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 1 Processor -- start 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a13ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a1b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5604b6a124e0 con 0x5604b6a0f590 2024-03-26T17:49:25.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5604b692a310 con 0x5604b6a09a30 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5604b6a12660 con 0x5604b6a0a420 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285f56a740 10 client.0 _ll_register_callbacks cb 0x5604b6965c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a13ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f2856d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a1b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a13ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49422/0 (socket says 172.21.15.105:49422) 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285659c640 1 -- 172.21.15.105:0/889083889 learned_addr learned my addr 172.21.15.105:0/889083889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f2855d9b640 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285659c640 1 -- 172.21.15.105:0/889083889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 msgr2=0x5604b6a14430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.413+0000 7f285659c640 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a14430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f285659c640 1 -- 172.21.15.105:0/889083889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 msgr2=0x5604b6a1b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f285659c640 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a1b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f285659c640 1 -- 172.21.15.105:0/889083889 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b674c7d0 con 0x5604b6a09a30 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f2856d9d640 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a1b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f285659c640 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a13ef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2848036040 tx=0x7f2848004e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f13f0510fbe4220 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f283dffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f283dffb640 1 -- 172.21.15.105:0/889083889 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848005250 con 0x5604b6a09a30 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f285f56a740 1 -- 172.21.15.105:0/889083889 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2848050070 con 0x5604b6a09a30 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f283dffb640 1 -- 172.21.15.105:0/889083889 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f28480053f0 con 0x5604b6a09a30 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f283dffb640 1 -- 172.21.15.105:0/889083889 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f28480035f0 con 0x5604b6a09a30 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f285f56a740 10 client.29270 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f285f56a740 1 -- 172.21.15.105:0/889083889 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5604b6a1e7e0 con 0x5604b6a09a30 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.414+0000 7f283dffb640 1 -- 172.21.15.105:0/889083889 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2848050070 con 0x5604b6a09a30 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f283dffb640 1 -- 172.21.15.105:0/889083889 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f28480417d0 con 0x5604b6a09a30 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f285f56a740 10 client.29270 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f285f56a740 10 client.29270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f285f56a740 20 client.29270 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f285f56a740 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604b6a1fb30 0x5604b6a21f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f285f56a740 4 client.29270 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f285f56a740 1 -- 172.21.15.105:0/889083889 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604b697beb0 con 0x5604b6a1fb30 2024-03-26T17:49:25.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f2855d9b640 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604b6a1fb30 0x5604b6a21f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f2855d9b640 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604b6a1fb30 0x5604b6a21f20 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4743160df0654a server_cookie=869a480cdb798eb9 in_seq=0 out_seq=0 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.415+0000 7f283dffb640 10 client.29270 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f283dffb640 1 -- 172.21.15.105:0/889083889 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5604b697beb0 con 0x5604b6a1fb30 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f283dffb640 10 client.29270 handle_command_reply: tid=0 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 client.29270 shutdown 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 -- 172.21.15.105:0/889083889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 msgr2=0x5604b6a13ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a13ef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2848036040 tx=0x7f2848004e10 comp rx=0 tx=0).stop 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 -- 172.21.15.105:0/889083889 shutdown_connections 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604b6a1fb30 0x5604b6a21f20 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a1b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f283dffb640 0 client.29270 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a14430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 --2- 172.21.15.105:0/889083889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a13ef0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.416+0000 7f285f56a740 1 -- 172.21.15.105:0/889083889 >> 172.21.15.105:0/889083889 conn(0x5604b6a04fa0 msgr2=0x5604b6a041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.417+0000 7f285f56a740 1 -- 172.21.15.105:0/889083889 shutdown_connections 2024-03-26T17:49:25.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.417+0000 7f285f56a740 1 -- 172.21.15.105:0/889083889 wait complete. 2024-03-26T17:49:25.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.417+0000 7f285f56a740 20 client.29270 trim_cache size 0 max 16384 2024-03-26T17:49:25.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.431+0000 7f285f56a740 1 Processor -- start 2024-03-26T17:49:25.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.431+0000 7f285f56a740 1 -- start start 2024-03-26T17:49:25.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.431+0000 7f285f56a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a14390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.431+0000 7f285f56a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a1b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.431+0000 7f285f56a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a08910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.431+0000 7f285f56a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28581a6c90 con 0x5604b6a0f590 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.431+0000 7f285f56a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28581a87d0 con 0x5604b6a09a30 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.431+0000 7f285f56a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28581a7360 con 0x5604b6a0a420 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a14390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f2855d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a1b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f2856d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a08910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a14390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49434/0 (socket says 172.21.15.105:49434) 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 -- 172.21.15.105:0/428109320 learned_addr learned my addr 172.21.15.105:0/428109320 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f2855d9b640 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a1b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37730/0 (socket says 172.21.15.105:37730) 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 -- 172.21.15.105:0/428109320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 msgr2=0x5604b6a1b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a1b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 -- 172.21.15.105:0/428109320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 msgr2=0x5604b6a08910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a08910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 -- 172.21.15.105:0/428109320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b6a22500 con 0x5604b6a09a30 2024-03-26T17:49:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f2856d9d640 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a08910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f285659c640 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a14390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f284802ca90 tx=0x7f284802cd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897d077eb8fc3ea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f283ffff640 1 -- 172.21.15.105:0/428109320 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284803a040 con 0x5604b6a09a30 2024-03-26T17:49:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f283ffff640 1 -- 172.21.15.105:0/428109320 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2848050040 con 0x5604b6a09a30 2024-03-26T17:49:25.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.432+0000 7f283ffff640 1 -- 172.21.15.105:0/428109320 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848003790 con 0x5604b6a09a30 2024-03-26T17:49:25.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 -- 172.21.15.105:0/428109320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 msgr2=0x5604b6a14390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a14390 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f284802ca90 tx=0x7f284802cd00 comp rx=0 tx=0).stop 2024-03-26T17:49:25.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 -- 172.21.15.105:0/428109320 shutdown_connections 2024-03-26T17:49:25.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0f590 0x5604b6a08910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a0a420 0x5604b6a1b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 --2- 172.21.15.105:0/428109320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a09a30 0x5604b6a14390 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 -- 172.21.15.105:0/428109320 >> 172.21.15.105:0/428109320 conn(0x5604b6a04fa0 msgr2=0x5604b6a08200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 -- 172.21.15.105:0/428109320 shutdown_connections 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 -- 172.21.15.105:0/428109320 wait complete. 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 -- start start 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.433+0000 7f285f56a740 1 Processor -- start 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285f56a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a09a30 0x5604b6a18750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285f56a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0a420 0x5604b6a18c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285f56a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a19c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285f56a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28581a8100 con 0x5604b6a0a420 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285f56a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28581a7a30 con 0x5604b6a0f590 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285f56a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28581a4c70 con 0x5604b6a09a30 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285f56a740 10 client.0 _ll_register_callbacks cb 0x7f2850004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:25.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a09a30 0x5604b6a18750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a09a30 0x5604b6a18750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37744/0 (socket says 172.21.15.105:37744) 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f2855d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0a420 0x5604b6a18c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 -- 172.21.15.105:0/2447411212 learned_addr learned my addr 172.21.15.105:0/2447411212 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f2856d9d640 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a19c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 -- 172.21.15.105:0/2447411212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 msgr2=0x5604b6a19c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a19c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 -- 172.21.15.105:0/2447411212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0a420 msgr2=0x5604b6a18c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0a420 0x5604b6a18c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 -- 172.21.15.105:0/2447411212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b6a1a240 con 0x5604b6a09a30 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f285659c640 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a09a30 0x5604b6a18750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2848041c40 tx=0x7f2848005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff10a707405ee9e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f283dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f283dffb640 1 -- 172.21.15.105:0/2447411212 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284803a040 con 0x5604b6a09a30 2024-03-26T17:49:25.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.434+0000 7f283dffb640 1 -- 172.21.15.105:0/2447411212 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f28480049d0 con 0x5604b6a09a30 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 1 -- 172.21.15.105:0/2447411212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2848050040 con 0x5604b6a09a30 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f283dffb640 1 -- 172.21.15.105:0/2447411212 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2848005460 con 0x5604b6a09a30 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 10 client.29266 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 1 -- 172.21.15.105:0/2447411212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5604b6a1b540 con 0x5604b6a09a30 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f283dffb640 1 -- 172.21.15.105:0/2447411212 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2848050040 con 0x5604b6a09a30 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f283dffb640 1 -- 172.21.15.105:0/2447411212 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f28480346a0 con 0x5604b6a09a30 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 10 client.29266 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 10 client.29266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 20 client.29266 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604b6a85ff0 0x5604b6a883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 4 client.29266 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f285f56a740 1 -- 172.21.15.105:0/2447411212 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604b6976200 con 0x5604b6a85ff0 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.435+0000 7f2855d9b640 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604b6a85ff0 0x5604b6a883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:25.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f2855d9b640 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604b6a85ff0 0x5604b6a883e0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925b6c2416c61013 server_cookie=b6a2466c54f4a964 in_seq=0 out_seq=0 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f283dffb640 10 client.29266 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f283dffb640 1 -- 172.21.15.105:0/2447411212 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604b6976200 con 0x5604b6a85ff0 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f283dffb640 10 client.29266 handle_command_reply: tid=0 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 client.29266 shutdown 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 -- 172.21.15.105:0/2447411212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a09a30 msgr2=0x5604b6a18750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a09a30 0x5604b6a18750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2848041c40 tx=0x7f2848005c90 comp rx=0 tx=0).stop 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 -- 172.21.15.105:0/2447411212 shutdown_connections 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5604b6a85ff0 0x5604b6a883e0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5604b6a0f590 0x5604b6a19c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5604b6a0a420 0x5604b6a18c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f283dffb640 0 client.29266 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 --2- 172.21.15.105:0/2447411212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5604b6a09a30 0x5604b6a18750 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 -- 172.21.15.105:0/2447411212 >> 172.21.15.105:0/2447411212 conn(0x5604b6a04fa0 msgr2=0x5604b6a14180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 -- 172.21.15.105:0/2447411212 shutdown_connections 2024-03-26T17:49:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 1 -- 172.21.15.105:0/2447411212 wait complete. 2024-03-26T17:49:25.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.436+0000 7f285f56a740 20 client.29266 trim_cache size 0 max 16384 2024-03-26T17:49:25.435 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 -- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28400a2fa0 msgr2=0x7f28400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28400a2fa0 0x7f28400a5460 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2848005ab0 tx=0x7f2848005fd0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 -- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2858102370 msgr2=0x7f285819e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2858102370 0x7f285819e000 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f285000eb10 tx=0x7f285000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 -- 172.21.15.105:0/1854225722 shutdown_connections 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28400a2fa0 0x7f28400a5460 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2858102370 0x7f285819e000 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28581019b0 0x7f2858196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 --2- 172.21.15.105:0/1854225722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28581007c0 0x7f2858196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 -- 172.21.15.105:0/1854225722 >> 172.21.15.105:0/1854225722 conn(0x7f28580fbd50 msgr2=0x7f2858104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 -- 172.21.15.105:0/1854225722 shutdown_connections 2024-03-26T17:49:25.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:25.437+0000 7f285c97e640 1 -- 172.21.15.105:0/1854225722 wait complete. 2024-03-26T17:49:26.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 356710 3.14 MB/sec execute 489 sec latency 19.384 ms 2024-03-26T17:49:26.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:25 smithi144 ceph-mon[36095]: pgmap v681: 97 pgs: 97 active+clean; 25 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:49:26.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:25 smithi105 ceph-mon[32280]: pgmap v681: 97 pgs: 97 active+clean; 25 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:49:26.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:25 smithi135 ceph-mon[35636]: pgmap v681: 97 pgs: 97 active+clean; 25 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:49:26.442 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:49:26.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:49:26.443 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.577+0000 7f4c15481640 1 -- 172.21.15.105:0/593250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c101007c0 msgr2=0x7f4c10100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.577+0000 7f4c15481640 1 --2- 172.21.15.105:0/593250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c101007c0 0x7f4c10100be0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009900 tx=0x7f4c0002bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.577+0000 7f4c15481640 1 -- 172.21.15.105:0/593250 shutdown_connections 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.577+0000 7f4c15481640 1 --2- 172.21.15.105:0/593250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c10102370 0x7f4c10108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.578+0000 7f4c15481640 1 --2- 172.21.15.105:0/593250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c101019b0 0x7f4c10101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.578+0000 7f4c15481640 1 --2- 172.21.15.105:0/593250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c101007c0 0x7f4c10100be0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.578+0000 7f4c15481640 1 -- 172.21.15.105:0/593250 >> 172.21.15.105:0/593250 conn(0x7f4c100fbd50 msgr2=0x7f4c100fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.578+0000 7f4c15481640 1 -- 172.21.15.105:0/593250 shutdown_connections 2024-03-26T17:49:26.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.578+0000 7f4c15481640 1 -- 172.21.15.105:0/593250 wait complete. 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.578+0000 7f4c15481640 1 Processor -- start 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.578+0000 7f4c15481640 1 -- start start 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c15481640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c101007c0 0x7f4c10197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c15481640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c101019b0 0x7f4c101980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c15481640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c10102370 0x7f4c1019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c15481640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1010c330 con 0x7f4c10102370 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c15481640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1010c1b0 con 0x7f4c101007c0 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c15481640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1010c4b0 con 0x7f4c101019b0 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c10102370 0x7f4c1019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c101007c0 0x7f4c10197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c101019b0 0x7f4c101980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c10102370 0x7f4c1019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45828/0 (socket says 172.21.15.105:45828) 2024-03-26T17:49:26.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/743419252 learned_addr learned my addr 172.21.15.105:0/743419252 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/743419252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c101019b0 msgr2=0x7f4c101980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c101019b0 0x7f4c101980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/743419252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c101007c0 msgr2=0x7f4c10197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c101007c0 0x7f4c10197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/743419252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1019f780 con 0x7f4c10102370 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0effd640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c101007c0 0x7f4c10197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.579+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c10102370 0x7f4c1019f110 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800eb10 tx=0x7f4c0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.580+0000 7f4bf7fff640 1 -- 172.21.15.105:0/743419252 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08004b20 con 0x7f4c10102370 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.580+0000 7f4c15481640 1 -- 172.21.15.105:0/743419252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1019fa70 con 0x7f4c10102370 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.580+0000 7f4bf7fff640 1 -- 172.21.15.105:0/743419252 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c08004cc0 con 0x7f4c10102370 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.580+0000 7f4bf7fff640 1 -- 172.21.15.105:0/743419252 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08010690 con 0x7f4c10102370 2024-03-26T17:49:26.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.580+0000 7f4c15481640 1 -- 172.21.15.105:0/743419252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c1019ff70 con 0x7f4c10102370 2024-03-26T17:49:26.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.582+0000 7f4bf7fff640 1 -- 172.21.15.105:0/743419252 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4c08005020 con 0x7f4c10102370 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4bf7fff640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c1806d740 1 Processor -- start 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c1806d740 1 -- start start 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4bf7fff640 1 -- 172.21.15.105:0/743419252 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4c080cf980 con 0x7f4c10102370 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81190 0x563796c815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c86680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796c89180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c1806d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563796be6630 con 0x563796c81b80 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c1806d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563796b8ee10 con 0x563796c86cf0 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c1806d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563796be25e0 con 0x563796c81190 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.583+0000 7f4c0effd640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796c89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796c89180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49462/0 (socket says 172.21.15.105:49462) 2024-03-26T17:49:26.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/3544615519 learned_addr learned my addr 172.21.15.105:0/3544615519 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/3544615519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81190 msgr2=0x563796c815b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/3544615519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81190 0x563796c815b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/3544615519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 msgr2=0x563796c86680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/3544615519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c86680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/3544615519 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563796ba5740 con 0x563796c86cf0 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0effd640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4c0002c240 tx=0x7f4c0002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.584+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/3544615519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796c89180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4c080cfca0 tx=0x7f4c08098440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecac3f9a8e5a3710 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.585+0000 7f4bf57fa640 1 -- 172.21.15.105:0/3544615519 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0809c070 con 0x563796c86cf0 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.585+0000 7f4bf57fa640 1 -- 172.21.15.105:0/3544615519 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c080a01c0 con 0x563796c86cf0 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.585+0000 7f4bf57fa640 1 -- 172.21.15.105:0/3544615519 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c080bc070 con 0x563796c86cf0 2024-03-26T17:49:26.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.585+0000 7f4c1806d740 1 -- 172.21.15.105:0/3544615519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 msgr2=0x563796c89180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.586+0000 7f4c1806d740 1 --2- 172.21.15.105:0/3544615519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796c89180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4c080cfca0 tx=0x7f4c08098440 comp rx=0 tx=0).stop 2024-03-26T17:49:26.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.586+0000 7f4c1806d740 1 -- 172.21.15.105:0/3544615519 shutdown_connections 2024-03-26T17:49:26.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.586+0000 7f4c1806d740 1 --2- 172.21.15.105:0/3544615519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796c89180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.586+0000 7f4c1806d740 1 --2- 172.21.15.105:0/3544615519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c86680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.586+0000 7f4c1806d740 1 --2- 172.21.15.105:0/3544615519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81190 0x563796c815b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.586+0000 7f4c1806d740 1 -- 172.21.15.105:0/3544615519 >> 172.21.15.105:0/3544615519 conn(0x563796c7c700 msgr2=0x563796c7cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.586+0000 7f4c1806d740 1 -- 172.21.15.105:0/3544615519 shutdown_connections 2024-03-26T17:49:26.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.586+0000 7f4c1806d740 1 -- 172.21.15.105:0/3544615519 wait complete. 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 1 -- start start 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 1 Processor -- start 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796c9c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81b80 0x563796c9cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796ca3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563796c9acb0 con 0x563796c81190 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563796ba21a0 con 0x563796c86cf0 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563796c9ae30 con 0x563796c81b80 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c1806d740 10 client.0 _ll_register_callbacks cb 0x5637968f9c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c0f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796ca3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c0e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81b80 0x563796c9cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c0f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796ca3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49478/0 (socket says 172.21.15.105:49478) 2024-03-26T17:49:26.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/476003973 learned_addr learned my addr 172.21.15.105:0/476003973 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.587+0000 7f4c0effd640 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796c9c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c0e7fc640 1 -- 172.21.15.105:0/476003973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 msgr2=0x563796ca3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c0e7fc640 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796ca3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c0e7fc640 1 -- 172.21.15.105:0/476003973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 msgr2=0x563796c9c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c0e7fc640 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796c9c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c0e7fc640 1 -- 172.21.15.105:0/476003973 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637969c27e0 con 0x563796c81b80 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c0e7fc640 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81b80 0x563796c9cc80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x563796c81f80 tx=0x7f4c04012b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b7e0830d0ca0ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4bf2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/476003973 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04004020 con 0x563796c81b80 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c1806d740 1 -- 172.21.15.105:0/476003973 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c080a01c0 con 0x563796c81b80 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/476003973 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4c040041c0 con 0x563796c81b80 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/476003973 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c040043c0 con 0x563796c81b80 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c1806d740 10 client.29275 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.588+0000 7f4c1806d740 1 -- 172.21.15.105:0/476003973 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563796a22f70 con 0x563796c81b80 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/476003973 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c080a01c0 con 0x563796c81b80 2024-03-26T17:49:26.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/476003973 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c04003b00 con 0x563796c81b80 2024-03-26T17:49:26.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4c1806d740 10 client.29275 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:26.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4c1806d740 10 client.29275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:26.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4c1806d740 20 client.29275 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:26.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4c1806d740 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563796ca8530 0x563796caa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4c1806d740 4 client.29275 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:26.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4c1806d740 1 -- 172.21.15.105:0/476003973 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563796bf3b70 con 0x563796ca8530 2024-03-26T17:49:26.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.589+0000 7f4c0effd640 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563796ca8530 0x563796caa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.591+0000 7f4bf2ff5640 10 client.29275 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.591+0000 7f4c0effd640 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563796ca8530 0x563796caa9d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df0dbc2e8147a63a server_cookie=fd97d5fa8066e02b in_seq=0 out_seq=0 2024-03-26T17:49:26.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.592+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/476003973 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563796bf3b70 con 0x563796ca8530 2024-03-26T17:49:26.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.592+0000 7f4bf2ff5640 10 client.29275 handle_command_reply: tid=0 2024-03-26T17:49:26.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.592+0000 7f4c1806d740 1 client.29275 shutdown 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4c1806d740 1 -- 172.21.15.105:0/476003973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81b80 msgr2=0x563796c9cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4c1806d740 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81b80 0x563796c9cc80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x563796c81f80 tx=0x7f4c04012b40 comp rx=0 tx=0).stop 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4c1806d740 1 -- 172.21.15.105:0/476003973 shutdown_connections 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4c1806d740 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563796ca8530 0x563796caa9d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4c1806d740 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c86cf0 0x563796ca3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4bf2ff5640 0 client.29275 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4c1806d740 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c81b80 0x563796c9cc80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4c1806d740 1 --2- 172.21.15.105:0/476003973 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796c9c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.593+0000 7f4c1806d740 1 -- 172.21.15.105:0/476003973 >> 172.21.15.105:0/476003973 conn(0x563796c7c700 msgr2=0x563796c7b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.594+0000 7f4c1806d740 1 -- 172.21.15.105:0/476003973 shutdown_connections 2024-03-26T17:49:26.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.594+0000 7f4c1806d740 1 -- 172.21.15.105:0/476003973 wait complete. 2024-03-26T17:49:26.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.594+0000 7f4c1806d740 20 client.29275 trim_cache size 0 max 16384 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.615+0000 7fba46b1b640 1 -- 172.21.15.105:0/787742302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba401023e0 msgr2=0x7fba401047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.615+0000 7fba46b1b640 1 --2- 172.21.15.105:0/787742302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba401023e0 0x7fba401047d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fba34009900 tx=0x7fba3402bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.615+0000 7fba46b1b640 1 -- 172.21.15.105:0/787742302 shutdown_connections 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.615+0000 7fba46b1b640 1 --2- 172.21.15.105:0/787742302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba40107640 0x7fba40109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.615+0000 7fba46b1b640 1 --2- 172.21.15.105:0/787742302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba40104d10 0x7fba40107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.615+0000 7fba46b1b640 1 --2- 172.21.15.105:0/787742302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba401023e0 0x7fba401047d0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.615+0000 7fba46b1b640 1 -- 172.21.15.105:0/787742302 >> 172.21.15.105:0/787742302 conn(0x7fba400fbd70 msgr2=0x7fba400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.616+0000 7fba46b1b640 1 -- 172.21.15.105:0/787742302 shutdown_connections 2024-03-26T17:49:26.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.616+0000 7fba46b1b640 1 -- 172.21.15.105:0/787742302 wait complete. 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.616+0000 7fba46b1b640 1 Processor -- start 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.616+0000 7fba46b1b640 1 -- start start 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba46b1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba401023e0 0x7fba40197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba46b1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba40104d10 0x7fba40198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba46b1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba40107640 0x7fba4019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba46b1b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010d420 con 0x7fba40107640 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba46b1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010d2a0 con 0x7fba401023e0 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba46b1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba4010d5a0 con 0x7fba40104d10 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba448b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba401023e0 0x7fba40197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba450b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba40107640 0x7fba4019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba448b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba401023e0 0x7fba40197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49504/0 (socket says 172.21.15.105:49504) 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba448b7640 1 -- 172.21.15.105:0/1331718851 learned_addr learned my addr 172.21.15.105:0/1331718851 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba3ffff640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba40104d10 0x7fba40198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba450b8640 1 -- 172.21.15.105:0/1331718851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba40104d10 msgr2=0x7fba40198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba450b8640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba40104d10 0x7fba40198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba450b8640 1 -- 172.21.15.105:0/1331718851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba401023e0 msgr2=0x7fba40197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba450b8640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba401023e0 0x7fba40197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba450b8640 1 -- 172.21.15.105:0/1331718851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba4019f710 con 0x7fba40107640 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba450b8640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba40107640 0x7fba4019f0a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fba38009fd0 tx=0x7fba3800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba3dffb640 1 -- 172.21.15.105:0/1331718851 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3800ce20 con 0x7fba40107640 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba3dffb640 1 -- 172.21.15.105:0/1331718851 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba38005020 con 0x7fba40107640 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba46b1b640 1 -- 172.21.15.105:0/1331718851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4019fa00 con 0x7fba40107640 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.617+0000 7fba3dffb640 1 -- 172.21.15.105:0/1331718851 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba38010690 con 0x7fba40107640 2024-03-26T17:49:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.618+0000 7fba46b1b640 1 -- 172.21.15.105:0/1331718851 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba4019ff00 con 0x7fba40107640 2024-03-26T17:49:26.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.619+0000 7f4c1806d740 1 Processor -- start 2024-03-26T17:49:26.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.619+0000 7f4c1806d740 1 -- start start 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.619+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796c7b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.619+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81190 0x563796ca3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c7b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c1806d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c10071c50 con 0x563796c81b80 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c1806d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c10073790 con 0x563796c81190 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c1806d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c10072320 con 0x563796c7c700 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796c7b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c7b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796c7b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37804/0 (socket says 172.21.15.105:37804) 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c7b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45878/0 (socket says 172.21.15.105:45878) 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81190 0x563796ca3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 -- 172.21.15.105:0/3212256222 learned_addr learned my addr 172.21.15.105:0/3212256222 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 -- 172.21.15.105:0/3212256222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81190 msgr2=0x563796ca3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 --2- 172.21.15.105:0/3212256222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81190 0x563796ca3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 -- 172.21.15.105:0/3212256222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 msgr2=0x563796c7b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 --2- 172.21.15.105:0/3212256222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c7b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 -- 172.21.15.105:0/3212256222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637968ba170 con 0x563796c7c700 2024-03-26T17:49:26.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0e7fc640 1 --2- 172.21.15.105:0/3212256222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81190 0x563796ca3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:26.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4c0effd640 1 --2- 172.21.15.105:0/3212256222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796c7b070 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009900 tx=0x7f4c0003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=310ad64b38cb0848 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4bf4ff9640 1 -- 172.21.15.105:0/3212256222 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0005b070 con 0x563796c7c700 2024-03-26T17:49:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.620+0000 7fba3dffb640 1 -- 172.21.15.105:0/1331718851 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fba380051d0 con 0x7fba40107640 2024-03-26T17:49:26.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.620+0000 7f4bf4ff9640 1 -- 172.21.15.105:0/3212256222 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c00005510 con 0x563796c7c700 2024-03-26T17:49:26.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.621+0000 7f4bf4ff9640 1 -- 172.21.15.105:0/3212256222 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c000038e0 con 0x563796c7c700 2024-03-26T17:49:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba497e4740 1 Processor -- start 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba497e4740 1 -- start start 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba497e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56262771e5d0 0x56262771e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba497e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627723ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba497e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562627724130 0x5626277265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba497e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5626276830b0 con 0x562627724130 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba497e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56262762b960 con 0x56262771efc0 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba497e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56262767f060 con 0x56262771e5d0 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562627724130 0x5626277265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba448b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627723ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562627724130 0x5626277265c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45882/0 (socket says 172.21.15.105:45882) 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba3ffff640 1 -- 172.21.15.105:0/145727470 learned_addr learned my addr 172.21.15.105:0/145727470 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba3ffff640 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56262771e5d0 0x56262771e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.621+0000 7fba3dffb640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba300a2fc0 0x7fba300a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.621+0000 7f4c1806d740 1 -- 172.21.15.105:0/3212256222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 msgr2=0x563796c7b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.621+0000 7f4c1806d740 1 --2- 172.21.15.105:0/3212256222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796c7b070 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009900 tx=0x7f4c0003e000 comp rx=0 tx=0).stop 2024-03-26T17:49:26.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.621+0000 7f4c1806d740 1 -- 172.21.15.105:0/3212256222 shutdown_connections 2024-03-26T17:49:26.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.621+0000 7f4c1806d740 1 --2- 172.21.15.105:0/3212256222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81b80 0x563796c7b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.621+0000 7f4c1806d740 1 --2- 172.21.15.105:0/3212256222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81190 0x563796ca3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.621+0000 7f4c1806d740 1 --2- 172.21.15.105:0/3212256222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796c7b070 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.622+0000 7f4c1806d740 1 -- 172.21.15.105:0/3212256222 >> 172.21.15.105:0/3212256222 conn(0x563796c77780 msgr2=0x563796c9b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba448b7640 1 -- 172.21.15.105:0/145727470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56262771e5d0 msgr2=0x56262771e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba448b7640 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56262771e5d0 0x56262771e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba448b7640 1 -- 172.21.15.105:0/145727470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562627724130 msgr2=0x5626277265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba448b7640 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562627724130 0x5626277265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba450b8640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba300a2fc0 0x7fba300a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba448b7640 1 -- 172.21.15.105:0/145727470 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262763dc60 con 0x56262771efc0 2024-03-26T17:49:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba3ffff640 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562627724130 0x5626277265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:26.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.622+0000 7f4c1806d740 1 -- 172.21.15.105:0/3212256222 shutdown_connections 2024-03-26T17:49:26.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.622+0000 7f4c1806d740 1 -- 172.21.15.105:0/3212256222 wait complete. 2024-03-26T17:49:26.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba448b7640 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627723ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fba34009230 tx=0x7fba340095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5821852720d514eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba3dffb640 1 -- 172.21.15.105:0/1331718851 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fba380cfb50 con 0x7fba40107640 2024-03-26T17:49:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba2affd640 1 -- 172.21.15.105:0/145727470 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34045020 con 0x56262771efc0 2024-03-26T17:49:26.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.622+0000 7f4c1806d740 1 -- start start 2024-03-26T17:49:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba450b8640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba300a2fc0 0x7fba300a5480 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fba38096c80 tx=0x7fba380984c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba2affd640 1 -- 172.21.15.105:0/145727470 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba3402c9a0 con 0x56262771efc0 2024-03-26T17:49:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.622+0000 7fba2affd640 1 -- 172.21.15.105:0/145727470 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34005510 con 0x56262771efc0 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.622+0000 7f4c1806d740 1 Processor -- start 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.622+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796ca1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.622+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796ca1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.622+0000 7f4c1806d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81b80 0x563796ca2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c1806d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c100730c0 con 0x563796c81190 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c1806d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c100729f0 con 0x563796c81b80 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c1806d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1006fc30 con 0x563796c7c700 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796ca1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796ca1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45888/0 (socket says 172.21.15.105:45888) 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0e7fc640 1 -- 172.21.15.105:0/613432229 learned_addr learned my addr 172.21.15.105:0/613432229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81b80 0x563796ca2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c1806d740 10 client.0 _ll_register_callbacks cb 0x7f4c04009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0effd640 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796ca1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/613432229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 msgr2=0x563796ca1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796ca1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/613432229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 msgr2=0x563796ca1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796ca1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0f7fe640 1 -- 172.21.15.105:0/613432229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563796ca3240 con 0x563796c81b80 2024-03-26T17:49:26.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0e7fc640 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796ca1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 -- 172.21.15.105:0/145727470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 msgr2=0x562627723ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627723ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fba34009230 tx=0x7fba340095e0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 -- 172.21.15.105:0/145727470 shutdown_connections 2024-03-26T17:49:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562627724130 0x5626277265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c0f7fe640 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81b80 0x563796ca2c50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4c08076c60 tx=0x7f4c08098310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2cb51c119d706f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4bf2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627723ac0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 --2- 172.21.15.105:0/145727470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56262771e5d0 0x56262771e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 -- 172.21.15.105:0/145727470 >> 172.21.15.105:0/145727470 conn(0x562627719b40 msgr2=0x562627719f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 -- 172.21.15.105:0/145727470 shutdown_connections 2024-03-26T17:49:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.623+0000 7fba497e4740 1 -- 172.21.15.105:0/145727470 wait complete. 2024-03-26T17:49:26.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/613432229 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c080a01c0 con 0x563796c81b80 2024-03-26T17:49:26.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/613432229 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4c080767e0 con 0x563796c81b80 2024-03-26T17:49:26.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.623+0000 7f4c1806d740 1 -- 172.21.15.105:0/613432229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c00005510 con 0x563796c81b80 2024-03-26T17:49:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 1 -- start start 2024-03-26T17:49:26.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/613432229 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c080769e0 con 0x563796c81b80 2024-03-26T17:49:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 1 Processor -- start 2024-03-26T17:49:26.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c1806d740 10 client.29303 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:26.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c1806d740 1 -- 172.21.15.105:0/613432229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563796d0ebd0 con 0x563796c81b80 2024-03-26T17:49:26.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/613432229 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c0809c2d0 con 0x563796c81b80 2024-03-26T17:49:26.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/613432229 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c08005310 con 0x563796c81b80 2024-03-26T17:49:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627728320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627728860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c1806d740 10 client.29303 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:26.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c1806d740 10 client.29303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:26.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c1806d740 20 client.29303 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c1806d740 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563796c86cf0 0x563796d11140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c1806d740 4 client.29303 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:49:26.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c1806d740 1 -- 172.21.15.105:0/613432229 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f4c00005510 con 0x563796c86cf0 2024-03-26T17:49:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562627727080 con 0x56262771e5d0 2024-03-26T17:49:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56262763edb0 con 0x56262771efc0 2024-03-26T17:49:26.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.624+0000 7f4c0effd640 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563796c86cf0 0x563796d11140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562627727200 con 0x562627724130 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba497e4740 10 client.0 _ll_register_callbacks cb 0x562627643110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba448b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627728860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba448b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627728860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49524/0 (socket says 172.21.15.105:49524) 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba448b7640 1 -- 172.21.15.105:0/785901322 learned_addr learned my addr 172.21.15.105:0/785901322 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.624+0000 7fba3ffff640 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627728320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba448b7640 1 -- 172.21.15.105:0/785901322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 msgr2=0x56262772f8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba448b7640 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba448b7640 1 -- 172.21.15.105:0/785901322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 msgr2=0x562627728320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba448b7640 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627728320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.625+0000 7f4bf2ff5640 10 client.29303 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba448b7640 1 -- 172.21.15.105:0/785901322 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562627397210 con 0x56262771efc0 2024-03-26T17:49:26.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba3ffff640 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627728320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:26.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.625+0000 7f4c0effd640 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563796c86cf0 0x563796d11140 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f8d36dc01e523ab server_cookie=f31a2c002a18837f in_seq=0 out_seq=0 2024-03-26T17:49:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba448b7640 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627728860 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba34000c00 tx=0x7fba34009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1da6cbb0ecad8dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:26.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba1ffff640 1 -- 172.21.15.105:0/785901322 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34045070 con 0x56262771efc0 2024-03-26T17:49:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba497e4740 1 -- 172.21.15.105:0/785901322 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba3402c9a0 con 0x56262771efc0 2024-03-26T17:49:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba1ffff640 1 -- 172.21.15.105:0/785901322 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fba34004450 con 0x56262771efc0 2024-03-26T17:49:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba1ffff640 1 -- 172.21.15.105:0/785901322 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fba340046c0 con 0x56262771efc0 2024-03-26T17:49:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba497e4740 10 client.29306 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba497e4740 1 -- 172.21.15.105:0/785901322 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562627463620 con 0x56262771efc0 2024-03-26T17:49:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.625+0000 7fba1ffff640 1 -- 172.21.15.105:0/785901322 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba3402c9a0 con 0x56262771efc0 2024-03-26T17:49:26.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4bf2ff5640 1 -- 172.21.15.105:0/613432229 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f4c00005510 con 0x563796c86cf0 2024-03-26T17:49:26.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4bf2ff5640 10 client.29303 handle_command_reply: tid=0 2024-03-26T17:49:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.626+0000 7fba1ffff640 1 -- 172.21.15.105:0/785901322 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fba34003b20 con 0x56262771efc0 2024-03-26T17:49:26.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 client.29303 shutdown 2024-03-26T17:49:26.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.626+0000 7fba497e4740 10 client.29306 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.626+0000 7fba497e4740 10 client.29306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.626+0000 7fba497e4740 20 client.29306 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.626+0000 7fba497e4740 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626277381a0 0x56262773a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.626+0000 7fba497e4740 4 client.29306 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.626+0000 7fba497e4740 1 -- 172.21.15.105:0/785901322 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626276905f0 con 0x5626277381a0 2024-03-26T17:49:26.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 -- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81b80 msgr2=0x563796ca2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81b80 0x563796ca2c50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4c08076c60 tx=0x7f4c08098310 comp rx=0 tx=0).stop 2024-03-26T17:49:26.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 -- 172.21.15.105:0/613432229 shutdown_connections 2024-03-26T17:49:26.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563796c86cf0 0x563796d11140 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563796c81b80 0x563796ca2c50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4bf2ff5640 0 client.29303 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.626+0000 7fba3ffff640 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626277381a0 0x56262773a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563796c81190 0x563796ca1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 --2- 172.21.15.105:0/613432229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563796c7c700 0x563796ca1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 -- 172.21.15.105:0/613432229 >> 172.21.15.105:0/613432229 conn(0x563796c77780 msgr2=0x563796c7f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.626+0000 7f4c1806d740 1 -- 172.21.15.105:0/613432229 shutdown_connections 2024-03-26T17:49:26.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.627+0000 7f4c1806d740 1 -- 172.21.15.105:0/613432229 wait complete. 2024-03-26T17:49:26.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.627+0000 7f4c1806d740 20 client.29303 trim_cache size 0 max 16384 2024-03-26T17:49:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.627+0000 7fba1ffff640 10 client.29306 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.627+0000 7fba3ffff640 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626277381a0 0x56262773a590 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24114fd949131ef server_cookie=3b4d4a1a3143bb4a in_seq=0 out_seq=0 2024-03-26T17:49:26.641 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:49:26.641 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:49:26.641 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "b6872ff8-9911-4356-bd82-e1bbd2aebfab", 2024-03-26T17:49:26.641 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:49:26.641 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:49:26.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.627+0000 7f4c15481640 1 -- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4bf80a2f20 msgr2=0x7f4bf80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.627+0000 7f4c15481640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4c0002c240 tx=0x7f4c0002c7e0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 -- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c10102370 msgr2=0x7f4c1019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c10102370 0x7f4c1019f110 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800eb10 tx=0x7f4c0800c2d0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 -- 172.21.15.105:0/743419252 shutdown_connections 2024-03-26T17:49:26.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4bf80a2f20 0x7f4bf80a53e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c10102370 0x7f4c1019f110 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c101019b0 0x7f4c101980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 --2- 172.21.15.105:0/743419252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c101007c0 0x7f4c10197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 -- 172.21.15.105:0/743419252 >> 172.21.15.105:0/743419252 conn(0x7f4c100fbd50 msgr2=0x7f4c10068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 -- 172.21.15.105:0/743419252 shutdown_connections 2024-03-26T17:49:26.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.628+0000 7f4c15481640 1 -- 172.21.15.105:0/743419252 wait complete. 2024-03-26T17:49:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba1ffff640 1 -- 172.21.15.105:0/785901322 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5626276905f0 con 0x5626277381a0 2024-03-26T17:49:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba1ffff640 10 client.29306 handle_command_reply: tid=0 2024-03-26T17:49:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 client.29306 shutdown 2024-03-26T17:49:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 -- 172.21.15.105:0/785901322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 msgr2=0x562627728860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627728860 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba34000c00 tx=0x7fba34009260 comp rx=0 tx=0).stop 2024-03-26T17:49:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 -- 172.21.15.105:0/785901322 shutdown_connections 2024-03-26T17:49:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5626277381a0 0x56262773a590 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x562627728860 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 --2- 172.21.15.105:0/785901322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627728320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 -- 172.21.15.105:0/785901322 >> 172.21.15.105:0/785901322 conn(0x562627719b40 msgr2=0x562627718df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba1ffff640 0 client.29306 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 -- 172.21.15.105:0/785901322 shutdown_connections 2024-03-26T17:49:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 1 -- 172.21.15.105:0/785901322 wait complete. 2024-03-26T17:49:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.629+0000 7fba497e4740 20 client.29306 trim_cache size 0 max 16384 2024-03-26T17:49:26.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:49:26.646 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.646+0000 7fba497e4740 1 Processor -- start 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba497e4740 1 -- start start 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba497e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627720140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba497e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x56262772f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba497e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x562627718560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba497e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba401a6a40 con 0x56262771e5d0 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba497e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba401a8580 con 0x56262771efc0 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba497e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba401a7110 con 0x562627724130 2024-03-26T17:49:26.651 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627720140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627720140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45906/0 (socket says 172.21.15.105:45906) 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 -- 172.21.15.105:0/2291921967 learned_addr learned my addr 172.21.15.105:0/2291921967 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba448b7640 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x56262772f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x562627718560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 -- 172.21.15.105:0/2291921967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 msgr2=0x562627718560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x562627718560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 -- 172.21.15.105:0/2291921967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 msgr2=0x56262772f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x56262772f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.647+0000 7fba3ffff640 1 -- 172.21.15.105:0/2291921967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262771a8b0 con 0x56262771e5d0 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.648+0000 7fba3ffff640 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627720140 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fba2c004700 tx=0x7fba2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5116a47386b3fda9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.648+0000 7fba2a7fc640 1 -- 172.21.15.105:0/2291921967 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0216c0 con 0x56262771e5d0 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.648+0000 7fba2a7fc640 1 -- 172.21.15.105:0/2291921967 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba2c003e70 con 0x56262771e5d0 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.648+0000 7fba2a7fc640 1 -- 172.21.15.105:0/2291921967 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0047a0 con 0x56262771e5d0 2024-03-26T17:49:26.652 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 -- 172.21.15.105:0/2291921967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 msgr2=0x562627720140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627720140 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fba2c004700 tx=0x7fba2c004730 comp rx=0 tx=0).stop 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 -- 172.21.15.105:0/2291921967 shutdown_connections 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x562627718560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771efc0 0x56262772f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 --2- 172.21.15.105:0/2291921967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771e5d0 0x562627720140 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 -- 172.21.15.105:0/2291921967 >> 172.21.15.105:0/2291921967 conn(0x562627719b40 msgr2=0x56262771fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 -- 172.21.15.105:0/2291921967 shutdown_connections 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 -- 172.21.15.105:0/2291921967 wait complete. 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 -- start start 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 Processor -- start 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771e5d0 0x56262772cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771efc0 0x56262772d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.649+0000 7fba497e4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.653 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba497e4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba401a7eb0 con 0x56262771efc0 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba497e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba401a77e0 con 0x56262771e5d0 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba497e4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba401a4a20 con 0x562627724130 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba497e4740 10 client.0 _ll_register_callbacks cb 0x7fba2c009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba448b7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771efc0 0x56262772d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37830/0 (socket says 172.21.15.105:37830) 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 -- 172.21.15.105:0/3886298301 learned_addr learned my addr 172.21.15.105:0/3886298301 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba448b7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771efc0 0x56262772d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45914/0 (socket says 172.21.15.105:45914) 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771e5d0 0x56262772cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 -- 172.21.15.105:0/3886298301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771e5d0 msgr2=0x56262772cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771e5d0 0x56262772cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 -- 172.21.15.105:0/3886298301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771efc0 msgr2=0x56262772d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771efc0 0x56262772d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 -- 172.21.15.105:0/3886298301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262772eaf0 con 0x562627724130 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba3ffff640 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772e500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fba2c005770 tx=0x7fba2c005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=590369a47a0d18f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba1ffff640 1 -- 172.21.15.105:0/3886298301 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c032020 con 0x562627724130 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba497e4740 1 -- 172.21.15.105:0/3886298301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba2c003e70 con 0x562627724130 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.650+0000 7fba1ffff640 1 -- 172.21.15.105:0/3886298301 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fba2c011250 con 0x562627724130 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba1ffff640 1 -- 172.21.15.105:0/3886298301 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fba2c021780 con 0x562627724130 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba497e4740 10 client.29293 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba497e4740 1 -- 172.21.15.105:0/3886298301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56262779e580 con 0x562627724130 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba1ffff640 1 -- 172.21.15.105:0/3886298301 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba2c003e70 con 0x562627724130 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba1ffff640 1 -- 172.21.15.105:0/3886298301 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fba2c020040 con 0x562627724130 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba497e4740 10 client.29293 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba497e4740 10 client.29293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba497e4740 20 client.29293 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba497e4740 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56262779ee70 0x5626277a1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba497e4740 4 client.29293 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:49:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.651+0000 7fba497e4740 1 -- 172.21.15.105:0/3886298301 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56262768a940 con 0x56262779ee70 2024-03-26T17:49:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.652+0000 7fba448b7640 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56262779ee70 0x5626277a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.652+0000 7fba1ffff640 10 client.29293 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.652+0000 7fba448b7640 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56262779ee70 0x5626277a1290 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51eb5ab61d74b53c server_cookie=6661fb5e3bf5746f in_seq=0 out_seq=0 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba1ffff640 1 -- 172.21.15.105:0/3886298301 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56262768a940 con 0x56262779ee70 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba1ffff640 10 client.29293 handle_command_reply: tid=0 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 client.29293 shutdown 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 -- 172.21.15.105:0/3886298301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 msgr2=0x56262772e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772e500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fba2c005770 tx=0x7fba2c005970 comp rx=0 tx=0).stop 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 -- 172.21.15.105:0/3886298301 shutdown_connections 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56262779ee70 0x5626277a1290 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562627724130 0x56262772e500 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56262771efc0 0x56262772d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 --2- 172.21.15.105:0/3886298301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56262771e5d0 0x56262772cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 -- 172.21.15.105:0/3886298301 >> 172.21.15.105:0/3886298301 conn(0x562627719b40 msgr2=0x56262771cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba1ffff640 0 client.29293 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 -- 172.21.15.105:0/3886298301 shutdown_connections 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 1 -- 172.21.15.105:0/3886298301 wait complete. 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.697+0000 7fba497e4740 20 client.29293 trim_cache size 0 max 16384 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:49:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:49:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 -- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba300a2fc0 msgr2=0x7fba300a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba300a2fc0 0x7fba300a5480 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fba38096c80 tx=0x7fba380984c0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 -- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba40107640 msgr2=0x7fba4019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba40107640 0x7fba4019f0a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fba38009fd0 tx=0x7fba3800ef10 comp rx=0 tx=0).stop 2024-03-26T17:49:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 -- 172.21.15.105:0/1331718851 shutdown_connections 2024-03-26T17:49:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba300a2fc0 0x7fba300a5480 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba40107640 0x7fba4019f0a0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba40104d10 0x7fba40198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 --2- 172.21.15.105:0/1331718851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba401023e0 0x7fba40197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 -- 172.21.15.105:0/1331718851 >> 172.21.15.105:0/1331718851 conn(0x7fba400fbd70 msgr2=0x7fba40102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 -- 172.21.15.105:0/1331718851 shutdown_connections 2024-03-26T17:49:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:26.698+0000 7fba46b1b640 1 -- 172.21.15.105:0/1331718851 wait complete. 2024-03-26T17:49:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 -- 172.21.15.105:0/1790197502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02701018f0 msgr2=0x7f0270101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 --2- 172.21.15.105:0/1790197502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02701018f0 0x7f0270101d10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f026c009900 tx=0x7f026c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 -- 172.21.15.105:0/1790197502 shutdown_connections 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 --2- 172.21.15.105:0/1790197502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f02701034a0 0x7f0270109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 --2- 172.21.15.105:0/1790197502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0270102ae0 0x7f0270102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 --2- 172.21.15.105:0/1790197502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02701018f0 0x7f0270101d10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 -- 172.21.15.105:0/1790197502 >> 172.21.15.105:0/1790197502 conn(0x7f02700fce60 msgr2=0x7f02700ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 -- 172.21.15.105:0/1790197502 shutdown_connections 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.792+0000 7f0278353640 1 -- 172.21.15.105:0/1790197502 wait complete. 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f0278353640 1 Processor -- start 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f0278353640 1 -- start start 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f0278353640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02701018f0 0x7f0270075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f0278353640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0270102ae0 0x7f0270073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f0278353640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02701034a0 0x7f02700741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f0278353640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f027010d3f0 con 0x7f02701018f0 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f0278353640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f027010d270 con 0x7f0270102ae0 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f0278353640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f027010d570 con 0x7f02701034a0 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f02760ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02701018f0 0x7f0270075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f02758ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0270102ae0 0x7f0270073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f02760ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02701018f0 0x7f0270075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45922/0 (socket says 172.21.15.105:45922) 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f02760ef640 1 -- 172.21.15.105:0/52909007 learned_addr learned my addr 172.21.15.105:0/52909007 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f02768f0640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02701034a0 0x7f02700741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f02758ee640 1 -- 172.21.15.105:0/52909007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02701034a0 msgr2=0x7f02700741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.793+0000 7f02758ee640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02701034a0 0x7f02700741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f02758ee640 1 -- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02701018f0 msgr2=0x7f0270075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f02758ee640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02701018f0 0x7f0270075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f02758ee640 1 -- 172.21.15.105:0/52909007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0270074a80 con 0x7f0270102ae0 2024-03-26T17:49:26.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f02760ef640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02701018f0 0x7f0270075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:26.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f02758ee640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0270102ae0 0x7f0270073cb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f026000e9f0 tx=0x7f026000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f02677fe640 1 -- 172.21.15.105:0/52909007 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026000ccc0 con 0x7f0270102ae0 2024-03-26T17:49:26.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f0278353640 1 -- 172.21.15.105:0/52909007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02701a1ca0 con 0x7f0270102ae0 2024-03-26T17:49:26.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f02677fe640 1 -- 172.21.15.105:0/52909007 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02600108e0 con 0x7f0270102ae0 2024-03-26T17:49:26.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f02677fe640 1 -- 172.21.15.105:0/52909007 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260018880 con 0x7f0270102ae0 2024-03-26T17:49:26.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.794+0000 7f0278353640 1 -- 172.21.15.105:0/52909007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02701a2190 con 0x7f0270102ae0 2024-03-26T17:49:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f027b018740 1 Processor -- start 2024-03-26T17:49:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f027b018740 1 -- start start 2024-03-26T17:49:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f027b018740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e668f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f027b018740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e668fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f027b018740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e669008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f027b018740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562e6685d980 con 0x562e668f92e0 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f027b018740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562e66805be0 con 0x562e668fe450 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f027b018740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562e66859930 con 0x562e668f88f0 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.797+0000 7f02768f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e668f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02760ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e668fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02768f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e668f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37862/0 (socket says 172.21.15.105:37862) 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02768f0640 1 -- 172.21.15.105:0/304059106 learned_addr learned my addr 172.21.15.105:0/304059106 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02677fe640 1 -- 172.21.15.105:0/52909007 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0260010a90 con 0x7f0270102ae0 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02760ef640 1 -- 172.21.15.105:0/304059106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 msgr2=0x562e668f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02760ef640 1 --2- 172.21.15.105:0/304059106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e668f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02760ef640 1 -- 172.21.15.105:0/304059106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 msgr2=0x562e669008e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02760ef640 1 --2- 172.21.15.105:0/304059106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e669008e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02760ef640 1 -- 172.21.15.105:0/304059106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6663b7d0 con 0x562e668f92e0 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02760ef640 1 --2- 172.21.15.105:0/304059106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e668fdde0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f026c006010 tx=0x7f026c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86459dafefd2fa26 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f0264bfe640 1 -- 172.21.15.105:0/304059106 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c045020 con 0x562e668f92e0 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f0264bfe640 1 -- 172.21.15.105:0/304059106 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c005040 con 0x562e668f92e0 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f0264bfe640 1 -- 172.21.15.105:0/304059106 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c0049f0 con 0x562e668f92e0 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.798+0000 7f02677fe640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f025c0a2f20 0x7f025c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f02768f0640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f025c0a2f20 0x7f025c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f02677fe640 1 -- 172.21.15.105:0/52909007 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f02600cf720 con 0x7f0270102ae0 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 -- 172.21.15.105:0/304059106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 msgr2=0x562e668fdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f02768f0640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f025c0a2f20 0x7f025c0a53e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0268009fd0 tx=0x7f02680096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 --2- 172.21.15.105:0/304059106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e668fdde0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f026c006010 tx=0x7f026c005e00 comp rx=0 tx=0).stop 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 -- 172.21.15.105:0/304059106 shutdown_connections 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 --2- 172.21.15.105:0/304059106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e669008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 --2- 172.21.15.105:0/304059106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e668fdde0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 --2- 172.21.15.105:0/304059106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e668f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 -- 172.21.15.105:0/304059106 >> 172.21.15.105:0/304059106 conn(0x562e668f3e60 msgr2=0x562e668f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 -- 172.21.15.105:0/304059106 shutdown_connections 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.799+0000 7f027b018740 1 -- 172.21.15.105:0/304059106 wait complete. 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 1 -- start start 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 1 Processor -- start 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f88f0 0x562e66902ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e66903020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e6690a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562e669013a0 con 0x562e668f88f0 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562e66819160 con 0x562e668fe450 2024-03-26T17:49:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562e66901520 con 0x562e668f92e0 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f027b018740 10 client.0 _ll_register_callbacks cb 0x562e66855720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02760ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f88f0 0x562e66902ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02768f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e6690a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02758ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e66903020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02758ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e66903020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37870/0 (socket says 172.21.15.105:37870) 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02758ee640 1 -- 172.21.15.105:0/524529579 learned_addr learned my addr 172.21.15.105:0/524529579 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02758ee640 1 -- 172.21.15.105:0/524529579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 msgr2=0x562e6690a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02758ee640 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e6690a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02758ee640 1 -- 172.21.15.105:0/524529579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f88f0 msgr2=0x562e66902ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02758ee640 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f88f0 0x562e66902ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.800+0000 7f02758ee640 1 -- 172.21.15.105:0/524529579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e665713f0 con 0x562e668f92e0 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f02758ee640 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e66903020 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f026009be50 tx=0x7f0260010e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa219f8e8261dd66 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f0259ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f0259ffb640 1 -- 172.21.15.105:0/524529579 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600a4590 con 0x562e668f92e0 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f0259ffb640 1 -- 172.21.15.105:0/524529579 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f02600bab80 con 0x562e668f92e0 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 1 -- 172.21.15.105:0/524529579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026c005040 con 0x562e668f92e0 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f0259ffb640 1 -- 172.21.15.105:0/524529579 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f02600a4820 con 0x562e668f92e0 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 10 client.29305 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 1 -- 172.21.15.105:0/524529579 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562e665310f0 con 0x562e668f92e0 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f0259ffb640 1 -- 172.21.15.105:0/524529579 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c005040 con 0x562e668f92e0 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f0259ffb640 1 -- 172.21.15.105:0/524529579 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0260097c10 con 0x562e668f92e0 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 10 client.29305 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 10 client.29305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 20 client.29305 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e6690e990 0x562e66910df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 4 client.29305 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.801+0000 7f027b018740 1 -- 172.21.15.105:0/524529579 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e6686aec0 con 0x562e6690e990 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.802+0000 7f02760ef640 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e6690e990 0x562e66910df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.802+0000 7f0259ffb640 10 client.29305 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.802+0000 7f02760ef640 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e6690e990 0x562e66910df0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60497a7c6eec86da server_cookie=4d4ad1a9a0ebad60 in_seq=0 out_seq=0 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f0259ffb640 1 -- 172.21.15.105:0/524529579 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562e6686aec0 con 0x562e6690e990 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f0259ffb640 10 client.29305 handle_command_reply: tid=0 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 client.29305 shutdown 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 -- 172.21.15.105:0/524529579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 msgr2=0x562e66903020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e66903020 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f026009be50 tx=0x7f0260010e80 comp rx=0 tx=0).stop 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 -- 172.21.15.105:0/524529579 shutdown_connections 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e6690e990 0x562e66910df0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e6690a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e66903020 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 --2- 172.21.15.105:0/524529579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f88f0 0x562e66902ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 -- 172.21.15.105:0/524529579 >> 172.21.15.105:0/524529579 conn(0x562e668f3e60 msgr2=0x562e668f3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f0259ffb640 0 client.29305 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 -- 172.21.15.105:0/524529579 shutdown_connections 2024-03-26T17:49:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.804+0000 7f027b018740 1 -- 172.21.15.105:0/524529579 wait complete. 2024-03-26T17:49:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.805+0000 7f027b018740 20 client.29305 trim_cache size 0 max 16384 2024-03-26T17:49:26.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f027b018740 1 Processor -- start 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f027b018740 1 -- start start 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f027b018740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668f88f0 0x562e668f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f027b018740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e6690a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f027b018740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668fe450 0x562e668f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f027b018740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02701aafa0 con 0x562e668fe450 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f027b018740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f02701acae0 con 0x562e668f88f0 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f027b018740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f02701ab670 con 0x562e668f92e0 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f02768f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668fe450 0x562e668f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f02768f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668fe450 0x562e668f2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45948/0 (socket says 172.21.15.105:45948) 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.824+0000 7f02768f0640 1 -- 172.21.15.105:0/2163564451 learned_addr learned my addr 172.21.15.105:0/2163564451 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f02760ef640 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668f88f0 0x562e668f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f02758ee640 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e6690a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f02768f0640 1 -- 172.21.15.105:0/2163564451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 msgr2=0x562e6690a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f02768f0640 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e6690a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f02768f0640 1 -- 172.21.15.105:0/2163564451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668f88f0 msgr2=0x562e668f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f02768f0640 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668f88f0 0x562e668f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f02768f0640 1 -- 172.21.15.105:0/2163564451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6690d310 con 0x562e668fe450 2024-03-26T17:49:26.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f02768f0640 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668fe450 0x562e668f2d30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f026801e7a0 tx=0x7f0268019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219cbe20ec9a3cab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f025bfff640 1 -- 172.21.15.105:0/2163564451 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268011a80 con 0x562e668fe450 2024-03-26T17:49:26.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f025bfff640 1 -- 172.21.15.105:0/2163564451 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0268011c20 con 0x562e668fe450 2024-03-26T17:49:26.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.825+0000 7f025bfff640 1 -- 172.21.15.105:0/2163564451 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268010690 con 0x562e668fe450 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 -- 172.21.15.105:0/2163564451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668fe450 msgr2=0x562e668f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668fe450 0x562e668f2d30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f026801e7a0 tx=0x7f0268019000 comp rx=0 tx=0).stop 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 -- 172.21.15.105:0/2163564451 shutdown_connections 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668fe450 0x562e668f2d30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f92e0 0x562e6690a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 --2- 172.21.15.105:0/2163564451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668f88f0 0x562e668f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 -- 172.21.15.105:0/2163564451 >> 172.21.15.105:0/2163564451 conn(0x562e668f3e60 msgr2=0x562e66902ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 -- 172.21.15.105:0/2163564451 shutdown_connections 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 -- 172.21.15.105:0/2163564451 wait complete. 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 -- start start 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 Processor -- start 2024-03-26T17:49:26.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e669079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.826+0000 7f027b018740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e66907f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f027b018740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e66908f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f027b018740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02701ac410 con 0x562e668f92e0 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f027b018740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f02701abd40 con 0x562e668fe450 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f027b018740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f02701a8f80 con 0x562e668f88f0 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e66907f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02760ef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e669079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e66907f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45950/0 (socket says 172.21.15.105:45950) 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f027b018740 10 client.0 _ll_register_callbacks cb 0x7f026000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 -- 172.21.15.105:0/136805959 learned_addr learned my addr 172.21.15.105:0/136805959 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02768f0640 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e66908f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 -- 172.21.15.105:0/136805959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 msgr2=0x562e669079f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e669079f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 -- 172.21.15.105:0/136805959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 msgr2=0x562e66908f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:26.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e66908f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 -- 172.21.15.105:0/136805959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e66909530 con 0x562e668f92e0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02760ef640 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e669079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f02758ee640 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e66907f30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f0260097fd0 tx=0x7f026009a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83965be9faaf05f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f0259ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f0259ffb640 1 -- 172.21.15.105:0/136805959 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026007e060 con 0x562e668f92e0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f0259ffb640 1 -- 172.21.15.105:0/136805959 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f02600a4390 con 0x562e668f92e0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f027b018740 1 -- 172.21.15.105:0/136805959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0268011c20 con 0x562e668f92e0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.827+0000 7f0259ffb640 1 -- 172.21.15.105:0/136805959 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f026009b820 con 0x562e668f92e0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f027b018740 10 client.19521 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f027b018740 1 -- 172.21.15.105:0/136805959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562e66975040 con 0x562e668f92e0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f0259ffb640 1 -- 172.21.15.105:0/136805959 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0268011c20 con 0x562e668f92e0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f0259ffb640 1 -- 172.21.15.105:0/136805959 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f02600bac10 con 0x562e668f92e0 2024-03-26T17:49:26.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f027b018740 10 client.19521 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f027b018740 10 client.19521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f027b018740 20 client.19521 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f027b018740 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e669759c0 0x562e66977de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f027b018740 4 client.19521 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f027b018740 1 -- 172.21.15.105:0/136805959 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e66865210 con 0x562e669759c0 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.828+0000 7f02760ef640 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e669759c0 0x562e66977de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.829+0000 7f02760ef640 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e669759c0 0x562e66977de0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f73415ed9c4aaedb server_cookie=d4e15667585d5b45 in_seq=0 out_seq=0 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.829+0000 7f0259ffb640 10 client.19521 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f0259ffb640 1 -- 172.21.15.105:0/136805959 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562e66865210 con 0x562e669759c0 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f0259ffb640 10 client.19521 handle_command_reply: tid=0 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 client.19521 shutdown 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 -- 172.21.15.105:0/136805959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 msgr2=0x562e66907f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e66907f30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f0260097fd0 tx=0x7f026009a3d0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 -- 172.21.15.105:0/136805959 shutdown_connections 2024-03-26T17:49:26.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562e669759c0 0x562e66977de0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562e668fe450 0x562e66908f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562e668f92e0 0x562e66907f30 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 --2- 172.21.15.105:0/136805959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562e668f88f0 0x562e669079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 -- 172.21.15.105:0/136805959 >> 172.21.15.105:0/136805959 conn(0x562e668f3e60 msgr2=0x562e668f70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f0259ffb640 0 client.19521 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 -- 172.21.15.105:0/136805959 shutdown_connections 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 1 -- 172.21.15.105:0/136805959 wait complete. 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.830+0000 7f027b018740 20 client.19521 trim_cache size 0 max 16384 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:49:26.828 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:49:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 -- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f025c0a2f20 msgr2=0x7f025c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f025c0a2f20 0x7f025c0a53e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0268009fd0 tx=0x7f02680096a0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 -- 172.21.15.105:0/52909007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0270102ae0 msgr2=0x7f0270073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0270102ae0 0x7f0270073cb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f026000e9f0 tx=0x7f026000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 -- 172.21.15.105:0/52909007 shutdown_connections 2024-03-26T17:49:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f025c0a2f20 0x7f025c0a53e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02701034a0 0x7f02700741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0270102ae0 0x7f0270073cb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 --2- 172.21.15.105:0/52909007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02701018f0 0x7f0270075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 -- 172.21.15.105:0/52909007 >> 172.21.15.105:0/52909007 conn(0x7f02700fce60 msgr2=0x7f0270105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 -- 172.21.15.105:0/52909007 shutdown_connections 2024-03-26T17:49:26.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:26.831+0000 7f0278353640 1 -- 172.21.15.105:0/52909007 wait complete. 2024-03-26T17:49:26.836 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:49:26.836 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:49:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 -- 172.21.15.105:0/2141712489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c102420 msgr2=0x7f4b5c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 --2- 172.21.15.105:0/2141712489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c102420 0x7f4b5c104810 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009900 tx=0x7f4b4c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 -- 172.21.15.105:0/2141712489 shutdown_connections 2024-03-26T17:49:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 --2- 172.21.15.105:0/2141712489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b5c107680 0x7f4b5c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 --2- 172.21.15.105:0/2141712489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b5c104d50 0x7f4b5c107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 --2- 172.21.15.105:0/2141712489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c102420 0x7f4b5c104810 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 -- 172.21.15.105:0/2141712489 >> 172.21.15.105:0/2141712489 conn(0x7f4b5c0fbd50 msgr2=0x7f4b5c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:26.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 -- 172.21.15.105:0/2141712489 shutdown_connections 2024-03-26T17:49:26.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 -- 172.21.15.105:0/2141712489 wait complete. 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.002+0000 7f4b611c6640 1 Processor -- start 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b611c6640 1 -- start start 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b611c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b5c102420 0x7f4b5c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b611c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c104d50 0x7f4b5c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b611c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b5c107680 0x7f4b5c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b611c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10d490 con 0x7f4b5c104d50 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b611c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10d310 con 0x7f4b5c107680 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b611c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c10d610 con 0x7f4b5c102420 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c104d50 0x7f4b5c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5ad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b5c102420 0x7f4b5c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c104d50 0x7f4b5c198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45964/0 (socket says 172.21.15.105:45964) 2024-03-26T17:49:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5a59c640 1 -- 172.21.15.105:0/3515279414 learned_addr learned my addr 172.21.15.105:0/3515279414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5b59e640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b5c107680 0x7f4b5c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5ad9d640 1 -- 172.21.15.105:0/3515279414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b5c107680 msgr2=0x7f4b5c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5ad9d640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b5c107680 0x7f4b5c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5ad9d640 1 -- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c104d50 msgr2=0x7f4b5c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5ad9d640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c104d50 0x7f4b5c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5ad9d640 1 -- 172.21.15.105:0/3515279414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c19f750 con 0x7f4b5c102420 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5a59c640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c104d50 0x7f4b5c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.003+0000 7f4b5ad9d640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b5c102420 0x7f4b5c197b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c02c170 tx=0x7f4b4c005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.004+0000 7f4b43fff640 1 -- 172.21.15.105:0/3515279414 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c043020 con 0x7f4b5c102420 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.004+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c19f9e0 con 0x7f4b5c102420 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.004+0000 7f4b43fff640 1 -- 172.21.15.105:0/3515279414 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b4c0043d0 con 0x7f4b5c102420 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.004+0000 7f4b43fff640 1 -- 172.21.15.105:0/3515279414 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c003b50 con 0x7f4b5c102420 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.004+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c19ff50 con 0x7f4b5c102420 2024-03-26T17:49:26.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.005+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c0686b0 con 0x7f4b5c102420 2024-03-26T17:49:26.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.007+0000 7f4b43fff640 1 -- 172.21.15.105:0/3515279414 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4b4c046020 con 0x7f4b5c102420 2024-03-26T17:49:26.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.007+0000 7f4b43fff640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b440a2f20 0x7f4b440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:26.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.007+0000 7f4b5a59c640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b440a2f20 0x7f4b440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:26.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.007+0000 7f4b43fff640 1 -- 172.21.15.105:0/3515279414 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4b4c0f18b0 con 0x7f4b5c102420 2024-03-26T17:49:27.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.008+0000 7f4b5a59c640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b440a2f20 0x7f4b440a53e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c199040 tx=0x7f4b50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.010+0000 7f4b43fff640 1 -- 172.21.15.105:0/3515279414 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4b4c04d020 con 0x7f4b5c102420 2024-03-26T17:49:27.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 357251 3.14 MB/sec execute 490 sec latency 22.092 ms 2024-03-26T17:49:27.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:26 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:49:27.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:27.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:26 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:49:27.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:27.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:26 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:49:27.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:26 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:49:27.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:27.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:26 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:49:27.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:27.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:26 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:49:27.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:26 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:49:27.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:27.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:26 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:49:27.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:27.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:26 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:49:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.263+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b5c02d2e0 con 0x7f4b5c102420 2024-03-26T17:49:27.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.263+0000 7f4b43fff640 1 -- 172.21.15.105:0/3515279414 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11941 (secure 0 0 0) 0x7f4b5c02d2e0 con 0x7f4b5c102420 2024-03-26T17:49:27.256 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:49:27.256 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:49:14.826136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24482,14628,24487,24466]},"id":1}]} 2024-03-26T17:49:27.256 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 32 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b440a2f20 msgr2=0x7f4b440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b440a2f20 0x7f4b440a53e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c199040 tx=0x7f4b50005f90 comp rx=0 tx=0).stop 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b5c102420 msgr2=0x7f4b5c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b5c102420 0x7f4b5c197b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c02c170 tx=0x7f4b4c005e60 comp rx=0 tx=0).stop 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 shutdown_connections 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b440a2f20 0x7f4b440a53e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b5c107680 0x7f4b5c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c104d50 0x7f4b5c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 --2- 172.21.15.105:0/3515279414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b5c102420 0x7f4b5c197b30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 >> 172.21.15.105:0/3515279414 conn(0x7f4b5c0fbd50 msgr2=0x7f4b5c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 shutdown_connections 2024-03-26T17:49:27.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.266+0000 7f4b611c6640 1 -- 172.21.15.105:0/3515279414 wait complete. 2024-03-26T17:49:27.270 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:49:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/295070173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc481075b0 msgr2=0x7fdc48109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/295070173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc481075b0 0x7fdc48109a00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009900 tx=0x7fdc4002bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/295070173 shutdown_connections 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/295070173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc481075b0 0x7fdc48109a00 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/295070173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc48104c80 0x7fdc48107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/295070173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48102350 0x7fdc48104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/295070173 >> 172.21.15.105:0/295070173 conn(0x7fdc480fbd60 msgr2=0x7fdc480fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/295070173 shutdown_connections 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.385+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/295070173 wait complete. 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc4e0c6640 1 Processor -- start 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc4e0c6640 1 -- start start 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc4e0c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc48102350 0x7fdc48197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc4e0c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc48104c80 0x7fdc48198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc4e0c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc481075b0 0x7fdc4819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc4e0c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4810d470 con 0x7fdc48104c80 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc4e0c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4810d2f0 con 0x7fdc48102350 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc4e0c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4810d5f0 con 0x7fdc481075b0 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc46ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc48104c80 0x7fdc48198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc46ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc48104c80 0x7fdc48198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45994/0 (socket says 172.21.15.105:45994) 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.386+0000 7fdc46ffd640 1 -- 172.21.15.105:0/3683659065 learned_addr learned my addr 172.21.15.105:0/3683659065 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc46ffd640 1 -- 172.21.15.105:0/3683659065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc481075b0 msgr2=0x7fdc4819f070 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc481075b0 0x7fdc4819f070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc46ffd640 1 -- 172.21.15.105:0/3683659065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc48102350 msgr2=0x7fdc48197ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc48102350 0x7fdc48197ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc46ffd640 1 -- 172.21.15.105:0/3683659065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4819f6e0 con 0x7fdc48104c80 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc48104c80 0x7fdc48198000 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000e9c0 tx=0x7fdc3000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc44ff9640 1 -- 172.21.15.105:0/3683659065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3000cd80 con 0x7fdc48104c80 2024-03-26T17:49:27.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc44ff9640 1 -- 172.21.15.105:0/3683659065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc300054d0 con 0x7fdc48104c80 2024-03-26T17:49:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc44ff9640 1 -- 172.21.15.105:0/3683659065 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30010430 con 0x7fdc48104c80 2024-03-26T17:49:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/3683659065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4819f9d0 con 0x7fdc48104c80 2024-03-26T17:49:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.387+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/3683659065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4819ff20 con 0x7fdc48104c80 2024-03-26T17:49:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.389+0000 7fdc44ff9640 1 -- 172.21.15.105:0/3683659065 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdc30004fd0 con 0x7fdc48104c80 2024-03-26T17:49:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.390+0000 7fdc44ff9640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdc340a2f20 0x7fdc340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.390+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdc340a2f20 0x7fdc340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.390+0000 7fdc44ff9640 1 -- 172.21.15.105:0/3683659065 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fdc300cf9b0 con 0x7fdc48104c80 2024-03-26T17:49:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.390+0000 7fdc50d90740 1 Processor -- start 2024-03-26T17:49:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.390+0000 7fdc50d90740 1 -- start start 2024-03-26T17:49:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.390+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb084e40 0x562ccb085260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.390+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc50d90740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562ccafe5970 con 0x562ccb08a9a0 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc50d90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ccaf8dd70 con 0x562ccb084e40 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc50d90740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ccafe1920 con 0x562ccb085830 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdc340a2f20 0x7fdc340a53e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc38005fd0 tx=0x7fdc38009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46002/0 (socket says 172.21.15.105:46002) 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 -- 172.21.15.105:0/3086584945 learned_addr learned my addr 172.21.15.105:0/3086584945 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb084e40 0x562ccb085260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 -- 172.21.15.105:0/3086584945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 msgr2=0x562ccb08a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 -- 172.21.15.105:0/3086584945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb084e40 msgr2=0x562ccb085260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb084e40 0x562ccb085260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 -- 172.21.15.105:0/3086584945 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ccacf93f0 con 0x562ccb08a9a0 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc47fff640 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08ce30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005ab0 tx=0x7fdc40005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e7ba50bd540b0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc3dffb640 1 -- 172.21.15.105:0/3086584945 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40042020 con 0x562ccb08a9a0 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc3dffb640 1 -- 172.21.15.105:0/3086584945 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc40009520 con 0x562ccb08a9a0 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.391+0000 7fdc3dffb640 1 -- 172.21.15.105:0/3086584945 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40003c00 con 0x562ccb08a9a0 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 -- 172.21.15.105:0/3086584945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 msgr2=0x562ccb08ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08ce30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005ab0 tx=0x7fdc40005760 comp rx=0 tx=0).stop 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 -- 172.21.15.105:0/3086584945 shutdown_connections 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08ce30 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3086584945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb084e40 0x562ccb085260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 -- 172.21.15.105:0/3086584945 >> 172.21.15.105:0/3086584945 conn(0x562ccb0803b0 msgr2=0x562ccb0807b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 -- 172.21.15.105:0/3086584945 shutdown_connections 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 -- 172.21.15.105:0/3086584945 wait complete. 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.392+0000 7fdc50d90740 1 -- start start 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc50d90740 1 Processor -- start 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb08f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb08a9a0 0x562ccb0967d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc50d90740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562ccb08d8f0 con 0x562ccb084e40 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc50d90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ccafa1310 con 0x562ccb08a9a0 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc50d90740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ccb08da70 con 0x562ccb085830 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc50d90740 10 client.0 _ll_register_callbacks cb 0x562ccafdcc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb08f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb08f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46016/0 (socket says 172.21.15.105:46016) 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc477fe640 1 -- 172.21.15.105:0/3874535820 learned_addr learned my addr 172.21.15.105:0/3874535820 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc47fff640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb08a9a0 0x562ccb0967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc477fe640 1 -- 172.21.15.105:0/3874535820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 msgr2=0x562ccb08f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc477fe640 1 -- 172.21.15.105:0/3874535820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb08a9a0 msgr2=0x562ccb0967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb08a9a0 0x562ccb0967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc477fe640 1 -- 172.21.15.105:0/3874535820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ccadc37d0 con 0x562ccb084e40 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc47fff640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb08a9a0 0x562ccb0967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.393+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb08f250 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdc380106b0 tx=0x7fdc38004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132e524c3f8144f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:27.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3874535820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800f840 con 0x562ccb084e40 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3874535820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fdc3800f9e0 con 0x562ccb084e40 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 1 -- 172.21.15.105:0/3874535820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc40009520 con 0x562ccb084e40 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3874535820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdc3800fbe0 con 0x562ccb084e40 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 10 client.19539 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 1 -- 172.21.15.105:0/3874535820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ccb099c10 con 0x562ccb084e40 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3874535820 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc40009520 con 0x562ccb084e40 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3874535820 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fdc38026400 con 0x562ccb084e40 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 10 client.19539 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 10 client.19539 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 20 client.19539 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562ccb09b3d0 0x562ccb09d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 4 client.19539 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:27.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.394+0000 7fdc50d90740 1 -- 172.21.15.105:0/3874535820 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ccaff2eb0 con 0x562ccb09b3d0 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.395+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562ccb09b3d0 0x562ccb09d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.395+0000 7fdc26ffd640 10 client.19539 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.395+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562ccb09b3d0 0x562ccb09d890 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84232b543158d55d server_cookie=15fbc61db128f6e0 in_seq=0 out_seq=0 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.395+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3874535820 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ccaff2eb0 con 0x562ccb09b3d0 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc26ffd640 10 client.19539 handle_command_reply: tid=0 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 client.19539 shutdown 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 -- 172.21.15.105:0/3874535820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 msgr2=0x562ccb08f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb08f250 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fdc380106b0 tx=0x7fdc38004ab0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 -- 172.21.15.105:0/3874535820 shutdown_connections 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562ccb09b3d0 0x562ccb09d890 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb08a9a0 0x562ccb0967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb085830 0x562ccb08f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc26ffd640 0 client.19539 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3874535820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb08f250 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 -- 172.21.15.105:0/3874535820 >> 172.21.15.105:0/3874535820 conn(0x562ccb0803b0 msgr2=0x562ccb07b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 -- 172.21.15.105:0/3874535820 shutdown_connections 2024-03-26T17:49:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 1 -- 172.21.15.105:0/3874535820 wait complete. 2024-03-26T17:49:27.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.396+0000 7fdc50d90740 20 client.19539 trim_cache size 0 max 16384 2024-03-26T17:49:27.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc50d90740 1 Processor -- start 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc50d90740 1 -- start start 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb084e40 0x562ccb08ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb096790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc50d90740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481a6ba0 con 0x562ccb08a9a0 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc50d90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481a86e0 con 0x562ccb085830 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc50d90740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481a7270 con 0x562ccb084e40 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.412+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc46ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb096790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc477fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb084e40 0x562ccb08ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc46ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb096790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49660/0 (socket says 172.21.15.105:49660) 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46020/0 (socket says 172.21.15.105:46020) 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc46ffd640 1 -- 172.21.15.105:0/712678933 learned_addr learned my addr 172.21.15.105:0/712678933 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc47fff640 1 -- 172.21.15.105:0/712678933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb084e40 msgr2=0x562ccb08ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc47fff640 1 --2- 172.21.15.105:0/712678933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb084e40 0x562ccb08ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc47fff640 1 -- 172.21.15.105:0/712678933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 msgr2=0x562ccb096790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc47fff640 1 --2- 172.21.15.105:0/712678933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb096790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc47fff640 1 -- 172.21.15.105:0/712678933 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ccb085c10 con 0x562ccb08a9a0 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/712678933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb096790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc47fff640 1 --2- 172.21.15.105:0/712678933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08f3a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc4002c170 tx=0x7fdc40005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b3e47875ec165b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc3d7fa640 1 -- 172.21.15.105:0/712678933 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40042030 con 0x562ccb08a9a0 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc3d7fa640 1 -- 172.21.15.105:0/712678933 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc40009520 con 0x562ccb08a9a0 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.413+0000 7fdc3d7fa640 1 -- 172.21.15.105:0/712678933 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40034440 con 0x562ccb08a9a0 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 -- 172.21.15.105:0/712678933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 msgr2=0x562ccb08f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 --2- 172.21.15.105:0/712678933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08f3a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc4002c170 tx=0x7fdc40005e50 comp rx=0 tx=0).stop 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 -- 172.21.15.105:0/712678933 shutdown_connections 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 --2- 172.21.15.105:0/712678933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb08a9a0 0x562ccb08f3a0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 --2- 172.21.15.105:0/712678933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb096790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 --2- 172.21.15.105:0/712678933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb084e40 0x562ccb08ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 -- 172.21.15.105:0/712678933 >> 172.21.15.105:0/712678933 conn(0x562ccb0803b0 msgr2=0x562ccb085f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 -- 172.21.15.105:0/712678933 shutdown_connections 2024-03-26T17:49:27.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 -- 172.21.15.105:0/712678933 wait complete. 2024-03-26T17:49:27.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.414+0000 7fdc50d90740 1 -- start start 2024-03-26T17:49:27.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc50d90740 1 Processor -- start 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb093970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb093eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc50d90740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb08a9a0 0x562ccb094ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc50d90740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481a8010 con 0x562ccb084e40 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc50d90740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481a7940 con 0x562ccb085830 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc50d90740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481a4b80 con 0x562ccb08a9a0 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc50d90740 10 client.0 _ll_register_callbacks cb 0x7fdc38014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb093970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb093970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46036/0 (socket says 172.21.15.105:46036) 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc477fe640 1 -- 172.21.15.105:0/3177028014 learned_addr learned my addr 172.21.15.105:0/3177028014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc47fff640 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb08a9a0 0x562ccb094ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb093eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc477fe640 1 -- 172.21.15.105:0/3177028014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb08a9a0 msgr2=0x562ccb094ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb08a9a0 0x562ccb094ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc477fe640 1 -- 172.21.15.105:0/3177028014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 msgr2=0x562ccb093eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb093eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc477fe640 1 -- 172.21.15.105:0/3177028014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ccb0954b0 con 0x562ccb084e40 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.415+0000 7fdc47fff640 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb08a9a0 0x562ccb094ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc477fe640 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb093970 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fdc38026870 tx=0x7fdc38009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05a3d71678a3d84 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3177028014 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38035040 con 0x562ccb084e40 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3177028014 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fdc38026ae0 con 0x562ccb084e40 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc50d90740 1 -- 172.21.15.105:0/3177028014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc40009520 con 0x562ccb084e40 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3177028014 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdc38026ce0 con 0x562ccb084e40 2024-03-26T17:49:27.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc50d90740 10 client.19545 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc50d90740 1 -- 172.21.15.105:0/3177028014 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ccb096820 con 0x562ccb084e40 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3177028014 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc40009520 con 0x562ccb084e40 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.416+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3177028014 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fdc3801eb80 con 0x562ccb084e40 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc50d90740 10 client.19545 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc50d90740 10 client.19545 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc50d90740 20 client.19545 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562ccb101cb0 0x562ccb1040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc50d90740 4 client.19545 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc50d90740 1 -- 172.21.15.105:0/3177028014 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ccafed200 con 0x562ccb101cb0 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562ccb101cb0 0x562ccb1040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc26ffd640 10 client.19545 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc46ffd640 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562ccb101cb0 0x562ccb1040d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acfa950f3ab881f5 server_cookie=2bf14e152eb60171 in_seq=0 out_seq=0 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc26ffd640 1 -- 172.21.15.105:0/3177028014 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ccafed200 con 0x562ccb101cb0 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc26ffd640 10 client.19545 handle_command_reply: tid=0 2024-03-26T17:49:27.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.417+0000 7fdc50d90740 1 client.19545 shutdown 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 -- 172.21.15.105:0/3177028014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 msgr2=0x562ccb093970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb093970 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fdc38026870 tx=0x7fdc38009790 comp rx=0 tx=0).stop 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 -- 172.21.15.105:0/3177028014 shutdown_connections 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562ccb101cb0 0x562ccb1040d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ccb08a9a0 0x562ccb094ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ccb085830 0x562ccb093eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc26ffd640 0 client.19545 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 --2- 172.21.15.105:0/3177028014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ccb084e40 0x562ccb093970 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 -- 172.21.15.105:0/3177028014 >> 172.21.15.105:0/3177028014 conn(0x562ccb0803b0 msgr2=0x562ccb08ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 -- 172.21.15.105:0/3177028014 shutdown_connections 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 1 -- 172.21.15.105:0/3177028014 wait complete. 2024-03-26T17:49:27.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc50d90740 20 client.19545 trim_cache size 0 max 16384 2024-03-26T17:49:27.417 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdc340a2f20 msgr2=0x7fdc340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdc340a2f20 0x7fdc340a53e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdc38005fd0 tx=0x7fdc38009500 comp rx=0 tx=0).stop 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc48104c80 msgr2=0x7fdc48198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.418+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc48104c80 0x7fdc48198000 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000e9c0 tx=0x7fdc3000ee80 comp rx=0 tx=0).stop 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.419+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/3683659065 shutdown_connections 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.419+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdc340a2f20 0x7fdc340a53e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.419+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc481075b0 0x7fdc4819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.419+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc48104c80 0x7fdc48198000 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.419+0000 7fdc4e0c6640 1 --2- 172.21.15.105:0/3683659065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdc48102350 0x7fdc48197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.419+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/3683659065 >> 172.21.15.105:0/3683659065 conn(0x7fdc480fbd60 msgr2=0x7fdc48102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.419+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/3683659065 shutdown_connections 2024-03-26T17:49:27.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.419+0000 7fdc4e0c6640 1 -- 172.21.15.105:0/3683659065 wait complete. 2024-03-26T17:49:27.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:49:27.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.550+0000 7feb2d3f3640 1 -- 172.21.15.105:0/4018479368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28104db0 msgr2=0x7feb281071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.550+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/4018479368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28104db0 0x7feb281071a0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7feb10009900 tx=0x7feb1002bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:27.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 -- 172.21.15.105:0/4018479368 shutdown_connections 2024-03-26T17:49:27.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/4018479368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb281076e0 0x7feb28109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/4018479368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28104db0 0x7feb281071a0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/4018479368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb28102480 0x7feb28104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 -- 172.21.15.105:0/4018479368 >> 172.21.15.105:0/4018479368 conn(0x7feb280fbdf0 msgr2=0x7feb280fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 -- 172.21.15.105:0/4018479368 shutdown_connections 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 -- 172.21.15.105:0/4018479368 wait complete. 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 Processor -- start 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 -- start start 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28102480 0x7feb28101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.551+0000 7feb2d3f3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb28104db0 0x7feb28100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb2d3f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb281076e0 0x7feb28100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb2d3f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb2810d3e0 con 0x7feb28102480 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb2d3f3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb2810d260 con 0x7feb28104db0 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb2d3f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb2810d560 con 0x7feb281076e0 2024-03-26T17:49:27.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28102480 0x7feb28101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28102480 0x7feb28101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46058/0 (socket says 172.21.15.105:46058) 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 -- 172.21.15.105:0/2748940041 learned_addr learned my addr 172.21.15.105:0/2748940041 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb28104db0 0x7feb28100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb277fe640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb281076e0 0x7feb28100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 -- 172.21.15.105:0/2748940041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb281076e0 msgr2=0x7feb28100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb281076e0 0x7feb28100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 -- 172.21.15.105:0/2748940041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb28104db0 msgr2=0x7feb28100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb28104db0 0x7feb28100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 -- 172.21.15.105:0/2748940041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb28101200 con 0x7feb28102480 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb277fe640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb281076e0 0x7feb28100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb26ffd640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28102480 0x7feb28101df0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7feb1800ea60 tx=0x7feb1800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb24ff9640 1 -- 172.21.15.105:0/2748940041 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1800cde0 con 0x7feb28102480 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb24ff9640 1 -- 172.21.15.105:0/2748940041 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb180035f0 con 0x7feb28102480 2024-03-26T17:49:27.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb2d3f3640 1 -- 172.21.15.105:0/2748940041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb281a1c90 con 0x7feb28102480 2024-03-26T17:49:27.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb24ff9640 1 -- 172.21.15.105:0/2748940041 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18010690 con 0x7feb28102480 2024-03-26T17:49:27.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.552+0000 7feb2d3f3640 1 -- 172.21.15.105:0/2748940041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb281a21b0 con 0x7feb28102480 2024-03-26T17:49:27.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.555+0000 7feb24ff9640 1 -- 172.21.15.105:0/2748940041 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7feb180037a0 con 0x7feb28102480 2024-03-26T17:49:27.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.555+0000 7feb2ffd7740 1 Processor -- start 2024-03-26T17:49:27.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.555+0000 7feb2ffd7740 1 -- start start 2024-03-26T17:49:27.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.555+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0056e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.555+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba005bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.555+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba005ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55db9ffbb970 con 0x55dba005c590 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db9ff63d70 con 0x55dba0057420 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55db9ffb7920 con 0x55dba0056a30 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba005ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb277fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0056e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba005ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46068/0 (socket says 172.21.15.105:46068) 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb277fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0056e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38012/0 (socket says 172.21.15.105:38012) 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 -- 172.21.15.105:0/3625193677 learned_addr learned my addr 172.21.15.105:0/3625193677 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb26ffd640 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba005bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb24ff9640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb140a2fa0 0x7feb140a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 -- 172.21.15.105:0/3625193677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 msgr2=0x55dba0056e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0056e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 -- 172.21.15.105:0/3625193677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 msgr2=0x55dba005bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba005bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 -- 172.21.15.105:0/3625193677 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db9fccf3f0 con 0x55dba005c590 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb26ffd640 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba005bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb277fe640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb140a2fa0 0x7feb140a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb24ff9640 1 -- 172.21.15.105:0/2748940041 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7feb180cf740 con 0x7feb28102480 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.556+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba005ea20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7feb2806a8d0 tx=0x7feb10009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d6534f4898db34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.557+0000 7feb1d7fa640 1 -- 172.21.15.105:0/3625193677 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10045020 con 0x55dba005c590 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.557+0000 7feb1d7fa640 1 -- 172.21.15.105:0/3625193677 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb10004040 con 0x55dba005c590 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.557+0000 7feb1d7fa640 1 -- 172.21.15.105:0/3625193677 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100055d0 con 0x55dba005c590 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 -- 172.21.15.105:0/3625193677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 msgr2=0x55dba005ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb277fe640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb140a2fa0 0x7feb140a5460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55dba0057c80 tx=0x7feb20004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba005ea20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7feb2806a8d0 tx=0x7feb10009560 comp rx=0 tx=0).stop 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 -- 172.21.15.105:0/3625193677 shutdown_connections 2024-03-26T17:49:27.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba005ea20 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba005bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/3625193677 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0056e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 -- 172.21.15.105:0/3625193677 >> 172.21.15.105:0/3625193677 conn(0x55dba0051fa0 msgr2=0x55dba00523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 -- 172.21.15.105:0/3625193677 shutdown_connections 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 -- 172.21.15.105:0/3625193677 wait complete. 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 -- start start 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.558+0000 7feb2ffd7740 1 Processor -- start 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0060ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba0061410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0068450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55dba005f4e0 con 0x55dba005c590 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db9ff77310 con 0x55dba0057420 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55dba005f660 con 0x55dba0056a30 2024-03-26T17:49:27.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb2ffd7740 10 client.0 _ll_register_callbacks cb 0x55db9ffb2c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0060ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0060ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38018/0 (socket says 172.21.15.105:38018) 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 -- 172.21.15.105:0/562035886 learned_addr learned my addr 172.21.15.105:0/562035886 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba0061410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb277fe640 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0068450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 -- 172.21.15.105:0/562035886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 msgr2=0x55dba0061410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba0061410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 -- 172.21.15.105:0/562035886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 msgr2=0x55dba0068450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0068450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 -- 172.21.15.105:0/562035886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db9fd997d0 con 0x55dba0056a30 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb26ffd640 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0060ed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feb1800ea30 tx=0x7feb18003d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fa311fcd78fd2d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb0e7fc640 1 -- 172.21.15.105:0/562035886 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1809bc00 con 0x55dba0056a30 2024-03-26T17:49:27.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb0e7fc640 1 -- 172.21.15.105:0/562035886 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7feb180baba0 con 0x55dba0056a30 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.559+0000 7feb2ffd7740 1 -- 172.21.15.105:0/562035886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb10004040 con 0x55dba0056a30 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb0e7fc640 1 -- 172.21.15.105:0/562035886 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb1809b790 con 0x55dba0056a30 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb2ffd7740 10 client.29344 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb2ffd7740 1 -- 172.21.15.105:0/562035886 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dba006b830 con 0x55dba0056a30 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb0e7fc640 1 -- 172.21.15.105:0/562035886 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb10004040 con 0x55dba0056a30 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb0e7fc640 1 -- 172.21.15.105:0/562035886 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7feb18004000 con 0x55dba0056a30 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb2ffd7740 10 client.29344 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb2ffd7740 10 client.29344 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb2ffd7740 20 client.29344 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55dba006cf10 0x55dba006f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb2ffd7740 4 client.29344 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb2ffd7740 1 -- 172.21.15.105:0/562035886 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db9ffc8eb0 con 0x55dba006cf10 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.560+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55dba006cf10 0x55dba006f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.561+0000 7feb0e7fc640 10 client.29344 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.561+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55dba006cf10 0x55dba006f380 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5140009ebcd2e4e7 server_cookie=9a3717e14527b56a in_seq=0 out_seq=0 2024-03-26T17:49:27.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.561+0000 7feb0e7fc640 1 -- 172.21.15.105:0/562035886 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55db9ffc8eb0 con 0x55dba006cf10 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.561+0000 7feb0e7fc640 10 client.29344 handle_command_reply: tid=0 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.561+0000 7feb2ffd7740 1 client.29344 shutdown 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.561+0000 7feb2ffd7740 1 -- 172.21.15.105:0/562035886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 msgr2=0x55dba0060ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.561+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0060ed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feb1800ea30 tx=0x7feb18003d20 comp rx=0 tx=0).stop 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 1 -- 172.21.15.105:0/562035886 shutdown_connections 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55dba006cf10 0x55dba006f380 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0068450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0057420 0x55dba0061410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb0e7fc640 0 client.29344 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/562035886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0056a30 0x55dba0060ed0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 1 -- 172.21.15.105:0/562035886 >> 172.21.15.105:0/562035886 conn(0x55dba0051fa0 msgr2=0x55dba00511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 1 -- 172.21.15.105:0/562035886 shutdown_connections 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 1 -- 172.21.15.105:0/562035886 wait complete. 2024-03-26T17:49:27.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.562+0000 7feb2ffd7740 20 client.29344 trim_cache size 0 max 16384 2024-03-26T17:49:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb2ffd7740 1 Processor -- start 2024-03-26T17:49:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb2ffd7740 1 -- start start 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0056a30 0x55dba0058110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0068410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0061370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb281aaff0 con 0x55dba005c590 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb281acb30 con 0x55dba0056a30 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb281ab6c0 con 0x55dba0057420 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0061370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.577+0000 7feb1e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0068410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb26ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0056a30 0x55dba0058110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0068410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38032/0 (socket says 172.21.15.105:38032) 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1e7fc640 1 -- 172.21.15.105:0/2512873144 learned_addr learned my addr 172.21.15.105:0/2512873144 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0061370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46094/0 (socket says 172.21.15.105:46094) 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1e7fc640 1 -- 172.21.15.105:0/2512873144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0056a30 msgr2=0x55dba0058110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/2512873144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0056a30 0x55dba0058110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1e7fc640 1 -- 172.21.15.105:0/2512873144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 msgr2=0x55dba0061370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/2512873144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0061370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1e7fc640 1 -- 172.21.15.105:0/2512873144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dba0057800 con 0x55dba0057420 2024-03-26T17:49:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/2512873144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0068410 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb10009900 tx=0x7feb1002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c19f1c8063af2f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1cff9640 1 -- 172.21.15.105:0/2512873144 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10053070 con 0x55dba0057420 2024-03-26T17:49:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1cff9640 1 -- 172.21.15.105:0/2512873144 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb10004470 con 0x55dba0057420 2024-03-26T17:49:27.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.578+0000 7feb1cff9640 1 -- 172.21.15.105:0/2512873144 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10045040 con 0x55dba0057420 2024-03-26T17:49:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 -- 172.21.15.105:0/2512873144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 msgr2=0x55dba0068410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/2512873144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0068410 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7feb10009900 tx=0x7feb1002bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 -- 172.21.15.105:0/2512873144 shutdown_connections 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/2512873144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba005c590 0x55dba0061370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/2512873144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0068410 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/2512873144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba0056a30 0x55dba0058110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 -- 172.21.15.105:0/2512873144 >> 172.21.15.105:0/2512873144 conn(0x55dba0051fa0 msgr2=0x55dba0055200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 -- 172.21.15.105:0/2512873144 shutdown_connections 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 -- 172.21.15.105:0/2512873144 wait complete. 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 -- start start 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.579+0000 7feb2ffd7740 1 Processor -- start 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba0056a30 0x55dba0065990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0065ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb2ffd7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba005c590 0x55dba0066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb281ac460 con 0x55dba0056a30 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb281abd90 con 0x55dba005c590 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb2ffd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb281a8fd0 con 0x55dba0057420 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba0056a30 0x55dba0065990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb2ffd7740 10 client.0 _ll_register_callbacks cb 0x7feb1800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba0056a30 0x55dba0065990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46096/0 (socket says 172.21.15.105:46096) 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 -- 172.21.15.105:0/1204909919 learned_addr learned my addr 172.21.15.105:0/1204909919 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0065ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb277fe640 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba005c590 0x55dba0066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 -- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 msgr2=0x55dba0065ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0065ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 -- 172.21.15.105:0/1204909919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba005c590 msgr2=0x55dba0066ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba005c590 0x55dba0066ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 -- 172.21.15.105:0/1204909919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dba00674d0 con 0x55dba0056a30 2024-03-26T17:49:27.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0065ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb26ffd640 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba0056a30 0x55dba0065990 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7feb180d5520 tx=0x7feb180cd250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89345879726a17e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb0e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.580+0000 7feb0e7fc640 1 -- 172.21.15.105:0/1204909919 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1809b970 con 0x55dba0056a30 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb0e7fc640 1 -- 172.21.15.105:0/1204909919 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7feb1809b280 con 0x55dba0056a30 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 1 -- 172.21.15.105:0/1204909919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb10004470 con 0x55dba0056a30 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb0e7fc640 1 -- 172.21.15.105:0/1204909919 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb1809bc00 con 0x55dba0056a30 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 10 client.19563 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 1 -- 172.21.15.105:0/1204909919 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dba00d3250 con 0x55dba0056a30 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb0e7fc640 1 -- 172.21.15.105:0/1204909919 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb10004470 con 0x55dba0056a30 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb0e7fc640 1 -- 172.21.15.105:0/1204909919 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7feb1809acf0 con 0x55dba0056a30 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 10 client.19563 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 10 client.19563 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 20 client.19563 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55dba00d3a30 0x55dba00d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 4 client.19563 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:27.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb2ffd7740 1 -- 172.21.15.105:0/1204909919 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db9ffc3200 con 0x55dba00d3a30 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.581+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55dba00d3a30 0x55dba00d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb1e7fc640 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55dba00d3a30 0x55dba00d5e50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4850aff6315f0ff server_cookie=25faba0c47ae95b9 in_seq=0 out_seq=0 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb0e7fc640 10 client.19563 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb0e7fc640 1 -- 172.21.15.105:0/1204909919 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db9ffc3200 con 0x55dba00d3a30 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb0e7fc640 10 client.19563 handle_command_reply: tid=0 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 client.19563 shutdown 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 -- 172.21.15.105:0/1204909919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba0056a30 msgr2=0x55dba0065990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba0056a30 0x55dba0065990 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7feb180d5520 tx=0x7feb180cd250 comp rx=0 tx=0).stop 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 -- 172.21.15.105:0/1204909919 shutdown_connections 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55dba00d3a30 0x55dba00d5e50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb0e7fc640 0 client.19563 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dba005c590 0x55dba0066ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55dba0057420 0x55dba0065ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 --2- 172.21.15.105:0/1204909919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55dba0056a30 0x55dba0065990 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 -- 172.21.15.105:0/1204909919 >> 172.21.15.105:0/1204909919 conn(0x55dba0051fa0 msgr2=0x55dba0055a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 -- 172.21.15.105:0/1204909919 shutdown_connections 2024-03-26T17:49:27.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 1 -- 172.21.15.105:0/1204909919 wait complete. 2024-03-26T17:49:27.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.582+0000 7feb2ffd7740 20 client.19563 trim_cache size 0 max 16384 2024-03-26T17:49:27.581 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:27.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 -- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb140a2fa0 msgr2=0x7feb140a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb140a2fa0 0x7feb140a5460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55dba0057c80 tx=0x7feb20004a40 comp rx=0 tx=0).stop 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 -- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28102480 msgr2=0x7feb28101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28102480 0x7feb28101df0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7feb1800ea60 tx=0x7feb1800ef20 comp rx=0 tx=0).stop 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 -- 172.21.15.105:0/2748940041 shutdown_connections 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb140a2fa0 0x7feb140a5460 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb281076e0 0x7feb28100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb28104db0 0x7feb28100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 --2- 172.21.15.105:0/2748940041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb28102480 0x7feb28101df0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 -- 172.21.15.105:0/2748940041 >> 172.21.15.105:0/2748940041 conn(0x7feb280fbdf0 msgr2=0x7feb28102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 -- 172.21.15.105:0/2748940041 shutdown_connections 2024-03-26T17:49:27.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.583+0000 7feb2d3f3640 1 -- 172.21.15.105:0/2748940041 wait complete. 2024-03-26T17:49:27.590 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:49:27.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 -- 172.21.15.105:0/1019327616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 msgr2=0x7f5268100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 --2- 172.21.15.105:0/1019327616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 0x7f5268100be0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5258009900 tx=0x7f525802bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 -- 172.21.15.105:0/1019327616 shutdown_connections 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 --2- 172.21.15.105:0/1019327616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5268102370 0x7f5268108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 --2- 172.21.15.105:0/1019327616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52681019b0 0x7f5268101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 --2- 172.21.15.105:0/1019327616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 0x7f5268100be0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 -- 172.21.15.105:0/1019327616 >> 172.21.15.105:0/1019327616 conn(0x7f52680fbd50 msgr2=0x7f52680fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 -- 172.21.15.105:0/1019327616 shutdown_connections 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.714+0000 7f526d7df640 1 -- 172.21.15.105:0/1019327616 wait complete. 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f526d7df640 1 Processor -- start 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f526d7df640 1 -- start start 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f526d7df640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 0x7f5268197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f526d7df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52681019b0 0x7f52681980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f526d7df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5268102370 0x7f526819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f526d7df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f526810d3d0 con 0x7f5268102370 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f526d7df640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f526810d250 con 0x7f52681007c0 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f526d7df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f526810d550 con 0x7f52681019b0 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5268102370 0x7f526819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 0x7f5268197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5268102370 0x7f526819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46128/0 (socket says 172.21.15.105:46128) 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 0x7f5268197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49742/0 (socket says 172.21.15.105:49742) 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 -- 172.21.15.105:0/426843438 learned_addr learned my addr 172.21.15.105:0/426843438 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f525ffff640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52681019b0 0x7f52681980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 -- 172.21.15.105:0/426843438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52681019b0 msgr2=0x7f52681980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52681019b0 0x7f52681980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 -- 172.21.15.105:0/426843438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 msgr2=0x7f5268197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 0x7f5268197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 -- 172.21.15.105:0/426843438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526819f7d0 con 0x7f5268102370 2024-03-26T17:49:27.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.715+0000 7f52677fe640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5268102370 0x7f526819f160 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f526000efd0 tx=0x7f526000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.716+0000 7f5264ff9640 1 -- 172.21.15.105:0/426843438 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260003b70 con 0x7f5268102370 2024-03-26T17:49:27.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.716+0000 7f526d7df640 1 -- 172.21.15.105:0/426843438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526819fac0 con 0x7f5268102370 2024-03-26T17:49:27.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.716+0000 7f5264ff9640 1 -- 172.21.15.105:0/426843438 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5260003d10 con 0x7f5268102370 2024-03-26T17:49:27.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.716+0000 7f5264ff9640 1 -- 172.21.15.105:0/426843438 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260010980 con 0x7f5268102370 2024-03-26T17:49:27.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.716+0000 7f526d7df640 1 -- 172.21.15.105:0/426843438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52681a0010 con 0x7f5268102370 2024-03-26T17:49:27.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.718+0000 7f5264ff9640 1 -- 172.21.15.105:0/426843438 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5260004020 con 0x7f5268102370 2024-03-26T17:49:27.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52703c6740 1 Processor -- start 2024-03-26T17:49:27.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52703c6740 1 -- start start 2024-03-26T17:49:27.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f5264ff9640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52540a2f20 0x7f52540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f525ffff640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52540a2f20 0x7f52540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52703c6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6b9a30 0x556b7f6b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52703c6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f5264ff9640 1 -- 172.21.15.105:0/426843438 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f52600cf830 con 0x7f5268102370 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52703c6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52703c6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556b7f61e970 con 0x556b7f6ba420 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52703c6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556b7f5c6d70 con 0x556b7f6b9a30 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52703c6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556b7f61a920 con 0x556b7f6bf590 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6b9a30 0x556b7f6b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f525ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6b9a30 0x556b7f6b9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49758/0 (socket says 172.21.15.105:49758) 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f5266ffd640 1 -- 172.21.15.105:0/70859200 learned_addr learned my addr 172.21.15.105:0/70859200 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.719+0000 7f52677fe640 1 --2- 172.21.15.105:0/70859200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f5266ffd640 1 -- 172.21.15.105:0/70859200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 msgr2=0x556b7f6c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f5266ffd640 1 --2- 172.21.15.105:0/70859200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f5266ffd640 1 -- 172.21.15.105:0/70859200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 msgr2=0x556b7f6bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f5266ffd640 1 --2- 172.21.15.105:0/70859200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f5266ffd640 1 -- 172.21.15.105:0/70859200 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b7f3323f0 con 0x556b7f6b9a30 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f525ffff640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52540a2f20 0x7f52540a53e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f52681990c0 tx=0x7f5250013830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f5266ffd640 1 --2- 172.21.15.105:0/70859200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6b9a30 0x556b7f6b9e50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5258009900 tx=0x7f5258009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8559d9f086816559 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f525d7fa640 1 -- 172.21.15.105:0/70859200 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258045020 con 0x556b7f6b9a30 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f525d7fa640 1 -- 172.21.15.105:0/70859200 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5258004040 con 0x556b7f6b9a30 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.720+0000 7f525d7fa640 1 -- 172.21.15.105:0/70859200 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258005540 con 0x556b7f6b9a30 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 -- 172.21.15.105:0/70859200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6b9a30 msgr2=0x556b7f6b9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 --2- 172.21.15.105:0/70859200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6b9a30 0x556b7f6b9e50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5258009900 tx=0x7f5258009560 comp rx=0 tx=0).stop 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 -- 172.21.15.105:0/70859200 shutdown_connections 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 --2- 172.21.15.105:0/70859200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 --2- 172.21.15.105:0/70859200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 --2- 172.21.15.105:0/70859200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6b9a30 0x556b7f6b9e50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 -- 172.21.15.105:0/70859200 >> 172.21.15.105:0/70859200 conn(0x556b7f6b4fa0 msgr2=0x556b7f6b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 -- 172.21.15.105:0/70859200 shutdown_connections 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.721+0000 7f52703c6740 1 -- 172.21.15.105:0/70859200 wait complete. 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 1 -- start start 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 1 Processor -- start 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6ba420 0x556b7f6c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556b7f6c24e0 con 0x556b7f6b9a30 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556b7f5da310 con 0x556b7f6ba420 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556b7f6c2660 con 0x556b7f6bf590 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52703c6740 10 client.0 _ll_register_callbacks cb 0x556b7f615c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f525ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6ba420 0x556b7f6c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46148/0 (socket says 172.21.15.105:46148) 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 -- 172.21.15.105:0/4012478037 learned_addr learned my addr 172.21.15.105:0/4012478037 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f525ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6ba420 0x556b7f6c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49772/0 (socket says 172.21.15.105:49772) 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f52677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38080/0 (socket says 172.21.15.105:38080) 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 -- 172.21.15.105:0/4012478037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 msgr2=0x556b7f6cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 -- 172.21.15.105:0/4012478037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6ba420 msgr2=0x556b7f6c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6ba420 0x556b7f6c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 -- 172.21.15.105:0/4012478037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b7f3fc7d0 con 0x556b7f6b9a30 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f5266ffd640 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c3ec0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f525802c170 tx=0x7f52580054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab6da704aa31313 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.722+0000 7f524e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f524e7fc640 1 -- 172.21.15.105:0/4012478037 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525804e070 con 0x556b7f6b9a30 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f524e7fc640 1 -- 172.21.15.105:0/4012478037 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f52580046c0 con 0x556b7f6b9a30 2024-03-26T17:49:27.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 1 -- 172.21.15.105:0/4012478037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5258004040 con 0x556b7f6b9a30 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f524e7fc640 1 -- 172.21.15.105:0/4012478037 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52580048c0 con 0x556b7f6b9a30 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 10 client.19575 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 1 -- 172.21.15.105:0/4012478037 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556b7f6ce7b0 con 0x556b7f6b9a30 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f524e7fc640 1 -- 172.21.15.105:0/4012478037 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5258004040 con 0x556b7f6b9a30 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f524e7fc640 1 -- 172.21.15.105:0/4012478037 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5258003b10 con 0x556b7f6b9a30 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 10 client.19575 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 10 client.19575 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 20 client.19575 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b7f6cfb50 0x556b7f6d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 4 client.19575 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f52703c6740 1 -- 172.21.15.105:0/4012478037 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b7f62beb0 con 0x556b7f6cfb50 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.723+0000 7f525ffff640 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b7f6cfb50 0x556b7f6d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.724+0000 7f524e7fc640 10 client.19575 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:49:27.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.724+0000 7f525ffff640 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b7f6cfb50 0x556b7f6d1f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=812f0df36bbfb14b server_cookie=7fe1e48b8f417fe4 in_seq=0 out_seq=0 2024-03-26T17:49:27.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.724+0000 7f524e7fc640 1 -- 172.21.15.105:0/4012478037 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556b7f62beb0 con 0x556b7f6cfb50 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.724+0000 7f524e7fc640 10 client.19575 handle_command_reply: tid=0 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 client.19575 shutdown 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 -- 172.21.15.105:0/4012478037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 msgr2=0x556b7f6c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c3ec0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f525802c170 tx=0x7f52580054f0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 -- 172.21.15.105:0/4012478037 shutdown_connections 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6bf590 0x556b7f6cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b7f6cfb50 0x556b7f6d1f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6ba420 0x556b7f6c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f524e7fc640 0 client.19575 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 --2- 172.21.15.105:0/4012478037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c3ec0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 -- 172.21.15.105:0/4012478037 >> 172.21.15.105:0/4012478037 conn(0x556b7f6b4fa0 msgr2=0x556b7f6b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 -- 172.21.15.105:0/4012478037 shutdown_connections 2024-03-26T17:49:27.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 1 -- 172.21.15.105:0/4012478037 wait complete. 2024-03-26T17:49:27.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.725+0000 7f52703c6740 20 client.19575 trim_cache size 0 max 16384 2024-03-26T17:49:27.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52703c6740 1 Processor -- start 2024-03-26T17:49:27.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52703c6740 1 -- start start 2024-03-26T17:49:27.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52703c6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52703c6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6ba420 0x556b7f6cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52703c6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 0x556b7f6b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52703c6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52681a8ca0 con 0x556b7f6b9a30 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52703c6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52681aa7e0 con 0x556b7f6bf590 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52703c6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52681a9370 con 0x556b7f6ba420 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46160/0 (socket says 172.21.15.105:46160) 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f5266ffd640 1 -- 172.21.15.105:0/3610056356 learned_addr learned my addr 172.21.15.105:0/3610056356 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f525ffff640 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6ba420 0x556b7f6cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.740+0000 7f52677fe640 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 0x556b7f6b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f5266ffd640 1 -- 172.21.15.105:0/3610056356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6ba420 msgr2=0x556b7f6cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f5266ffd640 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6ba420 0x556b7f6cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f5266ffd640 1 -- 172.21.15.105:0/3610056356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 msgr2=0x556b7f6b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f5266ffd640 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 0x556b7f6b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f5266ffd640 1 -- 172.21.15.105:0/3610056356 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b7f6b4b40 con 0x556b7f6b9a30 2024-03-26T17:49:27.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f525ffff640 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6ba420 0x556b7f6cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f5266ffd640 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c41c0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5258003eb0 tx=0x7f5258005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55e5045705a2ff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f525cff9640 1 -- 172.21.15.105:0/3610056356 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525804e040 con 0x556b7f6b9a30 2024-03-26T17:49:27.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f525cff9640 1 -- 172.21.15.105:0/3610056356 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5258058030 con 0x556b7f6b9a30 2024-03-26T17:49:27.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.741+0000 7f525cff9640 1 -- 172.21.15.105:0/3610056356 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258045020 con 0x556b7f6b9a30 2024-03-26T17:49:27.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 -- 172.21.15.105:0/3610056356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 msgr2=0x556b7f6c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c41c0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5258003eb0 tx=0x7f5258005dd0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 -- 172.21.15.105:0/3610056356 shutdown_connections 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 0x556b7f6b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6ba420 0x556b7f6cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 --2- 172.21.15.105:0/3610056356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6b9a30 0x556b7f6c41c0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 -- 172.21.15.105:0/3610056356 >> 172.21.15.105:0/3610056356 conn(0x556b7f6b4fa0 msgr2=0x556b7f6b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 -- 172.21.15.105:0/3610056356 shutdown_connections 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 -- 172.21.15.105:0/3610056356 wait complete. 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 -- start start 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.742+0000 7f52703c6740 1 Processor -- start 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52703c6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6b9a30 0x556b7f6c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52703c6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52703c6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 0x556b7f6ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52703c6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52681aa110 con 0x556b7f6ba420 2024-03-26T17:49:27.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52703c6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52681a9a40 con 0x556b7f6bf590 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52703c6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52681a6c80 con 0x556b7f6b9a30 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52703c6740 10 client.0 _ll_register_callbacks cb 0x7f5250009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f525ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6b9a30 0x556b7f6c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f525ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46166/0 (socket says 172.21.15.105:46166) 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f525ffff640 1 -- 172.21.15.105:0/3186813423 learned_addr learned my addr 172.21.15.105:0/3186813423 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52677fe640 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 0x556b7f6ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f5266ffd640 1 -- 172.21.15.105:0/3186813423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 msgr2=0x556b7f6ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f5266ffd640 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 0x556b7f6ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f5266ffd640 1 -- 172.21.15.105:0/3186813423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 msgr2=0x556b7f6c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f5266ffd640 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f5266ffd640 1 -- 172.21.15.105:0/3186813423 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b7f6b5d10 con 0x556b7f6b9a30 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f525ffff640 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f5266ffd640 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6b9a30 0x556b7f6c8d70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x556b7f6bab80 tx=0x7f5258005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88539dd03219b58 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f524e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f524e7fc640 1 -- 172.21.15.105:0/3186813423 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258045070 con 0x556b7f6b9a30 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f52703c6740 1 -- 172.21.15.105:0/3186813423 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5258058030 con 0x556b7f6b9a30 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.743+0000 7f524e7fc640 1 -- 172.21.15.105:0/3186813423 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f5258005010 con 0x556b7f6b9a30 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f524e7fc640 1 -- 172.21.15.105:0/3186813423 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f525802caf0 con 0x556b7f6b9a30 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f52703c6740 10 client.29368 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f52703c6740 1 -- 172.21.15.105:0/3186813423 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556b7f735fb0 con 0x556b7f6b9a30 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f524e7fc640 1 -- 172.21.15.105:0/3186813423 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5258058030 con 0x556b7f6b9a30 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f524e7fc640 1 -- 172.21.15.105:0/3186813423 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f52580416d0 con 0x556b7f6b9a30 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f52703c6740 10 client.29368 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f52703c6740 10 client.29368 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f52703c6740 20 client.29368 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f52703c6740 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b7f7366e0 0x556b7f738b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f52703c6740 4 client.29368 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f52703c6740 1 -- 172.21.15.105:0/3186813423 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b7f626200 con 0x556b7f7366e0 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.744+0000 7f525ffff640 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b7f7366e0 0x556b7f738b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f524e7fc640 10 client.29368 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f525ffff640 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b7f7366e0 0x556b7f738b00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=777477c920f83de0 server_cookie=5e88e7d8623835a6 in_seq=0 out_seq=0 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f524e7fc640 1 -- 172.21.15.105:0/3186813423 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b7f626200 con 0x556b7f7366e0 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f524e7fc640 10 client.29368 handle_command_reply: tid=0 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 client.29368 shutdown 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 -- 172.21.15.105:0/3186813423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6b9a30 msgr2=0x556b7f6c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6b9a30 0x556b7f6c8d70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x556b7f6bab80 tx=0x7f5258005a20 comp rx=0 tx=0).stop 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 -- 172.21.15.105:0/3186813423 shutdown_connections 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556b7f7366e0 0x556b7f738b00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556b7f6bf590 0x556b7f6ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f524e7fc640 0 client.29368 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:49:27.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556b7f6ba420 0x556b7f6c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 --2- 172.21.15.105:0/3186813423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556b7f6b9a30 0x556b7f6c8d70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 -- 172.21.15.105:0/3186813423 >> 172.21.15.105:0/3186813423 conn(0x556b7f6b4fa0 msgr2=0x556b7f6b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 -- 172.21.15.105:0/3186813423 shutdown_connections 2024-03-26T17:49:27.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 1 -- 172.21.15.105:0/3186813423 wait complete. 2024-03-26T17:49:27.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.745+0000 7f52703c6740 20 client.29368 trim_cache size 0 max 16384 2024-03-26T17:49:27.744 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 -- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52540a2f20 msgr2=0x7f52540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52540a2f20 0x7f52540a53e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f52681990c0 tx=0x7f5250013830 comp rx=0 tx=0).stop 2024-03-26T17:49:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 -- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5268102370 msgr2=0x7f526819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5268102370 0x7f526819f160 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f526000efd0 tx=0x7f526000c6a0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 -- 172.21.15.105:0/426843438 shutdown_connections 2024-03-26T17:49:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f52540a2f20 0x7f52540a53e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5268102370 0x7f526819f160 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52681019b0 0x7f52681980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 --2- 172.21.15.105:0/426843438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52681007c0 0x7f5268197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 -- 172.21.15.105:0/426843438 >> 172.21.15.105:0/426843438 conn(0x7f52680fbd50 msgr2=0x7f5268104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 -- 172.21.15.105:0/426843438 shutdown_connections 2024-03-26T17:49:27.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.746+0000 7f526d7df640 1 -- 172.21.15.105:0/426843438 wait complete. 2024-03-26T17:49:27.751 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:49:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 -- 172.21.15.105:0/2092965659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc109090 msgr2=0x7fb3bc109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/2092965659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc109090 0x7fb3bc109510 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009900 tx=0x7fb3b002bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:27.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 -- 172.21.15.105:0/2092965659 shutdown_connections 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/2092965659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc109a50 0x7fb3bc110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/2092965659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc109090 0x7fb3bc109510 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/2092965659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc107eb0 0x7fb3bc1082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 -- 172.21.15.105:0/2092965659 >> 172.21.15.105:0/2092965659 conn(0x7fb3bc076e90 msgr2=0x7fb3bc0792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 -- 172.21.15.105:0/2092965659 shutdown_connections 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 -- 172.21.15.105:0/2092965659 wait complete. 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.874+0000 7fb3c1b77640 1 Processor -- start 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3c1b77640 1 -- start start 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3c1b77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc107eb0 0x7fb3bc19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3c1b77640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc109090 0x7fb3bc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3c1b77640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc109a50 0x7fb3bc1a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3c1b77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc113a60 con 0x7fb3bc107eb0 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3c1b77640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1138e0 con 0x7fb3bc109090 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3c1b77640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc113be0 con 0x7fb3bc109a50 2024-03-26T17:49:27.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc107eb0 0x7fb3bc19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc107eb0 0x7fb3bc19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46170/0 (socket says 172.21.15.105:46170) 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bb7fe640 1 -- 172.21.15.105:0/1814664659 learned_addr learned my addr 172.21.15.105:0/1814664659 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3baffd640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc109090 0x7fb3bc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc109a50 0x7fb3bc1a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bb7fe640 1 -- 172.21.15.105:0/1814664659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc109a50 msgr2=0x7fb3bc1a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bb7fe640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc109a50 0x7fb3bc1a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bb7fe640 1 -- 172.21.15.105:0/1814664659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc109090 msgr2=0x7fb3bc19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bb7fe640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc109090 0x7fb3bc19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bb7fe640 1 -- 172.21.15.105:0/1814664659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc1a5d20 con 0x7fb3bc107eb0 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3baffd640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc109090 0x7fb3bc19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.875+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc109a50 0x7fb3bc1a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.876+0000 7fb3bb7fe640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc107eb0 0x7fb3bc19e100 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00e9c0 tx=0x7fb3ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.876+0000 7fb3b8ff9640 1 -- 172.21.15.105:0/1814664659 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00cd80 con 0x7fb3bc107eb0 2024-03-26T17:49:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.876+0000 7fb3b8ff9640 1 -- 172.21.15.105:0/1814664659 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3ac0035f0 con 0x7fb3bc107eb0 2024-03-26T17:49:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.876+0000 7fb3b8ff9640 1 -- 172.21.15.105:0/1814664659 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac010690 con 0x7fb3bc107eb0 2024-03-26T17:49:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.876+0000 7fb3c1b77640 1 -- 172.21.15.105:0/1814664659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc1a5fb0 con 0x7fb3bc107eb0 2024-03-26T17:49:27.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.876+0000 7fb3c1b77640 1 -- 172.21.15.105:0/1814664659 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc1a64c0 con 0x7fb3bc107eb0 2024-03-26T17:49:27.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3c4766740 1 Processor -- start 2024-03-26T17:49:27.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3c4766740 1 -- start start 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d223ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223b420 0x5600d223ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d2240590 0x5600d2242a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3c4766740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5600d219f970 con 0x5600d2240590 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3c4766740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5600d2147d70 con 0x5600d223b420 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3c4766740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5600d219b920 con 0x5600d223aa30 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3b8ff9640 1 -- 172.21.15.105:0/1814664659 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb3ac0037a0 con 0x7fb3bc107eb0 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d2240590 0x5600d2242a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d2240590 0x5600d2242a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46172/0 (socket says 172.21.15.105:46172) 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3baffd640 1 -- 172.21.15.105:0/4165424588 learned_addr learned my addr 172.21.15.105:0/4165424588 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d223ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3bb7fe640 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223b420 0x5600d223ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3bbfff640 1 -- 172.21.15.105:0/4165424588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223b420 msgr2=0x5600d223ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223b420 0x5600d223ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3bbfff640 1 -- 172.21.15.105:0/4165424588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d2240590 msgr2=0x5600d2242a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d2240590 0x5600d2242a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3bbfff640 1 -- 172.21.15.105:0/4165424588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d1eb33f0 con 0x5600d223aa30 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.879+0000 7fb3baffd640 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d2240590 0x5600d2242a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.880+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d223ae50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4005fd0 tx=0x7fb3b4011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968951a3c87aeb5d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.880+0000 7fb3b8ff9640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a2fa0 0x7fb3a40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.880+0000 7fb3a1ffb640 1 -- 172.21.15.105:0/4165424588 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004d20 con 0x5600d223aa30 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.880+0000 7fb3a1ffb640 1 -- 172.21.15.105:0/4165424588 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b400f9a0 con 0x5600d223aa30 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.880+0000 7fb3a1ffb640 1 -- 172.21.15.105:0/4165424588 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004470 con 0x5600d223aa30 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.880+0000 7fb3b8ff9640 1 -- 172.21.15.105:0/1814664659 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb3ac0cf6f0 con 0x7fb3bc107eb0 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.880+0000 7fb3bb7fe640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a2fa0 0x7fb3a40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3bb7fe640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a2fa0 0x7fb3a40a5460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0cd520 tx=0x7fb3ac003e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 -- 172.21.15.105:0/4165424588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 msgr2=0x5600d223ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d223ae50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4005fd0 tx=0x7fb3b4011bf0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 -- 172.21.15.105:0/4165424588 shutdown_connections 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d2240590 0x5600d2242a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223b420 0x5600d223ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 --2- 172.21.15.105:0/4165424588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d223ae50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 -- 172.21.15.105:0/4165424588 >> 172.21.15.105:0/4165424588 conn(0x5600d2235fa0 msgr2=0x5600d22363a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 -- 172.21.15.105:0/4165424588 shutdown_connections 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 -- 172.21.15.105:0/4165424588 wait complete. 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.881+0000 7fb3c4766740 1 -- start start 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3c4766740 1 Processor -- start 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d2244f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d22454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 0x5600d224c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3c4766740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5600d2243540 con 0x5600d223b420 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3c4766740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5600d215b310 con 0x5600d2240590 2024-03-26T17:49:27.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3c4766740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5600d22436c0 con 0x5600d223aa30 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3c4766740 10 client.0 _ll_register_callbacks cb 0x5600d2196c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d22454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3bbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d2244f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d22454c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46178/0 (socket says 172.21.15.105:46178) 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 -- 172.21.15.105:0/3019307465 learned_addr learned my addr 172.21.15.105:0/3019307465 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 0x5600d224c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 -- 172.21.15.105:0/3019307465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 msgr2=0x5600d2244f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d2244f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 -- 172.21.15.105:0/3019307465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 msgr2=0x5600d224c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 0x5600d224c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 -- 172.21.15.105:0/3019307465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d1f7d7d0 con 0x5600d223b420 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb3baffd640 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d22454c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0036040 tx=0x7fb3b0005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb11504bcf1b9b94 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.882+0000 7fb39affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:27.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb39affd640 1 -- 172.21.15.105:0/3019307465 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0046070 con 0x5600d223b420 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb39affd640 1 -- 172.21.15.105:0/3019307465 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb3b0005550 con 0x5600d223b420 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 1 -- 172.21.15.105:0/3019307465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b400f9a0 con 0x5600d223b420 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb39affd640 1 -- 172.21.15.105:0/3019307465 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb3b0005d50 con 0x5600d223b420 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 10 client.19593 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 1 -- 172.21.15.105:0/3019307465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5600d224f8e0 con 0x5600d223b420 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb39affd640 1 -- 172.21.15.105:0/3019307465 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b400f9a0 con 0x5600d223b420 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb39affd640 1 -- 172.21.15.105:0/3019307465 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b0003820 con 0x5600d223b420 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 10 client.19593 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 10 client.19593 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 20 client.19593 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5600d2250fc0 0x5600d22533e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 4 client.19593 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3c4766740 1 -- 172.21.15.105:0/3019307465 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600d21aceb0 con 0x5600d2250fc0 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.883+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5600d2250fc0 0x5600d22533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.884+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5600d2250fc0 0x5600d22533e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=892ee275a571eeb9 server_cookie=6621c7994ee8ae3b in_seq=0 out_seq=0 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.884+0000 7fb39affd640 10 client.19593 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.884+0000 7fb39affd640 1 -- 172.21.15.105:0/3019307465 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5600d21aceb0 con 0x5600d2250fc0 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.884+0000 7fb39affd640 10 client.19593 handle_command_reply: tid=0 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.884+0000 7fb3c4766740 1 client.19593 shutdown 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 -- 172.21.15.105:0/3019307465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 msgr2=0x5600d22454c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d22454c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0036040 tx=0x7fb3b0005250 comp rx=0 tx=0).stop 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 -- 172.21.15.105:0/3019307465 shutdown_connections 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5600d2250fc0 0x5600d22533e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 0x5600d224c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d22454c0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 --2- 172.21.15.105:0/3019307465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d2244f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb39affd640 0 client.19593 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 -- 172.21.15.105:0/3019307465 >> 172.21.15.105:0/3019307465 conn(0x5600d2235fa0 msgr2=0x5600d22351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 -- 172.21.15.105:0/3019307465 shutdown_connections 2024-03-26T17:49:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 1 -- 172.21.15.105:0/3019307465 wait complete. 2024-03-26T17:49:27.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.885+0000 7fb3c4766740 20 client.19593 trim_cache size 0 max 16384 2024-03-26T17:49:27.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:27 smithi105 ceph-mon[32280]: pgmap v682: 97 pgs: 97 active+clean; 27 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:49:27.891 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:27 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3515279414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:49:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3c4766740 1 Processor -- start 2024-03-26T17:49:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3c4766740 1 -- start start 2024-03-26T17:49:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d2234930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d224c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 0x5600d2234e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3c4766740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1af3d0 con 0x5600d223b420 2024-03-26T17:49:27.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3c4766740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1b0f10 con 0x5600d2240590 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3c4766740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1afaa0 con 0x5600d223aa30 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d224c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d224c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46184/0 (socket says 172.21.15.105:46184) 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 -- 172.21.15.105:0/828892968 learned_addr learned my addr 172.21.15.105:0/828892968 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d2234930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 0x5600d2234e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 -- 172.21.15.105:0/828892968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 msgr2=0x5600d2234930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d2234930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 -- 172.21.15.105:0/828892968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 msgr2=0x5600d2234e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 0x5600d2234e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 -- 172.21.15.105:0/828892968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d22437f0 con 0x5600d223b420 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.905+0000 7fb3baffd640 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d224c4c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0003de0 tx=0x7fb3b0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954a196e12710879 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.906+0000 7fb3a17fa640 1 -- 172.21.15.105:0/828892968 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0046070 con 0x5600d223b420 2024-03-26T17:49:27.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.906+0000 7fb3a17fa640 1 -- 172.21.15.105:0/828892968 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b002cc90 con 0x5600d223b420 2024-03-26T17:49:27.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.906+0000 7fb3a17fa640 1 -- 172.21.15.105:0/828892968 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0004b00 con 0x5600d223b420 2024-03-26T17:49:27.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- 172.21.15.105:0/828892968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 msgr2=0x5600d224c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d224c4c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0003de0 tx=0x7fb3b0009620 comp rx=0 tx=0).stop 2024-03-26T17:49:27.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- 172.21.15.105:0/828892968 shutdown_connections 2024-03-26T17:49:27.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d2240590 0x5600d2234e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d224c4c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 --2- 172.21.15.105:0/828892968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d223aa30 0x5600d2234930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- 172.21.15.105:0/828892968 >> 172.21.15.105:0/828892968 conn(0x5600d2235fa0 msgr2=0x5600d223c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- 172.21.15.105:0/828892968 shutdown_connections 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- 172.21.15.105:0/828892968 wait complete. 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- start start 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 Processor -- start 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223aa30 0x5600d2249830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d2249d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d2240590 0x5600d224ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1b0840 con 0x5600d223b420 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1b0170 con 0x5600d223aa30 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.907+0000 7fb3c4766740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1ad3b0 con 0x5600d2240590 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3c4766740 10 client.0 _ll_register_callbacks cb 0x7fb3b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d2249d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d2249d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46200/0 (socket says 172.21.15.105:46200) 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 -- 172.21.15.105:0/2948088660 learned_addr learned my addr 172.21.15.105:0/2948088660 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223aa30 0x5600d2249830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d2240590 0x5600d224ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 -- 172.21.15.105:0/2948088660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d2240590 msgr2=0x5600d224ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d2240590 0x5600d224ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 -- 172.21.15.105:0/2948088660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223aa30 msgr2=0x5600d2249830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223aa30 0x5600d2249830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 -- 172.21.15.105:0/2948088660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d224b370 con 0x5600d223b420 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3baffd640 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d2249d70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0006010 tx=0x7fb3b0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee7bbdd0420d36c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb39affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb39affd640 1 -- 172.21.15.105:0/2948088660 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b005a070 con 0x5600d223b420 2024-03-26T17:49:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb39affd640 1 -- 172.21.15.105:0/2948088660 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb3b0004a40 con 0x5600d223b420 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3c4766740 1 -- 172.21.15.105:0/2948088660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b002cc90 con 0x5600d223b420 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb39affd640 1 -- 172.21.15.105:0/2948088660 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb3b0065050 con 0x5600d223b420 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3c4766740 10 client.19599 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.908+0000 7fb3c4766740 1 -- 172.21.15.105:0/2948088660 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5600d224c670 con 0x5600d223b420 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb39affd640 1 -- 172.21.15.105:0/2948088660 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3b002cc90 con 0x5600d223b420 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb39affd640 1 -- 172.21.15.105:0/2948088660 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b0040ca0 con 0x5600d223b420 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb3c4766740 10 client.19599 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb3c4766740 10 client.19599 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb3c4766740 20 client.19599 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb3c4766740 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5600d22b7840 0x5600d22b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb3c4766740 4 client.19599 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb3c4766740 1 -- 172.21.15.105:0/2948088660 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600d21a7200 con 0x5600d22b7840 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5600d22b7840 0x5600d22b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb39affd640 10 client.19599 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.909+0000 7fb3bbfff640 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5600d22b7840 0x5600d22b9c60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7782f67f90004230 server_cookie=d21310b3fb125801 in_seq=0 out_seq=0 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb39affd640 1 -- 172.21.15.105:0/2948088660 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600d21a7200 con 0x5600d22b7840 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb39affd640 10 client.19599 handle_command_reply: tid=0 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 client.19599 shutdown 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 -- 172.21.15.105:0/2948088660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 msgr2=0x5600d2249d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d2249d70 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0006010 tx=0x7fb3b0003820 comp rx=0 tx=0).stop 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 -- 172.21.15.105:0/2948088660 shutdown_connections 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5600d22b7840 0x5600d22b9c60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5600d2240590 0x5600d224ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5600d223b420 0x5600d2249d70 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb39affd640 0 client.19599 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 --2- 172.21.15.105:0/2948088660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5600d223aa30 0x5600d2249830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 -- 172.21.15.105:0/2948088660 >> 172.21.15.105:0/2948088660 conn(0x5600d2235fa0 msgr2=0x5600d2239200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 -- 172.21.15.105:0/2948088660 shutdown_connections 2024-03-26T17:49:27.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 1 -- 172.21.15.105:0/2948088660 wait complete. 2024-03-26T17:49:27.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c4766740 20 client.19599 trim_cache size 0 max 16384 2024-03-26T17:49:27.909 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.910+0000 7fb3c1b77640 1 -- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a2fa0 msgr2=0x7fb3a40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a2fa0 0x7fb3a40a5460 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0cd520 tx=0x7fb3ac003e30 comp rx=0 tx=0).stop 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 -- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc107eb0 msgr2=0x7fb3bc19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc107eb0 0x7fb3bc19e100 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00e9c0 tx=0x7fb3ac00ee80 comp rx=0 tx=0).stop 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 -- 172.21.15.105:0/1814664659 shutdown_connections 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb3a40a2fa0 0x7fb3a40a5460 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3bc109a50 0x7fb3bc1a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb3bc109090 0x7fb3bc19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 --2- 172.21.15.105:0/1814664659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb3bc107eb0 0x7fb3bc19e100 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 -- 172.21.15.105:0/1814664659 >> 172.21.15.105:0/1814664659 conn(0x7fb3bc076e90 msgr2=0x7fb3bc10c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:27.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 -- 172.21.15.105:0/1814664659 shutdown_connections 2024-03-26T17:49:27.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:27.911+0000 7fb3c1b77640 1 -- 172.21.15.105:0/1814664659 wait complete. 2024-03-26T17:49:27.915 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.037+0000 7fda6348c640 1 -- 172.21.15.105:0/3609746886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c1019b0 msgr2=0x7fda5c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.037+0000 7fda6348c640 1 --2- 172.21.15.105:0/3609746886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c1019b0 0x7fda5c101e30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009860 tx=0x7fda4c02bb70 comp rx=0 tx=0).stop 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.037+0000 7fda6348c640 1 -- 172.21.15.105:0/3609746886 shutdown_connections 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.037+0000 7fda6348c640 1 --2- 172.21.15.105:0/3609746886 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda5c102370 0x7fda5c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.037+0000 7fda6348c640 1 --2- 172.21.15.105:0/3609746886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c1019b0 0x7fda5c101e30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.037+0000 7fda6348c640 1 --2- 172.21.15.105:0/3609746886 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda5c1007c0 0x7fda5c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.037+0000 7fda6348c640 1 -- 172.21.15.105:0/3609746886 >> 172.21.15.105:0/3609746886 conn(0x7fda5c0fbd50 msgr2=0x7fda5c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 -- 172.21.15.105:0/3609746886 shutdown_connections 2024-03-26T17:49:28.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 -- 172.21.15.105:0/3609746886 wait complete. 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 Processor -- start 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 -- start start 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda5c1007c0 0x7fda5c10b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda5c1019b0 0x7fda5c109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c102370 0x7fda5c109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10d410 con 0x7fda5c1007c0 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10d290 con 0x7fda5c1019b0 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda6348c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c10d590 con 0x7fda5c102370 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda61228640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda5c1007c0 0x7fda5c10b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda61228640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda5c1007c0 0x7fda5c10b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46226/0 (socket says 172.21.15.105:46226) 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda61228640 1 -- 172.21.15.105:0/558503314 learned_addr learned my addr 172.21.15.105:0/558503314 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda60a27640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda5c1019b0 0x7fda5c109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.038+0000 7fda61a29640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c102370 0x7fda5c109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda61228640 1 -- 172.21.15.105:0/558503314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c102370 msgr2=0x7fda5c109cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda61228640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c102370 0x7fda5c109cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda61228640 1 -- 172.21.15.105:0/558503314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda5c1019b0 msgr2=0x7fda5c109780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda61228640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda5c1019b0 0x7fda5c109780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda61228640 1 -- 172.21.15.105:0/558503314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c10a550 con 0x7fda5c1007c0 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda60a27640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda5c1019b0 0x7fda5c109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda61a29640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c102370 0x7fda5c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda61228640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda5c1007c0 0x7fda5c10b140 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fda5800e9c0 tx=0x7fda5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda527fc640 1 -- 172.21.15.105:0/558503314 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5800cd80 con 0x7fda5c1007c0 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda6348c640 1 -- 172.21.15.105:0/558503314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c1a1c10 con 0x7fda5c1007c0 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda527fc640 1 -- 172.21.15.105:0/558503314 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda580035f0 con 0x7fda5c1007c0 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda527fc640 1 -- 172.21.15.105:0/558503314 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58010690 con 0x7fda5c1007c0 2024-03-26T17:49:28.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.039+0000 7fda6348c640 1 -- 172.21.15.105:0/558503314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c1a2180 con 0x7fda5c1007c0 2024-03-26T17:49:28.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 357990 3.14 MB/sec execute 491 sec latency 87.453 ms 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.041+0000 7fda527fc640 1 -- 172.21.15.105:0/558503314 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fda580037a0 con 0x7fda5c1007c0 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda6607e740 1 Processor -- start 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda6607e740 1 -- start start 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda6607e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f26be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda6607e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f26c420 0x55886f270f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda6607e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f271590 0x55886f273a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda6607e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55886f1d0970 con 0x55886f26ba30 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda6607e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55886f178d70 con 0x55886f271590 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda6607e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55886f1cc920 con 0x55886f26c420 2024-03-26T17:49:28.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda61a29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f26be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda61228640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f26c420 0x55886f270f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda527fc640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda480a2f20 0x7fda480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda60a27640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f271590 0x55886f273a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda60a27640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f271590 0x55886f273a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49872/0 (socket says 172.21.15.105:49872) 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda61228640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f26c420 0x55886f270f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38190/0 (socket says 172.21.15.105:38190) 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda60a27640 1 -- 172.21.15.105:0/3610921078 learned_addr learned my addr 172.21.15.105:0/3610921078 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.042+0000 7fda61a29640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f26be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46238/0 (socket says 172.21.15.105:46238) 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda60a27640 1 -- 172.21.15.105:0/3610921078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f26c420 msgr2=0x55886f270f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda61a29640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda480a2f20 0x7fda480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda60a27640 1 --2- 172.21.15.105:0/3610921078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f26c420 0x55886f270f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda60a27640 1 -- 172.21.15.105:0/3610921078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 msgr2=0x55886f26be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda60a27640 1 --2- 172.21.15.105:0/3610921078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f26be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda60a27640 1 -- 172.21.15.105:0/3610921078 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886eee43f0 con 0x55886f271590 2024-03-26T17:49:28.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda527fc640 1 -- 172.21.15.105:0/558503314 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fda580cf7c0 con 0x7fda5c1007c0 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda60a27640 1 --2- 172.21.15.105:0/3610921078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f271590 0x55886f273a20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009860 tx=0x7fda4c005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8629a24c1644805 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda61a29640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda480a2f20 0x7fda480a53e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55886f26cc80 tx=0x7fda54004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda477fe640 1 -- 172.21.15.105:0/3610921078 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c039040 con 0x55886f271590 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda477fe640 1 -- 172.21.15.105:0/3610921078 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda4c034dc0 con 0x55886f271590 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.043+0000 7fda477fe640 1 -- 172.21.15.105:0/3610921078 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0035f0 con 0x55886f271590 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 -- 172.21.15.105:0/3610921078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f271590 msgr2=0x55886f273a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 --2- 172.21.15.105:0/3610921078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f271590 0x55886f273a20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009860 tx=0x7fda4c005db0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 -- 172.21.15.105:0/3610921078 shutdown_connections 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 --2- 172.21.15.105:0/3610921078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f271590 0x55886f273a20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 --2- 172.21.15.105:0/3610921078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f26c420 0x55886f270f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 --2- 172.21.15.105:0/3610921078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f26be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 -- 172.21.15.105:0/3610921078 >> 172.21.15.105:0/3610921078 conn(0x55886f266fa0 msgr2=0x55886f2673a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 -- 172.21.15.105:0/3610921078 shutdown_connections 2024-03-26T17:49:28.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.044+0000 7fda6607e740 1 -- 172.21.15.105:0/3610921078 wait complete. 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 1 -- start start 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 1 Processor -- start 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f275ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26c420 0x55886f276430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f27d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda61228640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f275ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55886f2744e0 con 0x55886f26ba30 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55886f18c310 con 0x55886f26c420 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55886f274660 con 0x55886f271590 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda61228640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f275ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46242/0 (socket says 172.21.15.105:46242) 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda61228640 1 -- 172.21.15.105:0/587958376 learned_addr learned my addr 172.21.15.105:0/587958376 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda6607e740 10 client.0 _ll_register_callbacks cb 0x55886f1c7c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.045+0000 7fda60a27640 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26c420 0x55886f276430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda61a29640 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f27d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda61228640 1 -- 172.21.15.105:0/587958376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 msgr2=0x55886f27d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda61228640 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f27d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda61228640 1 -- 172.21.15.105:0/587958376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26c420 msgr2=0x55886f276430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda61228640 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26c420 0x55886f276430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda61228640 1 -- 172.21.15.105:0/587958376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886efae7d0 con 0x55886f26ba30 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda61a29640 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f27d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda61228640 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f275ef0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fda5800cfd0 tx=0x7fda5809bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc7fc5431eef099 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda44ff9640 1 -- 172.21.15.105:0/587958376 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5809c5b0 con 0x55886f26ba30 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda6607e740 1 -- 172.21.15.105:0/587958376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda4c034dc0 con 0x55886f26ba30 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.046+0000 7fda44ff9640 1 -- 172.21.15.105:0/587958376 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fda5806a1b0 con 0x55886f26ba30 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda44ff9640 1 -- 172.21.15.105:0/587958376 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda5809beb0 con 0x55886f26ba30 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda6607e740 10 client.19611 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:28.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda6607e740 1 -- 172.21.15.105:0/587958376 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55886f2890b0 con 0x55886f26ba30 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda44ff9640 1 -- 172.21.15.105:0/587958376 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda4c034dc0 con 0x55886f26ba30 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda44ff9640 1 -- 172.21.15.105:0/587958376 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fda58097c00 con 0x55886f26ba30 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda6607e740 10 client.19611 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda6607e740 10 client.19611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda6607e740 20 client.19611 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda6607e740 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55886f28a720 0x55886f28cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda6607e740 4 client.19611 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda6607e740 1 -- 172.21.15.105:0/587958376 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55886f1ddeb0 con 0x55886f28a720 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.047+0000 7fda60a27640 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55886f28a720 0x55886f28cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.048+0000 7fda44ff9640 10 client.19611 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:28.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.048+0000 7fda60a27640 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55886f28a720 0x55886f28cb90 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a4cb83f5c9b4c9 server_cookie=717d2268d6c96a02 in_seq=0 out_seq=0 2024-03-26T17:49:28.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda44ff9640 1 -- 172.21.15.105:0/587958376 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55886f1ddeb0 con 0x55886f28a720 2024-03-26T17:49:28.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda44ff9640 10 client.19611 handle_command_reply: tid=0 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 client.19611 shutdown 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 -- 172.21.15.105:0/587958376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 msgr2=0x55886f275ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f275ef0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fda5800cfd0 tx=0x7fda5809bab0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 -- 172.21.15.105:0/587958376 shutdown_connections 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55886f28a720 0x55886f28cb90 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f27d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26c420 0x55886f276430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 --2- 172.21.15.105:0/587958376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26ba30 0x55886f275ef0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 -- 172.21.15.105:0/587958376 >> 172.21.15.105:0/587958376 conn(0x55886f266fa0 msgr2=0x55886f2661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda44ff9640 0 client.19611 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 -- 172.21.15.105:0/587958376 shutdown_connections 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.049+0000 7fda6607e740 1 -- 172.21.15.105:0/587958376 wait complete. 2024-03-26T17:49:28.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.050+0000 7fda6607e740 20 client.19611 trim_cache size 0 max 16384 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda6607e740 1 Processor -- start 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda6607e740 1 -- start start 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda6607e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f262050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda6607e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda6607e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f276210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda6607e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c1a6bc0 con 0x55886f26c420 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda6607e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c1a8700 con 0x55886f26ba30 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda6607e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c1a7290 con 0x55886f271590 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46252/0 (socket says 172.21.15.105:46252) 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda61228640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f262050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 -- 172.21.15.105:0/3438371642 learned_addr learned my addr 172.21.15.105:0/3438371642 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda61a29640 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f276210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 -- 172.21.15.105:0/3438371642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 msgr2=0x55886f276210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f276210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 -- 172.21.15.105:0/3438371642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 msgr2=0x55886f262050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f262050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 -- 172.21.15.105:0/3438371642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886f262820 con 0x55886f26c420 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda61228640 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f262050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.067+0000 7fda60a27640 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27d430 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0057c0 tx=0x7fda4c049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b280998367bd6ecd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda46ffd640 1 -- 172.21.15.105:0/3438371642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c039040 con 0x55886f26c420 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda46ffd640 1 -- 172.21.15.105:0/3438371642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda4c004030 con 0x55886f26c420 2024-03-26T17:49:28.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda46ffd640 1 -- 172.21.15.105:0/3438371642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c051280 con 0x55886f26c420 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 -- 172.21.15.105:0/3438371642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 msgr2=0x55886f27d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27d430 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0057c0 tx=0x7fda4c049040 comp rx=0 tx=0).stop 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 -- 172.21.15.105:0/3438371642 shutdown_connections 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f276210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27d430 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 --2- 172.21.15.105:0/3438371642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f262050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 -- 172.21.15.105:0/3438371642 >> 172.21.15.105:0/3438371642 conn(0x55886f266fa0 msgr2=0x55886f275b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 -- 172.21.15.105:0/3438371642 shutdown_connections 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.068+0000 7fda6607e740 1 -- 172.21.15.105:0/3438371642 wait complete. 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 1 -- start start 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 1 Processor -- start 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f27a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f2eea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c1a8030 con 0x55886f26c420 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c1a7960 con 0x55886f26ba30 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c1a4ba0 con 0x55886f271590 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda6607e740 10 client.0 _ll_register_callbacks cb 0x7fda58003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda61228640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f27a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda60a27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda61228640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f27a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49908/0 (socket says 172.21.15.105:49908) 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda61228640 1 -- 172.21.15.105:0/622749400 learned_addr learned my addr 172.21.15.105:0/622749400 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.069+0000 7fda61a29640 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f2eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda61228640 1 -- 172.21.15.105:0/622749400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 msgr2=0x55886f2eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda61228640 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f2eea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda61228640 1 -- 172.21.15.105:0/622749400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 msgr2=0x55886f27a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:28.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda61228640 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda61228640 1 -- 172.21.15.105:0/622749400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886f2eef90 con 0x55886f26ba30 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda60a27640 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda61228640 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f27a3e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda580cfc40 tx=0x7fda580980b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511b0d6049ab9930 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda44ff9640 1 -- 172.21.15.105:0/622749400 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda580a4630 con 0x55886f26ba30 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda6607e740 1 -- 172.21.15.105:0/622749400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda4c004030 con 0x55886f26ba30 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda44ff9640 1 -- 172.21.15.105:0/622749400 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fda5809b300 con 0x55886f26ba30 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda44ff9640 1 -- 172.21.15.105:0/622749400 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda580a48c0 con 0x55886f26ba30 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda6607e740 10 client.29420 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda6607e740 1 -- 172.21.15.105:0/622749400 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55886f2f0280 con 0x55886f26ba30 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.070+0000 7fda44ff9640 1 -- 172.21.15.105:0/622749400 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda4c004030 con 0x55886f26ba30 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda44ff9640 1 -- 172.21.15.105:0/622749400 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fda580a1220 con 0x55886f26ba30 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda6607e740 10 client.29420 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda6607e740 10 client.29420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:28.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda6607e740 20 client.29420 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda6607e740 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55886f2f0c00 0x55886f2f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda6607e740 4 client.29420 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda6607e740 1 -- 172.21.15.105:0/622749400 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55886f26d4c0 con 0x55886f2f0c00 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda60a27640 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55886f2f0c00 0x55886f2f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda44ff9640 10 client.29420 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda60a27640 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55886f2f0c00 0x55886f2f3020 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49780590a78bdf3a server_cookie=b63061f66c203cd3 in_seq=0 out_seq=0 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.071+0000 7fda44ff9640 1 -- 172.21.15.105:0/622749400 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55886f26d4c0 con 0x55886f2f0c00 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda44ff9640 10 client.29420 handle_command_reply: tid=0 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 client.29420 shutdown 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 -- 172.21.15.105:0/622749400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 msgr2=0x55886f27a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f27a3e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda580cfc40 tx=0x7fda580980b0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 -- 172.21.15.105:0/622749400 shutdown_connections 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55886f2f0c00 0x55886f2f3020 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55886f271590 0x55886f2eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55886f26c420 0x55886f27a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda44ff9640 0 client.29420 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 --2- 172.21.15.105:0/622749400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55886f26ba30 0x55886f27a3e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 -- 172.21.15.105:0/622749400 >> 172.21.15.105:0/622749400 conn(0x55886f266fa0 msgr2=0x55886f26a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 -- 172.21.15.105:0/622749400 shutdown_connections 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 1 -- 172.21.15.105:0/622749400 wait complete. 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6607e740 20 client.29420 trim_cache size 0 max 16384 2024-03-26T17:49:28.070 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:49:28.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6348c640 1 -- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda480a2f20 msgr2=0x7fda480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6348c640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda480a2f20 0x7fda480a53e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55886f26cc80 tx=0x7fda54004ab0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6348c640 1 -- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda5c1007c0 msgr2=0x7fda5c10b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:28.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.072+0000 7fda6348c640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda5c1007c0 0x7fda5c10b140 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fda5800e9c0 tx=0x7fda5800ee80 comp rx=0 tx=0).stop 2024-03-26T17:49:28.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.073+0000 7fda6348c640 1 -- 172.21.15.105:0/558503314 shutdown_connections 2024-03-26T17:49:28.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.073+0000 7fda6348c640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda480a2f20 0x7fda480a53e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.073+0000 7fda6348c640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c102370 0x7fda5c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.073+0000 7fda6348c640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda5c1019b0 0x7fda5c109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.073+0000 7fda6348c640 1 --2- 172.21.15.105:0/558503314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda5c1007c0 0x7fda5c10b140 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:28.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.073+0000 7fda6348c640 1 -- 172.21.15.105:0/558503314 >> 172.21.15.105:0/558503314 conn(0x7fda5c0fbd50 msgr2=0x7fda5c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:28.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.073+0000 7fda6348c640 1 -- 172.21.15.105:0/558503314 shutdown_connections 2024-03-26T17:49:28.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:28.073+0000 7fda6348c640 1 -- 172.21.15.105:0/558503314 wait complete. 2024-03-26T17:49:28.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:27 smithi144 ceph-mon[36095]: pgmap v682: 97 pgs: 97 active+clean; 27 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:49:28.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:27 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3515279414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:49:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:27 smithi135 ceph-mon[35636]: pgmap v682: 97 pgs: 97 active+clean; 27 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:49:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:27 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3515279414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:49:29.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 358477 3.15 MB/sec execute 492 sec latency 18.896 ms 2024-03-26T17:49:29.078 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:49:29.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:49:29.078 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 -- 172.21.15.105:0/2723520006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 msgr2=0x7f5e80108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2723520006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 0x7f5e80108c00 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009ca0 tx=0x7f5e7802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 -- 172.21.15.105:0/2723520006 shutdown_connections 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2723520006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 0x7f5e80108c00 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2723520006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e801019b0 0x7f5e80101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2723520006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e801007c0 0x7f5e80100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 -- 172.21.15.105:0/2723520006 >> 172.21.15.105:0/2723520006 conn(0x7f5e800fbd50 msgr2=0x7f5e800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 -- 172.21.15.105:0/2723520006 shutdown_connections 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.207+0000 7f5e884f8640 1 -- 172.21.15.105:0/2723520006 wait complete. 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e884f8640 1 Processor -- start 2024-03-26T17:49:29.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e884f8640 1 -- start start 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e884f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e801007c0 0x7f5e80069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e884f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e801019b0 0x7f5e8019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e884f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 0x7f5e8019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e884f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8010c2f0 con 0x7f5e80102370 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e884f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8010c170 con 0x7f5e801007c0 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e884f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8010c470 con 0x7f5e801019b0 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e86a95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 0x7f5e8019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e86294640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e801007c0 0x7f5e80069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e86a95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 0x7f5e8019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46266/0 (socket says 172.21.15.105:46266) 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e86294640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e801007c0 0x7f5e80069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49912/0 (socket says 172.21.15.105:49912) 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e86a95640 1 -- 172.21.15.105:0/2945129658 learned_addr learned my addr 172.21.15.105:0/2945129658 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.208+0000 7f5e85a93640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e801019b0 0x7f5e8019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e86a95640 1 -- 172.21.15.105:0/2945129658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e801019b0 msgr2=0x7f5e8019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e86a95640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e801019b0 0x7f5e8019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e86a95640 1 -- 172.21.15.105:0/2945129658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e801007c0 msgr2=0x7f5e80069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e86a95640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e801007c0 0x7f5e80069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e86a95640 1 -- 172.21.15.105:0/2945129658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8019f740 con 0x7f5e80102370 2024-03-26T17:49:29.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e86a95640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 0x7f5e8019ee30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5e7802c700 tx=0x7f5e780328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e777fe640 1 -- 172.21.15.105:0/2945129658 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78041020 con 0x7f5e80102370 2024-03-26T17:49:29.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e884f8640 1 -- 172.21.15.105:0/2945129658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8019f970 con 0x7f5e80102370 2024-03-26T17:49:29.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e777fe640 1 -- 172.21.15.105:0/2945129658 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e78003e20 con 0x7f5e80102370 2024-03-26T17:49:29.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e777fe640 1 -- 172.21.15.105:0/2945129658 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78039040 con 0x7f5e80102370 2024-03-26T17:49:29.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.209+0000 7f5e884f8640 1 -- 172.21.15.105:0/2945129658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8019feb0 con 0x7f5e80102370 2024-03-26T17:49:29.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.212+0000 7f5e777fe640 1 -- 172.21.15.105:0/2945129658 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5e78044020 con 0x7f5e80102370 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.212+0000 7f5e8b1c2740 1 Processor -- start 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.212+0000 7f5e8b1c2740 1 -- start start 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318eb35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318eb8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 0x558318ebb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558318e18630 con 0x558318eb8cf0 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558318dc0e10 con 0x558318eb3b80 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558318e145e0 con 0x558318eb3190 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318eb35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e86294640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318eb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e777fe640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e6c0a2f20 0x7f5e6c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e86294640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318eb8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49918/0 (socket says 172.21.15.105:49918) 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318eb35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38236/0 (socket says 172.21.15.105:38236) 2024-03-26T17:49:29.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e86294640 1 -- 172.21.15.105:0/974011026 learned_addr learned my addr 172.21.15.105:0/974011026 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 --2- 172.21.15.105:0/974011026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 0x558318ebb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e86a95640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e6c0a2f20 0x7f5e6c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e777fe640 1 -- 172.21.15.105:0/2945129658 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f5e780f1600 con 0x7f5e80102370 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 -- 172.21.15.105:0/974011026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 msgr2=0x558318eb8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 --2- 172.21.15.105:0/974011026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318eb8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 -- 172.21.15.105:0/974011026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 msgr2=0x558318ebb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 --2- 172.21.15.105:0/974011026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 0x558318ebb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 -- 172.21.15.105:0/974011026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558318dd7740 con 0x558318eb3190 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.213+0000 7f5e85a93640 1 --2- 172.21.15.105:0/974011026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318eb35b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5e80069300 tx=0x7f5e700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e968b80a8e765417 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.214+0000 7f5e86a95640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e6c0a2f20 0x7f5e6c0a53e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e780ba150 tx=0x7f5e780b31d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.214+0000 7f5e74bfe640 1 -- 172.21.15.105:0/974011026 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70004050 con 0x558318eb3190 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.214+0000 7f5e74bfe640 1 -- 172.21.15.105:0/974011026 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e7000fc70 con 0x558318eb3190 2024-03-26T17:49:29.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.214+0000 7f5e74bfe640 1 -- 172.21.15.105:0/974011026 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e700038e0 con 0x558318eb3190 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.214+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/974011026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 msgr2=0x558318eb35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.214+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/974011026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318eb35b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5e80069300 tx=0x7f5e700046c0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/974011026 shutdown_connections 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/974011026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 0x558318ebb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/974011026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318eb8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/974011026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318eb35b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/974011026 >> 172.21.15.105:0/974011026 conn(0x558318eae700 msgr2=0x558318eaeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/974011026 shutdown_connections 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/974011026 wait complete. 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 -- start start 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 Processor -- start 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ebd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ebdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 0x558318ec4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558318ebbca0 con 0x558318eb8cf0 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558318dd41a0 con 0x558318eb3b80 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558318ebbe20 con 0x558318eb3190 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.215+0000 7f5e8b1c2740 10 client.0 _ll_register_callbacks cb 0x558318b2bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e85a93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 0x558318ec4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e86294640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ebdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e85a93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ebd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e85a93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ebd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38242/0 (socket says 172.21.15.105:38242) 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e85a93640 1 -- 172.21.15.105:0/1264036700 learned_addr learned my addr 172.21.15.105:0/1264036700 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e86294640 1 -- 172.21.15.105:0/1264036700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 msgr2=0x558318ebd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e86294640 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ebd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e86294640 1 -- 172.21.15.105:0/1264036700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 msgr2=0x558318ec4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e86294640 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 0x558318ec4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e86294640 1 -- 172.21.15.105:0/1264036700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558318bf47e0 con 0x558318eb3b80 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e85a93640 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ebd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e86294640 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ebdca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x558318eb3f80 tx=0x7f5e7c012b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b913600469c3b93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e69ffb640 1 -- 172.21.15.105:0/1264036700 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c004050 con 0x558318eb3b80 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e69ffb640 1 -- 172.21.15.105:0/1264036700 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f5e7c0041f0 con 0x558318eb3b80 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.216+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/1264036700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7000fc70 con 0x558318eb3b80 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e69ffb640 1 -- 172.21.15.105:0/1264036700 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5e7c0043f0 con 0x558318eb3b80 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e8b1c2740 10 client.29432 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/1264036700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558318c54f70 con 0x558318eb3b80 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e69ffb640 1 -- 172.21.15.105:0/1264036700 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e7000fc70 con 0x558318eb3b80 2024-03-26T17:49:29.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e69ffb640 1 -- 172.21.15.105:0/1264036700 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5e7c003bd0 con 0x558318eb3b80 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e8b1c2740 10 client.29432 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e8b1c2740 10 client.29432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e8b1c2740 20 client.29432 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558318ec9580 0x558318ecb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e8b1c2740 4 client.29432 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/1264036700 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558318e25b70 con 0x558318ec9580 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.217+0000 7f5e85a93640 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558318ec9580 0x558318ecb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.218+0000 7f5e69ffb640 10 client.29432 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:29.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.218+0000 7f5e85a93640 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558318ec9580 0x558318ecb9c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f0ead17fd07096 server_cookie=b49293ffb41d8e6a in_seq=0 out_seq=0 2024-03-26T17:49:29.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e69ffb640 1 -- 172.21.15.105:0/1264036700 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558318e25b70 con 0x558318ec9580 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e69ffb640 10 client.29432 handle_command_reply: tid=0 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 client.29432 shutdown 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/1264036700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 msgr2=0x558318ebdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ebdca0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x558318eb3f80 tx=0x7f5e7c012b10 comp rx=0 tx=0).stop 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/1264036700 shutdown_connections 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558318ec9580 0x558318ecb9c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb8cf0 0x558318ec4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ebdca0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/1264036700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ebd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e69ffb640 0 client.29432 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/1264036700 >> 172.21.15.105:0/1264036700 conn(0x558318eae700 msgr2=0x558318ead900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/1264036700 shutdown_connections 2024-03-26T17:49:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/1264036700 wait complete. 2024-03-26T17:49:29.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.219+0000 7f5e8b1c2740 20 client.29432 trim_cache size 0 max 16384 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e8b1c2740 1 Processor -- start 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e8b1c2740 1 -- start start 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3190 0x558318eb50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ec4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb8cf0 0x558318eb5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801a8dc0 con 0x558318eb3190 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801aa900 con 0x558318eb3b80 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801a9490 con 0x558318eb8cf0 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3190 0x558318eb50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e86294640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ec4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3190 0x558318eb50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46288/0 (socket says 172.21.15.105:46288) 2024-03-26T17:49:29.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 -- 172.21.15.105:0/888614422 learned_addr learned my addr 172.21.15.105:0/888614422 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 --2- 172.21.15.105:0/888614422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb8cf0 0x558318eb5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 -- 172.21.15.105:0/888614422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb8cf0 msgr2=0x558318eb5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 --2- 172.21.15.105:0/888614422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb8cf0 0x558318eb5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 -- 172.21.15.105:0/888614422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 msgr2=0x558318ec4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 --2- 172.21.15.105:0/888614422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ec4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 -- 172.21.15.105:0/888614422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558318aec170 con 0x558318eb3190 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.242+0000 7f5e85a93640 1 --2- 172.21.15.105:0/888614422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3190 0x558318eb50c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5e700058c0 tx=0x7f5e70005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b19937e4c2573 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.243+0000 7f5e6bfff640 1 -- 172.21.15.105:0/888614422 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70028040 con 0x558318eb3190 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.243+0000 7f5e6bfff640 1 -- 172.21.15.105:0/888614422 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e70005040 con 0x558318eb3190 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.243+0000 7f5e6bfff640 1 -- 172.21.15.105:0/888614422 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70004990 con 0x558318eb3190 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.243+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/888614422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3190 msgr2=0x558318eb50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.243+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/888614422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3190 0x558318eb50c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5e700058c0 tx=0x7f5e70005850 comp rx=0 tx=0).stop 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.243+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/888614422 shutdown_connections 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.243+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/888614422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb8cf0 0x558318eb5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.243+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/888614422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb3b80 0x558318ec4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/888614422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3190 0x558318eb50c0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/888614422 >> 172.21.15.105:0/888614422 conn(0x558318eae700 msgr2=0x558318ebd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/888614422 shutdown_connections 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/888614422 wait complete. 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 -- start start 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 Processor -- start 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ec23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3b80 0x558318ec28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb8cf0 0x558318ec38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801aa230 con 0x558318eb3b80 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801a9b60 con 0x558318eb8cf0 2024-03-26T17:49:29.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801a6da0 con 0x558318eb3190 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e8b1c2740 10 client.0 _ll_register_callbacks cb 0x7f5e7c009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e86294640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3b80 0x558318ec28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e86294640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3b80 0x558318ec28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46300/0 (socket says 172.21.15.105:46300) 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.244+0000 7f5e86294640 1 -- 172.21.15.105:0/2429346419 learned_addr learned my addr 172.21.15.105:0/2429346419 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e85a93640 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ec23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e85a93640 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb8cf0 0x558318ec38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e86294640 1 -- 172.21.15.105:0/2429346419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 msgr2=0x558318ec23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e86294640 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ec23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e86294640 1 -- 172.21.15.105:0/2429346419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb8cf0 msgr2=0x558318ec38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e86294640 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb8cf0 0x558318ec38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e86294640 1 -- 172.21.15.105:0/2429346419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558318ec3e90 con 0x558318eb3b80 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e86294640 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3b80 0x558318ec28e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x558318ebec40 tx=0x7f5e7c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87ae6bd979d6be1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:29.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e69ffb640 1 -- 172.21.15.105:0/2429346419 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c034060 con 0x558318eb3b80 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e69ffb640 1 -- 172.21.15.105:0/2429346419 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f5e7c02c040 con 0x558318eb3b80 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/2429346419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e70005040 con 0x558318eb3b80 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e69ffb640 1 -- 172.21.15.105:0/2429346419 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5e7c010a10 con 0x558318eb3b80 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e8b1c2740 10 client.19635 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/2429346419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558318f2f900 con 0x558318eb3b80 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.245+0000 7f5e69ffb640 1 -- 172.21.15.105:0/2429346419 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e70005040 con 0x558318eb3b80 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e69ffb640 1 -- 172.21.15.105:0/2429346419 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5e7c005030 con 0x558318eb3b80 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e8b1c2740 10 client.19635 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e8b1c2740 10 client.19635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e8b1c2740 20 client.19635 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558318f301d0 0x558318f325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e8b1c2740 4 client.19635 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/2429346419 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558318e1fec0 con 0x558318f301d0 2024-03-26T17:49:29.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e85a93640 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558318f301d0 0x558318f325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e69ffb640 10 client.19635 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.246+0000 7f5e85a93640 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558318f301d0 0x558318f325f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91fba0dbf60f74e1 server_cookie=22ff33cdb547c919 in_seq=0 out_seq=0 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e69ffb640 1 -- 172.21.15.105:0/2429346419 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558318e1fec0 con 0x558318f301d0 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e69ffb640 10 client.19635 handle_command_reply: tid=0 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 client.19635 shutdown 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/2429346419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3b80 msgr2=0x558318ec28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3b80 0x558318ec28e0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x558318ebec40 tx=0x7f5e7c00f790 comp rx=0 tx=0).stop 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/2429346419 shutdown_connections 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558318f301d0 0x558318f325f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558318eb8cf0 0x558318ec38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558318eb3b80 0x558318ec28e0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 --2- 172.21.15.105:0/2429346419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558318eb3190 0x558318ec23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/2429346419 >> 172.21.15.105:0/2429346419 conn(0x558318eae700 msgr2=0x558318eb50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e69ffb640 0 client.19635 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:29.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/2429346419 shutdown_connections 2024-03-26T17:49:29.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 1 -- 172.21.15.105:0/2429346419 wait complete. 2024-03-26T17:49:29.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.247+0000 7f5e8b1c2740 20 client.19635 trim_cache size 0 max 16384 2024-03-26T17:49:29.246 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:49:29.246 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:49:29.246 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "23d0247e-4114-488f-8e1a-47598c044b17", 2024-03-26T17:49:29.246 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:49:29.246 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:49:29.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 -- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e6c0a2f20 msgr2=0x7f5e6c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e6c0a2f20 0x7f5e6c0a53e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e780ba150 tx=0x7f5e780b31d0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 -- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 msgr2=0x7f5e8019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 0x7f5e8019ee30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5e7802c700 tx=0x7f5e780328a0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 -- 172.21.15.105:0/2945129658 shutdown_connections 2024-03-26T17:49:29.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5e6c0a2f20 0x7f5e6c0a53e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e80102370 0x7f5e8019ee30 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e801019b0 0x7f5e8019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 --2- 172.21.15.105:0/2945129658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e801007c0 0x7f5e80069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 -- 172.21.15.105:0/2945129658 >> 172.21.15.105:0/2945129658 conn(0x7f5e800fbd50 msgr2=0x7f5e80104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 -- 172.21.15.105:0/2945129658 shutdown_connections 2024-03-26T17:49:29.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.248+0000 7f5e884f8640 1 -- 172.21.15.105:0/2945129658 wait complete. 2024-03-26T17:49:29.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:49:29.251 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:49:29.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 -- 172.21.15.105:0/3063685769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 msgr2=0x7f0b20100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 --2- 172.21.15.105:0/3063685769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 0x7f0b20100be0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009900 tx=0x7f0b1002bc30 comp rx=0 tx=0).stop 2024-03-26T17:49:29.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 -- 172.21.15.105:0/3063685769 shutdown_connections 2024-03-26T17:49:29.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 --2- 172.21.15.105:0/3063685769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b20102370 0x7f0b20108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 --2- 172.21.15.105:0/3063685769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b201019b0 0x7f0b20101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 --2- 172.21.15.105:0/3063685769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 0x7f0b20100be0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 -- 172.21.15.105:0/3063685769 >> 172.21.15.105:0/3063685769 conn(0x7f0b200fbd50 msgr2=0x7f0b200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 -- 172.21.15.105:0/3063685769 shutdown_connections 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.385+0000 7f0b25722640 1 -- 172.21.15.105:0/3063685769 wait complete. 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b25722640 1 Processor -- start 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b25722640 1 -- start start 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b25722640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 0x7f0b20196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b25722640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b201019b0 0x7f0b20196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b25722640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b20102370 0x7f0b2019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b25722640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010c310 con 0x7f0b201007c0 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b25722640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010c190 con 0x7f0b201019b0 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b25722640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2010c490 con 0x7f0b20102370 2024-03-26T17:49:29.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 0x7f0b20196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 0x7f0b20196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46322/0 (socket says 172.21.15.105:46322) 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b1effd640 1 -- 172.21.15.105:0/23855606 learned_addr learned my addr 172.21.15.105:0/23855606 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b201019b0 0x7f0b20196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.386+0000 7f0b1f7fe640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b20102370 0x7f0b2019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b1effd640 1 -- 172.21.15.105:0/23855606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b20102370 msgr2=0x7f0b2019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b1effd640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b20102370 0x7f0b2019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b1effd640 1 -- 172.21.15.105:0/23855606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b201019b0 msgr2=0x7f0b20196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b1effd640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b201019b0 0x7f0b20196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b1effd640 1 -- 172.21.15.105:0/23855606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2019e660 con 0x7f0b201007c0 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b1effd640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 0x7f0b20196a40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b10005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b07fff640 1 -- 172.21.15.105:0/23855606 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10043020 con 0x7f0b201007c0 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b07fff640 1 -- 172.21.15.105:0/23855606 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b100043d0 con 0x7f0b201007c0 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b07fff640 1 -- 172.21.15.105:0/23855606 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10003bb0 con 0x7f0b201007c0 2024-03-26T17:49:29.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b25722640 1 -- 172.21.15.105:0/23855606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2019e8f0 con 0x7f0b201007c0 2024-03-26T17:49:29.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.387+0000 7f0b25722640 1 -- 172.21.15.105:0/23855606 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2019edf0 con 0x7f0b201007c0 2024-03-26T17:49:29.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.389+0000 7f0b07fff640 1 -- 172.21.15.105:0/23855606 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0b10046020 con 0x7f0b201007c0 2024-03-26T17:49:29.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b283ec740 1 Processor -- start 2024-03-26T17:49:29.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b283ec740 1 -- start start 2024-03-26T17:49:29.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e16637b00 0x561e16637f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e1663cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e1663d660 0x561e1663faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b283ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e1659a980 con 0x561e16637b00 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b283ec740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e16542be0 con 0x561e1663d660 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b283ec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e16596930 con 0x561e166384f0 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e1663cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e16637b00 0x561e16637f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e1663cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38288/0 (socket says 172.21.15.105:38288) 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e16637b00 0x561e16637f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46326/0 (socket says 172.21.15.105:46326) 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b1effd640 1 -- 172.21.15.105:0/2230768713 learned_addr learned my addr 172.21.15.105:0/2230768713 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e1663d660 0x561e1663faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.390+0000 7f0b07fff640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b080a2fa0 0x7f0b080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1f7fe640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b080a2fa0 0x7f0b080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b07fff640 1 -- 172.21.15.105:0/23855606 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0b100f14d0 con 0x7f0b201007c0 2024-03-26T17:49:29.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1effd640 1 -- 172.21.15.105:0/2230768713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e1663d660 msgr2=0x561e1663faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1effd640 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e1663d660 0x561e1663faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1effd640 1 -- 172.21.15.105:0/2230768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e16637b00 msgr2=0x561e16637f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1effd640 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e16637b00 0x561e16637f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1effd640 1 -- 172.21.15.105:0/2230768713 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e163787d0 con 0x561e166384f0 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1f7fe640 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e16637b00 0x561e16637f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e1663d660 0x561e1663faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1effd640 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e1663cff0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0b100bd8c0 tx=0x7f0b10009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda37e8d7b08d3b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b1f7fe640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b080a2fa0 0x7f0b080a5460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x561e16638d50 tx=0x7f0b18004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b057fa640 1 -- 172.21.15.105:0/2230768713 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100bd700 con 0x561e166384f0 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b057fa640 1 -- 172.21.15.105:0/2230768713 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b100be2c0 con 0x561e166384f0 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.391+0000 7f0b057fa640 1 -- 172.21.15.105:0/2230768713 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100bd520 con 0x561e166384f0 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 -- 172.21.15.105:0/2230768713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 msgr2=0x561e1663cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e1663cff0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0b100bd8c0 tx=0x7f0b10009260 comp rx=0 tx=0).stop 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 -- 172.21.15.105:0/2230768713 shutdown_connections 2024-03-26T17:49:29.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e1663d660 0x561e1663faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e1663cff0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2230768713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e16637b00 0x561e16637f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 -- 172.21.15.105:0/2230768713 >> 172.21.15.105:0/2230768713 conn(0x561e16630e60 msgr2=0x561e16631260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 -- 172.21.15.105:0/2230768713 shutdown_connections 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 -- 172.21.15.105:0/2230768713 wait complete. 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 -- start start 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.392+0000 7f0b283ec740 1 Processor -- start 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 0x561e16648fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b283ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e166403d0 con 0x561e166384f0 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b283ec740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e16556160 con 0x561e16637b00 2024-03-26T17:49:29.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b283ec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e16640550 con 0x561e1663d660 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b283ec740 10 client.0 _ll_register_callbacks cb 0x561e16592720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50000/0 (socket says 172.21.15.105:50000) 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1effd640 1 -- 172.21.15.105:0/1230317833 learned_addr learned my addr 172.21.15.105:0/1230317833 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1f7fe640 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 0x561e16648fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/1230317833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 msgr2=0x561e16648fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 0x561e16648fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/1230317833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 msgr2=0x561e16641a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/1230317833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e162ae3f0 con 0x561e166384f0 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1effd640 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:29.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0b20197f50 tx=0x7f0b14012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7396855eefc3e945 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b02ff5640 1 -- 172.21.15.105:0/1230317833 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004030 con 0x561e166384f0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b02ff5640 1 -- 172.21.15.105:0/1230317833 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0b140041d0 con 0x561e166384f0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.393+0000 7f0b283ec740 1 -- 172.21.15.105:0/1230317833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b100be2c0 con 0x561e166384f0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b02ff5640 1 -- 172.21.15.105:0/1230317833 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0b140043d0 con 0x561e166384f0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b283ec740 10 client.19647 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b283ec740 1 -- 172.21.15.105:0/1230317833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561e1626e0f0 con 0x561e166384f0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b02ff5640 1 -- 172.21.15.105:0/1230317833 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b100be2c0 con 0x561e166384f0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b02ff5640 1 -- 172.21.15.105:0/1230317833 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0b14003ab0 con 0x561e166384f0 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b283ec740 10 client.19647 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b283ec740 10 client.19647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b283ec740 20 client.19647 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:29.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b283ec740 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e1664d840 0x561e1664fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b283ec740 4 client.19647 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b283ec740 1 -- 172.21.15.105:0/1230317833 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e165a7ec0 con 0x561e1664d840 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.394+0000 7f0b1effd640 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e1664d840 0x561e1664fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.395+0000 7f0b02ff5640 10 client.19647 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.395+0000 7f0b1effd640 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e1664d840 0x561e1664fcc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef59742bf651e05e server_cookie=cacc97bad0cbabeb in_seq=0 out_seq=0 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b02ff5640 1 -- 172.21.15.105:0/1230317833 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561e165a7ec0 con 0x561e1664d840 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b02ff5640 10 client.19647 handle_command_reply: tid=0 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 client.19647 shutdown 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 -- 172.21.15.105:0/1230317833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 msgr2=0x561e16641f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0b20197f50 tx=0x7f0b14012b60 comp rx=0 tx=0).stop 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 -- 172.21.15.105:0/1230317833 shutdown_connections 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e1664d840 0x561e1664fcc0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 0x561e16648fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b02ff5640 0 client.19647 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:29.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 --2- 172.21.15.105:0/1230317833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 -- 172.21.15.105:0/1230317833 >> 172.21.15.105:0/1230317833 conn(0x561e16630e60 msgr2=0x561e16630080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 -- 172.21.15.105:0/1230317833 shutdown_connections 2024-03-26T17:49:29.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 1 -- 172.21.15.105:0/1230317833 wait complete. 2024-03-26T17:49:29.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.396+0000 7f0b283ec740 20 client.19647 trim_cache size 0 max 16384 2024-03-26T17:49:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b283ec740 1 Processor -- start 2024-03-26T17:49:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b283ec740 1 -- start start 2024-03-26T17:49:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 0x561e16648f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b283ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a6c80 con 0x561e166384f0 2024-03-26T17:49:29.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b283ec740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a87c0 con 0x561e16637b00 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b283ec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a7350 con 0x561e1663d660 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46346/0 (socket says 172.21.15.105:46346) 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/2351600770 learned_addr learned my addr 172.21.15.105:0/2351600770 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1f7fe640 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 0x561e16648f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1effd640 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/2351600770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 msgr2=0x561e16648f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 0x561e16648f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/2351600770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 msgr2=0x561e16641a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.424+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/2351600770 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1663a0a0 con 0x561e166384f0 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.425+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0b14000c00 tx=0x7f0b14009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b83a24b5052d83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.425+0000 7f0b04ff9640 1 -- 172.21.15.105:0/2351600770 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b140040c0 con 0x561e166384f0 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.425+0000 7f0b04ff9640 1 -- 172.21.15.105:0/2351600770 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b14022030 con 0x561e166384f0 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.425+0000 7f0b04ff9640 1 -- 172.21.15.105:0/2351600770 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14003850 con 0x561e166384f0 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.425+0000 7f0b283ec740 1 -- 172.21.15.105:0/2351600770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 msgr2=0x561e16641f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.425+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0b14000c00 tx=0x7f0b14009850 comp rx=0 tx=0).stop 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 -- 172.21.15.105:0/2351600770 shutdown_connections 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e1663d660 0x561e16648f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e166384f0 0x561e16641f70 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2351600770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e16641a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 -- 172.21.15.105:0/2351600770 >> 172.21.15.105:0/2351600770 conn(0x561e16630e60 msgr2=0x561e16631240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 -- 172.21.15.105:0/2351600770 shutdown_connections 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 -- 172.21.15.105:0/2351600770 wait complete. 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 -- start start 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 Processor -- start 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e166463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e166468e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e1663d660 0x561e166478f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a80f0 con 0x561e1663d660 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a7a20 con 0x561e16637b00 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201a4c60 con 0x561e166384f0 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b283ec740 10 client.0 _ll_register_callbacks cb 0x7f0b14009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e1663d660 0x561e166478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b1e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e166468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e166463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b1e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e166468e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38334/0 (socket says 172.21.15.105:38334) 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b1effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e166463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50020/0 (socket says 172.21.15.105:50020) 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/2578485704 learned_addr learned my addr 172.21.15.105:0/2578485704 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.426+0000 7f0b1f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e1663d660 0x561e166478f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46350/0 (socket says 172.21.15.105:46350) 2024-03-26T17:49:29.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/2578485704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 msgr2=0x561e166463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e166463a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/2578485704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e1663d660 msgr2=0x561e166478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e1663d660 0x561e166478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b1e7fc640 1 -- 172.21.15.105:0/2578485704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e16647ee0 con 0x561e166384f0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b1e7fc640 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e166468e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b14003f70 tx=0x7f0b14012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36348b570e7e870c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b02ff5640 1 -- 172.21.15.105:0/2578485704 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14010bb0 con 0x561e166384f0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b02ff5640 1 -- 172.21.15.105:0/2578485704 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0b14010d50 con 0x561e166384f0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b283ec740 1 -- 172.21.15.105:0/2578485704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b14022030 con 0x561e166384f0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b02ff5640 1 -- 172.21.15.105:0/2578485704 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0b140035f0 con 0x561e166384f0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b283ec740 10 client.29437 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.427+0000 7f0b283ec740 1 -- 172.21.15.105:0/2578485704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561e166491f0 con 0x561e166384f0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b02ff5640 1 -- 172.21.15.105:0/2578485704 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b14022030 con 0x561e166384f0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b02ff5640 1 -- 172.21.15.105:0/2578485704 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0b1400fc30 con 0x561e166384f0 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b283ec740 10 client.29437 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:29.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b283ec740 10 client.29437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b283ec740 20 client.29437 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e1664d840 0x561e166b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b283ec740 4 client.29437 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:49:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b283ec740 1 -- 172.21.15.105:0/2578485704 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e165a2210 con 0x561e1664d840 2024-03-26T17:49:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b1effd640 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e1664d840 0x561e166b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.428+0000 7f0b02ff5640 10 client.29437 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:29.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.429+0000 7f0b1effd640 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e1664d840 0x561e166b6160 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f51d5b6ee05d445 server_cookie=c0823f79cc24db93 in_seq=0 out_seq=0 2024-03-26T17:49:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b02ff5640 1 -- 172.21.15.105:0/2578485704 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561e165a2210 con 0x561e1664d840 2024-03-26T17:49:29.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b02ff5640 10 client.29437 handle_command_reply: tid=0 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 client.29437 shutdown 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 -- 172.21.15.105:0/2578485704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 msgr2=0x561e166468e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e166468e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0b14003f70 tx=0x7f0b14012b60 comp rx=0 tx=0).stop 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 -- 172.21.15.105:0/2578485704 shutdown_connections 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e1664d840 0x561e166b6160 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e1663d660 0x561e166478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e166384f0 0x561e166468e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 --2- 172.21.15.105:0/2578485704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e16637b00 0x561e166463a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b02ff5640 0 client.29437 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 -- 172.21.15.105:0/2578485704 >> 172.21.15.105:0/2578485704 conn(0x561e16630e60 msgr2=0x561e16641a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 -- 172.21.15.105:0/2578485704 shutdown_connections 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 1 -- 172.21.15.105:0/2578485704 wait complete. 2024-03-26T17:49:29.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.430+0000 7f0b283ec740 20 client.29437 trim_cache size 0 max 16384 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: "23d0247e-4114-488f-8e1a-47598c044b17": { 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "23d0247e-4114-488f-8e1a-47598c044b17", 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:49:29.429 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:49:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 -- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b080a2fa0 msgr2=0x7f0b080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b080a2fa0 0x7f0b080a5460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x561e16638d50 tx=0x7f0b18004ab0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 -- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 msgr2=0x7f0b20196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 0x7f0b20196a40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000c00 tx=0x7f0b10005e60 comp rx=0 tx=0).stop 2024-03-26T17:49:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 -- 172.21.15.105:0/23855606 shutdown_connections 2024-03-26T17:49:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b080a2fa0 0x7f0b080a5460 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b20102370 0x7f0b2019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b201019b0 0x7f0b20196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 --2- 172.21.15.105:0/23855606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b201007c0 0x7f0b20196a40 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:29.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 -- 172.21.15.105:0/23855606 >> 172.21.15.105:0/23855606 conn(0x7f0b200fbd50 msgr2=0x7f0b20104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:29.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 -- 172.21.15.105:0/23855606 shutdown_connections 2024-03-26T17:49:29.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:29.431+0000 7f0b25722640 1 -- 172.21.15.105:0/23855606 wait complete. 2024-03-26T17:49:29.435 INFO:tasks.cephfs.filesystem:scrub status for tag:23d0247e-4114-488f-8e1a-47598c044b17 - {'path': '/', 'tag': '23d0247e-4114-488f-8e1a-47598c044b17', 'options': 'recursive,force'} 2024-03-26T17:49:30.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 358853 3.14 MB/sec execute 493 sec latency 16.664 ms 2024-03-26T17:49:30.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:29 smithi144 ceph-mon[36095]: pgmap v683: 97 pgs: 97 active+clean; 27 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:49:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:29 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:49:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:29 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:29 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:49:30.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:29 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:29 smithi105 ceph-mon[32280]: pgmap v683: 97 pgs: 97 active+clean; 27 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:49:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:29 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:49:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:29 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:29 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:49:30.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:29 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:29 smithi135 ceph-mon[35636]: pgmap v683: 97 pgs: 97 active+clean; 27 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:49:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:29 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:49:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:29 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:29 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:49:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:29 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:49:31.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 359417 3.14 MB/sec execute 494 sec latency 23.923 ms 2024-03-26T17:49:32.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 360114 3.14 MB/sec execute 495 sec latency 16.797 ms 2024-03-26T17:49:32.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:31 smithi144 ceph-mon[36095]: pgmap v684: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:49:32.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:31 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:49:32.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:31 smithi105 ceph-mon[32280]: pgmap v684: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:49:32.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:31 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:49:32.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:31 smithi135 ceph-mon[35636]: pgmap v684: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-03-26T17:49:32.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:31 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:49:33.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 360745 3.14 MB/sec execute 496 sec latency 15.419 ms 2024-03-26T17:49:34.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 361551 3.15 MB/sec execute 497 sec latency 16.675 ms 2024-03-26T17:49:34.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:33 smithi144 ceph-mon[36095]: pgmap v685: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:49:34.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:33 smithi105 ceph-mon[32280]: pgmap v685: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:49:34.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:33 smithi135 ceph-mon[35636]: pgmap v685: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:49:35.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 362040 3.15 MB/sec execute 498 sec latency 17.153 ms 2024-03-26T17:49:36.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 362413 3.14 MB/sec execute 499 sec latency 15.318 ms 2024-03-26T17:49:36.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:35 smithi144 ceph-mon[36095]: pgmap v686: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:35 smithi105 ceph-mon[32280]: pgmap v686: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:35 smithi135 ceph-mon[35636]: pgmap v686: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:37.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 362978 3.14 MB/sec execute 500 sec latency 23.952 ms 2024-03-26T17:49:38.032 INFO:tasks.workunit.client.0.smithi105.stdout: 1 363659 3.14 MB/sec execute 501 sec latency 17.700 ms 2024-03-26T17:49:38.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:37 smithi144 ceph-mon[36095]: pgmap v687: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:38.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:37 smithi105 ceph-mon[32280]: pgmap v687: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:38.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:37 smithi135 ceph-mon[35636]: pgmap v687: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:39.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 364263 3.14 MB/sec execute 502 sec latency 20.507 ms 2024-03-26T17:49:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 -- 172.21.15.105:0/4175246491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a40102390 msgr2=0x7f5a40108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/4175246491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a40102390 0x7f5a40108c20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ca0 tx=0x7f5a3802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 -- 172.21.15.105:0/4175246491 shutdown_connections 2024-03-26T17:49:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/4175246491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a40102390 0x7f5a40108c20 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/4175246491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a401019d0 0x7f5a40101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/4175246491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a401007e0 0x7f5a40100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 -- 172.21.15.105:0/4175246491 >> 172.21.15.105:0/4175246491 conn(0x7f5a400fbd50 msgr2=0x7f5a400fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 -- 172.21.15.105:0/4175246491 shutdown_connections 2024-03-26T17:49:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.802+0000 7f5a47d7e640 1 -- 172.21.15.105:0/4175246491 wait complete. 2024-03-26T17:49:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a47d7e640 1 Processor -- start 2024-03-26T17:49:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a47d7e640 1 -- start start 2024-03-26T17:49:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a47d7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a401007e0 0x7f5a40199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a47d7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a401019d0 0x7f5a4019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a47d7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40102390 0x7f5a401a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a47d7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010c2f0 con 0x7f5a401007e0 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a45b1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a401007e0 0x7f5a40199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a47d7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010c170 con 0x7f5a401019d0 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a45b1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a401007e0 0x7f5a40199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56820/0 (socket says 172.21.15.105:56820) 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a45b1a640 1 -- 172.21.15.105:0/18195710 learned_addr learned my addr 172.21.15.105:0/18195710 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010c470 con 0x7f5a40102390 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a45319640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a401019d0 0x7f5a4019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.803+0000 7f5a4631b640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40102390 0x7f5a401a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a45b1a640 1 -- 172.21.15.105:0/18195710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40102390 msgr2=0x7f5a401a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40102390 0x7f5a401a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a45b1a640 1 -- 172.21.15.105:0/18195710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a401019d0 msgr2=0x7f5a4019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a401019d0 0x7f5a4019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a45b1a640 1 -- 172.21.15.105:0/18195710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a401a1890 con 0x7f5a401007e0 2024-03-26T17:49:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a401007e0 0x7f5a40199c80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00e9c0 tx=0x7f5a3c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a36ffd640 1 -- 172.21.15.105:0/18195710 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c00cd80 con 0x7f5a401007e0 2024-03-26T17:49:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a36ffd640 1 -- 172.21.15.105:0/18195710 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a3c0035f0 con 0x7f5a401007e0 2024-03-26T17:49:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a401a1b80 con 0x7f5a401007e0 2024-03-26T17:49:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a36ffd640 1 -- 172.21.15.105:0/18195710 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c010690 con 0x7f5a401007e0 2024-03-26T17:49:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.804+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401a20d0 con 0x7f5a401007e0 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.806+0000 7f5a36ffd640 1 -- 172.21.15.105:0/18195710 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5a3c0037a0 con 0x7f5a401007e0 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.807+0000 7f5a4a96e740 1 Processor -- start 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.807+0000 7f5a36ffd640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a2c0a2fc0 0x7f5a2c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.807+0000 7f5a4a96e740 1 -- start start 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.807+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9e5d0 0x5635bde9e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9efc0 0x5635bdea3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bdea4130 0x5635bdea65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5635bde030b0 con 0x5635bdea4130 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5635bddab960 con 0x5635bde9efc0 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5635bddff060 con 0x5635bde9e5d0 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4631b640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a2c0a2fc0 0x7f5a2c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a36ffd640 1 -- 172.21.15.105:0/18195710 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f5a3c0cf830 con 0x7f5a401007e0 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9e5d0 0x5635bde9e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45319640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9efc0 0x5635bdea3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9e5d0 0x5635bde9e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49166/0 (socket says 172.21.15.105:49166) 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4631b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bdea4130 0x5635bdea65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45319640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9efc0 0x5635bdea3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59766/0 (socket says 172.21.15.105:59766) 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 -- 172.21.15.105:0/1840105405 learned_addr learned my addr 172.21.15.105:0/1840105405 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 -- 172.21.15.105:0/1840105405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9efc0 msgr2=0x5635bdea3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/1840105405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9efc0 0x5635bdea3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 -- 172.21.15.105:0/1840105405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bdea4130 msgr2=0x5635bdea65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/1840105405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bdea4130 0x5635bdea65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 -- 172.21.15.105:0/1840105405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635bddbdc60 con 0x5635bde9e5d0 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4631b640 1 --2- 172.21.15.105:0/1840105405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bdea4130 0x5635bdea65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a4631b640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a2c0a2fc0 0x7f5a2c0a5480 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a38006010 tx=0x7f5a38032ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/1840105405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9e5d0 0x5635bde9e9f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c098860 tx=0x7f5a3c09bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2147a9554a86183 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.808+0000 7f5a2bfff640 1 -- 172.21.15.105:0/1840105405 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c003b30 con 0x5635bde9e5d0 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a2bfff640 1 -- 172.21.15.105:0/1840105405 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a3c09c620 con 0x5635bde9e5d0 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a2bfff640 1 -- 172.21.15.105:0/1840105405 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c098500 con 0x5635bde9e5d0 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a4a96e740 1 -- 172.21.15.105:0/1840105405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9e5d0 msgr2=0x5635bde9e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/1840105405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9e5d0 0x5635bde9e9f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c098860 tx=0x7f5a3c09bdb0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a4a96e740 1 -- 172.21.15.105:0/1840105405 shutdown_connections 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/1840105405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bdea4130 0x5635bdea65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/1840105405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9efc0 0x5635bdea3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/1840105405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9e5d0 0x5635bde9e9f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.809+0000 7f5a4a96e740 1 -- 172.21.15.105:0/1840105405 >> 172.21.15.105:0/1840105405 conn(0x5635bde99b40 msgr2=0x5635bde99f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 -- 172.21.15.105:0/1840105405 shutdown_connections 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 -- 172.21.15.105:0/1840105405 wait complete. 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 -- start start 2024-03-26T17:49:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 Processor -- start 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9e5d0 0x5635bdea8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9efc0 0x5635bdea88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bdea4130 0x5635bdeaf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5635bdea70e0 con 0x5635bde9e5d0 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5635bddbedb0 con 0x5635bdea4130 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.810+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5635bdea7260 con 0x5635bde9efc0 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a4a96e740 10 client.0 _ll_register_callbacks cb 0x5635bddc3110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45b1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9e5d0 0x5635bdea8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45319640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9efc0 0x5635bdea88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45b1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9e5d0 0x5635bdea8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56832/0 (socket says 172.21.15.105:56832) 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45319640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9efc0 0x5635bdea88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49172/0 (socket says 172.21.15.105:49172) 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45b1a640 1 -- 172.21.15.105:0/2327372502 learned_addr learned my addr 172.21.15.105:0/2327372502 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a4631b640 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bdea4130 0x5635bdeaf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45319640 1 -- 172.21.15.105:0/2327372502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bdea4130 msgr2=0x5635bdeaf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45319640 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bdea4130 0x5635bdeaf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45319640 1 -- 172.21.15.105:0/2327372502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9e5d0 msgr2=0x5635bdea8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45319640 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9e5d0 0x5635bdea8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45319640 1 -- 172.21.15.105:0/2327372502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635bdb17210 con 0x5635bde9efc0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a45319640 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9efc0 0x5635bdea88d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5635bde9f3c0 tx=0x7f5a300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a3a41a4c48baf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a297fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a297fa640 1 -- 172.21.15.105:0/2327372502 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30004020 con 0x5635bde9efc0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a297fa640 1 -- 172.21.15.105:0/2327372502 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f5a300041c0 con 0x5635bde9efc0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a4a96e740 1 -- 172.21.15.105:0/2327372502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3c09c620 con 0x5635bde9efc0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a297fa640 1 -- 172.21.15.105:0/2327372502 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a300043c0 con 0x5635bde9efc0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.811+0000 7f5a4a96e740 10 client.29449 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a4a96e740 1 -- 172.21.15.105:0/2327372502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5635bdbe3620 con 0x5635bde9efc0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a297fa640 1 -- 172.21.15.105:0/2327372502 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a3c09c620 con 0x5635bde9efc0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a297fa640 1 -- 172.21.15.105:0/2327372502 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5a30004ca0 con 0x5635bde9efc0 2024-03-26T17:49:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a4a96e740 10 client.29449 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a4a96e740 10 client.29449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a4a96e740 20 client.29449 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5635bdeb4650 0x5635bdeb6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a4a96e740 4 client.29449 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a4a96e740 1 -- 172.21.15.105:0/2327372502 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635bde105f0 con 0x5635bdeb4650 2024-03-26T17:49:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.812+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5635bdeb4650 0x5635bdeb6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.813+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5635bdeb4650 0x5635bdeb6af0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f793a5301e5b3b4 server_cookie=55f43e03d45757ec in_seq=0 out_seq=0 2024-03-26T17:49:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.813+0000 7f5a297fa640 10 client.29449 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.813+0000 7f5a297fa640 1 -- 172.21.15.105:0/2327372502 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5635bde105f0 con 0x5635bdeb4650 2024-03-26T17:49:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.813+0000 7f5a297fa640 10 client.29449 handle_command_reply: tid=0 2024-03-26T17:49:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.813+0000 7f5a4a96e740 1 client.29449 shutdown 2024-03-26T17:49:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 -- 172.21.15.105:0/2327372502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9efc0 msgr2=0x5635bdea88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9efc0 0x5635bdea88d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5635bde9f3c0 tx=0x7f5a300118d0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 -- 172.21.15.105:0/2327372502 shutdown_connections 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5635bdeb4650 0x5635bdeb6af0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bdea4130 0x5635bdeaf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bde9efc0 0x5635bdea88d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a297fa640 0 client.29449 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/2327372502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9e5d0 0x5635bdea8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 -- 172.21.15.105:0/2327372502 >> 172.21.15.105:0/2327372502 conn(0x5635bde99b40 msgr2=0x5635bde98df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 -- 172.21.15.105:0/2327372502 shutdown_connections 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 1 -- 172.21.15.105:0/2327372502 wait complete. 2024-03-26T17:49:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.814+0000 7f5a4a96e740 20 client.29449 trim_cache size 0 max 16384 2024-03-26T17:49:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4a96e740 1 Processor -- start 2024-03-26T17:49:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4a96e740 1 -- start start 2024-03-26T17:49:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdea3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeb6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 0x5635bdeaf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ab040 con 0x5635bde9efc0 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401acb80 con 0x5635bde9e5d0 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ab710 con 0x5635bdea4130 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45319640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeb6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45319640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeb6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56838/0 (socket says 172.21.15.105:56838) 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a4631b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 0x5635bdeaf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45319640 1 -- 172.21.15.105:0/4192603964 learned_addr learned my addr 172.21.15.105:0/4192603964 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/4192603964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdea3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45319640 1 -- 172.21.15.105:0/4192603964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 msgr2=0x5635bdeaf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45319640 1 --2- 172.21.15.105:0/4192603964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 0x5635bdeaf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45319640 1 -- 172.21.15.105:0/4192603964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 msgr2=0x5635bdea3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45319640 1 --2- 172.21.15.105:0/4192603964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdea3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.832+0000 7f5a45319640 1 -- 172.21.15.105:0/4192603964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635bde95640 con 0x5635bde9efc0 2024-03-26T17:49:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.833+0000 7f5a45319640 1 --2- 172.21.15.105:0/4192603964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeb6a70 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005d00 tx=0x7f5a30004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f0bcbf922d4982 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.833+0000 7f5a2b7fe640 1 -- 172.21.15.105:0/4192603964 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30031070 con 0x5635bde9efc0 2024-03-26T17:49:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.833+0000 7f5a2b7fe640 1 -- 172.21.15.105:0/4192603964 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a30010e10 con 0x5635bde9efc0 2024-03-26T17:49:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.833+0000 7f5a2b7fe640 1 -- 172.21.15.105:0/4192603964 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30028040 con 0x5635bde9efc0 2024-03-26T17:49:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- 172.21.15.105:0/4192603964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 msgr2=0x5635bdeb6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/4192603964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeb6a70 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005d00 tx=0x7f5a30004560 comp rx=0 tx=0).stop 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- 172.21.15.105:0/4192603964 shutdown_connections 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/4192603964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 0x5635bdeaf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/4192603964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeb6a70 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/4192603964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdea3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- 172.21.15.105:0/4192603964 >> 172.21.15.105:0/4192603964 conn(0x5635bde99b40 msgr2=0x5635bdea3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- 172.21.15.105:0/4192603964 shutdown_connections 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- 172.21.15.105:0/4192603964 wait complete. 2024-03-26T17:49:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- start start 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 Processor -- start 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdeac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeacd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 0x5635bdf18a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401ac4b0 con 0x5635bde9efc0 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401abde0 con 0x5635bde9e5d0 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5a401a9020 con 0x5635bdea4130 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.834+0000 7f5a4a96e740 10 client.0 _ll_register_callbacks cb 0x7f5a3000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdeac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45319640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeacd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdeac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59794/0 (socket says 172.21.15.105:59794) 2024-03-26T17:49:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 -- 172.21.15.105:0/3838793505 learned_addr learned my addr 172.21.15.105:0/3838793505 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a4631b640 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 0x5635bdf18a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 -- 172.21.15.105:0/3838793505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 msgr2=0x5635bdf18a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 0x5635bdf18a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 -- 172.21.15.105:0/3838793505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 msgr2=0x5635bdeacd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeacd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 -- 172.21.15.105:0/3838793505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635bdf18f60 con 0x5635bde9e5d0 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45319640 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeacd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a45b1a640 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdeac850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0cd660 tx=0x7f5a3c0a4930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b60b33775a95cef7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a297fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a297fa640 1 -- 172.21.15.105:0/3838793505 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c09c620 con 0x5635bde9e5d0 2024-03-26T17:49:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a297fa640 1 -- 172.21.15.105:0/3838793505 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f5a3c055160 con 0x5635bde9e5d0 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.835+0000 7f5a4a96e740 1 -- 172.21.15.105:0/3838793505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a30010e10 con 0x5635bde9e5d0 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a297fa640 1 -- 172.21.15.105:0/3838793505 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c0a0070 con 0x5635bde9e5d0 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a4a96e740 10 client.29474 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a4a96e740 1 -- 172.21.15.105:0/3838793505 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5635bdf1a230 con 0x5635bde9e5d0 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a297fa640 1 -- 172.21.15.105:0/3838793505 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a3c098500 con 0x5635bde9e5d0 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a297fa640 1 -- 172.21.15.105:0/3838793505 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5a3c0b7070 con 0x5635bde9e5d0 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a4a96e740 10 client.29474 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a4a96e740 10 client.29474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a4a96e740 20 client.29474 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5635bdeb4650 0x5635bdf1c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a4a96e740 4 client.29474 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a4a96e740 1 -- 172.21.15.105:0/3838793505 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5a30010e10 con 0x5635bdeb4650 2024-03-26T17:49:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.836+0000 7f5a45319640 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5635bdeb4650 0x5635bdf1c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.837+0000 7f5a45319640 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5635bdeb4650 0x5635bdf1c750 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b359fdd964ec7288 server_cookie=cf120dd2184e5cf in_seq=0 out_seq=0 2024-03-26T17:49:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.837+0000 7f5a297fa640 10 client.29474 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a297fa640 1 -- 172.21.15.105:0/3838793505 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5a30010e10 con 0x5635bdeb4650 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a297fa640 10 client.29474 handle_command_reply: tid=0 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 client.29474 shutdown 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 -- 172.21.15.105:0/3838793505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 msgr2=0x5635bdeac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdeac850 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0cd660 tx=0x7f5a3c0a4930 comp rx=0 tx=0).stop 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 -- 172.21.15.105:0/3838793505 shutdown_connections 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5635bdeb4650 0x5635bdf1c750 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635bdea4130 0x5635bdf18a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635bde9efc0 0x5635bdeacd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a297fa640 0 client.29474 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 --2- 172.21.15.105:0/3838793505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635bde9e5d0 0x5635bdeac850 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 -- 172.21.15.105:0/3838793505 >> 172.21.15.105:0/3838793505 conn(0x5635bde99b40 msgr2=0x5635bde9cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 -- 172.21.15.105:0/3838793505 shutdown_connections 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 1 -- 172.21.15.105:0/3838793505 wait complete. 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.880+0000 7f5a4a96e740 20 client.29474 trim_cache size 0 max 16384 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:49:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:49:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:49:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a2c0a2fc0 msgr2=0x7f5a2c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a2c0a2fc0 0x7f5a2c0a5480 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a38006010 tx=0x7f5a38032ec0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a401007e0 msgr2=0x7f5a40199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a401007e0 0x7f5a40199c80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00e9c0 tx=0x7f5a3c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 shutdown_connections 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5a2c0a2fc0 0x7f5a2c0a5480 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a40102390 0x7f5a401a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5a401019d0 0x7f5a4019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 --2- 172.21.15.105:0/18195710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a401007e0 0x7f5a40199c80 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 >> 172.21.15.105:0/18195710 conn(0x7f5a400fbd50 msgr2=0x7f5a400fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 shutdown_connections 2024-03-26T17:49:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:39.881+0000 7f5a47d7e640 1 -- 172.21.15.105:0/18195710 wait complete. 2024-03-26T17:49:40.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 365080 3.15 MB/sec execute 503 sec latency 19.560 ms 2024-03-26T17:49:40.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:39 smithi105 ceph-mon[32280]: pgmap v688: 97 pgs: 97 active+clean; 31 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:40.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:39 smithi135 ceph-mon[35636]: pgmap v688: 97 pgs: 97 active+clean; 31 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:40.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:39 smithi144 ceph-mon[36095]: pgmap v688: 97 pgs: 97 active+clean; 31 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-03-26T17:49:41.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 365556 3.15 MB/sec execute 504 sec latency 16.614 ms 2024-03-26T17:49:42.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 365928 3.14 MB/sec execute 505 sec latency 14.662 ms 2024-03-26T17:49:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:41 smithi105 ceph-mon[32280]: pgmap v689: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:49:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:41 smithi135 ceph-mon[35636]: pgmap v689: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:49:42.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:41 smithi144 ceph-mon[36095]: pgmap v689: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.6 MiB/s wr, 111 op/s 2024-03-26T17:49:43.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 366505 3.14 MB/sec execute 506 sec latency 23.404 ms 2024-03-26T17:49:44.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 367169 3.14 MB/sec execute 507 sec latency 16.652 ms 2024-03-26T17:49:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:43 smithi105 ceph-mon[32280]: pgmap v690: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:49:44.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:43 smithi135 ceph-mon[35636]: pgmap v690: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:49:44.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:43 smithi144 ceph-mon[36095]: pgmap v690: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.6 MiB/s wr, 117 op/s 2024-03-26T17:49:45.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 367773 3.14 MB/sec execute 508 sec latency 18.246 ms 2024-03-26T17:49:46.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 368623 3.15 MB/sec execute 509 sec latency 16.169 ms 2024-03-26T17:49:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:45 smithi105 ceph-mon[32280]: pgmap v691: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:49:46.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:45 smithi135 ceph-mon[35636]: pgmap v691: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:49:46.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:45 smithi144 ceph-mon[36095]: pgmap v691: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:49:47.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 369115 3.15 MB/sec execute 510 sec latency 21.024 ms 2024-03-26T17:49:48.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 369497 3.14 MB/sec execute 511 sec latency 16.605 ms 2024-03-26T17:49:48.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:47 smithi105 ceph-mon[32280]: pgmap v692: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:47 smithi135 ceph-mon[35636]: pgmap v692: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:47 smithi144 ceph-mon[36095]: pgmap v692: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:49.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 370062 3.14 MB/sec execute 512 sec latency 24.534 ms 2024-03-26T17:49:50.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 370729 3.14 MB/sec execute 513 sec latency 16.312 ms 2024-03-26T17:49:50.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:49 smithi105 ceph-mon[32280]: pgmap v693: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:49:50.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:49 smithi135 ceph-mon[35636]: pgmap v693: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:49:50.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:49 smithi144 ceph-mon[36095]: pgmap v693: 97 pgs: 97 active+clean; 28 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:49:51.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 371352 3.14 MB/sec execute 514 sec latency 16.033 ms 2024-03-26T17:49:52.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 372179 3.15 MB/sec execute 515 sec latency 16.805 ms 2024-03-26T17:49:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:51 smithi105 ceph-mon[32280]: pgmap v694: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:49:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:51 smithi135 ceph-mon[35636]: pgmap v694: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:49:52.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:51 smithi144 ceph-mon[36095]: pgmap v694: 97 pgs: 97 active+clean; 30 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.6 MiB/s wr, 103 op/s 2024-03-26T17:49:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 -- 172.21.15.105:0/263416594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c102390 msgr2=0x7f824c108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 --2- 172.21.15.105:0/263416594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c102390 0x7f824c108c20 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f8244009ca0 tx=0x7f824402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 -- 172.21.15.105:0/263416594 shutdown_connections 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 --2- 172.21.15.105:0/263416594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c102390 0x7f824c108c20 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 --2- 172.21.15.105:0/263416594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f824c1019d0 0x7f824c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 --2- 172.21.15.105:0/263416594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824c1007e0 0x7f824c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 -- 172.21.15.105:0/263416594 >> 172.21.15.105:0/263416594 conn(0x7f824c0fbd50 msgr2=0x7f824c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 -- 172.21.15.105:0/263416594 shutdown_connections 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.976+0000 7f82512ea640 1 -- 172.21.15.105:0/263416594 wait complete. 2024-03-26T17:49:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f82512ea640 1 Processor -- start 2024-03-26T17:49:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f82512ea640 1 -- start start 2024-03-26T17:49:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f82512ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c1007e0 0x7f824c069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f82512ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824c1019d0 0x7f824c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f82512ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f824c102390 0x7f824c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f82512ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f824c10c2f0 con 0x7f824c1007e0 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f82512ea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f824c10c170 con 0x7f824c102390 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f82512ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f824c10c470 con 0x7f824c1019d0 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c1007e0 0x7f824c069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824c1019d0 0x7f824c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c1007e0 0x7f824c069ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42924/0 (socket says 172.21.15.105:42924) 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824c1019d0 0x7f824c19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46242/0 (socket says 172.21.15.105:46242) 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824affd640 1 -- 172.21.15.105:0/1294635530 learned_addr learned my addr 172.21.15.105:0/1294635530 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824b7fe640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f824c102390 0x7f824c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824a7fc640 1 -- 172.21.15.105:0/1294635530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f824c102390 msgr2=0x7f824c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824a7fc640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f824c102390 0x7f824c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824a7fc640 1 -- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c1007e0 msgr2=0x7f824c069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824a7fc640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c1007e0 0x7f824c069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.977+0000 7f824a7fc640 1 -- 172.21.15.105:0/1294635530 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c196510 con 0x7f824c1019d0 2024-03-26T17:49:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.978+0000 7f824a7fc640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824c1019d0 0x7f824c19c740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f824000e9f0 tx=0x7f824000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.978+0000 7f8233fff640 1 -- 172.21.15.105:0/1294635530 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000ccc0 con 0x7f824c1019d0 2024-03-26T17:49:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.978+0000 7f8233fff640 1 -- 172.21.15.105:0/1294635530 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82400108e0 con 0x7f824c1019d0 2024-03-26T17:49:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.978+0000 7f82512ea640 1 -- 172.21.15.105:0/1294635530 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c196800 con 0x7f824c1019d0 2024-03-26T17:49:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.978+0000 7f8233fff640 1 -- 172.21.15.105:0/1294635530 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240018880 con 0x7f824c1019d0 2024-03-26T17:49:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.978+0000 7f82512ea640 1 -- 172.21.15.105:0/1294635530 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c196d50 con 0x7f824c1019d0 2024-03-26T17:49:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8233fff640 1 -- 172.21.15.105:0/1294635530 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8240010a90 con 0x7f824c1019d0 2024-03-26T17:49:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8253fb5740 1 Processor -- start 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8253fb5740 1 -- start start 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8253fb5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c2320b0 con 0x55bf0c2d3130 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8253fb5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c1da960 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.981+0000 7f8253fb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c22e060 con 0x55bf0c2cd5d0 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f8233fff640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82340a2fc0 0x7f82340a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57820/0 (socket says 172.21.15.105:57820) 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 -- 172.21.15.105:0/126325180 learned_addr learned my addr 172.21.15.105:0/126325180 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46244/0 (socket says 172.21.15.105:46244) 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824b7fe640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82340a2fc0 0x7f82340a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f8233fff640 1 -- 172.21.15.105:0/1294635530 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f82400cf760 con 0x7f824c1019d0 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 -- 172.21.15.105:0/126325180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cd5d0 msgr2=0x55bf0c2cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 --2- 172.21.15.105:0/126325180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 -- 172.21.15.105:0/126325180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 msgr2=0x55bf0c2d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 --2- 172.21.15.105:0/126325180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 -- 172.21.15.105:0/126325180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0c1ecc60 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824b7fe640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82340a2fc0 0x7f82340a5480 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f824402c630 tx=0x7f8244009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f824affd640 1 --2- 172.21.15.105:0/126325180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d2ac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f823c009d80 tx=0x7f823c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d68a0fd6aa0912f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.982+0000 7f82317fa640 1 -- 172.21.15.105:0/126325180 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c01e070 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f82317fa640 1 -- 172.21.15.105:0/126325180 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f823c019ae0 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f82317fa640 1 -- 172.21.15.105:0/126325180 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c00e990 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 -- 172.21.15.105:0/126325180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 msgr2=0x55bf0c2d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 --2- 172.21.15.105:0/126325180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d2ac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f823c009d80 tx=0x7f823c011810 comp rx=0 tx=0).stop 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 -- 172.21.15.105:0/126325180 shutdown_connections 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 --2- 172.21.15.105:0/126325180 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 --2- 172.21.15.105:0/126325180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d2ac0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 --2- 172.21.15.105:0/126325180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 -- 172.21.15.105:0/126325180 >> 172.21.15.105:0/126325180 conn(0x55bf0c2c8b40 msgr2=0x55bf0c2c8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 -- 172.21.15.105:0/126325180 shutdown_connections 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.983+0000 7f8253fb5740 1 -- 172.21.15.105:0/126325180 wait complete. 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 1 -- start start 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 1 Processor -- start 2024-03-26T17:49:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c2d60e0 con 0x55bf0c2d3130 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c1eddb0 con 0x55bf0c2cd5d0 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf0c2d6260 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f8253fb5740 10 client.0 _ll_register_callbacks cb 0x55bf0c1f2110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2de960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42944/0 (socket says 172.21.15.105:42944) 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824b7fe640 1 -- 172.21.15.105:0/3344102247 learned_addr learned my addr 172.21.15.105:0/3344102247 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824a7fc640 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824affd640 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824a7fc640 1 -- 172.21.15.105:0/3344102247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 msgr2=0x55bf0c2d7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824a7fc640 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2d7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824a7fc640 1 -- 172.21.15.105:0/3344102247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 msgr2=0x55bf0c2de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824a7fc640 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2de960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824a7fc640 1 -- 172.21.15.105:0/3344102247 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0bf46210 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.984+0000 7f824b7fe640 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f824a7fc640 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d7990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f82400cd270 tx=0x7f824006aaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52a7183ee5b69a97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f822eff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f822eff5640 1 -- 172.21.15.105:0/3344102247 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400a45d0 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f822eff5640 1 -- 172.21.15.105:0/3344102247 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8240010430 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 1 -- 172.21.15.105:0/3344102247 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823c019ae0 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f822eff5640 1 -- 172.21.15.105:0/3344102247 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f824009bf00 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 10 client.29467 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 1 -- 172.21.15.105:0/3344102247 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf0c012620 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f822eff5640 1 -- 172.21.15.105:0/3344102247 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82400b6030 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f822eff5640 1 -- 172.21.15.105:0/3344102247 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8240097de0 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 10 client.29467 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 10 client.29467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 20 client.29467 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf0c2e3300 0x55bf0c2e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 4 client.29467 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.985+0000 7f8253fb5740 1 -- 172.21.15.105:0/3344102247 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf0c23f5f0 con 0x55bf0c2e3300 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.986+0000 7f824affd640 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf0c2e3300 0x55bf0c2e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.986+0000 7f822eff5640 10 client.29467 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.986+0000 7f824affd640 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf0c2e3300 0x55bf0c2e56f0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b9ad735771b7f7 server_cookie=92979899f2eb4c6d in_seq=0 out_seq=0 2024-03-26T17:49:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f822eff5640 1 -- 172.21.15.105:0/3344102247 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bf0c23f5f0 con 0x55bf0c2e3300 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f822eff5640 10 client.29467 handle_command_reply: tid=0 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 client.29467 shutdown 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 -- 172.21.15.105:0/3344102247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 msgr2=0x55bf0c2d7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d7990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f82400cd270 tx=0x7f824006aaf0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 -- 172.21.15.105:0/3344102247 shutdown_connections 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf0c2e3300 0x55bf0c2e56f0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2d7990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f822eff5640 0 client.29467 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3344102247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2d7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 -- 172.21.15.105:0/3344102247 >> 172.21.15.105:0/3344102247 conn(0x55bf0c2c8b40 msgr2=0x55bf0c2c7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 -- 172.21.15.105:0/3344102247 shutdown_connections 2024-03-26T17:49:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 1 -- 172.21.15.105:0/3344102247 wait complete. 2024-03-26T17:49:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:52.987+0000 7f8253fb5740 20 client.29467 trim_cache size 0 max 16384 2024-03-26T17:49:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f8253fb5740 1 Processor -- start 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f8253fb5740 1 -- start start 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f8253fb5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1aaf40 con 0x55bf0c2d3130 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f8253fb5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1aca80 con 0x55bf0c2cd5d0 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f8253fb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1ab610 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.005+0000 7f824a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2de920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46264/0 (socket says 172.21.15.105:46264) 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824a7fc640 1 -- 172.21.15.105:0/3287268372 learned_addr learned my addr 172.21.15.105:0/3287268372 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824b7fe640 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824affd640 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824a7fc640 1 -- 172.21.15.105:0/3287268372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 msgr2=0x55bf0c2cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824a7fc640 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824a7fc640 1 -- 172.21.15.105:0/3287268372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 msgr2=0x55bf0c2cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824a7fc640 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824a7fc640 1 -- 172.21.15.105:0/3287268372 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0c2c4450 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824affd640 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824b7fe640 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f824a7fc640 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2de920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82400cd590 tx=0x7f824009a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e583283e791b69 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.006+0000 7f8230ff9640 1 -- 172.21.15.105:0/3287268372 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824009afb0 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.007+0000 7f8230ff9640 1 -- 172.21.15.105:0/3287268372 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82400b6070 con 0x55bf0c2cdfc0 2024-03-26T17:49:52.999 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.007+0000 7f8230ff9640 1 -- 172.21.15.105:0/3287268372 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824009b350 con 0x55bf0c2cdfc0 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.007+0000 7f8253fb5740 1 -- 172.21.15.105:0/3287268372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 msgr2=0x55bf0c2de920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.007+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2de920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f82400cd590 tx=0x7f824009a410 comp rx=0 tx=0).stop 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 -- 172.21.15.105:0/3287268372 shutdown_connections 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2d3130 0x55bf0c2cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2de920 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 --2- 172.21.15.105:0/3287268372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 -- 172.21.15.105:0/3287268372 >> 172.21.15.105:0/3287268372 conn(0x55bf0c2c8b40 msgr2=0x55bf0c2d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 -- 172.21.15.105:0/3287268372 shutdown_connections 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 -- 172.21.15.105:0/3287268372 wait complete. 2024-03-26T17:49:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 -- start start 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 Processor -- start 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2db890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2d3130 0x55bf0c347620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1ac3b0 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1abce0 con 0x55bf0c2cdfc0 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1a8f20 con 0x55bf0c2d3130 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f8253fb5740 10 client.0 _ll_register_callbacks cb 0x7f824000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f824affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f824a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f824affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2db890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42964/0 (socket says 172.21.15.105:42964) 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f824a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57850/0 (socket says 172.21.15.105:57850) 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f824affd640 1 -- 172.21.15.105:0/111175360 learned_addr learned my addr 172.21.15.105:0/111175360 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.008+0000 7f824b7fe640 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2d3130 0x55bf0c347620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f824affd640 1 -- 172.21.15.105:0/111175360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2d3130 msgr2=0x55bf0c347620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f824affd640 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2d3130 0x55bf0c347620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f824affd640 1 -- 172.21.15.105:0/111175360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 msgr2=0x55bf0c2dbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f824affd640 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2dbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f824affd640 1 -- 172.21.15.105:0/111175360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0c347b60 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f824a7fc640 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2dbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f824affd640 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2db890 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f823c01c040 tx=0x7f823c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7777f3036a08cc4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f822eff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f822eff5640 1 -- 172.21.15.105:0/111175360 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c01e040 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f822eff5640 1 -- 172.21.15.105:0/111175360 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f823c004220 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f8253fb5740 1 -- 172.21.15.105:0/111175360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82400b6070 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f822eff5640 1 -- 172.21.15.105:0/111175360 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f823c03b050 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f8253fb5740 10 client.19686 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f8253fb5740 1 -- 172.21.15.105:0/111175360 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bf0c348de0 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.009+0000 7f822eff5640 1 -- 172.21.15.105:0/111175360 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82400b6070 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f822eff5640 1 -- 172.21.15.105:0/111175360 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f823c005040 con 0x55bf0c2cd5d0 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f8253fb5740 10 client.19686 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f8253fb5740 10 client.19686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f8253fb5740 20 client.19686 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f8253fb5740 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf0c349600 0x55bf0c34ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f8253fb5740 4 client.19686 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f8253fb5740 1 -- 172.21.15.105:0/111175360 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf0c239940 con 0x55bf0c349600 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f824a7fc640 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf0c349600 0x55bf0c34ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f824a7fc640 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf0c349600 0x55bf0c34ba20 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8a8814f9b1d8d0 server_cookie=7464bca531a8f7fa in_seq=0 out_seq=0 2024-03-26T17:49:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.010+0000 7f822eff5640 10 client.19686 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:53.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 372645 3.15 MB/sec execute 516 sec latency 18.570 ms 2024-03-26T17:49:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f822eff5640 1 -- 172.21.15.105:0/111175360 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf0c239940 con 0x55bf0c349600 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f822eff5640 10 client.19686 handle_command_reply: tid=0 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 client.19686 shutdown 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 -- 172.21.15.105:0/111175360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2cd5d0 msgr2=0x55bf0c2db890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2db890 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f823c01c040 tx=0x7f823c009c80 comp rx=0 tx=0).stop 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 -- 172.21.15.105:0/111175360 shutdown_connections 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf0c349600 0x55bf0c34ba20 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f822eff5640 0 client.19686 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf0c2d3130 0x55bf0c347620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf0c2cdfc0 0x55bf0c2dbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 --2- 172.21.15.105:0/111175360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf0c2cd5d0 0x55bf0c2db890 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 -- 172.21.15.105:0/111175360 >> 172.21.15.105:0/111175360 conn(0x55bf0c2c8b40 msgr2=0x55bf0c2d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 -- 172.21.15.105:0/111175360 shutdown_connections 2024-03-26T17:49:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 1 -- 172.21.15.105:0/111175360 wait complete. 2024-03-26T17:49:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.056+0000 7f8253fb5740 20 client.19686 trim_cache size 0 max 16384 2024-03-26T17:49:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:49:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:49:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:49:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:49:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 -- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82340a2fc0 msgr2=0x7f82340a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82340a2fc0 0x7f82340a5480 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f824402c630 tx=0x7f8244009360 comp rx=0 tx=0).stop 2024-03-26T17:49:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 -- 172.21.15.105:0/1294635530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824c1019d0 msgr2=0x7f824c19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824c1019d0 0x7f824c19c740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f824000e9f0 tx=0x7f824000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 -- 172.21.15.105:0/1294635530 shutdown_connections 2024-03-26T17:49:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82340a2fc0 0x7f82340a5480 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f824c102390 0x7f824c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f824c1019d0 0x7f824c19c740 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 --2- 172.21.15.105:0/1294635530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f824c1007e0 0x7f824c069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 -- 172.21.15.105:0/1294635530 >> 172.21.15.105:0/1294635530 conn(0x7f824c0fbd50 msgr2=0x7f824c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 -- 172.21.15.105:0/1294635530 shutdown_connections 2024-03-26T17:49:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:49:53.057+0000 7f82512ea640 1 -- 172.21.15.105:0/1294635530 wait complete. 2024-03-26T17:49:54.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 373005 3.14 MB/sec execute 517 sec latency 28.140 ms 2024-03-26T17:49:54.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:53 smithi105 ceph-mon[32280]: pgmap v695: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:54.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:53 smithi135 ceph-mon[35636]: pgmap v695: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:54.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:53 smithi144 ceph-mon[36095]: pgmap v695: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:55.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 373563 3.14 MB/sec execute 518 sec latency 21.166 ms 2024-03-26T17:49:56.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 374259 3.14 MB/sec execute 519 sec latency 19.972 ms 2024-03-26T17:49:56.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:55 smithi105 ceph-mon[32280]: pgmap v696: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:49:56.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:55 smithi135 ceph-mon[35636]: pgmap v696: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:49:56.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:55 smithi144 ceph-mon[36095]: pgmap v696: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-03-26T17:49:57.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 374865 3.14 MB/sec execute 520 sec latency 17.430 ms 2024-03-26T17:49:58.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 375702 3.15 MB/sec execute 521 sec latency 16.428 ms 2024-03-26T17:49:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:57 smithi105 ceph-mon[32280]: pgmap v697: 97 pgs: 97 active+clean; 31 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:58.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:57 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:49:58.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:57 smithi135 ceph-mon[35636]: pgmap v697: 97 pgs: 97 active+clean; 31 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:58.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:57 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:49:58.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:57 smithi144 ceph-mon[36095]: pgmap v697: 97 pgs: 97 active+clean; 31 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:49:58.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:57 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:49:59.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 376091 3.15 MB/sec execute 522 sec latency 17.255 ms 2024-03-26T17:49:59.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:49:59.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 -- 172.21.15.105:0/27004606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8218102370 msgr2=0x7f8218068a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 --2- 172.21.15.105:0/27004606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8218102370 0x7f8218068a20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8210009ca0 tx=0x7f821002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 -- 172.21.15.105:0/27004606 shutdown_connections 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 --2- 172.21.15.105:0/27004606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8218102370 0x7f8218068a20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 --2- 172.21.15.105:0/27004606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82181019b0 0x7f8218101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 --2- 172.21.15.105:0/27004606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82181007c0 0x7f8218100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 -- 172.21.15.105:0/27004606 >> 172.21.15.105:0/27004606 conn(0x7f82180fbd50 msgr2=0x7f82180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 -- 172.21.15.105:0/27004606 shutdown_connections 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.571+0000 7f82200ac640 1 -- 172.21.15.105:0/27004606 wait complete. 2024-03-26T17:49:59.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f82200ac640 1 Processor -- start 2024-03-26T17:49:59.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f82200ac640 1 -- start start 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f82200ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82181007c0 0x7f8218108770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f82200ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82181019b0 0x7f8218106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f82200ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218102370 0x7f82181072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f82200ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f821810d3f0 con 0x7f8218102370 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f82200ac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f821810d270 con 0x7f82181007c0 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f82200ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f821810d570 con 0x7f82181019b0 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821de48640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82181007c0 0x7f8218108770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821e649640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218102370 0x7f82181072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821de48640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82181007c0 0x7f8218108770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37016/0 (socket says 172.21.15.105:37016) 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821de48640 1 -- 172.21.15.105:0/2728355674 learned_addr learned my addr 172.21.15.105:0/2728355674 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821d647640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82181019b0 0x7f8218106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821de48640 1 -- 172.21.15.105:0/2728355674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82181019b0 msgr2=0x7f8218106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821de48640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82181019b0 0x7f8218106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821de48640 1 -- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218102370 msgr2=0x7f82181072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821de48640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218102370 0x7f82181072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.572+0000 7f821de48640 1 -- 172.21.15.105:0/2728355674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8218107b80 con 0x7f82181007c0 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.573+0000 7f821de48640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82181007c0 0x7f8218108770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f821400e9c0 tx=0x7f821400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.573+0000 7f820effd640 1 -- 172.21.15.105:0/2728355674 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821400cd80 con 0x7f82181007c0 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.573+0000 7f820effd640 1 -- 172.21.15.105:0/2728355674 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82140035f0 con 0x7f82181007c0 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.573+0000 7f820effd640 1 -- 172.21.15.105:0/2728355674 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214010690 con 0x7f82181007c0 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.573+0000 7f82200ac640 1 -- 172.21.15.105:0/2728355674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821806b340 con 0x7f82181007c0 2024-03-26T17:49:59.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.573+0000 7f82200ac640 1 -- 172.21.15.105:0/2728355674 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821806b890 con 0x7f82181007c0 2024-03-26T17:49:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.576+0000 7f820effd640 1 -- 172.21.15.105:0/2728355674 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f82140037a0 con 0x7f82181007c0 2024-03-26T17:49:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.576+0000 7f8222d72740 1 Processor -- start 2024-03-26T17:49:59.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.576+0000 7f8222d72740 1 -- start start 2024-03-26T17:49:59.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f820effd640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82040a2fa0 0x7f82040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f8222d72740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd68f0 0x5630f7fd6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f8222d72740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fdbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f8222d72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fde8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f8222d72740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5630f7f3b980 con 0x5630f7fd72e0 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f8222d72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5630f7ee3be0 con 0x5630f7fdc450 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f8222d72740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5630f7f37930 con 0x5630f7fd68f0 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f821d647640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fdbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f821d647640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fdbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43960/0 (socket says 172.21.15.105:43960) 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f821e649640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fde8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f821de48640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd68f0 0x5630f7fd6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f821d647640 1 -- 172.21.15.105:0/1161730998 learned_addr learned my addr 172.21.15.105:0/1161730998 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f821e649640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fde8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37028/0 (socket says 172.21.15.105:37028) 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.577+0000 7f821e649640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82040a2fa0 0x7f82040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821e649640 1 -- 172.21.15.105:0/1161730998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd68f0 msgr2=0x5630f7fd6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821e649640 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd68f0 0x5630f7fd6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821e649640 1 -- 172.21.15.105:0/1161730998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 msgr2=0x5630f7fdbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f820effd640 1 -- 172.21.15.105:0/2728355674 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f82140cf910 con 0x7f82181007c0 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821e649640 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fdbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821e649640 1 -- 172.21.15.105:0/1161730998 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f7d197d0 con 0x5630f7fdc450 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821de48640 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd68f0 0x5630f7fd6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821d647640 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fdbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821e649640 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fde8e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f821002c630 tx=0x7f82100328a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d10bf3a20b23b9bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f821e649640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82040a2fa0 0x7f82040a5460 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8210005e30 tx=0x7f8210042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f8203fff640 1 -- 172.21.15.105:0/1161730998 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210051070 con 0x5630f7fdc450 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f8203fff640 1 -- 172.21.15.105:0/1161730998 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8210004d40 con 0x5630f7fdc450 2024-03-26T17:49:59.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.578+0000 7f8203fff640 1 -- 172.21.15.105:0/1161730998 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210039040 con 0x5630f7fdc450 2024-03-26T17:49:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 -- 172.21.15.105:0/1161730998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 msgr2=0x5630f7fde8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fde8e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f821002c630 tx=0x7f82100328a0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 -- 172.21.15.105:0/1161730998 shutdown_connections 2024-03-26T17:49:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fde8e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fdbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 --2- 172.21.15.105:0/1161730998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd68f0 0x5630f7fd6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 -- 172.21.15.105:0/1161730998 >> 172.21.15.105:0/1161730998 conn(0x5630f7fd1e60 msgr2=0x5630f7fd2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 -- 172.21.15.105:0/1161730998 shutdown_connections 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.579+0000 7f8222d72740 1 -- 172.21.15.105:0/1161730998 wait complete. 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 1 -- start start 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 1 Processor -- start 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd72e0 0x5630f7fe10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fe80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5630f7fdf3a0 con 0x5630f7fd68f0 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5630f7ef7160 con 0x5630f7fdc450 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5630f7fdf520 con 0x5630f7fd72e0 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f8222d72740 10 client.0 _ll_register_callbacks cb 0x5630f7f33720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f821de48640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f821d647640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd72e0 0x5630f7fe10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f821de48640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43970/0 (socket says 172.21.15.105:43970) 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f821de48640 1 -- 172.21.15.105:0/1007756083 learned_addr learned my addr 172.21.15.105:0/1007756083 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.580+0000 7f821e649640 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fe80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f821de48640 1 -- 172.21.15.105:0/1007756083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd72e0 msgr2=0x5630f7fe10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f821de48640 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd72e0 0x5630f7fe10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f821de48640 1 -- 172.21.15.105:0/1007756083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 msgr2=0x5630f7fe80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f821de48640 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fe80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f821de48640 1 -- 172.21.15.105:0/1007756083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f7c4f3f0 con 0x5630f7fd68f0 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f821d647640 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd72e0 0x5630f7fe10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f821de48640 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0b60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8214009fd0 tx=0x7f8214003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a7bdfc40b8903d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f82017fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f82017fa640 1 -- 172.21.15.105:0/1007756083 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821409c000 con 0x5630f7fd68f0 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f8222d72740 1 -- 172.21.15.105:0/1007756083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8210004d40 con 0x5630f7fd68f0 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f82017fa640 1 -- 172.21.15.105:0/1007756083 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8214096b70 con 0x5630f7fd68f0 2024-03-26T17:49:59.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f82017fa640 1 -- 172.21.15.105:0/1007756083 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f82140a0150 con 0x5630f7fd68f0 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f8222d72740 10 client.19698 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.581+0000 7f8222d72740 1 -- 172.21.15.105:0/1007756083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5630f7c0f0f0 con 0x5630f7fd68f0 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f82017fa640 1 -- 172.21.15.105:0/1007756083 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8210004d40 con 0x5630f7fd68f0 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f82017fa640 1 -- 172.21.15.105:0/1007756083 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8214097dc0 con 0x5630f7fd68f0 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f8222d72740 10 client.19698 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f8222d72740 10 client.19698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f8222d72740 20 client.19698 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f8222d72740 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630f7fec9c0 0x5630f7feee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f8222d72740 4 client.19698 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f8222d72740 1 -- 172.21.15.105:0/1007756083 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630f7f48ec0 con 0x5630f7fec9c0 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.582+0000 7f821d647640 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630f7fec9c0 0x5630f7feee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.583+0000 7f82017fa640 10 client.19698 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.583+0000 7f821d647640 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630f7fec9c0 0x5630f7feee30 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e7d1e8f96cf4906 server_cookie=cc2f5d12c254c0c7 in_seq=0 out_seq=0 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f82017fa640 1 -- 172.21.15.105:0/1007756083 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5630f7f48ec0 con 0x5630f7fec9c0 2024-03-26T17:49:59.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f82017fa640 10 client.19698 handle_command_reply: tid=0 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f8222d72740 1 client.19698 shutdown 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f8222d72740 1 -- 172.21.15.105:0/1007756083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd68f0 msgr2=0x5630f7fe0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f8222d72740 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0b60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f8214009fd0 tx=0x7f8214003d00 comp rx=0 tx=0).stop 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f8222d72740 1 -- 172.21.15.105:0/1007756083 shutdown_connections 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f8222d72740 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630f7fec9c0 0x5630f7feee30 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f8222d72740 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fdc450 0x5630f7fe80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f8222d72740 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fd72e0 0x5630f7fe10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.584+0000 7f8222d72740 1 --2- 172.21.15.105:0/1007756083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0b60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.585+0000 7f8222d72740 1 -- 172.21.15.105:0/1007756083 >> 172.21.15.105:0/1007756083 conn(0x5630f7fd1e60 msgr2=0x5630f7fd1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.585+0000 7f82017fa640 0 client.19698 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.585+0000 7f8222d72740 1 -- 172.21.15.105:0/1007756083 shutdown_connections 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.585+0000 7f8222d72740 1 -- 172.21.15.105:0/1007756083 wait complete. 2024-03-26T17:49:59.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.585+0000 7f8222d72740 20 client.19698 trim_cache size 0 max 16384 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f8222d72740 1 Processor -- start 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f8222d72740 1 -- start start 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f8222d72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f8222d72740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f8222d72740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f7fe0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f8222d72740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82181a2760 con 0x5630f7fd72e0 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f8222d72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82181a42a0 con 0x5630f7fd68f0 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f8222d72740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82181a2e30 con 0x5630f7fdc450 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43986/0 (socket says 172.21.15.105:43986) 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 -- 172.21.15.105:0/2123371165 learned_addr learned my addr 172.21.15.105:0/2123371165 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821de48640 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821e649640 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f7fe0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 -- 172.21.15.105:0/2123371165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 msgr2=0x5630f7fe0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f7fe0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 -- 172.21.15.105:0/2123371165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 msgr2=0x5630f7fe0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 -- 172.21.15.105:0/2123371165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f7fcd8d0 con 0x5630f7fd72e0 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821e649640 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f7fe0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.604+0000 7f821d647640 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe8070 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8208004700 tx=0x7f8208005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedc129c49274e4c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f82037fe640 1 -- 172.21.15.105:0/2123371165 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208031270 con 0x5630f7fd72e0 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f82037fe640 1 -- 172.21.15.105:0/2123371165 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8208021e40 con 0x5630f7fd72e0 2024-03-26T17:49:59.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f82037fe640 1 -- 172.21.15.105:0/2123371165 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208030df0 con 0x5630f7fd72e0 2024-03-26T17:49:59.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:59 smithi144 ceph-mon[36095]: pgmap v698: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:49:59.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:49:59.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:49:59.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:49:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 -- 172.21.15.105:0/2123371165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 msgr2=0x5630f7fe8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe8070 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f8208004700 tx=0x7f8208005f90 comp rx=0 tx=0).stop 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 -- 172.21.15.105:0/2123371165 shutdown_connections 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f7fe0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe8070 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 --2- 172.21.15.105:0/2123371165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 -- 172.21.15.105:0/2123371165 >> 172.21.15.105:0/2123371165 conn(0x5630f7fd1e60 msgr2=0x5630f7fee9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 -- 172.21.15.105:0/2123371165 shutdown_connections 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.605+0000 7f8222d72740 1 -- 172.21.15.105:0/2123371165 wait complete. 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 1 -- start start 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 1 Processor -- start 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f8050cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82181a3bd0 con 0x5630f7fd72e0 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82181a3500 con 0x5630f7fd68f0 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8218072720 con 0x5630f7fdc450 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f8222d72740 10 client.0 _ll_register_callbacks cb 0x7f8214097c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f821de48640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f821de48640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37038/0 (socket says 172.21.15.105:37038) 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f821de48640 1 -- 172.21.15.105:0/2105029631 learned_addr learned my addr 172.21.15.105:0/2105029631 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f821e649640 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f8050cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.606+0000 7f821d647640 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f821de48640 1 -- 172.21.15.105:0/2105029631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 msgr2=0x5630f8050cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f821de48640 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f8050cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f821de48640 1 -- 172.21.15.105:0/2105029631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 msgr2=0x5630f7fe55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f821de48640 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f821de48640 1 -- 172.21.15.105:0/2105029631 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f80512e0 con 0x5630f7fd68f0 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f821de48640 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe5090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f821409a2b0 tx=0x7f8214097f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b30fb9594e67253c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:59.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f82017fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f82017fa640 1 -- 172.21.15.105:0/2105029631 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214077060 con 0x5630f7fd68f0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f82017fa640 1 -- 172.21.15.105:0/2105029631 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f821409b450 con 0x5630f7fd68f0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f8222d72740 1 -- 172.21.15.105:0/2105029631 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5630f7f43210 con 0x5630f7fd68f0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f82017fa640 1 -- 172.21.15.105:0/2105029631 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f821409c000 con 0x5630f7fd68f0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f8222d72740 10 client.29510 fetch_fsmap learned FSMap version 32 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.607+0000 7f8222d72740 1 -- 172.21.15.105:0/2105029631 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5630f8052660 con 0x5630f7fd68f0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f82017fa640 1 -- 172.21.15.105:0/2105029631 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5630f7f43210 con 0x5630f7fd68f0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f82017fa640 1 -- 172.21.15.105:0/2105029631 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8214004000 con 0x5630f7fd68f0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f8222d72740 10 client.29510 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f8222d72740 10 client.29510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f8222d72740 20 client.29510 populate_metadata read hostname 'smithi105' 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f8222d72740 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630f8052e50 0x5630f8055270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f8222d72740 4 client.29510 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f8222d72740 1 -- 172.21.15.105:0/2105029631 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630f7ec0540 con 0x5630f8052e50 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f821d647640 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630f8052e50 0x5630f8055270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f821d647640 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630f8052e50 0x5630f8055270 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3664c8c94c0bae9 server_cookie=5cc6b82c81b16820 in_seq=0 out_seq=0 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.608+0000 7f82017fa640 10 client.29510 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.610+0000 7f82017fa640 1 -- 172.21.15.105:0/2105029631 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5630f7ec0540 con 0x5630f8052e50 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.610+0000 7f82017fa640 10 client.29510 handle_command_reply: tid=0 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.610+0000 7f8222d72740 1 client.29510 shutdown 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.610+0000 7f8222d72740 1 -- 172.21.15.105:0/2105029631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 msgr2=0x5630f7fe5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.610+0000 7f8222d72740 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe5090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f821409a2b0 tx=0x7f8214097f80 comp rx=0 tx=0).stop 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 1 -- 172.21.15.105:0/2105029631 shutdown_connections 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5630f8052e50 0x5630f8055270 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5630f7fdc450 0x5630f8050cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5630f7fd72e0 0x5630f7fe55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 1 --2- 172.21.15.105:0/2105029631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5630f7fd68f0 0x5630f7fe5090 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 1 -- 172.21.15.105:0/2105029631 >> 172.21.15.105:0/2105029631 conn(0x5630f7fd1e60 msgr2=0x5630f7fee9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82017fa640 0 client.29510 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:49:59.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 1 -- 172.21.15.105:0/2105029631 shutdown_connections 2024-03-26T17:49:59.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 1 -- 172.21.15.105:0/2105029631 wait complete. 2024-03-26T17:49:59.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f8222d72740 20 client.29510 trim_cache size 0 max 16384 2024-03-26T17:49:59.607 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:49:59.607 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:49:59.607 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:49:59.607 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:49:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 -- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82040a2fa0 msgr2=0x7f82040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82040a2fa0 0x7f82040a5460 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8210005e30 tx=0x7f8210042000 comp rx=0 tx=0).stop 2024-03-26T17:49:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 -- 172.21.15.105:0/2728355674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82181007c0 msgr2=0x7f8218108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82181007c0 0x7f8218108770 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f821400e9c0 tx=0x7f821400ee80 comp rx=0 tx=0).stop 2024-03-26T17:49:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 -- 172.21.15.105:0/2728355674 shutdown_connections 2024-03-26T17:49:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82040a2fa0 0x7f82040a5460 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8218102370 0x7f82181072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82181019b0 0x7f8218106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 --2- 172.21.15.105:0/2728355674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f82181007c0 0x7f8218108770 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 -- 172.21.15.105:0/2728355674 >> 172.21.15.105:0/2728355674 conn(0x7f82180fbd50 msgr2=0x7f8218102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:59.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 -- 172.21.15.105:0/2728355674 shutdown_connections 2024-03-26T17:49:59.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.611+0000 7f82200ac640 1 -- 172.21.15.105:0/2728355674 wait complete. 2024-03-26T17:49:59.613 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.613 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.613 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:59 smithi105 ceph-mon[32280]: pgmap v698: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:49:59.613 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:49:59.613 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:49:59.613 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:49:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:49:59.614 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:49:59.665 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.665 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.665 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:59 smithi135 ceph-mon[35636]: pgmap v698: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:49:59.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:49:59.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:49:59.666 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:49:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:49:59.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 -- 172.21.15.105:0/2432401610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97901014b0 msgr2=0x7f9790101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 --2- 172.21.15.105:0/2432401610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97901014b0 0x7f9790101930 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f9788009ca0 tx=0x7f978802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 -- 172.21.15.105:0/2432401610 shutdown_connections 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 --2- 172.21.15.105:0/2432401610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97901014b0 0x7f9790101930 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 --2- 172.21.15.105:0/2432401610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9790100b10 0x7f9790100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 --2- 172.21.15.105:0/2432401610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97901001b0 0x7f97901005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 -- 172.21.15.105:0/2432401610 >> 172.21.15.105:0/2432401610 conn(0x7f97900fbd50 msgr2=0x7f97900fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 -- 172.21.15.105:0/2432401610 shutdown_connections 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 -- 172.21.15.105:0/2432401610 wait complete. 2024-03-26T17:49:59.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 Processor -- start 2024-03-26T17:49:59.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.746+0000 7f9796548640 1 -- start start 2024-03-26T17:49:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9796548640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97901001b0 0x7f9790199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9796548640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9790100b10 0x7f979019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9796548640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97901014b0 0x7f97901a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9796548640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f979010f5e0 con 0x7f9790100b10 2024-03-26T17:49:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9796548640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f979010f460 con 0x7f97901014b0 2024-03-26T17:49:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9796548640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f979010f760 con 0x7f97901001b0 2024-03-26T17:49:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f978f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9790100b10 0x7f979019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f978f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9790100b10 0x7f979019a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44008/0 (socket says 172.21.15.105:44008) 2024-03-26T17:49:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f978f7fe640 1 -- 172.21.15.105:0/3550419780 learned_addr learned my addr 172.21.15.105:0/3550419780 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:49:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9794ae5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97901014b0 0x7f97901a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f978ffff640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97901001b0 0x7f9790199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9794ae5640 1 -- 172.21.15.105:0/3550419780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97901001b0 msgr2=0x7f9790199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9794ae5640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97901001b0 0x7f9790199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9794ae5640 1 -- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9790100b10 msgr2=0x7f979019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:49:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9794ae5640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9790100b10 0x7f979019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:49:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f9794ae5640 1 -- 172.21.15.105:0/3550419780 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97901a18f0 con 0x7f97901014b0 2024-03-26T17:49:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f978ffff640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97901001b0 0x7f9790199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:49:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.747+0000 7f978f7fe640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9790100b10 0x7f979019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:49:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.748+0000 7f9794ae5640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97901014b0 0x7f97901a1250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f978802c630 tx=0x7f97880328a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.748+0000 7f978d7fa640 1 -- 172.21.15.105:0/3550419780 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788041020 con 0x7f97901014b0 2024-03-26T17:49:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.748+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97901a1b80 con 0x7f97901014b0 2024-03-26T17:49:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.748+0000 7f978d7fa640 1 -- 172.21.15.105:0/3550419780 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9788003e20 con 0x7f97901014b0 2024-03-26T17:49:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.748+0000 7f978d7fa640 1 -- 172.21.15.105:0/3550419780 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788039040 con 0x7f97901014b0 2024-03-26T17:49:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.748+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97901a20c0 con 0x7f97901014b0 2024-03-26T17:49:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.749+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979010b090 con 0x7f97901014b0 2024-03-26T17:49:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.750+0000 7f978d7fa640 1 -- 172.21.15.105:0/3550419780 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9788044020 con 0x7f97901014b0 2024-03-26T17:49:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.751+0000 7f978d7fa640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97780a2fa0 0x7f97780a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:49:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.751+0000 7f978ffff640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97780a2fa0 0x7f97780a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:49:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.751+0000 7f978d7fa640 1 -- 172.21.15.105:0/3550419780 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97880f1710 con 0x7f97901014b0 2024-03-26T17:49:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.752+0000 7f978ffff640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97780a2fa0 0x7f97780a5460 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f979019ab80 tx=0x7f9780005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:49:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:49:59.759+0000 7f978d7fa640 1 -- 172.21.15.105:0/3550419780 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f978804b020 con 0x7f97901014b0 2024-03-26T17:50:00.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 376489 3.14 MB/sec execute 523 sec latency 15.708 ms 2024-03-26T17:50:00.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.065+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f979002d240 con 0x7f97901014b0 2024-03-26T17:50:00.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.066+0000 7f978d7fa640 1 -- 172.21.15.105:0/3550419780 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11941 (secure 0 0 0) 0x7f97880be500 con 0x7f97901014b0 2024-03-26T17:50:00.057 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:50:00.058 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:49:14.826136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24482,14628,24487,24466]},"id":1}]} 2024-03-26T17:50:00.058 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 32 2024-03-26T17:50:00.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97780a2fa0 msgr2=0x7f97780a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97780a2fa0 0x7f97780a5460 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f979019ab80 tx=0x7f9780005f90 comp rx=0 tx=0).stop 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97901014b0 msgr2=0x7f97901a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97901014b0 0x7f97901a1250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f978802c630 tx=0x7f97880328a0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 shutdown_connections 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97780a2fa0 0x7f97780a5460 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97901014b0 0x7f97901a1250 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9790100b10 0x7f979019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 --2- 172.21.15.105:0/3550419780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97901001b0 0x7f9790199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 >> 172.21.15.105:0/3550419780 conn(0x7f97900fbd50 msgr2=0x7f97900fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 shutdown_connections 2024-03-26T17:50:00.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.069+0000 7f9796548640 1 -- 172.21.15.105:0/3550419780 wait complete. 2024-03-26T17:50:00.074 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:50:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 -- 172.21.15.105:0/57226757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b81007c0 msgr2=0x7fe8b8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/57226757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b81007c0 0x7fe8b8100be0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009900 tx=0x7fe8a802bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 -- 172.21.15.105:0/57226757 shutdown_connections 2024-03-26T17:50:00.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/57226757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8102370 0x7fe8b8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/57226757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8b81019b0 0x7fe8b8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/57226757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b81007c0 0x7fe8b8100be0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 -- 172.21.15.105:0/57226757 >> 172.21.15.105:0/57226757 conn(0x7fe8b80fbd50 msgr2=0x7fe8b80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 -- 172.21.15.105:0/57226757 shutdown_connections 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.196+0000 7fe8be8f8640 1 -- 172.21.15.105:0/57226757 wait complete. 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8be8f8640 1 Processor -- start 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8be8f8640 1 -- start start 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8be8f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b81007c0 0x7fe8b8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8be8f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8b81019b0 0x7fe8b819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8be8f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b8102370 0x7fe8b819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8be8f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810c2c0 con 0x7fe8b8102370 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8be8f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810c140 con 0x7fe8b81019b0 2024-03-26T17:50:00.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8be8f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b810c440 con 0x7fe8b81007c0 2024-03-26T17:50:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8bce95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b8102370 0x7fe8b819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8bce95640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b8102370 0x7fe8b819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44016/0 (socket says 172.21.15.105:44016) 2024-03-26T17:50:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8bce95640 1 -- 172.21.15.105:0/3446944587 learned_addr learned my addr 172.21.15.105:0/3446944587 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b81007c0 0x7fe8b8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8b81019b0 0x7fe8b819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8bce95640 1 -- 172.21.15.105:0/3446944587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b81007c0 msgr2=0x7fe8b8069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8bce95640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b81007c0 0x7fe8b8069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8bce95640 1 -- 172.21.15.105:0/3446944587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8b81019b0 msgr2=0x7fe8b819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8bce95640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8b81019b0 0x7fe8b819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.197+0000 7fe8bce95640 1 -- 172.21.15.105:0/3446944587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b819f7b0 con 0x7fe8b8102370 2024-03-26T17:50:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.198+0000 7fe8bce95640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b8102370 0x7fe8b819eea0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000eb10 tx=0x7fe8b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.198+0000 7fe8b57fa640 1 -- 172.21.15.105:0/3446944587 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0004b20 con 0x7fe8b8102370 2024-03-26T17:50:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.198+0000 7fe8b57fa640 1 -- 172.21.15.105:0/3446944587 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8b0004cc0 con 0x7fe8b8102370 2024-03-26T17:50:00.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.198+0000 7fe8be8f8640 1 -- 172.21.15.105:0/3446944587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b819fa40 con 0x7fe8b8102370 2024-03-26T17:50:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.198+0000 7fe8b57fa640 1 -- 172.21.15.105:0/3446944587 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0010690 con 0x7fe8b8102370 2024-03-26T17:50:00.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.198+0000 7fe8be8f8640 1 -- 172.21.15.105:0/3446944587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b819ff40 con 0x7fe8b8102370 2024-03-26T17:50:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8c15be740 1 Processor -- start 2024-03-26T17:50:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8b57fa640 1 -- 172.21.15.105:0/3446944587 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe8b0005020 con 0x7fe8b8102370 2024-03-26T17:50:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8c15be740 1 -- start start 2024-03-26T17:50:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284b9a30 0x5616284b9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284ba420 0x5616284bef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284bf590 0x5616284c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8c15be740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56162841e970 con 0x5616284bf590 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8c15be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5616283c6d70 con 0x5616284b9a30 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8c15be740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56162841a920 con 0x5616284ba420 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.201+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284bf590 0x5616284c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284bf590 0x5616284c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44032/0 (socket says 172.21.15.105:44032) 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b77fe640 1 -- 172.21.15.105:0/1447709777 learned_addr learned my addr 172.21.15.105:0/1447709777 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284ba420 0x5616284bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284b9a30 0x5616284b9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b57fa640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8a00a2f20 0x7fe8a00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b77fe640 1 -- 172.21.15.105:0/1447709777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284ba420 msgr2=0x5616284bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284ba420 0x5616284bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b77fe640 1 -- 172.21.15.105:0/1447709777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284b9a30 msgr2=0x5616284b9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284b9a30 0x5616284b9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b77fe640 1 -- 172.21.15.105:0/1447709777 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616281323f0 con 0x5616284bf590 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284ba420 0x5616284bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8bce95640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8a00a2f20 0x7fe8a00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.202+0000 7fe8b57fa640 1 -- 172.21.15.105:0/3446944587 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe8b00cf980 con 0x7fe8b8102370 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.203+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284bf590 0x5616284c1a20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009fd0 tx=0x7fe8ac005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fae13221c1d4628 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.203+0000 7fe8bce95640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8a00a2f20 0x7fe8a00a53e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe8b009c000 tx=0x7fe8b006bb90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.203+0000 7fe89e7fc640 1 -- 172.21.15.105:0/1447709777 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac004030 con 0x5616284bf590 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.203+0000 7fe89e7fc640 1 -- 172.21.15.105:0/1447709777 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ac00f790 con 0x5616284bf590 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.203+0000 7fe89e7fc640 1 -- 172.21.15.105:0/1447709777 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0038a0 con 0x5616284bf590 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 -- 172.21.15.105:0/1447709777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284bf590 msgr2=0x5616284c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284bf590 0x5616284c1a20 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009fd0 tx=0x7fe8ac005850 comp rx=0 tx=0).stop 2024-03-26T17:50:00.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 -- 172.21.15.105:0/1447709777 shutdown_connections 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284bf590 0x5616284c1a20 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284ba420 0x5616284bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 --2- 172.21.15.105:0/1447709777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284b9a30 0x5616284b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 -- 172.21.15.105:0/1447709777 >> 172.21.15.105:0/1447709777 conn(0x5616284b4fa0 msgr2=0x5616284b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 -- 172.21.15.105:0/1447709777 shutdown_connections 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 -- 172.21.15.105:0/1447709777 wait complete. 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 -- start start 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 Processor -- start 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284b9a30 0x5616284c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.204+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284ba420 0x5616284c4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284bf590 0x5616284cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8c15be740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5616284c2540 con 0x5616284ba420 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8c15be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5616283da310 con 0x5616284bf590 2024-03-26T17:50:00.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8c15be740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616284c26c0 con 0x5616284b9a30 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284ba420 0x5616284c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284b9a30 0x5616284c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8c15be740 10 client.0 _ll_register_callbacks cb 0x561628415c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284ba420 0x5616284c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44044/0 (socket says 172.21.15.105:44044) 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284b9a30 0x5616284c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33858/0 (socket says 172.21.15.105:33858) 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 -- 172.21.15.105:0/2064489369 learned_addr learned my addr 172.21.15.105:0/2064489369 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284bf590 0x5616284cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 -- 172.21.15.105:0/2064489369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284b9a30 msgr2=0x5616284c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284b9a30 0x5616284c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 -- 172.21.15.105:0/2064489369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284bf590 msgr2=0x5616284cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284bf590 0x5616284cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 -- 172.21.15.105:0/2064489369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616281fc7d0 con 0x5616284ba420 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284b9a30 0x5616284c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284ba420 0x5616284c4420 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009230 tx=0x7fe8a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8510bc068710016a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8977fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8977fe640 1 -- 172.21.15.105:0/2064489369 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8005d50 con 0x5616284ba420 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8977fe640 1 -- 172.21.15.105:0/2064489369 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe8a8005030 con 0x5616284ba420 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.205+0000 7fe8c15be740 1 -- 172.21.15.105:0/2064489369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ac00f790 con 0x5616284ba420 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8977fe640 1 -- 172.21.15.105:0/2064489369 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8a8051050 con 0x5616284ba420 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8c15be740 10 client.19722 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8c15be740 1 -- 172.21.15.105:0/2064489369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5616284ce840 con 0x5616284ba420 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8977fe640 1 -- 172.21.15.105:0/2064489369 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ac00f790 con 0x5616284ba420 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8977fe640 1 -- 172.21.15.105:0/2064489369 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe8a80035f0 con 0x5616284ba420 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8c15be740 10 client.19722 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8c15be740 10 client.19722 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8c15be740 20 client.19722 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8c15be740 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5616284cffd0 0x5616284d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8c15be740 4 client.19722 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:00.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8c15be740 1 -- 172.21.15.105:0/2064489369 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162842beb0 con 0x5616284cffd0 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.206+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5616284cffd0 0x5616284d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8977fe640 10 client.19722 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5616284cffd0 0x5616284d2410 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce0fb5d71a9af901 server_cookie=30faa405ea633146 in_seq=0 out_seq=0 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8977fe640 1 -- 172.21.15.105:0/2064489369 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56162842beb0 con 0x5616284cffd0 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8977fe640 10 client.19722 handle_command_reply: tid=0 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 client.19722 shutdown 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 -- 172.21.15.105:0/2064489369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284ba420 msgr2=0x5616284c4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284ba420 0x5616284c4420 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009230 tx=0x7fe8a8004270 comp rx=0 tx=0).stop 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 -- 172.21.15.105:0/2064489369 shutdown_connections 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5616284cffd0 0x5616284d2410 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284bf590 0x5616284cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8977fe640 0 client.19722 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284ba420 0x5616284c4420 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 --2- 172.21.15.105:0/2064489369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284b9a30 0x5616284c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.207+0000 7fe8c15be740 1 -- 172.21.15.105:0/2064489369 >> 172.21.15.105:0/2064489369 conn(0x5616284b4fa0 msgr2=0x5616284b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.208+0000 7fe8c15be740 1 -- 172.21.15.105:0/2064489369 shutdown_connections 2024-03-26T17:50:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.208+0000 7fe8c15be740 1 -- 172.21.15.105:0/2064489369 wait complete. 2024-03-26T17:50:00.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.208+0000 7fe8c15be740 20 client.19722 trim_cache size 0 max 16384 2024-03-26T17:50:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8c15be740 1 Processor -- start 2024-03-26T17:50:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8c15be740 1 -- start start 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 0x5616284b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8c15be740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81a8d60 con 0x5616284b9a30 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8c15be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81aa8a0 con 0x5616284ba420 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8c15be740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81a9430 con 0x5616284bf590 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44060/0 (socket says 172.21.15.105:44060) 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8b77fe640 1 -- 172.21.15.105:0/3215611978 learned_addr learned my addr 172.21.15.105:0/3215611978 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.222+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 0x5616284b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe8b77fe640 1 -- 172.21.15.105:0/3215611978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 msgr2=0x5616284b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 0x5616284b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe8b77fe640 1 -- 172.21.15.105:0/3215611978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 msgr2=0x5616284cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe8b77fe640 1 -- 172.21.15.105:0/3215611978 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616284ba800 con 0x5616284b9a30 2024-03-26T17:50:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284b3930 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0117f0 tx=0x7fe8ac005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44e08a8ecff6fee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe89dffb640 1 -- 172.21.15.105:0/3215611978 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00f790 con 0x5616284b9a30 2024-03-26T17:50:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe89dffb640 1 -- 172.21.15.105:0/3215611978 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ac02f030 con 0x5616284b9a30 2024-03-26T17:50:00.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.223+0000 7fe89dffb640 1 -- 172.21.15.105:0/3215611978 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0035f0 con 0x5616284b9a30 2024-03-26T17:50:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 -- 172.21.15.105:0/3215611978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 msgr2=0x5616284b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284b3930 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0117f0 tx=0x7fe8ac005850 comp rx=0 tx=0).stop 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 -- 172.21.15.105:0/3215611978 shutdown_connections 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 0x5616284b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 --2- 172.21.15.105:0/3215611978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284b3930 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 -- 172.21.15.105:0/3215611978 >> 172.21.15.105:0/3215611978 conn(0x5616284b4fa0 msgr2=0x5616284bb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 -- 172.21.15.105:0/3215611978 shutdown_connections 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 -- 172.21.15.105:0/3215611978 wait complete. 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.224+0000 7fe8c15be740 1 -- start start 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8c15be740 1 Processor -- start 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8c15be740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 0x5616284c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8c15be740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81aa1d0 con 0x5616284b9a30 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8c15be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81a9b00 con 0x5616284ba420 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8c15be740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81a6d40 con 0x5616284bf590 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8c15be740 10 client.0 _ll_register_callbacks cb 0x7fe8a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44064/0 (socket says 172.21.15.105:44064) 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 -- 172.21.15.105:0/289577171 learned_addr learned my addr 172.21.15.105:0/289577171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 0x5616284c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 -- 172.21.15.105:0/289577171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 msgr2=0x5616284c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 0x5616284c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 -- 172.21.15.105:0/289577171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 msgr2=0x5616284c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 -- 172.21.15.105:0/289577171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616284ca300 con 0x5616284b9a30 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8b77fe640 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284c87c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0058c0 tx=0x7fe8ac009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89399e3c26da6f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.225+0000 7fe8977fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8977fe640 1 -- 172.21.15.105:0/289577171 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac035040 con 0x5616284b9a30 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8977fe640 1 -- 172.21.15.105:0/289577171 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe8ac005450 con 0x5616284b9a30 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 1 -- 172.21.15.105:0/289577171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ac02f030 con 0x5616284b9a30 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8977fe640 1 -- 172.21.15.105:0/289577171 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe8ac005650 con 0x5616284b9a30 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 10 client.19728 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 1 -- 172.21.15.105:0/289577171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5616284cb600 con 0x5616284b9a30 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8977fe640 1 -- 172.21.15.105:0/289577171 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8ac02f030 con 0x5616284b9a30 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8977fe640 1 -- 172.21.15.105:0/289577171 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe8ac01fab0 con 0x5616284b9a30 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 10 client.19728 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 10 client.19728 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 20 client.19728 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5616285366f0 0x561628538ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 4 client.19728 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.226+0000 7fe8c15be740 1 -- 172.21.15.105:0/289577171 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561628426200 con 0x5616285366f0 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5616285366f0 0x561628538ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8977fe640 10 client.19728 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8b7fff640 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5616285366f0 0x561628538ae0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=149426a0c297e44d server_cookie=e5f779d59ce5363d in_seq=0 out_seq=0 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8977fe640 1 -- 172.21.15.105:0/289577171 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561628426200 con 0x5616285366f0 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8977fe640 10 client.19728 handle_command_reply: tid=0 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 client.19728 shutdown 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 -- 172.21.15.105:0/289577171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 msgr2=0x5616284c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284c87c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0058c0 tx=0x7fe8ac009d50 comp rx=0 tx=0).stop 2024-03-26T17:50:00.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 -- 172.21.15.105:0/289577171 shutdown_connections 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5616285366f0 0x561628538ae0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616284bf590 0x5616284c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616284ba420 0x5616284c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8977fe640 0 client.19728 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 --2- 172.21.15.105:0/289577171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616284b9a30 0x5616284c87c0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 -- 172.21.15.105:0/289577171 >> 172.21.15.105:0/289577171 conn(0x5616284b4fa0 msgr2=0x5616284b8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 -- 172.21.15.105:0/289577171 shutdown_connections 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.227+0000 7fe8c15be740 1 -- 172.21.15.105:0/289577171 wait complete. 2024-03-26T17:50:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8c15be740 20 client.19728 trim_cache size 0 max 16384 2024-03-26T17:50:00.263 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:00.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 -- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8a00a2f20 msgr2=0x7fe8a00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8a00a2f20 0x7fe8a00a53e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe8b009c000 tx=0x7fe8b006bb90 comp rx=0 tx=0).stop 2024-03-26T17:50:00.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 -- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b8102370 msgr2=0x7fe8b819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b8102370 0x7fe8b819eea0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000eb10 tx=0x7fe8b000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 -- 172.21.15.105:0/3446944587 shutdown_connections 2024-03-26T17:50:00.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe8a00a2f20 0x7fe8a00a53e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8b8102370 0x7fe8b819eea0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8b81019b0 0x7fe8b819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 --2- 172.21.15.105:0/3446944587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b81007c0 0x7fe8b8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 -- 172.21.15.105:0/3446944587 >> 172.21.15.105:0/3446944587 conn(0x7fe8b80fbd50 msgr2=0x7fe8b8104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 -- 172.21.15.105:0/3446944587 shutdown_connections 2024-03-26T17:50:00.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.228+0000 7fe8be8f8640 1 -- 172.21.15.105:0/3446944587 wait complete. 2024-03-26T17:50:00.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:50:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.391+0000 7fb69c73b640 1 -- 172.21.15.105:0/2048080415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6941007c0 msgr2=0x7fb694100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.391+0000 7fb69c73b640 1 --2- 172.21.15.105:0/2048080415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6941007c0 0x7fb694100be0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb690009cc0 tx=0x7fb69002e310 comp rx=0 tx=0).stop 2024-03-26T17:50:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 -- 172.21.15.105:0/2048080415 shutdown_connections 2024-03-26T17:50:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 --2- 172.21.15.105:0/2048080415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb694102370 0x7fb694108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 --2- 172.21.15.105:0/2048080415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6941019b0 0x7fb694101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 --2- 172.21.15.105:0/2048080415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6941007c0 0x7fb694100be0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 -- 172.21.15.105:0/2048080415 >> 172.21.15.105:0/2048080415 conn(0x7fb6940fbd50 msgr2=0x7fb6940fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 -- 172.21.15.105:0/2048080415 shutdown_connections 2024-03-26T17:50:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 -- 172.21.15.105:0/2048080415 wait complete. 2024-03-26T17:50:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 Processor -- start 2024-03-26T17:50:00.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.392+0000 7fb69c73b640 1 -- start start 2024-03-26T17:50:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69c73b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6941007c0 0x7fb694195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69c73b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6941019b0 0x7fb694195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69c73b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb694102370 0x7fb69419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69c73b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb69410b1f0 con 0x7fb694102370 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69c73b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb69410b070 con 0x7fb6941007c0 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69c73b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb69410b370 con 0x7fb6941019b0 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69a4d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6941007c0 0x7fb694195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69a4d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6941007c0 0x7fb694195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37152/0 (socket says 172.21.15.105:37152) 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69a4d7640 1 -- 172.21.15.105:0/743462534 learned_addr learned my addr 172.21.15.105:0/743462534 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69acd8640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb694102370 0x7fb69419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb699cd6640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6941019b0 0x7fb694195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb699cd6640 1 -- 172.21.15.105:0/743462534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6941007c0 msgr2=0x7fb694195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb699cd6640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6941007c0 0x7fb694195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb699cd6640 1 -- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb694102370 msgr2=0x7fb69419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb699cd6640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb694102370 0x7fb69419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb699cd6640 1 -- 172.21.15.105:0/743462534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69419d560 con 0x7fb6941019b0 2024-03-26T17:50:00.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb69acd8640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb694102370 0x7fb69419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.393+0000 7fb699cd6640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6941019b0 0x7fb694195e80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb68400e9f0 tx=0x7fb68400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.394+0000 7fb68b7fe640 1 -- 172.21.15.105:0/743462534 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68400cd80 con 0x7fb6941019b0 2024-03-26T17:50:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.394+0000 7fb69c73b640 1 -- 172.21.15.105:0/743462534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69419d850 con 0x7fb6941019b0 2024-03-26T17:50:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.394+0000 7fb68b7fe640 1 -- 172.21.15.105:0/743462534 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6840035f0 con 0x7fb6941019b0 2024-03-26T17:50:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.394+0000 7fb68b7fe640 1 -- 172.21.15.105:0/743462534 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684010690 con 0x7fb6941019b0 2024-03-26T17:50:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.394+0000 7fb69c73b640 1 -- 172.21.15.105:0/743462534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69419ddf0 con 0x7fb6941019b0 2024-03-26T17:50:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.396+0000 7fb68b7fe640 1 -- 172.21.15.105:0/743462534 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb6840037a0 con 0x7fb6941019b0 2024-03-26T17:50:00.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.396+0000 7fb68b7fe640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6800a2f20 0x7fb6800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb68b7fe640 1 -- 172.21.15.105:0/743462534 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb6840cf6d0 con 0x7fb6941019b0 2024-03-26T17:50:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6800a2f20 0x7fb6800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb69f3ff740 1 Processor -- start 2024-03-26T17:50:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb69f3ff740 1 -- start start 2024-03-26T17:50:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6800a2f20 0x7fb6800a53e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb690000f80 tx=0x7fb690004a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfa2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfa7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfaaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.397+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b04cf07970 con 0x55b04cfa2a30 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b04ceafd70 con 0x55b04cfa8590 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b04cf03920 con 0x55b04cfa3420 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb69a4d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfa2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb69a4d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfa2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44084/0 (socket says 172.21.15.105:44084) 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb699cd6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfa7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb69a4d7640 1 -- 172.21.15.105:0/1455019027 learned_addr learned my addr 172.21.15.105:0/1455019027 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb69acd8640 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfaaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb699cd6640 1 -- 172.21.15.105:0/1455019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 msgr2=0x55b04cfaaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb699cd6640 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfaaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb699cd6640 1 -- 172.21.15.105:0/1455019027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 msgr2=0x55b04cfa2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb699cd6640 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfa2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb699cd6640 1 -- 172.21.15.105:0/1455019027 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04cc1b3f0 con 0x55b04cfa3420 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb69acd8640 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfaaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb699cd6640 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfa7f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb68400e9c0 tx=0x7fb684003b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1663f8f3a19f07 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb688ff9640 1 -- 172.21.15.105:0/1455019027 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68409ff10 con 0x55b04cfa3420 2024-03-26T17:50:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb688ff9640 1 -- 172.21.15.105:0/1455019027 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6840a47d0 con 0x55b04cfa3420 2024-03-26T17:50:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.398+0000 7fb688ff9640 1 -- 172.21.15.105:0/1455019027 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68409b470 con 0x55b04cfa3420 2024-03-26T17:50:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.399+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1455019027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 msgr2=0x55b04cfa7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.399+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfa7f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb68400e9c0 tx=0x7fb684003b30 comp rx=0 tx=0).stop 2024-03-26T17:50:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.399+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1455019027 shutdown_connections 2024-03-26T17:50:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.399+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfaaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.399+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfa7f20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.399+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1455019027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfa2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.399+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1455019027 >> 172.21.15.105:0/1455019027 conn(0x55b04cf9dfa0 msgr2=0x55b04cf9e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.400+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1455019027 shutdown_connections 2024-03-26T17:50:00.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:00 smithi105 ceph-mon[32280]: overall HEALTH_OK 2024-03-26T17:50:00.393 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:00 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3550419780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:50:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.400+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1455019027 wait complete. 2024-03-26T17:50:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.400+0000 7fb69f3ff740 1 -- start start 2024-03-26T17:50:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.400+0000 7fb69f3ff740 1 Processor -- start 2024-03-26T17:50:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.400+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa2a30 0x55b04cfacf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.400+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa3420 0x55b04cfad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.400+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa8590 0x55b04cfb4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b04cfab4e0 con 0x55b04cfa3420 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b04cec3310 con 0x55b04cfa2a30 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b04cfab660 con 0x55b04cfa8590 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69f3ff740 10 client.0 _ll_register_callbacks cb 0x55b04cefec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa2a30 0x55b04cfacf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb699cd6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa3420 0x55b04cfad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa2a30 0x55b04cfacf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37168/0 (socket says 172.21.15.105:37168) 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 -- 172.21.15.105:0/3021512705 learned_addr learned my addr 172.21.15.105:0/3021512705 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69acd8640 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa8590 0x55b04cfb4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 -- 172.21.15.105:0/3021512705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa8590 msgr2=0x55b04cfb4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa8590 0x55b04cfb4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 -- 172.21.15.105:0/3021512705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa3420 msgr2=0x55b04cfad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa3420 0x55b04cfad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 -- 172.21.15.105:0/3021512705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04cce57d0 con 0x55b04cfa2a30 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa2a30 0x55b04cfacf10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb690005e30 tx=0x7fb69002e310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc29117267cf70cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb67dffb640 1 -- 172.21.15.105:0/3021512705 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690052020 con 0x55b04cfa2a30 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb67dffb640 1 -- 172.21.15.105:0/3021512705 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb690004450 con 0x55b04cfa2a30 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.401+0000 7fb69f3ff740 1 -- 172.21.15.105:0/3021512705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6840a47d0 con 0x55b04cfa2a30 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb67dffb640 1 -- 172.21.15.105:0/3021512705 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb69003b040 con 0x55b04cfa2a30 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb69f3ff740 10 client.29546 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb69f3ff740 1 -- 172.21.15.105:0/3021512705 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b04cfb7870 con 0x55b04cfa2a30 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb67dffb640 1 -- 172.21.15.105:0/3021512705 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6840a47d0 con 0x55b04cfa2a30 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb67dffb640 1 -- 172.21.15.105:0/3021512705 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb69004ecb0 con 0x55b04cfa2a30 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb69f3ff740 10 client.29546 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb69f3ff740 10 client.29546 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb69f3ff740 20 client.29546 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b04cfb9310 0x55b04cfbb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.402+0000 7fb69f3ff740 4 client.29546 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.403+0000 7fb69f3ff740 1 -- 172.21.15.105:0/3021512705 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b04cf14eb0 con 0x55b04cfb9310 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.403+0000 7fb699cd6640 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b04cfb9310 0x55b04cfbb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.403+0000 7fb67dffb640 10 client.29546 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.403+0000 7fb699cd6640 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b04cfb9310 0x55b04cfbb760 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=671f1d7d3b172f27 server_cookie=e285fc470ad6fc04 in_seq=0 out_seq=0 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb67dffb640 1 -- 172.21.15.105:0/3021512705 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b04cf14eb0 con 0x55b04cfb9310 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb67dffb640 10 client.29546 handle_command_reply: tid=0 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 client.29546 shutdown 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 -- 172.21.15.105:0/3021512705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa2a30 msgr2=0x55b04cfacf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa2a30 0x55b04cfacf10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb690005e30 tx=0x7fb69002e310 comp rx=0 tx=0).stop 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 -- 172.21.15.105:0/3021512705 shutdown_connections 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b04cfb9310 0x55b04cfbb760 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa8590 0x55b04cfb4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa3420 0x55b04cfad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/3021512705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa2a30 0x55b04cfacf10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb67dffb640 0 client.29546 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:50:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.404+0000 7fb69f3ff740 1 -- 172.21.15.105:0/3021512705 >> 172.21.15.105:0/3021512705 conn(0x55b04cf9dfa0 msgr2=0x55b04cf9d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.405+0000 7fb69f3ff740 1 -- 172.21.15.105:0/3021512705 shutdown_connections 2024-03-26T17:50:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.405+0000 7fb69f3ff740 1 -- 172.21.15.105:0/3021512705 wait complete. 2024-03-26T17:50:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.405+0000 7fb69f3ff740 20 client.29546 trim_cache size 0 max 16384 2024-03-26T17:50:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.420+0000 7fb69f3ff740 1 Processor -- start 2024-03-26T17:50:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.420+0000 7fb69f3ff740 1 -- start start 2024-03-26T17:50:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa2a30 0x55b04cfa4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa3420 0x55b04cfa4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa8590 0x55b04cfb4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941a6c50 con 0x55b04cfa8590 2024-03-26T17:50:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941a8790 con 0x55b04cfa3420 2024-03-26T17:50:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941a7320 con 0x55b04cfa2a30 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa8590 0x55b04cfb4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa8590 0x55b04cfb4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44100/0 (socket says 172.21.15.105:44100) 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 -- 172.21.15.105:0/1527065786 learned_addr learned my addr 172.21.15.105:0/1527065786 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb699cd6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa3420 0x55b04cfa4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa2a30 0x55b04cfa4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 -- 172.21.15.105:0/1527065786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa2a30 msgr2=0x55b04cfa4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa2a30 0x55b04cfa4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 -- 172.21.15.105:0/1527065786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa3420 msgr2=0x55b04cfa4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa3420 0x55b04cfa4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 -- 172.21.15.105:0/1527065786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04cf9e9f0 con 0x55b04cfa8590 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb699cd6640 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa3420 0x55b04cfa4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb69acd8640 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa8590 0x55b04cfb4450 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0127a0 tx=0x7fb68c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6e0e673532f36f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.421+0000 7fb67ffff640 1 -- 172.21.15.105:0/1527065786 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0120d0 con 0x55b04cfa8590 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb67ffff640 1 -- 172.21.15.105:0/1527065786 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb68c012270 con 0x55b04cfa8590 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb67ffff640 1 -- 172.21.15.105:0/1527065786 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68c0047a0 con 0x55b04cfa8590 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1527065786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa8590 msgr2=0x55b04cfb4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa8590 0x55b04cfb4450 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0127a0 tx=0x7fb68c005f90 comp rx=0 tx=0).stop 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1527065786 shutdown_connections 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa8590 0x55b04cfb4450 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa3420 0x55b04cfa4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1527065786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa2a30 0x55b04cfa4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1527065786 >> 172.21.15.105:0/1527065786 conn(0x55b04cf9dfa0 msgr2=0x55b04cfaba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1527065786 shutdown_connections 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.422+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1527065786 wait complete. 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 1 -- start start 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 1 Processor -- start 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfb3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfb4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfb07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941a80c0 con 0x55b04cfa2a30 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941a79f0 con 0x55b04cfa8590 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6941a4c30 con 0x55b04cfa3420 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69f3ff740 10 client.0 _ll_register_callbacks cb 0x7fb684005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69a4d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfb3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb699cd6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfb4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb699cd6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfb4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33930/0 (socket says 172.21.15.105:33930) 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb699cd6640 1 -- 172.21.15.105:0/1524519537 learned_addr learned my addr 172.21.15.105:0/1524519537 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69a4d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfb3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44116/0 (socket says 172.21.15.105:44116) 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.423+0000 7fb69acd8640 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfb07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb699cd6640 1 -- 172.21.15.105:0/1524519537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 msgr2=0x55b04cfb07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb699cd6640 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfb07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb699cd6640 1 -- 172.21.15.105:0/1524519537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 msgr2=0x55b04cfb3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb699cd6640 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfb3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb699cd6640 1 -- 172.21.15.105:0/1524519537 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04cfb0e00 con 0x55b04cfa3420 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfb3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb699cd6640 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfb4170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6840cd1e0 tx=0x7fb684005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea273cf5c3c0ced server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb67dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb67dffb640 1 -- 172.21.15.105:0/1524519537 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68407f060 con 0x55b04cfa3420 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb67dffb640 1 -- 172.21.15.105:0/1524519537 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb684009310 con 0x55b04cfa3420 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1524519537 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb68c012270 con 0x55b04cfa3420 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb67dffb640 1 -- 172.21.15.105:0/1524519537 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb6840a47d0 con 0x55b04cfa3420 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb69f3ff740 10 client.29533 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1524519537 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b04d01e2e0 con 0x55b04cfa3420 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.424+0000 7fb67dffb640 1 -- 172.21.15.105:0/1524519537 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb68c012270 con 0x55b04cfa3420 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb67dffb640 1 -- 172.21.15.105:0/1524519537 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb68409b330 con 0x55b04cfa3420 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 10 client.29533 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 10 client.29533 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 20 client.29533 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b04d01e850 0x55b04d020c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 4 client.29533 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1524519537 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b04cf0f200 con 0x55b04d01e850 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b04d01e850 0x55b04d020c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb67dffb640 10 client.29533 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69a4d7640 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b04d01e850 0x55b04d020c40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b40e6adb6cf2ce9 server_cookie=feef4d7e25d7f9b4 in_seq=0 out_seq=0 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb67dffb640 1 -- 172.21.15.105:0/1524519537 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b04cf0f200 con 0x55b04d01e850 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb67dffb640 10 client.29533 handle_command_reply: tid=0 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 1 client.29533 shutdown 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1524519537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 msgr2=0x55b04cfb4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.425+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfb4170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6840cd1e0 tx=0x7fb684005bb0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1524519537 shutdown_connections 2024-03-26T17:50:00.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b04d01e850 0x55b04d020c40 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b04cfa8590 0x55b04cfb07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb67dffb640 0 client.29533 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b04cfa3420 0x55b04cfb4170 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 1 --2- 172.21.15.105:0/1524519537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b04cfa2a30 0x55b04cfb3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1524519537 >> 172.21.15.105:0/1524519537 conn(0x55b04cf9dfa0 msgr2=0x55b04cfa4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1524519537 shutdown_connections 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 1 -- 172.21.15.105:0/1524519537 wait complete. 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69f3ff740 20 client.29533 trim_cache size 0 max 16384 2024-03-26T17:50:00.423 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 -- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6800a2f20 msgr2=0x7fb6800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6800a2f20 0x7fb6800a53e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb690000f80 tx=0x7fb690004a80 comp rx=0 tx=0).stop 2024-03-26T17:50:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 -- 172.21.15.105:0/743462534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6941019b0 msgr2=0x7fb694195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6941019b0 0x7fb694195e80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb68400e9f0 tx=0x7fb68400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 -- 172.21.15.105:0/743462534 shutdown_connections 2024-03-26T17:50:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6800a2f20 0x7fb6800a53e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb694102370 0x7fb69419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6941019b0 0x7fb694195e80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 --2- 172.21.15.105:0/743462534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6941007c0 0x7fb694195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 -- 172.21.15.105:0/743462534 >> 172.21.15.105:0/743462534 conn(0x7fb6940fbd50 msgr2=0x7fb694104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 -- 172.21.15.105:0/743462534 shutdown_connections 2024-03-26T17:50:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.426+0000 7fb69c73b640 1 -- 172.21.15.105:0/743462534 wait complete. 2024-03-26T17:50:00.429 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/1617505893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb01018d0 msgr2=0x7fbdb0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/1617505893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb01018d0 0x7fbdb0101cf0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009900 tx=0x7fbda402bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/1617505893 shutdown_connections 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/1617505893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb0103480 0x7fbdb0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/1617505893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb0102ac0 0x7fbdb0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/1617505893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb01018d0 0x7fbdb0101cf0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/1617505893 >> 172.21.15.105:0/1617505893 conn(0x7fbdb00fce20 msgr2=0x7fbdb00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/1617505893 shutdown_connections 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.560+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/1617505893 wait complete. 2024-03-26T17:50:00.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb6fb9640 1 Processor -- start 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb6fb9640 1 -- start start 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb6fb9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb01018d0 0x7fbdb0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb6fb9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb0102ac0 0x7fbdb01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb6fb9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb0103480 0x7fbdb019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb6fb9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb010d400 con 0x7fbdb0102ac0 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb6fb9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb010d280 con 0x7fbdb0103480 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb6fb9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb010d580 con 0x7fbdb01018d0 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb4d55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb01018d0 0x7fbdb0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb0102ac0 0x7fbdb01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb5556640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb0103480 0x7fbdb019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb4d55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb01018d0 0x7fbdb0197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33960/0 (socket says 172.21.15.105:33960) 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb4d55640 1 -- 172.21.15.105:0/4078067178 learned_addr learned my addr 172.21.15.105:0/4078067178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb4d55640 1 -- 172.21.15.105:0/4078067178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb0103480 msgr2=0x7fbdb019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb0103480 0x7fbdb019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb4d55640 1 -- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb0102ac0 msgr2=0x7fbdb01980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb0102ac0 0x7fbdb01980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdb4d55640 1 -- 172.21.15.105:0/4078067178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb019f770 con 0x7fbdb01018d0 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.561+0000 7fbdaffff640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb0102ac0 0x7fbdb01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.562+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb01018d0 0x7fbdb0197ba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000c00 tx=0x7fbda4005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.562+0000 7fbdadffb640 1 -- 172.21.15.105:0/4078067178 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4043020 con 0x7fbdb01018d0 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.562+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/4078067178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb019fa00 con 0x7fbdb01018d0 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.562+0000 7fbdadffb640 1 -- 172.21.15.105:0/4078067178 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbda4004350 con 0x7fbdb01018d0 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.562+0000 7fbdadffb640 1 -- 172.21.15.105:0/4078067178 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4003b80 con 0x7fbdb01018d0 2024-03-26T17:50:00.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.562+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/4078067178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb019ff50 con 0x7fbdb01018d0 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb9ba5740 1 Processor -- start 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb9ba5740 1 -- start start 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d98a30 0x55a632d98e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632d9df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632da0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a632cfd970 con 0x55a632d98a30 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a632ca5d70 con 0x55a632d9e590 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a632cf9920 con 0x55a632d99420 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdadffb640 1 -- 172.21.15.105:0/4078067178 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbda4046020 con 0x7fbdb01018d0 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb5556640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d98a30 0x55a632d98e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb5556640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d98a30 0x55a632d98e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44134/0 (socket says 172.21.15.105:44134) 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb5556640 1 -- 172.21.15.105:0/45743935 learned_addr learned my addr 172.21.15.105:0/45743935 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdaffff640 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632da0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.565+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632d9df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdaffff640 1 -- 172.21.15.105:0/45743935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 msgr2=0x55a632d9df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdaffff640 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632d9df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdaffff640 1 -- 172.21.15.105:0/45743935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d98a30 msgr2=0x55a632d98e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdaffff640 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d98a30 0x55a632d98e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdaffff640 1 -- 172.21.15.105:0/45743935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a632a113f0 con 0x55a632d9e590 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdb5556640 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d98a30 0x55a632d98e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdadffb640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbda00a2f20 0x7fbda00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdb5556640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbda00a2f20 0x7fbda00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdadffb640 1 -- 172.21.15.105:0/4078067178 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbda40f1580 con 0x7fbdb01018d0 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdaffff640 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632da0a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009fa0 tx=0x7fbd9c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19a91660f189090 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbdb5556640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbda00a2f20 0x7fbda00a53e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55a632d99c80 tx=0x7fbda8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.566+0000 7fbd9affd640 1 -- 172.21.15.105:0/45743935 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c004030 con 0x55a632d9e590 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbd9affd640 1 -- 172.21.15.105:0/45743935 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd9c0054d0 con 0x55a632d9e590 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbd9affd640 1 -- 172.21.15.105:0/45743935 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c004c40 con 0x55a632d9e590 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/45743935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 msgr2=0x55a632da0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632da0a20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009fa0 tx=0x7fbd9c0118d0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/45743935 shutdown_connections 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632da0a20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632d9df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/45743935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d98a30 0x55a632d98e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/45743935 >> 172.21.15.105:0/45743935 conn(0x55a632d93fa0 msgr2=0x55a632d943a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/45743935 shutdown_connections 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.567+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/45743935 wait complete. 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 1 -- start start 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 1 Processor -- start 2024-03-26T17:50:00.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d98a30 0x55a632da2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632daa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a632da1540 con 0x55a632d99420 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a632cb9310 con 0x55a632d9e590 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a632da16c0 con 0x55a632d98a30 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb9ba5740 10 client.0 _ll_register_callbacks cb 0x55a632cf4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb4d55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d98a30 0x55a632da2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44136/0 (socket says 172.21.15.105:44136) 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb5556640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632daa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdaffff640 1 -- 172.21.15.105:0/2503255440 learned_addr learned my addr 172.21.15.105:0/2503255440 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb4d55640 1 -- 172.21.15.105:0/2503255440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 msgr2=0x55a632daa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632daa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb4d55640 1 -- 172.21.15.105:0/2503255440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 msgr2=0x55a632da3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.568+0000 7fbdb4d55640 1 -- 172.21.15.105:0/2503255440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a632adb7d0 con 0x55a632d98a30 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbdaffff640 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbdb5556640 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632daa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d98a30 0x55a632da2f50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbda40ef090 tx=0x7fbda4004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb186c30f4701c73 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbd8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbd8ffff640 1 -- 172.21.15.105:0/2503255440 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda40bd7b0 con 0x55a632d98a30 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbd8ffff640 1 -- 172.21.15.105:0/2503255440 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbda4003d20 con 0x55a632d98a30 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2503255440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9c0054d0 con 0x55a632d98a30 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbd8ffff640 1 -- 172.21.15.105:0/2503255440 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbda40bd5d0 con 0x55a632d98a30 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbdb9ba5740 10 client.29545 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2503255440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a632da15b0 con 0x55a632d98a30 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.569+0000 7fbd8ffff640 1 -- 172.21.15.105:0/2503255440 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd9c0054d0 con 0x55a632d98a30 2024-03-26T17:50:00.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbd8ffff640 1 -- 172.21.15.105:0/2503255440 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbda40c61f0 con 0x55a632d98a30 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbdb9ba5740 10 client.29545 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbdb9ba5740 10 client.29545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbdb9ba5740 20 client.29545 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a632daef30 0x55a632db13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbdb9ba5740 4 client.29545 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2503255440 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a632d0aeb0 con 0x55a632daef30 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbdaffff640 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a632daef30 0x55a632db13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbdaffff640 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a632daef30 0x55a632db13c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4530ed7c09ad6f server_cookie=12732b7dc48c6100 in_seq=0 out_seq=0 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.570+0000 7fbd8ffff640 10 client.29545 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbd8ffff640 1 -- 172.21.15.105:0/2503255440 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a632d0aeb0 con 0x55a632daef30 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbd8ffff640 10 client.29545 handle_command_reply: tid=0 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 client.29545 shutdown 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2503255440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d98a30 msgr2=0x55a632da2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d98a30 0x55a632da2f50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbda40ef090 tx=0x7fbda4004580 comp rx=0 tx=0).stop 2024-03-26T17:50:00.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2503255440 shutdown_connections 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a632daef30 0x55a632db13c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d9e590 0x55a632daa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbd8ffff640 0 client.29545 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2503255440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d98a30 0x55a632da2f50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2503255440 >> 172.21.15.105:0/2503255440 conn(0x55a632d93fa0 msgr2=0x55a632d931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2503255440 shutdown_connections 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2503255440 wait complete. 2024-03-26T17:50:00.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.571+0000 7fbdb9ba5740 20 client.29545 trim_cache size 0 max 16384 2024-03-26T17:50:00.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.586+0000 7fbdb9ba5740 1 Processor -- start 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.586+0000 7fbdb9ba5740 1 -- start start 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632d99ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632daa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d9e590 0x55a632da2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01a8da0 con 0x55a632d9e590 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01aa8e0 con 0x55a632d98a30 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01a9470 con 0x55a632d99420 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632daa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdb5556640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d9e590 0x55a632da2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632daa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33982/0 (socket says 172.21.15.105:33982) 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 -- 172.21.15.105:0/2100478994 learned_addr learned my addr 172.21.15.105:0/2100478994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/2100478994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632d99ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 -- 172.21.15.105:0/2100478994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 msgr2=0x55a632d99ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 --2- 172.21.15.105:0/2100478994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632d99ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 -- 172.21.15.105:0/2100478994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d9e590 msgr2=0x55a632da2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 --2- 172.21.15.105:0/2100478994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d9e590 0x55a632da2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 -- 172.21.15.105:0/2100478994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a632d99cb0 con 0x55a632d99420 2024-03-26T17:50:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.587+0000 7fbdaffff640 1 --2- 172.21.15.105:0/2100478994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632daa490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55a632da4430 tx=0x7fbd9c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1589ef43cf51190 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbd9a7fc640 1 -- 172.21.15.105:0/2100478994 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c01d040 con 0x55a632d99420 2024-03-26T17:50:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbd9a7fc640 1 -- 172.21.15.105:0/2100478994 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd9c0044f0 con 0x55a632d99420 2024-03-26T17:50:00.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbd9a7fc640 1 -- 172.21.15.105:0/2100478994 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c003bb0 con 0x55a632d99420 2024-03-26T17:50:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2100478994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 msgr2=0x55a632daa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2100478994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632daa490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55a632da4430 tx=0x7fbd9c009310 comp rx=0 tx=0).stop 2024-03-26T17:50:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2100478994 shutdown_connections 2024-03-26T17:50:00.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2100478994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d9e590 0x55a632da2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2100478994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d99420 0x55a632daa490 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/2100478994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632d99ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2100478994 >> 172.21.15.105:0/2100478994 conn(0x55a632d93fa0 msgr2=0x55a632d94380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2100478994 shutdown_connections 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.588+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/2100478994 wait complete. 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 1 -- start start 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 1 Processor -- start 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632da76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d9e590 0x55a632da8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01aa210 con 0x55a632d99420 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01a9b40 con 0x55a632d98a30 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01a6d80 con 0x55a632d9e590 2024-03-26T17:50:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb9ba5740 10 client.0 _ll_register_callbacks cb 0x7fbda40057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb4d55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632da76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44148/0 (socket says 172.21.15.105:44148) 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb4d55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632da76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37236/0 (socket says 172.21.15.105:37236) 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdaffff640 1 -- 172.21.15.105:0/314180530 learned_addr learned my addr 172.21.15.105:0/314180530 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.589+0000 7fbdb5556640 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d9e590 0x55a632da8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdaffff640 1 -- 172.21.15.105:0/314180530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d9e590 msgr2=0x55a632da8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdaffff640 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d9e590 0x55a632da8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdaffff640 1 -- 172.21.15.105:0/314180530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 msgr2=0x55a632da76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdaffff640 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632da76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdaffff640 1 -- 172.21.15.105:0/314180530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a632da9200 con 0x55a632d99420 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632da76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdaffff640 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da7c00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009ef0 tx=0x7fbd9c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9ed9364f158284 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbd8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbd8ffff640 1 -- 172.21.15.105:0/314180530 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c01d030 con 0x55a632d99420 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/314180530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd9c0044f0 con 0x55a632d99420 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbd8ffff640 1 -- 172.21.15.105:0/314180530 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbd9c00fbd0 con 0x55a632d99420 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbd8ffff640 1 -- 172.21.15.105:0/314180530 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd9c00fdd0 con 0x55a632d99420 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdb9ba5740 10 client.19764 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.590+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/314180530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a632daa500 con 0x55a632d99420 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbd8ffff640 1 -- 172.21.15.105:0/314180530 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd9c0044f0 con 0x55a632d99420 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbd8ffff640 1 -- 172.21.15.105:0/314180530 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbd9c02d030 con 0x55a632d99420 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbdb9ba5740 10 client.19764 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbdb9ba5740 10 client.19764 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbdb9ba5740 20 client.19764 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a632daef30 0x55a632e17570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbdb9ba5740 4 client.19764 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/314180530 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a632d05200 con 0x55a632daef30 2024-03-26T17:50:00.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a632daef30 0x55a632e17570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbdb4d55640 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a632daef30 0x55a632e17570 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29fbe23907933dd7 server_cookie=a99e5d5d11904f0f in_seq=0 out_seq=0 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.591+0000 7fbd8ffff640 10 client.19764 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbd8ffff640 1 -- 172.21.15.105:0/314180530 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a632d05200 con 0x55a632daef30 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbd8ffff640 10 client.19764 handle_command_reply: tid=0 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 client.19764 shutdown 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/314180530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 msgr2=0x55a632da7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da7c00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009ef0 tx=0x7fbd9c009b70 comp rx=0 tx=0).stop 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/314180530 shutdown_connections 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a632d9e590 0x55a632da8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a632d99420 0x55a632da7c00 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a632daef30 0x55a632e17570 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 --2- 172.21.15.105:0/314180530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a632d98a30 0x55a632da76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/314180530 >> 172.21.15.105:0/314180530 conn(0x55a632d93fa0 msgr2=0x55a632d99e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbd8ffff640 0 client.19764 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:50:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/314180530 shutdown_connections 2024-03-26T17:50:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 1 -- 172.21.15.105:0/314180530 wait complete. 2024-03-26T17:50:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.592+0000 7fbdb9ba5740 20 client.19764 trim_cache size 0 max 16384 2024-03-26T17:50:00.589 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbda00a2f20 msgr2=0x7fbda00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbda00a2f20 0x7fbda00a53e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55a632d99c80 tx=0x7fbda8009500 comp rx=0 tx=0).stop 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/4078067178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb01018d0 msgr2=0x7fbdb0197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb01018d0 0x7fbdb0197ba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000c00 tx=0x7fbda4005dc0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/4078067178 shutdown_connections 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbda00a2f20 0x7fbda00a53e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbdb0103480 0x7fbdb019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdb0102ac0 0x7fbdb01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 --2- 172.21.15.105:0/4078067178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbdb01018d0 0x7fbdb0197ba0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/4078067178 >> 172.21.15.105:0/4078067178 conn(0x7fbdb00fce20 msgr2=0x7fbdb0105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/4078067178 shutdown_connections 2024-03-26T17:50:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.593+0000 7fbdb6fb9640 1 -- 172.21.15.105:0/4078067178 wait complete. 2024-03-26T17:50:00.597 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:50:00.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:00 smithi144 ceph-mon[36095]: overall HEALTH_OK 2024-03-26T17:50:00.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:00 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3550419780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:50:00.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:00 smithi135 ceph-mon[35636]: overall HEALTH_OK 2024-03-26T17:50:00.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:00 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3550419780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 -- 172.21.15.105:0/3222990215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 msgr2=0x7ff85c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 --2- 172.21.15.105:0/3222990215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 0x7ff85c100be0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009900 tx=0x7ff84c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 -- 172.21.15.105:0/3222990215 shutdown_connections 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 --2- 172.21.15.105:0/3222990215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85c102370 0x7ff85c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 --2- 172.21.15.105:0/3222990215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff85c1019b0 0x7ff85c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 --2- 172.21.15.105:0/3222990215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 0x7ff85c100be0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 -- 172.21.15.105:0/3222990215 >> 172.21.15.105:0/3222990215 conn(0x7ff85c0fbd50 msgr2=0x7ff85c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 -- 172.21.15.105:0/3222990215 shutdown_connections 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.723+0000 7ff862760640 1 -- 172.21.15.105:0/3222990215 wait complete. 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff862760640 1 Processor -- start 2024-03-26T17:50:00.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff862760640 1 -- start start 2024-03-26T17:50:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff862760640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 0x7ff85c196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff862760640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff85c1019b0 0x7ff85c196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff862760640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85c102370 0x7ff85c19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff862760640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10c2c0 con 0x7ff85c1007c0 2024-03-26T17:50:00.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff862760640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10c140 con 0x7ff85c102370 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff862760640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10c440 con 0x7ff85c1019b0 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 0x7ff85c196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff85bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 0x7ff85c196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44180/0 (socket says 172.21.15.105:44180) 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff85bfff640 1 -- 172.21.15.105:0/1223418100 learned_addr learned my addr 172.21.15.105:0/1223418100 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff860cfd640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85c102370 0x7ff85c19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff85c1019b0 0x7ff85c196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff85bfff640 1 -- 172.21.15.105:0/1223418100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff85c1019b0 msgr2=0x7ff85c196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff85bfff640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff85c1019b0 0x7ff85c196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.724+0000 7ff85bfff640 1 -- 172.21.15.105:0/1223418100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85c102370 msgr2=0x7ff85c19e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff85bfff640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85c102370 0x7ff85c19e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff85bfff640 1 -- 172.21.15.105:0/1223418100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c19e690 con 0x7ff85c1007c0 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff860cfd640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85c102370 0x7ff85c19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff85bfff640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 0x7ff85c196a70 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff84c000c00 tx=0x7ff84c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff8597fa640 1 -- 172.21.15.105:0/1223418100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c043020 con 0x7ff85c1007c0 2024-03-26T17:50:00.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff8597fa640 1 -- 172.21.15.105:0/1223418100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff84c004350 con 0x7ff85c1007c0 2024-03-26T17:50:00.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff8597fa640 1 -- 172.21.15.105:0/1223418100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c003b10 con 0x7ff85c1007c0 2024-03-26T17:50:00.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff862760640 1 -- 172.21.15.105:0/1223418100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c19e920 con 0x7ff85c1007c0 2024-03-26T17:50:00.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.725+0000 7ff862760640 1 -- 172.21.15.105:0/1223418100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c19ee70 con 0x7ff85c1007c0 2024-03-26T17:50:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff86541f740 1 Processor -- start 2024-03-26T17:50:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff86541f740 1 -- start start 2024-03-26T17:50:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff8597fa640 1 -- 172.21.15.105:0/1223418100 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff84c046020 con 0x7ff85c1007c0 2024-03-26T17:50:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff86541f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x56251543ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff86541f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543f420 0x562515443f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff86541f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562515444590 0x562515446a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff86541f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5625153a3970 con 0x56251543ea30 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff86541f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56251534bd70 con 0x562515444590 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff86541f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56251539f920 con 0x56251543f420 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff860cfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x56251543ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff860cfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x56251543ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44184/0 (socket says 172.21.15.105:44184) 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff860cfd640 1 -- 172.21.15.105:0/698656629 learned_addr learned my addr 172.21.15.105:0/698656629 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562515444590 0x562515446a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff85bfff640 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543f420 0x562515443f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff860cfd640 1 -- 172.21.15.105:0/698656629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543f420 msgr2=0x562515443f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff860cfd640 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543f420 0x562515443f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff860cfd640 1 -- 172.21.15.105:0/698656629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562515444590 msgr2=0x562515446a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff860cfd640 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562515444590 0x562515446a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff860cfd640 1 -- 172.21.15.105:0/698656629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625150b73f0 con 0x56251543ea30 2024-03-26T17:50:00.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.728+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562515444590 0x562515446a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.729+0000 7ff860cfd640 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x56251543ee50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff854011810 tx=0x7ff854011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8370bbaab0e566 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.729+0000 7ff8597fa640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8440a2f20 0x7ff8440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.729+0000 7ff8427fc640 1 -- 172.21.15.105:0/698656629 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854011310 con 0x56251543ea30 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.729+0000 7ff85bfff640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8440a2f20 0x7ff8440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.729+0000 7ff8427fc640 1 -- 172.21.15.105:0/698656629 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff85400f9a0 con 0x56251543ea30 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.729+0000 7ff8427fc640 1 -- 172.21.15.105:0/698656629 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8540055d0 con 0x56251543ea30 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.729+0000 7ff8597fa640 1 -- 172.21.15.105:0/1223418100 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff84c0f1510 con 0x7ff85c1007c0 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 -- 172.21.15.105:0/698656629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 msgr2=0x56251543ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff85bfff640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8440a2f20 0x7ff8440a53e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0ef340 tx=0x7ff84c0f6040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x56251543ee50 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ff854011810 tx=0x7ff854011cd0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 -- 172.21.15.105:0/698656629 shutdown_connections 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562515444590 0x562515446a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543f420 0x562515443f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 --2- 172.21.15.105:0/698656629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x56251543ee50 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 -- 172.21.15.105:0/698656629 >> 172.21.15.105:0/698656629 conn(0x562515439fa0 msgr2=0x56251543a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 -- 172.21.15.105:0/698656629 shutdown_connections 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 -- 172.21.15.105:0/698656629 wait complete. 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 -- start start 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.730+0000 7ff86541f740 1 Processor -- start 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff86541f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543ea30 0x562515448f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff86541f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515449470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff86541f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562515444590 0x5625154504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff86541f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562515447540 con 0x562515444590 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff86541f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56251535f310 con 0x56251543f420 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff86541f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625154476c0 con 0x56251543ea30 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff86541f740 10 client.0 _ll_register_callbacks cb 0x56251539ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515449470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff860cfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562515444590 0x5625154504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515449470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37268/0 (socket says 172.21.15.105:37268) 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff860cfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562515444590 0x5625154504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44186/0 (socket says 172.21.15.105:44186) 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 -- 172.21.15.105:0/3657567477 learned_addr learned my addr 172.21.15.105:0/3657567477 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff860cfd640 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543ea30 0x562515448f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 -- 172.21.15.105:0/3657567477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543ea30 msgr2=0x562515448f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543ea30 0x562515448f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 -- 172.21.15.105:0/3657567477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562515444590 msgr2=0x5625154504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562515444590 0x5625154504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 -- 172.21.15.105:0/3657567477 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625151817d0 con 0x56251543f420 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff860cfd640 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562515444590 0x5625154504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515449470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff850012700 tx=0x7ff850012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02e37a413d8c9e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff83b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.731+0000 7ff83b7fe640 1 -- 172.21.15.105:0/3657567477 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850004030 con 0x56251543f420 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 1 -- 172.21.15.105:0/3657567477 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85400f9a0 con 0x56251543f420 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff83b7fe640 1 -- 172.21.15.105:0/3657567477 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff8500041d0 con 0x56251543f420 2024-03-26T17:50:00.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff83b7fe640 1 -- 172.21.15.105:0/3657567477 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff8500043d0 con 0x56251543f420 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 10 client.29582 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 1 -- 172.21.15.105:0/3657567477 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562515455af0 con 0x56251543f420 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff83b7fe640 1 -- 172.21.15.105:0/3657567477 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff85400f9a0 con 0x56251543f420 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff83b7fe640 1 -- 172.21.15.105:0/3657567477 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff850003a70 con 0x56251543f420 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 10 client.29582 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 10 client.29582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 20 client.29582 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562515457160 0x562515459580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 4 client.29582 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff86541f740 1 -- 172.21.15.105:0/3657567477 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625153b0eb0 con 0x562515457160 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.732+0000 7ff860cfd640 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562515457160 0x562515459580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.733+0000 7ff83b7fe640 10 client.29582 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.733+0000 7ff860cfd640 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562515457160 0x562515459580 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa87be7ee1bbe0d4 server_cookie=c6dbbe086dc6a46d in_seq=0 out_seq=0 2024-03-26T17:50:00.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.733+0000 7ff83b7fe640 1 -- 172.21.15.105:0/3657567477 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5625153b0eb0 con 0x562515457160 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.733+0000 7ff83b7fe640 10 client.29582 handle_command_reply: tid=0 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 client.29582 shutdown 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 -- 172.21.15.105:0/3657567477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 msgr2=0x562515449470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515449470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff850012700 tx=0x7ff850012bc0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 -- 172.21.15.105:0/3657567477 shutdown_connections 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x562515457160 0x562515459580 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562515444590 0x5625154504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515449470 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff83b7fe640 0 client.29582 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 --2- 172.21.15.105:0/3657567477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56251543ea30 0x562515448f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 -- 172.21.15.105:0/3657567477 >> 172.21.15.105:0/3657567477 conn(0x562515439fa0 msgr2=0x5625154391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 -- 172.21.15.105:0/3657567477 shutdown_connections 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 1 -- 172.21.15.105:0/3657567477 wait complete. 2024-03-26T17:50:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.734+0000 7ff86541f740 20 client.29582 trim_cache size 0 max 16384 2024-03-26T17:50:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff86541f740 1 Processor -- start 2024-03-26T17:50:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff86541f740 1 -- start start 2024-03-26T17:50:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff86541f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x5625154409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff86541f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515440ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff86541f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 0x562515450470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff86541f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1a6c80 con 0x56251543ea30 2024-03-26T17:50:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff86541f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1a87c0 con 0x56251543f420 2024-03-26T17:50:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff86541f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1a7350 con 0x562515444590 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff860cfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x5625154409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515440ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff860cfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x5625154409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44202/0 (socket says 172.21.15.105:44202) 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff860cfd640 1 -- 172.21.15.105:0/3906218274 learned_addr learned my addr 172.21.15.105:0/3906218274 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff860cfd640 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 0x562515450470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.750+0000 7ff860cfd640 1 -- 172.21.15.105:0/3906218274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 msgr2=0x562515450470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff860cfd640 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 0x562515450470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff860cfd640 1 -- 172.21.15.105:0/3906218274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 msgr2=0x562515440ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff860cfd640 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515440ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff860cfd640 1 -- 172.21.15.105:0/3906218274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251543fcb0 con 0x56251543ea30 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515440ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff860cfd640 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x5625154409b0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff854014040 tx=0x7ff854009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c275de5e06dfb921 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff841ffb640 1 -- 172.21.15.105:0/3906218274 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85401e040 con 0x56251543ea30 2024-03-26T17:50:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff841ffb640 1 -- 172.21.15.105:0/3906218274 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff85400f9a0 con 0x56251543ea30 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.751+0000 7ff841ffb640 1 -- 172.21.15.105:0/3906218274 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854005770 con 0x56251543ea30 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 -- 172.21.15.105:0/3906218274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 msgr2=0x5625154409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x5625154409b0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff854014040 tx=0x7ff854009500 comp rx=0 tx=0).stop 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 -- 172.21.15.105:0/3906218274 shutdown_connections 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 0x562515450470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543f420 0x562515440ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 --2- 172.21.15.105:0/3906218274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543ea30 0x5625154409b0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 -- 172.21.15.105:0/3906218274 >> 172.21.15.105:0/3906218274 conn(0x562515439fa0 msgr2=0x56251543fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 -- 172.21.15.105:0/3906218274 shutdown_connections 2024-03-26T17:50:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 -- 172.21.15.105:0/3906218274 wait complete. 2024-03-26T17:50:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 -- start start 2024-03-26T17:50:00.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 Processor -- start 2024-03-26T17:50:00.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543ea30 0x56251544d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.752+0000 7ff86541f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543f420 0x56251544d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff86541f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 0x56251544e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff86541f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1a80f0 con 0x56251543f420 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff86541f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1a7a20 con 0x56251543ea30 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff86541f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1a4c60 con 0x562515444590 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff86541f740 10 client.0 _ll_register_callbacks cb 0x7ff850009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543f420 0x56251544d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543f420 0x56251544d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44218/0 (socket says 172.21.15.105:44218) 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff860cfd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543ea30 0x56251544d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 -- 172.21.15.105:0/2578396280 learned_addr learned my addr 172.21.15.105:0/2578396280 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff860cfd640 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 0x56251544e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 -- 172.21.15.105:0/2578396280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 msgr2=0x56251544e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 0x56251544e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 -- 172.21.15.105:0/2578396280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543ea30 msgr2=0x56251544d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543ea30 0x56251544d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 -- 172.21.15.105:0/2578396280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251544efc0 con 0x56251543f420 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff85b7fe640 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543f420 0x56251544d9c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff8500104d0 tx=0x7ff850005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c8daf8b42074e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff83b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff83b7fe640 1 -- 172.21.15.105:0/2578396280 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500041f0 con 0x56251543f420 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff83b7fe640 1 -- 172.21.15.105:0/2578396280 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff850010880 con 0x56251543f420 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.753+0000 7ff86541f740 1 -- 172.21.15.105:0/2578396280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85400f9a0 con 0x56251543f420 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff83b7fe640 1 -- 172.21.15.105:0/2578396280 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff850010a80 con 0x56251543f420 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff86541f740 10 client.19782 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff86541f740 1 -- 172.21.15.105:0/2578396280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562515450360 con 0x56251543f420 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff83b7fe640 1 -- 172.21.15.105:0/2578396280 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff85400f9a0 con 0x56251543f420 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff83b7fe640 1 -- 172.21.15.105:0/2578396280 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff8500036f0 con 0x56251543f420 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff86541f740 10 client.19782 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff86541f740 10 client.19782 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff86541f740 20 client.19782 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff86541f740 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5625154bd840 0x5625154bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff86541f740 4 client.19782 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff86541f740 1 -- 172.21.15.105:0/2578396280 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625153ab200 con 0x5625154bd840 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.754+0000 7ff860cfd640 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5625154bd840 0x5625154bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff860cfd640 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5625154bd840 0x5625154bfc60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1c45fef7fe06c9 server_cookie=75ddd4ab95f4fd38 in_seq=0 out_seq=0 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff83b7fe640 10 client.19782 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff83b7fe640 1 -- 172.21.15.105:0/2578396280 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625153ab200 con 0x5625154bd840 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff83b7fe640 10 client.19782 handle_command_reply: tid=0 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 client.19782 shutdown 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 -- 172.21.15.105:0/2578396280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543f420 msgr2=0x56251544d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543f420 0x56251544d9c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff8500104d0 tx=0x7ff850005ce0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 -- 172.21.15.105:0/2578396280 shutdown_connections 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5625154bd840 0x5625154bfc60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562515444590 0x56251544e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff83b7fe640 0 client.19782 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:50:00.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56251543f420 0x56251544d9c0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 --2- 172.21.15.105:0/2578396280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56251543ea30 0x56251544d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 -- 172.21.15.105:0/2578396280 >> 172.21.15.105:0/2578396280 conn(0x562515439fa0 msgr2=0x5625154409b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 -- 172.21.15.105:0/2578396280 shutdown_connections 2024-03-26T17:50:00.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 1 -- 172.21.15.105:0/2578396280 wait complete. 2024-03-26T17:50:00.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.755+0000 7ff86541f740 20 client.19782 trim_cache size 0 max 16384 2024-03-26T17:50:00.754 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 -- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8440a2f20 msgr2=0x7ff8440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8440a2f20 0x7ff8440a53e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0ef340 tx=0x7ff84c0f6040 comp rx=0 tx=0).stop 2024-03-26T17:50:00.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 -- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 msgr2=0x7ff85c196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 0x7ff85c196a70 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff84c000c00 tx=0x7ff84c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 -- 172.21.15.105:0/1223418100 shutdown_connections 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8440a2f20 0x7ff8440a53e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff85c102370 0x7ff85c19e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff85c1019b0 0x7ff85c196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 --2- 172.21.15.105:0/1223418100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff85c1007c0 0x7ff85c196a70 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 -- 172.21.15.105:0/1223418100 >> 172.21.15.105:0/1223418100 conn(0x7ff85c0fbd50 msgr2=0x7ff85c104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 -- 172.21.15.105:0/1223418100 shutdown_connections 2024-03-26T17:50:00.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.756+0000 7ff862760640 1 -- 172.21.15.105:0/1223418100 wait complete. 2024-03-26T17:50:00.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:50:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 -- 172.21.15.105:0/1343611692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 msgr2=0x7f06dc1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 --2- 172.21.15.105:0/1343611692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 0x7f06dc1099f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009ca0 tx=0x7f06d402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 -- 172.21.15.105:0/1343611692 shutdown_connections 2024-03-26T17:50:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 --2- 172.21.15.105:0/1343611692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 0x7f06dc1099f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 --2- 172.21.15.105:0/1343611692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06dc104c70 0x7f06dc107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 --2- 172.21.15.105:0/1343611692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc102340 0x7f06dc104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 -- 172.21.15.105:0/1343611692 >> 172.21.15.105:0/1343611692 conn(0x7f06dc0fbd50 msgr2=0x7f06dc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 -- 172.21.15.105:0/1343611692 shutdown_connections 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 -- 172.21.15.105:0/1343611692 wait complete. 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 Processor -- start 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.886+0000 7f06e39cd640 1 -- start start 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e39cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06dc102340 0x7f06dc101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e39cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc104c70 0x7f06dc1002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e39cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 0x7f06dc100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e39cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc10d3f0 con 0x7f06dc1075a0 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e39cd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc10d270 con 0x7f06dc102340 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e39cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc10d570 con 0x7f06dc104c70 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 0x7f06dc100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1769640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06dc102340 0x7f06dc101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 0x7f06dc100830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44230/0 (socket says 172.21.15.105:44230) 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 -- 172.21.15.105:0/4091046161 learned_addr learned my addr 172.21.15.105:0/4091046161 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e0f68640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc104c70 0x7f06dc1002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 -- 172.21.15.105:0/4091046161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc104c70 msgr2=0x7f06dc1002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc104c70 0x7f06dc1002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 -- 172.21.15.105:0/4091046161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06dc102340 msgr2=0x7f06dc101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06dc102340 0x7f06dc101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 -- 172.21.15.105:0/4091046161 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc1010c0 con 0x7f06dc1075a0 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1769640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06dc102340 0x7f06dc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06e1f6a640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 0x7f06dc100830 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f06d402c700 tx=0x7f06d40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.887+0000 7f06d27fc640 1 -- 172.21.15.105:0/4091046161 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4041020 con 0x7f06dc1075a0 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.888+0000 7f06d27fc640 1 -- 172.21.15.105:0/4091046161 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d4003e20 con 0x7f06dc1075a0 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.888+0000 7f06e39cd640 1 -- 172.21.15.105:0/4091046161 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06dc1a1bb0 con 0x7f06dc1075a0 2024-03-26T17:50:00.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.888+0000 7f06d27fc640 1 -- 172.21.15.105:0/4091046161 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4039040 con 0x7f06dc1075a0 2024-03-26T17:50:00.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.888+0000 7f06e39cd640 1 -- 172.21.15.105:0/4091046161 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06dc1a2100 con 0x7f06dc1075a0 2024-03-26T17:50:00.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.890+0000 7f06e65b0740 1 Processor -- start 2024-03-26T17:50:00.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06d27fc640 1 -- 172.21.15.105:0/4091046161 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f06d4044020 con 0x7f06dc1075a0 2024-03-26T17:50:00.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e65b0740 1 -- start start 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda7ea30 0x557edda7ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda83f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda84590 0x557edda86a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e65b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557edd9e3970 con 0x557edda7ea30 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e65b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557edd98bd70 con 0x557edda84590 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e65b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557edd9df920 con 0x557edda7f420 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda7ea30 0x557edda7ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e1769640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda83f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda7ea30 0x557edda7ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44242/0 (socket says 172.21.15.105:44242) 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e1769640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda83f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34080/0 (socket says 172.21.15.105:34080) 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e0f68640 1 -- 172.21.15.105:0/2393978453 learned_addr learned my addr 172.21.15.105:0/2393978453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.891+0000 7f06e0f68640 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda84590 0x557edda86a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06d27fc640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06c80a2f20 0x7f06c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06e0f68640 1 -- 172.21.15.105:0/2393978453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 msgr2=0x557edda83f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06e0f68640 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda83f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06e0f68640 1 -- 172.21.15.105:0/2393978453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda7ea30 msgr2=0x557edda7ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06e0f68640 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda7ea30 0x557edda7ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06e0f68640 1 -- 172.21.15.105:0/2393978453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557edd6f73f0 con 0x557edda84590 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06e1769640 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda83f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06e1f6a640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06c80a2f20 0x7f06c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06d27fc640 1 -- 172.21.15.105:0/4091046161 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f06d40f15c0 con 0x7f06dc1075a0 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06e0f68640 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda84590 0x557edda86a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009fd0 tx=0x7f06cc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a7c99c94ad1ca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06c77fe640 1 -- 172.21.15.105:0/2393978453 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc011df0 con 0x557edda84590 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06c77fe640 1 -- 172.21.15.105:0/2393978453 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06cc00f790 con 0x557edda84590 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.892+0000 7f06c77fe640 1 -- 172.21.15.105:0/2393978453 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0044e0 con 0x557edda84590 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 -- 172.21.15.105:0/2393978453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda84590 msgr2=0x557edda86a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda84590 0x557edda86a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009fd0 tx=0x7f06cc005f90 comp rx=0 tx=0).stop 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e1f6a640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06c80a2f20 0x7f06c80a53e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f06d40c6190 tx=0x7f06d4039bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 -- 172.21.15.105:0/2393978453 shutdown_connections 2024-03-26T17:50:00.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda84590 0x557edda86a20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda83f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2393978453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda7ea30 0x557edda7ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 -- 172.21.15.105:0/2393978453 >> 172.21.15.105:0/2393978453 conn(0x557edda79fa0 msgr2=0x557edda7a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 -- 172.21.15.105:0/2393978453 shutdown_connections 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 -- 172.21.15.105:0/2393978453 wait complete. 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 -- start start 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.893+0000 7f06e65b0740 1 Processor -- start 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda89500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e65b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557edda87540 con 0x557edda84590 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e65b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557edd99f310 con 0x557edda7ea30 2024-03-26T17:50:00.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e65b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557edda876c0 con 0x557edda7f420 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e65b0740 10 client.0 _ll_register_callbacks cb 0x557edd9dac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44250/0 (socket says 172.21.15.105:44250) 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 -- 172.21.15.105:0/2080114902 learned_addr learned my addr 172.21.15.105:0/2080114902 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e1769640 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda89500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 -- 172.21.15.105:0/2080114902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 msgr2=0x557edda89500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda89500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 -- 172.21.15.105:0/2080114902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 msgr2=0x557edda88fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 -- 172.21.15.105:0/2080114902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557edd7c17d0 con 0x557edda84590 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06e0f68640 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90540 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f06cc001eb0 tx=0x7f06cc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda236b572e61462 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.894+0000 7f06c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06c4ff9640 1 -- 172.21.15.105:0/2080114902 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc026020 con 0x557edda84590 2024-03-26T17:50:00.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06c4ff9640 1 -- 172.21.15.105:0/2080114902 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f06cc004030 con 0x557edda84590 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 1 -- 172.21.15.105:0/2080114902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06cc00f790 con 0x557edda84590 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06c4ff9640 1 -- 172.21.15.105:0/2080114902 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f06cc004230 con 0x557edda84590 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 10 client.19794 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 1 -- 172.21.15.105:0/2080114902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557edda938b0 con 0x557edda84590 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06c4ff9640 1 -- 172.21.15.105:0/2080114902 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06cc00f790 con 0x557edda84590 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06c4ff9640 1 -- 172.21.15.105:0/2080114902 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f06cc01d020 con 0x557edda84590 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 10 client.19794 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 10 client.19794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 20 client.19794 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557edda94df0 0x557edda971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 4 client.19794 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e65b0740 1 -- 172.21.15.105:0/2080114902 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557edd9f0eb0 con 0x557edda94df0 2024-03-26T17:50:00.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.895+0000 7f06e1769640 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557edda94df0 0x557edda971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.896+0000 7f06c4ff9640 10 client.19794 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.896+0000 7f06e1769640 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557edda94df0 0x557edda971e0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2364285c7c7145 server_cookie=78908f14d30320b in_seq=0 out_seq=0 2024-03-26T17:50:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.896+0000 7f06c4ff9640 1 -- 172.21.15.105:0/2080114902 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557edd9f0eb0 con 0x557edda94df0 2024-03-26T17:50:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.896+0000 7f06c4ff9640 10 client.19794 handle_command_reply: tid=0 2024-03-26T17:50:00.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 client.19794 shutdown 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 -- 172.21.15.105:0/2080114902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 msgr2=0x557edda90540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90540 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f06cc001eb0 tx=0x7f06cc01a040 comp rx=0 tx=0).stop 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 -- 172.21.15.105:0/2080114902 shutdown_connections 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557edda94df0 0x557edda971e0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90540 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06c4ff9640 0 client.19794 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda89500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 --2- 172.21.15.105:0/2080114902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 -- 172.21.15.105:0/2080114902 >> 172.21.15.105:0/2080114902 conn(0x557edda79fa0 msgr2=0x557edda791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 -- 172.21.15.105:0/2080114902 shutdown_connections 2024-03-26T17:50:00.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 1 -- 172.21.15.105:0/2080114902 wait complete. 2024-03-26T17:50:00.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.897+0000 7f06e65b0740 20 client.19794 trim_cache size 0 max 16384 2024-03-26T17:50:00.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e65b0740 1 Processor -- start 2024-03-26T17:50:00.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e65b0740 1 -- start start 2024-03-26T17:50:00.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7f420 0x557edda89500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e65b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc1aaf20 con 0x557edda84590 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e65b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc1aca60 con 0x557edda7f420 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e65b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc1ab5f0 con 0x557edda7ea30 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e1769640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7f420 0x557edda89500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e1769640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7f420 0x557edda89500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37328/0 (socket says 172.21.15.105:37328) 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34106/0 (socket says 172.21.15.105:34106) 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e1769640 1 -- 172.21.15.105:0/1659770018 learned_addr learned my addr 172.21.15.105:0/1659770018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e1769640 1 -- 172.21.15.105:0/1659770018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7ea30 msgr2=0x557edda88fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e1769640 1 --2- 172.21.15.105:0/1659770018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e1769640 1 -- 172.21.15.105:0/1659770018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 msgr2=0x557edda90500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e1769640 1 --2- 172.21.15.105:0/1659770018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.920+0000 7f06e1769640 1 -- 172.21.15.105:0/1659770018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557edda7ad10 con 0x557edda7f420 2024-03-26T17:50:00.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.921+0000 7f06e1769640 1 --2- 172.21.15.105:0/1659770018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7f420 0x557edda89500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x557edda7f820 tx=0x7f06d8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7bdb6e191700187 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.921+0000 7f06c6ffd640 1 -- 172.21.15.105:0/1659770018 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8004220 con 0x557edda7f420 2024-03-26T17:50:00.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.921+0000 7f06c6ffd640 1 -- 172.21.15.105:0/1659770018 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d8025070 con 0x557edda7f420 2024-03-26T17:50:00.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.921+0000 7f06c6ffd640 1 -- 172.21.15.105:0/1659770018 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8021720 con 0x557edda7f420 2024-03-26T17:50:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 -- 172.21.15.105:0/1659770018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7f420 msgr2=0x557edda89500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1659770018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7f420 0x557edda89500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x557edda7f820 tx=0x7f06d8005f90 comp rx=0 tx=0).stop 2024-03-26T17:50:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 -- 172.21.15.105:0/1659770018 shutdown_connections 2024-03-26T17:50:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1659770018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda90500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1659770018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7f420 0x557edda89500 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1659770018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7ea30 0x557edda88fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 -- 172.21.15.105:0/1659770018 >> 172.21.15.105:0/1659770018 conn(0x557edda79fa0 msgr2=0x557edda809c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 -- 172.21.15.105:0/1659770018 shutdown_connections 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 -- 172.21.15.105:0/1659770018 wait complete. 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 -- start start 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.922+0000 7f06e65b0740 1 Processor -- start 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 0x557edda902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda8cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e65b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda8d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e65b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc1ac390 con 0x557edda84590 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e65b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc1abcc0 con 0x557edda7ea30 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e65b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc1a8f00 con 0x557edda7f420 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e65b0740 10 client.0 _ll_register_callbacks cb 0x7f06cc009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda8d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e1769640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda8cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda8d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44268/0 (socket says 172.21.15.105:44268) 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 -- 172.21.15.105:0/1378838250 learned_addr learned my addr 172.21.15.105:0/1378838250 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 0x557edda902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 -- 172.21.15.105:0/1378838250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 msgr2=0x557edda8cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda8cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 -- 172.21.15.105:0/1378838250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 msgr2=0x557edda902a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 0x557edda902a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 -- 172.21.15.105:0/1378838250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557edda8d870 con 0x557edda84590 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e1769640 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda8cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06e0f68640 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda8d280 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00fcb0 tx=0x7f06cc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5e8d20e98e7868 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.923+0000 7f06c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06c4ff9640 1 -- 172.21.15.105:0/1378838250 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc026070 con 0x557edda84590 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06c4ff9640 1 -- 172.21.15.105:0/1378838250 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f06cc00ed00 con 0x557edda84590 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 1 -- 172.21.15.105:0/1378838250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d8025070 con 0x557edda84590 2024-03-26T17:50:00.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06c4ff9640 1 -- 172.21.15.105:0/1378838250 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f06cc00e860 con 0x557edda84590 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 10 client.19800 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 1 -- 172.21.15.105:0/1378838250 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557eddafa160 con 0x557edda84590 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06c4ff9640 1 -- 172.21.15.105:0/1378838250 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06d8025070 con 0x557edda84590 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06c4ff9640 1 -- 172.21.15.105:0/1378838250 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f06cc038040 con 0x557edda84590 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 10 client.19800 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 10 client.19800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 20 client.19800 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557edda94df0 0x557eddafca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 4 client.19800 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e65b0740 1 -- 172.21.15.105:0/1378838250 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557edd9eb200 con 0x557edda94df0 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.924+0000 7f06e1769640 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557edda94df0 0x557eddafca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.925+0000 7f06c4ff9640 10 client.19800 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:00.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.925+0000 7f06e1769640 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557edda94df0 0x557eddafca00 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1f8373ad62c52b server_cookie=c4dc909fc5c6c706 in_seq=0 out_seq=0 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.925+0000 7f06c4ff9640 1 -- 172.21.15.105:0/1378838250 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557edd9eb200 con 0x557edda94df0 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.925+0000 7f06c4ff9640 10 client.19800 handle_command_reply: tid=0 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 client.19800 shutdown 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 -- 172.21.15.105:0/1378838250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 msgr2=0x557edda8d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda8d280 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f06cc00fcb0 tx=0x7f06cc009f30 comp rx=0 tx=0).stop 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 -- 172.21.15.105:0/1378838250 shutdown_connections 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557edda94df0 0x557eddafca00 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557edda84590 0x557edda8d280 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557edda7f420 0x557edda8cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06c4ff9640 0 client.19800 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 --2- 172.21.15.105:0/1378838250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557edda7ea30 0x557edda902a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 -- 172.21.15.105:0/1378838250 >> 172.21.15.105:0/1378838250 conn(0x557edda79fa0 msgr2=0x557edda88fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 -- 172.21.15.105:0/1378838250 shutdown_connections 2024-03-26T17:50:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 1 -- 172.21.15.105:0/1378838250 wait complete. 2024-03-26T17:50:00.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e65b0740 20 client.19800 trim_cache size 0 max 16384 2024-03-26T17:50:00.924 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.926+0000 7f06e39cd640 1 -- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06c80a2f20 msgr2=0x7f06c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06c80a2f20 0x7f06c80a53e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f06d40c6190 tx=0x7f06d4039bb0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 -- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 msgr2=0x7f06dc100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 0x7f06dc100830 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f06d402c700 tx=0x7f06d40328a0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 -- 172.21.15.105:0/4091046161 shutdown_connections 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06c80a2f20 0x7f06c80a53e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06dc1075a0 0x7f06dc100830 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc104c70 0x7f06dc1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 --2- 172.21.15.105:0/4091046161 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06dc102340 0x7f06dc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 -- 172.21.15.105:0/4091046161 >> 172.21.15.105:0/4091046161 conn(0x7f06dc0fbd50 msgr2=0x7f06dc102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:00.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 -- 172.21.15.105:0/4091046161 shutdown_connections 2024-03-26T17:50:00.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:00.927+0000 7f06e39cd640 1 -- 172.21.15.105:0/4091046161 wait complete. 2024-03-26T17:50:01.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 377024 3.14 MB/sec execute 524 sec latency 20.554 ms 2024-03-26T17:50:01.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:01 smithi144 ceph-mon[36095]: pgmap v699: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:50:01.641 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:01 smithi105 ceph-mon[32280]: pgmap v699: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:50:01.664 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:01 smithi135 ceph-mon[35636]: pgmap v699: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.5 MiB/s wr, 112 op/s 2024-03-26T17:50:01.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:50:01.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:50:01.932 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:50:02.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 377633 3.14 MB/sec execute 525 sec latency 16.762 ms 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 -- 172.21.15.105:0/3314299994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 msgr2=0x7ff424105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 --2- 172.21.15.105:0/3314299994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 0x7ff424105db0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff414009900 tx=0x7ff41402bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 -- 172.21.15.105:0/3314299994 shutdown_connections 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 --2- 172.21.15.105:0/3314299994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff424106450 0x7ff424108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 --2- 172.21.15.105:0/3314299994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 0x7ff424105db0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 --2- 172.21.15.105:0/3314299994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff424101000 0x7ff4241033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 -- 172.21.15.105:0/3314299994 >> 172.21.15.105:0/3314299994 conn(0x7ff4240fb080 msgr2=0x7ff4240fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 -- 172.21.15.105:0/3314299994 shutdown_connections 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.066+0000 7ff42bf37640 1 -- 172.21.15.105:0/3314299994 wait complete. 2024-03-26T17:50:02.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42bf37640 1 Processor -- start 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42bf37640 1 -- start start 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42bf37640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff424101000 0x7ff424197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42bf37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 0x7ff4241980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42bf37640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff424106450 0x7ff42419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42bf37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff42410c300 con 0x7ff4241039c0 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42bf37640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff42410c180 con 0x7ff424101000 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42bf37640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff42410c480 con 0x7ff424106450 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42a4d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff424106450 0x7ff42419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff4294d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 0x7ff4241980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff429cd3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff424101000 0x7ff424197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42a4d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff424106450 0x7ff42419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34132/0 (socket says 172.21.15.105:34132) 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42a4d4640 1 -- 172.21.15.105:0/4076158926 learned_addr learned my addr 172.21.15.105:0/4076158926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff4294d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 0x7ff4241980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44278/0 (socket says 172.21.15.105:44278) 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42a4d4640 1 -- 172.21.15.105:0/4076158926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff424101000 msgr2=0x7ff424197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff424101000 0x7ff424197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42a4d4640 1 -- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 msgr2=0x7ff4241980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 0x7ff4241980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.067+0000 7ff42a4d4640 1 -- 172.21.15.105:0/4076158926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42419f7a0 con 0x7ff424106450 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.068+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff424106450 0x7ff42419f130 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff41c010ea0 tx=0x7ff41c010f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.068+0000 7ff41affd640 1 -- 172.21.15.105:0/4076158926 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c004b20 con 0x7ff424106450 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.068+0000 7ff42bf37640 1 -- 172.21.15.105:0/4076158926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff424101440 con 0x7ff424106450 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.068+0000 7ff41affd640 1 -- 172.21.15.105:0/4076158926 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff41c004cc0 con 0x7ff424106450 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.068+0000 7ff41affd640 1 -- 172.21.15.105:0/4076158926 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c019700 con 0x7ff424106450 2024-03-26T17:50:02.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.068+0000 7ff42bf37640 1 -- 172.21.15.105:0/4076158926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff424101910 con 0x7ff424106450 2024-03-26T17:50:02.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.070+0000 7ff41affd640 1 -- 172.21.15.105:0/4076158926 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff41c005020 con 0x7ff424106450 2024-03-26T17:50:02.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.071+0000 7ff41affd640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4100a2f20 0x7ff4100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.071+0000 7ff41affd640 1 -- 172.21.15.105:0/4076158926 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff41c0d2fe0 con 0x7ff424106450 2024-03-26T17:50:02.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.071+0000 7ff429cd3640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4100a2f20 0x7ff4100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.071+0000 7ff429cd3640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4100a2f20 0x7ff4100a53e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff4240681a0 tx=0x7ff420005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.071+0000 7ff42ebf6740 1 Processor -- start 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.071+0000 7ff42ebf6740 1 -- start start 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc1e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc21180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a29cb7e630 con 0x55a29cc19190 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a29cb26e10 con 0x55a29cc1ecf0 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a29cb7a5e0 con 0x55a29cc19b80 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc195b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44290/0 (socket says 172.21.15.105:44290) 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 -- 172.21.15.105:0/4208822325 learned_addr learned my addr 172.21.15.105:0/4208822325 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff4294d2640 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc1e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 -- 172.21.15.105:0/4208822325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 msgr2=0x55a29cc1e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc1e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 -- 172.21.15.105:0/4208822325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 msgr2=0x55a29cc21180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc21180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 -- 172.21.15.105:0/4208822325 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29cb3d740 con 0x55a29cc19190 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff4294d2640 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc1e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff429cd3640 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc195b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff42000e630 tx=0x7ff420014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7dacf729dc083 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff40ffff640 1 -- 172.21.15.105:0/4208822325 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420026430 con 0x55a29cc19190 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff40ffff640 1 -- 172.21.15.105:0/4208822325 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff42001c8c0 con 0x55a29cc19190 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.072+0000 7ff40ffff640 1 -- 172.21.15.105:0/4208822325 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42001d920 con 0x55a29cc19190 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 -- 172.21.15.105:0/4208822325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 msgr2=0x55a29cc195b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc195b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff42000e630 tx=0x7ff420014000 comp rx=0 tx=0).stop 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 -- 172.21.15.105:0/4208822325 shutdown_connections 2024-03-26T17:50:02.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc21180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc1e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/4208822325 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc195b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 -- 172.21.15.105:0/4208822325 >> 172.21.15.105:0/4208822325 conn(0x55a29cc14700 msgr2=0x55a29cc14b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 -- 172.21.15.105:0/4208822325 shutdown_connections 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.073+0000 7ff42ebf6740 1 -- 172.21.15.105:0/4208822325 wait complete. 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 1 -- start start 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 1 Processor -- start 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc23670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 0x55a29cc23bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc2abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a29cc21ca0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a29cb3a1a0 con 0x55a29cc19b80 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a29cc21e20 con 0x55a29cc19190 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42ebf6740 10 client.0 _ll_register_callbacks cb 0x55a29c891c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:02.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff429cd3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc23670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42a4d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc2abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff4294d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 0x55a29cc23bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff429cd3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc23670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34148/0 (socket says 172.21.15.105:34148) 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff42a4d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc2abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44306/0 (socket says 172.21.15.105:44306) 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.074+0000 7ff429cd3640 1 -- 172.21.15.105:0/3980551427 learned_addr learned my addr 172.21.15.105:0/3980551427 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42a4d4640 1 -- 172.21.15.105:0/3980551427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 msgr2=0x55a29cc23670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc23670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42a4d4640 1 -- 172.21.15.105:0/3980551427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 msgr2=0x55a29cc23bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 0x55a29cc23bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42a4d4640 1 -- 172.21.15.105:0/3980551427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29c95a7e0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc2abf0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0d0aa0 tx=0x7ff41c09f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66aa51eed52a6bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff40d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:02.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff40d7fa640 1 -- 172.21.15.105:0/3980551427 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0055f0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff40d7fa640 1 -- 172.21.15.105:0/3980551427 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff41c005310 con 0x55a29cc1ecf0 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42ebf6740 1 -- 172.21.15.105:0/3980551427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42001c8c0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff40d7fa640 1 -- 172.21.15.105:0/3980551427 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff41c0be170 con 0x55a29cc1ecf0 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42ebf6740 10 client.19812 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff42ebf6740 1 -- 172.21.15.105:0/3980551427 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a29c9baf70 con 0x55a29cc1ecf0 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.075+0000 7ff40d7fa640 1 -- 172.21.15.105:0/3980551427 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff42001c8c0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff40d7fa640 1 -- 172.21.15.105:0/3980551427 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff41c09b040 con 0x55a29cc1ecf0 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff42ebf6740 10 client.19812 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff42ebf6740 10 client.19812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff42ebf6740 20 client.19812 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a29cc2f590 0x55a29cc319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff42ebf6740 4 client.19812 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff42ebf6740 1 -- 172.21.15.105:0/3980551427 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a29cb8bb70 con 0x55a29cc2f590 2024-03-26T17:50:02.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff429cd3640 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a29cc2f590 0x55a29cc319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff40d7fa640 10 client.19812 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:02.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.076+0000 7ff429cd3640 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a29cc2f590 0x55a29cc319d0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b583840ac498938f server_cookie=7a7c8b16232e7de4 in_seq=0 out_seq=0 2024-03-26T17:50:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff40d7fa640 1 -- 172.21.15.105:0/3980551427 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a29cb8bb70 con 0x55a29cc2f590 2024-03-26T17:50:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff40d7fa640 10 client.19812 handle_command_reply: tid=0 2024-03-26T17:50:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 client.19812 shutdown 2024-03-26T17:50:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 -- 172.21.15.105:0/3980551427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 msgr2=0x55a29cc2abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc2abf0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0d0aa0 tx=0x7ff41c09f780 comp rx=0 tx=0).stop 2024-03-26T17:50:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 -- 172.21.15.105:0/3980551427 shutdown_connections 2024-03-26T17:50:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a29cc2f590 0x55a29cc319d0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff40d7fa640 0 client.19812 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc2abf0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 0x55a29cc23bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/3980551427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc23670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 -- 172.21.15.105:0/3980551427 >> 172.21.15.105:0/3980551427 conn(0x55a29cc14700 msgr2=0x55a29cc13900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 -- 172.21.15.105:0/3980551427 shutdown_connections 2024-03-26T17:50:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 1 -- 172.21.15.105:0/3980551427 wait complete. 2024-03-26T17:50:02.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.079+0000 7ff42ebf6740 20 client.19812 trim_cache size 0 max 16384 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.098+0000 7ff42ebf6740 1 Processor -- start 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.098+0000 7ff42ebf6740 1 -- start start 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc13070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 0x55a29cc2abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a6bb0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a86f0 con 0x55a29cc19b80 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a7280 con 0x55a29cc19190 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff429cd3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc13070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44312/0 (socket says 172.21.15.105:44312) 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 -- 172.21.15.105:0/382583774 learned_addr learned my addr 172.21.15.105:0/382583774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff4294d2640 1 --2- 172.21.15.105:0/382583774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 0x55a29cc2abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 -- 172.21.15.105:0/382583774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 msgr2=0x55a29cc13070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/382583774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc13070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 -- 172.21.15.105:0/382583774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 msgr2=0x55a29cc2abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/382583774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 0x55a29cc2abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 -- 172.21.15.105:0/382583774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a29c852170 con 0x55a29cc1ecf0 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/382583774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc135b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff41c09f710 tx=0x7ff41c09dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf7ffa84784862b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff40f7fe640 1 -- 172.21.15.105:0/382583774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0055f0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff40f7fe640 1 -- 172.21.15.105:0/382583774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff41c09bdf0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.099+0000 7ff40f7fe640 1 -- 172.21.15.105:0/382583774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c0a49b0 con 0x55a29cc1ecf0 2024-03-26T17:50:02.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 -- 172.21.15.105:0/382583774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 msgr2=0x55a29cc135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/382583774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc135b0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff41c09f710 tx=0x7ff41c09dc90 comp rx=0 tx=0).stop 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 -- 172.21.15.105:0/382583774 shutdown_connections 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/382583774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc1ecf0 0x55a29cc135b0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/382583774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc19b80 0x55a29cc2abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/382583774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19190 0x55a29cc13070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 -- 172.21.15.105:0/382583774 >> 172.21.15.105:0/382583774 conn(0x55a29cc14700 msgr2=0x55a29cc1b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 -- 172.21.15.105:0/382583774 shutdown_connections 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 -- 172.21.15.105:0/382583774 wait complete. 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.100+0000 7ff42ebf6740 1 -- start start 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42ebf6740 1 Processor -- start 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc28af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc29030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42ebf6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc29ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a8020 con 0x55a29cc19190 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a7950 con 0x55a29cc1ecf0 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42ebf6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241a4b90 con 0x55a29cc19b80 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42ebf6740 10 client.0 _ll_register_callbacks cb 0x7ff41c09f300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff4294d2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc29030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff4294d2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc29030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34170/0 (socket says 172.21.15.105:34170) 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff4294d2640 1 -- 172.21.15.105:0/662823259 learned_addr learned my addr 172.21.15.105:0/662823259 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc29ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff429cd3640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc28af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff4294d2640 1 -- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 msgr2=0x55a29cc29ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff4294d2640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc29ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff4294d2640 1 -- 172.21.15.105:0/662823259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 msgr2=0x55a29cc28af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff4294d2640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc28af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff4294d2640 1 -- 172.21.15.105:0/662823259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a29cc2a5e0 con 0x55a29cc19b80 2024-03-26T17:50:02.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff42a4d4640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc29ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.101+0000 7ff429cd3640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc28af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff4294d2640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc29030 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff414009900 tx=0x7ff414004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3d6169309afcf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff40d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff40d7fa640 1 -- 172.21.15.105:0/662823259 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41404e060 con 0x55a29cc19b80 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff42ebf6740 1 -- 172.21.15.105:0/662823259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff41c09bdf0 con 0x55a29cc19b80 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff40d7fa640 1 -- 172.21.15.105:0/662823259 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff41403a040 con 0x55a29cc19b80 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff40d7fa640 1 -- 172.21.15.105:0/662823259 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff414005030 con 0x55a29cc19b80 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff42ebf6740 10 client.29605 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff42ebf6740 1 -- 172.21.15.105:0/662823259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a29cc96180 con 0x55a29cc19b80 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff40d7fa640 1 -- 172.21.15.105:0/662823259 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff41c09bdf0 con 0x55a29cc19b80 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff40d7fa640 1 -- 172.21.15.105:0/662823259 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff414048070 con 0x55a29cc19b80 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff42ebf6740 10 client.29605 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.102+0000 7ff42ebf6740 10 client.29605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:02.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.103+0000 7ff42ebf6740 20 client.29605 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.103+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a29cc2f590 0x55a29cc98ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.103+0000 7ff42ebf6740 4 client.29605 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:50:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.103+0000 7ff42ebf6740 1 -- 172.21.15.105:0/662823259 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a29cc1ab90 con 0x55a29cc2f590 2024-03-26T17:50:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.103+0000 7ff429cd3640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a29cc2f590 0x55a29cc98ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.103+0000 7ff40d7fa640 10 client.29605 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:02.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.103+0000 7ff429cd3640 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a29cc2f590 0x55a29cc98ba0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86348855acf57e6b server_cookie=ec72dff321d1d9ea in_seq=0 out_seq=0 2024-03-26T17:50:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff40d7fa640 1 -- 172.21.15.105:0/662823259 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a29cc1ab90 con 0x55a29cc2f590 2024-03-26T17:50:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff40d7fa640 10 client.29605 handle_command_reply: tid=0 2024-03-26T17:50:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 client.29605 shutdown 2024-03-26T17:50:02.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 -- 172.21.15.105:0/662823259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 msgr2=0x55a29cc29030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc29030 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff414009900 tx=0x7ff414004270 comp rx=0 tx=0).stop 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 -- 172.21.15.105:0/662823259 shutdown_connections 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a29cc2f590 0x55a29cc98ba0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a29cc1ecf0 0x55a29cc29ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a29cc19b80 0x55a29cc29030 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 --2- 172.21.15.105:0/662823259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a29cc19190 0x55a29cc28af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 -- 172.21.15.105:0/662823259 >> 172.21.15.105:0/662823259 conn(0x55a29cc14700 msgr2=0x55a29cc17960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff40d7fa640 0 client.29605 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 -- 172.21.15.105:0/662823259 shutdown_connections 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 1 -- 172.21.15.105:0/662823259 wait complete. 2024-03-26T17:50:02.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.104+0000 7ff42ebf6740 20 client.29605 trim_cache size 0 max 16384 2024-03-26T17:50:02.104 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:50:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:50:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "fd065bba-6e97-4cf6-8033-4ad1aed23a00", 2024-03-26T17:50:02.104 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:50:02.104 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 -- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4100a2f20 msgr2=0x7ff4100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4100a2f20 0x7ff4100a53e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff4240681a0 tx=0x7ff420005f90 comp rx=0 tx=0).stop 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 -- 172.21.15.105:0/4076158926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff424106450 msgr2=0x7ff42419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff424106450 0x7ff42419f130 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff41c010ea0 tx=0x7ff41c010f90 comp rx=0 tx=0).stop 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 -- 172.21.15.105:0/4076158926 shutdown_connections 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4100a2f20 0x7ff4100a53e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff424106450 0x7ff42419f130 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4241039c0 0x7ff4241980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 --2- 172.21.15.105:0/4076158926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff424101000 0x7ff424197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 -- 172.21.15.105:0/4076158926 >> 172.21.15.105:0/4076158926 conn(0x7ff4240fb080 msgr2=0x7ff424100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 -- 172.21.15.105:0/4076158926 shutdown_connections 2024-03-26T17:50:02.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.105+0000 7ff42bf37640 1 -- 172.21.15.105:0/4076158926 wait complete. 2024-03-26T17:50:02.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:50:02.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:50:02.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.238+0000 7fd36adb2640 1 -- 172.21.15.105:0/4073928765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 msgr2=0x7fd364101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.238+0000 7fd36adb2640 1 --2- 172.21.15.105:0/4073928765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 0x7fd364101e30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fd350009900 tx=0x7fd35002bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.238+0000 7fd36adb2640 1 -- 172.21.15.105:0/4073928765 shutdown_connections 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.238+0000 7fd36adb2640 1 --2- 172.21.15.105:0/4073928765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd364102370 0x7fd364108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.238+0000 7fd36adb2640 1 --2- 172.21.15.105:0/4073928765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 0x7fd364101e30 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.238+0000 7fd36adb2640 1 --2- 172.21.15.105:0/4073928765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3641007c0 0x7fd364100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.238+0000 7fd36adb2640 1 -- 172.21.15.105:0/4073928765 >> 172.21.15.105:0/4073928765 conn(0x7fd3640fbd50 msgr2=0x7fd3640fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 -- 172.21.15.105:0/4073928765 shutdown_connections 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 -- 172.21.15.105:0/4073928765 wait complete. 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 Processor -- start 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 -- start start 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3641007c0 0x7fd364197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 0x7fd3641980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd364102370 0x7fd36419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd36410c2f0 con 0x7fd3641019b0 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd36410c170 con 0x7fd364102370 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36adb2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd36410c470 con 0x7fd3641007c0 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd363fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 0x7fd3641980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd368b4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3641007c0 0x7fd364197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd363fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 0x7fd3641980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44334/0 (socket says 172.21.15.105:44334) 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd363fff640 1 -- 172.21.15.105:0/2265632872 learned_addr learned my addr 172.21.15.105:0/2265632872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.239+0000 7fd36934f640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd364102370 0x7fd36419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd368b4e640 1 -- 172.21.15.105:0/2265632872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd364102370 msgr2=0x7fd36419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd368b4e640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd364102370 0x7fd36419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd368b4e640 1 -- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 msgr2=0x7fd3641980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd368b4e640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 0x7fd3641980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd368b4e640 1 -- 172.21.15.105:0/2265632872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36419f750 con 0x7fd3641007c0 2024-03-26T17:50:02.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd363fff640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 0x7fd3641980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:02.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd368b4e640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3641007c0 0x7fd364197b80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd35800e9c0 tx=0x7fd35800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd361ffb640 1 -- 172.21.15.105:0/2265632872 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35800ccc0 con 0x7fd3641007c0 2024-03-26T17:50:02.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd36adb2640 1 -- 172.21.15.105:0/2265632872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36419fa40 con 0x7fd3641007c0 2024-03-26T17:50:02.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd361ffb640 1 -- 172.21.15.105:0/2265632872 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3580035f0 con 0x7fd3641007c0 2024-03-26T17:50:02.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd361ffb640 1 -- 172.21.15.105:0/2265632872 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35800ce60 con 0x7fd3641007c0 2024-03-26T17:50:02.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.240+0000 7fd36adb2640 1 -- 172.21.15.105:0/2265632872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36419ffe0 con 0x7fd3641007c0 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.243+0000 7fd361ffb640 1 -- 172.21.15.105:0/2265632872 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd3580037a0 con 0x7fd3641007c0 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd361ffb640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd3540a2f20 0x7fd3540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36d9a0740 1 Processor -- start 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36d9a0740 1 -- start start 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd361ffb640 1 -- 172.21.15.105:0/2265632872 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd3580cf820 con 0x7fd3641007c0 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd363fff640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd3540a2f20 0x7fd3540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b18f0 0x5606a34b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a34b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b7450 0x5606a34b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5606a3416980 con 0x5606a34b7450 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606a33bebe0 con 0x5606a34b22e0 2024-03-26T17:50:02.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5606a3412930 con 0x5606a34b18f0 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36934f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b7450 0x5606a34b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36934f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b7450 0x5606a34b98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44342/0 (socket says 172.21.15.105:44342) 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd36934f640 1 -- 172.21.15.105:0/1066386890 learned_addr learned my addr 172.21.15.105:0/1066386890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.244+0000 7fd368b4e640 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b18f0 0x5606a34b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd368b4e640 1 -- 172.21.15.105:0/1066386890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 msgr2=0x5606a34b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd363fff640 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a34b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd368b4e640 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a34b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd368b4e640 1 -- 172.21.15.105:0/1066386890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b7450 msgr2=0x5606a34b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd368b4e640 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b7450 0x5606a34b98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd368b4e640 1 -- 172.21.15.105:0/1066386890 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606a31f47d0 con 0x5606a34b18f0 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd36934f640 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b7450 0x5606a34b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd363fff640 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a34b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd363fff640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd3540a2f20 0x7fd3540a53e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd364199040 tx=0x7fd350009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd368b4e640 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b18f0 0x5606a34b1d10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3580cd330 tx=0x7fd35809b970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e75f9b82d1f783 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd34effd640 1 -- 172.21.15.105:0/1066386890 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580a0060 con 0x5606a34b18f0 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd34effd640 1 -- 172.21.15.105:0/1066386890 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3580a4920 con 0x5606a34b18f0 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.245+0000 7fd34effd640 1 -- 172.21.15.105:0/1066386890 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358097ec0 con 0x5606a34b18f0 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1066386890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b18f0 msgr2=0x5606a34b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b18f0 0x5606a34b1d10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3580cd330 tx=0x7fd35809b970 comp rx=0 tx=0).stop 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1066386890 shutdown_connections 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b7450 0x5606a34b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a34b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1066386890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b18f0 0x5606a34b1d10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1066386890 >> 172.21.15.105:0/1066386890 conn(0x5606a34ace60 msgr2=0x5606a34ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1066386890 shutdown_connections 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1066386890 wait complete. 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 -- start start 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.246+0000 7fd36d9a0740 1 Processor -- start 2024-03-26T17:50:02.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b18f0 0x5606a34bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b22e0 0x5606a34bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b7450 0x5606a34c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5606a34ba400 con 0x5606a34b22e0 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606a33d2160 con 0x5606a34b18f0 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5606a34ba580 con 0x5606a34b7450 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36d9a0740 10 client.0 _ll_register_callbacks cb 0x5606a340e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd363fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b22e0 0x5606a34bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd368b4e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b18f0 0x5606a34bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd363fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b22e0 0x5606a34bc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44352/0 (socket says 172.21.15.105:44352) 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd363fff640 1 -- 172.21.15.105:0/2298199233 learned_addr learned my addr 172.21.15.105:0/2298199233 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36934f640 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b7450 0x5606a34c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd368b4e640 1 -- 172.21.15.105:0/2298199233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b7450 msgr2=0x5606a34c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd368b4e640 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b7450 0x5606a34c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd368b4e640 1 -- 172.21.15.105:0/2298199233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b22e0 msgr2=0x5606a34bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd368b4e640 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b22e0 0x5606a34bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd368b4e640 1 -- 172.21.15.105:0/2298199233 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606a312a3f0 con 0x5606a34b18f0 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd36934f640 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b7450 0x5606a34c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd363fff640 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b22e0 0x5606a34bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd368b4e640 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b18f0 0x5606a34bbb90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5606a34ba0d0 tx=0x7fd358097a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecdb1a9d95f80f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd343fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.247+0000 7fd343fff640 1 -- 172.21.15.105:0/2298199233 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35809b360 con 0x5606a34b18f0 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd343fff640 1 -- 172.21.15.105:0/2298199233 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd35809b500 con 0x5606a34b18f0 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 1 -- 172.21.15.105:0/2298199233 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3580a4920 con 0x5606a34b18f0 2024-03-26T17:50:02.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd343fff640 1 -- 172.21.15.105:0/2298199233 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd358097c60 con 0x5606a34b18f0 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 10 client.29633 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 1 -- 172.21.15.105:0/2298199233 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606a30ea0f0 con 0x5606a34b18f0 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd343fff640 1 -- 172.21.15.105:0/2298199233 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3580a4920 con 0x5606a34b18f0 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd343fff640 1 -- 172.21.15.105:0/2298199233 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd3580a11f0 con 0x5606a34b18f0 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 10 client.29633 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 10 client.29633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 20 client.29633 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5606a34c7570 0x5606a34c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 4 client.29633 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd36d9a0740 1 -- 172.21.15.105:0/2298199233 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606a3423ec0 con 0x5606a34c7570 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.248+0000 7fd363fff640 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5606a34c7570 0x5606a34c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.249+0000 7fd363fff640 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5606a34c7570 0x5606a34c9960 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81727471fcae4f47 server_cookie=ef9bda3a03bccc6 in_seq=0 out_seq=0 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.249+0000 7fd343fff640 10 client.29633 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:02.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd343fff640 1 -- 172.21.15.105:0/2298199233 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5606a3423ec0 con 0x5606a34c7570 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd343fff640 10 client.29633 handle_command_reply: tid=0 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 client.29633 shutdown 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 -- 172.21.15.105:0/2298199233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b18f0 msgr2=0x5606a34bbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b18f0 0x5606a34bbb90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5606a34ba0d0 tx=0x7fd358097a90 comp rx=0 tx=0).stop 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 -- 172.21.15.105:0/2298199233 shutdown_connections 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5606a34c7570 0x5606a34c9960 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b7450 0x5606a34c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b22e0 0x5606a34bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/2298199233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b18f0 0x5606a34bbb90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd343fff640 0 client.29633 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:02.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 -- 172.21.15.105:0/2298199233 >> 172.21.15.105:0/2298199233 conn(0x5606a34ace60 msgr2=0x5606a34ac080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 -- 172.21.15.105:0/2298199233 shutdown_connections 2024-03-26T17:50:02.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.250+0000 7fd36d9a0740 1 -- 172.21.15.105:0/2298199233 wait complete. 2024-03-26T17:50:02.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.251+0000 7fd36d9a0740 20 client.29633 trim_cache size 0 max 16384 2024-03-26T17:50:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.275+0000 7fd36d9a0740 1 Processor -- start 2024-03-26T17:50:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.275+0000 7fd36d9a0740 1 -- start start 2024-03-26T17:50:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34ace60 0x5606a34bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a34b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b22e0 0x5606a34ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641a8e80 con 0x5606a34b18f0 2024-03-26T17:50:02.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641aa9c0 con 0x5606a34ace60 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641a9550 con 0x5606a34b22e0 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a34b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a34b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44368/0 (socket says 172.21.15.105:44368) 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 -- 172.21.15.105:0/780159381 learned_addr learned my addr 172.21.15.105:0/780159381 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd368b4e640 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34ace60 0x5606a34bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd36934f640 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b22e0 0x5606a34ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 -- 172.21.15.105:0/780159381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b22e0 msgr2=0x5606a34ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b22e0 0x5606a34ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 -- 172.21.15.105:0/780159381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34ace60 msgr2=0x5606a34bc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34ace60 0x5606a34bc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 -- 172.21.15.105:0/780159381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606a34b2b70 con 0x5606a34b18f0 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd368b4e640 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34ace60 0x5606a34bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd363fff640 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a34b07d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd350005ab0 tx=0x7fd35002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b002e1de5bcd83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.276+0000 7fd34e7fc640 1 -- 172.21.15.105:0/780159381 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350049040 con 0x5606a34b18f0 2024-03-26T17:50:02.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd34e7fc640 1 -- 172.21.15.105:0/780159381 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd350005490 con 0x5606a34b18f0 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd34e7fc640 1 -- 172.21.15.105:0/780159381 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3500037e0 con 0x5606a34b18f0 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 -- 172.21.15.105:0/780159381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 msgr2=0x5606a34b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a34b07d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd350005ab0 tx=0x7fd35002c7e0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 -- 172.21.15.105:0/780159381 shutdown_connections 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34b22e0 0x5606a34ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a34b07d0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/780159381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34ace60 0x5606a34bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 -- 172.21.15.105:0/780159381 >> 172.21.15.105:0/780159381 conn(0x5606a34b6f20 msgr2=0x5606a34b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 -- 172.21.15.105:0/780159381 shutdown_connections 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.277+0000 7fd36d9a0740 1 -- 172.21.15.105:0/780159381 wait complete. 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 1 -- start start 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 1 Processor -- start 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34ace60 0x5606a3528dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a3529310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a35302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641aa2f0 con 0x5606a34b18f0 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641a9c20 con 0x5606a34b22e0 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3641a6e60 con 0x5606a34ace60 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36d9a0740 10 client.0 _ll_register_callbacks cb 0x7fd35c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd363fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a3529310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd368b4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34ace60 0x5606a3528dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd368b4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34ace60 0x5606a3528dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34234/0 (socket says 172.21.15.105:34234) 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd363fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a3529310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44376/0 (socket says 172.21.15.105:44376) 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd368b4e640 1 -- 172.21.15.105:0/1906814010 learned_addr learned my addr 172.21.15.105:0/1906814010 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.278+0000 7fd36934f640 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a35302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd368b4e640 1 -- 172.21.15.105:0/1906814010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 msgr2=0x5606a35302e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd368b4e640 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a35302e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd368b4e640 1 -- 172.21.15.105:0/1906814010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 msgr2=0x5606a3529310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd368b4e640 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a3529310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd368b4e640 1 -- 172.21.15.105:0/1906814010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606a35308d0 con 0x5606a34ace60 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd368b4e640 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34ace60 0x5606a3528dd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd3580cd650 tx=0x7fd3580a00e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2da1af1aed91923 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd343fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd343fff640 1 -- 172.21.15.105:0/1906814010 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3580981b0 con 0x5606a34ace60 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd343fff640 1 -- 172.21.15.105:0/1906814010 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd358097e00 con 0x5606a34ace60 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1906814010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd350005490 con 0x5606a34ace60 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd343fff640 1 -- 172.21.15.105:0/1906814010 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3580a4920 con 0x5606a34ace60 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd36d9a0740 10 client.29623 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.279+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1906814010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606a3531c50 con 0x5606a34ace60 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd343fff640 1 -- 172.21.15.105:0/1906814010 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd350005490 con 0x5606a34ace60 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd343fff640 1 -- 172.21.15.105:0/1906814010 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd35809ae40 con 0x5606a34ace60 2024-03-26T17:50:02.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd36d9a0740 10 client.29623 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd36d9a0740 10 client.29623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd36d9a0740 20 client.29623 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5606a34b7450 0x5606a3534520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd36d9a0740 4 client.29623 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1906814010 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606a341e210 con 0x5606a34b7450 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd363fff640 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5606a34b7450 0x5606a3534520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd363fff640 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5606a34b7450 0x5606a3534520 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ec7d3d7075002b server_cookie=c5c10713706e6179 in_seq=0 out_seq=0 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.280+0000 7fd343fff640 10 client.29623 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd343fff640 1 -- 172.21.15.105:0/1906814010 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5606a341e210 con 0x5606a34b7450 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd343fff640 10 client.29623 handle_command_reply: tid=0 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 client.29623 shutdown 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1906814010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34ace60 msgr2=0x5606a3528dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34ace60 0x5606a3528dd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd3580cd650 tx=0x7fd3580a00e0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1906814010 shutdown_connections 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5606a34b7450 0x5606a3534520 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5606a34b22e0 0x5606a35302e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5606a34b18f0 0x5606a3529310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd343fff640 0 client.29623 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 --2- 172.21.15.105:0/1906814010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5606a34ace60 0x5606a3528dd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1906814010 >> 172.21.15.105:0/1906814010 conn(0x5606a34b6f20 msgr2=0x5606a34b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1906814010 shutdown_connections 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 1 -- 172.21.15.105:0/1906814010 wait complete. 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.281+0000 7fd36d9a0740 20 client.29623 trim_cache size 0 max 16384 2024-03-26T17:50:02.280 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "fd065bba-6e97-4cf6-8033-4ad1aed23a00": { 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "fd065bba-6e97-4cf6-8033-4ad1aed23a00", 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:50:02.281 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:50:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 -- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd3540a2f20 msgr2=0x7fd3540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd3540a2f20 0x7fd3540a53e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd364199040 tx=0x7fd350009580 comp rx=0 tx=0).stop 2024-03-26T17:50:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 -- 172.21.15.105:0/2265632872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3641007c0 msgr2=0x7fd364197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3641007c0 0x7fd364197b80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd35800e9c0 tx=0x7fd35800ee80 comp rx=0 tx=0).stop 2024-03-26T17:50:02.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 -- 172.21.15.105:0/2265632872 shutdown_connections 2024-03-26T17:50:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd3540a2f20 0x7fd3540a53e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd364102370 0x7fd36419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3641019b0 0x7fd3641980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 --2- 172.21.15.105:0/2265632872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3641007c0 0x7fd364197b80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 -- 172.21.15.105:0/2265632872 >> 172.21.15.105:0/2265632872 conn(0x7fd3640fbd50 msgr2=0x7fd364068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 -- 172.21.15.105:0/2265632872 shutdown_connections 2024-03-26T17:50:02.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:02.282+0000 7fd36adb2640 1 -- 172.21.15.105:0/2265632872 wait complete. 2024-03-26T17:50:02.286 INFO:tasks.cephfs.filesystem:scrub status for tag:fd065bba-6e97-4cf6-8033-4ad1aed23a00 - {'path': '/', 'tag': 'fd065bba-6e97-4cf6-8033-4ad1aed23a00', 'options': 'recursive,force'} 2024-03-26T17:50:03.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 378373 3.14 MB/sec execute 526 sec latency 22.553 ms 2024-03-26T17:50:04.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 379042 3.14 MB/sec execute 527 sec latency 17.484 ms 2024-03-26T17:50:04.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:03 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:50:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:03 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:03 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:50:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:03 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:04.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:03 smithi144 ceph-mon[36095]: pgmap v700: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:50:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:03 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:50:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:03 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:03 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:50:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:03 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:03 smithi105 ceph-mon[32280]: pgmap v700: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:50:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:03 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:50:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:03 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:03 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:50:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:03 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:04.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:03 smithi135 ceph-mon[35636]: pgmap v700: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.4 MiB/s wr, 101 op/s 2024-03-26T17:50:05.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 379645 3.15 MB/sec execute 528 sec latency 15.026 ms 2024-03-26T17:50:06.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 380040 3.14 MB/sec execute 529 sec latency 16.994 ms 2024-03-26T17:50:06.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:05 smithi144 ceph-mon[36095]: pgmap v701: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:50:06.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:05 smithi105 ceph-mon[32280]: pgmap v701: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:50:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.161+0000 7f35e959a640 1 -- 172.21.15.105:0/889334925 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40328b0 con 0x7f35ec102390 2024-03-26T17:50:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 -- 172.21.15.105:0/889334925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec102390 msgr2=0x7f35ec108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/889334925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec102390 0x7f35ec108c20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009ca0 tx=0x7f35e402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 -- 172.21.15.105:0/889334925 shutdown_connections 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/889334925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec102390 0x7f35ec108c20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/889334925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec1019d0 0x7f35ec101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/889334925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec1007e0 0x7f35ec100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 -- 172.21.15.105:0/889334925 >> 172.21.15.105:0/889334925 conn(0x7f35ec0fbd50 msgr2=0x7f35ec0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 -- 172.21.15.105:0/889334925 shutdown_connections 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.162+0000 7f35f0ef8640 1 -- 172.21.15.105:0/889334925 wait complete. 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35f0ef8640 1 Processor -- start 2024-03-26T17:50:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35f0ef8640 1 -- start start 2024-03-26T17:50:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35f0ef8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec1007e0 0x7f35ec10cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35f0ef8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1019d0 0x7f35ec10b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35f0ef8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec102390 0x7f35ec10b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35f0ef8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec10d470 con 0x7f35ec102390 2024-03-26T17:50:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35f0ef8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec10d2f0 con 0x7f35ec1007e0 2024-03-26T17:50:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35f0ef8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec10d5f0 con 0x7f35ec1019d0 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35ea59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec1007e0 0x7f35ec10cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35e9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1019d0 0x7f35ec10b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35ea59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec1007e0 0x7f35ec10cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60500/0 (socket says 172.21.15.105:60500) 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35ea59c640 1 -- 172.21.15.105:0/132488185 learned_addr learned my addr 172.21.15.105:0/132488185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35ead9d640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec102390 0x7f35ec10b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35e9d9b640 1 -- 172.21.15.105:0/132488185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec1007e0 msgr2=0x7f35ec10cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec1007e0 0x7f35ec10cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35e9d9b640 1 -- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec102390 msgr2=0x7f35ec10b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec102390 0x7f35ec10b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.163+0000 7f35e9d9b640 1 -- 172.21.15.105:0/132488185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec10c070 con 0x7f35ec1019d0 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.164+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1019d0 0x7f35ec10b2a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35e000e9f0 tx=0x7f35e000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.164+0000 7f35db7fe640 1 -- 172.21.15.105:0/132488185 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e000ccc0 con 0x7f35ec1019d0 2024-03-26T17:50:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.164+0000 7f35db7fe640 1 -- 172.21.15.105:0/132488185 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e00108e0 con 0x7f35ec1019d0 2024-03-26T17:50:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.164+0000 7f35db7fe640 1 -- 172.21.15.105:0/132488185 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0018880 con 0x7f35ec1019d0 2024-03-26T17:50:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.164+0000 7f35f0ef8640 1 -- 172.21.15.105:0/132488185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec1a1b20 con 0x7f35ec1019d0 2024-03-26T17:50:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.164+0000 7f35f0ef8640 1 -- 172.21.15.105:0/132488185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec1a20c0 con 0x7f35ec1019d0 2024-03-26T17:50:06.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:05 smithi135 ceph-mon[35636]: pgmap v701: 97 pgs: 97 active+clean; 29 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.6 MiB/s wr, 114 op/s 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.166+0000 7f35db7fe640 1 -- 172.21.15.105:0/132488185 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f35e0010a90 con 0x7f35ec1019d0 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35f3bc3740 1 Processor -- start 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35f3bc3740 1 -- start start 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c65fc0 0x562ef5c6aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562ef5bca0b0 con 0x562ef5c6b130 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ef5b72960 con 0x562ef5c655d0 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ef5bc6060 con 0x562ef5c65fc0 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35db7fe640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d40a3040 0x7f35d40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35ead9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c65fc0 0x562ef5c6aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35ead9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60504/0 (socket says 172.21.15.105:60504) 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35ead9d640 1 -- 172.21.15.105:0/958214704 learned_addr learned my addr 172.21.15.105:0/958214704 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/958214704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35ead9d640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d40a3040 0x7f35d40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.167+0000 7f35db7fe640 1 -- 172.21.15.105:0/132488185 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f35e00cf6d0 con 0x7f35ec1019d0 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35ead9d640 1 -- 172.21.15.105:0/958214704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c65fc0 msgr2=0x562ef5c6aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35ead9d640 1 --2- 172.21.15.105:0/958214704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c65fc0 0x562ef5c6aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35ead9d640 1 -- 172.21.15.105:0/958214704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 msgr2=0x562ef5c6d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35ead9d640 1 --2- 172.21.15.105:0/958214704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c6d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35ead9d640 1 -- 172.21.15.105:0/958214704 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef5b84c60 con 0x562ef5c655d0 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35ead9d640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d40a3040 0x7f35d40a5500 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009a60 tx=0x7f35e4042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35ead9d640 1 --2- 172.21.15.105:0/958214704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c659f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f35e4005e30 tx=0x7f35e40328e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d5c619602b0857 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35d8ff9640 1 -- 172.21.15.105:0/958214704 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4051070 con 0x562ef5c655d0 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35d8ff9640 1 -- 172.21.15.105:0/958214704 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e4032cc0 con 0x562ef5c655d0 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.168+0000 7f35d8ff9640 1 -- 172.21.15.105:0/958214704 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4039040 con 0x562ef5c655d0 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 -- 172.21.15.105:0/958214704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 msgr2=0x562ef5c659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/958214704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c659f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f35e4005e30 tx=0x7f35e40328e0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 -- 172.21.15.105:0/958214704 shutdown_connections 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/958214704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/958214704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c65fc0 0x562ef5c6aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/958214704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c659f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 -- 172.21.15.105:0/958214704 >> 172.21.15.105:0/958214704 conn(0x562ef5c60b40 msgr2=0x562ef5c60f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 -- 172.21.15.105:0/958214704 shutdown_connections 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 -- 172.21.15.105:0/958214704 wait complete. 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.169+0000 7f35f3bc3740 1 -- start start 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35f3bc3740 1 Processor -- start 2024-03-26T17:50:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c655d0 0x562ef5c6f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c6f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5c76900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562ef5c6e080 con 0x562ef5c655d0 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ef5b85db0 con 0x562ef5c65fc0 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ef5c6e200 con 0x562ef5c6b130 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c655d0 0x562ef5c6f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35e9d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c6f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c655d0 0x562ef5c6f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36790/0 (socket says 172.21.15.105:36790) 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 -- 172.21.15.105:0/3418547378 learned_addr learned my addr 172.21.15.105:0/3418547378 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35f3bc3740 10 client.0 _ll_register_callbacks cb 0x562ef5b8a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ead9d640 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5c76900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 -- 172.21.15.105:0/3418547378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 msgr2=0x562ef5c76900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5c76900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 -- 172.21.15.105:0/3418547378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 msgr2=0x562ef5c6f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c6f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 -- 172.21.15.105:0/3418547378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef58de210 con 0x562ef5c655d0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c6f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35ea59c640 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c655d0 0x562ef5c6f380 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f35dc012700 tx=0x7f35dc012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733a21350dcd76c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.170+0000 7f35d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35d1ffb640 1 -- 172.21.15.105:0/3418547378 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00f8c0 con 0x562ef5c655d0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35d1ffb640 1 -- 172.21.15.105:0/3418547378 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f35dc00fa60 con 0x562ef5c655d0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 1 -- 172.21.15.105:0/3418547378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e4032cc0 con 0x562ef5c655d0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35d1ffb640 1 -- 172.21.15.105:0/3418547378 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f35dc00fc60 con 0x562ef5c655d0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 10 client.19848 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 1 -- 172.21.15.105:0/3418547378 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ef59aa620 con 0x562ef5c655d0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35d1ffb640 1 -- 172.21.15.105:0/3418547378 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35e4032cc0 con 0x562ef5c655d0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35d1ffb640 1 -- 172.21.15.105:0/3418547378 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f35dc010430 con 0x562ef5c655d0 2024-03-26T17:50:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 10 client.19848 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 10 client.19848 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 20 client.19848 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562ef5c7b1d0 0x562ef5c7d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 4 client.19848 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.171+0000 7f35f3bc3740 1 -- 172.21.15.105:0/3418547378 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ef5bd75f0 con 0x562ef5c7b1d0 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.172+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562ef5c7b1d0 0x562ef5c7d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.172+0000 7f35d1ffb640 10 client.19848 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.172+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562ef5c7b1d0 0x562ef5c7d620 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2161fe8acd9a56 server_cookie=ef23f91708e08d15 in_seq=0 out_seq=0 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35d1ffb640 1 -- 172.21.15.105:0/3418547378 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ef5bd75f0 con 0x562ef5c7b1d0 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35d1ffb640 10 client.19848 handle_command_reply: tid=0 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 client.19848 shutdown 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 -- 172.21.15.105:0/3418547378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c655d0 msgr2=0x562ef5c6f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c655d0 0x562ef5c6f380 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f35dc012700 tx=0x7f35dc012bc0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 -- 172.21.15.105:0/3418547378 shutdown_connections 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562ef5c7b1d0 0x562ef5c7d620 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5c76900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c6f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35d1ffb640 0 client.19848 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/3418547378 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c655d0 0x562ef5c6f380 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 -- 172.21.15.105:0/3418547378 >> 172.21.15.105:0/3418547378 conn(0x562ef5c60b40 msgr2=0x562ef5c5fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 -- 172.21.15.105:0/3418547378 shutdown_connections 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 1 -- 172.21.15.105:0/3418547378 wait complete. 2024-03-26T17:50:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.173+0000 7f35f3bc3740 20 client.19848 trim_cache size 0 max 16384 2024-03-26T17:50:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35f3bc3740 1 Processor -- start 2024-03-26T17:50:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35f3bc3740 1 -- start start 2024-03-26T17:50:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c655d0 0x562ef5c66850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c66d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1aaf80 con 0x562ef5c6b130 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1acac0 con 0x562ef5c65fc0 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1ab650 con 0x562ef5c655d0 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c655d0 0x562ef5c66850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c66d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35e9d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ea59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c655d0 0x562ef5c66850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59024/0 (socket says 172.21.15.105:59024) 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ea59c640 1 -- 172.21.15.105:0/1558772655 learned_addr learned my addr 172.21.15.105:0/1558772655 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:06.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c66d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36800/0 (socket says 172.21.15.105:36800) 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ea59c640 1 -- 172.21.15.105:0/1558772655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 msgr2=0x562ef5c768c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ea59c640 1 --2- 172.21.15.105:0/1558772655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c768c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ea59c640 1 -- 172.21.15.105:0/1558772655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 msgr2=0x562ef5c66d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ea59c640 1 --2- 172.21.15.105:0/1558772655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c66d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ea59c640 1 -- 172.21.15.105:0/1558772655 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef5c608b0 con 0x562ef5c655d0 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.188+0000 7f35ead9d640 1 --2- 172.21.15.105:0/1558772655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c66d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.189+0000 7f35ea59c640 1 --2- 172.21.15.105:0/1558772655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c655d0 0x562ef5c66850 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f35dc010850 tx=0x7f35dc00ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83b19457f07af59 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.189+0000 7f35d3fff640 1 -- 172.21.15.105:0/1558772655 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc010b80 con 0x562ef5c655d0 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.189+0000 7f35d3fff640 1 -- 172.21.15.105:0/1558772655 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35dc02b030 con 0x562ef5c655d0 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.189+0000 7f35d3fff640 1 -- 172.21.15.105:0/1558772655 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc005030 con 0x562ef5c655d0 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.189+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1558772655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c655d0 msgr2=0x562ef5c66850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.189+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1558772655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c655d0 0x562ef5c66850 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f35dc010850 tx=0x7f35dc00ff90 comp rx=0 tx=0).stop 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1558772655 shutdown_connections 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1558772655 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c6b130 0x562ef5c66d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1558772655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c65fc0 0x562ef5c768c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1558772655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c655d0 0x562ef5c66850 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1558772655 >> 172.21.15.105:0/1558772655 conn(0x562ef5c60b40 msgr2=0x562ef5c5fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1558772655 shutdown_connections 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1558772655 wait complete. 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 -- start start 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 Processor -- start 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c65fc0 0x562ef5c73c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5cdf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1ac3f0 con 0x562ef5c65fc0 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1abd20 con 0x562ef5c655d0 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1a8f60 con 0x562ef5c6b130 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35f3bc3740 10 client.0 _ll_register_callbacks cb 0x7f35dc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35e9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c65fc0 0x562ef5c73c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35ea59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35ead9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5cdf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35e9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c65fc0 0x562ef5c73c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36810/0 (socket says 172.21.15.105:36810) 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.190+0000 7f35e9d9b640 1 -- 172.21.15.105:0/1404516917 learned_addr learned my addr 172.21.15.105:0/1404516917 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35e9d9b640 1 -- 172.21.15.105:0/1404516917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 msgr2=0x562ef5cdf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5cdf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35e9d9b640 1 -- 172.21.15.105:0/1404516917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 msgr2=0x562ef5c736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35e9d9b640 1 -- 172.21.15.105:0/1404516917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef5cdfa90 con 0x562ef5c65fc0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35ea59c640 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35ead9d640 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5cdf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35e9d9b640 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c65fc0 0x562ef5c73c30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f35e000cf10 tx=0x7f35e0014820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d57cae46845e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35d1ffb640 1 -- 172.21.15.105:0/1404516917 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0026040 con 0x562ef5c65fc0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35d1ffb640 1 -- 172.21.15.105:0/1404516917 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f35e001cd30 con 0x562ef5c65fc0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1404516917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35dc02b030 con 0x562ef5c65fc0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35d1ffb640 1 -- 172.21.15.105:0/1404516917 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f35e00a47d0 con 0x562ef5c65fc0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35f3bc3740 10 client.19854 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1404516917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ef5ce0d70 con 0x562ef5c65fc0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.191+0000 7f35d1ffb640 1 -- 172.21.15.105:0/1404516917 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35dc02b030 con 0x562ef5c65fc0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35d1ffb640 1 -- 172.21.15.105:0/1404516917 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f35e007ace0 con 0x562ef5c65fc0 2024-03-26T17:50:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35f3bc3740 10 client.19854 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35f3bc3740 10 client.19854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35f3bc3740 20 client.19854 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562ef5ce1670 0x562ef5ce3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35f3bc3740 4 client.19854 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:50:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1404516917 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ef5bd1940 con 0x562ef5ce1670 2024-03-26T17:50:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35ea59c640 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562ef5ce1670 0x562ef5ce3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35d1ffb640 10 client.19854 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.192+0000 7f35ea59c640 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562ef5ce1670 0x562ef5ce3a60 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d1d92bfccb96e2 server_cookie=d8eef2547b16b8ef in_seq=0 out_seq=0 2024-03-26T17:50:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.443+0000 7f35d1ffb640 1 -- 172.21.15.105:0/1404516917 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ef5bd1940 con 0x562ef5ce1670 2024-03-26T17:50:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.443+0000 7f35d1ffb640 10 client.19854 handle_command_reply: tid=0 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.443+0000 7f35f3bc3740 1 client.19854 shutdown 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1404516917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c65fc0 msgr2=0x562ef5c73c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c65fc0 0x562ef5c73c30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f35e000cf10 tx=0x7f35e0014820 comp rx=0 tx=0).stop 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1404516917 shutdown_connections 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562ef5ce1670 0x562ef5ce3a60 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ef5c6b130 0x562ef5cdf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ef5c65fc0 0x562ef5c73c30 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35d1ffb640 0 client.19854 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 --2- 172.21.15.105:0/1404516917 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ef5c655d0 0x562ef5c736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1404516917 >> 172.21.15.105:0/1404516917 conn(0x562ef5c60b40 msgr2=0x562ef5c66850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1404516917 shutdown_connections 2024-03-26T17:50:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 1 -- 172.21.15.105:0/1404516917 wait complete. 2024-03-26T17:50:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.444+0000 7f35f3bc3740 20 client.19854 trim_cache size 0 max 16384 2024-03-26T17:50:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:50:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:50:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:50:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:50:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 -- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d40a3040 msgr2=0x7f35d40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d40a3040 0x7f35d40a5500 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009a60 tx=0x7f35e4042000 comp rx=0 tx=0).stop 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 -- 172.21.15.105:0/132488185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1019d0 msgr2=0x7f35ec10b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1019d0 0x7f35ec10b2a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35e000e9f0 tx=0x7f35e000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 -- 172.21.15.105:0/132488185 shutdown_connections 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35d40a3040 0x7f35d40a5500 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35ec102390 0x7f35ec10b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec1019d0 0x7f35ec10b2a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 --2- 172.21.15.105:0/132488185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35ec1007e0 0x7f35ec10cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 -- 172.21.15.105:0/132488185 >> 172.21.15.105:0/132488185 conn(0x7f35ec0fbd50 msgr2=0x7f35ec104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 -- 172.21.15.105:0/132488185 shutdown_connections 2024-03-26T17:50:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:06.445+0000 7f35f0ef8640 1 -- 172.21.15.105:0/132488185 wait complete. 2024-03-26T17:50:07.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 380543 3.14 MB/sec execute 530 sec latency 45.066 ms 2024-03-26T17:50:07.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:06 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:50:07.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:06 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:50:07.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:06 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:50:08.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 381175 3.14 MB/sec execute 531 sec latency 26.304 ms 2024-03-26T17:50:08.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:07 smithi144 ceph-mon[36095]: pgmap v702: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-03-26T17:50:08.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:07 smithi105 ceph-mon[32280]: pgmap v702: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-03-26T17:50:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:07 smithi135 ceph-mon[35636]: pgmap v702: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-03-26T17:50:09.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 381872 3.14 MB/sec execute 532 sec latency 20.207 ms 2024-03-26T17:50:10.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 382534 3.14 MB/sec execute 533 sec latency 16.526 ms 2024-03-26T17:50:10.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:09 smithi144 ceph-mon[36095]: pgmap v703: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:50:10.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:09 smithi105 ceph-mon[32280]: pgmap v703: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:50:10.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:09 smithi135 ceph-mon[35636]: pgmap v703: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 105 op/s 2024-03-26T17:50:11.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 383089 3.15 MB/sec execute 534 sec latency 23.122 ms 2024-03-26T17:50:12.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 383532 3.15 MB/sec execute 535 sec latency 22.466 ms 2024-03-26T17:50:12.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:11 smithi144 ceph-mon[36095]: pgmap v704: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:50:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:11 smithi105 ceph-mon[32280]: pgmap v704: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:50:12.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:11 smithi135 ceph-mon[35636]: pgmap v704: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:50:13.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 384043 3.14 MB/sec execute 536 sec latency 18.596 ms 2024-03-26T17:50:14.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 384665 3.14 MB/sec execute 537 sec latency 17.415 ms 2024-03-26T17:50:14.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:13 smithi144 ceph-mon[36095]: pgmap v705: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:50:14.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:13 smithi105 ceph-mon[32280]: pgmap v705: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:50:14.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:13 smithi135 ceph-mon[35636]: pgmap v705: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-03-26T17:50:15.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 385377 3.14 MB/sec execute 538 sec latency 15.694 ms 2024-03-26T17:50:16.033 INFO:tasks.workunit.client.0.smithi105.stdout: 1 386064 3.15 MB/sec execute 539 sec latency 17.330 ms 2024-03-26T17:50:16.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:15 smithi144 ceph-mon[36095]: pgmap v706: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:50:16.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:15 smithi105 ceph-mon[32280]: pgmap v706: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:50:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:15 smithi135 ceph-mon[35636]: pgmap v706: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-03-26T17:50:17.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 386654 3.15 MB/sec execute 540 sec latency 15.933 ms 2024-03-26T17:50:18.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 387123 3.15 MB/sec execute 541 sec latency 15.467 ms 2024-03-26T17:50:18.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:17 smithi144 ceph-mon[36095]: pgmap v707: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-03-26T17:50:18.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:17 smithi105 ceph-mon[32280]: pgmap v707: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-03-26T17:50:18.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:17 smithi135 ceph-mon[35636]: pgmap v707: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-03-26T17:50:19.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 387620 3.14 MB/sec execute 542 sec latency 26.245 ms 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 -- 172.21.15.105:0/3470258985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508104cf0 msgr2=0x7f65081070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 --2- 172.21.15.105:0/3470258985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508104cf0 0x7f65081070e0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009900 tx=0x7f64fc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 -- 172.21.15.105:0/3470258985 shutdown_connections 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 --2- 172.21.15.105:0/3470258985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6508107620 0x7f6508109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 --2- 172.21.15.105:0/3470258985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508104cf0 0x7f65081070e0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 --2- 172.21.15.105:0/3470258985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65081023c0 0x7f65081047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 -- 172.21.15.105:0/3470258985 >> 172.21.15.105:0/3470258985 conn(0x7f65080fbd50 msgr2=0x7f65080fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 -- 172.21.15.105:0/3470258985 shutdown_connections 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.543+0000 7f650d0ea640 1 -- 172.21.15.105:0/3470258985 wait complete. 2024-03-26T17:50:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650d0ea640 1 Processor -- start 2024-03-26T17:50:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650d0ea640 1 -- start start 2024-03-26T17:50:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650d0ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65081023c0 0x7f6508197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650d0ea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6508104cf0 0x7f6508197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650d0ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508107620 0x7f650819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650d0ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f650810d410 con 0x7f6508107620 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650d0ea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f650810d290 con 0x7f6508104cf0 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650d0ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f650810d590 con 0x7f65081023c0 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6508104cf0 0x7f6508197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508107620 0x7f650819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6508104cf0 0x7f6508197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54482/0 (socket says 172.21.15.105:54482) 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650659c640 1 -- 172.21.15.105:0/4187428489 learned_addr learned my addr 172.21.15.105:0/4187428489 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f6506d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65081023c0 0x7f6508197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.544+0000 7f650759e640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508107620 0x7f650819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33746/0 (socket says 172.21.15.105:33746) 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650759e640 1 -- 172.21.15.105:0/4187428489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65081023c0 msgr2=0x7f6508197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650759e640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65081023c0 0x7f6508197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650759e640 1 -- 172.21.15.105:0/4187428489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6508104cf0 msgr2=0x7f6508197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650759e640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6508104cf0 0x7f6508197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650759e640 1 -- 172.21.15.105:0/4187428489 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650819f680 con 0x7f6508107620 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650659c640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6508104cf0 0x7f6508197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650759e640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508107620 0x7f650819f010 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6500009fd0 tx=0x7f650000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f64effff640 1 -- 172.21.15.105:0/4187428489 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000ce20 con 0x7f6508107620 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f64effff640 1 -- 172.21.15.105:0/4187428489 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6500005020 con 0x7f6508107620 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650d0ea640 1 -- 172.21.15.105:0/4187428489 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650819f970 con 0x7f6508107620 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f64effff640 1 -- 172.21.15.105:0/4187428489 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500010690 con 0x7f6508107620 2024-03-26T17:50:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.545+0000 7f650d0ea640 1 -- 172.21.15.105:0/4187428489 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650819fec0 con 0x7f6508107620 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f64effff640 1 -- 172.21.15.105:0/4187428489 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f65000051d0 con 0x7f6508107620 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650fdb2740 1 Processor -- start 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650fdb2740 1 -- start start 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e9624359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e96243aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 0x55e96243d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650fdb2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e96239a0b0 con 0x55e9624355d0 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650fdb2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e962342960 con 0x55e962435fc0 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650fdb2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e962396060 con 0x55e96243b130 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e9624359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e9624359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33750/0 (socket says 172.21.15.105:33750) 2024-03-26T17:50:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 -- 172.21.15.105:0/309565434 learned_addr learned my addr 172.21.15.105:0/309565434 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f6506d9d640 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e96243aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 0x55e96243d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f64effff640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64f00a2fc0 0x7f64f00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650759e640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64f00a2fc0 0x7f64f00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 -- 172.21.15.105:0/309565434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 msgr2=0x55e96243d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 0x55e96243d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 -- 172.21.15.105:0/309565434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 msgr2=0x55e96243aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e96243aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f650659c640 1 -- 172.21.15.105:0/309565434 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e962354c60 con 0x55e9624355d0 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.548+0000 7f6506d9d640 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e96243aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.549+0000 7f64effff640 1 -- 172.21.15.105:0/4187428489 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f65000cfae0 con 0x7f6508107620 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.549+0000 7f650659c640 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e9624359f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0096c0 tx=0x7f64fc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa6525d623c68e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.549+0000 7f64ed7fa640 1 -- 172.21.15.105:0/309565434 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc004410 con 0x55e9624355d0 2024-03-26T17:50:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.549+0000 7f650759e640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64f00a2fc0 0x7f64f00a5480 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f650009bed0 tx=0x7f65000bc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.549+0000 7f64ed7fa640 1 -- 172.21.15.105:0/309565434 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64fc005500 con 0x55e9624355d0 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.549+0000 7f64ed7fa640 1 -- 172.21.15.105:0/309565434 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc004e50 con 0x55e9624355d0 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- 172.21.15.105:0/309565434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 msgr2=0x55e9624359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e9624359f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0096c0 tx=0x7f64fc005b50 comp rx=0 tx=0).stop 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- 172.21.15.105:0/309565434 shutdown_connections 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 0x55e96243d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e96243aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 --2- 172.21.15.105:0/309565434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e9624359f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- 172.21.15.105:0/309565434 >> 172.21.15.105:0/309565434 conn(0x55e962430b40 msgr2=0x55e962430f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- 172.21.15.105:0/309565434 shutdown_connections 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- 172.21.15.105:0/309565434 wait complete. 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- start start 2024-03-26T17:50:19.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 Processor -- start 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9624355d0 0x55e96243f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e962435fc0 0x55e96243f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e96243b130 0x55e962446920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e96243e080 con 0x55e962435fc0 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e962355db0 con 0x55e96243b130 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e96243e200 con 0x55e9624355d0 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650fdb2740 10 client.0 _ll_register_callbacks cb 0x55e96235a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f6506d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e962435fc0 0x55e96243f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f6506d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e962435fc0 0x55e96243f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33764/0 (socket says 172.21.15.105:33764) 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f6506d9d640 1 -- 172.21.15.105:0/3302211311 learned_addr learned my addr 172.21.15.105:0/3302211311 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.550+0000 7f650659c640 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9624355d0 0x55e96243f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f650659c640 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e96243b130 0x55e962446920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f6506d9d640 1 -- 172.21.15.105:0/3302211311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9624355d0 msgr2=0x55e96243f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f6506d9d640 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9624355d0 0x55e96243f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f6506d9d640 1 -- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e96243b130 msgr2=0x55e962446920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f6506d9d640 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e96243b130 0x55e962446920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f6506d9d640 1 -- 172.21.15.105:0/3302211311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9620ae210 con 0x55e962435fc0 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f650659c640 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e96243b130 0x55e962446920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f6506d9d640 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e962435fc0 0x55e96243f950 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f64f80126e0 tx=0x7f64f8012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd437d66beb0c41a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f64eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f64eaff5640 1 -- 172.21.15.105:0/3302211311 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8012190 con 0x55e962435fc0 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f64eaff5640 1 -- 172.21.15.105:0/3302211311 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f64f800f8c0 con 0x55e962435fc0 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f650fdb2740 1 -- 172.21.15.105:0/3302211311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64fc005500 con 0x55e962435fc0 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f64eaff5640 1 -- 172.21.15.105:0/3302211311 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f64f800fac0 con 0x55e962435fc0 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f650fdb2740 10 client.19866 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f650fdb2740 1 -- 172.21.15.105:0/3302211311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e96217a620 con 0x55e962435fc0 2024-03-26T17:50:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.551+0000 7f64eaff5640 1 -- 172.21.15.105:0/3302211311 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64fc005500 con 0x55e962435fc0 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f64eaff5640 1 -- 172.21.15.105:0/3302211311 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f64f8026be0 con 0x55e962435fc0 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f650fdb2740 10 client.19866 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f650fdb2740 10 client.19866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f650fdb2740 20 client.19866 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e96244b2d0 0x55e96244d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f650fdb2740 4 client.19866 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f650fdb2740 1 -- 172.21.15.105:0/3302211311 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9623a75f0 con 0x55e96244b2d0 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f650659c640 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e96244b2d0 0x55e96244d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f64eaff5640 10 client.19866 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.552+0000 7f650659c640 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e96244b2d0 0x55e96244d710 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad62d325b0c5277 server_cookie=df44902a1f9ae14b in_seq=0 out_seq=0 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.553+0000 7f64eaff5640 1 -- 172.21.15.105:0/3302211311 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e9623a75f0 con 0x55e96244b2d0 2024-03-26T17:50:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.553+0000 7f64eaff5640 10 client.19866 handle_command_reply: tid=0 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.553+0000 7f650fdb2740 1 client.19866 shutdown 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.553+0000 7f650fdb2740 1 -- 172.21.15.105:0/3302211311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e962435fc0 msgr2=0x55e96243f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.553+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e962435fc0 0x55e96243f950 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f64f80126e0 tx=0x7f64f8012ba0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 1 -- 172.21.15.105:0/3302211311 shutdown_connections 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e96244b2d0 0x55e96244d710 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e96243b130 0x55e962446920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e962435fc0 0x55e96243f950 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3302211311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e9624355d0 0x55e96243f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f64eaff5640 0 client.19866 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 1 -- 172.21.15.105:0/3302211311 >> 172.21.15.105:0/3302211311 conn(0x55e962430b40 msgr2=0x55e96242fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 1 -- 172.21.15.105:0/3302211311 shutdown_connections 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 1 -- 172.21.15.105:0/3302211311 wait complete. 2024-03-26T17:50:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.554+0000 7f650fdb2740 20 client.19866 trim_cache size 0 max 16384 2024-03-26T17:50:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650fdb2740 1 Processor -- start 2024-03-26T17:50:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650fdb2740 1 -- start start 2024-03-26T17:50:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e962436a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e962436f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 0x55e96242f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650fdb2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65081a8d30 con 0x55e9624355d0 2024-03-26T17:50:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650fdb2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65081aa870 con 0x55e962435fc0 2024-03-26T17:50:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650fdb2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65081a9400 con 0x55e96243b130 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e962436a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e962436a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33774/0 (socket says 172.21.15.105:33774) 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 -- 172.21.15.105:0/2202810485 learned_addr learned my addr 172.21.15.105:0/2202810485 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f6506d9d640 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e962436f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 0x55e96242f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 -- 172.21.15.105:0/2202810485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 msgr2=0x55e96242f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 0x55e96242f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 -- 172.21.15.105:0/2202810485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 msgr2=0x55e962436f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e962436f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.571+0000 7f650659c640 1 -- 172.21.15.105:0/2202810485 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9624315b0 con 0x55e9624355d0 2024-03-26T17:50:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.572+0000 7f650659c640 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e962436a30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0096c0 tx=0x7f64fc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fe72c17247eea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.572+0000 7f64ecff9640 1 -- 172.21.15.105:0/2202810485 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc059cb0 con 0x55e9624355d0 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.572+0000 7f64ecff9640 1 -- 172.21.15.105:0/2202810485 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64fc059e50 con 0x55e9624355d0 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.572+0000 7f64ecff9640 1 -- 172.21.15.105:0/2202810485 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc003d80 con 0x55e9624355d0 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- 172.21.15.105:0/2202810485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 msgr2=0x55e962436a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e962436a30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0096c0 tx=0x7f64fc005fd0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- 172.21.15.105:0/2202810485 shutdown_connections 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e96243b130 0x55e96242f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e962435fc0 0x55e962436f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 --2- 172.21.15.105:0/2202810485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e9624355d0 0x55e962436a30 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- 172.21.15.105:0/2202810485 >> 172.21.15.105:0/2202810485 conn(0x55e962430b40 msgr2=0x55e96243a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- 172.21.15.105:0/2202810485 shutdown_connections 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- 172.21.15.105:0/2202810485 wait complete. 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- start start 2024-03-26T17:50:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 Processor -- start 2024-03-26T17:50:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9624355d0 0x55e9624ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e962435fc0 0x55e9624ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e96243b130 0x55e9624b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65081aa1a0 con 0x55e96243b130 2024-03-26T17:50:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65081a9ad0 con 0x55e9624355d0 2024-03-26T17:50:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65081a6d10 con 0x55e962435fc0 2024-03-26T17:50:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650fdb2740 10 client.0 _ll_register_callbacks cb 0x7f64f8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e96243b130 0x55e9624b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f6506d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e962435fc0 0x55e9624ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e96243b130 0x55e9624b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33788/0 (socket says 172.21.15.105:33788) 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650659c640 1 -- 172.21.15.105:0/3596064240 learned_addr learned my addr 172.21.15.105:0/3596064240 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.573+0000 7f650659c640 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9624355d0 0x55e9624ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650659c640 1 -- 172.21.15.105:0/3596064240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e962435fc0 msgr2=0x55e9624ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650659c640 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e962435fc0 0x55e9624ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650659c640 1 -- 172.21.15.105:0/3596064240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9624355d0 msgr2=0x55e9624ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650659c640 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9624355d0 0x55e9624ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650659c640 1 -- 172.21.15.105:0/3596064240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9624b4450 con 0x55e96243b130 2024-03-26T17:50:19.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650659c640 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e96243b130 0x55e9624b3e60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0057c0 tx=0x7f64fc045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d373e6a3712b6a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f64eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f64eaff5640 1 -- 172.21.15.105:0/3596064240 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc003d00 con 0x55e96243b130 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f64eaff5640 1 -- 172.21.15.105:0/3596064240 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f64fc0592e0 con 0x55e96243b130 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650fdb2740 1 -- 172.21.15.105:0/3596064240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64fc059e50 con 0x55e96243b130 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f64eaff5640 1 -- 172.21.15.105:0/3596064240 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f64fc0594e0 con 0x55e96243b130 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650fdb2740 10 client.19872 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f650fdb2740 1 -- 172.21.15.105:0/3596064240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e9624b5760 con 0x55e96243b130 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.574+0000 7f64eaff5640 1 -- 172.21.15.105:0/3596064240 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64fc059e50 con 0x55e96243b130 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f64eaff5640 1 -- 172.21.15.105:0/3596064240 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f64fc034ad0 con 0x55e96243b130 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f650fdb2740 10 client.19872 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f650fdb2740 10 client.19872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f650fdb2740 20 client.19872 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e9624b5cb0 0x55e9624b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f650fdb2740 4 client.19872 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:50:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f650fdb2740 1 -- 172.21.15.105:0/3596064240 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e96243f200 con 0x55e9624b5cb0 2024-03-26T17:50:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f6506d9d640 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e9624b5cb0 0x55e9624b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f64eaff5640 10 client.19872 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.575+0000 7f6506d9d640 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e9624b5cb0 0x55e9624b80a0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd264465421c021 server_cookie=5ca66fea7362beeb in_seq=0 out_seq=0 2024-03-26T17:50:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.632+0000 7f64eaff5640 1 -- 172.21.15.105:0/3596064240 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e96243f200 con 0x55e9624b5cb0 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.632+0000 7f64eaff5640 10 client.19872 handle_command_reply: tid=0 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.632+0000 7f650fdb2740 1 client.19872 shutdown 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 -- 172.21.15.105:0/3596064240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e96243b130 msgr2=0x55e9624b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e96243b130 0x55e9624b3e60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0057c0 tx=0x7f64fc045040 comp rx=0 tx=0).stop 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 -- 172.21.15.105:0/3596064240 shutdown_connections 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e9624b5cb0 0x55e9624b80a0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e96243b130 0x55e9624b3e60 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e962435fc0 0x55e9624ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 --2- 172.21.15.105:0/3596064240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e9624355d0 0x55e9624ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f64eaff5640 0 client.19872 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 -- 172.21.15.105:0/3596064240 >> 172.21.15.105:0/3596064240 conn(0x55e962430b40 msgr2=0x55e962434630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 -- 172.21.15.105:0/3596064240 shutdown_connections 2024-03-26T17:50:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 1 -- 172.21.15.105:0/3596064240 wait complete. 2024-03-26T17:50:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650fdb2740 20 client.19872 trim_cache size 0 max 16384 2024-03-26T17:50:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:50:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:50:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:50:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:50:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 -- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64f00a2fc0 msgr2=0x7f64f00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64f00a2fc0 0x7f64f00a5480 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f650009bed0 tx=0x7f65000bc040 comp rx=0 tx=0).stop 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 -- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508107620 msgr2=0x7f650819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508107620 0x7f650819f010 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6500009fd0 tx=0x7f650000ef10 comp rx=0 tx=0).stop 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 -- 172.21.15.105:0/4187428489 shutdown_connections 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64f00a2fc0 0x7f64f00a5480 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6508107620 0x7f650819f010 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6508104cf0 0x7f6508197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 --2- 172.21.15.105:0/4187428489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65081023c0 0x7f6508197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.633+0000 7f650d0ea640 1 -- 172.21.15.105:0/4187428489 >> 172.21.15.105:0/4187428489 conn(0x7f65080fbd50 msgr2=0x7f6508102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.634+0000 7f650d0ea640 1 -- 172.21.15.105:0/4187428489 shutdown_connections 2024-03-26T17:50:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:19.634+0000 7f650d0ea640 1 -- 172.21.15.105:0/4187428489 wait complete. 2024-03-26T17:50:20.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 388237 3.14 MB/sec execute 543 sec latency 20.005 ms 2024-03-26T17:50:20.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:19 smithi144 ceph-mon[36095]: pgmap v708: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:50:20.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:19 smithi105 ceph-mon[32280]: pgmap v708: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:50:20.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:19 smithi135 ceph-mon[35636]: pgmap v708: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-03-26T17:50:21.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 388880 3.14 MB/sec execute 544 sec latency 15.749 ms 2024-03-26T17:50:22.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 389590 3.15 MB/sec execute 545 sec latency 19.404 ms 2024-03-26T17:50:22.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:21 smithi144 ceph-mon[36095]: pgmap v709: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:50:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:21 smithi105 ceph-mon[32280]: pgmap v709: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:50:22.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:21 smithi135 ceph-mon[35636]: pgmap v709: 97 pgs: 97 active+clean; 24 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-26T17:50:23.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 390164 3.15 MB/sec execute 546 sec latency 16.774 ms 2024-03-26T17:50:24.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 390625 3.15 MB/sec execute 547 sec latency 15.870 ms 2024-03-26T17:50:24.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:23 smithi144 ceph-mon[36095]: pgmap v710: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:50:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:23 smithi105 ceph-mon[32280]: pgmap v710: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:50:24.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:23 smithi135 ceph-mon[35636]: pgmap v710: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:50:25.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 391103 3.14 MB/sec execute 548 sec latency 23.740 ms 2024-03-26T17:50:26.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 391716 3.14 MB/sec execute 549 sec latency 17.304 ms 2024-03-26T17:50:26.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:25 smithi144 ceph-mon[36095]: pgmap v711: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-03-26T17:50:26.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:25 smithi105 ceph-mon[32280]: pgmap v711: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-03-26T17:50:26.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:25 smithi135 ceph-mon[35636]: pgmap v711: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-03-26T17:50:27.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 392398 3.14 MB/sec execute 550 sec latency 18.808 ms 2024-03-26T17:50:28.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 392999 3.14 MB/sec execute 551 sec latency 41.681 ms 2024-03-26T17:50:28.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:27 smithi144 ceph-mon[36095]: pgmap v712: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-03-26T17:50:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:27 smithi105 ceph-mon[32280]: pgmap v712: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-03-26T17:50:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:27 smithi135 ceph-mon[35636]: pgmap v712: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-03-26T17:50:29.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 393607 3.14 MB/sec execute 552 sec latency 17.287 ms 2024-03-26T17:50:30.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 394173 3.15 MB/sec execute 553 sec latency 15.443 ms 2024-03-26T17:50:30.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:29 smithi144 ceph-mon[36095]: pgmap v713: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:50:30.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:29 smithi105 ceph-mon[32280]: pgmap v713: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:50:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:29 smithi135 ceph-mon[35636]: pgmap v713: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-03-26T17:50:31.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 394631 3.14 MB/sec execute 554 sec latency 41.413 ms 2024-03-26T17:50:32.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 395276 3.14 MB/sec execute 555 sec latency 19.205 ms 2024-03-26T17:50:32.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:31 smithi144 ceph-mon[36095]: pgmap v714: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-26T17:50:32.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:31 smithi105 ceph-mon[32280]: pgmap v714: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-26T17:50:32.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:31 smithi135 ceph-mon[35636]: pgmap v714: 97 pgs: 97 active+clean; 26 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-03-26T17:50:32.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:50:32.288 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:50:32.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.418+0000 7f0612cf8640 1 -- 172.21.15.105:0/1105881717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f060c102370 msgr2=0x7f060c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.418+0000 7f0612cf8640 1 --2- 172.21.15.105:0/1105881717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f060c102370 0x7f060c108c00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f0604009ca0 tx=0x7f060402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 -- 172.21.15.105:0/1105881717 shutdown_connections 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 --2- 172.21.15.105:0/1105881717 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f060c102370 0x7f060c108c00 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 --2- 172.21.15.105:0/1105881717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f060c1019b0 0x7f060c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 --2- 172.21.15.105:0/1105881717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f060c1007c0 0x7f060c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 -- 172.21.15.105:0/1105881717 >> 172.21.15.105:0/1105881717 conn(0x7f060c0fbd50 msgr2=0x7f060c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 -- 172.21.15.105:0/1105881717 shutdown_connections 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 -- 172.21.15.105:0/1105881717 wait complete. 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 Processor -- start 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 -- start start 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f060c1007c0 0x7f060c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.419+0000 7f0612cf8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f060c1019b0 0x7f060c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0612cf8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f060c102370 0x7f060c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0612cf8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10c2f0 con 0x7f060c102370 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0612cf8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10c170 con 0x7f060c1019b0 2024-03-26T17:50:32.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0612cf8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f060c10c470 con 0x7f060c1007c0 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f060c1007c0 0x7f060c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f060c1007c0 0x7f060c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46762/0 (socket says 172.21.15.105:46762) 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 -- 172.21.15.105:0/3231763274 learned_addr learned my addr 172.21.15.105:0/3231763274 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f060bfff640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f060c1019b0 0x7f060c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0611295640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f060c102370 0x7f060c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 -- 172.21.15.105:0/3231763274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f060c1019b0 msgr2=0x7f060c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f060c1019b0 0x7f060c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 -- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f060c102370 msgr2=0x7f060c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f060c102370 0x7f060c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 -- 172.21.15.105:0/3231763274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c19f750 con 0x7f060c1007c0 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0610a94640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f060c1007c0 0x7f060c197b80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f060000e9c0 tx=0x7f060000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0609ffb640 1 -- 172.21.15.105:0/3231763274 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060000cd80 con 0x7f060c1007c0 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0612cf8640 1 -- 172.21.15.105:0/3231763274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c19fa40 con 0x7f060c1007c0 2024-03-26T17:50:32.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0609ffb640 1 -- 172.21.15.105:0/3231763274 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06000035f0 con 0x7f060c1007c0 2024-03-26T17:50:32.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.420+0000 7f0609ffb640 1 -- 172.21.15.105:0/3231763274 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600010690 con 0x7f060c1007c0 2024-03-26T17:50:32.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.421+0000 7f0612cf8640 1 -- 172.21.15.105:0/3231763274 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c19ff40 con 0x7f060c1007c0 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.423+0000 7f0609ffb640 1 -- 172.21.15.105:0/3231763274 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f06000037a0 con 0x7f060c1007c0 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.423+0000 7f06159c1740 1 Processor -- start 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.423+0000 7f06159c1740 1 -- start start 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.423+0000 7f06159c1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284efcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.423+0000 7f06159c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f01de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.423+0000 7f06159c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.423+0000 7f06159c1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d284e61980 con 0x55d284efc8f0 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f06159c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d284e09be0 con 0x55d284f02450 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f06159c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d284e5d930 con 0x55d284efd2e0 2024-03-26T17:50:32.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f0609ffb640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05fc0a2f20 0x7f05fc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f0611295640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284efcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f060bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f0611295640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284efcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37974/0 (socket says 172.21.15.105:37974) 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f060bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46582/0 (socket says 172.21.15.105:46582) 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f0611295640 1 -- 172.21.15.105:0/864999061 learned_addr learned my addr 172.21.15.105:0/864999061 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f0610a94640 1 --2- 172.21.15.105:0/864999061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f01de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f0611295640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05fc0a2f20 0x7f05fc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f060bfff640 1 -- 172.21.15.105:0/864999061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 msgr2=0x55d284f01de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f060bfff640 1 --2- 172.21.15.105:0/864999061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f01de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f060bfff640 1 -- 172.21.15.105:0/864999061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 msgr2=0x55d284efcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f060bfff640 1 --2- 172.21.15.105:0/864999061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284efcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f060bfff640 1 -- 172.21.15.105:0/864999061 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d284c3f7d0 con 0x55d284f02450 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f0609ffb640 1 -- 172.21.15.105:0/3231763274 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f06000cf930 con 0x7f060c1007c0 2024-03-26T17:50:32.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f060bfff640 1 --2- 172.21.15.105:0/864999061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f048e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009bf0 tx=0x7f05f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c04d59b72b47e9a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f05f6ffd640 1 -- 172.21.15.105:0/864999061 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8009da0 con 0x55d284f02450 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f0611295640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05fc0a2f20 0x7f05fc0a53e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55d284efdb40 tx=0x7f060402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f05f6ffd640 1 -- 172.21.15.105:0/864999061 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f8005020 con 0x55d284f02450 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.424+0000 7f05f6ffd640 1 -- 172.21.15.105:0/864999061 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004b20 con 0x55d284f02450 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 -- 172.21.15.105:0/864999061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 msgr2=0x55d284f048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 --2- 172.21.15.105:0/864999061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f048e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009bf0 tx=0x7f05f8011810 comp rx=0 tx=0).stop 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 -- 172.21.15.105:0/864999061 shutdown_connections 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 --2- 172.21.15.105:0/864999061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f048e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 --2- 172.21.15.105:0/864999061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f01de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 --2- 172.21.15.105:0/864999061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284efcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 -- 172.21.15.105:0/864999061 >> 172.21.15.105:0/864999061 conn(0x55d284ef7e60 msgr2=0x55d284ef8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 -- 172.21.15.105:0/864999061 shutdown_connections 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 -- 172.21.15.105:0/864999061 wait complete. 2024-03-26T17:50:32.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.425+0000 7f06159c1740 1 -- start start 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f06159c1740 1 Processor -- start 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f06159c1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f06159c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f07010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f06159c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f06159c1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d284f05400 con 0x55d284efc8f0 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f06159c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d284e1d160 con 0x55d284f02450 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f06159c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d284f05580 con 0x55d284efd2e0 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f06159c1740 10 client.0 _ll_register_callbacks cb 0x55d284e59720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0610a94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0610a94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37984/0 (socket says 172.21.15.105:37984) 2024-03-26T17:50:32.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0610a94640 1 -- 172.21.15.105:0/1351310482 learned_addr learned my addr 172.21.15.105:0/1351310482 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f060bfff640 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f07010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0611295640 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0610a94640 1 -- 172.21.15.105:0/1351310482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 msgr2=0x55d284f07010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0610a94640 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f07010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0610a94640 1 -- 172.21.15.105:0/1351310482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 msgr2=0x55d284f0e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0610a94640 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.426+0000 7f0610a94640 1 -- 172.21.15.105:0/1351310482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d284b753f0 con 0x55d284efc8f0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f0610a94640 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06ad0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f06000cd760 tx=0x7f060009bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5bf84776a4149d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f05ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f05ebfff640 1 -- 172.21.15.105:0/1351310482 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060009c020 con 0x55d284efc8f0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f05ebfff640 1 -- 172.21.15.105:0/1351310482 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f06000b00e0 con 0x55d284efc8f0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 1 -- 172.21.15.105:0/1351310482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f8005020 con 0x55d284efc8f0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f05ebfff640 1 -- 172.21.15.105:0/1351310482 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f06000a0170 con 0x55d284efc8f0 2024-03-26T17:50:32.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 10 client.19884 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 1 -- 172.21.15.105:0/1351310482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d284b350f0 con 0x55d284efc8f0 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f05ebfff640 1 -- 172.21.15.105:0/1351310482 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f8005020 con 0x55d284efc8f0 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f05ebfff640 1 -- 172.21.15.105:0/1351310482 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0600097e00 con 0x55d284efc8f0 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 10 client.19884 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 10 client.19884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 20 client.19884 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d284f12980 0x55d284f14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 4 client.19884 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f06159c1740 1 -- 172.21.15.105:0/1351310482 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d284e6eec0 con 0x55d284f12980 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.427+0000 7f060bfff640 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d284f12980 0x55d284f14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.428+0000 7f05ebfff640 10 client.19884 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.428+0000 7f060bfff640 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d284f12980 0x55d284f14e10 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b49eb6336d1fcbd9 server_cookie=54590c787320a173 in_seq=0 out_seq=0 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f05ebfff640 1 -- 172.21.15.105:0/1351310482 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d284e6eec0 con 0x55d284f12980 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f05ebfff640 10 client.19884 handle_command_reply: tid=0 2024-03-26T17:50:32.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 client.19884 shutdown 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 -- 172.21.15.105:0/1351310482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 msgr2=0x55d284f06ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06ad0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f06000cd760 tx=0x7f060009bc40 comp rx=0 tx=0).stop 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 -- 172.21.15.105:0/1351310482 shutdown_connections 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d284f12980 0x55d284f14e10 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f05ebfff640 0 client.19884 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f07010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 --2- 172.21.15.105:0/1351310482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06ad0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 -- 172.21.15.105:0/1351310482 >> 172.21.15.105:0/1351310482 conn(0x55d284ef7e60 msgr2=0x55d284ef7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 -- 172.21.15.105:0/1351310482 shutdown_connections 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 1 -- 172.21.15.105:0/1351310482 wait complete. 2024-03-26T17:50:32.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.429+0000 7f06159c1740 20 client.19884 trim_cache size 0 max 16384 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.452+0000 7f06159c1740 1 Processor -- start 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.452+0000 7f06159c1740 1 -- start start 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f06159c1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f06159c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284efd2e0 0x55d284f0dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f06159c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284f02450 0x55d284ef67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f06159c1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a8d70 con 0x55d284efc8f0 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f06159c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1aa8b0 con 0x55d284efd2e0 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f06159c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a9440 con 0x55d284f02450 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37994/0 (socket says 172.21.15.105:37994) 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 -- 172.21.15.105:0/2966813790 learned_addr learned my addr 172.21.15.105:0/2966813790 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f060bfff640 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284efd2e0 0x55d284f0dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0611295640 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284f02450 0x55d284ef67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 -- 172.21.15.105:0/2966813790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284f02450 msgr2=0x55d284ef67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284f02450 0x55d284ef67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 -- 172.21.15.105:0/2966813790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284efd2e0 msgr2=0x55d284f0dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284efd2e0 0x55d284f0dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 -- 172.21.15.105:0/2966813790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d284ef38d0 con 0x55d284efc8f0 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f0610a94640 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06df0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0600003db0 tx=0x7f06000cfcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be19c09d21051637 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f05f67fc640 1 -- 172.21.15.105:0/2966813790 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060009c020 con 0x55d284efc8f0 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f05f67fc640 1 -- 172.21.15.105:0/2966813790 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06000b5140 con 0x55d284efc8f0 2024-03-26T17:50:32.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.453+0000 7f05f67fc640 1 -- 172.21.15.105:0/2966813790 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600097d70 con 0x55d284efc8f0 2024-03-26T17:50:32.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 -- 172.21.15.105:0/2966813790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 msgr2=0x55d284f06df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06df0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0600003db0 tx=0x7f06000cfcc0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 -- 172.21.15.105:0/2966813790 shutdown_connections 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284f02450 0x55d284ef67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284efd2e0 0x55d284f0dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 --2- 172.21.15.105:0/2966813790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f06df0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 -- 172.21.15.105:0/2966813790 >> 172.21.15.105:0/2966813790 conn(0x55d284ef7e60 msgr2=0x55d284f066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 -- 172.21.15.105:0/2966813790 shutdown_connections 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.454+0000 7f06159c1740 1 -- 172.21.15.105:0/2966813790 wait complete. 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 1 -- start start 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 1 Processor -- start 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f0bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1aa1e0 con 0x55d284efc8f0 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a9b10 con 0x55d284f02450 2024-03-26T17:50:32.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1a6d50 con 0x55d284efd2e0 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f06159c1740 10 client.0 _ll_register_callbacks cb 0x7f0600003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f0610a94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f0610a94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f0b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38008/0 (socket says 172.21.15.105:38008) 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f0611295640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f0610a94640 1 -- 172.21.15.105:0/2330571450 learned_addr learned my addr 172.21.15.105:0/2330571450 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.455+0000 7f060bfff640 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f0bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f0610a94640 1 -- 172.21.15.105:0/2330571450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 msgr2=0x55d284f0bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f0610a94640 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f0bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f0610a94640 1 -- 172.21.15.105:0/2330571450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 msgr2=0x55d284f0cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f0610a94640 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f0610a94640 1 -- 172.21.15.105:0/2330571450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d284f0d250 con 0x55d284efc8f0 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f0611295640 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f060bfff640 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f0bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f0610a94640 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f0b760 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f060009bfb0 tx=0x7f0600098020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8990d56ce116d129 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f05ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f05ebfff640 1 -- 172.21.15.105:0/2330571450 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060009c020 con 0x55d284efc8f0 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f05ebfff640 1 -- 172.21.15.105:0/2330571450 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0600096d90 con 0x55d284efc8f0 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f06159c1740 1 -- 172.21.15.105:0/2330571450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06000b5140 con 0x55d284efc8f0 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f05ebfff640 1 -- 172.21.15.105:0/2330571450 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f060009aee0 con 0x55d284efc8f0 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f06159c1740 10 client.19890 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f06159c1740 1 -- 172.21.15.105:0/2330571450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d284f78d70 con 0x55d284efc8f0 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f05ebfff640 1 -- 172.21.15.105:0/2330571450 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06000b5140 con 0x55d284efc8f0 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f05ebfff640 1 -- 172.21.15.105:0/2330571450 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f06000a1300 con 0x55d284efc8f0 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f06159c1740 10 client.19890 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f06159c1740 10 client.19890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f06159c1740 20 client.19890 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f06159c1740 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d284f12980 0x55d284f7b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.456+0000 7f06159c1740 4 client.19890 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:50:32.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.457+0000 7f06159c1740 1 -- 172.21.15.105:0/2330571450 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d284e69210 con 0x55d284f12980 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.457+0000 7f060bfff640 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d284f12980 0x55d284f7b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.457+0000 7f05ebfff640 10 client.19890 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.457+0000 7f060bfff640 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d284f12980 0x55d284f7b480 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5e2b6c0174a94c server_cookie=a52c561f6ee5ba39 in_seq=0 out_seq=0 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f05ebfff640 1 -- 172.21.15.105:0/2330571450 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d284e69210 con 0x55d284f12980 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f05ebfff640 10 client.19890 handle_command_reply: tid=0 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 client.19890 shutdown 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 -- 172.21.15.105:0/2330571450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 msgr2=0x55d284f0b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f0b760 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f060009bfb0 tx=0x7f0600098020 comp rx=0 tx=0).stop 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 -- 172.21.15.105:0/2330571450 shutdown_connections 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d284f12980 0x55d284f7b480 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d284f02450 0x55d284f0cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d284efd2e0 0x55d284f0bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f05ebfff640 0 client.19890 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 --2- 172.21.15.105:0/2330571450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d284efc8f0 0x55d284f0b760 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 -- 172.21.15.105:0/2330571450 >> 172.21.15.105:0/2330571450 conn(0x55d284ef7e60 msgr2=0x55d284efb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 -- 172.21.15.105:0/2330571450 shutdown_connections 2024-03-26T17:50:32.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 1 -- 172.21.15.105:0/2330571450 wait complete. 2024-03-26T17:50:32.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.458+0000 7f06159c1740 20 client.19890 trim_cache size 0 max 16384 2024-03-26T17:50:32.457 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:50:32.457 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:50:32.457 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:50:32.457 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 -- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05fc0a2f20 msgr2=0x7f05fc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05fc0a2f20 0x7f05fc0a53e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55d284efdb40 tx=0x7f060402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 -- 172.21.15.105:0/3231763274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f060c1007c0 msgr2=0x7f060c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f060c1007c0 0x7f060c197b80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f060000e9c0 tx=0x7f060000ee80 comp rx=0 tx=0).stop 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 -- 172.21.15.105:0/3231763274 shutdown_connections 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05fc0a2f20 0x7f05fc0a53e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f060c102370 0x7f060c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f060c1019b0 0x7f060c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 --2- 172.21.15.105:0/3231763274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f060c1007c0 0x7f060c197b80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 -- 172.21.15.105:0/3231763274 >> 172.21.15.105:0/3231763274 conn(0x7f060c0fbd50 msgr2=0x7f060c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 -- 172.21.15.105:0/3231763274 shutdown_connections 2024-03-26T17:50:32.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.459+0000 7f0612cf8640 1 -- 172.21.15.105:0/3231763274 wait complete. 2024-03-26T17:50:32.464 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:50:32.464 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:50:32.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.585+0000 7f1fa0012640 1 -- 172.21.15.105:0/3262575437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 msgr2=0x7f1f98100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.585+0000 7f1fa0012640 1 --2- 172.21.15.105:0/3262575437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 0x7f1f98100c00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009860 tx=0x7f1f9402bb70 comp rx=0 tx=0).stop 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.585+0000 7f1fa0012640 1 -- 172.21.15.105:0/3262575437 shutdown_connections 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.585+0000 7f1fa0012640 1 --2- 172.21.15.105:0/3262575437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f98102390 0x7f1f98108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.585+0000 7f1fa0012640 1 --2- 172.21.15.105:0/3262575437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f981019d0 0x7f1f98101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.585+0000 7f1fa0012640 1 --2- 172.21.15.105:0/3262575437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 0x7f1f98100c00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.585+0000 7f1fa0012640 1 -- 172.21.15.105:0/3262575437 >> 172.21.15.105:0/3262575437 conn(0x7f1f980fbd50 msgr2=0x7f1f980fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 -- 172.21.15.105:0/3262575437 shutdown_connections 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 -- 172.21.15.105:0/3262575437 wait complete. 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 Processor -- start 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 -- start start 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 0x7f1f98197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f981019d0 0x7f1f981980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f98102390 0x7f1f9819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9810c310 con 0x7f1f981019d0 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9810c190 con 0x7f1f98102390 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.586+0000 7f1fa0012640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9810c490 con 0x7f1f981007e0 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f981019d0 0x7f1f981980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f981019d0 0x7f1f981980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38026/0 (socket says 172.21.15.105:38026) 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 -- 172.21.15.105:0/2625701026 learned_addr learned my addr 172.21.15.105:0/2625701026 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9ddae640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 0x7f1f98197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9e5af640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f98102390 0x7f1f9819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 -- 172.21.15.105:0/2625701026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 msgr2=0x7f1f98197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 0x7f1f98197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 -- 172.21.15.105:0/2625701026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f98102390 msgr2=0x7f1f9819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f98102390 0x7f1f9819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 -- 172.21.15.105:0/2625701026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9819f780 con 0x7f1f981019d0 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9ddae640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 0x7f1f98197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:32.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f9d5ad640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f981019d0 0x7f1f981980a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800e9c0 tx=0x7f1f8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f8effd640 1 -- 172.21.15.105:0/2625701026 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800cde0 con 0x7f1f981019d0 2024-03-26T17:50:32.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9819fa70 con 0x7f1f981019d0 2024-03-26T17:50:32.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f8effd640 1 -- 172.21.15.105:0/2625701026 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f880035f0 con 0x7f1f981019d0 2024-03-26T17:50:32.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1f8effd640 1 -- 172.21.15.105:0/2625701026 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88010690 con 0x7f1f981019d0 2024-03-26T17:50:32.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.587+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f981a0010 con 0x7f1f981019d0 2024-03-26T17:50:32.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.588+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f981a0240 con 0x7f1f981019d0 2024-03-26T17:50:32.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.590+0000 7f1f8effd640 1 -- 172.21.15.105:0/2625701026 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1f880037a0 con 0x7f1f981019d0 2024-03-26T17:50:32.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.590+0000 7f1f8effd640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f840a2fa0 0x7f1f840a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.591+0000 7f1f8effd640 1 -- 172.21.15.105:0/2625701026 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f1f880cf680 con 0x7f1f981019d0 2024-03-26T17:50:32.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.591+0000 7f1f9ddae640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f840a2fa0 0x7f1f840a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.591+0000 7f1f9ddae640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f840a2fa0 0x7f1f840a5460 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402c180 tx=0x7f1f940349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.596+0000 7f1f8effd640 1 -- 172.21.15.105:0/2625701026 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f88009510 con 0x7f1f981019d0 2024-03-26T17:50:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 -- 172.21.15.105:0/2117712053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe348100820 msgr2=0x7fe348100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 --2- 172.21.15.105:0/2117712053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe348100820 0x7fe348100c40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe344009900 tx=0x7fe34402bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 -- 172.21.15.105:0/2117712053 shutdown_connections 2024-03-26T17:50:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 --2- 172.21.15.105:0/2117712053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3481023d0 0x7fe348108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 --2- 172.21.15.105:0/2117712053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe348101a10 0x7fe348101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 --2- 172.21.15.105:0/2117712053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe348100820 0x7fe348100c40 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 -- 172.21.15.105:0/2117712053 >> 172.21.15.105:0/2117712053 conn(0x7fe3480fbd90 msgr2=0x7fe3480fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 -- 172.21.15.105:0/2117712053 shutdown_connections 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.758+0000 7fe3504cc640 1 -- 172.21.15.105:0/2117712053 wait complete. 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe3504cc640 1 Processor -- start 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe3504cc640 1 -- start start 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe3504cc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe348100820 0x7fe348197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe3504cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe348101a10 0x7fe348197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe3504cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3481023d0 0x7fe34819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe3504cc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe34810c370 con 0x7fe3481023d0 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe3504cc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe34810c1f0 con 0x7fe348100820 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe3504cc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe34810c4f0 con 0x7fe348101a10 2024-03-26T17:50:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe34ea69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3481023d0 0x7fe34819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe34da67640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe348101a10 0x7fe348197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe34e268640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe348100820 0x7fe348197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe34da67640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe348101a10 0x7fe348197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46826/0 (socket says 172.21.15.105:46826) 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe34ea69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3481023d0 0x7fe34819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38044/0 (socket says 172.21.15.105:38044) 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe34e268640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe348100820 0x7fe348197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46642/0 (socket says 172.21.15.105:46642) 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.759+0000 7fe34da67640 1 -- 172.21.15.105:0/1839436395 learned_addr learned my addr 172.21.15.105:0/1839436395 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe34da67640 1 -- 172.21.15.105:0/1839436395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe348100820 msgr2=0x7fe348197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe34da67640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe348100820 0x7fe348197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe34da67640 1 -- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3481023d0 msgr2=0x7fe34819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe34da67640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3481023d0 0x7fe34819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe34da67640 1 -- 172.21.15.105:0/1839436395 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34819f6c0 con 0x7fe348101a10 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe34e268640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe348100820 0x7fe348197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe34da67640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe348101a10 0x7fe348197fe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe33800ea60 tx=0x7fe33800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe33f7fe640 1 -- 172.21.15.105:0/1839436395 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33800cde0 con 0x7fe348101a10 2024-03-26T17:50:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe3504cc640 1 -- 172.21.15.105:0/1839436395 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34819f9b0 con 0x7fe348101a10 2024-03-26T17:50:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.760+0000 7fe33f7fe640 1 -- 172.21.15.105:0/1839436395 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3380035f0 con 0x7fe348101a10 2024-03-26T17:50:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.761+0000 7fe33f7fe640 1 -- 172.21.15.105:0/1839436395 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338010690 con 0x7fe348101a10 2024-03-26T17:50:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.761+0000 7fe3504cc640 1 -- 172.21.15.105:0/1839436395 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34819ff00 con 0x7fe348101a10 2024-03-26T17:50:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.763+0000 7fe33f7fe640 1 -- 172.21.15.105:0/1839436395 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe3380037a0 con 0x7fe348101a10 2024-03-26T17:50:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.763+0000 7fe35318a740 1 Processor -- start 2024-03-26T17:50:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe35318a740 1 -- start start 2024-03-26T17:50:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe35318a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe35318a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe35318a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe35318a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b6d550a0b0 con 0x55b6d55aa3d0 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe35318a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b6d54b2960 con 0x55b6d55a99e0 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe35318a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b6d5506060 con 0x55b6d55af540 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34ea69640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34da67640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34ea69640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55a9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46644/0 (socket says 172.21.15.105:46644) 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34ea69640 1 -- 172.21.15.105:0/62172732 learned_addr learned my addr 172.21.15.105:0/62172732 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34da67640 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46842/0 (socket says 172.21.15.105:46842) 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34e268640 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe33f7fe640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe3340a2fc0 0x7fe3340a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34da67640 1 -- 172.21.15.105:0/62172732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 msgr2=0x55b6d55a9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34da67640 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55a9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34da67640 1 -- 172.21.15.105:0/62172732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 msgr2=0x55b6d55aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34da67640 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.764+0000 7fe34da67640 1 -- 172.21.15.105:0/62172732 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d54c4c60 con 0x55b6d55af540 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.765+0000 7fe34ea69640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe3340a2fc0 0x7fe3340a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.765+0000 7fe33f7fe640 1 -- 172.21.15.105:0/1839436395 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe3380cf6a0 con 0x7fe348101a10 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.765+0000 7fe34da67640 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55b19d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cd130 tx=0x7fe338091270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810d6c832c6ab6c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.765+0000 7fe33cbfe640 1 -- 172.21.15.105:0/62172732 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33809b8d0 con 0x55b6d55af540 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.765+0000 7fe33cbfe640 1 -- 172.21.15.105:0/62172732 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe338003b30 con 0x55b6d55af540 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.765+0000 7fe33cbfe640 1 -- 172.21.15.105:0/62172732 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33809b6f0 con 0x55b6d55af540 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 -- 172.21.15.105:0/62172732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 msgr2=0x55b6d55b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55b19d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cd130 tx=0x7fe338091270 comp rx=0 tx=0).stop 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe34ea69640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe3340a2fc0 0x7fe3340a5480 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe340011dc0 tx=0x7fe340009ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 -- 172.21.15.105:0/62172732 shutdown_connections 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55b19d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 --2- 172.21.15.105:0/62172732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 -- 172.21.15.105:0/62172732 >> 172.21.15.105:0/62172732 conn(0x55b6d55a0b40 msgr2=0x55b6d55a0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 -- 172.21.15.105:0/62172732 shutdown_connections 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 -- 172.21.15.105:0/62172732 wait complete. 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 -- start start 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 Processor -- start 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.766+0000 7fe35318a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55babb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe35318a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b6d55b2490 con 0x55b6d55aa3d0 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe35318a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b6d54c5db0 con 0x55b6d55a99e0 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe35318a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b6d55b2610 con 0x55b6d55af540 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe35318a740 10 client.0 _ll_register_callbacks cb 0x55b6d54ca110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38070/0 (socket says 172.21.15.105:38070) 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34e268640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 -- 172.21.15.105:0/3518764135 learned_addr learned my addr 172.21.15.105:0/3518764135 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34ea69640 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55babb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 -- 172.21.15.105:0/3518764135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 msgr2=0x55b6d55babb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55babb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 -- 172.21.15.105:0/3518764135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 msgr2=0x55b6d55b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 -- 172.21.15.105:0/3518764135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d521e210 con 0x55b6d55aa3d0 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe34da67640 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b3be0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cd1b0 tx=0x7fe338003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0596e30bd41393 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe331ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe331ffb640 1 -- 172.21.15.105:0/3518764135 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33809b6f0 con 0x55b6d55aa3d0 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe331ffb640 1 -- 172.21.15.105:0/3518764135 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe3380d5130 con 0x55b6d55aa3d0 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe35318a740 1 -- 172.21.15.105:0/3518764135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6d55175f0 con 0x55b6d55aa3d0 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe331ffb640 1 -- 172.21.15.105:0/3518764135 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe33809b8d0 con 0x55b6d55aa3d0 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe35318a740 10 client.19908 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.767+0000 7fe35318a740 1 -- 172.21.15.105:0/3518764135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6d52ea620 con 0x55b6d55aa3d0 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe331ffb640 1 -- 172.21.15.105:0/3518764135 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b6d55175f0 con 0x55b6d55aa3d0 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe331ffb640 1 -- 172.21.15.105:0/3518764135 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe338098400 con 0x55b6d55aa3d0 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe35318a740 10 client.19908 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe35318a740 10 client.19908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe35318a740 20 client.19908 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe35318a740 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b6d55bf0a0 0x55b6d55c1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe35318a740 4 client.19908 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe35318a740 1 -- 172.21.15.105:0/3518764135 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6d5511940 con 0x55b6d55bf0a0 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.768+0000 7fe34e268640 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b6d55bf0a0 0x55b6d55c1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe331ffb640 10 client.19908 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe34e268640 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b6d55bf0a0 0x55b6d55c1490 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=172c771c73ce5d29 server_cookie=aca3a31ca3c77a03 in_seq=0 out_seq=0 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe331ffb640 1 -- 172.21.15.105:0/3518764135 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b6d5511940 con 0x55b6d55bf0a0 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe331ffb640 10 client.19908 handle_command_reply: tid=0 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 client.19908 shutdown 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 -- 172.21.15.105:0/3518764135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 msgr2=0x55b6d55b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b3be0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cd1b0 tx=0x7fe338003d90 comp rx=0 tx=0).stop 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 -- 172.21.15.105:0/3518764135 shutdown_connections 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b6d55bf0a0 0x55b6d55c1490 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55af540 0x55b6d55babb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe331ffb640 0 client.19908 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b3be0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 --2- 172.21.15.105:0/3518764135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.769+0000 7fe35318a740 1 -- 172.21.15.105:0/3518764135 >> 172.21.15.105:0/3518764135 conn(0x55b6d55a0b40 msgr2=0x55b6d559fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.770+0000 7fe35318a740 1 -- 172.21.15.105:0/3518764135 shutdown_connections 2024-03-26T17:50:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.770+0000 7fe35318a740 1 -- 172.21.15.105:0/3518764135 wait complete. 2024-03-26T17:50:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.770+0000 7fe35318a740 20 client.19908 trim_cache size 0 max 16384 2024-03-26T17:50:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.787+0000 7fe35318a740 1 Processor -- start 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.787+0000 7fe35318a740 1 -- start start 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.787+0000 7fe35318a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55a99e0 0x55b6d55ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.787+0000 7fe35318a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.787+0000 7fe35318a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55af540 0x55b6d559f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.787+0000 7fe35318a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481a6b50 con 0x55b6d55a99e0 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.787+0000 7fe35318a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481a8690 con 0x55b6d55af540 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.787+0000 7fe35318a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481a7220 con 0x55b6d55aa3d0 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55a99e0 0x55b6d55ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55a99e0 0x55b6d55ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38080/0 (socket says 172.21.15.105:38080) 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 -- 172.21.15.105:0/3656798529 learned_addr learned my addr 172.21.15.105:0/3656798529 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34ea69640 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55af540 0x55b6d559f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34da67640 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 -- 172.21.15.105:0/3656798529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 msgr2=0x55b6d55bab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55bab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 -- 172.21.15.105:0/3656798529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55af540 msgr2=0x55b6d559f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55af540 0x55b6d559f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 -- 172.21.15.105:0/3656798529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d55aa7b0 con 0x55b6d55a99e0 2024-03-26T17:50:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe34e268640 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55a99e0 0x55b6d55ab500 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe344009900 tx=0x7fe34402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ecb862a4ca79b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe333fff640 1 -- 172.21.15.105:0/3656798529 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34404f020 con 0x55b6d55a99e0 2024-03-26T17:50:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe333fff640 1 -- 172.21.15.105:0/3656798529 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3440040c0 con 0x55b6d55a99e0 2024-03-26T17:50:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.788+0000 7fe333fff640 1 -- 172.21.15.105:0/3656798529 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344049070 con 0x55b6d55a99e0 2024-03-26T17:50:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 -- 172.21.15.105:0/3656798529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55a99e0 msgr2=0x55b6d55ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55a99e0 0x55b6d55ab500 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe344009900 tx=0x7fe34402c7e0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 -- 172.21.15.105:0/3656798529 shutdown_connections 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55af540 0x55b6d559f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55bab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 --2- 172.21.15.105:0/3656798529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55a99e0 0x55b6d55ab500 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 -- 172.21.15.105:0/3656798529 >> 172.21.15.105:0/3656798529 conn(0x55b6d55a0b40 msgr2=0x55b6d55aadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 -- 172.21.15.105:0/3656798529 shutdown_connections 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.789+0000 7fe35318a740 1 -- 172.21.15.105:0/3656798529 wait complete. 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 1 -- start start 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 1 Processor -- start 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55af540 0x55b6d55b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481a7fc0 con 0x55b6d55af540 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481a78f0 con 0x55b6d55a99e0 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3481a4b30 con 0x55b6d55aa3d0 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe35318a740 10 client.0 _ll_register_callbacks cb 0x7fe3400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe34e268640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe34ea69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55af540 0x55b6d55b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe34e268640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46660/0 (socket says 172.21.15.105:46660) 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe34ea69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55af540 0x55b6d55b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38090/0 (socket says 172.21.15.105:38090) 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe34e268640 1 -- 172.21.15.105:0/2312207700 learned_addr learned my addr 172.21.15.105:0/2312207700 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.790+0000 7fe34da67640 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe34ea69640 1 -- 172.21.15.105:0/2312207700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 msgr2=0x55b6d55b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe34ea69640 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe34ea69640 1 -- 172.21.15.105:0/2312207700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 msgr2=0x55b6d55b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe34ea69640 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe34ea69640 1 -- 172.21.15.105:0/2312207700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d55ba3d0 con 0x55b6d55af540 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe34da67640 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe34e268640 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe34ea69640 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55af540 0x55b6d55b9de0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe340005fd0 tx=0x7fe34001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbabac1bb8e62ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe331ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe331ffb640 1 -- 172.21.15.105:0/2312207700 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34002d060 con 0x55b6d55af540 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe331ffb640 1 -- 172.21.15.105:0/2312207700 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe340003970 con 0x55b6d55af540 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe35318a740 1 -- 172.21.15.105:0/2312207700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3440040c0 con 0x55b6d55af540 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe331ffb640 1 -- 172.21.15.105:0/2312207700 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe340003b70 con 0x55b6d55af540 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe35318a740 10 client.19914 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.791+0000 7fe35318a740 1 -- 172.21.15.105:0/2312207700 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b6d56259a0 con 0x55b6d55af540 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe331ffb640 1 -- 172.21.15.105:0/2312207700 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3440040c0 con 0x55b6d55af540 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe331ffb640 1 -- 172.21.15.105:0/2312207700 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe34001f440 con 0x55b6d55af540 2024-03-26T17:50:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe35318a740 10 client.19914 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe35318a740 10 client.19914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe35318a740 20 client.19914 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe35318a740 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b6d55bf0a0 0x55b6d5628410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe35318a740 4 client.19914 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:50:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe35318a740 1 -- 172.21.15.105:0/2312207700 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6d548f770 con 0x55b6d55bf0a0 2024-03-26T17:50:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe34e268640 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b6d55bf0a0 0x55b6d5628410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe34e268640 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b6d55bf0a0 0x55b6d5628410 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4300f9f2de6ad92 server_cookie=56fd11f6a559f63a in_seq=0 out_seq=0 2024-03-26T17:50:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.792+0000 7fe331ffb640 10 client.19914 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.834+0000 7fe331ffb640 1 -- 172.21.15.105:0/2312207700 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6d548f770 con 0x55b6d55bf0a0 2024-03-26T17:50:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.834+0000 7fe331ffb640 10 client.19914 handle_command_reply: tid=0 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.834+0000 7fe35318a740 1 client.19914 shutdown 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 -- 172.21.15.105:0/2312207700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55af540 msgr2=0x55b6d55b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55af540 0x55b6d55b9de0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe340005fd0 tx=0x7fe34001d000 comp rx=0 tx=0).stop 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 -- 172.21.15.105:0/2312207700 shutdown_connections 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b6d55bf0a0 0x55b6d5628410 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b6d55af540 0x55b6d55b9de0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe331ffb640 0 client.19914 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b6d55aa3d0 0x55b6d55b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 --2- 172.21.15.105:0/2312207700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b6d55a99e0 0x55b6d55b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 -- 172.21.15.105:0/2312207700 >> 172.21.15.105:0/2312207700 conn(0x55b6d55a0b40 msgr2=0x55b6d55a3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 -- 172.21.15.105:0/2312207700 shutdown_connections 2024-03-26T17:50:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 1 -- 172.21.15.105:0/2312207700 wait complete. 2024-03-26T17:50:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.835+0000 7fe35318a740 20 client.19914 trim_cache size 0 max 16384 2024-03-26T17:50:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:50:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:50:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:50:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 -- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe3340a2fc0 msgr2=0x7fe3340a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe3340a2fc0 0x7fe3340a5480 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe340011dc0 tx=0x7fe340009ed0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 -- 172.21.15.105:0/1839436395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe348101a10 msgr2=0x7fe348197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe348101a10 0x7fe348197fe0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe33800ea60 tx=0x7fe33800ef20 comp rx=0 tx=0).stop 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 -- 172.21.15.105:0/1839436395 shutdown_connections 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe3340a2fc0 0x7fe3340a5480 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe3481023d0 0x7fe34819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe348101a10 0x7fe348197fe0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 --2- 172.21.15.105:0/1839436395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe348100820 0x7fe348197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 -- 172.21.15.105:0/1839436395 >> 172.21.15.105:0/1839436395 conn(0x7fe3480fbd90 msgr2=0x7fe3480fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 -- 172.21.15.105:0/1839436395 shutdown_connections 2024-03-26T17:50:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:32.836+0000 7fe3504cc640 1 -- 172.21.15.105:0/1839436395 wait complete. 2024-03-26T17:50:32.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.910+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f981a05d0 con 0x7f1f981019d0 2024-03-26T17:50:32.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.911+0000 7f1f8effd640 1 -- 172.21.15.105:0/2625701026 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11941 (secure 0 0 0) 0x7f1f8809c470 con 0x7f1f981019d0 2024-03-26T17:50:32.902 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:50:32.902 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:49:14.826136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24482,14628,24487,24466]},"id":1}]} 2024-03-26T17:50:32.903 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 32 2024-03-26T17:50:32.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f840a2fa0 msgr2=0x7f1f840a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f840a2fa0 0x7f1f840a5460 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402c180 tx=0x7f1f940349a0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f981019d0 msgr2=0x7f1f981980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:32.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f981019d0 0x7f1f981980a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800e9c0 tx=0x7f1f8800ee80 comp rx=0 tx=0).stop 2024-03-26T17:50:32.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 shutdown_connections 2024-03-26T17:50:32.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f840a2fa0 0x7f1f840a5460 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f98102390 0x7f1f9819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f981019d0 0x7f1f981980a0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 --2- 172.21.15.105:0/2625701026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f981007e0 0x7f1f98197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:32.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 >> 172.21.15.105:0/2625701026 conn(0x7f1f980fbd50 msgr2=0x7f1f98068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:32.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 shutdown_connections 2024-03-26T17:50:32.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:32.914+0000 7f1fa0012640 1 -- 172.21.15.105:0/2625701026 wait complete. 2024-03-26T17:50:32.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:50:33.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 395910 3.14 MB/sec execute 556 sec latency 23.787 ms 2024-03-26T17:50:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 -- 172.21.15.105:0/1933104542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb101018d0 msgr2=0x7fcb10101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 --2- 172.21.15.105:0/1933104542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb101018d0 0x7fcb10101cf0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 -- 172.21.15.105:0/1933104542 shutdown_connections 2024-03-26T17:50:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 --2- 172.21.15.105:0/1933104542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb10103480 0x7fcb10109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 --2- 172.21.15.105:0/1933104542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb10102ac0 0x7fcb10102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 --2- 172.21.15.105:0/1933104542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb101018d0 0x7fcb10101cf0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 -- 172.21.15.105:0/1933104542 >> 172.21.15.105:0/1933104542 conn(0x7fcb100fce20 msgr2=0x7fcb100ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 -- 172.21.15.105:0/1933104542 shutdown_connections 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 -- 172.21.15.105:0/1933104542 wait complete. 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 Processor -- start 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 -- start start 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb101018d0 0x7fcb10197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.052+0000 7fcb181d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10102ac0 0x7fcb10198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb181d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb10103480 0x7fcb1019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb181d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010d3d0 con 0x7fcb10102ac0 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb181d4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010d250 con 0x7fcb101018d0 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb181d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010d550 con 0x7fcb10103480 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10102ac0 0x7fcb10198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10102ac0 0x7fcb10198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38114/0 (socket says 172.21.15.105:38114) 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 -- 172.21.15.105:0/3733483910 learned_addr learned my addr 172.21.15.105:0/3733483910 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb15f70640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb101018d0 0x7fcb10197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb16771640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb10103480 0x7fcb1019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 -- 172.21.15.105:0/3733483910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb10103480 msgr2=0x7fcb1019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb10103480 0x7fcb1019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 -- 172.21.15.105:0/3733483910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb101018d0 msgr2=0x7fcb10197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb101018d0 0x7fcb10197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 -- 172.21.15.105:0/3733483910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1019f7b0 con 0x7fcb10102ac0 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb1576f640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10102ac0 0x7fcb10198120 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000e9f0 tx=0x7fcb0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb06ffd640 1 -- 172.21.15.105:0/3733483910 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000cd80 con 0x7fcb10102ac0 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb181d4640 1 -- 172.21.15.105:0/3733483910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1019faa0 con 0x7fcb10102ac0 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb06ffd640 1 -- 172.21.15.105:0/3733483910 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb000108e0 con 0x7fcb10102ac0 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb06ffd640 1 -- 172.21.15.105:0/3733483910 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00018920 con 0x7fcb10102ac0 2024-03-26T17:50:33.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.053+0000 7fcb181d4640 1 -- 172.21.15.105:0/3733483910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a0040 con 0x7fcb10102ac0 2024-03-26T17:50:33.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.056+0000 7fcb06ffd640 1 -- 172.21.15.105:0/3733483910 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcb00010a90 con 0x7fcb10102ac0 2024-03-26T17:50:33.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.056+0000 7fcb1adc0740 1 Processor -- start 2024-03-26T17:50:33.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb1adc0740 1 -- start start 2024-03-26T17:50:33.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d62a30 0x55e877d62e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d63420 0x55e877d67f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d68590 0x55e877d6aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e877cc7970 con 0x55e877d68590 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e877c6fd70 con 0x55e877d62a30 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e877cc3920 con 0x55e877d63420 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d62a30 0x55e877d62e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d62a30 0x55e877d62e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46684/0 (socket says 172.21.15.105:46684) 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb1576f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d68590 0x55e877d6aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb06ffd640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2f20 0x7fcafc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 -- 172.21.15.105:0/3373919856 learned_addr learned my addr 172.21.15.105:0/3373919856 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb15f70640 1 --2- 172.21.15.105:0/3373919856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d63420 0x55e877d67f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2f20 0x7fcafc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 -- 172.21.15.105:0/3373919856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d63420 msgr2=0x55e877d67f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 --2- 172.21.15.105:0/3373919856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d63420 0x55e877d67f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb06ffd640 1 -- 172.21.15.105:0/3733483910 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fcb000cf700 con 0x7fcb10102ac0 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 -- 172.21.15.105:0/3373919856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d68590 msgr2=0x55e877d6aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 --2- 172.21.15.105:0/3373919856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d68590 0x55e877d6aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.057+0000 7fcb16771640 1 -- 172.21.15.105:0/3373919856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8779db3f0 con 0x55e877d62a30 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.058+0000 7fcb16771640 1 --2- 172.21.15.105:0/3373919856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d62a30 0x55e877d62e50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcb08005c40 tx=0x7fcb0800f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5e1b61953711bc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.058+0000 7fcafbfff640 1 -- 172.21.15.105:0/3373919856 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08022020 con 0x55e877d62a30 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.058+0000 7fcb16771640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2f20 0x7fcafc0a53e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcb08023000 tx=0x7fcb08021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.058+0000 7fcafbfff640 1 -- 172.21.15.105:0/3373919856 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb08013170 con 0x55e877d62a30 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.058+0000 7fcafbfff640 1 -- 172.21.15.105:0/3373919856 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08010790 con 0x55e877d62a30 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.058+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3373919856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d62a30 msgr2=0x55e877d62e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.058+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3373919856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d62a30 0x55e877d62e50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcb08005c40 tx=0x7fcb0800f3c0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3373919856 shutdown_connections 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3373919856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d68590 0x55e877d6aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3373919856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d63420 0x55e877d67f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3373919856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d62a30 0x55e877d62e50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3373919856 >> 172.21.15.105:0/3373919856 conn(0x55e877d5dfa0 msgr2=0x55e877d5e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3373919856 shutdown_connections 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3373919856 wait complete. 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 -- start start 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.059+0000 7fcb1adc0740 1 Processor -- start 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d62a30 0x55e877d70420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d63420 0x55e877d70960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d62a30 0x55e877d70420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d62a30 0x55e877d70420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38130/0 (socket says 172.21.15.105:38130) 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 -- 172.21.15.105:0/3262071212 learned_addr learned my addr 172.21.15.105:0/3262071212 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d68590 0x55e877d6cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e877d6b540 con 0x55e877d62a30 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e877c83310 con 0x55e877d63420 2024-03-26T17:50:33.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e877d6b6c0 con 0x55e877d68590 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb16771640 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d68590 0x55e877d6cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1adc0740 10 client.0 _ll_register_callbacks cb 0x55e877cbec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1576f640 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d63420 0x55e877d70960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 -- 172.21.15.105:0/3262071212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d68590 msgr2=0x55e877d6cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d68590 0x55e877d6cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 -- 172.21.15.105:0/3262071212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d63420 msgr2=0x55e877d70960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d63420 0x55e877d70960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 -- 172.21.15.105:0/3262071212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e877aa57d0 con 0x55e877d62a30 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1576f640 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d63420 0x55e877d70960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb15f70640 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d62a30 0x55e877d70420 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af8c986fb0ba986 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcaf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb08013170 con 0x55e877d62a30 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcaf97fa640 1 -- 172.21.15.105:0/3262071212 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c005030 con 0x55e877d62a30 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.060+0000 7fcaf97fa640 1 -- 172.21.15.105:0/3262071212 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fcb0c0051d0 con 0x55e877d62a30 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcaf97fa640 1 -- 172.21.15.105:0/3262071212 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c0053d0 con 0x55e877d62a30 2024-03-26T17:50:33.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1adc0740 10 client.19926 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55e877d7fff0 con 0x55e877d62a30 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcaf97fa640 1 -- 172.21.15.105:0/3262071212 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcb0c0046c0 con 0x55e877d62a30 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1adc0740 10 client.19926 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1adc0740 10 client.19926 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1adc0740 20 client.19926 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e877d81600 0x55e877d83aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1adc0740 4 client.19926 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcb08013170 con 0x55e877d81600 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1576f640 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e877d81600 0x55e877d83aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcb1576f640 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e877d81600 0x55e877d83aa0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=196695725621264e server_cookie=293739602c536353 in_seq=0 out_seq=0 2024-03-26T17:50:33.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.061+0000 7fcaf97fa640 10 client.19926 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:50:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcaf97fa640 1 -- 172.21.15.105:0/3262071212 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fcb08013170 con 0x55e877d81600 2024-03-26T17:50:33.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcaf97fa640 10 client.19926 handle_command_reply: tid=0 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 client.19926 shutdown 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d62a30 msgr2=0x55e877d70420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d62a30 0x55e877d70420 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009900 tx=0x7fcb0c004270 comp rx=0 tx=0).stop 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 shutdown_connections 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e877d81600 0x55e877d83aa0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d68590 0x55e877d6cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d63420 0x55e877d70960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcaf97fa640 0 client.19926 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/3262071212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d62a30 0x55e877d70420 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 >> 172.21.15.105:0/3262071212 conn(0x55e877d5dfa0 msgr2=0x55e877d5d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 shutdown_connections 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.062+0000 7fcb1adc0740 1 -- 172.21.15.105:0/3262071212 wait complete. 2024-03-26T17:50:33.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.063+0000 7fcb1adc0740 20 client.19926 trim_cache size 0 max 16384 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1adc0740 1 Processor -- start 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1adc0740 1 -- start start 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d6cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d5c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d5ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101a6b70 con 0x55e877d63420 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101a86b0 con 0x55e877d68590 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101a7240 con 0x55e877d62a30 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb15f70640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d6cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1576f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d5c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb15f70640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d6cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46906/0 (socket says 172.21.15.105:46906) 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb15f70640 1 -- 172.21.15.105:0/261048036 learned_addr learned my addr 172.21.15.105:0/261048036 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb16771640 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d5ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb15f70640 1 -- 172.21.15.105:0/261048036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 msgr2=0x55e877d5ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb15f70640 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d5ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb15f70640 1 -- 172.21.15.105:0/261048036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 msgr2=0x55e877d5c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb15f70640 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d5c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb15f70640 1 -- 172.21.15.105:0/261048036 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e877d5d8f0 con 0x55e877d62a30 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.077+0000 7fcb1576f640 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d5c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.078+0000 7fcb15f70640 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d6cce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c02c840 tx=0x7fcb0c044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=339c8ceb075db52b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.078+0000 7fcafb7fe640 1 -- 172.21.15.105:0/261048036 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0046c0 con 0x55e877d62a30 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.078+0000 7fcafb7fe640 1 -- 172.21.15.105:0/261048036 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c059070 con 0x55e877d62a30 2024-03-26T17:50:33.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.078+0000 7fcafb7fe640 1 -- 172.21.15.105:0/261048036 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0051d0 con 0x55e877d62a30 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 -- 172.21.15.105:0/261048036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 msgr2=0x55e877d6cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d6cce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c02c840 tx=0x7fcb0c044000 comp rx=0 tx=0).stop 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 -- 172.21.15.105:0/261048036 shutdown_connections 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d5ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d5c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/261048036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d6cce0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 -- 172.21.15.105:0/261048036 >> 172.21.15.105:0/261048036 conn(0x55e877d5dfa0 msgr2=0x55e877d6c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 -- 172.21.15.105:0/261048036 shutdown_connections 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 -- 172.21.15.105:0/261048036 wait complete. 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 -- start start 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.079+0000 7fcb1adc0740 1 Processor -- start 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d92ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1adc0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d93ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101a7fe0 con 0x55e877d63420 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101a7910 con 0x55e877d68590 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1adc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101a4b50 con 0x55e877d62a30 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1adc0740 10 client.0 _ll_register_callbacks cb 0x7fcb0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1576f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d92ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb16771640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d93ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb15f70640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1576f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d92ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38152/0 (socket says 172.21.15.105:38152) 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1576f640 1 -- 172.21.15.105:0/1228648469 learned_addr learned my addr 172.21.15.105:0/1228648469 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb15f70640 1 -- 172.21.15.105:0/1228648469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 msgr2=0x55e877d93ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb15f70640 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d93ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb15f70640 1 -- 172.21.15.105:0/1228648469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 msgr2=0x55e877d92ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb15f70640 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d92ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb15f70640 1 -- 172.21.15.105:0/1228648469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e877d944e0 con 0x55e877d62a30 2024-03-26T17:50:33.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb1576f640 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d92ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb16771640 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d93ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcb15f70640 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d929a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c05f040 tx=0x7fcb0c0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2412bd2e067be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.080+0000 7fcaf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcaf97fa640 1 -- 172.21.15.105:0/1228648469 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0037f0 con 0x55e877d62a30 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 1 -- 172.21.15.105:0/1228648469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb0c059070 con 0x55e877d62a30 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcaf97fa640 1 -- 172.21.15.105:0/1228648469 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fcb0c003990 con 0x55e877d62a30 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcaf97fa640 1 -- 172.21.15.105:0/1228648469 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c003b90 con 0x55e877d62a30 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 10 client.29719 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 1 -- 172.21.15.105:0/1228648469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e877de7a80 con 0x55e877d62a30 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcaf97fa640 1 -- 172.21.15.105:0/1228648469 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c059070 con 0x55e877d62a30 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcaf97fa640 1 -- 172.21.15.105:0/1228648469 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcb0c06fac0 con 0x55e877d62a30 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 10 client.29719 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 10 client.29719 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:50:33.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 20 client.29719 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e877de83a0 0x55e877dea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 4 client.29719 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1adc0740 1 -- 172.21.15.105:0/1228648469 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e877cd4eb0 con 0x55e877de83a0 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.081+0000 7fcb1576f640 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e877de83a0 0x55e877dea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcaf97fa640 10 client.29719 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1576f640 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e877de83a0 0x55e877dea7c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cab3689c6667b326 server_cookie=8dd85ed9b9171469 in_seq=0 out_seq=0 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcaf97fa640 1 -- 172.21.15.105:0/1228648469 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e877cd4eb0 con 0x55e877de83a0 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcaf97fa640 10 client.29719 handle_command_reply: tid=0 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 client.29719 shutdown 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 -- 172.21.15.105:0/1228648469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 msgr2=0x55e877d929a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d929a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c05f040 tx=0x7fcb0c0048f0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 -- 172.21.15.105:0/1228648469 shutdown_connections 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e877de83a0 0x55e877dea7c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e877d68590 0x55e877d93ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e877d63420 0x55e877d92ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcaf97fa640 0 client.29719 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 --2- 172.21.15.105:0/1228648469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e877d62a30 0x55e877d929a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 -- 172.21.15.105:0/1228648469 >> 172.21.15.105:0/1228648469 conn(0x55e877d5dfa0 msgr2=0x55e877d5c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 -- 172.21.15.105:0/1228648469 shutdown_connections 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 1 -- 172.21.15.105:0/1228648469 wait complete. 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.082+0000 7fcb1adc0740 20 client.29719 trim_cache size 0 max 16384 2024-03-26T17:50:33.080 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 -- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2f20 msgr2=0x7fcafc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2f20 0x7fcafc0a53e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcb08023000 tx=0x7fcb08021000 comp rx=0 tx=0).stop 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 -- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10102ac0 msgr2=0x7fcb10198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10102ac0 0x7fcb10198120 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000e9f0 tx=0x7fcb0000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 -- 172.21.15.105:0/3733483910 shutdown_connections 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcafc0a2f20 0x7fcafc0a53e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb10103480 0x7fcb1019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb10102ac0 0x7fcb10198120 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 --2- 172.21.15.105:0/3733483910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb101018d0 0x7fcb10197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 -- 172.21.15.105:0/3733483910 >> 172.21.15.105:0/3733483910 conn(0x7fcb100fce20 msgr2=0x7fcb10105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 -- 172.21.15.105:0/3733483910 shutdown_connections 2024-03-26T17:50:33.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.083+0000 7fcb181d4640 1 -- 172.21.15.105:0/3733483910 wait complete. 2024-03-26T17:50:33.085 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:50:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 -- 172.21.15.105:0/1935507355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 msgr2=0x7fc814100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 --2- 172.21.15.105:0/1935507355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 0x7fc814100be0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc808009860 tx=0x7fc80802bb70 comp rx=0 tx=0).stop 2024-03-26T17:50:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 -- 172.21.15.105:0/1935507355 shutdown_connections 2024-03-26T17:50:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 --2- 172.21.15.105:0/1935507355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc814102370 0x7fc814108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 --2- 172.21.15.105:0/1935507355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8141019b0 0x7fc814101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 --2- 172.21.15.105:0/1935507355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 0x7fc814100be0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 -- 172.21.15.105:0/1935507355 >> 172.21.15.105:0/1935507355 conn(0x7fc8140fbd50 msgr2=0x7fc8140fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 -- 172.21.15.105:0/1935507355 shutdown_connections 2024-03-26T17:50:33.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 -- 172.21.15.105:0/1935507355 wait complete. 2024-03-26T17:50:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 Processor -- start 2024-03-26T17:50:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 -- start start 2024-03-26T17:50:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 0x7fc814197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8141019b0 0x7fc8141980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.215+0000 7fc81af90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc814102370 0x7fc81419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc81af90640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc81410c2c0 con 0x7fc8141007c0 2024-03-26T17:50:33.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc81af90640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc81410c140 con 0x7fc814102370 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc81af90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc81410c440 con 0x7fc8141019b0 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 0x7fc814197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 0x7fc814197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38168/0 (socket says 172.21.15.105:38168) 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 -- 172.21.15.105:0/159817704 learned_addr learned my addr 172.21.15.105:0/159817704 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc813fff640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8141019b0 0x7fc8141980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc81952d640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc814102370 0x7fc81419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 -- 172.21.15.105:0/159817704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8141019b0 msgr2=0x7fc8141980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8141019b0 0x7fc8141980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 -- 172.21.15.105:0/159817704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc814102370 msgr2=0x7fc81419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc814102370 0x7fc81419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 -- 172.21.15.105:0/159817704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81419f7c0 con 0x7fc8141007c0 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc818d2c640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 0x7fc814197ba0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc808005940 tx=0x7fc8080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc811ffb640 1 -- 172.21.15.105:0/159817704 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808041020 con 0x7fc8141007c0 2024-03-26T17:50:33.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc81af90640 1 -- 172.21.15.105:0/159817704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81419fa50 con 0x7fc8141007c0 2024-03-26T17:50:33.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc811ffb640 1 -- 172.21.15.105:0/159817704 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8080035f0 con 0x7fc8141007c0 2024-03-26T17:50:33.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.216+0000 7fc811ffb640 1 -- 172.21.15.105:0/159817704 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808039040 con 0x7fc8141007c0 2024-03-26T17:50:33.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.217+0000 7fc81af90640 1 -- 172.21.15.105:0/159817704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc814102db0 con 0x7fc8141007c0 2024-03-26T17:50:33.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.219+0000 7fc811ffb640 1 -- 172.21.15.105:0/159817704 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc808044020 con 0x7fc8141007c0 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.219+0000 7fc81db7c740 1 Processor -- start 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.219+0000 7fc81db7c740 1 -- start start 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb443ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4441f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb4442590 0x55bfb4444a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc81db7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bfb43a1970 con 0x55bfb4442590 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc81db7c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4349d70 con 0x55bfb443d420 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc81db7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bfb439d920 con 0x55bfb443ca30 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb4442590 0x55bfb4444a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb4442590 0x55bfb4444a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38184/0 (socket says 172.21.15.105:38184) 2024-03-26T17:50:33.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 -- 172.21.15.105:0/258583829 learned_addr learned my addr 172.21.15.105:0/258583829 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc81952d640 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb443ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc818d2c640 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4441f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc811ffb640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8040a2f20 0x7fc8040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 -- 172.21.15.105:0/258583829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 msgr2=0x55bfb443ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb443ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 -- 172.21.15.105:0/258583829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 msgr2=0x55bfb4441f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4441f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 -- 172.21.15.105:0/258583829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb40b53f0 con 0x55bfb4442590 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc81952d640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8040a2f20 0x7fc8040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.220+0000 7fc813fff640 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb4442590 0x55bfb4444a20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc800009c90 tx=0x7fc800011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47317f05684fd7b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc811ffb640 1 -- 172.21.15.105:0/159817704 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc8080f0dd0 con 0x7fc8141007c0 2024-03-26T17:50:33.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc7feffd640 1 -- 172.21.15.105:0/258583829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80001e070 con 0x55bfb4442590 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc7feffd640 1 -- 172.21.15.105:0/258583829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc800019ae0 con 0x55bfb4442590 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc7feffd640 1 -- 172.21.15.105:0/258583829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80000ea40 con 0x55bfb4442590 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc81db7c740 1 -- 172.21.15.105:0/258583829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb4442590 msgr2=0x55bfb4444a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc81db7c740 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb4442590 0x55bfb4444a20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc800009c90 tx=0x7fc800011810 comp rx=0 tx=0).stop 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc81952d640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8040a2f20 0x7fc8040a53e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009a60 tx=0x7fc80c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc81db7c740 1 -- 172.21.15.105:0/258583829 shutdown_connections 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc81db7c740 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb4442590 0x55bfb4444a20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc81db7c740 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4441f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc81db7c740 1 --2- 172.21.15.105:0/258583829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb443ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.221+0000 7fc81db7c740 1 -- 172.21.15.105:0/258583829 >> 172.21.15.105:0/258583829 conn(0x55bfb4437fa0 msgr2=0x55bfb44383a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 -- 172.21.15.105:0/258583829 shutdown_connections 2024-03-26T17:50:33.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 -- 172.21.15.105:0/258583829 wait complete. 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 -- start start 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 Processor -- start 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb4447140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4447680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb444eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bfb44455e0 con 0x55bfb444c6b0 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bfb435d310 con 0x55bfb443d420 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bfb4445760 con 0x55bfb443ca30 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.222+0000 7fc81db7c740 10 client.0 _ll_register_callbacks cb 0x55bfb4398c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc813fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4447680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc818d2c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb4447140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc81952d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb444eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc813fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4447680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46748/0 (socket says 172.21.15.105:46748) 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc813fff640 1 -- 172.21.15.105:0/3967467171 learned_addr learned my addr 172.21.15.105:0/3967467171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc818d2c640 1 -- 172.21.15.105:0/3967467171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 msgr2=0x55bfb4447680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc818d2c640 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4447680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc818d2c640 1 -- 172.21.15.105:0/3967467171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 msgr2=0x55bfb444eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc818d2c640 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb444eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc818d2c640 1 -- 172.21.15.105:0/3967467171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb417f7d0 con 0x55bfb443ca30 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc818d2c640 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb4447140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc8080ee8e0 tx=0x7fc8080091e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538f658b886b1922 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc7f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc7f3fff640 1 -- 172.21.15.105:0/3967467171 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8080bce20 con 0x55bfb443ca30 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc7f3fff640 1 -- 172.21.15.105:0/3967467171 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc8080037a0 con 0x55bfb443ca30 2024-03-26T17:50:33.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc81db7c740 1 -- 172.21.15.105:0/3967467171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc800019ae0 con 0x55bfb443ca30 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc7f3fff640 1 -- 172.21.15.105:0/3967467171 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc8080044e0 con 0x55bfb443ca30 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc81db7c740 10 client.29731 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.223+0000 7fc81db7c740 1 -- 172.21.15.105:0/3967467171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bfb44456b0 con 0x55bfb443ca30 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc7f3fff640 1 -- 172.21.15.105:0/3967467171 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc800019ae0 con 0x55bfb443ca30 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc7f3fff640 1 -- 172.21.15.105:0/3967467171 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc80803c030 con 0x55bfb443ca30 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc81db7c740 10 client.29731 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc81db7c740 10 client.29731 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc81db7c740 20 client.29731 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc81db7c740 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bfb4453510 0x55bfb44559a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc81db7c740 4 client.29731 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc81db7c740 1 -- 172.21.15.105:0/3967467171 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfb43aeeb0 con 0x55bfb4453510 2024-03-26T17:50:33.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc813fff640 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bfb4453510 0x55bfb44559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc7f3fff640 10 client.29731 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.224+0000 7fc813fff640 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bfb4453510 0x55bfb44559a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f79d725ea50d7c97 server_cookie=8aeab166c0a1152a in_seq=0 out_seq=0 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc7f3fff640 1 -- 172.21.15.105:0/3967467171 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bfb43aeeb0 con 0x55bfb4453510 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc7f3fff640 10 client.29731 handle_command_reply: tid=0 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 client.29731 shutdown 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 -- 172.21.15.105:0/3967467171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 msgr2=0x55bfb4447140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb4447140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc8080ee8e0 tx=0x7fc8080091e0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 -- 172.21.15.105:0/3967467171 shutdown_connections 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bfb4453510 0x55bfb44559a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb444eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4447680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc7f3fff640 0 client.29731 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 --2- 172.21.15.105:0/3967467171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443ca30 0x55bfb4447140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 -- 172.21.15.105:0/3967467171 >> 172.21.15.105:0/3967467171 conn(0x55bfb4437fa0 msgr2=0x55bfb44371c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 -- 172.21.15.105:0/3967467171 shutdown_connections 2024-03-26T17:50:33.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 1 -- 172.21.15.105:0/3967467171 wait complete. 2024-03-26T17:50:33.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.225+0000 7fc81db7c740 20 client.29731 trim_cache size 0 max 16384 2024-03-26T17:50:33.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81db7c740 1 Processor -- start 2024-03-26T17:50:33.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81db7c740 1 -- start start 2024-03-26T17:50:33.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443ca30 0x55bfb4444980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443d420 0x55bfb4446d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb4447290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81db7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141a6b70 con 0x55bfb444c6b0 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81db7c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141a86b0 con 0x55bfb443ca30 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81db7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141a7240 con 0x55bfb443d420 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81952d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb4447290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc818d2c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443ca30 0x55bfb4444980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81952d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb4447290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38200/0 (socket says 172.21.15.105:38200) 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.248+0000 7fc81952d640 1 -- 172.21.15.105:0/1567691944 learned_addr learned my addr 172.21.15.105:0/1567691944 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc813fff640 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443d420 0x55bfb4446d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc81952d640 1 -- 172.21.15.105:0/1567691944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443d420 msgr2=0x55bfb4446d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc81952d640 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443d420 0x55bfb4446d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc81952d640 1 -- 172.21.15.105:0/1567691944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443ca30 msgr2=0x55bfb4444980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc81952d640 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443ca30 0x55bfb4444980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc81952d640 1 -- 172.21.15.105:0/1567691944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfb443d800 con 0x55bfb444c6b0 2024-03-26T17:50:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc818d2c640 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443ca30 0x55bfb4444980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc81952d640 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb4447290 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc80c003a70 tx=0x7fc80c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f326eb2f373ff6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc7fe7fc640 1 -- 172.21.15.105:0/1567691944 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c004050 con 0x55bfb444c6b0 2024-03-26T17:50:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc7fe7fc640 1 -- 172.21.15.105:0/1567691944 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc80c0046a0 con 0x55bfb444c6b0 2024-03-26T17:50:33.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.249+0000 7fc7fe7fc640 1 -- 172.21.15.105:0/1567691944 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c020c00 con 0x55bfb444c6b0 2024-03-26T17:50:33.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 -- 172.21.15.105:0/1567691944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 msgr2=0x55bfb4447290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb4447290 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc80c003a70 tx=0x7fc80c0095c0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 -- 172.21.15.105:0/1567691944 shutdown_connections 2024-03-26T17:50:33.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb444c6b0 0x55bfb4447290 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb443d420 0x55bfb4446d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 --2- 172.21.15.105:0/1567691944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443ca30 0x55bfb4444980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 -- 172.21.15.105:0/1567691944 >> 172.21.15.105:0/1567691944 conn(0x55bfb4437fa0 msgr2=0x55bfb443eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 -- 172.21.15.105:0/1567691944 shutdown_connections 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 -- 172.21.15.105:0/1567691944 wait complete. 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 -- start start 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.250+0000 7fc81db7c740 1 Processor -- start 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb443ca30 0x55bfb4464850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4464d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81db7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb444c6b0 0x55bfb4465d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81db7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141a7fe0 con 0x55bfb443ca30 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81db7c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141a7910 con 0x55bfb443d420 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81db7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8141a4b50 con 0x55bfb444c6b0 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81db7c740 10 client.0 _ll_register_callbacks cb 0x7fc8080bd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb443ca30 0x55bfb4464850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb443ca30 0x55bfb4464850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38206/0 (socket says 172.21.15.105:38206) 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 -- 172.21.15.105:0/294247018 learned_addr learned my addr 172.21.15.105:0/294247018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc813fff640 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4464d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81952d640 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb444c6b0 0x55bfb4465d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 -- 172.21.15.105:0/294247018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb444c6b0 msgr2=0x55bfb4465d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb444c6b0 0x55bfb4465d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 -- 172.21.15.105:0/294247018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 msgr2=0x55bfb4464d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4464d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 -- 172.21.15.105:0/294247018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfb4466320 con 0x55bfb443ca30 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc813fff640 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4464d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc818d2c640 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb443ca30 0x55bfb4464850 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc8080eec00 tx=0x7fc8080bb9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194027d820efa9a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc7f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc7f3fff640 1 -- 172.21.15.105:0/294247018 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8080c5c90 con 0x55bfb443ca30 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc7f3fff640 1 -- 172.21.15.105:0/294247018 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc8080bc910 con 0x55bfb443ca30 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.251+0000 7fc81db7c740 1 -- 172.21.15.105:0/294247018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc80c0046a0 con 0x55bfb443ca30 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc7f3fff640 1 -- 172.21.15.105:0/294247018 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc8080bdbc0 con 0x55bfb443ca30 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc81db7c740 10 client.19950 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc81db7c740 1 -- 172.21.15.105:0/294247018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bfb44b98c0 con 0x55bfb443ca30 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc7f3fff640 1 -- 172.21.15.105:0/294247018 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc80c0046a0 con 0x55bfb443ca30 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc7f3fff640 1 -- 172.21.15.105:0/294247018 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc8080b9210 con 0x55bfb443ca30 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc81db7c740 10 client.19950 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc81db7c740 10 client.19950 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc81db7c740 20 client.19950 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc81db7c740 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bfb44b9e60 0x55bfb44bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc81db7c740 4 client.19950 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc81db7c740 1 -- 172.21.15.105:0/294247018 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfb43a9200 con 0x55bfb44b9e60 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.252+0000 7fc813fff640 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bfb44b9e60 0x55bfb44bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc813fff640 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bfb44b9e60 0x55bfb44bc2a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=718093b49ac1cc58 server_cookie=1e0985f18e4bee6b in_seq=0 out_seq=0 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc7f3fff640 10 client.19950 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc7f3fff640 1 -- 172.21.15.105:0/294247018 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfb43a9200 con 0x55bfb44b9e60 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc7f3fff640 10 client.19950 handle_command_reply: tid=0 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 client.19950 shutdown 2024-03-26T17:50:33.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 -- 172.21.15.105:0/294247018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb443ca30 msgr2=0x55bfb4464850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb443ca30 0x55bfb4464850 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc8080eec00 tx=0x7fc8080bb9d0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 -- 172.21.15.105:0/294247018 shutdown_connections 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bfb44b9e60 0x55bfb44bc2a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfb444c6b0 0x55bfb4465d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfb443d420 0x55bfb4464d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc7f3fff640 0 client.19950 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 --2- 172.21.15.105:0/294247018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfb443ca30 0x55bfb4464850 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 -- 172.21.15.105:0/294247018 >> 172.21.15.105:0/294247018 conn(0x55bfb4437fa0 msgr2=0x55bfb4446d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 -- 172.21.15.105:0/294247018 shutdown_connections 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 1 -- 172.21.15.105:0/294247018 wait complete. 2024-03-26T17:50:33.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.253+0000 7fc81db7c740 20 client.19950 trim_cache size 0 max 16384 2024-03-26T17:50:33.251 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:33.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 -- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8040a2f20 msgr2=0x7fc8040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8040a2f20 0x7fc8040a53e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009a60 tx=0x7fc80c004ab0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 -- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 msgr2=0x7fc814197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 0x7fc814197ba0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc808005940 tx=0x7fc8080042f0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 -- 172.21.15.105:0/159817704 shutdown_connections 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc8040a2f20 0x7fc8040a53e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc814102370 0x7fc81419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8141019b0 0x7fc8141980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 --2- 172.21.15.105:0/159817704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8141007c0 0x7fc814197ba0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 -- 172.21.15.105:0/159817704 >> 172.21.15.105:0/159817704 conn(0x7fc8140fbd50 msgr2=0x7fc814068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 -- 172.21.15.105:0/159817704 shutdown_connections 2024-03-26T17:50:33.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.254+0000 7fc81af90640 1 -- 172.21.15.105:0/159817704 wait complete. 2024-03-26T17:50:33.259 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 -- 172.21.15.105:0/2194830749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1007c0 msgr2=0x7f261c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 --2- 172.21.15.105:0/2194830749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1007c0 0x7f261c100be0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f2614009ca0 tx=0x7f261402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 -- 172.21.15.105:0/2194830749 shutdown_connections 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 --2- 172.21.15.105:0/2194830749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f261c102370 0x7f261c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 --2- 172.21.15.105:0/2194830749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f261c1019b0 0x7f261c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 --2- 172.21.15.105:0/2194830749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1007c0 0x7f261c100be0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 -- 172.21.15.105:0/2194830749 >> 172.21.15.105:0/2194830749 conn(0x7f261c0fbd50 msgr2=0x7f261c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 -- 172.21.15.105:0/2194830749 shutdown_connections 2024-03-26T17:50:33.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.385+0000 7f262126e640 1 -- 172.21.15.105:0/2194830749 wait complete. 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.386+0000 7f262126e640 1 Processor -- start 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.386+0000 7f262126e640 1 -- start start 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f262126e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f261c1007c0 0x7f261c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f262126e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1019b0 0x7f261c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f262126e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f261c102370 0x7f261c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f262126e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10c2f0 con 0x7f261c1019b0 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f262126e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10c170 con 0x7f261c1007c0 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f262126e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10c470 con 0x7f261c102370 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1019b0 0x7f261c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1019b0 0x7f261c196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38218/0 (socket says 172.21.15.105:38218) 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 -- 172.21.15.105:0/3297265232 learned_addr learned my addr 172.21.15.105:0/3297265232 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f261c102370 0x7f261c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261affd640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f261c1007c0 0x7f261c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 -- 172.21.15.105:0/3297265232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f261c102370 msgr2=0x7f261c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f261c102370 0x7f261c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 -- 172.21.15.105:0/3297265232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f261c1007c0 msgr2=0x7f261c196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f261c1007c0 0x7f261c196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 -- 172.21.15.105:0/3297265232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c19e670 con 0x7f261c1019b0 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261b7fe640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f261c102370 0x7f261c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f261a7fc640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1019b0 0x7f261c196f90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f261000eb00 tx=0x7f2610009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f2603fff640 1 -- 172.21.15.105:0/3297265232 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261001c070 con 0x7f261c1019b0 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f262126e640 1 -- 172.21.15.105:0/3297265232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c068a80 con 0x7f261c1019b0 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f2603fff640 1 -- 172.21.15.105:0/3297265232 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f261000bc30 con 0x7f261c1019b0 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f2603fff640 1 -- 172.21.15.105:0/3297265232 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610016610 con 0x7f261c1019b0 2024-03-26T17:50:33.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.387+0000 7f262126e640 1 -- 172.21.15.105:0/3297265232 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c069020 con 0x7f261c1019b0 2024-03-26T17:50:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.390+0000 7f2603fff640 1 -- 172.21.15.105:0/3297265232 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f261001f070 con 0x7f261c1019b0 2024-03-26T17:50:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.390+0000 7f2603fff640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26040a2f20 0x7f26040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.390+0000 7f261affd640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26040a2f20 0x7f26040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.390+0000 7f2603fff640 1 -- 172.21.15.105:0/3297265232 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f26100d09f0 con 0x7f261c1019b0 2024-03-26T17:50:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f2623e56740 1 Processor -- start 2024-03-26T17:50:33.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f2623e56740 1 -- start start 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f2623e56740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 0x563a62fe9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f2623e56740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62feef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f2623e56740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ff1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f2623e56740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563a62f4e970 con 0x563a62fea420 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f2623e56740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563a62ef6d70 con 0x563a62fe9a30 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f2623e56740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a62f4a920 con 0x563a62fef590 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261affd640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26040a2f20 0x7f26040a53e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2614032380 tx=0x7f2614000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ff1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ff1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47012/0 (socket says 172.21.15.105:47012) 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261b7fe640 1 -- 172.21.15.105:0/1718991554 learned_addr learned my addr 172.21.15.105:0/1718991554 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62feef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261affd640 1 --2- 172.21.15.105:0/1718991554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 0x563a62fe9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261b7fe640 1 -- 172.21.15.105:0/1718991554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 msgr2=0x563a62fe9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261b7fe640 1 --2- 172.21.15.105:0/1718991554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 0x563a62fe9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261b7fe640 1 -- 172.21.15.105:0/1718991554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 msgr2=0x563a62feef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261b7fe640 1 --2- 172.21.15.105:0/1718991554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62feef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.391+0000 7f261b7fe640 1 -- 172.21.15.105:0/1718991554 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a62c623f0 con 0x563a62fef590 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.392+0000 7f261b7fe640 1 --2- 172.21.15.105:0/1718991554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ff1a20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f260c011540 tx=0x7f260c011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6aa590618cf2a01 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.392+0000 7f26017fa640 1 -- 172.21.15.105:0/1718991554 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c00f5c0 con 0x563a62fef590 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.392+0000 7f26017fa640 1 -- 172.21.15.105:0/1718991554 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260c00fc10 con 0x563a62fef590 2024-03-26T17:50:33.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.392+0000 7f26017fa640 1 -- 172.21.15.105:0/1718991554 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c019c30 con 0x563a62fef590 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 -- 172.21.15.105:0/1718991554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 msgr2=0x563a62ff1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 --2- 172.21.15.105:0/1718991554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ff1a20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f260c011540 tx=0x7f260c011a00 comp rx=0 tx=0).stop 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 -- 172.21.15.105:0/1718991554 shutdown_connections 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 --2- 172.21.15.105:0/1718991554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ff1a20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 --2- 172.21.15.105:0/1718991554 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62feef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 --2- 172.21.15.105:0/1718991554 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 0x563a62fe9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 -- 172.21.15.105:0/1718991554 >> 172.21.15.105:0/1718991554 conn(0x563a62fe4fa0 msgr2=0x563a62fe53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 -- 172.21.15.105:0/1718991554 shutdown_connections 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 -- 172.21.15.105:0/1718991554 wait complete. 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 -- start start 2024-03-26T17:50:33.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 Processor -- start 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 0x563a62ff3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fea420 0x563a62ff4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fef590 0x563a62ffb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.393+0000 7f2623e56740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563a62ff2540 con 0x563a62fef590 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f2623e56740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563a62f0a310 con 0x563a62fe9a30 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f2623e56740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563a62ff26c0 con 0x563a62fea420 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f2623e56740 10 client.0 _ll_register_callbacks cb 0x563a62f45c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fef590 0x563a62ffb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fef590 0x563a62ffb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38238/0 (socket says 172.21.15.105:38238) 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 0x563a62ff3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 -- 172.21.15.105:0/2738575262 learned_addr learned my addr 172.21.15.105:0/2738575262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261a7fc640 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fea420 0x563a62ff4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 -- 172.21.15.105:0/2738575262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fea420 msgr2=0x563a62ff4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fea420 0x563a62ff4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 -- 172.21.15.105:0/2738575262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 msgr2=0x563a62ff3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 0x563a62ff3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 -- 172.21.15.105:0/2738575262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a62d2c7d0 con 0x563a62fef590 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f261b7fe640 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fef590 0x563a62ffb4d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f260c01b040 tx=0x7f260c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaad59de0b25926 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f25feff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f25feff5640 1 -- 172.21.15.105:0/2738575262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c004480 con 0x563a62fef590 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f25feff5640 1 -- 172.21.15.105:0/2738575262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f260c00f5c0 con 0x563a62fef590 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f2623e56740 1 -- 172.21.15.105:0/2738575262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260c00fc10 con 0x563a62fef590 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f25feff5640 1 -- 172.21.15.105:0/2738575262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f260c00f7c0 con 0x563a62fef590 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f2623e56740 10 client.19962 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.394+0000 7f2623e56740 1 -- 172.21.15.105:0/2738575262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a62ffe840 con 0x563a62fef590 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f25feff5640 1 -- 172.21.15.105:0/2738575262 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260c00fc10 con 0x563a62fef590 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f25feff5640 1 -- 172.21.15.105:0/2738575262 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f260c0046c0 con 0x563a62fef590 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f2623e56740 10 client.19962 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f2623e56740 10 client.19962 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f2623e56740 20 client.19962 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f2623e56740 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563a62fffd60 0x563a630021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f2623e56740 4 client.19962 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f2623e56740 1 -- 172.21.15.105:0/2738575262 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a62f5beb0 con 0x563a62fffd60 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.395+0000 7f261affd640 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563a62fffd60 0x563a630021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f25feff5640 10 client.19962 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f261affd640 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563a62fffd60 0x563a630021d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c1a2c44a439906e server_cookie=e7b7eb7e18b19b2 in_seq=0 out_seq=0 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f25feff5640 1 -- 172.21.15.105:0/2738575262 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563a62f5beb0 con 0x563a62fffd60 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f25feff5640 10 client.19962 handle_command_reply: tid=0 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 client.19962 shutdown 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 -- 172.21.15.105:0/2738575262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fef590 msgr2=0x563a62ffb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fef590 0x563a62ffb4d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f260c01b040 tx=0x7f260c009e60 comp rx=0 tx=0).stop 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 -- 172.21.15.105:0/2738575262 shutdown_connections 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fef590 0x563a62ffb4d0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fea420 0x563a62ff4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563a62fffd60 0x563a630021d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f25feff5640 0 client.19962 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:50:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 --2- 172.21.15.105:0/2738575262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fe9a30 0x563a62ff3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.396+0000 7f2623e56740 1 -- 172.21.15.105:0/2738575262 >> 172.21.15.105:0/2738575262 conn(0x563a62fe4fa0 msgr2=0x563a62fe41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.397+0000 7f2623e56740 1 -- 172.21.15.105:0/2738575262 shutdown_connections 2024-03-26T17:50:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.397+0000 7f2623e56740 1 -- 172.21.15.105:0/2738575262 wait complete. 2024-03-26T17:50:33.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.397+0000 7f2623e56740 20 client.19962 trim_cache size 0 max 16384 2024-03-26T17:50:33.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f2623e56740 1 Processor -- start 2024-03-26T17:50:33.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f2623e56740 1 -- start start 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f2623e56740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fe9a30 0x563a62fe0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f2623e56740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62ffb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f2623e56740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fef590 0x563a62fe3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f2623e56740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f261c1a6bf0 con 0x563a62fea420 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f2623e56740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f261c1a8730 con 0x563a62fef590 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f2623e56740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f261c1a72c0 con 0x563a62fe9a30 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f261a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62ffb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f261a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62ffb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38240/0 (socket says 172.21.15.105:38240) 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f261a7fc640 1 -- 172.21.15.105:0/3479746054 learned_addr learned my addr 172.21.15.105:0/3479746054 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f261b7fe640 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fef590 0x563a62fe3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.411+0000 7f261affd640 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fe9a30 0x563a62fe0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f261a7fc640 1 -- 172.21.15.105:0/3479746054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fe9a30 msgr2=0x563a62fe0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f261a7fc640 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fe9a30 0x563a62fe0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f261a7fc640 1 -- 172.21.15.105:0/3479746054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fef590 msgr2=0x563a62fe3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f261a7fc640 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fef590 0x563a62fe3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f261a7fc640 1 -- 172.21.15.105:0/3479746054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a62fe0820 con 0x563a62fea420 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f261a7fc640 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62ffb490 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f261009ce90 tx=0x7f26100d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125c2f2bfac97b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f2600ff9640 1 -- 172.21.15.105:0/3479746054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261009d190 con 0x563a62fea420 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f2600ff9640 1 -- 172.21.15.105:0/3479746054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26100bbe50 con 0x563a62fea420 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.412+0000 7f2600ff9640 1 -- 172.21.15.105:0/3479746054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100a5660 con 0x563a62fea420 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- 172.21.15.105:0/3479746054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 msgr2=0x563a62ffb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62ffb490 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f261009ce90 tx=0x7f26100d0040 comp rx=0 tx=0).stop 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- 172.21.15.105:0/3479746054 shutdown_connections 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fef590 0x563a62fe3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fea420 0x563a62ffb490 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 --2- 172.21.15.105:0/3479746054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fe9a30 0x563a62fe0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- 172.21.15.105:0/3479746054 >> 172.21.15.105:0/3479746054 conn(0x563a62fe4fa0 msgr2=0x563a62ff3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- 172.21.15.105:0/3479746054 shutdown_connections 2024-03-26T17:50:33.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- 172.21.15.105:0/3479746054 wait complete. 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- start start 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 Processor -- start 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fe9a30 0x563a62ff8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fea420 0x563a62ff9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ffa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f261c1a8060 con 0x563a62fe9a30 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f261c1a7990 con 0x563a62fea420 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f261c1a4bd0 con 0x563a62fef590 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f2623e56740 10 client.0 _ll_register_callbacks cb 0x7f260c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f261affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fe9a30 0x563a62ff8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f261affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fe9a30 0x563a62ff8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38254/0 (socket says 172.21.15.105:38254) 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f261affd640 1 -- 172.21.15.105:0/2148188537 learned_addr learned my addr 172.21.15.105:0/2148188537 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f261a7fc640 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fea420 0x563a62ff9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.413+0000 7f261b7fe640 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ffa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f261a7fc640 1 -- 172.21.15.105:0/2148188537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 msgr2=0x563a62ffa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f261a7fc640 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ffa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f261a7fc640 1 -- 172.21.15.105:0/2148188537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fe9a30 msgr2=0x563a62ff8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f261a7fc640 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fe9a30 0x563a62ff8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f261a7fc640 1 -- 172.21.15.105:0/2148188537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a62ffa800 con 0x563a62fea420 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f261affd640 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fe9a30 0x563a62ff8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f261a7fc640 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fea420 0x563a62ff9250 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f261009d120 tx=0x7f2610017b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9540e5c4da589e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f25feff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f25feff5640 1 -- 172.21.15.105:0/2148188537 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100a5af0 con 0x563a62fea420 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f2623e56740 1 -- 172.21.15.105:0/2148188537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26100bbe50 con 0x563a62fea420 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f25feff5640 1 -- 172.21.15.105:0/2148188537 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f26100a5870 con 0x563a62fea420 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f25feff5640 1 -- 172.21.15.105:0/2148188537 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f261009d190 con 0x563a62fea420 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f2623e56740 10 client.29768 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f2623e56740 1 -- 172.21.15.105:0/2148188537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a630661c0 con 0x563a62fea420 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.414+0000 7f25feff5640 1 -- 172.21.15.105:0/2148188537 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26100bbe50 con 0x563a62fea420 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f25feff5640 1 -- 172.21.15.105:0/2148188537 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f261009c250 con 0x563a62fea420 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 10 client.29768 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 10 client.29768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 20 client.29768 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563a62fffd60 0x563a63068b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 4 client.29768 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 1 -- 172.21.15.105:0/2148188537 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a62f56200 con 0x563a62fffd60 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f261affd640 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563a62fffd60 0x563a63068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f261affd640 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563a62fffd60 0x563a63068b50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8638c745da95391 server_cookie=50a74789e9f7a538 in_seq=0 out_seq=0 2024-03-26T17:50:33.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f25feff5640 10 client.29768 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f25feff5640 1 -- 172.21.15.105:0/2148188537 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a62f56200 con 0x563a62fffd60 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f25feff5640 10 client.29768 handle_command_reply: tid=0 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 1 client.29768 shutdown 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 1 -- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fea420 msgr2=0x563a62ff9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.415+0000 7f2623e56740 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fea420 0x563a62ff9250 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f261009d120 tx=0x7f2610017b90 comp rx=0 tx=0).stop 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 1 -- 172.21.15.105:0/2148188537 shutdown_connections 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563a62fef590 0x563a62ffa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563a62fffd60 0x563a63068b50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f25feff5640 0 client.29768 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563a62fea420 0x563a62ff9250 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 1 --2- 172.21.15.105:0/2148188537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563a62fe9a30 0x563a62ff8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 1 -- 172.21.15.105:0/2148188537 >> 172.21.15.105:0/2148188537 conn(0x563a62fe4fa0 msgr2=0x563a62fe8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 1 -- 172.21.15.105:0/2148188537 shutdown_connections 2024-03-26T17:50:33.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 1 -- 172.21.15.105:0/2148188537 wait complete. 2024-03-26T17:50:33.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f2623e56740 20 client.29768 trim_cache size 0 max 16384 2024-03-26T17:50:33.414 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 -- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26040a2f20 msgr2=0x7f26040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26040a2f20 0x7f26040a53e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2614032380 tx=0x7f2614000f40 comp rx=0 tx=0).stop 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 -- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1019b0 msgr2=0x7f261c196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1019b0 0x7f261c196f90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f261000eb00 tx=0x7f2610009f20 comp rx=0 tx=0).stop 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 -- 172.21.15.105:0/3297265232 shutdown_connections 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f26040a2f20 0x7f26040a53e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f261c102370 0x7f261c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f261c1019b0 0x7f261c196f90 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 --2- 172.21.15.105:0/3297265232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f261c1007c0 0x7f261c196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 -- 172.21.15.105:0/3297265232 >> 172.21.15.105:0/3297265232 conn(0x7f261c0fbd50 msgr2=0x7f261c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 -- 172.21.15.105:0/3297265232 shutdown_connections 2024-03-26T17:50:33.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.416+0000 7f262126e640 1 -- 172.21.15.105:0/3297265232 wait complete. 2024-03-26T17:50:33.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:50:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 -- 172.21.15.105:0/1264840580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4102340 msgr2=0x7f80f4104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 --2- 172.21.15.105:0/1264840580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4102340 0x7f80f4104730 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009900 tx=0x7f80e402bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:33.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 -- 172.21.15.105:0/1264840580 shutdown_connections 2024-03-26T17:50:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 --2- 172.21.15.105:0/1264840580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f41075a0 0x7f80f41099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 --2- 172.21.15.105:0/1264840580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4104c70 0x7f80f4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 --2- 172.21.15.105:0/1264840580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f4102340 0x7f80f4104730 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 -- 172.21.15.105:0/1264840580 >> 172.21.15.105:0/1264840580 conn(0x7f80f40fbd50 msgr2=0x7f80f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 -- 172.21.15.105:0/1264840580 shutdown_connections 2024-03-26T17:50:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 -- 172.21.15.105:0/1264840580 wait complete. 2024-03-26T17:50:33.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.553+0000 7f80f9752640 1 Processor -- start 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f9752640 1 -- start start 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f9752640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102340 0x7f80f4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f9752640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f4104c70 0x7f80f41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f9752640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f41075a0 0x7f80f419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f9752640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80f410d420 con 0x7f80f41075a0 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f9752640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80f410d2a0 con 0x7f80f4104c70 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f9752640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80f410d5a0 con 0x7f80f4102340 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102340 0x7f80f4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f41075a0 0x7f80f419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102340 0x7f80f4197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47068/0 (socket says 172.21.15.105:47068) 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f2ffd640 1 -- 172.21.15.105:0/3670987435 learned_addr learned my addr 172.21.15.105:0/3670987435 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f27fc640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f4104c70 0x7f80f41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f2ffd640 1 -- 172.21.15.105:0/3670987435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f4104c70 msgr2=0x7f80f41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f4104c70 0x7f80f41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f2ffd640 1 -- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f41075a0 msgr2=0x7f80f419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f41075a0 0x7f80f419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f2ffd640 1 -- 172.21.15.105:0/3670987435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f419f7a0 con 0x7f80f4102340 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.554+0000 7f80f37fe640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f41075a0 0x7f80f419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.555+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102340 0x7f80f4197b80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f80e4006010 tx=0x7f80e4005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.555+0000 7f80dbfff640 1 -- 172.21.15.105:0/3670987435 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4043020 con 0x7f80f4102340 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.555+0000 7f80f9752640 1 -- 172.21.15.105:0/3670987435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f419fa30 con 0x7f80f4102340 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.555+0000 7f80dbfff640 1 -- 172.21.15.105:0/3670987435 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80e40043d0 con 0x7f80f4102340 2024-03-26T17:50:33.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.555+0000 7f80dbfff640 1 -- 172.21.15.105:0/3670987435 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4003b80 con 0x7f80f4102340 2024-03-26T17:50:33.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.555+0000 7f80f9752640 1 -- 172.21.15.105:0/3670987435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f419ffd0 con 0x7f80f4102340 2024-03-26T17:50:33.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.557+0000 7f80fc40f740 1 Processor -- start 2024-03-26T17:50:33.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80fc40f740 1 -- start start 2024-03-26T17:50:33.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80dbfff640 1 -- 172.21.15.105:0/3670987435 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f80e4046020 con 0x7f80f4102340 2024-03-26T17:50:33.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbaba30 0x55a16dbabe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbac420 0x55a16dbb0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 0x55a16dbb3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80fc40f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a16db10970 con 0x55a16dbaba30 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80fc40f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a16dab8d70 con 0x55a16dbac420 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80fc40f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a16db0c920 con 0x55a16dbb1590 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbaba30 0x55a16dbabe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbac420 0x55a16dbb0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbaba30 0x55a16dbabe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38280/0 (socket says 172.21.15.105:38280) 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f37fe640 1 -- 172.21.15.105:0/2831757734 learned_addr learned my addr 172.21.15.105:0/2831757734 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f27fc640 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 0x55a16dbb3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f2ffd640 1 -- 172.21.15.105:0/2831757734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 msgr2=0x55a16dbb3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 0x55a16dbb3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f2ffd640 1 -- 172.21.15.105:0/2831757734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbaba30 msgr2=0x55a16dbabe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbaba30 0x55a16dbabe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f2ffd640 1 -- 172.21.15.105:0/2831757734 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16d8243f0 con 0x55a16dbac420 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80f37fe640 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbaba30 0x55a16dbabe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.558+0000 7f80dbfff640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.559+0000 7f80f37fe640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.559+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbac420 0x55a16dbb0f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f80e40ef090 tx=0x7f80e40f7b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3124b70dfe80aa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.559+0000 7f80dbfff640 1 -- 172.21.15.105:0/3670987435 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f80e40f1600 con 0x7f80f4102340 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.559+0000 7f80d97fa640 1 -- 172.21.15.105:0/2831757734 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e40b9f90 con 0x55a16dbac420 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.559+0000 7f80d97fa640 1 -- 172.21.15.105:0/2831757734 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80e40be3f0 con 0x55a16dbac420 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.559+0000 7f80d97fa640 1 -- 172.21.15.105:0/2831757734 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e40bd650 con 0x55a16dbac420 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80f37fe640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55a16dbacc80 tx=0x7f80ec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- 172.21.15.105:0/2831757734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbac420 msgr2=0x55a16dbb0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbac420 0x55a16dbb0f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f80e40ef090 tx=0x7f80e40f7b40 comp rx=0 tx=0).stop 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- 172.21.15.105:0/2831757734 shutdown_connections 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 0x55a16dbb3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbac420 0x55a16dbb0f20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2831757734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbaba30 0x55a16dbabe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- 172.21.15.105:0/2831757734 >> 172.21.15.105:0/2831757734 conn(0x55a16dba6fa0 msgr2=0x55a16dba73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- 172.21.15.105:0/2831757734 shutdown_connections 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- 172.21.15.105:0/2831757734 wait complete. 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- start start 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 Processor -- start 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dbb5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbb6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 0x55a16dbbd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a16dbb4300 con 0x55a16dbac420 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a16dacc310 con 0x55a16dbb1590 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a16dbb4480 con 0x55a16dbaba30 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.560+0000 7f80fc40f740 10 client.0 _ll_register_callbacks cb 0x55a16db07c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbb6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dbb5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbb6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38292/0 (socket says 172.21.15.105:38292) 2024-03-26T17:50:33.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 0x55a16dbbd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 -- 172.21.15.105:0/2368166910 learned_addr learned my addr 172.21.15.105:0/2368166910 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 -- 172.21.15.105:0/2368166910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 msgr2=0x55a16dbb5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dbb5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 -- 172.21.15.105:0/2368166910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 msgr2=0x55a16dbbd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 0x55a16dbbd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 -- 172.21.15.105:0/2368166910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16d8ee7d0 con 0x55a16dbac420 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dbb5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80f27fc640 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbb6180 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f80e8012700 tx=0x7f80e8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ed889c91954b98 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80d6ff5640 1 -- 172.21.15.105:0/2368166910 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8004020 con 0x55a16dbac420 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80d6ff5640 1 -- 172.21.15.105:0/2368166910 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f80e80041c0 con 0x55a16dbac420 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80fc40f740 1 -- 172.21.15.105:0/2368166910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e40be3f0 con 0x55a16dbac420 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80d6ff5640 1 -- 172.21.15.105:0/2368166910 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80e80043c0 con 0x55a16dbac420 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80fc40f740 10 client.19980 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.561+0000 7f80fc40f740 1 -- 172.21.15.105:0/2368166910 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a16dbc0600 con 0x55a16dbac420 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80d6ff5640 1 -- 172.21.15.105:0/2368166910 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80e40be3f0 con 0x55a16dbac420 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80d6ff5640 1 -- 172.21.15.105:0/2368166910 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f80e80039a0 con 0x55a16dbac420 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80fc40f740 10 client.19980 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80fc40f740 10 client.19980 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80fc40f740 20 client.19980 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a16dbc1cd0 0x55a16dbc4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80fc40f740 4 client.19980 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80fc40f740 1 -- 172.21.15.105:0/2368166910 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a16db1deb0 con 0x55a16dbc1cd0 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a16dbc1cd0 0x55a16dbc4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a16dbc1cd0 0x55a16dbc4190 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a5fefd4977728d server_cookie=d55fb14aa565ca64 in_seq=0 out_seq=0 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.562+0000 7f80d6ff5640 10 client.19980 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.563+0000 7f80d6ff5640 1 -- 172.21.15.105:0/2368166910 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a16db1deb0 con 0x55a16dbc1cd0 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.563+0000 7f80d6ff5640 10 client.19980 handle_command_reply: tid=0 2024-03-26T17:50:33.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.563+0000 7f80fc40f740 1 client.19980 shutdown 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.563+0000 7f80fc40f740 1 -- 172.21.15.105:0/2368166910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 msgr2=0x55a16dbb6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.563+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbb6180 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f80e8012700 tx=0x7f80e8012bc0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 1 -- 172.21.15.105:0/2368166910 shutdown_connections 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a16dbc1cd0 0x55a16dbc4190 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 0x55a16dbbd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbb6180 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80d6ff5640 0 client.19980 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2368166910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dbb5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 1 -- 172.21.15.105:0/2368166910 >> 172.21.15.105:0/2368166910 conn(0x55a16dba6fa0 msgr2=0x55a16dba61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 1 -- 172.21.15.105:0/2368166910 shutdown_connections 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 1 -- 172.21.15.105:0/2368166910 wait complete. 2024-03-26T17:50:33.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.564+0000 7f80fc40f740 20 client.19980 trim_cache size 0 max 16384 2024-03-26T17:50:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.580+0000 7f80fc40f740 1 Processor -- start 2024-03-26T17:50:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.580+0000 7f80fc40f740 1 -- start start 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dba2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 0x55a16dbaa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80fc40f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41a8df0 con 0x55a16dbac420 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80fc40f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41aa930 con 0x55a16dbb1590 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80fc40f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41a94c0 con 0x55a16dbaba30 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38298/0 (socket says 172.21.15.105:38298) 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 -- 172.21.15.105:0/2093586939 learned_addr learned my addr 172.21.15.105:0/2093586939 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f37fe640 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 0x55a16dbaa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dba2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 -- 172.21.15.105:0/2093586939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 msgr2=0x55a16dba2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dba2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 -- 172.21.15.105:0/2093586939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 msgr2=0x55a16dbaa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 0x55a16dbaa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 -- 172.21.15.105:0/2093586939 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16dba2820 con 0x55a16dbac420 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80f27fc640 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbd180 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f80e80096a0 tx=0x7f80e80096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf8b403240546aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80d8ff9640 1 -- 172.21.15.105:0/2093586939 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8004090 con 0x55a16dbac420 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80d8ff9640 1 -- 172.21.15.105:0/2093586939 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80e8003aa0 con 0x55a16dbac420 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.581+0000 7f80d8ff9640 1 -- 172.21.15.105:0/2093586939 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e801c4c0 con 0x55a16dbac420 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 -- 172.21.15.105:0/2093586939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 msgr2=0x55a16dbbd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbd180 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f80e80096a0 tx=0x7f80e80096d0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 -- 172.21.15.105:0/2093586939 shutdown_connections 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbb1590 0x55a16dbaa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbd180 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 --2- 172.21.15.105:0/2093586939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbaba30 0x55a16dba2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 -- 172.21.15.105:0/2093586939 >> 172.21.15.105:0/2093586939 conn(0x55a16dba6fa0 msgr2=0x55a16dbaa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 -- 172.21.15.105:0/2093586939 shutdown_connections 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.582+0000 7f80fc40f740 1 -- 172.21.15.105:0/2093586939 wait complete. 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 1 -- start start 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 1 Processor -- start 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbaba30 0x55a16dbbaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 0x55a16dbbc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41aa260 con 0x55a16dbac420 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41a9b90 con 0x55a16dbaba30 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80f41a6dd0 con 0x55a16dbb1590 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80fc40f740 10 client.0 _ll_register_callbacks cb 0x7f80e8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80f2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbaba30 0x55a16dbbaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80f2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbaba30 0x55a16dbbaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46862/0 (socket says 172.21.15.105:46862) 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80f2ffd640 1 -- 172.21.15.105:0/161485903 learned_addr learned my addr 172.21.15.105:0/161485903 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80f37fe640 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 0x55a16dbbc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.583+0000 7f80f27fc640 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80f2ffd640 1 -- 172.21.15.105:0/161485903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 msgr2=0x55a16dbbc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 0x55a16dbbc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80f2ffd640 1 -- 172.21.15.105:0/161485903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 msgr2=0x55a16dbbb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80f2ffd640 1 -- 172.21.15.105:0/161485903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a16dbbca50 con 0x55a16dbaba30 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80f27fc640 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80f2ffd640 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbaba30 0x55a16dbbaf10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f80e40ba670 tx=0x7f80e40f1900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c307482d1a0d3c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80d6ff5640 1 -- 172.21.15.105:0/161485903 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e40bd830 con 0x55a16dbaba30 2024-03-26T17:50:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80d6ff5640 1 -- 172.21.15.105:0/161485903 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f80e40f7d90 con 0x55a16dbaba30 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80fc40f740 1 -- 172.21.15.105:0/161485903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e8003aa0 con 0x55a16dbaba30 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80d6ff5640 1 -- 172.21.15.105:0/161485903 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80e40bdac0 con 0x55a16dbaba30 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80fc40f740 10 client.29786 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80fc40f740 1 -- 172.21.15.105:0/161485903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a16dc28740 con 0x55a16dbaba30 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.584+0000 7f80d6ff5640 1 -- 172.21.15.105:0/161485903 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80e8003aa0 con 0x55a16dbaba30 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80d6ff5640 1 -- 172.21.15.105:0/161485903 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f80e40054b0 con 0x55a16dbaba30 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80fc40f740 10 client.29786 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80fc40f740 10 client.29786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80fc40f740 20 client.29786 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80fc40f740 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a16dc28d30 0x55a16dc2b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80fc40f740 4 client.29786 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80fc40f740 1 -- 172.21.15.105:0/161485903 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a16db18200 con 0x55a16dc28d30 2024-03-26T17:50:33.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80f27fc640 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a16dc28d30 0x55a16dc2b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80d6ff5640 10 client.29786 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.585+0000 7f80f27fc640 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a16dc28d30 0x55a16dc2b150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c231205945c43c68 server_cookie=9a4d2a4d16f64cb9 in_seq=0 out_seq=0 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80d6ff5640 1 -- 172.21.15.105:0/161485903 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a16db18200 con 0x55a16dc28d30 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80d6ff5640 10 client.29786 handle_command_reply: tid=0 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 client.29786 shutdown 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 -- 172.21.15.105:0/161485903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbaba30 msgr2=0x55a16dbbaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbaba30 0x55a16dbbaf10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f80e40ba670 tx=0x7f80e40f1900 comp rx=0 tx=0).stop 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 -- 172.21.15.105:0/161485903 shutdown_connections 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55a16dc28d30 0x55a16dc2b150 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a16dbb1590 0x55a16dbbc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80d6ff5640 0 client.29786 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a16dbac420 0x55a16dbbb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 --2- 172.21.15.105:0/161485903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a16dbaba30 0x55a16dbbaf10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 -- 172.21.15.105:0/161485903 >> 172.21.15.105:0/161485903 conn(0x55a16dba6fa0 msgr2=0x55a16dbaaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 -- 172.21.15.105:0/161485903 shutdown_connections 2024-03-26T17:50:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 1 -- 172.21.15.105:0/161485903 wait complete. 2024-03-26T17:50:33.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80fc40f740 20 client.29786 trim_cache size 0 max 16384 2024-03-26T17:50:33.584 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 -- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 msgr2=0x7f80dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55a16dbacc80 tx=0x7f80ec009500 comp rx=0 tx=0).stop 2024-03-26T17:50:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 -- 172.21.15.105:0/3670987435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102340 msgr2=0x7f80f4197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102340 0x7f80f4197b80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f80e4006010 tx=0x7f80e4005e40 comp rx=0 tx=0).stop 2024-03-26T17:50:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 -- 172.21.15.105:0/3670987435 shutdown_connections 2024-03-26T17:50:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80dc0a2f20 0x7f80dc0a53e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80f41075a0 0x7f80f419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f80f4104c70 0x7f80f41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 --2- 172.21.15.105:0/3670987435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80f4102340 0x7f80f4197b80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 -- 172.21.15.105:0/3670987435 >> 172.21.15.105:0/3670987435 conn(0x7f80f40fbd50 msgr2=0x7f80f41041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 -- 172.21.15.105:0/3670987435 shutdown_connections 2024-03-26T17:50:33.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.586+0000 7f80f9752640 1 -- 172.21.15.105:0/3670987435 wait complete. 2024-03-26T17:50:33.588 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:50:33.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.717+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/160370927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 msgr2=0x7fa1a4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.717+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/160370927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 0x7fa1a4100be0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009900 tx=0x7fa1a002bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/160370927 shutdown_connections 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/160370927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a4102370 0x7fa1a4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/160370927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1a41019b0 0x7fa1a4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/160370927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 0x7fa1a4100be0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/160370927 >> 172.21.15.105:0/160370927 conn(0x7fa1a40fbd50 msgr2=0x7fa1a40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/160370927 shutdown_connections 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/160370927 wait complete. 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 Processor -- start 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 -- start start 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 0x7fa1a40695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.718+0000 7fa1ac3b9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a41019b0 0x7fa1a419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1ac3b9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1a4102370 0x7fa1a419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1ac3b9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410c2c0 con 0x7fa1a41007c0 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1ac3b9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410c140 con 0x7fa1a41019b0 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1ac3b9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410c440 con 0x7fa1a4102370 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 0x7fa1a40695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 0x7fa1a40695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38310/0 (socket says 172.21.15.105:38310) 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 -- 172.21.15.105:0/883359422 learned_addr learned my addr 172.21.15.105:0/883359422 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa956640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1a4102370 0x7fa1a419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1a9954640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a41019b0 0x7fa1a419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 -- 172.21.15.105:0/883359422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1a4102370 msgr2=0x7fa1a419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1a4102370 0x7fa1a419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 -- 172.21.15.105:0/883359422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a41019b0 msgr2=0x7fa1a419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a41019b0 0x7fa1a419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 -- 172.21.15.105:0/883359422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a419f760 con 0x7fa1a41007c0 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1aa155640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 0x7fa1a40695b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0000c00 tx=0x7fa1a0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa19b7fe640 1 -- 172.21.15.105:0/883359422 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0043020 con 0x7fa1a41007c0 2024-03-26T17:50:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/883359422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a419f9f0 con 0x7fa1a41007c0 2024-03-26T17:50:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa19b7fe640 1 -- 172.21.15.105:0/883359422 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1a0004350 con 0x7fa1a41007c0 2024-03-26T17:50:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa19b7fe640 1 -- 172.21.15.105:0/883359422 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0003b80 con 0x7fa1a41007c0 2024-03-26T17:50:33.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.719+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/883359422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a419ff90 con 0x7fa1a41007c0 2024-03-26T17:50:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.722+0000 7fa19b7fe640 1 -- 172.21.15.105:0/883359422 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa1a0046020 con 0x7fa1a41007c0 2024-03-26T17:50:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.722+0000 7fa19b7fe640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa1900a2f20 0x7fa1900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.722+0000 7fa1a9954640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa1900a2f20 0x7fa1900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.722+0000 7fa19b7fe640 1 -- 172.21.15.105:0/883359422 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa1a00f1580 con 0x7fa1a41007c0 2024-03-26T17:50:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.723+0000 7fa1a9954640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa1900a2f20 0x7fa1900a53e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4069320 tx=0x7fa194005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.723+0000 7fa1aefa1740 1 Processor -- start 2024-03-26T17:50:33.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.723+0000 7fa1aefa1740 1 -- start start 2024-03-26T17:50:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.723+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.723+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.723+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.723+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5610c352b970 con 0x5610c35cb830 2024-03-26T17:50:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.723+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610c34d3d70 con 0x5610c35d09a0 2024-03-26T17:50:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610c3527920 con 0x5610c35cae40 2024-03-26T17:50:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1a9954640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47124/0 (socket says 172.21.15.105:47124) 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1a9954640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38314/0 (socket says 172.21.15.105:38314) 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 -- 172.21.15.105:0/406674276 learned_addr learned my addr 172.21.15.105:0/406674276 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa956640 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 -- 172.21.15.105:0/406674276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 msgr2=0x5610c35d2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35d2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 -- 172.21.15.105:0/406674276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 msgr2=0x5610c35d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 -- 172.21.15.105:0/406674276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c323f3f0 con 0x5610c35cae40 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa956640 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35d2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1a9954640 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa1aa155640 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35cb260 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00bdc00 tx=0x7fa1a00bd7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d962f4f779f8e584 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa198bfe640 1 -- 172.21.15.105:0/406674276 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00bdc70 con 0x5610c35cae40 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa198bfe640 1 -- 172.21.15.105:0/406674276 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1a00c1dc0 con 0x5610c35cae40 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.724+0000 7fa198bfe640 1 -- 172.21.15.105:0/406674276 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00ba2e0 con 0x5610c35cae40 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 -- 172.21.15.105:0/406674276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 msgr2=0x5610c35cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35cb260 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00bdc00 tx=0x7fa1a00bd7b0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 -- 172.21.15.105:0/406674276 shutdown_connections 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35d2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/406674276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35cb260 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 -- 172.21.15.105:0/406674276 >> 172.21.15.105:0/406674276 conn(0x5610c35c1fa0 msgr2=0x5610c35c23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 -- 172.21.15.105:0/406674276 shutdown_connections 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.725+0000 7fa1aefa1740 1 -- 172.21.15.105:0/406674276 wait complete. 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 1 -- start start 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 1 Processor -- start 2024-03-26T17:50:33.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cae40 0x5610c35d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cb830 0x5610c35d5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5610c35d3710 con 0x5610c35cae40 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610c34e7310 con 0x5610c35d09a0 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610c35d3890 con 0x5610c35cb830 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa155640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cae40 0x5610c35d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aefa1740 10 client.0 _ll_register_callbacks cb 0x5610c3522c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa155640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cae40 0x5610c35d5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38322/0 (socket says 172.21.15.105:38322) 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa155640 1 -- 172.21.15.105:0/1743284139 learned_addr learned my addr 172.21.15.105:0/1743284139 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1a9954640 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cb830 0x5610c35d5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa956640 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa155640 1 -- 172.21.15.105:0/1743284139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cb830 msgr2=0x5610c35d5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa155640 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cb830 0x5610c35d5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa155640 1 -- 172.21.15.105:0/1743284139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 msgr2=0x5610c35dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa155640 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35dc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.726+0000 7fa1aa155640 1 -- 172.21.15.105:0/1743284139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c33097d0 con 0x5610c35cae40 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aa155640 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cae40 0x5610c35d5000 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00092a0 tx=0x7fa1a0003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2572b40e054bdfc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa18dffb640 1 -- 172.21.15.105:0/1743284139 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00bdc70 con 0x5610c35cae40 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aefa1740 1 -- 172.21.15.105:0/1743284139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a00c1dc0 con 0x5610c35cae40 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa18dffb640 1 -- 172.21.15.105:0/1743284139 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa1a00bbf70 con 0x5610c35cae40 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa18dffb640 1 -- 172.21.15.105:0/1743284139 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa1a00a2050 con 0x5610c35cae40 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aefa1740 10 client.19998 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aefa1740 1 -- 172.21.15.105:0/1743284139 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5610c35d3710 con 0x5610c35cae40 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa18dffb640 1 -- 172.21.15.105:0/1743284139 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1a00c1dc0 con 0x5610c35cae40 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa18dffb640 1 -- 172.21.15.105:0/1743284139 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa1a009cae0 con 0x5610c35cae40 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aefa1740 10 client.19998 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aefa1740 10 client.19998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:33.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aefa1740 20 client.19998 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610c35e0e00 0x5610c35e31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.727+0000 7fa1aefa1740 4 client.19998 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.728+0000 7fa1aefa1740 1 -- 172.21.15.105:0/1743284139 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610c3538eb0 con 0x5610c35e0e00 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.728+0000 7fa1a9954640 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610c35e0e00 0x5610c35e31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.728+0000 7fa1a9954640 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610c35e0e00 0x5610c35e31f0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a21a8ddcf2478ab4 server_cookie=bf5128cdd55a06fb in_seq=0 out_seq=0 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.728+0000 7fa18dffb640 10 client.19998 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.729+0000 7fa18dffb640 1 -- 172.21.15.105:0/1743284139 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5610c3538eb0 con 0x5610c35e0e00 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.729+0000 7fa18dffb640 10 client.19998 handle_command_reply: tid=0 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.729+0000 7fa1aefa1740 1 client.19998 shutdown 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 -- 172.21.15.105:0/1743284139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cae40 msgr2=0x5610c35d5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cae40 0x5610c35d5000 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00092a0 tx=0x7fa1a0003660 comp rx=0 tx=0).stop 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 -- 172.21.15.105:0/1743284139 shutdown_connections 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610c35e0e00 0x5610c35e31f0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cb830 0x5610c35d5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa18dffb640 0 client.19998 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:33.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/1743284139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cae40 0x5610c35d5000 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 -- 172.21.15.105:0/1743284139 >> 172.21.15.105:0/1743284139 conn(0x5610c35c1fa0 msgr2=0x5610c35c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 -- 172.21.15.105:0/1743284139 shutdown_connections 2024-03-26T17:50:33.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 1 -- 172.21.15.105:0/1743284139 wait complete. 2024-03-26T17:50:33.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.730+0000 7fa1aefa1740 20 client.19998 trim_cache size 0 max 16384 2024-03-26T17:50:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aefa1740 1 Processor -- start 2024-03-26T17:50:33.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aefa1740 1 -- start start 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35d5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a6e00 con 0x5610c35cb830 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a8940 con 0x5610c35d09a0 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a74d0 con 0x5610c35cae40 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1a9954640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1a9954640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38338/0 (socket says 172.21.15.105:38338) 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1a9954640 1 -- 172.21.15.105:0/2705335741 learned_addr learned my addr 172.21.15.105:0/2705335741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aa155640 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.744+0000 7fa1aa956640 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa1a9954640 1 -- 172.21.15.105:0/2705335741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 msgr2=0x5610c35d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa1a9954640 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35d5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa1a9954640 1 -- 172.21.15.105:0/2705335741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 msgr2=0x5610c35e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa1a9954640 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa1a9954640 1 -- 172.21.15.105:0/2705335741 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c35c2d10 con 0x5610c35cb830 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa1aa155640 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa1a9954640 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc510 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fa194009a90 tx=0x7fa194014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65181b808f4031a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa18ffff640 1 -- 172.21.15.105:0/2705335741 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194032070 con 0x5610c35cb830 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa18ffff640 1 -- 172.21.15.105:0/2705335741 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa19402c610 con 0x5610c35cb830 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.745+0000 7fa18ffff640 1 -- 172.21.15.105:0/2705335741 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19401dbd0 con 0x5610c35cb830 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2705335741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 msgr2=0x5610c35dc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc510 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fa194009a90 tx=0x7fa194014000 comp rx=0 tx=0).stop 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2705335741 shutdown_connections 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35d09a0 0x5610c35e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc510 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2705335741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35cae40 0x5610c35d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2705335741 >> 172.21.15.105:0/2705335741 conn(0x5610c35c1fa0 msgr2=0x5610c35c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2705335741 shutdown_connections 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2705335741 wait complete. 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- start start 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 Processor -- start 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35cae40 0x5610c35dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35d09a0 0x5610c35da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a8270 con 0x5610c35cb830 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a7ba0 con 0x5610c35cae40 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41a4d80 con 0x5610c35d09a0 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aefa1740 10 client.0 _ll_register_callbacks cb 0x7fa1940099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:33.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1a9954640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aa155640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35cae40 0x5610c35dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aa155640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35cae40 0x5610c35dbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46914/0 (socket says 172.21.15.105:46914) 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.746+0000 7fa1aa155640 1 -- 172.21.15.105:0/2055631474 learned_addr learned my addr 172.21.15.105:0/2055631474 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aa956640 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35d09a0 0x5610c35da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aa155640 1 -- 172.21.15.105:0/2055631474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35d09a0 msgr2=0x5610c35da5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aa155640 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35d09a0 0x5610c35da5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aa155640 1 -- 172.21.15.105:0/2055631474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 msgr2=0x5610c35dc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aa155640 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aa155640 1 -- 172.21.15.105:0/2055631474 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c35dade0 con 0x5610c35cae40 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aa155640 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35cae40 0x5610c35dbfa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00ef3b0 tx=0x7fa1a0003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3daa562ffd583c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa18dffb640 1 -- 172.21.15.105:0/2055631474 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00ca030 con 0x5610c35cae40 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa18dffb640 1 -- 172.21.15.105:0/2055631474 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa1a00bbf70 con 0x5610c35cae40 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2055631474 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19402c610 con 0x5610c35cae40 2024-03-26T17:50:33.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa18dffb640 1 -- 172.21.15.105:0/2055631474 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa1a00a2050 con 0x5610c35cae40 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aefa1740 10 client.29804 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.747+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2055631474 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5610c35dade0 con 0x5610c35cae40 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa18dffb640 1 -- 172.21.15.105:0/2055631474 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa19402c610 con 0x5610c35cae40 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa18dffb640 1 -- 172.21.15.105:0/2055631474 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa1a00c61f0 con 0x5610c35cae40 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa1aefa1740 10 client.29804 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa1aefa1740 10 client.29804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa1aefa1740 20 client.29804 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610c3648230 0x5610c364a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa1aefa1740 4 client.29804 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2055631474 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610c3533200 con 0x5610c3648230 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa1a9954640 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610c3648230 0x5610c364a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa18dffb640 10 client.29804 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.748+0000 7fa1a9954640 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610c3648230 0x5610c364a650 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa02c0ccad7874d8 server_cookie=71e618ff2b4f73c9 in_seq=0 out_seq=0 2024-03-26T17:50:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa18dffb640 1 -- 172.21.15.105:0/2055631474 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610c3533200 con 0x5610c3648230 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa18dffb640 10 client.29804 handle_command_reply: tid=0 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 client.29804 shutdown 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2055631474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35cae40 msgr2=0x5610c35dbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35cae40 0x5610c35dbfa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00ef3b0 tx=0x7fa1a0003d20 comp rx=0 tx=0).stop 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2055631474 shutdown_connections 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610c3648230 0x5610c364a650 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610c35d09a0 0x5610c35da5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610c35cb830 0x5610c35dc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 --2- 172.21.15.105:0/2055631474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610c35cae40 0x5610c35dbfa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa18dffb640 0 client.29804 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2055631474 >> 172.21.15.105:0/2055631474 conn(0x5610c35c1fa0 msgr2=0x5610c35c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2055631474 shutdown_connections 2024-03-26T17:50:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 1 -- 172.21.15.105:0/2055631474 wait complete. 2024-03-26T17:50:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.750+0000 7fa1aefa1740 20 client.29804 trim_cache size 0 max 16384 2024-03-26T17:50:33.748 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:50:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa1900a2f20 msgr2=0x7fa1900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa1900a2f20 0x7fa1900a53e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4069320 tx=0x7fa194005f90 comp rx=0 tx=0).stop 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 msgr2=0x7fa1a40695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 0x7fa1a40695b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0000c00 tx=0x7fa1a0005dc0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/883359422 shutdown_connections 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa1900a2f20 0x7fa1900a53e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1a4102370 0x7fa1a419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a41019b0 0x7fa1a419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 --2- 172.21.15.105:0/883359422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1a41007c0 0x7fa1a40695b0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/883359422 >> 172.21.15.105:0/883359422 conn(0x7fa1a40fbd50 msgr2=0x7fa1a4104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/883359422 shutdown_connections 2024-03-26T17:50:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:33.751+0000 7fa1ac3b9640 1 -- 172.21.15.105:0/883359422 wait complete. 2024-03-26T17:50:33.840 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:33 smithi105 ceph-mon[32280]: pgmap v715: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-03-26T17:50:33.840 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:33 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2625701026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:50:34.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 396388 3.14 MB/sec execute 557 sec latency 17.662 ms 2024-03-26T17:50:34.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:33 smithi144 ceph-mon[36095]: pgmap v715: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-03-26T17:50:34.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:33 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2625701026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:50:34.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:33 smithi135 ceph-mon[35636]: pgmap v715: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-03-26T17:50:34.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:33 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2625701026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:50:34.754 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:50:34.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:50:34.754 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:50:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 -- 172.21.15.105:0/3911103300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e0107620 msgr2=0x7f42e0109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 --2- 172.21.15.105:0/3911103300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e0107620 0x7f42e0109a10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009be0 tx=0x7f42d802bef0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 -- 172.21.15.105:0/3911103300 shutdown_connections 2024-03-26T17:50:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 --2- 172.21.15.105:0/3911103300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e0107620 0x7f42e0109a10 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 --2- 172.21.15.105:0/3911103300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e0104cf0 0x7f42e01070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 --2- 172.21.15.105:0/3911103300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e01023c0 0x7f42e01047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 -- 172.21.15.105:0/3911103300 >> 172.21.15.105:0/3911103300 conn(0x7f42e00fbd50 msgr2=0x7f42e00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 -- 172.21.15.105:0/3911103300 shutdown_connections 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 -- 172.21.15.105:0/3911103300 wait complete. 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 Processor -- start 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.879+0000 7f42e6c36640 1 -- start start 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e6c36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e01023c0 0x7f42e0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e6c36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e0104cf0 0x7f42e0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e6c36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e0107620 0x7f42e019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e6c36640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010d3f0 con 0x7f42e01023c0 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e6c36640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010d270 con 0x7f42e0104cf0 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e6c36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f42e010d570 con 0x7f42e0107620 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e01023c0 0x7f42e0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e51d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e0107620 0x7f42e019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42dffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e0104cf0 0x7f42e0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e01023c0 0x7f42e0197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40298/0 (socket says 172.21.15.105:40298) 2024-03-26T17:50:34.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e51d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e0107620 0x7f42e019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37806/0 (socket says 172.21.15.105:37806) 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 -- 172.21.15.105:0/2388294757 learned_addr learned my addr 172.21.15.105:0/2388294757 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 -- 172.21.15.105:0/2388294757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e0107620 msgr2=0x7f42e019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e0107620 0x7f42e019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 -- 172.21.15.105:0/2388294757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e0104cf0 msgr2=0x7f42e0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e0104cf0 0x7f42e0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 -- 172.21.15.105:0/2388294757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e019f740 con 0x7f42e01023c0 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42dffff640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e0104cf0 0x7f42e0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e51d3640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e0107620 0x7f42e019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42e49d2640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e01023c0 0x7f42e0197b20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f42d400e9f0 tx=0x7f42d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42ddffb640 1 -- 172.21.15.105:0/2388294757 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400ccc0 con 0x7f42e01023c0 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.881+0000 7f42e6c36640 1 -- 172.21.15.105:0/2388294757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e019fa30 con 0x7f42e01023c0 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.880+0000 7f42ddffb640 1 -- 172.21.15.105:0/2388294757 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42d40108e0 con 0x7f42e01023c0 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.881+0000 7f42ddffb640 1 -- 172.21.15.105:0/2388294757 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d40187c0 con 0x7f42e01023c0 2024-03-26T17:50:34.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.881+0000 7f42e6c36640 1 -- 172.21.15.105:0/2388294757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e019ffd0 con 0x7f42e01023c0 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.883+0000 7f42e9823740 1 Processor -- start 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.883+0000 7f42e9823740 1 -- start start 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e9823740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa205335b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e9823740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa20538680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e9823740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa2053b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e9823740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa20498630 con 0x55fa20533190 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e9823740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fa20440e10 con 0x55fa20538cf0 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e9823740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa204945e0 con 0x55fa20533b80 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42ddffb640 1 -- 172.21.15.105:0/2388294757 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f42d4010a90 con 0x7f42e01023c0 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e51d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa205335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e51d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa205335b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40304/0 (socket says 172.21.15.105:40304) 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e51d3640 1 -- 172.21.15.105:0/1509504267 learned_addr learned my addr 172.21.15.105:0/1509504267 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42dffff640 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa2053b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e49d2640 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa20538680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42dffff640 1 -- 172.21.15.105:0/1509504267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 msgr2=0x55fa20538680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42dffff640 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa20538680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42dffff640 1 -- 172.21.15.105:0/1509504267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 msgr2=0x55fa205335b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42dffff640 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa205335b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42dffff640 1 -- 172.21.15.105:0/1509504267 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa20457740 con 0x55fa20538cf0 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42e51d3640 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa205335b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.884+0000 7f42ddffb640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42d00a2fa0 0x7f42d00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.885+0000 7f42dffff640 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa2053b180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f42e0069500 tx=0x7f42cc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c44347272909f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.885+0000 7f42e51d3640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42d00a2fa0 0x7f42d00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.885+0000 7f42ddffb640 1 -- 172.21.15.105:0/2388294757 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f42d40cf7f0 con 0x7f42e01023c0 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.885+0000 7f42caffd640 1 -- 172.21.15.105:0/1509504267 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc004030 con 0x55fa20538cf0 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.885+0000 7f42caffd640 1 -- 172.21.15.105:0/1509504267 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42cc0041d0 con 0x55fa20538cf0 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.885+0000 7f42caffd640 1 -- 172.21.15.105:0/1509504267 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00f800 con 0x55fa20538cf0 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e51d3640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42d00a2fa0 0x7f42d00a5460 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55fa205343e0 tx=0x7f42d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 -- 172.21.15.105:0/1509504267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 msgr2=0x55fa2053b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa2053b180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f42e0069500 tx=0x7f42cc0118d0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 -- 172.21.15.105:0/1509504267 shutdown_connections 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa2053b180 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa20538680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 --2- 172.21.15.105:0/1509504267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa205335b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 -- 172.21.15.105:0/1509504267 >> 172.21.15.105:0/1509504267 conn(0x55fa2052e700 msgr2=0x55fa2052eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 -- 172.21.15.105:0/1509504267 shutdown_connections 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 -- 172.21.15.105:0/1509504267 wait complete. 2024-03-26T17:50:34.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 -- start start 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.886+0000 7f42e9823740 1 Processor -- start 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e9823740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa2053d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e9823740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa2053dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e9823740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa20544c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e9823740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa2053bca0 con 0x55fa20533190 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e9823740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fa204541a0 con 0x55fa20538cf0 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e9823740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa2053be20 con 0x55fa20533b80 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e49d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa2053d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e9823740 10 client.0 _ll_register_callbacks cb 0x55fa201abc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e49d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa2053d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40312/0 (socket says 172.21.15.105:40312) 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42dffff640 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa2053dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e49d2640 1 -- 172.21.15.105:0/3139837604 learned_addr learned my addr 172.21.15.105:0/3139837604 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e51d3640 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa20544c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42dffff640 1 -- 172.21.15.105:0/3139837604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 msgr2=0x55fa20544c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42dffff640 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa20544c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42dffff640 1 -- 172.21.15.105:0/3139837604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 msgr2=0x55fa2053d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42dffff640 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa2053d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42dffff640 1 -- 172.21.15.105:0/3139837604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa202747e0 con 0x55fa20533b80 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.887+0000 7f42e49d2640 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa2053d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42dffff640 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa2053dc10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55fa2052f670 tx=0x7f42cc009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60776c12fed4a1b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42bffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42bffff640 1 -- 172.21.15.105:0/3139837604 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc028020 con 0x55fa20533b80 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42bffff640 1 -- 172.21.15.105:0/3139837604 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f42cc016040 con 0x55fa20533b80 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 1 -- 172.21.15.105:0/3139837604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42cc0041d0 con 0x55fa20533b80 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42bffff640 1 -- 172.21.15.105:0/3139837604 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f42cc033050 con 0x55fa20533b80 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 10 client.29803 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 1 -- 172.21.15.105:0/3139837604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa202d4f70 con 0x55fa20533b80 2024-03-26T17:50:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42bffff640 1 -- 172.21.15.105:0/3139837604 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42cc0041d0 con 0x55fa20533b80 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42bffff640 1 -- 172.21.15.105:0/3139837604 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f42cc004a90 con 0x55fa20533b80 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 10 client.29803 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 10 client.29803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 20 client.29803 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa20549340 0x55fa2054b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 4 client.29803 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.888+0000 7f42e9823740 1 -- 172.21.15.105:0/3139837604 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa204a5b70 con 0x55fa20549340 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.889+0000 7f42e49d2640 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa20549340 0x55fa2054b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.889+0000 7f42bffff640 10 client.29803 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.889+0000 7f42e49d2640 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa20549340 0x55fa2054b7b0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b5167084d467fd server_cookie=eafab7c638873466 in_seq=0 out_seq=0 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42bffff640 1 -- 172.21.15.105:0/3139837604 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fa204a5b70 con 0x55fa20549340 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42bffff640 10 client.29803 handle_command_reply: tid=0 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 client.29803 shutdown 2024-03-26T17:50:34.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 -- 172.21.15.105:0/3139837604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 msgr2=0x55fa2053dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa2053dc10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55fa2052f670 tx=0x7f42cc009cb0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 -- 172.21.15.105:0/3139837604 shutdown_connections 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa20549340 0x55fa2054b7b0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20538cf0 0x55fa20544c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20533b80 0x55fa2053dc10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42bffff640 0 client.29803 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 --2- 172.21.15.105:0/3139837604 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa2053d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 -- 172.21.15.105:0/3139837604 >> 172.21.15.105:0/3139837604 conn(0x55fa2052e700 msgr2=0x55fa2052d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 -- 172.21.15.105:0/3139837604 shutdown_connections 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 1 -- 172.21.15.105:0/3139837604 wait complete. 2024-03-26T17:50:34.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.890+0000 7f42e9823740 20 client.29803 trim_cache size 0 max 16384 2024-03-26T17:50:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.913+0000 7f42e9823740 1 Processor -- start 2024-03-26T17:50:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.913+0000 7f42e9823740 1 -- start start 2024-03-26T17:50:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e9823740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533190 0x55fa2052d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e9823740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533b80 0x55fa20544c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e9823740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 0x55fa2052d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e9823740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01a8d90 con 0x55fa20533b80 2024-03-26T17:50:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e9823740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01aa8d0 con 0x55fa20533190 2024-03-26T17:50:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e9823740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01a9460 con 0x55fa20538cf0 2024-03-26T17:50:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533b80 0x55fa20544c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533b80 0x55fa20544c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40318/0 (socket says 172.21.15.105:40318) 2024-03-26T17:50:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e49d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533190 0x55fa2052d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 -- 172.21.15.105:0/2311597188 learned_addr learned my addr 172.21.15.105:0/2311597188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:34.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e51d3640 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 0x55fa2052d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 -- 172.21.15.105:0/2311597188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 msgr2=0x55fa2052d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 0x55fa2052d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 -- 172.21.15.105:0/2311597188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533190 msgr2=0x55fa2052d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533190 0x55fa2052d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 -- 172.21.15.105:0/2311597188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa2016c170 con 0x55fa20533b80 2024-03-26T17:50:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42e49d2640 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533190 0x55fa2052d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42dffff640 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533b80 0x55fa20544c10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009690 tx=0x7f42cc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130258a2b2091b9f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:34.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42ca7fc640 1 -- 172.21.15.105:0/2311597188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc028070 con 0x55fa20533b80 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42ca7fc640 1 -- 172.21.15.105:0/2311597188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42cc00faa0 con 0x55fa20533b80 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.914+0000 7f42ca7fc640 1 -- 172.21.15.105:0/2311597188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42cc004910 con 0x55fa20533b80 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 -- 172.21.15.105:0/2311597188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533b80 msgr2=0x55fa20544c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533b80 0x55fa20544c10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009690 tx=0x7f42cc005e50 comp rx=0 tx=0).stop 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 -- 172.21.15.105:0/2311597188 shutdown_connections 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 0x55fa2052d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533b80 0x55fa20544c10 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 --2- 172.21.15.105:0/2311597188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533190 0x55fa2052d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 -- 172.21.15.105:0/2311597188 >> 172.21.15.105:0/2311597188 conn(0x55fa2052e700 msgr2=0x55fa2053d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 -- 172.21.15.105:0/2311597188 shutdown_connections 2024-03-26T17:50:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.915+0000 7f42e9823740 1 -- 172.21.15.105:0/2311597188 wait complete. 2024-03-26T17:50:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 1 -- start start 2024-03-26T17:50:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 1 Processor -- start 2024-03-26T17:50:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa20542420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533b80 0x55fa20542960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 0x55fa20543900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01aa200 con 0x55fa20533190 2024-03-26T17:50:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01a9b30 con 0x55fa20533b80 2024-03-26T17:50:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f42e01a6d70 con 0x55fa20538cf0 2024-03-26T17:50:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e9823740 10 client.0 _ll_register_callbacks cb 0x7f42cc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e49d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa20542420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e49d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa20542420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40326/0 (socket says 172.21.15.105:40326) 2024-03-26T17:50:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e49d2640 1 -- 172.21.15.105:0/1481319122 learned_addr learned my addr 172.21.15.105:0/1481319122 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:34.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42dffff640 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533b80 0x55fa20542960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.916+0000 7f42e51d3640 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 0x55fa20543900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e49d2640 1 -- 172.21.15.105:0/1481319122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 msgr2=0x55fa20543900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e49d2640 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 0x55fa20543900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e49d2640 1 -- 172.21.15.105:0/1481319122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533b80 msgr2=0x55fa20542960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e49d2640 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533b80 0x55fa20542960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e49d2640 1 -- 172.21.15.105:0/1481319122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa20543ef0 con 0x55fa20533190 2024-03-26T17:50:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e49d2640 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa20542420 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f42d40982e0 tx=0x7f42d40097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75c6b5dcbaaec72 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:34.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42bffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42bffff640 1 -- 172.21.15.105:0/1481319122 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4010e00 con 0x55fa20533190 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42bffff640 1 -- 172.21.15.105:0/1481319122 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f42d40a4400 con 0x55fa20533190 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e9823740 1 -- 172.21.15.105:0/1481319122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42cc00faa0 con 0x55fa20533190 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42bffff640 1 -- 172.21.15.105:0/1481319122 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f42d409c580 con 0x55fa20533190 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e9823740 10 client.20022 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42e9823740 1 -- 172.21.15.105:0/1481319122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa205af7c0 con 0x55fa20533190 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42bffff640 1 -- 172.21.15.105:0/1481319122 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42cc00faa0 con 0x55fa20533190 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.917+0000 7f42bffff640 1 -- 172.21.15.105:0/1481319122 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f42d409b360 con 0x55fa20533190 2024-03-26T17:50:34.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42e9823740 10 client.20022 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42e9823740 10 client.20022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42e9823740 20 client.20022 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42e9823740 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa205b0190 0x55fa205b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42e9823740 4 client.20022 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42e9823740 1 -- 172.21.15.105:0/1481319122 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa2049fec0 con 0x55fa205b0190 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42dffff640 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa205b0190 0x55fa205b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42dffff640 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa205b0190 0x55fa205b25b0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a2843f18a6a9191 server_cookie=5d7c45719d78dcb in_seq=0 out_seq=0 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.918+0000 7f42bffff640 10 client.20022 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:34.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42bffff640 1 -- 172.21.15.105:0/1481319122 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa2049fec0 con 0x55fa205b0190 2024-03-26T17:50:34.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42bffff640 10 client.20022 handle_command_reply: tid=0 2024-03-26T17:50:34.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 client.20022 shutdown 2024-03-26T17:50:34.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 -- 172.21.15.105:0/1481319122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 msgr2=0x55fa20542420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa20542420 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f42d40982e0 tx=0x7f42d40097a0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 -- 172.21.15.105:0/1481319122 shutdown_connections 2024-03-26T17:50:34.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fa205b0190 0x55fa205b25b0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa20538cf0 0x55fa20543900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa20533b80 0x55fa20542960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 --2- 172.21.15.105:0/1481319122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa20533190 0x55fa20542420 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.919+0000 7f42e9823740 1 -- 172.21.15.105:0/1481319122 >> 172.21.15.105:0/1481319122 conn(0x55fa2052e700 msgr2=0x55fa20531960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:34.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.920+0000 7f42bffff640 0 client.20022 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:34.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.920+0000 7f42e9823740 1 -- 172.21.15.105:0/1481319122 shutdown_connections 2024-03-26T17:50:34.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.920+0000 7f42e9823740 1 -- 172.21.15.105:0/1481319122 wait complete. 2024-03-26T17:50:34.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.920+0000 7f42e9823740 20 client.20022 trim_cache size 0 max 16384 2024-03-26T17:50:34.923 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:50:34.923 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:50:34.923 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "c1cb8e36-97a1-4570-b1b9-5b569ef07041", 2024-03-26T17:50:34.923 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:50:34.923 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:50:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.920+0000 7f42e6c36640 1 -- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42d00a2fa0 msgr2=0x7f42d00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.920+0000 7f42e6c36640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42d00a2fa0 0x7f42d00a5460 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55fa205343e0 tx=0x7f42d8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.920+0000 7f42e6c36640 1 -- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e01023c0 msgr2=0x7f42e0197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.920+0000 7f42e6c36640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e01023c0 0x7f42e0197b20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f42d400e9f0 tx=0x7f42d400edf0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.921+0000 7f42e6c36640 1 -- 172.21.15.105:0/2388294757 shutdown_connections 2024-03-26T17:50:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.921+0000 7f42e6c36640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42d00a2fa0 0x7f42d00a5460 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.921+0000 7f42e6c36640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42e0107620 0x7f42e019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.921+0000 7f42e6c36640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42e0104cf0 0x7f42e0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.921+0000 7f42e6c36640 1 --2- 172.21.15.105:0/2388294757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42e01023c0 0x7f42e0197b20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:34.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.921+0000 7f42e6c36640 1 -- 172.21.15.105:0/2388294757 >> 172.21.15.105:0/2388294757 conn(0x7f42e00fbd50 msgr2=0x7f42e0102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:34.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.921+0000 7f42e6c36640 1 -- 172.21.15.105:0/2388294757 shutdown_connections 2024-03-26T17:50:34.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:34.921+0000 7f42e6c36640 1 -- 172.21.15.105:0/2388294757 wait complete. 2024-03-26T17:50:34.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:50:34.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:50:35.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 397091 3.14 MB/sec execute 558 sec latency 27.538 ms 2024-03-26T17:50:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.049+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/717641026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58104c70 msgr2=0x7f3b58107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.049+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/717641026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58104c70 0x7f3b58107060 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009900 tx=0x7f3b4402bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/717641026 shutdown_connections 2024-03-26T17:50:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/717641026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b581075a0 0x7f3b581099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/717641026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58104c70 0x7f3b58107060 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/717641026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b58102340 0x7f3b58104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/717641026 >> 172.21.15.105:0/717641026 conn(0x7f3b580fbd50 msgr2=0x7f3b580fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:35.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/717641026 shutdown_connections 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/717641026 wait complete. 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 Processor -- start 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 -- start start 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58102340 0x7f3b58197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b58104c70 0x7f3b581980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b581075a0 0x7f3b5819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5810d400 con 0x7f3b58102340 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5810d280 con 0x7f3b581075a0 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.050+0000 7f3b5ebcd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5810d580 con 0x7f3b58104c70 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58102340 0x7f3b58197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58102340 0x7f3b58197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40344/0 (socket says 172.21.15.105:40344) 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 -- 172.21.15.105:0/4113502564 learned_addr learned my addr 172.21.15.105:0/4113502564 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5d16a640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b581075a0 0x7f3b5819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b57fff640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b58104c70 0x7f3b581980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 -- 172.21.15.105:0/4113502564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b58104c70 msgr2=0x7f3b581980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b58104c70 0x7f3b581980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 -- 172.21.15.105:0/4113502564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b581075a0 msgr2=0x7f3b5819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b581075a0 0x7f3b5819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 -- 172.21.15.105:0/4113502564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5819f770 con 0x7f3b58102340 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5c969640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58102340 0x7f3b58197ba0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00e9f0 tx=0x7f3b4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b55ffb640 1 -- 172.21.15.105:0/4113502564 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00cd80 con 0x7f3b58102340 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b55ffb640 1 -- 172.21.15.105:0/4113502564 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b4c0054d0 con 0x7f3b58102340 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/4113502564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5819fa60 con 0x7f3b58102340 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b55ffb640 1 -- 172.21.15.105:0/4113502564 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c010490 con 0x7f3b58102340 2024-03-26T17:50:35.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.051+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/4113502564 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b5819ff30 con 0x7f3b58102340 2024-03-26T17:50:35.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b55ffb640 1 -- 172.21.15.105:0/4113502564 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3b4c005020 con 0x7f3b58102340 2024-03-26T17:50:35.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b617b0740 1 Processor -- start 2024-03-26T17:50:35.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b617b0740 1 -- start start 2024-03-26T17:50:35.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a949d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a94ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a9518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b617b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e5a8ae980 con 0x561e5a94f450 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b617b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e5a856be0 con 0x561e5a9498f0 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b55ffb640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b480a2f20 0x7f3b480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b617b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e5a8aa930 con 0x561e5a94a2e0 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b5d16a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a949d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b57fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a9518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b57fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a9518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40352/0 (socket says 172.21.15.105:40352) 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b5c969640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a94ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b57fff640 1 -- 172.21.15.105:0/1272184460 learned_addr learned my addr 172.21.15.105:0/1272184460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b5d16a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a949d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36660/0 (socket says 172.21.15.105:36660) 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b5d16a640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b480a2f20 0x7f3b480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.054+0000 7f3b55ffb640 1 -- 172.21.15.105:0/4113502564 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3b4c0cf790 con 0x7f3b58102340 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b57fff640 1 -- 172.21.15.105:0/1272184460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 msgr2=0x561e5a94ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b57fff640 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a94ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b57fff640 1 -- 172.21.15.105:0/1272184460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 msgr2=0x561e5a949d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b57fff640 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a949d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b57fff640 1 -- 172.21.15.105:0/1272184460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5a68c7d0 con 0x561e5a94f450 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b5c969640 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a94ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b5d16a640 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a949d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b57fff640 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a9518e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3b58069580 tx=0x7f3b44009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f929cd3dea1fdfa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b5d16a640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b480a2f20 0x7f3b480a53e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x561e5a94ab40 tx=0x7f3b50009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b42ffd640 1 -- 172.21.15.105:0/1272184460 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44045020 con 0x561e5a94f450 2024-03-26T17:50:35.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b42ffd640 1 -- 172.21.15.105:0/1272184460 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b44004040 con 0x561e5a94f450 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.055+0000 7f3b42ffd640 1 -- 172.21.15.105:0/1272184460 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44005540 con 0x561e5a94f450 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 -- 172.21.15.105:0/1272184460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 msgr2=0x561e5a9518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a9518e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3b58069580 tx=0x7f3b44009560 comp rx=0 tx=0).stop 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 -- 172.21.15.105:0/1272184460 shutdown_connections 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a9518e0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a94ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1272184460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a949d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 -- 172.21.15.105:0/1272184460 >> 172.21.15.105:0/1272184460 conn(0x561e5a944e60 msgr2=0x561e5a945260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 -- 172.21.15.105:0/1272184460 shutdown_connections 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 -- 172.21.15.105:0/1272184460 wait complete. 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 -- start start 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.056+0000 7f3b617b0740 1 Processor -- start 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a953a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94a2e0 0x561e5a953fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94f450 0x561e5a95b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b617b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e5a9523a0 con 0x561e5a94a2e0 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b617b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e5a86a160 con 0x561e5a9498f0 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b617b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e5a952520 con 0x561e5a94f450 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b617b0740 10 client.0 _ll_register_callbacks cb 0x561e5a8a6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b57fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94a2e0 0x561e5a953fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b5d16a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94f450 0x561e5a95b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b5c969640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a953a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b57fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94a2e0 0x561e5a953fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40356/0 (socket says 172.21.15.105:40356) 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b57fff640 1 -- 172.21.15.105:0/3174315460 learned_addr learned my addr 172.21.15.105:0/3174315460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b5c969640 1 -- 172.21.15.105:0/3174315460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94f450 msgr2=0x561e5a95b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:35.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b5c969640 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94f450 0x561e5a95b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b5c969640 1 -- 172.21.15.105:0/3174315460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94a2e0 msgr2=0x561e5a953fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b5c969640 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94a2e0 0x561e5a953fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b5c969640 1 -- 172.21.15.105:0/3174315460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5a5c23f0 con 0x561e5a9498f0 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b57fff640 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94a2e0 0x561e5a953fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b5c969640 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a953a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c098850 tx=0x7f3b4c01def0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d9935def506e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.057+0000 7f3b37fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b37fff640 1 -- 172.21.15.105:0/3174315460 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c005680 con 0x561e5a9498f0 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 1 -- 172.21.15.105:0/3174315460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b44004040 con 0x561e5a9498f0 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b37fff640 1 -- 172.21.15.105:0/3174315460 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3b4c0a4400 con 0x561e5a9498f0 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b37fff640 1 -- 172.21.15.105:0/3174315460 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b4c09c580 con 0x561e5a9498f0 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 10 client.29834 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 1 -- 172.21.15.105:0/3174315460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561e5a5820f0 con 0x561e5a9498f0 2024-03-26T17:50:35.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b37fff640 1 -- 172.21.15.105:0/3174315460 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b44004040 con 0x561e5a9498f0 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b37fff640 1 -- 172.21.15.105:0/3174315460 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3b4c097bd0 con 0x561e5a9498f0 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 10 client.29834 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 10 client.29834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 20 client.29834 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e5a95f810 0x561e5a961ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 4 client.29834 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.058+0000 7f3b617b0740 1 -- 172.21.15.105:0/3174315460 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e5a8bbec0 con 0x561e5a95f810 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.059+0000 7f3b57fff640 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e5a95f810 0x561e5a961ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.059+0000 7f3b57fff640 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e5a95f810 0x561e5a961ca0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6169ee0f5d722980 server_cookie=ec8b051103caeeaf in_seq=0 out_seq=0 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.059+0000 7f3b37fff640 10 client.29834 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b37fff640 1 -- 172.21.15.105:0/3174315460 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561e5a8bbec0 con 0x561e5a95f810 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b37fff640 10 client.29834 handle_command_reply: tid=0 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 client.29834 shutdown 2024-03-26T17:50:35.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 -- 172.21.15.105:0/3174315460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 msgr2=0x561e5a953a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a953a80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c098850 tx=0x7f3b4c01def0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 -- 172.21.15.105:0/3174315460 shutdown_connections 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e5a95f810 0x561e5a961ca0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94f450 0x561e5a95b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b37fff640 0 client.29834 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94a2e0 0x561e5a953fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3174315460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a9498f0 0x561e5a953a80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 -- 172.21.15.105:0/3174315460 >> 172.21.15.105:0/3174315460 conn(0x561e5a944e60 msgr2=0x561e5a944080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 -- 172.21.15.105:0/3174315460 shutdown_connections 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.060+0000 7f3b617b0740 1 -- 172.21.15.105:0/3174315460 wait complete. 2024-03-26T17:50:35.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.061+0000 7f3b617b0740 20 client.29834 trim_cache size 0 max 16384 2024-03-26T17:50:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.079+0000 7f3b617b0740 1 Processor -- start 2024-03-26T17:50:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.079+0000 7f3b617b0740 1 -- start start 2024-03-26T17:50:35.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.079+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a9498f0 0x561e5a953f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.079+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a943f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.079+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94f450 0x561e5a966560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.079+0000 7f3b617b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581a8d40 con 0x561e5a9498f0 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.079+0000 7f3b617b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581aa880 con 0x561e5a94f450 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.079+0000 7f3b617b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581a9410 con 0x561e5a94a2e0 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a9498f0 0x561e5a953f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a9498f0 0x561e5a953f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40360/0 (socket says 172.21.15.105:40360) 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 -- 172.21.15.105:0/3891863042 learned_addr learned my addr 172.21.15.105:0/3891863042 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5d16a640 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94f450 0x561e5a966560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b57fff640 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a943f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 -- 172.21.15.105:0/3891863042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 msgr2=0x561e5a943f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a943f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 -- 172.21.15.105:0/3891863042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94f450 msgr2=0x561e5a966560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94f450 0x561e5a966560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 -- 172.21.15.105:0/3891863042 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5a9447b0 con 0x561e5a9498f0 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b5c969640 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a9498f0 0x561e5a953f20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0cd5c0 tx=0x7f3b4c098220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af9494c3c936e10 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b427fc640 1 -- 172.21.15.105:0/3891863042 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c005680 con 0x561e5a9498f0 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b427fc640 1 -- 172.21.15.105:0/3891863042 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b4c0a4650 con 0x561e5a9498f0 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.080+0000 7f3b427fc640 1 -- 172.21.15.105:0/3891863042 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c09ad40 con 0x561e5a9498f0 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 -- 172.21.15.105:0/3891863042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a9498f0 msgr2=0x561e5a953f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a9498f0 0x561e5a953f20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0cd5c0 tx=0x7f3b4c098220 comp rx=0 tx=0).stop 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 -- 172.21.15.105:0/3891863042 shutdown_connections 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94f450 0x561e5a966560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a94a2e0 0x561e5a943f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 --2- 172.21.15.105:0/3891863042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a9498f0 0x561e5a953f20 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 -- 172.21.15.105:0/3891863042 >> 172.21.15.105:0/3891863042 conn(0x561e5a944e60 msgr2=0x561e5a9437f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 -- 172.21.15.105:0/3891863042 shutdown_connections 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 -- 172.21.15.105:0/3891863042 wait complete. 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 -- start start 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.081+0000 7f3b617b0740 1 Processor -- start 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a9498f0 0x561e5a958510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94a2e0 0x561e5a958a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b617b0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a959a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b617b0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581aa1b0 con 0x561e5a94f450 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b617b0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581a9ae0 con 0x561e5a94a2e0 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b617b0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581a6d20 con 0x561e5a9498f0 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b617b0740 10 client.0 _ll_register_callbacks cb 0x7f3b4c004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a959a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b57fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94a2e0 0x561e5a958a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5c969640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a9498f0 0x561e5a958510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a959a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40368/0 (socket says 172.21.15.105:40368) 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 -- 172.21.15.105:0/1646971452 learned_addr learned my addr 172.21.15.105:0/1646971452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b57fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94a2e0 0x561e5a958a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36674/0 (socket says 172.21.15.105:36674) 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 -- 172.21.15.105:0/1646971452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a9498f0 msgr2=0x561e5a958510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a9498f0 0x561e5a958510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 -- 172.21.15.105:0/1646971452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94a2e0 msgr2=0x561e5a958a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94a2e0 0x561e5a958a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 -- 172.21.15.105:0/1646971452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5a95a000 con 0x561e5a94f450 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5c969640 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a9498f0 0x561e5a958510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b5d16a640 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a959a10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f3b5001f550 tx=0x7f3b50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33be9a877e2cba0b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.082+0000 7f3b37fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b37fff640 1 -- 172.21.15.105:0/1646971452 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50004d20 con 0x561e5a94f450 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b37fff640 1 -- 172.21.15.105:0/1646971452 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3b50005020 con 0x561e5a94f450 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 1 -- 172.21.15.105:0/1646971452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b4c0a4650 con 0x561e5a94f450 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b37fff640 1 -- 172.21.15.105:0/1646971452 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3b50005220 con 0x561e5a94f450 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 10 client.20040 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 1 -- 172.21.15.105:0/1646971452 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561e5a9ca110 con 0x561e5a94f450 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b37fff640 1 -- 172.21.15.105:0/1646971452 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b4c0a4650 con 0x561e5a94f450 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b37fff640 1 -- 172.21.15.105:0/1646971452 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3b5001d070 con 0x561e5a94f450 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 10 client.20040 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 10 client.20040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:35.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 20 client.20040 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:35.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e5a9cabc0 0x561e5a9ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:35.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 4 client.20040 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:50:35.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.083+0000 7f3b617b0740 1 -- 172.21.15.105:0/1646971452 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e5a8b6210 con 0x561e5a9cabc0 2024-03-26T17:50:35.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.084+0000 7f3b5c969640 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e5a9cabc0 0x561e5a9ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:35.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.084+0000 7f3b5c969640 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e5a9cabc0 0x561e5a9ccfe0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e603274e1a2d268 server_cookie=f08fe5f1ac46d533 in_seq=0 out_seq=0 2024-03-26T17:50:35.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.084+0000 7f3b37fff640 10 client.20040 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b37fff640 1 -- 172.21.15.105:0/1646971452 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561e5a8b6210 con 0x561e5a9cabc0 2024-03-26T17:50:35.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b37fff640 10 client.20040 handle_command_reply: tid=0 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 client.20040 shutdown 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 -- 172.21.15.105:0/1646971452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 msgr2=0x561e5a959a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a959a10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f3b5001f550 tx=0x7f3b50004ab0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 -- 172.21.15.105:0/1646971452 shutdown_connections 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e5a9cabc0 0x561e5a9ccfe0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e5a94f450 0x561e5a959a10 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e5a94a2e0 0x561e5a958a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 --2- 172.21.15.105:0/1646971452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e5a9498f0 0x561e5a958510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b617b0740 1 -- 172.21.15.105:0/1646971452 >> 172.21.15.105:0/1646971452 conn(0x561e5a944e60 msgr2=0x561e5a9437f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.085+0000 7f3b37fff640 0 client.20040 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b617b0740 1 -- 172.21.15.105:0/1646971452 shutdown_connections 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b617b0740 1 -- 172.21.15.105:0/1646971452 wait complete. 2024-03-26T17:50:35.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b617b0740 20 client.20040 trim_cache size 0 max 16384 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout: "c1cb8e36-97a1-4570-b1b9-5b569ef07041": { 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "c1cb8e36-97a1-4570-b1b9-5b569ef07041", 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:50:35.083 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:50:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b480a2f20 msgr2=0x7f3b480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b480a2f20 0x7f3b480a53e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x561e5a94ab40 tx=0x7f3b50009500 comp rx=0 tx=0).stop 2024-03-26T17:50:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58102340 msgr2=0x7f3b58197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58102340 0x7f3b58197ba0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00e9f0 tx=0x7f3b4c00edf0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/4113502564 shutdown_connections 2024-03-26T17:50:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b480a2f20 0x7f3b480a53e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b581075a0 0x7f3b5819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b58104c70 0x7f3b581980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 --2- 172.21.15.105:0/4113502564 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b58102340 0x7f3b58197ba0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:35.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.086+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/4113502564 >> 172.21.15.105:0/4113502564 conn(0x7f3b580fbd50 msgr2=0x7f3b581041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:35.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.087+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/4113502564 shutdown_connections 2024-03-26T17:50:35.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:50:35.087+0000 7f3b5ebcd640 1 -- 172.21.15.105:0/4113502564 wait complete. 2024-03-26T17:50:35.092 INFO:tasks.cephfs.filesystem:scrub status for tag:c1cb8e36-97a1-4570-b1b9-5b569ef07041 - {'path': '/', 'tag': 'c1cb8e36-97a1-4570-b1b9-5b569ef07041', 'options': 'recursive,force'} 2024-03-26T17:50:36.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 397629 3.15 MB/sec execute 559 sec latency 17.574 ms 2024-03-26T17:50:36.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:35 smithi144 ceph-mon[36095]: pgmap v716: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-03-26T17:50:36.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:35 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:50:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:35 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:35 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:50:36.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:35 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:36.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:35 smithi105 ceph-mon[32280]: pgmap v716: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-03-26T17:50:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:35 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:50:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:35 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:35 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:50:36.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:35 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:35 smithi135 ceph-mon[35636]: pgmap v716: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-03-26T17:50:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:35 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:50:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:35 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:35 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:50:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:35 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:50:37.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 398061 3.14 MB/sec execute 560 sec latency 17.618 ms 2024-03-26T17:50:37.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:36 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:50:37.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:36 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:50:37.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:36 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:50:38.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 398692 3.15 MB/sec execute 561 sec latency 17.682 ms 2024-03-26T17:50:38.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:37 smithi105 ceph-mon[32280]: pgmap v717: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-03-26T17:50:38.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:37 smithi135 ceph-mon[35636]: pgmap v717: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-03-26T17:50:38.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:37 smithi144 ceph-mon[36095]: pgmap v717: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.4 MiB/s wr, 113 op/s 2024-03-26T17:50:39.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 399345 3.14 MB/sec execute 562 sec latency 17.952 ms 2024-03-26T17:50:40.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 399909 3.14 MB/sec execute 563 sec latency 17.470 ms 2024-03-26T17:50:40.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:39 smithi105 ceph-mon[32280]: pgmap v718: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:50:40.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:39 smithi135 ceph-mon[35636]: pgmap v718: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:50:40.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:39 smithi144 ceph-mon[36095]: pgmap v718: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-03-26T17:50:41.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 400633 3.14 MB/sec execute 564 sec latency 17.039 ms 2024-03-26T17:50:42.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 401174 3.15 MB/sec execute 565 sec latency 16.471 ms 2024-03-26T17:50:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:41 smithi105 ceph-mon[32280]: pgmap v719: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:50:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:41 smithi135 ceph-mon[35636]: pgmap v719: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:50:42.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:41 smithi144 ceph-mon[36095]: pgmap v719: 97 pgs: 97 active+clean; 25 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2024-03-26T17:50:43.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 401610 3.14 MB/sec execute 566 sec latency 19.226 ms 2024-03-26T17:50:44.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 402250 3.15 MB/sec execute 567 sec latency 19.418 ms 2024-03-26T17:50:44.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:43 smithi105 ceph-mon[32280]: pgmap v720: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:50:44.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:43 smithi135 ceph-mon[35636]: pgmap v720: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:50:44.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:43 smithi144 ceph-mon[36095]: pgmap v720: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.6 MiB/s wr, 121 op/s 2024-03-26T17:50:45.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 402896 3.14 MB/sec execute 568 sec latency 19.388 ms 2024-03-26T17:50:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 -- 172.21.15.105:0/2807235422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c104c90 msgr2=0x7f0c3c107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2807235422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c104c90 0x7f0c3c107080 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009900 tx=0x7f0c2c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 -- 172.21.15.105:0/2807235422 shutdown_connections 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2807235422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3c1075c0 0x7f0c3c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2807235422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c104c90 0x7f0c3c107080 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2807235422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c102360 0x7f0c3c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 -- 172.21.15.105:0/2807235422 >> 172.21.15.105:0/2807235422 conn(0x7f0c3c0fbd50 msgr2=0x7f0c3c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 -- 172.21.15.105:0/2807235422 shutdown_connections 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 -- 172.21.15.105:0/2807235422 wait complete. 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.939+0000 7f0c440bf640 1 Processor -- start 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c440bf640 1 -- start start 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c440bf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c102360 0x7f0c3c101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c440bf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c104c90 0x7f0c3c100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c440bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3c1075c0 0x7f0c3c100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c440bf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c10d400 con 0x7f0c3c102360 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c440bf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c10d280 con 0x7f0c3c104c90 2024-03-26T17:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c440bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c10d580 con 0x7f0c3c1075c0 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c102360 0x7f0c3c101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c102360 0x7f0c3c101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42896/0 (socket says 172.21.15.105:42896) 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2064949084 learned_addr learned my addr 172.21.15.105:0/2064949084 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c4165a640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c104c90 0x7f0c3c100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c4265c640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3c1075c0 0x7f0c3c100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2064949084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3c1075c0 msgr2=0x7f0c3c100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3c1075c0 0x7f0c3c100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2064949084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c104c90 msgr2=0x7f0c3c100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c104c90 0x7f0c3c100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2064949084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c1010e0 con 0x7f0c3c102360 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c4165a640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c104c90 0x7f0c3c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.940+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c102360 0x7f0c3c101cd0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800e9f0 tx=0x7f0c3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.941+0000 7f0c32ffd640 1 -- 172.21.15.105:0/2064949084 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800cd80 con 0x7f0c3c102360 2024-03-26T17:50:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.941+0000 7f0c32ffd640 1 -- 172.21.15.105:0/2064949084 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c380108e0 con 0x7f0c3c102360 2024-03-26T17:50:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.941+0000 7f0c440bf640 1 -- 172.21.15.105:0/2064949084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c1a1bc0 con 0x7f0c3c102360 2024-03-26T17:50:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.941+0000 7f0c32ffd640 1 -- 172.21.15.105:0/2064949084 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38018920 con 0x7f0c3c102360 2024-03-26T17:50:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.941+0000 7f0c440bf640 1 -- 172.21.15.105:0/2064949084 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c1a2130 con 0x7f0c3c102360 2024-03-26T17:50:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.943+0000 7f0c32ffd640 1 -- 172.21.15.105:0/2064949084 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0c38010a90 con 0x7f0c3c102360 2024-03-26T17:50:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c46d89740 1 Processor -- start 2024-03-26T17:50:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c46d89740 1 -- start start 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c32ffd640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c280a3040 0x7f0c280a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9d49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9da130 0x5589af9dc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c46d89740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5589af9390b0 con 0x5589af9d45d0 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c46d89740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5589af8e1960 con 0x5589af9d4fc0 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c46d89740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5589af935060 con 0x5589af9da130 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c4265c640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c280a3040 0x7f0c280a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c32ffd640 1 -- 172.21.15.105:0/2064949084 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0c380cf670 con 0x7f0c3c102360 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9d49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9d49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42906/0 (socket says 172.21.15.105:42906) 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2849705791 learned_addr learned my addr 172.21.15.105:0/2849705791 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c4165a640 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c4265c640 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9da130 0x5589af9dc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2849705791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9da130 msgr2=0x5589af9dc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9da130 0x5589af9dc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2849705791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 msgr2=0x5589af9d9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9d9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.944+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2849705791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589af8f3c60 con 0x5589af9d45d0 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.945+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9d49f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0c3809bda0 tx=0x7f0c3809b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd003a405ffc1ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.945+0000 7f0c4265c640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c280a3040 0x7f0c280a5500 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005fd0 tx=0x7f0c34011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.945+0000 7f0c27fff640 1 -- 172.21.15.105:0/2849705791 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3809be10 con 0x5589af9d45d0 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.945+0000 7f0c27fff640 1 -- 172.21.15.105:0/2849705791 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c3809feb0 con 0x5589af9d45d0 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.945+0000 7f0c27fff640 1 -- 172.21.15.105:0/2849705791 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38098480 con 0x5589af9d45d0 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 -- 172.21.15.105:0/2849705791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 msgr2=0x5589af9d49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9d49f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f0c3809bda0 tx=0x7f0c3809b830 comp rx=0 tx=0).stop 2024-03-26T17:50:45.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 -- 172.21.15.105:0/2849705791 shutdown_connections 2024-03-26T17:50:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9da130 0x5589af9dc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9d9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2849705791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9d49f0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 -- 172.21.15.105:0/2849705791 >> 172.21.15.105:0/2849705791 conn(0x5589af9cfb40 msgr2=0x5589af9cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 -- 172.21.15.105:0/2849705791 shutdown_connections 2024-03-26T17:50:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 -- 172.21.15.105:0/2849705791 wait complete. 2024-03-26T17:50:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.946+0000 7f0c46d89740 1 -- start start 2024-03-26T17:50:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c46d89740 1 Processor -- start 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d4fc0 0x5589af9de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9da130 0x5589af9e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c46d89740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5589af9dd0e0 con 0x5589af9d45d0 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c46d89740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5589af8f4db0 con 0x5589af9da130 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c46d89740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5589af9dd260 con 0x5589af9d4fc0 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c46d89740 10 client.0 _ll_register_callbacks cb 0x5589af8f9110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9de310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42920/0 (socket says 172.21.15.105:42920) 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2748611282 learned_addr learned my addr 172.21.15.105:0/2748611282 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c4165a640 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d4fc0 0x5589af9de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c4265c640 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9da130 0x5589af9e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2748611282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d4fc0 msgr2=0x5589af9de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d4fc0 0x5589af9de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2748611282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9da130 msgr2=0x5589af9e5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9da130 0x5589af9e5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.947+0000 7f0c41e5b640 1 -- 172.21.15.105:0/2748611282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589af64d210 con 0x5589af9d45d0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9de310 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0c3809bb10 tx=0x7f0c38097a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9d2f5adeaea4c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c257fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c257fa640 1 -- 172.21.15.105:0/2748611282 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38010630 con 0x5589af9d45d0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c257fa640 1 -- 172.21.15.105:0/2748611282 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0c3809b260 con 0x5589af9d45d0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 1 -- 172.21.15.105:0/2748611282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c3809feb0 con 0x5589af9d45d0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c257fa640 1 -- 172.21.15.105:0/2748611282 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c3809be10 con 0x5589af9d45d0 2024-03-26T17:50:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 10 client.20052 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 1 -- 172.21.15.105:0/2748611282 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589af719620 con 0x5589af9d45d0 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c257fa640 1 -- 172.21.15.105:0/2748611282 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c3809feb0 con 0x5589af9d45d0 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c257fa640 1 -- 172.21.15.105:0/2748611282 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0c3807fce0 con 0x5589af9d45d0 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 10 client.20052 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 10 client.20052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 20 client.20052 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5589af9e9d80 0x5589af9ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 4 client.20052 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:45.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.948+0000 7f0c46d89740 1 -- 172.21.15.105:0/2748611282 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589af9465f0 con 0x5589af9e9d80 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.949+0000 7f0c4165a640 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5589af9e9d80 0x5589af9ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.949+0000 7f0c257fa640 10 client.20052 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.949+0000 7f0c4165a640 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5589af9e9d80 0x5589af9ec170 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3688f8c5db12955 server_cookie=91d201691d6f8478 in_seq=0 out_seq=0 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c257fa640 1 -- 172.21.15.105:0/2748611282 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5589af9465f0 con 0x5589af9e9d80 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c257fa640 10 client.20052 handle_command_reply: tid=0 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 client.20052 shutdown 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 -- 172.21.15.105:0/2748611282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 msgr2=0x5589af9de310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9de310 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0c3809bb10 tx=0x7f0c38097a70 comp rx=0 tx=0).stop 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 -- 172.21.15.105:0/2748611282 shutdown_connections 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5589af9e9d80 0x5589af9ec170 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9da130 0x5589af9e5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c257fa640 0 client.20052 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d4fc0 0x5589af9de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 --2- 172.21.15.105:0/2748611282 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9d45d0 0x5589af9de310 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 -- 172.21.15.105:0/2748611282 >> 172.21.15.105:0/2748611282 conn(0x5589af9cfb40 msgr2=0x5589af9cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 -- 172.21.15.105:0/2748611282 shutdown_connections 2024-03-26T17:50:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 1 -- 172.21.15.105:0/2748611282 wait complete. 2024-03-26T17:50:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.950+0000 7f0c46d89740 20 client.20052 trim_cache size 0 max 16384 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c46d89740 1 Processor -- start 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c46d89740 1 -- start start 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c46d89740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1aafb0 con 0x5589af9da130 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c46d89740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1acaf0 con 0x5589af9d4fc0 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c46d89740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1ab680 con 0x5589af9d45d0 2024-03-26T17:50:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4165a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4265c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4165a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50356/0 (socket says 172.21.15.105:50356) 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38466/0 (socket says 172.21.15.105:38466) 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4265c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9d5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42926/0 (socket says 172.21.15.105:42926) 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4165a640 1 -- 172.21.15.105:0/1650255531 learned_addr learned my addr 172.21.15.105:0/1650255531 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4265c640 1 -- 172.21.15.105:0/1650255531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 msgr2=0x5589af9f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4265c640 1 --2- 172.21.15.105:0/1650255531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4265c640 1 -- 172.21.15.105:0/1650255531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 msgr2=0x5589af9e5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4265c640 1 --2- 172.21.15.105:0/1650255531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c4265c640 1 -- 172.21.15.105:0/1650255531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589af9cb450 con 0x5589af9da130 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.966+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/1650255531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.967+0000 7f0c4265c640 1 --2- 172.21.15.105:0/1650255531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9d5a30 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400f790 tx=0x7f0c34009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b6f253bce5694b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:45.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.967+0000 7f0c277fe640 1 -- 172.21.15.105:0/1650255531 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3401a040 con 0x5589af9da130 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.967+0000 7f0c277fe640 1 -- 172.21.15.105:0/1650255531 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c34018040 con 0x5589af9da130 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.967+0000 7f0c277fe640 1 -- 172.21.15.105:0/1650255531 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34021430 con 0x5589af9da130 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- 172.21.15.105:0/1650255531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 msgr2=0x5589af9d5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1650255531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9d5a30 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400f790 tx=0x7f0c34009500 comp rx=0 tx=0).stop 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- 172.21.15.105:0/1650255531 shutdown_connections 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1650255531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9d5a30 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1650255531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1650255531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- 172.21.15.105:0/1650255531 >> 172.21.15.105:0/1650255531 conn(0x5589af9cfb40 msgr2=0x5589af9d2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- 172.21.15.105:0/1650255531 shutdown_connections 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- 172.21.15.105:0/1650255531 wait complete. 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- start start 2024-03-26T17:50:45.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 Processor -- start 2024-03-26T17:50:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9e3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1ac420 con 0x5589af9da130 2024-03-26T17:50:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1abd50 con 0x5589af9d4fc0 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1a8f90 con 0x5589af9d45d0 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.968+0000 7f0c46d89740 10 client.0 _ll_register_callbacks cb 0x7f0c38087d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4165a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c41e5b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9e4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42930/0 (socket says 172.21.15.105:42930) 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 -- 172.21.15.105:0/1394441504 learned_addr learned my addr 172.21.15.105:0/1394441504 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4165a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50366/0 (socket says 172.21.15.105:50366) 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 -- 172.21.15.105:0/1394441504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 msgr2=0x5589af9e3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9e3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 -- 172.21.15.105:0/1394441504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 msgr2=0x5589af9e3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 -- 172.21.15.105:0/1394441504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589af9e4ef0 con 0x5589af9da130 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4165a640 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c4265c640 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9e4900 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005ea0 tx=0x7f0c34020ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24910eeacc90f817 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c257fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c257fa640 1 -- 172.21.15.105:0/1394441504 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3401a050 con 0x5589af9da130 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c257fa640 1 -- 172.21.15.105:0/1394441504 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0c34004c50 con 0x5589af9da130 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c46d89740 1 -- 172.21.15.105:0/1394441504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c34018040 con 0x5589af9da130 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c257fa640 1 -- 172.21.15.105:0/1394441504 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0c3400fdf0 con 0x5589af9da130 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c46d89740 10 client.20058 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.969+0000 7f0c46d89740 1 -- 172.21.15.105:0/1394441504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5589afa502c0 con 0x5589af9da130 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c257fa640 1 -- 172.21.15.105:0/1394441504 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c34018040 con 0x5589af9da130 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c257fa640 1 -- 172.21.15.105:0/1394441504 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0c34003cb0 con 0x5589af9da130 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c46d89740 10 client.20058 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c46d89740 10 client.20058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c46d89740 20 client.20058 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5589afa50cd0 0x5589afa530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c46d89740 4 client.20058 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:50:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c46d89740 1 -- 172.21.15.105:0/1394441504 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589af940940 con 0x5589afa50cd0 2024-03-26T17:50:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5589afa50cd0 0x5589afa530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c257fa640 10 client.20058 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:45.970+0000 7f0c41e5b640 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5589afa50cd0 0x5589afa530f0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a8f9911c8f14c5 server_cookie=d6b65b2b616109ce in_seq=0 out_seq=0 2024-03-26T17:50:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.010+0000 7f0c257fa640 1 -- 172.21.15.105:0/1394441504 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589af940940 con 0x5589afa50cd0 2024-03-26T17:50:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.010+0000 7f0c257fa640 10 client.20058 handle_command_reply: tid=0 2024-03-26T17:50:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.010+0000 7f0c46d89740 1 client.20058 shutdown 2024-03-26T17:50:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.010+0000 7f0c46d89740 1 -- 172.21.15.105:0/1394441504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 msgr2=0x5589af9e4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.010+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9e4900 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005ea0 tx=0x7f0c34020ed0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 1 -- 172.21.15.105:0/1394441504 shutdown_connections 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5589afa50cd0 0x5589afa530f0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5589af9da130 0x5589af9e4900 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589af9d4fc0 0x5589af9e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c257fa640 0 client.20058 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 1 --2- 172.21.15.105:0/1394441504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5589af9d45d0 0x5589af9e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 1 -- 172.21.15.105:0/1394441504 >> 172.21.15.105:0/1394441504 conn(0x5589af9cfb40 msgr2=0x5589af9ddf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 1 -- 172.21.15.105:0/1394441504 shutdown_connections 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 1 -- 172.21.15.105:0/1394441504 wait complete. 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c46d89740 20 client.20058 trim_cache size 0 max 16384 2024-03-26T17:50:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:50:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:50:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:50:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c440bf640 1 -- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c280a3040 msgr2=0x7f0c280a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.011+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c280a3040 0x7f0c280a5500 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005fd0 tx=0x7f0c34011bc0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 -- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c102360 msgr2=0x7f0c3c101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c102360 0x7f0c3c101cd0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800e9f0 tx=0x7f0c3800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 -- 172.21.15.105:0/2064949084 shutdown_connections 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0c280a3040 0x7f0c280a5500 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c3c1075c0 0x7f0c3c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0c3c104c90 0x7f0c3c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 --2- 172.21.15.105:0/2064949084 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c3c102360 0x7f0c3c101cd0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 -- 172.21.15.105:0/2064949084 >> 172.21.15.105:0/2064949084 conn(0x7f0c3c0fbd50 msgr2=0x7f0c3c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 -- 172.21.15.105:0/2064949084 shutdown_connections 2024-03-26T17:50:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:46.012+0000 7f0c440bf640 1 -- 172.21.15.105:0/2064949084 wait complete. 2024-03-26T17:50:46.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 403439 3.14 MB/sec execute 569 sec latency 17.996 ms 2024-03-26T17:50:46.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:45 smithi105 ceph-mon[32280]: pgmap v721: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:50:46.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:45 smithi135 ceph-mon[35636]: pgmap v721: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:50:46.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:45 smithi144 ceph-mon[36095]: pgmap v721: 97 pgs: 97 active+clean; 27 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-03-26T17:50:47.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 404169 3.15 MB/sec execute 570 sec latency 17.268 ms 2024-03-26T17:50:48.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 404681 3.15 MB/sec execute 571 sec latency 16.619 ms 2024-03-26T17:50:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:47 smithi105 ceph-mon[32280]: pgmap v722: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:50:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:47 smithi135 ceph-mon[35636]: pgmap v722: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:50:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:47 smithi144 ceph-mon[36095]: pgmap v722: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-03-26T17:50:49.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 405079 3.14 MB/sec execute 572 sec latency 22.677 ms 2024-03-26T17:50:50.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 405685 3.14 MB/sec execute 573 sec latency 16.190 ms 2024-03-26T17:50:50.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:49 smithi105 ceph-mon[32280]: pgmap v723: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-03-26T17:50:50.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:49 smithi135 ceph-mon[35636]: pgmap v723: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-03-26T17:50:50.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:49 smithi144 ceph-mon[36095]: pgmap v723: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-03-26T17:50:51.038 INFO:tasks.workunit.client.0.smithi105.stdout: 1 406390 3.14 MB/sec execute 574 sec latency 15.870 ms 2024-03-26T17:50:52.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 406992 3.14 MB/sec execute 575 sec latency 16.343 ms 2024-03-26T17:50:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:51 smithi105 ceph-mon[32280]: pgmap v724: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-03-26T17:50:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:51 smithi135 ceph-mon[35636]: pgmap v724: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-03-26T17:50:52.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:51 smithi144 ceph-mon[36095]: pgmap v724: 97 pgs: 97 active+clean; 28 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-03-26T17:50:53.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 407712 3.15 MB/sec execute 576 sec latency 64.857 ms 2024-03-26T17:50:54.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 408210 3.15 MB/sec execute 577 sec latency 51.372 ms 2024-03-26T17:50:54.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:53 smithi105 ceph-mon[32280]: pgmap v725: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:50:54.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:53 smithi135 ceph-mon[35636]: pgmap v725: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:50:54.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:53 smithi144 ceph-mon[36095]: pgmap v725: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2024-03-26T17:50:55.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 408575 3.14 MB/sec execute 578 sec latency 24.156 ms 2024-03-26T17:50:56.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 409160 3.14 MB/sec execute 579 sec latency 17.042 ms 2024-03-26T17:50:56.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:55 smithi105 ceph-mon[32280]: pgmap v726: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-03-26T17:50:56.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:55 smithi135 ceph-mon[35636]: pgmap v726: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-03-26T17:50:56.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:55 smithi144 ceph-mon[36095]: pgmap v726: 97 pgs: 97 active+clean; 29 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-03-26T17:50:57.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 409858 3.14 MB/sec execute 580 sec latency 18.866 ms 2024-03-26T17:50:58.034 INFO:tasks.workunit.client.0.smithi105.stdout: 1 410497 3.14 MB/sec execute 581 sec latency 17.341 ms 2024-03-26T17:50:58.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:57 smithi105 ceph-mon[32280]: pgmap v727: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:50:58.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:57 smithi135 ceph-mon[35636]: pgmap v727: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:50:58.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:57 smithi144 ceph-mon[36095]: pgmap v727: 97 pgs: 97 active+clean; 30 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:50:59.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 411255 3.15 MB/sec execute 582 sec latency 15.851 ms 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.108+0000 7fe9d475b640 1 -- 172.21.15.105:0/84791060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 msgr2=0x7fe9cc100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.108+0000 7fe9d475b640 1 --2- 172.21.15.105:0/84791060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 0x7fe9cc100c00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009900 tx=0x7fe9c802bc30 comp rx=0 tx=0).stop 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 -- 172.21.15.105:0/84791060 shutdown_connections 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 --2- 172.21.15.105:0/84791060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9cc102390 0x7fe9cc108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 --2- 172.21.15.105:0/84791060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9cc1019d0 0x7fe9cc101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 --2- 172.21.15.105:0/84791060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 0x7fe9cc100c00 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 -- 172.21.15.105:0/84791060 >> 172.21.15.105:0/84791060 conn(0x7fe9cc0fbd50 msgr2=0x7fe9cc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 -- 172.21.15.105:0/84791060 shutdown_connections 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 -- 172.21.15.105:0/84791060 wait complete. 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 Processor -- start 2024-03-26T17:50:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 -- start start 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 0x7fe9cc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9cc1019d0 0x7fe9cc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9cc102390 0x7fe9cc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc10c330 con 0x7fe9cc1007e0 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc10c1b0 con 0x7fe9cc102390 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.109+0000 7fe9d475b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc10c4b0 con 0x7fe9cc1019d0 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 0x7fe9cc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 0x7fe9cc197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44792/0 (socket says 172.21.15.105:44792) 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 -- 172.21.15.105:0/928854945 learned_addr learned my addr 172.21.15.105:0/928854945 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9cc1019d0 0x7fe9cc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d2cf8640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9cc102390 0x7fe9cc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 -- 172.21.15.105:0/928854945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9cc1019d0 msgr2=0x7fe9cc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9cc1019d0 0x7fe9cc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 -- 172.21.15.105:0/928854945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9cc102390 msgr2=0x7fe9cc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9cc102390 0x7fe9cc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 -- 172.21.15.105:0/928854945 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc19f6c0 con 0x7fe9cc1007e0 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 0x7fe9cc197aa0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8000c00 tx=0x7fe9c8005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9c37fe640 1 -- 172.21.15.105:0/928854945 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8043020 con 0x7fe9cc1007e0 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d475b640 1 -- 172.21.15.105:0/928854945 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc19f950 con 0x7fe9cc1007e0 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9c37fe640 1 -- 172.21.15.105:0/928854945 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9c80043d0 con 0x7fe9cc1007e0 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9c37fe640 1 -- 172.21.15.105:0/928854945 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8003b40 con 0x7fe9cc1007e0 2024-03-26T17:50:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.110+0000 7fe9d475b640 1 -- 172.21.15.105:0/928854945 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc19fea0 con 0x7fe9cc1007e0 2024-03-26T17:50:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9c37fe640 1 -- 172.21.15.105:0/928854945 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe9c8046020 con 0x7fe9cc1007e0 2024-03-26T17:50:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d7422740 1 Processor -- start 2024-03-26T17:50:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d7422740 1 -- start start 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a298fc0 0x561e0a29dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a29e130 0x561e0a2a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d7422740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e0a1fd0b0 con 0x561e0a2985d0 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d7422740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e0a1a5960 con 0x561e0a29e130 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d7422740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e0a1f9060 con 0x561e0a298fc0 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d2cf8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d1cf6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a29e130 0x561e0a2a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d2cf8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44806/0 (socket says 172.21.15.105:44806) 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d2cf8640 1 -- 172.21.15.105:0/774534514 learned_addr learned my addr 172.21.15.105:0/774534514 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a29e130 0x561e0a2a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48738/0 (socket says 172.21.15.105:48738) 2024-03-26T17:50:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.113+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a298fc0 0x561e0a29dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9c37fe640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9b80a2fc0 0x7fe9b80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d24f7640 1 -- 172.21.15.105:0/774534514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a29e130 msgr2=0x561e0a2a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a29e130 0x561e0a2a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d24f7640 1 -- 172.21.15.105:0/774534514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 msgr2=0x561e0a2989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d24f7640 1 -- 172.21.15.105:0/774534514 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0a1b7c60 con 0x561e0a298fc0 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d2cf8640 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d2cf8640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9b80a2fc0 0x7fe9b80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9c37fe640 1 -- 172.21.15.105:0/928854945 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe9c80f1580 con 0x7fe9cc1007e0 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a298fc0 0x561e0a29dac0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80ef010 tx=0x7fe9c80887c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c86387e2db5d173 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9c0ff9640 1 -- 172.21.15.105:0/774534514 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80bd7b0 con 0x561e0a298fc0 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9d2cf8640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9b80a2fc0 0x7fe9b80a5480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x561e0a299820 tx=0x7fe9c4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9c0ff9640 1 -- 172.21.15.105:0/774534514 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9c80be370 con 0x561e0a298fc0 2024-03-26T17:50:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.114+0000 7fe9c0ff9640 1 -- 172.21.15.105:0/774534514 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80bd5d0 con 0x561e0a298fc0 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 -- 172.21.15.105:0/774534514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a298fc0 msgr2=0x561e0a29dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a298fc0 0x561e0a29dac0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80ef010 tx=0x7fe9c80887c0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 -- 172.21.15.105:0/774534514 shutdown_connections 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a29e130 0x561e0a2a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a298fc0 0x561e0a29dac0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 --2- 172.21.15.105:0/774534514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 -- 172.21.15.105:0/774534514 >> 172.21.15.105:0/774534514 conn(0x561e0a293b40 msgr2=0x561e0a293f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 -- 172.21.15.105:0/774534514 shutdown_connections 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.115+0000 7fe9d7422740 1 -- 172.21.15.105:0/774534514 wait complete. 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 1 -- start start 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 1 Processor -- start 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a2a9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e0a2a0ea0 con 0x561e0a2985d0 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e0a1b8db0 con 0x561e0a298fc0 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e0a2a1020 con 0x561e0a29e130 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d7422740 10 client.0 _ll_register_callbacks cb 0x561e0a1bd110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d24f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d1cf6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d24f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2a1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44822/0 (socket says 172.21.15.105:44822) 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d24f7640 1 -- 172.21.15.105:0/433137949 learned_addr learned my addr 172.21.15.105:0/433137949 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.116+0000 7fe9d2cf8640 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a2a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d1cf6640 1 -- 172.21.15.105:0/433137949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 msgr2=0x561e0a2a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a2a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d1cf6640 1 -- 172.21.15.105:0/433137949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 msgr2=0x561e0a2a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d1cf6640 1 -- 172.21.15.105:0/433137949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e09f11210 con 0x561e0a298fc0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a2500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc198fb0 tx=0x7fe9bc012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f33b4677a598be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/433137949 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00f920 con 0x561e0a298fc0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d7422740 1 -- 172.21.15.105:0/433137949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c80be370 con 0x561e0a298fc0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/433137949 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe9bc00fac0 con 0x561e0a298fc0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/433137949 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe9bc00fcc0 con 0x561e0a298fc0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d7422740 10 client.29870 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9d7422740 1 -- 172.21.15.105:0/433137949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561e09fdd620 con 0x561e0a298fc0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.117+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/433137949 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9c80be370 con 0x561e0a298fc0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/433137949 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe9bc010430 con 0x561e0a298fc0 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9d7422740 10 client.29870 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9d7422740 10 client.29870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9d7422740 20 client.29870 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9d7422740 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e0a2addc0 0x561e0a2b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9d7422740 4 client.29870 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9d7422740 1 -- 172.21.15.105:0/433137949 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e0a20a5f0 con 0x561e0a2addc0 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e0a2addc0 0x561e0a2b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e0a2addc0 0x561e0a2b0230 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4c84cd0015a6bd server_cookie=47c041271eeb1a78 in_seq=0 out_seq=0 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.118+0000 7fe9b5ffb640 10 client.29870 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/433137949 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561e0a20a5f0 con 0x561e0a2addc0 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9b5ffb640 10 client.29870 handle_command_reply: tid=0 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 client.29870 shutdown 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 -- 172.21.15.105:0/433137949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 msgr2=0x561e0a2a2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a2500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc198fb0 tx=0x7fe9bc012b60 comp rx=0 tx=0).stop 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 -- 172.21.15.105:0/433137949 shutdown_connections 2024-03-26T17:50:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e0a2addc0 0x561e0a2b0230 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a2a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a2500 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 --2- 172.21.15.105:0/433137949 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a2a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9b5ffb640 0 client.29870 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 -- 172.21.15.105:0/433137949 >> 172.21.15.105:0/433137949 conn(0x561e0a293b40 msgr2=0x561e0a292df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 -- 172.21.15.105:0/433137949 shutdown_connections 2024-03-26T17:50:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 1 -- 172.21.15.105:0/433137949 wait complete. 2024-03-26T17:50:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.119+0000 7fe9d7422740 20 client.29870 trim_cache size 0 max 16384 2024-03-26T17:50:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d7422740 1 Processor -- start 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d7422740 1 -- start start 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a28ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a299850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d7422740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1a8de0 con 0x561e0a2985d0 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d7422740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1aa920 con 0x561e0a298fc0 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d7422740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1a94b0 con 0x561e0a29e130 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d24f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a28ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d24f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a28ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44824/0 (socket says 172.21.15.105:44824) 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d1cf6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d24f7640 1 -- 172.21.15.105:0/4290307750 learned_addr learned my addr 172.21.15.105:0/4290307750 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.136+0000 7fe9d2cf8640 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a299850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9d1cf6640 1 -- 172.21.15.105:0/4290307750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 msgr2=0x561e0a299850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a299850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9d1cf6640 1 -- 172.21.15.105:0/4290307750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 msgr2=0x561e0a28ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a28ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9d1cf6640 1 -- 172.21.15.105:0/4290307750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0a28f450 con 0x561e0a298fc0 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a28ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a9500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc010a90 tx=0x7fe9bc009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be713bf2fe7c6aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9b7fff640 1 -- 172.21.15.105:0/4290307750 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc000e60 con 0x561e0a298fc0 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9b7fff640 1 -- 172.21.15.105:0/4290307750 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9bc02b070 con 0x561e0a298fc0 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.137+0000 7fe9b7fff640 1 -- 172.21.15.105:0/4290307750 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc010d30 con 0x561e0a298fc0 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- 172.21.15.105:0/4290307750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 msgr2=0x561e0a2a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a9500 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc010a90 tx=0x7fe9bc009ea0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- 172.21.15.105:0/4290307750 shutdown_connections 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a299850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a298fc0 0x561e0a2a9500 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 --2- 172.21.15.105:0/4290307750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a2985d0 0x561e0a28ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- 172.21.15.105:0/4290307750 >> 172.21.15.105:0/4290307750 conn(0x561e0a293b40 msgr2=0x561e0a29d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- 172.21.15.105:0/4290307750 shutdown_connections 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- 172.21.15.105:0/4290307750 wait complete. 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- start start 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 Processor -- start 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a2985d0 0x561e0a2a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a298fc0 0x561e0a2a7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a2a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1aa250 con 0x561e0a298fc0 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1a9b80 con 0x561e0a2985d0 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1a6dc0 con 0x561e0a29e130 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.138+0000 7fe9d7422740 10 client.0 _ll_register_callbacks cb 0x7fe9bc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:50:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a2985d0 0x561e0a2a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d1cf6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a298fc0 0x561e0a2a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a2985d0 0x561e0a2a6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48768/0 (socket says 172.21.15.105:48768) 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 -- 172.21.15.105:0/3098689030 learned_addr learned my addr 172.21.15.105:0/3098689030 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d2cf8640 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a2a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 -- 172.21.15.105:0/3098689030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 msgr2=0x561e0a2a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a2a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 -- 172.21.15.105:0/3098689030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a298fc0 msgr2=0x561e0a2a7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a298fc0 0x561e0a2a7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 -- 172.21.15.105:0/3098689030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0a2a8ad0 con 0x561e0a2985d0 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a298fc0 0x561e0a2a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d24f7640 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a2985d0 0x561e0a2a6fe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80ef3b0 tx=0x7fe9c80838a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2026df55af85f37e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/3098689030 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80bd7b0 con 0x561e0a2985d0 2024-03-26T17:50:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/3098689030 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe9c80b9a80 con 0x561e0a2985d0 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d7422740 1 -- 172.21.15.105:0/3098689030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9bc02b070 con 0x561e0a2985d0 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/3098689030 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe9c80be370 con 0x561e0a2985d0 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d7422740 10 client.29876 fetch_fsmap learned FSMap version 32 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.139+0000 7fe9d7422740 1 -- 172.21.15.105:0/3098689030 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561e0a3144c0 con 0x561e0a2985d0 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/3098689030 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9bc02b070 con 0x561e0a2985d0 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/3098689030 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe9c8003ce0 con 0x561e0a2985d0 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9d7422740 10 client.29876 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9d7422740 10 client.29876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9d7422740 20 client.29876 populate_metadata read hostname 'smithi105' 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9d7422740 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e0a314cd0 0x561e0a3170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9d7422740 4 client.29876 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9d7422740 1 -- 172.21.15.105:0/3098689030 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e0a204940 con 0x561e0a314cd0 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e0a314cd0 0x561e0a3170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:50:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9d1cf6640 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e0a314cd0 0x561e0a3170f0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67e65a3ee901c62 server_cookie=e844ee59e9e8b467 in_seq=0 out_seq=0 2024-03-26T17:50:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.140+0000 7fe9b5ffb640 10 client.29876 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9b5ffb640 1 -- 172.21.15.105:0/3098689030 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e0a204940 con 0x561e0a314cd0 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9b5ffb640 10 client.29876 handle_command_reply: tid=0 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 client.29876 shutdown 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 -- 172.21.15.105:0/3098689030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a2985d0 msgr2=0x561e0a2a6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a2985d0 0x561e0a2a6fe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80ef3b0 tx=0x7fe9c80838a0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 -- 172.21.15.105:0/3098689030 shutdown_connections 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e0a314cd0 0x561e0a3170f0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e0a29e130 0x561e0a2a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9b5ffb640 0 client.29876 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e0a298fc0 0x561e0a2a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 --2- 172.21.15.105:0/3098689030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e0a2985d0 0x561e0a2a6fe0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 -- 172.21.15.105:0/3098689030 >> 172.21.15.105:0/3098689030 conn(0x561e0a293b40 msgr2=0x561e0a299850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 -- 172.21.15.105:0/3098689030 shutdown_connections 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 1 -- 172.21.15.105:0/3098689030 wait complete. 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.187+0000 7fe9d7422740 20 client.29876 trim_cache size 0 max 16384 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:50:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:50:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:50:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 -- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9b80a2fc0 msgr2=0x7fe9b80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9b80a2fc0 0x7fe9b80a5480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x561e0a299820 tx=0x7fe9c4004a40 comp rx=0 tx=0).stop 2024-03-26T17:50:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 -- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 msgr2=0x7fe9cc197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:50:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 0x7fe9cc197aa0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8000c00 tx=0x7fe9c8005e60 comp rx=0 tx=0).stop 2024-03-26T17:50:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 -- 172.21.15.105:0/928854945 shutdown_connections 2024-03-26T17:50:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9b80a2fc0 0x7fe9b80a5480 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9cc102390 0x7fe9cc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9cc1019d0 0x7fe9cc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 --2- 172.21.15.105:0/928854945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9cc1007e0 0x7fe9cc197aa0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:50:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 -- 172.21.15.105:0/928854945 >> 172.21.15.105:0/928854945 conn(0x7fe9cc0fbd50 msgr2=0x7fe9cc068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:50:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 -- 172.21.15.105:0/928854945 shutdown_connections 2024-03-26T17:50:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:50:59.188+0000 7fe9d475b640 1 -- 172.21.15.105:0/928854945 wait complete. 2024-03-26T17:51:00.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 411725 3.15 MB/sec execute 583 sec latency 40.050 ms 2024-03-26T17:51:00.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:59 smithi105 ceph-mon[32280]: pgmap v728: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:51:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:51:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:00.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:50:59 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:59 smithi135 ceph-mon[35636]: pgmap v728: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:51:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:51:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:50:59 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:00.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:59 smithi144 ceph-mon[36095]: pgmap v728: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.4 MiB/s wr, 98 op/s 2024-03-26T17:51:00.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:51:00.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:00.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:50:59 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:01.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 412094 3.14 MB/sec execute 584 sec latency 15.422 ms 2024-03-26T17:51:02.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 412658 3.14 MB/sec execute 585 sec latency 23.791 ms 2024-03-26T17:51:02.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:01 smithi105 ceph-mon[32280]: pgmap v729: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-03-26T17:51:02.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:01 smithi135 ceph-mon[35636]: pgmap v729: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-03-26T17:51:02.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:01 smithi144 ceph-mon[36095]: pgmap v729: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.4 MiB/s wr, 93 op/s 2024-03-26T17:51:03.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 413320 3.14 MB/sec execute 586 sec latency 18.655 ms 2024-03-26T17:51:03.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:51:03.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:51:03.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:02 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:02 smithi105 ceph-mon[32280]: pgmap v730: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:51:03.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:51:03.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:51:03.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:02 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:02 smithi135 ceph-mon[35636]: pgmap v730: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:51:03.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:51:03.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:51:03.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:02 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:51:03.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:02 smithi144 ceph-mon[36095]: pgmap v730: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-03-26T17:51:04.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 413934 3.14 MB/sec execute 587 sec latency 19.420 ms 2024-03-26T17:51:05.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 414780 3.15 MB/sec execute 588 sec latency 16.294 ms 2024-03-26T17:51:05.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:51:05.094 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:51:05.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- 172.21.15.105:0/1643101946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201019b0 msgr2=0x7f9720101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 --2- 172.21.15.105:0/1643101946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201019b0 0x7f9720101e30 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f970c009900 tx=0x7f970c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:05.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- 172.21.15.105:0/1643101946 shutdown_connections 2024-03-26T17:51:05.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 --2- 172.21.15.105:0/1643101946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9720102370 0x7f9720108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 --2- 172.21.15.105:0/1643101946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201019b0 0x7f9720101e30 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 --2- 172.21.15.105:0/1643101946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97201007c0 0x7f9720100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- 172.21.15.105:0/1643101946 >> 172.21.15.105:0/1643101946 conn(0x7f97200fbd50 msgr2=0x7f97200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- 172.21.15.105:0/1643101946 shutdown_connections 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- 172.21.15.105:0/1643101946 wait complete. 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 Processor -- start 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- start start 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201007c0 0x7f9720104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97201019b0 0x7f9720102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9720102370 0x7f97201032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f972010c2f0 con 0x7f97201007c0 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f972010c170 con 0x7f9720102370 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.232+0000 7f972713b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f972010c470 con 0x7f97201019b0 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201007c0 0x7f9720104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f97256d8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9720102370 0x7f97201032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201007c0 0x7f9720104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42858/0 (socket says 172.21.15.105:42858) 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 -- 172.21.15.105:0/2358447994 learned_addr learned my addr 172.21.15.105:0/2358447994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f971ffff640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97201019b0 0x7f9720102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 -- 172.21.15.105:0/2358447994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97201019b0 msgr2=0x7f9720102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97201019b0 0x7f9720102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 -- 172.21.15.105:0/2358447994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9720102370 msgr2=0x7f97201032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9720102370 0x7f97201032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 -- 172.21.15.105:0/2358447994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9720103b60 con 0x7f97201007c0 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f97256d8640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9720102370 0x7f97201032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f9724ed7640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201007c0 0x7f9720104750 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f971400e9c0 tx=0x7f971400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f971dffb640 1 -- 172.21.15.105:0/2358447994 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971400cd80 con 0x7f97201007c0 2024-03-26T17:51:05.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f971dffb640 1 -- 172.21.15.105:0/2358447994 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97140035f0 con 0x7f97201007c0 2024-03-26T17:51:05.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f971dffb640 1 -- 172.21.15.105:0/2358447994 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714010690 con 0x7f97201007c0 2024-03-26T17:51:05.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f972713b640 1 -- 172.21.15.105:0/2358447994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97201a1c60 con 0x7f97201007c0 2024-03-26T17:51:05.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.233+0000 7f972713b640 1 -- 172.21.15.105:0/2358447994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97201a2130 con 0x7f97201007c0 2024-03-26T17:51:05.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.236+0000 7f971dffb640 1 -- 172.21.15.105:0/2358447994 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f97140037a0 con 0x7f97201007c0 2024-03-26T17:51:05.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9729e08740 1 Processor -- start 2024-03-26T17:51:05.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9729e08740 1 -- start start 2024-03-26T17:51:05.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f971dffb640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97100a2f20 0x7f97100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9729e08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c8ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9729e08740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c8fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9729e08740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9729e08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557739bef980 con 0x557739c8a8f0 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9729e08740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557739b97be0 con 0x557739c90450 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9729e08740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557739beb930 con 0x557739c8b2e0 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f97256d8640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97100a2f20 0x7f97100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f971dffb640 1 -- 172.21.15.105:0/2358447994 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97140cf970 con 0x7f97201007c0 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c8ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f97256d8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f971ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c8fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c8ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42872/0 (socket says 172.21.15.105:42872) 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f97256d8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c928e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60306/0 (socket says 172.21.15.105:60306) 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f9724ed7640 1 -- 172.21.15.105:0/4135422056 learned_addr learned my addr 172.21.15.105:0/4135422056 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f971ffff640 1 -- 172.21.15.105:0/4135422056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 msgr2=0x557739c928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f971ffff640 1 --2- 172.21.15.105:0/4135422056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f971ffff640 1 -- 172.21.15.105:0/4135422056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 msgr2=0x557739c8ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f971ffff640 1 --2- 172.21.15.105:0/4135422056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c8ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.237+0000 7f971ffff640 1 -- 172.21.15.105:0/4135422056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577399cd7d0 con 0x557739c8b2e0 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.238+0000 7f97256d8640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97100a2f20 0x7f97100a53e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f97180099e0 tx=0x7f9718013660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.238+0000 7f971ffff640 1 --2- 172.21.15.105:0/4135422056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c8fde0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f970c000c00 tx=0x7f970c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21331e30c16fc97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.238+0000 7f970affd640 1 -- 172.21.15.105:0/4135422056 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c045020 con 0x557739c8b2e0 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.238+0000 7f970affd640 1 -- 172.21.15.105:0/4135422056 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f970c004040 con 0x557739c8b2e0 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.238+0000 7f970affd640 1 -- 172.21.15.105:0/4135422056 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c005560 con 0x557739c8b2e0 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.238+0000 7f9729e08740 1 -- 172.21.15.105:0/4135422056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 msgr2=0x557739c8fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.238+0000 7f9729e08740 1 --2- 172.21.15.105:0/4135422056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c8fde0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f970c000c00 tx=0x7f970c009560 comp rx=0 tx=0).stop 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 -- 172.21.15.105:0/4135422056 shutdown_connections 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 --2- 172.21.15.105:0/4135422056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 --2- 172.21.15.105:0/4135422056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c8fde0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 --2- 172.21.15.105:0/4135422056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c8ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 -- 172.21.15.105:0/4135422056 >> 172.21.15.105:0/4135422056 conn(0x557739c85e60 msgr2=0x557739c86260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 -- 172.21.15.105:0/4135422056 shutdown_connections 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 -- 172.21.15.105:0/4135422056 wait complete. 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 -- start start 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 Processor -- start 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c94a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c94f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c9bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557739c933a0 con 0x557739c8a8f0 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557739bab160 con 0x557739c90450 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557739c93520 con 0x557739c8b2e0 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9729e08740 10 client.0 _ll_register_callbacks cb 0x557739be7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:05.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c94a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c94a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42884/0 (socket says 172.21.15.105:42884) 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f971ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c94f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.239+0000 7f9724ed7640 1 -- 172.21.15.105:0/1604362469 learned_addr learned my addr 172.21.15.105:0/1604362469 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f97256d8640 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c9bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9724ed7640 1 -- 172.21.15.105:0/1604362469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 msgr2=0x557739c94f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9724ed7640 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c94f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9724ed7640 1 -- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 msgr2=0x557739c9bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9724ed7640 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c9bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9724ed7640 1 -- 172.21.15.105:0/1604362469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577399033f0 con 0x557739c8a8f0 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f971ffff640 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c94f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f97256d8640 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c9bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9724ed7640 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c94a50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f971400cfd0 tx=0x7f971409baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cc546c5245acea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f96fffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:05.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f96fffff640 1 -- 172.21.15.105:0/1604362469 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714003b10 con 0x557739c8a8f0 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f96fffff640 1 -- 172.21.15.105:0/1604362469 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9714076820 con 0x557739c8a8f0 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9729e08740 1 -- 172.21.15.105:0/1604362469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970c004040 con 0x557739c8a8f0 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f96fffff640 1 -- 172.21.15.105:0/1604362469 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f971409c760 con 0x557739c8a8f0 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9729e08740 10 client.20088 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f9729e08740 1 -- 172.21.15.105:0/1604362469 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5577398c30f0 con 0x557739c8a8f0 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.240+0000 7f96fffff640 1 -- 172.21.15.105:0/1604362469 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f970c004040 con 0x557739c8a8f0 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f96fffff640 1 -- 172.21.15.105:0/1604362469 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9714096bd0 con 0x557739c8a8f0 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f9729e08740 10 client.20088 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f9729e08740 10 client.20088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f9729e08740 20 client.20088 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f9729e08740 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557739ca08e0 0x557739ca2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f9729e08740 4 client.20088 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f9729e08740 1 -- 172.21.15.105:0/1604362469 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557739bfcec0 con 0x557739ca08e0 2024-03-26T17:51:05.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f971ffff640 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557739ca08e0 0x557739ca2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f971ffff640 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557739ca08e0 0x557739ca2da0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ee0020eedda207 server_cookie=985bb7eaf9a2d2c1 in_seq=0 out_seq=0 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.241+0000 7f96fffff640 10 client.20088 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f96fffff640 1 -- 172.21.15.105:0/1604362469 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557739bfcec0 con 0x557739ca08e0 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f96fffff640 10 client.20088 handle_command_reply: tid=0 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 client.20088 shutdown 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 -- 172.21.15.105:0/1604362469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 msgr2=0x557739c94a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c94a50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f971400cfd0 tx=0x7f971409baf0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 -- 172.21.15.105:0/1604362469 shutdown_connections 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557739ca08e0 0x557739ca2da0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c90450 0x557739c9bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f96fffff640 0 client.20088 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c94f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 --2- 172.21.15.105:0/1604362469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c94a50 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 -- 172.21.15.105:0/1604362469 >> 172.21.15.105:0/1604362469 conn(0x557739c85e60 msgr2=0x557739c85080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 -- 172.21.15.105:0/1604362469 shutdown_connections 2024-03-26T17:51:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 1 -- 172.21.15.105:0/1604362469 wait complete. 2024-03-26T17:51:05.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.243+0000 7f9729e08740 20 client.20088 trim_cache size 0 max 16384 2024-03-26T17:51:05.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.266+0000 7f9729e08740 1 Processor -- start 2024-03-26T17:51:05.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.266+0000 7f9729e08740 1 -- start start 2024-03-26T17:51:05.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f9729e08740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8a8f0 0x557739c86240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f9729e08740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c9bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f9729e08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c90450 0x557739c94660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f9729e08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97201aaea0 con 0x557739c90450 2024-03-26T17:51:05.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f9729e08740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97201ac9e0 con 0x557739c8a8f0 2024-03-26T17:51:05.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f9729e08740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97201ab570 con 0x557739c8b2e0 2024-03-26T17:51:05.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f97256d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c90450 0x557739c94660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f971ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c9bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8a8f0 0x557739c86240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f97256d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c90450 0x557739c94660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42890/0 (socket says 172.21.15.105:42890) 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f97256d8640 1 -- 172.21.15.105:0/2087964380 learned_addr learned my addr 172.21.15.105:0/2087964380 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f971ffff640 1 -- 172.21.15.105:0/2087964380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8a8f0 msgr2=0x557739c86240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f971ffff640 1 --2- 172.21.15.105:0/2087964380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8a8f0 0x557739c86240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f971ffff640 1 -- 172.21.15.105:0/2087964380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c90450 msgr2=0x557739c94660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f971ffff640 1 --2- 172.21.15.105:0/2087964380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c90450 0x557739c94660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f971ffff640 1 -- 172.21.15.105:0/2087964380 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557739c8bfb0 con 0x557739c8b2e0 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f97256d8640 1 --2- 172.21.15.105:0/2087964380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c90450 0x557739c94660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f971ffff640 1 --2- 172.21.15.105:0/2087964380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c9bf90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x557739c86dd0 tx=0x7f970c02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c6c96c848b240a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f970a7fc640 1 -- 172.21.15.105:0/2087964380 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c045040 con 0x557739c8b2e0 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f970a7fc640 1 -- 172.21.15.105:0/2087964380 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f970c054170 con 0x557739c8b2e0 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.267+0000 7f970a7fc640 1 -- 172.21.15.105:0/2087964380 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c0416c0 con 0x557739c8b2e0 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 -- 172.21.15.105:0/2087964380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 msgr2=0x557739c9bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 --2- 172.21.15.105:0/2087964380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c9bf90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x557739c86dd0 tx=0x7f970c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:05.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 -- 172.21.15.105:0/2087964380 shutdown_connections 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 --2- 172.21.15.105:0/2087964380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c90450 0x557739c94660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 --2- 172.21.15.105:0/2087964380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c8b2e0 0x557739c9bf90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 --2- 172.21.15.105:0/2087964380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8a8f0 0x557739c86240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 -- 172.21.15.105:0/2087964380 >> 172.21.15.105:0/2087964380 conn(0x557739c85e60 msgr2=0x557739c938c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 -- 172.21.15.105:0/2087964380 shutdown_connections 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 -- 172.21.15.105:0/2087964380 wait complete. 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 -- start start 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.268+0000 7f9729e08740 1 Processor -- start 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9729e08740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c99360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9729e08740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8b2e0 0x557739c998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9729e08740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c90450 0x557739c9a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9729e08740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97201ac310 con 0x557739c8a8f0 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9729e08740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97201abc40 con 0x557739c8b2e0 2024-03-26T17:51:05.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9729e08740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97201a8e80 con 0x557739c90450 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9729e08740 10 client.0 _ll_register_callbacks cb 0x7f971409be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c99360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c99360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42908/0 (socket says 172.21.15.105:42908) 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 -- 172.21.15.105:0/3228050791 learned_addr learned my addr 172.21.15.105:0/3228050791 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f971ffff640 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8b2e0 0x557739c998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f97256d8640 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c90450 0x557739c9a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 -- 172.21.15.105:0/3228050791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c90450 msgr2=0x557739c9a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c90450 0x557739c9a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 -- 172.21.15.105:0/3228050791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8b2e0 msgr2=0x557739c998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8b2e0 0x557739c998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 -- 172.21.15.105:0/3228050791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557739c9aea0 con 0x557739c8a8f0 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f9724ed7640 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c99360 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f971400c6a0 tx=0x7f97140a00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3aea9ca1162dc03 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.269+0000 7f96fffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:05.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f96fffff640 1 -- 172.21.15.105:0/3228050791 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714003b10 con 0x557739c8a8f0 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f96fffff640 1 -- 172.21.15.105:0/3228050791 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97140767d0 con 0x557739c8a8f0 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 1 -- 172.21.15.105:0/3228050791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970c054170 con 0x557739c8a8f0 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f96fffff640 1 -- 172.21.15.105:0/3228050791 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f97140a4a70 con 0x557739c8a8f0 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 10 client.20094 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 1 -- 172.21.15.105:0/3228050791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557739c9c220 con 0x557739c8a8f0 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f96fffff640 1 -- 172.21.15.105:0/3228050791 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f970c054170 con 0x557739c8a8f0 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f96fffff640 1 -- 172.21.15.105:0/3228050791 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9714097db0 con 0x557739c8a8f0 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 10 client.20094 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 10 client.20094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 20 client.20094 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557739d07380 0x557739d097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 4 client.20094 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:51:05.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f9729e08740 1 -- 172.21.15.105:0/3228050791 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557739bf7210 con 0x557739d07380 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.270+0000 7f971ffff640 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557739d07380 0x557739d097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f96fffff640 10 client.20094 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f971ffff640 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557739d07380 0x557739d097a0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=244240e1e25ace2c server_cookie=8ee14bda16f2d91e in_seq=0 out_seq=0 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f96fffff640 1 -- 172.21.15.105:0/3228050791 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557739bf7210 con 0x557739d07380 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f96fffff640 10 client.20094 handle_command_reply: tid=0 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 client.20094 shutdown 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 -- 172.21.15.105:0/3228050791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 msgr2=0x557739c99360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c99360 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f971400c6a0 tx=0x7f97140a00d0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 -- 172.21.15.105:0/3228050791 shutdown_connections 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557739d07380 0x557739d097a0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557739c90450 0x557739c9a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f96fffff640 0 client.20094 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557739c8b2e0 0x557739c998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 --2- 172.21.15.105:0/3228050791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557739c8a8f0 0x557739c99360 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 -- 172.21.15.105:0/3228050791 >> 172.21.15.105:0/3228050791 conn(0x557739c85e60 msgr2=0x557739c86240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 -- 172.21.15.105:0/3228050791 shutdown_connections 2024-03-26T17:51:05.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 1 -- 172.21.15.105:0/3228050791 wait complete. 2024-03-26T17:51:05.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.271+0000 7f9729e08740 20 client.20094 trim_cache size 0 max 16384 2024-03-26T17:51:05.272 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:51:05.272 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:51:05.272 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:51:05.272 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:51:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 -- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97100a2f20 msgr2=0x7f97100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97100a2f20 0x7f97100a53e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f97180099e0 tx=0x7f9718013660 comp rx=0 tx=0).stop 2024-03-26T17:51:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 -- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201007c0 msgr2=0x7f9720104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201007c0 0x7f9720104750 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f971400e9c0 tx=0x7f971400ee80 comp rx=0 tx=0).stop 2024-03-26T17:51:05.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 -- 172.21.15.105:0/2358447994 shutdown_connections 2024-03-26T17:51:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97100a2f20 0x7f97100a53e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9720102370 0x7f97201032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97201019b0 0x7f9720102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 --2- 172.21.15.105:0/2358447994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97201007c0 0x7f9720104750 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 -- 172.21.15.105:0/2358447994 >> 172.21.15.105:0/2358447994 conn(0x7f97200fbd50 msgr2=0x7f9720068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 -- 172.21.15.105:0/2358447994 shutdown_connections 2024-03-26T17:51:05.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.272+0000 7f972713b640 1 -- 172.21.15.105:0/2358447994 wait complete. 2024-03-26T17:51:05.275 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:51:05.276 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:51:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 -- 172.21.15.105:0/2658279977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e4101900 msgr2=0x7f65e4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 --2- 172.21.15.105:0/2658279977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e4101900 0x7f65e4101d20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009900 tx=0x7f65e002bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 -- 172.21.15.105:0/2658279977 shutdown_connections 2024-03-26T17:51:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 --2- 172.21.15.105:0/2658279977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e41034b0 0x7f65e4109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 --2- 172.21.15.105:0/2658279977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65e4102af0 0x7f65e4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 --2- 172.21.15.105:0/2658279977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e4101900 0x7f65e4101d20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 -- 172.21.15.105:0/2658279977 >> 172.21.15.105:0/2658279977 conn(0x7f65e40fce70 msgr2=0x7f65e40ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 -- 172.21.15.105:0/2658279977 shutdown_connections 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.397+0000 7f65eb553640 1 -- 172.21.15.105:0/2658279977 wait complete. 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65eb553640 1 Processor -- start 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65eb553640 1 -- start start 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65eb553640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65e4101900 0x7f65e4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65eb553640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e4102af0 0x7f65e41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65eb553640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e41034b0 0x7f65e419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65eb553640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65e410d420 con 0x7f65e4102af0 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65eb553640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65e410d2a0 con 0x7f65e4101900 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65eb553640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65e410d5a0 con 0x7f65e41034b0 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e8aee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e4102af0 0x7f65e41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e92ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65e4101900 0x7f65e4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e8aee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e4102af0 0x7f65e41980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42936/0 (socket says 172.21.15.105:42936) 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e8aee640 1 -- 172.21.15.105:0/3485078586 learned_addr learned my addr 172.21.15.105:0/3485078586 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e9af0640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e41034b0 0x7f65e419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e9af0640 1 -- 172.21.15.105:0/3485078586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65e4101900 msgr2=0x7f65e4197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e9af0640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65e4101900 0x7f65e4197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e9af0640 1 -- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e4102af0 msgr2=0x7f65e41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e9af0640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e4102af0 0x7f65e41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.398+0000 7f65e9af0640 1 -- 172.21.15.105:0/3485078586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e419f7d0 con 0x7f65e41034b0 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.399+0000 7f65e8aee640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e4102af0 0x7f65e41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.399+0000 7f65e9af0640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e41034b0 0x7f65e419f160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00ec50 tx=0x7f65dc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.399+0000 7f65da7fc640 1 -- 172.21.15.105:0/3485078586 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc003b70 con 0x7f65e41034b0 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.399+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e419fac0 con 0x7f65e41034b0 2024-03-26T17:51:05.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.399+0000 7f65da7fc640 1 -- 172.21.15.105:0/3485078586 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65dc003d10 con 0x7f65e41034b0 2024-03-26T17:51:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.399+0000 7f65da7fc640 1 -- 172.21.15.105:0/3485078586 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc010760 con 0x7f65e41034b0 2024-03-26T17:51:05.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.399+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e41a0010 con 0x7f65e41034b0 2024-03-26T17:51:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.400+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e4108d90 con 0x7f65e41034b0 2024-03-26T17:51:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.401+0000 7f65da7fc640 1 -- 172.21.15.105:0/3485078586 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f65dc004b40 con 0x7f65e41034b0 2024-03-26T17:51:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.402+0000 7f65da7fc640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65d00a2f20 0x7f65d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.402+0000 7f65e92ef640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65d00a2f20 0x7f65d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.402+0000 7f65da7fc640 1 -- 172.21.15.105:0/3485078586 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f65dc0cf940 con 0x7f65e41034b0 2024-03-26T17:51:05.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.402+0000 7f65e92ef640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65d00a2f20 0x7f65d00a53e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005ab0 tx=0x7f65e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.410+0000 7f65da7fc640 1 -- 172.21.15.105:0/3485078586 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f65dc09c730 con 0x7f65e41034b0 2024-03-26T17:51:05.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.711+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65e402d240 con 0x7f65e41034b0 2024-03-26T17:51:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.712+0000 7f65da7fc640 1 -- 172.21.15.105:0/3485078586 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11941 (secure 0 0 0) 0x7f65dc09c030 con 0x7f65e41034b0 2024-03-26T17:51:05.705 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:51:05.705 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:49:14.826136+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24482,14628,24487,24466]},"id":1}]} 2024-03-26T17:51:05.705 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 32 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.714+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65d00a2f20 msgr2=0x7f65d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.714+0000 7f65eb553640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65d00a2f20 0x7f65d00a53e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f65e0005ab0 tx=0x7f65e0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.714+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e41034b0 msgr2=0x7f65e419f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.714+0000 7f65eb553640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e41034b0 0x7f65e419f160 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00ec50 tx=0x7f65dc00c6a0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.715+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 shutdown_connections 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.715+0000 7f65eb553640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65d00a2f20 0x7f65d00a53e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.715+0000 7f65eb553640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65e41034b0 0x7f65e419f160 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.715+0000 7f65eb553640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e4102af0 0x7f65e41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.715+0000 7f65eb553640 1 --2- 172.21.15.105:0/3485078586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f65e4101900 0x7f65e4197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.715+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 >> 172.21.15.105:0/3485078586 conn(0x7f65e40fce70 msgr2=0x7f65e4105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.715+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 shutdown_connections 2024-03-26T17:51:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.715+0000 7f65eb553640 1 -- 172.21.15.105:0/3485078586 wait complete. 2024-03-26T17:51:05.721 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:51:05.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 -- 172.21.15.105:0/179160271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe518104cd0 msgr2=0x7fe5181070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 --2- 172.21.15.105:0/179160271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe518104cd0 0x7fe5181070c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009900 tx=0x7fe50c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 -- 172.21.15.105:0/179160271 shutdown_connections 2024-03-26T17:51:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 --2- 172.21.15.105:0/179160271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe518107600 0x7fe5181099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 --2- 172.21.15.105:0/179160271 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe518104cd0 0x7fe5181070c0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 --2- 172.21.15.105:0/179160271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5181023a0 0x7fe518104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 -- 172.21.15.105:0/179160271 >> 172.21.15.105:0/179160271 conn(0x7fe5180fbd50 msgr2=0x7fe5180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 -- 172.21.15.105:0/179160271 shutdown_connections 2024-03-26T17:51:05.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.851+0000 7fe51e40b640 1 -- 172.21.15.105:0/179160271 wait complete. 2024-03-26T17:51:05.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51e40b640 1 Processor -- start 2024-03-26T17:51:05.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51e40b640 1 -- start start 2024-03-26T17:51:05.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51e40b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5181023a0 0x7fe518101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51e40b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe518104cd0 0x7fe518100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51e40b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe518107600 0x7fe518100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51e40b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe51810d3f0 con 0x7fe5181023a0 2024-03-26T17:51:05.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51e40b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe51810d270 con 0x7fe518104cd0 2024-03-26T17:51:05.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51e40b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe51810d570 con 0x7fe518107600 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe517fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5181023a0 0x7fe518101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe517fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5181023a0 0x7fe518101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42952/0 (socket says 172.21.15.105:42952) 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe5177fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe518104cd0 0x7fe518100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe517fff640 1 -- 172.21.15.105:0/1679963195 learned_addr learned my addr 172.21.15.105:0/1679963195 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.852+0000 7fe51c9a8640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe518107600 0x7fe518100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5177fe640 1 -- 172.21.15.105:0/1679963195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe518107600 msgr2=0x7fe518100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5177fe640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe518107600 0x7fe518100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5177fe640 1 -- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5181023a0 msgr2=0x7fe518101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5177fe640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5181023a0 0x7fe518101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5177fe640 1 -- 172.21.15.105:0/1679963195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe518101120 con 0x7fe518104cd0 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe51c9a8640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe518107600 0x7fe518100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe517fff640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5181023a0 0x7fe518101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5177fe640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe518104cd0 0x7fe518100350 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe50c036040 tx=0x7fe50c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5157fa640 1 -- 172.21.15.105:0/1679963195 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c043020 con 0x7fe518104cd0 2024-03-26T17:51:05.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe51e40b640 1 -- 172.21.15.105:0/1679963195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5181a1cd0 con 0x7fe518104cd0 2024-03-26T17:51:05.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5157fa640 1 -- 172.21.15.105:0/1679963195 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe50c004350 con 0x7fe518104cd0 2024-03-26T17:51:05.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe5157fa640 1 -- 172.21.15.105:0/1679963195 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c003ab0 con 0x7fe518104cd0 2024-03-26T17:51:05.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.853+0000 7fe51e40b640 1 -- 172.21.15.105:0/1679963195 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5181a2190 con 0x7fe518104cd0 2024-03-26T17:51:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe520ffb740 1 Processor -- start 2024-03-26T17:51:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe520ffb740 1 -- start start 2024-03-26T17:51:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645682ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x556456831f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 0x556456834a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe520ffb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556456791970 con 0x55645682d420 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe520ffb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556456739d70 con 0x55645682ca30 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe520ffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55645678d920 con 0x556456832590 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe517fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x556456831f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe517fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x556456831f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42962/0 (socket says 172.21.15.105:42962) 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe517fff640 1 -- 172.21.15.105:0/677358994 learned_addr learned my addr 172.21.15.105:0/677358994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe51c9a8640 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645682ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe51c9a8640 1 -- 172.21.15.105:0/677358994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 msgr2=0x556456834a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe51c9a8640 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 0x556456834a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe51c9a8640 1 -- 172.21.15.105:0/677358994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 msgr2=0x556456831f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe51c9a8640 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x556456831f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe51c9a8640 1 -- 172.21.15.105:0/677358994 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564564a53f0 con 0x55645682ca30 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe517fff640 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x556456831f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe5157fa640 1 -- 172.21.15.105:0/1679963195 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe50c046020 con 0x7fe518104cd0 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.856+0000 7fe51c9a8640 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645682ce50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe510005fd0 tx=0x7fe510011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9d4b5fd90b3de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.857+0000 7fe4fe7fc640 1 -- 172.21.15.105:0/677358994 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510004d20 con 0x55645682ca30 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.857+0000 7fe4fe7fc640 1 -- 172.21.15.105:0/677358994 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe51000f9a0 con 0x55645682ca30 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.857+0000 7fe4fe7fc640 1 -- 172.21.15.105:0/677358994 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510004400 con 0x55645682ca30 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.857+0000 7fe5157fa640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5000a2f20 0x7fe5000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.857+0000 7fe517fff640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5000a2f20 0x7fe5000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.857+0000 7fe5157fa640 1 -- 172.21.15.105:0/1679963195 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe50c0f1530 con 0x7fe518104cd0 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.857+0000 7fe520ffb740 1 -- 172.21.15.105:0/677358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 msgr2=0x55645682ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.857+0000 7fe520ffb740 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645682ce50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe510005fd0 tx=0x7fe510011bf0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 -- 172.21.15.105:0/677358994 shutdown_connections 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 0x556456834a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x556456831f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 --2- 172.21.15.105:0/677358994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645682ce50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 -- 172.21.15.105:0/677358994 >> 172.21.15.105:0/677358994 conn(0x556456827fa0 msgr2=0x5564568283a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 -- 172.21.15.105:0/677358994 shutdown_connections 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 -- 172.21.15.105:0/677358994 wait complete. 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe517fff640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5000a2f20 0x7fe5000a53e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x556456828f10 tx=0x7fe508005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 -- start start 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.858+0000 7fe520ffb740 1 Processor -- start 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682ca30 0x556456836fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x5564568374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556456832590 0x55645683e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe520ffb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556456835540 con 0x55645682d420 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe520ffb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55645674d310 con 0x556456832590 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe520ffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5564568356c0 con 0x55645682ca30 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe520ffb740 10 client.0 _ll_register_callbacks cb 0x556456788c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682ca30 0x556456836fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682ca30 0x556456836fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58106/0 (socket says 172.21.15.105:58106) 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe51c9a8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556456832590 0x55645683e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 -- 172.21.15.105:0/2989002893 learned_addr learned my addr 172.21.15.105:0/2989002893 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe5177fe640 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x5564568374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 -- 172.21.15.105:0/2989002893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556456832590 msgr2=0x55645683e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556456832590 0x55645683e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 -- 172.21.15.105:0/2989002893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 msgr2=0x5564568374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x5564568374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 -- 172.21.15.105:0/2989002893 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645656f7d0 con 0x55645682ca30 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe517fff640 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682ca30 0x556456836fb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe508009780 tx=0x7fe508004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb18663e11621506 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe4f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe4f77fe640 1 -- 172.21.15.105:0/2989002893 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50800fc60 con 0x55645682ca30 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe520ffb740 1 -- 172.21.15.105:0/2989002893 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51000f9a0 con 0x55645682ca30 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.859+0000 7fe4f77fe640 1 -- 172.21.15.105:0/2989002893 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe50800fe00 con 0x55645682ca30 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe4f77fe640 1 -- 172.21.15.105:0/2989002893 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe5080035f0 con 0x55645682ca30 2024-03-26T17:51:05.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe520ffb740 10 client.29896 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe520ffb740 1 -- 172.21.15.105:0/2989002893 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564568419c0 con 0x55645682ca30 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe4f77fe640 1 -- 172.21.15.105:0/2989002893 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe51000f9a0 con 0x55645682ca30 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe4f77fe640 1 -- 172.21.15.105:0/2989002893 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe508011a50 con 0x55645682ca30 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe520ffb740 10 client.29896 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe520ffb740 10 client.29896 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe520ffb740 20 client.29896 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe520ffb740 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x556456843420 0x5564568458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe520ffb740 4 client.29896 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe520ffb740 1 -- 172.21.15.105:0/2989002893 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55645679eeb0 con 0x556456843420 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.860+0000 7fe5177fe640 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x556456843420 0x5564568458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.861+0000 7fe4f77fe640 10 client.29896 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.861+0000 7fe5177fe640 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x556456843420 0x5564568458c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5e81e672ca7addc8 server_cookie=5b8f31ccbde1b0c6 in_seq=0 out_seq=0 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.861+0000 7fe4f77fe640 1 -- 172.21.15.105:0/2989002893 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55645679eeb0 con 0x556456843420 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.861+0000 7fe4f77fe640 10 client.29896 handle_command_reply: tid=0 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.861+0000 7fe520ffb740 1 client.29896 shutdown 2024-03-26T17:51:05.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 -- 172.21.15.105:0/2989002893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682ca30 msgr2=0x556456836fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682ca30 0x556456836fb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe508009780 tx=0x7fe508004730 comp rx=0 tx=0).stop 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 -- 172.21.15.105:0/2989002893 shutdown_connections 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x556456843420 0x5564568458c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556456832590 0x55645683e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x5564568374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe4f77fe640 0 client.29896 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 --2- 172.21.15.105:0/2989002893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682ca30 0x556456836fb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 -- 172.21.15.105:0/2989002893 >> 172.21.15.105:0/2989002893 conn(0x556456827fa0 msgr2=0x5564568271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 -- 172.21.15.105:0/2989002893 shutdown_connections 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 1 -- 172.21.15.105:0/2989002893 wait complete. 2024-03-26T17:51:05.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.862+0000 7fe520ffb740 20 client.29896 trim_cache size 0 max 16384 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe520ffb740 1 Processor -- start 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe520ffb740 1 -- start start 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x5564568372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x55645683e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 0x556456826930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe520ffb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181aaf40 con 0x55645682d420 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe520ffb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181aca80 con 0x55645682ca30 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe520ffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181ab610 con 0x556456832590 2024-03-26T17:51:05.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe5177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x55645683e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe517fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x5564568372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe5177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x55645683e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42986/0 (socket says 172.21.15.105:42986) 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe5177fe640 1 -- 172.21.15.105:0/3737341947 learned_addr learned my addr 172.21.15.105:0/3737341947 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.878+0000 7fe51c9a8640 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 0x556456826930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe5177fe640 1 -- 172.21.15.105:0/3737341947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 msgr2=0x556456826930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe5177fe640 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 0x556456826930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe5177fe640 1 -- 172.21.15.105:0/3737341947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 msgr2=0x5564568372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe5177fe640 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x5564568372d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe5177fe640 1 -- 172.21.15.105:0/3737341947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564568417e0 con 0x55645682d420 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe517fff640 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x5564568372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe5177fe640 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x55645683e4f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0bd920 tx=0x7fe50c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ace27b19de8a123 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe4fdffb640 1 -- 172.21.15.105:0/3737341947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0c1d70 con 0x55645682d420 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe4fdffb640 1 -- 172.21.15.105:0/3737341947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe50c0c6630 con 0x55645682d420 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.879+0000 7fe4fdffb640 1 -- 172.21.15.105:0/3737341947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c005420 con 0x55645682d420 2024-03-26T17:51:05.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 -- 172.21.15.105:0/3737341947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 msgr2=0x55645683e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x55645683e4f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0bd920 tx=0x7fe50c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 -- 172.21.15.105:0/3737341947 shutdown_connections 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556456832590 0x556456826930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55645682d420 0x55645683e4f0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3737341947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x5564568372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 -- 172.21.15.105:0/3737341947 >> 172.21.15.105:0/3737341947 conn(0x556456827fa0 msgr2=0x556456836bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 -- 172.21.15.105:0/3737341947 shutdown_connections 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 -- 172.21.15.105:0/3737341947 wait complete. 2024-03-26T17:51:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 -- start start 2024-03-26T17:51:05.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.880+0000 7fe520ffb740 1 Processor -- start 2024-03-26T17:51:05.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645683b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682d420 0x55645683be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe520ffb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556456832590 0x55645683ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe520ffb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181ac3b0 con 0x556456832590 2024-03-26T17:51:05.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe520ffb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181abce0 con 0x55645682ca30 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe520ffb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5181a8f20 con 0x55645682d420 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe520ffb740 10 client.0 _ll_register_callbacks cb 0x7fe508012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe517fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645683b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe51c9a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556456832590 0x55645683ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe5177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682d420 0x55645683be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe517fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645683b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60434/0 (socket says 172.21.15.105:60434) 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe517fff640 1 -- 172.21.15.105:0/3708082662 learned_addr learned my addr 172.21.15.105:0/3708082662 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe5177fe640 1 -- 172.21.15.105:0/3708082662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 msgr2=0x55645683b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe5177fe640 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645683b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe5177fe640 1 -- 172.21.15.105:0/3708082662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556456832590 msgr2=0x55645683ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe5177fe640 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556456832590 0x55645683ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe5177fe640 1 -- 172.21.15.105:0/3708082662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55645683d410 con 0x55645682d420 2024-03-26T17:51:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe517fff640 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645683b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe51c9a8640 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556456832590 0x55645683ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe5177fe640 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682d420 0x55645683be10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0bd920 tx=0x7fe50c0c1de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f73f970702b0def server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe4f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.881+0000 7fe4f77fe640 1 -- 172.21.15.105:0/3708082662 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0da070 con 0x55645682d420 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 1 -- 172.21.15.105:0/3708082662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50c0c6630 con 0x55645682d420 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe4f77fe640 1 -- 172.21.15.105:0/3708082662 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe50c0f1310 con 0x55645682d420 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe4f77fe640 1 -- 172.21.15.105:0/3708082662 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe50c003e60 con 0x55645682d420 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 10 client.29902 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 1 -- 172.21.15.105:0/3708082662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55645683e710 con 0x55645682d420 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe4f77fe640 1 -- 172.21.15.105:0/3708082662 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe50c0c6630 con 0x55645682d420 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe4f77fe640 1 -- 172.21.15.105:0/3708082662 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe50c0d4070 con 0x55645682d420 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 10 client.29902 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 10 client.29902 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:51:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 20 client.29902 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5564568a9e00 0x5564568ac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 4 client.29902 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe520ffb740 1 -- 172.21.15.105:0/3708082662 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556456799200 con 0x5564568a9e00 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.882+0000 7fe517fff640 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5564568a9e00 0x5564568ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe4f77fe640 10 client.29902 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe517fff640 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5564568a9e00 0x5564568ac220 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ade6bb5cd6957f74 server_cookie=1200142ebe5a36b4 in_seq=0 out_seq=0 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe4f77fe640 1 -- 172.21.15.105:0/3708082662 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556456799200 con 0x5564568a9e00 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe4f77fe640 10 client.29902 handle_command_reply: tid=0 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 client.29902 shutdown 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 -- 172.21.15.105:0/3708082662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682d420 msgr2=0x55645683be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682d420 0x55645683be10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0bd920 tx=0x7fe50c0c1de0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 -- 172.21.15.105:0/3708082662 shutdown_connections 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5564568a9e00 0x5564568ac220 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556456832590 0x55645683ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe4f77fe640 0 client.29902 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:51:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55645682d420 0x55645683be10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 --2- 172.21.15.105:0/3708082662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55645682ca30 0x55645683b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 -- 172.21.15.105:0/3708082662 >> 172.21.15.105:0/3708082662 conn(0x556456827fa0 msgr2=0x55645682b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 -- 172.21.15.105:0/3708082662 shutdown_connections 2024-03-26T17:51:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.883+0000 7fe520ffb740 1 -- 172.21.15.105:0/3708082662 wait complete. 2024-03-26T17:51:05.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe520ffb740 20 client.29902 trim_cache size 0 max 16384 2024-03-26T17:51:05.882 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:05.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 -- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5000a2f20 msgr2=0x7fe5000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5000a2f20 0x7fe5000a53e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x556456828f10 tx=0x7fe508005f90 comp rx=0 tx=0).stop 2024-03-26T17:51:05.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 -- 172.21.15.105:0/1679963195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe518104cd0 msgr2=0x7fe518100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:05.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe518104cd0 0x7fe518100350 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe50c036040 tx=0x7fe50c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 -- 172.21.15.105:0/1679963195 shutdown_connections 2024-03-26T17:51:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe5000a2f20 0x7fe5000a53e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe518107600 0x7fe518100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe518104cd0 0x7fe518100350 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 --2- 172.21.15.105:0/1679963195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5181023a0 0x7fe518101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 -- 172.21.15.105:0/1679963195 >> 172.21.15.105:0/1679963195 conn(0x7fe5180fbd50 msgr2=0x7fe518104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 -- 172.21.15.105:0/1679963195 shutdown_connections 2024-03-26T17:51:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:05.884+0000 7fe51e40b640 1 -- 172.21.15.105:0/1679963195 wait complete. 2024-03-26T17:51:05.888 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:51:06.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/4067964690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e08102370 msgr2=0x7f0e08108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/4067964690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e08102370 0x7f0e08108c00 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009ca0 tx=0x7f0e0002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/4067964690 shutdown_connections 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/4067964690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e08102370 0x7f0e08108c00 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/4067964690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e081019b0 0x7f0e08101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/4067964690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e081007c0 0x7f0e08100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/4067964690 >> 172.21.15.105:0/4067964690 conn(0x7f0e080fbd50 msgr2=0x7f0e080fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/4067964690 shutdown_connections 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.018+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/4067964690 wait complete. 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0fc7e640 1 Processor -- start 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0fc7e640 1 -- start start 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0fc7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e081007c0 0x7f0e081969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0fc7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e081019b0 0x7f0e08196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0fc7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e08102370 0x7f0e0819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0fc7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0810c2f0 con 0x7f0e08102370 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0fc7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0810c170 con 0x7f0e081007c0 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0fc7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0810c470 con 0x7f0e081019b0 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0e21b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e08102370 0x7f0e0819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0d219640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e081019b0 0x7f0e08196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0da1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e081007c0 0x7f0e081969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0d219640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e081019b0 0x7f0e08196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58134/0 (socket says 172.21.15.105:58134) 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0d219640 1 -- 172.21.15.105:0/2048291210 learned_addr learned my addr 172.21.15.105:0/2048291210 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.019+0000 7f0e0da1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e081007c0 0x7f0e081969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60462/0 (socket says 172.21.15.105:60462) 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0e0d219640 1 -- 172.21.15.105:0/2048291210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e081007c0 msgr2=0x7f0e081969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0e0d219640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e081007c0 0x7f0e081969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0e0d219640 1 -- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e08102370 msgr2=0x7f0e0819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0e0d219640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e08102370 0x7f0e0819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0e0d219640 1 -- 172.21.15.105:0/2048291210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0819e5e0 con 0x7f0e081019b0 2024-03-26T17:51:06.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0e0d219640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e081019b0 0x7f0e08196f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0df800e9f0 tx=0x7f0df800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0dfeffd640 1 -- 172.21.15.105:0/2048291210 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df800cd80 con 0x7f0e081019b0 2024-03-26T17:51:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0dfeffd640 1 -- 172.21.15.105:0/2048291210 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0df80054d0 con 0x7f0e081019b0 2024-03-26T17:51:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/2048291210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e08068a80 con 0x7f0e081019b0 2024-03-26T17:51:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0dfeffd640 1 -- 172.21.15.105:0/2048291210 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8010490 con 0x7f0e081019b0 2024-03-26T17:51:06.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.020+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/2048291210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e08068f80 con 0x7f0e081019b0 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.023+0000 7f0e12865740 1 Processor -- start 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.023+0000 7f0e12865740 1 -- start start 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.023+0000 7f0e12865740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c88e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.023+0000 7f0e12865740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c89420 0x55e152c8df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.023+0000 7f0e12865740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c8e590 0x55e152c90a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.023+0000 7f0e12865740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e152bed970 con 0x55e152c8e590 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e12865740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e152b95d70 con 0x55e152c89420 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e12865740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e152be9920 con 0x55e152c88a30 2024-03-26T17:51:06.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0dfeffd640 1 -- 172.21.15.105:0/2048291210 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0df8005000 con 0x7f0e081019b0 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0da1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c89420 0x55e152c8df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0e21b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c88e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0da1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c89420 0x55e152c8df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60466/0 (socket says 172.21.15.105:60466) 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0da1a640 1 -- 172.21.15.105:0/439234651 learned_addr learned my addr 172.21.15.105:0/439234651 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0d219640 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c8e590 0x55e152c90a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0e21b640 1 -- 172.21.15.105:0/439234651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c89420 msgr2=0x55e152c8df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0e21b640 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c89420 0x55e152c8df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0e21b640 1 -- 172.21.15.105:0/439234651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c8e590 msgr2=0x55e152c90a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0e21b640 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c8e590 0x55e152c90a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0e21b640 1 -- 172.21.15.105:0/439234651 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1529013f0 con 0x55e152c88a30 2024-03-26T17:51:06.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c89420 0x55e152c8df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0e0e21b640 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c88e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0e0002c630 tx=0x7f0e00005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a244b72813250f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.024+0000 7f0dfeffd640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0df40a2f20 0x7f0df40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.025+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0df40a2f20 0x7f0df40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.025+0000 7f0df3fff640 1 -- 172.21.15.105:0/439234651 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00043070 con 0x55e152c88a30 2024-03-26T17:51:06.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.025+0000 7f0dfeffd640 1 -- 172.21.15.105:0/2048291210 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0df80cf790 con 0x7f0e081019b0 2024-03-26T17:51:06.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.025+0000 7f0df3fff640 1 -- 172.21.15.105:0/439234651 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e00032db0 con 0x55e152c88a30 2024-03-26T17:51:06.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.025+0000 7f0df3fff640 1 -- 172.21.15.105:0/439234651 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00039040 con 0x55e152c88a30 2024-03-26T17:51:06.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.026+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0df40a2f20 0x7f0df40a53e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55e152c89820 tx=0x7f0e04010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.026+0000 7f0e12865740 1 -- 172.21.15.105:0/439234651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 msgr2=0x55e152c88e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.026+0000 7f0e12865740 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c88e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0e0002c630 tx=0x7f0e00005fa0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.026+0000 7f0e12865740 1 -- 172.21.15.105:0/439234651 shutdown_connections 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.026+0000 7f0e12865740 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c8e590 0x55e152c90a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.026+0000 7f0e12865740 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c89420 0x55e152c8df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.026+0000 7f0e12865740 1 --2- 172.21.15.105:0/439234651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c88e50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.026+0000 7f0e12865740 1 -- 172.21.15.105:0/439234651 >> 172.21.15.105:0/439234651 conn(0x55e152c83fa0 msgr2=0x55e152c843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.027+0000 7f0e12865740 1 -- 172.21.15.105:0/439234651 shutdown_connections 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.027+0000 7f0e12865740 1 -- 172.21.15.105:0/439234651 wait complete. 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.027+0000 7f0e12865740 1 -- start start 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.027+0000 7f0e12865740 1 Processor -- start 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e12865740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c92f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e12865740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e12865740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e12865740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e152c914e0 con 0x55e152c88a30 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e12865740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e152ba9310 con 0x55e152c8e590 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e12865740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e152c91660 con 0x55e152c89420 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e12865740 10 client.0 _ll_register_callbacks cb 0x55e152be4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0da1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c92f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0da1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c92f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43020/0 (socket says 172.21.15.105:43020) 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0da1a640 1 -- 172.21.15.105:0/402932529 learned_addr learned my addr 172.21.15.105:0/402932529 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0e21b640 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c9a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0d219640 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0da1a640 1 -- 172.21.15.105:0/402932529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 msgr2=0x55e152c934b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c934b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0da1a640 1 -- 172.21.15.105:0/402932529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 msgr2=0x55e152c9a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c9a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.028+0000 7f0e0da1a640 1 -- 172.21.15.105:0/402932529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1529cb7d0 con 0x55e152c88a30 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c92f70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f0e040038a0 tx=0x7f0e04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175335d7529b305f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0df17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0df17fa640 1 -- 172.21.15.105:0/402932529 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04029060 con 0x55e152c88a30 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0df17fa640 1 -- 172.21.15.105:0/402932529 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0e0401a8f0 con 0x55e152c88a30 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0e12865740 1 -- 172.21.15.105:0/402932529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e00032db0 con 0x55e152c88a30 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0df17fa640 1 -- 172.21.15.105:0/402932529 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e04020040 con 0x55e152c88a30 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0e12865740 10 client.20130 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0e12865740 1 -- 172.21.15.105:0/402932529 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e152c9d930 con 0x55e152c88a30 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0df17fa640 1 -- 172.21.15.105:0/402932529 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e00032db0 con 0x55e152c88a30 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0df17fa640 1 -- 172.21.15.105:0/402932529 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0e0402f850 con 0x55e152c88a30 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0e12865740 10 client.20130 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:06.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0e12865740 10 client.20130 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0e12865740 20 client.20130 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.029+0000 7f0e12865740 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e152c9ef50 0x55e152ca13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.030+0000 7f0e12865740 4 client.20130 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.030+0000 7f0e12865740 1 -- 172.21.15.105:0/402932529 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e152bfaeb0 con 0x55e152c9ef50 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.030+0000 7f0e0d219640 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e152c9ef50 0x55e152ca13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.030+0000 7f0df17fa640 10 client.20130 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.030+0000 7f0e0d219640 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e152c9ef50 0x55e152ca13f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88bc7c867a372a2e server_cookie=2d19af6b2e31277d in_seq=0 out_seq=0 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0df17fa640 1 -- 172.21.15.105:0/402932529 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e152bfaeb0 con 0x55e152c9ef50 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0df17fa640 10 client.20130 handle_command_reply: tid=0 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 client.20130 shutdown 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 -- 172.21.15.105:0/402932529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 msgr2=0x55e152c92f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c92f70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f0e040038a0 tx=0x7f0e04004730 comp rx=0 tx=0).stop 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 -- 172.21.15.105:0/402932529 shutdown_connections 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e152c9ef50 0x55e152ca13f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c9a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0df17fa640 0 client.20130 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:51:06.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c934b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 --2- 172.21.15.105:0/402932529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c92f70 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 -- 172.21.15.105:0/402932529 >> 172.21.15.105:0/402932529 conn(0x55e152c83fa0 msgr2=0x55e152c831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 -- 172.21.15.105:0/402932529 shutdown_connections 2024-03-26T17:51:06.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 1 -- 172.21.15.105:0/402932529 wait complete. 2024-03-26T17:51:06.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.031+0000 7f0e12865740 20 client.20130 trim_cache size 0 max 16384 2024-03-26T17:51:06.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 415254 3.15 MB/sec execute 589 sec latency 15.388 ms 2024-03-26T17:51:06.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e12865740 1 Processor -- start 2024-03-26T17:51:06.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e12865740 1 -- start start 2024-03-26T17:51:06.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e12865740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c7f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e12865740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c9a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e12865740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c93290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e12865740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081aaf00 con 0x55e152c88a30 2024-03-26T17:51:06.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e12865740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081aca40 con 0x55e152c8e590 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e12865740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081ab5d0 con 0x55e152c89420 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e0da1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c7f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e0da1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c7f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43030/0 (socket says 172.21.15.105:43030) 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e0da1a640 1 -- 172.21.15.105:0/3907196305 learned_addr learned my addr 172.21.15.105:0/3907196305 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e0d219640 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c9a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.046+0000 7f0e0e21b640 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c93290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0e0da1a640 1 -- 172.21.15.105:0/3907196305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 msgr2=0x55e152c9a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c9a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0e0da1a640 1 -- 172.21.15.105:0/3907196305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 msgr2=0x55e152c93290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c93290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0e0da1a640 1 -- 172.21.15.105:0/3907196305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e152c8dc80 con 0x55e152c88a30 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0e0d219640 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c9a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c7f050 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0e040058c0 tx=0x7f0e040096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d07744874aa7ef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0df37fe640 1 -- 172.21.15.105:0/3907196305 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0403b020 con 0x55e152c88a30 2024-03-26T17:51:06.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0df37fe640 1 -- 172.21.15.105:0/3907196305 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e04009ca0 con 0x55e152c88a30 2024-03-26T17:51:06.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.047+0000 7f0df37fe640 1 -- 172.21.15.105:0/3907196305 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0401aa20 con 0x55e152c88a30 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 -- 172.21.15.105:0/3907196305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 msgr2=0x55e152c7f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c7f050 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0e040058c0 tx=0x7f0e040096e0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 -- 172.21.15.105:0/3907196305 shutdown_connections 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c93290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c89420 0x55e152c9a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 --2- 172.21.15.105:0/3907196305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c88a30 0x55e152c7f050 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 -- 172.21.15.105:0/3907196305 >> 172.21.15.105:0/3907196305 conn(0x55e152c83fa0 msgr2=0x55e152c92b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 -- 172.21.15.105:0/3907196305 shutdown_connections 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 -- 172.21.15.105:0/3907196305 wait complete. 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 -- start start 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.048+0000 7f0e12865740 1 Processor -- start 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e12865740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c97b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e12865740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c89420 0x55e152c98060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e12865740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c99020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e12865740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081ac370 con 0x55e152c89420 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e12865740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081abca0 con 0x55e152c8e590 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e12865740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e081a8ee0 con 0x55e152c88a30 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e12865740 10 client.0 _ll_register_callbacks cb 0x7f0e0401d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0d219640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c89420 0x55e152c98060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0d219640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c89420 0x55e152c98060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43046/0 (socket says 172.21.15.105:43046) 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0d219640 1 -- 172.21.15.105:0/3499391316 learned_addr learned my addr 172.21.15.105:0/3499391316 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c97b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0e21b640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c99020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0da1a640 1 -- 172.21.15.105:0/3499391316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 msgr2=0x55e152c99020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c99020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0da1a640 1 -- 172.21.15.105:0/3499391316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c89420 msgr2=0x55e152c98060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c89420 0x55e152c98060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0da1a640 1 -- 172.21.15.105:0/3499391316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e152c99610 con 0x55e152c88a30 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0d219640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c89420 0x55e152c98060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0e21b640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c99020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0e0da1a640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c97b20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0e040038a0 tx=0x7f0e04009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd0c159c54d0c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0df17fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.049+0000 7f0df17fa640 1 -- 172.21.15.105:0/3499391316 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04044070 con 0x55e152c88a30 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0df17fa640 1 -- 172.21.15.105:0/3499391316 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0e04005460 con 0x55e152c88a30 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 1 -- 172.21.15.105:0/3499391316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e04009ca0 con 0x55e152c88a30 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0df17fa640 1 -- 172.21.15.105:0/3499391316 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e04029020 con 0x55e152c88a30 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 10 client.29920 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 1 -- 172.21.15.105:0/3499391316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e152d052f0 con 0x55e152c88a30 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0df17fa640 1 -- 172.21.15.105:0/3499391316 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e04009ca0 con 0x55e152c88a30 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0df17fa640 1 -- 172.21.15.105:0/3499391316 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0e040048a0 con 0x55e152c88a30 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 10 client.29920 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:06.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 10 client.29920 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 20 client.29920 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e152d05b50 0x55e152d07f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 4 client.29920 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e12865740 1 -- 172.21.15.105:0/3499391316 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e152bf5200 con 0x55e152d05b50 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.050+0000 7f0e0d219640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e152d05b50 0x55e152d07f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0df17fa640 10 client.29920 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e0d219640 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e152d05b50 0x55e152d07f70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c282d0352a67e1b server_cookie=c4ea3db6c94d94ba in_seq=0 out_seq=0 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0df17fa640 1 -- 172.21.15.105:0/3499391316 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e152bf5200 con 0x55e152d05b50 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0df17fa640 10 client.29920 handle_command_reply: tid=0 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 client.29920 shutdown 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 -- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 msgr2=0x55e152c97b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c97b20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0e040038a0 tx=0x7f0e04009e50 comp rx=0 tx=0).stop 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 -- 172.21.15.105:0/3499391316 shutdown_connections 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e152d05b50 0x55e152d07f70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e152c8e590 0x55e152c99020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e152c89420 0x55e152c98060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0df17fa640 0 client.29920 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:51:06.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 --2- 172.21.15.105:0/3499391316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e152c88a30 0x55e152c97b20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 -- 172.21.15.105:0/3499391316 >> 172.21.15.105:0/3499391316 conn(0x55e152c83fa0 msgr2=0x55e152c87200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 -- 172.21.15.105:0/3499391316 shutdown_connections 2024-03-26T17:51:06.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 1 -- 172.21.15.105:0/3499391316 wait complete. 2024-03-26T17:51:06.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.051+0000 7f0e12865740 20 client.29920 trim_cache size 0 max 16384 2024-03-26T17:51:06.049 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0df40a2f20 msgr2=0x7f0df40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0df40a2f20 0x7f0df40a53e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55e152c89820 tx=0x7f0e04010040 comp rx=0 tx=0).stop 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/2048291210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e081019b0 msgr2=0x7f0e08196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e081019b0 0x7f0e08196f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0df800e9f0 tx=0x7f0df800edf0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/2048291210 shutdown_connections 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0df40a2f20 0x7f0df40a53e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e08102370 0x7f0e0819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e081019b0 0x7f0e08196f00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 --2- 172.21.15.105:0/2048291210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0e081007c0 0x7f0e081969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/2048291210 >> 172.21.15.105:0/2048291210 conn(0x7f0e080fbd50 msgr2=0x7f0e08104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/2048291210 shutdown_connections 2024-03-26T17:51:06.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.052+0000 7f0e0fc7e640 1 -- 172.21.15.105:0/2048291210 wait complete. 2024-03-26T17:51:06.053 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:05 smithi105 ceph-mon[32280]: pgmap v731: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-03-26T17:51:06.053 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:05 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3485078586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:51:06.054 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:51:06.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:05 smithi144 ceph-mon[36095]: pgmap v731: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-03-26T17:51:06.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:05 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3485078586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:51:06.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:05 smithi135 ceph-mon[35636]: pgmap v731: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.5 MiB/s wr, 101 op/s 2024-03-26T17:51:06.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:05 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3485078586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1323460596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81019b0 msgr2=0x7fb6d8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1323460596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81019b0 0x7fb6d8101e30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009900 tx=0x7fb6c802bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1323460596 shutdown_connections 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1323460596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6d8102370 0x7fb6d8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1323460596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81019b0 0x7fb6d8101e30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1323460596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d81007c0 0x7fb6d8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1323460596 >> 172.21.15.105:0/1323460596 conn(0x7fb6d80fbd50 msgr2=0x7fb6d80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1323460596 shutdown_connections 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1323460596 wait complete. 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 Processor -- start 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 -- start start 2024-03-26T17:51:06.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81007c0 0x7fb6d8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d81019b0 0x7fb6d81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.183+0000 7fb6dfa6a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6d8102370 0x7fb6d819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dfa6a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d810c2f0 con 0x7fb6d81019b0 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dfa6a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d810c170 con 0x7fb6d8102370 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dfa6a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d810c470 con 0x7fb6d81007c0 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd005640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d81019b0 0x7fb6d81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd806640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81007c0 0x7fb6d8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd005640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d81019b0 0x7fb6d81980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43060/0 (socket says 172.21.15.105:43060) 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd806640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81007c0 0x7fb6d8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58180/0 (socket says 172.21.15.105:58180) 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd005640 1 -- 172.21.15.105:0/1019058723 learned_addr learned my addr 172.21.15.105:0/1019058723 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6de007640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6d8102370 0x7fb6d819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd806640 1 -- 172.21.15.105:0/1019058723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6d8102370 msgr2=0x7fb6d819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd806640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6d8102370 0x7fb6d819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd806640 1 -- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d81019b0 msgr2=0x7fb6d81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd806640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d81019b0 0x7fb6d81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd806640 1 -- 172.21.15.105:0/1019058723 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d819f750 con 0x7fb6d81007c0 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd005640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d81019b0 0x7fb6d81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dd806640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81007c0 0x7fb6d8197b80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400e9c0 tx=0x7fb6d400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6ceffd640 1 -- 172.21.15.105:0/1019058723 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400cd80 con 0x7fb6d81007c0 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6ceffd640 1 -- 172.21.15.105:0/1019058723 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d40108e0 con 0x7fb6d81007c0 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1019058723 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d819fa40 con 0x7fb6d81007c0 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.184+0000 7fb6ceffd640 1 -- 172.21.15.105:0/1019058723 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4018860 con 0x7fb6d81007c0 2024-03-26T17:51:06.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.185+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1019058723 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d819ff90 con 0x7fb6d81007c0 2024-03-26T17:51:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.187+0000 7fb6ceffd640 1 -- 172.21.15.105:0/1019058723 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb6d401f020 con 0x7fb6d81007c0 2024-03-26T17:51:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6e264d740 1 Processor -- start 2024-03-26T17:51:06.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6e264d740 1 -- start start 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6ceffd640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6c40a2f20 0x7fb6c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x55930756c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x559307571330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 0x559307573e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6e264d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5593074cc970 con 0x55930756c830 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6e264d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559307474d70 con 0x55930756be40 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6e264d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5593074c8920 con 0x5593075719a0 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6dd005640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x559307571330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6de007640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6c40a2f20 0x7fb6c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.188+0000 7fb6ceffd640 1 -- 172.21.15.105:0/1019058723 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb6d40cf400 con 0x7fb6d81007c0 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd806640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x55930756c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd005640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x559307571330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43074/0 (socket says 172.21.15.105:43074) 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd005640 1 -- 172.21.15.105:0/3063387118 learned_addr learned my addr 172.21.15.105:0/3063387118 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6de007640 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 0x559307573e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd005640 1 -- 172.21.15.105:0/3063387118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 msgr2=0x559307573e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd005640 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 0x559307573e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd005640 1 -- 172.21.15.105:0/3063387118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 msgr2=0x55930756c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd005640 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x55930756c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd005640 1 -- 172.21.15.105:0/3063387118 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593071e03f0 con 0x55930756c830 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd806640 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x55930756c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6de007640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6c40a2f20 0x7fb6c40a53e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005fd0 tx=0x7fb6d0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6dd005640 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x559307571330 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8036040 tx=0x7fb6c8009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90200c048f6c262e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6c3fff640 1 -- 172.21.15.105:0/3063387118 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8045020 con 0x55930756c830 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6c3fff640 1 -- 172.21.15.105:0/3063387118 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6c802cc90 con 0x55930756c830 2024-03-26T17:51:06.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.189+0000 7fb6c3fff640 1 -- 172.21.15.105:0/3063387118 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8004900 con 0x55930756c830 2024-03-26T17:51:06.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.190+0000 7fb6e264d740 1 -- 172.21.15.105:0/3063387118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 msgr2=0x559307571330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.190+0000 7fb6e264d740 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x559307571330 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8036040 tx=0x7fb6c8009670 comp rx=0 tx=0).stop 2024-03-26T17:51:06.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.190+0000 7fb6e264d740 1 -- 172.21.15.105:0/3063387118 shutdown_connections 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.190+0000 7fb6e264d740 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 0x559307573e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.190+0000 7fb6e264d740 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x559307571330 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.190+0000 7fb6e264d740 1 --2- 172.21.15.105:0/3063387118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x55930756c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.190+0000 7fb6e264d740 1 -- 172.21.15.105:0/3063387118 >> 172.21.15.105:0/3063387118 conn(0x559307562fa0 msgr2=0x5593075633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.190+0000 7fb6e264d740 1 -- 172.21.15.105:0/3063387118 shutdown_connections 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 -- 172.21.15.105:0/3063387118 wait complete. 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 -- start start 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 Processor -- start 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x559307576290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756c830 0x5593075767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593075719a0 0x55930757d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5593075748f0 con 0x5593075719a0 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559307488310 con 0x55930756be40 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559307574a70 con 0x55930756c830 2024-03-26T17:51:06.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6e264d740 10 client.0 _ll_register_callbacks cb 0x5593074c3c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.191+0000 7fb6de007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593075719a0 0x55930757d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x559307576290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd005640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756c830 0x5593075767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x559307576290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60520/0 (socket says 172.21.15.105:60520) 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6de007640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593075719a0 0x55930757d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43078/0 (socket says 172.21.15.105:43078) 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 -- 172.21.15.105:0/509469900 learned_addr learned my addr 172.21.15.105:0/509469900 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 -- 172.21.15.105:0/509469900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756c830 msgr2=0x5593075767d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756c830 0x5593075767d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 -- 172.21.15.105:0/509469900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593075719a0 msgr2=0x55930757d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593075719a0 0x55930757d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 -- 172.21.15.105:0/509469900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593072aa7d0 con 0x55930756be40 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd005640 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756c830 0x5593075767d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6de007640 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593075719a0 0x55930757d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6dd806640 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x559307576290 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4097f50 tx=0x7fb6d40aec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00d32188476bc8a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6c17fa640 1 -- 172.21.15.105:0/509469900 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4010e20 con 0x55930756be40 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6c17fa640 1 -- 172.21.15.105:0/509469900 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb6d4010a90 con 0x55930756be40 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6e264d740 1 -- 172.21.15.105:0/509469900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c802cc90 con 0x55930756be40 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6c17fa640 1 -- 172.21.15.105:0/509469900 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb6d409c1f0 con 0x55930756be40 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6e264d740 10 client.29948 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.192+0000 7fb6e264d740 1 -- 172.21.15.105:0/509469900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559307580bf0 con 0x55930756be40 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6c17fa640 1 -- 172.21.15.105:0/509469900 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6c802cc90 con 0x55930756be40 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6c17fa640 1 -- 172.21.15.105:0/509469900 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb6d40af080 con 0x55930756be40 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6e264d740 10 client.29948 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6e264d740 10 client.29948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6e264d740 20 client.29948 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6e264d740 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559307582260 0x5593075846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6e264d740 4 client.29948 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6e264d740 1 -- 172.21.15.105:0/509469900 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593074d9eb0 con 0x559307582260 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6dd005640 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559307582260 0x5593075846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6c17fa640 10 client.29948 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.193+0000 7fb6dd005640 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559307582260 0x5593075846d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b06f0b6fd30efed server_cookie=e3e2e75ffc00bda4 in_seq=0 out_seq=0 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.194+0000 7fb6c17fa640 1 -- 172.21.15.105:0/509469900 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5593074d9eb0 con 0x559307582260 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.194+0000 7fb6c17fa640 10 client.29948 handle_command_reply: tid=0 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.194+0000 7fb6e264d740 1 client.29948 shutdown 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 -- 172.21.15.105:0/509469900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 msgr2=0x559307576290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x559307576290 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4097f50 tx=0x7fb6d40aec10 comp rx=0 tx=0).stop 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 -- 172.21.15.105:0/509469900 shutdown_connections 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5593075719a0 0x55930757d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756c830 0x5593075767d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559307582260 0x5593075846d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6c17fa640 0 client.29948 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:51:06.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 --2- 172.21.15.105:0/509469900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756be40 0x559307576290 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 -- 172.21.15.105:0/509469900 >> 172.21.15.105:0/509469900 conn(0x559307562fa0 msgr2=0x5593075621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 -- 172.21.15.105:0/509469900 shutdown_connections 2024-03-26T17:51:06.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 1 -- 172.21.15.105:0/509469900 wait complete. 2024-03-26T17:51:06.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.195+0000 7fb6e264d740 20 client.29948 trim_cache size 0 max 16384 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.211+0000 7fb6e264d740 1 Processor -- start 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.211+0000 7fb6e264d740 1 -- start start 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.211+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756be40 0x559307561930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.211+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x55930757d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593075719a0 0x559307561e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6e264d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81a6b10 con 0x55930756c830 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6e264d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81a8650 con 0x5593075719a0 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6e264d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81a71e0 con 0x55930756be40 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd005640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x55930757d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd005640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x55930757d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43082/0 (socket says 172.21.15.105:43082) 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd806640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756be40 0x559307561930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd005640 1 -- 172.21.15.105:0/2443242526 learned_addr learned my addr 172.21.15.105:0/2443242526 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6de007640 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593075719a0 0x559307561e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd806640 1 -- 172.21.15.105:0/2443242526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593075719a0 msgr2=0x559307561e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd806640 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593075719a0 0x559307561e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd806640 1 -- 172.21.15.105:0/2443242526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 msgr2=0x55930757d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd806640 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x55930757d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd806640 1 -- 172.21.15.105:0/2443242526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559307562db0 con 0x55930756be40 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd005640 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x55930757d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6dd806640 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756be40 0x559307561930 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb6d409fe00 tx=0x7fb6d409b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ca8e88097e1879 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6c37fe640 1 -- 172.21.15.105:0/2443242526 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40a42f0 con 0x55930756be40 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6c37fe640 1 -- 172.21.15.105:0/2443242526 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6d4010e20 con 0x55930756be40 2024-03-26T17:51:06.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.212+0000 7fb6c37fe640 1 -- 172.21.15.105:0/2443242526 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4010a90 con 0x55930756be40 2024-03-26T17:51:06.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.213+0000 7fb6e264d740 1 -- 172.21.15.105:0/2443242526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756be40 msgr2=0x559307561930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.213+0000 7fb6e264d740 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756be40 0x559307561930 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb6d409fe00 tx=0x7fb6d409b9f0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.213+0000 7fb6e264d740 1 -- 172.21.15.105:0/2443242526 shutdown_connections 2024-03-26T17:51:06.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.213+0000 7fb6e264d740 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5593075719a0 0x559307561e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.213+0000 7fb6e264d740 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756c830 0x55930757d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.213+0000 7fb6e264d740 1 --2- 172.21.15.105:0/2443242526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55930756be40 0x559307561930 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.213+0000 7fb6e264d740 1 -- 172.21.15.105:0/2443242526 >> 172.21.15.105:0/2443242526 conn(0x559307562fa0 msgr2=0x559307575ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.213+0000 7fb6e264d740 1 -- 172.21.15.105:0/2443242526 shutdown_connections 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 -- 172.21.15.105:0/2443242526 wait complete. 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 -- start start 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 Processor -- start 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756be40 0x55930757b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756c830 0x55930757b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 0x55930757c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81a7f80 con 0x55930756be40 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81a78b0 con 0x55930756c830 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81a4af0 con 0x5593075719a0 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6e264d740 10 client.0 _ll_register_callbacks cb 0x7fb6d4086040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6dd806640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756be40 0x55930757b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6dd005640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756c830 0x55930757b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6dd806640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756be40 0x55930757b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43088/0 (socket says 172.21.15.105:43088) 2024-03-26T17:51:06.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6dd806640 1 -- 172.21.15.105:0/1700851890 learned_addr learned my addr 172.21.15.105:0/1700851890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.214+0000 7fb6de007640 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 0x55930757c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6dd806640 1 -- 172.21.15.105:0/1700851890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 msgr2=0x55930757c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6dd806640 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 0x55930757c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6dd806640 1 -- 172.21.15.105:0/1700851890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756c830 msgr2=0x55930757b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6dd806640 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756c830 0x55930757b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6dd806640 1 -- 172.21.15.105:0/1700851890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55930757cca0 con 0x55930756be40 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6dd005640 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756c830 0x55930757b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6dd806640 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756be40 0x55930757b160 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb6d409ba80 tx=0x7fb6d40aeae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec21766b614901ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6c17fa640 1 -- 172.21.15.105:0/1700851890 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d407e020 con 0x55930756be40 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6c17fa640 1 -- 172.21.15.105:0/1700851890 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb6d40ae6f0 con 0x55930756be40 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6e264d740 1 -- 172.21.15.105:0/1700851890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5593074d4200 con 0x55930756be40 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6c17fa640 1 -- 172.21.15.105:0/1700851890 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb6d40a42d0 con 0x55930756be40 2024-03-26T17:51:06.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6e264d740 10 client.20154 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6e264d740 1 -- 172.21.15.105:0/1700851890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55930757cca0 con 0x55930756be40 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6c17fa640 1 -- 172.21.15.105:0/1700851890 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5593074d4200 con 0x55930756be40 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.215+0000 7fb6c17fa640 1 -- 172.21.15.105:0/1700851890 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb6d40ab020 con 0x55930756be40 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 10 client.20154 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 10 client.20154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 20 client.20154 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5593075e8d50 0x5593075eb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 4 client.20154 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 1 -- 172.21.15.105:0/1700851890 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593074516a0 con 0x5593075e8d50 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6dd005640 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5593075e8d50 0x5593075eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6c17fa640 10 client.20154 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6dd005640 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5593075e8d50 0x5593075eb140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=600bc25647d9d99f server_cookie=448e06b67e7ea12c in_seq=0 out_seq=0 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6c17fa640 1 -- 172.21.15.105:0/1700851890 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593074516a0 con 0x5593075e8d50 2024-03-26T17:51:06.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6c17fa640 10 client.20154 handle_command_reply: tid=0 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 1 client.20154 shutdown 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 1 -- 172.21.15.105:0/1700851890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756be40 msgr2=0x55930757b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.216+0000 7fb6e264d740 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756be40 0x55930757b160 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb6d409ba80 tx=0x7fb6d40aeae0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 1 -- 172.21.15.105:0/1700851890 shutdown_connections 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5593075719a0 0x55930757c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5593075e8d50 0x5593075eb140 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55930756c830 0x55930757b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 1 --2- 172.21.15.105:0/1700851890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55930756be40 0x55930757b160 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6c17fa640 0 client.20154 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 1 -- 172.21.15.105:0/1700851890 >> 172.21.15.105:0/1700851890 conn(0x559307562fa0 msgr2=0x559307566200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 1 -- 172.21.15.105:0/1700851890 shutdown_connections 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 1 -- 172.21.15.105:0/1700851890 wait complete. 2024-03-26T17:51:06.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6e264d740 20 client.20154 trim_cache size 0 max 16384 2024-03-26T17:51:06.216 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:06.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6c40a2f20 msgr2=0x7fb6c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6c40a2f20 0x7fb6c40a53e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005fd0 tx=0x7fb6d0009f10 comp rx=0 tx=0).stop 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1019058723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81007c0 msgr2=0x7fb6d8197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81007c0 0x7fb6d8197b80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400e9c0 tx=0x7fb6d400ee80 comp rx=0 tx=0).stop 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1019058723 shutdown_connections 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6c40a2f20 0x7fb6c40a53e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb6d8102370 0x7fb6d819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6d81019b0 0x7fb6d81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 --2- 172.21.15.105:0/1019058723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6d81007c0 0x7fb6d8197b80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.217+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1019058723 >> 172.21.15.105:0/1019058723 conn(0x7fb6d80fbd50 msgr2=0x7fb6d8068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.218+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1019058723 shutdown_connections 2024-03-26T17:51:06.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.218+0000 7fb6dfa6a640 1 -- 172.21.15.105:0/1019058723 wait complete. 2024-03-26T17:51:06.221 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:51:06.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 -- 172.21.15.105:0/152827783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec1023a0 msgr2=0x7f93ec104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 --2- 172.21.15.105:0/152827783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec1023a0 0x7f93ec104790 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009900 tx=0x7f93e802bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:06.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 -- 172.21.15.105:0/152827783 shutdown_connections 2024-03-26T17:51:06.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 --2- 172.21.15.105:0/152827783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec107600 0x7f93ec1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 --2- 172.21.15.105:0/152827783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93ec104cd0 0x7f93ec1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 --2- 172.21.15.105:0/152827783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec1023a0 0x7f93ec104790 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 -- 172.21.15.105:0/152827783 >> 172.21.15.105:0/152827783 conn(0x7f93ec0fbd50 msgr2=0x7f93ec0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 -- 172.21.15.105:0/152827783 shutdown_connections 2024-03-26T17:51:06.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.349+0000 7f93f4436640 1 -- 172.21.15.105:0/152827783 wait complete. 2024-03-26T17:51:06.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f4436640 1 Processor -- start 2024-03-26T17:51:06.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f4436640 1 -- start start 2024-03-26T17:51:06.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f4436640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec1023a0 0x7f93ec197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f4436640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93ec104cd0 0x7f93ec1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f4436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec107600 0x7f93ec19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f4436640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec10d410 con 0x7f93ec107600 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f4436640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec10d290 con 0x7f93ec104cd0 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f4436640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec10d590 con 0x7f93ec1023a0 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f29d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec107600 0x7f93ec19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f29d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec107600 0x7f93ec19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43116/0 (socket says 172.21.15.105:43116) 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f29d3640 1 -- 172.21.15.105:0/3333180368 learned_addr learned my addr 172.21.15.105:0/3333180368 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f21d2640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec1023a0 0x7f93ec197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.350+0000 7f93f19d1640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93ec104cd0 0x7f93ec1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93f29d3640 1 -- 172.21.15.105:0/3333180368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec1023a0 msgr2=0x7f93ec197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93f29d3640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec1023a0 0x7f93ec197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93f29d3640 1 -- 172.21.15.105:0/3333180368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93ec104cd0 msgr2=0x7f93ec1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93f29d3640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93ec104cd0 0x7f93ec1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93f29d3640 1 -- 172.21.15.105:0/3333180368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec19f7a0 con 0x7f93ec107600 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93f29d3640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec107600 0x7f93ec19f130 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f93e400eb10 tx=0x7f93e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93e37fe640 1 -- 172.21.15.105:0/3333180368 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4004b20 con 0x7f93ec107600 2024-03-26T17:51:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93f4436640 1 -- 172.21.15.105:0/3333180368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec19fa90 con 0x7f93ec107600 2024-03-26T17:51:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93e37fe640 1 -- 172.21.15.105:0/3333180368 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e4004cc0 con 0x7f93ec107600 2024-03-26T17:51:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93e37fe640 1 -- 172.21.15.105:0/3333180368 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4010690 con 0x7f93ec107600 2024-03-26T17:51:06.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.351+0000 7f93f4436640 1 -- 172.21.15.105:0/3333180368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec19ffe0 con 0x7f93ec107600 2024-03-26T17:51:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.353+0000 7f93e37fe640 1 -- 172.21.15.105:0/3333180368 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f93e4005020 con 0x7f93ec107600 2024-03-26T17:51:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.354+0000 7f93e37fe640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93d80a2f20 0x7f93d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.354+0000 7f93f21d2640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93d80a2f20 0x7f93d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.354+0000 7f93e37fe640 1 -- 172.21.15.105:0/3333180368 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f93e40cfae0 con 0x7f93ec107600 2024-03-26T17:51:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f21d2640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93d80a2f20 0x7f93d80a53e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009230 tx=0x7f93e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f7023740 1 Processor -- start 2024-03-26T17:51:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f7023740 1 -- start start 2024-03-26T17:51:06.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f7023740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e32a30 0x55d4e5e32e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f7023740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e37f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f7023740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e38590 0x55d4e5e3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f7023740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d4e5d97970 con 0x55d4e5e33420 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f7023740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d4e5d3fd70 con 0x55d4e5e32a30 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f7023740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4e5d93920 con 0x55d4e5e38590 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f19d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e37f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f21d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e32a30 0x55d4e5e32e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f29d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e38590 0x55d4e5e3aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f19d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e37f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43126/0 (socket says 172.21.15.105:43126) 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f21d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e32a30 0x55d4e5e32e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60554/0 (socket says 172.21.15.105:60554) 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f29d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e38590 0x55d4e5e3aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58236/0 (socket says 172.21.15.105:58236) 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.355+0000 7f93f19d1640 1 -- 172.21.15.105:0/4077562907 learned_addr learned my addr 172.21.15.105:0/4077562907 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93f29d3640 1 -- 172.21.15.105:0/4077562907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e32a30 msgr2=0x55d4e5e32e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93f29d3640 1 --2- 172.21.15.105:0/4077562907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e32a30 0x55d4e5e32e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93f29d3640 1 -- 172.21.15.105:0/4077562907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 msgr2=0x55d4e5e37f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93f29d3640 1 --2- 172.21.15.105:0/4077562907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e37f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93f29d3640 1 -- 172.21.15.105:0/4077562907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e5aab3f0 con 0x55d4e5e38590 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93f19d1640 1 --2- 172.21.15.105:0/4077562907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e37f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93f29d3640 1 --2- 172.21.15.105:0/4077562907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e38590 0x55d4e5e3aa20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93e40cd910 tx=0x7f93e409bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8afffc7f2c3653d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93e0bfe640 1 -- 172.21.15.105:0/4077562907 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e409c1d0 con 0x55d4e5e38590 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93e0bfe640 1 -- 172.21.15.105:0/4077562907 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e40a0320 con 0x55d4e5e38590 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.356+0000 7f93e0bfe640 1 -- 172.21.15.105:0/4077562907 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4005310 con 0x55d4e5e38590 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 -- 172.21.15.105:0/4077562907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e38590 msgr2=0x55d4e5e3aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 --2- 172.21.15.105:0/4077562907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e38590 0x55d4e5e3aa20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93e40cd910 tx=0x7f93e409bd10 comp rx=0 tx=0).stop 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 -- 172.21.15.105:0/4077562907 shutdown_connections 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 --2- 172.21.15.105:0/4077562907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e38590 0x55d4e5e3aa20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 --2- 172.21.15.105:0/4077562907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e37f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 --2- 172.21.15.105:0/4077562907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e32a30 0x55d4e5e32e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 -- 172.21.15.105:0/4077562907 >> 172.21.15.105:0/4077562907 conn(0x55d4e5e2dfa0 msgr2=0x55d4e5e2e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 -- 172.21.15.105:0/4077562907 shutdown_connections 2024-03-26T17:51:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 -- 172.21.15.105:0/4077562907 wait complete. 2024-03-26T17:51:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 -- start start 2024-03-26T17:51:06.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.357+0000 7f93f7023740 1 Processor -- start 2024-03-26T17:51:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f7023740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f7023740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e3d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f7023740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e44590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f7023740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d4e5e3b540 con 0x55d4e5e32a30 2024-03-26T17:51:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f7023740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d4e5d53310 con 0x55d4e5e38590 2024-03-26T17:51:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f7023740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4e5e3b6c0 con 0x55d4e5e33420 2024-03-26T17:51:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f7023740 10 client.0 _ll_register_callbacks cb 0x55d4e5d8ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:06.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f21d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f19d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e3d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f29d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e44590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f21d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43128/0 (socket says 172.21.15.105:43128) 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f21d2640 1 -- 172.21.15.105:0/3871038491 learned_addr learned my addr 172.21.15.105:0/3871038491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f29d3640 1 -- 172.21.15.105:0/3871038491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 msgr2=0x55d4e5e3d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f29d3640 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e3d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f29d3640 1 -- 172.21.15.105:0/3871038491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 msgr2=0x55d4e5e3d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f29d3640 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f29d3640 1 -- 172.21.15.105:0/3871038491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e5b757d0 con 0x55d4e5e38590 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f19d1640 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e3d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f29d3640 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e44590 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009270 tx=0x7f93e409c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a482d7c0823c9515 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3871038491 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e40ba060 con 0x55d4e5e38590 2024-03-26T17:51:06.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93f7023740 1 -- 172.21.15.105:0/3871038491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e40a0320 con 0x55d4e5e38590 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.358+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3871038491 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f93e409b620 con 0x55d4e5e38590 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3871038491 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f93e409c1d0 con 0x55d4e5e38590 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f7023740 10 client.29966 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f7023740 1 -- 172.21.15.105:0/3871038491 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4e5e47900 con 0x55d4e5e38590 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3871038491 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e40a0320 con 0x55d4e5e38590 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3871038491 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f93e409b090 con 0x55d4e5e38590 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f7023740 10 client.29966 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f7023740 10 client.29966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f7023740 20 client.29966 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f7023740 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d4e5e48c50 0x55d4e5e4b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f7023740 4 client.29966 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f7023740 1 -- 172.21.15.105:0/3871038491 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4e5da4eb0 con 0x55d4e5e48c50 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.359+0000 7f93f21d2640 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d4e5e48c50 0x55d4e5e4b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.360+0000 7f93f21d2640 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d4e5e48c50 0x55d4e5e4b040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d65253da14d98db server_cookie=3855951514079427 in_seq=0 out_seq=0 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.360+0000 7f93d5ffb640 10 client.29966 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3871038491 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d4e5da4eb0 con 0x55d4e5e48c50 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93d5ffb640 10 client.29966 handle_command_reply: tid=0 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 client.29966 shutdown 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 -- 172.21.15.105:0/3871038491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 msgr2=0x55d4e5e44590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e44590 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009270 tx=0x7f93e409c990 comp rx=0 tx=0).stop 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 -- 172.21.15.105:0/3871038491 shutdown_connections 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d4e5e48c50 0x55d4e5e4b040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e44590 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e3d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 --2- 172.21.15.105:0/3871038491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93d5ffb640 0 client.29966 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 -- 172.21.15.105:0/3871038491 >> 172.21.15.105:0/3871038491 conn(0x55d4e5e2dfa0 msgr2=0x55d4e5e2d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 -- 172.21.15.105:0/3871038491 shutdown_connections 2024-03-26T17:51:06.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 1 -- 172.21.15.105:0/3871038491 wait complete. 2024-03-26T17:51:06.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.361+0000 7f93f7023740 20 client.29966 trim_cache size 0 max 16384 2024-03-26T17:51:06.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f7023740 1 Processor -- start 2024-03-26T17:51:06.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f7023740 1 -- start start 2024-03-26T17:51:06.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f7023740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f7023740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e44550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f7023740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f7023740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1a8df0 con 0x55d4e5e32a30 2024-03-26T17:51:06.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f7023740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1aa930 con 0x55d4e5e38590 2024-03-26T17:51:06.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f7023740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1a94c0 con 0x55d4e5e33420 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f19d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e44550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f21d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f29d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f19d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e44550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58258/0 (socket says 172.21.15.105:58258) 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f19d1640 1 -- 172.21.15.105:0/4092680087 learned_addr learned my addr 172.21.15.105:0/4092680087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f29d3640 1 -- 172.21.15.105:0/4092680087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 msgr2=0x55d4e5e44550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f29d3640 1 --2- 172.21.15.105:0/4092680087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e44550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f29d3640 1 -- 172.21.15.105:0/4092680087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 msgr2=0x55d4e5e3ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f29d3640 1 --2- 172.21.15.105:0/4092680087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f29d3640 1 -- 172.21.15.105:0/4092680087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e5e3b7c0 con 0x55d4e5e38590 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.377+0000 7f93f21d2640 1 --2- 172.21.15.105:0/4092680087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.378+0000 7f93f29d3640 1 --2- 172.21.15.105:0/4092680087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e3bfa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f93e40cd910 tx=0x7f93e400efd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e069dc410c2c4b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.378+0000 7f93d7fff640 1 -- 172.21.15.105:0/4092680087 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e40a0320 con 0x55d4e5e38590 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.378+0000 7f93d7fff640 1 -- 172.21.15.105:0/4092680087 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e409a4d0 con 0x55d4e5e38590 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.378+0000 7f93d7fff640 1 -- 172.21.15.105:0/4092680087 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e409b090 con 0x55d4e5e38590 2024-03-26T17:51:06.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- 172.21.15.105:0/4092680087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 msgr2=0x55d4e5e3bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 --2- 172.21.15.105:0/4092680087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e3bfa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f93e40cd910 tx=0x7f93e400efd0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- 172.21.15.105:0/4092680087 shutdown_connections 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 --2- 172.21.15.105:0/4092680087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e3bfa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 --2- 172.21.15.105:0/4092680087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e33420 0x55d4e5e44550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 --2- 172.21.15.105:0/4092680087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e32a30 0x55d4e5e3ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- 172.21.15.105:0/4092680087 >> 172.21.15.105:0/4092680087 conn(0x55d4e5e2dfa0 msgr2=0x55d4e5e31a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- 172.21.15.105:0/4092680087 shutdown_connections 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- 172.21.15.105:0/4092680087 wait complete. 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- start start 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 Processor -- start 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e32a30 0x55d4e5e41570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e41ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e42ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1aa260 con 0x55d4e5e33420 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1a9b90 con 0x55d4e5e38590 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1a6dd0 con 0x55d4e5e32a30 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f7023740 10 client.0 _ll_register_callbacks cb 0x7f93dc009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f21d2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e32a30 0x55d4e5e41570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f19d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e41ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.379+0000 7f93f29d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e42ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f29d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e42ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60582/0 (socket says 172.21.15.105:60582) 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f19d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e41ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43138/0 (socket says 172.21.15.105:43138) 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f29d3640 1 -- 172.21.15.105:0/3675563772 learned_addr learned my addr 172.21.15.105:0/3675563772 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f21d2640 1 -- 172.21.15.105:0/3675563772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 msgr2=0x55d4e5e42ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f21d2640 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e42ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f21d2640 1 -- 172.21.15.105:0/3675563772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 msgr2=0x55d4e5e41ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f21d2640 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e41ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f21d2640 1 -- 172.21.15.105:0/3675563772 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e5e430b0 con 0x55d4e5e32a30 2024-03-26T17:51:06.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f19d1640 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e41ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f29d3640 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e42ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f21d2640 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e32a30 0x55d4e5e41570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55d4e5e4b770 tx=0x7f93e802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a60f4640515fc606 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3675563772 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8005040 con 0x55d4e5e32a30 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3675563772 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f93e80051e0 con 0x55d4e5e32a30 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f7023740 1 -- 172.21.15.105:0/3675563772 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e409a4d0 con 0x55d4e5e32a30 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3675563772 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f93e8064050 con 0x55d4e5e32a30 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f7023740 10 client.29956 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.380+0000 7f93f7023740 1 -- 172.21.15.105:0/3675563772 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d4e5e44420 con 0x55d4e5e32a30 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3675563772 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e409a4d0 con 0x55d4e5e32a30 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3675563772 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f93e80046c0 con 0x55d4e5e32a30 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93f7023740 10 client.29956 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93f7023740 10 client.29956 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:51:06.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93f7023740 20 client.29956 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93f7023740 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d4e5eaf170 0x55d4e5eb1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93f7023740 4 client.29956 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93f7023740 1 -- 172.21.15.105:0/3675563772 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4e5d9f200 con 0x55d4e5eaf170 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93f19d1640 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d4e5eaf170 0x55d4e5eb1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93f19d1640 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d4e5eaf170 0x55d4e5eb1590 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8dc1acd5156f5185 server_cookie=70fc73bf58362fff in_seq=0 out_seq=0 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.381+0000 7f93d5ffb640 10 client.29956 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93d5ffb640 1 -- 172.21.15.105:0/3675563772 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4e5d9f200 con 0x55d4e5eaf170 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93d5ffb640 10 client.29956 handle_command_reply: tid=0 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 client.29956 shutdown 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 -- 172.21.15.105:0/3675563772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e32a30 msgr2=0x55d4e5e41570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e32a30 0x55d4e5e41570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55d4e5e4b770 tx=0x7f93e802bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 -- 172.21.15.105:0/3675563772 shutdown_connections 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d4e5eaf170 0x55d4e5eb1590 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d4e5e38590 0x55d4e5e42ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93d5ffb640 0 client.29956 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:51:06.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d4e5e33420 0x55d4e5e41ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 --2- 172.21.15.105:0/3675563772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4e5e32a30 0x55d4e5e41570 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 -- 172.21.15.105:0/3675563772 >> 172.21.15.105:0/3675563772 conn(0x55d4e5e2dfa0 msgr2=0x55d4e5e2c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 -- 172.21.15.105:0/3675563772 shutdown_connections 2024-03-26T17:51:06.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 1 -- 172.21.15.105:0/3675563772 wait complete. 2024-03-26T17:51:06.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f7023740 20 client.29956 trim_cache size 0 max 16384 2024-03-26T17:51:06.384 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:06.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f4436640 1 -- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93d80a2f20 msgr2=0x7f93d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f4436640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93d80a2f20 0x7f93d80a53e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009230 tx=0x7f93e8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f4436640 1 -- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec107600 msgr2=0x7f93ec19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f4436640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec107600 0x7f93ec19f130 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f93e400eb10 tx=0x7f93e400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.382+0000 7f93f4436640 1 -- 172.21.15.105:0/3333180368 shutdown_connections 2024-03-26T17:51:06.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.383+0000 7f93f4436640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f93d80a2f20 0x7f93d80a53e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.383+0000 7f93f4436640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f93ec107600 0x7f93ec19f130 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.383+0000 7f93f4436640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f93ec104cd0 0x7f93ec1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.383+0000 7f93f4436640 1 --2- 172.21.15.105:0/3333180368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93ec1023a0 0x7f93ec197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.383+0000 7f93f4436640 1 -- 172.21.15.105:0/3333180368 >> 172.21.15.105:0/3333180368 conn(0x7f93ec0fbd50 msgr2=0x7f93ec102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.383+0000 7f93f4436640 1 -- 172.21.15.105:0/3333180368 shutdown_connections 2024-03-26T17:51:06.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.383+0000 7f93f4436640 1 -- 172.21.15.105:0/3333180368 wait complete. 2024-03-26T17:51:06.387 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:51:06.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.513+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/3763647146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b8102ae0 msgr2=0x7ff7b8102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.513+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/3763647146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b8102ae0 0x7ff7b8102f60 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009900 tx=0x7ff7a002bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.513+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/3763647146 shutdown_connections 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.513+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/3763647146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7b81034a0 0x7ff7b8109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.513+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/3763647146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b8102ae0 0x7ff7b8102f60 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.513+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/3763647146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7b81018f0 0x7ff7b8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.513+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/3763647146 >> 172.21.15.105:0/3763647146 conn(0x7ff7b80fce60 msgr2=0x7ff7b80ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/3763647146 shutdown_connections 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/3763647146 wait complete. 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 Processor -- start 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 -- start start 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b81018f0 0x7ff7b8075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7b8102ae0 0x7ff7b8073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7b81034a0 0x7ff7b80741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810d420 con 0x7ff7b81018f0 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810d2a0 con 0x7ff7b8102ae0 2024-03-26T17:51:06.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7bd7a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b810d5a0 con 0x7ff7b81034a0 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b81018f0 0x7ff7b8075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7affff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7b8102ae0 0x7ff7b8073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b81018f0 0x7ff7b8075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43150/0 (socket says 172.21.15.105:43150) 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7b6ffd640 1 -- 172.21.15.105:0/771646664 learned_addr learned my addr 172.21.15.105:0/771646664 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.514+0000 7ff7b77fe640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7b81034a0 0x7ff7b80741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7affff640 1 -- 172.21.15.105:0/771646664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7b81034a0 msgr2=0x7ff7b80741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7affff640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7b81034a0 0x7ff7b80741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7affff640 1 -- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b81018f0 msgr2=0x7ff7b8075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7affff640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b81018f0 0x7ff7b8075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7affff640 1 -- 172.21.15.105:0/771646664 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b8074a80 con 0x7ff7b8102ae0 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7affff640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7b8102ae0 0x7ff7b8073cb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0000c00 tx=0x7ff7a0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7b4ff9640 1 -- 172.21.15.105:0/771646664 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0043020 con 0x7ff7b8102ae0 2024-03-26T17:51:06.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/771646664 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b81a1c60 con 0x7ff7b8102ae0 2024-03-26T17:51:06.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7b4ff9640 1 -- 172.21.15.105:0/771646664 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7a0004350 con 0x7ff7b8102ae0 2024-03-26T17:51:06.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7b4ff9640 1 -- 172.21.15.105:0/771646664 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0003b50 con 0x7ff7b8102ae0 2024-03-26T17:51:06.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.515+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/771646664 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b81a21b0 con 0x7ff7b8102ae0 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.518+0000 7ff7b4ff9640 1 -- 172.21.15.105:0/771646664 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff7a0046020 con 0x7ff7b8102ae0 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.518+0000 7ff7c038c740 1 Processor -- start 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.518+0000 7ff7c038c740 1 -- start start 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1ba30 0x5650b9a1be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1c420 0x5650b9a20f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a23a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7c038c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5650b9980970 con 0x5650b9a1ba30 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7c038c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5650b9928d70 con 0x5650b9a1c420 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7c038c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650b997c920 con 0x5650b9a21590 2024-03-26T17:51:06.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7b4ff9640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7a40a2f20 0x7ff7a40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1ba30 0x5650b9a1be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1c420 0x5650b9a20f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1ba30 0x5650b9a1be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43154/0 (socket says 172.21.15.105:43154) 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1c420 0x5650b9a20f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60612/0 (socket says 172.21.15.105:60612) 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a23a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7b77fe640 1 -- 172.21.15.105:0/1600366308 learned_addr learned my addr 172.21.15.105:0/1600366308 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7b77fe640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7a40a2f20 0x7ff7a40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7b4ff9640 1 -- 172.21.15.105:0/771646664 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff7a00f1590 con 0x7ff7b8102ae0 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7affff640 1 -- 172.21.15.105:0/1600366308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1c420 msgr2=0x5650b9a20f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7affff640 1 --2- 172.21.15.105:0/1600366308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1c420 0x5650b9a20f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7affff640 1 -- 172.21.15.105:0/1600366308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1ba30 msgr2=0x5650b9a1be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7affff640 1 --2- 172.21.15.105:0/1600366308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1ba30 0x5650b9a1be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.519+0000 7ff7affff640 1 -- 172.21.15.105:0/1600366308 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b96943f0 con 0x5650b9a21590 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.520+0000 7ff7affff640 1 --2- 172.21.15.105:0/1600366308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a23a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00bd980 tx=0x7ff7a0088720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f06f6abe84512a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.520+0000 7ff7b77fe640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7a40a2f20 0x7ff7a40a53e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00099e0 tx=0x7ff7b0009c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.520+0000 7ff7ad7fa640 1 -- 172.21.15.105:0/1600366308 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00bda50 con 0x5650b9a21590 2024-03-26T17:51:06.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.520+0000 7ff7ad7fa640 1 -- 172.21.15.105:0/1600366308 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7a0088c30 con 0x5650b9a21590 2024-03-26T17:51:06.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.520+0000 7ff7ad7fa640 1 -- 172.21.15.105:0/1600366308 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00333a0 con 0x5650b9a21590 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- 172.21.15.105:0/1600366308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 msgr2=0x5650b9a23a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 --2- 172.21.15.105:0/1600366308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a23a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00bd980 tx=0x7ff7a0088720 comp rx=0 tx=0).stop 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- 172.21.15.105:0/1600366308 shutdown_connections 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 --2- 172.21.15.105:0/1600366308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a23a20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 --2- 172.21.15.105:0/1600366308 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1c420 0x5650b9a20f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 --2- 172.21.15.105:0/1600366308 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1ba30 0x5650b9a1be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- 172.21.15.105:0/1600366308 >> 172.21.15.105:0/1600366308 conn(0x5650b9a16fa0 msgr2=0x5650b9a173a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- 172.21.15.105:0/1600366308 shutdown_connections 2024-03-26T17:51:06.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- 172.21.15.105:0/1600366308 wait complete. 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- start start 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 Processor -- start 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a25c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a26140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 0x5650b9a2d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5650b9a24300 con 0x5650b9a1c420 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5650b993c310 con 0x5650b9a21590 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5650b9a24480 con 0x5650b9a1ba30 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.521+0000 7ff7c038c740 10 client.0 _ll_register_callbacks cb 0x5650b9977c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a25c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a26140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a25c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58312/0 (socket says 172.21.15.105:58312) 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 -- 172.21.15.105:0/3215166417 learned_addr learned my addr 172.21.15.105:0/3215166417 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b77fe640 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 0x5650b9a2d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 -- 172.21.15.105:0/3215166417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 msgr2=0x5650b9a2d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 0x5650b9a2d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 -- 172.21.15.105:0/3215166417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 msgr2=0x5650b9a26140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a26140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 -- 172.21.15.105:0/3215166417 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b975e7d0 con 0x5650b9a1ba30 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7affff640 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a26140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7b6ffd640 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a25c00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8012700 tx=0x7ff7a8012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86acdba3625e3bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff79e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff79e7fc640 1 -- 172.21.15.105:0/3215166417 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8004260 con 0x5650b9a1ba30 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff79e7fc640 1 -- 172.21.15.105:0/3215166417 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff7a8004400 con 0x5650b9a1ba30 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7c038c740 1 -- 172.21.15.105:0/3215166417 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a0088c30 con 0x5650b9a1ba30 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff79e7fc640 1 -- 172.21.15.105:0/3215166417 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff7a8011940 con 0x5650b9a1ba30 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7c038c740 10 client.29968 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.522+0000 7ff7c038c740 1 -- 172.21.15.105:0/3215166417 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5650b9a305c0 con 0x5650b9a1ba30 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff79e7fc640 1 -- 172.21.15.105:0/3215166417 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7a0088c30 con 0x5650b9a1ba30 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff79e7fc640 1 -- 172.21.15.105:0/3215166417 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff7a800f440 con 0x5650b9a1ba30 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff7c038c740 10 client.29968 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:06.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff7c038c740 10 client.29968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff7c038c740 20 client.29968 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff7c038c740 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5650b9a31c70 0x5650b9a34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff7c038c740 4 client.29968 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff7c038c740 1 -- 172.21.15.105:0/3215166417 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b998deb0 con 0x5650b9a31c70 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff7affff640 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5650b9a31c70 0x5650b9a34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff79e7fc640 10 client.29968 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.523+0000 7ff7affff640 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5650b9a31c70 0x5650b9a34130 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b90ff1a8a76a7dd server_cookie=b33ea0ae5c5682fe in_seq=0 out_seq=0 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff79e7fc640 1 -- 172.21.15.105:0/3215166417 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5650b998deb0 con 0x5650b9a31c70 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff79e7fc640 10 client.29968 handle_command_reply: tid=0 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 client.29968 shutdown 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 -- 172.21.15.105:0/3215166417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 msgr2=0x5650b9a25c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a25c00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8012700 tx=0x7ff7a8012bc0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 -- 172.21.15.105:0/3215166417 shutdown_connections 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5650b9a31c70 0x5650b9a34130 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 0x5650b9a2d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a26140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 --2- 172.21.15.105:0/3215166417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a25c00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff79e7fc640 0 client.29968 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:06.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 -- 172.21.15.105:0/3215166417 >> 172.21.15.105:0/3215166417 conn(0x5650b9a16fa0 msgr2=0x5650b9a161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 -- 172.21.15.105:0/3215166417 shutdown_connections 2024-03-26T17:51:06.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 1 -- 172.21.15.105:0/3215166417 wait complete. 2024-03-26T17:51:06.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.524+0000 7ff7c038c740 20 client.29968 trim_cache size 0 max 16384 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.538+0000 7ff7c038c740 1 Processor -- start 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7c038c740 1 -- start start 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a16040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a16580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 0x5650b9a2d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7c038c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81aaf70 con 0x5650b9a1c420 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7c038c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81acab0 con 0x5650b9a21590 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7c038c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81ab640 con 0x5650b9a1ba30 2024-03-26T17:51:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a16580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a16580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43174/0 (socket says 172.21.15.105:43174) 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7affff640 1 -- 172.21.15.105:0/1747974188 learned_addr learned my addr 172.21.15.105:0/1747974188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7b6ffd640 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a16040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7b77fe640 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 0x5650b9a2d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7affff640 1 -- 172.21.15.105:0/1747974188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 msgr2=0x5650b9a16040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7affff640 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a16040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7affff640 1 -- 172.21.15.105:0/1747974188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 msgr2=0x5650b9a2d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7affff640 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 0x5650b9a2d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.539+0000 7ff7affff640 1 -- 172.21.15.105:0/1747974188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b9a16b40 con 0x5650b9a1c420 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.540+0000 7ff7affff640 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a16580 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00096c0 tx=0x7ff7a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ca0c92bcd94f0b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.540+0000 7ff7acff9640 1 -- 172.21.15.105:0/1747974188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00bd5e0 con 0x5650b9a1c420 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.540+0000 7ff7acff9640 1 -- 172.21.15.105:0/1747974188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7a0088c30 con 0x5650b9a1c420 2024-03-26T17:51:06.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.540+0000 7ff7acff9640 1 -- 172.21.15.105:0/1747974188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00bd7c0 con 0x5650b9a1c420 2024-03-26T17:51:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.540+0000 7ff7c038c740 1 -- 172.21.15.105:0/1747974188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 msgr2=0x5650b9a16580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.540+0000 7ff7c038c740 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a16580 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00096c0 tx=0x7ff7a0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 -- 172.21.15.105:0/1747974188 shutdown_connections 2024-03-26T17:51:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a21590 0x5650b9a2d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a16580 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 --2- 172.21.15.105:0/1747974188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a1ba30 0x5650b9a16040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 -- 172.21.15.105:0/1747974188 >> 172.21.15.105:0/1747974188 conn(0x5650b9a16fa0 msgr2=0x5650b9a15930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 -- 172.21.15.105:0/1747974188 shutdown_connections 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 -- 172.21.15.105:0/1747974188 wait complete. 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 -- start start 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 Processor -- start 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1ba30 0x5650b9a2a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a2ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a2bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81ac3e0 con 0x5650b9a1c420 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81abd10 con 0x5650b9a1ba30 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b81a8f50 con 0x5650b9a21590 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7c038c740 10 client.0 _ll_register_callbacks cb 0x7ff7a8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a2ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1ba30 0x5650b9a2a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7affff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a2ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43176/0 (socket says 172.21.15.105:43176) 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7affff640 1 -- 172.21.15.105:0/382814203 learned_addr learned my addr 172.21.15.105:0/382814203 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:06.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.541+0000 7ff7b77fe640 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a2bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7affff640 1 -- 172.21.15.105:0/382814203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 msgr2=0x5650b9a2bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7affff640 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a2bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7affff640 1 -- 172.21.15.105:0/382814203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1ba30 msgr2=0x5650b9a2a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7affff640 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1ba30 0x5650b9a2a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7affff640 1 -- 172.21.15.105:0/382814203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b9a2c110 con 0x5650b9a1c420 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7b6ffd640 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1ba30 0x5650b9a2a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7affff640 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a2ab10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00ba0e0 tx=0x7ff7a0004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbb1c89d7b244a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff79e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff79e7fc640 1 -- 172.21.15.105:0/382814203 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00bd5e0 con 0x5650b9a1c420 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff79e7fc640 1 -- 172.21.15.105:0/382814203 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff7a00bbf80 con 0x5650b9a1c420 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7c038c740 1 -- 172.21.15.105:0/382814203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a0088c30 con 0x5650b9a1c420 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff79e7fc640 1 -- 172.21.15.105:0/382814203 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff7a00bda50 con 0x5650b9a1c420 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7c038c740 10 client.20190 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff7c038c740 1 -- 172.21.15.105:0/382814203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5650b9a2d410 con 0x5650b9a1c420 2024-03-26T17:51:06.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.542+0000 7ff79e7fc640 1 -- 172.21.15.105:0/382814203 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7a0088c30 con 0x5650b9a1c420 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff79e7fc640 1 -- 172.21.15.105:0/382814203 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff7a00c6200 con 0x5650b9a1c420 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff7c038c740 10 client.20190 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff7c038c740 10 client.20190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff7c038c740 20 client.20190 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff7c038c740 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5650b9a98310 0x5650b9a9a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff7c038c740 4 client.20190 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff7c038c740 1 -- 172.21.15.105:0/382814203 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650b9988200 con 0x5650b9a98310 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff7b6ffd640 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5650b9a98310 0x5650b9a9a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff79e7fc640 10 client.20190 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.543+0000 7ff7b6ffd640 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5650b9a98310 0x5650b9a9a700 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b82393276eab426 server_cookie=1b17d36a8612f9ba in_seq=0 out_seq=0 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff79e7fc640 1 -- 172.21.15.105:0/382814203 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650b9988200 con 0x5650b9a98310 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff79e7fc640 10 client.20190 handle_command_reply: tid=0 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 client.20190 shutdown 2024-03-26T17:51:06.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 -- 172.21.15.105:0/382814203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 msgr2=0x5650b9a2ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a2ab10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00ba0e0 tx=0x7ff7a0004580 comp rx=0 tx=0).stop 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 -- 172.21.15.105:0/382814203 shutdown_connections 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5650b9a98310 0x5650b9a9a700 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff79e7fc640 0 client.20190 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5650b9a21590 0x5650b9a2bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650b9a1c420 0x5650b9a2ab10 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 --2- 172.21.15.105:0/382814203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b9a1ba30 0x5650b9a2a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 -- 172.21.15.105:0/382814203 >> 172.21.15.105:0/382814203 conn(0x5650b9a16fa0 msgr2=0x5650b9a161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 -- 172.21.15.105:0/382814203 shutdown_connections 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 1 -- 172.21.15.105:0/382814203 wait complete. 2024-03-26T17:51:06.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.544+0000 7ff7c038c740 20 client.20190 trim_cache size 0 max 16384 2024-03-26T17:51:06.543 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:06.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7a40a2f20 msgr2=0x7ff7a40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7a40a2f20 0x7ff7a40a53e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00099e0 tx=0x7ff7b0009c80 comp rx=0 tx=0).stop 2024-03-26T17:51:06.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/771646664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7b8102ae0 msgr2=0x7ff7b8073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7b8102ae0 0x7ff7b8073cb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0000c00 tx=0x7ff7a0005dc0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/771646664 shutdown_connections 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff7a40a2f20 0x7ff7a40a53e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7b81034a0 0x7ff7b80741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff7b8102ae0 0x7ff7b8073cb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 --2- 172.21.15.105:0/771646664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7b81018f0 0x7ff7b8075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/771646664 >> 172.21.15.105:0/771646664 conn(0x7ff7b80fce60 msgr2=0x7ff7b8105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/771646664 shutdown_connections 2024-03-26T17:51:06.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:06.545+0000 7ff7bd7a9640 1 -- 172.21.15.105:0/771646664 wait complete. 2024-03-26T17:51:07.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 415601 3.14 MB/sec execute 590 sec latency 16.724 ms 2024-03-26T17:51:07.550 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:51:07.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:51:07.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:51:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 -- 172.21.15.105:0/2020204591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 msgr2=0x7f927c1007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 --2- 172.21.15.105:0/2020204591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 0x7f927c1007c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9274009be0 tx=0x7f927402bef0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 -- 172.21.15.105:0/2020204591 shutdown_connections 2024-03-26T17:51:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 --2- 172.21.15.105:0/2020204591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 0x7f927c1007c0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 --2- 172.21.15.105:0/2020204591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f927c0ff950 0x7f927c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 --2- 172.21.15.105:0/2020204591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c101e60 0x7f927c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 -- 172.21.15.105:0/2020204591 >> 172.21.15.105:0/2020204591 conn(0x7f927c0fb4d0 msgr2=0x7f927c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 -- 172.21.15.105:0/2020204591 shutdown_connections 2024-03-26T17:51:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 -- 172.21.15.105:0/2020204591 wait complete. 2024-03-26T17:51:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.678+0000 7f9283962640 1 Processor -- start 2024-03-26T17:51:07.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9283962640 1 -- start start 2024-03-26T17:51:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9283962640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f927c0ff950 0x7f927c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9283962640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 0x7f927c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9283962640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c101e60 0x7f927c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9283962640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f927c10f5f0 con 0x7f927c100310 2024-03-26T17:51:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9283962640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f927c10f470 con 0x7f927c0ff950 2024-03-26T17:51:07.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9283962640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f927c10f770 con 0x7f927c101e60 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9280efd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 0x7f927c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f92816fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f927c0ff950 0x7f927c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9280efd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 0x7f927c19a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43194/0 (socket says 172.21.15.105:43194) 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9280efd640 1 -- 172.21.15.105:0/3421377538 learned_addr learned my addr 172.21.15.105:0/3421377538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9281eff640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c101e60 0x7f927c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f92816fe640 1 -- 172.21.15.105:0/3421377538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c101e60 msgr2=0x7f927c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f92816fe640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c101e60 0x7f927c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f92816fe640 1 -- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 msgr2=0x7f927c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f92816fe640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 0x7f927c19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f92816fe640 1 -- 172.21.15.105:0/3421377538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c1a1900 con 0x7f927c0ff950 2024-03-26T17:51:07.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f9280efd640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 0x7f927c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.679+0000 7f92816fe640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f927c0ff950 0x7f927c199cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f927800e790 tx=0x7f927800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.680+0000 7f92727fc640 1 -- 172.21.15.105:0/3421377538 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927800eea0 con 0x7f927c0ff950 2024-03-26T17:51:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.680+0000 7f92727fc640 1 -- 172.21.15.105:0/3421377538 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f927800ccc0 con 0x7f927c0ff950 2024-03-26T17:51:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.680+0000 7f92727fc640 1 -- 172.21.15.105:0/3421377538 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278018710 con 0x7f927c0ff950 2024-03-26T17:51:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.680+0000 7f9283962640 1 -- 172.21.15.105:0/3421377538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c1a1bf0 con 0x7f927c0ff950 2024-03-26T17:51:07.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.680+0000 7f9283962640 1 -- 172.21.15.105:0/3421377538 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1a20f0 con 0x7f927c0ff950 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.682+0000 7f92727fc640 1 -- 172.21.15.105:0/3421377538 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9278010430 con 0x7f927c0ff950 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.682+0000 7f928662f740 1 Processor -- start 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.682+0000 7f928662f740 1 -- start start 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f92727fc640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92680a2f20 0x7f92680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f928662f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db0035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f928662f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f928662f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db00b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f928662f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a6daf68630 con 0x55a6db008cf0 2024-03-26T17:51:07.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f928662f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a6daf10e10 con 0x55a6db003b80 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f928662f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a6daf645e0 con 0x55a6db003190 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f92727fc640 1 -- 172.21.15.105:0/3421377538 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f92780cf590 con 0x7f927c0ff950 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9281eff640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92680a2f20 0x7f92680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db008680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60664/0 (socket says 172.21.15.105:60664) 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 -- 172.21.15.105:0/2827127508 learned_addr learned my addr 172.21.15.105:0/2827127508 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f92816fe640 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db0035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9281eff640 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db00b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 -- 172.21.15.105:0/2827127508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 msgr2=0x55a6db0035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db0035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 -- 172.21.15.105:0/2827127508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 msgr2=0x55a6db00b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db00b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 -- 172.21.15.105:0/2827127508 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6daf27740 con 0x55a6db003b80 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f92816fe640 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db0035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.683+0000 7f9280efd640 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db008680 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f926c0115c0 tx=0x7f926c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877d0b86270d3d41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f9281eff640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92680a2f20 0x7f92680a53e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9274004af0 tx=0x7f9274033450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f92677fe640 1 -- 172.21.15.105:0/2827127508 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c004030 con 0x55a6db003b80 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f92677fe640 1 -- 172.21.15.105:0/2827127508 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f926c0054d0 con 0x55a6db003b80 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f92677fe640 1 -- 172.21.15.105:0/2827127508 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c004c10 con 0x55a6db003b80 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f928662f740 1 -- 172.21.15.105:0/2827127508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 msgr2=0x55a6db008680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f928662f740 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db008680 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f926c0115c0 tx=0x7f926c009f90 comp rx=0 tx=0).stop 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f928662f740 1 -- 172.21.15.105:0/2827127508 shutdown_connections 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f928662f740 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db00b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f928662f740 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db008680 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f928662f740 1 --2- 172.21.15.105:0/2827127508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db0035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.684+0000 7f928662f740 1 -- 172.21.15.105:0/2827127508 >> 172.21.15.105:0/2827127508 conn(0x55a6daffe700 msgr2=0x55a6daffeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 -- 172.21.15.105:0/2827127508 shutdown_connections 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 -- 172.21.15.105:0/2827127508 wait complete. 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 -- start start 2024-03-26T17:51:07.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 Processor -- start 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db003190 0x55a6db00d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db008cf0 0x55a6db014c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a6db00bca0 con 0x55a6db003190 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a6daf241a0 con 0x55a6db008cf0 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a6db00be20 con 0x55a6db003b80 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f928662f740 10 client.0 _ll_register_callbacks cb 0x55a6dac7bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f92816fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db003190 0x55a6db00d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f92816fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db003190 0x55a6db00d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43210/0 (socket says 172.21.15.105:43210) 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f92816fe640 1 -- 172.21.15.105:0/504017735 learned_addr learned my addr 172.21.15.105:0/504017735 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.685+0000 7f9280efd640 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f9281eff640 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db008cf0 0x55a6db014c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f92816fe640 1 -- 172.21.15.105:0/504017735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 msgr2=0x55a6db00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f92816fe640 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f92816fe640 1 -- 172.21.15.105:0/504017735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db008cf0 msgr2=0x55a6db014c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f92816fe640 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db008cf0 0x55a6db014c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f92816fe640 1 -- 172.21.15.105:0/504017735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6dad447e0 con 0x55a6db003190 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f92816fe640 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db003190 0x55a6db00d6a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9278005fd0 tx=0x7f9278010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4870191e4e8712 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f9264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f9264ff9640 1 -- 172.21.15.105:0/504017735 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92780a4690 con 0x55a6db003190 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f9264ff9640 1 -- 172.21.15.105:0/504017735 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f92780cf110 con 0x55a6db003190 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f928662f740 1 -- 172.21.15.105:0/504017735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926c0054d0 con 0x55a6db003190 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f9264ff9640 1 -- 172.21.15.105:0/504017735 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f92780cf310 con 0x55a6db003190 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f928662f740 10 client.20202 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f928662f740 1 -- 172.21.15.105:0/504017735 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6dada4f70 con 0x55a6db003190 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.686+0000 7f9264ff9640 1 -- 172.21.15.105:0/504017735 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f926c0054d0 con 0x55a6db003190 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f9264ff9640 1 -- 172.21.15.105:0/504017735 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f927809ad80 con 0x55a6db003190 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f928662f740 10 client.20202 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f928662f740 10 client.20202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f928662f740 20 client.20202 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:07.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f928662f740 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a6db019550 0x55a6db01ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f928662f740 4 client.20202 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f928662f740 1 -- 172.21.15.105:0/504017735 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6daf75b70 con 0x55a6db019550 2024-03-26T17:51:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f9280efd640 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a6db019550 0x55a6db01ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f9264ff9640 10 client.20202 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:07.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.687+0000 7f9280efd640 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a6db019550 0x55a6db01ba10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2733bf10cff717e server_cookie=4f33d8e6ecfc3f9d in_seq=0 out_seq=0 2024-03-26T17:51:07.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f9264ff9640 1 -- 172.21.15.105:0/504017735 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a6daf75b70 con 0x55a6db019550 2024-03-26T17:51:07.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f9264ff9640 10 client.20202 handle_command_reply: tid=0 2024-03-26T17:51:07.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 client.20202 shutdown 2024-03-26T17:51:07.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 -- 172.21.15.105:0/504017735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db003190 msgr2=0x55a6db00d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db003190 0x55a6db00d6a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9278005fd0 tx=0x7f9278010b30 comp rx=0 tx=0).stop 2024-03-26T17:51:07.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 -- 172.21.15.105:0/504017735 shutdown_connections 2024-03-26T17:51:07.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a6db019550 0x55a6db01ba10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f9264ff9640 0 client.20202 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:07.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db008cf0 0x55a6db014c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db00dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 --2- 172.21.15.105:0/504017735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db003190 0x55a6db00d6a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 -- 172.21.15.105:0/504017735 >> 172.21.15.105:0/504017735 conn(0x55a6daffe700 msgr2=0x55a6daffd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 -- 172.21.15.105:0/504017735 shutdown_connections 2024-03-26T17:51:07.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 1 -- 172.21.15.105:0/504017735 wait complete. 2024-03-26T17:51:07.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.689+0000 7f928662f740 20 client.20202 trim_cache size 0 max 16384 2024-03-26T17:51:07.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.711+0000 7f928662f740 1 Processor -- start 2024-03-26T17:51:07.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.711+0000 7f928662f740 1 -- start start 2024-03-26T17:51:07.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.711+0000 7f928662f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db00db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.711+0000 7f928662f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db002070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.711+0000 7f928662f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db0202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f928662f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1aafd0 con 0x55a6db008cf0 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f928662f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1acb10 con 0x55a6db003b80 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f928662f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1ab6a0 con 0x55a6db003190 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db002070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9281eff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db0202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db002070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60690/0 (socket says 172.21.15.105:60690) 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f92816fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db00db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 -- 172.21.15.105:0/1101774953 learned_addr learned my addr 172.21.15.105:0/1101774953 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 -- 172.21.15.105:0/1101774953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 msgr2=0x55a6db00db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 --2- 172.21.15.105:0/1101774953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db00db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 -- 172.21.15.105:0/1101774953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 msgr2=0x55a6db0202d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 --2- 172.21.15.105:0/1101774953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db0202d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 -- 172.21.15.105:0/1101774953 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6dac3c170 con 0x55a6db003b80 2024-03-26T17:51:07.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9280efd640 1 --2- 172.21.15.105:0/1101774953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db002070 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f926c012fd0 tx=0x7f926c0044e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a56baac5eb01f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9266ffd640 1 -- 172.21.15.105:0/1101774953 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c009c10 con 0x55a6db003b80 2024-03-26T17:51:07.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9266ffd640 1 -- 172.21.15.105:0/1101774953 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f926c00fbf0 con 0x55a6db003b80 2024-03-26T17:51:07.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.712+0000 7f9266ffd640 1 -- 172.21.15.105:0/1101774953 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c019980 con 0x55a6db003b80 2024-03-26T17:51:07.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 -- 172.21.15.105:0/1101774953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 msgr2=0x55a6db002070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 --2- 172.21.15.105:0/1101774953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db002070 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f926c012fd0 tx=0x7f926c0044e0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 -- 172.21.15.105:0/1101774953 shutdown_connections 2024-03-26T17:51:07.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 --2- 172.21.15.105:0/1101774953 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db0202d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 --2- 172.21.15.105:0/1101774953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003b80 0x55a6db002070 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 --2- 172.21.15.105:0/1101774953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003190 0x55a6db00db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 -- 172.21.15.105:0/1101774953 >> 172.21.15.105:0/1101774953 conn(0x55a6daffe700 msgr2=0x55a6db001960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 -- 172.21.15.105:0/1101774953 shutdown_connections 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.713+0000 7f928662f740 1 -- 172.21.15.105:0/1101774953 wait complete. 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 1 -- start start 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 1 Processor -- start 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003190 0x55a6db012a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db012fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db013e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1ac440 con 0x55a6db008cf0 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1abd70 con 0x55a6db003190 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1a8fb0 con 0x55a6db003b80 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f928662f740 10 client.0 _ll_register_callbacks cb 0x7f927809bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:07.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f92816fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003190 0x55a6db012a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f9281eff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db013e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f9280efd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db012fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f92816fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003190 0x55a6db012a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60692/0 (socket says 172.21.15.105:60692) 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f9280efd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db012fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58394/0 (socket says 172.21.15.105:58394) 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.714+0000 7f92816fe640 1 -- 172.21.15.105:0/3944437721 learned_addr learned my addr 172.21.15.105:0/3944437721 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f92816fe640 1 -- 172.21.15.105:0/3944437721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 msgr2=0x55a6db012fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f92816fe640 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db012fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f92816fe640 1 -- 172.21.15.105:0/3944437721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 msgr2=0x55a6db013e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f92816fe640 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db013e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f92816fe640 1 -- 172.21.15.105:0/3944437721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6db0144c0 con 0x55a6db003190 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f9280efd640 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db012fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f9281eff640 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db013e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f92816fe640 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003190 0x55a6db012a90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f92780cd3c0 tx=0x7f92780cf4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32fce511a652e328 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f9264ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f9264ff9640 1 -- 172.21.15.105:0/3944437721 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92780a4690 con 0x55a6db003190 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f928662f740 1 -- 172.21.15.105:0/3944437721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926c00fbf0 con 0x55a6db003190 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f9264ff9640 1 -- 172.21.15.105:0/3944437721 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9278097c90 con 0x55a6db003190 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f9264ff9640 1 -- 172.21.15.105:0/3944437721 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f927809b690 con 0x55a6db003190 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f928662f740 10 client.30008 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f928662f740 1 -- 172.21.15.105:0/3944437721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a6db080360 con 0x55a6db003190 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.715+0000 7f9264ff9640 1 -- 172.21.15.105:0/3944437721 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f926c00fbf0 con 0x55a6db003190 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f9264ff9640 1 -- 172.21.15.105:0/3944437721 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9278086330 con 0x55a6db003190 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f928662f740 10 client.30008 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f928662f740 10 client.30008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f928662f740 20 client.30008 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f928662f740 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a6db080cf0 0x55a6db083110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f928662f740 4 client.30008 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:51:07.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f928662f740 1 -- 172.21.15.105:0/3944437721 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a6daf6fec0 con 0x55a6db080cf0 2024-03-26T17:51:07.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f9280efd640 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a6db080cf0 0x55a6db083110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f9280efd640 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a6db080cf0 0x55a6db083110 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24645ffa622154e4 server_cookie=cc57212627296913 in_seq=0 out_seq=0 2024-03-26T17:51:07.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.716+0000 7f9264ff9640 10 client.30008 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:07.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f9264ff9640 1 -- 172.21.15.105:0/3944437721 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a6daf6fec0 con 0x55a6db080cf0 2024-03-26T17:51:07.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f9264ff9640 10 client.30008 handle_command_reply: tid=0 2024-03-26T17:51:07.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 client.30008 shutdown 2024-03-26T17:51:07.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 -- 172.21.15.105:0/3944437721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003190 msgr2=0x55a6db012a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003190 0x55a6db012a90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f92780cd3c0 tx=0x7f92780cf4b0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 -- 172.21.15.105:0/3944437721 shutdown_connections 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a6db080cf0 0x55a6db083110 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a6db008cf0 0x55a6db013e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a6db003b80 0x55a6db012fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f9264ff9640 0 client.30008 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 --2- 172.21.15.105:0/3944437721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6db003190 0x55a6db012a90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 -- 172.21.15.105:0/3944437721 >> 172.21.15.105:0/3944437721 conn(0x55a6daffe700 msgr2=0x55a6db00d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 -- 172.21.15.105:0/3944437721 shutdown_connections 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 1 -- 172.21.15.105:0/3944437721 wait complete. 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.718+0000 7f928662f740 20 client.30008 trim_cache size 0 max 16384 2024-03-26T17:51:07.715 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:51:07.716 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:51:07.716 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "823fe247-fea0-4148-b49d-5d0ee9dd4a48", 2024-03-26T17:51:07.716 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:51:07.716 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 -- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92680a2f20 msgr2=0x7f92680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92680a2f20 0x7f92680a53e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9274004af0 tx=0x7f9274033450 comp rx=0 tx=0).stop 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 -- 172.21.15.105:0/3421377538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f927c0ff950 msgr2=0x7f927c199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f927c0ff950 0x7f927c199cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f927800e790 tx=0x7f927800ec50 comp rx=0 tx=0).stop 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 -- 172.21.15.105:0/3421377538 shutdown_connections 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f92680a2f20 0x7f92680a53e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f927c101e60 0x7f927c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f927c100310 0x7f927c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 --2- 172.21.15.105:0/3421377538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f927c0ff950 0x7f927c199cc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 -- 172.21.15.105:0/3421377538 >> 172.21.15.105:0/3421377538 conn(0x7f927c0fb4d0 msgr2=0x7f927c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 -- 172.21.15.105:0/3421377538 shutdown_connections 2024-03-26T17:51:07.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.719+0000 7f9283962640 1 -- 172.21.15.105:0/3421377538 wait complete. 2024-03-26T17:51:07.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:51:07.725 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:51:07.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/1408608495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41019b0 msgr2=0x7f0cf4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/1408608495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41019b0 0x7f0cf4101e30 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009900 tx=0x7f0cdc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:07.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/1408608495 shutdown_connections 2024-03-26T17:51:07.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/1408608495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf4102370 0x7f0cf4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/1408608495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41019b0 0x7f0cf4101e30 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/1408608495 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf41007c0 0x7f0cf4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/1408608495 >> 172.21.15.105:0/1408608495 conn(0x7f0cf40fbd50 msgr2=0x7f0cf40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/1408608495 shutdown_connections 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/1408608495 wait complete. 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.853+0000 7f0cf8bf3640 1 Processor -- start 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf8bf3640 1 -- start start 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf8bf3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41007c0 0x7f0cf4104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf8bf3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf41019b0 0x7f0cf4102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf8bf3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf4102370 0x7f0cf41032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf8bf3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410c2f0 con 0x7f0cf41007c0 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf8bf3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410c170 con 0x7f0cf4102370 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf8bf3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410c470 con 0x7f0cf41019b0 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41007c0 0x7f0cf4104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41007c0 0x7f0cf4104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43236/0 (socket says 172.21.15.105:43236) 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0ce9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf41019b0 0x7f0cf4102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf259c640 1 -- 172.21.15.105:0/857870988 learned_addr learned my addr 172.21.15.105:0/857870988 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf2d9d640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf4102370 0x7f0cf41032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/857870988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf4102370 msgr2=0x7f0cf41032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf4102370 0x7f0cf41032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41007c0 msgr2=0x7f0cf4104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41007c0 0x7f0cf4104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/857870988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf4103b60 con 0x7f0cf41019b0 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf2d9d640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf4102370 0x7f0cf41032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.854+0000 7f0cf259c640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41007c0 0x7f0cf4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.855+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf41019b0 0x7f0cf4102d90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc000c00 tx=0x7f0cdc005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.855+0000 7f0cebfff640 1 -- 172.21.15.105:0/857870988 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc043020 con 0x7f0cf41019b0 2024-03-26T17:51:07.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.855+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/857870988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf41a1c90 con 0x7f0cf41019b0 2024-03-26T17:51:07.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.855+0000 7f0cebfff640 1 -- 172.21.15.105:0/857870988 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cdc004350 con 0x7f0cf41019b0 2024-03-26T17:51:07.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.855+0000 7f0cebfff640 1 -- 172.21.15.105:0/857870988 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc003b50 con 0x7f0cf41019b0 2024-03-26T17:51:07.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.855+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/857870988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a2180 con 0x7f0cf41019b0 2024-03-26T17:51:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.857+0000 7f0cebfff640 1 -- 172.21.15.105:0/857870988 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0cdc046020 con 0x7f0cf41019b0 2024-03-26T17:51:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cfb7dc740 1 Processor -- start 2024-03-26T17:51:07.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cfb7dc740 1 -- start start 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b88f0 0x5556253b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b92e0 0x5556253bdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55562531d980 con 0x5556253b88f0 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5556252c5be0 con 0x5556253be450 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555625319930 con 0x5556253b92e0 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b92e0 0x5556253bdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b88f0 0x5556253b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b92e0 0x5556253bdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58430/0 (socket says 172.21.15.105:58430) 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b88f0 0x5556253b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43242/0 (socket says 172.21.15.105:43242) 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf259c640 1 -- 172.21.15.105:0/2842102693 learned_addr learned my addr 172.21.15.105:0/2842102693 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cebfff640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf259c640 1 -- 172.21.15.105:0/2842102693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 msgr2=0x5556253c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf259c640 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf259c640 1 -- 172.21.15.105:0/2842102693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b88f0 msgr2=0x5556253b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf259c640 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b88f0 0x5556253b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.858+0000 7f0cf259c640 1 -- 172.21.15.105:0/2842102693 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556250fb7d0 con 0x5556253b92e0 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.859+0000 7f0cf2d9d640 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b88f0 0x5556253b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.859+0000 7f0cebfff640 1 -- 172.21.15.105:0/857870988 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0cdc0f1490 con 0x7f0cf41019b0 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.859+0000 7f0cf2d9d640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.859+0000 7f0cf259c640 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b92e0 0x5556253bdde0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40115f0 tx=0x7f0ce4011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a588d608df16eef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.859+0000 7f0ce8d99640 1 -- 172.21.15.105:0/2842102693 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004020 con 0x5556253b92e0 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.859+0000 7f0ce8d99640 1 -- 172.21.15.105:0/2842102693 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce40041c0 con 0x5556253b92e0 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.859+0000 7f0ce8d99640 1 -- 172.21.15.105:0/2842102693 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40038b0 con 0x5556253b92e0 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/2842102693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b92e0 msgr2=0x5556253bdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cf2d9d640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5556253b9b40 tx=0x7f0cec004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b92e0 0x5556253bdde0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40115f0 tx=0x7f0ce4011ab0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/2842102693 shutdown_connections 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b92e0 0x5556253bdde0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/2842102693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b88f0 0x5556253b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/2842102693 >> 172.21.15.105:0/2842102693 conn(0x5556253b3e60 msgr2=0x5556253b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/2842102693 shutdown_connections 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/2842102693 wait complete. 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.860+0000 7f0cfb7dc740 1 -- start start 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cfb7dc740 1 Processor -- start 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253b92e0 0x5556253c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253be450 0x5556253ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5556253c1400 con 0x5556253be450 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5556252d9160 con 0x5556253b92e0 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58442/0 (socket says 172.21.15.105:58442) 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cf259c640 1 -- 172.21.15.105:0/1713884462 learned_addr learned my addr 172.21.15.105:0/1713884462 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5556253c1580 con 0x5556253b88f0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0cf2d9d640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253be450 0x5556253ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.861+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253b92e0 0x5556253c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cfb7dc740 10 client.0 _ll_register_callbacks cb 0x555625315720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/1713884462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 msgr2=0x5556253c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/1713884462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253be450 msgr2=0x5556253ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253be450 0x5556253ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/1713884462 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556250313f0 con 0x5556253b92e0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce40041c0 con 0x5556253b92e0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cf2d9d640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253be450 0x5556253ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cf259c640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253b92e0 0x5556253c3010 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0eefa0 tx=0x7f0cdc005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71db7d95bab7cc65 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cd9ffb640 10 client.30020 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/1713884462 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0bd950 con 0x5556253b92e0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/1713884462 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0cdc0ba1f0 con 0x5556253b92e0 2024-03-26T17:51:07.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/1713884462 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cdc0bd4e0 con 0x5556253b92e0 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cfb7dc740 10 client.30020 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.862+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=32}) -- 0x555624ff10f0 con 0x5556253b92e0 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.863+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/1713884462 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0cdc0f1080 con 0x5556253b92e0 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.863+0000 7f0cfb7dc740 10 client.30020 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.863+0000 7f0cfb7dc740 10 client.30020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.863+0000 7f0cfb7dc740 20 client.30020 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.863+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5556253ce770 0x5556253d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.863+0000 7f0cfb7dc740 4 client.30020 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.863+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55562532aec0 con 0x5556253ce770 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.863+0000 7f0cf259c640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5556253ce770 0x5556253d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cd9ffb640 10 client.30020 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:07.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cf259c640 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5556253ce770 0x5556253d0bb0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d18047d136af60 server_cookie=b04792d014d3a222 in_seq=0 out_seq=0 2024-03-26T17:51:07.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/1713884462 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55562532aec0 con 0x5556253ce770 2024-03-26T17:51:07.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cd9ffb640 10 client.30020 handle_command_reply: tid=0 2024-03-26T17:51:07.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 client.30020 shutdown 2024-03-26T17:51:07.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253b92e0 msgr2=0x5556253c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253b92e0 0x5556253c3010 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0eefa0 tx=0x7f0cdc005b70 comp rx=0 tx=0).stop 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 shutdown_connections 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5556253ce770 0x5556253d0bb0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253be450 0x5556253ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cd9ffb640 0 client.30020 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253b92e0 0x5556253c3010 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/1713884462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.864+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 >> 172.21.15.105:0/1713884462 conn(0x5556253b3e60 msgr2=0x5556253b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.865+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 shutdown_connections 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.865+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/1713884462 wait complete. 2024-03-26T17:51:07.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.865+0000 7f0cfb7dc740 20 client.30020 trim_cache size 0 max 16384 2024-03-26T17:51:07.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cfb7dc740 1 Processor -- start 2024-03-26T17:51:07.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cfb7dc740 1 -- start start 2024-03-26T17:51:07.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41aaf70 con 0x5556253b92e0 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41acab0 con 0x5556253be450 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41ab640 con 0x5556253b88f0 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0ce9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:58454/0 (socket says 172.21.15.105:58454) 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf259c640 1 -- 172.21.15.105:0/3968024120 learned_addr learned my addr 172.21.15.105:0/3968024120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf2d9d640 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf259c640 1 -- 172.21.15.105:0/3968024120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 msgr2=0x5556253b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf259c640 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf259c640 1 -- 172.21.15.105:0/3968024120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 msgr2=0x5556253ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf259c640 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.888+0000 7f0cf259c640 1 -- 172.21.15.105:0/3968024120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556253aee20 con 0x5556253b88f0 2024-03-26T17:51:07.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.889+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:07.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.889+0000 7f0cf259c640 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253b27f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40115f0 tx=0x7f0ce4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c36c3f75e1c6d168 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.889+0000 7f0cdbfff640 1 -- 172.21.15.105:0/3968024120 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400fda0 con 0x5556253b88f0 2024-03-26T17:51:07.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.889+0000 7f0cdbfff640 1 -- 172.21.15.105:0/3968024120 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce402e020 con 0x5556253b88f0 2024-03-26T17:51:07.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.889+0000 7f0cdbfff640 1 -- 172.21.15.105:0/3968024120 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4004610 con 0x5556253b88f0 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/3968024120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 msgr2=0x5556253b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253b27f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40115f0 tx=0x7f0ce4009310 comp rx=0 tx=0).stop 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/3968024120 shutdown_connections 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/3968024120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253b27f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/3968024120 >> 172.21.15.105:0/3968024120 conn(0x5556253b3e60 msgr2=0x5556253c2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/3968024120 shutdown_connections 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/3968024120 wait complete. 2024-03-26T17:51:07.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- start start 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 Processor -- start 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41ac3e0 con 0x5556253b92e0 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41abd10 con 0x5556253be450 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41a8f50 con 0x5556253b88f0 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cfb7dc740 10 client.0 _ll_register_callbacks cb 0x7f0cdc0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0ce9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cf259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0ce9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43264/0 (socket says 172.21.15.105:43264) 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/8690868 learned_addr learned my addr 172.21.15.105:0/8690868 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.890+0000 7f0cf2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/8690868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 msgr2=0x5556253c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/8690868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 msgr2=0x5556253c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0ce9d9b640 1 -- 172.21.15.105:0/8690868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556253c94b0 con 0x5556253b92e0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0ce9d9b640 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253c7f50 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0b9fe0 tx=0x7f0cdc0eee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42a3e0f71fb4919 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/8690868 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0da020 con 0x5556253b92e0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/8690868 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0cdc0f4a80 con 0x5556253b92e0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/8690868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce402e020 con 0x5556253b92e0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/8690868 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cdc0bd6c0 con 0x5556253b92e0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0cfb7dc740 10 client.20226 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/8690868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555625434c70 con 0x5556253b92e0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.891+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/8690868 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ce402e020 con 0x5556253b92e0 2024-03-26T17:51:07.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/8690868 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0cdc0b98d0 con 0x5556253b92e0 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cfb7dc740 10 client.20226 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cfb7dc740 10 client.20226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cfb7dc740 20 client.20226 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555625435490 0x5556254378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cfb7dc740 4 client.20226 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/8690868 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555625325210 con 0x555625435490 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cf259c640 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555625435490 0x5556254378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cf259c640 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555625435490 0x5556254378b0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e1339a44826b50 server_cookie=f6cc99c189d645b9 in_seq=0 out_seq=0 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.892+0000 7f0cd9ffb640 10 client.20226 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cd9ffb640 1 -- 172.21.15.105:0/8690868 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555625325210 con 0x555625435490 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cd9ffb640 10 client.20226 handle_command_reply: tid=0 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 client.20226 shutdown 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/8690868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 msgr2=0x5556253c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253c7f50 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0b9fe0 tx=0x7f0cdc0eee20 comp rx=0 tx=0).stop 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/8690868 shutdown_connections 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555625435490 0x5556254378b0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556253be450 0x5556253c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5556253b92e0 0x5556253c7f50 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 --2- 172.21.15.105:0/8690868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5556253b88f0 0x5556253c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/8690868 >> 172.21.15.105:0/8690868 conn(0x5556253b3e60 msgr2=0x5556253b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cd9ffb640 0 client.20226 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/8690868 shutdown_connections 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.893+0000 7f0cfb7dc740 1 -- 172.21.15.105:0/8690868 wait complete. 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cfb7dc740 20 client.20226 trim_cache size 0 max 16384 2024-03-26T17:51:07.891 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout: "823fe247-fea0-4148-b49d-5d0ee9dd4a48": { 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "823fe247-fea0-4148-b49d-5d0ee9dd4a48", 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:51:07.892 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:51:07.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 msgr2=0x7f0ce00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5556253b9b40 tx=0x7f0cec004a40 comp rx=0 tx=0).stop 2024-03-26T17:51:07.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/857870988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf41019b0 msgr2=0x7f0cf4102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:07.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf41019b0 0x7f0cf4102d90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc000c00 tx=0x7f0cdc005dc0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/857870988 shutdown_connections 2024-03-26T17:51:07.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ce00a2f20 0x7f0ce00a53e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0cf4102370 0x7f0cf41032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cf41019b0 0x7f0cf4102d90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 --2- 172.21.15.105:0/857870988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cf41007c0 0x7f0cf4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:07.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/857870988 >> 172.21.15.105:0/857870988 conn(0x7f0cf40fbd50 msgr2=0x7f0cf4068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:07.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/857870988 shutdown_connections 2024-03-26T17:51:07.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:07.894+0000 7f0cf8bf3640 1 -- 172.21.15.105:0/857870988 wait complete. 2024-03-26T17:51:07.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:07 smithi105 ceph-mon[32280]: pgmap v732: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:51:07.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:07 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:51:07.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:07.894 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:07 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:51:07.895 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:07.898 INFO:tasks.cephfs.filesystem:scrub status for tag:823fe247-fea0-4148-b49d-5d0ee9dd4a48 - {'path': '/', 'tag': '823fe247-fea0-4148-b49d-5d0ee9dd4a48', 'options': 'recursive,force'} 2024-03-26T17:51:08.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 416137 3.14 MB/sec execute 591 sec latency 24.563 ms 2024-03-26T17:51:08.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:07 smithi144 ceph-mon[36095]: pgmap v732: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:51:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:07 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:51:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:07 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:51:08.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:07 smithi135 ceph-mon[35636]: pgmap v732: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-03-26T17:51:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:07 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:51:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:07 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:51:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:09.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 416841 3.14 MB/sec execute 592 sec latency 18.757 ms 2024-03-26T17:51:10.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 417441 3.14 MB/sec execute 593 sec latency 17.469 ms 2024-03-26T17:51:10.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:09 smithi144 ceph-mon[36095]: pgmap v733: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.4 MiB/s wr, 94 op/s 2024-03-26T17:51:10.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:09 smithi105 ceph-mon[32280]: pgmap v733: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.4 MiB/s wr, 94 op/s 2024-03-26T17:51:10.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:09 smithi135 ceph-mon[35636]: pgmap v733: 97 pgs: 97 active+clean; 28 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.4 MiB/s wr, 94 op/s 2024-03-26T17:51:11.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 418306 3.15 MB/sec execute 594 sec latency 17.233 ms 2024-03-26T17:51:12.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 418701 3.15 MB/sec execute 595 sec latency 84.813 ms 2024-03-26T17:51:12.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:11 smithi144 ceph-mon[36095]: pgmap v734: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:51:12.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:11 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:51:12.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:11 smithi105 ceph-mon[32280]: pgmap v734: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:51:12.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:11 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:51:12.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:11 smithi135 ceph-mon[35636]: pgmap v734: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-03-26T17:51:12.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:11 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:51:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 -- 172.21.15.105:0/2687520858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f01023c0 msgr2=0x7f32f01047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 --2- 172.21.15.105:0/2687520858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f01023c0 0x7f32f01047b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009900 tx=0x7f32ec02bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 -- 172.21.15.105:0/2687520858 shutdown_connections 2024-03-26T17:51:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 --2- 172.21.15.105:0/2687520858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f0107620 0x7f32f0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 --2- 172.21.15.105:0/2687520858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f0104cf0 0x7f32f01070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 --2- 172.21.15.105:0/2687520858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f01023c0 0x7f32f01047b0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 -- 172.21.15.105:0/2687520858 >> 172.21.15.105:0/2687520858 conn(0x7f32f00fbd50 msgr2=0x7f32f00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 -- 172.21.15.105:0/2687520858 shutdown_connections 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 -- 172.21.15.105:0/2687520858 wait complete. 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.289+0000 7f32f7b13640 1 Processor -- start 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f7b13640 1 -- start start 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f7b13640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f01023c0 0x7f32f0199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f7b13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f0104cf0 0x7f32f019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f7b13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f0107620 0x7f32f01a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f7b13640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32f010d430 con 0x7f32f0104cf0 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f7b13640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32f010d2b0 con 0x7f32f01023c0 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f7b13640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32f010d5b0 con 0x7f32f0107620 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f0104cf0 0x7f32f019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f58af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f01023c0 0x7f32f0199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f0104cf0 0x7f32f019a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43272/0 (socket says 172.21.15.105:43272) 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 -- 172.21.15.105:0/3019235344 learned_addr learned my addr 172.21.15.105:0/3019235344 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f60b0640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f0107620 0x7f32f01a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 -- 172.21.15.105:0/3019235344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f0107620 msgr2=0x7f32f01a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f0107620 0x7f32f01a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 -- 172.21.15.105:0/3019235344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f01023c0 msgr2=0x7f32f0199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f01023c0 0x7f32f0199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 -- 172.21.15.105:0/3019235344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f01a18c0 con 0x7f32f0104cf0 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f58af640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f01023c0 0x7f32f0199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.290+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f0104cf0 0x7f32f019a1c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9c0 tx=0x7f32e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.291+0000 7f32e6ffd640 1 -- 172.21.15.105:0/3019235344 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd80 con 0x7f32f0104cf0 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.291+0000 7f32e6ffd640 1 -- 172.21.15.105:0/3019235344 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32e00054d0 con 0x7f32f0104cf0 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.291+0000 7f32f7b13640 1 -- 172.21.15.105:0/3019235344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f01a1bb0 con 0x7f32f0104cf0 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.291+0000 7f32e6ffd640 1 -- 172.21.15.105:0/3019235344 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0010430 con 0x7f32f0104cf0 2024-03-26T17:51:12.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.291+0000 7f32f7b13640 1 -- 172.21.15.105:0/3019235344 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f01a2100 con 0x7f32f0104cf0 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.293+0000 7f32e6ffd640 1 -- 172.21.15.105:0/3019235344 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f32e0004fb0 con 0x7f32f0104cf0 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.293+0000 7f32e6ffd640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32dc0a2fc0 0x7f32dc0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32f58af640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32dc0a2fc0 0x7f32dc0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32e6ffd640 1 -- 172.21.15.105:0/3019235344 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f32e00cf7e0 con 0x7f32f0104cf0 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32fa7d0740 1 Processor -- start 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32fa7d0740 1 -- start start 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32f58af640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32dc0a2fc0 0x7f32dc0a5480 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f32f019ab50 tx=0x7f32ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb674369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb67436fc0 0x55cb6743bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 0x55cb6743e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cb6739b0b0 con 0x55cb674365d0 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cb67343960 con 0x55cb6743c130 2024-03-26T17:51:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cb67397060 con 0x55cb67436fc0 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.294+0000 7f32f58af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb674369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f58af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb674369f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43278/0 (socket says 172.21.15.105:43278) 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f58af640 1 -- 172.21.15.105:0/475085872 learned_addr learned my addr 172.21.15.105:0/475085872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f60b0640 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 0x55cb6743e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f50ae640 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb67436fc0 0x55cb6743bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f50ae640 1 -- 172.21.15.105:0/475085872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 msgr2=0x55cb6743e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f50ae640 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 0x55cb6743e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f50ae640 1 -- 172.21.15.105:0/475085872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 msgr2=0x55cb674369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f50ae640 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb674369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f50ae640 1 -- 172.21.15.105:0/475085872 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb67355c60 con 0x55cb67436fc0 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f58af640 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb674369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32f50ae640 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb67436fc0 0x55cb6743bac0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f32e000c6a0 tx=0x7f32e009ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9848f6675b5b29d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32dbfff640 1 -- 172.21.15.105:0/475085872 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00b4020 con 0x55cb67436fc0 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32dbfff640 1 -- 172.21.15.105:0/475085872 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32e009bed0 con 0x55cb67436fc0 2024-03-26T17:51:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.295+0000 7f32dbfff640 1 -- 172.21.15.105:0/475085872 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0098540 con 0x55cb67436fc0 2024-03-26T17:51:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.296+0000 7f32fa7d0740 1 -- 172.21.15.105:0/475085872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb67436fc0 msgr2=0x55cb6743bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.296+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb67436fc0 0x55cb6743bac0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f32e000c6a0 tx=0x7f32e009ba10 comp rx=0 tx=0).stop 2024-03-26T17:51:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.296+0000 7f32fa7d0740 1 -- 172.21.15.105:0/475085872 shutdown_connections 2024-03-26T17:51:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.296+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 0x55cb6743e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.296+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb67436fc0 0x55cb6743bac0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.296+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/475085872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb674369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.296+0000 7f32fa7d0740 1 -- 172.21.15.105:0/475085872 >> 172.21.15.105:0/475085872 conn(0x55cb67431b40 msgr2=0x55cb67431f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.296+0000 7f32fa7d0740 1 -- 172.21.15.105:0/475085872 shutdown_connections 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 -- 172.21.15.105:0/475085872 wait complete. 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 -- start start 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 Processor -- start 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb674365d0 0x55cb674402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674407f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 0x55cb67447830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cb6743f080 con 0x55cb67436fc0 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cb67356db0 con 0x55cb6743c130 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cb6743f200 con 0x55cb674365d0 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32fa7d0740 10 client.0 _ll_register_callbacks cb 0x55cb6735b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32f50ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32f58af640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb674365d0 0x55cb674402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32f50ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674407f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43294/0 (socket says 172.21.15.105:43294) 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.297+0000 7f32f50ae640 1 -- 172.21.15.105:0/3350072628 learned_addr learned my addr 172.21.15.105:0/3350072628 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32f60b0640 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 0x55cb67447830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32f50ae640 1 -- 172.21.15.105:0/3350072628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb674365d0 msgr2=0x55cb674402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb674365d0 0x55cb674402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32f50ae640 1 -- 172.21.15.105:0/3350072628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 msgr2=0x55cb67447830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 0x55cb67447830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32f50ae640 1 -- 172.21.15.105:0/3350072628 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb670af210 con 0x55cb67436fc0 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32f58af640 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb674365d0 0x55cb674402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674407f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f32e00988a0 tx=0x7f32e009a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cceaade552f0815 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32d97fa640 1 -- 172.21.15.105:0/3350072628 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00b4070 con 0x55cb67436fc0 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32d97fa640 1 -- 172.21.15.105:0/3350072628 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f32e00a4660 con 0x55cb67436fc0 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3350072628 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e009bed0 con 0x55cb67436fc0 2024-03-26T17:51:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32d97fa640 1 -- 172.21.15.105:0/3350072628 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32e00a0020 con 0x55cb67436fc0 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32fa7d0740 10 client.20238 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3350072628 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cb6717b620 con 0x55cb67436fc0 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.298+0000 7f32d97fa640 1 -- 172.21.15.105:0/3350072628 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32e009bed0 con 0x55cb67436fc0 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32d97fa640 1 -- 172.21.15.105:0/3350072628 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f32e00b8030 con 0x55cb67436fc0 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32fa7d0740 10 client.20238 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32fa7d0740 10 client.20238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32fa7d0740 20 client.20238 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cb6744bd20 0x55cb6744e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32fa7d0740 4 client.20238 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3350072628 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb673a85f0 con 0x55cb6744bd20 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32f58af640 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cb6744bd20 0x55cb6744e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32d97fa640 10 client.20238 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.299+0000 7f32f58af640 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cb6744bd20 0x55cb6744e110 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56321d1e8f9a813 server_cookie=727ca6cb61641701 in_seq=0 out_seq=0 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32d97fa640 1 -- 172.21.15.105:0/3350072628 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cb673a85f0 con 0x55cb6744bd20 2024-03-26T17:51:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32d97fa640 10 client.20238 handle_command_reply: tid=0 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 client.20238 shutdown 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3350072628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 msgr2=0x55cb674407f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674407f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f32e00988a0 tx=0x7f32e009a3e0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3350072628 shutdown_connections 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cb6744bd20 0x55cb6744e110 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb6743c130 0x55cb67447830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674407f0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32d97fa640 0 client.20238 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3350072628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb674365d0 0x55cb674402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3350072628 >> 172.21.15.105:0/3350072628 conn(0x55cb67431b40 msgr2=0x55cb67430df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3350072628 shutdown_connections 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3350072628 wait complete. 2024-03-26T17:51:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.302+0000 7f32fa7d0740 20 client.20238 trim_cache size 0 max 16384 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.319+0000 7f32fa7d0740 1 Processor -- start 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32fa7d0740 1 -- start start 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb674365d0 0x55cb67453960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 0x55cb6743b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32f01aaff0 con 0x55cb67436fc0 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32f01acb30 con 0x55cb674365d0 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32f01ab6c0 con 0x55cb6743c130 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43304/0 (socket says 172.21.15.105:43304) 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f58af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb674365d0 0x55cb67453960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f60b0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 0x55cb6743b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 -- 172.21.15.105:0/263328438 learned_addr learned my addr 172.21.15.105:0/263328438 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 -- 172.21.15.105:0/263328438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 msgr2=0x55cb6743b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 --2- 172.21.15.105:0/263328438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 0x55cb6743b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 -- 172.21.15.105:0/263328438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb674365d0 msgr2=0x55cb67453960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 --2- 172.21.15.105:0/263328438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb674365d0 0x55cb67453960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 -- 172.21.15.105:0/263328438 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb6742d450 con 0x55cb67436fc0 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32f50ae640 1 --2- 172.21.15.105:0/263328438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674477f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f32e0098850 tx=0x7f32e0009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7792fe28c88e35c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.320+0000 7f32db7fe640 1 -- 172.21.15.105:0/263328438 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00b4070 con 0x55cb67436fc0 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32db7fe640 1 -- 172.21.15.105:0/263328438 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32e009bed0 con 0x55cb67436fc0 2024-03-26T17:51:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32db7fe640 1 -- 172.21.15.105:0/263328438 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00052a0 con 0x55cb67436fc0 2024-03-26T17:51:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 -- 172.21.15.105:0/263328438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 msgr2=0x55cb674477f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/263328438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674477f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f32e0098850 tx=0x7f32e0009700 comp rx=0 tx=0).stop 2024-03-26T17:51:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 -- 172.21.15.105:0/263328438 shutdown_connections 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/263328438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 0x55cb6743b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/263328438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb67436fc0 0x55cb674477f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/263328438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb674365d0 0x55cb67453960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 -- 172.21.15.105:0/263328438 >> 172.21.15.105:0/263328438 conn(0x55cb67431b40 msgr2=0x55cb674402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 -- 172.21.15.105:0/263328438 shutdown_connections 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.321+0000 7f32fa7d0740 1 -- 172.21.15.105:0/263328438 wait complete. 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 1 -- start start 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 1 Processor -- start 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb67444d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb67436fc0 0x55cb67445270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 0x55cb67446280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32f01ac460 con 0x55cb674365d0 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32f01abd90 con 0x55cb67436fc0 2024-03-26T17:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32f01a8fd0 con 0x55cb6743c130 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32fa7d0740 10 client.0 _ll_register_callbacks cb 0x7f32e80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32f58af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb67444d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32f50ae640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb67436fc0 0x55cb67445270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32f58af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb67444d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43310/0 (socket says 172.21.15.105:43310) 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32f58af640 1 -- 172.21.15.105:0/3183643459 learned_addr learned my addr 172.21.15.105:0/3183643459 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.322+0000 7f32f60b0640 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 0x55cb67446280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32f50ae640 1 -- 172.21.15.105:0/3183643459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 msgr2=0x55cb67446280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 0x55cb67446280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32f50ae640 1 -- 172.21.15.105:0/3183643459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 msgr2=0x55cb67444d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb67444d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32f50ae640 1 -- 172.21.15.105:0/3183643459 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb67446870 con 0x55cb67436fc0 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32f58af640 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb67444d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32f50ae640 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb67436fc0 0x55cb67445270 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f32e0098330 tx=0x7f32e000efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=287b2bd9bb0ba631 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32d97fa640 1 -- 172.21.15.105:0/3183643459 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00b4020 con 0x55cb67436fc0 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32d97fa640 1 -- 172.21.15.105:0/3183643459 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f32e00052a0 con 0x55cb67436fc0 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3183643459 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e009bed0 con 0x55cb67436fc0 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32d97fa640 1 -- 172.21.15.105:0/3183643459 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32e00bc0d0 con 0x55cb67436fc0 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32fa7d0740 10 client.30044 fetch_fsmap learned FSMap version 32 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3183643459 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cb67446870 con 0x55cb67436fc0 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.323+0000 7f32d97fa640 1 -- 172.21.15.105:0/3183643459 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32e009bed0 con 0x55cb67436fc0 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32d97fa640 1 -- 172.21.15.105:0/3183643459 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f32e00b8040 con 0x55cb67436fc0 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32fa7d0740 10 client.30044 fetch_fsmap finished waiting for FSMap version 32 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32fa7d0740 10 client.30044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32fa7d0740 20 client.30044 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cb674b2280 0x55cb674b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32fa7d0740 4 client.30044 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3183643459 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb673a2940 con 0x55cb674b2280 2024-03-26T17:51:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32f58af640 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cb674b2280 0x55cb674b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32d97fa640 10 client.30044 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.324+0000 7f32f58af640 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cb674b2280 0x55cb674b4670 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363157fd6a4ae2aa server_cookie=9f7f928462dc96f3 in_seq=0 out_seq=0 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32d97fa640 1 -- 172.21.15.105:0/3183643459 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb673a2940 con 0x55cb674b2280 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32d97fa640 10 client.30044 handle_command_reply: tid=0 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 client.30044 shutdown 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3183643459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb67436fc0 msgr2=0x55cb67445270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb67436fc0 0x55cb67445270 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f32e0098330 tx=0x7f32e000efb0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3183643459 shutdown_connections 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cb674b2280 0x55cb674b4670 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cb6743c130 0x55cb67446280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cb67436fc0 0x55cb67445270 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 --2- 172.21.15.105:0/3183643459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cb674365d0 0x55cb67444d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3183643459 >> 172.21.15.105:0/3183643459 conn(0x55cb67431b40 msgr2=0x55cb67437a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32d97fa640 0 client.30044 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3183643459 shutdown_connections 2024-03-26T17:51:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 1 -- 172.21.15.105:0/3183643459 wait complete. 2024-03-26T17:51:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.371+0000 7f32fa7d0740 20 client.30044 trim_cache size 0 max 16384 2024-03-26T17:51:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:51:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:51:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:51:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:51:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 -- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32dc0a2fc0 msgr2=0x7f32dc0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32dc0a2fc0 0x7f32dc0a5480 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f32f019ab50 tx=0x7f32ec005fd0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 -- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f0104cf0 msgr2=0x7f32f019a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f0104cf0 0x7f32f019a1c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9c0 tx=0x7f32e000ee80 comp rx=0 tx=0).stop 2024-03-26T17:51:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 -- 172.21.15.105:0/3019235344 shutdown_connections 2024-03-26T17:51:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32dc0a2fc0 0x7f32dc0a5480 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f0107620 0x7f32f01a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32f0104cf0 0x7f32f019a1c0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 --2- 172.21.15.105:0/3019235344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32f01023c0 0x7f32f0199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 -- 172.21.15.105:0/3019235344 >> 172.21.15.105:0/3019235344 conn(0x7f32f00fbd50 msgr2=0x7f32f0101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 -- 172.21.15.105:0/3019235344 shutdown_connections 2024-03-26T17:51:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:12.372+0000 7f32f7b13640 1 -- 172.21.15.105:0/3019235344 wait complete. 2024-03-26T17:51:13.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 419055 3.14 MB/sec execute 596 sec latency 94.518 ms 2024-03-26T17:51:14.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 419573 3.14 MB/sec execute 597 sec latency 23.204 ms 2024-03-26T17:51:14.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:13 smithi144 ceph-mon[36095]: pgmap v735: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:51:14.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:13 smithi105 ceph-mon[32280]: pgmap v735: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:51:14.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:13 smithi135 ceph-mon[35636]: pgmap v735: 97 pgs: 97 active+clean; 30 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-03-26T17:51:15.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 420206 3.14 MB/sec execute 598 sec latency 16.544 ms 2024-03-26T17:51:16.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 420908 3.14 MB/sec execute 599 sec latency 20.946 ms 2024-03-26T17:51:16.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:15 smithi144 ceph-mon[36095]: pgmap v736: 97 pgs: 97 active+clean; 31 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:51:16.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:15 smithi105 ceph-mon[32280]: pgmap v736: 97 pgs: 97 active+clean; 31 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:51:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:15 smithi135 ceph-mon[35636]: pgmap v736: 97 pgs: 97 active+clean; 31 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-03-26T17:51:17.035 INFO:tasks.workunit.client.0.smithi105.stdout: 1 cleanup 600 sec 2024-03-26T17:51:17.447 INFO:tasks.workunit.client.0.smithi105.stdout: 0 cleanup 600 sec 2024-03-26T17:51:17.447 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-03-26T17:51:17.447 INFO:tasks.workunit.client.0.smithi105.stdout: Operation Count AvgLat MaxLat 2024-03-26T17:51:17.447 INFO:tasks.workunit.client.0.smithi105.stdout: ---------------------------------------- 2024-03-26T17:51:17.447 INFO:tasks.workunit.client.0.smithi105.stdout: NTCreateX 60586 2.820 102.707 2024-03-26T17:51:17.448 INFO:tasks.workunit.client.0.smithi105.stdout: Close 44502 0.044 1.114 2024-03-26T17:51:17.448 INFO:tasks.workunit.client.0.smithi105.stdout: Rename 2568 8.596 97.263 2024-03-26T17:51:17.448 INFO:tasks.workunit.client.0.smithi105.stdout: Unlink 12239 3.163 87.444 2024-03-26T17:51:17.448 INFO:tasks.workunit.client.0.smithi105.stdout: Qpathinfo 54970 1.634 110.611 2024-03-26T17:51:17.448 INFO:tasks.workunit.client.0.smithi105.stdout: Qfileinfo 9588 0.072 0.381 2024-03-26T17:51:17.449 INFO:tasks.workunit.client.0.smithi105.stdout: Qfsinfo 10054 0.298 5.020 2024-03-26T17:51:17.449 INFO:tasks.workunit.client.0.smithi105.stdout: Sfileinfo 4950 0.625 3.229 2024-03-26T17:51:17.449 INFO:tasks.workunit.client.0.smithi105.stdout: Find 21219 11.796 105.669 2024-03-26T17:51:17.449 INFO:tasks.workunit.client.0.smithi105.stdout: WriteX 29934 0.158 1.508 2024-03-26T17:51:17.449 INFO:tasks.workunit.client.0.smithi105.stdout: ReadX 94975 0.004 0.573 2024-03-26T17:51:17.449 INFO:tasks.workunit.client.0.smithi105.stdout: LockX 198 0.645 1.674 2024-03-26T17:51:17.450 INFO:tasks.workunit.client.0.smithi105.stdout: UnlockX 198 0.631 1.216 2024-03-26T17:51:17.450 INFO:tasks.workunit.client.0.smithi105.stdout: Flush 4251 3.149 211.789 2024-03-26T17:51:17.450 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-03-26T17:51:17.450 INFO:tasks.workunit.client.0.smithi105.stdout:Throughput 3.14007 MB/sec 1 clients 1 procs max_latency=211.794 ms 2024-03-26T17:51:17.451 INFO:tasks.workunit.client.0.smithi105.stderr:+ dbench 10 2024-03-26T17:51:17.451 INFO:tasks.workunit.client.0.smithi105.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-03-26T17:51:17.452 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-03-26T17:51:17.452 INFO:tasks.workunit.client.0.smithi105.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-03-26T17:51:17.452 INFO:tasks.workunit.client.0.smithi105.stdout:8 of 10 processes prepared for launch 0 sec 2024-03-26T17:51:17.548 INFO:tasks.workunit.client.0.smithi105.stdout:10 of 10 processes prepared for launch 0 sec 2024-03-26T17:51:17.548 INFO:tasks.workunit.client.0.smithi105.stdout:releasing clients 2024-03-26T17:51:18.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:17 smithi144 ceph-mon[36095]: pgmap v737: 97 pgs: 97 active+clean; 26 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:51:18.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:17 smithi105 ceph-mon[32280]: pgmap v737: 97 pgs: 97 active+clean; 26 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:51:18.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:17 smithi135 ceph-mon[35636]: pgmap v737: 97 pgs: 97 active+clean; 26 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-03-26T17:51:18.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 211 82.34 MB/sec warmup 1 sec latency 55.332 ms 2024-03-26T17:51:19.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 363 64.97 MB/sec warmup 2 sec latency 65.408 ms 2024-03-26T17:51:20.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:19 smithi144 ceph-mon[36095]: pgmap v738: 97 pgs: 97 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 117 op/s 2024-03-26T17:51:20.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:19 smithi105 ceph-mon[32280]: pgmap v738: 97 pgs: 97 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 117 op/s 2024-03-26T17:51:20.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:19 smithi135 ceph-mon[35636]: pgmap v738: 97 pgs: 97 active+clean; 45 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 117 op/s 2024-03-26T17:51:20.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 506 58.87 MB/sec warmup 3 sec latency 91.037 ms 2024-03-26T17:51:21.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 668 52.83 MB/sec warmup 4 sec latency 181.074 ms 2024-03-26T17:51:22.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:21 smithi144 ceph-mon[36095]: pgmap v739: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.7 MiB/s wr, 130 op/s 2024-03-26T17:51:22.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:21 smithi105 ceph-mon[32280]: pgmap v739: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.7 MiB/s wr, 130 op/s 2024-03-26T17:51:22.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:21 smithi135 ceph-mon[35636]: pgmap v739: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.7 MiB/s wr, 130 op/s 2024-03-26T17:51:22.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 801 42.90 MB/sec warmup 5 sec latency 206.569 ms 2024-03-26T17:51:23.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:22 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:51:23.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:22 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:51:23.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:22 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:51:23.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 926 36.51 MB/sec warmup 6 sec latency 133.942 ms 2024-03-26T17:51:24.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:23 smithi105 ceph-mon[32280]: pgmap v740: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 13 MiB/s wr, 147 op/s 2024-03-26T17:51:24.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:23 smithi135 ceph-mon[35636]: pgmap v740: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 13 MiB/s wr, 147 op/s 2024-03-26T17:51:24.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:23 smithi144 ceph-mon[36095]: pgmap v740: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 13 MiB/s wr, 147 op/s 2024-03-26T17:51:24.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 1045 31.96 MB/sec warmup 7 sec latency 148.423 ms 2024-03-26T17:51:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 -- 172.21.15.105:0/2752734145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 msgr2=0x7f1548104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 --2- 172.21.15.105:0/2752734145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 0x7f1548104750 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1538009900 tx=0x7f153802bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 -- 172.21.15.105:0/2752734145 shutdown_connections 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 --2- 172.21.15.105:0/2752734145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15481075c0 0x7f1548109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 --2- 172.21.15.105:0/2752734145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1548104c90 0x7f1548107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 --2- 172.21.15.105:0/2752734145 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 0x7f1548104750 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 -- 172.21.15.105:0/2752734145 >> 172.21.15.105:0/2752734145 conn(0x7f15480fbd50 msgr2=0x7f15480fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 -- 172.21.15.105:0/2752734145 shutdown_connections 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.485+0000 7f154e7df640 1 -- 172.21.15.105:0/2752734145 wait complete. 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154e7df640 1 Processor -- start 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154e7df640 1 -- start start 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154e7df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 0x7f1548197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154e7df640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1548104c90 0x7f1548197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154e7df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15481075c0 0x7f154819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154e7df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f154810d430 con 0x7f1548102360 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154e7df640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f154810d2b0 con 0x7f1548104c90 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154e7df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f154810d5b0 con 0x7f15481075c0 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f1547fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 0x7f1548197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f154cd7c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15481075c0 0x7f154819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15477fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1548104c90 0x7f1548197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.486+0000 7f1547fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 0x7f1548197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59902/0 (socket says 172.21.15.105:59902) 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f1547fff640 1 -- 172.21.15.105:0/1036655744 learned_addr learned my addr 172.21.15.105:0/1036655744 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f154cd7c640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15481075c0 0x7f154819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55420/0 (socket says 172.21.15.105:55420) 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15477fe640 1 -- 172.21.15.105:0/1036655744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15481075c0 msgr2=0x7f154819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15477fe640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15481075c0 0x7f154819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15477fe640 1 -- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 msgr2=0x7f1548197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15477fe640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 0x7f1548197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15477fe640 1 -- 172.21.15.105:0/1036655744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15481001d0 con 0x7f1548104c90 2024-03-26T17:51:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f154cd7c640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15481075c0 0x7f154819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15477fe640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1548104c90 0x7f1548197fc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9c0 tx=0x7f153c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15457fa640 1 -- 172.21.15.105:0/1036655744 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c00ccc0 con 0x7f1548104c90 2024-03-26T17:51:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f154e7df640 1 -- 172.21.15.105:0/1036655744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15481004c0 con 0x7f1548104c90 2024-03-26T17:51:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15457fa640 1 -- 172.21.15.105:0/1036655744 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153c0108e0 con 0x7f1548104c90 2024-03-26T17:51:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f15457fa640 1 -- 172.21.15.105:0/1036655744 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c0187a0 con 0x7f1548104c90 2024-03-26T17:51:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.487+0000 7f154e7df640 1 -- 172.21.15.105:0/1036655744 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1548100a60 con 0x7f1548104c90 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15513c7740 1 Processor -- start 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15513c7740 1 -- start start 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15513c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15513c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1bbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15513c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1bc130 0x55d63b1be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15457fa640 1 -- 172.21.15.105:0/1036655744 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f153c010a90 con 0x7f1548104c90 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15513c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d63b11b0b0 con 0x55d63b1b6fc0 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15513c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d63b0c3960 con 0x55d63b1bc130 2024-03-26T17:51:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f15513c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d63b117060 con 0x55d63b1b65d0 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f154cd7c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f154cd7c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55428/0 (socket says 172.21.15.105:55428) 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.490+0000 7f154cd7c640 1 -- 172.21.15.105:0/3524364586 learned_addr learned my addr 172.21.15.105:0/3524364586 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f15477fe640 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1bc130 0x55d63b1be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f1547fff640 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1bbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f154cd7c640 1 -- 172.21.15.105:0/3524364586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1bc130 msgr2=0x55d63b1be5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f154cd7c640 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1bc130 0x55d63b1be5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f154cd7c640 1 -- 172.21.15.105:0/3524364586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 msgr2=0x55d63b1bbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f154cd7c640 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1bbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f154cd7c640 1 -- 172.21.15.105:0/3524364586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d63b0d5c60 con 0x55d63b1b65d0 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f1547fff640 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1bbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f15457fa640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15300a2fc0 0x7f15300a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f154cd7c640 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1b69f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f154000f760 tx=0x7f154000fc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2978aa18bbec626b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f1547fff640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15300a2fc0 0x7f15300a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f15457fa640 1 -- 172.21.15.105:0/1036655744 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f153c0cf780 con 0x7f1548104c90 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f152e7fc640 1 -- 172.21.15.105:0/3524364586 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540004d20 con 0x55d63b1b65d0 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f152e7fc640 1 -- 172.21.15.105:0/3524364586 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f154000ee40 con 0x55d63b1b65d0 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.491+0000 7f152e7fc640 1 -- 172.21.15.105:0/3524364586 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400042b0 con 0x55d63b1b65d0 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.492+0000 7f1547fff640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15300a2fc0 0x7f15300a5480 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55d63b1b7a30 tx=0x7f153802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.492+0000 7f15513c7740 1 -- 172.21.15.105:0/3524364586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 msgr2=0x55d63b1b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.492+0000 7f15513c7740 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1b69f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f154000f760 tx=0x7f154000fc20 comp rx=0 tx=0).stop 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.492+0000 7f15513c7740 1 -- 172.21.15.105:0/3524364586 shutdown_connections 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.492+0000 7f15513c7740 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1bc130 0x55d63b1be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.492+0000 7f15513c7740 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1bbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.492+0000 7f15513c7740 1 --2- 172.21.15.105:0/3524364586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1b69f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.492+0000 7f15513c7740 1 -- 172.21.15.105:0/3524364586 >> 172.21.15.105:0/3524364586 conn(0x55d63b1b1b40 msgr2=0x55d63b1b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 -- 172.21.15.105:0/3524364586 shutdown_connections 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 -- 172.21.15.105:0/3524364586 wait complete. 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 -- start start 2024-03-26T17:51:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 Processor -- start 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1bc130 0x55d63b1c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d63b1bf0e0 con 0x55d63b1b6fc0 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d63b0d6db0 con 0x55d63b1b65d0 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.493+0000 7f15513c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d63b1bf260 con 0x55d63b1bc130 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1c0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35124/0 (socket says 172.21.15.105:35124) 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 -- 172.21.15.105:0/2632034018 learned_addr learned my addr 172.21.15.105:0/2632034018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f154cd7c640 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1bc130 0x55d63b1c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f15477fe640 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f15513c7740 10 client.0 _ll_register_callbacks cb 0x55d63b0db110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 -- 172.21.15.105:0/2632034018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1bc130 msgr2=0x55d63b1c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1bc130 0x55d63b1c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 -- 172.21.15.105:0/2632034018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 msgr2=0x55d63b1c0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 -- 172.21.15.105:0/2632034018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d63ae2f210 con 0x55d63b1b65d0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f15477fe640 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f1547fff640 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1c0410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1538036040 tx=0x7f153802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae25ffca48e34fdc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f15277fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f15277fe640 1 -- 172.21.15.105:0/2632034018 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153804f030 con 0x55d63b1b65d0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f15277fe640 1 -- 172.21.15.105:0/2632034018 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f153803a040 con 0x55d63b1b65d0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.494+0000 7f15513c7740 1 -- 172.21.15.105:0/2632034018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d63b1285f0 con 0x55d63b1b65d0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15277fe640 1 -- 172.21.15.105:0/2632034018 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f15380046c0 con 0x55d63b1b65d0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15513c7740 10 client.30056 fetch_fsmap learned FSMap version 33 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15513c7740 1 -- 172.21.15.105:0/2632034018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d63aefb620 con 0x55d63b1b65d0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15277fe640 1 -- 172.21.15.105:0/2632034018 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d63b1285f0 con 0x55d63b1b65d0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15277fe640 1 -- 172.21.15.105:0/2632034018 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f1538003af0 con 0x55d63b1b65d0 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15513c7740 10 client.30056 fetch_fsmap finished waiting for FSMap version 33 2024-03-26T17:51:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15513c7740 10 client.30056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15513c7740 20 client.30056 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15513c7740 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d63b1cc270 0x55d63b1ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15513c7740 4 client.30056 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15513c7740 1 -- 172.21.15.105:0/2632034018 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d63b122940 con 0x55d63b1cc270 2024-03-26T17:51:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.495+0000 7f15477fe640 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d63b1cc270 0x55d63b1ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.496+0000 7f15277fe640 10 client.30056 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.496+0000 7f15477fe640 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d63b1cc270 0x55d63b1ce710 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7e51550a2357a8 server_cookie=a15322ef2ceb18c1 in_seq=0 out_seq=0 2024-03-26T17:51:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15277fe640 1 -- 172.21.15.105:0/2632034018 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d63b122940 con 0x55d63b1cc270 2024-03-26T17:51:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15277fe640 10 client.30056 handle_command_reply: tid=0 2024-03-26T17:51:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 client.30056 shutdown 2024-03-26T17:51:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 -- 172.21.15.105:0/2632034018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 msgr2=0x55d63b1c0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1c0410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1538036040 tx=0x7f153802bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 -- 172.21.15.105:0/2632034018 shutdown_connections 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d63b1cc270 0x55d63b1ce710 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1bc130 0x55d63b1c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15277fe640 0 client.30056 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 --2- 172.21.15.105:0/2632034018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1c0410 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 -- 172.21.15.105:0/2632034018 >> 172.21.15.105:0/2632034018 conn(0x55d63b1b1b40 msgr2=0x55d63b1b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 -- 172.21.15.105:0/2632034018 shutdown_connections 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 1 -- 172.21.15.105:0/2632034018 wait complete. 2024-03-26T17:51:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.497+0000 7f15513c7740 20 client.30056 trim_cache size 0 max 16384 2024-03-26T17:51:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.517+0000 7f15513c7740 1 Processor -- start 2024-03-26T17:51:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.517+0000 7f15513c7740 1 -- start start 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15513c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15513c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15513c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1bf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15513c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15481a6a50 con 0x55d63b1bc130 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15513c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15481a8620 con 0x55d63b1b65d0 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15513c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15481a7120 con 0x55d63b1b6fc0 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f154cd7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15477fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f1547fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f154cd7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1bf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59936/0 (socket says 172.21.15.105:59936) 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15477fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55454/0 (socket says 172.21.15.105:55454) 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f154cd7c640 1 -- 172.21.15.105:0/3576732702 learned_addr learned my addr 172.21.15.105:0/3576732702 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f1547fff640 1 -- 172.21.15.105:0/3576732702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b6fc0 msgr2=0x55d63b1c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f1547fff640 1 --2- 172.21.15.105:0/3576732702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f1547fff640 1 -- 172.21.15.105:0/3576732702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 msgr2=0x55d63b1bf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f1547fff640 1 --2- 172.21.15.105:0/3576732702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1bf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f1547fff640 1 -- 172.21.15.105:0/3576732702 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d63b1bbb10 con 0x55d63b1b65d0 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f15477fe640 1 --2- 172.21.15.105:0/3576732702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f1547fff640 1 --2- 172.21.15.105:0/3576732702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1b82f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1538000c00 tx=0x7f1538004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=247e55b932631b35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.518+0000 7f152dffb640 1 -- 172.21.15.105:0/3576732702 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153804f070 con 0x55d63b1b65d0 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f152dffb640 1 -- 172.21.15.105:0/3576732702 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1538062070 con 0x55d63b1b65d0 2024-03-26T17:51:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f152dffb640 1 -- 172.21.15.105:0/3576732702 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538003b40 con 0x55d63b1b65d0 2024-03-26T17:51:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 -- 172.21.15.105:0/3576732702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 msgr2=0x55d63b1b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 --2- 172.21.15.105:0/3576732702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1b82f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1538000c00 tx=0x7f1538004460 comp rx=0 tx=0).stop 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 -- 172.21.15.105:0/3576732702 shutdown_connections 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 --2- 172.21.15.105:0/3576732702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1bf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 --2- 172.21.15.105:0/3576732702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 --2- 172.21.15.105:0/3576732702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b65d0 0x55d63b1b82f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 -- 172.21.15.105:0/3576732702 >> 172.21.15.105:0/3576732702 conn(0x55d63b1b1b40 msgr2=0x55d63b1c0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 -- 172.21.15.105:0/3576732702 shutdown_connections 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.519+0000 7f15513c7740 1 -- 172.21.15.105:0/3576732702 wait complete. 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 1 -- start start 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 1 Processor -- start 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15481a7f50 con 0x55d63b1bc130 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15481a77f0 con 0x55d63b1b6fc0 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15481a4a30 con 0x55d63b1b65d0 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15513c7740 10 client.0 _ll_register_callbacks cb 0x7f1538044050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f1547fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f154cd7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f1547fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55456/0 (socket says 172.21.15.105:55456) 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f1547fff640 1 -- 172.21.15.105:0/739090014 learned_addr learned my addr 172.21.15.105:0/739090014 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.520+0000 7f15477fe640 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f1547fff640 1 -- 172.21.15.105:0/739090014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b6fc0 msgr2=0x55d63b1c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f1547fff640 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f1547fff640 1 -- 172.21.15.105:0/739090014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 msgr2=0x55d63b1c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f1547fff640 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1c6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f1547fff640 1 -- 172.21.15.105:0/739090014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d63b1c6730 con 0x55d63b1b65d0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f1547fff640 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1c4c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1538003f00 tx=0x7f1538003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea84c6fcbc387d2e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f15277fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f15277fe640 1 -- 172.21.15.105:0/739090014 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153804f040 con 0x55d63b1b65d0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f15513c7740 1 -- 172.21.15.105:0/739090014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1538062070 con 0x55d63b1b65d0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f15277fe640 1 -- 172.21.15.105:0/739090014 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f1538004860 con 0x55d63b1b65d0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f15277fe640 1 -- 172.21.15.105:0/739090014 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1538004a60 con 0x55d63b1b65d0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f15513c7740 10 client.30046 fetch_fsmap learned FSMap version 33 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f15513c7740 1 -- 172.21.15.105:0/739090014 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d63b1c7a40 con 0x55d63b1b65d0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.521+0000 7f15277fe640 1 -- 172.21.15.105:0/739090014 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1538062070 con 0x55d63b1b65d0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15277fe640 1 -- 172.21.15.105:0/739090014 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f1538071480 con 0x55d63b1b65d0 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15513c7740 10 client.30046 fetch_fsmap finished waiting for FSMap version 33 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15513c7740 10 client.30046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15513c7740 20 client.30046 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15513c7740 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d63b232bb0 0x55d63b234fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15513c7740 4 client.30046 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:51:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15513c7740 1 -- 172.21.15.105:0/739090014 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d63b0a0770 con 0x55d63b232bb0 2024-03-26T17:51:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15477fe640 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d63b232bb0 0x55d63b234fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15277fe640 10 client.30046 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.522+0000 7f15477fe640 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d63b232bb0 0x55d63b234fd0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857dfcdf0ed389ef server_cookie=f08ebce013ed01a3 in_seq=0 out_seq=0 2024-03-26T17:51:25.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 1182 29.46 MB/sec warmup 8 sec latency 143.409 ms 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15277fe640 1 -- 172.21.15.105:0/739090014 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d63b0a0770 con 0x55d63b232bb0 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15277fe640 10 client.30046 handle_command_reply: tid=0 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 client.30046 shutdown 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 -- 172.21.15.105:0/739090014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 msgr2=0x55d63b1c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1c4c40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1538003f00 tx=0x7f1538003f90 comp rx=0 tx=0).stop 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 -- 172.21.15.105:0/739090014 shutdown_connections 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d63b232bb0 0x55d63b234fd0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15277fe640 0 client.30046 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d63b1bc130 0x55d63b1c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d63b1b6fc0 0x55d63b1c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 --2- 172.21.15.105:0/739090014 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d63b1b65d0 0x55d63b1c4c40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 -- 172.21.15.105:0/739090014 >> 172.21.15.105:0/739090014 conn(0x55d63b1b1b40 msgr2=0x55d63b1b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 -- 172.21.15.105:0/739090014 shutdown_connections 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 1 -- 172.21.15.105:0/739090014 wait complete. 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.760+0000 7f15513c7740 20 client.30046 trim_cache size 0 max 16384 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:51:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:51:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 -- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15300a2fc0 msgr2=0x7f15300a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15300a2fc0 0x7f15300a5480 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55d63b1b7a30 tx=0x7f153802c7e0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 -- 172.21.15.105:0/1036655744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1548104c90 msgr2=0x7f1548197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1548104c90 0x7f1548197fc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9c0 tx=0x7f153c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:51:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 -- 172.21.15.105:0/1036655744 shutdown_connections 2024-03-26T17:51:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f15300a2fc0 0x7f15300a5480 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15481075c0 0x7f154819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1548104c90 0x7f1548197fc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 --2- 172.21.15.105:0/1036655744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1548102360 0x7f1548197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.761+0000 7f154e7df640 1 -- 172.21.15.105:0/1036655744 >> 172.21.15.105:0/1036655744 conn(0x7f15480fbd50 msgr2=0x7f1548102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.762+0000 7f154e7df640 1 -- 172.21.15.105:0/1036655744 shutdown_connections 2024-03-26T17:51:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:25.762+0000 7f154e7df640 1 -- 172.21.15.105:0/1036655744 wait complete. 2024-03-26T17:51:26.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:25 smithi105 ceph-mon[32280]: pgmap v741: 97 pgs: 97 active+clean; 203 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 17 MiB/s wr, 164 op/s 2024-03-26T17:51:26.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:25 smithi135 ceph-mon[35636]: pgmap v741: 97 pgs: 97 active+clean; 203 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 17 MiB/s wr, 164 op/s 2024-03-26T17:51:26.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:25 smithi144 ceph-mon[36095]: pgmap v741: 97 pgs: 97 active+clean; 203 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 17 MiB/s wr, 164 op/s 2024-03-26T17:51:26.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 1306 26.33 MB/sec warmup 9 sec latency 234.265 ms 2024-03-26T17:51:27.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 1443 24.00 MB/sec warmup 10 sec latency 121.809 ms 2024-03-26T17:51:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:27 smithi105 ceph-mon[32280]: pgmap v742: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 18 MiB/s wr, 221 op/s 2024-03-26T17:51:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:27 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:51:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:27 smithi135 ceph-mon[35636]: pgmap v742: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 18 MiB/s wr, 221 op/s 2024-03-26T17:51:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:27 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:51:28.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:27 smithi144 ceph-mon[36095]: pgmap v742: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 18 MiB/s wr, 221 op/s 2024-03-26T17:51:28.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:27 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:51:28.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 1637 24.40 MB/sec warmup 11 sec latency 120.120 ms 2024-03-26T17:51:29.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 1779 22.65 MB/sec warmup 12 sec latency 107.456 ms 2024-03-26T17:51:30.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:29 smithi105 ceph-mon[32280]: pgmap v743: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 19 MiB/s wr, 264 op/s 2024-03-26T17:51:30.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:29 smithi135 ceph-mon[35636]: pgmap v743: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 19 MiB/s wr, 264 op/s 2024-03-26T17:51:30.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:29 smithi144 ceph-mon[36095]: pgmap v743: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 19 MiB/s wr, 264 op/s 2024-03-26T17:51:30.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 1895 21.06 MB/sec warmup 13 sec latency 135.568 ms 2024-03-26T17:51:31.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 2102 20.20 MB/sec warmup 14 sec latency 104.927 ms 2024-03-26T17:51:32.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:31 smithi105 ceph-mon[32280]: pgmap v744: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 17 MiB/s wr, 313 op/s 2024-03-26T17:51:32.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:31 smithi135 ceph-mon[35636]: pgmap v744: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 17 MiB/s wr, 313 op/s 2024-03-26T17:51:32.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:31 smithi144 ceph-mon[36095]: pgmap v744: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 17 MiB/s wr, 313 op/s 2024-03-26T17:51:32.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 2249 19.12 MB/sec warmup 15 sec latency 114.941 ms 2024-03-26T17:51:33.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 2392 18.03 MB/sec warmup 16 sec latency 102.019 ms 2024-03-26T17:51:34.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:33 smithi105 ceph-mon[32280]: pgmap v745: 97 pgs: 97 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 15 MiB/s wr, 315 op/s 2024-03-26T17:51:34.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:33 smithi135 ceph-mon[35636]: pgmap v745: 97 pgs: 97 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 15 MiB/s wr, 315 op/s 2024-03-26T17:51:34.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:33 smithi144 ceph-mon[36095]: pgmap v745: 97 pgs: 97 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 15 MiB/s wr, 315 op/s 2024-03-26T17:51:34.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 2535 17.23 MB/sec warmup 17 sec latency 108.360 ms 2024-03-26T17:51:35.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 2706 16.91 MB/sec warmup 18 sec latency 107.178 ms 2024-03-26T17:51:36.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:35 smithi105 ceph-mon[32280]: pgmap v746: 97 pgs: 97 active+clean; 217 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 8.0 MiB/s wr, 313 op/s 2024-03-26T17:51:36.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:35 smithi135 ceph-mon[35636]: pgmap v746: 97 pgs: 97 active+clean; 217 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 8.0 MiB/s wr, 313 op/s 2024-03-26T17:51:36.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:35 smithi144 ceph-mon[36095]: pgmap v746: 97 pgs: 97 active+clean; 217 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 8.0 MiB/s wr, 313 op/s 2024-03-26T17:51:36.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 2804 16.24 MB/sec warmup 19 sec latency 111.303 ms 2024-03-26T17:51:37.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 2910 15.61 MB/sec warmup 20 sec latency 134.873 ms 2024-03-26T17:51:37.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:51:37.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:51:38.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 -- 172.21.15.105:0/1524816905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810c2d0 msgr2=0x7fa32810c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 --2- 172.21.15.105:0/1524816905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810c2d0 0x7fa32810c6f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa324009900 tx=0x7fa32402bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:38.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 -- 172.21.15.105:0/1524816905 shutdown_connections 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 --2- 172.21.15.105:0/1524816905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32810de80 0x7fa328114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 --2- 172.21.15.105:0/1524816905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32810d4c0 0x7fa32810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 --2- 172.21.15.105:0/1524816905 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810c2d0 0x7fa32810c6f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 -- 172.21.15.105:0/1524816905 >> 172.21.15.105:0/1524816905 conn(0x7fa32806c280 msgr2=0x7fa32806c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 -- 172.21.15.105:0/1524816905 shutdown_connections 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.053+0000 7fa32fe62640 1 -- 172.21.15.105:0/1524816905 wait complete. 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32fe62640 1 Processor -- start 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32fe62640 1 -- start start 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32fe62640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32810c2d0 0x7fa3281a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32fe62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32810d4c0 0x7fa3281a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32fe62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810de80 0x7fa3281a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32fe62640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa328117e10 con 0x7fa32810de80 2024-03-26T17:51:38.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32fe62640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa328117c90 con 0x7fa32810d4c0 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32fe62640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa328117f90 con 0x7fa32810c2d0 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32d3fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32810d4c0 0x7fa3281a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32d3fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32810d4c0 0x7fa3281a2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33934/0 (socket says 172.21.15.105:33934) 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32d3fd640 1 -- 172.21.15.105:0/2603789279 learned_addr learned my addr 172.21.15.105:0/2603789279 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32e3ff640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810de80 0x7fa3281a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.054+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32810c2d0 0x7fa3281a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32d3fd640 1 -- 172.21.15.105:0/2603789279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32810c2d0 msgr2=0x7fa3281a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32810c2d0 0x7fa3281a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32d3fd640 1 -- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810de80 msgr2=0x7fa3281a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810de80 0x7fa3281a9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32d3fd640 1 -- 172.21.15.105:0/2603789279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3281aa150 con 0x7fa32810d4c0 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32810c2d0 0x7fa3281a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32e3ff640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810de80 0x7fa3281a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32810d4c0 0x7fa3281a2a70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa31800ea60 tx=0x7fa31800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa31effd640 1 -- 172.21.15.105:0/2603789279 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31800cde0 con 0x7fa32810d4c0 2024-03-26T17:51:38.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32fe62640 1 -- 172.21.15.105:0/2603789279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3281aa440 con 0x7fa32810d4c0 2024-03-26T17:51:38.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa31effd640 1 -- 172.21.15.105:0/2603789279 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3180035f0 con 0x7fa32810d4c0 2024-03-26T17:51:38.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa31effd640 1 -- 172.21.15.105:0/2603789279 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318010690 con 0x7fa32810d4c0 2024-03-26T17:51:38.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.055+0000 7fa32fe62640 1 -- 172.21.15.105:0/2603789279 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3281aa9e0 con 0x7fa32810d4c0 2024-03-26T17:51:38.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.058+0000 7fa332a45740 1 Processor -- start 2024-03-26T17:51:38.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa332a45740 1 -- start start 2024-03-26T17:51:38.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa31effd640 1 -- 172.21.15.105:0/2603789279 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa3180037a0 con 0x7fa32810d4c0 2024-03-26T17:51:38.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa332a45740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f597d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa332a45740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f5982e0 0x56545f59cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa332a45740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f59d450 0x56545f59f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa332a45740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56545f4fc980 con 0x56545f59d450 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa332a45740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56545f4a4be0 con 0x56545f5978f0 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa332a45740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56545f4f8930 con 0x56545f5982e0 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32e3ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f597d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32dbfe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f5982e0 0x56545f59cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32e3ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f597d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33942/0 (socket says 172.21.15.105:33942) 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32e3ff640 1 -- 172.21.15.105:0/325951644 learned_addr learned my addr 172.21.15.105:0/325951644 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f59d450 0x56545f59f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32e3ff640 1 -- 172.21.15.105:0/325951644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f5982e0 msgr2=0x56545f59cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32e3ff640 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f5982e0 0x56545f59cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32e3ff640 1 -- 172.21.15.105:0/325951644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f59d450 msgr2=0x56545f59f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32e3ff640 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f59d450 0x56545f59f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32e3ff640 1 -- 172.21.15.105:0/325951644 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56545f2da7d0 con 0x56545f5978f0 2024-03-26T17:51:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f59d450 0x56545f59f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:51:38.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.059+0000 7fa31effd640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3140a2f20 0x7fa3140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.060+0000 7fa32e3ff640 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f597d10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa3200117e0 tx=0x7fa320011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea87f6fd5eefdba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.060+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3140a2f20 0x7fa3140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.060+0000 7fa31effd640 1 -- 172.21.15.105:0/2603789279 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa3180cf690 con 0x7fa32810d4c0 2024-03-26T17:51:38.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.060+0000 7fa313fff640 1 -- 172.21.15.105:0/325951644 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320011310 con 0x56545f5978f0 2024-03-26T17:51:38.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.060+0000 7fa313fff640 1 -- 172.21.15.105:0/325951644 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa320003e20 con 0x56545f5978f0 2024-03-26T17:51:38.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.060+0000 7fa313fff640 1 -- 172.21.15.105:0/325951644 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200044f0 con 0x56545f5978f0 2024-03-26T17:51:38.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3140a2f20 0x7fa3140a53e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa324005ab0 tx=0x7fa324005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 -- 172.21.15.105:0/325951644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 msgr2=0x56545f597d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f597d10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa3200117e0 tx=0x7fa320011ca0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 -- 172.21.15.105:0/325951644 shutdown_connections 2024-03-26T17:51:38.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f59d450 0x56545f59f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f5982e0 0x56545f59cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 --2- 172.21.15.105:0/325951644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f597d10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 -- 172.21.15.105:0/325951644 >> 172.21.15.105:0/325951644 conn(0x56545f592e60 msgr2=0x56545f593260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 -- 172.21.15.105:0/325951644 shutdown_connections 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.061+0000 7fa332a45740 1 -- 172.21.15.105:0/325951644 wait complete. 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 1 -- start start 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 1 Processor -- start 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f5aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5982e0 0x56545f5aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56545f5a04a0 con 0x56545f5982e0 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56545f4b8160 con 0x56545f5978f0 2024-03-26T17:51:38.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56545f5a0620 con 0x56545f59d450 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa332a45740 10 client.0 _ll_register_callbacks cb 0x56545f4f4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa32dbfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f5aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa32e3ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa32dbfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f5aa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33952/0 (socket says 172.21.15.105:33952) 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa32dbfe640 1 -- 172.21.15.105:0/3688957725 learned_addr learned my addr 172.21.15.105:0/3688957725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.062+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5982e0 0x56545f5aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa32dbfe640 1 -- 172.21.15.105:0/3688957725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 msgr2=0x56545f5b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa32dbfe640 1 -- 172.21.15.105:0/3688957725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5982e0 msgr2=0x56545f5aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5982e0 0x56545f5aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa32dbfe640 1 -- 172.21.15.105:0/3688957725 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56545f2103f0 con 0x56545f5978f0 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa32e3ff640 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f5aa3a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa324009900 tx=0x7fa32402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4447c76277c8569c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa3117fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:38.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa3117fa640 1 -- 172.21.15.105:0/3688957725 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324050030 con 0x56545f5978f0 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa3117fa640 1 -- 172.21.15.105:0/3688957725 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa32403a040 con 0x56545f5978f0 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.063+0000 7fa332a45740 1 -- 172.21.15.105:0/3688957725 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa320003e20 con 0x56545f5978f0 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa3117fa640 1 -- 172.21.15.105:0/3688957725 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa324004990 con 0x56545f5978f0 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa332a45740 10 client.30074 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa332a45740 1 -- 172.21.15.105:0/3688957725 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56545f1d00f0 con 0x56545f5978f0 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa3117fa640 1 -- 172.21.15.105:0/3688957725 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa320003e20 con 0x56545f5978f0 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa3117fa640 1 -- 172.21.15.105:0/3688957725 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa3240574e0 con 0x56545f5978f0 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa332a45740 10 client.30074 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa332a45740 10 client.30074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa332a45740 20 client.30074 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa332a45740 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56545f5b6200 0x56545f5b8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa332a45740 4 client.30074 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa332a45740 1 -- 172.21.15.105:0/3688957725 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56545f509ec0 con 0x56545f5b6200 2024-03-26T17:51:38.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.064+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56545f5b6200 0x56545f5b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.065+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56545f5b6200 0x56545f5b8680 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1e4814c9a5afb8 server_cookie=ddbb0f19d0d01f72 in_seq=0 out_seq=0 2024-03-26T17:51:38.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.065+0000 7fa3117fa640 10 client.30074 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa3117fa640 1 -- 172.21.15.105:0/3688957725 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56545f509ec0 con 0x56545f5b6200 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa3117fa640 10 client.30074 handle_command_reply: tid=0 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 client.30074 shutdown 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 -- 172.21.15.105:0/3688957725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 msgr2=0x56545f5aa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f5aa3a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa324009900 tx=0x7fa32402bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 -- 172.21.15.105:0/3688957725 shutdown_connections 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56545f5b6200 0x56545f5b8680 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5982e0 0x56545f5aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa3117fa640 0 client.30074 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 --2- 172.21.15.105:0/3688957725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5978f0 0x56545f5aa3a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.067+0000 7fa332a45740 1 -- 172.21.15.105:0/3688957725 >> 172.21.15.105:0/3688957725 conn(0x56545f592e60 msgr2=0x56545f592080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.068+0000 7fa332a45740 1 -- 172.21.15.105:0/3688957725 shutdown_connections 2024-03-26T17:51:38.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.068+0000 7fa332a45740 1 -- 172.21.15.105:0/3688957725 wait complete. 2024-03-26T17:51:38.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.068+0000 7fa332a45740 20 client.30074 trim_cache size 0 max 16384 2024-03-26T17:51:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.089+0000 7fa332a45740 1 Processor -- start 2024-03-26T17:51:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.089+0000 7fa332a45740 1 -- start start 2024-03-26T17:51:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.089+0000 7fa332a45740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5978f0 0x56545f5aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.089+0000 7fa332a45740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.089+0000 7fa332a45740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.089+0000 7fa332a45740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281af470 con 0x56545f5978f0 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.089+0000 7fa332a45740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281b1070 con 0x56545f5982e0 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.089+0000 7fa332a45740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281afb40 con 0x56545f59d450 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5978f0 0x56545f5aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32e3ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5978f0 0x56545f5aa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46274/0 (socket says 172.21.15.105:46274) 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32d3fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 -- 172.21.15.105:0/2352924424 learned_addr learned my addr 172.21.15.105:0/2352924424 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 -- 172.21.15.105:0/2352924424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 msgr2=0x56545f5aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2352924424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 -- 172.21.15.105:0/2352924424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 msgr2=0x56545f5b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2352924424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 -- 172.21.15.105:0/2352924424 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56545f58e8d0 con 0x56545f5978f0 2024-03-26T17:51:38.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32e3ff640 1 --2- 172.21.15.105:0/2352924424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2352924424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5978f0 0x56545f5aa070 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa32402c170 tx=0x7fa32402c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37b45e7366b14d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa3137fe640 1 -- 172.21.15.105:0/2352924424 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324050070 con 0x56545f5978f0 2024-03-26T17:51:38.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa3137fe640 1 -- 172.21.15.105:0/2352924424 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa324063070 con 0x56545f5978f0 2024-03-26T17:51:38.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.090+0000 7fa3137fe640 1 -- 172.21.15.105:0/2352924424 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa324003850 con 0x56545f5978f0 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 -- 172.21.15.105:0/2352924424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5978f0 msgr2=0x56545f5aa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 --2- 172.21.15.105:0/2352924424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5978f0 0x56545f5aa070 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa32402c170 tx=0x7fa32402c330 comp rx=0 tx=0).stop 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 -- 172.21.15.105:0/2352924424 shutdown_connections 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 --2- 172.21.15.105:0/2352924424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 --2- 172.21.15.105:0/2352924424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 --2- 172.21.15.105:0/2352924424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5978f0 0x56545f5aa070 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 -- 172.21.15.105:0/2352924424 >> 172.21.15.105:0/2352924424 conn(0x56545f592e60 msgr2=0x56545f593240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 -- 172.21.15.105:0/2352924424 shutdown_connections 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.091+0000 7fa332a45740 1 -- 172.21.15.105:0/2352924424 wait complete. 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 1 -- start start 2024-03-26T17:51:38.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 1 Processor -- start 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5b03c0 0x56545f5b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281b09a0 con 0x56545f5b03c0 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281b0210 con 0x56545f5982e0 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281ad450 con 0x56545f59d450 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa332a45740 10 client.0 _ll_register_callbacks cb 0x7fa324044050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa32d3fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa32dbfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa32d3fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46500/0 (socket says 172.21.15.105:46500) 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa32d3fd640 1 -- 172.21.15.105:0/2593828935 learned_addr learned my addr 172.21.15.105:0/2593828935 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa32dbfe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33976/0 (socket says 172.21.15.105:33976) 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.092+0000 7fa32e3ff640 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5b03c0 0x56545f5b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa32d3fd640 1 -- 172.21.15.105:0/2593828935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 msgr2=0x56545f5aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa32d3fd640 1 -- 172.21.15.105:0/2593828935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5b03c0 msgr2=0x56545f5b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5b03c0 0x56545f5b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa32d3fd640 1 -- 172.21.15.105:0/2593828935 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56545f5b0d80 con 0x56545f59d450 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa32d3fd640 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5af400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3180981e0 tx=0x7fa31809bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ccdba4f99cf2a53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa3117fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa3117fa640 1 -- 172.21.15.105:0/2593828935 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31809c480 con 0x56545f59d450 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa3117fa640 1 -- 172.21.15.105:0/2593828935 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa3180983f0 con 0x56545f59d450 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa332a45740 1 -- 172.21.15.105:0/2593828935 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa324063070 con 0x56545f59d450 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa3117fa640 1 -- 172.21.15.105:0/2593828935 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa31809bd80 con 0x56545f59d450 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa332a45740 10 client.30064 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa332a45740 1 -- 172.21.15.105:0/2593828935 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56545f61c860 con 0x56545f59d450 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa3117fa640 1 -- 172.21.15.105:0/2593828935 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa324063070 con 0x56545f59d450 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.093+0000 7fa3117fa640 1 -- 172.21.15.105:0/2593828935 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa31809b1d0 con 0x56545f59d450 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa332a45740 10 client.30064 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa332a45740 10 client.30064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa332a45740 20 client.30064 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa332a45740 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56545f61d2b0 0x56545f61f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa332a45740 4 client.30064 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:51:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa332a45740 1 -- 172.21.15.105:0/2593828935 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56545f504210 con 0x56545f61d2b0 2024-03-26T17:51:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56545f61d2b0 0x56545f61f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa32dbfe640 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56545f61d2b0 0x56545f61f6d0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b350487fee081b server_cookie=786c1d76a2562d2b in_seq=0 out_seq=0 2024-03-26T17:51:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.094+0000 7fa3117fa640 10 client.30064 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:38.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.096+0000 7fa3117fa640 1 -- 172.21.15.105:0/2593828935 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56545f504210 con 0x56545f61d2b0 2024-03-26T17:51:38.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.096+0000 7fa3117fa640 10 client.30064 handle_command_reply: tid=0 2024-03-26T17:51:38.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.096+0000 7fa332a45740 1 client.30064 shutdown 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.096+0000 7fa332a45740 1 -- 172.21.15.105:0/2593828935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 msgr2=0x56545f5af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.096+0000 7fa332a45740 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5af400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa3180981e0 tx=0x7fa31809bc50 comp rx=0 tx=0).stop 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.096+0000 7fa332a45740 1 -- 172.21.15.105:0/2593828935 shutdown_connections 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.096+0000 7fa332a45740 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56545f61d2b0 0x56545f61f6d0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa332a45740 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56545f5b03c0 0x56545f5b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa3117fa640 0 client.30064 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa332a45740 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56545f59d450 0x56545f5af400 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa332a45740 1 --2- 172.21.15.105:0/2593828935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56545f5982e0 0x56545f5aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa332a45740 1 -- 172.21.15.105:0/2593828935 >> 172.21.15.105:0/2593828935 conn(0x56545f592e60 msgr2=0x56545f5aa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa332a45740 1 -- 172.21.15.105:0/2593828935 shutdown_connections 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa332a45740 1 -- 172.21.15.105:0/2593828935 wait complete. 2024-03-26T17:51:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa332a45740 20 client.30064 trim_cache size 0 max 16384 2024-03-26T17:51:38.092 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:51:38.092 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:51:38.092 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:51:38.092 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 -- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3140a2f20 msgr2=0x7fa3140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3140a2f20 0x7fa3140a53e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa324005ab0 tx=0x7fa324005fd0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 -- 172.21.15.105:0/2603789279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32810d4c0 msgr2=0x7fa3281a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32810d4c0 0x7fa3281a2a70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa31800ea60 tx=0x7fa31800ef20 comp rx=0 tx=0).stop 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 -- 172.21.15.105:0/2603789279 shutdown_connections 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3140a2f20 0x7fa3140a53e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa32810de80 0x7fa3281a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa32810d4c0 0x7fa3281a2a70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 --2- 172.21.15.105:0/2603789279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa32810c2d0 0x7fa3281a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.097+0000 7fa32fe62640 1 -- 172.21.15.105:0/2603789279 >> 172.21.15.105:0/2603789279 conn(0x7fa32806c280 msgr2=0x7fa3281108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.098+0000 7fa32fe62640 1 -- 172.21.15.105:0/2603789279 shutdown_connections 2024-03-26T17:51:38.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.098+0000 7fa32fe62640 1 -- 172.21.15.105:0/2603789279 wait complete. 2024-03-26T17:51:38.102 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:51:38.102 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:51:38.131 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:37 smithi105 ceph-mon[32280]: pgmap v747: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-03-26T17:51:38.149 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:37 smithi144 ceph-mon[36095]: pgmap v747: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-03-26T17:51:38.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:37 smithi135 ceph-mon[35636]: pgmap v747: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 -- 172.21.15.105:0/2900272044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1023c0 msgr2=0x7f913c1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 --2- 172.21.15.105:0/2900272044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1023c0 0x7f913c1047b0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9138009900 tx=0x7f913802bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 -- 172.21.15.105:0/2900272044 shutdown_connections 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 --2- 172.21.15.105:0/2900272044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f913c107620 0x7f913c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 --2- 172.21.15.105:0/2900272044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f913c104cf0 0x7f913c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 --2- 172.21.15.105:0/2900272044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c1023c0 0x7f913c1047b0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 -- 172.21.15.105:0/2900272044 >> 172.21.15.105:0/2900272044 conn(0x7f913c0fbd50 msgr2=0x7f913c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 -- 172.21.15.105:0/2900272044 shutdown_connections 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.230+0000 7f914409a640 1 -- 172.21.15.105:0/2900272044 wait complete. 2024-03-26T17:51:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f914409a640 1 Processor -- start 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f914409a640 1 -- start start 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f914409a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f913c1023c0 0x7f913c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f914409a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f913c104cf0 0x7f913c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f914409a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c107620 0x7f913c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f914409a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f913c10d430 con 0x7f913c107620 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f914409a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f913c10d2b0 con 0x7f913c1023c0 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f914409a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f913c10d5b0 con 0x7f913c104cf0 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f9142637640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c107620 0x7f913c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f9141e36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f913c1023c0 0x7f913c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f9141e36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f913c1023c0 0x7f913c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34000/0 (socket says 172.21.15.105:34000) 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f9142637640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c107620 0x7f913c19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46304/0 (socket says 172.21.15.105:46304) 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f9141e36640 1 -- 172.21.15.105:0/3707062842 learned_addr learned my addr 172.21.15.105:0/3707062842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.231+0000 7f9141635640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f913c104cf0 0x7f913c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9142637640 1 -- 172.21.15.105:0/3707062842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f913c104cf0 msgr2=0x7f913c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9142637640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f913c104cf0 0x7f913c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9142637640 1 -- 172.21.15.105:0/3707062842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f913c1023c0 msgr2=0x7f913c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9142637640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f913c1023c0 0x7f913c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9142637640 1 -- 172.21.15.105:0/3707062842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c19f780 con 0x7f913c107620 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9141635640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f913c104cf0 0x7f913c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9141e36640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f913c1023c0 0x7f913c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9142637640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c107620 0x7f913c19f110 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9134009fd0 tx=0x7f913400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9132ffd640 1 -- 172.21.15.105:0/3707062842 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913400ce20 con 0x7f913c107620 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9132ffd640 1 -- 172.21.15.105:0/3707062842 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9134005020 con 0x7f913c107620 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c19fa70 con 0x7f913c107620 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f9132ffd640 1 -- 172.21.15.105:0/3707062842 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9134010690 con 0x7f913c107620 2024-03-26T17:51:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.232+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c19ff70 con 0x7f913c107620 2024-03-26T17:51:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.233+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913c0fd840 con 0x7f913c107620 2024-03-26T17:51:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.234+0000 7f9132ffd640 1 -- 172.21.15.105:0/3707062842 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f91340051d0 con 0x7f913c107620 2024-03-26T17:51:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.235+0000 7f9132ffd640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91280a2f20 0x7f91280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.235+0000 7f9141e36640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91280a2f20 0x7f91280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.235+0000 7f9132ffd640 1 -- 172.21.15.105:0/3707062842 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f91340cfaf0 con 0x7f913c107620 2024-03-26T17:51:38.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.235+0000 7f9141e36640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91280a2f20 0x7f91280a53e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f91380096c0 tx=0x7f913802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.240+0000 7f9132ffd640 1 -- 172.21.15.105:0/3707062842 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f913409c8e0 con 0x7f913c107620 2024-03-26T17:51:38.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 3042 15.16 MB/sec warmup 21 sec latency 187.465 ms 2024-03-26T17:51:38.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.578+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f913c02d240 con 0x7f913c107620 2024-03-26T17:51:38.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.579+0000 7f9132ffd640 1 -- 172.21.15.105:0/3707062842 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11948 (secure 0 0 0) 0x7f913409c1e0 con 0x7f913c107620 2024-03-26T17:51:38.570 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:51:38.571 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:51:26.802032+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:51:38.571 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 34 2024-03-26T17:51:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91280a2f20 msgr2=0x7f91280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91280a2f20 0x7f91280a53e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f91380096c0 tx=0x7f913802c7e0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c107620 msgr2=0x7f913c19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c107620 0x7f913c19f110 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9134009fd0 tx=0x7f913400ef10 comp rx=0 tx=0).stop 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 shutdown_connections 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f91280a2f20 0x7f91280a53e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f913c107620 0x7f913c19f110 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f913c104cf0 0x7f913c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 --2- 172.21.15.105:0/3707062842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f913c1023c0 0x7f913c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 >> 172.21.15.105:0/3707062842 conn(0x7f913c0fbd50 msgr2=0x7f913c102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 shutdown_connections 2024-03-26T17:51:38.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.582+0000 7f914409a640 1 -- 172.21.15.105:0/3707062842 wait complete. 2024-03-26T17:51:38.586 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:51:38.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 -- 172.21.15.105:0/151738044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb01012a0 msgr2=0x7fafb0101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 --2- 172.21.15.105:0/151738044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb01012a0 0x7fafb0101b20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009860 tx=0x7fafa002bb70 comp rx=0 tx=0).stop 2024-03-26T17:51:38.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 -- 172.21.15.105:0/151738044 shutdown_connections 2024-03-26T17:51:38.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 --2- 172.21.15.105:0/151738044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb01012a0 0x7fafb0101b20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 --2- 172.21.15.105:0/151738044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb01008e0 0x7fafb0100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 --2- 172.21.15.105:0/151738044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafb00ffb80 0x7fafb00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 -- 172.21.15.105:0/151738044 >> 172.21.15.105:0/151738044 conn(0x7fafb00fb4b0 msgr2=0x7fafb00fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 -- 172.21.15.105:0/151738044 shutdown_connections 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.774+0000 7fafb663f640 1 -- 172.21.15.105:0/151738044 wait complete. 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.775+0000 7fafb663f640 1 Processor -- start 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.775+0000 7fafb663f640 1 -- start start 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.775+0000 7fafb663f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafb00ffb80 0x7fafb01064c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.775+0000 7fafb663f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb01008e0 0x7fafb0106a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.775+0000 7fafb663f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb01012a0 0x7fafb0103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.775+0000 7fafb663f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010f5f0 con 0x7fafb00ffb80 2024-03-26T17:51:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.775+0000 7fafb663f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010f470 con 0x7fafb01012a0 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.775+0000 7fafb663f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010f770 con 0x7fafb01008e0 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafb00ffb80 0x7fafb01064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafb00ffb80 0x7fafb01064c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46326/0 (socket says 172.21.15.105:46326) 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafaffff640 1 -- 172.21.15.105:0/1868866558 learned_addr learned my addr 172.21.15.105:0/1868866558 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafb4bdc640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb01012a0 0x7fafb0103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb01008e0 0x7fafb0106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafb4bdc640 1 -- 172.21.15.105:0/1868866558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb01008e0 msgr2=0x7fafb0106a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafb4bdc640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb01008e0 0x7fafb0106a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafb4bdc640 1 -- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafb00ffb80 msgr2=0x7fafb01064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafb4bdc640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafb00ffb80 0x7fafb01064c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafb4bdc640 1 -- 172.21.15.105:0/1868866558 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb01038f0 con 0x7fafb01012a0 2024-03-26T17:51:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafaffff640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafb00ffb80 0x7fafb01064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafb4bdc640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb01012a0 0x7fafb0103030 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fafa002c180 tx=0x7fafa00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.776+0000 7fafad7fa640 1 -- 172.21.15.105:0/1868866558 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0041030 con 0x7fafb01012a0 2024-03-26T17:51:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.777+0000 7fafb663f640 1 -- 172.21.15.105:0/1868866558 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb01a6110 con 0x7fafb01012a0 2024-03-26T17:51:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.777+0000 7fafb663f640 1 -- 172.21.15.105:0/1868866558 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb01a64c0 con 0x7fafb01012a0 2024-03-26T17:51:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.777+0000 7fafad7fa640 1 -- 172.21.15.105:0/1868866558 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa002c9c0 con 0x7fafb01012a0 2024-03-26T17:51:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.777+0000 7fafad7fa640 1 -- 172.21.15.105:0/1868866558 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0039040 con 0x7fafb01012a0 2024-03-26T17:51:38.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafb921f740 1 Processor -- start 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafad7fa640 1 -- 172.21.15.105:0/1868866558 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fafa0044070 con 0x7fafb01012a0 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafb921f740 1 -- start start 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafb921f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafb921f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafb921f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7dda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafb921f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb73a970 con 0x55d3fb7db590 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafb921f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb6e2d70 con 0x55d3fb7d5a30 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.780+0000 7fafb921f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb736920 con 0x55d3fb7d6420 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7d5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34032/0 (socket says 172.21.15.105:34032) 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 -- 172.21.15.105:0/472509942 learned_addr learned my addr 172.21.15.105:0/472509942 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/472509942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7dda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafad7fa640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf980a2f20 0x7faf980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 -- 172.21.15.105:0/472509942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 msgr2=0x55d3fb7daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/472509942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7daf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 -- 172.21.15.105:0/472509942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 msgr2=0x55d3fb7dda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/472509942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7dda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 -- 172.21.15.105:0/472509942 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3fb44e3f0 con 0x55d3fb7d5a30 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafad7fa640 1 -- 172.21.15.105:0/1868866558 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fafa00f0900 con 0x7fafb01012a0 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/472509942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7d5e50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fafa40058c0 tx=0x7fafa4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830cffee88bb5869 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7fafb4bdc640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf980a2f20 0x7faf980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.781+0000 7faf967fc640 1 -- 172.21.15.105:0/472509942 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004050 con 0x55d3fb7d5a30 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7faf967fc640 1 -- 172.21.15.105:0/472509942 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa400fc70 con 0x55d3fb7d5a30 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7faf967fc640 1 -- 172.21.15.105:0/472509942 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4003a10 con 0x55d3fb7d5a30 2024-03-26T17:51:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 -- 172.21.15.105:0/472509942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 msgr2=0x55d3fb7d5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 --2- 172.21.15.105:0/472509942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7d5e50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fafa40058c0 tx=0x7fafa4005850 comp rx=0 tx=0).stop 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb4bdc640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf980a2f20 0x7faf980a53e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fafa0005820 tx=0x7fafa0009170 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 -- 172.21.15.105:0/472509942 shutdown_connections 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 --2- 172.21.15.105:0/472509942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7dda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 --2- 172.21.15.105:0/472509942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 --2- 172.21.15.105:0/472509942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7d5e50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 -- 172.21.15.105:0/472509942 >> 172.21.15.105:0/472509942 conn(0x55d3fb7d0fa0 msgr2=0x55d3fb7d13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 -- 172.21.15.105:0/472509942 shutdown_connections 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.782+0000 7fafb921f740 1 -- 172.21.15.105:0/472509942 wait complete. 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.783+0000 7fafb921f740 1 -- start start 2024-03-26T17:51:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.783+0000 7fafb921f740 1 Processor -- start 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafb921f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafb921f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafb921f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb7e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafb921f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb7de540 con 0x55d3fb7d6420 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafb921f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb6f6310 con 0x55d3fb7d5a30 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafb921f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3fb7de6c0 con 0x55d3fb7db590 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafb921f740 10 client.0 _ll_register_callbacks cb 0x55d3fb731c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46344/0 (socket says 172.21.15.105:46344) 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaffff640 1 -- 172.21.15.105:0/305795828 learned_addr learned my addr 172.21.15.105:0/305795828 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb7e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaf7fe640 1 -- 172.21.15.105:0/305795828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 msgr2=0x55d3fb7dff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7dff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaf7fe640 1 -- 172.21.15.105:0/305795828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d6420 msgr2=0x55d3fb7e0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaf7fe640 1 -- 172.21.15.105:0/305795828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3fb5187d0 con 0x55d3fb7db590 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaffff640 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb7e74b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009cb0 tx=0x7fafa4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0eb0ab4ea6f8094 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.784+0000 7faf8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.785+0000 7faf8f7fe640 1 -- 172.21.15.105:0/305795828 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4003a10 con 0x55d3fb7db590 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.785+0000 7fafb921f740 1 -- 172.21.15.105:0/305795828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa400fc70 con 0x55d3fb7db590 2024-03-26T17:51:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.785+0000 7faf8f7fe640 1 -- 172.21.15.105:0/305795828 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fafa40037b0 con 0x55d3fb7db590 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.785+0000 7faf8f7fe640 1 -- 172.21.15.105:0/305795828 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fafa4034050 con 0x55d3fb7db590 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.785+0000 7fafb921f740 10 client.30082 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.785+0000 7fafb921f740 1 -- 172.21.15.105:0/305795828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3fb7ea820 con 0x55d3fb7db590 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.785+0000 7faf8f7fe640 1 -- 172.21.15.105:0/305795828 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa400fc70 con 0x55d3fb7db590 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.786+0000 7faf8f7fe640 1 -- 172.21.15.105:0/305795828 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fafa402c020 con 0x55d3fb7db590 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.786+0000 7fafb921f740 10 client.30082 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.786+0000 7fafb921f740 10 client.30082 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.787+0000 7fafb921f740 20 client.30082 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.787+0000 7fafb921f740 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d3fb7ebde0 0x55d3fb7ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.787+0000 7fafb921f740 4 client.30082 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.787+0000 7fafb921f740 1 -- 172.21.15.105:0/305795828 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3fb747eb0 con 0x55d3fb7ebde0 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.787+0000 7fafaffff640 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d3fb7ebde0 0x55d3fb7ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.788+0000 7fafaffff640 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d3fb7ebde0 0x55d3fb7ee240 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4ec4495e3649a690 server_cookie=be17f3bcb4a8fbd0 in_seq=0 out_seq=0 2024-03-26T17:51:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.788+0000 7faf8f7fe640 10 client.30082 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:51:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7faf8f7fe640 1 -- 172.21.15.105:0/305795828 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d3fb747eb0 con 0x55d3fb7ebde0 2024-03-26T17:51:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7faf8f7fe640 10 client.30082 handle_command_reply: tid=0 2024-03-26T17:51:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 client.30082 shutdown 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 -- 172.21.15.105:0/305795828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 msgr2=0x55d3fb7e74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb7e74b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009cb0 tx=0x7fafa4005850 comp rx=0 tx=0).stop 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 -- 172.21.15.105:0/305795828 shutdown_connections 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d3fb7ebde0 0x55d3fb7ee240 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb7e74b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 --2- 172.21.15.105:0/305795828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7dff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 -- 172.21.15.105:0/305795828 >> 172.21.15.105:0/305795828 conn(0x55d3fb7d0fa0 msgr2=0x55d3fb7d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7faf8f7fe640 0 client.30082 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:51:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.789+0000 7fafb921f740 1 -- 172.21.15.105:0/305795828 shutdown_connections 2024-03-26T17:51:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.790+0000 7fafb921f740 1 -- 172.21.15.105:0/305795828 wait complete. 2024-03-26T17:51:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.790+0000 7fafb921f740 20 client.30082 trim_cache size 0 max 16384 2024-03-26T17:51:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafb921f740 1 Processor -- start 2024-03-26T17:51:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafb921f740 1 -- start start 2024-03-26T17:51:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafb921f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafb921f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafb921f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafb921f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00e530 con 0x55d3fb7db590 2024-03-26T17:51:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafb921f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c010130 con 0x55d3fb7d5a30 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafb921f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00ec00 con 0x55d3fb7d6420 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafaffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafaf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafaffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46562/0 (socket says 172.21.15.105:46562) 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafaffff640 1 -- 172.21.15.105:0/1257961008 learned_addr learned my addr 172.21.15.105:0/1257961008 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.815+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/1257961008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7fafaf7fe640 1 -- 172.21.15.105:0/1257961008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 msgr2=0x55d3fb7e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/1257961008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7fafaf7fe640 1 -- 172.21.15.105:0/1257961008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 msgr2=0x55d3fb7cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/1257961008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7fafaf7fe640 1 -- 172.21.15.105:0/1257961008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3fb7d1d10 con 0x55d3fb7db590 2024-03-26T17:51:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/1257961008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7cfe70 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fafa4004340 tx=0x7fafa4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d89c705eac27a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7faf95ffb640 1 -- 172.21.15.105:0/1257961008 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa400fc70 con 0x55d3fb7db590 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7faf95ffb640 1 -- 172.21.15.105:0/1257961008 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa4003800 con 0x55d3fb7db590 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.816+0000 7faf95ffb640 1 -- 172.21.15.105:0/1257961008 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004050 con 0x55d3fb7db590 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 -- 172.21.15.105:0/1257961008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 msgr2=0x55d3fb7cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 --2- 172.21.15.105:0/1257961008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7cfe70 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fafa4004340 tx=0x7fafa4005740 comp rx=0 tx=0).stop 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 -- 172.21.15.105:0/1257961008 shutdown_connections 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 --2- 172.21.15.105:0/1257961008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7db590 0x55d3fb7cfe70 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 --2- 172.21.15.105:0/1257961008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 --2- 172.21.15.105:0/1257961008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 -- 172.21.15.105:0/1257961008 >> 172.21.15.105:0/1257961008 conn(0x55d3fb7d0fa0 msgr2=0x55d3fb7d78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 -- 172.21.15.105:0/1257961008 shutdown_connections 2024-03-26T17:51:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.817+0000 7fafb921f740 1 -- 172.21.15.105:0/1257961008 wait complete. 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 1 -- start start 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 1 Processor -- start 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb850130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00fa60 con 0x55d3fb7d5a30 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00f2d0 con 0x55d3fb7d6420 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c00c510 con 0x55d3fb7db590 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafb921f740 10 client.0 _ll_register_callbacks cb 0x7fafa4009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafaf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafaf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7e43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46366/0 (socket says 172.21.15.105:46366) 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.818+0000 7fafaf7fe640 1 -- 172.21.15.105:0/3203643609 learned_addr learned my addr 172.21.15.105:0/3203643609 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaffff640 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb850130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaffff640 1 -- 172.21.15.105:0/3203643609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 msgr2=0x55d3fb850130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaffff640 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb850130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaffff640 1 -- 172.21.15.105:0/3203643609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d5a30 msgr2=0x55d3fb7e43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaffff640 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7e43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaffff640 1 -- 172.21.15.105:0/3203643609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3fb8506a0 con 0x55d3fb7d6420 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7e43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafaffff640 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e4900 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000f80 tx=0x7fafa801d840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82795b2150b88bea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7faf8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7faf8f7fe640 1 -- 172.21.15.105:0/3203643609 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa8028070 con 0x55d3fb7d6420 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7faf8f7fe640 1 -- 172.21.15.105:0/3203643609 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fafa8004b20 con 0x55d3fb7d6420 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.819+0000 7fafb921f740 1 -- 172.21.15.105:0/3203643609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa4003800 con 0x55d3fb7d6420 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7faf8f7fe640 1 -- 172.21.15.105:0/3203643609 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fafa8004d20 con 0x55d3fb7d6420 2024-03-26T17:51:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafb921f740 10 client.30104 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafb921f740 1 -- 172.21.15.105:0/3203643609 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3fb851a70 con 0x55d3fb7d6420 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7faf8f7fe640 1 -- 172.21.15.105:0/3203643609 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa4003800 con 0x55d3fb7d6420 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7faf8f7fe640 1 -- 172.21.15.105:0/3203643609 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fafa800b040 con 0x55d3fb7d6420 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafb921f740 10 client.30104 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafb921f740 10 client.30104 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafb921f740 20 client.30104 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafb921f740 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d3fb8523d0 0x55d3fb8547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafb921f740 4 client.30104 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafb921f740 1 -- 172.21.15.105:0/3203643609 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3fb742200 con 0x55d3fb8523d0 2024-03-26T17:51:38.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.820+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d3fb8523d0 0x55d3fb8547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.821+0000 7faf8f7fe640 10 client.30104 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:51:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.821+0000 7fafaf7fe640 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d3fb8523d0 0x55d3fb8547f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9c0f23ecdbfc379 server_cookie=68ad0858cd4b66b2 in_seq=0 out_seq=0 2024-03-26T17:51:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7faf8f7fe640 1 -- 172.21.15.105:0/3203643609 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3fb742200 con 0x55d3fb8523d0 2024-03-26T17:51:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7faf8f7fe640 10 client.30104 handle_command_reply: tid=0 2024-03-26T17:51:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 client.30104 shutdown 2024-03-26T17:51:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 -- 172.21.15.105:0/3203643609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d6420 msgr2=0x55d3fb7e4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e4900 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000f80 tx=0x7fafa801d840 comp rx=0 tx=0).stop 2024-03-26T17:51:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 -- 172.21.15.105:0/3203643609 shutdown_connections 2024-03-26T17:51:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55d3fb8523d0 0x55d3fb8547f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7faf8f7fe640 0 client.30104 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:51:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3fb7db590 0x55d3fb850130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3fb7d6420 0x55d3fb7e4900 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 --2- 172.21.15.105:0/3203643609 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3fb7d5a30 0x55d3fb7e43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 -- 172.21.15.105:0/3203643609 >> 172.21.15.105:0/3203643609 conn(0x55d3fb7d0fa0 msgr2=0x55d3fb7d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 -- 172.21.15.105:0/3203643609 shutdown_connections 2024-03-26T17:51:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.822+0000 7fafb921f740 1 -- 172.21.15.105:0/3203643609 wait complete. 2024-03-26T17:51:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.823+0000 7fafb921f740 20 client.30104 trim_cache size 0 max 16384 2024-03-26T17:51:38.820 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:38.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.823+0000 7fafb663f640 1 -- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf980a2f20 msgr2=0x7faf980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.823+0000 7fafb663f640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf980a2f20 0x7faf980a53e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fafa0005820 tx=0x7fafa0009170 comp rx=0 tx=0).stop 2024-03-26T17:51:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.823+0000 7fafb663f640 1 -- 172.21.15.105:0/1868866558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb01012a0 msgr2=0x7fafb0103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.823+0000 7fafb663f640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb01012a0 0x7fafb0103030 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fafa002c180 tx=0x7fafa00042f0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.824+0000 7fafb663f640 1 -- 172.21.15.105:0/1868866558 shutdown_connections 2024-03-26T17:51:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.824+0000 7fafb663f640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faf980a2f20 0x7faf980a53e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.824+0000 7fafb663f640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fafb01012a0 0x7fafb0103030 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.824+0000 7fafb663f640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafb01008e0 0x7fafb0106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.824+0000 7fafb663f640 1 --2- 172.21.15.105:0/1868866558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafb00ffb80 0x7fafb01064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.824+0000 7fafb663f640 1 -- 172.21.15.105:0/1868866558 >> 172.21.15.105:0/1868866558 conn(0x7fafb00fb4b0 msgr2=0x7fafb00fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.824+0000 7fafb663f640 1 -- 172.21.15.105:0/1868866558 shutdown_connections 2024-03-26T17:51:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:38.824+0000 7fafb663f640 1 -- 172.21.15.105:0/1868866558 wait complete. 2024-03-26T17:51:38.828 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:51:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.895+0000 7fb20d59a640 1 -- 172.21.15.105:0/1266418337 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20002cbe0 con 0x7fb21010ab70 2024-03-26T17:51:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 -- 172.21.15.105:0/1266418337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010ab70 msgr2=0x7fb21010cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 --2- 172.21.15.105:0/1266418337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010ab70 0x7fb21010cf60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb200009900 tx=0x7fb20002bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 -- 172.21.15.105:0/1266418337 shutdown_connections 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 --2- 172.21.15.105:0/1266418337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb21010fdd0 0x7fb210112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 --2- 172.21.15.105:0/1266418337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb21010d4a0 0x7fb21010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 --2- 172.21.15.105:0/1266418337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010ab70 0x7fb21010cf60 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 -- 172.21.15.105:0/1266418337 >> 172.21.15.105:0/1266418337 conn(0x7fb21006f830 msgr2=0x7fb210071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 -- 172.21.15.105:0/1266418337 shutdown_connections 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.896+0000 7fb214d52640 1 -- 172.21.15.105:0/1266418337 wait complete. 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb214d52640 1 Processor -- start 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb214d52640 1 -- start start 2024-03-26T17:51:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb214d52640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb21010ab70 0x7fb2101a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb214d52640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb21010d4a0 0x7fb2101a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb214d52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010fdd0 0x7fb2101a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb214d52640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb210115c30 con 0x7fb21010fdd0 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb214d52640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb210115ab0 con 0x7fb21010d4a0 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb214d52640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb210115db0 con 0x7fb21010ab70 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb21010ab70 0x7fb2101a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb20ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010fdd0 0x7fb2101a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb21010ab70 0x7fb2101a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46600/0 (socket says 172.21.15.105:46600) 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb20e59c640 1 -- 172.21.15.105:0/3736007334 learned_addr learned my addr 172.21.15.105:0/3736007334 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.897+0000 7fb20dd9b640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb21010d4a0 0x7fb2101a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb20e59c640 1 -- 172.21.15.105:0/3736007334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb21010d4a0 msgr2=0x7fb2101a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb20e59c640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb21010d4a0 0x7fb2101a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb20e59c640 1 -- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010fdd0 msgr2=0x7fb2101a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb20e59c640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010fdd0 0x7fb2101a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb20e59c640 1 -- 172.21.15.105:0/3736007334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb210108a60 con 0x7fb21010ab70 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010fdd0 0x7fb2101a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb20e59c640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb21010ab70 0x7fb2101a0250 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb200006010 tx=0x7fb200005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb1ff7fe640 1 -- 172.21.15.105:0/3736007334 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200043020 con 0x7fb21010ab70 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb1ff7fe640 1 -- 172.21.15.105:0/3736007334 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2000043d0 con 0x7fb21010ab70 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb214d52640 1 -- 172.21.15.105:0/3736007334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb210108cf0 con 0x7fb21010ab70 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb1ff7fe640 1 -- 172.21.15.105:0/3736007334 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200003b80 con 0x7fb21010ab70 2024-03-26T17:51:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.898+0000 7fb214d52640 1 -- 172.21.15.105:0/3736007334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb210109290 con 0x7fb21010ab70 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.901+0000 7fb1ff7fe640 1 -- 172.21.15.105:0/3736007334 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb200046020 con 0x7fb21010ab70 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.902+0000 7fb217a10740 1 Processor -- start 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.902+0000 7fb1ff7fe640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1f80a3040 0x7fb1f80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.902+0000 7fb1ff7fe640 1 -- 172.21.15.105:0/3736007334 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb2000f1720 con 0x7fb21010ab70 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.902+0000 7fb217a10740 1 -- start start 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.902+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1f80a3040 0x7fb1f80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.902+0000 7fb217a10740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb217a10740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc37fc0 0x56364cc3cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb217a10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc3f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb217a10740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56364cb9c0b0 con 0x56364cc37fc0 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb217a10740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56364cb44960 con 0x56364cc375d0 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb217a10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56364cb98060 con 0x56364cc3d130 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34084/0 (socket says 172.21.15.105:34084) 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 -- 172.21.15.105:0/1511020588 learned_addr learned my addr 172.21.15.105:0/1511020588 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20dd9b640 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc37fc0 0x56364cc3cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc3f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1f80a3040 0x7fb1f80a5500 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb208005fd0 tx=0x7fb208011b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 -- 172.21.15.105:0/1511020588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 msgr2=0x56364cc3f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc3f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 -- 172.21.15.105:0/1511020588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc37fc0 msgr2=0x56364cc3cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc37fc0 0x56364cc3cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 -- 172.21.15.105:0/1511020588 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364cb56c60 con 0x56364cc375d0 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.903+0000 7fb20e59c640 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc379f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb2000ba780 tx=0x7fb2000bd9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c69ee0e6b77456 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.904+0000 7fb1fcff9640 1 -- 172.21.15.105:0/1511020588 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000bddb0 con 0x56364cc375d0 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.904+0000 7fb1fcff9640 1 -- 172.21.15.105:0/1511020588 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2000c1030 con 0x56364cc375d0 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.904+0000 7fb1fcff9640 1 -- 172.21.15.105:0/1511020588 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000055b0 con 0x56364cc375d0 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 -- 172.21.15.105:0/1511020588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 msgr2=0x56364cc379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc379f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb2000ba780 tx=0x7fb2000bd9b0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 -- 172.21.15.105:0/1511020588 shutdown_connections 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc3f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc37fc0 0x56364cc3cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 --2- 172.21.15.105:0/1511020588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc379f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 -- 172.21.15.105:0/1511020588 >> 172.21.15.105:0/1511020588 conn(0x56364cc32b40 msgr2=0x56364cc32f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 -- 172.21.15.105:0/1511020588 shutdown_connections 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 -- 172.21.15.105:0/1511020588 wait complete. 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 -- start start 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.905+0000 7fb217a10740 1 Processor -- start 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb217a10740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc41170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb217a10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc37fc0 0x56364cc416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb217a10740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc3d130 0x56364cc486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc41170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb217a10740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56364cc3fea0 con 0x56364cc3d130 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc41170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34086/0 (socket says 172.21.15.105:34086) 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20e59c640 1 -- 172.21.15.105:0/1563062969 learned_addr learned my addr 172.21.15.105:0/1563062969 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56364cb57db0 con 0x56364cc375d0 2024-03-26T17:51:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56364cc40020 con 0x56364cc37fc0 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20dd9b640 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc37fc0 0x56364cc416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc3d130 0x56364cc486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb217a10740 10 client.0 _ll_register_callbacks cb 0x56364cb5c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20dd9b640 1 -- 172.21.15.105:0/1563062969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 msgr2=0x56364cc41170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20dd9b640 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc41170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20dd9b640 1 -- 172.21.15.105:0/1563062969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc3d130 msgr2=0x56364cc486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20dd9b640 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc3d130 0x56364cc486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20dd9b640 1 -- 172.21.15.105:0/1563062969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364c8b0210 con 0x56364cc37fc0 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.906+0000 7fb20e59c640 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc41170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb20dd9b640 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc37fc0 0x56364cc416b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56364cc383c0 tx=0x7fb204012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4d6a317fbac6ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb1f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/1563062969 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204004320 con 0x56364cc37fc0 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2000c1030 con 0x56364cc37fc0 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/1563062969 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb204011910 con 0x56364cc37fc0 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/1563062969 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb2040044c0 con 0x56364cc37fc0 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb217a10740 10 client.30100 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56364c97c620 con 0x56364cc37fc0 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.907+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/1563062969 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2000c1030 con 0x56364cc37fc0 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/1563062969 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb20400f440 con 0x56364cc37fc0 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb217a10740 10 client.30100 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb217a10740 10 client.30100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb217a10740 20 client.30100 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb217a10740 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56364cc4cec0 0x56364cc4f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb217a10740 4 client.30100 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56364cba95f0 con 0x56364cc4cec0 2024-03-26T17:51:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb20e59c640 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56364cc4cec0 0x56364cc4f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.908+0000 7fb1f5ffb640 10 client.30100 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.909+0000 7fb20e59c640 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56364cc4cec0 0x56364cc4f2f0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439803ddd3ecc275 server_cookie=f44b01417f56ae27 in_seq=0 out_seq=0 2024-03-26T17:51:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/1563062969 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56364cba95f0 con 0x56364cc4cec0 2024-03-26T17:51:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb1f5ffb640 10 client.30100 handle_command_reply: tid=0 2024-03-26T17:51:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 client.30100 shutdown 2024-03-26T17:51:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc37fc0 msgr2=0x56364cc416b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc37fc0 0x56364cc416b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56364cc383c0 tx=0x7fb204012b60 comp rx=0 tx=0).stop 2024-03-26T17:51:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 shutdown_connections 2024-03-26T17:51:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56364cc4cec0 0x56364cc4f2f0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb1f5ffb640 0 client.30100 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc3d130 0x56364cc486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc37fc0 0x56364cc416b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 --2- 172.21.15.105:0/1563062969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc375d0 0x56364cc41170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 >> 172.21.15.105:0/1563062969 conn(0x56364cc32b40 msgr2=0x56364cc31df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 shutdown_connections 2024-03-26T17:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 1 -- 172.21.15.105:0/1563062969 wait complete. 2024-03-26T17:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.911+0000 7fb217a10740 20 client.30100 trim_cache size 0 max 16384 2024-03-26T17:51:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.941+0000 7fb217a10740 1 Processor -- start 2024-03-26T17:51:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.941+0000 7fb217a10740 1 -- start start 2024-03-26T17:51:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb217a10740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc40a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb217a10740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 0x56364cc486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb217a10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc31560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb217a10740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2101af250 con 0x56364cc375d0 2024-03-26T17:51:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb217a10740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2101b0e50 con 0x56364cc37fc0 2024-03-26T17:51:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb217a10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2101af920 con 0x56364cc3d130 2024-03-26T17:51:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc40a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc40a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46400/0 (socket says 172.21.15.105:46400) 2024-03-26T17:51:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 -- 172.21.15.105:0/76352220 learned_addr learned my addr 172.21.15.105:0/76352220 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20dd9b640 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 0x56364cc486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc31560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 -- 172.21.15.105:0/76352220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 msgr2=0x56364cc31560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc31560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 -- 172.21.15.105:0/76352220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 msgr2=0x56364cc486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 0x56364cc486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 -- 172.21.15.105:0/76352220 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364cc335b0 con 0x56364cc375d0 2024-03-26T17:51:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.942+0000 7fb20e59c640 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc40a20 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb2000bdd40 tx=0x7fb2000bda20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c7491208030ff5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.943+0000 7fb1f7fff640 1 -- 172.21.15.105:0/76352220 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2000a1070 con 0x56364cc375d0 2024-03-26T17:51:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.943+0000 7fb1f7fff640 1 -- 172.21.15.105:0/76352220 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2000d9050 con 0x56364cc375d0 2024-03-26T17:51:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.943+0000 7fb1f7fff640 1 -- 172.21.15.105:0/76352220 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200003d20 con 0x56364cc375d0 2024-03-26T17:51:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 -- 172.21.15.105:0/76352220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 msgr2=0x56364cc40a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc40a20 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb2000bdd40 tx=0x7fb2000bda20 comp rx=0 tx=0).stop 2024-03-26T17:51:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 -- 172.21.15.105:0/76352220 shutdown_connections 2024-03-26T17:51:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc31560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 0x56364cc486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 --2- 172.21.15.105:0/76352220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc40a20 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 -- 172.21.15.105:0/76352220 >> 172.21.15.105:0/76352220 conn(0x56364cc32b40 msgr2=0x56364cc40310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 -- 172.21.15.105:0/76352220 shutdown_connections 2024-03-26T17:51:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.944+0000 7fb217a10740 1 -- 172.21.15.105:0/76352220 wait complete. 2024-03-26T17:51:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 1 -- start start 2024-03-26T17:51:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 1 Processor -- start 2024-03-26T17:51:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc46080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 0x56364cc465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc47580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2101b0780 con 0x56364cc375d0 2024-03-26T17:51:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb2101afff0 con 0x56364cc37fc0 2024-03-26T17:51:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2101ad230 con 0x56364cc3d130 2024-03-26T17:51:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb217a10740 10 client.0 _ll_register_callbacks cb 0x7fb2040097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb20ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc47580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb20ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc47580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46644/0 (socket says 172.21.15.105:46644) 2024-03-26T17:51:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb20ed9d640 1 -- 172.21.15.105:0/3996348639 learned_addr learned my addr 172.21.15.105:0/3996348639 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb20e59c640 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.945+0000 7fb20dd9b640 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 0x56364cc465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb20ed9d640 1 -- 172.21.15.105:0/3996348639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 msgr2=0x56364cc465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 0x56364cc465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb20ed9d640 1 -- 172.21.15.105:0/3996348639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 msgr2=0x56364cc46080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc46080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb20ed9d640 1 -- 172.21.15.105:0/3996348639 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364cc47b70 con 0x56364cc3d130 2024-03-26T17:51:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb20e59c640 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc46080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb20ed9d640 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc47580 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb208003e70 tx=0x7fb20801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=325f680de1d3f504 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb1f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/3996348639 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208035060 con 0x56364cc3d130 2024-03-26T17:51:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb217a10740 1 -- 172.21.15.105:0/3996348639 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2000d9050 con 0x56364cc3d130 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.946+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/3996348639 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb208030420 con 0x56364cc3d130 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.947+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/3996348639 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb208017040 con 0x56364cc3d130 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.947+0000 7fb217a10740 10 client.30106 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.947+0000 7fb217a10740 1 -- 172.21.15.105:0/3996348639 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56364ccb34c0 con 0x56364cc3d130 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.947+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/3996348639 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2000d9050 con 0x56364cc3d130 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/3996348639 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb20803b590 con 0x56364cc3d130 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb217a10740 10 client.30106 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb217a10740 10 client.30106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb217a10740 20 client.30106 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb217a10740 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56364cc4cec0 0x56364ccb5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb217a10740 4 client.30106 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:51:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb217a10740 1 -- 172.21.15.105:0/3996348639 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56364cba3940 con 0x56364cc4cec0 2024-03-26T17:51:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb20e59c640 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56364cc4cec0 0x56364ccb5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb20e59c640 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56364cc4cec0 0x56364ccb5fd0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a23da4be7136107 server_cookie=bac7973ea019c1fe in_seq=0 out_seq=0 2024-03-26T17:51:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:38.948+0000 7fb1f5ffb640 10 client.30106 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.000+0000 7f822d019640 1 -- 172.21.15.105:0/1749160395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228105450 msgr2=0x7f82281058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:38.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.000+0000 7f822d019640 1 --2- 172.21.15.105:0/1749160395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228105450 0x7f82281058d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f821c009900 tx=0x7f821c02bc60 comp rx=0 tx=0).stop 2024-03-26T17:51:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.004+0000 7f822d019640 1 -- 172.21.15.105:0/1749160395 shutdown_connections 2024-03-26T17:51:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.004+0000 7f822d019640 1 --2- 172.21.15.105:0/1749160395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228105450 0x7f82281058d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.004+0000 7f822d019640 1 --2- 172.21.15.105:0/1749160395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8228102aa0 0x7f8228104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.004+0000 7f822d019640 1 --2- 172.21.15.105:0/1749160395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8228100130 0x7f8228102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:38.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.004+0000 7f822d019640 1 -- 172.21.15.105:0/1749160395 >> 172.21.15.105:0/1749160395 conn(0x7f82280fbd50 msgr2=0x7f82280fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:38.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.008+0000 7f822d019640 1 -- 172.21.15.105:0/1749160395 shutdown_connections 2024-03-26T17:51:38.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.008+0000 7f822d019640 1 -- 172.21.15.105:0/1749160395 wait complete. 2024-03-26T17:51:39.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.009+0000 7f822d019640 1 Processor -- start 2024-03-26T17:51:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.010+0000 7f822d019640 1 -- start start 2024-03-26T17:51:39.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.010+0000 7f822d019640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228100130 0x7f82280702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.010+0000 7f822d019640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8228102aa0 0x7f8228070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.010+0000 7f822d019640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8228105450 0x7f82281ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.010+0000 7f822d019640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8228113a40 con 0x7f8228102aa0 2024-03-26T17:51:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.010+0000 7f822d019640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82281138c0 con 0x7f8228100130 2024-03-26T17:51:39.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.010+0000 7f822d019640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8228113bc0 con 0x7f8228105450 2024-03-26T17:51:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228100130 0x7f82280702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228100130 0x7f82280702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34100/0 (socket says 172.21.15.105:34100) 2024-03-26T17:51:39.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f8226d9d640 1 -- 172.21.15.105:0/598532407 learned_addr learned my addr 172.21.15.105:0/598532407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f822659c640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8228102aa0 0x7f8228070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f822759e640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8228105450 0x7f82281ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f8226d9d640 1 -- 172.21.15.105:0/598532407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8228105450 msgr2=0x7f82281ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f8226d9d640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8228105450 0x7f82281ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f8226d9d640 1 -- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8228102aa0 msgr2=0x7f8228070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f8226d9d640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8228102aa0 0x7f8228070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.011+0000 7f8226d9d640 1 -- 172.21.15.105:0/598532407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82281ae690 con 0x7f8228100130 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.012+0000 7f8226d9d640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228100130 0x7f82280702c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f822000d950 tx=0x7f822000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.012+0000 7f820ffff640 1 -- 172.21.15.105:0/598532407 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220017070 con 0x7f8228100130 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.012+0000 7f820ffff640 1 -- 172.21.15.105:0/598532407 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8220009c30 con 0x7f8228100130 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.013+0000 7f820ffff640 1 -- 172.21.15.105:0/598532407 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822000f620 con 0x7f8228100130 2024-03-26T17:51:39.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.013+0000 7f822d019640 1 -- 172.21.15.105:0/598532407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82281ae860 con 0x7f8228100130 2024-03-26T17:51:39.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.013+0000 7f822d019640 1 -- 172.21.15.105:0/598532407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281aedb0 con 0x7f8228100130 2024-03-26T17:51:39.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.016+0000 7f822fc06740 1 Processor -- start 2024-03-26T17:51:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.017+0000 7f822fc06740 1 -- start start 2024-03-26T17:51:39.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.017+0000 7f822fc06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac25a30 0x5574aac25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.017+0000 7f822fc06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac26420 0x5574aac2af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.017+0000 7f822fc06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 0x5574aac2da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.017+0000 7f822fc06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5574aab8a970 con 0x5574aac26420 2024-03-26T17:51:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.017+0000 7f822fc06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5574aab32d70 con 0x5574aac25a30 2024-03-26T17:51:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.017+0000 7f822fc06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574aab86920 con 0x5574aac2b590 2024-03-26T17:51:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.018+0000 7f822759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac25a30 0x5574aac25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.018+0000 7f822759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac25a30 0x5574aac25e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34116/0 (socket says 172.21.15.105:34116) 2024-03-26T17:51:39.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.018+0000 7f822759e640 1 -- 172.21.15.105:0/3727871001 learned_addr learned my addr 172.21.15.105:0/3727871001 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.018+0000 7f820ffff640 1 -- 172.21.15.105:0/598532407 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f82200109a0 con 0x7f8228100130 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.018+0000 7f820ffff640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82100a3070 0x7f82100a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f822659c640 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 0x5574aac2da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f820ffff640 1 -- 172.21.15.105:0/598532407 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f82200cf040 con 0x7f8228100130 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f822759e640 1 -- 172.21.15.105:0/3727871001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 msgr2=0x5574aac2da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f8226d9d640 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac26420 0x5574aac2af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f822759e640 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 0x5574aac2da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f822759e640 1 -- 172.21.15.105:0/3727871001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac26420 msgr2=0x5574aac2af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f822759e640 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac26420 0x5574aac2af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f822759e640 1 -- 172.21.15.105:0/3727871001 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574aa89e3f0 con 0x5574aac25a30 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f822759e640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82100a3070 0x7f82100a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.019+0000 7f8226d9d640 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac26420 0x5574aac2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.020+0000 7f822759e640 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac25a30 0x5574aac25e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f821c006010 tx=0x7f821c0095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281e8ed53becf505 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.020+0000 7f822759e640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82100a3070 0x7f82100a5530 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f821c005ab0 tx=0x7f821c04c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.020+0000 7f820d7fa640 1 -- 172.21.15.105:0/3727871001 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c045070 con 0x5574aac25a30 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.020+0000 7f820d7fa640 1 -- 172.21.15.105:0/3727871001 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f821c0046c0 con 0x5574aac25a30 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.020+0000 7f820d7fa640 1 -- 172.21.15.105:0/3727871001 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c003850 con 0x5574aac25a30 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 -- 172.21.15.105:0/3727871001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac25a30 msgr2=0x5574aac25e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac25a30 0x5574aac25e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f821c006010 tx=0x7f821c0095b0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 -- 172.21.15.105:0/3727871001 shutdown_connections 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 0x5574aac2da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac26420 0x5574aac2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 --2- 172.21.15.105:0/3727871001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac25a30 0x5574aac25e50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 -- 172.21.15.105:0/3727871001 >> 172.21.15.105:0/3727871001 conn(0x5574aac20fa0 msgr2=0x5574aac213a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 -- 172.21.15.105:0/3727871001 shutdown_connections 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.021+0000 7f822fc06740 1 -- 172.21.15.105:0/3727871001 wait complete. 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 1 -- start start 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 1 Processor -- start 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac41020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac26420 0x5574aac41560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac2b590 0x5574aac48530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5574aac3f550 con 0x5574aac25a30 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5574aab46310 con 0x5574aac2b590 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5574aac3f6d0 con 0x5574aac26420 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822fc06740 10 client.0 _ll_register_callbacks cb 0x5574aab81c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac41020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac26420 0x5574aac41560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac41020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46444/0 (socket says 172.21.15.105:46444) 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac26420 0x5574aac41560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46680/0 (socket says 172.21.15.105:46680) 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f8226d9d640 1 -- 172.21.15.105:0/3222229408 learned_addr learned my addr 172.21.15.105:0/3222229408 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822759e640 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac2b590 0x5574aac48530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822659c640 1 -- 172.21.15.105:0/3222229408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac2b590 msgr2=0x5574aac48530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822659c640 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac2b590 0x5574aac48530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822659c640 1 -- 172.21.15.105:0/3222229408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 msgr2=0x5574aac41020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822659c640 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac41020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f822659c640 1 -- 172.21.15.105:0/3222229408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574aa9687d0 con 0x5574aac26420 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.022+0000 7f8226d9d640 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822659c640 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac26420 0x5574aac41560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f82280717d0 tx=0x7f82180111b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dab4cac13c071143 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f820aff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f820aff5640 1 -- 172.21.15.105:0/3222229408 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821801a070 con 0x5574aac26420 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f820aff5640 1 -- 172.21.15.105:0/3222229408 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8218016450 con 0x5574aac26420 2024-03-26T17:51:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 1 -- 172.21.15.105:0/3222229408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f821c0046c0 con 0x5574aac26420 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f820aff5640 1 -- 172.21.15.105:0/3222229408 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8218026050 con 0x5574aac26420 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 10 client.30118 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 1 -- 172.21.15.105:0/3222229408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5574aac4b8e0 con 0x5574aac26420 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f820aff5640 1 -- 172.21.15.105:0/3222229408 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f821c0046c0 con 0x5574aac26420 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f820aff5640 1 -- 172.21.15.105:0/3222229408 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f821801e670 con 0x5574aac26420 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 10 client.30118 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 10 client.30118 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 20 client.30118 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5574aac4cfc0 0x5574aac4f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 4 client.30118 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.023+0000 7f822fc06740 1 -- 172.21.15.105:0/3222229408 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574aab97eb0 con 0x5574aac4cfc0 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.024+0000 7f8226d9d640 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5574aac4cfc0 0x5574aac4f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.024+0000 7f820aff5640 10 client.30118 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.024+0000 7f8226d9d640 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5574aac4cfc0 0x5574aac4f400 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd745b43ee5990e4 server_cookie=c783f549b2ace0e3 in_seq=0 out_seq=0 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f820aff5640 1 -- 172.21.15.105:0/3222229408 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5574aab97eb0 con 0x5574aac4cfc0 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f820aff5640 10 client.30118 handle_command_reply: tid=0 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 client.30118 shutdown 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 -- 172.21.15.105:0/3222229408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac26420 msgr2=0x5574aac41560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac26420 0x5574aac41560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f82280717d0 tx=0x7f82180111b0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 -- 172.21.15.105:0/3222229408 shutdown_connections 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5574aac4cfc0 0x5574aac4f400 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac2b590 0x5574aac48530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac26420 0x5574aac41560 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f820aff5640 0 client.30118 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:51:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 --2- 172.21.15.105:0/3222229408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac41020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 -- 172.21.15.105:0/3222229408 >> 172.21.15.105:0/3222229408 conn(0x5574aac20fa0 msgr2=0x5574aac201c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 -- 172.21.15.105:0/3222229408 shutdown_connections 2024-03-26T17:51:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.025+0000 7f822fc06740 1 -- 172.21.15.105:0/3222229408 wait complete. 2024-03-26T17:51:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.026+0000 7f822fc06740 20 client.30118 trim_cache size 0 max 16384 2024-03-26T17:51:39.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822fc06740 1 Processor -- start 2024-03-26T17:51:39.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822fc06740 1 -- start start 2024-03-26T17:51:39.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822fc06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac25a30 0x5574aac21380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822fc06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 0x5574aac484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822fc06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac2b590 0x5574aac279f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822fc06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82281af480 con 0x5574aac2b590 2024-03-26T17:51:39.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822fc06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82281b1080 con 0x5574aac26420 2024-03-26T17:51:39.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822fc06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82281afb50 con 0x5574aac25a30 2024-03-26T17:51:39.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac25a30 0x5574aac21380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac25a30 0x5574aac21380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46690/0 (socket says 172.21.15.105:46690) 2024-03-26T17:51:39.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f8226d9d640 1 -- 172.21.15.105:0/317762885 learned_addr learned my addr 172.21.15.105:0/317762885 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822659c640 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 0x5574aac484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f822759e640 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac2b590 0x5574aac279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f8226d9d640 1 -- 172.21.15.105:0/317762885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 msgr2=0x5574aac484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f8226d9d640 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 0x5574aac484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f8226d9d640 1 -- 172.21.15.105:0/317762885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac2b590 msgr2=0x5574aac279f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f8226d9d640 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac2b590 0x5574aac279f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.039+0000 7f8226d9d640 1 -- 172.21.15.105:0/317762885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574aac25820 con 0x5574aac25a30 2024-03-26T17:51:39.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f8226d9d640 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac25a30 0x5574aac21380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f822009b3d0 tx=0x7f822000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2555ef9ec47608e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f820cff9640 1 -- 172.21.15.105:0/317762885 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220020040 con 0x5574aac25a30 2024-03-26T17:51:39.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f820cff9640 1 -- 172.21.15.105:0/317762885 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f822009f820 con 0x5574aac25a30 2024-03-26T17:51:39.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f820cff9640 1 -- 172.21.15.105:0/317762885 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220097d40 con 0x5574aac25a30 2024-03-26T17:51:39.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 -- 172.21.15.105:0/317762885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac25a30 msgr2=0x5574aac21380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac25a30 0x5574aac21380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f822009b3d0 tx=0x7f822000bf60 comp rx=0 tx=0).stop 2024-03-26T17:51:39.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 -- 172.21.15.105:0/317762885 shutdown_connections 2024-03-26T17:51:39.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac2b590 0x5574aac279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 0x5574aac484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 --2- 172.21.15.105:0/317762885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac25a30 0x5574aac21380 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 -- 172.21.15.105:0/317762885 >> 172.21.15.105:0/317762885 conn(0x5574aac20fa0 msgr2=0x5574aac3fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 -- 172.21.15.105:0/317762885 shutdown_connections 2024-03-26T17:51:39.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.040+0000 7f822fc06740 1 -- 172.21.15.105:0/317762885 wait complete. 2024-03-26T17:51:39.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 1 -- start start 2024-03-26T17:51:39.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 1 Processor -- start 2024-03-26T17:51:39.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 0x5574aac457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 0x5574aacb1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f82281b09b0 con 0x5574aac25a30 2024-03-26T17:51:39.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f82281b0220 con 0x5574aac26420 2024-03-26T17:51:39.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f822810d350 con 0x5574aac2b590 2024-03-26T17:51:39.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822fc06740 10 client.0 _ll_register_callbacks cb 0x7f8218009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:39.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46456/0 (socket says 172.21.15.105:46456) 2024-03-26T17:51:39.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f8226d9d640 1 -- 172.21.15.105:0/599532571 learned_addr learned my addr 172.21.15.105:0/599532571 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822759e640 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 0x5574aacb1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f822659c640 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 0x5574aac457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f8226d9d640 1 -- 172.21.15.105:0/599532571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 msgr2=0x5574aacb1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f8226d9d640 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 0x5574aacb1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f8226d9d640 1 -- 172.21.15.105:0/599532571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 msgr2=0x5574aac457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f8226d9d640 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 0x5574aac457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.041+0000 7f8226d9d640 1 -- 172.21.15.105:0/599532571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574aac45e60 con 0x5574aac25a30 2024-03-26T17:51:39.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f8226d9d640 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac452b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f82200980a0 tx=0x7f8220009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b667667ba4b8b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f820aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:39.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f820aff5640 1 -- 172.21.15.105:0/599532571 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220097d40 con 0x5574aac25a30 2024-03-26T17:51:39.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f820aff5640 1 -- 172.21.15.105:0/599532571 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f822009ab20 con 0x5574aac25a30 2024-03-26T17:51:39.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f822fc06740 1 -- 172.21.15.105:0/599532571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822009f820 con 0x5574aac25a30 2024-03-26T17:51:39.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f820aff5640 1 -- 172.21.15.105:0/599532571 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8220081050 con 0x5574aac25a30 2024-03-26T17:51:39.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f822fc06740 10 client.20334 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:39.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f822fc06740 1 -- 172.21.15.105:0/599532571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5574aacb2970 con 0x5574aac25a30 2024-03-26T17:51:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f820aff5640 1 -- 172.21.15.105:0/599532571 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f822009f820 con 0x5574aac25a30 2024-03-26T17:51:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f820aff5640 1 -- 172.21.15.105:0/599532571 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8220010660 con 0x5574aac25a30 2024-03-26T17:51:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f822fc06740 10 client.20334 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f822fc06740 10 client.20334 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:51:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.042+0000 7f822fc06740 20 client.20334 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:39.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5574aacb3320 0x5574aacb5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 4 client.20334 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 -- 172.21.15.105:0/599532571 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574aab92200 con 0x5574aacb3320 2024-03-26T17:51:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822659c640 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5574aacb3320 0x5574aacb5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822659c640 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5574aacb3320 0x5574aacb5740 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c87ffaa7724d7c3 server_cookie=298a71995a2710c8 in_seq=0 out_seq=0 2024-03-26T17:51:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f820aff5640 10 client.20334 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:51:39.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f820aff5640 1 -- 172.21.15.105:0/599532571 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574aab92200 con 0x5574aacb3320 2024-03-26T17:51:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f820aff5640 10 client.20334 handle_command_reply: tid=0 2024-03-26T17:51:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 client.20334 shutdown 2024-03-26T17:51:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 -- 172.21.15.105:0/599532571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 msgr2=0x5574aac452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac452b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f82200980a0 tx=0x7f8220009740 comp rx=0 tx=0).stop 2024-03-26T17:51:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 -- 172.21.15.105:0/599532571 shutdown_connections 2024-03-26T17:51:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5574aacb3320 0x5574aacb5740 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5574aac2b590 0x5574aacb1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5574aac26420 0x5574aac457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 --2- 172.21.15.105:0/599532571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5574aac25a30 0x5574aac452b0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 -- 172.21.15.105:0/599532571 >> 172.21.15.105:0/599532571 conn(0x5574aac20fa0 msgr2=0x5574aac21380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f820aff5640 0 client.20334 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:51:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 -- 172.21.15.105:0/599532571 shutdown_connections 2024-03-26T17:51:39.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.043+0000 7f822fc06740 1 -- 172.21.15.105:0/599532571 wait complete. 2024-03-26T17:51:39.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822fc06740 20 client.20334 trim_cache size 0 max 16384 2024-03-26T17:51:39.052 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 -- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82100a3070 msgr2=0x7f82100a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82100a3070 0x7f82100a5530 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f821c005ab0 tx=0x7f821c04c000 comp rx=0 tx=0).stop 2024-03-26T17:51:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 -- 172.21.15.105:0/598532407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228100130 msgr2=0x7f82280702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228100130 0x7f82280702c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f822000d950 tx=0x7f822000de10 comp rx=0 tx=0).stop 2024-03-26T17:51:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 -- 172.21.15.105:0/598532407 shutdown_connections 2024-03-26T17:51:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f82100a3070 0x7f82100a5530 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8228105450 0x7f82281ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8228102aa0 0x7f8228070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 --2- 172.21.15.105:0/598532407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8228100130 0x7f82280702c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 -- 172.21.15.105:0/598532407 >> 172.21.15.105:0/598532407 conn(0x7f82280fbd50 msgr2=0x7f8228100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 -- 172.21.15.105:0/598532407 shutdown_connections 2024-03-26T17:51:39.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.044+0000 7f822d019640 1 -- 172.21.15.105:0/598532407 wait complete. 2024-03-26T17:51:39.057 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:51:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb1f5ffb640 1 -- 172.21.15.105:0/3996348639 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56364cba3940 con 0x56364cc4cec0 2024-03-26T17:51:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb1f5ffb640 10 client.30106 handle_command_reply: tid=0 2024-03-26T17:51:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 client.30106 shutdown 2024-03-26T17:51:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 -- 172.21.15.105:0/3996348639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 msgr2=0x56364cc47580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc47580 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb208003e70 tx=0x7fb20801c000 comp rx=0 tx=0).stop 2024-03-26T17:51:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 -- 172.21.15.105:0/3996348639 shutdown_connections 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56364cc4cec0 0x56364ccb5fd0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56364cc3d130 0x56364cc47580 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56364cc37fc0 0x56364cc465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb1f5ffb640 0 client.30106 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 --2- 172.21.15.105:0/3996348639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56364cc375d0 0x56364cc46080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 -- 172.21.15.105:0/3996348639 >> 172.21.15.105:0/3996348639 conn(0x56364cc32b40 msgr2=0x56364cc35da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 -- 172.21.15.105:0/3996348639 shutdown_connections 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 1 -- 172.21.15.105:0/3996348639 wait complete. 2024-03-26T17:51:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.069+0000 7fb217a10740 20 client.30106 trim_cache size 0 max 16384 2024-03-26T17:51:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:51:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:51:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:51:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:51:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 -- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1f80a3040 msgr2=0x7fb1f80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1f80a3040 0x7fb1f80a5500 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb208005fd0 tx=0x7fb208011b40 comp rx=0 tx=0).stop 2024-03-26T17:51:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 -- 172.21.15.105:0/3736007334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb21010ab70 msgr2=0x7fb2101a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb21010ab70 0x7fb2101a0250 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb200006010 tx=0x7fb200005e40 comp rx=0 tx=0).stop 2024-03-26T17:51:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 -- 172.21.15.105:0/3736007334 shutdown_connections 2024-03-26T17:51:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1f80a3040 0x7fb1f80a5500 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb21010fdd0 0x7fb2101a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb21010d4a0 0x7fb2101a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 --2- 172.21.15.105:0/3736007334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb21010ab70 0x7fb2101a0250 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 -- 172.21.15.105:0/3736007334 >> 172.21.15.105:0/3736007334 conn(0x7fb21006f830 msgr2=0x7fb21010b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 -- 172.21.15.105:0/3736007334 shutdown_connections 2024-03-26T17:51:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:39.070+0000 7fb214d52640 1 -- 172.21.15.105:0/3736007334 wait complete. 2024-03-26T17:51:39.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:38 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3707062842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:51:39.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:38 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3707062842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:51:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 -- 172.21.15.105:0/423492995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea741018d0 msgr2=0x7fea74101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 --2- 172.21.15.105:0/423492995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea741018d0 0x7fea74101cf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fea68009900 tx=0x7fea6802bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 -- 172.21.15.105:0/423492995 shutdown_connections 2024-03-26T17:51:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 --2- 172.21.15.105:0/423492995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea74103480 0x7fea74109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 --2- 172.21.15.105:0/423492995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea74102ac0 0x7fea74102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 --2- 172.21.15.105:0/423492995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea741018d0 0x7fea74101cf0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 -- 172.21.15.105:0/423492995 >> 172.21.15.105:0/423492995 conn(0x7fea740fce20 msgr2=0x7fea740ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 -- 172.21.15.105:0/423492995 shutdown_connections 2024-03-26T17:51:39.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 -- 172.21.15.105:0/423492995 wait complete. 2024-03-26T17:51:39.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 Processor -- start 2024-03-26T17:51:39.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.195+0000 7fea7abe6640 1 -- start start 2024-03-26T17:51:39.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea7abe6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea741018d0 0x7fea74075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea7abe6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea74102ac0 0x7fea74073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea7abe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea74103480 0x7fea740741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea7abe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea7410d3d0 con 0x7fea74103480 2024-03-26T17:51:39.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea7abe6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea7410d250 con 0x7fea741018d0 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea7abe6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea7410d550 con 0x7fea74102ac0 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea78982640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea741018d0 0x7fea74075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea78982640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea741018d0 0x7fea74075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34162/0 (socket says 172.21.15.105:34162) 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea78982640 1 -- 172.21.15.105:0/860459467 learned_addr learned my addr 172.21.15.105:0/860459467 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea79183640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea74103480 0x7fea740741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea73fff640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea74102ac0 0x7fea74073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea78982640 1 -- 172.21.15.105:0/860459467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea74102ac0 msgr2=0x7fea74073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea78982640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea74102ac0 0x7fea74073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea78982640 1 -- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea74103480 msgr2=0x7fea740741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea78982640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea74103480 0x7fea740741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea78982640 1 -- 172.21.15.105:0/860459467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea74074a80 con 0x7fea741018d0 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.196+0000 7fea79183640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea74103480 0x7fea740741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.197+0000 7fea78982640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea741018d0 0x7fea74075670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea68000c00 tx=0x7fea68005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.197+0000 7fea71ffb640 1 -- 172.21.15.105:0/860459467 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68043020 con 0x7fea741018d0 2024-03-26T17:51:39.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.197+0000 7fea7abe6640 1 -- 172.21.15.105:0/860459467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea741a1c80 con 0x7fea741018d0 2024-03-26T17:51:39.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.197+0000 7fea71ffb640 1 -- 172.21.15.105:0/860459467 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea68004350 con 0x7fea741018d0 2024-03-26T17:51:39.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.197+0000 7fea71ffb640 1 -- 172.21.15.105:0/860459467 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea68003b80 con 0x7fea741018d0 2024-03-26T17:51:39.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.197+0000 7fea7abe6640 1 -- 172.21.15.105:0/860459467 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea741a21d0 con 0x7fea741018d0 2024-03-26T17:51:39.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea7d7d2740 1 Processor -- start 2024-03-26T17:51:39.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea7d7d2740 1 -- start start 2024-03-26T17:51:39.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be09660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 0x55d04be0e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be11230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d04bd65970 con 0x55d04be0eda0 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d04bd0dd70 con 0x55d04be09240 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d04bd61920 con 0x55d04be09c30 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea79183640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be09660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea73fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be11230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea79183640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be09660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34164/0 (socket says 172.21.15.105:34164) 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea73fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be11230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46476/0 (socket says 172.21.15.105:46476) 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.200+0000 7fea79183640 1 -- 172.21.15.105:0/118123602 learned_addr learned my addr 172.21.15.105:0/118123602 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea73fff640 1 -- 172.21.15.105:0/118123602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 msgr2=0x55d04be0e730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea73fff640 1 --2- 172.21.15.105:0/118123602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 0x55d04be0e730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea73fff640 1 -- 172.21.15.105:0/118123602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 msgr2=0x55d04be09660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea73fff640 1 --2- 172.21.15.105:0/118123602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be09660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea73fff640 1 -- 172.21.15.105:0/118123602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04ba793f0 con 0x55d04be0eda0 2024-03-26T17:51:39.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea79183640 1 --2- 172.21.15.105:0/118123602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be09660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea71ffb640 1 -- 172.21.15.105:0/860459467 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fea68046020 con 0x7fea741018d0 2024-03-26T17:51:39.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea73fff640 1 --2- 172.21.15.105:0/118123602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be11230 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fea60009bf0 tx=0x7fea60011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a308d0e8a38e6dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea5effd640 1 -- 172.21.15.105:0/118123602 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6001e070 con 0x55d04be0eda0 2024-03-26T17:51:39.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea5effd640 1 -- 172.21.15.105:0/118123602 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea60009da0 con 0x55d04be0eda0 2024-03-26T17:51:39.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.201+0000 7fea5effd640 1 -- 172.21.15.105:0/118123602 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea600187d0 con 0x55d04be0eda0 2024-03-26T17:51:39.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.202+0000 7fea71ffb640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea640a2f20 0x7fea640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.202+0000 7fea79183640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea640a2f20 0x7fea640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.202+0000 7fea71ffb640 1 -- 172.21.15.105:0/860459467 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fea680f1550 con 0x7fea741018d0 2024-03-26T17:51:39.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.202+0000 7fea79183640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea640a2f20 0x7fea640a53e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55d04be0a490 tx=0x7fea6c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.202+0000 7fea7d7d2740 1 -- 172.21.15.105:0/118123602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 msgr2=0x55d04be11230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.202+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/118123602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be11230 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fea60009bf0 tx=0x7fea60011810 comp rx=0 tx=0).stop 2024-03-26T17:51:39.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 -- 172.21.15.105:0/118123602 shutdown_connections 2024-03-26T17:51:39.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/118123602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be11230 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/118123602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 0x55d04be0e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/118123602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be09660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 -- 172.21.15.105:0/118123602 >> 172.21.15.105:0/118123602 conn(0x55d04be047b0 msgr2=0x55d04be04bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 -- 172.21.15.105:0/118123602 shutdown_connections 2024-03-26T17:51:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 -- 172.21.15.105:0/118123602 wait complete. 2024-03-26T17:51:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 -- start start 2024-03-26T17:51:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.203+0000 7fea7d7d2740 1 Processor -- start 2024-03-26T17:51:39.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09240 0x55d04be137a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09c30 0x55d04be13ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d04be11d50 con 0x55d04be09240 2024-03-26T17:51:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d04bd21310 con 0x55d04be09c30 2024-03-26T17:51:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea78982640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09240 0x55d04be137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea78982640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09240 0x55d04be137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46488/0 (socket says 172.21.15.105:46488) 2024-03-26T17:51:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea78982640 1 -- 172.21.15.105:0/959664066 learned_addr learned my addr 172.21.15.105:0/959664066 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea79183640 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea73fff640 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09c30 0x55d04be13ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.204+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d04be11ed0 con 0x55d04be0eda0 2024-03-26T17:51:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea7d7d2740 10 client.0 _ll_register_callbacks cb 0x55d04bd5cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea79183640 1 -- 172.21.15.105:0/959664066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09c30 msgr2=0x55d04be13ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea79183640 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09c30 0x55d04be13ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea79183640 1 -- 172.21.15.105:0/959664066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09240 msgr2=0x55d04be137a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea79183640 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09240 0x55d04be137a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea79183640 1 -- 172.21.15.105:0/959664066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04bb437d0 con 0x55d04be0eda0 2024-03-26T17:51:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea79183640 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ad20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009fd0 tx=0x7fea6c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88e5ff2b44111b65 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea53fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:39.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea53fff640 1 -- 172.21.15.105:0/959664066 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c004770 con 0x55d04be0eda0 2024-03-26T17:51:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea53fff640 1 -- 172.21.15.105:0/959664066 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fea6c017030 con 0x55d04be0eda0 2024-03-26T17:51:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.205+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea60009da0 con 0x55d04be0eda0 2024-03-26T17:51:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea53fff640 1 -- 172.21.15.105:0/959664066 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea6c011020 con 0x55d04be0eda0 2024-03-26T17:51:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea7d7d2740 10 client.30133 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d04be1e0e0 con 0x55d04be0eda0 2024-03-26T17:51:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea53fff640 1 -- 172.21.15.105:0/959664066 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea60009da0 con 0x55d04be0eda0 2024-03-26T17:51:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea53fff640 1 -- 172.21.15.105:0/959664066 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fea6c0105f0 con 0x55d04be0eda0 2024-03-26T17:51:39.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea7d7d2740 10 client.30133 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea7d7d2740 10 client.30133 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea7d7d2740 20 client.30133 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d04be1f800 0x55d04be21cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea7d7d2740 4 client.30133 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.206+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d04bd72eb0 con 0x55d04be1f800 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.207+0000 7fea78982640 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d04be1f800 0x55d04be21cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.208+0000 7fea78982640 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d04be1f800 0x55d04be21cc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77b0306541b3b0db server_cookie=26041a7096b7e050 in_seq=0 out_seq=0 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.208+0000 7fea53fff640 10 client.30133 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.209+0000 7fea53fff640 1 -- 172.21.15.105:0/959664066 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d04bd72eb0 con 0x55d04be1f800 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.209+0000 7fea53fff640 10 client.30133 handle_command_reply: tid=0 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.209+0000 7fea7d7d2740 1 client.30133 shutdown 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 msgr2=0x55d04be1ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ad20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009fd0 tx=0x7fea6c004ab0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 shutdown_connections 2024-03-26T17:51:39.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ad20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d04be1f800 0x55d04be21cc0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09c30 0x55d04be13ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea53fff640 0 client.30133 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:51:39.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/959664066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09240 0x55d04be137a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 >> 172.21.15.105:0/959664066 conn(0x55d04be047b0 msgr2=0x55d04bdfb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 shutdown_connections 2024-03-26T17:51:39.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 1 -- 172.21.15.105:0/959664066 wait complete. 2024-03-26T17:51:39.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.210+0000 7fea7d7d2740 20 client.30133 trim_cache size 0 max 16384 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea7d7d2740 1 Processor -- start 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea7d7d2740 1 -- start start 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be0b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09c30 0x55d04be0b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea741a6c10 con 0x55d04be09c30 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea741a8810 con 0x55d04be09240 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea741a72e0 con 0x55d04be0eda0 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea73fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09c30 0x55d04be0b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea78982640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be0b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea79183640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea79183640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46750/0 (socket says 172.21.15.105:46750) 2024-03-26T17:51:39.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea79183640 1 -- 172.21.15.105:0/2647573342 learned_addr learned my addr 172.21.15.105:0/2647573342 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.232+0000 7fea78982640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be0b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34176/0 (socket says 172.21.15.105:34176) 2024-03-26T17:51:39.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea79183640 1 -- 172.21.15.105:0/2647573342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 msgr2=0x55d04be0b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea79183640 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be0b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea79183640 1 -- 172.21.15.105:0/2647573342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09c30 msgr2=0x55d04be0b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea79183640 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09c30 0x55d04be0b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea79183640 1 -- 172.21.15.105:0/2647573342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04be05520 con 0x55d04be0eda0 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea73fff640 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09c30 0x55d04be0b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea78982640 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be0b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea79183640 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ace0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fea6c010a60 tx=0x7fea6c005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe0d99e465b24048 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea5e7fc640 1 -- 172.21.15.105:0/2647573342 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c037070 con 0x55d04be0eda0 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea5e7fc640 1 -- 172.21.15.105:0/2647573342 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea6c011240 con 0x55d04be0eda0 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.233+0000 7fea5e7fc640 1 -- 172.21.15.105:0/2647573342 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c01dbe0 con 0x55d04be0eda0 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.234+0000 7fea7d7d2740 1 -- 172.21.15.105:0/2647573342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 msgr2=0x55d04be1ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.234+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ace0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fea6c010a60 tx=0x7fea6c005c50 comp rx=0 tx=0).stop 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.234+0000 7fea7d7d2740 1 -- 172.21.15.105:0/2647573342 shutdown_connections 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.234+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be0eda0 0x55d04be1ace0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.234+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be09c30 0x55d04be0b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.234+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/2647573342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be0b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.234+0000 7fea7d7d2740 1 -- 172.21.15.105:0/2647573342 >> 172.21.15.105:0/2647573342 conn(0x55d04be047b0 msgr2=0x55d04be12270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.235+0000 7fea7d7d2740 1 -- 172.21.15.105:0/2647573342 shutdown_connections 2024-03-26T17:51:39.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.235+0000 7fea7d7d2740 1 -- 172.21.15.105:0/2647573342 wait complete. 2024-03-26T17:51:39.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.235+0000 7fea7d7d2740 1 -- start start 2024-03-26T17:51:39.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.235+0000 7fea7d7d2740 1 Processor -- start 2024-03-26T17:51:39.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.235+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.235+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 0x55d04be186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea7d7d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be19680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea7d7d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea741a8140 con 0x55d04be0eda0 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be181a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34178/0 (socket says 172.21.15.105:34178) 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 -- 172.21.15.105:0/1428382826 learned_addr learned my addr 172.21.15.105:0/1428382826 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea79183640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be19680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea73fff640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 0x55d04be186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea741a79b0 con 0x55d04be09240 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea741a4bf0 con 0x55d04be09c30 2024-03-26T17:51:39.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea7d7d2740 10 client.0 _ll_register_callbacks cb 0x7fea6c003c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 -- 172.21.15.105:0/1428382826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 msgr2=0x55d04be186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 0x55d04be186e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 -- 172.21.15.105:0/1428382826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 msgr2=0x55d04be19680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be19680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 -- 172.21.15.105:0/1428382826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04be19c70 con 0x55d04be09240 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea73fff640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 0x55d04be186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea79183640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be19680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea78982640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be181a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fea680ba0a0 tx=0x7fea680f1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2005886a9668926e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.236+0000 7fea53fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea53fff640 1 -- 172.21.15.105:0/1428382826 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea680c1d90 con 0x55d04be09240 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea6c011240 con 0x55d04be09240 2024-03-26T17:51:39.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea53fff640 1 -- 172.21.15.105:0/1428382826 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fea680054d0 con 0x55d04be09240 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea53fff640 1 -- 172.21.15.105:0/1428382826 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea680c6650 con 0x55d04be09240 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 10 client.30158 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d04be1aff0 con 0x55d04be09240 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea53fff640 1 -- 172.21.15.105:0/1428382826 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea6c011240 con 0x55d04be09240 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea53fff640 1 -- 172.21.15.105:0/1428382826 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fea6809cbd0 con 0x55d04be09240 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 10 client.30158 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 10 client.30158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 20 client.30158 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d04be1f800 0x55d04be88380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 4 client.30158 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.237+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d04bd6d200 con 0x55d04be1f800 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.238+0000 7fea73fff640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d04be1f800 0x55d04be88380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.238+0000 7fea53fff640 10 client.30158 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.238+0000 7fea73fff640 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d04be1f800 0x55d04be88380 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9261e7c769ea77dd server_cookie=cf1acd3814c65f1d in_seq=0 out_seq=0 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea53fff640 1 -- 172.21.15.105:0/1428382826 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d04bd6d200 con 0x55d04be1f800 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea53fff640 10 client.30158 handle_command_reply: tid=0 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 client.30158 shutdown 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 msgr2=0x55d04be181a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be181a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fea680ba0a0 tx=0x7fea680f1040 comp rx=0 tx=0).stop 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 shutdown_connections 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d04be0eda0 0x55d04be19680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d04be09c30 0x55d04be186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d04be1f800 0x55d04be88380 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 --2- 172.21.15.105:0/1428382826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d04be09240 0x55d04be181a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 >> 172.21.15.105:0/1428382826 conn(0x55d04be047b0 msgr2=0x55d04be13760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea53fff640 0 client.30158 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:51:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 shutdown_connections 2024-03-26T17:51:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 1 -- 172.21.15.105:0/1428382826 wait complete. 2024-03-26T17:51:39.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.239+0000 7fea7d7d2740 20 client.30158 trim_cache size 0 max 16384 2024-03-26T17:51:39.237 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:39.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 -- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea640a2f20 msgr2=0x7fea640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea640a2f20 0x7fea640a53e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55d04be0a490 tx=0x7fea6c009500 comp rx=0 tx=0).stop 2024-03-26T17:51:39.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 -- 172.21.15.105:0/860459467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea741018d0 msgr2=0x7fea74075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea741018d0 0x7fea74075670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fea68000c00 tx=0x7fea68005dc0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 -- 172.21.15.105:0/860459467 shutdown_connections 2024-03-26T17:51:39.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea640a2f20 0x7fea640a53e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea74103480 0x7fea740741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea74102ac0 0x7fea74073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 --2- 172.21.15.105:0/860459467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea741018d0 0x7fea74075670 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 -- 172.21.15.105:0/860459467 >> 172.21.15.105:0/860459467 conn(0x7fea740fce20 msgr2=0x7fea74105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 -- 172.21.15.105:0/860459467 shutdown_connections 2024-03-26T17:51:39.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.240+0000 7fea7abe6640 1 -- 172.21.15.105:0/860459467 wait complete. 2024-03-26T17:51:39.246 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:51:39.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:38 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3707062842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:51:39.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 -- 172.21.15.105:0/2167454921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74103480 msgr2=0x7fdf74109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 --2- 172.21.15.105:0/2167454921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74103480 0x7fdf74109d10 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009ca0 tx=0x7fdf6c0301f0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 -- 172.21.15.105:0/2167454921 shutdown_connections 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 --2- 172.21.15.105:0/2167454921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74103480 0x7fdf74109d10 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 --2- 172.21.15.105:0/2167454921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf74102ac0 0x7fdf74102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 --2- 172.21.15.105:0/2167454921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf741018d0 0x7fdf74101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 -- 172.21.15.105:0/2167454921 >> 172.21.15.105:0/2167454921 conn(0x7fdf740fce40 msgr2=0x7fdf740ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 -- 172.21.15.105:0/2167454921 shutdown_connections 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.430+0000 7fdf796c6640 1 -- 172.21.15.105:0/2167454921 wait complete. 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf796c6640 1 Processor -- start 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf796c6640 1 -- start start 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf796c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf741018d0 0x7fdf741958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf796c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102ac0 0x7fdf74195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf796c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf74103480 0x7fdf7419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf796c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410b220 con 0x7fdf74102ac0 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf796c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410b0a0 con 0x7fdf741018d0 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf796c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410b3a0 con 0x7fdf74103480 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102ac0 0x7fdf74195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf741018d0 0x7fdf741958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102ac0 0x7fdf74195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46534/0 (socket says 172.21.15.105:46534) 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf741018d0 0x7fdf741958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34196/0 (socket says 172.21.15.105:34196) 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.431+0000 7fdf6bfff640 1 -- 172.21.15.105:0/1366181936 learned_addr learned my addr 172.21.15.105:0/1366181936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf737fe640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf74103480 0x7fdf7419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf6bfff640 1 -- 172.21.15.105:0/1366181936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf74103480 msgr2=0x7fdf7419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf74103480 0x7fdf7419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf6bfff640 1 -- 172.21.15.105:0/1366181936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf741018d0 msgr2=0x7fdf741958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf741018d0 0x7fdf741958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf6bfff640 1 -- 172.21.15.105:0/1366181936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7419d4d0 con 0x7fdf74102ac0 2024-03-26T17:51:39.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102ac0 0x7fdf74195df0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9f0 tx=0x7fdf5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf70ff9640 1 -- 172.21.15.105:0/1366181936 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00cd80 con 0x7fdf74102ac0 2024-03-26T17:51:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.432+0000 7fdf70ff9640 1 -- 172.21.15.105:0/1366181936 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf5c0054d0 con 0x7fdf74102ac0 2024-03-26T17:51:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.433+0000 7fdf796c6640 1 -- 172.21.15.105:0/1366181936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7419d7c0 con 0x7fdf74102ac0 2024-03-26T17:51:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.433+0000 7fdf70ff9640 1 -- 172.21.15.105:0/1366181936 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c010490 con 0x7fdf74102ac0 2024-03-26T17:51:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.433+0000 7fdf796c6640 1 -- 172.21.15.105:0/1366181936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7419dcb0 con 0x7fdf74102ac0 2024-03-26T17:51:39.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.436+0000 7fdf70ff9640 1 -- 172.21.15.105:0/1366181936 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdf5c005020 con 0x7fdf74102ac0 2024-03-26T17:51:39.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.436+0000 7fdf70ff9640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf70ff9640 1 -- 172.21.15.105:0/1366181936 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fdf5c0cf6f0 con 0x7fdf74102ac0 2024-03-26T17:51:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf7c392740 1 Processor -- start 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf7c392740 1 -- start start 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c5f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c5fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c5fea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf7c392740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55995c55b970 con 0x55995c5fc590 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf7c392740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55995c503d70 con 0x55995c5f6a30 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf7c392740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55995c557920 con 0x55995c5f7420 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdf74102920 tx=0x7fdf64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c5f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c5fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c5fbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46776/0 (socket says 172.21.15.105:46776) 2024-03-26T17:51:39.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf6bfff640 1 -- 172.21.15.105:0/2251653728 learned_addr learned my addr 172.21.15.105:0/2251653728 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.437+0000 7fdf737fe640 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c5fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf6bfff640 1 -- 172.21.15.105:0/2251653728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 msgr2=0x55995c5f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c5f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf6bfff640 1 -- 172.21.15.105:0/2251653728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 msgr2=0x55995c5fea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c5fea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf6bfff640 1 -- 172.21.15.105:0/2251653728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55995c26f3f0 con 0x55995c5f7420 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c5f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c5fbf20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c098240 tx=0x7fdf5c09b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57185db2a4ff0eae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf697fa640 1 -- 172.21.15.105:0/2251653728 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c09c4e0 con 0x55995c5f7420 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf697fa640 1 -- 172.21.15.105:0/2251653728 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf5c09bde0 con 0x55995c5f7420 2024-03-26T17:51:39.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.438+0000 7fdf697fa640 1 -- 172.21.15.105:0/2251653728 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0a4360 con 0x55995c5f7420 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 -- 172.21.15.105:0/2251653728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 msgr2=0x55995c5fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c5fbf20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c098240 tx=0x7fdf5c09b800 comp rx=0 tx=0).stop 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 -- 172.21.15.105:0/2251653728 shutdown_connections 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c5fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c5fbf20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 --2- 172.21.15.105:0/2251653728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c5f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 -- 172.21.15.105:0/2251653728 >> 172.21.15.105:0/2251653728 conn(0x55995c5f1fa0 msgr2=0x55995c5f23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 -- 172.21.15.105:0/2251653728 shutdown_connections 2024-03-26T17:51:39.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.439+0000 7fdf7c392740 1 -- 172.21.15.105:0/2251653728 wait complete. 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 1 -- start start 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 1 Processor -- start 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5f6a30 0x55995c609740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f7420 0x55995c609c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5fc590 0x55995c610cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55995c5ff540 con 0x55995c5f6a30 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55995c517310 con 0x55995c5f7420 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55995c5ff6c0 con 0x55995c5fc590 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf7c392740 10 client.0 _ll_register_callbacks cb 0x55995c552c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5f6a30 0x55995c609740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f7420 0x55995c609c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5f6a30 0x55995c609740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46540/0 (socket says 172.21.15.105:46540) 2024-03-26T17:51:39.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.440+0000 7fdf72ffd640 1 -- 172.21.15.105:0/752202849 learned_addr learned my addr 172.21.15.105:0/752202849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf737fe640 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5fc590 0x55995c610cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf72ffd640 1 -- 172.21.15.105:0/752202849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5fc590 msgr2=0x55995c610cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5fc590 0x55995c610cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf72ffd640 1 -- 172.21.15.105:0/752202849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f7420 msgr2=0x55995c609c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f7420 0x55995c609c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf72ffd640 1 -- 172.21.15.105:0/752202849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995c3397d0 con 0x55995c5f6a30 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f7420 0x55995c609c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5f6a30 0x55995c609740 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fdf6401eca0 tx=0x7fdf64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7e0dd2698375a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/752202849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6401e020 con 0x55995c5f6a30 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf7c392740 1 -- 172.21.15.105:0/752202849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5c09bde0 con 0x55995c5f6a30 2024-03-26T17:51:39.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.441+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/752202849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fdf6401e1c0 con 0x55995c5f6a30 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/752202849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdf6400eb40 con 0x55995c5f6a30 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf7c392740 10 client.20364 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf7c392740 1 -- 172.21.15.105:0/752202849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55995c614100 con 0x55995c5f6a30 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/752202849 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf5c09bde0 con 0x55995c5f6a30 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/752202849 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdf64025070 con 0x55995c5f6a30 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf7c392740 10 client.20364 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf7c392740 10 client.20364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf7c392740 20 client.20364 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf7c392740 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55995c615770 0x55995c617c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf7c392740 4 client.20364 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf7c392740 1 -- 172.21.15.105:0/752202849 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55995c568eb0 con 0x55995c615770 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.442+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55995c615770 0x55995c617c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.443+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55995c615770 0x55995c617c10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61818decfb14f9c5 server_cookie=716e9d6464862c8 in_seq=0 out_seq=0 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.443+0000 7fdf5a7fc640 10 client.20364 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/752202849 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55995c568eb0 con 0x55995c615770 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf5a7fc640 10 client.20364 handle_command_reply: tid=0 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 client.20364 shutdown 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 -- 172.21.15.105:0/752202849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5f6a30 msgr2=0x55995c609740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5f6a30 0x55995c609740 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fdf6401eca0 tx=0x7fdf64004730 comp rx=0 tx=0).stop 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 -- 172.21.15.105:0/752202849 shutdown_connections 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55995c615770 0x55995c617c10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5fc590 0x55995c610cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f7420 0x55995c609c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 --2- 172.21.15.105:0/752202849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5f6a30 0x55995c609740 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 -- 172.21.15.105:0/752202849 >> 172.21.15.105:0/752202849 conn(0x55995c5f1fa0 msgr2=0x55995c5f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf5a7fc640 0 client.20364 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:51:39.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 -- 172.21.15.105:0/752202849 shutdown_connections 2024-03-26T17:51:39.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 1 -- 172.21.15.105:0/752202849 wait complete. 2024-03-26T17:51:39.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.444+0000 7fdf7c392740 20 client.20364 trim_cache size 0 max 16384 2024-03-26T17:51:39.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf7c392740 1 Processor -- start 2024-03-26T17:51:39.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf7c392740 1 -- start start 2024-03-26T17:51:39.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c609350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c610c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c609890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf7c392740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741a6c20 con 0x55995c5fc590 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf7c392740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741a8820 con 0x55995c5f6a30 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf7c392740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741a72f0 con 0x55995c5f7420 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c610c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c609890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c609350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c610c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46796/0 (socket says 172.21.15.105:46796) 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c609350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34236/0 (socket says 172.21.15.105:34236) 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.464+0000 7fdf6bfff640 1 -- 172.21.15.105:0/1113714466 learned_addr learned my addr 172.21.15.105:0/1113714466 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf737fe640 1 -- 172.21.15.105:0/1113714466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 msgr2=0x55995c610c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf737fe640 1 --2- 172.21.15.105:0/1113714466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c610c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf737fe640 1 -- 172.21.15.105:0/1113714466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 msgr2=0x55995c609350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf737fe640 1 --2- 172.21.15.105:0/1113714466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c609350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf737fe640 1 -- 172.21.15.105:0/1113714466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55995c5ed820 con 0x55995c5fc590 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/1113714466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c609350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf737fe640 1 --2- 172.21.15.105:0/1113714466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c609890 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c03b040 tx=0x7fdf6c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1cc9648764773d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf68ff9640 1 -- 172.21.15.105:0/1113714466 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c03d040 con 0x55995c5fc590 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf68ff9640 1 -- 172.21.15.105:0/1113714466 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6c048070 con 0x55995c5fc590 2024-03-26T17:51:39.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.465+0000 7fdf68ff9640 1 -- 172.21.15.105:0/1113714466 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c00f2d0 con 0x55995c5fc590 2024-03-26T17:51:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 -- 172.21.15.105:0/1113714466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 msgr2=0x55995c609890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 --2- 172.21.15.105:0/1113714466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c609890 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c03b040 tx=0x7fdf6c009890 comp rx=0 tx=0).stop 2024-03-26T17:51:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 -- 172.21.15.105:0/1113714466 shutdown_connections 2024-03-26T17:51:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 --2- 172.21.15.105:0/1113714466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c609890 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 --2- 172.21.15.105:0/1113714466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c610c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 --2- 172.21.15.105:0/1113714466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c609350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 -- 172.21.15.105:0/1113714466 >> 172.21.15.105:0/1113714466 conn(0x55995c5f1fa0 msgr2=0x55995c5f8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 -- 172.21.15.105:0/1113714466 shutdown_connections 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 -- 172.21.15.105:0/1113714466 wait complete. 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.466+0000 7fdf7c392740 1 -- start start 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf7c392740 1 Processor -- start 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c60da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c60df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf7c392740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c679b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf7c392740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741a8150 con 0x55995c5fc590 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf7c392740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741a79c0 con 0x55995c5f6a30 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf7c392740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf741a4c00 con 0x55995c5f7420 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf7c392740 10 client.0 _ll_register_callbacks cb 0x7fdf640099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c60df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c60da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c60df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46804/0 (socket says 172.21.15.105:46804) 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf737fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c679b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 -- 172.21.15.105:0/3395428376 learned_addr learned my addr 172.21.15.105:0/3395428376 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf72ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c60da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34248/0 (socket says 172.21.15.105:34248) 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 -- 172.21.15.105:0/3395428376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 msgr2=0x55995c60da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c60da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 -- 172.21.15.105:0/3395428376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 msgr2=0x55995c679b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c679b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 -- 172.21.15.105:0/3395428376 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995c60e5c0 con 0x55995c5f7420 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf737fe640 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c679b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf6bfff640 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c60df80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0987b0 tx=0x7fdf5c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fadc25bf2af7bb50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.467+0000 7fdf5a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/3395428376 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0a4560 con 0x55995c5f7420 2024-03-26T17:51:39.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/3395428376 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fdf5c0bab50 con 0x55995c5f7420 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 1 -- 172.21.15.105:0/3395428376 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf6c048070 con 0x55995c5f7420 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/3395428376 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdf5c09c4e0 con 0x55995c5f7420 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 10 client.30157 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 1 -- 172.21.15.105:0/3395428376 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55995c67b110 con 0x55995c5f7420 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/3395428376 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf6c048070 con 0x55995c5f7420 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/3395428376 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdf5c097b30 con 0x55995c5f7420 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 10 client.30157 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 10 client.30157 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 20 client.30157 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55995c67b600 0x55995c67da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 4 client.30157 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.468+0000 7fdf7c392740 1 -- 172.21.15.105:0/3395428376 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55995c563200 con 0x55995c67b600 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.469+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55995c67b600 0x55995c67da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.469+0000 7fdf5a7fc640 10 client.30157 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.469+0000 7fdf72ffd640 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55995c67b600 0x55995c67da20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=805c50bc58021b4 server_cookie=faa0818f14b5738d in_seq=0 out_seq=0 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf5a7fc640 1 -- 172.21.15.105:0/3395428376 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55995c563200 con 0x55995c67b600 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf5a7fc640 10 client.30157 handle_command_reply: tid=0 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 client.30157 shutdown 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 -- 172.21.15.105:0/3395428376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 msgr2=0x55995c60df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c60df80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0987b0 tx=0x7fdf5c009310 comp rx=0 tx=0).stop 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 -- 172.21.15.105:0/3395428376 shutdown_connections 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55995c67b600 0x55995c67da20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55995c5fc590 0x55995c679b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf5a7fc640 0 client.30157 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995c5f7420 0x55995c60df80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 --2- 172.21.15.105:0/3395428376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55995c5f6a30 0x55995c60da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 -- 172.21.15.105:0/3395428376 >> 172.21.15.105:0/3395428376 conn(0x55995c5f1fa0 msgr2=0x55995c609350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 -- 172.21.15.105:0/3395428376 shutdown_connections 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 1 -- 172.21.15.105:0/3395428376 wait complete. 2024-03-26T17:51:39.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.470+0000 7fdf7c392740 20 client.30157 trim_cache size 0 max 16384 2024-03-26T17:51:39.469 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 -- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 msgr2=0x7fdf600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdf74102920 tx=0x7fdf64005f90 comp rx=0 tx=0).stop 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 -- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102ac0 msgr2=0x7fdf74195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102ac0 0x7fdf74195df0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9f0 tx=0x7fdf5c00edf0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 -- 172.21.15.105:0/1366181936 shutdown_connections 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdf600a2fa0 0x7fdf600a5460 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf74103480 0x7fdf7419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf74102ac0 0x7fdf74195df0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 --2- 172.21.15.105:0/1366181936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdf741018d0 0x7fdf741958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 -- 172.21.15.105:0/1366181936 >> 172.21.15.105:0/1366181936 conn(0x7fdf740fce40 msgr2=0x7fdf74105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 -- 172.21.15.105:0/1366181936 shutdown_connections 2024-03-26T17:51:39.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.471+0000 7fdf796c6640 1 -- 172.21.15.105:0/1366181936 wait complete. 2024-03-26T17:51:39.477 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:51:39.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 3137 14.83 MB/sec warmup 22 sec latency 216.762 ms 2024-03-26T17:51:39.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 -- 172.21.15.105:0/2099331736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e41088c0 msgr2=0x7f32e410acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 --2- 172.21.15.105:0/2099331736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e41088c0 0x7f32e410acf0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009860 tx=0x7f32e002bb70 comp rx=0 tx=0).stop 2024-03-26T17:51:39.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 -- 172.21.15.105:0/2099331736 shutdown_connections 2024-03-26T17:51:39.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 --2- 172.21.15.105:0/2099331736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e4112000 0x7f32e4114410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 --2- 172.21.15.105:0/2099331736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e410b230 0x7f32e410b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 --2- 172.21.15.105:0/2099331736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e41088c0 0x7f32e410acf0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 -- 172.21.15.105:0/2099331736 >> 172.21.15.105:0/2099331736 conn(0x7f32e406e720 msgr2=0x7f32e4070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 -- 172.21.15.105:0/2099331736 shutdown_connections 2024-03-26T17:51:39.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.623+0000 7f32ebc27640 1 -- 172.21.15.105:0/2099331736 wait complete. 2024-03-26T17:51:39.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ebc27640 1 Processor -- start 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ebc27640 1 -- start start 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ebc27640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e410b230 0x7f32e41a2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ebc27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e4112000 0x7f32e41a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ebc27640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e41a7d00 0x7f32e41aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ebc27640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4117e20 con 0x7f32e4112000 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ebc27640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4117ca0 con 0x7f32e41a7d00 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ebc27640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4117fa0 con 0x7f32e410b230 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32e99c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e410b230 0x7f32e41a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32e91c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e4112000 0x7f32e41a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32e91c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e4112000 0x7f32e41a2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46568/0 (socket says 172.21.15.105:46568) 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32e91c2640 1 -- 172.21.15.105:0/3189660002 learned_addr learned my addr 172.21.15.105:0/3189660002 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32e99c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e410b230 0x7f32e41a2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46822/0 (socket says 172.21.15.105:46822) 2024-03-26T17:51:39.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.624+0000 7f32ea1c4640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e41a7d00 0x7f32e41aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32e99c3640 1 -- 172.21.15.105:0/3189660002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e41a7d00 msgr2=0x7f32e41aa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32e99c3640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e41a7d00 0x7f32e41aa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32e99c3640 1 -- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e4112000 msgr2=0x7f32e41a2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32e99c3640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e4112000 0x7f32e41a2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32e99c3640 1 -- 172.21.15.105:0/3189660002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e41aa760 con 0x7f32e410b230 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32ea1c4640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e41a7d00 0x7f32e41aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32e99c3640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e410b230 0x7f32e41a2760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f32e0008000 tx=0x7f32e00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32daffd640 1 -- 172.21.15.105:0/3189660002 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0041020 con 0x7f32e410b230 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32daffd640 1 -- 172.21.15.105:0/3189660002 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32e0005020 con 0x7f32e410b230 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32ebc27640 1 -- 172.21.15.105:0/3189660002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e41aa9f0 con 0x7f32e410b230 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32daffd640 1 -- 172.21.15.105:0/3189660002 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0039040 con 0x7f32e410b230 2024-03-26T17:51:39.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.625+0000 7f32ebc27640 1 -- 172.21.15.105:0/3189660002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e41aaeb0 con 0x7f32e410b230 2024-03-26T17:51:39.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.628+0000 7f32daffd640 1 -- 172.21.15.105:0/3189660002 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f32e00051d0 con 0x7f32e410b230 2024-03-26T17:51:39.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.628+0000 7f32ee80e740 1 Processor -- start 2024-03-26T17:51:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ee80e740 1 -- start start 2024-03-26T17:51:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8bf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8c4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ee80e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55725c81b970 con 0x55725c8bf240 2024-03-26T17:51:39.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ee80e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55725c7c3d70 con 0x55725c8bfc30 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ee80e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55725c817920 con 0x55725c8c4da0 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ea1c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8bf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32daffd640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32d00a2f20 0x7f32d00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32e91c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32e91c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46832/0 (socket says 172.21.15.105:46832) 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32e91c2640 1 -- 172.21.15.105:0/2876181464 learned_addr learned my addr 172.21.15.105:0/2876181464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32daffd640 1 -- 172.21.15.105:0/3189660002 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f32e00f0780 con 0x7f32e410b230 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.629+0000 7f32ea1c4640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32d00a2f20 0x7f32d00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32e91c2640 1 -- 172.21.15.105:0/2876181464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 msgr2=0x55725c8c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32e91c2640 1 --2- 172.21.15.105:0/2876181464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32e91c2640 1 -- 172.21.15.105:0/2876181464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 msgr2=0x55725c8bf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32e91c2640 1 --2- 172.21.15.105:0/2876181464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8bf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32e91c2640 1 -- 172.21.15.105:0/2876181464 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55725c52f3f0 con 0x55725c8c4da0 2024-03-26T17:51:39.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32e91c2640 1 --2- 172.21.15.105:0/2876181464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8c7230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f32d400aca0 tx=0x7f32d400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131fc9f14b874994 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32ea1c4640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32d00a2f20 0x7f32d00a53e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55725c8c0490 tx=0x7f32dc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32cffff640 1 -- 172.21.15.105:0/2876181464 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d401d070 con 0x55725c8c4da0 2024-03-26T17:51:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32cffff640 1 -- 172.21.15.105:0/2876181464 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d4018af0 con 0x55725c8c4da0 2024-03-26T17:51:39.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.630+0000 7f32cffff640 1 -- 172.21.15.105:0/2876181464 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4010c60 con 0x55725c8c4da0 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 -- 172.21.15.105:0/2876181464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 msgr2=0x55725c8c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 --2- 172.21.15.105:0/2876181464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8c7230 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f32d400aca0 tx=0x7f32d400f790 comp rx=0 tx=0).stop 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 -- 172.21.15.105:0/2876181464 shutdown_connections 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 --2- 172.21.15.105:0/2876181464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8c7230 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 --2- 172.21.15.105:0/2876181464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 --2- 172.21.15.105:0/2876181464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8bf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 -- 172.21.15.105:0/2876181464 >> 172.21.15.105:0/2876181464 conn(0x55725c8ba7b0 msgr2=0x55725c8babb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 -- 172.21.15.105:0/2876181464 shutdown_connections 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 -- 172.21.15.105:0/2876181464 wait complete. 2024-03-26T17:51:39.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 -- start start 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.631+0000 7f32ee80e740 1 Processor -- start 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bf240 0x55725c8c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8bfc30 0x55725c8c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8c4da0 0x55725c8d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ee80e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55725c8c7d50 con 0x55725c8c4da0 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ee80e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55725c7d7310 con 0x55725c8bf240 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ee80e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55725c8c7ed0 con 0x55725c8bfc30 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ee80e740 10 client.0 _ll_register_callbacks cb 0x55725c812c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e91c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8bfc30 0x55725c8c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e99c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bf240 0x55725c8c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e91c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8bfc30 0x55725c8c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46838/0 (socket says 172.21.15.105:46838) 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e91c2640 1 -- 172.21.15.105:0/3762949233 learned_addr learned my addr 172.21.15.105:0/3762949233 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ea1c4640 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8c4da0 0x55725c8d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e91c2640 1 -- 172.21.15.105:0/3762949233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bf240 msgr2=0x55725c8c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e91c2640 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bf240 0x55725c8c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e91c2640 1 -- 172.21.15.105:0/3762949233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8c4da0 msgr2=0x55725c8d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e91c2640 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8c4da0 0x55725c8d0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e91c2640 1 -- 172.21.15.105:0/3762949233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55725c5f97d0 con 0x55725c8bfc30 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32e99c3640 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bf240 0x55725c8c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.632+0000 7f32ea1c4640 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8c4da0 0x55725c8d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32e91c2640 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8bfc30 0x55725c8c9d00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f32d4010c10 tx=0x7f32d4010c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faa9d362dc4696df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32cd7fa640 1 -- 172.21.15.105:0/3762949233 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4027020 con 0x55725c8bfc30 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32cd7fa640 1 -- 172.21.15.105:0/3762949233 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f32d4018640 con 0x55725c8bfc30 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32ee80e740 1 -- 172.21.15.105:0/3762949233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d4018af0 con 0x55725c8bfc30 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32cd7fa640 1 -- 172.21.15.105:0/3762949233 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f32d4032050 con 0x55725c8bfc30 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32ee80e740 10 client.30169 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32ee80e740 1 -- 172.21.15.105:0/3762949233 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55725c8d40b0 con 0x55725c8bfc30 2024-03-26T17:51:39.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.633+0000 7f32cd7fa640 1 -- 172.21.15.105:0/3762949233 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d4018af0 con 0x55725c8bfc30 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.634+0000 7f32cd7fa640 1 -- 172.21.15.105:0/3762949233 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f32d40046c0 con 0x55725c8bfc30 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.634+0000 7f32ee80e740 10 client.30169 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.634+0000 7f32ee80e740 10 client.30169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.634+0000 7f32ee80e740 20 client.30169 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.634+0000 7f32ee80e740 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55725c8d5400 0x55725c8d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.634+0000 7f32ee80e740 4 client.30169 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.634+0000 7f32ee80e740 1 -- 172.21.15.105:0/3762949233 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55725c828eb0 con 0x55725c8d5400 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.634+0000 7f32e99c3640 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55725c8d5400 0x55725c8d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.635+0000 7f32e99c3640 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55725c8d5400 0x55725c8d77f0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e837f872c45428ee server_cookie=3ad6f2d000635fc7 in_seq=0 out_seq=0 2024-03-26T17:51:39.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.635+0000 7f32cd7fa640 10 client.30169 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:39.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.636+0000 7f32cd7fa640 1 -- 172.21.15.105:0/3762949233 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55725c828eb0 con 0x55725c8d5400 2024-03-26T17:51:39.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.636+0000 7f32cd7fa640 10 client.30169 handle_command_reply: tid=0 2024-03-26T17:51:39.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.636+0000 7f32ee80e740 1 client.30169 shutdown 2024-03-26T17:51:39.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 -- 172.21.15.105:0/3762949233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8bfc30 msgr2=0x55725c8c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8bfc30 0x55725c8c9d00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f32d4010c10 tx=0x7f32d4010c40 comp rx=0 tx=0).stop 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 -- 172.21.15.105:0/3762949233 shutdown_connections 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55725c8d5400 0x55725c8d77f0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8c4da0 0x55725c8d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8bfc30 0x55725c8c9d00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32cd7fa640 0 client.30169 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 --2- 172.21.15.105:0/3762949233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bf240 0x55725c8c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 -- 172.21.15.105:0/3762949233 >> 172.21.15.105:0/3762949233 conn(0x55725c8ba7b0 msgr2=0x55725c8b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 -- 172.21.15.105:0/3762949233 shutdown_connections 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 1 -- 172.21.15.105:0/3762949233 wait complete. 2024-03-26T17:51:39.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.637+0000 7f32ee80e740 20 client.30169 trim_cache size 0 max 16384 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.654+0000 7f32ee80e740 1 Processor -- start 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.654+0000 7f32ee80e740 1 -- start start 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.654+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.654+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.654+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.654+0000 7f32ee80e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41af960 con 0x55725c8bf240 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.654+0000 7f32ee80e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b1560 con 0x55725c8bfc30 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.654+0000 7f32ee80e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b0030 con 0x55725c8c4da0 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46594/0 (socket says 172.21.15.105:46594) 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 -- 172.21.15.105:0/725209940 learned_addr learned my addr 172.21.15.105:0/725209940 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e91c2640 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32ea1c4640 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 -- 172.21.15.105:0/725209940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 msgr2=0x55725c8b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 -- 172.21.15.105:0/725209940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 msgr2=0x55725c8d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 -- 172.21.15.105:0/725209940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55725c8d7b60 con 0x55725c8bf240 2024-03-26T17:51:39.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32e99c3640 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8c91c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f32e00b98d0 tx=0x7f32e002bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf8fb9ef4bde5f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32cf7fe640 1 -- 172.21.15.105:0/725209940 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00bd600 con 0x55725c8bf240 2024-03-26T17:51:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32cf7fe640 1 -- 172.21.15.105:0/725209940 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32e00bcf00 con 0x55725c8bf240 2024-03-26T17:51:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.655+0000 7f32cf7fe640 1 -- 172.21.15.105:0/725209940 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00bd3c0 con 0x55725c8bf240 2024-03-26T17:51:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 -- 172.21.15.105:0/725209940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 msgr2=0x55725c8c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8c91c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f32e00b98d0 tx=0x7f32e002bb70 comp rx=0 tx=0).stop 2024-03-26T17:51:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 -- 172.21.15.105:0/725209940 shutdown_connections 2024-03-26T17:51:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 --2- 172.21.15.105:0/725209940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8c91c0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 -- 172.21.15.105:0/725209940 >> 172.21.15.105:0/725209940 conn(0x55725c8ba7b0 msgr2=0x55725c8c8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 -- 172.21.15.105:0/725209940 shutdown_connections 2024-03-26T17:51:39.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.656+0000 7f32ee80e740 1 -- 172.21.15.105:0/725209940 wait complete. 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 1 -- start start 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 1 Processor -- start 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b0e90 con 0x55725c8bf240 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41b0700 con 0x55725c8bfc30 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41ad940 con 0x55725c8c4da0 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32ee80e740 10 client.0 _ll_register_callbacks cb 0x7f32d401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.657+0000 7f32e99c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e99c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8d0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46598/0 (socket says 172.21.15.105:46598) 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e99c3640 1 -- 172.21.15.105:0/1318144394 learned_addr learned my addr 172.21.15.105:0/1318144394 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e91c2640 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32ea1c4640 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e99c3640 1 -- 172.21.15.105:0/1318144394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 msgr2=0x55725c8cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e99c3640 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e99c3640 1 -- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 msgr2=0x55725c8cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e99c3640 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e99c3640 1 -- 172.21.15.105:0/1318144394 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55725c939750 con 0x55725c8bf240 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e91c2640 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32e99c3640 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8d0c80 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f32e00ee210 tx=0x7f32e00f0b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8fdfa257078cde server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32cd7fa640 1 -- 172.21.15.105:0/1318144394 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00bd600 con 0x55725c8bf240 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32cd7fa640 1 -- 172.21.15.105:0/1318144394 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f32e00bd160 con 0x55725c8bf240 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32ee80e740 1 -- 172.21.15.105:0/1318144394 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e00bcf00 con 0x55725c8bf240 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.658+0000 7f32cd7fa640 1 -- 172.21.15.105:0/1318144394 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f32e00bd360 con 0x55725c8bf240 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32ee80e740 10 client.20388 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32ee80e740 1 -- 172.21.15.105:0/1318144394 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55725c93aa50 con 0x55725c8bf240 2024-03-26T17:51:39.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32cd7fa640 1 -- 172.21.15.105:0/1318144394 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32e00bcf00 con 0x55725c8bf240 2024-03-26T17:51:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32cd7fa640 1 -- 172.21.15.105:0/1318144394 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f32e00a4080 con 0x55725c8bf240 2024-03-26T17:51:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32ee80e740 10 client.20388 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32ee80e740 10 client.20388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32ee80e740 20 client.20388 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32ee80e740 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55725c93b260 0x55725c93d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32ee80e740 4 client.20388 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:51:39.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.659+0000 7f32ee80e740 1 -- 172.21.15.105:0/1318144394 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55725c823200 con 0x55725c93b260 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.660+0000 7f32e91c2640 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55725c93b260 0x55725c93d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.661+0000 7f32e91c2640 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55725c93b260 0x55725c93d680 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc09b94374a079cb server_cookie=eb34e4028dfaf39f in_seq=0 out_seq=0 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.661+0000 7f32cd7fa640 10 client.20388 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.663+0000 7f32cd7fa640 1 -- 172.21.15.105:0/1318144394 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55725c823200 con 0x55725c93b260 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.663+0000 7f32cd7fa640 10 client.20388 handle_command_reply: tid=0 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 client.20388 shutdown 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 -- 172.21.15.105:0/1318144394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 msgr2=0x55725c8d0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8d0c80 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f32e00ee210 tx=0x7f32e00f0b70 comp rx=0 tx=0).stop 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 -- 172.21.15.105:0/1318144394 shutdown_connections 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55725c93b260 0x55725c93d680 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55725c8c4da0 0x55725c8cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32cd7fa640 0 client.20388 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:39.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55725c8bfc30 0x55725c8cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 --2- 172.21.15.105:0/1318144394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55725c8bf240 0x55725c8d0c80 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 -- 172.21.15.105:0/1318144394 >> 172.21.15.105:0/1318144394 conn(0x55725c8ba7b0 msgr2=0x55725c8bda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 -- 172.21.15.105:0/1318144394 shutdown_connections 2024-03-26T17:51:39.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 1 -- 172.21.15.105:0/1318144394 wait complete. 2024-03-26T17:51:39.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.664+0000 7f32ee80e740 20 client.20388 trim_cache size 0 max 16384 2024-03-26T17:51:39.658 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 -- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32d00a2f20 msgr2=0x7f32d00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32d00a2f20 0x7f32d00a53e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55725c8c0490 tx=0x7f32dc004ab0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 -- 172.21.15.105:0/3189660002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e410b230 msgr2=0x7f32e41a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e410b230 0x7f32e41a2760 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f32e0008000 tx=0x7f32e00042f0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 -- 172.21.15.105:0/3189660002 shutdown_connections 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32d00a2f20 0x7f32d00a53e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e41a7d00 0x7f32e41aa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e4112000 0x7f32e41a2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 --2- 172.21.15.105:0/3189660002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e410b230 0x7f32e41a2760 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:39.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 -- 172.21.15.105:0/3189660002 >> 172.21.15.105:0/3189660002 conn(0x7f32e406e720 msgr2=0x7f32e4109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:39.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 -- 172.21.15.105:0/3189660002 shutdown_connections 2024-03-26T17:51:39.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:39.665+0000 7f32ebc27640 1 -- 172.21.15.105:0/3189660002 wait complete. 2024-03-26T17:51:40.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:39 smithi105 ceph-mon[32280]: pgmap v748: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-26T17:51:40.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:39 smithi135 ceph-mon[35636]: pgmap v748: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-26T17:51:40.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:39 smithi144 ceph-mon[36095]: pgmap v748: 97 pgs: 97 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-26T17:51:40.548 INFO:tasks.workunit.client.0.smithi105.stdout: 10 3380 15.34 MB/sec warmup 23 sec latency 126.267 ms 2024-03-26T17:51:40.671 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:51:40.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:51:40.671 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 -- 172.21.15.105:0/2249482167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c1018d0 msgr2=0x7fb98c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 --2- 172.21.15.105:0/2249482167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c1018d0 0x7fb98c101cf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb980009900 tx=0x7fb98002bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 -- 172.21.15.105:0/2249482167 shutdown_connections 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 --2- 172.21.15.105:0/2249482167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c103480 0x7fb98c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 --2- 172.21.15.105:0/2249482167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb98c102ac0 0x7fb98c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 --2- 172.21.15.105:0/2249482167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c1018d0 0x7fb98c101cf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 -- 172.21.15.105:0/2249482167 >> 172.21.15.105:0/2249482167 conn(0x7fb98c0fce60 msgr2=0x7fb98c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 -- 172.21.15.105:0/2249482167 shutdown_connections 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.827+0000 7fb992d87640 1 -- 172.21.15.105:0/2249482167 wait complete. 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb992d87640 1 Processor -- start 2024-03-26T17:51:40.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb992d87640 1 -- start start 2024-03-26T17:51:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb992d87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb98c1018d0 0x7fb98c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb992d87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c102ac0 0x7fb98c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb992d87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c103480 0x7fb98c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb992d87640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10d3d0 con 0x7fb98c1018d0 2024-03-26T17:51:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb992d87640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10d250 con 0x7fb98c102ac0 2024-03-26T17:51:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb992d87640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10d550 con 0x7fb98c103480 2024-03-26T17:51:40.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.828+0000 7fb990b23640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb98c1018d0 0x7fb98c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb990b23640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb98c1018d0 0x7fb98c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46606/0 (socket says 172.21.15.105:46606) 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb990b23640 1 -- 172.21.15.105:0/25691040 learned_addr learned my addr 172.21.15.105:0/25691040 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb990b23640 1 -- 172.21.15.105:0/25691040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c103480 msgr2=0x7fb98c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb991324640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c103480 0x7fb98c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb98bfff640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c102ac0 0x7fb98c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb990b23640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c103480 0x7fb98c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb990b23640 1 -- 172.21.15.105:0/25691040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c102ac0 msgr2=0x7fb98c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb990b23640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c102ac0 0x7fb98c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb990b23640 1 -- 172.21.15.105:0/25691040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c19f770 con 0x7fb98c1018d0 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb98bfff640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c102ac0 0x7fb98c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.829+0000 7fb990b23640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb98c1018d0 0x7fb98c197b50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb980000c00 tx=0x7fb980005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.830+0000 7fb989ffb640 1 -- 172.21.15.105:0/25691040 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980043020 con 0x7fb98c1018d0 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.830+0000 7fb989ffb640 1 -- 172.21.15.105:0/25691040 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb980004350 con 0x7fb98c1018d0 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.830+0000 7fb989ffb640 1 -- 172.21.15.105:0/25691040 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980003b80 con 0x7fb98c1018d0 2024-03-26T17:51:40.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.830+0000 7fb992d87640 1 -- 172.21.15.105:0/25691040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c19fa00 con 0x7fb98c1018d0 2024-03-26T17:51:40.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.830+0000 7fb992d87640 1 -- 172.21.15.105:0/25691040 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c073d30 con 0x7fb98c1018d0 2024-03-26T17:51:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.833+0000 7fb989ffb640 1 -- 172.21.15.105:0/25691040 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb980046020 con 0x7fb98c1018d0 2024-03-26T17:51:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.833+0000 7fb995970740 1 Processor -- start 2024-03-26T17:51:40.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb995970740 1 -- start start 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb989ffb640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb97c0a2fa0 0x7fb97c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb995970740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5921d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb995970740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921db80 0x55ad59222680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb995970740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59225180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb995970740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ad59182630 con 0x55ad5921db80 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb995970740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ad5912ae10 con 0x55ad5921d190 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb995970740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ad5917e5e0 con 0x55ad59222cf0 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb989ffb640 1 -- 172.21.15.105:0/25691040 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb9800f1630 con 0x7fb98c1018d0 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921db80 0x55ad59222680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb991324640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb97c0a2fa0 0x7fb97c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921db80 0x55ad59222680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46608/0 (socket says 172.21.15.105:46608) 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb98bfff640 1 -- 172.21.15.105:0/1777484509 learned_addr learned my addr 172.21.15.105:0/1777484509 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb990b23640 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5921d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb991324640 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59225180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb98bfff640 1 -- 172.21.15.105:0/1777484509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 msgr2=0x55ad59225180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb98bfff640 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59225180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb98bfff640 1 -- 172.21.15.105:0/1777484509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 msgr2=0x55ad5921d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb98bfff640 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5921d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb98bfff640 1 -- 172.21.15.105:0/1777484509 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad59141740 con 0x55ad5921db80 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.834+0000 7fb991324640 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59225180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.835+0000 7fb991324640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb97c0a2fa0 0x7fb97c0a5460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb984005fd0 tx=0x7fb984009a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:40.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.835+0000 7fb98bfff640 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921db80 0x55ad59222680 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb978009c00 tx=0x7fb978010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa531ee1c4d70d95 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.835+0000 7fb976ffd640 1 -- 172.21.15.105:0/1777484509 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978005020 con 0x55ad5921db80 2024-03-26T17:51:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.835+0000 7fb976ffd640 1 -- 172.21.15.105:0/1777484509 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9780047a0 con 0x55ad5921db80 2024-03-26T17:51:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.835+0000 7fb976ffd640 1 -- 172.21.15.105:0/1777484509 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780126b0 con 0x55ad5921db80 2024-03-26T17:51:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.836+0000 7fb995970740 1 -- 172.21.15.105:0/1777484509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921db80 msgr2=0x55ad59222680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.836+0000 7fb995970740 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921db80 0x55ad59222680 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb978009c00 tx=0x7fb978010740 comp rx=0 tx=0).stop 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.836+0000 7fb995970740 1 -- 172.21.15.105:0/1777484509 shutdown_connections 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.836+0000 7fb995970740 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59225180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.836+0000 7fb995970740 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921db80 0x55ad59222680 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.836+0000 7fb995970740 1 --2- 172.21.15.105:0/1777484509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5921d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.836+0000 7fb995970740 1 -- 172.21.15.105:0/1777484509 >> 172.21.15.105:0/1777484509 conn(0x55ad59218700 msgr2=0x55ad59218b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.836+0000 7fb995970740 1 -- 172.21.15.105:0/1777484509 shutdown_connections 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 -- 172.21.15.105:0/1777484509 wait complete. 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 -- start start 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 Processor -- start 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad592276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad59227c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad59222cf0 0x55ad5922ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ad59225ca0 con 0x55ad5921d190 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ad5913e1a0 con 0x55ad59222cf0 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.837+0000 7fb995970740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ad59225e20 con 0x55ad5921db80 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad592276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb995970740 10 client.0 _ll_register_callbacks cb 0x55ad58e95c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad59227c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad592276d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46624/0 (socket says 172.21.15.105:46624) 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 -- 172.21.15.105:0/3309596426 learned_addr learned my addr 172.21.15.105:0/3309596426 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb991324640 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad59222cf0 0x55ad5922ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 -- 172.21.15.105:0/3309596426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 msgr2=0x55ad59227c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad59227c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 -- 172.21.15.105:0/3309596426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad59222cf0 msgr2=0x55ad5922ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad59222cf0 0x55ad5922ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 -- 172.21.15.105:0/3309596426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad58f5e7e0 con 0x55ad5921d190 2024-03-26T17:51:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb990b23640 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad592276d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb9800bda20 tx=0x7fb9800bd740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0f9fee7b5c9b0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb96bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb96bfff640 1 -- 172.21.15.105:0/3309596426 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800c1e70 con 0x55ad5921d190 2024-03-26T17:51:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.838+0000 7fb96bfff640 1 -- 172.21.15.105:0/3309596426 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb980003d20 con 0x55ad5921d190 2024-03-26T17:51:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb995970740 1 -- 172.21.15.105:0/3309596426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9780047a0 con 0x55ad5921d190 2024-03-26T17:51:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb96bfff640 1 -- 172.21.15.105:0/3309596426 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9800c6730 con 0x55ad5921d190 2024-03-26T17:51:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb995970740 10 client.20400 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb995970740 1 -- 172.21.15.105:0/3309596426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad58fbef70 con 0x55ad5921d190 2024-03-26T17:51:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb96bfff640 1 -- 172.21.15.105:0/3309596426 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9780047a0 con 0x55ad5921d190 2024-03-26T17:51:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb96bfff640 1 -- 172.21.15.105:0/3309596426 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb9800b9a70 con 0x55ad5921d190 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb995970740 10 client.20400 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb995970740 10 client.20400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.839+0000 7fb995970740 20 client.20400 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.840+0000 7fb995970740 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad5923be00 0x55ad5923e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.840+0000 7fb995970740 4 client.20400 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.840+0000 7fb995970740 1 -- 172.21.15.105:0/3309596426 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad5918fb70 con 0x55ad5923be00 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.840+0000 7fb98bfff640 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad5923be00 0x55ad5923e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.840+0000 7fb96bfff640 10 client.20400 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.841+0000 7fb98bfff640 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad5923be00 0x55ad5923e2c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ebe00ee1d8d28f server_cookie=9c7bd9061bdd7b70 in_seq=0 out_seq=0 2024-03-26T17:51:40.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.843+0000 7fb96bfff640 1 -- 172.21.15.105:0/3309596426 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ad5918fb70 con 0x55ad5923be00 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.843+0000 7fb96bfff640 10 client.20400 handle_command_reply: tid=0 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.843+0000 7fb995970740 1 client.20400 shutdown 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 -- 172.21.15.105:0/3309596426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 msgr2=0x55ad592276d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad592276d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fb9800bda20 tx=0x7fb9800bd740 comp rx=0 tx=0).stop 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 -- 172.21.15.105:0/3309596426 shutdown_connections 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad5923be00 0x55ad5923e2c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad59222cf0 0x55ad5922ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb96bfff640 0 client.20400 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad59227c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 --2- 172.21.15.105:0/3309596426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad592276d0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 -- 172.21.15.105:0/3309596426 >> 172.21.15.105:0/3309596426 conn(0x55ad59218700 msgr2=0x55ad59217900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 -- 172.21.15.105:0/3309596426 shutdown_connections 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 1 -- 172.21.15.105:0/3309596426 wait complete. 2024-03-26T17:51:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.844+0000 7fb995970740 20 client.20400 trim_cache size 0 max 16384 2024-03-26T17:51:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb995970740 1 Processor -- start 2024-03-26T17:51:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb995970740 1 -- start start 2024-03-26T17:51:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb995970740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad5921bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb995970740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921db80 0x55ad5922ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb995970740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59243bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb995970740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1a6c70 con 0x55ad5921d190 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb995970740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1a8870 con 0x55ad5921db80 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb995970740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1a7340 con 0x55ad59222cf0 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb990b23640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad5921bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb990b23640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad5921bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46626/0 (socket says 172.21.15.105:46626) 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb990b23640 1 -- 172.21.15.105:0/1580540623 learned_addr learned my addr 172.21.15.105:0/1580540623 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb98bfff640 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921db80 0x55ad5922ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.871+0000 7fb991324640 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59243bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb990b23640 1 -- 172.21.15.105:0/1580540623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 msgr2=0x55ad59243bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb990b23640 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59243bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb990b23640 1 -- 172.21.15.105:0/1580540623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921db80 msgr2=0x55ad5922ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb990b23640 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921db80 0x55ad5922ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb990b23640 1 -- 172.21.15.105:0/1580540623 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad58e56170 con 0x55ad5921d190 2024-03-26T17:51:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb990b23640 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad5921bfd0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb98002c840 tx=0x7fb9800bdea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b23e96d990cb6a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb9767fc640 1 -- 172.21.15.105:0/1580540623 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800bdaf0 con 0x55ad5921d190 2024-03-26T17:51:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb9767fc640 1 -- 172.21.15.105:0/1580540623 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9800c1e70 con 0x55ad5921d190 2024-03-26T17:51:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.872+0000 7fb9767fc640 1 -- 172.21.15.105:0/1580540623 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800c64e0 con 0x55ad5921d190 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 -- 172.21.15.105:0/1580540623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 msgr2=0x55ad5921bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad5921bfd0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb98002c840 tx=0x7fb9800bdea0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 -- 172.21.15.105:0/1580540623 shutdown_connections 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad59222cf0 0x55ad59243bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921db80 0x55ad5922ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 --2- 172.21.15.105:0/1580540623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad5921d190 0x55ad5921bfd0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 -- 172.21.15.105:0/1580540623 >> 172.21.15.105:0/1580540623 conn(0x55ad59218700 msgr2=0x55ad59217070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 -- 172.21.15.105:0/1580540623 shutdown_connections 2024-03-26T17:51:40.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.873+0000 7fb995970740 1 -- 172.21.15.105:0/1580540623 wait complete. 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 1 -- start start 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 1 Processor -- start 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5922e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad5922e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad59222cf0 0x55ad592a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1a81a0 con 0x55ad59222cf0 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1a7a10 con 0x55ad5921d190 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1a4c50 con 0x55ad5921db80 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.874+0000 7fb995970740 10 client.0 _ll_register_callbacks cb 0x7fb9800b9fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5922e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad5922e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5922e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34390/0 (socket says 172.21.15.105:34390) 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 -- 172.21.15.105:0/1282129908 learned_addr learned my addr 172.21.15.105:0/1282129908 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb991324640 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad59222cf0 0x55ad592a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 -- 172.21.15.105:0/1282129908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 msgr2=0x55ad5922e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad5922e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 -- 172.21.15.105:0/1282129908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad59222cf0 msgr2=0x55ad592a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad59222cf0 0x55ad592a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 -- 172.21.15.105:0/1282129908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad592a2be0 con 0x55ad5921d190 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb991324640 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad59222cf0 0x55ad592a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb98bfff640 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad5922e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb990b23640 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5922e020 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb9800c6200 tx=0x7fb9800ef170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bac5a4d1208d57a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb96bfff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.875+0000 7fb96bfff640 1 -- 172.21.15.105:0/1282129908 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800bdaf0 con 0x55ad5921d190 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb96bfff640 1 -- 172.21.15.105:0/1282129908 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb98009cd90 con 0x55ad5921d190 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 1 -- 172.21.15.105:0/1282129908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9800c1e70 con 0x55ad5921d190 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb96bfff640 1 -- 172.21.15.105:0/1282129908 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9800bd170 con 0x55ad5921d190 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 10 client.30203 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 1 -- 172.21.15.105:0/1282129908 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad592a2be0 con 0x55ad5921d190 2024-03-26T17:51:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb96bfff640 1 -- 172.21.15.105:0/1282129908 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9800bd310 con 0x55ad5921d190 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb96bfff640 1 -- 172.21.15.105:0/1282129908 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb9800dca90 con 0x55ad5921d190 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 10 client.30203 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 10 client.30203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 20 client.30203 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad592a4450 0x55ad592a6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 4 client.30203 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.876+0000 7fb995970740 1 -- 172.21.15.105:0/1282129908 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb9800c1e70 con 0x55ad592a4450 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.877+0000 7fb98bfff640 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad592a4450 0x55ad592a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.877+0000 7fb98bfff640 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad592a4450 0x55ad592a6870 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a8e408c5828e7c8 server_cookie=373120a117e68d73 in_seq=0 out_seq=0 2024-03-26T17:51:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.877+0000 7fb96bfff640 10 client.30203 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:40.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb96bfff640 1 -- 172.21.15.105:0/1282129908 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb9800c1e70 con 0x55ad592a4450 2024-03-26T17:51:40.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb96bfff640 10 client.30203 handle_command_reply: tid=0 2024-03-26T17:51:40.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 client.30203 shutdown 2024-03-26T17:51:40.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 -- 172.21.15.105:0/1282129908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 msgr2=0x55ad5922e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5922e020 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb9800c6200 tx=0x7fb9800ef170 comp rx=0 tx=0).stop 2024-03-26T17:51:40.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 -- 172.21.15.105:0/1282129908 shutdown_connections 2024-03-26T17:51:40.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad592a4450 0x55ad592a6870 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad59222cf0 0x55ad592a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad5921db80 0x55ad5922e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 --2- 172.21.15.105:0/1282129908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad5921d190 0x55ad5922e020 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb995970740 1 -- 172.21.15.105:0/1282129908 >> 172.21.15.105:0/1282129908 conn(0x55ad59218700 msgr2=0x55ad59217900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.881+0000 7fb96bfff640 0 client.30203 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.882+0000 7fb995970740 1 -- 172.21.15.105:0/1282129908 shutdown_connections 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.882+0000 7fb995970740 1 -- 172.21.15.105:0/1282129908 wait complete. 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.882+0000 7fb995970740 20 client.30203 trim_cache size 0 max 16384 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "0b543f0e-6606-43ab-b57a-2eac67c41c8a", 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:51:40.875 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:51:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 -- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb97c0a2fa0 msgr2=0x7fb97c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb97c0a2fa0 0x7fb97c0a5460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb984005fd0 tx=0x7fb984009a40 comp rx=0 tx=0).stop 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 -- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb98c1018d0 msgr2=0x7fb98c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb98c1018d0 0x7fb98c197b50 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb980000c00 tx=0x7fb980005dc0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 -- 172.21.15.105:0/25691040 shutdown_connections 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb97c0a2fa0 0x7fb97c0a5460 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb98c103480 0x7fb98c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb98c102ac0 0x7fb98c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 --2- 172.21.15.105:0/25691040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb98c1018d0 0x7fb98c197b50 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 -- 172.21.15.105:0/25691040 >> 172.21.15.105:0/25691040 conn(0x7fb98c0fce60 msgr2=0x7fb98c105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 -- 172.21.15.105:0/25691040 shutdown_connections 2024-03-26T17:51:40.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:40.884+0000 7fb992d87640 1 -- 172.21.15.105:0/25691040 wait complete. 2024-03-26T17:51:40.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:51:40.890 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:51:41.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 -- 172.21.15.105:0/639364007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c241007c0 msgr2=0x7f8c24100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 --2- 172.21.15.105:0/639364007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c241007c0 0x7f8c24100be0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009900 tx=0x7f8c1402bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:41.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 -- 172.21.15.105:0/639364007 shutdown_connections 2024-03-26T17:51:41.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 --2- 172.21.15.105:0/639364007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c24102370 0x7f8c24108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 --2- 172.21.15.105:0/639364007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c241019b0 0x7f8c24101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 --2- 172.21.15.105:0/639364007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c241007c0 0x7f8c24100be0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 -- 172.21.15.105:0/639364007 >> 172.21.15.105:0/639364007 conn(0x7f8c240fbd50 msgr2=0x7f8c240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:41.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 -- 172.21.15.105:0/639364007 shutdown_connections 2024-03-26T17:51:41.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.032+0000 7f8c29db9640 1 -- 172.21.15.105:0/639364007 wait complete. 2024-03-26T17:51:41.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.033+0000 7f8c29db9640 1 Processor -- start 2024-03-26T17:51:41.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.033+0000 7f8c29db9640 1 -- start start 2024-03-26T17:51:41.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.033+0000 7f8c29db9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c241007c0 0x7f8c24195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.033+0000 7f8c29db9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c241019b0 0x7f8c24195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c29db9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c24102370 0x7f8c2419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c29db9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410b1f0 con 0x7f8c241019b0 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c29db9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410b070 con 0x7f8c24102370 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c29db9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410b370 con 0x7f8c241007c0 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c241007c0 0x7f8c24195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c241007c0 0x7f8c24195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46922/0 (socket says 172.21.15.105:46922) 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 -- 172.21.15.105:0/3669393679 learned_addr learned my addr 172.21.15.105:0/3669393679 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c241019b0 0x7f8c24195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c23fff640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c24102370 0x7f8c2419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 -- 172.21.15.105:0/3669393679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c24102370 msgr2=0x7f8c2419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c24102370 0x7f8c2419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 -- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c241019b0 msgr2=0x7f8c24195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c241019b0 0x7f8c24195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 -- 172.21.15.105:0/3669393679 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2419d560 con 0x7f8c241007c0 2024-03-26T17:51:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.034+0000 7f8c237fe640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c241007c0 0x7f8c24195940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8c14000c00 tx=0x7f8c14005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.035+0000 7f8c20ff9640 1 -- 172.21.15.105:0/3669393679 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14043020 con 0x7f8c241007c0 2024-03-26T17:51:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.035+0000 7f8c20ff9640 1 -- 172.21.15.105:0/3669393679 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c14004350 con 0x7f8c241007c0 2024-03-26T17:51:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.035+0000 7f8c29db9640 1 -- 172.21.15.105:0/3669393679 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2419d7f0 con 0x7f8c241007c0 2024-03-26T17:51:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.035+0000 7f8c20ff9640 1 -- 172.21.15.105:0/3669393679 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14003b80 con 0x7f8c241007c0 2024-03-26T17:51:41.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.035+0000 7f8c29db9640 1 -- 172.21.15.105:0/3669393679 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2419dd40 con 0x7f8c241007c0 2024-03-26T17:51:41.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.038+0000 7f8c20ff9640 1 -- 172.21.15.105:0/3669393679 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8c14046020 con 0x7f8c241007c0 2024-03-26T17:51:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.039+0000 7f8c2c9a3740 1 Processor -- start 2024-03-26T17:51:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.039+0000 7f8c2c9a3740 1 -- start start 2024-03-26T17:51:41.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.039+0000 7f8c20ff9640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c0c0a2f20 0x7f8c0c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.039+0000 7f8c23fff640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c0c0a2f20 0x7f8c0c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.039+0000 7f8c20ff9640 1 -- 172.21.15.105:0/3669393679 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c140f1550 con 0x7f8c241007c0 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19af8f0 0x5596f19afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19b02e0 0x5596f19b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5596f1914980 con 0x5596f19b02e0 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596f18bcbe0 con 0x5596f19b5450 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596f1910930 con 0x5596f19af8f0 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c23fff640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c0c0a2f20 0x7f8c0c0a53e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c005fd0 tx=0x7f8c1c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19b02e0 0x5596f19b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19b02e0 0x5596f19b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46646/0 (socket says 172.21.15.105:46646) 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c22ffd640 1 -- 172.21.15.105:0/3718043176 learned_addr learned my addr 172.21.15.105:0/3718043176 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c23fff640 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c237fe640 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19af8f0 0x5596f19afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c22ffd640 1 -- 172.21.15.105:0/3718043176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19af8f0 msgr2=0x5596f19afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:41.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19af8f0 0x5596f19afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c22ffd640 1 -- 172.21.15.105:0/3718043176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 msgr2=0x5596f19b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:41.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c22ffd640 1 -- 172.21.15.105:0/3718043176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f16f27d0 con 0x5596f19b02e0 2024-03-26T17:51:41.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.040+0000 7f8c237fe640 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19af8f0 0x5596f19afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:51:41.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.041+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19b02e0 0x5596f19b4de0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8c18010c50 tx=0x7f8c18005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d432a32d43ce1a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.041+0000 7f8c09ffb640 1 -- 172.21.15.105:0/3718043176 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1801f030 con 0x5596f19b02e0 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.041+0000 7f8c09ffb640 1 -- 172.21.15.105:0/3718043176 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c18010e00 con 0x5596f19b02e0 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.041+0000 7f8c09ffb640 1 -- 172.21.15.105:0/3718043176 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1801b680 con 0x5596f19b02e0 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/3718043176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19b02e0 msgr2=0x5596f19b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19b02e0 0x5596f19b4de0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8c18010c50 tx=0x7f8c18005f90 comp rx=0 tx=0).stop 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/3718043176 shutdown_connections 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19b02e0 0x5596f19b4de0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/3718043176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19af8f0 0x5596f19afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/3718043176 >> 172.21.15.105:0/3718043176 conn(0x5596f19aae60 msgr2=0x5596f19ab260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/3718043176 shutdown_connections 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.042+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/3718043176 wait complete. 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 1 -- start start 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 1 Processor -- start 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19bf370 0x5596f19c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5596f19b8400 con 0x5596f19bf370 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596f18d0160 con 0x5596f19af8f0 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5596f19b8580 con 0x5596f19b5450 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c22ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46934/0 (socket says 172.21.15.105:46934) 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c22ffd640 1 -- 172.21.15.105:0/1818573933 learned_addr learned my addr 172.21.15.105:0/1818573933 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c23fff640 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19bf370 0x5596f19c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.043+0000 7f8c2c9a3740 10 client.0 _ll_register_callbacks cb 0x5596f190c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c22ffd640 1 -- 172.21.15.105:0/1818573933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19af8f0 msgr2=0x5596f19b9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c22ffd640 1 -- 172.21.15.105:0/1818573933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19bf370 msgr2=0x5596f19c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19bf370 0x5596f19c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c22ffd640 1 -- 172.21.15.105:0/1818573933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f16283f0 con 0x5596f19b5450 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c237fe640 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8c18010c50 tx=0x7f8c1801bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1366add9a6a4e2ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c02ffd640 1 -- 172.21.15.105:0/1818573933 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18004230 con 0x5596f19b5450 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c02ffd640 1 -- 172.21.15.105:0/1818573933 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c180043d0 con 0x5596f19b5450 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/1818573933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c18010e00 con 0x5596f19b5450 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.044+0000 7f8c02ffd640 1 -- 172.21.15.105:0/1818573933 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c180035f0 con 0x5596f19b5450 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c2c9a3740 10 client.30199 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:41.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/1818573933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5596f15e80f0 con 0x5596f19b5450 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c02ffd640 1 -- 172.21.15.105:0/1818573933 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c18010e00 con 0x5596f19b5450 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c02ffd640 1 -- 172.21.15.105:0/1818573933 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8c18004d40 con 0x5596f19b5450 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c2c9a3740 10 client.30199 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c2c9a3740 10 client.30199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c2c9a3740 20 client.30199 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596f19c5d30 0x5596f19c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c2c9a3740 4 client.30199 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/1818573933 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f1921ec0 con 0x5596f19c5d30 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.045+0000 7f8c237fe640 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596f19c5d30 0x5596f19c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.046+0000 7f8c02ffd640 10 client.30199 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:41.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.046+0000 7f8c237fe640 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596f19c5d30 0x5596f19c81a0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe2f8509cb1966 server_cookie=8138c8fd69459ea in_seq=0 out_seq=0 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c02ffd640 1 -- 172.21.15.105:0/1818573933 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5596f1921ec0 con 0x5596f19c5d30 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c02ffd640 10 client.30199 handle_command_reply: tid=0 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 client.30199 shutdown 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/1818573933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 msgr2=0x5596f19ba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8c18010c50 tx=0x7f8c1801bd90 comp rx=0 tx=0).stop 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/1818573933 shutdown_connections 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596f19c5d30 0x5596f19c81a0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19bf370 0x5596f19c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/1818573933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/1818573933 >> 172.21.15.105:0/1818573933 conn(0x5596f19aae60 msgr2=0x5596f19aa080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c02ffd640 0 client.30199 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/1818573933 shutdown_connections 2024-03-26T17:51:41.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/1818573933 wait complete. 2024-03-26T17:51:41.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.048+0000 7f8c2c9a3740 20 client.30199 trim_cache size 0 max 16384 2024-03-26T17:51:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c2c9a3740 1 Processor -- start 2024-03-26T17:51:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c2c9a3740 1 -- start start 2024-03-26T17:51:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19bf370 0x5596f19b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241a6c40 con 0x5596f19af8f0 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241a8840 con 0x5596f19bf370 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241a7310 con 0x5596f19b5450 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46678/0 (socket says 172.21.15.105:46678) 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c237fe640 1 -- 172.21.15.105:0/2246959043 learned_addr learned my addr 172.21.15.105:0/2246959043 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.072+0000 7f8c23fff640 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19bf370 0x5596f19b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c237fe640 1 -- 172.21.15.105:0/2246959043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 msgr2=0x5596f19ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c237fe640 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c237fe640 1 -- 172.21.15.105:0/2246959043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19bf370 msgr2=0x5596f19b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c237fe640 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19bf370 0x5596f19b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c237fe640 1 -- 172.21.15.105:0/2246959043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f19a6530 con 0x5596f19af8f0 2024-03-26T17:51:41.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c237fe640 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f8c140ba0a0 tx=0x7f8c140eeee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6e0f68e7b3913e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c097fa640 1 -- 172.21.15.105:0/2246959043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140db070 con 0x5596f19af8f0 2024-03-26T17:51:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c097fa640 1 -- 172.21.15.105:0/2246959043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c140bdc40 con 0x5596f19af8f0 2024-03-26T17:51:41.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.073+0000 7f8c097fa640 1 -- 172.21.15.105:0/2246959043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140054d0 con 0x5596f19af8f0 2024-03-26T17:51:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.074+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/2246959043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 msgr2=0x5596f19b9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.074+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f8c140ba0a0 tx=0x7f8c140eeee0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.074+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/2246959043 shutdown_connections 2024-03-26T17:51:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.074+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19bf370 0x5596f19b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.074+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19b5450 0x5596f19ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.074+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/2246959043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19b9e00 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.074+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/2246959043 >> 172.21.15.105:0/2246959043 conn(0x5596f19aae60 msgr2=0x5596f19a5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.075+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/2246959043 shutdown_connections 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.075+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/2246959043 wait complete. 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.075+0000 7f8c2c9a3740 1 -- start start 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.075+0000 7f8c2c9a3740 1 Processor -- start 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c2c9a3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19bf370 0x5596f19d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241a8170 con 0x5596f19af8f0 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241a79e0 con 0x5596f19b5450 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c2c9a3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c241a4c20 con 0x5596f19bf370 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46692/0 (socket says 172.21.15.105:46692) 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c237fe640 1 -- 172.21.15.105:0/715971009 learned_addr learned my addr 172.21.15.105:0/715971009 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:41.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c23fff640 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19bf370 0x5596f19d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c2c9a3740 10 client.0 _ll_register_callbacks cb 0x7f8c1c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c22ffd640 1 -- 172.21.15.105:0/715971009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19bf370 msgr2=0x5596f19d7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19bf370 0x5596f19d7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c22ffd640 1 -- 172.21.15.105:0/715971009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 msgr2=0x5596f19d67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19d67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c22ffd640 1 -- 172.21.15.105:0/715971009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f19d8220 con 0x5596f19b5450 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c237fe640 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19d67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.076+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/715971009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c140bdc40 con 0x5596f19b5450 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c22ffd640 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19d6d00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8c180057c0 tx=0x7f8c180108f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17848aca9f5617cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c02ffd640 10 client.30218 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c02ffd640 1 -- 172.21.15.105:0/715971009 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1803c020 con 0x5596f19b5450 2024-03-26T17:51:41.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c02ffd640 1 -- 172.21.15.105:0/715971009 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c18004ad0 con 0x5596f19b5450 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c02ffd640 1 -- 172.21.15.105:0/715971009 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c1801f040 con 0x5596f19b5450 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c2c9a3740 10 client.30218 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/715971009 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5596f19d9450 con 0x5596f19b5450 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c02ffd640 1 -- 172.21.15.105:0/715971009 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8c18004d90 con 0x5596f19b5450 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c2c9a3740 10 client.30218 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c2c9a3740 10 client.30218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c2c9a3740 20 client.30218 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596f1a2bfc0 0x5596f1a2e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c2c9a3740 4 client.30218 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.077+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/715971009 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596f191c210 con 0x5596f1a2bfc0 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.078+0000 7f8c237fe640 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596f1a2bfc0 0x5596f1a2e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.078+0000 7f8c02ffd640 10 client.30218 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:41.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.078+0000 7f8c237fe640 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596f1a2bfc0 0x5596f1a2e3e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f476a3a4e00c8997 server_cookie=3c0d2655166b961d in_seq=0 out_seq=0 2024-03-26T17:51:41.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c02ffd640 1 -- 172.21.15.105:0/715971009 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5596f191c210 con 0x5596f1a2bfc0 2024-03-26T17:51:41.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c02ffd640 10 client.30218 handle_command_reply: tid=0 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 client.30218 shutdown 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 msgr2=0x5596f19d6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19d6d00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8c180057c0 tx=0x7f8c180108f0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/715971009 shutdown_connections 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5596f1a2bfc0 0x5596f1a2e3e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5596f19bf370 0x5596f19d7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c02ffd640 0 client.30218 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596f19b5450 0x5596f19d6d00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 --2- 172.21.15.105:0/715971009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5596f19af8f0 0x5596f19d67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/715971009 >> 172.21.15.105:0/715971009 conn(0x5596f19aae60 msgr2=0x5596f19b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/715971009 shutdown_connections 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 1 -- 172.21.15.105:0/715971009 wait complete. 2024-03-26T17:51:41.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.081+0000 7f8c2c9a3740 20 client.30218 trim_cache size 0 max 16384 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (285 inodes in the stack)", 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout: "0b543f0e-6606-43ab-b57a-2eac67c41c8a": { 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "0b543f0e-6606-43ab-b57a-2eac67c41c8a", 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:51:41.077 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:51:41.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 -- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c0c0a2f20 msgr2=0x7f8c0c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c0c0a2f20 0x7f8c0c0a53e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c005fd0 tx=0x7f8c1c009500 comp rx=0 tx=0).stop 2024-03-26T17:51:41.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 -- 172.21.15.105:0/3669393679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c241007c0 msgr2=0x7f8c24195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:41.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c241007c0 0x7f8c24195940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8c14000c00 tx=0x7f8c14005dc0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 -- 172.21.15.105:0/3669393679 shutdown_connections 2024-03-26T17:51:41.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c0c0a2f20 0x7f8c0c0a53e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c24102370 0x7f8c2419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c241019b0 0x7f8c24195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 --2- 172.21.15.105:0/3669393679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c241007c0 0x7f8c24195940 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:41.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 -- 172.21.15.105:0/3669393679 >> 172.21.15.105:0/3669393679 conn(0x7f8c240fbd50 msgr2=0x7f8c24104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:41.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 -- 172.21.15.105:0/3669393679 shutdown_connections 2024-03-26T17:51:41.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:51:41.082+0000 7f8c29db9640 1 -- 172.21.15.105:0/3669393679 wait complete. 2024-03-26T17:51:41.083 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:40 smithi105 ceph-mon[32280]: pgmap v749: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 3.0 MiB/s wr, 244 op/s 2024-03-26T17:51:41.085 INFO:tasks.cephfs.filesystem:scrub status for tag:0b543f0e-6606-43ab-b57a-2eac67c41c8a - {'path': '/', 'tag': '0b543f0e-6606-43ab-b57a-2eac67c41c8a', 'options': 'recursive,force'} 2024-03-26T17:51:41.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:40 smithi135 ceph-mon[35636]: pgmap v749: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 3.0 MiB/s wr, 244 op/s 2024-03-26T17:51:41.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:40 smithi144 ceph-mon[36095]: pgmap v749: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 3.0 MiB/s wr, 244 op/s 2024-03-26T17:51:41.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 3544 15.17 MB/sec warmup 24 sec latency 216.619 ms 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:51:42.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:51:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:51:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:51:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:51:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:51:42.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:51:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:51:42.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:51:42.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 3742 14.92 MB/sec warmup 25 sec latency 148.000 ms 2024-03-26T17:51:43.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:42 smithi135 ceph-mon[35636]: pgmap v750: 97 pgs: 97 active+clean; 225 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-26T17:51:43.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:42 smithi144 ceph-mon[36095]: pgmap v750: 97 pgs: 97 active+clean; 225 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-26T17:51:43.390 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:42 smithi105 ceph-mon[32280]: pgmap v750: 97 pgs: 97 active+clean; 225 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-26T17:51:43.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 3871 14.88 MB/sec warmup 26 sec latency 116.443 ms 2024-03-26T17:51:44.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 3941 14.35 MB/sec warmup 27 sec latency 118.364 ms 2024-03-26T17:51:45.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4006 13.87 MB/sec warmup 28 sec latency 131.417 ms 2024-03-26T17:51:46.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:45 smithi144 ceph-mon[36095]: pgmap v751: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-26T17:51:46.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:45 smithi105 ceph-mon[32280]: pgmap v751: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-26T17:51:46.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:45 smithi135 ceph-mon[35636]: pgmap v751: 97 pgs: 97 active+clean; 232 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-26T17:51:46.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4087 13.49 MB/sec warmup 29 sec latency 113.054 ms 2024-03-26T17:51:47.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4211 13.40 MB/sec warmup 30 sec latency 121.570 ms 2024-03-26T17:51:48.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:47 smithi144 ceph-mon[36095]: pgmap v752: 97 pgs: 97 active+clean; 235 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.8 MiB/s wr, 285 op/s 2024-03-26T17:51:48.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:47 smithi105 ceph-mon[32280]: pgmap v752: 97 pgs: 97 active+clean; 235 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.8 MiB/s wr, 285 op/s 2024-03-26T17:51:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:47 smithi135 ceph-mon[35636]: pgmap v752: 97 pgs: 97 active+clean; 235 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 3.8 MiB/s wr, 285 op/s 2024-03-26T17:51:48.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4410 13.67 MB/sec warmup 31 sec latency 122.374 ms 2024-03-26T17:51:49.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4585 13.48 MB/sec warmup 32 sec latency 119.224 ms 2024-03-26T17:51:50.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:49 smithi144 ceph-mon[36095]: pgmap v753: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-03-26T17:51:50.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:49 smithi105 ceph-mon[32280]: pgmap v753: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-03-26T17:51:50.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:49 smithi135 ceph-mon[35636]: pgmap v753: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-03-26T17:51:50.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4693 13.30 MB/sec warmup 33 sec latency 130.250 ms 2024-03-26T17:51:51.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4796 13.06 MB/sec warmup 34 sec latency 129.789 ms 2024-03-26T17:51:52.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:51 smithi144 ceph-mon[36095]: pgmap v754: 97 pgs: 97 active+clean; 235 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-03-26T17:51:52.098 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:51 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:51:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:51 smithi105 ceph-mon[32280]: pgmap v754: 97 pgs: 97 active+clean; 235 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-03-26T17:51:52.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:51 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:51:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:51 smithi135 ceph-mon[35636]: pgmap v754: 97 pgs: 97 active+clean; 235 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.7 MiB/s wr, 247 op/s 2024-03-26T17:51:52.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:51 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:51:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.179+0000 7fea5ca12640 1 -- 172.21.15.105:0/2033307210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea581023a0 msgr2=0x7fea58104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.179+0000 7fea5ca12640 1 --2- 172.21.15.105:0/2033307210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea581023a0 0x7fea58104790 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fea48009900 tx=0x7fea4802bc30 comp rx=0 tx=0).stop 2024-03-26T17:51:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.179+0000 7fea5ca12640 1 -- 172.21.15.105:0/2033307210 shutdown_connections 2024-03-26T17:51:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.179+0000 7fea5ca12640 1 --2- 172.21.15.105:0/2033307210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea58107600 0x7fea58109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.179+0000 7fea5ca12640 1 --2- 172.21.15.105:0/2033307210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea58104cd0 0x7fea581070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.179+0000 7fea5ca12640 1 --2- 172.21.15.105:0/2033307210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea581023a0 0x7fea58104790 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.179+0000 7fea5ca12640 1 -- 172.21.15.105:0/2033307210 >> 172.21.15.105:0/2033307210 conn(0x7fea580fbd90 msgr2=0x7fea580fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.180+0000 7fea5ca12640 1 -- 172.21.15.105:0/2033307210 shutdown_connections 2024-03-26T17:51:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.180+0000 7fea5ca12640 1 -- 172.21.15.105:0/2033307210 wait complete. 2024-03-26T17:51:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.180+0000 7fea5ca12640 1 Processor -- start 2024-03-26T17:51:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.180+0000 7fea5ca12640 1 -- start start 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea5ca12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea581023a0 0x7fea58101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea5ca12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea58104cd0 0x7fea58100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea5ca12640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea58107600 0x7fea58100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea5ca12640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810d480 con 0x7fea58104cd0 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea5ca12640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810d300 con 0x7fea58107600 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea5ca12640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810d600 con 0x7fea581023a0 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea55d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea58104cd0 0x7fea58100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea55d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea58104cd0 0x7fea58100350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45936/0 (socket says 172.21.15.105:45936) 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea55d9b640 1 -- 172.21.15.105:0/47939229 learned_addr learned my addr 172.21.15.105:0/47939229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea56d9d640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea58107600 0x7fea58100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea5659c640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea581023a0 0x7fea58101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea56d9d640 1 -- 172.21.15.105:0/47939229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea581023a0 msgr2=0x7fea58101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea56d9d640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea581023a0 0x7fea58101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea56d9d640 1 -- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea58104cd0 msgr2=0x7fea58100350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea56d9d640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea58104cd0 0x7fea58100350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea56d9d640 1 -- 172.21.15.105:0/47939229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea58101120 con 0x7fea58107600 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.181+0000 7fea55d9b640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea58104cd0 0x7fea58100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.182+0000 7fea56d9d640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea58107600 0x7fea58100890 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fea50009fd0 tx=0x7fea5000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.182+0000 7fea477fe640 1 -- 172.21.15.105:0/47939229 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5000ce20 con 0x7fea58107600 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.182+0000 7fea5ca12640 1 -- 172.21.15.105:0/47939229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea581a1c10 con 0x7fea58107600 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.182+0000 7fea477fe640 1 -- 172.21.15.105:0/47939229 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea50005020 con 0x7fea58107600 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.182+0000 7fea477fe640 1 -- 172.21.15.105:0/47939229 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50010690 con 0x7fea58107600 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.182+0000 7fea5ca12640 1 -- 172.21.15.105:0/47939229 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581a2110 con 0x7fea58107600 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.185+0000 7fea477fe640 1 -- 172.21.15.105:0/47939229 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fea500051d0 con 0x7fea58107600 2024-03-26T17:51:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.185+0000 7fea5f603740 1 Processor -- start 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.185+0000 7fea5f603740 1 -- start start 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5f603740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c2955d0 0x55a87c2959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5f603740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c295fc0 0x55a87c29aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5f603740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c29b130 0x55a87c29d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5f603740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a87c1fa0b0 con 0x55a87c29b130 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5f603740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a87c1a2960 con 0x55a87c2955d0 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5f603740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a87c1f6060 con 0x55a87c295fc0 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c295fc0 0x55a87c29aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c295fc0 0x55a87c29aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43048/0 (socket says 172.21.15.105:43048) 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 -- 172.21.15.105:0/2405038963 learned_addr learned my addr 172.21.15.105:0/2405038963 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea55d9b640 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c29b130 0x55a87c29d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea477fe640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea400a2fc0 0x7fea400a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea55d9b640 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c2955d0 0x55a87c2959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 -- 172.21.15.105:0/2405038963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c2955d0 msgr2=0x55a87c2959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c2955d0 0x55a87c2959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 -- 172.21.15.105:0/2405038963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c29b130 msgr2=0x55a87c29d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c29b130 0x55a87c29d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 -- 172.21.15.105:0/2405038963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87c1b4c60 con 0x55a87c295fc0 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea56d9d640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea400a2fc0 0x7fea400a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea477fe640 1 -- 172.21.15.105:0/47939229 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fea500cfb50 con 0x7fea58107600 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.186+0000 7fea5659c640 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c295fc0 0x55a87c29aac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fea48009900 tx=0x7fea48009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45845b7f5315bd9d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.187+0000 7fea44ff9640 1 -- 172.21.15.105:0/2405038963 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48045020 con 0x55a87c295fc0 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.187+0000 7fea44ff9640 1 -- 172.21.15.105:0/2405038963 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea48004040 con 0x55a87c295fc0 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.187+0000 7fea44ff9640 1 -- 172.21.15.105:0/2405038963 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48005590 con 0x55a87c295fc0 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 -- 172.21.15.105:0/2405038963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c295fc0 msgr2=0x55a87c29aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c295fc0 0x55a87c29aac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fea48009900 tx=0x7fea48009560 comp rx=0 tx=0).stop 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea56d9d640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea400a2fc0 0x7fea400a5480 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fea5009c1d0 tx=0x7fea50051040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 -- 172.21.15.105:0/2405038963 shutdown_connections 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c29b130 0x55a87c29d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c295fc0 0x55a87c29aac0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 --2- 172.21.15.105:0/2405038963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c2955d0 0x55a87c2959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 -- 172.21.15.105:0/2405038963 >> 172.21.15.105:0/2405038963 conn(0x55a87c290b40 msgr2=0x55a87c290f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 -- 172.21.15.105:0/2405038963 shutdown_connections 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 -- 172.21.15.105:0/2405038963 wait complete. 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 -- start start 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.188+0000 7fea5f603740 1 Processor -- start 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5f603740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c2955d0 0x55a87c29f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5f603740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c295fc0 0x55a87c29f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5f603740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5f603740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a87c29e080 con 0x55a87c2955d0 2024-03-26T17:51:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5f603740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a87c1b5db0 con 0x55a87c295fc0 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5f603740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a87c29e200 con 0x55a87c29b130 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5f603740 10 client.0 _ll_register_callbacks cb 0x55a87c1ba110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c2955d0 0x55a87c29f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c2955d0 0x55a87c29f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45954/0 (socket says 172.21.15.105:45954) 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 -- 172.21.15.105:0/2041718125 learned_addr learned my addr 172.21.15.105:0/2041718125 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5659c640 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c295fc0 0x55a87c29f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 -- 172.21.15.105:0/2041718125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 msgr2=0x55a87c2a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 -- 172.21.15.105:0/2041718125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c295fc0 msgr2=0x55a87c29f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c295fc0 0x55a87c29f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 -- 172.21.15.105:0/2041718125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87bf0e210 con 0x55a87c2955d0 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea55d9b640 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c2955d0 0x55a87c29f380 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fea4c013410 tx=0x7fea4c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0329bdc346b7ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea3dffb640 1 -- 172.21.15.105:0/2041718125 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c013c60 con 0x55a87c2955d0 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea5f603740 1 -- 172.21.15.105:0/2041718125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea48004040 con 0x55a87c2955d0 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.189+0000 7fea3dffb640 1 -- 172.21.15.105:0/2041718125 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fea4c013e00 con 0x55a87c2955d0 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea3dffb640 1 -- 172.21.15.105:0/2041718125 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea4c0047a0 con 0x55a87c2955d0 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5f603740 10 client.20436 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5f603740 1 -- 172.21.15.105:0/2041718125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a87bfda620 con 0x55a87c2955d0 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea3dffb640 1 -- 172.21.15.105:0/2041718125 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea48004040 con 0x55a87c2955d0 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea3dffb640 1 -- 172.21.15.105:0/2041718125 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fea4c022910 con 0x55a87c2955d0 2024-03-26T17:51:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5f603740 10 client.20436 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5f603740 10 client.20436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5f603740 20 client.20436 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5f603740 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a87c2ab270 0x55a87c2ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5f603740 4 client.20436 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:51:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5f603740 1 -- 172.21.15.105:0/2041718125 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a87c2075f0 con 0x55a87c2ab270 2024-03-26T17:51:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.190+0000 7fea5659c640 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a87c2ab270 0x55a87c2ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.191+0000 7fea3dffb640 10 client.20436 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.191+0000 7fea5659c640 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a87c2ab270 0x55a87c2ad6f0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559203cff429e87f server_cookie=15a21ceab5a8f4c6 in_seq=0 out_seq=0 2024-03-26T17:51:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.192+0000 7fea3dffb640 1 -- 172.21.15.105:0/2041718125 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a87c2075f0 con 0x55a87c2ab270 2024-03-26T17:51:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.192+0000 7fea3dffb640 10 client.20436 handle_command_reply: tid=0 2024-03-26T17:51:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.192+0000 7fea5f603740 1 client.20436 shutdown 2024-03-26T17:51:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.192+0000 7fea5f603740 1 -- 172.21.15.105:0/2041718125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c2955d0 msgr2=0x55a87c29f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.192+0000 7fea5f603740 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c2955d0 0x55a87c29f380 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fea4c013410 tx=0x7fea4c005f90 comp rx=0 tx=0).stop 2024-03-26T17:51:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.192+0000 7fea5f603740 1 -- 172.21.15.105:0/2041718125 shutdown_connections 2024-03-26T17:51:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.192+0000 7fea5f603740 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a87c2ab270 0x55a87c2ad6f0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.192+0000 7fea5f603740 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.193+0000 7fea5f603740 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c295fc0 0x55a87c29f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.193+0000 7fea5f603740 1 --2- 172.21.15.105:0/2041718125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c2955d0 0x55a87c29f380 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.193+0000 7fea5f603740 1 -- 172.21.15.105:0/2041718125 >> 172.21.15.105:0/2041718125 conn(0x55a87c290b40 msgr2=0x55a87c28fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.193+0000 7fea3dffb640 0 client.20436 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.193+0000 7fea5f603740 1 -- 172.21.15.105:0/2041718125 shutdown_connections 2024-03-26T17:51:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.193+0000 7fea5f603740 1 -- 172.21.15.105:0/2041718125 wait complete. 2024-03-26T17:51:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.193+0000 7fea5f603740 20 client.20436 trim_cache size 0 max 16384 2024-03-26T17:51:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5f603740 1 Processor -- start 2024-03-26T17:51:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5f603740 1 -- start start 2024-03-26T17:51:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5f603740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2972f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5f603740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29b130 0x55a87c2a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5f603740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29f6a0 0x55a87c28f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5f603740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea581aaed0 con 0x55a87c295fc0 2024-03-26T17:51:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5f603740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea581acad0 con 0x55a87c29b130 2024-03-26T17:51:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5f603740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea581ab5a0 con 0x55a87c29f6a0 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45968/0 (socket says 172.21.15.105:45968) 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 -- 172.21.15.105:0/365425229 learned_addr learned my addr 172.21.15.105:0/365425229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29b130 0x55a87c2a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29f6a0 0x55a87c28f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 -- 172.21.15.105:0/365425229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29f6a0 msgr2=0x55a87c28f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29f6a0 0x55a87c28f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 -- 172.21.15.105:0/365425229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29b130 msgr2=0x55a87c2a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29b130 0x55a87c2a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea55d9b640 1 -- 172.21.15.105:0/365425229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a87c2915b0 con 0x55a87c295fc0 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.212+0000 7fea5659c640 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29b130 0x55a87c2a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.213+0000 7fea55d9b640 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2972f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0059f0 tx=0x7fea4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa585b27945be75 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.213+0000 7fea3ffff640 1 -- 172.21.15.105:0/365425229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c032070 con 0x55a87c295fc0 2024-03-26T17:51:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.213+0000 7fea3ffff640 1 -- 172.21.15.105:0/365425229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea4c028030 con 0x55a87c295fc0 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.213+0000 7fea3ffff640 1 -- 172.21.15.105:0/365425229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c015320 con 0x55a87c295fc0 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 -- 172.21.15.105:0/365425229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 msgr2=0x55a87c2972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2972f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0059f0 tx=0x7fea4c009bd0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 -- 172.21.15.105:0/365425229 shutdown_connections 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29f6a0 0x55a87c28f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29b130 0x55a87c2a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 --2- 172.21.15.105:0/365425229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2972f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 -- 172.21.15.105:0/365425229 >> 172.21.15.105:0/365425229 conn(0x55a87c290b40 msgr2=0x55a87c29ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 -- 172.21.15.105:0/365425229 shutdown_connections 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 -- 172.21.15.105:0/365425229 wait complete. 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 -- start start 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 Processor -- start 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.214+0000 7fea5f603740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5f603740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5f603740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29f6a0 0x55a87c2a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5f603740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea581ac400 con 0x55a87c295fc0 2024-03-26T17:51:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5f603740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea581abc70 con 0x55a87c29f6a0 2024-03-26T17:51:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5f603740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea581a8eb0 con 0x55a87c29b130 2024-03-26T17:51:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5f603740 10 client.0 _ll_register_callbacks cb 0x7fea4c0135d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:51:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43082/0 (socket says 172.21.15.105:43082) 2024-03-26T17:51:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 -- 172.21.15.105:0/827915423 learned_addr learned my addr 172.21.15.105:0/827915423 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:51:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea55d9b640 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 -- 172.21.15.105:0/827915423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29f6a0 msgr2=0x55a87c2a5570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:51:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29f6a0 0x55a87c2a5570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 -- 172.21.15.105:0/827915423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 msgr2=0x55a87c2a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:51:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 -- 172.21.15.105:0/827915423 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87c2a5b60 con 0x55a87c29b130 2024-03-26T17:51:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea55d9b640 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5659c640 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a4560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea48009850 tx=0x7fea4802c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d3f53cf9c5d3f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea3dffb640 1 -- 172.21.15.105:0/827915423 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48050070 con 0x55a87c29b130 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.215+0000 7fea5f603740 1 -- 172.21.15.105:0/827915423 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4c028030 con 0x55a87c29b130 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea3dffb640 1 -- 172.21.15.105:0/827915423 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fea48054450 con 0x55a87c29b130 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea3dffb640 1 -- 172.21.15.105:0/827915423 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea48045040 con 0x55a87c29b130 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea5f603740 10 client.30223 fetch_fsmap learned FSMap version 34 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea5f603740 1 -- 172.21.15.105:0/827915423 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a87c311620 con 0x55a87c29b130 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea3dffb640 1 -- 172.21.15.105:0/827915423 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea4c028030 con 0x55a87c29b130 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea3dffb640 1 -- 172.21.15.105:0/827915423 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fea480050d0 con 0x55a87c29b130 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea5f603740 10 client.30223 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea5f603740 10 client.30223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.216+0000 7fea5f603740 20 client.30223 populate_metadata read hostname 'smithi105' 2024-03-26T17:51:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.217+0000 7fea5f603740 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a87c311ca0 0x55a87c314090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:51:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.217+0000 7fea5f603740 4 client.30223 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:51:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.217+0000 7fea5f603740 1 -- 172.21.15.105:0/827915423 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a87c201940 con 0x55a87c311ca0 2024-03-26T17:51:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.217+0000 7fea55d9b640 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a87c311ca0 0x55a87c314090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:51:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.217+0000 7fea3dffb640 10 client.30223 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.217+0000 7fea55d9b640 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a87c311ca0 0x55a87c314090 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6c173a3325796b server_cookie=c37111825aa7ee8a in_seq=0 out_seq=0 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea3dffb640 1 -- 172.21.15.105:0/827915423 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a87c201940 con 0x55a87c311ca0 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea3dffb640 10 client.30223 handle_command_reply: tid=0 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 client.30223 shutdown 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 -- 172.21.15.105:0/827915423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 msgr2=0x55a87c2a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a4560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea48009850 tx=0x7fea4802c1f0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 -- 172.21.15.105:0/827915423 shutdown_connections 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a87c311ca0 0x55a87c314090 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a87c29f6a0 0x55a87c2a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea3dffb640 0 client.30223 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a87c29b130 0x55a87c2a4560 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 --2- 172.21.15.105:0/827915423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a87c295fc0 0x55a87c2a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 -- 172.21.15.105:0/827915423 >> 172.21.15.105:0/827915423 conn(0x55a87c290b40 msgr2=0x55a87c293da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 -- 172.21.15.105:0/827915423 shutdown_connections 2024-03-26T17:51:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 1 -- 172.21.15.105:0/827915423 wait complete. 2024-03-26T17:51:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.375+0000 7fea5f603740 20 client.30223 trim_cache size 0 max 16384 2024-03-26T17:51:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:51:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:51:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:51:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 -- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea400a2fc0 msgr2=0x7fea400a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea400a2fc0 0x7fea400a5480 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fea5009c1d0 tx=0x7fea50051040 comp rx=0 tx=0).stop 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 -- 172.21.15.105:0/47939229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea58107600 msgr2=0x7fea58100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea58107600 0x7fea58100890 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fea50009fd0 tx=0x7fea5000ef10 comp rx=0 tx=0).stop 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 -- 172.21.15.105:0/47939229 shutdown_connections 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea400a2fc0 0x7fea400a5480 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea58107600 0x7fea58100890 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea58104cd0 0x7fea58100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 --2- 172.21.15.105:0/47939229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea581023a0 0x7fea58101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 -- 172.21.15.105:0/47939229 >> 172.21.15.105:0/47939229 conn(0x7fea580fbd90 msgr2=0x7fea58102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:51:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 -- 172.21.15.105:0/47939229 shutdown_connections 2024-03-26T17:51:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:51:52.376+0000 7fea5ca12640 1 -- 172.21.15.105:0/47939229 wait complete. 2024-03-26T17:51:52.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4907 12.91 MB/sec warmup 35 sec latency 222.268 ms 2024-03-26T17:51:53.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:52 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:51:53.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:52 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:51:53.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:52 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:51:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:52 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:51:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:52 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:51:53.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:52 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:51:53.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:52 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:51:53.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:52 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:51:53.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:52 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:51:53.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 4974 12.58 MB/sec warmup 36 sec latency 110.628 ms 2024-03-26T17:51:54.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:53 smithi144 ceph-mon[36095]: pgmap v755: 97 pgs: 97 active+clean; 240 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-03-26T17:51:54.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:53 smithi105 ceph-mon[32280]: pgmap v755: 97 pgs: 97 active+clean; 240 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-03-26T17:51:54.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:53 smithi135 ceph-mon[35636]: pgmap v755: 97 pgs: 97 active+clean; 240 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 4.0 MiB/s wr, 257 op/s 2024-03-26T17:51:54.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 5069 12.34 MB/sec warmup 37 sec latency 177.012 ms 2024-03-26T17:51:55.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 5164 12.06 MB/sec warmup 38 sec latency 191.714 ms 2024-03-26T17:51:56.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:55 smithi105 ceph-mon[32280]: pgmap v756: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T17:51:56.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:55 smithi135 ceph-mon[35636]: pgmap v756: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T17:51:56.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:55 smithi144 ceph-mon[36095]: pgmap v756: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T17:51:56.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 5271 11.78 MB/sec warmup 39 sec latency 181.512 ms 2024-03-26T17:51:57.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 5393 11.56 MB/sec warmup 40 sec latency 172.000 ms 2024-03-26T17:51:58.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:57 smithi105 ceph-mon[32280]: pgmap v757: 97 pgs: 97 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 3.4 MiB/s wr, 245 op/s 2024-03-26T17:51:58.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:57 smithi135 ceph-mon[35636]: pgmap v757: 97 pgs: 97 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 3.4 MiB/s wr, 245 op/s 2024-03-26T17:51:58.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:57 smithi144 ceph-mon[36095]: pgmap v757: 97 pgs: 97 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 3.4 MiB/s wr, 245 op/s 2024-03-26T17:51:58.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 5525 11.36 MB/sec warmup 41 sec latency 189.467 ms 2024-03-26T17:51:59.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 5653 11.26 MB/sec warmup 42 sec latency 131.790 ms 2024-03-26T17:52:00.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:51:59 smithi105 ceph-mon[32280]: pgmap v758: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-03-26T17:52:00.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:51:59 smithi135 ceph-mon[35636]: pgmap v758: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-03-26T17:52:00.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:51:59 smithi144 ceph-mon[36095]: pgmap v758: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-03-26T17:52:00.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 5793 11.19 MB/sec warmup 43 sec latency 124.395 ms 2024-03-26T17:52:01.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 5948 11.06 MB/sec warmup 44 sec latency 124.007 ms 2024-03-26T17:52:02.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:01 smithi105 ceph-mon[32280]: pgmap v759: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-03-26T17:52:02.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:01 smithi135 ceph-mon[35636]: pgmap v759: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-03-26T17:52:02.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:01 smithi144 ceph-mon[36095]: pgmap v759: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 3.2 MiB/s wr, 265 op/s 2024-03-26T17:52:02.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 6080 10.98 MB/sec warmup 45 sec latency 146.780 ms 2024-03-26T17:52:03.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 6249 11.23 MB/sec warmup 46 sec latency 148.527 ms 2024-03-26T17:52:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:03 smithi105 ceph-mon[32280]: pgmap v760: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-03-26T17:52:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:52:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:52:04.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:03 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:52:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:03 smithi135 ceph-mon[35636]: pgmap v760: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-03-26T17:52:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:52:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:52:04.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:03 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:52:04.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:03 smithi144 ceph-mon[36095]: pgmap v760: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-03-26T17:52:04.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:52:04.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:52:04.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:52:04.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:03 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:52:04.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 6389 11.05 MB/sec warmup 47 sec latency 115.259 ms 2024-03-26T17:52:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.499+0000 7f59e2a27640 1 -- 172.21.15.105:0/4126043241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1018f0 msgr2=0x7f59dc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.499+0000 7f59e2a27640 1 --2- 172.21.15.105:0/4126043241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1018f0 0x7f59dc101d10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009860 tx=0x7f59cc02bb70 comp rx=0 tx=0).stop 2024-03-26T17:52:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 -- 172.21.15.105:0/4126043241 shutdown_connections 2024-03-26T17:52:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 --2- 172.21.15.105:0/4126043241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59dc1034a0 0x7f59dc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 --2- 172.21.15.105:0/4126043241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc102ae0 0x7f59dc102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 --2- 172.21.15.105:0/4126043241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1018f0 0x7f59dc101d10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.491 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 -- 172.21.15.105:0/4126043241 >> 172.21.15.105:0/4126043241 conn(0x7f59dc0fce80 msgr2=0x7f59dc0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 -- 172.21.15.105:0/4126043241 shutdown_connections 2024-03-26T17:52:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 -- 172.21.15.105:0/4126043241 wait complete. 2024-03-26T17:52:05.492 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 Processor -- start 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.500+0000 7f59e2a27640 1 -- start start 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59e2a27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc1018f0 0x7f59dc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59e2a27640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59dc102ae0 0x7f59dc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc1018f0 0x7f59dc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc1018f0 0x7f59dc197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49568/0 (socket says 172.21.15.105:49568) 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59dbfff640 1 -- 172.21.15.105:0/2295668874 learned_addr learned my addr 172.21.15.105:0/2295668874 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59e2a27640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1034a0 0x7f59dc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10d420 con 0x7f59dc1018f0 2024-03-26T17:52:05.494 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10d2a0 con 0x7f59dc1034a0 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc10d5a0 con 0x7f59dc102ae0 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59db7fe640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59dc102ae0 0x7f59dc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59e0fc4640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1034a0 0x7f59dc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59dbfff640 1 -- 172.21.15.105:0/2295668874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59dc102ae0 msgr2=0x7f59dc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.501+0000 7f59dbfff640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59dc102ae0 0x7f59dc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59dbfff640 1 -- 172.21.15.105:0/2295668874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1034a0 msgr2=0x7f59dc19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59dbfff640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1034a0 0x7f59dc19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59dbfff640 1 -- 172.21.15.105:0/2295668874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc19f6b0 con 0x7f59dc1018f0 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59e0fc4640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1034a0 0x7f59dc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59db7fe640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59dc102ae0 0x7f59dc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59dbfff640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc1018f0 0x7f59dc197a90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f59cc005940 tx=0x7f59cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59d97fa640 1 -- 172.21.15.105:0/2295668874 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc041020 con 0x7f59dc1018f0 2024-03-26T17:52:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59d97fa640 1 -- 172.21.15.105:0/2295668874 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59cc0035f0 con 0x7f59dc1018f0 2024-03-26T17:52:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc19f940 con 0x7f59dc1018f0 2024-03-26T17:52:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc19fe30 con 0x7f59dc1018f0 2024-03-26T17:52:05.496 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.502+0000 7f59d97fa640 1 -- 172.21.15.105:0/2295668874 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc039040 con 0x7f59dc1018f0 2024-03-26T17:52:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.505+0000 7f59d97fa640 1 -- 172.21.15.105:0/2295668874 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f59cc044020 con 0x7f59dc1018f0 2024-03-26T17:52:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59d97fa640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59c40a3130 0x7f59c40a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e5615740 1 Processor -- start 2024-03-26T17:52:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e5615740 1 -- start start 2024-03-26T17:52:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59db7fe640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59c40a3130 0x7f59c40a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59d97fa640 1 -- 172.21.15.105:0/2295668874 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f59cc0f0c90 con 0x7f59dc1018f0 2024-03-26T17:52:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e5615740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5029970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e5615740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be502ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e5615740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be502d0a0 0x556be5039f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e5615740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556be4f8e0b0 con 0x556be5029550 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e5615740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556be4f36960 con 0x556be502d0a0 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e5615740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556be4f8a060 con 0x556be5029f40 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e0fc4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be502d0a0 0x556be5039f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e0fc4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be502d0a0 0x556be5039f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48464/0 (socket says 172.21.15.105:48464) 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59e0fc4640 1 -- 172.21.15.105:0/938977748 learned_addr learned my addr 172.21.15.105:0/938977748 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59dbfff640 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5029970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59db7fe640 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be502ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59db7fe640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59c40a3130 0x7f59c40a55f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f59d00096c0 tx=0x7f59d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59dbfff640 1 -- 172.21.15.105:0/938977748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 msgr2=0x556be502ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59dbfff640 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be502ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59dbfff640 1 -- 172.21.15.105:0/938977748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be502d0a0 msgr2=0x556be5039f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.506+0000 7f59dbfff640 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be502d0a0 0x556be5039f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.507+0000 7f59dbfff640 1 -- 172.21.15.105:0/938977748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556be4f48c60 con 0x556be5029550 2024-03-26T17:52:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.507+0000 7f59db7fe640 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be502ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:52:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.507+0000 7f59dbfff640 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5029970 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0ee750 tx=0x7f59cc0bcf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b80efe44e24cef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.507+0000 7f59c27fc640 1 -- 172.21.15.105:0/938977748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0d5020 con 0x556be5029550 2024-03-26T17:52:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.507+0000 7f59c27fc640 1 -- 172.21.15.105:0/938977748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59cc0bd380 con 0x556be5029550 2024-03-26T17:52:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59c27fc640 1 -- 172.21.15.105:0/938977748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0c1110 con 0x556be5029550 2024-03-26T17:52:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 -- 172.21.15.105:0/938977748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 msgr2=0x556be5029970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5029970 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0ee750 tx=0x7f59cc0bcf80 comp rx=0 tx=0).stop 2024-03-26T17:52:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 -- 172.21.15.105:0/938977748 shutdown_connections 2024-03-26T17:52:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be502d0a0 0x556be5039f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be502ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 --2- 172.21.15.105:0/938977748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5029970 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 -- 172.21.15.105:0/938977748 >> 172.21.15.105:0/938977748 conn(0x556be5024b40 msgr2=0x556be5024f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 -- 172.21.15.105:0/938977748 shutdown_connections 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 -- 172.21.15.105:0/938977748 wait complete. 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.508+0000 7f59e5615740 1 -- start start 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e5615740 1 Processor -- start 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e5615740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e5615740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be503be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e5615740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be502d0a0 0x556be5042e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e5615740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556be503a6d0 con 0x556be502d0a0 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e5615740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556be4f49db0 con 0x556be5029550 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e5615740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556be503a850 con 0x556be5029f40 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59dbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59db7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be503be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e5615740 10 client.0 _ll_register_callbacks cb 0x556be4f4e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59db7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be503be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46558/0 (socket says 172.21.15.105:46558) 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59db7fe640 1 -- 172.21.15.105:0/3172750079 learned_addr learned my addr 172.21.15.105:0/3172750079 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59e0fc4640 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be502d0a0 0x556be5042e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.509+0000 7f59dbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48478/0 (socket says 172.21.15.105:48478) 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59db7fe640 1 -- 172.21.15.105:0/3172750079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 msgr2=0x556be503b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59db7fe640 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59db7fe640 1 -- 172.21.15.105:0/3172750079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be502d0a0 msgr2=0x556be5042e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59db7fe640 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be502d0a0 0x556be5042e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59db7fe640 1 -- 172.21.15.105:0/3172750079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556be4ca2210 con 0x556be5029f40 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59e0fc4640 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be502d0a0 0x556be5042e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59db7fe640 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be503be10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009530 tx=0x7f59d0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b4a71ecdbd28956 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59bb7fe640 1 -- 172.21.15.105:0/3172750079 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d001cbe0 con 0x556be5029f40 2024-03-26T17:52:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59bb7fe640 1 -- 172.21.15.105:0/3172750079 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f59d001cd80 con 0x556be5029f40 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.510+0000 7f59e5615740 1 -- 172.21.15.105:0/3172750079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59cc0bd380 con 0x556be5029f40 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59bb7fe640 1 -- 172.21.15.105:0/3172750079 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f59d0004020 con 0x556be5029f40 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59e5615740 10 client.30232 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59e5615740 1 -- 172.21.15.105:0/3172750079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556be4d6e620 con 0x556be5029f40 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59bb7fe640 1 -- 172.21.15.105:0/3172750079 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59cc0bd380 con 0x556be5029f40 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59bb7fe640 1 -- 172.21.15.105:0/3172750079 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f59d0024070 con 0x556be5029f40 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59e5615740 10 client.30232 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59e5615740 10 client.30232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59e5615740 20 client.30232 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59e5615740 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556be5047810 0x556be5049c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59e5615740 4 client.30232 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.511+0000 7f59e5615740 1 -- 172.21.15.105:0/3172750079 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556be4f9b5f0 con 0x556be5047810 2024-03-26T17:52:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.512+0000 7f59dbfff640 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556be5047810 0x556be5049c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.512+0000 7f59dbfff640 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556be5047810 0x556be5049c70 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d904829e0a2ba1e server_cookie=1dccd026ebb8807e in_seq=0 out_seq=0 2024-03-26T17:52:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.512+0000 7f59bb7fe640 10 client.30232 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.513+0000 7f59bb7fe640 1 -- 172.21.15.105:0/3172750079 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556be4f9b5f0 con 0x556be5047810 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.513+0000 7f59bb7fe640 10 client.30232 handle_command_reply: tid=0 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.513+0000 7f59e5615740 1 client.30232 shutdown 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.513+0000 7f59e5615740 1 -- 172.21.15.105:0/3172750079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 msgr2=0x556be503be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.513+0000 7f59e5615740 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be503be10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009530 tx=0x7f59d0004730 comp rx=0 tx=0).stop 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 1 -- 172.21.15.105:0/3172750079 shutdown_connections 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556be5047810 0x556be5049c70 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be502d0a0 0x556be5042e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be5029f40 0x556be503be10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59bb7fe640 0 client.30232 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 1 --2- 172.21.15.105:0/3172750079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 1 -- 172.21.15.105:0/3172750079 >> 172.21.15.105:0/3172750079 conn(0x556be5024b40 msgr2=0x556be5023df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 1 -- 172.21.15.105:0/3172750079 shutdown_connections 2024-03-26T17:52:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 1 -- 172.21.15.105:0/3172750079 wait complete. 2024-03-26T17:52:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.514+0000 7f59e5615740 20 client.30232 trim_cache size 0 max 16384 2024-03-26T17:52:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.540+0000 7f59e5615740 1 Processor -- start 2024-03-26T17:52:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.540+0000 7f59e5615740 1 -- start start 2024-03-26T17:52:05.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59e5615740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59e5615740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029f40 0x556be5042e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59e5615740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5035ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59e5615740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800e500 con 0x556be5029f40 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59e5615740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59c8010100 con 0x556be5029550 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59e5615740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800ebd0 con 0x556be502d0a0 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029f40 0x556be5042e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59dbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029f40 0x556be5042e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49604/0 (socket says 172.21.15.105:49604) 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 -- 172.21.15.105:0/854429787 learned_addr learned my addr 172.21.15.105:0/854429787 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59e0fc4640 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5035ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 -- 172.21.15.105:0/854429787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 msgr2=0x556be5035ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5035ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 -- 172.21.15.105:0/854429787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 msgr2=0x556be503b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:05.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 -- 172.21.15.105:0/854429787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556be5028bd0 con 0x556be5029f40 2024-03-26T17:52:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59dbfff640 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:05.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.541+0000 7f59db7fe640 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029f40 0x556be5042e10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f59d0006010 tx=0x7f59d0009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c62fb79db8c588 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.542+0000 7f59c1ffb640 1 -- 172.21.15.105:0/854429787 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d001cbe0 con 0x556be5029f40 2024-03-26T17:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.542+0000 7f59c1ffb640 1 -- 172.21.15.105:0/854429787 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d001cd80 con 0x556be5029f40 2024-03-26T17:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.542+0000 7f59c1ffb640 1 -- 172.21.15.105:0/854429787 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0003d00 con 0x556be5029f40 2024-03-26T17:52:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 -- 172.21.15.105:0/854429787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029f40 msgr2=0x556be5042e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029f40 0x556be5042e10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f59d0006010 tx=0x7f59d0009e40 comp rx=0 tx=0).stop 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 -- 172.21.15.105:0/854429787 shutdown_connections 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5035ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029f40 0x556be5042e10 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 --2- 172.21.15.105:0/854429787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029550 0x556be503b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 -- 172.21.15.105:0/854429787 >> 172.21.15.105:0/854429787 conn(0x556be5024b40 msgr2=0x556be503ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 -- 172.21.15.105:0/854429787 shutdown_connections 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 -- 172.21.15.105:0/854429787 wait complete. 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.543+0000 7f59e5615740 1 -- start start 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e5615740 1 Processor -- start 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e5615740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5040700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e5615740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029f40 0x556be5040c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e5615740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5041c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e5615740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800fa30 con 0x556be5029550 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e5615740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800f2a0 con 0x556be5029f40 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e5615740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59c800c4e0 con 0x556be502d0a0 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e0fc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5041c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59db7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029f40 0x556be5040c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e0fc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5041c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46572/0 (socket says 172.21.15.105:46572) 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59db7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029f40 0x556be5040c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48500/0 (socket says 172.21.15.105:48500) 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e0fc4640 1 -- 172.21.15.105:0/402674811 learned_addr learned my addr 172.21.15.105:0/402674811 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e5615740 10 client.0 _ll_register_callbacks cb 0x7f59d0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59dbfff640 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5040700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e0fc4640 1 -- 172.21.15.105:0/402674811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029f40 msgr2=0x556be5040c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e0fc4640 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029f40 0x556be5040c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e0fc4640 1 -- 172.21.15.105:0/402674811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 msgr2=0x556be5040700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e0fc4640 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5040700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59e0fc4640 1 -- 172.21.15.105:0/402674811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556be5042240 con 0x556be502d0a0 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.544+0000 7f59db7fe640 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029f40 0x556be5040c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:05.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59e0fc4640 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5041c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005c40 tx=0x7f59d4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28f7cb9fa4d427ed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59bb7fe640 1 -- 172.21.15.105:0/402674811 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4021030 con 0x556be502d0a0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59e5615740 1 -- 172.21.15.105:0/402674811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59d001cd80 con 0x556be502d0a0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59bb7fe640 1 -- 172.21.15.105:0/402674811 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f59d4004b20 con 0x556be502d0a0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59bb7fe640 1 -- 172.21.15.105:0/402674811 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f59d4004d20 con 0x556be502d0a0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59e5615740 10 client.30238 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59e5615740 1 -- 172.21.15.105:0/402674811 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556be50adee0 con 0x556be502d0a0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.545+0000 7f59bb7fe640 1 -- 172.21.15.105:0/402674811 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59d001cd80 con 0x556be502d0a0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59bb7fe640 1 -- 172.21.15.105:0/402674811 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f59d40277a0 con 0x556be502d0a0 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59e5615740 10 client.30238 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59e5615740 10 client.30238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59e5615740 20 client.30238 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59e5615740 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556be50aea00 0x556be50b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59e5615740 4 client.30238 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:52:05.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59e5615740 1 -- 172.21.15.105:0/402674811 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556be4f95940 con 0x556be50aea00 2024-03-26T17:52:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59dbfff640 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556be50aea00 0x556be50b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59bb7fe640 10 client.30238 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.546+0000 7f59dbfff640 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556be50aea00 0x556be50b0e20 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556f4bed260797bd server_cookie=d137daa20f097fec in_seq=0 out_seq=0 2024-03-26T17:52:05.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 6517 10.92 MB/sec warmup 48 sec latency 128.363 ms 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59bb7fe640 1 -- 172.21.15.105:0/402674811 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556be4f95940 con 0x556be50aea00 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59bb7fe640 10 client.30238 handle_command_reply: tid=0 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 client.30238 shutdown 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 -- 172.21.15.105:0/402674811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 msgr2=0x556be5041c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5041c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005c40 tx=0x7f59d4010800 comp rx=0 tx=0).stop 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 -- 172.21.15.105:0/402674811 shutdown_connections 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556be50aea00 0x556be50b0e20 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556be502d0a0 0x556be5041c50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556be5029f40 0x556be5040c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 --2- 172.21.15.105:0/402674811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556be5029550 0x556be5040700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59bb7fe640 0 client.30238 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 -- 172.21.15.105:0/402674811 >> 172.21.15.105:0/402674811 conn(0x556be5024b40 msgr2=0x556be5023560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 -- 172.21.15.105:0/402674811 shutdown_connections 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 1 -- 172.21.15.105:0/402674811 wait complete. 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.687+0000 7f59e5615740 20 client.30238 trim_cache size 0 max 16384 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:52:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:52:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59c40a3130 msgr2=0x7f59c40a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59c40a3130 0x7f59c40a55f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f59d00096c0 tx=0x7f59d0009260 comp rx=0 tx=0).stop 2024-03-26T17:52:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc1018f0 msgr2=0x7f59dc197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc1018f0 0x7f59dc197a90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f59cc005940 tx=0x7f59cc0042f0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 shutdown_connections 2024-03-26T17:52:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59c40a3130 0x7f59c40a55f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f59dc1034a0 0x7f59dc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59dc102ae0 0x7f59dc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 --2- 172.21.15.105:0/2295668874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59dc1018f0 0x7f59dc197a90 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 >> 172.21.15.105:0/2295668874 conn(0x7f59dc0fce80 msgr2=0x7f59dc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 shutdown_connections 2024-03-26T17:52:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:05.688+0000 7f59e2a27640 1 -- 172.21.15.105:0/2295668874 wait complete. 2024-03-26T17:52:06.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:05 smithi105 ceph-mon[32280]: pgmap v761: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-03-26T17:52:06.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:05 smithi135 ceph-mon[35636]: pgmap v761: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-03-26T17:52:06.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:05 smithi144 ceph-mon[36095]: pgmap v761: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-03-26T17:52:06.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 6653 10.76 MB/sec warmup 49 sec latency 192.697 ms 2024-03-26T17:52:07.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 6806 10.64 MB/sec warmup 50 sec latency 139.665 ms 2024-03-26T17:52:08.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:07 smithi105 ceph-mon[32280]: pgmap v762: 97 pgs: 97 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-03-26T17:52:08.148 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:07 smithi144 ceph-mon[36095]: pgmap v762: 97 pgs: 97 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-03-26T17:52:08.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:07 smithi135 ceph-mon[35636]: pgmap v762: 97 pgs: 97 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-03-26T17:52:08.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 6968 10.51 MB/sec warmup 51 sec latency 119.958 ms 2024-03-26T17:52:09.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 7099 10.35 MB/sec warmup 52 sec latency 108.963 ms 2024-03-26T17:52:10.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:09 smithi105 ceph-mon[32280]: pgmap v763: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-03-26T17:52:10.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:09 smithi135 ceph-mon[35636]: pgmap v763: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-03-26T17:52:10.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:09 smithi144 ceph-mon[36095]: pgmap v763: 97 pgs: 97 active+clean; 224 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-03-26T17:52:10.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 7245 10.31 MB/sec warmup 53 sec latency 113.779 ms 2024-03-26T17:52:11.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:52:11.086 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:52:11.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 -- 172.21.15.105:0/1060024977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 msgr2=0x7f730c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 --2- 172.21.15.105:0/1060024977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 0x7f730c101e30 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009900 tx=0x7f72f402bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:11.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 -- 172.21.15.105:0/1060024977 shutdown_connections 2024-03-26T17:52:11.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 --2- 172.21.15.105:0/1060024977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c102370 0x7f730c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 --2- 172.21.15.105:0/1060024977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 0x7f730c101e30 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 --2- 172.21.15.105:0/1060024977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c1007c0 0x7f730c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 -- 172.21.15.105:0/1060024977 >> 172.21.15.105:0/1060024977 conn(0x7f730c0fbd50 msgr2=0x7f730c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 -- 172.21.15.105:0/1060024977 shutdown_connections 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.235+0000 7f7312a59640 1 -- 172.21.15.105:0/1060024977 wait complete. 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7312a59640 1 Processor -- start 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7312a59640 1 -- start start 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7312a59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c1007c0 0x7f730c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7312a59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 0x7f730c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7312a59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c102370 0x7f730c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7312a59640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10d3f0 con 0x7f730c1019b0 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7312a59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10d270 con 0x7f730c1007c0 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7312a59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f730c10d570 con 0x7f730c102370 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7303fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 0x7f730c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7303fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 0x7f730c198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49626/0 (socket says 172.21.15.105:49626) 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7303fff640 1 -- 172.21.15.105:0/3227326648 learned_addr learned my addr 172.21.15.105:0/3227326648 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f730bfff640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c1007c0 0x7f730c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7310ff6640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c102370 0x7f730c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7303fff640 1 -- 172.21.15.105:0/3227326648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c102370 msgr2=0x7f730c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7303fff640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c102370 0x7f730c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7303fff640 1 -- 172.21.15.105:0/3227326648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c1007c0 msgr2=0x7f730c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7303fff640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c1007c0 0x7f730c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.236+0000 7f7303fff640 1 -- 172.21.15.105:0/3227326648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c19f750 con 0x7f730c1019b0 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.237+0000 7f7303fff640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 0x7f730c198070 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f72f4006010 tx=0x7f72f4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.237+0000 7f7309ffb640 1 -- 172.21.15.105:0/3227326648 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4043020 con 0x7f730c1019b0 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.237+0000 7f7309ffb640 1 -- 172.21.15.105:0/3227326648 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72f4004440 con 0x7f730c1019b0 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.237+0000 7f7309ffb640 1 -- 172.21.15.105:0/3227326648 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4003bf0 con 0x7f730c1019b0 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.237+0000 7f7312a59640 1 -- 172.21.15.105:0/3227326648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c19f9e0 con 0x7f730c1019b0 2024-03-26T17:52:11.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.237+0000 7f7312a59640 1 -- 172.21.15.105:0/3227326648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c19ff80 con 0x7f730c1019b0 2024-03-26T17:52:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.240+0000 7f7309ffb640 1 -- 172.21.15.105:0/3227326648 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f72f4046020 con 0x7f730c1019b0 2024-03-26T17:52:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.240+0000 7f7315641740 1 Processor -- start 2024-03-26T17:52:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.240+0000 7f7315641740 1 -- start start 2024-03-26T17:52:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7315641740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7315641740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522b52e0 0x555c522b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7315641740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7315641740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555c52219980 con 0x555c522b48f0 2024-03-26T17:52:11.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7315641740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555c521c1be0 con 0x555c522ba450 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7315641740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555c52215930 con 0x555c522b52e0 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7309ffb640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72f80a2fa0 0x7f72f80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7303fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7310ff6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7303fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522bc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48520/0 (socket says 172.21.15.105:48520) 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7303fff640 1 -- 172.21.15.105:0/1649369482 learned_addr learned my addr 172.21.15.105:0/1649369482 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7310ff6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49638/0 (socket says 172.21.15.105:49638) 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f730bfff640 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522b52e0 0x555c522b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7309ffb640 1 -- 172.21.15.105:0/3227326648 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f72f40f16e0 con 0x7f730c1019b0 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7310ff6640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72f80a2fa0 0x7f72f80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7303fff640 1 -- 172.21.15.105:0/1649369482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522b52e0 msgr2=0x555c522b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7303fff640 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522b52e0 0x555c522b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7303fff640 1 -- 172.21.15.105:0/1649369482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 msgr2=0x555c522b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7303fff640 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7303fff640 1 -- 172.21.15.105:0/1649369482 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c51ff77d0 con 0x555c522ba450 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.241+0000 7f7310ff6640 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.242+0000 7f730bfff640 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522b52e0 0x555c522b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.242+0000 7f7310ff6640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72f80a2fa0 0x7f72f80a5460 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x555c522b5b40 tx=0x7f7304009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.242+0000 7f7303fff640 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522bc8e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72f4005ab0 tx=0x7f72f4003670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a0e0669356af8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.242+0000 7f73027fc640 1 -- 172.21.15.105:0/1649369482 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40bd910 con 0x555c522ba450 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.242+0000 7f73027fc640 1 -- 172.21.15.105:0/1649369482 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72f4005560 con 0x555c522ba450 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.242+0000 7f73027fc640 1 -- 172.21.15.105:0/1649369482 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40bd730 con 0x555c522ba450 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 -- 172.21.15.105:0/1649369482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 msgr2=0x555c522bc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522bc8e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72f4005ab0 tx=0x7f72f4003670 comp rx=0 tx=0).stop 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 -- 172.21.15.105:0/1649369482 shutdown_connections 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522bc8e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522b52e0 0x555c522b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 --2- 172.21.15.105:0/1649369482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 -- 172.21.15.105:0/1649369482 >> 172.21.15.105:0/1649369482 conn(0x555c522afe60 msgr2=0x555c522b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 -- 172.21.15.105:0/1649369482 shutdown_connections 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 -- 172.21.15.105:0/1649369482 wait complete. 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 -- start start 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.243+0000 7f7315641740 1 Processor -- start 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7315641740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522b48f0 0x555c522be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7315641740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b52e0 0x555c522bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7315641740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7303fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b52e0 0x555c522bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7303fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b52e0 0x555c522bed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49650/0 (socket says 172.21.15.105:49650) 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7303fff640 1 -- 172.21.15.105:0/2286677442 learned_addr learned my addr 172.21.15.105:0/2286677442 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555c522bd1c0 con 0x555c522b52e0 2024-03-26T17:52:11.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555c521d5160 con 0x555c522b48f0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555c522bd340 con 0x555c522ba450 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f730bfff640 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522b48f0 0x555c522be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7315641740 10 client.0 _ll_register_callbacks cb 0x555c52211720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.244+0000 7f7310ff6640 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f7303fff640 1 -- 172.21.15.105:0/2286677442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 msgr2=0x555c522c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f7303fff640 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f7303fff640 1 -- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522b48f0 msgr2=0x555c522be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f7303fff640 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522b48f0 0x555c522be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f7303fff640 1 -- 172.21.15.105:0/2286677442 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c51f2d3f0 con 0x555c522b52e0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f730bfff640 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522b48f0 0x555c522be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f7303fff640 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b52e0 0x555c522bed70 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f72f402bc70 tx=0x7f72f40bd7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701e2372d4dbd37e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f72eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f72eb7fe640 1 -- 172.21.15.105:0/2286677442 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40cb060 con 0x555c522b52e0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f72eb7fe640 1 -- 172.21.15.105:0/2286677442 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f72f40c6350 con 0x555c522b52e0 2024-03-26T17:52:11.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.245+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c52226ec0 con 0x555c522b52e0 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f72eb7fe640 1 -- 172.21.15.105:0/2286677442 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f72f40a2050 con 0x555c522b52e0 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f7315641740 10 client.20472 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555c51eed0f0 con 0x555c522b52e0 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f72eb7fe640 1 -- 172.21.15.105:0/2286677442 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555c52226ec0 con 0x555c522b52e0 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f72eb7fe640 1 -- 172.21.15.105:0/2286677442 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f72f40dc030 con 0x555c522b52e0 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f7315641740 10 client.20472 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f7315641740 10 client.20472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f7315641740 20 client.20472 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f7315641740 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555c522ca2a0 0x555c522cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f7315641740 4 client.20472 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c52221210 con 0x555c522ca2a0 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.246+0000 7f730bfff640 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555c522ca2a0 0x555c522cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.247+0000 7f72eb7fe640 10 client.20472 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:11.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.247+0000 7f730bfff640 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555c522ca2a0 0x555c522cc6e0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d5e466abb88d2e server_cookie=eab7a7e6d4053f06 in_seq=0 out_seq=0 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.248+0000 7f72eb7fe640 1 -- 172.21.15.105:0/2286677442 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555c52221210 con 0x555c522ca2a0 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.248+0000 7f72eb7fe640 10 client.20472 handle_command_reply: tid=0 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.248+0000 7f7315641740 1 client.20472 shutdown 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b52e0 msgr2=0x555c522bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b52e0 0x555c522bed70 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f72f402bc70 tx=0x7f72f40bd7c0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 shutdown_connections 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555c522ca2a0 0x555c522cc6e0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f72eb7fe640 0 client.20472 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b52e0 0x555c522bed70 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 --2- 172.21.15.105:0/2286677442 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522b48f0 0x555c522be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 >> 172.21.15.105:0/2286677442 conn(0x555c522afe60 msgr2=0x555c522af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 shutdown_connections 2024-03-26T17:52:11.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 1 -- 172.21.15.105:0/2286677442 wait complete. 2024-03-26T17:52:11.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.249+0000 7f7315641740 20 client.20472 trim_cache size 0 max 16384 2024-03-26T17:52:11.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7315641740 1 Processor -- start 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7315641740 1 -- start start 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7315641740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522afe60 0x555c522ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7315641740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7315641740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522aed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7315641740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a6c60 con 0x555c522b48f0 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7315641740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a8860 con 0x555c522afe60 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7315641740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a7330 con 0x555c522ba450 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7303fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7303fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49664/0 (socket says 172.21.15.105:49664) 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7310ff6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f7303fff640 1 -- 172.21.15.105:0/1637394407 learned_addr learned my addr 172.21.15.105:0/1637394407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.275+0000 7f730bfff640 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522afe60 0x555c522ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7303fff640 1 -- 172.21.15.105:0/1637394407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 msgr2=0x555c522aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7303fff640 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7303fff640 1 -- 172.21.15.105:0/1637394407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522afe60 msgr2=0x555c522ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7303fff640 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522afe60 0x555c522ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7303fff640 1 -- 172.21.15.105:0/1637394407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c522b5fb0 con 0x555c522b48f0 2024-03-26T17:52:11.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7310ff6640 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7303fff640 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c5d70 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f72f40ef510 tx=0x7f72f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5eae93f2b47f12 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7301ffb640 1 -- 172.21.15.105:0/1637394407 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40ac070 con 0x555c522b48f0 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7301ffb640 1 -- 172.21.15.105:0/1637394407 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72f40be4d0 con 0x555c522b48f0 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.276+0000 7f7301ffb640 1 -- 172.21.15.105:0/1637394407 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40c65b0 con 0x555c522b48f0 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 -- 172.21.15.105:0/1637394407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 msgr2=0x555c522c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c5d70 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f72f40ef510 tx=0x7f72f4000bc0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 -- 172.21.15.105:0/1637394407 shutdown_connections 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522ba450 0x555c522aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c5d70 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 --2- 172.21.15.105:0/1637394407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522afe60 0x555c522ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 -- 172.21.15.105:0/1637394407 >> 172.21.15.105:0/1637394407 conn(0x555c522b9e80 msgr2=0x555c522b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 -- 172.21.15.105:0/1637394407 shutdown_connections 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.277+0000 7f7315641740 1 -- 172.21.15.105:0/1637394407 wait complete. 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 1 -- start start 2024-03-26T17:52:11.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 1 Processor -- start 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522afe60 0x555c522c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a8190 con 0x555c522b48f0 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a7a00 con 0x555c522ba450 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1a4c40 con 0x555c522afe60 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7315641740 10 client.0 _ll_register_callbacks cb 0x7f7304009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7303fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f730bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522afe60 0x555c522c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7303fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49674/0 (socket says 172.21.15.105:49674) 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.278+0000 7f7303fff640 1 -- 172.21.15.105:0/1944604756 learned_addr learned my addr 172.21.15.105:0/1944604756 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7310ff6640 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7303fff640 1 -- 172.21.15.105:0/1944604756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522afe60 msgr2=0x555c522c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7303fff640 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522afe60 0x555c522c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7303fff640 1 -- 172.21.15.105:0/1944604756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 msgr2=0x555c522c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7303fff640 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7303fff640 1 -- 172.21.15.105:0/1944604756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c522c4f90 con 0x555c522b48f0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f730bfff640 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522afe60 0x555c522c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7303fff640 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c3990 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f72f40ba750 tx=0x7f72f40c67e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52631d5b88adaa0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f72eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f72eb7fe640 1 -- 172.21.15.105:0/1944604756 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f40b4070 con 0x555c522b48f0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f72eb7fe640 1 -- 172.21.15.105:0/1944604756 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f72f40b83b0 con 0x555c522b48f0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7315641740 1 -- 172.21.15.105:0/1944604756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72f40be4d0 con 0x555c522b48f0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f72eb7fe640 1 -- 172.21.15.105:0/1944604756 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f72f40ac050 con 0x555c522b48f0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7315641740 10 client.20478 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.279+0000 7f7315641740 1 -- 172.21.15.105:0/1944604756 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555c522c4f90 con 0x555c522b48f0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f72eb7fe640 1 -- 172.21.15.105:0/1944604756 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72f40be4d0 con 0x555c522b48f0 2024-03-26T17:52:11.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f72eb7fe640 1 -- 172.21.15.105:0/1944604756 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f72f40dc040 con 0x555c522b48f0 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f7315641740 10 client.20478 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f7315641740 10 client.20478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f7315641740 20 client.20478 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f7315641740 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555c52330bc0 0x555c52332fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f7315641740 4 client.20478 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f7315641740 1 -- 172.21.15.105:0/1944604756 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555c5219e540 con 0x555c52330bc0 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f730bfff640 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555c52330bc0 0x555c52332fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.280+0000 7f72eb7fe640 10 client.20478 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:11.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.281+0000 7f730bfff640 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555c52330bc0 0x555c52332fe0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579e91abf8628031 server_cookie=558e7482a0cdd394 in_seq=0 out_seq=0 2024-03-26T17:52:11.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f72eb7fe640 1 -- 172.21.15.105:0/1944604756 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555c5219e540 con 0x555c52330bc0 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f72eb7fe640 10 client.20478 handle_command_reply: tid=0 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 client.20478 shutdown 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 -- 172.21.15.105:0/1944604756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 msgr2=0x555c522c3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c3990 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f72f40ba750 tx=0x7f72f40c67e0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 -- 172.21.15.105:0/1944604756 shutdown_connections 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555c52330bc0 0x555c52332fe0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555c522ba450 0x555c522c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555c522b48f0 0x555c522c3990 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f72eb7fe640 0 client.20478 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 --2- 172.21.15.105:0/1944604756 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555c522afe60 0x555c522c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 -- 172.21.15.105:0/1944604756 >> 172.21.15.105:0/1944604756 conn(0x555c522b9e80 msgr2=0x555c522b96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 -- 172.21.15.105:0/1944604756 shutdown_connections 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 1 -- 172.21.15.105:0/1944604756 wait complete. 2024-03-26T17:52:11.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.282+0000 7f7315641740 20 client.20478 trim_cache size 0 max 16384 2024-03-26T17:52:11.278 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:52:11.278 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:52:11.278 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:52:11.278 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 -- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72f80a2fa0 msgr2=0x7f72f80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72f80a2fa0 0x7f72f80a5460 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x555c522b5b40 tx=0x7f7304009500 comp rx=0 tx=0).stop 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 -- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 msgr2=0x7f730c198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 0x7f730c198070 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f72f4006010 tx=0x7f72f4004040 comp rx=0 tx=0).stop 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 -- 172.21.15.105:0/3227326648 shutdown_connections 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f72f80a2fa0 0x7f72f80a5460 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c102370 0x7f730c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f730c1019b0 0x7f730c198070 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 --2- 172.21.15.105:0/3227326648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f730c1007c0 0x7f730c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 -- 172.21.15.105:0/3227326648 >> 172.21.15.105:0/3227326648 conn(0x7f730c0fbd50 msgr2=0x7f730c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 -- 172.21.15.105:0/3227326648 shutdown_connections 2024-03-26T17:52:11.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.283+0000 7f7312a59640 1 -- 172.21.15.105:0/3227326648 wait complete. 2024-03-26T17:52:11.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:52:11.287 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:52:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 -- 172.21.15.105:0/2994743201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41007e0 msgr2=0x7f7bc4100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2994743201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41007e0 0x7f7bc4100c00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009900 tx=0x7f7bc002bc40 comp rx=0 tx=0).stop 2024-03-26T17:52:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 -- 172.21.15.105:0/2994743201 shutdown_connections 2024-03-26T17:52:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2994743201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4102390 0x7f7bc4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2994743201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bc41019d0 0x7f7bc4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2994743201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41007e0 0x7f7bc4100c00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 -- 172.21.15.105:0/2994743201 >> 172.21.15.105:0/2994743201 conn(0x7f7bc40fbd50 msgr2=0x7f7bc40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 -- 172.21.15.105:0/2994743201 shutdown_connections 2024-03-26T17:52:11.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.430+0000 7f7bcc481640 1 -- 172.21.15.105:0/2994743201 wait complete. 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcc481640 1 Processor -- start 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcc481640 1 -- start start 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcc481640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bc41007e0 0x7f7bc410afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcc481640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41019d0 0x7f7bc4109620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcc481640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4102390 0x7f7bc4109b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcc481640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc410d470 con 0x7f7bc41019d0 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcc481640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc410d2f0 con 0x7f7bc41007e0 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcc481640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc410d5f0 con 0x7f7bc4102390 2024-03-26T17:52:11.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bc9a1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41019d0 0x7f7bc4109620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bca21d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bc41007e0 0x7f7bc410afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bc9a1c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41019d0 0x7f7bc4109620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49696/0 (socket says 172.21.15.105:49696) 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bc9a1c640 1 -- 172.21.15.105:0/2625673229 learned_addr learned my addr 172.21.15.105:0/2625673229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.431+0000 7f7bcaa1e640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4102390 0x7f7bc4109b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bc9a1c640 1 -- 172.21.15.105:0/2625673229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4102390 msgr2=0x7f7bc4109b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bc9a1c640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4102390 0x7f7bc4109b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bc9a1c640 1 -- 172.21.15.105:0/2625673229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bc41007e0 msgr2=0x7f7bc410afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bc9a1c640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bc41007e0 0x7f7bc410afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bc9a1c640 1 -- 172.21.15.105:0/2625673229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc410a3f0 con 0x7f7bc41019d0 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bca21d640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bc41007e0 0x7f7bc410afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bc9a1c640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41019d0 0x7f7bc4109620 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400b4a0 tx=0x7f7bb400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bbb7fe640 1 -- 172.21.15.105:0/2625673229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4015020 con 0x7f7bc41019d0 2024-03-26T17:52:11.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc41a1d20 con 0x7f7bc41019d0 2024-03-26T17:52:11.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bbb7fe640 1 -- 172.21.15.105:0/2625673229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bb4011ce0 con 0x7f7bc41019d0 2024-03-26T17:52:11.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc41a2230 con 0x7f7bc41019d0 2024-03-26T17:52:11.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.432+0000 7f7bbb7fe640 1 -- 172.21.15.105:0/2625673229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4010dc0 con 0x7f7bc41019d0 2024-03-26T17:52:11.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.434+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc4107c20 con 0x7f7bc41019d0 2024-03-26T17:52:11.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.435+0000 7f7bbb7fe640 1 -- 172.21.15.105:0/2625673229 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7bb4018020 con 0x7f7bc41019d0 2024-03-26T17:52:11.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.436+0000 7f7bbb7fe640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7bb00a2f20 0x7f7bb00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.436+0000 7f7bca21d640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7bb00a2f20 0x7f7bb00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.436+0000 7f7bbb7fe640 1 -- 172.21.15.105:0/2625673229 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f7bb40cf220 con 0x7f7bc41019d0 2024-03-26T17:52:11.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.437+0000 7f7bca21d640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7bb00a2f20 0x7f7bb00a53e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005ab0 tx=0x7f7bc0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.441+0000 7f7bbb7fe640 1 -- 172.21.15.105:0/2625673229 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7bb401f020 con 0x7f7bc41019d0 2024-03-26T17:52:11.549 INFO:tasks.workunit.client.0.smithi105.stdout: 10 7388 10.30 MB/sec warmup 54 sec latency 128.435 ms 2024-03-26T17:52:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.798+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bc402d400 con 0x7f7bc41019d0 2024-03-26T17:52:11.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.799+0000 7f7bbb7fe640 1 -- 172.21.15.105:0/2625673229 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11948 (secure 0 0 0) 0x7f7bb409bfb0 con 0x7f7bc41019d0 2024-03-26T17:52:11.790 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:52:11.791 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:51:26.802032+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:52:11.791 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 34 2024-03-26T17:52:11.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7bb00a2f20 msgr2=0x7f7bb00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7bb00a2f20 0x7f7bb00a53e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0005ab0 tx=0x7f7bc0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41019d0 msgr2=0x7f7bc4109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41019d0 0x7f7bc4109620 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400b4a0 tx=0x7f7bb400b8a0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 shutdown_connections 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7bb00a2f20 0x7f7bb00a53e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc4102390 0x7f7bc4109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7bc41019d0 0x7f7bc4109620 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 --2- 172.21.15.105:0/2625673229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bc41007e0 0x7f7bc410afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.802+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 >> 172.21.15.105:0/2625673229 conn(0x7f7bc40fbd50 msgr2=0x7f7bc4104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.803+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 shutdown_connections 2024-03-26T17:52:11.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.803+0000 7f7bcc481640 1 -- 172.21.15.105:0/2625673229 wait complete. 2024-03-26T17:52:11.807 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:52:11.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 -- 172.21.15.105:0/2411688445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340104cd0 msgr2=0x7f73401070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 --2- 172.21.15.105:0/2411688445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340104cd0 0x7f73401070c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f7334009860 tx=0x7f733402bb70 comp rx=0 tx=0).stop 2024-03-26T17:52:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 -- 172.21.15.105:0/2411688445 shutdown_connections 2024-03-26T17:52:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 --2- 172.21.15.105:0/2411688445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7340107600 0x7f73401099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 --2- 172.21.15.105:0/2411688445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340104cd0 0x7f73401070c0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 --2- 172.21.15.105:0/2411688445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73401023a0 0x7f7340104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 -- 172.21.15.105:0/2411688445 >> 172.21.15.105:0/2411688445 conn(0x7f73400fbd50 msgr2=0x7f73400fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 -- 172.21.15.105:0/2411688445 shutdown_connections 2024-03-26T17:52:11.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.953+0000 7f734489a640 1 -- 172.21.15.105:0/2411688445 wait complete. 2024-03-26T17:52:11.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.954+0000 7f734489a640 1 Processor -- start 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.954+0000 7f734489a640 1 -- start start 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.954+0000 7f734489a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73401023a0 0x7f734019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.954+0000 7f734489a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7340104cd0 0x7f734019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.954+0000 7f734489a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340107600 0x7f73401a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.954+0000 7f734489a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73401018c0 con 0x7f7340107600 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.954+0000 7f734489a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7340101740 con 0x7f73401023a0 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.954+0000 7f734489a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7340101a40 con 0x7f7340104cd0 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7340104cd0 0x7f734019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7340104cd0 0x7f734019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46646/0 (socket says 172.21.15.105:46646) 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 -- 172.21.15.105:0/2543015432 learned_addr learned my addr 172.21.15.105:0/2543015432 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733ed9d640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340107600 0x7f73401a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733e59c640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73401023a0 0x7f734019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 -- 172.21.15.105:0/2543015432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73401023a0 msgr2=0x7f734019e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73401023a0 0x7f734019e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 -- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340107600 msgr2=0x7f73401a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340107600 0x7f73401a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 -- 172.21.15.105:0/2543015432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73401a5d90 con 0x7f7340104cd0 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733ed9d640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340107600 0x7f73401a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f733dd9b640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7340104cd0 0x7f734019e6b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f73340057c0 tx=0x7f73340042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.955+0000 7f732f7fe640 1 -- 172.21.15.105:0/2543015432 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334041030 con 0x7f7340104cd0 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.956+0000 7f734489a640 1 -- 172.21.15.105:0/2543015432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73401a6020 con 0x7f7340104cd0 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.956+0000 7f732f7fe640 1 -- 172.21.15.105:0/2543015432 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f733402c9c0 con 0x7f7340104cd0 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.956+0000 7f732f7fe640 1 -- 172.21.15.105:0/2543015432 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334039040 con 0x7f7340104cd0 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.956+0000 7f734489a640 1 -- 172.21.15.105:0/2543015432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73401a6510 con 0x7f7340104cd0 2024-03-26T17:52:11.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.959+0000 7f732f7fe640 1 -- 172.21.15.105:0/2543015432 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7334044070 con 0x7f7340104cd0 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.959+0000 7f732f7fe640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73280a2f20 0x7f73280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.959+0000 7f7347565740 1 Processor -- start 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.959+0000 7f733e59c640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73280a2f20 0x7f73280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.959+0000 7f732f7fe640 1 -- 172.21.15.105:0/2543015432 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f73340f0a60 con 0x7f7340104cd0 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f7347565740 1 -- start start 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f7347565740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb47e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f7347565740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb4cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f7347565740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 0x55bc1cb4fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f7347565740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc1caac970 con 0x55bc1cb47a30 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f7347565740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc1ca54d70 con 0x55bc1cb48420 2024-03-26T17:52:11.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f7347565740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc1caa8920 con 0x55bc1cb4d590 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb4cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb4cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48586/0 (socket says 172.21.15.105:48586) 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 0x55bc1cb4fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb47e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733dd9b640 1 -- 172.21.15.105:0/1466513338 learned_addr learned my addr 172.21.15.105:0/1466513338 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733e59c640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73280a2f20 0x7f73280a53e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7330005fd0 tx=0x7f733000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733e59c640 1 -- 172.21.15.105:0/1466513338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 msgr2=0x55bc1cb4fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733e59c640 1 --2- 172.21.15.105:0/1466513338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 0x55bc1cb4fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733e59c640 1 -- 172.21.15.105:0/1466513338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 msgr2=0x55bc1cb4cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733e59c640 1 --2- 172.21.15.105:0/1466513338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb4cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733e59c640 1 -- 172.21.15.105:0/1466513338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1c7c03f0 con 0x55bc1cb47a30 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.960+0000 7f733dd9b640 1 --2- 172.21.15.105:0/1466513338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb4cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.961+0000 7f733e59c640 1 --2- 172.21.15.105:0/1466513338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb47e50 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7330014000 tx=0x7f7330013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65362ff5fc62491 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.961+0000 7f732cff9640 1 -- 172.21.15.105:0/1466513338 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330003e60 con 0x55bc1cb47a30 2024-03-26T17:52:11.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.961+0000 7f732cff9640 1 -- 172.21.15.105:0/1466513338 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f733001e8d0 con 0x55bc1cb47a30 2024-03-26T17:52:11.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.961+0000 7f732cff9640 1 -- 172.21.15.105:0/1466513338 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733001d9e0 con 0x55bc1cb47a30 2024-03-26T17:52:11.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 -- 172.21.15.105:0/1466513338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 msgr2=0x55bc1cb47e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 --2- 172.21.15.105:0/1466513338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb47e50 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f7330014000 tx=0x7f7330013000 comp rx=0 tx=0).stop 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 -- 172.21.15.105:0/1466513338 shutdown_connections 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 --2- 172.21.15.105:0/1466513338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 0x55bc1cb4fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 --2- 172.21.15.105:0/1466513338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb4cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 --2- 172.21.15.105:0/1466513338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb47e50 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 -- 172.21.15.105:0/1466513338 >> 172.21.15.105:0/1466513338 conn(0x55bc1cb42fa0 msgr2=0x55bc1cb433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 -- 172.21.15.105:0/1466513338 shutdown_connections 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 -- 172.21.15.105:0/1466513338 wait complete. 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.962+0000 7f7347565740 1 -- start start 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f7347565740 1 Processor -- start 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f7347565740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f7347565740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb48420 0x55bc1cb52510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f7347565740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb59550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f7347565740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc1cb50540 con 0x55bc1cb4d590 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f7347565740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc1ca68310 con 0x55bc1cb47a30 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f7347565740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc1cb506c0 con 0x55bc1cb48420 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f7347565740 10 client.0 _ll_register_callbacks cb 0x55bc1caa3c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb59550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb48420 0x55bc1cb52510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48588/0 (socket says 172.21.15.105:48588) 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb59550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49724/0 (socket says 172.21.15.105:49724) 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733e59c640 1 -- 172.21.15.105:0/548934286 learned_addr learned my addr 172.21.15.105:0/548934286 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733ed9d640 1 -- 172.21.15.105:0/548934286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb48420 msgr2=0x55bc1cb52510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733ed9d640 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb48420 0x55bc1cb52510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733ed9d640 1 -- 172.21.15.105:0/548934286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb47a30 msgr2=0x55bc1cb51fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733ed9d640 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.963+0000 7f733ed9d640 1 -- 172.21.15.105:0/548934286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1c88a7d0 con 0x55bc1cb4d590 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f733ed9d640 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb59550 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f7338009fd0 tx=0x7f7338012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d501501dbe63eda7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7325ffb640 1 -- 172.21.15.105:0/548934286 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338003970 con 0x55bc1cb4d590 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7347565740 1 -- 172.21.15.105:0/548934286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733001e8d0 con 0x55bc1cb4d590 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7325ffb640 1 -- 172.21.15.105:0/548934286 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f7338003b10 con 0x55bc1cb4d590 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7325ffb640 1 -- 172.21.15.105:0/548934286 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7338003d10 con 0x55bc1cb4d590 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7347565740 10 client.20496 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7347565740 1 -- 172.21.15.105:0/548934286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bc1cb5c9e0 con 0x55bc1cb4d590 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7325ffb640 1 -- 172.21.15.105:0/548934286 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f733001e8d0 con 0x55bc1cb4d590 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.964+0000 7f7325ffb640 1 -- 172.21.15.105:0/548934286 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f733801c510 con 0x55bc1cb4d590 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f7347565740 10 client.20496 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f7347565740 10 client.20496 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f7347565740 20 client.20496 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f7347565740 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55bc1cb5dfb0 0x55bc1cb60450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f7347565740 4 client.20496 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:11.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f7347565740 1 -- 172.21.15.105:0/548934286 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc1cab9eb0 con 0x55bc1cb5dfb0 2024-03-26T17:52:11.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f733e59c640 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55bc1cb5dfb0 0x55bc1cb60450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f733e59c640 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55bc1cb5dfb0 0x55bc1cb60450 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=901f22e697de6981 server_cookie=9d552eba118b5e57 in_seq=0 out_seq=0 2024-03-26T17:52:11.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.965+0000 7f7325ffb640 10 client.20496 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7325ffb640 1 -- 172.21.15.105:0/548934286 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc1cab9eb0 con 0x55bc1cb5dfb0 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7325ffb640 10 client.20496 handle_command_reply: tid=0 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 client.20496 shutdown 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 -- 172.21.15.105:0/548934286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 msgr2=0x55bc1cb59550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb59550 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f7338009fd0 tx=0x7f7338012820 comp rx=0 tx=0).stop 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 -- 172.21.15.105:0/548934286 shutdown_connections 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55bc1cb5dfb0 0x55bc1cb60450 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb59550 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb48420 0x55bc1cb52510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7325ffb640 0 client.20496 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 --2- 172.21.15.105:0/548934286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 -- 172.21.15.105:0/548934286 >> 172.21.15.105:0/548934286 conn(0x55bc1cb42fa0 msgr2=0x55bc1cb421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 -- 172.21.15.105:0/548934286 shutdown_connections 2024-03-26T17:52:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.966+0000 7f7347565740 1 -- 172.21.15.105:0/548934286 wait complete. 2024-03-26T17:52:11.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.967+0000 7f7347565740 20 client.20496 trim_cache size 0 max 16384 2024-03-26T17:52:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.987+0000 7f7347565740 1 Processor -- start 2024-03-26T17:52:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.987+0000 7f7347565740 1 -- start start 2024-03-26T17:52:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.987+0000 7f7347565740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.987+0000 7f7347565740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.987+0000 7f7347565740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 0x55bc1cb52510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.987+0000 7f7347565740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73401af3e0 con 0x55bc1cb47a30 2024-03-26T17:52:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.987+0000 7f7347565740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73401b0fe0 con 0x55bc1cb48420 2024-03-26T17:52:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.987+0000 7f7347565740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73401afab0 con 0x55bc1cb4d590 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb59510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48598/0 (socket says 172.21.15.105:48598) 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733dd9b640 1 -- 172.21.15.105:0/733144065 learned_addr learned my addr 172.21.15.105:0/733144065 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733ed9d640 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 0x55bc1cb52510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733e59c640 1 -- 172.21.15.105:0/733144065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 msgr2=0x55bc1cb52510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733e59c640 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 0x55bc1cb52510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733e59c640 1 -- 172.21.15.105:0/733144065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 msgr2=0x55bc1cb59510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733e59c640 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb59510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733e59c640 1 -- 172.21.15.105:0/733144065 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1cb4cc70 con 0x55bc1cb47a30 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733dd9b640 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb59510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f733e59c640 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x55bc1cb60c90 tx=0x7f733001d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6519123c0d95a94a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f7327fff640 1 -- 172.21.15.105:0/733144065 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330039070 con 0x55bc1cb47a30 2024-03-26T17:52:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.988+0000 7f7327fff640 1 -- 172.21.15.105:0/733144065 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7330003e60 con 0x55bc1cb47a30 2024-03-26T17:52:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.989+0000 7f7327fff640 1 -- 172.21.15.105:0/733144065 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330004680 con 0x55bc1cb47a30 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.989+0000 7f7347565740 1 -- 172.21.15.105:0/733144065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 msgr2=0x55bc1cb51fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.989+0000 7f7347565740 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x55bc1cb60c90 tx=0x7f733001d3b0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 -- 172.21.15.105:0/733144065 shutdown_connections 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb4d590 0x55bc1cb52510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb59510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 --2- 172.21.15.105:0/733144065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb47a30 0x55bc1cb51fd0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 -- 172.21.15.105:0/733144065 >> 172.21.15.105:0/733144065 conn(0x55bc1cb42fa0 msgr2=0x55bc1cb48b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 -- 172.21.15.105:0/733144065 shutdown_connections 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 -- 172.21.15.105:0/733144065 wait complete. 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 -- start start 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.990+0000 7f7347565740 1 Processor -- start 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f7347565740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb56980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f7347565740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f7347565740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb5dfb0 0x55bc1cb57e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f7347565740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73401b0910 con 0x55bc1cb4d590 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f7347565740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73401b0180 con 0x55bc1cb48420 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f7347565740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73401ad3c0 con 0x55bc1cb5dfb0 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb56980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb56980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48600/0 (socket says 172.21.15.105:48600) 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 -- 172.21.15.105:0/2395779046 learned_addr learned my addr 172.21.15.105:0/2395779046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb56ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49744/0 (socket says 172.21.15.105:49744) 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f7347565740 10 client.0 _ll_register_callbacks cb 0x7f733800f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733ed9d640 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb5dfb0 0x55bc1cb57e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 -- 172.21.15.105:0/2395779046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb5dfb0 msgr2=0x55bc1cb57e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb5dfb0 0x55bc1cb57e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 -- 172.21.15.105:0/2395779046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 msgr2=0x55bc1cb56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 -- 172.21.15.105:0/2395779046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1cb58470 con 0x55bc1cb48420 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733dd9b640 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f733e59c640 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb56980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7330000c00 tx=0x7f7330003940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eb95fc7dd9ac1a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.991+0000 7f7325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7325ffb640 1 -- 172.21.15.105:0/2395779046 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330039040 con 0x55bc1cb48420 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 1 -- 172.21.15.105:0/2395779046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7330003e60 con 0x55bc1cb48420 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7325ffb640 1 -- 172.21.15.105:0/2395779046 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f7330034030 con 0x55bc1cb48420 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7325ffb640 1 -- 172.21.15.105:0/2395779046 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7330034230 con 0x55bc1cb48420 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 10 client.30290 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 1 -- 172.21.15.105:0/2395779046 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bc1cb59750 con 0x55bc1cb48420 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7325ffb640 1 -- 172.21.15.105:0/2395779046 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7330003e60 con 0x55bc1cb48420 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7325ffb640 1 -- 172.21.15.105:0/2395779046 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f733003c070 con 0x55bc1cb48420 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 10 client.30290 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:11.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 10 client.30290 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:52:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 20 client.30290 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55bc1cbc4780 0x55bc1cbc6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 4 client.30290 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.992+0000 7f7347565740 1 -- 172.21.15.105:0/2395779046 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc1cab4200 con 0x55bc1cbc4780 2024-03-26T17:52:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.993+0000 7f733dd9b640 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55bc1cbc4780 0x55bc1cbc6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.993+0000 7f7325ffb640 10 client.30290 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:52:11.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.993+0000 7f733dd9b640 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55bc1cbc4780 0x55bc1cbc6b70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1b0a43ae11cbf93 server_cookie=f94fee53f813b226 in_seq=0 out_seq=0 2024-03-26T17:52:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.993+0000 7f7325ffb640 1 -- 172.21.15.105:0/2395779046 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc1cab4200 con 0x55bc1cbc4780 2024-03-26T17:52:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.993+0000 7f7325ffb640 10 client.30290 handle_command_reply: tid=0 2024-03-26T17:52:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 client.30290 shutdown 2024-03-26T17:52:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 -- 172.21.15.105:0/2395779046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 msgr2=0x55bc1cb56980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb56980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7330000c00 tx=0x7f7330003940 comp rx=0 tx=0).stop 2024-03-26T17:52:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 -- 172.21.15.105:0/2395779046 shutdown_connections 2024-03-26T17:52:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55bc1cbc4780 0x55bc1cbc6b70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc1cb5dfb0 0x55bc1cb57e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc1cb4d590 0x55bc1cb56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 --2- 172.21.15.105:0/2395779046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc1cb48420 0x55bc1cb56980 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 -- 172.21.15.105:0/2395779046 >> 172.21.15.105:0/2395779046 conn(0x55bc1cb42fa0 msgr2=0x55bc1cb50a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7325ffb640 0 client.30290 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:52:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 -- 172.21.15.105:0/2395779046 shutdown_connections 2024-03-26T17:52:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 1 -- 172.21.15.105:0/2395779046 wait complete. 2024-03-26T17:52:11.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.994+0000 7f7347565740 20 client.30290 trim_cache size 0 max 16384 2024-03-26T17:52:11.989 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 -- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73280a2f20 msgr2=0x7f73280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73280a2f20 0x7f73280a53e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7330005fd0 tx=0x7f733000b410 comp rx=0 tx=0).stop 2024-03-26T17:52:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 -- 172.21.15.105:0/2543015432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7340104cd0 msgr2=0x7f734019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7340104cd0 0x7f734019e6b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f73340057c0 tx=0x7f73340042f0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 -- 172.21.15.105:0/2543015432 shutdown_connections 2024-03-26T17:52:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73280a2f20 0x7f73280a53e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7340107600 0x7f73401a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7340104cd0 0x7f734019e6b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 --2- 172.21.15.105:0/2543015432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73401023a0 0x7f734019e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 -- 172.21.15.105:0/2543015432 >> 172.21.15.105:0/2543015432 conn(0x7f73400fbd50 msgr2=0x7f7340102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 -- 172.21.15.105:0/2543015432 shutdown_connections 2024-03-26T17:52:11.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:11.995+0000 7f734489a640 1 -- 172.21.15.105:0/2543015432 wait complete. 2024-03-26T17:52:11.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:52:12.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 -- 172.21.15.105:0/3824550948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08104cb0 msgr2=0x7f7a081070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 --2- 172.21.15.105:0/3824550948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08104cb0 0x7f7a081070a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009900 tx=0x7f79f802bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 -- 172.21.15.105:0/3824550948 shutdown_connections 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 --2- 172.21.15.105:0/3824550948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a081075e0 0x7f7a08109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 --2- 172.21.15.105:0/3824550948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08104cb0 0x7f7a081070a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 --2- 172.21.15.105:0/3824550948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a08102380 0x7f7a08104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 -- 172.21.15.105:0/3824550948 >> 172.21.15.105:0/3824550948 conn(0x7f7a080fbd70 msgr2=0x7f7a080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 -- 172.21.15.105:0/3824550948 shutdown_connections 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.148+0000 7f7a106e8640 1 -- 172.21.15.105:0/3824550948 wait complete. 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a106e8640 1 Processor -- start 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a106e8640 1 -- start start 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a106e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08102380 0x7f7a08199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a106e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a08104cb0 0x7f7a0819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a106e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a081075e0 0x7f7a081a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a106e8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08102040 con 0x7f7a08104cb0 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a106e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a08101ec0 con 0x7f7a081075e0 2024-03-26T17:52:12.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a106e8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081021c0 con 0x7f7a08102380 2024-03-26T17:52:12.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a0dc83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a08104cb0 0x7f7a0819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a0dc83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a08104cb0 0x7f7a0819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49764/0 (socket says 172.21.15.105:49764) 2024-03-26T17:52:12.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a0dc83640 1 -- 172.21.15.105:0/102248684 learned_addr learned my addr 172.21.15.105:0/102248684 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a0e484640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08102380 0x7f7a08199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.149+0000 7f7a0ec85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a081075e0 0x7f7a081a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:11 smithi105 ceph-mon[32280]: pgmap v764: 97 pgs: 97 active+clean; 222 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 270 op/s 2024-03-26T17:52:12.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:11 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2625673229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:52:12.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a0dc83640 1 -- 172.21.15.105:0/102248684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08102380 msgr2=0x7f7a08199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08102380 0x7f7a08199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a0dc83640 1 -- 172.21.15.105:0/102248684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a081075e0 msgr2=0x7f7a081a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a081075e0 0x7f7a081a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a0dc83640 1 -- 172.21.15.105:0/102248684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a081a19d0 con 0x7f7a08104cb0 2024-03-26T17:52:12.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a0e484640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08102380 0x7f7a08199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a08104cb0 0x7f7a0819a300 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000c00 tx=0x7f79f8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f79ff7fe640 1 -- 172.21.15.105:0/102248684 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8043020 con 0x7f7a08104cb0 2024-03-26T17:52:12.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a106e8640 1 -- 172.21.15.105:0/102248684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a081a1c60 con 0x7f7a08104cb0 2024-03-26T17:52:12.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f79ff7fe640 1 -- 172.21.15.105:0/102248684 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79f80042d0 con 0x7f7a08104cb0 2024-03-26T17:52:12.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f7a106e8640 1 -- 172.21.15.105:0/102248684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a081a21b0 con 0x7f7a08104cb0 2024-03-26T17:52:12.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.150+0000 7f79ff7fe640 1 -- 172.21.15.105:0/102248684 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8003b10 con 0x7f7a08104cb0 2024-03-26T17:52:12.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.153+0000 7f79ff7fe640 1 -- 172.21.15.105:0/102248684 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f79f8046020 con 0x7f7a08104cb0 2024-03-26T17:52:12.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a133ad740 1 Processor -- start 2024-03-26T17:52:12.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a133ad740 1 -- start start 2024-03-26T17:52:12.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f79ff7fe640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79f40a2f20 0x7f79f40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea26e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea27420 0x55e43ea2bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 0x55e43ea2ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a133ad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e43e98b970 con 0x55e43ea26a30 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a133ad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e43e933d70 con 0x55e43ea2c590 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a133ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e43e987920 con 0x55e43ea27420 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f79ff7fe640 1 -- 172.21.15.105:0/102248684 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f79f80f15c0 con 0x7f7a08104cb0 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a0ec85640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79f40a2f20 0x7f79f40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.154+0000 7f7a0e484640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea26e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0e484640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea26e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49766/0 (socket says 172.21.15.105:49766) 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0e484640 1 -- 172.21.15.105:0/3054520802 learned_addr learned my addr 172.21.15.105:0/3054520802 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea27420 0x55e43ea2bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0e484640 1 -- 172.21.15.105:0/3054520802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea27420 msgr2=0x55e43ea2bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0e484640 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea27420 0x55e43ea2bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0e484640 1 -- 172.21.15.105:0/3054520802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 msgr2=0x55e43ea2ea20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0e484640 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 0x55e43ea2ea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0e484640 1 -- 172.21.15.105:0/3054520802 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43e69f3f0 con 0x55e43ea26a30 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea27420 0x55e43ea2bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0ec85640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79f40a2f20 0x7f79f40a53e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005fd0 tx=0x7f7a00009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.155+0000 7f7a0e484640 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea26e50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7a040114f0 tx=0x7f7a040119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae0bc85a5722fae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.156+0000 7f79fcff9640 1 -- 172.21.15.105:0/3054520802 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04011250 con 0x55e43ea26a30 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.156+0000 7f79fcff9640 1 -- 172.21.15.105:0/3054520802 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a040035f0 con 0x55e43ea26a30 2024-03-26T17:52:12.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.156+0000 7f79fcff9640 1 -- 172.21.15.105:0/3054520802 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a040044e0 con 0x55e43ea26a30 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 -- 172.21.15.105:0/3054520802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 msgr2=0x55e43ea26e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea26e50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7a040114f0 tx=0x7f7a040119b0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 -- 172.21.15.105:0/3054520802 shutdown_connections 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 0x55e43ea2ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea27420 0x55e43ea2bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 --2- 172.21.15.105:0/3054520802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea26e50 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 -- 172.21.15.105:0/3054520802 >> 172.21.15.105:0/3054520802 conn(0x55e43ea21fa0 msgr2=0x55e43ea223a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 -- 172.21.15.105:0/3054520802 shutdown_connections 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 -- 172.21.15.105:0/3054520802 wait complete. 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.157+0000 7f7a133ad740 1 -- start start 2024-03-26T17:52:12.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a133ad740 1 Processor -- start 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea27420 0x55e43ea39cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 0x55e43ea40d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a133ad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e43ea37d50 con 0x55e43ea27420 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a133ad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e43e947310 con 0x55e43ea2c590 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a133ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e43ea37ed0 con 0x55e43ea26a30 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a133ad740 10 client.0 _ll_register_callbacks cb 0x55e43e982c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0e484640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0dc83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea27420 0x55e43ea39cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0e484640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46724/0 (socket says 172.21.15.105:46724) 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0e484640 1 -- 172.21.15.105:0/982267694 learned_addr learned my addr 172.21.15.105:0/982267694 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0ec85640 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 0x55e43ea40d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0dc83640 1 -- 172.21.15.105:0/982267694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 msgr2=0x55e43ea397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0dc83640 1 -- 172.21.15.105:0/982267694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 msgr2=0x55e43ea40d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 0x55e43ea40d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0dc83640 1 -- 172.21.15.105:0/982267694 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43e7697d0 con 0x55e43ea27420 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0e484640 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.158+0000 7f7a0ec85640 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 0x55e43ea40d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.159+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea27420 0x55e43ea39cf0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x55e43ea27e90 tx=0x7f79f80bd6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b53277fd34abbe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.159+0000 7f79f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.159+0000 7f79f1ffb640 1 -- 172.21.15.105:0/982267694 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f80c1e00 con 0x55e43ea27420 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.159+0000 7f79f1ffb640 1 -- 172.21.15.105:0/982267694 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f79f8003cb0 con 0x55e43ea27420 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.159+0000 7f7a133ad740 1 -- 172.21.15.105:0/982267694 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a040035f0 con 0x55e43ea27420 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.159+0000 7f79f1ffb640 1 -- 172.21.15.105:0/982267694 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f79f80c66c0 con 0x55e43ea27420 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.159+0000 7f7a133ad740 10 client.20514 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.159+0000 7f7a133ad740 1 -- 172.21.15.105:0/982267694 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e43ea44010 con 0x55e43ea27420 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f79f1ffb640 1 -- 172.21.15.105:0/982267694 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a040035f0 con 0x55e43ea27420 2024-03-26T17:52:12.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f79f1ffb640 1 -- 172.21.15.105:0/982267694 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f79f80f11f0 con 0x55e43ea27420 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f7a133ad740 10 client.20514 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f7a133ad740 10 client.20514 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f7a133ad740 20 client.20514 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f7a133ad740 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e43ea45570 0x55e43ea479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f7a133ad740 4 client.20514 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f7a133ad740 1 -- 172.21.15.105:0/982267694 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e43e998eb0 con 0x55e43ea45570 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f7a0e484640 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e43ea45570 0x55e43ea479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.160+0000 7f79f1ffb640 10 client.20514 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:52:12.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.161+0000 7f7a0e484640 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e43ea45570 0x55e43ea479f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6a3a747d9e8399fc server_cookie=bf830b2bea9b905a in_seq=0 out_seq=0 2024-03-26T17:52:12.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f79f1ffb640 1 -- 172.21.15.105:0/982267694 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e43e998eb0 con 0x55e43ea45570 2024-03-26T17:52:12.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f79f1ffb640 10 client.20514 handle_command_reply: tid=0 2024-03-26T17:52:12.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 client.20514 shutdown 2024-03-26T17:52:12.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 -- 172.21.15.105:0/982267694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea27420 msgr2=0x55e43ea39cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea27420 0x55e43ea39cf0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x55e43ea27e90 tx=0x7f79f80bd6d0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 -- 172.21.15.105:0/982267694 shutdown_connections 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e43ea45570 0x55e43ea479f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f79f1ffb640 0 client.20514 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea2c590 0x55e43ea40d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea27420 0x55e43ea39cf0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 --2- 172.21.15.105:0/982267694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 -- 172.21.15.105:0/982267694 >> 172.21.15.105:0/982267694 conn(0x55e43ea21fa0 msgr2=0x55e43ea211c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 -- 172.21.15.105:0/982267694 shutdown_connections 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.162+0000 7f7a133ad740 1 -- 172.21.15.105:0/982267694 wait complete. 2024-03-26T17:52:12.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.163+0000 7f7a133ad740 20 client.20514 trim_cache size 0 max 16384 2024-03-26T17:52:12.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:11 smithi135 ceph-mon[35636]: pgmap v764: 97 pgs: 97 active+clean; 222 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 270 op/s 2024-03-26T17:52:12.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:11 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2625673229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:52:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.185+0000 7f7a133ad740 1 Processor -- start 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.185+0000 7f7a133ad740 1 -- start start 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea40cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea2c590 0x55e43ea28ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a133ad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081a6d30 con 0x55e43ea2c590 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a133ad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081a8930 con 0x55e43ea27420 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a133ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081a7400 con 0x55e43ea26a30 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0ec85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea2c590 0x55e43ea28ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0ec85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea2c590 0x55e43ea28ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49798/0 (socket says 172.21.15.105:49798) 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0ec85640 1 -- 172.21.15.105:0/1922924854 learned_addr learned my addr 172.21.15.105:0/1922924854 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0e484640 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea40cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0e484640 1 -- 172.21.15.105:0/1922924854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 msgr2=0x55e43ea40cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0e484640 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea40cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0e484640 1 -- 172.21.15.105:0/1922924854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea2c590 msgr2=0x55e43ea28ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0e484640 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea2c590 0x55e43ea28ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0e484640 1 -- 172.21.15.105:0/1922924854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43ea26820 con 0x55e43ea26a30 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.186+0000 7f7a0ec85640 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea2c590 0x55e43ea28ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.187+0000 7f7a0e484640 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea28980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7a040114c0 tx=0x7f7a04003d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17227dca5657556b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.187+0000 7f79f3fff640 1 -- 172.21.15.105:0/1922924854 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a040035f0 con 0x55e43ea26a30 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.187+0000 7f79f3fff640 1 -- 172.21.15.105:0/1922924854 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a0402c7a0 con 0x55e43ea26a30 2024-03-26T17:52:12.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.187+0000 7f79f3fff640 1 -- 172.21.15.105:0/1922924854 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0400f5c0 con 0x55e43ea26a30 2024-03-26T17:52:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- 172.21.15.105:0/1922924854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 msgr2=0x55e43ea28980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea28980 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7a040114c0 tx=0x7f7a04003d00 comp rx=0 tx=0).stop 2024-03-26T17:52:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- 172.21.15.105:0/1922924854 shutdown_connections 2024-03-26T17:52:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea2c590 0x55e43ea28ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea40cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1922924854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea26a30 0x55e43ea28980 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- 172.21.15.105:0/1922924854 >> 172.21.15.105:0/1922924854 conn(0x55e43ea21fa0 msgr2=0x55e43ea22380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- 172.21.15.105:0/1922924854 shutdown_connections 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- 172.21.15.105:0/1922924854 wait complete. 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- start start 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 Processor -- start 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea3e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea2c590 0x55e43ea3f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081a8260 con 0x55e43ea26a30 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081a7ad0 con 0x55e43ea27420 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.188+0000 7f7a133ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a081a4d10 con 0x55e43ea2c590 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a133ad740 10 client.0 _ll_register_callbacks cb 0x7f79f8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0dc83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea3e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0e484640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0e484640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea3e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49812/0 (socket says 172.21.15.105:49812) 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0ec85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea2c590 0x55e43ea3f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0e484640 1 -- 172.21.15.105:0/1281384741 learned_addr learned my addr 172.21.15.105:0/1281384741 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0dc83640 1 -- 172.21.15.105:0/1281384741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea2c590 msgr2=0x55e43ea3f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea2c590 0x55e43ea3f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0dc83640 1 -- 172.21.15.105:0/1281384741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 msgr2=0x55e43ea3e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea3e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0dc83640 1 -- 172.21.15.105:0/1281384741 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43ea3fdd0 con 0x55e43ea27420 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0ec85640 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea2c590 0x55e43ea3f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0e484640 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea3e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a0dc83640 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea3e7d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f79f8005ab0 tx=0x7f79f80057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d632819342eb5b57 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f79f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f79f1ffb640 1 -- 172.21.15.105:0/1281384741 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f80bda80 con 0x55e43ea27420 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f7a133ad740 1 -- 172.21.15.105:0/1281384741 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a0402c7a0 con 0x55e43ea27420 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.189+0000 7f79f1ffb640 1 -- 172.21.15.105:0/1281384741 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f79f8004480 con 0x55e43ea27420 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f79f1ffb640 1 -- 172.21.15.105:0/1281384741 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f79f80c1e00 con 0x55e43ea27420 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a133ad740 10 client.30305 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a133ad740 1 -- 172.21.15.105:0/1281384741 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e43ea3fdd0 con 0x55e43ea27420 2024-03-26T17:52:12.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f79f1ffb640 1 -- 172.21.15.105:0/1281384741 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a0402c7a0 con 0x55e43ea27420 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f79f1ffb640 1 -- 172.21.15.105:0/1281384741 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f79f80bcb70 con 0x55e43ea27420 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a133ad740 10 client.30305 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a133ad740 10 client.30305 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a133ad740 20 client.30305 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e43eaabe40 0x55e43eaae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a133ad740 4 client.30305 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a133ad740 1 -- 172.21.15.105:0/1281384741 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e43e993200 con 0x55e43eaabe40 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.190+0000 7f7a0e484640 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e43eaabe40 0x55e43eaae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f79f1ffb640 10 client.30305 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a0e484640 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e43eaabe40 0x55e43eaae260 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b2f0846c4d4c919a server_cookie=f19dd4049b2160c6 in_seq=0 out_seq=0 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f79f1ffb640 1 -- 172.21.15.105:0/1281384741 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e43e993200 con 0x55e43eaabe40 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f79f1ffb640 10 client.30305 handle_command_reply: tid=0 2024-03-26T17:52:12.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 client.30305 shutdown 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 -- 172.21.15.105:0/1281384741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 msgr2=0x55e43ea3e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea3e7d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f79f8005ab0 tx=0x7f79f80057e0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 -- 172.21.15.105:0/1281384741 shutdown_connections 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55e43eaabe40 0x55e43eaae260 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e43ea2c590 0x55e43ea3f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e43ea27420 0x55e43ea3e7d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f79f1ffb640 0 client.30305 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 --2- 172.21.15.105:0/1281384741 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e43ea26a30 0x55e43ea3e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 -- 172.21.15.105:0/1281384741 >> 172.21.15.105:0/1281384741 conn(0x55e43ea21fa0 msgr2=0x55e43ea39790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 -- 172.21.15.105:0/1281384741 shutdown_connections 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 1 -- 172.21.15.105:0/1281384741 wait complete. 2024-03-26T17:52:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.191+0000 7f7a133ad740 20 client.30305 trim_cache size 0 max 16384 2024-03-26T17:52:12.188 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:12.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 -- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79f40a2f20 msgr2=0x7f79f40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79f40a2f20 0x7f79f40a53e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005fd0 tx=0x7f7a00009500 comp rx=0 tx=0).stop 2024-03-26T17:52:12.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 -- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a08104cb0 msgr2=0x7f7a0819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a08104cb0 0x7f7a0819a300 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000c00 tx=0x7f79f8005d50 comp rx=0 tx=0).stop 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 -- 172.21.15.105:0/102248684 shutdown_connections 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79f40a2f20 0x7f79f40a53e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a081075e0 0x7f7a081a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a08104cb0 0x7f7a0819a300 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 --2- 172.21.15.105:0/102248684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08102380 0x7f7a08199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 -- 172.21.15.105:0/102248684 >> 172.21.15.105:0/102248684 conn(0x7f7a080fbd70 msgr2=0x7f7a08104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 -- 172.21.15.105:0/102248684 shutdown_connections 2024-03-26T17:52:12.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.192+0000 7f7a106e8640 1 -- 172.21.15.105:0/102248684 wait complete. 2024-03-26T17:52:12.196 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:52:12.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 -- 172.21.15.105:0/1419455266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f81023b0 msgr2=0x7f84f81047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 --2- 172.21.15.105:0/1419455266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f81023b0 0x7f84f81047a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009900 tx=0x7f84e802bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:12.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 -- 172.21.15.105:0/1419455266 shutdown_connections 2024-03-26T17:52:12.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 --2- 172.21.15.105:0/1419455266 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f8107610 0x7f84f8109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 --2- 172.21.15.105:0/1419455266 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84f8104ce0 0x7f84f81070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 --2- 172.21.15.105:0/1419455266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f81023b0 0x7f84f81047a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 -- 172.21.15.105:0/1419455266 >> 172.21.15.105:0/1419455266 conn(0x7f84f80fbd60 msgr2=0x7f84f80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 -- 172.21.15.105:0/1419455266 shutdown_connections 2024-03-26T17:52:12.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 -- 172.21.15.105:0/1419455266 wait complete. 2024-03-26T17:52:12.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.346+0000 7f84fddda640 1 Processor -- start 2024-03-26T17:52:12.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84fddda640 1 -- start start 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84fddda640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84f81023b0 0x7f84f8195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84fddda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f8104ce0 0x7f84f8195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84fddda640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f8107610 0x7f84f819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84fddda640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84f810b250 con 0x7f84f8104ce0 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84fddda640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84f810b0d0 con 0x7f84f81023b0 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84fddda640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84f810b3d0 con 0x7f84f8107610 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f8107610 0x7f84f819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f8107610 0x7f84f819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46770/0 (socket says 172.21.15.105:46770) 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f7fff640 1 -- 172.21.15.105:0/2209528937 learned_addr learned my addr 172.21.15.105:0/2209528937 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84effff640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f8104ce0 0x7f84f8195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f77fe640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84f81023b0 0x7f84f8195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f7fff640 1 -- 172.21.15.105:0/2209528937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84f81023b0 msgr2=0x7f84f8195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f7fff640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84f81023b0 0x7f84f8195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f7fff640 1 -- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f8104ce0 msgr2=0x7f84f8195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f7fff640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f8104ce0 0x7f84f8195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.347+0000 7f84f7fff640 1 -- 172.21.15.105:0/2209528937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f819d5b0 con 0x7f84f8107610 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.348+0000 7f84f7fff640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f8107610 0x7f84f819cf40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f84f000eb10 tx=0x7f84f000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.348+0000 7f84f57fa640 1 -- 172.21.15.105:0/2209528937 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0004b20 con 0x7f84f8107610 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.348+0000 7f84fddda640 1 -- 172.21.15.105:0/2209528937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f819d8a0 con 0x7f84f8107610 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.348+0000 7f84f57fa640 1 -- 172.21.15.105:0/2209528937 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84f0004cc0 con 0x7f84f8107610 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.348+0000 7f84f57fa640 1 -- 172.21.15.105:0/2209528937 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0010690 con 0x7f84f8107610 2024-03-26T17:52:12.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.348+0000 7f84fddda640 1 -- 172.21.15.105:0/2209528937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f819dda0 con 0x7f84f8107610 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f85009bd740 1 Processor -- start 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f84f57fa640 1 -- 172.21.15.105:0/2209528937 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f84f0005020 con 0x7f84f8107610 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f85009bd740 1 -- start start 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f85009bd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36deb2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f85009bd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f85009bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 0x55a36deba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f85009bd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a36de0e970 con 0x55a36deb2c30 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f85009bd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a36ddb6d70 con 0x55a36deb2240 2024-03-26T17:52:12.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f85009bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a36de0a920 con 0x55a36deb7da0 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.351+0000 7f84f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84effff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36deb2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49834/0 (socket says 172.21.15.105:49834) 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84f77fe640 1 -- 172.21.15.105:0/181398234 learned_addr learned my addr 172.21.15.105:0/181398234 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84effff640 1 --2- 172.21.15.105:0/181398234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 0x55a36deba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84effff640 1 -- 172.21.15.105:0/181398234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 msgr2=0x55a36deba230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84effff640 1 --2- 172.21.15.105:0/181398234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 0x55a36deba230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84f57fa640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84d80a2f70 0x7f84d80a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84effff640 1 -- 172.21.15.105:0/181398234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 msgr2=0x55a36deb7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84effff640 1 --2- 172.21.15.105:0/181398234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84effff640 1 -- 172.21.15.105:0/181398234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36db223f0 con 0x55a36deb2240 2024-03-26T17:52:12.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84effff640 1 --2- 172.21.15.105:0/181398234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36deb2660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f84f8196ea0 tx=0x7f84e0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f446e3b6e48bc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84f57fa640 1 -- 172.21.15.105:0/2209528937 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f84f00cf980 con 0x7f84f8107610 2024-03-26T17:52:12.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84f7fff640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84d80a2f70 0x7f84d80a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84edffb640 1 -- 172.21.15.105:0/181398234 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004050 con 0x55a36deb2240 2024-03-26T17:52:12.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84edffb640 1 -- 172.21.15.105:0/181398234 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e000dc60 con 0x55a36deb2240 2024-03-26T17:52:12.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.352+0000 7f84edffb640 1 -- 172.21.15.105:0/181398234 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004b00 con 0x55a36deb2240 2024-03-26T17:52:12.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f85009bd740 1 -- 172.21.15.105:0/181398234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 msgr2=0x55a36deb2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f84f7fff640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84d80a2f70 0x7f84d80a5430 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f84f009c000 tx=0x7f84f0009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f85009bd740 1 --2- 172.21.15.105:0/181398234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36deb2660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f84f8196ea0 tx=0x7f84e0005f90 comp rx=0 tx=0).stop 2024-03-26T17:52:12.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f85009bd740 1 -- 172.21.15.105:0/181398234 shutdown_connections 2024-03-26T17:52:12.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f85009bd740 1 --2- 172.21.15.105:0/181398234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 0x55a36deba230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f85009bd740 1 --2- 172.21.15.105:0/181398234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f85009bd740 1 --2- 172.21.15.105:0/181398234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36deb2660 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f85009bd740 1 -- 172.21.15.105:0/181398234 >> 172.21.15.105:0/181398234 conn(0x55a36dea4fa0 msgr2=0x55a36dea53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.353+0000 7f85009bd740 1 -- 172.21.15.105:0/181398234 shutdown_connections 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 -- 172.21.15.105:0/181398234 wait complete. 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 -- start start 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 Processor -- start 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36debc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2c30 0x55a36debcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb7da0 0x55a36dec3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a36debacf0 con 0x55a36deb7da0 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a36ddca310 con 0x55a36deb2240 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a36debae70 con 0x55a36deb2c30 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.354+0000 7f85009bd740 10 client.0 _ll_register_callbacks cb 0x55a36de05c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:12.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36debc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb7da0 0x55a36dec3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb7da0 0x55a36dec3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49842/0 (socket says 172.21.15.105:49842) 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 -- 172.21.15.105:0/2629187525 learned_addr learned my addr 172.21.15.105:0/2629187525 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84f77fe640 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2c30 0x55a36debcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 -- 172.21.15.105:0/2629187525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2c30 msgr2=0x55a36debcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2c30 0x55a36debcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 -- 172.21.15.105:0/2629187525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 msgr2=0x55a36debc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36debc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 -- 172.21.15.105:0/2629187525 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36dbec7d0 con 0x55a36deb7da0 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84effff640 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb7da0 0x55a36dec3ca0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f84e000d660 tx=0x7f84e00183f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db1e86abf5c6074 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84e6ffd640 1 -- 172.21.15.105:0/2629187525 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0030060 con 0x55a36deb7da0 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84e6ffd640 1 -- 172.21.15.105:0/2629187525 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f84e0004d40 con 0x55a36deb7da0 2024-03-26T17:52:12.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f85009bd740 1 -- 172.21.15.105:0/2629187525 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e000dc60 con 0x55a36deb7da0 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.355+0000 7f84e6ffd640 1 -- 172.21.15.105:0/2629187525 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f84e000b630 con 0x55a36deb7da0 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f85009bd740 10 client.20532 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f85009bd740 1 -- 172.21.15.105:0/2629187525 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a36dec7010 con 0x55a36deb7da0 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f84e6ffd640 1 -- 172.21.15.105:0/2629187525 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e000dc60 con 0x55a36deb7da0 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f84e6ffd640 1 -- 172.21.15.105:0/2629187525 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f84e0028040 con 0x55a36deb7da0 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f85009bd740 10 client.20532 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f85009bd740 10 client.20532 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f85009bd740 20 client.20532 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f85009bd740 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a36dec8410 0x55a36deca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f85009bd740 4 client.20532 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.356+0000 7f85009bd740 1 -- 172.21.15.105:0/2629187525 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a36de1beb0 con 0x55a36dec8410 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.357+0000 7f84f77fe640 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a36dec8410 0x55a36deca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.357+0000 7f84f77fe640 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a36dec8410 0x55a36deca800 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53a4fbff9aa5f2c0 server_cookie=2b53ea007703eae in_seq=0 out_seq=0 2024-03-26T17:52:12.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.357+0000 7f84e6ffd640 10 client.20532 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:52:12.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:11 smithi144 ceph-mon[36095]: pgmap v764: 97 pgs: 97 active+clean; 222 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.7 MiB/s wr, 270 op/s 2024-03-26T17:52:12.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:11 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2625673229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:52:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.358+0000 7f84e6ffd640 1 -- 172.21.15.105:0/2629187525 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a36de1beb0 con 0x55a36dec8410 2024-03-26T17:52:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.358+0000 7f84e6ffd640 10 client.20532 handle_command_reply: tid=0 2024-03-26T17:52:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.358+0000 7f85009bd740 1 client.20532 shutdown 2024-03-26T17:52:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 -- 172.21.15.105:0/2629187525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb7da0 msgr2=0x55a36dec3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb7da0 0x55a36dec3ca0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f84e000d660 tx=0x7f84e00183f0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 -- 172.21.15.105:0/2629187525 shutdown_connections 2024-03-26T17:52:12.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb7da0 0x55a36dec3ca0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2c30 0x55a36debcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a36dec8410 0x55a36deca800 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 --2- 172.21.15.105:0/2629187525 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36debc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 -- 172.21.15.105:0/2629187525 >> 172.21.15.105:0/2629187525 conn(0x55a36dea4fa0 msgr2=0x55a36dea41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f84e6ffd640 0 client.20532 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:52:12.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 -- 172.21.15.105:0/2629187525 shutdown_connections 2024-03-26T17:52:12.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 1 -- 172.21.15.105:0/2629187525 wait complete. 2024-03-26T17:52:12.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.359+0000 7f85009bd740 20 client.20532 trim_cache size 0 max 16384 2024-03-26T17:52:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f85009bd740 1 Processor -- start 2024-03-26T17:52:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f85009bd740 1 -- start start 2024-03-26T17:52:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f85009bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2240 0x55a36deb4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f85009bd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f85009bd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb7da0 0x55a36dec3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f85009bd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84f81a6c00 con 0x55a36deb2c30 2024-03-26T17:52:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f85009bd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84f81a8800 con 0x55a36deb7da0 2024-03-26T17:52:12.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f85009bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84f81a72d0 con 0x55a36deb2240 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84effff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2240 0x55a36deb4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49852/0 (socket says 172.21.15.105:49852) 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84f77fe640 1 -- 172.21.15.105:0/4019065280 learned_addr learned my addr 172.21.15.105:0/4019065280 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84effff640 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb7da0 0x55a36dec3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84effff640 1 -- 172.21.15.105:0/4019065280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb7da0 msgr2=0x55a36dec3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84effff640 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb7da0 0x55a36dec3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84effff640 1 -- 172.21.15.105:0/4019065280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 msgr2=0x55a36deb46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84effff640 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.382+0000 7f84effff640 1 -- 172.21.15.105:0/4019065280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36dea5ee0 con 0x55a36deb2240 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.383+0000 7f84f77fe640 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.383+0000 7f84effff640 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2240 0x55a36deb4190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f84e0001eb0 tx=0x7f84e0005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e4ffd77b1ee774 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.383+0000 7f84ed7fa640 1 -- 172.21.15.105:0/4019065280 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0030070 con 0x55a36deb2240 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.383+0000 7f84ed7fa640 1 -- 172.21.15.105:0/4019065280 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e000d3c0 con 0x55a36deb2240 2024-03-26T17:52:12.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.383+0000 7f84ed7fa640 1 -- 172.21.15.105:0/4019065280 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0036940 con 0x55a36deb2240 2024-03-26T17:52:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.384+0000 7f85009bd740 1 -- 172.21.15.105:0/4019065280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2240 msgr2=0x55a36deb4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.384+0000 7f85009bd740 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2240 0x55a36deb4190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f84e0001eb0 tx=0x7f84e0005970 comp rx=0 tx=0).stop 2024-03-26T17:52:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.384+0000 7f85009bd740 1 -- 172.21.15.105:0/4019065280 shutdown_connections 2024-03-26T17:52:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.384+0000 7f85009bd740 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb7da0 0x55a36dec3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.384+0000 7f85009bd740 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36deb46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.384+0000 7f85009bd740 1 --2- 172.21.15.105:0/4019065280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb2240 0x55a36deb4190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.384+0000 7f85009bd740 1 -- 172.21.15.105:0/4019065280 >> 172.21.15.105:0/4019065280 conn(0x55a36dea4fa0 msgr2=0x55a36dea5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.384+0000 7f85009bd740 1 -- 172.21.15.105:0/4019065280 shutdown_connections 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.385+0000 7f85009bd740 1 -- 172.21.15.105:0/4019065280 wait complete. 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.385+0000 7f85009bd740 1 -- start start 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.385+0000 7f85009bd740 1 Processor -- start 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f85009bd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36dec1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f85009bd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36dec1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f85009bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 0x55a36dec2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f85009bd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84f81a8130 con 0x55a36deb2c30 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36dec1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f85009bd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84f81a79a0 con 0x55a36deb2240 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36dec1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49860/0 (socket says 172.21.15.105:49860) 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f85009bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84f81a4be0 con 0x55a36deb7da0 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 -- 172.21.15.105:0/3329577769 learned_addr learned my addr 172.21.15.105:0/3329577769 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84effff640 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36dec1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84effff640 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 0x55a36dec2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f85009bd740 10 client.0 _ll_register_callbacks cb 0x7f84e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 -- 172.21.15.105:0/3329577769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 msgr2=0x55a36dec2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 0x55a36dec2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 -- 172.21.15.105:0/3329577769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 msgr2=0x55a36dec1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36dec1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 -- 172.21.15.105:0/3329577769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36dec2f10 con 0x55a36deb2c30 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84f77fe640 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36dec1960 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f84e802c840 tx=0x7f84e8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14b7167dadac9b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.386+0000 7f84e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f84e6ffd640 1 -- 172.21.15.105:0/3329577769 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e804e020 con 0x55a36deb2c30 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 1 -- 172.21.15.105:0/3329577769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e000d3c0 con 0x55a36deb2c30 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f84e6ffd640 1 -- 172.21.15.105:0/3329577769 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f84e8004410 con 0x55a36deb2c30 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f84e6ffd640 1 -- 172.21.15.105:0/3329577769 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f84e804aab0 con 0x55a36deb2c30 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 10 client.20538 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:12.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 1 -- 172.21.15.105:0/3329577769 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a36df2e620 con 0x55a36deb2c30 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f84e6ffd640 1 -- 172.21.15.105:0/3329577769 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84e000d3c0 con 0x55a36deb2c30 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f84e6ffd640 1 -- 172.21.15.105:0/3329577769 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f84e8005390 con 0x55a36deb2c30 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 10 client.20538 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 10 client.20538 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 20 client.20538 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a36df2f050 0x55a36df31470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 4 client.20538 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.387+0000 7f85009bd740 1 -- 172.21.15.105:0/3329577769 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a36de16200 con 0x55a36df2f050 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.388+0000 7f84effff640 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a36df2f050 0x55a36df31470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.388+0000 7f84effff640 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a36df2f050 0x55a36df31470 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ac411a931b74504 server_cookie=4ff85f87b6af4c4b in_seq=0 out_seq=0 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.388+0000 7f84e6ffd640 10 client.20538 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.388+0000 7f84e6ffd640 1 -- 172.21.15.105:0/3329577769 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a36de16200 con 0x55a36df2f050 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.388+0000 7f84e6ffd640 10 client.20538 handle_command_reply: tid=0 2024-03-26T17:52:12.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.388+0000 7f85009bd740 1 client.20538 shutdown 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 -- 172.21.15.105:0/3329577769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 msgr2=0x55a36dec1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36dec1960 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f84e802c840 tx=0x7f84e8005a40 comp rx=0 tx=0).stop 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 -- 172.21.15.105:0/3329577769 shutdown_connections 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a36deb7da0 0x55a36dec2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a36deb2c30 0x55a36dec1960 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a36df2f050 0x55a36df31470 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f84e6ffd640 0 client.20538 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 --2- 172.21.15.105:0/3329577769 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a36deb2240 0x55a36dec1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 -- 172.21.15.105:0/3329577769 >> 172.21.15.105:0/3329577769 conn(0x55a36dea4fa0 msgr2=0x55a36deca420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 -- 172.21.15.105:0/3329577769 shutdown_connections 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 1 -- 172.21.15.105:0/3329577769 wait complete. 2024-03-26T17:52:12.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.389+0000 7f85009bd740 20 client.20538 trim_cache size 0 max 16384 2024-03-26T17:52:12.385 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:12.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 -- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84d80a2f70 msgr2=0x7f84d80a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84d80a2f70 0x7f84d80a5430 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f84f009c000 tx=0x7f84f0009a70 comp rx=0 tx=0).stop 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 -- 172.21.15.105:0/2209528937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f8107610 msgr2=0x7f84f819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f8107610 0x7f84f819cf40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f84f000eb10 tx=0x7f84f000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 -- 172.21.15.105:0/2209528937 shutdown_connections 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84d80a2f70 0x7f84d80a5430 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84f8107610 0x7f84f819cf40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f8104ce0 0x7f84f8195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 --2- 172.21.15.105:0/2209528937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84f81023b0 0x7f84f8195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 -- 172.21.15.105:0/2209528937 >> 172.21.15.105:0/2209528937 conn(0x7f84f80fbd60 msgr2=0x7f84f8102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 -- 172.21.15.105:0/2209528937 shutdown_connections 2024-03-26T17:52:12.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.390+0000 7f84fddda640 1 -- 172.21.15.105:0/2209528937 wait complete. 2024-03-26T17:52:12.395 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:52:12.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 -- 172.21.15.105:0/938025590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec104d20 msgr2=0x7f97ec107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 --2- 172.21.15.105:0/938025590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec104d20 0x7f97ec107110 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009900 tx=0x7f97e002bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:12.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 -- 172.21.15.105:0/938025590 shutdown_connections 2024-03-26T17:52:12.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 --2- 172.21.15.105:0/938025590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec107650 0x7f97ec109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 --2- 172.21.15.105:0/938025590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec104d20 0x7f97ec107110 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 --2- 172.21.15.105:0/938025590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec1023f0 0x7f97ec1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 -- 172.21.15.105:0/938025590 >> 172.21.15.105:0/938025590 conn(0x7f97ec0fbd60 msgr2=0x7f97ec0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 -- 172.21.15.105:0/938025590 shutdown_connections 2024-03-26T17:52:12.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.549+0000 7f97f13f3640 1 -- 172.21.15.105:0/938025590 wait complete. 2024-03-26T17:52:12.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97f13f3640 1 Processor -- start 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97f13f3640 1 -- start start 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97f13f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1023f0 0x7f97ec197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97f13f3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec104d20 0x7f97ec1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97f13f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec107650 0x7f97ec19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97f13f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec10d3d0 con 0x7f97ec1023f0 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97f13f3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec10d250 con 0x7f97ec104d20 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97f13f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec10d550 con 0x7f97ec107650 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97eaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1023f0 0x7f97ec197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97eaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1023f0 0x7f97ec197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49884/0 (socket says 172.21.15.105:49884) 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97eaffd640 1 -- 172.21.15.105:0/3490743709 learned_addr learned my addr 172.21.15.105:0/3490743709 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec107650 0x7f97ec19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.550+0000 7f97ea7fc640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec104d20 0x7f97ec1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97eaffd640 1 -- 172.21.15.105:0/3490743709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec107650 msgr2=0x7f97ec19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97eaffd640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec107650 0x7f97ec19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97eaffd640 1 -- 172.21.15.105:0/3490743709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec104d20 msgr2=0x7f97ec1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97eaffd640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec104d20 0x7f97ec1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97eaffd640 1 -- 172.21.15.105:0/3490743709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec19f790 con 0x7f97ec1023f0 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97eaffd640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1023f0 0x7f97ec197b70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00e9f0 tx=0x7f97dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97d3fff640 1 -- 172.21.15.105:0/3490743709 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00cd80 con 0x7f97ec1023f0 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97f13f3640 1 -- 172.21.15.105:0/3490743709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ec19fa80 con 0x7f97ec1023f0 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97d3fff640 1 -- 172.21.15.105:0/3490743709 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97dc0108e0 con 0x7f97ec1023f0 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97d3fff640 1 -- 172.21.15.105:0/3490743709 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc018920 con 0x7f97ec1023f0 2024-03-26T17:52:12.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.551+0000 7f97f13f3640 1 -- 172.21.15.105:0/3490743709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ec19fff0 con 0x7f97ec1023f0 2024-03-26T17:52:12.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97d3fff640 1 -- 172.21.15.105:0/3490743709 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f97dc010430 con 0x7f97ec1023f0 2024-03-26T17:52:12.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97f3fd9740 1 Processor -- start 2024-03-26T17:52:12.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97f3fd9740 1 -- start start 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f615432a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f61538f970 con 0x55f61542b420 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f615337d70 con 0x55f615430590 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f61538b920 con 0x55f61542aa30 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97ea7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f615432a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46826/0 (socket says 172.21.15.105:46826) 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97eb7fe640 1 -- 172.21.15.105:0/1907767037 learned_addr learned my addr 172.21.15.105:0/1907767037 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97eaffd640 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97ea7fc640 1 -- 172.21.15.105:0/1907767037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 msgr2=0x55f61542ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97ea7fc640 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97ea7fc640 1 -- 172.21.15.105:0/1907767037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 msgr2=0x55f61542ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.555+0000 7f97d3fff640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d40a2fa0 0x7f97d40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97ea7fc640 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97ea7fc640 1 -- 172.21.15.105:0/1907767037 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6150a33f0 con 0x55f615430590 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d40a2fa0 0x7f97d40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97eaffd640 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97d3fff640 1 -- 172.21.15.105:0/3490743709 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97dc0cf6c0 con 0x7f97ec1023f0 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97ea7fc640 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f615432a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f97e0036040 tx=0x7f97e0009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc25cf1e3c185d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97d17fa640 1 -- 172.21.15.105:0/1907767037 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0045020 con 0x55f615430590 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d40a2fa0 0x7f97d40a5460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55f61542bc80 tx=0x7f97e4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97d17fa640 1 -- 172.21.15.105:0/1907767037 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e0004040 con 0x55f615430590 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.556+0000 7f97d17fa640 1 -- 172.21.15.105:0/1907767037 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e00046c0 con 0x55f615430590 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1907767037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 msgr2=0x55f615432a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f615432a20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f97e0036040 tx=0x7f97e0009560 comp rx=0 tx=0).stop 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1907767037 shutdown_connections 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f615432a20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/1907767037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1907767037 >> 172.21.15.105:0/1907767037 conn(0x55f615425fa0 msgr2=0x55f6154263a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1907767037 shutdown_connections 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.557+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1907767037 wait complete. 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 1 -- start start 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 1 Processor -- start 2024-03-26T17:52:12.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f61542f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f6154332c0 con 0x55f61542b420 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f61534b310 con 0x55f615430590 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f615433440 con 0x55f61542aa30 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97ea7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f61542f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97ea7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49902/0 (socket says 172.21.15.105:49902) 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97ea7fc640 1 -- 172.21.15.105:0/807908313 learned_addr learned my addr 172.21.15.105:0/807908313 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97eaffd640 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.558+0000 7f97f3fd9740 10 client.0 _ll_register_callbacks cb 0x55f615386c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:12.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 7485 10.18 MB/sec warmup 55 sec latency 152.263 ms 2024-03-26T17:52:12.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97eb7fe640 1 -- 172.21.15.105:0/807908313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 msgr2=0x55f61542dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97eb7fe640 1 -- 172.21.15.105:0/807908313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 msgr2=0x55f61542e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97eb7fe640 1 -- 172.21.15.105:0/807908313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61516d7d0 con 0x55f615430590 2024-03-26T17:52:12.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f61542f260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f97e400f610 tx=0x7f97e4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a497d7df477861a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97ceff5640 1 -- 172.21.15.105:0/807908313 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4003970 con 0x55f615430590 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97f3fd9740 1 -- 172.21.15.105:0/807908313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e0004040 con 0x55f615430590 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97ceff5640 1 -- 172.21.15.105:0/807908313 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97e4017030 con 0x55f615430590 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97ceff5640 1 -- 172.21.15.105:0/807908313 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f97e4003b10 con 0x55f615430590 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97f3fd9740 10 client.30335 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.559+0000 7f97f3fd9740 1 -- 172.21.15.105:0/807908313 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f615448130 con 0x55f615430590 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97ceff5640 1 -- 172.21.15.105:0/807908313 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e0004040 con 0x55f615430590 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97ceff5640 1 -- 172.21.15.105:0/807908313 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f97e40042c0 con 0x55f615430590 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97f3fd9740 10 client.30335 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97f3fd9740 10 client.30335 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97f3fd9740 20 client.30335 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f615449c20 0x55f61544c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97f3fd9740 4 client.30335 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:12.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97f3fd9740 1 -- 172.21.15.105:0/807908313 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f61539ceb0 con 0x55f615449c20 2024-03-26T17:52:12.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.560+0000 7f97eaffd640 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f615449c20 0x55f61544c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.561+0000 7f97ceff5640 10 client.30335 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:52:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.565+0000 7f97eaffd640 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f615449c20 0x55f61544c080 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9526a050e1362ef3 server_cookie=17a521fa561f5bd1 in_seq=0 out_seq=0 2024-03-26T17:52:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.565+0000 7f97ceff5640 1 -- 172.21.15.105:0/807908313 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f61539ceb0 con 0x55f615449c20 2024-03-26T17:52:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.565+0000 7f97ceff5640 10 client.30335 handle_command_reply: tid=0 2024-03-26T17:52:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 client.30335 shutdown 2024-03-26T17:52:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 -- 172.21.15.105:0/807908313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 msgr2=0x55f61542f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f61542f260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f97e400f610 tx=0x7f97e4004ab0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 -- 172.21.15.105:0/807908313 shutdown_connections 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f615449c20 0x55f61544c080 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f615430590 0x55f61542f260 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97ceff5640 0 client.30335 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542b420 0x55f61542e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/807908313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f61542dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 -- 172.21.15.105:0/807908313 >> 172.21.15.105:0/807908313 conn(0x55f615425fa0 msgr2=0x55f6154251c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 -- 172.21.15.105:0/807908313 shutdown_connections 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 1 -- 172.21.15.105:0/807908313 wait complete. 2024-03-26T17:52:12.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.566+0000 7f97f3fd9740 20 client.30335 trim_cache size 0 max 16384 2024-03-26T17:52:12.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.585+0000 7f97f3fd9740 1 Processor -- start 2024-03-26T17:52:12.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.585+0000 7f97f3fd9740 1 -- start start 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.585+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542aa30 0x55f615434360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.585+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f6154348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.585+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f615430590 0x55f61542c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.585+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a6bf0 con 0x55f61542aa30 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.585+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a87f0 con 0x55f61542b420 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.585+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a72c0 con 0x55f615430590 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f6154348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f6154348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48724/0 (socket says 172.21.15.105:48724) 2024-03-26T17:52:12.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 -- 172.21.15.105:0/1272078110 learned_addr learned my addr 172.21.15.105:0/1272078110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97eaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542aa30 0x55f615434360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f615430590 0x55f61542c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 -- 172.21.15.105:0/1272078110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f615430590 msgr2=0x55f61542c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f615430590 0x55f61542c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 -- 172.21.15.105:0/1272078110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542aa30 msgr2=0x55f615434360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542aa30 0x55f615434360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 -- 172.21.15.105:0/1272078110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f615421a10 con 0x55f61542b420 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f615430590 0x55f61542c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97ea7fc640 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f6154348a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f97e002cb20 tx=0x7f97e002cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498a868324e13ce6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97d0ff9640 1 -- 172.21.15.105:0/1272078110 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e004f070 con 0x55f61542b420 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97d0ff9640 1 -- 172.21.15.105:0/1272078110 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e0005520 con 0x55f61542b420 2024-03-26T17:52:12.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.586+0000 7f97d0ff9640 1 -- 172.21.15.105:0/1272078110 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0045040 con 0x55f61542b420 2024-03-26T17:52:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1272078110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 msgr2=0x55f6154348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f6154348a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f97e002cb20 tx=0x7f97e002cb50 comp rx=0 tx=0).stop 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1272078110 shutdown_connections 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f615430590 0x55f61542c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f6154348a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/1272078110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f61542aa30 0x55f615434360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1272078110 >> 172.21.15.105:0/1272078110 conn(0x55f615425fa0 msgr2=0x55f61542dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1272078110 shutdown_connections 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.587+0000 7f97f3fd9740 1 -- 172.21.15.105:0/1272078110 wait complete. 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 1 -- start start 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 1 Processor -- start 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f615445d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f615442830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f615430590 0x55f615442dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a8120 con 0x55f615430590 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a7990 con 0x55f61542b420 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a4bd0 con 0x55f61542aa30 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97f3fd9740 10 client.0 _ll_register_callbacks cb 0x7f97e4009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97ea7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f615442830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97ea7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f615442830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48728/0 (socket says 172.21.15.105:48728) 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97ea7fc640 1 -- 172.21.15.105:0/828305137 learned_addr learned my addr 172.21.15.105:0/828305137 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f615430590 0x55f615442dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.588+0000 7f97eaffd640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f615445d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97eb7fe640 1 -- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 msgr2=0x55f615445d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f615445d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97eb7fe640 1 -- 172.21.15.105:0/828305137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 msgr2=0x55f615442830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f615442830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97eb7fe640 1 -- 172.21.15.105:0/828305137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6154adfe0 con 0x55f615430590 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97ea7fc640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f615442830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97eaffd640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f615445d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97eb7fe640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f615430590 0x55f615442dd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f97e4000f80 tx=0x7f97e4027690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de51a5188b18f8c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97ceff5640 1 -- 172.21.15.105:0/828305137 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4036040 con 0x55f615430590 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97ceff5640 1 -- 172.21.15.105:0/828305137 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97e401f240 con 0x55f615430590 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97f3fd9740 1 -- 172.21.15.105:0/828305137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e0005520 con 0x55f615430590 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97ceff5640 1 -- 172.21.15.105:0/828305137 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f97e40113f0 con 0x55f615430590 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97f3fd9740 10 client.20556 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:12.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.589+0000 7f97f3fd9740 1 -- 172.21.15.105:0/828305137 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f6154af310 con 0x55f615430590 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.590+0000 7f97ceff5640 1 -- 172.21.15.105:0/828305137 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e0005520 con 0x55f615430590 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.590+0000 7f97ceff5640 1 -- 172.21.15.105:0/828305137 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f97e4027aa0 con 0x55f615430590 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.590+0000 7f97f3fd9740 10 client.20556 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.590+0000 7f97f3fd9740 10 client.20556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.590+0000 7f97f3fd9740 20 client.20556 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.590+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f6154afc90 0x55f6154b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.590+0000 7f97f3fd9740 4 client.20556 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.590+0000 7f97f3fd9740 1 -- 172.21.15.105:0/828305137 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f615397200 con 0x55f6154afc90 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.591+0000 7f97eaffd640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f6154afc90 0x55f6154b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.591+0000 7f97ceff5640 10 client.20556 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.591+0000 7f97eaffd640 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f6154afc90 0x55f6154b20b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19124bbc9c0bf701 server_cookie=3a1939423398539e in_seq=0 out_seq=0 2024-03-26T17:52:12.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97ceff5640 1 -- 172.21.15.105:0/828305137 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f615397200 con 0x55f6154afc90 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97ceff5640 10 client.20556 handle_command_reply: tid=0 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 client.20556 shutdown 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 -- 172.21.15.105:0/828305137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f615430590 msgr2=0x55f615442dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f615430590 0x55f615442dd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f97e4000f80 tx=0x7f97e4027690 comp rx=0 tx=0).stop 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 -- 172.21.15.105:0/828305137 shutdown_connections 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f6154afc90 0x55f6154b20b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f615430590 0x55f615442dd0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f61542b420 0x55f615442830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 --2- 172.21.15.105:0/828305137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f61542aa30 0x55f615445d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 -- 172.21.15.105:0/828305137 >> 172.21.15.105:0/828305137 conn(0x55f615425fa0 msgr2=0x55f61542dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97ceff5640 0 client.20556 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 -- 172.21.15.105:0/828305137 shutdown_connections 2024-03-26T17:52:12.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 1 -- 172.21.15.105:0/828305137 wait complete. 2024-03-26T17:52:12.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.592+0000 7f97f3fd9740 20 client.20556 trim_cache size 0 max 16384 2024-03-26T17:52:12.590 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 -- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d40a2fa0 msgr2=0x7f97d40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d40a2fa0 0x7f97d40a5460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55f61542bc80 tx=0x7f97e4009500 comp rx=0 tx=0).stop 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 -- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1023f0 msgr2=0x7f97ec197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1023f0 0x7f97ec197b70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00e9f0 tx=0x7f97dc00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 -- 172.21.15.105:0/3490743709 shutdown_connections 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d40a2fa0 0x7f97d40a5460 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec107650 0x7f97ec19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec104d20 0x7f97ec1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 --2- 172.21.15.105:0/3490743709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1023f0 0x7f97ec197b70 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 -- 172.21.15.105:0/3490743709 >> 172.21.15.105:0/3490743709 conn(0x7f97ec0fbd60 msgr2=0x7f97ec104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 -- 172.21.15.105:0/3490743709 shutdown_connections 2024-03-26T17:52:12.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.593+0000 7f97f13f3640 1 -- 172.21.15.105:0/3490743709 wait complete. 2024-03-26T17:52:12.598 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:52:12.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 -- 172.21.15.105:0/865730352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b58101a10 msgr2=0x7f0b58101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 --2- 172.21.15.105:0/865730352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b58101a10 0x7f0b58101e90 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009900 tx=0x7f0b4c02bc40 comp rx=0 tx=0).stop 2024-03-26T17:52:12.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 -- 172.21.15.105:0/865730352 shutdown_connections 2024-03-26T17:52:12.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 --2- 172.21.15.105:0/865730352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b581023d0 0x7f0b58108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 --2- 172.21.15.105:0/865730352 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b58101a10 0x7f0b58101e90 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 --2- 172.21.15.105:0/865730352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b58100d00 0x7f0b58101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 -- 172.21.15.105:0/865730352 >> 172.21.15.105:0/865730352 conn(0x7f0b580fb4d0 msgr2=0x7f0b580fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 -- 172.21.15.105:0/865730352 shutdown_connections 2024-03-26T17:52:12.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.745+0000 7f0b5f959640 1 -- 172.21.15.105:0/865730352 wait complete. 2024-03-26T17:52:12.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5f959640 1 Processor -- start 2024-03-26T17:52:12.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5f959640 1 -- start start 2024-03-26T17:52:12.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5f959640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b58100d00 0x7f0b5819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5f959640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b58101a10 0x7f0b5819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5f959640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b581023d0 0x7f0b581a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5f959640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b58100150 con 0x7f0b581023d0 2024-03-26T17:52:12.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5f959640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b580fffd0 con 0x7f0b58100d00 2024-03-26T17:52:12.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5f959640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b581002d0 con 0x7f0b58101a10 2024-03-26T17:52:12.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5d6f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b58100d00 0x7f0b5819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5def6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b581023d0 0x7f0b581a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5def6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b581023d0 0x7f0b581a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49942/0 (socket says 172.21.15.105:49942) 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5d6f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b58100d00 0x7f0b5819e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48744/0 (socket says 172.21.15.105:48744) 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.746+0000 7f0b5def6640 1 -- 172.21.15.105:0/3772122964 learned_addr learned my addr 172.21.15.105:0/3772122964 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b58101a10 0x7f0b5819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5def6640 1 -- 172.21.15.105:0/3772122964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b58101a10 msgr2=0x7f0b5819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5def6640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b58101a10 0x7f0b5819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5def6640 1 -- 172.21.15.105:0/3772122964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b58100d00 msgr2=0x7f0b5819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5def6640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b58100d00 0x7f0b5819e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5def6640 1 -- 172.21.15.105:0/3772122964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b581a5cb0 con 0x7f0b581023d0 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b58101a10 0x7f0b5819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b58100d00 0x7f0b5819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.747+0000 7f0b5def6640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b581023d0 0x7f0b581a5640 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009fd0 tx=0x7f0b4800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.748+0000 7f0b567fc640 1 -- 172.21.15.105:0/3772122964 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48017070 con 0x7f0b581023d0 2024-03-26T17:52:12.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.748+0000 7f0b567fc640 1 -- 172.21.15.105:0/3772122964 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b48003e20 con 0x7f0b581023d0 2024-03-26T17:52:12.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.748+0000 7f0b567fc640 1 -- 172.21.15.105:0/3772122964 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48010460 con 0x7f0b581023d0 2024-03-26T17:52:12.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.748+0000 7f0b5f959640 1 -- 172.21.15.105:0/3772122964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b581a5fa0 con 0x7f0b581023d0 2024-03-26T17:52:12.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.748+0000 7f0b5f959640 1 -- 172.21.15.105:0/3772122964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b581a6410 con 0x7f0b581023d0 2024-03-26T17:52:12.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.751+0000 7f0b567fc640 1 -- 172.21.15.105:0/3772122964 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0b48003970 con 0x7f0b581023d0 2024-03-26T17:52:12.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.751+0000 7f0b62615740 1 Processor -- start 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.751+0000 7f0b62615740 1 -- start start 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.751+0000 7f0b62615740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c70ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.751+0000 7f0b62615740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c713f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.751+0000 7f0b62615740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c716a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b62615740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56044c673970 con 0x56044c70f420 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b62615740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56044c61bd70 con 0x56044c70ea30 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b62615740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56044c66f920 con 0x56044c714590 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c713f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5cef4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c70ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c713f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49956/0 (socket says 172.21.15.105:49956) 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5cef4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c70ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48758/0 (socket says 172.21.15.105:48758) 2024-03-26T17:52:12.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/1886881447 learned_addr learned my addr 172.21.15.105:0/1886881447 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c716a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/1886881447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 msgr2=0x56044c716a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c716a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/1886881447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 msgr2=0x56044c70ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c70ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/1886881447 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044c3873f0 con 0x56044c70f420 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c70ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b567fc640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b400a2fa0 0x7f0b400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c713f20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000f440 tx=0x7f0b5000f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e768a38d71a5d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.752+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b400a2fa0 0x7f0b400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.753+0000 7f0b567fc640 1 -- 172.21.15.105:0/3772122964 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0b480cf160 con 0x7f0b581023d0 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.753+0000 7f0b3f7fe640 1 -- 172.21.15.105:0/1886881447 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50004230 con 0x56044c70f420 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.753+0000 7f0b3f7fe640 1 -- 172.21.15.105:0/1886881447 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b500043d0 con 0x56044c70f420 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.753+0000 7f0b3f7fe640 1 -- 172.21.15.105:0/1886881447 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50010760 con 0x56044c70f420 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 -- 172.21.15.105:0/1886881447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 msgr2=0x56044c713f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c713f20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000f440 tx=0x7f0b5000f900 comp rx=0 tx=0).stop 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b400a2fa0 0x7f0b400a5460 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c000c00 tx=0x7f0b4c02c620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 -- 172.21.15.105:0/1886881447 shutdown_connections 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c716a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c713f20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 --2- 172.21.15.105:0/1886881447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c70ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 -- 172.21.15.105:0/1886881447 >> 172.21.15.105:0/1886881447 conn(0x56044c709fa0 msgr2=0x56044c70a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 -- 172.21.15.105:0/1886881447 shutdown_connections 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 -- 172.21.15.105:0/1886881447 wait complete. 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 -- start start 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.754+0000 7f0b62615740 1 Processor -- start 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b62615740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c718ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b62615740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c719420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b62615740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c720460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b62615740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56044c717540 con 0x56044c70f420 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b62615740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56044c62f310 con 0x56044c70ea30 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b62615740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56044c7176c0 con 0x56044c714590 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b62615740 10 client.0 _ll_register_callbacks cb 0x56044c66ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5d6f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c718ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5d6f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c718ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48770/0 (socket says 172.21.15.105:48770) 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/1386980168 learned_addr learned my addr 172.21.15.105:0/1386980168 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c719420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5def6640 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c720460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/1386980168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 msgr2=0x56044c720460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c720460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/1386980168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 msgr2=0x56044c719420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c719420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.755+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/1386980168 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044c4517d0 con 0x56044c70ea30 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.756+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c718ee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0b50005850 tx=0x7f0b50010f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b883c86245a51e6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.756+0000 7f0b3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.756+0000 7f0b3cff9640 1 -- 172.21.15.105:0/1386980168 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b50018440 con 0x56044c70ea30 2024-03-26T17:52:12.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.756+0000 7f0b3cff9640 1 -- 172.21.15.105:0/1386980168 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0b500185e0 con 0x56044c70ea30 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.756+0000 7f0b62615740 1 -- 172.21.15.105:0/1386980168 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b500043d0 con 0x56044c70ea30 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.756+0000 7f0b3cff9640 1 -- 172.21.15.105:0/1386980168 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0b500187e0 con 0x56044c70ea30 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.756+0000 7f0b62615740 10 client.30353 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.756+0000 7f0b62615740 1 -- 172.21.15.105:0/1386980168 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56044c723820 con 0x56044c70ea30 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b3cff9640 1 -- 172.21.15.105:0/1386980168 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b500043d0 con 0x56044c70ea30 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b3cff9640 1 -- 172.21.15.105:0/1386980168 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0b500179e0 con 0x56044c70ea30 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b62615740 10 client.30353 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b62615740 10 client.30353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b62615740 20 client.30353 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b62615740 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56044c724c20 0x56044c727050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b62615740 4 client.30353 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:12.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b62615740 1 -- 172.21.15.105:0/1386980168 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56044c680eb0 con 0x56044c724c20 2024-03-26T17:52:12.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.757+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56044c724c20 0x56044c727050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.758+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56044c724c20 0x56044c727050 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac2287256d5155b server_cookie=3766e6e5abe72c9d in_seq=0 out_seq=0 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.758+0000 7f0b3cff9640 10 client.30353 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b3cff9640 1 -- 172.21.15.105:0/1386980168 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56044c680eb0 con 0x56044c724c20 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b3cff9640 10 client.30353 handle_command_reply: tid=0 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 client.30353 shutdown 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 -- 172.21.15.105:0/1386980168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 msgr2=0x56044c718ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c718ee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0b50005850 tx=0x7f0b50010f20 comp rx=0 tx=0).stop 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 -- 172.21.15.105:0/1386980168 shutdown_connections 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56044c724c20 0x56044c727050 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c720460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b3cff9640 0 client.30353 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c719420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 --2- 172.21.15.105:0/1386980168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c718ee0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 -- 172.21.15.105:0/1386980168 >> 172.21.15.105:0/1386980168 conn(0x56044c709fa0 msgr2=0x56044c7091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 -- 172.21.15.105:0/1386980168 shutdown_connections 2024-03-26T17:52:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 1 -- 172.21.15.105:0/1386980168 wait complete. 2024-03-26T17:52:12.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.760+0000 7f0b62615740 20 client.30353 trim_cache size 0 max 16384 2024-03-26T17:52:12.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.784+0000 7f0b62615740 1 Processor -- start 2024-03-26T17:52:12.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.784+0000 7f0b62615740 1 -- start start 2024-03-26T17:52:12.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.784+0000 7f0b62615740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70ea30 0x56044c718500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.784+0000 7f0b62615740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c70f420 0x56044c720420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.784+0000 7f0b62615740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c714590 0x56044c718a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.784+0000 7f0b62615740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b581af300 con 0x56044c70ea30 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.784+0000 7f0b62615740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b581b0f00 con 0x56044c714590 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.784+0000 7f0b62615740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b581af9d0 con 0x56044c70f420 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5d6f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70ea30 0x56044c718500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5cef4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c70f420 0x56044c720420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5d6f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70ea30 0x56044c718500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49972/0 (socket says 172.21.15.105:49972) 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5d6f5640 1 -- 172.21.15.105:0/447859154 learned_addr learned my addr 172.21.15.105:0/447859154 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5def6640 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c714590 0x56044c718a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5cef4640 1 -- 172.21.15.105:0/447859154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c714590 msgr2=0x56044c718a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c714590 0x56044c718a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5cef4640 1 -- 172.21.15.105:0/447859154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70ea30 msgr2=0x56044c718500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70ea30 0x56044c718500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5cef4640 1 -- 172.21.15.105:0/447859154 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044c70ad10 con 0x56044c70f420 2024-03-26T17:52:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70ea30 0x56044c718500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c70f420 0x56044c720420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c02c180 tx=0x7f0b4c040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd9712f79c875bd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.785+0000 7f0b3effd640 1 -- 172.21.15.105:0/447859154 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c005600 con 0x56044c70f420 2024-03-26T17:52:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.786+0000 7f0b3effd640 1 -- 172.21.15.105:0/447859154 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b4c004c10 con 0x56044c70f420 2024-03-26T17:52:12.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.786+0000 7f0b3effd640 1 -- 172.21.15.105:0/447859154 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c044da0 con 0x56044c70f420 2024-03-26T17:52:12.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.786+0000 7f0b62615740 1 -- 172.21.15.105:0/447859154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c70f420 msgr2=0x56044c720420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.786+0000 7f0b62615740 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c70f420 0x56044c720420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c02c180 tx=0x7f0b4c040000 comp rx=0 tx=0).stop 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.786+0000 7f0b62615740 1 -- 172.21.15.105:0/447859154 shutdown_connections 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.786+0000 7f0b62615740 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c714590 0x56044c718a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.786+0000 7f0b62615740 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c70f420 0x56044c720420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.786+0000 7f0b62615740 1 --2- 172.21.15.105:0/447859154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70ea30 0x56044c718500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.787+0000 7f0b62615740 1 -- 172.21.15.105:0/447859154 >> 172.21.15.105:0/447859154 conn(0x56044c709fa0 msgr2=0x56044c70da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.787+0000 7f0b62615740 1 -- 172.21.15.105:0/447859154 shutdown_connections 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.787+0000 7f0b62615740 1 -- 172.21.15.105:0/447859154 wait complete. 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.787+0000 7f0b62615740 1 -- start start 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.787+0000 7f0b62615740 1 Processor -- start 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b62615740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c71d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b62615740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c71de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5cef4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c71de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5cef4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c71de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49974/0 (socket says 172.21.15.105:49974) 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b62615740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c71edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5cef4640 1 -- 172.21.15.105:0/3437175746 learned_addr learned my addr 172.21.15.105:0/3437175746 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b581b0830 con 0x56044c70f420 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b581b00a0 con 0x56044c70ea30 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b581ad2e0 con 0x56044c714590 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c71d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5def6640 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c71edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5cef4640 1 -- 172.21.15.105:0/3437175746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 msgr2=0x56044c71edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c71edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5cef4640 1 -- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 msgr2=0x56044c71d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c71d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5cef4640 1 -- 172.21.15.105:0/3437175746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044c71f3b0 con 0x56044c70f420 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c71d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b62615740 10 client.0 _ll_register_callbacks cb 0x7f0b5000aeb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.788+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b4c004c10 con 0x56044c70f420 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b5cef4640 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c71de00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c036040 tx=0x7f0b4c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea77fdfdadabe66b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b3cff9640 10 client.20574 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b3cff9640 1 -- 172.21.15.105:0/3437175746 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c05b030 con 0x56044c70f420 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b3cff9640 1 -- 172.21.15.105:0/3437175746 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0b4c056040 con 0x56044c70f420 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b3cff9640 1 -- 172.21.15.105:0/3437175746 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0b4c044d20 con 0x56044c70f420 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b62615740 10 client.20574 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:12.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=34}) -- 0x56044c7205b0 con 0x56044c70f420 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b3cff9640 1 -- 172.21.15.105:0/3437175746 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0b4c044460 con 0x56044c70f420 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b62615740 10 client.20574 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b62615740 10 client.20574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b62615740 20 client.20574 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.789+0000 7f0b62615740 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56044c724c20 0x56044c78d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.790+0000 7f0b62615740 4 client.20574 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.790+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56044c67b200 con 0x56044c724c20 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.790+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56044c724c20 0x56044c78d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.790+0000 7f0b3cff9640 10 client.20574 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:12.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.790+0000 7f0b5d6f5640 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56044c724c20 0x56044c78d5e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1eadd437d20334a server_cookie=9d0b91c5d0f2e1de in_seq=0 out_seq=0 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.791+0000 7f0b3cff9640 1 -- 172.21.15.105:0/3437175746 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56044c67b200 con 0x56044c724c20 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.791+0000 7f0b3cff9640 10 client.20574 handle_command_reply: tid=0 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.791+0000 7f0b62615740 1 client.20574 shutdown 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 msgr2=0x56044c71de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c71de00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c036040 tx=0x7f0b4c009300 comp rx=0 tx=0).stop 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 shutdown_connections 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56044c724c20 0x56044c78d5e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56044c714590 0x56044c71edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56044c70f420 0x56044c71de00 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 --2- 172.21.15.105:0/3437175746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56044c70ea30 0x56044c71d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 >> 172.21.15.105:0/3437175746 conn(0x56044c709fa0 msgr2=0x56044c708930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b3cff9640 0 client.20574 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 shutdown_connections 2024-03-26T17:52:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 1 -- 172.21.15.105:0/3437175746 wait complete. 2024-03-26T17:52:12.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.792+0000 7f0b62615740 20 client.20574 trim_cache size 0 max 16384 2024-03-26T17:52:12.788 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 -- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b400a2fa0 msgr2=0x7f0b400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b400a2fa0 0x7f0b400a5460 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c000c00 tx=0x7f0b4c02c620 comp rx=0 tx=0).stop 2024-03-26T17:52:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 -- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b581023d0 msgr2=0x7f0b581a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:12.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b581023d0 0x7f0b581a5640 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009fd0 tx=0x7f0b4800b260 comp rx=0 tx=0).stop 2024-03-26T17:52:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 -- 172.21.15.105:0/3772122964 shutdown_connections 2024-03-26T17:52:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0b400a2fa0 0x7f0b400a5460 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b581023d0 0x7f0b581a5640 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b58101a10 0x7f0b5819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 --2- 172.21.15.105:0/3772122964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b58100d00 0x7f0b5819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 -- 172.21.15.105:0/3772122964 >> 172.21.15.105:0/3772122964 conn(0x7f0b580fb4d0 msgr2=0x7f0b58102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 -- 172.21.15.105:0/3772122964 shutdown_connections 2024-03-26T17:52:12.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:12.793+0000 7f0b5f959640 1 -- 172.21.15.105:0/3772122964 wait complete. 2024-03-26T17:52:13.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 7603 10.09 MB/sec warmup 56 sec latency 126.783 ms 2024-03-26T17:52:13.798 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:52:13.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:52:13.799 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:52:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 -- 172.21.15.105:0/4068591408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48241019b0 msgr2=0x7f4824101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 --2- 172.21.15.105:0/4068591408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48241019b0 0x7f4824101e30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f4814009900 tx=0x7f481402bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 -- 172.21.15.105:0/4068591408 shutdown_connections 2024-03-26T17:52:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 --2- 172.21.15.105:0/4068591408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4824102370 0x7f4824108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 --2- 172.21.15.105:0/4068591408 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f48241019b0 0x7f4824101e30 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 --2- 172.21.15.105:0/4068591408 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48241007c0 0x7f4824100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 -- 172.21.15.105:0/4068591408 >> 172.21.15.105:0/4068591408 conn(0x7f48240fbd50 msgr2=0x7f48240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 -- 172.21.15.105:0/4068591408 shutdown_connections 2024-03-26T17:52:13.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.945+0000 7f482ae30640 1 -- 172.21.15.105:0/4068591408 wait complete. 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f482ae30640 1 Processor -- start 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f482ae30640 1 -- start start 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f482ae30640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48241007c0 0x7f4824197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f482ae30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102370 0x7f4824198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f482ae30640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482419d2b0 0x7f482419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f482ae30640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f482410c2f0 con 0x7f4824102370 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f482ae30640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f482410c170 con 0x7f48241007c0 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f482ae30640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f482410c470 con 0x7f482419d2b0 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f4828bcc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48241007c0 0x7f4824197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f4828bcc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48241007c0 0x7f4824197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48800/0 (socket says 172.21.15.105:48800) 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f4828bcc640 1 -- 172.21.15.105:0/832588129 learned_addr learned my addr 172.21.15.105:0/832588129 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f4823fff640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102370 0x7f4824198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.946+0000 7f48293cd640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482419d2b0 0x7f482419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4828bcc640 1 -- 172.21.15.105:0/832588129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482419d2b0 msgr2=0x7f482419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4828bcc640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482419d2b0 0x7f482419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4828bcc640 1 -- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102370 msgr2=0x7f4824198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4828bcc640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102370 0x7f4824198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4828bcc640 1 -- 172.21.15.105:0/832588129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482419fd40 con 0x7f48241007c0 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4823fff640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102370 0x7f4824198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f48293cd640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482419d2b0 0x7f482419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4828bcc640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48241007c0 0x7f4824197d50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f481800e9c0 tx=0x7f481800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4821ffb640 1 -- 172.21.15.105:0/832588129 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481800cd80 con 0x7f48241007c0 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f482ae30640 1 -- 172.21.15.105:0/832588129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482419ffd0 con 0x7f48241007c0 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4821ffb640 1 -- 172.21.15.105:0/832588129 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48180035f0 con 0x7f48241007c0 2024-03-26T17:52:13.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.947+0000 7f4821ffb640 1 -- 172.21.15.105:0/832588129 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818010690 con 0x7f48241007c0 2024-03-26T17:52:13.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.948+0000 7f482ae30640 1 -- 172.21.15.105:0/832588129 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48241a04e0 con 0x7f48241007c0 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.950+0000 7f4821ffb640 1 -- 172.21.15.105:0/832588129 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f48180037a0 con 0x7f48241007c0 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.950+0000 7f482da17740 1 Processor -- start 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f482da17740 1 -- start start 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f482da17740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1082530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f482da17740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 0x5561e10855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f482da17740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e1092ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f482da17740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5561e0fe7630 con 0x5561e1082110 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f482da17740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5561e0f8fe10 con 0x5561e1082b00 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f482da17740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5561e0fe35e0 con 0x5561e1085c60 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f48293cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1082530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f4828bcc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 0x5561e10855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f48293cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1082530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49994/0 (socket says 172.21.15.105:49994) 2024-03-26T17:52:13.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f48293cd640 1 -- 172.21.15.105:0/467311811 learned_addr learned my addr 172.21.15.105:0/467311811 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:13.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f4823fff640 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e1092ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f48293cd640 1 -- 172.21.15.105:0/467311811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 msgr2=0x5561e1092ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f48293cd640 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e1092ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f48293cd640 1 -- 172.21.15.105:0/467311811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 msgr2=0x5561e10855f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f48293cd640 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 0x5561e10855f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f48293cd640 1 -- 172.21.15.105:0/467311811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e0fa6740 con 0x5561e1082110 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f4823fff640 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e1092ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.951+0000 7f4821ffb640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f480c0a2f20 0x7f480c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.952+0000 7f4828bcc640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f480c0a2f20 0x7f480c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.952+0000 7f4821ffb640 1 -- 172.21.15.105:0/832588129 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f48180cf800 con 0x7f48241007c0 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.952+0000 7f48293cd640 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1082530 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f481c0117e0 tx=0x7f481c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90321857515112 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.952+0000 7f4828bcc640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f480c0a2f20 0x7f480c0a53e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f48180cd630 tx=0x7f48180d5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:13.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.952+0000 7f480affd640 1 -- 172.21.15.105:0/467311811 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c011310 con 0x5561e1082110 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.952+0000 7f480affd640 1 -- 172.21.15.105:0/467311811 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f481c003e20 con 0x5561e1082110 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.952+0000 7f480affd640 1 -- 172.21.15.105:0/467311811 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c0044f0 con 0x5561e1082110 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 -- 172.21.15.105:0/467311811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 msgr2=0x5561e1082530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1082530 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f481c0117e0 tx=0x7f481c011ca0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 -- 172.21.15.105:0/467311811 shutdown_connections 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e1092ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 0x5561e10855f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 --2- 172.21.15.105:0/467311811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1082530 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 -- 172.21.15.105:0/467311811 >> 172.21.15.105:0/467311811 conn(0x5561e107d700 msgr2=0x5561e107db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:13.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 -- 172.21.15.105:0/467311811 shutdown_connections 2024-03-26T17:52:13.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.953+0000 7f482da17740 1 -- 172.21.15.105:0/467311811 wait complete. 2024-03-26T17:52:13.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.954+0000 7f482da17740 1 -- start start 2024-03-26T17:52:13.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.954+0000 7f482da17740 1 Processor -- start 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.954+0000 7f482da17740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1094eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.954+0000 7f482da17740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082b00 0x5561e10953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f482da17740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1085c60 0x5561e109c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f482da17740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5561e10934d0 con 0x5561e1082110 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f48293cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1094eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082b00 0x5561e10953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082b00 0x5561e10953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46958/0 (socket says 172.21.15.105:46958) 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f48293cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1094eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49998/0 (socket says 172.21.15.105:49998) 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 -- 172.21.15.105:0/3684548452 learned_addr learned my addr 172.21.15.105:0/3684548452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f482da17740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5561e0fa31a0 con 0x5561e1085c60 2024-03-26T17:52:13.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5561e1093650 con 0x5561e1082b00 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f48293cd640 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1085c60 0x5561e109c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f482da17740 10 client.0 _ll_register_callbacks cb 0x5561e0cfac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 -- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1085c60 msgr2=0x5561e109c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1085c60 0x5561e109c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 -- 172.21.15.105:0/3684548452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 msgr2=0x5561e1094eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1094eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 -- 172.21.15.105:0/3684548452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e0dc37e0 con 0x5561e1082b00 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f48293cd640 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1085c60 0x5561e109c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4823fff640 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082b00 0x5561e10953f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4814009230 tx=0x7f4814005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2af74f0af8f0298d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.955+0000 7f4803fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f4803fff640 1 -- 172.21.15.105:0/3684548452 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814046070 con 0x5561e1082b00 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f4803fff640 1 -- 172.21.15.105:0/3684548452 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f48140046c0 con 0x5561e1082b00 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481c003e20 con 0x5561e1082b00 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f4803fff640 1 -- 172.21.15.105:0/3684548452 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4814005d50 con 0x5561e1082b00 2024-03-26T17:52:13.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f482da17740 10 client.30358 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5561e0e23f70 con 0x5561e1082b00 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f4803fff640 1 -- 172.21.15.105:0/3684548452 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f481c003e20 con 0x5561e1082b00 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f4803fff640 1 -- 172.21.15.105:0/3684548452 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4814003710 con 0x5561e1082b00 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f482da17740 10 client.30358 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.956+0000 7f482da17740 10 client.30358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.957+0000 7f482da17740 20 client.30358 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.957+0000 7f482da17740 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5561e10a0ce0 0x5561e10a3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.957+0000 7f482da17740 4 client.30358 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.957+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561e0ff4b70 con 0x5561e10a0ce0 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.957+0000 7f48293cd640 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5561e10a0ce0 0x5561e10a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.957+0000 7f4803fff640 10 client.30358 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.957+0000 7f48293cd640 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5561e10a0ce0 0x5561e10a3180 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f636894694069091 server_cookie=22767dc0989a81cb in_seq=0 out_seq=0 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f4803fff640 1 -- 172.21.15.105:0/3684548452 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5561e0ff4b70 con 0x5561e10a0ce0 2024-03-26T17:52:13.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f4803fff640 10 client.30358 handle_command_reply: tid=0 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 client.30358 shutdown 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082b00 msgr2=0x5561e10953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082b00 0x5561e10953f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4814009230 tx=0x7f4814005270 comp rx=0 tx=0).stop 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 shutdown_connections 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5561e10a0ce0 0x5561e10a3180 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1085c60 0x5561e109c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f4803fff640 0 client.30358 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082b00 0x5561e10953f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 --2- 172.21.15.105:0/3684548452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082110 0x5561e1094eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 >> 172.21.15.105:0/3684548452 conn(0x5561e107d700 msgr2=0x5561e107c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 shutdown_connections 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 1 -- 172.21.15.105:0/3684548452 wait complete. 2024-03-26T17:52:13.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.959+0000 7f482da17740 20 client.30358 trim_cache size 0 max 16384 2024-03-26T17:52:13.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f482da17740 1 Processor -- start 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f482da17740 1 -- start start 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f482da17740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082110 0x5561e107c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f482da17740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 0x5561e109c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f482da17740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1085c60 0x5561e107c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f482da17740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f48241a7100 con 0x5561e1085c60 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f482da17740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48241a8d00 con 0x5561e1082b00 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f482da17740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48241a77d0 con 0x5561e1082110 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f48293cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082110 0x5561e107c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f48293cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082110 0x5561e107c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46974/0 (socket says 172.21.15.105:46974) 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f4823fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 0x5561e109c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f48293cd640 1 -- 172.21.15.105:0/4139132276 learned_addr learned my addr 172.21.15.105:0/4139132276 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.988+0000 7f48293cd640 1 --2- 172.21.15.105:0/4139132276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1085c60 0x5561e107c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f48293cd640 1 -- 172.21.15.105:0/4139132276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 msgr2=0x5561e109c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:13.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f48293cd640 1 --2- 172.21.15.105:0/4139132276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 0x5561e109c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f48293cd640 1 -- 172.21.15.105:0/4139132276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1085c60 msgr2=0x5561e107c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f48293cd640 1 --2- 172.21.15.105:0/4139132276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1085c60 0x5561e107c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f48293cd640 1 -- 172.21.15.105:0/4139132276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561e0cbb170 con 0x5561e1082110 2024-03-26T17:52:13.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f48293cd640 1 --2- 172.21.15.105:0/4139132276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082110 0x5561e107c070 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f481c000f80 tx=0x7f481c01c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48dafc921e8d0988 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:13.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f480a7fc640 1 -- 172.21.15.105:0/4139132276 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c022040 con 0x5561e1082110 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f480a7fc640 1 -- 172.21.15.105:0/4139132276 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f481c00fe50 con 0x5561e1082110 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.989+0000 7f480a7fc640 1 -- 172.21.15.105:0/4139132276 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481c0050a0 con 0x5561e1082110 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 -- 172.21.15.105:0/4139132276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082110 msgr2=0x5561e107c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 --2- 172.21.15.105:0/4139132276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082110 0x5561e107c070 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f481c000f80 tx=0x7f481c01c040 comp rx=0 tx=0).stop 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 -- 172.21.15.105:0/4139132276 shutdown_connections 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 --2- 172.21.15.105:0/4139132276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1085c60 0x5561e107c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 --2- 172.21.15.105:0/4139132276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082b00 0x5561e109c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 --2- 172.21.15.105:0/4139132276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1082110 0x5561e107c070 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 -- 172.21.15.105:0/4139132276 >> 172.21.15.105:0/4139132276 conn(0x5561e107d700 msgr2=0x5561e1094eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 -- 172.21.15.105:0/4139132276 shutdown_connections 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.990+0000 7f482da17740 1 -- 172.21.15.105:0/4139132276 wait complete. 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f482da17740 1 -- start start 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f482da17740 1 Processor -- start 2024-03-26T17:52:13.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f482da17740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082110 0x5561e1099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f482da17740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082b00 0x5561e109a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f482da17740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e109b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f48293cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082110 0x5561e1099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f48293cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082110 0x5561e1099ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48836/0 (socket says 172.21.15.105:48836) 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f48293cd640 1 -- 172.21.15.105:0/3229022032 learned_addr learned my addr 172.21.15.105:0/3229022032 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f482da17740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f48241a8630 con 0x5561e1082b00 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f48241a7ea0 con 0x5561e1082110 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f4823fff640 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082b00 0x5561e109a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.991+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48241a50e0 con 0x5561e1085c60 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f482da17740 10 client.0 _ll_register_callbacks cb 0x7f4814009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f48293cd640 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e109b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f4823fff640 1 -- 172.21.15.105:0/3229022032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 msgr2=0x5561e109b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f4823fff640 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e109b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f4823fff640 1 -- 172.21.15.105:0/3229022032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082110 msgr2=0x5561e1099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f4823fff640 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082110 0x5561e1099ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f4823fff640 1 -- 172.21.15.105:0/3229022032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561e109b9e0 con 0x5561e1082b00 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f4823fff640 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082b00 0x5561e109a3e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f481402c170 tx=0x7f4814004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d0ae9703ad2668 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f4803fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.992+0000 7f4803fff640 1 -- 172.21.15.105:0/3229022032 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814046040 con 0x5561e1082b00 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f4803fff640 1 -- 172.21.15.105:0/3229022032 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f48140037f0 con 0x5561e1082b00 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f481c00fe50 con 0x5561e1082b00 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f4803fff640 1 -- 172.21.15.105:0/3229022032 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4814009520 con 0x5561e1082b00 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 10 client.20592 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:13.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5561e11074d0 con 0x5561e1082b00 2024-03-26T17:52:13.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f4803fff640 1 -- 172.21.15.105:0/3229022032 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f481c00fe50 con 0x5561e1082b00 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f4803fff640 1 -- 172.21.15.105:0/3229022032 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f48140404e0 con 0x5561e1082b00 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 10 client.20592 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 10 client.20592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 20 client.20592 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5561e10a0ce0 0x5561e1109fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 4 client.20592 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.993+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5561e0feeec0 con 0x5561e10a0ce0 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.994+0000 7f48293cd640 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5561e10a0ce0 0x5561e1109fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.994+0000 7f4803fff640 10 client.20592 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:13.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.994+0000 7f48293cd640 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5561e10a0ce0 0x5561e1109fa0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa8d30eaacf2878 server_cookie=224b6d4702b99200 in_seq=0 out_seq=0 2024-03-26T17:52:13.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.997+0000 7f4803fff640 1 -- 172.21.15.105:0/3229022032 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5561e0feeec0 con 0x5561e10a0ce0 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.997+0000 7f4803fff640 10 client.20592 handle_command_reply: tid=0 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 client.20592 shutdown 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082b00 msgr2=0x5561e109a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082b00 0x5561e109a3e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f481402c170 tx=0x7f4814004610 comp rx=0 tx=0).stop 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 shutdown_connections 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5561e10a0ce0 0x5561e1109fa0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5561e1085c60 0x5561e109b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5561e1082b00 0x5561e109a3e0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f4803fff640 0 client.20592 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 --2- 172.21.15.105:0/3229022032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5561e1082110 0x5561e1099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 >> 172.21.15.105:0/3229022032 conn(0x5561e107d700 msgr2=0x5561e107c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 shutdown_connections 2024-03-26T17:52:13.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 1 -- 172.21.15.105:0/3229022032 wait complete. 2024-03-26T17:52:13.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.998+0000 7f482da17740 20 client.20592 trim_cache size 0 max 16384 2024-03-26T17:52:13.991 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:52:13.991 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:52:13.991 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "d25aa312-f06f-464f-b42d-dd9baa917a9d", 2024-03-26T17:52:13.991 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:52:13.991 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:52:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 -- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f480c0a2f20 msgr2=0x7f480c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f480c0a2f20 0x7f480c0a53e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f48180cd630 tx=0x7f48180d5550 comp rx=0 tx=0).stop 2024-03-26T17:52:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 -- 172.21.15.105:0/832588129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48241007c0 msgr2=0x7f4824197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48241007c0 0x7f4824197d50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f481800e9c0 tx=0x7f481800ee80 comp rx=0 tx=0).stop 2024-03-26T17:52:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 -- 172.21.15.105:0/832588129 shutdown_connections 2024-03-26T17:52:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f480c0a2f20 0x7f480c0a53e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f482419d2b0 0x7f482419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102370 0x7f4824198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 --2- 172.21.15.105:0/832588129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f48241007c0 0x7f4824197d50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 -- 172.21.15.105:0/832588129 >> 172.21.15.105:0/832588129 conn(0x7f48240fbd50 msgr2=0x7f48240fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 -- 172.21.15.105:0/832588129 shutdown_connections 2024-03-26T17:52:13.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:13.999+0000 7f482ae30640 1 -- 172.21.15.105:0/832588129 wait complete. 2024-03-26T17:52:14.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:52:14.003 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:52:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.137+0000 7f8191636640 1 -- 172.21.15.105:0/2762188637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c1018f0 msgr2=0x7f818c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.137+0000 7f8191636640 1 --2- 172.21.15.105:0/2762188637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c1018f0 0x7f818c101d10 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f817c009900 tx=0x7f817c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:14.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.137+0000 7f8191636640 1 -- 172.21.15.105:0/2762188637 shutdown_connections 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.137+0000 7f8191636640 1 --2- 172.21.15.105:0/2762188637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f818c1034a0 0x7f818c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.137+0000 7f8191636640 1 --2- 172.21.15.105:0/2762188637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c102ae0 0x7f818c102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.137+0000 7f8191636640 1 --2- 172.21.15.105:0/2762188637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c1018f0 0x7f818c101d10 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.137+0000 7f8191636640 1 -- 172.21.15.105:0/2762188637 >> 172.21.15.105:0/2762188637 conn(0x7f818c0fce60 msgr2=0x7f818c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 -- 172.21.15.105:0/2762188637 shutdown_connections 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 -- 172.21.15.105:0/2762188637 wait complete. 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 Processor -- start 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 -- start start 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f818c1018f0 0x7f818c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c102ae0 0x7f818c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c1034a0 0x7f818c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10d3f0 con 0x7f818c102ae0 2024-03-26T17:52:14.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10d270 con 0x7f818c1018f0 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f8191636640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10d570 con 0x7f818c1034a0 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.138+0000 7f818a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c102ae0 0x7f818c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f818c1018f0 0x7f818c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c102ae0 0x7f818c198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50030/0 (socket says 172.21.15.105:50030) 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818a7fc640 1 -- 172.21.15.105:0/2573857257 learned_addr learned my addr 172.21.15.105:0/2573857257 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818b7fe640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c1034a0 0x7f818c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818a7fc640 1 -- 172.21.15.105:0/2573857257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c1034a0 msgr2=0x7f818c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818a7fc640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c1034a0 0x7f818c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818a7fc640 1 -- 172.21.15.105:0/2573857257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f818c1018f0 msgr2=0x7f818c197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818a7fc640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f818c1018f0 0x7f818c197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818a7fc640 1 -- 172.21.15.105:0/2573857257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c19f750 con 0x7f818c102ae0 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f818a7fc640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c102ae0 0x7f818c198070 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9f0 tx=0x7f818000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f8173fff640 1 -- 172.21.15.105:0/2573857257 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818000cd80 con 0x7f818c102ae0 2024-03-26T17:52:14.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f8173fff640 1 -- 172.21.15.105:0/2573857257 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81800108e0 con 0x7f818c102ae0 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f8191636640 1 -- 172.21.15.105:0/2573857257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c073b50 con 0x7f818c102ae0 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f8173fff640 1 -- 172.21.15.105:0/2573857257 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180018920 con 0x7f818c102ae0 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.139+0000 7f8191636640 1 -- 172.21.15.105:0/2573857257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c0740a0 con 0x7f818c102ae0 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8194223740 1 Processor -- start 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8194223740 1 -- start start 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8173fff640 1 -- 172.21.15.105:0/2573857257 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8180010a90 con 0x7f818c102ae0 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8194223740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a0dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8194223740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a12de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8194223740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 0x560948a158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8194223740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560948972980 con 0x560948a0e2e0 2024-03-26T17:52:14.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8194223740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56094891abe0 con 0x560948a13450 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f8194223740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56094896e930 con 0x560948a0d8f0 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a12de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a0dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a12de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50040/0 (socket says 172.21.15.105:50040) 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 0x560948a158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a0dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47008/0 (socket says 172.21.15.105:47008) 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818affd640 1 -- 172.21.15.105:0/2954213605 learned_addr learned my addr 172.21.15.105:0/2954213605 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818affd640 1 -- 172.21.15.105:0/2954213605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 msgr2=0x560948a0dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818affd640 1 --2- 172.21.15.105:0/2954213605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a0dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818affd640 1 -- 172.21.15.105:0/2954213605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 msgr2=0x560948a158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818affd640 1 --2- 172.21.15.105:0/2954213605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 0x560948a158e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818affd640 1 -- 172.21.15.105:0/2954213605 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609487507d0 con 0x560948a0e2e0 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.142+0000 7f818b7fe640 1 --2- 172.21.15.105:0/2954213605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a0dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.143+0000 7f8173fff640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81740a2f20 0x7f81740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.143+0000 7f818b7fe640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81740a2f20 0x7f81740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.143+0000 7f818affd640 1 --2- 172.21.15.105:0/2954213605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a12de0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f817c009900 tx=0x7f817c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96305a41a82c42ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.143+0000 7f8173fff640 1 -- 172.21.15.105:0/2573857257 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f81800cf700 con 0x7f818c102ae0 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.143+0000 7f81717fa640 1 -- 172.21.15.105:0/2954213605 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c04c070 con 0x560948a0e2e0 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.143+0000 7f81717fa640 1 -- 172.21.15.105:0/2954213605 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f817c004040 con 0x560948a0e2e0 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.143+0000 7f81717fa640 1 -- 172.21.15.105:0/2954213605 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c005490 con 0x560948a0e2e0 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 -- 172.21.15.105:0/2954213605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 msgr2=0x560948a12de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f818b7fe640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81740a2f20 0x7f81740a53e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x560948a0eb40 tx=0x7f8184009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:14.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 --2- 172.21.15.105:0/2954213605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a12de0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f817c009900 tx=0x7f817c009560 comp rx=0 tx=0).stop 2024-03-26T17:52:14.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 -- 172.21.15.105:0/2954213605 shutdown_connections 2024-03-26T17:52:14.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 --2- 172.21.15.105:0/2954213605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 0x560948a158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 --2- 172.21.15.105:0/2954213605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a12de0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 --2- 172.21.15.105:0/2954213605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a0dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 -- 172.21.15.105:0/2954213605 >> 172.21.15.105:0/2954213605 conn(0x560948a08e60 msgr2=0x560948a09260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:14.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 -- 172.21.15.105:0/2954213605 shutdown_connections 2024-03-26T17:52:14.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 -- 172.21.15.105:0/2954213605 wait complete. 2024-03-26T17:52:14.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 -- start start 2024-03-26T17:52:14.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.144+0000 7f8194223740 1 Processor -- start 2024-03-26T17:52:14.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f8194223740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 0x560948a17b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f8194223740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f8194223740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a1f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f8194223740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560948a163a0 con 0x560948a0e2e0 2024-03-26T17:52:14.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f8194223740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56094892e160 con 0x560948a0d8f0 2024-03-26T17:52:14.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f8194223740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560948a16520 con 0x560948a13450 2024-03-26T17:52:14.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f8194223740 10 client.0 _ll_register_callbacks cb 0x56094896a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:14.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50042/0 (socket says 172.21.15.105:50042) 2024-03-26T17:52:14.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 -- 172.21.15.105:0/3376187163 learned_addr learned my addr 172.21.15.105:0/3376187163 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:14.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818affd640 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 0x560948a17b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818b7fe640 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a1f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 -- 172.21.15.105:0/3376187163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 msgr2=0x560948a1f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a1f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 -- 172.21.15.105:0/3376187163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 msgr2=0x560948a17b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 0x560948a17b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 -- 172.21.15.105:0/3376187163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609486863f0 con 0x560948a0e2e0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f818a7fc640 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f818009bba0 tx=0x7f8180098200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28c4ac8e6d53d70 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f816eff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f816eff5640 1 -- 172.21.15.105:0/3376187163 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180010de0 con 0x560948a0e2e0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f816eff5640 1 -- 172.21.15.105:0/3376187163 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f81800cfbb0 con 0x560948a0e2e0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.145+0000 7f8194223740 1 -- 172.21.15.105:0/3376187163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f817c004040 con 0x560948a0e2e0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f816eff5640 1 -- 172.21.15.105:0/3376187163 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f818009bea0 con 0x560948a0e2e0 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f8194223740 10 client.20604 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:14.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f8194223740 1 -- 172.21.15.105:0/3376187163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5609486460f0 con 0x560948a0e2e0 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f816eff5640 1 -- 172.21.15.105:0/3376187163 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f817c004040 con 0x560948a0e2e0 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f816eff5640 1 -- 172.21.15.105:0/3376187163 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f81800bab60 con 0x560948a0e2e0 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f8194223740 10 client.20604 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f8194223740 10 client.20604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f8194223740 20 client.20604 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f8194223740 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560948a23a20 0x560948a25e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f8194223740 4 client.20604 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f8194223740 1 -- 172.21.15.105:0/3376187163 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56094897fec0 con 0x560948a23a20 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.146+0000 7f818affd640 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560948a23a20 0x560948a25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f818affd640 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560948a23a20 0x560948a25e40 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5ba3cfe1533714 server_cookie=72795aa5b078a201 in_seq=0 out_seq=0 2024-03-26T17:52:14.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f816eff5640 10 client.20604 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f816eff5640 1 -- 172.21.15.105:0/3376187163 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56094897fec0 con 0x560948a23a20 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f816eff5640 10 client.20604 handle_command_reply: tid=0 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 client.20604 shutdown 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 -- 172.21.15.105:0/3376187163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 msgr2=0x560948a18080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f818009bba0 tx=0x7f8180098200 comp rx=0 tx=0).stop 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 -- 172.21.15.105:0/3376187163 shutdown_connections 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560948a23a20 0x560948a25e40 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a1f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 --2- 172.21.15.105:0/3376187163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 0x560948a17b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f8194223740 1 -- 172.21.15.105:0/3376187163 >> 172.21.15.105:0/3376187163 conn(0x560948a08e60 msgr2=0x560948a08080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.147+0000 7f816eff5640 0 client.20604 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:14.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.148+0000 7f8194223740 1 -- 172.21.15.105:0/3376187163 shutdown_connections 2024-03-26T17:52:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.148+0000 7f8194223740 1 -- 172.21.15.105:0/3376187163 wait complete. 2024-03-26T17:52:14.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.148+0000 7f8194223740 20 client.20604 trim_cache size 0 max 16384 2024-03-26T17:52:14.152 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:13 smithi105 ceph-mon[32280]: pgmap v765: 97 pgs: 97 active+clean; 222 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.7 MiB/s wr, 244 op/s 2024-03-26T17:52:14.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:13 smithi135 ceph-mon[35636]: pgmap v765: 97 pgs: 97 active+clean; 222 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.7 MiB/s wr, 244 op/s 2024-03-26T17:52:14.164 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:52:14.057+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003c80 [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{1=1,2=1,3=1,4=1} v1333 ap=2 RANDEPHEMERALPIN f(v26 m2024-03-26T17:51:55.699520+0000 11=11+0) n(v3 rc2024-03-26T17:51:55.699520+0000 b2861056 12=11+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@769} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d142580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003c80:[<0x10000003bc0/COREL v50>,<0x10000003bb6/~dmtmp v48>,<0x10000000007/client8 v2078>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003c80:[<0x10000003bc0/COREL v1333>,<0x10000003bb6/~dmtmp v328>,<0x10000000007/client8 v3027>,<0x10000000006/clients v1815>,<0x10000000005/tmp v1767>,<0x10000000003/client.0 v1705>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1667>,<0x10000000001/sv_0 v1681>,<0x10000000000/qa v1625>,<0x1/volumes v1592>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:51:55.653709+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:51:55.699520+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v26 m2024-03-26T17:51:55.699520+0000 11=11+0)","memory_value.rstat":"n(v3 rc2024-03-26T17:51:55.699520+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.179+0000 7f8194223740 1 Processor -- start 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.179+0000 7f8194223740 1 -- start start 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8194223740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a17b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8194223740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8194223740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 0x560948a1f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8194223740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1a6b60 con 0x560948a0e2e0 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8194223740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1a8760 con 0x560948a13450 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8194223740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1a7230 con 0x560948a0d8f0 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50054/0 (socket says 172.21.15.105:50054) 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 -- 172.21.15.105:0/3292730125 learned_addr learned my addr 172.21.15.105:0/3292730125 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:14.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818affd640 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a17b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818b7fe640 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 0x560948a1f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 -- 172.21.15.105:0/3292730125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 msgr2=0x560948a17b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a17b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 -- 172.21.15.105:0/3292730125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 msgr2=0x560948a1f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 0x560948a1f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 -- 172.21.15.105:0/3292730125 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560948a098d0 con 0x560948a0e2e0 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f818a7fc640 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f8180009380 tx=0x7f8180000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49334243e3bb4557 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8170ff9640 1 -- 172.21.15.105:0/3292730125 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800b7070 con 0x560948a0e2e0 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8170ff9640 1 -- 172.21.15.105:0/3292730125 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f818009f070 con 0x560948a0e2e0 2024-03-26T17:52:14.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.180+0000 7f8170ff9640 1 -- 172.21.15.105:0/3292730125 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800a4800 con 0x560948a0e2e0 2024-03-26T17:52:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 -- 172.21.15.105:0/3292730125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 msgr2=0x560948a18080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f8180009380 tx=0x7f8180000bc0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 -- 172.21.15.105:0/3292730125 shutdown_connections 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a13450 0x560948a1f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a18080 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 --2- 172.21.15.105:0/3292730125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a0d8f0 0x560948a17b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 -- 172.21.15.105:0/3292730125 >> 172.21.15.105:0/3292730125 conn(0x560948a08e60 msgr2=0x560948a0f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 -- 172.21.15.105:0/3292730125 shutdown_connections 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.181+0000 7f8194223740 1 -- 172.21.15.105:0/3292730125 wait complete. 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 1 -- start start 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 1 Processor -- start 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 0x560948a1bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a1c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a90580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1a8090 con 0x560948a0e2e0 2024-03-26T17:52:14.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1a7900 con 0x560948a0d8f0 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f818c1a4b40 con 0x560948a13450 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f8194223740 10 client.0 _ll_register_callbacks cb 0x7f818000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f818affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 0x560948a1bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f818b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a90580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f818b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a90580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47038/0 (socket says 172.21.15.105:47038) 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f818b7fe640 1 -- 172.21.15.105:0/1537170603 learned_addr learned my addr 172.21.15.105:0/1537170603 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.182+0000 7f818a7fc640 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a1c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f818b7fe640 1 -- 172.21.15.105:0/1537170603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 msgr2=0x560948a1bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f818b7fe640 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 0x560948a1bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f818b7fe640 1 -- 172.21.15.105:0/1537170603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 msgr2=0x560948a1c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f818b7fe640 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a1c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f818b7fe640 1 -- 172.21.15.105:0/1537170603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560948a90ac0 con 0x560948a13450 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f818b7fe640 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a90580 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8184003ec0 tx=0x7f8184004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551368a1862a8ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:14.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f816eff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f816eff5640 1 -- 172.21.15.105:0/1537170603 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818402e060 con 0x560948a13450 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f8194223740 1 -- 172.21.15.105:0/1537170603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818009f070 con 0x560948a13450 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f816eff5640 1 -- 172.21.15.105:0/1537170603 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f81840045c0 con 0x560948a13450 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f816eff5640 1 -- 172.21.15.105:0/1537170603 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f818400fe60 con 0x560948a13450 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f8194223740 10 client.30382 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f8194223740 1 -- 172.21.15.105:0/1537170603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560948a91e50 con 0x560948a13450 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f816eff5640 1 -- 172.21.15.105:0/1537170603 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f818009f070 con 0x560948a13450 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.183+0000 7f816eff5640 1 -- 172.21.15.105:0/1537170603 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8184034420 con 0x560948a13450 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f8194223740 10 client.30382 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f8194223740 10 client.30382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f8194223740 20 client.30382 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f8194223740 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560948a928d0 0x560948a94cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f8194223740 4 client.30382 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f8194223740 1 -- 172.21.15.105:0/1537170603 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560948a17930 con 0x560948a928d0 2024-03-26T17:52:14.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f818affd640 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560948a928d0 0x560948a94cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f816eff5640 10 client.30382 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.184+0000 7f818affd640 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560948a928d0 0x560948a94cf0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb6bcfac589df4b server_cookie=b65e0075d16b4c5d in_seq=0 out_seq=0 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f816eff5640 1 -- 172.21.15.105:0/1537170603 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x560948a17930 con 0x560948a928d0 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f816eff5640 10 client.30382 handle_command_reply: tid=0 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 client.30382 shutdown 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 -- 172.21.15.105:0/1537170603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 msgr2=0x560948a90580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a90580 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8184003ec0 tx=0x7f8184004a40 comp rx=0 tx=0).stop 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 -- 172.21.15.105:0/1537170603 shutdown_connections 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560948a928d0 0x560948a94cf0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560948a13450 0x560948a90580 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560948a0e2e0 0x560948a1c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 --2- 172.21.15.105:0/1537170603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560948a0d8f0 0x560948a1bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 -- 172.21.15.105:0/1537170603 >> 172.21.15.105:0/1537170603 conn(0x560948a08e60 msgr2=0x560948a0f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:14.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f816eff5640 0 client.30382 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 -- 172.21.15.105:0/1537170603 shutdown_connections 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.186+0000 7f8194223740 1 -- 172.21.15.105:0/1537170603 wait complete. 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8194223740 20 client.30382 trim_cache size 0 max 16384 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (229 inodes in the stack)", 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stdout: "d25aa312-f06f-464f-b42d-dd9baa917a9d": { 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "d25aa312-f06f-464f-b42d-dd9baa917a9d", 2024-03-26T17:52:14.182 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:52:14.183 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:52:14.183 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:52:14.183 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:52:14.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 -- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81740a2f20 msgr2=0x7f81740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81740a2f20 0x7f81740a53e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x560948a0eb40 tx=0x7f8184009500 comp rx=0 tx=0).stop 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 -- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c102ae0 msgr2=0x7f818c198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c102ae0 0x7f818c198070 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9f0 tx=0x7f818000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 -- 172.21.15.105:0/2573857257 shutdown_connections 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f81740a2f20 0x7f81740a53e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f818c1034a0 0x7f818c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f818c102ae0 0x7f818c198070 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 --2- 172.21.15.105:0/2573857257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f818c1018f0 0x7f818c197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 -- 172.21.15.105:0/2573857257 >> 172.21.15.105:0/2573857257 conn(0x7f818c0fce60 msgr2=0x7f818c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 -- 172.21.15.105:0/2573857257 shutdown_connections 2024-03-26T17:52:14.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:14.187+0000 7f8191636640 1 -- 172.21.15.105:0/2573857257 wait complete. 2024-03-26T17:52:14.191 INFO:tasks.cephfs.filesystem:scrub status for tag:d25aa312-f06f-464f-b42d-dd9baa917a9d - {'path': '/', 'tag': 'd25aa312-f06f-464f-b42d-dd9baa917a9d', 'options': 'recursive,force'} 2024-03-26T17:52:14.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:13 smithi144 ceph-mon[36095]: pgmap v765: 97 pgs: 97 active+clean; 222 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.7 MiB/s wr, 244 op/s 2024-03-26T17:52:14.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 7711 10.06 MB/sec warmup 57 sec latency 283.721 ms 2024-03-26T17:52:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:52:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:15.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:52:15.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.165 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.166 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:15.348 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.349 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:15.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 7864 10.14 MB/sec warmup 58 sec latency 188.690 ms 2024-03-26T17:52:16.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:15 smithi105 ceph-mon[32280]: pgmap v766: 97 pgs: 97 active+clean; 226 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.1 MiB/s wr, 243 op/s 2024-03-26T17:52:16.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:15 smithi135 ceph-mon[35636]: pgmap v766: 97 pgs: 97 active+clean; 226 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.1 MiB/s wr, 243 op/s 2024-03-26T17:52:16.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:15 smithi144 ceph-mon[36095]: pgmap v766: 97 pgs: 97 active+clean; 226 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.1 MiB/s wr, 243 op/s 2024-03-26T17:52:16.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 8087 10.28 MB/sec warmup 59 sec latency 133.135 ms 2024-03-26T17:52:17.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 8247 10.30 MB/sec warmup 60 sec latency 128.093 ms 2024-03-26T17:52:18.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:17 smithi105 ceph-mon[32280]: pgmap v767: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.1 MiB/s wr, 268 op/s 2024-03-26T17:52:18.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:17 smithi135 ceph-mon[35636]: pgmap v767: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.1 MiB/s wr, 268 op/s 2024-03-26T17:52:18.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:17 smithi144 ceph-mon[36095]: pgmap v767: 97 pgs: 97 active+clean; 230 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.1 MiB/s wr, 268 op/s 2024-03-26T17:52:18.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 8450 10.37 MB/sec warmup 61 sec latency 121.723 ms 2024-03-26T17:52:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 -- 172.21.15.105:0/1489915977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac101a10 msgr2=0x7f3dac101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 --2- 172.21.15.105:0/1489915977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac101a10 0x7f3dac101e90 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009900 tx=0x7f3d9c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:18.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 -- 172.21.15.105:0/1489915977 shutdown_connections 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 --2- 172.21.15.105:0/1489915977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1023d0 0x7f3dac108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 --2- 172.21.15.105:0/1489915977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac101a10 0x7f3dac101e90 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 --2- 172.21.15.105:0/1489915977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac100820 0x7f3dac100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 -- 172.21.15.105:0/1489915977 >> 172.21.15.105:0/1489915977 conn(0x7f3dac0fbd90 msgr2=0x7f3dac0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 -- 172.21.15.105:0/1489915977 shutdown_connections 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 -- 172.21.15.105:0/1489915977 wait complete. 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 Processor -- start 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.801+0000 7f3db3a12640 1 -- start start 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db3a12640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac100820 0x7f3dac197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db3a12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac101a10 0x7f3dac197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db3a12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1023d0 0x7f3dac19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db3a12640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10c350 con 0x7f3dac1023d0 2024-03-26T17:52:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db3a12640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10c1d0 con 0x7f3dac100820 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db3a12640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10c4d0 con 0x7f3dac101a10 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac100820 0x7f3dac197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac100820 0x7f3dac197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44794/0 (socket says 172.21.15.105:44794) 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db0fad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac101a10 0x7f3dac197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db17ae640 1 -- 172.21.15.105:0/3243844678 learned_addr learned my addr 172.21.15.105:0/3243844678 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db1faf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1023d0 0x7f3dac19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db17ae640 1 -- 172.21.15.105:0/3243844678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac101a10 msgr2=0x7f3dac197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db17ae640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac101a10 0x7f3dac197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db17ae640 1 -- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1023d0 msgr2=0x7f3dac19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db17ae640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1023d0 0x7f3dac19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db17ae640 1 -- 172.21.15.105:0/3243844678 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac19f6a0 con 0x7f3dac100820 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db0fad640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac101a10 0x7f3dac197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:18.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.802+0000 7f3db1faf640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1023d0 0x7f3dac19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.803+0000 7f3db17ae640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac100820 0x7f3dac197a80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3da800e9c0 tx=0x7f3da800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.803+0000 7f3da27fc640 1 -- 172.21.15.105:0/3243844678 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da800cd80 con 0x7f3dac100820 2024-03-26T17:52:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.803+0000 7f3db3a12640 1 -- 172.21.15.105:0/3243844678 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac19f990 con 0x7f3dac100820 2024-03-26T17:52:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.803+0000 7f3da27fc640 1 -- 172.21.15.105:0/3243844678 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da80035f0 con 0x7f3dac100820 2024-03-26T17:52:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.803+0000 7f3da27fc640 1 -- 172.21.15.105:0/3243844678 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8010690 con 0x7f3dac100820 2024-03-26T17:52:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.803+0000 7f3db3a12640 1 -- 172.21.15.105:0/3243844678 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac19feb0 con 0x7f3dac100820 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.805+0000 7f3da27fc640 1 -- 172.21.15.105:0/3243844678 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3da80037a0 con 0x7f3dac100820 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db6604740 1 Processor -- start 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db6604740 1 -- start start 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db6604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db6604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 0x560c918aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db6604740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db6604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560c9180e0b0 con 0x560c918a95d0 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db6604740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560c917b6960 con 0x560c918af130 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db6604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560c9180a060 con 0x560c918a9fc0 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3da27fc640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fc0 0x7f3d980a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 0x560c918aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db1faf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db1faf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918a99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48352/0 (socket says 172.21.15.105:48352) 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db1faf640 1 -- 172.21.15.105:0/1313184311 learned_addr learned my addr 172.21.15.105:0/1313184311 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.806+0000 7f3db0fad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db1faf640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fc0 0x7f3d980a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3da27fc640 1 -- 172.21.15.105:0/3243844678 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3da80cf900 con 0x7f3dac100820 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db0fad640 1 -- 172.21.15.105:0/1313184311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 msgr2=0x560c918aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db0fad640 1 --2- 172.21.15.105:0/1313184311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 0x560c918aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db0fad640 1 -- 172.21.15.105:0/1313184311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 msgr2=0x560c918a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db0fad640 1 --2- 172.21.15.105:0/1313184311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918a99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db0fad640 1 -- 172.21.15.105:0/1313184311 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c917c8c60 con 0x560c918af130 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db1faf640 1 --2- 172.21.15.105:0/1313184311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db0fad640 1 --2- 172.21.15.105:0/1313184311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918b15c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c000c00 tx=0x7f3d9c0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473d4787f35294fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3db1faf640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fc0 0x7f3d980a5480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x560c918aa820 tx=0x7f3da4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3d977fe640 1 -- 172.21.15.105:0/1313184311 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c055070 con 0x560c918af130 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3d977fe640 1 -- 172.21.15.105:0/1313184311 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d9c02cc90 con 0x560c918af130 2024-03-26T17:52:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.807+0000 7f3d977fe640 1 -- 172.21.15.105:0/1313184311 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c004920 con 0x560c918af130 2024-03-26T17:52:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 -- 172.21.15.105:0/1313184311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 msgr2=0x560c918b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 --2- 172.21.15.105:0/1313184311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918b15c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c000c00 tx=0x7f3d9c0095f0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 -- 172.21.15.105:0/1313184311 shutdown_connections 2024-03-26T17:52:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 --2- 172.21.15.105:0/1313184311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918b15c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 --2- 172.21.15.105:0/1313184311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 0x560c918aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 --2- 172.21.15.105:0/1313184311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 -- 172.21.15.105:0/1313184311 >> 172.21.15.105:0/1313184311 conn(0x560c918a4b40 msgr2=0x560c918a4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 -- 172.21.15.105:0/1313184311 shutdown_connections 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.808+0000 7f3db6604740 1 -- 172.21.15.105:0/1313184311 wait complete. 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 1 -- start start 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 1 Processor -- start 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 0x560c918b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918ba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560c918b2080 con 0x560c918a95d0 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560c917c9db0 con 0x560c918af130 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560c918b2200 con 0x560c918a9fc0 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db6604740 10 client.0 _ll_register_callbacks cb 0x560c917ce110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48356/0 (socket says 172.21.15.105:48356) 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db17ae640 1 -- 172.21.15.105:0/1795603867 learned_addr learned my addr 172.21.15.105:0/1795603867 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db1faf640 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.809+0000 7f3db0fad640 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 0x560c918b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db17ae640 1 -- 172.21.15.105:0/1795603867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 msgr2=0x560c918b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db17ae640 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 0x560c918b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db17ae640 1 -- 172.21.15.105:0/1795603867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 msgr2=0x560c918ba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db17ae640 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918ba860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db17ae640 1 -- 172.21.15.105:0/1795603867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c91522210 con 0x560c918a95d0 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db17ae640 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918b3350 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3da809bf80 tx=0x7f3da8003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee76da0bc00296d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1795603867 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da80b3020 con 0x560c918a95d0 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1795603867 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3da8096b60 con 0x560c918a95d0 2024-03-26T17:52:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db6604740 1 -- 172.21.15.105:0/1795603867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d9c02cc90 con 0x560c918a95d0 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1795603867 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3da80a0140 con 0x560c918a95d0 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db6604740 10 client.20622 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3db6604740 1 -- 172.21.15.105:0/1795603867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560c915ee620 con 0x560c918a95d0 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.810+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1795603867 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d9c02cc90 con 0x560c918a95d0 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1795603867 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3da8097d40 con 0x560c918a95d0 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3db6604740 10 client.20622 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3db6604740 10 client.20622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3db6604740 20 client.20622 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3db6604740 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c918bf1b0 0x560c918c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3db6604740 4 client.20622 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3db6604740 1 -- 172.21.15.105:0/1795603867 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c9181b5f0 con 0x560c918bf1b0 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3db0fad640 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c918bf1b0 0x560c918c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3d94ff9640 10 client.20622 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.811+0000 7f3db0fad640 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c918bf1b0 0x560c918c1670 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8aedea35e92de3 server_cookie=5f02906bc6a66b46 in_seq=0 out_seq=0 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1795603867 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560c9181b5f0 con 0x560c918bf1b0 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3d94ff9640 10 client.20622 handle_command_reply: tid=0 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 client.20622 shutdown 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 -- 172.21.15.105:0/1795603867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 msgr2=0x560c918b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918b3350 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3da809bf80 tx=0x7f3da8003b10 comp rx=0 tx=0).stop 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 -- 172.21.15.105:0/1795603867 shutdown_connections 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c918bf1b0 0x560c918c1670 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918af130 0x560c918ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a9fc0 0x560c918b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 --2- 172.21.15.105:0/1795603867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918a95d0 0x560c918b3350 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 -- 172.21.15.105:0/1795603867 >> 172.21.15.105:0/1795603867 conn(0x560c918a4b40 msgr2=0x560c918a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3d94ff9640 0 client.20622 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 -- 172.21.15.105:0/1795603867 shutdown_connections 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 1 -- 172.21.15.105:0/1795603867 wait complete. 2024-03-26T17:52:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.813+0000 7f3db6604740 20 client.20622 trim_cache size 0 max 16384 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db6604740 1 Processor -- start 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db6604740 1 -- start start 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db6604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db6604740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db6604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db6604740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1a0460 con 0x560c918af130 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db6604740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1a2060 con 0x560c918a9fc0 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db6604740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1a0b30 con 0x560c918a95d0 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db1faf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db1faf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48370/0 (socket says 172.21.15.105:48370) 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db0fad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db1faf640 1 -- 172.21.15.105:0/2512438478 learned_addr learned my addr 172.21.15.105:0/2512438478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42066/0 (socket says 172.21.15.105:42066) 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db1faf640 1 -- 172.21.15.105:0/2512438478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 msgr2=0x560c918b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db1faf640 1 --2- 172.21.15.105:0/2512438478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db1faf640 1 -- 172.21.15.105:0/2512438478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 msgr2=0x560c918a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db1faf640 1 --2- 172.21.15.105:0/2512438478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db1faf640 1 -- 172.21.15.105:0/2512438478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c918aa3a0 con 0x560c918af130 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.833+0000 7f3db0fad640 1 --2- 172.21.15.105:0/2512438478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.834+0000 7f3db1faf640 1 --2- 172.21.15.105:0/2512438478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918a3aa0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3da400fee0 tx=0x7f3da400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de512e16ce108c2e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.834+0000 7f3d96ffd640 1 -- 172.21.15.105:0/2512438478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da40045d0 con 0x560c918af130 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.834+0000 7f3d96ffd640 1 -- 172.21.15.105:0/2512438478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da401d040 con 0x560c918af130 2024-03-26T17:52:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.834+0000 7f3d96ffd640 1 -- 172.21.15.105:0/2512438478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4005750 con 0x560c918af130 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 -- 172.21.15.105:0/2512438478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 msgr2=0x560c918a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 --2- 172.21.15.105:0/2512438478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918a3aa0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f3da400fee0 tx=0x7f3da400f3c0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 -- 172.21.15.105:0/2512438478 shutdown_connections 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 --2- 172.21.15.105:0/2512438478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918a3aa0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 --2- 172.21.15.105:0/2512438478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 --2- 172.21.15.105:0/2512438478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 -- 172.21.15.105:0/2512438478 >> 172.21.15.105:0/2512438478 conn(0x560c918a4b40 msgr2=0x560c918b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 -- 172.21.15.105:0/2512438478 shutdown_connections 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 -- 172.21.15.105:0/2512438478 wait complete. 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 -- start start 2024-03-26T17:52:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.835+0000 7f3db6604740 1 Processor -- start 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db6604740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db6604740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db6604740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db0fad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db1faf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db17ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42082/0 (socket says 172.21.15.105:42082) 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db1faf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918b9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48372/0 (socket says 172.21.15.105:48372) 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db0fad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918b8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:44840/0 (socket says 172.21.15.105:44840) 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db17ae640 1 -- 172.21.15.105:0/328947813 learned_addr learned my addr 172.21.15.105:0/328947813 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1a1990 con 0x560c918af130 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1a1200 con 0x560c918a9fc0 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac071ea0 con 0x560c918a95d0 2024-03-26T17:52:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db6604740 10 client.0 _ll_register_callbacks cb 0x7f3da809be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db17ae640 1 -- 172.21.15.105:0/328947813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 msgr2=0x560c918b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db17ae640 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db17ae640 1 -- 172.21.15.105:0/328947813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 msgr2=0x560c918b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db17ae640 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.836+0000 7f3db17ae640 1 -- 172.21.15.105:0/328947813 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c918b9700 con 0x560c918a95d0 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3db1faf640 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3db17ae640 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b7bc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3da80981a0 tx=0x7f3da809aeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b6d2d4b2bd93e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3d94ff9640 1 -- 172.21.15.105:0/328947813 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da807ae60 con 0x560c918a95d0 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3d94ff9640 1 -- 172.21.15.105:0/328947813 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3da809b440 con 0x560c918a95d0 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da401d040 con 0x560c918a95d0 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3d94ff9640 1 -- 172.21.15.105:0/328947813 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3da80b3070 con 0x560c918a95d0 2024-03-26T17:52:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3db6604740 10 client.30400 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.837+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560c919255b0 con 0x560c918a95d0 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.838+0000 7f3d94ff9640 1 -- 172.21.15.105:0/328947813 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da401d040 con 0x560c918a95d0 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3d94ff9640 1 -- 172.21.15.105:0/328947813 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3da8083080 con 0x560c918a95d0 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3db6604740 10 client.30400 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3db6604740 10 client.30400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3db6604740 20 client.30400 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3db6604740 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c91925e70 0x560c91928290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3db6604740 4 client.30400 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c91815940 con 0x560c91925e70 2024-03-26T17:52:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3db0fad640 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c91925e70 0x560c91928290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3db0fad640 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c91925e70 0x560c91928290 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9338bd439742a291 server_cookie=c8a29781b28a6e39 in_seq=0 out_seq=0 2024-03-26T17:52:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:18.839+0000 7f3d94ff9640 10 client.30400 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.049+0000 7f3d94ff9640 1 -- 172.21.15.105:0/328947813 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c91815940 con 0x560c91925e70 2024-03-26T17:52:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3d94ff9640 10 client.30400 handle_command_reply: tid=0 2024-03-26T17:52:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 client.30400 shutdown 2024-03-26T17:52:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 msgr2=0x560c918b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b7bc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3da80981a0 tx=0x7f3da809aeb0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 shutdown_connections 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c91925e70 0x560c91928290 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c918af130 0x560c918b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3d94ff9640 0 client.30400 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c918a9fc0 0x560c918b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 --2- 172.21.15.105:0/328947813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c918a95d0 0x560c918b7bc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 >> 172.21.15.105:0/328947813 conn(0x560c918a4b40 msgr2=0x560c918aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 shutdown_connections 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 1 -- 172.21.15.105:0/328947813 wait complete. 2024-03-26T17:52:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.050+0000 7f3db6604740 20 client.30400 trim_cache size 0 max 16384 2024-03-26T17:52:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:52:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:52:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:52:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:52:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 -- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fc0 msgr2=0x7f3d980a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fc0 0x7f3d980a5480 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x560c918aa820 tx=0x7f3da4004ab0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 -- 172.21.15.105:0/3243844678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac100820 msgr2=0x7f3dac197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac100820 0x7f3dac197a80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3da800e9c0 tx=0x7f3da800ee80 comp rx=0 tx=0).stop 2024-03-26T17:52:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 -- 172.21.15.105:0/3243844678 shutdown_connections 2024-03-26T17:52:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fc0 0x7f3d980a5480 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1023d0 0x7f3dac19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac101a10 0x7f3dac197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 --2- 172.21.15.105:0/3243844678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac100820 0x7f3dac197a80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 -- 172.21.15.105:0/3243844678 >> 172.21.15.105:0/3243844678 conn(0x7f3dac0fbd90 msgr2=0x7f3dac0fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 -- 172.21.15.105:0/3243844678 shutdown_connections 2024-03-26T17:52:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:19.051+0000 7f3db3a12640 1 -- 172.21.15.105:0/3243844678 wait complete. 2024-03-26T17:52:19.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 8511 10.25 MB/sec warmup 62 sec latency 194.876 ms 2024-03-26T17:52:20.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:19 smithi105 ceph-mon[32280]: pgmap v768: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-26T17:52:20.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:19 smithi135 ceph-mon[35636]: pgmap v768: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-26T17:52:20.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:19 smithi144 ceph-mon[36095]: pgmap v768: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-03-26T17:52:20.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 8577 10.10 MB/sec warmup 63 sec latency 112.212 ms 2024-03-26T17:52:21.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 8654 9.97 MB/sec warmup 64 sec latency 121.069 ms 2024-03-26T17:52:22.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:21 smithi105 ceph-mon[32280]: pgmap v769: 97 pgs: 97 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:52:22.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:21 smithi135 ceph-mon[35636]: pgmap v769: 97 pgs: 97 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:52:22.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:21 smithi144 ceph-mon[36095]: pgmap v769: 97 pgs: 97 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:52:22.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 8764 9.97 MB/sec warmup 65 sec latency 139.199 ms 2024-03-26T17:52:23.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:22 smithi144 ceph-mon[36095]: pgmap v770: 97 pgs: 97 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-26T17:52:23.390 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:22 smithi105 ceph-mon[32280]: pgmap v770: 97 pgs: 97 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-26T17:52:23.414 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:22 smithi135 ceph-mon[35636]: pgmap v770: 97 pgs: 97 active+clean; 242 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-26T17:52:23.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 8898 9.94 MB/sec warmup 66 sec latency 183.589 ms 2024-03-26T17:52:24.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9066 10.03 MB/sec warmup 67 sec latency 122.059 ms 2024-03-26T17:52:25.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9212 10.01 MB/sec warmup 68 sec latency 170.040 ms 2024-03-26T17:52:26.092 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:25 smithi105 ceph-mon[32280]: pgmap v771: 97 pgs: 97 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.1 MiB/s wr, 275 op/s 2024-03-26T17:52:26.097 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:25 smithi144 ceph-mon[36095]: pgmap v771: 97 pgs: 97 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.1 MiB/s wr, 275 op/s 2024-03-26T17:52:26.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:25 smithi135 ceph-mon[35636]: pgmap v771: 97 pgs: 97 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 4.1 MiB/s wr, 275 op/s 2024-03-26T17:52:26.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9350 10.03 MB/sec warmup 69 sec latency 136.510 ms 2024-03-26T17:52:27.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:26 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:52:27.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:26 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:52:27.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:26 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:52:27.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9458 9.94 MB/sec warmup 70 sec latency 165.178 ms 2024-03-26T17:52:28.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:27 smithi105 ceph-mon[32280]: pgmap v772: 97 pgs: 97 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-03-26T17:52:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:52:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:52:28.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:52:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:27 smithi135 ceph-mon[35636]: pgmap v772: 97 pgs: 97 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-03-26T17:52:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:52:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:52:28.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:52:28.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:27 smithi144 ceph-mon[36095]: pgmap v772: 97 pgs: 97 active+clean; 240 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-03-26T17:52:28.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:52:28.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:52:28.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:52:28.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9534 9.86 MB/sec warmup 71 sec latency 126.547 ms 2024-03-26T17:52:29.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9635 9.78 MB/sec warmup 72 sec latency 130.051 ms 2024-03-26T17:52:30.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:29 smithi105 ceph-mon[32280]: pgmap v773: 97 pgs: 97 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 4.1 MiB/s wr, 251 op/s 2024-03-26T17:52:30.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:29 smithi135 ceph-mon[35636]: pgmap v773: 97 pgs: 97 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 4.1 MiB/s wr, 251 op/s 2024-03-26T17:52:30.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:29 smithi144 ceph-mon[36095]: pgmap v773: 97 pgs: 97 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 4.1 MiB/s wr, 251 op/s 2024-03-26T17:52:30.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9718 9.67 MB/sec warmup 73 sec latency 185.374 ms 2024-03-26T17:52:31.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9814 9.56 MB/sec warmup 74 sec latency 161.498 ms 2024-03-26T17:52:32.142 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:31 smithi105 ceph-mon[32280]: pgmap v774: 97 pgs: 97 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-03-26T17:52:32.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:31 smithi135 ceph-mon[35636]: pgmap v774: 97 pgs: 97 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-03-26T17:52:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 -- 172.21.15.105:0/1824865418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e4102390 msgr2=0x7fc6e4108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 --2- 172.21.15.105:0/1824865418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e4102390 0x7fc6e4108c20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009900 tx=0x7fc6dc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 -- 172.21.15.105:0/1824865418 shutdown_connections 2024-03-26T17:52:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 --2- 172.21.15.105:0/1824865418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e4102390 0x7fc6e4108c20 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 --2- 172.21.15.105:0/1824865418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41019d0 0x7fc6e4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 --2- 172.21.15.105:0/1824865418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e41007e0 0x7fc6e4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 -- 172.21.15.105:0/1824865418 >> 172.21.15.105:0/1824865418 conn(0x7fc6e40fbd50 msgr2=0x7fc6e40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 -- 172.21.15.105:0/1824865418 shutdown_connections 2024-03-26T17:52:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.178+0000 7fc6ec085640 1 -- 172.21.15.105:0/1824865418 wait complete. 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.179+0000 7fc6ec085640 1 Processor -- start 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.179+0000 7fc6ec085640 1 -- start start 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.179+0000 7fc6ec085640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41007e0 0x7fc6e41a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.179+0000 7fc6ec085640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e41019d0 0x7fc6e41a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.179+0000 7fc6ec085640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e4102390 0x7fc6e41a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9e21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41007e0 0x7fc6e41a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9e21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41007e0 0x7fc6e41a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53034/0 (socket says 172.21.15.105:53034) 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9620640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e41019d0 0x7fc6e41a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9e21640 1 -- 172.21.15.105:0/3332872026 learned_addr learned my addr 172.21.15.105:0/3332872026 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410c2f0 con 0x7fc6e41019d0 2024-03-26T17:52:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410c170 con 0x7fc6e4102390 2024-03-26T17:52:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e410c470 con 0x7fc6e41007e0 2024-03-26T17:52:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6ea622640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e4102390 0x7fc6e41a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9e21640 1 -- 172.21.15.105:0/3332872026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e4102390 msgr2=0x7fc6e41a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e4102390 0x7fc6e41a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9e21640 1 -- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e41019d0 msgr2=0x7fc6e41a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e41019d0 0x7fc6e41a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.180+0000 7fc6e9e21640 1 -- 172.21.15.105:0/3332872026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e4196020 con 0x7fc6e41007e0 2024-03-26T17:52:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.181+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41007e0 0x7fc6e41a07e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6e001fbd0 tx=0x7fc6e001d6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.181+0000 7fc6daffd640 1 -- 172.21.15.105:0/3332872026 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0003b70 con 0x7fc6e41007e0 2024-03-26T17:52:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.181+0000 7fc6daffd640 1 -- 172.21.15.105:0/3332872026 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6e0003d10 con 0x7fc6e41007e0 2024-03-26T17:52:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.181+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e4196310 con 0x7fc6e41007e0 2024-03-26T17:52:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.181+0000 7fc6daffd640 1 -- 172.21.15.105:0/3332872026 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e002a880 con 0x7fc6e41007e0 2024-03-26T17:52:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.181+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e4196800 con 0x7fc6e41007e0 2024-03-26T17:52:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.184+0000 7fc6daffd640 1 -- 172.21.15.105:0/3332872026 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc6e0004030 con 0x7fc6e41007e0 2024-03-26T17:52:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.185+0000 7fc6daffd640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a3040 0x7fc6d00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.185+0000 7fc6eec74740 1 Processor -- start 2024-03-26T17:52:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.185+0000 7fc6daffd640 1 -- 172.21.15.105:0/3332872026 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc6e00e0bf0 con 0x7fc6e41007e0 2024-03-26T17:52:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.185+0000 7fc6ea622640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a3040 0x7fc6d00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.185+0000 7fc6eec74740 1 -- start start 2024-03-26T17:52:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa885d0 0x55d8faa889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa88fc0 0x55d8faa8dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faa905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6eec74740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d8fa9ed0b0 con 0x55d8faa8e130 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6eec74740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8fa995960 con 0x55d8faa88fc0 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6eec74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d8fa9e9060 con 0x55d8faa885d0 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6ea622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faa905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6ea622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faa905c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43532/0 (socket says 172.21.15.105:43532) 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6ea622640 1 -- 172.21.15.105:0/3357715143 learned_addr learned my addr 172.21.15.105:0/3357715143 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa885d0 0x55d8faa889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6e9620640 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa88fc0 0x55d8faa8dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6ea622640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a3040 0x7fc6d00a5500 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006010 tx=0x7fc6dc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6ea622640 1 -- 172.21.15.105:0/3357715143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa885d0 msgr2=0x55d8faa889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6ea622640 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa885d0 0x55d8faa889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6ea622640 1 -- 172.21.15.105:0/3357715143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa88fc0 msgr2=0x55d8faa8dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.186+0000 7fc6ea622640 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa88fc0 0x55d8faa8dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.187+0000 7fc6ea622640 1 -- 172.21.15.105:0/3357715143 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8fa9a7c60 con 0x55d8faa8e130 2024-03-26T17:52:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.187+0000 7fc6ea622640 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faa905c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009900 tx=0x7fc6dc046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e88b0f606120e33 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.187+0000 7fc6cffff640 1 -- 172.21.15.105:0/3357715143 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc047070 con 0x55d8faa8e130 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.187+0000 7fc6cffff640 1 -- 172.21.15.105:0/3357715143 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6dc050070 con 0x55d8faa8e130 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.187+0000 7fc6cffff640 1 -- 172.21.15.105:0/3357715143 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc004e40 con 0x55d8faa8e130 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 -- 172.21.15.105:0/3357715143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 msgr2=0x55d8faa905c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faa905c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009900 tx=0x7fc6dc046000 comp rx=0 tx=0).stop 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 -- 172.21.15.105:0/3357715143 shutdown_connections 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faa905c0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa88fc0 0x55d8faa8dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 --2- 172.21.15.105:0/3357715143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa885d0 0x55d8faa889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 -- 172.21.15.105:0/3357715143 >> 172.21.15.105:0/3357715143 conn(0x55d8faa83b40 msgr2=0x55d8faa83f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 -- 172.21.15.105:0/3357715143 shutdown_connections 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.188+0000 7fc6eec74740 1 -- 172.21.15.105:0/3357715143 wait complete. 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.189+0000 7fc6eec74740 1 -- start start 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.189+0000 7fc6eec74740 1 Processor -- start 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaaa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6eec74740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d8faaa2090 con 0x55d8faa8e130 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6eec74740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8fa9a8db0 con 0x55d8faa885d0 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6eec74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d8faaa2210 con 0x55d8faa88fc0 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6eec74740 10 client.0 _ll_register_callbacks cb 0x55d8fa9ad110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:33118/0 (socket says 172.21.15.105:33118) 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 -- 172.21.15.105:0/2478192958 learned_addr learned my addr 172.21.15.105:0/2478192958 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6ea622640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaaa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9620640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 -- 172.21.15.105:0/2478192958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 msgr2=0x55d8faaa38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 -- 172.21.15.105:0/2478192958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 msgr2=0x55d8faaaa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaaa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 -- 172.21.15.105:0/2478192958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8fa701210 con 0x55d8faa885d0 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9620640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6ea622640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaaa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa33a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc6e001dbb0 tx=0x7fc6e00ad980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e48009e91218d4df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.190+0000 7fc6cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/2478192958 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00c6030 con 0x55d8faa885d0 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/2478192958 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc6e00ab580 con 0x55d8faa885d0 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6eec74740 1 -- 172.21.15.105:0/2478192958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6dc050070 con 0x55d8faa885d0 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/2478192958 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc6e00b13d0 con 0x55d8faa885d0 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6eec74740 10 client.30425 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6eec74740 1 -- 172.21.15.105:0/2478192958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8fa7cd620 con 0x55d8faa885d0 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/2478192958 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6e00b5c90 con 0x55d8faa885d0 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/2478192958 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc6e00b5800 con 0x55d8faa885d0 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6eec74740 10 client.30425 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6eec74740 10 client.30425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6eec74740 20 client.30425 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.191+0000 7fc6eec74740 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8faaaf1c0 0x55d8faab1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.192+0000 7fc6eec74740 4 client.30425 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.192+0000 7fc6eec74740 1 -- 172.21.15.105:0/2478192958 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc6dc050070 con 0x55d8faaaf1c0 2024-03-26T17:52:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.192+0000 7fc6e9620640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8faaaf1c0 0x55d8faab1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.192+0000 7fc6e9620640 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8faaaf1c0 0x55d8faab1640 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94737946009c941 server_cookie=a860280660499f48 in_seq=0 out_seq=0 2024-03-26T17:52:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.192+0000 7fc6cd7fa640 10 client.30425 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/2478192958 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fc6dc050070 con 0x55d8faaaf1c0 2024-03-26T17:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6cd7fa640 10 client.30425 handle_command_reply: tid=0 2024-03-26T17:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 client.30425 shutdown 2024-03-26T17:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 -- 172.21.15.105:0/2478192958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 msgr2=0x55d8faaa33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.190 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa33a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc6e001dbb0 tx=0x7fc6e00ad980 comp rx=0 tx=0).stop 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 -- 172.21.15.105:0/2478192958 shutdown_connections 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8faaaf1c0 0x55d8faab1640 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaaa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 --2- 172.21.15.105:0/2478192958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa33a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6cd7fa640 0 client.30425 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 -- 172.21.15.105:0/2478192958 >> 172.21.15.105:0/2478192958 conn(0x55d8faa83b40 msgr2=0x55d8faa82df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 -- 172.21.15.105:0/2478192958 shutdown_connections 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 1 -- 172.21.15.105:0/2478192958 wait complete. 2024-03-26T17:52:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.195+0000 7fc6eec74740 20 client.30425 trim_cache size 0 max 16384 2024-03-26T17:52:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.215+0000 7fc6eec74740 1 Processor -- start 2024-03-26T17:52:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.215+0000 7fc6eec74740 1 -- start start 2024-03-26T17:52:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.215+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faa89a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.215+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa88fc0 0x55d8faaaa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.215+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa8e130 0x55d8faa89fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6eec74740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e406c540 con 0x55d8faa88fc0 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6eec74740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e406e140 con 0x55d8faa885d0 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6eec74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e406cc10 con 0x55d8faa8e130 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa88fc0 0x55d8faaaa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa88fc0 0x55d8faaaa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43550/0 (socket says 172.21.15.105:43550) 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 -- 172.21.15.105:0/3027200742 learned_addr learned my addr 172.21.15.105:0/3027200742 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faa89a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6ea622640 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa8e130 0x55d8faa89fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 -- 172.21.15.105:0/3027200742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa8e130 msgr2=0x55d8faa89fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa8e130 0x55d8faa89fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 -- 172.21.15.105:0/3027200742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 msgr2=0x55d8faa89a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faa89a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 -- 172.21.15.105:0/3027200742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8faa87630 con 0x55d8faa88fc0 2024-03-26T17:52:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6e9620640 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa88fc0 0x55d8faaaa8b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x55d8faa89a30 tx=0x7fc6d4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859ef0f575419c12 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6cf7fe640 1 -- 172.21.15.105:0/3027200742 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4004050 con 0x55d8faa88fc0 2024-03-26T17:52:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.216+0000 7fc6cf7fe640 1 -- 172.21.15.105:0/3027200742 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d401f5c0 con 0x55d8faa88fc0 2024-03-26T17:52:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6cf7fe640 1 -- 172.21.15.105:0/3027200742 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4003830 con 0x55d8faa88fc0 2024-03-26T17:52:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6eec74740 1 -- 172.21.15.105:0/3027200742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa88fc0 msgr2=0x55d8faaaa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6eec74740 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa88fc0 0x55d8faaaa8b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x55d8faa89a30 tx=0x7fc6d4012820 comp rx=0 tx=0).stop 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6eec74740 1 -- 172.21.15.105:0/3027200742 shutdown_connections 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6eec74740 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa8e130 0x55d8faa89fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6eec74740 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa88fc0 0x55d8faaaa8b0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6eec74740 1 --2- 172.21.15.105:0/3027200742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faa89a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6eec74740 1 -- 172.21.15.105:0/3027200742 >> 172.21.15.105:0/3027200742 conn(0x55d8faa83b40 msgr2=0x55d8faa8d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.217+0000 7fc6eec74740 1 -- 172.21.15.105:0/3027200742 shutdown_connections 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.218+0000 7fc6eec74740 1 -- 172.21.15.105:0/3027200742 wait complete. 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.218+0000 7fc6eec74740 1 -- start start 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.218+0000 7fc6eec74740 1 Processor -- start 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.218+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.218+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6eec74740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaa9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6eec74740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e406da70 con 0x55d8faa8e130 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6eec74740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e406d2e0 con 0x55d8faa885d0 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6eec74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e419d6c0 con 0x55d8faa88fc0 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6eec74740 10 client.0 _ll_register_callbacks cb 0x7fc6e0009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6ea622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaa9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53066/0 (socket says 172.21.15.105:53066) 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 -- 172.21.15.105:0/4231538755 learned_addr learned my addr 172.21.15.105:0/4231538755 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 -- 172.21.15.105:0/4231538755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 msgr2=0x55d8faaa7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 -- 172.21.15.105:0/4231538755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 msgr2=0x55d8faaa9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaa9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 -- 172.21.15.105:0/4231538755 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8faaa9620 con 0x55d8faa88fc0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6e9620640 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa8020 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4013010 tx=0x7fc6d4009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c783913fff67d1b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.219+0000 7fc6cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/4231538755 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4004250 con 0x55d8faa88fc0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/4231538755 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc6d401f110 con 0x55d8faa88fc0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 1 -- 172.21.15.105:0/4231538755 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d401f5c0 con 0x55d8faa88fc0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/4231538755 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc6d401f310 con 0x55d8faa88fc0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 10 client.30415 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 1 -- 172.21.15.105:0/4231538755 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8faaaa9d0 con 0x55d8faa88fc0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/4231538755 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6d401f5c0 con 0x55d8faa88fc0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/4231538755 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc6d402f070 con 0x55d8faa88fc0 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 10 client.30415 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 10 client.30415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 20 client.30415 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8faaaf1c0 0x55d8fab17c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 4 client.30415 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:52:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.220+0000 7fc6eec74740 1 -- 172.21.15.105:0/4231538755 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8fa9fa5f0 con 0x55d8faaaf1c0 2024-03-26T17:52:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.221+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8faaaf1c0 0x55d8fab17c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.221+0000 7fc6e9e21640 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8faaaf1c0 0x55d8fab17c60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f10906b94cf346 server_cookie=232439c63209ac94 in_seq=0 out_seq=0 2024-03-26T17:52:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.221+0000 7fc6cd7fa640 10 client.30415 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:32.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:31 smithi144 ceph-mon[36095]: pgmap v774: 97 pgs: 97 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 3.3 MiB/s wr, 192 op/s 2024-03-26T17:52:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6cd7fa640 1 -- 172.21.15.105:0/4231538755 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8fa9fa5f0 con 0x55d8faaaf1c0 2024-03-26T17:52:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6cd7fa640 10 client.30415 handle_command_reply: tid=0 2024-03-26T17:52:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 client.30415 shutdown 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 -- 172.21.15.105:0/4231538755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 msgr2=0x55d8faaa8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa8020 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4013010 tx=0x7fc6d4009f60 comp rx=0 tx=0).stop 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 -- 172.21.15.105:0/4231538755 shutdown_connections 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8faaaf1c0 0x55d8fab17c60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8faa8e130 0x55d8faaa9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8faa88fc0 0x55d8faaa8020 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6cd7fa640 0 client.30415 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 --2- 172.21.15.105:0/4231538755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8faa885d0 0x55d8faaa7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 -- 172.21.15.105:0/4231538755 >> 172.21.15.105:0/4231538755 conn(0x55d8faa83b40 msgr2=0x55d8faa89a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 -- 172.21.15.105:0/4231538755 shutdown_connections 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.376+0000 7fc6eec74740 1 -- 172.21.15.105:0/4231538755 wait complete. 2024-03-26T17:52:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6eec74740 20 client.30415 trim_cache size 0 max 16384 2024-03-26T17:52:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:52:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:52:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:52:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a3040 msgr2=0x7fc6d00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a3040 0x7fc6d00a5500 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc006010 tx=0x7fc6dc005e30 comp rx=0 tx=0).stop 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41007e0 msgr2=0x7fc6e41a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41007e0 0x7fc6e41a07e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc6e001fbd0 tx=0x7fc6e001d6b0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 shutdown_connections 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6d00a3040 0x7fc6d00a5500 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6e4102390 0x7fc6e41a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6e41019d0 0x7fc6e41a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 --2- 172.21.15.105:0/3332872026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6e41007e0 0x7fc6e41a07e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 >> 172.21.15.105:0/3332872026 conn(0x7fc6e40fbd50 msgr2=0x7fc6e4068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 shutdown_connections 2024-03-26T17:52:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:32.377+0000 7fc6ec085640 1 -- 172.21.15.105:0/3332872026 wait complete. 2024-03-26T17:52:32.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 9916 9.45 MB/sec warmup 75 sec latency 215.434 ms 2024-03-26T17:52:33.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 10036 9.38 MB/sec warmup 76 sec latency 128.763 ms 2024-03-26T17:52:34.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:33 smithi105 ceph-mon[32280]: pgmap v775: 97 pgs: 97 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 3.3 MiB/s wr, 178 op/s 2024-03-26T17:52:34.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:33 smithi135 ceph-mon[35636]: pgmap v775: 97 pgs: 97 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 3.3 MiB/s wr, 178 op/s 2024-03-26T17:52:34.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:33 smithi144 ceph-mon[36095]: pgmap v775: 97 pgs: 97 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 3.3 MiB/s wr, 178 op/s 2024-03-26T17:52:34.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 10164 9.33 MB/sec warmup 77 sec latency 129.740 ms 2024-03-26T17:52:35.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 10297 9.27 MB/sec warmup 78 sec latency 114.699 ms 2024-03-26T17:52:36.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:35 smithi105 ceph-mon[32280]: pgmap v776: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 3.6 MiB/s wr, 238 op/s 2024-03-26T17:52:36.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:35 smithi135 ceph-mon[35636]: pgmap v776: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 3.6 MiB/s wr, 238 op/s 2024-03-26T17:52:36.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:35 smithi144 ceph-mon[36095]: pgmap v776: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 3.6 MiB/s wr, 238 op/s 2024-03-26T17:52:36.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 10430 9.24 MB/sec warmup 79 sec latency 121.846 ms 2024-03-26T17:52:37.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 10578 9.19 MB/sec warmup 80 sec latency 124.378 ms 2024-03-26T17:52:38.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:37 smithi105 ceph-mon[32280]: pgmap v777: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-03-26T17:52:38.148 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:37 smithi144 ceph-mon[36095]: pgmap v777: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-03-26T17:52:38.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:37 smithi135 ceph-mon[35636]: pgmap v777: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-03-26T17:52:38.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 10730 9.25 MB/sec warmup 81 sec latency 115.564 ms 2024-03-26T17:52:39.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 10918 9.36 MB/sec warmup 82 sec latency 114.123 ms 2024-03-26T17:52:40.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:39 smithi105 ceph-mon[32280]: pgmap v778: 97 pgs: 97 active+clean; 231 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-03-26T17:52:40.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:39 smithi135 ceph-mon[35636]: pgmap v778: 97 pgs: 97 active+clean; 231 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-03-26T17:52:40.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:39 smithi144 ceph-mon[36095]: pgmap v778: 97 pgs: 97 active+clean; 231 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-03-26T17:52:40.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 11034 9.25 MB/sec warmup 83 sec latency 120.190 ms 2024-03-26T17:52:41.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 11196 9.21 MB/sec warmup 84 sec latency 120.037 ms 2024-03-26T17:52:42.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:41 smithi105 ceph-mon[32280]: pgmap v779: 97 pgs: 97 active+clean; 229 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-03-26T17:52:42.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:41 smithi135 ceph-mon[35636]: pgmap v779: 97 pgs: 97 active+clean; 229 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-03-26T17:52:42.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:41 smithi144 ceph-mon[36095]: pgmap v779: 97 pgs: 97 active+clean; 229 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 2.8 MiB/s wr, 256 op/s 2024-03-26T17:52:42.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 11340 9.13 MB/sec warmup 85 sec latency 126.103 ms 2024-03-26T17:52:43.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 11488 9.08 MB/sec warmup 86 sec latency 119.609 ms 2024-03-26T17:52:44.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:43 smithi105 ceph-mon[32280]: pgmap v780: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-26T17:52:44.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:43 smithi135 ceph-mon[35636]: pgmap v780: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-26T17:52:44.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:52:44.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:52:44.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.341+0000 7f4c58d37640 1 -- 172.21.15.105:0/2260036503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541007c0 msgr2=0x7f4c54100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.341+0000 7f4c58d37640 1 --2- 172.21.15.105:0/2260036503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541007c0 0x7f4c54100be0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009900 tx=0x7f4c4402bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.342+0000 7f4c58d37640 1 -- 172.21.15.105:0/2260036503 shutdown_connections 2024-03-26T17:52:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.342+0000 7f4c58d37640 1 --2- 172.21.15.105:0/2260036503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c54102370 0x7f4c54108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.342+0000 7f4c58d37640 1 --2- 172.21.15.105:0/2260036503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c541019b0 0x7f4c54101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.342+0000 7f4c58d37640 1 --2- 172.21.15.105:0/2260036503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541007c0 0x7f4c54100be0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.342+0000 7f4c58d37640 1 -- 172.21.15.105:0/2260036503 >> 172.21.15.105:0/2260036503 conn(0x7f4c540fbd50 msgr2=0x7f4c540fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.342+0000 7f4c58d37640 1 -- 172.21.15.105:0/2260036503 shutdown_connections 2024-03-26T17:52:44.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.342+0000 7f4c58d37640 1 -- 172.21.15.105:0/2260036503 wait complete. 2024-03-26T17:52:44.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.342+0000 7f4c58d37640 1 Processor -- start 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c58d37640 1 -- start start 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c58d37640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541007c0 0x7f4c54196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c58d37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541019b0 0x7f4c54196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c58d37640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54102370 0x7f4c5419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c58d37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410c2f0 con 0x7f4c541019b0 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c58d37640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410c170 con 0x7f4c54102370 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c58d37640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5410c470 con 0x7f4c541007c0 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541019b0 0x7f4c54196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541019b0 0x7f4c54196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42548/0 (socket says 172.21.15.105:42548) 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c51d9b640 1 -- 172.21.15.105:0/3624461369 learned_addr learned my addr 172.21.15.105:0/3624461369 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c52d9d640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54102370 0x7f4c5419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c5259c640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541007c0 0x7f4c54196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c52d9d640 1 -- 172.21.15.105:0/3624461369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541007c0 msgr2=0x7f4c54196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c52d9d640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541007c0 0x7f4c54196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c52d9d640 1 -- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541019b0 msgr2=0x7f4c54196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c52d9d640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541019b0 0x7f4c54196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c52d9d640 1 -- 172.21.15.105:0/3624461369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5419e670 con 0x7f4c54102370 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.343+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541019b0 0x7f4c54196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.344+0000 7f4c52d9d640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54102370 0x7f4c5419e000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009fd0 tx=0x7f4c4c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.344+0000 7f4c437fe640 1 -- 172.21.15.105:0/3624461369 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00ce20 con 0x7f4c54102370 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.344+0000 7f4c58d37640 1 -- 172.21.15.105:0/3624461369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c54068a80 con 0x7f4c54102370 2024-03-26T17:52:44.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.344+0000 7f4c437fe640 1 -- 172.21.15.105:0/3624461369 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c4c005020 con 0x7f4c54102370 2024-03-26T17:52:44.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.344+0000 7f4c437fe640 1 -- 172.21.15.105:0/3624461369 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c010690 con 0x7f4c54102370 2024-03-26T17:52:44.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.344+0000 7f4c58d37640 1 -- 172.21.15.105:0/3624461369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c54069020 con 0x7f4c54102370 2024-03-26T17:52:44.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.347+0000 7f4c437fe640 1 -- 172.21.15.105:0/3624461369 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4c4c0051d0 con 0x7f4c54102370 2024-03-26T17:52:44.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.347+0000 7f4c5b9fa740 1 Processor -- start 2024-03-26T17:52:44.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5b9fa740 1 -- start start 2024-03-26T17:52:44.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e52f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5619e5255980 con 0x5619e52f12e0 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5619e51fdbe0 con 0x5619e52f6450 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5619e5251930 con 0x5619e52f08f0 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42562/0 (socket says 172.21.15.105:42562) 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52f0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59980/0 (socket says 172.21.15.105:59980) 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5259c640 1 -- 172.21.15.105:0/4006265783 learned_addr learned my addr 172.21.15.105:0/4006265783 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:44.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/4006265783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e52f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c437fe640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c3c0a2f20 0x7f4c3c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5259c640 1 -- 172.21.15.105:0/4006265783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 msgr2=0x5619e52f0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5259c640 1 --2- 172.21.15.105:0/4006265783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52f0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5259c640 1 -- 172.21.15.105:0/4006265783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 msgr2=0x5619e52f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5259c640 1 --2- 172.21.15.105:0/4006265783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e52f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.348+0000 7f4c5259c640 1 -- 172.21.15.105:0/4006265783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619e50337d0 con 0x5619e52f12e0 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.349+0000 7f4c52d9d640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c3c0a2f20 0x7f4c3c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.349+0000 7f4c5259c640 1 --2- 172.21.15.105:0/4006265783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52f5de0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009900 tx=0x7f4c44009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514b9f91b284ee8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.349+0000 7f4c437fe640 1 -- 172.21.15.105:0/3624461369 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4c4c0cfb40 con 0x7f4c54102370 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.349+0000 7f4c40ff9640 1 -- 172.21.15.105:0/4006265783 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44045020 con 0x5619e52f12e0 2024-03-26T17:52:44.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.349+0000 7f4c40ff9640 1 -- 172.21.15.105:0/4006265783 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c44004040 con 0x5619e52f12e0 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.349+0000 7f4c40ff9640 1 -- 172.21.15.105:0/4006265783 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44005500 con 0x5619e52f12e0 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.349+0000 7f4c52d9d640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c3c0a2f20 0x7f4c3c0a53e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c09c160 tx=0x7f4c4c096b90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/4006265783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 msgr2=0x5619e52f5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/4006265783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52f5de0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009900 tx=0x7f4c44009560 comp rx=0 tx=0).stop 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/4006265783 shutdown_connections 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/4006265783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e52f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/4006265783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52f5de0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/4006265783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52f0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/4006265783 >> 172.21.15.105:0/4006265783 conn(0x5619e52ebe60 msgr2=0x5619e52ec260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/4006265783 shutdown_connections 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.350+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/4006265783 wait complete. 2024-03-26T17:52:44.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 1 -- start start 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 1 Processor -- start 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f08f0 0x5619e52fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e52fb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f6450 0x5619e5302080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5619e52f93a0 con 0x5619e52f6450 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5619e5211160 con 0x5619e52f08f0 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5619e52f9520 con 0x5619e52f12e0 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5b9fa740 10 client.0 _ll_register_callbacks cb 0x5619e524d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f08f0 0x5619e52fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e52fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f6450 0x5619e5302080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e52fb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59988/0 (socket says 172.21.15.105:59988) 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5259c640 1 -- 172.21.15.105:0/920351878 learned_addr learned my addr 172.21.15.105:0/920351878 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c51d9b640 1 -- 172.21.15.105:0/920351878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 msgr2=0x5619e52fb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e52fb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c51d9b640 1 -- 172.21.15.105:0/920351878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f6450 msgr2=0x5619e5302080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f6450 0x5619e5302080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c51d9b640 1 -- 172.21.15.105:0/920351878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619e4f693f0 con 0x5619e52f08f0 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.351+0000 7f4c5259c640 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e52fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.352+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f08f0 0x5619e52fab00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009620 tx=0x7f4c480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=992db3c8aa383bc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.352+0000 7f4c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.352+0000 7f4c39ffb640 1 -- 172.21.15.105:0/920351878 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48004260 con 0x5619e52f08f0 2024-03-26T17:52:44.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.352+0000 7f4c39ffb640 1 -- 172.21.15.105:0/920351878 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4c48004400 con 0x5619e52f08f0 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.352+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/920351878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c44004040 con 0x5619e52f08f0 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.352+0000 7f4c39ffb640 1 -- 172.21.15.105:0/920351878 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4c4802d050 con 0x5619e52f08f0 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.352+0000 7f4c5b9fa740 10 client.30443 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.352+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/920351878 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5619e4f290f0 con 0x5619e52f08f0 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c39ffb640 1 -- 172.21.15.105:0/920351878 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c44004040 con 0x5619e52f08f0 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c39ffb640 1 -- 172.21.15.105:0/920351878 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4c480035f0 con 0x5619e52f08f0 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c5b9fa740 10 client.30443 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c5b9fa740 10 client.30443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c5b9fa740 20 client.30443 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5619e5306940 0x5619e5308da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c5b9fa740 4 client.30443 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:44.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/920351878 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619e5262ec0 con 0x5619e5306940 2024-03-26T17:52:44.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.353+0000 7f4c5259c640 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5619e5306940 0x5619e5308da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.354+0000 7f4c39ffb640 10 client.30443 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:44.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.354+0000 7f4c5259c640 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5619e5306940 0x5619e5308da0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c64aaff98c0b06 server_cookie=285546424b9ac1bf in_seq=0 out_seq=0 2024-03-26T17:52:44.352 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:43 smithi144 ceph-mon[36095]: pgmap v780: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c39ffb640 1 -- 172.21.15.105:0/920351878 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5619e5262ec0 con 0x5619e5306940 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c39ffb640 10 client.30443 handle_command_reply: tid=0 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 client.30443 shutdown 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/920351878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f08f0 msgr2=0x5619e52fab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f08f0 0x5619e52fab00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009620 tx=0x7f4c480046c0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/920351878 shutdown_connections 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5619e5306940 0x5619e5308da0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f6450 0x5619e5302080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e52fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c39ffb640 0 client.30443 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:44.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/920351878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f08f0 0x5619e52fab00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/920351878 >> 172.21.15.105:0/920351878 conn(0x5619e52ebe60 msgr2=0x5619e52eb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/920351878 shutdown_connections 2024-03-26T17:52:44.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/920351878 wait complete. 2024-03-26T17:52:44.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.356+0000 7f4c5b9fa740 20 client.30443 trim_cache size 0 max 16384 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5b9fa740 1 Processor -- start 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5b9fa740 1 -- start start 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f08f0 0x5619e52fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e5302040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e52f8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a6b30 con 0x5619e52f08f0 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a8730 con 0x5619e52f6450 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a7200 con 0x5619e52f12e0 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f08f0 0x5619e52fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c5259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e5302040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f08f0 0x5619e52fafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42582/0 (socket says 172.21.15.105:42582) 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.380+0000 7f4c51d9b640 1 -- 172.21.15.105:0/3812560831 learned_addr learned my addr 172.21.15.105:0/3812560831 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/3812560831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e52f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c51d9b640 1 -- 172.21.15.105:0/3812560831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 msgr2=0x5619e5302040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/3812560831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e5302040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c51d9b640 1 -- 172.21.15.105:0/3812560831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 msgr2=0x5619e52f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/3812560831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e52f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c51d9b640 1 -- 172.21.15.105:0/3812560831 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619e52e78d0 con 0x5619e52f08f0 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/3812560831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f08f0 0x5619e52fafa0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4c48004020 tx=0x7f4c48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b5b6b4c106e153 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c3bfff640 1 -- 172.21.15.105:0/3812560831 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c480035f0 con 0x5619e52f08f0 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c3bfff640 1 -- 172.21.15.105:0/3812560831 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c48017e40 con 0x5619e52f08f0 2024-03-26T17:52:44.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.381+0000 7f4c3bfff640 1 -- 172.21.15.105:0/3812560831 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c480044a0 con 0x5619e52f08f0 2024-03-26T17:52:44.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.382+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/3812560831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f08f0 msgr2=0x5619e52fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.382+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/3812560831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f08f0 0x5619e52fafa0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4c48004020 tx=0x7f4c48009780 comp rx=0 tx=0).stop 2024-03-26T17:52:44.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.382+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/3812560831 shutdown_connections 2024-03-26T17:52:44.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.382+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/3812560831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e52f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.382+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/3812560831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f12e0 0x5619e5302040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.382+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/3812560831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f08f0 0x5619e52fafa0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.382+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/3812560831 >> 172.21.15.105:0/3812560831 conn(0x5619e52ebe60 msgr2=0x5619e52f5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/3812560831 shutdown_connections 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/3812560831 wait complete. 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 -- start start 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 Processor -- start 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52ff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e5300d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a8060 con 0x5619e52f12e0 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a78d0 con 0x5619e52f6450 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a4b10 con 0x5619e52f08f0 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c51d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60000/0 (socket says 172.21.15.105:60000) 2024-03-26T17:52:44.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c5b9fa740 10 client.0 _ll_register_callbacks cb 0x7f4c480104b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.383+0000 7f4c51d9b640 1 -- 172.21.15.105:0/2157163063 learned_addr learned my addr 172.21.15.105:0/2157163063 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c5259c640 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e5300d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c5259c640 1 -- 172.21.15.105:0/2157163063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 msgr2=0x5619e52ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c5259c640 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c5259c640 1 -- 172.21.15.105:0/2157163063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 msgr2=0x5619e5300d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c5259c640 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e5300d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c5259c640 1 -- 172.21.15.105:0/2157163063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619e5301360 con 0x5619e52f12e0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c5259c640 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52ffd60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005ab0 tx=0x7f4c440098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf875ca2a18b01c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c39ffb640 1 -- 172.21.15.105:0/2157163063 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44045040 con 0x5619e52f12e0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c39ffb640 1 -- 172.21.15.105:0/2157163063 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4c4402c8e0 con 0x5619e52f12e0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.384+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/2157163063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619e525d210 con 0x5619e52f12e0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c39ffb640 1 -- 172.21.15.105:0/2157163063 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4c4402cae0 con 0x5619e52f12e0 2024-03-26T17:52:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c5b9fa740 10 client.20664 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/2157163063 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5619e536cdb0 con 0x5619e52f12e0 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c39ffb640 1 -- 172.21.15.105:0/2157163063 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5619e525d210 con 0x5619e52f12e0 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c39ffb640 1 -- 172.21.15.105:0/2157163063 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4c44053030 con 0x5619e52f12e0 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c5b9fa740 10 client.20664 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c5b9fa740 10 client.20664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c5b9fa740 20 client.20664 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5619e536d3e0 0x5619e536f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c5b9fa740 4 client.20664 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/2157163063 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619e51da540 con 0x5619e536d3e0 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.385+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5619e536d3e0 0x5619e536f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.386+0000 7f4c39ffb640 10 client.20664 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.386+0000 7f4c51d9b640 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5619e536d3e0 0x5619e536f7d0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734330a63353021b server_cookie=4d78ad132575a08 in_seq=0 out_seq=0 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.387+0000 7f4c39ffb640 1 -- 172.21.15.105:0/2157163063 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5619e51da540 con 0x5619e536d3e0 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.387+0000 7f4c39ffb640 10 client.20664 handle_command_reply: tid=0 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.387+0000 7f4c5b9fa740 1 client.20664 shutdown 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/2157163063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 msgr2=0x5619e52ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52ffd60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005ab0 tx=0x7f4c440098a0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/2157163063 shutdown_connections 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5619e536d3e0 0x5619e536f7d0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5619e52f6450 0x5619e5300d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5619e52f12e0 0x5619e52ffd60 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 --2- 172.21.15.105:0/2157163063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5619e52f08f0 0x5619e52ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/2157163063 >> 172.21.15.105:0/2157163063 conn(0x5619e52ebe60 msgr2=0x5619e52ef0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c39ffb640 0 client.20664 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/2157163063 shutdown_connections 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 1 -- 172.21.15.105:0/2157163063 wait complete. 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.388+0000 7f4c5b9fa740 20 client.20664 trim_cache size 0 max 16384 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:52:44.384 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:52:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 -- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c3c0a2f20 msgr2=0x7f4c3c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c3c0a2f20 0x7f4c3c0a53e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c09c160 tx=0x7f4c4c096b90 comp rx=0 tx=0).stop 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 -- 172.21.15.105:0/3624461369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54102370 msgr2=0x7f4c5419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54102370 0x7f4c5419e000 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009fd0 tx=0x7f4c4c00ef10 comp rx=0 tx=0).stop 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 -- 172.21.15.105:0/3624461369 shutdown_connections 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4c3c0a2f20 0x7f4c3c0a53e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4c54102370 0x7f4c5419e000 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4c541019b0 0x7f4c54196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 --2- 172.21.15.105:0/3624461369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c541007c0 0x7f4c54196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 -- 172.21.15.105:0/3624461369 >> 172.21.15.105:0/3624461369 conn(0x7f4c540fbd50 msgr2=0x7f4c54104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 -- 172.21.15.105:0/3624461369 shutdown_connections 2024-03-26T17:52:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.389+0000 7f4c58d37640 1 -- 172.21.15.105:0/3624461369 wait complete. 2024-03-26T17:52:44.395 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:52:44.396 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:52:44.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 -- 172.21.15.105:0/2982051535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f576c0742c0 msgr2=0x7f576c074720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 --2- 172.21.15.105:0/2982051535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f576c0742c0 0x7f576c074720 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002bcd0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 -- 172.21.15.105:0/2982051535 shutdown_connections 2024-03-26T17:52:44.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 --2- 172.21.15.105:0/2982051535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c109770 0x7f576c10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 --2- 172.21.15.105:0/2982051535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f576c0742c0 0x7f576c074720 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 --2- 172.21.15.105:0/2982051535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f576c0758a0 0x7f576c073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 -- 172.21.15.105:0/2982051535 >> 172.21.15.105:0/2982051535 conn(0x7f576c0fce70 msgr2=0x7f576c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 -- 172.21.15.105:0/2982051535 shutdown_connections 2024-03-26T17:52:44.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.543+0000 7f57722f8640 1 -- 172.21.15.105:0/2982051535 wait complete. 2024-03-26T17:52:44.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f57722f8640 1 Processor -- start 2024-03-26T17:52:44.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f57722f8640 1 -- start start 2024-03-26T17:52:44.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f57722f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f576c0742c0 0x7f576c199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f57722f8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f576c0758a0 0x7f576c19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f57722f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c109770 0x7f576c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f57722f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f576c10f610 con 0x7f576c0742c0 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f57722f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f576c10f490 con 0x7f576c109770 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f57722f8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f576c10f790 con 0x7f576c0758a0 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f576b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f576c0758a0 0x7f576c19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f5770895640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c109770 0x7f576c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f576b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f576c0758a0 0x7f576c19a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52680/0 (socket says 172.21.15.105:52680) 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f576b7fe640 1 -- 172.21.15.105:0/1227906621 learned_addr learned my addr 172.21.15.105:0/1227906621 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.544+0000 7f576bfff640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f576c0742c0 0x7f576c199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f576b7fe640 1 -- 172.21.15.105:0/1227906621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c109770 msgr2=0x7f576c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f576b7fe640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c109770 0x7f576c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f576b7fe640 1 -- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f576c0742c0 msgr2=0x7f576c199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f576b7fe640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f576c0742c0 0x7f576c199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f576b7fe640 1 -- 172.21.15.105:0/1227906621 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c1a1990 con 0x7f576c0758a0 2024-03-26T17:52:44.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f576b7fe640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f576c0758a0 0x7f576c19a2b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5760006010 tx=0x7f5760005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f57697fa640 1 -- 172.21.15.105:0/1227906621 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760043020 con 0x7f576c0758a0 2024-03-26T17:52:44.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c1a1c20 con 0x7f576c0758a0 2024-03-26T17:52:44.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.545+0000 7f57697fa640 1 -- 172.21.15.105:0/1227906621 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f576002dc20 con 0x7f576c0758a0 2024-03-26T17:52:44.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.546+0000 7f57697fa640 1 -- 172.21.15.105:0/1227906621 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760003a10 con 0x7f576c0758a0 2024-03-26T17:52:44.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.546+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c1a2170 con 0x7f576c0758a0 2024-03-26T17:52:44.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.547+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576c108530 con 0x7f576c0758a0 2024-03-26T17:52:44.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.548+0000 7f57697fa640 1 -- 172.21.15.105:0/1227906621 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5760046020 con 0x7f576c0758a0 2024-03-26T17:52:44.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.549+0000 7f57697fa640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57540a2f20 0x7f57540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:44.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.549+0000 7f576bfff640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57540a2f20 0x7f57540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:44.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.549+0000 7f57697fa640 1 -- 172.21.15.105:0/1227906621 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f57600f14f0 con 0x7f576c0758a0 2024-03-26T17:52:44.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.549+0000 7f576bfff640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57540a2f20 0x7f57540a53e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f575c009bd0 tx=0x7f575c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:44.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.557+0000 7f57697fa640 1 -- 172.21.15.105:0/1227906621 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f576004d020 con 0x7f576c0758a0 2024-03-26T17:52:44.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 11650 9.02 MB/sec warmup 87 sec latency 108.991 ms 2024-03-26T17:52:44.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.912+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f576c02d240 con 0x7f576c0758a0 2024-03-26T17:52:44.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.913+0000 7f57697fa640 1 -- 172.21.15.105:0/1227906621 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11948 (secure 0 0 0) 0x7f57600be2e0 con 0x7f576c0758a0 2024-03-26T17:52:44.905 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:52:44.905 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:51:26.802032+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:52:44.905 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 34 2024-03-26T17:52:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57540a2f20 msgr2=0x7f57540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57540a2f20 0x7f57540a53e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f575c009bd0 tx=0x7f575c005f90 comp rx=0 tx=0).stop 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f576c0758a0 msgr2=0x7f576c19a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f576c0758a0 0x7f576c19a2b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5760006010 tx=0x7f5760005d50 comp rx=0 tx=0).stop 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 shutdown_connections 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f57540a2f20 0x7f57540a53e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f576c109770 0x7f576c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f576c0758a0 0x7f576c19a2b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 --2- 172.21.15.105:0/1227906621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f576c0742c0 0x7f576c199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 >> 172.21.15.105:0/1227906621 conn(0x7f576c0fce70 msgr2=0x7f576c105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 shutdown_connections 2024-03-26T17:52:44.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:44.916+0000 7f57722f8640 1 -- 172.21.15.105:0/1227906621 wait complete. 2024-03-26T17:52:44.922 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:52:45.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 -- 172.21.15.105:0/1866976245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc734104cf0 msgr2=0x7fc734105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 --2- 172.21.15.105:0/1866976245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc734104cf0 0x7fc734105170 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc728009900 tx=0x7fc72802bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:45.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 -- 172.21.15.105:0/1866976245 shutdown_connections 2024-03-26T17:52:45.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 --2- 172.21.15.105:0/1866976245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc7341056b0 0x7fc73410bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 --2- 172.21.15.105:0/1866976245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc734104cf0 0x7fc734105170 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 --2- 172.21.15.105:0/1866976245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc734103b00 0x7fc734103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 -- 172.21.15.105:0/1866976245 >> 172.21.15.105:0/1866976245 conn(0x7fc7340ff070 msgr2=0x7fc7341014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 -- 172.21.15.105:0/1866976245 shutdown_connections 2024-03-26T17:52:45.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.080+0000 7fc7391f4640 1 -- 172.21.15.105:0/1866976245 wait complete. 2024-03-26T17:52:45.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.081+0000 7fc7391f4640 1 Processor -- start 2024-03-26T17:52:45.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc7391f4640 1 -- start start 2024-03-26T17:52:45.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc7391f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc734103b00 0x7fc734199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc7391f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc734104cf0 0x7fc73419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc7391f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7341056b0 0x7fc7341a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc7391f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410f680 con 0x7fc734104cf0 2024-03-26T17:52:45.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc7391f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410f500 con 0x7fc734103b00 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc7391f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc73410f800 con 0x7fc7341056b0 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc73359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7341056b0 0x7fc7341a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc73359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7341056b0 0x7fc7341a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52692/0 (socket says 172.21.15.105:52692) 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc73259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc734104cf0 0x7fc73419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc73359e640 1 -- 172.21.15.105:0/3339817645 learned_addr learned my addr 172.21.15.105:0/3339817645 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc732d9d640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc734103b00 0x7fc734199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.082+0000 7fc73359e640 1 -- 172.21.15.105:0/3339817645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc734103b00 msgr2=0x7fc734199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc73359e640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc734103b00 0x7fc734199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc73359e640 1 -- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc734104cf0 msgr2=0x7fc73419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc73359e640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc734104cf0 0x7fc73419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc73359e640 1 -- 172.21.15.105:0/3339817645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7341a1950 con 0x7fc7341056b0 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc73259c640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc734104cf0 0x7fc73419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc73359e640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7341056b0 0x7fc7341a12b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00e970 tx=0x7fc72c00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc71bfff640 1 -- 172.21.15.105:0/3339817645 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00ccc0 con 0x7fc7341056b0 2024-03-26T17:52:45.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc7391f4640 1 -- 172.21.15.105:0/3339817645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341a1c40 con 0x7fc7341056b0 2024-03-26T17:52:45.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc71bfff640 1 -- 172.21.15.105:0/3339817645 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc72c00ce60 con 0x7fc7341056b0 2024-03-26T17:52:45.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc71bfff640 1 -- 172.21.15.105:0/3339817645 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c010430 con 0x7fc7341056b0 2024-03-26T17:52:45.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.083+0000 7fc7391f4640 1 -- 172.21.15.105:0/3339817645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a2190 con 0x7fc7341056b0 2024-03-26T17:52:45.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.086+0000 7fc71bfff640 1 -- 172.21.15.105:0/3339817645 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc72c004f90 con 0x7fc7341056b0 2024-03-26T17:52:45.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73bde9740 1 Processor -- start 2024-03-26T17:52:45.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73bde9740 1 -- start start 2024-03-26T17:52:45.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbb7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb8420 0x55b35dbbcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbbd590 0x55b35dbbfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73bde9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b35db1c970 con 0x55b35dbb8420 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73bde9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b35dac4d70 con 0x55b35dbbd590 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73bde9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b35db18920 con 0x55b35dbb7a30 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbb7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc732d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb8420 0x55b35dbbcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbb7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52700/0 (socket says 172.21.15.105:52700) 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 -- 172.21.15.105:0/576853247 learned_addr learned my addr 172.21.15.105:0/576853247 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbbd590 0x55b35dbbfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 -- 172.21.15.105:0/576853247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbbd590 msgr2=0x55b35dbbfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbbd590 0x55b35dbbfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 -- 172.21.15.105:0/576853247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb8420 msgr2=0x55b35dbbcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb8420 0x55b35dbbcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc73259c640 1 -- 172.21.15.105:0/576853247 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35d8303f0 con 0x55b35dbb7a30 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc71bfff640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.087+0000 7fc732d9d640 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb8420 0x55b35dbbcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.088+0000 7fc73259c640 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbb7e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc728005ec0 tx=0x7fc728005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427d229c40e31fe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.088+0000 7fc71bfff640 1 -- 172.21.15.105:0/3339817645 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc72c0cf890 con 0x7fc7341056b0 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.088+0000 7fc732d9d640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.088+0000 7fc7197fa640 1 -- 172.21.15.105:0/576853247 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72804a070 con 0x55b35dbb7a30 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.088+0000 7fc7197fa640 1 -- 172.21.15.105:0/576853247 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc728004410 con 0x55b35dbb7a30 2024-03-26T17:52:45.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.088+0000 7fc7197fa640 1 -- 172.21.15.105:0/576853247 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728004c90 con 0x55b35dbb7a30 2024-03-26T17:52:45.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc732d9d640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc724009fd0 tx=0x7fc724009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 -- 172.21.15.105:0/576853247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 msgr2=0x55b35dbb7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbb7e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc728005ec0 tx=0x7fc728005b50 comp rx=0 tx=0).stop 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 -- 172.21.15.105:0/576853247 shutdown_connections 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbbd590 0x55b35dbbfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb8420 0x55b35dbbcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 --2- 172.21.15.105:0/576853247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbb7e50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 -- 172.21.15.105:0/576853247 >> 172.21.15.105:0/576853247 conn(0x55b35dbb2fa0 msgr2=0x55b35dbb33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 -- 172.21.15.105:0/576853247 shutdown_connections 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 -- 172.21.15.105:0/576853247 wait complete. 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 -- start start 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.089+0000 7fc73bde9740 1 Processor -- start 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbc1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbbd590 0x55b35dbc9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73bde9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b35dbc04e0 con 0x55b35dbbd590 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73bde9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b35dad8310 con 0x55b35dbb8420 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73bde9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b35dbc0660 con 0x55b35dbb7a30 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73bde9740 10 client.0 _ll_register_callbacks cb 0x55b35db13c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc732d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbc1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc732d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbc1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52704/0 (socket says 172.21.15.105:52704) 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc732d9d640 1 -- 172.21.15.105:0/53803385 learned_addr learned my addr 172.21.15.105:0/53803385 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73259c640 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73359e640 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbbd590 0x55b35dbc9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc732d9d640 1 -- 172.21.15.105:0/53803385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 msgr2=0x55b35dbc2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc732d9d640 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc732d9d640 1 -- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbbd590 msgr2=0x55b35dbc9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc732d9d640 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbbd590 0x55b35dbc9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc732d9d640 1 -- 172.21.15.105:0/53803385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35d8fa7d0 con 0x55b35dbb7a30 2024-03-26T17:52:45.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.090+0000 7fc73359e640 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbbd590 0x55b35dbc9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc732d9d640 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbc1fd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc724005fd0 tx=0x7fc724004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c0338609ec901f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc716ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc716ff5640 1 -- 172.21.15.105:0/53803385 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72400fe30 con 0x55b35dbb7a30 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc716ff5640 1 -- 172.21.15.105:0/53803385 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc724011b50 con 0x55b35dbb7a30 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc73bde9740 1 -- 172.21.15.105:0/53803385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc728004410 con 0x55b35dbb7a30 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc716ff5640 1 -- 172.21.15.105:0/53803385 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc724011d50 con 0x55b35dbb7a30 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc73bde9740 10 client.30451 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc73bde9740 1 -- 172.21.15.105:0/53803385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b35dbcc9e0 con 0x55b35dbb7a30 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc716ff5640 1 -- 172.21.15.105:0/53803385 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc728004410 con 0x55b35dbb7a30 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc716ff5640 1 -- 172.21.15.105:0/53803385 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc72401f450 con 0x55b35dbb7a30 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc73bde9740 10 client.30451 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc73bde9740 10 client.30451 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc73bde9740 20 client.30451 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.091+0000 7fc73bde9740 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b35dbce440 0x55b35dbd08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.092+0000 7fc73bde9740 4 client.30451 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:45.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.092+0000 7fc73bde9740 1 -- 172.21.15.105:0/53803385 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b35db29eb0 con 0x55b35dbce440 2024-03-26T17:52:45.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.092+0000 7fc73259c640 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b35dbce440 0x55b35dbd08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.092+0000 7fc716ff5640 10 client.30451 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:52:45.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.097+0000 7fc73259c640 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b35dbce440 0x55b35dbd08e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e780867e2ffd1a05 server_cookie=37415f916ccdd089 in_seq=0 out_seq=0 2024-03-26T17:52:45.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc716ff5640 1 -- 172.21.15.105:0/53803385 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b35db29eb0 con 0x55b35dbce440 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc716ff5640 10 client.30451 handle_command_reply: tid=0 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 client.30451 shutdown 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 -- 172.21.15.105:0/53803385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 msgr2=0x55b35dbc1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbc1fd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc724005fd0 tx=0x7fc724004ab0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 -- 172.21.15.105:0/53803385 shutdown_connections 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b35dbce440 0x55b35dbd08e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbbd590 0x55b35dbc9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 --2- 172.21.15.105:0/53803385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbb7a30 0x55b35dbc1fd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 -- 172.21.15.105:0/53803385 >> 172.21.15.105:0/53803385 conn(0x55b35dbb2fa0 msgr2=0x55b35dbb21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc716ff5640 0 client.30451 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:52:45.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 -- 172.21.15.105:0/53803385 shutdown_connections 2024-03-26T17:52:45.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 1 -- 172.21.15.105:0/53803385 wait complete. 2024-03-26T17:52:45.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.098+0000 7fc73bde9740 20 client.30451 trim_cache size 0 max 16384 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.119+0000 7fc73bde9740 1 Processor -- start 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.119+0000 7fc73bde9740 1 -- start start 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.119+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.119+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.119+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 0x55b35dbc0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.119+0000 7fc73bde9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341aaf70 con 0x55b35dbb7a30 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.119+0000 7fc73bde9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341acb70 con 0x55b35dbb8420 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.119+0000 7fc73bde9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ab640 con 0x55b35dbbd590 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc73359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 0x55b35dbc0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57370/0 (socket says 172.21.15.105:57370) 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 -- 172.21.15.105:0/2418146384 learned_addr learned my addr 172.21.15.105:0/2418146384 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc73259c640 1 --2- 172.21.15.105:0/2418146384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 -- 172.21.15.105:0/2418146384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 msgr2=0x55b35dbc0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 --2- 172.21.15.105:0/2418146384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 0x55b35dbc0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 -- 172.21.15.105:0/2418146384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 msgr2=0x55b35dbc9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 --2- 172.21.15.105:0/2418146384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 -- 172.21.15.105:0/2418146384 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35dbcc800 con 0x55b35dbb7a30 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc732d9d640 1 --2- 172.21.15.105:0/2418146384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc0a30 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc724019040 tx=0x7fc7240095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bb45c0b0d81f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.120+0000 7fc718ff9640 1 -- 172.21.15.105:0/2418146384 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72401f430 con 0x55b35dbb7a30 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.121+0000 7fc718ff9640 1 -- 172.21.15.105:0/2418146384 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc72402f030 con 0x55b35dbb7a30 2024-03-26T17:52:45.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.121+0000 7fc718ff9640 1 -- 172.21.15.105:0/2418146384 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724029a30 con 0x55b35dbb7a30 2024-03-26T17:52:45.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 -- 172.21.15.105:0/2418146384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 msgr2=0x55b35dbc0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2418146384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc0a30 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc724019040 tx=0x7fc7240095f0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 -- 172.21.15.105:0/2418146384 shutdown_connections 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2418146384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 0x55b35dbc0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2418146384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2418146384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc0a30 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 -- 172.21.15.105:0/2418146384 >> 172.21.15.105:0/2418146384 conn(0x55b35dbb2fa0 msgr2=0x55b35dbb99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 -- 172.21.15.105:0/2418146384 shutdown_connections 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 -- 172.21.15.105:0/2418146384 wait complete. 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 -- start start 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.122+0000 7fc73bde9740 1 Processor -- start 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73bde9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 0x55b35dbc5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73bde9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ac4a0 con 0x55b35dbb7a30 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73bde9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341abd10 con 0x55b35dbb8420 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73bde9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341a8f50 con 0x55b35dbbd590 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73bde9740 10 client.0 _ll_register_callbacks cb 0x7fc724009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57376/0 (socket says 172.21.15.105:57376) 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 -- 172.21.15.105:0/2284073942 learned_addr learned my addr 172.21.15.105:0/2284073942 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73259c640 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73359e640 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 0x55b35dbc5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 -- 172.21.15.105:0/2284073942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 msgr2=0x55b35dbc5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 0x55b35dbc5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 -- 172.21.15.105:0/2284073942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 msgr2=0x55b35dbc9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 -- 172.21.15.105:0/2284073942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b35dbc6560 con 0x55b35dbb7a30 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc73259c640 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc732d9d640 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc9040 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc724009a00 tx=0x7fc724005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7a3eb5356f6af7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.123+0000 7fc716ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc716ff5640 1 -- 172.21.15.105:0/2284073942 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724049060 con 0x55b35dbb7a30 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc716ff5640 1 -- 172.21.15.105:0/2284073942 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc72401fa00 con 0x55b35dbb7a30 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 1 -- 172.21.15.105:0/2284073942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72402f030 con 0x55b35dbb7a30 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc716ff5640 1 -- 172.21.15.105:0/2284073942 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc72401fc40 con 0x55b35dbb7a30 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 10 client.20688 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 1 -- 172.21.15.105:0/2284073942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b35dc33670 con 0x55b35dbb7a30 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc716ff5640 1 -- 172.21.15.105:0/2284073942 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc72402f030 con 0x55b35dbb7a30 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc716ff5640 1 -- 172.21.15.105:0/2284073942 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc724020430 con 0x55b35dbb7a30 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 10 client.20688 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 10 client.20688 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:52:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 20 client.20688 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b35dc34000 0x55b35dc36420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 4 client.20688 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.124+0000 7fc73bde9740 1 -- 172.21.15.105:0/2284073942 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b35db24200 con 0x55b35dc34000 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73259c640 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b35dc34000 0x55b35dc36420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc716ff5640 10 client.20688 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73259c640 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b35dc34000 0x55b35dc36420 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=686a06c2701d2f12 server_cookie=ee1a42e9c3d89a5d in_seq=0 out_seq=0 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc716ff5640 1 -- 172.21.15.105:0/2284073942 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b35db24200 con 0x55b35dc34000 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc716ff5640 10 client.20688 handle_command_reply: tid=0 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 client.20688 shutdown 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 -- 172.21.15.105:0/2284073942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 msgr2=0x55b35dbc9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc9040 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc724009a00 tx=0x7fc724005cf0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 -- 172.21.15.105:0/2284073942 shutdown_connections 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b35dc34000 0x55b35dc36420 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b35dbbd590 0x55b35dbc5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b35dbb8420 0x55b35dbc9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 --2- 172.21.15.105:0/2284073942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b35dbb7a30 0x55b35dbc9040 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.125+0000 7fc73bde9740 1 -- 172.21.15.105:0/2284073942 >> 172.21.15.105:0/2284073942 conn(0x55b35dbb2fa0 msgr2=0x55b35dbc0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc716ff5640 0 client.20688 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:52:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc73bde9740 1 -- 172.21.15.105:0/2284073942 shutdown_connections 2024-03-26T17:52:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc73bde9740 1 -- 172.21.15.105:0/2284073942 wait complete. 2024-03-26T17:52:45.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc73bde9740 20 client.20688 trim_cache size 0 max 16384 2024-03-26T17:52:45.122 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 -- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc71c0a2f20 msgr2=0x7fc71c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc724009fd0 tx=0x7fc724009500 comp rx=0 tx=0).stop 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 -- 172.21.15.105:0/3339817645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7341056b0 msgr2=0x7fc7341a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7341056b0 0x7fc7341a12b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc72c00e970 tx=0x7fc72c00ed70 comp rx=0 tx=0).stop 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 -- 172.21.15.105:0/3339817645 shutdown_connections 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc71c0a2f20 0x7fc71c0a53e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7341056b0 0x7fc7341a12b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc734104cf0 0x7fc73419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 --2- 172.21.15.105:0/3339817645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc734103b00 0x7fc734199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.126+0000 7fc7391f4640 1 -- 172.21.15.105:0/3339817645 >> 172.21.15.105:0/3339817645 conn(0x7fc7340ff070 msgr2=0x7fc7341080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.127+0000 7fc7391f4640 1 -- 172.21.15.105:0/3339817645 shutdown_connections 2024-03-26T17:52:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.127+0000 7fc7391f4640 1 -- 172.21.15.105:0/3339817645 wait complete. 2024-03-26T17:52:45.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 -- 172.21.15.105:0/2200469153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c104c70 msgr2=0x7fed2c107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 --2- 172.21.15.105:0/2200469153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c104c70 0x7fed2c107060 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fed20009900 tx=0x7fed2002bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 -- 172.21.15.105:0/2200469153 shutdown_connections 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 --2- 172.21.15.105:0/2200469153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2c1075a0 0x7fed2c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 --2- 172.21.15.105:0/2200469153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c104c70 0x7fed2c107060 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 --2- 172.21.15.105:0/2200469153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c102340 0x7fed2c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 -- 172.21.15.105:0/2200469153 >> 172.21.15.105:0/2200469153 conn(0x7fed2c0fbd50 msgr2=0x7fed2c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 -- 172.21.15.105:0/2200469153 shutdown_connections 2024-03-26T17:52:45.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.274+0000 7fed32197640 1 -- 172.21.15.105:0/2200469153 wait complete. 2024-03-26T17:52:45.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.275+0000 7fed32197640 1 Processor -- start 2024-03-26T17:52:45.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.275+0000 7fed32197640 1 -- start start 2024-03-26T17:52:45.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed32197640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2c102340 0x7fed2c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed32197640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c104c70 0x7fed2c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed32197640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c1075a0 0x7fed2c1a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed32197640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10d400 con 0x7fed2c1075a0 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed32197640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10d280 con 0x7fed2c102340 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed32197640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c10d580 con 0x7fed2c104c70 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c1075a0 0x7fed2c1a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c1075a0 0x7fed2c1a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57394/0 (socket says 172.21.15.105:57394) 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 -- 172.21.15.105:0/1764342985 learned_addr learned my addr 172.21.15.105:0/1764342985 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2c102340 0x7fed2c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2affd640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c104c70 0x7fed2c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 -- 172.21.15.105:0/1764342985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c104c70 msgr2=0x7fed2c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c104c70 0x7fed2c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 -- 172.21.15.105:0/1764342985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2c102340 msgr2=0x7fed2c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2c102340 0x7fed2c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 -- 172.21.15.105:0/1764342985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c1a1970 con 0x7fed2c1075a0 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2c102340 0x7fed2c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2affd640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c104c70 0x7fed2c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed2bfff640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c1075a0 0x7fed2c1a1300 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fed2400eb10 tx=0x7fed2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.276+0000 7fed28ff9640 1 -- 172.21.15.105:0/1764342985 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24004b20 con 0x7fed2c1075a0 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.277+0000 7fed28ff9640 1 -- 172.21.15.105:0/1764342985 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed24004cc0 con 0x7fed2c1075a0 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.277+0000 7fed32197640 1 -- 172.21.15.105:0/1764342985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c1a1c60 con 0x7fed2c1075a0 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.277+0000 7fed28ff9640 1 -- 172.21.15.105:0/1764342985 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24010720 con 0x7fed2c1075a0 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.277+0000 7fed32197640 1 -- 172.21.15.105:0/1764342985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a2130 con 0x7fed2c1075a0 2024-03-26T17:52:45.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.279+0000 7fed28ff9640 1 -- 172.21.15.105:0/1764342985 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fed24005020 con 0x7fed2c1075a0 2024-03-26T17:52:45.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.280+0000 7fed34d83740 1 Processor -- start 2024-03-26T17:52:45.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.280+0000 7fed34d83740 1 -- start start 2024-03-26T17:52:45.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.280+0000 7fed28ff9640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed140a2f20 0x7fed140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.280+0000 7fed34d83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359a9a30 0x557e359a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.280+0000 7fed34d83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359aa420 0x557e359aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed34d83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e359b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed34d83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557e3590e970 con 0x557e359af590 2024-03-26T17:52:45.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed34d83740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557e358b6d70 con 0x557e359aa420 2024-03-26T17:52:45.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed34d83740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557e3590a920 con 0x557e359a9a30 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed28ff9640 1 -- 172.21.15.105:0/1764342985 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fed240cfad0 con 0x7fed2c1075a0 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359a9a30 0x557e359a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2affd640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed140a2f20 0x7fed140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e359b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359a9a30 0x557e359a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52754/0 (socket says 172.21.15.105:52754) 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 -- 172.21.15.105:0/1581863881 learned_addr learned my addr 172.21.15.105:0/1581863881 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2affd640 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359aa420 0x557e359aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 -- 172.21.15.105:0/1581863881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359aa420 msgr2=0x557e359aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359aa420 0x557e359aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 -- 172.21.15.105:0/1581863881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 msgr2=0x557e359b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e359b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 -- 172.21.15.105:0/1581863881 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e356223f0 con 0x557e359a9a30 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2affd640 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359aa420 0x557e359aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359a9a30 0x557e359a9e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009d40 tx=0x7fed1c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8e0996b9f720c2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.281+0000 7fed2affd640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed140a2f20 0x7fed140a53e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fed20006010 tx=0x7fed20005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.282+0000 7fed11ffb640 1 -- 172.21.15.105:0/1581863881 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c012650 con 0x557e359a9a30 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.282+0000 7fed11ffb640 1 -- 172.21.15.105:0/1581863881 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed1c009e30 con 0x557e359a9a30 2024-03-26T17:52:45.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.282+0000 7fed11ffb640 1 -- 172.21.15.105:0/1581863881 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00fc30 con 0x557e359a9a30 2024-03-26T17:52:45.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 -- 172.21.15.105:0/1581863881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359a9a30 msgr2=0x557e359a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359a9a30 0x557e359a9e50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009d40 tx=0x7fed1c0117b0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 -- 172.21.15.105:0/1581863881 shutdown_connections 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e359b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359aa420 0x557e359aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 --2- 172.21.15.105:0/1581863881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359a9a30 0x557e359a9e50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 -- 172.21.15.105:0/1581863881 >> 172.21.15.105:0/1581863881 conn(0x557e359a4fa0 msgr2=0x557e359a53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 -- 172.21.15.105:0/1581863881 shutdown_connections 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 -- 172.21.15.105:0/1581863881 wait complete. 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 -- start start 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.283+0000 7fed34d83740 1 Processor -- start 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed34d83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359a9a30 0x557e359bc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed34d83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 0x557e359bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed34d83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359af590 0x557e359c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed34d83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557e359b2540 con 0x557e359a9a30 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed34d83740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557e358ca310 con 0x557e359af590 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed34d83740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557e359b26c0 con 0x557e359aa420 2024-03-26T17:52:45.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359a9a30 0x557e359bc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359a9a30 0x557e359bc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57420/0 (socket says 172.21.15.105:57420) 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 -- 172.21.15.105:0/1918167166 learned_addr learned my addr 172.21.15.105:0/1918167166 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359af590 0x557e359c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2affd640 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 0x557e359bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed34d83740 10 client.0 _ll_register_callbacks cb 0x557e35905c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 -- 172.21.15.105:0/1918167166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 msgr2=0x557e359bcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 0x557e359bcd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 -- 172.21.15.105:0/1918167166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359af590 msgr2=0x557e359c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359af590 0x557e359c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 -- 172.21.15.105:0/1918167166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e356ec7d0 con 0x557e359a9a30 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed34d83740 1 -- 172.21.15.105:0/1918167166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed1c009e30 con 0x557e359a9a30 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.284+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359a9a30 0x557e359bc810 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fed1c01c040 tx=0x7fed1c00ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8d30f6dedeee6a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed0affd640 10 client.20700 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:45.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed0affd640 1 -- 172.21.15.105:0/1918167166 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c028020 con 0x557e359a9a30 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed0affd640 1 -- 172.21.15.105:0/1918167166 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fed1c005450 con 0x557e359a9a30 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed0affd640 1 -- 172.21.15.105:0/1918167166 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fed1c005650 con 0x557e359a9a30 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed34d83740 10 client.20700 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed34d83740 1 -- 172.21.15.105:0/1918167166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=34}) -- 0x557e359c6f70 con 0x557e359a9a30 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed0affd640 1 -- 172.21.15.105:0/1918167166 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fed1c0046c0 con 0x557e359a9a30 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed34d83740 10 client.20700 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed34d83740 10 client.20700 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed34d83740 20 client.20700 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed34d83740 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557e359c83b0 0x557e359ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed34d83740 4 client.20700 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.285+0000 7fed34d83740 1 -- 172.21.15.105:0/1918167166 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e3591beb0 con 0x557e359c83b0 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.286+0000 7fed2affd640 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557e359c83b0 0x557e359ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.286+0000 7fed2affd640 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557e359c83b0 0x557e359ca7c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7488edc1c140bfdc server_cookie=c35c848450913309 in_seq=0 out_seq=0 2024-03-26T17:52:45.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.286+0000 7fed0affd640 10 client.20700 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed0affd640 1 -- 172.21.15.105:0/1918167166 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557e3591beb0 con 0x557e359c83b0 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed0affd640 10 client.20700 handle_command_reply: tid=0 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 client.20700 shutdown 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 -- 172.21.15.105:0/1918167166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359a9a30 msgr2=0x557e359bc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359a9a30 0x557e359bc810 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fed1c01c040 tx=0x7fed1c00ffa0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 -- 172.21.15.105:0/1918167166 shutdown_connections 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557e359c83b0 0x557e359ca7c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359af590 0x557e359c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 0x557e359bcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 --2- 172.21.15.105:0/1918167166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359a9a30 0x557e359bc810 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 -- 172.21.15.105:0/1918167166 >> 172.21.15.105:0/1918167166 conn(0x557e359a4fa0 msgr2=0x557e359a41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed0affd640 0 client.20700 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:52:45.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 -- 172.21.15.105:0/1918167166 shutdown_connections 2024-03-26T17:52:45.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.287+0000 7fed34d83740 1 -- 172.21.15.105:0/1918167166 wait complete. 2024-03-26T17:52:45.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.288+0000 7fed34d83740 20 client.20700 trim_cache size 0 max 16384 2024-03-26T17:52:45.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed34d83740 1 Processor -- start 2024-03-26T17:52:45.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed34d83740 1 -- start start 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed34d83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed34d83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359aa420 0x557e359c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed34d83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359af590 0x557e359bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed34d83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1a6bf0 con 0x557e359aa420 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed34d83740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1a87f0 con 0x557e359a9a30 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed34d83740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1a72c0 con 0x557e359af590 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed2affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359aa420 0x557e359c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed2affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359aa420 0x557e359c3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57436/0 (socket says 172.21.15.105:57436) 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed2affd640 1 -- 172.21.15.105:0/2577252111 learned_addr learned my addr 172.21.15.105:0/2577252111 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.308+0000 7fed2bfff640 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359af590 0x557e359bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed2affd640 1 -- 172.21.15.105:0/2577252111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359af590 msgr2=0x557e359bca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed2affd640 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359af590 0x557e359bca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed2affd640 1 -- 172.21.15.105:0/2577252111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 msgr2=0x557e359bc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed2affd640 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359bc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed2affd640 1 -- 172.21.15.105:0/2577252111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e359aa800 con 0x557e359aa420 2024-03-26T17:52:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed2bfff640 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359af590 0x557e359bca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359bc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed2affd640 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359aa420 0x557e359c3d50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fed2c069580 tx=0x7fed2002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f924820db320e2a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed117fa640 1 -- 172.21.15.105:0/2577252111 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20053100 con 0x557e359aa420 2024-03-26T17:52:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed117fa640 1 -- 172.21.15.105:0/2577252111 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed20059070 con 0x557e359aa420 2024-03-26T17:52:45.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.309+0000 7fed117fa640 1 -- 172.21.15.105:0/2577252111 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20004bb0 con 0x557e359aa420 2024-03-26T17:52:45.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 -- 172.21.15.105:0/2577252111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359aa420 msgr2=0x557e359c3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359aa420 0x557e359c3d50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fed2c069580 tx=0x7fed2002bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:45.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 -- 172.21.15.105:0/2577252111 shutdown_connections 2024-03-26T17:52:45.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359af590 0x557e359bca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359aa420 0x557e359c3d50 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 --2- 172.21.15.105:0/2577252111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359bc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 -- 172.21.15.105:0/2577252111 >> 172.21.15.105:0/2577252111 conn(0x557e359a4fa0 msgr2=0x557e359b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 -- 172.21.15.105:0/2577252111 shutdown_connections 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.310+0000 7fed34d83740 1 -- 172.21.15.105:0/2577252111 wait complete. 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 1 -- start start 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 1 Processor -- start 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 0x557e359c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e35a2c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1a8120 con 0x557e359af590 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1a7990 con 0x557e359a9a30 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1a4bf0 con 0x557e359aa420 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed34d83740 10 client.0 _ll_register_callbacks cb 0x7fed24004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e35a2c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e35a2c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57440/0 (socket says 172.21.15.105:57440) 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359c0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35244/0 (socket says 172.21.15.105:35244) 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed2b7fe640 1 -- 172.21.15.105:0/2954732870 learned_addr learned my addr 172.21.15.105:0/2954732870 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.311+0000 7fed2affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 0x557e359c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed2b7fe640 1 -- 172.21.15.105:0/2954732870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 msgr2=0x557e359c0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 0x557e359c0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed2b7fe640 1 -- 172.21.15.105:0/2954732870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 msgr2=0x557e35a2c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e35a2c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed2b7fe640 1 -- 172.21.15.105:0/2954732870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e359c4040 con 0x557e359a9a30 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed2b7fe640 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359c0a30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed1c011cb0 tx=0x7fed1c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=234e8fdd68545465 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed0affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed0affd640 1 -- 172.21.15.105:0/2954732870 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c028070 con 0x557e359a9a30 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed34d83740 1 -- 172.21.15.105:0/2954732870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed20059070 con 0x557e359a9a30 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed0affd640 1 -- 172.21.15.105:0/2954732870 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fed1c005450 con 0x557e359a9a30 2024-03-26T17:52:45.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.312+0000 7fed0affd640 1 -- 172.21.15.105:0/2954732870 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fed1c045050 con 0x557e359a9a30 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed34d83740 10 client.30491 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed34d83740 1 -- 172.21.15.105:0/2954732870 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e35a2dc20 con 0x557e359a9a30 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed0affd640 1 -- 172.21.15.105:0/2954732870 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed20059070 con 0x557e359a9a30 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed0affd640 1 -- 172.21.15.105:0/2954732870 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fed1c012900 con 0x557e359a9a30 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed34d83740 10 client.30491 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed34d83740 10 client.30491 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed34d83740 20 client.30491 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed34d83740 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557e35a2e590 0x557e35a309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed34d83740 4 client.30491 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed34d83740 1 -- 172.21.15.105:0/2954732870 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e35916200 con 0x557e35a2e590 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.313+0000 7fed2affd640 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557e35a2e590 0x557e35a309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed0affd640 10 client.30491 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed2affd640 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557e35a2e590 0x557e35a309b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cea42b0ebd804224 server_cookie=74051d9aa7a25d69 in_seq=0 out_seq=0 2024-03-26T17:52:45.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed0affd640 1 -- 172.21.15.105:0/2954732870 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e35916200 con 0x557e35a2e590 2024-03-26T17:52:45.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed0affd640 10 client.30491 handle_command_reply: tid=0 2024-03-26T17:52:45.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 client.30491 shutdown 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 -- 172.21.15.105:0/2954732870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 msgr2=0x557e359c0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359c0a30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed1c011cb0 tx=0x7fed1c005f90 comp rx=0 tx=0).stop 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 -- 172.21.15.105:0/2954732870 shutdown_connections 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557e35a2e590 0x557e35a309b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557e359af590 0x557e35a2c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed0affd640 0 client.30491 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557e359aa420 0x557e359c0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 --2- 172.21.15.105:0/2954732870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557e359a9a30 0x557e359c0a30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 -- 172.21.15.105:0/2954732870 >> 172.21.15.105:0/2954732870 conn(0x557e359a4fa0 msgr2=0x557e359bc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.314+0000 7fed34d83740 1 -- 172.21.15.105:0/2954732870 shutdown_connections 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed34d83740 1 -- 172.21.15.105:0/2954732870 wait complete. 2024-03-26T17:52:45.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed34d83740 20 client.30491 trim_cache size 0 max 16384 2024-03-26T17:52:45.311 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:45.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 -- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed140a2f20 msgr2=0x7fed140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed140a2f20 0x7fed140a53e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fed20006010 tx=0x7fed20005b50 comp rx=0 tx=0).stop 2024-03-26T17:52:45.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 -- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c1075a0 msgr2=0x7fed2c1a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c1075a0 0x7fed2c1a1300 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fed2400eb10 tx=0x7fed2400c2d0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 -- 172.21.15.105:0/1764342985 shutdown_connections 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fed140a2f20 0x7fed140a53e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed2c1075a0 0x7fed2c1a1300 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed2c104c70 0x7fed2c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 --2- 172.21.15.105:0/1764342985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed2c102340 0x7fed2c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.315+0000 7fed32197640 1 -- 172.21.15.105:0/1764342985 >> 172.21.15.105:0/1764342985 conn(0x7fed2c0fbd50 msgr2=0x7fed2c101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.316+0000 7fed32197640 1 -- 172.21.15.105:0/1764342985 shutdown_connections 2024-03-26T17:52:45.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.316+0000 7fed32197640 1 -- 172.21.15.105:0/1764342985 wait complete. 2024-03-26T17:52:45.320 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:52:45.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 -- 172.21.15.105:0/4141808930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 msgr2=0x7f2930101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 --2- 172.21.15.105:0/4141808930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 0x7f2930101e30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2924009900 tx=0x7f292402bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 -- 172.21.15.105:0/4141808930 shutdown_connections 2024-03-26T17:52:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 --2- 172.21.15.105:0/4141808930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2930102370 0x7f2930108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 --2- 172.21.15.105:0/4141808930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 0x7f2930101e30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 --2- 172.21.15.105:0/4141808930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29301007c0 0x7f2930100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 -- 172.21.15.105:0/4141808930 >> 172.21.15.105:0/4141808930 conn(0x7f29300fbd50 msgr2=0x7f29300fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 -- 172.21.15.105:0/4141808930 shutdown_connections 2024-03-26T17:52:45.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.477+0000 7f2935c36640 1 -- 172.21.15.105:0/4141808930 wait complete. 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f2935c36640 1 Processor -- start 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f2935c36640 1 -- start start 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f2935c36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29301007c0 0x7f2930197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f2935c36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 0x7f2930198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f2935c36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2930102370 0x7f293019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f2935c36640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f293010c2d0 con 0x7f2930102370 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f2935c36640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f293010c150 con 0x7f29301007c0 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f2935c36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f293010c450 con 0x7f29301019b0 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.478+0000 7f292ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2930102370 0x7f293019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 0x7f2930198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29301007c0 0x7f2930197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2930102370 0x7f293019f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57466/0 (socket says 172.21.15.105:57466) 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 0x7f2930198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52794/0 (socket says 172.21.15.105:52794) 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292ffff640 1 -- 172.21.15.105:0/1702670149 learned_addr learned my addr 172.21.15.105:0/1702670149 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292effd640 1 -- 172.21.15.105:0/1702670149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29301007c0 msgr2=0x7f2930197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292effd640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29301007c0 0x7f2930197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292effd640 1 -- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2930102370 msgr2=0x7f293019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292effd640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2930102370 0x7f293019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292effd640 1 -- 172.21.15.105:0/1702670149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293019f770 con 0x7f29301019b0 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292effd640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 0x7f2930198090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2924000c00 tx=0x7f2924005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292cff9640 1 -- 172.21.15.105:0/1702670149 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924043020 con 0x7f29301019b0 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f292cff9640 1 -- 172.21.15.105:0/1702670149 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2924004410 con 0x7f29301019b0 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.479+0000 7f2935c36640 1 -- 172.21.15.105:0/1702670149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293019fa00 con 0x7f29301019b0 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.480+0000 7f2935c36640 1 -- 172.21.15.105:0/1702670149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293019ffa0 con 0x7f29301019b0 2024-03-26T17:52:45.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.480+0000 7f292cff9640 1 -- 172.21.15.105:0/1702670149 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924003a90 con 0x7f29301019b0 2024-03-26T17:52:45.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.482+0000 7f292cff9640 1 -- 172.21.15.105:0/1702670149 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2924046020 con 0x7f29301019b0 2024-03-26T17:52:45.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f2938826740 1 Processor -- start 2024-03-26T17:52:45.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f2938826740 1 -- start start 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f2938826740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d57a30 0x562828d57e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f2938826740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d5cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f2938826740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d5d590 0x562828d5fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f2938826740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562828cbc970 con 0x562828d57a30 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f2938826740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562828c64d70 con 0x562828d58420 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f2938826740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562828cb8920 con 0x562828d5d590 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d5cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f292ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d57a30 0x562828d57e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d5cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35276/0 (socket says 172.21.15.105:35276) 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f292ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d57a30 0x562828d57e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57468/0 (socket says 172.21.15.105:57468) 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f292f7fe640 1 -- 172.21.15.105:0/1972040370 learned_addr learned my addr 172.21.15.105:0/1972040370 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.483+0000 7f292effd640 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d5d590 0x562828d5fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292f7fe640 1 -- 172.21.15.105:0/1972040370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d5d590 msgr2=0x562828d5fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d5d590 0x562828d5fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292f7fe640 1 -- 172.21.15.105:0/1972040370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d57a30 msgr2=0x562828d57e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d57a30 0x562828d57e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292f7fe640 1 -- 172.21.15.105:0/1972040370 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628289d03f0 con 0x562828d58420 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292ffff640 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d57a30 0x562828d57e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d5cf20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2920009c80 tx=0x7f2920011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42566214e12a2211 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292cff9640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f29180a2fa0 0x7f29180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f2915ffb640 1 -- 172.21.15.105:0/1972040370 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920004230 con 0x562828d58420 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292ffff640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f29180a2fa0 0x7f29180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f2915ffb640 1 -- 172.21.15.105:0/1972040370 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29200043d0 con 0x562828d58420 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f2915ffb640 1 -- 172.21.15.105:0/1972040370 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920004d30 con 0x562828d58420 2024-03-26T17:52:45.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.484+0000 7f292cff9640 1 -- 172.21.15.105:0/1702670149 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f29240f12f0 con 0x7f29301019b0 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 -- 172.21.15.105:0/1972040370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 msgr2=0x562828d5cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d5cf20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2920009c80 tx=0x7f2920011810 comp rx=0 tx=0).stop 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f292ffff640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f29180a2fa0 0x7f29180a5460 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x562828d58c80 tx=0x7f2928009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 -- 172.21.15.105:0/1972040370 shutdown_connections 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d5d590 0x562828d5fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d5cf20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 --2- 172.21.15.105:0/1972040370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d57a30 0x562828d57e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 -- 172.21.15.105:0/1972040370 >> 172.21.15.105:0/1972040370 conn(0x562828d52fa0 msgr2=0x562828d533a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 -- 172.21.15.105:0/1972040370 shutdown_connections 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 -- 172.21.15.105:0/1972040370 wait complete. 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.485+0000 7f2938826740 1 -- start start 2024-03-26T17:52:45.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f2938826740 1 Processor -- start 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f2938826740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f2938826740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f2938826740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f2938826740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562828d60540 con 0x562828d5d590 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f2938826740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562828c78310 con 0x562828d58420 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f2938826740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562828d606c0 con 0x562828d57a30 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f2938826740 10 client.0 _ll_register_callbacks cb 0x562828cb3c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d61f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52808/0 (socket says 172.21.15.105:52808) 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292f7fe640 1 -- 172.21.15.105:0/1335371060 learned_addr learned my addr 172.21.15.105:0/1335371060 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292ffff640 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292effd640 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292f7fe640 1 -- 172.21.15.105:0/1335371060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 msgr2=0x562828d624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d624b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292f7fe640 1 -- 172.21.15.105:0/1335371060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 msgr2=0x562828d694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292f7fe640 1 -- 172.21.15.105:0/1335371060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562828a9a7d0 con 0x562828d57a30 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.486+0000 7f292ffff640 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d61f70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2920019000 tx=0x7f2920005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55515a70cacd55c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f290effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f290effd640 1 -- 172.21.15.105:0/1335371060 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920028020 con 0x562828d57a30 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f2938826740 1 -- 172.21.15.105:0/1335371060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29200043d0 con 0x562828d57a30 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f290effd640 1 -- 172.21.15.105:0/1335371060 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2920019630 con 0x562828d57a30 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f290effd640 1 -- 172.21.15.105:0/1335371060 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2920033050 con 0x562828d57a30 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f2938826740 10 client.30487 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f2938826740 1 -- 172.21.15.105:0/1335371060 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562828d6c860 con 0x562828d57a30 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.487+0000 7f290effd640 1 -- 172.21.15.105:0/1335371060 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29200043d0 con 0x562828d57a30 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.488+0000 7f290effd640 1 -- 172.21.15.105:0/1335371060 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f29200244b0 con 0x562828d57a30 2024-03-26T17:52:45.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.488+0000 7f2938826740 10 client.30487 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.488+0000 7f2938826740 10 client.30487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.488+0000 7f2938826740 20 client.30487 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.488+0000 7f2938826740 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562828d6dd30 0x562828d701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.488+0000 7f2938826740 4 client.30487 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.488+0000 7f2938826740 1 -- 172.21.15.105:0/1335371060 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562828cc9eb0 con 0x562828d6dd30 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.488+0000 7f292effd640 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562828d6dd30 0x562828d701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.489+0000 7f290effd640 10 client.30487 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.489+0000 7f292effd640 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562828d6dd30 0x562828d701d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=881c181db878682f server_cookie=4170e648b07af42e in_seq=0 out_seq=0 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f290effd640 1 -- 172.21.15.105:0/1335371060 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562828cc9eb0 con 0x562828d6dd30 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f290effd640 10 client.30487 handle_command_reply: tid=0 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 client.30487 shutdown 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 -- 172.21.15.105:0/1335371060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 msgr2=0x562828d61f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d61f70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2920019000 tx=0x7f2920005250 comp rx=0 tx=0).stop 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 -- 172.21.15.105:0/1335371060 shutdown_connections 2024-03-26T17:52:45.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562828d6dd30 0x562828d701d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 --2- 172.21.15.105:0/1335371060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d61f70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f290effd640 0 client.30487 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:52:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.490+0000 7f2938826740 1 -- 172.21.15.105:0/1335371060 >> 172.21.15.105:0/1335371060 conn(0x562828d52fa0 msgr2=0x562828d521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.491+0000 7f2938826740 1 -- 172.21.15.105:0/1335371060 shutdown_connections 2024-03-26T17:52:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.491+0000 7f2938826740 1 -- 172.21.15.105:0/1335371060 wait complete. 2024-03-26T17:52:45.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.491+0000 7f2938826740 20 client.30487 trim_cache size 0 max 16384 2024-03-26T17:52:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.506+0000 7f86af26c640 1 -- 172.21.15.105:0/4126909976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 msgr2=0x7f86a8109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.506+0000 7f86af26c640 1 --2- 172.21.15.105:0/4126909976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 0x7f86a8109d50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009ca0 tx=0x7f86a002c100 comp rx=0 tx=0).stop 2024-03-26T17:52:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.506+0000 7f86af26c640 1 -- 172.21.15.105:0/4126909976 shutdown_connections 2024-03-26T17:52:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.506+0000 7f86af26c640 1 --2- 172.21.15.105:0/4126909976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 0x7f86a8109d50 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.506+0000 7f86af26c640 1 --2- 172.21.15.105:0/4126909976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86a8102b00 0x7f86a8102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.506+0000 7f86af26c640 1 --2- 172.21.15.105:0/4126909976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86a8101910 0x7f86a8101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.506+0000 7f86af26c640 1 -- 172.21.15.105:0/4126909976 >> 172.21.15.105:0/4126909976 conn(0x7f86a80fce80 msgr2=0x7f86a80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.507+0000 7f86af26c640 1 -- 172.21.15.105:0/4126909976 shutdown_connections 2024-03-26T17:52:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.507+0000 7f86af26c640 1 -- 172.21.15.105:0/4126909976 wait complete. 2024-03-26T17:52:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.507+0000 7f86af26c640 1 Processor -- start 2024-03-26T17:52:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.507+0000 7f86af26c640 1 -- start start 2024-03-26T17:52:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86af26c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86a8101910 0x7f86a8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86af26c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86a8102b00 0x7f86a8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86af26c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 0x7f86a81a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86af26c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86a80743f0 con 0x7f86a81034c0 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86af26c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8074270 con 0x7f86a8102b00 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86af26c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8074570 con 0x7f86a8101910 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad809640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 0x7f86a81a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad008640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86a8101910 0x7f86a8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad809640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 0x7f86a81a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57486/0 (socket says 172.21.15.105:57486) 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad809640 1 -- 172.21.15.105:0/2282679673 learned_addr learned my addr 172.21.15.105:0/2282679673 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad008640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86a8101910 0x7f86a8199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52828/0 (socket says 172.21.15.105:52828) 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad008640 1 -- 172.21.15.105:0/2282679673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86a8102b00 msgr2=0x7f86a8199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad008640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86a8102b00 0x7f86a8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad008640 1 -- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 msgr2=0x7f86a81a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad008640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 0x7f86a81a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad008640 1 -- 172.21.15.105:0/2282679673 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a81a15e0 con 0x7f86a8101910 2024-03-26T17:52:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.508+0000 7f86ad809640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 0x7f86a81a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.509+0000 7f86ad008640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86a8101910 0x7f86a8199a10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f86a400d6a0 tx=0x7f86a400db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.509+0000 7f869e7fc640 1 -- 172.21.15.105:0/2282679673 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4017070 con 0x7f86a8101910 2024-03-26T17:52:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.509+0000 7f86af26c640 1 -- 172.21.15.105:0/2282679673 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a81a18d0 con 0x7f86a8101910 2024-03-26T17:52:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.509+0000 7f869e7fc640 1 -- 172.21.15.105:0/2282679673 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86a4009cd0 con 0x7f86a8101910 2024-03-26T17:52:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.509+0000 7f869e7fc640 1 -- 172.21.15.105:0/2282679673 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4003bf0 con 0x7f86a8101910 2024-03-26T17:52:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.509+0000 7f86af26c640 1 -- 172.21.15.105:0/2282679673 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a81a1e20 con 0x7f86a8101910 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.512+0000 7f869e7fc640 1 -- 172.21.15.105:0/2282679673 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f86a4010460 con 0x7f86a8101910 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.512+0000 7f86b1e58740 1 Processor -- start 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.512+0000 7f86b1e58740 1 -- start start 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe15849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1584fc0 0x563fe1589ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe158a130 0x563fe158c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86b1e58740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563fe14e90b0 con 0x563fe158a130 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86b1e58740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563fe1491960 con 0x563fe1584fc0 2024-03-26T17:52:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86b1e58740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563fe14e5060 con 0x563fe15845d0 2024-03-26T17:52:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86ad809640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe15849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86ad008640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1584fc0 0x563fe1589ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86ad809640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe15849f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52840/0 (socket says 172.21.15.105:52840) 2024-03-26T17:52:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86ad809640 1 -- 172.21.15.105:0/560903628 learned_addr learned my addr 172.21.15.105:0/560903628 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f86ac807640 1 --2- 172.21.15.105:0/560903628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe158a130 0x563fe158c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.513+0000 7f869e7fc640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86940a2fc0 0x7f86940a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86ac807640 1 -- 172.21.15.105:0/560903628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 msgr2=0x563fe15849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86ac807640 1 --2- 172.21.15.105:0/560903628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe15849f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86ac807640 1 -- 172.21.15.105:0/560903628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1584fc0 msgr2=0x563fe1589ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86ac807640 1 --2- 172.21.15.105:0/560903628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1584fc0 0x563fe1589ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86ac807640 1 -- 172.21.15.105:0/560903628 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe14a3c60 con 0x563fe158a130 2024-03-26T17:52:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f869e7fc640 1 -- 172.21.15.105:0/2282679673 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f86a40cf1a0 con 0x7f86a8101910 2024-03-26T17:52:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86ad809640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86940a2fc0 0x7f86940a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86ac807640 1 --2- 172.21.15.105:0/560903628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe158a130 0x563fe158c5c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f869800a9b0 tx=0x7f869800ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cb83df1d5307ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86937fe640 1 -- 172.21.15.105:0/560903628 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869800cd80 con 0x563fe158a130 2024-03-26T17:52:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.514+0000 7f86ad809640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86940a2fc0 0x7f86940a5480 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f86a0003b30 tx=0x7f86a00098c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f86937fe640 1 -- 172.21.15.105:0/560903628 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f869800f900 con 0x563fe158a130 2024-03-26T17:52:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f86937fe640 1 -- 172.21.15.105:0/560903628 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698018980 con 0x563fe158a130 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f2938826740 1 Processor -- start 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f2938826740 1 -- start start 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f2938826740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d74a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f2938826740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f2938826740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d74fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f2938826740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f291c00e530 con 0x562828d5d590 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f2938826740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f291c010130 con 0x562828d58420 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f2938826740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f291c00ec00 con 0x562828d57a30 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d74a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d74a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52842/0 (socket says 172.21.15.105:52842) 2024-03-26T17:52:45.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f292f7fe640 1 -- 172.21.15.105:0/1090043631 learned_addr learned my addr 172.21.15.105:0/1090043631 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f292effd640 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 -- 172.21.15.105:0/560903628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe158a130 msgr2=0x563fe158c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 --2- 172.21.15.105:0/560903628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe158a130 0x563fe158c5c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f869800a9b0 tx=0x7f869800ae70 comp rx=0 tx=0).stop 2024-03-26T17:52:45.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.515+0000 7f292ffff640 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d74fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f292f7fe640 1 -- 172.21.15.105:0/1090043631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 msgr2=0x562828d694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f292f7fe640 1 -- 172.21.15.105:0/1090043631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 msgr2=0x562828d74fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 -- 172.21.15.105:0/560903628 shutdown_connections 2024-03-26T17:52:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 --2- 172.21.15.105:0/560903628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe158a130 0x563fe158c5c0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 --2- 172.21.15.105:0/560903628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1584fc0 0x563fe1589ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 --2- 172.21.15.105:0/560903628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe15849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 -- 172.21.15.105:0/560903628 >> 172.21.15.105:0/560903628 conn(0x563fe157fb40 msgr2=0x563fe157ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d74fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f292f7fe640 1 -- 172.21.15.105:0/1090043631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562828d53d10 con 0x562828d57a30 2024-03-26T17:52:45.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f292ffff640 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d74fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:52:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 -- 172.21.15.105:0/560903628 shutdown_connections 2024-03-26T17:52:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f292f7fe640 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d74a90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2920024a90 tx=0x7f2920001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33f4f543de516a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 -- 172.21.15.105:0/560903628 wait complete. 2024-03-26T17:52:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f29157fa640 1 -- 172.21.15.105:0/1090043631 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920028070 con 0x562828d57a30 2024-03-26T17:52:45.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f29157fa640 1 -- 172.21.15.105:0/1090043631 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f292001f040 con 0x562828d57a30 2024-03-26T17:52:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f29157fa640 1 -- 172.21.15.105:0/1090043631 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920024680 con 0x562828d57a30 2024-03-26T17:52:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.516+0000 7f86b1e58740 1 -- start start 2024-03-26T17:52:45.513 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f86b1e58740 1 Processor -- start 2024-03-26T17:52:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 -- 172.21.15.105:0/1090043631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 msgr2=0x562828d74a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d74a90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2920024a90 tx=0x7f2920001e70 comp rx=0 tx=0).stop 2024-03-26T17:52:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe158e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.514 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 -- 172.21.15.105:0/1090043631 shutdown_connections 2024-03-26T17:52:45.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d74fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 --2- 172.21.15.105:0/1090043631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d74a90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 -- 172.21.15.105:0/1090043631 >> 172.21.15.105:0/1090043631 conn(0x562828d52fa0 msgr2=0x562828d56a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f86ad008640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe158e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f86ad008640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe158e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35332/0 (socket says 172.21.15.105:35332) 2024-03-26T17:52:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f86ad008640 1 -- 172.21.15.105:0/3816672122 learned_addr learned my addr 172.21.15.105:0/3816672122 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.515 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f86ac807640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 -- 172.21.15.105:0/1090043631 shutdown_connections 2024-03-26T17:52:45.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.517+0000 7f2938826740 1 -- 172.21.15.105:0/1090043631 wait complete. 2024-03-26T17:52:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86b1e58740 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1593af0 0x563fe1595ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.516 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563fe158d0e0 con 0x563fe1584fc0 2024-03-26T17:52:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563fe14a4db0 con 0x563fe15845d0 2024-03-26T17:52:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563fe158d260 con 0x563fe1593af0 2024-03-26T17:52:45.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 1 -- start start 2024-03-26T17:52:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86ad809640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1593af0 0x563fe1595ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86ad008640 1 -- 172.21.15.105:0/3816672122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1593af0 msgr2=0x563fe1595ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86ad008640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1593af0 0x563fe1595ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.517 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86ad008640 1 -- 172.21.15.105:0/3816672122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 msgr2=0x563fe158eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 1 Processor -- start 2024-03-26T17:52:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86ad008640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86ad008640 1 -- 172.21.15.105:0/3816672122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe11fd210 con 0x563fe15845d0 2024-03-26T17:52:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86ad809640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1593af0 0x563fe1595ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.518 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86ac807640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d66bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d67bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f291c00fa60 con 0x562828d5d590 2024-03-26T17:52:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f291c00f2d0 con 0x562828d58420 2024-03-26T17:52:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f291c00c510 con 0x562828d57a30 2024-03-26T17:52:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f2938826740 10 client.0 _ll_register_callbacks cb 0x7f2928009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d67bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.518+0000 7f86b1e58740 10 client.0 _ll_register_callbacks cb 0x563fe14a9110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f86ad008640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe158e580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f86a4098200 tx=0x7f86a409b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=619a37c2a11e5215 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f8690ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:45.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d67bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57522/0 (socket says 172.21.15.105:57522) 2024-03-26T17:52:45.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d66bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292ffff640 1 -- 172.21.15.105:0/3070739265 learned_addr learned my addr 172.21.15.105:0/3070739265 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f869800f900 con 0x563fe15845d0 2024-03-26T17:52:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f8690ff9640 1 -- 172.21.15.105:0/3816672122 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4020050 con 0x563fe15845d0 2024-03-26T17:52:45.521 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f8690ff9640 1 -- 172.21.15.105:0/3816672122 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f86a4004320 con 0x563fe15845d0 2024-03-26T17:52:45.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292effd640 1 -- 172.21.15.105:0/3070739265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 msgr2=0x562828d66670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292effd640 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d66670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292effd640 1 -- 172.21.15.105:0/3070739265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 msgr2=0x562828d67bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292effd640 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d67bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292effd640 1 -- 172.21.15.105:0/3070739265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562828d681b0 con 0x562828d58420 2024-03-26T17:52:45.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292ffff640 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d67bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.522 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f8690ff9640 1 -- 172.21.15.105:0/3816672122 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f86a409b830 con 0x563fe15845d0 2024-03-26T17:52:45.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f292effd640 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d66bb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f29240eee00 tx=0x7f292402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6190d76e521d3c1e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86b1e58740 10 client.30512 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=34}) -- 0x563fe12c9620 con 0x563fe15845d0 2024-03-26T17:52:45.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.519+0000 7f290effd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:45.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f290effd640 1 -- 172.21.15.105:0/3070739265 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29240be0e0 con 0x562828d58420 2024-03-26T17:52:45.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 1 -- 172.21.15.105:0/3070739265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f292001f040 con 0x562828d58420 2024-03-26T17:52:45.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f290effd640 1 -- 172.21.15.105:0/3070739265 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f29240ba050 con 0x562828d58420 2024-03-26T17:52:45.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f290effd640 1 -- 172.21.15.105:0/3070739265 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f29240bd9e0 con 0x562828d58420 2024-03-26T17:52:45.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 10 client.30515 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 1 -- 172.21.15.105:0/3070739265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562828d694e0 con 0x562828d58420 2024-03-26T17:52:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f8690ff9640 1 -- 172.21.15.105:0/3816672122 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f86a4097ea0 con 0x563fe15845d0 2024-03-26T17:52:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86b1e58740 10 client.30512 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86b1e58740 10 client.30512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:45.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f290effd640 1 -- 172.21.15.105:0/3070739265 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f292001f040 con 0x562828d58420 2024-03-26T17:52:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86b1e58740 20 client.30512 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86b1e58740 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563fe159a560 0x563fe159ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86b1e58740 4 client.30512 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fe14f65f0 con 0x563fe159a560 2024-03-26T17:52:45.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f290effd640 1 -- 172.21.15.105:0/3070739265 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f29240c5f60 con 0x562828d58420 2024-03-26T17:52:45.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 10 client.30515 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 10 client.30515 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:52:45.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 20 client.30515 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562828dd4490 0x562828dd68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 4 client.30515 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:45.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f2938826740 1 -- 172.21.15.105:0/3070739265 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562828cc4200 con 0x562828dd4490 2024-03-26T17:52:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.520+0000 7f86ac807640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563fe159a560 0x563fe159ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.521+0000 7f292f7fe640 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562828dd4490 0x562828dd68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.521+0000 7f8690ff9640 10 client.30512 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.521+0000 7f86ac807640 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563fe159a560 0x563fe159ca20 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f775ef44e1a4ac2 server_cookie=3b6488631cdb177c in_seq=0 out_seq=0 2024-03-26T17:52:45.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.521+0000 7f290effd640 10 client.30515 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:52:45.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.521+0000 7f292f7fe640 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562828dd4490 0x562828dd68b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=873abdd4a620f32e server_cookie=942157a1e17d9e51 in_seq=0 out_seq=0 2024-03-26T17:52:45.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.521+0000 7f290effd640 1 -- 172.21.15.105:0/3070739265 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562828cc4200 con 0x562828dd4490 2024-03-26T17:52:45.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.521+0000 7f290effd640 10 client.30515 handle_command_reply: tid=0 2024-03-26T17:52:45.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.521+0000 7f2938826740 1 client.30515 shutdown 2024-03-26T17:52:45.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 -- 172.21.15.105:0/3070739265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 msgr2=0x562828d66bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d66bb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f29240eee00 tx=0x7f292402bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 -- 172.21.15.105:0/3070739265 shutdown_connections 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562828d5d590 0x562828d67bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562828dd4490 0x562828dd68b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562828d58420 0x562828d66bb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 --2- 172.21.15.105:0/3070739265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562828d57a30 0x562828d66670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 -- 172.21.15.105:0/3070739265 >> 172.21.15.105:0/3070739265 conn(0x562828d52fa0 msgr2=0x562828d51930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f290effd640 0 client.30515 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 -- 172.21.15.105:0/3070739265 shutdown_connections 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 1 -- 172.21.15.105:0/3070739265 wait complete. 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f2938826740 20 client.30515 trim_cache size 0 max 16384 2024-03-26T17:52:45.529 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f8690ff9640 1 -- 172.21.15.105:0/3816672122 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563fe14f65f0 con 0x563fe159a560 2024-03-26T17:52:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f8690ff9640 10 client.30512 handle_command_reply: tid=0 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 client.30512 shutdown 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 msgr2=0x563fe158e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe158e580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f86a4098200 tx=0x7f86a409b6f0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 shutdown_connections 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563fe159a560 0x563fe159ca20 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f8690ff9640 0 client.30512 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1593af0 0x563fe1595ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 --2- 172.21.15.105:0/3816672122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe158e580 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.522+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 >> 172.21.15.105:0/3816672122 conn(0x563fe157fb40 msgr2=0x563fe157edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 shutdown_connections 2024-03-26T17:52:45.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 -- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f29180a2fa0 msgr2=0x7f29180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f86b1e58740 1 -- 172.21.15.105:0/3816672122 wait complete. 2024-03-26T17:52:45.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f29180a2fa0 0x7f29180a5460 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x562828d58c80 tx=0x7f2928009680 comp rx=0 tx=0).stop 2024-03-26T17:52:45.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 -- 172.21.15.105:0/1702670149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 msgr2=0x7f2930198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 0x7f2930198090 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2924000c00 tx=0x7f2924005dd0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f86b1e58740 20 client.30512 trim_cache size 0 max 16384 2024-03-26T17:52:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 -- 172.21.15.105:0/1702670149 shutdown_connections 2024-03-26T17:52:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f29180a2fa0 0x7f29180a5460 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2930102370 0x7f293019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29301019b0 0x7f2930198090 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 --2- 172.21.15.105:0/1702670149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f29301007c0 0x7f2930197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.523+0000 7f2935c36640 1 -- 172.21.15.105:0/1702670149 >> 172.21.15.105:0/1702670149 conn(0x7f29300fbd50 msgr2=0x7f29300fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.524+0000 7f2935c36640 1 -- 172.21.15.105:0/1702670149 shutdown_connections 2024-03-26T17:52:45.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.524+0000 7f2935c36640 1 -- 172.21.15.105:0/1702670149 wait complete. 2024-03-26T17:52:45.535 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:52:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86b1e58740 1 Processor -- start 2024-03-26T17:52:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86b1e58740 1 -- start start 2024-03-26T17:52:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe158c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1593af0 0x563fe1585880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86b1e58740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81aabe0 con 0x563fe1584fc0 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86b1e58740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81ac7e0 con 0x563fe1593af0 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86b1e58740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81ab2b0 con 0x563fe15845d0 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86ad008640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe158c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86ad809640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1593af0 0x563fe1585880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86ac807640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86ad809640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1593af0 0x563fe1585880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35352/0 (socket says 172.21.15.105:35352) 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.548+0000 7f86ad809640 1 -- 172.21.15.105:0/4081417119 learned_addr learned my addr 172.21.15.105:0/4081417119 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f86ad809640 1 -- 172.21.15.105:0/4081417119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 msgr2=0x563fe158c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f86ad809640 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe158c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f86ad809640 1 -- 172.21.15.105:0/4081417119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 msgr2=0x563fe158e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f86ad809640 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f86ad809640 1 -- 172.21.15.105:0/4081417119 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe1580590 con 0x563fe1593af0 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f86ad008640 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe158c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f86ac807640 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f86ad809640 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1593af0 0x563fe1585880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x563fe1585820 tx=0x7f86a003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1dba248ed8b59cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f8692ffd640 1 -- 172.21.15.105:0/4081417119 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0039040 con 0x563fe1593af0 2024-03-26T17:52:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.549+0000 7f8692ffd640 1 -- 172.21.15.105:0/4081417119 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86a0034cf0 con 0x563fe1593af0 2024-03-26T17:52:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.550+0000 7f8692ffd640 1 -- 172.21.15.105:0/4081417119 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0003e10 con 0x563fe1593af0 2024-03-26T17:52:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.550+0000 7f86b1e58740 1 -- 172.21.15.105:0/4081417119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1593af0 msgr2=0x563fe1585880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.550+0000 7f86b1e58740 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1593af0 0x563fe1585880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x563fe1585820 tx=0x7f86a003f000 comp rx=0 tx=0).stop 2024-03-26T17:52:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.550+0000 7f86b1e58740 1 -- 172.21.15.105:0/4081417119 shutdown_connections 2024-03-26T17:52:45.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.550+0000 7f86b1e58740 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe1593af0 0x563fe1585880 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.550+0000 7f86b1e58740 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1584fc0 0x563fe158e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.550+0000 7f86b1e58740 1 --2- 172.21.15.105:0/4081417119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe15845d0 0x563fe158c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.550+0000 7f86b1e58740 1 -- 172.21.15.105:0/4081417119 >> 172.21.15.105:0/4081417119 conn(0x563fe157fb40 msgr2=0x563fe158e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 -- 172.21.15.105:0/4081417119 shutdown_connections 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 -- 172.21.15.105:0/4081417119 wait complete. 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 -- start start 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 Processor -- start 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe15ab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1584fc0 0x563fe15abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1593af0 0x563fe15acab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81ac110 con 0x563fe1593af0 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81ab980 con 0x563fe15845d0 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81a8bc0 con 0x563fe1584fc0 2024-03-26T17:52:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.551+0000 7f86b1e58740 10 client.0 _ll_register_callbacks cb 0x7f86a40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe15ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe15ab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35358/0 (socket says 172.21.15.105:35358) 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 -- 172.21.15.105:0/2415133374 learned_addr learned my addr 172.21.15.105:0/2415133374 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad809640 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1593af0 0x563fe15acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ac807640 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1584fc0 0x563fe15abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 -- 172.21.15.105:0/2415133374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1584fc0 msgr2=0x563fe15abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1584fc0 0x563fe15abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 -- 172.21.15.105:0/2415133374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1593af0 msgr2=0x563fe15acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1593af0 0x563fe15acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 -- 172.21.15.105:0/2415133374 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe15ad0a0 con 0x563fe15845d0 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86ad008640 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe15ab560 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86a4004610 tx=0x7f86a40ccad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be5cfd3e9c9569d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f8690ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f8690ff9640 1 -- 172.21.15.105:0/2415133374 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4020070 con 0x563fe15845d0 2024-03-26T17:52:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.552+0000 7f86b1e58740 1 -- 172.21.15.105:0/2415133374 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a0034cf0 con 0x563fe15845d0 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f8690ff9640 1 -- 172.21.15.105:0/2415133374 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f86a40ccd10 con 0x563fe15845d0 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f8690ff9640 1 -- 172.21.15.105:0/2415133374 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f86a409f980 con 0x563fe15845d0 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86b1e58740 10 client.30521 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86b1e58740 1 -- 172.21.15.105:0/2415133374 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563fe1600640 con 0x563fe15845d0 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f8690ff9640 1 -- 172.21.15.105:0/2415133374 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86a0034cf0 con 0x563fe15845d0 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f8690ff9640 1 -- 172.21.15.105:0/2415133374 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f86a409cb60 con 0x563fe15845d0 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86b1e58740 10 client.30521 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86b1e58740 10 client.30521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86b1e58740 20 client.30521 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86b1e58740 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563fe1600fe0 0x563fe1603400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86b1e58740 4 client.30521 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86b1e58740 1 -- 172.21.15.105:0/2415133374 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fe14f0940 con 0x563fe1600fe0 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.553+0000 7f86ac807640 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563fe1600fe0 0x563fe1603400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.554+0000 7f8690ff9640 10 client.30521 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.554+0000 7f86ac807640 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563fe1600fe0 0x563fe1603400 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc5034957a1a8211 server_cookie=93436a86113e87 in_seq=0 out_seq=0 2024-03-26T17:52:45.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 11805 9.00 MB/sec warmup 88 sec latency 101.958 ms 2024-03-26T17:52:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f8690ff9640 1 -- 172.21.15.105:0/2415133374 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fe14f0940 con 0x563fe1600fe0 2024-03-26T17:52:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f8690ff9640 10 client.30521 handle_command_reply: tid=0 2024-03-26T17:52:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 client.30521 shutdown 2024-03-26T17:52:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 -- 172.21.15.105:0/2415133374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 msgr2=0x563fe15ab560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe15ab560 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86a4004610 tx=0x7f86a40ccad0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 -- 172.21.15.105:0/2415133374 shutdown_connections 2024-03-26T17:52:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563fe1600fe0 0x563fe1603400 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.660 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f8690ff9640 0 client.30521 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fe1593af0 0x563fe15acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fe1584fc0 0x563fe15abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 --2- 172.21.15.105:0/2415133374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fe15845d0 0x563fe15ab560 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 -- 172.21.15.105:0/2415133374 >> 172.21.15.105:0/2415133374 conn(0x563fe157fb40 msgr2=0x563fe158e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 -- 172.21.15.105:0/2415133374 shutdown_connections 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 1 -- 172.21.15.105:0/2415133374 wait complete. 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.668+0000 7f86b1e58740 20 client.30521 trim_cache size 0 max 16384 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:52:45.661 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:52:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:52:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 -- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86940a2fc0 msgr2=0x7f86940a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.662 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86940a2fc0 0x7f86940a5480 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f86a0003b30 tx=0x7f86a00098c0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 -- 172.21.15.105:0/2282679673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86a8101910 msgr2=0x7f86a8199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86a8101910 0x7f86a8199a10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f86a400d6a0 tx=0x7f86a400db60 comp rx=0 tx=0).stop 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 -- 172.21.15.105:0/2282679673 shutdown_connections 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f86940a2fc0 0x7f86940a5480 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86a81034c0 0x7f86a81a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86a8102b00 0x7f86a8199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 --2- 172.21.15.105:0/2282679673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86a8101910 0x7f86a8199a10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 -- 172.21.15.105:0/2282679673 >> 172.21.15.105:0/2282679673 conn(0x7f86a80fce80 msgr2=0x7f86a80fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.663 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 -- 172.21.15.105:0/2282679673 shutdown_connections 2024-03-26T17:52:45.664 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:45.669+0000 7f86af26c640 1 -- 172.21.15.105:0/2282679673 wait complete. 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 -- 172.21.15.105:0/4048105372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 msgr2=0x7f168c107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 --2- 172.21.15.105:0/4048105372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 0x7f168c107100 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f167c009860 tx=0x7f167c02bb90 comp rx=0 tx=0).stop 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 -- 172.21.15.105:0/4048105372 shutdown_connections 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 --2- 172.21.15.105:0/4048105372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c107640 0x7f168c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 --2- 172.21.15.105:0/4048105372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 0x7f168c107100 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 --2- 172.21.15.105:0/4048105372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f168c1023e0 0x7f168c1047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 -- 172.21.15.105:0/4048105372 >> 172.21.15.105:0/4048105372 conn(0x7f168c0fbd50 msgr2=0x7f168c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 -- 172.21.15.105:0/4048105372 shutdown_connections 2024-03-26T17:52:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.688+0000 7f1693630640 1 -- 172.21.15.105:0/4048105372 wait complete. 2024-03-26T17:52:45.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1693630640 1 Processor -- start 2024-03-26T17:52:45.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1693630640 1 -- start start 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1693630640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f168c1023e0 0x7f168c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1693630640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 0x7f168c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1693630640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c107640 0x7f168c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1693630640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f168c10d3c0 con 0x7f168c1023e0 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1693630640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f168c10d240 con 0x7f168c104d10 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1693630640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f168c10d540 con 0x7f168c107640 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f16913cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f168c1023e0 0x7f168c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f16913cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f168c1023e0 0x7f168c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57546/0 (socket says 172.21.15.105:57546) 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f16913cc640 1 -- 172.21.15.105:0/3484697823 learned_addr learned my addr 172.21.15.105:0/3484697823 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.689+0000 7f1690bcb640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 0x7f168c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f1691bcd640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c107640 0x7f168c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16913cc640 1 -- 172.21.15.105:0/3484697823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c107640 msgr2=0x7f168c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16913cc640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c107640 0x7f168c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16913cc640 1 -- 172.21.15.105:0/3484697823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 msgr2=0x7f168c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16913cc640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 0x7f168c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16913cc640 1 -- 172.21.15.105:0/3484697823 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c19f780 con 0x7f168c1023e0 2024-03-26T17:52:45.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f1690bcb640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 0x7f168c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16913cc640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f168c1023e0 0x7f168c197b60 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f168800e9c0 tx=0x7f168800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16827fc640 1 -- 172.21.15.105:0/3484697823 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168800cd80 con 0x7f168c1023e0 2024-03-26T17:52:45.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f1693630640 1 -- 172.21.15.105:0/3484697823 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c19fa70 con 0x7f168c1023e0 2024-03-26T17:52:45.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f1693630640 1 -- 172.21.15.105:0/3484697823 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c19ffe0 con 0x7f168c1023e0 2024-03-26T17:52:45.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16827fc640 1 -- 172.21.15.105:0/3484697823 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16880035f0 con 0x7f168c1023e0 2024-03-26T17:52:45.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.690+0000 7f16827fc640 1 -- 172.21.15.105:0/3484697823 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688010690 con 0x7f168c1023e0 2024-03-26T17:52:45.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.694+0000 7f16827fc640 1 -- 172.21.15.105:0/3484697823 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f16880037a0 con 0x7f168c1023e0 2024-03-26T17:52:45.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.694+0000 7f1696216740 1 Processor -- start 2024-03-26T17:52:45.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.694+0000 7f1696216740 1 -- start start 2024-03-26T17:52:45.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f1696216740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e64e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f1696216740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e69f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f1696216740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f1696216740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555cc1dc9970 con 0x555cc1e6a590 2024-03-26T17:52:45.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f1696216740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555cc1d71d70 con 0x555cc1e64a30 2024-03-26T17:52:45.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f1696216740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555cc1dc5920 con 0x555cc1e65420 2024-03-26T17:52:45.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f16913cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e69f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f1690bcb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f16913cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e69f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52894/0 (socket says 172.21.15.105:52894) 2024-03-26T17:52:45.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f16913cc640 1 -- 172.21.15.105:0/3717999722 learned_addr learned my addr 172.21.15.105:0/3717999722 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f16827fc640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16780a7330 0x7f16780a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f1691bcd640 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e64e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f16827fc640 1 -- 172.21.15.105:0/3484697823 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f16880cf710 con 0x7f168c1023e0 2024-03-26T17:52:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f16913cc640 1 -- 172.21.15.105:0/3717999722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 msgr2=0x555cc1e64e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f16913cc640 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e64e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.695+0000 7f16913cc640 1 -- 172.21.15.105:0/3717999722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 msgr2=0x555cc1e6ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.696+0000 7f16913cc640 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.696+0000 7f16913cc640 1 -- 172.21.15.105:0/3717999722 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc1add3f0 con 0x555cc1e65420 2024-03-26T17:52:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.696+0000 7f1690bcb640 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.696+0000 7f1691bcd640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16780a7330 0x7f16780a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.696+0000 7f16913cc640 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e69f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f168809baa0 tx=0x7f1688020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abee02ab7e2d459f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.697+0000 7f16777fe640 1 -- 172.21.15.105:0/3717999722 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168809b700 con 0x555cc1e65420 2024-03-26T17:52:45.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.697+0000 7f16777fe640 1 -- 172.21.15.105:0/3717999722 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1688003b10 con 0x555cc1e65420 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.697+0000 7f16777fe640 1 -- 172.21.15.105:0/3717999722 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16880cd280 con 0x555cc1e65420 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 -- 172.21.15.105:0/3717999722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 msgr2=0x555cc1e69f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e69f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f168809baa0 tx=0x7f1688020000 comp rx=0 tx=0).stop 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1691bcd640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16780a7330 0x7f16780a97f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f16840049d0 tx=0x7f1684004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 -- 172.21.15.105:0/3717999722 shutdown_connections 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e69f20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 --2- 172.21.15.105:0/3717999722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e64e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 -- 172.21.15.105:0/3717999722 >> 172.21.15.105:0/3717999722 conn(0x555cc1e5ffa0 msgr2=0x555cc1e603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 -- 172.21.15.105:0/3717999722 shutdown_connections 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 -- 172.21.15.105:0/3717999722 wait complete. 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 -- start start 2024-03-26T17:52:45.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.698+0000 7f1696216740 1 Processor -- start 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1696216740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1696216740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e6f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1696216740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e76490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1696216740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555cc1e6d480 con 0x555cc1e6a590 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1696216740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555cc1d85310 con 0x555cc1e64a30 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1696216740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555cc1e6d600 con 0x555cc1e65420 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1696216740 10 client.0 _ll_register_callbacks cb 0x555cc1dc0c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1691bcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e76490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f16913cc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f16913cc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35390/0 (socket says 172.21.15.105:35390) 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1691bcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e76490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57568/0 (socket says 172.21.15.105:57568) 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f16913cc640 1 -- 172.21.15.105:0/3807851042 learned_addr learned my addr 172.21.15.105:0/3807851042 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.699+0000 7f1690bcb640 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e6f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f16913cc640 1 -- 172.21.15.105:0/3807851042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 msgr2=0x555cc1e6f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f16913cc640 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e6f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f16913cc640 1 -- 172.21.15.105:0/3807851042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 msgr2=0x555cc1e76490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f16913cc640 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e76490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f16913cc640 1 -- 172.21.15.105:0/3807851042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc1ba77d0 con 0x555cc1e64a30 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f1691bcd640 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e76490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f16913cc640 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6ef10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x555cc1e6d210 tx=0x7f168809ff60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7fdd1b171b031e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f1674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f1674ff9640 1 -- 172.21.15.105:0/3807851042 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168809bda0 con 0x555cc1e64a30 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f1674ff9640 1 -- 172.21.15.105:0/3807851042 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f168809b1f0 con 0x555cc1e64a30 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f1696216740 1 -- 172.21.15.105:0/3807851042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555cc1dd6eb0 con 0x555cc1e64a30 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f1674ff9640 1 -- 172.21.15.105:0/3807851042 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f168809c4a0 con 0x555cc1e64a30 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f1696216740 10 client.30533 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.700+0000 7f1696216740 1 -- 172.21.15.105:0/3807851042 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555cc1e79800 con 0x555cc1e64a30 2024-03-26T17:52:45.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1674ff9640 1 -- 172.21.15.105:0/3807851042 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555cc1dd6eb0 con 0x555cc1e64a30 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1674ff9640 1 -- 172.21.15.105:0/3807851042 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1688097b60 con 0x555cc1e64a30 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1696216740 10 client.30533 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1696216740 10 client.30533 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1696216740 20 client.30533 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1696216740 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x555cc1e7ab50 0x555cc1e7cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1696216740 4 client.30533 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1696216740 1 -- 172.21.15.105:0/3807851042 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc1dd1200 con 0x555cc1e7ab50 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.701+0000 7f1690bcb640 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x555cc1e7ab50 0x555cc1e7cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.702+0000 7f1674ff9640 10 client.30533 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:52:45.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.702+0000 7f1690bcb640 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x555cc1e7ab50 0x555cc1e7cf40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15cc4471d41276f7 server_cookie=1d01adadbe5c9db4 in_seq=0 out_seq=0 2024-03-26T17:52:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1674ff9640 1 -- 172.21.15.105:0/3807851042 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555cc1dd1200 con 0x555cc1e7ab50 2024-03-26T17:52:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1674ff9640 10 client.30533 handle_command_reply: tid=0 2024-03-26T17:52:45.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 client.30533 shutdown 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 -- 172.21.15.105:0/3807851042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 msgr2=0x555cc1e6ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6ef10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x555cc1e6d210 tx=0x7f168809ff60 comp rx=0 tx=0).stop 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 -- 172.21.15.105:0/3807851042 shutdown_connections 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x555cc1e7ab50 0x555cc1e7cf40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e76490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e6f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 --2- 172.21.15.105:0/3807851042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6ef10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1696216740 1 -- 172.21.15.105:0/3807851042 >> 172.21.15.105:0/3807851042 conn(0x555cc1e5ffa0 msgr2=0x555cc1e5f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.703+0000 7f1674ff9640 0 client.30533 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.704+0000 7f1696216740 1 -- 172.21.15.105:0/3807851042 shutdown_connections 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.704+0000 7f1696216740 1 -- 172.21.15.105:0/3807851042 wait complete. 2024-03-26T17:52:45.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.704+0000 7f1696216740 20 client.30533 trim_cache size 0 max 16384 2024-03-26T17:52:45.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.727+0000 7f1696216740 1 Processor -- start 2024-03-26T17:52:45.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1696216740 1 -- start start 2024-03-26T17:52:45.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1696216740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1696216740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e76450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1696216740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1696216740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a6c30 con 0x555cc1e6a590 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1696216740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a8830 con 0x555cc1e64a30 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1696216740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a7300 con 0x555cc1e65420 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f16913cc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f16913cc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35404/0 (socket says 172.21.15.105:35404) 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1691bcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f16913cc640 1 -- 172.21.15.105:0/662038504 learned_addr learned my addr 172.21.15.105:0/662038504 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1690bcb640 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e76450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1691bcd640 1 -- 172.21.15.105:0/662038504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 msgr2=0x555cc1e76450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1691bcd640 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e76450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1691bcd640 1 -- 172.21.15.105:0/662038504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 msgr2=0x555cc1e6eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.728+0000 7f1691bcd640 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.729+0000 7f1691bcd640 1 -- 172.21.15.105:0/662038504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc1e6d740 con 0x555cc1e6a590 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.729+0000 7f1690bcb640 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e76450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.729+0000 7f1691bcd640 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6f060 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1684009fd0 tx=0x7f1684019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba09d528ceee64df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.729+0000 7f1676ffd640 1 -- 172.21.15.105:0/662038504 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168402e070 con 0x555cc1e6a590 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.729+0000 7f1676ffd640 1 -- 172.21.15.105:0/662038504 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1684029910 con 0x555cc1e6a590 2024-03-26T17:52:45.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.729+0000 7f1676ffd640 1 -- 172.21.15.105:0/662038504 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684010950 con 0x555cc1e6a590 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.730+0000 7f1696216740 1 -- 172.21.15.105:0/662038504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 msgr2=0x555cc1e6f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.730+0000 7f1696216740 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6f060 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1684009fd0 tx=0x7f1684019000 comp rx=0 tx=0).stop 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.730+0000 7f1696216740 1 -- 172.21.15.105:0/662038504 shutdown_connections 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.730+0000 7f1696216740 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e6a590 0x555cc1e6f060 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.730+0000 7f1696216740 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e65420 0x555cc1e76450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.730+0000 7f1696216740 1 --2- 172.21.15.105:0/662038504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e64a30 0x555cc1e6eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.731+0000 7f1696216740 1 -- 172.21.15.105:0/662038504 >> 172.21.15.105:0/662038504 conn(0x555cc1e5ffa0 msgr2=0x555cc1e6d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.731+0000 7f1696216740 1 -- 172.21.15.105:0/662038504 shutdown_connections 2024-03-26T17:52:45.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.731+0000 7f1696216740 1 -- 172.21.15.105:0/662038504 wait complete. 2024-03-26T17:52:45.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.731+0000 7f1696216740 1 -- start start 2024-03-26T17:52:45.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.731+0000 7f1696216740 1 Processor -- start 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.732+0000 7f1696216740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e64a30 0x555cc1e731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.732+0000 7f1696216740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e65420 0x555cc1e73710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f16913cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e64a30 0x555cc1e731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f16913cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e64a30 0x555cc1e731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52914/0 (socket says 172.21.15.105:52914) 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f1696216740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e6a590 0x555cc1ee7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a8160 con 0x555cc1e65420 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f16913cc640 1 -- 172.21.15.105:0/811370902 learned_addr learned my addr 172.21.15.105:0/811370902 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a79d0 con 0x555cc1e6a590 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a4c10 con 0x555cc1e64a30 2024-03-26T17:52:45.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f1696216740 10 client.0 _ll_register_callbacks cb 0x7f1688005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f1690bcb640 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e65420 0x555cc1e73710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f1691bcd640 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e6a590 0x555cc1ee7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f16913cc640 1 -- 172.21.15.105:0/811370902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e6a590 msgr2=0x555cc1ee7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f16913cc640 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e6a590 0x555cc1ee7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f16913cc640 1 -- 172.21.15.105:0/811370902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e65420 msgr2=0x555cc1e73710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f16913cc640 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e65420 0x555cc1e73710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f16913cc640 1 -- 172.21.15.105:0/811370902 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc1ee7a90 con 0x555cc1e64a30 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.733+0000 7f1690bcb640 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e65420 0x555cc1e73710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.734+0000 7f16913cc640 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e64a30 0x555cc1e731d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1688098770 tx=0x7f16880cd210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f3d429de0cd83f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.734+0000 7f1674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.735+0000 7f1674ff9640 1 -- 172.21.15.105:0/811370902 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168809b700 con 0x555cc1e64a30 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.735+0000 7f1674ff9640 1 -- 172.21.15.105:0/811370902 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f168809b400 con 0x555cc1e64a30 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.735+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1684029910 con 0x555cc1e64a30 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.736+0000 7f1674ff9640 1 -- 172.21.15.105:0/811370902 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f168809b010 con 0x555cc1e64a30 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.736+0000 7f1696216740 10 client.30526 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.736+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555cc1ee8b40 con 0x555cc1e64a30 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.736+0000 7f1674ff9640 1 -- 172.21.15.105:0/811370902 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f168807f020 con 0x555cc1e64a30 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.736+0000 7f1674ff9640 1 -- 172.21.15.105:0/811370902 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f16880a1220 con 0x555cc1e64a30 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1696216740 10 client.30526 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1696216740 10 client.30526 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1696216740 20 client.30526 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1696216740 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x555cc1ee9360 0x555cc1eeb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1696216740 4 client.30526 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:45.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1684029910 con 0x555cc1ee9360 2024-03-26T17:52:45.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1690bcb640 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x555cc1ee9360 0x555cc1eeb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1674ff9640 10 client.30526 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:52:45.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.737+0000 7f1690bcb640 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x555cc1ee9360 0x555cc1eeb780 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f53d43d73d94140 server_cookie=24c5d8be5f1a89f0 in_seq=0 out_seq=0 2024-03-26T17:52:45.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1674ff9640 1 -- 172.21.15.105:0/811370902 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1684029910 con 0x555cc1ee9360 2024-03-26T17:52:45.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1674ff9640 10 client.30526 handle_command_reply: tid=0 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 client.30526 shutdown 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e64a30 msgr2=0x555cc1e731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e64a30 0x555cc1e731d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1688098770 tx=0x7f16880cd210 comp rx=0 tx=0).stop 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 shutdown_connections 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x555cc1ee9360 0x555cc1eeb780 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1674ff9640 0 client.30526 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cc1e6a590 0x555cc1ee7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cc1e65420 0x555cc1e73710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 --2- 172.21.15.105:0/811370902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cc1e64a30 0x555cc1e731d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 >> 172.21.15.105:0/811370902 conn(0x555cc1e5ffa0 msgr2=0x555cc1e6eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 shutdown_connections 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 1 -- 172.21.15.105:0/811370902 wait complete. 2024-03-26T17:52:45.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.738+0000 7f1696216740 20 client.30526 trim_cache size 0 max 16384 2024-03-26T17:52:45.732 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 -- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16780a7330 msgr2=0x7f16780a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16780a7330 0x7f16780a97f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f16840049d0 tx=0x7f1684004a40 comp rx=0 tx=0).stop 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 -- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f168c1023e0 msgr2=0x7f168c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f168c1023e0 0x7f168c197b60 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f168800e9c0 tx=0x7f168800edc0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 -- 172.21.15.105:0/3484697823 shutdown_connections 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f16780a7330 0x7f16780a97f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f168c107640 0x7f168c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f168c104d10 0x7f168c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 --2- 172.21.15.105:0/3484697823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f168c1023e0 0x7f168c197b60 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 -- 172.21.15.105:0/3484697823 >> 172.21.15.105:0/3484697823 conn(0x7f168c0fbd50 msgr2=0x7f168c104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 -- 172.21.15.105:0/3484697823 shutdown_connections 2024-03-26T17:52:45.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.739+0000 7f1693630640 1 -- 172.21.15.105:0/3484697823 wait complete. 2024-03-26T17:52:45.747 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 -- 172.21.15.105:0/183218057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 msgr2=0x7ff8a8104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 --2- 172.21.15.105:0/183218057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 0x7ff8a8104730 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009900 tx=0x7ff8a402bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 -- 172.21.15.105:0/183218057 shutdown_connections 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 --2- 172.21.15.105:0/183218057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a81075a0 0x7ff8a81099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 --2- 172.21.15.105:0/183218057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8a8104c70 0x7ff8a8107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 --2- 172.21.15.105:0/183218057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 0x7ff8a8104730 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 -- 172.21.15.105:0/183218057 >> 172.21.15.105:0/183218057 conn(0x7ff8a80fbd50 msgr2=0x7ff8a80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 -- 172.21.15.105:0/183218057 shutdown_connections 2024-03-26T17:52:45.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.942+0000 7ff8afa19640 1 -- 172.21.15.105:0/183218057 wait complete. 2024-03-26T17:52:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8afa19640 1 Processor -- start 2024-03-26T17:52:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8afa19640 1 -- start start 2024-03-26T17:52:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8afa19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 0x7ff8a8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8afa19640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a8104c70 0x7ff8a819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8afa19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8a81075a0 0x7ff8a81a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8afa19640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a8100890 con 0x7ff8a8102340 2024-03-26T17:52:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8afa19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a8100710 con 0x7ff8a81075a0 2024-03-26T17:52:45.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8afa19640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a8100a10 con 0x7ff8a8104c70 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8acfb4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a8104c70 0x7ff8a819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8ad7b5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 0x7ff8a8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8acfb4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a8104c70 0x7ff8a819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52928/0 (socket says 172.21.15.105:52928) 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8adfb6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8a81075a0 0x7ff8a81a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.943+0000 7ff8acfb4640 1 -- 172.21.15.105:0/983120231 learned_addr learned my addr 172.21.15.105:0/983120231 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/983120231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a8104c70 msgr2=0x7ff8a819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a8104c70 0x7ff8a819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/983120231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8a81075a0 msgr2=0x7ff8a81a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8a81075a0 0x7ff8a81a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/983120231 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a81a19b0 con 0x7ff8a8102340 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 0x7ff8a8199da0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff8a402c170 tx=0x7ff8a4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff89e7fc640 1 -- 172.21.15.105:0/983120231 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4043020 con 0x7ff8a8102340 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff8afa19640 1 -- 172.21.15.105:0/983120231 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a81a1c40 con 0x7ff8a8102340 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff89e7fc640 1 -- 172.21.15.105:0/983120231 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8a40043d0 con 0x7ff8a8102340 2024-03-26T17:52:45.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff89e7fc640 1 -- 172.21.15.105:0/983120231 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4003ae0 con 0x7ff8a8102340 2024-03-26T17:52:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.944+0000 7ff8afa19640 1 -- 172.21.15.105:0/983120231 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a81a21b0 con 0x7ff8a8102340 2024-03-26T17:52:45.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.947+0000 7ff89e7fc640 1 -- 172.21.15.105:0/983120231 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff8a4046020 con 0x7ff8a8102340 2024-03-26T17:52:45.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8b2609740 1 Processor -- start 2024-03-26T17:52:45.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8b2609740 1 -- start start 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387808a30 0x564387808e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x56438780df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387810a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8b2609740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56438776d970 con 0x564387808a30 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8b2609740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564387715d70 con 0x56438780e590 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8b2609740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564387769920 con 0x564387809420 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8adfb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387808a30 0x564387808e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8adfb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387808a30 0x564387808e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57604/0 (socket says 172.21.15.105:57604) 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8adfb6640 1 -- 172.21.15.105:0/3287649093 learned_addr learned my addr 172.21.15.105:0/3287649093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x56438780df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff89e7fc640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8940a2fa0 0x7ff8940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.948+0000 7ff8acfb4640 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387810a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8adfb6640 1 -- 172.21.15.105:0/3287649093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 msgr2=0x56438780df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x56438780df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff89e7fc640 1 -- 172.21.15.105:0/983120231 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff8a40f1530 con 0x7ff8a8102340 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8adfb6640 1 -- 172.21.15.105:0/3287649093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 msgr2=0x564387810a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387810a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8adfb6640 1 -- 172.21.15.105:0/3287649093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643874813f0 con 0x564387808a30 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8acfb4640 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387810a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387808a30 0x564387808e50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009fd0 tx=0x7ff8a00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edee41c7b28e70e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8940a2fa0 0x7ff8940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8937fe640 1 -- 172.21.15.105:0/3287649093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0004250 con 0x564387808a30 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8937fe640 1 -- 172.21.15.105:0/3287649093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8a0003dd0 con 0x564387808a30 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8937fe640 1 -- 172.21.15.105:0/3287649093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a001bc10 con 0x564387808a30 2024-03-26T17:52:45.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.949+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8940a2fa0 0x7ff8940a5460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0020000 tx=0x7ff8a0021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.950+0000 7ff8b2609740 1 -- 172.21.15.105:0/3287649093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387808a30 msgr2=0x564387808e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.950+0000 7ff8b2609740 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387808a30 0x564387808e50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009fd0 tx=0x7ff8a00118d0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.950+0000 7ff8b2609740 1 -- 172.21.15.105:0/3287649093 shutdown_connections 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.950+0000 7ff8b2609740 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387810a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.950+0000 7ff8b2609740 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x56438780df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.950+0000 7ff8b2609740 1 --2- 172.21.15.105:0/3287649093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387808a30 0x564387808e50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.950+0000 7ff8b2609740 1 -- 172.21.15.105:0/3287649093 >> 172.21.15.105:0/3287649093 conn(0x564387803fa0 msgr2=0x5643878043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 -- 172.21.15.105:0/3287649093 shutdown_connections 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 -- 172.21.15.105:0/3287649093 wait complete. 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 -- start start 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 Processor -- start 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387808a30 0x564387812fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387809420 0x5643878134e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56438780e590 0x56438781a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564387811540 con 0x564387809420 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564387729310 con 0x564387808a30 2024-03-26T17:52:45.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.951+0000 7ff8b2609740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5643878116c0 con 0x56438780e590 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8acfb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387809420 0x5643878134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8b2609740 10 client.0 _ll_register_callbacks cb 0x564387764c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8acfb4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387809420 0x5643878134e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57616/0 (socket says 172.21.15.105:57616) 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8acfb4640 1 -- 172.21.15.105:0/1817617120 learned_addr learned my addr 172.21.15.105:0/1817617120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56438780e590 0x56438781a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387808a30 0x564387812fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/1817617120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56438780e590 msgr2=0x56438781a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56438780e590 0x56438781a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/1817617120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387809420 msgr2=0x5643878134e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387809420 0x5643878134e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/1817617120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438754b7d0 con 0x564387808a30 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387808a30 0x564387812fa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40ef040 tx=0x7ff8a40f4d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0160b82ce4670be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff890ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8acfb4640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387809420 0x5643878134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56438780e590 0x56438781a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff890ff9640 1 -- 172.21.15.105:0/1817617120 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40bd760 con 0x564387808a30 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff8b2609740 1 -- 172.21.15.105:0/1817617120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a0003dd0 con 0x564387808a30 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.952+0000 7ff890ff9640 1 -- 172.21.15.105:0/1817617120 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff8a40c63a0 con 0x564387808a30 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff890ff9640 1 -- 172.21.15.105:0/1817617120 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff8a40bbd60 con 0x564387808a30 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8b2609740 10 client.30551 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8b2609740 1 -- 172.21.15.105:0/1817617120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56438781d890 con 0x564387808a30 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff890ff9640 1 -- 172.21.15.105:0/1817617120 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8a0003dd0 con 0x564387808a30 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff890ff9640 1 -- 172.21.15.105:0/1817617120 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff8a40d5030 con 0x564387808a30 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8b2609740 10 client.30551 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8b2609740 10 client.30551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8b2609740 20 client.30551 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8b2609740 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56438781eee0 0x564387821380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8b2609740 4 client.30551 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:45.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8b2609740 1 -- 172.21.15.105:0/1817617120 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56438777aeb0 con 0x56438781eee0 2024-03-26T17:52:45.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.953+0000 7ff8acfb4640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56438781eee0 0x564387821380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.954+0000 7ff890ff9640 10 client.30551 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:45.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.954+0000 7ff8acfb4640 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56438781eee0 0x564387821380 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ec7045ebcf09ee server_cookie=e3c32e52b723571c in_seq=0 out_seq=0 2024-03-26T17:52:45.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.955+0000 7ff890ff9640 1 -- 172.21.15.105:0/1817617120 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56438777aeb0 con 0x56438781eee0 2024-03-26T17:52:45.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.955+0000 7ff890ff9640 10 client.30551 handle_command_reply: tid=0 2024-03-26T17:52:45.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.955+0000 7ff8b2609740 1 client.30551 shutdown 2024-03-26T17:52:45.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 -- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387808a30 msgr2=0x564387812fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387808a30 0x564387812fa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40ef040 tx=0x7ff8a40f4d00 comp rx=0 tx=0).stop 2024-03-26T17:52:45.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 -- 172.21.15.105:0/1817617120 shutdown_connections 2024-03-26T17:52:45.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56438781eee0 0x564387821380 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56438780e590 0x56438781a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387809420 0x5643878134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff890ff9640 0 client.30551 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:45.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 --2- 172.21.15.105:0/1817617120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387808a30 0x564387812fa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 -- 172.21.15.105:0/1817617120 >> 172.21.15.105:0/1817617120 conn(0x564387803fa0 msgr2=0x5643878031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 -- 172.21.15.105:0/1817617120 shutdown_connections 2024-03-26T17:52:45.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 1 -- 172.21.15.105:0/1817617120 wait complete. 2024-03-26T17:52:45.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.956+0000 7ff8b2609740 20 client.30551 trim_cache size 0 max 16384 2024-03-26T17:52:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.978+0000 7ff8b2609740 1 Processor -- start 2024-03-26T17:52:45.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.978+0000 7ff8b2609740 1 -- start start 2024-03-26T17:52:45.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.978+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387808a30 0x5643878132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.978+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387809420 0x56438781a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.978+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56438780e590 0x564387802930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8b2609740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a81aaff0 con 0x56438780e590 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8b2609740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a81acbf0 con 0x564387809420 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8b2609740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a81ab6c0 con 0x564387808a30 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387808a30 0x5643878132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387808a30 0x5643878132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52956/0 (socket says 172.21.15.105:52956) 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/2257384221 learned_addr learned my addr 172.21.15.105:0/2257384221 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8acfb4640 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387809420 0x56438781a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56438780e590 0x564387802930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/2257384221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387809420 msgr2=0x56438781a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387809420 0x56438781a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/2257384221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56438780e590 msgr2=0x564387802930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56438780e590 0x564387802930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 -- 172.21.15.105:0/2257384221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564387821a00 con 0x564387808a30 2024-03-26T17:52:45.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387808a30 0x5643878132c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40bdbb0 tx=0x7ff8a40035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84680b59841834c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.979+0000 7ff892ffd640 1 -- 172.21.15.105:0/2257384221 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40bd580 con 0x564387808a30 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff892ffd640 1 -- 172.21.15.105:0/2257384221 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8a40f10d0 con 0x564387808a30 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff892ffd640 1 -- 172.21.15.105:0/2257384221 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40bdc20 con 0x564387808a30 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff8b2609740 1 -- 172.21.15.105:0/2257384221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387808a30 msgr2=0x5643878132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff8b2609740 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387808a30 0x5643878132c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40bdbb0 tx=0x7ff8a40035f0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff8b2609740 1 -- 172.21.15.105:0/2257384221 shutdown_connections 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff8b2609740 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56438780e590 0x564387802930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff8b2609740 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564387809420 0x56438781a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff8b2609740 1 --2- 172.21.15.105:0/2257384221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387808a30 0x5643878132c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.980+0000 7ff8b2609740 1 -- 172.21.15.105:0/2257384221 >> 172.21.15.105:0/2257384221 conn(0x564387803fa0 msgr2=0x564387812bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.981+0000 7ff8b2609740 1 -- 172.21.15.105:0/2257384221 shutdown_connections 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.981+0000 7ff8b2609740 1 -- 172.21.15.105:0/2257384221 wait complete. 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.981+0000 7ff8b2609740 1 -- start start 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.981+0000 7ff8b2609740 1 Processor -- start 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x564387817640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387817b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8b2609740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387818b90 0x564387819010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8b2609740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a81ac520 con 0x564387818b90 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8b2609740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a81abd90 con 0x56438780e590 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8b2609740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a81a8fd0 con 0x564387809420 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8b2609740 10 client.0 _ll_register_callbacks cb 0x7ff8a4004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8adfb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387818b90 0x564387819010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8acfb4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387817b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8adfb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387818b90 0x564387819010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57634/0 (socket says 172.21.15.105:57634) 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8adfb6640 1 -- 172.21.15.105:0/724672305 learned_addr learned my addr 172.21.15.105:0/724672305 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x564387817640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8adfb6640 1 -- 172.21.15.105:0/724672305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 msgr2=0x564387817640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x564387817640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8adfb6640 1 -- 172.21.15.105:0/724672305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 msgr2=0x564387817b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387817b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8adfb6640 1 -- 172.21.15.105:0/724672305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564387819550 con 0x564387818b90 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8acfb4640 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387817b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.982+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x564387817640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff8adfb6640 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387818b90 0x564387819010 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x564387809c80 tx=0x7ff8a0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858d6db57a07eb80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff890ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff890ff9640 1 -- 172.21.15.105:0/724672305 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0029040 con 0x564387818b90 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff890ff9640 1 -- 172.21.15.105:0/724672305 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff8a0004000 con 0x564387818b90 2024-03-26T17:52:45.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff8b2609740 1 -- 172.21.15.105:0/724672305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a40f10d0 con 0x564387818b90 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff890ff9640 1 -- 172.21.15.105:0/724672305 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff8a0004200 con 0x564387818b90 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff8b2609740 10 client.20775 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff8b2609740 1 -- 172.21.15.105:0/724672305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56438781a850 con 0x564387818b90 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff890ff9640 1 -- 172.21.15.105:0/724672305 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8a0041030 con 0x564387818b90 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.983+0000 7ff890ff9640 1 -- 172.21.15.105:0/724672305 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff8a0031560 con 0x564387818b90 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff8b2609740 10 client.20775 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff8b2609740 10 client.20775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff8b2609740 20 client.20775 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff8b2609740 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564387885960 0x564387887d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff8b2609740 4 client.20775 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff8b2609740 1 -- 172.21.15.105:0/724672305 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff8a40f10d0 con 0x564387885960 2024-03-26T17:52:45.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564387885960 0x564387887d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:45.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff890ff9640 10 client.20775 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:45.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.984+0000 7ff8ad7b5640 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564387885960 0x564387887d80 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3121250ca98ad0ce server_cookie=a76e988a7e09b078 in_seq=0 out_seq=0 2024-03-26T17:52:45.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.987+0000 7ff890ff9640 1 -- 172.21.15.105:0/724672305 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff8a40f10d0 con 0x564387885960 2024-03-26T17:52:45.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.987+0000 7ff890ff9640 10 client.20775 handle_command_reply: tid=0 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.987+0000 7ff8b2609740 1 client.20775 shutdown 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.987+0000 7ff8b2609740 1 -- 172.21.15.105:0/724672305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387818b90 msgr2=0x564387819010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.987+0000 7ff8b2609740 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387818b90 0x564387819010 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x564387809c80 tx=0x7ff8a0003970 comp rx=0 tx=0).stop 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff8b2609740 1 -- 172.21.15.105:0/724672305 shutdown_connections 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff8b2609740 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564387885960 0x564387887d80 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff8b2609740 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564387818b90 0x564387819010 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff8b2609740 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56438780e590 0x564387817b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff8b2609740 1 --2- 172.21.15.105:0/724672305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564387809420 0x564387817640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff8b2609740 1 -- 172.21.15.105:0/724672305 >> 172.21.15.105:0/724672305 conn(0x564387803fa0 msgr2=0x564387807200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff890ff9640 0 client.20775 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff8b2609740 1 -- 172.21.15.105:0/724672305 shutdown_connections 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.988+0000 7ff8b2609740 1 -- 172.21.15.105:0/724672305 wait complete. 2024-03-26T17:52:45.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.989+0000 7ff8b2609740 20 client.20775 trim_cache size 0 max 16384 2024-03-26T17:52:45.982 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:52:45.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.990+0000 7ff8afa19640 1 -- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8940a2fa0 msgr2=0x7ff8940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.990+0000 7ff8afa19640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8940a2fa0 0x7ff8940a5460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0020000 tx=0x7ff8a0021000 comp rx=0 tx=0).stop 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.990+0000 7ff8afa19640 1 -- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 msgr2=0x7ff8a8199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.990+0000 7ff8afa19640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 0x7ff8a8199da0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff8a402c170 tx=0x7ff8a4005e60 comp rx=0 tx=0).stop 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.990+0000 7ff8afa19640 1 -- 172.21.15.105:0/983120231 shutdown_connections 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.990+0000 7ff8afa19640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8940a2fa0 0x7ff8940a5460 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.990+0000 7ff8afa19640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8a81075a0 0x7ff8a81a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.991+0000 7ff8afa19640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a8104c70 0x7ff8a819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.991+0000 7ff8afa19640 1 --2- 172.21.15.105:0/983120231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a8102340 0x7ff8a8199da0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.991+0000 7ff8afa19640 1 -- 172.21.15.105:0/983120231 >> 172.21.15.105:0/983120231 conn(0x7ff8a80fbd50 msgr2=0x7ff8a81079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.991+0000 7ff8afa19640 1 -- 172.21.15.105:0/983120231 shutdown_connections 2024-03-26T17:52:45.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:45.991+0000 7ff8afa19640 1 -- 172.21.15.105:0/983120231 wait complete. 2024-03-26T17:52:45.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:45 smithi105 ceph-mon[32280]: pgmap v781: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-03-26T17:52:45.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1227906621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:52:46.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:45 smithi135 ceph-mon[35636]: pgmap v781: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-03-26T17:52:46.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1227906621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:52:46.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:45 smithi144 ceph-mon[36095]: pgmap v781: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-03-26T17:52:46.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1227906621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:52:46.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 11916 8.94 MB/sec warmup 89 sec latency 171.753 ms 2024-03-26T17:52:46.998 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:52:46.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:52:46.999 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:52:47.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 -- 172.21.15.105:0/2193037686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07201007c0 msgr2=0x7f0720100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 --2- 172.21.15.105:0/2193037686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07201007c0 0x7f0720100be0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0710009900 tx=0x7f071002bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 -- 172.21.15.105:0/2193037686 shutdown_connections 2024-03-26T17:52:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 --2- 172.21.15.105:0/2193037686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0720102370 0x7f0720108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 --2- 172.21.15.105:0/2193037686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07201019b0 0x7f0720101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 --2- 172.21.15.105:0/2193037686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f07201007c0 0x7f0720100be0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 -- 172.21.15.105:0/2193037686 >> 172.21.15.105:0/2193037686 conn(0x7f07200fbd50 msgr2=0x7f07200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 -- 172.21.15.105:0/2193037686 shutdown_connections 2024-03-26T17:52:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.136+0000 7f0727e85640 1 -- 172.21.15.105:0/2193037686 wait complete. 2024-03-26T17:52:47.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0727e85640 1 Processor -- start 2024-03-26T17:52:47.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0727e85640 1 -- start start 2024-03-26T17:52:47.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0727e85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07201007c0 0x7f0720197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0727e85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07201019b0 0x7f0720198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0727e85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0720102370 0x7f072019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0727e85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f072010c2f0 con 0x7f0720102370 2024-03-26T17:52:47.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0727e85640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f072010c170 con 0x7f07201007c0 2024-03-26T17:52:47.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0727e85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f072010c470 con 0x7f07201019b0 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0726422640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0720102370 0x7f072019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725420640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07201019b0 0x7f0720198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725420640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07201019b0 0x7f0720198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52990/0 (socket says 172.21.15.105:52990) 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0726422640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0720102370 0x7f072019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57656/0 (socket says 172.21.15.105:57656) 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725420640 1 -- 172.21.15.105:0/9065752 learned_addr learned my addr 172.21.15.105:0/9065752 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725c21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07201007c0 0x7f0720197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725420640 1 -- 172.21.15.105:0/9065752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07201007c0 msgr2=0x7f0720197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725420640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07201007c0 0x7f0720197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725420640 1 -- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0720102370 msgr2=0x7f072019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725420640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0720102370 0x7f072019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.137+0000 7f0725420640 1 -- 172.21.15.105:0/9065752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072019f760 con 0x7f07201019b0 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.138+0000 7f0725420640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07201019b0 0x7f0720198080 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f071c00e9f0 tx=0x7f071c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.138+0000 7f0716ffd640 1 -- 172.21.15.105:0/9065752 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c00cd80 con 0x7f07201019b0 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.138+0000 7f0727e85640 1 -- 172.21.15.105:0/9065752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072019fa50 con 0x7f07201019b0 2024-03-26T17:52:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.138+0000 7f0727e85640 1 -- 172.21.15.105:0/9065752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072019fff0 con 0x7f07201019b0 2024-03-26T17:52:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.138+0000 7f0716ffd640 1 -- 172.21.15.105:0/9065752 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f071c0108e0 con 0x7f07201019b0 2024-03-26T17:52:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.139+0000 7f0716ffd640 1 -- 172.21.15.105:0/9065752 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c018920 con 0x7f07201019b0 2024-03-26T17:52:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.141+0000 7f072aa75740 1 Processor -- start 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.141+0000 7f072aa75740 1 -- start start 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.141+0000 7f0716ffd640 1 -- 172.21.15.105:0/9065752 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f071c010430 con 0x7f07201019b0 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.141+0000 7f072aa75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 0x559b56ed95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.141+0000 7f072aa75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ede680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.141+0000 7f072aa75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56ee1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.141+0000 7f072aa75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559b56e3e630 con 0x559b56edecf0 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.141+0000 7f072aa75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559b56de6e10 con 0x559b56ed9b80 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f072aa75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559b56e3a5e0 con 0x559b56ed9190 2024-03-26T17:52:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725420640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56ee1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725420640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56ee1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57670/0 (socket says 172.21.15.105:57670) 2024-03-26T17:52:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725420640 1 -- 172.21.15.105:0/1050120441 learned_addr learned my addr 172.21.15.105:0/1050120441 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725c21640 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ede680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0726422640 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 0x559b56ed95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725420640 1 -- 172.21.15.105:0/1050120441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 msgr2=0x559b56ed95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725420640 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 0x559b56ed95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725420640 1 -- 172.21.15.105:0/1050120441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 msgr2=0x559b56ede680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725420640 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ede680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0725420640 1 -- 172.21.15.105:0/1050120441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b56dfd740 con 0x559b56edecf0 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.142+0000 7f0716ffd640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f070c0a2fa0 0x7f070c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.143+0000 7f0726422640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f070c0a2fa0 0x7f070c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.143+0000 7f0716ffd640 1 -- 172.21.15.105:0/9065752 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f071c0cf5b0 con 0x7f07201019b0 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.143+0000 7f0725420640 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56ee1180 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f071c0d5720 tx=0x7f071c0d5be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437b99e3417af06f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.143+0000 7f070bfff640 1 -- 172.21.15.105:0/1050120441 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c09bb20 con 0x559b56edecf0 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.143+0000 7f0726422640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f070c0a2fa0 0x7f070c0a5460 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x559b56eda3e0 tx=0x7f0718015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.143+0000 7f070bfff640 1 -- 172.21.15.105:0/1050120441 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f071c010720 con 0x559b56edecf0 2024-03-26T17:52:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.143+0000 7f070bfff640 1 -- 172.21.15.105:0/1050120441 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c09b6b0 con 0x559b56edecf0 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 -- 172.21.15.105:0/1050120441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 msgr2=0x559b56ee1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56ee1180 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f071c0d5720 tx=0x7f071c0d5be0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 -- 172.21.15.105:0/1050120441 shutdown_connections 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56ee1180 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ede680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 --2- 172.21.15.105:0/1050120441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 0x559b56ed95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 -- 172.21.15.105:0/1050120441 >> 172.21.15.105:0/1050120441 conn(0x559b56ed4700 msgr2=0x559b56ed4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 -- 172.21.15.105:0/1050120441 shutdown_connections 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.144+0000 7f072aa75740 1 -- 172.21.15.105:0/1050120441 wait complete. 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.145+0000 7f072aa75740 1 -- start start 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.145+0000 7f072aa75740 1 Processor -- start 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.145+0000 7f072aa75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 0x559b56ee3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.145+0000 7f072aa75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ee3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.145+0000 7f072aa75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56eeac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.145+0000 7f072aa75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559b56ee1ca0 con 0x559b56edecf0 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.145+0000 7f072aa75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559b56dfa1a0 con 0x559b56ed9b80 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.145+0000 7f0725420640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ee3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725420640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ee3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35508/0 (socket says 172.21.15.105:35508) 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725420640 1 -- 172.21.15.105:0/1356701681 learned_addr learned my addr 172.21.15.105:0/1356701681 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725c21640 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 0x559b56ee3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0726422640 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56eeac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559b56ee1e20 con 0x559b56ed9190 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f072aa75740 10 client.0 _ll_register_callbacks cb 0x559b56b51c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725420640 1 -- 172.21.15.105:0/1356701681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 msgr2=0x559b56ee3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725420640 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 0x559b56ee3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725420640 1 -- 172.21.15.105:0/1356701681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 msgr2=0x559b56eeac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725420640 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56eeac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725420640 1 -- 172.21.15.105:0/1356701681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b56c1a7e0 con 0x559b56ed9b80 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0726422640 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56eeac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f0725420640 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ee3c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f071c0d74b0 tx=0x7f071c098110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc1e3b40591a251 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f07097fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.146+0000 7f07097fa640 1 -- 172.21.15.105:0/1356701681 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c09b890 con 0x559b56ed9b80 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559b56e4bb70 con 0x559b56ed9b80 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f07097fa640 1 -- 172.21.15.105:0/1356701681 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f071c0d5220 con 0x559b56ed9b80 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f07097fa640 1 -- 172.21.15.105:0/1356701681 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f071c09b6b0 con 0x559b56ed9b80 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 10 client.30569 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559b56c7af70 con 0x559b56ed9b80 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f07097fa640 1 -- 172.21.15.105:0/1356701681 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559b56e4bb70 con 0x559b56ed9b80 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f07097fa640 1 -- 172.21.15.105:0/1356701681 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f071c0b1060 con 0x559b56ed9b80 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 10 client.30569 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 10 client.30569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 20 client.30569 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:47.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b56eef590 0x559b56ef19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 4 client.30569 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.147+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b56e45ec0 con 0x559b56eef590 2024-03-26T17:52:47.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.148+0000 7f0725c21640 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b56eef590 0x559b56ef19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.148+0000 7f0725c21640 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b56eef590 0x559b56ef19b0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c333eabd7e242bff server_cookie=f79fc365552f3bfa in_seq=0 out_seq=0 2024-03-26T17:52:47.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.148+0000 7f07097fa640 10 client.30569 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:47.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.150+0000 7f07097fa640 1 -- 172.21.15.105:0/1356701681 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559b56e45ec0 con 0x559b56eef590 2024-03-26T17:52:47.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.150+0000 7f07097fa640 10 client.30569 handle_command_reply: tid=0 2024-03-26T17:52:47.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 client.30569 shutdown 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 msgr2=0x559b56ee3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ee3c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f071c0d74b0 tx=0x7f071c098110 comp rx=0 tx=0).stop 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 shutdown_connections 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b56eef590 0x559b56ef19b0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56eeac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f07097fa640 0 client.30569 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56ee3c80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 --2- 172.21.15.105:0/1356701681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9190 0x559b56ee3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.151+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 >> 172.21.15.105:0/1356701681 conn(0x559b56ed4700 msgr2=0x559b56ed3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.152+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 shutdown_connections 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.152+0000 7f072aa75740 1 -- 172.21.15.105:0/1356701681 wait complete. 2024-03-26T17:52:47.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.152+0000 7f072aa75740 20 client.30569 trim_cache size 0 max 16384 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f072aa75740 1 Processor -- start 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f072aa75740 1 -- start start 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f072aa75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56ed9190 0x559b56ee2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f072aa75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56eeac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f072aa75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56edecf0 0x559b56ee2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f072aa75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f070400e4d0 con 0x559b56ed9190 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f072aa75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07040100d0 con 0x559b56ed9b80 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f072aa75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f070400eba0 con 0x559b56edecf0 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f0725c21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56ed9190 0x559b56ee2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f0725c21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56ed9190 0x559b56ee2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57698/0 (socket says 172.21.15.105:57698) 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f0725c21640 1 -- 172.21.15.105:0/4282297778 learned_addr learned my addr 172.21.15.105:0/4282297778 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.176+0000 7f0726422640 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56edecf0 0x559b56ee2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f0725420640 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56eeac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f0725c21640 1 -- 172.21.15.105:0/4282297778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56edecf0 msgr2=0x559b56ee2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f0725c21640 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56edecf0 0x559b56ee2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f0725c21640 1 -- 172.21.15.105:0/4282297778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 msgr2=0x559b56eeac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f0725c21640 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56eeac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f0725c21640 1 -- 172.21.15.105:0/4282297778 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b56ef1d20 con 0x559b56ed9190 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f0725c21640 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56ed9190 0x559b56ee2a30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f0710005ab0 tx=0x7f071002cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc446f4eb0c10c44 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f070b7fe640 1 -- 172.21.15.105:0/4282297778 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071004bab0 con 0x559b56ed9190 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f070b7fe640 1 -- 172.21.15.105:0/4282297778 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f071004bc50 con 0x559b56ed9190 2024-03-26T17:52:47.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.177+0000 7f070b7fe640 1 -- 172.21.15.105:0/4282297778 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100052a0 con 0x559b56ed9190 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 -- 172.21.15.105:0/4282297778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56ed9190 msgr2=0x559b56ee2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56ed9190 0x559b56ee2a30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f0710005ab0 tx=0x7f071002cbc0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 -- 172.21.15.105:0/4282297778 shutdown_connections 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56edecf0 0x559b56ee2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9b80 0x559b56eeac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 --2- 172.21.15.105:0/4282297778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56ed9190 0x559b56ee2a30 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 -- 172.21.15.105:0/4282297778 >> 172.21.15.105:0/4282297778 conn(0x559b56ed4700 msgr2=0x559b56ed4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 -- 172.21.15.105:0/4282297778 shutdown_connections 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 -- 172.21.15.105:0/4282297778 wait complete. 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.178+0000 7f072aa75740 1 -- start start 2024-03-26T17:52:47.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f072aa75740 1 Processor -- start 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f072aa75740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9190 0x559b56ee7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f072aa75740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9b80 0x559b56ee7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f072aa75740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56f538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f072aa75740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f070400fa00 con 0x559b56edecf0 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f072aa75740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f070400f270 con 0x559b56ed9190 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f072aa75740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f070400c4b0 con 0x559b56ed9b80 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725c21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9190 0x559b56ee7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f072aa75740 10 client.0 _ll_register_callbacks cb 0x7f07180093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725c21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9190 0x559b56ee7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35526/0 (socket says 172.21.15.105:35526) 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725c21640 1 -- 172.21.15.105:0/410340576 learned_addr learned my addr 172.21.15.105:0/410340576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725420640 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9b80 0x559b56ee7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0726422640 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56f538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725c21640 1 -- 172.21.15.105:0/410340576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9b80 msgr2=0x559b56ee7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725c21640 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9b80 0x559b56ee7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725c21640 1 -- 172.21.15.105:0/410340576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 msgr2=0x559b56f538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725c21640 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56f538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.179+0000 7f0725c21640 1 -- 172.21.15.105:0/410340576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b56b12170 con 0x559b56ed9190 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.180+0000 7f0725c21640 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9190 0x559b56ee7840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f071002cb90 tx=0x7f071002c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987745a81b21827f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.180+0000 7f07097fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.180+0000 7f07097fa640 1 -- 172.21.15.105:0/410340576 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710059060 con 0x559b56ed9190 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.180+0000 7f07097fa640 1 -- 172.21.15.105:0/410340576 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f071004b420 con 0x559b56ed9190 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.180+0000 7f072aa75740 1 -- 172.21.15.105:0/410340576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f071004bc50 con 0x559b56ed9190 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.180+0000 7f07097fa640 1 -- 172.21.15.105:0/410340576 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f071004b620 con 0x559b56ed9190 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.180+0000 7f072aa75740 10 client.30575 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.180+0000 7f072aa75740 1 -- 172.21.15.105:0/410340576 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559b56f54bd0 con 0x559b56ed9190 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f07097fa640 1 -- 172.21.15.105:0/410340576 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0710005500 con 0x559b56ed9190 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f07097fa640 1 -- 172.21.15.105:0/410340576 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0710054460 con 0x559b56ed9190 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f072aa75740 10 client.30575 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:47.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f072aa75740 10 client.30575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f072aa75740 20 client.30575 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f072aa75740 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b56eef590 0x559b56f576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f072aa75740 4 client.30575 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f072aa75740 1 -- 172.21.15.105:0/410340576 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f071004bc50 con 0x559b56eef590 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f0725420640 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b56eef590 0x559b56f576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.181+0000 7f07097fa640 10 client.30575 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:47.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.182+0000 7f0725420640 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b56eef590 0x559b56f576a0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bccd411f9bc3a06 server_cookie=480c6efeb065a191 in_seq=0 out_seq=0 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f07097fa640 1 -- 172.21.15.105:0/410340576 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f071004bc50 con 0x559b56eef590 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f07097fa640 10 client.30575 handle_command_reply: tid=0 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 client.30575 shutdown 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 -- 172.21.15.105:0/410340576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9190 msgr2=0x559b56ee7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9190 0x559b56ee7840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f071002cb90 tx=0x7f071002c330 comp rx=0 tx=0).stop 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 -- 172.21.15.105:0/410340576 shutdown_connections 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b56eef590 0x559b56f576a0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b56edecf0 0x559b56f538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b56ed9b80 0x559b56ee7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 --2- 172.21.15.105:0/410340576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b56ed9190 0x559b56ee7840 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 -- 172.21.15.105:0/410340576 >> 172.21.15.105:0/410340576 conn(0x559b56ed4700 msgr2=0x559b56ee2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f07097fa640 0 client.30575 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 -- 172.21.15.105:0/410340576 shutdown_connections 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 1 -- 172.21.15.105:0/410340576 wait complete. 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.185+0000 7f072aa75740 20 client.30575 trim_cache size 0 max 16384 2024-03-26T17:52:47.179 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:52:47.180 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:52:47.180 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "1e477115-2ec2-475d-9db2-a6a5dcaaae34", 2024-03-26T17:52:47.180 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:52:47.180 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 -- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f070c0a2fa0 msgr2=0x7f070c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f070c0a2fa0 0x7f070c0a5460 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x559b56eda3e0 tx=0x7f0718015040 comp rx=0 tx=0).stop 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 -- 172.21.15.105:0/9065752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07201019b0 msgr2=0x7f0720198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07201019b0 0x7f0720198080 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f071c00e9f0 tx=0x7f071c00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 -- 172.21.15.105:0/9065752 shutdown_connections 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f070c0a2fa0 0x7f070c0a5460 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0720102370 0x7f072019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07201019b0 0x7f0720198080 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 --2- 172.21.15.105:0/9065752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f07201007c0 0x7f0720197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 -- 172.21.15.105:0/9065752 >> 172.21.15.105:0/9065752 conn(0x7f07200fbd50 msgr2=0x7f0720068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 -- 172.21.15.105:0/9065752 shutdown_connections 2024-03-26T17:52:47.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.186+0000 7f0727e85640 1 -- 172.21.15.105:0/9065752 wait complete. 2024-03-26T17:52:47.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:52:47.189 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/235631297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d01019b0 msgr2=0x7fd2d0101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/235631297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d01019b0 0x7fd2d0101e30 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009900 tx=0x7fd2c402bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/235631297 shutdown_connections 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/235631297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2d0102370 0x7fd2d0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/235631297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d01019b0 0x7fd2d0101e30 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/235631297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d01007c0 0x7fd2d0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/235631297 >> 172.21.15.105:0/235631297 conn(0x7fd2d00fbd50 msgr2=0x7fd2d00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/235631297 shutdown_connections 2024-03-26T17:52:47.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.326+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/235631297 wait complete. 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2d5d1b640 1 Processor -- start 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2d5d1b640 1 -- start start 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2d5d1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2d01007c0 0x7fd2d00695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2d5d1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d01019b0 0x7fd2d019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2d5d1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d0102370 0x7fd2d01a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2d5d1b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d010c2d0 con 0x7fd2d0102370 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2d5d1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d010c150 con 0x7fd2d01007c0 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2d5d1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d010c450 con 0x7fd2d01019b0 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2d01007c0 0x7fd2d00695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d0102370 0x7fd2d01a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2ceffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d01019b0 0x7fd2d019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2d01007c0 0x7fd2d00695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35544/0 (socket says 172.21.15.105:35544) 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.327+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/3243653916 learned_addr learned my addr 172.21.15.105:0/3243653916 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2cffff640 1 -- 172.21.15.105:0/3243653916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d01019b0 msgr2=0x7fd2d019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2cffff640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d01019b0 0x7fd2d019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2cffff640 1 -- 172.21.15.105:0/3243653916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2d01007c0 msgr2=0x7fd2d00695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2cffff640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2d01007c0 0x7fd2d00695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2cffff640 1 -- 172.21.15.105:0/3243653916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d01a1910 con 0x7fd2d0102370 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2d01007c0 0x7fd2d00695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2cffff640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d0102370 0x7fd2d01a1140 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009fd0 tx=0x7fd2c800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2ccff9640 1 -- 172.21.15.105:0/3243653916 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800ce20 con 0x7fd2d0102370 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/3243653916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d01a1c00 con 0x7fd2d0102370 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2ccff9640 1 -- 172.21.15.105:0/3243653916 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c8005020 con 0x7fd2d0102370 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2ccff9640 1 -- 172.21.15.105:0/3243653916 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8010690 con 0x7fd2d0102370 2024-03-26T17:52:47.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.328+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/3243653916 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d01a2120 con 0x7fd2d0102370 2024-03-26T17:52:47.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.331+0000 7fd2ccff9640 1 -- 172.21.15.105:0/3243653916 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd2c80051d0 con 0x7fd2d0102370 2024-03-26T17:52:47.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.331+0000 7fd2d89e6740 1 Processor -- start 2024-03-26T17:52:47.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.331+0000 7fd2d89e6740 1 -- start start 2024-03-26T17:52:47.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e3d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3daf0 0x55c675e425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e42c60 0x55c675e450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c675d99980 con 0x55c675e42c60 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c675d41be0 con 0x55c675e3daf0 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c675d95930 con 0x55c675e3d100 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3daf0 0x55c675e425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3daf0 0x55c675e425f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:35554/0 (socket says 172.21.15.105:35554) 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/2587132106 learned_addr learned my addr 172.21.15.105:0/2587132106 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e42c60 0x55c675e450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2ccff9640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2b80a2f20 0x7fd2b80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/2587132106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 msgr2=0x55c675e3d520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e3d520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/2587132106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e42c60 msgr2=0x55c675e450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e42c60 0x55c675e450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/2587132106 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c675b777d0 con 0x55c675e3daf0 2024-03-26T17:52:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e42c60 0x55c675e450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.332+0000 7fd2cffff640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2b80a2f20 0x7fd2b80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.333+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3daf0 0x55c675e425f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000f570 tx=0x7fd2c000fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b5c9508c9091a82 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.333+0000 7fd2ccff9640 1 -- 172.21.15.105:0/3243653916 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd2c80cfae0 con 0x7fd2d0102370 2024-03-26T17:52:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.333+0000 7fd2b5ffb640 1 -- 172.21.15.105:0/2587132106 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0004230 con 0x55c675e3daf0 2024-03-26T17:52:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.333+0000 7fd2b5ffb640 1 -- 172.21.15.105:0/2587132106 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c00043d0 con 0x55c675e3daf0 2024-03-26T17:52:47.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.333+0000 7fd2b5ffb640 1 -- 172.21.15.105:0/2587132106 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0010760 con 0x55c675e3daf0 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 -- 172.21.15.105:0/2587132106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3daf0 msgr2=0x55c675e425f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2cffff640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2b80a2f20 0x7fd2b80a53e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8098630 tx=0x7fd2c8005b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3daf0 0x55c675e425f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000f570 tx=0x7fd2c000fa30 comp rx=0 tx=0).stop 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 -- 172.21.15.105:0/2587132106 shutdown_connections 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e42c60 0x55c675e450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3daf0 0x55c675e425f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/2587132106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e3d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 -- 172.21.15.105:0/2587132106 >> 172.21.15.105:0/2587132106 conn(0x55c675e2fe60 msgr2=0x55c675e30260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 -- 172.21.15.105:0/2587132106 shutdown_connections 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 -- 172.21.15.105:0/2587132106 wait complete. 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.334+0000 7fd2d89e6740 1 -- start start 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2d89e6740 1 Processor -- start 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e3daf0 0x55c675e478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e42c60 0x55c675e4e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c675e45c10 con 0x55c675e3daf0 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c675d55160 con 0x55c675e42c60 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c675e45d90 con 0x55c675e3d100 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2ceffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2ceffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53052/0 (socket says 172.21.15.105:53052) 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2ceffd640 1 -- 172.21.15.105:0/1161231383 learned_addr learned my addr 172.21.15.105:0/1161231383 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2d89e6740 10 client.0 _ll_register_callbacks cb 0x55c675d91720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e3daf0 0x55c675e478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.335+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e42c60 0x55c675e4e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/1161231383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 msgr2=0x55c675e473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/1161231383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e42c60 msgr2=0x55c675e4e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e42c60 0x55c675e4e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/1161231383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c675aad3f0 con 0x55c675e3daf0 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:47.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e3daf0 0x55c675e478e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x55c675e30dd0 tx=0x7fd2c0010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c006c920fe3ee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2aeffd640 1 -- 172.21.15.105:0/1161231383 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c002d030 con 0x55c675e3daf0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2aeffd640 1 -- 172.21.15.105:0/1161231383 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd2c000add0 con 0x55c675e3daf0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1161231383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2c00043d0 con 0x55c675e3daf0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2aeffd640 1 -- 172.21.15.105:0/1161231383 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd2c00046c0 con 0x55c675e3daf0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2d89e6740 10 client.20805 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.336+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1161231383 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c675a6d0f0 con 0x55c675e3daf0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2aeffd640 1 -- 172.21.15.105:0/1161231383 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c00043d0 con 0x55c675e3daf0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2aeffd640 1 -- 172.21.15.105:0/1161231383 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd2c0033070 con 0x55c675e3daf0 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2d89e6740 10 client.20805 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2d89e6740 10 client.20805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2d89e6740 20 client.20805 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c675e35890 0x55c675e37cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2d89e6740 4 client.20805 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:47.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1161231383 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c675da6ec0 con 0x55c675e35890 2024-03-26T17:52:47.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.337+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c675e35890 0x55c675e37cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.338+0000 7fd2aeffd640 10 client.20805 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:47.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.338+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c675e35890 0x55c675e37cb0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7573a14f1ef9ce8e server_cookie=2ceaa3bd8d9dfea6 in_seq=0 out_seq=0 2024-03-26T17:52:47.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2aeffd640 1 -- 172.21.15.105:0/1161231383 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c675da6ec0 con 0x55c675e35890 2024-03-26T17:52:47.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2aeffd640 10 client.20805 handle_command_reply: tid=0 2024-03-26T17:52:47.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 client.20805 shutdown 2024-03-26T17:52:47.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1161231383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e3daf0 msgr2=0x55c675e478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e3daf0 0x55c675e478e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x55c675e30dd0 tx=0x7fd2c0010f00 comp rx=0 tx=0).stop 2024-03-26T17:52:47.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1161231383 shutdown_connections 2024-03-26T17:52:47.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c675e35890 0x55c675e37cb0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2aeffd640 0 client.20805 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:47.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e42c60 0x55c675e4e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e3daf0 0x55c675e478e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1161231383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e3d100 0x55c675e473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1161231383 >> 172.21.15.105:0/1161231383 conn(0x55c675e2fe60 msgr2=0x55c675e2f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1161231383 shutdown_connections 2024-03-26T17:52:47.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1161231383 wait complete. 2024-03-26T17:52:47.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.340+0000 7fd2d89e6740 20 client.20805 trim_cache size 0 max 16384 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2d89e6740 1 Processor -- start 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2d89e6740 1 -- start start 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e2e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675e2ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e55b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01a6ab0 con 0x55c675e306a0 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01a86b0 con 0x55c675e3d100 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01a7180 con 0x55c675e42c60 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e55b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2cf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675e2ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e55b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57768/0 (socket says 172.21.15.105:57768) 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2cf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675e2ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53068/0 (socket says 172.21.15.105:53068) 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 -- 172.21.15.105:0/1061396235 learned_addr learned my addr 172.21.15.105:0/1061396235 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/1061396235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e2e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 -- 172.21.15.105:0/1061396235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 msgr2=0x55c675e2ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/1061396235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675e2ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 -- 172.21.15.105:0/1061396235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 msgr2=0x55c675e2e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/1061396235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e2e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.365+0000 7fd2ceffd640 1 -- 172.21.15.105:0/1061396235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c675e42020 con 0x55c675e306a0 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.366+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/1061396235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e55b70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4006010 tx=0x7fd2c4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49068417717b86cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.366+0000 7fd2b57fa640 1 -- 172.21.15.105:0/1061396235 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4052020 con 0x55c675e306a0 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.366+0000 7fd2b57fa640 1 -- 172.21.15.105:0/1061396235 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c404c240 con 0x55c675e306a0 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.366+0000 7fd2b57fa640 1 -- 172.21.15.105:0/1061396235 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4005230 con 0x55c675e306a0 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1061396235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 msgr2=0x55c675e55b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1061396235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e55b70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4006010 tx=0x7fd2c4009680 comp rx=0 tx=0).stop 2024-03-26T17:52:47.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1061396235 shutdown_connections 2024-03-26T17:52:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1061396235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e55b70 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1061396235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675e2ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/1061396235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e2e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1061396235 >> 172.21.15.105:0/1061396235 conn(0x55c675e2fe60 msgr2=0x55c675e3c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1061396235 shutdown_connections 2024-03-26T17:52:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 -- 172.21.15.105:0/1061396235 wait complete. 2024-03-26T17:52:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.367+0000 7fd2d89e6740 1 -- start start 2024-03-26T17:52:47.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2d89e6740 1 Processor -- start 2024-03-26T17:52:47.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e56ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2d89e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675eaf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01a7fe0 con 0x55c675e306a0 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01a7850 con 0x55c675e3d100 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2ceffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2d89e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01a4a90 con 0x55c675e42c60 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2ceffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57776/0 (socket says 172.21.15.105:57776) 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2ceffd640 1 -- 172.21.15.105:0/3843330813 learned_addr learned my addr 172.21.15.105:0/3843330813 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e56ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2d89e6740 10 client.0 _ll_register_callbacks cb 0x7fd2c000f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:47.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.368+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675eaf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/3843330813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 msgr2=0x55c675eaf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675eaf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/3843330813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 msgr2=0x55c675e567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2cf7fe640 1 -- 172.21.15.105:0/3843330813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c675eaff50 con 0x55c675e3d100 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2cf7fe640 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e56ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd2c002d000 tx=0x7fd2c000ffa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f49a84ac46926c67 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2aeffd640 1 -- 172.21.15.105:0/3843330813 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0044080 con 0x55c675e3d100 2024-03-26T17:52:47.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2aeffd640 1 -- 172.21.15.105:0/3843330813 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd2c000ad90 con 0x55c675e3d100 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2d89e6740 1 -- 172.21.15.105:0/3843330813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2c404c240 con 0x55c675e3d100 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2aeffd640 1 -- 172.21.15.105:0/3843330813 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd2c0010a70 con 0x55c675e3d100 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2d89e6740 10 client.30593 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.369+0000 7fd2d89e6740 1 -- 172.21.15.105:0/3843330813 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c675eb1220 con 0x55c675e3d100 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2aeffd640 1 -- 172.21.15.105:0/3843330813 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2c404c240 con 0x55c675e3d100 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2aeffd640 1 -- 172.21.15.105:0/3843330813 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd2c0033070 con 0x55c675e3d100 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2d89e6740 10 client.30593 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2d89e6740 10 client.30593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:47.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2d89e6740 20 client.30593 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c675eb1af0 0x55c675eb3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2d89e6740 4 client.30593 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2d89e6740 1 -- 172.21.15.105:0/3843330813 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c675da1210 con 0x55c675eb1af0 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.370+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c675eb1af0 0x55c675eb3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.371+0000 7fd2ceffd640 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c675eb1af0 0x55c675eb3f10 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51fff5e0aed04f1a server_cookie=1806b96662d1656d in_seq=0 out_seq=0 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.371+0000 7fd2aeffd640 10 client.30593 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2aeffd640 1 -- 172.21.15.105:0/3843330813 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55c675da1210 con 0x55c675eb1af0 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2aeffd640 10 client.30593 handle_command_reply: tid=0 2024-03-26T17:52:47.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 client.30593 shutdown 2024-03-26T17:52:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 -- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 msgr2=0x55c675e56ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e56ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd2c002d000 tx=0x7fd2c000ffa0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 -- 172.21.15.105:0/3843330813 shutdown_connections 2024-03-26T17:52:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c675eb1af0 0x55c675eb3f10 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c675e42c60 0x55c675eaf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c675e3d100 0x55c675e56ce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 --2- 172.21.15.105:0/3843330813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c675e306a0 0x55c675e567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2aeffd640 0 client.30593 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:47.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 -- 172.21.15.105:0/3843330813 >> 172.21.15.105:0/3843330813 conn(0x55c675e2fe60 msgr2=0x55c675e330c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 -- 172.21.15.105:0/3843330813 shutdown_connections 2024-03-26T17:52:47.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 1 -- 172.21.15.105:0/3843330813 wait complete. 2024-03-26T17:52:47.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.373+0000 7fd2d89e6740 20 client.30593 trim_cache size 0 max 16384 2024-03-26T17:52:47.371 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:52:47.371 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (270 inodes in the stack)", 2024-03-26T17:52:47.372 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:52:47.372 INFO:teuthology.orchestra.run.smithi105.stdout: "1e477115-2ec2-475d-9db2-a6a5dcaaae34": { 2024-03-26T17:52:47.372 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:52:47.372 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "1e477115-2ec2-475d-9db2-a6a5dcaaae34", 2024-03-26T17:52:47.372 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:52:47.372 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:52:47.372 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:52:47.372 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:52:47.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2b80a2f20 msgr2=0x7fd2b80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2b80a2f20 0x7fd2b80a53e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8098630 tx=0x7fd2c8005b70 comp rx=0 tx=0).stop 2024-03-26T17:52:47.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d0102370 msgr2=0x7fd2d01a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:47.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d0102370 0x7fd2d01a1140 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009fd0 tx=0x7fd2c800ef10 comp rx=0 tx=0).stop 2024-03-26T17:52:47.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/3243653916 shutdown_connections 2024-03-26T17:52:47.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2b80a2f20 0x7fd2b80a53e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2d0102370 0x7fd2d01a1140 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2d01019b0 0x7fd2d019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 --2- 172.21.15.105:0/3243653916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2d01007c0 0x7fd2d00695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:47.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/3243653916 >> 172.21.15.105:0/3243653916 conn(0x7fd2d00fbd50 msgr2=0x7fd2d0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:47.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/3243653916 shutdown_connections 2024-03-26T17:52:47.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:52:47.374+0000 7fd2d5d1b640 1 -- 172.21.15.105:0/3243653916 wait complete. 2024-03-26T17:52:47.379 INFO:tasks.cephfs.filesystem:scrub status for tag:1e477115-2ec2-475d-9db2-a6a5dcaaae34 - {'path': '/', 'tag': '1e477115-2ec2-475d-9db2-a6a5dcaaae34', 'options': 'recursive,force'} 2024-03-26T17:52:47.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 12032 8.95 MB/sec warmup 90 sec latency 199.105 ms 2024-03-26T17:52:47.663 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:52:47.240+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003c80 [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{1=1,2=1,3=1,4=1} v2225 ap=2 RANDEPHEMERALPIN f(v52 m2024-03-26T17:52:31.877540+0000 11=11+0) n(v4 rc2024-03-26T17:52:31.877540+0000 b2861056 12=11+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@1220} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d142580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003c80:[<0x10000003bc0/COREL v50>,<0x10000003bb6/~dmtmp v48>,<0x10000000007/client8 v2078>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003c80:[<0x10000003bc0/COREL v2225>,<0x10000003bb6/~dmtmp v511>,<0x10000000007/client8 v3650>,<0x10000000006/clients v1896>,<0x10000000005/tmp v1843>,<0x10000000003/client.0 v1776>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1732>,<0x10000000001/sv_0 v1741>,<0x10000000000/qa v1691>,<0x1/volumes v1652>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:52:31.849495+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:52:31.877540+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v52 m2024-03-26T17:52:31.877540+0000 11=11+0)","memory_value.rstat":"n(v4 rc2024-03-26T17:52:31.877540+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T17:52:48.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: pgmap v782: 97 pgs: 97 active+clean; 232 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.6 MiB/s wr, 261 op/s 2024-03-26T17:52:48.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:52:48.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:52:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.141 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: pgmap v782: 97 pgs: 97 active+clean; 232 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.6 MiB/s wr, 261 op/s 2024-03-26T17:52:48.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:52:48.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: pgmap v782: 97 pgs: 97 active+clean; 232 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.6 MiB/s wr, 261 op/s 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:48.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 12160 8.91 MB/sec warmup 91 sec latency 140.555 ms 2024-03-26T17:52:49.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:48 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:49.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:48 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:49.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:48 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:52:49.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:48 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:49.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:48 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:49.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:48 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:49.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:48 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:52:49.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:48 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:49.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:48 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:49.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:48 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:49.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:48 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:52:49.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:48 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:52:49.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 12289 8.92 MB/sec warmup 92 sec latency 141.353 ms 2024-03-26T17:52:50.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:49 smithi105 ceph-mon[32280]: pgmap v783: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-03-26T17:52:50.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:49 smithi135 ceph-mon[35636]: pgmap v783: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-03-26T17:52:50.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:49 smithi144 ceph-mon[36095]: pgmap v783: 97 pgs: 97 active+clean; 230 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 3.1 MiB/s wr, 267 op/s 2024-03-26T17:52:50.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 12433 8.95 MB/sec warmup 93 sec latency 145.913 ms 2024-03-26T17:52:51.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 12658 9.09 MB/sec warmup 94 sec latency 154.358 ms 2024-03-26T17:52:52.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:51 smithi105 ceph-mon[32280]: pgmap v784: 97 pgs: 97 active+clean; 233 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-03-26T17:52:52.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:51 smithi135 ceph-mon[35636]: pgmap v784: 97 pgs: 97 active+clean; 233 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-03-26T17:52:52.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:51 smithi144 ceph-mon[36095]: pgmap v784: 97 pgs: 97 active+clean; 233 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 2.6 MiB/s wr, 255 op/s 2024-03-26T17:52:52.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 12800 9.09 MB/sec warmup 95 sec latency 141.105 ms 2024-03-26T17:52:53.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 12972 9.11 MB/sec warmup 96 sec latency 121.827 ms 2024-03-26T17:52:54.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:53 smithi105 ceph-mon[32280]: pgmap v785: 97 pgs: 97 active+clean; 239 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-03-26T17:52:54.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:53 smithi135 ceph-mon[35636]: pgmap v785: 97 pgs: 97 active+clean; 239 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-03-26T17:52:54.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:53 smithi144 ceph-mon[36095]: pgmap v785: 97 pgs: 97 active+clean; 239 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-03-26T17:52:54.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 13084 9.08 MB/sec warmup 97 sec latency 111.283 ms 2024-03-26T17:52:55.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 13151 9.03 MB/sec warmup 98 sec latency 129.755 ms 2024-03-26T17:52:56.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:55 smithi105 ceph-mon[32280]: pgmap v786: 97 pgs: 97 active+clean; 247 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-03-26T17:52:56.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:55 smithi135 ceph-mon[35636]: pgmap v786: 97 pgs: 97 active+clean; 247 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-03-26T17:52:56.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:55 smithi144 ceph-mon[36095]: pgmap v786: 97 pgs: 97 active+clean; 247 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 3.5 MiB/s wr, 264 op/s 2024-03-26T17:52:56.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 13232 8.96 MB/sec warmup 99 sec latency 129.349 ms 2024-03-26T17:52:57.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 13359 8.99 MB/sec warmup 100 sec latency 135.857 ms 2024-03-26T17:52:58.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:57 smithi105 ceph-mon[32280]: pgmap v787: 97 pgs: 97 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-03-26T17:52:58.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:57 smithi135 ceph-mon[35636]: pgmap v787: 97 pgs: 97 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-03-26T17:52:58.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:57 smithi144 ceph-mon[36095]: pgmap v787: 97 pgs: 97 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-03-26T17:52:58.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 13509 9.01 MB/sec warmup 101 sec latency 126.731 ms 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.798+0000 7f8c6100b640 1 -- 172.21.15.105:0/1367060916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5c102390 msgr2=0x7f8c5c104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.798+0000 7f8c6100b640 1 --2- 172.21.15.105:0/1367060916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5c102390 0x7f8c5c104780 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009900 tx=0x7f8c4c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.798+0000 7f8c6100b640 1 -- 172.21.15.105:0/1367060916 shutdown_connections 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.798+0000 7f8c6100b640 1 --2- 172.21.15.105:0/1367060916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c1075f0 0x7f8c5c109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.798+0000 7f8c6100b640 1 --2- 172.21.15.105:0/1367060916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c104cc0 0x7f8c5c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.798+0000 7f8c6100b640 1 --2- 172.21.15.105:0/1367060916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5c102390 0x7f8c5c104780 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.798+0000 7f8c6100b640 1 -- 172.21.15.105:0/1367060916 >> 172.21.15.105:0/1367060916 conn(0x7f8c5c0fbd60 msgr2=0x7f8c5c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.799+0000 7f8c6100b640 1 -- 172.21.15.105:0/1367060916 shutdown_connections 2024-03-26T17:52:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.799+0000 7f8c6100b640 1 -- 172.21.15.105:0/1367060916 wait complete. 2024-03-26T17:52:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.799+0000 7f8c6100b640 1 Processor -- start 2024-03-26T17:52:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c6100b640 1 -- start start 2024-03-26T17:52:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c6100b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c102390 0x7f8c5c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c6100b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c104cc0 0x7f8c5c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c6100b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5c1075f0 0x7f8c5c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c5a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c104cc0 0x7f8c5c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c102390 0x7f8c5c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c6100b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c10d440 con 0x7f8c5c1075f0 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c102390 0x7f8c5c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55386/0 (socket says 172.21.15.105:55386) 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c10d2c0 con 0x7f8c5c102390 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/2825965529 learned_addr learned my addr 172.21.15.105:0/2825965529 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c5b59e640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5c1075f0 0x7f8c5c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.800+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c10d5c0 con 0x7f8c5c104cc0 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c5a59c640 1 -- 172.21.15.105:0/2825965529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c102390 msgr2=0x7f8c5c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c102390 0x7f8c5c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c5a59c640 1 -- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5c1075f0 msgr2=0x7f8c5c19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5c1075f0 0x7f8c5c19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c5a59c640 1 -- 172.21.15.105:0/2825965529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c100200 con 0x7f8c5c104cc0 2024-03-26T17:52:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c104cc0 0x7f8c5c197fd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000e9c0 tx=0x7f8c5000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c43fff640 1 -- 172.21.15.105:0/2825965529 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5000cd80 con 0x7f8c5c104cc0 2024-03-26T17:52:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c43fff640 1 -- 172.21.15.105:0/2825965529 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c500035f0 con 0x7f8c5c104cc0 2024-03-26T17:52:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c43fff640 1 -- 172.21.15.105:0/2825965529 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50010690 con 0x7f8c5c104cc0 2024-03-26T17:52:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c1004f0 con 0x7f8c5c104cc0 2024-03-26T17:52:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.801+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c100a90 con 0x7f8c5c104cc0 2024-03-26T17:52:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.804+0000 7f8c43fff640 1 -- 172.21.15.105:0/2825965529 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8c500037a0 con 0x7f8c5c104cc0 2024-03-26T17:52:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c63bf8740 1 Processor -- start 2024-03-26T17:52:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c63bf8740 1 -- start start 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc19200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc20dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d06fb750b0 con 0x55d06fc18de0 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d06fb1d960 con 0x55d06fc1e940 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d06fb71060 con 0x55d06fc197d0 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c5a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc20dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c5a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc20dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55396/0 (socket says 172.21.15.105:55396) 2024-03-26T17:52:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c5a59c640 1 -- 172.21.15.105:0/3337783069 learned_addr learned my addr 172.21.15.105:0/3337783069 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.805+0000 7f8c5b59e640 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc19200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c5a59c640 1 -- 172.21.15.105:0/3337783069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 msgr2=0x55d06fc1e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c5a59c640 1 -- 172.21.15.105:0/3337783069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 msgr2=0x55d06fc19200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc19200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c5a59c640 1 -- 172.21.15.105:0/3337783069 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06fb2fc60 con 0x55d06fc1e940 2024-03-26T17:52:58.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:52:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc20dd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c500d5a80 tx=0x7f8c500d5f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=229f1251685b6539 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c417fa640 1 -- 172.21.15.105:0/3337783069 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500de070 con 0x55d06fc1e940 2024-03-26T17:52:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c417fa640 1 -- 172.21.15.105:0/3337783069 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c500d1070 con 0x55d06fc1e940 2024-03-26T17:52:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c417fa640 1 -- 172.21.15.105:0/3337783069 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500da6a0 con 0x55d06fc1e940 2024-03-26T17:52:58.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.806+0000 7f8c43fff640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c440a3040 0x7f8c440a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c43fff640 1 -- 172.21.15.105:0/2825965529 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c500cf680 con 0x7f8c5c104cc0 2024-03-26T17:52:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c5b59e640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c440a3040 0x7f8c440a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3337783069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 msgr2=0x55d06fc20dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc20dd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c500d5a80 tx=0x7f8c500d5f40 comp rx=0 tx=0).stop 2024-03-26T17:52:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3337783069 shutdown_connections 2024-03-26T17:52:58.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc20dd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3337783069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc19200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.807+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3337783069 >> 172.21.15.105:0/3337783069 conn(0x55d06fc14350 msgr2=0x55d06fc14750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3337783069 shutdown_connections 2024-03-26T17:52:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c5b59e640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c440a3040 0x7f8c440a5500 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55d06fc1a030 tx=0x7f8c54009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3337783069 wait complete. 2024-03-26T17:52:58.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 -- start start 2024-03-26T17:52:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 Processor -- start 2024-03-26T17:52:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc18de0 0x55d06fc33ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc1e940 0x55d06fc3b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d06fc218f0 con 0x55d06fc1e940 2024-03-26T17:52:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d06fb30db0 con 0x55d06fc18de0 2024-03-26T17:52:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d06fc21a70 con 0x55d06fc197d0 2024-03-26T17:52:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c63bf8740 10 client.0 _ll_register_callbacks cb 0x55d06fb35110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc18de0 0x55d06fc33ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc18de0 0x55d06fc33ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55412/0 (socket says 172.21.15.105:55412) 2024-03-26T17:52:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/3726399295 learned_addr learned my addr 172.21.15.105:0/3726399295 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.808+0000 7f8c5b59e640 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc1e940 0x55d06fc3b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/3726399295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 msgr2=0x55d06fc340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc340e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/3726399295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc1e940 msgr2=0x55d06fc3b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc1e940 0x55d06fc3b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/3726399295 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06f889210 con 0x55d06fc18de0 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc18de0 0x55d06fc33ba0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c036040 tx=0x7f8c4c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d73315eb591ab63 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c3eff5640 1 -- 172.21.15.105:0/3726399295 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c005230 con 0x55d06fc18de0 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3726399295 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c500d1070 con 0x55d06fc18de0 2024-03-26T17:52:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c3eff5640 1 -- 172.21.15.105:0/3726399295 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c4c0053d0 con 0x55d06fc18de0 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.809+0000 7f8c3eff5640 1 -- 172.21.15.105:0/3726399295 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c050050 con 0x55d06fc18de0 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c63bf8740 10 client.30605 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3726399295 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d06f955620 con 0x55d06fc18de0 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c3eff5640 1 -- 172.21.15.105:0/3726399295 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c500d1070 con 0x55d06fc18de0 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c3eff5640 1 -- 172.21.15.105:0/3726399295 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8c4c0037b0 con 0x55d06fc18de0 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c63bf8740 10 client.30605 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c63bf8740 10 client.30605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c63bf8740 20 client.30605 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d06fc26bc0 0x55d06fc29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c63bf8740 4 client.30605 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3726399295 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d06fb825f0 con 0x55d06fc26bc0 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.810+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d06fc26bc0 0x55d06fc29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.811+0000 7f8c3eff5640 10 client.30605 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.811+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d06fc26bc0 0x55d06fc29080 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef60caacb9b4a30a server_cookie=5d0ce1687d1307c0 in_seq=0 out_seq=0 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c3eff5640 1 -- 172.21.15.105:0/3726399295 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d06fb825f0 con 0x55d06fc26bc0 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c3eff5640 10 client.30605 handle_command_reply: tid=0 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 client.30605 shutdown 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3726399295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc18de0 msgr2=0x55d06fc33ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc18de0 0x55d06fc33ba0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c036040 tx=0x7f8c4c004270 comp rx=0 tx=0).stop 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3726399295 shutdown_connections 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d06fc26bc0 0x55d06fc29080 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c3eff5640 0 client.30605 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc1e940 0x55d06fc3b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3726399295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc18de0 0x55d06fc33ba0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3726399295 >> 172.21.15.105:0/3726399295 conn(0x55d06fc14350 msgr2=0x55d06fc0adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.813+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3726399295 shutdown_connections 2024-03-26T17:52:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.814+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3726399295 wait complete. 2024-03-26T17:52:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.814+0000 7f8c63bf8740 20 client.30605 trim_cache size 0 max 16384 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.833+0000 7f8c63bf8740 1 Processor -- start 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.833+0000 7f8c63bf8740 1 -- start start 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.833+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.833+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.833+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc1e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c0727d0 con 0x55d06fc18de0 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c19ff10 con 0x55d06fc1e940 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c072ea0 con 0x55d06fc197d0 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.824 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5b59e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc1e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49776/0 (socket says 172.21.15.105:49776) 2024-03-26T17:52:58.825 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/3466384638 learned_addr learned my addr 172.21.15.105:0/3466384638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/3466384638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 msgr2=0x55d06fc1dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/3466384638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/3466384638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 msgr2=0x55d06fc1e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/3466384638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc1e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/3466384638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06fc253c0 con 0x55d06fc18de0 2024-03-26T17:52:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.834+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/3466384638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc135b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c000c00 tx=0x7f8c4c02cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cc85c569961906 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.835+0000 7f8c40ff9640 1 -- 172.21.15.105:0/3466384638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c009520 con 0x55d06fc18de0 2024-03-26T17:52:58.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.835+0000 7f8c40ff9640 1 -- 172.21.15.105:0/3466384638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c4c003da0 con 0x55d06fc18de0 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.835+0000 7f8c40ff9640 1 -- 172.21.15.105:0/3466384638 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0046c0 con 0x55d06fc18de0 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.835+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3466384638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 msgr2=0x55d06fc135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3466384638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc135b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c000c00 tx=0x7f8c4c02cc80 comp rx=0 tx=0).stop 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3466384638 shutdown_connections 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3466384638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc1e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3466384638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc197d0 0x55d06fc1dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/3466384638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc18de0 0x55d06fc135b0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3466384638 >> 172.21.15.105:0/3466384638 conn(0x55d06fc14350 msgr2=0x55d06fc14730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3466384638 shutdown_connections 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 -- 172.21.15.105:0/3466384638 wait complete. 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 -- start start 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 Processor -- start 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc18de0 0x55d06fc3aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.836+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc197d0 0x55d06fc39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c63bf8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c19f840 con 0x55d06fc197d0 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c073570 con 0x55d06fc1e940 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c63bf8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c0707b0 con 0x55d06fc18de0 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c63bf8740 10 client.0 _ll_register_callbacks cb 0x7f8c4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc18de0 0x55d06fc3aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5ad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc18de0 0x55d06fc3aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49564/0 (socket says 172.21.15.105:49564) 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5ad9d640 1 -- 172.21.15.105:0/339771569 learned_addr learned my addr 172.21.15.105:0/339771569 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc197d0 0x55d06fc39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5b59e640 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5a59c640 1 -- 172.21.15.105:0/339771569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc18de0 msgr2=0x55d06fc3aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc18de0 0x55d06fc3aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5a59c640 1 -- 172.21.15.105:0/339771569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 msgr2=0x55d06fc39640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:52:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc39640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5a59c640 1 -- 172.21.15.105:0/339771569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06fc39c80 con 0x55d06fc197d0 2024-03-26T17:52:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc18de0 0x55d06fc3aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:52:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c5a59c640 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc197d0 0x55d06fc39100 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f8c500d51b0 tx=0x7f8c500d17e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acc68c918b72cf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:52:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.837+0000 7f8c3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:52:58.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c3eff5640 1 -- 172.21.15.105:0/339771569 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500de070 con 0x55d06fc197d0 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c3eff5640 1 -- 172.21.15.105:0/339771569 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c5009a070 con 0x55d06fc197d0 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c63bf8740 1 -- 172.21.15.105:0/339771569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c003da0 con 0x55d06fc197d0 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c3eff5640 1 -- 172.21.15.105:0/339771569 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c5009bb40 con 0x55d06fc197d0 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c63bf8740 10 client.20829 fetch_fsmap learned FSMap version 34 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c63bf8740 1 -- 172.21.15.105:0/339771569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d06fcbf190 con 0x55d06fc197d0 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c3eff5640 1 -- 172.21.15.105:0/339771569 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c4c003da0 con 0x55d06fc197d0 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c3eff5640 1 -- 172.21.15.105:0/339771569 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8c500d5240 con 0x55d06fc197d0 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c63bf8740 10 client.20829 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.838+0000 7f8c63bf8740 10 client.20829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.839+0000 7f8c63bf8740 20 client.20829 populate_metadata read hostname 'smithi105' 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.839+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d06fcbf760 0x55d06fcc1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.839+0000 7f8c63bf8740 4 client.20829 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.839+0000 7f8c63bf8740 1 -- 172.21.15.105:0/339771569 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d06fb7c940 con 0x55d06fcbf760 2024-03-26T17:52:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.839+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d06fcbf760 0x55d06fcc1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:52:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.839+0000 7f8c3eff5640 10 client.20829 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.839+0000 7f8c5ad9d640 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d06fcbf760 0x55d06fcc1b50 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30fb0f33215a3348 server_cookie=e0cf5061121297c6 in_seq=0 out_seq=0 2024-03-26T17:52:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c3eff5640 1 -- 172.21.15.105:0/339771569 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d06fb7c940 con 0x55d06fcbf760 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c3eff5640 10 client.20829 handle_command_reply: tid=0 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 client.20829 shutdown 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 -- 172.21.15.105:0/339771569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc197d0 msgr2=0x55d06fc39100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc197d0 0x55d06fc39100 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f8c500d51b0 tx=0x7f8c500d17e0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 -- 172.21.15.105:0/339771569 shutdown_connections 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d06fcbf760 0x55d06fcc1b50 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d06fc1e940 0x55d06fc39640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d06fc197d0 0x55d06fc39100 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 --2- 172.21.15.105:0/339771569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d06fc18de0 0x55d06fc3aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 -- 172.21.15.105:0/339771569 >> 172.21.15.105:0/339771569 conn(0x55d06fc14350 msgr2=0x55d06fc33970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c3eff5640 0 client.20829 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:52:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 -- 172.21.15.105:0/339771569 shutdown_connections 2024-03-26T17:52:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.973+0000 7f8c63bf8740 1 -- 172.21.15.105:0/339771569 wait complete. 2024-03-26T17:52:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c63bf8740 20 client.20829 trim_cache size 0 max 16384 2024-03-26T17:52:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:52:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:52:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:52:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:52:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c440a3040 msgr2=0x7f8c440a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c440a3040 0x7f8c440a5500 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55d06fc1a030 tx=0x7f8c54009500 comp rx=0 tx=0).stop 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c104cc0 msgr2=0x7f8c5c197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c104cc0 0x7f8c5c197fd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c5000e9c0 tx=0x7f8c5000ee80 comp rx=0 tx=0).stop 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 shutdown_connections 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c440a3040 0x7f8c440a5500 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5c1075f0 0x7f8c5c19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c104cc0 0x7f8c5c197fd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 --2- 172.21.15.105:0/2825965529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c5c102390 0x7f8c5c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.974+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 >> 172.21.15.105:0/2825965529 conn(0x7f8c5c0fbd60 msgr2=0x7f8c5c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.975+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 shutdown_connections 2024-03-26T17:52:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:52:58.975+0000 7f8c6100b640 1 -- 172.21.15.105:0/2825965529 wait complete. 2024-03-26T17:52:59.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:52:58 smithi144 ceph-mon[36095]: pgmap v788: 97 pgs: 97 active+clean; 251 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-26T17:52:59.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:52:58 smithi105 ceph-mon[32280]: pgmap v788: 97 pgs: 97 active+clean; 251 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-26T17:52:59.413 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:52:58 smithi135 ceph-mon[35636]: pgmap v788: 97 pgs: 97 active+clean; 251 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-03-26T17:52:59.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 13657 9.03 MB/sec warmup 102 sec latency 187.778 ms 2024-03-26T17:53:00.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 13785 8.99 MB/sec warmup 103 sec latency 123.201 ms 2024-03-26T17:53:01.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 13907 8.99 MB/sec warmup 104 sec latency 131.182 ms 2024-03-26T17:53:02.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:01 smithi105 ceph-mon[32280]: pgmap v789: 97 pgs: 97 active+clean; 247 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.6 MiB/s wr, 208 op/s 2024-03-26T17:53:02.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:01 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:53:02.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:01 smithi135 ceph-mon[35636]: pgmap v789: 97 pgs: 97 active+clean; 247 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.6 MiB/s wr, 208 op/s 2024-03-26T17:53:02.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:01 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:53:02.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:01 smithi144 ceph-mon[36095]: pgmap v789: 97 pgs: 97 active+clean; 247 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.6 MiB/s wr, 208 op/s 2024-03-26T17:53:02.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:01 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:53:02.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14044 8.97 MB/sec warmup 105 sec latency 134.119 ms 2024-03-26T17:53:03.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:02 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:53:03.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:02 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:53:03.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:02 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:53:03.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:02 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:53:03.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:02 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:53:03.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:02 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:53:03.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:02 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:53:03.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:02 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:53:03.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:02 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:53:03.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14138 8.92 MB/sec warmup 106 sec latency 153.793 ms 2024-03-26T17:53:04.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:03 smithi105 ceph-mon[32280]: pgmap v790: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 3.8 MiB/s wr, 213 op/s 2024-03-26T17:53:04.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:03 smithi135 ceph-mon[35636]: pgmap v790: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 3.8 MiB/s wr, 213 op/s 2024-03-26T17:53:04.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:03 smithi144 ceph-mon[36095]: pgmap v790: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 3.8 MiB/s wr, 213 op/s 2024-03-26T17:53:04.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14205 8.84 MB/sec warmup 107 sec latency 170.670 ms 2024-03-26T17:53:05.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:04 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:05.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:04 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:05.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:04 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:53:05.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14323 8.83 MB/sec warmup 108 sec latency 148.396 ms 2024-03-26T17:53:06.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:05 smithi105 ceph-mon[32280]: pgmap v791: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 4.2 MiB/s wr, 227 op/s 2024-03-26T17:53:06.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:05 smithi135 ceph-mon[35636]: pgmap v791: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 4.2 MiB/s wr, 227 op/s 2024-03-26T17:53:06.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:05 smithi144 ceph-mon[36095]: pgmap v791: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 4.2 MiB/s wr, 227 op/s 2024-03-26T17:53:06.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14436 8.77 MB/sec warmup 109 sec latency 144.846 ms 2024-03-26T17:53:07.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:07.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:07.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:53:07.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:07.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:07.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:53:07.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:53:07.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:53:07.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:53:07.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14545 8.71 MB/sec warmup 110 sec latency 141.300 ms 2024-03-26T17:53:08.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:07 smithi105 ceph-mon[32280]: pgmap v792: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 3.6 MiB/s wr, 226 op/s 2024-03-26T17:53:08.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:07 smithi135 ceph-mon[35636]: pgmap v792: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 3.6 MiB/s wr, 226 op/s 2024-03-26T17:53:08.247 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:07 smithi144 ceph-mon[36095]: pgmap v792: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 3.6 MiB/s wr, 226 op/s 2024-03-26T17:53:08.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14671 8.67 MB/sec warmup 111 sec latency 119.059 ms 2024-03-26T17:53:09.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14795 8.66 MB/sec warmup 112 sec latency 135.581 ms 2024-03-26T17:53:10.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:09 smithi105 ceph-mon[32280]: pgmap v793: 97 pgs: 97 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-03-26T17:53:10.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:09 smithi135 ceph-mon[35636]: pgmap v793: 97 pgs: 97 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-03-26T17:53:10.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:09 smithi144 ceph-mon[36095]: pgmap v793: 97 pgs: 97 active+clean; 253 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 3.5 MiB/s wr, 220 op/s 2024-03-26T17:53:10.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 14920 8.61 MB/sec warmup 113 sec latency 126.153 ms 2024-03-26T17:53:11.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 15058 8.57 MB/sec warmup 114 sec latency 126.235 ms 2024-03-26T17:53:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.094+0000 7ff453f10640 1 -- 172.21.15.105:0/651949111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 msgr2=0x7ff44c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.095+0000 7ff453f10640 1 --2- 172.21.15.105:0/651949111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 0x7ff44c101e50 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009900 tx=0x7ff43c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.095+0000 7ff453f10640 1 -- 172.21.15.105:0/651949111 shutdown_connections 2024-03-26T17:53:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.095+0000 7ff453f10640 1 --2- 172.21.15.105:0/651949111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff44c102390 0x7ff44c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.095+0000 7ff453f10640 1 --2- 172.21.15.105:0/651949111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 0x7ff44c101e50 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.095+0000 7ff453f10640 1 --2- 172.21.15.105:0/651949111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff44c1007e0 0x7ff44c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.095+0000 7ff453f10640 1 -- 172.21.15.105:0/651949111 >> 172.21.15.105:0/651949111 conn(0x7ff44c0fbd50 msgr2=0x7ff44c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.095+0000 7ff453f10640 1 -- 172.21.15.105:0/651949111 shutdown_connections 2024-03-26T17:53:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.095+0000 7ff453f10640 1 -- 172.21.15.105:0/651949111 wait complete. 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff453f10640 1 Processor -- start 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff453f10640 1 -- start start 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff453f10640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff44c1007e0 0x7ff44c1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff453f10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 0x7ff44c195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff453f10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff44c102390 0x7ff44c19ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff453f10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c10b210 con 0x7ff44c1019d0 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff453f10640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c10b090 con 0x7ff44c1007e0 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff453f10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c10b390 con 0x7ff44c102390 2024-03-26T17:53:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff4514ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 0x7ff44c195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff4514ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 0x7ff44c195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59020/0 (socket says 172.21.15.105:59020) 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff4514ab640 1 -- 172.21.15.105:0/3163495300 learned_addr learned my addr 172.21.15.105:0/3163495300 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff451cac640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff44c1007e0 0x7ff44c1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.096+0000 7ff4524ad640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff44c102390 0x7ff44c19ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff4514ab640 1 -- 172.21.15.105:0/3163495300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff44c102390 msgr2=0x7ff44c19ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff44c102390 0x7ff44c19ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff4514ab640 1 -- 172.21.15.105:0/3163495300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff44c1007e0 msgr2=0x7ff44c1958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff44c1007e0 0x7ff44c1958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff4514ab640 1 -- 172.21.15.105:0/3163495300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c19d490 con 0x7ff44c1019d0 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff451cac640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff44c1007e0 0x7ff44c1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 0x7ff44c195e00 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006010 tx=0x7ff43c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff442ffd640 1 -- 172.21.15.105:0/3163495300 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c043020 con 0x7ff44c1019d0 2024-03-26T17:53:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff442ffd640 1 -- 172.21.15.105:0/3163495300 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff43c0043d0 con 0x7ff44c1019d0 2024-03-26T17:53:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff453f10640 1 -- 172.21.15.105:0/3163495300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44c19d720 con 0x7ff44c1019d0 2024-03-26T17:53:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff442ffd640 1 -- 172.21.15.105:0/3163495300 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c003b80 con 0x7ff44c1019d0 2024-03-26T17:53:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.097+0000 7ff453f10640 1 -- 172.21.15.105:0/3163495300 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44c19dc70 con 0x7ff44c1019d0 2024-03-26T17:53:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.099+0000 7ff442ffd640 1 -- 172.21.15.105:0/3163495300 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff43c046020 con 0x7ff44c1019d0 2024-03-26T17:53:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.100+0000 7ff442ffd640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2fc0 0x7ff4380a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.100+0000 7ff451cac640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2fc0 0x7ff4380a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.100+0000 7ff442ffd640 1 -- 172.21.15.105:0/3163495300 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff43c0f1620 con 0x7ff44c1019d0 2024-03-26T17:53:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.100+0000 7ff456bdd740 1 Processor -- start 2024-03-26T17:53:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff456bdd740 1 -- start start 2024-03-26T17:53:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7d0540 0x55a2bf7d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff456bdd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf72b0b0 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff456bdd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf6d3960 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff456bdd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf727060 con 0x55a2bf7d0540 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4514ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff451cac640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2fc0 0x7ff4380a5480 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff44800b3c0 tx=0x7ff44800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4514ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59028/0 (socket says 172.21.15.105:59028) 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff451cac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4514ab640 1 -- 172.21.15.105:0/4044816149 learned_addr learned my addr 172.21.15.105:0/4044816149 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4524ad640 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7d0540 0x55a2bf7d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4514ab640 1 -- 172.21.15.105:0/4044816149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7d0540 msgr2=0x55a2bf7d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4514ab640 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7d0540 0x55a2bf7d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4514ab640 1 -- 172.21.15.105:0/4044816149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 msgr2=0x55a2bf7cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4514ab640 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4514ab640 1 -- 172.21.15.105:0/4044816149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2bf6e5c60 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.101+0000 7ff4524ad640 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7d0540 0x55a2bf7d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.102+0000 7ff4514ab640 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7cfed0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0ba170 tx=0x7ff43c0bd730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52887273ae04d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.102+0000 7ff437fff640 1 -- 172.21.15.105:0/4044816149 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0c1e60 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.102+0000 7ff437fff640 1 -- 172.21.15.105:0/4044816149 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff43c0c6720 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.102+0000 7ff437fff640 1 -- 172.21.15.105:0/4044816149 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0ba4f0 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 -- 172.21.15.105:0/4044816149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 msgr2=0x55a2bf7cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7cfed0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0ba170 tx=0x7ff43c0bd730 comp rx=0 tx=0).stop 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 -- 172.21.15.105:0/4044816149 shutdown_connections 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7d0540 0x55a2bf7d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7cfed0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 --2- 172.21.15.105:0/4044816149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 -- 172.21.15.105:0/4044816149 >> 172.21.15.105:0/4044816149 conn(0x55a2bf7c5f50 msgr2=0x55a2bf7c6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 -- 172.21.15.105:0/4044816149 shutdown_connections 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 -- 172.21.15.105:0/4044816149 wait complete. 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 -- start start 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.103+0000 7ff456bdd740 1 Processor -- start 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7d4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7d0540 0x55a2bf7dbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff451cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff451cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59032/0 (socket says 172.21.15.105:59032) 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff451cac640 1 -- 172.21.15.105:0/3894436623 learned_addr learned my addr 172.21.15.105:0/3894436623 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf7d32b0 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf6e6db0 con 0x55a2bf7d0540 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a2bf7d3430 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7d4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff456bdd740 10 client.0 _ll_register_callbacks cb 0x55a2bf6eb110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.104+0000 7ff4524ad640 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7d0540 0x55a2bf7dbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4514ab640 1 -- 172.21.15.105:0/3894436623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7d0540 msgr2=0x55a2bf7dbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7d0540 0x55a2bf7dbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4514ab640 1 -- 172.21.15.105:0/3894436623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7ca9e0 msgr2=0x55a2bf7d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4514ab640 1 -- 172.21.15.105:0/3894436623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2bf43f210 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7d4ac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55a2bf7cbe40 tx=0x7ff43c0bde40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9cc7262d3a48564 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4357fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4357fa640 1 -- 172.21.15.105:0/3894436623 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0c6490 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4357fa640 1 -- 172.21.15.105:0/3894436623 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff43c0bd160 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff43c0c6720 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff4357fa640 1 -- 172.21.15.105:0/3894436623 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff43c0c1e60 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff456bdd740 10 client.30604 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.105+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a2bf50b620 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff4357fa640 1 -- 172.21.15.105:0/3894436623 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff43c0c6720 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff4357fa640 1 -- 172.21.15.105:0/3894436623 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff43c0f10f0 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff456bdd740 10 client.30604 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff456bdd740 10 client.30604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff456bdd740 20 client.30604 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a2bf7c40f0 0x55a2bf7c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff456bdd740 4 client.30604 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2bf7385f0 con 0x55a2bf7c40f0 2024-03-26T17:53:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.106+0000 7ff451cac640 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a2bf7c40f0 0x55a2bf7c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.107+0000 7ff4357fa640 10 client.30604 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.107+0000 7ff451cac640 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a2bf7c40f0 0x55a2bf7c44f0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=725bfa841fc9b3b4 server_cookie=6ce39925f12a6eec in_seq=0 out_seq=0 2024-03-26T17:53:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff4357fa640 1 -- 172.21.15.105:0/3894436623 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a2bf7385f0 con 0x55a2bf7c40f0 2024-03-26T17:53:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff4357fa640 10 client.30604 handle_command_reply: tid=0 2024-03-26T17:53:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 client.30604 shutdown 2024-03-26T17:53:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7cb3d0 msgr2=0x55a2bf7d4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7d4ac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55a2bf7cbe40 tx=0x7ff43c0bde40 comp rx=0 tx=0).stop 2024-03-26T17:53:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 shutdown_connections 2024-03-26T17:53:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a2bf7c40f0 0x55a2bf7c44f0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff4357fa640 0 client.30604 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7d0540 0x55a2bf7dbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7d4ac0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3894436623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 >> 172.21.15.105:0/3894436623 conn(0x55a2bf7c5f50 msgr2=0x55a2bf7c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 shutdown_connections 2024-03-26T17:53:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.109+0000 7ff456bdd740 1 -- 172.21.15.105:0/3894436623 wait complete. 2024-03-26T17:53:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.110+0000 7ff456bdd740 20 client.30604 trim_cache size 0 max 16384 2024-03-26T17:53:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.131+0000 7ff456bdd740 1 Processor -- start 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.131+0000 7ff456bdd740 1 -- start start 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7c0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff456bdd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1a6b10 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff456bdd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1a8710 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff456bdd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1a71e0 con 0x55a2bf7c40f0 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7c0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59042/0 (socket says 172.21.15.105:59042) 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff451cac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 -- 172.21.15.105:0/49899335 learned_addr learned my addr 172.21.15.105:0/49899335 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4514ab640 1 --2- 172.21.15.105:0/49899335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7dbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 -- 172.21.15.105:0/49899335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7c40f0 msgr2=0x55a2bf7cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 --2- 172.21.15.105:0/49899335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 -- 172.21.15.105:0/49899335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 msgr2=0x55a2bf7dbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 --2- 172.21.15.105:0/49899335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7dbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 -- 172.21.15.105:0/49899335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2bf7c6cc0 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.132+0000 7ff4524ad640 1 --2- 172.21.15.105:0/49899335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7c0560 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff444011790 tx=0x7ff444011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a164422ef61b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff4377fe640 1 -- 172.21.15.105:0/49899335 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444004b20 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff4377fe640 1 -- 172.21.15.105:0/49899335 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff44400fdd0 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff4377fe640 1 -- 172.21.15.105:0/49899335 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4440055c0 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 -- 172.21.15.105:0/49899335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 msgr2=0x55a2bf7c0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 --2- 172.21.15.105:0/49899335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7c0560 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff444011790 tx=0x7ff444011c50 comp rx=0 tx=0).stop 2024-03-26T17:53:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 -- 172.21.15.105:0/49899335 shutdown_connections 2024-03-26T17:53:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 --2- 172.21.15.105:0/49899335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf7c0560 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 --2- 172.21.15.105:0/49899335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7dbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 --2- 172.21.15.105:0/49899335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 -- 172.21.15.105:0/49899335 >> 172.21.15.105:0/49899335 conn(0x55a2bf7c5f50 msgr2=0x55a2bf7cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 -- 172.21.15.105:0/49899335 shutdown_connections 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.133+0000 7ff456bdd740 1 -- 172.21.15.105:0/49899335 wait complete. 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.134+0000 7ff456bdd740 1 -- start start 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.134+0000 7ff456bdd740 1 Processor -- start 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.134+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff456bdd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf8491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff456bdd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1a8040 con 0x55a2bf7cb3d0 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff456bdd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1a78b0 con 0x55a2bf7c40f0 2024-03-26T17:53:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff456bdd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c1a4af0 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff451cac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff451cac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7d8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54192/0 (socket says 172.21.15.105:54192) 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff451cac640 1 -- 172.21.15.105:0/3267435859 learned_addr learned my addr 172.21.15.105:0/3267435859 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4524ad640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf8491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff456bdd740 10 client.0 _ll_register_callbacks cb 0x7ff44800be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4514ab640 1 -- 172.21.15.105:0/3267435859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7c40f0 msgr2=0x55a2bf7d8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7d8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4514ab640 1 -- 172.21.15.105:0/3267435859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 msgr2=0x55a2bf8491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf8491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4514ab640 1 -- 172.21.15.105:0/3267435859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2bf7bd450 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff451cac640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4524ad640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf8491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4514ab640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d9000 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0ba6e0 tx=0x7ff43c0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86c2590c67f2b57 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4357fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.135+0000 7ff4357fa640 1 -- 172.21.15.105:0/3267435859 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0bcf70 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff4357fa640 1 -- 172.21.15.105:0/3267435859 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff43c09cb80 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 1 -- 172.21.15.105:0/3267435859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff44400fdd0 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff4357fa640 1 -- 172.21.15.105:0/3267435859 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff43c0c6720 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 10 client.30610 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 1 -- 172.21.15.105:0/3267435859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a2bf84a8b0 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff4357fa640 1 -- 172.21.15.105:0/3267435859 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff44400fdd0 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff4357fa640 1 -- 172.21.15.105:0/3267435859 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff43c0b9a60 con 0x55a2bf7ca9e0 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 10 client.30610 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 10 client.30610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 20 client.30610 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a2bf7d0540 0x55a2bf84d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 4 client.30610 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:53:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff456bdd740 1 -- 172.21.15.105:0/3267435859 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2bf732940 con 0x55a2bf7d0540 2024-03-26T17:53:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.136+0000 7ff451cac640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a2bf7d0540 0x55a2bf84d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.137+0000 7ff4357fa640 10 client.30610 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.137+0000 7ff451cac640 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a2bf7d0540 0x55a2bf84d090 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354701e63c803a14 server_cookie=f60e7c9b89f765fa in_seq=0 out_seq=0 2024-03-26T17:53:12.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:11 smithi105 ceph-mon[32280]: pgmap v794: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2024-03-26T17:53:12.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:11 smithi135 ceph-mon[35636]: pgmap v794: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2024-03-26T17:53:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff4357fa640 1 -- 172.21.15.105:0/3267435859 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2bf732940 con 0x55a2bf7d0540 2024-03-26T17:53:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff4357fa640 10 client.30610 handle_command_reply: tid=0 2024-03-26T17:53:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 client.30610 shutdown 2024-03-26T17:53:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 -- 172.21.15.105:0/3267435859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7ca9e0 msgr2=0x55a2bf7d9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d9000 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0ba6e0 tx=0x7ff43c0092d0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 -- 172.21.15.105:0/3267435859 shutdown_connections 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a2bf7d0540 0x55a2bf84d090 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a2bf7cb3d0 0x55a2bf8491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff4357fa640 0 client.30610 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2bf7ca9e0 0x55a2bf7d9000 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 --2- 172.21.15.105:0/3267435859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2bf7c40f0 0x55a2bf7d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 -- 172.21.15.105:0/3267435859 >> 172.21.15.105:0/3267435859 conn(0x55a2bf7c5f50 msgr2=0x55a2bf7c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.274+0000 7ff456bdd740 1 -- 172.21.15.105:0/3267435859 shutdown_connections 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff456bdd740 1 -- 172.21.15.105:0/3267435859 wait complete. 2024-03-26T17:53:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff456bdd740 20 client.30610 trim_cache size 0 max 16384 2024-03-26T17:53:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:53:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:53:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:53:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:53:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 -- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2fc0 msgr2=0x7ff4380a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2fc0 0x7ff4380a5480 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff44800b3c0 tx=0x7ff44800d040 comp rx=0 tx=0).stop 2024-03-26T17:53:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 -- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 msgr2=0x7ff44c195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 0x7ff44c195e00 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006010 tx=0x7ff43c005e40 comp rx=0 tx=0).stop 2024-03-26T17:53:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 -- 172.21.15.105:0/3163495300 shutdown_connections 2024-03-26T17:53:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2fc0 0x7ff4380a5480 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff44c102390 0x7ff44c19ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff44c1019d0 0x7ff44c195e00 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.275+0000 7ff453f10640 1 --2- 172.21.15.105:0/3163495300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff44c1007e0 0x7ff44c1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.276+0000 7ff453f10640 1 -- 172.21.15.105:0/3163495300 >> 172.21.15.105:0/3163495300 conn(0x7ff44c0fbd50 msgr2=0x7ff44c104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.276+0000 7ff453f10640 1 -- 172.21.15.105:0/3163495300 shutdown_connections 2024-03-26T17:53:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:12.276+0000 7ff453f10640 1 -- 172.21.15.105:0/3163495300 wait complete. 2024-03-26T17:53:12.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:11 smithi144 ceph-mon[36095]: pgmap v794: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2024-03-26T17:53:12.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 15193 8.61 MB/sec warmup 115 sec latency 186.100 ms 2024-03-26T17:53:13.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 15331 8.60 MB/sec warmup 116 sec latency 129.874 ms 2024-03-26T17:53:14.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:13 smithi105 ceph-mon[32280]: pgmap v795: 97 pgs: 97 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-03-26T17:53:14.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:13 smithi135 ceph-mon[35636]: pgmap v795: 97 pgs: 97 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-03-26T17:53:14.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:13 smithi144 ceph-mon[36095]: pgmap v795: 97 pgs: 97 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-03-26T17:53:14.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 15517 8.66 MB/sec warmup 117 sec latency 116.536 ms 2024-03-26T17:53:15.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 15648 8.62 MB/sec warmup 118 sec latency 113.553 ms 2024-03-26T17:53:16.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:15 smithi105 ceph-mon[32280]: pgmap v796: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-03-26T17:53:16.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:15 smithi135 ceph-mon[35636]: pgmap v796: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-03-26T17:53:16.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:15 smithi144 ceph-mon[36095]: pgmap v796: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.5 MiB/s wr, 245 op/s 2024-03-26T17:53:16.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 15796 8.58 MB/sec warmup 119 sec latency 122.089 ms 2024-03-26T17:53:17.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:53:17.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:53:17.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 -- 172.21.15.105:0/1433447147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81034a0 msgr2=0x7fecc8109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1433447147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81034a0 0x7fecc8109d30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009be0 tx=0x7fecc002bef0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 -- 172.21.15.105:0/1433447147 shutdown_connections 2024-03-26T17:53:17.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1433447147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81034a0 0x7fecc8109d30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1433447147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc8102ae0 0x7fecc8102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1433447147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc81018f0 0x7fecc8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 -- 172.21.15.105:0/1433447147 >> 172.21.15.105:0/1433447147 conn(0x7fecc80fce60 msgr2=0x7fecc80ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 -- 172.21.15.105:0/1433447147 shutdown_connections 2024-03-26T17:53:17.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 -- 172.21.15.105:0/1433447147 wait complete. 2024-03-26T17:53:17.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 Processor -- start 2024-03-26T17:53:17.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.533+0000 7fecce3f4640 1 -- start start 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecce3f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81018f0 0x7fecc81958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecce3f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc8102ae0 0x7fecc8195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecce3f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc81034a0 0x7fecc819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecce3f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc810b240 con 0x7fecc81034a0 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecce3f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecc810b0c0 con 0x7fecc81018f0 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecce3f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecc810b3c0 con 0x7fecc8102ae0 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81018f0 0x7fecc81958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81018f0 0x7fecc81958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57454/0 (socket says 172.21.15.105:57454) 2024-03-26T17:53:17.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc7fff640 1 -- 172.21.15.105:0/1771360964 learned_addr learned my addr 172.21.15.105:0/1771360964 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc77fe640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc8102ae0 0x7fecc8195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7feccc991640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc81034a0 0x7fecc819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc7fff640 1 -- 172.21.15.105:0/1771360964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc8102ae0 msgr2=0x7fecc8195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc7fff640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc8102ae0 0x7fecc8195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc7fff640 1 -- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc81034a0 msgr2=0x7fecc819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc7fff640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc81034a0 0x7fecc819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.534+0000 7fecc7fff640 1 -- 172.21.15.105:0/1771360964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc819d4d0 con 0x7fecc81018f0 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.535+0000 7fecc77fe640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc8102ae0 0x7fecc8195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.535+0000 7feccc991640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc81034a0 0x7fecc819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.535+0000 7fecc7fff640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81018f0 0x7fecc81958b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fecb800e9c0 tx=0x7fecb800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.535+0000 7fecc57fa640 1 -- 172.21.15.105:0/1771360964 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb800cd80 con 0x7fecc81018f0 2024-03-26T17:53:17.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.535+0000 7fecc57fa640 1 -- 172.21.15.105:0/1771360964 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb80035f0 con 0x7fecc81018f0 2024-03-26T17:53:17.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.535+0000 7fecce3f4640 1 -- 172.21.15.105:0/1771360964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc819d7c0 con 0x7fecc81018f0 2024-03-26T17:53:17.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.535+0000 7fecc57fa640 1 -- 172.21.15.105:0/1771360964 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8010690 con 0x7fecc81018f0 2024-03-26T17:53:17.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.535+0000 7fecce3f4640 1 -- 172.21.15.105:0/1771360964 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc819dd60 con 0x7fecc81018f0 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.538+0000 7fecc57fa640 1 -- 172.21.15.105:0/1771360964 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fecb80037a0 con 0x7fecc81018f0 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.538+0000 7fecd0fe9740 1 Processor -- start 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.538+0000 7fecd0fe9740 1 -- start start 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f818f0 0x55d3b2f81d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f822e0 0x55d3b2f86de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 0x55d3b2f898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d3b2ee6980 con 0x55d3b2f822e0 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3b2e8ebe0 con 0x55d3b2f87450 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3b2ee2930 con 0x55d3b2f818f0 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7feccc991640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f818f0 0x55d3b2f81d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7feccc991640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f818f0 0x55d3b2f81d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33730/0 (socket says 172.21.15.105:33730) 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7feccc991640 1 -- 172.21.15.105:0/3075985884 learned_addr learned my addr 172.21.15.105:0/3075985884 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:17.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecc77fe640 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 0x55d3b2f898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecc7fff640 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f822e0 0x55d3b2f86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7feccc991640 1 -- 172.21.15.105:0/3075985884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 msgr2=0x55d3b2f898e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7feccc991640 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 0x55d3b2f898e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7feccc991640 1 -- 172.21.15.105:0/3075985884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f822e0 msgr2=0x55d3b2f86de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7feccc991640 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f822e0 0x55d3b2f86de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7feccc991640 1 -- 172.21.15.105:0/3075985884 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b2cc47d0 con 0x55d3b2f818f0 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecc57fa640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fecb00a2fa0 0x7fecb00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecc7fff640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fecb00a2fa0 0x7fecb00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.539+0000 7fecc57fa640 1 -- 172.21.15.105:0/1771360964 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fecb80cf720 con 0x7fecc81018f0 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.540+0000 7feccc991640 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f818f0 0x55d3b2f81d10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009be0 tx=0x7fecc0005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c639cc1ee045dea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.540+0000 7fecae7fc640 1 -- 172.21.15.105:0/3075985884 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0043030 con 0x55d3b2f818f0 2024-03-26T17:53:17.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.540+0000 7fecae7fc640 1 -- 172.21.15.105:0/3075985884 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc0003970 con 0x55d3b2f818f0 2024-03-26T17:53:17.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.540+0000 7fecae7fc640 1 -- 172.21.15.105:0/3075985884 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc002ca70 con 0x55d3b2f818f0 2024-03-26T17:53:17.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3075985884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f818f0 msgr2=0x55d3b2f81d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecc7fff640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fecb00a2fa0 0x7fecb00a5460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fecb80cd550 tx=0x7fecb80d5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f818f0 0x55d3b2f81d10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009be0 tx=0x7fecc0005e30 comp rx=0 tx=0).stop 2024-03-26T17:53:17.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3075985884 shutdown_connections 2024-03-26T17:53:17.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 0x55d3b2f898e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f822e0 0x55d3b2f86de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3075985884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f818f0 0x55d3b2f81d10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3075985884 >> 172.21.15.105:0/3075985884 conn(0x55d3b2f7ce60 msgr2=0x55d3b2f7d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3075985884 shutdown_connections 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3075985884 wait complete. 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.541+0000 7fecd0fe9740 1 -- start start 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecd0fe9740 1 Processor -- start 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f8c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f87450 0x55d3b2f93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d3b2f8a3a0 con 0x55d3b2f87450 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3b2ea2160 con 0x55d3b2f818f0 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3b2f8a520 con 0x55d3b2f822e0 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecd0fe9740 10 client.0 _ll_register_callbacks cb 0x55d3b2ede720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:17.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7feccc991640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7feccc991640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57482/0 (socket says 172.21.15.105:57482) 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7feccc991640 1 -- 172.21.15.105:0/3464136517 learned_addr learned my addr 172.21.15.105:0/3464136517 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.542+0000 7feccc991640 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f87450 0x55d3b2f93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feccc991640 1 -- 172.21.15.105:0/3464136517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 msgr2=0x55d3b2f8c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feccc991640 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f8c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feccc991640 1 -- 172.21.15.105:0/3464136517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f87450 msgr2=0x55d3b2f93140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feccc991640 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f87450 0x55d3b2f93140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feccc991640 1 -- 172.21.15.105:0/3464136517 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b2bfa3f0 con 0x55d3b2f818f0 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7fecc77fe640 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f8c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feccc991640 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bbc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fecc0036040 tx=0x7fecc0009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c550d81267f30616 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feca77fe640 1 -- 172.21.15.105:0/3464136517 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0043040 con 0x55d3b2f818f0 2024-03-26T17:53:17.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3464136517 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecc0003970 con 0x55d3b2f818f0 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feca77fe640 1 -- 172.21.15.105:0/3464136517 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fecc0034590 con 0x55d3b2f818f0 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7feca77fe640 1 -- 172.21.15.105:0/3464136517 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fecc0034790 con 0x55d3b2f818f0 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7fecd0fe9740 10 client.30641 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.543+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3464136517 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3b2bba0f0 con 0x55d3b2f818f0 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7feca77fe640 1 -- 172.21.15.105:0/3464136517 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc0003970 con 0x55d3b2f818f0 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7feca77fe640 1 -- 172.21.15.105:0/3464136517 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fecc003fb10 con 0x55d3b2f818f0 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7fecd0fe9740 10 client.30641 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7fecd0fe9740 10 client.30641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7fecd0fe9740 20 client.30641 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3b2f97630 0x55d3b2f99a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7fecd0fe9740 4 client.30641 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3464136517 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3b2ef3ec0 con 0x55d3b2f97630 2024-03-26T17:53:17.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.544+0000 7fecc77fe640 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3b2f97630 0x55d3b2f99a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.545+0000 7feca77fe640 10 client.30641 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:17.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.545+0000 7fecc77fe640 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3b2f97630 0x55d3b2f99a20 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb4cbf504a41437 server_cookie=3f06c2ec2db421d0 in_seq=0 out_seq=0 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.547+0000 7feca77fe640 1 -- 172.21.15.105:0/3464136517 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d3b2ef3ec0 con 0x55d3b2f97630 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.547+0000 7feca77fe640 10 client.30641 handle_command_reply: tid=0 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.547+0000 7fecd0fe9740 1 client.30641 shutdown 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3464136517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f818f0 msgr2=0x55d3b2f8bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bbc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fecc0036040 tx=0x7fecc0009280 comp rx=0 tx=0).stop 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3464136517 shutdown_connections 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3b2f97630 0x55d3b2f99a20 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f87450 0x55d3b2f93140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7feca77fe640 0 client.30641 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:17.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f8c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3464136517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bbc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3464136517 >> 172.21.15.105:0/3464136517 conn(0x55d3b2f7ce60 msgr2=0x55d3b2f7c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3464136517 shutdown_connections 2024-03-26T17:53:17.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3464136517 wait complete. 2024-03-26T17:53:17.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.548+0000 7fecd0fe9740 20 client.30641 trim_cache size 0 max 16384 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecd0fe9740 1 Processor -- start 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecd0fe9740 1 -- start start 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 0x55d3b2f7b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81a6b30 con 0x55d3b2f818f0 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81a8730 con 0x55d3b2f87450 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81a7200 con 0x55d3b2f822e0 2024-03-26T17:53:17.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7feccc991640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7feccc991640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47388/0 (socket says 172.21.15.105:47388) 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7feccc991640 1 -- 172.21.15.105:0/3798196843 learned_addr learned my addr 172.21.15.105:0/3798196843 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7fecc77fe640 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.572+0000 7feccc991640 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 0x55d3b2f7b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7feccc991640 1 -- 172.21.15.105:0/3798196843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 msgr2=0x55d3b2f930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7feccc991640 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7feccc991640 1 -- 172.21.15.105:0/3798196843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 msgr2=0x55d3b2f7b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7feccc991640 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 0x55d3b2f7b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7feccc991640 1 -- 172.21.15.105:0/3798196843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b2f7dbd0 con 0x55d3b2f818f0 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7feccc991640 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bee0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fecc003ff30 tx=0x7fecc0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5599070a0f8cef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7fecadffb640 1 -- 172.21.15.105:0/3798196843 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0043070 con 0x55d3b2f818f0 2024-03-26T17:53:17.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7fecadffb640 1 -- 172.21.15.105:0/3798196843 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc004c040 con 0x55d3b2f818f0 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.573+0000 7fecadffb640 1 -- 172.21.15.105:0/3798196843 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0034930 con 0x55d3b2f818f0 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3798196843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 msgr2=0x55d3b2f8bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bee0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fecc003ff30 tx=0x7fecc0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3798196843 shutdown_connections 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f87450 0x55d3b2f7b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f822e0 0x55d3b2f930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/3798196843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f8bee0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3798196843 >> 172.21.15.105:0/3798196843 conn(0x55d3b2f7ce60 msgr2=0x55d3b2f8b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3798196843 shutdown_connections 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 -- 172.21.15.105:0/3798196843 wait complete. 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 -- start start 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.574+0000 7fecd0fe9740 1 Processor -- start 2024-03-26T17:53:17.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f90250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f822e0 0x55d3b2f90790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecd0fe9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f87450 0x55d3b2f917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81a8060 con 0x55d3b2f818f0 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81a78d0 con 0x55d3b2f822e0 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecd0fe9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81a4b10 con 0x55d3b2f87450 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecd0fe9740 10 client.0 _ll_register_callbacks cb 0x7fecb80d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f822e0 0x55d3b2f90790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f822e0 0x55d3b2f90790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57506/0 (socket says 172.21.15.105:57506) 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feccc991640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f90250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7fecc77fe640 1 -- 172.21.15.105:0/1998436959 learned_addr learned my addr 172.21.15.105:0/1998436959 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feccc991640 1 -- 172.21.15.105:0/1998436959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f87450 msgr2=0x55d3b2f917a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:17.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feccc991640 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f87450 0x55d3b2f917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feccc991640 1 -- 172.21.15.105:0/1998436959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f822e0 msgr2=0x55d3b2f90790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feccc991640 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f822e0 0x55d3b2f90790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feccc991640 1 -- 172.21.15.105:0/1998436959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b2f91d90 con 0x55d3b2f818f0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feccc991640 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f90250 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fecc00484b0 tx=0x7fecc002da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c050e4dafca7821 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feca77fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.575+0000 7feca77fe640 1 -- 172.21.15.105:0/1998436959 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0063020 con 0x55d3b2f818f0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7feca77fe640 1 -- 172.21.15.105:0/1998436959 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fecc0004b40 con 0x55d3b2f818f0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 1 -- 172.21.15.105:0/1998436959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecc004c040 con 0x55d3b2f818f0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7feca77fe640 1 -- 172.21.15.105:0/1998436959 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fecc0043040 con 0x55d3b2f818f0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 10 client.20865 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 1 -- 172.21.15.105:0/1998436959 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d3b2f930a0 con 0x55d3b2f818f0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7feca77fe640 1 -- 172.21.15.105:0/1998436959 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc004c040 con 0x55d3b2f818f0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7feca77fe640 1 -- 172.21.15.105:0/1998436959 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fecc0003cc0 con 0x55d3b2f818f0 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 10 client.20865 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:17.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 10 client.20865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:17.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 20 client.20865 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:17.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3b2f97630 0x55d3b2fffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 4 client.20865 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:53:17.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.576+0000 7fecd0fe9740 1 -- 172.21.15.105:0/1998436959 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d3b2eee210 con 0x55d3b2f97630 2024-03-26T17:53:17.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.577+0000 7fecc77fe640 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3b2f97630 0x55d3b2fffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.577+0000 7feca77fe640 10 client.20865 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:17.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.577+0000 7fecc77fe640 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3b2f97630 0x55d3b2fffbd0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cc7df347906a49 server_cookie=cf18d0440fc26493 in_seq=0 out_seq=0 2024-03-26T17:53:17.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7feca77fe640 1 -- 172.21.15.105:0/1998436959 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d3b2eee210 con 0x55d3b2f97630 2024-03-26T17:53:17.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7feca77fe640 10 client.20865 handle_command_reply: tid=0 2024-03-26T17:53:17.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 client.20865 shutdown 2024-03-26T17:53:17.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 -- 172.21.15.105:0/1998436959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 msgr2=0x55d3b2f90250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f90250 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fecc00484b0 tx=0x7fecc002da80 comp rx=0 tx=0).stop 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 -- 172.21.15.105:0/1998436959 shutdown_connections 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3b2f97630 0x55d3b2fffbd0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7feca77fe640 0 client.20865 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3b2f87450 0x55d3b2f917a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3b2f822e0 0x55d3b2f90790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 --2- 172.21.15.105:0/1998436959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3b2f818f0 0x55d3b2f90250 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 -- 172.21.15.105:0/1998436959 >> 172.21.15.105:0/1998436959 conn(0x55d3b2f7ce60 msgr2=0x55d3b2f800c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 -- 172.21.15.105:0/1998436959 shutdown_connections 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 1 -- 172.21.15.105:0/1998436959 wait complete. 2024-03-26T17:53:17.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecd0fe9740 20 client.20865 trim_cache size 0 max 16384 2024-03-26T17:53:17.576 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:53:17.576 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:53:17.576 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:53:17.576 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecce3f4640 1 -- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fecb00a2fa0 msgr2=0x7fecb00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.579+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fecb00a2fa0 0x7fecb00a5460 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fecb80cd550 tx=0x7fecb80d5550 comp rx=0 tx=0).stop 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 -- 172.21.15.105:0/1771360964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81018f0 msgr2=0x7fecc81958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81018f0 0x7fecc81958b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fecb800e9c0 tx=0x7fecb800ee80 comp rx=0 tx=0).stop 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 -- 172.21.15.105:0/1771360964 shutdown_connections 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fecb00a2fa0 0x7fecb00a5460 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecc81034a0 0x7fecc819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecc8102ae0 0x7fecc8195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 --2- 172.21.15.105:0/1771360964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fecc81018f0 0x7fecc81958b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 -- 172.21.15.105:0/1771360964 >> 172.21.15.105:0/1771360964 conn(0x7fecc80fce60 msgr2=0x7fecc8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 -- 172.21.15.105:0/1771360964 shutdown_connections 2024-03-26T17:53:17.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.580+0000 7fecce3f4640 1 -- 172.21.15.105:0/1771360964 wait complete. 2024-03-26T17:53:17.584 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:53:17.584 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:53:17.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.740+0000 7fa33b8e8640 1 -- 172.21.15.105:0/2322206805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa33410dec0 msgr2=0x7fa3341102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.740+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/2322206805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa33410dec0 0x7fa3341102b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009ca0 tx=0x7fa32c02c100 comp rx=0 tx=0).stop 2024-03-26T17:53:17.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.740+0000 7fa33b8e8640 1 -- 172.21.15.105:0/2322206805 shutdown_connections 2024-03-26T17:53:17.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.740+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/2322206805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa33410dec0 0x7fa3341102b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.740+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/2322206805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa334074320 0x7fa3340747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.740+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/2322206805 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa334075900 0x7fa334073d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.740+0000 7fa33b8e8640 1 -- 172.21.15.105:0/2322206805 >> 172.21.15.105:0/2322206805 conn(0x7fa3340fce60 msgr2=0x7fa3340ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:17.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.741+0000 7fa33b8e8640 1 -- 172.21.15.105:0/2322206805 shutdown_connections 2024-03-26T17:53:17.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.741+0000 7fa33b8e8640 1 -- 172.21.15.105:0/2322206805 wait complete. 2024-03-26T17:53:17.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.742+0000 7fa33b8e8640 1 Processor -- start 2024-03-26T17:53:17.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.742+0000 7fa33b8e8640 1 -- start start 2024-03-26T17:53:17.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.742+0000 7fa33b8e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa334074320 0x7fa33419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa33b8e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa334075900 0x7fa33419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa33b8e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33410dec0 0x7fa3341a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa33b8e8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3341139f0 con 0x7fa334074320 2024-03-26T17:53:17.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa33b8e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113870 con 0x7fa33410dec0 2024-03-26T17:53:17.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa33b8e8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa334113b70 con 0x7fa334075900 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa339684640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa334074320 0x7fa33419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa339e85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33410dec0 0x7fa3341a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa338e83640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa334075900 0x7fa33419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa338e83640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa334075900 0x7fa33419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33788/0 (socket says 172.21.15.105:33788) 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa338e83640 1 -- 172.21.15.105:0/3927119958 learned_addr learned my addr 172.21.15.105:0/3927119958 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.743+0000 7fa339684640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa334074320 0x7fa33419e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47422/0 (socket says 172.21.15.105:47422) 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa339e85640 1 -- 172.21.15.105:0/3927119958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa334075900 msgr2=0x7fa33419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa339e85640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa334075900 0x7fa33419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa339e85640 1 -- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa334074320 msgr2=0x7fa33419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa339e85640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa334074320 0x7fa33419e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:17.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa339e85640 1 -- 172.21.15.105:0/3927119958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3341a5ca0 con 0x7fa33410dec0 2024-03-26T17:53:17.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa339684640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa334074320 0x7fa33419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:17.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa339e85640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33410dec0 0x7fa3341a5630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa32c02c640 tx=0x7fa32c0328a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa32a7fc640 1 -- 172.21.15.105:0/3927119958 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c041020 con 0x7fa33410dec0 2024-03-26T17:53:17.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.744+0000 7fa32a7fc640 1 -- 172.21.15.105:0/3927119958 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa32c003e20 con 0x7fa33410dec0 2024-03-26T17:53:17.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.745+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3341a5f30 con 0x7fa33410dec0 2024-03-26T17:53:17.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.745+0000 7fa32a7fc640 1 -- 172.21.15.105:0/3927119958 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c039040 con 0x7fa33410dec0 2024-03-26T17:53:17.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.745+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3341a6390 con 0x7fa33410dec0 2024-03-26T17:53:17.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.746+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3180052c0 con 0x7fa33410dec0 2024-03-26T17:53:17.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.747+0000 7fa32a7fc640 1 -- 172.21.15.105:0/3927119958 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa32c044020 con 0x7fa33410dec0 2024-03-26T17:53:17.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.748+0000 7fa32a7fc640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3200a2f20 0x7fa3200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:17.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.748+0000 7fa32a7fc640 1 -- 172.21.15.105:0/3927119958 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa32c0f1500 con 0x7fa33410dec0 2024-03-26T17:53:17.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.748+0000 7fa339684640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3200a2f20 0x7fa3200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:17.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.749+0000 7fa339684640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3200a2f20 0x7fa3200a53e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa324005d00 tx=0x7fa32400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:17.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:17.755+0000 7fa32a7fc640 1 -- 172.21.15.105:0/3927119958 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa32c04b020 con 0x7fa33410dec0 2024-03-26T17:53:18.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.113+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3180055b0 con 0x7fa33410dec0 2024-03-26T17:53:18.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.114+0000 7fa32a7fc640 1 -- 172.21.15.105:0/3927119958 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11948 (secure 0 0 0) 0x7fa32c0be2f0 con 0x7fa33410dec0 2024-03-26T17:53:18.105 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:53:18.105 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:51:26.802032+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:53:18.105 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 34 2024-03-26T17:53:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3200a2f20 msgr2=0x7fa3200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3200a2f20 0x7fa3200a53e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa324005d00 tx=0x7fa32400b410 comp rx=0 tx=0).stop 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33410dec0 msgr2=0x7fa3341a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33410dec0 0x7fa3341a5630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa32c02c640 tx=0x7fa32c0328a0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 shutdown_connections 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3200a2f20 0x7fa3200a53e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33410dec0 0x7fa3341a5630 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa334075900 0x7fa33419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 --2- 172.21.15.105:0/3927119958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa334074320 0x7fa33419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 >> 172.21.15.105:0/3927119958 conn(0x7fa3340fce60 msgr2=0x7fa334101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 shutdown_connections 2024-03-26T17:53:18.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.118+0000 7fa33b8e8640 1 -- 172.21.15.105:0/3927119958 wait complete. 2024-03-26T17:53:18.123 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:53:18.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:17 smithi105 ceph-mon[32280]: pgmap v797: 97 pgs: 97 active+clean; 238 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.9 MiB/s wr, 263 op/s 2024-03-26T17:53:18.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:17 smithi135 ceph-mon[35636]: pgmap v797: 97 pgs: 97 active+clean; 238 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.9 MiB/s wr, 263 op/s 2024-03-26T17:53:18.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 -- 172.21.15.105:0/1393399372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc1035d0 msgr2=0x7f0abc109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1393399372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc1035d0 0x7f0abc109e60 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009cd0 tx=0x7f0ab402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 -- 172.21.15.105:0/1393399372 shutdown_connections 2024-03-26T17:53:18.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1393399372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc1035d0 0x7f0abc109e60 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1393399372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0abc102c10 0x7f0abc103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1393399372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0abc0ffba0 0x7f0abc0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 -- 172.21.15.105:0/1393399372 >> 172.21.15.105:0/1393399372 conn(0x7f0abc0fb4b0 msgr2=0x7f0abc0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 -- 172.21.15.105:0/1393399372 shutdown_connections 2024-03-26T17:53:18.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.284+0000 7f0ac392f640 1 -- 172.21.15.105:0/1393399372 wait complete. 2024-03-26T17:53:18.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac392f640 1 Processor -- start 2024-03-26T17:53:18.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac392f640 1 -- start start 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac392f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc0ffba0 0x7f0abc1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac392f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0abc102c10 0x7f0abc195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac392f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0abc1035d0 0x7f0abc19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac392f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10b240 con 0x7f0abc0ffba0 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac392f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10b0c0 con 0x7f0abc1035d0 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac392f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10b3c0 con 0x7f0abc102c10 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac16cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc0ffba0 0x7f0abc1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac16cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc0ffba0 0x7f0abc1958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47438/0 (socket says 172.21.15.105:47438) 2024-03-26T17:53:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac16cb640 1 -- 172.21.15.105:0/1965390611 learned_addr learned my addr 172.21.15.105:0/1965390611 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac0eca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0abc102c10 0x7f0abc195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.285+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0abc1035d0 0x7f0abc19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac16cb640 1 -- 172.21.15.105:0/1965390611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0abc102c10 msgr2=0x7f0abc195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac16cb640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0abc102c10 0x7f0abc195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac16cb640 1 -- 172.21.15.105:0/1965390611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0abc1035d0 msgr2=0x7f0abc19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac16cb640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0abc1035d0 0x7f0abc19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac16cb640 1 -- 172.21.15.105:0/1965390611 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc19d500 con 0x7f0abc0ffba0 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0abc102c10 0x7f0abc195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0abc1035d0 0x7f0abc19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac16cb640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc0ffba0 0x7f0abc1958e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e9c0 tx=0x7f0ab800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ab27fc640 1 -- 172.21.15.105:0/1965390611 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800cd80 con 0x7f0abc0ffba0 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ab27fc640 1 -- 172.21.15.105:0/1965390611 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab80035f0 con 0x7f0abc0ffba0 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.286+0000 7f0ac392f640 1 -- 172.21.15.105:0/1965390611 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc19d7f0 con 0x7f0abc0ffba0 2024-03-26T17:53:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.287+0000 7f0ab27fc640 1 -- 172.21.15.105:0/1965390611 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8010690 con 0x7f0abc0ffba0 2024-03-26T17:53:18.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.287+0000 7f0ac392f640 1 -- 172.21.15.105:0/1965390611 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc19dd40 con 0x7f0abc0ffba0 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac65f0740 1 Processor -- start 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac65f0740 1 -- start start 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ab27fc640 1 -- 172.21.15.105:0/1965390611 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0ab80037a0 con 0x7f0abc0ffba0 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecaabf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaac590 0x55b8ecaaea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b8eca0b970 con 0x55b8ecaa7420 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec9b3d70 con 0x55b8ecaac590 2024-03-26T17:53:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b8eca07920 con 0x55b8ecaa6a30 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac1ecc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac1ecc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33820/0 (socket says 172.21.15.105:33820) 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/2541934118 learned_addr learned my addr 172.21.15.105:0/2541934118 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaac590 0x55b8ecaaea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.290+0000 7f0ac16cb640 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecaabf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/2541934118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaac590 msgr2=0x55b8ecaaea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaac590 0x55b8ecaaea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/2541934118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 msgr2=0x55b8ecaabf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecaabf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/2541934118 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ec71f3f0 con 0x55b8ecaa6a30 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ac16cb640 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecaabf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ab27fc640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0aa80a2f20 0x7f0aa80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ac16cb640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0aa80a2f20 0x7f0aa80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa6e50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4032380 tx=0x7f0ab4005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faaf57ffb0bb4347 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.291+0000 7f0aa77fe640 1 -- 172.21.15.105:0/2541934118 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4043070 con 0x55b8ecaa6a30 2024-03-26T17:53:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.292+0000 7f0aa77fe640 1 -- 172.21.15.105:0/2541934118 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab4003e20 con 0x55b8ecaa6a30 2024-03-26T17:53:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.292+0000 7f0aa77fe640 1 -- 172.21.15.105:0/2541934118 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4039040 con 0x55b8ecaa6a30 2024-03-26T17:53:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.292+0000 7f0ab27fc640 1 -- 172.21.15.105:0/1965390611 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0ab80cf800 con 0x7f0abc0ffba0 2024-03-26T17:53:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.292+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2541934118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 msgr2=0x55b8ecaa6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.292+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa6e50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4032380 tx=0x7f0ab4005f70 comp rx=0 tx=0).stop 2024-03-26T17:53:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.292+0000 7f0ac16cb640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0aa80a2f20 0x7f0aa80a53e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8091fd0 tx=0x7f0ab80d1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2541934118 shutdown_connections 2024-03-26T17:53:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaac590 0x55b8ecaaea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecaabf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2541934118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa6e50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2541934118 >> 172.21.15.105:0/2541934118 conn(0x55b8ecaa1fa0 msgr2=0x55b8ecaa23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2541934118 shutdown_connections 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2541934118 wait complete. 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 -- start start 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.293+0000 7f0ac65f0740 1 Processor -- start 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa6a30 0x55b8ecab95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa6a30 0x55b8ecab95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa6a30 0x55b8ecab95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47458/0 (socket says 172.21.15.105:47458) 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 0x55b8ecab9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/1495305056 learned_addr learned my addr 172.21.15.105:0/1495305056 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 0x55b8ecac0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b8ecaaf4e0 con 0x55b8ecaa6a30 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b8ec9c7310 con 0x55b8ecaa7420 2024-03-26T17:53:18.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b8ecaaf660 con 0x55b8ecaac590 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac65f0740 10 client.0 _ll_register_callbacks cb 0x55b8eca02c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 0x55b8ecab9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 0x55b8ecac0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/1495305056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 msgr2=0x55b8ecac0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 0x55b8ecac0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/1495305056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 msgr2=0x55b8ecab9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 0x55b8ecab9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.294+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/1495305056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ec7e97d0 con 0x55b8ecaa6a30 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa6a30 0x55b8ecab95b0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402c630 tx=0x7f0ab4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42b67582a71a39e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0aa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/1495305056 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4039070 con 0x55b8ecaa6a30 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/1495305056 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0ab40046c0 con 0x55b8ecaa6a30 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab4003e20 con 0x55b8ecaa6a30 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/1495305056 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ab4043040 con 0x55b8ecaa6a30 2024-03-26T17:53:18.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0ac65f0740 10 client.20883 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b8ecac3ea0 con 0x55b8ecaa6a30 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.295+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/1495305056 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab4003e20 con 0x55b8ecaa6a30 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/1495305056 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0ab4048450 con 0x55b8ecaa6a30 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0ac65f0740 10 client.20883 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0ac65f0740 10 client.20883 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0ac65f0740 20 client.20883 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b8ecac51f0 0x55b8ecac75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0ac65f0740 4 client.20883 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8eca18eb0 con 0x55b8ecac51f0 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b8ecac51f0 0x55b8ecac75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.296+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b8ecac51f0 0x55b8ecac75e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=18f4f4fc902c61bd server_cookie=a5fdadd94b4d032e in_seq=0 out_seq=0 2024-03-26T17:53:18.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.297+0000 7f0aa4ff9640 10 client.20883 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.297+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/1495305056 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b8eca18eb0 con 0x55b8ecac51f0 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.297+0000 7f0aa4ff9640 10 client.20883 handle_command_reply: tid=0 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.298+0000 7f0ac65f0740 1 client.20883 shutdown 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.298+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa6a30 msgr2=0x55b8ecab95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.298+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa6a30 0x55b8ecab95b0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402c630 tx=0x7f0ab4009620 comp rx=0 tx=0).stop 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.298+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 shutdown_connections 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.298+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b8ecac51f0 0x55b8ecac75e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.299+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 0x55b8ecac0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.299+0000 7f0aa4ff9640 0 client.20883 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:53:18.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.299+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 0x55b8ecab9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.299+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1495305056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa6a30 0x55b8ecab95b0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.299+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 >> 172.21.15.105:0/1495305056 conn(0x55b8ecaa1fa0 msgr2=0x55b8ecaa11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.299+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 shutdown_connections 2024-03-26T17:53:18.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.299+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1495305056 wait complete. 2024-03-26T17:53:18.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.299+0000 7f0ac65f0740 20 client.20883 trim_cache size 0 max 16384 2024-03-26T17:53:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac65f0740 1 Processor -- start 2024-03-26T17:53:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac65f0740 1 -- start start 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecac0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 0x55b8ecab91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a6b50 con 0x55b8ecaa7420 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a8750 con 0x55b8ecaa6a30 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a7220 con 0x55b8ecaac590 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac0eca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecac0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac1ecc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac0eca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecac0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47474/0 (socket says 172.21.15.105:47474) 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac0eca640 1 -- 172.21.15.105:0/1853572347 learned_addr learned my addr 172.21.15.105:0/1853572347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 0x55b8ecab91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/1853572347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 msgr2=0x55b8ecab91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 0x55b8ecab91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/1853572347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 msgr2=0x55b8ecac0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecac0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.322+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/1853572347 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ecaaba90 con 0x55b8ecaa6a30 2024-03-26T17:53:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.323+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecac0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.323+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa5910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40488a0 tx=0x7f0ab4032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2ec88baa2703ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.323+0000 7f0aa6ffd640 1 -- 172.21.15.105:0/1853572347 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4039050 con 0x55b8ecaa6a30 2024-03-26T17:53:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.323+0000 7f0aa6ffd640 1 -- 172.21.15.105:0/1853572347 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab4051070 con 0x55b8ecaa6a30 2024-03-26T17:53:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.323+0000 7f0aa6ffd640 1 -- 172.21.15.105:0/1853572347 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab403f440 con 0x55b8ecaa6a30 2024-03-26T17:53:18.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1853572347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa6a30 msgr2=0x55b8ecaa5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa5910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40488a0 tx=0x7f0ab4032340 comp rx=0 tx=0).stop 2024-03-26T17:53:18.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1853572347 shutdown_connections 2024-03-26T17:53:18.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaac590 0x55b8ecab91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaa7420 0x55b8ecac0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/1853572347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa6a30 0x55b8ecaa5910 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1853572347 >> 172.21.15.105:0/1853572347 conn(0x55b8ecaa1fa0 msgr2=0x55b8ecaa5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1853572347 shutdown_connections 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.324+0000 7f0ac65f0740 1 -- 172.21.15.105:0/1853572347 wait complete. 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 1 -- start start 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 1 Processor -- start 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecabdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 0x55b8ecabe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaac590 0x55b8ecabf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a8080 con 0x55b8ecaac590 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a78f0 con 0x55b8ecaa7420 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a4b30 con 0x55b8ecaa6a30 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.325+0000 7f0ac65f0740 10 client.0 _ll_register_callbacks cb 0x7f0aac0042d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecabdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecabdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33846/0 (socket says 172.21.15.105:33846) 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/2480516248 learned_addr learned my addr 172.21.15.105:0/2480516248 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 0x55b8ecabe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaac590 0x55b8ecabf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/2480516248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 msgr2=0x55b8ecabe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 0x55b8ecabe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/2480516248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaac590 msgr2=0x55b8ecabf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaac590 0x55b8ecabf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 -- 172.21.15.105:0/2480516248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ecabfaa0 con 0x55b8ecaa6a30 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac1ecc640 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecabdf60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4005e30 tx=0x7f0ab4000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd4ed7a8915ba95 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0aa4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/2480516248 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4039040 con 0x55b8ecaa6a30 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2480516248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab4051070 con 0x55b8ecaa6a30 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.326+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/2480516248 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0ab4004d40 con 0x55b8ecaa6a30 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/2480516248 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ab4004230 con 0x55b8ecaa6a30 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac65f0740 10 client.30649 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2480516248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b8ecac0da0 con 0x55b8ecaa6a30 2024-03-26T17:53:18.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/2480516248 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab4051070 con 0x55b8ecaa6a30 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/2480516248 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0ab406c020 con 0x55b8ecaa6a30 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac65f0740 10 client.30649 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac65f0740 10 client.30649 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac65f0740 20 client.30649 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b8ecac51f0 0x55b8ecb2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac65f0740 4 client.30649 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2480516248 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8eca13200 con 0x55b8ecac51f0 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.327+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b8ecac51f0 0x55b8ecb2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac0eca640 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b8ecac51f0 0x55b8ecb2da00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c31900e830c74db server_cookie=f5d782a6582ac5f0 in_seq=0 out_seq=0 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0aa4ff9640 10 client.30649 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0aa4ff9640 1 -- 172.21.15.105:0/2480516248 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8eca13200 con 0x55b8ecac51f0 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0aa4ff9640 10 client.30649 handle_command_reply: tid=0 2024-03-26T17:53:18.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 client.30649 shutdown 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2480516248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 msgr2=0x55b8ecabdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecabdf60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4005e30 tx=0x7f0ab4000f40 comp rx=0 tx=0).stop 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2480516248 shutdown_connections 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b8ecac51f0 0x55b8ecb2da00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b8ecaac590 0x55b8ecabf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0aa4ff9640 0 client.30649 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b8ecaa7420 0x55b8ecabe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 --2- 172.21.15.105:0/2480516248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8ecaa6a30 0x55b8ecabdf60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.328+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2480516248 >> 172.21.15.105:0/2480516248 conn(0x55b8ecaa1fa0 msgr2=0x55b8ecab9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2480516248 shutdown_connections 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac65f0740 1 -- 172.21.15.105:0/2480516248 wait complete. 2024-03-26T17:53:18.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac65f0740 20 client.30649 trim_cache size 0 max 16384 2024-03-26T17:53:18.325 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:18.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac392f640 1 -- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0aa80a2f20 msgr2=0x7f0aa80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0aa80a2f20 0x7f0aa80a53e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8091fd0 tx=0x7f0ab80d1040 comp rx=0 tx=0).stop 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac392f640 1 -- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc0ffba0 msgr2=0x7f0abc1958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc0ffba0 0x7f0abc1958e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e9c0 tx=0x7f0ab800ee80 comp rx=0 tx=0).stop 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac392f640 1 -- 172.21.15.105:0/1965390611 shutdown_connections 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0aa80a2f20 0x7f0aa80a53e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.329+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0abc1035d0 0x7f0abc19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.330+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0abc102c10 0x7f0abc195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.330+0000 7f0ac392f640 1 --2- 172.21.15.105:0/1965390611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0abc0ffba0 0x7f0abc1958e0 secure :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e9c0 tx=0x7f0ab800ee80 comp rx=0 tx=0).stop 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.330+0000 7f0ac392f640 1 -- 172.21.15.105:0/1965390611 >> 172.21.15.105:0/1965390611 conn(0x7f0abc0fb4b0 msgr2=0x7f0abc105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.330+0000 7f0ac392f640 1 -- 172.21.15.105:0/1965390611 shutdown_connections 2024-03-26T17:53:18.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.330+0000 7f0ac392f640 1 -- 172.21.15.105:0/1965390611 wait complete. 2024-03-26T17:53:18.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:53:18.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:17 smithi144 ceph-mon[36095]: pgmap v797: 97 pgs: 97 active+clean; 238 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 2.9 MiB/s wr, 263 op/s 2024-03-26T17:53:18.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.530+0000 7f2d0baf1640 1 -- 172.21.15.105:0/230952414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d041075a0 msgr2=0x7f2d041099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.530+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/230952414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d041075a0 0x7f2d041099f0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009ca0 tx=0x7f2cfc02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.530+0000 7f2d0baf1640 1 -- 172.21.15.105:0/230952414 shutdown_connections 2024-03-26T17:53:18.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.530+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/230952414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d041075a0 0x7f2d041099f0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.530+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/230952414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d04104c70 0x7f2d04107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.530+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/230952414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d04102340 0x7f2d04104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.530+0000 7f2d0baf1640 1 -- 172.21.15.105:0/230952414 >> 172.21.15.105:0/230952414 conn(0x7f2d040fbd50 msgr2=0x7f2d040fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.531+0000 7f2d0baf1640 1 -- 172.21.15.105:0/230952414 shutdown_connections 2024-03-26T17:53:18.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.531+0000 7f2d0baf1640 1 -- 172.21.15.105:0/230952414 wait complete. 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.531+0000 7f2d0baf1640 1 Processor -- start 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.531+0000 7f2d0baf1640 1 -- start start 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0baf1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d04102340 0x7f2d0419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0baf1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d04104c70 0x7f2d0419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0baf1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d041075a0 0x7f2d041a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0baf1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04101b00 con 0x7f2d04104c70 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0baf1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04101980 con 0x7f2d041075a0 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0baf1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d04101c80 con 0x7f2d04102340 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0908c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d04104c70 0x7f2d0419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0988d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d04102340 0x7f2d0419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0988d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d04102340 0x7f2d0419e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33862/0 (socket says 172.21.15.105:33862) 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0988d640 1 -- 172.21.15.105:0/937559970 learned_addr learned my addr 172.21.15.105:0/937559970 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d041075a0 0x7f2d041a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0988d640 1 -- 172.21.15.105:0/937559970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d041075a0 msgr2=0x7f2d041a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0988d640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d041075a0 0x7f2d041a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0988d640 1 -- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d04104c70 msgr2=0x7f2d0419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0988d640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d04104c70 0x7f2d0419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0988d640 1 -- 172.21.15.105:0/937559970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d041a5d30 con 0x7f2d04102340 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.532+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d041075a0 0x7f2d041a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.533+0000 7f2d0988d640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d04102340 0x7f2d0419e110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000ea60 tx=0x7f2d0000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.533+0000 7f2cfaffd640 1 -- 172.21.15.105:0/937559970 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000cde0 con 0x7f2d04102340 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.533+0000 7f2cfaffd640 1 -- 172.21.15.105:0/937559970 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d000035f0 con 0x7f2d04102340 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.533+0000 7f2cfaffd640 1 -- 172.21.15.105:0/937559970 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00010690 con 0x7f2d04102340 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.533+0000 7f2d0baf1640 1 -- 172.21.15.105:0/937559970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d041a5fc0 con 0x7f2d04102340 2024-03-26T17:53:18.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.533+0000 7f2d0baf1640 1 -- 172.21.15.105:0/937559970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d041a64d0 con 0x7f2d04102340 2024-03-26T17:53:18.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.536+0000 7f2d0e7b8740 1 Processor -- start 2024-03-26T17:53:18.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.536+0000 7f2d0e7b8740 1 -- start start 2024-03-26T17:53:18.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b1a30 0x5601b02b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b2420 0x5601b02b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5601b0216970 con 0x5601b02b2420 2024-03-26T17:53:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601b01bed70 con 0x5601b02b7590 2024-03-26T17:53:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5601b0212920 con 0x5601b02b1a30 2024-03-26T17:53:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0908c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0908c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02b9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57638/0 (socket says 172.21.15.105:57638) 2024-03-26T17:53:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0908c640 1 -- 172.21.15.105:0/3986401770 learned_addr learned my addr 172.21.15.105:0/3986401770 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2cfaffd640 1 -- 172.21.15.105:0/937559970 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2d000037a0 con 0x7f2d04102340 2024-03-26T17:53:18.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b1a30 0x5601b02b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0988d640 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b2420 0x5601b02b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0908c640 1 -- 172.21.15.105:0/3986401770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b1a30 msgr2=0x5601b02b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0908c640 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b1a30 0x5601b02b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0908c640 1 -- 172.21.15.105:0/3986401770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b2420 msgr2=0x5601b02b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0908c640 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b2420 0x5601b02b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0908c640 1 -- 172.21.15.105:0/3986401770 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601aff2a3f0 con 0x5601b02b7590 2024-03-26T17:53:18.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.537+0000 7f2d0988d640 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b2420 0x5601b02b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.538+0000 7f2d0908c640 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02b9a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009c60 tx=0x7f2cf4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38e9c073110d37eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.538+0000 7f2cfaffd640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2cf00a2f90 0x7f2cf00a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.538+0000 7f2ceffff640 1 -- 172.21.15.105:0/3986401770 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4004230 con 0x5601b02b7590 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.538+0000 7f2ceffff640 1 -- 172.21.15.105:0/3986401770 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cf40043d0 con 0x5601b02b7590 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.538+0000 7f2ceffff640 1 -- 172.21.15.105:0/3986401770 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4004d10 con 0x5601b02b7590 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.538+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2cf00a2f90 0x7f2cf00a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.538+0000 7f2cfaffd640 1 -- 172.21.15.105:0/937559970 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2d000cf6e0 con 0x7f2d04102340 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3986401770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 msgr2=0x5601b02b9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2cf00a2f90 0x7f2cf00a5450 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006010 tx=0x7f2cfc005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02b9a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009c60 tx=0x7f2cf4011810 comp rx=0 tx=0).stop 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3986401770 shutdown_connections 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02b9a20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b2420 0x5601b02b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3986401770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b1a30 0x5601b02b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3986401770 >> 172.21.15.105:0/3986401770 conn(0x5601b02acfa0 msgr2=0x5601b02ad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3986401770 shutdown_connections 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3986401770 wait complete. 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 -- start start 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.539+0000 7f2d0e7b8740 1 Processor -- start 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02bbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02bc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02c3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5601b02ba540 con 0x5601b02b1a30 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601b01d2310 con 0x5601b02b7590 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5601b02ba6c0 con 0x5601b02b2420 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0988d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02bbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0988d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02bbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47514/0 (socket says 172.21.15.105:47514) 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0988d640 1 -- 172.21.15.105:0/3190346612 learned_addr learned my addr 172.21.15.105:0/3190346612 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0908c640 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02bc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0e7b8740 10 client.0 _ll_register_callbacks cb 0x5601b020dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0988d640 1 -- 172.21.15.105:0/3190346612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 msgr2=0x5601b02bc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0988d640 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02bc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0988d640 1 -- 172.21.15.105:0/3190346612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 msgr2=0x5601b02c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0988d640 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.540+0000 7f2d0988d640 1 -- 172.21.15.105:0/3190346612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601afff47d0 con 0x5601b02b1a30 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2d0988d640 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02bbee0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2d0009bad0 tx=0x7f2d00098130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b3e75c0168c0b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2ced7fa640 1 -- 172.21.15.105:0/3190346612 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00003b30 con 0x5601b02b1a30 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3190346612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cf40043d0 con 0x5601b02b1a30 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2ced7fa640 1 -- 172.21.15.105:0/3190346612 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2d000cfb90 con 0x5601b02b1a30 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2ced7fa640 1 -- 172.21.15.105:0/3190346612 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2d0009c4d0 con 0x5601b02b1a30 2024-03-26T17:53:18.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2d0e7b8740 10 client.20898 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3190346612 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5601b02c67e0 con 0x5601b02b1a30 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.541+0000 7f2ced7fa640 1 -- 172.21.15.105:0/3190346612 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cf40043d0 con 0x5601b02b1a30 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2ced7fa640 1 -- 172.21.15.105:0/3190346612 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2d0009b220 con 0x5601b02b1a30 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2d0e7b8740 10 client.20898 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2d0e7b8740 10 client.20898 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2d0e7b8740 20 client.20898 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5601b02c7e10 0x5601b02ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2d0e7b8740 4 client.20898 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3190346612 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b0223eb0 con 0x5601b02c7e10 2024-03-26T17:53:18.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2d0908c640 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5601b02c7e10 0x5601b02ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2ced7fa640 10 client.20898 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:53:18.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.542+0000 7f2d0908c640 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5601b02c7e10 0x5601b02ca2a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3af50f5471b0b40 server_cookie=2b20e8464e5fa5c5 in_seq=0 out_seq=0 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.543+0000 7f2ced7fa640 1 -- 172.21.15.105:0/3190346612 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5601b0223eb0 con 0x5601b02c7e10 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.543+0000 7f2ced7fa640 10 client.20898 handle_command_reply: tid=0 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.543+0000 7f2d0e7b8740 1 client.20898 shutdown 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3190346612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 msgr2=0x5601b02bbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02bbee0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f2d0009bad0 tx=0x7f2d00098130 comp rx=0 tx=0).stop 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3190346612 shutdown_connections 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5601b02c7e10 0x5601b02ca2a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02bc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/3190346612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02bbee0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3190346612 >> 172.21.15.105:0/3190346612 conn(0x5601b02acfa0 msgr2=0x5601b02ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2ced7fa640 0 client.20898 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3190346612 shutdown_connections 2024-03-26T17:53:18.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/3190346612 wait complete. 2024-03-26T17:53:18.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.544+0000 7f2d0e7b8740 20 client.20898 trim_cache size 0 max 16384 2024-03-26T17:53:18.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 16096 2.53 MB/sec execute 1 sec latency 108.108 ms 2024-03-26T17:53:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.562+0000 7f2d0e7b8740 1 Processor -- start 2024-03-26T17:53:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0e7b8740 1 -- start start 2024-03-26T17:53:18.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02c3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02bafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041af3f0 con 0x5601b02b1a30 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041b0ff0 con 0x5601b02b7590 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041afac0 con 0x5601b02b2420 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0a08e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02bafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0988d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0a08e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02bafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57646/0 (socket says 172.21.15.105:57646) 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0a08e640 1 -- 172.21.15.105:0/1399453614 learned_addr learned my addr 172.21.15.105:0/1399453614 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0908c640 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0a08e640 1 -- 172.21.15.105:0/1399453614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 msgr2=0x5601b02c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0a08e640 1 -- 172.21.15.105:0/1399453614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 msgr2=0x5601b02baa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02baa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0a08e640 1 -- 172.21.15.105:0/1399453614 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b02ba590 con 0x5601b02b7590 2024-03-26T17:53:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.563+0000 7f2d0988d640 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02baa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.564+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02bafa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc032420 tx=0x7f2cfc040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=673b608db9940389 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.564+0000 7f2cef7fe640 1 -- 172.21.15.105:0/1399453614 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc039040 con 0x5601b02b7590 2024-03-26T17:53:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.564+0000 7f2cef7fe640 1 -- 172.21.15.105:0/1399453614 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cfc052070 con 0x5601b02b7590 2024-03-26T17:53:18.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.564+0000 7f2cef7fe640 1 -- 172.21.15.105:0/1399453614 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0406a0 con 0x5601b02b7590 2024-03-26T17:53:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/1399453614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 msgr2=0x5601b02bafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02bafa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc032420 tx=0x7f2cfc040000 comp rx=0 tx=0).stop 2024-03-26T17:53:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/1399453614 shutdown_connections 2024-03-26T17:53:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b02bafa0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/1399453614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02baa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/1399453614 >> 172.21.15.105:0/1399453614 conn(0x5601b02acfa0 msgr2=0x5601b02bbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/1399453614 shutdown_connections 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/1399453614 wait complete. 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 -- start start 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.565+0000 7f2d0e7b8740 1 Processor -- start 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0e7b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b032c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041b0920 con 0x5601b02b1a30 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0988d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041b0190 con 0x5601b02b7590 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0e7b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d041ad3d0 con 0x5601b02b2420 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0988d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02c0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47536/0 (socket says 172.21.15.105:47536) 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0988d640 1 -- 172.21.15.105:0/2989862071 learned_addr learned my addr 172.21.15.105:0/2989862071 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0908c640 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0a08e640 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b032c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0e7b8740 10 client.0 _ll_register_callbacks cb 0x7f2d0000e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0988d640 1 -- 172.21.15.105:0/2989862071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 msgr2=0x5601b02c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0988d640 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0988d640 1 -- 172.21.15.105:0/2989862071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 msgr2=0x5601b032c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0988d640 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b032c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.566+0000 7f2d0988d640 1 -- 172.21.15.105:0/2989862071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b032c700 con 0x5601b02b1a30 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0988d640 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02c0380 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2d0009bd60 tx=0x7f2d00003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a8d5ff3c4ff447 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2ced7fa640 1 -- 172.21.15.105:0/2989862071 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004330 con 0x5601b02b1a30 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2ced7fa640 1 -- 172.21.15.105:0/2989862071 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2d000bab40 con 0x5601b02b1a30 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/2989862071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cfc052070 con 0x5601b02b1a30 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2ced7fa640 1 -- 172.21.15.105:0/2989862071 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2d0009b910 con 0x5601b02b1a30 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 10 client.20904 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/2989862071 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5601b032da30 con 0x5601b02b1a30 2024-03-26T17:53:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2ced7fa640 1 -- 172.21.15.105:0/2989862071 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cfc052070 con 0x5601b02b1a30 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2ced7fa640 1 -- 172.21.15.105:0/2989862071 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2d0009ac90 con 0x5601b02b1a30 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 10 client.20904 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 10 client.20904 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 20 client.20904 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5601b032e1f0 0x5601b03305e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 4 client.20904 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.567+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/2989862071 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601b021e200 con 0x5601b032e1f0 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.568+0000 7f2d0908c640 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5601b032e1f0 0x5601b03305e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.568+0000 7f2ced7fa640 10 client.20904 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.568+0000 7f2d0908c640 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5601b032e1f0 0x5601b03305e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c2c9944b7546490 server_cookie=f64c29a62b580c68 in_seq=0 out_seq=0 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.568+0000 7f2ced7fa640 1 -- 172.21.15.105:0/2989862071 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601b021e200 con 0x5601b032e1f0 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.568+0000 7f2ced7fa640 10 client.20904 handle_command_reply: tid=0 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.568+0000 7f2d0e7b8740 1 client.20904 shutdown 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/2989862071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 msgr2=0x5601b02c0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02c0380 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2d0009bd60 tx=0x7f2d00003ea0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/2989862071 shutdown_connections 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5601b032e1f0 0x5601b03305e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601b02b7590 0x5601b032c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601b02b2420 0x5601b02c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2ced7fa640 0 client.20904 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 --2- 172.21.15.105:0/2989862071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601b02b1a30 0x5601b02c0380 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/2989862071 >> 172.21.15.105:0/2989862071 conn(0x5601b02acfa0 msgr2=0x5601b02b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/2989862071 shutdown_connections 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 1 -- 172.21.15.105:0/2989862071 wait complete. 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.569+0000 7f2d0e7b8740 20 client.20904 trim_cache size 0 max 16384 2024-03-26T17:53:18.564 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 -- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2cf00a2f90 msgr2=0x7f2cf00a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2cf00a2f90 0x7f2cf00a5450 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc006010 tx=0x7f2cfc005d80 comp rx=0 tx=0).stop 2024-03-26T17:53:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 -- 172.21.15.105:0/937559970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d04102340 msgr2=0x7f2d0419e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d04102340 0x7f2d0419e110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000ea60 tx=0x7f2d0000ef20 comp rx=0 tx=0).stop 2024-03-26T17:53:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 -- 172.21.15.105:0/937559970 shutdown_connections 2024-03-26T17:53:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2cf00a2f90 0x7f2cf00a5450 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d041075a0 0x7f2d041a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d04104c70 0x7f2d0419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 --2- 172.21.15.105:0/937559970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d04102340 0x7f2d0419e110 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 -- 172.21.15.105:0/937559970 >> 172.21.15.105:0/937559970 conn(0x7f2d040fbd50 msgr2=0x7f2d04102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 -- 172.21.15.105:0/937559970 shutdown_connections 2024-03-26T17:53:18.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.570+0000 7f2d0baf1640 1 -- 172.21.15.105:0/937559970 wait complete. 2024-03-26T17:53:18.574 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:53:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.723+0000 7efee871b640 1 -- 172.21.15.105:0/2914911990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 msgr2=0x7efee0100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.723+0000 7efee871b640 1 --2- 172.21.15.105:0/2914911990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 0x7efee0100360 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7efedc009860 tx=0x7efedc02bb70 comp rx=0 tx=0).stop 2024-03-26T17:53:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.723+0000 7efee871b640 1 -- 172.21.15.105:0/2914911990 shutdown_connections 2024-03-26T17:53:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.723+0000 7efee871b640 1 --2- 172.21.15.105:0/2914911990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efee0101a50 0x7efee0108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.723+0000 7efee871b640 1 --2- 172.21.15.105:0/2914911990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efee0101090 0x7efee0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.723+0000 7efee871b640 1 --2- 172.21.15.105:0/2914911990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 0x7efee0100360 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.723+0000 7efee871b640 1 -- 172.21.15.105:0/2914911990 >> 172.21.15.105:0/2914911990 conn(0x7efee00fb4b0 msgr2=0x7efee00fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.724+0000 7efee871b640 1 -- 172.21.15.105:0/2914911990 shutdown_connections 2024-03-26T17:53:18.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.724+0000 7efee871b640 1 -- 172.21.15.105:0/2914911990 wait complete. 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.724+0000 7efee871b640 1 Processor -- start 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.724+0000 7efee871b640 1 -- start start 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee871b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 0x7efee010c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee871b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efee0101090 0x7efee010b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee871b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efee0101a50 0x7efee010b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee871b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efee010d410 con 0x7efee00fff40 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee871b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efee010d290 con 0x7efee0101a50 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee871b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efee010d590 con 0x7efee0101090 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efee0101090 0x7efee010b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efee0101090 0x7efee010b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33934/0 (socket says 172.21.15.105:33934) 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 -- 172.21.15.105:0/2233459538 learned_addr learned my addr 172.21.15.105:0/2233459538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee64b7640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 0x7efee010c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee6cb8640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efee0101a50 0x7efee010b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 -- 172.21.15.105:0/2233459538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efee0101a50 msgr2=0x7efee010b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efee0101a50 0x7efee010b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 -- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 msgr2=0x7efee010c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 0x7efee010c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 -- 172.21.15.105:0/2233459538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee010be00 con 0x7efee0101090 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee64b7640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 0x7efee010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.725+0000 7efee5cb6640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efee0101090 0x7efee010b030 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efed000e9c0 tx=0x7efed000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.726+0000 7efed77fe640 1 -- 172.21.15.105:0/2233459538 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed000cd80 con 0x7efee0101090 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.726+0000 7efed77fe640 1 -- 172.21.15.105:0/2233459538 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed00035f0 con 0x7efee0101090 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.726+0000 7efed77fe640 1 -- 172.21.15.105:0/2233459538 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0010690 con 0x7efee0101090 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.726+0000 7efee871b640 1 -- 172.21.15.105:0/2233459538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee01a1c70 con 0x7efee0101090 2024-03-26T17:53:18.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.726+0000 7efee871b640 1 -- 172.21.15.105:0/2233459538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a2170 con 0x7efee0101090 2024-03-26T17:53:18.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efed77fe640 1 -- 172.21.15.105:0/2233459538 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7efed00037a0 con 0x7efee0101090 2024-03-26T17:53:18.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efeeb3e5740 1 Processor -- start 2024-03-26T17:53:18.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efeeb3e5740 1 -- start start 2024-03-26T17:53:18.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5682e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5683420 0x562ff5687f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff568aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efeeb3e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562ff55e7970 con 0x562ff5688590 2024-03-26T17:53:18.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efeeb3e5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ff558fd70 con 0x562ff5682a30 2024-03-26T17:53:18.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.729+0000 7efeeb3e5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ff55e3920 con 0x562ff5683420 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee64b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5683420 0x562ff5687f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee6cb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5682e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee5cb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff568aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee6cb8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5682e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57682/0 (socket says 172.21.15.105:57682) 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee64b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5683420 0x562ff5687f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33942/0 (socket says 172.21.15.105:33942) 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee5cb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff568aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47576/0 (socket says 172.21.15.105:47576) 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee6cb8640 1 -- 172.21.15.105:0/2482273131 learned_addr learned my addr 172.21.15.105:0/2482273131 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee5cb6640 1 -- 172.21.15.105:0/2482273131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5683420 msgr2=0x562ff5687f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee5cb6640 1 --2- 172.21.15.105:0/2482273131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5683420 0x562ff5687f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee5cb6640 1 -- 172.21.15.105:0/2482273131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 msgr2=0x562ff5682e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee5cb6640 1 --2- 172.21.15.105:0/2482273131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5682e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee5cb6640 1 -- 172.21.15.105:0/2482273131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff52fb3f0 con 0x562ff5688590 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efed77fe640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efecc0a2f20 0x7efecc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee6cb8640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efecc0a2f20 0x7efecc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.730+0000 7efee5cb6640 1 --2- 172.21.15.105:0/2482273131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff568aa20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7efed00cd400 tx=0x7efed00d5f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7a635d7c3b490b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.731+0000 7efed77fe640 1 -- 172.21.15.105:0/2233459538 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7efed00cf970 con 0x7efee0101090 2024-03-26T17:53:18.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.731+0000 7efed4ff9640 1 -- 172.21.15.105:0/2482273131 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0098300 con 0x562ff5688590 2024-03-26T17:53:18.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.731+0000 7efed4ff9640 1 -- 172.21.15.105:0/2482273131 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed0003b10 con 0x562ff5688590 2024-03-26T17:53:18.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.731+0000 7efed4ff9640 1 -- 172.21.15.105:0/2482273131 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed009b9c0 con 0x562ff5688590 2024-03-26T17:53:18.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efeeb3e5740 1 -- 172.21.15.105:0/2482273131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 msgr2=0x562ff568aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efee6cb8640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efecc0a2f20 0x7efecc0a53e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x562ff5683c80 tx=0x7efed8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/2482273131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff568aa20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7efed00cd400 tx=0x7efed00d5f10 comp rx=0 tx=0).stop 2024-03-26T17:53:18.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efeeb3e5740 1 -- 172.21.15.105:0/2482273131 shutdown_connections 2024-03-26T17:53:18.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/2482273131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff568aa20 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/2482273131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5683420 0x562ff5687f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/2482273131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5682e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efeeb3e5740 1 -- 172.21.15.105:0/2482273131 >> 172.21.15.105:0/2482273131 conn(0x562ff567dfa0 msgr2=0x562ff567e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.732+0000 7efeeb3e5740 1 -- 172.21.15.105:0/2482273131 shutdown_connections 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.733+0000 7efeeb3e5740 1 -- 172.21.15.105:0/2482273131 wait complete. 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.733+0000 7efeeb3e5740 1 -- start start 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.733+0000 7efeeb3e5740 1 Processor -- start 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff5695750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff5695c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efee64b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff5695750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efee64b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff5695750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33958/0 (socket says 172.21.15.105:33958) 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efee64b7640 1 -- 172.21.15.105:0/1317847113 learned_addr learned my addr 172.21.15.105:0/1317847113 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff569ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562ff568b540 con 0x562ff5688590 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562ff55a3310 con 0x562ff5683420 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efee5cb6640 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff5695c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ff568b6c0 con 0x562ff5682a30 2024-03-26T17:53:18.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.734+0000 7efee6cb8640 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff569ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efeeb3e5740 10 client.0 _ll_register_callbacks cb 0x562ff55dec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:18.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efee64b7640 1 -- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 msgr2=0x562ff5695c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efee64b7640 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff5695c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efee64b7640 1 -- 172.21.15.105:0/1317847113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 msgr2=0x562ff569ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efee64b7640 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff569ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efee64b7640 1 -- 172.21.15.105:0/1317847113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff53c57d0 con 0x562ff5682a30 2024-03-26T17:53:18.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efee5cb6640 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff5695c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efee64b7640 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff5695750 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efedc02c0b0 tx=0x7efedc005af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7e30623b3fd495 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efec9ffb640 1 -- 172.21.15.105:0/1317847113 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc043070 con 0x562ff5682a30 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efec9ffb640 1 -- 172.21.15.105:0/1317847113 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7efedc02c510 con 0x562ff5682a30 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.735+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ff55f4eb0 con 0x562ff5682a30 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efec9ffb640 1 -- 172.21.15.105:0/1317847113 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efedc04f050 con 0x562ff5682a30 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efeeb3e5740 10 client.30679 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562ff56a00e0 con 0x562ff5682a30 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efec9ffb640 1 -- 172.21.15.105:0/1317847113 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efedc0035f0 con 0x562ff5682a30 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efec9ffb640 1 -- 172.21.15.105:0/1317847113 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efedc03f450 con 0x562ff5682a30 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efeeb3e5740 10 client.30679 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efeeb3e5740 10 client.30679 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efeeb3e5740 20 client.30679 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562ff56a1700 0x562ff56a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efeeb3e5740 4 client.30679 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.736+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ff55f4eb0 con 0x562ff56a1700 2024-03-26T17:53:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.737+0000 7efee5cb6640 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562ff56a1700 0x562ff56a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.738+0000 7efee5cb6640 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562ff56a1700 0x562ff56a3b70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a05d8b7a396c70f3 server_cookie=de628eb2b7864d58 in_seq=0 out_seq=0 2024-03-26T17:53:18.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.738+0000 7efec9ffb640 10 client.30679 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efec9ffb640 1 -- 172.21.15.105:0/1317847113 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ff55f4eb0 con 0x562ff56a1700 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efec9ffb640 10 client.30679 handle_command_reply: tid=0 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 client.30679 shutdown 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 msgr2=0x562ff5695750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff5695750 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efedc02c0b0 tx=0x7efedc005af0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 shutdown_connections 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff569ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562ff56a1700 0x562ff56a3b70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff5695c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1317847113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff5695750 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efec9ffb640 0 client.30679 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:53:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.739+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 >> 172.21.15.105:0/1317847113 conn(0x562ff567dfa0 msgr2=0x562ff567d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.740+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 shutdown_connections 2024-03-26T17:53:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.740+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1317847113 wait complete. 2024-03-26T17:53:18.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.740+0000 7efeeb3e5740 20 client.30679 trim_cache size 0 max 16384 2024-03-26T17:53:18.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.764+0000 7efeeb3e5740 1 Processor -- start 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.764+0000 7efeeb3e5740 1 -- start start 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.764+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff567d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.764+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff569cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.764+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff5694d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.764+0000 7efee64b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff567d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.764+0000 7efee64b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff567d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:33964/0 (socket says 172.21.15.105:33964) 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee64b7640 1 -- 172.21.15.105:0/3434449500 learned_addr learned my addr 172.21.15.105:0/3434449500 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee6cb8640 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff5694d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efeeb3e5740 1 -- 172.21.15.105:0/3434449500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a6bf0 con 0x562ff5688590 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efeeb3e5740 1 -- 172.21.15.105:0/3434449500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a87f0 con 0x562ff5683420 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efeeb3e5740 1 -- 172.21.15.105:0/3434449500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a72c0 con 0x562ff5682a30 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee5cb6640 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff569cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee6cb8640 1 -- 172.21.15.105:0/3434449500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 msgr2=0x562ff567d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee6cb8640 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff567d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee6cb8640 1 -- 172.21.15.105:0/3434449500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 msgr2=0x562ff569cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee6cb8640 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff569cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee6cb8640 1 -- 172.21.15.105:0/3434449500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ff5683800 con 0x562ff5688590 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee64b7640 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff567d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.765+0000 7efee6cb8640 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff5694d70 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efed8009d60 tx=0x7efed8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149418edbb921a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.766+0000 7efecbfff640 1 -- 172.21.15.105:0/3434449500 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed801ab60 con 0x562ff5688590 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.766+0000 7efecbfff640 1 -- 172.21.15.105:0/3434449500 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed801ad00 con 0x562ff5688590 2024-03-26T17:53:18.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.766+0000 7efecbfff640 1 -- 172.21.15.105:0/3434449500 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed80056e0 con 0x562ff5688590 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 -- 172.21.15.105:0/3434449500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 msgr2=0x562ff5694d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff5694d70 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efed8009d60 tx=0x7efed8015000 comp rx=0 tx=0).stop 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 -- 172.21.15.105:0/3434449500 shutdown_connections 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5688590 0x562ff5694d70 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5683420 0x562ff569cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/3434449500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5682a30 0x562ff567d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 -- 172.21.15.105:0/3434449500 >> 172.21.15.105:0/3434449500 conn(0x562ff567dfa0 msgr2=0x562ff567c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 -- 172.21.15.105:0/3434449500 shutdown_connections 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 -- 172.21.15.105:0/3434449500 wait complete. 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 -- start start 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.767+0000 7efeeb3e5740 1 Processor -- start 2024-03-26T17:53:18.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5699ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5683420 0x562ff569a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efeeb3e5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5688590 0x562ff569b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efeeb3e5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a8120 con 0x562ff5683420 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efeeb3e5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a7990 con 0x562ff5682a30 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efeeb3e5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a4bd0 con 0x562ff5688590 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee64b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5699ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee64b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5699ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57712/0 (socket says 172.21.15.105:57712) 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee64b7640 1 -- 172.21.15.105:0/1547978198 learned_addr learned my addr 172.21.15.105:0/1547978198 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efeeb3e5740 10 client.0 _ll_register_callbacks cb 0x7efed0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee5cb6640 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5683420 0x562ff569a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee6cb8640 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5688590 0x562ff569b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee64b7640 1 -- 172.21.15.105:0/1547978198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5688590 msgr2=0x562ff569b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee64b7640 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5688590 0x562ff569b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee64b7640 1 -- 172.21.15.105:0/1547978198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5683420 msgr2=0x562ff569a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee64b7640 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5683420 0x562ff569a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.768+0000 7efee64b7640 1 -- 172.21.15.105:0/1547978198 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ff569bae0 con 0x562ff5682a30 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.769+0000 7efee64b7640 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5699ff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efedc03f9c0 tx=0x7efedc0045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5941054a12aacf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.769+0000 7efec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.769+0000 7efec9ffb640 1 -- 172.21.15.105:0/1547978198 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc057070 con 0x562ff5682a30 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.769+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1547978198 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed801ad00 con 0x562ff5682a30 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.769+0000 7efec9ffb640 1 -- 172.21.15.105:0/1547978198 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7efedc02c510 con 0x562ff5682a30 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.769+0000 7efec9ffb640 1 -- 172.21.15.105:0/1547978198 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efedc061050 con 0x562ff5682a30 2024-03-26T17:53:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.769+0000 7efeeb3e5740 10 client.30707 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.769+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1547978198 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562ff569ce30 con 0x562ff5682a30 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efec9ffb640 1 -- 172.21.15.105:0/1547978198 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed801ad00 con 0x562ff5682a30 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efec9ffb640 1 -- 172.21.15.105:0/1547978198 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efedc003860 con 0x562ff5682a30 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efeeb3e5740 10 client.30707 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efeeb3e5740 10 client.30707 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efeeb3e5740 20 client.30707 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562ff5708020 0x562ff570a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efeeb3e5740 4 client.30707 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1547978198 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ff55ef200 con 0x562ff5708020 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efee5cb6640 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562ff5708020 0x562ff570a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efec9ffb640 10 client.30707 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.770+0000 7efee5cb6640 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562ff5708020 0x562ff570a440 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24ff37a04b3d9d7f server_cookie=a6ffa23395e7f5e0 in_seq=0 out_seq=0 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efec9ffb640 1 -- 172.21.15.105:0/1547978198 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ff55ef200 con 0x562ff5708020 2024-03-26T17:53:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efec9ffb640 10 client.30707 handle_command_reply: tid=0 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 client.30707 shutdown 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1547978198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 msgr2=0x562ff5699ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5699ff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efedc03f9c0 tx=0x7efedc0045c0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1547978198 shutdown_connections 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ff5688590 0x562ff569b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ff5683420 0x562ff569a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x562ff5708020 0x562ff570a440 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 --2- 172.21.15.105:0/1547978198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562ff5682a30 0x562ff5699ff0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efec9ffb640 0 client.30707 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1547978198 >> 172.21.15.105:0/1547978198 conn(0x562ff567dfa0 msgr2=0x562ff5681a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1547978198 shutdown_connections 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 1 -- 172.21.15.105:0/1547978198 wait complete. 2024-03-26T17:53:18.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.771+0000 7efeeb3e5740 20 client.30707 trim_cache size 0 max 16384 2024-03-26T17:53:18.768 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:18.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 -- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efecc0a2f20 msgr2=0x7efecc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efecc0a2f20 0x7efecc0a53e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x562ff5683c80 tx=0x7efed8010040 comp rx=0 tx=0).stop 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 -- 172.21.15.105:0/2233459538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efee0101090 msgr2=0x7efee010b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efee0101090 0x7efee010b030 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efed000e9c0 tx=0x7efed000ee80 comp rx=0 tx=0).stop 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 -- 172.21.15.105:0/2233459538 shutdown_connections 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efecc0a2f20 0x7efecc0a53e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efee0101a50 0x7efee010b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efee0101090 0x7efee010b030 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 --2- 172.21.15.105:0/2233459538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efee00fff40 0x7efee010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 -- 172.21.15.105:0/2233459538 >> 172.21.15.105:0/2233459538 conn(0x7efee00fb4b0 msgr2=0x7efee01045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 -- 172.21.15.105:0/2233459538 shutdown_connections 2024-03-26T17:53:18.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.772+0000 7efee871b640 1 -- 172.21.15.105:0/2233459538 wait complete. 2024-03-26T17:53:18.777 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 -- 172.21.15.105:0/3820841488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe650104cd0 msgr2=0x7fe6501070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 --2- 172.21.15.105:0/3820841488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe650104cd0 0x7fe6501070c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fe644009900 tx=0x7fe64402bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 -- 172.21.15.105:0/3820841488 shutdown_connections 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 --2- 172.21.15.105:0/3820841488 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe650107600 0x7fe6501099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 --2- 172.21.15.105:0/3820841488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe650104cd0 0x7fe6501070c0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 --2- 172.21.15.105:0/3820841488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6501023a0 0x7fe650104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 -- 172.21.15.105:0/3820841488 >> 172.21.15.105:0/3820841488 conn(0x7fe6500fbd50 msgr2=0x7fe6500fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 -- 172.21.15.105:0/3820841488 shutdown_connections 2024-03-26T17:53:18.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.933+0000 7fe656659640 1 -- 172.21.15.105:0/3820841488 wait complete. 2024-03-26T17:53:18.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.934+0000 7fe656659640 1 Processor -- start 2024-03-26T17:53:18.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.934+0000 7fe656659640 1 -- start start 2024-03-26T17:53:18.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.934+0000 7fe656659640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6501023a0 0x7fe650101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.934+0000 7fe656659640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe650104cd0 0x7fe650100350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.934+0000 7fe656659640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe650107600 0x7fe650100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.934+0000 7fe656659640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe65010d3e0 con 0x7fe650107600 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.934+0000 7fe656659640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe65010d260 con 0x7fe6501023a0 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.934+0000 7fe656659640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe65010d560 con 0x7fe650104cd0 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6501023a0 0x7fe650101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe650104cd0 0x7fe650100350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6501023a0 0x7fe650101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57724/0 (socket says 172.21.15.105:57724) 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64ffff640 1 -- 172.21.15.105:0/2491195611 learned_addr learned my addr 172.21.15.105:0/2491195611 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe654bf6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe650107600 0x7fe650100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64f7fe640 1 -- 172.21.15.105:0/2491195611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6501023a0 msgr2=0x7fe650101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6501023a0 0x7fe650101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64f7fe640 1 -- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe650107600 msgr2=0x7fe650100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe650107600 0x7fe650100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64f7fe640 1 -- 172.21.15.105:0/2491195611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe650101120 con 0x7fe650104cd0 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe650104cd0 0x7fe650100350 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe644036040 tx=0x7fe644005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.935+0000 7fe64d7fa640 1 -- 172.21.15.105:0/2491195611 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644043020 con 0x7fe650104cd0 2024-03-26T17:53:18.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.936+0000 7fe64d7fa640 1 -- 172.21.15.105:0/2491195611 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe644004350 con 0x7fe650104cd0 2024-03-26T17:53:18.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.936+0000 7fe656659640 1 -- 172.21.15.105:0/2491195611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6501a1ca0 con 0x7fe650104cd0 2024-03-26T17:53:18.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.936+0000 7fe64d7fa640 1 -- 172.21.15.105:0/2491195611 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644003ab0 con 0x7fe650104cd0 2024-03-26T17:53:18.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.936+0000 7fe656659640 1 -- 172.21.15.105:0/2491195611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6501a21c0 con 0x7fe650104cd0 2024-03-26T17:53:18.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.938+0000 7fe64d7fa640 1 -- 172.21.15.105:0/2491195611 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe644046020 con 0x7fe650104cd0 2024-03-26T17:53:18.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.939+0000 7fe659243740 1 Processor -- start 2024-03-26T17:53:18.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.939+0000 7fe64d7fa640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe6380a3020 0x7fe6380a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64d7fa640 1 -- 172.21.15.105:0/2491195611 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe6440f1530 con 0x7fe650104cd0 2024-03-26T17:53:18.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe659243740 1 -- start start 2024-03-26T17:53:18.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe654bf6640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe6380a3020 0x7fe6380a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe659243740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349240 0x556963349660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe659243740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349c30 0x55696334e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe659243740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55696334eda0 0x556963351230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe659243740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5569632a5970 con 0x556963349c30 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe659243740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55696324dd70 con 0x55696334eda0 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe659243740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5569632a1920 con 0x556963349240 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349c30 0x55696334e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349c30 0x55696334e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47628/0 (socket says 172.21.15.105:47628) 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64f7fe640 1 -- 172.21.15.105:0/1483526350 learned_addr learned my addr 172.21.15.105:0/1483526350 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe654bf6640 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55696334eda0 0x556963351230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64ffff640 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349240 0x556963349660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64f7fe640 1 -- 172.21.15.105:0/1483526350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349240 msgr2=0x556963349660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349240 0x556963349660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64f7fe640 1 -- 172.21.15.105:0/1483526350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55696334eda0 msgr2=0x556963351230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55696334eda0 0x556963351230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe654bf6640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe6380a3020 0x7fe6380a54e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe648005fd0 tx=0x7fe648013b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.940+0000 7fe64f7fe640 1 -- 172.21.15.105:0/1483526350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556962fb93f0 con 0x556963349c30 2024-03-26T17:53:18.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.941+0000 7fe64ffff640 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349240 0x556963349660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:53:18.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.941+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349c30 0x55696334e730 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe6440ba590 tx=0x7fe6440bd5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f06012f6a90e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.941+0000 7fe6367fc640 1 -- 172.21.15.105:0/1483526350 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440c1d10 con 0x556963349c30 2024-03-26T17:53:18.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.941+0000 7fe6367fc640 1 -- 172.21.15.105:0/1483526350 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6440c65d0 con 0x556963349c30 2024-03-26T17:53:18.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.941+0000 7fe6367fc640 1 -- 172.21.15.105:0/1483526350 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440f12f0 con 0x556963349c30 2024-03-26T17:53:18.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 -- 172.21.15.105:0/1483526350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349c30 msgr2=0x55696334e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349c30 0x55696334e730 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe6440ba590 tx=0x7fe6440bd5e0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 -- 172.21.15.105:0/1483526350 shutdown_connections 2024-03-26T17:53:18.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55696334eda0 0x556963351230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349c30 0x55696334e730 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 --2- 172.21.15.105:0/1483526350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349240 0x556963349660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 -- 172.21.15.105:0/1483526350 >> 172.21.15.105:0/1483526350 conn(0x5569633447b0 msgr2=0x556963344bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 -- 172.21.15.105:0/1483526350 shutdown_connections 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 -- 172.21.15.105:0/1483526350 wait complete. 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.942+0000 7fe659243740 1 -- start start 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe659243740 1 Processor -- start 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe659243740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963353350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe659243740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x556963353890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe659243740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x55696335a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe659243740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5569632a5970 con 0x556963349240 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe659243740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556963261310 con 0x556963349c30 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe659243740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556963351b10 con 0x55696334eda0 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963353350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe659243740 10 client.0 _ll_register_callbacks cb 0x55696329cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963353350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47640/0 (socket says 172.21.15.105:47640) 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe64ffff640 1 -- 172.21.15.105:0/1977760095 learned_addr learned my addr 172.21.15.105:0/1977760095 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x556963353890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.943+0000 7fe654bf6640 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x55696335a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe64f7fe640 1 -- 172.21.15.105:0/1977760095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 msgr2=0x55696335a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x55696335a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe64f7fe640 1 -- 172.21.15.105:0/1977760095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 msgr2=0x556963353350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963353350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe64f7fe640 1 -- 172.21.15.105:0/1977760095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569630837d0 con 0x556963349c30 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe64ffff640 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963353350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x556963353890 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55696334a030 tx=0x7fe644003f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20572189ef837f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe62f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe62f7fe640 1 -- 172.21.15.105:0/1977760095 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440f1300 con 0x556963349c30 2024-03-26T17:53:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe62f7fe640 1 -- 172.21.15.105:0/1977760095 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe644005420 con 0x556963349c30 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe659243740 1 -- 172.21.15.105:0/1977760095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6440c65d0 con 0x556963349c30 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe62f7fe640 1 -- 172.21.15.105:0/1977760095 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe644005620 con 0x556963349c30 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe659243740 10 client.30719 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.944+0000 7fe659243740 1 -- 172.21.15.105:0/1977760095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55696335dbb0 con 0x556963349c30 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe62f7fe640 1 -- 172.21.15.105:0/1977760095 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6440c65d0 con 0x556963349c30 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe62f7fe640 1 -- 172.21.15.105:0/1977760095 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe64409eb00 con 0x556963349c30 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe659243740 10 client.30719 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe659243740 10 client.30719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe659243740 20 client.30719 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe659243740 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55696335f1d0 0x5569633615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe659243740 4 client.30719 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe659243740 1 -- 172.21.15.105:0/1977760095 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569632b2eb0 con 0x55696335f1d0 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.945+0000 7fe64ffff640 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55696335f1d0 0x5569633615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.946+0000 7fe62f7fe640 10 client.30719 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:53:18.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.946+0000 7fe64ffff640 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55696335f1d0 0x5569633615c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1ef0100dc9ce30f server_cookie=45a734f8327185bb in_seq=0 out_seq=0 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe62f7fe640 1 -- 172.21.15.105:0/1977760095 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5569632b2eb0 con 0x55696335f1d0 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe62f7fe640 10 client.30719 handle_command_reply: tid=0 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 client.30719 shutdown 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 -- 172.21.15.105:0/1977760095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 msgr2=0x556963353890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x556963353890 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55696334a030 tx=0x7fe644003f00 comp rx=0 tx=0).stop 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 -- 172.21.15.105:0/1977760095 shutdown_connections 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55696335f1d0 0x5569633615c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x55696335a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x556963353890 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 --2- 172.21.15.105:0/1977760095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963353350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 -- 172.21.15.105:0/1977760095 >> 172.21.15.105:0/1977760095 conn(0x5569633447b0 msgr2=0x55696333b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe62f7fe640 0 client.30719 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:53:18.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 -- 172.21.15.105:0/1977760095 shutdown_connections 2024-03-26T17:53:18.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 1 -- 172.21.15.105:0/1977760095 wait complete. 2024-03-26T17:53:18.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.947+0000 7fe659243740 20 client.30719 trim_cache size 0 max 16384 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe659243740 1 Processor -- start 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe659243740 1 -- start start 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe659243740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963348120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe659243740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x55696335a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe659243740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x556963351e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe659243740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501a6b70 con 0x556963349240 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe659243740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501a8770 con 0x556963349c30 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe659243740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501a7240 con 0x55696334eda0 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963348120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963348120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47642/0 (socket says 172.21.15.105:47642) 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe64ffff640 1 -- 172.21.15.105:0/586955799 learned_addr learned my addr 172.21.15.105:0/586955799 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe654bf6640 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x556963351e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.971+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x55696335a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe64ffff640 1 -- 172.21.15.105:0/586955799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 msgr2=0x556963351e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe64ffff640 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x556963351e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe64ffff640 1 -- 172.21.15.105:0/586955799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 msgr2=0x55696335a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe64ffff640 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x55696335a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe64ffff640 1 -- 172.21.15.105:0/586955799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556963337820 con 0x556963349240 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe654bf6640 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x556963351e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe64ffff640 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963348120 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe640009640 tx=0x7fe640013410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942f262ef62d7a21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe635ffb640 1 -- 172.21.15.105:0/586955799 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640024070 con 0x556963349240 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe635ffb640 1 -- 172.21.15.105:0/586955799 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe640014e30 con 0x556963349240 2024-03-26T17:53:18.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.972+0000 7fe635ffb640 1 -- 172.21.15.105:0/586955799 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640004a00 con 0x556963349240 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 -- 172.21.15.105:0/586955799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 msgr2=0x556963348120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963348120 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe640009640 tx=0x7fe640013410 comp rx=0 tx=0).stop 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 -- 172.21.15.105:0/586955799 shutdown_connections 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696334eda0 0x556963351e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349c30 0x55696335a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 --2- 172.21.15.105:0/586955799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556963349240 0x556963348120 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 -- 172.21.15.105:0/586955799 >> 172.21.15.105:0/586955799 conn(0x5569633447b0 msgr2=0x556963347a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 -- 172.21.15.105:0/586955799 shutdown_connections 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.973+0000 7fe659243740 1 -- 172.21.15.105:0/586955799 wait complete. 2024-03-26T17:53:18.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.974+0000 7fe659243740 1 -- start start 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.974+0000 7fe659243740 1 Processor -- start 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.974+0000 7fe659243740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349240 0x55696335a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.974+0000 7fe659243740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349c30 0x556963357160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.974+0000 7fe659243740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55696334eda0 0x5569633576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.974+0000 7fe659243740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501a80a0 con 0x55696334eda0 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.974+0000 7fe659243740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501a7910 con 0x556963349240 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe659243740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501a4b70 con 0x556963349c30 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe659243740 10 client.0 _ll_register_callbacks cb 0x7fe648009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349240 0x55696335a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349240 0x55696335a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57760/0 (socket says 172.21.15.105:57760) 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe64ffff640 1 -- 172.21.15.105:0/3950535732 learned_addr learned my addr 172.21.15.105:0/3950535732 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe654bf6640 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55696334eda0 0x5569633576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe64f7fe640 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349c30 0x556963357160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe654bf6640 1 -- 172.21.15.105:0/3950535732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349c30 msgr2=0x556963357160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe654bf6640 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349c30 0x556963357160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe654bf6640 1 -- 172.21.15.105:0/3950535732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349240 msgr2=0x55696335a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe654bf6640 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349240 0x55696335a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe654bf6640 1 -- 172.21.15.105:0/3950535732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569633c34c0 con 0x55696334eda0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe654bf6640 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55696334eda0 0x5569633576a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe648013570 tx=0x7fe64801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0a63aa5604d411 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.975+0000 7fe62f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe62f7fe640 1 -- 172.21.15.105:0/3950535732 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648004b20 con 0x55696334eda0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe62f7fe640 1 -- 172.21.15.105:0/3950535732 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe648004cc0 con 0x55696334eda0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 1 -- 172.21.15.105:0/3950535732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe640014e30 con 0x55696334eda0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe62f7fe640 1 -- 172.21.15.105:0/3950535732 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe648003970 con 0x55696334eda0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 10 client.20940 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 1 -- 172.21.15.105:0/3950535732 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5569633c46d0 con 0x55696334eda0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe62f7fe640 1 -- 172.21.15.105:0/3950535732 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe640014e30 con 0x55696334eda0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe62f7fe640 1 -- 172.21.15.105:0/3950535732 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe64801f070 con 0x55696334eda0 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 10 client.20940 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:18.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 10 client.20940 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 20 client.20940 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5569633c5100 0x5569633c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 4 client.20940 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.976+0000 7fe659243740 1 -- 172.21.15.105:0/3950535732 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569632ad200 con 0x5569633c5100 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.977+0000 7fe64ffff640 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5569633c5100 0x5569633c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.977+0000 7fe62f7fe640 10 client.20940 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.977+0000 7fe64ffff640 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5569633c5100 0x5569633c7520 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5e5022bc451ec7d server_cookie=f5ab3abb2290792f in_seq=0 out_seq=0 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe62f7fe640 1 -- 172.21.15.105:0/3950535732 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569632ad200 con 0x5569633c5100 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe62f7fe640 10 client.20940 handle_command_reply: tid=0 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 client.20940 shutdown 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 -- 172.21.15.105:0/3950535732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55696334eda0 msgr2=0x5569633576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55696334eda0 0x5569633576a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe648013570 tx=0x7fe64801b000 comp rx=0 tx=0).stop 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 -- 172.21.15.105:0/3950535732 shutdown_connections 2024-03-26T17:53:18.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5569633c5100 0x5569633c7520 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55696334eda0 0x5569633576a0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556963349c30 0x556963357160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 --2- 172.21.15.105:0/3950535732 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556963349240 0x55696335a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 -- 172.21.15.105:0/3950535732 >> 172.21.15.105:0/3950535732 conn(0x5569633447b0 msgr2=0x5569633482a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe62f7fe640 0 client.20940 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 -- 172.21.15.105:0/3950535732 shutdown_connections 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 1 -- 172.21.15.105:0/3950535732 wait complete. 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.978+0000 7fe659243740 20 client.20940 trim_cache size 0 max 16384 2024-03-26T17:53:18.973 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:18.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 -- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe6380a3020 msgr2=0x7fe6380a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe6380a3020 0x7fe6380a54e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe648005fd0 tx=0x7fe648013b60 comp rx=0 tx=0).stop 2024-03-26T17:53:18.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 -- 172.21.15.105:0/2491195611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe650104cd0 msgr2=0x7fe650100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:18.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe650104cd0 0x7fe650100350 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe644036040 tx=0x7fe644005dc0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 -- 172.21.15.105:0/2491195611 shutdown_connections 2024-03-26T17:53:18.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe6380a3020 0x7fe6380a54e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe650107600 0x7fe650100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe650104cd0 0x7fe650100350 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 --2- 172.21.15.105:0/2491195611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe6501023a0 0x7fe650101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:18.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 -- 172.21.15.105:0/2491195611 >> 172.21.15.105:0/2491195611 conn(0x7fe6500fbd50 msgr2=0x7fe650104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:18.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 -- 172.21.15.105:0/2491195611 shutdown_connections 2024-03-26T17:53:18.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:18.979+0000 7fe656659640 1 -- 172.21.15.105:0/2491195611 wait complete. 2024-03-26T17:53:18.984 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:53:19.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 -- 172.21.15.105:0/731954729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc102ac0 msgr2=0x7f2bcc102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 --2- 172.21.15.105:0/731954729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc102f40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009900 tx=0x7f2bc002bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:19.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 -- 172.21.15.105:0/731954729 shutdown_connections 2024-03-26T17:53:19.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 --2- 172.21.15.105:0/731954729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bcc103480 0x7f2bcc109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 --2- 172.21.15.105:0/731954729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc102f40 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 --2- 172.21.15.105:0/731954729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bcc1018d0 0x7f2bcc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 -- 172.21.15.105:0/731954729 >> 172.21.15.105:0/731954729 conn(0x7f2bcc0fce60 msgr2=0x7f2bcc0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:19.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 -- 172.21.15.105:0/731954729 shutdown_connections 2024-03-26T17:53:19.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.134+0000 7f2bd1075640 1 -- 172.21.15.105:0/731954729 wait complete. 2024-03-26T17:53:19.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.135+0000 7f2bd1075640 1 Processor -- start 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.135+0000 7f2bd1075640 1 -- start start 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.135+0000 7f2bd1075640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bcc1018d0 0x7f2bcc199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.135+0000 7f2bd1075640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc19a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.135+0000 7f2bd1075640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc103480 0x7f2bcc1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.135+0000 7f2bd1075640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc074100 con 0x7f2bcc103480 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.135+0000 7f2bd1075640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc073f80 con 0x7f2bcc102ac0 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.135+0000 7f2bd1075640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc074280 con 0x7f2bcc1018d0 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc103480 0x7f2bcc1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bcc1018d0 0x7f2bcc199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc103480 0x7f2bcc1a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47666/0 (socket says 172.21.15.105:47666) 2024-03-26T17:53:19.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bcc1018d0 0x7f2bcc199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34058/0 (socket says 172.21.15.105:34058) 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 -- 172.21.15.105:0/2801588493 learned_addr learned my addr 172.21.15.105:0/2801588493 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bca59c640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc19a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 -- 172.21.15.105:0/2801588493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bcc1018d0 msgr2=0x7f2bcc199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bcc1018d0 0x7f2bcc199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 -- 172.21.15.105:0/2801588493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bcc102ac0 msgr2=0x7f2bcc19a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc19a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 -- 172.21.15.105:0/2801588493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc1a1990 con 0x7f2bcc103480 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bcb59e640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc103480 0x7f2bcc1a1320 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400e9b0 tx=0x7f2bc400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bb3fff640 1 -- 172.21.15.105:0/2801588493 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc400cd80 con 0x7f2bcc103480 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bb3fff640 1 -- 172.21.15.105:0/2801588493 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bc40035f0 con 0x7f2bcc103480 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.136+0000 7f2bd1075640 1 -- 172.21.15.105:0/2801588493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc1a1c80 con 0x7f2bcc103480 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.137+0000 7f2bb3fff640 1 -- 172.21.15.105:0/2801588493 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4010690 con 0x7f2bcc103480 2024-03-26T17:53:19.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.137+0000 7f2bd1075640 1 -- 172.21.15.105:0/2801588493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc1a21a0 con 0x7f2bcc103480 2024-03-26T17:53:19.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bd3c59740 1 Processor -- start 2024-03-26T17:53:19.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bb3fff640 1 -- 172.21.15.105:0/2801588493 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2bc4010830 con 0x7f2bcc103480 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bd3c59740 1 -- start start 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8d9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bd3c59740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559c0f836970 con 0x559c0f8d1a30 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bd3c59740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559c0f7ded70 con 0x559c0f8d7590 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bd3c59740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559c0f832920 con 0x559c0f8d2420 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8d6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34066/0 (socket says 172.21.15.105:34066) 2024-03-26T17:53:19.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bcad9d640 1 -- 172.21.15.105:0/3956033582 learned_addr learned my addr 172.21.15.105:0/3956033582 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:19.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.140+0000 7f2bca59c640 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bca59c640 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8d9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bcad9d640 1 -- 172.21.15.105:0/3956033582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 msgr2=0x559c0f8d9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:19.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8d9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bcad9d640 1 -- 172.21.15.105:0/3956033582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 msgr2=0x559c0f8d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:19.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bcad9d640 1 -- 172.21.15.105:0/3956033582 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0f54a3f0 con 0x559c0f8d2420 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bca59c640 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8d9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8d6f20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc011470 tx=0x7f2bbc011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=528a88a03ff6fb29 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bb17fa640 1 -- 172.21.15.105:0/3956033582 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc003970 con 0x559c0f8d2420 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bb17fa640 1 -- 172.21.15.105:0/3956033582 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bbc004020 con 0x559c0f8d2420 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.141+0000 7f2bb17fa640 1 -- 172.21.15.105:0/3956033582 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00f8f0 con 0x559c0f8d2420 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bb3fff640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2bb40a2fa0 0x7f2bb40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bca59c640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2bb40a2fa0 0x7f2bb40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bb3fff640 1 -- 172.21.15.105:0/2801588493 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2bc40cf6c0 con 0x7f2bcc103480 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3956033582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 msgr2=0x559c0f8d6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8d6f20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc011470 tx=0x7f2bbc011930 comp rx=0 tx=0).stop 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3956033582 shutdown_connections 2024-03-26T17:53:19.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8d9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8d6f20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/3956033582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.142+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3956033582 >> 172.21.15.105:0/3956033582 conn(0x559c0f8ccfa0 msgr2=0x559c0f8cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bca59c640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2bb40a2fa0 0x7f2bb40a5460 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009230 tx=0x7f2bc002bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3956033582 shutdown_connections 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3956033582 wait complete. 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 -- start start 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 Processor -- start 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d7590 0x559c0f8e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559c0f8da540 con 0x559c0f8d7590 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559c0f7f2310 con 0x559c0f8d1a30 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559c0f8da6c0 con 0x559c0f8d2420 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.143+0000 7f2bd3c59740 10 client.0 _ll_register_callbacks cb 0x559c0f82dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:19.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bcb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d7590 0x559c0f8e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bca59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bcb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d7590 0x559c0f8e3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47680/0 (socket says 172.21.15.105:47680) 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bcb59e640 1 -- 172.21.15.105:0/588877680 learned_addr learned my addr 172.21.15.105:0/588877680 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bca59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34070/0 (socket says 172.21.15.105:34070) 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bca59c640 1 -- 172.21.15.105:0/588877680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d1a30 msgr2=0x559c0f8dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bca59c640 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bca59c640 1 -- 172.21.15.105:0/588877680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d7590 msgr2=0x559c0f8e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bca59c640 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d7590 0x559c0f8e3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bca59c640 1 -- 172.21.15.105:0/588877680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0f6147d0 con 0x559c0f8d2420 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bcb59e640 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d7590 0x559c0f8e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2bca59c640 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40fe94c68fb9de25 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2baeff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:19.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.144+0000 7f2baeff5640 1 -- 172.21.15.105:0/588877680 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0003a80 con 0x559c0f8d2420 2024-03-26T17:53:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2baeff5640 1 -- 172.21.15.105:0/588877680 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2bc0003c20 con 0x559c0f8d2420 2024-03-26T17:53:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 1 -- 172.21.15.105:0/588877680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bbc004020 con 0x559c0f8d2420 2024-03-26T17:53:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2baeff5640 1 -- 172.21.15.105:0/588877680 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2bc0003e20 con 0x559c0f8d2420 2024-03-26T17:53:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 10 client.30709 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 1 -- 172.21.15.105:0/588877680 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559c0f8e6840 con 0x559c0f8d2420 2024-03-26T17:53:19.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2baeff5640 1 -- 172.21.15.105:0/588877680 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bbc004020 con 0x559c0f8d2420 2024-03-26T17:53:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2baeff5640 1 -- 172.21.15.105:0/588877680 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2bc0051620 con 0x559c0f8d2420 2024-03-26T17:53:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 10 client.30709 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 10 client.30709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 20 client.30709 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559c0f8e8050 0x559c0f8ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 4 client.30709 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.145+0000 7f2bd3c59740 1 -- 172.21.15.105:0/588877680 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c0f843eb0 con 0x559c0f8e8050 2024-03-26T17:53:19.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.146+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559c0f8e8050 0x559c0f8ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.146+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559c0f8e8050 0x559c0f8ea4f0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c2d7bd1eb8bdaa server_cookie=98a463363e4bf95f in_seq=0 out_seq=0 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.146+0000 7f2baeff5640 10 client.30709 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2baeff5640 1 -- 172.21.15.105:0/588877680 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559c0f843eb0 con 0x559c0f8e8050 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2baeff5640 10 client.30709 handle_command_reply: tid=0 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 client.30709 shutdown 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 -- 172.21.15.105:0/588877680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 msgr2=0x559c0f8dc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 -- 172.21.15.105:0/588877680 shutdown_connections 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559c0f8e8050 0x559c0f8ea4f0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d7590 0x559c0f8e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/588877680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 -- 172.21.15.105:0/588877680 >> 172.21.15.105:0/588877680 conn(0x559c0f8ccfa0 msgr2=0x559c0f8cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2baeff5640 0 client.30709 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:19.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 -- 172.21.15.105:0/588877680 shutdown_connections 2024-03-26T17:53:19.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 1 -- 172.21.15.105:0/588877680 wait complete. 2024-03-26T17:53:19.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.148+0000 7f2bd3c59740 20 client.30709 trim_cache size 0 max 16384 2024-03-26T17:53:19.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:18 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3927119958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:53:19.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.170+0000 7f2bd3c59740 1 Processor -- start 2024-03-26T17:53:19.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.170+0000 7f2bd3c59740 1 -- start start 2024-03-26T17:53:19.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.170+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47682/0 (socket says 172.21.15.105:47682) 2024-03-26T17:53:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcad9d640 1 -- 172.21.15.105:0/3379732225 learned_addr learned my addr 172.21.15.105:0/3379732225 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3379732225 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1a2780 con 0x559c0f8d1a30 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3379732225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1a4380 con 0x559c0f8d7590 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3379732225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1a2e50 con 0x559c0f8d2420 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bca59c640 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcb59e640 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcad9d640 1 -- 172.21.15.105:0/3379732225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 msgr2=0x559c0f8dc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcad9d640 1 -- 172.21.15.105:0/3379732225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 msgr2=0x559c0f8e3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.171+0000 7f2bcad9d640 1 -- 172.21.15.105:0/3379732225 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0f8cd9f0 con 0x559c0f8d1a30 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.172+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc011440 tx=0x7f2bbc004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8fd4e0e786f22 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.172+0000 7f2bb0ff9640 1 -- 172.21.15.105:0/3379732225 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc004770 con 0x559c0f8d1a30 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.172+0000 7f2bb0ff9640 1 -- 172.21.15.105:0/3379732225 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bbc02a1d0 con 0x559c0f8d1a30 2024-03-26T17:53:19.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.172+0000 7f2bb0ff9640 1 -- 172.21.15.105:0/3379732225 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc003970 con 0x559c0f8d1a30 2024-03-26T17:53:19.164 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:18 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3927119958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3379732225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 msgr2=0x559c0f8dbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc011440 tx=0x7f2bbc004680 comp rx=0 tx=0).stop 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3379732225 shutdown_connections 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d2420 0x559c0f8dc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/3379732225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d1a30 0x559c0f8dbee0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3379732225 >> 172.21.15.105:0/3379732225 conn(0x559c0f8ccfa0 msgr2=0x559c0f8daa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3379732225 shutdown_connections 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.173+0000 7f2bd3c59740 1 -- 172.21.15.105:0/3379732225 wait complete. 2024-03-26T17:53:19.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bd3c59740 1 -- start start 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bd3c59740 1 Processor -- start 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d1a30 0x559c0f8e3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d2420 0x559c0f8dfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bd3c59740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d2420 0x559c0f8dfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d1a30 0x559c0f8e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d2420 0x559c0f8dfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47690/0 (socket says 172.21.15.105:47690) 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bca59c640 1 -- 172.21.15.105:0/1561622257 learned_addr learned my addr 172.21.15.105:0/1561622257 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bcad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d1a30 0x559c0f8e3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34090/0 (socket says 172.21.15.105:34090) 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bcb59e640 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bd3c59740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1a3cb0 con 0x559c0f8d2420 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.174+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc1a3520 con 0x559c0f8d7590 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc071f00 con 0x559c0f8d1a30 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bd3c59740 10 client.0 _ll_register_callbacks cb 0x7f2bc0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bca59c640 1 -- 172.21.15.105:0/1561622257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d1a30 msgr2=0x559c0f8e3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:19.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bca59c640 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d1a30 0x559c0f8e3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bca59c640 1 -- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 msgr2=0x559c0f8e02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bca59c640 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bca59c640 1 -- 172.21.15.105:0/1561622257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0f94c420 con 0x559c0f8d2420 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d1a30 0x559c0f8e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bcb59e640 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2bca59c640 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d2420 0x559c0f8dfd70 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0051cd0 tx=0x7f2bc0051f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4169c73c0988950a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2baeff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.175+0000 7f2baeff5640 1 -- 172.21.15.105:0/1561622257 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0005040 con 0x559c0f8d2420 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2baeff5640 1 -- 172.21.15.105:0/1561622257 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2bc00051e0 con 0x559c0f8d2420 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bbc02a1d0 con 0x559c0f8d2420 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2baeff5640 1 -- 172.21.15.105:0/1561622257 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2bc00053e0 con 0x559c0f8d2420 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 10 client.20958 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559c0f94d5c0 con 0x559c0f8d2420 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2baeff5640 1 -- 172.21.15.105:0/1561622257 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2bbc02a1d0 con 0x559c0f8d2420 2024-03-26T17:53:19.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2baeff5640 1 -- 172.21.15.105:0/1561622257 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2bc0060460 con 0x559c0f8d2420 2024-03-26T17:53:19.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 10 client.20958 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:19.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 10 client.20958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:19.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 20 client.20958 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:19.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559c0f94e060 0x559c0f950480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:19.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 4 client.20958 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:19.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.176+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c0f83e200 con 0x559c0f94e060 2024-03-26T17:53:19.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.177+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559c0f94e060 0x559c0f950480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.177+0000 7f2baeff5640 10 client.20958 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.177+0000 7f2bcad9d640 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559c0f94e060 0x559c0f950480 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20833bdc8d93103 server_cookie=77cf67b8440927f9 in_seq=0 out_seq=0 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2baeff5640 1 -- 172.21.15.105:0/1561622257 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c0f83e200 con 0x559c0f94e060 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2baeff5640 10 client.20958 handle_command_reply: tid=0 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 client.20958 shutdown 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d2420 msgr2=0x559c0f8dfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d2420 0x559c0f8dfd70 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0051cd0 tx=0x7f2bc0051f80 comp rx=0 tx=0).stop 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 shutdown_connections 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559c0f94e060 0x559c0f950480 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559c0f8d7590 0x559c0f8e02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2baeff5640 0 client.20958 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:19.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559c0f8d2420 0x559c0f8dfd70 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 --2- 172.21.15.105:0/1561622257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559c0f8d1a30 0x559c0f8e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 >> 172.21.15.105:0/1561622257 conn(0x559c0f8ccfa0 msgr2=0x559c0f8d3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:19.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 shutdown_connections 2024-03-26T17:53:19.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 1 -- 172.21.15.105:0/1561622257 wait complete. 2024-03-26T17:53:19.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.179+0000 7f2bd3c59740 20 client.20958 trim_cache size 0 max 16384 2024-03-26T17:53:19.173 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:19.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 -- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2bb40a2fa0 msgr2=0x7f2bb40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2bb40a2fa0 0x7f2bb40a5460 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009230 tx=0x7f2bc002bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:19.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 -- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc103480 msgr2=0x7f2bcc1a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:19.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc103480 0x7f2bcc1a1320 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400e9b0 tx=0x7f2bc400ee70 comp rx=0 tx=0).stop 2024-03-26T17:53:19.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 -- 172.21.15.105:0/2801588493 shutdown_connections 2024-03-26T17:53:19.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2bb40a2fa0 0x7f2bb40a5460 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bcc103480 0x7f2bcc1a1320 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc19a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 --2- 172.21.15.105:0/2801588493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bcc1018d0 0x7f2bcc199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.180+0000 7f2bd1075640 1 -- 172.21.15.105:0/2801588493 >> 172.21.15.105:0/2801588493 conn(0x7f2bcc0fce60 msgr2=0x7f2bcc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.181+0000 7f2bd1075640 1 -- 172.21.15.105:0/2801588493 shutdown_connections 2024-03-26T17:53:19.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:19.181+0000 7f2bd1075640 1 -- 172.21.15.105:0/2801588493 wait complete. 2024-03-26T17:53:19.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:18 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3927119958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:53:19.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 16227 3.38 MB/sec execute 2 sec latency 119.653 ms 2024-03-26T17:53:20.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:19 smithi105 ceph-mon[32280]: pgmap v798: 97 pgs: 97 active+clean; 238 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-26T17:53:20.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:19 smithi135 ceph-mon[35636]: pgmap v798: 97 pgs: 97 active+clean; 238 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-26T17:53:20.186 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:53:20.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:53:20.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:53:20.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 -- 172.21.15.105:0/753203116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0e01007c0 msgr2=0x7fc0e0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/753203116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0e01007c0 0x7fc0e0100be0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009860 tx=0x7fc0dc02bb70 comp rx=0 tx=0).stop 2024-03-26T17:53:20.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 -- 172.21.15.105:0/753203116 shutdown_connections 2024-03-26T17:53:20.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/753203116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e0102370 0x7fc0e0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/753203116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0e01019b0 0x7fc0e0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/753203116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0e01007c0 0x7fc0e0100be0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 -- 172.21.15.105:0/753203116 >> 172.21.15.105:0/753203116 conn(0x7fc0e00fbd50 msgr2=0x7fc0e00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 -- 172.21.15.105:0/753203116 shutdown_connections 2024-03-26T17:53:20.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.338+0000 7fc0e77e1640 1 -- 172.21.15.105:0/753203116 wait complete. 2024-03-26T17:53:20.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e77e1640 1 Processor -- start 2024-03-26T17:53:20.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e77e1640 1 -- start start 2024-03-26T17:53:20.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e77e1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0e01007c0 0x7fc0e0195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e77e1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e01019b0 0x7fc0e0195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e77e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0e0102370 0x7fc0e019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e77e1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010b1f0 con 0x7fc0e01007c0 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e77e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010b070 con 0x7fc0e0102370 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e77e1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010b370 con 0x7fc0e01019b0 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e557d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0e01007c0 0x7fc0e0195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e557d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0e01007c0 0x7fc0e0195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47700/0 (socket says 172.21.15.105:47700) 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e557d640 1 -- 172.21.15.105:0/3981962393 learned_addr learned my addr 172.21.15.105:0/3981962393 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.339+0000 7fc0e4d7c640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e01019b0 0x7fc0e0195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0e0102370 0x7fc0e019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e557d640 1 -- 172.21.15.105:0/3981962393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e01019b0 msgr2=0x7fc0e0195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e557d640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e01019b0 0x7fc0e0195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e557d640 1 -- 172.21.15.105:0/3981962393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0e0102370 msgr2=0x7fc0e019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e557d640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0e0102370 0x7fc0e019cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e557d640 1 -- 172.21.15.105:0/3981962393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e019d550 con 0x7fc0e01007c0 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e4d7c640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e01019b0 0x7fc0e0195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e557d640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0e01007c0 0x7fc0e0195930 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0057c0 tx=0x7fc0dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0d67fc640 1 -- 172.21.15.105:0/3981962393 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc041030 con 0x7fc0e01007c0 2024-03-26T17:53:20.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0d67fc640 1 -- 172.21.15.105:0/3981962393 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0dc02c9c0 con 0x7fc0e01007c0 2024-03-26T17:53:20.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0e77e1640 1 -- 172.21.15.105:0/3981962393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e019d7e0 con 0x7fc0e01007c0 2024-03-26T17:53:20.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.340+0000 7fc0d67fc640 1 -- 172.21.15.105:0/3981962393 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc039040 con 0x7fc0e01007c0 2024-03-26T17:53:20.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.341+0000 7fc0e77e1640 1 -- 172.21.15.105:0/3981962393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e019dc60 con 0x7fc0e01007c0 2024-03-26T17:53:20.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.343+0000 7fc0d67fc640 1 -- 172.21.15.105:0/3981962393 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc0dc044070 con 0x7fc0e01007c0 2024-03-26T17:53:20.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.344+0000 7fc0d67fc640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0cc0a2fa0 0x7fc0cc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.344+0000 7fc0ea3c5740 1 Processor -- start 2024-03-26T17:53:20.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.344+0000 7fc0ea3c5740 1 -- start start 2024-03-26T17:53:20.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0d67fc640 1 -- 172.21.15.105:0/3981962393 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc0dc0f0390 con 0x7fc0e01007c0 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e4d7c640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0cc0a2fa0 0x7fc0cc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6ca190 0x5585fc6ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6cab80 0x5585fc6cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5585fc62f630 con 0x5585fc6ca190 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5585fc5d7e10 con 0x5585fc6cfcf0 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5585fc62b5e0 con 0x5585fc6cab80 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e4d7c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6cab80 0x5585fc6cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e557d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6ca190 0x5585fc6ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e4d7c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6cab80 0x5585fc6cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34118/0 (socket says 172.21.15.105:34118) 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e4d7c640 1 -- 172.21.15.105:0/225619902 learned_addr learned my addr 172.21.15.105:0/225619902 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e4d7c640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0cc0a2fa0 0x7fc0cc0a5460 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000d690 tx=0x7fc0d0009830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e5d7e640 1 -- 172.21.15.105:0/225619902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6cab80 msgr2=0x5585fc6cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6cab80 0x5585fc6cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e5d7e640 1 -- 172.21.15.105:0/225619902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6ca190 msgr2=0x5585fc6ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6ca190 0x5585fc6ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e5d7e640 1 -- 172.21.15.105:0/225619902 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585fc5ee740 con 0x5585fc6cfcf0 2024-03-26T17:53:20.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.345+0000 7fc0e557d640 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6ca190 0x5585fc6ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.346+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6d2180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009f00 tx=0x7fc0d8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac8c42c1a6cee7f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.346+0000 7fc0cb7fe640 1 -- 172.21.15.105:0/225619902 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8004d20 con 0x5585fc6cfcf0 2024-03-26T17:53:20.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.346+0000 7fc0cb7fe640 1 -- 172.21.15.105:0/225619902 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0d8004050 con 0x5585fc6cfcf0 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.346+0000 7fc0cb7fe640 1 -- 172.21.15.105:0/225619902 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8005220 con 0x5585fc6cfcf0 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/225619902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 msgr2=0x5585fc6d2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6d2180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009f00 tx=0x7fc0d8011810 comp rx=0 tx=0).stop 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/225619902 shutdown_connections 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6d2180 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6cab80 0x5585fc6cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/225619902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6ca190 0x5585fc6ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/225619902 >> 172.21.15.105:0/225619902 conn(0x5585fc6c5700 msgr2=0x5585fc6c5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/225619902 shutdown_connections 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/225619902 wait complete. 2024-03-26T17:53:20.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 -- start start 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.347+0000 7fc0ea3c5740 1 Processor -- start 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6e0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5585fc6d7100 con 0x5585fc6cab80 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5585fc5eb1a0 con 0x5585fc6cfcf0 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5585fc6d7280 con 0x5585fc6ca190 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0ea3c5740 10 client.0 _ll_register_callbacks cb 0x5585fc342c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e5d7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e5d7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6e0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57866/0 (socket says 172.21.15.105:57866) 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e5d7e640 1 -- 172.21.15.105:0/2026953823 learned_addr learned my addr 172.21.15.105:0/2026953823 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e557d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e4d7c640 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e5d7e640 1 -- 172.21.15.105:0/2026953823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 msgr2=0x5585fc6d8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e5d7e640 1 -- 172.21.15.105:0/2026953823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 msgr2=0x5585fc6d8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e5d7e640 1 -- 172.21.15.105:0/2026953823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585fc40b7e0 con 0x5585fc6cfcf0 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.348+0000 7fc0e557d640 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6e0010 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc0d801b040 tx=0x7fc0d8004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e72ebff0bfbf4c4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/2026953823 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8028020 con 0x5585fc6cfcf0 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/2026953823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d8004050 con 0x5585fc6cfcf0 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/2026953823 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc0d8004410 con 0x5585fc6cfcf0 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/2026953823 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc0d8033050 con 0x5585fc6cfcf0 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0ea3c5740 10 client.30755 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:20.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/2026953823 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5585fc46bf70 con 0x5585fc6cfcf0 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/2026953823 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0d8004050 con 0x5585fc6cfcf0 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.349+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/2026953823 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc0d8023450 con 0x5585fc6cfcf0 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.350+0000 7fc0ea3c5740 10 client.30755 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.350+0000 7fc0ea3c5740 10 client.30755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.350+0000 7fc0ea3c5740 20 client.30755 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.350+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5585fc6e4db0 0x5585fc6e7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.350+0000 7fc0ea3c5740 4 client.30755 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.350+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/2026953823 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585fc63cb70 con 0x5585fc6e4db0 2024-03-26T17:53:20.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.350+0000 7fc0e557d640 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5585fc6e4db0 0x5585fc6e7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.351+0000 7fc0c8ff9640 10 client.30755 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.351+0000 7fc0e557d640 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5585fc6e4db0 0x5585fc6e7210 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceaa3ed45b604b67 server_cookie=61191238e593d9ac in_seq=0 out_seq=0 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/2026953823 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5585fc63cb70 con 0x5585fc6e4db0 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0c8ff9640 10 client.30755 handle_command_reply: tid=0 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 client.30755 shutdown 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/2026953823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 msgr2=0x5585fc6e0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6e0010 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc0d801b040 tx=0x7fc0d8004cf0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/2026953823 shutdown_connections 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5585fc6e4db0 0x5585fc6e7210 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6e0010 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/2026953823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/2026953823 >> 172.21.15.105:0/2026953823 conn(0x5585fc6c5700 msgr2=0x5585fc6c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0c8ff9640 0 client.30755 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:20.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/2026953823 shutdown_connections 2024-03-26T17:53:20.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/2026953823 wait complete. 2024-03-26T17:53:20.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.353+0000 7fc0ea3c5740 20 client.30755 trim_cache size 0 max 16384 2024-03-26T17:53:20.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:19 smithi144 ceph-mon[36095]: pgmap v798: 97 pgs: 97 active+clean; 238 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-03-26T17:53:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.381+0000 7fc0ea3c5740 1 Processor -- start 2024-03-26T17:53:20.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.381+0000 7fc0ea3c5740 1 -- start start 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.381+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.381+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.381+0000 7fc0e4d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.381+0000 7fc0e557d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.381+0000 7fc0e557d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34140/0 (socket says 172.21.15.105:34140) 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e4d7c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47742/0 (socket says 172.21.15.105:47742) 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e557d640 1 -- 172.21.15.105:0/409548354 learned_addr learned my addr 172.21.15.105:0/409548354 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/409548354 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a6b90 con 0x5585fc6cab80 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/409548354 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a8790 con 0x5585fc6cfcf0 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/409548354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a7260 con 0x5585fc6ca190 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e557d640 1 -- 172.21.15.105:0/409548354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 msgr2=0x5585fc6c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e557d640 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e557d640 1 -- 172.21.15.105:0/409548354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 msgr2=0x5585fc6d8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e557d640 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e557d640 1 -- 172.21.15.105:0/409548354 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585fc303170 con 0x5585fc6ca190 2024-03-26T17:53:20.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e4d7c640 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.382+0000 7fc0e557d640 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009830 tx=0x7fc0dc02bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9edf9fdedb75c1f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0caffd640 1 -- 172.21.15.105:0/409548354 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0bca80 con 0x5585fc6ca190 2024-03-26T17:53:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0caffd640 1 -- 172.21.15.105:0/409548354 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0dc0c0bd0 con 0x5585fc6ca190 2024-03-26T17:53:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0caffd640 1 -- 172.21.15.105:0/409548354 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0c5260 con 0x5585fc6ca190 2024-03-26T17:53:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/409548354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 msgr2=0x5585fc6d8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009830 tx=0x7fc0dc02bb70 comp rx=0 tx=0).stop 2024-03-26T17:53:20.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/409548354 shutdown_connections 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cfcf0 0x5585fc6c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cab80 0x5585fc6d8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/409548354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d8a90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.383+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/409548354 >> 172.21.15.105:0/409548354 conn(0x5585fc6c5700 msgr2=0x5585fc6c5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/409548354 shutdown_connections 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/409548354 wait complete. 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 -- start start 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 Processor -- start 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cab80 0x5585fc6d6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cfcf0 0x5585fc74d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a80c0 con 0x5585fc6cfcf0 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a7930 con 0x5585fc6cab80 2024-03-26T17:53:20.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.384+0000 7fc0ea3c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a4b70 con 0x5585fc6ca190 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0ea3c5740 10 client.0 _ll_register_callbacks cb 0x7fc0dc005620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e4d7c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cab80 0x5585fc6d6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e5d7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cfcf0 0x5585fc74d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e557d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e4d7c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cab80 0x5585fc6d6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57886/0 (socket says 172.21.15.105:57886) 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e4d7c640 1 -- 172.21.15.105:0/3006779946 learned_addr learned my addr 172.21.15.105:0/3006779946 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e557d640 1 -- 172.21.15.105:0/3006779946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cab80 msgr2=0x5585fc6d6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e557d640 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cab80 0x5585fc6d6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e557d640 1 -- 172.21.15.105:0/3006779946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cfcf0 msgr2=0x5585fc74d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e557d640 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cfcf0 0x5585fc74d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e557d640 1 -- 172.21.15.105:0/3006779946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585fc74df00 con 0x5585fc6ca190 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e5d7e640 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cfcf0 0x5585fc74d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0e557d640 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d5f10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x5585fc5d7f40 tx=0x7fc0dc0c5490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4f436f0c6b7fb78 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:20.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/3006779946 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc0c5000 con 0x5585fc6ca190 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/3006779946 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc0dc0c51a0 con 0x5585fc6ca190 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.385+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/3006779946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0dc0c0bd0 con 0x5585fc6ca190 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/3006779946 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc0dc02c510 con 0x5585fc6ca190 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0ea3c5740 10 client.30733 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/3006779946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5585fc74f2e0 con 0x5585fc6ca190 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/3006779946 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0dc0c0bd0 con 0x5585fc6ca190 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/3006779946 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc0dc03c030 con 0x5585fc6ca190 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0ea3c5740 10 client.30733 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0ea3c5740 10 client.30733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0ea3c5740 20 client.30733 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5585fc74fb30 0x5585fc751f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0ea3c5740 4 client.30733 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/3006779946 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5585fc636ec0 con 0x5585fc74fb30 2024-03-26T17:53:20.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.386+0000 7fc0e4d7c640 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5585fc74fb30 0x5585fc751f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.387+0000 7fc0c8ff9640 10 client.30733 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:20.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.387+0000 7fc0e4d7c640 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5585fc74fb30 0x5585fc751f50 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599609e7ac71f78e server_cookie=580b3aa33241feac in_seq=0 out_seq=0 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0c8ff9640 1 -- 172.21.15.105:0/3006779946 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5585fc636ec0 con 0x5585fc74fb30 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0c8ff9640 10 client.30733 handle_command_reply: tid=0 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 client.30733 shutdown 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/3006779946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 msgr2=0x5585fc6d5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d5f10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x5585fc5d7f40 tx=0x7fc0dc0c5490 comp rx=0 tx=0).stop 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/3006779946 shutdown_connections 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5585fc74fb30 0x5585fc751f50 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5585fc6cfcf0 0x5585fc74d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585fc6cab80 0x5585fc6d6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 --2- 172.21.15.105:0/3006779946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5585fc6ca190 0x5585fc6d5f10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/3006779946 >> 172.21.15.105:0/3006779946 conn(0x5585fc6c5700 msgr2=0x5585fc6cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.389+0000 7fc0c8ff9640 0 client.30733 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:20.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.390+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/3006779946 shutdown_connections 2024-03-26T17:53:20.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.390+0000 7fc0ea3c5740 1 -- 172.21.15.105:0/3006779946 wait complete. 2024-03-26T17:53:20.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.390+0000 7fc0ea3c5740 20 client.30733 trim_cache size 0 max 16384 2024-03-26T17:53:20.383 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:53:20.383 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:53:20.383 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "713b9a78-0b37-47c7-aaa9-43c50116053c", 2024-03-26T17:53:20.384 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:53:20.384 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:53:20.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 -- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0cc0a2fa0 msgr2=0x7fc0cc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0cc0a2fa0 0x7fc0cc0a5460 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000d690 tx=0x7fc0d0009830 comp rx=0 tx=0).stop 2024-03-26T17:53:20.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 -- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0e01007c0 msgr2=0x7fc0e0195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0e01007c0 0x7fc0e0195930 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0057c0 tx=0x7fc0dc0042f0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 -- 172.21.15.105:0/3981962393 shutdown_connections 2024-03-26T17:53:20.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc0cc0a2fa0 0x7fc0cc0a5460 secure :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc0d000d690 tx=0x7fc0d0009830 comp rx=0 tx=0).stop 2024-03-26T17:53:20.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0e0102370 0x7fc0e019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0e01019b0 0x7fc0e0195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 --2- 172.21.15.105:0/3981962393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0e01007c0 0x7fc0e0195930 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 -- 172.21.15.105:0/3981962393 >> 172.21.15.105:0/3981962393 conn(0x7fc0e00fbd50 msgr2=0x7fc0e0104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 -- 172.21.15.105:0/3981962393 shutdown_connections 2024-03-26T17:53:20.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.391+0000 7fc0e77e1640 1 -- 172.21.15.105:0/3981962393 wait complete. 2024-03-26T17:53:20.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:53:20.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 -- 172.21.15.105:0/1185144617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034102420 msgr2=0x7f8034104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1185144617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034102420 0x7f8034104810 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f8028009900 tx=0x7f802802bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 -- 172.21.15.105:0/1185144617 shutdown_connections 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1185144617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8034107680 0x7f8034109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1185144617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8034104d50 0x7f8034107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1185144617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034102420 0x7f8034104810 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 -- 172.21.15.105:0/1185144617 >> 172.21.15.105:0/1185144617 conn(0x7f80340fbd70 msgr2=0x7f80340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 -- 172.21.15.105:0/1185144617 shutdown_connections 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.537+0000 7f803ad1b640 1 -- 172.21.15.105:0/1185144617 wait complete. 2024-03-26T17:53:20.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f803ad1b640 1 Processor -- start 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f803ad1b640 1 -- start start 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f803ad1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8034102420 0x7f8034199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f803ad1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034104d50 0x7f803419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f803ad1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8034107680 0x7f80341a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f803ad1b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f803410d450 con 0x7f8034104d50 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f803ad1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f803410d2d0 con 0x7f8034102420 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f803ad1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f803410d5d0 con 0x7f8034107680 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8033fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034104d50 0x7f803419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8033fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034104d50 0x7f803419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47766/0 (socket says 172.21.15.105:47766) 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8033fff640 1 -- 172.21.15.105:0/1427611931 learned_addr learned my addr 172.21.15.105:0/1427611931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f80392b8640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8034107680 0x7f80341a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8038ab7640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8034102420 0x7f8034199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8033fff640 1 -- 172.21.15.105:0/1427611931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8034107680 msgr2=0x7f80341a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8033fff640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8034107680 0x7f80341a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8033fff640 1 -- 172.21.15.105:0/1427611931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8034102420 msgr2=0x7f8034199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8033fff640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8034102420 0x7f8034199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.538+0000 7f8033fff640 1 -- 172.21.15.105:0/1427611931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80341a19f0 con 0x7f8034104d50 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.539+0000 7f8033fff640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034104d50 0x7f803419a310 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f802000e9f0 tx=0x7f802000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.539+0000 7f8031ffb640 1 -- 172.21.15.105:0/1427611931 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802000ccc0 con 0x7f8034104d50 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.539+0000 7f8031ffb640 1 -- 172.21.15.105:0/1427611931 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80200054d0 con 0x7f8034104d50 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.539+0000 7f803ad1b640 1 -- 172.21.15.105:0/1427611931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80341a1ce0 con 0x7f8034104d50 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.539+0000 7f8031ffb640 1 -- 172.21.15.105:0/1427611931 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020010430 con 0x7f8034104d50 2024-03-26T17:53:20.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.539+0000 7f803ad1b640 1 -- 172.21.15.105:0/1427611931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80341a21d0 con 0x7f8034104d50 2024-03-26T17:53:20.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.542+0000 7f8031ffb640 1 -- 172.21.15.105:0/1427611931 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8020005000 con 0x7f8034104d50 2024-03-26T17:53:20.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.542+0000 7f803d9e6740 1 Processor -- start 2024-03-26T17:53:20.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.542+0000 7f803d9e6740 1 -- start start 2024-03-26T17:53:20.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.542+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd78d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.542+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd7dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.542+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f803d9e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bcdd980 con 0x55ac3bd7e450 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f803d9e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bc85be0 con 0x55ac3bd788f0 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f803d9e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bcd9930 con 0x55ac3bd792e0 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f80392b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd78d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8038ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd7dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8033fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f80392b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd78d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57916/0 (socket says 172.21.15.105:57916) 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8038ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd7dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34180/0 (socket says 172.21.15.105:34180) 2024-03-26T17:53:20.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f80392b8640 1 -- 172.21.15.105:0/4174175730 learned_addr learned my addr 172.21.15.105:0/4174175730 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8031ffb640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80240a2fa0 0x7f80240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8038ab7640 1 -- 172.21.15.105:0/4174175730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 msgr2=0x55ac3bd78d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8038ab7640 1 --2- 172.21.15.105:0/4174175730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd78d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8038ab7640 1 -- 172.21.15.105:0/4174175730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 msgr2=0x55ac3bd808e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8038ab7640 1 --2- 172.21.15.105:0/4174175730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd808e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8038ab7640 1 -- 172.21.15.105:0/4174175730 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3babb7d0 con 0x55ac3bd792e0 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8031ffb640 1 -- 172.21.15.105:0/1427611931 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f80200cf7a0 con 0x7f8034104d50 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f80392b8640 1 --2- 172.21.15.105:0/4174175730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd78d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f80392b8640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80240a2fa0 0x7f80240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.543+0000 7f8038ab7640 1 --2- 172.21.15.105:0/4174175730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd7dde0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8028000c00 tx=0x7f8028005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ce0aa95d074db5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f801effd640 1 -- 172.21.15.105:0/4174175730 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028045020 con 0x55ac3bd792e0 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f801effd640 1 -- 172.21.15.105:0/4174175730 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8028004040 con 0x55ac3bd792e0 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f801effd640 1 -- 172.21.15.105:0/4174175730 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280055a0 con 0x55ac3bd792e0 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f803d9e6740 1 -- 172.21.15.105:0/4174175730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 msgr2=0x55ac3bd7dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f803d9e6740 1 --2- 172.21.15.105:0/4174175730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd7dde0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8028000c00 tx=0x7f8028005c00 comp rx=0 tx=0).stop 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f80392b8640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80240a2fa0 0x7f80240a5460 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55ac3bd79b40 tx=0x7f802c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f803d9e6740 1 -- 172.21.15.105:0/4174175730 shutdown_connections 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f803d9e6740 1 --2- 172.21.15.105:0/4174175730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd808e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f803d9e6740 1 --2- 172.21.15.105:0/4174175730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd7dde0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f803d9e6740 1 --2- 172.21.15.105:0/4174175730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd78d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.544+0000 7f803d9e6740 1 -- 172.21.15.105:0/4174175730 >> 172.21.15.105:0/4174175730 conn(0x55ac3bd73e60 msgr2=0x55ac3bd74260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 -- 172.21.15.105:0/4174175730 shutdown_connections 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 -- 172.21.15.105:0/4174175730 wait complete. 2024-03-26T17:53:20.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 -- start start 2024-03-26T17:53:20.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 Processor -- start 2024-03-26T17:53:20.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd788f0 0x55ac3bd82a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd792e0 0x55ac3bd82fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd8a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bd813a0 con 0x55ac3bd7e450 2024-03-26T17:53:20.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bc99160 con 0x55ac3bd792e0 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bd81520 con 0x55ac3bd788f0 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f803d9e6740 10 client.0 _ll_register_callbacks cb 0x55ac3bcd5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.545+0000 7f8038ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd788f0 0x55ac3bd82a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8033fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd792e0 0x55ac3bd82fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8038ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd788f0 0x55ac3bd82a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34182/0 (socket says 172.21.15.105:34182) 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8038ab7640 1 -- 172.21.15.105:0/115275819 learned_addr learned my addr 172.21.15.105:0/115275819 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f80392b8640 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd8a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8038ab7640 1 -- 172.21.15.105:0/115275819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd792e0 msgr2=0x55ac3bd82fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8038ab7640 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd792e0 0x55ac3bd82fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8038ab7640 1 -- 172.21.15.105:0/115275819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 msgr2=0x55ac3bd8a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8038ab7640 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd8a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8038ab7640 1 -- 172.21.15.105:0/115275819 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3b9f13f0 con 0x55ac3bd788f0 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8033fff640 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd792e0 0x55ac3bd82fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8038ab7640 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd788f0 0x55ac3bd82a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8028043000 tx=0x7f80280055d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8df60879f3f36 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8013fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8013fff640 1 -- 172.21.15.105:0/115275819 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028050070 con 0x55ac3bd788f0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f803d9e6740 1 -- 172.21.15.105:0/115275819 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8028004040 con 0x55ac3bd788f0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.546+0000 7f8013fff640 1 -- 172.21.15.105:0/115275819 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f80280035f0 con 0x55ac3bd788f0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f8013fff640 1 -- 172.21.15.105:0/115275819 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8028045070 con 0x55ac3bd788f0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f803d9e6740 10 client.30745 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f803d9e6740 1 -- 172.21.15.105:0/115275819 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ac3b9b10f0 con 0x55ac3bd788f0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f8013fff640 1 -- 172.21.15.105:0/115275819 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8028004040 con 0x55ac3bd788f0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f8013fff640 1 -- 172.21.15.105:0/115275819 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8028053030 con 0x55ac3bd788f0 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f803d9e6740 10 client.30745 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f803d9e6740 10 client.30745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f803d9e6740 20 client.30745 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f803d9e6740 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac3bd96d00 0x55ac3bd99170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f803d9e6740 4 client.30745 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:20.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f803d9e6740 1 -- 172.21.15.105:0/115275819 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac3bceaec0 con 0x55ac3bd96d00 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.547+0000 7f8033fff640 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac3bd96d00 0x55ac3bd99170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.548+0000 7f8013fff640 10 client.30745 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.548+0000 7f8033fff640 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac3bd96d00 0x55ac3bd99170 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f0b10f102e2e5d server_cookie=6bc12b34fef45318 in_seq=0 out_seq=0 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f8013fff640 1 -- 172.21.15.105:0/115275819 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ac3bceaec0 con 0x55ac3bd96d00 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f8013fff640 10 client.30745 handle_command_reply: tid=0 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 client.30745 shutdown 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 -- 172.21.15.105:0/115275819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd788f0 msgr2=0x55ac3bd82a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd788f0 0x55ac3bd82a80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8028043000 tx=0x7f80280055d0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 -- 172.21.15.105:0/115275819 shutdown_connections 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac3bd96d00 0x55ac3bd99170 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd8a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd792e0 0x55ac3bd82fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f8013fff640 0 client.30745 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 --2- 172.21.15.105:0/115275819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd788f0 0x55ac3bd82a80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 -- 172.21.15.105:0/115275819 >> 172.21.15.105:0/115275819 conn(0x55ac3bd73e60 msgr2=0x55ac3bd73080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 -- 172.21.15.105:0/115275819 shutdown_connections 2024-03-26T17:53:20.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 1 -- 172.21.15.105:0/115275819 wait complete. 2024-03-26T17:53:20.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.551+0000 7f803d9e6740 20 client.30745 trim_cache size 0 max 16384 2024-03-26T17:53:20.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 16378 4.69 MB/sec execute 3 sec latency 176.238 ms 2024-03-26T17:53:20.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f803d9e6740 1 Processor -- start 2024-03-26T17:53:20.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f803d9e6740 1 -- start start 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd7db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd89f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f803d9e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80341a6b40 con 0x55ac3bd7e450 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f803d9e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80341a8740 con 0x55ac3bd788f0 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f803d9e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80341a7210 con 0x55ac3bd792e0 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f8033fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd89f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f8033fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd89f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34190/0 (socket says 172.21.15.105:34190) 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f8033fff640 1 -- 172.21.15.105:0/48772581 learned_addr learned my addr 172.21.15.105:0/48772581 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f8038ab7640 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd7db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.578+0000 7f80392b8640 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f8033fff640 1 -- 172.21.15.105:0/48772581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 msgr2=0x55ac3bd7db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f8033fff640 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd7db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f8033fff640 1 -- 172.21.15.105:0/48772581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 msgr2=0x55ac3bd820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f8033fff640 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd820a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f8033fff640 1 -- 172.21.15.105:0/48772581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3bd6f640 con 0x55ac3bd792e0 2024-03-26T17:53:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f8038ab7640 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd7db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f8033fff640 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd89f20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f80200cd260 tx=0x7f80200bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a866bfc1c1d62c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f801e7fc640 1 -- 172.21.15.105:0/48772581 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200bc0b0 con 0x55ac3bd792e0 2024-03-26T17:53:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f801e7fc640 1 -- 172.21.15.105:0/48772581 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80200046c0 con 0x55ac3bd792e0 2024-03-26T17:53:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.579+0000 7f801e7fc640 1 -- 172.21.15.105:0/48772581 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802009be90 con 0x55ac3bd792e0 2024-03-26T17:53:20.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 -- 172.21.15.105:0/48772581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 msgr2=0x55ac3bd89f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd89f20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f80200cd260 tx=0x7f80200bc040 comp rx=0 tx=0).stop 2024-03-26T17:53:20.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 -- 172.21.15.105:0/48772581 shutdown_connections 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd89f20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 --2- 172.21.15.105:0/48772581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd7db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 -- 172.21.15.105:0/48772581 >> 172.21.15.105:0/48772581 conn(0x55ac3bd73e60 msgr2=0x55ac3bd73080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 -- 172.21.15.105:0/48772581 shutdown_connections 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.580+0000 7f803d9e6740 1 -- 172.21.15.105:0/48772581 wait complete. 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 1 -- start start 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 1 Processor -- start 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd87b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd88b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80341a8070 con 0x55ac3bd7e450 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f80341a78e0 con 0x55ac3bd788f0 2024-03-26T17:53:20.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80341a4b40 con 0x55ac3bd792e0 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f803d9e6740 10 client.0 _ll_register_callbacks cb 0x7f8020097f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f80392b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd88b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f8033fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd87b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f8038ab7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f80392b8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd88b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47790/0 (socket says 172.21.15.105:47790) 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f8033fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd87b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34202/0 (socket says 172.21.15.105:34202) 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.581+0000 7f80392b8640 1 -- 172.21.15.105:0/530764386 learned_addr learned my addr 172.21.15.105:0/530764386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8038ab7640 1 -- 172.21.15.105:0/530764386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 msgr2=0x55ac3bd87b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8038ab7640 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd87b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8038ab7640 1 -- 172.21.15.105:0/530764386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 msgr2=0x55ac3bd88b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8038ab7640 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd88b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8038ab7640 1 -- 172.21.15.105:0/530764386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3bd89130 con 0x55ac3bd788f0 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8033fff640 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd87b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f80392b8640 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd88b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:20.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8038ab7640 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd875f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8028004b70 tx=0x7f802802c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e091eee7978b58e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8013fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8013fff640 1 -- 172.21.15.105:0/530764386 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028050030 con 0x55ac3bd788f0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f8013fff640 1 -- 172.21.15.105:0/530764386 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8028041480 con 0x55ac3bd788f0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.582+0000 7f803d9e6740 1 -- 172.21.15.105:0/530764386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac3bce5210 con 0x55ac3bd788f0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f8013fff640 1 -- 172.21.15.105:0/530764386 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8028065050 con 0x55ac3bd788f0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f803d9e6740 10 client.30779 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f803d9e6740 1 -- 172.21.15.105:0/530764386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ac3bdfd100 con 0x55ac3bd788f0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f8013fff640 1 -- 172.21.15.105:0/530764386 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ac3bce5210 con 0x55ac3bd788f0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f8013fff640 1 -- 172.21.15.105:0/530764386 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8028053070 con 0x55ac3bd788f0 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f803d9e6740 10 client.30779 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f803d9e6740 10 client.30779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f803d9e6740 20 client.30779 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f803d9e6740 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac3bdfd920 0x55ac3bdffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:20.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f803d9e6740 4 client.30779 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f803d9e6740 1 -- 172.21.15.105:0/530764386 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac3bc62540 con 0x55ac3bdfd920 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.583+0000 7f8033fff640 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac3bdfd920 0x55ac3bdffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.584+0000 7f8033fff640 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac3bdfd920 0x55ac3bdffd40 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85413ffee8a1232 server_cookie=a5deec1e854693a6 in_seq=0 out_seq=0 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.584+0000 7f8013fff640 10 client.30779 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f8013fff640 1 -- 172.21.15.105:0/530764386 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55ac3bc62540 con 0x55ac3bdfd920 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f8013fff640 10 client.30779 handle_command_reply: tid=0 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 client.30779 shutdown 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 -- 172.21.15.105:0/530764386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 msgr2=0x55ac3bd875f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd875f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8028004b70 tx=0x7f802802c3b0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 -- 172.21.15.105:0/530764386 shutdown_connections 2024-03-26T17:53:20.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ac3bdfd920 0x55ac3bdffd40 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ac3bd7e450 0x55ac3bd88b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ac3bd792e0 0x55ac3bd87b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 --2- 172.21.15.105:0/530764386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ac3bd788f0 0x55ac3bd875f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 -- 172.21.15.105:0/530764386 >> 172.21.15.105:0/530764386 conn(0x55ac3bd73e60 msgr2=0x55ac3bd77950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f8013fff640 0 client.30779 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 -- 172.21.15.105:0/530764386 shutdown_connections 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.585+0000 7f803d9e6740 1 -- 172.21.15.105:0/530764386 wait complete. 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803d9e6740 20 client.30779 trim_cache size 0 max 16384 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:53:20.581 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (232 inodes in the stack)", 2024-03-26T17:53:20.582 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:53:20.582 INFO:teuthology.orchestra.run.smithi105.stdout: "713b9a78-0b37-47c7-aaa9-43c50116053c": { 2024-03-26T17:53:20.582 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:53:20.582 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "713b9a78-0b37-47c7-aaa9-43c50116053c", 2024-03-26T17:53:20.582 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:53:20.582 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:53:20.582 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:53:20.582 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 -- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80240a2fa0 msgr2=0x7f80240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80240a2fa0 0x7f80240a5460 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55ac3bd79b40 tx=0x7f802c004a40 comp rx=0 tx=0).stop 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 -- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034104d50 msgr2=0x7f803419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034104d50 0x7f803419a310 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f802000e9f0 tx=0x7f802000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 -- 172.21.15.105:0/1427611931 shutdown_connections 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f80240a2fa0 0x7f80240a5460 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8034107680 0x7f80341a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8034104d50 0x7f803419a310 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 --2- 172.21.15.105:0/1427611931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8034102420 0x7f8034199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:20.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 -- 172.21.15.105:0/1427611931 >> 172.21.15.105:0/1427611931 conn(0x7f80340fbd70 msgr2=0x7f8034101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:20.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 -- 172.21.15.105:0/1427611931 shutdown_connections 2024-03-26T17:53:20.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:20.586+0000 7f803ad1b640 1 -- 172.21.15.105:0/1427611931 wait complete. 2024-03-26T17:53:20.591 INFO:tasks.cephfs.filesystem:scrub status for tag:713b9a78-0b37-47c7-aaa9-43c50116053c - {'path': '/', 'tag': '713b9a78-0b37-47c7-aaa9-43c50116053c', 'options': 'recursive,force'} 2024-03-26T17:53:21.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:20 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:53:21.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:21.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:21.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:21.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:20 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:53:21.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:21.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:21.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:21.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:20 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:53:21.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:21.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:21.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:21.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:21.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 16518 5.24 MB/sec execute 4 sec latency 183.231 ms 2024-03-26T17:53:22.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:21 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:22.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:21 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:21 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:22.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:21 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:21 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:22.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:21 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.140 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:21 smithi105 ceph-mon[32280]: pgmap v799: 97 pgs: 97 active+clean; 240 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-03-26T17:53:22.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:21 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:22.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:21 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:21 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:22.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:21 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:21 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:22.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:21 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:21 smithi135 ceph-mon[35636]: pgmap v799: 97 pgs: 97 active+clean; 240 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-03-26T17:53:22.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:21 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:22.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:21 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:21 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:22.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:21 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:21 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:22.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:21 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:22.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:21 smithi144 ceph-mon[36095]: pgmap v799: 97 pgs: 97 active+clean; 240 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-03-26T17:53:22.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 16639 5.80 MB/sec execute 5 sec latency 136.150 ms 2024-03-26T17:53:23.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 16766 6.19 MB/sec execute 6 sec latency 119.320 ms 2024-03-26T17:53:24.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:23 smithi105 ceph-mon[32280]: pgmap v800: 97 pgs: 97 active+clean; 242 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-26T17:53:24.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:23 smithi135 ceph-mon[35636]: pgmap v800: 97 pgs: 97 active+clean; 242 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-26T17:53:24.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:23 smithi144 ceph-mon[36095]: pgmap v800: 97 pgs: 97 active+clean; 242 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 2.8 MiB/s wr, 275 op/s 2024-03-26T17:53:24.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 16912 6.96 MB/sec execute 7 sec latency 134.910 ms 2024-03-26T17:53:25.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:24 smithi135 ceph-mon[35636]: pgmap v801: 97 pgs: 97 active+clean; 244 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-26T17:53:25.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:24 smithi144 ceph-mon[36095]: pgmap v801: 97 pgs: 97 active+clean; 244 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 -- 172.21.15.105:0/1110538130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41034a0 msgr2=0x7f4bb4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/1110538130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41034a0 0x7f4bb4109d30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009ca0 tx=0x7f4bac02c100 comp rx=0 tx=0).stop 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 -- 172.21.15.105:0/1110538130 shutdown_connections 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/1110538130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41034a0 0x7f4bb4109d30 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/1110538130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb4102ae0 0x7f4bb4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/1110538130 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb41018f0 0x7f4bb4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 -- 172.21.15.105:0/1110538130 >> 172.21.15.105:0/1110538130 conn(0x7f4bb40fce80 msgr2=0x7f4bb40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 -- 172.21.15.105:0/1110538130 shutdown_connections 2024-03-26T17:53:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.398+0000 7f4bbbe27640 1 -- 172.21.15.105:0/1110538130 wait complete. 2024-03-26T17:53:25.390 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:24 smithi105 ceph-mon[32280]: pgmap v801: 97 pgs: 97 active+clean; 244 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bbbe27640 1 Processor -- start 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bbbe27640 1 -- start start 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bbbe27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41018f0 0x7f4bb4075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bbbe27640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb4102ae0 0x7f4bb4073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bbbe27640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb41034a0 0x7f4bb40741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bbbe27640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb410d400 con 0x7f4bb41018f0 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bbbe27640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb410d280 con 0x7f4bb41034a0 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bbbe27640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb410d580 con 0x7f4bb4102ae0 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bb9bc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41018f0 0x7f4bb4075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bba3c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb41034a0 0x7f4bb40741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bb9bc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41018f0 0x7f4bb4075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33448/0 (socket says 172.21.15.105:33448) 2024-03-26T17:53:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/3711005929 learned_addr learned my addr 172.21.15.105:0/3711005929 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb4102ae0 0x7f4bb4073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bba3c4640 1 -- 172.21.15.105:0/3711005929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb4102ae0 msgr2=0x7f4bb4073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bba3c4640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb4102ae0 0x7f4bb4073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bba3c4640 1 -- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41018f0 msgr2=0x7f4bb4075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bba3c4640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41018f0 0x7f4bb4075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bba3c4640 1 -- 172.21.15.105:0/3711005929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb4074a80 con 0x7f4bb41034a0 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41018f0 0x7f4bb4075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.399+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb4102ae0 0x7f4bb4073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.400+0000 7f4bba3c4640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb41034a0 0x7f4bb40741f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4bac02c640 tx=0x7f4bac035d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.400+0000 7f4baaffd640 1 -- 172.21.15.105:0/3711005929 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac039040 con 0x7f4bb41034a0 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.400+0000 7f4bbbe27640 1 -- 172.21.15.105:0/3711005929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb41a1ab0 con 0x7f4bb41034a0 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.400+0000 7f4baaffd640 1 -- 172.21.15.105:0/3711005929 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bac003970 con 0x7f4bb41034a0 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.400+0000 7f4baaffd640 1 -- 172.21.15.105:0/3711005929 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac004410 con 0x7f4bb41034a0 2024-03-26T17:53:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.400+0000 7f4bbbe27640 1 -- 172.21.15.105:0/3711005929 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb41a2050 con 0x7f4bb41034a0 2024-03-26T17:53:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.403+0000 7f4baaffd640 1 -- 172.21.15.105:0/3711005929 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4bac046030 con 0x7f4bb41034a0 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.403+0000 7f4bbea15740 1 Processor -- start 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.403+0000 7f4bbea15740 1 -- start start 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4baaffd640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ba00a2fc0 0x7f4ba00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cb1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 0x55a5e8cb45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bbea15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a5e8c110b0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bbea15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5e8bb9960 con 0x55a5e8cb2130 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bbea15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a5e8c0d060 con 0x55a5e8cacfc0 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33462/0 (socket says 172.21.15.105:33462) 2024-03-26T17:53:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 -- 172.21.15.105:0/3087116020 learned_addr learned my addr 172.21.15.105:0/3087116020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4baaffd640 1 -- 172.21.15.105:0/3711005929 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4bac0f0ff0 con 0x7f4bb41034a0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ba00a2fc0 0x7f4ba00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bba3c4640 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 0x55a5e8cb45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cb1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 -- 172.21.15.105:0/3087116020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 msgr2=0x55a5e8cb1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cb1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 -- 172.21.15.105:0/3087116020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 msgr2=0x55a5e8cb45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 0x55a5e8cb45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.404+0000 7f4bb93c2640 1 -- 172.21.15.105:0/3087116020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e8bcbc60 con 0x55a5e8cac5d0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.405+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cac9f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009730 tx=0x7f4ba4009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a37dbafe9914fe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.405+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ba00a2fc0 0x7f4ba00a5480 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0005fd0 tx=0x7f4bb000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.405+0000 7f4b9ffff640 1 -- 172.21.15.105:0/3087116020 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba401a070 con 0x55a5e8cac5d0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.405+0000 7f4b9ffff640 1 -- 172.21.15.105:0/3087116020 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba400fdf0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.405+0000 7f4b9ffff640 1 -- 172.21.15.105:0/3087116020 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4005620 con 0x55a5e8cac5d0 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087116020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 msgr2=0x55a5e8cac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cac9f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009730 tx=0x7f4ba4009cc0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087116020 shutdown_connections 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 0x55a5e8cb45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cb1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087116020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cac9f0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087116020 >> 172.21.15.105:0/3087116020 conn(0x55a5e8ca7b40 msgr2=0x55a5e8ca7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087116020 shutdown_connections 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.406+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087116020 wait complete. 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 1 -- start start 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 1 Processor -- start 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cb6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 0x55a5e8cbd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a5e8cb50e0 con 0x55a5e8cb2130 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5e8bccdb0 con 0x55a5e8cacfc0 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a5e8cb5260 con 0x55a5e8cac5d0 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bb9bc3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bba3c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 0x55a5e8cbd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb9bc3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51402/0 (socket says 172.21.15.105:51402) 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/1409417976 learned_addr learned my addr 172.21.15.105:0/1409417976 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cb6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.407+0000 7f4bbea15740 10 client.0 _ll_register_callbacks cb 0x55a5e8bd1110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/1409417976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 msgr2=0x55a5e8cb6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cb6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/1409417976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 msgr2=0x55a5e8cbd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 0x55a5e8cbd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/1409417976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e8925210 con 0x55a5e8cac5d0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb6430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000bfd0 tx=0x7f4bb0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e90109476210064f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4b9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/1409417976 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000edc0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bbea15740 1 -- 172.21.15.105:0/1409417976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ba400fdf0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/1409417976 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4bb0004020 con 0x55a5e8cac5d0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/1409417976 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4bb0004220 con 0x55a5e8cac5d0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bbea15740 10 client.30763 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.408+0000 7f4bbea15740 1 -- 172.21.15.105:0/1409417976 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a5e89f1620 con 0x55a5e8cac5d0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/1409417976 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba400fdf0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/1409417976 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4bb001e4b0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4bbea15740 10 client.30763 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4bbea15740 10 client.30763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4bbea15740 20 client.30763 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4bbea15740 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5e8cc2430 0x55a5e8cc48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4bbea15740 4 client.30763 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4bbea15740 1 -- 172.21.15.105:0/1409417976 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e8c1e5f0 con 0x55a5e8cc2430 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.409+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5e8cc2430 0x55a5e8cc48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.410+0000 7f4b9d7fa640 10 client.30763 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.410+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5e8cc2430 0x55a5e8cc48f0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0a5ffd3420e468 server_cookie=1a9c0d0b0bc8c320 in_seq=0 out_seq=0 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/1409417976 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a5e8c1e5f0 con 0x55a5e8cc2430 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4b9d7fa640 10 client.30763 handle_command_reply: tid=0 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 client.30763 shutdown 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 -- 172.21.15.105:0/1409417976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 msgr2=0x55a5e8cb6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb6430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000bfd0 tx=0x7f4bb0004730 comp rx=0 tx=0).stop 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 -- 172.21.15.105:0/1409417976 shutdown_connections 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5e8cc2430 0x55a5e8cc48f0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 0x55a5e8cbd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cb6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 --2- 172.21.15.105:0/1409417976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb6430 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4b9d7fa640 0 client.30763 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 -- 172.21.15.105:0/1409417976 >> 172.21.15.105:0/1409417976 conn(0x55a5e8ca7b40 msgr2=0x55a5e8ca6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 -- 172.21.15.105:0/1409417976 shutdown_connections 2024-03-26T17:53:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.412+0000 7f4bbea15740 1 -- 172.21.15.105:0/1409417976 wait complete. 2024-03-26T17:53:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.413+0000 7f4bbea15740 20 client.30763 trim_cache size 0 max 16384 2024-03-26T17:53:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.434+0000 7f4bbea15740 1 Processor -- start 2024-03-26T17:53:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bbea15740 1 -- start start 2024-03-26T17:53:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 0x55a5e8cab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bbea15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41aae80 con 0x55a5e8cb2130 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bbea15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41aca80 con 0x55a5e8cacfc0 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bbea15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41ab550 con 0x55a5e8cac5d0 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb9bc3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb9bc3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51414/0 (socket says 172.21.15.105:51414) 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/659059279 learned_addr learned my addr 172.21.15.105:0/659059279 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bba3c4640 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 0x55a5e8cab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/659059279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 msgr2=0x55a5e8cbd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/659059279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 msgr2=0x55a5e8cab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 0x55a5e8cab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.435+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/659059279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e8cb1ae0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.436+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb68d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00116c0 tx=0x7f4bb0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d8a7f60c948f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.436+0000 7f4b9f7fe640 1 -- 172.21.15.105:0/659059279 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb001edf0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.436+0000 7f4b9f7fe640 1 -- 172.21.15.105:0/659059279 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bb000edc0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.436+0000 7f4b9f7fe640 1 -- 172.21.15.105:0/659059279 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00047a0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 -- 172.21.15.105:0/659059279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 msgr2=0x55a5e8cb68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb68d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00116c0 tx=0x7f4bb0000bc0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 -- 172.21.15.105:0/659059279 shutdown_connections 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cb2130 0x55a5e8cab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 --2- 172.21.15.105:0/659059279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cb68d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 -- 172.21.15.105:0/659059279 >> 172.21.15.105:0/659059279 conn(0x55a5e8ca7b40 msgr2=0x55a5e8caada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 -- 172.21.15.105:0/659059279 shutdown_connections 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 -- 172.21.15.105:0/659059279 wait complete. 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 -- start start 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 Processor -- start 2024-03-26T17:53:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cbacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.437+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bbea15740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 0x55a5e8cbc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bbea15740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41ac3b0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bbea15740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41abc20 con 0x55a5e8cb2130 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bbea15740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41a8e60 con 0x55a5e8cacfc0 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bbea15740 10 client.0 _ll_register_callbacks cb 0x7f4bb000be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb93c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb9bc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cbacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb93c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51424/0 (socket says 172.21.15.105:51424) 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb93c2640 1 -- 172.21.15.105:0/3087462995 learned_addr learned my addr 172.21.15.105:0/3087462995 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bba3c4640 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 0x55a5e8cbc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/3087462995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 msgr2=0x55a5e8cbb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/3087462995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 msgr2=0x55a5e8cbc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 0x55a5e8cbc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb9bc3640 1 -- 172.21.15.105:0/3087462995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e8cbc830 con 0x55a5e8cac5d0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4bb9bc3640 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cbacf0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x55a5e8cb5cf0 tx=0x7f4bb001e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd8d22e483c47ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4b9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.438+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/3087462995 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0048020 con 0x55a5e8cac5d0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/3087462995 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4bb000ebc0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087462995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bb000edc0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/3087462995 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4bb0053050 con 0x55a5e8cac5d0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 10 client.21012 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087462995 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a5e8cbdb40 con 0x55a5e8cac5d0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/3087462995 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bb000edc0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/3087462995 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4bb004f4c0 con 0x55a5e8cac5d0 2024-03-26T17:53:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 10 client.21012 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 10 client.21012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 20 client.21012 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5e8d317b0 0x55a5e8d33bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 4 client.21012 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:53:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.439+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087462995 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5e8c18940 con 0x55a5e8d317b0 2024-03-26T17:53:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.440+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5e8d317b0 0x55a5e8d33bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.440+0000 7f4b9d7fa640 10 client.21012 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.440+0000 7f4bb93c2640 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5e8d317b0 0x55a5e8d33bd0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f7a36fd438fa7d server_cookie=1e5e8190a39f07b8 in_seq=0 out_seq=0 2024-03-26T17:53:25.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 17039 7.23 MB/sec execute 8 sec latency 150.653 ms 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.641+0000 7f4b9d7fa640 1 -- 172.21.15.105:0/3087462995 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5e8c18940 con 0x55a5e8d317b0 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.641+0000 7f4b9d7fa640 10 client.21012 handle_command_reply: tid=0 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.641+0000 7f4bbea15740 1 client.21012 shutdown 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.641+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087462995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 msgr2=0x55a5e8cbacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.641+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cbacf0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x55a5e8cb5cf0 tx=0x7f4bb001e880 comp rx=0 tx=0).stop 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087462995 shutdown_connections 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a5e8d317b0 0x55a5e8d33bd0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5e8cb2130 0x55a5e8cbc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4b9d7fa640 0 client.21012 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a5e8cacfc0 0x55a5e8cbb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 1 --2- 172.21.15.105:0/3087462995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a5e8cac5d0 0x55a5e8cbacf0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087462995 >> 172.21.15.105:0/3087462995 conn(0x55a5e8ca7b40 msgr2=0x55a5e8cab630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087462995 shutdown_connections 2024-03-26T17:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 1 -- 172.21.15.105:0/3087462995 wait complete. 2024-03-26T17:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbea15740 20 client.21012 trim_cache size 0 max 16384 2024-03-26T17:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:53:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbbe27640 1 -- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ba00a2fc0 msgr2=0x7f4ba00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ba00a2fc0 0x7f4ba00a5480 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0005fd0 tx=0x7f4bb000b4a0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbbe27640 1 -- 172.21.15.105:0/3711005929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb41034a0 msgr2=0x7f4bb40741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.642+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb41034a0 0x7f4bb40741f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4bac02c640 tx=0x7f4bac035d60 comp rx=0 tx=0).stop 2024-03-26T17:53:25.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.643+0000 7f4bbbe27640 1 -- 172.21.15.105:0/3711005929 shutdown_connections 2024-03-26T17:53:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.643+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4ba00a2fc0 0x7f4ba00a5480 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.643+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4bb41034a0 0x7f4bb40741f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.643+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb4102ae0 0x7f4bb4073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.643+0000 7f4bbbe27640 1 --2- 172.21.15.105:0/3711005929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4bb41018f0 0x7f4bb4075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.643+0000 7f4bbbe27640 1 -- 172.21.15.105:0/3711005929 >> 172.21.15.105:0/3711005929 conn(0x7f4bb40fce80 msgr2=0x7f4bb4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.643+0000 7f4bbbe27640 1 -- 172.21.15.105:0/3711005929 shutdown_connections 2024-03-26T17:53:25.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:25.643+0000 7f4bbbe27640 1 -- 172.21.15.105:0/3711005929 wait complete. 2024-03-26T17:53:26.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 17244 8.30 MB/sec execute 9 sec latency 222.795 ms 2024-03-26T17:53:27.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 17383 8.77 MB/sec execute 10 sec latency 125.777 ms 2024-03-26T17:53:28.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:27 smithi105 ceph-mon[32280]: pgmap v802: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-03-26T17:53:28.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:27 smithi135 ceph-mon[35636]: pgmap v802: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-03-26T17:53:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:27 smithi144 ceph-mon[36095]: pgmap v802: 97 pgs: 97 active+clean; 249 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-03-26T17:53:28.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 17545 8.96 MB/sec execute 11 sec latency 182.546 ms 2024-03-26T17:53:29.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 17660 8.64 MB/sec execute 12 sec latency 106.511 ms 2024-03-26T17:53:30.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:29 smithi105 ceph-mon[32280]: pgmap v803: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T17:53:30.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:29 smithi135 ceph-mon[35636]: pgmap v803: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T17:53:30.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:29 smithi144 ceph-mon[36095]: pgmap v803: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T17:53:30.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 17733 8.20 MB/sec execute 13 sec latency 119.750 ms 2024-03-26T17:53:31.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 17821 7.91 MB/sec execute 14 sec latency 117.166 ms 2024-03-26T17:53:32.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:31 smithi105 ceph-mon[32280]: pgmap v804: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 3.5 MiB/s wr, 268 op/s 2024-03-26T17:53:32.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:31 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:53:32.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:31 smithi135 ceph-mon[35636]: pgmap v804: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 3.5 MiB/s wr, 268 op/s 2024-03-26T17:53:32.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:31 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:53:32.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:31 smithi144 ceph-mon[36095]: pgmap v804: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 3.5 MiB/s wr, 268 op/s 2024-03-26T17:53:32.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:31 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:53:32.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 17942 8.26 MB/sec execute 15 sec latency 130.105 ms 2024-03-26T17:53:33.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:32 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:53:33.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:32 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:53:33.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:32 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:53:33.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:32 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:53:33.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:32 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:53:33.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:32 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:53:33.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:32 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:53:33.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:32 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:53:33.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:32 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:53:33.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 18075 8.10 MB/sec execute 16 sec latency 120.343 ms 2024-03-26T17:53:34.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:33 smithi105 ceph-mon[32280]: pgmap v805: 97 pgs: 97 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-03-26T17:53:34.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:33 smithi135 ceph-mon[35636]: pgmap v805: 97 pgs: 97 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-03-26T17:53:34.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:33 smithi144 ceph-mon[36095]: pgmap v805: 97 pgs: 97 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-03-26T17:53:34.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 18221 8.16 MB/sec execute 17 sec latency 151.890 ms 2024-03-26T17:53:35.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 18369 8.29 MB/sec execute 18 sec latency 134.152 ms 2024-03-26T17:53:36.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:35 smithi105 ceph-mon[32280]: pgmap v806: 97 pgs: 97 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-03-26T17:53:36.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:35 smithi135 ceph-mon[35636]: pgmap v806: 97 pgs: 97 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-03-26T17:53:36.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:35 smithi144 ceph-mon[36095]: pgmap v806: 97 pgs: 97 active+clean; 259 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 3.9 MiB/s wr, 231 op/s 2024-03-26T17:53:36.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 18479 8.13 MB/sec execute 19 sec latency 136.821 ms 2024-03-26T17:53:37.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 18632 8.32 MB/sec execute 20 sec latency 127.905 ms 2024-03-26T17:53:38.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:37 smithi105 ceph-mon[32280]: pgmap v807: 97 pgs: 97 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T17:53:38.147 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:37 smithi144 ceph-mon[36095]: pgmap v807: 97 pgs: 97 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T17:53:38.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:37 smithi135 ceph-mon[35636]: pgmap v807: 97 pgs: 97 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T17:53:38.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 18729 8.06 MB/sec execute 21 sec latency 134.617 ms 2024-03-26T17:53:38.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.766+0000 7f4747fff640 1 -- 172.21.15.105:0/2700375490 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740034d80 con 0x7f4750104c90 2024-03-26T17:53:38.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.766+0000 7f47577c4640 1 -- 172.21.15.105:0/2700375490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4750104c90 msgr2=0x7f4750107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.766+0000 7f47577c4640 1 --2- 172.21.15.105:0/2700375490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4750104c90 0x7f4750107080 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4740009860 tx=0x7f474002bb70 comp rx=0 tx=0).stop 2024-03-26T17:53:38.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.767+0000 7f47577c4640 1 -- 172.21.15.105:0/2700375490 shutdown_connections 2024-03-26T17:53:38.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.767+0000 7f47577c4640 1 --2- 172.21.15.105:0/2700375490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47501075c0 0x7f4750109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.767+0000 7f47577c4640 1 --2- 172.21.15.105:0/2700375490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4750104c90 0x7f4750107080 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.767+0000 7f47577c4640 1 --2- 172.21.15.105:0/2700375490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4750102360 0x7f4750104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.767+0000 7f47577c4640 1 -- 172.21.15.105:0/2700375490 >> 172.21.15.105:0/2700375490 conn(0x7f47500fbd50 msgr2=0x7f47500fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.767+0000 7f47577c4640 1 -- 172.21.15.105:0/2700375490 shutdown_connections 2024-03-26T17:53:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.767+0000 7f47577c4640 1 -- 172.21.15.105:0/2700375490 wait complete. 2024-03-26T17:53:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f47577c4640 1 Processor -- start 2024-03-26T17:53:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f47577c4640 1 -- start start 2024-03-26T17:53:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f47577c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4750102360 0x7f475019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f47577c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4750104c90 0x7f475019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f47577c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47501075c0 0x7f47501a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f47577c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f475010f670 con 0x7f4750104c90 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f47577c4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f475010f4f0 con 0x7f4750102360 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f47577c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f475010f7f0 con 0x7f47501075c0 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f4755560640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4750102360 0x7f475019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f4755560640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4750102360 0x7f475019e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36884/0 (socket says 172.21.15.105:36884) 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f4755560640 1 -- 172.21.15.105:0/314819900 learned_addr learned my addr 172.21.15.105:0/314819900 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f4754d5f640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4750104c90 0x7f475019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f4755d61640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47501075c0 0x7f47501a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.768+0000 7f4755560640 1 -- 172.21.15.105:0/314819900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47501075c0 msgr2=0x7f47501a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f4755560640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47501075c0 0x7f47501a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f4755560640 1 -- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4750104c90 msgr2=0x7f475019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f4755560640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4750104c90 0x7f475019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f4755560640 1 -- 172.21.15.105:0/314819900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47501a5d20 con 0x7f4750102360 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f4755d61640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47501075c0 0x7f47501a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f4755560640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4750102360 0x7f475019e100 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e9c0 tx=0x7f474c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f47467fc640 1 -- 172.21.15.105:0/314819900 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c00ccc0 con 0x7f4750102360 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f47467fc640 1 -- 172.21.15.105:0/314819900 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f474c0054d0 con 0x7f4750102360 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f47577c4640 1 -- 172.21.15.105:0/314819900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47501a5fb0 con 0x7f4750102360 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f47467fc640 1 -- 172.21.15.105:0/314819900 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c010430 con 0x7f4750102360 2024-03-26T17:53:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.769+0000 7f47577c4640 1 -- 172.21.15.105:0/314819900 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47501a6470 con 0x7f4750102360 2024-03-26T17:53:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.772+0000 7f47467fc640 1 -- 172.21.15.105:0/314819900 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f474c004ef0 con 0x7f4750102360 2024-03-26T17:53:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.772+0000 7f475a3ac740 1 Processor -- start 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.772+0000 7f475a3ac740 1 -- start start 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.772+0000 7f47467fc640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f473c0a2fc0 0x7f473c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d78169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d781bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f475a3ac740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5563d777b0b0 con 0x5563d78165d0 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f475a3ac740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5563d7723960 con 0x5563d781c130 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f475a3ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5563d7777060 con 0x5563d7816fc0 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4755d61640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f473c0a2fc0 0x7f473c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f47467fc640 1 -- 172.21.15.105:0/314819900 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f474c0cf9a0 con 0x7f4750102360 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d781bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4755560640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d78169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d781bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47756/0 (socket says 172.21.15.105:47756) 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 -- 172.21.15.105:0/1754825475 learned_addr learned my addr 172.21.15.105:0/1754825475 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4755d61640 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 -- 172.21.15.105:0/1754825475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 msgr2=0x5563d781e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 -- 172.21.15.105:0/1754825475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 msgr2=0x5563d78169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d78169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 -- 172.21.15.105:0/1754825475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d7735c60 con 0x5563d7816fc0 2024-03-26T17:53:38.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4755560640 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d78169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4755d61640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f473c0a2fc0 0x7f473c0a5480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4748005fd0 tx=0x7f474800fed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.773+0000 7f4754d5f640 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d781bac0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4750069aa0 tx=0x7f4740005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e706ff7d5ceb3c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.774+0000 7f473b7fe640 1 -- 172.21.15.105:0/1754825475 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740039040 con 0x5563d7816fc0 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.774+0000 7f473b7fe640 1 -- 172.21.15.105:0/1754825475 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4740004030 con 0x5563d7816fc0 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.774+0000 7f473b7fe640 1 -- 172.21.15.105:0/1754825475 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474002c9e0 con 0x5563d7816fc0 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 -- 172.21.15.105:0/1754825475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 msgr2=0x5563d781bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d781bac0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4750069aa0 tx=0x7f4740005b20 comp rx=0 tx=0).stop 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 -- 172.21.15.105:0/1754825475 shutdown_connections 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d781bac0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 --2- 172.21.15.105:0/1754825475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d78169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 -- 172.21.15.105:0/1754825475 >> 172.21.15.105:0/1754825475 conn(0x5563d7811b40 msgr2=0x5563d7811f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:38.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 -- 172.21.15.105:0/1754825475 shutdown_connections 2024-03-26T17:53:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 -- 172.21.15.105:0/1754825475 wait complete. 2024-03-26T17:53:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 -- start start 2024-03-26T17:53:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.775+0000 7f475a3ac740 1 Processor -- start 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d7820370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d78208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d78278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f475a3ac740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5563d781f080 con 0x5563d78165d0 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f475a3ac740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5563d7736db0 con 0x5563d781c130 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f475a3ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5563d781f200 con 0x5563d7816fc0 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f4754d5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d78208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f4755560640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d7820370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f475a3ac740 10 client.0 _ll_register_callbacks cb 0x5563d773b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f4754d5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d78208b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47762/0 (socket says 172.21.15.105:47762) 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f4754d5f640 1 -- 172.21.15.105:0/2505278611 learned_addr learned my addr 172.21.15.105:0/2505278611 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f4755d61640 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d78278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f4754d5f640 1 -- 172.21.15.105:0/2505278611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 msgr2=0x5563d78278f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f4754d5f640 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d78278f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.776+0000 7f4754d5f640 1 -- 172.21.15.105:0/2505278611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 msgr2=0x5563d7820370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.777+0000 7f4754d5f640 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d7820370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.777+0000 7f4754d5f640 1 -- 172.21.15.105:0/2505278611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d748f210 con 0x5563d7816fc0 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.777+0000 7f4754d5f640 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d78208b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4740000c00 tx=0x7f4740004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbc9bc09238c5720 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.777+0000 7f4738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.777+0000 7f4738ff9640 1 -- 172.21.15.105:0/2505278611 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740039040 con 0x5563d7816fc0 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.777+0000 7f475a3ac740 1 -- 172.21.15.105:0/2505278611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4740004030 con 0x5563d7816fc0 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.777+0000 7f4738ff9640 1 -- 172.21.15.105:0/2505278611 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4740034c80 con 0x5563d7816fc0 2024-03-26T17:53:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.777+0000 7f4738ff9640 1 -- 172.21.15.105:0/2505278611 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f474002c720 con 0x5563d7816fc0 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f475a3ac740 10 client.30781 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f475a3ac740 1 -- 172.21.15.105:0/2505278611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5563d755b620 con 0x5563d7816fc0 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f4738ff9640 1 -- 172.21.15.105:0/2505278611 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f474004b070 con 0x5563d7816fc0 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f4738ff9640 1 -- 172.21.15.105:0/2505278611 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f47400046c0 con 0x5563d7816fc0 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f475a3ac740 10 client.30781 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f475a3ac740 10 client.30781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f475a3ac740 20 client.30781 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563d782c390 0x5563d782e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f475a3ac740 4 client.30781 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f475a3ac740 1 -- 172.21.15.105:0/2505278611 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4740004030 con 0x5563d782c390 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.778+0000 7f4755560640 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563d782c390 0x5563d782e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.779+0000 7f4738ff9640 10 client.30781 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.779+0000 7f4755560640 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563d782c390 0x5563d782e7f0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0a99afe7b7d811 server_cookie=c31e8b9c084887ba in_seq=0 out_seq=0 2024-03-26T17:53:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f4738ff9640 1 -- 172.21.15.105:0/2505278611 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f4740004030 con 0x5563d782c390 2024-03-26T17:53:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f4738ff9640 10 client.30781 handle_command_reply: tid=0 2024-03-26T17:53:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 client.30781 shutdown 2024-03-26T17:53:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 -- 172.21.15.105:0/2505278611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 msgr2=0x5563d78208b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d78208b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4740000c00 tx=0x7f4740004570 comp rx=0 tx=0).stop 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 -- 172.21.15.105:0/2505278611 shutdown_connections 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563d782c390 0x5563d782e7f0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d78278f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d78208b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f4738ff9640 0 client.30781 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2505278611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d78165d0 0x5563d7820370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 -- 172.21.15.105:0/2505278611 >> 172.21.15.105:0/2505278611 conn(0x5563d7811b40 msgr2=0x5563d7810df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 -- 172.21.15.105:0/2505278611 shutdown_connections 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 1 -- 172.21.15.105:0/2505278611 wait complete. 2024-03-26T17:53:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.782+0000 7f475a3ac740 20 client.30781 trim_cache size 0 max 16384 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.805+0000 7f475a3ac740 1 Processor -- start 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f475a3ac740 1 -- start start 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d78165d0 0x5563d782e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d7816fc0 0x5563d78278b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f475a3ac740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47501af330 con 0x5563d7816fc0 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f475a3ac740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47501b0f30 con 0x5563d781c130 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4754d5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d7816fc0 0x5563d78278b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f475a3ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47501afa00 con 0x5563d78165d0 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4754d5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d7816fc0 0x5563d78278b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42430/0 (socket says 172.21.15.105:42430) 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4754d5f640 1 -- 172.21.15.105:0/1067142806 learned_addr learned my addr 172.21.15.105:0/1067142806 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4755560640 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d78165d0 0x5563d782e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4755d61640 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4754d5f640 1 -- 172.21.15.105:0/1067142806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d78165d0 msgr2=0x5563d782e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4754d5f640 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d78165d0 0x5563d782e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4754d5f640 1 -- 172.21.15.105:0/1067142806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 msgr2=0x5563d781b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4754d5f640 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4754d5f640 1 -- 172.21.15.105:0/1067142806 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d7821360 con 0x5563d7816fc0 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.806+0000 7f4755d61640 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.807+0000 7f4754d5f640 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d7816fc0 0x5563d78278b0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f474002c0b0 tx=0x7f4740005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a3450d3b89ec40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.807+0000 7f473affd640 1 -- 172.21.15.105:0/1067142806 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740005850 con 0x5563d7816fc0 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.807+0000 7f473affd640 1 -- 172.21.15.105:0/1067142806 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f474004b070 con 0x5563d7816fc0 2024-03-26T17:53:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.807+0000 7f473affd640 1 -- 172.21.15.105:0/1067142806 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740004d00 con 0x5563d7816fc0 2024-03-26T17:53:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 -- 172.21.15.105:0/1067142806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d7816fc0 msgr2=0x5563d78278b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d7816fc0 0x5563d78278b0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f474002c0b0 tx=0x7f4740005bd0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 -- 172.21.15.105:0/1067142806 shutdown_connections 2024-03-26T17:53:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d781c130 0x5563d781b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d7816fc0 0x5563d78278b0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 --2- 172.21.15.105:0/1067142806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d78165d0 0x5563d782e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 -- 172.21.15.105:0/1067142806 >> 172.21.15.105:0/1067142806 conn(0x5563d7811b40 msgr2=0x5563d7814da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 -- 172.21.15.105:0/1067142806 shutdown_connections 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 -- 172.21.15.105:0/1067142806 wait complete. 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 -- start start 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.808+0000 7f475a3ac740 1 Processor -- start 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d78165d0 0x5563d78248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d7824de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f475a3ac740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d781c130 0x5563d7825df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f475a3ac740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47501b0860 con 0x5563d781c130 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f475a3ac740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47501b00d0 con 0x5563d78165d0 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f475a3ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47501ad310 con 0x5563d7816fc0 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f475a3ac740 10 client.0 _ll_register_callbacks cb 0x7f474c098330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4755560640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d78165d0 0x5563d78248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4754d5f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d7824de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4755d61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d781c130 0x5563d7825df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4755560640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d78165d0 0x5563d78248a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36940/0 (socket says 172.21.15.105:36940) 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4755560640 1 -- 172.21.15.105:0/2147776825 learned_addr learned my addr 172.21.15.105:0/2147776825 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4754d5f640 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d7824de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47784/0 (socket says 172.21.15.105:47784) 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4754d5f640 1 -- 172.21.15.105:0/2147776825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d78165d0 msgr2=0x5563d78248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4754d5f640 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d78165d0 0x5563d78248a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4754d5f640 1 -- 172.21.15.105:0/2147776825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d781c130 msgr2=0x5563d7825df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4754d5f640 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d781c130 0x5563d7825df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4754d5f640 1 -- 172.21.15.105:0/2147776825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d78263e0 con 0x5563d7816fc0 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4755d61640 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d781c130 0x5563d7825df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4754d5f640 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d7824de0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4740005c40 tx=0x7f474002cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0eb9c7f7533e225 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.809+0000 7f4738ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f4738ff9640 1 -- 172.21.15.105:0/2147776825 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740005850 con 0x5563d7816fc0 2024-03-26T17:53:38.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f4738ff9640 1 -- 172.21.15.105:0/2147776825 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4740004f10 con 0x5563d7816fc0 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f475a3ac740 1 -- 172.21.15.105:0/2147776825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f474004b070 con 0x5563d7816fc0 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f4738ff9640 1 -- 172.21.15.105:0/2147776825 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4740005150 con 0x5563d7816fc0 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f475a3ac740 10 client.30787 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f475a3ac740 1 -- 172.21.15.105:0/2147776825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5563d78276f0 con 0x5563d7816fc0 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f4738ff9640 1 -- 172.21.15.105:0/2147776825 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f474004b070 con 0x5563d7816fc0 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f4738ff9640 1 -- 172.21.15.105:0/2147776825 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4740050920 con 0x5563d7816fc0 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f475a3ac740 10 client.30787 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f475a3ac740 10 client.30787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f475a3ac740 20 client.30787 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563d7892a00 0x5563d7894e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.810+0000 7f475a3ac740 4 client.30787 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.811+0000 7f475a3ac740 1 -- 172.21.15.105:0/2147776825 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563d77885f0 con 0x5563d7892a00 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.811+0000 7f4755560640 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563d7892a00 0x5563d7894e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.811+0000 7f4738ff9640 10 client.30787 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:38.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.811+0000 7f4755560640 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563d7892a00 0x5563d7894e20 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ef40365b068582 server_cookie=d0ff53f611c88dfa in_seq=0 out_seq=0 2024-03-26T17:53:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.959+0000 7f4738ff9640 1 -- 172.21.15.105:0/2147776825 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563d77885f0 con 0x5563d7892a00 2024-03-26T17:53:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.959+0000 7f4738ff9640 10 client.30787 handle_command_reply: tid=0 2024-03-26T17:53:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.959+0000 7f475a3ac740 1 client.30787 shutdown 2024-03-26T17:53:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.959+0000 7f475a3ac740 1 -- 172.21.15.105:0/2147776825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 msgr2=0x5563d7824de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.959+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d7824de0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4740005c40 tx=0x7f474002cb70 comp rx=0 tx=0).stop 2024-03-26T17:53:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.959+0000 7f475a3ac740 1 -- 172.21.15.105:0/2147776825 shutdown_connections 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.959+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563d7892a00 0x5563d7894e20 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.960+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563d781c130 0x5563d7825df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.960+0000 7f4738ff9640 0 client.30787 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.960+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563d7816fc0 0x5563d7824de0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.960+0000 7f475a3ac740 1 --2- 172.21.15.105:0/2147776825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563d78165d0 0x5563d78248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.960+0000 7f475a3ac740 1 -- 172.21.15.105:0/2147776825 >> 172.21.15.105:0/2147776825 conn(0x5563d7811b40 msgr2=0x5563d781b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.960+0000 7f475a3ac740 1 -- 172.21.15.105:0/2147776825 shutdown_connections 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.960+0000 7f475a3ac740 1 -- 172.21.15.105:0/2147776825 wait complete. 2024-03-26T17:53:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.960+0000 7f475a3ac740 20 client.30787 trim_cache size 0 max 16384 2024-03-26T17:53:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:53:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:53:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:53:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:53:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 -- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f473c0a2fc0 msgr2=0x7f473c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f473c0a2fc0 0x7f473c0a5480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4748005fd0 tx=0x7f474800fed0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 -- 172.21.15.105:0/314819900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4750102360 msgr2=0x7f475019e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4750102360 0x7f475019e100 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e9c0 tx=0x7f474c00edc0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 -- 172.21.15.105:0/314819900 shutdown_connections 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f473c0a2fc0 0x7f473c0a5480 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47501075c0 0x7f47501a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4750104c90 0x7f475019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 --2- 172.21.15.105:0/314819900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4750102360 0x7f475019e100 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 -- 172.21.15.105:0/314819900 >> 172.21.15.105:0/314819900 conn(0x7f47500fbd50 msgr2=0x7f4750102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 -- 172.21.15.105:0/314819900 shutdown_connections 2024-03-26T17:53:38.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:38.961+0000 7f47577c4640 1 -- 172.21.15.105:0/314819900 wait complete. 2024-03-26T17:53:39.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 18815 7.91 MB/sec execute 22 sec latency 238.526 ms 2024-03-26T17:53:40.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:39 smithi105 ceph-mon[32280]: pgmap v808: 97 pgs: 97 active+clean; 266 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 4.2 MiB/s wr, 195 op/s 2024-03-26T17:53:40.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:39 smithi135 ceph-mon[35636]: pgmap v808: 97 pgs: 97 active+clean; 266 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 4.2 MiB/s wr, 195 op/s 2024-03-26T17:53:40.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:39 smithi144 ceph-mon[36095]: pgmap v808: 97 pgs: 97 active+clean; 266 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 4.2 MiB/s wr, 195 op/s 2024-03-26T17:53:40.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 18901 7.64 MB/sec execute 23 sec latency 166.305 ms 2024-03-26T17:53:41.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 19014 7.50 MB/sec execute 24 sec latency 138.692 ms 2024-03-26T17:53:42.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:41 smithi105 ceph-mon[32280]: pgmap v809: 97 pgs: 97 active+clean; 267 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 3.5 MiB/s wr, 163 op/s 2024-03-26T17:53:42.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:41 smithi135 ceph-mon[35636]: pgmap v809: 97 pgs: 97 active+clean; 267 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 3.5 MiB/s wr, 163 op/s 2024-03-26T17:53:42.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:41 smithi144 ceph-mon[36095]: pgmap v809: 97 pgs: 97 active+clean; 267 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 3.5 MiB/s wr, 163 op/s 2024-03-26T17:53:42.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 19133 7.32 MB/sec execute 25 sec latency 144.254 ms 2024-03-26T17:53:43.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 19252 7.16 MB/sec execute 26 sec latency 131.399 ms 2024-03-26T17:53:44.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:43 smithi105 ceph-mon[32280]: pgmap v810: 97 pgs: 97 active+clean; 266 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-03-26T17:53:44.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:43 smithi135 ceph-mon[35636]: pgmap v810: 97 pgs: 97 active+clean; 266 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-03-26T17:53:44.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:43 smithi144 ceph-mon[36095]: pgmap v810: 97 pgs: 97 active+clean; 266 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.6 MiB/s wr, 200 op/s 2024-03-26T17:53:44.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 19384 7.19 MB/sec execute 27 sec latency 133.282 ms 2024-03-26T17:53:45.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 19505 7.03 MB/sec execute 28 sec latency 130.089 ms 2024-03-26T17:53:46.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:45 smithi105 ceph-mon[32280]: pgmap v811: 97 pgs: 97 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:53:46.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:45 smithi135 ceph-mon[35636]: pgmap v811: 97 pgs: 97 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:53:46.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:45 smithi144 ceph-mon[36095]: pgmap v811: 97 pgs: 97 active+clean; 260 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:53:46.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 19613 6.90 MB/sec execute 29 sec latency 204.218 ms 2024-03-26T17:53:47.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 19751 6.98 MB/sec execute 30 sec latency 208.021 ms 2024-03-26T17:53:48.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:47 smithi105 ceph-mon[32280]: pgmap v812: 97 pgs: 97 active+clean; 255 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-03-26T17:53:48.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:47 smithi135 ceph-mon[35636]: pgmap v812: 97 pgs: 97 active+clean; 255 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-03-26T17:53:48.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:47 smithi144 ceph-mon[36095]: pgmap v812: 97 pgs: 97 active+clean; 255 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-03-26T17:53:48.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 19897 7.07 MB/sec execute 31 sec latency 130.443 ms 2024-03-26T17:53:49.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 20067 7.25 MB/sec execute 32 sec latency 127.869 ms 2024-03-26T17:53:50.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:49 smithi105 ceph-mon[32280]: pgmap v813: 97 pgs: 97 active+clean; 251 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-03-26T17:53:50.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:49 smithi135 ceph-mon[35636]: pgmap v813: 97 pgs: 97 active+clean; 251 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-03-26T17:53:50.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:49 smithi144 ceph-mon[36095]: pgmap v813: 97 pgs: 97 active+clean; 251 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 3.5 MiB/s wr, 275 op/s 2024-03-26T17:53:50.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 20200 7.31 MB/sec execute 33 sec latency 123.516 ms 2024-03-26T17:53:50.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:53:50.592 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:53:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 -- 172.21.15.105:0/3949670103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a41018d0 msgr2=0x7f77a4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 --2- 172.21.15.105:0/3949670103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a41018d0 0x7f77a4101cf0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7798009900 tx=0x7f779802bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 -- 172.21.15.105:0/3949670103 shutdown_connections 2024-03-26T17:53:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 --2- 172.21.15.105:0/3949670103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4103480 0x7f77a4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 --2- 172.21.15.105:0/3949670103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a4102ac0 0x7f77a4102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 --2- 172.21.15.105:0/3949670103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a41018d0 0x7f77a4101cf0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 -- 172.21.15.105:0/3949670103 >> 172.21.15.105:0/3949670103 conn(0x7f77a40fce20 msgr2=0x7f77a40ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:50.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 -- 172.21.15.105:0/3949670103 shutdown_connections 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.759+0000 7f77aab3b640 1 -- 172.21.15.105:0/3949670103 wait complete. 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.760+0000 7f77aab3b640 1 Processor -- start 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.760+0000 7f77aab3b640 1 -- start start 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.760+0000 7f77aab3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a41018d0 0x7f77a4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.760+0000 7f77aab3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a4102ac0 0x7f77a41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.760+0000 7f77aab3b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4103480 0x7f77a419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.760+0000 7f77aab3b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d420 con 0x7f77a4102ac0 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.760+0000 7f77aab3b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d2a0 con 0x7f77a4103480 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.760+0000 7f77aab3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410d5a0 con 0x7f77a41018d0 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.761+0000 7f77a88d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a41018d0 0x7f77a4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.761+0000 7f77a90d8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4103480 0x7f77a419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.761+0000 7f77a90d8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4103480 0x7f77a419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36278/0 (socket says 172.21.15.105:36278) 2024-03-26T17:53:50.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.761+0000 7f77a90d8640 1 -- 172.21.15.105:0/1544005921 learned_addr learned my addr 172.21.15.105:0/1544005921 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.761+0000 7f77a88d7640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a41018d0 0x7f77a4197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51398/0 (socket says 172.21.15.105:51398) 2024-03-26T17:53:50.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.761+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a4102ac0 0x7f77a41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.761+0000 7f77a3fff640 1 -- 172.21.15.105:0/1544005921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a41018d0 msgr2=0x7f77a4197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.761+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a41018d0 0x7f77a4197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a3fff640 1 -- 172.21.15.105:0/1544005921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4103480 msgr2=0x7f77a419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4103480 0x7f77a419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a3fff640 1 -- 172.21.15.105:0/1544005921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a4073bf0 con 0x7f77a4102ac0 2024-03-26T17:53:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a88d7640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a41018d0 0x7f77a4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a90d8640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4103480 0x7f77a419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:50.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a4102ac0 0x7f77a41980d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f779000d6d0 tx=0x7f779000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a1ffb640 1 -- 172.21.15.105:0/1544005921 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790017070 con 0x7f77a4102ac0 2024-03-26T17:53:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a1ffb640 1 -- 172.21.15.105:0/1544005921 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7790009820 con 0x7f77a4102ac0 2024-03-26T17:53:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77a1ffb640 1 -- 172.21.15.105:0/1544005921 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790004c40 con 0x7f77a4102ac0 2024-03-26T17:53:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.762+0000 7f77aab3b640 1 -- 172.21.15.105:0/1544005921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a4073ee0 con 0x7f77a4102ac0 2024-03-26T17:53:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.763+0000 7f77aab3b640 1 -- 172.21.15.105:0/1544005921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a4074480 con 0x7f77a4102ac0 2024-03-26T17:53:50.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.766+0000 7f77a1ffb640 1 -- 172.21.15.105:0/1544005921 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f77900099d0 con 0x7f77a4102ac0 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.766+0000 7f77ad808740 1 Processor -- start 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77ad808740 1 -- start start 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77a1ffb640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77940a2fa0 0x7f77940a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77ad808740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc35d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77ad808740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77ad808740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc3b450 0x55e37cc3d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77ad808740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e37cb9a980 con 0x55e37cc358f0 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77ad808740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e37cb42be0 con 0x55e37cc3b450 2024-03-26T17:53:50.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77ad808740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e37cb96930 con 0x55e37cc362e0 2024-03-26T17:53:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77a1ffb640 1 -- 172.21.15.105:0/1544005921 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f77900cede0 con 0x7f77a4102ac0 2024-03-26T17:53:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77a88d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc35d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77a90d8640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77940a2fa0 0x7f77940a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77a88d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc35d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48326/0 (socket says 172.21.15.105:48326) 2024-03-26T17:53:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77a88d7640 1 -- 172.21.15.105:0/1724112532 learned_addr learned my addr 172.21.15.105:0/1724112532 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.767+0000 7f77a90d8640 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc3b450 0x55e37cc3d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f77a3fff640 1 -- 172.21.15.105:0/1724112532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc3b450 msgr2=0x55e37cc3d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:50.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc3b450 0x55e37cc3d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f77a3fff640 1 -- 172.21.15.105:0/1724112532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 msgr2=0x55e37cc35d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:50.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc35d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f77a3fff640 1 -- 172.21.15.105:0/1724112532 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37c9787d0 con 0x55e37cc362e0 2024-03-26T17:53:50.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f77a88d7640 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc35d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:50.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f77a90d8640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77940a2fa0 0x7f77940a5460 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f779c009fd0 tx=0x7f779c0123f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3ade0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7790009610 tx=0x7f7790090cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc6a47161956358 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f778effd640 1 -- 172.21.15.105:0/1724112532 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790020040 con 0x55e37cc362e0 2024-03-26T17:53:50.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f778effd640 1 -- 172.21.15.105:0/1724112532 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779009b580 con 0x55e37cc362e0 2024-03-26T17:53:50.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.768+0000 7f778effd640 1 -- 172.21.15.105:0/1724112532 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790097bf0 con 0x55e37cc362e0 2024-03-26T17:53:50.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 -- 172.21.15.105:0/1724112532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 msgr2=0x55e37cc3ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3ade0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7790009610 tx=0x7f7790090cc0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 -- 172.21.15.105:0/1724112532 shutdown_connections 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc3b450 0x55e37cc3d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3ade0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 --2- 172.21.15.105:0/1724112532 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc35d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 -- 172.21.15.105:0/1724112532 >> 172.21.15.105:0/1724112532 conn(0x55e37cc30e60 msgr2=0x55e37cc31260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 -- 172.21.15.105:0/1724112532 shutdown_connections 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.770+0000 7f77ad808740 1 -- 172.21.15.105:0/1724112532 wait complete. 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.771+0000 7f77ad808740 1 -- start start 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.771+0000 7f77ad808740 1 Processor -- start 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.771+0000 7f77ad808740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc3f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.771+0000 7f77ad808740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.771+0000 7f77ad808740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 0x55e37cc46eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.771+0000 7f77ad808740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e37cc3e3a0 con 0x55e37cc3b450 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.771+0000 7f77ad808740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e37cb56160 con 0x55e37cc358f0 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.771+0000 7f77ad808740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e37cc3e520 con 0x55e37cc362e0 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77ad808740 10 client.0 _ll_register_callbacks cb 0x55e37cb92720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51406/0 (socket says 172.21.15.105:51406) 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 -- 172.21.15.105:0/1438452506 learned_addr learned my addr 172.21.15.105:0/1438452506 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a88d7640 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc3f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a90d8640 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 0x55e37cc46eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 -- 172.21.15.105:0/1438452506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 msgr2=0x55e37cc3f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc3f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 -- 172.21.15.105:0/1438452506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 msgr2=0x55e37cc46eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 0x55e37cc46eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 -- 172.21.15.105:0/1438452506 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37c8ae3f0 con 0x55e37cc362e0 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.772+0000 7f77a3fff640 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3fe70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f77900cc8f0 tx=0x7f7790090750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e1802a83c4ec7cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f7783fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f7783fff640 1 -- 172.21.15.105:0/1438452506 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77900b4060 con 0x55e37cc362e0 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f7783fff640 1 -- 172.21.15.105:0/1438452506 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f779009abd0 con 0x55e37cc362e0 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 1 -- 172.21.15.105:0/1438452506 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779009b580 con 0x55e37cc362e0 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f7783fff640 1 -- 172.21.15.105:0/1438452506 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7790020070 con 0x55e37cc362e0 2024-03-26T17:53:50.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 10 client.30799 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 1 -- 172.21.15.105:0/1438452506 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e37c86e0f0 con 0x55e37cc362e0 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f7783fff640 1 -- 172.21.15.105:0/1438452506 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779009b580 con 0x55e37cc362e0 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f7783fff640 1 -- 172.21.15.105:0/1438452506 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f77900972d0 con 0x55e37cc362e0 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 10 client.30799 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 10 client.30799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 20 client.30799 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e37cc4b3a0 0x55e37cc4d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 4 client.30799 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.773+0000 7f77ad808740 1 -- 172.21.15.105:0/1438452506 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e37cba7ec0 con 0x55e37cc4b3a0 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.774+0000 7f77a88d7640 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e37cc4b3a0 0x55e37cc4d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.774+0000 7f77a88d7640 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e37cc4b3a0 0x55e37cc4d790 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87114caee5400dad server_cookie=7b333a00537ecc43 in_seq=0 out_seq=0 2024-03-26T17:53:50.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.774+0000 7f7783fff640 10 client.30799 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.776+0000 7f7783fff640 1 -- 172.21.15.105:0/1438452506 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e37cba7ec0 con 0x55e37cc4b3a0 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.776+0000 7f7783fff640 10 client.30799 handle_command_reply: tid=0 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.776+0000 7f77ad808740 1 client.30799 shutdown 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 -- 172.21.15.105:0/1438452506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 msgr2=0x55e37cc3fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3fe70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f77900cc8f0 tx=0x7f7790090750 comp rx=0 tx=0).stop 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 -- 172.21.15.105:0/1438452506 shutdown_connections 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e37cc4b3a0 0x55e37cc4d790 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 0x55e37cc46eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f7783fff640 0 client.30799 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc3fe70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 --2- 172.21.15.105:0/1438452506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc3f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 -- 172.21.15.105:0/1438452506 >> 172.21.15.105:0/1438452506 conn(0x55e37cc30e60 msgr2=0x55e37cc30080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:50.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 -- 172.21.15.105:0/1438452506 shutdown_connections 2024-03-26T17:53:50.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 1 -- 172.21.15.105:0/1438452506 wait complete. 2024-03-26T17:53:50.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.777+0000 7f77ad808740 20 client.30799 trim_cache size 0 max 16384 2024-03-26T17:53:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.805+0000 7f77ad808740 1 Processor -- start 2024-03-26T17:53:50.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.805+0000 7f77ad808740 1 -- start start 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.805+0000 7f77ad808740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc2c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.805+0000 7f77ad808740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc362e0 0x55e37cc46e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.805+0000 7f77ad808740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc3b450 0x55e37cc2ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77ad808740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41aaf90 con 0x55e37cc358f0 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77ad808740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41acb90 con 0x55e37cc362e0 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77ad808740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41ab660 con 0x55e37cc3b450 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc2c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc2c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48336/0 (socket says 172.21.15.105:48336) 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 -- 172.21.15.105:0/3655725294 learned_addr learned my addr 172.21.15.105:0/3655725294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a3fff640 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc362e0 0x55e37cc46e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a90d8640 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc3b450 0x55e37cc2ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 -- 172.21.15.105:0/3655725294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc3b450 msgr2=0x55e37cc2ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc3b450 0x55e37cc2ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 -- 172.21.15.105:0/3655725294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc362e0 msgr2=0x55e37cc46e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc362e0 0x55e37cc46e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 -- 172.21.15.105:0/3655725294 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37cc366c0 con 0x55e37cc358f0 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f77a88d7640 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc2c050 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f7798006010 tx=0x7f779802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7095f0e3c59e09a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f778e7fc640 1 -- 172.21.15.105:0/3655725294 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798004320 con 0x55e37cc358f0 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f778e7fc640 1 -- 172.21.15.105:0/3655725294 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779804d070 con 0x55e37cc358f0 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.806+0000 7f778e7fc640 1 -- 172.21.15.105:0/3655725294 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798005040 con 0x55e37cc358f0 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 -- 172.21.15.105:0/3655725294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 msgr2=0x55e37cc2c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc2c050 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f7798006010 tx=0x7f779802c7e0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 -- 172.21.15.105:0/3655725294 shutdown_connections 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc3b450 0x55e37cc2ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc362e0 0x55e37cc46e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 --2- 172.21.15.105:0/3655725294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc358f0 0x55e37cc2c050 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 -- 172.21.15.105:0/3655725294 >> 172.21.15.105:0/3655725294 conn(0x55e37cc30e60 msgr2=0x55e37cc2f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 -- 172.21.15.105:0/3655725294 shutdown_connections 2024-03-26T17:53:50.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.807+0000 7f77ad808740 1 -- 172.21.15.105:0/3655725294 wait complete. 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77ad808740 1 -- start start 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77ad808740 1 Processor -- start 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77ad808740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc44e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77ad808740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc45340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77ad808740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 0x55e37cc462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77ad808740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41ac4c0 con 0x55e37cc3b450 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77ad808740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41abd30 con 0x55e37cc358f0 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77ad808740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41a8f70 con 0x55e37cc362e0 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.808+0000 7f77a88d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc44e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc45340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a88d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc44e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36294/0 (socket says 172.21.15.105:36294) 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc45340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51422/0 (socket says 172.21.15.105:51422) 2024-03-26T17:53:50.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a88d7640 1 -- 172.21.15.105:0/913897536 learned_addr learned my addr 172.21.15.105:0/913897536 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77ad808740 10 client.0 _ll_register_callbacks cb 0x7f779802c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a90d8640 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 0x55e37cc462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a88d7640 1 -- 172.21.15.105:0/913897536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 msgr2=0x55e37cc45340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a88d7640 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc45340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a88d7640 1 -- 172.21.15.105:0/913897536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 msgr2=0x55e37cc462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a88d7640 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 0x55e37cc462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a88d7640 1 -- 172.21.15.105:0/913897536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37cc468d0 con 0x55e37cc358f0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a3fff640 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc45340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77a88d7640 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc44e00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7798036040 tx=0x7f7798004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85651363485cba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f7783fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f7783fff640 1 -- 172.21.15.105:0/913897536 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798005250 con 0x55e37cc358f0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f7783fff640 1 -- 172.21.15.105:0/913897536 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f77980053f0 con 0x55e37cc358f0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.809+0000 7f77ad808740 1 -- 172.21.15.105:0/913897536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779804d070 con 0x55e37cc358f0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f7783fff640 1 -- 172.21.15.105:0/913897536 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77980055f0 con 0x55e37cc358f0 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77ad808740 10 client.30833 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:50.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77ad808740 1 -- 172.21.15.105:0/913897536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e37ccb1e50 con 0x55e37cc358f0 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f7783fff640 1 -- 172.21.15.105:0/913897536 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7798048cf0 con 0x55e37cc358f0 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f7783fff640 1 -- 172.21.15.105:0/913897536 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7798059030 con 0x55e37cc358f0 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77ad808740 10 client.30833 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77ad808740 10 client.30833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77ad808740 20 client.30833 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77ad808740 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e37ccb28e0 0x55e37ccb4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77ad808740 4 client.30833 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77ad808740 1 -- 172.21.15.105:0/913897536 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f779804d070 con 0x55e37ccb28e0 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.810+0000 7f77a3fff640 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e37ccb28e0 0x55e37ccb4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.811+0000 7f7783fff640 10 client.30833 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:50.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.811+0000 7f77a3fff640 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e37ccb28e0 0x55e37ccb4d00 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a5b4cec36e94db server_cookie=4ea265a5d4f6ab3e in_seq=0 out_seq=0 2024-03-26T17:53:50.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f7783fff640 1 -- 172.21.15.105:0/913897536 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f779804d070 con 0x55e37ccb28e0 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f7783fff640 10 client.30833 handle_command_reply: tid=0 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 client.30833 shutdown 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 -- 172.21.15.105:0/913897536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 msgr2=0x55e37cc44e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc44e00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7798036040 tx=0x7f7798004320 comp rx=0 tx=0).stop 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 -- 172.21.15.105:0/913897536 shutdown_connections 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e37ccb28e0 0x55e37ccb4d00 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e37cc3b450 0x55e37cc462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f7783fff640 0 client.30833 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e37cc362e0 0x55e37cc45340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 --2- 172.21.15.105:0/913897536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e37cc358f0 0x55e37cc44e00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 -- 172.21.15.105:0/913897536 >> 172.21.15.105:0/913897536 conn(0x55e37cc30e60 msgr2=0x55e37cc30080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:50.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 -- 172.21.15.105:0/913897536 shutdown_connections 2024-03-26T17:53:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 1 -- 172.21.15.105:0/913897536 wait complete. 2024-03-26T17:53:50.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.812+0000 7f77ad808740 20 client.30833 trim_cache size 0 max 16384 2024-03-26T17:53:50.807 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:53:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:53:50.807 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:53:50.807 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 -- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77940a2fa0 msgr2=0x7f77940a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77940a2fa0 0x7f77940a5460 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f779c009fd0 tx=0x7f779c0123f0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 -- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a4102ac0 msgr2=0x7f77a41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a4102ac0 0x7f77a41980d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f779000d6d0 tx=0x7f779000db90 comp rx=0 tx=0).stop 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 -- 172.21.15.105:0/1544005921 shutdown_connections 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f77940a2fa0 0x7f77940a5460 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77a4103480 0x7f77a419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77a4102ac0 0x7f77a41980d0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 --2- 172.21.15.105:0/1544005921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77a41018d0 0x7f77a4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 -- 172.21.15.105:0/1544005921 >> 172.21.15.105:0/1544005921 conn(0x7f77a40fce20 msgr2=0x7f77a4105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:50.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.813+0000 7f77aab3b640 1 -- 172.21.15.105:0/1544005921 shutdown_connections 2024-03-26T17:53:50.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.814+0000 7f77aab3b640 1 -- 172.21.15.105:0/1544005921 wait complete. 2024-03-26T17:53:50.816 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:53:50.816 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:53:50.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3194955786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81019d0 msgr2=0x7fc6d8101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3194955786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81019d0 0x7fc6d8101e50 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009900 tx=0x7fc6c802bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3194955786 shutdown_connections 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3194955786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d8102390 0x7fc6d8108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3194955786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81019d0 0x7fc6d8101e50 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3194955786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6d81007e0 0x7fc6d8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3194955786 >> 172.21.15.105:0/3194955786 conn(0x7fc6d80fbd50 msgr2=0x7fc6d80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3194955786 shutdown_connections 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.974+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3194955786 wait complete. 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dffa9640 1 Processor -- start 2024-03-26T17:53:50.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dffa9640 1 -- start start 2024-03-26T17:53:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dffa9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81007e0 0x7fc6d8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dffa9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6d81019d0 0x7fc6d8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dffa9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d8102390 0x7fc6d819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dffa9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d810c340 con 0x7fc6d81007e0 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dffa9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d810c1c0 con 0x7fc6d81019d0 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dffa9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d810c4c0 con 0x7fc6d8102390 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6ddd45640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81007e0 0x7fc6d8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6ddd45640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81007e0 0x7fc6d8197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48368/0 (socket says 172.21.15.105:48368) 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6de546640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d8102390 0x7fc6d819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6ddd45640 1 -- 172.21.15.105:0/3748080086 learned_addr learned my addr 172.21.15.105:0/3748080086 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.975+0000 7fc6dd544640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6d81019d0 0x7fc6d8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6dd544640 1 -- 172.21.15.105:0/3748080086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d8102390 msgr2=0x7fc6d819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6dd544640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d8102390 0x7fc6d819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6dd544640 1 -- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81007e0 msgr2=0x7fc6d8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6dd544640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81007e0 0x7fc6d8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6dd544640 1 -- 172.21.15.105:0/3748080086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d819f760 con 0x7fc6d81019d0 2024-03-26T17:53:50.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6ddd45640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81007e0 0x7fc6d8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6dd544640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6d81019d0 0x7fc6d8198080 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009850 tx=0x7fc6c8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6ceffd640 1 -- 172.21.15.105:0/3748080086 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8043020 con 0x7fc6d81019d0 2024-03-26T17:53:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d819f9f0 con 0x7fc6d81019d0 2024-03-26T17:53:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6ceffd640 1 -- 172.21.15.105:0/3748080086 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6c80043c0 con 0x7fc6d81019d0 2024-03-26T17:53:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6ceffd640 1 -- 172.21.15.105:0/3748080086 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8003a80 con 0x7fc6d81019d0 2024-03-26T17:53:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.976+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d819ff60 con 0x7fc6d81019d0 2024-03-26T17:53:50.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.978+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d81a0130 con 0x7fc6d81019d0 2024-03-26T17:53:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.979+0000 7fc6ceffd640 1 -- 172.21.15.105:0/3748080086 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc6c8046020 con 0x7fc6d81019d0 2024-03-26T17:53:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.979+0000 7fc6ceffd640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6c40a2ff0 0x7fc6c40a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.980+0000 7fc6ddd45640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6c40a2ff0 0x7fc6c40a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.980+0000 7fc6ceffd640 1 -- 172.21.15.105:0/3748080086 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc6c80f1550 con 0x7fc6d81019d0 2024-03-26T17:53:50.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.980+0000 7fc6ddd45640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6c40a2ff0 0x7fc6c40a54b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40095f0 tx=0x7fc6d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:50.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:50.984+0000 7fc6ceffd640 1 -- 172.21.15.105:0/3748080086 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6c804d020 con 0x7fc6d81019d0 2024-03-26T17:53:51.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.359+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6d81a03a0 con 0x7fc6d81019d0 2024-03-26T17:53:51.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.360+0000 7fc6ceffd640 1 -- 172.21.15.105:0/3748080086 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11948 (secure 0 0 0) 0x7fc6c80be340 con 0x7fc6d81019d0 2024-03-26T17:53:51.350 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:53:51.350 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:51:26.802032+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24482,24521,14628,24466,24487]},"id":1}]} 2024-03-26T17:53:51.350 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 34 2024-03-26T17:53:51.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6c40a2ff0 msgr2=0x7fc6c40a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6c40a2ff0 0x7fc6c40a54b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40095f0 tx=0x7fc6d4005f90 comp rx=0 tx=0).stop 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6d81019d0 msgr2=0x7fc6d8198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6d81019d0 0x7fc6d8198080 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009850 tx=0x7fc6c8004040 comp rx=0 tx=0).stop 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 shutdown_connections 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6c40a2ff0 0x7fc6c40a54b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6d8102390 0x7fc6d819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6d81019d0 0x7fc6d8198080 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 --2- 172.21.15.105:0/3748080086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6d81007e0 0x7fc6d8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.363+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 >> 172.21.15.105:0/3748080086 conn(0x7fc6d80fbd50 msgr2=0x7fc6d8068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.364+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 shutdown_connections 2024-03-26T17:53:51.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.364+0000 7fc6dffa9640 1 -- 172.21.15.105:0/3748080086 wait complete. 2024-03-26T17:53:51.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:53:51.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 -- 172.21.15.105:0/1217734339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc70104c70 msgr2=0x7efc70107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 --2- 172.21.15.105:0/1217734339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc70104c70 0x7efc70107060 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efc64009860 tx=0x7efc6402bb90 comp rx=0 tx=0).stop 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 -- 172.21.15.105:0/1217734339 shutdown_connections 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 --2- 172.21.15.105:0/1217734339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc701075a0 0x7efc701099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 --2- 172.21.15.105:0/1217734339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc70104c70 0x7efc70107060 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 --2- 172.21.15.105:0/1217734339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc70102340 0x7efc70104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 -- 172.21.15.105:0/1217734339 >> 172.21.15.105:0/1217734339 conn(0x7efc700fbd50 msgr2=0x7efc700fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 -- 172.21.15.105:0/1217734339 shutdown_connections 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.525+0000 7efc7555b640 1 -- 172.21.15.105:0/1217734339 wait complete. 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.526+0000 7efc7555b640 1 Processor -- start 2024-03-26T17:53:51.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.526+0000 7efc7555b640 1 -- start start 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc7555b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70102340 0x7efc7019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc7555b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc70104c70 0x7efc7019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc7555b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc701075a0 0x7efc701a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc7555b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc70101d00 con 0x7efc70104c70 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70102340 0x7efc7019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70102340 0x7efc7019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36328/0 (socket says 172.21.15.105:36328) 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6effd640 1 -- 172.21.15.105:0/2173248291 learned_addr learned my addr 172.21.15.105:0/2173248291 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc70104c70 0x7efc7019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc7555b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc70101b80 con 0x7efc70102340 2024-03-26T17:53:51.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6f7fe640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc701075a0 0x7efc701a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc70101e80 con 0x7efc701075a0 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6f7fe640 1 -- 172.21.15.105:0/2173248291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70102340 msgr2=0x7efc7019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6f7fe640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70102340 0x7efc7019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6f7fe640 1 -- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc70104c70 msgr2=0x7efc7019e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6f7fe640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc70104c70 0x7efc7019e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6f7fe640 1 -- 172.21.15.105:0/2173248291 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc701a5ba0 con 0x7efc701075a0 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.527+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc70104c70 0x7efc7019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.528+0000 7efc6f7fe640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc701075a0 0x7efc701a5530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efc68009fd0 tx=0x7efc6800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.528+0000 7efc57fff640 1 -- 172.21.15.105:0/2173248291 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6800ce20 con 0x7efc701075a0 2024-03-26T17:53:51.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.528+0000 7efc57fff640 1 -- 172.21.15.105:0/2173248291 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc68005020 con 0x7efc701075a0 2024-03-26T17:53:51.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.528+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc701a5e90 con 0x7efc701075a0 2024-03-26T17:53:51.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.528+0000 7efc57fff640 1 -- 172.21.15.105:0/2173248291 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68010690 con 0x7efc701075a0 2024-03-26T17:53:51.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.528+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701a6350 con 0x7efc701075a0 2024-03-26T17:53:51.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc78226740 1 Processor -- start 2024-03-26T17:53:51.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc57fff640 1 -- 172.21.15.105:0/2173248291 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7efc68004b20 con 0x7efc701075a0 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc78226740 1 -- start start 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc78226740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80eede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc78226740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80ef2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc78226740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 0x561f80ef5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc78226740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561f80e52970 con 0x561f80eeda30 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc78226740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f80dfad70 con 0x561f80eee420 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.531+0000 7efc78226740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561f80e4e920 con 0x561f80ef3590 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc6effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80ef2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc57fff640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efc580a2ff0 0x7efc580a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc6f7fe640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efc580a2ff0 0x7efc580a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc6e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80eede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc6effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80ef2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36340/0 (socket says 172.21.15.105:36340) 2024-03-26T17:53:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc6e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80eede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48394/0 (socket says 172.21.15.105:48394) 2024-03-26T17:53:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc6effd640 1 -- 172.21.15.105:0/2664920510 learned_addr learned my addr 172.21.15.105:0/2664920510 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc6f7fe640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efc580a2ff0 0x7efc580a54b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efc68098b30 tx=0x7efc68097cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.532+0000 7efc57fff640 1 -- 172.21.15.105:0/2173248291 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7efc680cfa70 con 0x7efc701075a0 2024-03-26T17:53:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc6e7fc640 1 -- 172.21.15.105:0/2664920510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 msgr2=0x561f80ef5a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:53:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/2664920510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 0x561f80ef5a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc6e7fc640 1 -- 172.21.15.105:0/2664920510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 msgr2=0x561f80ef2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/2664920510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80ef2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc6e7fc640 1 -- 172.21.15.105:0/2664920510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f80b663f0 con 0x561f80eeda30 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/2664920510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80eede50 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7efc64009780 tx=0x7efc64034890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05f45203bed23f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc557fa640 1 -- 172.21.15.105:0/2664920510 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64042020 con 0x561f80eeda30 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc557fa640 1 -- 172.21.15.105:0/2664920510 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc64034ce0 con 0x561f80eeda30 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.533+0000 7efc557fa640 1 -- 172.21.15.105:0/2664920510 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64039040 con 0x561f80eeda30 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 -- 172.21.15.105:0/2664920510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 msgr2=0x561f80eede50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 --2- 172.21.15.105:0/2664920510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80eede50 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7efc64009780 tx=0x7efc64034890 comp rx=0 tx=0).stop 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 -- 172.21.15.105:0/2664920510 shutdown_connections 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 --2- 172.21.15.105:0/2664920510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 0x561f80ef5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 --2- 172.21.15.105:0/2664920510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80ef2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 --2- 172.21.15.105:0/2664920510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80eede50 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 -- 172.21.15.105:0/2664920510 >> 172.21.15.105:0/2664920510 conn(0x561f80ee8fa0 msgr2=0x561f80ee93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 -- 172.21.15.105:0/2664920510 shutdown_connections 2024-03-26T17:53:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 -- 172.21.15.105:0/2664920510 wait complete. 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.534+0000 7efc78226740 1 -- start start 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc78226740 1 Processor -- start 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc78226740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f08f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc78226740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc78226740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 0x561f80f10510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc78226740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561f80f074f0 con 0x561f80eeda30 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc78226740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561f80e0e310 con 0x561f80ef3590 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc78226740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561f80f07670 con 0x561f80eee420 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc78226740 10 client.0 _ll_register_callbacks cb 0x561f80e49c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f08f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f08f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48396/0 (socket says 172.21.15.105:48396) 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 -- 172.21.15.105:0/353511188 learned_addr learned my addr 172.21.15.105:0/353511188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 0x561f80f10510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6effd640 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 -- 172.21.15.105:0/353511188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 msgr2=0x561f80f094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f094d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 -- 172.21.15.105:0/353511188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 msgr2=0x561f80f10510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 0x561f80f10510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 -- 172.21.15.105:0/353511188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f80c307d0 con 0x561f80eeda30 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.535+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f08f90 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7efc64037040 tx=0x7efc6402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d2aab8fa446cc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc52ff5640 1 -- 172.21.15.105:0/353511188 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64050060 con 0x561f80eeda30 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc52ff5640 1 -- 172.21.15.105:0/353511188 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7efc640037f0 con 0x561f80eeda30 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 1 -- 172.21.15.105:0/353511188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc64034ce0 con 0x561f80eeda30 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc52ff5640 1 -- 172.21.15.105:0/353511188 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc64039030 con 0x561f80eeda30 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 10 client.21066 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 1 -- 172.21.15.105:0/353511188 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561f80f13880 con 0x561f80eeda30 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc52ff5640 1 -- 172.21.15.105:0/353511188 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc64034ce0 con 0x561f80eeda30 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc52ff5640 1 -- 172.21.15.105:0/353511188 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efc6403e880 con 0x561f80eeda30 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 10 client.21066 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 10 client.21066 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 20 client.21066 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:51.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561f80f15060 0x561f80f17480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 4 client.21066 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:51.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.536+0000 7efc78226740 1 -- 172.21.15.105:0/353511188 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f80e5feb0 con 0x561f80f15060 2024-03-26T17:53:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.537+0000 7efc6effd640 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561f80f15060 0x561f80f17480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.537+0000 7efc52ff5640 10 client.21066 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:53:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.537+0000 7efc6effd640 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561f80f15060 0x561f80f17480 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0b655c8c29e1e17 server_cookie=922fbbe12ef2d388 in_seq=0 out_seq=0 2024-03-26T17:53:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.538+0000 7efc52ff5640 1 -- 172.21.15.105:0/353511188 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561f80e5feb0 con 0x561f80f15060 2024-03-26T17:53:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.538+0000 7efc52ff5640 10 client.21066 handle_command_reply: tid=0 2024-03-26T17:53:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.538+0000 7efc78226740 1 client.21066 shutdown 2024-03-26T17:53:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.538+0000 7efc78226740 1 -- 172.21.15.105:0/353511188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 msgr2=0x561f80f08f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.538+0000 7efc78226740 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f08f90 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7efc64037040 tx=0x7efc6402e040 comp rx=0 tx=0).stop 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 1 -- 172.21.15.105:0/353511188 shutdown_connections 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561f80f15060 0x561f80f17480 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 0x561f80f10510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc52ff5640 0 client.21066 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 1 --2- 172.21.15.105:0/353511188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f08f90 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 1 -- 172.21.15.105:0/353511188 >> 172.21.15.105:0/353511188 conn(0x561f80ee8fa0 msgr2=0x561f80ee81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 1 -- 172.21.15.105:0/353511188 shutdown_connections 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 1 -- 172.21.15.105:0/353511188 wait complete. 2024-03-26T17:53:51.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.539+0000 7efc78226740 20 client.21066 trim_cache size 0 max 16384 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc78226740 1 Processor -- start 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc78226740 1 -- start start 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc78226740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc78226740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc78226740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 0x561f80ee7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc78226740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a68e0 con 0x561f80eeda30 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc78226740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a84e0 con 0x561f80ef3590 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc78226740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a6fb0 con 0x561f80eee420 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51492/0 (socket says 172.21.15.105:51492) 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6effd640 1 -- 172.21.15.105:0/1282215930 learned_addr learned my addr 172.21.15.105:0/1282215930 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 0x561f80ee7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6effd640 1 -- 172.21.15.105:0/1282215930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 msgr2=0x561f80ee7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6effd640 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 0x561f80ee7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6effd640 1 -- 172.21.15.105:0/1282215930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 msgr2=0x561f80f092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6effd640 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6effd640 1 -- 172.21.15.105:0/1282215930 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f80ef2c60 con 0x561f80eee420 2024-03-26T17:53:51.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.561+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:51.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 20347 7.22 MB/sec execute 34 sec latency 109.397 ms 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.562+0000 7efc6effd640 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f104a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x561f80eee820 tx=0x7efc60009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8c605453e06e1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.562+0000 7efc54ff9640 1 -- 172.21.15.105:0/1282215930 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60004250 con 0x561f80eee420 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.562+0000 7efc54ff9640 1 -- 172.21.15.105:0/1282215930 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc600043f0 con 0x561f80eee420 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.562+0000 7efc54ff9640 1 -- 172.21.15.105:0/1282215930 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60004e40 con 0x561f80eee420 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 -- 172.21.15.105:0/1282215930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 msgr2=0x561f80f104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f104a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x561f80eee820 tx=0x7efc60009260 comp rx=0 tx=0).stop 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 -- 172.21.15.105:0/1282215930 shutdown_connections 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80ef3590 0x561f80ee7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80eee420 0x561f80f104a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 --2- 172.21.15.105:0/1282215930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 -- 172.21.15.105:0/1282215930 >> 172.21.15.105:0/1282215930 conn(0x561f80ee8fa0 msgr2=0x561f80f08ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 -- 172.21.15.105:0/1282215930 shutdown_connections 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 -- 172.21.15.105:0/1282215930 wait complete. 2024-03-26T17:53:51.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 -- start start 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.563+0000 7efc78226740 1 Processor -- start 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc78226740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f0d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc78226740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80f0d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc78226740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 0x561f80f793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc78226740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a7e10 con 0x561f80eeda30 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f0d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f0d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48408/0 (socket says 172.21.15.105:48408) 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 -- 172.21.15.105:0/1383920511 learned_addr learned my addr 172.21.15.105:0/1383920511 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc701a7680 con 0x561f80eee420 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc70071dd0 con 0x561f80ef3590 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6effd640 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80f0d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc78226740 10 client.0 _ll_register_callbacks cb 0x7efc60009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 0x561f80f793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 -- 172.21.15.105:0/1383920511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 msgr2=0x561f80f793b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 0x561f80f793b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 -- 172.21.15.105:0/1383920511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 msgr2=0x561f80f0d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80f0d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 -- 172.21.15.105:0/1383920511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f80f798f0 con 0x561f80eeda30 2024-03-26T17:53:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc6e7fc640 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f0d320 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7efc64009860 tx=0x7efc64001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c481a28523a66380 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.564+0000 7efc52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc52ff5640 1 -- 172.21.15.105:0/1383920511 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6405a030 con 0x561f80eeda30 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc52ff5640 1 -- 172.21.15.105:0/1383920511 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7efc64005230 con 0x561f80eeda30 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc600043f0 con 0x561f80eeda30 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc52ff5640 1 -- 172.21.15.105:0/1383920511 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efc64050020 con 0x561f80eeda30 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc78226740 10 client.21072 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561f80f7aa90 con 0x561f80eeda30 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc52ff5640 1 -- 172.21.15.105:0/1383920511 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc600043f0 con 0x561f80eeda30 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc52ff5640 1 -- 172.21.15.105:0/1383920511 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efc64060040 con 0x561f80eeda30 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc78226740 10 client.21072 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc78226740 10 client.21072 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc78226740 20 client.21072 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.565+0000 7efc78226740 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561f80f7b390 0x561f80f7d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.566+0000 7efc78226740 4 client.21072 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.566+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f80e5a200 con 0x561f80f7b390 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.566+0000 7efc6effd640 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561f80f7b390 0x561f80f7d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.566+0000 7efc52ff5640 10 client.21072 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.566+0000 7efc6effd640 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561f80f7b390 0x561f80f7d7b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7926e42f6c0408bf server_cookie=ab80dd3e04796b08 in_seq=0 out_seq=0 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.566+0000 7efc52ff5640 1 -- 172.21.15.105:0/1383920511 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f80e5a200 con 0x561f80f7b390 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.566+0000 7efc52ff5640 10 client.21072 handle_command_reply: tid=0 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.566+0000 7efc78226740 1 client.21072 shutdown 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 msgr2=0x561f80f0d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f0d320 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7efc64009860 tx=0x7efc64001e50 comp rx=0 tx=0).stop 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 shutdown_connections 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561f80f7b390 0x561f80f7d7b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f80ef3590 0x561f80f793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561f80eee420 0x561f80f0d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc52ff5640 0 client.21072 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:53:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 --2- 172.21.15.105:0/1383920511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561f80eeda30 0x561f80f0d320 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 >> 172.21.15.105:0/1383920511 conn(0x561f80ee8fa0 msgr2=0x561f80eec200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 shutdown_connections 2024-03-26T17:53:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 1 -- 172.21.15.105:0/1383920511 wait complete. 2024-03-26T17:53:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.567+0000 7efc78226740 20 client.21072 trim_cache size 0 max 16384 2024-03-26T17:53:51.561 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efc580a2ff0 msgr2=0x7efc580a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efc580a2ff0 0x7efc580a54b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efc68098b30 tx=0x7efc68097cd0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc701075a0 msgr2=0x7efc701a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc701075a0 0x7efc701a5530 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efc68009fd0 tx=0x7efc6800ef10 comp rx=0 tx=0).stop 2024-03-26T17:53:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 shutdown_connections 2024-03-26T17:53:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7efc580a2ff0 0x7efc580a54b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc701075a0 0x7efc701a5530 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc70104c70 0x7efc7019e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 --2- 172.21.15.105:0/2173248291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc70102340 0x7efc7019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 >> 172.21.15.105:0/2173248291 conn(0x7efc700fbd50 msgr2=0x7efc70105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 shutdown_connections 2024-03-26T17:53:51.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.568+0000 7efc7555b640 1 -- 172.21.15.105:0/2173248291 wait complete. 2024-03-26T17:53:51.570 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:53:51.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.733+0000 7faeabe61640 1 -- 172.21.15.105:0/3783681377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea4100a60 msgr2=0x7faea4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.733+0000 7faeabe61640 1 --2- 172.21.15.105:0/3783681377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea4100a60 0x7faea4100ee0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fae94009930 tx=0x7fae9402bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.733+0000 7faeabe61640 1 -- 172.21.15.105:0/3783681377 shutdown_connections 2024-03-26T17:53:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.733+0000 7faeabe61640 1 --2- 172.21.15.105:0/3783681377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faea4101420 0x7faea4108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.733+0000 7faeabe61640 1 --2- 172.21.15.105:0/3783681377 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea4100a60 0x7faea4100ee0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.733+0000 7faeabe61640 1 --2- 172.21.15.105:0/3783681377 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faea4101dc0 0x7faea41021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.733+0000 7faeabe61640 1 -- 172.21.15.105:0/3783681377 >> 172.21.15.105:0/3783681377 conn(0x7faea40fb4b0 msgr2=0x7faea40fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.734+0000 7faeabe61640 1 -- 172.21.15.105:0/3783681377 shutdown_connections 2024-03-26T17:53:51.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.734+0000 7faeabe61640 1 -- 172.21.15.105:0/3783681377 wait complete. 2024-03-26T17:53:51.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.734+0000 7faeabe61640 1 Processor -- start 2024-03-26T17:53:51.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.734+0000 7faeabe61640 1 -- start start 2024-03-26T17:53:51.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.734+0000 7faeabe61640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faea4100a60 0x7faea4197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.734+0000 7faeabe61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faea4101420 0x7faea41980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faeabe61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea4101dc0 0x7faea419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faeabe61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faea40ffe20 con 0x7faea4101dc0 2024-03-26T17:53:51.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea9bfd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faea4100a60 0x7faea4197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faea4101420 0x7faea41980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faea4101420 0x7faea41980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51520/0 (socket says 172.21.15.105:51520) 2024-03-26T17:53:51.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 -- 172.21.15.105:0/3569536001 learned_addr learned my addr 172.21.15.105:0/3569536001 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faeabe61640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faea40ffca0 con 0x7faea4100a60 2024-03-26T17:53:51.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faea40fffa0 con 0x7faea4101420 2024-03-26T17:53:51.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea4101dc0 0x7faea419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 -- 172.21.15.105:0/3569536001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faea4100a60 msgr2=0x7faea4197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faea4100a60 0x7faea4197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 -- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea4101dc0 msgr2=0x7faea419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea4101dc0 0x7faea419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 -- 172.21.15.105:0/3569536001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea419f790 con 0x7faea4101420 2024-03-26T17:53:51.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.735+0000 7faea93fc640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faea4101420 0x7faea41980b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fae94009230 tx=0x7fae940040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.736+0000 7fae9affd640 1 -- 172.21.15.105:0/3569536001 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94043020 con 0x7faea4101420 2024-03-26T17:53:51.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.736+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea419fa20 con 0x7faea4101420 2024-03-26T17:53:51.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.736+0000 7fae9affd640 1 -- 172.21.15.105:0/3569536001 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae940044c0 con 0x7faea4101420 2024-03-26T17:53:51.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.736+0000 7fae9affd640 1 -- 172.21.15.105:0/3569536001 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94003bd0 con 0x7faea4101420 2024-03-26T17:53:51.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.736+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea419ffc0 con 0x7faea4101420 2024-03-26T17:53:51.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.738+0000 7fae9affd640 1 -- 172.21.15.105:0/3569536001 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fae94046020 con 0x7faea4101420 2024-03-26T17:53:51.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.739+0000 7faeaea55740 1 Processor -- start 2024-03-26T17:53:51.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.739+0000 7faeaea55740 1 -- start start 2024-03-26T17:53:51.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.739+0000 7faeaea55740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c84930e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.739+0000 7faeaea55740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c84935f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.739+0000 7faeaea55740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84936590 0x562c84938a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.739+0000 7faeaea55740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562c84895970 con 0x562c84936590 2024-03-26T17:53:51.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.739+0000 7faeaea55740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562c8483dd70 con 0x562c84931420 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.739+0000 7faeaea55740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562c84891920 con 0x562c84930a30 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7fae9affd640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fae900a2fa0 0x7fae900a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84936590 0x562c84938a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84936590 0x562c84938a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48428/0 (socket says 172.21.15.105:48428) 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 -- 172.21.15.105:0/3440610092 learned_addr learned my addr 172.21.15.105:0/3440610092 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c84930e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea9bfd640 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c84935f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7fae9affd640 1 -- 172.21.15.105:0/3569536001 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fae940f1690 con 0x7faea4101420 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fae900a2fa0 0x7fae900a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 -- 172.21.15.105:0/3440610092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 msgr2=0x562c84930e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c84930e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 -- 172.21.15.105:0/3440610092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 msgr2=0x562c84935f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c84935f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 -- 172.21.15.105:0/3440610092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c845a93f0 con 0x562c84936590 2024-03-26T17:53:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.740+0000 7faea93fc640 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84936590 0x562c84938a20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fae940ba110 tx=0x7fae9404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432e1c2583bc9ed9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.741+0000 7fae8ffff640 1 -- 172.21.15.105:0/3440610092 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940bd680 con 0x562c84936590 2024-03-26T17:53:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.741+0000 7fae8ffff640 1 -- 172.21.15.105:0/3440610092 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae940be420 con 0x562c84936590 2024-03-26T17:53:51.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.741+0000 7fae8ffff640 1 -- 172.21.15.105:0/3440610092 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940ba390 con 0x562c84936590 2024-03-26T17:53:51.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 -- 172.21.15.105:0/3440610092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84936590 msgr2=0x562c84938a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84936590 0x562c84938a20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fae940ba110 tx=0x7fae9404f000 comp rx=0 tx=0).stop 2024-03-26T17:53:51.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fae900a2fa0 0x7fae900a5460 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x562c84931c80 tx=0x7fae9c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 -- 172.21.15.105:0/3440610092 shutdown_connections 2024-03-26T17:53:51.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84936590 0x562c84938a20 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c84935f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 --2- 172.21.15.105:0/3440610092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c84930e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 -- 172.21.15.105:0/3440610092 >> 172.21.15.105:0/3440610092 conn(0x562c8492bfa0 msgr2=0x562c8492c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 -- 172.21.15.105:0/3440610092 shutdown_connections 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.742+0000 7faeaea55740 1 -- 172.21.15.105:0/3440610092 wait complete. 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 1 -- start start 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 1 Processor -- start 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c8493ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c8493b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84942750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562c84939300 con 0x562c84931420 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562c84851310 con 0x562c84940360 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562c84939480 con 0x562c84930a30 2024-03-26T17:53:51.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.743+0000 7faeaea55740 10 client.0 _ll_register_callbacks cb 0x562c8488cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c8493b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faeaa3fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84942750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c8493b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48434/0 (socket says 172.21.15.105:48434) 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 -- 172.21.15.105:0/1769323931 learned_addr learned my addr 172.21.15.105:0/1769323931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faeaa3fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84942750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36408/0 (socket says 172.21.15.105:36408) 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea9bfd640 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c8493ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 -- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 msgr2=0x562c8493ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c8493ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 -- 172.21.15.105:0/1769323931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 msgr2=0x562c84942750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84942750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 -- 172.21.15.105:0/1769323931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c846737d0 con 0x562c84931420 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea93fc640 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c8493b3a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fae940ba6f0 tx=0x7fae9402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ef95f4c8ff1277 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7faea9bfd640 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c8493ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7fae8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:51.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1769323931 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940dc060 con 0x562c84931420 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.744+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1769323931 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fae940c64b0 con 0x562c84931420 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7faeaea55740 1 -- 172.21.15.105:0/1769323931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae940be420 con 0x562c84931420 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1769323931 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae940bdd20 con 0x562c84931420 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7faeaea55740 10 client.21084 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7faeaea55740 1 -- 172.21.15.105:0/1769323931 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562c84939300 con 0x562c84931420 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1769323931 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae940be420 con 0x562c84931420 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1769323931 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fae940e2070 con 0x562c84931420 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7faeaea55740 10 client.21084 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7faeaea55740 10 client.21084 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.745+0000 7faeaea55740 20 client.21084 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.746+0000 7faeaea55740 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x562c84946ec0 0x562c849492d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.746+0000 7faeaea55740 4 client.21084 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.746+0000 7faeaea55740 1 -- 172.21.15.105:0/1769323931 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c848a2eb0 con 0x562c84946ec0 2024-03-26T17:53:51.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.746+0000 7faea9bfd640 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x562c84946ec0 0x562c849492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.746+0000 7fae8d7fa640 10 client.21084 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:53:51.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.747+0000 7faea9bfd640 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x562c84946ec0 0x562c849492d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ee622da7a5c1af93 server_cookie=769d6f865995d24b in_seq=0 out_seq=0 2024-03-26T17:53:51.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.747+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1769323931 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562c848a2eb0 con 0x562c84946ec0 2024-03-26T17:53:51.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.747+0000 7fae8d7fa640 10 client.21084 handle_command_reply: tid=0 2024-03-26T17:53:51.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7faeaea55740 1 client.21084 shutdown 2024-03-26T17:53:51.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7faeaea55740 1 -- 172.21.15.105:0/1769323931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 msgr2=0x562c8493b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7faeaea55740 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c8493b3a0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fae940ba6f0 tx=0x7fae9402bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:51.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7faeaea55740 1 -- 172.21.15.105:0/1769323931 shutdown_connections 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7faeaea55740 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x562c84946ec0 0x562c849492d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7faeaea55740 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84942750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7fae8d7fa640 0 client.21084 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7faeaea55740 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c8493b3a0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.748+0000 7faeaea55740 1 --2- 172.21.15.105:0/1769323931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c8493ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.749+0000 7faeaea55740 1 -- 172.21.15.105:0/1769323931 >> 172.21.15.105:0/1769323931 conn(0x562c8492bfa0 msgr2=0x562c8492b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.749+0000 7faeaea55740 1 -- 172.21.15.105:0/1769323931 shutdown_connections 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.749+0000 7faeaea55740 1 -- 172.21.15.105:0/1769323931 wait complete. 2024-03-26T17:53:51.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.749+0000 7faeaea55740 20 client.21084 trim_cache size 0 max 16384 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaea55740 1 Processor -- start 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaea55740 1 -- start start 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaea55740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c84932ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaea55740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c849330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaea55740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84940360 0x562c84939700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaea55740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faea41a6b80 con 0x562c84940360 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaea55740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faea41a8780 con 0x562c84931420 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaea55740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faea41a7250 con 0x562c84930a30 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faea93fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c849330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faea9bfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c84932ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faea93fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c849330e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36414/0 (socket says 172.21.15.105:36414) 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faea93fc640 1 -- 172.21.15.105:0/3615979727 learned_addr learned my addr 172.21.15.105:0/3615979727 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.771+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/3615979727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84940360 0x562c84939700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7faea93fc640 1 -- 172.21.15.105:0/3615979727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 msgr2=0x562c84932ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7faea93fc640 1 --2- 172.21.15.105:0/3615979727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c84932ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7faea93fc640 1 -- 172.21.15.105:0/3615979727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84940360 msgr2=0x562c84939700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7faea93fc640 1 --2- 172.21.15.105:0/3615979727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84940360 0x562c84939700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7faea93fc640 1 -- 172.21.15.105:0/3615979727 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c84931cb0 con 0x562c84931420 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7faea93fc640 1 --2- 172.21.15.105:0/3615979727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c849330e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fae94009900 tx=0x7fae940f1910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29c4b07fb957c4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7fae8f7fe640 1 -- 172.21.15.105:0/3615979727 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94098030 con 0x562c84931420 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7fae8f7fe640 1 -- 172.21.15.105:0/3615979727 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae940be420 con 0x562c84931420 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.772+0000 7fae8f7fe640 1 -- 172.21.15.105:0/3615979727 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940dc040 con 0x562c84931420 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.773+0000 7faeaea55740 1 -- 172.21.15.105:0/3615979727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 msgr2=0x562c849330e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.773+0000 7faeaea55740 1 --2- 172.21.15.105:0/3615979727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c849330e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fae94009900 tx=0x7fae940f1910 comp rx=0 tx=0).stop 2024-03-26T17:53:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.773+0000 7faeaea55740 1 -- 172.21.15.105:0/3615979727 shutdown_connections 2024-03-26T17:53:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.773+0000 7faeaea55740 1 --2- 172.21.15.105:0/3615979727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84940360 0x562c84939700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.773+0000 7faeaea55740 1 --2- 172.21.15.105:0/3615979727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84931420 0x562c849330e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.773+0000 7faeaea55740 1 --2- 172.21.15.105:0/3615979727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c84932ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.773+0000 7faeaea55740 1 -- 172.21.15.105:0/3615979727 >> 172.21.15.105:0/3615979727 conn(0x562c8492bfa0 msgr2=0x562c8492c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.774+0000 7faeaea55740 1 -- 172.21.15.105:0/3615979727 shutdown_connections 2024-03-26T17:53:51.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.774+0000 7faeaea55740 1 -- 172.21.15.105:0/3615979727 wait complete. 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.774+0000 7faeaea55740 1 -- start start 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.774+0000 7faeaea55740 1 Processor -- start 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.775+0000 7faeaea55740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c849577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.775+0000 7faeaea55740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c84957ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faea93fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c84957ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faea93fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c84957ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48452/0 (socket says 172.21.15.105:48452) 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faea9bfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c849577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faea9bfd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c849577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51544/0 (socket says 172.21.15.105:51544) 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faeaea55740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84958cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faea93fc640 1 -- 172.21.15.105:0/1343682057 learned_addr learned my addr 172.21.15.105:0/1343682057 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84958cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faea41a80b0 con 0x562c84931420 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faea41a7920 con 0x562c84940360 2024-03-26T17:53:51.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faea41a4b60 con 0x562c84930a30 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.776+0000 7faeaea55740 10 client.0 _ll_register_callbacks cb 0x7fae9c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faeaa3fe640 1 -- 172.21.15.105:0/1343682057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 msgr2=0x562c849577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c849577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faeaa3fe640 1 -- 172.21.15.105:0/1343682057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 msgr2=0x562c84957ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c84957ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faeaa3fe640 1 -- 172.21.15.105:0/1343682057 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c849592e0 con 0x562c84940360 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faea93fc640 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c84957ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faea9bfd640 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c849577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faeaa3fe640 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84958cf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005fd0 tx=0x7fae9c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8148305ccf9fa064 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7fae8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:51.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1343682057 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c004b20 con 0x562c84940360 2024-03-26T17:53:51.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1343682057 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fae9c004cc0 con 0x562c84940360 2024-03-26T17:53:51.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.777+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae940be420 con 0x562c84940360 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.778+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1343682057 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae9c004450 con 0x562c84940360 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.778+0000 7faeaea55740 10 client.30875 fetch_fsmap learned FSMap version 34 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.778+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562c8495a650 con 0x562c84940360 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.778+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1343682057 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae940be420 con 0x562c84940360 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.779+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1343682057 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fae9c033030 con 0x562c84940360 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.779+0000 7faeaea55740 10 client.30875 fetch_fsmap finished waiting for FSMap version 34 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.779+0000 7faeaea55740 10 client.30875 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.779+0000 7faeaea55740 20 client.30875 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.779+0000 7faeaea55740 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x562c84946ec0 0x562c849af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.779+0000 7faeaea55740 4 client.30875 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:51.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.779+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c8489d200 con 0x562c84946ec0 2024-03-26T17:53:51.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.779+0000 7faea9bfd640 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x562c84946ec0 0x562c849af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.780+0000 7fae8d7fa640 10 client.30875 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:53:51.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.780+0000 7faea9bfd640 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x562c84946ec0 0x562c849af330 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=453a945d7cdc0923 server_cookie=c6f2e0044ada29b7 in_seq=0 out_seq=0 2024-03-26T17:53:51.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.780+0000 7fae8d7fa640 1 -- 172.21.15.105:0/1343682057 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c8489d200 con 0x562c84946ec0 2024-03-26T17:53:51.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.780+0000 7fae8d7fa640 10 client.30875 handle_command_reply: tid=0 2024-03-26T17:53:51.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.780+0000 7faeaea55740 1 client.30875 shutdown 2024-03-26T17:53:51.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 msgr2=0x562c84958cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84958cf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005fd0 tx=0x7fae9c004a40 comp rx=0 tx=0).stop 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 shutdown_connections 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x562c84946ec0 0x562c849af330 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562c84940360 0x562c84958cf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7fae8d7fa640 0 client.30875 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562c84931420 0x562c84957ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 --2- 172.21.15.105:0/1343682057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562c84930a30 0x562c849577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 >> 172.21.15.105:0/1343682057 conn(0x562c8492bfa0 msgr2=0x562c8493ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 shutdown_connections 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 1 -- 172.21.15.105:0/1343682057 wait complete. 2024-03-26T17:53:51.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.781+0000 7faeaea55740 20 client.30875 trim_cache size 0 max 16384 2024-03-26T17:53:51.775 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:51.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fae900a2fa0 msgr2=0x7fae900a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fae900a2fa0 0x7fae900a5460 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x562c84931c80 tx=0x7fae9c004ab0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faea4101420 msgr2=0x7faea41980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faea4101420 0x7faea41980b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fae94009230 tx=0x7fae940040c0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 shutdown_connections 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fae900a2fa0 0x7fae900a5460 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faea4101dc0 0x7faea419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faea4101420 0x7faea41980b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 --2- 172.21.15.105:0/3569536001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faea4100a60 0x7faea4197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 >> 172.21.15.105:0/3569536001 conn(0x7faea40fb4b0 msgr2=0x7faea4104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 shutdown_connections 2024-03-26T17:53:51.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.782+0000 7faeabe61640 1 -- 172.21.15.105:0/3569536001 wait complete. 2024-03-26T17:53:51.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:53:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 -- 172.21.15.105:0/49884720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581019b0 msgr2=0x7f0558101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 --2- 172.21.15.105:0/49884720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581019b0 0x7f0558101e30 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f054c009900 tx=0x7f054c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 -- 172.21.15.105:0/49884720 shutdown_connections 2024-03-26T17:53:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 --2- 172.21.15.105:0/49884720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0558102370 0x7f0558068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 --2- 172.21.15.105:0/49884720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581019b0 0x7f0558101e30 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 --2- 172.21.15.105:0/49884720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05581007c0 0x7f0558100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 -- 172.21.15.105:0/49884720 >> 172.21.15.105:0/49884720 conn(0x7f05580fbd50 msgr2=0x7f05580fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 -- 172.21.15.105:0/49884720 shutdown_connections 2024-03-26T17:53:51.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.948+0000 7f055e15b640 1 -- 172.21.15.105:0/49884720 wait complete. 2024-03-26T17:53:51.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.949+0000 7f055e15b640 1 Processor -- start 2024-03-26T17:53:51.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.949+0000 7f055e15b640 1 -- start start 2024-03-26T17:53:51.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.949+0000 7f055e15b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581007c0 0x7f0558195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.949+0000 7f055e15b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05581019b0 0x7f0558195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.949+0000 7f055e15b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558102370 0x7f055819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.949+0000 7f055e15b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f055810b250 con 0x7f05581007c0 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.949+0000 7f055e15b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f055810b0d0 con 0x7f05581019b0 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.949+0000 7f055e15b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f055810b3d0 con 0x7f0558102370 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05581019b0 0x7f0558195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0557fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558102370 0x7f055819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05581019b0 0x7f0558195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36442/0 (socket says 172.21.15.105:36442) 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0556ffd640 1 -- 172.21.15.105:0/847347838 learned_addr learned my addr 172.21.15.105:0/847347838 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f05577fe640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581007c0 0x7f0558195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0557fff640 1 -- 172.21.15.105:0/847347838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05581019b0 msgr2=0x7f0558195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0557fff640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05581019b0 0x7f0558195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0557fff640 1 -- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581007c0 msgr2=0x7f0558195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0557fff640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581007c0 0x7f0558195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0557fff640 1 -- 172.21.15.105:0/847347838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055819d520 con 0x7f0558102370 2024-03-26T17:53:51.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f05577fe640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581007c0 0x7f0558195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:51.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.950+0000 7f0557fff640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558102370 0x7f055819ceb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0550005fd0 tx=0x7f0550016850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.951+0000 7f0554ff9640 1 -- 172.21.15.105:0/847347838 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550020020 con 0x7f0558102370 2024-03-26T17:53:51.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.951+0000 7f055e15b640 1 -- 172.21.15.105:0/847347838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055819d810 con 0x7f0558102370 2024-03-26T17:53:51.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.951+0000 7f0554ff9640 1 -- 172.21.15.105:0/847347838 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f055000fde0 con 0x7f0558102370 2024-03-26T17:53:51.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.951+0000 7f055e15b640 1 -- 172.21.15.105:0/847347838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055819dd60 con 0x7f0558102370 2024-03-26T17:53:51.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.951+0000 7f0554ff9640 1 -- 172.21.15.105:0/847347838 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500114f0 con 0x7f0558102370 2024-03-26T17:53:51.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0554ff9640 1 -- 172.21.15.105:0/847347838 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f055001a050 con 0x7f0558102370 2024-03-26T17:53:51.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0560e24740 1 Processor -- start 2024-03-26T17:53:51.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0560e24740 1 -- start start 2024-03-26T17:53:51.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0560e24740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109a6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0560e24740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0560e24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109ac590 0x563f109aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0560e24740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563f1090b970 con 0x563f109ac590 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0560e24740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563f108b3d70 con 0x563f109a6a30 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.954+0000 7f0560e24740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563f10907920 con 0x563f109a7420 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109a6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109a6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36448/0 (socket says 172.21.15.105:36448) 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 -- 172.21.15.105:0/3173657937 learned_addr learned my addr 172.21.15.105:0/3173657937 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109ac590 0x563f109aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0554ff9640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05400a3020 0x7f05400a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f05577fe640 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0557fff640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05400a3020 0x7f05400a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0554ff9640 1 -- 172.21.15.105:0/847347838 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f05500d73f0 con 0x7f0558102370 2024-03-26T17:53:51.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 -- 172.21.15.105:0/3173657937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 msgr2=0x563f109abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 -- 172.21.15.105:0/3173657937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109ac590 msgr2=0x563f109aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109ac590 0x563f109aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 -- 172.21.15.105:0/3173657937 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f1061f3f0 con 0x563f109a6a30 2024-03-26T17:53:51.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0556ffd640 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109a6e50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f054c02c240 tx=0x7f054c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36ed4f1ed016779a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f0557fff640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05400a3020 0x7f05400a54e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0550011fd0 tx=0x7f0550020fc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.955+0000 7f053dffb640 1 -- 172.21.15.105:0/3173657937 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c004030 con 0x563f109a6a30 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f053dffb640 1 -- 172.21.15.105:0/3173657937 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f054c048070 con 0x563f109a6a30 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f053dffb640 1 -- 172.21.15.105:0/3173657937 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c004900 con 0x563f109a6a30 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f0560e24740 1 -- 172.21.15.105:0/3173657937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 msgr2=0x563f109a6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f0560e24740 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109a6e50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f054c02c240 tx=0x7f054c005d50 comp rx=0 tx=0).stop 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f0560e24740 1 -- 172.21.15.105:0/3173657937 shutdown_connections 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f0560e24740 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109ac590 0x563f109aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f0560e24740 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f0560e24740 1 --2- 172.21.15.105:0/3173657937 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109a6e50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.956+0000 7f0560e24740 1 -- 172.21.15.105:0/3173657937 >> 172.21.15.105:0/3173657937 conn(0x563f109a1fa0 msgr2=0x563f109a23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.957+0000 7f0560e24740 1 -- 172.21.15.105:0/3173657937 shutdown_connections 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.957+0000 7f0560e24740 1 -- 172.21.15.105:0/3173657937 wait complete. 2024-03-26T17:53:51.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.957+0000 7f0560e24740 1 -- start start 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.957+0000 7f0560e24740 1 Processor -- start 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0560e24740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0560e24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 0x563f109b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0560e24740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 0x563f109b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0560e24740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563f109af4e0 con 0x563f109a7420 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0560e24740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563f108c7310 con 0x563f109a6a30 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0560e24740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563f109af660 con 0x563f109ac590 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0560e24740 10 client.0 _ll_register_callbacks cb 0x563f10902c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f05577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 0x563f109b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36450/0 (socket says 172.21.15.105:36450) 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 -- 172.21.15.105:0/1100582842 learned_addr learned my addr 172.21.15.105:0/1100582842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 0x563f109b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 -- 172.21.15.105:0/1100582842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 msgr2=0x563f109b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 0x563f109b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 -- 172.21.15.105:0/1100582842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 msgr2=0x563f109b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 0x563f109b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 -- 172.21.15.105:0/1100582842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f106e97d0 con 0x563f109a6a30 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0556ffd640 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b0fa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f054c02c840 tx=0x7f054c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32526154b39f16d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0536ffd640 1 -- 172.21.15.105:0/1100582842 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c004ad0 con 0x563f109a6a30 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.958+0000 7f0536ffd640 1 -- 172.21.15.105:0/1100582842 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f054c004c70 con 0x563f109a6a30 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.959+0000 7f0560e24740 1 -- 172.21.15.105:0/1100582842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f054c048070 con 0x563f109a6a30 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.959+0000 7f0536ffd640 1 -- 172.21.15.105:0/1100582842 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f054c004e70 con 0x563f109a6a30 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.959+0000 7f0560e24740 10 client.30887 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.959+0000 7f0560e24740 1 -- 172.21.15.105:0/1100582842 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563f109bb870 con 0x563f109a6a30 2024-03-26T17:53:51.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.959+0000 7f0536ffd640 1 -- 172.21.15.105:0/1100582842 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f054c048070 con 0x563f109a6a30 2024-03-26T17:53:51.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f0536ffd640 1 -- 172.21.15.105:0/1100582842 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f054c0404f0 con 0x563f109a6a30 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f0560e24740 10 client.30887 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f0560e24740 10 client.30887 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f0560e24740 20 client.30887 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f0560e24740 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563f109bcbc0 0x563f109befb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f0560e24740 4 client.30887 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f0560e24740 1 -- 172.21.15.105:0/1100582842 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f10918eb0 con 0x563f109bcbc0 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f05577fe640 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563f109bcbc0 0x563f109befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f0536ffd640 10 client.30887 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.960+0000 7f05577fe640 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563f109bcbc0 0x563f109befb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb402511738578d2 server_cookie=1389a7b2169ae525 in_seq=0 out_seq=0 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0536ffd640 1 -- 172.21.15.105:0/1100582842 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563f10918eb0 con 0x563f109bcbc0 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0536ffd640 10 client.30887 handle_command_reply: tid=0 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 client.30887 shutdown 2024-03-26T17:53:51.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 -- 172.21.15.105:0/1100582842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 msgr2=0x563f109b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b0fa0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f054c02c840 tx=0x7f054c009260 comp rx=0 tx=0).stop 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 -- 172.21.15.105:0/1100582842 shutdown_connections 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 0x563f109b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 0x563f109b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563f109bcbc0 0x563f109befb0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 --2- 172.21.15.105:0/1100582842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b0fa0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.961+0000 7f0560e24740 1 -- 172.21.15.105:0/1100582842 >> 172.21.15.105:0/1100582842 conn(0x563f109a1fa0 msgr2=0x563f109a11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.962+0000 7f0536ffd640 0 client.30887 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.962+0000 7f0560e24740 1 -- 172.21.15.105:0/1100582842 shutdown_connections 2024-03-26T17:53:51.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.962+0000 7f0560e24740 1 -- 172.21.15.105:0/1100582842 wait complete. 2024-03-26T17:53:51.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.962+0000 7f0560e24740 20 client.30887 trim_cache size 0 max 16384 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.985+0000 7f0560e24740 1 Processor -- start 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.985+0000 7f0560e24740 1 -- start start 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0560e24740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0560e24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 0x563f109b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0560e24740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 0x563f109a0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0560e24740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05581a6b80 con 0x563f109a7420 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0560e24740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05581a8720 con 0x563f109a6a30 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0560e24740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05581a7250 con 0x563f109ac590 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 0x563f109a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36458/0 (socket says 172.21.15.105:36458) 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 -- 172.21.15.105:0/2429421309 learned_addr learned my addr 172.21.15.105:0/2429421309 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f05577fe640 1 --2- 172.21.15.105:0/2429421309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 0x563f109b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 -- 172.21.15.105:0/2429421309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 msgr2=0x563f109a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 --2- 172.21.15.105:0/2429421309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 0x563f109a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 -- 172.21.15.105:0/2429421309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 msgr2=0x563f109b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 --2- 172.21.15.105:0/2429421309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 0x563f109b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.986+0000 7f0556ffd640 1 -- 172.21.15.105:0/2429421309 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f109a2d10 con 0x563f109a6a30 2024-03-26T17:53:51.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.987+0000 7f0556ffd640 1 --2- 172.21.15.105:0/2429421309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b12c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f054c040a30 tx=0x7f054c005950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc58d7ea884abf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.987+0000 7f053d7fa640 1 -- 172.21.15.105:0/2429421309 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c064040 con 0x563f109a6a30 2024-03-26T17:53:51.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.987+0000 7f053d7fa640 1 -- 172.21.15.105:0/2429421309 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f054c048040 con 0x563f109a6a30 2024-03-26T17:53:51.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.987+0000 7f053d7fa640 1 -- 172.21.15.105:0/2429421309 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c0055b0 con 0x563f109a6a30 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 -- 172.21.15.105:0/2429421309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 msgr2=0x563f109b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 --2- 172.21.15.105:0/2429421309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b12c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f054c040a30 tx=0x7f054c005950 comp rx=0 tx=0).stop 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 -- 172.21.15.105:0/2429421309 shutdown_connections 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 --2- 172.21.15.105:0/2429421309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109ac590 0x563f109a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 --2- 172.21.15.105:0/2429421309 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a7420 0x563f109b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 --2- 172.21.15.105:0/2429421309 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109a6a30 0x563f109b12c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 -- 172.21.15.105:0/2429421309 >> 172.21.15.105:0/2429421309 conn(0x563f109a1fa0 msgr2=0x563f109b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 -- 172.21.15.105:0/2429421309 shutdown_connections 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.988+0000 7f0560e24740 1 -- 172.21.15.105:0/2429421309 wait complete. 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.989+0000 7f0560e24740 1 -- start start 2024-03-26T17:53:51.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.989+0000 7f0560e24740 1 Processor -- start 2024-03-26T17:53:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.989+0000 7f0560e24740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a6a30 0x563f109b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.989+0000 7f0560e24740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.989+0000 7f0560e24740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109ac590 0x563f10a20d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.989+0000 7f0560e24740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05581a8050 con 0x563f109a6a30 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.989+0000 7f0560e24740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05581a7920 con 0x563f109ac590 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.989+0000 7f0560e24740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05581a4b60 con 0x563f109a7420 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f0560e24740 10 client.0 _ll_register_callbacks cb 0x7f054800bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a6a30 0x563f109b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f0556ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109ac590 0x563f10a20d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51606/0 (socket says 172.21.15.105:51606) 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 -- 172.21.15.105:0/3436713742 learned_addr learned my addr 172.21.15.105:0/3436713742 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 -- 172.21.15.105:0/3436713742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109ac590 msgr2=0x563f10a20d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109ac590 0x563f10a20d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 -- 172.21.15.105:0/3436713742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a6a30 msgr2=0x563f109b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a6a30 0x563f109b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 -- 172.21.15.105:0/3436713742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f10a21250 con 0x563f109a7420 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f0556ffd640 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109ac590 0x563f10a20d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f05577fe640 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109b5940 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0548000c00 tx=0x7f05480128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43400ee33861d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.990+0000 7f0536ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0536ffd640 1 -- 172.21.15.105:0/3436713742 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548020640 con 0x563f109a7420 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0536ffd640 1 -- 172.21.15.105:0/3436713742 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0548029420 con 0x563f109a7420 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0560e24740 1 -- 172.21.15.105:0/3436713742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f054c048040 con 0x563f109a7420 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0536ffd640 1 -- 172.21.15.105:0/3436713742 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f05480207e0 con 0x563f109a7420 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0560e24740 10 client.30862 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0560e24740 1 -- 172.21.15.105:0/3436713742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563f10a22490 con 0x563f109a7420 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0536ffd640 1 -- 172.21.15.105:0/3436713742 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f054c048040 con 0x563f109a7420 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0536ffd640 1 -- 172.21.15.105:0/3436713742 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f0548010550 con 0x563f109a7420 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0560e24740 10 client.30862 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:51.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0560e24740 10 client.30862 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:53:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.991+0000 7f0560e24740 20 client.30862 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.992+0000 7f0560e24740 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563f10a22ec0 0x563f10a252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.992+0000 7f0560e24740 4 client.30862 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.992+0000 7f0560e24740 1 -- 172.21.15.105:0/3436713742 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f10913200 con 0x563f10a22ec0 2024-03-26T17:53:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.992+0000 7f0556ffd640 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563f10a22ec0 0x563f10a252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:51.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.992+0000 7f0556ffd640 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563f10a22ec0 0x563f10a252e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d45d178e05f9e35e server_cookie=ebb7db3d38595a8e in_seq=0 out_seq=0 2024-03-26T17:53:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.992+0000 7f0536ffd640 10 client.30862 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:53:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0536ffd640 1 -- 172.21.15.105:0/3436713742 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f10913200 con 0x563f10a22ec0 2024-03-26T17:53:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0536ffd640 10 client.30862 handle_command_reply: tid=0 2024-03-26T17:53:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 client.30862 shutdown 2024-03-26T17:53:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 -- 172.21.15.105:0/3436713742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 msgr2=0x563f109b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109b5940 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0548000c00 tx=0x7f05480128e0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 -- 172.21.15.105:0/3436713742 shutdown_connections 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x563f10a22ec0 0x563f10a252e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563f109ac590 0x563f10a20d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563f109a7420 0x563f109b5940 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 --2- 172.21.15.105:0/3436713742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563f109a6a30 0x563f109b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 -- 172.21.15.105:0/3436713742 >> 172.21.15.105:0/3436713742 conn(0x563f109a1fa0 msgr2=0x563f109a5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0536ffd640 0 client.30862 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 -- 172.21.15.105:0/3436713742 shutdown_connections 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 1 -- 172.21.15.105:0/3436713742 wait complete. 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.993+0000 7f0560e24740 20 client.30862 trim_cache size 0 max 16384 2024-03-26T17:53:51.987 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 -- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05400a3020 msgr2=0x7f05400a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05400a3020 0x7f05400a54e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0550011fd0 tx=0x7f0550020fc0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 -- 172.21.15.105:0/847347838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558102370 msgr2=0x7f055819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558102370 0x7f055819ceb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0550005fd0 tx=0x7f0550016850 comp rx=0 tx=0).stop 2024-03-26T17:53:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 -- 172.21.15.105:0/847347838 shutdown_connections 2024-03-26T17:53:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05400a3020 0x7f05400a54e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0558102370 0x7f055819ceb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f05581019b0 0x7f0558195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 --2- 172.21.15.105:0/847347838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05581007c0 0x7f0558195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.994+0000 7f055e15b640 1 -- 172.21.15.105:0/847347838 >> 172.21.15.105:0/847347838 conn(0x7f05580fbd50 msgr2=0x7f0558102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.995+0000 7f055e15b640 1 -- 172.21.15.105:0/847347838 shutdown_connections 2024-03-26T17:53:51.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:51.995+0000 7f055e15b640 1 -- 172.21.15.105:0/847347838 wait complete. 2024-03-26T17:53:51.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:53:52.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 -- 172.21.15.105:0/569314969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab341023d0 msgr2=0x7fab34108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 --2- 172.21.15.105:0/569314969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab341023d0 0x7fab34108c60 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ca0 tx=0x7fab2c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 -- 172.21.15.105:0/569314969 shutdown_connections 2024-03-26T17:53:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 --2- 172.21.15.105:0/569314969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab341023d0 0x7fab34108c60 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 --2- 172.21.15.105:0/569314969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab34101a10 0x7fab34101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 --2- 172.21.15.105:0/569314969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab34100820 0x7fab34100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 -- 172.21.15.105:0/569314969 >> 172.21.15.105:0/569314969 conn(0x7fab340fbd90 msgr2=0x7fab340fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 -- 172.21.15.105:0/569314969 shutdown_connections 2024-03-26T17:53:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.094+0000 7fab39737640 1 -- 172.21.15.105:0/569314969 wait complete. 2024-03-26T17:53:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.095+0000 7fab39737640 1 Processor -- start 2024-03-26T17:53:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab39737640 1 -- start start 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab39737640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab34100820 0x7fab34196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab39737640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab34101a10 0x7fab34196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab39737640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab341023d0 0x7fab3419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab34100820 0x7fab34196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab34100820 0x7fab34196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48546/0 (socket says 172.21.15.105:48546) 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab34101a10 0x7fab34196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab32ffd640 1 -- 172.21.15.105:0/1402785822 learned_addr learned my addr 172.21.15.105:0/1402785822 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410c350 con 0x7fab34100820 2024-03-26T17:53:52.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410c1d0 con 0x7fab34101a10 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.096+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410c4d0 con 0x7fab341023d0 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab337fe640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab341023d0 0x7fab3419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab32ffd640 1 -- 172.21.15.105:0/1402785822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab341023d0 msgr2=0x7fab3419dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab32ffd640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab341023d0 0x7fab3419dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab32ffd640 1 -- 172.21.15.105:0/1402785822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab34101a10 msgr2=0x7fab34196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab32ffd640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab34101a10 0x7fab34196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab32ffd640 1 -- 172.21.15.105:0/1402785822 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3406a060 con 0x7fab34100820 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab2bfff640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab34101a10 0x7fab34196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab32ffd640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab34100820 0x7fab34196720 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fab2400e9c0 tx=0x7fab2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab30ff9640 1 -- 172.21.15.105:0/1402785822 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2400cd80 con 0x7fab34100820 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3406a350 con 0x7fab34100820 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3406a8a0 con 0x7fab34100820 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.097+0000 7fab30ff9640 1 -- 172.21.15.105:0/1402785822 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab240035f0 con 0x7fab34100820 2024-03-26T17:53:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.098+0000 7fab30ff9640 1 -- 172.21.15.105:0/1402785822 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24010690 con 0x7fab34100820 2024-03-26T17:53:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.101+0000 7fab30ff9640 1 -- 172.21.15.105:0/1402785822 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fab24010830 con 0x7fab34100820 2024-03-26T17:53:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.101+0000 7fab30ff9640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab200a3040 0x7fab200a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.102+0000 7fab2bfff640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab200a3040 0x7fab200a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.102+0000 7fab30ff9640 1 -- 172.21.15.105:0/1402785822 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fab240d0940 con 0x7fab34100820 2024-03-26T17:53:52.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab2bfff640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab200a3040 0x7fab200a5500 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fab34197c30 tx=0x7fab1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab3c3fc740 1 Processor -- start 2024-03-26T17:53:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab3c3fc740 1 -- start start 2024-03-26T17:53:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a40ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab3c3fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5627549a00b0 con 0x562754a41130 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab3c3fc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562754948960 con 0x562754a3bfc0 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.110+0000 7fab3c3fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56275499c060 con 0x562754a3b5d0 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a40ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a40ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36494/0 (socket says 172.21.15.105:36494) 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 -- 172.21.15.105:0/3857591817 learned_addr learned my addr 172.21.15.105:0/3857591817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab32ffd640 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab337fe640 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 -- 172.21.15.105:0/3857591817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 msgr2=0x562754a3b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 -- 172.21.15.105:0/3857591817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 msgr2=0x562754a435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 -- 172.21.15.105:0/3857591817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56275495ac60 con 0x562754a3bfc0 2024-03-26T17:53:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab337fe640 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.111+0000 7fab2bfff640 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a40ac0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fab1c004130 tx=0x7fab1c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e058c97e48cbbf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.112+0000 7fab297fa640 1 -- 172.21.15.105:0/3857591817 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c003db0 con 0x562754a3bfc0 2024-03-26T17:53:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.112+0000 7fab297fa640 1 -- 172.21.15.105:0/3857591817 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab1c025070 con 0x562754a3bfc0 2024-03-26T17:53:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.112+0000 7fab297fa640 1 -- 172.21.15.105:0/3857591817 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c003db0 con 0x562754a3bfc0 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3857591817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 msgr2=0x562754a40ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a40ac0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fab1c004130 tx=0x7fab1c014000 comp rx=0 tx=0).stop 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3857591817 shutdown_connections 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a40ac0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3857591817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3857591817 >> 172.21.15.105:0/3857591817 conn(0x562754a36b40 msgr2=0x562754a36f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3857591817 shutdown_connections 2024-03-26T17:53:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3857591817 wait complete. 2024-03-26T17:53:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 -- start start 2024-03-26T17:53:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.113+0000 7fab3c3fc740 1 Processor -- start 2024-03-26T17:53:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3b5d0 0x562754a4da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3bfc0 0x562754a4dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a54ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab3c3fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562754a440e0 con 0x562754a41130 2024-03-26T17:53:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab3c3fc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56275495bdb0 con 0x562754a3b5d0 2024-03-26T17:53:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab3c3fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562754a44260 con 0x562754a3bfc0 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab3c3fc740 10 client.0 _ll_register_callbacks cb 0x562754960110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3bfc0 0x562754a4dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3bfc0 0x562754a4dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51652/0 (socket says 172.21.15.105:51652) 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab337fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a54ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3b5d0 0x562754a4da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3b5d0 0x562754a4da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36506/0 (socket says 172.21.15.105:36506) 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab32ffd640 1 -- 172.21.15.105:0/3155834848 learned_addr learned my addr 172.21.15.105:0/3155834848 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab337fe640 1 -- 172.21.15.105:0/3155834848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3bfc0 msgr2=0x562754a4dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab337fe640 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3bfc0 0x562754a4dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab337fe640 1 -- 172.21.15.105:0/3155834848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3b5d0 msgr2=0x562754a4da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab337fe640 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3b5d0 0x562754a4da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab337fe640 1 -- 172.21.15.105:0/3155834848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627546b4210 con 0x562754a41130 2024-03-26T17:53:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.114+0000 7fab2bfff640 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3bfc0 0x562754a4dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.115+0000 7fab337fe640 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a54ff0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009bf0 tx=0x7fab2c032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcc6a775793a5ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.115+0000 7fab1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.115+0000 7fab1a7fc640 1 -- 172.21.15.105:0/3155834848 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c044070 con 0x562754a41130 2024-03-26T17:53:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.115+0000 7fab1a7fc640 1 -- 172.21.15.105:0/3155834848 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fab2c003970 con 0x562754a41130 2024-03-26T17:53:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.115+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3155834848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1c025070 con 0x562754a41130 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab1a7fc640 1 -- 172.21.15.105:0/3155834848 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fab2c04e050 con 0x562754a41130 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab3c3fc740 10 client.21120 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3155834848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562754780620 con 0x562754a41130 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab1a7fc640 1 -- 172.21.15.105:0/3155834848 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab1c025070 con 0x562754a41130 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab1a7fc640 1 -- 172.21.15.105:0/3155834848 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fab2c0046b0 con 0x562754a41130 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab3c3fc740 10 client.21120 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab3c3fc740 10 client.21120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab3c3fc740 20 client.21120 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562754a59960 0x562754a5bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab3c3fc740 4 client.21120 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.116+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3155834848 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627549ad5f0 con 0x562754a59960 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.117+0000 7fab32ffd640 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562754a59960 0x562754a5bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.117+0000 7fab1a7fc640 10 client.21120 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.117+0000 7fab32ffd640 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562754a59960 0x562754a5bdd0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0785529566040d3 server_cookie=6a77ae0004a40174 in_seq=0 out_seq=0 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.118+0000 7fab1a7fc640 1 -- 172.21.15.105:0/3155834848 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5627549ad5f0 con 0x562754a59960 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.118+0000 7fab1a7fc640 10 client.21120 handle_command_reply: tid=0 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.118+0000 7fab3c3fc740 1 client.21120 shutdown 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.118+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3155834848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 msgr2=0x562754a54ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.118+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a54ff0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009bf0 tx=0x7fab2c032de0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3155834848 shutdown_connections 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562754a59960 0x562754a5bdd0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a54ff0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3bfc0 0x562754a4dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/3155834848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3b5d0 0x562754a4da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3155834848 >> 172.21.15.105:0/3155834848 conn(0x562754a36b40 msgr2=0x562754a35df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab1a7fc640 0 client.21120 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3155834848 shutdown_connections 2024-03-26T17:53:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 1 -- 172.21.15.105:0/3155834848 wait complete. 2024-03-26T17:53:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.119+0000 7fab3c3fc740 20 client.21120 trim_cache size 0 max 16384 2024-03-26T17:53:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.148+0000 7fab3c3fc740 1 Processor -- start 2024-03-26T17:53:52.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.148+0000 7fab3c3fc740 1 -- start start 2024-03-26T17:53:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a54fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51666/0 (socket says 172.21.15.105:51666) 2024-03-26T17:53:52.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab32ffd640 1 -- 172.21.15.105:0/2206901428 learned_addr learned my addr 172.21.15.105:0/2206901428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab2bfff640 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a54fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a35560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab337fe640 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a35560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.149+0000 7fab3c3fc740 1 -- 172.21.15.105:0/2206901428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab1400e530 con 0x562754a41130 2024-03-26T17:53:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.150+0000 7fab3c3fc740 1 -- 172.21.15.105:0/2206901428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab140100d0 con 0x562754a3bfc0 2024-03-26T17:53:52.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.150+0000 7fab3c3fc740 1 -- 172.21.15.105:0/2206901428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab1400ec00 con 0x562754a3b5d0 2024-03-26T17:53:52.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:51 smithi105 ceph-mon[32280]: pgmap v814: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-26T17:53:52.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:51 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3748080086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:53:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.150+0000 7fab32ffd640 1 -- 172.21.15.105:0/2206901428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 msgr2=0x562754a54fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.150+0000 7fab32ffd640 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a54fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.150+0000 7fab32ffd640 1 -- 172.21.15.105:0/2206901428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 msgr2=0x562754a35560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.150+0000 7fab32ffd640 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a35560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.150+0000 7fab32ffd640 1 -- 172.21.15.105:0/2206901428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562754a443e0 con 0x562754a3b5d0 2024-03-26T17:53:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.151+0000 7fab2bfff640 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a54fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.151+0000 7fab32ffd640 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3d170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fab2400cfd0 tx=0x7fab240bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ab5956e642b573 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.151+0000 7fab28ff9640 1 -- 172.21.15.105:0/2206901428 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2409b2d0 con 0x562754a3b5d0 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.151+0000 7fab28ff9640 1 -- 172.21.15.105:0/2206901428 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab240bd0b0 con 0x562754a3b5d0 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.151+0000 7fab28ff9640 1 -- 172.21.15.105:0/2206901428 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2409c680 con 0x562754a3b5d0 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.152+0000 7fab3c3fc740 1 -- 172.21.15.105:0/2206901428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 msgr2=0x562754a3d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.152+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3d170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fab2400cfd0 tx=0x7fab240bd040 comp rx=0 tx=0).stop 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.152+0000 7fab3c3fc740 1 -- 172.21.15.105:0/2206901428 shutdown_connections 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.152+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a41130 0x562754a35560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.152+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a3bfc0 0x562754a54fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.152+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/2206901428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a3d170 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.152+0000 7fab3c3fc740 1 -- 172.21.15.105:0/2206901428 >> 172.21.15.105:0/2206901428 conn(0x562754a36b40 msgr2=0x562754a3ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.153+0000 7fab3c3fc740 1 -- 172.21.15.105:0/2206901428 shutdown_connections 2024-03-26T17:53:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.153+0000 7fab3c3fc740 1 -- 172.21.15.105:0/2206901428 wait complete. 2024-03-26T17:53:52.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.154+0000 7fab3c3fc740 1 -- start start 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.154+0000 7fab3c3fc740 1 Processor -- start 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.154+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab3c3fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a3bfc0 0x562754a52ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51672/0 (socket says 172.21.15.105:51672) 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a3bfc0 0x562754a52ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a3bfc0 0x562754a52ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48588/0 (socket says 172.21.15.105:48588) 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab32ffd640 1 -- 172.21.15.105:0/1214849 learned_addr learned my addr 172.21.15.105:0/1214849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a41130 0x562754a53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fab1400fa00 con 0x562754a3bfc0 2024-03-26T17:53:52.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab1400f2d0 con 0x562754a41130 2024-03-26T17:53:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab1400c510 con 0x562754a3b5d0 2024-03-26T17:53:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab3c3fc740 10 client.0 _ll_register_callbacks cb 0x7fab2c009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab337fe640 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a41130 0x562754a53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab2bfff640 1 -- 172.21.15.105:0/1214849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 msgr2=0x562754a529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab2bfff640 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab2bfff640 1 -- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a41130 msgr2=0x562754a53db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.155+0000 7fab2bfff640 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a41130 0x562754a53db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.156+0000 7fab2bfff640 1 -- 172.21.15.105:0/1214849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562754a543a0 con 0x562754a3bfc0 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.156+0000 7fab337fe640 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a41130 0x562754a53db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.156+0000 7fab32ffd640 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.156+0000 7fab2bfff640 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a3bfc0 0x562754a52ee0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009550 tx=0x7fab1c012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98ba990fa26f204 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.156+0000 7fab1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.156+0000 7fab1a7fc640 1 -- 172.21.15.105:0/1214849 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c003db0 con 0x562754a3bfc0 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.156+0000 7fab1a7fc640 1 -- 172.21.15.105:0/1214849 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fab1c003bb0 con 0x562754a3bfc0 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.156+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab240bd0b0 con 0x562754a3bfc0 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab1a7fc640 1 -- 172.21.15.105:0/1214849 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fab1c03b050 con 0x562754a3bfc0 2024-03-26T17:53:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab3c3fc740 10 client.21126 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562754abfe70 con 0x562754a3bfc0 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab1a7fc640 1 -- 172.21.15.105:0/1214849 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab240bd0b0 con 0x562754a3bfc0 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab1a7fc640 1 -- 172.21.15.105:0/1214849 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7fab1c0374d0 con 0x562754a3bfc0 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab3c3fc740 10 client.21126 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab3c3fc740 10 client.21126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab3c3fc740 20 client.21126 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.157+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562754ac05d0 0x562754ac29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.158+0000 7fab3c3fc740 4 client.21126 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.158+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627549a7940 con 0x562754ac05d0 2024-03-26T17:53:52.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.158+0000 7fab32ffd640 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562754ac05d0 0x562754ac29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.161+0000 7fab32ffd640 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562754ac05d0 0x562754ac29f0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd654b8f950efbf server_cookie=c98378088671b37 in_seq=0 out_seq=0 2024-03-26T17:53:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.161+0000 7fab1a7fc640 10 client.21126 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 -- 172.21.15.105:0/3326776359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e81019b0 msgr2=0x7f78e8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 --2- 172.21.15.105:0/3326776359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e81019b0 0x7f78e8101e30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009900 tx=0x7f78d002bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 -- 172.21.15.105:0/3326776359 shutdown_connections 2024-03-26T17:53:52.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 --2- 172.21.15.105:0/3326776359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78e8102370 0x7f78e8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 --2- 172.21.15.105:0/3326776359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e81019b0 0x7f78e8101e30 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 --2- 172.21.15.105:0/3326776359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78e81007c0 0x7f78e8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 -- 172.21.15.105:0/3326776359 >> 172.21.15.105:0/3326776359 conn(0x7f78e80fbd50 msgr2=0x7f78e80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 -- 172.21.15.105:0/3326776359 shutdown_connections 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.172+0000 7f78edf9e640 1 -- 172.21.15.105:0/3326776359 wait complete. 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.173+0000 7f78edf9e640 1 Processor -- start 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.173+0000 7f78edf9e640 1 -- start start 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.173+0000 7f78edf9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78e81007c0 0x7f78e819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.173+0000 7f78edf9e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78e81019b0 0x7f78e819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.173+0000 7f78edf9e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8102370 0x7f78e81a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78edf9e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78e810f600 con 0x7f78e8102370 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78edf9e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78e810f480 con 0x7f78e81019b0 2024-03-26T17:53:52.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78edf9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78e810f780 con 0x7f78e81007c0 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78deffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78e81019b0 0x7f78e819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8102370 0x7f78e81a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8102370 0x7f78e81a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48608/0 (socket says 172.21.15.105:48608) 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e7fff640 1 -- 172.21.15.105:0/884989373 learned_addr learned my addr 172.21.15.105:0/884989373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e77fe640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78e81007c0 0x7f78e819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e77fe640 1 -- 172.21.15.105:0/884989373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78e81019b0 msgr2=0x7f78e819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e77fe640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78e81019b0 0x7f78e819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e77fe640 1 -- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8102370 msgr2=0x7f78e81a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e77fe640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8102370 0x7f78e81a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e77fe640 1 -- 172.21.15.105:0/884989373 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e81a5d10 con 0x7f78e81007c0 2024-03-26T17:53:52.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78deffd640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78e81019b0 0x7f78e819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e7fff640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8102370 0x7f78e81a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e77fe640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78e81007c0 0x7f78e819e0f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f78d8005fd0 tx=0x7f78d800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.174+0000 7f78e57fa640 1 -- 172.21.15.105:0/884989373 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8003970 con 0x7f78e81007c0 2024-03-26T17:53:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.175+0000 7f78edf9e640 1 -- 172.21.15.105:0/884989373 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e81a5fa0 con 0x7f78e81007c0 2024-03-26T17:53:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.175+0000 7f78edf9e640 1 -- 172.21.15.105:0/884989373 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e81a64b0 con 0x7f78e81007c0 2024-03-26T17:53:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.175+0000 7f78e57fa640 1 -- 172.21.15.105:0/884989373 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78d8004030 con 0x7f78e81007c0 2024-03-26T17:53:52.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.175+0000 7f78e57fa640 1 -- 172.21.15.105:0/884989373 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8010720 con 0x7f78e81007c0 2024-03-26T17:53:52.167 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:51 smithi135 ceph-mon[35636]: pgmap v814: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-26T17:53:52.168 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:51 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3748080086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:53:52.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.177+0000 7f78e57fa640 1 -- 172.21.15.105:0/884989373 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f78d80041e0 con 0x7f78e81007c0 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78f0b88740 1 Processor -- start 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78f0b88740 1 -- start start 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b66a30 0x55ad06b66e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b6bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 0x55ad06b6ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78f0b88740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ad06acb970 con 0x55ad06b67420 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78f0b88740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ad06a73d70 con 0x55ad06b66a30 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78f0b88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ad06ac7920 con 0x55ad06b6c590 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e57fa640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f78d40a2f00 0x7f78d40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b66a30 0x55ad06b66e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b66a30 0x55ad06b66e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36542/0 (socket says 172.21.15.105:36542) 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 -- 172.21.15.105:0/2082779730 learned_addr learned my addr 172.21.15.105:0/2082779730 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f78d40a2f00 0x7f78d40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e57fa640 1 -- 172.21.15.105:0/884989373 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f78d80cf8a0 con 0x7f78e81007c0 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78deffd640 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 0x55ad06b6ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e77fe640 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b6bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 -- 172.21.15.105:0/2082779730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 msgr2=0x55ad06b6ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 0x55ad06b6ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 -- 172.21.15.105:0/2082779730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 msgr2=0x55ad06b6bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b6bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.178+0000 7f78e7fff640 1 -- 172.21.15.105:0/2082779730 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad067df3f0 con 0x55ad06b66a30 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.179+0000 7f78e77fe640 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b6bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.179+0000 7f78e7fff640 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b66a30 0x55ad06b66e50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f78e0011fb0 tx=0x7f78e000f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1546eee372f4201 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.179+0000 7f78e7fff640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f78d40a2f00 0x7f78d40a53c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f78e000fd50 tx=0x7f78e001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.179+0000 7f78ddffb640 1 -- 172.21.15.105:0/2082779730 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0022070 con 0x55ad06b66a30 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.179+0000 7f78ddffb640 1 -- 172.21.15.105:0/2082779730 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e000ed60 con 0x55ad06b66a30 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.179+0000 7f78ddffb640 1 -- 172.21.15.105:0/2082779730 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e001e660 con 0x55ad06b66a30 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 -- 172.21.15.105:0/2082779730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b66a30 msgr2=0x55ad06b66e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b66a30 0x55ad06b66e50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f78e0011fb0 tx=0x7f78e000f850 comp rx=0 tx=0).stop 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 -- 172.21.15.105:0/2082779730 shutdown_connections 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 0x55ad06b6ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b6bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2082779730 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b66a30 0x55ad06b66e50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 -- 172.21.15.105:0/2082779730 >> 172.21.15.105:0/2082779730 conn(0x55ad06b61fa0 msgr2=0x55ad06b623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 -- 172.21.15.105:0/2082779730 shutdown_connections 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 -- 172.21.15.105:0/2082779730 wait complete. 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.180+0000 7f78f0b88740 1 -- start start 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78f0b88740 1 Processor -- start 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b67420 0x55ad06b71400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 0x55ad06b78440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78f0b88740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ad06b6f4d0 con 0x55ad06b66a30 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78f0b88740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ad06a87310 con 0x55ad06b67420 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78f0b88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ad06b6f650 con 0x55ad06b6c590 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78f0b88740 10 client.0 _ll_register_callbacks cb 0x55ad06ac2c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78deffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b67420 0x55ad06b71400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78deffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b67420 0x55ad06b71400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36556/0 (socket says 172.21.15.105:36556) 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78deffd640 1 -- 172.21.15.105:0/1929847246 learned_addr learned my addr 172.21.15.105:0/1929847246 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78e7fff640 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 0x55ad06b78440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.181+0000 7f78e77fe640 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78deffd640 1 -- 172.21.15.105:0/1929847246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 msgr2=0x55ad06b78440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78deffd640 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 0x55ad06b78440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78deffd640 1 -- 172.21.15.105:0/1929847246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b66a30 msgr2=0x55ad06b70ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78deffd640 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78deffd640 1 -- 172.21.15.105:0/1929847246 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad068a97d0 con 0x55ad06b67420 2024-03-26T17:53:52.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78deffd640 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b67420 0x55ad06b71400 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f78d002c170 tx=0x7f78d0005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd2f62fdef70e40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78c6ffd640 1 -- 172.21.15.105:0/1929847246 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0005d50 con 0x55ad06b67420 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78f0b88740 1 -- 172.21.15.105:0/1929847246 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78e000ed60 con 0x55ad06b67420 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78c6ffd640 1 -- 172.21.15.105:0/1929847246 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f78d00046c0 con 0x55ad06b67420 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78c6ffd640 1 -- 172.21.15.105:0/1929847246 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78d002cc90 con 0x55ad06b67420 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78f0b88740 10 client.30923 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.182+0000 7f78f0b88740 1 -- 172.21.15.105:0/1929847246 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad06b7b870 con 0x55ad06b67420 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78c6ffd640 1 -- 172.21.15.105:0/1929847246 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78e000ed60 con 0x55ad06b67420 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78c6ffd640 1 -- 172.21.15.105:0/1929847246 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f78d0003880 con 0x55ad06b67420 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78f0b88740 10 client.30923 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78f0b88740 10 client.30923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78f0b88740 20 client.30923 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78f0b88740 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55ad06b7ce30 0x55ad06b7f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78f0b88740 4 client.30923 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:52.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78f0b88740 1 -- 172.21.15.105:0/1929847246 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad06ad8eb0 con 0x55ad06b7ce30 2024-03-26T17:53:52.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.183+0000 7f78e77fe640 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55ad06b7ce30 0x55ad06b7f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.184+0000 7f78c6ffd640 10 client.30923 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:53:52.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.184+0000 7f78e77fe640 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55ad06b7ce30 0x55ad06b7f2a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0043151667565bd server_cookie=72da95c78dd62039 in_seq=0 out_seq=0 2024-03-26T17:53:52.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78c6ffd640 1 -- 172.21.15.105:0/1929847246 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ad06ad8eb0 con 0x55ad06b7ce30 2024-03-26T17:53:52.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78c6ffd640 10 client.30923 handle_command_reply: tid=0 2024-03-26T17:53:52.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 client.30923 shutdown 2024-03-26T17:53:52.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 -- 172.21.15.105:0/1929847246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b67420 msgr2=0x55ad06b71400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b67420 0x55ad06b71400 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f78d002c170 tx=0x7f78d0005250 comp rx=0 tx=0).stop 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 -- 172.21.15.105:0/1929847246 shutdown_connections 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55ad06b7ce30 0x55ad06b7f2a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b6c590 0x55ad06b78440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78c6ffd640 0 client.30923 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b67420 0x55ad06b71400 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 --2- 172.21.15.105:0/1929847246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 -- 172.21.15.105:0/1929847246 >> 172.21.15.105:0/1929847246 conn(0x55ad06b61fa0 msgr2=0x55ad06b611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 -- 172.21.15.105:0/1929847246 shutdown_connections 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 1 -- 172.21.15.105:0/1929847246 wait complete. 2024-03-26T17:53:52.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.185+0000 7f78f0b88740 20 client.30923 trim_cache size 0 max 16384 2024-03-26T17:53:52.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.202+0000 7f78f0b88740 1 Processor -- start 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.202+0000 7f78f0b88740 1 -- start start 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b71400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b78400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78f0b88740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81a6bb0 con 0x55ad06b67420 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78f0b88740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81a8750 con 0x55ad06b6c590 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78deffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b71400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78e77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78e77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51702/0 (socket says 172.21.15.105:51702) 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78e77fe640 1 -- 172.21.15.105:0/2967883324 learned_addr learned my addr 172.21.15.105:0/2967883324 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78f0b88740 1 -- 172.21.15.105:0/2967883324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81a7280 con 0x55ad06b66a30 2024-03-26T17:53:52.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78e7fff640 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b78400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78deffd640 1 -- 172.21.15.105:0/2967883324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 msgr2=0x55ad06b70ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78deffd640 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78deffd640 1 -- 172.21.15.105:0/2967883324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 msgr2=0x55ad06b78400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78deffd640 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b78400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78deffd640 1 -- 172.21.15.105:0/2967883324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad06b6bc00 con 0x55ad06b67420 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.203+0000 7f78e7fff640 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b78400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.204+0000 7f78deffd640 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b71400 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f78d0003e90 tx=0x7f78d002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03d20633940c795 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.204+0000 7f78dd7fa640 1 -- 172.21.15.105:0/2967883324 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0000e60 con 0x55ad06b67420 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.204+0000 7f78dd7fa640 1 -- 172.21.15.105:0/2967883324 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78d002cc90 con 0x55ad06b67420 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.204+0000 7f78dd7fa640 1 -- 172.21.15.105:0/2967883324 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0034a70 con 0x55ad06b67420 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 -- 172.21.15.105:0/2967883324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 msgr2=0x55ad06b71400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b71400 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f78d0003e90 tx=0x7f78d002bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 -- 172.21.15.105:0/2967883324 shutdown_connections 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b78400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b71400 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2967883324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b70ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 -- 172.21.15.105:0/2967883324 >> 172.21.15.105:0/2967883324 conn(0x55ad06b61fa0 msgr2=0x55ad06b62380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 -- 172.21.15.105:0/2967883324 shutdown_connections 2024-03-26T17:53:52.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 -- 172.21.15.105:0/2967883324 wait complete. 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 -- start start 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 Processor -- start 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b75b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.205+0000 7f78f0b88740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78f0b88740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81a8080 con 0x55ad06b67420 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78f0b88740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81a7950 con 0x55ad06b6c590 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78f0b88740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8071ed0 con 0x55ad06b66a30 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78f0b88740 10 client.0 _ll_register_callbacks cb 0x7f78d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b75b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b75b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51710/0 (socket says 172.21.15.105:51710) 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e77fe640 1 -- 172.21.15.105:0/2150640940 learned_addr learned my addr 172.21.15.105:0/2150640940 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e7fff640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78deffd640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e77fe640 1 -- 172.21.15.105:0/2150640940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 msgr2=0x55ad06b770e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e77fe640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b770e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e77fe640 1 -- 172.21.15.105:0/2150640940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 msgr2=0x55ad06b760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e77fe640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b760d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e77fe640 1 -- 172.21.15.105:0/2150640940 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad06b776d0 con 0x55ad06b66a30 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78e7fff640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.206+0000 7f78deffd640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78e77fe640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b75b90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f78d80cd6d0 tx=0x7f78d80cfaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf739cd100ad18fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78c6ffd640 1 -- 172.21.15.105:0/2150640940 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80b4070 con 0x55ad06b66a30 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78f0b88740 1 -- 172.21.15.105:0/2150640940 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d002cc90 con 0x55ad06b66a30 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78c6ffd640 1 -- 172.21.15.105:0/2150640940 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f78d80bc0b0 con 0x55ad06b66a30 2024-03-26T17:53:52.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78c6ffd640 1 -- 172.21.15.105:0/2150640940 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78d80a49a0 con 0x55ad06b66a30 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78f0b88740 10 client.30898 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78f0b88740 1 -- 172.21.15.105:0/2150640940 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ad06be3210 con 0x55ad06b66a30 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.207+0000 7f78c6ffd640 1 -- 172.21.15.105:0/2150640940 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78d8098600 con 0x55ad06b66a30 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.208+0000 7f78c6ffd640 1 -- 172.21.15.105:0/2150640940 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f78d8020020 con 0x55ad06b66a30 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.208+0000 7f78f0b88740 10 client.30898 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.208+0000 7f78f0b88740 10 client.30898 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.208+0000 7f78f0b88740 20 client.30898 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.208+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55ad06be3be0 0x55ad06be6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.208+0000 7f78f0b88740 4 client.30898 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.208+0000 7f78f0b88740 1 -- 172.21.15.105:0/2150640940 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f78d002cc90 con 0x55ad06be3be0 2024-03-26T17:53:52.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.209+0000 7f78deffd640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55ad06be3be0 0x55ad06be6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.209+0000 7f78c6ffd640 10 client.30898 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:53:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.209+0000 7f78deffd640 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55ad06be3be0 0x55ad06be6000 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa59d5a2b64ca466 server_cookie=221b702be655555b in_seq=0 out_seq=0 2024-03-26T17:53:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78c6ffd640 1 -- 172.21.15.105:0/2150640940 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f78d002cc90 con 0x55ad06be3be0 2024-03-26T17:53:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78c6ffd640 10 client.30898 handle_command_reply: tid=0 2024-03-26T17:53:52.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 client.30898 shutdown 2024-03-26T17:53:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 -- 172.21.15.105:0/2150640940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 msgr2=0x55ad06b75b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b75b90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f78d80cd6d0 tx=0x7f78d80cfaa0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 -- 172.21.15.105:0/2150640940 shutdown_connections 2024-03-26T17:53:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55ad06be3be0 0x55ad06be6000 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78c6ffd640 0 client.30898 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:53:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad06b6c590 0x55ad06b770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad06b67420 0x55ad06b760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 --2- 172.21.15.105:0/2150640940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad06b66a30 0x55ad06b75b90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 -- 172.21.15.105:0/2150640940 >> 172.21.15.105:0/2150640940 conn(0x55ad06b61fa0 msgr2=0x55ad06b689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 -- 172.21.15.105:0/2150640940 shutdown_connections 2024-03-26T17:53:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.212+0000 7f78f0b88740 1 -- 172.21.15.105:0/2150640940 wait complete. 2024-03-26T17:53:52.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78f0b88740 20 client.30898 trim_cache size 0 max 16384 2024-03-26T17:53:52.202 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 -- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f78d40a2f00 msgr2=0x7f78d40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f78d40a2f00 0x7f78d40a53c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f78e000fd50 tx=0x7f78e001e000 comp rx=0 tx=0).stop 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 -- 172.21.15.105:0/884989373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78e81007c0 msgr2=0x7f78e819e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78e81007c0 0x7f78e819e0f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f78d8005fd0 tx=0x7f78d800ef90 comp rx=0 tx=0).stop 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 -- 172.21.15.105:0/884989373 shutdown_connections 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f78d40a2f00 0x7f78d40a53c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78e8102370 0x7f78e81a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78e81019b0 0x7f78e819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 --2- 172.21.15.105:0/884989373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78e81007c0 0x7f78e819e0f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.213+0000 7f78edf9e640 1 -- 172.21.15.105:0/884989373 >> 172.21.15.105:0/884989373 conn(0x7f78e80fbd50 msgr2=0x7f78e810c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.214+0000 7f78edf9e640 1 -- 172.21.15.105:0/884989373 shutdown_connections 2024-03-26T17:53:52.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.214+0000 7f78edf9e640 1 -- 172.21.15.105:0/884989373 wait complete. 2024-03-26T17:53:52.217 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:53:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab1a7fc640 1 -- 172.21.15.105:0/1214849 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627549a7940 con 0x562754ac05d0 2024-03-26T17:53:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab1a7fc640 10 client.21126 handle_command_reply: tid=0 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 client.21126 shutdown 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a3bfc0 msgr2=0x562754a52ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a3bfc0 0x562754a52ee0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009550 tx=0x7fab1c012ed0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 shutdown_connections 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562754ac05d0 0x562754ac29f0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562754a41130 0x562754a53db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562754a3bfc0 0x562754a52ee0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 --2- 172.21.15.105:0/1214849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562754a3b5d0 0x562754a529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.353+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 >> 172.21.15.105:0/1214849 conn(0x562754a36b40 msgr2=0x562754a39da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab1a7fc640 0 client.21126 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 shutdown_connections 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab3c3fc740 1 -- 172.21.15.105:0/1214849 wait complete. 2024-03-26T17:53:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab3c3fc740 20 client.21126 trim_cache size 0 max 16384 2024-03-26T17:53:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:53:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:53:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:53:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:53:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab200a3040 msgr2=0x7fab200a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab200a3040 0x7fab200a5500 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fab34197c30 tx=0x7fab1c005f90 comp rx=0 tx=0).stop 2024-03-26T17:53:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab34100820 msgr2=0x7fab34196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab34100820 0x7fab34196720 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fab2400e9c0 tx=0x7fab2400ee80 comp rx=0 tx=0).stop 2024-03-26T17:53:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 shutdown_connections 2024-03-26T17:53:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fab200a3040 0x7fab200a5500 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab341023d0 0x7fab3419dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab34101a10 0x7fab34196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 --2- 172.21.15.105:0/1402785822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fab34100820 0x7fab34196720 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.354+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 >> 172.21.15.105:0/1402785822 conn(0x7fab340fbd90 msgr2=0x7fab34104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.355+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 shutdown_connections 2024-03-26T17:53:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:53:52.355+0000 7fab39737640 1 -- 172.21.15.105:0/1402785822 wait complete. 2024-03-26T17:53:52.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:51 smithi144 ceph-mon[36095]: pgmap v814: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-03-26T17:53:52.347 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:51 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3748080086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:53:52.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 -- 172.21.15.105:0/3893615801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b4102370 msgr2=0x7f87b4108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/3893615801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b4102370 0x7f87b4108c00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009ca0 tx=0x7f87ac0301e0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 -- 172.21.15.105:0/3893615801 shutdown_connections 2024-03-26T17:53:52.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/3893615801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b4102370 0x7f87b4108c00 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/3893615801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87b41019b0 0x7f87b4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/3893615801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b41007c0 0x7f87b4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 -- 172.21.15.105:0/3893615801 >> 172.21.15.105:0/3893615801 conn(0x7f87b40fbd50 msgr2=0x7f87b40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 -- 172.21.15.105:0/3893615801 shutdown_connections 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.373+0000 7f87b9f5b640 1 -- 172.21.15.105:0/3893615801 wait complete. 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.374+0000 7f87b9f5b640 1 Processor -- start 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.374+0000 7f87b9f5b640 1 -- start start 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b9f5b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87b41007c0 0x7f87b41956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b9f5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b41019b0 0x7f87b4195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b9f5b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4102370 0x7f87b419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b9f5b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410b200 con 0x7f87b41019b0 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b9f5b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410b080 con 0x7f87b4102370 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b9f5b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87b410b380 con 0x7f87b41007c0 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b41019b0 0x7f87b4195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b41019b0 0x7f87b4195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48664/0 (socket says 172.21.15.105:48664) 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1087955660 learned_addr learned my addr 172.21.15.105:0/1087955660 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b3fff640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4102370 0x7f87b419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b37fe640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87b41007c0 0x7f87b41956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b3fff640 1 -- 172.21.15.105:0/1087955660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87b41007c0 msgr2=0x7f87b41956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b3fff640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87b41007c0 0x7f87b41956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b3fff640 1 -- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b41019b0 msgr2=0x7f87b4195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b3fff640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b41019b0 0x7f87b4195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b3fff640 1 -- 172.21.15.105:0/1087955660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b419d2d0 con 0x7f87b4102370 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b37fe640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87b41007c0 0x7f87b41956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.375+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b41019b0 0x7f87b4195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.376+0000 7f87b3fff640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4102370 0x7f87b419cc60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009bf0 tx=0x7f87ac039ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.376+0000 7f87b0ff9640 1 -- 172.21.15.105:0/1087955660 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac03d040 con 0x7f87b4102370 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.376+0000 7f87b9f5b640 1 -- 172.21.15.105:0/1087955660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b419d560 con 0x7f87b4102370 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.376+0000 7f87b0ff9640 1 -- 172.21.15.105:0/1087955660 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87ac048070 con 0x7f87b4102370 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.376+0000 7f87b0ff9640 1 -- 172.21.15.105:0/1087955660 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac004510 con 0x7f87b4102370 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.376+0000 7f87b9f5b640 1 -- 172.21.15.105:0/1087955660 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b419da70 con 0x7f87b4102370 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87b0ff9640 1 -- 172.21.15.105:0/1087955660 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f87ac0046b0 con 0x7f87b4102370 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87bcc21740 1 Processor -- start 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87bcc21740 1 -- start start 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 0x555d3fa3bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa3c590 0x555d3fa3ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87bcc21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555d3f99b970 con 0x555d3fa36a30 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87bcc21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d3f943d70 con 0x555d3fa3c590 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87bcc21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d3f997920 con 0x555d3fa37420 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.379+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa36e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa36e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48672/0 (socket says 172.21.15.105:48672) 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 -- 172.21.15.105:0/43616653 learned_addr learned my addr 172.21.15.105:0/43616653 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b0ff9640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f879c0a2f20 0x7f879c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa3c590 0x555d3fa3ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b37fe640 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 0x555d3fa3bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b3fff640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f879c0a2f20 0x7f879c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 -- 172.21.15.105:0/43616653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 msgr2=0x555d3fa3bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 0x555d3fa3bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 -- 172.21.15.105:0/43616653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa3c590 msgr2=0x555d3fa3ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa3c590 0x555d3fa3ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 -- 172.21.15.105:0/43616653 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d3f6af3f0 con 0x555d3fa36a30 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa36e50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f87a8013ce0 tx=0x7f87a8009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e424e38dd31eeb8e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.380+0000 7f8799ffb640 1 -- 172.21.15.105:0/43616653 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8013170 con 0x555d3fa36a30 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.381+0000 7f87b0ff9640 1 -- 172.21.15.105:0/1087955660 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f87ac0f5c60 con 0x7f87b4102370 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.381+0000 7f87b3fff640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f879c0a2f20 0x7f879c0a53e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0f3770 tx=0x7f87ac0b7830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.381+0000 7f8799ffb640 1 -- 172.21.15.105:0/43616653 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87a8013310 con 0x555d3fa36a30 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.381+0000 7f8799ffb640 1 -- 172.21.15.105:0/43616653 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8010d00 con 0x555d3fa36a30 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.381+0000 7f87bcc21740 1 -- 172.21.15.105:0/43616653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 msgr2=0x555d3fa36e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.381+0000 7f87bcc21740 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa36e50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f87a8013ce0 tx=0x7f87a8009c10 comp rx=0 tx=0).stop 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 -- 172.21.15.105:0/43616653 shutdown_connections 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa3c590 0x555d3fa3ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 0x555d3fa3bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 --2- 172.21.15.105:0/43616653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa36e50 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 -- 172.21.15.105:0/43616653 >> 172.21.15.105:0/43616653 conn(0x555d3fa31fa0 msgr2=0x555d3fa323a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 -- 172.21.15.105:0/43616653 shutdown_connections 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 -- 172.21.15.105:0/43616653 wait complete. 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 -- start start 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.382+0000 7f87bcc21740 1 Processor -- start 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa36a30 0x555d3fa40fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 0x555d3fa414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa48520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87bcc21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555d3fa3f540 con 0x555d3fa3c590 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87bcc21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d3f957310 con 0x555d3fa36a30 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87bcc21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555d3fa3f6c0 con 0x555d3fa37420 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa36a30 0x555d3fa40fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa36a30 0x555d3fa40fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36600/0 (socket says 172.21.15.105:36600) 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87b2ffd640 1 -- 172.21.15.105:0/3456379296 learned_addr learned my addr 172.21.15.105:0/3456379296 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa48520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87b37fe640 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 0x555d3fa414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.383+0000 7f87bcc21740 10 client.0 _ll_register_callbacks cb 0x555d3f992c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87b2ffd640 1 -- 172.21.15.105:0/3456379296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 msgr2=0x555d3fa414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 0x555d3fa414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87b2ffd640 1 -- 172.21.15.105:0/3456379296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 msgr2=0x555d3fa48520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa48520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87b2ffd640 1 -- 172.21.15.105:0/3456379296 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d3f7797d0 con 0x555d3fa36a30 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa36a30 0x555d3fa40fa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87a80132d0 tx=0x7f87a8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e626ccb1e1f9be05 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f8792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f8792ffd640 1 -- 172.21.15.105:0/3456379296 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8011ed0 con 0x555d3fa36a30 2024-03-26T17:53:52.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f8792ffd640 1 -- 172.21.15.105:0/3456379296 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f87a8004030 con 0x555d3fa36a30 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87bcc21740 1 -- 172.21.15.105:0/3456379296 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a8013310 con 0x555d3fa36a30 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f8792ffd640 1 -- 172.21.15.105:0/3456379296 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f87a8034050 con 0x555d3fa36a30 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87bcc21740 10 client.30941 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.384+0000 7f87bcc21740 1 -- 172.21.15.105:0/3456379296 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555d3fa4b850 con 0x555d3fa36a30 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f8792ffd640 1 -- 172.21.15.105:0/3456379296 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87a8013310 con 0x555d3fa36a30 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f8792ffd640 1 -- 172.21.15.105:0/3456379296 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f87a8003b30 con 0x555d3fa36a30 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f87bcc21740 10 client.30941 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f87bcc21740 10 client.30941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f87bcc21740 20 client.30941 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f87bcc21740 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555d3fa4cd20 0x555d3fa4f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f87bcc21740 4 client.30941 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:52.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f87bcc21740 1 -- 172.21.15.105:0/3456379296 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d3f9a8eb0 con 0x555d3fa4cd20 2024-03-26T17:53:52.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.385+0000 7f87b37fe640 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555d3fa4cd20 0x555d3fa4f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.386+0000 7f87b37fe640 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555d3fa4cd20 0x555d3fa4f170 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c25a0c084491a7 server_cookie=490cf498beea0f1c in_seq=0 out_seq=0 2024-03-26T17:53:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.386+0000 7f8792ffd640 10 client.30941 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.387+0000 7f8792ffd640 1 -- 172.21.15.105:0/3456379296 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555d3f9a8eb0 con 0x555d3fa4cd20 2024-03-26T17:53:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.387+0000 7f8792ffd640 10 client.30941 handle_command_reply: tid=0 2024-03-26T17:53:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.387+0000 7f87bcc21740 1 client.30941 shutdown 2024-03-26T17:53:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 -- 172.21.15.105:0/3456379296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa36a30 msgr2=0x555d3fa40fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa36a30 0x555d3fa40fa0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87a80132d0 tx=0x7f87a8009260 comp rx=0 tx=0).stop 2024-03-26T17:53:52.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 -- 172.21.15.105:0/3456379296 shutdown_connections 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555d3fa4cd20 0x555d3fa4f170 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa48520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa37420 0x555d3fa414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 --2- 172.21.15.105:0/3456379296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa36a30 0x555d3fa40fa0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f8792ffd640 0 client.30941 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 -- 172.21.15.105:0/3456379296 >> 172.21.15.105:0/3456379296 conn(0x555d3fa31fa0 msgr2=0x555d3fa311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 -- 172.21.15.105:0/3456379296 shutdown_connections 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 1 -- 172.21.15.105:0/3456379296 wait complete. 2024-03-26T17:53:52.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.388+0000 7f87bcc21740 20 client.30941 trim_cache size 0 max 16384 2024-03-26T17:53:52.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87bcc21740 1 Processor -- start 2024-03-26T17:53:52.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87bcc21740 1 -- start start 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa3c590 0x555d3fa30930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87bcc21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a68d0 con 0x555d3fa36a30 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87bcc21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a8470 con 0x555d3fa37420 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87bcc21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a6fa0 con 0x555d3fa3c590 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa412c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48684/0 (socket says 172.21.15.105:48684) 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1154228534 learned_addr learned my addr 172.21.15.105:0/1154228534 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87b37fe640 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.409+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa3c590 0x555d3fa30930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1154228534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa3c590 msgr2=0x555d3fa30930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa3c590 0x555d3fa30930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1154228534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 msgr2=0x555d3fa484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1154228534 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d3fa2d820 con 0x555d3fa36a30 2024-03-26T17:53:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87b37fe640 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa412c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5294eea817c4f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87997fa640 1 -- 172.21.15.105:0/1154228534 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a800fcc0 con 0x555d3fa36a30 2024-03-26T17:53:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87997fa640 1 -- 172.21.15.105:0/1154228534 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87a800fe60 con 0x555d3fa36a30 2024-03-26T17:53:52.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.410+0000 7f87997fa640 1 -- 172.21.15.105:0/1154228534 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8005110 con 0x555d3fa36a30 2024-03-26T17:53:52.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 -- 172.21.15.105:0/1154228534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 msgr2=0x555d3fa412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa412c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a8005d30 comp rx=0 tx=0).stop 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 -- 172.21.15.105:0/1154228534 shutdown_connections 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa3c590 0x555d3fa30930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1154228534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa36a30 0x555d3fa412c0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 -- 172.21.15.105:0/1154228534 >> 172.21.15.105:0/1154228534 conn(0x555d3fa31fa0 msgr2=0x555d3fa40bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 -- 172.21.15.105:0/1154228534 shutdown_connections 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.411+0000 7f87bcc21740 1 -- 172.21.15.105:0/1154228534 wait complete. 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 1 -- start start 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 1 Processor -- start 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa36a30 0x555d3fa48340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa44e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa45390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa44e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa45390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa44e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36618/0 (socket says 172.21.15.105:36618) 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa45390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48694/0 (socket says 172.21.15.105:48694) 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a7da0 con 0x555d3fa3c590 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1163386925 learned_addr learned my addr 172.21.15.105:0/1163386925 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a7670 con 0x555d3fa37420 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41a48b0 con 0x555d3fa36a30 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87bcc21740 10 client.0 _ll_register_callbacks cb 0x7f87a8013770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.412+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa36a30 0x555d3fa48340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1163386925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa36a30 msgr2=0x555d3fa48340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa36a30 0x555d3fa48340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1163386925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 msgr2=0x555d3fa44e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa44e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87b2ffd640 1 -- 172.21.15.105:0/1163386925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d3fab98b0 con 0x555d3fa3c590 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87b2ffd640 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa45390 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f87a8020500 tx=0x7f87a8011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10c405d327d98cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f8792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f8792ffd640 1 -- 172.21.15.105:0/1163386925 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8040020 con 0x555d3fa3c590 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f8792ffd640 1 -- 172.21.15.105:0/1163386925 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f87a80132a0 con 0x555d3fa3c590 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a800fe60 con 0x555d3fa3c590 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f8792ffd640 1 -- 172.21.15.105:0/1163386925 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f87a80134a0 con 0x555d3fa3c590 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87bcc21740 10 client.21162 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.413+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555d3fabaab0 con 0x555d3fa3c590 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f8792ffd640 1 -- 172.21.15.105:0/1163386925 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87a80055c0 con 0x555d3fa3c590 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f8792ffd640 1 -- 172.21.15.105:0/1163386925 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f87a80035f0 con 0x555d3fa3c590 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f87bcc21740 10 client.21162 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:52.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f87bcc21740 10 client.21162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:52.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f87bcc21740 20 client.21162 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:52.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555d3fabb2d0 0x555d3fabd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:52.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f87bcc21740 4 client.21162 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:53:52.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f87a800fe60 con 0x555d3fabb2d0 2024-03-26T17:53:52.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f87b37fe640 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555d3fabb2d0 0x555d3fabd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:52.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.414+0000 7f8792ffd640 10 client.21162 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:52.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.415+0000 7f87b37fe640 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555d3fabb2d0 0x555d3fabd6f0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b85ebaf3c52b62e server_cookie=a1f72f90cecd6498 in_seq=0 out_seq=0 2024-03-26T17:53:52.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.415+0000 7f8792ffd640 1 -- 172.21.15.105:0/1163386925 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f87a800fe60 con 0x555d3fabb2d0 2024-03-26T17:53:52.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.415+0000 7f8792ffd640 10 client.21162 handle_command_reply: tid=0 2024-03-26T17:53:52.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 client.21162 shutdown 2024-03-26T17:53:52.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 msgr2=0x555d3fa45390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa45390 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f87a8020500 tx=0x7f87a8011fd0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 shutdown_connections 2024-03-26T17:53:52.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555d3fabb2d0 0x555d3fabd6f0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555d3fa3c590 0x555d3fa45390 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d3fa37420 0x555d3fa44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 --2- 172.21.15.105:0/1163386925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555d3fa36a30 0x555d3fa48340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 >> 172.21.15.105:0/1163386925 conn(0x555d3fa31fa0 msgr2=0x555d3fa35200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f8792ffd640 0 client.21162 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:52.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 shutdown_connections 2024-03-26T17:53:52.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.416+0000 7f87bcc21740 1 -- 172.21.15.105:0/1163386925 wait complete. 2024-03-26T17:53:52.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87bcc21740 20 client.21162 trim_cache size 0 max 16384 2024-03-26T17:53:52.409 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 -- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f879c0a2f20 msgr2=0x7f879c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f879c0a2f20 0x7f879c0a53e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0f3770 tx=0x7f87ac0b7830 comp rx=0 tx=0).stop 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 -- 172.21.15.105:0/1087955660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4102370 msgr2=0x7f87b419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4102370 0x7f87b419cc60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009bf0 tx=0x7f87ac039ce0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 -- 172.21.15.105:0/1087955660 shutdown_connections 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f879c0a2f20 0x7f879c0a53e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4102370 0x7f87b419cc60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87b41019b0 0x7f87b4195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 --2- 172.21.15.105:0/1087955660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87b41007c0 0x7f87b41956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:52.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.417+0000 7f87b9f5b640 1 -- 172.21.15.105:0/1087955660 >> 172.21.15.105:0/1087955660 conn(0x7f87b40fbd50 msgr2=0x7f87b4104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:52.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.418+0000 7f87b9f5b640 1 -- 172.21.15.105:0/1087955660 shutdown_connections 2024-03-26T17:53:52.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:52.418+0000 7f87b9f5b640 1 -- 172.21.15.105:0/1087955660 wait complete. 2024-03-26T17:53:52.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 20481 7.08 MB/sec execute 35 sec latency 174.950 ms 2024-03-26T17:53:53.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:52 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:53:53.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:52 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:53:53.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:52 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:53:53.422 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:53:53.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:53:53.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:53:53.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 20625 6.93 MB/sec execute 36 sec latency 106.466 ms 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 -- 172.21.15.105:0/3863437907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 msgr2=0x7f2760100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 --2- 172.21.15.105:0/3863437907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 0x7f2760100f70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2750009900 tx=0x7f275002bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 -- 172.21.15.105:0/3863437907 shutdown_connections 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 --2- 172.21.15.105:0/3863437907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27601014b0 0x7f2760101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 --2- 172.21.15.105:0/3863437907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 0x7f2760100f70 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 --2- 172.21.15.105:0/3863437907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27601001b0 0x7f27601005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 -- 172.21.15.105:0/3863437907 >> 172.21.15.105:0/3863437907 conn(0x7f27600fbdf0 msgr2=0x7f27600fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 -- 172.21.15.105:0/3863437907 shutdown_connections 2024-03-26T17:53:53.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.581+0000 7f2768063640 1 -- 172.21.15.105:0/3863437907 wait complete. 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2768063640 1 Processor -- start 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2768063640 1 -- start start 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2768063640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27601001b0 0x7f2760199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2768063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 0x7f276019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2768063640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27601014b0 0x7f27601a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2768063640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f276010f5c0 con 0x7f2760100b10 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2768063640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f276010f440 con 0x7f27601014b0 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2768063640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f276010f740 con 0x7f27601001b0 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f27655fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 0x7f276019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2765dff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27601001b0 0x7f2760199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2765dff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27601001b0 0x7f2760199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51798/0 (socket says 172.21.15.105:51798) 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2765dff640 1 -- 172.21.15.105:0/2550311373 learned_addr learned my addr 172.21.15.105:0/2550311373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2766600640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27601014b0 0x7f27601a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f27655fe640 1 -- 172.21.15.105:0/2550311373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27601001b0 msgr2=0x7f2760199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f27655fe640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27601001b0 0x7f2760199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f27655fe640 1 -- 172.21.15.105:0/2550311373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27601014b0 msgr2=0x7f27601a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f27655fe640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27601014b0 0x7f27601a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f27655fe640 1 -- 172.21.15.105:0/2550311373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27601a1960 con 0x7f2760100b10 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.582+0000 7f2765dff640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27601001b0 0x7f2760199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.583+0000 7f27655fe640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 0x7f276019a290 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2750000c00 tx=0x7f2750005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.583+0000 7f2756ffd640 1 -- 172.21.15.105:0/2550311373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750043020 con 0x7f2760100b10 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.583+0000 7f2756ffd640 1 -- 172.21.15.105:0/2550311373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2750004350 con 0x7f2760100b10 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.583+0000 7f2768063640 1 -- 172.21.15.105:0/2550311373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27601a1bf0 con 0x7f2760100b10 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.583+0000 7f2756ffd640 1 -- 172.21.15.105:0/2550311373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750003b50 con 0x7f2760100b10 2024-03-26T17:53:53.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.583+0000 7f2768063640 1 -- 172.21.15.105:0/2550311373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27601a2190 con 0x7f2760100b10 2024-03-26T17:53:53.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f276ac48740 1 Processor -- start 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f276ac48740 1 -- start start 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f276ac48740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a3c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f276ac48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3cb80 0x55bb56a41680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f276ac48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56a44180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f276ac48740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bb569a1630 con 0x55bb56a3c190 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f276ac48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bb56949e10 con 0x55bb56a41cf0 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f276ac48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bb5699d5e0 con 0x55bb56a3cb80 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f2756ffd640 1 -- 172.21.15.105:0/2550311373 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2750046020 con 0x7f2760100b10 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f2765dff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3cb80 0x55bb56a41680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f2765dff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3cb80 0x55bb56a41680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51800/0 (socket says 172.21.15.105:51800) 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f2766600640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a3c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.586+0000 7f2765dff640 1 -- 172.21.15.105:0/3090684370 learned_addr learned my addr 172.21.15.105:0/3090684370 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f27655fe640 1 --2- 172.21.15.105:0/3090684370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56a44180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2766600640 1 -- 172.21.15.105:0/3090684370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3cb80 msgr2=0x55bb56a41680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2766600640 1 --2- 172.21.15.105:0/3090684370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3cb80 0x55bb56a41680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2766600640 1 -- 172.21.15.105:0/3090684370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 msgr2=0x55bb56a44180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2766600640 1 --2- 172.21.15.105:0/3090684370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56a44180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2766600640 1 -- 172.21.15.105:0/3090684370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb56960740 con 0x55bb56a3c190 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2766600640 1 --2- 172.21.15.105:0/3090684370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a3c5b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f2758009ed0 tx=0x7f27580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da93756ca3c3649 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2756ffd640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f274c0a2fa0 0x7f274c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f274bfff640 1 -- 172.21.15.105:0/3090684370 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758003b70 con 0x55bb56a3c190 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2765dff640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f274c0a2fa0 0x7f274c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f2756ffd640 1 -- 172.21.15.105:0/2550311373 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f27500f1520 con 0x7f2760100b10 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f274bfff640 1 -- 172.21.15.105:0/3090684370 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2758015070 con 0x55bb56a3c190 2024-03-26T17:53:53.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.587+0000 7f274bfff640 1 -- 172.21.15.105:0/3090684370 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580117b0 con 0x55bb56a3c190 2024-03-26T17:53:53.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.588+0000 7f276ac48740 1 -- 172.21.15.105:0/3090684370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 msgr2=0x55bb56a3c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.588+0000 7f2765dff640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f274c0a2fa0 0x7f274c0a5460 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55bb56a3cf80 tx=0x7f275c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.588+0000 7f276ac48740 1 --2- 172.21.15.105:0/3090684370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a3c5b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f2758009ed0 tx=0x7f27580107a0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.588+0000 7f276ac48740 1 -- 172.21.15.105:0/3090684370 shutdown_connections 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.588+0000 7f276ac48740 1 --2- 172.21.15.105:0/3090684370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56a44180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.588+0000 7f276ac48740 1 --2- 172.21.15.105:0/3090684370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3cb80 0x55bb56a41680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.588+0000 7f276ac48740 1 --2- 172.21.15.105:0/3090684370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a3c5b0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.588+0000 7f276ac48740 1 -- 172.21.15.105:0/3090684370 >> 172.21.15.105:0/3090684370 conn(0x55bb56a37700 msgr2=0x55bb56a37b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.589+0000 7f276ac48740 1 -- 172.21.15.105:0/3090684370 shutdown_connections 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.589+0000 7f276ac48740 1 -- 172.21.15.105:0/3090684370 wait complete. 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.589+0000 7f276ac48740 1 -- start start 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.589+0000 7f276ac48740 1 Processor -- start 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f276ac48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3c190 0x55bb56a467a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f276ac48740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56a46ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f276ac48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a4dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f276ac48740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bb56a44ca0 con 0x55bb56a3cb80 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f276ac48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bb5695d1a0 con 0x55bb56a3c190 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f276ac48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bb56a44e20 con 0x55bb56a41cf0 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f276ac48740 10 client.0 _ll_register_callbacks cb 0x55bb566b4c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f27655fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56a46ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3c190 0x55bb56a467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3c190 0x55bb56a467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36658/0 (socket says 172.21.15.105:36658) 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f27655fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56a46ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48746/0 (socket says 172.21.15.105:48746) 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 -- 172.21.15.105:0/1997562061 learned_addr learned my addr 172.21.15.105:0/1997562061 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2766600640 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a4dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 -- 172.21.15.105:0/1997562061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 msgr2=0x55bb56a4dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a4dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 -- 172.21.15.105:0/1997562061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 msgr2=0x55bb56a46ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56a46ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 -- 172.21.15.105:0/1997562061 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5677d7e0 con 0x55bb56a3c190 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2766600640 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a4dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f2765dff640 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3c190 0x55bb56a467a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f275c00f850 tx=0x7f275c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527474586d344642 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.590+0000 7f27497fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f27497fa640 1 -- 172.21.15.105:0/1997562061 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c011b50 con 0x55bb56a3c190 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f276ac48740 1 -- 172.21.15.105:0/1997562061 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2758015070 con 0x55bb56a3c190 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f27497fa640 1 -- 172.21.15.105:0/1997562061 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f275c011cf0 con 0x55bb56a3c190 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f27497fa640 1 -- 172.21.15.105:0/1997562061 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f275c005020 con 0x55bb56a3c190 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f276ac48740 10 client.30959 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f276ac48740 1 -- 172.21.15.105:0/1997562061 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bb567ddf70 con 0x55bb56a3c190 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f27497fa640 1 -- 172.21.15.105:0/1997562061 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2758015070 con 0x55bb56a3c190 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f27497fa640 1 -- 172.21.15.105:0/1997562061 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f275c027070 con 0x55bb56a3c190 2024-03-26T17:53:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f276ac48740 10 client.30959 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f276ac48740 10 client.30959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f276ac48740 20 client.30959 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.591+0000 7f276ac48740 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb56a5b1d0 0x55bb56a5d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.592+0000 7f276ac48740 4 client.30959 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.592+0000 7f276ac48740 1 -- 172.21.15.105:0/1997562061 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb569aeb70 con 0x55bb56a5b1d0 2024-03-26T17:53:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.592+0000 7f27655fe640 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb56a5b1d0 0x55bb56a5d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.592+0000 7f27497fa640 10 client.30959 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.592+0000 7f27655fe640 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb56a5b1d0 0x55bb56a5d670 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f6476ebfc08665 server_cookie=8c0868a947ff1708 in_seq=0 out_seq=0 2024-03-26T17:53:53.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f27497fa640 1 -- 172.21.15.105:0/1997562061 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bb569aeb70 con 0x55bb56a5b1d0 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f27497fa640 10 client.30959 handle_command_reply: tid=0 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 client.30959 shutdown 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 -- 172.21.15.105:0/1997562061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3c190 msgr2=0x55bb56a467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3c190 0x55bb56a467a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f275c00f850 tx=0x7f275c004730 comp rx=0 tx=0).stop 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 -- 172.21.15.105:0/1997562061 shutdown_connections 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb56a5b1d0 0x55bb56a5d670 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a4dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f27497fa640 0 client.30959 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56a46ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 --2- 172.21.15.105:0/1997562061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3c190 0x55bb56a467a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 -- 172.21.15.105:0/1997562061 >> 172.21.15.105:0/1997562061 conn(0x55bb56a37700 msgr2=0x55bb56a36900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 -- 172.21.15.105:0/1997562061 shutdown_connections 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.594+0000 7f276ac48740 1 -- 172.21.15.105:0/1997562061 wait complete. 2024-03-26T17:53:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.595+0000 7f276ac48740 20 client.30959 trim_cache size 0 max 16384 2024-03-26T17:53:53.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f276ac48740 1 Processor -- start 2024-03-26T17:53:53.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f276ac48740 1 -- start start 2024-03-26T17:53:53.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f276ac48740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f276ac48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3cb80 0x55bb56a468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f276ac48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a36070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f276ac48740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27601a6c60 con 0x55bb56a3c190 2024-03-26T17:53:53.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f276ac48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27601a8800 con 0x55bb56a3cb80 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f276ac48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27601a7330 con 0x55bb56a41cf0 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f2765dff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f2765dff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48754/0 (socket says 172.21.15.105:48754) 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f2765dff640 1 -- 172.21.15.105:0/419336936 learned_addr learned my addr 172.21.15.105:0/419336936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f27655fe640 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3cb80 0x55bb56a468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f2766600640 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a36070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f2765dff640 1 -- 172.21.15.105:0/419336936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 msgr2=0x55bb56a36070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f2765dff640 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a36070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.623+0000 7f2765dff640 1 -- 172.21.15.105:0/419336936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3cb80 msgr2=0x55bb56a468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.624+0000 7f2765dff640 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3cb80 0x55bb56a468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.624+0000 7f2765dff640 1 -- 172.21.15.105:0/419336936 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb56675170 con 0x55bb56a3c190 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.624+0000 7f27655fe640 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3cb80 0x55bb56a468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.624+0000 7f2765dff640 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a463b0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f275c019040 tx=0x7f275c00f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b90e93d1f3f48cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.624+0000 7f274b7fe640 1 -- 172.21.15.105:0/419336936 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c037030 con 0x55bb56a3c190 2024-03-26T17:53:53.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.624+0000 7f274b7fe640 1 -- 172.21.15.105:0/419336936 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f275c0051c0 con 0x55bb56a3c190 2024-03-26T17:53:53.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.624+0000 7f274b7fe640 1 -- 172.21.15.105:0/419336936 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c004ad0 con 0x55bb56a3c190 2024-03-26T17:53:53.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 -- 172.21.15.105:0/419336936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 msgr2=0x55bb56a463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a463b0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f275c019040 tx=0x7f275c00f410 comp rx=0 tx=0).stop 2024-03-26T17:53:53.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 -- 172.21.15.105:0/419336936 shutdown_connections 2024-03-26T17:53:53.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a41cf0 0x55bb56a36070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a3cb80 0x55bb56a468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 --2- 172.21.15.105:0/419336936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3c190 0x55bb56a463b0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 -- 172.21.15.105:0/419336936 >> 172.21.15.105:0/419336936 conn(0x55bb56a37700 msgr2=0x55bb56a37ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 -- 172.21.15.105:0/419336936 shutdown_connections 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.625+0000 7f276ac48740 1 -- 172.21.15.105:0/419336936 wait complete. 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 1 -- start start 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 1 Processor -- start 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3c190 0x55bb56abb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56abdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56ac0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f27601a8130 con 0x55bb56a3cb80 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27601a7a00 con 0x55bb56a41cf0 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27601a4c40 con 0x55bb56a3c190 2024-03-26T17:53:53.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.626+0000 7f276ac48740 10 client.0 _ll_register_callbacks cb 0x7f275c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f27655fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56abdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2766600640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56ac0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2765dff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3c190 0x55bb56abb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f27655fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56abdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48766/0 (socket says 172.21.15.105:48766) 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f27655fe640 1 -- 172.21.15.105:0/168587800 learned_addr learned my addr 172.21.15.105:0/168587800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2766600640 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56ac0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36682/0 (socket says 172.21.15.105:36682) 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2766600640 1 -- 172.21.15.105:0/168587800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3c190 msgr2=0x55bb56abb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2766600640 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3c190 0x55bb56abb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2766600640 1 -- 172.21.15.105:0/168587800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 msgr2=0x55bb56abdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2766600640 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56abdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2766600640 1 -- 172.21.15.105:0/168587800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb56ac09a0 con 0x55bb56a41cf0 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2765dff640 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3c190 0x55bb56abb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:53.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f27655fe640 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56abdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f2766600640 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56ac0170 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2758000f80 tx=0x7f2758009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6f429f0741f3f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f27497fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f27497fa640 1 -- 172.21.15.105:0/168587800 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758005540 con 0x55bb56a41cf0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f27497fa640 1 -- 172.21.15.105:0/168587800 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f27580056e0 con 0x55bb56a41cf0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.627+0000 7f276ac48740 1 -- 172.21.15.105:0/168587800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f275c0051c0 con 0x55bb56a41cf0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f27497fa640 1 -- 172.21.15.105:0/168587800 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f27580058e0 con 0x55bb56a41cf0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f276ac48740 10 client.30965 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f276ac48740 1 -- 172.21.15.105:0/168587800 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bb56ac1ce0 con 0x55bb56a41cf0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f27497fa640 1 -- 172.21.15.105:0/168587800 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f275c0051c0 con 0x55bb56a41cf0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f27497fa640 1 -- 172.21.15.105:0/168587800 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f2758033570 con 0x55bb56a41cf0 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f276ac48740 10 client.30965 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f276ac48740 10 client.30965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f276ac48740 20 client.30965 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:53.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f276ac48740 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb56a5b1d0 0x55bb56ac4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f276ac48740 4 client.30965 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:53:53.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f276ac48740 1 -- 172.21.15.105:0/168587800 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bb569a8ec0 con 0x55bb56a5b1d0 2024-03-26T17:53:53.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.628+0000 7f2765dff640 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb56a5b1d0 0x55bb56ac4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.629+0000 7f27497fa640 10 client.30965 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:53.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.629+0000 7f2765dff640 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb56a5b1d0 0x55bb56ac4410 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d5fe6edb0e4fafe server_cookie=46935fd6d0a5c67e in_seq=0 out_seq=0 2024-03-26T17:53:53.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f27497fa640 1 -- 172.21.15.105:0/168587800 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb569a8ec0 con 0x55bb56a5b1d0 2024-03-26T17:53:53.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f27497fa640 10 client.30965 handle_command_reply: tid=0 2024-03-26T17:53:53.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 client.30965 shutdown 2024-03-26T17:53:53.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 -- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 msgr2=0x55bb56ac0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56ac0170 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2758000f80 tx=0x7f2758009cd0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 -- 172.21.15.105:0/168587800 shutdown_connections 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bb56a5b1d0 0x55bb56ac4410 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb56a41cf0 0x55bb56ac0170 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb56a3cb80 0x55bb56abdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 --2- 172.21.15.105:0/168587800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb56a3c190 0x55bb56abb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 -- 172.21.15.105:0/168587800 >> 172.21.15.105:0/168587800 conn(0x55bb56a37700 msgr2=0x55bb56a46320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f27497fa640 0 client.30965 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 -- 172.21.15.105:0/168587800 shutdown_connections 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.631+0000 7f276ac48740 1 -- 172.21.15.105:0/168587800 wait complete. 2024-03-26T17:53:53.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.632+0000 7f276ac48740 20 client.30965 trim_cache size 0 max 16384 2024-03-26T17:53:53.625 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:53:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:53:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "f439887a-6e31-4b32-aece-5ae6b2ae0e47", 2024-03-26T17:53:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:53:53.625 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:53:53.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.632+0000 7f2768063640 1 -- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f274c0a2fa0 msgr2=0x7f274c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.632+0000 7f2768063640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f274c0a2fa0 0x7f274c0a5460 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55bb56a3cf80 tx=0x7f275c005f90 comp rx=0 tx=0).stop 2024-03-26T17:53:53.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.632+0000 7f2768063640 1 -- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 msgr2=0x7f276019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 0x7f276019a290 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2750000c00 tx=0x7f2750005dc0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 -- 172.21.15.105:0/2550311373 shutdown_connections 2024-03-26T17:53:53.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f274c0a2fa0 0x7f274c0a5460 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27601014b0 0x7f27601a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2760100b10 0x7f276019a290 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 --2- 172.21.15.105:0/2550311373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27601001b0 0x7f2760199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 -- 172.21.15.105:0/2550311373 >> 172.21.15.105:0/2550311373 conn(0x7f27600fbdf0 msgr2=0x7f27600fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 -- 172.21.15.105:0/2550311373 shutdown_connections 2024-03-26T17:53:53.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.633+0000 7f2768063640 1 -- 172.21.15.105:0/2550311373 wait complete. 2024-03-26T17:53:53.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:53:53.636 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 -- 172.21.15.105:0/2673071761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81007c0 msgr2=0x7f0ae8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/2673071761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81007c0 0x7f0ae8100be0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009900 tx=0x7f0ad802bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 -- 172.21.15.105:0/2673071761 shutdown_connections 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/2673071761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ae8102370 0x7f0ae8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/2673071761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae81019b0 0x7f0ae8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/2673071761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81007c0 0x7f0ae8100be0 secure :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009900 tx=0x7f0ad802bc30 comp rx=0 tx=0).stop 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 -- 172.21.15.105:0/2673071761 >> 172.21.15.105:0/2673071761 conn(0x7f0ae80fbd50 msgr2=0x7f0ae80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 -- 172.21.15.105:0/2673071761 shutdown_connections 2024-03-26T17:53:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.791+0000 7f0aed0a1640 1 -- 172.21.15.105:0/2673071761 wait complete. 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.792+0000 7f0aed0a1640 1 Processor -- start 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.792+0000 7f0aed0a1640 1 -- start start 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.792+0000 7f0aed0a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81019b0 0x7f0ae8195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.792+0000 7f0aed0a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae8102370 0x7f0ae8196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.792+0000 7f0aed0a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ae819b0e0 0x7f0ae819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.792+0000 7f0aed0a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae810b240 con 0x7f0ae81019b0 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.792+0000 7f0aed0a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae810b0c0 con 0x7f0ae8102370 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.792+0000 7f0aed0a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae810b3c0 con 0x7f0ae819b0e0 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ae819b0e0 0x7f0ae819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ae819b0e0 0x7f0ae819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51852/0 (socket says 172.21.15.105:51852) 2024-03-26T17:53:53.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 -- 172.21.15.105:0/3266201734 learned_addr learned my addr 172.21.15.105:0/3266201734 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81019b0 0x7f0ae8195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ade59c640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae8102370 0x7f0ae8196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 -- 172.21.15.105:0/3266201734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae8102370 msgr2=0x7f0ae8196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae8102370 0x7f0ae8196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 -- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81019b0 msgr2=0x7f0ae8195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81019b0 0x7f0ae8195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 -- 172.21.15.105:0/3266201734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae819db40 con 0x7f0ae819b0e0 2024-03-26T17:53:53.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81019b0 0x7f0ae8195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:53.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.793+0000 7f0ae759e640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ae819b0e0 0x7f0ae819d4d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000eb10 tx=0x7f0ae000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.794+0000 7f0ae4d99640 1 -- 172.21.15.105:0/3266201734 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0004b20 con 0x7f0ae819b0e0 2024-03-26T17:53:53.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.794+0000 7f0ae4d99640 1 -- 172.21.15.105:0/3266201734 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ae0004cc0 con 0x7f0ae819b0e0 2024-03-26T17:53:53.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.794+0000 7f0aed0a1640 1 -- 172.21.15.105:0/3266201734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae819de30 con 0x7f0ae819b0e0 2024-03-26T17:53:53.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.794+0000 7f0ae4d99640 1 -- 172.21.15.105:0/3266201734 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0010690 con 0x7f0ae819b0e0 2024-03-26T17:53:53.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.794+0000 7f0aed0a1640 1 -- 172.21.15.105:0/3266201734 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae819e2f0 con 0x7f0ae819b0e0 2024-03-26T17:53:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.797+0000 7f0aefd69740 1 Processor -- start 2024-03-26T17:53:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0ae4d99640 1 -- 172.21.15.105:0/3266201734 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0ae0005020 con 0x7f0ae819b0e0 2024-03-26T17:53:53.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0aefd69740 1 -- start start 2024-03-26T17:53:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133c8f0 0x56405133cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133d2e0 0x564051341de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051342450 0x5640513448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0aefd69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5640512a1980 con 0x56405133d2e0 2024-03-26T17:53:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0aefd69740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564051249be0 con 0x564051342450 2024-03-26T17:53:53.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0aefd69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56405129d930 con 0x56405133c8f0 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133c8f0 0x56405133cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133c8f0 0x56405133cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51862/0 (socket says 172.21.15.105:51862) 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0ade59c640 1 -- 172.21.15.105:0/35408223 learned_addr learned my addr 172.21.15.105:0/35408223 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133d2e0 0x564051341de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.798+0000 7f0ade59c640 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051342450 0x5640513448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ade59c640 1 -- 172.21.15.105:0/35408223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051342450 msgr2=0x5640513448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ae4d99640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ad40a2fa0 0x7f0ad40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ade59c640 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051342450 0x5640513448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ade59c640 1 -- 172.21.15.105:0/35408223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133d2e0 msgr2=0x564051341de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ade59c640 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133d2e0 0x564051341de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ade59c640 1 -- 172.21.15.105:0/35408223 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405107f7d0 con 0x56405133c8f0 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133d2e0 0x564051341de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ae759e640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ad40a2fa0 0x7f0ad40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ade59c640 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133c8f0 0x56405133cd10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00198d0 tx=0x7f0ad0009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc9f7d33632efe2e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0ae4d99640 1 -- 172.21.15.105:0/3266201734 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0ae00cf9b0 con 0x7f0ae819b0e0 2024-03-26T17:53:53.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0add59a640 1 -- 172.21.15.105:0/35408223 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0010850 con 0x56405133c8f0 2024-03-26T17:53:53.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0add59a640 1 -- 172.21.15.105:0/35408223 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad0017450 con 0x56405133c8f0 2024-03-26T17:53:53.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.799+0000 7f0add59a640 1 -- 172.21.15.105:0/35408223 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0010ea0 con 0x56405133c8f0 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.800+0000 7f0aefd69740 1 -- 172.21.15.105:0/35408223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133c8f0 msgr2=0x56405133cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.800+0000 7f0aefd69740 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133c8f0 0x56405133cd10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00198d0 tx=0x7f0ad0009700 comp rx=0 tx=0).stop 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.800+0000 7f0ae759e640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ad40a2fa0 0x7f0ad40a5460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0ae001c000 tx=0x7f0ae006edd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 -- 172.21.15.105:0/35408223 shutdown_connections 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051342450 0x5640513448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133d2e0 0x564051341de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 --2- 172.21.15.105:0/35408223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133c8f0 0x56405133cd10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 -- 172.21.15.105:0/35408223 >> 172.21.15.105:0/35408223 conn(0x564051337e60 msgr2=0x564051338260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 -- 172.21.15.105:0/35408223 shutdown_connections 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 -- 172.21.15.105:0/35408223 wait complete. 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 -- start start 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 Processor -- start 2024-03-26T17:53:53.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133c8f0 0x564051346b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.801+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x564051347070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564051342450 0x56405134e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0aefd69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564051345400 con 0x564051342450 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0aefd69740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56405125d160 con 0x56405133c8f0 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0aefd69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564051345580 con 0x56405133d2e0 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0aefd69740 10 client.0 _ll_register_callbacks cb 0x564051299720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564051342450 0x56405134e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ae6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x564051347070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564051342450 0x56405134e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48794/0 (socket says 172.21.15.105:48794) 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 -- 172.21.15.105:0/4182913083 learned_addr learned my addr 172.21.15.105:0/4182913083 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133c8f0 0x564051346b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 -- 172.21.15.105:0/4182913083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 msgr2=0x564051347070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x564051347070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 -- 172.21.15.105:0/4182913083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133c8f0 msgr2=0x564051346b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133c8f0 0x564051346b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 -- 172.21.15.105:0/4182913083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564050fb53f0 con 0x564051342450 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ade59c640 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564051342450 0x56405134e0b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009d60 tx=0x7f0ad001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9902e5e4367cca0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ace7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.802+0000 7f0ace7fc640 1 -- 172.21.15.105:0/4182913083 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0017810 con 0x564051342450 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 1 -- 172.21.15.105:0/4182913083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad0017450 con 0x564051342450 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0ace7fc640 1 -- 172.21.15.105:0/4182913083 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0ad00179b0 con 0x564051342450 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0ace7fc640 1 -- 172.21.15.105:0/4182913083 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0ad003b050 con 0x564051342450 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 10 client.21192 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 1 -- 172.21.15.105:0/4182913083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564050f750f0 con 0x564051342450 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0ace7fc640 1 -- 172.21.15.105:0/4182913083 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad0017450 con 0x564051342450 2024-03-26T17:53:53.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0ace7fc640 1 -- 172.21.15.105:0/4182913083 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f0ad0023070 con 0x564051342450 2024-03-26T17:53:53.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 10 client.21192 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:53.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 10 client.21192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:53.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 20 client.21192 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:53.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564051352ca0 0x564051355100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 4 client.21192 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:53:53.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.803+0000 7f0aefd69740 1 -- 172.21.15.105:0/4182913083 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640512aeec0 con 0x564051352ca0 2024-03-26T17:53:53.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.804+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564051352ca0 0x564051355100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.805+0000 7f0ace7fc640 10 client.21192 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:53.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.805+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564051352ca0 0x564051355100 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9a4543ea0c91f2 server_cookie=690267da95ea7d in_seq=0 out_seq=0 2024-03-26T17:53:53.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.806+0000 7f0ace7fc640 1 -- 172.21.15.105:0/4182913083 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5640512aeec0 con 0x564051352ca0 2024-03-26T17:53:53.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.806+0000 7f0ace7fc640 10 client.21192 handle_command_reply: tid=0 2024-03-26T17:53:53.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.806+0000 7f0aefd69740 1 client.21192 shutdown 2024-03-26T17:53:53.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 -- 172.21.15.105:0/4182913083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564051342450 msgr2=0x56405134e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564051342450 0x56405134e0b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009d60 tx=0x7f0ad001b040 comp rx=0 tx=0).stop 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 -- 172.21.15.105:0/4182913083 shutdown_connections 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564051352ca0 0x564051355100 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564051342450 0x56405134e0b0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x564051347070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 --2- 172.21.15.105:0/4182913083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133c8f0 0x564051346b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 -- 172.21.15.105:0/4182913083 >> 172.21.15.105:0/4182913083 conn(0x564051337e60 msgr2=0x564051337080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0ace7fc640 0 client.21192 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 -- 172.21.15.105:0/4182913083 shutdown_connections 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 1 -- 172.21.15.105:0/4182913083 wait complete. 2024-03-26T17:53:53.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.807+0000 7f0aefd69740 20 client.21192 trim_cache size 0 max 16384 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0aefd69740 1 Processor -- start 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0aefd69740 1 -- start start 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x5640513367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x56405134e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051336d30 0x5640513371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0aefd69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81a7230 con 0x56405133c8f0 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0aefd69740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81a8dd0 con 0x564051336d30 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0aefd69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81a7900 con 0x56405133d2e0 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x5640513367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x5640513367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48806/0 (socket says 172.21.15.105:48806) 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0ade59c640 1 -- 172.21.15.105:0/179857246 learned_addr learned my addr 172.21.15.105:0/179857246 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0ade59c640 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051336d30 0x5640513371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.834+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x56405134e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0ade59c640 1 -- 172.21.15.105:0/179857246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 msgr2=0x56405134e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0ade59c640 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x56405134e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0ade59c640 1 -- 172.21.15.105:0/179857246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051336d30 msgr2=0x5640513371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0ade59c640 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051336d30 0x5640513371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0ade59c640 1 -- 172.21.15.105:0/179857246 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405133d080 con 0x56405133c8f0 2024-03-26T17:53:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x56405134e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:53:53.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0ade59c640 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x5640513367f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c00 tx=0x7f0ad0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5d01c2a0cccd48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0adcd99640 1 -- 172.21.15.105:0/179857246 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0025480 con 0x56405133c8f0 2024-03-26T17:53:53.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0adcd99640 1 -- 172.21.15.105:0/179857246 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad0025970 con 0x56405133c8f0 2024-03-26T17:53:53.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.835+0000 7f0adcd99640 1 -- 172.21.15.105:0/179857246 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00052a0 con 0x56405133c8f0 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.836+0000 7f0aefd69740 1 -- 172.21.15.105:0/179857246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 msgr2=0x5640513367f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.836+0000 7f0aefd69740 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x5640513367f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0000c00 tx=0x7f0ad0005fd0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 -- 172.21.15.105:0/179857246 shutdown_connections 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564051336d30 0x5640513371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56405133d2e0 0x56405134e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 --2- 172.21.15.105:0/179857246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x5640513367f0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 -- 172.21.15.105:0/179857246 >> 172.21.15.105:0/179857246 conn(0x564051337e60 msgr2=0x564051338240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 -- 172.21.15.105:0/179857246 shutdown_connections 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 -- 172.21.15.105:0/179857246 wait complete. 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 -- start start 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 Processor -- start 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564051336d30 0x56405134bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x56405134c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133d2e0 0x56405134d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81a8700 con 0x56405133c8f0 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81a7fd0 con 0x56405133d2e0 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae81a5210 con 0x564051336d30 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0aefd69740 10 client.0 _ll_register_callbacks cb 0x7f0ae00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564051336d30 0x56405134bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.837+0000 7f0ae6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x56405134c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ae6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x56405134c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48822/0 (socket says 172.21.15.105:48822) 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ade59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564051336d30 0x56405134bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51882/0 (socket says 172.21.15.105:51882) 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ae6d9d640 1 -- 172.21.15.105:0/1721703584 learned_addr learned my addr 172.21.15.105:0/1721703584 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ade59c640 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133d2e0 0x56405134d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ade59c640 1 -- 172.21.15.105:0/1721703584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133d2e0 msgr2=0x56405134d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ade59c640 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133d2e0 0x56405134d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ade59c640 1 -- 172.21.15.105:0/1721703584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 msgr2=0x56405134c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ade59c640 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x56405134c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ade59c640 1 -- 172.21.15.105:0/1721703584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56405134d780 con 0x564051336d30 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x56405134c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ade59c640 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564051336d30 0x56405134bd20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0001eb0 tx=0x7f0ad0005990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3932851964b0b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ace7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ace7fc640 1 -- 172.21.15.105:0/1721703584 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0017150 con 0x564051336d30 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0aefd69740 1 -- 172.21.15.105:0/1721703584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad0025970 con 0x564051336d30 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.838+0000 7f0ace7fc640 1 -- 172.21.15.105:0/1721703584 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0ad00172f0 con 0x564051336d30 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0ace7fc640 1 -- 172.21.15.105:0/1721703584 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0ad0005040 con 0x564051336d30 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0aefd69740 10 client.30946 fetch_fsmap learned FSMap version 35 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0aefd69740 1 -- 172.21.15.105:0/1721703584 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5640513c1cb0 con 0x564051336d30 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0ace7fc640 1 -- 172.21.15.105:0/1721703584 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ad0025970 con 0x564051336d30 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0ace7fc640 1 -- 172.21.15.105:0/1721703584 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 35) ==== 8362+0+0 (secure 0 0 0) 0x7f0ad0023040 con 0x564051336d30 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0aefd69740 10 client.30946 fetch_fsmap finished waiting for FSMap version 35 2024-03-26T17:53:53.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0aefd69740 10 client.30946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:53:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0aefd69740 20 client.30946 populate_metadata read hostname 'smithi105' 2024-03-26T17:53:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0aefd69740 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5640513c2480 0x5640513c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:53:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0aefd69740 4 client.30946 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:53:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0aefd69740 1 -- 172.21.15.105:0/1721703584 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640512a9210 con 0x5640513c2480 2024-03-26T17:53:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.839+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5640513c2480 0x5640513c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:53:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.840+0000 7f0ace7fc640 10 client.30946 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:53.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.840+0000 7f0ae6d9d640 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5640513c2480 0x5640513c48a0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed70890bb8ac4583 server_cookie=df48429e04fdd2d7 in_seq=0 out_seq=0 2024-03-26T17:53:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0ace7fc640 1 -- 172.21.15.105:0/1721703584 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5640512a9210 con 0x5640513c2480 2024-03-26T17:53:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0ace7fc640 10 client.30946 handle_command_reply: tid=0 2024-03-26T17:53:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 client.30946 shutdown 2024-03-26T17:53:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 -- 172.21.15.105:0/1721703584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564051336d30 msgr2=0x56405134bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564051336d30 0x56405134bd20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0001eb0 tx=0x7f0ad0005990 comp rx=0 tx=0).stop 2024-03-26T17:53:53.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 -- 172.21.15.105:0/1721703584 shutdown_connections 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5640513c2480 0x5640513c48a0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56405133d2e0 0x56405134d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56405133c8f0 0x56405134c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0ace7fc640 0 client.30946 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 --2- 172.21.15.105:0/1721703584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564051336d30 0x56405134bd20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 -- 172.21.15.105:0/1721703584 >> 172.21.15.105:0/1721703584 conn(0x564051337e60 msgr2=0x5640513371b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 -- 172.21.15.105:0/1721703584 shutdown_connections 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 1 -- 172.21.15.105:0/1721703584 wait complete. 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.842+0000 7f0aefd69740 20 client.30946 trim_cache size 0 max 16384 2024-03-26T17:53:53.834 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (202 inodes in the stack)", 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "f439887a-6e31-4b32-aece-5ae6b2ae0e47": { 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "f439887a-6e31-4b32-aece-5ae6b2ae0e47", 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:53:53.835 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:53:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 -- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ad40a2fa0 msgr2=0x7f0ad40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ad40a2fa0 0x7f0ad40a5460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0ae001c000 tx=0x7f0ae006edd0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 -- 172.21.15.105:0/3266201734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ae819b0e0 msgr2=0x7f0ae819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ae819b0e0 0x7f0ae819d4d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000eb10 tx=0x7f0ae000c2d0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 -- 172.21.15.105:0/3266201734 shutdown_connections 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ad40a2fa0 0x7f0ad40a5460 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ae819b0e0 0x7f0ae819d4d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0ae8102370 0x7f0ae8196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 --2- 172.21.15.105:0/3266201734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ae81019b0 0x7f0ae8195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.843+0000 7f0aed0a1640 1 -- 172.21.15.105:0/3266201734 >> 172.21.15.105:0/3266201734 conn(0x7f0ae80fbd50 msgr2=0x7f0ae8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.844+0000 7f0aed0a1640 1 -- 172.21.15.105:0/3266201734 shutdown_connections 2024-03-26T17:53:53.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:53:53.844+0000 7f0aed0a1640 1 -- 172.21.15.105:0/3266201734 wait complete. 2024-03-26T17:53:53.847 INFO:tasks.cephfs.filesystem:scrub status for tag:f439887a-6e31-4b32-aece-5ae6b2ae0e47 - {'path': '/', 'tag': 'f439887a-6e31-4b32-aece-5ae6b2ae0e47', 'options': 'recursive,force'} 2024-03-26T17:53:54.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:53 smithi105 ceph-mon[32280]: pgmap v815: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.9 MiB/s wr, 298 op/s 2024-03-26T17:53:54.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:53 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:53:54.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:54.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:54.139 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:54.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:53 smithi135 ceph-mon[35636]: pgmap v815: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.9 MiB/s wr, 298 op/s 2024-03-26T17:53:54.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:53 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:53:54.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:54.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:54.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:54.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:53 smithi144 ceph-mon[36095]: pgmap v815: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 2.9 MiB/s wr, 298 op/s 2024-03-26T17:53:54.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:53 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:53:54.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:54.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:54.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:54.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 20760 6.83 MB/sec execute 37 sec latency 163.956 ms 2024-03-26T17:53:55.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:55.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:55.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:55.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:53:55.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.163 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:54 smithi135 ceph-mon[35636]: pgmap v816: 97 pgs: 97 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-03-26T17:53:55.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:55.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:55.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:55.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:53:55.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:54 smithi144 ceph-mon[36095]: pgmap v816: 97 pgs: 97 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-03-26T17:53:55.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:53:55.389 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:54 smithi105 ceph-mon[32280]: pgmap v816: 97 pgs: 97 active+clean; 247 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 2.7 MiB/s wr, 278 op/s 2024-03-26T17:53:55.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 20907 6.77 MB/sec execute 38 sec latency 117.212 ms 2024-03-26T17:53:56.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 21053 6.82 MB/sec execute 39 sec latency 103.583 ms 2024-03-26T17:53:57.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 21218 6.91 MB/sec execute 40 sec latency 118.314 ms 2024-03-26T17:53:58.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:57 smithi105 ceph-mon[32280]: pgmap v817: 97 pgs: 97 active+clean; 245 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-03-26T17:53:58.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:57 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:53:58.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:57 smithi135 ceph-mon[35636]: pgmap v817: 97 pgs: 97 active+clean; 245 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-03-26T17:53:58.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:57 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:53:58.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:57 smithi144 ceph-mon[36095]: pgmap v817: 97 pgs: 97 active+clean; 245 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-03-26T17:53:58.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:57 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:53:58.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 21328 6.84 MB/sec execute 41 sec latency 126.690 ms 2024-03-26T17:53:59.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 21481 6.98 MB/sec execute 42 sec latency 117.856 ms 2024-03-26T17:54:00.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:53:59 smithi105 ceph-mon[32280]: pgmap v818: 97 pgs: 97 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-03-26T17:54:00.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:53:59 smithi135 ceph-mon[35636]: pgmap v818: 97 pgs: 97 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-03-26T17:54:00.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:53:59 smithi144 ceph-mon[36095]: pgmap v818: 97 pgs: 97 active+clean; 246 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-03-26T17:54:00.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 21624 7.09 MB/sec execute 43 sec latency 170.732 ms 2024-03-26T17:54:01.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 21815 7.32 MB/sec execute 44 sec latency 197.799 ms 2024-03-26T17:54:02.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:01 smithi105 ceph-mon[32280]: pgmap v819: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.6 MiB/s wr, 276 op/s 2024-03-26T17:54:02.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:01 smithi135 ceph-mon[35636]: pgmap v819: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.6 MiB/s wr, 276 op/s 2024-03-26T17:54:02.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:01 smithi144 ceph-mon[36095]: pgmap v819: 97 pgs: 97 active+clean; 249 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 2.6 MiB/s wr, 276 op/s 2024-03-26T17:54:02.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 21968 7.41 MB/sec execute 45 sec latency 145.697 ms 2024-03-26T17:54:03.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:02 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:54:03.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:02 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:54:03.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:02 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:54:03.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 22116 7.53 MB/sec execute 46 sec latency 136.160 ms 2024-03-26T17:54:04.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:03 smithi105 ceph-mon[32280]: pgmap v820: 97 pgs: 97 active+clean; 254 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-26T17:54:04.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:03 smithi135 ceph-mon[35636]: pgmap v820: 97 pgs: 97 active+clean; 254 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-26T17:54:04.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:03 smithi144 ceph-mon[36095]: pgmap v820: 97 pgs: 97 active+clean; 254 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-03-26T17:54:04.550 INFO:tasks.workunit.client.0.smithi105.stdout: 10 22216 7.46 MB/sec execute 47 sec latency 126.148 ms 2024-03-26T17:54:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 -- 172.21.15.105:0/431057484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608c1019d0 msgr2=0x7f608c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 --2- 172.21.15.105:0/431057484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608c1019d0 0x7f608c101e50 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f6080009900 tx=0x7f608002bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 -- 172.21.15.105:0/431057484 shutdown_connections 2024-03-26T17:54:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 --2- 172.21.15.105:0/431057484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608c102390 0x7f608c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 --2- 172.21.15.105:0/431057484 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608c1019d0 0x7f608c101e50 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 --2- 172.21.15.105:0/431057484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f608c1007e0 0x7f608c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 -- 172.21.15.105:0/431057484 >> 172.21.15.105:0/431057484 conn(0x7f608c0fbd50 msgr2=0x7f608c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 -- 172.21.15.105:0/431057484 shutdown_connections 2024-03-26T17:54:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.471+0000 7f6092427640 1 -- 172.21.15.105:0/431057484 wait complete. 2024-03-26T17:54:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f6092427640 1 Processor -- start 2024-03-26T17:54:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f6092427640 1 -- start start 2024-03-26T17:54:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f6092427640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608c1007e0 0x7f608c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f6092427640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f608c1019d0 0x7f608c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f6092427640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608c102390 0x7f608c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f608bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608c1007e0 0x7f608c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f608bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608c1007e0 0x7f608c197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44398/0 (socket says 172.21.15.105:44398) 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f608b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f608c1019d0 0x7f608c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f608bfff640 1 -- 172.21.15.105:0/2801121350 learned_addr learned my addr 172.21.15.105:0/2801121350 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f608c10c2f0 con 0x7f608c102390 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f608c10c170 con 0x7f608c1019d0 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f608c10c470 con 0x7f608c1007e0 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.472+0000 7f60909c4640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608c102390 0x7f608c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f608b7fe640 1 -- 172.21.15.105:0/2801121350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608c1007e0 msgr2=0x7f608c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f608b7fe640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608c1007e0 0x7f608c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f608b7fe640 1 -- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608c102390 msgr2=0x7f608c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f608b7fe640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608c102390 0x7f608c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f608b7fe640 1 -- 172.21.15.105:0/2801121350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608c19f6c0 con 0x7f608c1019d0 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f608bfff640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608c1007e0 0x7f608c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f608b7fe640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f608c1019d0 0x7f608c197fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6080000c00 tx=0x7f6080005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f60897fa640 1 -- 172.21.15.105:0/2801121350 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080043020 con 0x7f608c1019d0 2024-03-26T17:54:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f60897fa640 1 -- 172.21.15.105:0/2801121350 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6080004350 con 0x7f608c1019d0 2024-03-26T17:54:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f60897fa640 1 -- 172.21.15.105:0/2801121350 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080003ae0 con 0x7f608c1019d0 2024-03-26T17:54:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608c19f950 con 0x7f608c1019d0 2024-03-26T17:54:05.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.473+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608c19fe50 con 0x7f608c1019d0 2024-03-26T17:54:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.476+0000 7f609500f740 1 Processor -- start 2024-03-26T17:54:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.476+0000 7f609500f740 1 -- start start 2024-03-26T17:54:05.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.476+0000 7f60897fa640 1 -- 172.21.15.105:0/2801121350 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6080046020 con 0x7f608c1019d0 2024-03-26T17:54:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f609500f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f609500f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac031ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f609500f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac0345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f609500f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561fabf910b0 con 0x561fac02c5d0 2024-03-26T17:54:05.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f609500f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561fabf39960 con 0x561fac02cfc0 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f609500f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561fabf8d060 con 0x561fac032130 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f60897fa640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f60740a2fc0 0x7f60740a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f60909c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f60909c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47224/0 (socket says 172.21.15.105:47224) 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f60909c4640 1 -- 172.21.15.105:0/2568028539 learned_addr learned my addr 172.21.15.105:0/2568028539 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f608bfff640 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac031ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.477+0000 7f608bfff640 1 -- 172.21.15.105:0/2568028539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 msgr2=0x561fac0345c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f608b7fe640 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac0345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f608bfff640 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac0345c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f608bfff640 1 -- 172.21.15.105:0/2568028539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 msgr2=0x561fac02c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f608bfff640 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f608bfff640 1 -- 172.21.15.105:0/2568028539 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fabf4bc60 con 0x561fac02cfc0 2024-03-26T17:54:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f60909c4640 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f60909c4640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f60740a2fc0 0x7f60740a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f608bfff640 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac031ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f607c0114c0 tx=0x7f607c011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1a382b60532e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f60727fc640 1 -- 172.21.15.105:0/2568028539 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c005020 con 0x561fac02cfc0 2024-03-26T17:54:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f60727fc640 1 -- 172.21.15.105:0/2568028539 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f607c0047a0 con 0x561fac02cfc0 2024-03-26T17:54:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f608b7fe640 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac0345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:54:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.478+0000 7f60727fc640 1 -- 172.21.15.105:0/2568028539 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c00f7e0 con 0x561fac02cfc0 2024-03-26T17:54:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.479+0000 7f60897fa640 1 -- 172.21.15.105:0/2801121350 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f60800f1480 con 0x7f608c1019d0 2024-03-26T17:54:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.479+0000 7f60909c4640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f60740a2fc0 0x7f60740a5480 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x561fac02d820 tx=0x7f6084004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.479+0000 7f609500f740 1 -- 172.21.15.105:0/2568028539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 msgr2=0x561fac031ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.479+0000 7f609500f740 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac031ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f607c0114c0 tx=0x7f607c011980 comp rx=0 tx=0).stop 2024-03-26T17:54:05.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 -- 172.21.15.105:0/2568028539 shutdown_connections 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac0345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac031ac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 --2- 172.21.15.105:0/2568028539 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 -- 172.21.15.105:0/2568028539 >> 172.21.15.105:0/2568028539 conn(0x561fac027b40 msgr2=0x561fac027f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 -- 172.21.15.105:0/2568028539 shutdown_connections 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 -- 172.21.15.105:0/2568028539 wait complete. 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 -- start start 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.480+0000 7f609500f740 1 Processor -- start 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f609500f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02c5d0 0x561fac036370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f609500f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02cfc0 0x561fac0368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f609500f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac03d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f609500f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561fac0350e0 con 0x561fac02cfc0 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f609500f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561fabf4cdb0 con 0x561fac02c5d0 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f609500f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561fac035260 con 0x561fac032130 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02cfc0 0x561fac0368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02cfc0 0x561fac0368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47228/0 (socket says 172.21.15.105:47228) 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02c5d0 0x561fac036370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608b7fe640 1 -- 172.21.15.105:0/607857318 learned_addr learned my addr 172.21.15.105:0/607857318 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f609500f740 10 client.0 _ll_register_callbacks cb 0x561fabf51110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608b7fe640 1 -- 172.21.15.105:0/607857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 msgr2=0x561fac03d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f60909c4640 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac03d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608b7fe640 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac03d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608b7fe640 1 -- 172.21.15.105:0/607857318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02c5d0 msgr2=0x561fac036370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608b7fe640 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02c5d0 0x561fac036370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608b7fe640 1 -- 172.21.15.105:0/607857318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fabca5210 con 0x561fac02cfc0 2024-03-26T17:54:05.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.481+0000 7f608bfff640 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02c5d0 0x561fac036370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.482+0000 7f608b7fe640 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02cfc0 0x561fac0368b0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f60800b9fd0 tx=0x7f60800b9ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9911f2a16e5b2fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.482+0000 7f606b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.482+0000 7f606b7fe640 1 -- 172.21.15.105:0/607857318 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800c67d0 con 0x561fac02cfc0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.482+0000 7f609500f740 1 -- 172.21.15.105:0/607857318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f607c0047a0 con 0x561fac02cfc0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.482+0000 7f606b7fe640 1 -- 172.21.15.105:0/607857318 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f60800c6b50 con 0x561fac02cfc0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.482+0000 7f606b7fe640 1 -- 172.21.15.105:0/607857318 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f60800be270 con 0x561fac02cfc0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.482+0000 7f609500f740 10 client.21210 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.482+0000 7f609500f740 1 -- 172.21.15.105:0/607857318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561fabd71620 con 0x561fac02cfc0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.483+0000 7f606b7fe640 1 -- 172.21.15.105:0/607857318 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f607c0047a0 con 0x561fac02cfc0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.483+0000 7f606b7fe640 1 -- 172.21.15.105:0/607857318 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f60800f10f0 con 0x561fac02cfc0 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f609500f740 10 client.21210 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f609500f740 10 client.21210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:05.475 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f609500f740 20 client.21210 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f609500f740 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561fac0416f0 0x561fac041b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f609500f740 4 client.21210 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f609500f740 1 -- 172.21.15.105:0/607857318 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fabf9e5f0 con 0x561fac0416f0 2024-03-26T17:54:05.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f608bfff640 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561fac0416f0 0x561fac041b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f606b7fe640 10 client.21210 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.484+0000 7f608bfff640 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561fac0416f0 0x561fac041b80 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ccd2f2bb2810ce server_cookie=ac9d6f60f583934a in_seq=0 out_seq=0 2024-03-26T17:54:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.485+0000 7f606b7fe640 1 -- 172.21.15.105:0/607857318 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561fabf9e5f0 con 0x561fac0416f0 2024-03-26T17:54:05.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.485+0000 7f606b7fe640 10 client.21210 handle_command_reply: tid=0 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.485+0000 7f609500f740 1 client.21210 shutdown 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 -- 172.21.15.105:0/607857318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02cfc0 msgr2=0x561fac0368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02cfc0 0x561fac0368b0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f60800b9fd0 tx=0x7f60800b9ed0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 -- 172.21.15.105:0/607857318 shutdown_connections 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561fac0416f0 0x561fac041b80 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac03d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02cfc0 0x561fac0368b0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 --2- 172.21.15.105:0/607857318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02c5d0 0x561fac036370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 -- 172.21.15.105:0/607857318 >> 172.21.15.105:0/607857318 conn(0x561fac027b40 msgr2=0x561fac026df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f606b7fe640 0 client.21210 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 -- 172.21.15.105:0/607857318 shutdown_connections 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 1 -- 172.21.15.105:0/607857318 wait complete. 2024-03-26T17:54:05.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.486+0000 7f609500f740 20 client.21210 trim_cache size 0 max 16384 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f609500f740 1 Processor -- start 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f609500f740 1 -- start start 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f609500f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f609500f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f609500f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac026560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f609500f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1a69a0 con 0x561fac02c5d0 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f609500f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1a84e0 con 0x561fac02cfc0 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f609500f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1a7070 con 0x561fac032130 2024-03-26T17:54:05.499 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f608bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.509+0000 7f60909c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac026560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f60909c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac026560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44440/0 (socket says 172.21.15.105:44440) 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f60909c4640 1 -- 172.21.15.105:0/2324365098 learned_addr learned my addr 172.21.15.105:0/2324365098 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f608b7fe640 1 --2- 172.21.15.105:0/2324365098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f60909c4640 1 -- 172.21.15.105:0/2324365098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 msgr2=0x561fac03d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f60909c4640 1 --2- 172.21.15.105:0/2324365098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f60909c4640 1 -- 172.21.15.105:0/2324365098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 msgr2=0x561fac02e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f60909c4640 1 --2- 172.21.15.105:0/2324365098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f60909c4640 1 -- 172.21.15.105:0/2324365098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fac0288b0 con 0x561fac032130 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f60909c4640 1 --2- 172.21.15.105:0/2324365098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac026560 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f60840104b0 tx=0x7f6084013030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be97e63552b8bca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f6071ffb640 1 -- 172.21.15.105:0/2324365098 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608401c670 con 0x561fac032130 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f6071ffb640 1 -- 172.21.15.105:0/2324365098 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f608401ccc0 con 0x561fac032130 2024-03-26T17:54:05.500 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.510+0000 7f6071ffb640 1 -- 172.21.15.105:0/2324365098 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6084029c70 con 0x561fac032130 2024-03-26T17:54:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 -- 172.21.15.105:0/2324365098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 msgr2=0x561fac026560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 --2- 172.21.15.105:0/2324365098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac026560 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f60840104b0 tx=0x7f6084013030 comp rx=0 tx=0).stop 2024-03-26T17:54:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 -- 172.21.15.105:0/2324365098 shutdown_connections 2024-03-26T17:54:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 --2- 172.21.15.105:0/2324365098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac032130 0x561fac026560 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 --2- 172.21.15.105:0/2324365098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 --2- 172.21.15.105:0/2324365098 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac02c5d0 0x561fac02e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 -- 172.21.15.105:0/2324365098 >> 172.21.15.105:0/2324365098 conn(0x561fac027b40 msgr2=0x561fac02da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 -- 172.21.15.105:0/2324365098 shutdown_connections 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.511+0000 7f609500f740 1 -- 172.21.15.105:0/2324365098 wait complete. 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 1 -- start start 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 1 Processor -- start 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac02c5d0 0x561fac03afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac032130 0x561fac03c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1a7e10 con 0x561fac032130 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1a7740 con 0x561fac02cfc0 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1a4980 con 0x561fac02c5d0 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f609500f740 10 client.0 _ll_register_callbacks cb 0x7f60800b96f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f60909c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac032130 0x561fac03c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f608bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac02c5d0 0x561fac03afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f608b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f608bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac02c5d0 0x561fac03afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:44452/0 (socket says 172.21.15.105:44452) 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f60909c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac032130 0x561fac03c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47246/0 (socket says 172.21.15.105:47246) 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f608b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49462/0 (socket says 172.21.15.105:49462) 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.512+0000 7f608bfff640 1 -- 172.21.15.105:0/942960391 learned_addr learned my addr 172.21.15.105:0/942960391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.513+0000 7f608bfff640 1 -- 172.21.15.105:0/942960391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 msgr2=0x561fac03b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.513+0000 7f608bfff640 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.513+0000 7f608bfff640 1 -- 172.21.15.105:0/942960391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac032130 msgr2=0x561fac03c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.513+0000 7f608bfff640 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac032130 0x561fac03c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.513+0000 7f608bfff640 1 -- 172.21.15.105:0/942960391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fac03cad0 con 0x561fac02c5d0 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.513+0000 7f608b7fe640 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.513+0000 7f608bfff640 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac02c5d0 0x561fac03afe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f607c011e80 tx=0x7f607c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94878964ca7eca18 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.513+0000 7f606b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.514+0000 7f606b7fe640 1 -- 172.21.15.105:0/942960391 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c01d040 con 0x561fac02c5d0 2024-03-26T17:54:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.514+0000 7f606b7fe640 1 -- 172.21.15.105:0/942960391 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f607c0043c0 con 0x561fac02c5d0 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.514+0000 7f609500f740 1 -- 172.21.15.105:0/942960391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608401ccc0 con 0x561fac02c5d0 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.514+0000 7f606b7fe640 1 -- 172.21.15.105:0/942960391 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f607c0112d0 con 0x561fac02c5d0 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.514+0000 7f609500f740 10 client.30958 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.514+0000 7f609500f740 1 -- 172.21.15.105:0/942960391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561fac0a8530 con 0x561fac02c5d0 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.514+0000 7f606b7fe640 1 -- 172.21.15.105:0/942960391 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f608401ccc0 con 0x561fac02c5d0 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f606b7fe640 1 -- 172.21.15.105:0/942960391 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f607c00f5c0 con 0x561fac02c5d0 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f609500f740 10 client.30958 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f609500f740 10 client.30958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f609500f740 20 client.30958 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f609500f740 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561fac0a8c90 0x561fac0ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f609500f740 4 client.30958 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f609500f740 1 -- 172.21.15.105:0/942960391 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fabf98940 con 0x561fac0a8c90 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f608b7fe640 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561fac0a8c90 0x561fac0ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.515+0000 7f606b7fe640 10 client.30958 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.516+0000 7f608b7fe640 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561fac0a8c90 0x561fac0ab0b0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d8abc97c8c1788 server_cookie=dbec0e505d92c6d8 in_seq=0 out_seq=0 2024-03-26T17:54:05.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 22308 7.37 MB/sec execute 48 sec latency 108.979 ms 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.631+0000 7f606b7fe640 1 -- 172.21.15.105:0/942960391 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fabf98940 con 0x561fac0a8c90 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.631+0000 7f606b7fe640 10 client.30958 handle_command_reply: tid=0 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.631+0000 7f609500f740 1 client.30958 shutdown 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 -- 172.21.15.105:0/942960391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac02c5d0 msgr2=0x561fac03afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac02c5d0 0x561fac03afe0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f607c011e80 tx=0x7f607c000bc0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 -- 172.21.15.105:0/942960391 shutdown_connections 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561fac0a8c90 0x561fac0ab0b0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561fac032130 0x561fac03c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561fac02cfc0 0x561fac03b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f606b7fe640 0 client.30958 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 --2- 172.21.15.105:0/942960391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561fac02c5d0 0x561fac03afe0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 -- 172.21.15.105:0/942960391 >> 172.21.15.105:0/942960391 conn(0x561fac027b40 msgr2=0x561fac02ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 -- 172.21.15.105:0/942960391 shutdown_connections 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 1 -- 172.21.15.105:0/942960391 wait complete. 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f609500f740 20 client.30958 trim_cache size 0 max 16384 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:54:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:54:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f60740a2fc0 msgr2=0x7f60740a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f6092427640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f60740a2fc0 0x7f60740a5480 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x561fac02d820 tx=0x7f6084004a40 comp rx=0 tx=0).stop 2024-03-26T17:54:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f608c1019d0 msgr2=0x7f608c197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.632+0000 7f6092427640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f608c1019d0 0x7f608c197fe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6080000c00 tx=0x7f6080005dc0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.633+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 shutdown_connections 2024-03-26T17:54:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.633+0000 7f6092427640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f60740a2fc0 0x7f60740a5480 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.633+0000 7f6092427640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f608c102390 0x7f608c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.633+0000 7f6092427640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f608c1019d0 0x7f608c197fe0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.633+0000 7f6092427640 1 --2- 172.21.15.105:0/2801121350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f608c1007e0 0x7f608c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.633+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 >> 172.21.15.105:0/2801121350 conn(0x7f608c0fbd50 msgr2=0x7f608c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.633+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 shutdown_connections 2024-03-26T17:54:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:05.633+0000 7f6092427640 1 -- 172.21.15.105:0/2801121350 wait complete. 2024-03-26T17:54:06.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:05 smithi105 ceph-mon[32280]: pgmap v821: 97 pgs: 97 active+clean; 261 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-03-26T17:54:06.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:05 smithi135 ceph-mon[35636]: pgmap v821: 97 pgs: 97 active+clean; 261 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-03-26T17:54:06.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:05 smithi144 ceph-mon[36095]: pgmap v821: 97 pgs: 97 active+clean; 261 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-03-26T17:54:06.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 22385 7.27 MB/sec execute 49 sec latency 174.068 ms 2024-03-26T17:54:07.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:06 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:07.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:06 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:54:07.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:06 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:07.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:06 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:54:07.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:06 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:54:07.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:06 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:54:07.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 22498 7.33 MB/sec execute 50 sec latency 123.615 ms 2024-03-26T17:54:08.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:07 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:54:08.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:07 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:54:08.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:07 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:54:08.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:07 smithi135 ceph-mon[35636]: pgmap v822: 97 pgs: 97 active+clean; 267 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-03-26T17:54:08.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:08.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:08.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:54:08.246 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:07 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:54:08.246 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:07 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:54:08.246 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:07 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:54:08.246 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:07 smithi144 ceph-mon[36095]: pgmap v822: 97 pgs: 97 active+clean; 267 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-03-26T17:54:08.246 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:08.246 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:08.246 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:54:08.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:07 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:54:08.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:07 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:54:08.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:07 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:54:08.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:07 smithi105 ceph-mon[32280]: pgmap v822: 97 pgs: 97 active+clean; 267 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.7 MiB/s wr, 269 op/s 2024-03-26T17:54:08.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:54:08.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:54:08.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:54:08.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 22654 7.52 MB/sec execute 51 sec latency 129.080 ms 2024-03-26T17:54:09.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 22788 7.44 MB/sec execute 52 sec latency 142.427 ms 2024-03-26T17:54:10.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:09 smithi135 ceph-mon[35636]: pgmap v823: 97 pgs: 97 active+clean; 268 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-03-26T17:54:10.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:09 smithi144 ceph-mon[36095]: pgmap v823: 97 pgs: 97 active+clean; 268 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-03-26T17:54:10.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:09 smithi105 ceph-mon[32280]: pgmap v823: 97 pgs: 97 active+clean; 268 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-03-26T17:54:10.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 22918 7.42 MB/sec execute 53 sec latency 158.994 ms 2024-03-26T17:54:11.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23036 7.42 MB/sec execute 54 sec latency 148.997 ms 2024-03-26T17:54:12.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:11 smithi135 ceph-mon[35636]: pgmap v824: 97 pgs: 97 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 3.7 MiB/s wr, 196 op/s 2024-03-26T17:54:12.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:11 smithi144 ceph-mon[36095]: pgmap v824: 97 pgs: 97 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 3.7 MiB/s wr, 196 op/s 2024-03-26T17:54:12.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:11 smithi105 ceph-mon[32280]: pgmap v824: 97 pgs: 97 active+clean; 264 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 3.7 MiB/s wr, 196 op/s 2024-03-26T17:54:12.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23159 7.42 MB/sec execute 55 sec latency 150.091 ms 2024-03-26T17:54:13.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:12 smithi135 ceph-mon[35636]: pgmap v825: 97 pgs: 97 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-03-26T17:54:13.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:12 smithi144 ceph-mon[36095]: pgmap v825: 97 pgs: 97 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-03-26T17:54:13.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:12 smithi105 ceph-mon[32280]: pgmap v825: 97 pgs: 97 active+clean; 265 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.9 MiB/s wr, 207 op/s 2024-03-26T17:54:13.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23267 7.37 MB/sec execute 56 sec latency 125.200 ms 2024-03-26T17:54:14.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23378 7.38 MB/sec execute 57 sec latency 164.915 ms 2024-03-26T17:54:15.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23501 7.32 MB/sec execute 58 sec latency 167.966 ms 2024-03-26T17:54:16.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:15 smithi105 ceph-mon[32280]: pgmap v826: 97 pgs: 97 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-03-26T17:54:16.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:15 smithi135 ceph-mon[35636]: pgmap v826: 97 pgs: 97 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-03-26T17:54:16.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:15 smithi144 ceph-mon[36095]: pgmap v826: 97 pgs: 97 active+clean; 268 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 4.1 MiB/s wr, 207 op/s 2024-03-26T17:54:16.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23589 7.21 MB/sec execute 59 sec latency 193.403 ms 2024-03-26T17:54:17.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23710 7.17 MB/sec execute 60 sec latency 147.248 ms 2024-03-26T17:54:18.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:17 smithi105 ceph-mon[32280]: pgmap v827: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-03-26T17:54:18.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:17 smithi135 ceph-mon[35636]: pgmap v827: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-03-26T17:54:18.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:17 smithi144 ceph-mon[36095]: pgmap v827: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.6 MiB/s wr, 210 op/s 2024-03-26T17:54:18.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23838 7.12 MB/sec execute 61 sec latency 161.536 ms 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 -- 172.21.15.105:0/3969054493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39301018f0 msgr2=0x7f3930101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 --2- 172.21.15.105:0/3969054493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39301018f0 0x7f3930101d10 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f3920009900 tx=0x7f392002bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 -- 172.21.15.105:0/3969054493 shutdown_connections 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 --2- 172.21.15.105:0/3969054493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39301034a0 0x7f3930109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 --2- 172.21.15.105:0/3969054493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3930102ae0 0x7f3930102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 --2- 172.21.15.105:0/3969054493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39301018f0 0x7f3930101d10 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 -- 172.21.15.105:0/3969054493 >> 172.21.15.105:0/3969054493 conn(0x7f39300fce80 msgr2=0x7f39300ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 -- 172.21.15.105:0/3969054493 shutdown_connections 2024-03-26T17:54:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.749+0000 7f3934dc6640 1 -- 172.21.15.105:0/3969054493 wait complete. 2024-03-26T17:54:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.750+0000 7f3934dc6640 1 Processor -- start 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.750+0000 7f3934dc6640 1 -- start start 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.750+0000 7f3934dc6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39301018f0 0x7f3930075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.750+0000 7f3934dc6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3930102ae0 0x7f3930073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.750+0000 7f3934dc6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39301034a0 0x7f39300741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f3934dc6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f393010d3f0 con 0x7f3930102ae0 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f3934dc6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f393010d270 con 0x7f39301018f0 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f3934dc6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f393010d570 con 0x7f39301034a0 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39301018f0 0x7f3930075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3930102ae0 0x7f3930073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39301018f0 0x7f3930075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39686/0 (socket says 172.21.15.105:39686) 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39301034a0 0x7f39300741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392e59c640 1 -- 172.21.15.105:0/821325263 learned_addr learned my addr 172.21.15.105:0/821325263 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392dd9b640 1 -- 172.21.15.105:0/821325263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39301034a0 msgr2=0x7f39300741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392dd9b640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39301034a0 0x7f39300741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392dd9b640 1 -- 172.21.15.105:0/821325263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39301018f0 msgr2=0x7f3930075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392dd9b640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39301018f0 0x7f3930075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392dd9b640 1 -- 172.21.15.105:0/821325263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3930074730 con 0x7f3930102ae0 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.751+0000 7f392dd9b640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3930102ae0 0x7f3930073cb0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f392400e9f0 tx=0x7f392400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.752+0000 7f391f7fe640 1 -- 172.21.15.105:0/821325263 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392400ccc0 con 0x7f3930102ae0 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.752+0000 7f3934dc6640 1 -- 172.21.15.105:0/821325263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3930074a20 con 0x7f3930102ae0 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.752+0000 7f391f7fe640 1 -- 172.21.15.105:0/821325263 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39240108e0 con 0x7f3930102ae0 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.752+0000 7f3934dc6640 1 -- 172.21.15.105:0/821325263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39301a6520 con 0x7f3930102ae0 2024-03-26T17:54:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.752+0000 7f391f7fe640 1 -- 172.21.15.105:0/821325263 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924018880 con 0x7f3930102ae0 2024-03-26T17:54:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f391f7fe640 1 -- 172.21.15.105:0/821325263 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3924018a20 con 0x7f3930102ae0 2024-03-26T17:54:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f39379b1740 1 Processor -- start 2024-03-26T17:54:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f39379b1740 1 -- start start 2024-03-26T17:54:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f39379b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd75d0 0x55db81fd79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f39379b1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fdcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f39379b1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fdf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f39379b1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55db81f3c0b0 con 0x55db81fd7fc0 2024-03-26T17:54:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f39379b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db81ee4960 con 0x55db81fd75d0 2024-03-26T17:54:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.755+0000 7f39379b1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55db81f38060 con 0x55db81fdd130 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd75d0 0x55db81fd79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd75d0 0x55db81fd79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39690/0 (socket says 172.21.15.105:39690) 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fdcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392ed9d640 1 -- 172.21.15.105:0/3167499375 learned_addr learned my addr 172.21.15.105:0/3167499375 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392dd9b640 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fdf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f391f7fe640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39180a3040 0x7f39180a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392dd9b640 1 -- 172.21.15.105:0/3167499375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd75d0 msgr2=0x55db81fd79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392dd9b640 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd75d0 0x55db81fd79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392dd9b640 1 -- 172.21.15.105:0/3167499375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 msgr2=0x55db81fdcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392dd9b640 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fdcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392dd9b640 1 -- 172.21.15.105:0/3167499375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db81ef6c60 con 0x55db81fdd130 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392e59c640 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fdcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392ed9d640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39180a3040 0x7f39180a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.747 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f392dd9b640 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fdf5c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f39240cd050 tx=0x7f39240aff40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71f1d69930ba1059 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.756+0000 7f391f7fe640 1 -- 172.21.15.105:0/821325263 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f39240cf5c0 con 0x7f3930102ae0 2024-03-26T17:54:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.757+0000 7f391cff9640 1 -- 172.21.15.105:0/3167499375 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392405ce60 con 0x55db81fdd130 2024-03-26T17:54:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.757+0000 7f391cff9640 1 -- 172.21.15.105:0/3167499375 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392409bd60 con 0x55db81fdd130 2024-03-26T17:54:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.757+0000 7f391cff9640 1 -- 172.21.15.105:0/3167499375 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924026030 con 0x55db81fdd130 2024-03-26T17:54:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.757+0000 7f39379b1740 1 -- 172.21.15.105:0/3167499375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 msgr2=0x55db81fdf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.757+0000 7f39379b1740 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fdf5c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f39240cd050 tx=0x7f39240aff40 comp rx=0 tx=0).stop 2024-03-26T17:54:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.757+0000 7f392ed9d640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39180a3040 0x7f39180a5500 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3928009fd0 tx=0x7f3928004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 -- 172.21.15.105:0/3167499375 shutdown_connections 2024-03-26T17:54:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fdf5c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fdcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 --2- 172.21.15.105:0/3167499375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd75d0 0x55db81fd79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 -- 172.21.15.105:0/3167499375 >> 172.21.15.105:0/3167499375 conn(0x55db81fd2b40 msgr2=0x55db81fd2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 -- 172.21.15.105:0/3167499375 shutdown_connections 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 -- 172.21.15.105:0/3167499375 wait complete. 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 -- start start 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 Processor -- start 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd75d0 0x55db81fe1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.758+0000 7f39379b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fe18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f39379b1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fe8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f39379b1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55db81fe00e0 con 0x55db81fd75d0 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f39379b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db81ef7db0 con 0x55db81fd7fc0 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f39379b1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55db81fe0260 con 0x55db81fdd130 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f39379b1740 10 client.0 _ll_register_callbacks cb 0x55db81efc110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:18.754 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd75d0 0x55db81fe1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd75d0 0x55db81fe1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34750/0 (socket says 172.21.15.105:34750) 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 -- 172.21.15.105:0/1572339237 learned_addr learned my addr 172.21.15.105:0/1572339237 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392dd9b640 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fe18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392ed9d640 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fe8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 -- 172.21.15.105:0/1572339237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 msgr2=0x55db81fe8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fe8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 -- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 msgr2=0x55db81fe18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fe18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 -- 172.21.15.105:0/1572339237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db81c50210 con 0x55db81fd75d0 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392dd9b640 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fe18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f392e59c640 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd75d0 0x55db81fe1390 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f3920000c00 tx=0x7f3920004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be3f21434c5dc48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f3915ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:18.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f3915ffb640 1 -- 172.21.15.105:0/1572339237 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920005030 con 0x55db81fd75d0 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f3915ffb640 1 -- 172.21.15.105:0/1572339237 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f39200051d0 con 0x55db81fd75d0 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.759+0000 7f39379b1740 1 -- 172.21.15.105:0/1572339237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392409bd60 con 0x55db81fd75d0 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f3915ffb640 1 -- 172.21.15.105:0/1572339237 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f39200053d0 con 0x55db81fd75d0 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f39379b1740 10 client.21228 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f39379b1740 1 -- 172.21.15.105:0/1572339237 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55db81d1c620 con 0x55db81fd75d0 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f3915ffb640 1 -- 172.21.15.105:0/1572339237 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392409bd60 con 0x55db81fd75d0 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f3915ffb640 1 -- 172.21.15.105:0/1572339237 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3920003820 con 0x55db81fd75d0 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f39379b1740 10 client.21228 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f39379b1740 10 client.21228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f39379b1740 20 client.21228 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f39379b1740 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55db81fed260 0x55db81fef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f39379b1740 4 client.21228 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.760+0000 7f39379b1740 1 -- 172.21.15.105:0/1572339237 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db81f495f0 con 0x55db81fed260 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.761+0000 7f392dd9b640 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55db81fed260 0x55db81fef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.761+0000 7f3915ffb640 10 client.21228 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.761+0000 7f392dd9b640 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55db81fed260 0x55db81fef6c0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f24584a99d470c server_cookie=15a6b93112f5f306 in_seq=0 out_seq=0 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.762+0000 7f3915ffb640 1 -- 172.21.15.105:0/1572339237 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55db81f495f0 con 0x55db81fed260 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.762+0000 7f3915ffb640 10 client.21228 handle_command_reply: tid=0 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 client.21228 shutdown 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 -- 172.21.15.105:0/1572339237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd75d0 msgr2=0x55db81fe1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd75d0 0x55db81fe1390 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f3920000c00 tx=0x7f3920004270 comp rx=0 tx=0).stop 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 -- 172.21.15.105:0/1572339237 shutdown_connections 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55db81fed260 0x55db81fef6c0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fe8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fe18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 --2- 172.21.15.105:0/1572339237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd75d0 0x55db81fe1390 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 -- 172.21.15.105:0/1572339237 >> 172.21.15.105:0/1572339237 conn(0x55db81fd2b40 msgr2=0x55db81fd1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f3915ffb640 0 client.21228 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:18.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 -- 172.21.15.105:0/1572339237 shutdown_connections 2024-03-26T17:54:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 1 -- 172.21.15.105:0/1572339237 wait complete. 2024-03-26T17:54:18.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.763+0000 7f39379b1740 20 client.21228 trim_cache size 0 max 16384 2024-03-26T17:54:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f39379b1740 1 Processor -- start 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f39379b1740 1 -- start start 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f39379b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fd9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f39379b1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fe8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f39379b1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fdc360 0x55db81fd64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f39379b1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f391000e5f0 con 0x55db81fdc360 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f39379b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3910010130 con 0x55db81fd7fc0 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f39379b1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f391000ecc0 con 0x55db81fdd130 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f392ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fdc360 0x55db81fd64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f392ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fdc360 0x55db81fd64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34754/0 (socket says 172.21.15.105:34754) 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f392ed9d640 1 -- 172.21.15.105:0/2613969689 learned_addr learned my addr 172.21.15.105:0/2613969689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f392e59c640 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fd9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.784+0000 7f392dd9b640 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fe8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f392ed9d640 1 -- 172.21.15.105:0/2613969689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 msgr2=0x55db81fe8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f392ed9d640 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fe8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f392ed9d640 1 -- 172.21.15.105:0/2613969689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 msgr2=0x55db81fd9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f392ed9d640 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fd9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f392ed9d640 1 -- 172.21.15.105:0/2613969689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db81fdca40 con 0x55db81fdc360 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f392e59c640 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fd9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f392ed9d640 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fdc360 0x55db81fd64b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3928020710 tx=0x7f3928013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3ba6ef1bae4013 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f3917fff640 1 -- 172.21.15.105:0/2613969689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928004d20 con 0x55db81fdc360 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f3917fff640 1 -- 172.21.15.105:0/2613969689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392801ebe0 con 0x55db81fdc360 2024-03-26T17:54:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.785+0000 7f3917fff640 1 -- 172.21.15.105:0/2613969689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39280057e0 con 0x55db81fdc360 2024-03-26T17:54:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 -- 172.21.15.105:0/2613969689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fdc360 msgr2=0x55db81fd64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fdc360 0x55db81fd64b0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3928020710 tx=0x7f3928013030 comp rx=0 tx=0).stop 2024-03-26T17:54:18.779 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 -- 172.21.15.105:0/2613969689 shutdown_connections 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fdc360 0x55db81fd64b0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdd130 0x55db81fe8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 --2- 172.21.15.105:0/2613969689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fd7fc0 0x55db81fd9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 -- 172.21.15.105:0/2613969689 >> 172.21.15.105:0/2613969689 conn(0x55db81fd2b40 msgr2=0x55db81fd5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 -- 172.21.15.105:0/2613969689 shutdown_connections 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 -- 172.21.15.105:0/2613969689 wait complete. 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.786+0000 7f39379b1740 1 -- start start 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f39379b1740 1 Processor -- start 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f39379b1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fe5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f39379b1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdc360 0x55db81fe6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f39379b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fdd130 0x55db81fe7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f39379b1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f391000fa60 con 0x55db81fd7fc0 2024-03-26T17:54:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f39379b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f391000f390 con 0x55db81fdd130 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f39379b1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f391000c5d0 con 0x55db81fdc360 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fe5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fe5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34760/0 (socket says 172.21.15.105:34760) 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392e59c640 1 -- 172.21.15.105:0/3813197707 learned_addr learned my addr 172.21.15.105:0/3813197707 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392dd9b640 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdc360 0x55db81fe6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f39379b1740 10 client.0 _ll_register_callbacks cb 0x7f3920009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392ed9d640 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fdd130 0x55db81fe7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392e59c640 1 -- 172.21.15.105:0/3813197707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdc360 msgr2=0x55db81fe6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392e59c640 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdc360 0x55db81fe6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392e59c640 1 -- 172.21.15.105:0/3813197707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fdd130 msgr2=0x55db81fe7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392e59c640 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fdd130 0x55db81fe7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392e59c640 1 -- 172.21.15.105:0/3813197707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db81fe7a40 con 0x55db81fd7fc0 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.787+0000 7f392dd9b640 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdc360 0x55db81fe6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f392e59c640 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fe5f00 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3920003de0 tx=0x7f3920003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ac08a2442817da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f3915ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f3915ffb640 1 -- 172.21.15.105:0/3813197707 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3920059020 con 0x55db81fd7fc0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f3915ffb640 1 -- 172.21.15.105:0/3813197707 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f39200407f0 con 0x55db81fd7fc0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f39379b1740 1 -- 172.21.15.105:0/3813197707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392801ebe0 con 0x55db81fd7fc0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f3915ffb640 1 -- 172.21.15.105:0/3813197707 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3920000e60 con 0x55db81fd7fc0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f39379b1740 10 client.21234 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f39379b1740 1 -- 172.21.15.105:0/3813197707 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55db82053670 con 0x55db81fd7fc0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f3915ffb640 1 -- 172.21.15.105:0/3813197707 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f392801ebe0 con 0x55db81fd7fc0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.788+0000 7f3915ffb640 1 -- 172.21.15.105:0/3813197707 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3920055560 con 0x55db81fd7fc0 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f39379b1740 10 client.21234 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f39379b1740 10 client.21234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f39379b1740 20 client.21234 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f39379b1740 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55db820540c0 0x55db820564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f39379b1740 4 client.21234 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:54:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f39379b1740 1 -- 172.21.15.105:0/3813197707 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db81f43940 con 0x55db820540c0 2024-03-26T17:54:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f392dd9b640 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55db820540c0 0x55db820564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f392dd9b640 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55db820540c0 0x55db820564e0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25105720934f7a36 server_cookie=617ed97117f159ea in_seq=0 out_seq=0 2024-03-26T17:54:18.783 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.789+0000 7f3915ffb640 10 client.21234 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f3915ffb640 1 -- 172.21.15.105:0/3813197707 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db81f43940 con 0x55db820540c0 2024-03-26T17:54:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f3915ffb640 10 client.21234 handle_command_reply: tid=0 2024-03-26T17:54:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 client.21234 shutdown 2024-03-26T17:54:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 -- 172.21.15.105:0/3813197707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 msgr2=0x55db81fe5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fe5f00 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3920003de0 tx=0x7f3920003820 comp rx=0 tx=0).stop 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 -- 172.21.15.105:0/3813197707 shutdown_connections 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55db820540c0 0x55db820564e0 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db81fdd130 0x55db81fe7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db81fdc360 0x55db81fe6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f3915ffb640 0 client.21234 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 --2- 172.21.15.105:0/3813197707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db81fd7fc0 0x55db81fe5f00 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 -- 172.21.15.105:0/3813197707 >> 172.21.15.105:0/3813197707 conn(0x55db81fd2b40 msgr2=0x55db81fd6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 -- 172.21.15.105:0/3813197707 shutdown_connections 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.909+0000 7f39379b1740 1 -- 172.21.15.105:0/3813197707 wait complete. 2024-03-26T17:54:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f39379b1740 20 client.21234 trim_cache size 0 max 16384 2024-03-26T17:54:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:54:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:54:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:54:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 -- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39180a3040 msgr2=0x7f39180a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39180a3040 0x7f39180a5500 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3928009fd0 tx=0x7f3928004a40 comp rx=0 tx=0).stop 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 -- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3930102ae0 msgr2=0x7f3930073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3930102ae0 0x7f3930073cb0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f392400e9f0 tx=0x7f392400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 -- 172.21.15.105:0/821325263 shutdown_connections 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39180a3040 0x7f39180a5500 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39301034a0 0x7f39300741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3930102ae0 0x7f3930073cb0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.910+0000 7f3934dc6640 1 --2- 172.21.15.105:0/821325263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39301018f0 0x7f3930075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.911+0000 7f3934dc6640 1 -- 172.21.15.105:0/821325263 >> 172.21.15.105:0/821325263 conn(0x7f39300fce80 msgr2=0x7f3930105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.911+0000 7f3934dc6640 1 -- 172.21.15.105:0/821325263 shutdown_connections 2024-03-26T17:54:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:18.911+0000 7f3934dc6640 1 -- 172.21.15.105:0/821325263 wait complete. 2024-03-26T17:54:19.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 23954 7.11 MB/sec execute 62 sec latency 209.985 ms 2024-03-26T17:54:20.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:19 smithi105 ceph-mon[32280]: pgmap v828: 97 pgs: 97 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.5 MiB/s wr, 189 op/s 2024-03-26T17:54:20.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:19 smithi135 ceph-mon[35636]: pgmap v828: 97 pgs: 97 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.5 MiB/s wr, 189 op/s 2024-03-26T17:54:20.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:19 smithi144 ceph-mon[36095]: pgmap v828: 97 pgs: 97 active+clean; 269 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.5 MiB/s wr, 189 op/s 2024-03-26T17:54:20.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 24079 7.06 MB/sec execute 63 sec latency 125.682 ms 2024-03-26T17:54:21.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 24200 7.01 MB/sec execute 64 sec latency 135.383 ms 2024-03-26T17:54:22.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:21 smithi105 ceph-mon[32280]: pgmap v829: 97 pgs: 97 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-03-26T17:54:22.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:21 smithi135 ceph-mon[35636]: pgmap v829: 97 pgs: 97 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-03-26T17:54:22.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:21 smithi144 ceph-mon[36095]: pgmap v829: 97 pgs: 97 active+clean; 270 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.3 MiB/s wr, 189 op/s 2024-03-26T17:54:22.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 24334 7.05 MB/sec execute 65 sec latency 138.129 ms 2024-03-26T17:54:23.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 24492 7.11 MB/sec execute 66 sec latency 138.443 ms 2024-03-26T17:54:23.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:54:23.849 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:54:23.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 -- 172.21.15.105:0/2860033274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d4102990 msgr2=0x7f83d4102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:23.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/2860033274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d4102990 0x7f83d4102e10 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f83c400fb10 tx=0x7f83c402fdc0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 -- 172.21.15.105:0/2860033274 shutdown_connections 2024-03-26T17:54:23.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/2860033274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83d4103350 0x7f83d4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/2860033274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d4102990 0x7f83d4102e10 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/2860033274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d41017a0 0x7f83d4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 -- 172.21.15.105:0/2860033274 >> 172.21.15.105:0/2860033274 conn(0x7f83d40fcd30 msgr2=0x7f83d40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:23.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 -- 172.21.15.105:0/2860033274 shutdown_connections 2024-03-26T17:54:23.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.003+0000 7f83d8d9d640 1 -- 172.21.15.105:0/2860033274 wait complete. 2024-03-26T17:54:23.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d8d9d640 1 Processor -- start 2024-03-26T17:54:23.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d8d9d640 1 -- start start 2024-03-26T17:54:23.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d8d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d41017a0 0x7f83d4195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d8d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83d4102990 0x7f83d4195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d8d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d4103350 0x7f83d419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d8d9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f83d410b0a0 con 0x7f83d41017a0 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d8d9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83d410af20 con 0x7f83d4102990 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d8d9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83d410b220 con 0x7f83d4103350 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.004+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d41017a0 0x7f83d4195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d41017a0 0x7f83d4195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34788/0 (socket says 172.21.15.105:34788) 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d37fe640 1 -- 172.21.15.105:0/1955952243 learned_addr learned my addr 172.21.15.105:0/1955952243 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83d4102990 0x7f83d4195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:23.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d3fff640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d4103350 0x7f83d419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d37fe640 1 -- 172.21.15.105:0/1955952243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d4103350 msgr2=0x7f83d419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d37fe640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d4103350 0x7f83d419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d37fe640 1 -- 172.21.15.105:0/1955952243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83d4102990 msgr2=0x7f83d4195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d37fe640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83d4102990 0x7f83d4195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d37fe640 1 -- 172.21.15.105:0/1955952243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d419d360 con 0x7f83d41017a0 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d37fe640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d41017a0 0x7f83d4195740 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f83c800e9c0 tx=0x7f83c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d0ff9640 1 -- 172.21.15.105:0/1955952243 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c800cd80 con 0x7f83d41017a0 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d8d9d640 1 -- 172.21.15.105:0/1955952243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d419d650 con 0x7f83d41017a0 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d0ff9640 1 -- 172.21.15.105:0/1955952243 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83c80035f0 con 0x7f83d41017a0 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d0ff9640 1 -- 172.21.15.105:0/1955952243 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c8010690 con 0x7f83d41017a0 2024-03-26T17:54:23.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.005+0000 7f83d8d9d640 1 -- 172.21.15.105:0/1955952243 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d419db70 con 0x7f83d41017a0 2024-03-26T17:54:23.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.008+0000 7f83d0ff9640 1 -- 172.21.15.105:0/1955952243 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f83c80037a0 con 0x7f83d41017a0 2024-03-26T17:54:23.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.008+0000 7f83dcbe2740 1 Processor -- start 2024-03-26T17:54:23.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.008+0000 7f83dcbe2740 1 -- start start 2024-03-26T17:54:23.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 0x5646a063aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a063ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:23.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a0642a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5646a059f980 con 0x5646a0640600 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646a0547be0 con 0x5646a063aaa0 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5646a059b930 con 0x5646a063b490 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a0642a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 0x5646a063aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a063ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d0ff9640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83bc0a2f20 0x7f83bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a0642a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34792/0 (socket says 172.21.15.105:34792) 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d2ffd640 1 -- 172.21.15.105:0/3493588490 learned_addr learned my addr 172.21.15.105:0/3493588490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d3fff640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83bc0a2f20 0x7f83bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d0ff9640 1 -- 172.21.15.105:0/1955952243 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f83c80cf800 con 0x7f83d41017a0 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d37fe640 1 -- 172.21.15.105:0/3493588490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 msgr2=0x5646a063aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d37fe640 1 --2- 172.21.15.105:0/3493588490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 0x5646a063aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d37fe640 1 -- 172.21.15.105:0/3493588490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 msgr2=0x5646a0642a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d37fe640 1 --2- 172.21.15.105:0/3493588490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a0642a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d37fe640 1 -- 172.21.15.105:0/3493588490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a037d7d0 con 0x5646a063b490 2024-03-26T17:54:24.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.009+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/3493588490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a0642a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.010+0000 7f83d37fe640 1 --2- 172.21.15.105:0/3493588490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a063ff90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f83c8098860 tx=0x7f83c8003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0199fa00419c613 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.010+0000 7f83d3fff640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83bc0a2f20 0x7f83bc0a53e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5646a063bcf0 tx=0x7f83cc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.010+0000 7f83b9ffb640 1 -- 172.21.15.105:0/3493588490 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c809bc60 con 0x5646a063b490 2024-03-26T17:54:24.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.010+0000 7f83b9ffb640 1 -- 172.21.15.105:0/3493588490 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83c8003b30 con 0x5646a063b490 2024-03-26T17:54:24.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.010+0000 7f83b9ffb640 1 -- 172.21.15.105:0/3493588490 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c809b7f0 con 0x5646a063b490 2024-03-26T17:54:24.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3493588490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 msgr2=0x5646a063ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3493588490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a063ff90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f83c8098860 tx=0x7f83c8003e30 comp rx=0 tx=0).stop 2024-03-26T17:54:24.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3493588490 shutdown_connections 2024-03-26T17:54:24.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3493588490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a0642a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3493588490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a063ff90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3493588490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 0x5646a063aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3493588490 >> 172.21.15.105:0/3493588490 conn(0x5646a0636010 msgr2=0x5646a0636410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3493588490 shutdown_connections 2024-03-26T17:54:24.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3493588490 wait complete. 2024-03-26T17:54:24.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 -- start start 2024-03-26T17:54:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.011+0000 7f83dcbe2740 1 Processor -- start 2024-03-26T17:54:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063aaa0 0x5646a0644cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063b490 0x5646a06451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a064c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5646a0643550 con 0x5646a0640600 2024-03-26T17:54:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646a055b160 con 0x5646a063b490 2024-03-26T17:54:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5646a06436d0 con 0x5646a063aaa0 2024-03-26T17:54:24.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83dcbe2740 10 client.0 _ll_register_callbacks cb 0x5646a0597720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063aaa0 0x5646a0644cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a064c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063aaa0 0x5646a0644cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41560/0 (socket says 172.21.15.105:41560) 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d37fe640 1 -- 172.21.15.105:0/3742808693 learned_addr learned my addr 172.21.15.105:0/3742808693 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063b490 0x5646a06451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d3fff640 1 -- 172.21.15.105:0/3742808693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063aaa0 msgr2=0x5646a0644cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d3fff640 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063aaa0 0x5646a0644cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d3fff640 1 -- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063b490 msgr2=0x5646a06451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d3fff640 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063b490 0x5646a06451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d3fff640 1 -- 172.21.15.105:0/3742808693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a02b33f0 con 0x5646a0640600 2024-03-26T17:54:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063b490 0x5646a06451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83d3fff640 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a064c1f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f83cc007f70 tx=0x7f83cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1604daff59786498 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.012+0000 7f83b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.013+0000 7f83b2ffd640 1 -- 172.21.15.105:0/3742808693 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc019070 con 0x5646a0640600 2024-03-26T17:54:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.013+0000 7f83b2ffd640 1 -- 172.21.15.105:0/3742808693 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f83cc020810 con 0x5646a0640600 2024-03-26T17:54:24.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.013+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3742808693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646a05acec0 con 0x5646a0640600 2024-03-26T17:54:24.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.013+0000 7f83b2ffd640 1 -- 172.21.15.105:0/3742808693 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f83cc020a10 con 0x5646a0640600 2024-03-26T17:54:24.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.013+0000 7f83dcbe2740 10 client.21246 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:24.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.013+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3742808693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5646a02730f0 con 0x5646a0640600 2024-03-26T17:54:24.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.013+0000 7f83b2ffd640 1 -- 172.21.15.105:0/3742808693 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5646a05acec0 con 0x5646a0640600 2024-03-26T17:54:24.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.014+0000 7f83b2ffd640 1 -- 172.21.15.105:0/3742808693 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f83cc00ec50 con 0x5646a0640600 2024-03-26T17:54:24.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.014+0000 7f83dcbe2740 10 client.21246 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:24.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.014+0000 7f83dcbe2740 10 client.21246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:24.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.014+0000 7f83dcbe2740 20 client.21246 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:24.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.014+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646a0650b10 0x5646a0652f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.014+0000 7f83dcbe2740 4 client.21246 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:24.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.014+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3742808693 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a05a7210 con 0x5646a0650b10 2024-03-26T17:54:24.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.014+0000 7f83d37fe640 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646a0650b10 0x5646a0652f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.015+0000 7f83b2ffd640 10 client.21246 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:24.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.015+0000 7f83d37fe640 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646a0650b10 0x5646a0652f70 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d497b9051796b6f server_cookie=69db76fde7561440 in_seq=0 out_seq=0 2024-03-26T17:54:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.019+0000 7f83b2ffd640 1 -- 172.21.15.105:0/3742808693 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5646a05a7210 con 0x5646a0650b10 2024-03-26T17:54:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.019+0000 7f83b2ffd640 10 client.21246 handle_command_reply: tid=0 2024-03-26T17:54:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.019+0000 7f83dcbe2740 1 client.21246 shutdown 2024-03-26T17:54:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.019+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3742808693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 msgr2=0x5646a064c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.019+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a064c1f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f83cc007f70 tx=0x7f83cc004a40 comp rx=0 tx=0).stop 2024-03-26T17:54:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3742808693 shutdown_connections 2024-03-26T17:54:24.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646a0650b10 0x5646a0652f70 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a0640600 0x5646a064c1f0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83b2ffd640 0 client.21246 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063b490 0x5646a06451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/3742808693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063aaa0 0x5646a0644cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3742808693 >> 172.21.15.105:0/3742808693 conn(0x5646a0636010 msgr2=0x5646a0635210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3742808693 shutdown_connections 2024-03-26T17:54:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 1 -- 172.21.15.105:0/3742808693 wait complete. 2024-03-26T17:54:24.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.020+0000 7f83dcbe2740 20 client.21246 trim_cache size 0 max 16384 2024-03-26T17:54:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83dcbe2740 1 Processor -- start 2024-03-26T17:54:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83dcbe2740 1 -- start start 2024-03-26T17:54:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 0x5646a0644cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063b490 0x5646a064c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a0640600 0x5646a0657830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41a6900 con 0x5646a063b490 2024-03-26T17:54:24.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41a8440 con 0x5646a063aaa0 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41a6fd0 con 0x5646a0640600 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063b490 0x5646a064c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063b490 0x5646a064c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34806/0 (socket says 172.21.15.105:34806) 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83d2ffd640 1 -- 172.21.15.105:0/2516297571 learned_addr learned my addr 172.21.15.105:0/2516297571 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83d3fff640 1 --2- 172.21.15.105:0/2516297571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a0640600 0x5646a0657830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.047+0000 7f83d2ffd640 1 -- 172.21.15.105:0/2516297571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a0640600 msgr2=0x5646a0657830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.048+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/2516297571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a0640600 0x5646a0657830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.048+0000 7f83d2ffd640 1 -- 172.21.15.105:0/2516297571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 msgr2=0x5646a0644cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.048+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/2516297571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 0x5646a0644cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.048+0000 7f83d2ffd640 1 -- 172.21.15.105:0/2516297571 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a063b870 con 0x5646a063b490 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.048+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/2516297571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063b490 0x5646a064c1b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f83c4038000 tx=0x7f83c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f875f4a62ed41d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.048+0000 7f83b97fa640 1 -- 172.21.15.105:0/2516297571 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40046c0 con 0x5646a063b490 2024-03-26T17:54:24.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.048+0000 7f83b97fa640 1 -- 172.21.15.105:0/2516297571 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83c4047070 con 0x5646a063b490 2024-03-26T17:54:24.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.048+0000 7f83b97fa640 1 -- 172.21.15.105:0/2516297571 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40309c0 con 0x5646a063b490 2024-03-26T17:54:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 -- 172.21.15.105:0/2516297571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063b490 msgr2=0x5646a064c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/2516297571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063b490 0x5646a064c1b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f83c4038000 tx=0x7f83c4005fd0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 -- 172.21.15.105:0/2516297571 shutdown_connections 2024-03-26T17:54:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/2516297571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a0640600 0x5646a0657830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/2516297571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063b490 0x5646a064c1b0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/2516297571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a063aaa0 0x5646a0644cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 -- 172.21.15.105:0/2516297571 >> 172.21.15.105:0/2516297571 conn(0x5646a0636010 msgr2=0x5646a0635210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 -- 172.21.15.105:0/2516297571 shutdown_connections 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 -- 172.21.15.105:0/2516297571 wait complete. 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.049+0000 7f83dcbe2740 1 -- start start 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83dcbe2740 1 Processor -- start 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063aaa0 0x5646a064aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a064b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83dcbe2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a0640600 0x5646a064b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41a7d70 con 0x5646a063aaa0 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41a76a0 con 0x5646a0640600 2024-03-26T17:54:24.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83dcbe2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83d41a48e0 con 0x5646a063b490 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83dcbe2740 10 client.0 _ll_register_callbacks cb 0x7f83cc0134c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063aaa0 0x5646a064aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063aaa0 0x5646a064aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34812/0 (socket says 172.21.15.105:34812) 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d37fe640 1 -- 172.21.15.105:0/1255842528 learned_addr learned my addr 172.21.15.105:0/1255842528 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d3fff640 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a0640600 0x5646a064b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a064b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d37fe640 1 -- 172.21.15.105:0/1255842528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 msgr2=0x5646a064b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d37fe640 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a064b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d37fe640 1 -- 172.21.15.105:0/1255842528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a0640600 msgr2=0x5646a064b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d37fe640 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a0640600 0x5646a064b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.050+0000 7f83d37fe640 1 -- 172.21.15.105:0/1255842528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a06b6ff0 con 0x5646a063aaa0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a064b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83d37fe640 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063aaa0 0x5646a064aaf0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f83c8098860 tx=0x7f83c8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5176cbe3d6adf611 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83b2ffd640 1 -- 172.21.15.105:0/1255842528 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c809b9d0 con 0x5646a063aaa0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83b2ffd640 1 -- 172.21.15.105:0/1255842528 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f83c80a4410 con 0x5646a063aaa0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83dcbe2740 1 -- 172.21.15.105:0/1255842528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83c4047070 con 0x5646a063aaa0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83b2ffd640 1 -- 172.21.15.105:0/1255842528 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f83c809bc60 con 0x5646a063aaa0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83dcbe2740 10 client.21252 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.051+0000 7f83dcbe2740 1 -- 172.21.15.105:0/1255842528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5646a06b8300 con 0x5646a063aaa0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83b2ffd640 1 -- 172.21.15.105:0/1255842528 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83c809c590 con 0x5646a063aaa0 2024-03-26T17:54:24.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83b2ffd640 1 -- 172.21.15.105:0/1255842528 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f83c8097da0 con 0x5646a063aaa0 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83dcbe2740 10 client.21252 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83dcbe2740 10 client.21252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83dcbe2740 20 client.21252 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646a06b88e0 0x5646a06bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83dcbe2740 4 client.21252 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83dcbe2740 1 -- 172.21.15.105:0/1255842528 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f83c4047070 con 0x5646a06b88e0 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.052+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646a06b88e0 0x5646a06bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.053+0000 7f83b2ffd640 10 client.21252 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:24.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.053+0000 7f83d2ffd640 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646a06b88e0 0x5646a06bacd0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99f95b0d1beaa5a server_cookie=c054355dafa99b4c in_seq=0 out_seq=0 2024-03-26T17:54:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.054+0000 7f83b2ffd640 1 -- 172.21.15.105:0/1255842528 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f83c4047070 con 0x5646a06b88e0 2024-03-26T17:54:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.054+0000 7f83b2ffd640 10 client.21252 handle_command_reply: tid=0 2024-03-26T17:54:24.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.054+0000 7f83dcbe2740 1 client.21252 shutdown 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 -- 172.21.15.105:0/1255842528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063aaa0 msgr2=0x5646a064aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063aaa0 0x5646a064aaf0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f83c8098860 tx=0x7f83c8010830 comp rx=0 tx=0).stop 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 -- 172.21.15.105:0/1255842528 shutdown_connections 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5646a06b88e0 0x5646a06bacd0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646a0640600 0x5646a064b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5646a063b490 0x5646a064b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 --2- 172.21.15.105:0/1255842528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5646a063aaa0 0x5646a064aaf0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 -- 172.21.15.105:0/1255842528 >> 172.21.15.105:0/1255842528 conn(0x5646a0636010 msgr2=0x5646a0634980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83b2ffd640 0 client.21252 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 -- 172.21.15.105:0/1255842528 shutdown_connections 2024-03-26T17:54:24.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 1 -- 172.21.15.105:0/1255842528 wait complete. 2024-03-26T17:54:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83dcbe2740 20 client.21252 trim_cache size 0 max 16384 2024-03-26T17:54:24.049 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:54:24.049 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:54:24.049 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:54:24.049 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:54:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83d8d9d640 1 -- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83bc0a2f20 msgr2=0x7f83bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83bc0a2f20 0x7f83bc0a53e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5646a063bcf0 tx=0x7f83cc004ab0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83d8d9d640 1 -- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d41017a0 msgr2=0x7f83d4195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.055+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d41017a0 0x7f83d4195740 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f83c800e9c0 tx=0x7f83c800ee80 comp rx=0 tx=0).stop 2024-03-26T17:54:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.056+0000 7f83d8d9d640 1 -- 172.21.15.105:0/1955952243 shutdown_connections 2024-03-26T17:54:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.056+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83bc0a2f20 0x7f83bc0a53e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.056+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d4103350 0x7f83d419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.056+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83d4102990 0x7f83d4195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.056+0000 7f83d8d9d640 1 --2- 172.21.15.105:0/1955952243 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83d41017a0 0x7f83d4195740 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.056+0000 7f83d8d9d640 1 -- 172.21.15.105:0/1955952243 >> 172.21.15.105:0/1955952243 conn(0x7f83d40fcd30 msgr2=0x7f83d4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.056+0000 7f83d8d9d640 1 -- 172.21.15.105:0/1955952243 shutdown_connections 2024-03-26T17:54:24.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.056+0000 7f83d8d9d640 1 -- 172.21.15.105:0/1955952243 wait complete. 2024-03-26T17:54:24.057 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:54:24.058 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:54:24.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:23 smithi105 ceph-mon[32280]: pgmap v830: 97 pgs: 97 active+clean; 257 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.5 MiB/s wr, 217 op/s 2024-03-26T17:54:24.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:23 smithi135 ceph-mon[35636]: pgmap v830: 97 pgs: 97 active+clean; 257 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.5 MiB/s wr, 217 op/s 2024-03-26T17:54:24.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.201+0000 7f30705a9640 1 -- 172.21.15.105:0/610948561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3068102ae0 msgr2=0x7f3068102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.201+0000 7f30705a9640 1 --2- 172.21.15.105:0/610948561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3068102ae0 0x7f3068102f60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3058009900 tx=0x7f305802bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:24.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.202+0000 7f30705a9640 1 -- 172.21.15.105:0/610948561 shutdown_connections 2024-03-26T17:54:24.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.202+0000 7f30705a9640 1 --2- 172.21.15.105:0/610948561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30681034a0 0x7f3068109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.202+0000 7f30705a9640 1 --2- 172.21.15.105:0/610948561 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3068102ae0 0x7f3068102f60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.202+0000 7f30705a9640 1 --2- 172.21.15.105:0/610948561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30681018f0 0x7f3068101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.202+0000 7f30705a9640 1 -- 172.21.15.105:0/610948561 >> 172.21.15.105:0/610948561 conn(0x7f30680fce60 msgr2=0x7f30680ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.202+0000 7f30705a9640 1 -- 172.21.15.105:0/610948561 shutdown_connections 2024-03-26T17:54:24.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.202+0000 7f30705a9640 1 -- 172.21.15.105:0/610948561 wait complete. 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f30705a9640 1 Processor -- start 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f30705a9640 1 -- start start 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f30705a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30681018f0 0x7f3068197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f30705a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3068102ae0 0x7f3068198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f30705a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30681034a0 0x7f306819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f30705a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f306810d470 con 0x7f30681018f0 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f30705a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f306810d2f0 con 0x7f3068102ae0 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f30705a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f306810d5f0 con 0x7f30681034a0 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30681018f0 0x7f3068197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30681018f0 0x7f3068197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34840/0 (socket says 172.21.15.105:34840) 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 -- 172.21.15.105:0/2784792095 learned_addr learned my addr 172.21.15.105:0/2784792095 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306eb46640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30681034a0 0x7f306819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306db44640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3068102ae0 0x7f3068198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 -- 172.21.15.105:0/2784792095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30681034a0 msgr2=0x7f306819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30681034a0 0x7f306819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 -- 172.21.15.105:0/2784792095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3068102ae0 msgr2=0x7f3068198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3068102ae0 0x7f3068198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 -- 172.21.15.105:0/2784792095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306819f760 con 0x7f30681018f0 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.203+0000 7f306e345640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30681018f0 0x7f3068197b40 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9c0 tx=0x7f306400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.204+0000 7f305f7fe640 1 -- 172.21.15.105:0/2784792095 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306400cde0 con 0x7f30681018f0 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.204+0000 7f305f7fe640 1 -- 172.21.15.105:0/2784792095 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30640108e0 con 0x7f30681018f0 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.204+0000 7f305f7fe640 1 -- 172.21.15.105:0/2784792095 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064018990 con 0x7f30681018f0 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.204+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3068073b50 con 0x7f30681018f0 2024-03-26T17:54:24.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.204+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3068074070 con 0x7f30681018f0 2024-03-26T17:54:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.205+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30680686b0 con 0x7f30681018f0 2024-03-26T17:54:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.206+0000 7f305f7fe640 1 -- 172.21.15.105:0/2784792095 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f306401f020 con 0x7f30681018f0 2024-03-26T17:54:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.207+0000 7f305f7fe640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30540a2f20 0x7f30540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.207+0000 7f305f7fe640 1 -- 172.21.15.105:0/2784792095 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f30640cf670 con 0x7f30681018f0 2024-03-26T17:54:24.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.207+0000 7f306db44640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30540a2f20 0x7f30540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.208+0000 7f306db44640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30540a2f20 0x7f30540a53e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3058006010 tx=0x7f3058000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.214+0000 7f305f7fe640 1 -- 172.21.15.105:0/2784792095 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f306409c460 con 0x7f30681018f0 2024-03-26T17:54:24.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:23 smithi144 ceph-mon[36095]: pgmap v830: 97 pgs: 97 active+clean; 257 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.5 MiB/s wr, 217 op/s 2024-03-26T17:54:24.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.559+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3068108cb0 con 0x7f30681018f0 2024-03-26T17:54:24.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.560+0000 7f305f7fe640 1 -- 172.21.15.105:0/2784792095 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11941 (secure 0 0 0) 0x7f306409bd60 con 0x7f30681018f0 2024-03-26T17:54:24.549 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:54:24.550 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:54:01.817010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:54:24.550 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 37 2024-03-26T17:54:24.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 24642 7.15 MB/sec execute 67 sec latency 125.710 ms 2024-03-26T17:54:24.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30540a2f20 msgr2=0x7f30540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30540a2f20 0x7f30540a53e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3058006010 tx=0x7f3058000bc0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30681018f0 msgr2=0x7f3068197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30681018f0 0x7f3068197b40 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f306400e9c0 tx=0x7f306400ee80 comp rx=0 tx=0).stop 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 shutdown_connections 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30540a2f20 0x7f30540a53e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30681034a0 0x7f306819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3068102ae0 0x7f3068198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 --2- 172.21.15.105:0/2784792095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30681018f0 0x7f3068197b40 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 >> 172.21.15.105:0/2784792095 conn(0x7f30680fce60 msgr2=0x7f3068105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 shutdown_connections 2024-03-26T17:54:24.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.563+0000 7f30705a9640 1 -- 172.21.15.105:0/2784792095 wait complete. 2024-03-26T17:54:24.566 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:54:24.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 -- 172.21.15.105:0/2426306959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc25c102380 msgr2=0x7fc25c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 --2- 172.21.15.105:0/2426306959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc25c102380 0x7fc25c104770 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00fb10 tx=0x7fc24c02fdb0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 -- 172.21.15.105:0/2426306959 shutdown_connections 2024-03-26T17:54:24.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 --2- 172.21.15.105:0/2426306959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25c1075e0 0x7fc25c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 --2- 172.21.15.105:0/2426306959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc25c104cb0 0x7fc25c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 --2- 172.21.15.105:0/2426306959 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc25c102380 0x7fc25c104770 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 -- 172.21.15.105:0/2426306959 >> 172.21.15.105:0/2426306959 conn(0x7fc25c0fbd70 msgr2=0x7fc25c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 -- 172.21.15.105:0/2426306959 shutdown_connections 2024-03-26T17:54:24.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.716+0000 7fc262a26640 1 -- 172.21.15.105:0/2426306959 wait complete. 2024-03-26T17:54:24.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.717+0000 7fc262a26640 1 Processor -- start 2024-03-26T17:54:24.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.717+0000 7fc262a26640 1 -- start start 2024-03-26T17:54:24.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.717+0000 7fc262a26640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25c102380 0x7fc25c1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.717+0000 7fc262a26640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc25c104cb0 0x7fc25c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.717+0000 7fc262a26640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc25c1075e0 0x7fc25c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.717+0000 7fc262a26640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c10b270 con 0x7fc25c1075e0 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.717+0000 7fc262a26640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c10b0f0 con 0x7fc25c104cb0 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.717+0000 7fc262a26640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c10b3f0 con 0x7fc25c102380 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25c102380 0x7fc25c1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25c102380 0x7fc25c1959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59350/0 (socket says 172.21.15.105:59350) 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 -- 172.21.15.105:0/1977342311 learned_addr learned my addr 172.21.15.105:0/1977342311 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc25c104cb0 0x7fc25c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc260fc3640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc25c1075e0 0x7fc25c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 -- 172.21.15.105:0/1977342311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc25c104cb0 msgr2=0x7fc25c195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc25c104cb0 0x7fc25c195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 -- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc25c1075e0 msgr2=0x7fc25c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc25c1075e0 0x7fc25c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 -- 172.21.15.105:0/1977342311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25c19d5d0 con 0x7fc25c102380 2024-03-26T17:54:24.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc25c104cb0 0x7fc25c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.718+0000 7fc25bfff640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25c102380 0x7fc25c1959b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006010 tx=0x7fc24c0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.719+0000 7fc2597fa640 1 -- 172.21.15.105:0/1977342311 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00f540 con 0x7fc25c102380 2024-03-26T17:54:24.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.719+0000 7fc262a26640 1 -- 172.21.15.105:0/1977342311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25c19d860 con 0x7fc25c102380 2024-03-26T17:54:24.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.719+0000 7fc2597fa640 1 -- 172.21.15.105:0/1977342311 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc24c004030 con 0x7fc25c102380 2024-03-26T17:54:24.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.719+0000 7fc262a26640 1 -- 172.21.15.105:0/1977342311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25c19dda0 con 0x7fc25c102380 2024-03-26T17:54:24.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.719+0000 7fc2597fa640 1 -- 172.21.15.105:0/1977342311 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c030d50 con 0x7fc25c102380 2024-03-26T17:54:24.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.721+0000 7fc2597fa640 1 -- 172.21.15.105:0/1977342311 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc24c040070 con 0x7fc25c102380 2024-03-26T17:54:24.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc26560c740 1 Processor -- start 2024-03-26T17:54:24.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc26560c740 1 -- start start 2024-03-26T17:54:24.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc26560c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x562608078e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc2597fa640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2440a2fa0 0x7fc2440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc26560c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 0x56260807df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc26560c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56260807e590 0x562608080a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc26560c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562607fdd970 con 0x562608078a30 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc26560c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562607f85d70 con 0x562608079420 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc26560c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562607fd9920 con 0x56260807e590 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.722+0000 7fc25bfff640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2440a2fa0 0x7fc2440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x562608078e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x562608078e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34938/0 (socket says 172.21.15.105:34938) 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 -- 172.21.15.105:0/2142765076 learned_addr learned my addr 172.21.15.105:0/2142765076 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc25b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 0x56260807df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc2597fa640 1 -- 172.21.15.105:0/1977342311 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc24c0f4b20 con 0x7fc25c102380 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 -- 172.21.15.105:0/2142765076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56260807e590 msgr2=0x562608080a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2142765076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56260807e590 0x562608080a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 -- 172.21.15.105:0/2142765076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 msgr2=0x56260807df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2142765076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 0x56260807df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 -- 172.21.15.105:0/2142765076 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562607cf13f0 con 0x562608078a30 2024-03-26T17:54:24.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc25bfff640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2440a2fa0 0x7fc2440a5460 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0c96f0 tx=0x7fc24c0bcd80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2142765076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x562608078e50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc254011470 tx=0x7fc254011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c0bb1de7792af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.723+0000 7fc2427fc640 1 -- 172.21.15.105:0/2142765076 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254004b20 con 0x562608078a30 2024-03-26T17:54:24.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.724+0000 7fc2427fc640 1 -- 172.21.15.105:0/2142765076 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc254003e20 con 0x562608078a30 2024-03-26T17:54:24.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.724+0000 7fc2427fc640 1 -- 172.21.15.105:0/2142765076 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25400f4e0 con 0x562608078a30 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.724+0000 7fc26560c740 1 -- 172.21.15.105:0/2142765076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 msgr2=0x562608078e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.724+0000 7fc26560c740 1 --2- 172.21.15.105:0/2142765076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x562608078e50 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fc254011470 tx=0x7fc254011930 comp rx=0 tx=0).stop 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 -- 172.21.15.105:0/2142765076 shutdown_connections 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 --2- 172.21.15.105:0/2142765076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56260807e590 0x562608080a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 --2- 172.21.15.105:0/2142765076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 0x56260807df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 --2- 172.21.15.105:0/2142765076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x562608078e50 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 -- 172.21.15.105:0/2142765076 >> 172.21.15.105:0/2142765076 conn(0x562608073fa0 msgr2=0x5626080743a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 -- 172.21.15.105:0/2142765076 shutdown_connections 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 -- 172.21.15.105:0/2142765076 wait complete. 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 -- start start 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.725+0000 7fc26560c740 1 Processor -- start 2024-03-26T17:54:24.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc26560c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260808b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc26560c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 0x56260808bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc26560c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56260807e590 0x562608092cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc26560c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5626080814e0 con 0x562608078a30 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc26560c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562607f99310 con 0x56260807e590 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc26560c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562608081660 con 0x562608079420 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc26560c740 10 client.0 _ll_register_callbacks cb 0x562607fd4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260808b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260808b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34944/0 (socket says 172.21.15.105:34944) 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 -- 172.21.15.105:0/2422435046 learned_addr learned my addr 172.21.15.105:0/2422435046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 0x56260808bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 -- 172.21.15.105:0/2422435046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 msgr2=0x56260808bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 0x56260808bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 -- 172.21.15.105:0/2422435046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56260807e590 msgr2=0x562608092cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56260807e590 0x562608092cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 -- 172.21.15.105:0/2422435046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562607dbb7d0 con 0x562608078a30 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260808b750 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc254006010 tx=0x7fc254003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ac58f0b558d6cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.726+0000 7fc23b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2422435046 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254026070 con 0x562608078a30 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2422435046 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc25400f4e0 con 0x562608078a30 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 1 -- 172.21.15.105:0/2422435046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc254003e20 con 0x562608078a30 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2422435046 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc254032050 con 0x562608078a30 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 10 client.21270 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 1 -- 172.21.15.105:0/2422435046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562608096040 con 0x562608078a30 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2422435046 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc254003e20 con 0x562608078a30 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2422435046 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fc2540226c0 con 0x562608078a30 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 10 client.21270 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 10 client.21270 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 20 client.21270 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562608097a40 0x562608099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 4 client.21270 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:24.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc26560c740 1 -- 172.21.15.105:0/2422435046 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562607feaeb0 con 0x562608097a40 2024-03-26T17:54:24.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.727+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562608097a40 0x562608099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.728+0000 7fc23b7fe640 10 client.21270 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:54:24.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.728+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562608097a40 0x562608099e80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c298c03e6aaf0f server_cookie=60a8a4b5bd0a285b in_seq=0 out_seq=0 2024-03-26T17:54:24.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2422435046 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562607feaeb0 con 0x562608097a40 2024-03-26T17:54:24.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc23b7fe640 10 client.21270 handle_command_reply: tid=0 2024-03-26T17:54:24.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 client.21270 shutdown 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 -- 172.21.15.105:0/2422435046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 msgr2=0x56260808b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260808b750 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc254006010 tx=0x7fc254003ba0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 -- 172.21.15.105:0/2422435046 shutdown_connections 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562608097a40 0x562608099e80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56260807e590 0x562608092cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 0x56260808bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 --2- 172.21.15.105:0/2422435046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260808b750 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 -- 172.21.15.105:0/2422435046 >> 172.21.15.105:0/2422435046 conn(0x562608073fa0 msgr2=0x5626080731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc23b7fe640 0 client.21270 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 -- 172.21.15.105:0/2422435046 shutdown_connections 2024-03-26T17:54:24.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.729+0000 7fc26560c740 1 -- 172.21.15.105:0/2422435046 wait complete. 2024-03-26T17:54:24.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.730+0000 7fc26560c740 20 client.21270 trim_cache size 0 max 16384 2024-03-26T17:54:24.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.752+0000 7fc26560c740 1 Processor -- start 2024-03-26T17:54:24.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.752+0000 7fc26560c740 1 -- start start 2024-03-26T17:54:24.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.752+0000 7fc26560c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608073fa0 0x56260807a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc26560c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260807aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc26560c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 0x562608092c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc26560c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c1a6cd0 con 0x562608078a30 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc26560c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c1a8810 con 0x562608073fa0 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc26560c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c1a73a0 con 0x562608079420 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260807aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260807aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34960/0 (socket says 172.21.15.105:34960) 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 -- 172.21.15.105:0/2352118027 learned_addr learned my addr 172.21.15.105:0/2352118027 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608073fa0 0x56260807a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 0x562608092c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 -- 172.21.15.105:0/2352118027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 msgr2=0x562608092c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 0x562608092c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 -- 172.21.15.105:0/2352118027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608073fa0 msgr2=0x56260807a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608073fa0 0x56260807a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 -- 172.21.15.105:0/2352118027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562608074d10 con 0x562608078a30 2024-03-26T17:54:24.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.753+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260807aef0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc250009fd0 tx=0x7fc250005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218de0b7cd018142 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.754+0000 7fc241ffb640 1 -- 172.21.15.105:0/2352118027 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250004050 con 0x562608078a30 2024-03-26T17:54:24.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.754+0000 7fc241ffb640 1 -- 172.21.15.105:0/2352118027 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2500054d0 con 0x562608078a30 2024-03-26T17:54:24.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.754+0000 7fc241ffb640 1 -- 172.21.15.105:0/2352118027 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25000fc80 con 0x562608078a30 2024-03-26T17:54:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 -- 172.21.15.105:0/2352118027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 msgr2=0x56260807aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260807aef0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc250009fd0 tx=0x7fc250005f90 comp rx=0 tx=0).stop 2024-03-26T17:54:24.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 -- 172.21.15.105:0/2352118027 shutdown_connections 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608079420 0x562608092c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608078a30 0x56260807aef0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 --2- 172.21.15.105:0/2352118027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608073fa0 0x56260807a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 -- 172.21.15.105:0/2352118027 >> 172.21.15.105:0/2352118027 conn(0x56260808a600 msgr2=0x562608074380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 -- 172.21.15.105:0/2352118027 shutdown_connections 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 -- 172.21.15.105:0/2352118027 wait complete. 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 -- start start 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.755+0000 7fc26560c740 1 Processor -- start 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc26560c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608073fa0 0x562608092610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc26560c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608078a30 0x562608092b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc26560c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 0x56260808f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc26560c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c1a8140 con 0x562608073fa0 2024-03-26T17:54:24.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc26560c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c1a7a70 con 0x562608079420 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc26560c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc25c1a4cb0 con 0x562608078a30 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc26560c740 10 client.0 _ll_register_callbacks cb 0x7fc250009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc260fc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608073fa0 0x562608092610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc260fc3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608073fa0 0x562608092610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34974/0 (socket says 172.21.15.105:34974) 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc25b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608078a30 0x562608092b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc260fc3640 1 -- 172.21.15.105:0/2835585348 learned_addr learned my addr 172.21.15.105:0/2835585348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 0x56260808f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc25b7fe640 1 -- 172.21.15.105:0/2835585348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 msgr2=0x56260808f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 0x56260808f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc25b7fe640 1 -- 172.21.15.105:0/2835585348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608073fa0 msgr2=0x562608092610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608073fa0 0x562608092610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc25b7fe640 1 -- 172.21.15.105:0/2835585348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260808f7a0 con 0x562608078a30 2024-03-26T17:54:24.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608073fa0 0x562608092610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc25b7fe640 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608078a30 0x562608092b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc250000c00 tx=0x7fc250005130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41e0824aed931334 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.756+0000 7fc23b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2835585348 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25002f060 con 0x562608078a30 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2835585348 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fc250004050 con 0x562608078a30 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 1 -- 172.21.15.105:0/2835585348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2500054d0 con 0x562608078a30 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2835585348 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc250005320 con 0x562608078a30 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 10 client.31015 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 1 -- 172.21.15.105:0/2835585348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5626080fca60 con 0x562608078a30 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2835585348 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2500054d0 con 0x562608078a30 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2835585348 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fc2500202a0 con 0x562608078a30 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 10 client.31015 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 10 client.31015 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 20 client.31015 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:24.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56260807e590 0x5626080ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 4 client.31015 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.757+0000 7fc26560c740 1 -- 172.21.15.105:0/2835585348 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562607fe5200 con 0x56260807e590 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56260807e590 0x5626080ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc23b7fe640 10 client.31015 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc260fc3640 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56260807e590 0x5626080ff420 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf1109a812e12773 server_cookie=c6306f1e12dfd474 in_seq=0 out_seq=0 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc23b7fe640 1 -- 172.21.15.105:0/2835585348 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562607fe5200 con 0x56260807e590 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc23b7fe640 10 client.31015 handle_command_reply: tid=0 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 client.31015 shutdown 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 -- 172.21.15.105:0/2835585348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608078a30 msgr2=0x562608092b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608078a30 0x562608092b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc250000c00 tx=0x7fc250005130 comp rx=0 tx=0).stop 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 -- 172.21.15.105:0/2835585348 shutdown_connections 2024-03-26T17:54:24.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56260807e590 0x5626080ff420 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562608079420 0x56260808f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562608078a30 0x562608092b50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc23b7fe640 0 client.31015 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 --2- 172.21.15.105:0/2835585348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562608073fa0 0x562608092610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.758+0000 7fc26560c740 1 -- 172.21.15.105:0/2835585348 >> 172.21.15.105:0/2835585348 conn(0x56260808a600 msgr2=0x56260807a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.759+0000 7fc26560c740 1 -- 172.21.15.105:0/2835585348 shutdown_connections 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.759+0000 7fc26560c740 1 -- 172.21.15.105:0/2835585348 wait complete. 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.759+0000 7fc26560c740 20 client.31015 trim_cache size 0 max 16384 2024-03-26T17:54:24.753 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.759+0000 7fc262a26640 1 -- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2440a2fa0 msgr2=0x7fc2440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.759+0000 7fc262a26640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2440a2fa0 0x7fc2440a5460 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0c96f0 tx=0x7fc24c0bcd80 comp rx=0 tx=0).stop 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.759+0000 7fc262a26640 1 -- 172.21.15.105:0/1977342311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25c102380 msgr2=0x7fc25c1959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.759+0000 7fc262a26640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25c102380 0x7fc25c1959b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc24c006010 tx=0x7fc24c0048f0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.760+0000 7fc262a26640 1 -- 172.21.15.105:0/1977342311 shutdown_connections 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.760+0000 7fc262a26640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc2440a2fa0 0x7fc2440a5460 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.760+0000 7fc262a26640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc25c1075e0 0x7fc25c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.760+0000 7fc262a26640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc25c104cb0 0x7fc25c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.760+0000 7fc262a26640 1 --2- 172.21.15.105:0/1977342311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25c102380 0x7fc25c1959b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.760+0000 7fc262a26640 1 -- 172.21.15.105:0/1977342311 >> 172.21.15.105:0/1977342311 conn(0x7fc25c0fbd70 msgr2=0x7fc25c1042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.760+0000 7fc262a26640 1 -- 172.21.15.105:0/1977342311 shutdown_connections 2024-03-26T17:54:24.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.760+0000 7fc262a26640 1 -- 172.21.15.105:0/1977342311 wait complete. 2024-03-26T17:54:24.763 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:54:24.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 -- 172.21.15.105:0/4178376830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 msgr2=0x7fbfac101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/4178376830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 0x7fbfac101e30 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009900 tx=0x7fbfa002bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:24.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 -- 172.21.15.105:0/4178376830 shutdown_connections 2024-03-26T17:54:24.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/4178376830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfac102370 0x7fbfac108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/4178376830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 0x7fbfac101e30 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/4178376830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfac1007c0 0x7fbfac100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 -- 172.21.15.105:0/4178376830 >> 172.21.15.105:0/4178376830 conn(0x7fbfac0fbd50 msgr2=0x7fbfac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 -- 172.21.15.105:0/4178376830 shutdown_connections 2024-03-26T17:54:24.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 -- 172.21.15.105:0/4178376830 wait complete. 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.924+0000 7fbfb1b53640 1 Processor -- start 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfb1b53640 1 -- start start 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfb1b53640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfac1007c0 0x7fbfac10e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfb1b53640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 0x7fbfac10ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfb1b53640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfac102370 0x7fbfac10d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfb1b53640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac10c2d0 con 0x7fbfac1019b0 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfb1b53640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac10c150 con 0x7fbfac102370 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfb1b53640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac10c450 con 0x7fbfac1007c0 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfac1007c0 0x7fbfac10e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfaaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 0x7fbfac10ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfac1007c0 0x7fbfac10e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59394/0 (socket says 172.21.15.105:59394) 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1278127126 learned_addr learned my addr 172.21.15.105:0/1278127126 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfabfff640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfac102370 0x7fbfac10d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1278127126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfac102370 msgr2=0x7fbfac10d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.925+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfac102370 0x7fbfac10d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 msgr2=0x7fbfac10ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 0x7fbfac10ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1278127126 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac10d8e0 con 0x7fbfac1007c0 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 0x7fbfac10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfac1007c0 0x7fbfac10e820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00e9c0 tx=0x7fbf9c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfa8ff9640 1 -- 172.21.15.105:0/1278127126 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00cde0 con 0x7fbfac1007c0 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfb1b53640 1 -- 172.21.15.105:0/1278127126 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac10dbd0 con 0x7fbfac1007c0 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfa8ff9640 1 -- 172.21.15.105:0/1278127126 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf9c0108e0 con 0x7fbfac1007c0 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfa8ff9640 1 -- 172.21.15.105:0/1278127126 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c0188e0 con 0x7fbfac1007c0 2024-03-26T17:54:24.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.926+0000 7fbfb1b53640 1 -- 172.21.15.105:0/1278127126 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac1a6610 con 0x7fbfac1007c0 2024-03-26T17:54:24.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfb4819740 1 Processor -- start 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfb4819740 1 -- start start 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfa8ff9640 1 -- 172.21.15.105:0/1278127126 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbf9c01f020 con 0x7fbfac1007c0 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2dc4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2dc9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2dcca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfb4819740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5612c2d29970 con 0x5612c2dca590 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfb4819740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5612c2cd1d70 con 0x5612c2dc4a30 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.929+0000 7fbfb4819740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5612c2d25920 con 0x5612c2dc5420 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2dc9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2dcca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfabfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2dc4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2dcca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35006/0 (socket says 172.21.15.105:35006) 2024-03-26T17:54:24.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2dc9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59396/0 (socket says 172.21.15.105:59396) 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 -- 172.21.15.105:0/3350627437 learned_addr learned my addr 172.21.15.105:0/3350627437 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 -- 172.21.15.105:0/3350627437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 msgr2=0x5612c2dc9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2dc9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 -- 172.21.15.105:0/3350627437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 msgr2=0x5612c2dc4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2dc4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 -- 172.21.15.105:0/3350627437 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c2a3d3f0 con 0x5612c2dca590 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfa8ff9640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbf940a2f20 0x7fbf940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2dc9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfabfff640 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2dc4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfa8ff9640 1 -- 172.21.15.105:0/1278127126 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbf9c0cf5b0 con 0x7fbfac1007c0 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2dcca20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009900 tx=0x7fbfa00095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fc53ea37e2511a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.930+0000 7fbfabfff640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbf940a2f20 0x7fbf940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.931+0000 7fbf91ffb640 1 -- 172.21.15.105:0/3350627437 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0045020 con 0x5612c2dca590 2024-03-26T17:54:24.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.931+0000 7fbf91ffb640 1 -- 172.21.15.105:0/3350627437 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa002cc90 con 0x5612c2dca590 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.931+0000 7fbf91ffb640 1 -- 172.21.15.105:0/3350627437 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0004920 con 0x5612c2dca590 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.931+0000 7fbfb4819740 1 -- 172.21.15.105:0/3350627437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 msgr2=0x5612c2dcca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.931+0000 7fbfabfff640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbf940a2f20 0x7fbf940a53e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x5612c2dc5c80 tx=0x7fbfa4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.931+0000 7fbfb4819740 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2dcca20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009900 tx=0x7fbfa00095f0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 -- 172.21.15.105:0/3350627437 shutdown_connections 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2dcca20 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2dc9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 --2- 172.21.15.105:0/3350627437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2dc4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 -- 172.21.15.105:0/3350627437 >> 172.21.15.105:0/3350627437 conn(0x5612c2dbffa0 msgr2=0x5612c2dc03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 -- 172.21.15.105:0/3350627437 shutdown_connections 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 -- 172.21.15.105:0/3350627437 wait complete. 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 -- start start 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.932+0000 7fbfb4819740 1 Processor -- start 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2ddfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2de01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2de7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfb4819740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5612c2dde4f0 con 0x5612c2dca590 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfb4819740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5612c2ce5310 con 0x5612c2dc4a30 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfb4819740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5612c2dde670 con 0x5612c2dc5420 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfab7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2ddfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2de7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfabfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2de7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35018/0 (socket says 172.21.15.105:35018) 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfabfff640 1 -- 172.21.15.105:0/1671268095 learned_addr learned my addr 172.21.15.105:0/1671268095 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2ddfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56872/0 (socket says 172.21.15.105:56872) 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfb4819740 10 client.0 _ll_register_callbacks cb 0x5612c2d20c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2de01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1671268095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 msgr2=0x5612c2de01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2de01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1671268095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 msgr2=0x5612c2de7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2de7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1671268095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c2b077d0 con 0x5612c2dc4a30 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfabfff640 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2de7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2ddfc80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0cd0c0 tx=0x7fbf9c097f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=839a0231d9a64957 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.933+0000 7fbf8affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbf8affd640 1 -- 172.21.15.105:0/1671268095 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c09b600 con 0x5612c2dc4a30 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 1 -- 172.21.15.105:0/1671268095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa002cc90 con 0x5612c2dc4a30 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbf8affd640 1 -- 172.21.15.105:0/1671268095 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbf9c0a4220 con 0x5612c2dc4a30 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbf8affd640 1 -- 172.21.15.105:0/1671268095 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbf9c010e20 con 0x5612c2dc4a30 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 10 client.31064 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 1 -- 172.21.15.105:0/1671268095 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5612c2dea630 con 0x5612c2dc4a30 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbf8affd640 1 -- 172.21.15.105:0/1671268095 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa002cc90 con 0x5612c2dc4a30 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbf8affd640 1 -- 172.21.15.105:0/1671268095 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fbf9c0b1030 con 0x5612c2dc4a30 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 10 client.31064 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:24.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 10 client.31064 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 20 client.31064 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5612c2debc40 0x5612c2dee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 4 client.31064 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfb4819740 1 -- 172.21.15.105:0/1671268095 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612c2d36eb0 con 0x5612c2debc40 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.934+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5612c2debc40 0x5612c2dee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.935+0000 7fbf8affd640 10 client.31064 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.935+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5612c2debc40 0x5612c2dee0b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c1ba34c048edf84 server_cookie=2f28978b6748e47f in_seq=0 out_seq=0 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbf8affd640 1 -- 172.21.15.105:0/1671268095 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5612c2d36eb0 con 0x5612c2debc40 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbf8affd640 10 client.31064 handle_command_reply: tid=0 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 client.31064 shutdown 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 -- 172.21.15.105:0/1671268095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 msgr2=0x5612c2ddfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2ddfc80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0cd0c0 tx=0x7fbf9c097f40 comp rx=0 tx=0).stop 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 -- 172.21.15.105:0/1671268095 shutdown_connections 2024-03-26T17:54:24.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5612c2debc40 0x5612c2dee0b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dca590 0x5612c2de7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc5420 0x5612c2de01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbf8affd640 0 client.31064 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:54:24.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1671268095 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc4a30 0x5612c2ddfc80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 -- 172.21.15.105:0/1671268095 >> 172.21.15.105:0/1671268095 conn(0x5612c2dbffa0 msgr2=0x5612c2dbf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 -- 172.21.15.105:0/1671268095 shutdown_connections 2024-03-26T17:54:24.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.936+0000 7fbfb4819740 1 -- 172.21.15.105:0/1671268095 wait complete. 2024-03-26T17:54:24.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.937+0000 7fbfb4819740 20 client.31064 trim_cache size 0 max 16384 2024-03-26T17:54:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.958+0000 7fbfb4819740 1 Processor -- start 2024-03-26T17:54:24.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.958+0000 7fbfb4819740 1 -- start start 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc4a30 0x5612c2dc3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc5420 0x5612c2de7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dca590 0x5612c2dc3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfaaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc5420 0x5612c2de7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfaaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc5420 0x5612c2de7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35032/0 (socket says 172.21.15.105:35032) 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc4a30 0x5612c2dc3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfaaffd640 1 -- 172.21.15.105:0/4049174397 learned_addr learned my addr 172.21.15.105:0/4049174397 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfb4819740 1 -- 172.21.15.105:0/4049174397 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1a6af0 con 0x5612c2dc5420 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfb4819740 1 -- 172.21.15.105:0/4049174397 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1a8630 con 0x5612c2dca590 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfb4819740 1 -- 172.21.15.105:0/4049174397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1a71c0 con 0x5612c2dc4a30 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfabfff640 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dca590 0x5612c2dc3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfab7fe640 1 -- 172.21.15.105:0/4049174397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dca590 msgr2=0x5612c2dc3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dca590 0x5612c2dc3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfab7fe640 1 -- 172.21.15.105:0/4049174397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc5420 msgr2=0x5612c2de7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc5420 0x5612c2de7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfab7fe640 1 -- 172.21.15.105:0/4049174397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c2dc5cb0 con 0x5612c2dc4a30 2024-03-26T17:54:24.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfabfff640 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dca590 0x5612c2dc3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:54:24.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.959+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc5420 0x5612c2de7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:24.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.960+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc4a30 0x5612c2dc3200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0cd3e0 tx=0x7fbf9c09a1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4198045bc40347 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.960+0000 7fbf917fa640 1 -- 172.21.15.105:0/4049174397 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c09b600 con 0x5612c2dc4a30 2024-03-26T17:54:24.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.960+0000 7fbf917fa640 1 -- 172.21.15.105:0/4049174397 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf9c09b0f0 con 0x5612c2dc4a30 2024-03-26T17:54:24.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.960+0000 7fbf917fa640 1 -- 172.21.15.105:0/4049174397 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c09ba70 con 0x5612c2dc4a30 2024-03-26T17:54:24.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 -- 172.21.15.105:0/4049174397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc4a30 msgr2=0x5612c2dc3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc4a30 0x5612c2dc3200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0cd3e0 tx=0x7fbf9c09a1b0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 -- 172.21.15.105:0/4049174397 shutdown_connections 2024-03-26T17:54:24.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dca590 0x5612c2dc3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc5420 0x5612c2de7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 --2- 172.21.15.105:0/4049174397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dc4a30 0x5612c2dc3200 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 -- 172.21.15.105:0/4049174397 >> 172.21.15.105:0/4049174397 conn(0x5612c2dbffa0 msgr2=0x5612c2ddea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 -- 172.21.15.105:0/4049174397 shutdown_connections 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 -- 172.21.15.105:0/4049174397 wait complete. 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.961+0000 7fbfb4819740 1 -- start start 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfb4819740 1 Processor -- start 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc4a30 0x5612c2de4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc5420 0x5612c2de4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfb4819740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dca590 0x5612c2de5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfb4819740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1a7f60 con 0x5612c2dc4a30 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfb4819740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1a7890 con 0x5612c2dc5420 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfb4819740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac071de0 con 0x5612c2dca590 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfb4819740 10 client.0 _ll_register_callbacks cb 0x7fbf9c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc4a30 0x5612c2de4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc4a30 0x5612c2de4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35040/0 (socket says 172.21.15.105:35040) 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1673704163 learned_addr learned my addr 172.21.15.105:0/1673704163 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfabfff640 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dca590 0x5612c2de5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc5420 0x5612c2de4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1673704163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dca590 msgr2=0x5612c2de5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dca590 0x5612c2de5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1673704163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc5420 msgr2=0x5612c2de4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc5420 0x5612c2de4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.962+0000 7fbfab7fe640 1 -- 172.21.15.105:0/1673704163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c2de6010 con 0x5612c2dc4a30 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbfab7fe640 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc4a30 0x5612c2de4590 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c09bc30 tx=0x7fbf9c098070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aae4bf2474123bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbf8affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbf8affd640 1 -- 172.21.15.105:0/1673704163 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c09ba70 con 0x5612c2dc4a30 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbfb4819740 1 -- 172.21.15.105:0/1673704163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612c2d31200 con 0x5612c2dc4a30 2024-03-26T17:54:24.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbf8affd640 1 -- 172.21.15.105:0/1673704163 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbf9c0979f0 con 0x5612c2dc4a30 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbf8affd640 1 -- 172.21.15.105:0/1673704163 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbf9c09b600 con 0x5612c2dc4a30 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbfb4819740 10 client.21294 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbfb4819740 1 -- 172.21.15.105:0/1673704163 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5612c2de7280 con 0x5612c2dc4a30 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbf8affd640 1 -- 172.21.15.105:0/1673704163 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5612c2d31200 con 0x5612c2dc4a30 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbf8affd640 1 -- 172.21.15.105:0/1673704163 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fbf9c0b1070 con 0x5612c2dc4a30 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbfb4819740 10 client.21294 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbfb4819740 10 client.21294 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbfb4819740 20 client.21294 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.963+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5612c2e52460 0x5612c2e54880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.964+0000 7fbfb4819740 4 client.21294 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.964+0000 7fbfb4819740 1 -- 172.21.15.105:0/1673704163 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612c2cae6a0 con 0x5612c2e52460 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.964+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5612c2e52460 0x5612c2e54880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.964+0000 7fbf8affd640 10 client.21294 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:54:24.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.964+0000 7fbfaaffd640 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5612c2e52460 0x5612c2e54880 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8509030b912c17d4 server_cookie=684c797648443156 in_seq=0 out_seq=0 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.964+0000 7fbf8affd640 1 -- 172.21.15.105:0/1673704163 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612c2cae6a0 con 0x5612c2e52460 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.964+0000 7fbf8affd640 10 client.21294 handle_command_reply: tid=0 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.964+0000 7fbfb4819740 1 client.21294 shutdown 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 -- 172.21.15.105:0/1673704163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc4a30 msgr2=0x5612c2de4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc4a30 0x5612c2de4590 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c09bc30 tx=0x7fbf9c098070 comp rx=0 tx=0).stop 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 -- 172.21.15.105:0/1673704163 shutdown_connections 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5612c2e52460 0x5612c2e54880 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5612c2dca590 0x5612c2de5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5612c2dc5420 0x5612c2de4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 --2- 172.21.15.105:0/1673704163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5612c2dc4a30 0x5612c2de4590 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 -- 172.21.15.105:0/1673704163 >> 172.21.15.105:0/1673704163 conn(0x5612c2dbffa0 msgr2=0x5612c2dc66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbf8affd640 0 client.21294 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 -- 172.21.15.105:0/1673704163 shutdown_connections 2024-03-26T17:54:24.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 1 -- 172.21.15.105:0/1673704163 wait complete. 2024-03-26T17:54:24.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.965+0000 7fbfb4819740 20 client.21294 trim_cache size 0 max 16384 2024-03-26T17:54:24.960 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 -- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbf940a2f20 msgr2=0x7fbf940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbf940a2f20 0x7fbf940a53e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x5612c2dc5c80 tx=0x7fbfa4009500 comp rx=0 tx=0).stop 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 -- 172.21.15.105:0/1278127126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfac1007c0 msgr2=0x7fbfac10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfac1007c0 0x7fbfac10e820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00e9c0 tx=0x7fbf9c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 -- 172.21.15.105:0/1278127126 shutdown_connections 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbf940a2f20 0x7fbf940a53e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbfac102370 0x7fbfac10d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfac1019b0 0x7fbfac10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 --2- 172.21.15.105:0/1278127126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfac1007c0 0x7fbfac10e820 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:24.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 -- 172.21.15.105:0/1278127126 >> 172.21.15.105:0/1278127126 conn(0x7fbfac0fbd50 msgr2=0x7fbfac0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:24.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 -- 172.21.15.105:0/1278127126 shutdown_connections 2024-03-26T17:54:24.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:24.966+0000 7fbfb1b53640 1 -- 172.21.15.105:0/1278127126 wait complete. 2024-03-26T17:54:24.967 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:54:25.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 -- 172.21.15.105:0/3262022468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 msgr2=0x7feea0104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 --2- 172.21.15.105:0/3262022468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 0x7feea0104730 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009900 tx=0x7fee9c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 -- 172.21.15.105:0/3262022468 shutdown_connections 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 --2- 172.21.15.105:0/3262022468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea01075a0 0x7feea01099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 --2- 172.21.15.105:0/3262022468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feea0104c70 0x7feea0107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 --2- 172.21.15.105:0/3262022468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 0x7feea0104730 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 -- 172.21.15.105:0/3262022468 >> 172.21.15.105:0/3262022468 conn(0x7feea00fbd50 msgr2=0x7feea00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 -- 172.21.15.105:0/3262022468 shutdown_connections 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.121+0000 7feea77d4640 1 -- 172.21.15.105:0/3262022468 wait complete. 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea77d4640 1 Processor -- start 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea77d4640 1 -- start start 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea77d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 0x7feea0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea77d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feea0104c70 0x7feea01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea77d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea01075a0 0x7feea019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea77d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feea010d460 con 0x7feea0102340 2024-03-26T17:54:25.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea77d4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feea010d2e0 con 0x7feea0104c70 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea77d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feea010d5e0 con 0x7feea01075a0 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea5570640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 0x7feea0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea5570640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 0x7feea0197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35064/0 (socket says 172.21.15.105:35064) 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea5570640 1 -- 172.21.15.105:0/1302443761 learned_addr learned my addr 172.21.15.105:0/1302443761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea4d6f640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feea0104c70 0x7feea01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.122+0000 7feea5d71640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea01075a0 0x7feea019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7feea5570640 1 -- 172.21.15.105:0/1302443761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea01075a0 msgr2=0x7feea019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7feea5570640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea01075a0 0x7feea019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7feea5570640 1 -- 172.21.15.105:0/1302443761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feea0104c70 msgr2=0x7feea01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7feea5570640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feea0104c70 0x7feea01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7feea5570640 1 -- 172.21.15.105:0/1302443761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea019f7b0 con 0x7feea0102340 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7feea5570640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 0x7feea0197b90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fee9c000c00 tx=0x7fee9c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7fee967fc640 1 -- 172.21.15.105:0/1302443761 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c043020 con 0x7feea0102340 2024-03-26T17:54:25.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7feea77d4640 1 -- 172.21.15.105:0/1302443761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea019fa40 con 0x7feea0102340 2024-03-26T17:54:25.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7fee967fc640 1 -- 172.21.15.105:0/1302443761 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee9c0044c0 con 0x7feea0102340 2024-03-26T17:54:25.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7fee967fc640 1 -- 172.21.15.105:0/1302443761 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c003c30 con 0x7feea0102340 2024-03-26T17:54:25.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.123+0000 7feea77d4640 1 -- 172.21.15.105:0/1302443761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea019ffe0 con 0x7feea0102340 2024-03-26T17:54:25.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.126+0000 7fee967fc640 1 -- 172.21.15.105:0/1302443761 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fee9c046020 con 0x7feea0102340 2024-03-26T17:54:25.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.126+0000 7feeaa3c2740 1 Processor -- start 2024-03-26T17:54:25.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.126+0000 7feeaa3c2740 1 -- start start 2024-03-26T17:54:25.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.126+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b0e40 0x55ffae7b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.126+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ffae711970 con 0x55ffae7b1830 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ffae6b9d70 con 0x55ffae7b0e40 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ffae70d920 con 0x55ffae7b69a0 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5d71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b0e40 0x55ffae7b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5d71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b0e40 0x55ffae7b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56894/0 (socket says 172.21.15.105:56894) 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5d71640 1 -- 172.21.15.105:0/1145996409 learned_addr learned my addr 172.21.15.105:0/1145996409 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5570640 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7fee967fc640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fee8c0a2f20 0x7fee8c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea4d6f640 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5d71640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fee8c0a2f20 0x7fee8c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5570640 1 -- 172.21.15.105:0/1145996409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 msgr2=0x55ffae7b8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5570640 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7b8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5570640 1 -- 172.21.15.105:0/1145996409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b0e40 msgr2=0x55ffae7b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5570640 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b0e40 0x55ffae7b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea5570640 1 -- 172.21.15.105:0/1145996409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffae4253f0 con 0x55ffae7b1830 2024-03-26T17:54:25.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.127+0000 7feea4d6f640 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:54:25.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.128+0000 7feea5570640 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b6330 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0ba6a0 tx=0x7fee9c0bd400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473e2f2c6a1eb80a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.128+0000 7fee8b7fe640 1 -- 172.21.15.105:0/1145996409 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0f1c30 con 0x55ffae7b1830 2024-03-26T17:54:25.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.128+0000 7fee8b7fe640 1 -- 172.21.15.105:0/1145996409 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee9c0ba4c0 con 0x55ffae7b1830 2024-03-26T17:54:25.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.128+0000 7fee8b7fe640 1 -- 172.21.15.105:0/1145996409 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c087f90 con 0x55ffae7b1830 2024-03-26T17:54:25.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.128+0000 7fee967fc640 1 -- 172.21.15.105:0/1302443761 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fee9c0f1640 con 0x7feea0102340 2024-03-26T17:54:25.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 -- 172.21.15.105:0/1145996409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 msgr2=0x55ffae7b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feea5d71640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fee8c0a2f20 0x7fee8c0a53e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fee98013a40 tx=0x7fee98009c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b6330 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0ba6a0 tx=0x7fee9c0bd400 comp rx=0 tx=0).stop 2024-03-26T17:54:25.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 -- 172.21.15.105:0/1145996409 shutdown_connections 2024-03-26T17:54:25.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b6330 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/1145996409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b0e40 0x55ffae7b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 -- 172.21.15.105:0/1145996409 >> 172.21.15.105:0/1145996409 conn(0x55ffae7a7fa0 msgr2=0x55ffae7a83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 -- 172.21.15.105:0/1145996409 shutdown_connections 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 -- 172.21.15.105:0/1145996409 wait complete. 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 -- start start 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.129+0000 7feeaa3c2740 1 Processor -- start 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 0x55ffae7b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 0x55ffae7b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ffae7b9710 con 0x55ffae7b1830 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ffae6cd310 con 0x55ffae7b69a0 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ffae7b9890 con 0x55ffae7b0e40 2024-03-26T17:54:25.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feeaa3c2740 10 client.0 _ll_register_callbacks cb 0x55ffae708c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea4d6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea4d6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35070/0 (socket says 172.21.15.105:35070) 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea4d6f640 1 -- 172.21.15.105:0/121006800 learned_addr learned my addr 172.21.15.105:0/121006800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea5d71640 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 0x55ffae7b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea4d6f640 1 -- 172.21.15.105:0/121006800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 msgr2=0x55ffae7b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea4d6f640 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 0x55ffae7b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea4d6f640 1 -- 172.21.15.105:0/121006800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 msgr2=0x55ffae7b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea4d6f640 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 0x55ffae7b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.130+0000 7feea4d6f640 1 -- 172.21.15.105:0/121006800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffae4ef7d0 con 0x55ffae7b1830 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7feea4d6f640 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b3d90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fee900136e0 tx=0x7fee90009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d739f2f8de1a1ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7fee88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7fee88ff9640 1 -- 172.21.15.105:0/121006800 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90004440 con 0x55ffae7b1830 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7feeaa3c2740 1 -- 172.21.15.105:0/121006800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffae71eeb0 con 0x55ffae7b1830 2024-03-26T17:54:25.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7fee88ff9640 1 -- 172.21.15.105:0/121006800 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fee90011960 con 0x55ffae7b1830 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7fee88ff9640 1 -- 172.21.15.105:0/121006800 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fee90011b60 con 0x55ffae7b1830 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7feeaa3c2740 10 client.21306 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7feeaa3c2740 1 -- 172.21.15.105:0/121006800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ffae7c9d60 con 0x55ffae7b1830 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.131+0000 7fee88ff9640 1 -- 172.21.15.105:0/121006800 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55ffae71eeb0 con 0x55ffae7b1830 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7fee88ff9640 1 -- 172.21.15.105:0/121006800 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fee9001a070 con 0x55ffae7b1830 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7feeaa3c2740 10 client.21306 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7feeaa3c2740 10 client.21306 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7feeaa3c2740 20 client.21306 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ffae7cb500 0x55ffae7cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7feeaa3c2740 4 client.21306 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7feeaa3c2740 1 -- 172.21.15.105:0/121006800 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffae719200 con 0x55ffae7cb500 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7feea5570640 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ffae7cb500 0x55ffae7cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7fee88ff9640 10 client.21306 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.132+0000 7feea5570640 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ffae7cb500 0x55ffae7cd9c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7b9ae912004fc8 server_cookie=b4cf2cf647e7e969 in_seq=0 out_seq=0 2024-03-26T17:54:25.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7fee88ff9640 1 -- 172.21.15.105:0/121006800 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ffae719200 con 0x55ffae7cb500 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7fee88ff9640 10 client.21306 handle_command_reply: tid=0 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 client.21306 shutdown 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 -- 172.21.15.105:0/121006800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 msgr2=0x55ffae7b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b3d90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fee900136e0 tx=0x7fee90009f90 comp rx=0 tx=0).stop 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 -- 172.21.15.105:0/121006800 shutdown_connections 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ffae7cb500 0x55ffae7cd9c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 0x55ffae7b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b1830 0x55ffae7b3d90 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7fee88ff9640 0 client.21306 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/121006800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 0x55ffae7b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.133+0000 7feeaa3c2740 1 -- 172.21.15.105:0/121006800 >> 172.21.15.105:0/121006800 conn(0x55ffae7a7fa0 msgr2=0x55ffae7a71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.134+0000 7feeaa3c2740 1 -- 172.21.15.105:0/121006800 shutdown_connections 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.134+0000 7feeaa3c2740 1 -- 172.21.15.105:0/121006800 wait complete. 2024-03-26T17:54:25.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.134+0000 7feeaa3c2740 20 client.21306 trim_cache size 0 max 16384 2024-03-26T17:54:25.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:24 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2784792095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:54:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.152+0000 7feeaa3c2740 1 Processor -- start 2024-03-26T17:54:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.152+0000 7feeaa3c2740 1 -- start start 2024-03-26T17:54:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.152+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b0e40 0x55ffae7ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.152+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.152+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b21d0 0x55ffae7b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.152+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feea01a6cb0 con 0x55ffae7b0e40 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.152+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feea01a87f0 con 0x55ffae7b21d0 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.152+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feea01a7380 con 0x55ffae7b69a0 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5570640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b0e40 0x55ffae7ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea4d6f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5d71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b21d0 0x55ffae7b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5570640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b0e40 0x55ffae7ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35084/0 (socket says 172.21.15.105:35084) 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5570640 1 -- 172.21.15.105:0/2746461429 learned_addr learned my addr 172.21.15.105:0/2746461429 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea4d6f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59468/0 (socket says 172.21.15.105:59468) 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5d71640 1 -- 172.21.15.105:0/2746461429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 msgr2=0x55ffae7ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5d71640 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5d71640 1 -- 172.21.15.105:0/2746461429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b0e40 msgr2=0x55ffae7ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5d71640 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b0e40 0x55ffae7ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5d71640 1 -- 172.21.15.105:0/2746461429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffae7a8d10 con 0x55ffae7b21d0 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea4d6f640 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:25.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5570640 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b0e40 0x55ffae7ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7feea5d71640 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b21d0 0x55ffae7b2650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fee98009fd0 tx=0x7fee9801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e65758b8c7a9cd3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7fee8affd640 1 -- 172.21.15.105:0/2746461429 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98003970 con 0x55ffae7b21d0 2024-03-26T17:54:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7fee8affd640 1 -- 172.21.15.105:0/2746461429 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee98004050 con 0x55ffae7b21d0 2024-03-26T17:54:25.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.153+0000 7fee8affd640 1 -- 172.21.15.105:0/2746461429 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98010690 con 0x55ffae7b21d0 2024-03-26T17:54:25.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 -- 172.21.15.105:0/2746461429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b21d0 msgr2=0x55ffae7b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b21d0 0x55ffae7b2650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fee98009fd0 tx=0x7fee9801c000 comp rx=0 tx=0).stop 2024-03-26T17:54:25.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 -- 172.21.15.105:0/2746461429 shutdown_connections 2024-03-26T17:54:25.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b21d0 0x55ffae7b2650 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b69a0 0x55ffae7ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/2746461429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b0e40 0x55ffae7ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 -- 172.21.15.105:0/2746461429 >> 172.21.15.105:0/2746461429 conn(0x55ffae7a7fa0 msgr2=0x55ffae7b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 -- 172.21.15.105:0/2746461429 shutdown_connections 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.154+0000 7feeaa3c2740 1 -- 172.21.15.105:0/2746461429 wait complete. 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 1 -- start start 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 1 Processor -- start 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 0x55ffae7c5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b21d0 0x55ffae7c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 0x55ffae7c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feea01a8120 con 0x55ffae7b21d0 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feea01a7a50 con 0x55ffae7b69a0 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feea01a4c90 con 0x55ffae7b0e40 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feeaa3c2740 10 client.0 _ll_register_callbacks cb 0x7fee900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feea4d6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b21d0 0x55ffae7c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feea5570640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 0x55ffae7c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feea4d6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b21d0 0x55ffae7c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35092/0 (socket says 172.21.15.105:35092) 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.155+0000 7feea4d6f640 1 -- 172.21.15.105:0/420692006 learned_addr learned my addr 172.21.15.105:0/420692006 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7feea5d71640 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 0x55ffae7c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7feea4d6f640 1 -- 172.21.15.105:0/420692006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 msgr2=0x55ffae7c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7feea4d6f640 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 0x55ffae7c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7feea4d6f640 1 -- 172.21.15.105:0/420692006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 msgr2=0x55ffae7c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7feea4d6f640 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 0x55ffae7c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7feea4d6f640 1 -- 172.21.15.105:0/420692006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffae7c6da0 con 0x55ffae7b21d0 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7feea5570640 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 0x55ffae7c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7feea4d6f640 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b21d0 0x55ffae7c5860 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fee90001eb0 tx=0x7fee90026510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c09f604f3c773ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.156+0000 7fee88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.157+0000 7fee88ff9640 1 -- 172.21.15.105:0/420692006 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee900052b0 con 0x55ffae7b21d0 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.157+0000 7fee88ff9640 1 -- 172.21.15.105:0/420692006 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fee90005450 con 0x55ffae7b21d0 2024-03-26T17:54:25.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.157+0000 7feeaa3c2740 1 -- 172.21.15.105:0/420692006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee98004050 con 0x55ffae7b21d0 2024-03-26T17:54:25.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.157+0000 7fee88ff9640 1 -- 172.21.15.105:0/420692006 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fee90005650 con 0x55ffae7b21d0 2024-03-26T17:54:25.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.157+0000 7feeaa3c2740 10 client.21312 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:25.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.157+0000 7feeaa3c2740 1 -- 172.21.15.105:0/420692006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ffae7c8140 con 0x55ffae7b21d0 2024-03-26T17:54:25.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.157+0000 7fee88ff9640 1 -- 172.21.15.105:0/420692006 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee98004050 con 0x55ffae7b21d0 2024-03-26T17:54:25.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.157+0000 7fee88ff9640 1 -- 172.21.15.105:0/420692006 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fee9002e030 con 0x55ffae7b21d0 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.158+0000 7feeaa3c2740 10 client.21312 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.158+0000 7feeaa3c2740 10 client.21312 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.158+0000 7feeaa3c2740 20 client.21312 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.158+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ffae831f70 0x55ffae834390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.158+0000 7feeaa3c2740 4 client.21312 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.158+0000 7feeaa3c2740 1 -- 172.21.15.105:0/420692006 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffae6966a0 con 0x55ffae831f70 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.158+0000 7feea5570640 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ffae831f70 0x55ffae834390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7fee88ff9640 10 client.21312 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feea5570640 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ffae831f70 0x55ffae834390 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=373bc6958de6a3ab server_cookie=1585c46dac2f6987 in_seq=0 out_seq=0 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7fee88ff9640 1 -- 172.21.15.105:0/420692006 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffae6966a0 con 0x55ffae831f70 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7fee88ff9640 10 client.21312 handle_command_reply: tid=0 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 client.21312 shutdown 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 -- 172.21.15.105:0/420692006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b21d0 msgr2=0x55ffae7c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b21d0 0x55ffae7c5860 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fee90001eb0 tx=0x7fee90026510 comp rx=0 tx=0).stop 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 -- 172.21.15.105:0/420692006 shutdown_connections 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ffae831f70 0x55ffae834390 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ffae7b69a0 0x55ffae7c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7fee88ff9640 0 client.21312 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ffae7b21d0 0x55ffae7c5860 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 --2- 172.21.15.105:0/420692006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ffae7b0e40 0x55ffae7c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.159+0000 7feeaa3c2740 1 -- 172.21.15.105:0/420692006 >> 172.21.15.105:0/420692006 conn(0x55ffae7a7fa0 msgr2=0x55ffae7ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.160+0000 7feeaa3c2740 1 -- 172.21.15.105:0/420692006 shutdown_connections 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.160+0000 7feeaa3c2740 1 -- 172.21.15.105:0/420692006 wait complete. 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.160+0000 7feeaa3c2740 20 client.21312 trim_cache size 0 max 16384 2024-03-26T17:54:25.152 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:25.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 -- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fee8c0a2f20 msgr2=0x7fee8c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fee8c0a2f20 0x7fee8c0a53e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fee98013a40 tx=0x7fee98009c60 comp rx=0 tx=0).stop 2024-03-26T17:54:25.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 -- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 msgr2=0x7feea0197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 0x7feea0197b90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fee9c000c00 tx=0x7fee9c0040c0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 -- 172.21.15.105:0/1302443761 shutdown_connections 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fee8c0a2f20 0x7fee8c0a53e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feea01075a0 0x7feea019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feea0104c70 0x7feea01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 --2- 172.21.15.105:0/1302443761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feea0102340 0x7feea0197b90 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 -- 172.21.15.105:0/1302443761 >> 172.21.15.105:0/1302443761 conn(0x7feea00fbd50 msgr2=0x7feea0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 -- 172.21.15.105:0/1302443761 shutdown_connections 2024-03-26T17:54:25.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.161+0000 7feea77d4640 1 -- 172.21.15.105:0/1302443761 wait complete. 2024-03-26T17:54:25.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:24 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2784792095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:54:25.163 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 -- 172.21.15.105:0/1935143072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d381001b0 msgr2=0x7f9d381005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 --2- 172.21.15.105:0/1935143072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d381001b0 0x7f9d381005d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009900 tx=0x7f9d3402bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 -- 172.21.15.105:0/1935143072 shutdown_connections 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 --2- 172.21.15.105:0/1935143072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d381014b0 0x7f9d38101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 --2- 172.21.15.105:0/1935143072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d38100b10 0x7f9d38100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 --2- 172.21.15.105:0/1935143072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d381001b0 0x7f9d381005d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 -- 172.21.15.105:0/1935143072 >> 172.21.15.105:0/1935143072 conn(0x7f9d380fbdf0 msgr2=0x7f9d380fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 -- 172.21.15.105:0/1935143072 shutdown_connections 2024-03-26T17:54:25.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.315+0000 7f9d40452640 1 -- 172.21.15.105:0/1935143072 wait complete. 2024-03-26T17:54:25.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d40452640 1 Processor -- start 2024-03-26T17:54:25.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d40452640 1 -- start start 2024-03-26T17:54:25.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d40452640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d381001b0 0x7f9d38199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d40452640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d38100b10 0x7f9d3819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d40452640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d381014b0 0x7f9d381a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d40452640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810f5e0 con 0x7f9d38100b10 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d40452640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810f460 con 0x7f9d381014b0 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d40452640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3810f760 con 0x7f9d381001b0 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.316+0000 7f9d3d9ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d38100b10 0x7f9d3819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3d9ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d38100b10 0x7f9d3819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35118/0 (socket says 172.21.15.105:35118) 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3d9ed640 1 -- 172.21.15.105:0/3985714969 learned_addr learned my addr 172.21.15.105:0/3985714969 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d381001b0 0x7f9d38199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d381014b0 0x7f9d381a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3d9ed640 1 -- 172.21.15.105:0/3985714969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d381001b0 msgr2=0x7f9d38199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d381001b0 0x7f9d38199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3d9ed640 1 -- 172.21.15.105:0/3985714969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d381014b0 msgr2=0x7f9d381a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d381014b0 0x7f9d381a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3d9ed640 1 -- 172.21.15.105:0/3985714969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d381a1960 con 0x7f9d38100b10 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d38100b10 0x7f9d3819a280 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800e9f0 tx=0x7f9d2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d2f7fe640 1 -- 172.21.15.105:0/3985714969 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800ccc0 con 0x7f9d38100b10 2024-03-26T17:54:25.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d2f7fe640 1 -- 172.21.15.105:0/3985714969 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d2800ce60 con 0x7f9d38100b10 2024-03-26T17:54:25.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d40452640 1 -- 172.21.15.105:0/3985714969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d381a1c50 con 0x7f9d38100b10 2024-03-26T17:54:25.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d2f7fe640 1 -- 172.21.15.105:0/3985714969 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28010430 con 0x7f9d38100b10 2024-03-26T17:54:25.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.317+0000 7f9d40452640 1 -- 172.21.15.105:0/3985714969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d381a21a0 con 0x7f9d38100b10 2024-03-26T17:54:25.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.320+0000 7f9d4303f740 1 Processor -- start 2024-03-26T17:54:25.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.320+0000 7f9d2f7fe640 1 -- 172.21.15.105:0/3985714969 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9d28005020 con 0x7f9d38100b10 2024-03-26T17:54:25.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d4303f740 1 -- start start 2024-03-26T17:54:25.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccbfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccc4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 0x558eaccc7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d4303f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558eacc24970 con 0x558eaccbfa30 2024-03-26T17:54:25.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d4303f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558eacbccd70 con 0x558eaccc0420 2024-03-26T17:54:25.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d4303f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558eacc20920 con 0x558eaccc5590 2024-03-26T17:54:25.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e1ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccc4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e1ee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccc4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56926/0 (socket says 172.21.15.105:56926) 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e1ee640 1 -- 172.21.15.105:0/3403751321 learned_addr learned my addr 172.21.15.105:0/3403751321 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 0x558eaccc7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccbfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d2f7fe640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9d240a2fa0 0x7f9d240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e1ee640 1 -- 172.21.15.105:0/3403751321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 msgr2=0x558eaccc7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 0x558eaccc7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e1ee640 1 -- 172.21.15.105:0/3403751321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 msgr2=0x558eaccbfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccbfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d3e1ee640 1 -- 172.21.15.105:0/3403751321 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eac9383f0 con 0x558eaccc0420 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.321+0000 7f9d2f7fe640 1 -- 172.21.15.105:0/3985714969 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9d280cf720 con 0x7f9d38100b10 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.322+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9d240a2fa0 0x7f9d240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.322+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccc4f20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009900 tx=0x7f9d34009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eca2abf9a515f35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.322+0000 7f9d2cbfe640 1 -- 172.21.15.105:0/3403751321 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34045020 con 0x558eaccc0420 2024-03-26T17:54:25.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.322+0000 7f9d2cbfe640 1 -- 172.21.15.105:0/3403751321 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d34004040 con 0x558eaccc0420 2024-03-26T17:54:25.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.322+0000 7f9d2cbfe640 1 -- 172.21.15.105:0/3403751321 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34005600 con 0x558eaccc0420 2024-03-26T17:54:25.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 -- 172.21.15.105:0/3403751321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 msgr2=0x558eaccc4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9d240a2fa0 0x7f9d240a5460 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9d300049d0 tx=0x7f9d30004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccc4f20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009900 tx=0x7f9d34009560 comp rx=0 tx=0).stop 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 -- 172.21.15.105:0/3403751321 shutdown_connections 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 0x558eaccc7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccc4f20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3403751321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccbfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 -- 172.21.15.105:0/3403751321 >> 172.21.15.105:0/3403751321 conn(0x558eaccbafa0 msgr2=0x558eaccbb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 -- 172.21.15.105:0/3403751321 shutdown_connections 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 -- 172.21.15.105:0/3403751321 wait complete. 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.323+0000 7f9d4303f740 1 -- start start 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d4303f740 1 Processor -- start 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccc9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc0420 0x558eaccca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc5590 0x558eaccd14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d4303f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558eaccc84e0 con 0x558eaccbfa30 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d4303f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558eacbe0310 con 0x558eaccc5590 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d4303f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558eaccc8660 con 0x558eaccc0420 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d4303f740 10 client.0 _ll_register_callbacks cb 0x558eacc1bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e9ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc5590 0x558eaccd14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e9ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc5590 0x558eaccd14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56928/0 (socket says 172.21.15.105:56928) 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e9ef640 1 -- 172.21.15.105:0/519465853 learned_addr learned my addr 172.21.15.105:0/519465853 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccc9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc0420 0x558eaccca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e9ef640 1 -- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc0420 msgr2=0x558eaccca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc0420 0x558eaccca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e9ef640 1 -- 172.21.15.105:0/519465853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 msgr2=0x558eaccc9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccc9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3e9ef640 1 -- 172.21.15.105:0/519465853 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eaca027d0 con 0x558eaccc5590 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.324+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc0420 0x558eaccca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:25.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc5590 0x558eaccd14a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000f910 tx=0x7f9d30009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408a21b2572d1c40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d21ffb640 1 -- 172.21.15.105:0/519465853 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30010700 con 0x558eaccc5590 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d21ffb640 1 -- 172.21.15.105:0/519465853 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9d30018050 con 0x558eaccc5590 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d4303f740 1 -- 172.21.15.105:0/519465853 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d34004040 con 0x558eaccc5590 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d21ffb640 1 -- 172.21.15.105:0/519465853 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9d300108a0 con 0x558eaccc5590 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d4303f740 10 client.31097 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d4303f740 1 -- 172.21.15.105:0/519465853 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558eaccd4880 con 0x558eaccc5590 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.325+0000 7f9d21ffb640 1 -- 172.21.15.105:0/519465853 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d34004040 con 0x558eaccc5590 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d21ffb640 1 -- 172.21.15.105:0/519465853 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9d3001a070 con 0x558eaccc5590 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d4303f740 10 client.31097 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d4303f740 10 client.31097 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d4303f740 20 client.31097 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d4303f740 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558eaccd5f00 0x558eaccd82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d4303f740 4 client.31097 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:25.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d4303f740 1 -- 172.21.15.105:0/519465853 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eacc31eb0 con 0x558eaccd5f00 2024-03-26T17:54:25.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558eaccd5f00 0x558eaccd82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d21ffb640 10 client.31097 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:54:25.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.326+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558eaccd5f00 0x558eaccd82f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0ce48ea8af3c56c server_cookie=9235011e62e74fd5 in_seq=0 out_seq=0 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.327+0000 7f9d21ffb640 1 -- 172.21.15.105:0/519465853 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558eacc31eb0 con 0x558eaccd5f00 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.327+0000 7f9d21ffb640 10 client.31097 handle_command_reply: tid=0 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.327+0000 7f9d4303f740 1 client.31097 shutdown 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 -- 172.21.15.105:0/519465853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc5590 msgr2=0x558eaccd14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc5590 0x558eaccd14a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000f910 tx=0x7f9d30009500 comp rx=0 tx=0).stop 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 -- 172.21.15.105:0/519465853 shutdown_connections 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558eaccd5f00 0x558eaccd82f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d21ffb640 0 client.31097 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc5590 0x558eaccd14a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc0420 0x558eaccca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 --2- 172.21.15.105:0/519465853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccc9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 -- 172.21.15.105:0/519465853 >> 172.21.15.105:0/519465853 conn(0x558eaccbafa0 msgr2=0x558eaccba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 -- 172.21.15.105:0/519465853 shutdown_connections 2024-03-26T17:54:25.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 1 -- 172.21.15.105:0/519465853 wait complete. 2024-03-26T17:54:25.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.328+0000 7f9d4303f740 20 client.31097 trim_cache size 0 max 16384 2024-03-26T17:54:25.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d4303f740 1 Processor -- start 2024-03-26T17:54:25.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d4303f740 1 -- start start 2024-03-26T17:54:25.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccbfa30 0x558eaccbe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccd1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccc5590 0x558eaccdcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d4303f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9d381aaf30 con 0x558eaccc5590 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d4303f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9d381aca70 con 0x558eaccc0420 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d4303f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d381ab600 con 0x558eaccbfa30 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d3e9ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccc5590 0x558eaccdcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d3e9ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccc5590 0x558eaccdcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35150/0 (socket says 172.21.15.105:35150) 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d3e9ef640 1 -- 172.21.15.105:0/3060024335 learned_addr learned my addr 172.21.15.105:0/3060024335 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.351+0000 7f9d3e1ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccbfa30 0x558eaccbe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/3060024335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccd1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d3e9ef640 1 -- 172.21.15.105:0/3060024335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccbfa30 msgr2=0x558eaccbe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/3060024335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccbfa30 0x558eaccbe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d3e9ef640 1 -- 172.21.15.105:0/3060024335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 msgr2=0x558eaccd1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/3060024335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccd1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d3e9ef640 1 -- 172.21.15.105:0/3060024335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eaccb6820 con 0x558eaccc5590 2024-03-26T17:54:25.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/3060024335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccc5590 0x558eaccdcbf0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9d30013010 tx=0x7f9d30005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f748c299b9faa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d23fff640 1 -- 172.21.15.105:0/3060024335 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30010a40 con 0x558eaccc5590 2024-03-26T17:54:25.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d23fff640 1 -- 172.21.15.105:0/3060024335 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d30037020 con 0x558eaccc5590 2024-03-26T17:54:25.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.352+0000 7f9d23fff640 1 -- 172.21.15.105:0/3060024335 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3001c570 con 0x558eaccc5590 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 -- 172.21.15.105:0/3060024335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccc5590 msgr2=0x558eaccdcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3060024335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccc5590 0x558eaccdcbf0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9d30013010 tx=0x7f9d30005c70 comp rx=0 tx=0).stop 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 -- 172.21.15.105:0/3060024335 shutdown_connections 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3060024335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccc5590 0x558eaccdcbf0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3060024335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eaccd1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3060024335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccbfa30 0x558eaccbe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 -- 172.21.15.105:0/3060024335 >> 172.21.15.105:0/3060024335 conn(0x558eaccbafa0 msgr2=0x558eaccbe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 -- 172.21.15.105:0/3060024335 shutdown_connections 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.353+0000 7f9d4303f740 1 -- 172.21.15.105:0/3060024335 wait complete. 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 1 -- start start 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 1 Processor -- start 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eacccef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 0x558eacccff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9d381ac3a0 con 0x558eaccbfa30 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9d381abcd0 con 0x558eaccc0420 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d381a8f10 con 0x558eaccc5590 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d4303f740 10 client.0 _ll_register_callbacks cb 0x7f9d3000f5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e1ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3d9ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eacccef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e1ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35156/0 (socket says 172.21.15.105:35156) 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e1ee640 1 -- 172.21.15.105:0/3280226980 learned_addr learned my addr 172.21.15.105:0/3280226980 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e9ef640 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 0x558eacccff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e1ee640 1 -- 172.21.15.105:0/3280226980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 msgr2=0x558eacccff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 0x558eacccff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e1ee640 1 -- 172.21.15.105:0/3280226980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 msgr2=0x558eacccef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eacccef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3e1ee640 1 -- 172.21.15.105:0/3280226980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eaccd0520 con 0x558eaccbfa30 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.354+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eacccef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d3e1ee640 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccce9e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009900 tx=0x7f9d3402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c007be7178d0edc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d21ffb640 1 -- 172.21.15.105:0/3280226980 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d34045040 con 0x558eaccbfa30 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d4303f740 1 -- 172.21.15.105:0/3280226980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d30037020 con 0x558eaccbfa30 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d21ffb640 1 -- 172.21.15.105:0/3280226980 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9d3402caf0 con 0x558eaccbfa30 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d21ffb640 1 -- 172.21.15.105:0/3280226980 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9d34004040 con 0x558eaccbfa30 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d4303f740 10 client.21330 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d4303f740 1 -- 172.21.15.105:0/3280226980 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558eacd3c230 con 0x558eaccbfa30 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d21ffb640 1 -- 172.21.15.105:0/3280226980 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d30037020 con 0x558eaccbfa30 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.355+0000 7f9d21ffb640 1 -- 172.21.15.105:0/3280226980 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9d34003ce0 con 0x558eaccbfa30 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d4303f740 10 client.21330 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d4303f740 10 client.21330 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:54:25.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d4303f740 20 client.21330 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558eaccd5f00 0x558eacd3e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d4303f740 4 client.21330 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d4303f740 1 -- 172.21.15.105:0/3280226980 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558eacc2c200 con 0x558eaccd5f00 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558eaccd5f00 0x558eacd3e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d3d9ed640 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558eaccd5f00 0x558eacd3e730 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b1084e0c6c7f6a4 server_cookie=9785be2d3cfc256 in_seq=0 out_seq=0 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d21ffb640 10 client.21330 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d21ffb640 1 -- 172.21.15.105:0/3280226980 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558eacc2c200 con 0x558eaccd5f00 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d21ffb640 10 client.21330 handle_command_reply: tid=0 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.356+0000 7f9d4303f740 1 client.21330 shutdown 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 -- 172.21.15.105:0/3280226980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 msgr2=0x558eaccce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccce9e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f9d34009900 tx=0x7f9d3402bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 -- 172.21.15.105:0/3280226980 shutdown_connections 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558eaccd5f00 0x558eacd3e730 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558eaccc5590 0x558eacccff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558eaccc0420 0x558eacccef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d21ffb640 0 client.21330 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 --2- 172.21.15.105:0/3280226980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558eaccbfa30 0x558eaccce9e0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 -- 172.21.15.105:0/3280226980 >> 172.21.15.105:0/3280226980 conn(0x558eaccbafa0 msgr2=0x558eaccbea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 -- 172.21.15.105:0/3280226980 shutdown_connections 2024-03-26T17:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 1 -- 172.21.15.105:0/3280226980 wait complete. 2024-03-26T17:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d4303f740 20 client.21330 trim_cache size 0 max 16384 2024-03-26T17:54:25.351 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d40452640 1 -- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9d240a2fa0 msgr2=0x7f9d240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d40452640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9d240a2fa0 0x7f9d240a5460 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9d300049d0 tx=0x7f9d30004a40 comp rx=0 tx=0).stop 2024-03-26T17:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d40452640 1 -- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d38100b10 msgr2=0x7f9d3819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.357+0000 7f9d40452640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d38100b10 0x7f9d3819a280 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800e9f0 tx=0x7f9d2800edf0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.358+0000 7f9d40452640 1 -- 172.21.15.105:0/3985714969 shutdown_connections 2024-03-26T17:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.358+0000 7f9d40452640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9d240a2fa0 0x7f9d240a5460 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.358+0000 7f9d40452640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9d381014b0 0x7f9d381a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.358+0000 7f9d40452640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d38100b10 0x7f9d3819a280 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.358+0000 7f9d40452640 1 --2- 172.21.15.105:0/3985714969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d381001b0 0x7f9d38199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.358+0000 7f9d40452640 1 -- 172.21.15.105:0/3985714969 >> 172.21.15.105:0/3985714969 conn(0x7f9d380fbdf0 msgr2=0x7f9d380fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.358+0000 7f9d40452640 1 -- 172.21.15.105:0/3985714969 shutdown_connections 2024-03-26T17:54:25.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.358+0000 7f9d40452640 1 -- 172.21.15.105:0/3985714969 wait complete. 2024-03-26T17:54:25.353 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:24 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2784792095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:54:25.359 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:54:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 -- 172.21.15.105:0/1578135826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c104d20 msgr2=0x7f987c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 --2- 172.21.15.105:0/1578135826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c104d20 0x7f987c107110 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f986c009900 tx=0x7f986c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 -- 172.21.15.105:0/1578135826 shutdown_connections 2024-03-26T17:54:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 --2- 172.21.15.105:0/1578135826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f987c107650 0x7f987c109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 --2- 172.21.15.105:0/1578135826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c104d20 0x7f987c107110 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 --2- 172.21.15.105:0/1578135826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f987c1023f0 0x7f987c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 -- 172.21.15.105:0/1578135826 >> 172.21.15.105:0/1578135826 conn(0x7f987c0fbd60 msgr2=0x7f987c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 -- 172.21.15.105:0/1578135826 shutdown_connections 2024-03-26T17:54:25.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.522+0000 7f9883427640 1 -- 172.21.15.105:0/1578135826 wait complete. 2024-03-26T17:54:25.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.523+0000 7f9883427640 1 Processor -- start 2024-03-26T17:54:25.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.523+0000 7f9883427640 1 -- start start 2024-03-26T17:54:25.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f9883427640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1023f0 0x7f987c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f9883427640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f987c104d20 0x7f987c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f98811c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1023f0 0x7f987c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f98811c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1023f0 0x7f987c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56974/0 (socket says 172.21.15.105:56974) 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f98811c3640 1 -- 172.21.15.105:0/11680015 learned_addr learned my addr 172.21.15.105:0/11680015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f9883427640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f987c107650 0x7f987c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10d420 con 0x7f987c107650 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10d2a0 con 0x7f987c1023f0 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f98809c2640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f987c104d20 0x7f987c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.524+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f987c10d5a0 con 0x7f987c104d20 2024-03-26T17:54:25.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98819c4640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f987c107650 0x7f987c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98811c3640 1 -- 172.21.15.105:0/11680015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f987c104d20 msgr2=0x7f987c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98811c3640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f987c104d20 0x7f987c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98811c3640 1 -- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f987c107650 msgr2=0x7f987c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98811c3640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f987c107650 0x7f987c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98811c3640 1 -- 172.21.15.105:0/11680015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c19f7a0 con 0x7f987c1023f0 2024-03-26T17:54:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98811c3640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1023f0 0x7f987c197b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9878009f20 tx=0x7f987800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98727fc640 1 -- 172.21.15.105:0/11680015 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878020070 con 0x7f987c1023f0 2024-03-26T17:54:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.525+0000 7f98727fc640 1 -- 172.21.15.105:0/11680015 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f987800cd80 con 0x7f987c1023f0 2024-03-26T17:54:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.526+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c100240 con 0x7f987c1023f0 2024-03-26T17:54:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.526+0000 7f98727fc640 1 -- 172.21.15.105:0/11680015 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987801bae0 con 0x7f987c1023f0 2024-03-26T17:54:25.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.526+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c1007b0 con 0x7f987c1023f0 2024-03-26T17:54:25.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.529+0000 7f988600e740 1 Processor -- start 2024-03-26T17:54:25.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.529+0000 7f988600e740 1 -- start start 2024-03-26T17:54:25.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f988600e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a82e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f98727fc640 1 -- 172.21.15.105:0/11680015 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9878023070 con 0x7f987c1023f0 2024-03-26T17:54:25.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f988600e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a87f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f988600e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a8aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f98819c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a82e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f98819c4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a82e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:56988/0 (socket says 172.21.15.105:56988) 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f98819c4640 1 -- 172.21.15.105:0/162045031 learned_addr learned my addr 172.21.15.105:0/162045031 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f98809c2640 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a8aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f988600e740 1 -- 172.21.15.105:0/162045031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5563c49d6970 con 0x5563c4a83430 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f988600e740 1 -- 172.21.15.105:0/162045031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5563c497ed70 con 0x5563c4a82a40 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.530+0000 7f988600e740 1 -- 172.21.15.105:0/162045031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5563c49d2920 con 0x5563c4a885a0 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.531+0000 7f98811c3640 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a87f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.531+0000 7f98809c2640 1 -- 172.21.15.105:0/162045031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 msgr2=0x5563c4a82e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.531+0000 7f98809c2640 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a82e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.531+0000 7f98809c2640 1 -- 172.21.15.105:0/162045031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 msgr2=0x5563c4a87f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.531+0000 7f98809c2640 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a87f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.531+0000 7f98809c2640 1 -- 172.21.15.105:0/162045031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c46ea3f0 con 0x5563c4a885a0 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.531+0000 7f98811c3640 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a87f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.531+0000 7f98727fc640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98680a3020 0x7f98680a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.532+0000 7f98809c2640 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a8aa30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f986c036040 tx=0x7f986c0095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f048fc28ce421e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.532+0000 7f98819c4640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98680a3020 0x7f98680a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.532+0000 7f98677fe640 1 -- 172.21.15.105:0/162045031 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c045020 con 0x5563c4a885a0 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.532+0000 7f98677fe640 1 -- 172.21.15.105:0/162045031 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f986c02c9a0 con 0x5563c4a885a0 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.532+0000 7f98677fe640 1 -- 172.21.15.105:0/162045031 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c0055f0 con 0x5563c4a885a0 2024-03-26T17:54:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.532+0000 7f98727fc640 1 -- 172.21.15.105:0/11680015 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f98780d35b0 con 0x7f987c1023f0 2024-03-26T17:54:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f98819c4640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98680a3020 0x7f98680a54e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x5563c4a83c90 tx=0x7f9874009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 -- 172.21.15.105:0/162045031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 msgr2=0x5563c4a8aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a8aa30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f986c036040 tx=0x7f986c0095e0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 -- 172.21.15.105:0/162045031 shutdown_connections 2024-03-26T17:54:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a8aa30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a87f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 --2- 172.21.15.105:0/162045031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a82e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 -- 172.21.15.105:0/162045031 >> 172.21.15.105:0/162045031 conn(0x5563c4a6cfa0 msgr2=0x5563c4a6d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 -- 172.21.15.105:0/162045031 shutdown_connections 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 -- 172.21.15.105:0/162045031 wait complete. 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 -- start start 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.533+0000 7f988600e740 1 Processor -- start 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f988600e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f988600e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f988600e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a7a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f988600e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5563c4a718b0 con 0x5563c4a83430 2024-03-26T17:54:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f988600e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5563c4992310 con 0x5563c4a82a40 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f988600e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5563c4a71a30 con 0x5563c4a885a0 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f988600e740 10 client.0 _ll_register_callbacks cb 0x5563c49cdc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98811c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98819c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a7a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98811c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a73180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57000/0 (socket says 172.21.15.105:57000) 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98811c3640 1 -- 172.21.15.105:0/602121428 learned_addr learned my addr 172.21.15.105:0/602121428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98809c2640 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98811c3640 1 -- 172.21.15.105:0/602121428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 msgr2=0x5563c4a7a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98811c3640 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a7a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98811c3640 1 -- 172.21.15.105:0/602121428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 msgr2=0x5563c4a736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98811c3640 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98811c3640 1 -- 172.21.15.105:0/602121428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c47b47d0 con 0x5563c4a82a40 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.534+0000 7f98819c4640 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a7a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:25.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f98811c3640 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a73180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f98780a8120 tx=0x7f987809f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d0928b7747d0129 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f9864ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f9864ff9640 1 -- 172.21.15.105:0/602121428 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98780a83c0 con 0x5563c4a82a40 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f9864ff9640 1 -- 172.21.15.105:0/602121428 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f98780d30d0 con 0x5563c4a82a40 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f988600e740 1 -- 172.21.15.105:0/602121428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f986c02c9a0 con 0x5563c4a82a40 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f9864ff9640 1 -- 172.21.15.105:0/602121428 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9878013da0 con 0x5563c4a82a40 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f988600e740 10 client.31115 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f988600e740 1 -- 172.21.15.105:0/602121428 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5563c4a7db30 con 0x5563c4a82a40 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f9864ff9640 1 -- 172.21.15.105:0/602121428 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f986c02c9a0 con 0x5563c4a82a40 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.535+0000 7f9864ff9640 1 -- 172.21.15.105:0/602121428 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f987809ba40 con 0x5563c4a82a40 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f988600e740 10 client.31115 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f988600e740 10 client.31115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f988600e740 20 client.31115 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f988600e740 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563c4a7f1d0 0x5563c4a81670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f988600e740 4 client.31115 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:25.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f988600e740 1 -- 172.21.15.105:0/602121428 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563c49e3eb0 con 0x5563c4a7f1d0 2024-03-26T17:54:25.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f98809c2640 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563c4a7f1d0 0x5563c4a81670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f98809c2640 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563c4a7f1d0 0x5563c4a81670 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa7a701a8ec5ca39 server_cookie=5fc12a4b867d8d7 in_seq=0 out_seq=0 2024-03-26T17:54:25.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.536+0000 7f9864ff9640 10 client.31115 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f9864ff9640 1 -- 172.21.15.105:0/602121428 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5563c49e3eb0 con 0x5563c4a7f1d0 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f9864ff9640 10 client.31115 handle_command_reply: tid=0 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 client.31115 shutdown 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 -- 172.21.15.105:0/602121428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 msgr2=0x5563c4a73180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a73180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f98780a8120 tx=0x7f987809f830 comp rx=0 tx=0).stop 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 -- 172.21.15.105:0/602121428 shutdown_connections 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563c4a7f1d0 0x5563c4a81670 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a885a0 0x5563c4a7a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a83430 0x5563c4a736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 --2- 172.21.15.105:0/602121428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a82a40 0x5563c4a73180 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f9864ff9640 0 client.31115 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 -- 172.21.15.105:0/602121428 >> 172.21.15.105:0/602121428 conn(0x5563c4a6cfa0 msgr2=0x5563c4a6c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 -- 172.21.15.105:0/602121428 shutdown_connections 2024-03-26T17:54:25.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 1 -- 172.21.15.105:0/602121428 wait complete. 2024-03-26T17:54:25.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.538+0000 7f988600e740 20 client.31115 trim_cache size 0 max 16384 2024-03-26T17:54:25.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.561+0000 7f988600e740 1 Processor -- start 2024-03-26T17:54:25.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.561+0000 7f988600e740 1 -- start start 2024-03-26T17:54:25.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.561+0000 7f988600e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a82a40 0x5563c4a81f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.561+0000 7f988600e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a83430 0x5563c4a82450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.561+0000 7f988600e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a885a0 0x5563c4a70200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.561+0000 7f988600e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f987c072580 con 0x5563c4a82a40 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98809c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a83430 0x5563c4a82450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98809c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a83430 0x5563c4a82450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59558/0 (socket says 172.21.15.105:59558) 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98809c2640 1 -- 172.21.15.105:0/152131122 learned_addr learned my addr 172.21.15.105:0/152131122 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98811c3640 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a82a40 0x5563c4a81f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f988600e740 1 -- 172.21.15.105:0/152131122 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f987c19ff50 con 0x5563c4a885a0 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f988600e740 1 -- 172.21.15.105:0/152131122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f987c072c50 con 0x5563c4a83430 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98819c4640 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a885a0 0x5563c4a70200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98811c3640 1 -- 172.21.15.105:0/152131122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a83430 msgr2=0x5563c4a82450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98811c3640 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a83430 0x5563c4a82450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98811c3640 1 -- 172.21.15.105:0/152131122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a885a0 msgr2=0x5563c4a70200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98811c3640 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a885a0 0x5563c4a70200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98811c3640 1 -- 172.21.15.105:0/152131122 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c4a83810 con 0x5563c4a82a40 2024-03-26T17:54:25.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 24758 7.12 MB/sec execute 68 sec latency 113.148 ms 2024-03-26T17:54:25.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.562+0000 7f98811c3640 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a82a40 0x5563c4a81f10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f987800c6a0 tx=0x7f987809e200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0ce8ea4ced6bd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.563+0000 7f9866ffd640 1 -- 172.21.15.105:0/152131122 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987809f650 con 0x5563c4a82a40 2024-03-26T17:54:25.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.563+0000 7f9866ffd640 1 -- 172.21.15.105:0/152131122 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f987809fcf0 con 0x5563c4a82a40 2024-03-26T17:54:25.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.563+0000 7f9866ffd640 1 -- 172.21.15.105:0/152131122 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987809ec00 con 0x5563c4a82a40 2024-03-26T17:54:25.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.563+0000 7f988600e740 1 -- 172.21.15.105:0/152131122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a82a40 msgr2=0x5563c4a81f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.563+0000 7f988600e740 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a82a40 0x5563c4a81f10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f987800c6a0 tx=0x7f987809e200 comp rx=0 tx=0).stop 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 -- 172.21.15.105:0/152131122 shutdown_connections 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a885a0 0x5563c4a70200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a83430 0x5563c4a82450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 --2- 172.21.15.105:0/152131122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a82a40 0x5563c4a81f10 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 -- 172.21.15.105:0/152131122 >> 172.21.15.105:0/152131122 conn(0x5563c4a6cfa0 msgr2=0x5563c4a84830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 -- 172.21.15.105:0/152131122 shutdown_connections 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 -- 172.21.15.105:0/152131122 wait complete. 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 -- start start 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.564+0000 7f988600e740 1 Processor -- start 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f988600e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a82a40 0x5563c4a8eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f988600e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a83430 0x5563c4a8f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f988600e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a885a0 0x5563c4a8f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f988600e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f987c19f970 con 0x5563c4a885a0 2024-03-26T17:54:25.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f988600e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f987c073320 con 0x5563c4a83430 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f988600e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f987c070560 con 0x5563c4a82a40 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f988600e740 10 client.0 _ll_register_callbacks cb 0x7f9878009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98811c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a82a40 0x5563c4a8eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98809c2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a83430 0x5563c4a8f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98811c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a82a40 0x5563c4a8eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59570/0 (socket says 172.21.15.105:59570) 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98809c2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a83430 0x5563c4a8f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57024/0 (socket says 172.21.15.105:57024) 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98811c3640 1 -- 172.21.15.105:0/3504334093 learned_addr learned my addr 172.21.15.105:0/3504334093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98819c4640 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a885a0 0x5563c4a8f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98809c2640 1 -- 172.21.15.105:0/3504334093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a82a40 msgr2=0x5563c4a8eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98809c2640 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a82a40 0x5563c4a8eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98809c2640 1 -- 172.21.15.105:0/3504334093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a885a0 msgr2=0x5563c4a8f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:25.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98809c2640 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a885a0 0x5563c4a8f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98809c2640 1 -- 172.21.15.105:0/3504334093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c4aec960 con 0x5563c4a83430 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98819c4640 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a885a0 0x5563c4a8f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.565+0000 7f98809c2640 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a83430 0x5563c4a8f430 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f986c009900 tx=0x7f986c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3303249f5d468a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f9864ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f9864ff9640 1 -- 172.21.15.105:0/3504334093 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c045040 con 0x5563c4a83430 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f9864ff9640 1 -- 172.21.15.105:0/3504334093 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f986c004240 con 0x5563c4a83430 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f988600e740 1 -- 172.21.15.105:0/3504334093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f987809fcf0 con 0x5563c4a83430 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f9864ff9640 1 -- 172.21.15.105:0/3504334093 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f986c004440 con 0x5563c4a83430 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f988600e740 10 client.31121 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f988600e740 1 -- 172.21.15.105:0/3504334093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5563c4aedc60 con 0x5563c4a83430 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f9864ff9640 1 -- 172.21.15.105:0/3504334093 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f987809fcf0 con 0x5563c4a83430 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.566+0000 7f9864ff9640 1 -- 172.21.15.105:0/3504334093 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f986c003920 con 0x5563c4a83430 2024-03-26T17:54:25.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.567+0000 7f988600e740 10 client.31121 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:25.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.567+0000 7f988600e740 10 client.31121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:25.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.567+0000 7f988600e740 20 client.31121 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:25.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.567+0000 7f988600e740 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563c4aee660 0x5563c4af0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:25.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.567+0000 7f988600e740 4 client.31121 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:25.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.567+0000 7f988600e740 1 -- 172.21.15.105:0/3504334093 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563c49de200 con 0x5563c4aee660 2024-03-26T17:54:25.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.568+0000 7f98811c3640 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563c4aee660 0x5563c4af0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:25.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.568+0000 7f9864ff9640 10 client.31121 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:25.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.575+0000 7f98811c3640 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563c4aee660 0x5563c4af0a80 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed871934cb20fad server_cookie=a7ab4e4b2097bbdb in_seq=0 out_seq=0 2024-03-26T17:54:25.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.577+0000 7f9864ff9640 1 -- 172.21.15.105:0/3504334093 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563c49de200 con 0x5563c4aee660 2024-03-26T17:54:25.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.577+0000 7f9864ff9640 10 client.31121 handle_command_reply: tid=0 2024-03-26T17:54:25.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.577+0000 7f988600e740 1 client.31121 shutdown 2024-03-26T17:54:25.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 -- 172.21.15.105:0/3504334093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a83430 msgr2=0x5563c4a8f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a83430 0x5563c4a8f430 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f986c009900 tx=0x7f986c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 -- 172.21.15.105:0/3504334093 shutdown_connections 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5563c4aee660 0x5563c4af0a80 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563c4a885a0 0x5563c4a8f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f9864ff9640 0 client.31121 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5563c4a83430 0x5563c4a8f430 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 --2- 172.21.15.105:0/3504334093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5563c4a82a40 0x5563c4a8eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 -- 172.21.15.105:0/3504334093 >> 172.21.15.105:0/3504334093 conn(0x5563c4a6cfa0 msgr2=0x5563c4a84830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 -- 172.21.15.105:0/3504334093 shutdown_connections 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 1 -- 172.21.15.105:0/3504334093 wait complete. 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.578+0000 7f988600e740 20 client.31121 trim_cache size 0 max 16384 2024-03-26T17:54:25.568 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:25.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98680a3020 msgr2=0x7f98680a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98680a3020 0x7f98680a54e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x5563c4a83c90 tx=0x7f9874009500 comp rx=0 tx=0).stop 2024-03-26T17:54:25.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1023f0 msgr2=0x7f987c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:25.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1023f0 0x7f987c197b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9878009f20 tx=0x7f987800ee80 comp rx=0 tx=0).stop 2024-03-26T17:54:25.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 shutdown_connections 2024-03-26T17:54:25.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f98680a3020 0x7f98680a54e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f987c107650 0x7f987c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f987c104d20 0x7f987c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 --2- 172.21.15.105:0/11680015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f987c1023f0 0x7f987c197b80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:25.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 >> 172.21.15.105:0/11680015 conn(0x7f987c0fbd60 msgr2=0x7f987c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:25.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 shutdown_connections 2024-03-26T17:54:25.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:25.579+0000 7f9883427640 1 -- 172.21.15.105:0/11680015 wait complete. 2024-03-26T17:54:26.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:25 smithi105 ceph-mon[32280]: pgmap v831: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-03-26T17:54:26.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:25 smithi135 ceph-mon[35636]: pgmap v831: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-03-26T17:54:26.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:25 smithi144 ceph-mon[36095]: pgmap v831: 97 pgs: 97 active+clean; 254 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.3 MiB/s wr, 226 op/s 2024-03-26T17:54:26.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 24925 7.17 MB/sec execute 69 sec latency 130.101 ms 2024-03-26T17:54:26.584 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:54:26.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:54:26.584 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:54:26.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 -- 172.21.15.105:0/2480517896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201019b0 msgr2=0x7fa920101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 --2- 172.21.15.105:0/2480517896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201019b0 0x7fa920101e30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fa914009900 tx=0x7fa91402bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:26.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 -- 172.21.15.105:0/2480517896 shutdown_connections 2024-03-26T17:54:26.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 --2- 172.21.15.105:0/2480517896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa920102370 0x7fa920108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 --2- 172.21.15.105:0/2480517896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201019b0 0x7fa920101e30 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 --2- 172.21.15.105:0/2480517896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9201007c0 0x7fa920100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 -- 172.21.15.105:0/2480517896 >> 172.21.15.105:0/2480517896 conn(0x7fa9200fbd50 msgr2=0x7fa9200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 -- 172.21.15.105:0/2480517896 shutdown_connections 2024-03-26T17:54:26.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.750+0000 7fa924d7e640 1 -- 172.21.15.105:0/2480517896 wait complete. 2024-03-26T17:54:26.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa924d7e640 1 Processor -- start 2024-03-26T17:54:26.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa924d7e640 1 -- start start 2024-03-26T17:54:26.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa924d7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201007c0 0x7fa920196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa924d7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9201019b0 0x7fa920196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa924d7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920102370 0x7fa92019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa924d7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c320 con 0x7fa9201007c0 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa924d7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c1a0 con 0x7fa9201019b0 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa924d7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c4a0 con 0x7fa920102370 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201007c0 0x7fa920196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201007c0 0x7fa920196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35244/0 (socket says 172.21.15.105:35244) 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa91e59c640 1 -- 172.21.15.105:0/3090554134 learned_addr learned my addr 172.21.15.105:0/3090554134 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9201019b0 0x7fa920196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.751+0000 7fa91ed9d640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920102370 0x7fa92019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa91e59c640 1 -- 172.21.15.105:0/3090554134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920102370 msgr2=0x7fa92019e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa91e59c640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920102370 0x7fa92019e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa91e59c640 1 -- 172.21.15.105:0/3090554134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9201019b0 msgr2=0x7fa920196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa91e59c640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9201019b0 0x7fa920196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa91e59c640 1 -- 172.21.15.105:0/3090554134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92019e670 con 0x7fa9201007c0 2024-03-26T17:54:26.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa91e59c640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201007c0 0x7fa920196a50 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fa91000e9c0 tx=0x7fa91000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa90f7fe640 1 -- 172.21.15.105:0/3090554134 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91000cde0 con 0x7fa9201007c0 2024-03-26T17:54:26.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa90f7fe640 1 -- 172.21.15.105:0/3090554134 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9100108e0 con 0x7fa9201007c0 2024-03-26T17:54:26.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa924d7e640 1 -- 172.21.15.105:0/3090554134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92019e960 con 0x7fa9201007c0 2024-03-26T17:54:26.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa90f7fe640 1 -- 172.21.15.105:0/3090554134 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910018990 con 0x7fa9201007c0 2024-03-26T17:54:26.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.752+0000 7fa924d7e640 1 -- 172.21.15.105:0/3090554134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92019ee30 con 0x7fa9201007c0 2024-03-26T17:54:26.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.755+0000 7fa92796c740 1 Processor -- start 2024-03-26T17:54:26.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa92796c740 1 -- start start 2024-03-26T17:54:26.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa90f7fe640 1 -- 172.21.15.105:0/3090554134 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa91001f020 con 0x7fa9201007c0 2024-03-26T17:54:26.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa92796c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc59d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa92796c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc5a350 0x55e05dc5ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa92796c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5f0b0 0x55e05dc61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa92796c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e05dbba630 con 0x55e05dc5f0b0 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa92796c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e05db62e10 con 0x55e05dc5a350 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa92796c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e05dbb65e0 con 0x55e05dc59960 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.756+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc5a350 0x55e05dc5ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5f0b0 0x55e05dc61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc5a350 0x55e05dc5ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57046/0 (socket says 172.21.15.105:57046) 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91e59c640 1 -- 172.21.15.105:0/2281402326 learned_addr learned my addr 172.21.15.105:0/2281402326 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91ed9d640 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc59d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91e59c640 1 -- 172.21.15.105:0/2281402326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 msgr2=0x55e05dc59d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91e59c640 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc59d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91e59c640 1 -- 172.21.15.105:0/2281402326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5f0b0 msgr2=0x55e05dc61570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa90f7fe640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9080a2fa0 0x7fa9080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91e59c640 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5f0b0 0x55e05dc61570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91e59c640 1 -- 172.21.15.105:0/2281402326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05db79740 con 0x55e05dc5a350 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5f0b0 0x55e05dc61570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91ed9d640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9080a2fa0 0x7fa9080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.757+0000 7fa91e59c640 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc5a350 0x55e05dc5ea20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9100add40 tx=0x7fa910098040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1747c8fe76c74d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.758+0000 7fa90f7fe640 1 -- 172.21.15.105:0/3090554134 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa9100cf650 con 0x7fa9201007c0 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.758+0000 7fa90cff9640 1 -- 172.21.15.105:0/2281402326 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91009bab0 con 0x55e05dc5a350 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.758+0000 7fa90cff9640 1 -- 172.21.15.105:0/2281402326 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9100cbe50 con 0x55e05dc5a350 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.758+0000 7fa90cff9640 1 -- 172.21.15.105:0/2281402326 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91009b640 con 0x55e05dc5a350 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 -- 172.21.15.105:0/2281402326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc5a350 msgr2=0x55e05dc5ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc5a350 0x55e05dc5ea20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9100add40 tx=0x7fa910098040 comp rx=0 tx=0).stop 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa91ed9d640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9080a2fa0 0x7fa9080a5460 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa918005fd0 tx=0x7fa918009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 -- 172.21.15.105:0/2281402326 shutdown_connections 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5f0b0 0x55e05dc61570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc5a350 0x55e05dc5ea20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 --2- 172.21.15.105:0/2281402326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc59d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 -- 172.21.15.105:0/2281402326 >> 172.21.15.105:0/2281402326 conn(0x55e05dc51d40 msgr2=0x55e05dc50290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 -- 172.21.15.105:0/2281402326 shutdown_connections 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 -- 172.21.15.105:0/2281402326 wait complete. 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 -- start start 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.759+0000 7fa92796c740 1 Processor -- start 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa92796c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc63ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa92796c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc64010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa92796c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc5f0b0 0x55e05dc6b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa92796c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e05dc62130 con 0x55e05dc5a350 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa92796c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e05db761a0 con 0x55e05dc59960 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa92796c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e05dc622b0 con 0x55e05dc5f0b0 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc64010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc64010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35250/0 (socket says 172.21.15.105:35250) 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91dd9b640 1 -- 172.21.15.105:0/653889395 learned_addr learned my addr 172.21.15.105:0/653889395 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91e59c640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc63ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91ed9d640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc5f0b0 0x55e05dc6b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa92796c740 10 client.0 _ll_register_callbacks cb 0x55e05d8cdc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91dd9b640 1 -- 172.21.15.105:0/653889395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc5f0b0 msgr2=0x55e05dc6b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc5f0b0 0x55e05dc6b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91dd9b640 1 -- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 msgr2=0x55e05dc63ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc63ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91dd9b640 1 -- 172.21.15.105:0/653889395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05d9967e0 con 0x55e05dc5a350 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91e59c640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc63ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.760+0000 7fa91ed9d640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc5f0b0 0x55e05dc6b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc64010 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa914005ab0 tx=0x7fa914004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b2e1d2054620c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa905ffb640 1 -- 172.21.15.105:0/653889395 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914005030 con 0x55e05dc5a350 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa92796c740 1 -- 172.21.15.105:0/653889395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9100cbe50 con 0x55e05dc5a350 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa905ffb640 1 -- 172.21.15.105:0/653889395 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa9140051d0 con 0x55e05dc5a350 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa905ffb640 1 -- 172.21.15.105:0/653889395 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa9140053d0 con 0x55e05dc5a350 2024-03-26T17:54:26.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa92796c740 10 client.21360 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa92796c740 1 -- 172.21.15.105:0/653889395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e05d9f6f70 con 0x55e05dc5a350 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.761+0000 7fa905ffb640 1 -- 172.21.15.105:0/653889395 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9100cbe50 con 0x55e05dc5a350 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa905ffb640 1 -- 172.21.15.105:0/653889395 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa914003680 con 0x55e05dc5a350 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa92796c740 10 client.21360 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa92796c740 10 client.21360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa92796c740 20 client.21360 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa92796c740 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e05dc6f840 0x55e05dc71cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa92796c740 4 client.21360 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa92796c740 1 -- 172.21.15.105:0/653889395 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e05dbc7b70 con 0x55e05dc6f840 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa91e59c640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e05dc6f840 0x55e05dc71cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.762+0000 7fa905ffb640 10 client.21360 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:26.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.764+0000 7fa91e59c640 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e05dc6f840 0x55e05dc71cc0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76a579f2f97600f server_cookie=91fc8902ed1e8528 in_seq=0 out_seq=0 2024-03-26T17:54:26.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa905ffb640 1 -- 172.21.15.105:0/653889395 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e05dbc7b70 con 0x55e05dc6f840 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa905ffb640 10 client.21360 handle_command_reply: tid=0 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 client.21360 shutdown 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 -- 172.21.15.105:0/653889395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 msgr2=0x55e05dc64010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc64010 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa914005ab0 tx=0x7fa914004270 comp rx=0 tx=0).stop 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 -- 172.21.15.105:0/653889395 shutdown_connections 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e05dc6f840 0x55e05dc71cc0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc5f0b0 0x55e05dc6b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa905ffb640 0 client.21360 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc64010 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 --2- 172.21.15.105:0/653889395 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc63ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 -- 172.21.15.105:0/653889395 >> 172.21.15.105:0/653889395 conn(0x55e05dc51d40 msgr2=0x55e05dc4f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 -- 172.21.15.105:0/653889395 shutdown_connections 2024-03-26T17:54:26.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.765+0000 7fa92796c740 1 -- 172.21.15.105:0/653889395 wait complete. 2024-03-26T17:54:26.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.766+0000 7fa92796c740 20 client.21360 trim_cache size 0 max 16384 2024-03-26T17:54:26.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.796+0000 7fa92796c740 1 Processor -- start 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.796+0000 7fa92796c740 1 -- start start 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.796+0000 7fa92796c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc51d40 0x55e05dc4f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa92796c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc6b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa92796c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc4f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa92796c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201a6bf0 con 0x55e05dc5a350 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa92796c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201a8730 con 0x55e05dc51d40 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa92796c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201a72c0 con 0x55e05dc59960 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc4f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc4f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35262/0 (socket says 172.21.15.105:35262) 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91ed9d640 1 -- 172.21.15.105:0/756407796 learned_addr learned my addr 172.21.15.105:0/756407796 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc6b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91e59c640 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc51d40 0x55e05dc4f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91dd9b640 1 -- 172.21.15.105:0/756407796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc51d40 msgr2=0x55e05dc4f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc51d40 0x55e05dc4f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91dd9b640 1 -- 172.21.15.105:0/756407796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 msgr2=0x55e05dc4f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc4f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91dd9b640 1 -- 172.21.15.105:0/756407796 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05d88e170 con 0x55e05dc59960 2024-03-26T17:54:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc6b010 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa91402c170 tx=0x7fa91402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55a56afa67ba471 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.797+0000 7fa907fff640 1 -- 172.21.15.105:0/756407796 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9140035f0 con 0x55e05dc59960 2024-03-26T17:54:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.798+0000 7fa907fff640 1 -- 172.21.15.105:0/756407796 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa914003790 con 0x55e05dc59960 2024-03-26T17:54:26.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.798+0000 7fa907fff640 1 -- 172.21.15.105:0/756407796 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914034440 con 0x55e05dc59960 2024-03-26T17:54:26.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.798+0000 7fa92796c740 1 -- 172.21.15.105:0/756407796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 msgr2=0x55e05dc6b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.798+0000 7fa92796c740 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc6b010 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa91402c170 tx=0x7fa91402bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:26.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 -- 172.21.15.105:0/756407796 shutdown_connections 2024-03-26T17:54:26.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc4f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc59960 0x55e05dc6b010 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 --2- 172.21.15.105:0/756407796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc51d40 0x55e05dc4f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 -- 172.21.15.105:0/756407796 >> 172.21.15.105:0/756407796 conn(0x55e05dc4b780 msgr2=0x55e05dc5b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 -- 172.21.15.105:0/756407796 shutdown_connections 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 -- 172.21.15.105:0/756407796 wait complete. 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 -- start start 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.799+0000 7fa92796c740 1 Processor -- start 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc51d40 0x55e05dc687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc68d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc69d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201a8060 con 0x55e05dc5a350 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201a7990 con 0x55e05dc59960 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201a4bd0 con 0x55e05dc51d40 2024-03-26T17:54:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 10 client.0 _ll_register_callbacks cb 0x7fa914009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc69d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc51d40 0x55e05dc687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc69d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35264/0 (socket says 172.21.15.105:35264) 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91ed9d640 1 -- 172.21.15.105:0/3499084853 learned_addr learned my addr 172.21.15.105:0/3499084853 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91e59c640 1 -- 172.21.15.105:0/3499084853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 msgr2=0x55e05dc68d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91e59c640 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc68d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91e59c640 1 -- 172.21.15.105:0/3499084853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 msgr2=0x55e05dc69d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91e59c640 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc69d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91e59c640 1 -- 172.21.15.105:0/3499084853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05dc6a320 con 0x55e05dc51d40 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc68d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91ed9d640 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc69d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa91e59c640 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc51d40 0x55e05dc687e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa9100d51e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54bc59db606ea977 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa905ffb640 1 -- 172.21.15.105:0/3499084853 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91009b820 con 0x55e05dc51d40 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa905ffb640 1 -- 172.21.15.105:0/3499084853 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa910010a90 con 0x55e05dc51d40 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 1 -- 172.21.15.105:0/3499084853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa914003790 con 0x55e05dc51d40 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa905ffb640 1 -- 172.21.15.105:0/3499084853 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa91009b640 con 0x55e05dc51d40 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 10 client.31099 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 1 -- 172.21.15.105:0/3499084853 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e05dcd5d20 con 0x55e05dc51d40 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa905ffb640 1 -- 172.21.15.105:0/3499084853 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9100b9070 con 0x55e05dc51d40 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa905ffb640 1 -- 172.21.15.105:0/3499084853 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa910097a30 con 0x55e05dc51d40 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 10 client.31099 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.801+0000 7fa92796c740 10 client.31099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.802+0000 7fa92796c740 20 client.31099 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.802+0000 7fa92796c740 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e05dc5f0b0 0x55e05dcd82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.802+0000 7fa92796c740 4 client.31099 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:54:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.802+0000 7fa92796c740 1 -- 172.21.15.105:0/3499084853 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fa914003790 con 0x55e05dc5f0b0 2024-03-26T17:54:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.802+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e05dc5f0b0 0x55e05dcd82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.802+0000 7fa905ffb640 10 client.31099 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:26.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.802+0000 7fa91dd9b640 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e05dc5f0b0 0x55e05dcd82c0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=735d099e74d2a1e5 server_cookie=fd14e76f5a7cca77 in_seq=0 out_seq=0 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa905ffb640 1 -- 172.21.15.105:0/3499084853 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fa914003790 con 0x55e05dc5f0b0 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa905ffb640 10 client.31099 handle_command_reply: tid=0 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 client.31099 shutdown 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 -- 172.21.15.105:0/3499084853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc51d40 msgr2=0x55e05dc687e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc51d40 0x55e05dc687e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa9100d51e0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 -- 172.21.15.105:0/3499084853 shutdown_connections 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e05dc5f0b0 0x55e05dcd82c0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e05dc5a350 0x55e05dc69d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e05dc59960 0x55e05dc68d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 --2- 172.21.15.105:0/3499084853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e05dc51d40 0x55e05dc687e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa905ffb640 0 client.31099 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 -- 172.21.15.105:0/3499084853 >> 172.21.15.105:0/3499084853 conn(0x55e05dc4b780 msgr2=0x55e05dc58760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 -- 172.21.15.105:0/3499084853 shutdown_connections 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.805+0000 7fa92796c740 1 -- 172.21.15.105:0/3499084853 wait complete. 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa92796c740 20 client.31099 trim_cache size 0 max 16384 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "4fbf0e62-7fd8-4017-bb4f-63588cb5bcf9", 2024-03-26T17:54:26.797 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:54:26.798 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 -- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9080a2fa0 msgr2=0x7fa9080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9080a2fa0 0x7fa9080a5460 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa918005fd0 tx=0x7fa918009500 comp rx=0 tx=0).stop 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 -- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201007c0 msgr2=0x7fa920196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201007c0 0x7fa920196a50 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fa91000e9c0 tx=0x7fa91000ee80 comp rx=0 tx=0).stop 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 -- 172.21.15.105:0/3090554134 shutdown_connections 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9080a2fa0 0x7fa9080a5460 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa920102370 0x7fa92019e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9201019b0 0x7fa920196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 --2- 172.21.15.105:0/3090554134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9201007c0 0x7fa920196a50 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 -- 172.21.15.105:0/3090554134 >> 172.21.15.105:0/3090554134 conn(0x7fa9200fbd50 msgr2=0x7fa920104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 -- 172.21.15.105:0/3090554134 shutdown_connections 2024-03-26T17:54:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.806+0000 7fa924d7e640 1 -- 172.21.15.105:0/3090554134 wait complete. 2024-03-26T17:54:26.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:54:26.809 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:54:26.886 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:26 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:54:26.886 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:26.887 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:26 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:54:26.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 -- 172.21.15.105:0/2440837848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf41007c0 msgr2=0x7f9bf4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/2440837848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf41007c0 0x7f9bf4100be0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009860 tx=0x7f9bf002bb70 comp rx=0 tx=0).stop 2024-03-26T17:54:26.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 -- 172.21.15.105:0/2440837848 shutdown_connections 2024-03-26T17:54:26.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/2440837848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf4102370 0x7f9bf4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/2440837848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf41019b0 0x7f9bf4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/2440837848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf41007c0 0x7f9bf4100be0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 -- 172.21.15.105:0/2440837848 >> 172.21.15.105:0/2440837848 conn(0x7f9bf40fbd50 msgr2=0x7f9bf40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 -- 172.21.15.105:0/2440837848 shutdown_connections 2024-03-26T17:54:26.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.978+0000 7f9bfba3d640 1 -- 172.21.15.105:0/2440837848 wait complete. 2024-03-26T17:54:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.979+0000 7f9bfba3d640 1 Processor -- start 2024-03-26T17:54:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.979+0000 7f9bfba3d640 1 -- start start 2024-03-26T17:54:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.979+0000 7f9bfba3d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf41007c0 0x7f9bf4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.979+0000 7f9bfba3d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf41019b0 0x7f9bf419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.979+0000 7f9bfba3d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf4102370 0x7f9bf419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.979+0000 7f9bfba3d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf410c2c0 con 0x7f9bf4102370 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.979+0000 7f9bfba3d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf410c140 con 0x7f9bf41007c0 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.979+0000 7f9bfba3d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf410c440 con 0x7f9bf41019b0 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf8fd8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf41019b0 0x7f9bf419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf9fda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf4102370 0x7f9bf419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf8fd8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf41019b0 0x7f9bf419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59640/0 (socket says 172.21.15.105:59640) 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf8fd8640 1 -- 172.21.15.105:0/16789712 learned_addr learned my addr 172.21.15.105:0/16789712 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf41007c0 0x7f9bf4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf9fda640 1 -- 172.21.15.105:0/16789712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf41019b0 msgr2=0x7f9bf419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf9fda640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf41019b0 0x7f9bf419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf9fda640 1 -- 172.21.15.105:0/16789712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf41007c0 msgr2=0x7f9bf4069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf9fda640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf41007c0 0x7f9bf4069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf9fda640 1 -- 172.21.15.105:0/16789712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf419f7c0 con 0x7f9bf4102370 2024-03-26T17:54:26.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bf9fda640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf4102370 0x7f9bf419eeb0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00efc0 tx=0x7f9bec00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bea7fc640 1 -- 172.21.15.105:0/16789712 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec003970 con 0x7f9bf4102370 2024-03-26T17:54:26.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bea7fc640 1 -- 172.21.15.105:0/16789712 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bec004020 con 0x7f9bf4102370 2024-03-26T17:54:26.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bfba3d640 1 -- 172.21.15.105:0/16789712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf419fa50 con 0x7f9bf4102370 2024-03-26T17:54:26.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.980+0000 7f9bfba3d640 1 -- 172.21.15.105:0/16789712 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf41968d0 con 0x7f9bf4102370 2024-03-26T17:54:26.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.981+0000 7f9bea7fc640 1 -- 172.21.15.105:0/16789712 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00b940 con 0x7f9bf4102370 2024-03-26T17:54:26.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.983+0000 7f9bea7fc640 1 -- 172.21.15.105:0/16789712 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9bec01d020 con 0x7f9bf4102370 2024-03-26T17:54:26.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bfe622740 1 Processor -- start 2024-03-26T17:54:26.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bfe622740 1 -- start start 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bea7fc640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9be00a2fa0 0x7f9be00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9e2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2af0 0x55781e9e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e7c60 0x55781e9ea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bfe622740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55781e93e980 con 0x55781e9e7c60 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bfe622740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55781e8e6be0 con 0x55781e9e2af0 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.984+0000 7f9bfe622740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55781e93a930 con 0x55781e9e2100 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bea7fc640 1 -- 172.21.15.105:0/16789712 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9bec0cf190 con 0x7f9bf4102370 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9e2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59646/0 (socket says 172.21.15.105:59646) 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 -- 172.21.15.105:0/648142250 learned_addr learned my addr 172.21.15.105:0/648142250 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf9fda640 1 --2- 172.21.15.105:0/648142250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e7c60 0x55781e9ea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9be00a2fa0 0x7f9be00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 -- 172.21.15.105:0/648142250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2af0 msgr2=0x55781e9e75f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:54:26.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/648142250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2af0 0x55781e9e75f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 -- 172.21.15.105:0/648142250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e7c60 msgr2=0x55781e9ea0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/648142250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e7c60 0x55781e9ea0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 -- 172.21.15.105:0/648142250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55781e71c7d0 con 0x55781e9e2100 2024-03-26T17:54:26.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/648142250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9e2520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009860 tx=0x7f9bf0005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95e8576fb9e9fbb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9be00a2fa0 0x7f9be00a5460 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55781e9e2ef0 tx=0x7f9be4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.985+0000 7f9bdf7fe640 1 -- 172.21.15.105:0/648142250 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0039040 con 0x55781e9e2100 2024-03-26T17:54:26.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bdf7fe640 1 -- 172.21.15.105:0/648142250 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bf0004030 con 0x55781e9e2100 2024-03-26T17:54:26.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bdf7fe640 1 -- 172.21.15.105:0/648142250 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf002caa0 con 0x55781e9e2100 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bfe622740 1 -- 172.21.15.105:0/648142250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 msgr2=0x55781e9e2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bfe622740 1 --2- 172.21.15.105:0/648142250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9e2520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009860 tx=0x7f9bf0005b20 comp rx=0 tx=0).stop 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bfe622740 1 -- 172.21.15.105:0/648142250 shutdown_connections 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bfe622740 1 --2- 172.21.15.105:0/648142250 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e7c60 0x55781e9ea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bfe622740 1 --2- 172.21.15.105:0/648142250 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2af0 0x55781e9e75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bfe622740 1 --2- 172.21.15.105:0/648142250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9e2520 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bfe622740 1 -- 172.21.15.105:0/648142250 >> 172.21.15.105:0/648142250 conn(0x55781e9dd670 msgr2=0x55781e9dda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.986+0000 7f9bfe622740 1 -- 172.21.15.105:0/648142250 shutdown_connections 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.987+0000 7f9bfe622740 1 -- 172.21.15.105:0/648142250 wait complete. 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.987+0000 7f9bfe622740 1 -- start start 2024-03-26T17:54:26.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.987+0000 7f9bfe622740 1 Processor -- start 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2100 0x55781e9ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2af0 0x55781e9ec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 0x55781e9f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bfe622740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55781e9eac50 con 0x55781e9e2100 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf97d9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2100 0x55781e9ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bfe622740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55781e8fa160 con 0x55781e9e7c60 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bfe622740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55781e9eadd0 con 0x55781e9e2af0 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf97d9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2100 0x55781e9ec350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35306/0 (socket says 172.21.15.105:35306) 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf97d9640 1 -- 172.21.15.105:0/130973453 learned_addr learned my addr 172.21.15.105:0/130973453 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf8fd8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2af0 0x55781e9ec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf9fda640 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 0x55781e9f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bfe622740 10 client.0 _ll_register_callbacks cb 0x55781e936720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf97d9640 1 -- 172.21.15.105:0/130973453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2af0 msgr2=0x55781e9ec890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2af0 0x55781e9ec890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf97d9640 1 -- 172.21.15.105:0/130973453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 msgr2=0x55781e9f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 0x55781e9f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf97d9640 1 -- 172.21.15.105:0/130973453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781e6523f0 con 0x55781e9e2100 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.988+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2af0 0x55781e9ec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.989+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2100 0x55781e9ec350 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00057c0 tx=0x7f9bf0004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7848774fda363f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.989+0000 7f9bdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.989+0000 7f9bdcff9640 1 -- 172.21.15.105:0/130973453 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0039040 con 0x55781e9e2100 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.989+0000 7f9bdcff9640 1 -- 172.21.15.105:0/130973453 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9bf002c7e0 con 0x55781e9e2100 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.989+0000 7f9bfe622740 1 -- 172.21.15.105:0/130973453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf0004030 con 0x55781e9e2100 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.989+0000 7f9bdcff9640 1 -- 172.21.15.105:0/130973453 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9bf0034640 con 0x55781e9e2100 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.989+0000 7f9bfe622740 10 client.21378 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.989+0000 7f9bfe622740 1 -- 172.21.15.105:0/130973453 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55781e6120f0 con 0x55781e9e2100 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bdcff9640 1 -- 172.21.15.105:0/130973453 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bf0004030 con 0x55781e9e2100 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bdcff9640 1 -- 172.21.15.105:0/130973453 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9bf00047c0 con 0x55781e9e2100 2024-03-26T17:54:26.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bfe622740 10 client.21378 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bfe622740 10 client.21378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bfe622740 20 client.21378 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bfe622740 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55781e9f8320 0x55781e9fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bfe622740 4 client.21378 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bfe622740 1 -- 172.21.15.105:0/130973453 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55781e94bec0 con 0x55781e9f8320 2024-03-26T17:54:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.990+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55781e9f8320 0x55781e9fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.991+0000 7f9bdcff9640 10 client.21378 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:26.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.991+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55781e9f8320 0x55781e9fa7c0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4823a75398ef693b server_cookie=e618a5631fc535c6 in_seq=0 out_seq=0 2024-03-26T17:54:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bdcff9640 1 -- 172.21.15.105:0/130973453 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55781e94bec0 con 0x55781e9f8320 2024-03-26T17:54:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bdcff9640 10 client.21378 handle_command_reply: tid=0 2024-03-26T17:54:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 client.21378 shutdown 2024-03-26T17:54:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 -- 172.21.15.105:0/130973453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2100 msgr2=0x55781e9ec350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:26.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2100 0x55781e9ec350 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00057c0 tx=0x7f9bf0004570 comp rx=0 tx=0).stop 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 -- 172.21.15.105:0/130973453 shutdown_connections 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55781e9f8320 0x55781e9fa7c0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 0x55781e9f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2af0 0x55781e9ec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 --2- 172.21.15.105:0/130973453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2100 0x55781e9ec350 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 -- 172.21.15.105:0/130973453 >> 172.21.15.105:0/130973453 conn(0x55781e9dd670 msgr2=0x55781e9d4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bdcff9640 0 client.21378 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 -- 172.21.15.105:0/130973453 shutdown_connections 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 1 -- 172.21.15.105:0/130973453 wait complete. 2024-03-26T17:54:26.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:26.993+0000 7f9bfe622740 20 client.21378 trim_cache size 0 max 16384 2024-03-26T17:54:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bfe622740 1 Processor -- start 2024-03-26T17:54:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bfe622740 1 -- start start 2024-03-26T17:54:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 0x55781e9d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bfe622740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd800e530 con 0x55781e9e2af0 2024-03-26T17:54:27.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bfe622740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd8010070 con 0x55781e9e7c60 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bfe622740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd800ec00 con 0x55781e9e2100 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf8fd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf97d9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf97d9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9d37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59670/0 (socket says 172.21.15.105:59670) 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf8fd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35320/0 (socket says 172.21.15.105:35320) 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf97d9640 1 -- 172.21.15.105:0/3477697538 learned_addr learned my addr 172.21.15.105:0/3477697538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf9fda640 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 0x55781e9d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf97d9640 1 -- 172.21.15.105:0/3477697538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 msgr2=0x55781e9d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 0x55781e9d3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf97d9640 1 -- 172.21.15.105:0/3477697538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 msgr2=0x55781e9f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf97d9640 1 -- 172.21.15.105:0/3477697538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55781e9e6f20 con 0x55781e9e2100 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.018+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:27.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.019+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9d37f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0000c00 tx=0x7f9bf0005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b73682b9dac1589 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.019+0000 7f9bdeffd640 1 -- 172.21.15.105:0/3477697538 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0034640 con 0x55781e9e2100 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.019+0000 7f9bdeffd640 1 -- 172.21.15.105:0/3477697538 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bf004b070 con 0x55781e9e2100 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.019+0000 7f9bdeffd640 1 -- 172.21.15.105:0/3477697538 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0005060 con 0x55781e9e2100 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 -- 172.21.15.105:0/3477697538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 msgr2=0x55781e9d37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9d37f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0000c00 tx=0x7f9bf0005970 comp rx=0 tx=0).stop 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 -- 172.21.15.105:0/3477697538 shutdown_connections 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e7c60 0x55781e9d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 --2- 172.21.15.105:0/3477697538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e2100 0x55781e9d37f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 -- 172.21.15.105:0/3477697538 >> 172.21.15.105:0/3477697538 conn(0x55781e9dd670 msgr2=0x55781e9e72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 -- 172.21.15.105:0/3477697538 shutdown_connections 2024-03-26T17:54:27.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 -- 172.21.15.105:0/3477697538 wait complete. 2024-03-26T17:54:27.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.020+0000 7f9bfe622740 1 -- start start 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bfe622740 1 Processor -- start 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2100 0x55781e9f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bfe622740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e7c60 0x55781e9f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bf97d9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2100 0x55781e9f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bf97d9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2100 0x55781e9f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57128/0 (socket says 172.21.15.105:57128) 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bf97d9640 1 -- 172.21.15.105:0/828295761 learned_addr learned my addr 172.21.15.105:0/828295761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd800f9a0 con 0x55781e9e2af0 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd800f2d0 con 0x55781e9e2100 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd800c510 con 0x55781e9e7c60 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.021+0000 7f9bf9fda640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e7c60 0x55781e9f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bfe622740 10 client.0 _ll_register_callbacks cb 0x7f9bec0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:27.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf97d9640 1 -- 172.21.15.105:0/828295761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e7c60 msgr2=0x55781e9f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e7c60 0x55781e9f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf97d9640 1 -- 172.21.15.105:0/828295761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 msgr2=0x55781e9f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf97d9640 1 -- 172.21.15.105:0/828295761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781e9f3b40 con 0x55781e9e2100 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf9fda640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e7c60 0x55781e9f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bf97d9640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2100 0x55781e9f3510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9bf002c0b0 tx=0x7f9bf002cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c97b8fc4f84cbd1b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.022+0000 7f9bdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.023+0000 7f9bdcff9640 1 -- 172.21.15.105:0/828295761 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0034640 con 0x55781e9e2100 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.023+0000 7f9bdcff9640 1 -- 172.21.15.105:0/828295761 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9bf0005010 con 0x55781e9e2100 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.023+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf004b070 con 0x55781e9e2100 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bdcff9640 1 -- 172.21.15.105:0/828295761 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9bf0005210 con 0x55781e9e2100 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bfe622740 10 client.31154 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:27.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55781ea5d640 con 0x55781e9e2100 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bdcff9640 1 -- 172.21.15.105:0/828295761 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bf00053b0 con 0x55781e9e2100 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bdcff9640 1 -- 172.21.15.105:0/828295761 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9bf0050b90 con 0x55781e9e2100 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bfe622740 10 client.31154 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bfe622740 10 client.31154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bfe622740 20 client.31154 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.024+0000 7f9bfe622740 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55781ea5de30 0x55781ea60250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.025+0000 7f9bfe622740 4 client.31154 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.025+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9bf004b070 con 0x55781ea5de30 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.025+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55781ea5de30 0x55781ea60250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.025+0000 7f9bf8fd8640 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55781ea5de30 0x55781ea60250 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26335fa3550845 server_cookie=b59695928c13fb4d in_seq=0 out_seq=0 2024-03-26T17:54:27.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.025+0000 7f9bdcff9640 10 client.31154 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.028+0000 7f9bdcff9640 1 -- 172.21.15.105:0/828295761 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x7f9bf004b070 con 0x55781ea5de30 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.028+0000 7f9bdcff9640 10 client.31154 handle_command_reply: tid=0 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.028+0000 7f9bfe622740 1 client.31154 shutdown 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2100 msgr2=0x55781e9f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2100 0x55781e9f3510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9bf002c0b0 tx=0x7f9bf002cba0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 shutdown_connections 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55781ea5de30 0x55781ea60250 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55781e9e7c60 0x55781e9f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55781e9e2af0 0x55781e9f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 --2- 172.21.15.105:0/828295761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55781e9e2100 0x55781e9f3510 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bdcff9640 0 client.31154 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 >> 172.21.15.105:0/828295761 conn(0x55781e9dd670 msgr2=0x55781e9e08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 shutdown_connections 2024-03-26T17:54:27.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 1 -- 172.21.15.105:0/828295761 wait complete. 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.029+0000 7f9bfe622740 20 client.31154 trim_cache size 0 max 16384 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (240 inodes in the stack)", 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: "4fbf0e62-7fd8-4017-bb4f-63588cb5bcf9": { 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "4fbf0e62-7fd8-4017-bb4f-63588cb5bcf9", 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:54:27.020 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:54:27.021 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:54:27.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 -- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9be00a2fa0 msgr2=0x7f9be00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:27.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9be00a2fa0 0x7f9be00a5460 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55781e9e2ef0 tx=0x7f9be4005f90 comp rx=0 tx=0).stop 2024-03-26T17:54:27.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 -- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf4102370 msgr2=0x7f9bf419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:27.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf4102370 0x7f9bf419eeb0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00efc0 tx=0x7f9bec00c800 comp rx=0 tx=0).stop 2024-03-26T17:54:27.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 -- 172.21.15.105:0/16789712 shutdown_connections 2024-03-26T17:54:27.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9be00a2fa0 0x7f9be00a5460 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9bf4102370 0x7f9bf419eeb0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bf41019b0 0x7f9bf419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 --2- 172.21.15.105:0/16789712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bf41007c0 0x7f9bf4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:27.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 -- 172.21.15.105:0/16789712 >> 172.21.15.105:0/16789712 conn(0x7f9bf40fbd50 msgr2=0x7f9bf4104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:27.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 -- 172.21.15.105:0/16789712 shutdown_connections 2024-03-26T17:54:27.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:27.030+0000 7f9bfba3d640 1 -- 172.21.15.105:0/16789712 wait complete. 2024-03-26T17:54:27.034 INFO:tasks.cephfs.filesystem:scrub status for tag:4fbf0e62-7fd8-4017-bb4f-63588cb5bcf9 - {'path': '/', 'tag': '4fbf0e62-7fd8-4017-bb4f-63588cb5bcf9', 'options': 'recursive,force'} 2024-03-26T17:54:27.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:26 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:54:27.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:27.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:26 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:54:27.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:26 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:54:27.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:27.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:26 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:54:27.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 25052 7.09 MB/sec execute 70 sec latency 219.596 ms 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: pgmap v832: 97 pgs: 97 active+clean; 255 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:27 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:54:28.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: pgmap v832: 97 pgs: 97 active+clean; 255 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-03-26T17:54:28.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:54:28.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:54:28.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:54:28.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:27 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:54:28.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: pgmap v832: 97 pgs: 97 active+clean; 255 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 802 KiB/s rd, 3.1 MiB/s wr, 263 op/s 2024-03-26T17:54:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:54:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:54:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:54:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.345 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:27 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:54:28.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 25192 7.00 MB/sec execute 71 sec latency 103.190 ms 2024-03-26T17:54:29.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 25351 6.94 MB/sec execute 72 sec latency 109.096 ms 2024-03-26T17:54:30.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:29 smithi135 ceph-mon[35636]: pgmap v833: 97 pgs: 97 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 3.1 MiB/s wr, 276 op/s 2024-03-26T17:54:30.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:29 smithi144 ceph-mon[36095]: pgmap v833: 97 pgs: 97 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 3.1 MiB/s wr, 276 op/s 2024-03-26T17:54:30.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:29 smithi105 ceph-mon[32280]: pgmap v833: 97 pgs: 97 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 3.1 MiB/s wr, 276 op/s 2024-03-26T17:54:30.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 25520 6.93 MB/sec execute 73 sec latency 104.836 ms 2024-03-26T17:54:31.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 25663 6.95 MB/sec execute 74 sec latency 113.761 ms 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 -- 172.21.15.105:0/1390656626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb441023a0 msgr2=0x7feb44104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 --2- 172.21.15.105:0/1390656626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb441023a0 0x7feb44104790 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7feb38009900 tx=0x7feb3802bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 -- 172.21.15.105:0/1390656626 shutdown_connections 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 --2- 172.21.15.105:0/1390656626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb44107600 0x7feb44109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 --2- 172.21.15.105:0/1390656626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb44104cd0 0x7feb441070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 --2- 172.21.15.105:0/1390656626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb441023a0 0x7feb44104790 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 -- 172.21.15.105:0/1390656626 >> 172.21.15.105:0/1390656626 conn(0x7feb440fbd90 msgr2=0x7feb440fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 -- 172.21.15.105:0/1390656626 shutdown_connections 2024-03-26T17:54:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.032+0000 7feb4af10640 1 -- 172.21.15.105:0/1390656626 wait complete. 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb4af10640 1 Processor -- start 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb4af10640 1 -- start start 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb4af10640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb441023a0 0x7feb44199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb4af10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb44104cd0 0x7feb4419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb4af10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb44107600 0x7feb441a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb4af10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb4410d480 con 0x7feb44104cd0 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb4af10640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb4410d300 con 0x7feb441023a0 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb4af10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb4410d600 con 0x7feb44107600 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb43fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb44104cd0 0x7feb4419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb48cac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb441023a0 0x7feb44199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb43fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb44104cd0 0x7feb4419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35360/0 (socket says 172.21.15.105:35360) 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb43fff640 1 -- 172.21.15.105:0/2519774023 learned_addr learned my addr 172.21.15.105:0/2519774023 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.033+0000 7feb494ad640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb44107600 0x7feb441a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb43fff640 1 -- 172.21.15.105:0/2519774023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb44107600 msgr2=0x7feb441a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb43fff640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb44107600 0x7feb441a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb43fff640 1 -- 172.21.15.105:0/2519774023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb441023a0 msgr2=0x7feb44199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb43fff640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb441023a0 0x7feb44199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb43fff640 1 -- 172.21.15.105:0/2519774023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb441a1900 con 0x7feb44104cd0 2024-03-26T17:54:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb48cac640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb441023a0 0x7feb44199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb43fff640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb44104cd0 0x7feb4419a230 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7feb3000e9f0 tx=0x7feb3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb41ffb640 1 -- 172.21.15.105:0/2519774023 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3000ccc0 con 0x7feb44104cd0 2024-03-26T17:54:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb41ffb640 1 -- 172.21.15.105:0/2519774023 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb300054d0 con 0x7feb44104cd0 2024-03-26T17:54:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb41ffb640 1 -- 172.21.15.105:0/2519774023 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30010430 con 0x7feb44104cd0 2024-03-26T17:54:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.034+0000 7feb4af10640 1 -- 172.21.15.105:0/2519774023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb441a1bf0 con 0x7feb44104cd0 2024-03-26T17:54:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.035+0000 7feb4af10640 1 -- 172.21.15.105:0/2519774023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb441a2090 con 0x7feb44104cd0 2024-03-26T17:54:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.037+0000 7feb41ffb640 1 -- 172.21.15.105:0/2519774023 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7feb30005020 con 0x7feb44104cd0 2024-03-26T17:54:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.038+0000 7feb41ffb640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb340a3130 0x7feb340a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.039+0000 7feb41ffb640 1 -- 172.21.15.105:0/2519774023 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7feb300cf750 con 0x7feb44104cd0 2024-03-26T17:54:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.039+0000 7feb48cac640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb340a3130 0x7feb340a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.039+0000 7feb4dbd8740 1 Processor -- start 2024-03-26T17:54:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.039+0000 7feb48cac640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb340a3130 0x7feb340a55f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feb4419b1c0 tx=0x7feb38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.039+0000 7feb4dbd8740 1 -- start start 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.039+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d15d0 0x555bea2d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d1fc0 0x555bea2d6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d7130 0x555bea2d95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d1fc0 0x555bea2d6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb48cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d15d0 0x555bea2d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d1fc0 0x555bea2d6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57166/0 (socket says 172.21.15.105:57166) 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 -- 172.21.15.105:0/2856055920 learned_addr learned my addr 172.21.15.105:0/2856055920 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555bea2360b0 con 0x555bea2d15d0 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb494ad640 1 --2- 172.21.15.105:0/2856055920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d7130 0x555bea2d95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb4dbd8740 1 -- 172.21.15.105:0/2856055920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555bea1de960 con 0x555bea2d1fc0 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb4dbd8740 1 -- 172.21.15.105:0/2856055920 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555bea232060 con 0x555bea2d7130 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 -- 172.21.15.105:0/2856055920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d7130 msgr2=0x555bea2d95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 --2- 172.21.15.105:0/2856055920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d7130 0x555bea2d95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 -- 172.21.15.105:0/2856055920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d15d0 msgr2=0x555bea2d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 --2- 172.21.15.105:0/2856055920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d15d0 0x555bea2d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 -- 172.21.15.105:0/2856055920 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bea1f0c60 con 0x555bea2d1fc0 2024-03-26T17:54:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb43fff640 1 --2- 172.21.15.105:0/2856055920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d1fc0 0x555bea2d6ac0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feb3009bdd0 tx=0x7feb3009a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a68ed0493b041d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.040+0000 7feb2effd640 1 -- 172.21.15.105:0/2856055920 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300b6030 con 0x555bea2d1fc0 2024-03-26T17:54:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.041+0000 7feb2effd640 1 -- 172.21.15.105:0/2856055920 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb30005310 con 0x555bea2d1fc0 2024-03-26T17:54:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.041+0000 7feb2effd640 1 -- 172.21.15.105:0/2856055920 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300b4020 con 0x555bea2d1fc0 2024-03-26T17:54:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.041+0000 7feb4dbd8740 1 -- 172.21.15.105:0/2856055920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d1fc0 msgr2=0x555bea2d6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.041+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/2856055920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d1fc0 0x555bea2d6ac0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feb3009bdd0 tx=0x7feb3009a040 comp rx=0 tx=0).stop 2024-03-26T17:54:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 -- 172.21.15.105:0/2856055920 shutdown_connections 2024-03-26T17:54:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/2856055920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d7130 0x555bea2d95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/2856055920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d1fc0 0x555bea2d6ac0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/2856055920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d15d0 0x555bea2d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 -- 172.21.15.105:0/2856055920 >> 172.21.15.105:0/2856055920 conn(0x555bea2ccb40 msgr2=0x555bea2ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 -- 172.21.15.105:0/2856055920 shutdown_connections 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 -- 172.21.15.105:0/2856055920 wait complete. 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 -- start start 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.042+0000 7feb4dbd8740 1 Processor -- start 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d15d0 0x555bea2db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d1fc0 0x555bea2db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555bea2da0e0 con 0x555bea2d7130 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555bea1f1db0 con 0x555bea2d15d0 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555bea2da260 con 0x555bea2d1fc0 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb494ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb43fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d1fc0 0x555bea2db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb494ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35368/0 (socket says 172.21.15.105:35368) 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb43fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d1fc0 0x555bea2db8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59720/0 (socket says 172.21.15.105:59720) 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb43fff640 1 -- 172.21.15.105:0/702170248 learned_addr learned my addr 172.21.15.105:0/702170248 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb48cac640 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d15d0 0x555bea2db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb4dbd8740 10 client.0 _ll_register_callbacks cb 0x555bea1f6110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb494ad640 1 -- 172.21.15.105:0/702170248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d1fc0 msgr2=0x555bea2db8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb494ad640 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d1fc0 0x555bea2db8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb494ad640 1 -- 172.21.15.105:0/702170248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d15d0 msgr2=0x555bea2db390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb494ad640 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d15d0 0x555bea2db390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb494ad640 1 -- 172.21.15.105:0/702170248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be9f4a210 con 0x555bea2d7130 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb43fff640 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d1fc0 0x555bea2db8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb494ad640 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e2910 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7feb3c005fd0 tx=0x7feb3c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f116d2b65eea27 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.043+0000 7feb23fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.044+0000 7feb23fff640 1 -- 172.21.15.105:0/702170248 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c009e30 con 0x555bea2d7130 2024-03-26T17:54:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.044+0000 7feb4dbd8740 1 -- 172.21.15.105:0/702170248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bea2435f0 con 0x555bea2d7130 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.044+0000 7feb23fff640 1 -- 172.21.15.105:0/702170248 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7feb3c004b20 con 0x555bea2d7130 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.044+0000 7feb23fff640 1 -- 172.21.15.105:0/702170248 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feb3c004d20 con 0x555bea2d7130 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.044+0000 7feb4dbd8740 10 client.21396 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.044+0000 7feb4dbd8740 1 -- 172.21.15.105:0/702170248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555bea016620 con 0x555bea2d7130 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.044+0000 7feb23fff640 1 -- 172.21.15.105:0/702170248 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x555bea2435f0 con 0x555bea2d7130 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb23fff640 1 -- 172.21.15.105:0/702170248 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7feb3c019070 con 0x555bea2d7130 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb4dbd8740 10 client.21396 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb4dbd8740 10 client.21396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb4dbd8740 20 client.21396 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555bea2e7020 0x555bea2e9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb4dbd8740 4 client.21396 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb4dbd8740 1 -- 172.21.15.105:0/702170248 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bea23d940 con 0x555bea2e7020 2024-03-26T17:54:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb48cac640 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555bea2e7020 0x555bea2e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb48cac640 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555bea2e7020 0x555bea2e9470 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca28d012d248fcb3 server_cookie=80f0ed400f0ea44e in_seq=0 out_seq=0 2024-03-26T17:54:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.045+0000 7feb23fff640 10 client.21396 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.048+0000 7feb23fff640 1 -- 172.21.15.105:0/702170248 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555bea23d940 con 0x555bea2e7020 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.048+0000 7feb23fff640 10 client.21396 handle_command_reply: tid=0 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.048+0000 7feb4dbd8740 1 client.21396 shutdown 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 -- 172.21.15.105:0/702170248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 msgr2=0x555bea2e2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e2910 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7feb3c005fd0 tx=0x7feb3c00f790 comp rx=0 tx=0).stop 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 -- 172.21.15.105:0/702170248 shutdown_connections 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555bea2e7020 0x555bea2e9470 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e2910 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d1fc0 0x555bea2db8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/702170248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d15d0 0x555bea2db390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 -- 172.21.15.105:0/702170248 >> 172.21.15.105:0/702170248 conn(0x555bea2ccb40 msgr2=0x555bea2cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb23fff640 0 client.21396 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 -- 172.21.15.105:0/702170248 shutdown_connections 2024-03-26T17:54:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.049+0000 7feb4dbd8740 1 -- 172.21.15.105:0/702170248 wait complete. 2024-03-26T17:54:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.050+0000 7feb4dbd8740 20 client.21396 trim_cache size 0 max 16384 2024-03-26T17:54:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.073+0000 7feb4dbd8740 1 Processor -- start 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.073+0000 7feb4dbd8740 1 -- start start 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.073+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.073+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d1fc0 0x555bea2e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.073+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d7130 0x555bea2db8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb441aaf60 con 0x555bea2d1fc0 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb441acaa0 con 0x555bea2d7130 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb441ab630 con 0x555bea2d15d0 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d1fc0 0x555bea2e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb48cac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d1fc0 0x555bea2e28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35382/0 (socket says 172.21.15.105:35382) 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 -- 172.21.15.105:0/4170787883 learned_addr learned my addr 172.21.15.105:0/4170787883 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb494ad640 1 --2- 172.21.15.105:0/4170787883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d7130 0x555bea2db8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 -- 172.21.15.105:0/4170787883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 msgr2=0x555bea2db390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 --2- 172.21.15.105:0/4170787883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2db390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 -- 172.21.15.105:0/4170787883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d7130 msgr2=0x555bea2db8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 --2- 172.21.15.105:0/4170787883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d7130 0x555bea2db8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 -- 172.21.15.105:0/4170787883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bea2cd8b0 con 0x555bea2d1fc0 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb43fff640 1 --2- 172.21.15.105:0/4170787883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d1fc0 0x555bea2e28d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7feb30098730 tx=0x7feb30099e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61af30a062879ab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.074+0000 7feb2e7fc640 1 -- 172.21.15.105:0/4170787883 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300a9070 con 0x555bea2d1fc0 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.075+0000 7feb2e7fc640 1 -- 172.21.15.105:0/4170787883 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb30005680 con 0x555bea2d1fc0 2024-03-26T17:54:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.075+0000 7feb2e7fc640 1 -- 172.21.15.105:0/4170787883 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb300b6070 con 0x555bea2d1fc0 2024-03-26T17:54:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.075+0000 7feb4dbd8740 1 -- 172.21.15.105:0/4170787883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d1fc0 msgr2=0x555bea2e28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.075+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/4170787883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d1fc0 0x555bea2e28d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7feb30098730 tx=0x7feb30099e50 comp rx=0 tx=0).stop 2024-03-26T17:54:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 -- 172.21.15.105:0/4170787883 shutdown_connections 2024-03-26T17:54:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/4170787883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2d7130 0x555bea2db8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/4170787883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d1fc0 0x555bea2e28d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/4170787883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2db390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 -- 172.21.15.105:0/4170787883 >> 172.21.15.105:0/4170787883 conn(0x555bea2ccb40 msgr2=0x555bea2cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 -- 172.21.15.105:0/4170787883 shutdown_connections 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 -- 172.21.15.105:0/4170787883 wait complete. 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 -- start start 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.076+0000 7feb4dbd8740 1 Processor -- start 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb4dbd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2e10d0 0x555bea2e1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb441ac3d0 con 0x555bea2d7130 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb441abd00 con 0x555bea2e10d0 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb4dbd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb441a8f40 con 0x555bea2d15d0 2024-03-26T17:54:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb4dbd8740 10 client.0 _ll_register_callbacks cb 0x7feb3c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb48cac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb48cac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2dfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59750/0 (socket says 172.21.15.105:59750) 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb48cac640 1 -- 172.21.15.105:0/3977393755 learned_addr learned my addr 172.21.15.105:0/3977393755 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb494ad640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2e10d0 0x555bea2e1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb43fff640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb48cac640 1 -- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2e10d0 msgr2=0x555bea2e1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb48cac640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2e10d0 0x555bea2e1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb48cac640 1 -- 172.21.15.105:0/3977393755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 msgr2=0x555bea2e00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb48cac640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb48cac640 1 -- 172.21.15.105:0/3977393755 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bea2e1a90 con 0x555bea2d15d0 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb494ad640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2e10d0 0x555bea2e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.077+0000 7feb43fff640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb48cac640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2dfb80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feb3802c170 tx=0x7feb3803e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b84c6946600c8e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:32.069 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb23fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb23fff640 1 -- 172.21.15.105:0/3977393755 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38005490 con 0x555bea2d15d0 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb23fff640 1 -- 172.21.15.105:0/3977393755 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7feb380541a0 con 0x555bea2d15d0 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb4dbd8740 1 -- 172.21.15.105:0/3977393755 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb30005680 con 0x555bea2d15d0 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb23fff640 1 -- 172.21.15.105:0/3977393755 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feb38005630 con 0x555bea2d15d0 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb4dbd8740 10 client.31135 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb4dbd8740 1 -- 172.21.15.105:0/3977393755 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555bea355bc0 con 0x555bea2d15d0 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.078+0000 7feb23fff640 1 -- 172.21.15.105:0/3977393755 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb30005680 con 0x555bea2d15d0 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb23fff640 1 -- 172.21.15.105:0/3977393755 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7feb3804a040 con 0x555bea2d15d0 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb4dbd8740 10 client.31135 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb4dbd8740 10 client.31135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb4dbd8740 20 client.31135 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555bea3564f0 0x555bea358910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb4dbd8740 4 client.31135 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:54:32.070 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb4dbd8740 1 -- 172.21.15.105:0/3977393755 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bea1bb770 con 0x555bea3564f0 2024-03-26T17:54:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb43fff640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555bea3564f0 0x555bea358910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb43fff640 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555bea3564f0 0x555bea358910 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7aedc34134896d server_cookie=7850b0b911c38422 in_seq=0 out_seq=0 2024-03-26T17:54:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.079+0000 7feb23fff640 10 client.31135 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:32.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:31 smithi135 ceph-mon[35636]: pgmap v834: 97 pgs: 97 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.7 MiB/s wr, 285 op/s 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb23fff640 1 -- 172.21.15.105:0/3977393755 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bea1bb770 con 0x555bea3564f0 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb23fff640 10 client.31135 handle_command_reply: tid=0 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 client.31135 shutdown 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 -- 172.21.15.105:0/3977393755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 msgr2=0x555bea2dfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2dfb80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feb3802c170 tx=0x7feb3803e000 comp rx=0 tx=0).stop 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 -- 172.21.15.105:0/3977393755 shutdown_connections 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555bea3564f0 0x555bea358910 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555bea2e10d0 0x555bea2e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555bea2d7130 0x555bea2e00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 --2- 172.21.15.105:0/3977393755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555bea2d15d0 0x555bea2dfb80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb23fff640 0 client.31135 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.262+0000 7feb4dbd8740 1 -- 172.21.15.105:0/3977393755 >> 172.21.15.105:0/3977393755 conn(0x555bea2ccb40 msgr2=0x555bea2db350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4dbd8740 1 -- 172.21.15.105:0/3977393755 shutdown_connections 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4dbd8740 1 -- 172.21.15.105:0/3977393755 wait complete. 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4dbd8740 20 client.31135 trim_cache size 0 max 16384 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:54:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:54:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 -- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb340a3130 msgr2=0x7feb340a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb340a3130 0x7feb340a55f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feb4419b1c0 tx=0x7feb38005fd0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 -- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb44104cd0 msgr2=0x7feb4419a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb44104cd0 0x7feb4419a230 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7feb3000e9f0 tx=0x7feb3000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 -- 172.21.15.105:0/2519774023 shutdown_connections 2024-03-26T17:54:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feb340a3130 0x7feb340a55f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb44107600 0x7feb441a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb44104cd0 0x7feb4419a230 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 --2- 172.21.15.105:0/2519774023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb441023a0 0x7feb44199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 -- 172.21.15.105:0/2519774023 >> 172.21.15.105:0/2519774023 conn(0x7feb440fbd90 msgr2=0x7feb44101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 -- 172.21.15.105:0/2519774023 shutdown_connections 2024-03-26T17:54:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:32.263+0000 7feb4af10640 1 -- 172.21.15.105:0/2519774023 wait complete. 2024-03-26T17:54:32.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:31 smithi144 ceph-mon[36095]: pgmap v834: 97 pgs: 97 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.7 MiB/s wr, 285 op/s 2024-03-26T17:54:32.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:31 smithi105 ceph-mon[32280]: pgmap v834: 97 pgs: 97 active+clean; 256 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 2.7 MiB/s wr, 285 op/s 2024-03-26T17:54:32.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 25771 6.90 MB/sec execute 75 sec latency 179.415 ms 2024-03-26T17:54:33.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 25924 6.92 MB/sec execute 76 sec latency 118.562 ms 2024-03-26T17:54:34.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:33 smithi135 ceph-mon[35636]: pgmap v835: 97 pgs: 97 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-03-26T17:54:34.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:33 smithi144 ceph-mon[36095]: pgmap v835: 97 pgs: 97 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-03-26T17:54:34.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:33 smithi105 ceph-mon[32280]: pgmap v835: 97 pgs: 97 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.5 MiB/s wr, 283 op/s 2024-03-26T17:54:34.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 26081 6.98 MB/sec execute 77 sec latency 121.697 ms 2024-03-26T17:54:35.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:34 smithi144 ceph-mon[36095]: pgmap v836: 97 pgs: 97 active+clean; 261 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-03-26T17:54:35.388 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:34 smithi105 ceph-mon[32280]: pgmap v836: 97 pgs: 97 active+clean; 261 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-03-26T17:54:35.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:34 smithi135 ceph-mon[35636]: pgmap v836: 97 pgs: 97 active+clean; 261 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-03-26T17:54:35.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 26233 7.08 MB/sec execute 78 sec latency 130.782 ms 2024-03-26T17:54:36.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 26473 7.25 MB/sec execute 79 sec latency 158.681 ms 2024-03-26T17:54:37.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 26598 7.28 MB/sec execute 80 sec latency 170.619 ms 2024-03-26T17:54:38.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:37 smithi105 ceph-mon[32280]: pgmap v837: 97 pgs: 97 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-03-26T17:54:38.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:37 smithi135 ceph-mon[35636]: pgmap v837: 97 pgs: 97 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-03-26T17:54:38.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:37 smithi144 ceph-mon[36095]: pgmap v837: 97 pgs: 97 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-03-26T17:54:38.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 26738 7.31 MB/sec execute 81 sec latency 124.991 ms 2024-03-26T17:54:39.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 26837 7.30 MB/sec execute 82 sec latency 111.575 ms 2024-03-26T17:54:40.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:39 smithi105 ceph-mon[32280]: pgmap v838: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-03-26T17:54:40.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:39 smithi135 ceph-mon[35636]: pgmap v838: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-03-26T17:54:40.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:39 smithi144 ceph-mon[36095]: pgmap v838: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 3.7 MiB/s wr, 277 op/s 2024-03-26T17:54:40.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 26907 7.23 MB/sec execute 83 sec latency 123.294 ms 2024-03-26T17:54:41.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27010 7.20 MB/sec execute 84 sec latency 134.852 ms 2024-03-26T17:54:42.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:41 smithi105 ceph-mon[32280]: pgmap v839: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 3.5 MiB/s wr, 230 op/s 2024-03-26T17:54:42.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:41 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:54:42.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:41 smithi135 ceph-mon[35636]: pgmap v839: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 3.5 MiB/s wr, 230 op/s 2024-03-26T17:54:42.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:41 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:54:42.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:41 smithi144 ceph-mon[36095]: pgmap v839: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 3.5 MiB/s wr, 230 op/s 2024-03-26T17:54:42.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:41 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:54:42.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27135 7.26 MB/sec execute 85 sec latency 148.608 ms 2024-03-26T17:54:43.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:42 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:54:43.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:42 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:54:43.138 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:42 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:54:43.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:42 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:54:43.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:42 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:54:43.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:42 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:54:43.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:42 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:54:43.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:42 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:54:43.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:42 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:54:43.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27269 7.29 MB/sec execute 86 sec latency 138.568 ms 2024-03-26T17:54:44.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:43 smithi105 ceph-mon[32280]: pgmap v840: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-03-26T17:54:44.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:43 smithi135 ceph-mon[35636]: pgmap v840: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-03-26T17:54:44.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:43 smithi144 ceph-mon[36095]: pgmap v840: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-03-26T17:54:44.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27410 7.30 MB/sec execute 87 sec latency 120.884 ms 2024-03-26T17:54:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.386+0000 7f54fa553640 1 -- 172.21.15.105:0/2266090217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 msgr2=0x7f54f4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.386+0000 7f54fa553640 1 --2- 172.21.15.105:0/2266090217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 0x7f54f4102220 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009890 tx=0x7f54e402bb70 comp rx=0 tx=0).stop 2024-03-26T17:54:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.387+0000 7f54fa553640 1 -- 172.21.15.105:0/2266090217 shutdown_connections 2024-03-26T17:54:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.387+0000 7f54fa553640 1 --2- 172.21.15.105:0/2266090217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f4101460 0x7f54f4108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.387+0000 7f54fa553640 1 --2- 172.21.15.105:0/2266090217 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54f4100aa0 0x7f54f4100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.387+0000 7f54fa553640 1 --2- 172.21.15.105:0/2266090217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 0x7f54f4102220 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.387+0000 7f54fa553640 1 -- 172.21.15.105:0/2266090217 >> 172.21.15.105:0/2266090217 conn(0x7f54f40fb510 msgr2=0x7f54f40fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.388+0000 7f54fa553640 1 -- 172.21.15.105:0/2266090217 shutdown_connections 2024-03-26T17:54:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.388+0000 7f54fa553640 1 -- 172.21.15.105:0/2266090217 wait complete. 2024-03-26T17:54:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.389+0000 7f54fa553640 1 Processor -- start 2024-03-26T17:54:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.389+0000 7f54fa553640 1 -- start start 2024-03-26T17:54:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54fa553640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54f4100aa0 0x7f54f41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54fa553640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f4101460 0x7f54f4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54fa553640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 0x7f54f419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54fa553640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40ffe60 con 0x7f54f4101460 2024-03-26T17:54:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54fa553640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40ffce0 con 0x7f54f4100aa0 2024-03-26T17:54:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54fa553640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40fffe0 con 0x7f54f4101e00 2024-03-26T17:54:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54f4100aa0 0x7f54f41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f4101460 0x7f54f4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54f4100aa0 0x7f54f41978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41352/0 (socket says 172.21.15.105:41352) 2024-03-26T17:54:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f3fff640 1 -- 172.21.15.105:0/3816923393 learned_addr learned my addr 172.21.15.105:0/3816923393 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f8af0640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 0x7f54f419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f37fe640 1 -- 172.21.15.105:0/3816923393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 msgr2=0x7f54f419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f37fe640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 0x7f54f419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f37fe640 1 -- 172.21.15.105:0/3816923393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54f4100aa0 msgr2=0x7f54f41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f37fe640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54f4100aa0 0x7f54f41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f37fe640 1 -- 172.21.15.105:0/3816923393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f419f510 con 0x7f54f4101460 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f8af0640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 0x7f54f419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.390+0000 7f54f3fff640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54f4100aa0 0x7f54f41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.391+0000 7f54f37fe640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f4101460 0x7f54f4197e30 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009c40 tx=0x7f54e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.391+0000 7f54f17fa640 1 -- 172.21.15.105:0/3816923393 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8010250 con 0x7f54f4101460 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.391+0000 7f54f17fa640 1 -- 172.21.15.105:0/3816923393 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e800ce70 con 0x7f54f4101460 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.391+0000 7f54fa553640 1 -- 172.21.15.105:0/3816923393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f410bed0 con 0x7f54f4101460 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.391+0000 7f54f17fa640 1 -- 172.21.15.105:0/3816923393 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e800f920 con 0x7f54f4101460 2024-03-26T17:54:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.391+0000 7f54fa553640 1 -- 172.21.15.105:0/3816923393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f410c470 con 0x7f54f4101460 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.394+0000 7f54fd21a740 1 Processor -- start 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.394+0000 7f54fd21a740 1 -- start start 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.394+0000 7f54f17fa640 1 -- 172.21.15.105:0/3816923393 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f54e800fac0 con 0x7f54f4101460 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c2a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c2fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54fd21a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556030b8f0b0 con 0x556030c2a5d0 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54fd21a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556030b37960 con 0x556030c2afc0 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54fd21a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556030b8b060 con 0x556030c30130 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f8af0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c2a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c2fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c2fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41358/0 (socket says 172.21.15.105:41358) 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 -- 172.21.15.105:0/1330161107 learned_addr learned my addr 172.21.15.105:0/1330161107 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f37fe640 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 -- 172.21.15.105:0/1330161107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 msgr2=0x556030c325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 -- 172.21.15.105:0/1330161107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 msgr2=0x556030c2a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c2a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 -- 172.21.15.105:0/1330161107 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556030b49c60 con 0x556030c2afc0 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f37fe640 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f17fa640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54dc0a3040 0x7f54dc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.395+0000 7f54f3fff640 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c2fac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009540 tx=0x7f54e4034870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f580aa3bced64c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.396+0000 7f54f8af0640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54dc0a3040 0x7f54dc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.396+0000 7f54f17fa640 1 -- 172.21.15.105:0/3816923393 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f54e80d7e90 con 0x7f54f4101460 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.396+0000 7f54da7fc640 1 -- 172.21.15.105:0/1330161107 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4043020 con 0x556030c2afc0 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.396+0000 7f54da7fc640 1 -- 172.21.15.105:0/1330161107 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e4034cf0 con 0x556030c2afc0 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.396+0000 7f54da7fc640 1 -- 172.21.15.105:0/1330161107 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4039040 con 0x556030c2afc0 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 -- 172.21.15.105:0/1330161107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 msgr2=0x556030c2fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54f8af0640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54dc0a3040 0x7f54dc0a5500 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x556030c2b820 tx=0x7f54ec011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c2fac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009540 tx=0x7f54e4034870 comp rx=0 tx=0).stop 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 -- 172.21.15.105:0/1330161107 shutdown_connections 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c2fac0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1330161107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c2a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 -- 172.21.15.105:0/1330161107 >> 172.21.15.105:0/1330161107 conn(0x556030c25b40 msgr2=0x556030c25f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 -- 172.21.15.105:0/1330161107 shutdown_connections 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 -- 172.21.15.105:0/1330161107 wait complete. 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.397+0000 7f54fd21a740 1 -- start start 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54fd21a740 1 Processor -- start 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c34310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c34850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c3b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54fd21a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556030c33080 con 0x556030c2a5d0 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54fd21a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556030b4adb0 con 0x556030c2afc0 2024-03-26T17:54:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54fd21a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556030c33200 con 0x556030c30130 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54fd21a740 10 client.0 _ll_register_callbacks cb 0x556030b4f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c34310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c34310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52514/0 (socket says 172.21.15.105:52514) 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54f3fff640 1 -- 172.21.15.105:0/2775418906 learned_addr learned my addr 172.21.15.105:0/2775418906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54f8af0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c3b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54f37fe640 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c34850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54f3fff640 1 -- 172.21.15.105:0/2775418906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 msgr2=0x556030c3b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54f3fff640 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c3b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.398+0000 7f54f3fff640 1 -- 172.21.15.105:0/2775418906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 msgr2=0x556030c34850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54f3fff640 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c34850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54f3fff640 1 -- 172.21.15.105:0/2775418906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560308a3210 con 0x556030c2a5d0 2024-03-26T17:54:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54f8af0640 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c3b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54f3fff640 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c34310 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f54e40057c0 tx=0x7f54e4034ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f07b6fb6ae005de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54d37fe640 1 -- 172.21.15.105:0/2775418906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4039050 con 0x556030c2a5d0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54d37fe640 1 -- 172.21.15.105:0/2775418906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f54e40035f0 con 0x556030c2a5d0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54fd21a740 1 -- 172.21.15.105:0/2775418906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e4034cf0 con 0x556030c2a5d0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54d37fe640 1 -- 172.21.15.105:0/2775418906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f54e4043040 con 0x556030c2a5d0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54fd21a740 10 client.21414 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.399+0000 7f54fd21a740 1 -- 172.21.15.105:0/2775418906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55603096f620 con 0x556030c2a5d0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54d37fe640 1 -- 172.21.15.105:0/2775418906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e4034cf0 con 0x556030c2a5d0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54d37fe640 1 -- 172.21.15.105:0/2775418906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f54e403f6b0 con 0x556030c2a5d0 2024-03-26T17:54:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54fd21a740 10 client.21414 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54fd21a740 10 client.21414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54fd21a740 20 client.21414 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54fd21a740 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556030c3fd80 0x556030c42170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54fd21a740 4 client.21414 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54fd21a740 1 -- 172.21.15.105:0/2775418906 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556030b9c5f0 con 0x556030c3fd80 2024-03-26T17:54:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.400+0000 7f54f37fe640 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556030c3fd80 0x556030c42170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.401+0000 7f54d37fe640 10 client.21414 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.401+0000 7f54f37fe640 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556030c3fd80 0x556030c42170 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd4f0fc82e25078 server_cookie=7415734e88363f82 in_seq=0 out_seq=0 2024-03-26T17:54:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54d37fe640 1 -- 172.21.15.105:0/2775418906 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556030b9c5f0 con 0x556030c3fd80 2024-03-26T17:54:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54d37fe640 10 client.21414 handle_command_reply: tid=0 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 client.21414 shutdown 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 -- 172.21.15.105:0/2775418906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 msgr2=0x556030c34310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c34310 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f54e40057c0 tx=0x7f54e4034ea0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 -- 172.21.15.105:0/2775418906 shutdown_connections 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556030c3fd80 0x556030c42170 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c3b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2afc0 0x556030c34850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 --2- 172.21.15.105:0/2775418906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2a5d0 0x556030c34310 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54fd21a740 1 -- 172.21.15.105:0/2775418906 >> 172.21.15.105:0/2775418906 conn(0x556030c25b40 msgr2=0x556030c24df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.402+0000 7f54d37fe640 0 client.21414 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.403+0000 7f54fd21a740 1 -- 172.21.15.105:0/2775418906 shutdown_connections 2024-03-26T17:54:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.403+0000 7f54fd21a740 1 -- 172.21.15.105:0/2775418906 wait complete. 2024-03-26T17:54:45.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.403+0000 7f54fd21a740 20 client.21414 trim_cache size 0 max 16384 2024-03-26T17:54:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.426+0000 7f54fd21a740 1 Processor -- start 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.426+0000 7f54fd21a740 1 -- start start 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c335a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2afc0 0x556030c3b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c33ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54fd21a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41a68b0 con 0x556030c2afc0 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54fd21a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41a83f0 con 0x556030c2a5d0 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54fd21a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41a6f80 con 0x556030c30130 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f8af0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c33ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2afc0 0x556030c3b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f8af0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c33ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59310/0 (socket says 172.21.15.105:59310) 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f8af0640 1 -- 172.21.15.105:0/471622647 learned_addr learned my addr 172.21.15.105:0/471622647 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f3fff640 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c335a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f37fe640 1 -- 172.21.15.105:0/471622647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 msgr2=0x556030c33ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f37fe640 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c33ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f37fe640 1 -- 172.21.15.105:0/471622647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 msgr2=0x556030c335a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f37fe640 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c335a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f37fe640 1 -- 172.21.15.105:0/471622647 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556030c2a3c0 con 0x556030c2afc0 2024-03-26T17:54:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f3fff640 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.427+0000 7f54f37fe640 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2afc0 0x556030c3b850 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f54e80d59a0 tx=0x7f54e808f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c2ea9b7466bff7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.428+0000 7f54d9ffb640 1 -- 172.21.15.105:0/471622647 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e80a4c80 con 0x556030c2afc0 2024-03-26T17:54:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.428+0000 7f54d9ffb640 1 -- 172.21.15.105:0/471622647 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e80a4580 con 0x556030c2afc0 2024-03-26T17:54:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.428+0000 7f54d9ffb640 1 -- 172.21.15.105:0/471622647 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e800fdb0 con 0x556030c2afc0 2024-03-26T17:54:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 -- 172.21.15.105:0/471622647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2afc0 msgr2=0x556030c3b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2afc0 0x556030c3b850 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f54e80d59a0 tx=0x7f54e808f040 comp rx=0 tx=0).stop 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 -- 172.21.15.105:0/471622647 shutdown_connections 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c33ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c2afc0 0x556030c3b850 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 --2- 172.21.15.105:0/471622647 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c335a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 -- 172.21.15.105:0/471622647 >> 172.21.15.105:0/471622647 conn(0x556030c25b40 msgr2=0x556030c29630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 -- 172.21.15.105:0/471622647 shutdown_connections 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 -- 172.21.15.105:0/471622647 wait complete. 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.429+0000 7f54fd21a740 1 -- start start 2024-03-26T17:54:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.430+0000 7f54fd21a740 1 Processor -- start 2024-03-26T17:54:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.430+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.430+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.430+0000 7f54fd21a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c3a2f0 0x556030c3a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.430+0000 7f54fd21a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41a7d20 con 0x556030c3a2f0 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54fd21a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41a7650 con 0x556030c2a5d0 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54fd21a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41a4890 con 0x556030c30130 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54fd21a740 10 client.0 _ll_register_callbacks cb 0x7f54ec009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f8af0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c3a2f0 0x556030c3a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c38da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41378/0 (socket says 172.21.15.105:41378) 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f3fff640 1 -- 172.21.15.105:0/1352267827 learned_addr learned my addr 172.21.15.105:0/1352267827 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f37fe640 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f8af0640 1 -- 172.21.15.105:0/1352267827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 msgr2=0x556030c392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f8af0640 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f8af0640 1 -- 172.21.15.105:0/1352267827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 msgr2=0x556030c38da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.431+0000 7f54f8af0640 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c38da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54f8af0640 1 -- 172.21.15.105:0/1352267827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556030c3acb0 con 0x556030c3a2f0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54f8af0640 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c3a2f0 0x556030c3a770 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f54ec005fd0 tx=0x7f54ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f9a85d675494cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54d37fe640 1 -- 172.21.15.105:0/1352267827 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec02c030 con 0x556030c3a2f0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54d37fe640 1 -- 172.21.15.105:0/1352267827 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f54ec004b20 con 0x556030c3a2f0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54fd21a740 1 -- 172.21.15.105:0/1352267827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e80a4580 con 0x556030c3a2f0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54d37fe640 1 -- 172.21.15.105:0/1352267827 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f54ec038050 con 0x556030c3a2f0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54fd21a740 10 client.21420 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.432+0000 7f54fd21a740 1 -- 172.21.15.105:0/1352267827 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556030ca62e0 con 0x556030c3a2f0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54d37fe640 1 -- 172.21.15.105:0/1352267827 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e80a4580 con 0x556030c3a2f0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54d37fe640 1 -- 172.21.15.105:0/1352267827 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f54ec00b040 con 0x556030c3a2f0 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54fd21a740 10 client.21420 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54fd21a740 10 client.21420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54fd21a740 20 client.21420 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556030ca6c70 0x556030ca9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54fd21a740 4 client.21420 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54fd21a740 1 -- 172.21.15.105:0/1352267827 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556030c42790 con 0x556030ca6c70 2024-03-26T17:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.433+0000 7f54f3fff640 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556030ca6c70 0x556030ca9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.434+0000 7f54d37fe640 10 client.21420 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.434+0000 7f54f3fff640 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556030ca6c70 0x556030ca9090 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4723165d04975d73 server_cookie=f646d58b6fc3eb94 in_seq=0 out_seq=0 2024-03-26T17:54:45.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27531 7.31 MB/sec execute 88 sec latency 139.601 ms 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54d37fe640 1 -- 172.21.15.105:0/1352267827 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556030c42790 con 0x556030ca6c70 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54d37fe640 10 client.21420 handle_command_reply: tid=0 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 client.21420 shutdown 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 -- 172.21.15.105:0/1352267827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c3a2f0 msgr2=0x556030c3a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c3a2f0 0x556030c3a770 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f54ec005fd0 tx=0x7f54ec004ab0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 -- 172.21.15.105:0/1352267827 shutdown_connections 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556030ca6c70 0x556030ca9090 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556030c3a2f0 0x556030c3a770 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54d37fe640 0 client.21420 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556030c30130 0x556030c392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 --2- 172.21.15.105:0/1352267827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556030c2a5d0 0x556030c38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.628+0000 7f54fd21a740 1 -- 172.21.15.105:0/1352267827 >> 172.21.15.105:0/1352267827 conn(0x556030c25b40 msgr2=0x556030c24560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.629+0000 7f54fd21a740 1 -- 172.21.15.105:0/1352267827 shutdown_connections 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.629+0000 7f54fd21a740 1 -- 172.21.15.105:0/1352267827 wait complete. 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.629+0000 7f54fd21a740 20 client.21420 trim_cache size 0 max 16384 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:54:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:54:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 -- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54dc0a3040 msgr2=0x7f54dc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54dc0a3040 0x7f54dc0a5500 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x556030c2b820 tx=0x7f54ec011040 comp rx=0 tx=0).stop 2024-03-26T17:54:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 -- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f4101460 msgr2=0x7f54f4197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f4101460 0x7f54f4197e30 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009c40 tx=0x7f54e800c6a0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 -- 172.21.15.105:0/3816923393 shutdown_connections 2024-03-26T17:54:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f54dc0a3040 0x7f54dc0a5500 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54f4101e00 0x7f54f419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54f4101460 0x7f54f4197e30 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 --2- 172.21.15.105:0/3816923393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54f4100aa0 0x7f54f41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.630+0000 7f54fa553640 1 -- 172.21.15.105:0/3816923393 >> 172.21.15.105:0/3816923393 conn(0x7f54f40fb510 msgr2=0x7f54f40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.631+0000 7f54fa553640 1 -- 172.21.15.105:0/3816923393 shutdown_connections 2024-03-26T17:54:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:45.631+0000 7f54fa553640 1 -- 172.21.15.105:0/3816923393 wait complete. 2024-03-26T17:54:46.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:45 smithi105 ceph-mon[32280]: pgmap v841: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-26T17:54:46.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:45 smithi135 ceph-mon[35636]: pgmap v841: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-26T17:54:46.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:45 smithi144 ceph-mon[36095]: pgmap v841: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 4.1 MiB/s wr, 201 op/s 2024-03-26T17:54:46.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27636 7.26 MB/sec execute 89 sec latency 215.470 ms 2024-03-26T17:54:47.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27776 7.29 MB/sec execute 90 sec latency 127.845 ms 2024-03-26T17:54:48.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:47 smithi105 ceph-mon[32280]: pgmap v842: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-03-26T17:54:48.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:47 smithi135 ceph-mon[35636]: pgmap v842: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-03-26T17:54:48.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:47 smithi144 ceph-mon[36095]: pgmap v842: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-03-26T17:54:48.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27885 7.25 MB/sec execute 91 sec latency 125.693 ms 2024-03-26T17:54:49.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 27984 7.22 MB/sec execute 92 sec latency 139.339 ms 2024-03-26T17:54:50.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:49 smithi105 ceph-mon[32280]: pgmap v843: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-26T17:54:50.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:49 smithi135 ceph-mon[35636]: pgmap v843: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-26T17:54:50.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:49 smithi144 ceph-mon[36095]: pgmap v843: 97 pgs: 97 active+clean; 276 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-26T17:54:50.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 28112 7.20 MB/sec execute 93 sec latency 136.812 ms 2024-03-26T17:54:51.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 28234 7.17 MB/sec execute 94 sec latency 149.694 ms 2024-03-26T17:54:52.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:51 smithi135 ceph-mon[35636]: pgmap v844: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-03-26T17:54:52.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:51 smithi144 ceph-mon[36095]: pgmap v844: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-03-26T17:54:52.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:51 smithi105 ceph-mon[32280]: pgmap v844: 97 pgs: 97 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-03-26T17:54:52.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 28353 7.12 MB/sec execute 95 sec latency 152.181 ms 2024-03-26T17:54:53.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 28469 7.08 MB/sec execute 96 sec latency 137.835 ms 2024-03-26T17:54:54.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:53 smithi135 ceph-mon[35636]: pgmap v845: 97 pgs: 97 active+clean; 281 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.4 MiB/s wr, 228 op/s 2024-03-26T17:54:54.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:53 smithi144 ceph-mon[36095]: pgmap v845: 97 pgs: 97 active+clean; 281 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.4 MiB/s wr, 228 op/s 2024-03-26T17:54:54.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:53 smithi105 ceph-mon[32280]: pgmap v845: 97 pgs: 97 active+clean; 281 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.4 MiB/s wr, 228 op/s 2024-03-26T17:54:54.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 28610 7.12 MB/sec execute 97 sec latency 124.503 ms 2024-03-26T17:54:55.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 28737 7.10 MB/sec execute 98 sec latency 120.413 ms 2024-03-26T17:54:56.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:55 smithi135 ceph-mon[35636]: pgmap v846: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-03-26T17:54:56.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:55 smithi144 ceph-mon[36095]: pgmap v846: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-03-26T17:54:56.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:55 smithi105 ceph-mon[32280]: pgmap v846: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 3.5 MiB/s wr, 248 op/s 2024-03-26T17:54:56.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 28851 7.05 MB/sec execute 99 sec latency 187.237 ms 2024-03-26T17:54:57.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:54:57.035 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:54:57.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.195+0000 7f43027e6640 1 -- 172.21.15.105:0/2073293549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc103480 msgr2=0x7f42fc109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.195+0000 7f43027e6640 1 --2- 172.21.15.105:0/2073293549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc103480 0x7f42fc109d10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009be0 tx=0x7f42f402bef0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.195+0000 7f43027e6640 1 -- 172.21.15.105:0/2073293549 shutdown_connections 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.195+0000 7f43027e6640 1 --2- 172.21.15.105:0/2073293549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc103480 0x7f42fc109d10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.195+0000 7f43027e6640 1 --2- 172.21.15.105:0/2073293549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42fc102ac0 0x7f42fc102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.195+0000 7f43027e6640 1 --2- 172.21.15.105:0/2073293549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42fc1018d0 0x7f42fc101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.195+0000 7f43027e6640 1 -- 172.21.15.105:0/2073293549 >> 172.21.15.105:0/2073293549 conn(0x7f42fc0fce60 msgr2=0x7f42fc0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.196+0000 7f43027e6640 1 -- 172.21.15.105:0/2073293549 shutdown_connections 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.196+0000 7f43027e6640 1 -- 172.21.15.105:0/2073293549 wait complete. 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.196+0000 7f43027e6640 1 Processor -- start 2024-03-26T17:54:57.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.196+0000 7f43027e6640 1 -- start start 2024-03-26T17:54:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f43027e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42fc1018d0 0x7f42fc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f43027e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc102ac0 0x7f42fc198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f43027e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42fc103480 0x7f42fc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f43027e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc10d400 con 0x7f42fc103480 2024-03-26T17:54:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f43027e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc10d280 con 0x7f42fc102ac0 2024-03-26T17:54:57.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f43027e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc10d580 con 0x7f42fc1018d0 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f42fb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc102ac0 0x7f42fc198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f42fb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc102ac0 0x7f42fc198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39072/0 (socket says 172.21.15.105:39072) 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f42fb7fe640 1 -- 172.21.15.105:0/3261070753 learned_addr learned my addr 172.21.15.105:0/3261070753 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f4300d83640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42fc103480 0x7f42fc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f42fbfff640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42fc1018d0 0x7f42fc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f4300d83640 1 -- 172.21.15.105:0/3261070753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42fc1018d0 msgr2=0x7f42fc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f4300d83640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42fc1018d0 0x7f42fc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f4300d83640 1 -- 172.21.15.105:0/3261070753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc102ac0 msgr2=0x7f42fc198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f4300d83640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc102ac0 0x7f42fc198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f4300d83640 1 -- 172.21.15.105:0/3261070753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc19f710 con 0x7f42fc103480 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.197+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc102ac0 0x7f42fc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:57.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.198+0000 7f4300d83640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42fc103480 0x7f42fc19f0a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f42f402c430 tx=0x7f42f4033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.198+0000 7f42f97fa640 1 -- 172.21.15.105:0/3261070753 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4043020 con 0x7f42fc103480 2024-03-26T17:54:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.198+0000 7f42f97fa640 1 -- 172.21.15.105:0/3261070753 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f402ca60 con 0x7f42fc103480 2024-03-26T17:54:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.198+0000 7f43027e6640 1 -- 172.21.15.105:0/3261070753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc19f9a0 con 0x7f42fc103480 2024-03-26T17:54:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.198+0000 7f42f97fa640 1 -- 172.21.15.105:0/3261070753 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4034430 con 0x7f42fc103480 2024-03-26T17:54:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.198+0000 7f43027e6640 1 -- 172.21.15.105:0/3261070753 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc19fec0 con 0x7f42fc103480 2024-03-26T17:54:57.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.201+0000 7f42f97fa640 1 -- 172.21.15.105:0/3261070753 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f42f4046020 con 0x7f42fc103480 2024-03-26T17:54:57.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f42f97fa640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42e40a2f20 0x7f42e40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f43053d0740 1 Processor -- start 2024-03-26T17:54:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f42fbfff640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42e40a2f20 0x7f42e40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f43053d0740 1 -- start start 2024-03-26T17:54:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f42f97fa640 1 -- 172.21.15.105:0/3261070753 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f42f40f19d0 con 0x7f42fc103480 2024-03-26T17:54:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f43053d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x56184022c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f43053d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f43053d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x561840233cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f43053d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56184018c980 con 0x56184022c6f0 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f43053d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561840134be0 con 0x56184022bd00 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.202+0000 7f43053d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561840188930 con 0x561840231860 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50376/0 (socket says 172.21.15.105:50376) 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 -- 172.21.15.105:0/3036096080 learned_addr learned my addr 172.21.15.105:0/3036096080 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fbfff640 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x56184022c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f4300d83640 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x561840233cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fbfff640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42e40a2f20 0x7f42e40a53e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009530 tx=0x7f42ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 -- 172.21.15.105:0/3036096080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 msgr2=0x561840233cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x561840233cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 -- 172.21.15.105:0/3036096080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 msgr2=0x56184022c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x56184022c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 -- 172.21.15.105:0/3036096080 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183ff6a7d0 con 0x56184022c6f0 2024-03-26T17:54:57.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402311f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f42f00115f0 tx=0x7f42f0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19caf83651afff5d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.203+0000 7f42e27fc640 1 -- 172.21.15.105:0/3036096080 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f001e070 con 0x56184022c6f0 2024-03-26T17:54:57.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.204+0000 7f42e27fc640 1 -- 172.21.15.105:0/3036096080 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f000fb70 con 0x56184022c6f0 2024-03-26T17:54:57.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.204+0000 7f42e27fc640 1 -- 172.21.15.105:0/3036096080 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0019c10 con 0x56184022c6f0 2024-03-26T17:54:57.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.204+0000 7f43053d0740 1 -- 172.21.15.105:0/3036096080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 msgr2=0x5618402311f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.204+0000 7f43053d0740 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402311f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f42f00115f0 tx=0x7f42f0011ab0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 -- 172.21.15.105:0/3036096080 shutdown_connections 2024-03-26T17:54:57.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x561840233cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402311f0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 --2- 172.21.15.105:0/3036096080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x56184022c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 -- 172.21.15.105:0/3036096080 >> 172.21.15.105:0/3036096080 conn(0x561840227270 msgr2=0x561840227670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 -- 172.21.15.105:0/3036096080 shutdown_connections 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 -- 172.21.15.105:0/3036096080 wait complete. 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 -- start start 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.205+0000 7f43053d0740 1 Processor -- start 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f43053d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x561840235f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f43053d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f43053d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x56184023d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f43053d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561840234810 con 0x56184022c6f0 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f43053d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561840148160 con 0x56184022bd00 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f43053d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561840234990 con 0x561840231860 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50380/0 (socket says 172.21.15.105:50380) 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 -- 172.21.15.105:0/2549973960 learned_addr learned my addr 172.21.15.105:0/2549973960 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f43053d0740 10 client.0 _ll_register_callbacks cb 0x561840184720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f4300d83640 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x56184023d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fbfff640 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x561840235f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 -- 172.21.15.105:0/2549973960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 msgr2=0x56184023d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x56184023d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 -- 172.21.15.105:0/2549973960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 msgr2=0x561840235f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x561840235f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 -- 172.21.15.105:0/2549973960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183fea03f0 con 0x56184022c6f0 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.206+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f42f00115c0 tx=0x7f42f000ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da45443389ef81c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f42db7fe640 1 -- 172.21.15.105:0/2549973960 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0026020 con 0x56184022c6f0 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f42db7fe640 1 -- 172.21.15.105:0/2549973960 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f42f0019950 con 0x56184022c6f0 2024-03-26T17:54:57.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f43053d0740 1 -- 172.21.15.105:0/2549973960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f000fb70 con 0x56184022c6f0 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f42db7fe640 1 -- 172.21.15.105:0/2549973960 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f42f0031050 con 0x56184022c6f0 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f43053d0740 10 client.21432 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f43053d0740 1 -- 172.21.15.105:0/2549973960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56183fe600f0 con 0x56184022c6f0 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f42db7fe640 1 -- 172.21.15.105:0/2549973960 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f0009da0 con 0x56184022c6f0 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f42db7fe640 1 -- 172.21.15.105:0/2549973960 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f42f0021030 con 0x56184022c6f0 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.207+0000 7f43053d0740 10 client.21432 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.208+0000 7f43053d0740 10 client.21432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.208+0000 7f43053d0740 20 client.21432 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.208+0000 7f43053d0740 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5618402420f0 0x561840244590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.208+0000 7f43053d0740 4 client.21432 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.208+0000 7f43053d0740 1 -- 172.21.15.105:0/2549973960 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f42f000fb70 con 0x5618402420f0 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.208+0000 7f42fbfff640 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5618402420f0 0x561840244590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.208+0000 7f42db7fe640 10 client.21432 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:57.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.208+0000 7f42fbfff640 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5618402420f0 0x561840244590 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9be10bc738417b4 server_cookie=9e45f9e80f1ea49a in_seq=0 out_seq=0 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.209+0000 7f42db7fe640 1 -- 172.21.15.105:0/2549973960 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f42f000fb70 con 0x5618402420f0 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.209+0000 7f42db7fe640 10 client.21432 handle_command_reply: tid=0 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.209+0000 7f43053d0740 1 client.21432 shutdown 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 -- 172.21.15.105:0/2549973960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 msgr2=0x5618402364b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f42f00115c0 tx=0x7f42f000ff50 comp rx=0 tx=0).stop 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 -- 172.21.15.105:0/2549973960 shutdown_connections 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5618402420f0 0x561840244590 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x56184023d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 --2- 172.21.15.105:0/2549973960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x561840235f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 -- 172.21.15.105:0/2549973960 >> 172.21.15.105:0/2549973960 conn(0x561840227270 msgr2=0x561840222080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f42db7fe640 0 client.21432 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:57.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 -- 172.21.15.105:0/2549973960 shutdown_connections 2024-03-26T17:54:57.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 1 -- 172.21.15.105:0/2549973960 wait complete. 2024-03-26T17:54:57.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.210+0000 7f43053d0740 20 client.21432 trim_cache size 0 max 16384 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f43053d0740 1 Processor -- start 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f43053d0740 1 -- start start 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f43053d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x561840235f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f43053d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f43053d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x56184023d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f43053d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a6b20 con 0x56184022c6f0 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f43053d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a8660 con 0x56184022bd00 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f43053d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a71f0 con 0x561840231860 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f42fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f42fb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50388/0 (socket says 172.21.15.105:50388) 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f42fb7fe640 1 -- 172.21.15.105:0/2030587269 learned_addr learned my addr 172.21.15.105:0/2030587269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.240+0000 7f4300d83640 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x56184023d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42fbfff640 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x561840235f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42fb7fe640 1 -- 172.21.15.105:0/2030587269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 msgr2=0x56184023d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x56184023d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42fb7fe640 1 -- 172.21.15.105:0/2030587269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 msgr2=0x561840235f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x561840235f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42fb7fe640 1 -- 172.21.15.105:0/2030587269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561840227fe0 con 0x56184022c6f0 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f42f00049a0 tx=0x7f42f0005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b953405d2c033ed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42e1ffb640 1 -- 172.21.15.105:0/2030587269 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f001e040 con 0x56184022c6f0 2024-03-26T17:54:57.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42e1ffb640 1 -- 172.21.15.105:0/2030587269 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f0004680 con 0x56184022c6f0 2024-03-26T17:54:57.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.241+0000 7f42e1ffb640 1 -- 172.21.15.105:0/2030587269 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0019d00 con 0x56184022c6f0 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 -- 172.21.15.105:0/2030587269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 msgr2=0x5618402364b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f42f00049a0 tx=0x7f42f0005b90 comp rx=0 tx=0).stop 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 -- 172.21.15.105:0/2030587269 shutdown_connections 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561840231860 0x56184023d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022c6f0 0x5618402364b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 --2- 172.21.15.105:0/2030587269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56184022bd00 0x561840235f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 -- 172.21.15.105:0/2030587269 >> 172.21.15.105:0/2030587269 conn(0x561840227270 msgr2=0x561840227650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 -- 172.21.15.105:0/2030587269 shutdown_connections 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 -- 172.21.15.105:0/2030587269 wait complete. 2024-03-26T17:54:57.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 -- start start 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.242+0000 7f43053d0740 1 Processor -- start 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f43053d0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022bd00 0x56184023a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f43053d0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56184022c6f0 0x56184023a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f43053d0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561840231860 0x5618402a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f43053d0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a7f90 con 0x56184022bd00 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f43053d0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a78c0 con 0x561840231860 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f43053d0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc1a4b00 con 0x56184022c6f0 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022bd00 0x56184023a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f43053d0740 10 client.0 _ll_register_callbacks cb 0x7f42f001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022bd00 0x56184023a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50404/0 (socket says 172.21.15.105:50404) 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 -- 172.21.15.105:0/2799884412 learned_addr learned my addr 172.21.15.105:0/2799884412 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56184022c6f0 0x56184023a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f4300d83640 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561840231860 0x5618402a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 -- 172.21.15.105:0/2799884412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56184022c6f0 msgr2=0x56184023a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56184022c6f0 0x56184023a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 -- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561840231860 msgr2=0x5618402a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561840231860 0x5618402a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 -- 172.21.15.105:0/2799884412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56184023d790 con 0x56184022bd00 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f4300d83640 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561840231860 0x5618402a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42fbfff640 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022bd00 0x56184023a180 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x561840244e40 tx=0x7f42ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b7d0882eda4053 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.243+0000 7f42db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.244+0000 7f42db7fe640 1 -- 172.21.15.105:0/2799884412 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec01bb80 con 0x56184022bd00 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.244+0000 7f42db7fe640 1 -- 172.21.15.105:0/2799884412 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f42ec01bd20 con 0x56184022bd00 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.244+0000 7f43053d0740 1 -- 172.21.15.105:0/2799884412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f0004680 con 0x56184022bd00 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.244+0000 7f42db7fe640 1 -- 172.21.15.105:0/2799884412 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f42ec02c0d0 con 0x56184022bd00 2024-03-26T17:54:57.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.244+0000 7f43053d0740 10 client.21438 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.244+0000 7f43053d0740 1 -- 172.21.15.105:0/2799884412 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5618402a7a90 con 0x56184022bd00 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.244+0000 7f42db7fe640 1 -- 172.21.15.105:0/2799884412 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42f0004680 con 0x56184022bd00 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f42db7fe640 1 -- 172.21.15.105:0/2799884412 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f42ec02a780 con 0x56184022bd00 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f43053d0740 10 client.21438 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f43053d0740 10 client.21438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f43053d0740 20 client.21438 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f43053d0740 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5618402a8470 0x5618402aa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f43053d0740 4 client.21438 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f43053d0740 1 -- 172.21.15.105:0/2799884412 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561840199ec0 con 0x5618402a8470 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5618402a8470 0x5618402aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f42db7fe640 10 client.21438 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.245+0000 7f42fb7fe640 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5618402a8470 0x5618402aa890 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e45ae33a8723eb server_cookie=8c87023f352f3ee6 in_seq=0 out_seq=0 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.246+0000 7f42db7fe640 1 -- 172.21.15.105:0/2799884412 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561840199ec0 con 0x5618402a8470 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.246+0000 7f42db7fe640 10 client.21438 handle_command_reply: tid=0 2024-03-26T17:54:57.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.246+0000 7f43053d0740 1 client.21438 shutdown 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.246+0000 7f43053d0740 1 -- 172.21.15.105:0/2799884412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022bd00 msgr2=0x56184023a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.246+0000 7f43053d0740 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022bd00 0x56184023a180 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x561840244e40 tx=0x7f42ec004730 comp rx=0 tx=0).stop 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 1 -- 172.21.15.105:0/2799884412 shutdown_connections 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5618402a8470 0x5618402aa890 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561840231860 0x5618402a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56184022c6f0 0x56184023a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f42db7fe640 0 client.21438 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 1 --2- 172.21.15.105:0/2799884412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56184022bd00 0x56184023a180 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 1 -- 172.21.15.105:0/2799884412 >> 172.21.15.105:0/2799884412 conn(0x561840227270 msgr2=0x56184022d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 1 -- 172.21.15.105:0/2799884412 shutdown_connections 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 1 -- 172.21.15.105:0/2799884412 wait complete. 2024-03-26T17:54:57.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.247+0000 7f43053d0740 20 client.21438 trim_cache size 0 max 16384 2024-03-26T17:54:57.241 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:54:57.241 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:54:57.241 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:54:57.241 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 -- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42e40a2f20 msgr2=0x7f42e40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42e40a2f20 0x7f42e40a53e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009530 tx=0x7f42ec009260 comp rx=0 tx=0).stop 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 -- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42fc103480 msgr2=0x7f42fc19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42fc103480 0x7f42fc19f0a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f42f402c430 tx=0x7f42f4033c90 comp rx=0 tx=0).stop 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 -- 172.21.15.105:0/3261070753 shutdown_connections 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f42e40a2f20 0x7f42e40a53e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42fc103480 0x7f42fc19f0a0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f42fc102ac0 0x7f42fc198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 --2- 172.21.15.105:0/3261070753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42fc1018d0 0x7f42fc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 -- 172.21.15.105:0/3261070753 >> 172.21.15.105:0/3261070753 conn(0x7f42fc0fce60 msgr2=0x7f42fc105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 -- 172.21.15.105:0/3261070753 shutdown_connections 2024-03-26T17:54:57.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.248+0000 7f43027e6640 1 -- 172.21.15.105:0/3261070753 wait complete. 2024-03-26T17:54:57.249 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:54:57.249 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 -- 172.21.15.105:0/2721488774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9058102390 msgr2=0x7f9058108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 --2- 172.21.15.105:0/2721488774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9058102390 0x7f9058108c20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9050009ca0 tx=0x7f905002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 -- 172.21.15.105:0/2721488774 shutdown_connections 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 --2- 172.21.15.105:0/2721488774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9058102390 0x7f9058108c20 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 --2- 172.21.15.105:0/2721488774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90581019d0 0x7f9058101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 --2- 172.21.15.105:0/2721488774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90581007e0 0x7f9058100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 -- 172.21.15.105:0/2721488774 >> 172.21.15.105:0/2721488774 conn(0x7f90580fbd50 msgr2=0x7f90580fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 -- 172.21.15.105:0/2721488774 shutdown_connections 2024-03-26T17:54:57.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.401+0000 7f90605ab640 1 -- 172.21.15.105:0/2721488774 wait complete. 2024-03-26T17:54:57.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f90605ab640 1 Processor -- start 2024-03-26T17:54:57.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f90605ab640 1 -- start start 2024-03-26T17:54:57.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f90605ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581007e0 0x7f9058199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f90605ab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90581019d0 0x7f905819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f90605ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9058102390 0x7f90581a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f90605ab640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f905810c310 con 0x7f90581007e0 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f90605ab640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f905810c190 con 0x7f90581019d0 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f90605ab640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f905810c490 con 0x7f9058102390 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905e347640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581007e0 0x7f9058199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905e347640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581007e0 0x7f9058199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50430/0 (socket says 172.21.15.105:50430) 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905e347640 1 -- 172.21.15.105:0/3520152441 learned_addr learned my addr 172.21.15.105:0/3520152441 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905eb48640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9058102390 0x7f90581a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905db46640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90581019d0 0x7f905819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905e347640 1 -- 172.21.15.105:0/3520152441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9058102390 msgr2=0x7f90581a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905e347640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9058102390 0x7f90581a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905e347640 1 -- 172.21.15.105:0/3520152441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90581019d0 msgr2=0x7f905819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.402+0000 7f905e347640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90581019d0 0x7f905819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.403+0000 7f905e347640 1 -- 172.21.15.105:0/3520152441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90581a1960 con 0x7f90581007e0 2024-03-26T17:54:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.403+0000 7f905e347640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581007e0 0x7f9058199d40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f905400e9c0 tx=0x7f905400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.403+0000 7f904f7fe640 1 -- 172.21.15.105:0/3520152441 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905400cd00 con 0x7f90581007e0 2024-03-26T17:54:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.403+0000 7f904f7fe640 1 -- 172.21.15.105:0/3520152441 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90540035f0 con 0x7f90581007e0 2024-03-26T17:54:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.403+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90581a1c50 con 0x7f90581007e0 2024-03-26T17:54:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.403+0000 7f904f7fe640 1 -- 172.21.15.105:0/3520152441 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054010690 con 0x7f90581007e0 2024-03-26T17:54:57.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.403+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90581a21a0 con 0x7f90581007e0 2024-03-26T17:54:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.405+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90581a23d0 con 0x7f90581007e0 2024-03-26T17:54:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.406+0000 7f904f7fe640 1 -- 172.21.15.105:0/3520152441 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f90540037a0 con 0x7f90581007e0 2024-03-26T17:54:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.406+0000 7f904f7fe640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90440a30a0 0x7f90440a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.407+0000 7f904f7fe640 1 -- 172.21.15.105:0/3520152441 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f90540cf790 con 0x7f90581007e0 2024-03-26T17:54:57.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.407+0000 7f905db46640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90440a30a0 0x7f90440a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.407+0000 7f905db46640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90440a30a0 0x7f90440a5560 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f905819b250 tx=0x7f9048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.414+0000 7f904f7fe640 1 -- 172.21.15.105:0/3520152441 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f905409c580 con 0x7f90581007e0 2024-03-26T17:54:57.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 28988 7.09 MB/sec execute 100 sec latency 120.323 ms 2024-03-26T17:54:57.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.771+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90581a25a0 con 0x7f90581007e0 2024-03-26T17:54:57.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.772+0000 7f904f7fe640 1 -- 172.21.15.105:0/3520152441 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11941 (secure 0 0 0) 0x7f905409be80 con 0x7f90581007e0 2024-03-26T17:54:57.760 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:54:57.761 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:54:01.817010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:54:57.761 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 37 2024-03-26T17:54:57.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90440a30a0 msgr2=0x7f90440a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90440a30a0 0x7f90440a5560 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f905819b250 tx=0x7f9048005f90 comp rx=0 tx=0).stop 2024-03-26T17:54:57.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581007e0 msgr2=0x7f9058199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581007e0 0x7f9058199d40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f905400e9c0 tx=0x7f905400ee80 comp rx=0 tx=0).stop 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 shutdown_connections 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90440a30a0 0x7f90440a5560 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9058102390 0x7f90581a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90581019d0 0x7f905819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 --2- 172.21.15.105:0/3520152441 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90581007e0 0x7f9058199d40 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 >> 172.21.15.105:0/3520152441 conn(0x7f90580fbd50 msgr2=0x7f90580fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 shutdown_connections 2024-03-26T17:54:57.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.775+0000 7f90605ab640 1 -- 172.21.15.105:0/3520152441 wait complete. 2024-03-26T17:54:57.778 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:54:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.935+0000 7fd2a42dc640 1 -- 172.21.15.105:0/790693833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 msgr2=0x7fd29c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.935+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/790693833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 0x7fd29c101e30 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009900 tx=0x7fd28c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.935+0000 7fd2a42dc640 1 -- 172.21.15.105:0/790693833 shutdown_connections 2024-03-26T17:54:57.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.935+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/790693833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c102370 0x7fd29c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.935+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/790693833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 0x7fd29c101e30 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.935+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/790693833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd29c1007c0 0x7fd29c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.935+0000 7fd2a42dc640 1 -- 172.21.15.105:0/790693833 >> 172.21.15.105:0/790693833 conn(0x7fd29c0fbd50 msgr2=0x7fd29c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.936+0000 7fd2a42dc640 1 -- 172.21.15.105:0/790693833 shutdown_connections 2024-03-26T17:54:57.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.936+0000 7fd2a42dc640 1 -- 172.21.15.105:0/790693833 wait complete. 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.936+0000 7fd2a42dc640 1 Processor -- start 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.936+0000 7fd2a42dc640 1 -- start start 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a42dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c1007c0 0x7fd29c196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a42dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 0x7fd29c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a42dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd29c102370 0x7fd29c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a42dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c10c2d0 con 0x7fd29c1019b0 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a42dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c10c150 con 0x7fd29c102370 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a42dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd29c10c450 con 0x7fd29c1007c0 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a1877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 0x7fd29c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a1877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 0x7fd29c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50440/0 (socket says 172.21.15.105:50440) 2024-03-26T17:54:57.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a1877640 1 -- 172.21.15.105:0/117554386 learned_addr learned my addr 172.21.15.105:0/117554386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a2879640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd29c102370 0x7fd29c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a2078640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c1007c0 0x7fd29c196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a2879640 1 -- 172.21.15.105:0/117554386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c1007c0 msgr2=0x7fd29c196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a2879640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c1007c0 0x7fd29c196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a2879640 1 -- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 msgr2=0x7fd29c196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a2879640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 0x7fd29c196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a2879640 1 -- 172.21.15.105:0/117554386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29c19e660 con 0x7fd29c102370 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a1877640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 0x7fd29c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.937+0000 7fd2a2879640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd29c102370 0x7fd29c19dff0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd294009fd0 tx=0x7fd29400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.938+0000 7fd2937fe640 1 -- 172.21.15.105:0/117554386 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294004d20 con 0x7fd29c102370 2024-03-26T17:54:57.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.938+0000 7fd2a42dc640 1 -- 172.21.15.105:0/117554386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29c068a80 con 0x7fd29c102370 2024-03-26T17:54:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.938+0000 7fd2a42dc640 1 -- 172.21.15.105:0/117554386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29c068fd0 con 0x7fd29c102370 2024-03-26T17:54:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.938+0000 7fd2937fe640 1 -- 172.21.15.105:0/117554386 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd29400b980 con 0x7fd29c102370 2024-03-26T17:54:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.938+0000 7fd2937fe640 1 -- 172.21.15.105:0/117554386 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294005560 con 0x7fd29c102370 2024-03-26T17:54:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.941+0000 7fd2a6fa1740 1 Processor -- start 2024-03-26T17:54:57.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.941+0000 7fd2937fe640 1 -- 172.21.15.105:0/117554386 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd2940057e0 con 0x7fd29c102370 2024-03-26T17:54:57.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.941+0000 7fd2a6fa1740 1 -- start start 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e685e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e68af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 0x55e50e68da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e50e5ea970 con 0x55e50e68b590 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e50e592d70 con 0x55e50e686420 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e50e5e6920 con 0x55e50e685a30 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e685e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a2078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e68af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e685e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34780/0 (socket says 172.21.15.105:34780) 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a2078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e68af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39170/0 (socket says 172.21.15.105:39170) 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 -- 172.21.15.105:0/2884175859 learned_addr learned my addr 172.21.15.105:0/2884175859 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 0x55e50e68da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2937fe640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2880a2fa0 0x7fd2880a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 -- 172.21.15.105:0/2884175859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 msgr2=0x55e50e68af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e68af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 -- 172.21.15.105:0/2884175859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 msgr2=0x55e50e68da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 0x55e50e68da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 -- 172.21.15.105:0/2884175859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50e2fe3f0 con 0x55e50e685a30 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a2078640 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e68af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a2879640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2880a2fa0 0x7fd2880a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.942+0000 7fd2a1877640 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e685e50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0096c0 tx=0x7fd28c005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fd85b1f4b316bd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.943+0000 7fd2937fe640 1 -- 172.21.15.105:0/117554386 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd2940d1790 con 0x7fd29c102370 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.943+0000 7fd290bfe640 1 -- 172.21.15.105:0/2884175859 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c04a070 con 0x55e50e685a30 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.943+0000 7fd290bfe640 1 -- 172.21.15.105:0/2884175859 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd28c004410 con 0x55e50e685a30 2024-03-26T17:54:57.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.943+0000 7fd290bfe640 1 -- 172.21.15.105:0/2884175859 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c004bf0 con 0x55e50e685a30 2024-03-26T17:54:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/2884175859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 msgr2=0x55e50e685e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a2879640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2880a2fa0 0x7fd2880a5460 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd29409de10 tx=0x7fd29406f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e685e50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0096c0 tx=0x7fd28c005b50 comp rx=0 tx=0).stop 2024-03-26T17:54:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/2884175859 shutdown_connections 2024-03-26T17:54:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 0x55e50e68da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e68af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/2884175859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e685e50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/2884175859 >> 172.21.15.105:0/2884175859 conn(0x55e50e680fa0 msgr2=0x55e50e6813a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/2884175859 shutdown_connections 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/2884175859 wait complete. 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 -- start start 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 Processor -- start 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e685a30 0x55e50e68ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.944+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e686420 0x55e50e690510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e68b590 0x55e50e697550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e50e68e4e0 con 0x55e50e686420 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e50e5a6310 con 0x55e50e685a30 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e50e68e660 con 0x55e50e68b590 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a6fa1740 10 client.0 _ll_register_callbacks cb 0x55e50e5e1c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e686420 0x55e50e690510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e686420 0x55e50e690510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50464/0 (socket says 172.21.15.105:50464) 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 -- 172.21.15.105:0/1493834898 learned_addr learned my addr 172.21.15.105:0/1493834898 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a1877640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e685a30 0x55e50e68ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a1877640 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e68b590 0x55e50e697550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 -- 172.21.15.105:0/1493834898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e68b590 msgr2=0x55e50e697550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e68b590 0x55e50e697550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 -- 172.21.15.105:0/1493834898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e685a30 msgr2=0x55e50e68ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e685a30 0x55e50e68ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 -- 172.21.15.105:0/1493834898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50e3c87d0 con 0x55e50e686420 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a1877640 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e685a30 0x55e50e68ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a2078640 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e686420 0x55e50e690510 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fd298012700 tx=0x7fd298012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bee080c60ee1e02 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd285ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd285ffb640 1 -- 172.21.15.105:0/1493834898 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298004030 con 0x55e50e686420 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd285ffb640 1 -- 172.21.15.105:0/1493834898 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd2980041d0 con 0x55e50e686420 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.945+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/1493834898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd28c004410 con 0x55e50e686420 2024-03-26T17:54:57.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd285ffb640 1 -- 172.21.15.105:0/1493834898 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd2980043d0 con 0x55e50e686420 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a6fa1740 10 client.21456 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/1493834898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e50e69a9c0 con 0x55e50e686420 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd285ffb640 1 -- 172.21.15.105:0/1493834898 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd28c004410 con 0x55e50e686420 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd285ffb640 1 -- 172.21.15.105:0/1493834898 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd2980039a0 con 0x55e50e686420 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a6fa1740 10 client.21456 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a6fa1740 10 client.21456 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a6fa1740 20 client.21456 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e50e69c040 0x55e50e69e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a6fa1740 4 client.21456 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/1493834898 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e50e5f7eb0 con 0x55e50e69c040 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.946+0000 7fd2a1877640 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e50e69c040 0x55e50e69e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd285ffb640 10 client.21456 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a1877640 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e50e69c040 0x55e50e69e4c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf55d4815a911719 server_cookie=bf542a78e013036c in_seq=0 out_seq=0 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd285ffb640 1 -- 172.21.15.105:0/1493834898 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e50e5f7eb0 con 0x55e50e69c040 2024-03-26T17:54:57.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd285ffb640 10 client.21456 handle_command_reply: tid=0 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a6fa1740 1 client.21456 shutdown 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/1493834898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e686420 msgr2=0x55e50e690510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e686420 0x55e50e690510 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fd298012700 tx=0x7fd298012bc0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/1493834898 shutdown_connections 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e50e69c040 0x55e50e69e4c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e68b590 0x55e50e697550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e686420 0x55e50e690510 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd285ffb640 0 client.21456 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.947+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/1493834898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e685a30 0x55e50e68ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.948+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/1493834898 >> 172.21.15.105:0/1493834898 conn(0x55e50e680fa0 msgr2=0x55e50e6801c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.948+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/1493834898 shutdown_connections 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.948+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/1493834898 wait complete. 2024-03-26T17:54:57.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.948+0000 7fd2a6fa1740 20 client.21456 trim_cache size 0 max 16384 2024-03-26T17:54:57.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a6fa1740 1 Processor -- start 2024-03-26T17:54:57.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a6fa1740 1 -- start start 2024-03-26T17:54:57.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e685a30 0x55e50e67f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e686420 0x55e50e697510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e68b590 0x55e50e67fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd28000e520 con 0x55e50e685a30 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd280010060 con 0x55e50e68b590 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd28000ebf0 con 0x55e50e686420 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a1877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e685a30 0x55e50e67f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a2078640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e686420 0x55e50e697510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a1877640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e685a30 0x55e50e67f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50470/0 (socket says 172.21.15.105:50470) 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a1877640 1 -- 172.21.15.105:0/4028341448 learned_addr learned my addr 172.21.15.105:0/4028341448 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.968+0000 7fd2a2078640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e686420 0x55e50e697510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34800/0 (socket says 172.21.15.105:34800) 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd2a1877640 1 -- 172.21.15.105:0/4028341448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e686420 msgr2=0x55e50e697510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd2a1877640 1 --2- 172.21.15.105:0/4028341448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e686420 0x55e50e697510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd2a1877640 1 -- 172.21.15.105:0/4028341448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e68b590 msgr2=0x55e50e67fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd2a1877640 1 --2- 172.21.15.105:0/4028341448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e68b590 0x55e50e67fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd2a1877640 1 -- 172.21.15.105:0/4028341448 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50e681d10 con 0x55e50e685a30 2024-03-26T17:54:57.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd2a1877640 1 --2- 172.21.15.105:0/4028341448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e685a30 0x55e50e67f930 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009430 tx=0x7fd28c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b2c277ab31c016 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd287fff640 1 -- 172.21.15.105:0/4028341448 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c05b030 con 0x55e50e685a30 2024-03-26T17:54:57.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd287fff640 1 -- 172.21.15.105:0/4028341448 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd28c04b040 con 0x55e50e685a30 2024-03-26T17:54:57.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.969+0000 7fd287fff640 1 -- 172.21.15.105:0/4028341448 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c04a040 con 0x55e50e685a30 2024-03-26T17:54:57.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/4028341448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e685a30 msgr2=0x55e50e67f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/4028341448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e685a30 0x55e50e67f930 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009430 tx=0x7fd28c005e40 comp rx=0 tx=0).stop 2024-03-26T17:54:57.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/4028341448 shutdown_connections 2024-03-26T17:54:57.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/4028341448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e68b590 0x55e50e67fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/4028341448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e686420 0x55e50e697510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/4028341448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e685a30 0x55e50e67f930 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/4028341448 >> 172.21.15.105:0/4028341448 conn(0x55e50e680fa0 msgr2=0x55e50e687980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/4028341448 shutdown_connections 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.970+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/4028341448 wait complete. 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 1 -- start start 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 1 Processor -- start 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e6943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e694920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 0x55e50e7003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd28000f990 con 0x55e50e68b590 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd28000f2c0 con 0x55e50e686420 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd28000c500 con 0x55e50e685a30 2024-03-26T17:54:57.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a6fa1740 10 client.0 _ll_register_callbacks cb 0x7fd298009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a1877640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e6943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a2078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e694920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a2078640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e694920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39200/0 (socket says 172.21.15.105:39200) 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a2078640 1 -- 172.21.15.105:0/989626445 learned_addr learned my addr 172.21.15.105:0/989626445 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a1877640 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 0x55e50e7003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a1877640 1 -- 172.21.15.105:0/989626445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 msgr2=0x55e50e694920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a1877640 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e694920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a1877640 1 -- 172.21.15.105:0/989626445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 msgr2=0x55e50e7003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a1877640 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 0x55e50e7003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.971+0000 7fd2a1877640 1 -- 172.21.15.105:0/989626445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50e7008f0 con 0x55e50e685a30 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd2a1877640 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e6943e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005ab0 tx=0x7fd28c045040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d96f957ef67a0d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd285ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd285ffb640 1 -- 172.21.15.105:0/989626445 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c064020 con 0x55e50e685a30 2024-03-26T17:54:57.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd285ffb640 1 -- 172.21.15.105:0/989626445 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fd28c003af0 con 0x55e50e685a30 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/989626445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd28c04b040 con 0x55e50e685a30 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd285ffb640 1 -- 172.21.15.105:0/989626445 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd28c05b040 con 0x55e50e685a30 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd2a6fa1740 10 client.31195 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/989626445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e50e701b90 con 0x55e50e685a30 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd285ffb640 1 -- 172.21.15.105:0/989626445 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd28c04b040 con 0x55e50e685a30 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.972+0000 7fd285ffb640 1 -- 172.21.15.105:0/989626445 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd28c043450 con 0x55e50e685a30 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 10 client.31195 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 10 client.31195 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 20 client.31195 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e50e702010 0x55e50e704400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 4 client.31195 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/989626445 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e50e5f2200 con 0x55e50e702010 2024-03-26T17:54:57.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a2078640 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e50e702010 0x55e50e704400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd285ffb640 10 client.31195 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a2078640 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e50e702010 0x55e50e704400 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5fcda841eec11bb server_cookie=8ad0ac81689a7bde in_seq=0 out_seq=0 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd285ffb640 1 -- 172.21.15.105:0/989626445 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e50e5f2200 con 0x55e50e702010 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd285ffb640 10 client.31195 handle_command_reply: tid=0 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 client.31195 shutdown 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/989626445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 msgr2=0x55e50e6943e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e6943e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd28c005ab0 tx=0x7fd28c045040 comp rx=0 tx=0).stop 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/989626445 shutdown_connections 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55e50e702010 0x55e50e704400 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e50e68b590 0x55e50e7003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e50e686420 0x55e50e694920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 --2- 172.21.15.105:0/989626445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e50e685a30 0x55e50e6943e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.973+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/989626445 >> 172.21.15.105:0/989626445 conn(0x55e50e680fa0 msgr2=0x55e50e684200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.974+0000 7fd285ffb640 0 client.31195 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:54:57.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.974+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/989626445 shutdown_connections 2024-03-26T17:54:57.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.974+0000 7fd2a6fa1740 1 -- 172.21.15.105:0/989626445 wait complete. 2024-03-26T17:54:57.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.974+0000 7fd2a6fa1740 20 client.31195 trim_cache size 0 max 16384 2024-03-26T17:54:57.968 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 -- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2880a2fa0 msgr2=0x7fd2880a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2880a2fa0 0x7fd2880a5460 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd29409de10 tx=0x7fd29406f040 comp rx=0 tx=0).stop 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 -- 172.21.15.105:0/117554386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd29c102370 msgr2=0x7fd29c19dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd29c102370 0x7fd29c19dff0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd294009fd0 tx=0x7fd29400c6a0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 -- 172.21.15.105:0/117554386 shutdown_connections 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2880a2fa0 0x7fd2880a5460 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd29c102370 0x7fd29c19dff0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd29c1019b0 0x7fd29c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 --2- 172.21.15.105:0/117554386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c1007c0 0x7fd29c196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 -- 172.21.15.105:0/117554386 >> 172.21.15.105:0/117554386 conn(0x7fd29c0fbd50 msgr2=0x7fd29c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 -- 172.21.15.105:0/117554386 shutdown_connections 2024-03-26T17:54:57.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:57.975+0000 7fd2a42dc640 1 -- 172.21.15.105:0/117554386 wait complete. 2024-03-26T17:54:57.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:54:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 -- 172.21.15.105:0/1324773714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 msgr2=0x7fcb24100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 --2- 172.21.15.105:0/1324773714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 0x7fcb24100be0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009ca0 tx=0x7fcb140301f0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 -- 172.21.15.105:0/1324773714 shutdown_connections 2024-03-26T17:54:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 --2- 172.21.15.105:0/1324773714 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb24102370 0x7fcb24108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 --2- 172.21.15.105:0/1324773714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241019b0 0x7fcb24101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 --2- 172.21.15.105:0/1324773714 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 0x7fcb24100be0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 -- 172.21.15.105:0/1324773714 >> 172.21.15.105:0/1324773714 conn(0x7fcb240fbd50 msgr2=0x7fcb240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 -- 172.21.15.105:0/1324773714 shutdown_connections 2024-03-26T17:54:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.131+0000 7fcb290dc640 1 -- 172.21.15.105:0/1324773714 wait complete. 2024-03-26T17:54:58.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb290dc640 1 Processor -- start 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb290dc640 1 -- start start 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb290dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 0x7fcb24195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb290dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241019b0 0x7fcb24195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb290dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb24102370 0x7fcb2419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb290dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2410b1b0 con 0x7fcb241007c0 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb290dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2410b030 con 0x7fcb24102370 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb290dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2410b330 con 0x7fcb241019b0 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb2259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241019b0 0x7fcb24195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb2359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb24102370 0x7fcb2419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 0x7fcb24195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb2359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb24102370 0x7fcb2419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39220/0 (socket says 172.21.15.105:39220) 2024-03-26T17:54:58.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.132+0000 7fcb2359e640 1 -- 172.21.15.105:0/3582863802 learned_addr learned my addr 172.21.15.105:0/3582863802 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb2259c640 1 -- 172.21.15.105:0/3582863802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb24102370 msgr2=0x7fcb2419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb2259c640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb24102370 0x7fcb2419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb2259c640 1 -- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 msgr2=0x7fcb24195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb2259c640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 0x7fcb24195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb2259c640 1 -- 172.21.15.105:0/3582863802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2419d560 con 0x7fcb241019b0 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb2359e640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb24102370 0x7fcb2419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 0x7fcb24195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb2259c640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241019b0 0x7fcb24195e80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb0bfff640 1 -- 172.21.15.105:0/3582863802 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1800ccc0 con 0x7fcb241019b0 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb290dc640 1 -- 172.21.15.105:0/3582863802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2419d850 con 0x7fcb241019b0 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb290dc640 1 -- 172.21.15.105:0/3582863802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2419ddf0 con 0x7fcb241019b0 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb0bfff640 1 -- 172.21.15.105:0/3582863802 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb180108e0 con 0x7fcb241019b0 2024-03-26T17:54:58.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.133+0000 7fcb0bfff640 1 -- 172.21.15.105:0/3582863802 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18018880 con 0x7fcb241019b0 2024-03-26T17:54:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.136+0000 7fcb2bda7740 1 Processor -- start 2024-03-26T17:54:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.136+0000 7fcb2bda7740 1 -- start start 2024-03-26T17:54:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.136+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe54de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.136+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe552f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.136+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe553590 0x559cfe555a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.136+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559cfe4b2970 con 0x559cfe54e420 2024-03-26T17:54:58.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.136+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559cfe45ad70 con 0x559cfe553590 2024-03-26T17:54:58.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.136+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559cfe4ae920 con 0x559cfe54da30 2024-03-26T17:54:58.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe552f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe552f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50510/0 (socket says 172.21.15.105:50510) 2024-03-26T17:54:58.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb2359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe54de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 -- 172.21.15.105:0/1857227372 learned_addr learned my addr 172.21.15.105:0/1857227372 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb0bfff640 1 -- 172.21.15.105:0/3582863802 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcb18010a90 con 0x7fcb241019b0 2024-03-26T17:54:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 -- 172.21.15.105:0/1857227372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 msgr2=0x559cfe54de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/1857227372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe54de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 -- 172.21.15.105:0/1857227372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe553590 msgr2=0x559cfe555a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:54:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/1857227372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe553590 0x559cfe555a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 -- 172.21.15.105:0/1857227372 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cfe1c63f0 con 0x559cfe54e420 2024-03-26T17:54:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/1857227372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe552f20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcb14030800 tx=0x7fcb14005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d1e8c064cb9561 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb097fa640 1 -- 172.21.15.105:0/1857227372 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14047070 con 0x559cfe54e420 2024-03-26T17:54:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb097fa640 1 -- 172.21.15.105:0/1857227372 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb14003970 con 0x559cfe54e420 2024-03-26T17:54:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.137+0000 7fcb097fa640 1 -- 172.21.15.105:0/1857227372 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1403d040 con 0x559cfe54e420 2024-03-26T17:54:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb0bfff640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcb0c0a2fa0 0x7fcb0c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2359e640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcb0c0a2fa0 0x7fcb0c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb0bfff640 1 -- 172.21.15.105:0/3582863802 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fcb180cf6e0 con 0x7fcb241019b0 2024-03-26T17:54:58.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1857227372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 msgr2=0x559cfe552f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2359e640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcb0c0a2fa0 0x7fcb0c0a5460 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00bfd0 tx=0x7fcb1c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1857227372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe552f20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcb14030800 tx=0x7fcb14005ef0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1857227372 shutdown_connections 2024-03-26T17:54:58.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1857227372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe553590 0x559cfe555a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1857227372 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe552f20 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1857227372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe54de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1857227372 >> 172.21.15.105:0/1857227372 conn(0x559cfe548fa0 msgr2=0x559cfe5493a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1857227372 shutdown_connections 2024-03-26T17:54:58.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.138+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1857227372 wait complete. 2024-03-26T17:54:58.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 1 -- start start 2024-03-26T17:54:58.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 1 Processor -- start 2024-03-26T17:54:58.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe557fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe5584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559cfe5564e0 con 0x559cfe54e420 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559cfe46e310 con 0x559cfe54da30 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559cfe556660 con 0x559cfe553590 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2bda7740 10 client.0 _ll_register_callbacks cb 0x559cfe4a9c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe5584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe5584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50522/0 (socket says 172.21.15.105:50522) 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2259c640 1 -- 172.21.15.105:0/640560430 learned_addr learned my addr 172.21.15.105:0/640560430 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe557fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2359e640 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2259c640 1 -- 172.21.15.105:0/640560430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 msgr2=0x559cfe55f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2259c640 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2259c640 1 -- 172.21.15.105:0/640560430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 msgr2=0x559cfe557fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2259c640 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe557fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.139+0000 7fcb2259c640 1 -- 172.21.15.105:0/640560430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cfe2907d0 con 0x559cfe54e420 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb2259c640 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe5584f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fcb1809be10 tx=0x7fcb1809be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9c5e903398be4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb06ff5640 1 -- 172.21.15.105:0/640560430 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb180a4550 con 0x559cfe54e420 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb06ff5640 1 -- 172.21.15.105:0/640560430 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fcb18010430 con 0x559cfe54e420 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb2bda7740 1 -- 172.21.15.105:0/640560430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb14003970 con 0x559cfe54e420 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb06ff5640 1 -- 172.21.15.105:0/640560430 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcb18010630 con 0x559cfe54e420 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb2bda7740 10 client.21474 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb2bda7740 1 -- 172.21.15.105:0/640560430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559cfe562910 con 0x559cfe54e420 2024-03-26T17:54:58.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb06ff5640 1 -- 172.21.15.105:0/640560430 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb14003970 con 0x559cfe54e420 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb06ff5640 1 -- 172.21.15.105:0/640560430 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fcb18097c60 con 0x559cfe54e420 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb2bda7740 10 client.21474 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb2bda7740 10 client.21474 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.140+0000 7fcb2bda7740 20 client.21474 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.141+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559cfe563fe0 0x559cfe566440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.141+0000 7fcb2bda7740 4 client.21474 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.141+0000 7fcb2bda7740 1 -- 172.21.15.105:0/640560430 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cfe4bfeb0 con 0x559cfe563fe0 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.141+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559cfe563fe0 0x559cfe566440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.141+0000 7fcb06ff5640 10 client.21474 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.141+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559cfe563fe0 0x559cfe566440 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e3d70308e2b8fc7 server_cookie=9e627267a902b812 in_seq=0 out_seq=0 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb06ff5640 1 -- 172.21.15.105:0/640560430 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559cfe4bfeb0 con 0x559cfe563fe0 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb06ff5640 10 client.21474 handle_command_reply: tid=0 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 client.21474 shutdown 2024-03-26T17:54:58.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 -- 172.21.15.105:0/640560430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 msgr2=0x559cfe5584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe5584f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fcb1809be10 tx=0x7fcb1809be80 comp rx=0 tx=0).stop 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 -- 172.21.15.105:0/640560430 shutdown_connections 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559cfe563fe0 0x559cfe566440 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe5584f0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/640560430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe557fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb06ff5640 0 client.21474 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 -- 172.21.15.105:0/640560430 >> 172.21.15.105:0/640560430 conn(0x559cfe548fa0 msgr2=0x559cfe5481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 -- 172.21.15.105:0/640560430 shutdown_connections 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 1 -- 172.21.15.105:0/640560430 wait complete. 2024-03-26T17:54:58.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.142+0000 7fcb2bda7740 20 client.21474 trim_cache size 0 max 16384 2024-03-26T17:54:58.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:57 smithi105 ceph-mon[32280]: pgmap v847: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-03-26T17:54:58.143 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:57 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3520152441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:54:58.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.163+0000 7fcb2bda7740 1 Processor -- start 2024-03-26T17:54:58.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.163+0000 7fcb2bda7740 1 -- start start 2024-03-26T17:54:58.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.163+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe54f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.163+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe54ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.163+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.163+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241a6ca0 con 0x559cfe54e420 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.163+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241a87e0 con 0x559cfe54da30 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.163+0000 7fcb2bda7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241a7370 con 0x559cfe553590 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe54ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe54f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe54ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50524/0 (socket says 172.21.15.105:50524) 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 -- 172.21.15.105:0/1635221176 learned_addr learned my addr 172.21.15.105:0/1635221176 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe54f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39256/0 (socket says 172.21.15.105:39256) 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2359e640 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 -- 172.21.15.105:0/1635221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 msgr2=0x559cfe55f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 -- 172.21.15.105:0/1635221176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 msgr2=0x559cfe54f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe54f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 -- 172.21.15.105:0/1635221176 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cfe54ecb0 con 0x559cfe54e420 2024-03-26T17:54:58.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2359e640 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe54f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb2259c640 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe54ff30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fcb1809bb80 tx=0x7fcb180bc0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e021b6cb0a4c6fb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb08ff9640 1 -- 172.21.15.105:0/1635221176 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1807e030 con 0x559cfe54e420 2024-03-26T17:54:58.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb08ff9640 1 -- 172.21.15.105:0/1635221176 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb180b8070 con 0x559cfe54e420 2024-03-26T17:54:58.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.164+0000 7fcb08ff9640 1 -- 172.21.15.105:0/1635221176 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb180a4350 con 0x559cfe54e420 2024-03-26T17:54:58.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.165+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1635221176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 msgr2=0x559cfe54ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.165+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe54ff30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fcb1809bb80 tx=0x7fcb180bc0f0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1635221176 shutdown_connections 2024-03-26T17:54:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe553590 0x559cfe55f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe54e420 0x559cfe54ff30 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1635221176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54da30 0x559cfe54f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1635221176 >> 172.21.15.105:0/1635221176 conn(0x559cfe548fa0 msgr2=0x559cfe556a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1635221176 shutdown_connections 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1635221176 wait complete. 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 -- start start 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 Processor -- start 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.166+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe55c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54e420 0x559cfe55ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe55c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb22d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe55c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34870/0 (socket says 172.21.15.105:34870) 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb2259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54e420 0x559cfe55ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb2259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54e420 0x559cfe55ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39266/0 (socket says 172.21.15.105:39266) 2024-03-26T17:54:58.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb2bda7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe553590 0x559cfe55de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb22d9d640 1 -- 172.21.15.105:0/1645032373 learned_addr learned my addr 172.21.15.105:0/1645032373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb2359e640 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe553590 0x559cfe55de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241a8110 con 0x559cfe553590 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241a7a40 con 0x559cfe54e420 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.167+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb241a4c80 con 0x559cfe54da30 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb2bda7740 10 client.0 _ll_register_callbacks cb 0x7fcb18098120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb2359e640 1 -- 172.21.15.105:0/1645032373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 msgr2=0x559cfe55c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb2359e640 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe55c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb2359e640 1 -- 172.21.15.105:0/1645032373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54e420 msgr2=0x559cfe55ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb2359e640 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54e420 0x559cfe55ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb2359e640 1 -- 172.21.15.105:0/1645032373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cfe55e420 con 0x559cfe553590 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe55c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb2359e640 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe553590 0x559cfe55de30 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x559cfe54ec80 tx=0x7fcb1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e1fdf0d847cde4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:58.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb06ff5640 1 -- 172.21.15.105:0/1645032373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c031060 con 0x559cfe553590 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb06ff5640 1 -- 172.21.15.105:0/1645032373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fcb1c022680 con 0x559cfe553590 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.168+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb180b8070 con 0x559cfe553590 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb06ff5640 1 -- 172.21.15.105:0/1645032373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcb1c022880 con 0x559cfe553590 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb2bda7740 10 client.21480 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559cfe55f730 con 0x559cfe553590 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb06ff5640 1 -- 172.21.15.105:0/1645032373 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb180b8070 con 0x559cfe553590 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb06ff5640 1 -- 172.21.15.105:0/1645032373 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fcb1c037b10 con 0x559cfe553590 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb2bda7740 10 client.21480 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb2bda7740 10 client.21480 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb2bda7740 20 client.21480 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559cfe5ca9b0 0x559cfe5ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb2bda7740 4 client.21480 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cfe4ba200 con 0x559cfe5ca9b0 2024-03-26T17:54:58.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.169+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559cfe5ca9b0 0x559cfe5ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb06ff5640 10 client.21480 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb22d9d640 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559cfe5ca9b0 0x559cfe5ccdd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bfdcb113646afc21 server_cookie=b8d8d38be2f09544 in_seq=0 out_seq=0 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb06ff5640 1 -- 172.21.15.105:0/1645032373 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cfe4ba200 con 0x559cfe5ca9b0 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb06ff5640 10 client.21480 handle_command_reply: tid=0 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 client.21480 shutdown 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe553590 msgr2=0x559cfe55de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe553590 0x559cfe55de30 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x559cfe54ec80 tx=0x7fcb1c004730 comp rx=0 tx=0).stop 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 shutdown_connections 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x559cfe5ca9b0 0x559cfe5ccdd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559cfe553590 0x559cfe55de30 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cfe54e420 0x559cfe55ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 --2- 172.21.15.105:0/1645032373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559cfe54da30 0x559cfe55c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb06ff5640 0 client.21480 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:54:58.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 >> 172.21.15.105:0/1645032373 conn(0x559cfe548fa0 msgr2=0x559cfe557fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 shutdown_connections 2024-03-26T17:54:58.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 1 -- 172.21.15.105:0/1645032373 wait complete. 2024-03-26T17:54:58.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.170+0000 7fcb2bda7740 20 client.21480 trim_cache size 0 max 16384 2024-03-26T17:54:58.167 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 -- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcb0c0a2fa0 msgr2=0x7fcb0c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcb0c0a2fa0 0x7fcb0c0a5460 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00bfd0 tx=0x7fcb1c00d040 comp rx=0 tx=0).stop 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 -- 172.21.15.105:0/3582863802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241019b0 msgr2=0x7fcb24195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241019b0 0x7fcb24195e80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800e9f0 tx=0x7fcb1800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 -- 172.21.15.105:0/3582863802 shutdown_connections 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcb0c0a2fa0 0x7fcb0c0a5460 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcb24102370 0x7fcb2419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb241019b0 0x7fcb24195e80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 --2- 172.21.15.105:0/3582863802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb241007c0 0x7fcb24195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 -- 172.21.15.105:0/3582863802 >> 172.21.15.105:0/3582863802 conn(0x7fcb240fbd50 msgr2=0x7fcb24104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 -- 172.21.15.105:0/3582863802 shutdown_connections 2024-03-26T17:54:58.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.171+0000 7fcb290dc640 1 -- 172.21.15.105:0/3582863802 wait complete. 2024-03-26T17:54:58.170 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:57 smithi135 ceph-mon[35636]: pgmap v847: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-03-26T17:54:58.170 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:57 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3520152441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:54:58.172 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 -- 172.21.15.105:0/1040773128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec102250 msgr2=0x7f97ec108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 --2- 172.21.15.105:0/1040773128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec102250 0x7f97ec108ae0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009ca0 tx=0x7f97e402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 -- 172.21.15.105:0/1040773128 shutdown_connections 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 --2- 172.21.15.105:0/1040773128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec102250 0x7f97ec108ae0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 --2- 172.21.15.105:0/1040773128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec101890 0x7f97ec101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 --2- 172.21.15.105:0/1040773128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec1006a0 0x7f97ec100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 -- 172.21.15.105:0/1040773128 >> 172.21.15.105:0/1040773128 conn(0x7f97ec0fbc30 msgr2=0x7f97ec0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 -- 172.21.15.105:0/1040773128 shutdown_connections 2024-03-26T17:54:58.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.326+0000 7f97f09fe640 1 -- 172.21.15.105:0/1040773128 wait complete. 2024-03-26T17:54:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.327+0000 7f97f09fe640 1 Processor -- start 2024-03-26T17:54:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.327+0000 7f97f09fe640 1 -- start start 2024-03-26T17:54:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.327+0000 7f97f09fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1006a0 0x7f97ec197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.327+0000 7f97f09fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec101890 0x7f97ec197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.327+0000 7f97f09fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec102250 0x7f97ec19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.327+0000 7f97f09fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec10c1d0 con 0x7f97ec1006a0 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97f09fe640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec10c050 con 0x7f97ec102250 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97f09fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec10c350 con 0x7f97ec101890 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1006a0 0x7f97ec197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1006a0 0x7f97ec197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50568/0 (socket says 172.21.15.105:50568) 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 -- 172.21.15.105:0/2387674362 learned_addr learned my addr 172.21.15.105:0/2387674362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97e3fff640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec101890 0x7f97ec197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97eb59e640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec102250 0x7f97ec19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 -- 172.21.15.105:0/2387674362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec101890 msgr2=0x7f97ec197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec101890 0x7f97ec197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 -- 172.21.15.105:0/2387674362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec102250 msgr2=0x7f97ec19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec102250 0x7f97ec19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 -- 172.21.15.105:0/2387674362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec19f580 con 0x7f97ec1006a0 2024-03-26T17:54:58.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97e3fff640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec101890 0x7f97ec197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97ead9d640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1006a0 0x7f97ec197960 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009f00 tx=0x7f97dc00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.328+0000 7f97e8d99640 1 -- 172.21.15.105:0/2387674362 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00ccc0 con 0x7f97ec1006a0 2024-03-26T17:54:58.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.329+0000 7f97f09fe640 1 -- 172.21.15.105:0/2387674362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ec102b10 con 0x7f97ec1006a0 2024-03-26T17:54:58.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.329+0000 7f97e8d99640 1 -- 172.21.15.105:0/2387674362 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97dc0137d0 con 0x7f97ec1006a0 2024-03-26T17:54:58.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.329+0000 7f97e8d99640 1 -- 172.21.15.105:0/2387674362 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc003850 con 0x7f97ec1006a0 2024-03-26T17:54:58.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.329+0000 7f97f09fe640 1 -- 172.21.15.105:0/2387674362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ec103060 con 0x7f97ec1006a0 2024-03-26T17:54:58.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.331+0000 7f97e8d99640 1 -- 172.21.15.105:0/2387674362 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f97dc013980 con 0x7f97ec1006a0 2024-03-26T17:54:58.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.332+0000 7f97e8d99640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d80a2f20 0x7f97d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.332+0000 7f97f3deb740 1 Processor -- start 2024-03-26T17:54:58.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.332+0000 7f97e8d99640 1 -- 172.21.15.105:0/2387674362 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97dc0d37f0 con 0x7f97ec1006a0 2024-03-26T17:54:58.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.332+0000 7f97e3fff640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d80a2f20 0x7f97d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.332+0000 7f97f3deb740 1 -- start start 2024-03-26T17:54:58.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee50221060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50226130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50228c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97f3deb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ee50185970 con 0x55ee50220c40 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97f3deb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ee5012dd70 con 0x55ee50221630 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97f3deb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee50181920 con 0x55ee502267a0 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee50221060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97e3fff640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d80a2f20 0x7f97d80a53e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f97ec198e70 tx=0x7f97d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee50221060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50570/0 (socket says 172.21.15.105:50570) 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97ead9d640 1 -- 172.21.15.105:0/3107854299 learned_addr learned my addr 172.21.15.105:0/3107854299 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97eb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50228c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97e3fff640 1 --2- 172.21.15.105:0/3107854299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50226130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97eb59e640 1 -- 172.21.15.105:0/3107854299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 msgr2=0x55ee50226130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97eb59e640 1 --2- 172.21.15.105:0/3107854299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50226130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97eb59e640 1 -- 172.21.15.105:0/3107854299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 msgr2=0x55ee50221060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97eb59e640 1 --2- 172.21.15.105:0/3107854299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee50221060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97eb59e640 1 -- 172.21.15.105:0/3107854299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4fe993f0 con 0x55ee502267a0 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97eb59e640 1 --2- 172.21.15.105:0/3107854299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50228c30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f97e4037040 tx=0x7f97e4005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ac5ee73a675e6b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.333+0000 7f97e17fa640 1 -- 172.21.15.105:0/3107854299 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4043070 con 0x55ee502267a0 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.334+0000 7f97e17fa640 1 -- 172.21.15.105:0/3107854299 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97e4032db0 con 0x55ee502267a0 2024-03-26T17:54:58.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.334+0000 7f97e17fa640 1 -- 172.21.15.105:0/3107854299 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4039040 con 0x55ee502267a0 2024-03-26T17:54:58.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.334+0000 7f97f3deb740 1 -- 172.21.15.105:0/3107854299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 msgr2=0x55ee50228c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.334+0000 7f97f3deb740 1 --2- 172.21.15.105:0/3107854299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50228c30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f97e4037040 tx=0x7f97e4005f80 comp rx=0 tx=0).stop 2024-03-26T17:54:58.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 -- 172.21.15.105:0/3107854299 shutdown_connections 2024-03-26T17:54:58.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 --2- 172.21.15.105:0/3107854299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50228c30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 --2- 172.21.15.105:0/3107854299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50226130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 --2- 172.21.15.105:0/3107854299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee50221060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 -- 172.21.15.105:0/3107854299 >> 172.21.15.105:0/3107854299 conn(0x55ee5021c1b0 msgr2=0x55ee5021c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 -- 172.21.15.105:0/3107854299 shutdown_connections 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 -- 172.21.15.105:0/3107854299 wait complete. 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 -- start start 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.335+0000 7f97f3deb740 1 Processor -- start 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50220c40 0x55ee5022b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50221630 0x55ee5022b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50232640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97f3deb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ee502296f0 con 0x55ee50221630 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97f3deb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ee50141310 con 0x55ee50220c40 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97f3deb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee50229870 con 0x55ee502267a0 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50220c40 0x55ee5022b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50221630 0x55ee5022b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50220c40 0x55ee5022b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39298/0 (socket says 172.21.15.105:39298) 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97ead9d640 1 -- 172.21.15.105:0/1426569455 learned_addr learned my addr 172.21.15.105:0/1426569455 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50221630 0x55ee5022b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50580/0 (socket says 172.21.15.105:50580) 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97eb59e640 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50232640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97f3deb740 10 client.0 _ll_register_callbacks cb 0x55ee5017cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97ead9d640 1 -- 172.21.15.105:0/1426569455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 msgr2=0x55ee50232640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97ead9d640 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50232640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97ead9d640 1 -- 172.21.15.105:0/1426569455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50221630 msgr2=0x55ee5022b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97ead9d640 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50221630 0x55ee5022b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97ead9d640 1 -- 172.21.15.105:0/1426569455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4ff637d0 con 0x55ee50220c40 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97eb59e640 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50232640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.336+0000 7f97e3fff640 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50221630 0x55ee5022b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97ead9d640 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50220c40 0x55ee5022b0c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0d1620 tx=0x7f97dc09fe80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b48147305a3ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:54:58.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97d27fc640 1 -- 172.21.15.105:0/1426569455 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0892e0 con 0x55ee50220c40 2024-03-26T17:54:58.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97d27fc640 1 -- 172.21.15.105:0/1426569455 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97dc09f3e0 con 0x55ee50220c40 2024-03-26T17:54:58.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97f3deb740 1 -- 172.21.15.105:0/1426569455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e4032db0 con 0x55ee50220c40 2024-03-26T17:54:58.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97d27fc640 1 -- 172.21.15.105:0/1426569455 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97dc0a88f0 con 0x55ee50220c40 2024-03-26T17:54:58.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97f3deb740 10 client.31256 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97f3deb740 1 -- 172.21.15.105:0/1426569455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ee50235a20 con 0x55ee50220c40 2024-03-26T17:54:58.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.337+0000 7f97d27fc640 1 -- 172.21.15.105:0/1426569455 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97dc0c00c0 con 0x55ee50220c40 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.338+0000 7f97d27fc640 1 -- 172.21.15.105:0/1426569455 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f97dc0a51c0 con 0x55ee50220c40 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.338+0000 7f97f3deb740 10 client.31256 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.338+0000 7f97f3deb740 10 client.31256 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.338+0000 7f97f3deb740 20 client.31256 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.338+0000 7f97f3deb740 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ee50237080 0x55ee50239510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.338+0000 7f97f3deb740 4 client.31256 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.338+0000 7f97f3deb740 1 -- 172.21.15.105:0/1426569455 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f97e4032db0 con 0x55ee50237080 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.338+0000 7f97e3fff640 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ee50237080 0x55ee50239510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97d27fc640 10 client.31256 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97e3fff640 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ee50237080 0x55ee50239510 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d349731da92aa10 server_cookie=c99e7545ddd41789 in_seq=0 out_seq=0 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97d27fc640 1 -- 172.21.15.105:0/1426569455 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f97e4032db0 con 0x55ee50237080 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97d27fc640 10 client.31256 handle_command_reply: tid=0 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97f3deb740 1 client.31256 shutdown 2024-03-26T17:54:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97f3deb740 1 -- 172.21.15.105:0/1426569455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50220c40 msgr2=0x55ee5022b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97f3deb740 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50220c40 0x55ee5022b0c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0d1620 tx=0x7f97dc09fe80 comp rx=0 tx=0).stop 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97f3deb740 1 -- 172.21.15.105:0/1426569455 shutdown_connections 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97f3deb740 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50232640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.339+0000 7f97f3deb740 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50221630 0x55ee5022b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.340+0000 7f97f3deb740 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ee50237080 0x55ee50239510 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.340+0000 7f97f3deb740 1 --2- 172.21.15.105:0/1426569455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50220c40 0x55ee5022b0c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.340+0000 7f97f3deb740 1 -- 172.21.15.105:0/1426569455 >> 172.21.15.105:0/1426569455 conn(0x55ee5021c1b0 msgr2=0x55ee5021b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.340+0000 7f97d27fc640 0 client.31256 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.340+0000 7f97f3deb740 1 -- 172.21.15.105:0/1426569455 shutdown_connections 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.340+0000 7f97f3deb740 1 -- 172.21.15.105:0/1426569455 wait complete. 2024-03-26T17:54:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.340+0000 7f97f3deb740 20 client.31256 trim_cache size 0 max 16384 2024-03-26T17:54:58.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:57 smithi144 ceph-mon[36095]: pgmap v847: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-03-26T17:54:58.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:57 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3520152441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:54:58.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97f3deb740 1 Processor -- start 2024-03-26T17:54:58.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97f3deb740 1 -- start start 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50232600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee5022b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97f3deb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a69c0 con 0x55ee50220c40 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97f3deb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a8500 con 0x55ee50221630 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97f3deb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a7090 con 0x55ee502267a0 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50232600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97eb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee5022b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50232600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39306/0 (socket says 172.21.15.105:39306) 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50582/0 (socket says 172.21.15.105:50582) 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97e3fff640 1 -- 172.21.15.105:0/4033876356 learned_addr learned my addr 172.21.15.105:0/4033876356 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.359+0000 7f97e3fff640 1 -- 172.21.15.105:0/4033876356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 msgr2=0x55ee5022b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97e3fff640 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee5022b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97e3fff640 1 -- 172.21.15.105:0/4033876356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 msgr2=0x55ee5022acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97e3fff640 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97e3fff640 1 -- 172.21.15.105:0/4033876356 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5021cf20 con 0x55ee50221630 2024-03-26T17:54:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97eb59e640 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee5022b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97ead9d640 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97e3fff640 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50232600 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f97d4013010 tx=0x7f97d4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c549519d8c1c301 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97e0ff9640 1 -- 172.21.15.105:0/4033876356 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4031070 con 0x55ee50221630 2024-03-26T17:54:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97e0ff9640 1 -- 172.21.15.105:0/4033876356 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d400fd80 con 0x55ee50221630 2024-03-26T17:54:58.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.360+0000 7f97e0ff9640 1 -- 172.21.15.105:0/4033876356 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d401d3f0 con 0x55ee50221630 2024-03-26T17:54:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 -- 172.21.15.105:0/4033876356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 msgr2=0x55ee50232600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50232600 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f97d4013010 tx=0x7f97d4014000 comp rx=0 tx=0).stop 2024-03-26T17:54:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 -- 172.21.15.105:0/4033876356 shutdown_connections 2024-03-26T17:54:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee5022b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee50232600 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 --2- 172.21.15.105:0/4033876356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 -- 172.21.15.105:0/4033876356 >> 172.21.15.105:0/4033876356 conn(0x55ee5021c1b0 msgr2=0x55ee5021c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 -- 172.21.15.105:0/4033876356 shutdown_connections 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 -- 172.21.15.105:0/4033876356 wait complete. 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 -- start start 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 Processor -- start 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee5022fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50230e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.361+0000 7f97f3deb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a7e30 con 0x55ee50220c40 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97f3deb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a7760 con 0x55ee50221630 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97f3deb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1a49a0 con 0x55ee502267a0 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee5022fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97f3deb740 10 client.0 _ll_register_callbacks cb 0x7f97dc09f8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50590/0 (socket says 172.21.15.105:50590) 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 -- 172.21.15.105:0/2440990171 learned_addr learned my addr 172.21.15.105:0/2440990171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97eb59e640 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50230e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 -- 172.21.15.105:0/2440990171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 msgr2=0x55ee50230e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50230e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 -- 172.21.15.105:0/2440990171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 msgr2=0x55ee5022fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee5022fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 -- 172.21.15.105:0/2440990171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee50231460 con 0x55ee50220c40 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97ead9d640 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022f920 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0a5580 tx=0x7f97dc0d1300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5996b868065ca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97d27fc640 1 -- 172.21.15.105:0/2440990171 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc09ff90 con 0x55ee50220c40 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97d27fc640 1 -- 172.21.15.105:0/2440990171 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f97dc0890b0 con 0x55ee50220c40 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.362+0000 7f97f3deb740 1 -- 172.21.15.105:0/2440990171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d400fd80 con 0x55ee50220c40 2024-03-26T17:54:58.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97d27fc640 1 -- 172.21.15.105:0/2440990171 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97dc0892b0 con 0x55ee50220c40 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97f3deb740 10 client.21498 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97f3deb740 1 -- 172.21.15.105:0/2440990171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ee502327d0 con 0x55ee50220c40 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97d27fc640 1 -- 172.21.15.105:0/2440990171 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d400fd80 con 0x55ee50220c40 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97d27fc640 1 -- 172.21.15.105:0/2440990171 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f97dc09bce0 con 0x55ee50220c40 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97f3deb740 10 client.21498 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97f3deb740 10 client.21498 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97f3deb740 20 client.21498 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97f3deb740 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ee5029dab0 0x55ee5029fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97f3deb740 4 client.21498 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97f3deb740 1 -- 172.21.15.105:0/2440990171 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee50192eb0 con 0x55ee5029dab0 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.363+0000 7f97e3fff640 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ee5029dab0 0x55ee5029fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97d27fc640 10 client.21498 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97e3fff640 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ee5029dab0 0x55ee5029fed0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=807aa278e498efa9 server_cookie=2f14f95957a993f7 in_seq=0 out_seq=0 2024-03-26T17:54:58.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97d27fc640 1 -- 172.21.15.105:0/2440990171 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee50192eb0 con 0x55ee5029dab0 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97d27fc640 10 client.21498 handle_command_reply: tid=0 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 client.21498 shutdown 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 -- 172.21.15.105:0/2440990171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 msgr2=0x55ee5022f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022f920 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0a5580 tx=0x7f97dc0d1300 comp rx=0 tx=0).stop 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 -- 172.21.15.105:0/2440990171 shutdown_connections 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee502267a0 0x55ee50230e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55ee5029dab0 0x55ee5029fed0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ee50221630 0x55ee5022fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 --2- 172.21.15.105:0/2440990171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ee50220c40 0x55ee5022f920 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97d27fc640 0 client.21498 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 -- 172.21.15.105:0/2440990171 >> 172.21.15.105:0/2440990171 conn(0x55ee5021c1b0 msgr2=0x55ee5022ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 -- 172.21.15.105:0/2440990171 shutdown_connections 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 1 -- 172.21.15.105:0/2440990171 wait complete. 2024-03-26T17:54:58.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.364+0000 7f97f3deb740 20 client.21498 trim_cache size 0 max 16384 2024-03-26T17:54:58.359 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 -- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d80a2f20 msgr2=0x7f97d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d80a2f20 0x7f97d80a53e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f97ec198e70 tx=0x7f97d4005f90 comp rx=0 tx=0).stop 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 -- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1006a0 msgr2=0x7f97ec197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1006a0 0x7f97ec197960 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009f00 tx=0x7f97dc00ed90 comp rx=0 tx=0).stop 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 -- 172.21.15.105:0/2387674362 shutdown_connections 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97d80a2f20 0x7f97d80a53e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97ec102250 0x7f97ec19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97ec101890 0x7f97ec197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 --2- 172.21.15.105:0/2387674362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97ec1006a0 0x7f97ec197960 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 -- 172.21.15.105:0/2387674362 >> 172.21.15.105:0/2387674362 conn(0x7f97ec0fbc30 msgr2=0x7f97ec0fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 -- 172.21.15.105:0/2387674362 shutdown_connections 2024-03-26T17:54:58.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.365+0000 7f97f09fe640 1 -- 172.21.15.105:0/2387674362 wait complete. 2024-03-26T17:54:58.368 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:54:58.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.523+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3243252529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac1018f0 msgr2=0x7f25ac101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.523+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3243252529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac1018f0 0x7f25ac101d10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009900 tx=0x7f25a002bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:58.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.524+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3243252529 shutdown_connections 2024-03-26T17:54:58.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.524+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3243252529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ac1034a0 0x7f25ac109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.524+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3243252529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25ac102ae0 0x7f25ac102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.524+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3243252529 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac1018f0 0x7f25ac101d10 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.524+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3243252529 >> 172.21.15.105:0/3243252529 conn(0x7f25ac0fce60 msgr2=0x7f25ac0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.524+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3243252529 shutdown_connections 2024-03-26T17:54:58.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.524+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3243252529 wait complete. 2024-03-26T17:54:58.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.524+0000 7f25b2bcd640 1 Processor -- start 2024-03-26T17:54:58.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b2bcd640 1 -- start start 2024-03-26T17:54:58.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b2bcd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25ac1018f0 0x7f25ac197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b2bcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac102ae0 0x7f25ac198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b2bcd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ac1034a0 0x7f25ac19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac102ae0 0x7f25ac198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b0969640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25ac1018f0 0x7f25ac197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b0969640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25ac1018f0 0x7f25ac197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39350/0 (socket says 172.21.15.105:39350) 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac102ae0 0x7f25ac198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50600/0 (socket says 172.21.15.105:50600) 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b0969640 1 -- 172.21.15.105:0/3867234760 learned_addr learned my addr 172.21.15.105:0/3867234760 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac10d3f0 con 0x7f25ac102ae0 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac10d270 con 0x7f25ac1018f0 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac10d570 con 0x7f25ac1034a0 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b116a640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ac1034a0 0x7f25ac19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b0969640 1 -- 172.21.15.105:0/3867234760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ac1034a0 msgr2=0x7f25ac19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b0969640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ac1034a0 0x7f25ac19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b0969640 1 -- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac102ae0 msgr2=0x7f25ac198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b0969640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac102ae0 0x7f25ac198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25b0969640 1 -- 172.21.15.105:0/3867234760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac19f750 con 0x7f25ac1018f0 2024-03-26T17:54:58.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.525+0000 7f25abfff640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac102ae0 0x7f25ac198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.526+0000 7f25b0969640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25ac1018f0 0x7f25ac197b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f25a0000c00 tx=0x7f25a0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.526+0000 7f25a9ffb640 1 -- 172.21.15.105:0/3867234760 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0043020 con 0x7f25ac1018f0 2024-03-26T17:54:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.526+0000 7f25a9ffb640 1 -- 172.21.15.105:0/3867234760 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25a0004350 con 0x7f25ac1018f0 2024-03-26T17:54:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.526+0000 7f25a9ffb640 1 -- 172.21.15.105:0/3867234760 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0003b80 con 0x7f25ac1018f0 2024-03-26T17:54:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.526+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac19f9e0 con 0x7f25ac1018f0 2024-03-26T17:54:58.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.526+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac19fee0 con 0x7f25ac1018f0 2024-03-26T17:54:58.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.529+0000 7f25a9ffb640 1 -- 172.21.15.105:0/3867234760 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f25a0046020 con 0x7f25ac1018f0 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.529+0000 7f25b57ba740 1 Processor -- start 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.529+0000 7f25b57ba740 1 -- start start 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981682e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 0x557981687f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798168aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b57ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5579815e7970 con 0x557981682a30 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b57ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55798158fd70 con 0x557981688590 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b57ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5579815e3920 con 0x557981683420 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b116a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981682e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b116a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981682e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50616/0 (socket says 172.21.15.105:50616) 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b116a640 1 -- 172.21.15.105:0/2720309316 learned_addr learned my addr 172.21.15.105:0/2720309316 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b0969640 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 0x557981687f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25abfff640 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798168aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b116a640 1 -- 172.21.15.105:0/2720309316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 msgr2=0x557981687f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b116a640 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 0x557981687f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25a9ffb640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f259c0a2f20 0x7f259c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b116a640 1 -- 172.21.15.105:0/2720309316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 msgr2=0x55798168aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b116a640 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798168aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25b116a640 1 -- 172.21.15.105:0/2720309316 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579812fb3f0 con 0x557981682a30 2024-03-26T17:54:58.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25abfff640 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798168aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:54:58.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.530+0000 7f25a9ffb640 1 -- 172.21.15.105:0/3867234760 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f25a00f1580 con 0x7f25ac1018f0 2024-03-26T17:54:58.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.531+0000 7f25b116a640 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981682e50 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009fd0 tx=0x7f25a40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0ec2cd07f6ed32 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.531+0000 7f25b116a640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f259c0a2f20 0x7f259c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.531+0000 7f2596ffd640 1 -- 172.21.15.105:0/2720309316 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4003ab0 con 0x557981682a30 2024-03-26T17:54:58.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.531+0000 7f2596ffd640 1 -- 172.21.15.105:0/2720309316 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25a401a070 con 0x557981682a30 2024-03-26T17:54:58.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.531+0000 7f2596ffd640 1 -- 172.21.15.105:0/2720309316 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a40169e0 con 0x557981682a30 2024-03-26T17:54:58.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b116a640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f259c0a2f20 0x7f259c0a53e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f25a4016fd0 tx=0x7f25a4016c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b57ba740 1 -- 172.21.15.105:0/2720309316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 msgr2=0x557981682e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b57ba740 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981682e50 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009fd0 tx=0x7f25a40118d0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b57ba740 1 -- 172.21.15.105:0/2720309316 shutdown_connections 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b57ba740 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798168aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b57ba740 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 0x557981687f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b57ba740 1 --2- 172.21.15.105:0/2720309316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981682e50 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b57ba740 1 -- 172.21.15.105:0/2720309316 >> 172.21.15.105:0/2720309316 conn(0x55798167dfa0 msgr2=0x55798167e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.532+0000 7f25b57ba740 1 -- 172.21.15.105:0/2720309316 shutdown_connections 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.533+0000 7f25b57ba740 1 -- 172.21.15.105:0/2720309316 wait complete. 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.533+0000 7f25b57ba740 1 -- start start 2024-03-26T17:54:58.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.533+0000 7f25b57ba740 1 Processor -- start 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.533+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.533+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 0x557981695c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798169cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b57ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55798168b540 con 0x557981682a30 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50632/0 (socket says 172.21.15.105:50632) 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b57ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5579815a3310 con 0x557981688590 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b57ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55798168b6c0 con 0x557981683420 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 -- 172.21.15.105:0/4003414804 learned_addr learned my addr 172.21.15.105:0/4003414804 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25abfff640 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 0x557981695c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b57ba740 10 client.0 _ll_register_callbacks cb 0x5579815dec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b116a640 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798169cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 -- 172.21.15.105:0/4003414804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 msgr2=0x557981695c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 0x557981695c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 -- 172.21.15.105:0/4003414804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 msgr2=0x55798169cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798169cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 -- 172.21.15.105:0/4003414804 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579813c57d0 con 0x557981682a30 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f25b0969640 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695720 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006010 tx=0x7f25a0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b820fb1c1a15fcb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.534+0000 7f258bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f258bfff640 1 -- 172.21.15.105:0/4003414804 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a00bd7b0 con 0x557981682a30 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f25b57ba740 1 -- 172.21.15.105:0/4003414804 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25a401a070 con 0x557981682a30 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f258bfff640 1 -- 172.21.15.105:0/4003414804 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f25a0003d20 con 0x557981682a30 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f258bfff640 1 -- 172.21.15.105:0/4003414804 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f25a00bd5d0 con 0x557981682a30 2024-03-26T17:54:58.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f25b57ba740 10 client.21510 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f25b57ba740 1 -- 172.21.15.105:0/4003414804 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55798169ff80 con 0x557981682a30 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f258bfff640 1 -- 172.21.15.105:0/4003414804 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25a401a070 con 0x557981682a30 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f258bfff640 1 -- 172.21.15.105:0/4003414804 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f25a00c61f0 con 0x557981682a30 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f25b57ba740 10 client.21510 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.535+0000 7f25b57ba740 10 client.21510 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.536+0000 7f25b57ba740 20 client.21510 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.536+0000 7f25b57ba740 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5579816a1780 0x5579816a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.536+0000 7f25b57ba740 4 client.21510 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.536+0000 7f25b57ba740 1 -- 172.21.15.105:0/4003414804 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579815f4eb0 con 0x5579816a1780 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.536+0000 7f25abfff640 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5579816a1780 0x5579816a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.536+0000 7f25abfff640 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5579816a1780 0x5579816a3c40 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61a1211eac87dab4 server_cookie=fb2f432acb0c3f38 in_seq=0 out_seq=0 2024-03-26T17:54:58.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.536+0000 7f258bfff640 10 client.21510 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:54:58.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f258bfff640 1 -- 172.21.15.105:0/4003414804 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5579815f4eb0 con 0x5579816a1780 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f258bfff640 10 client.21510 handle_command_reply: tid=0 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f25b57ba740 1 client.21510 shutdown 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f25b57ba740 1 -- 172.21.15.105:0/4003414804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 msgr2=0x557981695720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f25b57ba740 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695720 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f25a0006010 tx=0x7f25a0009560 comp rx=0 tx=0).stop 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f25b57ba740 1 -- 172.21.15.105:0/4003414804 shutdown_connections 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f25b57ba740 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5579816a1780 0x5579816a3c40 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f25b57ba740 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981688590 0x55798169cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f25b57ba740 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981683420 0x557981695c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.537+0000 7f25b57ba740 1 --2- 172.21.15.105:0/4003414804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695720 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.538+0000 7f25b57ba740 1 -- 172.21.15.105:0/4003414804 >> 172.21.15.105:0/4003414804 conn(0x55798167dfa0 msgr2=0x55798167d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.538+0000 7f258bfff640 0 client.21510 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.538+0000 7f25b57ba740 1 -- 172.21.15.105:0/4003414804 shutdown_connections 2024-03-26T17:54:58.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.538+0000 7f25b57ba740 1 -- 172.21.15.105:0/4003414804 wait complete. 2024-03-26T17:54:58.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.538+0000 7f25b57ba740 20 client.21510 trim_cache size 0 max 16384 2024-03-26T17:54:58.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b57ba740 1 Processor -- start 2024-03-26T17:54:58.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b57ba740 1 -- start start 2024-03-26T17:54:58.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 0x55798167c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b57ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac1a6b60 con 0x557981682a30 2024-03-26T17:54:58.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b57ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac1a86a0 con 0x557981683420 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b57ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac1a7230 con 0x557981688590 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b0969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b0969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50644/0 (socket says 172.21.15.105:50644) 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b0969640 1 -- 172.21.15.105:0/3891118439 learned_addr learned my addr 172.21.15.105:0/3891118439 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25abfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.557+0000 7f25b116a640 1 --2- 172.21.15.105:0/3891118439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 0x55798167c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25b0969640 1 -- 172.21.15.105:0/3891118439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 msgr2=0x55798167c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25b0969640 1 --2- 172.21.15.105:0/3891118439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 0x55798167c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25b0969640 1 -- 172.21.15.105:0/3891118439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 msgr2=0x55798169cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25b0969640 1 --2- 172.21.15.105:0/3891118439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25b0969640 1 -- 172.21.15.105:0/3891118439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798167d8f0 con 0x557981682a30 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25b0969640 1 --2- 172.21.15.105:0/3891118439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695a40 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f25a00ba640 tx=0x7f25a00c1dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be345510c2ab1a35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25967fc640 1 -- 172.21.15.105:0/3891118439 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a00bd7b0 con 0x557981682a30 2024-03-26T17:54:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25967fc640 1 -- 172.21.15.105:0/3891118439 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25a00be370 con 0x557981682a30 2024-03-26T17:54:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.558+0000 7f25967fc640 1 -- 172.21.15.105:0/3891118439 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a00b9f10 con 0x557981682a30 2024-03-26T17:54:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 -- 172.21.15.105:0/3891118439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 msgr2=0x557981695a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 --2- 172.21.15.105:0/3891118439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695a40 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f25a00ba640 tx=0x7f25a00c1dc0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 -- 172.21.15.105:0/3891118439 shutdown_connections 2024-03-26T17:54:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 --2- 172.21.15.105:0/3891118439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 0x55798167c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 --2- 172.21.15.105:0/3891118439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 --2- 172.21.15.105:0/3891118439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x557981695a40 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 -- 172.21.15.105:0/3891118439 >> 172.21.15.105:0/3891118439 conn(0x55798167dfa0 msgr2=0x557981695330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 -- 172.21.15.105:0/3891118439 shutdown_connections 2024-03-26T17:54:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 -- 172.21.15.105:0/3891118439 wait complete. 2024-03-26T17:54:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.559+0000 7f25b57ba740 1 -- start start 2024-03-26T17:54:58.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b57ba740 1 Processor -- start 2024-03-26T17:54:58.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x55798169a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b0969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x55798169a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b0969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x55798169a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50654/0 (socket says 172.21.15.105:50654) 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b57ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b0969640 1 -- 172.21.15.105:0/571225951 learned_addr learned my addr 172.21.15.105:0/571225951 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b57ba740 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 0x55798169bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac1a7fd0 con 0x557981682a30 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac1a7900 con 0x557981683420 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac1a4b40 con 0x557981688590 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b57ba740 10 client.0 _ll_register_callbacks cb 0x7f25a00057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b116a640 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 0x55798169bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25abfff640 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b0969640 1 -- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 msgr2=0x55798169bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b0969640 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 0x55798169bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.560+0000 7f25b0969640 1 -- 172.21.15.105:0/571225951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 msgr2=0x55798169abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f25b0969640 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f25b0969640 1 -- 172.21.15.105:0/571225951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798169c1c0 con 0x557981682a30 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f25abfff640 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f25b0969640 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x55798169a680 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f25a00ef120 tx=0x7f25a0003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d79fc92718259bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f258bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f258bfff640 1 -- 172.21.15.105:0/571225951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a00c66d0 con 0x557981682a30 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f258bfff640 1 -- 172.21.15.105:0/571225951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f25a009cae0 con 0x557981682a30 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25a00be370 con 0x557981682a30 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f258bfff640 1 -- 172.21.15.105:0/571225951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f25a00b9f10 con 0x557981682a30 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f25b57ba740 10 client.21516 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.561+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55798169c1c0 con 0x557981682a30 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f258bfff640 1 -- 172.21.15.105:0/571225951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25a00be370 con 0x557981682a30 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f258bfff640 1 -- 172.21.15.105:0/571225951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f25a00c2f50 con 0x557981682a30 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f25b57ba740 10 client.21516 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f25b57ba740 10 client.21516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:54:58.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f25b57ba740 20 client.21516 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f25b57ba740 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557981708310 0x55798170a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f25b57ba740 4 client.21516 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:58.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579815ef200 con 0x557981708310 2024-03-26T17:54:58.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.562+0000 7f25abfff640 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557981708310 0x55798170a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.563+0000 7f25abfff640 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557981708310 0x55798170a700 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2d4fbdece770227 server_cookie=21abed44b61d4923 in_seq=0 out_seq=0 2024-03-26T17:54:58.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.563+0000 7f258bfff640 10 client.21516 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:54:58.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 29133 7.15 MB/sec execute 101 sec latency 121.113 ms 2024-03-26T17:54:58.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.563+0000 7f258bfff640 1 -- 172.21.15.105:0/571225951 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579815ef200 con 0x557981708310 2024-03-26T17:54:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.563+0000 7f258bfff640 10 client.21516 handle_command_reply: tid=0 2024-03-26T17:54:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.563+0000 7f25b57ba740 1 client.21516 shutdown 2024-03-26T17:54:58.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 msgr2=0x55798169a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x55798169a680 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f25a00ef120 tx=0x7f25a0003f50 comp rx=0 tx=0).stop 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 shutdown_connections 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x557981708310 0x55798170a700 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557981688590 0x55798169bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557981683420 0x55798169abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f258bfff640 0 client.21516 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 --2- 172.21.15.105:0/571225951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557981682a30 0x55798169a680 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 >> 172.21.15.105:0/571225951 conn(0x55798167dfa0 msgr2=0x55798167c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 shutdown_connections 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 1 -- 172.21.15.105:0/571225951 wait complete. 2024-03-26T17:54:58.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.564+0000 7f25b57ba740 20 client.21516 trim_cache size 0 max 16384 2024-03-26T17:54:58.560 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f259c0a2f20 msgr2=0x7f259c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f259c0a2f20 0x7f259c0a53e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f25a4016fd0 tx=0x7f25a4016c80 comp rx=0 tx=0).stop 2024-03-26T17:54:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25ac1018f0 msgr2=0x7f25ac197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25ac1018f0 0x7f25ac197b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f25a0000c00 tx=0x7f25a0005dc0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 shutdown_connections 2024-03-26T17:54:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f259c0a2f20 0x7f259c0a53e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25ac1034a0 0x7f25ac19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25ac102ae0 0x7f25ac198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 --2- 172.21.15.105:0/3867234760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25ac1018f0 0x7f25ac197b30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 >> 172.21.15.105:0/3867234760 conn(0x7f25ac0fce60 msgr2=0x7f25ac105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 shutdown_connections 2024-03-26T17:54:58.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.565+0000 7f25b2bcd640 1 -- 172.21.15.105:0/3867234760 wait complete. 2024-03-26T17:54:58.568 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 -- 172.21.15.105:0/1976638290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a101023a0 msgr2=0x7f8a10104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/1976638290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a101023a0 0x7f8a10104790 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009900 tx=0x7f8a0402bc30 comp rx=0 tx=0).stop 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 -- 172.21.15.105:0/1976638290 shutdown_connections 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/1976638290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a10107600 0x7f8a101099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/1976638290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a10104cd0 0x7f8a101070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/1976638290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a101023a0 0x7f8a10104790 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 -- 172.21.15.105:0/1976638290 >> 172.21.15.105:0/1976638290 conn(0x7f8a100fbd50 msgr2=0x7f8a100fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 -- 172.21.15.105:0/1976638290 shutdown_connections 2024-03-26T17:54:58.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.733+0000 7f8a16bcd640 1 -- 172.21.15.105:0/1976638290 wait complete. 2024-03-26T17:54:58.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a16bcd640 1 Processor -- start 2024-03-26T17:54:58.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a16bcd640 1 -- start start 2024-03-26T17:54:58.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a16bcd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a101023a0 0x7f8a10195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a16bcd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a10104cd0 0x7f8a10195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a16bcd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a10107600 0x7f8a1019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a16bcd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1010b230 con 0x7f8a101023a0 2024-03-26T17:54:58.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a16bcd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1010b0b0 con 0x7f8a10107600 2024-03-26T17:54:58.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a16bcd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1010b3b0 con 0x7f8a10104cd0 2024-03-26T17:54:58.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a14969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a101023a0 0x7f8a10195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a1516a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a10107600 0x7f8a1019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a14969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a101023a0 0x7f8a10195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50668/0 (socket says 172.21.15.105:50668) 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a14969640 1 -- 172.21.15.105:0/3675790985 learned_addr learned my addr 172.21.15.105:0/3675790985 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.734+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a10104cd0 0x7f8a10195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a1516a640 1 -- 172.21.15.105:0/3675790985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a10104cd0 msgr2=0x7f8a10195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a1516a640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a10104cd0 0x7f8a10195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a1516a640 1 -- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a101023a0 msgr2=0x7f8a10195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a1516a640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a101023a0 0x7f8a10195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a1516a640 1 -- 172.21.15.105:0/3675790985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1019d560 con 0x7f8a10107600 2024-03-26T17:54:58.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a14969640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a101023a0 0x7f8a10195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a1516a640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a10107600 0x7f8a1019cef0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800eef0 tx=0x7f8a08012490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a0dffb640 1 -- 172.21.15.105:0/3675790985 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0801c030 con 0x7f8a10107600 2024-03-26T17:54:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a0dffb640 1 -- 172.21.15.105:0/3675790985 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a08012e70 con 0x7f8a10107600 2024-03-26T17:54:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a16bcd640 1 -- 172.21.15.105:0/3675790985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1019d850 con 0x7f8a10107600 2024-03-26T17:54:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a0dffb640 1 -- 172.21.15.105:0/3675790985 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08018400 con 0x7f8a10107600 2024-03-26T17:54:58.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.735+0000 7f8a16bcd640 1 -- 172.21.15.105:0/3675790985 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1019dda0 con 0x7f8a10107600 2024-03-26T17:54:58.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a0dffb640 1 -- 172.21.15.105:0/3675790985 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8a0801f070 con 0x7f8a10107600 2024-03-26T17:54:58.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a197b8740 1 Processor -- start 2024-03-26T17:54:58.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a197b8740 1 -- start start 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a4318660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a431d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a431dda0 0x5614a4320230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a197b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5614a4274970 con 0x5614a431dda0 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a197b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5614a421cd70 con 0x5614a4318c30 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a197b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614a4270920 con 0x5614a4318240 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a4318660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a4318660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35016/0 (socket says 172.21.15.105:35016) 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a0ffff640 1 -- 172.21.15.105:0/3649388050 learned_addr learned my addr 172.21.15.105:0/3649388050 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a431dda0 0x5614a4320230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a0dffb640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a000a2fa0 0x7f8a000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.739+0000 7f8a14969640 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a431d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a1516a640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a000a2fa0 0x7f8a000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a0ffff640 1 -- 172.21.15.105:0/3649388050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 msgr2=0x5614a431d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a0dffb640 1 -- 172.21.15.105:0/3675790985 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8a080d3000 con 0x7f8a10107600 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a431d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a0ffff640 1 -- 172.21.15.105:0/3649388050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a431dda0 msgr2=0x5614a4320230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a431dda0 0x5614a4320230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a0ffff640 1 -- 172.21.15.105:0/3649388050 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a3f883f0 con 0x5614a4318240 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a14969640 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a431d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a4318660 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0058c0 tx=0x7f89fc005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61830c983f7feb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f8a1516a640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a000a2fa0 0x7f8a000a5460 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8a08021000 tx=0x7f8a08022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f89faffd640 1 -- 172.21.15.105:0/3649388050 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc004030 con 0x5614a4318240 2024-03-26T17:54:58.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f89faffd640 1 -- 172.21.15.105:0/3649388050 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc005020 con 0x5614a4318240 2024-03-26T17:54:58.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.740+0000 7f89faffd640 1 -- 172.21.15.105:0/3649388050 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc004c60 con 0x5614a4318240 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 -- 172.21.15.105:0/3649388050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 msgr2=0x5614a4318660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a4318660 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0058c0 tx=0x7f89fc005850 comp rx=0 tx=0).stop 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 -- 172.21.15.105:0/3649388050 shutdown_connections 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a431dda0 0x5614a4320230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a431d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 --2- 172.21.15.105:0/3649388050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a4318660 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 -- 172.21.15.105:0/3649388050 >> 172.21.15.105:0/3649388050 conn(0x5614a43137b0 msgr2=0x5614a4313bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 -- 172.21.15.105:0/3649388050 shutdown_connections 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.741+0000 7f8a197b8740 1 -- 172.21.15.105:0/3649388050 wait complete. 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.742+0000 7f8a197b8740 1 -- start start 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.742+0000 7f8a197b8740 1 Processor -- start 2024-03-26T17:54:58.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.742+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318240 0x5614a4322780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.742+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a4322cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 0x5614a4329d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318240 0x5614a4322780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318240 0x5614a4322780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50694/0 (socket says 172.21.15.105:50694) 2024-03-26T17:54:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a0ffff640 1 -- 172.21.15.105:0/1877322 learned_addr learned my addr 172.21.15.105:0/1877322 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5614a4320d50 con 0x5614a4318240 2024-03-26T17:54:58.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5614a4230310 con 0x5614a4318c30 2024-03-26T17:54:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614a4320ed0 con 0x5614a431dda0 2024-03-26T17:54:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a14969640 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a4322cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a197b8740 10 client.0 _ll_register_callbacks cb 0x5614a426bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a14969640 1 -- 172.21.15.105:0/1877322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 msgr2=0x5614a4329d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:54:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a14969640 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 0x5614a4329d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a14969640 1 -- 172.21.15.105:0/1877322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318240 msgr2=0x5614a4322780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a14969640 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318240 0x5614a4322780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a14969640 1 -- 172.21.15.105:0/1877322 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a40527d0 con 0x5614a4318c30 2024-03-26T17:54:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318240 0x5614a4322780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89fc005020 con 0x5614a4318c30 2024-03-26T17:54:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f8a14969640 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a4322cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009230 tx=0x7f8a04005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f93349f54a0555 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f89effff640 10 client.31283 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:54:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.743+0000 7f89effff640 1 -- 172.21.15.105:0/1877322 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0404f050 con 0x5614a4318c30 2024-03-26T17:54:58.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f89effff640 1 -- 172.21.15.105:0/1877322 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8a040046c0 con 0x5614a4318c30 2024-03-26T17:54:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f89effff640 1 -- 172.21.15.105:0/1877322 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8a04045070 con 0x5614a4318c30 2024-03-26T17:54:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a197b8740 10 client.31283 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5614a432cf90 con 0x5614a4318c30 2024-03-26T17:54:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f89effff640 1 -- 172.21.15.105:0/1877322 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8a040048c0 con 0x5614a4318c30 2024-03-26T17:54:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a197b8740 10 client.31283 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a197b8740 10 client.31283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a197b8740 20 client.31283 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5614a432e560 0x5614a43309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a197b8740 4 client.31283 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614a4281eb0 con 0x5614a432e560 2024-03-26T17:54:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.744+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5614a432e560 0x5614a43309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.745+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5614a432e560 0x5614a43309b0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63252a962d8abf4d server_cookie=2c57d9f5c5ef174 in_seq=0 out_seq=0 2024-03-26T17:54:58.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.745+0000 7f89effff640 10 client.31283 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:58.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f89effff640 1 -- 172.21.15.105:0/1877322 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5614a4281eb0 con 0x5614a432e560 2024-03-26T17:54:58.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f89effff640 10 client.31283 handle_command_reply: tid=0 2024-03-26T17:54:58.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 client.31283 shutdown 2024-03-26T17:54:58.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 msgr2=0x5614a4322cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a4322cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009230 tx=0x7f8a04005270 comp rx=0 tx=0).stop 2024-03-26T17:54:58.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 shutdown_connections 2024-03-26T17:54:58.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5614a432e560 0x5614a43309b0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f89effff640 0 client.31283 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:58.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 0x5614a4329d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318c30 0x5614a4322cc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1877322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318240 0x5614a4322780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 >> 172.21.15.105:0/1877322 conn(0x5614a43137b0 msgr2=0x5614a430a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 shutdown_connections 2024-03-26T17:54:58.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.747+0000 7f8a197b8740 1 -- 172.21.15.105:0/1877322 wait complete. 2024-03-26T17:54:58.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.748+0000 7f8a197b8740 20 client.31283 trim_cache size 0 max 16384 2024-03-26T17:54:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.755+0000 7fe996d1b640 1 -- 172.21.15.105:0/2473967318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901019d0 msgr2=0x7fe990101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.756+0000 7fe996d1b640 1 --2- 172.21.15.105:0/2473967318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901019d0 0x7fe990101e50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009860 tx=0x7fe97c02bb70 comp rx=0 tx=0).stop 2024-03-26T17:54:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.756+0000 7fe996d1b640 1 -- 172.21.15.105:0/2473967318 shutdown_connections 2024-03-26T17:54:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.756+0000 7fe996d1b640 1 --2- 172.21.15.105:0/2473967318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe990102390 0x7fe990108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.756+0000 7fe996d1b640 1 --2- 172.21.15.105:0/2473967318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901019d0 0x7fe990101e50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.756+0000 7fe996d1b640 1 --2- 172.21.15.105:0/2473967318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe9901007e0 0x7fe990100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.756+0000 7fe996d1b640 1 -- 172.21.15.105:0/2473967318 >> 172.21.15.105:0/2473967318 conn(0x7fe9900fbd50 msgr2=0x7fe9900fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.756+0000 7fe996d1b640 1 -- 172.21.15.105:0/2473967318 shutdown_connections 2024-03-26T17:54:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.756+0000 7fe996d1b640 1 -- 172.21.15.105:0/2473967318 wait complete. 2024-03-26T17:54:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe996d1b640 1 Processor -- start 2024-03-26T17:54:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe996d1b640 1 -- start start 2024-03-26T17:54:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe996d1b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901007e0 0x7fe990197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe996d1b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9901019d0 0x7fe990198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe996d1b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe990102390 0x7fe99019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe996d1b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe99010d430 con 0x7fe990102390 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe996d1b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe99010d2b0 con 0x7fe9901019d0 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe996d1b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe99010d5b0 con 0x7fe9901007e0 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe994ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901007e0 0x7fe990197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe994ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901007e0 0x7fe990197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35048/0 (socket says 172.21.15.105:35048) 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.757+0000 7fe994ab7640 1 -- 172.21.15.105:0/3997152135 learned_addr learned my addr 172.21.15.105:0/3997152135 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe98ffff640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9901019d0 0x7fe990198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe9952b8640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe990102390 0x7fe99019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe994ab7640 1 -- 172.21.15.105:0/3997152135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9901019d0 msgr2=0x7fe990198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe994ab7640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9901019d0 0x7fe990198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe994ab7640 1 -- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe990102390 msgr2=0x7fe99019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe994ab7640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe990102390 0x7fe99019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe994ab7640 1 -- 172.21.15.105:0/3997152135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99019f690 con 0x7fe9901007e0 2024-03-26T17:54:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe994ab7640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901007e0 0x7fe990197ac0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe98400e9a0 tx=0x7fe98400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe98dffb640 1 -- 172.21.15.105:0/3997152135 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98400cd80 con 0x7fe9901007e0 2024-03-26T17:54:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe98dffb640 1 -- 172.21.15.105:0/3997152135 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9840108e0 con 0x7fe9901007e0 2024-03-26T17:54:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.758+0000 7fe996d1b640 1 -- 172.21.15.105:0/3997152135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe990109640 con 0x7fe9901007e0 2024-03-26T17:54:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.759+0000 7fe98dffb640 1 -- 172.21.15.105:0/3997152135 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984018950 con 0x7fe9901007e0 2024-03-26T17:54:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.759+0000 7fe996d1b640 1 -- 172.21.15.105:0/3997152135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe990109b60 con 0x7fe9901007e0 2024-03-26T17:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe98dffb640 1 -- 172.21.15.105:0/3997152135 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe984010a90 con 0x7fe9901007e0 2024-03-26T17:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe9999e2740 1 Processor -- start 2024-03-26T17:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe9999e2740 1 -- start start 2024-03-26T17:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cf45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe9999e2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557967c510b0 con 0x557967cecfc0 2024-03-26T17:54:58.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe9999e2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557967bf9960 con 0x557967cf2130 2024-03-26T17:54:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.762+0000 7fe9999e2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557967c4d060 con 0x557967cec5d0 2024-03-26T17:54:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe994ab7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe98ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cf45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe98ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cf45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39464/0 (socket says 172.21.15.105:39464) 2024-03-26T17:54:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe994ab7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50722/0 (socket says 172.21.15.105:50722) 2024-03-26T17:54:58.764 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe98ffff640 1 -- 172.21.15.105:0/1727217739 learned_addr learned my addr 172.21.15.105:0/1727217739 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe98dffb640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9800a3040 0x7fe9800a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe9952b8640 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe9952b8640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9800a3040 0x7fe9800a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe98dffb640 1 -- 172.21.15.105:0/3997152135 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe9840cf800 con 0x7fe9901007e0 2024-03-26T17:54:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe9952b8640 1 -- 172.21.15.105:0/1727217739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 msgr2=0x557967cf45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe9952b8640 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cf45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe9952b8640 1 -- 172.21.15.105:0/1727217739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 msgr2=0x557967cf1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe9952b8640 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe9952b8640 1 -- 172.21.15.105:0/1727217739 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557967c0bc60 con 0x557967cec5d0 2024-03-26T17:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe98ffff640 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cf45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.763+0000 7fe9952b8640 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cec9f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe988005fd0 tx=0x7fe988013ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca560d0a85bbe1c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.764+0000 7fe9952b8640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9800a3040 0x7fe9800a5500 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe98801e000 tx=0x7fe988020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.764+0000 7fe97affd640 1 -- 172.21.15.105:0/1727217739 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988013120 con 0x557967cec5d0 2024-03-26T17:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.764+0000 7fe97affd640 1 -- 172.21.15.105:0/1727217739 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9880132c0 con 0x557967cec5d0 2024-03-26T17:54:58.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.764+0000 7fe97affd640 1 -- 172.21.15.105:0/1727217739 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988004300 con 0x557967cec5d0 2024-03-26T17:54:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 -- 172.21.15.105:0/1727217739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 msgr2=0x557967cec9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cec9f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe988005fd0 tx=0x7fe988013ef0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 -- 172.21.15.105:0/1727217739 shutdown_connections 2024-03-26T17:54:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cf45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1727217739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cec9f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 -- 172.21.15.105:0/1727217739 >> 172.21.15.105:0/1727217739 conn(0x557967ce7b40 msgr2=0x557967ce7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 -- 172.21.15.105:0/1727217739 shutdown_connections 2024-03-26T17:54:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 -- 172.21.15.105:0/1727217739 wait complete. 2024-03-26T17:54:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 -- start start 2024-03-26T17:54:58.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.765+0000 7fe9999e2740 1 Processor -- start 2024-03-26T17:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.766+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.766+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.766+0000 7fe994ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.766+0000 7fe994ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35060/0 (socket says 172.21.15.105:35060) 2024-03-26T17:54:58.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.766+0000 7fe994ab7640 1 -- 172.21.15.105:0/3579557105 learned_addr learned my addr 172.21.15.105:0/3579557105 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe98ffff640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe9999e2740 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cfd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557967cf5080 con 0x557967cecfc0 2024-03-26T17:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557967c0cdb0 con 0x557967cf2130 2024-03-26T17:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557967cf5200 con 0x557967cec5d0 2024-03-26T17:54:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe9952b8640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cfd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe9999e2740 10 client.0 _ll_register_callbacks cb 0x557967c11110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe994ab7640 1 -- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 msgr2=0x557967cfd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe994ab7640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cfd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe994ab7640 1 -- 172.21.15.105:0/3579557105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 msgr2=0x557967cf6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe994ab7640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe994ab7640 1 -- 172.21.15.105:0/3579557105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557967965210 con 0x557967cec5d0 2024-03-26T17:54:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe9952b8640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cfd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.767+0000 7fe98ffff640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.768+0000 7fe994ab7640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6400 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe984009230 tx=0x7fe984009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70b5fd76355061f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.768+0000 7fe96ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:54:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.768+0000 7fe96ffff640 1 -- 172.21.15.105:0/3579557105 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984010e20 con 0x557967cec5d0 2024-03-26T17:54:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.768+0000 7fe96ffff640 1 -- 172.21.15.105:0/3579557105 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe9840cfbb0 con 0x557967cec5d0 2024-03-26T17:54:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.768+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9880132c0 con 0x557967cec5d0 2024-03-26T17:54:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.768+0000 7fe96ffff640 1 -- 172.21.15.105:0/3579557105 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe98409c590 con 0x557967cec5d0 2024-03-26T17:54:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.768+0000 7fe9999e2740 10 client.31270 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.768+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557967a31620 con 0x557967cec5d0 2024-03-26T17:54:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe96ffff640 1 -- 172.21.15.105:0/3579557105 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9880132c0 con 0x557967cec5d0 2024-03-26T17:54:58.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe96ffff640 1 -- 172.21.15.105:0/3579557105 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe98409b2e0 con 0x557967cec5d0 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe9999e2740 10 client.31270 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe9999e2740 10 client.31270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe9999e2740 20 client.31270 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe9999e2740 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557967d021b0 0x557967d045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe9999e2740 4 client.31270 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557967c5e5f0 con 0x557967d021b0 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.769+0000 7fe98ffff640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557967d021b0 0x557967d045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.770+0000 7fe96ffff640 10 client.31270 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.770+0000 7fe98ffff640 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557967d021b0 0x557967d045e0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b0736d041374a4 server_cookie=68eb3e7fb4fd0338 in_seq=0 out_seq=0 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe96ffff640 1 -- 172.21.15.105:0/3579557105 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557967c5e5f0 con 0x557967d021b0 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe96ffff640 10 client.31270 handle_command_reply: tid=0 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 client.31270 shutdown 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 msgr2=0x557967cf6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6400 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe984009230 tx=0x7fe984009490 comp rx=0 tx=0).stop 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 shutdown_connections 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557967d021b0 0x557967d045e0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967cfd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cf6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 --2- 172.21.15.105:0/3579557105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6400 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe96ffff640 0 client.31270 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 >> 172.21.15.105:0/3579557105 conn(0x557967ce7b40 msgr2=0x557967ce6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 shutdown_connections 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.771+0000 7fe9999e2740 1 -- 172.21.15.105:0/3579557105 wait complete. 2024-03-26T17:54:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.772+0000 7fe9999e2740 20 client.31270 trim_cache size 0 max 16384 2024-03-26T17:54:58.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a197b8740 1 Processor -- start 2024-03-26T17:54:58.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a197b8740 1 -- start start 2024-03-26T17:54:58.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a431a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4329cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a431dda0 0x5614a431a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a197b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a6b60 con 0x5614a4318c30 2024-03-26T17:54:58.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a197b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a86a0 con 0x5614a431dda0 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a197b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a7230 con 0x5614a4318240 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a14969640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4329cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a431a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a431a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35070/0 (socket says 172.21.15.105:35070) 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a0ffff640 1 -- 172.21.15.105:0/1587137535 learned_addr learned my addr 172.21.15.105:0/1587137535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.778+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a431dda0 0x5614a431a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.779+0000 7f8a0ffff640 1 -- 172.21.15.105:0/1587137535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a431dda0 msgr2=0x5614a431a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.779+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a431dda0 0x5614a431a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.779+0000 7f8a0ffff640 1 -- 172.21.15.105:0/1587137535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 msgr2=0x5614a4329cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.779+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4329cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.779+0000 7f8a0ffff640 1 -- 172.21.15.105:0/1587137535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a431d420 con 0x5614a4318240 2024-03-26T17:54:58.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.779+0000 7f8a14969640 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4329cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.779+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a431a170 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009dd0 tx=0x7f89fc009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a6fb3196cfa810 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.779+0000 7f89fa7fc640 1 -- 172.21.15.105:0/1587137535 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc004cb0 con 0x5614a4318240 2024-03-26T17:54:58.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.780+0000 7f89fa7fc640 1 -- 172.21.15.105:0/1587137535 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc004e50 con 0x5614a4318240 2024-03-26T17:54:58.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.780+0000 7f89fa7fc640 1 -- 172.21.15.105:0/1587137535 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00fa00 con 0x5614a4318240 2024-03-26T17:54:58.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 -- 172.21.15.105:0/1587137535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 msgr2=0x5614a431a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a431a170 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009dd0 tx=0x7f89fc009fd0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 -- 172.21.15.105:0/1587137535 shutdown_connections 2024-03-26T17:54:58.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a431dda0 0x5614a431a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4329cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1587137535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a4318240 0x5614a431a170 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 -- 172.21.15.105:0/1587137535 >> 172.21.15.105:0/1587137535 conn(0x5614a43137b0 msgr2=0x5614a4313b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 -- 172.21.15.105:0/1587137535 shutdown_connections 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 -- 172.21.15.105:0/1587137535 wait complete. 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 -- start start 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.781+0000 7f8a197b8740 1 Processor -- start 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318240 0x5614a4326f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4327450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a197b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 0x5614a4328460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a197b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a7fd0 con 0x5614a4318c30 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a197b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a7900 con 0x5614a4318240 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a197b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101a4b40 con 0x5614a431dda0 2024-03-26T17:54:58.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318240 0x5614a4326f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318240 0x5614a4326f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39488/0 (socket says 172.21.15.105:39488) 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a0ffff640 1 -- 172.21.15.105:0/1457361081 learned_addr learned my addr 172.21.15.105:0/1457361081 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a14969640 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4327450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a197b8740 10 client.0 _ll_register_callbacks cb 0x7f89fc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.782+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 0x5614a4328460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f8a14969640 1 -- 172.21.15.105:0/1457361081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 msgr2=0x5614a4328460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f8a14969640 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 0x5614a4328460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f8a14969640 1 -- 172.21.15.105:0/1457361081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318240 msgr2=0x5614a4326f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f8a14969640 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318240 0x5614a4326f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f8a14969640 1 -- 172.21.15.105:0/1457361081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a4328a50 con 0x5614a4318c30 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 0x5614a4328460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f8a14969640 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4327450 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009430 tx=0x7f8a0403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b945901e26e1e85 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f89effff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f89effff640 1 -- 172.21.15.105:0/1457361081 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0404f040 con 0x5614a4318c30 2024-03-26T17:54:58.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f89effff640 1 -- 172.21.15.105:0/1457361081 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8a040048c0 con 0x5614a4318c30 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.783+0000 7f8a197b8740 1 -- 172.21.15.105:0/1457361081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89fc004e50 con 0x5614a4318c30 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.784+0000 7f89effff640 1 -- 172.21.15.105:0/1457361081 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8a040043e0 con 0x5614a4318c30 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.784+0000 7f8a197b8740 10 client.21546 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.784+0000 7f8a197b8740 1 -- 172.21.15.105:0/1457361081 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5614a4329da0 con 0x5614a4318c30 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.784+0000 7f89effff640 1 -- 172.21.15.105:0/1457361081 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc005020 con 0x5614a4318c30 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.784+0000 7f89effff640 1 -- 172.21.15.105:0/1457361081 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8a04054480 con 0x5614a4318c30 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f8a197b8740 10 client.21546 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f8a197b8740 10 client.21546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f8a197b8740 20 client.21546 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5614a432e560 0x5614a4396d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f8a197b8740 4 client.21546 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f8a197b8740 1 -- 172.21.15.105:0/1457361081 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f89fc004e50 con 0x5614a432e560 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5614a432e560 0x5614a4396d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f89effff640 10 client.21546 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:58.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.785+0000 7f8a0ffff640 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5614a432e560 0x5614a4396d80 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a37dc708a301b3 server_cookie=3b9c5113b8ad6c3a in_seq=0 out_seq=0 2024-03-26T17:54:58.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.786+0000 7f89effff640 1 -- 172.21.15.105:0/1457361081 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f89fc004e50 con 0x5614a432e560 2024-03-26T17:54:58.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f89effff640 10 client.21546 handle_command_reply: tid=0 2024-03-26T17:54:58.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 client.21546 shutdown 2024-03-26T17:54:58.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 -- 172.21.15.105:0/1457361081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 msgr2=0x5614a4327450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4327450 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009430 tx=0x7f8a0403f000 comp rx=0 tx=0).stop 2024-03-26T17:54:58.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 -- 172.21.15.105:0/1457361081 shutdown_connections 2024-03-26T17:54:58.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5614a432e560 0x5614a4396d80 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614a431dda0 0x5614a4328460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f89effff640 0 client.21546 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5614a4318c30 0x5614a4327450 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 --2- 172.21.15.105:0/1457361081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5614a4318240 0x5614a4326f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 -- 172.21.15.105:0/1457361081 >> 172.21.15.105:0/1457361081 conn(0x5614a43137b0 msgr2=0x5614a4321270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 -- 172.21.15.105:0/1457361081 shutdown_connections 2024-03-26T17:54:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 1 -- 172.21.15.105:0/1457361081 wait complete. 2024-03-26T17:54:58.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.787+0000 7f8a197b8740 20 client.21546 trim_cache size 0 max 16384 2024-03-26T17:54:58.795 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:54:58.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 -- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a000a2fa0 msgr2=0x7f8a000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a000a2fa0 0x7f8a000a5460 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8a08021000 tx=0x7f8a08022000 comp rx=0 tx=0).stop 2024-03-26T17:54:58.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 -- 172.21.15.105:0/3675790985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a10107600 msgr2=0x7f8a1019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a10107600 0x7f8a1019cef0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800eef0 tx=0x7f8a08012490 comp rx=0 tx=0).stop 2024-03-26T17:54:58.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 -- 172.21.15.105:0/3675790985 shutdown_connections 2024-03-26T17:54:58.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a000a2fa0 0x7f8a000a5460 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a10107600 0x7f8a1019cef0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a10104cd0 0x7f8a10195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 --2- 172.21.15.105:0/3675790985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a101023a0 0x7f8a10195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 -- 172.21.15.105:0/3675790985 >> 172.21.15.105:0/3675790985 conn(0x7f8a100fbd50 msgr2=0x7f8a10102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 -- 172.21.15.105:0/3675790985 shutdown_connections 2024-03-26T17:54:58.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:58.788+0000 7f8a16bcd640 1 -- 172.21.15.105:0/3675790985 wait complete. 2024-03-26T17:54:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.796+0000 7fe9999e2740 1 Processor -- start 2024-03-26T17:54:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.796+0000 7fe9999e2740 1 -- start start 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.796+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.796+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.796+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967ced9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.796+0000 7fe9999e2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901aaea0 con 0x557967cecfc0 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.796+0000 7fe9999e2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ac9e0 con 0x557967cf2130 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.796+0000 7fe9999e2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ab570 con 0x557967cec5d0 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe9952b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967ced9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe9952b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967ced9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39490/0 (socket says 172.21.15.105:39490) 2024-03-26T17:54:58.804 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe9952b8640 1 -- 172.21.15.105:0/4291624932 learned_addr learned my addr 172.21.15.105:0/4291624932 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe994ab7640 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe98ffff640 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe994ab7640 1 -- 172.21.15.105:0/4291624932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 msgr2=0x557967ced9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe994ab7640 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967ced9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe994ab7640 1 -- 172.21.15.105:0/4291624932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 msgr2=0x557967cfd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe994ab7640 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe994ab7640 1 -- 172.21.15.105:0/4291624932 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557967ce3450 con 0x557967cec5d0 2024-03-26T17:54:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe9952b8640 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967ced9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe98ffff640 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe994ab7640 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6720 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe98409a140 tx=0x7fe98409a3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f877bc0b17f53d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe97a7fc640 1 -- 172.21.15.105:0/4291624932 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984010e20 con 0x557967cec5d0 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe97a7fc640 1 -- 172.21.15.105:0/4291624932 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe98407f020 con 0x557967cec5d0 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.797+0000 7fe97a7fc640 1 -- 172.21.15.105:0/4291624932 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98409b2e0 con 0x557967cec5d0 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.798+0000 7fe9999e2740 1 -- 172.21.15.105:0/4291624932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 msgr2=0x557967cf6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6720 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe98409a140 tx=0x7fe98409a3a0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 -- 172.21.15.105:0/4291624932 shutdown_connections 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967ced9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 --2- 172.21.15.105:0/4291624932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cf6720 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 -- 172.21.15.105:0/4291624932 >> 172.21.15.105:0/4291624932 conn(0x557967ce7b40 msgr2=0x557967cf6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.806 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 -- 172.21.15.105:0/4291624932 shutdown_connections 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 -- 172.21.15.105:0/4291624932 wait complete. 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 -- start start 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.799+0000 7fe9999e2740 1 Processor -- start 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.800+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cfa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.800+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.800+0000 7fe9999e2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967d664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.800+0000 7fe994ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cfa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.800+0000 7fe98ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.800+0000 7fe994ab7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cfa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35096/0 (socket says 172.21.15.105:35096) 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.800+0000 7fe994ab7640 1 -- 172.21.15.105:0/1369282259 learned_addr learned my addr 172.21.15.105:0/1369282259 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901ac310 con 0x557967cecfc0 2024-03-26T17:54:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901abc40 con 0x557967cf2130 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9901a8e80 con 0x557967cec5d0 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe9999e2740 10 client.0 _ll_register_callbacks cb 0x7fe984098220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe9952b8640 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967d664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe994ab7640 1 -- 172.21.15.105:0/1369282259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 msgr2=0x557967d664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe994ab7640 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967d664e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe994ab7640 1 -- 172.21.15.105:0/1369282259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 msgr2=0x557967cfaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe994ab7640 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.801+0000 7fe994ab7640 1 -- 172.21.15.105:0/1369282259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557967cfdb40 con 0x557967cec5d0 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe994ab7640 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cfa530 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe9840d51f0 tx=0x7fe98409b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53b64f8e7d6a059 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe96ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe96ffff640 1 -- 172.21.15.105:0/1369282259 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984010e20 con 0x557967cec5d0 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe96ffff640 1 -- 172.21.15.105:0/1369282259 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe9840cfb10 con 0x557967cec5d0 2024-03-26T17:54:58.808 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98407f020 con 0x557967cec5d0 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe96ffff640 1 -- 172.21.15.105:0/1369282259 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe9840ab050 con 0x557967cec5d0 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 10 client.31282 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557967cfdb40 con 0x557967cec5d0 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe96ffff640 1 -- 172.21.15.105:0/1369282259 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe9840cfd50 con 0x557967cec5d0 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe96ffff640 1 -- 172.21.15.105:0/1369282259 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe984097be0 con 0x557967cec5d0 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 10 client.31282 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 10 client.31282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 20 client.31282 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557967d67f30 0x557967d6a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:58.809 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 4 client.31282 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:54:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.802+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe98407f020 con 0x557967d67f30 2024-03-26T17:54:58.810 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.803+0000 7fe98ffff640 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557967d67f30 0x557967d6a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.803+0000 7fe96ffff640 10 client.31282 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:58.811 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.803+0000 7fe98ffff640 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557967d67f30 0x557967d6a320 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756f580bec3f582c server_cookie=d1089d9155226777 in_seq=0 out_seq=0 2024-03-26T17:54:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe96ffff640 1 -- 172.21.15.105:0/1369282259 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe98407f020 con 0x557967d67f30 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe96ffff640 10 client.31282 handle_command_reply: tid=0 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 client.31282 shutdown 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 msgr2=0x557967cfa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cfa530 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe9840d51f0 tx=0x7fe98409b510 comp rx=0 tx=0).stop 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 shutdown_connections 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557967d67f30 0x557967d6a320 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557967cf2130 0x557967d664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557967cecfc0 0x557967cfaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 --2- 172.21.15.105:0/1369282259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557967cec5d0 0x557967cfa530 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe96ffff640 0 client.31282 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.943+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 >> 172.21.15.105:0/1369282259 conn(0x557967ce7b40 msgr2=0x557967ced9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.944+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 shutdown_connections 2024-03-26T17:54:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.944+0000 7fe9999e2740 1 -- 172.21.15.105:0/1369282259 wait complete. 2024-03-26T17:54:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.944+0000 7fe9999e2740 20 client.31282 trim_cache size 0 max 16384 2024-03-26T17:54:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:54:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:54:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:54:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:54:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 -- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9800a3040 msgr2=0x7fe9800a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9800a3040 0x7fe9800a5500 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe98801e000 tx=0x7fe988020000 comp rx=0 tx=0).stop 2024-03-26T17:54:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 -- 172.21.15.105:0/3997152135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901007e0 msgr2=0x7fe990197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901007e0 0x7fe990197ac0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe98400e9a0 tx=0x7fe98400ee60 comp rx=0 tx=0).stop 2024-03-26T17:54:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 -- 172.21.15.105:0/3997152135 shutdown_connections 2024-03-26T17:54:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe9800a3040 0x7fe9800a5500 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe990102390 0x7fe99019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe9901019d0 0x7fe990198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 --2- 172.21.15.105:0/3997152135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9901007e0 0x7fe990197ac0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 -- 172.21.15.105:0/3997152135 >> 172.21.15.105:0/3997152135 conn(0x7fe9900fbd50 msgr2=0x7fe990104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 -- 172.21.15.105:0/3997152135 shutdown_connections 2024-03-26T17:54:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:54:58.945+0000 7fe996d1b640 1 -- 172.21.15.105:0/3997152135 wait complete. 2024-03-26T17:54:59.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:54:58 smithi105 ceph-mon[32280]: pgmap v848: 97 pgs: 97 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 3.7 MiB/s wr, 264 op/s 2024-03-26T17:54:59.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:54:58 smithi144 ceph-mon[36095]: pgmap v848: 97 pgs: 97 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 3.7 MiB/s wr, 264 op/s 2024-03-26T17:54:59.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:54:58 smithi135 ceph-mon[35636]: pgmap v848: 97 pgs: 97 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 3.7 MiB/s wr, 264 op/s 2024-03-26T17:54:59.551 INFO:tasks.workunit.client.0.smithi105.stdout: 10 29261 7.13 MB/sec execute 102 sec latency 205.488 ms 2024-03-26T17:54:59.812 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:54:59.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:54:59.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:54:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 -- 172.21.15.105:0/2114073589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 msgr2=0x7f95dc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:59.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 --2- 172.21.15.105:0/2114073589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 0x7f95dc100be0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009860 tx=0x7f95cc02bb70 comp rx=0 tx=0).stop 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 -- 172.21.15.105:0/2114073589 shutdown_connections 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 --2- 172.21.15.105:0/2114073589 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc102370 0x7f95dc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 --2- 172.21.15.105:0/2114073589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95dc1019b0 0x7f95dc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 --2- 172.21.15.105:0/2114073589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 0x7f95dc100be0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 -- 172.21.15.105:0/2114073589 >> 172.21.15.105:0/2114073589 conn(0x7f95dc0fbd50 msgr2=0x7f95dc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 -- 172.21.15.105:0/2114073589 shutdown_connections 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.974+0000 7f95e1559640 1 -- 172.21.15.105:0/2114073589 wait complete. 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95e1559640 1 Processor -- start 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95e1559640 1 -- start start 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95e1559640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 0x7f95dc197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95e1559640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95dc1019b0 0x7f95dc198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95e1559640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc102370 0x7f95dc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95e1559640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc10c2e0 con 0x7f95dc102370 2024-03-26T17:54:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95e1559640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc10c160 con 0x7f95dc1019b0 2024-03-26T17:54:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc102370 0x7f95dc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95e1559640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc10c460 con 0x7f95dc1007c0 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc102370 0x7f95dc19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50772/0 (socket says 172.21.15.105:50772) 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.975+0000 7f95db7fe640 1 -- 172.21.15.105:0/3826559757 learned_addr learned my addr 172.21.15.105:0/3826559757 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.976+0000 7f95daffd640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 0x7f95dc197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.976+0000 7f95d3fff640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95dc1019b0 0x7f95dc198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.976+0000 7f95db7fe640 1 -- 172.21.15.105:0/3826559757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 msgr2=0x7f95dc197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.976+0000 7f95db7fe640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 0x7f95dc197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.977+0000 7f95db7fe640 1 -- 172.21.15.105:0/3826559757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95dc1019b0 msgr2=0x7f95dc198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.977+0000 7f95db7fe640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95dc1019b0 0x7f95dc198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.977+0000 7f95db7fe640 1 -- 172.21.15.105:0/3826559757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc19f7e0 con 0x7f95dc102370 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.977+0000 7f95d3fff640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95dc1019b0 0x7f95dc198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.977+0000 7f95daffd640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 0x7f95dc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:54:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.977+0000 7f95db7fe640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc102370 0x7f95dc19f170 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f95d400efc0 tx=0x7f95d400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.978+0000 7f95d8ff9640 1 -- 172.21.15.105:0/3826559757 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4003970 con 0x7f95dc102370 2024-03-26T17:54:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.978+0000 7f95d8ff9640 1 -- 172.21.15.105:0/3826559757 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95d4004020 con 0x7f95dc102370 2024-03-26T17:54:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.978+0000 7f95e1559640 1 -- 172.21.15.105:0/3826559757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95dc19fad0 con 0x7f95dc102370 2024-03-26T17:54:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.978+0000 7f95e1559640 1 -- 172.21.15.105:0/3826559757 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95dc1a0020 con 0x7f95dc102370 2024-03-26T17:54:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.979+0000 7f95d8ff9640 1 -- 172.21.15.105:0/3826559757 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d400b9b0 con 0x7f95dc102370 2024-03-26T17:54:59.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.980+0000 7f95d8ff9640 1 -- 172.21.15.105:0/3826559757 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f95d401d020 con 0x7f95dc102370 2024-03-26T17:54:59.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.981+0000 7f95d8ff9640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c80a2f50 0x7f95c80a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.981+0000 7f95e4211740 1 Processor -- start 2024-03-26T17:54:59.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.981+0000 7f95d8ff9640 1 -- 172.21.15.105:0/3826559757 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f95d40cf2d0 con 0x7f95dc102370 2024-03-26T17:54:59.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.981+0000 7f95e4211740 1 -- start start 2024-03-26T17:54:59.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95daffd640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c80a2f50 0x7f95c80a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95e4211740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42190 0x55601dd425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95e4211740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42b80 0x55601dd47680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95e4211740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd4a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95e4211740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55601dca7630 con 0x55601dd42b80 2024-03-26T17:54:59.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95e4211740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55601dc4fe10 con 0x55601dd47cf0 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95e4211740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55601dca35e0 con 0x55601dd42190 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95daffd640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c80a2f50 0x7f95c80a5410 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f95cc005b60 tx=0x7f95cc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd4a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd4a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39534/0 (socket says 172.21.15.105:39534) 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.982+0000 7f95db7fe640 1 -- 172.21.15.105:0/1940609415 learned_addr learned my addr 172.21.15.105:0/1940609415 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95d3fff640 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42b80 0x55601dd47680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95daffd640 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42190 0x55601dd425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95db7fe640 1 -- 172.21.15.105:0/1940609415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42190 msgr2=0x55601dd425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95db7fe640 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42190 0x55601dd425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95db7fe640 1 -- 172.21.15.105:0/1940609415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42b80 msgr2=0x55601dd47680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95db7fe640 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42b80 0x55601dd47680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95db7fe640 1 -- 172.21.15.105:0/1940609415 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601dc66740 con 0x55601dd47cf0 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95daffd640 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42190 0x55601dd425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95d3fff640 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42b80 0x55601dd47680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:54:59.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.983+0000 7f95db7fe640 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd4a180 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f95d4097dc0 tx=0x7f95d400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=381c6d1fc40fa81b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:59.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.984+0000 7f95d17fa640 1 -- 172.21.15.105:0/1940609415 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d40a4370 con 0x55601dd47cf0 2024-03-26T17:54:59.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.984+0000 7f95d17fa640 1 -- 172.21.15.105:0/1940609415 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95d4099c60 con 0x55601dd47cf0 2024-03-26T17:54:59.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.984+0000 7f95d17fa640 1 -- 172.21.15.105:0/1940609415 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d40a3ee0 con 0x55601dd47cf0 2024-03-26T17:54:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 -- 172.21.15.105:0/1940609415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 msgr2=0x55601dd4a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd4a180 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f95d4097dc0 tx=0x7f95d400a430 comp rx=0 tx=0).stop 2024-03-26T17:54:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 -- 172.21.15.105:0/1940609415 shutdown_connections 2024-03-26T17:54:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd4a180 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42b80 0x55601dd47680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 --2- 172.21.15.105:0/1940609415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42190 0x55601dd425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 -- 172.21.15.105:0/1940609415 >> 172.21.15.105:0/1940609415 conn(0x55601dd3d700 msgr2=0x55601dd3db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:59.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 -- 172.21.15.105:0/1940609415 shutdown_connections 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 -- 172.21.15.105:0/1940609415 wait complete. 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 -- start start 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.985+0000 7f95e4211740 1 Processor -- start 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95e4211740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd50ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95e4211740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd42b80 0x55601dd50ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95e4211740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd47cf0 0x55601dd58030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95e4211740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55601dd4aca0 con 0x55601dd42190 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95e4211740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55601dc631a0 con 0x55601dd42b80 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd50ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd50ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50786/0 (socket says 172.21.15.105:50786) 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd42b80 0x55601dd50ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95daffd640 1 -- 172.21.15.105:0/3741255856 learned_addr learned my addr 172.21.15.105:0/3741255856 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95e4211740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55601dd4ae20 con 0x55601dd47cf0 2024-03-26T17:54:59.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.986+0000 7f95db7fe640 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd47cf0 0x55601dd58030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95e4211740 10 client.0 _ll_register_callbacks cb 0x55601d9bac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95d3fff640 1 -- 172.21.15.105:0/3741255856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd47cf0 msgr2=0x55601dd58030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95d3fff640 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd47cf0 0x55601dd58030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95d3fff640 1 -- 172.21.15.105:0/3741255856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 msgr2=0x55601dd50ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95d3fff640 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd50ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95d3fff640 1 -- 172.21.15.105:0/3741255856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601da837e0 con 0x55601dd42b80 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95d3fff640 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd42b80 0x55601dd50ff0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55601dd42f80 tx=0x7f95c4013bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f946d7f7526948 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95c27fc640 1 -- 172.21.15.105:0/3741255856 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4011e60 con 0x55601dd42b80 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95c27fc640 1 -- 172.21.15.105:0/3741255856 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f95c4005020 con 0x55601dd42b80 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.987+0000 7f95e4211740 1 -- 172.21.15.105:0/3741255856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d4099c60 con 0x55601dd42b80 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95c27fc640 1 -- 172.21.15.105:0/3741255856 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f95c4005220 con 0x55601dd42b80 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95e4211740 10 client.31319 fetch_fsmap learned FSMap version 37 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95e4211740 1 -- 172.21.15.105:0/3741255856 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55601dae3f70 con 0x55601dd42b80 2024-03-26T17:54:59.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95c27fc640 1 -- 172.21.15.105:0/3741255856 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95c4004c50 con 0x55601dd42b80 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95c27fc640 1 -- 172.21.15.105:0/3741255856 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f95c401a030 con 0x55601dd42b80 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95e4211740 10 client.31319 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95e4211740 10 client.31319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95e4211740 20 client.31319 populate_metadata read hostname 'smithi105' 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95e4211740 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55601dd5c910 0x55601dd5eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95e4211740 4 client.31319 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.988+0000 7f95e4211740 1 -- 172.21.15.105:0/3741255856 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f95d4099c60 con 0x55601dd5c910 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.989+0000 7f95daffd640 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55601dd5c910 0x55601dd5eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.990+0000 7f95daffd640 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55601dd5c910 0x55601dd5eda0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a3d22049e1c935 server_cookie=6232e09deba732ff in_seq=0 out_seq=0 2024-03-26T17:54:59.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.990+0000 7f95c27fc640 10 client.31319 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.991+0000 7f95c27fc640 1 -- 172.21.15.105:0/3741255856 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f95d4099c60 con 0x55601dd5c910 2024-03-26T17:54:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.991+0000 7f95c27fc640 10 client.31319 handle_command_reply: tid=0 2024-03-26T17:54:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.991+0000 7f95e4211740 1 client.31319 shutdown 2024-03-26T17:54:59.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 -- 172.21.15.105:0/3741255856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd42b80 msgr2=0x55601dd50ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd42b80 0x55601dd50ff0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55601dd42f80 tx=0x7f95c4013bf0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 -- 172.21.15.105:0/3741255856 shutdown_connections 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55601dd5c910 0x55601dd5eda0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd47cf0 0x55601dd58030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd42b80 0x55601dd50ff0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 --2- 172.21.15.105:0/3741255856 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd50ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 -- 172.21.15.105:0/3741255856 >> 172.21.15.105:0/3741255856 conn(0x55601dd3d700 msgr2=0x55601dd3c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95c27fc640 0 client.31319 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 -- 172.21.15.105:0/3741255856 shutdown_connections 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 1 -- 172.21.15.105:0/3741255856 wait complete. 2024-03-26T17:54:59.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:54:59.992+0000 7f95e4211740 20 client.31319 trim_cache size 0 max 16384 2024-03-26T17:55:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.021+0000 7f95e4211740 1 Processor -- start 2024-03-26T17:55:00.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.021+0000 7f95e4211740 1 -- start start 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95e4211740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd41070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95e4211740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd57ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95e4211740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd63660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95e4211740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1a6aa0 con 0x55601dd42190 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95e4211740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1a85e0 con 0x55601dd47cf0 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95e4211740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1a7170 con 0x55601dd42b80 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95daffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd41070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd63660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd57ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95db7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd63660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39552/0 (socket says 172.21.15.105:39552) 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd57ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35134/0 (socket says 172.21.15.105:35134) 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95db7fe640 1 -- 172.21.15.105:0/376402979 learned_addr learned my addr 172.21.15.105:0/376402979 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:00.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95daffd640 1 -- 172.21.15.105:0/376402979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 msgr2=0x55601dd57ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95daffd640 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd57ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95daffd640 1 -- 172.21.15.105:0/376402979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 msgr2=0x55601dd63660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95daffd640 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd63660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95daffd640 1 -- 172.21.15.105:0/376402979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601d97b170 con 0x55601dd42190 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95db7fe640 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd63660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.022+0000 7f95d3fff640 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd57ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.023+0000 7f95daffd640 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd41070 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x55601dd5f610 tx=0x7f95cc03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a60392bbd131bf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.023+0000 7f95d0ff9640 1 -- 172.21.15.105:0/376402979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc052070 con 0x55601dd42190 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.023+0000 7f95d0ff9640 1 -- 172.21.15.105:0/376402979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95cc045040 con 0x55601dd42190 2024-03-26T17:55:00.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.023+0000 7f95d0ff9640 1 -- 172.21.15.105:0/376402979 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc044040 con 0x55601dd42190 2024-03-26T17:55:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 -- 172.21.15.105:0/376402979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 msgr2=0x55601dd41070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd41070 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x55601dd5f610 tx=0x7f95cc03c000 comp rx=0 tx=0).stop 2024-03-26T17:55:00.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 -- 172.21.15.105:0/376402979 shutdown_connections 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd63660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd57ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 --2- 172.21.15.105:0/376402979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd41070 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 -- 172.21.15.105:0/376402979 >> 172.21.15.105:0/376402979 conn(0x55601dd3d700 msgr2=0x55601dd40960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 -- 172.21.15.105:0/376402979 shutdown_connections 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 -- 172.21.15.105:0/376402979 wait complete. 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 -- start start 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.024+0000 7f95e4211740 1 Processor -- start 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95e4211740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95e4211740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95e4211740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95e4211740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1a7f10 con 0x55601dd42190 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95e4211740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1a7840 con 0x55601dd47cf0 2024-03-26T17:55:00.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95e4211740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1a4a80 con 0x55601dd42b80 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95e4211740 10 client.0 _ll_register_callbacks cb 0x7f95c4009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95d3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35148/0 (socket says 172.21.15.105:35148) 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95db7fe640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95d3fff640 1 -- 172.21.15.105:0/469435898 learned_addr learned my addr 172.21.15.105:0/469435898 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.025+0000 7f95daffd640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95d3fff640 1 -- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 msgr2=0x55601dd562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95d3fff640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95d3fff640 1 -- 172.21.15.105:0/469435898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 msgr2=0x55601dd57b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95d3fff640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd57b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95d3fff640 1 -- 172.21.15.105:0/469435898 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601dd56b10 con 0x55601dd42b80 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95db7fe640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95daffd640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:00.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95d3fff640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd580a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c4009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fba0d9b40c1222 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95c27fc640 1 -- 172.21.15.105:0/469435898 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c402a050 con 0x55601dd42b80 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95e4211740 1 -- 172.21.15.105:0/469435898 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95cc045040 con 0x55601dd42b80 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95c27fc640 1 -- 172.21.15.105:0/469435898 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f95c4011c60 con 0x55601dd42b80 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95c27fc640 1 -- 172.21.15.105:0/469435898 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f95c400fc40 con 0x55601dd42b80 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95e4211740 10 client.31297 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.026+0000 7f95e4211740 1 -- 172.21.15.105:0/469435898 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55601ddc3960 con 0x55601dd42b80 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95c27fc640 1 -- 172.21.15.105:0/469435898 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95cc045040 con 0x55601dd42b80 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95c27fc640 1 -- 172.21.15.105:0/469435898 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f95c401a040 con 0x55601dd42b80 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95e4211740 10 client.31297 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95e4211740 10 client.31297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95e4211740 20 client.31297 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95e4211740 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55601ddc4330 0x55601ddc6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95e4211740 4 client.31297 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:55:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95e4211740 1 -- 172.21.15.105:0/469435898 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55601dcb4b70 con 0x55601ddc4330 2024-03-26T17:55:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.027+0000 7f95daffd640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55601ddc4330 0x55601ddc6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.028+0000 7f95daffd640 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55601ddc4330 0x55601ddc6750 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eac91020c1747e0 server_cookie=89ef9705798b4de in_seq=0 out_seq=0 2024-03-26T17:55:00.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.028+0000 7f95c27fc640 10 client.31297 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95c27fc640 1 -- 172.21.15.105:0/469435898 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55601dcb4b70 con 0x55601ddc4330 2024-03-26T17:55:00.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95c27fc640 10 client.31297 handle_command_reply: tid=0 2024-03-26T17:55:00.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 client.31297 shutdown 2024-03-26T17:55:00.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 -- 172.21.15.105:0/469435898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 msgr2=0x55601dd580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd580a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c4009d30 comp rx=0 tx=0).stop 2024-03-26T17:55:00.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 -- 172.21.15.105:0/469435898 shutdown_connections 2024-03-26T17:55:00.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55601ddc4330 0x55601ddc6750 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55601dd47cf0 0x55601dd562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95c27fc640 0 client.31297 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:00.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55601dd42b80 0x55601dd580a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 --2- 172.21.15.105:0/469435898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55601dd42190 0x55601dd57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 -- 172.21.15.105:0/469435898 >> 172.21.15.105:0/469435898 conn(0x55601dd3d700 msgr2=0x55601dd411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 -- 172.21.15.105:0/469435898 shutdown_connections 2024-03-26T17:55:00.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 1 -- 172.21.15.105:0/469435898 wait complete. 2024-03-26T17:55:00.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.030+0000 7f95e4211740 20 client.31297 trim_cache size 0 max 16384 2024-03-26T17:55:00.028 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:55:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:55:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "343dbd81-eb85-47d5-ad5e-b9fc23e3b92e", 2024-03-26T17:55:00.028 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:55:00.028 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:55:00.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 -- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c80a2f50 msgr2=0x7f95c80a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c80a2f50 0x7f95c80a5410 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f95cc005b60 tx=0x7f95cc005a90 comp rx=0 tx=0).stop 2024-03-26T17:55:00.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 -- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc102370 msgr2=0x7f95dc19f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc102370 0x7f95dc19f170 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f95d400efc0 tx=0x7f95d400c800 comp rx=0 tx=0).stop 2024-03-26T17:55:00.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 -- 172.21.15.105:0/3826559757 shutdown_connections 2024-03-26T17:55:00.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95c80a2f50 0x7f95c80a5410 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc102370 0x7f95dc19f170 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95dc1019b0 0x7f95dc198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 --2- 172.21.15.105:0/3826559757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95dc1007c0 0x7f95dc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 -- 172.21.15.105:0/3826559757 >> 172.21.15.105:0/3826559757 conn(0x7f95dc0fbd50 msgr2=0x7f95dc068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 -- 172.21.15.105:0/3826559757 shutdown_connections 2024-03-26T17:55:00.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.031+0000 7f95e1559640 1 -- 172.21.15.105:0/3826559757 wait complete. 2024-03-26T17:55:00.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:55:00.033 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:55:00.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 -- 172.21.15.105:0/3950663729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424100320 msgr2=0x7f74241007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3950663729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424100320 0x7f74241007d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f741c009ca0 tx=0x7f741c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 -- 172.21.15.105:0/3950663729 shutdown_connections 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3950663729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424100320 0x7f74241007d0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3950663729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f74240ff960 0x7f74240ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3950663729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7424101e70 0x7f7424102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 -- 172.21.15.105:0/3950663729 >> 172.21.15.105:0/3950663729 conn(0x7f74240fb4e0 msgr2=0x7f74240fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 -- 172.21.15.105:0/3950663729 shutdown_connections 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 -- 172.21.15.105:0/3950663729 wait complete. 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.181+0000 7f742c5b2640 1 Processor -- start 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742c5b2640 1 -- start start 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742c5b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74240ff960 0x7f7424199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742c5b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7424100320 0x7f742419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742c5b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424101e70 0x7f74241a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742c5b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f742410f640 con 0x7f7424101e70 2024-03-26T17:55:00.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742c5b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f742410f4c0 con 0x7f7424100320 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742c5b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f742410f7c0 con 0x7f74240ff960 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742ab4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424101e70 0x7f74241a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742a34e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74240ff960 0x7f7424199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742a34e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74240ff960 0x7f7424199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35174/0 (socket says 172.21.15.105:35174) 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742a34e640 1 -- 172.21.15.105:0/3796778089 learned_addr learned my addr 172.21.15.105:0/3796778089 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f7429b4d640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7424100320 0x7f742419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742ab4f640 1 -- 172.21.15.105:0/3796778089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74240ff960 msgr2=0x7f7424199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742ab4f640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74240ff960 0x7f7424199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742ab4f640 1 -- 172.21.15.105:0/3796778089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7424100320 msgr2=0x7f742419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742ab4f640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7424100320 0x7f742419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.182+0000 7f742ab4f640 1 -- 172.21.15.105:0/3796778089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74241a1900 con 0x7f7424101e70 2024-03-26T17:55:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.183+0000 7f742ab4f640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424101e70 0x7f74241a1260 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f741c02c630 tx=0x7f741c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.183+0000 7f741b7fe640 1 -- 172.21.15.105:0/3796778089 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c04d070 con 0x7f7424101e70 2024-03-26T17:55:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.183+0000 7f741b7fe640 1 -- 172.21.15.105:0/3796778089 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741c040e30 con 0x7f7424101e70 2024-03-26T17:55:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.183+0000 7f742c5b2640 1 -- 172.21.15.105:0/3796778089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7424102c40 con 0x7f7424101e70 2024-03-26T17:55:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.183+0000 7f741b7fe640 1 -- 172.21.15.105:0/3796778089 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c004060 con 0x7f7424101e70 2024-03-26T17:55:00.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.183+0000 7f742c5b2640 1 -- 172.21.15.105:0/3796778089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7424103130 con 0x7f7424101e70 2024-03-26T17:55:00.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.185+0000 7f741b7fe640 1 -- 172.21.15.105:0/3796778089 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f741c050070 con 0x7f7424101e70 2024-03-26T17:55:00.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.186+0000 7f741b7fe640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74100a2f20 0x7f74100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.186+0000 7f741b7fe640 1 -- 172.21.15.105:0/3796778089 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f741c0f9580 con 0x7f7424101e70 2024-03-26T17:55:00.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.186+0000 7f742a34e640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74100a2f20 0x7f74100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.186+0000 7f742f1a0740 1 Processor -- start 2024-03-26T17:55:00.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742f1a0740 1 -- start start 2024-03-26T17:55:00.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceaa8f0 0x564ffceaad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceab2e0 0x564ffceafde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceb0450 0x564ffceb28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742f1a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564ffce0f980 con 0x564ffceaa8f0 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74100a2f20 0x7f74100a53e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f742419b1c0 tx=0x7f7420005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742f1a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ffcdb7be0 con 0x564ffceab2e0 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742f1a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ffce0b930 con 0x564ffceb0450 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceaa8f0 0x564ffceaad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceaa8f0 0x564ffceaad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50814/0 (socket says 172.21.15.105:50814) 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 -- 172.21.15.105:0/2294008629 learned_addr learned my addr 172.21.15.105:0/2294008629 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742ab4f640 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceb0450 0x564ffceb28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f7429b4d640 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceab2e0 0x564ffceafde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 -- 172.21.15.105:0/2294008629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceb0450 msgr2=0x564ffceb28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceb0450 0x564ffceb28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 -- 172.21.15.105:0/2294008629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceab2e0 msgr2=0x564ffceafde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceab2e0 0x564ffceafde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742a34e640 1 -- 172.21.15.105:0/2294008629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ffcbed7d0 con 0x564ffceaa8f0 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.187+0000 7f742ab4f640 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceb0450 0x564ffceb28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.188+0000 7f742a34e640 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceaa8f0 0x564ffceaad10 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7420009f80 tx=0x7f7420014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d41d545252a9641 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.188+0000 7f7418bfe640 1 -- 172.21.15.105:0/2294008629 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200265f0 con 0x564ffceaa8f0 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.188+0000 7f7418bfe640 1 -- 172.21.15.105:0/2294008629 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7420026c40 con 0x564ffceaa8f0 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.188+0000 7f7418bfe640 1 -- 172.21.15.105:0/2294008629 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420025ce0 con 0x564ffceaa8f0 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- 172.21.15.105:0/2294008629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceaa8f0 msgr2=0x564ffceaad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceaa8f0 0x564ffceaad10 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7420009f80 tx=0x7f7420014000 comp rx=0 tx=0).stop 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- 172.21.15.105:0/2294008629 shutdown_connections 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceb0450 0x564ffceb28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceab2e0 0x564ffceafde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2294008629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceaa8f0 0x564ffceaad10 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- 172.21.15.105:0/2294008629 >> 172.21.15.105:0/2294008629 conn(0x564ffcea5e60 msgr2=0x564ffcea6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- 172.21.15.105:0/2294008629 shutdown_connections 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- 172.21.15.105:0/2294008629 wait complete. 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- start start 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 Processor -- start 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceaa8f0 0x564ffceb4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceab2e0 0x564ffceb50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceb0450 0x564ffcebc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564ffceb3400 con 0x564ffceab2e0 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ffcdcb160 con 0x564ffceb0450 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564ffceb3580 con 0x564ffceaa8f0 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.189+0000 7f742f1a0740 10 client.0 _ll_register_callbacks cb 0x564ffce07720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceab2e0 0x564ffceb50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceab2e0 0x564ffceb50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50820/0 (socket says 172.21.15.105:50820) 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 -- 172.21.15.105:0/4127620133 learned_addr learned my addr 172.21.15.105:0/4127620133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f742a34e640 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceaa8f0 0x564ffceb4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f742ab4f640 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceb0450 0x564ffcebc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 -- 172.21.15.105:0/4127620133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceaa8f0 msgr2=0x564ffceb4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceaa8f0 0x564ffceb4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 -- 172.21.15.105:0/4127620133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceb0450 msgr2=0x564ffcebc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceb0450 0x564ffcebc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 -- 172.21.15.105:0/4127620133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ffcb233f0 con 0x564ffceab2e0 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f742a34e640 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceaa8f0 0x564ffceb4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f7429b4d640 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceab2e0 0x564ffceb50d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7414012700 tx=0x7f7414012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c6499062f7a443 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:00.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f740dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f740dffb640 1 -- 172.21.15.105:0/4127620133 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414004050 con 0x564ffceab2e0 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f740dffb640 1 -- 172.21.15.105:0/4127620133 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f74140041f0 con 0x564ffceab2e0 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.190+0000 7f742f1a0740 1 -- 172.21.15.105:0/4127620133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7420026c40 con 0x564ffceab2e0 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f740dffb640 1 -- 172.21.15.105:0/4127620133 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f74140043f0 con 0x564ffceab2e0 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742f1a0740 10 client.21582 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742f1a0740 1 -- 172.21.15.105:0/4127620133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564ffcae30f0 con 0x564ffceab2e0 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f740dffb640 1 -- 172.21.15.105:0/4127620133 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7420026c40 con 0x564ffceab2e0 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f740dffb640 1 -- 172.21.15.105:0/4127620133 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7414003b00 con 0x564ffceab2e0 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742f1a0740 10 client.21582 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742f1a0740 10 client.21582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742f1a0740 20 client.21582 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742f1a0740 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ffcec0a80 0x564ffcec2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742f1a0740 4 client.21582 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:00.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742f1a0740 1 -- 172.21.15.105:0/4127620133 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ffce1cec0 con 0x564ffcec0a80 2024-03-26T17:55:00.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.191+0000 7f742a34e640 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ffcec0a80 0x564ffcec2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.192+0000 7f742a34e640 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ffcec0a80 0x564ffcec2f00 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9918326a22cd7ade server_cookie=e4482056e56958cc in_seq=0 out_seq=0 2024-03-26T17:55:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.192+0000 7f740dffb640 10 client.21582 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.194+0000 7f740dffb640 1 -- 172.21.15.105:0/4127620133 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564ffce1cec0 con 0x564ffcec0a80 2024-03-26T17:55:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.194+0000 7f740dffb640 10 client.21582 handle_command_reply: tid=0 2024-03-26T17:55:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 client.21582 shutdown 2024-03-26T17:55:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 -- 172.21.15.105:0/4127620133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceab2e0 msgr2=0x564ffceb50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceab2e0 0x564ffceb50d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7414012700 tx=0x7f7414012bc0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 -- 172.21.15.105:0/4127620133 shutdown_connections 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ffcec0a80 0x564ffcec2f00 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceb0450 0x564ffcebc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffceab2e0 0x564ffceb50d0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 --2- 172.21.15.105:0/4127620133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceaa8f0 0x564ffceb4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f740dffb640 0 client.21582 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 -- 172.21.15.105:0/4127620133 >> 172.21.15.105:0/4127620133 conn(0x564ffcea5e60 msgr2=0x564ffcea5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 -- 172.21.15.105:0/4127620133 shutdown_connections 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 1 -- 172.21.15.105:0/4127620133 wait complete. 2024-03-26T17:55:00.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.195+0000 7f742f1a0740 20 client.21582 trim_cache size 0 max 16384 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.223+0000 7f742f1a0740 1 Processor -- start 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.223+0000 7f742f1a0740 1 -- start start 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.223+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffcea6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.223+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffcebc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.223+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcea47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.223+0000 7f742f1a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74241a6af0 con 0x564ffcea5e60 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.223+0000 7f742f1a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f74241a8630 con 0x564ffceaa8f0 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.223+0000 7f742f1a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74241a71c0 con 0x564ffceab2e0 2024-03-26T17:55:00.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffcea6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffcea6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50826/0 (socket says 172.21.15.105:50826) 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 -- 172.21.15.105:0/3064259538 learned_addr learned my addr 172.21.15.105:0/3064259538 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f7429b4d640 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffcebc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742ab4f640 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcea47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 -- 172.21.15.105:0/3064259538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 msgr2=0x564ffcea47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcea47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 -- 172.21.15.105:0/3064259538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 msgr2=0x564ffcebc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffcebc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 -- 172.21.15.105:0/3064259538 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ffceb5b80 con 0x564ffcea5e60 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f742a34e640 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffcea6240 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7420028040 tx=0x7f7420009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81c1a3d19909d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f740ffff640 1 -- 172.21.15.105:0/3064259538 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420025770 con 0x564ffcea5e60 2024-03-26T17:55:00.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f740ffff640 1 -- 172.21.15.105:0/3064259538 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7420039070 con 0x564ffcea5e60 2024-03-26T17:55:00.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.224+0000 7f740ffff640 1 -- 172.21.15.105:0/3064259538 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420033260 con 0x564ffcea5e60 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.225+0000 7f742f1a0740 1 -- 172.21.15.105:0/3064259538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 msgr2=0x564ffcea6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.225+0000 7f742f1a0740 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffcea6240 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f7420028040 tx=0x7f7420009fb0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.225+0000 7f742f1a0740 1 -- 172.21.15.105:0/3064259538 shutdown_connections 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.225+0000 7f742f1a0740 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcea47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.225+0000 7f742f1a0740 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffcebc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.225+0000 7f742f1a0740 1 --2- 172.21.15.105:0/3064259538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffcea6240 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.225+0000 7f742f1a0740 1 -- 172.21.15.105:0/3064259538 >> 172.21.15.105:0/3064259538 conn(0x564ffceaff20 msgr2=0x564ffceb3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.225+0000 7f742f1a0740 1 -- 172.21.15.105:0/3064259538 shutdown_connections 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 -- 172.21.15.105:0/3064259538 wait complete. 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 -- start start 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 Processor -- start 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffceb9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffceb9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcebaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74241a7f60 con 0x564ffcea5e60 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f74241a7890 con 0x564ffceaa8f0 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74241a4ad0 con 0x564ffceab2e0 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742f1a0740 10 client.0 _ll_register_callbacks cb 0x7f7414009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742ab4f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcebaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742ab4f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcebaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35232/0 (socket says 172.21.15.105:35232) 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.226+0000 7f742ab4f640 1 -- 172.21.15.105:0/2503136648 learned_addr learned my addr 172.21.15.105:0/2503136648 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742a34e640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffceb9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f7429b4d640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffceb9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742ab4f640 1 -- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 msgr2=0x564ffceb9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742ab4f640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffceb9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742ab4f640 1 -- 172.21.15.105:0/2503136648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 msgr2=0x564ffceb9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:00.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742ab4f640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffceb9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742ab4f640 1 -- 172.21.15.105:0/2503136648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ffcebb090 con 0x564ffceab2e0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f7429b4d640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffceb9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742a34e640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffceb9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742ab4f640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcebaaa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f741c032420 tx=0x7f741c032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=735ddadde0229cc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f740dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f740dffb640 1 -- 172.21.15.105:0/2503136648 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c0ce390 con 0x564ffceab2e0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f740dffb640 1 -- 172.21.15.105:0/2503136648 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f741c0f90b0 con 0x564ffceab2e0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.227+0000 7f742f1a0740 1 -- 172.21.15.105:0/2503136648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7420039070 con 0x564ffceab2e0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f740dffb640 1 -- 172.21.15.105:0/2503136648 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f741c0c5cc0 con 0x564ffceab2e0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f742f1a0740 10 client.31315 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f742f1a0740 1 -- 172.21.15.105:0/2503136648 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564ffcebc410 con 0x564ffceab2e0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f740dffb640 1 -- 172.21.15.105:0/2503136648 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741c0c9d60 con 0x564ffceab2e0 2024-03-26T17:55:00.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f740dffb640 1 -- 172.21.15.105:0/2503136648 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f741c047030 con 0x564ffceab2e0 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f742f1a0740 10 client.31315 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f742f1a0740 10 client.31315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f742f1a0740 20 client.31315 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ffceb0450 0x564ffcf29670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f742f1a0740 4 client.31315 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.228+0000 7f742f1a0740 1 -- 172.21.15.105:0/2503136648 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7420039070 con 0x564ffceb0450 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.229+0000 7f742a34e640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ffceb0450 0x564ffcf29670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.229+0000 7f742a34e640 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ffceb0450 0x564ffcf29670 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e3394169fb0f448 server_cookie=c522e99e59a6bc5f in_seq=0 out_seq=0 2024-03-26T17:55:00.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.229+0000 7f740dffb640 10 client.31315 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f740dffb640 1 -- 172.21.15.105:0/2503136648 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x7f7420039070 con 0x564ffceb0450 2024-03-26T17:55:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f740dffb640 10 client.31315 handle_command_reply: tid=0 2024-03-26T17:55:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 client.31315 shutdown 2024-03-26T17:55:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 -- 172.21.15.105:0/2503136648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 msgr2=0x564ffcebaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcebaaa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f741c032420 tx=0x7f741c032340 comp rx=0 tx=0).stop 2024-03-26T17:55:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 -- 172.21.15.105:0/2503136648 shutdown_connections 2024-03-26T17:55:00.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564ffceb0450 0x564ffcf29670 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564ffceab2e0 0x564ffcebaaa0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ffceaa8f0 0x564ffceb9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f740dffb640 0 client.31315 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 --2- 172.21.15.105:0/2503136648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564ffcea5e60 0x564ffceb9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 -- 172.21.15.105:0/2503136648 >> 172.21.15.105:0/2503136648 conn(0x564ffceaff20 msgr2=0x564ffcea6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 -- 172.21.15.105:0/2503136648 shutdown_connections 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 1 -- 172.21.15.105:0/2503136648 wait complete. 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.231+0000 7f742f1a0740 20 client.31315 trim_cache size 0 max 16384 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:55:00.223 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (213 inodes in the stack)", 2024-03-26T17:55:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:55:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "343dbd81-eb85-47d5-ad5e-b9fc23e3b92e": { 2024-03-26T17:55:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:55:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "343dbd81-eb85-47d5-ad5e-b9fc23e3b92e", 2024-03-26T17:55:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:55:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:55:00.224 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:55:00.224 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 -- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74100a2f20 msgr2=0x7f74100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74100a2f20 0x7f74100a53e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f742419b1c0 tx=0x7f7420005f90 comp rx=0 tx=0).stop 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 -- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424101e70 msgr2=0x7f74241a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424101e70 0x7f74241a1260 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f741c02c630 tx=0x7f741c0328a0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 -- 172.21.15.105:0/3796778089 shutdown_connections 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f74100a2f20 0x7f74100a53e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7424101e70 0x7f74241a1260 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7424100320 0x7f742419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 --2- 172.21.15.105:0/3796778089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74240ff960 0x7f7424199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.232+0000 7f742c5b2640 1 -- 172.21.15.105:0/3796778089 >> 172.21.15.105:0/3796778089 conn(0x7f74240fb4e0 msgr2=0x7f74240fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.233+0000 7f742c5b2640 1 -- 172.21.15.105:0/3796778089 shutdown_connections 2024-03-26T17:55:00.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:00.233+0000 7f742c5b2640 1 -- 172.21.15.105:0/3796778089 wait complete. 2024-03-26T17:55:00.236 INFO:tasks.cephfs.filesystem:scrub status for tag:343dbd81-eb85-47d5-ad5e-b9fc23e3b92e - {'path': '/', 'tag': '343dbd81-eb85-47d5-ad5e-b9fc23e3b92e', 'options': 'recursive,force'} 2024-03-26T17:55:00.410 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:55:00 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:55:00.120+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003d3d [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client5/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} v6073 ap=2 RANDEPHEMERALPIN f(v210 m2024-03-26T17:54:52.968028+0000 20=20+0) n(v20 rc2024-03-26T17:54:52.968028+0000 b5201920 21=20+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@3803} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d471600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003d3d:[<0x10000003bbc/PARADOX v108>,<0x10000003bb2/~dmtmp v50>,<0x10000000007/client5 v2084>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003d3d:[<0x10000003bbc/PARADOX v6073>,<0x10000003bb2/~dmtmp v920>,<0x10000000007/client5 v6025>,<0x10000000006/clients v2254>,<0x10000000005/tmp v2137>,<0x10000000003/client.0 v2050>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1989>,<0x10000000001/sv_0 v2000>,<0x10000000000/qa v1933>,<0x1/volumes v1903>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:54:52.939096+0000 21=21+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:54:52.968028+0000 b5201920 21=20+1)","memory_value.dirstat":"f(v210 m2024-03-26T17:54:52.968028+0000 20=20+0)","memory_value.rstat":"n(v20 rc2024-03-26T17:54:52.968028+0000 b5201920 21=20+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T17:55:00.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:00 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:00.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:00 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:00.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 29379 7.10 MB/sec execute 103 sec latency 170.064 ms 2024-03-26T17:55:00.593 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:00 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:00.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:00 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:00.637 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:00 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:00.637 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:00 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:55:01.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:01.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:01.411 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:01.412 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.412 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.412 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:01.412 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:01.412 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.412 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.413 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003d3d (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:55:01.413 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.413 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:01 smithi135 ceph-mon[35636]: pgmap v849: 97 pgs: 97 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-03-26T17:55:01.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 29530 7.09 MB/sec execute 104 sec latency 123.115 ms 2024-03-26T17:55:01.593 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.595 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003d3d (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:55:01.595 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.595 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:01 smithi144 ceph-mon[36095]: pgmap v849: 97 pgs: 97 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-03-26T17:55:01.638 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:55:01.638 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.638 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:01.638 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003d3d (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:01.639 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:01 smithi105 ceph-mon[32280]: pgmap v849: 97 pgs: 97 active+clean; 263 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-03-26T17:55:02.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 29688 7.07 MB/sec execute 105 sec latency 113.164 ms 2024-03-26T17:55:03.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 29832 7.04 MB/sec execute 106 sec latency 117.468 ms 2024-03-26T17:55:04.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:03 smithi105 ceph-mon[32280]: pgmap v850: 97 pgs: 97 active+clean; 264 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-03-26T17:55:04.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:03 smithi135 ceph-mon[35636]: pgmap v850: 97 pgs: 97 active+clean; 264 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-03-26T17:55:04.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:03 smithi144 ceph-mon[36095]: pgmap v850: 97 pgs: 97 active+clean; 264 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.0 MiB/s wr, 258 op/s 2024-03-26T17:55:04.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 29966 7.01 MB/sec execute 107 sec latency 188.390 ms 2024-03-26T17:55:05.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 30123 6.98 MB/sec execute 108 sec latency 107.428 ms 2024-03-26T17:55:06.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:05 smithi105 ceph-mon[32280]: pgmap v851: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-26T17:55:06.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:05 smithi135 ceph-mon[35636]: pgmap v851: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-26T17:55:06.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:05 smithi144 ceph-mon[36095]: pgmap v851: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-03-26T17:55:06.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 30262 6.97 MB/sec execute 109 sec latency 103.318 ms 2024-03-26T17:55:07.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 30389 6.97 MB/sec execute 110 sec latency 125.457 ms 2024-03-26T17:55:08.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:07 smithi105 ceph-mon[32280]: pgmap v852: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 2.7 MiB/s wr, 307 op/s 2024-03-26T17:55:08.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:07 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:08.144 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:07 smithi144 ceph-mon[36095]: pgmap v852: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 2.7 MiB/s wr, 307 op/s 2024-03-26T17:55:08.144 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:07 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:07 smithi135 ceph-mon[35636]: pgmap v852: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 2.7 MiB/s wr, 307 op/s 2024-03-26T17:55:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:07 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:55:08.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 30521 6.98 MB/sec execute 111 sec latency 129.090 ms 2024-03-26T17:55:09.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:09.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:09.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:55:09.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:09.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:09.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:55:09.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:55:09.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:55:09.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:55:09.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:55:09.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 30710 7.07 MB/sec execute 112 sec latency 120.556 ms 2024-03-26T17:55:10.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:09 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-03-26T17:55:10.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:09 smithi105 ceph-mon[32280]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-03-26T17:55:10.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:09 smithi105 ceph-mon[32280]: pgmap v853: 97 pgs: 97 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-03-26T17:55:10.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:09 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-03-26T17:55:10.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:09 smithi135 ceph-mon[35636]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-03-26T17:55:10.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:09 smithi135 ceph-mon[35636]: pgmap v853: 97 pgs: 97 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-03-26T17:55:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:09 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi105 to 176.2M 2024-03-26T17:55:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:09 smithi144 ceph-mon[36095]: Unable to set osd_memory_target on smithi105 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-03-26T17:55:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:09 smithi144 ceph-mon[36095]: pgmap v853: 97 pgs: 97 active+clean; 266 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.1 MiB/s wr, 319 op/s 2024-03-26T17:55:10.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 30873 7.13 MB/sec execute 113 sec latency 131.770 ms 2024-03-26T17:55:11.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 31075 7.18 MB/sec execute 114 sec latency 122.629 ms 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/3390137940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081007e0 msgr2=0x7f8a08100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/3390137940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081007e0 0x7f8a08100c00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009900 tx=0x7f89f802bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/3390137940 shutdown_connections 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/3390137940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a08102390 0x7f8a08108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/3390137940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a081019d0 0x7f8a08101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/3390137940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081007e0 0x7f8a08100c00 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/3390137940 >> 172.21.15.105:0/3390137940 conn(0x7f8a080fbd50 msgr2=0x7f8a080fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/3390137940 shutdown_connections 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.062+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/3390137940 wait complete. 2024-03-26T17:55:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.063+0000 7f8a0e5b2640 1 Processor -- start 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.063+0000 7f8a0e5b2640 1 -- start start 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.063+0000 7f8a0e5b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a081007e0 0x7f8a08197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.063+0000 7f8a0e5b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081019d0 0x7f8a08198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.063+0000 7f8a0e5b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a08102390 0x7f8a0819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.063+0000 7f8a0e5b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0810c300 con 0x7f8a081019d0 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.063+0000 7f8a0e5b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0810c180 con 0x7f8a08102390 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.063+0000 7f8a0e5b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0810c480 con 0x7f8a081007e0 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081019d0 0x7f8a08198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a081007e0 0x7f8a08197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081019d0 0x7f8a08198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49894/0 (socket says 172.21.15.105:49894) 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a081007e0 0x7f8a08197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43142/0 (socket says 172.21.15.105:43142) 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 -- 172.21.15.105:0/847104229 learned_addr learned my addr 172.21.15.105:0/847104229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a0cb4f640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a08102390 0x7f8a0819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 -- 172.21.15.105:0/847104229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a081007e0 msgr2=0x7f8a08197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a081007e0 0x7f8a08197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 -- 172.21.15.105:0/847104229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a08102390 msgr2=0x7f8a0819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a08102390 0x7f8a0819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 -- 172.21.15.105:0/847104229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0819f6e0 con 0x7f8a081019d0 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a077fe640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081019d0 0x7f8a08198000 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00e9f0 tx=0x7f89fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a057fa640 1 -- 172.21.15.105:0/847104229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00cd80 con 0x7f8a081019d0 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a057fa640 1 -- 172.21.15.105:0/847104229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc0054d0 con 0x7f8a081019d0 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.064+0000 7f8a057fa640 1 -- 172.21.15.105:0/847104229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc010490 con 0x7f8a081019d0 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.065+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/847104229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a08102c50 con 0x7f8a081019d0 2024-03-26T17:55:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.065+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/847104229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a08103150 con 0x7f8a081019d0 2024-03-26T17:55:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.067+0000 7f8a057fa640 1 -- 172.21.15.105:0/847104229 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f89fc005020 con 0x7f8a081019d0 2024-03-26T17:55:12.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a1119e740 1 Processor -- start 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a1119e740 1 -- start start 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e49165d0 0x5623e49169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e491bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e491e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a1119e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5623e487b0b0 con 0x5623e4916fc0 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a1119e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5623e4823960 con 0x5623e491c130 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a1119e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5623e4877060 con 0x5623e49165d0 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a0cb4f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e49165d0 0x5623e49169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e491bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a0cb4f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e49165d0 0x5623e49169f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43144/0 (socket says 172.21.15.105:43144) 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a0cb4f640 1 -- 172.21.15.105:0/2917044643 learned_addr learned my addr 172.21.15.105:0/2917044643 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a077fe640 1 --2- 172.21.15.105:0/2917044643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e491e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.068+0000 7f8a057fa640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89f00a3040 0x7f89f00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a0cb4f640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89f00a3040 0x7f89f00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a077fe640 1 -- 172.21.15.105:0/2917044643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e49165d0 msgr2=0x5623e49169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a077fe640 1 --2- 172.21.15.105:0/2917044643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e49165d0 0x5623e49169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a077fe640 1 -- 172.21.15.105:0/2917044643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 msgr2=0x5623e491bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a077fe640 1 --2- 172.21.15.105:0/2917044643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e491bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a077fe640 1 -- 172.21.15.105:0/2917044643 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e4835c60 con 0x5623e491c130 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a057fa640 1 -- 172.21.15.105:0/847104229 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f89fc0cf700 con 0x7f8a081019d0 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a077fe640 1 --2- 172.21.15.105:0/2917044643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e491e5c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0cd210 tx=0x7f89fc01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c706ec4e16209a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f8a0cb4f640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89f00a3040 0x7f89f00a5500 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x5623e4917820 tx=0x7f8a00004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f89ee7fc640 1 -- 172.21.15.105:0/2917044643 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc01c420 con 0x5623e491c130 2024-03-26T17:55:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f89ee7fc640 1 -- 172.21.15.105:0/2917044643 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc005310 con 0x5623e491c130 2024-03-26T17:55:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.069+0000 7f89ee7fc640 1 -- 172.21.15.105:0/2917044643 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc01e2b0 con 0x5623e491c130 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 -- 172.21.15.105:0/2917044643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 msgr2=0x5623e491e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2917044643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e491e5c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0cd210 tx=0x7f89fc01d000 comp rx=0 tx=0).stop 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 -- 172.21.15.105:0/2917044643 shutdown_connections 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2917044643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e491e5c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2917044643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e491bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2917044643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e49165d0 0x5623e49169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 -- 172.21.15.105:0/2917044643 >> 172.21.15.105:0/2917044643 conn(0x5623e4911b40 msgr2=0x5623e4911f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 -- 172.21.15.105:0/2917044643 shutdown_connections 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.070+0000 7f8a1119e740 1 -- 172.21.15.105:0/2917044643 wait complete. 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.071+0000 7f8a1119e740 1 -- start start 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.071+0000 7f8a1119e740 1 Processor -- start 2024-03-26T17:55:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.071+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e4920350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.071+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4920890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.071+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e491c130 0x5623e49278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e4920350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a1119e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5623e491f080 con 0x5623e491c130 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a0cb4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e491c130 0x5623e49278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e4920350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:37770/0 (socket says 172.21.15.105:37770) 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 -- 172.21.15.105:0/3812336368 learned_addr learned my addr 172.21.15.105:0/3812336368 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a077fe640 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4920890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5623e4836db0 con 0x5623e49165d0 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5623e491f200 con 0x5623e4916fc0 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 -- 172.21.15.105:0/3812336368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 msgr2=0x5623e4920890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4920890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 -- 172.21.15.105:0/3812336368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e491c130 msgr2=0x5623e49278d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e491c130 0x5623e49278d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 -- 172.21.15.105:0/3812336368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e458f210 con 0x5623e49165d0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a1119e740 10 client.0 _ll_register_callbacks cb 0x5623e483b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623e48885f0 con 0x5623e49165d0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a077fe640 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4920890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a0cb4f640 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e491c130 0x5623e49278d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f8a07fff640 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e4920350 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009900 tx=0x7f89f8004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b4ca777de06642 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f89e77fe640 10 client.31352 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.072+0000 7f89e77fe640 1 -- 172.21.15.105:0/3812336368 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8005d50 con 0x5623e49165d0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f89e77fe640 1 -- 172.21.15.105:0/3812336368 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f89f8005230 con 0x5623e49165d0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f89e77fe640 1 -- 172.21.15.105:0/3812336368 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f89f8005430 con 0x5623e49165d0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f8a1119e740 10 client.31352 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5623e465b620 con 0x5623e49165d0 2024-03-26T17:55:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f89e77fe640 1 -- 172.21.15.105:0/3812336368 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f89f804f030 con 0x5623e49165d0 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f8a1119e740 10 client.31352 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f8a1119e740 10 client.31352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f8a1119e740 20 client.31352 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f8a1119e740 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5623e492c090 0x5623e492e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f8a1119e740 4 client.31352 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.073+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623e4882940 con 0x5623e492c090 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.074+0000 7f8a077fe640 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5623e492c090 0x5623e492e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.074+0000 7f89e77fe640 10 client.31352 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:12.066 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.074+0000 7f8a077fe640 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5623e492c090 0x5623e492e4a0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3dc4589b57ce84 server_cookie=480ba0834680bf5d in_seq=0 out_seq=0 2024-03-26T17:55:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f89e77fe640 1 -- 172.21.15.105:0/3812336368 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5623e4882940 con 0x5623e492c090 2024-03-26T17:55:12.067 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f89e77fe640 10 client.31352 handle_command_reply: tid=0 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 client.31352 shutdown 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 msgr2=0x5623e4920350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e4920350 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009900 tx=0x7f89f8004250 comp rx=0 tx=0).stop 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 shutdown_connections 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5623e492c090 0x5623e492e4a0 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e491c130 0x5623e49278d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4920890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 --2- 172.21.15.105:0/3812336368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e4920350 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 >> 172.21.15.105:0/3812336368 conn(0x5623e4911b40 msgr2=0x5623e4910df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f89e77fe640 0 client.31352 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 shutdown_connections 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 1 -- 172.21.15.105:0/3812336368 wait complete. 2024-03-26T17:55:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.076+0000 7f8a1119e740 20 client.31352 trim_cache size 0 max 16384 2024-03-26T17:55:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.097+0000 7f8a1119e740 1 Processor -- start 2024-03-26T17:55:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.097+0000 7f8a1119e740 1 -- start start 2024-03-26T17:55:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e491bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e4927890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e491c130 0x5623e4910560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a1119e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081a6b50 con 0x5623e4916fc0 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a1119e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081a8690 con 0x5623e49165d0 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a1119e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081a7220 con 0x5623e491c130 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e4927890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e4927890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49916/0 (socket says 172.21.15.105:49916) 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a0cb4f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e491c130 0x5623e4910560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 -- 172.21.15.105:0/4023942771 learned_addr learned my addr 172.21.15.105:0/4023942771 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a07fff640 1 --2- 172.21.15.105:0/4023942771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e491bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 -- 172.21.15.105:0/4023942771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e491c130 msgr2=0x5623e4910560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 --2- 172.21.15.105:0/4023942771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e491c130 0x5623e4910560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 -- 172.21.15.105:0/4023942771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 msgr2=0x5623e491bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 --2- 172.21.15.105:0/4023942771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e491bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 -- 172.21.15.105:0/4023942771 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e49173a0 con 0x5623e4916fc0 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f8a077fe640 1 --2- 172.21.15.105:0/4023942771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e4927890 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f89fc098250 tx=0x7f89fc0bc110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72f91574b23997b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.098+0000 7f89edffb640 1 -- 172.21.15.105:0/4023942771 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc09c4f0 con 0x5623e4916fc0 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.099+0000 7f89edffb640 1 -- 172.21.15.105:0/4023942771 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc09ff40 con 0x5623e4916fc0 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.099+0000 7f89edffb640 1 -- 172.21.15.105:0/4023942771 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc098460 con 0x5623e4916fc0 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.099+0000 7f8a1119e740 1 -- 172.21.15.105:0/4023942771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 msgr2=0x5623e4927890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.099+0000 7f8a1119e740 1 --2- 172.21.15.105:0/4023942771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e4927890 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f89fc098250 tx=0x7f89fc0bc110 comp rx=0 tx=0).stop 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 -- 172.21.15.105:0/4023942771 shutdown_connections 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 --2- 172.21.15.105:0/4023942771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e491c130 0x5623e4910560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 --2- 172.21.15.105:0/4023942771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e4916fc0 0x5623e4927890 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 --2- 172.21.15.105:0/4023942771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e49165d0 0x5623e491bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 -- 172.21.15.105:0/4023942771 >> 172.21.15.105:0/4023942771 conn(0x5623e4911b40 msgr2=0x5623e491b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 -- 172.21.15.105:0/4023942771 shutdown_connections 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 -- 172.21.15.105:0/4023942771 wait complete. 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 -- start start 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.100+0000 7f8a1119e740 1 Processor -- start 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e49165d0 0x5623e4927160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4927700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e49165d0 0x5623e4927160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e49165d0 0x5623e4927160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49924/0 (socket says 172.21.15.105:49924) 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a077fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4927700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a077fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4927700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43168/0 (socket says 172.21.15.105:43168) 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a1119e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e4923e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a07fff640 1 -- 172.21.15.105:0/2332303158 learned_addr learned my addr 172.21.15.105:0/2332303158 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081a7fc0 con 0x5623e49165d0 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081a78f0 con 0x5623e491c130 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081a4b30 con 0x5623e4916fc0 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a0cb4f640 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e4923e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.101+0000 7f8a1119e740 10 client.0 _ll_register_callbacks cb 0x7f89f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a077fe640 1 -- 172.21.15.105:0/2332303158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 msgr2=0x5623e4923e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a077fe640 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e4923e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a077fe640 1 -- 172.21.15.105:0/2332303158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e49165d0 msgr2=0x5623e4927160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a077fe640 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e49165d0 0x5623e4927160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a077fe640 1 -- 172.21.15.105:0/2332303158 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e4924630 con 0x5623e4916fc0 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a07fff640 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e49165d0 0x5623e4927160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a077fe640 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4927700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f89fc09bfb0 tx=0x7f89fc00c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6940672b616ac948 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f89e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f89e77fe640 1 -- 172.21.15.105:0/2332303158 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0b4070 con 0x5623e4916fc0 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f89e77fe640 1 -- 172.21.15.105:0/2332303158 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f89fc098460 con 0x5623e4916fc0 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89fc09ff40 con 0x5623e4916fc0 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f89e77fe640 1 -- 172.21.15.105:0/2332303158 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f89fc0b7050 con 0x5623e4916fc0 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.102+0000 7f8a1119e740 10 client.31333 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5623e4990fd0 con 0x5623e4916fc0 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f89e77fe640 1 -- 172.21.15.105:0/2332303158 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89fc0a4800 con 0x5623e4916fc0 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f89e77fe640 1 -- 172.21.15.105:0/2332303158 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f89fc00b2c0 con 0x5623e4916fc0 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f8a1119e740 10 client.31333 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f8a1119e740 10 client.31333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f8a1119e740 20 client.31333 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5623e4991470 0x5623e4993860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f8a1119e740 4 client.31333 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f89fc09ff40 con 0x5623e4991470 2024-03-26T17:55:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.103+0000 7f8a07fff640 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5623e4991470 0x5623e4993860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.104+0000 7f89e77fe640 10 client.31333 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.105+0000 7f8a07fff640 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5623e4991470 0x5623e4993860 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a441ff23429553 server_cookie=997dbf89375f233c in_seq=0 out_seq=0 2024-03-26T17:55:12.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:11 smithi105 ceph-mon[32280]: pgmap v854: 97 pgs: 97 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-03-26T17:55:12.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:11 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:55:12.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:11 smithi135 ceph-mon[35636]: pgmap v854: 97 pgs: 97 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-03-26T17:55:12.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:11 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f89e77fe640 1 -- 172.21.15.105:0/2332303158 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f89fc09ff40 con 0x5623e4991470 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f89e77fe640 10 client.31333 handle_command_reply: tid=0 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 client.31333 shutdown 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 msgr2=0x5623e4927700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4927700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f89fc09bfb0 tx=0x7f89fc00c840 comp rx=0 tx=0).stop 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 shutdown_connections 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5623e4991470 0x5623e4993860 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623e491c130 0x5623e4923e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5623e4916fc0 0x5623e4927700 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f89e77fe640 0 client.31333 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 --2- 172.21.15.105:0/2332303158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5623e49165d0 0x5623e4927160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 >> 172.21.15.105:0/2332303158 conn(0x5623e4911b40 msgr2=0x5623e4914da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 shutdown_connections 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 1 -- 172.21.15.105:0/2332303158 wait complete. 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.262+0000 7f8a1119e740 20 client.31333 trim_cache size 0 max 16384 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:55:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:55:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89f00a3040 msgr2=0x7f89f00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89f00a3040 0x7f89f00a5500 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x5623e4917820 tx=0x7f8a00004ab0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081019d0 msgr2=0x7f8a08198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081019d0 0x7f8a08198000 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00e9f0 tx=0x7f89fc00edf0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/847104229 shutdown_connections 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f89f00a3040 0x7f89f00a5500 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a08102390 0x7f8a0819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a081019d0 0x7f8a08198000 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 --2- 172.21.15.105:0/847104229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a081007e0 0x7f8a08197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/847104229 >> 172.21.15.105:0/847104229 conn(0x7f8a080fbd50 msgr2=0x7f8a080fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/847104229 shutdown_connections 2024-03-26T17:55:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:12.263+0000 7f8a0e5b2640 1 -- 172.21.15.105:0/847104229 wait complete. 2024-03-26T17:55:12.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:11 smithi144 ceph-mon[36095]: pgmap v854: 97 pgs: 97 active+clean; 267 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-03-26T17:55:12.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:11 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:55:12.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 31211 7.23 MB/sec execute 115 sec latency 199.085 ms 2024-03-26T17:55:13.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:12 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:55:13.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:12 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:55:13.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:12 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:55:13.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:12 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:55:13.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:12 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:55:13.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:12 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:55:13.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:12 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:55:13.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:12 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:55:13.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:12 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:55:13.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 31336 7.25 MB/sec execute 116 sec latency 131.884 ms 2024-03-26T17:55:14.162 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:13 smithi135 ceph-mon[35636]: pgmap v855: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-03-26T17:55:14.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:13 smithi144 ceph-mon[36095]: pgmap v855: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-03-26T17:55:14.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:13 smithi105 ceph-mon[32280]: pgmap v855: 97 pgs: 97 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.2 MiB/s wr, 287 op/s 2024-03-26T17:55:14.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 31423 7.23 MB/sec execute 117 sec latency 125.929 ms 2024-03-26T17:55:15.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 31497 7.19 MB/sec execute 118 sec latency 113.946 ms 2024-03-26T17:55:16.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:15 smithi135 ceph-mon[35636]: pgmap v856: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-03-26T17:55:16.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:15 smithi144 ceph-mon[36095]: pgmap v856: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-03-26T17:55:16.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:15 smithi105 ceph-mon[32280]: pgmap v856: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-03-26T17:55:16.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 31616 7.21 MB/sec execute 119 sec latency 115.765 ms 2024-03-26T17:55:17.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 31753 7.24 MB/sec execute 120 sec latency 132.609 ms 2024-03-26T17:55:18.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:17 smithi135 ceph-mon[35636]: pgmap v857: 97 pgs: 97 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.8 MiB/s wr, 234 op/s 2024-03-26T17:55:18.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:17 smithi144 ceph-mon[36095]: pgmap v857: 97 pgs: 97 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.8 MiB/s wr, 234 op/s 2024-03-26T17:55:18.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:17 smithi105 ceph-mon[32280]: pgmap v857: 97 pgs: 97 active+clean; 286 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.8 MiB/s wr, 234 op/s 2024-03-26T17:55:18.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 31877 7.26 MB/sec execute 121 sec latency 212.262 ms 2024-03-26T17:55:19.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:18 smithi135 ceph-mon[35636]: pgmap v858: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 4.3 MiB/s wr, 226 op/s 2024-03-26T17:55:19.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:18 smithi144 ceph-mon[36095]: pgmap v858: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 4.3 MiB/s wr, 226 op/s 2024-03-26T17:55:19.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:18 smithi105 ceph-mon[32280]: pgmap v858: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 4.3 MiB/s wr, 226 op/s 2024-03-26T17:55:19.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32003 7.23 MB/sec execute 122 sec latency 204.930 ms 2024-03-26T17:55:20.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32129 7.26 MB/sec execute 123 sec latency 177.642 ms 2024-03-26T17:55:21.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32227 7.21 MB/sec execute 124 sec latency 156.603 ms 2024-03-26T17:55:22.136 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:21 smithi105 ceph-mon[32280]: pgmap v859: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.8 MiB/s wr, 215 op/s 2024-03-26T17:55:22.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:21 smithi135 ceph-mon[35636]: pgmap v859: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.8 MiB/s wr, 215 op/s 2024-03-26T17:55:22.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:21 smithi144 ceph-mon[36095]: pgmap v859: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.8 MiB/s wr, 215 op/s 2024-03-26T17:55:22.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32381 7.24 MB/sec execute 125 sec latency 140.145 ms 2024-03-26T17:55:23.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32474 7.20 MB/sec execute 126 sec latency 137.431 ms 2024-03-26T17:55:24.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:23 smithi105 ceph-mon[32280]: pgmap v860: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 4.2 MiB/s wr, 219 op/s 2024-03-26T17:55:24.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:23 smithi135 ceph-mon[35636]: pgmap v860: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 4.2 MiB/s wr, 219 op/s 2024-03-26T17:55:24.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:23 smithi144 ceph-mon[36095]: pgmap v860: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 4.2 MiB/s wr, 219 op/s 2024-03-26T17:55:24.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32586 7.19 MB/sec execute 127 sec latency 123.091 ms 2024-03-26T17:55:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 -- 172.21.15.105:0/3498689028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 msgr2=0x7ff440104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 --2- 172.21.15.105:0/3498689028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 0x7ff440104fb0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009900 tx=0x7ff43c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 -- 172.21.15.105:0/3498689028 shutdown_connections 2024-03-26T17:55:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 --2- 172.21.15.105:0/3498689028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4401054f0 0x7ff44010bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 --2- 172.21.15.105:0/3498689028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 0x7ff440104fb0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 --2- 172.21.15.105:0/3498689028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff440103940 0x7ff440103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 -- 172.21.15.105:0/3498689028 >> 172.21.15.105:0/3498689028 conn(0x7ff4400feed0 msgr2=0x7ff4401012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 -- 172.21.15.105:0/3498689028 shutdown_connections 2024-03-26T17:55:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.436+0000 7ff446cd9640 1 -- 172.21.15.105:0/3498689028 wait complete. 2024-03-26T17:55:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff446cd9640 1 Processor -- start 2024-03-26T17:55:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff446cd9640 1 -- start start 2024-03-26T17:55:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff446cd9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff440103940 0x7ff44019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff446cd9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 0x7ff44019e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff446cd9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4401054f0 0x7ff4401a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff446cd9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff440076f10 con 0x7ff440104b30 2024-03-26T17:55:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff446cd9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff440076d90 con 0x7ff440103940 2024-03-26T17:55:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff446cd9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff440077090 con 0x7ff4401054f0 2024-03-26T17:55:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff4464d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4401054f0 0x7ff4401a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4464d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4401054f0 0x7ff4401a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53088/0 (socket says 172.21.15.105:53088) 2024-03-26T17:55:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4464d8640 1 -- 172.21.15.105:0/4188686373 learned_addr learned my addr 172.21.15.105:0/4188686373 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4454d6640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 0x7ff44019e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.437+0000 7ff445cd7640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff440103940 0x7ff44019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4454d6640 1 -- 172.21.15.105:0/4188686373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4401054f0 msgr2=0x7ff4401a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4454d6640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4401054f0 0x7ff4401a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4454d6640 1 -- 172.21.15.105:0/4188686373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff440103940 msgr2=0x7ff44019dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4454d6640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff440103940 0x7ff44019dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4454d6640 1 -- 172.21.15.105:0/4188686373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4401a5be0 con 0x7ff440104b30 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff445cd7640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff440103940 0x7ff44019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.438+0000 7ff4454d6640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 0x7ff44019e500 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ff43c02c170 tx=0x7ff43c004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.439+0000 7ff436ffd640 1 -- 172.21.15.105:0/4188686373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c043020 con 0x7ff440104b30 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.439+0000 7ff436ffd640 1 -- 172.21.15.105:0/4188686373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff43c0052b0 con 0x7ff440104b30 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.439+0000 7ff446cd9640 1 -- 172.21.15.105:0/4188686373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4401a5e70 con 0x7ff440104b30 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.439+0000 7ff446cd9640 1 -- 172.21.15.105:0/4188686373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4401a6400 con 0x7ff440104b30 2024-03-26T17:55:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.439+0000 7ff436ffd640 1 -- 172.21.15.105:0/4188686373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c003df0 con 0x7ff440104b30 2024-03-26T17:55:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.444+0000 7ff44ac09740 1 Processor -- start 2024-03-26T17:55:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.444+0000 7ff436ffd640 1 -- 172.21.15.105:0/4188686373 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff43c046020 con 0x7ff440104b30 2024-03-26T17:55:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff44ac09740 1 -- start start 2024-03-26T17:55:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b801790 0x55a28b801bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b806c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b8072f0 0x55a28b809780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff44ac09740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a28b7660b0 con 0x55a28b802180 2024-03-26T17:55:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff44ac09740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a28b70e960 con 0x55a28b8072f0 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff44ac09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a28b762060 con 0x55a28b801790 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff4464d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b801790 0x55a28b801bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff4464d8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b801790 0x55a28b801bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53096/0 (socket says 172.21.15.105:53096) 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff4464d8640 1 -- 172.21.15.105:0/1324641534 learned_addr learned my addr 172.21.15.105:0/1324641534 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff4454d6640 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b8072f0 0x55a28b809780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.445+0000 7ff445cd7640 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b806c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff4464d8640 1 -- 172.21.15.105:0/1324641534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b8072f0 msgr2=0x55a28b809780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff4464d8640 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b8072f0 0x55a28b809780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff4464d8640 1 -- 172.21.15.105:0/1324641534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 msgr2=0x55a28b806c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff4464d8640 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b806c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff4464d8640 1 -- 172.21.15.105:0/1324641534 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28b720c60 con 0x55a28b801790 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff445cd7640 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b806c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:55:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff4464d8640 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b801790 0x55a28b801bb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff430009aa0 tx=0x7ff430009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be6b9bb09bc964ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff42bfff640 1 -- 172.21.15.105:0/1324641534 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430017070 con 0x55a28b801790 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff42bfff640 1 -- 172.21.15.105:0/1324641534 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff430013470 con 0x55a28b801790 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.446+0000 7ff42bfff640 1 -- 172.21.15.105:0/1324641534 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430012610 con 0x55a28b801790 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff436ffd640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff42c0a2f90 0x7ff42c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff445cd7640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff42c0a2f90 0x7ff42c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff436ffd640 1 -- 172.21.15.105:0/4188686373 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff43c0f1630 con 0x7ff440104b30 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff44ac09740 1 -- 172.21.15.105:0/1324641534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b801790 msgr2=0x55a28b801bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b801790 0x55a28b801bb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff430009aa0 tx=0x7ff430009f60 comp rx=0 tx=0).stop 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff44ac09740 1 -- 172.21.15.105:0/1324641534 shutdown_connections 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b8072f0 0x55a28b809780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b806c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1324641534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b801790 0x55a28b801bb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff44ac09740 1 -- 172.21.15.105:0/1324641534 >> 172.21.15.105:0/1324641534 conn(0x55a28b7fcd00 msgr2=0x55a28b7fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff44ac09740 1 -- 172.21.15.105:0/1324641534 shutdown_connections 2024-03-26T17:55:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.447+0000 7ff445cd7640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff42c0a2f90 0x7ff42c0a5450 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff43800afd0 tx=0x7ff43800a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.448+0000 7ff44ac09740 1 -- 172.21.15.105:0/1324641534 wait complete. 2024-03-26T17:55:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.448+0000 7ff44ac09740 1 -- start start 2024-03-26T17:55:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.448+0000 7ff44ac09740 1 Processor -- start 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b80bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b812af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff44ac09740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a28b80a2a0 con 0x55a28b8072f0 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff44ac09740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a28b721db0 con 0x55a28b801790 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff44ac09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a28b80a420 con 0x55a28b802180 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff44ac09740 10 client.0 _ll_register_callbacks cb 0x55a28b726110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff4454d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b80bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48812/0 (socket says 172.21.15.105:48812) 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 -- 172.21.15.105:0/1148774566 learned_addr learned my addr 172.21.15.105:0/1148774566 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff4464d8640 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b812af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 -- 172.21.15.105:0/1148774566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 msgr2=0x55a28b80bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b80bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 -- 172.21.15.105:0/1148774566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 msgr2=0x55a28b812af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b812af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 -- 172.21.15.105:0/1148774566 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28b47a210 con 0x55a28b801790 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff4464d8640 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b812af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff445cd7640 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80b570 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4380116c0 tx=0x7ff438004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea5c51b17c813457 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.449+0000 7ff4297fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.450+0000 7ff4297fa640 1 -- 172.21.15.105:0/1148774566 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438017070 con 0x55a28b801790 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.450+0000 7ff4297fa640 1 -- 172.21.15.105:0/1148774566 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff43801c420 con 0x55a28b801790 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.450+0000 7ff44ac09740 1 -- 172.21.15.105:0/1148774566 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff430013470 con 0x55a28b801790 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.450+0000 7ff4297fa640 1 -- 172.21.15.105:0/1148774566 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff438026630 con 0x55a28b801790 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.450+0000 7ff44ac09740 10 client.31367 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.450+0000 7ff44ac09740 1 -- 172.21.15.105:0/1148774566 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a28b546620 con 0x55a28b801790 2024-03-26T17:55:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff4297fa640 1 -- 172.21.15.105:0/1148774566 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff430013470 con 0x55a28b801790 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff4297fa640 1 -- 172.21.15.105:0/1148774566 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff43801ca70 con 0x55a28b801790 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff44ac09740 10 client.31367 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff44ac09740 10 client.31367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff44ac09740 20 client.31367 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28b8173d0 0x55a28b819860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff44ac09740 4 client.31367 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff44ac09740 1 -- 172.21.15.105:0/1148774566 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a28b7735f0 con 0x55a28b8173d0 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.451+0000 7ff4454d6640 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28b8173d0 0x55a28b819860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.452+0000 7ff4297fa640 10 client.31367 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.452+0000 7ff4454d6640 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28b8173d0 0x55a28b819860 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c6d4912843b7e1 server_cookie=1a0fab3c0d278fba in_seq=0 out_seq=0 2024-03-26T17:55:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.454+0000 7ff4297fa640 1 -- 172.21.15.105:0/1148774566 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a28b7735f0 con 0x55a28b8173d0 2024-03-26T17:55:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.454+0000 7ff4297fa640 10 client.31367 handle_command_reply: tid=0 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 client.31367 shutdown 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 -- 172.21.15.105:0/1148774566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 msgr2=0x55a28b80b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80b570 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4380116c0 tx=0x7ff438004a40 comp rx=0 tx=0).stop 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 -- 172.21.15.105:0/1148774566 shutdown_connections 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28b8173d0 0x55a28b819860 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b812af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b80bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 --2- 172.21.15.105:0/1148774566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80b570 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 -- 172.21.15.105:0/1148774566 >> 172.21.15.105:0/1148774566 conn(0x55a28b7fcd00 msgr2=0x55a28b7fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff4297fa640 0 client.31367 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 -- 172.21.15.105:0/1148774566 shutdown_connections 2024-03-26T17:55:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 1 -- 172.21.15.105:0/1148774566 wait complete. 2024-03-26T17:55:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.455+0000 7ff44ac09740 20 client.31367 trim_cache size 0 max 16384 2024-03-26T17:55:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.477+0000 7ff44ac09740 1 Processor -- start 2024-03-26T17:55:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.477+0000 7ff44ac09740 1 -- start start 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.477+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b803450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.477+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b812a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.477+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b8072f0 0x55a28b806510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff44ac09740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff42400e530 con 0x55a28b802180 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff44ac09740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff424010070 con 0x55a28b801790 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff44ac09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff42400ec00 con 0x55a28b8072f0 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b812a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b812a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37004/0 (socket says 172.21.15.105:37004) 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 -- 172.21.15.105:0/582916332 learned_addr learned my addr 172.21.15.105:0/582916332 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff445cd7640 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b803450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4464d8640 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b8072f0 0x55a28b806510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 -- 172.21.15.105:0/582916332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b8072f0 msgr2=0x55a28b806510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b8072f0 0x55a28b806510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 -- 172.21.15.105:0/582916332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 msgr2=0x55a28b803450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b803450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 -- 172.21.15.105:0/582916332 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28b806bf0 con 0x55a28b802180 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4464d8640 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b8072f0 0x55a28b806510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff4454d6640 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b812a80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0bdb40 tx=0x7ff43c0f3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22617d01d3268d7f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff42b7fe640 1 -- 172.21.15.105:0/582916332 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0c0070 con 0x55a28b802180 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff42b7fe640 1 -- 172.21.15.105:0/582916332 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff43c0be540 con 0x55a28b802180 2024-03-26T17:55:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.478+0000 7ff42b7fe640 1 -- 172.21.15.105:0/582916332 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c0bc1d0 con 0x55a28b802180 2024-03-26T17:55:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 -- 172.21.15.105:0/582916332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 msgr2=0x55a28b812a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b812a80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0bdb40 tx=0x7ff43c0f3040 comp rx=0 tx=0).stop 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 -- 172.21.15.105:0/582916332 shutdown_connections 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b8072f0 0x55a28b806510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b802180 0x55a28b812a80 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 --2- 172.21.15.105:0/582916332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b803450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 -- 172.21.15.105:0/582916332 >> 172.21.15.105:0/582916332 conn(0x55a28b7fcd00 msgr2=0x55a28b80b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 -- 172.21.15.105:0/582916332 shutdown_connections 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.479+0000 7ff44ac09740 1 -- 172.21.15.105:0/582916332 wait complete. 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 1 -- start start 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 1 Processor -- start 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b810150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b811160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff42400f9a0 con 0x55a28b8072f0 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff42400f2d0 con 0x55a28b801790 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff42400c510 con 0x55a28b802180 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff445cd7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff4464d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b811160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff4454d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b810150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff445cd7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48830/0 (socket says 172.21.15.105:48830) 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff445cd7640 1 -- 172.21.15.105:0/2738326087 learned_addr learned my addr 172.21.15.105:0/2738326087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff4454d6640 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b810150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53118/0 (socket says 172.21.15.105:53118) 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.480+0000 7ff44ac09740 10 client.0 _ll_register_callbacks cb 0x7ff43800e6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff445cd7640 1 -- 172.21.15.105:0/2738326087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 msgr2=0x55a28b810150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff445cd7640 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b810150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff445cd7640 1 -- 172.21.15.105:0/2738326087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 msgr2=0x55a28b811160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff445cd7640 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b811160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff445cd7640 1 -- 172.21.15.105:0/2738326087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28b811750 con 0x55a28b801790 2024-03-26T17:55:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff4464d8640 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b811160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff445cd7640 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80fc10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff43801cf10 tx=0x7ff438004d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0756ac2f7961c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff4297fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff4297fa640 1 -- 172.21.15.105:0/2738326087 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438017070 con 0x55a28b801790 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff44ac09740 1 -- 172.21.15.105:0/2738326087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff43c0be540 con 0x55a28b801790 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.481+0000 7ff4297fa640 1 -- 172.21.15.105:0/2738326087 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff438030bb0 con 0x55a28b801790 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff4297fa640 1 -- 172.21.15.105:0/2738326087 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff438030db0 con 0x55a28b801790 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff44ac09740 10 client.31373 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff44ac09740 1 -- 172.21.15.105:0/2738326087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a28b812ab0 con 0x55a28b801790 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff4297fa640 1 -- 172.21.15.105:0/2738326087 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff43c0be540 con 0x55a28b801790 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff4297fa640 1 -- 172.21.15.105:0/2738326087 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ff43801c4b0 con 0x55a28b801790 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff44ac09740 10 client.31373 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff44ac09740 10 client.31373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff44ac09740 20 client.31373 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff44ac09740 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28b87dc30 0x55a28b880050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff44ac09740 4 client.31373 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:55:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff44ac09740 1 -- 172.21.15.105:0/2738326087 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a28b76d940 con 0x55a28b87dc30 2024-03-26T17:55:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.482+0000 7ff4454d6640 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28b87dc30 0x55a28b880050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.483+0000 7ff4297fa640 10 client.31373 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.483+0000 7ff4454d6640 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28b87dc30 0x55a28b880050 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c33f59f3db703c8 server_cookie=a45724d8190ff17 in_seq=0 out_seq=0 2024-03-26T17:55:25.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32684 7.15 MB/sec execute 128 sec latency 163.164 ms 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff4297fa640 1 -- 172.21.15.105:0/2738326087 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a28b76d940 con 0x55a28b87dc30 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff4297fa640 10 client.31373 handle_command_reply: tid=0 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 client.31373 shutdown 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 -- 172.21.15.105:0/2738326087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 msgr2=0x55a28b80fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80fc10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff43801cf10 tx=0x7ff438004d60 comp rx=0 tx=0).stop 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 -- 172.21.15.105:0/2738326087 shutdown_connections 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28b87dc30 0x55a28b880050 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28b8072f0 0x55a28b811160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff4297fa640 0 client.31373 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28b802180 0x55a28b810150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 --2- 172.21.15.105:0/2738326087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28b801790 0x55a28b80fc10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 -- 172.21.15.105:0/2738326087 >> 172.21.15.105:0/2738326087 conn(0x55a28b7fcd00 msgr2=0x55a28b802a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 -- 172.21.15.105:0/2738326087 shutdown_connections 2024-03-26T17:55:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 1 -- 172.21.15.105:0/2738326087 wait complete. 2024-03-26T17:55:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.630+0000 7ff44ac09740 20 client.31373 trim_cache size 0 max 16384 2024-03-26T17:55:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:55:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:55:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:55:25.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 -- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff42c0a2f90 msgr2=0x7ff42c0a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff42c0a2f90 0x7ff42c0a5450 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff43800afd0 tx=0x7ff43800a590 comp rx=0 tx=0).stop 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 -- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 msgr2=0x7ff44019e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 0x7ff44019e500 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ff43c02c170 tx=0x7ff43c004450 comp rx=0 tx=0).stop 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 -- 172.21.15.105:0/4188686373 shutdown_connections 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff42c0a2f90 0x7ff42c0a5450 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4401054f0 0x7ff4401a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff440104b30 0x7ff44019e500 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 --2- 172.21.15.105:0/4188686373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff440103940 0x7ff44019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 -- 172.21.15.105:0/4188686373 >> 172.21.15.105:0/4188686373 conn(0x7ff4400feed0 msgr2=0x7ff440105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 -- 172.21.15.105:0/4188686373 shutdown_connections 2024-03-26T17:55:25.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:25.631+0000 7ff446cd9640 1 -- 172.21.15.105:0/4188686373 wait complete. 2024-03-26T17:55:26.136 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:25 smithi105 ceph-mon[32280]: pgmap v861: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 3.8 MiB/s wr, 194 op/s 2024-03-26T17:55:26.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:25 smithi135 ceph-mon[35636]: pgmap v861: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 3.8 MiB/s wr, 194 op/s 2024-03-26T17:55:26.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:25 smithi144 ceph-mon[36095]: pgmap v861: 97 pgs: 97 active+clean; 289 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 3.8 MiB/s wr, 194 op/s 2024-03-26T17:55:26.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32799 7.13 MB/sec execute 129 sec latency 234.497 ms 2024-03-26T17:55:27.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 32926 7.10 MB/sec execute 130 sec latency 115.933 ms 2024-03-26T17:55:28.136 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:27 smithi105 ceph-mon[32280]: pgmap v862: 97 pgs: 97 active+clean; 289 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-03-26T17:55:28.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:27 smithi135 ceph-mon[35636]: pgmap v862: 97 pgs: 97 active+clean; 289 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-03-26T17:55:28.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:27 smithi144 ceph-mon[36095]: pgmap v862: 97 pgs: 97 active+clean; 289 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-03-26T17:55:28.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 33050 7.08 MB/sec execute 131 sec latency 218.244 ms 2024-03-26T17:55:29.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 33168 7.07 MB/sec execute 132 sec latency 222.716 ms 2024-03-26T17:55:29.843 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:29 smithi144 ceph-mon[36095]: pgmap v863: 97 pgs: 97 active+clean; 284 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-03-26T17:55:29.886 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:29 smithi105 ceph-mon[32280]: pgmap v863: 97 pgs: 97 active+clean; 284 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-03-26T17:55:29.910 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:29 smithi135 ceph-mon[35636]: pgmap v863: 97 pgs: 97 active+clean; 284 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.5 MiB/s wr, 226 op/s 2024-03-26T17:55:30.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:55:30.238 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:55:30.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 33312 7.09 MB/sec execute 133 sec latency 118.104 ms 2024-03-26T17:55:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 -- 172.21.15.105:0/686537733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 msgr2=0x7fe4f0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 --2- 172.21.15.105:0/686537733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 0x7fe4f0100be0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009900 tx=0x7fe4ec02bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 -- 172.21.15.105:0/686537733 shutdown_connections 2024-03-26T17:55:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 --2- 172.21.15.105:0/686537733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f0102370 0x7fe4f0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 --2- 172.21.15.105:0/686537733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4f01019b0 0x7fe4f0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 --2- 172.21.15.105:0/686537733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 0x7fe4f0100be0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 -- 172.21.15.105:0/686537733 >> 172.21.15.105:0/686537733 conn(0x7fe4f00fbd50 msgr2=0x7fe4f00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:30.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 -- 172.21.15.105:0/686537733 shutdown_connections 2024-03-26T17:55:30.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.717+0000 7fe4f8059640 1 -- 172.21.15.105:0/686537733 wait complete. 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f8059640 1 Processor -- start 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f8059640 1 -- start start 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f8059640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 0x7fe4f00695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f8059640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f01019b0 0x7fe4f019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f8059640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4f0102370 0x7fe4f019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f8059640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f010c2c0 con 0x7fe4f01007c0 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f8059640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f010c140 con 0x7fe4f0102370 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f8059640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f010c440 con 0x7fe4f01019b0 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 0x7fe4f00695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 0x7fe4f00695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37036/0 (socket says 172.21.15.105:37036) 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f5df5640 1 -- 172.21.15.105:0/2500714839 learned_addr learned my addr 172.21.15.105:0/2500714839 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f01019b0 0x7fe4f019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.718+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4f0102370 0x7fe4f019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f5df5640 1 -- 172.21.15.105:0/2500714839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f01019b0 msgr2=0x7fe4f019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f01019b0 0x7fe4f019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f5df5640 1 -- 172.21.15.105:0/2500714839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4f0102370 msgr2=0x7fe4f019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4f0102370 0x7fe4f019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f5df5640 1 -- 172.21.15.105:0/2500714839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f019f760 con 0x7fe4f01007c0 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f01019b0 0x7fe4f019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4f0102370 0x7fe4f019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 0x7fe4f00695b0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4e6ffd640 1 -- 172.21.15.105:0/2500714839 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec043020 con 0x7fe4f01007c0 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4f8059640 1 -- 172.21.15.105:0/2500714839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f019f9f0 con 0x7fe4f01007c0 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4e6ffd640 1 -- 172.21.15.105:0/2500714839 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4ec004350 con 0x7fe4f01007c0 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.719+0000 7fe4e6ffd640 1 -- 172.21.15.105:0/2500714839 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec003b80 con 0x7fe4f01007c0 2024-03-26T17:55:30.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.720+0000 7fe4f8059640 1 -- 172.21.15.105:0/2500714839 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f019ff90 con 0x7fe4f01007c0 2024-03-26T17:55:30.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.722+0000 7fe4e6ffd640 1 -- 172.21.15.105:0/2500714839 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe4ec046020 con 0x7fe4f01007c0 2024-03-26T17:55:30.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4fac43740 1 Processor -- start 2024-03-26T17:55:30.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4fac43740 1 -- start start 2024-03-26T17:55:30.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab345d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3462e0 0x5603ab34ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab34d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4fac43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5603ab2aa980 con 0x5603ab3462e0 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4fac43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603ab252be0 con 0x5603ab34b450 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4fac43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5603ab2a6930 con 0x5603ab3458f0 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3462e0 0x5603ab34ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4f55f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab34d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4f65f6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab345d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4f55f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab34d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48854/0 (socket says 172.21.15.105:48854) 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4f65f6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab345d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53136/0 (socket says 172.21.15.105:53136) 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3462e0 0x5603ab34ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37052/0 (socket says 172.21.15.105:37052) 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.723+0000 7fe4f55f4640 1 -- 172.21.15.105:0/1142372165 learned_addr learned my addr 172.21.15.105:0/1142372165 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4e6ffd640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4f55f4640 1 -- 172.21.15.105:0/1142372165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 msgr2=0x5603ab345d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/1142372165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab345d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4f55f4640 1 -- 172.21.15.105:0/1142372165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3462e0 msgr2=0x5603ab34ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/1142372165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3462e0 0x5603ab34ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4f55f4640 1 -- 172.21.15.105:0/1142372165 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ab0887d0 con 0x5603ab34b450 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4e6ffd640 1 -- 172.21.15.105:0/2500714839 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe4ec0f1230 con 0x7fe4f01007c0 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/1142372165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab34d8e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00114f0 tx=0x7fe4e00119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d217e9f0f7ca554 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4dbfff640 1 -- 172.21.15.105:0/1142372165 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e001e070 con 0x5603ab34b450 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.724+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x5603ab346b40 tx=0x7fe4e8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.725+0000 7fe4dbfff640 1 -- 172.21.15.105:0/1142372165 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4e00035f0 con 0x5603ab34b450 2024-03-26T17:55:30.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.725+0000 7fe4dbfff640 1 -- 172.21.15.105:0/1142372165 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e000f820 con 0x5603ab34b450 2024-03-26T17:55:30.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.725+0000 7fe4fac43740 1 -- 172.21.15.105:0/1142372165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 msgr2=0x5603ab34d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.725+0000 7fe4fac43740 1 --2- 172.21.15.105:0/1142372165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab34d8e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00114f0 tx=0x7fe4e00119b0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 -- 172.21.15.105:0/1142372165 shutdown_connections 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 --2- 172.21.15.105:0/1142372165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab34d8e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 --2- 172.21.15.105:0/1142372165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3462e0 0x5603ab34ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 --2- 172.21.15.105:0/1142372165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab345d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 -- 172.21.15.105:0/1142372165 >> 172.21.15.105:0/1142372165 conn(0x5603ab340e60 msgr2=0x5603ab341260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 -- 172.21.15.105:0/1142372165 shutdown_connections 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 -- 172.21.15.105:0/1142372165 wait complete. 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 -- start start 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.726+0000 7fe4fac43740 1 Processor -- start 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab353f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3462e0 0x5603ab3544b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4fac43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5603ab34e400 con 0x5603ab34b450 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4fac43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603ab266160 con 0x5603ab3462e0 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4fac43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5603ab34e580 con 0x5603ab3458f0 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4fac43740 10 client.0 _ll_register_callbacks cb 0x5603ab2a2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f65f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab353f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f65f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37060/0 (socket says 172.21.15.105:37060) 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f65f6640 1 -- 172.21.15.105:0/50328546 learned_addr learned my addr 172.21.15.105:0/50328546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3462e0 0x5603ab3544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f65f6640 1 -- 172.21.15.105:0/50328546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 msgr2=0x5603ab353f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab353f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f65f6640 1 -- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3462e0 msgr2=0x5603ab3544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:30.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3462e0 0x5603ab3544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f65f6640 1 -- 172.21.15.105:0/50328546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603aafbe3f0 con 0x5603ab34b450 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.727+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3462e0 0x5603ab3544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35b480 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800ff20 tx=0x7fe4e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c540cee0ae9c0937 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4d97fa640 1 -- 172.21.15.105:0/50328546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8021850 con 0x5603ab34b450 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4fac43740 1 -- 172.21.15.105:0/50328546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e00035f0 con 0x5603ab34b450 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4d97fa640 1 -- 172.21.15.105:0/50328546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe4e80219f0 con 0x5603ab34b450 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4d97fa640 1 -- 172.21.15.105:0/50328546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe4e8021bf0 con 0x5603ab34b450 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4fac43740 10 client.21633 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4fac43740 1 -- 172.21.15.105:0/50328546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5603aaf7e0f0 con 0x5603ab34b450 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4d97fa640 1 -- 172.21.15.105:0/50328546 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4e00035f0 con 0x5603ab34b450 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4d97fa640 1 -- 172.21.15.105:0/50328546 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe4e8022440 con 0x5603ab34b450 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4fac43740 10 client.21633 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:30.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.728+0000 7fe4fac43740 10 client.21633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:30.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.729+0000 7fe4fac43740 20 client.21633 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:30.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.729+0000 7fe4fac43740 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5603ab35fd70 0x5603ab3621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.729+0000 7fe4fac43740 4 client.21633 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:30.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.729+0000 7fe4fac43740 1 -- 172.21.15.105:0/50328546 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603ab2b7ec0 con 0x5603ab35fd70 2024-03-26T17:55:30.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.729+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5603ab35fd70 0x5603ab3621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.729+0000 7fe4d97fa640 10 client.21633 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:30.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.731+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5603ab35fd70 0x5603ab3621d0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97d38c272415f4d server_cookie=5b2f1efdf87c538d in_seq=0 out_seq=0 2024-03-26T17:55:30.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4d97fa640 1 -- 172.21.15.105:0/50328546 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5603ab2b7ec0 con 0x5603ab35fd70 2024-03-26T17:55:30.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4d97fa640 10 client.21633 handle_command_reply: tid=0 2024-03-26T17:55:30.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 client.21633 shutdown 2024-03-26T17:55:30.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 -- 172.21.15.105:0/50328546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 msgr2=0x5603ab35b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35b480 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800ff20 tx=0x7fe4e8004a40 comp rx=0 tx=0).stop 2024-03-26T17:55:30.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 -- 172.21.15.105:0/50328546 shutdown_connections 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5603ab35fd70 0x5603ab3621d0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4d97fa640 0 client.21633 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35b480 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3462e0 0x5603ab3544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 --2- 172.21.15.105:0/50328546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3458f0 0x5603ab353f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 -- 172.21.15.105:0/50328546 >> 172.21.15.105:0/50328546 conn(0x5603ab340e60 msgr2=0x5603ab340080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 -- 172.21.15.105:0/50328546 shutdown_connections 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.735+0000 7fe4fac43740 1 -- 172.21.15.105:0/50328546 wait complete. 2024-03-26T17:55:30.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.736+0000 7fe4fac43740 20 client.21633 trim_cache size 0 max 16384 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4fac43740 1 Processor -- start 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4fac43740 1 -- start start 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3458f0 0x5603ab34ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 0x5603ab35b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab33f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4fac43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01a6ba0 con 0x5603ab3458f0 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4fac43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01a86e0 con 0x5603ab34b450 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4fac43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01a7270 con 0x5603ab3462e0 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3458f0 0x5603ab34ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3458f0 0x5603ab34ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37074/0 (socket says 172.21.15.105:37074) 2024-03-26T17:55:30.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4f5df5640 1 -- 172.21.15.105:0/3516425103 learned_addr learned my addr 172.21.15.105:0/3516425103 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab33f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.766+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 0x5603ab35b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4f5df5640 1 -- 172.21.15.105:0/3516425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 msgr2=0x5603ab35b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 0x5603ab35b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4f5df5640 1 -- 172.21.15.105:0/3516425103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 msgr2=0x5603ab33f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab33f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4f5df5640 1 -- 172.21.15.105:0/3516425103 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ab35e730 con 0x5603ab3458f0 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab33f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3458f0 0x5603ab34ab90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0b9d80 tx=0x7fe4ec0f3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227a48e28aa59fa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4db7fe640 1 -- 172.21.15.105:0/3516425103 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0bd460 con 0x5603ab3458f0 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4db7fe640 1 -- 172.21.15.105:0/3516425103 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4ec0d6070 con 0x5603ab3458f0 2024-03-26T17:55:30.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.767+0000 7fe4db7fe640 1 -- 172.21.15.105:0/3516425103 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0bd280 con 0x5603ab3458f0 2024-03-26T17:55:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 -- 172.21.15.105:0/3516425103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3458f0 msgr2=0x5603ab34ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3458f0 0x5603ab34ab90 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0b9d80 tx=0x7fe4ec0f3040 comp rx=0 tx=0).stop 2024-03-26T17:55:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 -- 172.21.15.105:0/3516425103 shutdown_connections 2024-03-26T17:55:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab34b450 0x5603ab33f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 0x5603ab35b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3516425103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab3458f0 0x5603ab34ab90 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 -- 172.21.15.105:0/3516425103 >> 172.21.15.105:0/3516425103 conn(0x5603ab340e60 msgr2=0x5603ab341240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 -- 172.21.15.105:0/3516425103 shutdown_connections 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.768+0000 7fe4fac43740 1 -- 172.21.15.105:0/3516425103 wait complete. 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4fac43740 1 -- start start 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4fac43740 1 Processor -- start 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3458f0 0x5603ab358b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 0x5603ab3590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4fac43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4fac43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01a8010 con 0x5603ab34b450 2024-03-26T17:55:30.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4fac43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01a7940 con 0x5603ab3458f0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4fac43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f01a4b80 con 0x5603ab3462e0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.769+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3458f0 0x5603ab358b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f5df5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3458f0 0x5603ab358b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48880/0 (socket says 172.21.15.105:48880) 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f5df5640 1 -- 172.21.15.105:0/3493049946 learned_addr learned my addr 172.21.15.105:0/3493049946 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 0x5603ab3590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f65f6640 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4fac43740 10 client.0 _ll_register_callbacks cb 0x7fe4e8003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f5df5640 1 -- 172.21.15.105:0/3493049946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 msgr2=0x5603ab3590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 0x5603ab3590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f5df5640 1 -- 172.21.15.105:0/3493049946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 msgr2=0x5603ab35a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f5df5640 1 -- 172.21.15.105:0/3493049946 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ab353900 con 0x5603ab3458f0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4f5df5640 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3458f0 0x5603ab358b80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0bd620 tx=0x7fe4ec02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07c45521b67c1ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:55:30.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4d97fa640 1 -- 172.21.15.105:0/3493049946 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0bd460 con 0x5603ab3458f0 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4d97fa640 1 -- 172.21.15.105:0/3493049946 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe4ec0eed40 con 0x5603ab3458f0 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.770+0000 7fe4fac43740 1 -- 172.21.15.105:0/3493049946 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4ec0d6070 con 0x5603ab3458f0 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4d97fa640 1 -- 172.21.15.105:0/3493049946 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe4ec0bd920 con 0x5603ab3458f0 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4fac43740 10 client.31391 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4fac43740 1 -- 172.21.15.105:0/3493049946 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5603ab35b600 con 0x5603ab3458f0 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4d97fa640 1 -- 172.21.15.105:0/3493049946 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4ec0a1020 con 0x5603ab3458f0 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4d97fa640 1 -- 172.21.15.105:0/3493049946 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe4ec0dc2b0 con 0x5603ab3458f0 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4fac43740 10 client.31391 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4fac43740 10 client.31391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4fac43740 20 client.31391 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5603ab3cae10 0x5603ab3cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4fac43740 4 client.31391 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.771+0000 7fe4fac43740 1 -- 172.21.15.105:0/3493049946 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe4ec0d6070 con 0x5603ab3cae10 2024-03-26T17:55:30.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.772+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5603ab3cae10 0x5603ab3cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.772+0000 7fe4d97fa640 10 client.31391 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:30.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.772+0000 7fe4f55f4640 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5603ab3cae10 0x5603ab3cd230 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345f07ecc4874a87 server_cookie=1cab313b63ae0cc2 in_seq=0 out_seq=0 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.773+0000 7fe4d97fa640 1 -- 172.21.15.105:0/3493049946 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fe4ec0d6070 con 0x5603ab3cae10 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.773+0000 7fe4d97fa640 10 client.31391 handle_command_reply: tid=0 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.773+0000 7fe4fac43740 1 client.31391 shutdown 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.773+0000 7fe4fac43740 1 -- 172.21.15.105:0/3493049946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3458f0 msgr2=0x5603ab358b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.773+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3458f0 0x5603ab358b80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0bd620 tx=0x7fe4ec02bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 1 -- 172.21.15.105:0/3493049946 shutdown_connections 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5603ab3cae10 0x5603ab3cd230 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5603ab34b450 0x5603ab35a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5603ab3462e0 0x5603ab3590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4d97fa640 0 client.31391 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 1 --2- 172.21.15.105:0/3493049946 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603ab3458f0 0x5603ab358b80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 1 -- 172.21.15.105:0/3493049946 >> 172.21.15.105:0/3493049946 conn(0x5603ab340e60 msgr2=0x5603ab347560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 1 -- 172.21.15.105:0/3493049946 shutdown_connections 2024-03-26T17:55:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 1 -- 172.21.15.105:0/3493049946 wait complete. 2024-03-26T17:55:30.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4fac43740 20 client.31391 trim_cache size 0 max 16384 2024-03-26T17:55:30.766 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:55:30.766 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:55:30.766 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:55:30.766 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4f8059640 1 -- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4dc0a2f20 msgr2=0x7fe4dc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4f8059640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x5603ab346b40 tx=0x7fe4e8004ab0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4f8059640 1 -- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 msgr2=0x7fe4f00695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.774+0000 7fe4f8059640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 0x7fe4f00695b0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec005dc0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.775+0000 7fe4f8059640 1 -- 172.21.15.105:0/2500714839 shutdown_connections 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.775+0000 7fe4f8059640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4dc0a2f20 0x7fe4dc0a53e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.775+0000 7fe4f8059640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4f0102370 0x7fe4f019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.775+0000 7fe4f8059640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f01019b0 0x7fe4f019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.775+0000 7fe4f8059640 1 --2- 172.21.15.105:0/2500714839 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4f01007c0 0x7fe4f00695b0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.775+0000 7fe4f8059640 1 -- 172.21.15.105:0/2500714839 >> 172.21.15.105:0/2500714839 conn(0x7fe4f00fbd50 msgr2=0x7fe4f0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.775+0000 7fe4f8059640 1 -- 172.21.15.105:0/2500714839 shutdown_connections 2024-03-26T17:55:30.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.775+0000 7fe4f8059640 1 -- 172.21.15.105:0/2500714839 wait complete. 2024-03-26T17:55:30.789 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:55:30.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:55:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 -- 172.21.15.105:0/2229948521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c1076e0 msgr2=0x7f5f6c109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2229948521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c1076e0 0x7f5f6c109ad0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009ca0 tx=0x7f5f6402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 -- 172.21.15.105:0/2229948521 shutdown_connections 2024-03-26T17:55:30.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2229948521 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c1076e0 0x7f5f6c109ad0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2229948521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6c104db0 0x7f5f6c1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2229948521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f6c102480 0x7f5f6c104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 -- 172.21.15.105:0/2229948521 >> 172.21.15.105:0/2229948521 conn(0x7f5f6c0fbe10 msgr2=0x7f5f6c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 -- 172.21.15.105:0/2229948521 shutdown_connections 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.949+0000 7f5f7369a640 1 -- 172.21.15.105:0/2229948521 wait complete. 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f7369a640 1 Processor -- start 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f7369a640 1 -- start start 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f7369a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c102480 0x7f5f6c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f7369a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6c104db0 0x7f5f6c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f7369a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f6c1076e0 0x7f5f6c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f7369a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c10d490 con 0x7f5f6c102480 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f7369a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c10d310 con 0x7f5f6c104db0 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f7369a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c10d610 con 0x7f5f6c1076e0 2024-03-26T17:55:30.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f71436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c102480 0x7f5f6c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f71436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c102480 0x7f5f6c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37116/0 (socket says 172.21.15.105:37116) 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f71436640 1 -- 172.21.15.105:0/2231210064 learned_addr learned my addr 172.21.15.105:0/2231210064 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.950+0000 7f5f70c35640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6c104db0 0x7f5f6c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f71c37640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f6c1076e0 0x7f5f6c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f71436640 1 -- 172.21.15.105:0/2231210064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f6c1076e0 msgr2=0x7f5f6c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f71436640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f6c1076e0 0x7f5f6c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f71436640 1 -- 172.21.15.105:0/2231210064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6c104db0 msgr2=0x7f5f6c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f71436640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6c104db0 0x7f5f6c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f71436640 1 -- 172.21.15.105:0/2231210064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c19f7b0 con 0x7f5f6c102480 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f71436640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c102480 0x7f5f6c197b90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800ea60 tx=0x7f5f6800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f627fc640 1 -- 172.21.15.105:0/2231210064 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800cde0 con 0x7f5f6c102480 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f627fc640 1 -- 172.21.15.105:0/2231210064 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f680035f0 con 0x7f5f6c102480 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c19faa0 con 0x7f5f6c102480 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f627fc640 1 -- 172.21.15.105:0/2231210064 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f68010690 con 0x7f5f6c102480 2024-03-26T17:55:30.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.951+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c19fff0 con 0x7f5f6c102480 2024-03-26T17:55:30.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.953+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f6c068b00 con 0x7f5f6c102480 2024-03-26T17:55:30.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.954+0000 7f5f627fc640 1 -- 172.21.15.105:0/2231210064 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5f680037a0 con 0x7f5f6c102480 2024-03-26T17:55:30.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.955+0000 7f5f627fc640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5f580a3070 0x7f5f580a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:30.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.955+0000 7f5f627fc640 1 -- 172.21.15.105:0/2231210064 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f5f680cf5c0 con 0x7f5f6c102480 2024-03-26T17:55:30.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.955+0000 7f5f70c35640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5f580a3070 0x7f5f580a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:30.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.956+0000 7f5f70c35640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5f580a3070 0x7f5f580a5530 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c1990a0 tx=0x7f5f5c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:30.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:30.963+0000 7f5f627fc640 1 -- 172.21.15.105:0/2231210064 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5f6809c3b0 con 0x7f5f6c102480 2024-03-26T17:55:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.318+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f6c0fd900 con 0x7f5f6c102480 2024-03-26T17:55:31.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.319+0000 7f5f627fc640 1 -- 172.21.15.105:0/2231210064 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11941 (secure 0 0 0) 0x7f5f6809bcb0 con 0x7f5f6c102480 2024-03-26T17:55:31.306 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:55:31.306 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:54:01.817010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:55:31.306 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 37 2024-03-26T17:55:31.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5f580a3070 msgr2=0x7f5f580a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5f580a3070 0x7f5f580a5530 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c1990a0 tx=0x7f5f5c00b390 comp rx=0 tx=0).stop 2024-03-26T17:55:31.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c102480 msgr2=0x7f5f6c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c102480 0x7f5f6c197b90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800ea60 tx=0x7f5f6800ef20 comp rx=0 tx=0).stop 2024-03-26T17:55:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 shutdown_connections 2024-03-26T17:55:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5f580a3070 0x7f5f580a5530 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f6c1076e0 0x7f5f6c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5f6c104db0 0x7f5f6c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 --2- 172.21.15.105:0/2231210064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6c102480 0x7f5f6c197b90 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 >> 172.21.15.105:0/2231210064 conn(0x7f5f6c0fbe10 msgr2=0x7f5f6c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 shutdown_connections 2024-03-26T17:55:31.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.321+0000 7f5f7369a640 1 -- 172.21.15.105:0/2231210064 wait complete. 2024-03-26T17:55:31.321 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:55:31.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 -- 172.21.15.105:0/1123468859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c102340 msgr2=0x7f2c5c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 --2- 172.21.15.105:0/1123468859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c102340 0x7f2c5c104730 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009900 tx=0x7f2c4c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:31.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 -- 172.21.15.105:0/1123468859 shutdown_connections 2024-03-26T17:55:31.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 --2- 172.21.15.105:0/1123468859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c5c1075a0 0x7f2c5c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 --2- 172.21.15.105:0/1123468859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c104c70 0x7f2c5c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 --2- 172.21.15.105:0/1123468859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c102340 0x7f2c5c104730 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 -- 172.21.15.105:0/1123468859 >> 172.21.15.105:0/1123468859 conn(0x7f2c5c0fbd50 msgr2=0x7f2c5c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 -- 172.21.15.105:0/1123468859 shutdown_connections 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.480+0000 7f2c6217e640 1 -- 172.21.15.105:0/1123468859 wait complete. 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.481+0000 7f2c6217e640 1 Processor -- start 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.481+0000 7f2c6217e640 1 -- start start 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.481+0000 7f2c6217e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c102340 0x7f2c5c101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.481+0000 7f2c6217e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c5c104c70 0x7f2c5c1002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.481+0000 7f2c6217e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c1075a0 0x7f2c5c100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.481+0000 7f2c6217e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c10d440 con 0x7f2c5c104c70 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.481+0000 7f2c6217e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c10d2c0 con 0x7f2c5c102340 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.481+0000 7f2c6217e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c10d5c0 con 0x7f2c5c1075a0 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c5c104c70 0x7f2c5c1002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c102340 0x7f2c5c101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c1075a0 0x7f2c5c100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c5c104c70 0x7f2c5c1002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37132/0 (socket says 172.21.15.105:37132) 2024-03-26T17:55:31.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5affd640 1 -- 172.21.15.105:0/563708867 learned_addr learned my addr 172.21.15.105:0/563708867 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5bfff640 1 -- 172.21.15.105:0/563708867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c102340 msgr2=0x7f2c5c101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5bfff640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c102340 0x7f2c5c101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5bfff640 1 -- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c5c104c70 msgr2=0x7f2c5c1002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5bfff640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c5c104c70 0x7f2c5c1002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5bfff640 1 -- 172.21.15.105:0/563708867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c1010c0 con 0x7f2c5c1075a0 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5affd640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c5c104c70 0x7f2c5c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c5bfff640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c1075a0 0x7f2c5c100830 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400ec50 tx=0x7f2c5400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c58ff9640 1 -- 172.21.15.105:0/563708867 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54003b70 con 0x7f2c5c1075a0 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c58ff9640 1 -- 172.21.15.105:0/563708867 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c54003d10 con 0x7f2c5c1075a0 2024-03-26T17:55:31.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c6217e640 1 -- 172.21.15.105:0/563708867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5c1a1ce0 con 0x7f2c5c1075a0 2024-03-26T17:55:31.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c58ff9640 1 -- 172.21.15.105:0/563708867 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54010760 con 0x7f2c5c1075a0 2024-03-26T17:55:31.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.482+0000 7f2c6217e640 1 -- 172.21.15.105:0/563708867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5c1a2180 con 0x7f2c5c1075a0 2024-03-26T17:55:31.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.485+0000 7f2c58ff9640 1 -- 172.21.15.105:0/563708867 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2c54004020 con 0x7f2c5c1075a0 2024-03-26T17:55:31.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c64d6b740 1 Processor -- start 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c64d6b740 1 -- start start 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72dfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c64d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5641f723c970 con 0x5641f72d7a30 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c64d6b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5641f71e4d70 con 0x5641f72d8420 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c64d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5641f7238920 con 0x5641f72dd590 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72d7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37138/0 (socket says 172.21.15.105:37138) 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5affd640 1 -- 172.21.15.105:0/1703156455 learned_addr learned my addr 172.21.15.105:0/1703156455 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5b7fe640 1 -- 172.21.15.105:0/1703156455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 msgr2=0x5641f72dfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c58ff9640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c440a2f20 0x7f2c440a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72dfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5b7fe640 1 -- 172.21.15.105:0/1703156455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 msgr2=0x5641f72d7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72d7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5b7fe640 1 -- 172.21.15.105:0/1703156455 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f6f503f0 con 0x5641f72d8420 2024-03-26T17:55:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.486+0000 7f2c5affd640 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.487+0000 7f2c58ff9640 1 -- 172.21.15.105:0/563708867 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2c540cfa20 con 0x7f2c5c1075a0 2024-03-26T17:55:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.487+0000 7f2c5bfff640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c440a2f20 0x7f2c440a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.487+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72dcf20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c02c170 tx=0x7f2c4c005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22b2ce69877b213 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.487+0000 7f2c41ffb640 1 -- 172.21.15.105:0/1703156455 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c045020 con 0x5641f72d8420 2024-03-26T17:55:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.487+0000 7f2c41ffb640 1 -- 172.21.15.105:0/1703156455 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c02c9a0 con 0x5641f72d8420 2024-03-26T17:55:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.487+0000 7f2c41ffb640 1 -- 172.21.15.105:0/1703156455 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c005630 con 0x5641f72d8420 2024-03-26T17:55:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 -- 172.21.15.105:0/1703156455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 msgr2=0x5641f72dcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c5bfff640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c440a2f20 0x7f2c440a53e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2c540cd4e0 tx=0x7f2c5400ce70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72dcf20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c02c170 tx=0x7f2c4c005df0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 -- 172.21.15.105:0/1703156455 shutdown_connections 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72dfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72dcf20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/1703156455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 -- 172.21.15.105:0/1703156455 >> 172.21.15.105:0/1703156455 conn(0x5641f72d2fa0 msgr2=0x5641f72d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 -- 172.21.15.105:0/1703156455 shutdown_connections 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 -- 172.21.15.105:0/1703156455 wait complete. 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.488+0000 7f2c64d6b740 1 -- start start 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.489+0000 7f2c64d6b740 1 Processor -- start 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.489+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d7a30 0x5641f72e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.489+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d8420 0x5641f72e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.489+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72e9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.489+0000 7f2c64d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5641f72e04e0 con 0x5641f72d8420 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.489+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d7a30 0x5641f72e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d7a30 0x5641f72e1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48932/0 (socket says 172.21.15.105:48932) 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 -- 172.21.15.105:0/3569138544 learned_addr learned my addr 172.21.15.105:0/3569138544 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5641f71f8310 con 0x5641f72d7a30 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5641f72e0660 con 0x5641f72dd590 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d8420 0x5641f72e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c64d6b740 10 client.0 _ll_register_callbacks cb 0x5641f7233c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 -- 172.21.15.105:0/3569138544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 msgr2=0x5641f72e9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72e9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 -- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d8420 msgr2=0x5641f72e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d8420 0x5641f72e2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 -- 172.21.15.105:0/3569138544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f701a7d0 con 0x5641f72d7a30 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d8420 0x5641f72e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d7a30 0x5641f72e1ef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2c50012eb0 tx=0x7f2c50009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b780d6cbf93badf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:55:31.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c3affd640 1 -- 172.21.15.105:0/3569138544 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500120d0 con 0x5641f72d7a30 2024-03-26T17:55:31.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.490+0000 7f2c3affd640 1 -- 172.21.15.105:0/3569138544 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2c50004030 con 0x5641f72d7a30 2024-03-26T17:55:31.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.491+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c4c02c9a0 con 0x5641f72d7a30 2024-03-26T17:55:31.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.491+0000 7f2c3affd640 1 -- 172.21.15.105:0/3569138544 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2c50027050 con 0x5641f72d7a30 2024-03-26T17:55:31.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.491+0000 7f2c64d6b740 10 client.31409 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:31.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.491+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5641f72ec810 con 0x5641f72d7a30 2024-03-26T17:55:31.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.491+0000 7f2c3affd640 1 -- 172.21.15.105:0/3569138544 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c02c9a0 con 0x5641f72d7a30 2024-03-26T17:55:31.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.491+0000 7f2c3affd640 1 -- 172.21.15.105:0/3569138544 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2c5001f020 con 0x5641f72d7a30 2024-03-26T17:55:31.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.491+0000 7f2c64d6b740 10 client.31409 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:31.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.491+0000 7f2c64d6b740 10 client.31409 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:55:31.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.492+0000 7f2c64d6b740 20 client.31409 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:31.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.492+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5641f72eded0 0x5641f72f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.492+0000 7f2c64d6b740 4 client.31409 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.492+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f7249eb0 con 0x5641f72eded0 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.492+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5641f72eded0 0x5641f72f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.492+0000 7f2c3affd640 10 client.31409 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.492+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5641f72eded0 0x5641f72f0350 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b082cc6ff482a7f server_cookie=7ac24be86903b0df in_seq=0 out_seq=0 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c3affd640 1 -- 172.21.15.105:0/3569138544 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5641f7249eb0 con 0x5641f72eded0 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c3affd640 10 client.31409 handle_command_reply: tid=0 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 client.31409 shutdown 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d7a30 msgr2=0x5641f72e1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d7a30 0x5641f72e1ef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2c50012eb0 tx=0x7f2c50009f50 comp rx=0 tx=0).stop 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 shutdown_connections 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5641f72eded0 0x5641f72f0350 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c3affd640 0 client.31409 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d8420 0x5641f72e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3569138544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d7a30 0x5641f72e1ef0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.493+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 >> 172.21.15.105:0/3569138544 conn(0x5641f72d2fa0 msgr2=0x5641f72d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.494+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 shutdown_connections 2024-03-26T17:55:31.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.494+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3569138544 wait complete. 2024-03-26T17:55:31.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.494+0000 7f2c64d6b740 20 client.31409 trim_cache size 0 max 16384 2024-03-26T17:55:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c64d6b740 1 Processor -- start 2024-03-26T17:55:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c64d6b740 1 -- start start 2024-03-26T17:55:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72d8420 0x5641f72e9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72dd590 0x5641f72d6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c64d6b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1aaff0 con 0x5641f72d7a30 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c64d6b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1acb30 con 0x5641f72dd590 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c64d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1ab6c0 con 0x5641f72d8420 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37164/0 (socket says 172.21.15.105:37164) 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c5affd640 1 -- 172.21.15.105:0/4121788343 learned_addr learned my addr 172.21.15.105:0/4121788343 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.515+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72d8420 0x5641f72e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c5affd640 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72dd590 0x5641f72d6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c5affd640 1 -- 172.21.15.105:0/4121788343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72d8420 msgr2=0x5641f72e9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c5affd640 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72d8420 0x5641f72e9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c5affd640 1 -- 172.21.15.105:0/4121788343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72dd590 msgr2=0x5641f72d6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c5affd640 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72dd590 0x5641f72d6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c5affd640 1 -- 172.21.15.105:0/4121788343 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f72d3d10 con 0x5641f72d7a30 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c5affd640 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e2390 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f2c50012cb0 tx=0x7f2c50001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4231dd58a6af8f94 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c417fa640 1 -- 172.21.15.105:0/4121788343 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500121e0 con 0x5641f72d7a30 2024-03-26T17:55:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c417fa640 1 -- 172.21.15.105:0/4121788343 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c5001c040 con 0x5641f72d7a30 2024-03-26T17:55:31.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.516+0000 7f2c417fa640 1 -- 172.21.15.105:0/4121788343 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c500036c0 con 0x5641f72d7a30 2024-03-26T17:55:31.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.517+0000 7f2c64d6b740 1 -- 172.21.15.105:0/4121788343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 msgr2=0x5641f72e2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.517+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e2390 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f2c50012cb0 tx=0x7f2c50001e50 comp rx=0 tx=0).stop 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.517+0000 7f2c64d6b740 1 -- 172.21.15.105:0/4121788343 shutdown_connections 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.517+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72dd590 0x5641f72d6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.517+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72d8420 0x5641f72e9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.518+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/4121788343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e2390 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.518+0000 7f2c64d6b740 1 -- 172.21.15.105:0/4121788343 >> 172.21.15.105:0/4121788343 conn(0x5641f72d2fa0 msgr2=0x5641f72d6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.518+0000 7f2c64d6b740 1 -- 172.21.15.105:0/4121788343 shutdown_connections 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.518+0000 7f2c64d6b740 1 -- 172.21.15.105:0/4121788343 wait complete. 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.518+0000 7f2c64d6b740 1 -- start start 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.519+0000 7f2c64d6b740 1 Processor -- start 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.519+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.519+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.519+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.519+0000 7f2c5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37168/0 (socket says 172.21.15.105:37168) 2024-03-26T17:55:31.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.519+0000 7f2c64d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.519+0000 7f2c5affd640 1 -- 172.21.15.105:0/3051652622 learned_addr learned my addr 172.21.15.105:0/3051652622 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.519+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1ac460 con 0x5641f72d7a30 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1abd90 con 0x5641f72d8420 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c1a8fd0 con 0x5641f72dd590 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c64d6b740 10 client.0 _ll_register_callbacks cb 0x7f2c50012d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c5b7fe640 1 -- 172.21.15.105:0/3051652622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 msgr2=0x5641f72e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c5b7fe640 1 -- 172.21.15.105:0/3051652622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 msgr2=0x5641f72e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c5b7fe640 1 -- 172.21.15.105:0/3051652622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f72e8530 con 0x5641f72d8420 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c5b7fe640 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72e6fa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5641f72d3f10 tx=0x7f2c4c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6648962135cff9b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.520+0000 7f2c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.521+0000 7f2c3affd640 1 -- 172.21.15.105:0/3051652622 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c045040 con 0x5641f72d8420 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.521+0000 7f2c3affd640 1 -- 172.21.15.105:0/3051652622 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2c4c054cb0 con 0x5641f72d8420 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.521+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c5001c040 con 0x5641f72d8420 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.521+0000 7f2c3affd640 1 -- 172.21.15.105:0/3051652622 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2c4c004040 con 0x5641f72d8420 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.521+0000 7f2c64d6b740 10 client.31415 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.521+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5641f7354250 con 0x5641f72d8420 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c3affd640 1 -- 172.21.15.105:0/3051652622 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c5001c040 con 0x5641f72d8420 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c3affd640 1 -- 172.21.15.105:0/3051652622 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2c4c004300 con 0x5641f72d8420 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c64d6b740 10 client.31415 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c64d6b740 10 client.31415 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c64d6b740 20 client.31415 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5641f7354680 0x5641f7356aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c64d6b740 4 client.31415 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641f7244200 con 0x5641f7354680 2024-03-26T17:55:31.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.522+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5641f7354680 0x5641f7356aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c5affd640 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5641f7354680 0x5641f7356aa0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5211739827338cb7 server_cookie=1b731eb0d7fbcff9 in_seq=0 out_seq=0 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c3affd640 10 client.31415 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c3affd640 1 -- 172.21.15.105:0/3051652622 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641f7244200 con 0x5641f7354680 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c3affd640 10 client.31415 handle_command_reply: tid=0 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 client.31415 shutdown 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 msgr2=0x5641f72e6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72e6fa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x5641f72d3f10 tx=0x7f2c4c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 shutdown_connections 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5641f7354680 0x5641f7356aa0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5641f72dd590 0x5641f72e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c3affd640 0 client.31415 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641f72d8420 0x5641f72e6fa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 --2- 172.21.15.105:0/3051652622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5641f72d7a30 0x5641f72e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.523+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 >> 172.21.15.105:0/3051652622 conn(0x5641f72d2fa0 msgr2=0x5641f72d6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.524+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 shutdown_connections 2024-03-26T17:55:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.524+0000 7f2c64d6b740 1 -- 172.21.15.105:0/3051652622 wait complete. 2024-03-26T17:55:31.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.524+0000 7f2c64d6b740 20 client.31415 trim_cache size 0 max 16384 2024-03-26T17:55:31.515 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 -- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c440a2f20 msgr2=0x7f2c440a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c440a2f20 0x7f2c440a53e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2c540cd4e0 tx=0x7f2c5400ce70 comp rx=0 tx=0).stop 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 -- 172.21.15.105:0/563708867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c1075a0 msgr2=0x7f2c5c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c1075a0 0x7f2c5c100830 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400ec50 tx=0x7f2c5400c6a0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 -- 172.21.15.105:0/563708867 shutdown_connections 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c440a2f20 0x7f2c440a53e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c5c1075a0 0x7f2c5c100830 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c5c104c70 0x7f2c5c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 --2- 172.21.15.105:0/563708867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c5c102340 0x7f2c5c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 -- 172.21.15.105:0/563708867 >> 172.21.15.105:0/563708867 conn(0x7f2c5c0fbd50 msgr2=0x7f2c5c102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 -- 172.21.15.105:0/563708867 shutdown_connections 2024-03-26T17:55:31.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.525+0000 7f2c6217e640 1 -- 172.21.15.105:0/563708867 wait complete. 2024-03-26T17:55:31.526 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:55:31.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 33438 7.06 MB/sec execute 134 sec latency 125.461 ms 2024-03-26T17:55:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.684+0000 7f66416c6640 1 -- 172.21.15.105:0/2259667716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c1018e0 msgr2=0x7f663c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.684+0000 7f66416c6640 1 --2- 172.21.15.105:0/2259667716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c1018e0 0x7f663c101d00 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f662c009900 tx=0x7f662c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 -- 172.21.15.105:0/2259667716 shutdown_connections 2024-03-26T17:55:31.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 --2- 172.21.15.105:0/2259667716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c103490 0x7f663c109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 --2- 172.21.15.105:0/2259667716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f663c102ad0 0x7f663c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 --2- 172.21.15.105:0/2259667716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c1018e0 0x7f663c101d00 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 -- 172.21.15.105:0/2259667716 >> 172.21.15.105:0/2259667716 conn(0x7f663c0fce30 msgr2=0x7f663c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 -- 172.21.15.105:0/2259667716 shutdown_connections 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 -- 172.21.15.105:0/2259667716 wait complete. 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 Processor -- start 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.685+0000 7f66416c6640 1 -- start start 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f66416c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c1018e0 0x7f663c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f66416c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c102ad0 0x7f663c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f66416c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f663c103490 0x7f663c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f66416c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f663c10d460 con 0x7f663c102ad0 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f66416c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f663c10d2e0 con 0x7f663c103490 2024-03-26T17:55:31.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f66416c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f663c10d5e0 con 0x7f663c1018e0 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c102ad0 0x7f663c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f663affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c1018e0 0x7f663c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c102ad0 0x7f663c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37184/0 (socket says 172.21.15.105:37184) 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f663affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c1018e0 0x7f663c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53244/0 (socket says 172.21.15.105:53244) 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 -- 172.21.15.105:0/2197778429 learned_addr learned my addr 172.21.15.105:0/2197778429 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f663b7fe640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f663c103490 0x7f663c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 -- 172.21.15.105:0/2197778429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c1018e0 msgr2=0x7f663c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c1018e0 0x7f663c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 -- 172.21.15.105:0/2197778429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f663c103490 msgr2=0x7f663c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f663c103490 0x7f663c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 -- 172.21.15.105:0/2197778429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c19f720 con 0x7f663c102ad0 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f663affd640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c1018e0 0x7f663c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6633fff640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c102ad0 0x7f663c198040 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f662400e9f0 tx=0x7f662400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.686+0000 7f6638ff9640 1 -- 172.21.15.105:0/2197778429 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662400cd80 con 0x7f663c102ad0 2024-03-26T17:55:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.687+0000 7f6638ff9640 1 -- 172.21.15.105:0/2197778429 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66240108e0 con 0x7f663c102ad0 2024-03-26T17:55:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.687+0000 7f66416c6640 1 -- 172.21.15.105:0/2197778429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c19fa10 con 0x7f663c102ad0 2024-03-26T17:55:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.687+0000 7f6638ff9640 1 -- 172.21.15.105:0/2197778429 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624018920 con 0x7f663c102ad0 2024-03-26T17:55:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.687+0000 7f66416c6640 1 -- 172.21.15.105:0/2197778429 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c19ff60 con 0x7f663c102ad0 2024-03-26T17:55:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.689+0000 7f6638ff9640 1 -- 172.21.15.105:0/2197778429 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6624010a90 con 0x7f663c102ad0 2024-03-26T17:55:31.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6644393740 1 Processor -- start 2024-03-26T17:55:31.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6644393740 1 -- start start 2024-03-26T17:55:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6644393740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b48fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6644393740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b494f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6644393740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b497a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6644393740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a30b3f4970 con 0x55a30b495590 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6644393740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a30b39cd70 con 0x55a30b490420 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6644393740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a30b3f0920 con 0x55a30b48fa30 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.690+0000 7f6638ff9640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66280a2f20 0x7f66280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b494f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f6633fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b497a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b494f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48972/0 (socket says 172.21.15.105:48972) 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 -- 172.21.15.105:0/3203004281 learned_addr learned my addr 172.21.15.105:0/3203004281 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663b7fe640 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b48fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663b7fe640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66280a2f20 0x7f66280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 -- 172.21.15.105:0/3203004281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 msgr2=0x55a30b48fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b48fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 -- 172.21.15.105:0/3203004281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 msgr2=0x55a30b497a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f6638ff9640 1 -- 172.21.15.105:0/2197778429 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f66240cf760 con 0x7f663c102ad0 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b497a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 -- 172.21.15.105:0/3203004281 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a30b1083f0 con 0x55a30b490420 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663b7fe640 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b48fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f6633fff640 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b497a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663b7fe640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66280a2f20 0x7f66280a53e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x55a30b490c80 tx=0x7f6634009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.691+0000 7f663affd640 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b494f20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f662c009900 tx=0x7f662c0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3cc40d23e39de17 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.692+0000 7f66317fa640 1 -- 172.21.15.105:0/3203004281 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c045020 con 0x55a30b490420 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.692+0000 7f66317fa640 1 -- 172.21.15.105:0/3203004281 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f662c02cc90 con 0x55a30b490420 2024-03-26T17:55:31.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.692+0000 7f66317fa640 1 -- 172.21.15.105:0/3203004281 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c004920 con 0x55a30b490420 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.692+0000 7f6644393740 1 -- 172.21.15.105:0/3203004281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 msgr2=0x55a30b494f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.692+0000 7f6644393740 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b494f20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f662c009900 tx=0x7f662c0095f0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 -- 172.21.15.105:0/3203004281 shutdown_connections 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b497a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b494f20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 --2- 172.21.15.105:0/3203004281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b48fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 -- 172.21.15.105:0/3203004281 >> 172.21.15.105:0/3203004281 conn(0x55a30b48afa0 msgr2=0x55a30b48b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 -- 172.21.15.105:0/3203004281 shutdown_connections 2024-03-26T17:55:31.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 -- 172.21.15.105:0/3203004281 wait complete. 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 -- start start 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 Processor -- start 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48fa30 0x55a30b499f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 0x55a30b49a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b4a1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a30b4984e0 con 0x55a30b495590 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a30b3b0310 con 0x55a30b48fa30 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a30b498660 con 0x55a30b490420 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.693+0000 7f6644393740 10 client.0 _ll_register_callbacks cb 0x55a30b3ebc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f663affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48fa30 0x55a30b499f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f6633fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 0x55a30b49a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f663affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48fa30 0x55a30b499f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:48982/0 (socket says 172.21.15.105:48982) 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f663affd640 1 -- 172.21.15.105:0/388742907 learned_addr learned my addr 172.21.15.105:0/388742907 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f663b7fe640 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b4a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f6633fff640 1 -- 172.21.15.105:0/388742907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48fa30 msgr2=0x55a30b499f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f6633fff640 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48fa30 0x55a30b499f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f6633fff640 1 -- 172.21.15.105:0/388742907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 msgr2=0x55a30b4a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f6633fff640 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b4a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f6633fff640 1 -- 172.21.15.105:0/388742907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30b1d27d0 con 0x55a30b490420 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f663affd640 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48fa30 0x55a30b499f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f6633fff640 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 0x55a30b49a450 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6624097b80 tx=0x7f66240a4160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b154e849b56b84d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f66227fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f66227fc640 1 -- 172.21.15.105:0/388742907 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624010de0 con 0x55a30b490420 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f6644393740 1 -- 172.21.15.105:0/388742907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662c02cc90 con 0x55a30b490420 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.694+0000 7f66227fc640 1 -- 172.21.15.105:0/388742907 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6624098510 con 0x55a30b490420 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f66227fc640 1 -- 172.21.15.105:0/388742907 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f662409bea0 con 0x55a30b490420 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f6644393740 10 client.31399 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:31.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f6644393740 1 -- 172.21.15.105:0/388742907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a30b4a4870 con 0x55a30b490420 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f66227fc640 1 -- 172.21.15.105:0/388742907 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f662c02cc90 con 0x55a30b490420 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f66227fc640 1 -- 172.21.15.105:0/388742907 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f66240a2060 con 0x55a30b490420 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f6644393740 10 client.31399 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f6644393740 10 client.31399 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f6644393740 20 client.31399 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f6644393740 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55a30b4a5f90 0x55a30b4a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f6644393740 4 client.31399 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f6644393740 1 -- 172.21.15.105:0/388742907 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a30b401eb0 con 0x55a30b4a5f90 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.695+0000 7f663affd640 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55a30b4a5f90 0x55a30b4a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.697+0000 7f663affd640 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55a30b4a5f90 0x55a30b4a8400 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec37311a061843f3 server_cookie=658bf354f7fa85fa in_seq=0 out_seq=0 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.697+0000 7f66227fc640 10 client.31399 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.698+0000 7f66227fc640 1 -- 172.21.15.105:0/388742907 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a30b401eb0 con 0x55a30b4a5f90 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.698+0000 7f66227fc640 10 client.31399 handle_command_reply: tid=0 2024-03-26T17:55:31.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.698+0000 7f6644393740 1 client.31399 shutdown 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.698+0000 7f6644393740 1 -- 172.21.15.105:0/388742907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 msgr2=0x55a30b49a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.698+0000 7f6644393740 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 0x55a30b49a450 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6624097b80 tx=0x7f66240a4160 comp rx=0 tx=0).stop 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 1 -- 172.21.15.105:0/388742907 shutdown_connections 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55a30b4a5f90 0x55a30b4a8400 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b495590 0x55a30b4a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 0x55a30b49a450 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 1 --2- 172.21.15.105:0/388742907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48fa30 0x55a30b499f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 1 -- 172.21.15.105:0/388742907 >> 172.21.15.105:0/388742907 conn(0x55a30b48afa0 msgr2=0x55a30b48a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f66227fc640 0 client.31399 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 1 -- 172.21.15.105:0/388742907 shutdown_connections 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 1 -- 172.21.15.105:0/388742907 wait complete. 2024-03-26T17:55:31.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.699+0000 7f6644393740 20 client.31399 trim_cache size 0 max 16384 2024-03-26T17:55:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6644393740 1 Processor -- start 2024-03-26T17:55:31.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6644393740 1 -- start start 2024-03-26T17:55:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6644393740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48afa0 0x55a30b499f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6644393740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48fa30 0x55a30b49a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6633fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48fa30 0x55a30b49a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6633fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48fa30 0x55a30b49a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37204/0 (socket says 172.21.15.105:37204) 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f663affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48afa0 0x55a30b499f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6633fff640 1 -- 172.21.15.105:0/2112175150 learned_addr learned my addr 172.21.15.105:0/2112175150 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6644393740 1 --2- 172.21.15.105:0/2112175150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 0x55a30b4a1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6644393740 1 -- 172.21.15.105:0/2112175150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1a8dc0 con 0x55a30b48fa30 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6644393740 1 -- 172.21.15.105:0/2112175150 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1aa900 con 0x55a30b48afa0 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.721+0000 7f6644393740 1 -- 172.21.15.105:0/2112175150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1a9490 con 0x55a30b490420 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6633fff640 1 -- 172.21.15.105:0/2112175150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 msgr2=0x55a30b4a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6633fff640 1 --2- 172.21.15.105:0/2112175150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 0x55a30b4a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6633fff640 1 -- 172.21.15.105:0/2112175150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48afa0 msgr2=0x55a30b499f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6633fff640 1 --2- 172.21.15.105:0/2112175150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48afa0 0x55a30b499f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6633fff640 1 -- 172.21.15.105:0/2112175150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a30b4952e0 con 0x55a30b48fa30 2024-03-26T17:55:31.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6633fff640 1 --2- 172.21.15.105:0/2112175150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48fa30 0x55a30b49a450 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6624098300 tx=0x7f6624098270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacc8d6a8b822344 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6630ff9640 1 -- 172.21.15.105:0/2112175150 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66240cd100 con 0x55a30b48fa30 2024-03-26T17:55:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6630ff9640 1 -- 172.21.15.105:0/2112175150 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66240a4800 con 0x55a30b48fa30 2024-03-26T17:55:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.722+0000 7f6630ff9640 1 -- 172.21.15.105:0/2112175150 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662409b2f0 con 0x55a30b48fa30 2024-03-26T17:55:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.723+0000 7f6644393740 1 -- 172.21.15.105:0/2112175150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48fa30 msgr2=0x55a30b49a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.723+0000 7f6644393740 1 --2- 172.21.15.105:0/2112175150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48fa30 0x55a30b49a450 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6624098300 tx=0x7f6624098270 comp rx=0 tx=0).stop 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.723+0000 7f6644393740 1 -- 172.21.15.105:0/2112175150 shutdown_connections 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.723+0000 7f6644393740 1 --2- 172.21.15.105:0/2112175150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b490420 0x55a30b4a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.723+0000 7f6644393740 1 --2- 172.21.15.105:0/2112175150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48fa30 0x55a30b49a450 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.723+0000 7f6644393740 1 --2- 172.21.15.105:0/2112175150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b48afa0 0x55a30b499f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.723+0000 7f6644393740 1 -- 172.21.15.105:0/2112175150 >> 172.21.15.105:0/2112175150 conn(0x55a30b491200 msgr2=0x55a30b48b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.723+0000 7f6644393740 1 -- 172.21.15.105:0/2112175150 shutdown_connections 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 -- 172.21.15.105:0/2112175150 wait complete. 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 -- start start 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 Processor -- start 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48afa0 0x55a30b4a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b49df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b49e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1aa230 con 0x55a30b48afa0 2024-03-26T17:55:31.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1a9b60 con 0x55a30b490420 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f6644393740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f663c1a6da0 con 0x55a30b48fa30 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.724+0000 7f663affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48afa0 0x55a30b4a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48afa0 0x55a30b4a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37220/0 (socket says 172.21.15.105:37220) 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663affd640 1 -- 172.21.15.105:0/364252160 learned_addr learned my addr 172.21.15.105:0/364252160 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f6633fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b49df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663b7fe640 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b49e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f6644393740 10 client.0 _ll_register_callbacks cb 0x7f6624009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663affd640 1 -- 172.21.15.105:0/364252160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 msgr2=0x55a30b49df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663affd640 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b49df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663affd640 1 -- 172.21.15.105:0/364252160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 msgr2=0x55a30b49e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663affd640 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b49e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663affd640 1 -- 172.21.15.105:0/364252160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30b50a340 con 0x55a30b48afa0 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f663affd640 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48afa0 0x55a30b4a14f0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f662c000c00 tx=0x7f662c02cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bbb29402a60cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f66227fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:31.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f66227fc640 1 -- 172.21.15.105:0/364252160 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c045040 con 0x55a30b48afa0 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f66227fc640 1 -- 172.21.15.105:0/364252160 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f662c005240 con 0x55a30b48afa0 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.725+0000 7f6644393740 1 -- 172.21.15.105:0/364252160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66240a4800 con 0x55a30b48afa0 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f66227fc640 1 -- 172.21.15.105:0/364252160 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f662c005440 con 0x55a30b48afa0 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6644393740 10 client.21681 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6644393740 1 -- 172.21.15.105:0/364252160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a30b50b690 con 0x55a30b48afa0 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f66227fc640 1 -- 172.21.15.105:0/364252160 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66240a4800 con 0x55a30b48afa0 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f66227fc640 1 -- 172.21.15.105:0/364252160 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f662c003800 con 0x55a30b48afa0 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6644393740 10 client.21681 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6644393740 10 client.21681 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6644393740 20 client.21681 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6644393740 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55a30b495590 0x55a30b50dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6644393740 4 client.21681 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6644393740 1 -- 172.21.15.105:0/364252160 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a30b3fc200 con 0x55a30b495590 2024-03-26T17:55:31.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.726+0000 7f6633fff640 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55a30b495590 0x55a30b50dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f66227fc640 10 client.21681 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6633fff640 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55a30b495590 0x55a30b50dc10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c82d8705294d2fa server_cookie=23de695d2e35b3f9 in_seq=0 out_seq=0 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f66227fc640 1 -- 172.21.15.105:0/364252160 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a30b3fc200 con 0x55a30b495590 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f66227fc640 10 client.21681 handle_command_reply: tid=0 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 client.21681 shutdown 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 -- 172.21.15.105:0/364252160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48afa0 msgr2=0x55a30b4a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48afa0 0x55a30b4a14f0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f662c000c00 tx=0x7f662c02cd20 comp rx=0 tx=0).stop 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 -- 172.21.15.105:0/364252160 shutdown_connections 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55a30b495590 0x55a30b50dc10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a30b490420 0x55a30b49e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f66227fc640 0 client.21681 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a30b48fa30 0x55a30b49df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 --2- 172.21.15.105:0/364252160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a30b48afa0 0x55a30b4a14f0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 -- 172.21.15.105:0/364252160 >> 172.21.15.105:0/364252160 conn(0x55a30b491200 msgr2=0x55a30b491920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 -- 172.21.15.105:0/364252160 shutdown_connections 2024-03-26T17:55:31.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 1 -- 172.21.15.105:0/364252160 wait complete. 2024-03-26T17:55:31.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.727+0000 7f6644393740 20 client.21681 trim_cache size 0 max 16384 2024-03-26T17:55:31.719 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 -- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66280a2f20 msgr2=0x7f66280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66280a2f20 0x7f66280a53e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x55a30b490c80 tx=0x7f6634009500 comp rx=0 tx=0).stop 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 -- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c102ad0 msgr2=0x7f663c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c102ad0 0x7f663c198040 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f662400e9f0 tx=0x7f662400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 -- 172.21.15.105:0/2197778429 shutdown_connections 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66280a2f20 0x7f66280a53e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f663c103490 0x7f663c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f663c102ad0 0x7f663c198040 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 --2- 172.21.15.105:0/2197778429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c1018e0 0x7f663c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 -- 172.21.15.105:0/2197778429 >> 172.21.15.105:0/2197778429 conn(0x7f663c0fce30 msgr2=0x7f663c105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 -- 172.21.15.105:0/2197778429 shutdown_connections 2024-03-26T17:55:31.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.728+0000 7f66416c6640 1 -- 172.21.15.105:0/2197778429 wait complete. 2024-03-26T17:55:31.728 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 -- 172.21.15.105:0/4154737100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01023b0 msgr2=0x7f0bf0108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 --2- 172.21.15.105:0/4154737100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01023b0 0x7f0bf0108c40 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009ca0 tx=0x7f0be802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 -- 172.21.15.105:0/4154737100 shutdown_connections 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 --2- 172.21.15.105:0/4154737100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01023b0 0x7f0bf0108c40 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 --2- 172.21.15.105:0/4154737100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bf01019f0 0x7f0bf0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 --2- 172.21.15.105:0/4154737100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bf0100800 0x7f0bf0100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 -- 172.21.15.105:0/4154737100 >> 172.21.15.105:0/4154737100 conn(0x7f0bf00fbd70 msgr2=0x7f0bf00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 -- 172.21.15.105:0/4154737100 shutdown_connections 2024-03-26T17:55:31.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.923+0000 7f0bf8537640 1 -- 172.21.15.105:0/4154737100 wait complete. 2024-03-26T17:55:31.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf8537640 1 Processor -- start 2024-03-26T17:55:31.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf8537640 1 -- start start 2024-03-26T17:55:31.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf8537640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bf0100800 0x7f0bf006a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf8537640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01019f0 0x7f0bf019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf8537640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bf01023b0 0x7f0bf019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf8537640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf010c330 con 0x7f0bf01019f0 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf8537640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf010c1b0 con 0x7f0bf0100800 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf8537640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf010c4b0 con 0x7f0bf01023b0 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf62d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bf0100800 0x7f0bf006a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf5ad2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01019f0 0x7f0bf019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf6ad4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bf01023b0 0x7f0bf019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf62d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bf0100800 0x7f0bf006a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49004/0 (socket says 172.21.15.105:49004) 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.924+0000 7f0bf62d3640 1 -- 172.21.15.105:0/894580824 learned_addr learned my addr 172.21.15.105:0/894580824 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf62d3640 1 -- 172.21.15.105:0/894580824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bf01023b0 msgr2=0x7f0bf019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bf01023b0 0x7f0bf019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf62d3640 1 -- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01019f0 msgr2=0x7f0bf019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01019f0 0x7f0bf019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf62d3640 1 -- 172.21.15.105:0/894580824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf019f770 con 0x7f0bf0100800 2024-03-26T17:55:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01019f0 0x7f0bf019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bf0100800 0x7f0bf006a790 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009bd0 tx=0x7f0bec00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0be77fe640 1 -- 172.21.15.105:0/894580824 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec01e070 con 0x7f0bf0100800 2024-03-26T17:55:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf8537640 1 -- 172.21.15.105:0/894580824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf0196670 con 0x7f0bf0100800 2024-03-26T17:55:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0be77fe640 1 -- 172.21.15.105:0/894580824 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bec0035f0 con 0x7f0bf0100800 2024-03-26T17:55:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0be77fe640 1 -- 172.21.15.105:0/894580824 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0196b0 con 0x7f0bf0100800 2024-03-26T17:55:31.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.925+0000 7f0bf8537640 1 -- 172.21.15.105:0/894580824 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf0196bc0 con 0x7f0bf0100800 2024-03-26T17:55:31.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.929+0000 7f0be77fe640 1 -- 172.21.15.105:0/894580824 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0bec021070 con 0x7f0bf0100800 2024-03-26T17:55:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.929+0000 7f0bfb1fc740 1 Processor -- start 2024-03-26T17:55:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.929+0000 7f0be77fe640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0bdc0a2f20 0x7f0bdc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0be77fe640 1 -- 172.21.15.105:0/894580824 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0bec0d2c90 con 0x7f0bf0100800 2024-03-26T17:55:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bf6ad4640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0bdc0a2f20 0x7f0bdc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bfb1fc740 1 -- start start 2024-03-26T17:55:31.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5607c4106970 con 0x5607c41a2420 2024-03-26T17:55:31.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5607c40aed70 con 0x5607c41a7590 2024-03-26T17:55:31.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5607c4102920 con 0x5607c41a1a30 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bf5ad2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bf62d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bf62d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53292/0 (socket says 172.21.15.105:53292) 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.930+0000 7f0bf62d3640 1 -- 172.21.15.105:0/1291618838 learned_addr learned my addr 172.21.15.105:0/1291618838 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf6ad4640 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf62d3640 1 -- 172.21.15.105:0/1291618838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 msgr2=0x5607c41a9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41a9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf62d3640 1 -- 172.21.15.105:0/1291618838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 msgr2=0x5607c41a6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf6ad4640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0bdc0a2f20 0x7f0bdc0a53e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006010 tx=0x7f0be80324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41a6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf62d3640 1 -- 172.21.15.105:0/1291618838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607c3e1a3f0 con 0x5607c41a1a30 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41a1e50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0a7860 tx=0x7f0bec09b0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f85f0577eb47cd97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0be4bfe640 1 -- 172.21.15.105:0/1291618838 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0a34d0 con 0x5607c41a1a30 2024-03-26T17:55:31.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0be4bfe640 1 -- 172.21.15.105:0/1291618838 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bec0a7d90 con 0x5607c41a1a30 2024-03-26T17:55:31.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.931+0000 7f0be4bfe640 1 -- 172.21.15.105:0/1291618838 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec09b180 con 0x5607c41a1a30 2024-03-26T17:55:31.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1291618838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 msgr2=0x5607c41a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41a1e50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0a7860 tx=0x7f0bec09b0d0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1291618838 shutdown_connections 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1291618838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41a1e50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1291618838 >> 172.21.15.105:0/1291618838 conn(0x5607c419cfa0 msgr2=0x5607c419d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1291618838 shutdown_connections 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.932+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1291618838 wait complete. 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.933+0000 7f0bfb1fc740 1 -- start start 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.933+0000 7f0bfb1fc740 1 Processor -- start 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.933+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.933+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37256/0 (socket says 172.21.15.105:37256) 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 -- 172.21.15.105:0/1420968287 learned_addr learned my addr 172.21.15.105:0/1420968287 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5607c41aa540 con 0x5607c41a1a30 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5607c40c2310 con 0x5607c41a7590 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5607c41aa6c0 con 0x5607c41a2420 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bfb1fc740 10 client.0 _ll_register_callbacks cb 0x5607c40fdc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 -- 172.21.15.105:0/1420968287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 msgr2=0x5607c41ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 -- 172.21.15.105:0/1420968287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 msgr2=0x5607c41b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 -- 172.21.15.105:0/1420968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607c3ee47d0 con 0x5607c41a1a30 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.934+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41ac010 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0d2fb0 tx=0x7f0bec0a34d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8058a115f40edfe8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.935+0000 7f0bd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.935+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/1420968287 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0bd060 con 0x5607c41a1a30 2024-03-26T17:55:31.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.935+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bec0a7d90 con 0x5607c41a1a30 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.935+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/1420968287 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0bec0b8260 con 0x5607c41a1a30 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.935+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/1420968287 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0bec0b8460 con 0x5607c41a1a30 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.935+0000 7f0bfb1fc740 10 client.21693 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.935+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5607c41b6900 con 0x5607c41a1a30 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.935+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/1420968287 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bec0a7d90 con 0x5607c41a1a30 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/1420968287 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0bec0a4660 con 0x5607c41a1a30 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bfb1fc740 10 client.21693 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bfb1fc740 10 client.21693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bfb1fc740 20 client.21693 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5607c41b7c50 0x5607c41ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bfb1fc740 4 client.21693 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607c4113eb0 con 0x5607c41b7c50 2024-03-26T17:55:31.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5607c41b7c50 0x5607c41ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bd9ffb640 10 client.21693 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.936+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5607c41b7c50 0x5607c41ba040 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=150d10804538dde4 server_cookie=d394f6446317a6d2 in_seq=0 out_seq=0 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.937+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/1420968287 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5607c4113eb0 con 0x5607c41b7c50 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.937+0000 7f0bd9ffb640 10 client.21693 handle_command_reply: tid=0 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.937+0000 7f0bfb1fc740 1 client.21693 shutdown 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.937+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 msgr2=0x5607c41ac010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.937+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41ac010 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0d2fb0 tx=0x7f0bec0a34d0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 shutdown_connections 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5607c41b7c50 0x5607c41ba040 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bd9ffb640 0 client.21693 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1420968287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41ac010 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 >> 172.21.15.105:0/1420968287 conn(0x5607c419cfa0 msgr2=0x5607c419c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 shutdown_connections 2024-03-26T17:55:31.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1420968287 wait complete. 2024-03-26T17:55:31.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.938+0000 7f0bfb1fc740 20 client.21693 trim_cache size 0 max 16384 2024-03-26T17:55:31.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bfb1fc740 1 Processor -- start 2024-03-26T17:55:31.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bfb1fc740 1 -- start start 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01a6c50 con 0x5607c41a1a30 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01a8790 con 0x5607c41a7590 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01a7320 con 0x5607c41a2420 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bf5ad2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bf5ad2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53300/0 (socket says 172.21.15.105:53300) 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bf5ad2640 1 -- 172.21.15.105:0/1608647660 learned_addr learned my addr 172.21.15.105:0/1608647660 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.961+0000 7f0bf6ad4640 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bf5ad2640 1 -- 172.21.15.105:0/1608647660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 msgr2=0x5607c41ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bf5ad2640 1 -- 172.21.15.105:0/1608647660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 msgr2=0x5607c41abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bf5ad2640 1 -- 172.21.15.105:0/1608647660 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607c419dd10 con 0x5607c41a2420 2024-03-26T17:55:31.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:55:31.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41b3520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0bf006a500 tx=0x7f0be0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=803a5b14ed7c207f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bdbfff640 1 -- 172.21.15.105:0/1608647660 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be001e700 con 0x5607c41a2420 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bdbfff640 1 -- 172.21.15.105:0/1608647660 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0be00054d0 con 0x5607c41a2420 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.962+0000 7f0bdbfff640 1 -- 172.21.15.105:0/1608647660 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be001d3a0 con 0x5607c41a2420 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1608647660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 msgr2=0x5607c41b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41b3520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0bf006a500 tx=0x7f0be0005f90 comp rx=0 tx=0).stop 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1608647660 shutdown_connections 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a2420 0x5607c41b3520 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/1608647660 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a1a30 0x5607c41abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1608647660 >> 172.21.15.105:0/1608647660 conn(0x5607c419cfa0 msgr2=0x5607c41aaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1608647660 shutdown_connections 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/1608647660 wait complete. 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 -- start start 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.963+0000 7f0bfb1fc740 1 Processor -- start 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bfb1fc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01a80c0 con 0x5607c41a2420 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01a79f0 con 0x5607c41a7590 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bfb1fc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01a4c30 con 0x5607c41a1a30 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bfb1fc740 10 client.0 _ll_register_callbacks cb 0x7f0be0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37272/0 (socket says 172.21.15.105:37272) 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 -- 172.21.15.105:0/3981674674 learned_addr learned my addr 172.21.15.105:0/3981674674 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 -- 172.21.15.105:0/3981674674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 msgr2=0x5607c41b3430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 -- 172.21.15.105:0/3981674674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 msgr2=0x5607c41b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 -- 172.21.15.105:0/3981674674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607c4224630 con 0x5607c41a2420 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bf5ad2640 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41afed0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009560 tx=0x7f0be001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcd3c8de3540185 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.964+0000 7f0bd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/3981674674 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0005680 con 0x5607c41a2420 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/3981674674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be00054d0 con 0x5607c41a2420 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/3981674674 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0be001e500 con 0x5607c41a2420 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/3981674674 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0be001e720 con 0x5607c41a2420 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 10 client.21699 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:31.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/3981674674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5607c4225980 con 0x5607c41a2420 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/3981674674 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0be00054d0 con 0x5607c41a2420 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/3981674674 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0be0010670 con 0x5607c41a2420 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 10 client.21699 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 10 client.21699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 20 client.21699 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5607c41b7c50 0x5607c42282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 4 client.21699 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.965+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/3981674674 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607c410e200 con 0x5607c41b7c50 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5607c41b7c50 0x5607c42282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bd9ffb640 10 client.21699 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bf62d3640 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5607c41b7c50 0x5607c42282f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6549d3620e85bc1 server_cookie=2d27fbb3835c4e01 in_seq=0 out_seq=0 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bd9ffb640 1 -- 172.21.15.105:0/3981674674 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607c410e200 con 0x5607c41b7c50 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bd9ffb640 10 client.21699 handle_command_reply: tid=0 2024-03-26T17:55:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 client.21699 shutdown 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/3981674674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 msgr2=0x5607c41afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41afed0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f0be0009560 tx=0x7f0be001ec10 comp rx=0 tx=0).stop 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/3981674674 shutdown_connections 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5607c41b7c50 0x5607c42282f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5607c41a7590 0x5607c41b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5607c41a2420 0x5607c41afed0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bd9ffb640 0 client.21699 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 --2- 172.21.15.105:0/3981674674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5607c41a1a30 0x5607c41b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.966+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/3981674674 >> 172.21.15.105:0/3981674674 conn(0x5607c419cfa0 msgr2=0x5607c41abc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/3981674674 shutdown_connections 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bfb1fc740 1 -- 172.21.15.105:0/3981674674 wait complete. 2024-03-26T17:55:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bfb1fc740 20 client.21699 trim_cache size 0 max 16384 2024-03-26T17:55:31.959 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:55:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 -- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0bdc0a2f20 msgr2=0x7f0bdc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0bdc0a2f20 0x7f0bdc0a53e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006010 tx=0x7f0be80324f0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 -- 172.21.15.105:0/894580824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bf0100800 msgr2=0x7f0bf006a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bf0100800 0x7f0bf006a790 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009bd0 tx=0x7f0bec00b690 comp rx=0 tx=0).stop 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 -- 172.21.15.105:0/894580824 shutdown_connections 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0bdc0a2f20 0x7f0bdc0a53e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bf01023b0 0x7f0bf019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf01019f0 0x7f0bf019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 --2- 172.21.15.105:0/894580824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bf0100800 0x7f0bf006a790 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 -- 172.21.15.105:0/894580824 >> 172.21.15.105:0/894580824 conn(0x7f0bf00fbd70 msgr2=0x7f0bf0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.967+0000 7f0bf8537640 1 -- 172.21.15.105:0/894580824 shutdown_connections 2024-03-26T17:55:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:31.968+0000 7f0bf8537640 1 -- 172.21.15.105:0/894580824 wait complete. 2024-03-26T17:55:31.967 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:55:31.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:31 smithi105 ceph-mon[32280]: pgmap v864: 97 pgs: 97 active+clean; 283 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-03-26T17:55:31.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:31 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2231210064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:55:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 -- 172.21.15.105:0/1733637089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c104c70 msgr2=0x7f2f7c107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1733637089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c104c70 0x7f2f7c107060 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009900 tx=0x7f2f7002bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 -- 172.21.15.105:0/1733637089 shutdown_connections 2024-03-26T17:55:32.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1733637089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f7c1075a0 0x7f2f7c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1733637089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c104c70 0x7f2f7c107060 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1733637089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f7c102340 0x7f2f7c104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 -- 172.21.15.105:0/1733637089 >> 172.21.15.105:0/1733637089 conn(0x7f2f7c0fbd50 msgr2=0x7f2f7c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 -- 172.21.15.105:0/1733637089 shutdown_connections 2024-03-26T17:55:32.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.121+0000 7f2f829f4640 1 -- 172.21.15.105:0/1733637089 wait complete. 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.122+0000 7f2f829f4640 1 Processor -- start 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.122+0000 7f2f829f4640 1 -- start start 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.122+0000 7f2f829f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c102340 0x7f2f7c101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f829f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f7c104c70 0x7f2f7c1002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f829f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f7c1075a0 0x7f2f7c100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f829f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c10d430 con 0x7f2f7c102340 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f829f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c10d2b0 con 0x7f2f7c104c70 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f829f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c10d5b0 con 0x7f2f7c1075a0 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c102340 0x7f2f7c101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c102340 0x7f2f7c101cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37296/0 (socket says 172.21.15.105:37296) 2024-03-26T17:55:32.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7bfff640 1 -- 172.21.15.105:0/1914365614 learned_addr learned my addr 172.21.15.105:0/1914365614 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f7c104c70 0x7f2f7c1002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f80f91640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f7c1075a0 0x7f2f7c100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7bfff640 1 -- 172.21.15.105:0/1914365614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f7c1075a0 msgr2=0x7f2f7c100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f7c1075a0 0x7f2f7c100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7bfff640 1 -- 172.21.15.105:0/1914365614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f7c104c70 msgr2=0x7f2f7c1002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f7c104c70 0x7f2f7c1002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f7bfff640 1 -- 172.21.15.105:0/1914365614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7c1010c0 con 0x7f2f7c102340 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.123+0000 7f2f80f91640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f7c1075a0 0x7f2f7c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.124+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c102340 0x7f2f7c101cb0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c920 tx=0x7f2f6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.124+0000 7f2f797fa640 1 -- 172.21.15.105:0/1914365614 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c007c70 con 0x7f2f7c102340 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.124+0000 7f2f797fa640 1 -- 172.21.15.105:0/1914365614 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f6c007e10 con 0x7f2f7c102340 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.124+0000 7f2f797fa640 1 -- 172.21.15.105:0/1914365614 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c018820 con 0x7f2f7c102340 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.124+0000 7f2f829f4640 1 -- 172.21.15.105:0/1914365614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7c1a1c00 con 0x7f2f7c102340 2024-03-26T17:55:32.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.124+0000 7f2f829f4640 1 -- 172.21.15.105:0/1914365614 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7c1a20d0 con 0x7f2f7c102340 2024-03-26T17:55:32.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.127+0000 7f2f797fa640 1 -- 172.21.15.105:0/1914365614 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2f6c0189c0 con 0x7f2f7c102340 2024-03-26T17:55:32.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f855e9740 1 Processor -- start 2024-03-26T17:55:32.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f797fa640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f640a2fa0 0x7f2f640a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f855e9740 1 -- start start 2024-03-26T17:55:32.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdc240 0x55f334cdc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f797fa640 1 -- 172.21.15.105:0/1914365614 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2f6c0cf540 con 0x7f2f7c102340 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f80f91640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f640a2fa0 0x7f2f640a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334ce1da0 0x55f334ce4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f855e9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f334c38970 con 0x55f334ce1da0 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f855e9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f334be0d70 con 0x55f334cdcc30 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f855e9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f334c34920 con 0x55f334cdc240 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdc240 0x55f334cdc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdc240 0x55f334cdc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53352/0 (socket says 172.21.15.105:53352) 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f7bfff640 1 -- 172.21.15.105:0/2109732314 learned_addr learned my addr 172.21.15.105:0/2109732314 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f80f91640 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334ce1da0 0x55f334ce4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.128+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f7bfff640 1 -- 172.21.15.105:0/2109732314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 msgr2=0x55f334ce1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f7bfff640 1 -- 172.21.15.105:0/2109732314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334ce1da0 msgr2=0x55f334ce4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334ce1da0 0x55f334ce4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f7bfff640 1 -- 172.21.15.105:0/2109732314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33494c3f0 con 0x55f334cdc240 2024-03-26T17:55:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:55:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdc240 0x55f334cdc660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c09b930 tx=0x7f2f6c00fbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e868fb6ae7676d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f80f91640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f640a2fa0 0x7f2f640a5460 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005fd0 tx=0x7f2f74009bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f627fc640 1 -- 172.21.15.105:0/2109732314 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c09fd80 con 0x55f334cdc240 2024-03-26T17:55:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f627fc640 1 -- 172.21.15.105:0/2109732314 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f6c0a4640 con 0x55f334cdc240 2024-03-26T17:55:32.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.129+0000 7f2f627fc640 1 -- 172.21.15.105:0/2109732314 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0cf0f0 con 0x55f334cdc240 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 -- 172.21.15.105:0/2109732314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdc240 msgr2=0x55f334cdc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdc240 0x55f334cdc660 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c09b930 tx=0x7f2f6c00fbc0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 -- 172.21.15.105:0/2109732314 shutdown_connections 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334ce1da0 0x55f334ce4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2109732314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdc240 0x55f334cdc660 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 -- 172.21.15.105:0/2109732314 >> 172.21.15.105:0/2109732314 conn(0x55f334ccefa0 msgr2=0x55f334ccf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 -- 172.21.15.105:0/2109732314 shutdown_connections 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 -- 172.21.15.105:0/2109732314 wait complete. 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.130+0000 7f2f855e9740 1 -- start start 2024-03-26T17:55:32.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f855e9740 1 Processor -- start 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ce6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334ce1da0 0x55f334ceddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f855e9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f334ce4d50 con 0x55f334cdc240 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f855e9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f334bf4310 con 0x55f334cdcc30 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f855e9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f334ce4ed0 con 0x55f334ce1da0 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f855e9740 10 client.0 _ll_register_callbacks cb 0x55f334c2fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ce6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f7b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ce6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37312/0 (socket says 172.21.15.105:37312) 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f7bfff640 1 -- 172.21.15.105:0/2907762364 learned_addr learned my addr 172.21.15.105:0/2907762364 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.131+0000 7f2f80f91640 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334ce1da0 0x55f334ceddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f7b7fe640 1 -- 172.21.15.105:0/2907762364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334ce1da0 msgr2=0x55f334ceddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334ce1da0 0x55f334ceddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f7b7fe640 1 -- 172.21.15.105:0/2907762364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 msgr2=0x55f334ce6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ce6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f7b7fe640 1 -- 172.21.15.105:0/2907762364 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f334a167d0 con 0x55f334cdcc30 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ce6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce6d80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2f70036040 tx=0x7f2f700042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b34b3ff35687d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/2907762364 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70005d50 con 0x55f334cdcc30 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f855e9740 1 -- 172.21.15.105:0/2907762364 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f6c0a4640 con 0x55f334cdcc30 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/2907762364 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2f70005250 con 0x55f334cdcc30 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/2907762364 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2f700044e0 con 0x55f334cdcc30 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f855e9740 10 client.31454 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.132+0000 7f2f855e9740 1 -- 172.21.15.105:0/2907762364 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f334cf11f0 con 0x55f334cdcc30 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/2907762364 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f6c0a4640 con 0x55f334cdcc30 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/2907762364 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2f70052080 con 0x55f334cdcc30 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f855e9740 10 client.31454 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:32.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f855e9740 10 client.31454 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f855e9740 20 client.31454 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f334cf2760 0x55f334cf4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f855e9740 4 client.31454 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f855e9740 1 -- 172.21.15.105:0/2907762364 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f334c45eb0 con 0x55f334cf2760 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f334cf2760 0x55f334cf4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f5b7fe640 10 client.31454 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.133+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f334cf2760 0x55f334cf4bc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b28cd6ff2ebd670c server_cookie=9446951e7ab819f4 in_seq=0 out_seq=0 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/2907762364 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f334c45eb0 con 0x55f334cf2760 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f5b7fe640 10 client.31454 handle_command_reply: tid=0 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 client.31454 shutdown 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 -- 172.21.15.105:0/2907762364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 msgr2=0x55f334ce6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce6d80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2f70036040 tx=0x7f2f700042f0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 -- 172.21.15.105:0/2907762364 shutdown_connections 2024-03-26T17:55:32.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f334cf2760 0x55f334cf4bc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f5b7fe640 0 client.31454 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:55:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334ce1da0 0x55f334ceddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334cdcc30 0x55f334ce6d80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 --2- 172.21.15.105:0/2907762364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ce6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 -- 172.21.15.105:0/2907762364 >> 172.21.15.105:0/2907762364 conn(0x55f334ccefa0 msgr2=0x55f334cce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.134+0000 7f2f855e9740 1 -- 172.21.15.105:0/2907762364 shutdown_connections 2024-03-26T17:55:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.135+0000 7f2f855e9740 1 -- 172.21.15.105:0/2907762364 wait complete. 2024-03-26T17:55:32.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.135+0000 7f2f855e9740 20 client.31454 trim_cache size 0 max 16384 2024-03-26T17:55:32.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.153+0000 7f2f855e9740 1 Processor -- start 2024-03-26T17:55:32.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.153+0000 7f2f855e9740 1 -- start start 2024-03-26T17:55:32.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334cdd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334cedd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 0x55f334ccd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f855e9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1a6be0 con 0x55f334cdc240 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f855e9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1a8720 con 0x55f334ce1da0 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f855e9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1a72b0 con 0x55f334cdcc30 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334cdd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334cdd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37326/0 (socket says 172.21.15.105:37326) 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7bfff640 1 -- 172.21.15.105:0/3018504837 learned_addr learned my addr 172.21.15.105:0/3018504837 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334cedd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f80f91640 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 0x55f334ccd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7b7fe640 1 -- 172.21.15.105:0/3018504837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 msgr2=0x55f334ccd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 0x55f334ccd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7b7fe640 1 -- 172.21.15.105:0/3018504837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 msgr2=0x55f334cdd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334cdd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7b7fe640 1 -- 172.21.15.105:0/3018504837 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f334cdd010 con 0x55f334cdcc30 2024-03-26T17:55:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334cdd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.154+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334cedd80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009850 tx=0x7f2f7002cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0db4db7384637e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.155+0000 7f2f61ffb640 1 -- 172.21.15.105:0/3018504837 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70005d50 con 0x55f334cdcc30 2024-03-26T17:55:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.155+0000 7f2f61ffb640 1 -- 172.21.15.105:0/3018504837 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f70003e20 con 0x55f334cdcc30 2024-03-26T17:55:32.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.155+0000 7f2f61ffb640 1 -- 172.21.15.105:0/3018504837 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70004900 con 0x55f334cdcc30 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.155+0000 7f2f855e9740 1 -- 172.21.15.105:0/3018504837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 msgr2=0x55f334cedd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.155+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334cedd80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009850 tx=0x7f2f7002cca0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 -- 172.21.15.105:0/3018504837 shutdown_connections 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 0x55f334ccd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334cedd80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3018504837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334cdd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 -- 172.21.15.105:0/3018504837 >> 172.21.15.105:0/3018504837 conn(0x55f334ccefa0 msgr2=0x55f334ce52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 -- 172.21.15.105:0/3018504837 shutdown_connections 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 -- 172.21.15.105:0/3018504837 wait complete. 2024-03-26T17:55:32.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 -- start start 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 Processor -- start 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ceb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.156+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334ceb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f855e9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 0x55f334cec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ceb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f855e9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1a8050 con 0x55f334cdc240 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f855e9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1a7980 con 0x55f334ce1da0 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f855e9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c1a4bc0 con 0x55f334cdcc30 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ceb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37338/0 (socket says 172.21.15.105:37338) 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 -- 172.21.15.105:0/3906199955 learned_addr learned my addr 172.21.15.105:0/3906199955 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334ceb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f80f91640 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 0x55f334cec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f855e9740 10 client.0 _ll_register_callbacks cb 0x7f2f70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 -- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 msgr2=0x55f334ceb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334ceb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 -- 172.21.15.105:0/3906199955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 msgr2=0x55f334cec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 0x55f334cec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 -- 172.21.15.105:0/3906199955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f334cece00 con 0x55f334cdc240 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334ceb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f7bfff640 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ceb360 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0cd370 tx=0x7f2f6c09baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e978ac863d2e325a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.157+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/3906199955 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c07e070 con 0x55f334cdc240 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/3906199955 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2f6c099f30 con 0x55f334cdc240 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 1 -- 172.21.15.105:0/3906199955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f70003e20 con 0x55f334cdc240 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/3906199955 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2f6c0a9050 con 0x55f334cdc240 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 10 client.21714 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 1 -- 172.21.15.105:0/3906199955 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f334cee130 con 0x55f334cdc240 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/3906199955 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f70003e20 con 0x55f334cdc240 2024-03-26T17:55:32.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/3906199955 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2f6c0cf210 con 0x55f334cdc240 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 10 client.21714 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 10 client.21714 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 20 client.21714 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f334d58e70 0x55f334d5b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 4 client.21714 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f855e9740 1 -- 172.21.15.105:0/3906199955 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f334cddd10 con 0x55f334d58e70 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.158+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f334d58e70 0x55f334d5b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f5b7fe640 10 client.21714 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f7b7fe640 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f334d58e70 0x55f334d5b260 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7881913522f55a6 server_cookie=e75ef6c879bfaeec in_seq=0 out_seq=0 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f5b7fe640 1 -- 172.21.15.105:0/3906199955 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f334cddd10 con 0x55f334d58e70 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f5b7fe640 10 client.21714 handle_command_reply: tid=0 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 client.21714 shutdown 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 -- 172.21.15.105:0/3906199955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 msgr2=0x55f334ceb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ceb360 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0cd370 tx=0x7f2f6c09baf0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 -- 172.21.15.105:0/3906199955 shutdown_connections 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55f334d58e70 0x55f334d5b260 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f5b7fe640 0 client.21714 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f334ce1da0 0x55f334cec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f334cdcc30 0x55f334ceb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 --2- 172.21.15.105:0/3906199955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f334cdc240 0x55f334ceb360 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.159+0000 7f2f855e9740 1 -- 172.21.15.105:0/3906199955 >> 172.21.15.105:0/3906199955 conn(0x55f334ccefa0 msgr2=0x55f334cd2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.160+0000 7f2f855e9740 1 -- 172.21.15.105:0/3906199955 shutdown_connections 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.160+0000 7f2f855e9740 1 -- 172.21.15.105:0/3906199955 wait complete. 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.160+0000 7f2f855e9740 20 client.21714 trim_cache size 0 max 16384 2024-03-26T17:55:32.152 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.160+0000 7f2f829f4640 1 -- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f640a2fa0 msgr2=0x7f2f640a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f640a2fa0 0x7f2f640a5460 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2f74005fd0 tx=0x7f2f74009bb0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 -- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c102340 msgr2=0x7f2f7c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c102340 0x7f2f7c101cb0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c920 tx=0x7f2f6c00cde0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 -- 172.21.15.105:0/1914365614 shutdown_connections 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2f640a2fa0 0x7f2f640a5460 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f7c1075a0 0x7f2f7c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2f7c104c70 0x7f2f7c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 --2- 172.21.15.105:0/1914365614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f7c102340 0x7f2f7c101cb0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 -- 172.21.15.105:0/1914365614 >> 172.21.15.105:0/1914365614 conn(0x7f2f7c0fbd50 msgr2=0x7f2f7c102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 -- 172.21.15.105:0/1914365614 shutdown_connections 2024-03-26T17:55:32.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.161+0000 7f2f829f4640 1 -- 172.21.15.105:0/1914365614 wait complete. 2024-03-26T17:55:32.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:31 smithi135 ceph-mon[35636]: pgmap v864: 97 pgs: 97 active+clean; 283 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-03-26T17:55:32.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:31 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2231210064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:55:32.161 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:55:32.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/3131885478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0102ac0 msgr2=0x7f3fb0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/3131885478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0102ac0 0x7f3fb0102f40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009860 tx=0x7f3fa002bb70 comp rx=0 tx=0).stop 2024-03-26T17:55:32.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/3131885478 shutdown_connections 2024-03-26T17:55:32.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/3131885478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb0103480 0x7f3fb0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/3131885478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0102ac0 0x7f3fb0102f40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/3131885478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb01018d0 0x7f3fb0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/3131885478 >> 172.21.15.105:0/3131885478 conn(0x7f3fb00fce20 msgr2=0x7f3fb00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/3131885478 shutdown_connections 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.353+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/3131885478 wait complete. 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb7e9a640 1 Processor -- start 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb7e9a640 1 -- start start 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb7e9a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb01018d0 0x7f3fb019e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb7e9a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb0102ac0 0x7f3fb019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb7e9a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0103480 0x7f3fb01a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb7e9a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0113a10 con 0x7f3fb0102ac0 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb7e9a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0113890 con 0x7f3fb01018d0 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb7e9a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0113b90 con 0x7f3fb0103480 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.354+0000 7f3fb5435640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb0102ac0 0x7f3fb019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5435640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb0102ac0 0x7f3fb019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37352/0 (socket says 172.21.15.105:37352) 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5435640 1 -- 172.21.15.105:0/1320427107 learned_addr learned my addr 172.21.15.105:0/1320427107 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb01018d0 0x7f3fb019e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb6437640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0103480 0x7f3fb01a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5435640 1 -- 172.21.15.105:0/1320427107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0103480 msgr2=0x7f3fb01a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5435640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0103480 0x7f3fb01a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5435640 1 -- 172.21.15.105:0/1320427107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb01018d0 msgr2=0x7f3fb019e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5435640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb01018d0 0x7f3fb019e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5435640 1 -- 172.21.15.105:0/1320427107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb01a5d50 con 0x7f3fb0102ac0 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb6437640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0103480 0x7f3fb01a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb5435640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb0102ac0 0x7f3fb019e670 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0005940 tx=0x7f3fa0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fa6ffd640 1 -- 172.21.15.105:0/1320427107 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0041020 con 0x7f3fb0102ac0 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/1320427107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb01a5fe0 con 0x7f3fb0102ac0 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/1320427107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01a6450 con 0x7f3fb0102ac0 2024-03-26T17:55:32.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fa6ffd640 1 -- 172.21.15.105:0/1320427107 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fa002c9c0 con 0x7f3fb0102ac0 2024-03-26T17:55:32.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.355+0000 7f3fa6ffd640 1 -- 172.21.15.105:0/1320427107 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0039040 con 0x7f3fb0102ac0 2024-03-26T17:55:32.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:31 smithi144 ceph-mon[36095]: pgmap v864: 97 pgs: 97 active+clean; 283 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-03-26T17:55:32.346 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:31 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2231210064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:55:32.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fbab63740 1 Processor -- start 2024-03-26T17:55:32.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fbab63740 1 -- start start 2024-03-26T17:55:32.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fa6ffd640 1 -- 172.21.15.105:0/1320427107 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3fa0044020 con 0x7f3fb0102ac0 2024-03-26T17:55:32.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b4a30 0x555e8c4b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fbab63740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555e8c419970 con 0x555e8c4b5420 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fbab63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555e8c3c1d70 con 0x555e8c4b4a30 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fbab63740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e8c415920 con 0x555e8c4ba590 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fb5c36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fb6437640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b4a30 0x555e8c4b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fb6437640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b4a30 0x555e8c4b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49140/0 (socket says 172.21.15.105:49140) 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fb6437640 1 -- 172.21.15.105:0/1924611629 learned_addr learned my addr 172.21.15.105:0/1924611629 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.359+0000 7f3fb5435640 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb6437640 1 -- 172.21.15.105:0/1924611629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 msgr2=0x555e8c4bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb6437640 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb6437640 1 -- 172.21.15.105:0/1924611629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 msgr2=0x555e8c4b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb6437640 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb6437640 1 -- 172.21.15.105:0/1924611629 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8c12d3f0 con 0x555e8c4b4a30 2024-03-26T17:55:32.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb5435640 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fa6ffd640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f9c0a2f10 0x7f3f9c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb6437640 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b4a30 0x555e8c4b4e50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80117e0 tx=0x7f3fa8011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd72a2df8c7ed655 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f9c0a2f10 0x7f3f9c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3fa6ffd640 1 -- 172.21.15.105:0/1320427107 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3fa00f0d60 con 0x7f3fb0102ac0 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3f9bfff640 1 -- 172.21.15.105:0/1924611629 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8011310 con 0x555e8c4b4a30 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3f9bfff640 1 -- 172.21.15.105:0/1924611629 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fa8003e20 con 0x555e8c4b4a30 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.360+0000 7f3f9bfff640 1 -- 172.21.15.105:0/1924611629 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80044f0 con 0x555e8c4b4a30 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.361+0000 7f3fbab63740 1 -- 172.21.15.105:0/1924611629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b4a30 msgr2=0x555e8c4b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.361+0000 7f3fbab63740 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b4a30 0x555e8c4b4e50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80117e0 tx=0x7f3fa8011ca0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.361+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f9c0a2f10 0x7f3f9c0a53d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x555e8c4b5820 tx=0x7f3fac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 -- 172.21.15.105:0/1924611629 shutdown_connections 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 --2- 172.21.15.105:0/1924611629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b4a30 0x555e8c4b4e50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 -- 172.21.15.105:0/1924611629 >> 172.21.15.105:0/1924611629 conn(0x555e8c4affa0 msgr2=0x555e8c4b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 -- 172.21.15.105:0/1924611629 shutdown_connections 2024-03-26T17:55:32.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 -- 172.21.15.105:0/1924611629 wait complete. 2024-03-26T17:55:32.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 -- start start 2024-03-26T17:55:32.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.362+0000 7f3fbab63740 1 Processor -- start 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4bef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4bf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fbab63740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555e8c4bd540 con 0x555e8c4b4a30 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fbab63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555e8c3d5310 con 0x555e8c4b5420 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fbab63740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e8c4bd6c0 con 0x555e8c4ba590 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fbab63740 10 client.0 _ll_register_callbacks cb 0x555e8c410c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5c36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4bef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5c36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4bef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37380/0 (socket says 172.21.15.105:37380) 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5c36640 1 -- 172.21.15.105:0/3215257238 learned_addr learned my addr 172.21.15.105:0/3215257238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5435640 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4bf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb6437640 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5c36640 1 -- 172.21.15.105:0/3215257238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 msgr2=0x555e8c4c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5c36640 1 -- 172.21.15.105:0/3215257238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 msgr2=0x555e8c4bf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4bf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.363+0000 7f3fb5c36640 1 -- 172.21.15.105:0/3215257238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8c1f77d0 con 0x555e8c4b4a30 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.364+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4bef80 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009fd0 tx=0x7f3fac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492f45a0d84c938f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.364+0000 7f3f997fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.364+0000 7f3f997fa640 1 -- 172.21.15.105:0/3215257238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac01dd00 con 0x555e8c4b4a30 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.364+0000 7f3f997fa640 1 -- 172.21.15.105:0/3215257238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3fac018030 con 0x555e8c4b4a30 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.364+0000 7f3fbab63740 1 -- 172.21.15.105:0/3215257238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fa8003e20 con 0x555e8c4b4a30 2024-03-26T17:55:32.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.364+0000 7f3f997fa640 1 -- 172.21.15.105:0/3215257238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3fac005020 con 0x555e8c4b4a30 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.364+0000 7f3fbab63740 10 client.21726 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.364+0000 7f3fbab63740 1 -- 172.21.15.105:0/3215257238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555e8c4c9930 con 0x555e8c4b4a30 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.365+0000 7f3f997fa640 1 -- 172.21.15.105:0/3215257238 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fa8003e20 con 0x555e8c4b4a30 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.365+0000 7f3f997fa640 1 -- 172.21.15.105:0/3215257238 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3fac0047a0 con 0x555e8c4b4a30 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.365+0000 7f3fbab63740 10 client.21726 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.365+0000 7f3fbab63740 10 client.21726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.365+0000 7f3fbab63740 20 client.21726 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.365+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e8c4cb000 0x555e8c4cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.365+0000 7f3fbab63740 4 client.21726 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.365+0000 7f3fbab63740 1 -- 172.21.15.105:0/3215257238 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e8c426eb0 con 0x555e8c4cb000 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.366+0000 7f3fb5435640 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e8c4cb000 0x555e8c4cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.366+0000 7f3fb5435640 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e8c4cb000 0x555e8c4cd450 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6cde6913483e343 server_cookie=d9be943976f3b4f6 in_seq=0 out_seq=0 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.366+0000 7f3f997fa640 10 client.21726 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:32.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3f997fa640 1 -- 172.21.15.105:0/3215257238 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555e8c426eb0 con 0x555e8c4cb000 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3f997fa640 10 client.21726 handle_command_reply: tid=0 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 client.21726 shutdown 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 -- 172.21.15.105:0/3215257238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 msgr2=0x555e8c4bef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4bef80 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009fd0 tx=0x7f3fac004730 comp rx=0 tx=0).stop 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 -- 172.21.15.105:0/3215257238 shutdown_connections 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e8c4cb000 0x555e8c4cd450 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4bf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3f997fa640 0 client.21726 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3215257238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4bef80 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 -- 172.21.15.105:0/3215257238 >> 172.21.15.105:0/3215257238 conn(0x555e8c4affa0 msgr2=0x555e8c4af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 -- 172.21.15.105:0/3215257238 shutdown_connections 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.368+0000 7f3fbab63740 1 -- 172.21.15.105:0/3215257238 wait complete. 2024-03-26T17:55:32.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.369+0000 7f3fbab63740 20 client.21726 trim_cache size 0 max 16384 2024-03-26T17:55:32.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.390+0000 7f3fbab63740 1 Processor -- start 2024-03-26T17:55:32.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.390+0000 7f3fbab63740 1 -- start start 2024-03-26T17:55:32.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.390+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4b4a30 0x555e8c4b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.390+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.390+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4ba590 0x555e8c4bf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.390+0000 7f3fbab63740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01af440 con 0x555e8c4b5420 2024-03-26T17:55:32.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.390+0000 7f3fbab63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b0f80 con 0x555e8c4ba590 2024-03-26T17:55:32.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.390+0000 7f3fbab63740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01afb10 con 0x555e8c4b4a30 2024-03-26T17:55:32.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5c36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4b4a30 0x555e8c4b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5c36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4b4a30 0x555e8c4b6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53424/0 (socket says 172.21.15.105:53424) 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5c36640 1 -- 172.21.15.105:0/3284298816 learned_addr learned my addr 172.21.15.105:0/3284298816 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5435640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb6437640 1 --2- 172.21.15.105:0/3284298816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4ba590 0x555e8c4bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5435640 1 -- 172.21.15.105:0/3284298816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4b4a30 msgr2=0x555e8c4b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5435640 1 --2- 172.21.15.105:0/3284298816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4b4a30 0x555e8c4b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5435640 1 -- 172.21.15.105:0/3284298816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4ba590 msgr2=0x555e8c4bf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5435640 1 --2- 172.21.15.105:0/3284298816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4ba590 0x555e8c4bf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5435640 1 -- 172.21.15.105:0/3284298816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8c4b9c00 con 0x555e8c4b5420 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.391+0000 7f3fb5435640 1 --2- 172.21.15.105:0/3284298816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4c6490 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00ee870 tx=0x7f3fa00f4840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b16d5dda41101a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.392+0000 7f3f9b7fe640 1 -- 172.21.15.105:0/3284298816 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00b96f0 con 0x555e8c4b5420 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.392+0000 7f3f9b7fe640 1 -- 172.21.15.105:0/3284298816 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fa00bdb50 con 0x555e8c4b5420 2024-03-26T17:55:32.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.392+0000 7f3f9b7fe640 1 -- 172.21.15.105:0/3284298816 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00bcdb0 con 0x555e8c4b5420 2024-03-26T17:55:32.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 -- 172.21.15.105:0/3284298816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 msgr2=0x555e8c4c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3284298816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4c6490 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00ee870 tx=0x7f3fa00f4840 comp rx=0 tx=0).stop 2024-03-26T17:55:32.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 -- 172.21.15.105:0/3284298816 shutdown_connections 2024-03-26T17:55:32.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3284298816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4ba590 0x555e8c4bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3284298816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b5420 0x555e8c4c6490 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 --2- 172.21.15.105:0/3284298816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4b4a30 0x555e8c4b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 -- 172.21.15.105:0/3284298816 >> 172.21.15.105:0/3284298816 conn(0x555e8c4affa0 msgr2=0x555e8c4b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 -- 172.21.15.105:0/3284298816 shutdown_connections 2024-03-26T17:55:32.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 -- 172.21.15.105:0/3284298816 wait complete. 2024-03-26T17:55:32.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.393+0000 7f3fbab63740 1 -- start start 2024-03-26T17:55:32.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.394+0000 7f3fbab63740 1 Processor -- start 2024-03-26T17:55:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.394+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.394+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4c4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.394+0000 7f3fbab63740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.394+0000 7f3fbab63740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b08b0 con 0x555e8c4b4a30 2024-03-26T17:55:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.394+0000 7f3fbab63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b01e0 con 0x555e8c4b5420 2024-03-26T17:55:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.394+0000 7f3fbab63740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01ad420 con 0x555e8c4ba590 2024-03-26T17:55:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.394+0000 7f3fbab63740 10 client.0 _ll_register_callbacks cb 0x7f3fac009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:32.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53432/0 (socket says 172.21.15.105:53432) 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 -- 172.21.15.105:0/940762261 learned_addr learned my addr 172.21.15.105:0/940762261 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb5435640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 -- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 msgr2=0x555e8c4c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 -- 172.21.15.105:0/940762261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 msgr2=0x555e8c4c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 -- 172.21.15.105:0/940762261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8c4c5920 con 0x555e8c4ba590 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb5435640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.395+0000 7f3fb6437640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c5330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800f790 tx=0x7f3fa8003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa227be7f3750d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:32.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.396+0000 7f3f997fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.396+0000 7f3f997fa640 1 -- 172.21.15.105:0/940762261 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800f9a0 con 0x555e8c4ba590 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.396+0000 7f3f997fa640 1 -- 172.21.15.105:0/940762261 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3fa800fb40 con 0x555e8c4ba590 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.396+0000 7f3fbab63740 1 -- 172.21.15.105:0/940762261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fa00bdb50 con 0x555e8c4ba590 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.396+0000 7f3f997fa640 1 -- 172.21.15.105:0/940762261 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3fa8036050 con 0x555e8c4ba590 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.396+0000 7f3fbab63740 10 client.31447 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.396+0000 7f3fbab63740 1 -- 172.21.15.105:0/940762261 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555e8c531480 con 0x555e8c4ba590 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.396+0000 7f3f997fa640 1 -- 172.21.15.105:0/940762261 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fa00bdb50 con 0x555e8c4ba590 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3f997fa640 1 -- 172.21.15.105:0/940762261 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3fa8018020 con 0x555e8c4ba590 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3fbab63740 10 client.31447 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3fbab63740 10 client.31447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3fbab63740 20 client.31447 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3fbab63740 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e8c531cc0 0x555e8c5340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:32.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3fbab63740 4 client.31447 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:55:32.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3fbab63740 1 -- 172.21.15.105:0/940762261 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e8c421200 con 0x555e8c531cc0 2024-03-26T17:55:32.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e8c531cc0 0x555e8c5340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:32.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3f997fa640 10 client.31447 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:32.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.397+0000 7f3fb5c36640 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e8c531cc0 0x555e8c5340e0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6100a78b60167dc server_cookie=ab5a8aa65d220496 in_seq=0 out_seq=0 2024-03-26T17:55:32.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.399+0000 7f3f997fa640 1 -- 172.21.15.105:0/940762261 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e8c421200 con 0x555e8c531cc0 2024-03-26T17:55:32.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.399+0000 7f3f997fa640 10 client.31447 handle_command_reply: tid=0 2024-03-26T17:55:32.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.399+0000 7f3fbab63740 1 client.31447 shutdown 2024-03-26T17:55:32.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 -- 172.21.15.105:0/940762261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 msgr2=0x555e8c4c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c5330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800f790 tx=0x7f3fa8003ba0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 -- 172.21.15.105:0/940762261 shutdown_connections 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e8c531cc0 0x555e8c5340e0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e8c4ba590 0x555e8c4c5330 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e8c4b5420 0x555e8c4c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 --2- 172.21.15.105:0/940762261 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e8c4b4a30 0x555e8c4c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 -- 172.21.15.105:0/940762261 >> 172.21.15.105:0/940762261 conn(0x555e8c4affa0 msgr2=0x555e8c4b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3f997fa640 0 client.31447 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 -- 172.21.15.105:0/940762261 shutdown_connections 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 1 -- 172.21.15.105:0/940762261 wait complete. 2024-03-26T17:55:32.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.400+0000 7f3fbab63740 20 client.31447 trim_cache size 0 max 16384 2024-03-26T17:55:32.391 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:55:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f9c0a2f10 msgr2=0x7f3f9c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f9c0a2f10 0x7f3f9c0a53d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x555e8c4b5820 tx=0x7f3fac009260 comp rx=0 tx=0).stop 2024-03-26T17:55:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb0102ac0 msgr2=0x7f3fb019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb0102ac0 0x7f3fb019e670 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0005940 tx=0x7f3fa0004270 comp rx=0 tx=0).stop 2024-03-26T17:55:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/1320427107 shutdown_connections 2024-03-26T17:55:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f9c0a2f10 0x7f3f9c0a53d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fb0103480 0x7f3fb01a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3fb0102ac0 0x7f3fb019e670 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 --2- 172.21.15.105:0/1320427107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb01018d0 0x7f3fb019e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:32.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/1320427107 >> 172.21.15.105:0/1320427107 conn(0x7f3fb00fce20 msgr2=0x7f3fb0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:32.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.401+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/1320427107 shutdown_connections 2024-03-26T17:55:32.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:32.402+0000 7f3fb7e9a640 1 -- 172.21.15.105:0/1320427107 wait complete. 2024-03-26T17:55:32.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 33581 7.09 MB/sec execute 135 sec latency 125.466 ms 2024-03-26T17:55:33.406 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:55:33.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:55:33.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:55:33.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 33706 7.12 MB/sec execute 136 sec latency 135.933 ms 2024-03-26T17:55:33.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3505491678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 msgr2=0x7f69b40ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3505491678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 0x7f69b40ffdb0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009900 tx=0x7f69a802bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:33.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3505491678 shutdown_connections 2024-03-26T17:55:33.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3505491678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b41002f0 0x7f69b41007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3505491678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 0x7f69b40ffdb0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3505491678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69b4101e40 0x7f69b4102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3505491678 >> 172.21.15.105:0/3505491678 conn(0x7f69b40fb4b0 msgr2=0x7f69b40fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3505491678 shutdown_connections 2024-03-26T17:55:33.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.574+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3505491678 wait complete. 2024-03-26T17:55:33.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b8cc0640 1 Processor -- start 2024-03-26T17:55:33.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b8cc0640 1 -- start start 2024-03-26T17:55:33.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b8cc0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 0x7f69b4199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b8cc0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69b41002f0 0x7f69b419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b8cc0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b4101e40 0x7f69b41a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b8cc0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69b410f660 con 0x7f69b40ff930 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b8cc0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69b410f4e0 con 0x7f69b41002f0 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b8cc0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69b410f7e0 con 0x7f69b4101e40 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 0x7f69b4199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 0x7f69b4199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37416/0 (socket says 172.21.15.105:37416) 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b259c640 1 -- 172.21.15.105:0/3259795184 learned_addr learned my addr 172.21.15.105:0/3259795184 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69b41002f0 0x7f69b419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.575+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b4101e40 0x7f69b41a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69b259c640 1 -- 172.21.15.105:0/3259795184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b4101e40 msgr2=0x7f69b41a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69b259c640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b4101e40 0x7f69b41a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69b259c640 1 -- 172.21.15.105:0/3259795184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69b41002f0 msgr2=0x7f69b419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69b259c640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69b41002f0 0x7f69b419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69b259c640 1 -- 172.21.15.105:0/3259795184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b41a1970 con 0x7f69b40ff930 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b4101e40 0x7f69b41a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69b259c640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 0x7f69b4199d50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f69a400e9f0 tx=0x7f69a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69a37fe640 1 -- 172.21.15.105:0/3259795184 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a400cd80 con 0x7f69b40ff930 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69a37fe640 1 -- 172.21.15.105:0/3259795184 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69a40108e0 con 0x7f69b40ff930 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.576+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3259795184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b41a1c60 con 0x7f69b40ff930 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.577+0000 7f69a37fe640 1 -- 172.21.15.105:0/3259795184 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4018920 con 0x7f69b40ff930 2024-03-26T17:55:33.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.577+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3259795184 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b41a21d0 con 0x7f69b40ff930 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69a37fe640 1 -- 172.21.15.105:0/3259795184 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f69a4010a90 con 0x7f69b40ff930 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69bb986740 1 Processor -- start 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69bb986740 1 -- start start 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69bb986740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1190 0x560e12ed15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69bb986740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12ed6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69bb986740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed6cf0 0x560e12ed9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69bb986740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560e12e36630 con 0x560e12ed1190 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69bb986740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560e12ddee10 con 0x560e12ed1b80 2024-03-26T17:55:33.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69bb986740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560e12e325e0 con 0x560e12ed6cf0 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.580+0000 7f69a37fe640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f699c0a2fa0 0x7f699c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed6cf0 0x560e12ed9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12ed6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed6cf0 0x560e12ed9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53440/0 (socket says 172.21.15.105:53440) 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 -- 172.21.15.105:0/1164171069 learned_addr learned my addr 172.21.15.105:0/1164171069 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/1164171069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1190 0x560e12ed15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f699c0a2fa0 0x7f699c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69a37fe640 1 -- 172.21.15.105:0/3259795184 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f69a40cf670 con 0x7f69b40ff930 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 -- 172.21.15.105:0/1164171069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 msgr2=0x560e12ed6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/1164171069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12ed6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 -- 172.21.15.105:0/1164171069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1190 msgr2=0x560e12ed15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/1164171069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1190 0x560e12ed15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 -- 172.21.15.105:0/1164171069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e12df5740 con 0x560e12ed6cf0 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/1164171069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed6cf0 0x560e12ed9180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f69a8000c00 tx=0x7f69a80095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46be5f39ddc34248 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f699c0a2fa0 0x7f699c0a5460 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0099e0 tx=0x7f69ac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69a0ff9640 1 -- 172.21.15.105:0/1164171069 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8045020 con 0x560e12ed6cf0 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69a0ff9640 1 -- 172.21.15.105:0/1164171069 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69a802c9a0 con 0x560e12ed6cf0 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.581+0000 7f69a0ff9640 1 -- 172.21.15.105:0/1164171069 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a80055c0 con 0x560e12ed6cf0 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.582+0000 7f69bb986740 1 -- 172.21.15.105:0/1164171069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed6cf0 msgr2=0x560e12ed9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.582+0000 7f69bb986740 1 --2- 172.21.15.105:0/1164171069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed6cf0 0x560e12ed9180 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f69a8000c00 tx=0x7f69a80095f0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.582+0000 7f69bb986740 1 -- 172.21.15.105:0/1164171069 shutdown_connections 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.582+0000 7f69bb986740 1 --2- 172.21.15.105:0/1164171069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed6cf0 0x560e12ed9180 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.582+0000 7f69bb986740 1 --2- 172.21.15.105:0/1164171069 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12ed6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.582+0000 7f69bb986740 1 --2- 172.21.15.105:0/1164171069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1190 0x560e12ed15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.582+0000 7f69bb986740 1 -- 172.21.15.105:0/1164171069 >> 172.21.15.105:0/1164171069 conn(0x560e12ecc700 msgr2=0x560e12eccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 -- 172.21.15.105:0/1164171069 shutdown_connections 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 -- 172.21.15.105:0/1164171069 wait complete. 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 -- start start 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 Processor -- start 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 0x560e12edb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12edbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560e12ed9c40 con 0x560e12ed6cf0 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560e12df21a0 con 0x560e12ed1b80 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560e12ed9dc0 con 0x560e12ed1190 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69b259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 0x560e12edb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69b259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 0x560e12edb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53446/0 (socket says 172.21.15.105:53446) 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69b259c640 1 -- 172.21.15.105:0/2894381527 learned_addr learned my addr 172.21.15.105:0/2894381527 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69bb986740 10 client.0 _ll_register_callbacks cb 0x560e12b49c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.583+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12edbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69b2d9d640 1 -- 172.21.15.105:0/2894381527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 msgr2=0x560e12edb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 0x560e12edb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69b2d9d640 1 -- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 msgr2=0x560e12edbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12edbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69b2d9d640 1 -- 172.21.15.105:0/2894381527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e12c127e0 con 0x560e12ed6cf0 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12edbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee2b90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f69ac005fd0 tx=0x7f69ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d567fec5677220fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f6999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f6999ffb640 1 -- 172.21.15.105:0/2894381527 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac011020 con 0x560e12ed6cf0 2024-03-26T17:55:33.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f6999ffb640 1 -- 172.21.15.105:0/2894381527 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f69ac0111c0 con 0x560e12ed6cf0 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69bb986740 1 -- 172.21.15.105:0/2894381527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a802c9a0 con 0x560e12ed6cf0 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f6999ffb640 1 -- 172.21.15.105:0/2894381527 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f69ac01f670 con 0x560e12ed6cf0 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69bb986740 10 client.21744 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f69bb986740 1 -- 172.21.15.105:0/2894381527 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560e12c72f70 con 0x560e12ed6cf0 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.584+0000 7f6999ffb640 1 -- 172.21.15.105:0/2894381527 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69a802c9a0 con 0x560e12ed6cf0 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f6999ffb640 1 -- 172.21.15.105:0/2894381527 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f69ac028070 con 0x560e12ed6cf0 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f69bb986740 10 client.21744 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f69bb986740 10 client.21744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f69bb986740 20 client.21744 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f69bb986740 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560e12ee74a0 0x560e12ee9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f69bb986740 4 client.21744 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f69bb986740 1 -- 172.21.15.105:0/2894381527 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e12e43b70 con 0x560e12ee74a0 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f69b259c640 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560e12ee74a0 0x560e12ee9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f6999ffb640 10 client.21744 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:33.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.585+0000 7f69b259c640 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560e12ee74a0 0x560e12ee9940 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b47f524157d698 server_cookie=6d07740f548d3310 in_seq=0 out_seq=0 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f6999ffb640 1 -- 172.21.15.105:0/2894381527 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560e12e43b70 con 0x560e12ee74a0 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f6999ffb640 10 client.21744 handle_command_reply: tid=0 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 client.21744 shutdown 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 -- 172.21.15.105:0/2894381527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 msgr2=0x560e12ee2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee2b90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f69ac005fd0 tx=0x7f69ac004a40 comp rx=0 tx=0).stop 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 -- 172.21.15.105:0/2894381527 shutdown_connections 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560e12ee74a0 0x560e12ee9940 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee2b90 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f6999ffb640 0 client.21744 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1b80 0x560e12edbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 --2- 172.21.15.105:0/2894381527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 0x560e12edb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 -- 172.21.15.105:0/2894381527 >> 172.21.15.105:0/2894381527 conn(0x560e12ecc700 msgr2=0x560e12ecb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.587+0000 7f69bb986740 1 -- 172.21.15.105:0/2894381527 shutdown_connections 2024-03-26T17:55:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.588+0000 7f69bb986740 1 -- 172.21.15.105:0/2894381527 wait complete. 2024-03-26T17:55:33.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.588+0000 7f69bb986740 20 client.21744 trim_cache size 0 max 16384 2024-03-26T17:55:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69bb986740 1 Processor -- start 2024-03-26T17:55:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69bb986740 1 -- start start 2024-03-26T17:55:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69bb986740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 0x560e12ecb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69bb986740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1b80 0x560e12ee2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69bb986740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed6cf0 0x560e12ecb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69bb986740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69b41aaf20 con 0x560e12ed1b80 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69bb986740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69b41aca60 con 0x560e12ed6cf0 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69bb986740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69b41ab5f0 con 0x560e12ed1190 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69b1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1b80 0x560e12ee2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69b1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1b80 0x560e12ee2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37448/0 (socket says 172.21.15.105:37448) 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69b1d9b640 1 -- 172.21.15.105:0/3716339288 learned_addr learned my addr 172.21.15.105:0/3716339288 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69b1d9b640 1 -- 172.21.15.105:0/3716339288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 msgr2=0x560e12ecb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.616+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/3716339288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 0x560e12ecb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.617+0000 7f69b1d9b640 1 -- 172.21.15.105:0/3716339288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed6cf0 msgr2=0x560e12ecb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.617+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/3716339288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed6cf0 0x560e12ecb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.617+0000 7f69b1d9b640 1 -- 172.21.15.105:0/3716339288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e12b0a170 con 0x560e12ed1b80 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.617+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/3716339288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1b80 0x560e12ee2b50 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005ec0 tx=0x7f69a8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7523cd012e7d3f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.617+0000 7f699bfff640 1 -- 172.21.15.105:0/3716339288 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8045030 con 0x560e12ed1b80 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.617+0000 7f699bfff640 1 -- 172.21.15.105:0/3716339288 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69a80035f0 con 0x560e12ed1b80 2024-03-26T17:55:33.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.617+0000 7f699bfff640 1 -- 172.21.15.105:0/3716339288 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a802cca0 con 0x560e12ed1b80 2024-03-26T17:55:33.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 -- 172.21.15.105:0/3716339288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1b80 msgr2=0x560e12ee2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 --2- 172.21.15.105:0/3716339288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1b80 0x560e12ee2b50 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005ec0 tx=0x7f69a8005b50 comp rx=0 tx=0).stop 2024-03-26T17:55:33.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 -- 172.21.15.105:0/3716339288 shutdown_connections 2024-03-26T17:55:33.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 --2- 172.21.15.105:0/3716339288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed6cf0 0x560e12ecb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 --2- 172.21.15.105:0/3716339288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed1b80 0x560e12ee2b50 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 --2- 172.21.15.105:0/3716339288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1190 0x560e12ecb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 -- 172.21.15.105:0/3716339288 >> 172.21.15.105:0/3716339288 conn(0x560e12ecc700 msgr2=0x560e12ed3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 -- 172.21.15.105:0/3716339288 shutdown_connections 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 -- 172.21.15.105:0/3716339288 wait complete. 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.618+0000 7f69bb986740 1 -- start start 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69bb986740 1 Processor -- start 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69bb986740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1190 0x560e12ee03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69bb986740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1b80 0x560e12ee0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69bb986740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69bb986740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69b41ac390 con 0x560e12ed6cf0 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69bb986740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69b41abcc0 con 0x560e12ed1190 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69bb986740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69b41a8f00 con 0x560e12ed1b80 2024-03-26T17:55:33.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1b80 0x560e12ee0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1b80 0x560e12ee0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53466/0 (socket says 172.21.15.105:53466) 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b1d9b640 1 -- 172.21.15.105:0/3785067420 learned_addr learned my addr 172.21.15.105:0/3785067420 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b259c640 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1190 0x560e12ee03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b2d9d640 1 -- 172.21.15.105:0/3785067420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1b80 msgr2=0x560e12ee0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1b80 0x560e12ee0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b2d9d640 1 -- 172.21.15.105:0/3785067420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1190 msgr2=0x560e12ee03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1190 0x560e12ee03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b2d9d640 1 -- 172.21.15.105:0/3785067420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e12ee1f30 con 0x560e12ed6cf0 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.619+0000 7f69b1d9b640 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1b80 0x560e12ee0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f69bb986740 10 client.0 _ll_register_callbacks cb 0x7f69ac003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f69b2d9d640 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee1940 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f69ac033840 tx=0x7f69ac033c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28282751a7babe8c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f6999ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:33.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f69bb986740 1 -- 172.21.15.105:0/3785067420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a80035f0 con 0x560e12ed6cf0 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f6999ffb640 1 -- 172.21.15.105:0/3785067420 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac003e90 con 0x560e12ed6cf0 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f6999ffb640 1 -- 172.21.15.105:0/3785067420 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f69ac02f030 con 0x560e12ed6cf0 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f6999ffb640 1 -- 172.21.15.105:0/3785067420 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f69ac00fe40 con 0x560e12ed6cf0 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f69bb986740 10 client.21750 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f69bb986740 1 -- 172.21.15.105:0/3785067420 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=37}) -- 0x560e12f4d840 con 0x560e12ed6cf0 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.620+0000 7f6999ffb640 1 -- 172.21.15.105:0/3785067420 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f69ac028050 con 0x560e12ed6cf0 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f69bb986740 10 client.21750 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f69bb986740 10 client.21750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f69bb986740 20 client.21750 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f69bb986740 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560e12ee74a0 0x560e12f50170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f69bb986740 4 client.21750 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f69bb986740 1 -- 172.21.15.105:0/3785067420 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e12e3dec0 con 0x560e12ee74a0 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f69b259c640 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560e12ee74a0 0x560e12f50170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f6999ffb640 10 client.21750 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:33.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.621+0000 7f69b259c640 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560e12ee74a0 0x560e12f50170 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a6352fcf42fde0a server_cookie=388d86314494dc1e in_seq=0 out_seq=0 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f6999ffb640 1 -- 172.21.15.105:0/3785067420 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e12e3dec0 con 0x560e12ee74a0 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f6999ffb640 10 client.21750 handle_command_reply: tid=0 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 client.21750 shutdown 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 -- 172.21.15.105:0/3785067420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 msgr2=0x560e12ee1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee1940 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f69ac033840 tx=0x7f69ac033c60 comp rx=0 tx=0).stop 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 -- 172.21.15.105:0/3785067420 shutdown_connections 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560e12ee74a0 0x560e12f50170 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560e12ed6cf0 0x560e12ee1940 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f6999ffb640 0 client.21750 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560e12ed1b80 0x560e12ee0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 --2- 172.21.15.105:0/3785067420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560e12ed1190 0x560e12ee03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 -- 172.21.15.105:0/3785067420 >> 172.21.15.105:0/3785067420 conn(0x560e12ecc700 msgr2=0x560e12ecf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 -- 172.21.15.105:0/3785067420 shutdown_connections 2024-03-26T17:55:33.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 1 -- 172.21.15.105:0/3785067420 wait complete. 2024-03-26T17:55:33.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.624+0000 7f69bb986740 20 client.21750 trim_cache size 0 max 16384 2024-03-26T17:55:33.619 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:55:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:55:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "1ae9201e-b1f5-40a6-b5b0-f2e0e43be53d", 2024-03-26T17:55:33.619 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:55:33.619 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:55:33.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f699c0a2fa0 msgr2=0x7f699c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f699c0a2fa0 0x7f699c0a5460 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0099e0 tx=0x7f69ac009500 comp rx=0 tx=0).stop 2024-03-26T17:55:33.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 msgr2=0x7f69b4199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 0x7f69b4199d50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f69a400e9f0 tx=0x7f69a400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3259795184 shutdown_connections 2024-03-26T17:55:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f699c0a2fa0 0x7f699c0a5460 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b4101e40 0x7f69b41a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69b41002f0 0x7f69b419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 --2- 172.21.15.105:0/3259795184 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69b40ff930 0x7f69b4199d50 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3259795184 >> 172.21.15.105:0/3259795184 conn(0x7f69b40fb4b0 msgr2=0x7f69b40fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3259795184 shutdown_connections 2024-03-26T17:55:33.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.625+0000 7f69b8cc0640 1 -- 172.21.15.105:0/3259795184 wait complete. 2024-03-26T17:55:33.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:55:33.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:55:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 -- 172.21.15.105:0/1585094031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a4100800 msgr2=0x7fe5a4100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/1585094031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a4100800 0x7fe5a4100c20 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fe594009900 tx=0x7fe59402bc30 comp rx=0 tx=0).stop 2024-03-26T17:55:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 -- 172.21.15.105:0/1585094031 shutdown_connections 2024-03-26T17:55:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/1585094031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a41023b0 0x7fe5a4108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/1585094031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41019f0 0x7fe5a4101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/1585094031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a4100800 0x7fe5a4100c20 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 -- 172.21.15.105:0/1585094031 >> 172.21.15.105:0/1585094031 conn(0x7fe5a40fbd70 msgr2=0x7fe5a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 -- 172.21.15.105:0/1585094031 shutdown_connections 2024-03-26T17:55:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.772+0000 7fe5a97c6640 1 -- 172.21.15.105:0/1585094031 wait complete. 2024-03-26T17:55:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.773+0000 7fe5a97c6640 1 Processor -- start 2024-03-26T17:55:33.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.773+0000 7fe5a97c6640 1 -- start start 2024-03-26T17:55:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.773+0000 7fe5a97c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a4100800 0x7fe5a410c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.773+0000 7fe5a97c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a41019f0 0x7fe5a410acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.773+0000 7fe5a97c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41023b0 0x7fe5a410b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a97c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a410d420 con 0x7fe5a41019f0 2024-03-26T17:55:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a97c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a410d2a0 con 0x7fe5a4100800 2024-03-26T17:55:33.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a97c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a410d5a0 con 0x7fe5a41023b0 2024-03-26T17:55:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a4100800 0x7fe5a410c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a4100800 0x7fe5a410c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49258/0 (socket says 172.21.15.105:49258) 2024-03-26T17:55:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a2ffd640 1 -- 172.21.15.105:0/960195269 learned_addr learned my addr 172.21.15.105:0/960195269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41023b0 0x7fe5a410b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a41019f0 0x7fe5a410acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a2ffd640 1 -- 172.21.15.105:0/960195269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41023b0 msgr2=0x7fe5a410b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a2ffd640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41023b0 0x7fe5a410b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a2ffd640 1 -- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a41019f0 msgr2=0x7fe5a410acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a2ffd640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a41019f0 0x7fe5a410acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a2ffd640 1 -- 172.21.15.105:0/960195269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a410b750 con 0x7fe5a4100800 2024-03-26T17:55:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.774+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a41019f0 0x7fe5a410acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:33.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.775+0000 7fe5a2ffd640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a4100800 0x7fe5a410c690 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe594000c00 tx=0x7fe594005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.775+0000 7fe58bfff640 1 -- 172.21.15.105:0/960195269 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594043020 con 0x7fe5a4100800 2024-03-26T17:55:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.775+0000 7fe58bfff640 1 -- 172.21.15.105:0/960195269 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5940042d0 con 0x7fe5a4100800 2024-03-26T17:55:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.775+0000 7fe58bfff640 1 -- 172.21.15.105:0/960195269 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594003b10 con 0x7fe5a4100800 2024-03-26T17:55:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.775+0000 7fe5a97c6640 1 -- 172.21.15.105:0/960195269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a410b9e0 con 0x7fe5a4100800 2024-03-26T17:55:33.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.775+0000 7fe5a97c6640 1 -- 172.21.15.105:0/960195269 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a41a6610 con 0x7fe5a4100800 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.778+0000 7fe58bfff640 1 -- 172.21.15.105:0/960195269 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe594046020 con 0x7fe5a4100800 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.778+0000 7fe5ac3b3740 1 Processor -- start 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.778+0000 7fe5ac3b3740 1 -- start start 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070e8f0 0x558c9070ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90713de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 0x558c907168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5ac3b3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c90673980 con 0x558c90714450 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5ac3b3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c9061bbe0 con 0x558c9070e8f0 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5ac3b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c9066f930 con 0x558c9070f2e0 2024-03-26T17:55:33.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90713de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90713de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53496/0 (socket says 172.21.15.105:53496) 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070e8f0 0x558c9070ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.779+0000 7fe5a2ffd640 1 -- 172.21.15.105:0/2928090535 learned_addr learned my addr 172.21.15.105:0/2928090535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.780+0000 7fe5a37fe640 1 -- 172.21.15.105:0/2928090535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 msgr2=0x558c90713de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.780+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/2928090535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90713de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.780+0000 7fe5a37fe640 1 -- 172.21.15.105:0/2928090535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 msgr2=0x558c907168e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.780+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/2928090535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 0x558c907168e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.780+0000 7fe5a37fe640 1 -- 172.21.15.105:0/2928090535 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c904517d0 con 0x558c9070e8f0 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.780+0000 7fe5a2ffd640 1 --2- 172.21.15.105:0/2928090535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90713de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.780+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/2928090535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070e8f0 0x558c9070ed10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009f00 tx=0x7fe59c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7fe6ed9a2490710 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.780+0000 7fe5897fa640 1 -- 172.21.15.105:0/2928090535 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004d20 con 0x558c9070e8f0 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.781+0000 7fe58bfff640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe58c0a2fa0 0x7fe58c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.781+0000 7fe5897fa640 1 -- 172.21.15.105:0/2928090535 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe59c004030 con 0x558c9070e8f0 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.781+0000 7fe5897fa640 1 -- 172.21.15.105:0/2928090535 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c005260 con 0x558c9070e8f0 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.781+0000 7fe58bfff640 1 -- 172.21.15.105:0/960195269 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe5940f14c0 con 0x7fe5a4100800 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.781+0000 7fe5a2ffd640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe58c0a2fa0 0x7fe58c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/2928090535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070e8f0 msgr2=0x558c9070ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5a2ffd640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe58c0a2fa0 0x7fe58c0a5460 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe594006010 tx=0x7fe5940f4f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/2928090535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070e8f0 0x558c9070ed10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009f00 tx=0x7fe59c011810 comp rx=0 tx=0).stop 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/2928090535 shutdown_connections 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/2928090535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 0x558c907168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/2928090535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90713de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/2928090535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070e8f0 0x558c9070ed10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/2928090535 >> 172.21.15.105:0/2928090535 conn(0x558c90709e60 msgr2=0x558c9070a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/2928090535 shutdown_connections 2024-03-26T17:55:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/2928090535 wait complete. 2024-03-26T17:55:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.782+0000 7fe5ac3b3740 1 -- start start 2024-03-26T17:55:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5ac3b3740 1 Processor -- start 2024-03-26T17:55:33.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070e8f0 0x558c90729b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070f2e0 0x558c9072a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 0x558c90731100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070e8f0 0x558c90729b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070e8f0 0x558c90729b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53498/0 (socket says 172.21.15.105:53498) 2024-03-26T17:55:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5a37fe640 1 -- 172.21.15.105:0/3395752079 learned_addr learned my addr 172.21.15.105:0/3395752079 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070f2e0 0x558c9072a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558c90728410 con 0x558c90714450 2024-03-26T17:55:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558c9062f160 con 0x558c9070f2e0 2024-03-26T17:55:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558c90728590 con 0x558c9070e8f0 2024-03-26T17:55:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.783+0000 7fe5ac3b3740 10 client.0 _ll_register_callbacks cb 0x558c9066b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 0x558c90731100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe5a37fe640 1 -- 172.21.15.105:0/3395752079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070f2e0 msgr2=0x558c9072a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070f2e0 0x558c9072a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe5a37fe640 1 -- 172.21.15.105:0/3395752079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 msgr2=0x558c90731100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 0x558c90731100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe5a37fe640 1 -- 172.21.15.105:0/3395752079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c903873f0 con 0x558c9070e8f0 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070e8f0 0x558c90729b80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009370 tx=0x7fe59c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7448ccc6df660d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe586ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe586ff5640 1 -- 172.21.15.105:0/3395752079 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c02d060 con 0x558c9070e8f0 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe586ff5640 1 -- 172.21.15.105:0/3395752079 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe59c005040 con 0x558c9070e8f0 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.784+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c004030 con 0x558c9070e8f0 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe586ff5640 1 -- 172.21.15.105:0/3395752079 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe59c005240 con 0x558c9070e8f0 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5ac3b3740 10 client.31474 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558c903470f0 con 0x558c9070e8f0 2024-03-26T17:55:33.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe586ff5640 1 -- 172.21.15.105:0/3395752079 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe59c004030 con 0x558c9070e8f0 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe586ff5640 1 -- 172.21.15.105:0/3395752079 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe59c024070 con 0x558c9070e8f0 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5ac3b3740 10 client.31474 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5ac3b3740 10 client.31474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5ac3b3740 20 client.31474 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c907356d0 0x558c90737ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5ac3b3740 4 client.31474 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c90680ec0 con 0x558c907356d0 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.785+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c907356d0 0x558c90737ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.786+0000 7fe586ff5640 10 client.31474 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.786+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c907356d0 0x558c90737ae0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e223d895b63d8de server_cookie=cef57a21f078189b in_seq=0 out_seq=0 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.788+0000 7fe586ff5640 1 -- 172.21.15.105:0/3395752079 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558c90680ec0 con 0x558c907356d0 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.788+0000 7fe586ff5640 10 client.31474 handle_command_reply: tid=0 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.788+0000 7fe5ac3b3740 1 client.31474 shutdown 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.788+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070e8f0 msgr2=0x558c90729b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.788+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070e8f0 0x558c90729b80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009370 tx=0x7fe59c000f40 comp rx=0 tx=0).stop 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.788+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 shutdown_connections 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.788+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c907356d0 0x558c90737ae0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.789+0000 7fe586ff5640 0 client.31474 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.789+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c90714450 0x558c90731100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.789+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c9070f2e0 0x558c9072a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.789+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3395752079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070e8f0 0x558c90729b80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.789+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 >> 172.21.15.105:0/3395752079 conn(0x558c90709e60 msgr2=0x558c90709080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.789+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 shutdown_connections 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.789+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3395752079 wait complete. 2024-03-26T17:55:33.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.789+0000 7fe5ac3b3740 20 client.31474 trim_cache size 0 max 16384 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.814+0000 7fe5ac3b3740 1 Processor -- start 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.814+0000 7fe5ac3b3740 1 -- start start 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.814+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.814+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90731090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.814+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 0x558c907087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.814+0000 7fe5ac3b3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41a6c70 con 0x558c9070e8f0 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.814+0000 7fe5ac3b3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41a87b0 con 0x558c90714450 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.814+0000 7fe5ac3b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41a7340 con 0x558c9070f2e0 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37488/0 (socket says 172.21.15.105:37488) 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a27fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90731090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 -- 172.21.15.105:0/3704039154 learned_addr learned my addr 172.21.15.105:0/3704039154 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/3704039154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 0x558c907087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 -- 172.21.15.105:0/3704039154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 msgr2=0x558c90731090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/3704039154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90731090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 -- 172.21.15.105:0/3704039154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 msgr2=0x558c907087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/3704039154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 0x558c907087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 -- 172.21.15.105:0/3704039154 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c907137f0 con 0x558c9070e8f0 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/3704039154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072a020 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe59c011850 tx=0x7fe59c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61024b0a37117aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe588ff9640 1 -- 172.21.15.105:0/3704039154 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c02d040 con 0x558c9070e8f0 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe588ff9640 1 -- 172.21.15.105:0/3704039154 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe59c01f040 con 0x558c9070e8f0 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.815+0000 7fe588ff9640 1 -- 172.21.15.105:0/3704039154 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00f6e0 con 0x558c9070e8f0 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3704039154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 msgr2=0x558c9072a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3704039154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072a020 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe59c011850 tx=0x7fe59c014040 comp rx=0 tx=0).stop 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3704039154 shutdown_connections 2024-03-26T17:55:33.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3704039154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 0x558c907087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3704039154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c90731090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/3704039154 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072a020 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3704039154 >> 172.21.15.105:0/3704039154 conn(0x558c90709e60 msgr2=0x558c90713b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3704039154 shutdown_connections 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.816+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/3704039154 wait complete. 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 1 -- start start 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 1 Processor -- start 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c9072f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37492/0 (socket says 172.21.15.105:37492) 2024-03-26T17:55:33.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 0x558c90730320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a37fe640 1 -- 172.21.15.105:0/674147586 learned_addr learned my addr 172.21.15.105:0/674147586 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:33.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41a80e0 con 0x558c9070e8f0 2024-03-26T17:55:33.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c9072f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41a7a10 con 0x558c90714450 2024-03-26T17:55:33.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4071f60 con 0x558c9070f2e0 2024-03-26T17:55:33.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5ac3b3740 10 client.0 _ll_register_callbacks cb 0x7fe5940f4dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:33.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a27fc640 1 -- 172.21.15.105:0/674147586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 msgr2=0x558c90730320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 0x558c90730320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a27fc640 1 -- 172.21.15.105:0/674147586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 msgr2=0x558c9072ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a27fc640 1 -- 172.21.15.105:0/674147586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c90730910 con 0x558c9070f2e0 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.817+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe5a27fc640 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c9072f360 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x558c90738330 tx=0x7fe59801ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed10b01b10f47e1c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe586ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe586ff5640 1 -- 172.21.15.105:0/674147586 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598004250 con 0x558c9070f2e0 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c01f040 con 0x558c9070f2e0 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe586ff5640 1 -- 172.21.15.105:0/674147586 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fe5980043f0 con 0x558c9070f2e0 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe586ff5640 1 -- 172.21.15.105:0/674147586 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe59801f390 con 0x558c9070f2e0 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe5ac3b3740 10 client.31480 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558c9079c0a0 con 0x558c9070f2e0 2024-03-26T17:55:33.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe586ff5640 1 -- 172.21.15.105:0/674147586 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe59c01f040 con 0x558c9070f2e0 2024-03-26T17:55:33.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.818+0000 7fe586ff5640 1 -- 172.21.15.105:0/674147586 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fe598010430 con 0x558c9070f2e0 2024-03-26T17:55:33.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe5ac3b3740 10 client.31480 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:33.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe5ac3b3740 10 client.31480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:33.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe5ac3b3740 20 client.31480 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:33.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c907356d0 0x558c9079eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:33.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe5ac3b3740 4 client.31480 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:55:33.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c9067b210 con 0x558c907356d0 2024-03-26T17:55:33.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c907356d0 0x558c9079eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:33.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe5a37fe640 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c907356d0 0x558c9079eac0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333bf9187fce426f server_cookie=aa68d515675a54b4 in_seq=0 out_seq=0 2024-03-26T17:55:33.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.819+0000 7fe586ff5640 10 client.31480 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:33.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.821+0000 7fe586ff5640 1 -- 172.21.15.105:0/674147586 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x558c9067b210 con 0x558c907356d0 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.821+0000 7fe586ff5640 10 client.31480 handle_command_reply: tid=0 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.821+0000 7fe5ac3b3740 1 client.31480 shutdown 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 msgr2=0x558c9072f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c9072f360 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x558c90738330 tx=0x7fe59801ff70 comp rx=0 tx=0).stop 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 shutdown_connections 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558c907356d0 0x558c9079eac0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558c90714450 0x558c90730320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558c9070f2e0 0x558c9072f360 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 --2- 172.21.15.105:0/674147586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558c9070e8f0 0x558c9072ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 >> 172.21.15.105:0/674147586 conn(0x558c90709e60 msgr2=0x558c9070d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe586ff5640 0 client.31480 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 shutdown_connections 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 1 -- 172.21.15.105:0/674147586 wait complete. 2024-03-26T17:55:33.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.822+0000 7fe5ac3b3740 20 client.31480 trim_cache size 0 max 16384 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (233 inodes in the stack)", 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout: "1ae9201e-b1f5-40a6-b5b0-f2e0e43be53d": { 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "1ae9201e-b1f5-40a6-b5b0-f2e0e43be53d", 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:55:33.814 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:55:33.815 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:55:33.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 -- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe58c0a2fa0 msgr2=0x7fe58c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe58c0a2fa0 0x7fe58c0a5460 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe594006010 tx=0x7fe5940f4f00 comp rx=0 tx=0).stop 2024-03-26T17:55:33.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 -- 172.21.15.105:0/960195269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a4100800 msgr2=0x7fe5a410c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:33.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a4100800 0x7fe5a410c690 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe594000c00 tx=0x7fe594005d50 comp rx=0 tx=0).stop 2024-03-26T17:55:33.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 -- 172.21.15.105:0/960195269 shutdown_connections 2024-03-26T17:55:33.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe58c0a2fa0 0x7fe58c0a5460 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41023b0 0x7fe5a410b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5a41019f0 0x7fe5a410acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 --2- 172.21.15.105:0/960195269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a4100800 0x7fe5a410c690 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:33.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 -- 172.21.15.105:0/960195269 >> 172.21.15.105:0/960195269 conn(0x7fe5a40fbd70 msgr2=0x7fe5a4104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:33.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 -- 172.21.15.105:0/960195269 shutdown_connections 2024-03-26T17:55:33.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:55:33.823+0000 7fe5a97c6640 1 -- 172.21.15.105:0/960195269 wait complete. 2024-03-26T17:55:33.824 INFO:tasks.cephfs.filesystem:scrub status for tag:1ae9201e-b1f5-40a6-b5b0-f2e0e43be53d - {'path': '/', 'tag': '1ae9201e-b1f5-40a6-b5b0-f2e0e43be53d', 'options': 'recursive,force'} 2024-03-26T17:55:34.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:33 smithi105 ceph-mon[32280]: pgmap v865: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-03-26T17:55:34.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:33 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:55:34.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:33 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:34.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:33 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:34.137 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:33 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:34.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:33 smithi135 ceph-mon[35636]: pgmap v865: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-03-26T17:55:34.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:33 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:55:34.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:33 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:34.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:33 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:34.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:33 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:34.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:33 smithi144 ceph-mon[36095]: pgmap v865: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-03-26T17:55:34.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:33 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:55:34.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:33 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:34.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:33 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:34.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:33 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:34.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 33832 7.11 MB/sec execute 137 sec latency 182.739 ms 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:55:35.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.161 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:34 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:35.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:35.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:35.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:55:35.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.344 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:34 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:35.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:35.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:35.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:55:35.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:34 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:55:35.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 33978 7.10 MB/sec execute 138 sec latency 130.067 ms 2024-03-26T17:55:36.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:35 smithi135 ceph-mon[35636]: pgmap v866: 97 pgs: 97 active+clean; 276 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T17:55:36.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:35 smithi144 ceph-mon[36095]: pgmap v866: 97 pgs: 97 active+clean; 276 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T17:55:36.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:35 smithi105 ceph-mon[32280]: pgmap v866: 97 pgs: 97 active+clean; 276 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T17:55:36.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 34105 7.06 MB/sec execute 139 sec latency 129.120 ms 2024-03-26T17:55:37.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 34259 7.08 MB/sec execute 140 sec latency 168.717 ms 2024-03-26T17:55:38.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:37 smithi135 ceph-mon[35636]: pgmap v867: 97 pgs: 97 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 3.0 MiB/s wr, 305 op/s 2024-03-26T17:55:38.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:37 smithi144 ceph-mon[36095]: pgmap v867: 97 pgs: 97 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 3.0 MiB/s wr, 305 op/s 2024-03-26T17:55:38.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:37 smithi105 ceph-mon[32280]: pgmap v867: 97 pgs: 97 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 3.0 MiB/s wr, 305 op/s 2024-03-26T17:55:38.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 34396 7.04 MB/sec execute 141 sec latency 113.459 ms 2024-03-26T17:55:39.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:38 smithi144 ceph-mon[36095]: pgmap v868: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-03-26T17:55:39.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:38 smithi105 ceph-mon[32280]: pgmap v868: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-03-26T17:55:39.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:38 smithi135 ceph-mon[35636]: pgmap v868: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-03-26T17:55:39.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 34540 7.03 MB/sec execute 142 sec latency 114.444 ms 2024-03-26T17:55:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 -- 172.21.15.105:0/3950168852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c10100820 msgr2=0x7f1c10100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 --2- 172.21.15.105:0/3950168852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c10100820 0x7f1c10100c40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009900 tx=0x7f1c0c02bc50 comp rx=0 tx=0).stop 2024-03-26T17:55:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 -- 172.21.15.105:0/3950168852 shutdown_connections 2024-03-26T17:55:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 --2- 172.21.15.105:0/3950168852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c101023d0 0x7f1c10108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 --2- 172.21.15.105:0/3950168852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c10101a10 0x7f1c10101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 --2- 172.21.15.105:0/3950168852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c10100820 0x7f1c10100c40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 -- 172.21.15.105:0/3950168852 >> 172.21.15.105:0/3950168852 conn(0x7f1c100fbd90 msgr2=0x7f1c100fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:40.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 -- 172.21.15.105:0/3950168852 shutdown_connections 2024-03-26T17:55:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.059+0000 7f1c17c97640 1 -- 172.21.15.105:0/3950168852 wait complete. 2024-03-26T17:55:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c17c97640 1 Processor -- start 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c17c97640 1 -- start start 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c17c97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c10100820 0x7f1c101047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c17c97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c10101a10 0x7f1c10102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c17c97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c101023d0 0x7f1c10103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c17c97640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010c320 con 0x7f1c101023d0 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c17c97640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010c1a0 con 0x7f1c10100820 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c17c97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010c4a0 con 0x7f1c10101a10 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c16234640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c101023d0 0x7f1c10103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c16234640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c101023d0 0x7f1c10103330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42138/0 (socket says 172.21.15.105:42138) 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c15a33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c10100820 0x7f1c101047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c16234640 1 -- 172.21.15.105:0/1350839978 learned_addr learned my addr 172.21.15.105:0/1350839978 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.060+0000 7f1c15232640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c10101a10 0x7f1c10102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c15a33640 1 -- 172.21.15.105:0/1350839978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c10101a10 msgr2=0x7f1c10102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c15a33640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c10101a10 0x7f1c10102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c15a33640 1 -- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c101023d0 msgr2=0x7f1c10103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c15a33640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c101023d0 0x7f1c10103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c15a33640 1 -- 172.21.15.105:0/1350839978 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c10103870 con 0x7f1c10100820 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c16234640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c101023d0 0x7f1c10103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c15a33640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c10100820 0x7f1c101047b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c036040 tx=0x7f1c0c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c06ffd640 1 -- 172.21.15.105:0/1350839978 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c043020 con 0x7f1c10100820 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c06ffd640 1 -- 172.21.15.105:0/1350839978 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c0c004350 con 0x7f1c10100820 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c06ffd640 1 -- 172.21.15.105:0/1350839978 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c003ab0 con 0x7f1c10100820 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c17c97640 1 -- 172.21.15.105:0/1350839978 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c10103b00 con 0x7f1c10100820 2024-03-26T17:55:40.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.061+0000 7f1c17c97640 1 -- 172.21.15.105:0/1350839978 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c101a6520 con 0x7f1c10100820 2024-03-26T17:55:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.072+0000 7f1c06ffd640 1 -- 172.21.15.105:0/1350839978 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1c0c046020 con 0x7f1c10100820 2024-03-26T17:55:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.073+0000 7f1c06ffd640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1bfc0a3130 0x7f1bfc0a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.073+0000 7f1c15232640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1bfc0a3130 0x7f1bfc0a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.074+0000 7f1c06ffd640 1 -- 172.21.15.105:0/1350839978 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f1c0c0f1470 con 0x7f1c10100820 2024-03-26T17:55:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.074+0000 7f1c15232640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1bfc0a3130 0x7f1bfc0a55f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1c10104520 tx=0x7f1c00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.127+0000 7f1c1a522740 1 Processor -- start 2024-03-26T17:55:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.127+0000 7f1c1a522740 1 -- start start 2024-03-26T17:55:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.127+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b70125d0 0x55a7b70129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.127+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b7017ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7018130 0x55a7b701a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c1a522740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a7b6f770b0 con 0x55a7b7018130 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c1a522740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a7b6f1f960 con 0x55a7b70125d0 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c1a522740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7b6f73060 con 0x55a7b7012fc0 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c16234640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7018130 0x55a7b701a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15232640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b7017ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15a33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b70125d0 0x55a7b70129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c16234640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7018130 0x55a7b701a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42146/0 (socket says 172.21.15.105:42146) 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15232640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b7017ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51746/0 (socket says 172.21.15.105:51746) 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c16234640 1 -- 172.21.15.105:0/3192588576 learned_addr learned my addr 172.21.15.105:0/3192588576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15232640 1 -- 172.21.15.105:0/3192588576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b70125d0 msgr2=0x55a7b70129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15232640 1 --2- 172.21.15.105:0/3192588576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b70125d0 0x55a7b70129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15232640 1 -- 172.21.15.105:0/3192588576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7018130 msgr2=0x55a7b701a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15232640 1 --2- 172.21.15.105:0/3192588576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7018130 0x55a7b701a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15232640 1 -- 172.21.15.105:0/3192588576 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b6f31c60 con 0x55a7b7012fc0 2024-03-26T17:55:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15a33640 1 --2- 172.21.15.105:0/3192588576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b70125d0 0x55a7b70129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.128+0000 7f1c15232640 1 --2- 172.21.15.105:0/3192588576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b7017ac0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1c000118d0 tx=0x7f1c00014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64608a2d6a9ea736 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.129+0000 7f1bfbfff640 1 -- 172.21.15.105:0/3192588576 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00004030 con 0x55a7b7012fc0 2024-03-26T17:55:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.129+0000 7f1bfbfff640 1 -- 172.21.15.105:0/3192588576 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c000041d0 con 0x55a7b7012fc0 2024-03-26T17:55:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.129+0000 7f1bfbfff640 1 -- 172.21.15.105:0/3192588576 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000266d0 con 0x55a7b7012fc0 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 -- 172.21.15.105:0/3192588576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 msgr2=0x55a7b7017ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3192588576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b7017ac0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1c000118d0 tx=0x7f1c00014000 comp rx=0 tx=0).stop 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 -- 172.21.15.105:0/3192588576 shutdown_connections 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3192588576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7018130 0x55a7b701a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3192588576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b7017ac0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3192588576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b70125d0 0x55a7b70129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 -- 172.21.15.105:0/3192588576 >> 172.21.15.105:0/3192588576 conn(0x55a7b700db40 msgr2=0x55a7b700df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 -- 172.21.15.105:0/3192588576 shutdown_connections 2024-03-26T17:55:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.130+0000 7f1c1a522740 1 -- 172.21.15.105:0/3192588576 wait complete. 2024-03-26T17:55:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.138+0000 7f1c1a522740 1 -- start start 2024-03-26T17:55:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c1a522740 1 Processor -- start 2024-03-26T17:55:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b701c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7012fc0 0x55a7b701ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7018130 0x55a7b70239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c1a522740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a7b701b0e0 con 0x55a7b70125d0 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c1a522740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a7b6f32db0 con 0x55a7b7012fc0 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c1a522740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7b701b260 con 0x55a7b7018130 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c15a33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b701c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c15a33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b701c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42150/0 (socket says 172.21.15.105:42150) 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c15a33640 1 -- 172.21.15.105:0/581071482 learned_addr learned my addr 172.21.15.105:0/581071482 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.139+0000 7f1c15232640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7012fc0 0x55a7b701ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c16234640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7018130 0x55a7b70239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c1a522740 10 client.0 _ll_register_callbacks cb 0x55a7b6f37110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c15a33640 1 -- 172.21.15.105:0/581071482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7018130 msgr2=0x55a7b70239e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c15a33640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7018130 0x55a7b70239e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c15a33640 1 -- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7012fc0 msgr2=0x55a7b701ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c15a33640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7012fc0 0x55a7b701ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c15a33640 1 -- 172.21.15.105:0/581071482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b6c8b210 con 0x55a7b70125d0 2024-03-26T17:55:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c16234640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7018130 0x55a7b70239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c15232640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7012fc0 0x55a7b701ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c15a33640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b701c540 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0bda90 tx=0x7f1c0c0be340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c572798527498a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1c1a522740 1 -- 172.21.15.105:0/581071482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c000041d0 con 0x55a7b70125d0 2024-03-26T17:55:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1bf97fa640 1 -- 172.21.15.105:0/581071482 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0db060 con 0x55a7b70125d0 2024-03-26T17:55:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.140+0000 7f1bf97fa640 1 -- 172.21.15.105:0/581071482 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f1c0c005420 con 0x55a7b70125d0 2024-03-26T17:55:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1bf97fa640 1 -- 172.21.15.105:0/581071482 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c0d6030 con 0x55a7b70125d0 2024-03-26T17:55:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1c1a522740 10 client.21777 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1c1a522740 1 -- 172.21.15.105:0/581071482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55a7b6d57620 con 0x55a7b70125d0 2024-03-26T17:55:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1bf97fa640 1 -- 172.21.15.105:0/581071482 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1c0c0c6080 con 0x55a7b70125d0 2024-03-26T17:55:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1c1a522740 10 client.21777 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1c1a522740 10 client.21777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1c1a522740 20 client.21777 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1c1a522740 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a7b7027f00 0x55a7b702a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1c1a522740 4 client.21777 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.150+0000 7f1c1a522740 1 -- 172.21.15.105:0/581071482 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7b6f845f0 con 0x55a7b7027f00 2024-03-26T17:55:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.151+0000 7f1c15232640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a7b7027f00 0x55a7b702a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.151+0000 7f1bf97fa640 10 client.21777 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.151+0000 7f1c15232640 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a7b7027f00 0x55a7b702a350 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262fe66d0cc30cfa server_cookie=a43cd314999b3715 in_seq=0 out_seq=0 2024-03-26T17:55:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1bf97fa640 1 -- 172.21.15.105:0/581071482 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a7b6f845f0 con 0x55a7b7027f00 2024-03-26T17:55:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1bf97fa640 10 client.21777 handle_command_reply: tid=0 2024-03-26T17:55:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 client.21777 shutdown 2024-03-26T17:55:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 -- 172.21.15.105:0/581071482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 msgr2=0x55a7b701c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b701c540 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0bda90 tx=0x7f1c0c0be340 comp rx=0 tx=0).stop 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 -- 172.21.15.105:0/581071482 shutdown_connections 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a7b7027f00 0x55a7b702a350 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7018130 0x55a7b70239e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7012fc0 0x55a7b701ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 --2- 172.21.15.105:0/581071482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b701c540 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 -- 172.21.15.105:0/581071482 >> 172.21.15.105:0/581071482 conn(0x55a7b700db40 msgr2=0x55a7b700cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1bf97fa640 0 client.21777 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 -- 172.21.15.105:0/581071482 shutdown_connections 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.154+0000 7f1c1a522740 1 -- 172.21.15.105:0/581071482 wait complete. 2024-03-26T17:55:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.155+0000 7f1c1a522740 20 client.21777 trim_cache size 0 max 16384 2024-03-26T17:55:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c1a522740 1 Processor -- start 2024-03-26T17:55:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c1a522740 1 -- start start 2024-03-26T17:55:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b7014170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b70239a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b701bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c1a522740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101af2f0 con 0x55a7b70125d0 2024-03-26T17:55:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c1a522740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101b0e30 con 0x55a7b7018130 2024-03-26T17:55:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c1a522740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101af9c0 con 0x55a7b7012fc0 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15a33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b7014170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15a33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b7014170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42166/0 (socket says 172.21.15.105:42166) 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15a33640 1 -- 172.21.15.105:0/3429378580 learned_addr learned my addr 172.21.15.105:0/3429378580 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15232640 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b70239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c16234640 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b701bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15a33640 1 -- 172.21.15.105:0/3429378580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 msgr2=0x55a7b70239a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15a33640 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b70239a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15a33640 1 -- 172.21.15.105:0/3429378580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 msgr2=0x55a7b701bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15a33640 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b701bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.180+0000 7f1c15a33640 1 -- 172.21.15.105:0/3429378580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b701b600 con 0x55a7b70125d0 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.181+0000 7f1c15a33640 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b7014170 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0bd800 tx=0x7f1c0c0c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96615391461bcf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.181+0000 7f1bfb7fe640 1 -- 172.21.15.105:0/3429378580 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0c6560 con 0x55a7b70125d0 2024-03-26T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.181+0000 7f1bfb7fe640 1 -- 172.21.15.105:0/3429378580 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c000041d0 con 0x55a7b70125d0 2024-03-26T17:55:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.181+0000 7f1bfb7fe640 1 -- 172.21.15.105:0/3429378580 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0b9850 con 0x55a7b70125d0 2024-03-26T17:55:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 -- 172.21.15.105:0/3429378580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 msgr2=0x55a7b7014170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b7014170 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0bd800 tx=0x7f1c0c0c0000 comp rx=0 tx=0).stop 2024-03-26T17:55:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 -- 172.21.15.105:0/3429378580 shutdown_connections 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b701bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b7012fc0 0x55a7b70239a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3429378580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b70125d0 0x55a7b7014170 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 -- 172.21.15.105:0/3429378580 >> 172.21.15.105:0/3429378580 conn(0x55a7b700db40 msgr2=0x55a7b7013a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 -- 172.21.15.105:0/3429378580 shutdown_connections 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 -- 172.21.15.105:0/3429378580 wait complete. 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.182+0000 7f1c1a522740 1 -- start start 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c1a522740 1 Processor -- start 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b70125d0 0x55a7b7020dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7012fc0 0x55a7b7021310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c1a522740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b70222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c1a522740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101b0760 con 0x55a7b7012fc0 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c1a522740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101b0090 con 0x55a7b7018130 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c1a522740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c101ad2d0 con 0x55a7b70125d0 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c1a522740 10 client.0 _ll_register_callbacks cb 0x7f1c0c0bd640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c15232640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7012fc0 0x55a7b7021310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c15232640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7012fc0 0x55a7b7021310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42176/0 (socket says 172.21.15.105:42176) 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c15232640 1 -- 172.21.15.105:0/3870585164 learned_addr learned my addr 172.21.15.105:0/3870585164 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c15a33640 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b70125d0 0x55a7b7020dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.183+0000 7f1c16234640 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b70222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1c15232640 1 -- 172.21.15.105:0/3870585164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b70125d0 msgr2=0x55a7b7020dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1c15232640 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b70125d0 0x55a7b7020dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1c15232640 1 -- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 msgr2=0x55a7b70222d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1c15232640 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b70222d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1c15232640 1 -- 172.21.15.105:0/3870585164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b70228c0 con 0x55a7b7012fc0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1c16234640 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b70222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1c15232640 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7012fc0 0x55a7b7021310 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f1c00000c00 tx=0x7f1c00005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12323ca4778e095e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1bf97fa640 1 -- 172.21.15.105:0/3870585164 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00004380 con 0x55a7b7012fc0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1bf97fa640 1 -- 172.21.15.105:0/3870585164 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f1c00003ec0 con 0x55a7b7012fc0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.184+0000 7f1c1a522740 1 -- 172.21.15.105:0/3870585164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c000041d0 con 0x55a7b7012fc0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1bf97fa640 1 -- 172.21.15.105:0/3870585164 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1c000036f0 con 0x55a7b7012fc0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1c1a522740 10 client.21783 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1c1a522740 1 -- 172.21.15.105:0/3870585164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a7b7023c70 con 0x55a7b7012fc0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1bf97fa640 1 -- 172.21.15.105:0/3870585164 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c000041d0 con 0x55a7b7012fc0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1bf97fa640 1 -- 172.21.15.105:0/3870585164 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f1c00034490 con 0x55a7b7012fc0 2024-03-26T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1c1a522740 10 client.21783 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1c1a522740 10 client.21783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1c1a522740 20 client.21783 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a7b708e620 0x55a7b7090a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1c1a522740 4 client.21783 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.185+0000 7f1c1a522740 1 -- 172.21.15.105:0/3870585164 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7b6f7e940 con 0x55a7b708e620 2024-03-26T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.186+0000 7f1c15a33640 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a7b708e620 0x55a7b7090a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.186+0000 7f1bf97fa640 10 client.21783 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.186+0000 7f1c15a33640 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a7b708e620 0x55a7b7090a40 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce55d6ef91f9e09 server_cookie=a2fa50d033b8807d in_seq=0 out_seq=0 2024-03-26T17:55:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1bf97fa640 1 -- 172.21.15.105:0/3870585164 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7b6f7e940 con 0x55a7b708e620 2024-03-26T17:55:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1bf97fa640 10 client.21783 handle_command_reply: tid=0 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 client.21783 shutdown 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 -- 172.21.15.105:0/3870585164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7012fc0 msgr2=0x55a7b7021310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7012fc0 0x55a7b7021310 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f1c00000c00 tx=0x7f1c00005650 comp rx=0 tx=0).stop 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 -- 172.21.15.105:0/3870585164 shutdown_connections 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a7b708e620 0x55a7b7090a40 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1bf97fa640 0 client.21783 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a7b7018130 0x55a7b70222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a7b7012fc0 0x55a7b7021310 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 --2- 172.21.15.105:0/3870585164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a7b70125d0 0x55a7b7020dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 -- 172.21.15.105:0/3870585164 >> 172.21.15.105:0/3870585164 conn(0x55a7b700db40 msgr2=0x55a7b70142f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 -- 172.21.15.105:0/3870585164 shutdown_connections 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.325+0000 7f1c1a522740 1 -- 172.21.15.105:0/3870585164 wait complete. 2024-03-26T17:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c1a522740 20 client.21783 trim_cache size 0 max 16384 2024-03-26T17:55:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:55:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:55:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:55:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 -- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1bfc0a3130 msgr2=0x7f1bfc0a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1bfc0a3130 0x7f1bfc0a55f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1c10104520 tx=0x7f1c00005f90 comp rx=0 tx=0).stop 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 -- 172.21.15.105:0/1350839978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c10100820 msgr2=0x7f1c101047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c10100820 0x7f1c101047b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c036040 tx=0x7f1c0c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 -- 172.21.15.105:0/1350839978 shutdown_connections 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1bfc0a3130 0x7f1bfc0a55f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c101023d0 0x7f1c10103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c10101a10 0x7f1c10102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 --2- 172.21.15.105:0/1350839978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c10100820 0x7f1c101047b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.326+0000 7f1c17c97640 1 -- 172.21.15.105:0/1350839978 >> 172.21.15.105:0/1350839978 conn(0x7f1c100fbd90 msgr2=0x7f1c100fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.327+0000 7f1c17c97640 1 -- 172.21.15.105:0/1350839978 shutdown_connections 2024-03-26T17:55:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:40.327+0000 7f1c17c97640 1 -- 172.21.15.105:0/1350839978 wait complete. 2024-03-26T17:55:40.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 34667 7.00 MB/sec execute 143 sec latency 173.546 ms 2024-03-26T17:55:41.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 34804 6.99 MB/sec execute 144 sec latency 109.372 ms 2024-03-26T17:55:42.136 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:41 smithi105 ceph-mon[32280]: pgmap v869: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-03-26T17:55:42.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:41 smithi135 ceph-mon[35636]: pgmap v869: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-03-26T17:55:42.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:41 smithi144 ceph-mon[36095]: pgmap v869: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-03-26T17:55:42.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 34940 6.99 MB/sec execute 145 sec latency 121.050 ms 2024-03-26T17:55:43.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 35092 7.01 MB/sec execute 146 sec latency 105.200 ms 2024-03-26T17:55:44.136 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:43 smithi105 ceph-mon[32280]: pgmap v870: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 2.9 MiB/s wr, 298 op/s 2024-03-26T17:55:44.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:43 smithi135 ceph-mon[35636]: pgmap v870: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 2.9 MiB/s wr, 298 op/s 2024-03-26T17:55:44.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:43 smithi144 ceph-mon[36095]: pgmap v870: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 2.9 MiB/s wr, 298 op/s 2024-03-26T17:55:44.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 35263 7.04 MB/sec execute 147 sec latency 119.381 ms 2024-03-26T17:55:45.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 35472 7.12 MB/sec execute 148 sec latency 137.320 ms 2024-03-26T17:55:46.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:45 smithi135 ceph-mon[35636]: pgmap v871: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-03-26T17:55:46.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:45 smithi144 ceph-mon[36095]: pgmap v871: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-03-26T17:55:46.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:45 smithi105 ceph-mon[32280]: pgmap v871: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-03-26T17:55:46.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 35664 7.16 MB/sec execute 149 sec latency 125.582 ms 2024-03-26T17:55:47.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:46 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:55:47.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:46 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:55:47.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:46 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:55:47.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 35832 7.20 MB/sec execute 150 sec latency 121.844 ms 2024-03-26T17:55:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:47 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:55:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:47 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:55:48.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:47 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:55:48.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:47 smithi135 ceph-mon[35636]: pgmap v872: 97 pgs: 97 active+clean; 277 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-03-26T17:55:48.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:47 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:55:48.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:47 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:55:48.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:47 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:55:48.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:47 smithi144 ceph-mon[36095]: pgmap v872: 97 pgs: 97 active+clean; 277 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-03-26T17:55:48.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:47 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:55:48.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:47 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:55:48.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:47 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:55:48.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:47 smithi105 ceph-mon[32280]: pgmap v872: 97 pgs: 97 active+clean; 277 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 2.8 MiB/s wr, 281 op/s 2024-03-26T17:55:48.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 35904 7.18 MB/sec execute 151 sec latency 117.290 ms 2024-03-26T17:55:49.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36025 7.21 MB/sec execute 152 sec latency 118.932 ms 2024-03-26T17:55:50.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:49 smithi135 ceph-mon[35636]: pgmap v873: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-03-26T17:55:50.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:49 smithi144 ceph-mon[36095]: pgmap v873: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-03-26T17:55:50.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:49 smithi105 ceph-mon[32280]: pgmap v873: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-03-26T17:55:50.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36107 7.19 MB/sec execute 153 sec latency 116.772 ms 2024-03-26T17:55:51.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36217 7.18 MB/sec execute 154 sec latency 148.070 ms 2024-03-26T17:55:52.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:51 smithi135 ceph-mon[35636]: pgmap v874: 97 pgs: 97 active+clean; 290 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 3.3 MiB/s wr, 198 op/s 2024-03-26T17:55:52.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:51 smithi144 ceph-mon[36095]: pgmap v874: 97 pgs: 97 active+clean; 290 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 3.3 MiB/s wr, 198 op/s 2024-03-26T17:55:52.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:51 smithi105 ceph-mon[32280]: pgmap v874: 97 pgs: 97 active+clean; 290 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 3.3 MiB/s wr, 198 op/s 2024-03-26T17:55:52.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36348 7.22 MB/sec execute 155 sec latency 139.568 ms 2024-03-26T17:55:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/138831693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 msgr2=0x7fa9b0109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/138831693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 0x7fa9b0109d50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009be0 tx=0x7fa9a802bef0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/138831693 shutdown_connections 2024-03-26T17:55:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/138831693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 0x7fa9b0109d50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/138831693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0102b00 0x7fa9b0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/138831693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b0101910 0x7fa9b0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/138831693 >> 172.21.15.105:0/138831693 conn(0x7fa9b00fce80 msgr2=0x7fa9b00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/138831693 shutdown_connections 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.542+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/138831693 wait complete. 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b7e0b640 1 Processor -- start 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b7e0b640 1 -- start start 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b7e0b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0101910 0x7fa9b019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b7e0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b0102b00 0x7fa9b019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b7e0b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 0x7fa9b01a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b7e0b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b010d420 con 0x7fa9b0101910 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b7e0b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b010d2a0 con 0x7fa9b01034c0 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b7e0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b010d5a0 con 0x7fa9b0102b00 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0101910 0x7fa9b019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b53a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b0102b00 0x7fa9b019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b53a6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b0102b00 0x7fa9b019e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35182/0 (socket says 172.21.15.105:35182) 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0101910 0x7fa9b019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52142/0 (socket says 172.21.15.105:52142) 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b53a6640 1 -- 172.21.15.105:0/782847164 learned_addr learned my addr 172.21.15.105:0/782847164 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 0x7fa9b01a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/782847164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b0102b00 msgr2=0x7fa9b019e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.543+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b0102b00 0x7fa9b019e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/782847164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 msgr2=0x7fa9b01a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 0x7fa9b01a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/782847164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b01a5cb0 con 0x7fa9b0101910 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 0x7fa9b01a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0101910 0x7fa9b019e090 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00e9c0 tx=0x7fa9ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9a6ffd640 1 -- 172.21.15.105:0/782847164 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00ccc0 con 0x7fa9b0101910 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/782847164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b01a5fa0 con 0x7fa9b0101910 2024-03-26T17:55:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9a6ffd640 1 -- 172.21.15.105:0/782847164 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9ac0035f0 con 0x7fa9b0101910 2024-03-26T17:55:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/782847164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01a6460 con 0x7fa9b0101910 2024-03-26T17:55:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.544+0000 7fa9a6ffd640 1 -- 172.21.15.105:0/782847164 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00ce60 con 0x7fa9b0101910 2024-03-26T17:55:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.547+0000 7fa9a6ffd640 1 -- 172.21.15.105:0/782847164 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa9ac0037a0 con 0x7fa9b0101910 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.547+0000 7fa9ba9f8740 1 Processor -- start 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9ba9f8740 1 -- start start 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c17649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c1764fc0 0x5628c1769ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 0x5628c176c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5628c16c90b0 con 0x5628c1764fc0 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5628c1671960 con 0x5628c176a130 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628c16c5060 con 0x5628c17645d0 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9b63a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c17649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c1764fc0 0x5628c1769ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9a6ffd640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa99c0a2fc0 0x7fa99c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c1764fc0 0x5628c1769ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52144/0 (socket says 172.21.15.105:52144) 2024-03-26T17:55:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/3040297052 learned_addr learned my addr 172.21.15.105:0/3040297052 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/3040297052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 0x5628c176c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa99c0a2fc0 0x7fa99c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.548+0000 7fa9a6ffd640 1 -- 172.21.15.105:0/782847164 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa9ac0cf820 con 0x7fa9b0101910 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa9b63a8640 1 -- 172.21.15.105:0/3040297052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 msgr2=0x5628c176c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/3040297052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 0x5628c176c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa9b63a8640 1 -- 172.21.15.105:0/3040297052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c1764fc0 msgr2=0x5628c1769ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/3040297052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c1764fc0 0x5628c1769ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa9b63a8640 1 -- 172.21.15.105:0/3040297052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c1683c60 con 0x5628c17645d0 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/3040297052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c17649f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8034fd0 tx=0x7fa9a8033d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bbe664cd3655425 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa99c0a2fc0 0x7fa99c0a5480 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8005cc0 tx=0x7fa9a804a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa99bfff640 1 -- 172.21.15.105:0/3040297052 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a802cd20 con 0x5628c17645d0 2024-03-26T17:55:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa99bfff640 1 -- 172.21.15.105:0/3040297052 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a80044f0 con 0x5628c17645d0 2024-03-26T17:55:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.549+0000 7fa99bfff640 1 -- 172.21.15.105:0/3040297052 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8034900 con 0x5628c17645d0 2024-03-26T17:55:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.550+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/3040297052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 msgr2=0x5628c17649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.550+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/3040297052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c17649f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8034fd0 tx=0x7fa9a8033d40 comp rx=0 tx=0).stop 2024-03-26T17:55:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.550+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/3040297052 shutdown_connections 2024-03-26T17:55:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.550+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/3040297052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 0x5628c176c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.550+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/3040297052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c1764fc0 0x5628c1769ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.550+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/3040297052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c17649f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.550+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/3040297052 >> 172.21.15.105:0/3040297052 conn(0x5628c175fb40 msgr2=0x5628c175ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.551+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/3040297052 shutdown_connections 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.551+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/3040297052 wait complete. 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.551+0000 7fa9ba9f8740 1 -- start start 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.551+0000 7fa9ba9f8740 1 Processor -- start 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c176e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 0x5628c176e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c17758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5628c176d020 con 0x5628c176a130 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5628c1684db0 con 0x5628c1764fc0 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5628c176d1a0 con 0x5628c17645d0 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9ba9f8740 10 client.0 _ll_register_callbacks cb 0x5628c1689110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b63a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c17758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c176e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b53a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 0x5628c176e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c176e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35198/0 (socket says 172.21.15.105:35198) 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b63a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c17758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52158/0 (socket says 172.21.15.105:52158) 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/2075747346 learned_addr learned my addr 172.21.15.105:0/2075747346 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/2075747346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 msgr2=0x5628c176e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 0x5628c176e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/2075747346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 msgr2=0x5628c17758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c17758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/2075747346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c13dd210 con 0x5628c17645d0 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c17758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c176e320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac012020 tx=0x7fa9ac09bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b64d66865099af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9997fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.552+0000 7fa9997fa640 1 -- 172.21.15.105:0/2075747346 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac09bf10 con 0x5628c17645d0 2024-03-26T17:55:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9997fa640 1 -- 172.21.15.105:0/2075747346 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa9ac09a210 con 0x5628c17645d0 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/2075747346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9a80044f0 con 0x5628c17645d0 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9997fa640 1 -- 172.21.15.105:0/2075747346 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa9ac0a0060 con 0x5628c17645d0 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 10 client.31510 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/2075747346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5628c14a9620 con 0x5628c17645d0 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9997fa640 1 -- 172.21.15.105:0/2075747346 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a80044f0 con 0x5628c17645d0 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9997fa640 1 -- 172.21.15.105:0/2075747346 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa9ac09add0 con 0x5628c17645d0 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 10 client.31510 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 10 client.31510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 20 client.31510 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628c177a1d0 0x5628c177c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 4 client.31510 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.553+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/2075747346 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628c16d65f0 con 0x5628c177a1d0 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.554+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628c177a1d0 0x5628c177c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.554+0000 7fa9997fa640 10 client.31510 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.554+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628c177a1d0 0x5628c177c640 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e126030b88ca79 server_cookie=172e8e4786441099 in_seq=0 out_seq=0 2024-03-26T17:55:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9997fa640 1 -- 172.21.15.105:0/2075747346 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5628c16d65f0 con 0x5628c177a1d0 2024-03-26T17:55:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9997fa640 10 client.31510 handle_command_reply: tid=0 2024-03-26T17:55:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 client.31510 shutdown 2024-03-26T17:55:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/2075747346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 msgr2=0x5628c176e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c176e320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac012020 tx=0x7fa9ac09bac0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/2075747346 shutdown_connections 2024-03-26T17:55:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628c177a1d0 0x5628c177c640 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c17758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 0x5628c176e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/2075747346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c176e320 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/2075747346 >> 172.21.15.105:0/2075747346 conn(0x5628c175fb40 msgr2=0x5628c175edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9997fa640 0 client.31510 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/2075747346 shutdown_connections 2024-03-26T17:55:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/2075747346 wait complete. 2024-03-26T17:55:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.556+0000 7fa9ba9f8740 20 client.31510 trim_cache size 0 max 16384 2024-03-26T17:55:53.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36478 7.23 MB/sec execute 156 sec latency 127.186 ms 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9ba9f8740 1 Processor -- start 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9ba9f8740 1 -- start start 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c17645d0 0x5628c17693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c1764fc0 0x5628c1769910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 0x5628c176df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa99400e440 con 0x5628c17645d0 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa99400ff80 con 0x5628c176a130 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa99400eb10 con 0x5628c1764fc0 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c17645d0 0x5628c17693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c17645d0 0x5628c17693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52172/0 (socket says 172.21.15.105:52172) 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.575+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/1927277978 learned_addr learned my addr 172.21.15.105:0/1927277978 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c1764fc0 0x5628c1769910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 0x5628c176df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b53a6640 1 -- 172.21.15.105:0/1927277978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 msgr2=0x5628c176df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 0x5628c176df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b53a6640 1 -- 172.21.15.105:0/1927277978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c17645d0 msgr2=0x5628c17693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c17645d0 0x5628c17693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b53a6640 1 -- 172.21.15.105:0/1927277978 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c1769e50 con 0x5628c1764fc0 2024-03-26T17:55:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c17645d0 0x5628c17693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c1764fc0 0x5628c1769910 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x5628c177cec0 tx=0x7fa9a0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fbf7ef12b52b5b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa99b7fe640 1 -- 172.21.15.105:0/1927277978 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0004020 con 0x5628c1764fc0 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa99b7fe640 1 -- 172.21.15.105:0/1927277978 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a0024030 con 0x5628c1764fc0 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.576+0000 7fa99b7fe640 1 -- 172.21.15.105:0/1927277978 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0003850 con 0x5628c1764fc0 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.577+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/1927277978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c1764fc0 msgr2=0x5628c1769910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.577+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c1764fc0 0x5628c1769910 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x5628c177cec0 tx=0x7fa9a0009260 comp rx=0 tx=0).stop 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.577+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/1927277978 shutdown_connections 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.577+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c176a130 0x5628c176df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.577+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c1764fc0 0x5628c1769910 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.577+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/1927277978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c17645d0 0x5628c17693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.577+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/1927277978 >> 172.21.15.105:0/1927277978 conn(0x5628c175fb40 msgr2=0x5628c177bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.578+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/1927277978 shutdown_connections 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.578+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/1927277978 wait complete. 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.578+0000 7fa9ba9f8740 1 -- start start 2024-03-26T17:55:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.578+0000 7fa9ba9f8740 1 Processor -- start 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c1775520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 0x5628c1772030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9ba9f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c1772660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa99400f8b0 con 0x5628c176a130 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa99400f1e0 con 0x5628c1764fc0 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9ba9f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa99400c420 con 0x5628c17645d0 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c1775520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c1775520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35210/0 (socket says 172.21.15.105:35210) 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/556769911 learned_addr learned my addr 172.21.15.105:0/556769911 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b63a8640 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c1772660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 0x5628c1772030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/556769911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 msgr2=0x5628c1772030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 0x5628c1772030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/556769911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 msgr2=0x5628c1772660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:55:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c1772660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 -- 172.21.15.105:0/556769911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c17de5f0 con 0x5628c17645d0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9b5ba7640 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c1775520 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00c6a0 tx=0x7fa9ac09c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73feb1fb5cb4e46d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9997fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9ba9f8740 10 client.0 _ll_register_callbacks cb 0x7fa9ac097af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/556769911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9a0024030 con 0x5628c17645d0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9997fa640 1 -- 172.21.15.105:0/556769911 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac09bce0 con 0x5628c17645d0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.579+0000 7fa9997fa640 1 -- 172.21.15.105:0/556769911 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa9ac09b360 con 0x5628c17645d0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9997fa640 1 -- 172.21.15.105:0/556769911 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa9ac09b870 con 0x5628c17645d0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9ba9f8740 10 client.31516 fetch_fsmap learned FSMap version 37 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/556769911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5628c17df870 con 0x5628c17645d0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9997fa640 1 -- 172.21.15.105:0/556769911 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa9ac097c60 con 0x5628c17645d0 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9ba9f8740 10 client.31516 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9ba9f8740 10 client.31516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:55:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9ba9f8740 20 client.31516 populate_metadata read hostname 'smithi105' 2024-03-26T17:55:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628c17e0100 0x5628c17e2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:55:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9ba9f8740 4 client.31516 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:55:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/556769911 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628c16d0940 con 0x5628c17e0100 2024-03-26T17:55:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.580+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628c17e0100 0x5628c17e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:55:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.581+0000 7fa9997fa640 10 client.31516 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.581+0000 7fa9b53a6640 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628c17e0100 0x5628c17e2520 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7121674ccb86414 server_cookie=750b29980b848e3d in_seq=0 out_seq=0 2024-03-26T17:55:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9997fa640 1 -- 172.21.15.105:0/556769911 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628c16d0940 con 0x5628c17e0100 2024-03-26T17:55:53.701 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9997fa640 10 client.31516 handle_command_reply: tid=0 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 client.31516 shutdown 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/556769911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 msgr2=0x5628c1775520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c1775520 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00c6a0 tx=0x7fa9ac09c6d0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/556769911 shutdown_connections 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5628c17e0100 0x5628c17e2520 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5628c176a130 0x5628c1772660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5628c1764fc0 0x5628c1772030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9997fa640 0 client.31516 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 --2- 172.21.15.105:0/556769911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5628c17645d0 0x5628c1775520 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.713+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/556769911 >> 172.21.15.105:0/556769911 conn(0x5628c175fb40 msgr2=0x5628c177bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.714+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/556769911 shutdown_connections 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.714+0000 7fa9ba9f8740 1 -- 172.21.15.105:0/556769911 wait complete. 2024-03-26T17:55:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.714+0000 7fa9ba9f8740 20 client.31516 trim_cache size 0 max 16384 2024-03-26T17:55:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:55:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:55:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:55:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.714+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa99c0a2fc0 msgr2=0x7fa99c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.714+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa99c0a2fc0 0x7fa99c0a5480 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8005cc0 tx=0x7fa9a804a000 comp rx=0 tx=0).stop 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.714+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0101910 msgr2=0x7fa9b019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.714+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0101910 0x7fa9b019e090 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00e9c0 tx=0x7fa9ac00ee80 comp rx=0 tx=0).stop 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.715+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/782847164 shutdown_connections 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.715+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa99c0a2fc0 0x7fa99c0a5480 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.715+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01034c0 0x7fa9b01a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.715+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b0102b00 0x7fa9b019e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.715+0000 7fa9b7e0b640 1 --2- 172.21.15.105:0/782847164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0101910 0x7fa9b019e090 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:55:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.715+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/782847164 >> 172.21.15.105:0/782847164 conn(0x7fa9b00fce80 msgr2=0x7fa9b0103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:55:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.715+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/782847164 shutdown_connections 2024-03-26T17:55:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:55:53.715+0000 7fa9b7e0b640 1 -- 172.21.15.105:0/782847164 wait complete. 2024-03-26T17:55:54.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:53 smithi135 ceph-mon[35636]: pgmap v875: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-03-26T17:55:54.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:53 smithi144 ceph-mon[36095]: pgmap v875: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-03-26T17:55:54.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:53 smithi105 ceph-mon[32280]: pgmap v875: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 4.1 MiB/s wr, 226 op/s 2024-03-26T17:55:54.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36576 7.20 MB/sec execute 157 sec latency 211.118 ms 2024-03-26T17:55:55.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:54 smithi135 ceph-mon[35636]: pgmap v876: 97 pgs: 97 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-03-26T17:55:55.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:54 smithi144 ceph-mon[36095]: pgmap v876: 97 pgs: 97 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-03-26T17:55:55.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:54 smithi105 ceph-mon[32280]: pgmap v876: 97 pgs: 97 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 4.3 MiB/s wr, 212 op/s 2024-03-26T17:55:55.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36699 7.20 MB/sec execute 158 sec latency 148.521 ms 2024-03-26T17:55:56.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36809 7.17 MB/sec execute 159 sec latency 119.386 ms 2024-03-26T17:55:57.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 36922 7.18 MB/sec execute 160 sec latency 130.528 ms 2024-03-26T17:55:58.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:57 smithi135 ceph-mon[35636]: pgmap v877: 97 pgs: 97 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-03-26T17:55:58.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:57 smithi144 ceph-mon[36095]: pgmap v877: 97 pgs: 97 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-03-26T17:55:58.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:57 smithi105 ceph-mon[32280]: pgmap v877: 97 pgs: 97 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-03-26T17:55:58.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 37037 7.16 MB/sec execute 161 sec latency 168.748 ms 2024-03-26T17:55:59.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 37164 7.14 MB/sec execute 162 sec latency 123.793 ms 2024-03-26T17:56:00.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:55:59 smithi135 ceph-mon[35636]: pgmap v878: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 4.4 MiB/s wr, 237 op/s 2024-03-26T17:56:00.187 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:55:59 smithi105 ceph-mon[32280]: pgmap v878: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 4.4 MiB/s wr, 237 op/s 2024-03-26T17:56:00.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:55:59 smithi144 ceph-mon[36095]: pgmap v878: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 4.4 MiB/s wr, 237 op/s 2024-03-26T17:56:00.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 37284 7.12 MB/sec execute 163 sec latency 142.364 ms 2024-03-26T17:56:01.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 37408 7.11 MB/sec execute 164 sec latency 137.044 ms 2024-03-26T17:56:02.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:01 smithi135 ceph-mon[35636]: pgmap v879: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-26T17:56:02.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:01 smithi144 ceph-mon[36095]: pgmap v879: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-26T17:56:02.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:01 smithi105 ceph-mon[32280]: pgmap v879: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-03-26T17:56:02.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 37524 7.09 MB/sec execute 165 sec latency 129.509 ms 2024-03-26T17:56:03.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 37652 7.08 MB/sec execute 166 sec latency 177.536 ms 2024-03-26T17:56:03.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:56:03.825 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:56:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.064+0000 7fa64f974640 1 -- 172.21.15.105:0/2605626366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6481023e0 msgr2=0x7fa6481047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.064+0000 7fa64f974640 1 --2- 172.21.15.105:0/2605626366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6481023e0 0x7fa6481047d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa644009900 tx=0x7fa64402bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.064+0000 7fa64f974640 1 -- 172.21.15.105:0/2605626366 shutdown_connections 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.064+0000 7fa64f974640 1 --2- 172.21.15.105:0/2605626366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa648107640 0x7fa648109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.064+0000 7fa64f974640 1 --2- 172.21.15.105:0/2605626366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa648104d10 0x7fa648107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.064+0000 7fa64f974640 1 --2- 172.21.15.105:0/2605626366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa6481023e0 0x7fa6481047d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.064+0000 7fa64f974640 1 -- 172.21.15.105:0/2605626366 >> 172.21.15.105:0/2605626366 conn(0x7fa6480fbd50 msgr2=0x7fa6480fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.065+0000 7fa64f974640 1 -- 172.21.15.105:0/2605626366 shutdown_connections 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.065+0000 7fa64f974640 1 -- 172.21.15.105:0/2605626366 wait complete. 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.065+0000 7fa64f974640 1 Processor -- start 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.065+0000 7fa64f974640 1 -- start start 2024-03-26T17:56:04.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64f974640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6481023e0 0x7fa648101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64f974640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa648104d10 0x7fa648100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64f974640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa648107640 0x7fa6481008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64f974640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa64810d400 con 0x7fa6481023e0 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64f974640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa64810d280 con 0x7fa648104d10 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64f974640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa64810d580 con 0x7fa648107640 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa648104d10 0x7fa648100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa648104d10 0x7fa648100390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46940/0 (socket says 172.21.15.105:46940) 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64d710640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6481023e0 0x7fa648101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64df11640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa648107640 0x7fa6481008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64cf0f640 1 -- 172.21.15.105:0/1099654040 learned_addr learned my addr 172.21.15.105:0/1099654040 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64df11640 1 -- 172.21.15.105:0/1099654040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa648104d10 msgr2=0x7fa648100390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64df11640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa648104d10 0x7fa648100390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64df11640 1 -- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6481023e0 msgr2=0x7fa648101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64df11640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6481023e0 0x7fa648101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64df11640 1 -- 172.21.15.105:0/1099654040 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa648101160 con 0x7fa648107640 2024-03-26T17:56:04.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.066+0000 7fa64d710640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6481023e0 0x7fa648101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:04.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.067+0000 7fa64df11640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa648107640 0x7fa6481008d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa640009fd0 tx=0x7fa64000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.067+0000 7fa63e7fc640 1 -- 172.21.15.105:0/1099654040 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64000ce20 con 0x7fa648107640 2024-03-26T17:56:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.067+0000 7fa64f974640 1 -- 172.21.15.105:0/1099654040 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6481a1c50 con 0x7fa648107640 2024-03-26T17:56:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.067+0000 7fa63e7fc640 1 -- 172.21.15.105:0/1099654040 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa640005020 con 0x7fa648107640 2024-03-26T17:56:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.067+0000 7fa63e7fc640 1 -- 172.21.15.105:0/1099654040 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640010690 con 0x7fa648107640 2024-03-26T17:56:04.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.068+0000 7fa64f974640 1 -- 172.21.15.105:0/1099654040 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6481a2150 con 0x7fa648107640 2024-03-26T17:56:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.070+0000 7fa63e7fc640 1 -- 172.21.15.105:0/1099654040 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa6400051d0 con 0x7fa648107640 2024-03-26T17:56:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa652636740 1 Processor -- start 2024-03-26T17:56:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa652636740 1 -- start start 2024-03-26T17:56:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa652636740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48100 0x55c12ff48520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa652636740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48af0 0x55c12ff4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa652636740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa652636740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c12fea4980 con 0x55c12ff4dc60 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa652636740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c12fe4cbe0 con 0x55c12ff48af0 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa652636740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c12fea0930 con 0x55c12ff48100 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa63e7fc640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6340a2f20 0x7fa6340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48100 0x55c12ff48520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa64d710640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48af0 0x55c12ff4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa64d710640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48af0 0x55c12ff4d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46946/0 (socket says 172.21.15.105:46946) 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48100 0x55c12ff48520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37466/0 (socket says 172.21.15.105:37466) 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa64d710640 1 -- 172.21.15.105:0/2281448721 learned_addr learned my addr 172.21.15.105:0/2281448721 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/2281448721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa63e7fc640 1 -- 172.21.15.105:0/1099654040 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa6400cfaf0 con 0x7fa648107640 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.071+0000 7fa64df11640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6340a2f20 0x7fa6340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa64d710640 1 -- 172.21.15.105:0/2281448721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48100 msgr2=0x55c12ff48520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa64d710640 1 --2- 172.21.15.105:0/2281448721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48100 0x55c12ff48520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa64d710640 1 -- 172.21.15.105:0/2281448721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 msgr2=0x55c12ff500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa64d710640 1 --2- 172.21.15.105:0/2281448721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa64d710640 1 -- 172.21.15.105:0/2281448721 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12fc827d0 con 0x55c12ff48af0 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa64d710640 1 --2- 172.21.15.105:0/2281448721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48af0 0x55c12ff4d5f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa644009900 tx=0x7fa64402df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=747081983ad83f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa64df11640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6340a2f20 0x7fa6340a53e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa640000f80 tx=0x7fa640005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa6337fe640 1 -- 172.21.15.105:0/2281448721 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644045030 con 0x55c12ff48af0 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa6337fe640 1 -- 172.21.15.105:0/2281448721 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa644005040 con 0x55c12ff48af0 2024-03-26T17:56:04.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.072+0000 7fa6337fe640 1 -- 172.21.15.105:0/2281448721 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644005d50 con 0x55c12ff48af0 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 -- 172.21.15.105:0/2281448721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48af0 msgr2=0x55c12ff4d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 --2- 172.21.15.105:0/2281448721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48af0 0x55c12ff4d5f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa644009900 tx=0x7fa64402df40 comp rx=0 tx=0).stop 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 -- 172.21.15.105:0/2281448721 shutdown_connections 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 --2- 172.21.15.105:0/2281448721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 --2- 172.21.15.105:0/2281448721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48af0 0x55c12ff4d5f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 --2- 172.21.15.105:0/2281448721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48100 0x55c12ff48520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 -- 172.21.15.105:0/2281448721 >> 172.21.15.105:0/2281448721 conn(0x55c12ff3ae60 msgr2=0x55c12ff3b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 -- 172.21.15.105:0/2281448721 shutdown_connections 2024-03-26T17:56:04.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 -- 172.21.15.105:0/2281448721 wait complete. 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 -- start start 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.073+0000 7fa652636740 1 Processor -- start 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa652636740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa652636740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa652636740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa652636740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c12ff50bb0 con 0x55c12ff4dc60 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa652636740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c12fe60160 con 0x55c12ff48100 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa652636740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c12ff50d30 con 0x55c12ff48af0 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46952/0 (socket says 172.21.15.105:46952) 2024-03-26T17:56:04.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa64cf0f640 1 -- 172.21.15.105:0/2891797725 learned_addr learned my addr 172.21.15.105:0/2891797725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa64d710640 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa652636740 10 client.0 _ll_register_callbacks cb 0x55c12fe9c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa64d710640 1 -- 172.21.15.105:0/2891797725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 msgr2=0x55c12ff523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa64d710640 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.074+0000 7fa64d710640 1 -- 172.21.15.105:0/2891797725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 msgr2=0x55c12ff598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa64d710640 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa64d710640 1 -- 172.21.15.105:0/2891797725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c12fbb83f0 con 0x55c12ff48af0 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa652636740 1 -- 172.21.15.105:0/2891797725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa644005040 con 0x55c12ff48af0 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa64d710640 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff528f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55c12ff48ef0 tx=0x7fa6440048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d6417bcb714f36f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa630ff9640 10 client.31528 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa630ff9640 1 -- 172.21.15.105:0/2891797725 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644005d50 con 0x55c12ff48af0 2024-03-26T17:56:04.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa630ff9640 1 -- 172.21.15.105:0/2891797725 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa644034440 con 0x55c12ff48af0 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa630ff9640 1 -- 172.21.15.105:0/2891797725 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa6440346c0 con 0x55c12ff48af0 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa652636740 10 client.31528 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.075+0000 7fa652636740 1 -- 172.21.15.105:0/2891797725 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55c12fb780f0 con 0x55c12ff48af0 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa630ff9640 1 -- 172.21.15.105:0/2891797725 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa644034860 con 0x55c12ff48af0 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa652636740 10 client.31528 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa652636740 10 client.31528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa652636740 20 client.31528 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa652636740 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c12ff5dc80 0x55c12ff60070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa652636740 4 client.31528 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa652636740 1 -- 172.21.15.105:0/2891797725 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c12feb1ec0 con 0x55c12ff5dc80 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c12ff5dc80 0x55c12ff60070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c12ff5dc80 0x55c12ff60070 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199b6e23704a5cae server_cookie=a956761d3171b645 in_seq=0 out_seq=0 2024-03-26T17:56:04.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.076+0000 7fa630ff9640 10 client.31528 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.078+0000 7fa630ff9640 1 -- 172.21.15.105:0/2891797725 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c12feb1ec0 con 0x55c12ff5dc80 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.078+0000 7fa630ff9640 10 client.31528 handle_command_reply: tid=0 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 client.31528 shutdown 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 -- 172.21.15.105:0/2891797725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 msgr2=0x55c12ff528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff528f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55c12ff48ef0 tx=0x7fa6440048f0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 -- 172.21.15.105:0/2891797725 shutdown_connections 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c12ff5dc80 0x55c12ff60070 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ff598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff528f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 --2- 172.21.15.105:0/2891797725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 -- 172.21.15.105:0/2891797725 >> 172.21.15.105:0/2891797725 conn(0x55c12ff3ae60 msgr2=0x55c12ff3a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa630ff9640 0 client.31528 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 -- 172.21.15.105:0/2891797725 shutdown_connections 2024-03-26T17:56:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 1 -- 172.21.15.105:0/2891797725 wait complete. 2024-03-26T17:56:04.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.079+0000 7fa652636740 20 client.31528 trim_cache size 0 max 16384 2024-03-26T17:56:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa652636740 1 Processor -- start 2024-03-26T17:56:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa652636740 1 -- start start 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa652636740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff48100 0x55c12ff49d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa652636740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff59880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa652636740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff4dc60 0x55c12ff4a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa652636740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481a6b30 con 0x55c12ff48100 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa652636740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481a8670 con 0x55c12ff4dc60 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa652636740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481a7200 con 0x55c12ff48af0 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff48100 0x55c12ff49d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff48100 0x55c12ff49d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57880/0 (socket says 172.21.15.105:57880) 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa64cf0f640 1 -- 172.21.15.105:0/3027588058 learned_addr learned my addr 172.21.15.105:0/3027588058 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.108+0000 7fa64d710640 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff59880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff4dc60 0x55c12ff4a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa64d710640 1 -- 172.21.15.105:0/3027588058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff4dc60 msgr2=0x55c12ff4a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa64d710640 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff4dc60 0x55c12ff4a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa64d710640 1 -- 172.21.15.105:0/3027588058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff48100 msgr2=0x55c12ff49d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa64d710640 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff48100 0x55c12ff49d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa64d710640 1 -- 172.21.15.105:0/3027588058 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12ff3b8d0 con 0x55c12ff48af0 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff4dc60 0x55c12ff4a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa64d710640 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff59880 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa64402c170 tx=0x7fa644035000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c14a6324f03facce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa632ffd640 1 -- 172.21.15.105:0/3027588058 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644045020 con 0x55c12ff48af0 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa632ffd640 1 -- 172.21.15.105:0/3027588058 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa644005040 con 0x55c12ff48af0 2024-03-26T17:56:04.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.109+0000 7fa632ffd640 1 -- 172.21.15.105:0/3027588058 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64403fb40 con 0x55c12ff48af0 2024-03-26T17:56:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 -- 172.21.15.105:0/3027588058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 msgr2=0x55c12ff59880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff59880 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa64402c170 tx=0x7fa644035000 comp rx=0 tx=0).stop 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 -- 172.21.15.105:0/3027588058 shutdown_connections 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff4dc60 0x55c12ff4a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff59880 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 --2- 172.21.15.105:0/3027588058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff48100 0x55c12ff49d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 -- 172.21.15.105:0/3027588058 >> 172.21.15.105:0/3027588058 conn(0x55c12ff3ae60 msgr2=0x55c12ff5fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 -- 172.21.15.105:0/3027588058 shutdown_connections 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.110+0000 7fa652636740 1 -- 172.21.15.105:0/3027588058 wait complete. 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.111+0000 7fa652636740 1 -- start start 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.111+0000 7fa652636740 1 Processor -- start 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa652636740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff56620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa652636740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff56b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa652636740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ffc1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa652636740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481a7fa0 con 0x55c12ff4dc60 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa652636740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481a78d0 con 0x55c12ff48100 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa652636740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481a4b10 con 0x55c12ff48af0 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa652636740 10 client.0 _ll_register_callbacks cb 0x7fa638009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ffc1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ffc1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57894/0 (socket says 172.21.15.105:57894) 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 -- 172.21.15.105:0/3184442787 learned_addr learned my addr 172.21.15.105:0/3184442787 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff56620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64d710640 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff56b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 -- 172.21.15.105:0/3184442787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 msgr2=0x55c12ff56b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff56b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 -- 172.21.15.105:0/3184442787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 msgr2=0x55c12ff56620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff56620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 -- 172.21.15.105:0/3184442787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c12ff57150 con 0x55c12ff4dc60 2024-03-26T17:56:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa64cf0f640 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ffc1e50 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa638009f10 tx=0x7fa638005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d29015e69902d91 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.112+0000 7fa630ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.113+0000 7fa630ff9640 1 -- 172.21.15.105:0/3184442787 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638004350 con 0x55c12ff4dc60 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.113+0000 7fa652636740 1 -- 172.21.15.105:0/3184442787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa644005040 con 0x55c12ff4dc60 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.113+0000 7fa630ff9640 1 -- 172.21.15.105:0/3184442787 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fa638005020 con 0x55c12ff4dc60 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.113+0000 7fa630ff9640 1 -- 172.21.15.105:0/3184442787 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa638005220 con 0x55c12ff4dc60 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.113+0000 7fa652636740 10 client.21819 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.113+0000 7fa652636740 1 -- 172.21.15.105:0/3184442787 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c12ffc34d0 con 0x55c12ff4dc60 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa630ff9640 1 -- 172.21.15.105:0/3184442787 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa644005040 con 0x55c12ff4dc60 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa630ff9640 1 -- 172.21.15.105:0/3184442787 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fa638034070 con 0x55c12ff4dc60 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa652636740 10 client.21819 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa652636740 10 client.21819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa652636740 20 client.21819 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa652636740 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c12ffc3e00 0x55c12ffc6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa652636740 4 client.21819 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:56:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa652636740 1 -- 172.21.15.105:0/3184442787 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c12feac210 con 0x55c12ffc3e00 2024-03-26T17:56:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa64d710640 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c12ffc3e00 0x55c12ffc6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa630ff9640 10 client.21819 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.114+0000 7fa64d710640 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c12ffc3e00 0x55c12ffc6220 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76597e4401b42fc server_cookie=f33324d378a4ab3c in_seq=0 out_seq=0 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa630ff9640 1 -- 172.21.15.105:0/3184442787 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c12feac210 con 0x55c12ffc3e00 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa630ff9640 10 client.21819 handle_command_reply: tid=0 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 client.21819 shutdown 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 -- 172.21.15.105:0/3184442787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 msgr2=0x55c12ffc1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ffc1e50 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa638009f10 tx=0x7fa638005f90 comp rx=0 tx=0).stop 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 -- 172.21.15.105:0/3184442787 shutdown_connections 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c12ffc3e00 0x55c12ffc6220 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c12ff4dc60 0x55c12ffc1e50 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c12ff48af0 0x55c12ff56b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 --2- 172.21.15.105:0/3184442787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c12ff48100 0x55c12ff56620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 -- 172.21.15.105:0/3184442787 >> 172.21.15.105:0/3184442787 conn(0x55c12ff3ae60 msgr2=0x55c12ff523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa630ff9640 0 client.21819 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.116+0000 7fa652636740 1 -- 172.21.15.105:0/3184442787 shutdown_connections 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa652636740 1 -- 172.21.15.105:0/3184442787 wait complete. 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa652636740 20 client.21819 trim_cache size 0 max 16384 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:56:04.107 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:56:04.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 -- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6340a2f20 msgr2=0x7fa6340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6340a2f20 0x7fa6340a53e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa640000f80 tx=0x7fa640005f70 comp rx=0 tx=0).stop 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 -- 172.21.15.105:0/1099654040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa648107640 msgr2=0x7fa6481008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa648107640 0x7fa6481008d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa640009fd0 tx=0x7fa64000ef10 comp rx=0 tx=0).stop 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 -- 172.21.15.105:0/1099654040 shutdown_connections 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa6340a2f20 0x7fa6340a53e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa648107640 0x7fa6481008d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa648104d10 0x7fa648100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 --2- 172.21.15.105:0/1099654040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6481023e0 0x7fa648101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.117+0000 7fa64f974640 1 -- 172.21.15.105:0/1099654040 >> 172.21.15.105:0/1099654040 conn(0x7fa6480fbd50 msgr2=0x7fa648102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.118+0000 7fa64f974640 1 -- 172.21.15.105:0/1099654040 shutdown_connections 2024-03-26T17:56:04.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.118+0000 7fa64f974640 1 -- 172.21.15.105:0/1099654040 wait complete. 2024-03-26T17:56:04.118 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:56:04.119 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:56:04.147 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:03 smithi105 ceph-mon[32280]: pgmap v880: 97 pgs: 97 active+clean; 293 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-03-26T17:56:04.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:03 smithi135 ceph-mon[35636]: pgmap v880: 97 pgs: 97 active+clean; 293 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-03-26T17:56:04.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 -- 172.21.15.105:0/1067196820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63c06e9d0 msgr2=0x7ff63c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 --2- 172.21.15.105:0/1067196820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63c06e9d0 0x7ff63c06ee30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009900 tx=0x7ff62c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 -- 172.21.15.105:0/1067196820 shutdown_connections 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 --2- 172.21.15.105:0/1067196820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff63c06f370 0x7ff63c06fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 --2- 172.21.15.105:0/1067196820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63c06e9d0 0x7ff63c06ee30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 --2- 172.21.15.105:0/1067196820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c06d790 0x7ff63c06dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 -- 172.21.15.105:0/1067196820 >> 172.21.15.105:0/1067196820 conn(0x7ff63c06c360 msgr2=0x7ff63c06c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 -- 172.21.15.105:0/1067196820 shutdown_connections 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.272+0000 7ff6438a1640 1 -- 172.21.15.105:0/1067196820 wait complete. 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff6438a1640 1 Processor -- start 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff6438a1640 1 -- start start 2024-03-26T17:56:04.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff6438a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff63c06d790 0x7ff63c1a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff6438a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63c06e9d0 0x7ff63c1a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff6438a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c06f370 0x7ff63c1adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff6438a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c11c230 con 0x7ff63c06f370 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff6438a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c11c0b0 con 0x7ff63c06d790 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff6438a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c11c3b0 con 0x7ff63c06e9d0 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff64163d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff63c06d790 0x7ff63c1a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff640e3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63c06e9d0 0x7ff63c1a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff64163d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff63c06d790 0x7ff63c1a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:47002/0 (socket says 172.21.15.105:47002) 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.273+0000 7ff64163d640 1 -- 172.21.15.105:0/2718734837 learned_addr learned my addr 172.21.15.105:0/2718734837 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff641e3e640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c06f370 0x7ff63c1adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff64163d640 1 -- 172.21.15.105:0/2718734837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63c06e9d0 msgr2=0x7ff63c1a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff64163d640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63c06e9d0 0x7ff63c1a6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff64163d640 1 -- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c06f370 msgr2=0x7ff63c1adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff64163d640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c06f370 0x7ff63c1adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff64163d640 1 -- 172.21.15.105:0/2718734837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c1ae570 con 0x7ff63c06d790 2024-03-26T17:56:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff641e3e640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c06f370 0x7ff63c1adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff64163d640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff63c06d790 0x7ff63c1a6950 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff63400ebc0 tx=0x7ff63400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff6327fc640 1 -- 172.21.15.105:0/2718734837 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634010630 con 0x7ff63c06d790 2024-03-26T17:56:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c1ae860 con 0x7ff63c06d790 2024-03-26T17:56:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff6327fc640 1 -- 172.21.15.105:0/2718734837 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff634010c80 con 0x7ff63c06d790 2024-03-26T17:56:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff6327fc640 1 -- 172.21.15.105:0/2718734837 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634018b30 con 0x7ff63c06d790 2024-03-26T17:56:04.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.274+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c1aedd0 con 0x7ff63c06d790 2024-03-26T17:56:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.275+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63c118c60 con 0x7ff63c06d790 2024-03-26T17:56:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.277+0000 7ff6327fc640 1 -- 172.21.15.105:0/2718734837 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff634018cd0 con 0x7ff63c06d790 2024-03-26T17:56:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.277+0000 7ff6327fc640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6280a2fa0 0x7ff6280a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.278+0000 7ff6327fc640 1 -- 172.21.15.105:0/2718734837 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ff6340cff60 con 0x7ff63c06d790 2024-03-26T17:56:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.278+0000 7ff640e3c640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6280a2fa0 0x7ff6280a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.278+0000 7ff640e3c640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6280a2fa0 0x7ff6280a5460 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009230 tx=0x7ff62c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.283+0000 7ff6327fc640 1 -- 172.21.15.105:0/2718734837 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff63409ccf0 con 0x7ff63c06d790 2024-03-26T17:56:04.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:03 smithi144 ceph-mon[36095]: pgmap v880: 97 pgs: 97 active+clean; 293 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 3.4 MiB/s wr, 225 op/s 2024-03-26T17:56:04.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 37796 7.09 MB/sec execute 167 sec latency 126.709 ms 2024-03-26T17:56:04.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.650+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff63c02d240 con 0x7ff63c06d790 2024-03-26T17:56:04.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.651+0000 7ff6327fc640 1 -- 172.21.15.105:0/2718734837 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11941 (secure 0 0 0) 0x7ff63409c5f0 con 0x7ff63c06d790 2024-03-26T17:56:04.638 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:56:04.639 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:54:01.817010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:56:04.639 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 37 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.653+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6280a2fa0 msgr2=0x7ff6280a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.653+0000 7ff6438a1640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6280a2fa0 0x7ff6280a5460 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009230 tx=0x7ff62c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.653+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff63c06d790 msgr2=0x7ff63c1a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff63c06d790 0x7ff63c1a6950 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff63400ebc0 tx=0x7ff63400c6a0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 shutdown_connections 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6280a2fa0 0x7ff6280a5460 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff63c06f370 0x7ff63c1adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63c06e9d0 0x7ff63c1a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 --2- 172.21.15.105:0/2718734837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff63c06d790 0x7ff63c1a6950 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 >> 172.21.15.105:0/2718734837 conn(0x7ff63c06c360 msgr2=0x7ff63c10a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 shutdown_connections 2024-03-26T17:56:04.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.654+0000 7ff6438a1640 1 -- 172.21.15.105:0/2718734837 wait complete. 2024-03-26T17:56:04.664 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 -- 172.21.15.105:0/453356662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560104d30 msgr2=0x7f0560107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 --2- 172.21.15.105:0/453356662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560104d30 0x7f0560107120 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f0554009900 tx=0x7f055402bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 -- 172.21.15.105:0/453356662 shutdown_connections 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 --2- 172.21.15.105:0/453356662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0560107660 0x7f0560109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 --2- 172.21.15.105:0/453356662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560104d30 0x7f0560107120 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 --2- 172.21.15.105:0/453356662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0560102400 0x7f05601047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 -- 172.21.15.105:0/453356662 >> 172.21.15.105:0/453356662 conn(0x7f05600fbd50 msgr2=0x7f05600fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 -- 172.21.15.105:0/453356662 shutdown_connections 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.862+0000 7f056640b640 1 -- 172.21.15.105:0/453356662 wait complete. 2024-03-26T17:56:04.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f056640b640 1 Processor -- start 2024-03-26T17:56:04.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f056640b640 1 -- start start 2024-03-26T17:56:04.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f056640b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560102400 0x7f0560199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f056640b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0560104d30 0x7f056019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f056640b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0560107660 0x7f05601a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f056640b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f056010d3d0 con 0x7f0560102400 2024-03-26T17:56:04.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f056640b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f056010d250 con 0x7f0560104d30 2024-03-26T17:56:04.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f056640b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f056010d550 con 0x7f0560107660 2024-03-26T17:56:04.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f055ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560102400 0x7f0560199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0560104d30 0x7f056019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f055ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560102400 0x7f0560199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59628/0 (socket says 172.21.15.105:59628) 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f055ffff640 1 -- 172.21.15.105:0/270889697 learned_addr learned my addr 172.21.15.105:0/270889697 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0560104d30 0x7f056019a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45586/0 (socket says 172.21.15.105:45586) 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.863+0000 7f05649a8640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0560107660 0x7f05601a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055ffff640 1 -- 172.21.15.105:0/270889697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0560107660 msgr2=0x7f05601a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055ffff640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0560107660 0x7f05601a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055ffff640 1 -- 172.21.15.105:0/270889697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0560104d30 msgr2=0x7f056019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055ffff640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0560104d30 0x7f056019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055ffff640 1 -- 172.21.15.105:0/270889697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05601a1990 con 0x7f0560102400 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055ffff640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560102400 0x7f0560199d70 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f055000e9f0 tx=0x7f055000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055d7fa640 1 -- 172.21.15.105:0/270889697 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055000cd80 con 0x7f0560102400 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f056640b640 1 -- 172.21.15.105:0/270889697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05601a1c80 con 0x7f0560102400 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055d7fa640 1 -- 172.21.15.105:0/270889697 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05500108e0 con 0x7f0560102400 2024-03-26T17:56:04.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f055d7fa640 1 -- 172.21.15.105:0/270889697 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550018920 con 0x7f0560102400 2024-03-26T17:56:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.864+0000 7f056640b640 1 -- 172.21.15.105:0/270889697 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05601a21a0 con 0x7f0560102400 2024-03-26T17:56:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.867+0000 7f055d7fa640 1 -- 172.21.15.105:0/270889697 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0550010a90 con 0x7f0560102400 2024-03-26T17:56:04.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.867+0000 7f0568ffc740 1 Processor -- start 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.867+0000 7f0568ffc740 1 -- start start 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.867+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a5ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5d420 0x55c740a61f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a64a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f0568ffc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c7409c1970 con 0x55c740a5d420 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f0568ffc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c740969d70 con 0x55c740a62590 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f0568ffc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c7409bd920 con 0x55c740a5ca30 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f05649a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a5ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f05649a8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a5ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34464/0 (socket says 172.21.15.105:34464) 2024-03-26T17:56:04.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f05649a8640 1 -- 172.21.15.105:0/2883614469 learned_addr learned my addr 172.21.15.105:0/2883614469 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055d7fa640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05480a2f20 0x7f05480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055ffff640 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5d420 0x55c740a61f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055f7fe640 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a64a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f05649a8640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05480a2f20 0x7f05480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055f7fe640 1 -- 172.21.15.105:0/2883614469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 msgr2=0x55c740a5ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055f7fe640 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a5ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055f7fe640 1 -- 172.21.15.105:0/2883614469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5d420 msgr2=0x55c740a61f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055f7fe640 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5d420 0x55c740a61f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055d7fa640 1 -- 172.21.15.105:0/270889697 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f05500cf700 con 0x7f0560102400 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.868+0000 7f055f7fe640 1 -- 172.21.15.105:0/2883614469 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7406d53f0 con 0x55c740a62590 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.869+0000 7f055f7fe640 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a64a20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0554036040 tx=0x7f0554009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b81e3f2ab6c757 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.869+0000 7f05649a8640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05480a2f20 0x7f05480a53e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0558009fd0 tx=0x7f0558009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.869+0000 7f05467fc640 1 -- 172.21.15.105:0/2883614469 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554045020 con 0x55c740a62590 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.869+0000 7f05467fc640 1 -- 172.21.15.105:0/2883614469 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0554004040 con 0x55c740a62590 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.869+0000 7f05467fc640 1 -- 172.21.15.105:0/2883614469 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540055a0 con 0x55c740a62590 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 -- 172.21.15.105:0/2883614469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 msgr2=0x55c740a64a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a64a20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0554036040 tx=0x7f0554009560 comp rx=0 tx=0).stop 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 -- 172.21.15.105:0/2883614469 shutdown_connections 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a64a20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5d420 0x55c740a61f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 --2- 172.21.15.105:0/2883614469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a5ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 -- 172.21.15.105:0/2883614469 >> 172.21.15.105:0/2883614469 conn(0x55c740a57fa0 msgr2=0x55c740a583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 -- 172.21.15.105:0/2883614469 shutdown_connections 2024-03-26T17:56:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 -- 172.21.15.105:0/2883614469 wait complete. 2024-03-26T17:56:04.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.870+0000 7f0568ffc740 1 -- start start 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f0568ffc740 1 Processor -- start 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5ca30 0x55c740a77eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f0568ffc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c740a764f0 con 0x55c740a5ca30 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f0568ffc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c74097d310 con 0x55c740a62590 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f0568ffc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c740a76670 con 0x55c740a5d420 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f0568ffc740 10 client.0 _ll_register_callbacks cb 0x55c7409b8c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a783f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34478/0 (socket says 172.21.15.105:34478) 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f055f7fe640 1 -- 172.21.15.105:0/861941368 learned_addr learned my addr 172.21.15.105:0/861941368 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f05649a8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.871+0000 7f055ffff640 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5ca30 0x55c740a77eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f055f7fe640 1 -- 172.21.15.105:0/861941368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 msgr2=0x55c740a7f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f055f7fe640 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a7f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f055f7fe640 1 -- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5ca30 msgr2=0x55c740a77eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f055f7fe640 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5ca30 0x55c740a77eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f055f7fe640 1 -- 172.21.15.105:0/861941368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74079f7d0 con 0x55c740a5d420 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f055ffff640 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5ca30 0x55c740a77eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f055f7fe640 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a783f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f055402c170 tx=0x7f05540044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a964e5219894576 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f053f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f053f7fe640 1 -- 172.21.15.105:0/861941368 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055404e070 con 0x55c740a5d420 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f0568ffc740 1 -- 172.21.15.105:0/861941368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0554004040 con 0x55c740a5d420 2024-03-26T17:56:04.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f053f7fe640 1 -- 172.21.15.105:0/861941368 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f05540035f0 con 0x55c740a5d420 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f053f7fe640 1 -- 172.21.15.105:0/861941368 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f055405a050 con 0x55c740a5d420 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f0568ffc740 10 client.31552 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f0568ffc740 1 -- 172.21.15.105:0/861941368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c740a76540 con 0x55c740a5d420 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.872+0000 7f053f7fe640 1 -- 172.21.15.105:0/861941368 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0554004040 con 0x55c740a5d420 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f053f7fe640 1 -- 172.21.15.105:0/861941368 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0554004b50 con 0x55c740a5d420 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f0568ffc740 10 client.31552 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f0568ffc740 10 client.31552 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f0568ffc740 20 client.31552 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f0568ffc740 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55c740a83970 0x55c740a85de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f0568ffc740 4 client.31552 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f0568ffc740 1 -- 172.21.15.105:0/861941368 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7409ceeb0 con 0x55c740a83970 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f055ffff640 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55c740a83970 0x55c740a85de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f053f7fe640 10 client.31552 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.873+0000 7f055ffff640 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55c740a83970 0x55c740a85de0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16418f33fb3b5983 server_cookie=f2d4811a6f81ac61 in_seq=0 out_seq=0 2024-03-26T17:56:04.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.874+0000 7f053f7fe640 1 -- 172.21.15.105:0/861941368 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c7409ceeb0 con 0x55c740a83970 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.874+0000 7f053f7fe640 10 client.31552 handle_command_reply: tid=0 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.874+0000 7f0568ffc740 1 client.31552 shutdown 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.874+0000 7f0568ffc740 1 -- 172.21.15.105:0/861941368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 msgr2=0x55c740a783f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.874+0000 7f0568ffc740 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a783f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f055402c170 tx=0x7f05540044a0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 1 -- 172.21.15.105:0/861941368 shutdown_connections 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55c740a83970 0x55c740a85de0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f053f7fe640 0 client.31552 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a62590 0x55c740a7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a783f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 1 --2- 172.21.15.105:0/861941368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a5ca30 0x55c740a77eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 1 -- 172.21.15.105:0/861941368 >> 172.21.15.105:0/861941368 conn(0x55c740a57fa0 msgr2=0x55c740a571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 1 -- 172.21.15.105:0/861941368 shutdown_connections 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 1 -- 172.21.15.105:0/861941368 wait complete. 2024-03-26T17:56:04.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.875+0000 7f0568ffc740 20 client.31552 trim_cache size 0 max 16384 2024-03-26T17:56:04.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f0568ffc740 1 Processor -- start 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f0568ffc740 1 -- start start 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5ca30 0x55c740a53050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a7f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a56930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f0568ffc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05601a2730 con 0x55c740a62590 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f0568ffc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05601a4270 con 0x55c740a5ca30 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f0568ffc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05601a2e00 con 0x55c740a5d420 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f05649a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a56930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.900+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a7f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f05649a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a56930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59656/0 (socket says 172.21.15.105:59656) 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f05649a8640 1 -- 172.21.15.105:0/1078881171 learned_addr learned my addr 172.21.15.105:0/1078881171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f055ffff640 1 --2- 172.21.15.105:0/1078881171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5ca30 0x55c740a53050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f05649a8640 1 -- 172.21.15.105:0/1078881171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 msgr2=0x55c740a7f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f05649a8640 1 --2- 172.21.15.105:0/1078881171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a7f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f05649a8640 1 -- 172.21.15.105:0/1078881171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5ca30 msgr2=0x55c740a53050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f05649a8640 1 --2- 172.21.15.105:0/1078881171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5ca30 0x55c740a53050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f05649a8640 1 -- 172.21.15.105:0/1078881171 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c740a53820 con 0x55c740a62590 2024-03-26T17:56:04.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f05649a8640 1 --2- 172.21.15.105:0/1078881171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a56930 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f0558009d30 tx=0x7f055801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273e34182ca799a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.901+0000 7f0545ffb640 1 -- 172.21.15.105:0/1078881171 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558003970 con 0x55c740a62590 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.902+0000 7f0545ffb640 1 -- 172.21.15.105:0/1078881171 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0558020900 con 0x55c740a62590 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.902+0000 7f0545ffb640 1 -- 172.21.15.105:0/1078881171 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580041b0 con 0x55c740a62590 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 -- 172.21.15.105:0/1078881171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 msgr2=0x55c740a56930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 --2- 172.21.15.105:0/1078881171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a56930 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f0558009d30 tx=0x7f055801d000 comp rx=0 tx=0).stop 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 -- 172.21.15.105:0/1078881171 shutdown_connections 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 --2- 172.21.15.105:0/1078881171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a56930 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 --2- 172.21.15.105:0/1078881171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5d420 0x55c740a7f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 --2- 172.21.15.105:0/1078881171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5ca30 0x55c740a53050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 -- 172.21.15.105:0/1078881171 >> 172.21.15.105:0/1078881171 conn(0x55c740a57fa0 msgr2=0x55c740a5e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 -- 172.21.15.105:0/1078881171 shutdown_connections 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 -- 172.21.15.105:0/1078881171 wait complete. 2024-03-26T17:56:04.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 -- start start 2024-03-26T17:56:04.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.903+0000 7f0568ffc740 1 Processor -- start 2024-03-26T17:56:04.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5d420 0x55c740a7ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f0568ffc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a7dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f0568ffc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05601a3ba0 con 0x55c740a62590 2024-03-26T17:56:04.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f0568ffc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05601a34d0 con 0x55c740a5d420 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f0568ffc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0560071dd0 con 0x55c740a5ca30 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f0568ffc740 10 client.0 _ll_register_callbacks cb 0x7f055009bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f05649a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a7dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5d420 0x55c740a7ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f05649a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a7dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59672/0 (socket says 172.21.15.105:59672) 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f05649a8640 1 -- 172.21.15.105:0/3295039831 learned_addr learned my addr 172.21.15.105:0/3295039831 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f055ffff640 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a7c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f05649a8640 1 -- 172.21.15.105:0/3295039831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 msgr2=0x55c740a7c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f05649a8640 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a7c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f05649a8640 1 -- 172.21.15.105:0/3295039831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5d420 msgr2=0x55c740a7ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f05649a8640 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5d420 0x55c740a7ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.904+0000 7f05649a8640 1 -- 172.21.15.105:0/3295039831 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c740a7e2f0 con 0x55c740a62590 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.905+0000 7f05649a8640 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a7dd00 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0558009d00 tx=0x7f0558020ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6253ae113e6dfa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:04.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.905+0000 7f053f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.905+0000 7f053f7fe640 1 -- 172.21.15.105:0/3295039831 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558003b80 con 0x55c740a62590 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.905+0000 7f053f7fe640 1 -- 172.21.15.105:0/3295039831 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f0558026040 con 0x55c740a62590 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.905+0000 7f0568ffc740 1 -- 172.21.15.105:0/3295039831 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0558020900 con 0x55c740a62590 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.905+0000 7f053f7fe640 1 -- 172.21.15.105:0/3295039831 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0558003d20 con 0x55c740a62590 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.905+0000 7f0568ffc740 10 client.21843 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.905+0000 7f0568ffc740 1 -- 172.21.15.105:0/3295039831 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c740a7f5d0 con 0x55c740a62590 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.906+0000 7f053f7fe640 1 -- 172.21.15.105:0/3295039831 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0558020900 con 0x55c740a62590 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.906+0000 7f053f7fe640 1 -- 172.21.15.105:0/3295039831 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f055802a660 con 0x55c740a62590 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.906+0000 7f0568ffc740 10 client.21843 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.906+0000 7f0568ffc740 10 client.21843 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.906+0000 7f0568ffc740 20 client.21843 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.906+0000 7f0568ffc740 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55c740a83970 0x55c740aec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.906+0000 7f0568ffc740 4 client.21843 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.906+0000 7f0568ffc740 1 -- 172.21.15.105:0/3295039831 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7409c9200 con 0x55c740a83970 2024-03-26T17:56:04.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.907+0000 7f055ffff640 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55c740a83970 0x55c740aec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.907+0000 7f053f7fe640 10 client.21843 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.907+0000 7f055ffff640 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55c740a83970 0x55c740aec340 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd66f31aee2cecde server_cookie=623ec8b006d9bfe5 in_seq=0 out_seq=0 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.907+0000 7f053f7fe640 1 -- 172.21.15.105:0/3295039831 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7409c9200 con 0x55c740a83970 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.907+0000 7f053f7fe640 10 client.21843 handle_command_reply: tid=0 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.907+0000 7f0568ffc740 1 client.21843 shutdown 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.907+0000 7f0568ffc740 1 -- 172.21.15.105:0/3295039831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 msgr2=0x55c740a7dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.907+0000 7f0568ffc740 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a7dd00 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f0558009d00 tx=0x7f0558020ce0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 1 -- 172.21.15.105:0/3295039831 shutdown_connections 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55c740a83970 0x55c740aec340 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c740a62590 0x55c740a7dd00 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c740a5d420 0x55c740a7ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 1 --2- 172.21.15.105:0/3295039831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c740a5ca30 0x55c740a7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 1 -- 172.21.15.105:0/3295039831 >> 172.21.15.105:0/3295039831 conn(0x55c740a57fa0 msgr2=0x55c740a5b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f053f7fe640 0 client.21843 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:56:04.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 1 -- 172.21.15.105:0/3295039831 shutdown_connections 2024-03-26T17:56:04.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 1 -- 172.21.15.105:0/3295039831 wait complete. 2024-03-26T17:56:04.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.908+0000 7f0568ffc740 20 client.21843 trim_cache size 0 max 16384 2024-03-26T17:56:04.899 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:04.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 -- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05480a2f20 msgr2=0x7f05480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05480a2f20 0x7f05480a53e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0558009fd0 tx=0x7f0558009f00 comp rx=0 tx=0).stop 2024-03-26T17:56:04.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 -- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560102400 msgr2=0x7f0560199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:04.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560102400 0x7f0560199d70 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f055000e9f0 tx=0x7f055000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 -- 172.21.15.105:0/270889697 shutdown_connections 2024-03-26T17:56:04.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f05480a2f20 0x7f05480a53e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0560107660 0x7f05601a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0560104d30 0x7f056019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 --2- 172.21.15.105:0/270889697 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0560102400 0x7f0560199d70 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:04.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 -- 172.21.15.105:0/270889697 >> 172.21.15.105:0/270889697 conn(0x7f05600fbd50 msgr2=0x7f0560101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:04.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 -- 172.21.15.105:0/270889697 shutdown_connections 2024-03-26T17:56:04.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:04.909+0000 7f056640b640 1 -- 172.21.15.105:0/270889697 wait complete. 2024-03-26T17:56:04.912 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:56:04.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:04 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2718734837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:56:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 -- 172.21.15.105:0/208028360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc103b00 msgr2=0x7f2dbc103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/208028360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc103b00 0x7f2dbc103f20 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009900 tx=0x7f2db802bc50 comp rx=0 tx=0).stop 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 -- 172.21.15.105:0/208028360 shutdown_connections 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/208028360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dbc1056b0 0x7f2dbc10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/208028360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc104cf0 0x7f2dbc105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/208028360 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc103b00 0x7f2dbc103f20 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 -- 172.21.15.105:0/208028360 >> 172.21.15.105:0/208028360 conn(0x7f2dbc0ff070 msgr2=0x7f2dbc1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 -- 172.21.15.105:0/208028360 shutdown_connections 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.069+0000 7f2dc45cd640 1 -- 172.21.15.105:0/208028360 wait complete. 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc45cd640 1 Processor -- start 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc45cd640 1 -- start start 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc45cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dbc103b00 0x7f2dbc199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc45cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc104cf0 0x7f2dbc19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc45cd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc1056b0 0x7f2dbc1a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc45cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc10f6a0 con 0x7f2dbc104cf0 2024-03-26T17:56:05.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc45cd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc10f520 con 0x7f2dbc103b00 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc45cd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc10f820 con 0x7f2dbc1056b0 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc1b68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc104cf0 0x7f2dbc19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc1b68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc104cf0 0x7f2dbc19a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59684/0 (socket says 172.21.15.105:59684) 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc1b68640 1 -- 172.21.15.105:0/3725046872 learned_addr learned my addr 172.21.15.105:0/3725046872 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc2b6a640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc1056b0 0x7f2dbc1a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.070+0000 7f2dc2369640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dbc103b00 0x7f2dbc199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2dc1b68640 1 -- 172.21.15.105:0/3725046872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc1056b0 msgr2=0x7f2dbc1a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc1056b0 0x7f2dbc1a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2dc1b68640 1 -- 172.21.15.105:0/3725046872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dbc103b00 msgr2=0x7f2dbc199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dbc103b00 0x7f2dbc199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2dc1b68640 1 -- 172.21.15.105:0/3725046872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dbc1a1a20 con 0x7f2dbc104cf0 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc104cf0 0x7f2dbc19a340 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b450 tx=0x7f2dac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2db37fe640 1 -- 172.21.15.105:0/3725046872 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac015020 con 0x7f2dbc104cf0 2024-03-26T17:56:05.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2db37fe640 1 -- 172.21.15.105:0/3725046872 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dac011d40 con 0x7f2dbc104cf0 2024-03-26T17:56:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2db37fe640 1 -- 172.21.15.105:0/3725046872 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00f340 con 0x7f2dbc104cf0 2024-03-26T17:56:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2dc45cd640 1 -- 172.21.15.105:0/3725046872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dbc1a1d10 con 0x7f2dbc104cf0 2024-03-26T17:56:05.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.071+0000 7f2dc45cd640 1 -- 172.21.15.105:0/3725046872 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dbc1a21f0 con 0x7f2dbc104cf0 2024-03-26T17:56:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.074+0000 7f2db37fe640 1 -- 172.21.15.105:0/3725046872 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2dac018020 con 0x7f2dbc104cf0 2024-03-26T17:56:05.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.074+0000 7f2dc71ba740 1 Processor -- start 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc71ba740 1 -- start start 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee644e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee649f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee64ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560aee5a9970 con 0x560aee645420 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560aee551d70 con 0x560aee644a30 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560aee5a5920 con 0x560aee64a590 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc1b68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee64ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc2369640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee649f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc1b68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee64ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34518/0 (socket says 172.21.15.105:34518) 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc1b68640 1 -- 172.21.15.105:0/1718355738 learned_addr learned my addr 172.21.15.105:0/1718355738 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2dc2b6a640 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee644e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.075+0000 7f2db37fe640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2da80a2fa0 0x7f2da80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.076+0000 7f2db37fe640 1 -- 172.21.15.105:0/3725046872 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2dac0ced60 con 0x7f2dbc104cf0 2024-03-26T17:56:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.076+0000 7f2dc2369640 1 -- 172.21.15.105:0/1718355738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 msgr2=0x560aee64ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.076+0000 7f2dc2369640 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee64ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.076+0000 7f2dc2369640 1 -- 172.21.15.105:0/1718355738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 msgr2=0x560aee644e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.076+0000 7f2dc2369640 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee644e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.076+0000 7f2dc2369640 1 -- 172.21.15.105:0/1718355738 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aee2bd3f0 con 0x560aee645420 2024-03-26T17:56:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.076+0000 7f2dc2b6a640 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee644e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.076+0000 7f2dc2b6a640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2da80a2fa0 0x7f2da80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.077+0000 7f2dc2369640 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee649f20 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f2db8036040 tx=0x7f2db8005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b429465a32f16ba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.077+0000 7f2dc2b6a640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2da80a2fa0 0x7f2da80a5460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x560aee645c80 tx=0x7f2db4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.077+0000 7f2db0bfe640 1 -- 172.21.15.105:0/1718355738 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8045020 con 0x560aee645420 2024-03-26T17:56:05.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.077+0000 7f2db0bfe640 1 -- 172.21.15.105:0/1718355738 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2db802cb40 con 0x560aee645420 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.078+0000 7f2db0bfe640 1 -- 172.21.15.105:0/1718355738 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8004920 con 0x560aee645420 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.078+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1718355738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 msgr2=0x560aee649f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.078+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee649f20 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f2db8036040 tx=0x7f2db8005e20 comp rx=0 tx=0).stop 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1718355738 shutdown_connections 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee64ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee649f20 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1718355738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee644e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1718355738 >> 172.21.15.105:0/1718355738 conn(0x560aee63ffa0 msgr2=0x560aee6403a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1718355738 shutdown_connections 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1718355738 wait complete. 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 -- start start 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.079+0000 7f2dc71ba740 1 Processor -- start 2024-03-26T17:56:05.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee64ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee64f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee6564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560aee64d530 con 0x560aee644a30 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560aee565310 con 0x560aee645420 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560aee64d6b0 con 0x560aee64a590 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc71ba740 10 client.0 _ll_register_callbacks cb 0x560aee5a0c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2369640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee64ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2369640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee64ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59700/0 (socket says 172.21.15.105:59700) 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2369640 1 -- 172.21.15.105:0/1076148899 learned_addr learned my addr 172.21.15.105:0/1076148899 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2b6a640 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee6564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee64f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2369640 1 -- 172.21.15.105:0/1076148899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 msgr2=0x560aee6564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2369640 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee6564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2369640 1 -- 172.21.15.105:0/1076148899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 msgr2=0x560aee64f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2369640 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee64f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.080+0000 7f2dc2369640 1 -- 172.21.15.105:0/1076148899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aee3877d0 con 0x560aee644a30 2024-03-26T17:56:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.081+0000 7f2dc2369640 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee64ef40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2db80096c0 tx=0x7f2db8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d5201167079914 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.081+0000 7f2da5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:05.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.081+0000 7f2da5ffb640 1 -- 172.21.15.105:0/1076148899 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8051060 con 0x560aee644a30 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.081+0000 7f2da5ffb640 1 -- 172.21.15.105:0/1076148899 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2db8003d20 con 0x560aee644a30 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.081+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1076148899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db802cb40 con 0x560aee644a30 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.081+0000 7f2da5ffb640 1 -- 172.21.15.105:0/1076148899 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2db8045070 con 0x560aee644a30 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.081+0000 7f2dc71ba740 10 client.21855 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.081+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1076148899 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560aee659830 con 0x560aee644a30 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2da5ffb640 1 -- 172.21.15.105:0/1076148899 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2db802cb40 con 0x560aee644a30 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2da5ffb640 1 -- 172.21.15.105:0/1076148899 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2db804a450 con 0x560aee644a30 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2dc71ba740 10 client.21855 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2dc71ba740 10 client.21855 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:56:05.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2dc71ba740 20 client.21855 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560aee65ab80 0x560aee65cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2dc71ba740 4 client.21855 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1076148899 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aee5b6eb0 con 0x560aee65ab80 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.082+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560aee65ab80 0x560aee65cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.083+0000 7f2da5ffb640 10 client.21855 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.083+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560aee65ab80 0x560aee65cf70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=336b95756a950c82 server_cookie=839ec01ac83369b6 in_seq=0 out_seq=0 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.083+0000 7f2da5ffb640 1 -- 172.21.15.105:0/1076148899 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560aee5b6eb0 con 0x560aee65ab80 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.083+0000 7f2da5ffb640 10 client.21855 handle_command_reply: tid=0 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.083+0000 7f2dc71ba740 1 client.21855 shutdown 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.083+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1076148899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 msgr2=0x560aee64ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.083+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee64ef40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2db80096c0 tx=0x7f2db8003820 comp rx=0 tx=0).stop 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1076148899 shutdown_connections 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560aee65ab80 0x560aee65cf70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee6564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee64f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/1076148899 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee64ef40 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1076148899 >> 172.21.15.105:0/1076148899 conn(0x560aee63ffa0 msgr2=0x560aee63f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2da5ffb640 0 client.21855 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:56:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1076148899 shutdown_connections 2024-03-26T17:56:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 1 -- 172.21.15.105:0/1076148899 wait complete. 2024-03-26T17:56:05.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.084+0000 7f2dc71ba740 20 client.21855 trim_cache size 0 max 16384 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.105+0000 7f2dc71ba740 1 Processor -- start 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc71ba740 1 -- start start 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee64da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee656480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee643910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1aafe0 con 0x560aee645420 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1acb20 con 0x560aee644a30 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1ab6b0 con 0x560aee64a590 2024-03-26T17:56:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc1b68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee656480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2369640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee64da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2b6a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee643910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2369640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee64da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45668/0 (socket says 172.21.15.105:45668) 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2369640 1 -- 172.21.15.105:0/53887545 learned_addr learned my addr 172.21.15.105:0/53887545 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2b6a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee643910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34540/0 (socket says 172.21.15.105:34540) 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee656480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59708/0 (socket says 172.21.15.105:59708) 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2369640 1 -- 172.21.15.105:0/53887545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 msgr2=0x560aee643910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2369640 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee643910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2369640 1 -- 172.21.15.105:0/53887545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 msgr2=0x560aee656480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2369640 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee656480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc2369640 1 -- 172.21.15.105:0/53887545 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aee64a3c0 con 0x560aee644a30 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.106+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee656480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.107+0000 7f2dc2369640 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee64da80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009900 tx=0x7f2db8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8ca88caeeafbe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.107+0000 7f2da7fff640 1 -- 172.21.15.105:0/53887545 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8005d50 con 0x560aee644a30 2024-03-26T17:56:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.107+0000 7f2da7fff640 1 -- 172.21.15.105:0/53887545 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2db8004490 con 0x560aee644a30 2024-03-26T17:56:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.107+0000 7f2da7fff640 1 -- 172.21.15.105:0/53887545 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80052d0 con 0x560aee644a30 2024-03-26T17:56:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 -- 172.21.15.105:0/53887545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 msgr2=0x560aee64da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee64da80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009900 tx=0x7f2db8005fd0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 -- 172.21.15.105:0/53887545 shutdown_connections 2024-03-26T17:56:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee643910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee645420 0x560aee656480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/53887545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee644a30 0x560aee64da80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 -- 172.21.15.105:0/53887545 >> 172.21.15.105:0/53887545 conn(0x560aee63ffa0 msgr2=0x560aee643200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 -- 172.21.15.105:0/53887545 shutdown_connections 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 -- 172.21.15.105:0/53887545 wait complete. 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 -- start start 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.108+0000 7f2dc71ba740 1 Processor -- start 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee653820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee653d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc71ba740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee654d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1ac450 con 0x560aee644a30 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1abd80 con 0x560aee645420 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc71ba740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2dbc1a8fc0 con 0x560aee64a590 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc71ba740 10 client.0 _ll_register_callbacks cb 0x7f2db4009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc2369640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee653820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc1b68640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee653d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc2369640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee653820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59716/0 (socket says 172.21.15.105:59716) 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc2369640 1 -- 172.21.15.105:0/2102719712 learned_addr learned my addr 172.21.15.105:0/2102719712 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc2b6a640 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee654d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc1b68640 1 -- 172.21.15.105:0/2102719712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 msgr2=0x560aee654d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee654d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc1b68640 1 -- 172.21.15.105:0/2102719712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 msgr2=0x560aee653820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee653820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc1b68640 1 -- 172.21.15.105:0/2102719712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aee655360 con 0x560aee645420 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc2369640 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee653820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2dc1b68640 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee653d60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0ccb90 tx=0x7f2dac097040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3065d57a19e1a54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.109+0000 7f2da5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2da5ffb640 1 -- 172.21.15.105:0/2102719712 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac09bb50 con 0x560aee645420 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2da5ffb640 1 -- 172.21.15.105:0/2102719712 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2dac097ac0 con 0x560aee645420 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 1 -- 172.21.15.105:0/2102719712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db8004490 con 0x560aee645420 2024-03-26T17:56:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2da5ffb640 1 -- 172.21.15.105:0/2102719712 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2dac09b450 con 0x560aee645420 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 10 client.31598 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 1 -- 172.21.15.105:0/2102719712 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560aee6566d0 con 0x560aee645420 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2da5ffb640 1 -- 172.21.15.105:0/2102719712 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2db8004490 con 0x560aee645420 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2da5ffb640 1 -- 172.21.15.105:0/2102719712 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2dac0ccce0 con 0x560aee645420 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 10 client.31598 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 10 client.31598 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 20 client.31598 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560aee6c1330 0x560aee6c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 4 client.31598 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.110+0000 7f2dc71ba740 1 -- 172.21.15.105:0/2102719712 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560aee5b1200 con 0x560aee6c1330 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2dc2369640 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560aee6c1330 0x560aee6c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2da5ffb640 10 client.31598 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2dc2369640 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560aee6c1330 0x560aee6c3750 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=846e6104c38fac2f server_cookie=f38af030ff164d1d in_seq=0 out_seq=0 2024-03-26T17:56:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2da5ffb640 1 -- 172.21.15.105:0/2102719712 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560aee5b1200 con 0x560aee6c1330 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2da5ffb640 10 client.31598 handle_command_reply: tid=0 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2dc71ba740 1 client.31598 shutdown 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2dc71ba740 1 -- 172.21.15.105:0/2102719712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 msgr2=0x560aee653d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee653d60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0ccb90 tx=0x7f2dac097040 comp rx=0 tx=0).stop 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2dc71ba740 1 -- 172.21.15.105:0/2102719712 shutdown_connections 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.111+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x560aee6c1330 0x560aee6c3750 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2da5ffb640 0 client.31598 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560aee64a590 0x560aee654d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560aee645420 0x560aee653d60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc71ba740 1 --2- 172.21.15.105:0/2102719712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560aee644a30 0x560aee653820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc71ba740 1 -- 172.21.15.105:0/2102719712 >> 172.21.15.105:0/2102719712 conn(0x560aee63ffa0 msgr2=0x560aee643a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc71ba740 1 -- 172.21.15.105:0/2102719712 shutdown_connections 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc71ba740 1 -- 172.21.15.105:0/2102719712 wait complete. 2024-03-26T17:56:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc71ba740 20 client.31598 trim_cache size 0 max 16384 2024-03-26T17:56:05.105 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 -- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2da80a2fa0 msgr2=0x7f2da80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2da80a2fa0 0x7f2da80a5460 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x560aee645c80 tx=0x7f2db4004a40 comp rx=0 tx=0).stop 2024-03-26T17:56:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 -- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc104cf0 msgr2=0x7f2dbc19a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc104cf0 0x7f2dbc19a340 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00b450 tx=0x7f2dac00b910 comp rx=0 tx=0).stop 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 -- 172.21.15.105:0/3725046872 shutdown_connections 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2da80a2fa0 0x7f2da80a5460 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc1056b0 0x7f2dbc1a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dbc104cf0 0x7f2dbc19a340 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 --2- 172.21.15.105:0/3725046872 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2dbc103b00 0x7f2dbc199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.112+0000 7f2dc45cd640 1 -- 172.21.15.105:0/3725046872 >> 172.21.15.105:0/3725046872 conn(0x7f2dbc0ff070 msgr2=0x7f2dbc108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.113+0000 7f2dc45cd640 1 -- 172.21.15.105:0/3725046872 shutdown_connections 2024-03-26T17:56:05.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.113+0000 7f2dc45cd640 1 -- 172.21.15.105:0/3725046872 wait complete. 2024-03-26T17:56:05.115 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:56:05.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:04 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2718734837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.313+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2268096402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 msgr2=0x7f8c68100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.313+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2268096402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 0x7f8c68100bf0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009900 tx=0x7f8c5802bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.313+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2268096402 shutdown_connections 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.313+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2268096402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c68102380 0x7f8c68108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.313+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2268096402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c681019c0 0x7f8c68101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.313+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2268096402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 0x7f8c68100bf0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.313+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2268096402 >> 172.21.15.105:0/2268096402 conn(0x7f8c680fbd60 msgr2=0x7f8c680fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.313+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2268096402 shutdown_connections 2024-03-26T17:56:05.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.314+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2268096402 wait complete. 2024-03-26T17:56:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.314+0000 7f8c6dd09640 1 Processor -- start 2024-03-26T17:56:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c6dd09640 1 -- start start 2024-03-26T17:56:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c6dd09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 0x7f8c68197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c6dd09640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c681019c0 0x7f8c681980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c6dd09640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c68102380 0x7f8c6819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c6dd09640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6810d480 con 0x7f8c681007d0 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c6dd09640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6810d300 con 0x7f8c68102380 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c6dd09640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6810d600 con 0x7f8c681019c0 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 0x7f8c68197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c66ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c681019c0 0x7f8c681980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 0x7f8c68197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59744/0 (socket says 172.21.15.105:59744) 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 -- 172.21.15.105:0/2868586418 learned_addr learned my addr 172.21.15.105:0/2868586418 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c67fff640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c68102380 0x7f8c6819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 -- 172.21.15.105:0/2868586418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c681019c0 msgr2=0x7f8c681980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c681019c0 0x7f8c681980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 -- 172.21.15.105:0/2868586418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c68102380 msgr2=0x7f8c6819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c68102380 0x7f8c6819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 -- 172.21.15.105:0/2868586418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6819f780 con 0x7f8c681007d0 2024-03-26T17:56:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c681019c0 0x7f8c681980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.315+0000 7f8c677fe640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 0x7f8c68197b60 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f8c58000c00 tx=0x7f8c58004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.316+0000 7f8c64ff9640 1 -- 172.21.15.105:0/2868586418 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58043020 con 0x7f8c681007d0 2024-03-26T17:56:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.316+0000 7f8c64ff9640 1 -- 172.21.15.105:0/2868586418 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c580043c0 con 0x7f8c681007d0 2024-03-26T17:56:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.316+0000 7f8c64ff9640 1 -- 172.21.15.105:0/2868586418 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58003b80 con 0x7f8c681007d0 2024-03-26T17:56:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.316+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2868586418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c681095d0 con 0x7f8c681007d0 2024-03-26T17:56:05.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.316+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2868586418 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c68109ad0 con 0x7f8c681007d0 2024-03-26T17:56:05.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.320+0000 7f8c64ff9640 1 -- 172.21.15.105:0/2868586418 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8c58046020 con 0x7f8c681007d0 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.321+0000 7f8c64ff9640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c500a2ff0 0x7f8c500a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.321+0000 7f8c709d2740 1 Processor -- start 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.321+0000 7f8c64ff9640 1 -- 172.21.15.105:0/2868586418 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c580f1620 con 0x7f8c681007d0 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.321+0000 7f8c709d2740 1 -- start start 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db117f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117f7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117fa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c709d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55db1174e970 con 0x55db117f7da0 2024-03-26T17:56:05.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c709d2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db116f6d70 con 0x55db117f2c30 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c709d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55db1174a920 con 0x55db117f2240 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db117f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db117f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34584/0 (socket says 172.21.15.105:34584) 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c677fe640 1 -- 172.21.15.105:0/3497336409 learned_addr learned my addr 172.21.15.105:0/3497336409 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c67fff640 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c500a2ff0 0x7f8c500a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.322+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c67fff640 1 -- 172.21.15.105:0/3497336409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 msgr2=0x55db117f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c67fff640 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db117f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c67fff640 1 -- 172.21.15.105:0/3497336409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 msgr2=0x55db117f7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c67fff640 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117f7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c67fff640 1 -- 172.21.15.105:0/3497336409 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db114623f0 con 0x55db117f7da0 2024-03-26T17:56:05.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:05.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c67fff640 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117fa230 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009f50 tx=0x7f8c600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a04bcc3f1ff2c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c500a2ff0 0x7f8c500a54b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55db117f3030 tx=0x7f8c5c010c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c4dffb640 1 -- 172.21.15.105:0/3497336409 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60004d20 con 0x55db117f7da0 2024-03-26T17:56:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c4dffb640 1 -- 172.21.15.105:0/3497336409 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c60004050 con 0x55db117f7da0 2024-03-26T17:56:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.323+0000 7f8c4dffb640 1 -- 172.21.15.105:0/3497336409 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60005280 con 0x55db117f7da0 2024-03-26T17:56:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 -- 172.21.15.105:0/3497336409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 msgr2=0x55db117fa230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117fa230 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8c60009f50 tx=0x7f8c600118d0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 -- 172.21.15.105:0/3497336409 shutdown_connections 2024-03-26T17:56:05.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117fa230 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 --2- 172.21.15.105:0/3497336409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db117f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 -- 172.21.15.105:0/3497336409 >> 172.21.15.105:0/3497336409 conn(0x55db117ed7b0 msgr2=0x55db117edbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 -- 172.21.15.105:0/3497336409 shutdown_connections 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.324+0000 7f8c709d2740 1 -- 172.21.15.105:0/3497336409 wait complete. 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 1 -- start start 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 1 Processor -- start 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f2240 0x55db117fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f7da0 0x55db11803d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55db117fad50 con 0x55db117f2240 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db1170a310 con 0x55db117f2c30 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55db117faed0 con 0x55db117f7da0 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f2240 0x55db117fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c67fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f7da0 0x55db11803d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c67fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f7da0 0x55db11803d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34592/0 (socket says 172.21.15.105:34592) 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f2240 0x55db117fc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59764/0 (socket says 172.21.15.105:59764) 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c67fff640 1 -- 172.21.15.105:0/4037375654 learned_addr learned my addr 172.21.15.105:0/4037375654 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.325+0000 7f8c709d2740 10 client.0 _ll_register_callbacks cb 0x55db11745c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c67fff640 1 -- 172.21.15.105:0/4037375654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 msgr2=0x55db117fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c67fff640 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c67fff640 1 -- 172.21.15.105:0/4037375654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f2240 msgr2=0x55db117fc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c67fff640 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f2240 0x55db117fc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c67fff640 1 -- 172.21.15.105:0/4037375654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db1152c7d0 con 0x55db117f7da0 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c67fff640 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f7da0 0x55db11803d00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8c60005c40 tx=0x7f8c60004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff2a7b227f8a9c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c46ffd640 1 -- 172.21.15.105:0/4037375654 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c60004610 con 0x55db117f7da0 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c709d2740 1 -- 172.21.15.105:0/4037375654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c60004050 con 0x55db117f7da0 2024-03-26T17:56:05.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c46ffd640 1 -- 172.21.15.105:0/4037375654 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c60025470 con 0x55db117f7da0 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c46ffd640 1 -- 172.21.15.105:0/4037375654 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8c60025670 con 0x55db117f7da0 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c709d2740 10 client.31585 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c709d2740 1 -- 172.21.15.105:0/4037375654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55db118070c0 con 0x55db117f7da0 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.326+0000 7f8c46ffd640 1 -- 172.21.15.105:0/4037375654 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c60004050 con 0x55db117f7da0 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c46ffd640 1 -- 172.21.15.105:0/4037375654 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8c60023a00 con 0x55db117f7da0 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c709d2740 10 client.31585 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c709d2740 10 client.31585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c709d2740 20 client.31585 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c709d2740 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55db11808530 0x55db1180a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c709d2740 4 client.31585 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c709d2740 1 -- 172.21.15.105:0/4037375654 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db1175beb0 con 0x55db11808530 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c677fe640 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55db11808530 0x55db1180a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c46ffd640 10 client.31585 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.327+0000 7f8c677fe640 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55db11808530 0x55db1180a9d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d8b2103dc1d492c server_cookie=2427eaeb60c0d150 in_seq=0 out_seq=0 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.328+0000 7f8c46ffd640 1 -- 172.21.15.105:0/4037375654 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55db1175beb0 con 0x55db11808530 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.328+0000 7f8c46ffd640 10 client.31585 handle_command_reply: tid=0 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.328+0000 7f8c709d2740 1 client.31585 shutdown 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.328+0000 7f8c709d2740 1 -- 172.21.15.105:0/4037375654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f7da0 msgr2=0x55db11803d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.328+0000 7f8c709d2740 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f7da0 0x55db11803d00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8c60005c40 tx=0x7f8c60004ce0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.328+0000 7f8c709d2740 1 -- 172.21.15.105:0/4037375654 shutdown_connections 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c709d2740 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f7da0 0x55db11803d00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c709d2740 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55db11808530 0x55db1180a9d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c709d2740 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db117fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c709d2740 1 --2- 172.21.15.105:0/4037375654 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f2240 0x55db117fc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c709d2740 1 -- 172.21.15.105:0/4037375654 >> 172.21.15.105:0/4037375654 conn(0x55db117ed7b0 msgr2=0x55db117e41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c46ffd640 0 client.31585 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:56:05.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c709d2740 1 -- 172.21.15.105:0/4037375654 shutdown_connections 2024-03-26T17:56:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c709d2740 1 -- 172.21.15.105:0/4037375654 wait complete. 2024-03-26T17:56:05.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.329+0000 7f8c709d2740 20 client.31585 trim_cache size 0 max 16384 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.348+0000 7f8c709d2740 1 Processor -- start 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.348+0000 7f8c709d2740 1 -- start start 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.348+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2240 0x55db117e3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.348+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2c30 0x55db11803cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117e3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2c30 0x55db11803cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2c30 0x55db11803cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34596/0 (socket says 172.21.15.105:34596) 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2240 0x55db117e3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2240 0x55db117e3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45720/0 (socket says 172.21.15.105:45720) 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 -- 172.21.15.105:0/3808278431 learned_addr learned my addr 172.21.15.105:0/3808278431 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c709d2740 1 -- 172.21.15.105:0/3808278431 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681a6c40 con 0x55db117f7da0 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c709d2740 1 -- 172.21.15.105:0/3808278431 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681a8780 con 0x55db117f2240 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c709d2740 1 -- 172.21.15.105:0/3808278431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681a7310 con 0x55db117f2c30 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c67fff640 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117e3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 -- 172.21.15.105:0/3808278431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2240 msgr2=0x55db117e3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2240 0x55db117e3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 -- 172.21.15.105:0/3808278431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 msgr2=0x55db117e3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117e3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 -- 172.21.15.105:0/3808278431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db117e0a10 con 0x55db117f2c30 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c67fff640 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.349+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2c30 0x55db11803cc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8c68199070 tx=0x7f8c5c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe26f8836872f758 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.350+0000 7f8c4d7fa640 1 -- 172.21.15.105:0/3808278431 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00eea0 con 0x55db117f2c30 2024-03-26T17:56:05.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.350+0000 7f8c4d7fa640 1 -- 172.21.15.105:0/3808278431 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c5c02d070 con 0x55db117f2c30 2024-03-26T17:56:05.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.350+0000 7f8c4d7fa640 1 -- 172.21.15.105:0/3808278431 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c028890 con 0x55db117f2c30 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.350+0000 7f8c709d2740 1 -- 172.21.15.105:0/3808278431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2c30 msgr2=0x55db11803cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.350+0000 7f8c709d2740 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2c30 0x55db11803cc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8c68199070 tx=0x7f8c5c015000 comp rx=0 tx=0).stop 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 -- 172.21.15.105:0/3808278431 shutdown_connections 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db117e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2c30 0x55db11803cc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 --2- 172.21.15.105:0/3808278431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2240 0x55db117e3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 -- 172.21.15.105:0/3808278431 >> 172.21.15.105:0/3808278431 conn(0x55db117ed7b0 msgr2=0x55db117fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 -- 172.21.15.105:0/3808278431 shutdown_connections 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 -- 172.21.15.105:0/3808278431 wait complete. 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 -- start start 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 Processor -- start 2024-03-26T17:56:05.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db11800a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db11800fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db1186c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681a80b0 con 0x55db117f7da0 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681a79e0 con 0x55db117f2c30 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681a4c20 con 0x55db117f2240 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.351+0000 7f8c709d2740 10 client.0 _ll_register_callbacks cb 0x7f8c60009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db11800a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db11800a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34602/0 (socket says 172.21.15.105:34602) 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 -- 172.21.15.105:0/1282766402 learned_addr learned my addr 172.21.15.105:0/1282766402 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db11800fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c67fff640 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db1186c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 -- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 msgr2=0x55db11800fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db11800fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 -- 172.21.15.105:0/1282766402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 msgr2=0x55db1186c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db1186c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 -- 172.21.15.105:0/1282766402 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db11801660 con 0x55db117f2240 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db11800fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c677fe640 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db11800a80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8c580ef450 tx=0x7f8c580f1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2890691ba32c4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c46ffd640 1 -- 172.21.15.105:0/1282766402 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c580be410 con 0x55db117f2240 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c709d2740 1 -- 172.21.15.105:0/1282766402 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c5c02d070 con 0x55db117f2240 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.352+0000 7f8c46ffd640 1 -- 172.21.15.105:0/1282766402 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8c58003d20 con 0x55db117f2240 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.353+0000 7f8c46ffd640 1 -- 172.21.15.105:0/1282766402 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8c580bdd10 con 0x55db117f2240 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.353+0000 7f8c709d2740 10 client.31591 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:05.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.353+0000 7f8c709d2740 1 -- 172.21.15.105:0/1282766402 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55db1186df80 con 0x55db117f2240 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.353+0000 7f8c46ffd640 1 -- 172.21.15.105:0/1282766402 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c5c02d070 con 0x55db117f2240 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.353+0000 7f8c46ffd640 1 -- 172.21.15.105:0/1282766402 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8c580c6290 con 0x55db117f2240 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.354+0000 7f8c709d2740 10 client.31591 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.354+0000 7f8c709d2740 10 client.31591 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.354+0000 7f8c709d2740 20 client.31591 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.354+0000 7f8c709d2740 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55db1186e8e0 0x55db11870d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.354+0000 7f8c709d2740 4 client.31591 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.354+0000 7f8c709d2740 1 -- 172.21.15.105:0/1282766402 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db117f7b10 con 0x55db1186e8e0 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.354+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55db1186e8e0 0x55db11870d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c66ffd640 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55db1186e8e0 0x55db11870d00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4f2f79b42f33e50 server_cookie=dfa33fbc2ed157e0 in_seq=0 out_seq=0 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c46ffd640 10 client.31591 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c46ffd640 1 -- 172.21.15.105:0/1282766402 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db117f7b10 con 0x55db1186e8e0 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c46ffd640 10 client.31591 handle_command_reply: tid=0 2024-03-26T17:56:05.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 client.31591 shutdown 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 -- 172.21.15.105:0/1282766402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 msgr2=0x55db11800a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db11800a80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8c580ef450 tx=0x7f8c580f1040 comp rx=0 tx=0).stop 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 -- 172.21.15.105:0/1282766402 shutdown_connections 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55db117f7da0 0x55db1186c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55db1186e8e0 0x55db11870d00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db117f2c30 0x55db11800fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c46ffd640 0 client.31591 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 --2- 172.21.15.105:0/1282766402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55db117f2240 0x55db11800a80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.355+0000 7f8c709d2740 1 -- 172.21.15.105:0/1282766402 >> 172.21.15.105:0/1282766402 conn(0x55db117ed7b0 msgr2=0x55db117e3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c709d2740 1 -- 172.21.15.105:0/1282766402 shutdown_connections 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c709d2740 1 -- 172.21.15.105:0/1282766402 wait complete. 2024-03-26T17:56:05.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c709d2740 20 client.31591 trim_cache size 0 max 16384 2024-03-26T17:56:05.348 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:05.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c500a2ff0 msgr2=0x7f8c500a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c500a2ff0 0x7f8c500a54b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55db117f3030 tx=0x7f8c5c010c60 comp rx=0 tx=0).stop 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 msgr2=0x7f8c68197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 0x7f8c68197b60 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f8c58000c00 tx=0x7f8c58004040 comp rx=0 tx=0).stop 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2868586418 shutdown_connections 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8c500a2ff0 0x7f8c500a54b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c68102380 0x7f8c6819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c681019c0 0x7f8c681980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 --2- 172.21.15.105:0/2868586418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c681007d0 0x7f8c68197b60 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.356+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2868586418 >> 172.21.15.105:0/2868586418 conn(0x7f8c680fbd60 msgr2=0x7f8c68104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.357+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2868586418 shutdown_connections 2024-03-26T17:56:05.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.357+0000 7f8c6dd09640 1 -- 172.21.15.105:0/2868586418 wait complete. 2024-03-26T17:56:05.350 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:04 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2718734837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:56:05.358 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:56:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 -- 172.21.15.105:0/3262661799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac105690 msgr2=0x7ffbac10bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 --2- 172.21.15.105:0/3262661799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac105690 0x7ffbac10bf20 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009ca0 tx=0x7ffba402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 -- 172.21.15.105:0/3262661799 shutdown_connections 2024-03-26T17:56:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 --2- 172.21.15.105:0/3262661799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac105690 0x7ffbac10bf20 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 --2- 172.21.15.105:0/3262661799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac104cd0 0x7ffbac105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 --2- 172.21.15.105:0/3262661799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbac103ae0 0x7ffbac103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 -- 172.21.15.105:0/3262661799 >> 172.21.15.105:0/3262661799 conn(0x7ffbac0ff070 msgr2=0x7ffbac101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 -- 172.21.15.105:0/3262661799 shutdown_connections 2024-03-26T17:56:05.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.515+0000 7ffbb3463640 1 -- 172.21.15.105:0/3262661799 wait complete. 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb3463640 1 Processor -- start 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb3463640 1 -- start start 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb3463640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac103ae0 0x7ffbac199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb3463640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbac104cd0 0x7ffbac19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb3463640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac105690 0x7ffbac1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb3463640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac10f5f0 con 0x7ffbac103ae0 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb3463640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac10f470 con 0x7ffbac104cd0 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb3463640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac10f770 con 0x7ffbac105690 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac103ae0 0x7ffbac199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac103ae0 0x7ffbac199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59814/0 (socket says 172.21.15.105:59814) 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb11ff640 1 -- 172.21.15.105:0/2820053642 learned_addr learned my addr 172.21.15.105:0/2820053642 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbac104cd0 0x7ffbac19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.516+0000 7ffbb1a00640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac105690 0x7ffbac1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb09fe640 1 -- 172.21.15.105:0/2820053642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac105690 msgr2=0x7ffbac1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac105690 0x7ffbac1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb09fe640 1 -- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac103ae0 msgr2=0x7ffbac199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac103ae0 0x7ffbac199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb09fe640 1 -- 172.21.15.105:0/2820053642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbac1a1900 con 0x7ffbac104cd0 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac103ae0 0x7ffbac199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb1a00640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac105690 0x7ffbac1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbac104cd0 0x7ffbac19a200 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00e9f0 tx=0x7ffb9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffba27fc640 1 -- 172.21.15.105:0/2820053642 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c00ccc0 con 0x7ffbac104cd0 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb3463640 1 -- 172.21.15.105:0/2820053642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbac1a1bf0 con 0x7ffbac104cd0 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffba27fc640 1 -- 172.21.15.105:0/2820053642 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb9c0108e0 con 0x7ffbac104cd0 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffba27fc640 1 -- 172.21.15.105:0/2820053642 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c018880 con 0x7ffbac104cd0 2024-03-26T17:56:05.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.517+0000 7ffbb3463640 1 -- 172.21.15.105:0/2820053642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbac1a2140 con 0x7ffbac104cd0 2024-03-26T17:56:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.520+0000 7ffba27fc640 1 -- 172.21.15.105:0/2820053642 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ffb9c010a90 con 0x7ffbac104cd0 2024-03-26T17:56:05.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.520+0000 7ffba27fc640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb980a2f20 0x7ffb980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffba27fc640 1 -- 172.21.15.105:0/2820053642 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ffb9c0cf720 con 0x7ffbac104cd0 2024-03-26T17:56:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb980a2f20 0x7ffb980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb6047740 1 Processor -- start 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb6047740 1 -- start start 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d1a30 0x5626bb6d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d7590 0x5626bb6d9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb6047740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5626bb636970 con 0x5626bb6d1a30 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb6047740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5626bb5ded70 con 0x5626bb6d2420 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb6047740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626bb632920 con 0x5626bb6d7590 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.521+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb980a2f20 0x7ffb980a53e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ffbac19ab90 tx=0x7ffba8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d1a30 0x5626bb6d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6d6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45748/0 (socket says 172.21.15.105:45748) 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d1a30 0x5626bb6d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59822/0 (socket says 172.21.15.105:59822) 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 -- 172.21.15.105:0/585369031 learned_addr learned my addr 172.21.15.105:0/585369031 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb1a00640 1 --2- 172.21.15.105:0/585369031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d7590 0x5626bb6d9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 -- 172.21.15.105:0/585369031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d7590 msgr2=0x5626bb6d9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/585369031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d7590 0x5626bb6d9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 -- 172.21.15.105:0/585369031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d1a30 msgr2=0x5626bb6d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/585369031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d1a30 0x5626bb6d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 -- 172.21.15.105:0/585369031 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626bb34a3f0 con 0x5626bb6d2420 2024-03-26T17:56:05.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/585369031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6d6f20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c09be50 tx=0x7ffb9c010e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2974f01452a84f9a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.522+0000 7ffb977fe640 1 -- 172.21.15.105:0/585369031 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c09ff60 con 0x5626bb6d2420 2024-03-26T17:56:05.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.523+0000 7ffb977fe640 1 -- 172.21.15.105:0/585369031 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb9c0a4820 con 0x5626bb6d2420 2024-03-26T17:56:05.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.523+0000 7ffb977fe640 1 -- 172.21.15.105:0/585369031 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0badc0 con 0x5626bb6d2420 2024-03-26T17:56:05.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.523+0000 7ffbb6047740 1 -- 172.21.15.105:0/585369031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 msgr2=0x5626bb6d6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.523+0000 7ffbb6047740 1 --2- 172.21.15.105:0/585369031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6d6f20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c09be50 tx=0x7ffb9c010e00 comp rx=0 tx=0).stop 2024-03-26T17:56:05.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 -- 172.21.15.105:0/585369031 shutdown_connections 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 --2- 172.21.15.105:0/585369031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d7590 0x5626bb6d9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 --2- 172.21.15.105:0/585369031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6d6f20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 --2- 172.21.15.105:0/585369031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d1a30 0x5626bb6d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 -- 172.21.15.105:0/585369031 >> 172.21.15.105:0/585369031 conn(0x5626bb6ccfa0 msgr2=0x5626bb6cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 -- 172.21.15.105:0/585369031 shutdown_connections 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 -- 172.21.15.105:0/585369031 wait complete. 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 -- start start 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 Processor -- start 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6ed4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.524+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 0x5626bb6f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb6047740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5626bb6eb490 con 0x5626bb6d7590 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb6047740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5626bb5f2310 con 0x5626bb6d2420 2024-03-26T17:56:05.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb6047740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5626bb6eb610 con 0x5626bb6d1a30 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6ecf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34636/0 (socket says 172.21.15.105:34636) 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 -- 172.21.15.105:0/3684429474 learned_addr learned my addr 172.21.15.105:0/3684429474 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb6047740 10 client.0 _ll_register_callbacks cb 0x5626bb62dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb1a00640 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 0x5626bb6f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6ed4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 -- 172.21.15.105:0/3684429474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 msgr2=0x5626bb6ed4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6ed4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 -- 172.21.15.105:0/3684429474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 msgr2=0x5626bb6f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 0x5626bb6f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 -- 172.21.15.105:0/3684429474 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626bb4147d0 con 0x5626bb6d1a30 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6ecf60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffba8003710 tx=0x7ffba8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f4e2ca2ad70c24b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffb94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffb94ff9640 1 -- 172.21.15.105:0/3684429474 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba800f8b0 con 0x5626bb6d1a30 2024-03-26T17:56:05.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffbb6047740 1 -- 172.21.15.105:0/3684429474 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9c0a4820 con 0x5626bb6d1a30 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.525+0000 7ffb94ff9640 1 -- 172.21.15.105:0/3684429474 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ffba800fa50 con 0x5626bb6d1a30 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffb94ff9640 1 -- 172.21.15.105:0/3684429474 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffba800fc90 con 0x5626bb6d1a30 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb6047740 10 client.31603 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb6047740 1 -- 172.21.15.105:0/3684429474 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5626bb6f7910 con 0x5626bb6d1a30 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffb94ff9640 1 -- 172.21.15.105:0/3684429474 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb9c0a4820 con 0x5626bb6d1a30 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffb94ff9640 1 -- 172.21.15.105:0/3684429474 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ffba801a050 con 0x5626bb6d1a30 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb6047740 10 client.31603 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb6047740 10 client.31603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb6047740 20 client.31603 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5626bb6f8f30 0x5626bb6fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb6047740 4 client.31603 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb6047740 1 -- 172.21.15.105:0/3684429474 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626bb643eb0 con 0x5626bb6f8f30 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.526+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5626bb6f8f30 0x5626bb6fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.527+0000 7ffb94ff9640 10 client.31603 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:56:05.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.527+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5626bb6f8f30 0x5626bb6fb3d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d5f3f425ae80290 server_cookie=8966d7ea5839801f in_seq=0 out_seq=0 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.528+0000 7ffb94ff9640 1 -- 172.21.15.105:0/3684429474 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5626bb643eb0 con 0x5626bb6f8f30 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.528+0000 7ffb94ff9640 10 client.31603 handle_command_reply: tid=0 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.528+0000 7ffbb6047740 1 client.31603 shutdown 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 -- 172.21.15.105:0/3684429474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 msgr2=0x5626bb6ecf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6ecf60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffba8003710 tx=0x7ffba8004730 comp rx=0 tx=0).stop 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 -- 172.21.15.105:0/3684429474 shutdown_connections 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5626bb6f8f30 0x5626bb6fb3d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 0x5626bb6f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d2420 0x5626bb6ed4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3684429474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6ecf60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 -- 172.21.15.105:0/3684429474 >> 172.21.15.105:0/3684429474 conn(0x5626bb6ccfa0 msgr2=0x5626bb6cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffb94ff9640 0 client.31603 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:56:05.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 -- 172.21.15.105:0/3684429474 shutdown_connections 2024-03-26T17:56:05.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 1 -- 172.21.15.105:0/3684429474 wait complete. 2024-03-26T17:56:05.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.529+0000 7ffbb6047740 20 client.31603 trim_cache size 0 max 16384 2024-03-26T17:56:05.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.553+0000 7ffbb6047740 1 Processor -- start 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.553+0000 7ffbb6047740 1 -- start start 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.553+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d1a30 0x5626bb6ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.553+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d2420 0x5626bb6f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 0x5626bb6cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb6047740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1a2870 con 0x5626bb6d7590 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb6047740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1a43b0 con 0x5626bb6d1a30 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb6047740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1a2f40 con 0x5626bb6d2420 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb09fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d2420 0x5626bb6f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d1a30 0x5626bb6ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d1a30 0x5626bb6ed280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45766/0 (socket says 172.21.15.105:45766) 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb09fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d2420 0x5626bb6f44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34640/0 (socket says 172.21.15.105:34640) 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 -- 172.21.15.105:0/3306761546 learned_addr learned my addr 172.21.15.105:0/3306761546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb1a00640 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 0x5626bb6cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 -- 172.21.15.105:0/3306761546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d2420 msgr2=0x5626bb6f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d2420 0x5626bb6f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 -- 172.21.15.105:0/3306761546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 msgr2=0x5626bb6cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 0x5626bb6cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 -- 172.21.15.105:0/3306761546 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626bb6d6c90 con 0x5626bb6d1a30 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d2420 0x5626bb6f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.554+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d1a30 0x5626bb6ed280 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009f40 tx=0x7ffba8004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8ab4666868c9e56 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffb96ffd640 1 -- 172.21.15.105:0/3306761546 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8037030 con 0x5626bb6d1a30 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffb96ffd640 1 -- 172.21.15.105:0/3306761546 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba800fe30 con 0x5626bb6d1a30 2024-03-26T17:56:05.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffb96ffd640 1 -- 172.21.15.105:0/3306761546 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8004890 con 0x5626bb6d1a30 2024-03-26T17:56:05.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffbb6047740 1 -- 172.21.15.105:0/3306761546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d1a30 msgr2=0x5626bb6ed280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d1a30 0x5626bb6ed280 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009f40 tx=0x7ffba8004680 comp rx=0 tx=0).stop 2024-03-26T17:56:05.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffbb6047740 1 -- 172.21.15.105:0/3306761546 shutdown_connections 2024-03-26T17:56:05.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d7590 0x5626bb6cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d2420 0x5626bb6f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffbb6047740 1 --2- 172.21.15.105:0/3306761546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d1a30 0x5626bb6ed280 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.555+0000 7ffbb6047740 1 -- 172.21.15.105:0/3306761546 >> 172.21.15.105:0/3306761546 conn(0x5626bb6ccfa0 msgr2=0x5626bb6ecb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 -- 172.21.15.105:0/3306761546 shutdown_connections 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 -- 172.21.15.105:0/3306761546 wait complete. 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 -- start start 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 Processor -- start 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d2420 0x5626bb6f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d7590 0x5626bb6f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1a3ce0 con 0x5626bb6d2420 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac1a3610 con 0x5626bb6d7590 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.556+0000 7ffbb6047740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac072030 con 0x5626bb6d1a30 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb6047740 10 client.0 _ll_register_callbacks cb 0x7ffba80035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34656/0 (socket says 172.21.15.105:34656) 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb09fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d2420 0x5626bb6f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb1a00640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d7590 0x5626bb6f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 -- 172.21.15.105:0/114667800 learned_addr learned my addr 172.21.15.105:0/114667800 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 -- 172.21.15.105:0/114667800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d7590 msgr2=0x5626bb6f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d7590 0x5626bb6f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 -- 172.21.15.105:0/114667800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d2420 msgr2=0x5626bb6f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d2420 0x5626bb6f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 -- 172.21.15.105:0/114667800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626bb6f3060 con 0x5626bb6d1a30 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d2420 0x5626bb6f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb1a00640 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d7590 0x5626bb6f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffbb11ff640 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6f1520 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffba80284f0 tx=0x7ffba8028d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc1c4d2d7bd39eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffb94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.557+0000 7ffb94ff9640 1 -- 172.21.15.105:0/114667800 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba8041070 con 0x5626bb6d1a30 2024-03-26T17:56:05.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.558+0000 7ffbb6047740 1 -- 172.21.15.105:0/114667800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffba800fe30 con 0x5626bb6d1a30 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.558+0000 7ffb94ff9640 1 -- 172.21.15.105:0/114667800 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7ffba802e030 con 0x5626bb6d1a30 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.558+0000 7ffb94ff9640 1 -- 172.21.15.105:0/114667800 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ffba804d050 con 0x5626bb6d1a30 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.558+0000 7ffbb6047740 10 client.31609 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.558+0000 7ffbb6047740 1 -- 172.21.15.105:0/114667800 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5626bb6f4360 con 0x5626bb6d1a30 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffb94ff9640 1 -- 172.21.15.105:0/114667800 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffba800fe30 con 0x5626bb6d1a30 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffb94ff9640 1 -- 172.21.15.105:0/114667800 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7ffba801a040 con 0x5626bb6d1a30 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffbb6047740 10 client.31609 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffbb6047740 10 client.31609 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffbb6047740 20 client.31609 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffbb6047740 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5626bb75f500 0x5626bb761920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffbb6047740 4 client.31609 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffbb6047740 1 -- 172.21.15.105:0/114667800 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626bb63e200 con 0x5626bb75f500 2024-03-26T17:56:05.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.559+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5626bb75f500 0x5626bb761920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.560+0000 7ffbb09fe640 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5626bb75f500 0x5626bb761920 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=356fa53d93d4f663 server_cookie=63d966d51c78673c in_seq=0 out_seq=0 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.560+0000 7ffb94ff9640 10 client.31609 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffb94ff9640 1 -- 172.21.15.105:0/114667800 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626bb63e200 con 0x5626bb75f500 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffb94ff9640 10 client.31609 handle_command_reply: tid=0 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 client.31609 shutdown 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 -- 172.21.15.105:0/114667800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 msgr2=0x5626bb6f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6f1520 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffba80284f0 tx=0x7ffba8028d40 comp rx=0 tx=0).stop 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 -- 172.21.15.105:0/114667800 shutdown_connections 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5626bb75f500 0x5626bb761920 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5626bb6d7590 0x5626bb6f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5626bb6d2420 0x5626bb6f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 --2- 172.21.15.105:0/114667800 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5626bb6d1a30 0x5626bb6f1520 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 -- 172.21.15.105:0/114667800 >> 172.21.15.105:0/114667800 conn(0x5626bb6ccfa0 msgr2=0x5626bb6d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffb94ff9640 0 client.31609 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:56:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 -- 172.21.15.105:0/114667800 shutdown_connections 2024-03-26T17:56:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 1 -- 172.21.15.105:0/114667800 wait complete. 2024-03-26T17:56:05.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.561+0000 7ffbb6047740 20 client.31609 trim_cache size 0 max 16384 2024-03-26T17:56:05.551 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 -- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb980a2f20 msgr2=0x7ffb980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb980a2f20 0x7ffb980a53e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ffbac19ab90 tx=0x7ffba8005f90 comp rx=0 tx=0).stop 2024-03-26T17:56:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 -- 172.21.15.105:0/2820053642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbac104cd0 msgr2=0x7ffbac19a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbac104cd0 0x7ffbac19a200 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00e9f0 tx=0x7ffb9c00eeb0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 -- 172.21.15.105:0/2820053642 shutdown_connections 2024-03-26T17:56:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ffb980a2f20 0x7ffb980a53e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbac105690 0x7ffbac1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffbac104cd0 0x7ffbac19a200 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 --2- 172.21.15.105:0/2820053642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbac103ae0 0x7ffbac199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 -- 172.21.15.105:0/2820053642 >> 172.21.15.105:0/2820053642 conn(0x7ffbac0ff070 msgr2=0x7ffbac108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 -- 172.21.15.105:0/2820053642 shutdown_connections 2024-03-26T17:56:05.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.562+0000 7ffbb3463640 1 -- 172.21.15.105:0/2820053642 wait complete. 2024-03-26T17:56:05.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 37912 7.08 MB/sec execute 168 sec latency 130.562 ms 2024-03-26T17:56:05.563 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2815986860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 msgr2=0x7fb4e8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2815986860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 0x7fb4e8101cf0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009900 tx=0x7fb4d802bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2815986860 shutdown_connections 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2815986860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4e8103480 0x7fb4e8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2815986860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4e8102ac0 0x7fb4e8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2815986860 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 0x7fb4e8101cf0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2815986860 >> 172.21.15.105:0/2815986860 conn(0x7fb4e80fce60 msgr2=0x7fb4e80ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2815986860 shutdown_connections 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.717+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2815986860 wait complete. 2024-03-26T17:56:05.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.718+0000 7fb4ed8dc640 1 Processor -- start 2024-03-26T17:56:05.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.718+0000 7fb4ed8dc640 1 -- start start 2024-03-26T17:56:05.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.718+0000 7fb4ed8dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 0x7fb4e8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.718+0000 7fb4ed8dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4e8102ac0 0x7fb4e81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.718+0000 7fb4ed8dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4e8103480 0x7fb4e819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.718+0000 7fb4ed8dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810d3f0 con 0x7fb4e81018d0 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.718+0000 7fb4ed8dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810d270 con 0x7fb4e8102ac0 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.718+0000 7fb4ed8dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810d570 con 0x7fb4e8103480 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 0x7fb4e8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 0x7fb4e8197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59878/0 (socket says 172.21.15.105:59878) 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e6ffd640 1 -- 172.21.15.105:0/2868225082 learned_addr learned my addr 172.21.15.105:0/2868225082 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4e8103480 0x7fb4e819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4e8102ac0 0x7fb4e81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e77fe640 1 -- 172.21.15.105:0/2868225082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4e8102ac0 msgr2=0x7fb4e81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4e8102ac0 0x7fb4e81980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e77fe640 1 -- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 msgr2=0x7fb4e8197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 0x7fb4e8197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e77fe640 1 -- 172.21.15.105:0/2868225082 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e819f7a0 con 0x7fb4e8103480 2024-03-26T17:56:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e6ffd640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 0x7fb4e8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.719+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4e8103480 0x7fb4e819f130 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009fd0 tx=0x7fb4e000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.720+0000 7fb4ec8da640 1 -- 172.21.15.105:0/2868225082 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000ce20 con 0x7fb4e8103480 2024-03-26T17:56:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.720+0000 7fb4ec8da640 1 -- 172.21.15.105:0/2868225082 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4e0005020 con 0x7fb4e8103480 2024-03-26T17:56:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.720+0000 7fb4ec8da640 1 -- 172.21.15.105:0/2868225082 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0010690 con 0x7fb4e8103480 2024-03-26T17:56:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.720+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2868225082 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e819fa90 con 0x7fb4e8103480 2024-03-26T17:56:05.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.720+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2868225082 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e819ff90 con 0x7fb4e8103480 2024-03-26T17:56:05.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.723+0000 7fb4ec8da640 1 -- 172.21.15.105:0/2868225082 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb4e00051d0 con 0x7fb4e8103480 2024-03-26T17:56:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4f05a5740 1 Processor -- start 2024-03-26T17:56:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4f05a5740 1 -- start start 2024-03-26T17:56:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4ec8da640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4d00a3070 0x7fb4d00a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfa7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 0x5652bcfac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfaf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5652bcf03970 con 0x5652bcface00 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5652bceabd70 con 0x5652bcfa7850 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5652bceff920 con 0x5652bcfa8240 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfa7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4d00a3070 0x7fb4d00a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfaf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfa7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45810/0 (socket says 172.21.15.105:45810) 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4e6ffd640 1 -- 172.21.15.105:0/1324302129 learned_addr learned my addr 172.21.15.105:0/1324302129 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfaf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59894/0 (socket says 172.21.15.105:59894) 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.724+0000 7fb4ec8da640 1 -- 172.21.15.105:0/2868225082 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb4e00cfa00 con 0x7fb4e8103480 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4e77fe640 1 -- 172.21.15.105:0/1324302129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 msgr2=0x5652bcfac790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 0x5652bcfac790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4e77fe640 1 -- 172.21.15.105:0/1324302129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 msgr2=0x5652bcfa7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfa7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4e77fe640 1 -- 172.21.15.105:0/1324302129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652bcc173f0 con 0x5652bcface00 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4e6ffd640 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfa7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfaf290 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00cd510 tx=0x7fb4e009ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4011159f44f801 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4d00a3070 0x7fb4d00a5530 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009710 tx=0x7fb4dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4cdffb640 1 -- 172.21.15.105:0/1324302129 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00bae60 con 0x5652bcface00 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4cdffb640 1 -- 172.21.15.105:0/1324302129 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4e009c0f0 con 0x5652bcface00 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.725+0000 7fb4cdffb640 1 -- 172.21.15.105:0/1324302129 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0098760 con 0x5652bcface00 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1324302129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 msgr2=0x5652bcfaf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfaf290 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb4e00cd510 tx=0x7fb4e009ba50 comp rx=0 tx=0).stop 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1324302129 shutdown_connections 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfaf290 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 0x5652bcfac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1324302129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfa7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.714 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1324302129 >> 172.21.15.105:0/1324302129 conn(0x5652bcf99f20 msgr2=0x5652bcf9a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1324302129 shutdown_connections 2024-03-26T17:56:05.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.726+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1324302129 wait complete. 2024-03-26T17:56:05.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4f05a5740 1 -- start start 2024-03-26T17:56:05.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4f05a5740 1 Processor -- start 2024-03-26T17:56:05.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa7850 0x5652bcfb1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcface00 0x5652bcfb8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5652bcfafd50 con 0x5652bcfa8240 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa7850 0x5652bcfb1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa7850 0x5652bcfb1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34686/0 (socket says 172.21.15.105:34686) 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4e6ffd640 1 -- 172.21.15.105:0/942126366 learned_addr learned my addr 172.21.15.105:0/942126366 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5652bcebf310 con 0x5652bcface00 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.727+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5652bcfafed0 con 0x5652bcfa7850 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcface00 0x5652bcfb8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4f05a5740 10 client.0 _ll_register_callbacks cb 0x5652bcefac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e6ffd640 1 -- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcface00 msgr2=0x5652bcfb8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e6ffd640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcface00 0x5652bcfb8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e6ffd640 1 -- 172.21.15.105:0/942126366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 msgr2=0x5652bcfb1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e6ffd640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e6ffd640 1 -- 172.21.15.105:0/942126366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652bcce17d0 con 0x5652bcfa7850 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcface00 0x5652bcfb8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4e6ffd640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa7850 0x5652bcfb1800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009230 tx=0x7fb4d8005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab50d225e0fd71d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.728+0000 7fb4c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/942126366 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8045070 con 0x5652bcfa7850 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/942126366 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb4d80046c0 con 0x5652bcfa7850 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e009c0f0 con 0x5652bcfa7850 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/942126366 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb4d8051050 con 0x5652bcfa7850 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 10 client.31618 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:05.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5652bcfcd130 con 0x5652bcfa7850 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/942126366 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4d802cc90 con 0x5652bcfa7850 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/942126366 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb4d80039c0 con 0x5652bcfa7850 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 10 client.31618 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 10 client.31618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 20 client.31618 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5652bcfce7c0 0x5652bcfd0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 4 client.31618 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.729+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb4e009c0f0 con 0x5652bcfce7c0 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.730+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5652bcfce7c0 0x5652bcfd0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.730+0000 7fb4c6ffd640 10 client.31618 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:05.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.730+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5652bcfce7c0 0x5652bcfd0bf0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a7123fa0a52016 server_cookie=31fe5f1b9e334707 in_seq=0 out_seq=0 2024-03-26T17:56:05.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.734+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/942126366 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fb4e009c0f0 con 0x5652bcfce7c0 2024-03-26T17:56:05.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.734+0000 7fb4c6ffd640 10 client.31618 handle_command_reply: tid=0 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.734+0000 7fb4f05a5740 1 client.31618 shutdown 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa7850 msgr2=0x5652bcfb1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa7850 0x5652bcfb1800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009230 tx=0x7fb4d8005270 comp rx=0 tx=0).stop 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 shutdown_connections 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5652bcfce7c0 0x5652bcfd0bf0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcface00 0x5652bcfb8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4c6ffd640 0 client.31618 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/942126366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa7850 0x5652bcfb1800 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 >> 172.21.15.105:0/942126366 conn(0x5652bcf99f20 msgr2=0x5652bcfa6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 shutdown_connections 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 1 -- 172.21.15.105:0/942126366 wait complete. 2024-03-26T17:56:05.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.735+0000 7fb4f05a5740 20 client.31618 trim_cache size 0 max 16384 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4f05a5740 1 Processor -- start 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4f05a5740 1 -- start start 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcf99040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 0x5652bcfb8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfb0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8073dc0 con 0x5652bcface00 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8075900 con 0x5652bcfa7850 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8074490 con 0x5652bcfa8240 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfb0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfb0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59904/0 (socket says 172.21.15.105:59904) 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4e77fe640 1 -- 172.21.15.105:0/135398482 learned_addr learned my addr 172.21.15.105:0/135398482 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4e6ffd640 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcf99040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.754+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 0x5652bcfb8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4e77fe640 1 -- 172.21.15.105:0/135398482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 msgr2=0x5652bcfb8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 0x5652bcfb8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4e77fe640 1 -- 172.21.15.105:0/135398482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 msgr2=0x5652bcf99040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcf99040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4e77fe640 1 -- 172.21.15.105:0/135398482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652bcf9a990 con 0x5652bcface00 2024-03-26T17:56:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4e67fc640 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 0x5652bcfb8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfb0e20 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000c760 tx=0x7fb4e009bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77adcdd48892ffee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4cd7fa640 1 -- 172.21.15.105:0/135398482 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00bae60 con 0x5652bcface00 2024-03-26T17:56:05.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4cd7fa640 1 -- 172.21.15.105:0/135398482 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4e009a190 con 0x5652bcface00 2024-03-26T17:56:05.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.755+0000 7fb4cd7fa640 1 -- 172.21.15.105:0/135398482 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e00bc320 con 0x5652bcface00 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 -- 172.21.15.105:0/135398482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 msgr2=0x5652bcfb0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfb0e20 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000c760 tx=0x7fb4e009bb80 comp rx=0 tx=0).stop 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 -- 172.21.15.105:0/135398482 shutdown_connections 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcface00 0x5652bcfb0e20 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcfa8240 0x5652bcfb8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/135398482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcf99040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 -- 172.21.15.105:0/135398482 >> 172.21.15.105:0/135398482 conn(0x5652bcf99f20 msgr2=0x5652bcf98930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 -- 172.21.15.105:0/135398482 shutdown_connections 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 -- 172.21.15.105:0/135398482 wait complete. 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.756+0000 7fb4f05a5740 1 -- start start 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4f05a5740 1 Processor -- start 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfb5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4f05a5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcface00 0x5652bcfb7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8075230 con 0x5652bcfa8240 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8074b60 con 0x5652bcfa7850 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4f05a5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8071da0 con 0x5652bcface00 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4f05a5740 10 client.0 _ll_register_callbacks cb 0x7fb4dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4e67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4e6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfb5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4e77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcface00 0x5652bcfb7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4e77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcface00 0x5652bcfb7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34704/0 (socket says 172.21.15.105:34704) 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4e77fe640 1 -- 172.21.15.105:0/1848388752 learned_addr learned my addr 172.21.15.105:0/1848388752 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.757+0000 7fb4e67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59916/0 (socket says 172.21.15.105:59916) 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4e77fe640 1 -- 172.21.15.105:0/1848388752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 msgr2=0x5652bcfb5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfb5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4e77fe640 1 -- 172.21.15.105:0/1848388752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 msgr2=0x5652bcfb63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4e77fe640 1 -- 172.21.15.105:0/1848388752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652bcfb7950 con 0x5652bcface00 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4e77fe640 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcface00 0x5652bcfb7360 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0098ac0 tx=0x7fb4e009c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=325053b2a60d4001 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/1848388752 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0083070 con 0x5652bcface00 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1848388752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4e009a190 con 0x5652bcface00 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.758+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/1848388752 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fb4e0008430 con 0x5652bcface00 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/1848388752 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb4e00bae60 con 0x5652bcface00 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4f05a5740 10 client.31624 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:05.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1848388752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5652bcfb8c50 con 0x5652bcface00 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/1848388752 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4e009a190 con 0x5652bcface00 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/1848388752 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fb4e0004b40 con 0x5652bcface00 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4f05a5740 10 client.31624 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4f05a5740 10 client.31624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4f05a5740 20 client.31624 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5652bd034b40 0x5652bd036f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4f05a5740 4 client.31624 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1848388752 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652bcf10eb0 con 0x5652bd034b40 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.759+0000 7fb4e6ffd640 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5652bd034b40 0x5652bd036f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.760+0000 7fb4e6ffd640 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5652bd034b40 0x5652bd036f30 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb9cbf217137049 server_cookie=1babc9abe7dbadc1 in_seq=0 out_seq=0 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.760+0000 7fb4c6ffd640 10 client.31624 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4c6ffd640 1 -- 172.21.15.105:0/1848388752 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652bcf10eb0 con 0x5652bd034b40 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4c6ffd640 10 client.31624 handle_command_reply: tid=0 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 client.31624 shutdown 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1848388752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcface00 msgr2=0x5652bcfb7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcface00 0x5652bcfb7360 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0098ac0 tx=0x7fb4e009c8b0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1848388752 shutdown_connections 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5652bd034b40 0x5652bd036f30 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5652bcface00 0x5652bcfb7360 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5652bcfa8240 0x5652bcfb63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 --2- 172.21.15.105:0/1848388752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5652bcfa7850 0x5652bcfb5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.761+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1848388752 >> 172.21.15.105:0/1848388752 conn(0x5652bcf99f20 msgr2=0x5652bcf98930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4c6ffd640 0 client.31624 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1848388752 shutdown_connections 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4f05a5740 1 -- 172.21.15.105:0/1848388752 wait complete. 2024-03-26T17:56:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4f05a5740 20 client.31624 trim_cache size 0 max 16384 2024-03-26T17:56:05.751 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4d00a3070 msgr2=0x7fb4d00a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4d00a3070 0x7fb4d00a5530 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009710 tx=0x7fb4dc005f90 comp rx=0 tx=0).stop 2024-03-26T17:56:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2868225082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4e8103480 msgr2=0x7fb4e819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4e8103480 0x7fb4e819f130 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009fd0 tx=0x7fb4e000ef10 comp rx=0 tx=0).stop 2024-03-26T17:56:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2868225082 shutdown_connections 2024-03-26T17:56:05.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4d00a3070 0x7fb4d00a5530 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4e8103480 0x7fb4e819f130 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4e8102ac0 0x7fb4e81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 --2- 172.21.15.105:0/2868225082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4e81018d0 0x7fb4e8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:05.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.762+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2868225082 >> 172.21.15.105:0/2868225082 conn(0x7fb4e80fce60 msgr2=0x7fb4e8105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:05.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.763+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2868225082 shutdown_connections 2024-03-26T17:56:05.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:05.763+0000 7fb4ed8dc640 1 -- 172.21.15.105:0/2868225082 wait complete. 2024-03-26T17:56:06.135 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:05 smithi105 ceph-mon[32280]: pgmap v881: 97 pgs: 97 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-03-26T17:56:06.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:05 smithi135 ceph-mon[35636]: pgmap v881: 97 pgs: 97 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-03-26T17:56:06.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:05 smithi144 ceph-mon[36095]: pgmap v881: 97 pgs: 97 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 3.3 MiB/s wr, 242 op/s 2024-03-26T17:56:06.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 38048 7.08 MB/sec execute 169 sec latency 107.909 ms 2024-03-26T17:56:06.763 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:56:06.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:56:06.764 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:56:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 -- 172.21.15.105:0/299610653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70f41019e0 msgr2=0x7f70f4101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/299610653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70f41019e0 0x7f70f4101e60 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009900 tx=0x7f70e802bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 -- 172.21.15.105:0/299610653 shutdown_connections 2024-03-26T17:56:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/299610653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f41023a0 0x7f70f4108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/299610653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70f41019e0 0x7f70f4101e60 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/299610653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f41007f0 0x7f70f4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 -- 172.21.15.105:0/299610653 >> 172.21.15.105:0/299610653 conn(0x7f70f40fbd80 msgr2=0x7f70f40fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 -- 172.21.15.105:0/299610653 shutdown_connections 2024-03-26T17:56:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.840+0000 7f70f9b0c640 1 -- 172.21.15.105:0/299610653 wait complete. 2024-03-26T17:56:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.841+0000 7f70f9b0c640 1 Processor -- start 2024-03-26T17:56:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.841+0000 7f70f9b0c640 1 -- start start 2024-03-26T17:56:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.841+0000 7f70f9b0c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70f41007f0 0x7f70f406abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.841+0000 7f70f9b0c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f41019e0 0x7f70f419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.841+0000 7f70f9b0c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f41023a0 0x7f70f419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.841+0000 7f70f9b0c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70f410c320 con 0x7f70f41007f0 2024-03-26T17:56:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.841+0000 7f70f9b0c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70f410c1a0 con 0x7f70f41019e0 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.841+0000 7f70f9b0c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70f410c4a0 con 0x7f70f41023a0 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f41023a0 0x7f70f419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70f41007f0 0x7f70f406abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f41023a0 0x7f70f419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34720/0 (socket says 172.21.15.105:34720) 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f41019e0 0x7f70f419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 -- 172.21.15.105:0/1034912443 learned_addr learned my addr 172.21.15.105:0/1034912443 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 -- 172.21.15.105:0/1034912443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f41019e0 msgr2=0x7f70f419c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f41019e0 0x7f70f419c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 -- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70f41007f0 msgr2=0x7f70f406abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70f41007f0 0x7f70f406abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 -- 172.21.15.105:0/1034912443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f419f6c0 con 0x7f70f41023a0 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f41019e0 0x7f70f419c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f3fff640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f41023a0 0x7f70f419edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00eb10 tx=0x7f70ec00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.842+0000 7f70f0ff9640 1 -- 172.21.15.105:0/1034912443 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec004b20 con 0x7f70f41023a0 2024-03-26T17:56:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.843+0000 7f70f0ff9640 1 -- 172.21.15.105:0/1034912443 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70ec004cc0 con 0x7f70f41023a0 2024-03-26T17:56:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.843+0000 7f70f9b0c640 1 -- 172.21.15.105:0/1034912443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f419f950 con 0x7f70f41023a0 2024-03-26T17:56:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.843+0000 7f70f9b0c640 1 -- 172.21.15.105:0/1034912443 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f419fe70 con 0x7f70f41023a0 2024-03-26T17:56:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.843+0000 7f70f0ff9640 1 -- 172.21.15.105:0/1034912443 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec010690 con 0x7f70f41023a0 2024-03-26T17:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70f0ff9640 1 -- 172.21.15.105:0/1034912443 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f70ec005020 con 0x7f70f41023a0 2024-03-26T17:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70fc7cb740 1 Processor -- start 2024-03-26T17:56:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70fc7cb740 1 -- start start 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5570807130b0 con 0x5570807aefc0 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5570806bb960 con 0x5570807ae5d0 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.846+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55708070f060 con 0x5570807b4130 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59942/0 (socket says 172.21.15.105:59942) 2024-03-26T17:56:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 -- 172.21.15.105:0/1455698740 learned_addr learned my addr 172.21.15.105:0/1455698740 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45848/0 (socket says 172.21.15.105:45848) 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 -- 172.21.15.105:0/1455698740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 msgr2=0x5570807b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 -- 172.21.15.105:0/1455698740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 msgr2=0x5570807ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f0ff9640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70dc0a2fc0 0x7f70dc0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 -- 172.21.15.105:0/1455698740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570806cdc60 con 0x5570807aefc0 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70dc0a2fc0 0x7f70dc0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.847+0000 7f70f37fe640 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807b3ac0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc9568df783d67 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.848+0000 7f70d9ffb640 1 -- 172.21.15.105:0/1455698740 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4004220 con 0x5570807aefc0 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.848+0000 7f70d9ffb640 1 -- 172.21.15.105:0/1455698740 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70e40043c0 con 0x5570807aefc0 2024-03-26T17:56:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.848+0000 7f70f0ff9640 1 -- 172.21.15.105:0/1034912443 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f70ec0cfb40 con 0x7f70f41023a0 2024-03-26T17:56:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.848+0000 7f70d9ffb640 1 -- 172.21.15.105:0/1455698740 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4019890 con 0x5570807aefc0 2024-03-26T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70dc0a2fc0 0x7f70dc0a5480 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009230 tx=0x7f70e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 -- 172.21.15.105:0/1455698740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 msgr2=0x5570807b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807b3ac0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e4011810 comp rx=0 tx=0).stop 2024-03-26T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 -- 172.21.15.105:0/1455698740 shutdown_connections 2024-03-26T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807b3ac0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/1455698740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 -- 172.21.15.105:0/1455698740 >> 172.21.15.105:0/1455698740 conn(0x5570807a9b40 msgr2=0x5570807a9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 -- 172.21.15.105:0/1455698740 shutdown_connections 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 -- 172.21.15.105:0/1455698740 wait complete. 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.849+0000 7f70fc7cb740 1 -- start start 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70fc7cb740 1 Processor -- start 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5570807bb540 con 0x5570807aefc0 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5570806cedb0 con 0x5570807ae5d0 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5570807bb6c0 con 0x5570807b4130 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70fc7cb740 10 client.0 _ll_register_callbacks cb 0x5570806d3110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.839 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70f3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807bc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45850/0 (socket says 172.21.15.105:45850) 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70f37fe640 1 -- 172.21.15.105:0/263118447 learned_addr learned my addr 172.21.15.105:0/263118447 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70f3fff640 1 -- 172.21.15.105:0/263118447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 msgr2=0x5570807bc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70f3fff640 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807bc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.850+0000 7f70f3fff640 1 -- 172.21.15.105:0/263118447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 msgr2=0x5570807bccb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.851+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.851+0000 7f70f3fff640 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807bccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.851+0000 7f70f3fff640 1 -- 172.21.15.105:0/263118447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557080427210 con 0x5570807b4130 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.851+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.851+0000 7f70f3fff640 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807c3cf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0049d0 tx=0x7f70ec09bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33393660155d8f97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.851+0000 7f70d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.851+0000 7f70d2ffd640 1 -- 172.21.15.105:0/263118447 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec005510 con 0x5570807b4130 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.851+0000 7f70fc7cb740 1 -- 172.21.15.105:0/263118447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70e40043c0 con 0x5570807b4130 2024-03-26T17:56:06.840 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.852+0000 7f70d2ffd640 1 -- 172.21.15.105:0/263118447 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f70ec0b51d0 con 0x5570807b4130 2024-03-26T17:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.852+0000 7f70d2ffd640 1 -- 172.21.15.105:0/263118447 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f70ec0a0320 con 0x5570807b4130 2024-03-26T17:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.852+0000 7f70fc7cb740 10 client.31636 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.852+0000 7f70fc7cb740 1 -- 172.21.15.105:0/263118447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5570804f3620 con 0x5570807b4130 2024-03-26T17:56:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.852+0000 7f70d2ffd640 1 -- 172.21.15.105:0/263118447 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70e40043c0 con 0x5570807b4130 2024-03-26T17:56:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70d2ffd640 1 -- 172.21.15.105:0/263118447 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f70ec0bc070 con 0x5570807b4130 2024-03-26T17:56:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70fc7cb740 10 client.31636 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70fc7cb740 10 client.31636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70fc7cb740 20 client.31636 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5570807c86a0 0x5570807caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70fc7cb740 4 client.31636 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70fc7cb740 1 -- 172.21.15.105:0/263118447 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570807205f0 con 0x5570807c86a0 2024-03-26T17:56:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70f37fe640 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5570807c86a0 0x5570807caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70d2ffd640 10 client.31636 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.853+0000 7f70f37fe640 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5570807c86a0 0x5570807caad0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6289ad426f768865 server_cookie=8f55e56d633db8b9 in_seq=0 out_seq=0 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70d2ffd640 1 -- 172.21.15.105:0/263118447 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5570807205f0 con 0x5570807c86a0 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70d2ffd640 10 client.31636 handle_command_reply: tid=0 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 client.31636 shutdown 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 -- 172.21.15.105:0/263118447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 msgr2=0x5570807c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807c3cf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0049d0 tx=0x7f70ec09bbf0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 -- 172.21.15.105:0/263118447 shutdown_connections 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5570807c86a0 0x5570807caad0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807b4130 0x5570807c3cf0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70d2ffd640 0 client.31636 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/263118447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807bc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 -- 172.21.15.105:0/263118447 >> 172.21.15.105:0/263118447 conn(0x5570807a9b40 msgr2=0x5570807a8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 -- 172.21.15.105:0/263118447 shutdown_connections 2024-03-26T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 1 -- 172.21.15.105:0/263118447 wait complete. 2024-03-26T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.857+0000 7f70fc7cb740 20 client.31636 trim_cache size 0 max 16384 2024-03-26T17:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70fc7cb740 1 Processor -- start 2024-03-26T17:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70fc7cb740 1 -- start start 2024-03-26T17:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807aefc0 0x5570807c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807b4130 0x5570807cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41a6a30 con 0x5570807b4130 2024-03-26T17:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41a8570 con 0x5570807ae5d0 2024-03-26T17:56:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41a7100 con 0x5570807aefc0 2024-03-26T17:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70f2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807aefc0 0x5570807c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807b4130 0x5570807cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807a8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45864/0 (socket says 172.21.15.105:45864) 2024-03-26T17:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70f3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807b4130 0x5570807cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59960/0 (socket says 172.21.15.105:59960) 2024-03-26T17:56:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.886+0000 7f70f37fe640 1 -- 172.21.15.105:0/3849789405 learned_addr learned my addr 172.21.15.105:0/3849789405 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.887+0000 7f70f2ffd640 1 -- 172.21.15.105:0/3849789405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 msgr2=0x5570807a8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.887+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/3849789405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807a8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.887+0000 7f70f2ffd640 1 -- 172.21.15.105:0/3849789405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807b4130 msgr2=0x5570807cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.887+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/3849789405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807b4130 0x5570807cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.887+0000 7f70f2ffd640 1 -- 172.21.15.105:0/3849789405 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570807bbfc0 con 0x5570807aefc0 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.887+0000 7f70f3fff640 1 --2- 172.21.15.105:0/3849789405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807b4130 0x5570807cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.887+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/3849789405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807aefc0 0x5570807c3cb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f70e8000c00 tx=0x7f70e8042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a8b0849516f6538 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.887+0000 7f70d97fa640 1 -- 172.21.15.105:0/3849789405 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8005510 con 0x5570807aefc0 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.888+0000 7f70d97fa640 1 -- 172.21.15.105:0/3849789405 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70e8049070 con 0x5570807aefc0 2024-03-26T17:56:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.888+0000 7f70d97fa640 1 -- 172.21.15.105:0/3849789405 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8003a50 con 0x5570807aefc0 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.888+0000 7f70fc7cb740 1 -- 172.21.15.105:0/3849789405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807aefc0 msgr2=0x5570807c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.888+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/3849789405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807aefc0 0x5570807c3cb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f70e8000c00 tx=0x7f70e8042000 comp rx=0 tx=0).stop 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.888+0000 7f70fc7cb740 1 -- 172.21.15.105:0/3849789405 shutdown_connections 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.888+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/3849789405 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807b4130 0x5570807cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.888+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/3849789405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807aefc0 0x5570807c3cb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/3849789405 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807ae5d0 0x5570807a8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 -- 172.21.15.105:0/3849789405 >> 172.21.15.105:0/3849789405 conn(0x5570807a9b40 msgr2=0x5570807a8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 -- 172.21.15.105:0/3849789405 shutdown_connections 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 -- 172.21.15.105:0/3849789405 wait complete. 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 -- start start 2024-03-26T17:56:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 Processor -- start 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807ae5d0 0x5570807c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807b4130 0x5570807c2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41a7ea0 con 0x5570807aefc0 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41a77d0 con 0x5570807b4130 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41a4a10 con 0x5570807ae5d0 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.889+0000 7f70fc7cb740 10 client.0 _ll_register_callbacks cb 0x7f70ec0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807ae5d0 0x5570807c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807c1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59972/0 (socket says 172.21.15.105:59972) 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f2ffd640 1 -- 172.21.15.105:0/2656092867 learned_addr learned my addr 172.21.15.105:0/2656092867 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f3fff640 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807b4130 0x5570807c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f37fe640 1 -- 172.21.15.105:0/2656092867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807b4130 msgr2=0x5570807c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f37fe640 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807b4130 0x5570807c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f37fe640 1 -- 172.21.15.105:0/2656092867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 msgr2=0x5570807c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f37fe640 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f37fe640 1 -- 172.21.15.105:0/2656092867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570807c2af0 con 0x5570807ae5d0 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70f37fe640 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807ae5d0 0x5570807c39e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009bc0 tx=0x7f70e4005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c4a174a97567b5a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70d2ffd640 1 -- 172.21.15.105:0/2656092867 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e402bd00 con 0x5570807ae5d0 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70d2ffd640 1 -- 172.21.15.105:0/2656092867 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f70e401f040 con 0x5570807ae5d0 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70fc7cb740 1 -- 172.21.15.105:0/2656092867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70e8049070 con 0x5570807ae5d0 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.890+0000 7f70d2ffd640 1 -- 172.21.15.105:0/2656092867 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f70e403a050 con 0x5570807ae5d0 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70fc7cb740 10 client.31642 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70fc7cb740 1 -- 172.21.15.105:0/2656092867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55708082f810 con 0x5570807ae5d0 2024-03-26T17:56:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70d2ffd640 1 -- 172.21.15.105:0/2656092867 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70e402f020 con 0x5570807ae5d0 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70d2ffd640 1 -- 172.21.15.105:0/2656092867 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f70e4005460 con 0x5570807ae5d0 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70fc7cb740 10 client.31642 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70fc7cb740 10 client.31642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70fc7cb740 20 client.31642 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5570808300b0 0x5570808324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70fc7cb740 4 client.31642 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70fc7cb740 1 -- 172.21.15.105:0/2656092867 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f70e8049070 con 0x5570808300b0 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.891+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5570808300b0 0x5570808324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.892+0000 7f70d2ffd640 10 client.31642 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:06.892+0000 7f70f2ffd640 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5570808300b0 0x5570808324d0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e947090c482d82e3 server_cookie=460c8338a6c52648 in_seq=0 out_seq=0 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 -- 172.21.15.105:0/827624555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e64102400 msgr2=0x7f2e641047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 --2- 172.21.15.105:0/827624555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e64102400 0x7f2e641047f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009900 tx=0x7f2e5402bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 -- 172.21.15.105:0/827624555 shutdown_connections 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 --2- 172.21.15.105:0/827624555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e64107660 0x7f2e64109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 --2- 172.21.15.105:0/827624555 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e64104d30 0x7f2e64107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 --2- 172.21.15.105:0/827624555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e64102400 0x7f2e641047f0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 -- 172.21.15.105:0/827624555 >> 172.21.15.105:0/827624555 conn(0x7f2e640fbd50 msgr2=0x7f2e640fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 -- 172.21.15.105:0/827624555 shutdown_connections 2024-03-26T17:56:06.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.924+0000 7f2e69c97640 1 -- 172.21.15.105:0/827624555 wait complete. 2024-03-26T17:56:06.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.926+0000 7f2e69c97640 1 Processor -- start 2024-03-26T17:56:06.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.927+0000 7f2e69c97640 1 -- start start 2024-03-26T17:56:06.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.927+0000 7f2e69c97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e64102400 0x7f2e64197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.927+0000 7f2e69c97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e64104d30 0x7f2e641980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.927+0000 7f2e69c97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e64107660 0x7f2e6419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.927+0000 7f2e69c97640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6410d3f0 con 0x7f2e64102400 2024-03-26T17:56:06.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.927+0000 7f2e69c97640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6410d270 con 0x7f2e64107660 2024-03-26T17:56:06.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.927+0000 7f2e69c97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6410d570 con 0x7f2e64104d30 2024-03-26T17:56:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.931+0000 7f2e5affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e64104d30 0x7f2e641980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.931+0000 7f2e5affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e64104d30 0x7f2e641980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34782/0 (socket says 172.21.15.105:34782) 2024-03-26T17:56:06.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.931+0000 7f2e5affd640 1 -- 172.21.15.105:0/2260130434 learned_addr learned my addr 172.21.15.105:0/2260130434 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.932+0000 7f2e637fe640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e64102400 0x7f2e64197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.932+0000 7f2e5affd640 1 -- 172.21.15.105:0/2260130434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e64107660 msgr2=0x7f2e6419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.932+0000 7f2e5affd640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e64107660 0x7f2e6419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.932+0000 7f2e5affd640 1 -- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e64102400 msgr2=0x7f2e64197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.932+0000 7f2e5affd640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e64102400 0x7f2e64197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.932+0000 7f2e5affd640 1 -- 172.21.15.105:0/2260130434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6419f7d0 con 0x7f2e64104d30 2024-03-26T17:56:06.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.942+0000 7f2e5affd640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e64104d30 0x7f2e641980f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00e9c0 tx=0x7f2e4c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.944+0000 7f2e617fa640 1 -- 172.21.15.105:0/2260130434 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00cd80 con 0x7f2e64104d30 2024-03-26T17:56:06.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.944+0000 7f2e617fa640 1 -- 172.21.15.105:0/2260130434 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c0054d0 con 0x7f2e64104d30 2024-03-26T17:56:06.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.944+0000 7f2e617fa640 1 -- 172.21.15.105:0/2260130434 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c010430 con 0x7f2e64104d30 2024-03-26T17:56:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.946+0000 7f2e69c97640 1 -- 172.21.15.105:0/2260130434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e64100250 con 0x7f2e64104d30 2024-03-26T17:56:06.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.946+0000 7f2e69c97640 1 -- 172.21.15.105:0/2260130434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e64100720 con 0x7f2e64104d30 2024-03-26T17:56:06.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.950+0000 7f2e6c883740 1 Processor -- start 2024-03-26T17:56:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.951+0000 7f2e617fa640 1 -- 172.21.15.105:0/2260130434 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2e4c004f90 con 0x7f2e64104d30 2024-03-26T17:56:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.951+0000 7f2e6c883740 1 -- start start 2024-03-26T17:56:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25d6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 0x5637a25dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a25de990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e6c883740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637a2533630 con 0x5637a25d69a0 2024-03-26T17:56:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e6c883740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5637a24dbe10 con 0x5637a25dc500 2024-03-26T17:56:06.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e6c883740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637a252f5e0 con 0x5637a25d7390 2024-03-26T17:56:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e617fa640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2e500a30e0 0x7f2e500a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25d6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25d6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60000/0 (socket says 172.21.15.105:60000) 2024-03-26T17:56:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.952+0000 7f2e63fff640 1 -- 172.21.15.105:0/1089559777 learned_addr learned my addr 172.21.15.105:0/1089559777 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e63fff640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2e500a30e0 0x7f2e500a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e5affd640 1 --2- 172.21.15.105:0/1089559777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a25de990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e617fa640 1 -- 172.21.15.105:0/2260130434 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2e4c0cf950 con 0x7f2e64104d30 2024-03-26T17:56:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e5affd640 1 -- 172.21.15.105:0/1089559777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 msgr2=0x5637a25dbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e5affd640 1 --2- 172.21.15.105:0/1089559777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 0x5637a25dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e5affd640 1 -- 172.21.15.105:0/1089559777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 msgr2=0x5637a25d6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e5affd640 1 --2- 172.21.15.105:0/1089559777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25d6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e5affd640 1 -- 172.21.15.105:0/1089559777 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a24f2740 con 0x5637a25dc500 2024-03-26T17:56:06.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.953+0000 7f2e5affd640 1 --2- 172.21.15.105:0/1089559777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a25de990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c020000 tx=0x7f2e4c004780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9b11e5bcfda99f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.958+0000 7f2e63fff640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2e500a30e0 0x7f2e500a55a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x5637a25d7bf0 tx=0x7f2e5c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.961+0000 7f2e59ffb640 1 -- 172.21.15.105:0/1089559777 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c09b940 con 0x5637a25dc500 2024-03-26T17:56:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.961+0000 7f2e59ffb640 1 -- 172.21.15.105:0/1089559777 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c09c6e0 con 0x5637a25dc500 2024-03-26T17:56:06.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.961+0000 7f2e59ffb640 1 -- 172.21.15.105:0/1089559777 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c01f330 con 0x5637a25dc500 2024-03-26T17:56:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.963+0000 7f2e6c883740 1 -- 172.21.15.105:0/1089559777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 msgr2=0x5637a25de990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.963+0000 7f2e6c883740 1 --2- 172.21.15.105:0/1089559777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a25de990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c020000 tx=0x7f2e4c004780 comp rx=0 tx=0).stop 2024-03-26T17:56:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.964+0000 7f2e6c883740 1 -- 172.21.15.105:0/1089559777 shutdown_connections 2024-03-26T17:56:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.964+0000 7f2e6c883740 1 --2- 172.21.15.105:0/1089559777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a25de990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.964+0000 7f2e6c883740 1 --2- 172.21.15.105:0/1089559777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 0x5637a25dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.964+0000 7f2e6c883740 1 --2- 172.21.15.105:0/1089559777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25d6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.964+0000 7f2e6c883740 1 -- 172.21.15.105:0/1089559777 >> 172.21.15.105:0/1089559777 conn(0x5637a25d1f10 msgr2=0x5637a25d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.966+0000 7f2e6c883740 1 -- 172.21.15.105:0/1089559777 shutdown_connections 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.966+0000 7f2e6c883740 1 -- 172.21.15.105:0/1089559777 wait complete. 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.966+0000 7f2e6c883740 1 -- start start 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.966+0000 7f2e6c883740 1 Processor -- start 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dc500 0x5637a25f9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e6c883740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637a25df450 con 0x5637a25d69a0 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e6c883740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5637a24ef1a0 con 0x5637a25d7390 2024-03-26T17:56:06.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e6c883740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637a25df5d0 con 0x5637a25dc500 2024-03-26T17:56:06.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e6c883740 10 client.0 _ll_register_callbacks cb 0x5637a2246c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60006/0 (socket says 172.21.15.105:60006) 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.967+0000 7f2e637fe640 1 -- 172.21.15.105:0/3950560129 learned_addr learned my addr 172.21.15.105:0/3950560129 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e63fff640 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dc500 0x5637a25f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e5affd640 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e637fe640 1 -- 172.21.15.105:0/3950560129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dc500 msgr2=0x5637a25f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e637fe640 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dc500 0x5637a25f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e637fe640 1 -- 172.21.15.105:0/3950560129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 msgr2=0x5637a25f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e637fe640 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e637fe640 1 -- 172.21.15.105:0/3950560129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a230f7e0 con 0x5637a25d69a0 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e637fe640 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25f1d40 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2e5402c170 tx=0x7f2e54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9c2a3dacc57631 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:06.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e42ffd640 1 -- 172.21.15.105:0/3950560129 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54044020 con 0x5637a25d69a0 2024-03-26T17:56:06.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.968+0000 7f2e6c883740 1 -- 172.21.15.105:0/3950560129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c09c6e0 con 0x5637a25d69a0 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.969+0000 7f2e42ffd640 1 -- 172.21.15.105:0/3950560129 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2e54005240 con 0x5637a25d69a0 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.969+0000 7f2e42ffd640 1 -- 172.21.15.105:0/3950560129 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2e5404f050 con 0x5637a25d69a0 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.969+0000 7f2e6c883740 10 client.21942 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.969+0000 7f2e6c883740 1 -- 172.21.15.105:0/3950560129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5637a236ff70 con 0x5637a25d69a0 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.969+0000 7f2e42ffd640 1 -- 172.21.15.105:0/3950560129 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c09c6e0 con 0x5637a25d69a0 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.969+0000 7f2e42ffd640 1 -- 172.21.15.105:0/3950560129 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2e540035f0 con 0x5637a25d69a0 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.970+0000 7f2e6c883740 10 client.21942 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.970+0000 7f2e6c883740 10 client.21942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.970+0000 7f2e6c883740 20 client.21942 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:06.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.970+0000 7f2e6c883740 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a25cac40 0x5637a25cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.970+0000 7f2e6c883740 4 client.21942 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.970+0000 7f2e6c883740 1 -- 172.21.15.105:0/3950560129 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637a2540b70 con 0x5637a25cac40 2024-03-26T17:56:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.971+0000 7f2e5affd640 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a25cac40 0x5637a25cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.975+0000 7f2e5affd640 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a25cac40 0x5637a25cd100 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78194d8429fcf93d server_cookie=28c5db778880442a in_seq=0 out_seq=0 2024-03-26T17:56:06.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:06.975+0000 7f2e42ffd640 10 client.21942 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.050+0000 7f70d2ffd640 1 -- 172.21.15.105:0/2656092867 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f70e8049070 con 0x5570808300b0 2024-03-26T17:56:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.050+0000 7f70d2ffd640 10 client.31642 handle_command_reply: tid=0 2024-03-26T17:56:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.050+0000 7f70fc7cb740 1 client.31642 shutdown 2024-03-26T17:56:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 -- 172.21.15.105:0/2656092867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807ae5d0 msgr2=0x5570807c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807ae5d0 0x5570807c39e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009bc0 tx=0x7f70e4005d50 comp rx=0 tx=0).stop 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 -- 172.21.15.105:0/2656092867 shutdown_connections 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5570808300b0 0x5570808324d0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570807b4130 0x5570807c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5570807aefc0 0x5570807c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 --2- 172.21.15.105:0/2656092867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5570807ae5d0 0x5570807c39e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 -- 172.21.15.105:0/2656092867 >> 172.21.15.105:0/2656092867 conn(0x5570807a9b40 msgr2=0x5570807aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70d2ffd640 0 client.31642 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 -- 172.21.15.105:0/2656092867 shutdown_connections 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 1 -- 172.21.15.105:0/2656092867 wait complete. 2024-03-26T17:56:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f70fc7cb740 20 client.31642 trim_cache size 0 max 16384 2024-03-26T17:56:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f2e42ffd640 1 -- 172.21.15.105:0/3950560129 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5637a2540b70 con 0x5637a25cac40 2024-03-26T17:56:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f2e42ffd640 10 client.21942 handle_command_reply: tid=0 2024-03-26T17:56:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f2e6c883740 1 client.21942 shutdown 2024-03-26T17:56:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:56:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:56:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:56:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:56:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f2e6c883740 1 -- 172.21.15.105:0/3950560129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 msgr2=0x5637a25f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.051+0000 7f2e6c883740 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25f1d40 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f2e5402c170 tx=0x7f2e54004270 comp rx=0 tx=0).stop 2024-03-26T17:56:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 1 -- 172.21.15.105:0/3950560129 shutdown_connections 2024-03-26T17:56:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a25cac40 0x5637a25cd100 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dc500 0x5637a25f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 1 --2- 172.21.15.105:0/3950560129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25d69a0 0x5637a25f1d40 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 1 -- 172.21.15.105:0/3950560129 >> 172.21.15.105:0/3950560129 conn(0x5637a25d1f10 msgr2=0x5637a25c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e42ffd640 0 client.21942 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 1 -- 172.21.15.105:0/3950560129 shutdown_connections 2024-03-26T17:56:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 1 -- 172.21.15.105:0/3950560129 wait complete. 2024-03-26T17:56:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f70f9b0c640 1 -- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70dc0a2fc0 msgr2=0x7f70dc0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70dc0a2fc0 0x7f70dc0a5480 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009230 tx=0x7f70e8009260 comp rx=0 tx=0).stop 2024-03-26T17:56:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f2e6c883740 20 client.21942 trim_cache size 0 max 16384 2024-03-26T17:56:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f70f9b0c640 1 -- 172.21.15.105:0/1034912443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f41023a0 msgr2=0x7f70f419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.052+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f41023a0 0x7f70f419edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00eb10 tx=0x7f70ec00c2d0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.053+0000 7f70f9b0c640 1 -- 172.21.15.105:0/1034912443 shutdown_connections 2024-03-26T17:56:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.053+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f70dc0a2fc0 0x7f70dc0a5480 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.053+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f41023a0 0x7f70f419edb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.053+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70f41019e0 0x7f70f419c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.053+0000 7f70f9b0c640 1 --2- 172.21.15.105:0/1034912443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70f41007f0 0x7f70f406abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.053+0000 7f70f9b0c640 1 -- 172.21.15.105:0/1034912443 >> 172.21.15.105:0/1034912443 conn(0x7f70f40fbd80 msgr2=0x7f70f4104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.053+0000 7f70f9b0c640 1 -- 172.21.15.105:0/1034912443 shutdown_connections 2024-03-26T17:56:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:07.053+0000 7f70f9b0c640 1 -- 172.21.15.105:0/1034912443 wait complete. 2024-03-26T17:56:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.083+0000 7f2e6c883740 1 Processor -- start 2024-03-26T17:56:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e6c883740 1 -- start start 2024-03-26T17:56:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dc500 0x5637a25f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dbff0 0x5637a25c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e6c883740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64072680 con 0x5637a25dc500 2024-03-26T17:56:07.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e6c883740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641b1080 con 0x5637a25d7390 2024-03-26T17:56:07.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e6c883740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64072d50 con 0x5637a25dbff0 2024-03-26T17:56:07.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dc500 0x5637a25f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e637fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dc500 0x5637a25f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60010/0 (socket says 172.21.15.105:60010) 2024-03-26T17:56:07.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dbff0 0x5637a25c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e5affd640 1 -- 172.21.15.105:0/1696119523 learned_addr learned my addr 172.21.15.105:0/1696119523 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:07.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e5affd640 1 -- 172.21.15.105:0/1696119523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dbff0 msgr2=0x5637a25c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e5affd640 1 --2- 172.21.15.105:0/1696119523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dbff0 0x5637a25c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.084+0000 7f2e5affd640 1 -- 172.21.15.105:0/1696119523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 msgr2=0x5637a25f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.085+0000 7f2e5affd640 1 --2- 172.21.15.105:0/1696119523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.085+0000 7f2e5affd640 1 -- 172.21.15.105:0/1696119523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637a2207170 con 0x5637a25dc500 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.085+0000 7f2e637fe640 1 --2- 172.21.15.105:0/1696119523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.085+0000 7f2e5affd640 1 --2- 172.21.15.105:0/1696119523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dc500 0x5637a25f1ed0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c098440 tx=0x7f2e4c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f7ee36649698e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.085+0000 7f2e597fa640 1 -- 172.21.15.105:0/1696119523 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c005280 con 0x5637a25dc500 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.085+0000 7f2e597fa640 1 -- 172.21.15.105:0/1696119523 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c09c6e0 con 0x5637a25dc500 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.085+0000 7f2e597fa640 1 -- 172.21.15.105:0/1696119523 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0a0130 con 0x5637a25dc500 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 -- 172.21.15.105:0/1696119523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dc500 msgr2=0x5637a25f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 --2- 172.21.15.105:0/1696119523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dc500 0x5637a25f1ed0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c098440 tx=0x7f2e4c005f10 comp rx=0 tx=0).stop 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 -- 172.21.15.105:0/1696119523 shutdown_connections 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 --2- 172.21.15.105:0/1696119523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25dbff0 0x5637a25c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 --2- 172.21.15.105:0/1696119523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dc500 0x5637a25f1ed0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 --2- 172.21.15.105:0/1696119523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25d7390 0x5637a25f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 -- 172.21.15.105:0/1696119523 >> 172.21.15.105:0/1696119523 conn(0x5637a25d1f10 msgr2=0x5637a25d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 -- 172.21.15.105:0/1696119523 shutdown_connections 2024-03-26T17:56:07.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 -- 172.21.15.105:0/1696119523 wait complete. 2024-03-26T17:56:07.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.086+0000 7f2e6c883740 1 -- start start 2024-03-26T17:56:07.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e6c883740 1 Processor -- start 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 0x5637a2655bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dbff0 0x5637a2658120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e6c883740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a265a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e6c883740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e641b09b0 con 0x5637a25dbff0 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e6c883740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64073420 con 0x5637a25dc500 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e6c883740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64070660 con 0x5637a25d7390 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e5affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dbff0 0x5637a2658120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a265a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e6c883740 10 client.0 _ll_register_callbacks cb 0x7f2e4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a265a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45924/0 (socket says 172.21.15.105:45924) 2024-03-26T17:56:07.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e63fff640 1 -- 172.21.15.105:0/562391263 learned_addr learned my addr 172.21.15.105:0/562391263 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.087+0000 7f2e637fe640 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 0x5637a2655bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e5affd640 1 -- 172.21.15.105:0/562391263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 msgr2=0x5637a2655bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e5affd640 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 0x5637a2655bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e5affd640 1 -- 172.21.15.105:0/562391263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 msgr2=0x5637a265a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e5affd640 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a265a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e5affd640 1 -- 172.21.15.105:0/562391263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637a265ac30 con 0x5637a25dbff0 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e63fff640 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a265a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e5affd640 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dbff0 0x5637a2658120 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0058c0 tx=0x7f2e4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2857feeab0e25bf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e42ffd640 1 -- 172.21.15.105:0/562391263 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0a8030 con 0x5637a25dbff0 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e42ffd640 1 -- 172.21.15.105:0/562391263 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f2e4c09a080 con 0x5637a25dbff0 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e6c883740 1 -- 172.21.15.105:0/562391263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c09c6e0 con 0x5637a25dbff0 2024-03-26T17:56:07.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e42ffd640 1 -- 172.21.15.105:0/562391263 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2e4c005280 con 0x5637a25dbff0 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e6c883740 10 client.21948 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.088+0000 7f2e6c883740 1 -- 172.21.15.105:0/562391263 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5637a265bf50 con 0x5637a25dbff0 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e42ffd640 1 -- 172.21.15.105:0/562391263 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c09c6e0 con 0x5637a25dbff0 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e42ffd640 1 -- 172.21.15.105:0/562391263 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2e4c097d30 con 0x5637a25dbff0 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e6c883740 10 client.21948 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e6c883740 10 client.21948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e6c883740 20 client.21948 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e6c883740 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a265c290 0x5637a265e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e6c883740 4 client.21948 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e6c883740 1 -- 172.21.15.105:0/562391263 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5637a253aec0 con 0x5637a265c290 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.089+0000 7f2e637fe640 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a265c290 0x5637a265e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.090+0000 7f2e637fe640 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a265c290 0x5637a265e680 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fe83d9250de3ba server_cookie=fc85f176894b44ea in_seq=0 out_seq=0 2024-03-26T17:56:07.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.090+0000 7f2e42ffd640 10 client.21948 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.092+0000 7f2e42ffd640 1 -- 172.21.15.105:0/562391263 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5637a253aec0 con 0x5637a265c290 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.092+0000 7f2e42ffd640 10 client.21948 handle_command_reply: tid=0 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.092+0000 7f2e6c883740 1 client.21948 shutdown 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 -- 172.21.15.105:0/562391263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dbff0 msgr2=0x5637a2658120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dbff0 0x5637a2658120 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0058c0 tx=0x7f2e4c00cf90 comp rx=0 tx=0).stop 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 -- 172.21.15.105:0/562391263 shutdown_connections 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5637a265c290 0x5637a265e680 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637a25dc500 0x5637a265a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637a25dbff0 0x5637a2658120 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 --2- 172.21.15.105:0/562391263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637a25d7390 0x5637a2655bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 -- 172.21.15.105:0/562391263 >> 172.21.15.105:0/562391263 conn(0x5637a25d1f10 msgr2=0x5637a25f1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e42ffd640 0 client.21948 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 -- 172.21.15.105:0/562391263 shutdown_connections 2024-03-26T17:56:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 1 -- 172.21.15.105:0/562391263 wait complete. 2024-03-26T17:56:07.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.093+0000 7f2e6c883740 20 client.21948 trim_cache size 0 max 16384 2024-03-26T17:56:07.082 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:56:07.082 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:56:07.082 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "ba382905-caef-45a4-b2d3-7d53b115a6f7", 2024-03-26T17:56:07.082 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:56:07.082 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:56:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 -- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2e500a30e0 msgr2=0x7f2e500a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2e500a30e0 0x7f2e500a55a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x5637a25d7bf0 tx=0x7f2e5c004a40 comp rx=0 tx=0).stop 2024-03-26T17:56:07.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 -- 172.21.15.105:0/2260130434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e64104d30 msgr2=0x7f2e641980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e64104d30 0x7f2e641980f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00e9c0 tx=0x7f2e4c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 -- 172.21.15.105:0/2260130434 shutdown_connections 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2e500a30e0 0x7f2e500a55a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e64107660 0x7f2e6419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e64104d30 0x7f2e641980f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 --2- 172.21.15.105:0/2260130434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e64102400 0x7f2e64197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 -- 172.21.15.105:0/2260130434 >> 172.21.15.105:0/2260130434 conn(0x7f2e640fbd50 msgr2=0x7f2e64102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 -- 172.21.15.105:0/2260130434 shutdown_connections 2024-03-26T17:56:07.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.094+0000 7f2e69c97640 1 -- 172.21.15.105:0/2260130434 wait complete. 2024-03-26T17:56:07.093 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:06 smithi105 ceph-mon[32280]: pgmap v882: 97 pgs: 97 active+clean; 287 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-03-26T17:56:07.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:56:07.095 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:56:07.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:06 smithi135 ceph-mon[35636]: pgmap v882: 97 pgs: 97 active+clean; 287 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-03-26T17:56:07.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.289+0000 7f3db396f640 1 -- 172.21.15.105:0/3929256776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1019b0 msgr2=0x7f3dac101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.289+0000 7f3db396f640 1 --2- 172.21.15.105:0/3929256776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1019b0 0x7f3dac101e30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c009900 tx=0x7f3d9c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:07.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.289+0000 7f3db396f640 1 -- 172.21.15.105:0/3929256776 shutdown_connections 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.289+0000 7f3db396f640 1 --2- 172.21.15.105:0/3929256776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac102370 0x7f3dac108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.289+0000 7f3db396f640 1 --2- 172.21.15.105:0/3929256776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1019b0 0x7f3dac101e30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.289+0000 7f3db396f640 1 --2- 172.21.15.105:0/3929256776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1007c0 0x7f3dac100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.289+0000 7f3db396f640 1 -- 172.21.15.105:0/3929256776 >> 172.21.15.105:0/3929256776 conn(0x7f3dac0fbd50 msgr2=0x7f3dac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.290+0000 7f3db396f640 1 -- 172.21.15.105:0/3929256776 shutdown_connections 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.290+0000 7f3db396f640 1 -- 172.21.15.105:0/3929256776 wait complete. 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.290+0000 7f3db396f640 1 Processor -- start 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.290+0000 7f3db396f640 1 -- start start 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.290+0000 7f3db396f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1007c0 0x7f3dac197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.290+0000 7f3db396f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1019b0 0x7f3dac198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db396f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac102370 0x7f3dac19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db396f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10c2d0 con 0x7f3dac1019b0 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db396f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10c150 con 0x7f3dac1007c0 2024-03-26T17:56:07.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db396f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac10c450 con 0x7f3dac102370 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1007c0 0x7f3dac197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1007c0 0x7f3dac197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45942/0 (socket says 172.21.15.105:45942) 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 -- 172.21.15.105:0/1466442656 learned_addr learned my addr 172.21.15.105:0/1466442656 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac102370 0x7f3dac19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1019b0 0x7f3dac198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 -- 172.21.15.105:0/1466442656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac102370 msgr2=0x7f3dac19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac102370 0x7f3dac19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 -- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1019b0 msgr2=0x7f3dac198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1019b0 0x7f3dac198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 -- 172.21.15.105:0/1466442656 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac19f710 con 0x7f3dac1007c0 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.291+0000 7f3db170b640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1007c0 0x7f3dac197af0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d20 tx=0x7f3da800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:07.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.292+0000 7f3da27fc640 1 -- 172.21.15.105:0/1466442656 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8010630 con 0x7f3dac1007c0 2024-03-26T17:56:07.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.292+0000 7f3da27fc640 1 -- 172.21.15.105:0/1466442656 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da8010c80 con 0x7f3dac1007c0 2024-03-26T17:56:07.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.292+0000 7f3db396f640 1 -- 172.21.15.105:0/1466442656 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac19fa00 con 0x7f3dac1007c0 2024-03-26T17:56:07.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.292+0000 7f3da27fc640 1 -- 172.21.15.105:0/1466442656 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8018d70 con 0x7f3dac1007c0 2024-03-26T17:56:07.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.292+0000 7f3db396f640 1 -- 172.21.15.105:0/1466442656 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac19ff70 con 0x7f3dac1007c0 2024-03-26T17:56:07.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.295+0000 7f3db6630740 1 Processor -- start 2024-03-26T17:56:07.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db6630740 1 -- start start 2024-03-26T17:56:07.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db6630740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781bbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db6630740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 0x5616781c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db6630740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 0x5616781c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db6630740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561678120980 con 0x5616781bc2e0 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db6630740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5616780c8be0 con 0x5616781bb8f0 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db6630740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56167811c930 con 0x5616781c1450 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781bbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781bbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45944/0 (socket says 172.21.15.105:45944) 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 -- 172.21.15.105:0/943188791 learned_addr learned my addr 172.21.15.105:0/943188791 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/943188791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 0x5616781c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 -- 172.21.15.105:0/943188791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 msgr2=0x5616781c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/943188791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 0x5616781c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 -- 172.21.15.105:0/943188791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 msgr2=0x5616781c0de0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/943188791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 0x5616781c0de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 -- 172.21.15.105:0/943188791 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561677efe7d0 con 0x5616781bb8f0 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3da27fc640 1 -- 172.21.15.105:0/1466442656 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3da801f020 con 0x7f3dac1007c0 2024-03-26T17:56:07.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.296+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/943188791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781bbd10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3da400b700 tx=0x7f3da400bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e10528a8af55dac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:07.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.297+0000 7f3d977fe640 1 -- 172.21.15.105:0/943188791 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4003dd0 con 0x5616781bb8f0 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.297+0000 7f3d977fe640 1 -- 172.21.15.105:0/943188791 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da4015070 con 0x5616781bb8f0 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.297+0000 7f3d977fe640 1 -- 172.21.15.105:0/943188791 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da40106a0 con 0x5616781bb8f0 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.297+0000 7f3da27fc640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fa0 0x7f3d980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.297+0000 7f3da27fc640 1 -- 172.21.15.105:0/1466442656 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3da80cf860 con 0x7f3dac1007c0 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.297+0000 7f3db170b640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fa0 0x7f3d980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 -- 172.21.15.105:0/943188791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 msgr2=0x5616781bbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db170b640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fa0 0x7f3d980a5460 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3da80a43d0 tx=0x7f3da8009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 --2- 172.21.15.105:0/943188791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781bbd10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3da400b700 tx=0x7f3da400bbc0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 -- 172.21.15.105:0/943188791 shutdown_connections 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 --2- 172.21.15.105:0/943188791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 0x5616781c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 --2- 172.21.15.105:0/943188791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 0x5616781c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 --2- 172.21.15.105:0/943188791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781bbd10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 -- 172.21.15.105:0/943188791 >> 172.21.15.105:0/943188791 conn(0x5616781b6e60 msgr2=0x5616781b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 -- 172.21.15.105:0/943188791 shutdown_connections 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.298+0000 7f3db6630740 1 -- 172.21.15.105:0/943188791 wait complete. 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 1 -- start start 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 1 Processor -- start 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 0x5616781c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 0x5616781cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5616781c4400 con 0x5616781bc2e0 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5616780dc160 con 0x5616781bb8f0 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5616781c4580 con 0x5616781c1450 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db1f0c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db6630740 10 client.0 _ll_register_callbacks cb 0x561678118720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db1f0c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45958/0 (socket says 172.21.15.105:45958) 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db1f0c640 1 -- 172.21.15.105:0/1199785241 learned_addr learned my addr 172.21.15.105:0/1199785241 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:07.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 0x5616781c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.299+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 0x5616781cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3db1f0c640 1 -- 172.21.15.105:0/1199785241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 msgr2=0x5616781cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 0x5616781cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3db1f0c640 1 -- 172.21.15.105:0/1199785241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 msgr2=0x5616781c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 0x5616781c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3db1f0c640 1 -- 172.21.15.105:0/1199785241 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561677e343f0 con 0x5616781bb8f0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 0x5616781c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c5b40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3da4010760 tx=0x7f3da4011f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f63d3d9484543e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1199785241 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4011640 con 0x5616781bb8f0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3db6630740 1 -- 172.21.15.105:0/1199785241 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da4015070 con 0x5616781bb8f0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.300+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1199785241 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3da40117e0 con 0x5616781bb8f0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1199785241 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3da4005020 con 0x5616781bb8f0 2024-03-26T17:56:07.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db6630740 10 client.31688 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db6630740 1 -- 172.21.15.105:0/1199785241 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561677df40f0 con 0x5616781bb8f0 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1199785241 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da4015070 con 0x5616781bb8f0 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1199785241 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3da40046c0 con 0x5616781bb8f0 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db6630740 10 client.31688 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db6630740 10 client.31688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db6630740 20 client.31688 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db6630740 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5616781d1c30 0x5616781d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db6630740 4 client.31688 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db6630740 1 -- 172.21.15.105:0/1199785241 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56167812dec0 con 0x5616781d1c30 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.301+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5616781d1c30 0x5616781d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.302+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5616781d1c30 0x5616781d4070 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f355de21479d5b4 server_cookie=6a3bd3b6d2a34b77 in_seq=0 out_seq=0 2024-03-26T17:56:07.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.302+0000 7f3d94ff9640 10 client.31688 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3d94ff9640 1 -- 172.21.15.105:0/1199785241 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56167812dec0 con 0x5616781d1c30 2024-03-26T17:56:07.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3d94ff9640 10 client.31688 handle_command_reply: tid=0 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 client.31688 shutdown 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 -- 172.21.15.105:0/1199785241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 msgr2=0x5616781c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c5b40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3da4010760 tx=0x7f3da4011f70 comp rx=0 tx=0).stop 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 -- 172.21.15.105:0/1199785241 shutdown_connections 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5616781d1c30 0x5616781d4070 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781c1450 0x5616781cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781bc2e0 0x5616781c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3d94ff9640 0 client.31688 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 --2- 172.21.15.105:0/1199785241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c5b40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 -- 172.21.15.105:0/1199785241 >> 172.21.15.105:0/1199785241 conn(0x5616781b6e60 msgr2=0x5616781b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 -- 172.21.15.105:0/1199785241 shutdown_connections 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.304+0000 7f3db6630740 1 -- 172.21.15.105:0/1199785241 wait complete. 2024-03-26T17:56:07.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.305+0000 7f3db6630740 20 client.31688 trim_cache size 0 max 16384 2024-03-26T17:56:07.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db6630740 1 Processor -- start 2024-03-26T17:56:07.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db6630740 1 -- start start 2024-03-26T17:56:07.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db6630740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db6630740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db6630740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x5616781b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db6630740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1aaff0 con 0x5616781c1450 2024-03-26T17:56:07.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db6630740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1acb30 con 0x5616781bb8f0 2024-03-26T17:56:07.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db6630740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1ab6c0 con 0x5616781bc2e0 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db1f0c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x5616781b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db0f0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db1f0c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x5616781b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60054/0 (socket says 172.21.15.105:60054) 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db0f0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34842/0 (socket says 172.21.15.105:34842) 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db1f0c640 1 -- 172.21.15.105:0/26147943 learned_addr learned my addr 172.21.15.105:0/26147943 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.333+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3db0f0a640 1 -- 172.21.15.105:0/26147943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 msgr2=0x5616781c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3db0f0a640 1 -- 172.21.15.105:0/26147943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 msgr2=0x5616781b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x5616781b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3db0f0a640 1 -- 172.21.15.105:0/26147943 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616781c4be0 con 0x5616781bc2e0 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x5616781b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:07.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781c5e60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c000c00 tx=0x7f3d9c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d974b0ae12278bd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:07.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3d96ffd640 1 -- 172.21.15.105:0/26147943 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c049050 con 0x5616781bc2e0 2024-03-26T17:56:07.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3d96ffd640 1 -- 172.21.15.105:0/26147943 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d9c04bab0 con 0x5616781bc2e0 2024-03-26T17:56:07.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.334+0000 7f3d96ffd640 1 -- 172.21.15.105:0/26147943 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c04b820 con 0x5616781bc2e0 2024-03-26T17:56:07.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 -- 172.21.15.105:0/26147943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 msgr2=0x5616781c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781c5e60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c000c00 tx=0x7f3d9c009260 comp rx=0 tx=0).stop 2024-03-26T17:56:07.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 -- 172.21.15.105:0/26147943 shutdown_connections 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x5616781b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781c5e60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 --2- 172.21.15.105:0/26147943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 -- 172.21.15.105:0/26147943 >> 172.21.15.105:0/26147943 conn(0x5616781b6e60 msgr2=0x5616781c5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 -- 172.21.15.105:0/26147943 shutdown_connections 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.335+0000 7f3db6630740 1 -- 172.21.15.105:0/26147943 wait complete. 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db6630740 1 -- start start 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db6630740 1 Processor -- start 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db6630740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db6630740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db6630740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x561678235fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db6630740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1ac460 con 0x5616781c1450 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db6630740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1abd90 con 0x5616781bb8f0 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db0f0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db0f0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34858/0 (socket says 172.21.15.105:34858) 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db0f0a640 1 -- 172.21.15.105:0/155441686 learned_addr learned my addr 172.21.15.105:0/155441686 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac1a8fd0 con 0x5616781bc2e0 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.336+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3db6630740 10 client.0 _ll_register_callbacks cb 0x7f3d9c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x561678235fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3db0f0a640 1 -- 172.21.15.105:0/155441686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 msgr2=0x5616781c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3db0f0a640 1 -- 172.21.15.105:0/155441686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 msgr2=0x561678235fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:07.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x561678235fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3db0f0a640 1 -- 172.21.15.105:0/155441686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561678236560 con 0x5616781bc2e0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3db0f0a640 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781ca2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c036040 tx=0x7f3d9c04b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e971a59f229dfe8d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3d94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.337+0000 7f3d94ff9640 1 -- 172.21.15.105:0/155441686 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c057020 con 0x5616781bc2e0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.338+0000 7f3d94ff9640 1 -- 172.21.15.105:0/155441686 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f3d9c004180 con 0x5616781bc2e0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.338+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d9c04bab0 con 0x5616781bc2e0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.338+0000 7f3d94ff9640 1 -- 172.21.15.105:0/155441686 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3d9c04b430 con 0x5616781bc2e0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.338+0000 7f3db6630740 10 client.31675 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.338+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5616782378c0 con 0x5616781bc2e0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.338+0000 7f3d94ff9640 1 -- 172.21.15.105:0/155441686 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d9c04bab0 con 0x5616781bc2e0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.338+0000 7f3d94ff9640 1 -- 172.21.15.105:0/155441686 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3d9c0047e0 con 0x5616781bc2e0 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.338+0000 7f3db6630740 10 client.31675 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.339+0000 7f3db6630740 10 client.31675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:07.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.339+0000 7f3db6630740 20 client.31675 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:07.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.339+0000 7f3db6630740 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561678238230 0x56167823a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:07.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.339+0000 7f3db6630740 4 client.31675 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:56:07.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.339+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561678128210 con 0x561678238230 2024-03-26T17:56:07.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.339+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561678238230 0x56167823a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:07.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.339+0000 7f3d94ff9640 10 client.31675 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.339+0000 7f3db1f0c640 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561678238230 0x56167823a650 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dafc8c1288b7cf3 server_cookie=c87b1392ee9b3fa0 in_seq=0 out_seq=0 2024-03-26T17:56:07.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3d94ff9640 1 -- 172.21.15.105:0/155441686 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x561678128210 con 0x561678238230 2024-03-26T17:56:07.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3d94ff9640 10 client.31675 handle_command_reply: tid=0 2024-03-26T17:56:07.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 client.31675 shutdown 2024-03-26T17:56:07.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 msgr2=0x5616781ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781ca2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c036040 tx=0x7f3d9c04b820 comp rx=0 tx=0).stop 2024-03-26T17:56:07.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 shutdown_connections 2024-03-26T17:56:07.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561678238230 0x56167823a650 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5616781c1450 0x561678235fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5616781bc2e0 0x5616781ca2f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3d94ff9640 0 client.31675 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 --2- 172.21.15.105:0/155441686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616781bb8f0 0x5616781c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 >> 172.21.15.105:0/155441686 conn(0x5616781b6e60 msgr2=0x5616781ba0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 shutdown_connections 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.341+0000 7f3db6630740 1 -- 172.21.15.105:0/155441686 wait complete. 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db6630740 20 client.31675 trim_cache size 0 max 16384 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (142 inodes in the stack)", 2024-03-26T17:56:07.331 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:56:07.332 INFO:teuthology.orchestra.run.smithi105.stdout: "ba382905-caef-45a4-b2d3-7d53b115a6f7": { 2024-03-26T17:56:07.332 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:56:07.332 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "ba382905-caef-45a4-b2d3-7d53b115a6f7", 2024-03-26T17:56:07.332 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:56:07.332 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:56:07.332 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:56:07.332 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 -- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fa0 msgr2=0x7f3d980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fa0 0x7f3d980a5460 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3da80a43d0 tx=0x7f3da8009200 comp rx=0 tx=0).stop 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 -- 172.21.15.105:0/1466442656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1007c0 msgr2=0x7f3dac197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1007c0 0x7f3dac197af0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009d20 tx=0x7f3da800c6a0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 -- 172.21.15.105:0/1466442656 shutdown_connections 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3d980a2fa0 0x7f3d980a5460 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dac102370 0x7f3dac19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dac1019b0 0x7f3dac198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 --2- 172.21.15.105:0/1466442656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3dac1007c0 0x7f3dac197af0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 -- 172.21.15.105:0/1466442656 >> 172.21.15.105:0/1466442656 conn(0x7f3dac0fbd50 msgr2=0x7f3dac068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 -- 172.21.15.105:0/1466442656 shutdown_connections 2024-03-26T17:56:07.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:07.342+0000 7f3db396f640 1 -- 172.21.15.105:0/1466442656 wait complete. 2024-03-26T17:56:07.340 INFO:tasks.cephfs.filesystem:scrub status for tag:ba382905-caef-45a4-b2d3-7d53b115a6f7 - {'path': '/', 'tag': 'ba382905-caef-45a4-b2d3-7d53b115a6f7', 'options': 'recursive,force'} 2024-03-26T17:56:07.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:06 smithi144 ceph-mon[36095]: pgmap v882: 97 pgs: 97 active+clean; 287 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 3.1 MiB/s wr, 233 op/s 2024-03-26T17:56:07.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 38181 7.11 MB/sec execute 170 sec latency 196.925 ms 2024-03-26T17:56:08.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:56:08.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:07 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.243 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.244 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:07 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:56:08.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:08.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:08.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:08.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:08.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:08.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:08.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:07 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:08.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 38310 7.10 MB/sec execute 171 sec latency 118.937 ms 2024-03-26T17:56:09.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:08 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:09.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:08 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:09.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:08 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:09.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:08 smithi144 ceph-mon[36095]: pgmap v883: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-03-26T17:56:09.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:08 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:09.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:08 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:09.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:08 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:09.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:08 smithi105 ceph-mon[32280]: pgmap v883: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-03-26T17:56:09.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:08 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:09.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:08 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:09.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:08 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:56:09.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:08 smithi135 ceph-mon[35636]: pgmap v883: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-03-26T17:56:09.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 38422 7.08 MB/sec execute 172 sec latency 161.150 ms 2024-03-26T17:56:10.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: Unable to set osd_memory_target on smithi144 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-03-26T17:56:10.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:09 smithi144 ceph-mon[36095]: Unable to set osd_memory_target on smithi135 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:56:10.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-03-26T17:56:10.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: Unable to set osd_memory_target on smithi144 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:56:10.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-03-26T17:56:10.387 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:09 smithi105 ceph-mon[32280]: Unable to set osd_memory_target on smithi135 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:56:10.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi144 to 176.2M 2024-03-26T17:56:10.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: Unable to set osd_memory_target on smithi144 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:56:10.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T17:56:10.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: Adjusting osd_memory_target on smithi135 to 176.2M 2024-03-26T17:56:10.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:09 smithi135 ceph-mon[35636]: Unable to set osd_memory_target on smithi135 to 184768102: error parsing value: Value '184768102' is below minimum 939524096 2024-03-26T17:56:10.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 38570 7.09 MB/sec execute 173 sec latency 123.722 ms 2024-03-26T17:56:11.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:10 smithi144 ceph-mon[36095]: pgmap v884: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.6 MiB/s wr, 233 op/s 2024-03-26T17:56:11.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:10 smithi105 ceph-mon[32280]: pgmap v884: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.6 MiB/s wr, 233 op/s 2024-03-26T17:56:11.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:10 smithi135 ceph-mon[35636]: pgmap v884: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 2.6 MiB/s wr, 233 op/s 2024-03-26T17:56:11.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 38724 7.07 MB/sec execute 174 sec latency 121.449 ms 2024-03-26T17:56:12.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 38853 7.05 MB/sec execute 175 sec latency 122.543 ms 2024-03-26T17:56:13.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 39012 7.04 MB/sec execute 176 sec latency 110.968 ms 2024-03-26T17:56:14.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:13 smithi135 ceph-mon[35636]: pgmap v885: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-26T17:56:14.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:13 smithi144 ceph-mon[36095]: pgmap v885: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-26T17:56:14.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:13 smithi105 ceph-mon[32280]: pgmap v885: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-03-26T17:56:14.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 39141 7.03 MB/sec execute 177 sec latency 176.089 ms 2024-03-26T17:56:15.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 39269 7.02 MB/sec execute 178 sec latency 105.250 ms 2024-03-26T17:56:16.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:15 smithi135 ceph-mon[35636]: pgmap v886: 97 pgs: 97 active+clean; 283 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.9 MiB/s wr, 311 op/s 2024-03-26T17:56:16.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:15 smithi144 ceph-mon[36095]: pgmap v886: 97 pgs: 97 active+clean; 283 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.9 MiB/s wr, 311 op/s 2024-03-26T17:56:16.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:15 smithi105 ceph-mon[32280]: pgmap v886: 97 pgs: 97 active+clean; 283 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 2.9 MiB/s wr, 311 op/s 2024-03-26T17:56:16.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 39398 7.01 MB/sec execute 179 sec latency 114.701 ms 2024-03-26T17:56:17.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:16 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:17.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:16 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:17.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:16 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:56:17.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:16 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:17.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:16 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:17.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:16 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:56:17.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:16 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:56:17.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:16 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:56:17.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:16 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:56:17.552 INFO:tasks.workunit.client.0.smithi105.stdout: 10 39549 7.03 MB/sec execute 180 sec latency 124.006 ms 2024-03-26T17:56:18.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:17 smithi135 ceph-mon[35636]: pgmap v887: 97 pgs: 97 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-03-26T17:56:18.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:17 smithi144 ceph-mon[36095]: pgmap v887: 97 pgs: 97 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-03-26T17:56:18.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:17 smithi105 ceph-mon[32280]: pgmap v887: 97 pgs: 97 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-03-26T17:56:18.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 39717 7.06 MB/sec execute 181 sec latency 133.329 ms 2024-03-26T17:56:19.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:18 smithi144 ceph-mon[36095]: pgmap v888: 97 pgs: 97 active+clean; 286 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-03-26T17:56:19.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:18 smithi105 ceph-mon[32280]: pgmap v888: 97 pgs: 97 active+clean; 286 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-03-26T17:56:19.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:18 smithi135 ceph-mon[35636]: pgmap v888: 97 pgs: 97 active+clean; 286 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-03-26T17:56:19.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 39905 7.09 MB/sec execute 182 sec latency 118.132 ms 2024-03-26T17:56:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/339793549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 msgr2=0x7fbcc4109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/339793549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 0x7fbcc4109a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009ca0 tx=0x7fbcbc02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/339793549 shutdown_connections 2024-03-26T17:56:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/339793549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 0x7fbcc4109a10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/339793549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcc4104c90 0x7fbcc4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/339793549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4102360 0x7fbcc4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/339793549 >> 172.21.15.105:0/339793549 conn(0x7fbcc40fbd50 msgr2=0x7fbcc40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/339793549 shutdown_connections 2024-03-26T17:56:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.241+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/339793549 wait complete. 2024-03-26T17:56:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.242+0000 7fbcc8dc4640 1 Processor -- start 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.242+0000 7fbcc8dc4640 1 -- start start 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.242+0000 7fbcc8dc4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4102360 0x7fbcc4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.242+0000 7fbcc8dc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcc4104c90 0x7fbcc4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.242+0000 7fbcc8dc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 0x7fbcc419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.242+0000 7fbcc8dc4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc410d430 con 0x7fbcc4104c90 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.242+0000 7fbcc8dc4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc410d2b0 con 0x7fbcc4102360 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.242+0000 7fbcc8dc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc410d5b0 con 0x7fbcc41075c0 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 0x7fbcc419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 0x7fbcc419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49766/0 (socket says 172.21.15.105:49766) 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4102360 0x7fbcc4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 -- 172.21.15.105:0/1156005222 learned_addr learned my addr 172.21.15.105:0/1156005222 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcc4104c90 0x7fbcc4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 -- 172.21.15.105:0/1156005222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4102360 msgr2=0x7fbcc4197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4102360 0x7fbcc4197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 -- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcc4104c90 msgr2=0x7fbcc4197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcc4104c90 0x7fbcc4197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 -- 172.21.15.105:0/1156005222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc406b410 con 0x7fbcc41075c0 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.243+0000 7fbcc2d9d640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 0x7fbcc419efd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02c700 tx=0x7fbcbc0328a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.244+0000 7fbcbbfff640 1 -- 172.21.15.105:0/1156005222 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc041020 con 0x7fbcc41075c0 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.244+0000 7fbcbbfff640 1 -- 172.21.15.105:0/1156005222 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcbc003e20 con 0x7fbcc41075c0 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.244+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/1156005222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc406b6a0 con 0x7fbcc41075c0 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.244+0000 7fbcbbfff640 1 -- 172.21.15.105:0/1156005222 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc039040 con 0x7fbcc41075c0 2024-03-26T17:56:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.244+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/1156005222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc406bbe0 con 0x7fbcc41075c0 2024-03-26T17:56:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.247+0000 7fbcbbfff640 1 -- 172.21.15.105:0/1156005222 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbcbc044020 con 0x7fbcc41075c0 2024-03-26T17:56:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.247+0000 7fbccb9ac740 1 Processor -- start 2024-03-26T17:56:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.247+0000 7fbccb9ac740 1 -- start start 2024-03-26T17:56:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbb49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbb4fc0 0x561b2dbb9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbba130 0x561b2dbbc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561b2db190b0 con 0x561b2dbba130 2024-03-26T17:56:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561b2dac1960 con 0x561b2dbb4fc0 2024-03-26T17:56:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b2db15060 con 0x561b2dbb45d0 2024-03-26T17:56:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcbbfff640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbcb00a30c0 0x7fbcb00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbb4fc0 0x561b2dbb9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbb4fc0 0x561b2dbb9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40922/0 (socket says 172.21.15.105:40922) 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc259c640 1 -- 172.21.15.105:0/1628449626 learned_addr learned my addr 172.21.15.105:0/1628449626 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbb49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc2d9d640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbcb00a30c0 0x7fbcb00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbba130 0x561b2dbbc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc259c640 1 -- 172.21.15.105:0/1628449626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 msgr2=0x561b2dbb49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc259c640 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbb49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc259c640 1 -- 172.21.15.105:0/1628449626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbba130 msgr2=0x561b2dbbc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc259c640 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbba130 0x561b2dbbc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcc259c640 1 -- 172.21.15.105:0/1628449626 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2dad3c60 con 0x561b2dbb4fc0 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.248+0000 7fbcbbfff640 1 -- 172.21.15.105:0/1156005222 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbcbc0f15d0 con 0x7fbcc41075c0 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.249+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbba130 0x561b2dbbc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.249+0000 7fbcc259c640 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbb4fc0 0x561b2dbb9ac0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009d20 tx=0x7fbcb4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e33b871fa55de8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.249+0000 7fbcc2d9d640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbcb00a30c0 0x7fbcb00a5580 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0b9940 tx=0x7fbcbc0b97d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.249+0000 7fbcb8d99640 1 -- 172.21.15.105:0/1628449626 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb401e070 con 0x561b2dbb4fc0 2024-03-26T17:56:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.249+0000 7fbcb8d99640 1 -- 172.21.15.105:0/1628449626 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcb4019ae0 con 0x561b2dbb4fc0 2024-03-26T17:56:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.249+0000 7fbcb8d99640 1 -- 172.21.15.105:0/1628449626 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400ea80 con 0x561b2dbb4fc0 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1628449626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbb4fc0 msgr2=0x561b2dbb9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbb4fc0 0x561b2dbb9ac0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009d20 tx=0x7fbcb4011810 comp rx=0 tx=0).stop 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1628449626 shutdown_connections 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbba130 0x561b2dbbc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbb4fc0 0x561b2dbb9ac0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1628449626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbb49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1628449626 >> 172.21.15.105:0/1628449626 conn(0x561b2dbafb40 msgr2=0x561b2dbaff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1628449626 shutdown_connections 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1628449626 wait complete. 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.250+0000 7fbccb9ac740 1 -- start start 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbccb9ac740 1 Processor -- start 2024-03-26T17:56:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 0x561b2dbbe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbc5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561b2dbbd130 con 0x561b2dbb45d0 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbcb9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbcb9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51822/0 (socket says 172.21.15.105:51822) 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/1618422807 learned_addr learned my addr 172.21.15.105:0/1618422807 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbcc259c640 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 0x561b2dbbe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561b2dad4db0 con 0x561b2dbba130 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b2dbbd2b0 con 0x561b2dbb4fc0 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbccb9ac740 10 client.0 _ll_register_callbacks cb 0x561b2dad9110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.251+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbc5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/1618422807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 msgr2=0x561b2dbbe9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 0x561b2dbbe9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/1618422807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 msgr2=0x561b2dbc5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbc5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/1618422807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2d82d210 con 0x561b2dbb45d0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe460 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fbcac011990 tx=0x7fbcac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f544b0bc0c4f1810 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbca9ffb640 1 -- 172.21.15.105:0/1618422807 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac004240 con 0x561b2dbb45d0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcb4019ae0 con 0x561b2dbb45d0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbca9ffb640 1 -- 172.21.15.105:0/1618422807 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbcac0043e0 con 0x561b2dbb45d0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbca9ffb640 1 -- 172.21.15.105:0/1618422807 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbcac010950 con 0x561b2dbb45d0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbccb9ac740 10 client.21972 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.252+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561b2d8f9620 con 0x561b2dbb45d0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbca9ffb640 1 -- 172.21.15.105:0/1618422807 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcb4019ae0 con 0x561b2dbb45d0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbca9ffb640 1 -- 172.21.15.105:0/1618422807 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fbcac0283f0 con 0x561b2dbb45d0 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbccb9ac740 10 client.21972 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbccb9ac740 10 client.21972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbccb9ac740 20 client.21972 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561b2dbca640 0x561b2dbccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbccb9ac740 4 client.21972 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2db265f0 con 0x561b2dbca640 2024-03-26T17:56:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.253+0000 7fbcc259c640 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561b2dbca640 0x561b2dbccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.254+0000 7fbca9ffb640 10 client.21972 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.254+0000 7fbcc259c640 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561b2dbca640 0x561b2dbccae0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a03647a2165b4c5b server_cookie=baa1b934b48874f3 in_seq=0 out_seq=0 2024-03-26T17:56:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbca9ffb640 1 -- 172.21.15.105:0/1618422807 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561b2db265f0 con 0x561b2dbca640 2024-03-26T17:56:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbca9ffb640 10 client.21972 handle_command_reply: tid=0 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 client.21972 shutdown 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 msgr2=0x561b2dbbe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe460 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fbcac011990 tx=0x7fbcac004730 comp rx=0 tx=0).stop 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 shutdown_connections 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561b2dbca640 0x561b2dbccae0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbc5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbca9ffb640 0 client.21972 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 0x561b2dbbe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/1618422807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe460 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 >> 172.21.15.105:0/1618422807 conn(0x561b2dbafb40 msgr2=0x561b2dbaedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.256+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 shutdown_connections 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.257+0000 7fbccb9ac740 1 -- 172.21.15.105:0/1618422807 wait complete. 2024-03-26T17:56:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.257+0000 7fbccb9ac740 20 client.21972 trim_cache size 0 max 16384 2024-03-26T17:56:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.280+0000 7fbccb9ac740 1 Processor -- start 2024-03-26T17:56:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.280+0000 7fbccb9ac740 1 -- start start 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.280+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.280+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.280+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbca400e530 con 0x561b2dbb45d0 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbca4010070 con 0x561b2dbba130 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbca400ec00 con 0x561b2dbb4fc0 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51832/0 (socket says 172.21.15.105:51832) 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/2190034831 learned_addr learned my addr 172.21.15.105:0/2190034831 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcc259c640 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/2190034831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 msgr2=0x561b2dbc5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/2190034831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 msgr2=0x561b2dbae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/2190034831 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2dbaf8b0 con 0x561b2dbb45d0 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe900 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fbcac006010 tx=0x7fbcac0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06c074226f390d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcabfff640 1 -- 172.21.15.105:0/2190034831 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0284b0 con 0x561b2dbb45d0 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.281+0000 7fbcabfff640 1 -- 172.21.15.105:0/2190034831 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcac024030 con 0x561b2dbb45d0 2024-03-26T17:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbcabfff640 1 -- 172.21.15.105:0/2190034831 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0035f0 con 0x561b2dbb45d0 2024-03-26T17:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 -- 172.21.15.105:0/2190034831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 msgr2=0x561b2dbbe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe900 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fbcac006010 tx=0x7fbcac0098a0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 -- 172.21.15.105:0/2190034831 shutdown_connections 2024-03-26T17:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/2190034831 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb45d0 0x561b2dbbe900 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 -- 172.21.15.105:0/2190034831 >> 172.21.15.105:0/2190034831 conn(0x561b2dbafb40 msgr2=0x561b2dbb60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 -- 172.21.15.105:0/2190034831 shutdown_connections 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.282+0000 7fbccb9ac740 1 -- 172.21.15.105:0/2190034831 wait complete. 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 1 -- start start 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 1 Processor -- start 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbc2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbc3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbca400f9a0 con 0x561b2dbb4fc0 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbca400f2d0 con 0x561b2dbba130 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbca400c510 con 0x561b2dbb45d0 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbcb9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbc2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbcc259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbcb9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbc2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:49790/0 (socket says 172.21.15.105:49790) 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbcc259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51834/0 (socket says 172.21.15.105:51834) 2024-03-26T17:56:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbcb9d9b640 1 -- 172.21.15.105:0/3139894602 learned_addr learned my addr 172.21.15.105:0/3139894602 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbccb9ac740 10 client.0 _ll_register_callbacks cb 0x7fbcac009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.283+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbc3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbcc259c640 1 -- 172.21.15.105:0/3139894602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 msgr2=0x561b2dbc2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbcc259c640 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbc2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbcc259c640 1 -- 172.21.15.105:0/3139894602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 msgr2=0x561b2dbc3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbcc259c640 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbc3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbcc259c640 1 -- 172.21.15.105:0/3139894602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2dbc45c0 con 0x561b2dbb4fc0 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbc2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbcc259c640 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc2fc0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x561b2dbb0ab0 tx=0x7fbcb4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67a2a4ae2f7b51c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbca9ffb640 1 -- 172.21.15.105:0/3139894602 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb401e040 con 0x561b2dbb4fc0 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbca9ffb640 1 -- 172.21.15.105:0/3139894602 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbcb4005240 con 0x561b2dbb4fc0 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbccb9ac740 1 -- 172.21.15.105:0/3139894602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcac024030 con 0x561b2dbb4fc0 2024-03-26T17:56:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbca9ffb640 1 -- 172.21.15.105:0/3139894602 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbcb4005480 con 0x561b2dbb4fc0 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbccb9ac740 10 client.21978 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.284+0000 7fbccb9ac740 1 -- 172.21.15.105:0/3139894602 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561b2dbc5900 con 0x561b2dbb4fc0 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbca9ffb640 1 -- 172.21.15.105:0/3139894602 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcac024030 con 0x561b2dbb4fc0 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbca9ffb640 1 -- 172.21.15.105:0/3139894602 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fbcb4034470 con 0x561b2dbb4fc0 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbccb9ac740 10 client.21978 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbccb9ac740 10 client.21978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbccb9ac740 20 client.21978 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561b2dc30ca0 0x561b2dc33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbccb9ac740 4 client.21978 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbccb9ac740 1 -- 172.21.15.105:0/3139894602 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b2db20940 con 0x561b2dc30ca0 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.285+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561b2dc30ca0 0x561b2dc33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.286+0000 7fbcb9d9b640 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561b2dc30ca0 0x561b2dc33090 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea071e804aae1330 server_cookie=704c67760f09058b in_seq=0 out_seq=0 2024-03-26T17:56:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.286+0000 7fbca9ffb640 10 client.21978 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.423+0000 7fbca9ffb640 1 -- 172.21.15.105:0/3139894602 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b2db20940 con 0x561b2dc30ca0 2024-03-26T17:56:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.423+0000 7fbca9ffb640 10 client.21978 handle_command_reply: tid=0 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.423+0000 7fbccb9ac740 1 client.21978 shutdown 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 -- 172.21.15.105:0/3139894602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb4fc0 msgr2=0x561b2dbc2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc2fc0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x561b2dbb0ab0 tx=0x7fbcb4009f60 comp rx=0 tx=0).stop 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 -- 172.21.15.105:0/3139894602 shutdown_connections 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561b2dc30ca0 0x561b2dc33090 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbca9ffb640 0 client.21978 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b2dbba130 0x561b2dbc3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b2dbb4fc0 0x561b2dbc2fc0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 --2- 172.21.15.105:0/3139894602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b2dbb45d0 0x561b2dbc2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 -- 172.21.15.105:0/3139894602 >> 172.21.15.105:0/3139894602 conn(0x561b2dbafb40 msgr2=0x561b2dbb2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 -- 172.21.15.105:0/3139894602 shutdown_connections 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 1 -- 172.21.15.105:0/3139894602 wait complete. 2024-03-26T17:56:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.424+0000 7fbccb9ac740 20 client.21978 trim_cache size 0 max 16384 2024-03-26T17:56:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:56:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:56:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:56:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:56:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbcb00a30c0 msgr2=0x7fbcb00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbcb00a30c0 0x7fbcb00a5580 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0b9940 tx=0x7fbcbc0b97d0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/1156005222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 msgr2=0x7fbcc419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 0x7fbcc419efd0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc02c700 tx=0x7fbcbc0328a0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/1156005222 shutdown_connections 2024-03-26T17:56:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbcb00a30c0 0x7fbcb00a5580 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcc41075c0 0x7fbcc419efd0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcc4104c90 0x7fbcc4197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 --2- 172.21.15.105:0/1156005222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcc4102360 0x7fbcc4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/1156005222 >> 172.21.15.105:0/1156005222 conn(0x7fbcc40fbd50 msgr2=0x7fbcc4102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/1156005222 shutdown_connections 2024-03-26T17:56:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:20.425+0000 7fbcc8dc4640 1 -- 172.21.15.105:0/1156005222 wait complete. 2024-03-26T17:56:20.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 40060 7.11 MB/sec execute 183 sec latency 199.070 ms 2024-03-26T17:56:21.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 40237 7.13 MB/sec execute 184 sec latency 122.850 ms 2024-03-26T17:56:22.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:21 smithi135 ceph-mon[35636]: pgmap v889: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-03-26T17:56:22.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:21 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:56:22.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:21 smithi144 ceph-mon[36095]: pgmap v889: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-03-26T17:56:22.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:21 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:56:22.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:21 smithi105 ceph-mon[32280]: pgmap v889: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-03-26T17:56:22.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:21 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:56:22.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 40384 7.16 MB/sec execute 185 sec latency 108.973 ms 2024-03-26T17:56:23.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:22 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:56:23.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:22 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:56:23.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:22 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:56:23.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:22 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:56:23.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:22 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:56:23.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:22 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:56:23.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:22 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:56:23.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:22 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:56:23.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:22 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:56:23.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 40486 7.15 MB/sec execute 186 sec latency 208.088 ms 2024-03-26T17:56:24.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:23 smithi135 ceph-mon[35636]: pgmap v890: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-03-26T17:56:24.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:23 smithi144 ceph-mon[36095]: pgmap v890: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-03-26T17:56:24.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:23 smithi105 ceph-mon[32280]: pgmap v890: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-03-26T17:56:24.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 40554 7.14 MB/sec execute 187 sec latency 205.261 ms 2024-03-26T17:56:25.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 40693 7.18 MB/sec execute 188 sec latency 117.551 ms 2024-03-26T17:56:26.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:25 smithi135 ceph-mon[35636]: pgmap v891: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-26T17:56:26.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:25 smithi144 ceph-mon[36095]: pgmap v891: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-26T17:56:26.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:25 smithi105 ceph-mon[32280]: pgmap v891: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.7 MiB/s wr, 239 op/s 2024-03-26T17:56:26.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 40808 7.19 MB/sec execute 189 sec latency 122.487 ms 2024-03-26T17:56:27.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 40942 7.21 MB/sec execute 190 sec latency 148.482 ms 2024-03-26T17:56:28.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:27 smithi135 ceph-mon[35636]: pgmap v892: 97 pgs: 97 active+clean; 300 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 3.8 MiB/s wr, 222 op/s 2024-03-26T17:56:28.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:27 smithi144 ceph-mon[36095]: pgmap v892: 97 pgs: 97 active+clean; 300 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 3.8 MiB/s wr, 222 op/s 2024-03-26T17:56:28.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:27 smithi105 ceph-mon[32280]: pgmap v892: 97 pgs: 97 active+clean; 300 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 3.8 MiB/s wr, 222 op/s 2024-03-26T17:56:28.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41047 7.18 MB/sec execute 191 sec latency 139.667 ms 2024-03-26T17:56:29.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:28 smithi144 ceph-mon[36095]: pgmap v893: 97 pgs: 97 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-03-26T17:56:29.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:28 smithi105 ceph-mon[32280]: pgmap v893: 97 pgs: 97 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-03-26T17:56:29.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:28 smithi135 ceph-mon[35636]: pgmap v893: 97 pgs: 97 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 4.6 MiB/s wr, 235 op/s 2024-03-26T17:56:29.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41146 7.17 MB/sec execute 192 sec latency 129.461 ms 2024-03-26T17:56:30.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41260 7.17 MB/sec execute 193 sec latency 125.635 ms 2024-03-26T17:56:31.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41371 7.14 MB/sec execute 194 sec latency 140.865 ms 2024-03-26T17:56:32.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:31 smithi135 ceph-mon[35636]: pgmap v894: 97 pgs: 97 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 3.6 MiB/s wr, 196 op/s 2024-03-26T17:56:32.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:31 smithi144 ceph-mon[36095]: pgmap v894: 97 pgs: 97 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 3.6 MiB/s wr, 196 op/s 2024-03-26T17:56:32.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:31 smithi105 ceph-mon[32280]: pgmap v894: 97 pgs: 97 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 3.6 MiB/s wr, 196 op/s 2024-03-26T17:56:32.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41477 7.13 MB/sec execute 195 sec latency 140.776 ms 2024-03-26T17:56:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.561+0000 7f837464b640 1 -- 172.21.15.105:0/369728172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c1007e0 msgr2=0x7f836c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.561+0000 7f837464b640 1 --2- 172.21.15.105:0/369728172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c1007e0 0x7f836c100c00 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f8368009860 tx=0x7f836802bb90 comp rx=0 tx=0).stop 2024-03-26T17:56:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.561+0000 7f837464b640 1 -- 172.21.15.105:0/369728172 shutdown_connections 2024-03-26T17:56:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.561+0000 7f837464b640 1 --2- 172.21.15.105:0/369728172 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836c102390 0x7f836c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.561+0000 7f837464b640 1 --2- 172.21.15.105:0/369728172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836c1019d0 0x7f836c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.561+0000 7f837464b640 1 --2- 172.21.15.105:0/369728172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c1007e0 0x7f836c100c00 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.561+0000 7f837464b640 1 -- 172.21.15.105:0/369728172 >> 172.21.15.105:0/369728172 conn(0x7f836c0fbd50 msgr2=0x7f836c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.561+0000 7f837464b640 1 -- 172.21.15.105:0/369728172 shutdown_connections 2024-03-26T17:56:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.562+0000 7f837464b640 1 -- 172.21.15.105:0/369728172 wait complete. 2024-03-26T17:56:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.562+0000 7f837464b640 1 Processor -- start 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.562+0000 7f837464b640 1 -- start start 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f837464b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836c1007e0 0x7f836c069b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f837464b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836c1019d0 0x7f836c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f837464b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c102390 0x7f836c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f837464b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f836c10c380 con 0x7f836c102390 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f837464b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f836c10c200 con 0x7f836c1007e0 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f837464b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f836c10c500 con 0x7f836c1019d0 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f83723e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836c1007e0 0x7f836c069b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f8372be8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c102390 0x7f836c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f83723e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836c1007e0 0x7f836c069b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60146/0 (socket says 172.21.15.105:60146) 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f83723e7640 1 -- 172.21.15.105:0/3421559751 learned_addr learned my addr 172.21.15.105:0/3421559751 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f8371be6640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836c1019d0 0x7f836c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f8372be8640 1 -- 172.21.15.105:0/3421559751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836c1019d0 msgr2=0x7f836c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f8372be8640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836c1019d0 0x7f836c19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f8372be8640 1 -- 172.21.15.105:0/3421559751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836c1007e0 msgr2=0x7f836c069b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f8372be8640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836c1007e0 0x7f836c069b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.563+0000 7f8372be8640 1 -- 172.21.15.105:0/3421559751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836c19f750 con 0x7f836c102390 2024-03-26T17:56:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.564+0000 7f8372be8640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c102390 0x7f836c19ee90 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f836400be20 tx=0x7f8364009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.564+0000 7f83637fe640 1 -- 172.21.15.105:0/3421559751 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364019070 con 0x7f836c102390 2024-03-26T17:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.564+0000 7f83637fe640 1 -- 172.21.15.105:0/3421559751 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f836400cdb0 con 0x7f836c102390 2024-03-26T17:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.564+0000 7f837464b640 1 -- 172.21.15.105:0/3421559751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836c196650 con 0x7f836c102390 2024-03-26T17:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.565+0000 7f83637fe640 1 -- 172.21.15.105:0/3421559751 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8364019070 con 0x7f836c102390 2024-03-26T17:56:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.565+0000 7f837464b640 1 -- 172.21.15.105:0/3421559751 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836c196bd0 con 0x7f836c102390 2024-03-26T17:56:33.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41597 7.13 MB/sec execute 196 sec latency 130.060 ms 2024-03-26T17:56:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.568+0000 7f83637fe640 1 -- 172.21.15.105:0/3421559751 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f836401c070 con 0x7f836c102390 2024-03-26T17:56:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.568+0000 7f8377239740 1 Processor -- start 2024-03-26T17:56:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.568+0000 7f8377239740 1 -- start start 2024-03-26T17:56:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8377239740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb05d0 0x562f22cb09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8377239740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb0fc0 0x562f22cb5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8377239740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 0x562f22cb85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8377239740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562f22c150b0 con 0x562f22cb05d0 2024-03-26T17:56:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8377239740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562f22bbd960 con 0x562f22cb0fc0 2024-03-26T17:56:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8377239740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f22c11060 con 0x562f22cb6130 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8371be6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb05d0 0x562f22cb09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8371be6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb05d0 0x562f22cb09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33424/0 (socket says 172.21.15.105:33424) 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f8371be6640 1 -- 172.21.15.105:0/153827035 learned_addr learned my addr 172.21.15.105:0/153827035 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.569+0000 7f83723e7640 1 --2- 172.21.15.105:0/153827035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb0fc0 0x562f22cb5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f8371be6640 1 -- 172.21.15.105:0/153827035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 msgr2=0x562f22cb85c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f8371be6640 1 --2- 172.21.15.105:0/153827035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 0x562f22cb85c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f8371be6640 1 -- 172.21.15.105:0/153827035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb0fc0 msgr2=0x562f22cb5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f8371be6640 1 --2- 172.21.15.105:0/153827035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb0fc0 0x562f22cb5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f8371be6640 1 -- 172.21.15.105:0/153827035 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f22bcfc60 con 0x562f22cb05d0 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f83637fe640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83580a31d0 0x7f83580a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f8371be6640 1 --2- 172.21.15.105:0/153827035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb05d0 0x562f22cb09f0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f835c010d50 tx=0x7f835c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb9aa7339e6dda1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f83723e7640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83580a31d0 0x7f83580a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f83637fe640 1 -- 172.21.15.105:0/3421559751 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f83640ce7d0 con 0x7f836c102390 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.570+0000 7f8360bfe640 1 -- 172.21.15.105:0/153827035 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c023070 con 0x562f22cb05d0 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.571+0000 7f8360bfe640 1 -- 172.21.15.105:0/153827035 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f835c004310 con 0x562f22cb05d0 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.571+0000 7f8360bfe640 1 -- 172.21.15.105:0/153827035 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c003850 con 0x562f22cb05d0 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f83723e7640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83580a31d0 0x7f83580a5690 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f83680057c0 tx=0x7f83680349b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 -- 172.21.15.105:0/153827035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb05d0 msgr2=0x562f22cb09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 --2- 172.21.15.105:0/153827035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb05d0 0x562f22cb09f0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f835c010d50 tx=0x7f835c009260 comp rx=0 tx=0).stop 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 -- 172.21.15.105:0/153827035 shutdown_connections 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 --2- 172.21.15.105:0/153827035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 0x562f22cb85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 --2- 172.21.15.105:0/153827035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb0fc0 0x562f22cb5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 --2- 172.21.15.105:0/153827035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb05d0 0x562f22cb09f0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 -- 172.21.15.105:0/153827035 >> 172.21.15.105:0/153827035 conn(0x562f22cabb40 msgr2=0x562f22cabf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 -- 172.21.15.105:0/153827035 shutdown_connections 2024-03-26T17:56:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.572+0000 7f8377239740 1 -- 172.21.15.105:0/153827035 wait complete. 2024-03-26T17:56:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.573+0000 7f8377239740 1 -- start start 2024-03-26T17:56:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.573+0000 7f8377239740 1 Processor -- start 2024-03-26T17:56:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.573+0000 7f8377239740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 0x562f22cba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.573+0000 7f8377239740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.573+0000 7f8371be6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.573+0000 7f8371be6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cba930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33426/0 (socket says 172.21.15.105:33426) 2024-03-26T17:56:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.573+0000 7f8371be6640 1 -- 172.21.15.105:0/1642840335 learned_addr learned my addr 172.21.15.105:0/1642840335 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8377239740 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 0x562f22cc1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562f22cb90e0 con 0x562f22cb0fc0 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562f22bd0db0 con 0x562f22cb05d0 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562f22cb9260 con 0x562f22cb6130 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8372be8640 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 0x562f22cc1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8377239740 10 client.0 _ll_register_callbacks cb 0x562f22bd5110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8371be6640 1 -- 172.21.15.105:0/1642840335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 msgr2=0x562f22cc1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8371be6640 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 0x562f22cc1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8371be6640 1 -- 172.21.15.105:0/1642840335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 msgr2=0x562f22cba3f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8371be6640 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 0x562f22cba3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8371be6640 1 -- 172.21.15.105:0/1642840335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f22929210 con 0x562f22cb0fc0 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8371be6640 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cba930 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f835c005eb0 tx=0x7f835c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0a6de47b407376 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.574+0000 7f8355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.575+0000 7f8355ffb640 1 -- 172.21.15.105:0/1642840335 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c02e020 con 0x562f22cb0fc0 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.575+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f835c004310 con 0x562f22cb0fc0 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.575+0000 7f8355ffb640 1 -- 172.21.15.105:0/1642840335 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f835c004110 con 0x562f22cb0fc0 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.575+0000 7f8355ffb640 1 -- 172.21.15.105:0/1642840335 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f835c039050 con 0x562f22cb0fc0 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.576+0000 7f8377239740 10 client.21990 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.576+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562f229f5620 con 0x562f22cb0fc0 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.576+0000 7f8355ffb640 1 -- 172.21.15.105:0/1642840335 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f835c004310 con 0x562f22cb0fc0 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.576+0000 7f8355ffb640 1 -- 172.21.15.105:0/1642840335 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f835c029070 con 0x562f22cb0fc0 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.576+0000 7f8377239740 10 client.21990 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.577+0000 7f8377239740 10 client.21990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.577+0000 7f8377239740 20 client.21990 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.577+0000 7f8377239740 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562f22cc5ed0 0x562f22cc82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.577+0000 7f83723e7640 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562f22cc5ed0 0x562f22cc82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.578+0000 7f8377239740 4 client.21990 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.578+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f22c225f0 con 0x562f22cc5ed0 2024-03-26T17:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.578+0000 7f8355ffb640 10 client.21990 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.578+0000 7f83723e7640 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562f22cc5ed0 0x562f22cc82c0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84bbc3f3f38ed43a server_cookie=82469d9c4b12b444 in_seq=0 out_seq=0 2024-03-26T17:56:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8355ffb640 1 -- 172.21.15.105:0/1642840335 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562f22c225f0 con 0x562f22cc5ed0 2024-03-26T17:56:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8355ffb640 10 client.21990 handle_command_reply: tid=0 2024-03-26T17:56:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 client.21990 shutdown 2024-03-26T17:56:33.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 msgr2=0x562f22cba930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cba930 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f835c005eb0 tx=0x7f835c0056d0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 shutdown_connections 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562f22cc5ed0 0x562f22cc82c0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb6130 0x562f22cc1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8355ffb640 0 client.21990 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cba930 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 --2- 172.21.15.105:0/1642840335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 0x562f22cba3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.581+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 >> 172.21.15.105:0/1642840335 conn(0x562f22cabb40 msgr2=0x562f22caadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.582+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 shutdown_connections 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.582+0000 7f8377239740 1 -- 172.21.15.105:0/1642840335 wait complete. 2024-03-26T17:56:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.582+0000 7f8377239740 20 client.21990 trim_cache size 0 max 16384 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.604+0000 7f8377239740 1 Processor -- start 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8377239740 1 -- start start 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8377239740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 0x562f22cb5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8377239740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb0fc0 0x562f22cc1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8377239740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb6130 0x562f22cb5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8377239740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f836c1a6c10 con 0x562f22cb6130 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8377239740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f836c1a8750 con 0x562f22cb05d0 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8377239740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f836c1a72e0 con 0x562f22cb0fc0 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8372be8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb6130 0x562f22cb5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8372be8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb6130 0x562f22cb5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:33442/0 (socket says 172.21.15.105:33442) 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8372be8640 1 -- 172.21.15.105:0/2906477533 learned_addr learned my addr 172.21.15.105:0/2906477533 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8371be6640 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb0fc0 0x562f22cc1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f83723e7640 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 0x562f22cb5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8372be8640 1 -- 172.21.15.105:0/2906477533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb0fc0 msgr2=0x562f22cc1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8372be8640 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb0fc0 0x562f22cc1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8372be8640 1 -- 172.21.15.105:0/2906477533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 msgr2=0x562f22cb5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8372be8640 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 0x562f22cb5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.605+0000 7f8372be8640 1 -- 172.21.15.105:0/2906477533 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f22cbb520 con 0x562f22cb6130 2024-03-26T17:56:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.606+0000 7f8372be8640 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb6130 0x562f22cb5950 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f836400bdf0 tx=0x7f836409a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9b21e9dd34f6cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.606+0000 7f8357fff640 1 -- 172.21.15.105:0/2906477533 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83640bb0c0 con 0x562f22cb6130 2024-03-26T17:56:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.606+0000 7f8357fff640 1 -- 172.21.15.105:0/2906477533 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83640a38d0 con 0x562f22cb6130 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.606+0000 7f8357fff640 1 -- 172.21.15.105:0/2906477533 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83640a3440 con 0x562f22cb6130 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 -- 172.21.15.105:0/2906477533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb6130 msgr2=0x562f22cb5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb6130 0x562f22cb5950 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f836400bdf0 tx=0x7f836409a8d0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 -- 172.21.15.105:0/2906477533 shutdown_connections 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb6130 0x562f22cb5950 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb0fc0 0x562f22cc1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 --2- 172.21.15.105:0/2906477533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb05d0 0x562f22cb5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 -- 172.21.15.105:0/2906477533 >> 172.21.15.105:0/2906477533 conn(0x562f22cabb40 msgr2=0x562f22cba3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 -- 172.21.15.105:0/2906477533 shutdown_connections 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 -- 172.21.15.105:0/2906477533 wait complete. 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 -- start start 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.607+0000 7f8377239740 1 Processor -- start 2024-03-26T17:56:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8377239740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb05d0 0x562f22cbecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8377239740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cbf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8377239740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb6130 0x562f22cc0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8377239740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f836c1a8080 con 0x562f22cb0fc0 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8377239740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f836c1a79b0 con 0x562f22cb6130 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f83723e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb05d0 0x562f22cbecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8371be6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cbf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f83723e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb05d0 0x562f22cbecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39968/0 (socket says 172.21.15.105:39968) 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f83723e7640 1 -- 172.21.15.105:0/1043361523 learned_addr learned my addr 172.21.15.105:0/1043361523 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8377239740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f836c1a4bf0 con 0x562f22cb05d0 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8377239740 10 client.0 _ll_register_callbacks cb 0x7f835c021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.608+0000 7f8372be8640 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb6130 0x562f22cc0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8371be6640 1 -- 172.21.15.105:0/1043361523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb05d0 msgr2=0x562f22cbecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8371be6640 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb05d0 0x562f22cbecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8371be6640 1 -- 172.21.15.105:0/1043361523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb6130 msgr2=0x562f22cc0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8371be6640 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb6130 0x562f22cc0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8371be6640 1 -- 172.21.15.105:0/1043361523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f22cc0810 con 0x562f22cb0fc0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8371be6640 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cbf210 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f835c005f70 tx=0x7f835c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661348e2dfe0a9eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8355ffb640 1 -- 172.21.15.105:0/1043361523 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c042060 con 0x562f22cb0fc0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8355ffb640 1 -- 172.21.15.105:0/1043361523 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f835c02aaa0 con 0x562f22cb0fc0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8377239740 1 -- 172.21.15.105:0/1043361523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83640a38d0 con 0x562f22cb0fc0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8355ffb640 1 -- 172.21.15.105:0/1043361523 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f835c02e040 con 0x562f22cb0fc0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8377239740 10 client.21996 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.609+0000 7f8377239740 1 -- 172.21.15.105:0/1043361523 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562f22cc1b00 con 0x562f22cb0fc0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.610+0000 7f8355ffb640 1 -- 172.21.15.105:0/1043361523 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83640a38d0 con 0x562f22cb0fc0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.610+0000 7f8355ffb640 1 -- 172.21.15.105:0/1043361523 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f835c029040 con 0x562f22cb0fc0 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.610+0000 7f8377239740 10 client.21996 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.610+0000 7f8377239740 10 client.21996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.610+0000 7f8377239740 20 client.21996 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.610+0000 7f8377239740 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562f22cc5ed0 0x562f22d2e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.610+0000 7f8377239740 4 client.21996 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.610+0000 7f8377239740 1 -- 172.21.15.105:0/1043361523 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f22c1c940 con 0x562f22cc5ed0 2024-03-26T17:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.611+0000 7f83723e7640 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562f22cc5ed0 0x562f22d2e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.612+0000 7f83723e7640 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562f22cc5ed0 0x562f22d2e790 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79378f667872abb6 server_cookie=ff372d4582217e83 in_seq=0 out_seq=0 2024-03-26T17:56:33.600 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.612+0000 7f8355ffb640 10 client.21996 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8355ffb640 1 -- 172.21.15.105:0/1043361523 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f22c1c940 con 0x562f22cc5ed0 2024-03-26T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8355ffb640 10 client.21996 handle_command_reply: tid=0 2024-03-26T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 client.21996 shutdown 2024-03-26T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 -- 172.21.15.105:0/1043361523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 msgr2=0x562f22cbf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cbf210 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f835c005f70 tx=0x7f835c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 -- 172.21.15.105:0/1043361523 shutdown_connections 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562f22cc5ed0 0x562f22d2e790 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8355ffb640 0 client.21996 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562f22cb6130 0x562f22cc0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562f22cb0fc0 0x562f22cbf210 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 --2- 172.21.15.105:0/1043361523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562f22cb05d0 0x562f22cbecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 -- 172.21.15.105:0/1043361523 >> 172.21.15.105:0/1043361523 conn(0x562f22cabb40 msgr2=0x562f22cb5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 -- 172.21.15.105:0/1043361523 shutdown_connections 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.798+0000 7f8377239740 1 -- 172.21.15.105:0/1043361523 wait complete. 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.799+0000 7f8377239740 20 client.21996 trim_cache size 0 max 16384 2024-03-26T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:56:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:56:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:56:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:56:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.799+0000 7f837464b640 1 -- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83580a31d0 msgr2=0x7f83580a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.799+0000 7f837464b640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83580a31d0 0x7f83580a5690 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f83680057c0 tx=0x7f83680349b0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.799+0000 7f837464b640 1 -- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c102390 msgr2=0x7f836c19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.799+0000 7f837464b640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c102390 0x7f836c19ee90 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f836400be20 tx=0x7f8364009ef0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.800+0000 7f837464b640 1 -- 172.21.15.105:0/3421559751 shutdown_connections 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.800+0000 7f837464b640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f83580a31d0 0x7f83580a5690 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.800+0000 7f837464b640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f836c102390 0x7f836c19ee90 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.800+0000 7f837464b640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f836c1019d0 0x7f836c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.800+0000 7f837464b640 1 --2- 172.21.15.105:0/3421559751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f836c1007e0 0x7f836c069b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.800+0000 7f837464b640 1 -- 172.21.15.105:0/3421559751 >> 172.21.15.105:0/3421559751 conn(0x7f836c0fbd50 msgr2=0x7f836c104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.800+0000 7f837464b640 1 -- 172.21.15.105:0/3421559751 shutdown_connections 2024-03-26T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:33.800+0000 7f837464b640 1 -- 172.21.15.105:0/3421559751 wait complete. 2024-03-26T17:56:34.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:33 smithi135 ceph-mon[35636]: pgmap v895: 97 pgs: 97 active+clean; 306 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.9 MiB/s wr, 223 op/s 2024-03-26T17:56:34.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:33 smithi144 ceph-mon[36095]: pgmap v895: 97 pgs: 97 active+clean; 306 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.9 MiB/s wr, 223 op/s 2024-03-26T17:56:34.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:33 smithi105 ceph-mon[32280]: pgmap v895: 97 pgs: 97 active+clean; 306 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.9 MiB/s wr, 223 op/s 2024-03-26T17:56:34.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41716 7.12 MB/sec execute 197 sec latency 241.179 ms 2024-03-26T17:56:35.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41851 7.11 MB/sec execute 198 sec latency 166.672 ms 2024-03-26T17:56:36.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:35 smithi135 ceph-mon[35636]: pgmap v896: 97 pgs: 97 active+clean; 307 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-03-26T17:56:36.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:35 smithi144 ceph-mon[36095]: pgmap v896: 97 pgs: 97 active+clean; 307 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-03-26T17:56:36.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:35 smithi105 ceph-mon[32280]: pgmap v896: 97 pgs: 97 active+clean; 307 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.8 MiB/s wr, 205 op/s 2024-03-26T17:56:36.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 41955 7.08 MB/sec execute 199 sec latency 142.770 ms 2024-03-26T17:56:37.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:56:37.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:56:37.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 -- 172.21.15.105:0/334598381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40104c70 msgr2=0x7f4b40107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 --2- 172.21.15.105:0/334598381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40104c70 0x7f4b40107060 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009900 tx=0x7f4b2802bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:37.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 -- 172.21.15.105:0/334598381 shutdown_connections 2024-03-26T17:56:37.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 --2- 172.21.15.105:0/334598381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b401075a0 0x7f4b401099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 --2- 172.21.15.105:0/334598381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40104c70 0x7f4b40107060 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 --2- 172.21.15.105:0/334598381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b40102340 0x7f4b40104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 -- 172.21.15.105:0/334598381 >> 172.21.15.105:0/334598381 conn(0x7f4b400fbd50 msgr2=0x7f4b400fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:37.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 -- 172.21.15.105:0/334598381 shutdown_connections 2024-03-26T17:56:37.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.498+0000 7f4b4503d640 1 -- 172.21.15.105:0/334598381 wait complete. 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.499+0000 7f4b4503d640 1 Processor -- start 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.499+0000 7f4b4503d640 1 -- start start 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b4503d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40102340 0x7f4b40101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b4503d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40104c70 0x7f4b401002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b4503d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b401075a0 0x7f4b40100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b4503d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010d400 con 0x7f4b40102340 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b4503d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010d280 con 0x7f4b40104c70 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b4503d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4010d580 con 0x7f4b401075a0 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b3ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40102340 0x7f4b40101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b37fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40104c70 0x7f4b401002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b3ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40102340 0x7f4b40101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48198/0 (socket says 172.21.15.105:48198) 2024-03-26T17:56:37.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b3ed9d640 1 -- 172.21.15.105:0/3339739116 learned_addr learned my addr 172.21.15.105:0/3339739116 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b3f59e640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b401075a0 0x7f4b40100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b37fff640 1 -- 172.21.15.105:0/3339739116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b401075a0 msgr2=0x7f4b40100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b37fff640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b401075a0 0x7f4b40100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b37fff640 1 -- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40102340 msgr2=0x7f4b40101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b37fff640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40102340 0x7f4b40101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b37fff640 1 -- 172.21.15.105:0/3339739116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b401010f0 con 0x7f4b40104c70 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40102340 0x7f4b40101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.500+0000 7f4b37fff640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40104c70 0x7f4b401002f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4b28036040 tx=0x7f4b28005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.501+0000 7f4b3cd99640 1 -- 172.21.15.105:0/3339739116 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28043020 con 0x7f4b40104c70 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.501+0000 7f4b4503d640 1 -- 172.21.15.105:0/3339739116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b401a1c60 con 0x7f4b40104c70 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.501+0000 7f4b3cd99640 1 -- 172.21.15.105:0/3339739116 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b28004350 con 0x7f4b40104c70 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.501+0000 7f4b3cd99640 1 -- 172.21.15.105:0/3339739116 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28003ab0 con 0x7f4b40104c70 2024-03-26T17:56:37.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.501+0000 7f4b4503d640 1 -- 172.21.15.105:0/3339739116 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b401a21d0 con 0x7f4b40104c70 2024-03-26T17:56:37.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.504+0000 7f4b3cd99640 1 -- 172.21.15.105:0/3339739116 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4b28046020 con 0x7f4b40104c70 2024-03-26T17:56:37.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.504+0000 7f4b47c21740 1 Processor -- start 2024-03-26T17:56:37.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.504+0000 7f4b47c21740 1 -- start start 2024-03-26T17:56:37.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ad8f0 0x55ff162add10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ae2e0 0x55ff162b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b47c21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ff16212980 con 0x55ff162ae2e0 2024-03-26T17:56:37.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b47c21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ff161babe0 con 0x55ff162b3450 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b47c21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff1620e930 con 0x55ff162ad8f0 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b3f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ad8f0 0x55ff162add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b3f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ad8f0 0x55ff162add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39732/0 (socket says 172.21.15.105:39732) 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b3cd99640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b2c0a2fa0 0x7f4b2c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b3f59e640 1 -- 172.21.15.105:0/1158471227 learned_addr learned my addr 172.21.15.105:0/1158471227 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ae2e0 0x55ff162b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b37fff640 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b3f59e640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b2c0a2fa0 0x7f4b2c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b3cd99640 1 -- 172.21.15.105:0/3339739116 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4b280f1560 con 0x7f4b40104c70 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b37fff640 1 -- 172.21.15.105:0/1158471227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ad8f0 msgr2=0x55ff162add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b37fff640 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ad8f0 0x55ff162add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b37fff640 1 -- 172.21.15.105:0/1158471227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ae2e0 msgr2=0x55ff162b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b37fff640 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ae2e0 0x55ff162b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b37fff640 1 -- 172.21.15.105:0/1158471227 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff15ff07d0 con 0x55ff162b3450 2024-03-26T17:56:37.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.505+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ae2e0 0x55ff162b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:37.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.506+0000 7f4b37fff640 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162b58e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4b280eeff0 tx=0x7f4b28000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb6da9dc8bc8967 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:37.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.506+0000 7f4b3f59e640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b2c0a2fa0 0x7f4b2c0a5460 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009fd0 tx=0x7f4b38009ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:37.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.506+0000 7f4b357fa640 1 -- 172.21.15.105:0/1158471227 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280be350 con 0x55ff162b3450 2024-03-26T17:56:37.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.506+0000 7f4b357fa640 1 -- 172.21.15.105:0/1158471227 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b280bdc50 con 0x55ff162b3450 2024-03-26T17:56:37.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.506+0000 7f4b357fa640 1 -- 172.21.15.105:0/1158471227 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280ba2c0 con 0x55ff162b3450 2024-03-26T17:56:37.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 -- 172.21.15.105:0/1158471227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 msgr2=0x55ff162b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162b58e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4b280eeff0 tx=0x7f4b28000bc0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 -- 172.21.15.105:0/1158471227 shutdown_connections 2024-03-26T17:56:37.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162b58e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ae2e0 0x55ff162b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 --2- 172.21.15.105:0/1158471227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ad8f0 0x55ff162add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 -- 172.21.15.105:0/1158471227 >> 172.21.15.105:0/1158471227 conn(0x55ff162a8e60 msgr2=0x55ff162a9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 -- 172.21.15.105:0/1158471227 shutdown_connections 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 -- 172.21.15.105:0/1158471227 wait complete. 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 -- start start 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.507+0000 7f4b47c21740 1 Processor -- start 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ad8f0 0x55ff162b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162b3450 0x55ff162bee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b47c21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ff162b61c0 con 0x55ff162b3450 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b47c21740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ff161ce160 con 0x55ff162ad8f0 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b47c21740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ff162b6340 con 0x55ff162ae2e0 2024-03-26T17:56:37.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b47c21740 10 client.0 _ll_register_callbacks cb 0x55ff1620a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39748/0 (socket says 172.21.15.105:39748) 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 -- 172.21.15.105:0/2571350847 learned_addr learned my addr 172.21.15.105:0/2571350847 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b3f59e640 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162b3450 0x55ff162bee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ad8f0 0x55ff162b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 -- 172.21.15.105:0/2571350847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ad8f0 msgr2=0x55ff162b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ad8f0 0x55ff162b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 -- 172.21.15.105:0/2571350847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162b3450 msgr2=0x55ff162bee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162b3450 0x55ff162bee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 -- 172.21.15.105:0/2571350847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff15f263f0 con 0x55ff162ae2e0 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b37fff640 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162b7e30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4b280bd950 tx=0x7f4b280c6660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cdaa27bb858cb18 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.508+0000 7f4b267fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.509+0000 7f4b267fc640 1 -- 172.21.15.105:0/2571350847 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280c1da0 con 0x55ff162ae2e0 2024-03-26T17:56:37.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.509+0000 7f4b267fc640 1 -- 172.21.15.105:0/2571350847 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4b28003c50 con 0x55ff162ae2e0 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.509+0000 7f4b47c21740 1 -- 172.21.15.105:0/2571350847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b280bdc50 con 0x55ff162ae2e0 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.509+0000 7f4b267fc640 1 -- 172.21.15.105:0/2571350847 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4b280dc050 con 0x55ff162ae2e0 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.509+0000 7f4b47c21740 10 client.31720 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.509+0000 7f4b47c21740 1 -- 172.21.15.105:0/2571350847 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ff15ee60f0 con 0x55ff162ae2e0 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.509+0000 7f4b267fc640 1 -- 172.21.15.105:0/2571350847 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b280bdc50 con 0x55ff162ae2e0 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b267fc640 1 -- 172.21.15.105:0/2571350847 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f4b280d4030 con 0x55ff162ae2e0 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b47c21740 10 client.31720 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b47c21740 10 client.31720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b47c21740 20 client.31720 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ff162c3400 0x55ff162c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b47c21740 4 client.31720 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b47c21740 1 -- 172.21.15.105:0/2571350847 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff1621fec0 con 0x55ff162c3400 2024-03-26T17:56:37.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ff162c3400 0x55ff162c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b267fc640 10 client.31720 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:37.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.510+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ff162c3400 0x55ff162c57f0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d562b616b1b9d305 server_cookie=e9a8640b35e24dfd in_seq=0 out_seq=0 2024-03-26T17:56:37.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b267fc640 1 -- 172.21.15.105:0/2571350847 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ff1621fec0 con 0x55ff162c3400 2024-03-26T17:56:37.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b267fc640 10 client.31720 handle_command_reply: tid=0 2024-03-26T17:56:37.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 client.31720 shutdown 2024-03-26T17:56:37.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 -- 172.21.15.105:0/2571350847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 msgr2=0x55ff162b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162b7e30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4b280bd950 tx=0x7f4b280c6660 comp rx=0 tx=0).stop 2024-03-26T17:56:37.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 -- 172.21.15.105:0/2571350847 shutdown_connections 2024-03-26T17:56:37.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ff162c3400 0x55ff162c57f0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162b3450 0x55ff162bee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b267fc640 0 client.31720 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:37.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162b7e30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2571350847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ad8f0 0x55ff162b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 -- 172.21.15.105:0/2571350847 >> 172.21.15.105:0/2571350847 conn(0x55ff162a8e60 msgr2=0x55ff162a8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:37.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 -- 172.21.15.105:0/2571350847 shutdown_connections 2024-03-26T17:56:37.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 1 -- 172.21.15.105:0/2571350847 wait complete. 2024-03-26T17:56:37.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.512+0000 7f4b47c21740 20 client.31720 trim_cache size 0 max 16384 2024-03-26T17:56:37.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.540+0000 7f4b47c21740 1 Processor -- start 2024-03-26T17:56:37.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.540+0000 7f4b47c21740 1 -- start start 2024-03-26T17:56:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ae2e0 0x55ff162bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162b3450 0x55ff162afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b47c21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401aaf00 con 0x55ff162ad8f0 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b3ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b37fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ae2e0 0x55ff162bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b37fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ae2e0 0x55ff162bedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57642/0 (socket says 172.21.15.105:57642) 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b37fff640 1 -- 172.21.15.105:0/3101526361 learned_addr learned my addr 172.21.15.105:0/3101526361 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b47c21740 1 -- 172.21.15.105:0/3101526361 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401aca40 con 0x55ff162ae2e0 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b47c21740 1 -- 172.21.15.105:0/3101526361 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401ab5d0 con 0x55ff162b3450 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b3f59e640 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162b3450 0x55ff162afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b37fff640 1 -- 172.21.15.105:0/3101526361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162b3450 msgr2=0x55ff162afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b37fff640 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162b3450 0x55ff162afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b37fff640 1 -- 172.21.15.105:0/3101526361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 msgr2=0x55ff162af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:37.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b37fff640 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b37fff640 1 -- 172.21.15.105:0/3101526361 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff162aeb70 con 0x55ff162ae2e0 2024-03-26T17:56:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.541+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.542+0000 7f4b37fff640 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ae2e0 0x55ff162bedc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4b2802c170 tx=0x7f4b28004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6955593d22fb5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.542+0000 7f4b34ff9640 1 -- 172.21.15.105:0/3101526361 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280cb080 con 0x55ff162ae2e0 2024-03-26T17:56:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.542+0000 7f4b34ff9640 1 -- 172.21.15.105:0/3101526361 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b280f10d0 con 0x55ff162ae2e0 2024-03-26T17:56:37.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.542+0000 7f4b34ff9640 1 -- 172.21.15.105:0/3101526361 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280b87c0 con 0x55ff162ae2e0 2024-03-26T17:56:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 -- 172.21.15.105:0/3101526361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ae2e0 msgr2=0x55ff162bedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ae2e0 0x55ff162bedc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4b2802c170 tx=0x7f4b28004590 comp rx=0 tx=0).stop 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 -- 172.21.15.105:0/3101526361 shutdown_connections 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162b3450 0x55ff162afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162ae2e0 0x55ff162bedc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 --2- 172.21.15.105:0/3101526361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 -- 172.21.15.105:0/3101526361 >> 172.21.15.105:0/3101526361 conn(0x55ff162a8e60 msgr2=0x55ff162a9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 -- 172.21.15.105:0/3101526361 shutdown_connections 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.543+0000 7f4b47c21740 1 -- 172.21.15.105:0/3101526361 wait complete. 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b47c21740 1 -- start start 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b47c21740 1 Processor -- start 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162bc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b47c21740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162bd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b47c21740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401ac370 con 0x55ff162ad8f0 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b37fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162bc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b3ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b37fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162bc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39766/0 (socket says 172.21.15.105:39766) 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b37fff640 1 -- 172.21.15.105:0/2073855257 learned_addr learned my addr 172.21.15.105:0/2073855257 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401abca0 con 0x55ff162b3450 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.544+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4b401a8ee0 con 0x55ff162ae2e0 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b47c21740 10 client.0 _ll_register_callbacks cb 0x7f4b30009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b3f59e640 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162bd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b3ed9d640 1 -- 172.21.15.105:0/2073855257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 msgr2=0x55ff162bc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162bc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b3ed9d640 1 -- 172.21.15.105:0/2073855257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 msgr2=0x55ff162bd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162bd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b3ed9d640 1 -- 172.21.15.105:0/2073855257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff162bdf00 con 0x55ff162ad8f0 2024-03-26T17:56:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b3ed9d640 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162bc410 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x55ff162c6030 tx=0x7f4b30010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcc87dd26a6b348 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.545+0000 7f4b267fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.546+0000 7f4b267fc640 1 -- 172.21.15.105:0/2073855257 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000fde0 con 0x55ff162ad8f0 2024-03-26T17:56:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.546+0000 7f4b267fc640 1 -- 172.21.15.105:0/2073855257 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f4b30005050 con 0x55ff162ad8f0 2024-03-26T17:56:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.546+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b280f10d0 con 0x55ff162ad8f0 2024-03-26T17:56:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.546+0000 7f4b267fc640 1 -- 172.21.15.105:0/2073855257 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4b30005250 con 0x55ff162ad8f0 2024-03-26T17:56:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.546+0000 7f4b47c21740 10 client.22014 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:37.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.546+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ff163296c0 con 0x55ff162ad8f0 2024-03-26T17:56:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.546+0000 7f4b267fc640 1 -- 172.21.15.105:0/2073855257 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b30004c50 con 0x55ff162ad8f0 2024-03-26T17:56:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b267fc640 1 -- 172.21.15.105:0/2073855257 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f4b30020330 con 0x55ff162ad8f0 2024-03-26T17:56:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b47c21740 10 client.22014 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b47c21740 10 client.22014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b47c21740 20 client.22014 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ff16329f50 0x55ff1632c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b47c21740 4 client.22014 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4b280f10d0 con 0x55ff16329f50 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b37fff640 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ff16329f50 0x55ff1632c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b267fc640 10 client.22014 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.547+0000 7f4b37fff640 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ff16329f50 0x55ff1632c370 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef7de846a84d347 server_cookie=db50fd48d709038e in_seq=0 out_seq=0 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b267fc640 1 -- 172.21.15.105:0/2073855257 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f4b280f10d0 con 0x55ff16329f50 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b267fc640 10 client.22014 handle_command_reply: tid=0 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 client.22014 shutdown 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 msgr2=0x55ff162bc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162bc410 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x55ff162c6030 tx=0x7f4b30010ee0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 shutdown_connections 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ff16329f50 0x55ff1632c370 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ff162b3450 0x55ff162bd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ff162ae2e0 0x55ff162bc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b267fc640 0 client.22014 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 --2- 172.21.15.105:0/2073855257 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ff162ad8f0 0x55ff162bc410 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 >> 172.21.15.105:0/2073855257 conn(0x55ff162a8e60 msgr2=0x55ff162af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 shutdown_connections 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 1 -- 172.21.15.105:0/2073855257 wait complete. 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.550+0000 7f4b47c21740 20 client.22014 trim_cache size 0 max 16384 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:56:37.539 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:56:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 -- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b2c0a2fa0 msgr2=0x7f4b2c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b2c0a2fa0 0x7f4b2c0a5460 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009fd0 tx=0x7f4b38009ec0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 -- 172.21.15.105:0/3339739116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40104c70 msgr2=0x7f4b401002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40104c70 0x7f4b401002f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4b28036040 tx=0x7f4b28005dc0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 -- 172.21.15.105:0/3339739116 shutdown_connections 2024-03-26T17:56:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4b2c0a2fa0 0x7f4b2c0a5460 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b401075a0 0x7f4b40100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4b40104c70 0x7f4b401002f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 --2- 172.21.15.105:0/3339739116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b40102340 0x7f4b40101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 -- 172.21.15.105:0/3339739116 >> 172.21.15.105:0/3339739116 conn(0x7f4b400fbd50 msgr2=0x7f4b401041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 -- 172.21.15.105:0/3339739116 shutdown_connections 2024-03-26T17:56:37.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.551+0000 7f4b4503d640 1 -- 172.21.15.105:0/3339739116 wait complete. 2024-03-26T17:56:37.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:56:37.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:56:37.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 42090 7.08 MB/sec execute 200 sec latency 139.785 ms 2024-03-26T17:56:37.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/3502129259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 msgr2=0x7fbd84109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/3502129259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 0x7fbd84109d30 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009ca0 tx=0x7fbd7c02c120 comp rx=0 tx=0).stop 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/3502129259 shutdown_connections 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/3502129259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 0x7fbd84109d30 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/3502129259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd84102ae0 0x7fbd84102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/3502129259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd841018f0 0x7fbd84101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/3502129259 >> 172.21.15.105:0/3502129259 conn(0x7fbd840fce60 msgr2=0x7fbd840ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/3502129259 shutdown_connections 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.709+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/3502129259 wait complete. 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd8b4dc640 1 Processor -- start 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd8b4dc640 1 -- start start 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd8b4dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd841018f0 0x7fbd84075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd8b4dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd84102ae0 0x7fbd84073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd8b4dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 0x7fbd840741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd8b4dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8410d490 con 0x7fbd841034a0 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd8b4dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8410d310 con 0x7fbd841018f0 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd8b4dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8410d610 con 0x7fbd84102ae0 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd89a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 0x7fbd840741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd89a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 0x7fbd840741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48236/0 (socket says 172.21.15.105:48236) 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd89a79640 1 -- 172.21.15.105:0/2250743478 learned_addr learned my addr 172.21.15.105:0/2250743478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.710+0000 7fbd88a77640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd84102ae0 0x7fbd84073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd89a79640 1 -- 172.21.15.105:0/2250743478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd84102ae0 msgr2=0x7fbd84073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd89a79640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd84102ae0 0x7fbd84073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd89a79640 1 -- 172.21.15.105:0/2250743478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd841018f0 msgr2=0x7fbd84075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd89a79640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd841018f0 0x7fbd84075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd89a79640 1 -- 172.21.15.105:0/2250743478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd84074a80 con 0x7fbd841034a0 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd89a79640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 0x7fbd840741f0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c02c730 tx=0x7fbd7c0347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd7a7fc640 1 -- 172.21.15.105:0/2250743478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c039040 con 0x7fbd841034a0 2024-03-26T17:56:37.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd7a7fc640 1 -- 172.21.15.105:0/2250743478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd7c044070 con 0x7fbd841034a0 2024-03-26T17:56:37.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd7a7fc640 1 -- 172.21.15.105:0/2250743478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0042a0 con 0x7fbd841034a0 2024-03-26T17:56:37.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd841a1c00 con 0x7fbd841034a0 2024-03-26T17:56:37.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.711+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd841a2150 con 0x7fbd841034a0 2024-03-26T17:56:37.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.712+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd84108c10 con 0x7fbd841034a0 2024-03-26T17:56:37.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.713+0000 7fbd7a7fc640 1 -- 172.21.15.105:0/2250743478 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbd7c003970 con 0x7fbd841034a0 2024-03-26T17:56:37.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.714+0000 7fbd7a7fc640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbd700a2f20 0x7fbd700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:37.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.714+0000 7fbd7a7fc640 1 -- 172.21.15.105:0/2250743478 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fbd7c0f18c0 con 0x7fbd841034a0 2024-03-26T17:56:37.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.714+0000 7fbd89278640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbd700a2f20 0x7fbd700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:37.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.715+0000 7fbd89278640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbd700a2f20 0x7fbd700a53e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbd84102940 tx=0x7fbd80006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:37.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:37.721+0000 7fbd7a7fc640 1 -- 172.21.15.105:0/2250743478 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbd7c0be740 con 0x7fbd841034a0 2024-03-26T17:56:38.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.065+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd84062560 con 0x7fbd841034a0 2024-03-26T17:56:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.066+0000 7fbd7a7fc640 1 -- 172.21.15.105:0/2250743478 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11941 (secure 0 0 0) 0x7fbd7c0be040 con 0x7fbd841034a0 2024-03-26T17:56:38.052 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:56:38.052 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:54:01.817010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_29155":{"gid":29155,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3150554209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3150554209},{"type":"v1","addr":"172.21.15.135:6835","nonce":3150554209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:56:38.052 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 37 2024-03-26T17:56:38.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbd700a2f20 msgr2=0x7fbd700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbd700a2f20 0x7fbd700a53e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbd84102940 tx=0x7fbd80006c40 comp rx=0 tx=0).stop 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 msgr2=0x7fbd840741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 0x7fbd840741f0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c02c730 tx=0x7fbd7c0347b0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 shutdown_connections 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbd700a2f20 0x7fbd700a53e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd841034a0 0x7fbd840741f0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd84102ae0 0x7fbd84073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 --2- 172.21.15.105:0/2250743478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbd841018f0 0x7fbd84075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 >> 172.21.15.105:0/2250743478 conn(0x7fbd840fce60 msgr2=0x7fbd84105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 shutdown_connections 2024-03-26T17:56:38.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.068+0000 7fbd8b4dc640 1 -- 172.21.15.105:0/2250743478 wait complete. 2024-03-26T17:56:38.069 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:56:38.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:37 smithi135 ceph-mon[35636]: pgmap v897: 97 pgs: 97 active+clean; 304 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 3.3 MiB/s wr, 204 op/s 2024-03-26T17:56:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.216+0000 7f67b744b640 1 -- 172.21.15.105:0/2002186105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b0102370 msgr2=0x7f67b0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.216+0000 7f67b744b640 1 --2- 172.21.15.105:0/2002186105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b0102370 0x7f67b0108c00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009ca0 tx=0x7f67a802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.216+0000 7f67b744b640 1 -- 172.21.15.105:0/2002186105 shutdown_connections 2024-03-26T17:56:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.216+0000 7f67b744b640 1 --2- 172.21.15.105:0/2002186105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b0102370 0x7f67b0108c00 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.216+0000 7f67b744b640 1 --2- 172.21.15.105:0/2002186105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67b01019b0 0x7f67b0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.216+0000 7f67b744b640 1 --2- 172.21.15.105:0/2002186105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01007c0 0x7f67b0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.216+0000 7f67b744b640 1 -- 172.21.15.105:0/2002186105 >> 172.21.15.105:0/2002186105 conn(0x7f67b00fbd50 msgr2=0x7f67b00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.217+0000 7f67b744b640 1 -- 172.21.15.105:0/2002186105 shutdown_connections 2024-03-26T17:56:38.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.217+0000 7f67b744b640 1 -- 172.21.15.105:0/2002186105 wait complete. 2024-03-26T17:56:38.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.217+0000 7f67b744b640 1 Processor -- start 2024-03-26T17:56:38.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.217+0000 7f67b744b640 1 -- start start 2024-03-26T17:56:38.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b744b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b01007c0 0x7f67b0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b744b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01019b0 0x7f67b019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b744b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67b0102370 0x7f67b019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b744b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f67b010c370 con 0x7f67b01007c0 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b744b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67b010c1f0 con 0x7f67b0102370 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b744b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67b010c4f0 con 0x7f67b01019b0 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b01007c0 0x7f67b0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b01007c0 0x7f67b0069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48256/0 (socket says 172.21.15.105:48256) 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 -- 172.21.15.105:0/397106283 learned_addr learned my addr 172.21.15.105:0/397106283 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 -- 172.21.15.105:0/397106283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01019b0 msgr2=0x7f67b019c8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01019b0 0x7f67b019c8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 -- 172.21.15.105:0/397106283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67b0102370 msgr2=0x7f67b019eec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67b0102370 0x7f67b019eec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 -- 172.21.15.105:0/397106283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b019f7d0 con 0x7f67b01007c0 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67b51e7640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b01007c0 0x7f67b0069590 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9c0 tx=0x7f67ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67a67fc640 1 -- 172.21.15.105:0/397106283 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00cd80 con 0x7f67b01007c0 2024-03-26T17:56:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67a67fc640 1 -- 172.21.15.105:0/397106283 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67ac0035f0 con 0x7f67b01007c0 2024-03-26T17:56:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.218+0000 7f67a67fc640 1 -- 172.21.15.105:0/397106283 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac010690 con 0x7f67b01007c0 2024-03-26T17:56:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.219+0000 7f67b744b640 1 -- 172.21.15.105:0/397106283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b019fa60 con 0x7f67b01007c0 2024-03-26T17:56:38.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.219+0000 7f67b744b640 1 -- 172.21.15.105:0/397106283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b01a0030 con 0x7f67b01007c0 2024-03-26T17:56:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.222+0000 7f67a67fc640 1 -- 172.21.15.105:0/397106283 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f67ac0037a0 con 0x7f67b01007c0 2024-03-26T17:56:38.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67ba038740 1 Processor -- start 2024-03-26T17:56:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67ba038740 1 -- start start 2024-03-26T17:56:38.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67ba038740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb21c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67ba038740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb22220 0x55737bb267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67ba038740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb26e20 0x55737bb29290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67ba038740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55737ba7d970 con 0x55737bb26e20 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67ba038740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55737ba25d70 con 0x55737bb22220 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67ba038740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55737ba79920 con 0x55737bb21830 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.223+0000 7f67a67fc640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f679c0a2fa0 0x7f679c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb21c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb21c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39806/0 (socket says 172.21.15.105:39806) 2024-03-26T17:56:38.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 -- 172.21.15.105:0/942420728 learned_addr learned my addr 172.21.15.105:0/942420728 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f679c0a2fa0 0x7f679c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67a67fc640 1 -- 172.21.15.105:0/397106283 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f67ac0cfa80 con 0x7f67b01007c0 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 -- 172.21.15.105:0/942420728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb22220 msgr2=0x55737bb267b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 --2- 172.21.15.105:0/942420728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb22220 0x55737bb267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 -- 172.21.15.105:0/942420728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb26e20 msgr2=0x55737bb29290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 --2- 172.21.15.105:0/942420728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb26e20 0x55737bb29290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.224+0000 7f67b59e8640 1 -- 172.21.15.105:0/942420728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55737b7913f0 con 0x55737bb21830 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.225+0000 7f67b59e8640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f679c0a2fa0 0x7f679c0a5460 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f67a8032420 tx=0x7f67a8040000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.225+0000 7f67b59e8640 1 --2- 172.21.15.105:0/942420728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb21c50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f67a8032550 tx=0x7f67a8032e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c537eb60a1c0b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.225+0000 7f679b7fe640 1 -- 172.21.15.105:0/942420728 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8039040 con 0x55737bb21830 2024-03-26T17:56:38.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.225+0000 7f679b7fe640 1 -- 172.21.15.105:0/942420728 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67a8047070 con 0x55737bb21830 2024-03-26T17:56:38.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.225+0000 7f679b7fe640 1 -- 172.21.15.105:0/942420728 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8004700 con 0x55737bb21830 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 -- 172.21.15.105:0/942420728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 msgr2=0x55737bb21c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 --2- 172.21.15.105:0/942420728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb21c50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f67a8032550 tx=0x7f67a8032e40 comp rx=0 tx=0).stop 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 -- 172.21.15.105:0/942420728 shutdown_connections 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 --2- 172.21.15.105:0/942420728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb26e20 0x55737bb29290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 --2- 172.21.15.105:0/942420728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb22220 0x55737bb267b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 --2- 172.21.15.105:0/942420728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb21c50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 -- 172.21.15.105:0/942420728 >> 172.21.15.105:0/942420728 conn(0x55737bb13f20 msgr2=0x55737bb14320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 -- 172.21.15.105:0/942420728 shutdown_connections 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 -- 172.21.15.105:0/942420728 wait complete. 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 -- start start 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.226+0000 7f67ba038740 1 Processor -- start 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67ba038740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb21830 0x55737bb2b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67ba038740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb22220 0x55737bb2bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67ba038740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 0x55737bb32d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67ba038740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55737bb29d50 con 0x55737bb21830 2024-03-26T17:56:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67ba038740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55737ba39310 con 0x55737bb26e20 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67ba038740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55737bb29ed0 con 0x55737bb22220 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67ba038740 10 client.0 _ll_register_callbacks cb 0x55737ba74c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67b51e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb21830 0x55737bb2b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67b49e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb22220 0x55737bb2bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67b59e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 0x55737bb32d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67b49e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb22220 0x55737bb2bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39814/0 (socket says 172.21.15.105:39814) 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.227+0000 7f67b49e6640 1 -- 172.21.15.105:0/2022114030 learned_addr learned my addr 172.21.15.105:0/2022114030 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67b49e6640 1 -- 172.21.15.105:0/2022114030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 msgr2=0x55737bb32d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67b49e6640 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 0x55737bb32d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67b49e6640 1 -- 172.21.15.105:0/2022114030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb21830 msgr2=0x55737bb2b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67b49e6640 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb21830 0x55737bb2b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67b49e6640 1 -- 172.21.15.105:0/2022114030 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55737b85b7d0 con 0x55737bb22220 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67b49e6640 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb22220 0x55737bb2bcf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009dd0 tx=0x7f67a000b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ac7df25c534fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f6798ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f6798ff9640 1 -- 172.21.15.105:0/2022114030 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0015020 con 0x55737bb22220 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67ba038740 1 -- 172.21.15.105:0/2022114030 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a8047070 con 0x55737bb22220 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f6798ff9640 1 -- 172.21.15.105:0/2022114030 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f67a0011420 con 0x55737bb22220 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f6798ff9640 1 -- 172.21.15.105:0/2022114030 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f67a0020050 con 0x55737bb22220 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67ba038740 10 client.31738 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.228+0000 7f67ba038740 1 -- 172.21.15.105:0/2022114030 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55737bb361c0 con 0x55737bb22220 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.229+0000 7f6798ff9640 1 -- 172.21.15.105:0/2022114030 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67a8047070 con 0x55737bb22220 2024-03-26T17:56:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.229+0000 7f6798ff9640 1 -- 172.21.15.105:0/2022114030 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f67a00197f0 con 0x55737bb22220 2024-03-26T17:56:38.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.229+0000 7f67ba038740 10 client.31738 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.229+0000 7f67ba038740 10 client.31738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.230+0000 7f67ba038740 20 client.31738 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.230+0000 7f67ba038740 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55737bb37830 0x55737bb39c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.230+0000 7f67ba038740 4 client.31738 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.230+0000 7f67ba038740 1 -- 172.21.15.105:0/2022114030 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55737ba8aeb0 con 0x55737bb37830 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.230+0000 7f67b51e7640 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55737bb37830 0x55737bb39c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.231+0000 7f67b51e7640 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55737bb37830 0x55737bb39c60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f141a283d3264fc3 server_cookie=e7e59e3b885befb4 in_seq=0 out_seq=0 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.231+0000 7f6798ff9640 10 client.31738 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.231+0000 7f6798ff9640 1 -- 172.21.15.105:0/2022114030 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55737ba8aeb0 con 0x55737bb37830 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.231+0000 7f6798ff9640 10 client.31738 handle_command_reply: tid=0 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.231+0000 7f67ba038740 1 client.31738 shutdown 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 -- 172.21.15.105:0/2022114030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb22220 msgr2=0x55737bb2bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb22220 0x55737bb2bcf0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009dd0 tx=0x7f67a000b4b0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 -- 172.21.15.105:0/2022114030 shutdown_connections 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55737bb37830 0x55737bb39c60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f6798ff9640 0 client.31738 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 0x55737bb32d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb22220 0x55737bb2bcf0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 --2- 172.21.15.105:0/2022114030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb21830 0x55737bb2b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 -- 172.21.15.105:0/2022114030 >> 172.21.15.105:0/2022114030 conn(0x55737bb13f20 msgr2=0x55737bb20ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 -- 172.21.15.105:0/2022114030 shutdown_connections 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 1 -- 172.21.15.105:0/2022114030 wait complete. 2024-03-26T17:56:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.232+0000 7f67ba038740 20 client.31738 trim_cache size 0 max 16384 2024-03-26T17:56:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.250+0000 7f67ba038740 1 Processor -- start 2024-03-26T17:56:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.250+0000 7f67ba038740 1 -- start start 2024-03-26T17:56:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.250+0000 7f67ba038740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb14300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.250+0000 7f67ba038740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67ba038740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 0x55737bb20630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67ba038740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01a6c80 con 0x55737bb22220 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67ba038740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01a87c0 con 0x55737bb26e20 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67ba038740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01a7350 con 0x55737bb21830 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b49e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb14300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b49e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb32cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48284/0 (socket says 172.21.15.105:48284) 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb14300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39816/0 (socket says 172.21.15.105:39816) 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 -- 172.21.15.105:0/3381798361 learned_addr learned my addr 172.21.15.105:0/3381798361 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b59e8640 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 0x55737bb20630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 -- 172.21.15.105:0/3381798361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 msgr2=0x55737bb20630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 0x55737bb20630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 -- 172.21.15.105:0/3381798361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 msgr2=0x55737bb32cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb32cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 -- 172.21.15.105:0/3381798361 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55737bb0fa10 con 0x55737bb21830 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b49e6640 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f67b51e7640 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb14300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f67ac098ae0 tx=0x7f67ac009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dbc59679a338739 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.251+0000 7f679affd640 1 -- 172.21.15.105:0/3381798361 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac09c110 con 0x55737bb21830 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f679affd640 1 -- 172.21.15.105:0/3381798361 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67ac0a0260 con 0x55737bb21830 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f679affd640 1 -- 172.21.15.105:0/3381798361 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac098780 con 0x55737bb21830 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 -- 172.21.15.105:0/3381798361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 msgr2=0x55737bb14300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb14300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f67ac098ae0 tx=0x7f67ac009740 comp rx=0 tx=0).stop 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 -- 172.21.15.105:0/3381798361 shutdown_connections 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb26e20 0x55737bb20630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 --2- 172.21.15.105:0/3381798361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb21830 0x55737bb14300 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 -- 172.21.15.105:0/3381798361 >> 172.21.15.105:0/3381798361 conn(0x55737bb13f20 msgr2=0x55737bb2a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 -- 172.21.15.105:0/3381798361 shutdown_connections 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.252+0000 7f67ba038740 1 -- 172.21.15.105:0/3381798361 wait complete. 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 1 -- start start 2024-03-26T17:56:38.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 1 Processor -- start 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb21830 0x55737bb30590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb30ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb26e20 0x55737bb31ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01a80f0 con 0x55737bb22220 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01a7a20 con 0x55737bb21830 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67b01a4c60 con 0x55737bb26e20 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67ba038740 10 client.0 _ll_register_callbacks cb 0x7f67a0006ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67b51e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb21830 0x55737bb30590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67b59e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb26e20 0x55737bb31ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67b51e7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb21830 0x55737bb30590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57720/0 (socket says 172.21.15.105:57720) 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67b51e7640 1 -- 172.21.15.105:0/4181793581 learned_addr learned my addr 172.21.15.105:0/4181793581 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.253+0000 7f67b49e6640 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb30ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f67b59e8640 1 -- 172.21.15.105:0/4181793581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb21830 msgr2=0x55737bb30590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f67b59e8640 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb21830 0x55737bb30590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f67b59e8640 1 -- 172.21.15.105:0/4181793581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 msgr2=0x55737bb30ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f67b59e8640 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb30ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f67b59e8640 1 -- 172.21.15.105:0/4181793581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55737bb320d0 con 0x55737bb26e20 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f67b59e8640 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb26e20 0x55737bb31ae0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f67a802c610 tx=0x7f67a8004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccd27aa4e99d4548 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f6798ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f6798ff9640 1 -- 172.21.15.105:0/4181793581 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8039070 con 0x55737bb26e20 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f6798ff9640 1 -- 172.21.15.105:0/4181793581 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f67a8032910 con 0x55737bb26e20 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.254+0000 7f67ba038740 1 -- 172.21.15.105:0/4181793581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ac0a0260 con 0x55737bb26e20 2024-03-26T17:56:38.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f6798ff9640 1 -- 172.21.15.105:0/4181793581 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f67a8004050 con 0x55737bb26e20 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67ba038740 10 client.31744 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67ba038740 1 -- 172.21.15.105:0/4181793581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55737bb9dd20 con 0x55737bb26e20 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f6798ff9640 1 -- 172.21.15.105:0/4181793581 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67ac0a0260 con 0x55737bb26e20 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f6798ff9640 1 -- 172.21.15.105:0/4181793581 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f67a8034560 con 0x55737bb26e20 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67ba038740 10 client.31744 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67ba038740 10 client.31744 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67ba038740 20 client.31744 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67ba038740 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55737bb37830 0x55737bba0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67ba038740 4 client.31744 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67ba038740 1 -- 172.21.15.105:0/4181793581 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55737ba85200 con 0x55737bb37830 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.255+0000 7f67b51e7640 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55737bb37830 0x55737bba0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f6798ff9640 10 client.31744 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:56:38.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67b51e7640 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55737bb37830 0x55737bba0350 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7cd32be824adfe1 server_cookie=47a6cb369034f159 in_seq=0 out_seq=0 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f6798ff9640 1 -- 172.21.15.105:0/4181793581 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55737ba85200 con 0x55737bb37830 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f6798ff9640 10 client.31744 handle_command_reply: tid=0 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 client.31744 shutdown 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 -- 172.21.15.105:0/4181793581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb26e20 msgr2=0x55737bb31ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb26e20 0x55737bb31ae0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f67a802c610 tx=0x7f67a8004f40 comp rx=0 tx=0).stop 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 -- 172.21.15.105:0/4181793581 shutdown_connections 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55737bb37830 0x55737bba0350 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55737bb26e20 0x55737bb31ae0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55737bb22220 0x55737bb30ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f6798ff9640 0 client.31744 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 --2- 172.21.15.105:0/4181793581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55737bb21830 0x55737bb30590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 -- 172.21.15.105:0/4181793581 >> 172.21.15.105:0/4181793581 conn(0x55737bb13f20 msgr2=0x55737bb14300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 -- 172.21.15.105:0/4181793581 shutdown_connections 2024-03-26T17:56:38.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.256+0000 7f67ba038740 1 -- 172.21.15.105:0/4181793581 wait complete. 2024-03-26T17:56:38.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67ba038740 20 client.31744 trim_cache size 0 max 16384 2024-03-26T17:56:38.249 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:38.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 -- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f679c0a2fa0 msgr2=0x7f679c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f679c0a2fa0 0x7f679c0a5460 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f67a8032420 tx=0x7f67a8040000 comp rx=0 tx=0).stop 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 -- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b01007c0 msgr2=0x7f67b0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b01007c0 0x7f67b0069590 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9c0 tx=0x7f67ac00ee80 comp rx=0 tx=0).stop 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 -- 172.21.15.105:0/397106283 shutdown_connections 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f679c0a2fa0 0x7f679c0a5460 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67b0102370 0x7f67b019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01019b0 0x7f67b019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 --2- 172.21.15.105:0/397106283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67b01007c0 0x7f67b0069590 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 -- 172.21.15.105:0/397106283 >> 172.21.15.105:0/397106283 conn(0x7f67b00fbd50 msgr2=0x7f67b0104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 -- 172.21.15.105:0/397106283 shutdown_connections 2024-03-26T17:56:38.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.257+0000 7f67b744b640 1 -- 172.21.15.105:0/397106283 wait complete. 2024-03-26T17:56:38.256 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:37 smithi105 ceph-mon[32280]: pgmap v897: 97 pgs: 97 active+clean; 304 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 3.3 MiB/s wr, 204 op/s 2024-03-26T17:56:38.257 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:56:38.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:37 smithi144 ceph-mon[36095]: pgmap v897: 97 pgs: 97 active+clean; 304 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 3.3 MiB/s wr, 204 op/s 2024-03-26T17:56:38.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.410+0000 7f6461459640 1 -- 172.21.15.105:0/368742060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 msgr2=0x7f645c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.410+0000 7f6461459640 1 --2- 172.21.15.105:0/368742060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 0x7f645c100be0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f644c009900 tx=0x7f644c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:38.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.410+0000 7f6461459640 1 -- 172.21.15.105:0/368742060 shutdown_connections 2024-03-26T17:56:38.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.410+0000 7f6461459640 1 --2- 172.21.15.105:0/368742060 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f645c102370 0x7f645c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.410+0000 7f6461459640 1 --2- 172.21.15.105:0/368742060 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645c1019b0 0x7f645c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.410+0000 7f6461459640 1 --2- 172.21.15.105:0/368742060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 0x7f645c100be0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.410+0000 7f6461459640 1 -- 172.21.15.105:0/368742060 >> 172.21.15.105:0/368742060 conn(0x7f645c0fbd50 msgr2=0x7f645c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.410+0000 7f6461459640 1 -- 172.21.15.105:0/368742060 shutdown_connections 2024-03-26T17:56:38.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.411+0000 7f6461459640 1 -- 172.21.15.105:0/368742060 wait complete. 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.411+0000 7f6461459640 1 Processor -- start 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f6461459640 1 -- start start 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f6461459640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 0x7f645c0695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f6461459640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645c1019b0 0x7f645c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f6461459640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f645c102370 0x7f645c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f6461459640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f645c10c340 con 0x7f645c1007c0 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f6461459640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f645c10c1c0 con 0x7f645c1019b0 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f6461459640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f645c10c4c0 con 0x7f645c102370 2024-03-26T17:56:38.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 0x7f645c0695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 0x7f645c0695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48318/0 (socket says 172.21.15.105:48318) 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645affd640 1 -- 172.21.15.105:0/2783568248 learned_addr learned my addr 172.21.15.105:0/2783568248 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645a7fc640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645c1019b0 0x7f645c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645b7fe640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f645c102370 0x7f645c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645affd640 1 -- 172.21.15.105:0/2783568248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f645c102370 msgr2=0x7f645c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645affd640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f645c102370 0x7f645c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645affd640 1 -- 172.21.15.105:0/2783568248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645c1019b0 msgr2=0x7f645c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645affd640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645c1019b0 0x7f645c19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645affd640 1 -- 172.21.15.105:0/2783568248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c19f770 con 0x7f645c1007c0 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.412+0000 7f645b7fe640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f645c102370 0x7f645c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.413+0000 7f645a7fc640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645c1019b0 0x7f645c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.413+0000 7f645affd640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 0x7f645c0695b0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f644c000c00 tx=0x7f644c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.413+0000 7f6443fff640 1 -- 172.21.15.105:0/2783568248 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c043020 con 0x7f645c1007c0 2024-03-26T17:56:38.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.413+0000 7f6443fff640 1 -- 172.21.15.105:0/2783568248 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f644c0044c0 con 0x7f645c1007c0 2024-03-26T17:56:38.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.413+0000 7f6443fff640 1 -- 172.21.15.105:0/2783568248 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c003c60 con 0x7f645c1007c0 2024-03-26T17:56:38.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.413+0000 7f6461459640 1 -- 172.21.15.105:0/2783568248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c19fa00 con 0x7f645c1007c0 2024-03-26T17:56:38.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.413+0000 7f6461459640 1 -- 172.21.15.105:0/2783568248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c19ff50 con 0x7f645c1007c0 2024-03-26T17:56:38.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.416+0000 7f6443fff640 1 -- 172.21.15.105:0/2783568248 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f644c046020 con 0x7f645c1007c0 2024-03-26T17:56:38.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6464040740 1 Processor -- start 2024-03-26T17:56:38.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6464040740 1 -- start start 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6464040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc32e40 0x555e7fc33260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6464040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc33830 0x555e7fc38330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6464040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 0x555e7fc3ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6464040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555e7fb93970 con 0x555e7fc32e40 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6464040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555e7fb3bd70 con 0x555e7fc389a0 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6464040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e7fb8f920 con 0x555e7fc33830 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f645affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc33830 0x555e7fc38330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f6443fff640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64440a3070 0x7f64440a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f645a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 0x555e7fc3ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f645affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc33830 0x555e7fc38330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39842/0 (socket says 172.21.15.105:39842) 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f645affd640 1 -- 172.21.15.105:0/3436530048 learned_addr learned my addr 172.21.15.105:0/3436530048 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.417+0000 7f645b7fe640 1 --2- 172.21.15.105:0/3436530048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc32e40 0x555e7fc33260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f6443fff640 1 -- 172.21.15.105:0/2783568248 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f644c0f1620 con 0x7f645c1007c0 2024-03-26T17:56:38.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f645affd640 1 -- 172.21.15.105:0/3436530048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 msgr2=0x555e7fc3ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f645affd640 1 --2- 172.21.15.105:0/3436530048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 0x555e7fc3ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f645affd640 1 -- 172.21.15.105:0/3436530048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc32e40 msgr2=0x555e7fc33260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f645affd640 1 --2- 172.21.15.105:0/3436530048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc32e40 0x555e7fc33260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f645affd640 1 -- 172.21.15.105:0/3436530048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e7f8a73f0 con 0x555e7fc33830 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f645affd640 1 --2- 172.21.15.105:0/3436530048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc33830 0x555e7fc38330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f644c0ef080 tx=0x7f644c02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=934dfe7b7dc724d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f645b7fe640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64440a3070 0x7f64440a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f64417fa640 1 -- 172.21.15.105:0/3436530048 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c005600 con 0x555e7fc33830 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f64417fa640 1 -- 172.21.15.105:0/3436530048 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f644c0c1e00 con 0x555e7fc33830 2024-03-26T17:56:38.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.418+0000 7f64417fa640 1 -- 172.21.15.105:0/3436530048 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c0ba320 con 0x555e7fc33830 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 -- 172.21.15.105:0/3436530048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc33830 msgr2=0x555e7fc38330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f645b7fe640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64440a3070 0x7f64440a5530 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x555e7fc34090 tx=0x7f6454004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 --2- 172.21.15.105:0/3436530048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc33830 0x555e7fc38330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f644c0ef080 tx=0x7f644c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 -- 172.21.15.105:0/3436530048 shutdown_connections 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 --2- 172.21.15.105:0/3436530048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 0x555e7fc3ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 --2- 172.21.15.105:0/3436530048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc33830 0x555e7fc38330 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 --2- 172.21.15.105:0/3436530048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc32e40 0x555e7fc33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 -- 172.21.15.105:0/3436530048 >> 172.21.15.105:0/3436530048 conn(0x555e7fc29fa0 msgr2=0x555e7fc2a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 -- 172.21.15.105:0/3436530048 shutdown_connections 2024-03-26T17:56:38.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.419+0000 7f6464040740 1 -- 172.21.15.105:0/3436530048 wait complete. 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 1 -- start start 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 1 Processor -- start 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc3cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc3d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 0x555e7fc44560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555e7fc3b710 con 0x555e7fc33830 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555e7fb4f310 con 0x555e7fc389a0 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e7fc3b890 con 0x555e7fc32e40 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f645affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc3cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f645affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc3cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39854/0 (socket says 172.21.15.105:39854) 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f645affd640 1 -- 172.21.15.105:0/3903852906 learned_addr learned my addr 172.21.15.105:0/3903852906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f645a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc3d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f6464040740 10 client.0 _ll_register_callbacks cb 0x555e7fb8ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.420+0000 7f645b7fe640 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 0x555e7fc44560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f645a7fc640 1 -- 172.21.15.105:0/3903852906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 msgr2=0x555e7fc3cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f645a7fc640 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc3cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f645a7fc640 1 -- 172.21.15.105:0/3903852906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 msgr2=0x555e7fc44560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f645a7fc640 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 0x555e7fc44560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f645a7fc640 1 -- 172.21.15.105:0/3903852906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e7f9717d0 con 0x555e7fc33830 2024-03-26T17:56:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f645a7fc640 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc3d520 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f645c069320 tx=0x7f645000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6099c0e09a50df6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f643eff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f643eff5640 1 -- 172.21.15.105:0/3903852906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645001b640 con 0x555e7fc33830 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f643eff5640 1 -- 172.21.15.105:0/3903852906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6450011440 con 0x555e7fc33830 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f6464040740 1 -- 172.21.15.105:0/3903852906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e7fba0eb0 con 0x555e7fc33830 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f643eff5640 1 -- 172.21.15.105:0/3903852906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f645001b7e0 con 0x555e7fc33830 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f6464040740 10 client.22047 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.421+0000 7f6464040740 1 -- 172.21.15.105:0/3903852906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555e7fc479a0 con 0x555e7fc33830 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f643eff5640 1 -- 172.21.15.105:0/3903852906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f645001be30 con 0x555e7fc33830 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f643eff5640 1 -- 172.21.15.105:0/3903852906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6450019020 con 0x555e7fc33830 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f6464040740 10 client.22047 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f6464040740 10 client.22047 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:56:38.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f6464040740 20 client.22047 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:38.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f6464040740 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555e7fc49070 0x555e7fc4b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f6464040740 4 client.22047 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:38.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f6464040740 1 -- 172.21.15.105:0/3903852906 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e7fba0eb0 con 0x555e7fc49070 2024-03-26T17:56:38.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.422+0000 7f645affd640 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555e7fc49070 0x555e7fc4b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.423+0000 7f645affd640 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555e7fc49070 0x555e7fc4b4f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ae1ec4f213cc3c6 server_cookie=692569dc89fe2356 in_seq=0 out_seq=0 2024-03-26T17:56:38.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.423+0000 7f643eff5640 10 client.22047 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:56:38.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.423+0000 7f643eff5640 1 -- 172.21.15.105:0/3903852906 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555e7fba0eb0 con 0x555e7fc49070 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.423+0000 7f643eff5640 10 client.22047 handle_command_reply: tid=0 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.423+0000 7f6464040740 1 client.22047 shutdown 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 -- 172.21.15.105:0/3903852906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 msgr2=0x555e7fc3d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc3d520 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f645c069320 tx=0x7f645000bf50 comp rx=0 tx=0).stop 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 -- 172.21.15.105:0/3903852906 shutdown_connections 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555e7fc49070 0x555e7fc4b4f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc389a0 0x555e7fc44560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc3d520 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 --2- 172.21.15.105:0/3903852906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc3cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 -- 172.21.15.105:0/3903852906 >> 172.21.15.105:0/3903852906 conn(0x555e7fc29fa0 msgr2=0x555e7fc291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f643eff5640 0 client.22047 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:56:38.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 -- 172.21.15.105:0/3903852906 shutdown_connections 2024-03-26T17:56:38.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 1 -- 172.21.15.105:0/3903852906 wait complete. 2024-03-26T17:56:38.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.424+0000 7f6464040740 20 client.22047 trim_cache size 0 max 16384 2024-03-26T17:56:38.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.444+0000 7f6464040740 1 Processor -- start 2024-03-26T17:56:38.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.444+0000 7f6464040740 1 -- start start 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f6464040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc32e40 0x555e7fc28930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f6464040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc44520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f6464040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc389a0 0x555e7fc28e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f6464040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1a6be0 con 0x555e7fc33830 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f6464040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1a8720 con 0x555e7fc32e40 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f6464040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1a72b0 con 0x555e7fc389a0 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc44520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc44520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48340/0 (socket says 172.21.15.105:48340) 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 -- 172.21.15.105:0/3903535738 learned_addr learned my addr 172.21.15.105:0/3903535738 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645b7fe640 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc389a0 0x555e7fc28e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645affd640 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc32e40 0x555e7fc28930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 -- 172.21.15.105:0/3903535738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc389a0 msgr2=0x555e7fc28e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc389a0 0x555e7fc28e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 -- 172.21.15.105:0/3903535738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc32e40 msgr2=0x555e7fc28930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc32e40 0x555e7fc28930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 -- 172.21.15.105:0/3903535738 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e7fc3de80 con 0x555e7fc33830 2024-03-26T17:56:38.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f645a7fc640 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc44520 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f6450011ee0 tx=0x7f64500039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b3ac194b04399e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.445+0000 7f6440ff9640 1 -- 172.21.15.105:0/3903535738 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645001b640 con 0x555e7fc33830 2024-03-26T17:56:38.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.446+0000 7f6440ff9640 1 -- 172.21.15.105:0/3903535738 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f645001be30 con 0x555e7fc33830 2024-03-26T17:56:38.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.446+0000 7f6440ff9640 1 -- 172.21.15.105:0/3903535738 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64500048e0 con 0x555e7fc33830 2024-03-26T17:56:38.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.446+0000 7f6464040740 1 -- 172.21.15.105:0/3903535738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 msgr2=0x555e7fc44520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.446+0000 7f6464040740 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc44520 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f6450011ee0 tx=0x7f64500039c0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 -- 172.21.15.105:0/3903535738 shutdown_connections 2024-03-26T17:56:38.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc389a0 0x555e7fc28e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc33830 0x555e7fc44520 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 --2- 172.21.15.105:0/3903535738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc32e40 0x555e7fc28930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 -- 172.21.15.105:0/3903535738 >> 172.21.15.105:0/3903535738 conn(0x555e7fc29fa0 msgr2=0x555e7fc3cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 -- 172.21.15.105:0/3903535738 shutdown_connections 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 -- 172.21.15.105:0/3903535738 wait complete. 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 -- start start 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.447+0000 7f6464040740 1 Processor -- start 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f6464040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc41b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f6464040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc33830 0x555e7fc42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f6464040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc389a0 0x555e7fc430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f645b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc389a0 0x555e7fc430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f645b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc389a0 0x555e7fc430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48356/0 (socket says 172.21.15.105:48356) 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f645b7fe640 1 -- 172.21.15.105:0/4222971265 learned_addr learned my addr 172.21.15.105:0/4222971265 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f6464040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1a8050 con 0x555e7fc389a0 2024-03-26T17:56:38.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1a7980 con 0x555e7fc33830 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1a4bc0 con 0x555e7fc32e40 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.448+0000 7f645affd640 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc41b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f6464040740 10 client.0 _ll_register_callbacks cb 0x7f6450004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f645a7fc640 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc33830 0x555e7fc42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f645b7fe640 1 -- 172.21.15.105:0/4222971265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 msgr2=0x555e7fc41b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f645b7fe640 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc41b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f645b7fe640 1 -- 172.21.15.105:0/4222971265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc33830 msgr2=0x555e7fc42090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f645b7fe640 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc33830 0x555e7fc42090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f645b7fe640 1 -- 172.21.15.105:0/4222971265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e7fc43690 con 0x555e7fc389a0 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f645b7fe640 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc389a0 0x555e7fc430a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6454012c80 tx=0x7f6454004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5453575385c855bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f643eff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f643eff5640 1 -- 172.21.15.105:0/4222971265 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454029420 con 0x555e7fc389a0 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f643eff5640 1 -- 172.21.15.105:0/4222971265 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6454010450 con 0x555e7fc389a0 2024-03-26T17:56:38.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f645001be30 con 0x555e7fc389a0 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f643eff5640 1 -- 172.21.15.105:0/4222971265 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6454010650 con 0x555e7fc389a0 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f6464040740 10 client.22053 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.449+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555e7fcaf3f0 con 0x555e7fc389a0 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f643eff5640 1 -- 172.21.15.105:0/4222971265 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f645001be30 con 0x555e7fc389a0 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f643eff5640 1 -- 172.21.15.105:0/4222971265 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6454029b90 con 0x555e7fc389a0 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f6464040740 10 client.22053 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f6464040740 10 client.22053 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f6464040740 20 client.22053 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f6464040740 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555e7fcafde0 0x555e7fcb2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f6464040740 4 client.22053 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e7fb9b200 con 0x555e7fcafde0 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.450+0000 7f645affd640 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555e7fcafde0 0x555e7fcb2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f645affd640 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555e7fcafde0 0x555e7fcb2200 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c092745b9be3caf8 server_cookie=5b61e38b01bd4b83 in_seq=0 out_seq=0 2024-03-26T17:56:38.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f643eff5640 10 client.22053 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f643eff5640 1 -- 172.21.15.105:0/4222971265 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e7fb9b200 con 0x555e7fcafde0 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f643eff5640 10 client.22053 handle_command_reply: tid=0 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 client.22053 shutdown 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc389a0 msgr2=0x555e7fc430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc389a0 0x555e7fc430a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6454012c80 tx=0x7f6454004ab0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 shutdown_connections 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x555e7fcafde0 0x555e7fcb2200 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e7fc389a0 0x555e7fc430a0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f643eff5640 0 client.22053 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e7fc33830 0x555e7fc42090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 --2- 172.21.15.105:0/4222971265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e7fc32e40 0x555e7fc41b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 >> 172.21.15.105:0/4222971265 conn(0x555e7fc29fa0 msgr2=0x555e7fc2d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 shutdown_connections 2024-03-26T17:56:38.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.451+0000 7f6464040740 1 -- 172.21.15.105:0/4222971265 wait complete. 2024-03-26T17:56:38.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6464040740 20 client.22053 trim_cache size 0 max 16384 2024-03-26T17:56:38.444 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 -- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64440a3070 msgr2=0x7f64440a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64440a3070 0x7f64440a5530 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x555e7fc34090 tx=0x7f6454004a40 comp rx=0 tx=0).stop 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 -- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 msgr2=0x7f645c0695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 0x7f645c0695b0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f644c000c00 tx=0x7f644c0040c0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 -- 172.21.15.105:0/2783568248 shutdown_connections 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64440a3070 0x7f64440a5530 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f645c102370 0x7f645c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f645c1019b0 0x7f645c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 --2- 172.21.15.105:0/2783568248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f645c1007c0 0x7f645c0695b0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 -- 172.21.15.105:0/2783568248 >> 172.21.15.105:0/2783568248 conn(0x7f645c0fbd50 msgr2=0x7f645c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 -- 172.21.15.105:0/2783568248 shutdown_connections 2024-03-26T17:56:38.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.452+0000 7f6461459640 1 -- 172.21.15.105:0/2783568248 wait complete. 2024-03-26T17:56:38.453 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:56:38.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 42197 7.06 MB/sec execute 201 sec latency 123.806 ms 2024-03-26T17:56:38.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 -- 172.21.15.105:0/3244306617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 msgr2=0x7f6e3c100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3244306617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 0x7f6e3c100c20 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009900 tx=0x7f6e2c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 -- 172.21.15.105:0/3244306617 shutdown_connections 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3244306617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c1023b0 0x7f6e3c108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3244306617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c1019f0 0x7f6e3c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3244306617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 0x7f6e3c100c20 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 -- 172.21.15.105:0/3244306617 >> 172.21.15.105:0/3244306617 conn(0x7f6e3c0fbd70 msgr2=0x7f6e3c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 -- 172.21.15.105:0/3244306617 shutdown_connections 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.611+0000 7f6e4146c640 1 -- 172.21.15.105:0/3244306617 wait complete. 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.612+0000 7f6e4146c640 1 Processor -- start 2024-03-26T17:56:38.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.612+0000 7f6e4146c640 1 -- start start 2024-03-26T17:56:38.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.612+0000 7f6e4146c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 0x7f6e3c197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.612+0000 7f6e4146c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c1019f0 0x7f6e3c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.612+0000 7f6e4146c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c1023b0 0x7f6e3c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.612+0000 7f6e4146c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10c300 con 0x7f6e3c100800 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.612+0000 7f6e4146c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10c180 con 0x7f6e3c1023b0 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.612+0000 7f6e4146c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c10c480 con 0x7f6e3c1019f0 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 0x7f6e3c197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 0x7f6e3c197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48374/0 (socket says 172.21.15.105:48374) 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 -- 172.21.15.105:0/3468920478 learned_addr learned my addr 172.21.15.105:0/3468920478 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c1019f0 0x7f6e3c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3b7fe640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c1023b0 0x7f6e3c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 -- 172.21.15.105:0/3468920478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c1019f0 msgr2=0x7f6e3c198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c1019f0 0x7f6e3c198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 -- 172.21.15.105:0/3468920478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c1023b0 msgr2=0x7f6e3c19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c1023b0 0x7f6e3c19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 -- 172.21.15.105:0/3468920478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c19f7e0 con 0x7f6e3c100800 2024-03-26T17:56:38.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e3affd640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 0x7f6e3c197c10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009850 tx=0x7f6e2c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.613+0000 7f6e38ff9640 1 -- 172.21.15.105:0/3468920478 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c043020 con 0x7f6e3c100800 2024-03-26T17:56:38.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.614+0000 7f6e38ff9640 1 -- 172.21.15.105:0/3468920478 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e2c0042d0 con 0x7f6e3c100800 2024-03-26T17:56:38.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.614+0000 7f6e4146c640 1 -- 172.21.15.105:0/3468920478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c102ec0 con 0x7f6e3c100800 2024-03-26T17:56:38.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.614+0000 7f6e38ff9640 1 -- 172.21.15.105:0/3468920478 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c003b10 con 0x7f6e3c100800 2024-03-26T17:56:38.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.614+0000 7f6e4146c640 1 -- 172.21.15.105:0/3468920478 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c103350 con 0x7f6e3c100800 2024-03-26T17:56:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.617+0000 7f6e4404b740 1 Processor -- start 2024-03-26T17:56:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.617+0000 7f6e4404b740 1 -- start start 2024-03-26T17:56:38.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 0x5651cf2d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d5430 0x5651cf2d9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e4404b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5651cf228970 con 0x5651cf2da5a0 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e4404b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5651cf1d0d70 con 0x5651cf2d5430 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e4404b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651cf224920 con 0x5651cf2d4a40 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e38ff9640 1 -- 172.21.15.105:0/3468920478 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6e2c046020 con 0x7f6e3c100800 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2dca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48390/0 (socket says 172.21.15.105:48390) 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e327fc640 1 -- 172.21.15.105:0/3491753733 learned_addr learned my addr 172.21.15.105:0/3491753733 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e327fc640 1 -- 172.21.15.105:0/3491753733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 msgr2=0x5651cf2d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3491753733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 0x5651cf2d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e327fc640 1 -- 172.21.15.105:0/3491753733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d5430 msgr2=0x5651cf2d9f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:56:38.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.618+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3491753733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d5430 0x5651cf2d9f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.619+0000 7f6e327fc640 1 -- 172.21.15.105:0/3491753733 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651cef3c3f0 con 0x5651cf2da5a0 2024-03-26T17:56:38.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.619+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3491753733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2dca30 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6e24010dc0 tx=0x7f6e24005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f397907cf96e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.619+0000 7f6e317fa640 1 -- 172.21.15.105:0/3491753733 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401a6c0 con 0x5651cf2da5a0 2024-03-26T17:56:38.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.619+0000 7f6e38ff9640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e280a3070 0x7f6e280a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.619+0000 7f6e317fa640 1 -- 172.21.15.105:0/3491753733 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e2401ad10 con 0x5651cf2da5a0 2024-03-26T17:56:38.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.619+0000 7f6e317fa640 1 -- 172.21.15.105:0/3491753733 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24019d80 con 0x5651cf2da5a0 2024-03-26T17:56:38.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.619+0000 7f6e38ff9640 1 -- 172.21.15.105:0/3468920478 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6e2c0f14d0 con 0x7f6e3c100800 2024-03-26T17:56:38.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.619+0000 7f6e3b7fe640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e280a3070 0x7f6e280a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.620+0000 7f6e3b7fe640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e280a3070 0x7f6e280a5530 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x5651cf2d5c90 tx=0x7f6e34009b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.620+0000 7f6e4404b740 1 -- 172.21.15.105:0/3491753733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 msgr2=0x5651cf2dca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.620+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3491753733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2dca30 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6e24010dc0 tx=0x7f6e24005f90 comp rx=0 tx=0).stop 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 -- 172.21.15.105:0/3491753733 shutdown_connections 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3491753733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2dca30 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3491753733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d5430 0x5651cf2d9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3491753733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 0x5651cf2d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 -- 172.21.15.105:0/3491753733 >> 172.21.15.105:0/3491753733 conn(0x5651cf2befa0 msgr2=0x5651cf2bf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 -- 172.21.15.105:0/3491753733 shutdown_connections 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 -- 172.21.15.105:0/3491753733 wait complete. 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 -- start start 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.621+0000 7f6e4404b740 1 Processor -- start 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d4a40 0x5651cf2dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2e6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e4404b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5651cf2dd550 con 0x5651cf2da5a0 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e4404b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5651cf1e4310 con 0x5651cf2d4a40 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e4404b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651cf2dd6d0 con 0x5651cf2d5430 2024-03-26T17:56:38.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2df340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39932/0 (socket says 172.21.15.105:39932) 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e327fc640 1 -- 172.21.15.105:0/307842672 learned_addr learned my addr 172.21.15.105:0/307842672 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e4404b740 10 client.0 _ll_register_callbacks cb 0x5651cf21fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e3affd640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d4a40 0x5651cf2dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e3b7fe640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.622+0000 7f6e327fc640 1 -- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d4a40 msgr2=0x5651cf2dee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e327fc640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d4a40 0x5651cf2dee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e327fc640 1 -- 172.21.15.105:0/307842672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 msgr2=0x5651cf2e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e327fc640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2e6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e327fc640 1 -- 172.21.15.105:0/307842672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651cf0067d0 con 0x5651cf2d5430 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e3b7fe640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:38.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e3affd640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d4a40 0x5651cf2dee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e327fc640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2df340 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6e2401a040 tx=0x7f6e240196f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edc0cb3661bb6744 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e227fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e227fc640 1 -- 172.21.15.105:0/307842672 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24005590 con 0x5651cf2d5430 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e227fc640 1 -- 172.21.15.105:0/307842672 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6e2401a4c0 con 0x5651cf2d5430 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e4404b740 1 -- 172.21.15.105:0/307842672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2401ad10 con 0x5651cf2d5430 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e227fc640 1 -- 172.21.15.105:0/307842672 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e2401a6c0 con 0x5651cf2d5430 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e4404b740 10 client.31768 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.623+0000 7f6e4404b740 1 -- 172.21.15.105:0/307842672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5651cf2e96d0 con 0x5651cf2d5430 2024-03-26T17:56:38.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e227fc640 1 -- 172.21.15.105:0/307842672 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e2401ad10 con 0x5651cf2d5430 2024-03-26T17:56:38.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e227fc640 1 -- 172.21.15.105:0/307842672 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6e2402e470 con 0x5651cf2d5430 2024-03-26T17:56:38.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e4404b740 10 client.31768 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:38.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e4404b740 10 client.31768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:56:38.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e4404b740 20 client.31768 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:38.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e4404b740 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5651cf2eaa20 0x5651cf2ece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e4404b740 4 client.31768 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:38.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e4404b740 1 -- 172.21.15.105:0/307842672 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651cf235eb0 con 0x5651cf2eaa20 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.624+0000 7f6e3affd640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5651cf2eaa20 0x5651cf2ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.625+0000 7f6e227fc640 10 client.31768 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.625+0000 7f6e3affd640 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5651cf2eaa20 0x5651cf2ece10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d04321810d901867 server_cookie=c84e63b203b0c8c in_seq=0 out_seq=0 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e227fc640 1 -- 172.21.15.105:0/307842672 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5651cf235eb0 con 0x5651cf2eaa20 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e227fc640 10 client.31768 handle_command_reply: tid=0 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 client.31768 shutdown 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 -- 172.21.15.105:0/307842672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 msgr2=0x5651cf2df340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2df340 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6e2401a040 tx=0x7f6e240196f0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 -- 172.21.15.105:0/307842672 shutdown_connections 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2da5a0 0x5651cf2e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2df340 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5651cf2eaa20 0x5651cf2ece10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 --2- 172.21.15.105:0/307842672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2d4a40 0x5651cf2dee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 -- 172.21.15.105:0/307842672 >> 172.21.15.105:0/307842672 conn(0x5651cf2befa0 msgr2=0x5651cf2be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e227fc640 0 client.31768 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:56:38.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 -- 172.21.15.105:0/307842672 shutdown_connections 2024-03-26T17:56:38.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 1 -- 172.21.15.105:0/307842672 wait complete. 2024-03-26T17:56:38.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.626+0000 7f6e4404b740 20 client.31768 trim_cache size 0 max 16384 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e4404b740 1 Processor -- start 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e4404b740 1 -- start start 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 0x5651cf2c2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d5430 0x5651cf2e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 0x5651cf2c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e4404b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c1a05d0 con 0x5651cf2d5430 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e4404b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0724f0 con 0x5651cf2da5a0 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e4404b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c19ff00 con 0x5651cf2d4a40 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d5430 0x5651cf2e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d5430 0x5651cf2e62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48418/0 (socket says 172.21.15.105:48418) 2024-03-26T17:56:38.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e327fc640 1 -- 172.21.15.105:0/3878112189 learned_addr learned my addr 172.21.15.105:0/3878112189 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.647+0000 7f6e3affd640 1 --2- 172.21.15.105:0/3878112189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 0x5651cf2c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.648+0000 7f6e327fc640 1 -- 172.21.15.105:0/3878112189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 msgr2=0x5651cf2c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.648+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3878112189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 0x5651cf2c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.648+0000 7f6e327fc640 1 -- 172.21.15.105:0/3878112189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 msgr2=0x5651cf2c2740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:56:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.648+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3878112189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 0x5651cf2c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.648+0000 7f6e327fc640 1 -- 172.21.15.105:0/3878112189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651cf2bfd10 con 0x5651cf2d5430 2024-03-26T17:56:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.648+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3878112189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d5430 0x5651cf2e62d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005db0 tx=0x7f6e24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b326c6b9a0e92313 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.648+0000 7f6e30ff9640 1 -- 172.21.15.105:0/3878112189 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24019760 con 0x5651cf2d5430 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.649+0000 7f6e30ff9640 1 -- 172.21.15.105:0/3878112189 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e24034030 con 0x5651cf2d5430 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.649+0000 7f6e30ff9640 1 -- 172.21.15.105:0/3878112189 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24005390 con 0x5651cf2d5430 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 -- 172.21.15.105:0/3878112189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d5430 msgr2=0x5651cf2e62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3878112189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d5430 0x5651cf2e62d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005db0 tx=0x7f6e24000bc0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 -- 172.21.15.105:0/3878112189 shutdown_connections 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3878112189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 0x5651cf2c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3878112189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d5430 0x5651cf2e62d0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3878112189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d4a40 0x5651cf2c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 -- 172.21.15.105:0/3878112189 >> 172.21.15.105:0/3878112189 conn(0x5651cf2befa0 msgr2=0x5651cf2d6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 -- 172.21.15.105:0/3878112189 shutdown_connections 2024-03-26T17:56:38.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 -- 172.21.15.105:0/3878112189 wait complete. 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 -- start start 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.650+0000 7f6e4404b740 1 Processor -- start 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d4a40 0x5651cf2e3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e4404b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 0x5651cf2e4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e4404b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c073290 con 0x5651cf2d4a40 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e4404b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c072bc0 con 0x5651cf2da5a0 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e4404b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0704d0 con 0x5651cf2d5430 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d4a40 0x5651cf2e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e4404b740 10 client.0 _ll_register_callbacks cb 0x7f6e24010cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d4a40 0x5651cf2e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48430/0 (socket says 172.21.15.105:48430) 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3affd640 1 -- 172.21.15.105:0/3390059811 learned_addr learned my addr 172.21.15.105:0/3390059811 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3b7fe640 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 0x5651cf2e4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3affd640 1 -- 172.21.15.105:0/3390059811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 msgr2=0x5651cf2e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3affd640 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3affd640 1 -- 172.21.15.105:0/3390059811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 msgr2=0x5651cf2e4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3affd640 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 0x5651cf2e4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.651+0000 7f6e3affd640 1 -- 172.21.15.105:0/3390059811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651cf2e51b0 con 0x5651cf2d4a40 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e3affd640 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d4a40 0x5651cf2e3670 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0bdaf0 tx=0x7f6e2c0f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786426ed0ce04b3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e227fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e227fc640 1 -- 172.21.15.105:0/3390059811 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0db070 con 0x5651cf2d4a40 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e227fc640 1 -- 172.21.15.105:0/3390059811 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f6e2c0ef040 con 0x5651cf2d4a40 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e4404b740 1 -- 172.21.15.105:0/3390059811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e24034030 con 0x5651cf2d4a40 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e227fc640 1 -- 172.21.15.105:0/3390059811 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e2c0c1cb0 con 0x5651cf2d4a40 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e4404b740 10 client.22071 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e4404b740 1 -- 172.21.15.105:0/3390059811 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5651cf2e6580 con 0x5651cf2d4a40 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.652+0000 7f6e227fc640 1 -- 172.21.15.105:0/3390059811 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e2c0c6570 con 0x5651cf2d4a40 2024-03-26T17:56:38.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e227fc640 1 -- 172.21.15.105:0/3390059811 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6e2c0bddc0 con 0x5651cf2d4a40 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e4404b740 10 client.22071 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e4404b740 10 client.22071 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e4404b740 20 client.22071 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5651cf3513d0 0x5651cf3537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e4404b740 4 client.22071 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e4404b740 1 -- 172.21.15.105:0/3390059811 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6e24034030 con 0x5651cf3513d0 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5651cf3513d0 0x5651cf3537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e227fc640 10 client.22071 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.653+0000 7f6e327fc640 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5651cf3513d0 0x5651cf3537f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c533b686775cf126 server_cookie=555546024d7ae25a in_seq=0 out_seq=0 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e227fc640 1 -- 172.21.15.105:0/3390059811 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6e24034030 con 0x5651cf3513d0 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e227fc640 10 client.22071 handle_command_reply: tid=0 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 client.22071 shutdown 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 -- 172.21.15.105:0/3390059811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d4a40 msgr2=0x5651cf2e3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d4a40 0x5651cf2e3670 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0bdaf0 tx=0x7f6e2c0f1040 comp rx=0 tx=0).stop 2024-03-26T17:56:38.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 -- 172.21.15.105:0/3390059811 shutdown_connections 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5651cf3513d0 0x5651cf3537f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e227fc640 0 client.22071 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5651cf2da5a0 0x5651cf2e4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651cf2d5430 0x5651cf2e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 --2- 172.21.15.105:0/3390059811 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5651cf2d4a40 0x5651cf2e3670 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 -- 172.21.15.105:0/3390059811 >> 172.21.15.105:0/3390059811 conn(0x5651cf2befa0 msgr2=0x5651cf2c2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 -- 172.21.15.105:0/3390059811 shutdown_connections 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 1 -- 172.21.15.105:0/3390059811 wait complete. 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.654+0000 7f6e4404b740 20 client.22071 trim_cache size 0 max 16384 2024-03-26T17:56:38.645 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 -- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e280a3070 msgr2=0x7f6e280a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e280a3070 0x7f6e280a5530 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x5651cf2d5c90 tx=0x7f6e34009b10 comp rx=0 tx=0).stop 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 -- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 msgr2=0x7f6e3c197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 0x7f6e3c197c10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009850 tx=0x7f6e2c005d50 comp rx=0 tx=0).stop 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 -- 172.21.15.105:0/3468920478 shutdown_connections 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e280a3070 0x7f6e280a5530 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e3c1023b0 0x7f6e3c19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e3c1019f0 0x7f6e3c198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 --2- 172.21.15.105:0/3468920478 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e3c100800 0x7f6e3c197c10 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 -- 172.21.15.105:0/3468920478 >> 172.21.15.105:0/3468920478 conn(0x7f6e3c0fbd70 msgr2=0x7f6e3c069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 -- 172.21.15.105:0/3468920478 shutdown_connections 2024-03-26T17:56:38.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.655+0000 7f6e4146c640 1 -- 172.21.15.105:0/3468920478 wait complete. 2024-03-26T17:56:38.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:56:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 -- 172.21.15.105:0/4276733631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41019b0 msgr2=0x7f55d4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 --2- 172.21.15.105:0/4276733631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41019b0 0x7f55d4101e30 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009860 tx=0x7f55c402bb70 comp rx=0 tx=0).stop 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 -- 172.21.15.105:0/4276733631 shutdown_connections 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 --2- 172.21.15.105:0/4276733631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55d4102370 0x7f55d4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 --2- 172.21.15.105:0/4276733631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41019b0 0x7f55d4101e30 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 --2- 172.21.15.105:0/4276733631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55d41007c0 0x7f55d4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 -- 172.21.15.105:0/4276733631 >> 172.21.15.105:0/4276733631 conn(0x7f55d40fbd50 msgr2=0x7f55d40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 -- 172.21.15.105:0/4276733631 shutdown_connections 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.813+0000 7f55db668640 1 -- 172.21.15.105:0/4276733631 wait complete. 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55db668640 1 Processor -- start 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55db668640 1 -- start start 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55db668640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41007c0 0x7f55d4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55db668640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55d41019b0 0x7f55d4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55db668640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55d4102370 0x7f55d419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55db668640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f55d410c360 con 0x7f55d41007c0 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55db668640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55d410c1e0 con 0x7f55d4102370 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55db668640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55d410c4e0 con 0x7f55d41019b0 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55d9404640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41007c0 0x7f55d4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55d9404640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41007c0 0x7f55d4197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48444/0 (socket says 172.21.15.105:48444) 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55d9404640 1 -- 172.21.15.105:0/2419839292 learned_addr learned my addr 172.21.15.105:0/2419839292 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.814+0000 7f55d9c05640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55d4102370 0x7f55d419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d9404640 1 -- 172.21.15.105:0/2419839292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55d41019b0 msgr2=0x7f55d4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d8c03640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55d41019b0 0x7f55d4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d9404640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55d41019b0 0x7f55d4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d9404640 1 -- 172.21.15.105:0/2419839292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55d4102370 msgr2=0x7f55d419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d9404640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55d4102370 0x7f55d419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d9404640 1 -- 172.21.15.105:0/2419839292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d419f810 con 0x7f55d41007c0 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d8c03640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55d41019b0 0x7f55d4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:56:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d9c05640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55d4102370 0x7f55d419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:56:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55d9404640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41007c0 0x7f55d4197bf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b450 tx=0x7f55d000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55ca7fc640 1 -- 172.21.15.105:0/2419839292 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0015020 con 0x7f55d41007c0 2024-03-26T17:56:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55ca7fc640 1 -- 172.21.15.105:0/2419839292 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55d0011d40 con 0x7f55d41007c0 2024-03-26T17:56:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55db668640 1 -- 172.21.15.105:0/2419839292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d419fb00 con 0x7f55d41007c0 2024-03-26T17:56:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55db668640 1 -- 172.21.15.105:0/2419839292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d41a00a0 con 0x7f55d41007c0 2024-03-26T17:56:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.815+0000 7f55ca7fc640 1 -- 172.21.15.105:0/2419839292 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d000f340 con 0x7f55d41007c0 2024-03-26T17:56:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.818+0000 7f55de24e740 1 Processor -- start 2024-03-26T17:56:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55de24e740 1 -- start start 2024-03-26T17:56:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55de24e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55de24e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f6420 0x5635da5faf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55de24e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5fda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55de24e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5635da55a970 con 0x5635da5fb590 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55de24e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5635da502d70 con 0x5635da5f5a30 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55de24e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5635da556920 con 0x5635da5f6420 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d9c05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d9c05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57864/0 (socket says 172.21.15.105:57864) 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d9c05640 1 -- 172.21.15.105:0/1422516193 learned_addr learned my addr 172.21.15.105:0/1422516193 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d8c03640 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55ca7fc640 1 -- 172.21.15.105:0/2419839292 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f55d0018020 con 0x7f55d41007c0 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d9c05640 1 -- 172.21.15.105:0/1422516193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f6420 msgr2=0x5635da5faf20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d9c05640 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f6420 0x5635da5faf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d9c05640 1 -- 172.21.15.105:0/1422516193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 msgr2=0x5635da5fda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d9c05640 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5fda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d9c05640 1 -- 172.21.15.105:0/1422516193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635da26e3f0 con 0x5635da5f5a30 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.819+0000 7f55d8c03640 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5fda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.820+0000 7f55d9c05640 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5f5e50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009fb0 tx=0x7f55cc00f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27683123b5440a7b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.820+0000 7f55bf7fe640 1 -- 172.21.15.105:0/1422516193 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc003970 con 0x5635da5f5a30 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.820+0000 7f55bf7fe640 1 -- 172.21.15.105:0/1422516193 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55cc004020 con 0x5635da5f5a30 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.820+0000 7f55bf7fe640 1 -- 172.21.15.105:0/1422516193 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00c9a0 con 0x5635da5f5a30 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.820+0000 7f55ca7fc640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f55c00a3020 0x7f55c00a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.820+0000 7f55ca7fc640 1 -- 172.21.15.105:0/2419839292 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f55d00cef40 con 0x7f55d41007c0 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.820+0000 7f55d9404640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f55c00a3020 0x7f55c00a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 -- 172.21.15.105:0/1422516193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 msgr2=0x5635da5f5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5f5e50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009fb0 tx=0x7f55cc00f890 comp rx=0 tx=0).stop 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55d9404640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f55c00a3020 0x7f55c00a54e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f55d009b560 tx=0x7f55d009b3f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 -- 172.21.15.105:0/1422516193 shutdown_connections 2024-03-26T17:56:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5fda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f6420 0x5635da5faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 --2- 172.21.15.105:0/1422516193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5f5e50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 -- 172.21.15.105:0/1422516193 >> 172.21.15.105:0/1422516193 conn(0x5635da5f0fa0 msgr2=0x5635da5f13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 -- 172.21.15.105:0/1422516193 shutdown_connections 2024-03-26T17:56:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 -- 172.21.15.105:0/1422516193 wait complete. 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.821+0000 7f55de24e740 1 -- start start 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.822+0000 7f55de24e740 1 Processor -- start 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.822+0000 7f55de24e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5fff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.822+0000 7f55de24e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5f6420 0x5635da600460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.822+0000 7f55de24e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5fb590 0x5635da6074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.822+0000 7f55d8c03640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5f6420 0x5635da600460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.822+0000 7f55d8c03640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5f6420 0x5635da600460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48466/0 (socket says 172.21.15.105:48466) 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.822+0000 7f55d8c03640 1 -- 172.21.15.105:0/2207094642 learned_addr learned my addr 172.21.15.105:0/2207094642 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.822+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5635da5fe540 con 0x5635da5f6420 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5635da516310 con 0x5635da5f5a30 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5635da5fe6c0 con 0x5635da5fb590 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55de24e740 10 client.0 _ll_register_callbacks cb 0x5635da551c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55d8c03640 1 -- 172.21.15.105:0/2207094642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5fb590 msgr2=0x5635da6074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55d8c03640 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5fb590 0x5635da6074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55d8c03640 1 -- 172.21.15.105:0/2207094642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 msgr2=0x5635da5fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55d8c03640 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55d8c03640 1 -- 172.21.15.105:0/2207094642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635da3387d0 con 0x5635da5f6420 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55d8c03640 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5f6420 0x5635da600460 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009830 tx=0x7f55c4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbd24248e2ef468 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55cc004020 con 0x5635da5f6420 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.823+0000 7f55bcff9640 10 client.22083 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.824+0000 7f55bcff9640 1 -- 172.21.15.105:0/2207094642 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4039040 con 0x5635da5f6420 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.824+0000 7f55bcff9640 1 -- 172.21.15.105:0/2207094642 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f55c4004240 con 0x5635da5f6420 2024-03-26T17:56:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55bcff9640 1 -- 172.21.15.105:0/2207094642 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f55c40044e0 con 0x5635da5f6420 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55de24e740 10 client.22083 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5635da60a7c0 con 0x5635da5f6420 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55bcff9640 1 -- 172.21.15.105:0/2207094642 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f55c4005030 con 0x5635da5f6420 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55de24e740 10 client.22083 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55de24e740 10 client.22083 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55de24e740 20 client.22083 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55de24e740 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5635da60bcf0 0x5635da60e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55de24e740 4 client.22083 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55cc004020 con 0x5635da60bcf0 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.825+0000 7f55d9c05640 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5635da60bcf0 0x5635da60e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.826+0000 7f55d9c05640 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5635da60bcf0 0x5635da60e180 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57345ef681bcf6cd server_cookie=3f5c0cefeb3b51aa in_seq=0 out_seq=0 2024-03-26T17:56:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.826+0000 7f55bcff9640 10 client.22083 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.827+0000 7f55bcff9640 1 -- 172.21.15.105:0/2207094642 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f55cc004020 con 0x5635da60bcf0 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55bcff9640 10 client.22083 handle_command_reply: tid=0 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 client.22083 shutdown 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5f6420 msgr2=0x5635da600460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5f6420 0x5635da600460 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009830 tx=0x7f55c4005db0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 shutdown_connections 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5635da60bcf0 0x5635da60e180 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5fb590 0x5635da6074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5f6420 0x5635da600460 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 --2- 172.21.15.105:0/2207094642 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5f5a30 0x5635da5fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 >> 172.21.15.105:0/2207094642 conn(0x5635da5f0fa0 msgr2=0x5635da5f01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.828+0000 7f55bcff9640 0 client.22083 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:56:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.829+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 shutdown_connections 2024-03-26T17:56:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.829+0000 7f55de24e740 1 -- 172.21.15.105:0/2207094642 wait complete. 2024-03-26T17:56:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.829+0000 7f55de24e740 20 client.22083 trim_cache size 0 max 16384 2024-03-26T17:56:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55de24e740 1 Processor -- start 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55de24e740 1 -- start start 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55de24e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da5f7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55de24e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55de24e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da607460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55de24e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41a6d10 con 0x5635da5fb590 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55de24e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41a8850 con 0x5635da5ec540 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55de24e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41a73e0 con 0x5635da5f5a30 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55d8c03640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55d9c05640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da5f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55d9c05640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da5f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39990/0 (socket says 172.21.15.105:39990) 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55d8c03640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5f7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48472/0 (socket says 172.21.15.105:48472) 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55d9c05640 1 -- 172.21.15.105:0/3756510867 learned_addr learned my addr 172.21.15.105:0/3756510867 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.850+0000 7f55d9c05640 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da607460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55d9c05640 1 -- 172.21.15.105:0/3756510867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 msgr2=0x5635da607460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55d9c05640 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da607460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55d9c05640 1 -- 172.21.15.105:0/3756510867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 msgr2=0x5635da5f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55d9c05640 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55d9c05640 1 -- 172.21.15.105:0/3756510867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635da5f0dd0 con 0x5635da5f5a30 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55d8c03640 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55d9c05640 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da5f7920 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0096c0 tx=0x7f55cc009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c42d7f7bfd66245 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55beffd640 1 -- 172.21.15.105:0/3756510867 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc026190 con 0x5635da5f5a30 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55beffd640 1 -- 172.21.15.105:0/3756510867 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55cc0267e0 con 0x5635da5f5a30 2024-03-26T17:56:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.851+0000 7f55beffd640 1 -- 172.21.15.105:0/3756510867 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc024420 con 0x5635da5f5a30 2024-03-26T17:56:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 -- 172.21.15.105:0/3756510867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 msgr2=0x5635da5f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da5f7920 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0096c0 tx=0x7f55cc009630 comp rx=0 tx=0).stop 2024-03-26T17:56:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 -- 172.21.15.105:0/3756510867 shutdown_connections 2024-03-26T17:56:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da607460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da5f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 --2- 172.21.15.105:0/3756510867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da5f7920 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 -- 172.21.15.105:0/3756510867 >> 172.21.15.105:0/3756510867 conn(0x5635da5f0fa0 msgr2=0x5635da5fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 -- 172.21.15.105:0/3756510867 shutdown_connections 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.852+0000 7f55de24e740 1 -- 172.21.15.105:0/3756510867 wait complete. 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 1 -- start start 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 1 Processor -- start 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da605190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da6056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da6066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41a8180 con 0x5635da5fb590 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41a7ab0 con 0x5635da5ec540 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41a4cf0 con 0x5635da5f5a30 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55de24e740 10 client.0 _ll_register_callbacks cb 0x7f55cc009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55d9c05640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da6066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55d9c05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da605190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55d9c05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da605190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57888/0 (socket says 172.21.15.105:57888) 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.853+0000 7f55d9c05640 1 -- 172.21.15.105:0/1453889537 learned_addr learned my addr 172.21.15.105:0/1453889537 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55d9c05640 1 -- 172.21.15.105:0/1453889537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 msgr2=0x5635da6056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55d9c05640 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da6056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55d9c05640 1 -- 172.21.15.105:0/1453889537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 msgr2=0x5635da6066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55d9c05640 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da6066e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55d9c05640 1 -- 172.21.15.105:0/1453889537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635da606cd0 con 0x5635da5ec540 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55d9c05640 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da605190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f55cc004990 tx=0x7f55cc00f890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5705d0cea189670d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55bcff9640 1 -- 172.21.15.105:0/1453889537 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00c850 con 0x5635da5ec540 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55de24e740 1 -- 172.21.15.105:0/1453889537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55cc0267e0 con 0x5635da5ec540 2024-03-26T17:56:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55bcff9640 1 -- 172.21.15.105:0/1453889537 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f55cc00c9f0 con 0x5635da5ec540 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55bcff9640 1 -- 172.21.15.105:0/1453889537 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f55cc026190 con 0x5635da5ec540 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55de24e740 10 client.31793 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.854+0000 7f55de24e740 1 -- 172.21.15.105:0/1453889537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5635da672670 con 0x5635da5ec540 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55bcff9640 1 -- 172.21.15.105:0/1453889537 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55cc0267e0 con 0x5635da5ec540 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55bcff9640 1 -- 172.21.15.105:0/1453889537 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f55cc00b040 con 0x5635da5ec540 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55de24e740 10 client.31793 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55de24e740 10 client.31793 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55de24e740 20 client.31793 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55de24e740 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5635da672c20 0x5635da675010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55de24e740 4 client.31793 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55de24e740 1 -- 172.21.15.105:0/1453889537 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635da567eb0 con 0x5635da672c20 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.855+0000 7f55d8c03640 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5635da672c20 0x5635da675010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.856+0000 7f55d8c03640 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5635da672c20 0x5635da675010 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4aadfd989f3b778 server_cookie=dddfd920aed41a9e in_seq=0 out_seq=0 2024-03-26T17:56:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.856+0000 7f55bcff9640 10 client.31793 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:56:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.856+0000 7f55bcff9640 1 -- 172.21.15.105:0/1453889537 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635da567eb0 con 0x5635da672c20 2024-03-26T17:56:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.856+0000 7f55bcff9640 10 client.31793 handle_command_reply: tid=0 2024-03-26T17:56:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 client.31793 shutdown 2024-03-26T17:56:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 -- 172.21.15.105:0/1453889537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 msgr2=0x5635da605190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da605190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f55cc004990 tx=0x7f55cc00f890 comp rx=0 tx=0).stop 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 -- 172.21.15.105:0/1453889537 shutdown_connections 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5635da672c20 0x5635da675010 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5635da5fb590 0x5635da6066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5635da5f5a30 0x5635da6056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 --2- 172.21.15.105:0/1453889537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635da5ec540 0x5635da605190 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 -- 172.21.15.105:0/1453889537 >> 172.21.15.105:0/1453889537 conn(0x5635da5f0fa0 msgr2=0x5635da5ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55bcff9640 0 client.31793 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 -- 172.21.15.105:0/1453889537 shutdown_connections 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 1 -- 172.21.15.105:0/1453889537 wait complete. 2024-03-26T17:56:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.857+0000 7f55de24e740 20 client.31793 trim_cache size 0 max 16384 2024-03-26T17:56:38.849 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 -- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f55c00a3020 msgr2=0x7f55c00a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f55c00a3020 0x7f55c00a54e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f55d009b560 tx=0x7f55d009b3f0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 -- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41007c0 msgr2=0x7f55d4197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41007c0 0x7f55d4197bf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b450 tx=0x7f55d000b910 comp rx=0 tx=0).stop 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 -- 172.21.15.105:0/2419839292 shutdown_connections 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f55c00a3020 0x7f55c00a54e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f55d4102370 0x7f55d419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55d41019b0 0x7f55d4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 --2- 172.21.15.105:0/2419839292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55d41007c0 0x7f55d4197bf0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.858+0000 7f55db668640 1 -- 172.21.15.105:0/2419839292 >> 172.21.15.105:0/2419839292 conn(0x7f55d40fbd50 msgr2=0x7f55d40fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.859+0000 7f55db668640 1 -- 172.21.15.105:0/2419839292 shutdown_connections 2024-03-26T17:56:38.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:38.859+0000 7f55db668640 1 -- 172.21.15.105:0/2419839292 wait complete. 2024-03-26T17:56:38.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:56:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 -- 172.21.15.105:0/3431672188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181007c0 msgr2=0x7f8518100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 --2- 172.21.15.105:0/3431672188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181007c0 0x7f8518100be0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8514009900 tx=0x7f851402bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 -- 172.21.15.105:0/3431672188 shutdown_connections 2024-03-26T17:56:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 --2- 172.21.15.105:0/3431672188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8518102370 0x7f8518108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 --2- 172.21.15.105:0/3431672188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85181019b0 0x7f8518101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 --2- 172.21.15.105:0/3431672188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181007c0 0x7f8518100be0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 -- 172.21.15.105:0/3431672188 >> 172.21.15.105:0/3431672188 conn(0x7f85180fbd50 msgr2=0x7f85180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 -- 172.21.15.105:0/3431672188 shutdown_connections 2024-03-26T17:56:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.026+0000 7f851fa73640 1 -- 172.21.15.105:0/3431672188 wait complete. 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851fa73640 1 Processor -- start 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851fa73640 1 -- start start 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851fa73640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85181007c0 0x7f85180695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851fa73640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181019b0 0x7f851819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851fa73640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8518102370 0x7f851819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851fa73640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f851810c2c0 con 0x7f85181019b0 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851fa73640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f851810c140 con 0x7f8518102370 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851fa73640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f851810c440 con 0x7f85181007c0 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.027+0000 7f851d80f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85181007c0 0x7f85180695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d00e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181019b0 0x7f851819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d80f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85181007c0 0x7f85180695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40022/0 (socket says 172.21.15.105:40022) 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d80f640 1 -- 172.21.15.105:0/4278297215 learned_addr learned my addr 172.21.15.105:0/4278297215 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851e010640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8518102370 0x7f851819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d80f640 1 -- 172.21.15.105:0/4278297215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8518102370 msgr2=0x7f851819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d80f640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8518102370 0x7f851819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d80f640 1 -- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181019b0 msgr2=0x7f851819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d80f640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181019b0 0x7f851819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d80f640 1 -- 172.21.15.105:0/4278297215 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851819f760 con 0x7f85181007c0 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d00e640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181019b0 0x7f851819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.028+0000 7f851d80f640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85181007c0 0x7f85180695b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8514000c00 tx=0x7f8514005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.029+0000 7f850effd640 1 -- 172.21.15.105:0/4278297215 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514043020 con 0x7f85181007c0 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.029+0000 7f850effd640 1 -- 172.21.15.105:0/4278297215 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8514004350 con 0x7f85181007c0 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.029+0000 7f851fa73640 1 -- 172.21.15.105:0/4278297215 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851819f9f0 con 0x7f85181007c0 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.029+0000 7f850effd640 1 -- 172.21.15.105:0/4278297215 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514003b80 con 0x7f85181007c0 2024-03-26T17:56:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.029+0000 7f851fa73640 1 -- 172.21.15.105:0/4278297215 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851819ff90 con 0x7f85181007c0 2024-03-26T17:56:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.033+0000 7f852265d740 1 Processor -- start 2024-03-26T17:56:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.033+0000 7f852265d740 1 -- start start 2024-03-26T17:56:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.033+0000 7f852265d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b3e40 0x5592e71b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.033+0000 7f852265d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f850effd640 1 -- 172.21.15.105:0/4278297215 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8514046020 con 0x7f85181007c0 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f852265d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b99a0 0x5592e71bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851e010640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b3e40 0x5592e71b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851e010640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b3e40 0x5592e71b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:57912/0 (socket says 172.21.15.105:57912) 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851e010640 1 -- 172.21.15.105:0/2159656256 learned_addr learned my addr 172.21.15.105:0/2159656256 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f852265d740 1 -- 172.21.15.105:0/2159656256 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5592e7114970 con 0x5592e71b4830 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851d80f640 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851d00e640 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b99a0 0x5592e71bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f852265d740 1 -- 172.21.15.105:0/2159656256 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5592e70bcd70 con 0x5592e71b3e40 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f852265d740 1 -- 172.21.15.105:0/2159656256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5592e7110920 con 0x5592e71b99a0 2024-03-26T17:56:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851e010640 1 -- 172.21.15.105:0/2159656256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b99a0 msgr2=0x5592e71bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851e010640 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b99a0 0x5592e71bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851e010640 1 -- 172.21.15.105:0/2159656256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 msgr2=0x5592e71b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851e010640 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851e010640 1 -- 172.21.15.105:0/2159656256 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e6e283f0 con 0x5592e71b3e40 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.034+0000 7f851d80f640 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.035+0000 7f850effd640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f85040a2fa0 0x7f85040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.035+0000 7f850effd640 1 -- 172.21.15.105:0/4278297215 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f85140f1530 con 0x7f85181007c0 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.035+0000 7f851e010640 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b3e40 0x5592e71b4260 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8510009fd0 tx=0x7f85100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa0a07aa48f1a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.035+0000 7f851d80f640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f85040a2fa0 0x7f85040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.035+0000 7f8503fff640 1 -- 172.21.15.105:0/2159656256 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510004d20 con 0x5592e71b3e40 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.035+0000 7f8503fff640 1 -- 172.21.15.105:0/2159656256 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8510004050 con 0x5592e71b3e40 2024-03-26T17:56:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.035+0000 7f8503fff640 1 -- 172.21.15.105:0/2159656256 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510005260 con 0x5592e71b3e40 2024-03-26T17:56:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.037+0000 7f851d80f640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f85040a2fa0 0x7f85040a5460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f851402c840 tx=0x7f85140f4f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.037+0000 7f852265d740 1 -- 172.21.15.105:0/2159656256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b3e40 msgr2=0x5592e71b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.037+0000 7f852265d740 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b3e40 0x5592e71b4260 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8510009fd0 tx=0x7f85100118d0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.038+0000 7f852265d740 1 -- 172.21.15.105:0/2159656256 shutdown_connections 2024-03-26T17:56:39.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.038+0000 7f852265d740 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b99a0 0x5592e71bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.038+0000 7f852265d740 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.038+0000 7f852265d740 1 --2- 172.21.15.105:0/2159656256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b3e40 0x5592e71b4260 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.038+0000 7f852265d740 1 -- 172.21.15.105:0/2159656256 >> 172.21.15.105:0/2159656256 conn(0x5592e71aafa0 msgr2=0x5592e71ab3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:39.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.038+0000 7f852265d740 1 -- 172.21.15.105:0/2159656256 shutdown_connections 2024-03-26T17:56:39.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.038+0000 7f852265d740 1 -- 172.21.15.105:0/2159656256 wait complete. 2024-03-26T17:56:39.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.038+0000 7f852265d740 1 -- start start 2024-03-26T17:56:39.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.039+0000 7f852265d740 1 Processor -- start 2024-03-26T17:56:39.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.039+0000 7f852265d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.039+0000 7f852265d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.039+0000 7f851e010640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.039+0000 7f851e010640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71be1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40034/0 (socket says 172.21.15.105:40034) 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.039+0000 7f851e010640 1 -- 172.21.15.105:0/1361909480 learned_addr learned my addr 172.21.15.105:0/1361909480 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f852265d740 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 0x5592e71c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5592e71bc950 con 0x5592e71b4830 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5592e70d0310 con 0x5592e71b99a0 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5592e71bcad0 con 0x5592e71b3e40 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f851d00e640 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f851e010640 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 0x5592e71c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f852265d740 10 client.0 _ll_register_callbacks cb 0x5592e710bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f851d00e640 1 -- 172.21.15.105:0/1361909480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 msgr2=0x5592e71be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f851d00e640 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f851d00e640 1 -- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 msgr2=0x5592e71c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:39.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f851d00e640 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 0x5592e71c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f851d00e640 1 -- 172.21.15.105:0/1361909480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e6ef27d0 con 0x5592e71b4830 2024-03-26T17:56:39.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8510004050 con 0x5592e71b4830 2024-03-26T17:56:39.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f851d00e640 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71be6f0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f8518069320 tx=0x7f850800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19531204d43d686 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:39.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f85017fa640 10 client.22101 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:39.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.040+0000 7f85017fa640 1 -- 172.21.15.105:0/1361909480 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508012110 con 0x5592e71b4830 2024-03-26T17:56:39.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f85017fa640 1 -- 172.21.15.105:0/1361909480 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8508004050 con 0x5592e71b4830 2024-03-26T17:56:39.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f85017fa640 1 -- 172.21.15.105:0/1361909480 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f85080122b0 con 0x5592e71b4830 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f852265d740 10 client.22101 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5592e71c8a80 con 0x5592e71b4830 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f85017fa640 1 -- 172.21.15.105:0/1361909480 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8508004250 con 0x5592e71b4830 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f852265d740 10 client.22101 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f852265d740 10 client.22101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f852265d740 20 client.22101 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f852265d740 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5592e71ca0b0 0x5592e71cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f852265d740 4 client.22101 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592e7121eb0 con 0x5592e71ca0b0 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.041+0000 7f851e010640 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5592e71ca0b0 0x5592e71cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.042+0000 7f85017fa640 10 client.22101 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:39.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.042+0000 7f851e010640 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5592e71ca0b0 0x5592e71cc4c0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf2f5411bbeee00 server_cookie=a50930c3689f1b3 in_seq=0 out_seq=0 2024-03-26T17:56:39.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.044+0000 7f85017fa640 1 -- 172.21.15.105:0/1361909480 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5592e7121eb0 con 0x5592e71ca0b0 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.044+0000 7f85017fa640 10 client.22101 handle_command_reply: tid=0 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.044+0000 7f852265d740 1 client.22101 shutdown 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.044+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 msgr2=0x5592e71be6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.044+0000 7f852265d740 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71be6f0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f8518069320 tx=0x7f850800f790 comp rx=0 tx=0).stop 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.044+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 shutdown_connections 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.044+0000 7f852265d740 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5592e71ca0b0 0x5592e71cc4c0 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.045+0000 7f852265d740 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 0x5592e71c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.045+0000 7f85017fa640 0 client.22101 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.045+0000 7f852265d740 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b4830 0x5592e71be6f0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.045+0000 7f852265d740 1 --2- 172.21.15.105:0/1361909480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.045+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 >> 172.21.15.105:0/1361909480 conn(0x5592e71aafa0 msgr2=0x5592e71aa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:39.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.045+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 shutdown_connections 2024-03-26T17:56:39.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.045+0000 7f852265d740 1 -- 172.21.15.105:0/1361909480 wait complete. 2024-03-26T17:56:39.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.045+0000 7f852265d740 20 client.22101 trim_cache size 0 max 16384 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.066+0000 7f852265d740 1 Processor -- start 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f852265d740 1 -- start start 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f852265d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b3e40 0x5592e71bce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f852265d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b4830 0x5592e71c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f852265d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 0x5592e71b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f852265d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85181a6ad0 con 0x5592e71b3e40 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f852265d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85181a8610 con 0x5592e71b99a0 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f852265d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85181a71a0 con 0x5592e71b4830 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851e010640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b3e40 0x5592e71bce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851d00e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b4830 0x5592e71c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851e010640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b3e40 0x5592e71bce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48540/0 (socket says 172.21.15.105:48540) 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851e010640 1 -- 172.21.15.105:0/2414226065 learned_addr learned my addr 172.21.15.105:0/2414226065 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851e010640 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 0x5592e71b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851d00e640 1 -- 172.21.15.105:0/2414226065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 msgr2=0x5592e71b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851d00e640 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 0x5592e71b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851d00e640 1 -- 172.21.15.105:0/2414226065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b3e40 msgr2=0x5592e71bce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851d00e640 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b3e40 0x5592e71bce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851d00e640 1 -- 172.21.15.105:0/2414226065 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e71bf2e0 con 0x5592e71b4830 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.067+0000 7f851e010640 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b3e40 0x5592e71bce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:39.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.068+0000 7f851d00e640 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b4830 0x5592e71c56f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8508009730 tx=0x7f8508013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ee674c6fc615086 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.068+0000 7f85037fe640 1 -- 172.21.15.105:0/2414226065 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85080035f0 con 0x5592e71b4830 2024-03-26T17:56:39.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.068+0000 7f85037fe640 1 -- 172.21.15.105:0/2414226065 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8510004050 con 0x5592e71b4830 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.068+0000 7f85037fe640 1 -- 172.21.15.105:0/2414226065 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850801c620 con 0x5592e71b4830 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 -- 172.21.15.105:0/2414226065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b4830 msgr2=0x5592e71c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b4830 0x5592e71c56f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8508009730 tx=0x7f8508013000 comp rx=0 tx=0).stop 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 -- 172.21.15.105:0/2414226065 shutdown_connections 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b99a0 0x5592e71b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b4830 0x5592e71c56f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 --2- 172.21.15.105:0/2414226065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b3e40 0x5592e71bce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 -- 172.21.15.105:0/2414226065 >> 172.21.15.105:0/2414226065 conn(0x5592e71aafa0 msgr2=0x5592e71ab380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 -- 172.21.15.105:0/2414226065 shutdown_connections 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 -- 172.21.15.105:0/2414226065 wait complete. 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.069+0000 7f852265d740 1 -- start start 2024-03-26T17:56:39.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.070+0000 7f852265d740 1 Processor -- start 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.070+0000 7f852265d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.070+0000 7f852265d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b4830 0x5592e71c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.070+0000 7f852265d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b99a0 0x5592e71c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.070+0000 7f851e010640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.070+0000 7f851e010640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40038/0 (socket says 172.21.15.105:40038) 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.070+0000 7f851e010640 1 -- 172.21.15.105:0/2544256135 learned_addr learned my addr 172.21.15.105:0/2544256135 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.070+0000 7f851d00e640 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b4830 0x5592e71c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85181a7f40 con 0x5592e71b99a0 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85181a7870 con 0x5592e71b4830 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85181a4ab0 con 0x5592e71b3e40 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f852265d740 10 client.0 _ll_register_callbacks cb 0x7f8508009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:39.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f851e010640 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b99a0 0x5592e71c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f851e010640 1 -- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b4830 msgr2=0x5592e71c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f851e010640 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b4830 0x5592e71c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f851e010640 1 -- 172.21.15.105:0/2544256135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b99a0 msgr2=0x5592e71c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f851e010640 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b99a0 0x5592e71c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f851e010640 1 -- 172.21.15.105:0/2544256135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e71c4dc0 con 0x5592e71b3e40 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f851d00e640 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b4830 0x5592e71c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f851e010640 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71c3280 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8510009fa0 tx=0x7f8510005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=447cd277abcc9f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f85017fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f85017fa640 1 -- 172.21.15.105:0/2544256135 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8510004ac0 con 0x5592e71b3e40 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.071+0000 7f85017fa640 1 -- 172.21.15.105:0/2544256135 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f8510004c60 con 0x5592e71b3e40 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8510004050 con 0x5592e71b3e40 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f85017fa640 1 -- 172.21.15.105:0/2544256135 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f851002d120 con 0x5592e71b3e40 2024-03-26T17:56:39.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 10 client.31795 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5592e7230870 con 0x5592e71b3e40 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f85017fa640 1 -- 172.21.15.105:0/2544256135 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f851002d770 con 0x5592e71b3e40 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f85017fa640 1 -- 172.21.15.105:0/2544256135 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8510038030 con 0x5592e71b3e40 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 10 client.31795 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 10 client.31795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 20 client.31795 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5592e71ca0b0 0x5592e7233090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 4 client.31795 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.072+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8510004050 con 0x5592e71ca0b0 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.073+0000 7f851d00e640 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5592e71ca0b0 0x5592e7233090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.073+0000 7f851d00e640 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5592e71ca0b0 0x5592e7233090 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2802e4d50c22141 server_cookie=f6d3c6fa380d833b in_seq=0 out_seq=0 2024-03-26T17:56:39.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.073+0000 7f85017fa640 10 client.31795 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.075+0000 7f85017fa640 1 -- 172.21.15.105:0/2544256135 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8510004050 con 0x5592e71ca0b0 2024-03-26T17:56:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.075+0000 7f85017fa640 10 client.31795 handle_command_reply: tid=0 2024-03-26T17:56:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.075+0000 7f852265d740 1 client.31795 shutdown 2024-03-26T17:56:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.075+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 msgr2=0x5592e71c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.075+0000 7f852265d740 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71c3280 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8510009fa0 tx=0x7f8510005fd0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 shutdown_connections 2024-03-26T17:56:39.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5592e71ca0b0 0x5592e7233090 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592e71b99a0 0x5592e71c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f85017fa640 0 client.31795 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592e71b4830 0x5592e71c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 1 --2- 172.21.15.105:0/2544256135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5592e71b3e40 0x5592e71c3280 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 >> 172.21.15.105:0/2544256135 conn(0x5592e71aafa0 msgr2=0x5592e71b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 shutdown_connections 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 1 -- 172.21.15.105:0/2544256135 wait complete. 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.076+0000 7f852265d740 20 client.31795 trim_cache size 0 max 16384 2024-03-26T17:56:39.066 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:56:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 -- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f85040a2fa0 msgr2=0x7f85040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f85040a2fa0 0x7f85040a5460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f851402c840 tx=0x7f85140f4f70 comp rx=0 tx=0).stop 2024-03-26T17:56:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 -- 172.21.15.105:0/4278297215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85181007c0 msgr2=0x7f85180695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85181007c0 0x7f85180695b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8514000c00 tx=0x7f8514005dc0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 -- 172.21.15.105:0/4278297215 shutdown_connections 2024-03-26T17:56:39.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f85040a2fa0 0x7f85040a5460 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8518102370 0x7f851819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85181019b0 0x7f851819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 --2- 172.21.15.105:0/4278297215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85181007c0 0x7f85180695b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 -- 172.21.15.105:0/4278297215 >> 172.21.15.105:0/4278297215 conn(0x7f85180fbd50 msgr2=0x7f8518104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 -- 172.21.15.105:0/4278297215 shutdown_connections 2024-03-26T17:56:39.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:39.077+0000 7f851fa73640 1 -- 172.21.15.105:0/4278297215 wait complete. 2024-03-26T17:56:39.077 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:38 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2250743478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:56:39.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:38 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2250743478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:56:39.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:38 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2250743478' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:56:39.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 42319 7.06 MB/sec execute 202 sec latency 124.695 ms 2024-03-26T17:56:40.080 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:56:40.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:56:40.080 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:56:40.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:39 smithi135 ceph-mon[35636]: pgmap v898: 97 pgs: 97 active+clean; 295 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 -- 172.21.15.105:0/3313465206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 msgr2=0x7f511c10a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 --2- 172.21.15.105:0/3313465206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 0x7f511c10a500 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5118009900 tx=0x7f511802bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 -- 172.21.15.105:0/3313465206 shutdown_connections 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 --2- 172.21.15.105:0/3313465206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f511c10bc90 0x7f511c112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 --2- 172.21.15.105:0/3313465206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c10b2d0 0x7f511c10b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 --2- 172.21.15.105:0/3313465206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 0x7f511c10a500 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 -- 172.21.15.105:0/3313465206 >> 172.21.15.105:0/3313465206 conn(0x7f511c070870 msgr2=0x7f511c072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 -- 172.21.15.105:0/3313465206 shutdown_connections 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 -- 172.21.15.105:0/3313465206 wait complete. 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.236+0000 7f5123e1f640 1 Processor -- start 2024-03-26T17:56:40.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5123e1f640 1 -- start start 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5123e1f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 0x7f511c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5123e1f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c10b2d0 0x7f511c1a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5123e1f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f511c10bc90 0x7f511c1a7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5123e1f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f511c115be0 con 0x7f511c10a0e0 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5123e1f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f511c115a60 con 0x7f511c10bc90 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5123e1f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f511c115d60 con 0x7f511c10b2d0 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5121bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 0x7f511c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5121bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 0x7f511c1a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48574/0 (socket says 172.21.15.105:48574) 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5121bbb640 1 -- 172.21.15.105:0/488729299 learned_addr learned my addr 172.21.15.105:0/488729299 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f51223bc640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f511c10bc90 0x7f511c1a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f51213ba640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c10b2d0 0x7f511c1a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5121bbb640 1 -- 172.21.15.105:0/488729299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c10b2d0 msgr2=0x7f511c1a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5121bbb640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c10b2d0 0x7f511c1a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5121bbb640 1 -- 172.21.15.105:0/488729299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f511c10bc90 msgr2=0x7f511c1a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5121bbb640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f511c10bc90 0x7f511c1a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.237+0000 7f5121bbb640 1 -- 172.21.15.105:0/488729299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511c06b060 con 0x7f511c10a0e0 2024-03-26T17:56:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.238+0000 7f5121bbb640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 0x7f511c1a03a0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5118000c00 tx=0x7f5118005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.238+0000 7f5112ffd640 1 -- 172.21.15.105:0/488729299 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118043020 con 0x7f511c10a0e0 2024-03-26T17:56:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.238+0000 7f5112ffd640 1 -- 172.21.15.105:0/488729299 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5118004350 con 0x7f511c10a0e0 2024-03-26T17:56:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.238+0000 7f5112ffd640 1 -- 172.21.15.105:0/488729299 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118003b10 con 0x7f511c10a0e0 2024-03-26T17:56:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.238+0000 7f5123e1f640 1 -- 172.21.15.105:0/488729299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511c06b2f0 con 0x7f511c10a0e0 2024-03-26T17:56:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.238+0000 7f5123e1f640 1 -- 172.21.15.105:0/488729299 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511c06b7f0 con 0x7f511c10a0e0 2024-03-26T17:56:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.241+0000 7f5112ffd640 1 -- 172.21.15.105:0/488729299 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5118046020 con 0x7f511c10a0e0 2024-03-26T17:56:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.241+0000 7f5126a05740 1 Processor -- start 2024-03-26T17:56:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5126a05740 1 -- start start 2024-03-26T17:56:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5126a05740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7190 0x55579b5e75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5126a05740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5ec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5126a05740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5ef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5126a05740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55579b54c630 con 0x55579b5e7b80 2024-03-26T17:56:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5126a05740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55579b4f4e10 con 0x55579b5eccf0 2024-03-26T17:56:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5126a05740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55579b5485e0 con 0x55579b5e7190 2024-03-26T17:56:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5121bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f51223bc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7190 0x55579b5e75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5121bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48590/0 (socket says 172.21.15.105:48590) 2024-03-26T17:56:40.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5121bbb640 1 -- 172.21.15.105:0/3754827073 learned_addr learned my addr 172.21.15.105:0/3754827073 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5112ffd640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f51080a2fa0 0x7f51080a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5121bbb640 1 -- 172.21.15.105:0/3754827073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7190 msgr2=0x55579b5e75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5121bbb640 1 --2- 172.21.15.105:0/3754827073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7190 0x55579b5e75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5121bbb640 1 -- 172.21.15.105:0/3754827073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 msgr2=0x55579b5ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5121bbb640 1 --2- 172.21.15.105:0/3754827073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f5121bbb640 1 -- 172.21.15.105:0/3754827073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55579b50b740 con 0x55579b5e7b80 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.242+0000 7f51223bc640 1 --2- 172.21.15.105:0/3754827073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7190 0x55579b5e75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.243+0000 7f5112ffd640 1 -- 172.21.15.105:0/488729299 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f51180f1510 con 0x7f511c10a0e0 2024-03-26T17:56:40.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.243+0000 7f5121bbb640 1 --2- 172.21.15.105:0/3754827073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5ec680 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f51180a11d0 tx=0x7f5118004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2362a01cb2718555 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.243+0000 7f51223bc640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f51080a2fa0 0x7f51080a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.243+0000 7f5107fff640 1 -- 172.21.15.105:0/3754827073 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180bd9d0 con 0x55579b5e7b80 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.243+0000 7f5107fff640 1 -- 172.21.15.105:0/3754827073 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51180dc970 con 0x55579b5e7b80 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.243+0000 7f5107fff640 1 -- 172.21.15.105:0/3754827073 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180bd560 con 0x55579b5e7b80 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 -- 172.21.15.105:0/3754827073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 msgr2=0x55579b5ec680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f51223bc640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f51080a2fa0 0x7f51080a5460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55579b5e83e0 tx=0x7f5114004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 --2- 172.21.15.105:0/3754827073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5ec680 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f51180a11d0 tx=0x7f5118004500 comp rx=0 tx=0).stop 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 -- 172.21.15.105:0/3754827073 shutdown_connections 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 --2- 172.21.15.105:0/3754827073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 --2- 172.21.15.105:0/3754827073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5ec680 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 --2- 172.21.15.105:0/3754827073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7190 0x55579b5e75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 -- 172.21.15.105:0/3754827073 >> 172.21.15.105:0/3754827073 conn(0x55579b5e2700 msgr2=0x55579b5e2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 -- 172.21.15.105:0/3754827073 shutdown_connections 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.244+0000 7f5126a05740 1 -- 172.21.15.105:0/3754827073 wait complete. 2024-03-26T17:56:40.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 1 -- start start 2024-03-26T17:56:40.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 1 Processor -- start 2024-03-26T17:56:40.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55579b5efa60 con 0x55579b5e7190 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55579b5081a0 con 0x55579b5eccf0 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55579b5efbe0 con 0x55579b5e7b80 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5126a05740 10 client.0 _ll_register_callbacks cb 0x55579b25fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f51213ba640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5121bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f51213ba640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40062/0 (socket says 172.21.15.105:40062) 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f51223bc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f5121bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48602/0 (socket says 172.21.15.105:48602) 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.245+0000 7f51213ba640 1 -- 172.21.15.105:0/4180546613 learned_addr learned my addr 172.21.15.105:0/4180546613 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.246+0000 7f51213ba640 1 -- 172.21.15.105:0/4180546613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 msgr2=0x55579b5f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.246+0000 7f51213ba640 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.246+0000 7f51213ba640 1 -- 172.21.15.105:0/4180546613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 msgr2=0x55579b5f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.246+0000 7f51213ba640 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.246+0000 7f51213ba640 1 -- 172.21.15.105:0/4180546613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55579b3287e0 con 0x55579b5e7b80 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.246+0000 7f5121bbb640 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.247+0000 7f51213ba640 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f1890 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f510c0116c0 tx=0x7f510c011b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8443bc3564e5d988 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.247+0000 7f51057fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.247+0000 7f51057fa640 1 -- 172.21.15.105:0/4180546613 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c00fcd0 con 0x55579b5e7b80 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.248+0000 7f5126a05740 1 -- 172.21.15.105:0/4180546613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51180dc970 con 0x55579b5e7b80 2024-03-26T17:56:40.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.248+0000 7f51057fa640 1 -- 172.21.15.105:0/4180546613 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f510c004020 con 0x55579b5e7b80 2024-03-26T17:56:40.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.249+0000 7f51057fa640 1 -- 172.21.15.105:0/4180546613 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f510c00fe70 con 0x55579b5e7b80 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.249+0000 7f5126a05740 10 client.31807 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.249+0000 7f5126a05740 1 -- 172.21.15.105:0/4180546613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55579b388f70 con 0x55579b5e7b80 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.249+0000 7f51057fa640 1 -- 172.21.15.105:0/4180546613 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51180dc970 con 0x55579b5e7b80 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f51057fa640 1 -- 172.21.15.105:0/4180546613 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f510c019430 con 0x55579b5e7b80 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f5126a05740 10 client.31807 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f5126a05740 10 client.31807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f5126a05740 20 client.31807 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f5126a05740 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55579b5fd1d0 0x55579b5ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f5126a05740 4 client.31807 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f5126a05740 1 -- 172.21.15.105:0/4180546613 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55579b559b70 con 0x55579b5fd1d0 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f5121bbb640 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55579b5fd1d0 0x55579b5ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f51057fa640 10 client.31807 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:40.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.250+0000 7f5121bbb640 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55579b5fd1d0 0x55579b5ff630 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d6ae2c372f84c7 server_cookie=7eb8b45325aca4de in_seq=0 out_seq=0 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f51057fa640 1 -- 172.21.15.105:0/4180546613 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55579b559b70 con 0x55579b5fd1d0 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f51057fa640 10 client.31807 handle_command_reply: tid=0 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 client.31807 shutdown 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 -- 172.21.15.105:0/4180546613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 msgr2=0x55579b5f1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f1890 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f510c0116c0 tx=0x7f510c011b80 comp rx=0 tx=0).stop 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 -- 172.21.15.105:0/4180546613 shutdown_connections 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55579b5fd1d0 0x55579b5ff630 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f1890 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 --2- 172.21.15.105:0/4180546613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 -- 172.21.15.105:0/4180546613 >> 172.21.15.105:0/4180546613 conn(0x55579b5e2700 msgr2=0x55579b5e1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f51057fa640 0 client.31807 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:40.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 -- 172.21.15.105:0/4180546613 shutdown_connections 2024-03-26T17:56:40.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.252+0000 7f5126a05740 1 -- 172.21.15.105:0/4180546613 wait complete. 2024-03-26T17:56:40.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.253+0000 7f5126a05740 20 client.31807 trim_cache size 0 max 16384 2024-03-26T17:56:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.278+0000 7f5126a05740 1 Processor -- start 2024-03-26T17:56:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.278+0000 7f5126a05740 1 -- start start 2024-03-26T17:56:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f5126a05740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5e7190 0x55579b5fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f5126a05740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5f8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f5126a05740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5eccf0 0x55579b5ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f5126a05740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1b38c0 con 0x55579b5e7b80 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f5126a05740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1b5400 con 0x55579b5e7190 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f5126a05740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1b3f90 con 0x55579b5eccf0 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48610/0 (socket says 172.21.15.105:48610) 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 -- 172.21.15.105:0/3054532015 learned_addr learned my addr 172.21.15.105:0/3054532015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f5121bbb640 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5e7190 0x55579b5fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51223bc640 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5eccf0 0x55579b5ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 -- 172.21.15.105:0/3054532015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5eccf0 msgr2=0x55579b5ff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5eccf0 0x55579b5ff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 -- 172.21.15.105:0/3054532015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5e7190 msgr2=0x55579b5fefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5e7190 0x55579b5fefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 -- 172.21.15.105:0/3054532015 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55579b220170 con 0x55579b5e7b80 2024-03-26T17:56:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f5121bbb640 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5e7190 0x55579b5fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.279+0000 7f51213ba640 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5f8890 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f510c0058c0 tx=0x7f510c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77c0f8a05d2e54a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.280+0000 7f51077fe640 1 -- 172.21.15.105:0/3054532015 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c030070 con 0x55579b5e7b80 2024-03-26T17:56:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.280+0000 7f51077fe640 1 -- 172.21.15.105:0/3054532015 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f510c005560 con 0x55579b5e7b80 2024-03-26T17:56:40.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.280+0000 7f51077fe640 1 -- 172.21.15.105:0/3054532015 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c027070 con 0x55579b5e7b80 2024-03-26T17:56:40.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.280+0000 7f5126a05740 1 -- 172.21.15.105:0/3054532015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 msgr2=0x55579b5f8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.280+0000 7f5126a05740 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5f8890 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f510c0058c0 tx=0x7f510c005f90 comp rx=0 tx=0).stop 2024-03-26T17:56:40.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 -- 172.21.15.105:0/3054532015 shutdown_connections 2024-03-26T17:56:40.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5eccf0 0x55579b5ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7b80 0x55579b5f8890 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 --2- 172.21.15.105:0/3054532015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5e7190 0x55579b5fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 -- 172.21.15.105:0/3054532015 >> 172.21.15.105:0/3054532015 conn(0x55579b5e2700 msgr2=0x55579b5e2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 -- 172.21.15.105:0/3054532015 shutdown_connections 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 -- 172.21.15.105:0/3054532015 wait complete. 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 -- start start 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.281+0000 7f5126a05740 1 Processor -- start 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5126a05740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5126a05740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5126a05740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5126a05740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1b4d30 con 0x55579b5e7190 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5126a05740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1b4660 con 0x55579b5eccf0 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5126a05740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f511c1b18a0 con 0x55579b5e7b80 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5126a05740 10 client.0 _ll_register_callbacks cb 0x7f510c009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48616/0 (socket says 172.21.15.105:48616) 2024-03-26T17:56:40.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 -- 172.21.15.105:0/80445869 learned_addr learned my addr 172.21.15.105:0/80445869 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f51223bc640 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f51213ba640 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 -- 172.21.15.105:0/80445869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 msgr2=0x55579b5f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 -- 172.21.15.105:0/80445869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 msgr2=0x55579b5f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 -- 172.21.15.105:0/80445869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55579b5f7e90 con 0x55579b5e7190 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5121bbb640 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f6430 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5118005ab0 tx=0x7f51180035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96392c92ae588121 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f51057fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f51057fa640 1 -- 172.21.15.105:0/80445869 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180bd740 con 0x55579b5e7190 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f51057fa640 1 -- 172.21.15.105:0/80445869 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f51180b9a90 con 0x55579b5e7190 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.282+0000 7f5126a05740 1 -- 172.21.15.105:0/80445869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f510c005560 con 0x55579b5e7190 2024-03-26T17:56:40.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f51057fa640 1 -- 172.21.15.105:0/80445869 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f51180bd9d0 con 0x55579b5e7190 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f5126a05740 10 client.22125 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f5126a05740 1 -- 172.21.15.105:0/80445869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55579b663930 con 0x55579b5e7190 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f51057fa640 1 -- 172.21.15.105:0/80445869 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51180be300 con 0x55579b5e7190 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f51057fa640 1 -- 172.21.15.105:0/80445869 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f5118003cb0 con 0x55579b5e7190 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f5126a05740 10 client.22125 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f5126a05740 10 client.22125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f5126a05740 20 client.22125 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f5126a05740 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55579b664170 0x55579b666590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f5126a05740 4 client.22125 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f5126a05740 1 -- 172.21.15.105:0/80445869 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f510c005560 con 0x55579b664170 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.283+0000 7f51213ba640 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55579b664170 0x55579b666590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.284+0000 7f51057fa640 10 client.22125 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:40.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.284+0000 7f51213ba640 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55579b664170 0x55579b666590 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbac09bf68bb3143 server_cookie=4cf503559f890cc0 in_seq=0 out_seq=0 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.286+0000 7f51057fa640 1 -- 172.21.15.105:0/80445869 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f510c005560 con 0x55579b664170 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.286+0000 7f51057fa640 10 client.22125 handle_command_reply: tid=0 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.286+0000 7f5126a05740 1 client.22125 shutdown 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.286+0000 7f5126a05740 1 -- 172.21.15.105:0/80445869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 msgr2=0x55579b5f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.286+0000 7f5126a05740 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f6430 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5118005ab0 tx=0x7f51180035f0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.286+0000 7f5126a05740 1 -- 172.21.15.105:0/80445869 shutdown_connections 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.286+0000 7f5126a05740 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55579b664170 0x55579b666590 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.286+0000 7f5126a05740 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55579b5eccf0 0x55579b5f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.287+0000 7f51057fa640 0 client.22125 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.287+0000 7f5126a05740 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55579b5e7b80 0x55579b5f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.287+0000 7f5126a05740 1 --2- 172.21.15.105:0/80445869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55579b5e7190 0x55579b5f6430 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.287+0000 7f5126a05740 1 -- 172.21.15.105:0/80445869 >> 172.21.15.105:0/80445869 conn(0x55579b5e2700 msgr2=0x55579b5e9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.287+0000 7f5126a05740 1 -- 172.21.15.105:0/80445869 shutdown_connections 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.287+0000 7f5126a05740 1 -- 172.21.15.105:0/80445869 wait complete. 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.287+0000 7f5126a05740 20 client.22125 trim_cache size 0 max 16384 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "d021a0f9-131f-4897-92ce-658715b1ffa3", 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:56:40.277 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:56:40.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 -- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f51080a2fa0 msgr2=0x7f51080a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f51080a2fa0 0x7f51080a5460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55579b5e83e0 tx=0x7f5114004a40 comp rx=0 tx=0).stop 2024-03-26T17:56:40.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 -- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 msgr2=0x7f511c1a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 0x7f511c1a03a0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5118000c00 tx=0x7f5118005dc0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 -- 172.21.15.105:0/488729299 shutdown_connections 2024-03-26T17:56:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f51080a2fa0 0x7f51080a5460 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f511c10bc90 0x7f511c1a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511c10b2d0 0x7f511c1a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 --2- 172.21.15.105:0/488729299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f511c10a0e0 0x7f511c1a03a0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.288+0000 7f5123e1f640 1 -- 172.21.15.105:0/488729299 >> 172.21.15.105:0/488729299 conn(0x7f511c070870 msgr2=0x7f511c10e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.289+0000 7f5123e1f640 1 -- 172.21.15.105:0/488729299 shutdown_connections 2024-03-26T17:56:40.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.289+0000 7f5123e1f640 1 -- 172.21.15.105:0/488729299 wait complete. 2024-03-26T17:56:40.288 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:39 smithi105 ceph-mon[32280]: pgmap v898: 97 pgs: 97 active+clean; 295 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-03-26T17:56:40.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:56:40.290 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:56:40.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:39 smithi144 ceph-mon[36095]: pgmap v898: 97 pgs: 97 active+clean; 295 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 3.7 MiB/s wr, 256 op/s 2024-03-26T17:56:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 -- 172.21.15.105:0/3192677789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba481023c0 msgr2=0x7fba481047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 --2- 172.21.15.105:0/3192677789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba481023c0 0x7fba481047b0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009900 tx=0x7fba3c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 -- 172.21.15.105:0/3192677789 shutdown_connections 2024-03-26T17:56:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 --2- 172.21.15.105:0/3192677789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba48107620 0x7fba48109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 --2- 172.21.15.105:0/3192677789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba48104cf0 0x7fba481070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 --2- 172.21.15.105:0/3192677789 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba481023c0 0x7fba481047b0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 -- 172.21.15.105:0/3192677789 >> 172.21.15.105:0/3192677789 conn(0x7fba480fbd50 msgr2=0x7fba480fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 -- 172.21.15.105:0/3192677789 shutdown_connections 2024-03-26T17:56:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.442+0000 7fba4eb77640 1 -- 172.21.15.105:0/3192677789 wait complete. 2024-03-26T17:56:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4eb77640 1 Processor -- start 2024-03-26T17:56:40.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4eb77640 1 -- start start 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4eb77640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba481023c0 0x7fba48101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4eb77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba48104cf0 0x7fba48100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4eb77640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba48107620 0x7fba481008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4eb77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba4810d410 con 0x7fba48104cf0 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4eb77640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba4810d290 con 0x7fba481023c0 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4eb77640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba4810d590 con 0x7fba48107620 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4d114640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba48107620 0x7fba481008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4d114640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba48107620 0x7fba481008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40100/0 (socket says 172.21.15.105:40100) 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4c913640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba481023c0 0x7fba48101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4d114640 1 -- 172.21.15.105:0/4055822185 learned_addr learned my addr 172.21.15.105:0/4055822185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba47fff640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba48104cf0 0x7fba48100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4d114640 1 -- 172.21.15.105:0/4055822185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba481023c0 msgr2=0x7fba48101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4d114640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba481023c0 0x7fba48101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.443+0000 7fba4d114640 1 -- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba48104cf0 msgr2=0x7fba48100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.444+0000 7fba4d114640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba48104cf0 0x7fba48100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.444+0000 7fba4d114640 1 -- 172.21.15.105:0/4055822185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba48101140 con 0x7fba48107620 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.444+0000 7fba47fff640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba48104cf0 0x7fba48100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.444+0000 7fba4d114640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba48107620 0x7fba481008b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fba4000b5b0 tx=0x7fba4000ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.444+0000 7fba45ffb640 1 -- 172.21.15.105:0/4055822185 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40013070 con 0x7fba48107620 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.444+0000 7fba4eb77640 1 -- 172.21.15.105:0/4055822185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba481977c0 con 0x7fba48107620 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.444+0000 7fba45ffb640 1 -- 172.21.15.105:0/4055822185 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba40004d20 con 0x7fba48107620 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.444+0000 7fba45ffb640 1 -- 172.21.15.105:0/4055822185 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4000fd10 con 0x7fba48107620 2024-03-26T17:56:40.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.445+0000 7fba4eb77640 1 -- 172.21.15.105:0/4055822185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba48197d60 con 0x7fba48107620 2024-03-26T17:56:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.447+0000 7fba45ffb640 1 -- 172.21.15.105:0/4055822185 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fba40014270 con 0x7fba48107620 2024-03-26T17:56:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba45ffb640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba380a2f20 0x7fba380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba51764740 1 Processor -- start 2024-03-26T17:56:40.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba51764740 1 -- start start 2024-03-26T17:56:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba45ffb640 1 -- 172.21.15.105:0/4055822185 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fba400cf410 con 0x7fba48107620 2024-03-26T17:56:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba4c913640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba380a2f20 0x7fba380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba51764740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c925bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba51764740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c9260de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba51764740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c92638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba51764740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b7c91c0980 con 0x55b7c925c2e0 2024-03-26T17:56:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba51764740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9168be0 con 0x55b7c9261450 2024-03-26T17:56:40.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba51764740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7c91bc930 con 0x55b7c925b8f0 2024-03-26T17:56:40.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba4c913640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c925bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.448+0000 7fba47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c9260de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4d114640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c92638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4c913640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c925bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40116/0 (socket says 172.21.15.105:40116) 2024-03-26T17:56:40.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4c913640 1 -- 172.21.15.105:0/3023705841 learned_addr learned my addr 172.21.15.105:0/3023705841 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4d114640 1 -- 172.21.15.105:0/3023705841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 msgr2=0x55b7c925bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4d114640 1 --2- 172.21.15.105:0/3023705841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c925bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4d114640 1 -- 172.21.15.105:0/3023705841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 msgr2=0x55b7c9260de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4d114640 1 --2- 172.21.15.105:0/3023705841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c9260de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4d114640 1 -- 172.21.15.105:0/3023705841 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c8f9e7d0 con 0x55b7c9261450 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba47fff640 1 --2- 172.21.15.105:0/3023705841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c9260de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4c913640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba380a2f20 0x7fba380a53e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fba3c005ec0 tx=0x7fba3c02cbb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba4d114640 1 --2- 172.21.15.105:0/3023705841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c92638e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fba400ccf20 tx=0x7fba4009ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c66780ac50cc360 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba32ffd640 1 -- 172.21.15.105:0/3023705841 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4009bb90 con 0x55b7c9261450 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba32ffd640 1 -- 172.21.15.105:0/3023705841 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba4009fce0 con 0x55b7c9261450 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.449+0000 7fba32ffd640 1 -- 172.21.15.105:0/3023705841 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40014560 con 0x55b7c9261450 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 -- 172.21.15.105:0/3023705841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 msgr2=0x55b7c92638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 --2- 172.21.15.105:0/3023705841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c92638e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fba400ccf20 tx=0x7fba4009ba20 comp rx=0 tx=0).stop 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 -- 172.21.15.105:0/3023705841 shutdown_connections 2024-03-26T17:56:40.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 --2- 172.21.15.105:0/3023705841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c92638e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 --2- 172.21.15.105:0/3023705841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c9260de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 --2- 172.21.15.105:0/3023705841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c925bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 -- 172.21.15.105:0/3023705841 >> 172.21.15.105:0/3023705841 conn(0x55b7c9256e60 msgr2=0x55b7c9257260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 -- 172.21.15.105:0/3023705841 shutdown_connections 2024-03-26T17:56:40.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.450+0000 7fba51764740 1 -- 172.21.15.105:0/3023705841 wait complete. 2024-03-26T17:56:40.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.451+0000 7fba51764740 1 -- start start 2024-03-26T17:56:40.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.451+0000 7fba51764740 1 Processor -- start 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.451+0000 7fba51764740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925b8f0 0x55b7c9265b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.451+0000 7fba51764740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c925c2e0 0x55b7c92660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.451+0000 7fba51764740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c9261450 0x55b7c926d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.451+0000 7fba51764740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9264400 con 0x55b7c925b8f0 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.451+0000 7fba51764740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7c917c160 con 0x55b7c925c2e0 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.451+0000 7fba51764740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9264580 con 0x55b7c9261450 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925b8f0 0x55b7c9265b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925b8f0 0x55b7c9265b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48652/0 (socket says 172.21.15.105:48652) 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 -- 172.21.15.105:0/1408224619 learned_addr learned my addr 172.21.15.105:0/1408224619 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba51764740 10 client.0 _ll_register_callbacks cb 0x55b7c91b8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba47fff640 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c925c2e0 0x55b7c92660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4d114640 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c9261450 0x55b7c926d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 -- 172.21.15.105:0/1408224619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c9261450 msgr2=0x55b7c926d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c9261450 0x55b7c926d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 -- 172.21.15.105:0/1408224619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c925c2e0 msgr2=0x55b7c92660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c925c2e0 0x55b7c92660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 -- 172.21.15.105:0/1408224619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c8ed43f0 con 0x55b7c925b8f0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba4c913640 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925b8f0 0x55b7c9265b80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fba3c036040 tx=0x7fba3c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a165b195b1478875 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba27fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.452+0000 7fba27fff640 1 -- 172.21.15.105:0/1408224619 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c05a070 con 0x55b7c925b8f0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba27fff640 1 -- 172.21.15.105:0/1408224619 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fba3c005310 con 0x55b7c925b8f0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 1 -- 172.21.15.105:0/1408224619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba4009fce0 con 0x55b7c925b8f0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba27fff640 1 -- 172.21.15.105:0/1408224619 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fba3c005510 con 0x55b7c925b8f0 2024-03-26T17:56:40.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 10 client.22137 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 1 -- 172.21.15.105:0/1408224619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b7c8e940f0 con 0x55b7c925b8f0 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba27fff640 1 -- 172.21.15.105:0/1408224619 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba3c004b70 con 0x55b7c925b8f0 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba27fff640 1 -- 172.21.15.105:0/1408224619 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fba3c003a20 con 0x55b7c925b8f0 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 10 client.22137 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 10 client.22137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 20 client.22137 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b7c9271960 0x55b7c9273db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 4 client.22137 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.453+0000 7fba51764740 1 -- 172.21.15.105:0/1408224619 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fba4009fce0 con 0x55b7c9271960 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.454+0000 7fba47fff640 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b7c9271960 0x55b7c9273db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.454+0000 7fba27fff640 10 client.22137 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.454+0000 7fba47fff640 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b7c9271960 0x55b7c9273db0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f4f00f0da779c6 server_cookie=7a14fe3c082cbbd in_seq=0 out_seq=0 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba27fff640 1 -- 172.21.15.105:0/1408224619 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fba4009fce0 con 0x55b7c9271960 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba27fff640 10 client.22137 handle_command_reply: tid=0 2024-03-26T17:56:40.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 client.22137 shutdown 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 -- 172.21.15.105:0/1408224619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925b8f0 msgr2=0x55b7c9265b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925b8f0 0x55b7c9265b80 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fba3c036040 tx=0x7fba3c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 -- 172.21.15.105:0/1408224619 shutdown_connections 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b7c9271960 0x55b7c9273db0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c9261450 0x55b7c926d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba27fff640 0 client.22137 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c925c2e0 0x55b7c92660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 --2- 172.21.15.105:0/1408224619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925b8f0 0x55b7c9265b80 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 -- 172.21.15.105:0/1408224619 >> 172.21.15.105:0/1408224619 conn(0x55b7c9256e60 msgr2=0x55b7c9256080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 -- 172.21.15.105:0/1408224619 shutdown_connections 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 1 -- 172.21.15.105:0/1408224619 wait complete. 2024-03-26T17:56:40.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.456+0000 7fba51764740 20 client.22137 trim_cache size 0 max 16384 2024-03-26T17:56:40.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.480+0000 7fba51764740 1 Processor -- start 2024-03-26T17:56:40.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.480+0000 7fba51764740 1 -- start start 2024-03-26T17:56:40.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.480+0000 7fba51764740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c9266020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.480+0000 7fba51764740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.480+0000 7fba51764740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c9257240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba51764740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba481a6c60 con 0x55b7c925c2e0 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba51764740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba481a87a0 con 0x55b7c9261450 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba51764740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba481a7330 con 0x55b7c925b8f0 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48664/0 (socket says 172.21.15.105:48664) 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba4c913640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c9266020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 -- 172.21.15.105:0/969078338 learned_addr learned my addr 172.21.15.105:0/969078338 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba4d114640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c9257240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 -- 172.21.15.105:0/969078338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 msgr2=0x55b7c9266020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 --2- 172.21.15.105:0/969078338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c9266020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 -- 172.21.15.105:0/969078338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 msgr2=0x55b7c9257240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 --2- 172.21.15.105:0/969078338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c9257240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 -- 172.21.15.105:0/969078338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c9256800 con 0x55b7c925c2e0 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba47fff640 1 --2- 172.21.15.105:0/969078338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926d0c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x55b7c925cd50 tx=0x7fba34005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb814dcb5e690c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba327fc640 1 -- 172.21.15.105:0/969078338 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34004230 con 0x55b7c925c2e0 2024-03-26T17:56:40.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba327fc640 1 -- 172.21.15.105:0/969078338 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba34025030 con 0x55b7c925c2e0 2024-03-26T17:56:40.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.481+0000 7fba327fc640 1 -- 172.21.15.105:0/969078338 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34005590 con 0x55b7c925c2e0 2024-03-26T17:56:40.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 -- 172.21.15.105:0/969078338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 msgr2=0x55b7c926d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 --2- 172.21.15.105:0/969078338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926d0c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x55b7c925cd50 tx=0x7fba34005f90 comp rx=0 tx=0).stop 2024-03-26T17:56:40.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 -- 172.21.15.105:0/969078338 shutdown_connections 2024-03-26T17:56:40.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 --2- 172.21.15.105:0/969078338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c9257240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 --2- 172.21.15.105:0/969078338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926d0c0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 --2- 172.21.15.105:0/969078338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c9266020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 -- 172.21.15.105:0/969078338 >> 172.21.15.105:0/969078338 conn(0x55b7c9256e60 msgr2=0x55b7c925a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 -- 172.21.15.105:0/969078338 shutdown_connections 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.482+0000 7fba51764740 1 -- 172.21.15.105:0/969078338 wait complete. 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 1 -- start start 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 1 Processor -- start 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c926ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c926c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fba481a80d0 con 0x55b7c925c2e0 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba481a7a00 con 0x55b7c9261450 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fba481a4c60 con 0x55b7c925b8f0 2024-03-26T17:56:40.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba4c913640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c926ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba51764740 10 client.0 _ll_register_callbacks cb 0x7fba400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba4c913640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c926ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40136/0 (socket says 172.21.15.105:40136) 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba4c913640 1 -- 172.21.15.105:0/2803554226 learned_addr learned my addr 172.21.15.105:0/2803554226 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.483+0000 7fba4d114640 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c926c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba4c913640 1 -- 172.21.15.105:0/2803554226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 msgr2=0x55b7c926c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba4c913640 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c926c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba4c913640 1 -- 172.21.15.105:0/2803554226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 msgr2=0x55b7c926b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba4c913640 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba4c913640 1 -- 172.21.15.105:0/2803554226 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c926c780 con 0x55b7c925b8f0 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba4c913640 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c926ad00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fba3c02c170 tx=0x7fba3c05eae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c901fa3c1dbd96c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba27fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba27fff640 1 -- 172.21.15.105:0/2803554226 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c075060 con 0x55b7c925b8f0 2024-03-26T17:56:40.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba27fff640 1 -- 172.21.15.105:0/2803554226 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7fba3c0035f0 con 0x55b7c925b8f0 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba51764740 1 -- 172.21.15.105:0/2803554226 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba34025030 con 0x55b7c925b8f0 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba27fff640 1 -- 172.21.15.105:0/2803554226 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fba3c05ee60 con 0x55b7c925b8f0 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.484+0000 7fba51764740 10 client.31831 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba51764740 1 -- 172.21.15.105:0/2803554226 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b7c92d81b0 con 0x55b7c925b8f0 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba27fff640 1 -- 172.21.15.105:0/2803554226 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba34025030 con 0x55b7c925b8f0 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba27fff640 1 -- 172.21.15.105:0/2803554226 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fba3c06f030 con 0x55b7c925b8f0 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba51764740 10 client.31831 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba51764740 10 client.31831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba51764740 20 client.31831 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba51764740 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b7c9271960 0x55b7c92dab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba51764740 4 client.31831 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba51764740 1 -- 172.21.15.105:0/2803554226 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7c91cdec0 con 0x55b7c9271960 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.485+0000 7fba47fff640 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b7c9271960 0x55b7c92dab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:40.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.486+0000 7fba27fff640 10 client.31831 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:40.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.486+0000 7fba47fff640 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b7c9271960 0x55b7c92dab10 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d87e9cfb61b506 server_cookie=3b29d71b832c006f in_seq=0 out_seq=0 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.488+0000 7fba27fff640 1 -- 172.21.15.105:0/2803554226 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55b7c91cdec0 con 0x55b7c9271960 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.488+0000 7fba27fff640 10 client.31831 handle_command_reply: tid=0 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.488+0000 7fba51764740 1 client.31831 shutdown 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.488+0000 7fba51764740 1 -- 172.21.15.105:0/2803554226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 msgr2=0x55b7c926ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.488+0000 7fba51764740 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c926ad00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fba3c02c170 tx=0x7fba3c05eae0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 1 -- 172.21.15.105:0/2803554226 shutdown_connections 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b7c9271960 0x55b7c92dab10 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9261450 0x55b7c926c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7c925c2e0 0x55b7c926b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 1 --2- 172.21.15.105:0/2803554226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7c925b8f0 0x55b7c926ad00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 1 -- 172.21.15.105:0/2803554226 >> 172.21.15.105:0/2803554226 conn(0x55b7c9256e60 msgr2=0x55b7c925a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba27fff640 0 client.31831 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 1 -- 172.21.15.105:0/2803554226 shutdown_connections 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 1 -- 172.21.15.105:0/2803554226 wait complete. 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.489+0000 7fba51764740 20 client.31831 trim_cache size 0 max 16384 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (208 inodes in the stack)", 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stdout: "d021a0f9-131f-4897-92ce-658715b1ffa3": { 2024-03-26T17:56:40.479 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:56:40.480 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "d021a0f9-131f-4897-92ce-658715b1ffa3", 2024-03-26T17:56:40.480 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:56:40.480 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:56:40.480 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:56:40.480 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 -- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba380a2f20 msgr2=0x7fba380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba380a2f20 0x7fba380a53e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fba3c005ec0 tx=0x7fba3c02cbb0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 -- 172.21.15.105:0/4055822185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba48107620 msgr2=0x7fba481008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba48107620 0x7fba481008b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fba4000b5b0 tx=0x7fba4000ba70 comp rx=0 tx=0).stop 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 -- 172.21.15.105:0/4055822185 shutdown_connections 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fba380a2f20 0x7fba380a53e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba48107620 0x7fba481008b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fba48104cf0 0x7fba48100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 --2- 172.21.15.105:0/4055822185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba481023c0 0x7fba48101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:40.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 -- 172.21.15.105:0/4055822185 >> 172.21.15.105:0/4055822185 conn(0x7fba480fbd50 msgr2=0x7fba48102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:40.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 -- 172.21.15.105:0/4055822185 shutdown_connections 2024-03-26T17:56:40.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:56:40.490+0000 7fba4eb77640 1 -- 172.21.15.105:0/4055822185 wait complete. 2024-03-26T17:56:40.492 INFO:tasks.cephfs.filesystem:scrub status for tag:d021a0f9-131f-4897-92ce-658715b1ffa3 - {'path': '/', 'tag': 'd021a0f9-131f-4897-92ce-658715b1ffa3', 'options': 'recursive,force'} 2024-03-26T17:56:40.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 42456 7.07 MB/sec execute 203 sec latency 172.357 ms 2024-03-26T17:56:40.658 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:56:40.355+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003d46 [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} v8177 ap=2 RANDEPHEMERALPIN f(v311 m2024-03-26T17:56:20.202415+0000 24=24+0) n(v27 rc2024-03-26T17:56:20.202415+0000 b5230239 25=24+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@5209} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d47cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003d46:[<0x10000003bc0/PARADOX v108>,<0x10000003bb6/~dmtmp v48>,<0x10000000007/client8 v2078>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003d46:[<0x10000003bc0/PARADOX v8177>,<0x10000003bb6/~dmtmp v1723>,<0x10000000007/client8 v7809>,<0x10000000006/clients v2532>,<0x10000000005/tmp v2346>,<0x10000000003/client.0 v2247>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v2184>,<0x10000000001/sv_0 v2194>,<0x10000000000/qa v2120>,<0x1/volumes v2094>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:56:20.085711+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:56:20.202415+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v311 m2024-03-26T17:56:20.202415+0000 24=24+0)","memory_value.rstat":"n(v27 rc2024-03-26T17:56:20.202415+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T17:56:41.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:56:41.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:41.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:41.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003d46 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:56:41.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:41.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.343 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:40 smithi144 ceph-mon[36095]: pgmap v899: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-03-26T17:56:41.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:56:41.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:41.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:41.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003d46 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:56:41.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:41.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:40 smithi105 ceph-mon[32280]: pgmap v899: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-03-26T17:56:41.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:56:41.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:41.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:41.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003d46 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:56:41.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:41.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:41.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:40 smithi135 ceph-mon[35636]: pgmap v899: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-03-26T17:56:41.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 42612 7.08 MB/sec execute 204 sec latency 151.600 ms 2024-03-26T17:56:42.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:41 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:42.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:41 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:42.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:41 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:56:42.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:41 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:41 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:42.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:41 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:42.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:41 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:56:42.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:41 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:41 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:42.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:41 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:42.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:41 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:56:42.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:41 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:56:42.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 42742 7.11 MB/sec execute 205 sec latency 125.939 ms 2024-03-26T17:56:43.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:42 smithi144 ceph-mon[36095]: pgmap v900: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.3 MiB/s wr, 276 op/s 2024-03-26T17:56:43.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:42 smithi105 ceph-mon[32280]: pgmap v900: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.3 MiB/s wr, 276 op/s 2024-03-26T17:56:43.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:42 smithi135 ceph-mon[35636]: pgmap v900: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.3 MiB/s wr, 276 op/s 2024-03-26T17:56:43.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 42887 7.09 MB/sec execute 206 sec latency 135.416 ms 2024-03-26T17:56:44.553 INFO:tasks.workunit.client.0.smithi105.stdout: 10 43038 7.09 MB/sec execute 207 sec latency 124.783 ms 2024-03-26T17:56:45.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 43173 7.07 MB/sec execute 208 sec latency 137.712 ms 2024-03-26T17:56:46.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:45 smithi135 ceph-mon[35636]: pgmap v901: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 3.4 MiB/s wr, 323 op/s 2024-03-26T17:56:46.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:45 smithi144 ceph-mon[36095]: pgmap v901: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 3.4 MiB/s wr, 323 op/s 2024-03-26T17:56:46.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:45 smithi105 ceph-mon[32280]: pgmap v901: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 3.4 MiB/s wr, 323 op/s 2024-03-26T17:56:46.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 43319 7.07 MB/sec execute 209 sec latency 103.219 ms 2024-03-26T17:56:46.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.953+0000 7f904832f640 1 -- 172.21.15.105:0/254774575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040101900 msgr2=0x7f9040101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.953+0000 7f904832f640 1 --2- 172.21.15.105:0/254774575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040101900 0x7f9040101d20 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f903c009900 tx=0x7f903c02bc50 comp rx=0 tx=0).stop 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.953+0000 7f904832f640 1 -- 172.21.15.105:0/254774575 shutdown_connections 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.953+0000 7f904832f640 1 --2- 172.21.15.105:0/254774575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90401034b0 0x7f9040109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.953+0000 7f904832f640 1 --2- 172.21.15.105:0/254774575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9040102af0 0x7f9040102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.953+0000 7f904832f640 1 --2- 172.21.15.105:0/254774575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040101900 0x7f9040101d20 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.953+0000 7f904832f640 1 -- 172.21.15.105:0/254774575 >> 172.21.15.105:0/254774575 conn(0x7f90400fce50 msgr2=0x7f90400ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 -- 172.21.15.105:0/254774575 shutdown_connections 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 -- 172.21.15.105:0/254774575 wait complete. 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 Processor -- start 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 -- start start 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9040101900 0x7f9040075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.941 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040102af0 0x7f9040073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90401034b0 0x7f90400741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f904010d4d0 con 0x7f9040102af0 2024-03-26T17:56:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f904010d350 con 0x7f9040101900 2024-03-26T17:56:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f904832f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f904010d650 con 0x7f90401034b0 2024-03-26T17:56:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f90458ca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040102af0 0x7f9040073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f90458ca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040102af0 0x7f9040073cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46142/0 (socket says 172.21.15.105:46142) 2024-03-26T17:56:46.942 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.954+0000 7f90458ca640 1 -- 172.21.15.105:0/1064932471 learned_addr learned my addr 172.21.15.105:0/1064932471 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90468cc640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90401034b0 0x7f90400741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90458ca640 1 -- 172.21.15.105:0/1064932471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90401034b0 msgr2=0x7f90400741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90458ca640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90401034b0 0x7f90400741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90458ca640 1 -- 172.21.15.105:0/1064932471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9040101900 msgr2=0x7f9040075670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T17:56:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90458ca640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9040101900 0x7f9040075670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90458ca640 1 -- 172.21.15.105:0/1064932471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9040074a80 con 0x7f9040102af0 2024-03-26T17:56:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90468cc640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90401034b0 0x7f90400741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:56:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90458ca640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040102af0 0x7f9040073cb0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f903000b450 tx=0x7f903000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90377fe640 1 -- 172.21.15.105:0/1064932471 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030015020 con 0x7f9040102af0 2024-03-26T17:56:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f904832f640 1 -- 172.21.15.105:0/1064932471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90401a1c60 con 0x7f9040102af0 2024-03-26T17:56:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90377fe640 1 -- 172.21.15.105:0/1064932471 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9030011d40 con 0x7f9040102af0 2024-03-26T17:56:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f904832f640 1 -- 172.21.15.105:0/1064932471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90401a21b0 con 0x7f9040102af0 2024-03-26T17:56:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.955+0000 7f90377fe640 1 -- 172.21.15.105:0/1064932471 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903000f340 con 0x7f9040102af0 2024-03-26T17:56:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.958+0000 7f90377fe640 1 -- 172.21.15.105:0/1064932471 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9030018020 con 0x7f9040102af0 2024-03-26T17:56:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.958+0000 7f904aff0740 1 Processor -- start 2024-03-26T17:56:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f904aff0740 1 -- start start 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f90377fe640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f902c0a2fc0 0x7f902c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f904aff0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f904aff0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b712ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f904aff0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 0x55a42b7155c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f904aff0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a42b6720b0 con 0x55a42b70d5d0 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f904aff0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a42b61a960 con 0x55a42b713130 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f904aff0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a42b66e060 con 0x55a42b70dfc0 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f90377fe640 1 -- 172.21.15.105:0/1064932471 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f90300cef40 con 0x7f9040102af0 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f90458ca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b712ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f90468cc640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f902c0a2fc0 0x7f902c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.946 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f90458ca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b712ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:51322/0 (socket says 172.21.15.105:51322) 2024-03-26T17:56:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f90458ca640 1 -- 172.21.15.105:0/2740513301 learned_addr learned my addr 172.21.15.105:0/2740513301 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.959+0000 7f90460cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90468cc640 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 0x55a42b7155c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90458ca640 1 -- 172.21.15.105:0/2740513301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 msgr2=0x55a42b7155c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90458ca640 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 0x55a42b7155c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90458ca640 1 -- 172.21.15.105:0/2740513301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 msgr2=0x55a42b70d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90458ca640 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90468cc640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f902c0a2fc0 0x7f902c0a5480 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9038009810 tx=0x7f9038009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90458ca640 1 -- 172.21.15.105:0/2740513301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42b62cc60 con 0x55a42b70dfc0 2024-03-26T17:56:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90460cb640 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f90458ca640 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b712ac0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f903009b5c0 tx=0x7f903009bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78817d7098a097df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f9034bfe640 1 -- 172.21.15.105:0/2740513301 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903009f780 con 0x55a42b70dfc0 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f9034bfe640 1 -- 172.21.15.105:0/2740513301 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90300a4040 con 0x55a42b70dfc0 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.960+0000 7f9034bfe640 1 -- 172.21.15.105:0/2740513301 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903009aa80 con 0x55a42b70dfc0 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 -- 172.21.15.105:0/2740513301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 msgr2=0x55a42b712ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b712ac0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f903009b5c0 tx=0x7f903009bdf0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 -- 172.21.15.105:0/2740513301 shutdown_connections 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 0x55a42b7155c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b712ac0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 --2- 172.21.15.105:0/2740513301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 -- 172.21.15.105:0/2740513301 >> 172.21.15.105:0/2740513301 conn(0x55a42b708b40 msgr2=0x55a42b708f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 -- 172.21.15.105:0/2740513301 shutdown_connections 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 -- 172.21.15.105:0/2740513301 wait complete. 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 -- start start 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.962+0000 7f904aff0740 1 Processor -- start 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f904aff0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b717280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f904aff0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70dfc0 0x55a42b7177c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f904aff0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f904aff0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a42b716020 con 0x55a42b70d5d0 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f904aff0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a42b62ddb0 con 0x55a42b70dfc0 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f904aff0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a42b7161a0 con 0x55a42b713130 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f904aff0740 10 client.0 _ll_register_callbacks cb 0x55a42b632110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90460cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b717280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90458ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70dfc0 0x55a42b7177c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90468cc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90458ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70dfc0 0x55a42b7177c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:39250/0 (socket says 172.21.15.105:39250) 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90460cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b717280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46152/0 (socket says 172.21.15.105:46152) 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90458ca640 1 -- 172.21.15.105:0/3109125212 learned_addr learned my addr 172.21.15.105:0/3109125212 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90468cc640 1 -- 172.21.15.105:0/3109125212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70dfc0 msgr2=0x55a42b7177c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90468cc640 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70dfc0 0x55a42b7177c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90468cc640 1 -- 172.21.15.105:0/3109125212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 msgr2=0x55a42b717280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90468cc640 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b717280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.963+0000 7f90468cc640 1 -- 172.21.15.105:0/3109125212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42b386210 con 0x55a42b713130 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f90468cc640 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71e800 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9038005c40 tx=0x7f9038004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=263f06724e84dd7b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f9029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f9029ffb640 1 -- 172.21.15.105:0/3109125212 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038026020 con 0x55a42b713130 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f904aff0740 1 -- 172.21.15.105:0/3109125212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90300a4040 con 0x55a42b713130 2024-03-26T17:56:46.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f9029ffb640 1 -- 172.21.15.105:0/3109125212 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9038004580 con 0x55a42b713130 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f9029ffb640 1 -- 172.21.15.105:0/3109125212 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9038004780 con 0x55a42b713130 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f904aff0740 10 client.31840 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f904aff0740 1 -- 172.21.15.105:0/3109125212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a42b452620 con 0x55a42b713130 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f9029ffb640 1 -- 172.21.15.105:0/3109125212 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90300a4040 con 0x55a42b713130 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f9029ffb640 1 -- 172.21.15.105:0/3109125212 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9038012450 con 0x55a42b713130 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f904aff0740 10 client.31840 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f904aff0740 10 client.31840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.964+0000 7f904aff0740 20 client.31840 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.965+0000 7f904aff0740 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a42b7231c0 0x55a42b725630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.965+0000 7f904aff0740 4 client.31840 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.965+0000 7f904aff0740 1 -- 172.21.15.105:0/3109125212 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a42b67f5f0 con 0x55a42b7231c0 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.965+0000 7f90460cb640 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a42b7231c0 0x55a42b725630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.965+0000 7f9029ffb640 10 client.31840 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.965+0000 7f90460cb640 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a42b7231c0 0x55a42b725630 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9a80fee8af3b89 server_cookie=65124431102e7915 in_seq=0 out_seq=0 2024-03-26T17:56:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f9029ffb640 1 -- 172.21.15.105:0/3109125212 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a42b67f5f0 con 0x55a42b7231c0 2024-03-26T17:56:46.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f9029ffb640 10 client.31840 handle_command_reply: tid=0 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 client.31840 shutdown 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 -- 172.21.15.105:0/3109125212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 msgr2=0x55a42b71e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71e800 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9038005c40 tx=0x7f9038004ab0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 -- 172.21.15.105:0/3109125212 shutdown_connections 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a42b7231c0 0x55a42b725630 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71e800 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f9029ffb640 0 client.31840 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70dfc0 0x55a42b7177c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 --2- 172.21.15.105:0/3109125212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b717280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 -- 172.21.15.105:0/3109125212 >> 172.21.15.105:0/3109125212 conn(0x55a42b708b40 msgr2=0x55a42b707df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 -- 172.21.15.105:0/3109125212 shutdown_connections 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.966+0000 7f904aff0740 1 -- 172.21.15.105:0/3109125212 wait complete. 2024-03-26T17:56:46.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.967+0000 7f904aff0740 20 client.31840 trim_cache size 0 max 16384 2024-03-26T17:56:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.990+0000 7f904aff0740 1 Processor -- start 2024-03-26T17:56:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.990+0000 7f904aff0740 1 -- start start 2024-03-26T17:56:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f904aff0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f904aff0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b71e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f904aff0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 0x55a42b70ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f904aff0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f902400e530 con 0x55a42b70d5d0 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f904aff0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9024010070 con 0x55a42b713130 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f904aff0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f902400ec00 con 0x55a42b70dfc0 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90460cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90458ca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b71e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90460cb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46160/0 (socket says 172.21.15.105:46160) 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90460cb640 1 -- 172.21.15.105:0/480781124 learned_addr learned my addr 172.21.15.105:0/480781124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90468cc640 1 --2- 172.21.15.105:0/480781124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 0x55a42b70ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90458ca640 1 -- 172.21.15.105:0/480781124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 msgr2=0x55a42b70ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90458ca640 1 --2- 172.21.15.105:0/480781124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 0x55a42b70ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90458ca640 1 -- 172.21.15.105:0/480781124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 msgr2=0x55a42b70ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90458ca640 1 --2- 172.21.15.105:0/480781124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.991+0000 7f90458ca640 1 -- 172.21.15.105:0/480781124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42b717b10 con 0x55a42b70dfc0 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.992+0000 7f90458ca640 1 --2- 172.21.15.105:0/480781124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b71e720 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f90300ccd70 tx=0x7f9030084bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f0e4b4b0ab1588 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.992+0000 7f902bfff640 1 -- 172.21.15.105:0/480781124 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90300b3020 con 0x55a42b70dfc0 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.992+0000 7f902bfff640 1 -- 172.21.15.105:0/480781124 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90300a4040 con 0x55a42b70dfc0 2024-03-26T17:56:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.992+0000 7f902bfff640 1 -- 172.21.15.105:0/480781124 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903009ace0 con 0x55a42b70dfc0 2024-03-26T17:56:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 -- 172.21.15.105:0/480781124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 msgr2=0x55a42b71e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 --2- 172.21.15.105:0/480781124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b71e720 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f90300ccd70 tx=0x7f9030084bc0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 -- 172.21.15.105:0/480781124 shutdown_connections 2024-03-26T17:56:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 --2- 172.21.15.105:0/480781124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b713130 0x55a42b70ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 --2- 172.21.15.105:0/480781124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b70dfc0 0x55a42b71e720 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 --2- 172.21.15.105:0/480781124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70d5d0 0x55a42b70ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 -- 172.21.15.105:0/480781124 >> 172.21.15.105:0/480781124 conn(0x55a42b708b40 msgr2=0x55a42b712330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 -- 172.21.15.105:0/480781124 shutdown_connections 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 -- 172.21.15.105:0/480781124 wait complete. 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 -- start start 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.993+0000 7f904aff0740 1 Processor -- start 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f904aff0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70d5d0 0x55a42b71c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f904aff0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70dfc0 0x55a42b71c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f904aff0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f904aff0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f902400f9a0 con 0x55a42b70dfc0 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90458ca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70dfc0 0x55a42b71c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90458ca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70dfc0 0x55a42b71c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46176/0 (socket says 172.21.15.105:46176) 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90458ca640 1 -- 172.21.15.105:0/2850319862 learned_addr learned my addr 172.21.15.105:0/2850319862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90460cb640 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70d5d0 0x55a42b71c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90468cc640 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f904aff0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f902400f2d0 con 0x55a42b70d5d0 2024-03-26T17:56:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f902400c510 con 0x55a42b713130 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f904aff0740 10 client.0 _ll_register_callbacks cb 0x7f90380099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90468cc640 1 -- 172.21.15.105:0/2850319862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70d5d0 msgr2=0x55a42b71c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90468cc640 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70d5d0 0x55a42b71c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90468cc640 1 -- 172.21.15.105:0/2850319862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70dfc0 msgr2=0x55a42b71c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90468cc640 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70dfc0 0x55a42b71c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90468cc640 1 -- 172.21.15.105:0/2850319862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42b71df20 con 0x55a42b713130 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90458ca640 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70dfc0 0x55a42b71c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f90468cc640 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71d930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9038014440 tx=0x7f90380046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fcccd91de667aa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.994+0000 7f9029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f9029ffb640 1 -- 172.21.15.105:0/2850319862 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038026040 con 0x55a42b713130 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f9029ffb640 1 -- 172.21.15.105:0/2850319862 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8293+0+0 (secure 0 0 0) 0x7f9038022470 con 0x55a42b713130 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90300a4040 con 0x55a42b713130 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f9029ffb640 1 -- 172.21.15.105:0/2850319862 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9038014a30 con 0x55a42b713130 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 10 client.31846 fetch_fsmap learned FSMap version 37 2024-03-26T17:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a42b789b90 con 0x55a42b713130 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f9029ffb640 1 -- 172.21.15.105:0/2850319862 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90300a4040 con 0x55a42b713130 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f9029ffb640 1 -- 172.21.15.105:0/2850319862 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f9038037440 con 0x55a42b713130 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 10 client.31846 fetch_fsmap finished waiting for FSMap version 37 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 10 client.31846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 20 client.31846 populate_metadata read hostname 'smithi105' 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a42b78a5e0 0x55a42b78ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 4 client.31846 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.995+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a42b679940 con 0x55a42b78a5e0 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.996+0000 7f90460cb640 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a42b78a5e0 0x55a42b78ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.996+0000 7f9029ffb640 10 client.31846 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:46.996+0000 7f90460cb640 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a42b78a5e0 0x55a42b78ca00 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80d6bab51fd502e server_cookie=4f7da56545f617fa in_seq=0 out_seq=0 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f9029ffb640 1 -- 172.21.15.105:0/2850319862 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a42b679940 con 0x55a42b78a5e0 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f9029ffb640 10 client.31846 handle_command_reply: tid=0 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f904aff0740 1 client.31846 shutdown 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 msgr2=0x55a42b71d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f904aff0740 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71d930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9038014440 tx=0x7f90380046b0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 shutdown_connections 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f904aff0740 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a42b78a5e0 0x55a42b78ca00 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f904aff0740 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a42b713130 0x55a42b71d930 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f9029ffb640 0 client.31846 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f904aff0740 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a42b70dfc0 0x55a42b71c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.133+0000 7f904aff0740 1 --2- 172.21.15.105:0/2850319862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a42b70d5d0 0x55a42b71c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.134+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 >> 172.21.15.105:0/2850319862 conn(0x55a42b708b40 msgr2=0x55a42b70e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.134+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 shutdown_connections 2024-03-26T17:56:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.134+0000 7f904aff0740 1 -- 172.21.15.105:0/2850319862 wait complete. 2024-03-26T17:56:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.134+0000 7f904aff0740 20 client.31846 trim_cache size 0 max 16384 2024-03-26T17:56:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:56:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:56:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:56:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:56:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.134+0000 7f904832f640 1 -- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f902c0a2fc0 msgr2=0x7f902c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.134+0000 7f904832f640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f902c0a2fc0 0x7f902c0a5480 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9038009810 tx=0x7f9038009760 comp rx=0 tx=0).stop 2024-03-26T17:56:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.134+0000 7f904832f640 1 -- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040102af0 msgr2=0x7f9040073cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:56:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.134+0000 7f904832f640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040102af0 0x7f9040073cb0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f903000b450 tx=0x7f903000b910 comp rx=0 tx=0).stop 2024-03-26T17:56:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.135+0000 7f904832f640 1 -- 172.21.15.105:0/1064932471 shutdown_connections 2024-03-26T17:56:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.135+0000 7f904832f640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f902c0a2fc0 0x7f902c0a5480 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.135+0000 7f904832f640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90401034b0 0x7f90400741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.135+0000 7f904832f640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9040102af0 0x7f9040073cb0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.135+0000 7f904832f640 1 --2- 172.21.15.105:0/1064932471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9040101900 0x7f9040075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:56:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.135+0000 7f904832f640 1 -- 172.21.15.105:0/1064932471 >> 172.21.15.105:0/1064932471 conn(0x7f90400fce50 msgr2=0x7f9040105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:56:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.135+0000 7f904832f640 1 -- 172.21.15.105:0/1064932471 shutdown_connections 2024-03-26T17:56:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:56:47.135+0000 7f904832f640 1 -- 172.21.15.105:0/1064932471 wait complete. 2024-03-26T17:56:47.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 43456 7.05 MB/sec execute 210 sec latency 215.188 ms 2024-03-26T17:56:47.907 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:56:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setuser ceph since I am not root 2024-03-26T17:56:47.907 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:56:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setgroup ceph since I am not root 2024-03-26T17:56:47.907 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:56:47 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/2185726813 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560df9277a40 con 0x560df924ac00 2024-03-26T17:56:47.908 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:56:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T17:56:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:47 smithi135 ceph-mon[35636]: pgmap v902: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-03-26T17:56:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:47 smithi135 ceph-mon[35636]: daemon mds.e restarted 2024-03-26T17:56:48.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:47 smithi144 ceph-mon[36095]: pgmap v902: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-03-26T17:56:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:47 smithi144 ceph-mon[36095]: daemon mds.e restarted 2024-03-26T17:56:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:47 smithi105 ceph-mon[32280]: pgmap v902: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-03-26T17:56:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:47 smithi105 ceph-mon[32280]: daemon mds.e restarted 2024-03-26T17:56:48.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 43574 7.04 MB/sec execute 211 sec latency 124.352 ms 2024-03-26T17:56:49.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:48 smithi135 ceph-mon[35636]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T17:56:49.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:48 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/2697191136,v1:172.21.15.135:6835/2697191136] up:boot 2024-03-26T17:56:49.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:48 smithi135 ceph-mon[35636]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:56:49.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:48 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:56:49.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:48 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:56:49.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:48 smithi135 ceph-mon[35636]: pgmap v904: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-26T17:56:49.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:48 smithi144 ceph-mon[36095]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T17:56:49.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:48 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/2697191136,v1:172.21.15.135:6835/2697191136] up:boot 2024-03-26T17:56:49.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:48 smithi144 ceph-mon[36095]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:56:49.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:48 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:56:49.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:48 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:56:49.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:48 smithi144 ceph-mon[36095]: pgmap v904: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-26T17:56:49.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:48 smithi105 ceph-mon[32280]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T17:56:49.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:48 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/2697191136,v1:172.21.15.135:6835/2697191136] up:boot 2024-03-26T17:56:49.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:48 smithi105 ceph-mon[32280]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:56:49.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:48 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:56:49.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:48 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:56:49.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:48 smithi105 ceph-mon[32280]: pgmap v904: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 2.7 MiB/s wr, 291 op/s 2024-03-26T17:56:49.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 43729 7.04 MB/sec execute 212 sec latency 96.535 ms 2024-03-26T17:56:50.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 43852 7.03 MB/sec execute 213 sec latency 108.326 ms 2024-03-26T17:56:51.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 43980 7.03 MB/sec execute 214 sec latency 107.065 ms 2024-03-26T17:56:52.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:51 smithi135 ceph-mon[35636]: pgmap v905: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-03-26T17:56:52.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:51 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:56:52.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:51 smithi144 ceph-mon[36095]: pgmap v905: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-03-26T17:56:52.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:51 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:56:52.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:51 smithi105 ceph-mon[32280]: pgmap v905: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-03-26T17:56:52.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:51 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:56:52.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 44144 7.05 MB/sec execute 215 sec latency 110.939 ms 2024-03-26T17:56:53.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:52 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:56:53.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:52 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:56:53.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:52 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:56:53.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:52 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:56:53.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:52 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:56:53.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:52 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:56:53.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:52 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:56:53.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:52 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:56:53.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:52 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:56:53.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 44276 7.06 MB/sec execute 216 sec latency 182.763 ms 2024-03-26T17:56:54.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:53 smithi135 ceph-mon[35636]: pgmap v906: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-03-26T17:56:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:53 smithi144 ceph-mon[36095]: pgmap v906: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-03-26T17:56:54.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:53 smithi105 ceph-mon[32280]: pgmap v906: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-03-26T17:56:54.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 44470 7.10 MB/sec execute 217 sec latency 145.180 ms 2024-03-26T17:56:55.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:54 smithi144 ceph-mon[36095]: pgmap v907: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-26T17:56:55.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:54 smithi105 ceph-mon[32280]: pgmap v907: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-26T17:56:55.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:54 smithi135 ceph-mon[35636]: pgmap v907: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-03-26T17:56:55.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 44625 7.10 MB/sec execute 218 sec latency 134.654 ms 2024-03-26T17:56:56.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 44808 7.11 MB/sec execute 219 sec latency 138.941 ms 2024-03-26T17:56:57.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 44907 7.11 MB/sec execute 220 sec latency 123.903 ms 2024-03-26T17:56:58.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:57 smithi135 ceph-mon[35636]: pgmap v908: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-03-26T17:56:58.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:57 smithi144 ceph-mon[36095]: pgmap v908: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-03-26T17:56:58.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:57 smithi105 ceph-mon[32280]: pgmap v908: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-03-26T17:56:58.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45015 7.12 MB/sec execute 221 sec latency 138.428 ms 2024-03-26T17:56:59.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45161 7.13 MB/sec execute 222 sec latency 151.650 ms 2024-03-26T17:57:00.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:56:59 smithi135 ceph-mon[35636]: pgmap v909: 97 pgs: 97 active+clean; 305 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 4.5 MiB/s wr, 270 op/s 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 -- 172.21.15.105:0/3459010314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a4102390 msgr2=0x7fe4a4108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/3459010314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a4102390 0x7fe4a4108c20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009ca0 tx=0x7fe49c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 -- 172.21.15.105:0/3459010314 shutdown_connections 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/3459010314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a4102390 0x7fe4a4108c20 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/3459010314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a41019d0 0x7fe4a4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/3459010314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a41007e0 0x7fe4a4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 -- 172.21.15.105:0/3459010314 >> 172.21.15.105:0/3459010314 conn(0x7fe4a40fbd50 msgr2=0x7fe4a40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 -- 172.21.15.105:0/3459010314 shutdown_connections 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.250+0000 7fe4abbd4640 1 -- 172.21.15.105:0/3459010314 wait complete. 2024-03-26T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4abbd4640 1 Processor -- start 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4abbd4640 1 -- start start 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4abbd4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a41007e0 0x7fe4a4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4abbd4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a41019d0 0x7fe4a4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4abbd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a4102390 0x7fe4a419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4abbd4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a410c310 con 0x7fe4a41007e0 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4abbd4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a410c190 con 0x7fe4a41019d0 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4abbd4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a410c490 con 0x7fe4a4102390 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4a9970640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a41007e0 0x7fe4a4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4a916f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a41019d0 0x7fe4a4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4a9970640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a41007e0 0x7fe4a4197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50778/0 (socket says 172.21.15.105:50778) 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4a9970640 1 -- 172.21.15.105:0/970304464 learned_addr learned my addr 172.21.15.105:0/970304464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.251+0000 7fe4aa171640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a4102390 0x7fe4a419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4a9970640 1 -- 172.21.15.105:0/970304464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a4102390 msgr2=0x7fe4a419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4a9970640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a4102390 0x7fe4a419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4a9970640 1 -- 172.21.15.105:0/970304464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a41019d0 msgr2=0x7fe4a4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4a9970640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a41019d0 0x7fe4a4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4a9970640 1 -- 172.21.15.105:0/970304464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a419f690 con 0x7fe4a41007e0 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4a916f640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a41019d0 0x7fe4a4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4aa171640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a4102390 0x7fe4a419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4a9970640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a41007e0 0x7fe4a4197a70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000e9c0 tx=0x7fe4a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe49affd640 1 -- 172.21.15.105:0/970304464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a000cde0 con 0x7fe4a41007e0 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe49affd640 1 -- 172.21.15.105:0/970304464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4a00108e0 con 0x7fe4a41007e0 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4abbd4640 1 -- 172.21.15.105:0/970304464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a419f980 con 0x7fe4a41007e0 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe49affd640 1 -- 172.21.15.105:0/970304464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0018990 con 0x7fe4a41007e0 2024-03-26T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.252+0000 7fe4abbd4640 1 -- 172.21.15.105:0/970304464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a419fed0 con 0x7fe4a41007e0 2024-03-26T17:57:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.255+0000 7fe49affd640 1 -- 172.21.15.105:0/970304464 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe4a001f020 con 0x7fe4a41007e0 2024-03-26T17:57:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4ae7c2740 1 Processor -- start 2024-03-26T17:57:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4ae7c2740 1 -- start start 2024-03-26T17:57:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55726a4060b0 con 0x55726a4ab540 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55726a3ae960 con 0x55726a4a59e0 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55726a402060 con 0x55726a4a63d0 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe49affd640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4900a3040 0x7fe4900a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.256+0000 7fe4a916f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a916f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50794/0 (socket says 172.21.15.105:50794) 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4aa171640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a916f640 1 -- 172.21.15.105:0/4126035921 learned_addr learned my addr 172.21.15.105:0/4126035921 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a9970640 1 --2- 172.21.15.105:0/4126035921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4aa171640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4900a3040 0x7fe4900a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe49affd640 1 -- 172.21.15.105:0/970304464 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fe4a00cf550 con 0x7fe4a41007e0 2024-03-26T17:57:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a916f640 1 -- 172.21.15.105:0/4126035921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 msgr2=0x55726a4aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a916f640 1 --2- 172.21.15.105:0/4126035921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a916f640 1 -- 172.21.15.105:0/4126035921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 msgr2=0x55726a4a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a916f640 1 --2- 172.21.15.105:0/4126035921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a916f640 1 -- 172.21.15.105:0/4126035921 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726a3c0c60 con 0x55726a4ab540 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4a916f640 1 --2- 172.21.15.105:0/4126035921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4ad9d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fe4940114f0 tx=0x7fe4940118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0c549040d6d114 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe4aa171640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4900a3040 0x7fe4900a5500 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe49c02c630 tx=0x7fe49c032f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe48ffff640 1 -- 172.21.15.105:0/4126035921 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494004020 con 0x55726a4ab540 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe48ffff640 1 -- 172.21.15.105:0/4126035921 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4940041c0 con 0x55726a4ab540 2024-03-26T17:57:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.257+0000 7fe48ffff640 1 -- 172.21.15.105:0/4126035921 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49400f780 con 0x55726a4ab540 2024-03-26T17:57:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/4126035921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 msgr2=0x55726a4ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/4126035921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4ad9d0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fe4940114f0 tx=0x7fe4940118f0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/4126035921 shutdown_connections 2024-03-26T17:57:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/4126035921 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4ad9d0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/4126035921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/4126035921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/4126035921 >> 172.21.15.105:0/4126035921 conn(0x55726a49cb40 msgr2=0x55726a49cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/4126035921 shutdown_connections 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/4126035921 wait complete. 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 -- start start 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.258+0000 7fe4ae7c2740 1 Processor -- start 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55726a4ae4f0 con 0x55726a4ab540 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55726a3c1db0 con 0x55726a4a59e0 2024-03-26T17:57:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55726a4ae670 con 0x55726a4a63d0 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4ae7c2740 10 client.0 _ll_register_callbacks cb 0x55726a3c6110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4aa171640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a9970640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43662/0 (socket says 172.21.15.105:43662) 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 -- 172.21.15.105:0/2368753344 learned_addr learned my addr 172.21.15.105:0/2368753344 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 -- 172.21.15.105:0/2368753344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 msgr2=0x55726a4af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 -- 172.21.15.105:0/2368753344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 msgr2=0x55726a4b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 -- 172.21.15.105:0/2368753344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726a11a210 con 0x55726a4a63d0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4aa171640 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe4a916f640 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe494009350 tx=0x7fe494009dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0285375ee5ff1ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.259+0000 7fe48d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe48d7fa640 1 -- 172.21.15.105:0/2368753344 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494027070 con 0x55726a4a63d0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/2368753344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4940041c0 con 0x55726a4a63d0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe48d7fa640 1 -- 172.21.15.105:0/2368753344 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fe494005040 con 0x55726a4a63d0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe48d7fa640 1 -- 172.21.15.105:0/2368753344 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe494033050 con 0x55726a4a63d0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 10 client.31864 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/2368753344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55726a1e6620 con 0x55726a4a63d0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe48d7fa640 1 -- 172.21.15.105:0/2368753344 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4940041c0 con 0x55726a4a63d0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe48d7fa640 1 -- 172.21.15.105:0/2368753344 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fe49400f4e0 con 0x55726a4a63d0 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 10 client.31864 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 10 client.31864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 20 client.31864 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55726a4bb460 0x55726a4bd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 4 client.31864 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/2368753344 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55726a4135f0 con 0x55726a4bb460 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.260+0000 7fe4a9970640 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55726a4bb460 0x55726a4bd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.261+0000 7fe4a9970640 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55726a4bb460 0x55726a4bd850 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9513573f7d3f1b50 server_cookie=55af8bc119438747 in_seq=0 out_seq=0 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.261+0000 7fe48d7fa640 10 client.31864 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.262+0000 7fe48d7fa640 1 -- 172.21.15.105:0/2368753344 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55726a4135f0 con 0x55726a4bb460 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.262+0000 7fe48d7fa640 10 client.31864 handle_command_reply: tid=0 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.262+0000 7fe4ae7c2740 1 client.31864 shutdown 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/2368753344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 msgr2=0x55726a4afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe494009350 tx=0x7fe494009dc0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/2368753344 shutdown_connections 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55726a4bb460 0x55726a4bd850 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4ab540 0x55726a4b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe48d7fa640 0 client.31864 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/2368753344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/2368753344 >> 172.21.15.105:0/2368753344 conn(0x55726a49cb40 msgr2=0x55726a49bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/2368753344 shutdown_connections 2024-03-26T17:57:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/2368753344 wait complete. 2024-03-26T17:57:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.263+0000 7fe4ae7c2740 20 client.31864 trim_cache size 0 max 16384 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.285+0000 7fe4ae7c2740 1 Processor -- start 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4ae7c2740 1 -- start start 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a59e0 0x55726a4af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a4b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a41a6af0 con 0x55726a4a59e0 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a41a8630 con 0x55726a4a63d0 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a41a71c0 con 0x55726a4ab540 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a9970640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a59e0 0x55726a4af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4aa171640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a4b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4aa171640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a4b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:43672/0 (socket says 172.21.15.105:43672) 2024-03-26T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4aa171640 1 -- 172.21.15.105:0/247407022 learned_addr learned my addr 172.21.15.105:0/247407022 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a916f640 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a9970640 1 -- 172.21.15.105:0/247407022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 msgr2=0x55726a4b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a9970640 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a4b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a9970640 1 -- 172.21.15.105:0/247407022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a63d0 msgr2=0x55726a4afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a9970640 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a9970640 1 -- 172.21.15.105:0/247407022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726a4b8160 con 0x55726a4a59e0 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a916f640 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.286+0000 7fe4a9970640 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a59e0 0x55726a4af760 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00cd060 tx=0x7fe4a0007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa17121619092ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.287+0000 7fe48f7fe640 1 -- 172.21.15.105:0/247407022 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a00a4650 con 0x55726a4a59e0 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.287+0000 7fe48f7fe640 1 -- 172.21.15.105:0/247407022 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4a001d220 con 0x55726a4a59e0 2024-03-26T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.287+0000 7fe48f7fe640 1 -- 172.21.15.105:0/247407022 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a009bea0 con 0x55726a4a59e0 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/247407022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a59e0 msgr2=0x55726a4af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a59e0 0x55726a4af760 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00cd060 tx=0x7fe4a0007b90 comp rx=0 tx=0).stop 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/247407022 shutdown_connections 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a4b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a63d0 0x55726a4afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/247407022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a59e0 0x55726a4af760 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/247407022 >> 172.21.15.105:0/247407022 conn(0x55726a49cb40 msgr2=0x55726a49fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/247407022 shutdown_connections 2024-03-26T17:57:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/247407022 wait complete. 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 -- start start 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.288+0000 7fe4ae7c2740 1 Processor -- start 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a63d0 0x55726a4b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4ae7c2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a51f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a41a7f60 con 0x55726a4a63d0 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a41a7890 con 0x55726a4a59e0 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4ae7c2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a41a4ad0 con 0x55726a4ab540 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4ae7c2740 10 client.0 _ll_register_callbacks cb 0x7fe4a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a63d0 0x55726a4b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a9970640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a63d0 0x55726a4b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50820/0 (socket says 172.21.15.105:50820) 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 -- 172.21.15.105:0/1690453854 learned_addr learned my addr 172.21.15.105:0/1690453854 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4aa171640 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a51f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 -- 172.21.15.105:0/1690453854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 msgr2=0x55726a51f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a51f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 -- 172.21.15.105:0/1690453854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 msgr2=0x55726a4b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 -- 172.21.15.105:0/1690453854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726a4b6de0 con 0x55726a4a63d0 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a9970640 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4aa171640 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a51f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe4a916f640 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a63d0 0x55726a4b3d10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fe494006010 tx=0x7fe494011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74de69f218833613 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe48d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.289+0000 7fe48d7fa640 1 -- 172.21.15.105:0/1690453854 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49403c060 con 0x55726a4a63d0 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe48d7fa640 1 -- 172.21.15.105:0/1690453854 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fe494003ce0 con 0x55726a4a63d0 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/1690453854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a001d220 con 0x55726a4a63d0 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe48d7fa640 1 -- 172.21.15.105:0/1690453854 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe494027040 con 0x55726a4a63d0 2024-03-26T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 10 client.22185 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/1690453854 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55726a520b10 con 0x55726a4a63d0 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe48d7fa640 1 -- 172.21.15.105:0/1690453854 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4a001d220 con 0x55726a4a63d0 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe48d7fa640 1 -- 172.21.15.105:0/1690453854 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fe494004640 con 0x55726a4a63d0 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 10 client.22185 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 10 client.22185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 20 client.22185 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55726a521480 0x55726a5238a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 4 client.22185 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/1690453854 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55726a40d940 con 0x55726a521480 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.290+0000 7fe4a9970640 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55726a521480 0x55726a5238a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.291+0000 7fe48d7fa640 10 client.22185 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.291+0000 7fe4a9970640 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55726a521480 0x55726a5238a0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc2573da0d25427 server_cookie=1cd819cbebab3835 in_seq=0 out_seq=0 2024-03-26T17:57:00.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:56:59 smithi144 ceph-mon[36095]: pgmap v909: 97 pgs: 97 active+clean; 305 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 4.5 MiB/s wr, 270 op/s 2024-03-26T17:57:00.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:56:59 smithi105 ceph-mon[32280]: pgmap v909: 97 pgs: 97 active+clean; 305 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 4.5 MiB/s wr, 270 op/s 2024-03-26T17:57:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.436+0000 7fe48d7fa640 1 -- 172.21.15.105:0/1690453854 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55726a40d940 con 0x55726a521480 2024-03-26T17:57:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.436+0000 7fe48d7fa640 10 client.22185 handle_command_reply: tid=0 2024-03-26T17:57:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 client.22185 shutdown 2024-03-26T17:57:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/1690453854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a63d0 msgr2=0x55726a4b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a63d0 0x55726a4b3d10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fe494006010 tx=0x7fe494011e70 comp rx=0 tx=0).stop 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/1690453854 shutdown_connections 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55726a521480 0x55726a5238a0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe48d7fa640 0 client.22185 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55726a4ab540 0x55726a51f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55726a4a63d0 0x55726a4b3d10 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 --2- 172.21.15.105:0/1690453854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55726a4a59e0 0x55726a4b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/1690453854 >> 172.21.15.105:0/1690453854 conn(0x55726a49cb40 msgr2=0x55726a49bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/1690453854 shutdown_connections 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 1 -- 172.21.15.105:0/1690453854 wait complete. 2024-03-26T17:57:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.437+0000 7fe4ae7c2740 20 client.22185 trim_cache size 0 max 16384 2024-03-26T17:57:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:57:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:57:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:57:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:57:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 -- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4900a3040 msgr2=0x7fe4900a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4900a3040 0x7fe4900a5500 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe49c02c630 tx=0x7fe49c032f60 comp rx=0 tx=0).stop 2024-03-26T17:57:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 -- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a41007e0 msgr2=0x7fe4a4197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a41007e0 0x7fe4a4197a70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000e9c0 tx=0x7fe4a000ee80 comp rx=0 tx=0).stop 2024-03-26T17:57:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 -- 172.21.15.105:0/970304464 shutdown_connections 2024-03-26T17:57:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe4900a3040 0x7fe4900a5500 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a4102390 0x7fe4a419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4a41019d0 0x7fe4a4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 --2- 172.21.15.105:0/970304464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4a41007e0 0x7fe4a4197a70 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 -- 172.21.15.105:0/970304464 >> 172.21.15.105:0/970304464 conn(0x7fe4a40fbd50 msgr2=0x7fe4a40fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 -- 172.21.15.105:0/970304464 shutdown_connections 2024-03-26T17:57:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:00.438+0000 7fe4abbd4640 1 -- 172.21.15.105:0/970304464 wait complete. 2024-03-26T17:57:00.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45228 7.13 MB/sec execute 223 sec latency 198.441 ms 2024-03-26T17:57:01.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45384 7.17 MB/sec execute 224 sec latency 130.019 ms 2024-03-26T17:57:02.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:01 smithi135 ceph-mon[35636]: pgmap v910: 97 pgs: 97 active+clean; 305 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 3.7 MiB/s wr, 195 op/s 2024-03-26T17:57:02.186 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:01 smithi105 ceph-mon[32280]: pgmap v910: 97 pgs: 97 active+clean; 305 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 3.7 MiB/s wr, 195 op/s 2024-03-26T17:57:02.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:01 smithi144 ceph-mon[36095]: pgmap v910: 97 pgs: 97 active+clean; 305 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 3.7 MiB/s wr, 195 op/s 2024-03-26T17:57:02.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45523 7.18 MB/sec execute 225 sec latency 120.844 ms 2024-03-26T17:57:03.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:02 smithi144 ceph-mon[36095]: pgmap v911: 97 pgs: 97 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 4.2 MiB/s wr, 206 op/s 2024-03-26T17:57:03.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:02 smithi105 ceph-mon[32280]: pgmap v911: 97 pgs: 97 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 4.2 MiB/s wr, 206 op/s 2024-03-26T17:57:03.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:02 smithi135 ceph-mon[35636]: pgmap v911: 97 pgs: 97 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 4.2 MiB/s wr, 206 op/s 2024-03-26T17:57:03.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45626 7.17 MB/sec execute 226 sec latency 149.410 ms 2024-03-26T17:57:04.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45724 7.16 MB/sec execute 227 sec latency 145.558 ms 2024-03-26T17:57:05.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45826 7.16 MB/sec execute 228 sec latency 130.936 ms 2024-03-26T17:57:06.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:05 smithi135 ceph-mon[35636]: pgmap v912: 97 pgs: 97 active+clean; 314 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 4.4 MiB/s wr, 228 op/s 2024-03-26T17:57:06.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:05 smithi144 ceph-mon[36095]: pgmap v912: 97 pgs: 97 active+clean; 314 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 4.4 MiB/s wr, 228 op/s 2024-03-26T17:57:06.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:05 smithi105 ceph-mon[32280]: pgmap v912: 97 pgs: 97 active+clean; 314 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 4.4 MiB/s wr, 228 op/s 2024-03-26T17:57:06.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 45922 7.13 MB/sec execute 229 sec latency 135.940 ms 2024-03-26T17:57:07.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 46030 7.11 MB/sec execute 230 sec latency 150.989 ms 2024-03-26T17:57:08.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:07 smithi135 ceph-mon[35636]: pgmap v913: 97 pgs: 97 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 3.7 MiB/s wr, 197 op/s 2024-03-26T17:57:08.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:07 smithi144 ceph-mon[36095]: pgmap v913: 97 pgs: 97 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 3.7 MiB/s wr, 197 op/s 2024-03-26T17:57:08.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:07 smithi105 ceph-mon[32280]: pgmap v913: 97 pgs: 97 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 3.7 MiB/s wr, 197 op/s 2024-03-26T17:57:08.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 46139 7.09 MB/sec execute 231 sec latency 121.907 ms 2024-03-26T17:57:09.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 46263 7.08 MB/sec execute 232 sec latency 134.580 ms 2024-03-26T17:57:10.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:09 smithi135 ceph-mon[35636]: pgmap v914: 97 pgs: 97 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-26T17:57:10.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:09 smithi144 ceph-mon[36095]: pgmap v914: 97 pgs: 97 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-26T17:57:10.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:09 smithi105 ceph-mon[32280]: pgmap v914: 97 pgs: 97 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 4.0 MiB/s wr, 220 op/s 2024-03-26T17:57:10.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:57:10.493 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:57:10.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 46428 7.10 MB/sec execute 233 sec latency 121.556 ms 2024-03-26T17:57:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.662+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/2637589536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d841023b0 msgr2=0x7f7d84108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.662+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/2637589536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d841023b0 0x7f7d84108c40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009be0 tx=0x7f7d7c02bef0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.663+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/2637589536 shutdown_connections 2024-03-26T17:57:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.663+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/2637589536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d841023b0 0x7f7d84108c40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.663+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/2637589536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d841019f0 0x7f7d84101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.663+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/2637589536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d84100800 0x7f7d84100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.663+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/2637589536 >> 172.21.15.105:0/2637589536 conn(0x7f7d840fbd70 msgr2=0x7f7d840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:10.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.663+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/2637589536 shutdown_connections 2024-03-26T17:57:10.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.663+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/2637589536 wait complete. 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.664+0000 7f7d8a0e8640 1 Processor -- start 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.664+0000 7f7d8a0e8640 1 -- start start 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d8a0e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d84100800 0x7f7d84199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d8a0e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d841019f0 0x7f7d8419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d8a0e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d841023b0 0x7f7d841a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d8a0e8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410c330 con 0x7f7d84100800 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d8a0e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410c1b0 con 0x7f7d841019f0 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d8a0e8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8410c4b0 con 0x7f7d841023b0 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d84100800 0x7f7d84199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d84100800 0x7f7d84199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44722/0 (socket says 172.21.15.105:44722) 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 -- 172.21.15.105:0/1702083938 learned_addr learned my addr 172.21.15.105:0/1702083938 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:10.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d83fff640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d841023b0 0x7f7d841a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d841019f0 0x7f7d8419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 -- 172.21.15.105:0/1702083938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d841023b0 msgr2=0x7f7d841a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d841023b0 0x7f7d841a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 -- 172.21.15.105:0/1702083938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d841019f0 msgr2=0x7f7d8419a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d841019f0 0x7f7d8419a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 -- 172.21.15.105:0/1702083938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d841a1860 con 0x7f7d84100800 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.665+0000 7f7d837fe640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d84100800 0x7f7d84199c50 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400e9c0 tx=0x7f7d7400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.666+0000 7f7d80ff9640 1 -- 172.21.15.105:0/1702083938 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400ccc0 con 0x7f7d84100800 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.666+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/1702083938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d841a1b50 con 0x7f7d84100800 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.666+0000 7f7d80ff9640 1 -- 172.21.15.105:0/1702083938 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d740035f0 con 0x7f7d84100800 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.666+0000 7f7d80ff9640 1 -- 172.21.15.105:0/1702083938 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400ce60 con 0x7f7d84100800 2024-03-26T17:57:10.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.666+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/1702083938 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d841a20a0 con 0x7f7d84100800 2024-03-26T17:57:10.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.669+0000 7f7d80ff9640 1 -- 172.21.15.105:0/1702083938 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7d74010690 con 0x7f7d84100800 2024-03-26T17:57:10.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.669+0000 7f7d8cdad740 1 Processor -- start 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.669+0000 7f7d8cdad740 1 -- start start 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d0423520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d04285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d042b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b1d037f980 con 0x55b1d0428c60 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b1d0327be0 con 0x55b1d0423100 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b1d037b930 con 0x55b1d0423af0 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d0423520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d04285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d0423520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60200/0 (socket says 172.21.15.105:60200) 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 -- 172.21.15.105:0/3093043898 learned_addr learned my addr 172.21.15.105:0/3093043898 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d042b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d80ff9640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d6c0a2fa0 0x7f7d6c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d80ff9640 1 -- 172.21.15.105:0/1702083938 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f7d740cf950 con 0x7f7d84100800 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d6c0a2fa0 0x7f7d6c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 -- 172.21.15.105:0/3093043898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 msgr2=0x55b1d04285f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d04285f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 -- 172.21.15.105:0/3093043898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 msgr2=0x55b1d042b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d042b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.670+0000 7f7d83fff640 1 -- 172.21.15.105:0/3093043898 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d015d7d0 con 0x55b1d0423100 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.671+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d042b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.671+0000 7f7d83fff640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d6c0a2fa0 0x7f7d6c0a5460 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c033830 tx=0x7f7d7c039000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.671+0000 7f7d83fff640 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d0423520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c035000 tx=0x7f7d7c0337c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2637d947a0550fab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.671+0000 7f7d69ffb640 1 -- 172.21.15.105:0/3093043898 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c0333e0 con 0x55b1d0423100 2024-03-26T17:57:10.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.671+0000 7f7d69ffb640 1 -- 172.21.15.105:0/3093043898 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d7c050070 con 0x55b1d0423100 2024-03-26T17:57:10.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.671+0000 7f7d69ffb640 1 -- 172.21.15.105:0/3093043898 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c0047e0 con 0x55b1d0423100 2024-03-26T17:57:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.672+0000 7f7d8cdad740 1 -- 172.21.15.105:0/3093043898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 msgr2=0x55b1d0423520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.672+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d0423520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c035000 tx=0x7f7d7c0337c0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.672+0000 7f7d8cdad740 1 -- 172.21.15.105:0/3093043898 shutdown_connections 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.672+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d042b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.672+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d04285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.672+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/3093043898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d0423520 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.672+0000 7f7d8cdad740 1 -- 172.21.15.105:0/3093043898 >> 172.21.15.105:0/3093043898 conn(0x55b1d0415e60 msgr2=0x55b1d0416260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.673+0000 7f7d8cdad740 1 -- 172.21.15.105:0/3093043898 shutdown_connections 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.673+0000 7f7d8cdad740 1 -- 172.21.15.105:0/3093043898 wait complete. 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.673+0000 7f7d8cdad740 1 -- start start 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.673+0000 7f7d8cdad740 1 Processor -- start 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0423100 0x55b1d042d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d042d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0423100 0x55b1d042d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0423100 0x55b1d042d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44748/0 (socket says 172.21.15.105:44748) 2024-03-26T17:57:10.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d837fe640 1 -- 172.21.15.105:0/1766164078 learned_addr learned my addr 172.21.15.105:0/1766164078 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0428c60 0x55b1d0434830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b1d042bbb0 con 0x55b1d0423100 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b1d033b160 con 0x55b1d0428c60 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b1d042bd30 con 0x55b1d0423af0 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d8cdad740 10 client.0 _ll_register_callbacks cb 0x55b1d0377720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d042d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.674+0000 7f7d83fff640 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0428c60 0x55b1d0434830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.675+0000 7f7d837fe640 1 -- 172.21.15.105:0/1766164078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 msgr2=0x55b1d042d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.675+0000 7f7d837fe640 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d042d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.675+0000 7f7d837fe640 1 -- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0428c60 msgr2=0x55b1d0434830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.675+0000 7f7d837fe640 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0428c60 0x55b1d0434830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.675+0000 7f7d837fe640 1 -- 172.21.15.105:0/1766164078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d00933f0 con 0x55b1d0423100 2024-03-26T17:57:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.675+0000 7f7d837fe640 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0423100 0x55b1d042d320 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f7d7409bce0 tx=0x7f7d7409a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b02575c41635ca9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.675+0000 7f7d62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:10.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.676+0000 7f7d62ffd640 1 -- 172.21.15.105:0/1766164078 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7409b630 con 0x55b1d0423100 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.676+0000 7f7d62ffd640 1 -- 172.21.15.105:0/1766164078 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f7d74097ca0 con 0x55b1d0423100 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.676+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d7c050070 con 0x55b1d0423100 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.676+0000 7f7d62ffd640 1 -- 172.21.15.105:0/1766164078 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7d740a0130 con 0x55b1d0423100 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.676+0000 7f7d8cdad740 10 client.22197 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.676+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b1d00530f0 con 0x55b1d0423100 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.676+0000 7f7d62ffd640 1 -- 172.21.15.105:0/1766164078 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d7c050070 con 0x55b1d0423100 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d62ffd640 1 -- 172.21.15.105:0/1766164078 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7d740a12c0 con 0x55b1d0423100 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d8cdad740 10 client.22197 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d8cdad740 10 client.22197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d8cdad740 20 client.22197 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b1d0439590 0x55b1d043ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d8cdad740 4 client.22197 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1d038cec0 con 0x55b1d0439590 2024-03-26T17:57:10.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b1d0439590 0x55b1d043ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.677+0000 7f7d62ffd640 10 client.22197 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.678+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b1d0439590 0x55b1d043ba30 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679a3db1e02b699 server_cookie=41ff15609badd9b0 in_seq=0 out_seq=0 2024-03-26T17:57:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.679+0000 7f7d62ffd640 1 -- 172.21.15.105:0/1766164078 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b1d038cec0 con 0x55b1d0439590 2024-03-26T17:57:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.679+0000 7f7d62ffd640 10 client.22197 handle_command_reply: tid=0 2024-03-26T17:57:10.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.680+0000 7f7d8cdad740 1 client.22197 shutdown 2024-03-26T17:57:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.680+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0423100 msgr2=0x55b1d042d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.680+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0423100 0x55b1d042d320 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f7d7409bce0 tx=0x7f7d7409a120 comp rx=0 tx=0).stop 2024-03-26T17:57:10.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 shutdown_connections 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b1d0439590 0x55b1d043ba30 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0428c60 0x55b1d0434830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d042d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/1766164078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0423100 0x55b1d042d320 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 >> 172.21.15.105:0/1766164078 conn(0x55b1d0415e60 msgr2=0x55b1d0415080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d62ffd640 0 client.22197 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 shutdown_connections 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.681+0000 7f7d8cdad740 1 -- 172.21.15.105:0/1766164078 wait complete. 2024-03-26T17:57:10.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.682+0000 7f7d8cdad740 20 client.22197 trim_cache size 0 max 16384 2024-03-26T17:57:10.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.713+0000 7f7d8cdad740 1 Processor -- start 2024-03-26T17:57:10.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.713+0000 7f7d8cdad740 1 -- start start 2024-03-26T17:57:10.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.713+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d042cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.713+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d042d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.713+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d04147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.713+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7000e480 con 0x55b1d0428c60 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.713+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7000ffc0 con 0x55b1d0423100 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.713+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7000eb50 con 0x55b1d0423af0 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d042cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d042cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60216/0 (socket says 172.21.15.105:60216) 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 -- 172.21.15.105:0/2845412794 learned_addr learned my addr 172.21.15.105:0/2845412794 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d83fff640 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d04147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d042d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 -- 172.21.15.105:0/2845412794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 msgr2=0x55b1d042d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d042d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 -- 172.21.15.105:0/2845412794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 msgr2=0x55b1d04147f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d04147f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 -- 172.21.15.105:0/2845412794 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1d04255c0 con 0x55b1d0423100 2024-03-26T17:57:10.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d83fff640 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d04147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:10.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.714+0000 7f7d837fe640 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d042cf30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d7409bf70 tx=0x7f7d7409c160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a36441ac0898b482 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:10.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.715+0000 7f7d697fa640 1 -- 172.21.15.105:0/2845412794 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740a49f0 con 0x55b1d0423100 2024-03-26T17:57:10.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.715+0000 7f7d697fa640 1 -- 172.21.15.105:0/2845412794 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d74004000 con 0x55b1d0423100 2024-03-26T17:57:10.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.715+0000 7f7d697fa640 1 -- 172.21.15.105:0/2845412794 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74097f00 con 0x55b1d0423100 2024-03-26T17:57:10.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2845412794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 msgr2=0x55b1d042cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d042cf30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d7409bf70 tx=0x7f7d7409c160 comp rx=0 tx=0).stop 2024-03-26T17:57:10.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2845412794 shutdown_connections 2024-03-26T17:57:10.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d04147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423af0 0x55b1d042d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2845412794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423100 0x55b1d042cf30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2845412794 >> 172.21.15.105:0/2845412794 conn(0x55b1d0415e60 msgr2=0x55b1d0416240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2845412794 shutdown_connections 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2845412794 wait complete. 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 -- start start 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.716+0000 7f7d8cdad740 1 Processor -- start 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423100 0x55b1d0440c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423af0 0x55b1d0441180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d8cdad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d049de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7000f8f0 con 0x55b1d0428c60 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7000f220 con 0x55b1d0423af0 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d8cdad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7000c460 con 0x55b1d0423100 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d8cdad740 10 client.0 _ll_register_callbacks cb 0x7f7d740981f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d82ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423af0 0x55b1d0441180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d82ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423af0 0x55b1d0441180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60230/0 (socket says 172.21.15.105:60230) 2024-03-26T17:57:10.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d82ffd640 1 -- 172.21.15.105:0/2074079275 learned_addr learned my addr 172.21.15.105:0/2074079275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d83fff640 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d049de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d837fe640 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423100 0x55b1d0440c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d82ffd640 1 -- 172.21.15.105:0/2074079275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423100 msgr2=0x55b1d0440c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423100 0x55b1d0440c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d82ffd640 1 -- 172.21.15.105:0/2074079275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 msgr2=0x55b1d049de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d049de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.717+0000 7f7d82ffd640 1 -- 172.21.15.105:0/2074079275 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1d049e420 con 0x55b1d0423af0 2024-03-26T17:57:10.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.718+0000 7f7d82ffd640 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423af0 0x55b1d0441180 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55b1d0416dd0 tx=0x7f7d780108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11abb65a79f35970 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:10.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.718+0000 7f7d62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:10.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.718+0000 7f7d62ffd640 1 -- 172.21.15.105:0/2074079275 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78005050 con 0x55b1d0423af0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.718+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2074079275 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d74004000 con 0x55b1d0423af0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.718+0000 7f7d62ffd640 1 -- 172.21.15.105:0/2074079275 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f7d780051f0 con 0x55b1d0423af0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d62ffd640 1 -- 172.21.15.105:0/2074079275 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7d780053f0 con 0x55b1d0423af0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d8cdad740 10 client.31898 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2074079275 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b1d049f790 con 0x55b1d0423af0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d62ffd640 1 -- 172.21.15.105:0/2074079275 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d74004000 con 0x55b1d0423af0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d62ffd640 1 -- 172.21.15.105:0/2074079275 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7d78020250 con 0x55b1d0423af0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d8cdad740 10 client.31898 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d8cdad740 10 client.31898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d8cdad740 20 client.31898 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b1d04a01a0 0x55b1d04a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d8cdad740 4 client.31898 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.719+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2074079275 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1d0387210 con 0x55b1d04a01a0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.720+0000 7f7d837fe640 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b1d04a01a0 0x55b1d04a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.720+0000 7f7d837fe640 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b1d04a01a0 0x55b1d04a25c0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7429dfd481a63053 server_cookie=40d5eb23352a2a38 in_seq=0 out_seq=0 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.720+0000 7f7d62ffd640 10 client.31898 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.720+0000 7f7d62ffd640 1 -- 172.21.15.105:0/2074079275 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b1d0387210 con 0x55b1d04a01a0 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.720+0000 7f7d62ffd640 10 client.31898 handle_command_reply: tid=0 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.720+0000 7f7d8cdad740 1 client.31898 shutdown 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2074079275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423af0 msgr2=0x55b1d0441180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423af0 0x55b1d0441180 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55b1d0416dd0 tx=0x7f7d780108c0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2074079275 shutdown_connections 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b1d04a01a0 0x55b1d04a25c0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b1d0428c60 0x55b1d049de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b1d0423af0 0x55b1d0441180 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0x55b1d0416dd0 tx=0x7f7d780108c0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 --2- 172.21.15.105:0/2074079275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b1d0423100 0x55b1d0440c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2074079275 >> 172.21.15.105:0/2074079275 conn(0x55b1d0415e60 msgr2=0x55b1d042cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d62ffd640 0 client.31898 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.721+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2074079275 shutdown_connections 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.722+0000 7f7d8cdad740 1 -- 172.21.15.105:0/2074079275 wait complete. 2024-03-26T17:57:10.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.722+0000 7f7d8cdad740 20 client.31898 trim_cache size 0 max 16384 2024-03-26T17:57:10.709 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:57:10.709 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:57:10.709 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:57:10.709 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d6c0a2fa0 msgr2=0x7f7d6c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d6c0a2fa0 0x7f7d6c0a5460 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c033830 tx=0x7f7d7c039000 comp rx=0 tx=0).stop 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d84100800 msgr2=0x7f7d84199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d84100800 0x7f7d84199c50 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400e9c0 tx=0x7f7d7400edc0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/1702083938 shutdown_connections 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7d6c0a2fa0 0x7f7d6c0a5460 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d841023b0 0x7f7d841a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d841019f0 0x7f7d8419a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 --2- 172.21.15.105:0/1702083938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d84100800 0x7f7d84199c50 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/1702083938 >> 172.21.15.105:0/1702083938 conn(0x7f7d840fbd70 msgr2=0x7f7d840fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/1702083938 shutdown_connections 2024-03-26T17:57:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.723+0000 7f7d8a0e8640 1 -- 172.21.15.105:0/1702083938 wait complete. 2024-03-26T17:57:10.725 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:57:10.725 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:57:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 -- 172.21.15.105:0/2441129311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 msgr2=0x7f278c10bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 --2- 172.21.15.105:0/2441129311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 0x7f278c10bf40 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2784009ca0 tx=0x7f278402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 -- 172.21.15.105:0/2441129311 shutdown_connections 2024-03-26T17:57:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 --2- 172.21.15.105:0/2441129311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 0x7f278c10bf40 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 --2- 172.21.15.105:0/2441129311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f278c104cf0 0x7f278c105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 --2- 172.21.15.105:0/2441129311 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f278c103b00 0x7f278c103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 -- 172.21.15.105:0/2441129311 >> 172.21.15.105:0/2441129311 conn(0x7f278c0ff070 msgr2=0x7f278c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 -- 172.21.15.105:0/2441129311 shutdown_connections 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.886+0000 7f2792df4640 1 -- 172.21.15.105:0/2441129311 wait complete. 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2792df4640 1 Processor -- start 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2792df4640 1 -- start start 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2792df4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f278c103b00 0x7f278c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2792df4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f278c104cf0 0x7f278c19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2792df4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 0x7f278c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2792df4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f278c10f630 con 0x7f278c1056b0 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2792df4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f278c10f4b0 con 0x7f278c104cf0 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2792df4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f278c10f7b0 con 0x7f278c103b00 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 0x7f278c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 0x7f278c1a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44788/0 (socket says 172.21.15.105:44788) 2024-03-26T17:57:10.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 -- 172.21.15.105:0/3954062 learned_addr learned my addr 172.21.15.105:0/3954062 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2790b90640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f278c103b00 0x7f278c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f278bfff640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f278c104cf0 0x7f278c19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 -- 172.21.15.105:0/3954062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f278c103b00 msgr2=0x7f278c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f278c103b00 0x7f278c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 -- 172.21.15.105:0/3954062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f278c104cf0 msgr2=0x7f278c19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f278c104cf0 0x7f278c19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 -- 172.21.15.105:0/3954062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c1a1910 con 0x7f278c1056b0 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.887+0000 7f2791391640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 0x7f278c1a1270 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f278402c700 tx=0x7f27840328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.888+0000 7f2789ffb640 1 -- 172.21.15.105:0/3954062 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784041020 con 0x7f278c1056b0 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.888+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c1a1ba0 con 0x7f278c1056b0 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.888+0000 7f2789ffb640 1 -- 172.21.15.105:0/3954062 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2784003e20 con 0x7f278c1056b0 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.888+0000 7f2789ffb640 1 -- 172.21.15.105:0/3954062 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784039040 con 0x7f278c1056b0 2024-03-26T17:57:10.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.888+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c1a20e0 con 0x7f278c1056b0 2024-03-26T17:57:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.889+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278c10aef0 con 0x7f278c1056b0 2024-03-26T17:57:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.891+0000 7f2789ffb640 1 -- 172.21.15.105:0/3954062 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2784044020 con 0x7f278c1056b0 2024-03-26T17:57:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.891+0000 7f2789ffb640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f277c0a2f20 0x7f277c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:10.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.892+0000 7f2790b90640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f277c0a2f20 0x7f277c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.892+0000 7f2789ffb640 1 -- 172.21.15.105:0/3954062 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f27840f1a30 con 0x7f278c1056b0 2024-03-26T17:57:10.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.892+0000 7f2790b90640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f277c0a2f20 0x7f277c0a53e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f278c19b1d0 tx=0x7f2780005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:10.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:10.897+0000 7f2789ffb640 1 -- 172.21.15.105:0/3954062 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f278404b020 con 0x7f278c1056b0 2024-03-26T17:57:11.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.240+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f278c02d240 con 0x7f278c1056b0 2024-03-26T17:57:11.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.241+0000 7f2789ffb640 1 -- 172.21.15.105:0/3954062 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11941 (secure 0 0 0) 0x7f27840be820 con 0x7f278c1056b0 2024-03-26T17:57:11.227 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:57:11.227 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:56:47.920346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_31852":{"gid":31852,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/2697191136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2697191136},{"type":"v1","addr":"172.21.15.135:6835","nonce":2697191136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T17:57:11.227 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 39 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.243+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f277c0a2f20 msgr2=0x7f277c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.243+0000 7f2792df4640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f277c0a2f20 0x7f277c0a53e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f278c19b1d0 tx=0x7f2780005f90 comp rx=0 tx=0).stop 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.243+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 msgr2=0x7f278c1a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.243+0000 7f2792df4640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 0x7f278c1a1270 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f278402c700 tx=0x7f27840328a0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.243+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 shutdown_connections 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.244+0000 7f2792df4640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f277c0a2f20 0x7f277c0a53e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.244+0000 7f2792df4640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f278c1056b0 0x7f278c1a1270 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.244+0000 7f2792df4640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f278c104cf0 0x7f278c19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.244+0000 7f2792df4640 1 --2- 172.21.15.105:0/3954062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f278c103b00 0x7f278c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.244+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 >> 172.21.15.105:0/3954062 conn(0x7f278c0ff070 msgr2=0x7f278c1080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.244+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 shutdown_connections 2024-03-26T17:57:11.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.244+0000 7f2792df4640 1 -- 172.21.15.105:0/3954062 wait complete. 2024-03-26T17:57:11.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:10 smithi105 ceph-mon[32280]: pgmap v915: 97 pgs: 97 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 2.6 MiB/s wr, 173 op/s 2024-03-26T17:57:11.243 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:57:11.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:10 smithi144 ceph-mon[36095]: pgmap v915: 97 pgs: 97 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 2.6 MiB/s wr, 173 op/s 2024-03-26T17:57:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2275484876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a38102370 msgr2=0x7f7a38108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2275484876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a38102370 0x7f7a38108c00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009ca0 tx=0x7f7a3002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2275484876 shutdown_connections 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2275484876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a38102370 0x7f7a38108c00 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2275484876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a381019b0 0x7f7a38101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2275484876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a381007c0 0x7f7a38100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2275484876 >> 172.21.15.105:0/2275484876 conn(0x7f7a380fbd50 msgr2=0x7f7a380fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2275484876 shutdown_connections 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2275484876 wait complete. 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.392+0000 7f7a3dbe6640 1 Processor -- start 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a3dbe6640 1 -- start start 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a3dbe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a381007c0 0x7f7a381969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a3dbe6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a381019b0 0x7f7a38196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a3dbe6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a38102370 0x7f7a3819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a3dbe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c2f0 con 0x7f7a381007c0 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a3dbe6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c170 con 0x7f7a381019b0 2024-03-26T17:57:11.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a3dbe6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810c470 con 0x7f7a38102370 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a381007c0 0x7f7a381969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a381007c0 0x7f7a381969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44806/0 (socket says 172.21.15.105:44806) 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a377fe640 1 -- 172.21.15.105:0/2939249252 learned_addr learned my addr 172.21.15.105:0/2939249252 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a37fff640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a38102370 0x7f7a3819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a377fe640 1 -- 172.21.15.105:0/2939249252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a38102370 msgr2=0x7f7a3819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a38102370 0x7f7a3819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a377fe640 1 -- 172.21.15.105:0/2939249252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a381019b0 msgr2=0x7f7a38196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a381019b0 0x7f7a38196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a377fe640 1 -- 172.21.15.105:0/2939249252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a38068a20 con 0x7f7a381007c0 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.393+0000 7f7a37fff640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a38102370 0x7f7a3819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.394+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a381007c0 0x7f7a381969c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800e9c0 tx=0x7f7a2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.394+0000 7f7a34ff9640 1 -- 172.21.15.105:0/2939249252 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800cd80 con 0x7f7a381007c0 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.394+0000 7f7a34ff9640 1 -- 172.21.15.105:0/2939249252 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a280035f0 con 0x7f7a381007c0 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.394+0000 7f7a34ff9640 1 -- 172.21.15.105:0/2939249252 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28010690 con 0x7f7a381007c0 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.394+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2939249252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a38068d10 con 0x7f7a381007c0 2024-03-26T17:57:11.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.394+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2939249252 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a38069260 con 0x7f7a381007c0 2024-03-26T17:57:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.396+0000 7f7a34ff9640 1 -- 172.21.15.105:0/2939249252 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7a280037a0 con 0x7f7a381007c0 2024-03-26T17:57:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.397+0000 7f7a34ff9640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a200a2f20 0x7f7a200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.397+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a200a2f20 0x7f7a200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.397+0000 7f7a34ff9640 1 -- 172.21.15.105:0/2939249252 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f7a280cf880 con 0x7f7a381007c0 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.397+0000 7f7a407ce740 1 Processor -- start 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.397+0000 7f7a407ce740 1 -- start start 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843ca30 0x559bf843ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843d420 0x559bf8441f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8444a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a407ce740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559bf83a1970 con 0x559bf8442590 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a407ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559bf8349d70 con 0x559bf843d420 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a407ce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559bf839d920 con 0x559bf843ca30 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843ca30 0x559bf843ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8444a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a200a2f20 0x7f7a200a53e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0058c0 tx=0x7f7a2c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843ca30 0x559bf843ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56540/0 (socket says 172.21.15.105:56540) 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 -- 172.21.15.105:0/2742325870 learned_addr learned my addr 172.21.15.105:0/2742325870 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843d420 0x559bf8441f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 -- 172.21.15.105:0/2742325870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843d420 msgr2=0x559bf8441f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843d420 0x559bf8441f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 -- 172.21.15.105:0/2742325870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 msgr2=0x559bf8444a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8444a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 -- 172.21.15.105:0/2742325870 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf80b53f0 con 0x559bf843ca30 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a37fff640 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8444a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843ca30 0x559bf843ce50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7a280a4450 tx=0x7f7a2809b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d97881998730586 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.398+0000 7f7a1dffb640 1 -- 172.21.15.105:0/2742325870 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a280b4070 con 0x559bf843ca30 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a1dffb640 1 -- 172.21.15.105:0/2742325870 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a2809bf70 con 0x559bf843ca30 2024-03-26T17:57:11.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a1dffb640 1 -- 172.21.15.105:0/2742325870 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a280a44f0 con 0x559bf843ca30 2024-03-26T17:57:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 -- 172.21.15.105:0/2742325870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843ca30 msgr2=0x559bf843ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843ca30 0x559bf843ce50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7a280a4450 tx=0x7f7a2809b8d0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 -- 172.21.15.105:0/2742325870 shutdown_connections 2024-03-26T17:57:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8444a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843d420 0x559bf8441f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2742325870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843ca30 0x559bf843ce50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 -- 172.21.15.105:0/2742325870 >> 172.21.15.105:0/2742325870 conn(0x559bf8437fa0 msgr2=0x559bf84383a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 -- 172.21.15.105:0/2742325870 shutdown_connections 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.399+0000 7f7a407ce740 1 -- 172.21.15.105:0/2742325870 wait complete. 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 1 -- start start 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 1 Processor -- start 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843d420 0x559bf844b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf8442590 0x559bf84528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559bf8449950 con 0x559bf843d420 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559bf835d310 con 0x559bf843ca30 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559bf8449ad0 con 0x559bf8442590 2024-03-26T17:57:11.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.400+0000 7f7a407ce740 10 client.0 _ll_register_callbacks cb 0x559bf8398c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf8442590 0x559bf84528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a36ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843d420 0x559bf844b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf8442590 0x559bf84528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56542/0 (socket says 172.21.15.105:56542) 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 -- 172.21.15.105:0/2643648960 learned_addr learned my addr 172.21.15.105:0/2643648960 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 -- 172.21.15.105:0/2643648960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 msgr2=0x559bf844b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 -- 172.21.15.105:0/2643648960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843d420 msgr2=0x559bf844b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843d420 0x559bf844b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 -- 172.21.15.105:0/2643648960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf817f7d0 con 0x559bf8442590 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843d420 0x559bf844b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a37fff640 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf8442590 0x559bf84528b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005e30 tx=0x7f7a300098f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158bb373b7369349 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a16ffd640 1 -- 172.21.15.105:0/2643648960 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30044070 con 0x559bf8442590 2024-03-26T17:57:11.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a407ce740 1 -- 172.21.15.105:0/2643648960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a2809bf70 con 0x559bf8442590 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.401+0000 7f7a16ffd640 1 -- 172.21.15.105:0/2643648960 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f7a30003970 con 0x559bf8442590 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a16ffd640 1 -- 172.21.15.105:0/2643648960 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7a30050050 con 0x559bf8442590 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a407ce740 10 client.31906 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a407ce740 1 -- 172.21.15.105:0/2643648960 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559bf8455ce0 con 0x559bf8442590 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a16ffd640 1 -- 172.21.15.105:0/2643648960 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a2809bf70 con 0x559bf8442590 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a16ffd640 1 -- 172.21.15.105:0/2643648960 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7a30040460 con 0x559bf8442590 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a407ce740 10 client.31906 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a407ce740 10 client.31906 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a407ce740 20 client.31906 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559bf8457370 0x559bf84597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a407ce740 4 client.31906 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a407ce740 1 -- 172.21.15.105:0/2643648960 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bf83aeeb0 con 0x559bf8457370 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.402+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559bf8457370 0x559bf84597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.403+0000 7f7a377fe640 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559bf8457370 0x559bf84597d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=562a12806fa54bd server_cookie=f873a27ffd9849fc in_seq=0 out_seq=0 2024-03-26T17:57:11.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.403+0000 7f7a16ffd640 10 client.31906 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.403+0000 7f7a16ffd640 1 -- 172.21.15.105:0/2643648960 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559bf83aeeb0 con 0x559bf8457370 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.403+0000 7f7a16ffd640 10 client.31906 handle_command_reply: tid=0 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.403+0000 7f7a407ce740 1 client.31906 shutdown 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 -- 172.21.15.105:0/2643648960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf8442590 msgr2=0x559bf84528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf8442590 0x559bf84528b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005e30 tx=0x7f7a300098f0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 -- 172.21.15.105:0/2643648960 shutdown_connections 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559bf8457370 0x559bf84597d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf8442590 0x559bf84528b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843d420 0x559bf844b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a16ffd640 0 client.31906 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 --2- 172.21.15.105:0/2643648960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 -- 172.21.15.105:0/2643648960 >> 172.21.15.105:0/2643648960 conn(0x559bf8437fa0 msgr2=0x559bf84371c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 -- 172.21.15.105:0/2643648960 shutdown_connections 2024-03-26T17:57:11.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 1 -- 172.21.15.105:0/2643648960 wait complete. 2024-03-26T17:57:11.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.404+0000 7f7a407ce740 20 client.31906 trim_cache size 0 max 16384 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a407ce740 1 Processor -- start 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a407ce740 1 -- start start 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf8452870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8436930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a407ce740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a6bf0 con 0x559bf8442590 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a407ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a8730 con 0x559bf843ca30 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a407ce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a72c0 con 0x559bf843d420 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a36ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf8452870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8436930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60276/0 (socket says 172.21.15.105:60276) 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a377fe640 1 -- 172.21.15.105:0/1841004823 learned_addr learned my addr 172.21.15.105:0/1841004823 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a36ffd640 1 -- 172.21.15.105:0/1841004823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 msgr2=0x559bf844adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/1841004823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a36ffd640 1 -- 172.21.15.105:0/1841004823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 msgr2=0x559bf8436930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/1841004823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8436930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a36ffd640 1 -- 172.21.15.105:0/1841004823 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bf843dcb0 con 0x559bf843d420 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.420+0000 7f7a37fff640 1 --2- 172.21.15.105:0/1841004823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8436930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.421+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/1841004823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf8452870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x559bf843d820 tx=0x7f7a2c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80dee73750c51cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.421+0000 7f7a1d7fa640 1 -- 172.21.15.105:0/1841004823 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c005020 con 0x559bf843d420 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.421+0000 7f7a1d7fa640 1 -- 172.21.15.105:0/1841004823 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a2c0047a0 con 0x559bf843d420 2024-03-26T17:57:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.421+0000 7f7a1d7fa640 1 -- 172.21.15.105:0/1841004823 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c010840 con 0x559bf843d420 2024-03-26T17:57:11.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:10 smithi135 ceph-mon[35636]: pgmap v915: 97 pgs: 97 active+clean; 315 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 2.6 MiB/s wr, 173 op/s 2024-03-26T17:57:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 -- 172.21.15.105:0/1841004823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 msgr2=0x559bf8452870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 --2- 172.21.15.105:0/1841004823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf8452870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x559bf843d820 tx=0x7f7a2c009f90 comp rx=0 tx=0).stop 2024-03-26T17:57:11.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 -- 172.21.15.105:0/1841004823 shutdown_connections 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 --2- 172.21.15.105:0/1841004823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf8442590 0x559bf8436930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 --2- 172.21.15.105:0/1841004823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf8452870 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 --2- 172.21.15.105:0/1841004823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf843ca30 0x559bf844adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 -- 172.21.15.105:0/1841004823 >> 172.21.15.105:0/1841004823 conn(0x559bf8437fa0 msgr2=0x559bf844b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 -- 172.21.15.105:0/1841004823 shutdown_connections 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 -- 172.21.15.105:0/1841004823 wait complete. 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 -- start start 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 Processor -- start 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843ca30 0x559bf844f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf844fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.422+0000 7f7a407ce740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf8442590 0x559bf84bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a407ce740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a8060 con 0x559bf843ca30 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a407ce740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a7990 con 0x559bf8442590 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a407ce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381a4bd0 con 0x559bf843d420 2024-03-26T17:57:11.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a407ce740 10 client.0 _ll_register_callbacks cb 0x7f7a2c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a36ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf844fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843ca30 0x559bf844f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf8442590 0x559bf84bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a36ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf844fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56558/0 (socket says 172.21.15.105:56558) 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a36ffd640 1 -- 172.21.15.105:0/3321416535 learned_addr learned my addr 172.21.15.105:0/3321416535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a377fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843ca30 0x559bf844f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44834/0 (socket says 172.21.15.105:44834) 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a377fe640 1 -- 172.21.15.105:0/3321416535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 msgr2=0x559bf844fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a377fe640 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf844fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a377fe640 1 -- 172.21.15.105:0/3321416535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf8442590 msgr2=0x559bf84bb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a377fe640 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf8442590 0x559bf84bb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a377fe640 1 -- 172.21.15.105:0/3321416535 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf84bbcb0 con 0x559bf843ca30 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf844fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a377fe640 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843ca30 0x559bf844f890 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f7a280cd390 tx=0x7f7a2809ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c809c990d50f4b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a16ffd640 1 -- 172.21.15.105:0/3321416535 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a280b4040 con 0x559bf843ca30 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a16ffd640 1 -- 172.21.15.105:0/3321416535 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f7a2809a270 con 0x559bf843ca30 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.423+0000 7f7a407ce740 1 -- 172.21.15.105:0/3321416535 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a2c0047a0 con 0x559bf843ca30 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a16ffd640 1 -- 172.21.15.105:0/3321416535 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7a2809bf70 con 0x559bf843ca30 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a407ce740 10 client.22227 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a407ce740 1 -- 172.21.15.105:0/3321416535 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559bf84bcfc0 con 0x559bf843ca30 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a16ffd640 1 -- 172.21.15.105:0/3321416535 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a2c0047a0 con 0x559bf843ca30 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a16ffd640 1 -- 172.21.15.105:0/3321416535 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f7a280b7070 con 0x559bf843ca30 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a407ce740 10 client.22227 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:11.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a407ce740 10 client.22227 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a407ce740 20 client.22227 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a407ce740 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559bf84bd520 0x559bf84bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a407ce740 4 client.22227 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a407ce740 1 -- 172.21.15.105:0/3321416535 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bf83a9200 con 0x559bf84bd520 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.424+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559bf84bd520 0x559bf84bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a36ffd640 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559bf84bd520 0x559bf84bf910 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=306c8d338ba2e05b server_cookie=c46033db7e5ba9df in_seq=0 out_seq=0 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a16ffd640 10 client.22227 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a16ffd640 1 -- 172.21.15.105:0/3321416535 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bf83a9200 con 0x559bf84bd520 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a16ffd640 10 client.22227 handle_command_reply: tid=0 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 client.22227 shutdown 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 -- 172.21.15.105:0/3321416535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843ca30 msgr2=0x559bf844f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843ca30 0x559bf844f890 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f7a280cd390 tx=0x7f7a2809ba00 comp rx=0 tx=0).stop 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 -- 172.21.15.105:0/3321416535 shutdown_connections 2024-03-26T17:57:11.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x559bf84bd520 0x559bf84bf910 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bf8442590 0x559bf84bb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559bf843d420 0x559bf844fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a16ffd640 0 client.22227 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 --2- 172.21.15.105:0/3321416535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559bf843ca30 0x559bf844f890 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.425+0000 7f7a407ce740 1 -- 172.21.15.105:0/3321416535 >> 172.21.15.105:0/3321416535 conn(0x559bf8437fa0 msgr2=0x559bf844b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.426+0000 7f7a407ce740 1 -- 172.21.15.105:0/3321416535 shutdown_connections 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.426+0000 7f7a407ce740 1 -- 172.21.15.105:0/3321416535 wait complete. 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.426+0000 7f7a407ce740 20 client.22227 trim_cache size 0 max 16384 2024-03-26T17:57:11.417 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.426+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a200a2f20 msgr2=0x7f7a200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.426+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a200a2f20 0x7f7a200a53e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0058c0 tx=0x7f7a2c005850 comp rx=0 tx=0).stop 2024-03-26T17:57:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.426+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a381007c0 msgr2=0x7f7a381969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.426+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a381007c0 0x7f7a381969c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800e9c0 tx=0x7f7a2800ee80 comp rx=0 tx=0).stop 2024-03-26T17:57:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.426+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2939249252 shutdown_connections 2024-03-26T17:57:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.427+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7a200a2f20 0x7f7a200a53e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.427+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a38102370 0x7f7a3819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.427+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7a381019b0 0x7f7a38196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.427+0000 7f7a3dbe6640 1 --2- 172.21.15.105:0/2939249252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a381007c0 0x7f7a381969c0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.427+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2939249252 >> 172.21.15.105:0/2939249252 conn(0x7f7a380fbd50 msgr2=0x7f7a38104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.427+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2939249252 shutdown_connections 2024-03-26T17:57:11.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.427+0000 7f7a3dbe6640 1 -- 172.21.15.105:0/2939249252 wait complete. 2024-03-26T17:57:11.425 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:57:11.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 46544 7.09 MB/sec execute 234 sec latency 139.830 ms 2024-03-26T17:57:11.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.579+0000 7f17d0a34640 1 -- 172.21.15.105:0/2442954847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1007c0 msgr2=0x7f17cc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.579+0000 7f17d0a34640 1 --2- 172.21.15.105:0/2442954847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1007c0 0x7f17cc100be0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009900 tx=0x7f17bc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.579+0000 7f17d0a34640 1 -- 172.21.15.105:0/2442954847 shutdown_connections 2024-03-26T17:57:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.579+0000 7f17d0a34640 1 --2- 172.21.15.105:0/2442954847 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17cc102370 0x7f17cc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.579+0000 7f17d0a34640 1 --2- 172.21.15.105:0/2442954847 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17cc1019b0 0x7f17cc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.579+0000 7f17d0a34640 1 --2- 172.21.15.105:0/2442954847 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1007c0 0x7f17cc100be0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.579+0000 7f17d0a34640 1 -- 172.21.15.105:0/2442954847 >> 172.21.15.105:0/2442954847 conn(0x7f17cc0fbd50 msgr2=0x7f17cc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.579+0000 7f17d0a34640 1 -- 172.21.15.105:0/2442954847 shutdown_connections 2024-03-26T17:57:11.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.580+0000 7f17d0a34640 1 -- 172.21.15.105:0/2442954847 wait complete. 2024-03-26T17:57:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.580+0000 7f17d0a34640 1 Processor -- start 2024-03-26T17:57:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.580+0000 7f17d0a34640 1 -- start start 2024-03-26T17:57:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17d0a34640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17cc1007c0 0x7f17cc199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17d0a34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1019b0 0x7f17cc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17d0a34640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17cc102370 0x7f17cc1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17d0a34640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10d3d0 con 0x7f17cc1019b0 2024-03-26T17:57:11.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17d0a34640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10d250 con 0x7f17cc1007c0 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17d0a34640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc10d550 con 0x7f17cc102370 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17c9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1019b0 0x7f17cc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17ca59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17cc1007c0 0x7f17cc199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17c9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1019b0 0x7f17cc19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44842/0 (socket says 172.21.15.105:44842) 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17c9d9b640 1 -- 172.21.15.105:0/1073993546 learned_addr learned my addr 172.21.15.105:0/1073993546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17cad9d640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17cc102370 0x7f17cc1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17c9d9b640 1 -- 172.21.15.105:0/1073993546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17cc102370 msgr2=0x7f17cc1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17cc102370 0x7f17cc1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17c9d9b640 1 -- 172.21.15.105:0/1073993546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17cc1007c0 msgr2=0x7f17cc199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17cc1007c0 0x7f17cc199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.581+0000 7f17c9d9b640 1 -- 172.21.15.105:0/1073993546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc1a19b0 con 0x7f17cc1019b0 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.582+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1019b0 0x7f17cc19a330 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f17c000e9f0 tx=0x7f17c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.582+0000 7f17bb7fe640 1 -- 172.21.15.105:0/1073993546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c000cd80 con 0x7f17cc1019b0 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.582+0000 7f17bb7fe640 1 -- 172.21.15.105:0/1073993546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c00108e0 con 0x7f17cc1019b0 2024-03-26T17:57:11.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.582+0000 7f17d0a34640 1 -- 172.21.15.105:0/1073993546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1a1ca0 con 0x7f17cc1019b0 2024-03-26T17:57:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.582+0000 7f17bb7fe640 1 -- 172.21.15.105:0/1073993546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0018920 con 0x7f17cc1019b0 2024-03-26T17:57:11.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.582+0000 7f17d0a34640 1 -- 172.21.15.105:0/1073993546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1a21c0 con 0x7f17cc1019b0 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.585+0000 7f17bb7fe640 1 -- 172.21.15.105:0/1073993546 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f17c0018ac0 con 0x7f17cc1019b0 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.585+0000 7f17d361a740 1 Processor -- start 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.585+0000 7f17d361a740 1 -- start start 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17d361a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041cdbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17d361a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdc420 0x557041ce0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17d361a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041ce1590 0x557041ce3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17d361a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557041c40970 con 0x557041cdba30 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17d361a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557041be8d70 con 0x557041cdc420 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17d361a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557041c3c920 con 0x557041ce1590 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041cdbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17c9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041ce1590 0x557041ce3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041cdbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44854/0 (socket says 172.21.15.105:44854) 2024-03-26T17:57:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 -- 172.21.15.105:0/682518238 learned_addr learned my addr 172.21.15.105:0/682518238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17ca59c640 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdc420 0x557041ce0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 -- 172.21.15.105:0/682518238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041ce1590 msgr2=0x557041ce3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041ce1590 0x557041ce3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 -- 172.21.15.105:0/682518238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdc420 msgr2=0x557041ce0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdc420 0x557041ce0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 -- 172.21.15.105:0/682518238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570419543f0 con 0x557041cdba30 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041ce1590 0x557041ce3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17bb7fe640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17b40a3070 0x7f17b40a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.586+0000 7f17cad9d640 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041cdbe50 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009fd0 tx=0x7f17c4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f22a8e4fc9ac67 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.587+0000 7f17bb7fe640 1 -- 172.21.15.105:0/1073993546 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f17c00cf780 con 0x7f17cc1019b0 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.587+0000 7f17ca59c640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17b40a3070 0x7f17b40a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.587+0000 7f17b8ff9640 1 -- 172.21.15.105:0/682518238 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4011700 con 0x557041cdba30 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.587+0000 7f17b8ff9640 1 -- 172.21.15.105:0/682518238 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c4011d50 con 0x557041cdba30 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.587+0000 7f17b8ff9640 1 -- 172.21.15.105:0/682518238 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c401a380 con 0x557041cdba30 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17ca59c640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17b40a3070 0x7f17b40a5530 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f17bc005ab0 tx=0x7f17bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 -- 172.21.15.105:0/682518238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 msgr2=0x557041cdbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041cdbe50 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009fd0 tx=0x7f17c4010860 comp rx=0 tx=0).stop 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 -- 172.21.15.105:0/682518238 shutdown_connections 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041ce1590 0x557041ce3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdc420 0x557041ce0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 --2- 172.21.15.105:0/682518238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041cdbe50 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 -- 172.21.15.105:0/682518238 >> 172.21.15.105:0/682518238 conn(0x557041cd6fa0 msgr2=0x557041cd73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 -- 172.21.15.105:0/682518238 shutdown_connections 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.588+0000 7f17d361a740 1 -- 172.21.15.105:0/682518238 wait complete. 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 1 -- start start 2024-03-26T17:57:11.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 1 Processor -- start 2024-03-26T17:57:11.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041ce5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041ce5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ce6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041ce5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44866/0 (socket says 172.21.15.105:44866) 2024-03-26T17:57:11.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17ca59c640 1 -- 172.21.15.105:0/1753029197 learned_addr learned my addr 172.21.15.105:0/1753029197 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041ce1590 0x557041ced4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557041ce4540 con 0x557041cdba30 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557041bfc310 con 0x557041ce1590 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557041ce46c0 con 0x557041cdc420 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ce6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17cad9d640 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041ce1590 0x557041ced4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.589+0000 7f17d361a740 10 client.0 _ll_register_callbacks cb 0x557041c37c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17ca59c640 1 -- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 msgr2=0x557041ce6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17ca59c640 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ce6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17ca59c640 1 -- 172.21.15.105:0/1753029197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041ce1590 msgr2=0x557041ced4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17ca59c640 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041ce1590 0x557041ced4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17ca59c640 1 -- 172.21.15.105:0/1753029197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557041a1e7d0 con 0x557041cdba30 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ce6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17ca59c640 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041ce5f50 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0096c0 tx=0x7f17bc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3dc533d1c81dcb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17b1ffb640 1 -- 172.21.15.105:0/1753029197 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc04f030 con 0x557041cdba30 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17b1ffb640 1 -- 172.21.15.105:0/1753029197 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f17bc03a040 con 0x557041cdba30 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c4011d50 con 0x557041cdba30 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17b1ffb640 1 -- 172.21.15.105:0/1753029197 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17bc005250 con 0x557041cdba30 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17d361a740 10 client.22239 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.590+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557041cf08e0 con 0x557041cdba30 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17b1ffb640 1 -- 172.21.15.105:0/1753029197 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17bc0053f0 con 0x557041cdba30 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17b1ffb640 1 -- 172.21.15.105:0/1753029197 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f17bc0037b0 con 0x557041cdba30 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17d361a740 10 client.22239 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17d361a740 10 client.22239 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17d361a740 20 client.22239 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17d361a740 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557041cf2000 0x557041cf4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17d361a740 4 client.22239 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f17c4011d50 con 0x557041cf2000 2024-03-26T17:57:11.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.591+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557041cf2000 0x557041cf4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.592+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557041cf2000 0x557041cf4470 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5aecd7007a7ccb82 server_cookie=9b42a887fd87090 in_seq=0 out_seq=0 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.592+0000 7f17b1ffb640 10 client.22239 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.592+0000 7f17b1ffb640 1 -- 172.21.15.105:0/1753029197 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f17c4011d50 con 0x557041cf2000 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.592+0000 7f17b1ffb640 10 client.22239 handle_command_reply: tid=0 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.592+0000 7f17d361a740 1 client.22239 shutdown 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 msgr2=0x557041ce5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041ce5f50 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0096c0 tx=0x7f17bc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 shutdown_connections 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557041cf2000 0x557041cf4470 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17b1ffb640 0 client.22239 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041ce1590 0x557041ced4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ce6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 --2- 172.21.15.105:0/1753029197 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041cdba30 0x557041ce5f50 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 >> 172.21.15.105:0/1753029197 conn(0x557041cd6fa0 msgr2=0x557041cd61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 shutdown_connections 2024-03-26T17:57:11.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 1 -- 172.21.15.105:0/1753029197 wait complete. 2024-03-26T17:57:11.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.593+0000 7f17d361a740 20 client.22239 trim_cache size 0 max 16384 2024-03-26T17:57:11.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.611+0000 7f17d361a740 1 Processor -- start 2024-03-26T17:57:11.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.611+0000 7f17d361a740 1 -- start start 2024-03-26T17:57:11.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.611+0000 7f17d361a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cdd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.611+0000 7f17d361a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ced490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.611+0000 7f17d361a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041ce63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.611+0000 7f17d361a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1aafb0 con 0x557041ce1590 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.611+0000 7f17d361a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1acaf0 con 0x557041cdba30 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.611+0000 7f17d361a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1ab680 con 0x557041cdc420 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17c9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ced490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17cad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041ce63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17ca59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cdd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17c9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ced490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56610/0 (socket says 172.21.15.105:56610) 2024-03-26T17:57:11.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17c9d9b640 1 -- 172.21.15.105:0/3517601557 learned_addr learned my addr 172.21.15.105:0/3517601557 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17cad9d640 1 -- 172.21.15.105:0/3517601557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 msgr2=0x557041ced490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17cad9d640 1 --2- 172.21.15.105:0/3517601557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ced490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17cad9d640 1 -- 172.21.15.105:0/3517601557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 msgr2=0x557041cdd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17cad9d640 1 --2- 172.21.15.105:0/3517601557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cdd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17cad9d640 1 -- 172.21.15.105:0/3517601557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557041cd7d10 con 0x557041ce1590 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17ca59c640 1 --2- 172.21.15.105:0/3517601557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cdd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17cad9d640 1 --2- 172.21.15.105:0/3517601557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041ce63f0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f17c40099a0 tx=0x7f17c4009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7b9c0a2896406b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.612+0000 7f17b3fff640 1 -- 172.21.15.105:0/3517601557 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4004c60 con 0x557041ce1590 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.613+0000 7f17b3fff640 1 -- 172.21.15.105:0/3517601557 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c4003e20 con 0x557041ce1590 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.613+0000 7f17b3fff640 1 -- 172.21.15.105:0/3517601557 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4004810 con 0x557041ce1590 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- 172.21.15.105:0/3517601557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 msgr2=0x557041ce63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 --2- 172.21.15.105:0/3517601557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041ce63f0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f17c40099a0 tx=0x7f17c4009c50 comp rx=0 tx=0).stop 2024-03-26T17:57:11.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- 172.21.15.105:0/3517601557 shutdown_connections 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 --2- 172.21.15.105:0/3517601557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041ce63f0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 --2- 172.21.15.105:0/3517601557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ced490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 --2- 172.21.15.105:0/3517601557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cdd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- 172.21.15.105:0/3517601557 >> 172.21.15.105:0/3517601557 conn(0x557041cd6fa0 msgr2=0x557041cda200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- 172.21.15.105:0/3517601557 shutdown_connections 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- 172.21.15.105:0/3517601557 wait complete. 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- start start 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 Processor -- start 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ceaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041cebea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1ac420 con 0x557041ce1590 2024-03-26T17:57:11.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1abd50 con 0x557041cdba30 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.614+0000 7f17d361a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc1a8f90 con 0x557041cdc420 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17d361a740 10 client.0 _ll_register_callbacks cb 0x7f17bc043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60340/0 (socket says 172.21.15.105:60340) 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17c9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ceaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 -- 172.21.15.105:0/3262193005 learned_addr learned my addr 172.21.15.105:0/3262193005 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17cad9d640 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041cebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 -- 172.21.15.105:0/3262193005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 msgr2=0x557041ceaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ceaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 -- 172.21.15.105:0/3262193005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 msgr2=0x557041cebea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041cebea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 -- 172.21.15.105:0/3262193005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557041cec490 con 0x557041cdba30 2024-03-26T17:57:11.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17cad9d640 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041cebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17ca59c640 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cea9a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f17bc003c20 tx=0x7f17bc003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b781f13089fe52f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.615+0000 7f17b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:11.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.616+0000 7f17b1ffb640 1 -- 172.21.15.105:0/3262193005 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc04f070 con 0x557041cdba30 2024-03-26T17:57:11.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.616+0000 7f17d361a740 1 -- 172.21.15.105:0/3262193005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c4003e20 con 0x557041cdba30 2024-03-26T17:57:11.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.616+0000 7f17b1ffb640 1 -- 172.21.15.105:0/3262193005 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f17bc003e40 con 0x557041cdba30 2024-03-26T17:57:11.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.616+0000 7f17b1ffb640 1 -- 172.21.15.105:0/3262193005 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f17bc06c050 con 0x557041cdba30 2024-03-26T17:57:11.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.616+0000 7f17d361a740 10 client.31937 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:11.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.616+0000 7f17d361a740 1 -- 172.21.15.105:0/3262193005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557041ced800 con 0x557041cdba30 2024-03-26T17:57:11.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.616+0000 7f17b1ffb640 1 -- 172.21.15.105:0/3262193005 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17c4003e20 con 0x557041cdba30 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17b1ffb640 1 -- 172.21.15.105:0/3262193005 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f17bc04b7d0 con 0x557041cdba30 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17d361a740 10 client.31937 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17d361a740 10 client.31937 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17d361a740 20 client.31937 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17d361a740 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557041cf2000 0x557041d5ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17d361a740 4 client.31937 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17d361a740 1 -- 172.21.15.105:0/3262193005 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557041c4deb0 con 0x557041cf2000 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557041cf2000 0x557041d5ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17b1ffb640 10 client.31937 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.617+0000 7f17c9d9b640 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557041cf2000 0x557041d5ab60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=33dedce409753e69 server_cookie=559026b47f7aec0a in_seq=0 out_seq=0 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17b1ffb640 1 -- 172.21.15.105:0/3262193005 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557041c4deb0 con 0x557041cf2000 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17b1ffb640 10 client.31937 handle_command_reply: tid=0 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 client.31937 shutdown 2024-03-26T17:57:11.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 -- 172.21.15.105:0/3262193005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 msgr2=0x557041cea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cea9a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f17bc003c20 tx=0x7f17bc003820 comp rx=0 tx=0).stop 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 -- 172.21.15.105:0/3262193005 shutdown_connections 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557041cf2000 0x557041d5ab60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557041ce1590 0x557041cebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557041cdc420 0x557041ceaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17b1ffb640 0 client.31937 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 --2- 172.21.15.105:0/3262193005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557041cdba30 0x557041cea9a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 -- 172.21.15.105:0/3262193005 >> 172.21.15.105:0/3262193005 conn(0x557041cd6fa0 msgr2=0x557041ce6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 -- 172.21.15.105:0/3262193005 shutdown_connections 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 1 -- 172.21.15.105:0/3262193005 wait complete. 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.618+0000 7f17d361a740 20 client.31937 trim_cache size 0 max 16384 2024-03-26T17:57:11.607 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:11.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 -- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17b40a3070 msgr2=0x7f17b40a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17b40a3070 0x7f17b40a5530 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f17bc005ab0 tx=0x7f17bc005fd0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 -- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1019b0 msgr2=0x7f17cc19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1019b0 0x7f17cc19a330 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f17c000e9f0 tx=0x7f17c000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 -- 172.21.15.105:0/1073993546 shutdown_connections 2024-03-26T17:57:11.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f17b40a3070 0x7f17b40a5530 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17cc102370 0x7f17cc1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17cc1019b0 0x7f17cc19a330 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 --2- 172.21.15.105:0/1073993546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17cc1007c0 0x7f17cc199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 -- 172.21.15.105:0/1073993546 >> 172.21.15.105:0/1073993546 conn(0x7f17cc0fbd50 msgr2=0x7f17cc107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 -- 172.21.15.105:0/1073993546 shutdown_connections 2024-03-26T17:57:11.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.619+0000 7f17d0a34640 1 -- 172.21.15.105:0/1073993546 wait complete. 2024-03-26T17:57:11.620 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:57:11.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 -- 172.21.15.105:0/158029425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c104cd0 msgr2=0x7f001c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 --2- 172.21.15.105:0/158029425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c104cd0 0x7f001c1070c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0004009900 tx=0x7f000402bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:11.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 -- 172.21.15.105:0/158029425 shutdown_connections 2024-03-26T17:57:11.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 --2- 172.21.15.105:0/158029425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f001c107600 0x7f001c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 --2- 172.21.15.105:0/158029425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c104cd0 0x7f001c1070c0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 --2- 172.21.15.105:0/158029425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f001c1023a0 0x7f001c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 -- 172.21.15.105:0/158029425 >> 172.21.15.105:0/158029425 conn(0x7f001c0fbd50 msgr2=0x7f001c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 -- 172.21.15.105:0/158029425 shutdown_connections 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.788+0000 7f0020953640 1 -- 172.21.15.105:0/158029425 wait complete. 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0020953640 1 Processor -- start 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0020953640 1 -- start start 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0020953640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c1023a0 0x7f001c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0020953640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f001c104cd0 0x7f001c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0020953640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f001c107600 0x7f001c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0020953640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f001c10d440 con 0x7f001c1023a0 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0020953640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f001c10d2c0 con 0x7f001c104cd0 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0020953640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f001c10d5c0 con 0x7f001c107600 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0019d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f001c104cd0 0x7f001c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0019d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f001c104cd0 0x7f001c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60346/0 (socket says 172.21.15.105:60346) 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f001ad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f001c107600 0x7f001c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f0019d9b640 1 -- 172.21.15.105:0/3898529428 learned_addr learned my addr 172.21.15.105:0/3898529428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.789+0000 7f001a59c640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c1023a0 0x7f001c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f001ad9d640 1 -- 172.21.15.105:0/3898529428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f001c104cd0 msgr2=0x7f001c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f001ad9d640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f001c104cd0 0x7f001c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f001ad9d640 1 -- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c1023a0 msgr2=0x7f001c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f001ad9d640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c1023a0 0x7f001c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f001ad9d640 1 -- 172.21.15.105:0/3898529428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c19f7b0 con 0x7f001c107600 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f0019d9b640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f001c104cd0 0x7f001c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f001a59c640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c1023a0 0x7f001c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f001ad9d640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f001c107600 0x7f001c19f140 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0014009b50 tx=0x7f001400b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f00137fe640 1 -- 172.21.15.105:0/3898529428 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014009e50 con 0x7f001c107600 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f0020953640 1 -- 172.21.15.105:0/3898529428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001c1001f0 con 0x7f001c107600 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f00137fe640 1 -- 172.21.15.105:0/3898529428 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f001400be40 con 0x7f001c107600 2024-03-26T17:57:11.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f00137fe640 1 -- 172.21.15.105:0/3898529428 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014019410 con 0x7f001c107600 2024-03-26T17:57:11.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.790+0000 7f0020953640 1 -- 172.21.15.105:0/3898529428 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001c100790 con 0x7f001c107600 2024-03-26T17:57:11.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.793+0000 7f00137fe640 1 -- 172.21.15.105:0/3898529428 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0014020020 con 0x7f001c107600 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.793+0000 7f00137fe640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00080a3110 0x7f00080a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.793+0000 7f002361c740 1 Processor -- start 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001a59c640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00080a3110 0x7f00080a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f002361c740 1 -- start start 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f00137fe640 1 -- 172.21.15.105:0/3898529428 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f00140d2c30 con 0x7f001c107600 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f002361c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd2a30 0x55b38efd2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f002361c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd3420 0x55b38efd7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f002361c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd8590 0x55b38efdaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f002361c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b38ef37970 con 0x55b38efd2a30 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f002361c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b38eedfd70 con 0x55b38efd3420 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f002361c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b38ef33920 con 0x55b38efd8590 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd2a30 0x55b38efd2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f0019d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd3420 0x55b38efd7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001ad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd8590 0x55b38efdaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001ad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd8590 0x55b38efdaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56640/0 (socket says 172.21.15.105:56640) 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f0019d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd3420 0x55b38efd7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60352/0 (socket says 172.21.15.105:60352) 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001ad9d640 1 -- 172.21.15.105:0/1614553116 learned_addr learned my addr 172.21.15.105:0/1614553116 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001ad9d640 1 -- 172.21.15.105:0/1614553116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd3420 msgr2=0x55b38efd7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001ad9d640 1 --2- 172.21.15.105:0/1614553116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd3420 0x55b38efd7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001ad9d640 1 -- 172.21.15.105:0/1614553116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd2a30 msgr2=0x55b38efd2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001ad9d640 1 --2- 172.21.15.105:0/1614553116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd2a30 0x55b38efd2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.794+0000 7f001ad9d640 1 -- 172.21.15.105:0/1614553116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b38ec4b3f0 con 0x55b38efd8590 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.795+0000 7f001a59c640 1 --2- 172.21.15.105:0/1614553116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd2a30 0x55b38efd2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.795+0000 7f001a59c640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00080a3110 0x7f00080a55d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f000c005fd0 tx=0x7f000c011da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.795+0000 7f001ad9d640 1 --2- 172.21.15.105:0/1614553116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd8590 0x55b38efdaa20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f001409bcf0 tx=0x7f001409f220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e9fbd767822c598 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.795+0000 7f0010ff9640 1 -- 172.21.15.105:0/1614553116 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140a7d30 con 0x55b38efd8590 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.795+0000 7f0010ff9640 1 -- 172.21.15.105:0/1614553116 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f001409d620 con 0x55b38efd8590 2024-03-26T17:57:11.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.795+0000 7f0010ff9640 1 -- 172.21.15.105:0/1614553116 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001409bb30 con 0x55b38efd8590 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 -- 172.21.15.105:0/1614553116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd8590 msgr2=0x55b38efdaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 --2- 172.21.15.105:0/1614553116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd8590 0x55b38efdaa20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f001409bcf0 tx=0x7f001409f220 comp rx=0 tx=0).stop 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 -- 172.21.15.105:0/1614553116 shutdown_connections 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 --2- 172.21.15.105:0/1614553116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd8590 0x55b38efdaa20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 --2- 172.21.15.105:0/1614553116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd3420 0x55b38efd7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 --2- 172.21.15.105:0/1614553116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd2a30 0x55b38efd2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 -- 172.21.15.105:0/1614553116 >> 172.21.15.105:0/1614553116 conn(0x55b38efcdfa0 msgr2=0x55b38efce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 -- 172.21.15.105:0/1614553116 shutdown_connections 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.796+0000 7f002361c740 1 -- 172.21.15.105:0/1614553116 wait complete. 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 1 -- start start 2024-03-26T17:57:11.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 1 Processor -- start 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efe5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f0019d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efe5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f001a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f001a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56646/0 (socket says 172.21.15.105:56646) 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f0019d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efe5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44928/0 (socket says 172.21.15.105:44928) 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f001a59c640 1 -- 172.21.15.105:0/590603166 learned_addr learned my addr 172.21.15.105:0/590603166 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 0x55b38efecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b38efe3d50 con 0x55b38efd3420 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b38eef3310 con 0x55b38efd8590 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b38efe3ed0 con 0x55b38efd2a30 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f002361c740 10 client.0 _ll_register_callbacks cb 0x55b38ef2ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.797+0000 7f001ad9d640 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 0x55b38efecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f001a59c640 1 -- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 msgr2=0x55b38efecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f001a59c640 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 0x55b38efecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f001a59c640 1 -- 172.21.15.105:0/590603166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 msgr2=0x55b38efe5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f001a59c640 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efe5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f001a59c640 1 -- 172.21.15.105:0/590603166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b38ed157d0 con 0x55b38efd2a30 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f001a59c640 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe5780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f000c01e760 tx=0x7f000c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d91a78574d619d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f0001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f0001ffb640 1 -- 172.21.15.105:0/590603166 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c02b060 con 0x55b38efd2a30 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f0001ffb640 1 -- 172.21.15.105:0/590603166 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f000c01d9d0 con 0x55b38efd2a30 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f001409d620 con 0x55b38efd2a30 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f0001ffb640 1 -- 172.21.15.105:0/590603166 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f000c01dbd0 con 0x55b38efd2a30 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f002361c740 10 client.31942 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.798+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b38eff0130 con 0x55b38efd2a30 2024-03-26T17:57:11.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f0001ffb640 1 -- 172.21.15.105:0/590603166 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f001409d620 con 0x55b38efd2a30 2024-03-26T17:57:11.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f0001ffb640 1 -- 172.21.15.105:0/590603166 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f000c025030 con 0x55b38efd2a30 2024-03-26T17:57:11.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f002361c740 10 client.31942 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:11.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f002361c740 10 client.31942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:57:11.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f002361c740 20 client.31942 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:11.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f002361c740 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b38eff1800 0x55b38eff3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f002361c740 4 client.31942 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:11.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b38ef44eb0 con 0x55b38eff1800 2024-03-26T17:57:11.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.799+0000 7f0019d9b640 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b38eff1800 0x55b38eff3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.800+0000 7f0019d9b640 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b38eff1800 0x55b38eff3cc0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8415a9824402d89b server_cookie=1e054778e119f46c in_seq=0 out_seq=0 2024-03-26T17:57:11.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.800+0000 7f0001ffb640 10 client.31942 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:57:11.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.800+0000 7f0001ffb640 1 -- 172.21.15.105:0/590603166 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b38ef44eb0 con 0x55b38eff1800 2024-03-26T17:57:11.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.800+0000 7f0001ffb640 10 client.31942 handle_command_reply: tid=0 2024-03-26T17:57:11.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 client.31942 shutdown 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 msgr2=0x55b38efe5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe5780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f000c01e760 tx=0x7f000c004ab0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 shutdown_connections 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b38eff1800 0x55b38eff3cc0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f0001ffb640 0 client.31942 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 0x55b38efecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efe5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 --2- 172.21.15.105:0/590603166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe5780 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 >> 172.21.15.105:0/590603166 conn(0x55b38efcdfa0 msgr2=0x55b38efcd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 shutdown_connections 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 1 -- 172.21.15.105:0/590603166 wait complete. 2024-03-26T17:57:11.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.801+0000 7f002361c740 20 client.31942 trim_cache size 0 max 16384 2024-03-26T17:57:11.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f002361c740 1 Processor -- start 2024-03-26T17:57:11.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f002361c740 1 -- start start 2024-03-26T17:57:11.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f002361c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd2a30 0x55b38efe5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f002361c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd3420 0x55b38efeccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f002361c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd8590 0x55b38efe58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f002361c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1a6bf0 con 0x55b38efd8590 2024-03-26T17:57:11.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f002361c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1a8730 con 0x55b38efd2a30 2024-03-26T17:57:11.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f002361c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1a72c0 con 0x55b38efd3420 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd8590 0x55b38efe58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd8590 0x55b38efe58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44930/0 (socket says 172.21.15.105:44930) 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001ad9d640 1 -- 172.21.15.105:0/1673386638 learned_addr learned my addr 172.21.15.105:0/1673386638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f0019d9b640 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd3420 0x55b38efeccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001a59c640 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd2a30 0x55b38efe5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001ad9d640 1 -- 172.21.15.105:0/1673386638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd3420 msgr2=0x55b38efeccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001ad9d640 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd3420 0x55b38efeccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001ad9d640 1 -- 172.21.15.105:0/1673386638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd2a30 msgr2=0x55b38efe5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001ad9d640 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd2a30 0x55b38efe5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.818+0000 7f001ad9d640 1 -- 172.21.15.105:0/1673386638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b38efd7c20 con 0x55b38efd8590 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.819+0000 7f001ad9d640 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd8590 0x55b38efe58d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f001409f4e0 tx=0x7f001409d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208744739f21888d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.819+0000 7f0003fff640 1 -- 172.21.15.105:0/1673386638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140a3470 con 0x55b38efd8590 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.819+0000 7f0003fff640 1 -- 172.21.15.105:0/1673386638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00140a3170 con 0x55b38efd8590 2024-03-26T17:57:11.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.819+0000 7f0003fff640 1 -- 172.21.15.105:0/1673386638 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140d0bb0 con 0x55b38efd8590 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.819+0000 7f002361c740 1 -- 172.21.15.105:0/1673386638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd8590 msgr2=0x55b38efe58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.819+0000 7f002361c740 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd8590 0x55b38efe58d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f001409f4e0 tx=0x7f001409d830 comp rx=0 tx=0).stop 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 -- 172.21.15.105:0/1673386638 shutdown_connections 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd8590 0x55b38efe58d0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd3420 0x55b38efeccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 --2- 172.21.15.105:0/1673386638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd2a30 0x55b38efe5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 -- 172.21.15.105:0/1673386638 >> 172.21.15.105:0/1673386638 conn(0x55b38efcdfa0 msgr2=0x55b38efd4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 -- 172.21.15.105:0/1673386638 shutdown_connections 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 -- 172.21.15.105:0/1673386638 wait complete. 2024-03-26T17:57:11.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 -- start start 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 Processor -- start 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.820+0000 7f002361c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 0x55b38efeb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f002361c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1a8060 con 0x55b38efd3420 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f002361c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1a7990 con 0x55b38efd8590 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f002361c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1a4bd0 con 0x55b38efd2a30 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efea510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44946/0 (socket says 172.21.15.105:44946) 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 -- 172.21.15.105:0/3116417302 learned_addr learned my addr 172.21.15.105:0/3116417302 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f002361c740 10 client.0 _ll_register_callbacks cb 0x7f000c00fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f001ad9d640 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 0x55b38efeb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f001a59c640 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 -- 172.21.15.105:0/3116417302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 msgr2=0x55b38efe9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 -- 172.21.15.105:0/3116417302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 msgr2=0x55b38efeb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 0x55b38efeb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 -- 172.21.15.105:0/3116417302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b38efebac0 con 0x55b38efd3420 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0019d9b640 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efea510 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f000402c6a0 tx=0x7f0004004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b65229b4bf5157 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0001ffb640 1 -- 172.21.15.105:0/3116417302 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000404bd80 con 0x55b38efd3420 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f0001ffb640 1 -- 172.21.15.105:0/3116417302 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f000404b3b0 con 0x55b38efd3420 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.821+0000 7f002361c740 1 -- 172.21.15.105:0/3116417302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00140a3170 con 0x55b38efd3420 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f0001ffb640 1 -- 172.21.15.105:0/3116417302 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f000405a050 con 0x55b38efd3420 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f002361c740 10 client.22263 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f002361c740 1 -- 172.21.15.105:0/3116417302 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b38efece60 con 0x55b38efd3420 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f0001ffb640 1 -- 172.21.15.105:0/3116417302 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00140a3170 con 0x55b38efd3420 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f0001ffb640 1 -- 172.21.15.105:0/3116417302 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0004004700 con 0x55b38efd3420 2024-03-26T17:57:11.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f002361c740 10 client.22263 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f002361c740 10 client.22263 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f002361c740 20 client.22263 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f002361c740 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b38eff1800 0x55b38f05a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f002361c740 4 client.22263 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f002361c740 1 -- 172.21.15.105:0/3116417302 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b38ef3f200 con 0x55b38eff1800 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.822+0000 7f001a59c640 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b38eff1800 0x55b38f05a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.823+0000 7f001a59c640 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b38eff1800 0x55b38f05a250 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f059143d0f43851 server_cookie=7d08c1ab1fa48693 in_seq=0 out_seq=0 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.823+0000 7f0001ffb640 10 client.22263 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f0001ffb640 1 -- 172.21.15.105:0/3116417302 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b38ef3f200 con 0x55b38eff1800 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f0001ffb640 10 client.22263 handle_command_reply: tid=0 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 client.22263 shutdown 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 -- 172.21.15.105:0/3116417302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 msgr2=0x55b38efea510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efea510 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f000402c6a0 tx=0x7f0004004510 comp rx=0 tx=0).stop 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 -- 172.21.15.105:0/3116417302 shutdown_connections 2024-03-26T17:57:11.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b38eff1800 0x55b38f05a250 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b38efd8590 0x55b38efeb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b38efd3420 0x55b38efea510 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f0001ffb640 0 client.22263 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 --2- 172.21.15.105:0/3116417302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b38efd2a30 0x55b38efe9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 -- 172.21.15.105:0/3116417302 >> 172.21.15.105:0/3116417302 conn(0x55b38efcdfa0 msgr2=0x55b38efe5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 -- 172.21.15.105:0/3116417302 shutdown_connections 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 1 -- 172.21.15.105:0/3116417302 wait complete. 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.824+0000 7f002361c740 20 client.22263 trim_cache size 0 max 16384 2024-03-26T17:57:11.814 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:11.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 -- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00080a3110 msgr2=0x7f00080a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00080a3110 0x7f00080a55d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f000c005fd0 tx=0x7f000c011da0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 -- 172.21.15.105:0/3898529428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f001c107600 msgr2=0x7f001c19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f001c107600 0x7f001c19f140 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0014009b50 tx=0x7f001400b690 comp rx=0 tx=0).stop 2024-03-26T17:57:11.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 -- 172.21.15.105:0/3898529428 shutdown_connections 2024-03-26T17:57:11.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f00080a3110 0x7f00080a55d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f001c107600 0x7f001c19f140 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f001c104cd0 0x7f001c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 --2- 172.21.15.105:0/3898529428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f001c1023a0 0x7f001c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 -- 172.21.15.105:0/3898529428 >> 172.21.15.105:0/3898529428 conn(0x7f001c0fbd50 msgr2=0x7f001c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 -- 172.21.15.105:0/3898529428 shutdown_connections 2024-03-26T17:57:11.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.825+0000 7f0020953640 1 -- 172.21.15.105:0/3898529428 wait complete. 2024-03-26T17:57:11.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:57:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.979+0000 7fcbf49e1640 1 -- 172.21.15.105:0/714571275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0104cf0 msgr2=0x7fcbf01070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.979+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/714571275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0104cf0 0x7fcbf01070e0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009900 tx=0x7fcbd802bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.979+0000 7fcbf49e1640 1 -- 172.21.15.105:0/714571275 shutdown_connections 2024-03-26T17:57:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.979+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/714571275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf0107620 0x7fcbf0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.979+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/714571275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0104cf0 0x7fcbf01070e0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.979+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/714571275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf01023c0 0x7fcbf01047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.979+0000 7fcbf49e1640 1 -- 172.21.15.105:0/714571275 >> 172.21.15.105:0/714571275 conn(0x7fcbf00fbd70 msgr2=0x7fcbf00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.980+0000 7fcbf49e1640 1 -- 172.21.15.105:0/714571275 shutdown_connections 2024-03-26T17:57:11.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.980+0000 7fcbf49e1640 1 -- 172.21.15.105:0/714571275 wait complete. 2024-03-26T17:57:11.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.980+0000 7fcbf49e1640 1 Processor -- start 2024-03-26T17:57:11.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.980+0000 7fcbf49e1640 1 -- start start 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbf49e1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf01023c0 0x7fcbf0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbf49e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf0104cf0 0x7fcbf0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbf49e1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0107620 0x7fcbf019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbf49e1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf010d3e0 con 0x7fcbf0107620 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbf49e1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf010d260 con 0x7fcbf0104cf0 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbf49e1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf010d560 con 0x7fcbf01023c0 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbeed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0107620 0x7fcbf019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf01023c0 0x7fcbf0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf01023c0 0x7fcbf0197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56674/0 (socket says 172.21.15.105:56674) 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 -- 172.21.15.105:0/2075500100 learned_addr learned my addr 172.21.15.105:0/2075500100 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf0104cf0 0x7fcbf0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 -- 172.21.15.105:0/2075500100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf0104cf0 msgr2=0x7fcbf0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf0104cf0 0x7fcbf0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 -- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0107620 msgr2=0x7fcbf019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0107620 0x7fcbf019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 -- 172.21.15.105:0/2075500100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf019f710 con 0x7fcbf01023c0 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbeed9d640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0107620 0x7fcbf019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf0104cf0 0x7fcbf0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.981+0000 7fcbee59c640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf01023c0 0x7fcbf0197af0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c920 tx=0x7fcbe000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.982+0000 7fcbe77fe640 1 -- 172.21.15.105:0/2075500100 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0007c70 con 0x7fcbf01023c0 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.982+0000 7fcbe77fe640 1 -- 172.21.15.105:0/2075500100 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbe0007e10 con 0x7fcbf01023c0 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.982+0000 7fcbf49e1640 1 -- 172.21.15.105:0/2075500100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf019fa00 con 0x7fcbf01023c0 2024-03-26T17:57:11.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.982+0000 7fcbe77fe640 1 -- 172.21.15.105:0/2075500100 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0018820 con 0x7fcbf01023c0 2024-03-26T17:57:11.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.982+0000 7fcbf49e1640 1 -- 172.21.15.105:0/2075500100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf019ff20 con 0x7fcbf01023c0 2024-03-26T17:57:11.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbf75c5740 1 Processor -- start 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbf75c5740 1 -- start start 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbe77fe640 1 -- 172.21.15.105:0/2075500100 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcbe00189c0 con 0x7fcbf01023c0 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 0x55644fdc9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55644fd29970 con 0x55644fdc9830 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55644fcd1d70 con 0x55644fdce9a0 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55644fd25920 con 0x55644fdc8e40 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbe7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbeed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 0x55644fdc9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44962/0 (socket says 172.21.15.105:44962) 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbe7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60402/0 (socket says 172.21.15.105:60402) 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.985+0000 7fcbee59c640 1 -- 172.21.15.105:0/1958663362 learned_addr learned my addr 172.21.15.105:0/1958663362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbee59c640 1 -- 172.21.15.105:0/1958663362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 msgr2=0x55644fdc9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbee59c640 1 --2- 172.21.15.105:0/1958663362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 0x55644fdc9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbee59c640 1 -- 172.21.15.105:0/1958663362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 msgr2=0x55644fdd0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbee59c640 1 --2- 172.21.15.105:0/1958663362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbee59c640 1 -- 172.21.15.105:0/1958663362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55644fa3d3f0 con 0x55644fdc9830 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbe77fe640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbdc0a2f20 0x7fcbdc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbee59c640 1 --2- 172.21.15.105:0/1958663362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdce330 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00d5610 tx=0x7fcbe00d5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d80a763dd0c389 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbeed9d640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbdc0a2f20 0x7fcbdc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbe4ff9640 1 -- 172.21.15.105:0/1958663362 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0097e40 con 0x55644fdc9830 2024-03-26T17:57:11.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbe77fe640 1 -- 172.21.15.105:0/2075500100 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fcbe00cf510 con 0x7fcbf01023c0 2024-03-26T17:57:11.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbe4ff9640 1 -- 172.21.15.105:0/1958663362 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbe00bf070 con 0x55644fdc9830 2024-03-26T17:57:11.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.986+0000 7fcbe4ff9640 1 -- 172.21.15.105:0/1958663362 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00c5bb0 con 0x55644fdc9830 2024-03-26T17:57:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.987+0000 7fcbf75c5740 1 -- 172.21.15.105:0/1958663362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 msgr2=0x55644fdce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.987+0000 7fcbeed9d640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbdc0a2f20 0x7fcbdc0a53e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x55644fdca090 tx=0x7fcbe8009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.987+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/1958663362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdce330 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00d5610 tx=0x7fcbe00d5700 comp rx=0 tx=0).stop 2024-03-26T17:57:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.987+0000 7fcbf75c5740 1 -- 172.21.15.105:0/1958663362 shutdown_connections 2024-03-26T17:57:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.987+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/1958663362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.987+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/1958663362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdce330 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.987+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/1958663362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 0x55644fdc9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.987+0000 7fcbf75c5740 1 -- 172.21.15.105:0/1958663362 >> 172.21.15.105:0/1958663362 conn(0x55644fdbffa0 msgr2=0x55644fdc03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.988+0000 7fcbf75c5740 1 -- 172.21.15.105:0/1958663362 shutdown_connections 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.988+0000 7fcbf75c5740 1 -- 172.21.15.105:0/1958663362 wait complete. 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.988+0000 7fcbf75c5740 1 -- start start 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.988+0000 7fcbf75c5740 1 Processor -- start 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 0x55644fdd3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdd38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdda900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55644fdd1950 con 0x55644fdc9830 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55644fce5310 con 0x55644fdce9a0 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55644fdd1ad0 con 0x55644fdc8e40 2024-03-26T17:57:11.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbf75c5740 10 client.0 _ll_register_callbacks cb 0x55644fd20c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdd38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdd38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44966/0 (socket says 172.21.15.105:44966) 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 0x55644fdd3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 -- 172.21.15.105:0/3291516774 learned_addr learned my addr 172.21.15.105:0/3291516774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbeed9d640 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdda900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 -- 172.21.15.105:0/3291516774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 msgr2=0x55644fdd3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 0x55644fdd3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 -- 172.21.15.105:0/3291516774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 msgr2=0x55644fdda900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdda900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 -- 172.21.15.105:0/3291516774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55644fb077d0 con 0x55644fdc9830 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbeed9d640 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdda900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdd38c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009230 tx=0x7fcbd8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589665c3118d94d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.989+0000 7fcbd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:11.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.990+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3291516774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd802cc90 con 0x55644fdc9830 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.990+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3291516774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fcbd80046c0 con 0x55644fdc9830 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.990+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3291516774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe00bf070 con 0x55644fdc9830 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.990+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3291516774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcbd80055d0 con 0x55644fdc9830 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.990+0000 7fcbf75c5740 10 client.22275 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.990+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3291516774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55644fdddce0 con 0x55644fdc9830 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.990+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3291516774 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbe00bf070 con 0x55644fdc9830 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.990+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3291516774 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcbd8003c20 con 0x55644fdc9830 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbf75c5740 10 client.22275 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbf75c5740 10 client.22275 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbf75c5740 20 client.22275 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55644fddf300 0x55644fde1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbf75c5740 4 client.22275 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3291516774 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55644fd36eb0 con 0x55644fddf300 2024-03-26T17:57:11.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbee59c640 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55644fddf300 0x55644fde1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbd5ffb640 10 client.22275 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.991+0000 7fcbee59c640 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55644fddf300 0x55644fde1790 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=186b7c0f0f8de409 server_cookie=9a74ddb4733ddda7 in_seq=0 out_seq=0 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.992+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3291516774 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55644fd36eb0 con 0x55644fddf300 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.992+0000 7fcbd5ffb640 10 client.22275 handle_command_reply: tid=0 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.992+0000 7fcbf75c5740 1 client.22275 shutdown 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.992+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3291516774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 msgr2=0x55644fdd38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.992+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdd38c0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009230 tx=0x7fcbd8005250 comp rx=0 tx=0).stop 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3291516774 shutdown_connections 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55644fddf300 0x55644fde1790 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdda900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc9830 0x55644fdd38c0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3291516774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc8e40 0x55644fdd3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:11.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3291516774 >> 172.21.15.105:0/3291516774 conn(0x55644fdbffa0 msgr2=0x55644fdbf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:11.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbd5ffb640 0 client.22275 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:57:11.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3291516774 shutdown_connections 2024-03-26T17:57:11.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3291516774 wait complete. 2024-03-26T17:57:11.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:11.993+0000 7fcbf75c5740 20 client.22275 trim_cache size 0 max 16384 2024-03-26T17:57:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbf75c5740 1 Processor -- start 2024-03-26T17:57:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbf75c5740 1 -- start start 2024-03-26T17:57:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdc8e40 0x55644fde1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdda8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdce9a0 0x55644fdbe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:11.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01a6b10 con 0x55644fdce9a0 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01a8650 con 0x55644fdc8e40 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01a71e0 con 0x55644fdc9830 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbe7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdda8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdc8e40 0x55644fde1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbeed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdce9a0 0x55644fdbe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdc8e40 0x55644fde1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60406/0 (socket says 172.21.15.105:60406) 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbeed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdce9a0 0x55644fdbe930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44974/0 (socket says 172.21.15.105:44974) 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbee59c640 1 -- 172.21.15.105:0/465934864 learned_addr learned my addr 172.21.15.105:0/465934864 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:12.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.014+0000 7fcbe7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdda8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56704/0 (socket says 172.21.15.105:56704) 2024-03-26T17:57:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbe7fff640 1 -- 172.21.15.105:0/465934864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdc8e40 msgr2=0x55644fde1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/465934864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdc8e40 0x55644fde1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbe7fff640 1 -- 172.21.15.105:0/465934864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdce9a0 msgr2=0x55644fdbe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/465934864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdce9a0 0x55644fdbe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbe7fff640 1 -- 172.21.15.105:0/465934864 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55644fdca0c0 con 0x55644fdc9830 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbee59c640 1 --2- 172.21.15.105:0/465934864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdc8e40 0x55644fde1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/465934864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdda8c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcbd802c170 tx=0x7fcbd802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222976d217ca750b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbd7fff640 1 -- 172.21.15.105:0/465934864 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd802cc90 con 0x55644fdc9830 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbd7fff640 1 -- 172.21.15.105:0/465934864 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbd8059020 con 0x55644fdc9830 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.015+0000 7fcbd7fff640 1 -- 172.21.15.105:0/465934864 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8004c40 con 0x55644fdc9830 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 -- 172.21.15.105:0/465934864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 msgr2=0x55644fdda8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/465934864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdda8c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcbd802c170 tx=0x7fcbd802bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 -- 172.21.15.105:0/465934864 shutdown_connections 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/465934864 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdce9a0 0x55644fdbe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/465934864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdda8c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/465934864 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdc8e40 0x55644fde1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 -- 172.21.15.105:0/465934864 >> 172.21.15.105:0/465934864 conn(0x55644fdbffa0 msgr2=0x55644fdcad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 -- 172.21.15.105:0/465934864 shutdown_connections 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 -- 172.21.15.105:0/465934864 wait complete. 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.016+0000 7fcbf75c5740 1 -- start start 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbf75c5740 1 Processor -- start 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc8e40 0x55644fdd8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdd87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbf75c5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01a7f80 con 0x55644fdc8e40 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01a78b0 con 0x55644fdce9a0 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbf75c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01a4af0 con 0x55644fdc9830 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbf75c5740 10 client.0 _ll_register_callbacks cb 0x7fcbd8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbe7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdd87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbe7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdd87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56714/0 (socket says 172.21.15.105:56714) 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbe7fff640 1 -- 172.21.15.105:0/3518259884 learned_addr learned my addr 172.21.15.105:0/3518259884 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbee59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc8e40 0x55644fdd8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbeed9d640 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbe7fff640 1 -- 172.21.15.105:0/3518259884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 msgr2=0x55644fdd9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.017+0000 7fcbe7fff640 1 -- 172.21.15.105:0/3518259884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc8e40 msgr2=0x55644fdd8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.018+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc8e40 0x55644fdd8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.018+0000 7fcbe7fff640 1 -- 172.21.15.105:0/3518259884 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55644fdd9d00 con 0x55644fdc9830 2024-03-26T17:57:12.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.018+0000 7fcbee59c640 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc8e40 0x55644fdd8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.018+0000 7fcbe7fff640 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdd87c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcbd802c840 tx=0x7fcbd8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64016928a139ce08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.018+0000 7fcbd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.018+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3518259884 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd802cc90 con 0x55644fdc9830 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.018+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3518259884 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fcbd8034cc0 con 0x55644fdc9830 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.018+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3518259884 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbd8059020 con 0x55644fdc9830 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3518259884 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcbd8004980 con 0x55644fdc9830 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbf75c5740 10 client.31963 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3518259884 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55644fe45850 con 0x55644fdc9830 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3518259884 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcbd8059020 con 0x55644fdc9830 2024-03-26T17:57:12.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3518259884 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fcbd8003c50 con 0x55644fdc9830 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbf75c5740 10 client.31963 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbf75c5740 10 client.31963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbf75c5740 20 client.31963 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55644fe46190 0x55644fe48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbf75c5740 4 client.31963 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3518259884 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55644fd31200 con 0x55644fe46190 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.019+0000 7fcbee59c640 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55644fe46190 0x55644fe48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.020+0000 7fcbd5ffb640 10 client.31963 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:57:12.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.020+0000 7fcbee59c640 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55644fe46190 0x55644fe48580 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28356cf97635d92a server_cookie=6544dae781ed6d8a in_seq=0 out_seq=0 2024-03-26T17:57:12.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbd5ffb640 1 -- 172.21.15.105:0/3518259884 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55644fd31200 con 0x55644fe46190 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbd5ffb640 10 client.31963 handle_command_reply: tid=0 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 client.31963 shutdown 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3518259884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 msgr2=0x55644fdd87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdd87c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcbd802c840 tx=0x7fcbd8009260 comp rx=0 tx=0).stop 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3518259884 shutdown_connections 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55644fe46190 0x55644fe48580 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55644fdce9a0 0x55644fdd9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55644fdc9830 0x55644fdd87c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbd5ffb640 0 client.31963 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 --2- 172.21.15.105:0/3518259884 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55644fdc8e40 0x55644fdd8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3518259884 >> 172.21.15.105:0/3518259884 conn(0x55644fdbffa0 msgr2=0x55644fdc3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3518259884 shutdown_connections 2024-03-26T17:57:12.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 1 -- 172.21.15.105:0/3518259884 wait complete. 2024-03-26T17:57:12.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.023+0000 7fcbf75c5740 20 client.31963 trim_cache size 0 max 16384 2024-03-26T17:57:12.011 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 -- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbdc0a2f20 msgr2=0x7fcbdc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbdc0a2f20 0x7fcbdc0a53e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x55644fdca090 tx=0x7fcbe8009600 comp rx=0 tx=0).stop 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 -- 172.21.15.105:0/2075500100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf01023c0 msgr2=0x7fcbf0197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf01023c0 0x7fcbf0197af0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c920 tx=0x7fcbe000cde0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 -- 172.21.15.105:0/2075500100 shutdown_connections 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcbdc0a2f20 0x7fcbdc0a53e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbf0107620 0x7fcbf019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcbf0104cf0 0x7fcbf0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 --2- 172.21.15.105:0/2075500100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbf01023c0 0x7fcbf0197af0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.024+0000 7fcbf49e1640 1 -- 172.21.15.105:0/2075500100 >> 172.21.15.105:0/2075500100 conn(0x7fcbf00fbd70 msgr2=0x7fcbf0102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.025+0000 7fcbf49e1640 1 -- 172.21.15.105:0/2075500100 shutdown_connections 2024-03-26T17:57:12.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.025+0000 7fcbf49e1640 1 -- 172.21.15.105:0/2075500100 wait complete. 2024-03-26T17:57:12.023 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:57:12.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 -- 172.21.15.105:0/601357639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41007c0 msgr2=0x7fd0d4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/601357639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41007c0 0x7fd0d4100be0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009900 tx=0x7fd0d002bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:12.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 -- 172.21.15.105:0/601357639 shutdown_connections 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/601357639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4102370 0x7fd0d4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/601357639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d41019b0 0x7fd0d4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/601357639 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41007c0 0x7fd0d4100be0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 -- 172.21.15.105:0/601357639 >> 172.21.15.105:0/601357639 conn(0x7fd0d40fbd50 msgr2=0x7fd0d40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 -- 172.21.15.105:0/601357639 shutdown_connections 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.175+0000 7fd0dbe19640 1 -- 172.21.15.105:0/601357639 wait complete. 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0dbe19640 1 Processor -- start 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0dbe19640 1 -- start start 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0dbe19640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d41007c0 0x7fd0d4196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0dbe19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41019b0 0x7fd0d4196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0dbe19640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4102370 0x7fd0d419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0dbe19640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410c330 con 0x7fd0d41019b0 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0dbe19640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410c1b0 con 0x7fd0d4102370 2024-03-26T17:57:12.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0dbe19640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d410c4b0 con 0x7fd0d41007c0 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0d93b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41019b0 0x7fd0d4196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0d9bb5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d41007c0 0x7fd0d4196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0d93b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41019b0 0x7fd0d4196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44990/0 (socket says 172.21.15.105:44990) 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0d93b4640 1 -- 172.21.15.105:0/1751131523 learned_addr learned my addr 172.21.15.105:0/1751131523 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.176+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4102370 0x7fd0d419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0d9bb5640 1 -- 172.21.15.105:0/1751131523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4102370 msgr2=0x7fd0d419e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4102370 0x7fd0d419e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0d9bb5640 1 -- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41019b0 msgr2=0x7fd0d4196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41019b0 0x7fd0d4196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0d9bb5640 1 -- 172.21.15.105:0/1751131523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d419e680 con 0x7fd0d41007c0 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41019b0 0x7fd0d4196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d41007c0 0x7fd0d4196a60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0036040 tx=0x7fd0d00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0caffd640 1 -- 172.21.15.105:0/1751131523 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0043020 con 0x7fd0d41007c0 2024-03-26T17:57:12.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0dbe19640 1 -- 172.21.15.105:0/1751131523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d419e910 con 0x7fd0d41007c0 2024-03-26T17:57:12.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0caffd640 1 -- 172.21.15.105:0/1751131523 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0d0004a50 con 0x7fd0d41007c0 2024-03-26T17:57:12.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0caffd640 1 -- 172.21.15.105:0/1751131523 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0003b50 con 0x7fd0d41007c0 2024-03-26T17:57:12.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.177+0000 7fd0dbe19640 1 -- 172.21.15.105:0/1751131523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d419ee30 con 0x7fd0d41007c0 2024-03-26T17:57:12.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.180+0000 7fd0caffd640 1 -- 172.21.15.105:0/1751131523 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd0d0046020 con 0x7fd0d41007c0 2024-03-26T17:57:12.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0dea0a740 1 Processor -- start 2024-03-26T17:57:12.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0dea0a740 1 -- start start 2024-03-26T17:57:12.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fb17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fb21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb78e0 0x5609f0fc24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0dea0a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609f0f16970 con 0x5609f0fb1d20 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0dea0a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5609f0ebed70 con 0x5609f0fb78e0 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0da3b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fb17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0dea0a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609f0f12920 con 0x5609f0fb3090 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0da3b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fb17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56730/0 (socket says 172.21.15.105:56730) 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0da3b6640 1 -- 172.21.15.105:0/2643812242 learned_addr learned my addr 172.21.15.105:0/2643812242 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fb21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.181+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb78e0 0x5609f0fc24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0da3b6640 1 -- 172.21.15.105:0/2643812242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb78e0 msgr2=0x5609f0fc24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb78e0 0x5609f0fc24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0da3b6640 1 -- 172.21.15.105:0/2643812242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 msgr2=0x5609f0fb21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fb21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0caffd640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c00a2f20 0x7fd0c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0da3b6640 1 -- 172.21.15.105:0/2643812242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f0c2a3f0 con 0x5609f0fb3090 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fb21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:12.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0caffd640 1 -- 172.21.15.105:0/1751131523 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd0d00f1540 con 0x7fd0d41007c0 2024-03-26T17:57:12.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c00a2f20 0x7fd0c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fb17e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009fd0 tx=0x7fd0cc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1fd80eecc5866e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:12.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0bffff640 1 -- 172.21.15.105:0/2643812242 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc003b90 con 0x5609f0fb3090 2024-03-26T17:57:12.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0bffff640 1 -- 172.21.15.105:0/2643812242 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0cc0045a0 con 0x5609f0fb3090 2024-03-26T17:57:12.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.182+0000 7fd0bffff640 1 -- 172.21.15.105:0/2643812242 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc019400 con 0x5609f0fb3090 2024-03-26T17:57:12.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 -- 172.21.15.105:0/2643812242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 msgr2=0x5609f0fb17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c00a2f20 0x7fd0c00a53e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc01d000 tx=0x7fd0cc019750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fb17e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009fd0 tx=0x7fd0cc0118d0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 -- 172.21.15.105:0/2643812242 shutdown_connections 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb78e0 0x5609f0fc24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fb21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/2643812242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fb17e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 -- 172.21.15.105:0/2643812242 >> 172.21.15.105:0/2643812242 conn(0x5609f0facfa0 msgr2=0x5609f0fad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 -- 172.21.15.105:0/2643812242 shutdown_connections 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.183+0000 7fd0dea0a740 1 -- 172.21.15.105:0/2643812242 wait complete. 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0dea0a740 1 -- start start 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0dea0a740 1 Processor -- start 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fc4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fc4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fc9c70 0x5609f0fcc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0dea0a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5609f0fc2d60 con 0x5609f0fb1d20 2024-03-26T17:57:12.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0dea0a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5609f0ed2310 con 0x5609f0fc9c70 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0dea0a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609f0fc2ee0 con 0x5609f0fb3090 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0d9bb5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fc4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.184+0000 7fd0d93b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fc4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0d93b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fc4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56742/0 (socket says 172.21.15.105:56742) 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0d93b4640 1 -- 172.21.15.105:0/3544090851 learned_addr learned my addr 172.21.15.105:0/3544090851 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0dea0a740 10 client.0 _ll_register_callbacks cb 0x5609f0f0dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fc9c70 0x5609f0fcc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0d93b4640 1 -- 172.21.15.105:0/3544090851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fc9c70 msgr2=0x5609f0fcc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fc9c70 0x5609f0fcc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0d93b4640 1 -- 172.21.15.105:0/3544090851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 msgr2=0x5609f0fc4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fc4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0d93b4640 1 -- 172.21.15.105:0/3544090851 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f0cf47d0 con 0x5609f0fb3090 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fc4c40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4012d20 tx=0x7fd0c400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6908017e7515de0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:12.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:12.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3544090851 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40120d0 con 0x5609f0fb3090 2024-03-26T17:57:12.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3544090851 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd0c4010430 con 0x5609f0fb3090 2024-03-26T17:57:12.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3544090851 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0cc0045a0 con 0x5609f0fb3090 2024-03-26T17:57:12.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.185+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3544090851 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd0c4012270 con 0x5609f0fb3090 2024-03-26T17:57:12.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0dea0a740 10 client.31975 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:12.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3544090851 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5609f0fcf480 con 0x5609f0fb3090 2024-03-26T17:57:12.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3544090851 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0cc0045a0 con 0x5609f0fb3090 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3544090851 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd0c4026660 con 0x5609f0fb3090 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0dea0a740 10 client.31975 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0dea0a740 10 client.31975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0dea0a740 20 client.31975 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5609f0fd0b10 0x5609f0fd2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0dea0a740 4 client.31975 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3544090851 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609f0f23eb0 con 0x5609f0fd0b10 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.186+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5609f0fd0b10 0x5609f0fd2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.187+0000 7fd0bd7fa640 10 client.31975 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.187+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5609f0fd0b10 0x5609f0fd2fb0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb30d173174d7317 server_cookie=d7c1a815f7014629 in_seq=0 out_seq=0 2024-03-26T17:57:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.188+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3544090851 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5609f0f23eb0 con 0x5609f0fd0b10 2024-03-26T17:57:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.188+0000 7fd0bd7fa640 10 client.31975 handle_command_reply: tid=0 2024-03-26T17:57:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.188+0000 7fd0dea0a740 1 client.31975 shutdown 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.188+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3544090851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 msgr2=0x5609f0fc4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.188+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fc4c40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4012d20 tx=0x7fd0c400f790 comp rx=0 tx=0).stop 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3544090851 shutdown_connections 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5609f0fd0b10 0x5609f0fd2fb0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fc9c70 0x5609f0fcc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fb3090 0x5609f0fc4c40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3544090851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fc4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3544090851 >> 172.21.15.105:0/3544090851 conn(0x5609f0facfa0 msgr2=0x5609f0fac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0bd7fa640 0 client.31975 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3544090851 shutdown_connections 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3544090851 wait complete. 2024-03-26T17:57:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.189+0000 7fd0dea0a740 20 client.31975 trim_cache size 0 max 16384 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.209+0000 7fd0dea0a740 1 Processor -- start 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.209+0000 7fd0dea0a740 1 -- start start 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb1d20 0x5609f0fc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb3090 0x5609f0fc37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d93b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb3090 0x5609f0fc37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d93b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb3090 0x5609f0fc37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45022/0 (socket says 172.21.15.105:45022) 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d93b4640 1 -- 172.21.15.105:0/3291170160 learned_addr learned my addr 172.21.15.105:0/3291170160 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fc3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3291170160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d41a6bc0 con 0x5609f0fb3090 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3291170160 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d41a8700 con 0x5609f0fb1d20 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3291170160 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d41a7290 con 0x5609f0fc9c70 2024-03-26T17:57:12.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fc3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb1d20 0x5609f0fc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d93b4640 1 -- 172.21.15.105:0/3291170160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 msgr2=0x5609f0fc3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fc3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d93b4640 1 -- 172.21.15.105:0/3291170160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb1d20 msgr2=0x5609f0fc3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb1d20 0x5609f0fc3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d93b4640 1 -- 172.21.15.105:0/3291170160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609f0fac8f0 con 0x5609f0fb3090 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.210+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb1d20 0x5609f0fc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.211+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb3090 0x5609f0fc37c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4026cc0 tx=0x7fd0c400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8e66ff55dfe8de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.211+0000 7fd0bf7fe640 1 -- 172.21.15.105:0/3291170160 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4026440 con 0x5609f0fb3090 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.211+0000 7fd0bf7fe640 1 -- 172.21.15.105:0/3291170160 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0c40265e0 con 0x5609f0fb3090 2024-03-26T17:57:12.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.211+0000 7fd0bf7fe640 1 -- 172.21.15.105:0/3291170160 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c401cdf0 con 0x5609f0fb3090 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3291170160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb3090 msgr2=0x5609f0fc37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb3090 0x5609f0fc37c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4026cc0 tx=0x7fd0c400fd10 comp rx=0 tx=0).stop 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3291170160 shutdown_connections 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fc3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb3090 0x5609f0fc37c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3291170160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb1d20 0x5609f0fc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3291170160 >> 172.21.15.105:0/3291170160 conn(0x5609f0facfa0 msgr2=0x5609f0fac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3291170160 shutdown_connections 2024-03-26T17:57:12.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3291170160 wait complete. 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 -- start start 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.212+0000 7fd0dea0a740 1 Processor -- start 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.213+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fe2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.213+0000 7fd0dea0a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb3090 0x5609f0fe2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0d9bb5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fe2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0d9bb5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fe2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45032/0 (socket says 172.21.15.105:45032) 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0d9bb5640 1 -- 172.21.15.105:0/3938099567 learned_addr learned my addr 172.21.15.105:0/3938099567 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fe35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d41a8030 con 0x5609f0fb1d20 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb3090 0x5609f0fe2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d41a7960 con 0x5609f0fb3090 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d41a4ba0 con 0x5609f0fc9c70 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fe35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0dea0a740 10 client.0 _ll_register_callbacks cb 0x7fd0c4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:12.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0da3b6640 1 -- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb3090 msgr2=0x5609f0fe2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb3090 0x5609f0fe2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0da3b6640 1 -- 172.21.15.105:0/3938099567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 msgr2=0x5609f0fe2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fe2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0da3b6640 1 -- 172.21.15.105:0/3938099567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609f0fadd10 con 0x5609f0fc9c70 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0d93b4640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb3090 0x5609f0fe2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fe2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c40265e0 con 0x5609f0fc9c70 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0da3b6640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fe35a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009fa0 tx=0x7fd0cc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474c8f5149793025 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.214+0000 7fd0bd7fa640 10 client.31981 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3938099567 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc03d060 con 0x5609f0fc9c70 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3938099567 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd0cc01f040 con 0x5609f0fc9c70 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3938099567 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd0cc004050 con 0x5609f0fc9c70 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0dea0a740 10 client.31981 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:12.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5609f1036f80 con 0x5609f0fc9c70 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3938099567 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd0cc0041f0 con 0x5609f0fc9c70 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0dea0a740 10 client.31981 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0dea0a740 10 client.31981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0dea0a740 20 client.31981 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5609f1037670 0x5609f1039a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0dea0a740 4 client.31981 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.215+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609f0f1e200 con 0x5609f1037670 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.216+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5609f1037670 0x5609f1039a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.216+0000 7fd0d9bb5640 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5609f1037670 0x5609f1039a60 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44d1c6493d56f47 server_cookie=6e6d7b2d71af9d2b in_seq=0 out_seq=0 2024-03-26T17:57:12.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.216+0000 7fd0bd7fa640 10 client.31981 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:12.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0bd7fa640 1 -- 172.21.15.105:0/3938099567 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609f0f1e200 con 0x5609f1037670 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0bd7fa640 10 client.31981 handle_command_reply: tid=0 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 client.31981 shutdown 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 msgr2=0x5609f0fe35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fe35a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009fa0 tx=0x7fd0cc017000 comp rx=0 tx=0).stop 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 shutdown_connections 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5609f1037670 0x5609f1039a60 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609f0fc9c70 0x5609f0fe35a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609f0fb3090 0x5609f0fe2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 --2- 172.21.15.105:0/3938099567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5609f0fb1d20 0x5609f0fe2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 >> 172.21.15.105:0/3938099567 conn(0x5609f0facfa0 msgr2=0x5609f0fc3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0bd7fa640 0 client.31981 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 shutdown_connections 2024-03-26T17:57:12.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 1 -- 172.21.15.105:0/3938099567 wait complete. 2024-03-26T17:57:12.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.217+0000 7fd0dea0a740 20 client.31981 trim_cache size 0 max 16384 2024-03-26T17:57:12.207 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 -- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c00a2f20 msgr2=0x7fd0c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c00a2f20 0x7fd0c00a53e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc01d000 tx=0x7fd0cc019750 comp rx=0 tx=0).stop 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 -- 172.21.15.105:0/1751131523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d41007c0 msgr2=0x7fd0d4196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d41007c0 0x7fd0d4196a60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0036040 tx=0x7fd0d00042f0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 -- 172.21.15.105:0/1751131523 shutdown_connections 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0c00a2f20 0x7fd0c00a53e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0d4102370 0x7fd0d419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0d41019b0 0x7fd0d4196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 --2- 172.21.15.105:0/1751131523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0d41007c0 0x7fd0d4196a60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 -- 172.21.15.105:0/1751131523 >> 172.21.15.105:0/1751131523 conn(0x7fd0d40fbd50 msgr2=0x7fd0d4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 -- 172.21.15.105:0/1751131523 shutdown_connections 2024-03-26T17:57:12.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:12.218+0000 7fd0dbe19640 1 -- 172.21.15.105:0/1751131523 wait complete. 2024-03-26T17:57:12.215 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:11 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3954062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:57:12.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:11 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3954062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:57:12.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:11 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3954062' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:57:12.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 46661 7.08 MB/sec execute 235 sec latency 127.026 ms 2024-03-26T17:57:13.217 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:57:13.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:57:13.218 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:57:13.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:12 smithi105 ceph-mon[32280]: pgmap v916: 97 pgs: 97 active+clean; 312 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-03-26T17:57:13.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:12 smithi144 ceph-mon[36095]: pgmap v916: 97 pgs: 97 active+clean; 312 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-03-26T17:57:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 -- 172.21.15.105:0/1661008457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ddc102380 msgr2=0x7f4ddc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 --2- 172.21.15.105:0/1661008457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ddc102380 0x7f4ddc108c10 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009900 tx=0x7f4dd402bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 -- 172.21.15.105:0/1661008457 shutdown_connections 2024-03-26T17:57:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 --2- 172.21.15.105:0/1661008457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ddc102380 0x7f4ddc108c10 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 --2- 172.21.15.105:0/1661008457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ddc1019c0 0x7f4ddc101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 --2- 172.21.15.105:0/1661008457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc1007d0 0x7f4ddc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 -- 172.21.15.105:0/1661008457 >> 172.21.15.105:0/1661008457 conn(0x7f4ddc0fbd60 msgr2=0x7f4ddc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 -- 172.21.15.105:0/1661008457 shutdown_connections 2024-03-26T17:57:13.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 -- 172.21.15.105:0/1661008457 wait complete. 2024-03-26T17:57:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.371+0000 7f4de1a43640 1 Processor -- start 2024-03-26T17:57:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4de1a43640 1 -- start start 2024-03-26T17:57:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4de1a43640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc1007d0 0x7f4ddc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4de1a43640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ddc1019c0 0x7f4ddc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4de1a43640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ddc102380 0x7f4ddc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4de1a43640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc10d3e0 con 0x7f4ddc102380 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4de1a43640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc10d260 con 0x7f4ddc1019c0 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4de1a43640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc10d560 con 0x7f4ddc1007d0 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4ddaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc1007d0 0x7f4ddc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4dda7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ddc1019c0 0x7f4ddc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4ddaffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc1007d0 0x7f4ddc197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56784/0 (socket says 172.21.15.105:56784) 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4ddaffd640 1 -- 172.21.15.105:0/4018135185 learned_addr learned my addr 172.21.15.105:0/4018135185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.372+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ddc102380 0x7f4ddc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4dda7fc640 1 -- 172.21.15.105:0/4018135185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc1007d0 msgr2=0x7f4ddc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc1007d0 0x7f4ddc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4dda7fc640 1 -- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ddc102380 msgr2=0x7f4ddc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ddc102380 0x7f4ddc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4dda7fc640 1 -- 172.21.15.105:0/4018135185 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc19f730 con 0x7f4ddc1019c0 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ddc1019c0 0x7f4ddc198050 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0005fd0 tx=0x7f4dd000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4de0a41640 1 -- 172.21.15.105:0/4018135185 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0009c30 con 0x7f4ddc1019c0 2024-03-26T17:57:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4de1a43640 1 -- 172.21.15.105:0/4018135185 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc19fa20 con 0x7f4ddc1019c0 2024-03-26T17:57:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4de1a43640 1 -- 172.21.15.105:0/4018135185 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc19ffc0 con 0x7f4ddc1019c0 2024-03-26T17:57:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4de0a41640 1 -- 172.21.15.105:0/4018135185 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dd0009dd0 con 0x7f4ddc1019c0 2024-03-26T17:57:13.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.373+0000 7f4de0a41640 1 -- 172.21.15.105:0/4018135185 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0003e20 con 0x7f4ddc1019c0 2024-03-26T17:57:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.376+0000 7f4de0a41640 1 -- 172.21.15.105:0/4018135185 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4dd0010460 con 0x7f4ddc1019c0 2024-03-26T17:57:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de462a740 1 Processor -- start 2024-03-26T17:57:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de462a740 1 -- start start 2024-03-26T17:57:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de462a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2d29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de462a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d2f90 0x55890e2d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de462a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de462a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55890e233630 con 0x55890e2d25a0 2024-03-26T17:57:13.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de462a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55890e1dbe10 con 0x55890e2d2f90 2024-03-26T17:57:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de462a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55890e22f5e0 con 0x55890e2d8100 2024-03-26T17:57:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2d29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2d29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45056/0 (socket says 172.21.15.105:45056) 2024-03-26T17:57:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4ddb7fe640 1 -- 172.21.15.105:0/2437661644 learned_addr learned my addr 172.21.15.105:0/2437661644 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4de0a41640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4dc40a2f20 0x7f4dc40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.377+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d2f90 0x55890e2d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4de0a41640 1 -- 172.21.15.105:0/4018135185 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f4dd00cee60 con 0x7f4ddc1019c0 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4ddb7fe640 1 -- 172.21.15.105:0/2437661644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 msgr2=0x55890e2da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4ddb7fe640 1 -- 172.21.15.105:0/2437661644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d2f90 msgr2=0x55890e2d7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d2f90 0x55890e2d7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4ddb7fe640 1 -- 172.21.15.105:0/2437661644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890e1f2740 con 0x55890e2d25a0 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2d29c0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009900 tx=0x7f4dd4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39761b5e0c52604 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.378+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4dc40a2f20 0x7f4dc40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.379+0000 7f4dc1ffb640 1 -- 172.21.15.105:0/2437661644 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4045020 con 0x55890e2d25a0 2024-03-26T17:57:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.379+0000 7f4dc1ffb640 1 -- 172.21.15.105:0/2437661644 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dd402cc60 con 0x55890e2d25a0 2024-03-26T17:57:13.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.379+0000 7f4dc1ffb640 1 -- 172.21.15.105:0/2437661644 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4004a80 con 0x55890e2d25a0 2024-03-26T17:57:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 -- 172.21.15.105:0/2437661644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 msgr2=0x55890e2d29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2d29c0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009900 tx=0x7f4dd4005d50 comp rx=0 tx=0).stop 2024-03-26T17:57:13.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4dc40a2f20 0x7f4dc40a53e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4005c40 tx=0x7f4dd4004d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 -- 172.21.15.105:0/2437661644 shutdown_connections 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d2f90 0x55890e2d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 --2- 172.21.15.105:0/2437661644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2d29c0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 -- 172.21.15.105:0/2437661644 >> 172.21.15.105:0/2437661644 conn(0x55890e2c9700 msgr2=0x55890e2c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 -- 172.21.15.105:0/2437661644 shutdown_connections 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 -- 172.21.15.105:0/2437661644 wait complete. 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.380+0000 7f4de462a740 1 -- start start 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4de462a740 1 Processor -- start 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4de462a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2dcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4de462a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2dd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4de462a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4de462a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55890e2db050 con 0x55890e2d25a0 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4de462a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55890e1ef1a0 con 0x55890e2d8100 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4de462a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55890e2db1d0 con 0x55890e2d2f90 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4dda7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2dd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4dda7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2dd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56812/0 (socket says 172.21.15.105:56812) 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4dda7fc640 1 -- 172.21.15.105:0/193522779 learned_addr learned my addr 172.21.15.105:0/193522779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2dcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4de462a740 10 client.0 _ll_register_callbacks cb 0x55890df46c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:13.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.381+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dda7fc640 1 -- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 msgr2=0x55890e2e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dda7fc640 1 -- 172.21.15.105:0/193522779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 msgr2=0x55890e2dcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2dcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dda7fc640 1 -- 172.21.15.105:0/193522779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890e00f7e0 con 0x55890e2d2f90 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2dcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2dd060 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009610 tx=0x7f4dd0097960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef967b24c2874f97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dbaffd640 1 -- 172.21.15.105:0/193522779 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0020050 con 0x55890e2d2f90 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4dbaffd640 1 -- 172.21.15.105:0/193522779 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f4dd009f150 con 0x55890e2d2f90 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.382+0000 7f4de462a740 1 -- 172.21.15.105:0/193522779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd402cc60 con 0x55890e2d2f90 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4dbaffd640 1 -- 172.21.15.105:0/193522779 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4dd009f640 con 0x55890e2d2f90 2024-03-26T17:57:13.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4de462a740 10 client.31993 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4de462a740 1 -- 172.21.15.105:0/193522779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55890e06ff70 con 0x55890e2d2f90 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4dbaffd640 1 -- 172.21.15.105:0/193522779 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dd402cc60 con 0x55890e2d2f90 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4dbaffd640 1 -- 172.21.15.105:0/193522779 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4dd0084c50 con 0x55890e2d2f90 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4de462a740 10 client.31993 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4de462a740 10 client.31993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4de462a740 20 client.31993 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4de462a740 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890e2e88f0 0x55890e2ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4de462a740 4 client.31993 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.383+0000 7f4de462a740 1 -- 172.21.15.105:0/193522779 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55890e240b70 con 0x55890e2e88f0 2024-03-26T17:57:13.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.384+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890e2e88f0 0x55890e2ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.384+0000 7f4dbaffd640 10 client.31993 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.384+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890e2e88f0 0x55890e2ead90 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec2168355a3938e server_cookie=c523db8a55b8f1f in_seq=0 out_seq=0 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4dbaffd640 1 -- 172.21.15.105:0/193522779 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55890e240b70 con 0x55890e2e88f0 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4dbaffd640 10 client.31993 handle_command_reply: tid=0 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 client.31993 shutdown 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 -- 172.21.15.105:0/193522779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 msgr2=0x55890e2dd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2dd060 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009610 tx=0x7f4dd0097960 comp rx=0 tx=0).stop 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 -- 172.21.15.105:0/193522779 shutdown_connections 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890e2e88f0 0x55890e2ead90 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4dbaffd640 0 client.31993 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2dd060 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 --2- 172.21.15.105:0/193522779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2dcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 -- 172.21.15.105:0/193522779 >> 172.21.15.105:0/193522779 conn(0x55890e2c9700 msgr2=0x55890e2c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 -- 172.21.15.105:0/193522779 shutdown_connections 2024-03-26T17:57:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.386+0000 7f4de462a740 1 -- 172.21.15.105:0/193522779 wait complete. 2024-03-26T17:57:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.387+0000 7f4de462a740 20 client.31993 trim_cache size 0 max 16384 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.416+0000 7f4de462a740 1 Processor -- start 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.416+0000 7f4de462a740 1 -- start start 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.416+0000 7f4de462a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.416+0000 7f4de462a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2e4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.416+0000 7f4de462a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.416+0000 7f4de462a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc800e520 con 0x55890e2d25a0 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.416+0000 7f4de462a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8010060 con 0x55890e2d8100 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.416+0000 7f4de462a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc800ebf0 con 0x55890e2d2f90 2024-03-26T17:57:13.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4dda7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4dda7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2e4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56818/0 (socket says 172.21.15.105:56818) 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60520/0 (socket says 172.21.15.105:60520) 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4dda7fc640 1 -- 172.21.15.105:0/48893673 learned_addr learned my addr 172.21.15.105:0/48893673 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddb7fe640 1 -- 172.21.15.105:0/48893673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 msgr2=0x55890e2e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2e4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddb7fe640 1 -- 172.21.15.105:0/48893673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 msgr2=0x55890e2c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddb7fe640 1 -- 172.21.15.105:0/48893673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890df07170 con 0x55890e2d8100 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2c85b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4036040 tx=0x7f4dd4053000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812b78bb955b28bd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.417+0000 7f4dc17fa640 1 -- 172.21.15.105:0/48893673 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4045070 con 0x55890e2d8100 2024-03-26T17:57:13.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.418+0000 7f4dc17fa640 1 -- 172.21.15.105:0/48893673 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dd402cc60 con 0x55890e2d8100 2024-03-26T17:57:13.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.418+0000 7f4dc17fa640 1 -- 172.21.15.105:0/48893673 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4003b40 con 0x55890e2d8100 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.418+0000 7f4de462a740 1 -- 172.21.15.105:0/48893673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 msgr2=0x55890e2c85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.418+0000 7f4de462a740 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2c85b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4036040 tx=0x7f4dd4053000 comp rx=0 tx=0).stop 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 -- 172.21.15.105:0/48893673 shutdown_connections 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d8100 0x55890e2c85b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d2f90 0x55890e2e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 --2- 172.21.15.105:0/48893673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d25a0 0x55890e2c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 -- 172.21.15.105:0/48893673 >> 172.21.15.105:0/48893673 conn(0x55890e2c9700 msgr2=0x55890e2c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 -- 172.21.15.105:0/48893673 shutdown_connections 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 -- 172.21.15.105:0/48893673 wait complete. 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 -- start start 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 Processor -- start 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d25a0 0x55890e2e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.419+0000 7f4de462a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d2f90 0x55890e2e2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4de462a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2e3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4de462a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc800f990 con 0x55890e2d2f90 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4de462a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc800f2c0 con 0x55890e2d25a0 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4de462a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc800c500 con 0x55890e2d8100 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4de462a740 10 client.0 _ll_register_callbacks cb 0x7f4dd000df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d25a0 0x55890e2e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d25a0 0x55890e2e1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60536/0 (socket says 172.21.15.105:60536) 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 -- 172.21.15.105:0/14670896 learned_addr learned my addr 172.21.15.105:0/14670896 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d2f90 0x55890e2e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 -- 172.21.15.105:0/14670896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 msgr2=0x55890e2e3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2e3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 -- 172.21.15.105:0/14670896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d2f90 msgr2=0x55890e2e2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d2f90 0x55890e2e2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 -- 172.21.15.105:0/14670896 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890e2e3980 con 0x55890e2d25a0 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddb7fe640 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4ddaffd640 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d25a0 0x55890e2e1e40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc000f80 tx=0x7f4dcc01f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2a109e00aa573f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4dbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.420+0000 7f4dbaffd640 1 -- 172.21.15.105:0/14670896 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc004d20 con 0x55890e2d25a0 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.421+0000 7f4de462a740 1 -- 172.21.15.105:0/14670896 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd402cc60 con 0x55890e2d25a0 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.421+0000 7f4dbaffd640 1 -- 172.21.15.105:0/14670896 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f4dcc022150 con 0x55890e2d25a0 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.421+0000 7f4dbaffd640 1 -- 172.21.15.105:0/14670896 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4dcc031050 con 0x55890e2d25a0 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.421+0000 7f4de462a740 10 client.32009 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:13.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.422+0000 7f4de462a740 1 -- 172.21.15.105:0/14670896 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55890e34f420 con 0x55890e2d25a0 2024-03-26T17:57:13.410 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:12 smithi135 ceph-mon[35636]: pgmap v916: 97 pgs: 97 active+clean; 312 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 3.1 MiB/s wr, 219 op/s 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.422+0000 7f4dbaffd640 1 -- 172.21.15.105:0/14670896 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dd402cc60 con 0x55890e2d25a0 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.423+0000 7f4dbaffd640 1 -- 172.21.15.105:0/14670896 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f4dcc0108b0 con 0x55890e2d25a0 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.423+0000 7f4de462a740 10 client.32009 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.423+0000 7f4de462a740 10 client.32009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.423+0000 7f4de462a740 20 client.32009 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.423+0000 7f4de462a740 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890e34fdf0 0x55890e352210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.423+0000 7f4de462a740 4 client.32009 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.423+0000 7f4de462a740 1 -- 172.21.15.105:0/14670896 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55890e23aec0 con 0x55890e34fdf0 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.423+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890e34fdf0 0x55890e352210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.424+0000 7f4dbaffd640 10 client.32009 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.424+0000 7f4dda7fc640 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890e34fdf0 0x55890e352210 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6816d482c559e83c server_cookie=bbbcfa4df8b9124f in_seq=0 out_seq=0 2024-03-26T17:57:13.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.426+0000 7f4dbaffd640 1 -- 172.21.15.105:0/14670896 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55890e23aec0 con 0x55890e34fdf0 2024-03-26T17:57:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.426+0000 7f4dbaffd640 10 client.32009 handle_command_reply: tid=0 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.426+0000 7f4de462a740 1 client.32009 shutdown 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.426+0000 7f4de462a740 1 -- 172.21.15.105:0/14670896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d25a0 msgr2=0x55890e2e1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.426+0000 7f4de462a740 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d25a0 0x55890e2e1e40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc000f80 tx=0x7f4dcc01f7c0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 1 -- 172.21.15.105:0/14670896 shutdown_connections 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890e34fdf0 0x55890e352210 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890e2d8100 0x55890e2e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890e2d2f90 0x55890e2e2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4dbaffd640 0 client.32009 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 1 --2- 172.21.15.105:0/14670896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890e2d25a0 0x55890e2e1e40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 1 -- 172.21.15.105:0/14670896 >> 172.21.15.105:0/14670896 conn(0x55890e2c9700 msgr2=0x55890e2cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 1 -- 172.21.15.105:0/14670896 shutdown_connections 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 1 -- 172.21.15.105:0/14670896 wait complete. 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.427+0000 7f4de462a740 20 client.32009 trim_cache size 0 max 16384 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "9b0e103a-07f0-45dc-b4e9-9695f7c8e921", 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:57:13.414 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:57:13.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 -- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4dc40a2f20 msgr2=0x7f4dc40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4dc40a2f20 0x7f4dc40a53e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4005c40 tx=0x7f4dd4004d60 comp rx=0 tx=0).stop 2024-03-26T17:57:13.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 -- 172.21.15.105:0/4018135185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ddc1019c0 msgr2=0x7f4ddc198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ddc1019c0 0x7f4ddc198050 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0005fd0 tx=0x7f4dd000dde0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 -- 172.21.15.105:0/4018135185 shutdown_connections 2024-03-26T17:57:13.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4dc40a2f20 0x7f4dc40a53e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ddc102380 0x7f4ddc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4ddc1019c0 0x7f4ddc198050 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 --2- 172.21.15.105:0/4018135185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc1007d0 0x7f4ddc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 -- 172.21.15.105:0/4018135185 >> 172.21.15.105:0/4018135185 conn(0x7f4ddc0fbd60 msgr2=0x7f4ddc104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 -- 172.21.15.105:0/4018135185 shutdown_connections 2024-03-26T17:57:13.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.428+0000 7f4de1a43640 1 -- 172.21.15.105:0/4018135185 wait complete. 2024-03-26T17:57:13.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:57:13.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:57:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 -- 172.21.15.105:0/2954617868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 msgr2=0x7fd640104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 --2- 172.21.15.105:0/2954617868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 0x7fd640104750 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd630009ca0 tx=0x7fd63002c100 comp rx=0 tx=0).stop 2024-03-26T17:57:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 -- 172.21.15.105:0/2954617868 shutdown_connections 2024-03-26T17:57:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 --2- 172.21.15.105:0/2954617868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd6401075c0 0x7fd640109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 --2- 172.21.15.105:0/2954617868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd640104c90 0x7fd640107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 --2- 172.21.15.105:0/2954617868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 0x7fd640104750 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 -- 172.21.15.105:0/2954617868 >> 172.21.15.105:0/2954617868 conn(0x7fd6400fbd50 msgr2=0x7fd6400fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 -- 172.21.15.105:0/2954617868 shutdown_connections 2024-03-26T17:57:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.561+0000 7fd645937640 1 -- 172.21.15.105:0/2954617868 wait complete. 2024-03-26T17:57:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd645937640 1 Processor -- start 2024-03-26T17:57:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd645937640 1 -- start start 2024-03-26T17:57:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd645937640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 0x7fd640199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd645937640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd640104c90 0x7fd64019a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd645937640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6401075c0 0x7fd6401a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd645937640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010d430 con 0x7fd6401075c0 2024-03-26T17:57:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd645937640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010d2b0 con 0x7fd640104c90 2024-03-26T17:57:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd645937640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd64010d5b0 con 0x7fd640102360 2024-03-26T17:57:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd63f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6401075c0 0x7fd6401a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd63effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 0x7fd640199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd63effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 0x7fd640199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56834/0 (socket says 172.21.15.105:56834) 2024-03-26T17:57:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd63f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6401075c0 0x7fd6401a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45110/0 (socket says 172.21.15.105:45110) 2024-03-26T17:57:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd63effd640 1 -- 172.21.15.105:0/1673453106 learned_addr learned my addr 172.21.15.105:0/1673453106 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.562+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd640104c90 0x7fd64019a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1673453106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 msgr2=0x7fd640199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 0x7fd640199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1673453106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd640104c90 msgr2=0x7fd64019a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd640104c90 0x7fd64019a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1673453106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd640192b10 con 0x7fd6401075c0 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd63effd640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 0x7fd640199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6401075c0 0x7fd6401a1220 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd638009e70 tx=0x7fd63800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd644935640 1 -- 172.21.15.105:0/1673453106 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63800ccc0 con 0x7fd6401075c0 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd644935640 1 -- 172.21.15.105:0/1673453106 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd638005020 con 0x7fd6401075c0 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd644935640 1 -- 172.21.15.105:0/1673453106 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63800ce60 con 0x7fd6401075c0 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd645937640 1 -- 172.21.15.105:0/1673453106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd640192e00 con 0x7fd6401075c0 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.563+0000 7fd645937640 1 -- 172.21.15.105:0/1673453106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6401932a0 con 0x7fd6401075c0 2024-03-26T17:57:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.566+0000 7fd644935640 1 -- 172.21.15.105:0/1673453106 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd6380051d0 con 0x7fd6401075c0 2024-03-26T17:57:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd6485f8740 1 Processor -- start 2024-03-26T17:57:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd644935640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6280a3040 0x7fd6280a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd6485f8740 1 -- start start 2024-03-26T17:57:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd644935640 1 -- 172.21.15.105:0/1673453106 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd6380e0380 con 0x7fd6401075c0 2024-03-26T17:57:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c85d0 0x55d0382c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382ce130 0x55d0382d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6280a3040 0x7fd6280a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd6485f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d03822d0b0 con 0x55d0382c8fc0 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd6485f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0381d5960 con 0x55d0382ce130 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd6485f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d038229060 con 0x55d0382c85d0 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd63f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382ce130 0x55d0382d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd63effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c85d0 0x55d0382c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd63f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382ce130 0x55d0382d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60552/0 (socket says 172.21.15.105:60552) 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd63effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c85d0 0x55d0382c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56848/0 (socket says 172.21.15.105:56848) 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1069308126 learned_addr learned my addr 172.21.15.105:0/1069308126 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.567+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1069308126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c85d0 msgr2=0x55d0382c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c85d0 0x55d0382c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1069308126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 msgr2=0x55d0382cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1069308126 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0381e7c60 con 0x55d0382ce130 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd63effd640 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c85d0 0x55d0382c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382ce130 0x55d0382d05c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd6380a93e0 tx=0x7fd6380ad110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0f458bad038c90 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6280a3040 0x7fd6280a5500 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd64019b180 tx=0x7fd634009bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd625ffb640 1 -- 172.21.15.105:0/1069308126 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63802d040 con 0x55d0382ce130 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd625ffb640 1 -- 172.21.15.105:0/1069308126 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6380b5420 con 0x55d0382ce130 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.568+0000 7fd625ffb640 1 -- 172.21.15.105:0/1069308126 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380a8760 con 0x55d0382ce130 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 -- 172.21.15.105:0/1069308126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382ce130 msgr2=0x55d0382d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382ce130 0x55d0382d05c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd6380a93e0 tx=0x7fd6380ad110 comp rx=0 tx=0).stop 2024-03-26T17:57:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 -- 172.21.15.105:0/1069308126 shutdown_connections 2024-03-26T17:57:13.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 46784 7.07 MB/sec execute 236 sec latency 183.567 ms 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382ce130 0x55d0382d05c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1069308126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c85d0 0x55d0382c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 -- 172.21.15.105:0/1069308126 >> 172.21.15.105:0/1069308126 conn(0x55d0382c3b40 msgr2=0x55d0382c3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 -- 172.21.15.105:0/1069308126 shutdown_connections 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.569+0000 7fd6485f8740 1 -- 172.21.15.105:0/1069308126 wait complete. 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 1 -- start start 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 1 Processor -- start 2024-03-26T17:57:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d0382d9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d0382d10e0 con 0x55d0382c8fc0 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0381e8db0 con 0x55d0382c85d0 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0382d1260 con 0x55d0382ce130 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.570+0000 7fd6485f8740 10 client.0 _ll_register_callbacks cb 0x55d0381ed110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60554/0 (socket says 172.21.15.105:60554) 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63effd640 1 -- 172.21.15.105:0/1059831889 learned_addr learned my addr 172.21.15.105:0/1059831889 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d0382d9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1059831889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 msgr2=0x55d0382d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1059831889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 msgr2=0x55d0382d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63f7fe640 1 -- 172.21.15.105:0/1059831889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d037f41210 con 0x55d0382ce130 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63effd640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.571+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d0382d9970 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd6380a8e20 tx=0x7fd6380acb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=495a4aca68608fe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd61effd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd61effd640 1 -- 172.21.15.105:0/1059831889 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380c9030 con 0x55d0382ce130 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 1 -- 172.21.15.105:0/1059831889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6380b5420 con 0x55d0382ce130 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd61effd640 1 -- 172.21.15.105:0/1059831889 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd6380aad10 con 0x55d0382ce130 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd61effd640 1 -- 172.21.15.105:0/1059831889 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd63802d070 con 0x55d0382ce130 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 10 client.32011 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 1 -- 172.21.15.105:0/1059831889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d03800d620 con 0x55d0382ce130 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd61effd640 1 -- 172.21.15.105:0/1059831889 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6380b5420 con 0x55d0382ce130 2024-03-26T17:57:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd61effd640 1 -- 172.21.15.105:0/1059831889 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd6380ab8d0 con 0x55d0382ce130 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 10 client.32011 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 10 client.32011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 20 client.32011 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0382dde60 0x55d0382e0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 4 client.32011 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.572+0000 7fd6485f8740 1 -- 172.21.15.105:0/1059831889 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03823a5f0 con 0x55d0382dde60 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.573+0000 7fd63effd640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0382dde60 0x55d0382e0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.573+0000 7fd61effd640 10 client.32011 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.573+0000 7fd63effd640 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0382dde60 0x55d0382e0250 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f548bbe79f319527 server_cookie=19d868f1b642881e in_seq=0 out_seq=0 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd61effd640 1 -- 172.21.15.105:0/1059831889 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d03823a5f0 con 0x55d0382dde60 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd61effd640 10 client.32011 handle_command_reply: tid=0 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 client.32011 shutdown 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 -- 172.21.15.105:0/1059831889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 msgr2=0x55d0382d9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d0382d9970 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd6380a8e20 tx=0x7fd6380acb70 comp rx=0 tx=0).stop 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 -- 172.21.15.105:0/1059831889 shutdown_connections 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d0382dde60 0x55d0382e0250 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d0382d9970 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 --2- 172.21.15.105:0/1059831889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd61effd640 0 client.32011 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 -- 172.21.15.105:0/1059831889 >> 172.21.15.105:0/1059831889 conn(0x55d0382c3b40 msgr2=0x55d0382c2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 -- 172.21.15.105:0/1059831889 shutdown_connections 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.575+0000 7fd6485f8740 1 -- 172.21.15.105:0/1059831889 wait complete. 2024-03-26T17:57:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.576+0000 7fd6485f8740 20 client.32011 trim_cache size 0 max 16384 2024-03-26T17:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.598+0000 7fd6485f8740 1 Processor -- start 2024-03-26T17:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.598+0000 7fd6485f8740 1 -- start start 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.598+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.598+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c8fc0 0x55d0382d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.598+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382ce130 0x55d0382cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd6485f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd640070600 con 0x55d0382ce130 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd6485f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd640072140 con 0x55d0382c85d0 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd6485f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd640070cd0 con 0x55d0382c8fc0 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c8fc0 0x55d0382d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382ce130 0x55d0382cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c8fc0 0x55d0382d9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56856/0 (socket says 172.21.15.105:56856) 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382e5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60564/0 (socket says 172.21.15.105:60564) 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63e7fc640 1 -- 172.21.15.105:0/2442862524 learned_addr learned my addr 172.21.15.105:0/2442862524 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63effd640 1 -- 172.21.15.105:0/2442862524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c8fc0 msgr2=0x55d0382d9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63effd640 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c8fc0 0x55d0382d9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63effd640 1 -- 172.21.15.105:0/2442862524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382ce130 msgr2=0x55d0382cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63effd640 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382ce130 0x55d0382cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63effd640 1 -- 172.21.15.105:0/2442862524 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0382e05c0 con 0x55d0382c85d0 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c8fc0 0x55d0382d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382ce130 0x55d0382cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd63effd640 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382e5a30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55d0382e09a0 tx=0x7fd6300324c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4a8fae3514e726 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.599+0000 7fd6257fa640 1 -- 172.21.15.105:0/2442862524 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630051070 con 0x55d0382c85d0 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.600+0000 7fd6257fa640 1 -- 172.21.15.105:0/2442862524 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd630004040 con 0x55d0382c85d0 2024-03-26T17:57:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.600+0000 7fd6257fa640 1 -- 172.21.15.105:0/2442862524 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630039040 con 0x55d0382c85d0 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 -- 172.21.15.105:0/2442862524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 msgr2=0x55d0382e5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382e5a30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55d0382e09a0 tx=0x7fd6300324c0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 -- 172.21.15.105:0/2442862524 shutdown_connections 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382ce130 0x55d0382cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382c8fc0 0x55d0382d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2442862524 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382e5a30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 -- 172.21.15.105:0/2442862524 >> 172.21.15.105:0/2442862524 conn(0x55d0382c3b40 msgr2=0x55d0382d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 -- 172.21.15.105:0/2442862524 shutdown_connections 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 -- 172.21.15.105:0/2442862524 wait complete. 2024-03-26T17:57:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 -- start start 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.601+0000 7fd6485f8740 1 Processor -- start 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd6485f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d038342050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd6485f8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd640071a70 con 0x55d0382c8fc0 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd6485f8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd6400713a0 con 0x55d0382c85d0 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd6485f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd64006e5e0 con 0x55d0382ce130 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd6485f8740 10 client.0 _ll_register_callbacks cb 0x7fd634009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45140/0 (socket says 172.21.15.105:45140) 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 -- 172.21.15.105:0/2606043451 learned_addr learned my addr 172.21.15.105:0/2606043451 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d038342050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 -- 172.21.15.105:0/2606043451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 msgr2=0x55d038342050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d038342050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 -- 172.21.15.105:0/2606043451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 msgr2=0x55d0382d6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 -- 172.21.15.105:0/2606043451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0382d9b10 con 0x55d0382c8fc0 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63f7fe640 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d038342050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63effd640 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd63e7fc640 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d6a40 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd634013750 tx=0x7fd634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f49df30d8c60e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.602+0000 7fd61effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.603+0000 7fd61effd640 1 -- 172.21.15.105:0/2606043451 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6340044c0 con 0x55d0382c8fc0 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.603+0000 7fd61effd640 1 -- 172.21.15.105:0/2606043451 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd634003b40 con 0x55d0382c8fc0 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.603+0000 7fd6485f8740 1 -- 172.21.15.105:0/2606043451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd630004040 con 0x55d0382c8fc0 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.603+0000 7fd61effd640 1 -- 172.21.15.105:0/2606043451 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd634003d40 con 0x55d0382c8fc0 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.603+0000 7fd6485f8740 10 client.22335 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.603+0000 7fd6485f8740 1 -- 172.21.15.105:0/2606043451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d0383435e0 con 0x55d0382c8fc0 2024-03-26T17:57:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.603+0000 7fd61effd640 1 -- 172.21.15.105:0/2606043451 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd630004040 con 0x55d0382c8fc0 2024-03-26T17:57:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.605+0000 7fd61effd640 1 -- 172.21.15.105:0/2606043451 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd63401f040 con 0x55d0382c8fc0 2024-03-26T17:57:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.605+0000 7fd6485f8740 10 client.22335 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.605+0000 7fd6485f8740 10 client.22335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.605+0000 7fd6485f8740 20 client.22335 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.605+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d038343f80 0x55d0383463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.605+0000 7fd6485f8740 4 client.22335 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.605+0000 7fd6485f8740 1 -- 172.21.15.105:0/2606043451 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d038234940 con 0x55d038343f80 2024-03-26T17:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.605+0000 7fd63effd640 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d038343f80 0x55d0383463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.608+0000 7fd63effd640 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d038343f80 0x55d0383463a0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed425c7d0a8678a3 server_cookie=17d4c8904624455e in_seq=0 out_seq=0 2024-03-26T17:57:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.608+0000 7fd61effd640 10 client.22335 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1214184182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e541018f0 msgr2=0x7f3e54101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1214184182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e541018f0 0x7f3e54101d10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009900 tx=0x7f3e5002bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1214184182 shutdown_connections 2024-03-26T17:57:13.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1214184182 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e541034a0 0x7f3e54109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1214184182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e54102ae0 0x7f3e54102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1214184182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e541018f0 0x7f3e54101d10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1214184182 >> 172.21.15.105:0/1214184182 conn(0x7f3e540fce60 msgr2=0x7f3e540ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1214184182 shutdown_connections 2024-03-26T17:57:13.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.632+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1214184182 wait complete. 2024-03-26T17:57:13.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.633+0000 7f3e5bd90640 1 Processor -- start 2024-03-26T17:57:13.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.633+0000 7f3e5bd90640 1 -- start start 2024-03-26T17:57:13.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5bd90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e541018f0 0x7f3e54197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5bd90640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e54102ae0 0x7f3e54198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5bd90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e541034a0 0x7f3e5419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5bd90640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5410d410 con 0x7f3e54102ae0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5bd90640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5410d290 con 0x7f3e541018f0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5bd90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5410d590 con 0x7f3e541034a0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e54102ae0 0x7f3e54198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e59b2c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e541018f0 0x7f3e54197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e54102ae0 0x7f3e54198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45148/0 (socket says 172.21.15.105:45148) 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 -- 172.21.15.105:0/1096485684 learned_addr learned my addr 172.21.15.105:0/1096485684 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5a32d640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e541034a0 0x7f3e5419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 -- 172.21.15.105:0/1096485684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e541034a0 msgr2=0x7f3e5419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e541034a0 0x7f3e5419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 -- 172.21.15.105:0/1096485684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e541018f0 msgr2=0x7f3e54197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e541018f0 0x7f3e54197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 -- 172.21.15.105:0/1096485684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5419f770 con 0x7f3e54102ae0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e541018f0 0x7f3e54197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.634+0000 7f3e5932b640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e54102ae0 0x7f3e54198090 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400e9f0 tx=0x7f3e4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.635+0000 7f3e4affd640 1 -- 172.21.15.105:0/1096485684 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400cd80 con 0x7f3e54102ae0 2024-03-26T17:57:13.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.635+0000 7f3e4affd640 1 -- 172.21.15.105:0/1096485684 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e440054d0 con 0x7f3e54102ae0 2024-03-26T17:57:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.635+0000 7f3e4affd640 1 -- 172.21.15.105:0/1096485684 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44010490 con 0x7f3e54102ae0 2024-03-26T17:57:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.635+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1096485684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e54073b50 con 0x7f3e54102ae0 2024-03-26T17:57:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.635+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1096485684 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e54074050 con 0x7f3e54102ae0 2024-03-26T17:57:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.637+0000 7f3e4affd640 1 -- 172.21.15.105:0/1096485684 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3e44005020 con 0x7f3e54102ae0 2024-03-26T17:57:13.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.638+0000 7f3e4affd640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e400a30f0 0x7f3e400a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.638+0000 7f3e4affd640 1 -- 172.21.15.105:0/1096485684 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f3e440cf6f0 con 0x7f3e54102ae0 2024-03-26T17:57:13.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.638+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e400a30f0 0x7f3e400a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.638+0000 7f3e5e97f740 1 Processor -- start 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5e97f740 1 -- start start 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e400a30f0 0x7f3e400a55b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3e50005ab0 tx=0x7f3e50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56328f0 0x55bea5632d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56332e0 0x55bea5637de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bea5597980 con 0x55bea56332e0 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bea553fbe0 con 0x55bea56328f0 2024-03-26T17:57:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bea5593930 con 0x55bea5638450 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5932b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56332e0 0x55bea5637de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5932b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56332e0 0x55bea5637de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45160/0 (socket says 172.21.15.105:45160) 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5932b640 1 -- 172.21.15.105:0/2859661335 learned_addr learned my addr 172.21.15.105:0/2859661335 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56328f0 0x55bea5632d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5a32d640 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5932b640 1 -- 172.21.15.105:0/2859661335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 msgr2=0x55bea563a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5932b640 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5932b640 1 -- 172.21.15.105:0/2859661335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56328f0 msgr2=0x55bea5632d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.639+0000 7f3e5932b640 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56328f0 0x55bea5632d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.640+0000 7f3e5932b640 1 -- 172.21.15.105:0/2859661335 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea53757d0 con 0x55bea56332e0 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.640+0000 7f3e5932b640 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56332e0 0x55bea5637de0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3e440981e0 tx=0x7f3e44005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef38da57be2dab4f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.640+0000 7f3e3ffff640 1 -- 172.21.15.105:0/2859661335 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4409bd80 con 0x55bea56332e0 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.640+0000 7f3e3ffff640 1 -- 172.21.15.105:0/2859661335 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e4409fed0 con 0x55bea56332e0 2024-03-26T17:57:13.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.640+0000 7f3e3ffff640 1 -- 172.21.15.105:0/2859661335 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440a4300 con 0x55bea56332e0 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 -- 172.21.15.105:0/2859661335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56332e0 msgr2=0x55bea5637de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56332e0 0x55bea5637de0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3e440981e0 tx=0x7f3e44005310 comp rx=0 tx=0).stop 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 -- 172.21.15.105:0/2859661335 shutdown_connections 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56332e0 0x55bea5637de0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/2859661335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56328f0 0x55bea5632d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 -- 172.21.15.105:0/2859661335 >> 172.21.15.105:0/2859661335 conn(0x55bea562de60 msgr2=0x55bea562e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 -- 172.21.15.105:0/2859661335 shutdown_connections 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 -- 172.21.15.105:0/2859661335 wait complete. 2024-03-26T17:57:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.641+0000 7f3e5e97f740 1 -- start start 2024-03-26T17:57:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5e97f740 1 Processor -- start 2024-03-26T17:57:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea563ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea5644790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bea563b400 con 0x55bea56328f0 2024-03-26T17:57:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bea5553160 con 0x55bea56423a0 2024-03-26T17:57:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bea563b580 con 0x55bea5638450 2024-03-26T17:57:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5e97f740 10 client.0 _ll_register_callbacks cb 0x55bea558f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5932b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5a32d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea5644790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5932b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56896/0 (socket says 172.21.15.105:56896) 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e5932b640 1 -- 172.21.15.105:0/117410252 learned_addr learned my addr 172.21.15.105:0/117410252 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.642+0000 7f3e59b2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea563ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e5932b640 1 -- 172.21.15.105:0/117410252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 msgr2=0x55bea5644790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e5932b640 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea5644790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e5932b640 1 -- 172.21.15.105:0/117410252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 msgr2=0x55bea563ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e5932b640 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea563ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e5932b640 1 -- 172.21.15.105:0/117410252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea52ab3f0 con 0x55bea5638450 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea563ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e5a32d640 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea5644790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e5932b640 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563d370 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3e44098190 tx=0x7f3e44004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ced09f5089f9beb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/117410252 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440a9060 con 0x55bea5638450 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.643+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/117410252 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f3e4409b1d0 con 0x55bea5638450 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 1 -- 172.21.15.105:0/117410252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e4409fed0 con 0x55bea5638450 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/117410252 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3e440a4790 con 0x55bea5638450 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 10 client.32026 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 1 -- 172.21.15.105:0/117410252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bea526b0f0 con 0x55bea5638450 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/117410252 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e4409b410 con 0x55bea5638450 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/117410252 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3e440ba070 con 0x55bea5638450 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 10 client.32026 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 10 client.32026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 20 client.32026 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bea5648c60 0x55bea564b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 4 client.32026 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e5e97f740 1 -- 172.21.15.105:0/117410252 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3e4409fed0 con 0x55bea5648c60 2024-03-26T17:57:13.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.644+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bea5648c60 0x55bea564b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.645+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bea5648c60 0x55bea564b050 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f501f1a2d1505f0 server_cookie=3e260979edd8ee50 in_seq=0 out_seq=0 2024-03-26T17:57:13.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.645+0000 7f3e3d7fa640 10 client.32026 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.806+0000 7fd61effd640 1 -- 172.21.15.105:0/2606043451 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d038234940 con 0x55d038343f80 2024-03-26T17:57:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.806+0000 7fd61effd640 10 client.22335 handle_command_reply: tid=0 2024-03-26T17:57:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.806+0000 7fd6485f8740 1 client.22335 shutdown 2024-03-26T17:57:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd6485f8740 1 -- 172.21.15.105:0/2606043451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 msgr2=0x55d0382d6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d6a40 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd634013750 tx=0x7fd634004730 comp rx=0 tx=0).stop 2024-03-26T17:57:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd6485f8740 1 -- 172.21.15.105:0/2606043451 shutdown_connections 2024-03-26T17:57:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d038343f80 0x55d0383463a0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0382ce130 0x55d038342050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0382c8fc0 0x55d0382d6a40 secure :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd634013750 tx=0x7fd634004730 comp rx=0 tx=0).stop 2024-03-26T17:57:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd6485f8740 1 --2- 172.21.15.105:0/2606043451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0382c85d0 0x55d0382d6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd6485f8740 1 -- 172.21.15.105:0/2606043451 >> 172.21.15.105:0/2606043451 conn(0x55d0382c3b40 msgr2=0x55d0382cd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.807+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/117410252 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f3e4409fed0 con 0x55bea5648c60 2024-03-26T17:57:13.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.807+0000 7f3e3d7fa640 10 client.32026 handle_command_reply: tid=0 2024-03-26T17:57:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.807+0000 7fd61effd640 0 client.22335 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.808+0000 7fd6485f8740 1 -- 172.21.15.105:0/2606043451 shutdown_connections 2024-03-26T17:57:13.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.807+0000 7f3e5e97f740 1 client.32026 shutdown 2024-03-26T17:57:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.808+0000 7fd6485f8740 1 -- 172.21.15.105:0/2606043451 wait complete. 2024-03-26T17:57:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.808+0000 7fd6485f8740 20 client.22335 trim_cache size 0 max 16384 2024-03-26T17:57:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:57:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:57:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:57:13.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e5e97f740 1 -- 172.21.15.105:0/117410252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 msgr2=0x55bea563d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563d370 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3e44098190 tx=0x7f3e44004990 comp rx=0 tx=0).stop 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e5e97f740 1 -- 172.21.15.105:0/117410252 shutdown_connections 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bea5648c60 0x55bea564b050 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea5644790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea563d370 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/117410252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea563ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e5e97f740 1 -- 172.21.15.105:0/117410252 >> 172.21.15.105:0/117410252 conn(0x55bea562de60 msgr2=0x55bea562d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.809+0000 7f3e3d7fa640 0 client.32026 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.809+0000 7fd645937640 1 -- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6280a3040 msgr2=0x7fd6280a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.809+0000 7fd645937640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6280a3040 0x7fd6280a5500 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd64019b180 tx=0x7fd634009bf0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.809+0000 7fd645937640 1 -- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6401075c0 msgr2=0x7fd6401a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.809+0000 7fd645937640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6401075c0 0x7fd6401a1220 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd638009e70 tx=0x7fd63800ed10 comp rx=0 tx=0).stop 2024-03-26T17:57:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.810+0000 7fd645937640 1 -- 172.21.15.105:0/1673453106 shutdown_connections 2024-03-26T17:57:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.810+0000 7fd645937640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd6280a3040 0x7fd6280a5500 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.810+0000 7fd645937640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6401075c0 0x7fd6401a1220 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.810+0000 7fd645937640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd640104c90 0x7fd64019a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.810+0000 7fd645937640 1 --2- 172.21.15.105:0/1673453106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd640102360 0x7fd640199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.810+0000 7fd645937640 1 -- 172.21.15.105:0/1673453106 >> 172.21.15.105:0/1673453106 conn(0x7fd6400fbd50 msgr2=0x7fd640101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.810+0000 7f3e5e97f740 1 -- 172.21.15.105:0/117410252 shutdown_connections 2024-03-26T17:57:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.810+0000 7fd645937640 1 -- 172.21.15.105:0/1673453106 shutdown_connections 2024-03-26T17:57:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.810+0000 7f3e5e97f740 1 -- 172.21.15.105:0/117410252 wait complete. 2024-03-26T17:57:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:13.810+0000 7fd645937640 1 -- 172.21.15.105:0/1673453106 wait complete. 2024-03-26T17:57:13.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.810+0000 7f3e5e97f740 20 client.32026 trim_cache size 0 max 16384 2024-03-26T17:57:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e5e97f740 1 Processor -- start 2024-03-26T17:57:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e5e97f740 1 -- start start 2024-03-26T17:57:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5628e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea5638450 0x55bea564ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea56423a0 0x55bea564b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e540725c0 con 0x55bea56328f0 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5419fef0 con 0x55bea5638450 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54072c90 con 0x55bea56423a0 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e59b2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5628e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e59b2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5628e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45184/0 (socket says 172.21.15.105:45184) 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.839+0000 7f3e59b2c640 1 -- 172.21.15.105:0/1579274135 learned_addr learned my addr 172.21.15.105:0/1579274135 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e5932b640 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea5638450 0x55bea564ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e5a32d640 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea56423a0 0x55bea564b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e59b2c640 1 -- 172.21.15.105:0/1579274135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea56423a0 msgr2=0x55bea564b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea56423a0 0x55bea564b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e59b2c640 1 -- 172.21.15.105:0/1579274135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea5638450 msgr2=0x55bea564ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea5638450 0x55bea564ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e59b2c640 1 -- 172.21.15.105:0/1579274135 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea56298d0 con 0x55bea56328f0 2024-03-26T17:57:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5628e20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009230 tx=0x7f3e50045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6221101348ca0b64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e3f7fe640 1 -- 172.21.15.105:0/1579274135 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50005100 con 0x55bea56328f0 2024-03-26T17:57:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e3f7fe640 1 -- 172.21.15.105:0/1579274135 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e500046c0 con 0x55bea56328f0 2024-03-26T17:57:13.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.840+0000 7f3e3f7fe640 1 -- 172.21.15.105:0/1579274135 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50058290 con 0x55bea56328f0 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 -- 172.21.15.105:0/1579274135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 msgr2=0x55bea5628e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5628e20 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009230 tx=0x7f3e50045000 comp rx=0 tx=0).stop 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 -- 172.21.15.105:0/1579274135 shutdown_connections 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea56423a0 0x55bea564b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea5638450 0x55bea564ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/1579274135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5628e20 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 -- 172.21.15.105:0/1579274135 >> 172.21.15.105:0/1579274135 conn(0x55bea562de60 msgr2=0x55bea563b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 -- 172.21.15.105:0/1579274135 shutdown_connections 2024-03-26T17:57:13.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.841+0000 7f3e5e97f740 1 -- 172.21.15.105:0/1579274135 wait complete. 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 1 -- start start 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 1 Processor -- start 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5659a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea5659f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea565af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5419f940 con 0x55bea56328f0 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e54073360 con 0x55bea56423a0 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e540705a0 con 0x55bea5638450 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5e97f740 10 client.0 _ll_register_callbacks cb 0x7f3e44005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e59b2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5659a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e59b2c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5659a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45194/0 (socket says 172.21.15.105:45194) 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5932b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea5659f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e59b2c640 1 -- 172.21.15.105:0/3635197645 learned_addr learned my addr 172.21.15.105:0/3635197645 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.842+0000 7f3e5a32d640 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea565af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5932b640 1 -- 172.21.15.105:0/3635197645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 msgr2=0x55bea565af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5932b640 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea565af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5932b640 1 -- 172.21.15.105:0/3635197645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 msgr2=0x55bea5659a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5932b640 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5659a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5932b640 1 -- 172.21.15.105:0/3635197645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea565b580 con 0x55bea5638450 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5659a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5932b640 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea5659f80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3e4409a030 tx=0x7f3e44005310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c90161b5cf6c4fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/3635197645 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440a9070 con 0x55bea5638450 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5e97f740 1 -- 172.21.15.105:0/3635197645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e500046c0 con 0x55bea5638450 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/3635197645 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f3e44086d30 con 0x55bea5638450 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/3635197645 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3e4409fed0 con 0x55bea5638450 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5e97f740 10 client.32032 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.843+0000 7f3e5e97f740 1 -- 172.21.15.105:0/3635197645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bea56aeb20 con 0x55bea5638450 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/3635197645 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e500046c0 con 0x55bea5638450 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/3635197645 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3e440ba040 con 0x55bea5638450 2024-03-26T17:57:13.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e5e97f740 10 client.32032 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e5e97f740 10 client.32032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e5e97f740 20 client.32032 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bea56af230 0x55bea56b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e5e97f740 4 client.32032 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:57:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e5e97f740 1 -- 172.21.15.105:0/3635197645 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bea55a4ec0 con 0x55bea56af230 2024-03-26T17:57:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.844+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bea56af230 0x55bea56b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.845+0000 7f3e59b2c640 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bea56af230 0x55bea56b1650 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3030a713538ffade server_cookie=bbc837b53c64c8c6 in_seq=0 out_seq=0 2024-03-26T17:57:13.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.845+0000 7f3e3d7fa640 10 client.32032 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.846+0000 7f3e3d7fa640 1 -- 172.21.15.105:0/3635197645 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55bea55a4ec0 con 0x55bea56af230 2024-03-26T17:57:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.846+0000 7f3e3d7fa640 10 client.32032 handle_command_reply: tid=0 2024-03-26T17:57:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.846+0000 7f3e5e97f740 1 client.32032 shutdown 2024-03-26T17:57:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 -- 172.21.15.105:0/3635197645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 msgr2=0x55bea5659f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea5659f80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3e4409a030 tx=0x7f3e44005310 comp rx=0 tx=0).stop 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 -- 172.21.15.105:0/3635197645 shutdown_connections 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bea56af230 0x55bea56b1650 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bea56423a0 0x55bea565af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bea5638450 0x55bea5659f80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 --2- 172.21.15.105:0/3635197645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bea56328f0 0x55bea5659a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e3d7fa640 0 client.32032 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 -- 172.21.15.105:0/3635197645 >> 172.21.15.105:0/3635197645 conn(0x55bea562de60 msgr2=0x55bea563b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 -- 172.21.15.105:0/3635197645 shutdown_connections 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 1 -- 172.21.15.105:0/3635197645 wait complete. 2024-03-26T17:57:13.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5e97f740 20 client.32032 trim_cache size 0 max 16384 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (116 inodes in the stack)", 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout: "9b0e103a-07f0-45dc-b4e9-9695f7c8e921": { 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "9b0e103a-07f0-45dc-b4e9-9695f7c8e921", 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:57:13.836 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e400a30f0 msgr2=0x7f3e400a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e400a30f0 0x7f3e400a55b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3e50005ab0 tx=0x7f3e50005fd0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e54102ae0 msgr2=0x7f3e54198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e54102ae0 0x7f3e54198090 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400e9f0 tx=0x7f3e4400edf0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1096485684 shutdown_connections 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3e400a30f0 0x7f3e400a55b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e541034a0 0x7f3e5419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e54102ae0 0x7f3e54198090 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 --2- 172.21.15.105:0/1096485684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e541018f0 0x7f3e54197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.847+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1096485684 >> 172.21.15.105:0/1096485684 conn(0x7f3e540fce60 msgr2=0x7f3e54105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:13.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.848+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1096485684 shutdown_connections 2024-03-26T17:57:13.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:13.848+0000 7f3e5bd90640 1 -- 172.21.15.105:0/1096485684 wait complete. 2024-03-26T17:57:13.846 INFO:tasks.cephfs.filesystem:scrub status for tag:9b0e103a-07f0-45dc-b4e9-9695f7c8e921 - {'path': '/', 'tag': '9b0e103a-07f0-45dc-b4e9-9695f7c8e921', 'options': 'recursive,force'} 2024-03-26T17:57:14.134 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:13 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:57:14.135 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:13 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:14.135 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:13 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:14.135 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:13 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:14.135 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:13 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:14.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:13 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:57:14.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:13 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:14.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:13 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:14.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:13 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:14.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:13 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:14.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:13 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:57:14.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:13 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:14.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:13 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:14.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:13 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:14.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:13 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:14.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 46901 7.07 MB/sec execute 237 sec latency 221.162 ms 2024-03-26T17:57:15.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:15.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:15.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:14 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:15.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:14 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:14 smithi144 ceph-mon[36095]: pgmap v917: 97 pgs: 97 active+clean; 303 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-26T17:57:15.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:15.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:15.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:14 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:15.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:14 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.386 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:14 smithi105 ceph-mon[32280]: pgmap v917: 97 pgs: 97 active+clean; 303 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-26T17:57:15.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:15.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:15.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:14 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:15.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:14 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:15.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:14 smithi135 ceph-mon[35636]: pgmap v917: 97 pgs: 97 active+clean; 303 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 3.1 MiB/s wr, 253 op/s 2024-03-26T17:57:15.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 47027 7.06 MB/sec execute 238 sec latency 111.816 ms 2024-03-26T17:57:16.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:16 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:16.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 47178 7.08 MB/sec execute 239 sec latency 147.541 ms 2024-03-26T17:57:16.591 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:16 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:16.634 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:16 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:57:17.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 47330 7.09 MB/sec execute 240 sec latency 151.555 ms 2024-03-26T17:57:17.591 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:17 smithi144 ceph-mon[36095]: pgmap v918: 97 pgs: 97 active+clean; 300 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-03-26T17:57:17.634 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:17 smithi105 ceph-mon[32280]: pgmap v918: 97 pgs: 97 active+clean; 300 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-03-26T17:57:17.658 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:17 smithi135 ceph-mon[35636]: pgmap v918: 97 pgs: 97 active+clean; 300 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-03-26T17:57:18.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 47450 7.08 MB/sec execute 241 sec latency 184.966 ms 2024-03-26T17:57:19.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 47575 7.06 MB/sec execute 242 sec latency 184.236 ms 2024-03-26T17:57:20.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:19 smithi135 ceph-mon[35636]: pgmap v919: 97 pgs: 97 active+clean; 296 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-03-26T17:57:20.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:20.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:20.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:57:20.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:19 smithi144 ceph-mon[36095]: pgmap v919: 97 pgs: 97 active+clean; 296 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-03-26T17:57:20.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:20.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:20.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:57:20.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:19 smithi105 ceph-mon[32280]: pgmap v919: 97 pgs: 97 active+clean; 296 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-03-26T17:57:20.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:57:20.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:57:20.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:57:20.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 47744 7.06 MB/sec execute 243 sec latency 119.087 ms 2024-03-26T17:57:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:20 smithi144 ceph-mon[36095]: pgmap v920: 97 pgs: 97 active+clean; 297 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-03-26T17:57:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:20 smithi105 ceph-mon[32280]: pgmap v920: 97 pgs: 97 active+clean; 297 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-03-26T17:57:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:20 smithi135 ceph-mon[35636]: pgmap v920: 97 pgs: 97 active+clean; 297 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-03-26T17:57:21.554 INFO:tasks.workunit.client.0.smithi105.stdout: 10 47874 7.05 MB/sec execute 244 sec latency 114.379 ms 2024-03-26T17:57:22.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 48031 7.05 MB/sec execute 245 sec latency 116.312 ms 2024-03-26T17:57:23.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 48178 7.05 MB/sec execute 246 sec latency 104.278 ms 2024-03-26T17:57:24.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:23 smithi135 ceph-mon[35636]: pgmap v921: 97 pgs: 97 active+clean; 296 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-03-26T17:57:24.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:23 smithi144 ceph-mon[36095]: pgmap v921: 97 pgs: 97 active+clean; 296 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-03-26T17:57:24.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:23 smithi105 ceph-mon[32280]: pgmap v921: 97 pgs: 97 active+clean; 296 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-03-26T17:57:24.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 48292 7.04 MB/sec execute 247 sec latency 111.519 ms 2024-03-26T17:57:25.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:24 smithi144 ceph-mon[36095]: pgmap v922: 97 pgs: 97 active+clean; 299 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-03-26T17:57:25.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:24 smithi105 ceph-mon[32280]: pgmap v922: 97 pgs: 97 active+clean; 299 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-03-26T17:57:25.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:24 smithi135 ceph-mon[35636]: pgmap v922: 97 pgs: 97 active+clean; 299 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-03-26T17:57:25.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 48433 7.04 MB/sec execute 248 sec latency 117.426 ms 2024-03-26T17:57:26.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 48555 7.04 MB/sec execute 249 sec latency 113.753 ms 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 -- 172.21.15.105:0/2923860263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 msgr2=0x7fdac0100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 --2- 172.21.15.105:0/2923860263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 0x7fdac0100c00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009900 tx=0x7fdab002bc40 comp rx=0 tx=0).stop 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 -- 172.21.15.105:0/2923860263 shutdown_connections 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 --2- 172.21.15.105:0/2923860263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdac0102390 0x7fdac0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 --2- 172.21.15.105:0/2923860263 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdac01019d0 0x7fdac0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 --2- 172.21.15.105:0/2923860263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 0x7fdac0100c00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 -- 172.21.15.105:0/2923860263 >> 172.21.15.105:0/2923860263 conn(0x7fdac00fbd50 msgr2=0x7fdac00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 -- 172.21.15.105:0/2923860263 shutdown_connections 2024-03-26T17:57:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.937+0000 7fdac4c75640 1 -- 172.21.15.105:0/2923860263 wait complete. 2024-03-26T17:57:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdac4c75640 1 Processor -- start 2024-03-26T17:57:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdac4c75640 1 -- start start 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdac4c75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 0x7fdac0069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdac4c75640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdac01019d0 0x7fdac019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdac4c75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdac0102390 0x7fdac019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdac4c75640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010c2e0 con 0x7fdac0102390 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdac4c75640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010c160 con 0x7fdac01019d0 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdac4c75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010c460 con 0x7fdac01007e0 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdac0102390 0x7fdac019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdac0102390 0x7fdac019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42360/0 (socket says 172.21.15.105:42360) 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdabed9d640 1 -- 172.21.15.105:0/1740650659 learned_addr learned my addr 172.21.15.105:0/1740650659 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdab7fff640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdac01019d0 0x7fdac019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.938+0000 7fdabe59c640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 0x7fdac0069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdabe59c640 1 -- 172.21.15.105:0/1740650659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdac01019d0 msgr2=0x7fdac019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdabe59c640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdac01019d0 0x7fdac019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdabe59c640 1 -- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdac0102390 msgr2=0x7fdac019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdabe59c640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdac0102390 0x7fdac019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdabe59c640 1 -- 172.21.15.105:0/1740650659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac019f6b0 con 0x7fdac01007e0 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdabed9d640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdac0102390 0x7fdac019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdabe59c640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 0x7fdac0069ad0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdab0036040 tx=0x7fdab0005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdab77fe640 1 -- 172.21.15.105:0/1740650659 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0043020 con 0x7fdac01007e0 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdab77fe640 1 -- 172.21.15.105:0/1740650659 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab0004350 con 0x7fdac01007e0 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdac4c75640 1 -- 172.21.15.105:0/1740650659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac019f940 con 0x7fdac01007e0 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdab77fe640 1 -- 172.21.15.105:0/1740650659 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0003ab0 con 0x7fdac01007e0 2024-03-26T17:57:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.939+0000 7fdac4c75640 1 -- 172.21.15.105:0/1740650659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac019fe60 con 0x7fdac01007e0 2024-03-26T17:57:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.942+0000 7fdab77fe640 1 -- 172.21.15.105:0/1740650659 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdab0046020 con 0x7fdac01007e0 2024-03-26T17:57:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdac785b740 1 Processor -- start 2024-03-26T17:57:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdac785b740 1 -- start start 2024-03-26T17:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdac785b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96c8a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdac785b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c8fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdac785b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 0x561e96c925c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdac785b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e96bef0b0 con 0x561e96c8a5d0 2024-03-26T17:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdac785b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e96b97960 con 0x561e96c90130 2024-03-26T17:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdac785b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e96beb060 con 0x561e96c8afc0 2024-03-26T17:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96c8a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96c8a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42364/0 (socket says 172.21.15.105:42364) 2024-03-26T17:57:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdabed9d640 1 -- 172.21.15.105:0/4136056118 learned_addr learned my addr 172.21.15.105:0/4136056118 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdabe59c640 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c8fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdab77fe640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdaac0a2fc0 0x7fdaac0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.943+0000 7fdab7fff640 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 0x561e96c925c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdabed9d640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdaac0a2fc0 0x7fdaac0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdabe59c640 1 -- 172.21.15.105:0/4136056118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 msgr2=0x561e96c925c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdabe59c640 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 0x561e96c925c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdabe59c640 1 -- 172.21.15.105:0/4136056118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 msgr2=0x561e96c8a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdabe59c640 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96c8a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdab77fe640 1 -- 172.21.15.105:0/1740650659 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fdab00f1530 con 0x7fdac01007e0 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdabe59c640 1 -- 172.21.15.105:0/4136056118 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e96ba9c60 con 0x561e96c8afc0 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdabe59c640 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c8fac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdab00ef040 tx=0x7fdab00b9ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158fd35dd4bc8af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdab4ff9640 1 -- 172.21.15.105:0/4136056118 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab00bd9f0 con 0x561e96c8afc0 2024-03-26T17:57:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdab4ff9640 1 -- 172.21.15.105:0/4136056118 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab00be320 con 0x561e96c8afc0 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.944+0000 7fdab4ff9640 1 -- 172.21.15.105:0/4136056118 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab00bd580 con 0x561e96c8afc0 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 -- 172.21.15.105:0/4136056118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 msgr2=0x561e96c8fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdabed9d640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdaac0a2fc0 0x7fdaac0a5480 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x561e96c8b820 tx=0x7fdab8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c8fac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdab00ef040 tx=0x7fdab00b9ff0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 -- 172.21.15.105:0/4136056118 shutdown_connections 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 0x561e96c925c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c8fac0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 --2- 172.21.15.105:0/4136056118 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96c8a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 -- 172.21.15.105:0/4136056118 >> 172.21.15.105:0/4136056118 conn(0x561e96c85b40 msgr2=0x561e96c85f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 -- 172.21.15.105:0/4136056118 shutdown_connections 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 -- 172.21.15.105:0/4136056118 wait complete. 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.945+0000 7fdac785b740 1 -- start start 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdac785b740 1 Processor -- start 2024-03-26T17:57:26.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdac785b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8a5d0 0x561e96c8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdac785b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8afc0 0x561e96c8d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdac785b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 0x561e96c8e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdac785b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e96c92c80 con 0x561e96c8afc0 2024-03-26T17:57:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdac785b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e96baadb0 con 0x561e96c90130 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdac785b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e96c92e00 con 0x561e96c8a5d0 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdab7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8afc0 0x561e96c8d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdab7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8afc0 0x561e96c8d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42368/0 (socket says 172.21.15.105:42368) 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdab7fff640 1 -- 172.21.15.105:0/3464504862 learned_addr learned my addr 172.21.15.105:0/3464504862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdac785b740 10 client.0 _ll_register_callbacks cb 0x561e96baf110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdabe59c640 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8a5d0 0x561e96c8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdabed9d640 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 0x561e96c8e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdab7fff640 1 -- 172.21.15.105:0/3464504862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8a5d0 msgr2=0x561e96c8ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdab7fff640 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8a5d0 0x561e96c8ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdab7fff640 1 -- 172.21.15.105:0/3464504862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 msgr2=0x561e96c8e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdab7fff640 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 0x561e96c8e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.946+0000 7fdab7fff640 1 -- 172.21.15.105:0/3464504862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e96903210 con 0x561e96c8afc0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdab7fff640 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8afc0 0x561e96c8d370 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8012700 tx=0x7fdaa8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d233b14db6af5101 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3464504862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800f8c0 con 0x561e96c8afc0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdac785b740 1 -- 172.21.15.105:0/3464504862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab00be320 con 0x561e96c8afc0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3464504862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fdaa800fa60 con 0x561e96c8afc0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3464504862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdaa800fc60 con 0x561e96c8afc0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdac785b740 10 client.22365 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdac785b740 1 -- 172.21.15.105:0/3464504862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561e969cf620 con 0x561e96c8afc0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3464504862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab00be320 con 0x561e96c8afc0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.947+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3464504862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdaa8010430 con 0x561e96c8afc0 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdac785b740 10 client.22365 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdac785b740 10 client.22365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdac785b740 20 client.22365 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdac785b740 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e96c9ffb0 0x561e96ca2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdac785b740 4 client.22365 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdac785b740 1 -- 172.21.15.105:0/3464504862 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e96bfc5f0 con 0x561e96c9ffb0 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdabe59c640 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e96c9ffb0 0x561e96ca2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdabe59c640 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e96c9ffb0 0x561e96ca2400 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=544dc14328608370 server_cookie=e5a5fdd5d2e35de9 in_seq=0 out_seq=0 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.948+0000 7fdaa5ffb640 10 client.22365 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.949+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3464504862 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561e96bfc5f0 con 0x561e96c9ffb0 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.949+0000 7fdaa5ffb640 10 client.22365 handle_command_reply: tid=0 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 client.22365 shutdown 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 -- 172.21.15.105:0/3464504862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8afc0 msgr2=0x561e96c8d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8afc0 0x561e96c8d370 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8012700 tx=0x7fdaa8012bc0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 -- 172.21.15.105:0/3464504862 shutdown_connections 2024-03-26T17:57:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e96c9ffb0 0x561e96ca2400 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c90130 0x561e96c8e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8afc0 0x561e96c8d370 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 --2- 172.21.15.105:0/3464504862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8a5d0 0x561e96c8ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 -- 172.21.15.105:0/3464504862 >> 172.21.15.105:0/3464504862 conn(0x561e96c85b40 msgr2=0x561e96c84df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdaa5ffb640 0 client.22365 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 -- 172.21.15.105:0/3464504862 shutdown_connections 2024-03-26T17:57:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 1 -- 172.21.15.105:0/3464504862 wait complete. 2024-03-26T17:57:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.950+0000 7fdac785b740 20 client.22365 trim_cache size 0 max 16384 2024-03-26T17:57:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.973+0000 7fdac785b740 1 Processor -- start 2024-03-26T17:57:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.973+0000 7fdac785b740 1 -- start start 2024-03-26T17:57:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.973+0000 7fdac785b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96ca7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.973+0000 7fdac785b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8afc0 0x561e96c88da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.973+0000 7fdac785b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c90130 0x561e96c8ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.973+0000 7fdac785b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a6b30 con 0x561e96c8a5d0 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.973+0000 7fdac785b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a8670 con 0x561e96c8afc0 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.973+0000 7fdac785b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a7200 con 0x561e96c90130 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96ca7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96ca7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42374/0 (socket says 172.21.15.105:42374) 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 -- 172.21.15.105:0/143088262 learned_addr learned my addr 172.21.15.105:0/143088262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabed9d640 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c90130 0x561e96c8ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdab7fff640 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8afc0 0x561e96c88da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 -- 172.21.15.105:0/143088262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c90130 msgr2=0x561e96c8ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c90130 0x561e96c8ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 -- 172.21.15.105:0/143088262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8afc0 msgr2=0x561e96c88da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8afc0 0x561e96c88da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 -- 172.21.15.105:0/143088262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e96c8bec0 con 0x561e96c8a5d0 2024-03-26T17:57:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.974+0000 7fdabe59c640 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96ca7d10 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fdab00bd920 tx=0x7fdab0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96745e6f0a23524 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdaa7fff640 1 -- 172.21.15.105:0/143088262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab00bd760 con 0x561e96c8a5d0 2024-03-26T17:57:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdaa7fff640 1 -- 172.21.15.105:0/143088262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab00be320 con 0x561e96c8a5d0 2024-03-26T17:57:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdaa7fff640 1 -- 172.21.15.105:0/143088262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0003c50 con 0x561e96c8a5d0 2024-03-26T17:57:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdac785b740 1 -- 172.21.15.105:0/143088262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 msgr2=0x561e96ca7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdac785b740 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96ca7d10 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fdab00bd920 tx=0x7fdab0000bc0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdac785b740 1 -- 172.21.15.105:0/143088262 shutdown_connections 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdac785b740 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c90130 0x561e96c8ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdac785b740 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8afc0 0x561e96c88da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdac785b740 1 --2- 172.21.15.105:0/143088262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c8a5d0 0x561e96ca7d10 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.975+0000 7fdac785b740 1 -- 172.21.15.105:0/143088262 >> 172.21.15.105:0/143088262 conn(0x561e96c85b40 msgr2=0x561e96c80c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 -- 172.21.15.105:0/143088262 shutdown_connections 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 -- 172.21.15.105:0/143088262 wait complete. 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 -- start start 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 Processor -- start 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8a5d0 0x561e96c9a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c9a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c90130 0x561e96c9b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a7fa0 con 0x561e96c90130 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a78d0 con 0x561e96c8a5d0 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdac785b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a4b10 con 0x561e96c8afc0 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c90130 0x561e96c9b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8a5d0 0x561e96c9a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8a5d0 0x561e96c9a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41416/0 (socket says 172.21.15.105:41416) 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c90130 0x561e96c9b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42380/0 (socket says 172.21.15.105:42380) 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.976+0000 7fdabe59c640 1 -- 172.21.15.105:0/3864668880 learned_addr learned my addr 172.21.15.105:0/3864668880 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdac785b740 10 client.0 _ll_register_callbacks cb 0x7fdaa8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdab7fff640 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c9a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdabe59c640 1 -- 172.21.15.105:0/3864668880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 msgr2=0x561e96c9a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdabe59c640 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c9a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdabe59c640 1 -- 172.21.15.105:0/3864668880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c90130 msgr2=0x561e96c9b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdabe59c640 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c90130 0x561e96c9b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdabe59c640 1 -- 172.21.15.105:0/3864668880 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e96c9bb40 con 0x561e96c8a5d0 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdab7fff640 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c9a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdabed9d640 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c90130 0x561e96c9b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdabe59c640 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8a5d0 0x561e96c9a000 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdab00bdbb0 tx=0x7fdab0004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca53429a17296e8a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3864668880 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab00bd760 con 0x561e96c8a5d0 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3864668880 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fdab00882f0 con 0x561e96c8a5d0 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.977+0000 7fdac785b740 1 -- 172.21.15.105:0/3864668880 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab00be320 con 0x561e96c8a5d0 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3864668880 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdab00c1d70 con 0x561e96c8a5d0 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdac785b740 10 client.32063 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdac785b740 1 -- 172.21.15.105:0/3864668880 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561e96c9ce50 con 0x561e96c8a5d0 2024-03-26T17:57:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3864668880 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdab00be320 con 0x561e96c8a5d0 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3864668880 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fdab00bdf50 con 0x561e96c8a5d0 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdac785b740 10 client.32063 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdac785b740 10 client.32063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdac785b740 20 client.32063 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdac785b740 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e96d064e0 0x561e96d088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdac785b740 4 client.32063 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdac785b740 1 -- 172.21.15.105:0/3864668880 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e96bf6940 con 0x561e96d064e0 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.978+0000 7fdab7fff640 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e96d064e0 0x561e96d088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.979+0000 7fdaa5ffb640 10 client.32063 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:26.979+0000 7fdab7fff640 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e96d064e0 0x561e96d088d0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2507f433c834d7 server_cookie=75565a729b407d33 in_seq=0 out_seq=0 2024-03-26T17:57:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.111+0000 7fdaa5ffb640 1 -- 172.21.15.105:0/3864668880 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e96bf6940 con 0x561e96d064e0 2024-03-26T17:57:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.111+0000 7fdaa5ffb640 10 client.32063 handle_command_reply: tid=0 2024-03-26T17:57:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 client.32063 shutdown 2024-03-26T17:57:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 -- 172.21.15.105:0/3864668880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8a5d0 msgr2=0x561e96c9a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8a5d0 0x561e96c9a000 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdab00bdbb0 tx=0x7fdab0004590 comp rx=0 tx=0).stop 2024-03-26T17:57:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 -- 172.21.15.105:0/3864668880 shutdown_connections 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e96d064e0 0x561e96d088d0 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e96c90130 0x561e96c9b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e96c8afc0 0x561e96c9a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdaa5ffb640 0 client.32063 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 --2- 172.21.15.105:0/3864668880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e96c8a5d0 0x561e96c9a000 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 -- 172.21.15.105:0/3864668880 >> 172.21.15.105:0/3864668880 conn(0x561e96c85b40 msgr2=0x561e96c88da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 -- 172.21.15.105:0/3864668880 shutdown_connections 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 1 -- 172.21.15.105:0/3864668880 wait complete. 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.112+0000 7fdac785b740 20 client.32063 trim_cache size 0 max 16384 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:57:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:57:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:57:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 -- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdaac0a2fc0 msgr2=0x7fdaac0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdaac0a2fc0 0x7fdaac0a5480 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x561e96c8b820 tx=0x7fdab8004ab0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 -- 172.21.15.105:0/1740650659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 msgr2=0x7fdac0069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 0x7fdac0069ad0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdab0036040 tx=0x7fdab0005dc0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 -- 172.21.15.105:0/1740650659 shutdown_connections 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdaac0a2fc0 0x7fdaac0a5480 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdac0102390 0x7fdac019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdac01019d0 0x7fdac019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 --2- 172.21.15.105:0/1740650659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac01007e0 0x7fdac0069ad0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 -- 172.21.15.105:0/1740650659 >> 172.21.15.105:0/1740650659 conn(0x7fdac00fbd50 msgr2=0x7fdac0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 -- 172.21.15.105:0/1740650659 shutdown_connections 2024-03-26T17:57:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:27.113+0000 7fdac4c75640 1 -- 172.21.15.105:0/1740650659 wait complete. 2024-03-26T17:57:27.134 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:26 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:57:27.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:26 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:57:27.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:26 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:57:27.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 48690 7.04 MB/sec execute 250 sec latency 170.804 ms 2024-03-26T17:57:28.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:57:28.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:57:28.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:57:28.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:27 smithi144 ceph-mon[36095]: pgmap v923: 97 pgs: 97 active+clean; 301 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T17:57:28.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:57:28.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:57:28.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:57:28.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:27 smithi105 ceph-mon[32280]: pgmap v923: 97 pgs: 97 active+clean; 301 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T17:57:28.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:57:28.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:57:28.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:57:28.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:27 smithi135 ceph-mon[35636]: pgmap v923: 97 pgs: 97 active+clean; 301 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T17:57:28.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 48855 7.07 MB/sec execute 251 sec latency 153.672 ms 2024-03-26T17:57:29.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:28 smithi144 ceph-mon[36095]: pgmap v924: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-03-26T17:57:29.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:28 smithi105 ceph-mon[32280]: pgmap v924: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-03-26T17:57:29.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:28 smithi135 ceph-mon[35636]: pgmap v924: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-03-26T17:57:29.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 49029 7.09 MB/sec execute 252 sec latency 131.872 ms 2024-03-26T17:57:30.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 49214 7.10 MB/sec execute 253 sec latency 128.920 ms 2024-03-26T17:57:31.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:30 smithi144 ceph-mon[36095]: pgmap v925: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-03-26T17:57:31.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:30 smithi105 ceph-mon[32280]: pgmap v925: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-03-26T17:57:31.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:30 smithi135 ceph-mon[35636]: pgmap v925: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-03-26T17:57:31.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 49367 7.10 MB/sec execute 254 sec latency 114.983 ms 2024-03-26T17:57:32.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 49481 7.10 MB/sec execute 255 sec latency 131.706 ms 2024-03-26T17:57:33.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:32 smithi144 ceph-mon[36095]: pgmap v926: 97 pgs: 97 active+clean; 307 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 3.4 MiB/s wr, 254 op/s 2024-03-26T17:57:33.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:32 smithi105 ceph-mon[32280]: pgmap v926: 97 pgs: 97 active+clean; 307 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 3.4 MiB/s wr, 254 op/s 2024-03-26T17:57:33.409 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:32 smithi135 ceph-mon[35636]: pgmap v926: 97 pgs: 97 active+clean; 307 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 3.4 MiB/s wr, 254 op/s 2024-03-26T17:57:33.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 49565 7.09 MB/sec execute 256 sec latency 104.846 ms 2024-03-26T17:57:34.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 49729 7.12 MB/sec execute 257 sec latency 198.930 ms 2024-03-26T17:57:35.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:34 smithi144 ceph-mon[36095]: pgmap v927: 97 pgs: 97 active+clean; 312 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-03-26T17:57:35.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:34 smithi105 ceph-mon[32280]: pgmap v927: 97 pgs: 97 active+clean; 312 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-03-26T17:57:35.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:34 smithi135 ceph-mon[35636]: pgmap v927: 97 pgs: 97 active+clean; 312 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.9 MiB/s wr, 244 op/s 2024-03-26T17:57:35.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 49838 7.13 MB/sec execute 258 sec latency 140.890 ms 2024-03-26T17:57:36.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 49969 7.15 MB/sec execute 259 sec latency 139.714 ms 2024-03-26T17:57:37.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:36 smithi144 ceph-mon[36095]: pgmap v928: 97 pgs: 97 active+clean; 313 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-26T17:57:37.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:36 smithi105 ceph-mon[32280]: pgmap v928: 97 pgs: 97 active+clean; 313 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-26T17:57:37.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:36 smithi135 ceph-mon[35636]: pgmap v928: 97 pgs: 97 active+clean; 313 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.6 MiB/s wr, 223 op/s 2024-03-26T17:57:37.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50105 7.17 MB/sec execute 260 sec latency 138.247 ms 2024-03-26T17:57:38.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50228 7.16 MB/sec execute 261 sec latency 138.901 ms 2024-03-26T17:57:39.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:38 smithi144 ceph-mon[36095]: pgmap v929: 97 pgs: 97 active+clean; 321 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-03-26T17:57:39.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:38 smithi105 ceph-mon[32280]: pgmap v929: 97 pgs: 97 active+clean; 321 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-03-26T17:57:39.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:38 smithi135 ceph-mon[35636]: pgmap v929: 97 pgs: 97 active+clean; 321 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 4.7 MiB/s wr, 215 op/s 2024-03-26T17:57:39.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50323 7.15 MB/sec execute 262 sec latency 137.768 ms 2024-03-26T17:57:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.234+0000 7fd41ba61640 1 -- 172.21.15.105:0/3501056530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd414103530 msgr2=0x7fd414109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.234+0000 7fd41ba61640 1 --2- 172.21.15.105:0/3501056530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd414103530 0x7fd414109dc0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009cd0 tx=0x7fd40c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.234+0000 7fd41ba61640 1 -- 172.21.15.105:0/3501056530 shutdown_connections 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.234+0000 7fd41ba61640 1 --2- 172.21.15.105:0/3501056530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd414103530 0x7fd414109dc0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.234+0000 7fd41ba61640 1 --2- 172.21.15.105:0/3501056530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd414102b70 0x7fd414102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.234+0000 7fd41ba61640 1 --2- 172.21.15.105:0/3501056530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4140ffba0 0x7fd4140fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.234+0000 7fd41ba61640 1 -- 172.21.15.105:0/3501056530 >> 172.21.15.105:0/3501056530 conn(0x7fd4140fb510 msgr2=0x7fd4140fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.235+0000 7fd41ba61640 1 -- 172.21.15.105:0/3501056530 shutdown_connections 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.235+0000 7fd41ba61640 1 -- 172.21.15.105:0/3501056530 wait complete. 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.235+0000 7fd41ba61640 1 Processor -- start 2024-03-26T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.235+0000 7fd41ba61640 1 -- start start 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd41ba61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140ffba0 0x7fd414197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd41ba61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd414102b70 0x7fd414197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd41ba61640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd414103530 0x7fd41419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd41ba61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd41410d400 con 0x7fd4140ffba0 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd41ba61640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd41410d280 con 0x7fd414103530 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd41ba61640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd41410d580 con 0x7fd414102b70 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140ffba0 0x7fd414197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140ffba0 0x7fd414197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53756/0 (socket says 172.21.15.105:53756) 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 -- 172.21.15.105:0/1513935813 learned_addr learned my addr 172.21.15.105:0/1513935813 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd418ffc640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd414102b70 0x7fd414197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd419ffe640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd414103530 0x7fd41419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 -- 172.21.15.105:0/1513935813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd414102b70 msgr2=0x7fd414197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd414102b70 0x7fd414197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 -- 172.21.15.105:0/1513935813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd414103530 msgr2=0x7fd41419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd414103530 0x7fd41419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 -- 172.21.15.105:0/1513935813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41419f690 con 0x7fd4140ffba0 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd4197fd640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140ffba0 0x7fd414197a70 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e9c0 tx=0x7fd41000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd40a7fc640 1 -- 172.21.15.105:0/1513935813 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41000cd80 con 0x7fd4140ffba0 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.236+0000 7fd41ba61640 1 -- 172.21.15.105:0/1513935813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41419f980 con 0x7fd4140ffba0 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.237+0000 7fd40a7fc640 1 -- 172.21.15.105:0/1513935813 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4100035f0 con 0x7fd4140ffba0 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.237+0000 7fd40a7fc640 1 -- 172.21.15.105:0/1513935813 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410010690 con 0x7fd4140ffba0 2024-03-26T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.237+0000 7fd41ba61640 1 -- 172.21.15.105:0/1513935813 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41419ff20 con 0x7fd4140ffba0 2024-03-26T17:57:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.239+0000 7fd40a7fc640 1 -- 172.21.15.105:0/1513935813 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd4100037a0 con 0x7fd4140ffba0 2024-03-26T17:57:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd41e653740 1 Processor -- start 2024-03-26T17:57:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd41e653740 1 -- start start 2024-03-26T17:57:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd41e653740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f7635d0 0x561c6f7639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd41e653740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f768ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd41e653740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f769130 0x561c6f76b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd41e653740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c6f6c80b0 con 0x561c6f7635d0 2024-03-26T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd41e653740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561c6f670960 con 0x561c6f769130 2024-03-26T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd41e653740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c6f6c4060 con 0x561c6f763fc0 2024-03-26T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd419ffe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f7635d0 0x561c6f7639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd4197fd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f768ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd419ffe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f7635d0 0x561c6f7639f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53758/0 (socket says 172.21.15.105:53758) 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd419ffe640 1 -- 172.21.15.105:0/363814806 learned_addr learned my addr 172.21.15.105:0/363814806 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd418ffc640 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f769130 0x561c6f76b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.240+0000 7fd40a7fc640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4000a3040 0x7fd4000a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd419ffe640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4000a3040 0x7fd4000a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd4197fd640 1 -- 172.21.15.105:0/363814806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f769130 msgr2=0x561c6f76b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd4197fd640 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f769130 0x561c6f76b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd4197fd640 1 -- 172.21.15.105:0/363814806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f7635d0 msgr2=0x561c6f7639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd4197fd640 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f7635d0 0x561c6f7639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd40a7fc640 1 -- 172.21.15.105:0/1513935813 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd4100cf8c0 con 0x7fd4140ffba0 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd4197fd640 1 -- 172.21.15.105:0/363814806 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6f682c60 con 0x561c6f763fc0 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd418ffc640 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f769130 0x561c6f76b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd4197fd640 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f768ac0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd4100cd2f0 tx=0x7fd410004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a806b78f81e23786 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd419ffe640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4000a3040 0x7fd4000a5500 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x561c6f764820 tx=0x7fd40c02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd3ff7fe640 1 -- 172.21.15.105:0/363814806 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410003b30 con 0x561c6f763fc0 2024-03-26T17:57:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd3ff7fe640 1 -- 172.21.15.105:0/363814806 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd41009c650 con 0x561c6f763fc0 2024-03-26T17:57:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.241+0000 7fd3ff7fe640 1 -- 172.21.15.105:0/363814806 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100985c0 con 0x561c6f763fc0 2024-03-26T17:57:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.242+0000 7fd41e653740 1 -- 172.21.15.105:0/363814806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 msgr2=0x561c6f768ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.242+0000 7fd41e653740 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f768ac0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd4100cd2f0 tx=0x7fd410004730 comp rx=0 tx=0).stop 2024-03-26T17:57:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.242+0000 7fd41e653740 1 -- 172.21.15.105:0/363814806 shutdown_connections 2024-03-26T17:57:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.242+0000 7fd41e653740 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f769130 0x561c6f76b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.242+0000 7fd41e653740 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f768ac0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.242+0000 7fd41e653740 1 --2- 172.21.15.105:0/363814806 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f7635d0 0x561c6f7639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.242+0000 7fd41e653740 1 -- 172.21.15.105:0/363814806 >> 172.21.15.105:0/363814806 conn(0x561c6f75eb40 msgr2=0x561c6f75ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.243+0000 7fd41e653740 1 -- 172.21.15.105:0/363814806 shutdown_connections 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.243+0000 7fd41e653740 1 -- 172.21.15.105:0/363814806 wait complete. 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.243+0000 7fd41e653740 1 -- start start 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.243+0000 7fd41e653740 1 Processor -- start 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.243+0000 7fd41e653740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f766160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.243+0000 7fd41e653740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f7666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd41e653740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f767640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd41e653740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c6f76bf40 con 0x561c6f769130 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd41e653740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561c6f683db0 con 0x561c6f7635d0 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd41e653740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c6f76c0c0 con 0x561c6f763fc0 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f7666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f7666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55662/0 (socket says 172.21.15.105:55662) 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd4197fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f766160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 -- 172.21.15.105:0/3853856801 learned_addr learned my addr 172.21.15.105:0/3853856801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd419ffe640 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f767640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd41e653740 10 client.0 _ll_register_callbacks cb 0x561c6f688110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 -- 172.21.15.105:0/3853856801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 msgr2=0x561c6f766160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f766160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 -- 172.21.15.105:0/3853856801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 msgr2=0x561c6f767640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f767640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 -- 172.21.15.105:0/3853856801 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6f3dc210 con 0x561c6f763fc0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd4197fd640 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f766160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd418ffc640 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f7666a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd404012720 tx=0x7fd404012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=583cfe0b37a2b2ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd3fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.244+0000 7fd3fcff9640 1 -- 172.21.15.105:0/3853856801 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404004030 con 0x561c6f763fc0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.245+0000 7fd41e653740 1 -- 172.21.15.105:0/3853856801 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41009c650 con 0x561c6f763fc0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.245+0000 7fd3fcff9640 1 -- 172.21.15.105:0/3853856801 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd4040041d0 con 0x561c6f763fc0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.245+0000 7fd3fcff9640 1 -- 172.21.15.105:0/3853856801 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd404004410 con 0x561c6f763fc0 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.245+0000 7fd41e653740 10 client.32062 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.245+0000 7fd41e653740 1 -- 172.21.15.105:0/3853856801 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c6f4a8620 con 0x561c6f763fc0 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.245+0000 7fd3fcff9640 1 -- 172.21.15.105:0/3853856801 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd41009c650 con 0x561c6f763fc0 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.245+0000 7fd3fcff9640 1 -- 172.21.15.105:0/3853856801 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd404003b20 con 0x561c6f763fc0 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.246+0000 7fd41e653740 10 client.32062 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.246+0000 7fd41e653740 10 client.32062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.246+0000 7fd41e653740 20 client.32062 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.246+0000 7fd41e653740 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c6f7797e0 0x561c6f77bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.246+0000 7fd41e653740 4 client.32062 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.246+0000 7fd41e653740 1 -- 172.21.15.105:0/3853856801 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c6f6d55f0 con 0x561c6f7797e0 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.246+0000 7fd4197fd640 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c6f7797e0 0x561c6f77bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.247+0000 7fd4197fd640 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c6f7797e0 0x561c6f77bca0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6921005046b4bb34 server_cookie=1d456cecfccf232f in_seq=0 out_seq=0 2024-03-26T17:57:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.247+0000 7fd3fcff9640 10 client.32062 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.249+0000 7fd3fcff9640 1 -- 172.21.15.105:0/3853856801 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c6f6d55f0 con 0x561c6f7797e0 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.249+0000 7fd3fcff9640 10 client.32062 handle_command_reply: tid=0 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 client.32062 shutdown 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 -- 172.21.15.105:0/3853856801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 msgr2=0x561c6f7666a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f7666a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd404012720 tx=0x7fd404012be0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 -- 172.21.15.105:0/3853856801 shutdown_connections 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c6f7797e0 0x561c6f77bca0 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f767640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f7666a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd3fcff9640 0 client.32062 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 --2- 172.21.15.105:0/3853856801 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f766160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 -- 172.21.15.105:0/3853856801 >> 172.21.15.105:0/3853856801 conn(0x561c6f75eb40 msgr2=0x561c6f75ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 -- 172.21.15.105:0/3853856801 shutdown_connections 2024-03-26T17:57:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 1 -- 172.21.15.105:0/3853856801 wait complete. 2024-03-26T17:57:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.250+0000 7fd41e653740 20 client.32062 trim_cache size 0 max 16384 2024-03-26T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.271+0000 7fd41e653740 1 Processor -- start 2024-03-26T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd41e653740 1 -- start start 2024-03-26T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd41e653740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f767600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd41e653740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f76c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd41e653740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f76c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd41e653740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141a6be0 con 0x561c6f769130 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd41e653740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141a8720 con 0x561c6f7635d0 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd41e653740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141a72b0 con 0x561c6f763fc0 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd418ffc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f76c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd419ffe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f76c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd418ffc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f76c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55678/0 (socket says 172.21.15.105:55678) 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd419ffe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f76c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53784/0 (socket says 172.21.15.105:53784) 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd418ffc640 1 -- 172.21.15.105:0/3176225425 learned_addr learned my addr 172.21.15.105:0/3176225425 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd4197fd640 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f767600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd418ffc640 1 -- 172.21.15.105:0/3176225425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 msgr2=0x561c6f767600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd418ffc640 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f767600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd418ffc640 1 -- 172.21.15.105:0/3176225425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 msgr2=0x561c6f76c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd418ffc640 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f76c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd418ffc640 1 -- 172.21.15.105:0/3176225425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c6f7643a0 con 0x561c6f763fc0 2024-03-26T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.272+0000 7fd419ffe640 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f76c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.273+0000 7fd418ffc640 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f76c490 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd4040126f0 tx=0x7fd404009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a10ac9a113e722 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.273+0000 7fd3feffd640 1 -- 172.21.15.105:0/3176225425 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404010730 con 0x561c6f763fc0 2024-03-26T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.273+0000 7fd3feffd640 1 -- 172.21.15.105:0/3176225425 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd404022030 con 0x561c6f763fc0 2024-03-26T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.273+0000 7fd3feffd640 1 -- 172.21.15.105:0/3176225425 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40400fa80 con 0x561c6f763fc0 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 -- 172.21.15.105:0/3176225425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 msgr2=0x561c6f76c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f76c490 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd4040126f0 tx=0x7fd404009f40 comp rx=0 tx=0).stop 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 -- 172.21.15.105:0/3176225425 shutdown_connections 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f769130 0x561c6f76c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f763fc0 0x561c6f76c490 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 --2- 172.21.15.105:0/3176225425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f767600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 -- 172.21.15.105:0/3176225425 >> 172.21.15.105:0/3176225425 conn(0x561c6f75eb40 msgr2=0x561c6f766160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 -- 172.21.15.105:0/3176225425 shutdown_connections 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 -- 172.21.15.105:0/3176225425 wait complete. 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 -- start start 2024-03-26T17:57:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 Processor -- start 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.274+0000 7fd41e653740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f772fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd41e653740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f763fc0 0x561c6f773500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd41e653740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f769130 0x561c6f774510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd41e653740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141a8050 con 0x561c6f763fc0 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd41e653740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141a7980 con 0x561c6f7635d0 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd41e653740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141a4bc0 con 0x561c6f769130 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd41e653740 10 client.0 _ll_register_callbacks cb 0x7fd404009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f772fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f772fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58728/0 (socket says 172.21.15.105:58728) 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 -- 172.21.15.105:0/1887508549 learned_addr learned my addr 172.21.15.105:0/1887508549 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd418ffc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f763fc0 0x561c6f773500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd419ffe640 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f769130 0x561c6f774510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 -- 172.21.15.105:0/1887508549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f769130 msgr2=0x561c6f774510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f769130 0x561c6f774510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 -- 172.21.15.105:0/1887508549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f763fc0 msgr2=0x561c6f773500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f763fc0 0x561c6f773500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 -- 172.21.15.105:0/1887508549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6f774b00 con 0x561c6f7635d0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd418ffc640 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f763fc0 0x561c6f773500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd4197fd640 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f772fc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd41009bee0 tx=0x7fd410097bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db15e18186d85c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd3fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd3fcff9640 1 -- 172.21.15.105:0/1887508549 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41009c650 con 0x561c6f7635d0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd3fcff9640 1 -- 172.21.15.105:0/1887508549 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fd4100cd3a0 con 0x561c6f7635d0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.275+0000 7fd41e653740 1 -- 172.21.15.105:0/1887508549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd404022030 con 0x561c6f7635d0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd3fcff9640 1 -- 172.21.15.105:0/1887508549 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd41009bf50 con 0x561c6f7635d0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd41e653740 10 client.32081 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd41e653740 1 -- 172.21.15.105:0/1887508549 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c6f775e80 con 0x561c6f7635d0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd3fcff9640 1 -- 172.21.15.105:0/1887508549 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd404022030 con 0x561c6f7635d0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd3fcff9640 1 -- 172.21.15.105:0/1887508549 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fd410096ac0 con 0x561c6f7635d0 2024-03-26T17:57:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd41e653740 10 client.32081 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd41e653740 10 client.32081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd41e653740 20 client.32081 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd41e653740 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c6f7dfb40 0x561c6f7e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd41e653740 4 client.32081 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:57:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.276+0000 7fd41e653740 1 -- 172.21.15.105:0/1887508549 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c6f6cf940 con 0x561c6f7dfb40 2024-03-26T17:57:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.277+0000 7fd418ffc640 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c6f7dfb40 0x561c6f7e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.277+0000 7fd418ffc640 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c6f7dfb40 0x561c6f7e1f30 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8c40fc7bcfd2a6 server_cookie=fd2cd77c213ed552 in_seq=0 out_seq=0 2024-03-26T17:57:40.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.277+0000 7fd3fcff9640 10 client.32081 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd3fcff9640 1 -- 172.21.15.105:0/1887508549 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c6f6cf940 con 0x561c6f7dfb40 2024-03-26T17:57:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd3fcff9640 10 client.32081 handle_command_reply: tid=0 2024-03-26T17:57:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 client.32081 shutdown 2024-03-26T17:57:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 -- 172.21.15.105:0/1887508549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 msgr2=0x561c6f772fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f772fc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd41009bee0 tx=0x7fd410097bc0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 -- 172.21.15.105:0/1887508549 shutdown_connections 2024-03-26T17:57:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c6f7dfb40 0x561c6f7e1f30 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c6f769130 0x561c6f774510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd3fcff9640 0 client.32081 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c6f763fc0 0x561c6f773500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 --2- 172.21.15.105:0/1887508549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c6f7635d0 0x561c6f772fc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.443+0000 7fd41e653740 1 -- 172.21.15.105:0/1887508549 >> 172.21.15.105:0/1887508549 conn(0x561c6f75eb40 msgr2=0x561c6f76c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.444+0000 7fd41e653740 1 -- 172.21.15.105:0/1887508549 shutdown_connections 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.444+0000 7fd41e653740 1 -- 172.21.15.105:0/1887508549 wait complete. 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.444+0000 7fd41e653740 20 client.32081 trim_cache size 0 max 16384 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:57:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:57:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 -- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4000a3040 msgr2=0x7fd4000a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4000a3040 0x7fd4000a5500 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x561c6f764820 tx=0x7fd40c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 -- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140ffba0 msgr2=0x7fd414197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140ffba0 0x7fd414197a70 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fd41000e9c0 tx=0x7fd41000ee80 comp rx=0 tx=0).stop 2024-03-26T17:57:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 -- 172.21.15.105:0/1513935813 shutdown_connections 2024-03-26T17:57:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4000a3040 0x7fd4000a5500 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd414103530 0x7fd41419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd414102b70 0x7fd414197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 --2- 172.21.15.105:0/1513935813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4140ffba0 0x7fd414197a70 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 -- 172.21.15.105:0/1513935813 >> 172.21.15.105:0/1513935813 conn(0x7fd4140fb510 msgr2=0x7fd414105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.445+0000 7fd41ba61640 1 -- 172.21.15.105:0/1513935813 shutdown_connections 2024-03-26T17:57:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:40.446+0000 7fd41ba61640 1 -- 172.21.15.105:0/1513935813 wait complete. 2024-03-26T17:57:40.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50405 7.14 MB/sec execute 263 sec latency 254.138 ms 2024-03-26T17:57:41.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:40 smithi144 ceph-mon[36095]: pgmap v930: 97 pgs: 97 active+clean; 322 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 3.6 MiB/s wr, 168 op/s 2024-03-26T17:57:41.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:40 smithi105 ceph-mon[32280]: pgmap v930: 97 pgs: 97 active+clean; 322 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 3.6 MiB/s wr, 168 op/s 2024-03-26T17:57:41.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:40 smithi135 ceph-mon[35636]: pgmap v930: 97 pgs: 97 active+clean; 322 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 3.6 MiB/s wr, 168 op/s 2024-03-26T17:57:41.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50499 7.13 MB/sec execute 264 sec latency 171.733 ms 2024-03-26T17:57:42.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50595 7.11 MB/sec execute 265 sec latency 146.742 ms 2024-03-26T17:57:43.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:42 smithi144 ceph-mon[36095]: pgmap v931: 97 pgs: 97 active+clean; 324 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-03-26T17:57:43.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:42 smithi105 ceph-mon[32280]: pgmap v931: 97 pgs: 97 active+clean; 324 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-03-26T17:57:43.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:42 smithi135 ceph-mon[35636]: pgmap v931: 97 pgs: 97 active+clean; 324 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-03-26T17:57:43.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50700 7.08 MB/sec execute 266 sec latency 134.636 ms 2024-03-26T17:57:43.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:57:43.847 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:57:43.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 -- 172.21.15.105:0/755288591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb900107620 msgr2=0x7fb900109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:43.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 --2- 172.21.15.105:0/755288591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb900107620 0x7fb900109a70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009ca0 tx=0x7fb8f802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:43.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 -- 172.21.15.105:0/755288591 shutdown_connections 2024-03-26T17:57:43.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 --2- 172.21.15.105:0/755288591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb900107620 0x7fb900109a70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:43.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 --2- 172.21.15.105:0/755288591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb900104cf0 0x7fb9001070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:43.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 --2- 172.21.15.105:0/755288591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9001023c0 0x7fb9001047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:43.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 -- 172.21.15.105:0/755288591 >> 172.21.15.105:0/755288591 conn(0x7fb9000fbdf0 msgr2=0x7fb9000fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 -- 172.21.15.105:0/755288591 shutdown_connections 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.009+0000 7fb907bdf640 1 -- 172.21.15.105:0/755288591 wait complete. 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb907bdf640 1 Processor -- start 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb907bdf640 1 -- start start 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb907bdf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9001023c0 0x7fb900199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb907bdf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb900104cf0 0x7fb90019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb907bdf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb900107620 0x7fb9001a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb907bdf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb90010f610 con 0x7fb900107620 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb907bdf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb90010f490 con 0x7fb900104cf0 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb907bdf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb90010f790 con 0x7fb9001023c0 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb90597b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9001023c0 0x7fb900199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb90617c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb900107620 0x7fb9001a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb90597b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9001023c0 0x7fb900199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55704/0 (socket says 172.21.15.105:55704) 2024-03-26T17:57:43.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb90597b640 1 -- 172.21.15.105:0/2904127565 learned_addr learned my addr 172.21.15.105:0/2904127565 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.010+0000 7fb90517a640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb900104cf0 0x7fb90019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb90617c640 1 -- 172.21.15.105:0/2904127565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9001023c0 msgr2=0x7fb900199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb90617c640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9001023c0 0x7fb900199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb90617c640 1 -- 172.21.15.105:0/2904127565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb900104cf0 msgr2=0x7fb90019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb90617c640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb900104cf0 0x7fb90019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb90617c640 1 -- 172.21.15.105:0/2904127565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9001a18f0 con 0x7fb900107620 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb90597b640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9001023c0 0x7fb900199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb90617c640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb900107620 0x7fb9001a1250 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fb8f802c630 tx=0x7fb8f80328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb8f6ffd640 1 -- 172.21.15.105:0/2904127565 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8041020 con 0x7fb900107620 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb907bdf640 1 -- 172.21.15.105:0/2904127565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9001a1b80 con 0x7fb900107620 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb8f6ffd640 1 -- 172.21.15.105:0/2904127565 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f8003e20 con 0x7fb900107620 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb8f6ffd640 1 -- 172.21.15.105:0/2904127565 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8039040 con 0x7fb900107620 2024-03-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.011+0000 7fb907bdf640 1 -- 172.21.15.105:0/2904127565 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9001a20c0 con 0x7fb900107620 2024-03-26T17:57:43.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.014+0000 7fb90a7c7740 1 Processor -- start 2024-03-26T17:57:43.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.014+0000 7fb8f6ffd640 1 -- 172.21.15.105:0/2904127565 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb8f8044020 con 0x7fb900107620 2024-03-26T17:57:43.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90a7c7740 1 -- start start 2024-03-26T17:57:43.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e79520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:43.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e7e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e7ec60 0x561d22e810f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90a7c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561d22dd5980 con 0x561d22e79100 2024-03-26T17:57:44.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90a7c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561d22d7dbe0 con 0x561d22e79af0 2024-03-26T17:57:44.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90a7c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561d22dd1930 con 0x561d22e7ec60 2024-03-26T17:57:44.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90517a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e79520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90597b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e7e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90517a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e79520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53820/0 (socket says 172.21.15.105:53820) 2024-03-26T17:57:44.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb90517a640 1 -- 172.21.15.105:0/2212361678 learned_addr learned my addr 172.21.15.105:0/2212361678 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.015+0000 7fb8f6ffd640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8ec0a2f20 0x7fb8ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90517a640 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e7ec60 0x561d22e810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90597b640 1 -- 172.21.15.105:0/2212361678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e7ec60 msgr2=0x561d22e810f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90597b640 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e7ec60 0x561d22e810f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90597b640 1 -- 172.21.15.105:0/2212361678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 msgr2=0x561d22e79520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90597b640 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e79520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90597b640 1 -- 172.21.15.105:0/2212361678 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d22bb37d0 con 0x561d22e79af0 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90517a640 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e7ec60 0x561d22e810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90617c640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8ec0a2f20 0x7fb8ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb8f6ffd640 1 -- 172.21.15.105:0/2904127565 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb8f80f1590 con 0x7fb900107620 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb90597b640 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e7e5f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb90019ab80 tx=0x7fb8fc0119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=973b06b8eb830325 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb8ebfff640 1 -- 172.21.15.105:0/2212361678 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc005020 con 0x561d22e79af0 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb8ebfff640 1 -- 172.21.15.105:0/2212361678 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc0047a0 con 0x561d22e79af0 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.016+0000 7fb8ebfff640 1 -- 172.21.15.105:0/2212361678 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc00f9e0 con 0x561d22e79af0 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 -- 172.21.15.105:0/2212361678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 msgr2=0x561d22e7e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90617c640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8ec0a2f20 0x7fb8ec0a53e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80bd980 tx=0x7fb8f802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e7e5f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb90019ab80 tx=0x7fb8fc0119a0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 -- 172.21.15.105:0/2212361678 shutdown_connections 2024-03-26T17:57:44.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e7ec60 0x561d22e810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e7e5f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/2212361678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e79520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 -- 172.21.15.105:0/2212361678 >> 172.21.15.105:0/2212361678 conn(0x561d22e74670 msgr2=0x561d22e74a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 -- 172.21.15.105:0/2212361678 shutdown_connections 2024-03-26T17:57:44.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.017+0000 7fb90a7c7740 1 -- 172.21.15.105:0/2212361678 wait complete. 2024-03-26T17:57:44.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.018+0000 7fb90a7c7740 1 -- start start 2024-03-26T17:57:44.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.018+0000 7fb90a7c7740 1 Processor -- start 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.018+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e83320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.018+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79af0 0x561d22e83860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e8a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90a7c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561d22e81c10 con 0x561d22e79100 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90a7c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561d22d91160 con 0x561d22e7ec60 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90a7c7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561d22e81d90 con 0x561d22e79af0 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90517a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e83320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90597b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79af0 0x561d22e83860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90517a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e83320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53822/0 (socket says 172.21.15.105:53822) 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90517a640 1 -- 172.21.15.105:0/4259539015 learned_addr learned my addr 172.21.15.105:0/4259539015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90a7c7740 10 client.0 _ll_register_callbacks cb 0x561d22dcd720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90517a640 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e8a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90597b640 1 -- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 msgr2=0x561d22e8a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90597b640 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e8a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90597b640 1 -- 172.21.15.105:0/4259539015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 msgr2=0x561d22e83320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90597b640 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e83320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90597b640 1 -- 172.21.15.105:0/4259539015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d22ae93f0 con 0x561d22e79af0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90517a640 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90597b640 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79af0 0x561d22e83860 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x561d22e755e0 tx=0x7fb8fc005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e016136f925764b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb8e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb8e97fa640 1 -- 172.21.15.105:0/4259539015 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc028070 con 0x561d22e79af0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb8e97fa640 1 -- 172.21.15.105:0/4259539015 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb8fc00f760 con 0x561d22e79af0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.019+0000 7fb90a7c7740 1 -- 172.21.15.105:0/4259539015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc0047a0 con 0x561d22e79af0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.020+0000 7fb8e97fa640 1 -- 172.21.15.105:0/4259539015 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb8fc034050 con 0x561d22e79af0 2024-03-26T17:57:44.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.020+0000 7fb90a7c7740 10 client.32077 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.020+0000 7fb90a7c7740 1 -- 172.21.15.105:0/4259539015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561d22aa90f0 con 0x561d22e79af0 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.020+0000 7fb8e97fa640 1 -- 172.21.15.105:0/4259539015 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc009cf0 con 0x561d22e79af0 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb8e97fa640 1 -- 172.21.15.105:0/4259539015 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb8fc0249d0 con 0x561d22e79af0 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb90a7c7740 10 client.32077 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb90a7c7740 10 client.32077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb90a7c7740 20 client.32077 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d22e8ee00 0x561d22e91280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb90a7c7740 4 client.32077 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb90a7c7740 1 -- 172.21.15.105:0/4259539015 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb8fc0047a0 con 0x561d22e8ee00 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb90517a640 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d22e8ee00 0x561d22e91280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb8e97fa640 10 client.32077 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.021+0000 7fb90517a640 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d22e8ee00 0x561d22e91280 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5c603dc1f5f8e1 server_cookie=c908f2cc30d7f1ab in_seq=0 out_seq=0 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb8e97fa640 1 -- 172.21.15.105:0/4259539015 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fb8fc0047a0 con 0x561d22e8ee00 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb8e97fa640 10 client.32077 handle_command_reply: tid=0 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 client.32077 shutdown 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 -- 172.21.15.105:0/4259539015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79af0 msgr2=0x561d22e83860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79af0 0x561d22e83860 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x561d22e755e0 tx=0x7fb8fc005250 comp rx=0 tx=0).stop 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 -- 172.21.15.105:0/4259539015 shutdown_connections 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d22e8ee00 0x561d22e91280 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e8a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79af0 0x561d22e83860 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/4259539015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79100 0x561d22e83320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 -- 172.21.15.105:0/4259539015 >> 172.21.15.105:0/4259539015 conn(0x561d22e74670 msgr2=0x561d22e6b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb8e97fa640 0 client.32077 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:44.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 -- 172.21.15.105:0/4259539015 shutdown_connections 2024-03-26T17:57:44.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.024+0000 7fb90a7c7740 1 -- 172.21.15.105:0/4259539015 wait complete. 2024-03-26T17:57:44.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.025+0000 7fb90a7c7740 20 client.32077 trim_cache size 0 max 16384 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.054+0000 7fb90a7c7740 1 Processor -- start 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90a7c7740 1 -- start start 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e7e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79af0 0x561d22e8a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e7acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90a7c7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001a6ae0 con 0x561d22e79af0 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90517a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e7e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90517a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e7e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55726/0 (socket says 172.21.15.105:55726) 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90517a640 1 -- 172.21.15.105:0/3359896029 learned_addr learned my addr 172.21.15.105:0/3359896029 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90a7c7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001a8620 con 0x561d22e7ec60 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90a7c7740 1 -- 172.21.15.105:0/3359896029 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001a71b0 con 0x561d22e79100 2024-03-26T17:57:44.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90517a640 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e7acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.055+0000 7fb90597b640 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79af0 0x561d22e8a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb90517a640 1 -- 172.21.15.105:0/3359896029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 msgr2=0x561d22e7acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb90517a640 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e7acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb90517a640 1 -- 172.21.15.105:0/3359896029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79af0 msgr2=0x561d22e8a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb90517a640 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79af0 0x561d22e8a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb90517a640 1 -- 172.21.15.105:0/3359896029 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d22e7a790 con 0x561d22e79100 2024-03-26T17:57:44.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb90517a640 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e7e320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0024170 tx=0x7fb8f0009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb9b6f2d44ef617 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb8eb7fe640 1 -- 172.21.15.105:0/3359896029 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00247b0 con 0x561d22e79100 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb8eb7fe640 1 -- 172.21.15.105:0/3359896029 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f001b040 con 0x561d22e79100 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.056+0000 7fb8eb7fe640 1 -- 172.21.15.105:0/3359896029 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00047a0 con 0x561d22e79100 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 -- 172.21.15.105:0/3359896029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 msgr2=0x561d22e7e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e7e320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0024170 tx=0x7fb8f0009910 comp rx=0 tx=0).stop 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 -- 172.21.15.105:0/3359896029 shutdown_connections 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e7ec60 0x561d22e7acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e79af0 0x561d22e8a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/3359896029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e7e320 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 -- 172.21.15.105:0/3359896029 >> 172.21.15.105:0/3359896029 conn(0x561d22e74670 msgr2=0x561d22e74a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 -- 172.21.15.105:0/3359896029 shutdown_connections 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.057+0000 7fb90a7c7740 1 -- 172.21.15.105:0/3359896029 wait complete. 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.058+0000 7fb90a7c7740 1 -- start start 2024-03-26T17:57:44.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.058+0000 7fb90a7c7740 1 Processor -- start 2024-03-26T17:57:44.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.058+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e87c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.058+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e88170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.058+0000 7fb90a7c7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e7ec60 0x561d22e89180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.058+0000 7fb90517a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e87c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.058+0000 7fb90517a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e87c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55742/0 (socket says 172.21.15.105:55742) 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90517a640 1 -- 172.21.15.105:0/430093458 learned_addr learned my addr 172.21.15.105:0/430093458 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001a7f50 con 0x561d22e7ec60 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001a7880 con 0x561d22e79af0 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9001a4ac0 con 0x561d22e79100 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90597b640 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e88170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90a7c7740 10 client.0 _ll_register_callbacks cb 0x7fb8f0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90517a640 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e7ec60 0x561d22e89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90517a640 1 -- 172.21.15.105:0/430093458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 msgr2=0x561d22e87c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90517a640 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e87c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90517a640 1 -- 172.21.15.105:0/430093458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 msgr2=0x561d22e88170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90517a640 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e88170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.059+0000 7fb90517a640 1 -- 172.21.15.105:0/430093458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d22e89770 con 0x561d22e7ec60 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.060+0000 7fb90517a640 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e7ec60 0x561d22e89180 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0015040 tx=0x7fb8f0009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2373d9e407fd4f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.060+0000 7fb8e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.060+0000 7fb8e97fa640 1 -- 172.21.15.105:0/430093458 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f001a040 con 0x561d22e7ec60 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.060+0000 7fb8e97fa640 1 -- 172.21.15.105:0/430093458 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb8f00035f0 con 0x561d22e7ec60 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.060+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f001b040 con 0x561d22e7ec60 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.060+0000 7fb8e97fa640 1 -- 172.21.15.105:0/430093458 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb8f000fe70 con 0x561d22e7ec60 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.060+0000 7fb90a7c7740 10 client.22407 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.060+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561d22e8aa80 con 0x561d22e7ec60 2024-03-26T17:57:44.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.061+0000 7fb8e97fa640 1 -- 172.21.15.105:0/430093458 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f001b040 con 0x561d22e7ec60 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.061+0000 7fb8e97fa640 1 -- 172.21.15.105:0/430093458 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb8f0031070 con 0x561d22e7ec60 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.061+0000 7fb90a7c7740 10 client.22407 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.061+0000 7fb90a7c7740 10 client.22407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.061+0000 7fb90a7c7740 20 client.22407 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.061+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d22ef5840 0x561d22ef7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.061+0000 7fb90a7c7740 4 client.22407 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.061+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d22de2ec0 con 0x561d22ef5840 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.062+0000 7fb90597b640 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d22ef5840 0x561d22ef7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.062+0000 7fb90597b640 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d22ef5840 0x561d22ef7c60 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a124528e2e1edd server_cookie=f7086f833984560b in_seq=0 out_seq=0 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.062+0000 7fb8e97fa640 10 client.22407 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb8e97fa640 1 -- 172.21.15.105:0/430093458 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561d22de2ec0 con 0x561d22ef5840 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb8e97fa640 10 client.22407 handle_command_reply: tid=0 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 client.22407 shutdown 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e7ec60 msgr2=0x561d22e89180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e7ec60 0x561d22e89180 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0015040 tx=0x7fb8f0009910 comp rx=0 tx=0).stop 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 shutdown_connections 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561d22ef5840 0x561d22ef7c60 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d22e7ec60 0x561d22e89180 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d22e79af0 0x561d22e88170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb8e97fa640 0 client.22407 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 --2- 172.21.15.105:0/430093458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d22e79100 0x561d22e87c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.064+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 >> 172.21.15.105:0/430093458 conn(0x561d22e74670 msgr2=0x561d22e7e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.065+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 shutdown_connections 2024-03-26T17:57:44.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.065+0000 7fb90a7c7740 1 -- 172.21.15.105:0/430093458 wait complete. 2024-03-26T17:57:44.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.065+0000 7fb90a7c7740 20 client.22407 trim_cache size 0 max 16384 2024-03-26T17:57:44.051 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:57:44.051 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:57:44.051 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:57:44.051 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:57:44.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 -- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8ec0a2f20 msgr2=0x7fb8ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8ec0a2f20 0x7fb8ec0a53e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80bd980 tx=0x7fb8f802e040 comp rx=0 tx=0).stop 2024-03-26T17:57:44.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 -- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb900107620 msgr2=0x7fb9001a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb900107620 0x7fb9001a1250 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fb8f802c630 tx=0x7fb8f80328a0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 -- 172.21.15.105:0/2904127565 shutdown_connections 2024-03-26T17:57:44.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8ec0a2f20 0x7fb8ec0a53e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb900107620 0x7fb9001a1250 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb900104cf0 0x7fb90019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 --2- 172.21.15.105:0/2904127565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9001023c0 0x7fb900199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 -- 172.21.15.105:0/2904127565 >> 172.21.15.105:0/2904127565 conn(0x7fb9000fbdf0 msgr2=0x7fb900102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 -- 172.21.15.105:0/2904127565 shutdown_connections 2024-03-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.066+0000 7fb907bdf640 1 -- 172.21.15.105:0/2904127565 wait complete. 2024-03-26T17:57:44.066 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:57:44.067 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:57:44.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 -- 172.21.15.105:0/1600305506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 msgr2=0x7ff158110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 --2- 172.21.15.105:0/1600305506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 0x7ff158110370 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ff150009ca0 tx=0x7ff15002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 -- 172.21.15.105:0/1600305506 shutdown_connections 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 --2- 172.21.15.105:0/1600305506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 0x7ff158110370 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 --2- 172.21.15.105:0/1600305506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff158109090 0x7ff158109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 --2- 172.21.15.105:0/1600305506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff158107eb0 0x7ff1581082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 -- 172.21.15.105:0/1600305506 >> 172.21.15.105:0/1600305506 conn(0x7ff158076ed0 msgr2=0x7ff1580792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 -- 172.21.15.105:0/1600305506 shutdown_connections 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.225+0000 7ff15cfab640 1 -- 172.21.15.105:0/1600305506 wait complete. 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15cfab640 1 Processor -- start 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15cfab640 1 -- start start 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15cfab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff158107eb0 0x7ff15819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15cfab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff158109090 0x7ff15819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15cfab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 0x7ff1581a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15cfab640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff158113a30 con 0x7ff158109a50 2024-03-26T17:57:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15cfab640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1581138b0 con 0x7ff158109090 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15cfab640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff158113bb0 con 0x7ff158107eb0 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff156d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 0x7ff1581a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff155d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff158109090 0x7ff15819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff158107eb0 0x7ff15819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff155d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff158109090 0x7ff15819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:58788/0 (socket says 172.21.15.105:58788) 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff155d9b640 1 -- 172.21.15.105:0/343505338 learned_addr learned my addr 172.21.15.105:0/343505338 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff158107eb0 0x7ff15819e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55754/0 (socket says 172.21.15.105:55754) 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff156d9d640 1 -- 172.21.15.105:0/343505338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff158107eb0 msgr2=0x7ff15819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff156d9d640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff158107eb0 0x7ff15819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff156d9d640 1 -- 172.21.15.105:0/343505338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff158109090 msgr2=0x7ff15819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff156d9d640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff158109090 0x7ff15819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff156d9d640 1 -- 172.21.15.105:0/343505338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1581a5ca0 con 0x7ff158109a50 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff15659c640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff158107eb0 0x7ff15819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.226+0000 7ff156d9d640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 0x7ff1581a5630 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff150037040 tx=0x7ff150032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.227+0000 7ff1477fe640 1 -- 172.21.15.105:0/343505338 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150041020 con 0x7ff158109a50 2024-03-26T17:57:44.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.227+0000 7ff1477fe640 1 -- 172.21.15.105:0/343505338 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff150003e20 con 0x7ff158109a50 2024-03-26T17:57:44.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.227+0000 7ff1477fe640 1 -- 172.21.15.105:0/343505338 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150039040 con 0x7ff158109a50 2024-03-26T17:57:44.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.227+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1581a5f30 con 0x7ff158109a50 2024-03-26T17:57:44.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.227+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1581a6340 con 0x7ff158109a50 2024-03-26T17:57:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.228+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15810f250 con 0x7ff158109a50 2024-03-26T17:57:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.229+0000 7ff1477fe640 1 -- 172.21.15.105:0/343505338 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff150044020 con 0x7ff158109a50 2024-03-26T17:57:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.230+0000 7ff1477fe640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1400a2f20 0x7ff1400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.230+0000 7ff15659c640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1400a2f20 0x7ff1400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.230+0000 7ff1477fe640 1 -- 172.21.15.105:0/343505338 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7ff1500f17b0 con 0x7ff158109a50 2024-03-26T17:57:44.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.231+0000 7ff15659c640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1400a2f20 0x7ff1400a53e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff158108c00 tx=0x7ff148005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.236+0000 7ff1477fe640 1 -- 172.21.15.105:0/343505338 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff15004b020 con 0x7ff158109a50 2024-03-26T17:57:44.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50839 7.08 MB/sec execute 267 sec latency 147.395 ms 2024-03-26T17:57:44.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.584+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff15802d240 con 0x7ff158109a50 2024-03-26T17:57:44.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.585+0000 7ff1477fe640 1 -- 172.21.15.105:0/343505338 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11941 (secure 0 0 0) 0x7ff1500be5a0 con 0x7ff158109a50 2024-03-26T17:57:44.570 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:57:44.570 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:56:47.920346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_31852":{"gid":31852,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/2697191136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2697191136},{"type":"v1","addr":"172.21.15.135:6835","nonce":2697191136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T17:57:44.570 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 39 2024-03-26T17:57:44.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1400a2f20 msgr2=0x7ff1400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1400a2f20 0x7ff1400a53e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff158108c00 tx=0x7ff148005f90 comp rx=0 tx=0).stop 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 msgr2=0x7ff1581a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 0x7ff1581a5630 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff150037040 tx=0x7ff150032820 comp rx=0 tx=0).stop 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 shutdown_connections 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff1400a2f20 0x7ff1400a53e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff158109a50 0x7ff1581a5630 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff158109090 0x7ff15819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 --2- 172.21.15.105:0/343505338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff158107eb0 0x7ff15819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 >> 172.21.15.105:0/343505338 conn(0x7ff158076ed0 msgr2=0x7ff15810c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 shutdown_connections 2024-03-26T17:57:44.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.588+0000 7ff15cfab640 1 -- 172.21.15.105:0/343505338 wait complete. 2024-03-26T17:57:44.586 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:57:44.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 -- 172.21.15.105:0/1135172927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb14102370 msgr2=0x7feb14108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 --2- 172.21.15.105:0/1135172927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb14102370 0x7feb14108c00 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009ca0 tx=0x7feb0c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 -- 172.21.15.105:0/1135172927 shutdown_connections 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 --2- 172.21.15.105:0/1135172927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb14102370 0x7feb14108c00 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 --2- 172.21.15.105:0/1135172927 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb141019b0 0x7feb14101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 --2- 172.21.15.105:0/1135172927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb141007c0 0x7feb14100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 -- 172.21.15.105:0/1135172927 >> 172.21.15.105:0/1135172927 conn(0x7feb140fbd50 msgr2=0x7feb140fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 -- 172.21.15.105:0/1135172927 shutdown_connections 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 -- 172.21.15.105:0/1135172927 wait complete. 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.752+0000 7feb1933b640 1 Processor -- start 2024-03-26T17:57:44.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb1933b640 1 -- start start 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb1933b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb141007c0 0x7feb14104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb1933b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb141019b0 0x7feb14102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb1933b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14102370 0x7feb141032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb1933b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb1410c2d0 con 0x7feb141019b0 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb1933b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb1410c150 con 0x7feb141007c0 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb1933b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb1410c450 con 0x7feb14102370 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb141019b0 0x7feb14102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb141007c0 0x7feb14104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb127fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb141019b0 0x7feb14102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55610/0 (socket says 172.21.15.105:55610) 2024-03-26T17:57:44.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb127fc640 1 -- 172.21.15.105:0/3960249149 learned_addr learned my addr 172.21.15.105:0/3960249149 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb137fe640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14102370 0x7feb141032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb127fc640 1 -- 172.21.15.105:0/3960249149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14102370 msgr2=0x7feb141032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb127fc640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14102370 0x7feb141032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb127fc640 1 -- 172.21.15.105:0/3960249149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb141007c0 msgr2=0x7feb14104750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb127fc640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb141007c0 0x7feb14104750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb127fc640 1 -- 172.21.15.105:0/3960249149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb14103b60 con 0x7feb141019b0 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.753+0000 7feb12ffd640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb141007c0 0x7feb14104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.754+0000 7feb127fc640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb141019b0 0x7feb14102d90 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7feb0800e9f0 tx=0x7feb0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.754+0000 7feafbfff640 1 -- 172.21.15.105:0/3960249149 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0800cd80 con 0x7feb141019b0 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.754+0000 7feb1933b640 1 -- 172.21.15.105:0/3960249149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb141a1bf0 con 0x7feb141019b0 2024-03-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.754+0000 7feafbfff640 1 -- 172.21.15.105:0/3960249149 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb080108e0 con 0x7feb141019b0 2024-03-26T17:57:44.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.754+0000 7feafbfff640 1 -- 172.21.15.105:0/3960249149 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08018920 con 0x7feb141019b0 2024-03-26T17:57:44.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.754+0000 7feb1933b640 1 -- 172.21.15.105:0/3960249149 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb141a2190 con 0x7feb141019b0 2024-03-26T17:57:44.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.757+0000 7feb1c009740 1 Processor -- start 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.757+0000 7feb1c009740 1 -- start start 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.757+0000 7feafbfff640 1 -- 172.21.15.105:0/3960249149 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7feb08010a90 con 0x7feb141019b0 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.757+0000 7feb1c009740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x55845816ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.757+0000 7feb1c009740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816b420 0x55845816ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.757+0000 7feb1c009740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x558458172a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb1c009740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5584580cf970 con 0x55845816b420 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb1c009740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558458077d70 con 0x558458170590 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb1c009740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5584580cb920 con 0x55845816aa30 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816b420 0x55845816ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816b420 0x55845816ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55622/0 (socket says 172.21.15.105:55622) 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb12ffd640 1 -- 172.21.15.105:0/752618452 learned_addr learned my addr 172.21.15.105:0/752618452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb127fc640 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x558458172a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb137fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x55845816ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb137fe640 1 -- 172.21.15.105:0/752618452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 msgr2=0x558458172a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb137fe640 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x558458172a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb137fe640 1 -- 172.21.15.105:0/752618452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816b420 msgr2=0x55845816ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb137fe640 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816b420 0x55845816ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb137fe640 1 -- 172.21.15.105:0/752618452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558457de33f0 con 0x55845816aa30 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb12ffd640 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816b420 0x55845816ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feafbfff640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feafc0a2f20 0x7feafc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb127fc640 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x558458172a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:44.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb12ffd640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feafc0a2f20 0x7feafc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feafbfff640 1 -- 172.21.15.105:0/3960249149 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7feb080cf700 con 0x7feb141019b0 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.758+0000 7feb137fe640 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x55845816ae50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feb0c037040 tx=0x7feb0c005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f582168e8c8c03 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.759+0000 7feaf97fa640 1 -- 172.21.15.105:0/752618452 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c043070 con 0x55845816aa30 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.759+0000 7feaf97fa640 1 -- 172.21.15.105:0/752618452 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb0c032db0 con 0x55845816aa30 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.759+0000 7feaf97fa640 1 -- 172.21.15.105:0/752618452 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c039040 con 0x55845816aa30 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.759+0000 7feb12ffd640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feafc0a2f20 0x7feafc0a53e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55845816b820 tx=0x7feb04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.759+0000 7feb1c009740 1 -- 172.21.15.105:0/752618452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 msgr2=0x55845816ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.759+0000 7feb1c009740 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x55845816ae50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feb0c037040 tx=0x7feb0c005f80 comp rx=0 tx=0).stop 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 -- 172.21.15.105:0/752618452 shutdown_connections 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x558458172a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816b420 0x55845816ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 --2- 172.21.15.105:0/752618452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x55845816ae50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 -- 172.21.15.105:0/752618452 >> 172.21.15.105:0/752618452 conn(0x558458165fa0 msgr2=0x5584581663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 -- 172.21.15.105:0/752618452 shutdown_connections 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 -- 172.21.15.105:0/752618452 wait complete. 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 -- start start 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 Processor -- start 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.760+0000 7feb1c009740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb1c009740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816b420 0x5584581754c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb1c009740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x55845817c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb1c009740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5584581734e0 con 0x55845816aa30 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb1c009740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55845808b310 con 0x558458170590 2024-03-26T17:57:44.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb1c009740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558458173660 con 0x55845816b420 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb127fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816b420 0x5584581754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55630/0 (socket says 172.21.15.105:55630) 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb12ffd640 1 -- 172.21.15.105:0/1086529505 learned_addr learned my addr 172.21.15.105:0/1086529505 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb1c009740 10 client.0 _ll_register_callbacks cb 0x5584580c6c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb137fe640 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x55845817c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb127fc640 1 -- 172.21.15.105:0/1086529505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 msgr2=0x55845817c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb127fc640 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x55845817c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb127fc640 1 -- 172.21.15.105:0/1086529505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 msgr2=0x558458174f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb127fc640 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb127fc640 1 -- 172.21.15.105:0/1086529505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558457ead7d0 con 0x55845816b420 2024-03-26T17:57:44.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb12ffd640 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb137fe640 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x55845817c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feb127fc640 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816b420 0x5584581754c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feb080cd210 tx=0x7feb0809bae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45d267f6da063d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feaf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.761+0000 7feaf6ff5640 1 -- 172.21.15.105:0/1086529505 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0809ff40 con 0x55845816b420 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feaf6ff5640 1 -- 172.21.15.105:0/1086529505 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7feb080cfbb0 con 0x55845816b420 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 1 -- 172.21.15.105:0/1086529505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0c032db0 con 0x55845816b420 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feaf6ff5640 1 -- 172.21.15.105:0/1086529505 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7feb080a4800 con 0x55845816b420 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 10 client.32101 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 1 -- 172.21.15.105:0/1086529505 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55845817f870 con 0x55845816b420 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feaf6ff5640 1 -- 172.21.15.105:0/1086529505 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb0c032db0 con 0x55845816b420 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feaf6ff5640 1 -- 172.21.15.105:0/1086529505 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7feb0809b2f0 con 0x55845816b420 2024-03-26T17:57:44.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 10 client.32101 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 10 client.32101 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:57:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 20 client.32101 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558458180fc0 0x558458183400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 4 client.32101 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb1c009740 1 -- 172.21.15.105:0/1086529505 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584580dceb0 con 0x558458180fc0 2024-03-26T17:57:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.762+0000 7feb12ffd640 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558458180fc0 0x558458183400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.763+0000 7feaf6ff5640 10 client.32101 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:57:44.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.763+0000 7feb12ffd640 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558458180fc0 0x558458183400 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d94c7a4472afd4af server_cookie=f24440e6f4f4c26a in_seq=0 out_seq=0 2024-03-26T17:57:44.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feaf6ff5640 1 -- 172.21.15.105:0/1086529505 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5584580dceb0 con 0x558458180fc0 2024-03-26T17:57:44.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feaf6ff5640 10 client.32101 handle_command_reply: tid=0 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 client.32101 shutdown 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 -- 172.21.15.105:0/1086529505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816b420 msgr2=0x5584581754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816b420 0x5584581754c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feb080cd210 tx=0x7feb0809bae0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 -- 172.21.15.105:0/1086529505 shutdown_connections 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x558458180fc0 0x558458183400 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feaf6ff5640 0 client.32101 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558458170590 0x55845817c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816b420 0x5584581754c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 --2- 172.21.15.105:0/1086529505 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 -- 172.21.15.105:0/1086529505 >> 172.21.15.105:0/1086529505 conn(0x558458165fa0 msgr2=0x5584581651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 -- 172.21.15.105:0/1086529505 shutdown_connections 2024-03-26T17:57:44.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 1 -- 172.21.15.105:0/1086529505 wait complete. 2024-03-26T17:57:44.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.764+0000 7feb1c009740 20 client.32101 trim_cache size 0 max 16384 2024-03-26T17:57:44.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.784+0000 7feb1c009740 1 Processor -- start 2024-03-26T17:57:44.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.784+0000 7feb1c009740 1 -- start start 2024-03-26T17:57:44.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.784+0000 7feb1c009740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.784+0000 7feb1c009740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x55845817c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.784+0000 7feb1c009740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558458170590 0x5584581754c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.784+0000 7feb1c009740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb141aaff0 con 0x55845816aa30 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.784+0000 7feb1c009740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb141acb30 con 0x55845816b420 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.784+0000 7feb1c009740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ab6c0 con 0x558458170590 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55634/0 (socket says 172.21.15.105:55634) 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 -- 172.21.15.105:0/3045108900 learned_addr learned my addr 172.21.15.105:0/3045108900 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb137fe640 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558458170590 0x5584581754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb127fc640 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x55845817c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 -- 172.21.15.105:0/3045108900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558458170590 msgr2=0x5584581754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558458170590 0x5584581754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 -- 172.21.15.105:0/3045108900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 msgr2=0x55845817c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x55845817c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 -- 172.21.15.105:0/3045108900 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55845816c3c0 con 0x55845816aa30 2024-03-26T17:57:44.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feb12ffd640 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7feb04009580 tx=0x7feb0401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2f7e26694af7d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.785+0000 7feaf8ff9640 1 -- 172.21.15.105:0/3045108900 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04004020 con 0x55845816aa30 2024-03-26T17:57:44.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.786+0000 7feaf8ff9640 1 -- 172.21.15.105:0/3045108900 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb0401de10 con 0x55845816aa30 2024-03-26T17:57:44.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.786+0000 7feaf8ff9640 1 -- 172.21.15.105:0/3045108900 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb040047a0 con 0x55845816aa30 2024-03-26T17:57:44.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.786+0000 7feb1c009740 1 -- 172.21.15.105:0/3045108900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 msgr2=0x558458174f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.786+0000 7feb1c009740 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7feb04009580 tx=0x7feb0401b000 comp rx=0 tx=0).stop 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.786+0000 7feb1c009740 1 -- 172.21.15.105:0/3045108900 shutdown_connections 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.786+0000 7feb1c009740 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558458170590 0x5584581754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x55845817c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 --2- 172.21.15.105:0/3045108900 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55845816aa30 0x558458174f80 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 -- 172.21.15.105:0/3045108900 >> 172.21.15.105:0/3045108900 conn(0x558458165fa0 msgr2=0x55845816bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 -- 172.21.15.105:0/3045108900 shutdown_connections 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 -- 172.21.15.105:0/3045108900 wait complete. 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 -- start start 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 Processor -- start 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x558458179390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x5584581798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558458170590 0x5584581e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feb141ac460 con 0x558458170590 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feb141abd90 con 0x55845816b420 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb141a8fd0 con 0x55845816aa30 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb1c009740 10 client.0 _ll_register_callbacks cb 0x7feb0809bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558458170590 0x5584581e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb127fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x5584581798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb12ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x558458179390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558458170590 0x5584581e52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55640/0 (socket says 172.21.15.105:55640) 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.787+0000 7feb137fe640 1 -- 172.21.15.105:0/940011104 learned_addr learned my addr 172.21.15.105:0/940011104 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb137fe640 1 -- 172.21.15.105:0/940011104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 msgr2=0x558458179390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb137fe640 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x558458179390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb137fe640 1 -- 172.21.15.105:0/940011104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 msgr2=0x5584581798d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb137fe640 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x5584581798d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb137fe640 1 -- 172.21.15.105:0/940011104 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584581e5830 con 0x558458170590 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb12ffd640 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x558458179390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb127fc640 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x5584581798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb137fe640 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558458170590 0x5584581e52f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7feb0c032420 tx=0x7feb0c032990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80d7d8c1b27b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feaf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feaf6ff5640 1 -- 172.21.15.105:0/940011104 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c050060 con 0x558458170590 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feaf6ff5640 1 -- 172.21.15.105:0/940011104 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7feb0c003d80 con 0x558458170590 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.788+0000 7feb1c009740 1 -- 172.21.15.105:0/940011104 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0401de10 con 0x558458170590 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.789+0000 7feaf6ff5640 1 -- 172.21.15.105:0/940011104 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7feb0c039030 con 0x558458170590 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.789+0000 7feb1c009740 10 client.22431 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.789+0000 7feb1c009740 1 -- 172.21.15.105:0/940011104 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5584581e6ba0 con 0x558458170590 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.789+0000 7feaf6ff5640 1 -- 172.21.15.105:0/940011104 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb0401de10 con 0x558458170590 2024-03-26T17:57:44.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feaf6ff5640 1 -- 172.21.15.105:0/940011104 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7feb0c048450 con 0x558458170590 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feb1c009740 10 client.22431 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feb1c009740 10 client.22431 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feb1c009740 20 client.22431 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feb1c009740 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5584581e7170 0x5584581e9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feb1c009740 4 client.22431 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feb1c009740 1 -- 172.21.15.105:0/940011104 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584580d7200 con 0x5584581e7170 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feb12ffd640 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5584581e7170 0x5584581e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feaf6ff5640 10 client.22431 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.790+0000 7feb12ffd640 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5584581e7170 0x5584581e9590 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72862bf54349132f server_cookie=9168e387b4c27c5d in_seq=0 out_seq=0 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feaf6ff5640 1 -- 172.21.15.105:0/940011104 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584580d7200 con 0x5584581e7170 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feaf6ff5640 10 client.22431 handle_command_reply: tid=0 2024-03-26T17:57:44.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feb1c009740 1 client.22431 shutdown 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feb1c009740 1 -- 172.21.15.105:0/940011104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558458170590 msgr2=0x5584581e52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feb1c009740 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558458170590 0x5584581e52f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7feb0c032420 tx=0x7feb0c032990 comp rx=0 tx=0).stop 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feb1c009740 1 -- 172.21.15.105:0/940011104 shutdown_connections 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feb1c009740 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5584581e7170 0x5584581e9590 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feb1c009740 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558458170590 0x5584581e52f0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feb1c009740 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55845816b420 0x5584581798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feb1c009740 1 --2- 172.21.15.105:0/940011104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55845816aa30 0x558458179390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.791+0000 7feaf6ff5640 0 client.22431 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.792+0000 7feb1c009740 1 -- 172.21.15.105:0/940011104 >> 172.21.15.105:0/940011104 conn(0x558458165fa0 msgr2=0x55845816c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.792+0000 7feb1c009740 1 -- 172.21.15.105:0/940011104 shutdown_connections 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.792+0000 7feb1c009740 1 -- 172.21.15.105:0/940011104 wait complete. 2024-03-26T17:57:44.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.792+0000 7feb1c009740 20 client.22431 trim_cache size 0 max 16384 2024-03-26T17:57:44.782 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 -- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feafc0a2f20 msgr2=0x7feafc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feafc0a2f20 0x7feafc0a53e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55845816b820 tx=0x7feb04005f90 comp rx=0 tx=0).stop 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 -- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb141019b0 msgr2=0x7feb14102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb141019b0 0x7feb14102d90 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7feb0800e9f0 tx=0x7feb0800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 -- 172.21.15.105:0/3960249149 shutdown_connections 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7feafc0a2f20 0x7feafc0a53e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14102370 0x7feb141032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feb141019b0 0x7feb14102d90 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 --2- 172.21.15.105:0/3960249149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feb141007c0 0x7feb14104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.793+0000 7feb1933b640 1 -- 172.21.15.105:0/3960249149 >> 172.21.15.105:0/3960249149 conn(0x7feb140fbd50 msgr2=0x7feb140fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.794+0000 7feb1933b640 1 -- 172.21.15.105:0/3960249149 shutdown_connections 2024-03-26T17:57:44.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.794+0000 7feb1933b640 1 -- 172.21.15.105:0/3960249149 wait complete. 2024-03-26T17:57:44.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:57:44.884 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:44 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/343505338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:57:44.907 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:44 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/343505338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:57:44.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 -- 172.21.15.105:0/2140909183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 msgr2=0x7f6cb01082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 --2- 172.21.15.105:0/2140909183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 0x7f6cb01082b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009ca0 tx=0x7f6ca002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 -- 172.21.15.105:0/2140909183 shutdown_connections 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 --2- 172.21.15.105:0/2140909183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb0109a30 0x7f6cb0110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 --2- 172.21.15.105:0/2140909183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb0109070 0x7f6cb01094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 --2- 172.21.15.105:0/2140909183 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 0x7f6cb01082b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 -- 172.21.15.105:0/2140909183 >> 172.21.15.105:0/2140909183 conn(0x7f6cb0076e70 msgr2=0x7f6cb00792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 -- 172.21.15.105:0/2140909183 shutdown_connections 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.957+0000 7f6cb635b640 1 -- 172.21.15.105:0/2140909183 wait complete. 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb635b640 1 Processor -- start 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb635b640 1 -- start start 2024-03-26T17:57:44.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb635b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 0x7f6cb019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb635b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb0109070 0x7f6cb019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb635b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb0109a30 0x7f6cb01a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb635b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0113a10 con 0x7f6cb0109070 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb635b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0113890 con 0x7f6cb0107e90 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb635b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0113b90 con 0x7f6cb0109a30 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6ca77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb0109070 0x7f6cb019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6caffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 0x7f6cb019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6caffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 0x7f6cb019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52122/0 (socket says 172.21.15.105:52122) 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6ca77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb0109070 0x7f6cb019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55646/0 (socket says 172.21.15.105:55646) 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6caffff640 1 -- 172.21.15.105:0/3601202355 learned_addr learned my addr 172.21.15.105:0/3601202355 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.958+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb0109a30 0x7f6cb01a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6caffff640 1 -- 172.21.15.105:0/3601202355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb0109a30 msgr2=0x7f6cb01a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6caffff640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb0109a30 0x7f6cb01a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6caffff640 1 -- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb0109070 msgr2=0x7f6cb019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6caffff640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb0109070 0x7f6cb019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6caffff640 1 -- 172.21.15.105:0/3601202355 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb01a5d90 con 0x7f6cb0107e90 2024-03-26T17:57:44.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb0109a30 0x7f6cb01a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb0109070 0x7f6cb019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6caffff640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 0x7f6cb019e170 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0032380 tx=0x7f6ca0032820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6cadffb640 1 -- 172.21.15.105:0/3601202355 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0041020 con 0x7f6cb0107e90 2024-03-26T17:57:44.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6cb635b640 1 -- 172.21.15.105:0/3601202355 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb01a6020 con 0x7f6cb0107e90 2024-03-26T17:57:44.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6cadffb640 1 -- 172.21.15.105:0/3601202355 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ca0003e20 con 0x7f6cb0107e90 2024-03-26T17:57:44.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6cadffb640 1 -- 172.21.15.105:0/3601202355 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0039040 con 0x7f6cb0107e90 2024-03-26T17:57:44.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.959+0000 7f6cb635b640 1 -- 172.21.15.105:0/3601202355 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb01a6560 con 0x7f6cb0107e90 2024-03-26T17:57:44.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.962+0000 7f6cadffb640 1 -- 172.21.15.105:0/3601202355 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6ca0044020 con 0x7f6cb0107e90 2024-03-26T17:57:44.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb9021740 1 Processor -- start 2024-03-26T17:57:44.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb9021740 1 -- start start 2024-03-26T17:57:44.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cadffb640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c9c0a2f20 0x7f6c9c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc1ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 0x5577acc21f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc24a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb9021740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5577acb81970 con 0x5577acc22590 2024-03-26T17:57:44.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb9021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5577acb29d70 con 0x5577acc1d420 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb9021740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5577acb7d920 con 0x5577acc1ca30 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cadffb640 1 -- 172.21.15.105:0/3601202355 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f6ca00f13e0 con 0x7f6cb0107e90 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c9c0a2f20 0x7f6c9c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6caffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc1ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6caffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc1ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39626/0 (socket says 172.21.15.105:39626) 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6caffff640 1 -- 172.21.15.105:0/2857377584 learned_addr learned my addr 172.21.15.105:0/2857377584 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc24a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.963+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 0x5577acc21f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.964+0000 7f6cb48f8640 1 -- 172.21.15.105:0/2857377584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 msgr2=0x5577acc1ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.964+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc1ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.964+0000 7f6cb48f8640 1 -- 172.21.15.105:0/2857377584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 msgr2=0x5577acc21f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.964+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 0x5577acc21f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.964+0000 7f6cb48f8640 1 -- 172.21.15.105:0/2857377584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577ac8953f0 con 0x5577acc22590 2024-03-26T17:57:44.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.964+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 0x5577acc21f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:44.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.964+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c9c0a2f20 0x7f6c9c0a53e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80095f0 tx=0x7f6ca8011000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.965+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc24a20 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8013d00 tx=0x7f6ca800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a7e7fb78422ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.965+0000 7f6ca67fc640 1 -- 172.21.15.105:0/2857377584 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8013580 con 0x5577acc22590 2024-03-26T17:57:44.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.965+0000 7f6ca67fc640 1 -- 172.21.15.105:0/2857377584 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ca8013180 con 0x5577acc22590 2024-03-26T17:57:44.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.965+0000 7f6ca67fc640 1 -- 172.21.15.105:0/2857377584 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8004a00 con 0x5577acc22590 2024-03-26T17:57:44.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 -- 172.21.15.105:0/2857377584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 msgr2=0x5577acc24a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc24a20 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8013d00 tx=0x7f6ca800f790 comp rx=0 tx=0).stop 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 -- 172.21.15.105:0/2857377584 shutdown_connections 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc24a20 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 0x5577acc21f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2857377584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc1ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 -- 172.21.15.105:0/2857377584 >> 172.21.15.105:0/2857377584 conn(0x5577acc17fa0 msgr2=0x5577acc183a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 -- 172.21.15.105:0/2857377584 shutdown_connections 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 -- 172.21.15.105:0/2857377584 wait complete. 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.966+0000 7f6cb9021740 1 -- start start 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb9021740 1 Processor -- start 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1ca30 0x5577acc38020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc38560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc22590 0x5577acc3f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb9021740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5577acc364f0 con 0x5577acc1d420 2024-03-26T17:57:44.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb9021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5577acb3d310 con 0x5577acc1ca30 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb9021740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5577acc36670 con 0x5577acc22590 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb9021740 10 client.0 _ll_register_callbacks cb 0x5577acb78c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6ca77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc38560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6ca77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc38560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55652/0 (socket says 172.21.15.105:55652) 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6ca77fe640 1 -- 172.21.15.105:0/934439721 learned_addr learned my addr 172.21.15.105:0/934439721 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6caffff640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1ca30 0x5577acc38020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc22590 0x5577acc3f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6ca77fe640 1 -- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc22590 msgr2=0x5577acc3f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc22590 0x5577acc3f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6ca77fe640 1 -- 172.21.15.105:0/934439721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1ca30 msgr2=0x5577acc38020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1ca30 0x5577acc38020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6ca77fe640 1 -- 172.21.15.105:0/934439721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577ac95f7d0 con 0x5577acc1d420 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6caffff640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1ca30 0x5577acc38020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:44.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.967+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc22590 0x5577acc3f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc38560 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f6c98012960 tx=0x7f6c98012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5611af816ded5ed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6c8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/934439721 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800fa00 con 0x5577acc1d420 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6cb9021740 1 -- 172.21.15.105:0/934439721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca8013180 con 0x5577acc1d420 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/934439721 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f6c9800fba0 con 0x5577acc1d420 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/934439721 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6c9800fda0 con 0x5577acc1d420 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6cb9021740 10 client.22443 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6cb9021740 1 -- 172.21.15.105:0/934439721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5577acc42970 con 0x5577acc1d420 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/934439721 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ca8013180 con 0x5577acc1d420 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/934439721 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f6c98010430 con 0x5577acc1d420 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6cb9021740 10 client.22443 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6cb9021740 10 client.22443 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:57:44.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.968+0000 7f6cb9021740 20 client.22443 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.969+0000 7f6cb9021740 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5577acc44030 0x5577acc46490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.969+0000 7f6cb9021740 4 client.22443 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.969+0000 7f6cb9021740 1 -- 172.21.15.105:0/934439721 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577acb8eeb0 con 0x5577acc44030 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.969+0000 7f6caffff640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5577acc44030 0x5577acc46490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.969+0000 7f6c8f7fe640 10 client.22443 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.969+0000 7f6caffff640 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5577acc44030 0x5577acc46490 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94c3ed8bc333b29c server_cookie=3e4fd35c5c3c3174 in_seq=0 out_seq=0 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/934439721 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5577acb8eeb0 con 0x5577acc44030 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6c8f7fe640 10 client.22443 handle_command_reply: tid=0 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 client.22443 shutdown 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 -- 172.21.15.105:0/934439721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 msgr2=0x5577acc38560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc38560 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f6c98012960 tx=0x7f6c98012e20 comp rx=0 tx=0).stop 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 -- 172.21.15.105:0/934439721 shutdown_connections 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5577acc44030 0x5577acc46490 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc22590 0x5577acc3f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc38560 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 --2- 172.21.15.105:0/934439721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1ca30 0x5577acc38020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6c8f7fe640 0 client.22443 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:57:44.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 -- 172.21.15.105:0/934439721 >> 172.21.15.105:0/934439721 conn(0x5577acc17fa0 msgr2=0x5577acc171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 -- 172.21.15.105:0/934439721 shutdown_connections 2024-03-26T17:57:44.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 1 -- 172.21.15.105:0/934439721 wait complete. 2024-03-26T17:57:44.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.970+0000 7f6cb9021740 20 client.22443 trim_cache size 0 max 16384 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.990+0000 7f6cb9021740 1 Processor -- start 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6cb9021740 1 -- start start 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc16930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 0x5577acc3f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc16e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6cb9021740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01a6bf0 con 0x5577acc22590 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6cb9021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01a8730 con 0x5577acc1d420 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6cb9021740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01a72c0 con 0x5577acc1ca30 2024-03-26T17:57:44.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6caffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc16930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6caffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc16930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39654/0 (socket says 172.21.15.105:39654) 2024-03-26T17:57:44.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6caffff640 1 -- 172.21.15.105:0/1364023134 learned_addr learned my addr 172.21.15.105:0/1364023134 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.991+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc16e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.992+0000 7f6caffff640 1 -- 172.21.15.105:0/1364023134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 msgr2=0x5577acc3f4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T17:57:44.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.992+0000 7f6caffff640 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 0x5577acc3f4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.992+0000 7f6caffff640 1 -- 172.21.15.105:0/1364023134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 msgr2=0x5577acc16e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.992+0000 7f6caffff640 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc16e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.992+0000 7f6caffff640 1 -- 172.21.15.105:0/1364023134 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577acc1d800 con 0x5577acc1ca30 2024-03-26T17:57:44.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.992+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc16e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:57:44.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.993+0000 7f6caffff640 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc16930 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0032ac0 tx=0x7f6ca00eed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354d9e87a08fe3a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.993+0000 7f6ca5ffb640 1 -- 172.21.15.105:0/1364023134 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00da070 con 0x5577acc1ca30 2024-03-26T17:57:44.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.993+0000 7f6ca5ffb640 1 -- 172.21.15.105:0/1364023134 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ca00be1d0 con 0x5577acc1ca30 2024-03-26T17:57:44.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.993+0000 7f6ca5ffb640 1 -- 172.21.15.105:0/1364023134 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00bbb70 con 0x5577acc1ca30 2024-03-26T17:57:44.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 -- 172.21.15.105:0/1364023134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 msgr2=0x5577acc16930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc16930 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0032ac0 tx=0x7f6ca00eed70 comp rx=0 tx=0).stop 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 -- 172.21.15.105:0/1364023134 shutdown_connections 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc22590 0x5577acc16e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc1d420 0x5577acc3f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 --2- 172.21.15.105:0/1364023134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc16930 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 -- 172.21.15.105:0/1364023134 >> 172.21.15.105:0/1364023134 conn(0x5577acc17fa0 msgr2=0x5577acc1ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 -- 172.21.15.105:0/1364023134 shutdown_connections 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 -- 172.21.15.105:0/1364023134 wait complete. 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.994+0000 7f6cb9021740 1 -- start start 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb9021740 1 Processor -- start 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc3ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc3d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb9021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc22590 0x5577acc3e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb9021740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01a8060 con 0x5577acc1d420 2024-03-26T17:57:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb9021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb01a7990 con 0x5577acc22590 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb9021740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb0080c70 con 0x5577acc1ca30 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb9021740 10 client.0 _ll_register_callbacks cb 0x7f6c98009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6ca77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc3d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6caffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc3ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6ca77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc3d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55668/0 (socket says 172.21.15.105:55668) 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6ca77fe640 1 -- 172.21.15.105:0/2440268000 learned_addr learned my addr 172.21.15.105:0/2440268000 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6caffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc3ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39670/0 (socket says 172.21.15.105:39670) 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6cb48f8640 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc22590 0x5577acc3e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6ca77fe640 1 -- 172.21.15.105:0/2440268000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 msgr2=0x5577acc3ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc3ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6ca77fe640 1 -- 172.21.15.105:0/2440268000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc22590 msgr2=0x5577acc3e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc22590 0x5577acc3e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6ca77fe640 1 -- 172.21.15.105:0/2440268000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577acc3e8f0 con 0x5577acc1d420 2024-03-26T17:57:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.995+0000 7f6caffff640 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc3ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6ca77fe640 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc3d340 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f6c980108d0 tx=0x7f6c98009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f34940eecf7e197 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6c8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/2440268000 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800fa00 con 0x5577acc1d420 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/2440268000 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f6c9800fba0 con 0x5577acc1d420 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6cb9021740 1 -- 172.21.15.105:0/2440268000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca00be1d0 con 0x5577acc1d420 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/2440268000 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6c98010430 con 0x5577acc1d420 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6cb9021740 10 client.22446 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6cb9021740 1 -- 172.21.15.105:0/2440268000 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5577accaa3f0 con 0x5577acc1d420 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.996+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/2440268000 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ca00be1d0 con 0x5577acc1d420 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/2440268000 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f6c98004840 con 0x5577acc1d420 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6cb9021740 10 client.22446 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6cb9021740 10 client.22446 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6cb9021740 20 client.22446 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5577accaae70 0x5577accad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6cb9021740 4 client.22446 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6cb9021740 1 -- 172.21.15.105:0/2440268000 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577acc37e10 con 0x5577accaae70 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6caffff640 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5577accaae70 0x5577accad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6caffff640 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5577accaae70 0x5577accad290 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e82ed0f712827fd server_cookie=4a98c50abcfea7a6 in_seq=0 out_seq=0 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.997+0000 7f6c8f7fe640 10 client.22446 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6c8f7fe640 1 -- 172.21.15.105:0/2440268000 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577acc37e10 con 0x5577accaae70 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6c8f7fe640 10 client.22446 handle_command_reply: tid=0 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 client.22446 shutdown 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 -- 172.21.15.105:0/2440268000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 msgr2=0x5577acc3d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc3d340 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f6c980108d0 tx=0x7f6c98009280 comp rx=0 tx=0).stop 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 -- 172.21.15.105:0/2440268000 shutdown_connections 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5577accaae70 0x5577accad290 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5577acc22590 0x5577acc3e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5577acc1d420 0x5577acc3d340 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 --2- 172.21.15.105:0/2440268000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5577acc1ca30 0x5577acc3ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 -- 172.21.15.105:0/2440268000 >> 172.21.15.105:0/2440268000 conn(0x5577acc17fa0 msgr2=0x5577acc1b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6c8f7fe640 0 client.22446 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:57:44.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 -- 172.21.15.105:0/2440268000 shutdown_connections 2024-03-26T17:57:44.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 1 -- 172.21.15.105:0/2440268000 wait complete. 2024-03-26T17:57:44.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.998+0000 7f6cb9021740 20 client.22446 trim_cache size 0 max 16384 2024-03-26T17:57:44.987 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 -- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c9c0a2f20 msgr2=0x7f6c9c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c9c0a2f20 0x7f6c9c0a53e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80095f0 tx=0x7f6ca8011000 comp rx=0 tx=0).stop 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 -- 172.21.15.105:0/3601202355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 msgr2=0x7f6cb019e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 0x7f6cb019e170 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0032380 tx=0x7f6ca0032820 comp rx=0 tx=0).stop 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 -- 172.21.15.105:0/3601202355 shutdown_connections 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6c9c0a2f20 0x7f6c9c0a53e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cb0109a30 0x7f6cb01a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cb0109070 0x7f6cb019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 --2- 172.21.15.105:0/3601202355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cb0107e90 0x7f6cb019e170 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:44.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 -- 172.21.15.105:0/3601202355 >> 172.21.15.105:0/3601202355 conn(0x7f6cb0076e70 msgr2=0x7f6cb010c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:44.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 -- 172.21.15.105:0/3601202355 shutdown_connections 2024-03-26T17:57:44.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:44.999+0000 7f6cb635b640 1 -- 172.21.15.105:0/3601202355 wait complete. 2024-03-26T17:57:44.997 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:57:45.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:44 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/343505338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 -- 172.21.15.105:0/1407077490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 msgr2=0x7fbf00100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 --2- 172.21.15.105:0/1407077490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 0x7fbf00100be0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009900 tx=0x7fbef002bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 -- 172.21.15.105:0/1407077490 shutdown_connections 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 --2- 172.21.15.105:0/1407077490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf00102370 0x7fbf00108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 --2- 172.21.15.105:0/1407077490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf001019b0 0x7fbf00101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 --2- 172.21.15.105:0/1407077490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 0x7fbf00100be0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 -- 172.21.15.105:0/1407077490 >> 172.21.15.105:0/1407077490 conn(0x7fbf000fbd50 msgr2=0x7fbf000fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 -- 172.21.15.105:0/1407077490 shutdown_connections 2024-03-26T17:57:45.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.158+0000 7fbf06027640 1 -- 172.21.15.105:0/1407077490 wait complete. 2024-03-26T17:57:45.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbf06027640 1 Processor -- start 2024-03-26T17:57:45.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbf06027640 1 -- start start 2024-03-26T17:57:45.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbf06027640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 0x7fbf0019e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbf06027640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf001019b0 0x7fbf0019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 0x7fbf0019e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 0x7fbf0019e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55678/0 (socket says 172.21.15.105:55678) 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbeff7fe640 1 -- 172.21.15.105:0/4118900871 learned_addr learned my addr 172.21.15.105:0/4118900871 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbf06027640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf00102370 0x7fbf001a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010b200 con 0x7fbf001007c0 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010b080 con 0x7fbf00102370 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0010b380 con 0x7fbf001019b0 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbefeffd640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf001019b0 0x7fbf0019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.159+0000 7fbefffff640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf00102370 0x7fbf001a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefeffd640 1 -- 172.21.15.105:0/4118900871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf00102370 msgr2=0x7fbf001a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefeffd640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf00102370 0x7fbf001a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefeffd640 1 -- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 msgr2=0x7fbf0019e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefeffd640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 0x7fbf0019e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefeffd640 1 -- 172.21.15.105:0/4118900871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf001a5d80 con 0x7fbf001019b0 2024-03-26T17:57:45.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefeffd640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf001019b0 0x7fbf0019e6a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbef400e9f0 tx=0x7fbef400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefcff9640 1 -- 172.21.15.105:0/4118900871 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef400cd80 con 0x7fbf001019b0 2024-03-26T17:57:45.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefcff9640 1 -- 172.21.15.105:0/4118900871 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbef40035f0 con 0x7fbf001019b0 2024-03-26T17:57:45.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf001a6070 con 0x7fbf001019b0 2024-03-26T17:57:45.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbefcff9640 1 -- 172.21.15.105:0/4118900871 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4010670 con 0x7fbf001019b0 2024-03-26T17:57:45.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.160+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf001a65c0 con 0x7fbf001019b0 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.163+0000 7fbf08c13740 1 Processor -- start 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.163+0000 7fbf08c13740 1 -- start start 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c833be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbefcff9640 1 -- 172.21.15.105:0/4118900871 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbef40037a0 con 0x7fbf001019b0 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c8340f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c8343a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbefffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c833be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbefffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c833be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52186/0 (socket says 172.21.15.105:52186) 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbefffff640 1 -- 172.21.15.105:0/2726035918 learned_addr learned my addr 172.21.15.105:0/2726035918 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbf08c13740 1 -- 172.21.15.105:0/2726035918 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5601c82a0970 con 0x5601c8341590 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbf08c13740 1 -- 172.21.15.105:0/2726035918 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601c8248d70 con 0x5601c833ba30 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbf08c13740 1 -- 172.21.15.105:0/2726035918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5601c829c920 con 0x5601c833c420 2024-03-26T17:57:45.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c8340f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbefeffd640 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c8343a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbeff7fe640 1 -- 172.21.15.105:0/2726035918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 msgr2=0x5601c833be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c833be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbeff7fe640 1 -- 172.21.15.105:0/2726035918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 msgr2=0x5601c8343a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c8343a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbefcff9640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbee80a2f20 0x7fbee80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.164+0000 7fbeff7fe640 1 -- 172.21.15.105:0/2726035918 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c7fb43f0 con 0x5601c833c420 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.165+0000 7fbefcff9640 1 -- 172.21.15.105:0/4118900871 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fbef40cf670 con 0x7fbf001019b0 2024-03-26T17:57:45.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.165+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c8340f20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbef0005ab0 tx=0x7fbef0005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607591a00b82a370 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.165+0000 7fbee5ffb640 1 -- 172.21.15.105:0/2726035918 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0045020 con 0x5601c833c420 2024-03-26T17:57:45.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.165+0000 7fbee5ffb640 1 -- 172.21.15.105:0/2726035918 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbef002c9a0 con 0x5601c833c420 2024-03-26T17:57:45.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.165+0000 7fbee5ffb640 1 -- 172.21.15.105:0/2726035918 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0005620 con 0x5601c833c420 2024-03-26T17:57:45.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.165+0000 7fbefffff640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbee80a2f20 0x7fbee80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbf08c13740 1 -- 172.21.15.105:0/2726035918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 msgr2=0x5601c8340f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c8340f20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fbef0005ab0 tx=0x7fbef0005df0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbf08c13740 1 -- 172.21.15.105:0/2726035918 shutdown_connections 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c8343a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c8340f20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2726035918 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c833be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbf08c13740 1 -- 172.21.15.105:0/2726035918 >> 172.21.15.105:0/2726035918 conn(0x5601c8336fa0 msgr2=0x5601c83373a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbf08c13740 1 -- 172.21.15.105:0/2726035918 shutdown_connections 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.166+0000 7fbefffff640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbee80a2f20 0x7fbee80a53e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x5601c833cc80 tx=0x7fbef8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 -- 172.21.15.105:0/2726035918 wait complete. 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 -- start start 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 Processor -- start 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c8345d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c83462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c834d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5601c83444e0 con 0x5601c833ba30 2024-03-26T17:57:45.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601c825c310 con 0x5601c833c420 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.167+0000 7fbf08c13740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5601c8344660 con 0x5601c8341590 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbf08c13740 10 client.0 _ll_register_callbacks cb 0x5601c8297c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c83462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c83462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52196/0 (socket says 172.21.15.105:52196) 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 -- 172.21.15.105:0/2262743814 learned_addr learned my addr 172.21.15.105:0/2262743814 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c8345d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 -- 172.21.15.105:0/2262743814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 msgr2=0x5601c834d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefffff640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c834d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c834d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 -- 172.21.15.105:0/2262743814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 msgr2=0x5601c8345d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c8345d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 -- 172.21.15.105:0/2262743814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c807e7d0 con 0x5601c833c420 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c8345d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefffff640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c834d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbedeffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.168+0000 7fbefeffd640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c83462d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbef40a4240 tx=0x7fbef4098000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15f58fb40184fe35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbedeffd640 1 -- 172.21.15.105:0/2262743814 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef409b6c0 con 0x5601c833c420 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 1 -- 172.21.15.105:0/2262743814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef002c9a0 con 0x5601c833c420 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbedeffd640 1 -- 172.21.15.105:0/2262743814 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fbef40983d0 con 0x5601c833c420 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbedeffd640 1 -- 172.21.15.105:0/2262743814 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbef4003b30 con 0x5601c833c420 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 10 client.32150 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:45.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 1 -- 172.21.15.105:0/2262743814 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5601c83506d0 con 0x5601c833c420 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbedeffd640 1 -- 172.21.15.105:0/2262743814 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbef002c9a0 con 0x5601c833c420 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbedeffd640 1 -- 172.21.15.105:0/2262743814 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbef4086040 con 0x5601c833c420 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 10 client.32150 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 10 client.32150 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 20 client.32150 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5601c8351d40 0x5601c83541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 4 client.32150 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.169+0000 7fbf08c13740 1 -- 172.21.15.105:0/2262743814 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601c82adeb0 con 0x5601c8351d40 2024-03-26T17:57:45.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.170+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5601c8351d40 0x5601c83541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.170+0000 7fbedeffd640 10 client.32150 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.170+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5601c8351d40 0x5601c83541a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ecc0a2f90324597 server_cookie=3069111c42213cb2 in_seq=0 out_seq=0 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.171+0000 7fbedeffd640 1 -- 172.21.15.105:0/2262743814 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5601c82adeb0 con 0x5601c8351d40 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.171+0000 7fbedeffd640 10 client.32150 handle_command_reply: tid=0 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.171+0000 7fbf08c13740 1 client.32150 shutdown 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.171+0000 7fbf08c13740 1 -- 172.21.15.105:0/2262743814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 msgr2=0x5601c83462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.171+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c83462d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbef40a4240 tx=0x7fbef4098000 comp rx=0 tx=0).stop 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 1 -- 172.21.15.105:0/2262743814 shutdown_connections 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c834d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5601c8351d40 0x5601c83541a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c83462d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2262743814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c8345d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 1 -- 172.21.15.105:0/2262743814 >> 172.21.15.105:0/2262743814 conn(0x5601c8336fa0 msgr2=0x5601c83361c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbedeffd640 0 client.32150 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:57:45.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 1 -- 172.21.15.105:0/2262743814 shutdown_connections 2024-03-26T17:57:45.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 1 -- 172.21.15.105:0/2262743814 wait complete. 2024-03-26T17:57:45.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.172+0000 7fbf08c13740 20 client.32150 trim_cache size 0 max 16384 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.192+0000 7fbf08c13740 1 Processor -- start 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.192+0000 7fbf08c13740 1 -- start start 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.192+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c833d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.192+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c834d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.192+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c833dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.192+0000 7fbf08c13740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001af2b0 con 0x5601c833ba30 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.192+0000 7fbf08c13740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001b0df0 con 0x5601c833c420 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.192+0000 7fbf08c13740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001af980 con 0x5601c8341590 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c833d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c833d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55698/0 (socket says 172.21.15.105:55698) 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 -- 172.21.15.105:0/618923055 learned_addr learned my addr 172.21.15.105:0/618923055 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbefeffd640 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c834d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbefffff640 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c833dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 -- 172.21.15.105:0/618923055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 msgr2=0x5601c833dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c833dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 -- 172.21.15.105:0/618923055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 msgr2=0x5601c834d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c834d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 -- 172.21.15.105:0/618923055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c8340aa0 con 0x5601c833ba30 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c833d7e0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fbef00096c0 tx=0x7fbef0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39444a66ffd0f4d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbee57fa640 1 -- 172.21.15.105:0/618923055 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0045040 con 0x5601c833ba30 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbee57fa640 1 -- 172.21.15.105:0/618923055 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbef004f070 con 0x5601c833ba30 2024-03-26T17:57:45.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.193+0000 7fbee57fa640 1 -- 172.21.15.105:0/618923055 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00035f0 con 0x5601c833ba30 2024-03-26T17:57:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 -- 172.21.15.105:0/618923055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 msgr2=0x5601c833d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c833d7e0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fbef00096c0 tx=0x7fbef0009260 comp rx=0 tx=0).stop 2024-03-26T17:57:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 -- 172.21.15.105:0/618923055 shutdown_connections 2024-03-26T17:57:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c8341590 0x5601c833dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833c420 0x5601c834d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 --2- 172.21.15.105:0/618923055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c833ba30 0x5601c833d7e0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 -- 172.21.15.105:0/618923055 >> 172.21.15.105:0/618923055 conn(0x5601c8336fa0 msgr2=0x5601c8337380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 -- 172.21.15.105:0/618923055 shutdown_connections 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.194+0000 7fbf08c13740 1 -- 172.21.15.105:0/618923055 wait complete. 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.195+0000 7fbf08c13740 1 -- start start 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.195+0000 7fbf08c13740 1 Processor -- start 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.195+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c834a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.195+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c834a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.195+0000 7fbf08c13740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c834b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.195+0000 7fbf08c13740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001b0720 con 0x5601c8341590 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.195+0000 7fbf08c13740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001b0050 con 0x5601c833ba30 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.195+0000 7fbf08c13740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001ad290 con 0x5601c833c420 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbf08c13740 10 client.0 _ll_register_callbacks cb 0x7fbef40d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:45.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c834a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbefeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c834a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbefffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c834b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c834a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52218/0 (socket says 172.21.15.105:52218) 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbefeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c834a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39738/0 (socket says 172.21.15.105:39738) 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 -- 172.21.15.105:0/2357876955 learned_addr learned my addr 172.21.15.105:0/2357876955 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 -- 172.21.15.105:0/2357876955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 msgr2=0x5601c834a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c834a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 -- 172.21.15.105:0/2357876955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 msgr2=0x5601c834b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c834b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 -- 172.21.15.105:0/2357876955 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c834be10 con 0x5601c833ba30 2024-03-26T17:57:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbefffff640 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c834b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbeff7fe640 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c834a340 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbef002c170 tx=0x7fbef0054860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fb0b2977479853a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbedeffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbedeffd640 1 -- 172.21.15.105:0/2357876955 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef005a070 con 0x5601c833ba30 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbf08c13740 1 -- 172.21.15.105:0/2357876955 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef004f070 con 0x5601c833ba30 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.196+0000 7fbedeffd640 1 -- 172.21.15.105:0/2357876955 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fbef0004140 con 0x5601c833ba30 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbedeffd640 1 -- 172.21.15.105:0/2357876955 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbef0009520 con 0x5601c833ba30 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbf08c13740 10 client.32156 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbf08c13740 1 -- 172.21.15.105:0/2357876955 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5601c834d110 con 0x5601c833ba30 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbedeffd640 1 -- 172.21.15.105:0/2357876955 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbef004f070 con 0x5601c833ba30 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbedeffd640 1 -- 172.21.15.105:0/2357876955 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fbef0053070 con 0x5601c833ba30 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbf08c13740 10 client.32156 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbf08c13740 10 client.32156 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbf08c13740 20 client.32156 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5601c83b80a0 0x5601c83ba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbf08c13740 4 client.32156 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbf08c13740 1 -- 172.21.15.105:0/2357876955 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601c82a8200 con 0x5601c83b80a0 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.197+0000 7fbefeffd640 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5601c83b80a0 0x5601c83ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbedeffd640 10 client.32156 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbefeffd640 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5601c83b80a0 0x5601c83ba490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a371a502664d741 server_cookie=513a186b354b29d8 in_seq=0 out_seq=0 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbedeffd640 1 -- 172.21.15.105:0/2357876955 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601c82a8200 con 0x5601c83b80a0 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbedeffd640 10 client.32156 handle_command_reply: tid=0 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 client.32156 shutdown 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 -- 172.21.15.105:0/2357876955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 msgr2=0x5601c834a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c834a340 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbef002c170 tx=0x7fbef0054860 comp rx=0 tx=0).stop 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 -- 172.21.15.105:0/2357876955 shutdown_connections 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5601c8341590 0x5601c834b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5601c833c420 0x5601c834a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5601c83b80a0 0x5601c83ba490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 --2- 172.21.15.105:0/2357876955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601c833ba30 0x5601c834a340 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbedeffd640 0 client.32156 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:57:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 -- 172.21.15.105:0/2357876955 >> 172.21.15.105:0/2357876955 conn(0x5601c8336fa0 msgr2=0x5601c8345d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.198+0000 7fbf08c13740 1 -- 172.21.15.105:0/2357876955 shutdown_connections 2024-03-26T17:57:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf08c13740 1 -- 172.21.15.105:0/2357876955 wait complete. 2024-03-26T17:57:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf08c13740 20 client.32156 trim_cache size 0 max 16384 2024-03-26T17:57:45.188 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbee80a2f20 msgr2=0x7fbee80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbee80a2f20 0x7fbee80a53e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x5601c833cc80 tx=0x7fbef8009500 comp rx=0 tx=0).stop 2024-03-26T17:57:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf001019b0 msgr2=0x7fbf0019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf001019b0 0x7fbf0019e6a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbef400e9f0 tx=0x7fbef400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 shutdown_connections 2024-03-26T17:57:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbee80a2f20 0x7fbee80a53e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf00102370 0x7fbf001a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf001019b0 0x7fbf0019e6a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 --2- 172.21.15.105:0/4118900871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf001007c0 0x7fbf0019e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.199+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 >> 172.21.15.105:0/4118900871 conn(0x7fbf000fbd50 msgr2=0x7fbf0010dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.200+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 shutdown_connections 2024-03-26T17:57:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.200+0000 7fbf06027640 1 -- 172.21.15.105:0/4118900871 wait complete. 2024-03-26T17:57:45.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:57:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 -- 172.21.15.105:0/36058550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc938104c70 msgr2=0x7fc938107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 --2- 172.21.15.105:0/36058550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc938104c70 0x7fc938107060 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fc920009900 tx=0x7fc92002bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 -- 172.21.15.105:0/36058550 shutdown_connections 2024-03-26T17:57:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 --2- 172.21.15.105:0/36058550 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc9381075a0 0x7fc9381099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 --2- 172.21.15.105:0/36058550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc938104c70 0x7fc938107060 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 --2- 172.21.15.105:0/36058550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc938102340 0x7fc938104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 -- 172.21.15.105:0/36058550 >> 172.21.15.105:0/36058550 conn(0x7fc9380fbd50 msgr2=0x7fc9380fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 -- 172.21.15.105:0/36058550 shutdown_connections 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 -- 172.21.15.105:0/36058550 wait complete. 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 Processor -- start 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.361+0000 7fc93d412640 1 -- start start 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc93d412640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc938102340 0x7fc938197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc93d412640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc938104c70 0x7fc938198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc93d412640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9381075a0 0x7fc93819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc93d412640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc93810d3e0 con 0x7fc9381075a0 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc93d412640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc93810d260 con 0x7fc938104c70 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc93d412640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc93810d560 con 0x7fc938102340 2024-03-26T17:57:45.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc938102340 0x7fc938197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc92ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc938104c70 0x7fc938198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc938102340 0x7fc938197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39766/0 (socket says 172.21.15.105:39766) 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 -- 172.21.15.105:0/3676838200 learned_addr learned my addr 172.21.15.105:0/3676838200 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc9377fe640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9381075a0 0x7fc93819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 -- 172.21.15.105:0/3676838200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc938104c70 msgr2=0x7fc938198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc938104c70 0x7fc938198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 -- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9381075a0 msgr2=0x7fc93819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9381075a0 0x7fc93819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 -- 172.21.15.105:0/3676838200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93819f760 con 0x7fc938102340 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc936ffd640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc938102340 0x7fc938197b40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc92800e9f0 tx=0x7fc92800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.362+0000 7fc934ff9640 1 -- 172.21.15.105:0/3676838200 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92800cd80 con 0x7fc938102340 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.363+0000 7fc934ff9640 1 -- 172.21.15.105:0/3676838200 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9280108e0 con 0x7fc938102340 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.363+0000 7fc93d412640 1 -- 172.21.15.105:0/3676838200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93819fa50 con 0x7fc938102340 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.363+0000 7fc934ff9640 1 -- 172.21.15.105:0/3676838200 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928018920 con 0x7fc938102340 2024-03-26T17:57:45.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.363+0000 7fc93d412640 1 -- 172.21.15.105:0/3676838200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93819ffc0 con 0x7fc938102340 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.365+0000 7fc934ff9640 1 -- 172.21.15.105:0/3676838200 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc928010a90 con 0x7fc938102340 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.365+0000 7fc934ff9640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc9240a2fa0 0x7fc9240a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc9240a2fa0 0x7fc9240a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc934ff9640 1 -- 172.21.15.105:0/3676838200 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fc9280cf6a0 con 0x7fc938102340 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc940003740 1 Processor -- start 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc940003740 1 -- start start 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc9240a2fa0 0x7fc9240a5460 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc920005ab0 tx=0x7fc920005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc940003740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e7607f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc940003740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e7607f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc940003740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e7607fca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc940003740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e760759970 con 0x55e7607f5420 2024-03-26T17:57:45.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc940003740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e760701d70 con 0x55e7607f4a30 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc940003740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e760755920 con 0x55e7607fa590 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc92ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e7607f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc92ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e7607f9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55726/0 (socket says 172.21.15.105:55726) 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.366+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e7607f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc92ffff640 1 -- 172.21.15.105:0/3644086868 learned_addr learned my addr 172.21.15.105:0/3644086868 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc9377fe640 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e7607fca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc92ffff640 1 -- 172.21.15.105:0/3644086868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 msgr2=0x55e7607fca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e7607fca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc92ffff640 1 -- 172.21.15.105:0/3644086868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 msgr2=0x55e7607f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e7607f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc92ffff640 1 -- 172.21.15.105:0/3644086868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76046d3f0 con 0x55e7607f5420 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc9377fe640 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e7607fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e7607f9f20 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc920009900 tx=0x7fc92003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68d9f2c3d679767 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.367+0000 7fc92d7fa640 1 -- 172.21.15.105:0/3644086868 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92002cc40 con 0x55e7607f5420 2024-03-26T17:57:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.368+0000 7fc92d7fa640 1 -- 172.21.15.105:0/3644086868 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9200054f0 con 0x55e7607f5420 2024-03-26T17:57:45.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.368+0000 7fc92d7fa640 1 -- 172.21.15.105:0/3644086868 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92004b620 con 0x55e7607f5420 2024-03-26T17:57:45.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 -- 172.21.15.105:0/3644086868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 msgr2=0x55e7607f9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e7607f9f20 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc920009900 tx=0x7fc92003e000 comp rx=0 tx=0).stop 2024-03-26T17:57:45.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 -- 172.21.15.105:0/3644086868 shutdown_connections 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e7607fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e7607f9f20 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 --2- 172.21.15.105:0/3644086868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e7607f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 -- 172.21.15.105:0/3644086868 >> 172.21.15.105:0/3644086868 conn(0x55e7607effa0 msgr2=0x55e7607f03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 -- 172.21.15.105:0/3644086868 shutdown_connections 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 -- 172.21.15.105:0/3644086868 wait complete. 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.369+0000 7fc940003740 1 -- start start 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc940003740 1 Processor -- start 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc940003740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f4a30 0x55e7607fef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc940003740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f5420 0x55e7607ff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc940003740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e760806490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc940003740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e7607fd4e0 con 0x55e7607f4a30 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc940003740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e760715310 con 0x55e7607fa590 2024-03-26T17:57:45.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc940003740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e7607fd660 con 0x55e7607f5420 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f4a30 0x55e7607fef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f4a30 0x55e7607fef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55738/0 (socket says 172.21.15.105:55738) 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc936ffd640 1 -- 172.21.15.105:0/3629614542 learned_addr learned my addr 172.21.15.105:0/3629614542 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f5420 0x55e7607ff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc940003740 10 client.0 _ll_register_callbacks cb 0x55e760750c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc9377fe640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e760806490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc936ffd640 1 -- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f5420 msgr2=0x55e7607ff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc936ffd640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f5420 0x55e7607ff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc936ffd640 1 -- 172.21.15.105:0/3629614542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 msgr2=0x55e760806490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc936ffd640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e760806490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc936ffd640 1 -- 172.21.15.105:0/3629614542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7605377d0 con 0x55e7607f4a30 2024-03-26T17:57:45.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f5420 0x55e7607ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.370+0000 7fc9377fe640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e760806490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc936ffd640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f4a30 0x55e7607fef10 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fc92809bdd0 tx=0x7fc928010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8801f13e4f04ccf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc91e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc91e7fc640 1 -- 172.21.15.105:0/3629614542 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928010630 con 0x55e7607f4a30 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc91e7fc640 1 -- 172.21.15.105:0/3629614542 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fc92809b290 con 0x55e7607f4a30 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc940003740 1 -- 172.21.15.105:0/3629614542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9200054f0 con 0x55e7607f4a30 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc91e7fc640 1 -- 172.21.15.105:0/3629614542 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9280a47a0 con 0x55e7607f4a30 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc940003740 10 client.22476 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc940003740 1 -- 172.21.15.105:0/3629614542 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e7608098b0 con 0x55e7607f4a30 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc91e7fc640 1 -- 172.21.15.105:0/3629614542 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9200054f0 con 0x55e7607f4a30 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.371+0000 7fc91e7fc640 1 -- 172.21.15.105:0/3629614542 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc928018ac0 con 0x55e7607f4a30 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc940003740 10 client.22476 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc940003740 10 client.22476 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc940003740 20 client.22476 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:45.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc940003740 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e76080af00 0x55e76080d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc940003740 4 client.22476 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc940003740 1 -- 172.21.15.105:0/3629614542 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e760766eb0 con 0x55e76080af00 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e76080af00 0x55e76080d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc92ffff640 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e76080af00 0x55e76080d370 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6370929559cb130 server_cookie=a43dbdb611efae45 in_seq=0 out_seq=0 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.372+0000 7fc91e7fc640 10 client.22476 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.373+0000 7fc91e7fc640 1 -- 172.21.15.105:0/3629614542 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e760766eb0 con 0x55e76080af00 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.373+0000 7fc91e7fc640 10 client.22476 handle_command_reply: tid=0 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.373+0000 7fc940003740 1 client.22476 shutdown 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 -- 172.21.15.105:0/3629614542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f4a30 msgr2=0x55e7607fef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f4a30 0x55e7607fef10 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fc92809bdd0 tx=0x7fc928010ea0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 -- 172.21.15.105:0/3629614542 shutdown_connections 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e76080af00 0x55e76080d370 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e760806490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc91e7fc640 0 client.22476 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f5420 0x55e7607ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 --2- 172.21.15.105:0/3629614542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f4a30 0x55e7607fef10 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 -- 172.21.15.105:0/3629614542 >> 172.21.15.105:0/3629614542 conn(0x55e7607effa0 msgr2=0x55e7607ef1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 -- 172.21.15.105:0/3629614542 shutdown_connections 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 1 -- 172.21.15.105:0/3629614542 wait complete. 2024-03-26T17:57:45.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.374+0000 7fc940003740 20 client.22476 trim_cache size 0 max 16384 2024-03-26T17:57:45.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.395+0000 7fc940003740 1 Processor -- start 2024-03-26T17:57:45.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.395+0000 7fc940003740 1 -- start start 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc940003740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f4a30 0x55e7607ee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc940003740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760806450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc940003740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e7607eee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc940003740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381a6be0 con 0x55e7607f5420 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc940003740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381a8720 con 0x55e7607fa590 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc940003740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381a72b0 con 0x55e7607f4a30 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f4a30 0x55e7607ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f4a30 0x55e7607ee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39800/0 (socket says 172.21.15.105:39800) 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc936ffd640 1 -- 172.21.15.105:0/903805625 learned_addr learned my addr 172.21.15.105:0/903805625 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc9377fe640 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e7607eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc92ffff640 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760806450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc936ffd640 1 -- 172.21.15.105:0/903805625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 msgr2=0x55e7607eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc936ffd640 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e7607eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc936ffd640 1 -- 172.21.15.105:0/903805625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 msgr2=0x55e760806450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc936ffd640 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760806450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc936ffd640 1 -- 172.21.15.105:0/903805625 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7607f0d10 con 0x55e7607f4a30 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc92ffff640 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760806450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.396+0000 7fc9377fe640 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e7607eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.397+0000 7fc936ffd640 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f4a30 0x55e7607ee930 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc92809bb40 tx=0x7fc9280bc0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5846f77ed52ae8dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.397+0000 7fc92cff9640 1 -- 172.21.15.105:0/903805625 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92809b980 con 0x55e7607f4a30 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.397+0000 7fc92cff9640 1 -- 172.21.15.105:0/903805625 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc92809b290 con 0x55e7607f4a30 2024-03-26T17:57:45.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.397+0000 7fc92cff9640 1 -- 172.21.15.105:0/903805625 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928097b90 con 0x55e7607f4a30 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 -- 172.21.15.105:0/903805625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f4a30 msgr2=0x55e7607ee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f4a30 0x55e7607ee930 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc92809bb40 tx=0x7fc9280bc0c0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 -- 172.21.15.105:0/903805625 shutdown_connections 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607fa590 0x55e7607eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760806450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 --2- 172.21.15.105:0/903805625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607f4a30 0x55e7607ee930 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 -- 172.21.15.105:0/903805625 >> 172.21.15.105:0/903805625 conn(0x55e7607effa0 msgr2=0x55e7607f6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 -- 172.21.15.105:0/903805625 shutdown_connections 2024-03-26T17:57:45.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 -- 172.21.15.105:0/903805625 wait complete. 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.398+0000 7fc940003740 1 -- start start 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc940003740 1 Processor -- start 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc940003740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e760803420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc940003740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760803960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc940003740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e76086f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc940003740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381a8050 con 0x55e7607f5420 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc940003740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381a7980 con 0x55e7607f4a30 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc940003740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc9381a4bc0 con 0x55e7607fa590 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc940003740 10 client.0 _ll_register_callbacks cb 0x7fc92809bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc92ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760803960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e760803420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc92ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760803960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55758/0 (socket says 172.21.15.105:55758) 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc9377fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e76086f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e760803420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52288/0 (socket says 172.21.15.105:52288) 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc92ffff640 1 -- 172.21.15.105:0/838650938 learned_addr learned my addr 172.21.15.105:0/838650938 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc936ffd640 1 -- 172.21.15.105:0/838650938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 msgr2=0x55e76086f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc936ffd640 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e76086f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc936ffd640 1 -- 172.21.15.105:0/838650938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 msgr2=0x55e760803960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc936ffd640 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760803960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.399+0000 7fc936ffd640 1 -- 172.21.15.105:0/838650938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76086f820 con 0x55e7607f4a30 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc92ffff640 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760803960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc936ffd640 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e760803420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc928098810 tx=0x7fc92809b1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a28f771de76f3af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc91e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc91e7fc640 1 -- 172.21.15.105:0/838650938 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92809b980 con 0x55e7607f4a30 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc940003740 1 -- 172.21.15.105:0/838650938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92809b290 con 0x55e7607f4a30 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc91e7fc640 1 -- 172.21.15.105:0/838650938 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fc928086b50 con 0x55e7607f4a30 2024-03-26T17:57:45.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc91e7fc640 1 -- 172.21.15.105:0/838650938 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc9280b0170 con 0x55e7607f4a30 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc940003740 10 client.32174 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.400+0000 7fc940003740 1 -- 172.21.15.105:0/838650938 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e76086f820 con 0x55e7607f4a30 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc91e7fc640 1 -- 172.21.15.105:0/838650938 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc92809b290 con 0x55e7607f4a30 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc91e7fc640 1 -- 172.21.15.105:0/838650938 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fc9280ab070 con 0x55e7607f4a30 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc940003740 10 client.32174 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc940003740 10 client.32174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc940003740 20 client.32174 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc940003740 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e760870fc0 0x55e7608733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc940003740 4 client.32174 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc940003740 1 -- 172.21.15.105:0/838650938 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7607fa300 con 0x55e760870fc0 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc92ffff640 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e760870fc0 0x55e7608733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc91e7fc640 10 client.32174 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:57:45.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.401+0000 7fc92ffff640 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e760870fc0 0x55e7608733e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=621e4f7adf7055cc server_cookie=cfdec6fe963d26c7 in_seq=0 out_seq=0 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.402+0000 7fc91e7fc640 1 -- 172.21.15.105:0/838650938 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7607fa300 con 0x55e760870fc0 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.402+0000 7fc91e7fc640 10 client.32174 handle_command_reply: tid=0 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.402+0000 7fc940003740 1 client.32174 shutdown 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.402+0000 7fc940003740 1 -- 172.21.15.105:0/838650938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 msgr2=0x55e760803420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.402+0000 7fc940003740 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e760803420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc928098810 tx=0x7fc92809b1e0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 1 -- 172.21.15.105:0/838650938 shutdown_connections 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55e760870fc0 0x55e7608733e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e7607fa590 0x55e76086f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e7607f5420 0x55e760803960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 1 --2- 172.21.15.105:0/838650938 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e7607f4a30 0x55e760803420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 1 -- 172.21.15.105:0/838650938 >> 172.21.15.105:0/838650938 conn(0x55e7607effa0 msgr2=0x55e7607f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc91e7fc640 0 client.32174 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 1 -- 172.21.15.105:0/838650938 shutdown_connections 2024-03-26T17:57:45.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 1 -- 172.21.15.105:0/838650938 wait complete. 2024-03-26T17:57:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.403+0000 7fc940003740 20 client.32174 trim_cache size 0 max 16384 2024-03-26T17:57:45.393 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:45.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 -- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc9240a2fa0 msgr2=0x7fc9240a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc9240a2fa0 0x7fc9240a5460 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc920005ab0 tx=0x7fc920005fd0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 -- 172.21.15.105:0/3676838200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc938102340 msgr2=0x7fc938197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc938102340 0x7fc938197b40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc92800e9f0 tx=0x7fc92800eeb0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 -- 172.21.15.105:0/3676838200 shutdown_connections 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc9240a2fa0 0x7fc9240a5460 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9381075a0 0x7fc93819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc938104c70 0x7fc938198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 --2- 172.21.15.105:0/3676838200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc938102340 0x7fc938197b40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 -- 172.21.15.105:0/3676838200 >> 172.21.15.105:0/3676838200 conn(0x7fc9380fbd50 msgr2=0x7fc9381041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 -- 172.21.15.105:0/3676838200 shutdown_connections 2024-03-26T17:57:45.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.404+0000 7fc93d412640 1 -- 172.21.15.105:0/3676838200 wait complete. 2024-03-26T17:57:45.403 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:57:45.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 -- 172.21.15.105:0/3372375759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 msgr2=0x7f35dc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 --2- 172.21.15.105:0/3372375759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 0x7f35dc100be0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009900 tx=0x7f35d802bc30 comp rx=0 tx=0).stop 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 -- 172.21.15.105:0/3372375759 shutdown_connections 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 --2- 172.21.15.105:0/3372375759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc102370 0x7f35dc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 --2- 172.21.15.105:0/3372375759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc1019b0 0x7f35dc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 --2- 172.21.15.105:0/3372375759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 0x7f35dc100be0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 -- 172.21.15.105:0/3372375759 >> 172.21.15.105:0/3372375759 conn(0x7f35dc0fbd50 msgr2=0x7f35dc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 -- 172.21.15.105:0/3372375759 shutdown_connections 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.568+0000 7f35e35b3640 1 -- 172.21.15.105:0/3372375759 wait complete. 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e35b3640 1 Processor -- start 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e35b3640 1 -- start start 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e35b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 0x7f35dc104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e35b3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc1019b0 0x7f35dc102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e35b3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc102370 0x7f35dc1032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e35b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc10c2e0 con 0x7f35dc1007c0 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e35b3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc10c160 con 0x7f35dc102370 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e35b3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc10c460 con 0x7f35dc1019b0 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e134f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 0x7f35dc104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e134f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 0x7f35dc104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55784/0 (socket says 172.21.15.105:55784) 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e134f640 1 -- 172.21.15.105:0/4073931430 learned_addr learned my addr 172.21.15.105:0/4073931430 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e1b50640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc102370 0x7f35dc1032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e0b4e640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc1019b0 0x7f35dc102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e134f640 1 -- 172.21.15.105:0/4073931430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc1019b0 msgr2=0x7f35dc102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e134f640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc1019b0 0x7f35dc102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e134f640 1 -- 172.21.15.105:0/4073931430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc102370 msgr2=0x7f35dc1032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e134f640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc102370 0x7f35dc1032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.569+0000 7f35e134f640 1 -- 172.21.15.105:0/4073931430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc103b60 con 0x7f35dc1007c0 2024-03-26T17:57:45.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.570+0000 7f35e134f640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 0x7f35dc104750 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f35d8000c00 tx=0x7f35d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.570+0000 7f35d27fc640 1 -- 172.21.15.105:0/4073931430 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8043020 con 0x7f35dc1007c0 2024-03-26T17:57:45.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.570+0000 7f35e35b3640 1 -- 172.21.15.105:0/4073931430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc1a1c90 con 0x7f35dc1007c0 2024-03-26T17:57:45.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.570+0000 7f35e35b3640 1 -- 172.21.15.105:0/4073931430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc1a2180 con 0x7f35dc1007c0 2024-03-26T17:57:45.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.570+0000 7f35d27fc640 1 -- 172.21.15.105:0/4073931430 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35d8004aa0 con 0x7f35dc1007c0 2024-03-26T17:57:45.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.570+0000 7f35d27fc640 1 -- 172.21.15.105:0/4073931430 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8003d00 con 0x7f35dc1007c0 2024-03-26T17:57:45.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 50957 7.07 MB/sec execute 268 sec latency 130.303 ms 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.573+0000 7f35d27fc640 1 -- 172.21.15.105:0/4073931430 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f35d8046020 con 0x7f35dc1007c0 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.573+0000 7f35e6199740 1 Processor -- start 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e6199740 1 -- start start 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35d27fc640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35c80a3020 0x7f35c80a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e6199740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d967a30 0x55d37d967e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e6199740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d96cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e6199740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d96d590 0x55d37d96fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e6199740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d37d8cc970 con 0x55d37d96d590 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e6199740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d37d874d70 con 0x55d37d967a30 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e6199740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d37d8c8920 con 0x55d37d968420 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35d27fc640 1 -- 172.21.15.105:0/4073931430 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f35d80f1930 con 0x7f35dc1007c0 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e1b50640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d96d590 0x55d37d96fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e0b4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d96cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e134f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d967a30 0x55d37d967e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e0b4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d96cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39840/0 (socket says 172.21.15.105:39840) 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e0b4e640 1 -- 172.21.15.105:0/152773412 learned_addr learned my addr 172.21.15.105:0/152773412 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e0b4e640 1 -- 172.21.15.105:0/152773412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d967a30 msgr2=0x55d37d967e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e0b4e640 1 --2- 172.21.15.105:0/152773412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d967a30 0x55d37d967e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e0b4e640 1 -- 172.21.15.105:0/152773412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d96d590 msgr2=0x55d37d96fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.574+0000 7f35e1b50640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35c80a3020 0x7f35c80a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.575+0000 7f35e0b4e640 1 --2- 172.21.15.105:0/152773412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d96d590 0x55d37d96fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.575+0000 7f35e0b4e640 1 -- 172.21.15.105:0/152773412 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d37d5e03f0 con 0x55d37d968420 2024-03-26T17:57:45.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.575+0000 7f35e0b4e640 1 --2- 172.21.15.105:0/152773412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d96cf20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b740 tx=0x7f35cc00bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb51c88f7f6156e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.575+0000 7f35c77fe640 1 -- 172.21.15.105:0/152773412 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc003dd0 con 0x55d37d968420 2024-03-26T17:57:45.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.575+0000 7f35c77fe640 1 -- 172.21.15.105:0/152773412 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35cc015070 con 0x55d37d968420 2024-03-26T17:57:45.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.575+0000 7f35c77fe640 1 -- 172.21.15.105:0/152773412 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc004e40 con 0x55d37d968420 2024-03-26T17:57:45.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 -- 172.21.15.105:0/152773412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 msgr2=0x55d37d96cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 --2- 172.21.15.105:0/152773412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d96cf20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b740 tx=0x7f35cc00bc00 comp rx=0 tx=0).stop 2024-03-26T17:57:45.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e1b50640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35c80a3020 0x7f35c80a54e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005fd0 tx=0x7f35d4009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 -- 172.21.15.105:0/152773412 shutdown_connections 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 --2- 172.21.15.105:0/152773412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d96d590 0x55d37d96fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 --2- 172.21.15.105:0/152773412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d96cf20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 --2- 172.21.15.105:0/152773412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d967a30 0x55d37d967e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 -- 172.21.15.105:0/152773412 >> 172.21.15.105:0/152773412 conn(0x55d37d962fa0 msgr2=0x55d37d9633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 -- 172.21.15.105:0/152773412 shutdown_connections 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 -- 172.21.15.105:0/152773412 wait complete. 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 -- start start 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.576+0000 7f35e6199740 1 Processor -- start 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e6199740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d971f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e6199740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d9724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e6199740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e6199740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d37d9705e0 con 0x55d37d967a30 2024-03-26T17:57:45.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e6199740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d37d888310 con 0x55d37d96d590 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e6199740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d37d970760 con 0x55d37d968420 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e1b50640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e0b4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d9724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e1b50640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52324/0 (socket says 172.21.15.105:52324) 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e1b50640 1 -- 172.21.15.105:0/3572258794 learned_addr learned my addr 172.21.15.105:0/3572258794 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e6199740 10 client.0 _ll_register_callbacks cb 0x55d37d8c3c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e134f640 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d971f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e1b50640 1 -- 172.21.15.105:0/3572258794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 msgr2=0x55d37d9724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e1b50640 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d9724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e1b50640 1 -- 172.21.15.105:0/3572258794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 msgr2=0x55d37d971f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e1b50640 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d971f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.577+0000 7f35e1b50640 1 -- 172.21.15.105:0/3572258794 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d37d6aa7d0 con 0x55d37d96d590 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35e1b50640 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979500 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f35d4010c40 tx=0x7f35d4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6546d5a69c0a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35c4ff9640 1 -- 172.21.15.105:0/3572258794 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d401fd30 con 0x55d37d96d590 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35c4ff9640 1 -- 172.21.15.105:0/3572258794 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f35d4020410 con 0x55d37d96d590 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35e6199740 1 -- 172.21.15.105:0/3572258794 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35cc015070 con 0x55d37d96d590 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35c4ff9640 1 -- 172.21.15.105:0/3572258794 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f35d4003970 con 0x55d37d96d590 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35e6199740 10 client.32186 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35e6199740 1 -- 172.21.15.105:0/3572258794 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d37d97c8c0 con 0x55d37d96d590 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.578+0000 7f35c4ff9640 1 -- 172.21.15.105:0/3572258794 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35cc015070 con 0x55d37d96d590 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35c4ff9640 1 -- 172.21.15.105:0/3572258794 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f35d400f040 con 0x55d37d96d590 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35e6199740 10 client.32186 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35e6199740 10 client.32186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35e6199740 20 client.32186 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35e6199740 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d37d97e010 0x55d37d9804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35e6199740 4 client.32186 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35e6199740 1 -- 172.21.15.105:0/3572258794 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d37d8d9eb0 con 0x55d37d97e010 2024-03-26T17:57:45.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35e134f640 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d37d97e010 0x55d37d9804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.579+0000 7f35c4ff9640 10 client.32186 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:45.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.580+0000 7f35e134f640 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d37d97e010 0x55d37d9804d0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9614835c6128b3ba server_cookie=a6ae66696cd44f29 in_seq=0 out_seq=0 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35c4ff9640 1 -- 172.21.15.105:0/3572258794 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d37d8d9eb0 con 0x55d37d97e010 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35c4ff9640 10 client.32186 handle_command_reply: tid=0 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 client.32186 shutdown 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 -- 172.21.15.105:0/3572258794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 msgr2=0x55d37d979500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979500 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f35d4010c40 tx=0x7f35d4004ab0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 -- 172.21.15.105:0/3572258794 shutdown_connections 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d37d97e010 0x55d37d9804d0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979500 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d9724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 --2- 172.21.15.105:0/3572258794 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d971f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 -- 172.21.15.105:0/3572258794 >> 172.21.15.105:0/3572258794 conn(0x55d37d962fa0 msgr2=0x55d37d9621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35c4ff9640 0 client.32186 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:45.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.582+0000 7f35e6199740 1 -- 172.21.15.105:0/3572258794 shutdown_connections 2024-03-26T17:57:45.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.583+0000 7f35e6199740 1 -- 172.21.15.105:0/3572258794 wait complete. 2024-03-26T17:57:45.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.583+0000 7f35e6199740 20 client.32186 trim_cache size 0 max 16384 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.605+0000 7f35e6199740 1 Processor -- start 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.605+0000 7f35e6199740 1 -- start start 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.605+0000 7f35e6199740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d969880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.605+0000 7f35e6199740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d969dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.605+0000 7f35e6199740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e6199740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000e500 con 0x55d37d967a30 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e6199740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35c0010040 con 0x55d37d96d590 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e6199740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000ebd0 con 0x55d37d968420 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d969880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d969880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55798/0 (socket says 172.21.15.105:55798) 2024-03-26T17:57:45.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 -- 172.21.15.105:0/19754148 learned_addr learned my addr 172.21.15.105:0/19754148 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e1b50640 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e0b4e640 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d969dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 -- 172.21.15.105:0/19754148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 msgr2=0x55d37d969dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d969dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 -- 172.21.15.105:0/19754148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 msgr2=0x55d37d979420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 -- 172.21.15.105:0/19754148 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d37d962b40 con 0x55d37d967a30 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35e134f640 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d969880 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005ab0 tx=0x7f35d80de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d02e935802f8207 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.606+0000 7f35c6ffd640 1 -- 172.21.15.105:0/19754148 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d80d70d0 con 0x55d37d967a30 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35c6ffd640 1 -- 172.21.15.105:0/19754148 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35d80bef20 con 0x55d37d967a30 2024-03-26T17:57:45.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35c6ffd640 1 -- 172.21.15.105:0/19754148 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d80de0b0 con 0x55d37d967a30 2024-03-26T17:57:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35e6199740 1 -- 172.21.15.105:0/19754148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 msgr2=0x55d37d969880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35e6199740 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d969880 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005ab0 tx=0x7f35d80de040 comp rx=0 tx=0).stop 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35e6199740 1 -- 172.21.15.105:0/19754148 shutdown_connections 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35e6199740 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d96d590 0x55d37d979420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35e6199740 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d968420 0x55d37d969dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35e6199740 1 --2- 172.21.15.105:0/19754148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d967a30 0x55d37d969880 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.607+0000 7f35e6199740 1 -- 172.21.15.105:0/19754148 >> 172.21.15.105:0/19754148 conn(0x55d37d962fa0 msgr2=0x55d37d963380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 -- 172.21.15.105:0/19754148 shutdown_connections 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 -- 172.21.15.105:0/19754148 wait complete. 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 -- start start 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 Processor -- start 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d968420 0x55d37d9769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d96d590 0x55d37d976f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d977eb0 0x55d37d978330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000f970 con 0x55d37d977eb0 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000f2a0 con 0x55d37d968420 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.608+0000 7f35e6199740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35c000c4e0 con 0x55d37d96d590 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e6199740 10 client.0 _ll_register_callbacks cb 0x7f35d4012470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d977eb0 0x55d37d978330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e134f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d968420 0x55d37d9769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e0b4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d96d590 0x55d37d976f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d977eb0 0x55d37d978330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55806/0 (socket says 172.21.15.105:55806) 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 -- 172.21.15.105:0/415656079 learned_addr learned my addr 172.21.15.105:0/415656079 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e0b4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d96d590 0x55d37d976f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39868/0 (socket says 172.21.15.105:39868) 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 -- 172.21.15.105:0/415656079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d96d590 msgr2=0x55d37d976f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d96d590 0x55d37d976f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 -- 172.21.15.105:0/415656079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d968420 msgr2=0x55d37d9769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d968420 0x55d37d9769d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 -- 172.21.15.105:0/415656079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d37d978870 con 0x55d37d977eb0 2024-03-26T17:57:45.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e1b50640 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d977eb0 0x55d37d978330 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f35d40107a0 tx=0x7f35d4011160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ab14a38706f8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35c4ff9640 1 -- 172.21.15.105:0/415656079 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4036040 con 0x55d37d977eb0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35c4ff9640 1 -- 172.21.15.105:0/415656079 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f35d4003d10 con 0x55d37d977eb0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.609+0000 7f35e6199740 1 -- 172.21.15.105:0/415656079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d80bef20 con 0x55d37d977eb0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35c4ff9640 1 -- 172.21.15.105:0/415656079 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f35d4010820 con 0x55d37d977eb0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e6199740 10 client.22500 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e6199740 1 -- 172.21.15.105:0/415656079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d37d9e4690 con 0x55d37d977eb0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35c4ff9640 1 -- 172.21.15.105:0/415656079 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35d80bef20 con 0x55d37d977eb0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35c4ff9640 1 -- 172.21.15.105:0/415656079 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f35d4042030 con 0x55d37d977eb0 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e6199740 10 client.22500 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e6199740 10 client.22500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e6199740 20 client.22500 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e6199740 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d37d9e5090 0x55d37d9e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:45.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e6199740 4 client.22500 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:57:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e6199740 1 -- 172.21.15.105:0/415656079 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d37d8d4200 con 0x55d37d9e5090 2024-03-26T17:57:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.610+0000 7f35e134f640 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d37d9e5090 0x55d37d9e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.611+0000 7f35e134f640 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d37d9e5090 0x55d37d9e7480 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40e5d3425dbf709 server_cookie=198b2e3e9ae5f711 in_seq=0 out_seq=0 2024-03-26T17:57:45.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.611+0000 7f35c4ff9640 10 client.22500 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35c4ff9640 1 -- 172.21.15.105:0/415656079 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d37d8d4200 con 0x55d37d9e5090 2024-03-26T17:57:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35c4ff9640 10 client.22500 handle_command_reply: tid=0 2024-03-26T17:57:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 client.22500 shutdown 2024-03-26T17:57:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 -- 172.21.15.105:0/415656079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d977eb0 msgr2=0x55d37d978330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d977eb0 0x55d37d978330 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f35d40107a0 tx=0x7f35d4011160 comp rx=0 tx=0).stop 2024-03-26T17:57:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 -- 172.21.15.105:0/415656079 shutdown_connections 2024-03-26T17:57:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d37d9e5090 0x55d37d9e7480 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d37d977eb0 0x55d37d978330 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d37d96d590 0x55d37d976f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 --2- 172.21.15.105:0/415656079 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d37d968420 0x55d37d9769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35c4ff9640 0 client.22500 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 -- 172.21.15.105:0/415656079 >> 172.21.15.105:0/415656079 conn(0x55d37d962fa0 msgr2=0x55d37d971f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 -- 172.21.15.105:0/415656079 shutdown_connections 2024-03-26T17:57:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 1 -- 172.21.15.105:0/415656079 wait complete. 2024-03-26T17:57:45.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.612+0000 7f35e6199740 20 client.22500 trim_cache size 0 max 16384 2024-03-26T17:57:45.601 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 -- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35c80a3020 msgr2=0x7f35c80a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35c80a3020 0x7f35c80a54e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f35d4005fd0 tx=0x7f35d4009600 comp rx=0 tx=0).stop 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 -- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 msgr2=0x7f35dc104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 0x7f35dc104750 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f35d8000c00 tx=0x7f35d8004270 comp rx=0 tx=0).stop 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 -- 172.21.15.105:0/4073931430 shutdown_connections 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f35c80a3020 0x7f35c80a54e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc102370 0x7f35dc1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35dc1019b0 0x7f35dc102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 --2- 172.21.15.105:0/4073931430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f35dc1007c0 0x7f35dc104750 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:45.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 -- 172.21.15.105:0/4073931430 >> 172.21.15.105:0/4073931430 conn(0x7f35dc0fbd50 msgr2=0x7f35dc068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:45.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 -- 172.21.15.105:0/4073931430 shutdown_connections 2024-03-26T17:57:45.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:45.613+0000 7f35e35b3640 1 -- 172.21.15.105:0/4073931430 wait complete. 2024-03-26T17:57:45.884 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:45 smithi105 ceph-mon[32280]: pgmap v932: 97 pgs: 97 active+clean; 318 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-03-26T17:57:45.907 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:45 smithi135 ceph-mon[35636]: pgmap v932: 97 pgs: 97 active+clean; 318 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-03-26T17:57:46.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:45 smithi144 ceph-mon[36095]: pgmap v932: 97 pgs: 97 active+clean; 318 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-03-26T17:57:46.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 51124 7.09 MB/sec execute 269 sec latency 130.125 ms 2024-03-26T17:57:46.613 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:57:46.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:57:46.613 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:57:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 -- 172.21.15.105:0/3989636133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95741014a0 msgr2=0x7f9574101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 --2- 172.21.15.105:0/3989636133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95741014a0 0x7f9574101920 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f956c009ca0 tx=0x7f956c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 -- 172.21.15.105:0/3989636133 shutdown_connections 2024-03-26T17:57:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 --2- 172.21.15.105:0/3989636133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95741014a0 0x7f9574101920 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 --2- 172.21.15.105:0/3989636133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9574100b00 0x7f9574100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 --2- 172.21.15.105:0/3989636133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95741001a0 0x7f95741005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 -- 172.21.15.105:0/3989636133 >> 172.21.15.105:0/3989636133 conn(0x7f95740fbd60 msgr2=0x7f95740fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 -- 172.21.15.105:0/3989636133 shutdown_connections 2024-03-26T17:57:46.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.778+0000 7f95788b2640 1 -- 172.21.15.105:0/3989636133 wait complete. 2024-03-26T17:57:46.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.779+0000 7f95788b2640 1 Processor -- start 2024-03-26T17:57:46.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.779+0000 7f95788b2640 1 -- start start 2024-03-26T17:57:46.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.779+0000 7f95788b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95741001a0 0x7f9574199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f95788b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9574100b00 0x7f957419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f95788b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95741014a0 0x7f95741a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f95788b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f957410f5f0 con 0x7f95741001a0 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f95788b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f957410f470 con 0x7f95741014a0 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f95788b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f957410f770 con 0x7f9574100b00 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95741001a0 0x7f9574199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95741001a0 0x7f9574199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55828/0 (socket says 172.21.15.105:55828) 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 -- 172.21.15.105:0/965498925 learned_addr learned my addr 172.21.15.105:0/965498925 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f9571d9b640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9574100b00 0x7f957419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f9572d9d640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95741014a0 0x7f95741a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 -- 172.21.15.105:0/965498925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9574100b00 msgr2=0x7f957419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9574100b00 0x7f957419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 -- 172.21.15.105:0/965498925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95741014a0 msgr2=0x7f95741a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95741014a0 0x7f95741a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 -- 172.21.15.105:0/965498925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95741a1900 con 0x7f95741001a0 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f9571d9b640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9574100b00 0x7f957419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f9572d9d640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95741014a0 0x7f95741a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f957259c640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95741001a0 0x7f9574199cc0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f956400e9f0 tx=0x7f956400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f95637fe640 1 -- 172.21.15.105:0/965498925 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956400cd80 con 0x7f95741001a0 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f95637fe640 1 -- 172.21.15.105:0/965498925 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95640054d0 con 0x7f95741001a0 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.780+0000 7f95788b2640 1 -- 172.21.15.105:0/965498925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95741a1bf0 con 0x7f95741001a0 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.781+0000 7f95637fe640 1 -- 172.21.15.105:0/965498925 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564010490 con 0x7f95741001a0 2024-03-26T17:57:46.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.781+0000 7f95788b2640 1 -- 172.21.15.105:0/965498925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95741a20f0 con 0x7f95741001a0 2024-03-26T17:57:46.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.783+0000 7f95637fe640 1 -- 172.21.15.105:0/965498925 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9564005020 con 0x7f95741001a0 2024-03-26T17:57:46.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957b57d740 1 Processor -- start 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957b57d740 1 -- start start 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f95637fe640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f955c0a2fa0 0x7f955c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957b57d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65225b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957b57d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6527680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957b57d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca652a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957b57d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cca6487630 con 0x55cca6522190 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957b57d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cca642fe10 con 0x55cca6527cf0 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957b57d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cca64835e0 con 0x55cca6522b80 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f95637fe640 1 -- 172.21.15.105:0/965498925 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f95640cf7d0 con 0x7f95741001a0 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f9572d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca652a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f9572d9d640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f955c0a2fa0 0x7f955c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65225b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55840/0 (socket says 172.21.15.105:55840) 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f957259c640 1 -- 172.21.15.105:0/2159032392 learned_addr learned my addr 172.21.15.105:0/2159032392 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.784+0000 7f9571d9b640 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6527680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f957259c640 1 -- 172.21.15.105:0/2159032392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 msgr2=0x55cca6527680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f957259c640 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6527680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f957259c640 1 -- 172.21.15.105:0/2159032392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 msgr2=0x55cca652a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f957259c640 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca652a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f957259c640 1 -- 172.21.15.105:0/2159032392 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca6446740 con 0x55cca6522190 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f9571d9b640 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6527680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f957259c640 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65225b0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f95640a4340 tx=0x7f9564005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1bb849d953f28f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f9560ff9640 1 -- 172.21.15.105:0/2159032392 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956409be60 con 0x55cca6522190 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f9560ff9640 1 -- 172.21.15.105:0/2159032392 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956409ffb0 con 0x55cca6522190 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f9572d9d640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f955c0a2fa0 0x7f955c0a5460 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f956c006010 tx=0x7f956c032f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.785+0000 7f9560ff9640 1 -- 172.21.15.105:0/2159032392 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640babd0 con 0x55cca6522190 2024-03-26T17:57:46.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 -- 172.21.15.105:0/2159032392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 msgr2=0x55cca65225b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65225b0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f95640a4340 tx=0x7f9564005310 comp rx=0 tx=0).stop 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 -- 172.21.15.105:0/2159032392 shutdown_connections 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca652a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6527680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 --2- 172.21.15.105:0/2159032392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65225b0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 -- 172.21.15.105:0/2159032392 >> 172.21.15.105:0/2159032392 conn(0x55cca651d700 msgr2=0x55cca651db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 -- 172.21.15.105:0/2159032392 shutdown_connections 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.786+0000 7f957b57d740 1 -- 172.21.15.105:0/2159032392 wait complete. 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 1 -- start start 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 1 Processor -- start 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca652c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca652cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6527cf0 0x55cca6533cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55cca652aca0 con 0x55cca6527cf0 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cca64431a0 con 0x55cca6522190 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cca652ae20 con 0x55cca6522b80 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957b57d740 10 client.0 _ll_register_callbacks cb 0x55cca619ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f9572d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6527cf0 0x55cca6533cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f9571d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca652cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f9571d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca652cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39884/0 (socket says 172.21.15.105:39884) 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f9571d9b640 1 -- 172.21.15.105:0/3978414141 learned_addr learned my addr 172.21.15.105:0/3978414141 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.787+0000 7f957259c640 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca652c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9571d9b640 1 -- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 msgr2=0x55cca652c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9571d9b640 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca652c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9571d9b640 1 -- 172.21.15.105:0/3978414141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6527cf0 msgr2=0x55cca6533cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9571d9b640 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6527cf0 0x55cca6533cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9571d9b640 1 -- 172.21.15.105:0/3978414141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca62637e0 con 0x55cca6522b80 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f957259c640 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca652c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9572d9d640 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6527cf0 0x55cca6533cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9571d9b640 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca652cc70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9568012d00 tx=0x7f956800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9830ced177f364b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9559ffb640 1 -- 172.21.15.105:0/3978414141 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680120d0 con 0x55cca6522b80 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9559ffb640 1 -- 172.21.15.105:0/3978414141 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f9568004020 con 0x55cca6522b80 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f957b57d740 1 -- 172.21.15.105:0/3978414141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f956409ffb0 con 0x55cca6522b80 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f9559ffb640 1 -- 172.21.15.105:0/3978414141 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9568012270 con 0x55cca6522b80 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f957b57d740 10 client.32188 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.788+0000 7f957b57d740 1 -- 172.21.15.105:0/3978414141 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cca62c3f70 con 0x55cca6522b80 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f9559ffb640 1 -- 172.21.15.105:0/3978414141 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f956409ffb0 con 0x55cca6522b80 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f9559ffb640 1 -- 172.21.15.105:0/3978414141 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f956801c470 con 0x55cca6522b80 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f957b57d740 10 client.32188 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f957b57d740 10 client.32188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:46.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f957b57d740 20 client.32188 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f957b57d740 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cca6538620 0x55cca653aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f957b57d740 4 client.32188 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f957b57d740 1 -- 172.21.15.105:0/3978414141 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cca6494b70 con 0x55cca6538620 2024-03-26T17:57:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.789+0000 7f957259c640 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cca6538620 0x55cca653aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.790+0000 7f9559ffb640 10 client.32188 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:46.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.790+0000 7f957259c640 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cca6538620 0x55cca653aa80 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af9c9555e3cb99a server_cookie=7c803d339c6514fc in_seq=0 out_seq=0 2024-03-26T17:57:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.791+0000 7f9559ffb640 1 -- 172.21.15.105:0/3978414141 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cca6494b70 con 0x55cca6538620 2024-03-26T17:57:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.791+0000 7f9559ffb640 10 client.32188 handle_command_reply: tid=0 2024-03-26T17:57:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.791+0000 7f957b57d740 1 client.32188 shutdown 2024-03-26T17:57:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.791+0000 7f957b57d740 1 -- 172.21.15.105:0/3978414141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 msgr2=0x55cca652cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.791+0000 7f957b57d740 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca652cc70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9568012d00 tx=0x7f956800f790 comp rx=0 tx=0).stop 2024-03-26T17:57:46.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.791+0000 7f957b57d740 1 -- 172.21.15.105:0/3978414141 shutdown_connections 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f957b57d740 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cca6538620 0x55cca653aa80 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f9559ffb640 0 client.32188 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f957b57d740 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6527cf0 0x55cca6533cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f957b57d740 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca652cc70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f957b57d740 1 --2- 172.21.15.105:0/3978414141 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca652c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f957b57d740 1 -- 172.21.15.105:0/3978414141 >> 172.21.15.105:0/3978414141 conn(0x55cca651d700 msgr2=0x55cca651c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f957b57d740 1 -- 172.21.15.105:0/3978414141 shutdown_connections 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f957b57d740 1 -- 172.21.15.105:0/3978414141 wait complete. 2024-03-26T17:57:46.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.792+0000 7f957b57d740 20 client.32188 trim_cache size 0 max 16384 2024-03-26T17:57:46.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f957b57d740 1 Processor -- start 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f957b57d740 1 -- start start 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f957b57d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca651c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f957b57d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522b80 0x55cca6533c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f957b57d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6527cf0 0x55cca651c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f957b57d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95741ab080 con 0x55cca6522b80 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f957b57d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95741acbc0 con 0x55cca6522190 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f957b57d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95741ab750 con 0x55cca6527cf0 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f9571d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522b80 0x55cca6533c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f9571d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522b80 0x55cca6533c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55858/0 (socket says 172.21.15.105:55858) 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f9571d9b640 1 -- 172.21.15.105:0/1125115667 learned_addr learned my addr 172.21.15.105:0/1125115667 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.811+0000 7f9572d9d640 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6527cf0 0x55cca651c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f957259c640 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca651c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f9571d9b640 1 -- 172.21.15.105:0/1125115667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6527cf0 msgr2=0x55cca651c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f9571d9b640 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6527cf0 0x55cca651c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f9571d9b640 1 -- 172.21.15.105:0/1125115667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 msgr2=0x55cca651c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f9571d9b640 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca651c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f9571d9b640 1 -- 172.21.15.105:0/1125115667 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca615b170 con 0x55cca6522b80 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f9571d9b640 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522b80 0x55cca6533c70 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f956801ca80 tx=0x7f9568005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc93cc09a425ee1c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f955bfff640 1 -- 172.21.15.105:0/1125115667 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956801cd10 con 0x55cca6522b80 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f955bfff640 1 -- 172.21.15.105:0/1125115667 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9568005580 con 0x55cca6522b80 2024-03-26T17:57:46.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.812+0000 7f955bfff640 1 -- 172.21.15.105:0/1125115667 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568003830 con 0x55cca6522b80 2024-03-26T17:57:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 -- 172.21.15.105:0/1125115667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522b80 msgr2=0x55cca6533c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522b80 0x55cca6533c70 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f956801ca80 tx=0x7f9568005d30 comp rx=0 tx=0).stop 2024-03-26T17:57:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 -- 172.21.15.105:0/1125115667 shutdown_connections 2024-03-26T17:57:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6527cf0 0x55cca651c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522b80 0x55cca6533c70 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 --2- 172.21.15.105:0/1125115667 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6522190 0x55cca651c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 -- 172.21.15.105:0/1125115667 >> 172.21.15.105:0/1125115667 conn(0x55cca651d700 msgr2=0x55cca65240c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 -- 172.21.15.105:0/1125115667 shutdown_connections 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.813+0000 7f957b57d740 1 -- 172.21.15.105:0/1125115667 wait complete. 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 1 -- start start 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 1 Processor -- start 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6531b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca6532b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95741ac4f0 con 0x55cca6522190 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95741abe20 con 0x55cca6527cf0 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95741a9060 con 0x55cca6522b80 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957b57d740 10 client.0 _ll_register_callbacks cb 0x7f9568009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:46.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55860/0 (socket says 172.21.15.105:55860) 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957259c640 1 -- 172.21.15.105:0/3824619983 learned_addr learned my addr 172.21.15.105:0/3824619983 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f9572d9d640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca6532b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f9571d9b640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6531b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957259c640 1 -- 172.21.15.105:0/3824619983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 msgr2=0x55cca6531b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957259c640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6531b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.814+0000 7f957259c640 1 -- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 msgr2=0x55cca6532b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f957259c640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca6532b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f957259c640 1 -- 172.21.15.105:0/3824619983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca6533100 con 0x55cca6522190 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f9571d9b640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6531b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f9572d9d640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca6532b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f957259c640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65315c0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f956409a110 tx=0x7f956409b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bea9228fa9a8f52 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f9559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:46.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f9559ffb640 1 -- 172.21.15.105:0/3824619983 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640cd3c0 con 0x55cca6522190 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f9559ffb640 1 -- 172.21.15.105:0/3824619983 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f95640babd0 con 0x55cca6522190 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f957b57d740 1 -- 172.21.15.105:0/3824619983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9568005580 con 0x55cca6522190 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f9559ffb640 1 -- 172.21.15.105:0/3824619983 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f956409c560 con 0x55cca6522190 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f957b57d740 10 client.22518 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.815+0000 7f957b57d740 1 -- 172.21.15.105:0/3824619983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cca659ec40 con 0x55cca6522190 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f9559ffb640 1 -- 172.21.15.105:0/3824619983 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9568005580 con 0x55cca6522190 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f9559ffb640 1 -- 172.21.15.105:0/3824619983 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f95640b00d0 con 0x55cca6522190 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f957b57d740 10 client.22518 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f957b57d740 10 client.22518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f957b57d740 20 client.22518 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f957b57d740 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cca659f210 0x55cca65a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f957b57d740 4 client.22518 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f957b57d740 1 -- 172.21.15.105:0/3824619983 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cca648eec0 con 0x55cca659f210 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.816+0000 7f9571d9b640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cca659f210 0x55cca65a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.817+0000 7f9559ffb640 10 client.22518 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:46.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.817+0000 7f9571d9b640 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cca659f210 0x55cca65a1600 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4502c8a65fb29ce server_cookie=c1dcd68c8ba34175 in_seq=0 out_seq=0 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.819+0000 7f9559ffb640 1 -- 172.21.15.105:0/3824619983 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cca648eec0 con 0x55cca659f210 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.819+0000 7f9559ffb640 10 client.22518 handle_command_reply: tid=0 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.819+0000 7f957b57d740 1 client.22518 shutdown 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 -- 172.21.15.105:0/3824619983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 msgr2=0x55cca65315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65315c0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f956409a110 tx=0x7f956409b7c0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 -- 172.21.15.105:0/3824619983 shutdown_connections 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55cca659f210 0x55cca65a1600 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cca6527cf0 0x55cca6532b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cca6522b80 0x55cca6531b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 --2- 172.21.15.105:0/3824619983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55cca6522190 0x55cca65315c0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 -- 172.21.15.105:0/3824619983 >> 172.21.15.105:0/3824619983 conn(0x55cca651d700 msgr2=0x55cca6520960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f9559ffb640 0 client.22518 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 -- 172.21.15.105:0/3824619983 shutdown_connections 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 1 -- 172.21.15.105:0/3824619983 wait complete. 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.820+0000 7f957b57d740 20 client.22518 trim_cache size 0 max 16384 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "7b013d01-c157-4314-bb64-7c6a82567b08", 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:57:46.807 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:57:46.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 -- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f955c0a2fa0 msgr2=0x7f955c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f955c0a2fa0 0x7f955c0a5460 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f956c006010 tx=0x7f956c032f60 comp rx=0 tx=0).stop 2024-03-26T17:57:46.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 -- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95741001a0 msgr2=0x7f9574199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95741001a0 0x7f9574199cc0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f956400e9f0 tx=0x7f956400edf0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 -- 172.21.15.105:0/965498925 shutdown_connections 2024-03-26T17:57:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f955c0a2fa0 0x7f955c0a5460 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95741014a0 0x7f95741a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9574100b00 0x7f957419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 --2- 172.21.15.105:0/965498925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95741001a0 0x7f9574199cc0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 -- 172.21.15.105:0/965498925 >> 172.21.15.105:0/965498925 conn(0x7f95740fbd60 msgr2=0x7f95740fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 -- 172.21.15.105:0/965498925 shutdown_connections 2024-03-26T17:57:46.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.821+0000 7f95788b2640 1 -- 172.21.15.105:0/965498925 wait complete. 2024-03-26T17:57:46.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:57:46.819 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:57:46.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 -- 172.21.15.105:0/4231983747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381023b0 msgr2=0x7f2538108c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 --2- 172.21.15.105:0/4231983747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381023b0 0x7f2538108c40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f2530009ca0 tx=0x7f253002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 -- 172.21.15.105:0/4231983747 shutdown_connections 2024-03-26T17:57:46.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 --2- 172.21.15.105:0/4231983747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381023b0 0x7f2538108c40 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 --2- 172.21.15.105:0/4231983747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25381019f0 0x7f2538101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 --2- 172.21.15.105:0/4231983747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2538100ce0 0x7f2538101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 -- 172.21.15.105:0/4231983747 >> 172.21.15.105:0/4231983747 conn(0x7f25380fb4b0 msgr2=0x7f25380fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 -- 172.21.15.105:0/4231983747 shutdown_connections 2024-03-26T17:57:46.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.971+0000 7f253ece8640 1 -- 172.21.15.105:0/4231983747 wait complete. 2024-03-26T17:57:46.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.972+0000 7f253ece8640 1 Processor -- start 2024-03-26T17:57:46.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.972+0000 7f253ece8640 1 -- start start 2024-03-26T17:57:46.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.972+0000 7f253ece8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2538100ce0 0x7f2538197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.972+0000 7f253ece8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381019f0 0x7f2538198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.972+0000 7f253ece8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25381023b0 0x7f253819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.972+0000 7f253ece8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f253810d430 con 0x7f25381019f0 2024-03-26T17:57:46.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.972+0000 7f253ece8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f253810d2b0 con 0x7f25381023b0 2024-03-26T17:57:46.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.972+0000 7f253ece8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f253810d5b0 con 0x7f2538100ce0 2024-03-26T17:57:46.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381019f0 0x7f2538198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f253ca84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2538100ce0 0x7f2538197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381019f0 0x7f2538198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55876/0 (socket says 172.21.15.105:55876) 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 -- 172.21.15.105:0/2930070592 learned_addr learned my addr 172.21.15.105:0/2930070592 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 -- 172.21.15.105:0/2930070592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2538100ce0 msgr2=0x7f2538197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2538100ce0 0x7f2538197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 -- 172.21.15.105:0/2930070592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25381023b0 msgr2=0x7f253819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25381023b0 0x7f253819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 -- 172.21.15.105:0/2930070592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253819f710 con 0x7f25381019f0 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2537fff640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381019f0 0x7f2538198030 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f252400e9f0 tx=0x7f252400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2535ffb640 1 -- 172.21.15.105:0/2930070592 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252400cd80 con 0x7f25381019f0 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2535ffb640 1 -- 172.21.15.105:0/2930070592 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25240108e0 con 0x7f25381019f0 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f2535ffb640 1 -- 172.21.15.105:0/2930070592 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524018920 con 0x7f25381019f0 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.973+0000 7f253ece8640 1 -- 172.21.15.105:0/2930070592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253819fa00 con 0x7f25381019f0 2024-03-26T17:57:46.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.974+0000 7f253ece8640 1 -- 172.21.15.105:0/2930070592 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253819ff00 con 0x7f25381019f0 2024-03-26T17:57:46.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.976+0000 7f2535ffb640 1 -- 172.21.15.105:0/2930070592 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2524010a90 con 0x7f25381019f0 2024-03-26T17:57:46.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f25419ae740 1 Processor -- start 2024-03-26T17:57:46.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f2535ffb640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25280a2fa0 0x7f25280a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f25419ae740 1 -- start start 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f25419ae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6986f0 0x561c1b698b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f25419ae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b69d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f253d285640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25280a2fa0 0x7f25280a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f25419ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f25419ae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c1b5f4980 con 0x561c1b6986f0 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f25419ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561c1b59cbe0 con 0x561c1b69dce0 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.977+0000 7f25419ae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c1b5f0930 con 0x561c1b6990e0 2024-03-26T17:57:46.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.978+0000 7f2535ffb640 1 -- 172.21.15.105:0/2930070592 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f25240cf750 con 0x7f25381019f0 2024-03-26T17:57:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.978+0000 7f253ca84640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6986f0 0x561c1b698b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.978+0000 7f253ca84640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6986f0 0x561c1b698b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55880/0 (socket says 172.21.15.105:55880) 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.978+0000 7f253ca84640 1 -- 172.21.15.105:0/1945537304 learned_addr learned my addr 172.21.15.105:0/1945537304 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.978+0000 7f2537fff640 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b69d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.978+0000 7f253d285640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25280a2fa0 0x7f25280a5460 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2530006010 tx=0x7f25300093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f253d285640 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f253ca84640 1 -- 172.21.15.105:0/1945537304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 msgr2=0x561c1b69d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f253ca84640 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b69d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f253ca84640 1 -- 172.21.15.105:0/1945537304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 msgr2=0x561c1b6a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f253ca84640 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f253ca84640 1 -- 172.21.15.105:0/1945537304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1b3d27d0 con 0x561c1b6986f0 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f253d285640 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:46.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f2537fff640 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b69d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f253ca84640 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6986f0 0x561c1b698b10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f252c009ca0 tx=0x7f252c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9adcbbc1b95a99 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f2522ffd640 1 -- 172.21.15.105:0/1945537304 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c004220 con 0x561c1b6986f0 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f2522ffd640 1 -- 172.21.15.105:0/1945537304 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f252c0043c0 con 0x561c1b6986f0 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.979+0000 7f2522ffd640 1 -- 172.21.15.105:0/1945537304 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c0198b0 con 0x561c1b6986f0 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 -- 172.21.15.105:0/1945537304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6986f0 msgr2=0x561c1b698b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6986f0 0x561c1b698b10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f252c009ca0 tx=0x7f252c011810 comp rx=0 tx=0).stop 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 -- 172.21.15.105:0/1945537304 shutdown_connections 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b69d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 --2- 172.21.15.105:0/1945537304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6986f0 0x561c1b698b10 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 -- 172.21.15.105:0/1945537304 >> 172.21.15.105:0/1945537304 conn(0x561c1b68ade0 msgr2=0x561c1b68b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 -- 172.21.15.105:0/1945537304 shutdown_connections 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.980+0000 7f25419ae740 1 -- 172.21.15.105:0/1945537304 wait complete. 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 1 -- start start 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 1 Processor -- start 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b6919f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b69dce0 0x561c1b691f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b6973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c1b68f0e0 con 0x561c1b69dce0 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561c1b5b0160 con 0x561c1b696f60 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c1b68f260 con 0x561c1b6990e0 2024-03-26T17:57:46.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.981+0000 7f25419ae740 10 client.0 _ll_register_callbacks cb 0x561c1b5ec720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f253ca84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b6919f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f253d285640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b6973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f253d285640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b6973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52418/0 (socket says 172.21.15.105:52418) 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2537fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b69dce0 0x561c1b691f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f253d285640 1 -- 172.21.15.105:0/1927891369 learned_addr learned my addr 172.21.15.105:0/1927891369 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2537fff640 1 -- 172.21.15.105:0/1927891369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 msgr2=0x561c1b6919f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2537fff640 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b6919f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2537fff640 1 -- 172.21.15.105:0/1927891369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 msgr2=0x561c1b6973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2537fff640 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b6973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2537fff640 1 -- 172.21.15.105:0/1927891369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1b3083f0 con 0x561c1b69dce0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f253d285640 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b6973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2537fff640 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b69dce0 0x561c1b691f30 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f25240cd580 tx=0x7f2524010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f4854a80e60016 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2517fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2517fff640 1 -- 172.21.15.105:0/1927891369 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25240b6060 con 0x561c1b69dce0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f2517fff640 1 -- 172.21.15.105:0/1927891369 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f25240badb0 con 0x561c1b69dce0 2024-03-26T17:57:46.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.982+0000 7f25419ae740 1 -- 172.21.15.105:0/1927891369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f252c0043c0 con 0x561c1b69dce0 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f2517fff640 1 -- 172.21.15.105:0/1927891369 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f252409e070 con 0x561c1b69dce0 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f25419ae740 10 client.22530 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f25419ae740 1 -- 172.21.15.105:0/1927891369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c1b2c80f0 con 0x561c1b69dce0 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f2517fff640 1 -- 172.21.15.105:0/1927891369 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f252c0043c0 con 0x561c1b69dce0 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f2517fff640 1 -- 172.21.15.105:0/1927891369 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f2524097b90 con 0x561c1b69dce0 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f25419ae740 10 client.22530 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f25419ae740 10 client.22530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f25419ae740 20 client.22530 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f25419ae740 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c1b6b73a0 0x561c1b6b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f25419ae740 4 client.22530 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f25419ae740 1 -- 172.21.15.105:0/1927891369 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1b601ec0 con 0x561c1b6b73a0 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.983+0000 7f253ca84640 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c1b6b73a0 0x561c1b6b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.984+0000 7f2517fff640 10 client.22530 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:46.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.984+0000 7f253ca84640 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c1b6b73a0 0x561c1b6b97e0 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6a3016774f227b server_cookie=78d9f3c476bfc882 in_seq=0 out_seq=0 2024-03-26T17:57:46.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f2517fff640 1 -- 172.21.15.105:0/1927891369 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c1b601ec0 con 0x561c1b6b73a0 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f2517fff640 10 client.22530 handle_command_reply: tid=0 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 client.22530 shutdown 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 -- 172.21.15.105:0/1927891369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b69dce0 msgr2=0x561c1b691f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b69dce0 0x561c1b691f30 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f25240cd580 tx=0x7f2524010ea0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 -- 172.21.15.105:0/1927891369 shutdown_connections 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c1b6b73a0 0x561c1b6b97e0 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b6973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b69dce0 0x561c1b691f30 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f2517fff640 0 client.22530 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 --2- 172.21.15.105:0/1927891369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b6990e0 0x561c1b6919f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 -- 172.21.15.105:0/1927891369 >> 172.21.15.105:0/1927891369 conn(0x561c1b68ade0 msgr2=0x561c1b68a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:46.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 -- 172.21.15.105:0/1927891369 shutdown_connections 2024-03-26T17:57:46.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 1 -- 172.21.15.105:0/1927891369 wait complete. 2024-03-26T17:57:46.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:46.986+0000 7f25419ae740 20 client.22530 trim_cache size 0 max 16384 2024-03-26T17:57:46.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.013+0000 7f25419ae740 1 Processor -- start 2024-03-26T17:57:46.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.013+0000 7f25419ae740 1 -- start start 2024-03-26T17:57:46.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f25419ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b69afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f25419ae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b69b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f25419ae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b69dce0 0x561c1b690050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:46.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f25419ae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25381a6cb0 con 0x561c1b6990e0 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f25419ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25381a87f0 con 0x561c1b696f60 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f25419ae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25381a7380 con 0x561c1b69dce0 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b69b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b69b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55908/0 (socket says 172.21.15.105:55908) 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 -- 172.21.15.105:0/4216636285 learned_addr learned my addr 172.21.15.105:0/4216636285 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f253ca84640 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b69afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f253d285640 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b69dce0 0x561c1b690050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 -- 172.21.15.105:0/4216636285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b69dce0 msgr2=0x561c1b690050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b69dce0 0x561c1b690050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 -- 172.21.15.105:0/4216636285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 msgr2=0x561c1b69afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b69afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 -- 172.21.15.105:0/4216636285 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1b6866b0 con 0x561c1b6990e0 2024-03-26T17:57:47.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f2537fff640 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b69b510 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f252409bb40 tx=0x7f252409bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8695715760349dbd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.014+0000 7f25227fc640 1 -- 172.21.15.105:0/4216636285 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25240b6070 con 0x561c1b6990e0 2024-03-26T17:57:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.015+0000 7f25227fc640 1 -- 172.21.15.105:0/4216636285 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25240a4850 con 0x561c1b6990e0 2024-03-26T17:57:47.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.015+0000 7f25227fc640 1 -- 172.21.15.105:0/4216636285 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2524018ac0 con 0x561c1b6990e0 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 -- 172.21.15.105:0/4216636285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 msgr2=0x561c1b69b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b69b510 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f252409bb40 tx=0x7f252409bfc0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 -- 172.21.15.105:0/4216636285 shutdown_connections 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b69dce0 0x561c1b690050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b69b510 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 --2- 172.21.15.105:0/4216636285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b696f60 0x561c1b69afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 -- 172.21.15.105:0/4216636285 >> 172.21.15.105:0/4216636285 conn(0x561c1b68ade0 msgr2=0x561c1b6919f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 -- 172.21.15.105:0/4216636285 shutdown_connections 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 -- 172.21.15.105:0/4216636285 wait complete. 2024-03-26T17:57:47.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 -- start start 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.016+0000 7f25419ae740 1 Processor -- start 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f25419ae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b696f60 0x561c1b6c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f25419ae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b6c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f25419ae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f25419ae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25381a8120 con 0x561c1b6990e0 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f25419ae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f25381a7a50 con 0x561c1b69dce0 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f25419ae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25381a4c90 con 0x561c1b696f60 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f25419ae740 10 client.0 _ll_register_callbacks cb 0x7f2524097a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253ca84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b696f60 0x561c1b6c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253d285640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253ca84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b696f60 0x561c1b6c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39964/0 (socket says 172.21.15.105:39964) 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253ca84640 1 -- 172.21.15.105:0/2592198392 learned_addr learned my addr 172.21.15.105:0/2592198392 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:47.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f2537fff640 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b6c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253ca84640 1 -- 172.21.15.105:0/2592198392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 msgr2=0x561c1b6c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253ca84640 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253ca84640 1 -- 172.21.15.105:0/2592198392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 msgr2=0x561c1b6c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253ca84640 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b6c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.017+0000 7f253ca84640 1 -- 172.21.15.105:0/2592198392 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1b6c9700 con 0x561c1b696f60 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f253d285640 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f2537fff640 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b6c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f253ca84640 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b696f60 0x561c1b6c7c60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f252c001eb0 tx=0x7f252c003da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39ad65d8673df88b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f2517fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f2517fff640 1 -- 172.21.15.105:0/2592198392 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c003e10 con 0x561c1b696f60 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f2517fff640 1 -- 172.21.15.105:0/2592198392 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f252c02bc40 con 0x561c1b696f60 2024-03-26T17:57:47.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f25419ae740 1 -- 172.21.15.105:0/2592198392 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25240a4850 con 0x561c1b696f60 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f2517fff640 1 -- 172.21.15.105:0/2592198392 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f252c02b320 con 0x561c1b696f60 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f25419ae740 10 client.32209 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f25419ae740 1 -- 172.21.15.105:0/2592198392 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561c1b6ca9d0 con 0x561c1b696f60 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.018+0000 7f2517fff640 1 -- 172.21.15.105:0/2592198392 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25240a4850 con 0x561c1b696f60 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f2517fff640 1 -- 172.21.15.105:0/2592198392 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f252c0035f0 con 0x561c1b696f60 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f25419ae740 10 client.32209 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f25419ae740 10 client.32209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f25419ae740 20 client.32209 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f25419ae740 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c1b71d420 0x561c1b71f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f25419ae740 4 client.32209 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f25419ae740 1 -- 172.21.15.105:0/2592198392 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c1b5fc210 con 0x561c1b71d420 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f2537fff640 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c1b71d420 0x561c1b71f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f2517fff640 10 client.32209 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:47.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.019+0000 7f2537fff640 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c1b71d420 0x561c1b71f840 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a38981f8a347427 server_cookie=df1b394a6c332460 in_seq=0 out_seq=0 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f2517fff640 1 -- 172.21.15.105:0/2592198392 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x561c1b5fc210 con 0x561c1b71d420 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f2517fff640 10 client.32209 handle_command_reply: tid=0 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 client.32209 shutdown 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 -- 172.21.15.105:0/2592198392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b696f60 msgr2=0x561c1b6c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b696f60 0x561c1b6c7c60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f252c001eb0 tx=0x7f252c003da0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 -- 172.21.15.105:0/2592198392 shutdown_connections 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c1b71d420 0x561c1b71f840 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c1b69dce0 0x561c1b6c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c1b6990e0 0x561c1b6c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 --2- 172.21.15.105:0/2592198392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c1b696f60 0x561c1b6c7c60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f2517fff640 0 client.32209 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 -- 172.21.15.105:0/2592198392 >> 172.21.15.105:0/2592198392 conn(0x561c1b68ade0 msgr2=0x561c1b69afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 -- 172.21.15.105:0/2592198392 shutdown_connections 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 1 -- 172.21.15.105:0/2592198392 wait complete. 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.022+0000 7f25419ae740 20 client.32209 trim_cache size 0 max 16384 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (276 inodes in the stack)", 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stdout: "7b013d01-c157-4314-bb64-7c6a82567b08": { 2024-03-26T17:57:47.010 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:57:47.011 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "7b013d01-c157-4314-bb64-7c6a82567b08", 2024-03-26T17:57:47.011 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:57:47.011 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:57:47.011 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:57:47.011 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 -- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25280a2fa0 msgr2=0x7f25280a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25280a2fa0 0x7f25280a5460 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2530006010 tx=0x7f25300093f0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 -- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381019f0 msgr2=0x7f2538198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381019f0 0x7f2538198030 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f252400e9f0 tx=0x7f252400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 -- 172.21.15.105:0/2930070592 shutdown_connections 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f25280a2fa0 0x7f25280a5460 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f25381023b0 0x7f253819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25381019f0 0x7f2538198030 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 --2- 172.21.15.105:0/2930070592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2538100ce0 0x7f2538197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 -- 172.21.15.105:0/2930070592 >> 172.21.15.105:0/2930070592 conn(0x7f25380fb4b0 msgr2=0x7f2538104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:47.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 -- 172.21.15.105:0/2930070592 shutdown_connections 2024-03-26T17:57:47.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:57:47.023+0000 7f253ece8640 1 -- 172.21.15.105:0/2930070592 wait complete. 2024-03-26T17:57:47.020 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:46 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:57:47.020 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:47.020 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:47.021 INFO:tasks.cephfs.filesystem:scrub status for tag:7b013d01-c157-4314-bb64-7c6a82567b08 - {'path': '/', 'tag': '7b013d01-c157-4314-bb64-7c6a82567b08', 'options': 'recursive,force'} 2024-03-26T17:57:47.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:46 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:57:47.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:47.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:47.158 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:57:46 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:57:46.929+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003d3d [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client5/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} v10285 ap=2 RANDEPHEMERALPIN f(v381 m2024-03-26T17:57:27.331990+0000 24=24+0) n(v34 rc2024-03-26T17:57:27.331990+0000 b5230239 25=24+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@6727} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d471600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003d3d:[<0x10000003bbc/PARADOX v108>,<0x10000003bb2/~dmtmp v50>,<0x10000000007/client5 v2084>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003d3d:[<0x10000003bbc/PARADOX v10285>,<0x10000003bb2/~dmtmp v1590>,<0x10000000007/client5 v9048>,<0x10000000006/clients v2705>,<0x10000000005/tmp v2491>,<0x10000000003/client.0 v2390>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v2326>,<0x10000000001/sv_0 v2336>,<0x10000000000/qa v2258>,<0x1/volumes v2229>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:57:27.294067+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:57:27.331990+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v381 m2024-03-26T17:57:27.331990+0000 24=24+0)","memory_value.rstat":"n(v34 rc2024-03-26T17:57:27.331990+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T17:57:47.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:46 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:57:47.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:47.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:47.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 51207 7.07 MB/sec execute 270 sec latency 223.768 ms 2024-03-26T17:57:48.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:48.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:48.158 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: pgmap v933: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-03-26T17:57:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:57:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.159 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003d3d (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:57:48.160 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:47 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:48.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:48.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: pgmap v933: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-03-26T17:57:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:57:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003d3d (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:57:48.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:47 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:48.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:48.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: pgmap v933: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 3.3 MiB/s wr, 232 op/s 2024-03-26T17:57:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:57:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003d3d (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client5/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-03-26T17:57:48.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:47 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:57:48.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 51336 7.07 MB/sec execute 271 sec latency 129.525 ms 2024-03-26T17:57:49.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 51461 7.07 MB/sec execute 272 sec latency 213.587 ms 2024-03-26T17:57:50.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:49 smithi135 ceph-mon[35636]: pgmap v934: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-03-26T17:57:50.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:49 smithi144 ceph-mon[36095]: pgmap v934: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-03-26T17:57:50.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:49 smithi105 ceph-mon[32280]: pgmap v934: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-03-26T17:57:50.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 51583 7.06 MB/sec execute 273 sec latency 155.159 ms 2024-03-26T17:57:51.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 51714 7.06 MB/sec execute 274 sec latency 127.838 ms 2024-03-26T17:57:52.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:51 smithi135 ceph-mon[35636]: pgmap v935: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.5 MiB/s wr, 279 op/s 2024-03-26T17:57:52.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:51 smithi144 ceph-mon[36095]: pgmap v935: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.5 MiB/s wr, 279 op/s 2024-03-26T17:57:52.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:51 smithi105 ceph-mon[32280]: pgmap v935: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.5 MiB/s wr, 279 op/s 2024-03-26T17:57:52.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 51852 7.07 MB/sec execute 275 sec latency 141.993 ms 2024-03-26T17:57:53.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 52011 7.08 MB/sec execute 276 sec latency 118.441 ms 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.573+0000 7f0915fa9640 1 -- 172.21.15.105:0/2924370054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 msgr2=0x7f0910100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.573+0000 7f0915fa9640 1 --2- 172.21.15.105:0/2924370054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 0x7f0910100ec0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f090400d910 tx=0x7f090402b9a0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.574+0000 7f0915fa9640 1 -- 172.21.15.105:0/2924370054 shutdown_connections 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.574+0000 7f0915fa9640 1 --2- 172.21.15.105:0/2924370054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0910101400 0x7f0910108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.574+0000 7f0915fa9640 1 --2- 172.21.15.105:0/2924370054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 0x7f0910100ec0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.574+0000 7f0915fa9640 1 --2- 172.21.15.105:0/2924370054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0910101df0 0x7f0910102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.574+0000 7f0915fa9640 1 -- 172.21.15.105:0/2924370054 >> 172.21.15.105:0/2924370054 conn(0x7f09100fb510 msgr2=0x7f09100fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.574+0000 7f0915fa9640 1 -- 172.21.15.105:0/2924370054 shutdown_connections 2024-03-26T17:57:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.574+0000 7f0915fa9640 1 -- 172.21.15.105:0/2924370054 wait complete. 2024-03-26T17:57:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.574+0000 7f0915fa9640 1 Processor -- start 2024-03-26T17:57:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f0915fa9640 1 -- start start 2024-03-26T17:57:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f0915fa9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 0x7f0910195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f0915fa9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0910101400 0x7f0910195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 0x7f0910195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f0915fa9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0910101df0 0x7f091019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 0x7f0910195680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55922/0 (socket says 172.21.15.105:55922) 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090f7fe640 1 -- 172.21.15.105:0/722724430 learned_addr learned my addr 172.21.15.105:0/722724430 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f091010b280 con 0x7f0910100a40 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f091010b100 con 0x7f0910101df0 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090effd640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0910101400 0x7f0910195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f091010b400 con 0x7f0910101400 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090ffff640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0910101df0 0x7f091019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090f7fe640 1 -- 172.21.15.105:0/722724430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0910101400 msgr2=0x7f0910195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090f7fe640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0910101400 0x7f0910195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090f7fe640 1 -- 172.21.15.105:0/722724430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0910101df0 msgr2=0x7f091019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090f7fe640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0910101df0 0x7f091019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.575+0000 7f090f7fe640 1 -- 172.21.15.105:0/722724430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091019d2a0 con 0x7f0910100a40 2024-03-26T17:57:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.576+0000 7f090f7fe640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 0x7f0910195680 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f090000e9c0 tx=0x7f090000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.576+0000 7f090cff9640 1 -- 172.21.15.105:0/722724430 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090000cd80 con 0x7f0910100a40 2024-03-26T17:57:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.576+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091019d590 con 0x7f0910100a40 2024-03-26T17:57:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.576+0000 7f090cff9640 1 -- 172.21.15.105:0/722724430 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09000035f0 con 0x7f0910100a40 2024-03-26T17:57:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.576+0000 7f090cff9640 1 -- 172.21.15.105:0/722724430 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900010690 con 0x7f0910100a40 2024-03-26T17:57:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.576+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091019dab0 con 0x7f0910100a40 2024-03-26T17:57:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.579+0000 7f090cff9640 1 -- 172.21.15.105:0/722724430 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f09000037a0 con 0x7f0910100a40 2024-03-26T17:57:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.579+0000 7f0918b8c740 1 Processor -- start 2024-03-26T17:57:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.579+0000 7f0918b8c740 1 -- start start 2024-03-26T17:57:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.579+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac05d0 0x5583fcac09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.579+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac0fc0 0x5583fcac5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.579+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcac85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f0918b8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5583fca250b0 con 0x5583fcac0fc0 2024-03-26T17:57:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f0918b8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583fc9cd960 con 0x5583fcac6130 2024-03-26T17:57:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f0918b8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583fca21060 con 0x5583fcac05d0 2024-03-26T17:57:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090cff9640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08f80a2fc0 0x7f08f80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac05d0 0x5583fcac09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac0fc0 0x5583fcac5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcac85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac0fc0 0x5583fcac5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55936/0 (socket says 172.21.15.105:55936) 2024-03-26T17:57:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 -- 172.21.15.105:0/2353663672 learned_addr learned my addr 172.21.15.105:0/2353663672 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090ffff640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08f80a2fc0 0x7f08f80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090cff9640 1 -- 172.21.15.105:0/722724430 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f09000cf880 con 0x7f0910100a40 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 -- 172.21.15.105:0/2353663672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac05d0 msgr2=0x5583fcac09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 --2- 172.21.15.105:0/2353663672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac05d0 0x5583fcac09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 -- 172.21.15.105:0/2353663672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 msgr2=0x5583fcac85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 --2- 172.21.15.105:0/2353663672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcac85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 -- 172.21.15.105:0/2353663672 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fc9dfc60 con 0x5583fcac0fc0 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090effd640 1 --2- 172.21.15.105:0/2353663672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcac85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.580+0000 7f090f7fe640 1 --2- 172.21.15.105:0/2353663672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac0fc0 0x5583fcac5ac0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f090009bf00 tx=0x7f0900003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea4b1cec29ad185 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.581+0000 7f08f5ffb640 1 -- 172.21.15.105:0/2353663672 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090009bd40 con 0x5583fcac0fc0 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.581+0000 7f090ffff640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08f80a2fc0 0x7f08f80a5480 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f090800d680 tx=0x7f090800dcf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.581+0000 7f08f5ffb640 1 -- 172.21.15.105:0/2353663672 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0900003b30 con 0x5583fcac0fc0 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.581+0000 7f08f5ffb640 1 -- 172.21.15.105:0/2353663672 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09000bc070 con 0x5583fcac0fc0 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 -- 172.21.15.105:0/2353663672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac0fc0 msgr2=0x5583fcac5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2353663672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac0fc0 0x5583fcac5ac0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f090009bf00 tx=0x7f0900003e30 comp rx=0 tx=0).stop 2024-03-26T17:57:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 -- 172.21.15.105:0/2353663672 shutdown_connections 2024-03-26T17:57:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2353663672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcac85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2353663672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac0fc0 0x5583fcac5ac0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2353663672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac05d0 0x5583fcac09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 -- 172.21.15.105:0/2353663672 >> 172.21.15.105:0/2353663672 conn(0x5583fcabbb40 msgr2=0x5583fcabbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 -- 172.21.15.105:0/2353663672 shutdown_connections 2024-03-26T17:57:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 -- 172.21.15.105:0/2353663672 wait complete. 2024-03-26T17:57:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.582+0000 7f0918b8c740 1 -- start start 2024-03-26T17:57:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f0918b8c740 1 Processor -- start 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcaca920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcad1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f0918b8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5583fcac9120 con 0x5583fcac05d0 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f0918b8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583fc9e0db0 con 0x5583fcac0fc0 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f0918b8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583fcac92a0 con 0x5583fcac6130 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55944/0 (socket says 172.21.15.105:55944) 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f090effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcaca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f090f7fe640 1 -- 172.21.15.105:0/4195085774 learned_addr learned my addr 172.21.15.105:0/4195085774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f0918b8c740 10 client.0 _ll_register_callbacks cb 0x5583fc9e5110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f090ffff640 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcad1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f090f7fe640 1 -- 172.21.15.105:0/4195085774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 msgr2=0x5583fcad1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.583+0000 7f090f7fe640 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcad1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f090f7fe640 1 -- 172.21.15.105:0/4195085774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 msgr2=0x5583fcaca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f090f7fe640 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcaca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f090f7fe640 1 -- 172.21.15.105:0/4195085774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fc739210 con 0x5583fcac05d0 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f090effd640 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcaca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f090f7fe640 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca3e0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f09000a0280 tx=0x7f090009bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90847d1912525402 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f08eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f08eeffd640 1 -- 172.21.15.105:0/4195085774 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09000a44f0 con 0x5583fcac05d0 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f0918b8c740 1 -- 172.21.15.105:0/4195085774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583fca325f0 con 0x5583fcac05d0 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f08eeffd640 1 -- 172.21.15.105:0/4195085774 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f09000a4690 con 0x5583fcac05d0 2024-03-26T17:57:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f08eeffd640 1 -- 172.21.15.105:0/4195085774 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f090009b3c0 con 0x5583fcac05d0 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f0918b8c740 10 client.22548 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.584+0000 7f0918b8c740 1 -- 172.21.15.105:0/4195085774 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5583fc805620 con 0x5583fcac05d0 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f08eeffd640 1 -- 172.21.15.105:0/4195085774 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5583fca325f0 con 0x5583fcac05d0 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f08eeffd640 1 -- 172.21.15.105:0/4195085774 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f09000bace0 con 0x5583fcac05d0 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f0918b8c740 10 client.22548 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f0918b8c740 10 client.22548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f0918b8c740 20 client.22548 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f0918b8c740 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583fcad5e10 0x5583fcad8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f0918b8c740 4 client.22548 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f0918b8c740 1 -- 172.21.15.105:0/4195085774 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583fca2c940 con 0x5583fcad5e10 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.585+0000 7f090effd640 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583fcad5e10 0x5583fcad8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.586+0000 7f08eeffd640 10 client.22548 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.586+0000 7f090effd640 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583fcad5e10 0x5583fcad8200 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d93dafa55cd348e server_cookie=d8fe0f8c3b68d32f in_seq=0 out_seq=0 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f08eeffd640 1 -- 172.21.15.105:0/4195085774 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5583fca2c940 con 0x5583fcad5e10 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f08eeffd640 10 client.22548 handle_command_reply: tid=0 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 client.22548 shutdown 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 -- 172.21.15.105:0/4195085774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 msgr2=0x5583fcaca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca3e0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f09000a0280 tx=0x7f090009bf70 comp rx=0 tx=0).stop 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 -- 172.21.15.105:0/4195085774 shutdown_connections 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583fcad5e10 0x5583fcad8200 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcad1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcaca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 --2- 172.21.15.105:0/4195085774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca3e0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 -- 172.21.15.105:0/4195085774 >> 172.21.15.105:0/4195085774 conn(0x5583fcabbb40 msgr2=0x5583fcabadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f08eeffd640 0 client.22548 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 -- 172.21.15.105:0/4195085774 shutdown_connections 2024-03-26T17:57:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.588+0000 7f0918b8c740 1 -- 172.21.15.105:0/4195085774 wait complete. 2024-03-26T17:57:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.589+0000 7f0918b8c740 20 client.22548 trim_cache size 0 max 16384 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.614+0000 7f0918b8c740 1 Processor -- start 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.614+0000 7f0918b8c740 1 -- start start 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.614+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.614+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcad1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.614+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcac1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.614+0000 7f0918b8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a6980 con 0x5583fcac05d0 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.614+0000 7f0918b8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a84c0 con 0x5583fcac0fc0 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.614+0000 7f0918b8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a7050 con 0x5583fcac6130 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcad1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55946/0 (socket says 172.21.15.105:55946) 2024-03-26T17:57:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcad1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52500/0 (socket says 172.21.15.105:52500) 2024-03-26T17:57:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcac1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 -- 172.21.15.105:0/1834440175 learned_addr learned my addr 172.21.15.105:0/1834440175 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 -- 172.21.15.105:0/1834440175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 msgr2=0x5583fcac1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcac1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 -- 172.21.15.105:0/1834440175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 msgr2=0x5583fcaca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 -- 172.21.15.105:0/1834440175 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fcabc590 con 0x5583fcac0fc0 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090ffff640 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcac1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090f7fe640 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.615+0000 7f090effd640 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcad1920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0904005940 tx=0x7f0904000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9eabdc1bcf8a082 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.616+0000 7f08f57fa640 1 -- 172.21.15.105:0/1834440175 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904051070 con 0x5583fcac0fc0 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.616+0000 7f08f57fa640 1 -- 172.21.15.105:0/1834440175 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f090404b030 con 0x5583fcac0fc0 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.616+0000 7f08f57fa640 1 -- 172.21.15.105:0/1834440175 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904030aa0 con 0x5583fcac0fc0 2024-03-26T17:57:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 -- 172.21.15.105:0/1834440175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 msgr2=0x5583fcad1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcad1920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0904005940 tx=0x7f0904000bc0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 -- 172.21.15.105:0/1834440175 shutdown_connections 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac6130 0x5583fcac1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac0fc0 0x5583fcad1920 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 --2- 172.21.15.105:0/1834440175 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcaca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 -- 172.21.15.105:0/1834440175 >> 172.21.15.105:0/1834440175 conn(0x5583fcabbb40 msgr2=0x5583fcac9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 -- 172.21.15.105:0/1834440175 shutdown_connections 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 -- 172.21.15.105:0/1834440175 wait complete. 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.617+0000 7f0918b8c740 1 -- start start 2024-03-26T17:57:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f0918b8c740 1 Processor -- start 2024-03-26T17:57:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcad1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac0fc0 0x5583fcace410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f0918b8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcace9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f0918b8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a7df0 con 0x5583fcac05d0 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f0918b8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a7720 con 0x5583fcac6130 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f0918b8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09101a4960 con 0x5583fcac0fc0 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f0918b8c740 10 client.0 _ll_register_callbacks cb 0x7f090800d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcad1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac0fc0 0x5583fcace410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcace9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcace9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:52510/0 (socket says 172.21.15.105:52510) 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac0fc0 0x5583fcace410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40028/0 (socket says 172.21.15.105:40028) 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcad1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55958/0 (socket says 172.21.15.105:55958) 2024-03-26T17:57:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090ffff640 1 -- 172.21.15.105:0/2883394387 learned_addr learned my addr 172.21.15.105:0/2883394387 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090ffff640 1 -- 172.21.15.105:0/2883394387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac0fc0 msgr2=0x5583fcace410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090ffff640 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac0fc0 0x5583fcace410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090ffff640 1 -- 172.21.15.105:0/2883394387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 msgr2=0x5583fcad1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090ffff640 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcad1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.618+0000 7f090ffff640 1 -- 172.21.15.105:0/2883394387 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fcb39f70 con 0x5583fcac6130 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.619+0000 7f090ffff640 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcace9b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f090801bfc0 tx=0x7f090800d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4bab469084e52b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.619+0000 7f08eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.619+0000 7f08eeffd640 1 -- 172.21.15.105:0/2883394387 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090802d030 con 0x5583fcac6130 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.619+0000 7f08eeffd640 1 -- 172.21.15.105:0/2883394387 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f0908004b20 con 0x5583fcac6130 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.619+0000 7f0918b8c740 1 -- 172.21.15.105:0/2883394387 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f090404b030 con 0x5583fcac6130 2024-03-26T17:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.619+0000 7f08eeffd640 1 -- 172.21.15.105:0/2883394387 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f090801de40 con 0x5583fcac6130 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.619+0000 7f0918b8c740 10 client.32240 fetch_fsmap learned FSMap version 39 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.619+0000 7f0918b8c740 1 -- 172.21.15.105:0/2883394387 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5583fcb43ae0 con 0x5583fcac6130 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f08eeffd640 1 -- 172.21.15.105:0/2883394387 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f090404b030 con 0x5583fcac6130 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f08eeffd640 1 -- 172.21.15.105:0/2883394387 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f0908005020 con 0x5583fcac6130 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f0918b8c740 10 client.32240 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f0918b8c740 10 client.32240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f0918b8c740 20 client.32240 populate_metadata read hostname 'smithi105' 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583fcb44540 0x5583fcb46960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f0918b8c740 4 client.32240 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f0918b8c740 1 -- 172.21.15.105:0/2883394387 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583fc9aa770 con 0x5583fcb44540 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.620+0000 7f090f7fe640 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583fcb44540 0x5583fcb46960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.621+0000 7f090f7fe640 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583fcb44540 0x5583fcb46960 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcde0ee079d0a170 server_cookie=76c9603fe2785497 in_seq=0 out_seq=0 2024-03-26T17:57:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.621+0000 7f08eeffd640 10 client.32240 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f08eeffd640 1 -- 172.21.15.105:0/2883394387 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583fc9aa770 con 0x5583fcb44540 2024-03-26T17:57:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f08eeffd640 10 client.32240 handle_command_reply: tid=0 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f0918b8c740 1 client.32240 shutdown 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f0918b8c740 1 -- 172.21.15.105:0/2883394387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 msgr2=0x5583fcace9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcace9b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f090801bfc0 tx=0x7f090800d910 comp rx=0 tx=0).stop 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f0918b8c740 1 -- 172.21.15.105:0/2883394387 shutdown_connections 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583fcb44540 0x5583fcb46960 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583fcac6130 0x5583fcace9b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.786+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583fcac0fc0 0x5583fcace410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.786+0000 7f0918b8c740 1 --2- 172.21.15.105:0/2883394387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583fcac05d0 0x5583fcad1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.786+0000 7f0918b8c740 1 -- 172.21.15.105:0/2883394387 >> 172.21.15.105:0/2883394387 conn(0x5583fcabbb40 msgr2=0x5583fcac1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.785+0000 7f08eeffd640 0 client.32240 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.786+0000 7f0918b8c740 1 -- 172.21.15.105:0/2883394387 shutdown_connections 2024-03-26T17:57:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.786+0000 7f0918b8c740 1 -- 172.21.15.105:0/2883394387 wait complete. 2024-03-26T17:57:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.786+0000 7f0918b8c740 20 client.32240 trim_cache size 0 max 16384 2024-03-26T17:57:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:57:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:57:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:57:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:57:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08f80a2fc0 msgr2=0x7f08f80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08f80a2fc0 0x7f08f80a5480 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f090800d680 tx=0x7f090800dcf0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 msgr2=0x7f0910195680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:57:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 0x7f0910195680 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f090000e9c0 tx=0x7f090000ee80 comp rx=0 tx=0).stop 2024-03-26T17:57:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 shutdown_connections 2024-03-26T17:57:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08f80a2fc0 0x7f08f80a5480 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0910101df0 0x7f091019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0910101400 0x7f0910195bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 --2- 172.21.15.105:0/722724430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0910100a40 0x7f0910195680 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:57:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 >> 172.21.15.105:0/722724430 conn(0x7f09100fb510 msgr2=0x7f0910104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:57:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 shutdown_connections 2024-03-26T17:57:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:57:53.787+0000 7f0915fa9640 1 -- 172.21.15.105:0/722724430 wait complete. 2024-03-26T17:57:54.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:53 smithi135 ceph-mon[35636]: pgmap v936: 97 pgs: 97 active+clean; 308 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.8 MiB/s wr, 301 op/s 2024-03-26T17:57:54.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:53 smithi144 ceph-mon[36095]: pgmap v936: 97 pgs: 97 active+clean; 308 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.8 MiB/s wr, 301 op/s 2024-03-26T17:57:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:53 smithi105 ceph-mon[32280]: pgmap v936: 97 pgs: 97 active+clean; 308 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.8 MiB/s wr, 301 op/s 2024-03-26T17:57:54.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 52150 7.06 MB/sec execute 277 sec latency 232.904 ms 2024-03-26T17:57:55.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 52282 7.05 MB/sec execute 278 sec latency 152.404 ms 2024-03-26T17:57:56.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:55 smithi135 ceph-mon[35636]: pgmap v937: 97 pgs: 97 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-26T17:57:56.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:55 smithi144 ceph-mon[36095]: pgmap v937: 97 pgs: 97 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-26T17:57:56.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:55 smithi105 ceph-mon[32280]: pgmap v937: 97 pgs: 97 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-03-26T17:57:56.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 52431 7.04 MB/sec execute 279 sec latency 110.772 ms 2024-03-26T17:57:57.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 52584 7.04 MB/sec execute 280 sec latency 117.083 ms 2024-03-26T17:57:58.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:57 smithi135 ceph-mon[35636]: pgmap v938: 97 pgs: 97 active+clean; 305 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 290 op/s 2024-03-26T17:57:58.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:57 smithi144 ceph-mon[36095]: pgmap v938: 97 pgs: 97 active+clean; 305 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 290 op/s 2024-03-26T17:57:58.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:57 smithi105 ceph-mon[32280]: pgmap v938: 97 pgs: 97 active+clean; 305 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 290 op/s 2024-03-26T17:57:58.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 52735 7.04 MB/sec execute 281 sec latency 108.945 ms 2024-03-26T17:57:59.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 52861 7.04 MB/sec execute 282 sec latency 158.176 ms 2024-03-26T17:58:00.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:57:59 smithi135 ceph-mon[35636]: pgmap v939: 97 pgs: 97 active+clean; 305 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-03-26T17:58:00.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:57:59 smithi144 ceph-mon[36095]: pgmap v939: 97 pgs: 97 active+clean; 305 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-03-26T17:58:00.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:57:59 smithi105 ceph-mon[32280]: pgmap v939: 97 pgs: 97 active+clean; 305 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 3.1 MiB/s wr, 318 op/s 2024-03-26T17:58:00.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 52972 7.04 MB/sec execute 283 sec latency 157.107 ms 2024-03-26T17:58:01.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 53091 7.04 MB/sec execute 284 sec latency 112.612 ms 2024-03-26T17:58:02.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:01 smithi135 ceph-mon[35636]: pgmap v940: 97 pgs: 97 active+clean; 307 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 2.4 MiB/s wr, 243 op/s 2024-03-26T17:58:02.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:01 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:58:02.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:01 smithi144 ceph-mon[36095]: pgmap v940: 97 pgs: 97 active+clean; 307 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 2.4 MiB/s wr, 243 op/s 2024-03-26T17:58:02.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:01 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:58:02.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:01 smithi105 ceph-mon[32280]: pgmap v940: 97 pgs: 97 active+clean; 307 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 2.4 MiB/s wr, 243 op/s 2024-03-26T17:58:02.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:01 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:58:02.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 53228 7.04 MB/sec execute 285 sec latency 132.508 ms 2024-03-26T17:58:03.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:02 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:58:03.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:02 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:58:03.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:02 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:58:03.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:02 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:58:03.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:02 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:58:03.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:02 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:58:03.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:02 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:58:03.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:02 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:58:03.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:02 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:58:03.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 53402 7.05 MB/sec execute 286 sec latency 120.766 ms 2024-03-26T17:58:04.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:03 smithi144 ceph-mon[36095]: pgmap v941: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-03-26T17:58:04.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:03 smithi105 ceph-mon[32280]: pgmap v941: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-03-26T17:58:04.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:03 smithi135 ceph-mon[35636]: pgmap v941: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 2.9 MiB/s wr, 260 op/s 2024-03-26T17:58:04.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 53576 7.08 MB/sec execute 287 sec latency 139.534 ms 2024-03-26T17:58:05.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 53752 7.10 MB/sec execute 288 sec latency 143.085 ms 2024-03-26T17:58:06.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:05 smithi144 ceph-mon[36095]: pgmap v942: 97 pgs: 97 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-03-26T17:58:06.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:05 smithi105 ceph-mon[32280]: pgmap v942: 97 pgs: 97 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-03-26T17:58:06.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:05 smithi135 ceph-mon[35636]: pgmap v942: 97 pgs: 97 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 3.4 MiB/s wr, 270 op/s 2024-03-26T17:58:06.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 53905 7.09 MB/sec execute 289 sec latency 121.704 ms 2024-03-26T17:58:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 -- 172.21.15.105:0/3203382587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac1023d0 msgr2=0x7f04ac108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3203382587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac1023d0 0x7f04ac108c60 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009be0 tx=0x7f04a402bef0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 -- 172.21.15.105:0/3203382587 shutdown_connections 2024-03-26T17:58:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3203382587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac1023d0 0x7f04ac108c60 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3203382587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ac101a10 0x7f04ac101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3203382587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04ac100820 0x7f04ac100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 -- 172.21.15.105:0/3203382587 >> 172.21.15.105:0/3203382587 conn(0x7f04ac0fbd90 msgr2=0x7f04ac0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 -- 172.21.15.105:0/3203382587 shutdown_connections 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.903+0000 7f04b2c85640 1 -- 172.21.15.105:0/3203382587 wait complete. 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b2c85640 1 Processor -- start 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b2c85640 1 -- start start 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b2c85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac100820 0x7f04ac1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b0a21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac100820 0x7f04ac1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b0a21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac100820 0x7f04ac1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51298/0 (socket says 172.21.15.105:51298) 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b2c85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ac101a10 0x7f04ac197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b0a21640 1 -- 172.21.15.105:0/3070932165 learned_addr learned my addr 172.21.15.105:0/3070932165 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04ac1023d0 0x7f04ac19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac10d470 con 0x7f04ac100820 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac10d2f0 con 0x7f04ac1023d0 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac10d5f0 con 0x7f04ac101a10 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04abfff640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ac101a10 0x7f04ac197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.904+0000 7f04b1222640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04ac1023d0 0x7f04ac19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04abfff640 1 -- 172.21.15.105:0/3070932165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04ac1023d0 msgr2=0x7f04ac19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04abfff640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04ac1023d0 0x7f04ac19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04abfff640 1 -- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac100820 msgr2=0x7f04ac1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04abfff640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac100820 0x7f04ac1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04abfff640 1 -- 172.21.15.105:0/3070932165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac19f600 con 0x7f04ac101a10 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04b0a21640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac100820 0x7f04ac1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04abfff640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ac101a10 0x7f04ac197f20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f049800e790 tx=0x7f049800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04a9ffb640 1 -- 172.21.15.105:0/3070932165 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049800eea0 con 0x7f04ac101a10 2024-03-26T17:58:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac109680 con 0x7f04ac101a10 2024-03-26T17:58:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04a9ffb640 1 -- 172.21.15.105:0/3070932165 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f049800ccc0 con 0x7f04ac101a10 2024-03-26T17:58:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04a9ffb640 1 -- 172.21.15.105:0/3070932165 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498018710 con 0x7f04ac101a10 2024-03-26T17:58:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.905+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac109c20 con 0x7f04ac101a10 2024-03-26T17:58:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.908+0000 7f04a9ffb640 1 -- 172.21.15.105:0/3070932165 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f04980188b0 con 0x7f04ac101a10 2024-03-26T17:58:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.909+0000 7f04a9ffb640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f049c0a3040 0x7f049c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.909+0000 7f04b0a21640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f049c0a3040 0x7f049c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.909+0000 7f04a9ffb640 1 -- 172.21.15.105:0/3070932165 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f04980d04d0 con 0x7f04ac101a10 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.909+0000 7f04b0a21640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f049c0a3040 0x7f049c0a5500 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f04a00058c0 tx=0x7f04a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.909+0000 7f04b5878740 1 Processor -- start 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b5878740 1 -- start start 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b5878740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b5878740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d92098fc0 0x556d9209dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b5878740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b5878740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556d91ffd0b0 con 0x556d92098fc0 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b5878740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556d91fa5960 con 0x556d920985d0 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b5878740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d91ff9060 con 0x556d9209e130 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d92098fc0 0x556d9209dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d92098fc0 0x556d9209dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51302/0 (socket says 172.21.15.105:51302) 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 -- 172.21.15.105:0/1549004613 learned_addr learned my addr 172.21.15.105:0/1549004613 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b0a21640 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04b1222640 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 -- 172.21.15.105:0/1549004613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 msgr2=0x556d920a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 -- 172.21.15.105:0/1549004613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 msgr2=0x556d920989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 -- 172.21.15.105:0/1549004613 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d91fb7c60 con 0x556d92098fc0 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.910+0000 7f04abfff640 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d92098fc0 0x556d9209dac0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0498099640 tx=0x7f0498010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779dbf714c922e40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.911+0000 7f0496ffd640 1 -- 172.21.15.105:0/1549004613 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980a0d10 con 0x556d92098fc0 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.911+0000 7f0496ffd640 1 -- 172.21.15.105:0/1549004613 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04980a55d0 con 0x556d92098fc0 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.911+0000 7f0496ffd640 1 -- 172.21.15.105:0/1549004613 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498018e00 con 0x556d92098fc0 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 -- 172.21.15.105:0/1549004613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d92098fc0 msgr2=0x556d9209dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d92098fc0 0x556d9209dac0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0498099640 tx=0x7f0498010a00 comp rx=0 tx=0).stop 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 -- 172.21.15.105:0/1549004613 shutdown_connections 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d92098fc0 0x556d9209dac0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 --2- 172.21.15.105:0/1549004613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 -- 172.21.15.105:0/1549004613 >> 172.21.15.105:0/1549004613 conn(0x556d92093b40 msgr2=0x556d92093f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 -- 172.21.15.105:0/1549004613 shutdown_connections 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.912+0000 7f04b5878740 1 -- 172.21.15.105:0/1549004613 wait complete. 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 1 -- start start 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 1 Processor -- start 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920985d0 0x556d920a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d9209e130 0x556d920a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920a7a30 0x556d920a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556d920a1080 con 0x556d9209e130 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556d91fb8db0 con 0x556d920a7a30 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556d920a1200 con 0x556d920985d0 2024-03-26T17:58:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b0a21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920985d0 0x556d920a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04abfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d9209e130 0x556d920a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b0a21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920985d0 0x556d920a24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40874/0 (socket says 172.21.15.105:40874) 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b0a21640 1 -- 172.21.15.105:0/2429672747 learned_addr learned my addr 172.21.15.105:0/2429672747 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b5878740 10 client.0 _ll_register_callbacks cb 0x556d91fbd110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.913+0000 7f04b1222640 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920a7a30 0x556d920a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b0a21640 1 -- 172.21.15.105:0/2429672747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920a7a30 msgr2=0x556d920a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b0a21640 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920a7a30 0x556d920a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b0a21640 1 -- 172.21.15.105:0/2429672747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d9209e130 msgr2=0x556d920a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b0a21640 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d9209e130 0x556d920a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b0a21640 1 -- 172.21.15.105:0/2429672747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d91d11210 con 0x556d920985d0 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04abfff640 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d9209e130 0x556d920a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b0a21640 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920985d0 0x556d920a24c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f04a0011e90 tx=0x7f04a0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fad6de6675f1a46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f048bfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:58:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f048bfff640 1 -- 172.21.15.105:0/2429672747 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a001fbf0 con 0x556d920985d0 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f048bfff640 1 -- 172.21.15.105:0/2429672747 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f04a001fd90 con 0x556d920985d0 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b5878740 1 -- 172.21.15.105:0/2429672747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04980a55d0 con 0x556d920985d0 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f048bfff640 1 -- 172.21.15.105:0/2429672747 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f04a001c430 con 0x556d920985d0 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b5878740 10 client.32239 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.914+0000 7f04b5878740 1 -- 172.21.15.105:0/2429672747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556d91ddd620 con 0x556d920985d0 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f048bfff640 1 -- 172.21.15.105:0/2429672747 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a001ca80 con 0x556d920985d0 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f048bfff640 1 -- 172.21.15.105:0/2429672747 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f04a0033900 con 0x556d920985d0 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f04b5878740 10 client.32239 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f04b5878740 10 client.32239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f04b5878740 20 client.32239 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f04b5878740 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556d920aeb10 0x556d920b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f04b5878740 4 client.32239 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f04b5878740 1 -- 172.21.15.105:0/2429672747 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f04980a55d0 con 0x556d920aeb10 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.915+0000 7f04abfff640 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556d920aeb10 0x556d920b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.916+0000 7f048bfff640 10 client.32239 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.916+0000 7f04abfff640 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556d920aeb10 0x556d920b0fb0 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6370f42f8bd0e2d server_cookie=96aa42dbad704daf in_seq=0 out_seq=0 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f048bfff640 1 -- 172.21.15.105:0/2429672747 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f04980a55d0 con 0x556d920aeb10 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f048bfff640 10 client.32239 handle_command_reply: tid=0 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 client.32239 shutdown 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 -- 172.21.15.105:0/2429672747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920985d0 msgr2=0x556d920a24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920985d0 0x556d920a24c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f04a0011e90 tx=0x7f04a0004730 comp rx=0 tx=0).stop 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 -- 172.21.15.105:0/2429672747 shutdown_connections 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556d920aeb10 0x556d920b0fb0 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920a7a30 0x556d920a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f048bfff640 0 client.32239 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d9209e130 0x556d920a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 --2- 172.21.15.105:0/2429672747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920985d0 0x556d920a24c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 -- 172.21.15.105:0/2429672747 >> 172.21.15.105:0/2429672747 conn(0x556d92093b40 msgr2=0x556d92092df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.917+0000 7f04b5878740 1 -- 172.21.15.105:0/2429672747 shutdown_connections 2024-03-26T17:58:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.918+0000 7f04b5878740 1 -- 172.21.15.105:0/2429672747 wait complete. 2024-03-26T17:58:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.918+0000 7f04b5878740 20 client.32239 trim_cache size 0 max 16384 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.941+0000 7f04b5878740 1 Processor -- start 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b5878740 1 -- start start 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b5878740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b5878740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b5878740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920a7a30 0x556d920a20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b5878740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1af3b0 con 0x556d920a7a30 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b5878740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1b0ef0 con 0x556d920985d0 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b5878740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1afa80 con 0x556d9209e130 2024-03-26T17:58:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b0a21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04abfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b1222640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920a7a30 0x556d920a20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04abfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40880/0 (socket says 172.21.15.105:40880) 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b0a21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43734/0 (socket says 172.21.15.105:43734) 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04abfff640 1 -- 172.21.15.105:0/522521008 learned_addr learned my addr 172.21.15.105:0/522521008 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b1222640 1 -- 172.21.15.105:0/522521008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 msgr2=0x556d920b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b1222640 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b1222640 1 -- 172.21.15.105:0/522521008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 msgr2=0x556d920a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b1222640 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b1222640 1 -- 172.21.15.105:0/522521008 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d920a15a0 con 0x556d920a7a30 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04abfff640 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.942+0000 7f04b0a21640 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.943+0000 7f04b1222640 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920a7a30 0x556d920a20d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f04a402c430 tx=0x7f04a4033450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c800b6602d8b3564 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.943+0000 7f04967fc640 1 -- 172.21.15.105:0/522521008 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4033680 con 0x556d920a7a30 2024-03-26T17:58:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.943+0000 7f04967fc640 1 -- 172.21.15.105:0/522521008 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a4003970 con 0x556d920a7a30 2024-03-26T17:58:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.943+0000 7f04967fc640 1 -- 172.21.15.105:0/522521008 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a402cc90 con 0x556d920a7a30 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 -- 172.21.15.105:0/522521008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920a7a30 msgr2=0x556d920a20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920a7a30 0x556d920a20d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f04a402c430 tx=0x7f04a4033450 comp rx=0 tx=0).stop 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 -- 172.21.15.105:0/522521008 shutdown_connections 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920a7a30 0x556d920a20d0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d9209e130 0x556d920b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 --2- 172.21.15.105:0/522521008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d920985d0 0x556d920a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 -- 172.21.15.105:0/522521008 >> 172.21.15.105:0/522521008 conn(0x556d92093b40 msgr2=0x556d92093f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 -- 172.21.15.105:0/522521008 shutdown_connections 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 -- 172.21.15.105:0/522521008 wait complete. 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 -- start start 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.944+0000 7f04b5878740 1 Processor -- start 2024-03-26T17:58:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b5878740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920985d0 0x556d920bfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b5878740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d9209e130 0x556d920c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b5878740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920a7a30 0x556d920c1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b5878740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1b0820 con 0x556d920985d0 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b5878740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1b0150 con 0x556d9209e130 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b5878740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac1ad390 con 0x556d920a7a30 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b5878740 10 client.0 _ll_register_callbacks cb 0x7f04a0012e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04abfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d9209e130 0x556d920c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b1222640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920a7a30 0x556d920c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b1222640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920a7a30 0x556d920c1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40888/0 (socket says 172.21.15.105:40888) 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b1222640 1 -- 172.21.15.105:0/626753833 learned_addr learned my addr 172.21.15.105:0/626753833 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b0a21640 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920985d0 0x556d920bfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b0a21640 1 -- 172.21.15.105:0/626753833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920a7a30 msgr2=0x556d920c1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b0a21640 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920a7a30 0x556d920c1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b0a21640 1 -- 172.21.15.105:0/626753833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d9209e130 msgr2=0x556d920c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b0a21640 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d9209e130 0x556d920c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04b0a21640 1 -- 172.21.15.105:0/626753833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d920c1740 con 0x556d920985d0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.945+0000 7f04abfff640 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d9209e130 0x556d920c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f04b0a21640 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920985d0 0x556d920bfc00 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f04a0033da0 tx=0x7f04a0009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235ae13c398776e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f048bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f048bfff640 1 -- 172.21.15.105:0/626753833 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0011a40 con 0x556d920985d0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f048bfff640 1 -- 172.21.15.105:0/626753833 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f04a002f030 con 0x556d920985d0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f04b5878740 1 -- 172.21.15.105:0/626753833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a4003970 con 0x556d920985d0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f048bfff640 1 -- 172.21.15.105:0/626753833 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f04a0004020 con 0x556d920985d0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f04b5878740 10 client.22572 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f04b5878740 1 -- 172.21.15.105:0/626753833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556d92114d90 con 0x556d920985d0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.946+0000 7f048bfff640 1 -- 172.21.15.105:0/626753833 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04a4003970 con 0x556d920985d0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f048bfff640 1 -- 172.21.15.105:0/626753833 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f04a003c070 con 0x556d920985d0 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f04b5878740 10 client.22572 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f04b5878740 10 client.22572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f04b5878740 20 client.22572 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f04b5878740 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556d920aeb10 0x556d92117720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f04b5878740 4 client.22572 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:58:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f04b5878740 1 -- 172.21.15.105:0/626753833 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d9200a5f0 con 0x556d920aeb10 2024-03-26T17:58:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f04abfff640 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556d920aeb10 0x556d92117720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f048bfff640 10 client.22572 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:06.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:06.947+0000 7f04abfff640 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556d920aeb10 0x556d92117720 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=470d24c0439c2efa server_cookie=aadd3e769aed6e35 in_seq=0 out_seq=0 2024-03-26T17:58:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f048bfff640 1 -- 172.21.15.105:0/626753833 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d9200a5f0 con 0x556d920aeb10 2024-03-26T17:58:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f048bfff640 10 client.22572 handle_command_reply: tid=0 2024-03-26T17:58:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 client.22572 shutdown 2024-03-26T17:58:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 -- 172.21.15.105:0/626753833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920985d0 msgr2=0x556d920bfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920985d0 0x556d920bfc00 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f04a0033da0 tx=0x7f04a0009fa0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 -- 172.21.15.105:0/626753833 shutdown_connections 2024-03-26T17:58:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556d920aeb10 0x556d92117720 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.086 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556d920a7a30 0x556d920c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d9209e130 0x556d920c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f048bfff640 0 client.22572 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 --2- 172.21.15.105:0/626753833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556d920985d0 0x556d920bfc00 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 -- 172.21.15.105:0/626753833 >> 172.21.15.105:0/626753833 conn(0x556d92093b40 msgr2=0x556d920a2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 -- 172.21.15.105:0/626753833 shutdown_connections 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 1 -- 172.21.15.105:0/626753833 wait complete. 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.101+0000 7f04b5878740 20 client.22572 trim_cache size 0 max 16384 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:58:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:58:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:58:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f049c0a3040 msgr2=0x7f049c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f049c0a3040 0x7f049c0a5500 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f04a00058c0 tx=0x7f04a0005f90 comp rx=0 tx=0).stop 2024-03-26T17:58:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ac101a10 msgr2=0x7f04ac197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ac101a10 0x7f04ac197f20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f049800e790 tx=0x7f049800ec50 comp rx=0 tx=0).stop 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 shutdown_connections 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f049c0a3040 0x7f049c0a5500 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04ac1023d0 0x7f04ac19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ac101a10 0x7f04ac197f20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 --2- 172.21.15.105:0/3070932165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04ac100820 0x7f04ac1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 >> 172.21.15.105:0/3070932165 conn(0x7f04ac0fbd90 msgr2=0x7f04ac104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 shutdown_connections 2024-03-26T17:58:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:07.102+0000 7f04b2c85640 1 -- 172.21.15.105:0/3070932165 wait complete. 2024-03-26T17:58:07.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54004 7.09 MB/sec execute 290 sec latency 193.661 ms 2024-03-26T17:58:08.208 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:07 smithi135 ceph-mon[35636]: pgmap v943: 97 pgs: 97 active+clean; 320 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-03-26T17:58:08.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:07 smithi144 ceph-mon[36095]: pgmap v943: 97 pgs: 97 active+clean; 320 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-03-26T17:58:08.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:07 smithi105 ceph-mon[32280]: pgmap v943: 97 pgs: 97 active+clean; 320 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-03-26T17:58:08.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54100 7.08 MB/sec execute 291 sec latency 122.648 ms 2024-03-26T17:58:09.241 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:08 smithi144 ceph-mon[36095]: pgmap v944: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-03-26T17:58:09.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:08 smithi105 ceph-mon[32280]: pgmap v944: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-03-26T17:58:09.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:08 smithi135 ceph-mon[35636]: pgmap v944: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 3.8 MiB/s wr, 254 op/s 2024-03-26T17:58:09.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54231 7.08 MB/sec execute 292 sec latency 123.415 ms 2024-03-26T17:58:10.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54357 7.09 MB/sec execute 293 sec latency 138.429 ms 2024-03-26T17:58:11.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:10 smithi144 ceph-mon[36095]: pgmap v945: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 3.3 MiB/s wr, 193 op/s 2024-03-26T17:58:11.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:10 smithi105 ceph-mon[32280]: pgmap v945: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 3.3 MiB/s wr, 193 op/s 2024-03-26T17:58:11.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:10 smithi135 ceph-mon[35636]: pgmap v945: 97 pgs: 97 active+clean; 319 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 3.3 MiB/s wr, 193 op/s 2024-03-26T17:58:11.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54482 7.11 MB/sec execute 294 sec latency 136.665 ms 2024-03-26T17:58:12.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54654 7.14 MB/sec execute 295 sec latency 125.700 ms 2024-03-26T17:58:13.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:12 smithi144 ceph-mon[36095]: pgmap v946: 97 pgs: 97 active+clean; 321 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 3.7 MiB/s wr, 230 op/s 2024-03-26T17:58:13.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:12 smithi105 ceph-mon[32280]: pgmap v946: 97 pgs: 97 active+clean; 321 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 3.7 MiB/s wr, 230 op/s 2024-03-26T17:58:13.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:12 smithi135 ceph-mon[35636]: pgmap v946: 97 pgs: 97 active+clean; 321 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 3.7 MiB/s wr, 230 op/s 2024-03-26T17:58:13.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54749 7.13 MB/sec execute 296 sec latency 125.478 ms 2024-03-26T17:58:14.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54873 7.13 MB/sec execute 297 sec latency 156.959 ms 2024-03-26T17:58:15.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:14 smithi144 ceph-mon[36095]: pgmap v947: 97 pgs: 97 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 4.2 MiB/s wr, 226 op/s 2024-03-26T17:58:15.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:14 smithi105 ceph-mon[32280]: pgmap v947: 97 pgs: 97 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 4.2 MiB/s wr, 226 op/s 2024-03-26T17:58:15.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:14 smithi135 ceph-mon[35636]: pgmap v947: 97 pgs: 97 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 4.2 MiB/s wr, 226 op/s 2024-03-26T17:58:15.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 54971 7.12 MB/sec execute 298 sec latency 167.804 ms 2024-03-26T17:58:16.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 55078 7.12 MB/sec execute 299 sec latency 135.816 ms 2024-03-26T17:58:17.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:58:17.022 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:58:17.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.174+0000 7f38f7fff640 1 -- 172.21.15.105:0/3559051818 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f002cbe0 con 0x7f39001019b0 2024-03-26T17:58:17.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 -- 172.21.15.105:0/3559051818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39001019b0 msgr2=0x7f3900101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 --2- 172.21.15.105:0/3559051818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39001019b0 0x7f3900101e30 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009900 tx=0x7f38f002bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:17.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 -- 172.21.15.105:0/3559051818 shutdown_connections 2024-03-26T17:58:17.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 --2- 172.21.15.105:0/3559051818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3900102370 0x7f3900108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 --2- 172.21.15.105:0/3559051818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39001019b0 0x7f3900101e30 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 --2- 172.21.15.105:0/3559051818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39001007c0 0x7f3900100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 -- 172.21.15.105:0/3559051818 >> 172.21.15.105:0/3559051818 conn(0x7f39000fbd50 msgr2=0x7f39000fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 -- 172.21.15.105:0/3559051818 shutdown_connections 2024-03-26T17:58:17.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.175+0000 7f39073a9640 1 -- 172.21.15.105:0/3559051818 wait complete. 2024-03-26T17:58:17.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.176+0000 7f39073a9640 1 Processor -- start 2024-03-26T17:58:17.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.176+0000 7f39073a9640 1 -- start start 2024-03-26T17:58:17.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.176+0000 7f39073a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39001007c0 0x7f390010b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.176+0000 7f39073a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39001019b0 0x7f3900109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.176+0000 7f39073a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3900102370 0x7f3900109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.176+0000 7f39073a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f390010d470 con 0x7f39001019b0 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.176+0000 7f39073a9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f390010d2f0 con 0x7f39001007c0 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.176+0000 7f39073a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f390010d5f0 con 0x7f3900102370 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905946640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3900102370 0x7f3900109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905946640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3900102370 0x7f3900109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54564/0 (socket says 172.21.15.105:54564) 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905946640 1 -- 172.21.15.105:0/2823144125 learned_addr learned my addr 172.21.15.105:0/2823144125 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905145640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39001007c0 0x7f390010b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3904944640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39001019b0 0x7f3900109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905145640 1 -- 172.21.15.105:0/2823144125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3900102370 msgr2=0x7f3900109cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905145640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3900102370 0x7f3900109cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905145640 1 -- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39001019b0 msgr2=0x7f3900109780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905145640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39001019b0 0x7f3900109780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905145640 1 -- 172.21.15.105:0/2823144125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390010a550 con 0x7f39001007c0 2024-03-26T17:58:17.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f3905145640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39001007c0 0x7f390010b140 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00e9c0 tx=0x7f38fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f38f67fc640 1 -- 172.21.15.105:0/2823144125 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00ccc0 con 0x7f39001007c0 2024-03-26T17:58:17.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f39073a9640 1 -- 172.21.15.105:0/2823144125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39001a1c40 con 0x7f39001007c0 2024-03-26T17:58:17.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f38f67fc640 1 -- 172.21.15.105:0/2823144125 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38fc0035f0 con 0x7f39001007c0 2024-03-26T17:58:17.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f38f67fc640 1 -- 172.21.15.105:0/2823144125 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00ccc0 con 0x7f39001007c0 2024-03-26T17:58:17.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.177+0000 7f39073a9640 1 -- 172.21.15.105:0/2823144125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39001a21b0 con 0x7f39001007c0 2024-03-26T17:58:17.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3909f8c740 1 Processor -- start 2024-03-26T17:58:17.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f38f67fc640 1 -- 172.21.15.105:0/2823144125 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f38fc0037a0 con 0x7f39001007c0 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3909f8c740 1 -- start start 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d7450 0x555e1c5d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3909f8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555e1c536980 con 0x555e1c5d7450 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3909f8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555e1c4debe0 con 0x555e1c5d22e0 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3909f8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e1c532930 con 0x555e1c5d18f0 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3904944640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d7450 0x555e1c5d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3905145640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3905946640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3905145640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5d6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40142/0 (socket says 172.21.15.105:40142) 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3904944640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d7450 0x555e1c5d98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35108/0 (socket says 172.21.15.105:35108) 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.181+0000 7f3905145640 1 -- 172.21.15.105:0/1197006171 learned_addr learned my addr 172.21.15.105:0/1197006171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.182+0000 7f3905946640 1 -- 172.21.15.105:0/1197006171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 msgr2=0x555e1c5d6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.182+0000 7f38f67fc640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38ec0a3120 0x7f38ec0a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.182+0000 7f38f67fc640 1 -- 172.21.15.105:0/2823144125 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f38fc0cf4f0 con 0x7f39001007c0 2024-03-26T17:58:17.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.182+0000 7f3905946640 1 --2- 172.21.15.105:0/1197006171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5d6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.182+0000 7f3905946640 1 -- 172.21.15.105:0/1197006171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d7450 msgr2=0x555e1c5d98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.182+0000 7f3905946640 1 --2- 172.21.15.105:0/1197006171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d7450 0x555e1c5d98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.182+0000 7f3905946640 1 -- 172.21.15.105:0/1197006171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1c3147d0 con 0x555e1c5d18f0 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.182+0000 7f3904944640 1 --2- 172.21.15.105:0/1197006171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d7450 0x555e1c5d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.183+0000 7f3905946640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38ec0a3120 0x7f38ec0a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.183+0000 7f3905946640 1 --2- 172.21.15.105:0/1197006171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5d1d10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009f50 tx=0x7f38f8011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6f088a37775f29a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.183+0000 7f38eb7fe640 1 -- 172.21.15.105:0/1197006171 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8012950 con 0x555e1c5d18f0 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.183+0000 7f38eb7fe640 1 -- 172.21.15.105:0/1197006171 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38f801cac0 con 0x555e1c5d18f0 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.183+0000 7f38eb7fe640 1 -- 172.21.15.105:0/1197006171 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f801b870 con 0x555e1c5d18f0 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 -- 172.21.15.105:0/1197006171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 msgr2=0x555e1c5d1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3905946640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38ec0a3120 0x7f38ec0a55e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f38f8020000 tx=0x7f38f8021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 --2- 172.21.15.105:0/1197006171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5d1d10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009f50 tx=0x7f38f8011870 comp rx=0 tx=0).stop 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 -- 172.21.15.105:0/1197006171 shutdown_connections 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 --2- 172.21.15.105:0/1197006171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d7450 0x555e1c5d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 --2- 172.21.15.105:0/1197006171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 --2- 172.21.15.105:0/1197006171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5d1d10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 -- 172.21.15.105:0/1197006171 >> 172.21.15.105:0/1197006171 conn(0x555e1c5cce60 msgr2=0x555e1c5cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 -- 172.21.15.105:0/1197006171 shutdown_connections 2024-03-26T17:58:17.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 -- 172.21.15.105:0/1197006171 wait complete. 2024-03-26T17:58:17.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 -- start start 2024-03-26T17:58:17.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.184+0000 7f3909f8c740 1 Processor -- start 2024-03-26T17:58:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5dbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 0x555e1c5dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d7450 0x555e1c5e3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3909f8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555e1c5da400 con 0x555e1c5d22e0 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3909f8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555e1c4f2160 con 0x555e1c5d7450 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3909f8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555e1c5da580 con 0x555e1c5d18f0 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3904944640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 0x555e1c5dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3909f8c740 10 client.0 _ll_register_callbacks cb 0x555e1c52e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3905946640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d7450 0x555e1c5e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3905145640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5dbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3905946640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d7450 0x555e1c5e3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40150/0 (socket says 172.21.15.105:40150) 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.185+0000 7f3905946640 1 -- 172.21.15.105:0/4143405477 learned_addr learned my addr 172.21.15.105:0/4143405477 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f3905946640 1 -- 172.21.15.105:0/4143405477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 msgr2=0x555e1c5dbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f3905946640 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5dbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f3905946640 1 -- 172.21.15.105:0/4143405477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 msgr2=0x555e1c5dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f3905946640 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 0x555e1c5dc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f3905946640 1 -- 172.21.15.105:0/4143405477 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1c24a3f0 con 0x555e1c5d7450 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f3905946640 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d7450 0x555e1c5e3080 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f38f8011f40 tx=0x7f38f80127d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae44cfb1962e9dc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f38e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f38e8ff9640 1 -- 172.21.15.105:0/4143405477 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8036060 con 0x555e1c5d7450 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f3909f8c740 1 -- 172.21.15.105:0/4143405477 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f801cac0 con 0x555e1c5d7450 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.186+0000 7f38e8ff9640 1 -- 172.21.15.105:0/4143405477 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f38f802e070 con 0x555e1c5d7450 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f38e8ff9640 1 -- 172.21.15.105:0/4143405477 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f38f801b670 con 0x555e1c5d7450 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3909f8c740 10 client.32270 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3909f8c740 1 -- 172.21.15.105:0/4143405477 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555e1c20a0f0 con 0x555e1c5d7450 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f38e8ff9640 1 -- 172.21.15.105:0/4143405477 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38f801cac0 con 0x555e1c5d7450 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f38e8ff9640 1 -- 172.21.15.105:0/4143405477 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f38f800f040 con 0x555e1c5d7450 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3909f8c740 10 client.32270 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3909f8c740 10 client.32270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3909f8c740 20 client.32270 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3909f8c740 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e1c5e7cb0 0x555e1c5ea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3909f8c740 4 client.32270 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:17.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3909f8c740 1 -- 172.21.15.105:0/4143405477 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e1c543ec0 con 0x555e1c5e7cb0 2024-03-26T17:58:17.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.187+0000 7f3905145640 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e1c5e7cb0 0x555e1c5ea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.188+0000 7f38e8ff9640 10 client.32270 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:17.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.188+0000 7f3905145640 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e1c5e7cb0 0x555e1c5ea0f0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd12c755e3a6d75 server_cookie=867db28a32b48f63 in_seq=0 out_seq=0 2024-03-26T17:58:17.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f38e8ff9640 1 -- 172.21.15.105:0/4143405477 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555e1c543ec0 con 0x555e1c5e7cb0 2024-03-26T17:58:17.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f38e8ff9640 10 client.32270 handle_command_reply: tid=0 2024-03-26T17:58:17.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 client.32270 shutdown 2024-03-26T17:58:17.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 -- 172.21.15.105:0/4143405477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d7450 msgr2=0x555e1c5e3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d7450 0x555e1c5e3080 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f38f8011f40 tx=0x7f38f80127d0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 -- 172.21.15.105:0/4143405477 shutdown_connections 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e1c5e7cb0 0x555e1c5ea0f0 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d7450 0x555e1c5e3080 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 0x555e1c5dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 --2- 172.21.15.105:0/4143405477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d18f0 0x555e1c5dbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f3909f8c740 1 -- 172.21.15.105:0/4143405477 >> 172.21.15.105:0/4143405477 conn(0x555e1c5cce60 msgr2=0x555e1c5cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.190+0000 7f38e8ff9640 0 client.32270 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.191+0000 7f3909f8c740 1 -- 172.21.15.105:0/4143405477 shutdown_connections 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.191+0000 7f3909f8c740 1 -- 172.21.15.105:0/4143405477 wait complete. 2024-03-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.191+0000 7f3909f8c740 20 client.32270 trim_cache size 0 max 16384 2024-03-26T17:58:17.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3909f8c740 1 Processor -- start 2024-03-26T17:58:17.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3909f8c740 1 -- start start 2024-03-26T17:58:17.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d18f0 0x555e1c5d9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5e3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3905145640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d18f0 0x555e1c5d9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 0x555e1c5dbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3905145640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d18f0 0x555e1c5d9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35128/0 (socket says 172.21.15.105:35128) 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3905145640 1 -- 172.21.15.105:0/2504898740 learned_addr learned my addr 172.21.15.105:0/2504898740 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3909f8c740 1 -- 172.21.15.105:0/2504898740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39001ab070 con 0x555e1c5d18f0 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3909f8c740 1 -- 172.21.15.105:0/2504898740 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f39001acbb0 con 0x555e1c5d22e0 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.215+0000 7f3909f8c740 1 -- 172.21.15.105:0/2504898740 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39001ab740 con 0x555e1c5d7450 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f3904944640 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f3905946640 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 0x555e1c5dbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f3905145640 1 -- 172.21.15.105:0/2504898740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 msgr2=0x555e1c5dbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f3905145640 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 0x555e1c5dbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f3905145640 1 -- 172.21.15.105:0/2504898740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 msgr2=0x555e1c5e3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f3905145640 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5e3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f3905145640 1 -- 172.21.15.105:0/2504898740 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1c5d26c0 con 0x555e1c5d18f0 2024-03-26T17:58:17.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f3905145640 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d18f0 0x555e1c5d9840 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f38fc097fe0 tx=0x7f38fc004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b896995f6c06fea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f38eaffd640 1 -- 172.21.15.105:0/2504898740 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc09c280 con 0x555e1c5d18f0 2024-03-26T17:58:17.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f38eaffd640 1 -- 172.21.15.105:0/2504898740 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38fc09bb80 con 0x555e1c5d18f0 2024-03-26T17:58:17.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.216+0000 7f38eaffd640 1 -- 172.21.15.105:0/2504898740 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0981f0 con 0x555e1c5d18f0 2024-03-26T17:58:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.217+0000 7f3909f8c740 1 -- 172.21.15.105:0/2504898740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d18f0 msgr2=0x555e1c5d9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.217+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d18f0 0x555e1c5d9840 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f38fc097fe0 tx=0x7f38fc004570 comp rx=0 tx=0).stop 2024-03-26T17:58:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.217+0000 7f3909f8c740 1 -- 172.21.15.105:0/2504898740 shutdown_connections 2024-03-26T17:58:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.217+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 0x555e1c5dbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.217+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d22e0 0x555e1c5e3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.217+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2504898740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d18f0 0x555e1c5d9840 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.217+0000 7f3909f8c740 1 -- 172.21.15.105:0/2504898740 >> 172.21.15.105:0/2504898740 conn(0x555e1c5cce60 msgr2=0x555e1c5d6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.218+0000 7f3909f8c740 1 -- 172.21.15.105:0/2504898740 shutdown_connections 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.218+0000 7f3909f8c740 1 -- 172.21.15.105:0/2504898740 wait complete. 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.218+0000 7f3909f8c740 1 -- start start 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.218+0000 7f3909f8c740 1 Processor -- start 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.218+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d18f0 0x555e1c5e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.218+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 0x555e1c5df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.218+0000 7f3909f8c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 0x555e1c5dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3909f8c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39001ac4e0 con 0x555e1c5d22e0 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3909f8c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f39001abe10 con 0x555e1c5d18f0 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3909f8c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39001a9050 con 0x555e1c5d7450 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3904944640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 0x555e1c5df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d18f0 0x555e1c5e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3909f8c740 10 client.0 _ll_register_callbacks cb 0x7f38f801f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d18f0 0x555e1c5e2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40162/0 (socket says 172.21.15.105:40162) 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 -- 172.21.15.105:0/2607894500 learned_addr learned my addr 172.21.15.105:0/2607894500 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905946640 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 0x555e1c5dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 -- 172.21.15.105:0/2607894500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 msgr2=0x555e1c5dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 0x555e1c5dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 -- 172.21.15.105:0/2607894500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 msgr2=0x555e1c5df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 0x555e1c5df7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 -- 172.21.15.105:0/2607894500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1c5e3290 con 0x555e1c5d18f0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3905145640 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d18f0 0x555e1c5e2c60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c6a0 tx=0x7f38fc00c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570664d76d74cca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f38e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f38e8ff9640 1 -- 172.21.15.105:0/2607894500 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc09c280 con 0x555e1c5d18f0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f38e8ff9640 1 -- 172.21.15.105:0/2607894500 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f38fc09afd0 con 0x555e1c5d18f0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.219+0000 7f3909f8c740 1 -- 172.21.15.105:0/2607894500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38fc09bb80 con 0x555e1c5d18f0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f38e8ff9640 1 -- 172.21.15.105:0/2607894500 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f38fc081050 con 0x555e1c5d18f0 2024-03-26T17:58:17.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3909f8c740 10 client.32276 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3909f8c740 1 -- 172.21.15.105:0/2607894500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555e1c64cf10 con 0x555e1c5d18f0 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f38e8ff9640 1 -- 172.21.15.105:0/2607894500 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38fc09bb80 con 0x555e1c5d18f0 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f38e8ff9640 1 -- 172.21.15.105:0/2607894500 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f38fc09ab60 con 0x555e1c5d18f0 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3909f8c740 10 client.32276 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3909f8c740 10 client.32276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3909f8c740 20 client.32276 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e1c64d810 0x555e1c64fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3909f8c740 4 client.32276 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3909f8c740 1 -- 172.21.15.105:0/2607894500 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e1c53e210 con 0x555e1c64d810 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.220+0000 7f3904944640 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e1c64d810 0x555e1c64fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.221+0000 7f3904944640 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e1c64d810 0x555e1c64fc30 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcec0b6d66172521 server_cookie=d492f2e4a4c7c93 in_seq=0 out_seq=0 2024-03-26T17:58:17.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.221+0000 7f38e8ff9640 10 client.32276 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:17.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.223+0000 7f38e8ff9640 1 -- 172.21.15.105:0/2607894500 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e1c53e210 con 0x555e1c64d810 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.223+0000 7f38e8ff9640 10 client.32276 handle_command_reply: tid=0 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.223+0000 7f3909f8c740 1 client.32276 shutdown 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 -- 172.21.15.105:0/2607894500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d18f0 msgr2=0x555e1c5e2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d18f0 0x555e1c5e2c60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c6a0 tx=0x7f38fc00c840 comp rx=0 tx=0).stop 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 -- 172.21.15.105:0/2607894500 shutdown_connections 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555e1c64d810 0x555e1c64fc30 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f38e8ff9640 0 client.32276 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555e1c5d7450 0x555e1c5dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555e1c5d22e0 0x555e1c5df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 --2- 172.21.15.105:0/2607894500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555e1c5d18f0 0x555e1c5e2c60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 -- 172.21.15.105:0/2607894500 >> 172.21.15.105:0/2607894500 conn(0x555e1c5cce60 msgr2=0x555e1c5dbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 -- 172.21.15.105:0/2607894500 shutdown_connections 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 1 -- 172.21.15.105:0/2607894500 wait complete. 2024-03-26T17:58:17.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.224+0000 7f3909f8c740 20 client.32276 trim_cache size 0 max 16384 2024-03-26T17:58:17.213 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:58:17.213 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:58:17.213 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:58:17.213 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:58:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.225+0000 7f39073a9640 1 -- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38ec0a3120 msgr2=0x7f38ec0a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.225+0000 7f39073a9640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38ec0a3120 0x7f38ec0a55e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f38f8020000 tx=0x7f38f8021000 comp rx=0 tx=0).stop 2024-03-26T17:58:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.225+0000 7f39073a9640 1 -- 172.21.15.105:0/2823144125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39001007c0 msgr2=0x7f390010b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.225+0000 7f39073a9640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39001007c0 0x7f390010b140 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00e9c0 tx=0x7f38fc00ee80 comp rx=0 tx=0).stop 2024-03-26T17:58:17.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.225+0000 7f39073a9640 1 -- 172.21.15.105:0/2823144125 shutdown_connections 2024-03-26T17:58:17.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.225+0000 7f39073a9640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38ec0a3120 0x7f38ec0a55e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.225+0000 7f39073a9640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3900102370 0x7f3900109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.226+0000 7f39073a9640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39001019b0 0x7f3900109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.226+0000 7f39073a9640 1 --2- 172.21.15.105:0/2823144125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39001007c0 0x7f390010b140 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.226+0000 7f39073a9640 1 -- 172.21.15.105:0/2823144125 >> 172.21.15.105:0/2823144125 conn(0x7f39000fbd50 msgr2=0x7f3900104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.226+0000 7f39073a9640 1 -- 172.21.15.105:0/2823144125 shutdown_connections 2024-03-26T17:58:17.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.226+0000 7f39073a9640 1 -- 172.21.15.105:0/2823144125 wait complete. 2024-03-26T17:58:17.223 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:16 smithi105 ceph-mon[32280]: pgmap v948: 97 pgs: 97 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 3.6 MiB/s wr, 193 op/s 2024-03-26T17:58:17.225 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:58:17.225 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:58:17.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:16 smithi144 ceph-mon[36095]: pgmap v948: 97 pgs: 97 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 3.6 MiB/s wr, 193 op/s 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 -- 172.21.15.105:0/1261107592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06601007e0 msgr2=0x7f0660100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 --2- 172.21.15.105:0/1261107592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06601007e0 0x7f0660100c00 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f0650009900 tx=0x7f065002bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 -- 172.21.15.105:0/1261107592 shutdown_connections 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 --2- 172.21.15.105:0/1261107592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0660102390 0x7f0660108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 --2- 172.21.15.105:0/1261107592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06601019d0 0x7f0660101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 --2- 172.21.15.105:0/1261107592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06601007e0 0x7f0660100c00 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 -- 172.21.15.105:0/1261107592 >> 172.21.15.105:0/1261107592 conn(0x7f06600fbd70 msgr2=0x7f06600fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 -- 172.21.15.105:0/1261107592 shutdown_connections 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.377+0000 7f066603d640 1 -- 172.21.15.105:0/1261107592 wait complete. 2024-03-26T17:58:17.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.378+0000 7f066603d640 1 Processor -- start 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.378+0000 7f066603d640 1 -- start start 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f066603d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06601007e0 0x7f066006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f066603d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06601019d0 0x7f066019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f066603d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660102390 0x7f066019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f066603d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f066010c300 con 0x7f06601019d0 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f066603d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f066010c180 con 0x7f06601007e0 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f066603d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f066010c480 con 0x7f0660102390 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06601007e0 0x7f066006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06601007e0 0x7f066006aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40178/0 (socket says 172.21.15.105:40178) 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 -- 172.21.15.105:0/3479261076 learned_addr learned my addr 172.21.15.105:0/3479261076 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065ffff640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660102390 0x7f066019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065effd640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06601019d0 0x7f066019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 -- 172.21.15.105:0/3479261076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660102390 msgr2=0x7f066019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660102390 0x7f066019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 -- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06601019d0 msgr2=0x7f066019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06601019d0 0x7f066019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 -- 172.21.15.105:0/3479261076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066019f7d0 con 0x7f06601007e0 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.379+0000 7f065f7fe640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06601007e0 0x7f066006aa00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0650000c00 tx=0x7f0650005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.380+0000 7f065cff9640 1 -- 172.21.15.105:0/3479261076 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650004350 con 0x7f06601007e0 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.380+0000 7f065cff9640 1 -- 172.21.15.105:0/3479261076 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06500052b0 con 0x7f06601007e0 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.380+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0660196670 con 0x7f06601007e0 2024-03-26T17:58:17.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.380+0000 7f065cff9640 1 -- 172.21.15.105:0/3479261076 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065003edb0 con 0x7f06601007e0 2024-03-26T17:58:17.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.380+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0660196c10 con 0x7f06601007e0 2024-03-26T17:58:17.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.382+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0660107c20 con 0x7f06601007e0 2024-03-26T17:58:17.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.382+0000 7f065cff9640 1 -- 172.21.15.105:0/3479261076 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0650005460 con 0x7f06601007e0 2024-03-26T17:58:17.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.383+0000 7f065cff9640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06480a3020 0x7f06480a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.383+0000 7f065cff9640 1 -- 172.21.15.105:0/3479261076 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f06500f5180 con 0x7f06601007e0 2024-03-26T17:58:17.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.383+0000 7f065effd640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06480a3020 0x7f06480a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.384+0000 7f065effd640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06480a3020 0x7f06480a54e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f066006a770 tx=0x7f0654005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.392+0000 7f065cff9640 1 -- 172.21.15.105:0/3479261076 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f06500c1f10 con 0x7f06601007e0 2024-03-26T17:58:17.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:16 smithi135 ceph-mon[35636]: pgmap v948: 97 pgs: 97 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 3.6 MiB/s wr, 193 op/s 2024-03-26T17:58:17.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 55166 7.10 MB/sec execute 300 sec latency 156.382 ms 2024-03-26T17:58:17.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.718+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06601980b0 con 0x7f06601007e0 2024-03-26T17:58:17.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.719+0000 7f065cff9640 1 -- 172.21.15.105:0/3479261076 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11941 (secure 0 0 0) 0x7f06500c1810 con 0x7f06601007e0 2024-03-26T17:58:17.704 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:58:17.704 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:56:47.920346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_31852":{"gid":31852,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/2697191136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":2697191136},{"type":"v1","addr":"172.21.15.135:6835","nonce":2697191136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T17:58:17.705 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 39 2024-03-26T17:58:17.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06480a3020 msgr2=0x7f06480a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06480a3020 0x7f06480a54e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f066006a770 tx=0x7f0654005f90 comp rx=0 tx=0).stop 2024-03-26T17:58:17.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06601007e0 msgr2=0x7f066006aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06601007e0 0x7f066006aa00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0650000c00 tx=0x7f0650005dc0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 shutdown_connections 2024-03-26T17:58:17.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f06480a3020 0x7f06480a54e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660102390 0x7f066019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06601019d0 0x7f066019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 --2- 172.21.15.105:0/3479261076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f06601007e0 0x7f066006aa00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 >> 172.21.15.105:0/3479261076 conn(0x7f06600fbd70 msgr2=0x7f0660104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 shutdown_connections 2024-03-26T17:58:17.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.722+0000 7f066603d640 1 -- 172.21.15.105:0/3479261076 wait complete. 2024-03-26T17:58:17.722 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:58:17.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 -- 172.21.15.105:0/3673504763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3960102390 msgr2=0x7f3960069c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 --2- 172.21.15.105:0/3673504763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3960102390 0x7f3960069c40 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f3958009be0 tx=0x7f395802bef0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 -- 172.21.15.105:0/3673504763 shutdown_connections 2024-03-26T17:58:17.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 --2- 172.21.15.105:0/3673504763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3960102390 0x7f3960069c40 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 --2- 172.21.15.105:0/3673504763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39601019d0 0x7f3960101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 --2- 172.21.15.105:0/3673504763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39601007e0 0x7f3960100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 -- 172.21.15.105:0/3673504763 >> 172.21.15.105:0/3673504763 conn(0x7f39600fbd70 msgr2=0x7f39600fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 -- 172.21.15.105:0/3673504763 shutdown_connections 2024-03-26T17:58:17.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.890+0000 7f3964ef8640 1 -- 172.21.15.105:0/3673504763 wait complete. 2024-03-26T17:58:17.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3964ef8640 1 Processor -- start 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3964ef8640 1 -- start start 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3964ef8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39601007e0 0x7f3960199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3964ef8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39601019d0 0x7f396019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3964ef8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3960102390 0x7f39601a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3964ef8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f396010f680 con 0x7f39601019d0 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3964ef8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f396010f500 con 0x7f39601007e0 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3964ef8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f396010f800 con 0x7f3960102390 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3957fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39601019d0 0x7f396019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f395e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39601007e0 0x7f3960199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3957fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39601019d0 0x7f396019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35180/0 (socket says 172.21.15.105:35180) 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f3957fff640 1 -- 172.21.15.105:0/1019163021 learned_addr learned my addr 172.21.15.105:0/1019163021 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f395ed9d640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3960102390 0x7f39601a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.891+0000 7f395e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39601007e0 0x7f3960199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40192/0 (socket says 172.21.15.105:40192) 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f395ed9d640 1 -- 172.21.15.105:0/1019163021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39601007e0 msgr2=0x7f3960199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f395ed9d640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39601007e0 0x7f3960199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f395ed9d640 1 -- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39601019d0 msgr2=0x7f396019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f395ed9d640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39601019d0 0x7f396019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f395ed9d640 1 -- 172.21.15.105:0/1019163021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39601a1900 con 0x7f3960102390 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f395e59c640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39601007e0 0x7f3960199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f3957fff640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39601019d0 0x7f396019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f395ed9d640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3960102390 0x7f39601a1260 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3958036040 tx=0x7f3958033c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f39577fe640 1 -- 172.21.15.105:0/1019163021 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958043020 con 0x7f3960102390 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f39577fe640 1 -- 172.21.15.105:0/1019163021 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3958003e20 con 0x7f3960102390 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f3964ef8640 1 -- 172.21.15.105:0/1019163021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39601a1b90 con 0x7f3960102390 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f39577fe640 1 -- 172.21.15.105:0/1019163021 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958034430 con 0x7f3960102390 2024-03-26T17:58:17.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.892+0000 7f3964ef8640 1 -- 172.21.15.105:0/1019163021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601a20b0 con 0x7f3960102390 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.895+0000 7f39577fe640 1 -- 172.21.15.105:0/1019163021 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3958046020 con 0x7f3960102390 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.895+0000 7f3967bc0740 1 Processor -- start 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3967bc0740 1 -- start start 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29656e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f39577fe640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39400a2f20 0x7f39400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae2965bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae2965ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3967bc0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ae295bb970 con 0x55ae29657420 2024-03-26T17:58:17.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3967bc0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ae29563d70 con 0x55ae2965c590 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3967bc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ae295b7920 con 0x55ae29656a30 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f395e59c640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39400a2f20 0x7f39400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3957fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae2965bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f39577fe640 1 -- 172.21.15.105:0/1019163021 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f39580f1bc0 con 0x7f3960102390 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3957fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29656e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3957fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29656e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54634/0 (socket says 172.21.15.105:54634) 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f3957fff640 1 -- 172.21.15.105:0/1456091795 learned_addr learned my addr 172.21.15.105:0/1456091795 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.896+0000 7f395ed9d640 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae2965ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3957fff640 1 -- 172.21.15.105:0/1456091795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 msgr2=0x55ae2965ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3957fff640 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae2965ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3957fff640 1 -- 172.21.15.105:0/1456091795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 msgr2=0x55ae2965bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3957fff640 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae2965bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3957fff640 1 -- 172.21.15.105:0/1456091795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae292cf3f0 con 0x55ae29656a30 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f395ed9d640 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae2965ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:17.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f395e59c640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39400a2f20 0x7f39400a53e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f396019b1c0 tx=0x7f3948009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3957fff640 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29656e50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3950013cf0 tx=0x7f3950004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19af36329561de18 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3954ff9640 1 -- 172.21.15.105:0/1456091795 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950013110 con 0x55ae29656a30 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3954ff9640 1 -- 172.21.15.105:0/1456091795 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39500132b0 con 0x55ae29656a30 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.897+0000 7f3954ff9640 1 -- 172.21.15.105:0/1456091795 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950004a90 con 0x55ae29656a30 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 -- 172.21.15.105:0/1456091795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 msgr2=0x55ae29656e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29656e50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3950013cf0 tx=0x7f3950004730 comp rx=0 tx=0).stop 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 -- 172.21.15.105:0/1456091795 shutdown_connections 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae2965ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae2965bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 --2- 172.21.15.105:0/1456091795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29656e50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 -- 172.21.15.105:0/1456091795 >> 172.21.15.105:0/1456091795 conn(0x55ae29651fa0 msgr2=0x55ae296523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 -- 172.21.15.105:0/1456091795 shutdown_connections 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 -- 172.21.15.105:0/1456091795 wait complete. 2024-03-26T17:58:17.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.898+0000 7f3967bc0740 1 -- start start 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3967bc0740 1 Processor -- start 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29660fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae29657420 0x55ae296614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae2965c590 0x55ae29668530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3967bc0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ae2965f540 con 0x55ae2965c590 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3967bc0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ae29577310 con 0x55ae29657420 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3967bc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ae2965f6c0 con 0x55ae29656a30 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3967bc0740 10 client.0 _ll_register_callbacks cb 0x55ae295b2c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae2965c590 0x55ae29668530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29660fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae2965c590 0x55ae29668530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35196/0 (socket says 172.21.15.105:35196) 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 -- 172.21.15.105:0/809258710 learned_addr learned my addr 172.21.15.105:0/809258710 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f3957fff640 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae29657420 0x55ae296614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 -- 172.21.15.105:0/809258710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 msgr2=0x55ae29660fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29660fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 -- 172.21.15.105:0/809258710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae29657420 msgr2=0x55ae296614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae29657420 0x55ae296614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 -- 172.21.15.105:0/809258710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae293997d0 con 0x55ae2965c590 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f395ed9d640 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae2965c590 0x55ae29668530 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3958006010 tx=0x7f39580bc450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cde858d56aab4ba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.899+0000 7f394dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f394dffb640 1 -- 172.21.15.105:0/809258710 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580c23a0 con 0x55ae2965c590 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f394dffb640 1 -- 172.21.15.105:0/809258710 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f39580c2120 con 0x55ae2965c590 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 1 -- 172.21.15.105:0/809258710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39500132b0 con 0x55ae2965c590 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f394dffb640 1 -- 172.21.15.105:0/809258710 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f39580c6c60 con 0x55ae2965c590 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 10 client.22608 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 1 -- 172.21.15.105:0/809258710 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae2966b8a0 con 0x55ae2965c590 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f394dffb640 1 -- 172.21.15.105:0/809258710 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39500132b0 con 0x55ae2965c590 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f394dffb640 1 -- 172.21.15.105:0/809258710 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3958004b40 con 0x55ae2965c590 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 10 client.22608 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:17.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 10 client.22608 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 20 client.22608 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ae2966cfd0 0x55ae2966f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 4 client.22608 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f3967bc0740 1 -- 172.21.15.105:0/809258710 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae295c8eb0 con 0x55ae2966cfd0 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.900+0000 7f395e59c640 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ae2966cfd0 0x55ae2966f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.901+0000 7f395e59c640 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ae2966cfd0 0x55ae2966f430 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e0bab7dc822c58c server_cookie=b10697c78b95b8f1 in_seq=0 out_seq=0 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.901+0000 7f394dffb640 10 client.22608 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.901+0000 7f394dffb640 1 -- 172.21.15.105:0/809258710 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ae295c8eb0 con 0x55ae2966cfd0 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.901+0000 7f394dffb640 10 client.22608 handle_command_reply: tid=0 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.901+0000 7f3967bc0740 1 client.22608 shutdown 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 -- 172.21.15.105:0/809258710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae2965c590 msgr2=0x55ae29668530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae2965c590 0x55ae29668530 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3958006010 tx=0x7f39580bc450 comp rx=0 tx=0).stop 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 -- 172.21.15.105:0/809258710 shutdown_connections 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ae2966cfd0 0x55ae2966f430 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f394dffb640 0 client.22608 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:58:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae2965c590 0x55ae29668530 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae29657420 0x55ae296614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 --2- 172.21.15.105:0/809258710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29660fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 -- 172.21.15.105:0/809258710 >> 172.21.15.105:0/809258710 conn(0x55ae29651fa0 msgr2=0x55ae296511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 -- 172.21.15.105:0/809258710 shutdown_connections 2024-03-26T17:58:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 1 -- 172.21.15.105:0/809258710 wait complete. 2024-03-26T17:58:17.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.902+0000 7f3967bc0740 20 client.22608 trim_cache size 0 max 16384 2024-03-26T17:58:17.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3967bc0740 1 Processor -- start 2024-03-26T17:58:17.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3967bc0740 1 -- start start 2024-03-26T17:58:17.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29650930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae296684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae29650e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3967bc0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39601aaf10 con 0x55ae29657420 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3967bc0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f39601aca50 con 0x55ae2965c590 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3967bc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39601ab5e0 con 0x55ae29656a30 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.920+0000 7f3957fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae296684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29650930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f3957fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae296684f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35208/0 (socket says 172.21.15.105:35208) 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29650930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54662/0 (socket says 172.21.15.105:54662) 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f3957fff640 1 -- 172.21.15.105:0/3763752290 learned_addr learned my addr 172.21.15.105:0/3763752290 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395ed9d640 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae29650e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395e59c640 1 -- 172.21.15.105:0/3763752290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 msgr2=0x55ae29650e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395e59c640 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae29650e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395e59c640 1 -- 172.21.15.105:0/3763752290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 msgr2=0x55ae296684f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395e59c640 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae296684f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395e59c640 1 -- 172.21.15.105:0/3763752290 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae29652d10 con 0x55ae29656a30 2024-03-26T17:58:17.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f3957fff640 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae296684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:17.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f395e59c640 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29650930 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55ae2966fce0 tx=0x7f3948014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d444db8598402f50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f394ffff640 1 -- 172.21.15.105:0/3763752290 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948012b40 con 0x55ae29656a30 2024-03-26T17:58:17.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f394ffff640 1 -- 172.21.15.105:0/3763752290 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3948012ce0 con 0x55ae29656a30 2024-03-26T17:58:17.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.921+0000 7f394ffff640 1 -- 172.21.15.105:0/3763752290 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480035f0 con 0x55ae29656a30 2024-03-26T17:58:17.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 -- 172.21.15.105:0/3763752290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 msgr2=0x55ae29650930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29650930 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x55ae2966fce0 tx=0x7f3948014000 comp rx=0 tx=0).stop 2024-03-26T17:58:17.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 -- 172.21.15.105:0/3763752290 shutdown_connections 2024-03-26T17:58:17.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae29650e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29657420 0x55ae296684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 --2- 172.21.15.105:0/3763752290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29656a30 0x55ae29650930 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 -- 172.21.15.105:0/3763752290 >> 172.21.15.105:0/3763752290 conn(0x55ae29651fa0 msgr2=0x55ae29658980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 -- 172.21.15.105:0/3763752290 shutdown_connections 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.922+0000 7f3967bc0740 1 -- 172.21.15.105:0/3763752290 wait complete. 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 1 -- start start 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 1 Processor -- start 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29656a30 0x55ae29665d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29657420 0x55ae29666250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae296671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f39601ac380 con 0x55ae29656a30 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f39601abcb0 con 0x55ae2965c590 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39601a8ef0 con 0x55ae29657420 2024-03-26T17:58:17.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3967bc0740 10 client.0 _ll_register_callbacks cb 0x7f39580bdbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29656a30 0x55ae29665d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29656a30 0x55ae29665d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35210/0 (socket says 172.21.15.105:35210) 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395e59c640 1 -- 172.21.15.105:0/2681614091 learned_addr learned my addr 172.21.15.105:0/2681614091 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395ed9d640 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae296671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f3957fff640 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29657420 0x55ae29666250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395e59c640 1 -- 172.21.15.105:0/2681614091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29657420 msgr2=0x55ae29666250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395e59c640 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29657420 0x55ae29666250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395e59c640 1 -- 172.21.15.105:0/2681614091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 msgr2=0x55ae296671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395e59c640 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae296671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.923+0000 7f395e59c640 1 -- 172.21.15.105:0/2681614091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae296677b0 con 0x55ae29656a30 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f394dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f395e59c640 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29656a30 0x55ae29665d10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3948011ef0 tx=0x7f3948012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8208492bd3d8b9ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f394dffb640 1 -- 172.21.15.105:0/2681614091 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480282d0 con 0x55ae29656a30 2024-03-26T17:58:17.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f394dffb640 1 -- 172.21.15.105:0/2681614091 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f3948004020 con 0x55ae29656a30 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f3967bc0740 1 -- 172.21.15.105:0/2681614091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3948012ce0 con 0x55ae29656a30 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f394dffb640 1 -- 172.21.15.105:0/2681614091 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3948004280 con 0x55ae29656a30 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f3967bc0740 10 client.22614 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f3967bc0740 1 -- 172.21.15.105:0/2681614091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae296d3440 con 0x55ae29656a30 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.924+0000 7f394dffb640 1 -- 172.21.15.105:0/2681614091 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3948012ce0 con 0x55ae29656a30 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f394dffb640 1 -- 172.21.15.105:0/2681614091 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3948032470 con 0x55ae29656a30 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f3967bc0740 10 client.22614 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f3967bc0740 10 client.22614 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f3967bc0740 20 client.22614 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:17.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f3967bc0740 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ae2966cfd0 0x55ae296d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f3967bc0740 4 client.22614 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f3967bc0740 1 -- 172.21.15.105:0/2681614091 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae29660da0 con 0x55ae2966cfd0 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f3957fff640 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ae2966cfd0 0x55ae296d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f394dffb640 10 client.22614 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f3957fff640 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ae2966cfd0 0x55ae296d5db0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49aa10ddd1cb8aaf server_cookie=54ceddd05f4f2d5c in_seq=0 out_seq=0 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f394dffb640 1 -- 172.21.15.105:0/2681614091 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae29660da0 con 0x55ae2966cfd0 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.925+0000 7f394dffb640 10 client.22614 handle_command_reply: tid=0 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 client.22614 shutdown 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 -- 172.21.15.105:0/2681614091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29656a30 msgr2=0x55ae29665d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29656a30 0x55ae29665d10 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3948011ef0 tx=0x7f3948012bf0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 -- 172.21.15.105:0/2681614091 shutdown_connections 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ae2966cfd0 0x55ae296d5db0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f394dffb640 0 client.22614 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:58:17.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ae2965c590 0x55ae296671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ae29657420 0x55ae29666250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 --2- 172.21.15.105:0/2681614091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ae29656a30 0x55ae29665d10 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 -- 172.21.15.105:0/2681614091 >> 172.21.15.105:0/2681614091 conn(0x55ae29651fa0 msgr2=0x55ae29655200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 -- 172.21.15.105:0/2681614091 shutdown_connections 2024-03-26T17:58:17.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 1 -- 172.21.15.105:0/2681614091 wait complete. 2024-03-26T17:58:17.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.926+0000 7f3967bc0740 20 client.22614 trim_cache size 0 max 16384 2024-03-26T17:58:17.917 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 -- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39400a2f20 msgr2=0x7f39400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39400a2f20 0x7f39400a53e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f396019b1c0 tx=0x7f3948009260 comp rx=0 tx=0).stop 2024-03-26T17:58:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 -- 172.21.15.105:0/1019163021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3960102390 msgr2=0x7f39601a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3960102390 0x7f39601a1260 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3958036040 tx=0x7f3958033c40 comp rx=0 tx=0).stop 2024-03-26T17:58:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 -- 172.21.15.105:0/1019163021 shutdown_connections 2024-03-26T17:58:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39400a2f20 0x7f39400a53e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3960102390 0x7f39601a1260 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39601019d0 0x7f396019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 --2- 172.21.15.105:0/1019163021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f39601007e0 0x7f3960199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:17.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 -- 172.21.15.105:0/1019163021 >> 172.21.15.105:0/1019163021 conn(0x7f39600fbd70 msgr2=0x7f39600fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:17.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 -- 172.21.15.105:0/1019163021 shutdown_connections 2024-03-26T17:58:17.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:17.927+0000 7f3964ef8640 1 -- 172.21.15.105:0/1019163021 wait complete. 2024-03-26T17:58:17.926 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:58:18.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.092+0000 7fb896a80640 1 -- 172.21.15.105:0/1419864379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901019b0 msgr2=0x7fb890101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.092+0000 7fb896a80640 1 --2- 172.21.15.105:0/1419864379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901019b0 0x7fb890101e30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00b9f0 tx=0x7fb87c02d8e0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.093+0000 7fb896a80640 1 -- 172.21.15.105:0/1419864379 shutdown_connections 2024-03-26T17:58:18.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.093+0000 7fb896a80640 1 --2- 172.21.15.105:0/1419864379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb890102370 0x7fb890108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.093+0000 7fb896a80640 1 --2- 172.21.15.105:0/1419864379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901019b0 0x7fb890101e30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.093+0000 7fb896a80640 1 --2- 172.21.15.105:0/1419864379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8901007c0 0x7fb890100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.093+0000 7fb896a80640 1 -- 172.21.15.105:0/1419864379 >> 172.21.15.105:0/1419864379 conn(0x7fb8900fbd50 msgr2=0x7fb8900fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.093+0000 7fb896a80640 1 -- 172.21.15.105:0/1419864379 shutdown_connections 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.093+0000 7fb896a80640 1 -- 172.21.15.105:0/1419864379 wait complete. 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.093+0000 7fb896a80640 1 Processor -- start 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb896a80640 1 -- start start 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb896a80640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901007c0 0x7fb890198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb896a80640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8901019b0 0x7fb8901991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb896a80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb890102370 0x7fb890195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb896a80640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010b1f0 con 0x7fb890102370 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb896a80640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010b070 con 0x7fb8901019b0 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb896a80640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb89010b370 con 0x7fb8901007c0 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901007c0 0x7fb890198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901007c0 0x7fb890198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54676/0 (socket says 172.21.15.105:54676) 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 -- 172.21.15.105:0/2745146638 learned_addr learned my addr 172.21.15.105:0/2745146638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89501d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb890102370 0x7fb890195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb88ffff640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8901019b0 0x7fb8901991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 -- 172.21.15.105:0/2745146638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8901019b0 msgr2=0x7fb8901991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8901019b0 0x7fb8901991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 -- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb890102370 msgr2=0x7fb890195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb890102370 0x7fb890195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 -- 172.21.15.105:0/2745146638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8901960e0 con 0x7fb8901007c0 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.094+0000 7fb89481c640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901007c0 0x7fb890198cb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e9c0 tx=0x7fb88400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.095+0000 7fb88dffb640 1 -- 172.21.15.105:0/2745146638 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88400cd80 con 0x7fb8901007c0 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.095+0000 7fb88dffb640 1 -- 172.21.15.105:0/2745146638 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8840035f0 con 0x7fb8901007c0 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.095+0000 7fb896a80640 1 -- 172.21.15.105:0/2745146638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8901a5f80 con 0x7fb8901007c0 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.095+0000 7fb88dffb640 1 -- 172.21.15.105:0/2745146638 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884010690 con 0x7fb8901007c0 2024-03-26T17:58:18.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.095+0000 7fb896a80640 1 -- 172.21.15.105:0/2745146638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8901a6330 con 0x7fb8901007c0 2024-03-26T17:58:18.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.097+0000 7fb88dffb640 1 -- 172.21.15.105:0/2745146638 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb8840037a0 con 0x7fb8901007c0 2024-03-26T17:58:18.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb88dffb640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8800a2f20 0x7fb8800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89965f740 1 Processor -- start 2024-03-26T17:58:18.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89965f740 1 -- start start 2024-03-26T17:58:18.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb88ffff640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8800a2f20 0x7fb8800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb88dffb640 1 -- 172.21.15.105:0/2745146638 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb8840cf900 con 0x7fb8901007c0 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89965f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89965f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653f420 0x55bc96543f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89965f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc96544590 0x55bc96546a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89965f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc964a3970 con 0x55bc96544590 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89965f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc9644bd70 con 0x55bc9653ea30 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89965f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc9649f920 con 0x55bc9653f420 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89481c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89501d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc96544590 0x55bc96546a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89481c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40242/0 (socket says 172.21.15.105:40242) 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.098+0000 7fb89481c640 1 -- 172.21.15.105:0/276523108 learned_addr learned my addr 172.21.15.105:0/276523108 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb89501d640 1 -- 172.21.15.105:0/276523108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653f420 msgr2=0x55bc96543f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb89501d640 1 --2- 172.21.15.105:0/276523108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653f420 0x55bc96543f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb89501d640 1 -- 172.21.15.105:0/276523108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 msgr2=0x55bc9653ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb89501d640 1 --2- 172.21.15.105:0/276523108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb89501d640 1 -- 172.21.15.105:0/276523108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc961b73f0 con 0x55bc96544590 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb88ffff640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8800a2f20 0x7fb8800a53e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009fb0 tx=0x7fb87c009f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb89481c640 1 --2- 172.21.15.105:0/276523108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb89501d640 1 --2- 172.21.15.105:0/276523108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc96544590 0x55bc96546a20 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb88800e450 tx=0x7fb8880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b6b7bb9cd9cb70 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb87affd640 1 -- 172.21.15.105:0/276523108 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888004b20 con 0x55bc96544590 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb87affd640 1 -- 172.21.15.105:0/276523108 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb888004cc0 con 0x55bc96544590 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.099+0000 7fb87affd640 1 -- 172.21.15.105:0/276523108 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888005560 con 0x55bc96544590 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 -- 172.21.15.105:0/276523108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc96544590 msgr2=0x55bc96546a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 --2- 172.21.15.105:0/276523108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc96544590 0x55bc96546a20 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb88800e450 tx=0x7fb8880095c0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 -- 172.21.15.105:0/276523108 shutdown_connections 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 --2- 172.21.15.105:0/276523108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc96544590 0x55bc96546a20 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 --2- 172.21.15.105:0/276523108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653f420 0x55bc96543f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 --2- 172.21.15.105:0/276523108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 -- 172.21.15.105:0/276523108 >> 172.21.15.105:0/276523108 conn(0x55bc96539fa0 msgr2=0x55bc9653a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 -- 172.21.15.105:0/276523108 shutdown_connections 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.100+0000 7fb89965f740 1 -- 172.21.15.105:0/276523108 wait complete. 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 1 -- start start 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 1 Processor -- start 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653ea30 0x55bc96548f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96549440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc96544590 0x55bc96550480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc96547540 con 0x55bc9653f420 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc9645f310 con 0x55bc96544590 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc965476c0 con 0x55bc9653ea30 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89965f740 10 client.0 _ll_register_callbacks cb 0x55bc9649ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96549440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96549440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35242/0 (socket says 172.21.15.105:35242) 2024-03-26T17:58:18.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb88ffff640 1 -- 172.21.15.105:0/3299044837 learned_addr learned my addr 172.21.15.105:0/3299044837 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89501d640 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc96544590 0x55bc96550480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.101+0000 7fb89481c640 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653ea30 0x55bc96548f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb88ffff640 1 -- 172.21.15.105:0/3299044837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653ea30 msgr2=0x55bc96548f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb88ffff640 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653ea30 0x55bc96548f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb88ffff640 1 -- 172.21.15.105:0/3299044837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc96544590 msgr2=0x55bc96550480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb88ffff640 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc96544590 0x55bc96550480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb88ffff640 1 -- 172.21.15.105:0/3299044837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc962817d0 con 0x55bc9653f420 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb88ffff640 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96549440 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb87c02de20 tx=0x7fb87c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edfb53ea68b39 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb86ffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb86ffff640 1 -- 172.21.15.105:0/3299044837 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c037040 con 0x55bc9653f420 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb89965f740 1 -- 172.21.15.105:0/3299044837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb888004cc0 con 0x55bc9653f420 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb86ffff640 1 -- 172.21.15.105:0/3299044837 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb87c004b90 con 0x55bc9653f420 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb86ffff640 1 -- 172.21.15.105:0/3299044837 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb87c004d90 con 0x55bc9653f420 2024-03-26T17:58:18.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb89965f740 10 client.22626 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.102+0000 7fb89965f740 1 -- 172.21.15.105:0/3299044837 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bc965537f0 con 0x55bc9653f420 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb86ffff640 1 -- 172.21.15.105:0/3299044837 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb888004cc0 con 0x55bc9653f420 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb86ffff640 1 -- 172.21.15.105:0/3299044837 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb87c059080 con 0x55bc9653f420 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb89965f740 10 client.22626 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb89965f740 10 client.22626 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb89965f740 20 client.22626 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb89965f740 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bc96555070 0x55bc965574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb89965f740 4 client.22626 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb89965f740 1 -- 172.21.15.105:0/3299044837 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc964b0eb0 con 0x55bc96555070 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.103+0000 7fb89481c640 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bc96555070 0x55bc965574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.104+0000 7fb89481c640 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bc96555070 0x55bc965574f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63ee8a6bf3caf5f8 server_cookie=a4308c82cdf21ad4 in_seq=0 out_seq=0 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.104+0000 7fb86ffff640 10 client.22626 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.104+0000 7fb86ffff640 1 -- 172.21.15.105:0/3299044837 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc964b0eb0 con 0x55bc96555070 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.104+0000 7fb86ffff640 10 client.22626 handle_command_reply: tid=0 2024-03-26T17:58:18.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.104+0000 7fb89965f740 1 client.22626 shutdown 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 -- 172.21.15.105:0/3299044837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 msgr2=0x55bc96549440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96549440 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb87c02de20 tx=0x7fb87c000bc0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 -- 172.21.15.105:0/3299044837 shutdown_connections 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bc96555070 0x55bc965574f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc96544590 0x55bc96550480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96549440 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 --2- 172.21.15.105:0/3299044837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc9653ea30 0x55bc96548f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 -- 172.21.15.105:0/3299044837 >> 172.21.15.105:0/3299044837 conn(0x55bc96539fa0 msgr2=0x55bc965391c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb86ffff640 0 client.22626 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 -- 172.21.15.105:0/3299044837 shutdown_connections 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 1 -- 172.21.15.105:0/3299044837 wait complete. 2024-03-26T17:58:18.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.105+0000 7fb89965f740 20 client.22626 trim_cache size 0 max 16384 2024-03-26T17:58:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89965f740 1 Processor -- start 2024-03-26T17:58:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89965f740 1 -- start start 2024-03-26T17:58:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89965f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89965f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96550440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89965f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc96540920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89965f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901af2e0 con 0x55bc9653f420 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89965f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901b0e20 con 0x55bc9653ea30 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89965f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901af9b0 con 0x55bc96544590 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96550440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89501d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc96540920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89481c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96550440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35254/0 (socket says 172.21.15.105:35254) 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb89501d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc96540920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54698/0 (socket says 172.21.15.105:54698) 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.125+0000 7fb88ffff640 1 -- 172.21.15.105:0/3571621882 learned_addr learned my addr 172.21.15.105:0/3571621882 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb89481c640 1 -- 172.21.15.105:0/3571621882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 msgr2=0x55bc96540920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb89481c640 1 --2- 172.21.15.105:0/3571621882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc96540920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb89481c640 1 -- 172.21.15.105:0/3571621882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 msgr2=0x55bc96550440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb89481c640 1 --2- 172.21.15.105:0/3571621882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96550440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb89481c640 1 -- 172.21.15.105:0/3571621882 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc9653ad10 con 0x55bc9653ea30 2024-03-26T17:58:18.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb88ffff640 1 --2- 172.21.15.105:0/3571621882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96550440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb89481c640 1 --2- 172.21.15.105:0/3571621882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653fc60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb8840a44d0 tx=0x7fb884005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb6c79668f91cc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb87a7fc640 1 -- 172.21.15.105:0/3571621882 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88407ae60 con 0x55bc9653ea30 2024-03-26T17:58:18.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb87a7fc640 1 -- 172.21.15.105:0/3571621882 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb88409bff0 con 0x55bc9653ea30 2024-03-26T17:58:18.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.126+0000 7fb87a7fc640 1 -- 172.21.15.105:0/3571621882 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884098660 con 0x55bc9653ea30 2024-03-26T17:58:18.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 -- 172.21.15.105:0/3571621882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 msgr2=0x55bc9653fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 --2- 172.21.15.105:0/3571621882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653fc60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb8840a44d0 tx=0x7fb884005f10 comp rx=0 tx=0).stop 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 -- 172.21.15.105:0/3571621882 shutdown_connections 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 --2- 172.21.15.105:0/3571621882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc96540920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 --2- 172.21.15.105:0/3571621882 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653f420 0x55bc96550440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 --2- 172.21.15.105:0/3571621882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653ea30 0x55bc9653fc60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 -- 172.21.15.105:0/3571621882 >> 172.21.15.105:0/3571621882 conn(0x55bc96539fa0 msgr2=0x55bc96548f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 -- 172.21.15.105:0/3571621882 shutdown_connections 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.127+0000 7fb89965f740 1 -- 172.21.15.105:0/3571621882 wait complete. 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 1 -- start start 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 1 Processor -- start 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653ea30 0x55bc9654d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653f420 0x55bc9654d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc965b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901b0750 con 0x55bc9653ea30 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901b0080 con 0x55bc9653f420 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901ad2c0 con 0x55bc96544590 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89965f740 10 client.0 _ll_register_callbacks cb 0x7fb884005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89481c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653ea30 0x55bc9654d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89481c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653ea30 0x55bc9654d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35260/0 (socket says 172.21.15.105:35260) 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89481c640 1 -- 172.21.15.105:0/2196711903 learned_addr learned my addr 172.21.15.105:0/2196711903 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653f420 0x55bc9654d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.128+0000 7fb89501d640 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc965b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb88ffff640 1 -- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 msgr2=0x55bc965b93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb88ffff640 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc965b93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb88ffff640 1 -- 172.21.15.105:0/2196711903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653ea30 msgr2=0x55bc9654d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb88ffff640 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653ea30 0x55bc9654d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb88ffff640 1 -- 172.21.15.105:0/2196711903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc96550670 con 0x55bc9653f420 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb89481c640 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653ea30 0x55bc9654d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb89501d640 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc965b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb88ffff640 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653f420 0x55bc9654d5f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00b460 tx=0x7fb87c004fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfa14c142e9753f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb86ffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb86ffff640 1 -- 172.21.15.105:0/2196711903 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c037040 con 0x55bc9653f420 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb86ffff640 1 -- 172.21.15.105:0/2196711903 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb87c050690 con 0x55bc9653f420 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb89965f740 1 -- 172.21.15.105:0/2196711903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb88409bff0 con 0x55bc9653f420 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb86ffff640 1 -- 172.21.15.105:0/2196711903 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb87c033640 con 0x55bc9653f420 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb89965f740 10 client.32318 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.129+0000 7fb89965f740 1 -- 172.21.15.105:0/2196711903 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55bc965ba980 con 0x55bc9653f420 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb86ffff640 1 -- 172.21.15.105:0/2196711903 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb88409bff0 con 0x55bc9653f420 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb86ffff640 1 -- 172.21.15.105:0/2196711903 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb87c044040 con 0x55bc9653f420 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb89965f740 10 client.32318 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:18.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb89965f740 10 client.32318 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb89965f740 20 client.32318 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb89965f740 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bc965bb2a0 0x55bc965bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb89965f740 4 client.32318 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb89965f740 1 -- 172.21.15.105:0/2196711903 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc96540430 con 0x55bc965bb2a0 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.130+0000 7fb89481c640 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bc965bb2a0 0x55bc965bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb86ffff640 10 client.32318 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89481c640 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bc965bb2a0 0x55bc965bd6c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d513da3b406cf2c1 server_cookie=2bc700877ccc2ddf in_seq=0 out_seq=0 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb86ffff640 1 -- 172.21.15.105:0/2196711903 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc96540430 con 0x55bc965bb2a0 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb86ffff640 10 client.32318 handle_command_reply: tid=0 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 client.32318 shutdown 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 -- 172.21.15.105:0/2196711903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653f420 msgr2=0x55bc9654d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653f420 0x55bc9654d5f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00b460 tx=0x7fb87c004fa0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 -- 172.21.15.105:0/2196711903 shutdown_connections 2024-03-26T17:58:18.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55bc965bb2a0 0x55bc965bd6c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc96544590 0x55bc965b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc9653f420 0x55bc9654d5f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb86ffff640 0 client.32318 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 --2- 172.21.15.105:0/2196711903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc9653ea30 0x55bc9654d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 -- 172.21.15.105:0/2196711903 >> 172.21.15.105:0/2196711903 conn(0x55bc96539fa0 msgr2=0x55bc96540920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 -- 172.21.15.105:0/2196711903 shutdown_connections 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 1 -- 172.21.15.105:0/2196711903 wait complete. 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.131+0000 7fb89965f740 20 client.32318 trim_cache size 0 max 16384 2024-03-26T17:58:18.122 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:18.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 -- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8800a2f20 msgr2=0x7fb8800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8800a2f20 0x7fb8800a53e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009fb0 tx=0x7fb87c009f40 comp rx=0 tx=0).stop 2024-03-26T17:58:18.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 -- 172.21.15.105:0/2745146638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901007c0 msgr2=0x7fb890198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901007c0 0x7fb890198cb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e9c0 tx=0x7fb88400ee80 comp rx=0 tx=0).stop 2024-03-26T17:58:18.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 -- 172.21.15.105:0/2745146638 shutdown_connections 2024-03-26T17:58:18.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb8800a2f20 0x7fb8800a53e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb890102370 0x7fb890195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb8901019b0 0x7fb8901991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 --2- 172.21.15.105:0/2745146638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8901007c0 0x7fb890198cb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.132+0000 7fb896a80640 1 -- 172.21.15.105:0/2745146638 >> 172.21.15.105:0/2745146638 conn(0x7fb8900fbd50 msgr2=0x7fb890104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.133+0000 7fb896a80640 1 -- 172.21.15.105:0/2745146638 shutdown_connections 2024-03-26T17:58:18.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.133+0000 7fb896a80640 1 -- 172.21.15.105:0/2745146638 wait complete. 2024-03-26T17:58:18.128 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:17 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3479261076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:58:18.129 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:58:18.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 -- 172.21.15.105:0/3784381761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 msgr2=0x7f75d0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 --2- 172.21.15.105:0/3784381761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 0x7f75d0108c00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009be0 tx=0x7f75c802bef0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 -- 172.21.15.105:0/3784381761 shutdown_connections 2024-03-26T17:58:18.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 --2- 172.21.15.105:0/3784381761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 0x7f75d0108c00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 --2- 172.21.15.105:0/3784381761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75d01019b0 0x7f75d0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 --2- 172.21.15.105:0/3784381761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75d01007c0 0x7f75d0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 -- 172.21.15.105:0/3784381761 >> 172.21.15.105:0/3784381761 conn(0x7f75d00fbd50 msgr2=0x7f75d00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 -- 172.21.15.105:0/3784381761 shutdown_connections 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.285+0000 7f75d84fe640 1 -- 172.21.15.105:0/3784381761 wait complete. 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d84fe640 1 Processor -- start 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d84fe640 1 -- start start 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d84fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75d01007c0 0x7f75d0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d84fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75d01019b0 0x7f75d019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d84fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 0x7f75d01a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d84fe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010c320 con 0x7f75d01019b0 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d84fe640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010c1a0 con 0x7f75d01007c0 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d84fe640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75d010c4a0 con 0x7f75d0102370 2024-03-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d5a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75d01019b0 0x7f75d019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d629a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75d01007c0 0x7f75d0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d5a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75d01019b0 0x7f75d019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35286/0 (socket says 172.21.15.105:35286) 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d6a9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 0x7f75d01a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.286+0000 7f75d5a99640 1 -- 172.21.15.105:0/1588551803 learned_addr learned my addr 172.21.15.105:0/1588551803 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d5a99640 1 -- 172.21.15.105:0/1588551803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 msgr2=0x7f75d01a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d5a99640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 0x7f75d01a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d5a99640 1 -- 172.21.15.105:0/1588551803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75d01007c0 msgr2=0x7f75d0199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d5a99640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75d01007c0 0x7f75d0199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d5a99640 1 -- 172.21.15.105:0/1588551803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01a1920 con 0x7f75d01019b0 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 0x7f75d01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d5a99640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75d01019b0 0x7f75d019a250 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f75c000e9c0 tx=0x7f75c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75c77fe640 1 -- 172.21.15.105:0/1588551803 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c000cde0 con 0x7f75d01019b0 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75c77fe640 1 -- 172.21.15.105:0/1588551803 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c00035f0 con 0x7f75d01019b0 2024-03-26T17:58:18.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d84fe640 1 -- 172.21.15.105:0/1588551803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01a1c10 con 0x7f75d01019b0 2024-03-26T17:58:18.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75c77fe640 1 -- 172.21.15.105:0/1588551803 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c0010690 con 0x7f75d01019b0 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.287+0000 7f75d84fe640 1 -- 172.21.15.105:0/1588551803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a2160 con 0x7f75d01019b0 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.290+0000 7f75c77fe640 1 -- 172.21.15.105:0/1588551803 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f75c00037a0 con 0x7f75d01019b0 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.290+0000 7f75db1bd740 1 Processor -- start 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.290+0000 7f75db1bd740 1 -- start start 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e18730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e1b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75db1bd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556244d6f970 con 0x556244e13c30 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75db1bd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556244d17d70 con 0x556244e13240 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75db1bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556244d6b920 con 0x556244e18da0 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d6a9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d6a9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e13660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40292/0 (socket says 172.21.15.105:40292) 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d629a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e18730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d6a9b640 1 -- 172.21.15.105:0/59971935 learned_addr learned my addr 172.21.15.105:0/59971935 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d5a99640 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e1b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d6a9b640 1 -- 172.21.15.105:0/59971935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 msgr2=0x556244e1b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e1b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d6a9b640 1 -- 172.21.15.105:0/59971935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 msgr2=0x556244e18730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e18730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d6a9b640 1 -- 172.21.15.105:0/59971935 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556244a833f0 con 0x556244e13240 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d5a99640 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e1b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.291+0000 7f75d629a640 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e18730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.292+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e13660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f75c802c500 tx=0x7f75c8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=814ca4cd187eda6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.292+0000 7f75c4bfe640 1 -- 172.21.15.105:0/59971935 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8043030 con 0x556244e13240 2024-03-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.292+0000 7f75c4bfe640 1 -- 172.21.15.105:0/59971935 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c8003970 con 0x556244e13240 2024-03-26T17:58:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.292+0000 7f75c4bfe640 1 -- 172.21.15.105:0/59971935 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c802cae0 con 0x556244e13240 2024-03-26T17:58:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.292+0000 7f75c77fe640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75bc0a2f20 0x7f75bc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.292+0000 7f75d629a640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75bc0a2f20 0x7f75bc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.292+0000 7f75c77fe640 1 -- 172.21.15.105:0/1588551803 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f75c00cf7f0 con 0x7f75d01019b0 2024-03-26T17:58:18.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 -- 172.21.15.105:0/59971935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 msgr2=0x556244e13660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e13660 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f75c802c500 tx=0x7f75c8009840 comp rx=0 tx=0).stop 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75d629a640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75bc0a2f20 0x7f75bc0a53e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x556244e14030 tx=0x7f75cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 -- 172.21.15.105:0/59971935 shutdown_connections 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e1b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e18730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 --2- 172.21.15.105:0/59971935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e13660 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 -- 172.21.15.105:0/59971935 >> 172.21.15.105:0/59971935 conn(0x556244e0e7b0 msgr2=0x556244e0ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 -- 172.21.15.105:0/59971935 shutdown_connections 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 -- 172.21.15.105:0/59971935 wait complete. 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.293+0000 7f75db1bd740 1 -- start start 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75db1bd740 1 Processor -- start 2024-03-26T17:58:18.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e1d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e1dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e24c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75db1bd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556244e1bcf0 con 0x556244e13c30 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75db1bd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556244d2b310 con 0x556244e13240 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75db1bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556244e1be70 con 0x556244e18da0 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75db1bd740 10 client.0 _ll_register_callbacks cb 0x556244d66c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d5a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e1dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d6a9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e24c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d629a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e1d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d5a99640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e1dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35304/0 (socket says 172.21.15.105:35304) 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d5a99640 1 -- 172.21.15.105:0/1004779018 learned_addr learned my addr 172.21.15.105:0/1004779018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d629a640 1 -- 172.21.15.105:0/1004779018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 msgr2=0x556244e24c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d629a640 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e24c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d629a640 1 -- 172.21.15.105:0/1004779018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 msgr2=0x556244e1dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d629a640 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e1dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.294+0000 7f75d629a640 1 -- 172.21.15.105:0/1004779018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556244b4d7d0 con 0x556244e13240 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75d629a640 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e1d6e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x556244e0f720 tx=0x7f75cc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4963073ce38c815 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75b9ffb640 1 -- 172.21.15.105:0/1004779018 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc004020 con 0x556244e13240 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75db1bd740 1 -- 172.21.15.105:0/1004779018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c8003970 con 0x556244e13240 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75b9ffb640 1 -- 172.21.15.105:0/1004779018 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f75cc017030 con 0x556244e13240 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75b9ffb640 1 -- 172.21.15.105:0/1004779018 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f75cc0041c0 con 0x556244e13240 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75db1bd740 10 client.32330 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75db1bd740 1 -- 172.21.15.105:0/1004779018 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556244e28090 con 0x556244e13240 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.295+0000 7f75b9ffb640 1 -- 172.21.15.105:0/1004779018 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c8003970 con 0x556244e13240 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.296+0000 7f75b9ffb640 1 -- 172.21.15.105:0/1004779018 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f75cc027530 con 0x556244e13240 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.296+0000 7f75db1bd740 10 client.32330 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.296+0000 7f75db1bd740 10 client.32330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:58:18.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.296+0000 7f75db1bd740 20 client.32330 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.296+0000 7f75db1bd740 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556244e29670 0x556244e2bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.296+0000 7f75db1bd740 4 client.32330 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.296+0000 7f75db1bd740 1 -- 172.21.15.105:0/1004779018 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556244d7ceb0 con 0x556244e29670 2024-03-26T17:58:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.296+0000 7f75d5a99640 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556244e29670 0x556244e2bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.297+0000 7f75d5a99640 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556244e29670 0x556244e2bb10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20f0377bf1d4e148 server_cookie=f5f3116e7a3263dc in_seq=0 out_seq=0 2024-03-26T17:58:18.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.297+0000 7f75b9ffb640 10 client.32330 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:58:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.297+0000 7f75b9ffb640 1 -- 172.21.15.105:0/1004779018 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556244d7ceb0 con 0x556244e29670 2024-03-26T17:58:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.297+0000 7f75b9ffb640 10 client.32330 handle_command_reply: tid=0 2024-03-26T17:58:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.297+0000 7f75db1bd740 1 client.32330 shutdown 2024-03-26T17:58:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 -- 172.21.15.105:0/1004779018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 msgr2=0x556244e1d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e1d6e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x556244e0f720 tx=0x7f75cc004730 comp rx=0 tx=0).stop 2024-03-26T17:58:18.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 -- 172.21.15.105:0/1004779018 shutdown_connections 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e18da0 0x556244e24c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e13c30 0x556244e1dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556244e29670 0x556244e2bb10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 --2- 172.21.15.105:0/1004779018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e1d6e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 -- 172.21.15.105:0/1004779018 >> 172.21.15.105:0/1004779018 conn(0x556244e0e7b0 msgr2=0x556244e051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75b9ffb640 0 client.32330 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 -- 172.21.15.105:0/1004779018 shutdown_connections 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 1 -- 172.21.15.105:0/1004779018 wait complete. 2024-03-26T17:58:18.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.298+0000 7f75db1bd740 20 client.32330 trim_cache size 0 max 16384 2024-03-26T17:58:18.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75db1bd740 1 Processor -- start 2024-03-26T17:58:18.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75db1bd740 1 -- start start 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13240 0x556244e1db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13c30 0x556244e24c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e04930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75db1bd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a6c40 con 0x556244e18da0 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75db1bd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a8780 con 0x556244e13c30 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75db1bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a7310 con 0x556244e13240 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75d629a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13240 0x556244e1db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75d629a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13240 0x556244e1db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54750/0 (socket says 172.21.15.105:54750) 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75d629a640 1 -- 172.21.15.105:0/256599498 learned_addr learned my addr 172.21.15.105:0/256599498 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75d5a99640 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13c30 0x556244e24c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.319+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e04930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75d6a9b640 1 -- 172.21.15.105:0/256599498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13240 msgr2=0x556244e1db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13240 0x556244e1db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75d6a9b640 1 -- 172.21.15.105:0/256599498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13c30 msgr2=0x556244e24c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13c30 0x556244e24c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75d6a9b640 1 -- 172.21.15.105:0/256599498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556244e1c7e0 con 0x556244e18da0 2024-03-26T17:58:18.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e04930 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f75c8004af0 tx=0x7f75c8009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3619051a70d8c3f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75bbfff640 1 -- 172.21.15.105:0/256599498 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c804c070 con 0x556244e18da0 2024-03-26T17:58:18.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75bbfff640 1 -- 172.21.15.105:0/256599498 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c8003e20 con 0x556244e18da0 2024-03-26T17:58:18.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.320+0000 7f75bbfff640 1 -- 172.21.15.105:0/256599498 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8043040 con 0x556244e18da0 2024-03-26T17:58:18.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 -- 172.21.15.105:0/256599498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 msgr2=0x556244e04930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e04930 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f75c8004af0 tx=0x7f75c8009280 comp rx=0 tx=0).stop 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 -- 172.21.15.105:0/256599498 shutdown_connections 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e04930 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13c30 0x556244e24c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 --2- 172.21.15.105:0/256599498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13240 0x556244e1db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 -- 172.21.15.105:0/256599498 >> 172.21.15.105:0/256599498 conn(0x556244e0e7b0 msgr2=0x556244e0eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 -- 172.21.15.105:0/256599498 shutdown_connections 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 -- 172.21.15.105:0/256599498 wait complete. 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.321+0000 7f75db1bd740 1 -- start start 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75db1bd740 1 Processor -- start 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13c30 0x556244e22730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75db1bd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75db1bd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a80b0 con 0x556244e18da0 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75db1bd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a79e0 con 0x556244e13240 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75db1bd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01a4c80 con 0x556244e13c30 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75db1bd740 10 client.0 _ll_register_callbacks cb 0x7f75cc00b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d6a9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d6a9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e236a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35324/0 (socket says 172.21.15.105:35324) 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d6a9b640 1 -- 172.21.15.105:0/3828747038 learned_addr learned my addr 172.21.15.105:0/3828747038 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d629a640 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d5a99640 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13c30 0x556244e22730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d6a9b640 1 -- 172.21.15.105:0/3828747038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13c30 msgr2=0x556244e22730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13c30 0x556244e22730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d6a9b640 1 -- 172.21.15.105:0/3828747038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 msgr2=0x556244e221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.322+0000 7f75d6a9b640 1 -- 172.21.15.105:0/3828747038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556244e23c90 con 0x556244e18da0 2024-03-26T17:58:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.323+0000 7f75d6a9b640 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e236a0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009600 tx=0x7f75c802cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e62c73b94820ea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.323+0000 7f75b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.323+0000 7f75b9ffb640 1 -- 172.21.15.105:0/3828747038 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8043050 con 0x556244e18da0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.323+0000 7f75b9ffb640 1 -- 172.21.15.105:0/3828747038 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f75c8003970 con 0x556244e18da0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.323+0000 7f75db1bd740 1 -- 172.21.15.105:0/3828747038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c8003e20 con 0x556244e18da0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.323+0000 7f75b9ffb640 1 -- 172.21.15.105:0/3828747038 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f75c804c040 con 0x556244e18da0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.323+0000 7f75db1bd740 10 client.22650 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.323+0000 7f75db1bd740 1 -- 172.21.15.105:0/3828747038 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556244e24f90 con 0x556244e18da0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75b9ffb640 1 -- 172.21.15.105:0/3828747038 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c8003e20 con 0x556244e18da0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75b9ffb640 1 -- 172.21.15.105:0/3828747038 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f75c8003fd0 con 0x556244e18da0 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75db1bd740 10 client.22650 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75db1bd740 10 client.22650 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75db1bd740 20 client.22650 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75db1bd740 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556244e8fb60 0x556244e91f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75db1bd740 4 client.22650 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75db1bd740 1 -- 172.21.15.105:0/3828747038 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556244d77200 con 0x556244e8fb60 2024-03-26T17:58:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.324+0000 7f75d629a640 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556244e8fb60 0x556244e91f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75d629a640 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556244e8fb60 0x556244e91f80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7ff2582dff5fcdb server_cookie=7e86f60b22eccbdb in_seq=0 out_seq=0 2024-03-26T17:58:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75b9ffb640 10 client.22650 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:58:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75b9ffb640 1 -- 172.21.15.105:0/3828747038 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556244d77200 con 0x556244e8fb60 2024-03-26T17:58:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75b9ffb640 10 client.22650 handle_command_reply: tid=0 2024-03-26T17:58:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 client.22650 shutdown 2024-03-26T17:58:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 -- 172.21.15.105:0/3828747038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 msgr2=0x556244e236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e236a0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009600 tx=0x7f75c802cd80 comp rx=0 tx=0).stop 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 -- 172.21.15.105:0/3828747038 shutdown_connections 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556244e18da0 0x556244e236a0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556244e13c30 0x556244e22730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x556244e8fb60 0x556244e91f80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75b9ffb640 0 client.22650 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 --2- 172.21.15.105:0/3828747038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556244e13240 0x556244e221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.325+0000 7f75db1bd740 1 -- 172.21.15.105:0/3828747038 >> 172.21.15.105:0/3828747038 conn(0x556244e0e7b0 msgr2=0x556244e11a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75db1bd740 1 -- 172.21.15.105:0/3828747038 shutdown_connections 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75db1bd740 1 -- 172.21.15.105:0/3828747038 wait complete. 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75db1bd740 20 client.22650 trim_cache size 0 max 16384 2024-03-26T17:58:18.315 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:18.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75d84fe640 1 -- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75bc0a2f20 msgr2=0x7f75bc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75d84fe640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75bc0a2f20 0x7f75bc0a53e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x556244e14030 tx=0x7f75cc00b390 comp rx=0 tx=0).stop 2024-03-26T17:58:18.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75d84fe640 1 -- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75d01019b0 msgr2=0x7f75d019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75d84fe640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75d01019b0 0x7f75d019a250 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f75c000e9c0 tx=0x7f75c000ee80 comp rx=0 tx=0).stop 2024-03-26T17:58:18.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75d84fe640 1 -- 172.21.15.105:0/1588551803 shutdown_connections 2024-03-26T17:58:18.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75d84fe640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f75bc0a2f20 0x7f75bc0a53e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75d84fe640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0102370 0x7f75d01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.326+0000 7f75d84fe640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f75d01019b0 0x7f75d019a250 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.327+0000 7f75d84fe640 1 --2- 172.21.15.105:0/1588551803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75d01007c0 0x7f75d0199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.327+0000 7f75d84fe640 1 -- 172.21.15.105:0/1588551803 >> 172.21.15.105:0/1588551803 conn(0x7f75d00fbd50 msgr2=0x7f75d00fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.327+0000 7f75d84fe640 1 -- 172.21.15.105:0/1588551803 shutdown_connections 2024-03-26T17:58:18.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.327+0000 7f75d84fe640 1 -- 172.21.15.105:0/1588551803 wait complete. 2024-03-26T17:58:18.325 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:58:18.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:17 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3479261076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:58:18.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:17 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3479261076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 -- 172.21.15.105:0/1245866422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f391c1019b0 msgr2=0x7f391c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 --2- 172.21.15.105:0/1245866422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f391c1019b0 0x7f391c101e30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3910009860 tx=0x7f391002bb70 comp rx=0 tx=0).stop 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 -- 172.21.15.105:0/1245866422 shutdown_connections 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 --2- 172.21.15.105:0/1245866422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c102370 0x7f391c068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 --2- 172.21.15.105:0/1245866422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f391c1019b0 0x7f391c101e30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 --2- 172.21.15.105:0/1245866422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f391c1007c0 0x7f391c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 -- 172.21.15.105:0/1245866422 >> 172.21.15.105:0/1245866422 conn(0x7f391c0fbd50 msgr2=0x7f391c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 -- 172.21.15.105:0/1245866422 shutdown_connections 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.484+0000 7f3922875640 1 -- 172.21.15.105:0/1245866422 wait complete. 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f3922875640 1 Processor -- start 2024-03-26T17:58:18.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f3922875640 1 -- start start 2024-03-26T17:58:18.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f3922875640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c1007c0 0x7f391c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f3922875640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f391c1019b0 0x7f391c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f3922875640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f391c102370 0x7f391c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f3922875640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10f610 con 0x7f391c1007c0 2024-03-26T17:58:18.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f3922875640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10f490 con 0x7f391c1019b0 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f3922875640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10f790 con 0x7f391c102370 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.485+0000 7f391bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c1007c0 0x7f391c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f391c1019b0 0x7f391c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c1007c0 0x7f391c199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35346/0 (socket says 172.21.15.105:35346) 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391bfff640 1 -- 172.21.15.105:0/3497840111 learned_addr learned my addr 172.21.15.105:0/3497840111 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f3920e12640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f391c102370 0x7f391c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391bfff640 1 -- 172.21.15.105:0/3497840111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f391c102370 msgr2=0x7f391c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391bfff640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f391c102370 0x7f391c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391bfff640 1 -- 172.21.15.105:0/3497840111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f391c1019b0 msgr2=0x7f391c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391bfff640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f391c1019b0 0x7f391c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391bfff640 1 -- 172.21.15.105:0/3497840111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c1a1940 con 0x7f391c1007c0 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f391bfff640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c1007c0 0x7f391c199d30 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f390c00d950 tx=0x7f390c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f39197fa640 1 -- 172.21.15.105:0/3497840111 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c017020 con 0x7f391c1007c0 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f3922875640 1 -- 172.21.15.105:0/3497840111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c1a1c30 con 0x7f391c1007c0 2024-03-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f39197fa640 1 -- 172.21.15.105:0/3497840111 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c009c30 con 0x7f391c1007c0 2024-03-26T17:58:18.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f39197fa640 1 -- 172.21.15.105:0/3497840111 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c00f6b0 con 0x7f391c1007c0 2024-03-26T17:58:18.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.486+0000 7f3922875640 1 -- 172.21.15.105:0/3497840111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a21d0 con 0x7f391c1007c0 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f392545b740 1 Processor -- start 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f39197fa640 1 -- 172.21.15.105:0/3497840111 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f390c01a020 con 0x7f391c1007c0 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f392545b740 1 -- start start 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f392545b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db66e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f392545b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db6bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f392545b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db6ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f392545b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563e9dacb970 con 0x563e9db67420 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f392545b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e9da73d70 con 0x563e9db6c590 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f392545b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e9dac7920 con 0x563e9db66a30 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f3920e12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db66e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.489+0000 7f391bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db6bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f3920e12640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db66e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54780/0 (socket says 172.21.15.105:54780) 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f3920e12640 1 -- 172.21.15.105:0/3009172413 learned_addr learned my addr 172.21.15.105:0/3009172413 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f391b7fe640 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db6ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f39197fa640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39040a2f20 0x7f39040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f391bfff640 1 -- 172.21.15.105:0/3009172413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 msgr2=0x563e9db66e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f391bfff640 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db66e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f391bfff640 1 -- 172.21.15.105:0/3009172413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 msgr2=0x563e9db6ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f391bfff640 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db6ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f391bfff640 1 -- 172.21.15.105:0/3009172413 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9d7df3f0 con 0x563e9db67420 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f3920e12640 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db66e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f3920e12640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39040a2f20 0x7f39040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f39197fa640 1 -- 172.21.15.105:0/3497840111 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f390c0cf020 con 0x7f391c1007c0 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.490+0000 7f391bfff640 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db6bf20 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f390c0ccb50 tx=0x7f390c0afa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d29a899bf553f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.491+0000 7f39027fc640 1 -- 172.21.15.105:0/3009172413 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c09b580 con 0x563e9db67420 2024-03-26T17:58:18.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.491+0000 7f39027fc640 1 -- 172.21.15.105:0/3009172413 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c09beb0 con 0x563e9db67420 2024-03-26T17:58:18.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.491+0000 7f39027fc640 1 -- 172.21.15.105:0/3009172413 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c0998f0 con 0x563e9db67420 2024-03-26T17:58:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.491+0000 7f392545b740 1 -- 172.21.15.105:0/3009172413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 msgr2=0x563e9db6bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.491+0000 7f3920e12640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39040a2f20 0x7f39040a53e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x563e9db67c80 tx=0x7f3914004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.491+0000 7f392545b740 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db6bf20 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f390c0ccb50 tx=0x7f390c0afa40 comp rx=0 tx=0).stop 2024-03-26T17:58:18.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 -- 172.21.15.105:0/3009172413 shutdown_connections 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db6ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db6bf20 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 --2- 172.21.15.105:0/3009172413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db66e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 -- 172.21.15.105:0/3009172413 >> 172.21.15.105:0/3009172413 conn(0x563e9db61fa0 msgr2=0x563e9db623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 -- 172.21.15.105:0/3009172413 shutdown_connections 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 -- 172.21.15.105:0/3009172413 wait complete. 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 -- start start 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.492+0000 7f392545b740 1 Processor -- start 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f392545b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db66a30 0x563e9db70f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f392545b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db71490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f392545b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db6c590 0x563e9db784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db71490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f392545b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563e9db6f540 con 0x563e9db67420 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db71490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35372/0 (socket says 172.21.15.105:35372) 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f392545b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e9da87310 con 0x563e9db66a30 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 -- 172.21.15.105:0/2240359657 learned_addr learned my addr 172.21.15.105:0/2240359657 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563e9db6f6c0 con 0x563e9db6c590 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391bfff640 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db66a30 0x563e9db70f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f392545b740 10 client.0 _ll_register_callbacks cb 0x563e9dac2c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f3920e12640 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db6c590 0x563e9db784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 -- 172.21.15.105:0/2240359657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db6c590 msgr2=0x563e9db784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db6c590 0x563e9db784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 -- 172.21.15.105:0/2240359657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db66a30 msgr2=0x563e9db70f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db66a30 0x563e9db70f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 -- 172.21.15.105:0/2240359657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9d8a97d0 con 0x563e9db67420 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391bfff640 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db66a30 0x563e9db70f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f391b7fe640 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db71490 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f3910009780 tx=0x7f3910034be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd6d06360bcf2a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.493+0000 7f38fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.494+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2240359657 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910039040 con 0x563e9db67420 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.494+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390c09beb0 con 0x563e9db67420 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.494+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2240359657 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f3910004450 con 0x563e9db67420 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.494+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2240359657 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f391004e050 con 0x563e9db67420 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.494+0000 7f392545b740 10 client.22662 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.494+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e9db7b8e0 con 0x563e9db67420 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.494+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2240359657 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c09beb0 con 0x563e9db67420 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2240359657 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3910003a80 con 0x563e9db67420 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f392545b740 10 client.22662 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f392545b740 10 client.22662 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f392545b740 20 client.22662 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f392545b740 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e9db7d380 0x563e9db7f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f392545b740 4 client.22662 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:18.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e9dad8eb0 con 0x563e9db7d380 2024-03-26T17:58:18.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f391bfff640 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e9db7d380 0x563e9db7f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.495+0000 7f38fb7fe640 10 client.22662 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:58:18.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.496+0000 7f391bfff640 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e9db7d380 0x563e9db7f840 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=136bc33d420f3dda server_cookie=921eea6b0ee6e55b in_seq=0 out_seq=0 2024-03-26T17:58:18.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.496+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2240359657 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563e9dad8eb0 con 0x563e9db7d380 2024-03-26T17:58:18.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.496+0000 7f38fb7fe640 10 client.22662 handle_command_reply: tid=0 2024-03-26T17:58:18.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 client.22662 shutdown 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 msgr2=0x563e9db71490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db71490 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f3910009780 tx=0x7f3910034be0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 shutdown_connections 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e9db7d380 0x563e9db7f840 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db6c590 0x563e9db784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db67420 0x563e9db71490 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 --2- 172.21.15.105:0/2240359657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db66a30 0x563e9db70f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 >> 172.21.15.105:0/2240359657 conn(0x563e9db61fa0 msgr2=0x563e9db611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f38fb7fe640 0 client.22662 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.497+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 shutdown_connections 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.498+0000 7f392545b740 1 -- 172.21.15.105:0/2240359657 wait complete. 2024-03-26T17:58:18.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.498+0000 7f392545b740 20 client.22662 trim_cache size 0 max 16384 2024-03-26T17:58:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f392545b740 1 Processor -- start 2024-03-26T17:58:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f392545b740 1 -- start start 2024-03-26T17:58:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f392545b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db70f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f392545b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db71490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f392545b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db68920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f392545b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1aaee0 con 0x563e9db78490 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f392545b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1aca20 con 0x563e9db6c590 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f392545b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ab5b0 con 0x563e9db66a30 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f3920e12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db68920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.514+0000 7f391bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db70f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f3920e12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db68920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35378/0 (socket says 172.21.15.105:35378) 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f3920e12640 1 -- 172.21.15.105:0/1600496392 learned_addr learned my addr 172.21.15.105:0/1600496392 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f391b7fe640 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db71490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f391bfff640 1 -- 172.21.15.105:0/1600496392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 msgr2=0x563e9db71490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f391bfff640 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db71490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f391bfff640 1 -- 172.21.15.105:0/1600496392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 msgr2=0x563e9db68920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f391bfff640 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db68920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f391bfff640 1 -- 172.21.15.105:0/1600496392 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e9db70030 con 0x563e9db66a30 2024-03-26T17:58:18.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f3920e12640 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db68920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f391bfff640 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db70f50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f390c09c060 tx=0x7f390c0997c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1027f1641c124521 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f3901ffb640 1 -- 172.21.15.105:0/1600496392 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c014070 con 0x563e9db66a30 2024-03-26T17:58:18.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f3901ffb640 1 -- 172.21.15.105:0/1600496392 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c09beb0 con 0x563e9db66a30 2024-03-26T17:58:18.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.515+0000 7f3901ffb640 1 -- 172.21.15.105:0/1600496392 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c09b110 con 0x563e9db66a30 2024-03-26T17:58:18.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 -- 172.21.15.105:0/1600496392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 msgr2=0x563e9db70f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db70f50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f390c09c060 tx=0x7f390c0997c0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 -- 172.21.15.105:0/1600496392 shutdown_connections 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db68920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db71490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 --2- 172.21.15.105:0/1600496392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db70f50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 -- 172.21.15.105:0/1600496392 >> 172.21.15.105:0/1600496392 conn(0x563e9db61fa0 msgr2=0x563e9db7f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 -- 172.21.15.105:0/1600496392 shutdown_connections 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 -- 172.21.15.105:0/1600496392 wait complete. 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.516+0000 7f392545b740 1 -- start start 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f392545b740 1 Processor -- start 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f392545b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f392545b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db75800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f392545b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db767c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f392545b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1ac350 con 0x563e9db78490 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f392545b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1abc80 con 0x563e9db6c590 2024-03-26T17:58:18.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f392545b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1a8ec0 con 0x563e9db66a30 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f392545b740 10 client.0 _ll_register_callbacks cb 0x7f39100096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f391bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f391bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db752c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54816/0 (socket says 172.21.15.105:54816) 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f391bfff640 1 -- 172.21.15.105:0/2943499350 learned_addr learned my addr 172.21.15.105:0/2943499350 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f3920e12640 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db767c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f391b7fe640 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db75800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f391bfff640 1 -- 172.21.15.105:0/2943499350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 msgr2=0x563e9db75800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f391bfff640 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db75800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.517+0000 7f391bfff640 1 -- 172.21.15.105:0/2943499350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 msgr2=0x563e9db767c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f391bfff640 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db767c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f391bfff640 1 -- 172.21.15.105:0/2943499350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9db76db0 con 0x563e9db66a30 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f3920e12640 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db767c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f391bfff640 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db752c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c0ba400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca8c5c80601e63b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f38fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:58:18.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2943499350 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c014040 con 0x563e9db66a30 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2943499350 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f390c0860d0 con 0x563e9db66a30 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f392545b740 1 -- 172.21.15.105:0/2943499350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390c09beb0 con 0x563e9db66a30 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2943499350 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f390c09b2f0 con 0x563e9db66a30 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f392545b740 10 client.32338 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f392545b740 1 -- 172.21.15.105:0/2943499350 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563e9db780b0 con 0x563e9db66a30 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.518+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2943499350 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f390c09beb0 con 0x563e9db66a30 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2943499350 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f390c024060 con 0x563e9db66a30 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f392545b740 10 client.32338 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f392545b740 10 client.32338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f392545b740 20 client.32338 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f392545b740 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e9dbe3820 0x563e9dbe5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f392545b740 4 client.32338 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f392545b740 1 -- 172.21.15.105:0/2943499350 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e9dad3200 con 0x563e9dbe3820 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f391b7fe640 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e9dbe3820 0x563e9dbe5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f38fb7fe640 10 client.32338 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.519+0000 7f391b7fe640 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e9dbe3820 0x563e9dbe5c10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d31319a349a77cc3 server_cookie=151d53021bf45be3 in_seq=0 out_seq=0 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f38fb7fe640 1 -- 172.21.15.105:0/2943499350 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e9dad3200 con 0x563e9dbe3820 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f38fb7fe640 10 client.32338 handle_command_reply: tid=0 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 client.32338 shutdown 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 -- 172.21.15.105:0/2943499350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 msgr2=0x563e9db752c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db752c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c0ba400 comp rx=0 tx=0).stop 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 -- 172.21.15.105:0/2943499350 shutdown_connections 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x563e9dbe3820 0x563e9dbe5c10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563e9db78490 0x563e9db767c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e9db6c590 0x563e9db75800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 --2- 172.21.15.105:0/2943499350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563e9db66a30 0x563e9db752c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f38fb7fe640 0 client.32338 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 -- 172.21.15.105:0/2943499350 >> 172.21.15.105:0/2943499350 conn(0x563e9db61fa0 msgr2=0x563e9db68920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 -- 172.21.15.105:0/2943499350 shutdown_connections 2024-03-26T17:58:18.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 1 -- 172.21.15.105:0/2943499350 wait complete. 2024-03-26T17:58:18.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.520+0000 7f392545b740 20 client.32338 trim_cache size 0 max 16384 2024-03-26T17:58:18.511 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 -- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39040a2f20 msgr2=0x7f39040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39040a2f20 0x7f39040a53e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x563e9db67c80 tx=0x7f3914004a40 comp rx=0 tx=0).stop 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 -- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c1007c0 msgr2=0x7f391c199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c1007c0 0x7f391c199d30 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f390c00d950 tx=0x7f390c00de10 comp rx=0 tx=0).stop 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 -- 172.21.15.105:0/3497840111 shutdown_connections 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f39040a2f20 0x7f39040a53e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f391c102370 0x7f391c1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f391c1019b0 0x7f391c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 --2- 172.21.15.105:0/3497840111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f391c1007c0 0x7f391c199d30 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 -- 172.21.15.105:0/3497840111 >> 172.21.15.105:0/3497840111 conn(0x7f391c0fbd50 msgr2=0x7f391c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 -- 172.21.15.105:0/3497840111 shutdown_connections 2024-03-26T17:58:18.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.521+0000 7f3922875640 1 -- 172.21.15.105:0/3497840111 wait complete. 2024-03-26T17:58:18.519 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:58:18.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 55267 7.08 MB/sec execute 301 sec latency 140.202 ms 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.666+0000 7f502cee8640 1 -- 172.21.15.105:0/1698989458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50281019b0 msgr2=0x7f5028101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.666+0000 7f502cee8640 1 --2- 172.21.15.105:0/1698989458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50281019b0 0x7f5028101e30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f501c009900 tx=0x7f501c02bc40 comp rx=0 tx=0).stop 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.666+0000 7f502cee8640 1 -- 172.21.15.105:0/1698989458 shutdown_connections 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.666+0000 7f502cee8640 1 --2- 172.21.15.105:0/1698989458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5028102370 0x7f5028108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.666+0000 7f502cee8640 1 --2- 172.21.15.105:0/1698989458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f50281019b0 0x7f5028101e30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.666+0000 7f502cee8640 1 --2- 172.21.15.105:0/1698989458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50281007c0 0x7f5028100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.666+0000 7f502cee8640 1 -- 172.21.15.105:0/1698989458 >> 172.21.15.105:0/1698989458 conn(0x7f50280fbd50 msgr2=0x7f50280fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.666+0000 7f502cee8640 1 -- 172.21.15.105:0/1698989458 shutdown_connections 2024-03-26T17:58:18.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.667+0000 7f502cee8640 1 -- 172.21.15.105:0/1698989458 wait complete. 2024-03-26T17:58:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.667+0000 7f502cee8640 1 Processor -- start 2024-03-26T17:58:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.667+0000 7f502cee8640 1 -- start start 2024-03-26T17:58:18.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502cee8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50281007c0 0x7f5028197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502cee8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50281019b0 0x7f5028198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502cee8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5028102370 0x7f502819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502cee8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f502810d470 con 0x7f50281007c0 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502cee8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f502810d2f0 con 0x7f5028102370 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502cee8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f502810d5f0 con 0x7f50281019b0 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50281007c0 0x7f5028197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50281007c0 0x7f5028197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35402/0 (socket says 172.21.15.105:35402) 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502659c640 1 -- 172.21.15.105:0/1039328486 learned_addr learned my addr 172.21.15.105:0/1039328486 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f5025d9b640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50281019b0 0x7f5028198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f5026d9d640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5028102370 0x7f502819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f5025d9b640 1 -- 172.21.15.105:0/1039328486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5028102370 msgr2=0x7f502819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f5025d9b640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5028102370 0x7f502819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f5025d9b640 1 -- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50281007c0 msgr2=0x7f5028197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f5025d9b640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50281007c0 0x7f5028197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f5025d9b640 1 -- 172.21.15.105:0/1039328486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502819f800 con 0x7f50281019b0 2024-03-26T17:58:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f502659c640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50281007c0 0x7f5028197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.668+0000 7f5025d9b640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50281019b0 0x7f5028198120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f501c000c00 tx=0x7f501c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.669+0000 7f50177fe640 1 -- 172.21.15.105:0/1039328486 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c043020 con 0x7f50281019b0 2024-03-26T17:58:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.669+0000 7f50177fe640 1 -- 172.21.15.105:0/1039328486 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f501c004440 con 0x7f50281019b0 2024-03-26T17:58:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.669+0000 7f502cee8640 1 -- 172.21.15.105:0/1039328486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50281095c0 con 0x7f50281019b0 2024-03-26T17:58:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.669+0000 7f50177fe640 1 -- 172.21.15.105:0/1039328486 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c003c60 con 0x7f50281019b0 2024-03-26T17:58:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.669+0000 7f502cee8640 1 -- 172.21.15.105:0/1039328486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5028109be0 con 0x7f50281019b0 2024-03-26T17:58:18.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.671+0000 7f50177fe640 1 -- 172.21.15.105:0/1039328486 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f501c046020 con 0x7f50281019b0 2024-03-26T17:58:18.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.672+0000 7f50177fe640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50100a2fa0 0x7f50100a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.672+0000 7f502659c640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50100a2fa0 0x7f50100a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.672+0000 7f502fbaa740 1 Processor -- start 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.672+0000 7f50177fe640 1 -- 172.21.15.105:0/1039328486 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f501c0f14a0 con 0x7f50281019b0 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.672+0000 7f502fbaa740 1 -- start start 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f8a30 0x55d99c4f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c4fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4fe590 0x55d99c500a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f502fbaa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d99c45d970 con 0x55d99c4f8a30 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f502fbaa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d99c405d70 con 0x55d99c4f9420 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f502fbaa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d99c459920 con 0x55d99c4fe590 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4fe590 0x55d99c500a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5025d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c4fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4fe590 0x55d99c500a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54838/0 (socket says 172.21.15.105:54838) 2024-03-26T17:58:18.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 -- 172.21.15.105:0/2648743022 learned_addr learned my addr 172.21.15.105:0/2648743022 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f502659c640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50100a2fa0 0x7f50100a5460 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f50180058c0 tx=0x7f5018005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f502659c640 1 --2- 172.21.15.105:0/2648743022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f8a30 0x55d99c4f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 -- 172.21.15.105:0/2648743022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 msgr2=0x55d99c4fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 --2- 172.21.15.105:0/2648743022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c4fdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 -- 172.21.15.105:0/2648743022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f8a30 msgr2=0x55d99c4f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 --2- 172.21.15.105:0/2648743022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f8a30 0x55d99c4f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 -- 172.21.15.105:0/2648743022 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99c1713f0 con 0x55d99c4fe590 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.673+0000 7f5026d9d640 1 --2- 172.21.15.105:0/2648743022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4fe590 0x55d99c500a20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5020011150 tx=0x7f5020011610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7a7e7b94252575 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.674+0000 7f5014ff9640 1 -- 172.21.15.105:0/2648743022 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50200138e0 con 0x55d99c4fe590 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.674+0000 7f5014ff9640 1 -- 172.21.15.105:0/2648743022 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5020022070 con 0x55d99c4fe590 2024-03-26T17:58:18.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.674+0000 7f5014ff9640 1 -- 172.21.15.105:0/2648743022 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502001b420 con 0x55d99c4fe590 2024-03-26T17:58:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 -- 172.21.15.105:0/2648743022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4fe590 msgr2=0x55d99c500a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2648743022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4fe590 0x55d99c500a20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5020011150 tx=0x7f5020011610 comp rx=0 tx=0).stop 2024-03-26T17:58:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 -- 172.21.15.105:0/2648743022 shutdown_connections 2024-03-26T17:58:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2648743022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4fe590 0x55d99c500a20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2648743022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c4fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2648743022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f8a30 0x55d99c4f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 -- 172.21.15.105:0/2648743022 >> 172.21.15.105:0/2648743022 conn(0x55d99c4f3fa0 msgr2=0x55d99c4f43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 -- 172.21.15.105:0/2648743022 shutdown_connections 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 -- 172.21.15.105:0/2648743022 wait complete. 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 -- start start 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 Processor -- start 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.675+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502fbaa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d99c501540 con 0x55d99c4f9420 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502fbaa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d99c419310 con 0x55d99c4fe590 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502fbaa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d99c5016c0 con 0x55d99c4f8a30 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502fbaa740 10 client.0 _ll_register_callbacks cb 0x55d99c454c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35412/0 (socket says 172.21.15.105:35412) 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 -- 172.21.15.105:0/2682415690 learned_addr learned my addr 172.21.15.105:0/2682415690 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5026d9d640 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 -- 172.21.15.105:0/2682415690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 msgr2=0x55d99c502ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 -- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 msgr2=0x55d99c50a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 -- 172.21.15.105:0/2682415690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99c23b7d0 con 0x55d99c4f9420 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502659c640 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5026d9d640 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f5025d9b640 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f501c0eefb0 tx=0x7f501c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b799e0bcd2ae05 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f500dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f500dffb640 1 -- 172.21.15.105:0/2682415690 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0bdb90 con 0x55d99c4f9420 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f500dffb640 1 -- 172.21.15.105:0/2682415690 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f501c0f1110 con 0x55d99c4f9420 2024-03-26T17:58:18.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502fbaa740 1 -- 172.21.15.105:0/2682415690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5020022070 con 0x55d99c4f9420 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f500dffb640 1 -- 172.21.15.105:0/2682415690 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f501c0c1ce0 con 0x55d99c4f9420 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502fbaa740 10 client.22677 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.676+0000 7f502fbaa740 1 -- 172.21.15.105:0/2682415690 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d99c50d7e0 con 0x55d99c4f9420 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f500dffb640 1 -- 172.21.15.105:0/2682415690 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5020022070 con 0x55d99c4f9420 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f500dffb640 1 -- 172.21.15.105:0/2682415690 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f501c0b98e0 con 0x55d99c4f9420 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f502fbaa740 10 client.22677 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f502fbaa740 10 client.22677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f502fbaa740 20 client.22677 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d99c50ee90 0x55d99c511320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f502fbaa740 4 client.22677 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f502fbaa740 1 -- 172.21.15.105:0/2682415690 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d99c46aeb0 con 0x55d99c50ee90 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f502659c640 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d99c50ee90 0x55d99c511320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f500dffb640 10 client.22677 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:18.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.677+0000 7f502659c640 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d99c50ee90 0x55d99c511320 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a3c63e8dc966f19 server_cookie=e303b0588251fe14 in_seq=0 out_seq=0 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.679+0000 7f500dffb640 1 -- 172.21.15.105:0/2682415690 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d99c46aeb0 con 0x55d99c50ee90 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.679+0000 7f500dffb640 10 client.22677 handle_command_reply: tid=0 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.679+0000 7f502fbaa740 1 client.22677 shutdown 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.679+0000 7f502fbaa740 1 -- 172.21.15.105:0/2682415690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 msgr2=0x55d99c503430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.679+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f501c0eefb0 tx=0x7f501c0035f0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.679+0000 7f502fbaa740 1 -- 172.21.15.105:0/2682415690 shutdown_connections 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.679+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d99c50ee90 0x55d99c511320 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.680+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.680+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.680+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2682415690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.680+0000 7f502fbaa740 1 -- 172.21.15.105:0/2682415690 >> 172.21.15.105:0/2682415690 conn(0x55d99c4f3fa0 msgr2=0x55d99c4f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.680+0000 7f500dffb640 0 client.22677 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.680+0000 7f502fbaa740 1 -- 172.21.15.105:0/2682415690 shutdown_connections 2024-03-26T17:58:18.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.680+0000 7f502fbaa740 1 -- 172.21.15.105:0/2682415690 wait complete. 2024-03-26T17:58:18.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.680+0000 7f502fbaa740 20 client.22677 trim_cache size 0 max 16384 2024-03-26T17:58:18.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502fbaa740 1 Processor -- start 2024-03-26T17:58:18.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502fbaa740 1 -- start start 2024-03-26T17:58:18.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502fbaa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a6c90 con 0x55d99c4f9420 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502fbaa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a87d0 con 0x55d99c4fe590 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502fbaa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a7360 con 0x55d99c4f8a30 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f5025d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f5025d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35426/0 (socket says 172.21.15.105:35426) 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f5025d9b640 1 -- 172.21.15.105:0/2637285842 learned_addr learned my addr 172.21.15.105:0/2637285842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.701+0000 7f502659c640 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f5026d9d640 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f5025d9b640 1 -- 172.21.15.105:0/2637285842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 msgr2=0x55d99c502ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f5025d9b640 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f5025d9b640 1 -- 172.21.15.105:0/2637285842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 msgr2=0x55d99c50a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f5025d9b640 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f5025d9b640 1 -- 172.21.15.105:0/2637285842 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99c4f9800 con 0x55d99c4f9420 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f5025d9b640 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f501c0bdb20 tx=0x7f501c0bda90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0738bcb01f3020c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f500ffff640 1 -- 172.21.15.105:0/2637285842 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0c65a0 con 0x55d99c4f9420 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f500ffff640 1 -- 172.21.15.105:0/2637285842 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f501c003e00 con 0x55d99c4f9420 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.702+0000 7f500ffff640 1 -- 172.21.15.105:0/2637285842 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0c6110 con 0x55d99c4f9420 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 -- 172.21.15.105:0/2637285842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 msgr2=0x55d99c503430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f501c0bdb20 tx=0x7f501c0bda90 comp rx=0 tx=0).stop 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 -- 172.21.15.105:0/2637285842 shutdown_connections 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4fe590 0x55d99c50a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4f9420 0x55d99c503430 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 --2- 172.21.15.105:0/2637285842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c502ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 -- 172.21.15.105:0/2637285842 >> 172.21.15.105:0/2637285842 conn(0x55d99c4f3fa0 msgr2=0x55d99c510ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 -- 172.21.15.105:0/2637285842 shutdown_connections 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.703+0000 7f502fbaa740 1 -- 172.21.15.105:0/2637285842 wait complete. 2024-03-26T17:58:18.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 1 -- start start 2024-03-26T17:58:18.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 1 Processor -- start 2024-03-26T17:58:18.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c507460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c5079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4fe590 0x55d99c573250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f5026d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4fe590 0x55d99c573250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c507460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f5026d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4fe590 0x55d99c573250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35442/0 (socket says 172.21.15.105:35442) 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c507460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54854/0 (socket says 172.21.15.105:54854) 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f5025d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c5079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f5026d9d640 1 -- 172.21.15.105:0/962345393 learned_addr learned my addr 172.21.15.105:0/962345393 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a8100 con 0x55d99c4fe590 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a7a30 con 0x55d99c4f9420 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a4c70 con 0x55d99c4f8a30 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.704+0000 7f502fbaa740 10 client.0 _ll_register_callbacks cb 0x7f501c0bd960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f5025d9b640 1 -- 172.21.15.105:0/962345393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 msgr2=0x55d99c507460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f5025d9b640 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c507460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f5025d9b640 1 -- 172.21.15.105:0/962345393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4fe590 msgr2=0x55d99c573250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f5025d9b640 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4fe590 0x55d99c573250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f5025d9b640 1 -- 172.21.15.105:0/962345393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99c573790 con 0x55d99c4f9420 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f502659c640 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c507460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f5025d9b640 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c5079a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f501c0b9fa0 tx=0x7f501c0b9bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c55d1381ade642de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f500dffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f500dffb640 1 -- 172.21.15.105:0/962345393 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0a7020 con 0x55d99c4f9420 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f501c003e00 con 0x55d99c4f9420 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f500dffb640 1 -- 172.21.15.105:0/962345393 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f501c0f1210 con 0x55d99c4f9420 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f500dffb640 1 -- 172.21.15.105:0/962345393 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f501c0c65a0 con 0x55d99c4f9420 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f502fbaa740 10 client.32369 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.705+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d99c573790 con 0x55d99c4f9420 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f500dffb640 1 -- 172.21.15.105:0/962345393 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f501c003e00 con 0x55d99c4f9420 2024-03-26T17:58:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f500dffb640 1 -- 172.21.15.105:0/962345393 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f501c0a3020 con 0x55d99c4f9420 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f502fbaa740 10 client.32369 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f502fbaa740 10 client.32369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f502fbaa740 20 client.32369 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f502fbaa740 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d99c574f80 0x55d99c5773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f502fbaa740 4 client.32369 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d99c465200 con 0x55d99c574f80 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f502659c640 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d99c574f80 0x55d99c5773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f500dffb640 10 client.32369 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.706+0000 7f502659c640 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d99c574f80 0x55d99c5773a0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a4e8af92be5a23 server_cookie=5de2e44b3e6a073a in_seq=0 out_seq=0 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f500dffb640 1 -- 172.21.15.105:0/962345393 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d99c465200 con 0x55d99c574f80 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f500dffb640 10 client.32369 handle_command_reply: tid=0 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 client.32369 shutdown 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 msgr2=0x55d99c5079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c5079a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f501c0b9fa0 tx=0x7f501c0b9bd0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 shutdown_connections 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d99c574f80 0x55d99c5773a0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d99c4fe590 0x55d99c573250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f500dffb640 0 client.32369 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d99c4f9420 0x55d99c5079a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 --2- 172.21.15.105:0/962345393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d99c4f8a30 0x55d99c507460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 >> 172.21.15.105:0/962345393 conn(0x55d99c4f3fa0 msgr2=0x55d99c4fa920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 shutdown_connections 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.708+0000 7f502fbaa740 1 -- 172.21.15.105:0/962345393 wait complete. 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.709+0000 7f502fbaa740 20 client.32369 trim_cache size 0 max 16384 2024-03-26T17:58:18.701 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.709+0000 7f502cee8640 1 -- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50100a2fa0 msgr2=0x7f50100a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.709+0000 7f502cee8640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50100a2fa0 0x7f50100a5460 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f50180058c0 tx=0x7f5018005f90 comp rx=0 tx=0).stop 2024-03-26T17:58:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 -- 172.21.15.105:0/1039328486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50281019b0 msgr2=0x7f5028198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50281019b0 0x7f5028198120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f501c000c00 tx=0x7f501c004040 comp rx=0 tx=0).stop 2024-03-26T17:58:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 -- 172.21.15.105:0/1039328486 shutdown_connections 2024-03-26T17:58:18.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f50100a2fa0 0x7f50100a5460 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5028102370 0x7f502819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50281019b0 0x7f5028198120 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 --2- 172.21.15.105:0/1039328486 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50281007c0 0x7f5028197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 -- 172.21.15.105:0/1039328486 >> 172.21.15.105:0/1039328486 conn(0x7f50280fbd50 msgr2=0x7f5028104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 -- 172.21.15.105:0/1039328486 shutdown_connections 2024-03-26T17:58:18.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:18.710+0000 7f502cee8640 1 -- 172.21.15.105:0/1039328486 wait complete. 2024-03-26T17:58:19.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:18 smithi144 ceph-mon[36095]: pgmap v949: 97 pgs: 97 active+clean; 332 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-03-26T17:58:19.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:18 smithi105 ceph-mon[32280]: pgmap v949: 97 pgs: 97 active+clean; 332 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-03-26T17:58:19.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:18 smithi135 ceph-mon[35636]: pgmap v949: 97 pgs: 97 active+clean; 332 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 4.0 MiB/s wr, 205 op/s 2024-03-26T17:58:19.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 55372 7.07 MB/sec execute 302 sec latency 174.195 ms 2024-03-26T17:58:19.711 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:58:19.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:58:19.712 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:58:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 -- 172.21.15.105:0/772879053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f01075a0 msgr2=0x7f49f01099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 --2- 172.21.15.105:0/772879053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f01075a0 0x7f49f01099f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009be0 tx=0x7f49e802bef0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 -- 172.21.15.105:0/772879053 shutdown_connections 2024-03-26T17:58:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 --2- 172.21.15.105:0/772879053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f01075a0 0x7f49f01099f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 --2- 172.21.15.105:0/772879053 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49f0104c70 0x7f49f0107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 --2- 172.21.15.105:0/772879053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49f0102340 0x7f49f0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 -- 172.21.15.105:0/772879053 >> 172.21.15.105:0/772879053 conn(0x7f49f00fbd50 msgr2=0x7f49f00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 -- 172.21.15.105:0/772879053 shutdown_connections 2024-03-26T17:58:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.883+0000 7f49f5722640 1 -- 172.21.15.105:0/772879053 wait complete. 2024-03-26T17:58:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.884+0000 7f49f5722640 1 Processor -- start 2024-03-26T17:58:19.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.885+0000 7f49f5722640 1 -- start start 2024-03-26T17:58:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.885+0000 7f49f5722640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f0102340 0x7f49f0199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.885+0000 7f49f5722640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49f0104c70 0x7f49f019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.885+0000 7f49f5722640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49f01075a0 0x7f49f01a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.885+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f0102340 0x7f49f0199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.885+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f0102340 0x7f49f0199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54874/0 (socket says 172.21.15.105:54874) 2024-03-26T17:58:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.885+0000 7f49ef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49f01075a0 0x7f49f01a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.885+0000 7f49ef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49f01075a0 0x7f49f01a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35474/0 (socket says 172.21.15.105:35474) 2024-03-26T17:58:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.886+0000 7f49ee7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49f0104c70 0x7f49f019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.886+0000 7f49ee7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49f0104c70 0x7f49f019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40450/0 (socket says 172.21.15.105:40450) 2024-03-26T17:58:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.886+0000 7f49ee7fc640 1 -- 172.21.15.105:0/2848047007 learned_addr learned my addr 172.21.15.105:0/2848047007 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.886+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0101d40 con 0x7f49f01075a0 2024-03-26T17:58:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.886+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0101bc0 con 0x7f49f0104c70 2024-03-26T17:58:19.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.886+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49f0101ec0 con 0x7f49f0102340 2024-03-26T17:58:19.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49ee7fc640 1 -- 172.21.15.105:0/2848047007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f0102340 msgr2=0x7f49f0199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f0102340 0x7f49f0199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49ee7fc640 1 -- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49f01075a0 msgr2=0x7f49f01a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49f01075a0 0x7f49f01a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49ee7fc640 1 -- 172.21.15.105:0/2848047007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f01a1910 con 0x7f49f0104c70 2024-03-26T17:58:19.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49f01075a0 0x7f49f01a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:19.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49f0104c70 0x7f49f019a240 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9c0 tx=0x7f49e400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:19.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49d7fff640 1 -- 172.21.15.105:0/2848047007 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400cd80 con 0x7f49f0104c70 2024-03-26T17:58:19.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f01a1c00 con 0x7f49f0104c70 2024-03-26T17:58:19.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49d7fff640 1 -- 172.21.15.105:0/2848047007 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49e40054d0 con 0x7f49f0104c70 2024-03-26T17:58:19.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.887+0000 7f49d7fff640 1 -- 172.21.15.105:0/2848047007 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4010430 con 0x7f49f0104c70 2024-03-26T17:58:19.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.888+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f01a20f0 con 0x7f49f0104c70 2024-03-26T17:58:19.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49d7fff640 1 -- 172.21.15.105:0/2848047007 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f49e4004fd0 con 0x7f49f0104c70 2024-03-26T17:58:19.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49f83ed740 1 Processor -- start 2024-03-26T17:58:19.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49f83ed740 1 -- start start 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3ba1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3ba6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3ba9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49f83ed740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fad3afe630 con 0x55fad3ba19a0 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49f83ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fad3aa6e10 con 0x55fad3ba2390 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.891+0000 7f49f83ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fad3afa5e0 con 0x55fad3ba7500 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49d7fff640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49d80a3020 0x7f49d80a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3ba6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3ba6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40452/0 (socket says 172.21.15.105:40452) 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49eeffd640 1 -- 172.21.15.105:0/4245078344 learned_addr learned my addr 172.21.15.105:0/4245078344 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3ba1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49d7fff640 1 -- 172.21.15.105:0/2848047007 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f49e40cf890 con 0x7f49f0104c70 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3ba9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49eeffd640 1 -- 172.21.15.105:0/4245078344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 msgr2=0x55fad3ba9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49eeffd640 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3ba9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49eeffd640 1 -- 172.21.15.105:0/4245078344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 msgr2=0x55fad3ba1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49eeffd640 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3ba1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49eeffd640 1 -- 172.21.15.105:0/4245078344 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad3abd740 con 0x55fad3ba2390 2024-03-26T17:58:19.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.892+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3ba1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.893+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49d80a3020 0x7f49d80a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.893+0000 7f49eeffd640 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3ba6e90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009c80 tx=0x7f49e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc3f133df117bc54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.893+0000 7f49d57fa640 1 -- 172.21.15.105:0/4245078344 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0004220 con 0x55fad3ba2390 2024-03-26T17:58:19.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.893+0000 7f49d57fa640 1 -- 172.21.15.105:0/4245078344 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49e00043c0 con 0x55fad3ba2390 2024-03-26T17:58:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.893+0000 7f49d57fa640 1 -- 172.21.15.105:0/4245078344 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0019790 con 0x55fad3ba2390 2024-03-26T17:58:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.894+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49d80a3020 0x7f49d80a54e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f49e8034fd0 tx=0x7f49e8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.894+0000 7f49f83ed740 1 -- 172.21.15.105:0/4245078344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 msgr2=0x55fad3ba6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.894+0000 7f49f83ed740 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3ba6e90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009c80 tx=0x7f49e0011810 comp rx=0 tx=0).stop 2024-03-26T17:58:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 -- 172.21.15.105:0/4245078344 shutdown_connections 2024-03-26T17:58:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3ba9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3ba6e90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 --2- 172.21.15.105:0/4245078344 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3ba1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 -- 172.21.15.105:0/4245078344 >> 172.21.15.105:0/4245078344 conn(0x55fad3b9cf10 msgr2=0x55fad3b9d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:19.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 -- 172.21.15.105:0/4245078344 shutdown_connections 2024-03-26T17:58:19.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 -- 172.21.15.105:0/4245078344 wait complete. 2024-03-26T17:58:19.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 -- start start 2024-03-26T17:58:19.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.895+0000 7f49f83ed740 1 Processor -- start 2024-03-26T17:58:19.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba19a0 0x55fad3babef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3bac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba7500 0x55fad3bb3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba19a0 0x55fad3babef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba19a0 0x55fad3babef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54886/0 (socket says 172.21.15.105:54886) 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49ef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba7500 0x55fad3bb3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49ef7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba7500 0x55fad3bb3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35494/0 (socket says 172.21.15.105:35494) 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49ee7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3bac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49ee7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3bac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40462/0 (socket says 172.21.15.105:40462) 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49eeffd640 1 -- 172.21.15.105:0/3865550485 learned_addr learned my addr 172.21.15.105:0/3865550485 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fad3baa4b0 con 0x55fad3ba7500 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fad3aba1a0 con 0x55fad3ba2390 2024-03-26T17:58:19.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.896+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fad3baa630 con 0x55fad3ba19a0 2024-03-26T17:58:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49eeffd640 1 -- 172.21.15.105:0/3865550485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 msgr2=0x55fad3bac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49eeffd640 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3bac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49eeffd640 1 -- 172.21.15.105:0/3865550485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba7500 msgr2=0x55fad3bb3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49eeffd640 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba7500 0x55fad3bb3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49eeffd640 1 -- 172.21.15.105:0/3865550485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad38da7e0 con 0x55fad3ba19a0 2024-03-26T17:58:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba7500 0x55fad3bb3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3bac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:58:19.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49f83ed740 10 client.0 _ll_register_callbacks cb 0x55fad3811c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e00043c0 con 0x55fad3ba19a0 2024-03-26T17:58:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49eeffd640 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba19a0 0x55fad3babef0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009820 tx=0x7f49e0005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf79b1f19f3f178d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49d2ff5640 10 client.32368 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:58:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49d2ff5640 1 -- 172.21.15.105:0/3865550485 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0032050 con 0x55fad3ba19a0 2024-03-26T17:58:19.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.897+0000 7f49d2ff5640 1 -- 172.21.15.105:0/3865550485 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f49e00035f0 con 0x55fad3ba19a0 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49d2ff5640 1 -- 172.21.15.105:0/3865550485 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f49e0028070 con 0x55fad3ba19a0 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49f83ed740 10 client.32368 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55fad393af70 con 0x55fad3ba19a0 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49d2ff5640 1 -- 172.21.15.105:0/3865550485 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f49e0035020 con 0x55fad3ba19a0 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49f83ed740 10 client.32368 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49f83ed740 10 client.32368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49f83ed740 20 client.32368 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49f83ed740 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fad3bb7860 0x55fad3bb9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49f83ed740 4 client.32368 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad3b0bb70 con 0x55fad3bb7860 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.898+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fad3bb7860 0x55fad3bb9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.899+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fad3bb7860 0x55fad3bb9cd0 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c8c3f0564cc63cd server_cookie=110efc1619547382 in_seq=0 out_seq=0 2024-03-26T17:58:19.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.899+0000 7f49d2ff5640 10 client.32368 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.901+0000 7f49d2ff5640 1 -- 172.21.15.105:0/3865550485 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fad3b0bb70 con 0x55fad3bb7860 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.901+0000 7f49d2ff5640 10 client.32368 handle_command_reply: tid=0 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.901+0000 7f49f83ed740 1 client.32368 shutdown 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba19a0 msgr2=0x55fad3babef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba19a0 0x55fad3babef0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009820 tx=0x7f49e0005270 comp rx=0 tx=0).stop 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 shutdown_connections 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fad3bb7860 0x55fad3bb9cd0 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba7500 0x55fad3bb3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba2390 0x55fad3bac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 --2- 172.21.15.105:0/3865550485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba19a0 0x55fad3babef0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 >> 172.21.15.105:0/3865550485 conn(0x55fad3b9cf10 msgr2=0x55fad3b93900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:19.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49d2ff5640 0 client.32368 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:19.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 shutdown_connections 2024-03-26T17:58:19.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 1 -- 172.21.15.105:0/3865550485 wait complete. 2024-03-26T17:58:19.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.902+0000 7f49f83ed740 20 client.32368 trim_cache size 0 max 16384 2024-03-26T17:58:19.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49f83ed740 1 Processor -- start 2024-03-26T17:58:19.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49f83ed740 1 -- start start 2024-03-26T17:58:19.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3bab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba2390 0x55fad3bb33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba7500 0x55fad3baba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49f83ed740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49f01a6bf0 con 0x55fad3ba19a0 2024-03-26T17:58:19.918 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49f83ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49f01a8730 con 0x55fad3ba7500 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49f83ed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49f01a72c0 con 0x55fad3ba2390 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3bab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3bab510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35510/0 (socket says 172.21.15.105:35510) 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49eeffd640 1 -- 172.21.15.105:0/236827216 learned_addr learned my addr 172.21.15.105:0/236827216 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba2390 0x55fad3bb33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.932+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba7500 0x55fad3baba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49eeffd640 1 -- 172.21.15.105:0/236827216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba2390 msgr2=0x55fad3bb33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49eeffd640 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba2390 0x55fad3bb33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49eeffd640 1 -- 172.21.15.105:0/236827216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba7500 msgr2=0x55fad3baba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49eeffd640 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba7500 0x55fad3baba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49eeffd640 1 -- 172.21.15.105:0/236827216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad37d2170 con 0x55fad3ba19a0 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49eeffd640 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3bab510 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f49e0004610 tx=0x7f49e0025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364f6417532bb04d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49d4ff9640 1 -- 172.21.15.105:0/236827216 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0032070 con 0x55fad3ba19a0 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49d4ff9640 1 -- 172.21.15.105:0/236827216 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49e00043c0 con 0x55fad3ba19a0 2024-03-26T17:58:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.933+0000 7f49d4ff9640 1 -- 172.21.15.105:0/236827216 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0004d40 con 0x55fad3ba19a0 2024-03-26T17:58:19.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 -- 172.21.15.105:0/236827216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 msgr2=0x55fad3bab510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3bab510 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f49e0004610 tx=0x7f49e0025000 comp rx=0 tx=0).stop 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 -- 172.21.15.105:0/236827216 shutdown_connections 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba7500 0x55fad3baba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba2390 0x55fad3bb33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 --2- 172.21.15.105:0/236827216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba19a0 0x55fad3bab510 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 -- 172.21.15.105:0/236827216 >> 172.21.15.105:0/236827216 conn(0x55fad3b9cf10 msgr2=0x55fad3ba0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 -- 172.21.15.105:0/236827216 shutdown_connections 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.934+0000 7f49f83ed740 1 -- 172.21.15.105:0/236827216 wait complete. 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49f83ed740 1 -- start start 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49f83ed740 1 Processor -- start 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba19a0 0x55fad3bb0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49f83ed740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba2390 0x55fad3bb11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba19a0 0x55fad3bb0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba19a0 0x55fad3bb0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40480/0 (socket says 172.21.15.105:40480) 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49eeffd640 1 -- 172.21.15.105:0/538249883 learned_addr learned my addr 172.21.15.105:0/538249883 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49f83ed740 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3bb21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.935+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba2390 0x55fad3bb11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49f01a8060 con 0x55fad3ba2390 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49f01a7990 con 0x55fad3ba19a0 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3bb21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49f01a4bd0 con 0x55fad3ba7500 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49f83ed740 10 client.0 _ll_register_callbacks cb 0x7f49e0009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49eeffd640 1 -- 172.21.15.105:0/538249883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 msgr2=0x55fad3bb21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49eeffd640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3bb21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49eeffd640 1 -- 172.21.15.105:0/538249883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba2390 msgr2=0x55fad3bb11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49eeffd640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba2390 0x55fad3bb11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49eeffd640 1 -- 172.21.15.105:0/538249883 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad3bb27d0 con 0x55fad3ba19a0 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49ef7fe640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3bb21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e00043c0 con 0x55fad3ba19a0 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba2390 0x55fad3bb11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49eeffd640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba19a0 0x55fad3bb0c90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f49e0005ec0 tx=0x7f49e0009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bba56915e2be5d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.936+0000 7f49d2ff5640 10 client.32384 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49d2ff5640 1 -- 172.21.15.105:0/538249883 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0032050 con 0x55fad3ba19a0 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49d2ff5640 1 -- 172.21.15.105:0/538249883 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f49e0003920 con 0x55fad3ba19a0 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49d2ff5640 1 -- 172.21.15.105:0/538249883 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f49e0003b60 con 0x55fad3ba19a0 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49f83ed740 10 client.32384 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55fad3c1dcf0 con 0x55fad3ba19a0 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49d2ff5640 1 -- 172.21.15.105:0/538249883 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f49e0035040 con 0x55fad3ba19a0 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49f83ed740 10 client.32384 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49f83ed740 10 client.32384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49f83ed740 20 client.32384 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49f83ed740 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fad3c1e350 0x55fad3c20740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49f83ed740 4 client.32384 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:58:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.937+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fad3b05ec0 con 0x55fad3c1e350 2024-03-26T17:58:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.938+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fad3c1e350 0x55fad3c20740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.938+0000 7f49ee7fc640 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fad3c1e350 0x55fad3c20740 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34b66d4096d8e18 server_cookie=780c049559d39719 in_seq=0 out_seq=0 2024-03-26T17:58:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.938+0000 7f49d2ff5640 10 client.32384 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.941+0000 7f49d2ff5640 1 -- 172.21.15.105:0/538249883 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fad3b05ec0 con 0x55fad3c1e350 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.941+0000 7f49d2ff5640 10 client.32384 handle_command_reply: tid=0 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.941+0000 7f49f83ed740 1 client.32384 shutdown 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.941+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba19a0 msgr2=0x55fad3bb0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.941+0000 7f49f83ed740 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba19a0 0x55fad3bb0c90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f49e0005ec0 tx=0x7f49e0009870 comp rx=0 tx=0).stop 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 shutdown_connections 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49f83ed740 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fad3c1e350 0x55fad3c20740 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49f83ed740 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fad3ba7500 0x55fad3bb21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49f83ed740 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fad3ba2390 0x55fad3bb11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49d2ff5640 0 client.32384 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49f83ed740 1 --2- 172.21.15.105:0/538249883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fad3ba19a0 0x55fad3bb0c90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 >> 172.21.15.105:0/538249883 conn(0x55fad3b9cf10 msgr2=0x55fad3b93070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 shutdown_connections 2024-03-26T17:58:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.942+0000 7f49f83ed740 1 -- 172.21.15.105:0/538249883 wait complete. 2024-03-26T17:58:19.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f83ed740 20 client.32384 trim_cache size 0 max 16384 2024-03-26T17:58:19.930 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:58:19.930 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:58:19.930 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "45fe17c3-260a-49e6-afd8-cadadd423adb", 2024-03-26T17:58:19.930 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:58:19.930 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:58:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49d80a3020 msgr2=0x7f49d80a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49d80a3020 0x7f49d80a54e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f49e8034fd0 tx=0x7f49e8009500 comp rx=0 tx=0).stop 2024-03-26T17:58:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49f0104c70 msgr2=0x7f49f019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49f0104c70 0x7f49f019a240 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9c0 tx=0x7f49e400ee80 comp rx=0 tx=0).stop 2024-03-26T17:58:19.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 shutdown_connections 2024-03-26T17:58:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f49d80a3020 0x7f49d80a54e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49f01075a0 0x7f49f01a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49f0104c70 0x7f49f019a240 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 --2- 172.21.15.105:0/2848047007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f0102340 0x7f49f0199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 >> 172.21.15.105:0/2848047007 conn(0x7f49f00fbd50 msgr2=0x7f49f0102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.943+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 shutdown_connections 2024-03-26T17:58:19.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:19.944+0000 7f49f5722640 1 -- 172.21.15.105:0/2848047007 wait complete. 2024-03-26T17:58:19.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:58:19.942 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:58:20.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.100+0000 7f3d171f7640 1 -- 172.21.15.105:0/1989273997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d10104bb0 msgr2=0x7f3d10105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.100+0000 7f3d171f7640 1 --2- 172.21.15.105:0/1989273997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d10104bb0 0x7f3d10105030 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009900 tx=0x7f3d0002bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.100+0000 7f3d171f7640 1 -- 172.21.15.105:0/1989273997 shutdown_connections 2024-03-26T17:58:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.100+0000 7f3d171f7640 1 --2- 172.21.15.105:0/1989273997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d10105570 0x7f3d1010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.100+0000 7f3d171f7640 1 --2- 172.21.15.105:0/1989273997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d10104bb0 0x7f3d10105030 secure :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009900 tx=0x7f3d0002bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.100+0000 7f3d171f7640 1 --2- 172.21.15.105:0/1989273997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d101039c0 0x7f3d10103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.100+0000 7f3d171f7640 1 -- 172.21.15.105:0/1989273997 >> 172.21.15.105:0/1989273997 conn(0x7f3d100fef30 msgr2=0x7f3d10101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.100+0000 7f3d171f7640 1 -- 172.21.15.105:0/1989273997 shutdown_connections 2024-03-26T17:58:20.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.101+0000 7f3d171f7640 1 -- 172.21.15.105:0/1989273997 wait complete. 2024-03-26T17:58:20.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.101+0000 7f3d171f7640 1 Processor -- start 2024-03-26T17:58:20.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d171f7640 1 -- start start 2024-03-26T17:58:20.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d171f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d101039c0 0x7f3d10199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d171f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d10105570 0x7f3d1019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d171f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d1019f350 0x7f3d101a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d171f7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010f520 con 0x7f3d1019f350 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d171f7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010f3a0 con 0x7f3d101039c0 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d161f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d101039c0 0x7f3d10199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d161f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d101039c0 0x7f3d10199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40494/0 (socket says 172.21.15.105:40494) 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d161f5640 1 -- 172.21.15.105:0/269998867 learned_addr learned my addr 172.21.15.105:0/269998867 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d169f6640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d1019f350 0x7f3d101a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1010f6a0 con 0x7f3d10105570 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.102+0000 7f3d159f4640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d10105570 0x7f3d1019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d161f5640 1 -- 172.21.15.105:0/269998867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d10105570 msgr2=0x7f3d1019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d161f5640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d10105570 0x7f3d1019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d161f5640 1 -- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d1019f350 msgr2=0x7f3d101a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d161f5640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d1019f350 0x7f3d101a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d161f5640 1 -- 172.21.15.105:0/269998867 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d101a1db0 con 0x7f3d101039c0 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d159f4640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d10105570 0x7f3d1019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d161f5640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d101039c0 0x7f3d10199db0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00e9c0 tx=0x7f3d0c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d077fe640 1 -- 172.21.15.105:0/269998867 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00cd80 con 0x7f3d101039c0 2024-03-26T17:58:20.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d101a20a0 con 0x7f3d101039c0 2024-03-26T17:58:20.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.103+0000 7f3d077fe640 1 -- 172.21.15.105:0/269998867 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d0c0035f0 con 0x7f3d101039c0 2024-03-26T17:58:20.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.104+0000 7f3d077fe640 1 -- 172.21.15.105:0/269998867 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c010690 con 0x7f3d101039c0 2024-03-26T17:58:20.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.104+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d101a2610 con 0x7f3d101039c0 2024-03-26T17:58:20.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.106+0000 7f3d077fe640 1 -- 172.21.15.105:0/269998867 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3d0c0037a0 con 0x7f3d101039c0 2024-03-26T17:58:20.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.107+0000 7f3d077fe640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3cfc0a2fa0 0x7f3cfc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.107+0000 7f3d077fe640 1 -- 172.21.15.105:0/269998867 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f3d0c0cf7d0 con 0x7f3d101039c0 2024-03-26T17:58:20.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.107+0000 7f3d159f4640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3cfc0a2fa0 0x7f3cfc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.107+0000 7f3d159f4640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3cfc0a2fa0 0x7f3cfc0a5460 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3d00036040 tx=0x7f3d0002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d1b040740 1 Processor -- start 2024-03-26T17:58:20.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d1b040740 1 -- start start 2024-03-26T17:58:20.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0018352b0 0x55c0018356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c001835ca0 0x55c00183a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c00183d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d159f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c001835ca0 0x55c00183a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d161f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0018352b0 0x55c0018356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d1b040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c001791980 con 0x55c00183ae10 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d1b040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c001739be0 con 0x55c0018352b0 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d159f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c001835ca0 0x55c00183a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54932/0 (socket says 172.21.15.105:54932) 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d161f5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0018352b0 0x55c0018356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40498/0 (socket says 172.21.15.105:40498) 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d159f4640 1 -- 172.21.15.105:0/2599775153 learned_addr learned my addr 172.21.15.105:0/2599775153 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d169f6640 1 --2- 172.21.15.105:0/2599775153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c00183d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.112+0000 7f3d1b040740 1 -- 172.21.15.105:0/2599775153 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c00178d930 con 0x55c001835ca0 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d161f5640 1 -- 172.21.15.105:0/2599775153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c001835ca0 msgr2=0x55c00183a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d161f5640 1 --2- 172.21.15.105:0/2599775153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c001835ca0 0x55c00183a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d161f5640 1 -- 172.21.15.105:0/2599775153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 msgr2=0x55c00183d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d161f5640 1 --2- 172.21.15.105:0/2599775153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c00183d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d161f5640 1 -- 172.21.15.105:0/2599775153 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c00156f7d0 con 0x55c0018352b0 2024-03-26T17:58:20.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d161f5640 1 --2- 172.21.15.105:0/2599775153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0018352b0 0x55c0018356d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0cd2e0 tx=0x7f3d0c09bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22d163843673f72b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d04bfe640 1 -- 172.21.15.105:0/2599775153 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c09bec0 con 0x55c0018352b0 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d04bfe640 1 -- 172.21.15.105:0/2599775153 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d0c0a0010 con 0x55c0018352b0 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.113+0000 7f3d04bfe640 1 -- 172.21.15.105:0/2599775153 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0a4440 con 0x55c0018352b0 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.114+0000 7f3d1b040740 1 -- 172.21.15.105:0/2599775153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0018352b0 msgr2=0x55c0018356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.114+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2599775153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0018352b0 0x55c0018356d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0cd2e0 tx=0x7f3d0c09bac0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.115+0000 7f3d1b040740 1 -- 172.21.15.105:0/2599775153 shutdown_connections 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.115+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2599775153 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c00183d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.115+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2599775153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c001835ca0 0x55c00183a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.115+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2599775153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0018352b0 0x55c0018356d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.115+0000 7f3d1b040740 1 -- 172.21.15.105:0/2599775153 >> 172.21.15.105:0/2599775153 conn(0x55c001830820 msgr2=0x55c001830c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.115+0000 7f3d1b040740 1 -- 172.21.15.105:0/2599775153 shutdown_connections 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.115+0000 7f3d1b040740 1 -- 172.21.15.105:0/2599775153 wait complete. 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 1 -- start start 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 1 Processor -- start 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c00183f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c00183fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001846a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c00183dd00 con 0x55c0018352b0 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c00174d160 con 0x55c001835ca0 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c00183de80 con 0x55c00183ae10 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d1b040740 10 client.0 _ll_register_callbacks cb 0x55c001789720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d161f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c00183f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d161f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c00183f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35556/0 (socket says 172.21.15.105:35556) 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d161f5640 1 -- 172.21.15.105:0/2502035343 learned_addr learned my addr 172.21.15.105:0/2502035343 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.116+0000 7f3d159f4640 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c00183fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d169f6640 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001846a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d159f4640 1 -- 172.21.15.105:0/2502035343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 msgr2=0x55c001846a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d159f4640 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001846a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d159f4640 1 -- 172.21.15.105:0/2502035343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 msgr2=0x55c00183f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d159f4640 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c00183f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d159f4640 1 -- 172.21.15.105:0/2502035343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0014a53f0 con 0x55c001835ca0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d169f6640 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001846a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d159f4640 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c00183fa00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55c0018360a0 tx=0x7f3d00009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8d18887257a508 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/2502035343 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0004f030 con 0x55c001835ca0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/2502035343 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f3d000048d0 con 0x55c001835ca0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3d1b040740 1 -- 172.21.15.105:0/2502035343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d0c0a0010 con 0x55c001835ca0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.117+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/2502035343 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3d00005650 con 0x55c001835ca0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d1b040740 10 client.32393 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d1b040740 1 -- 172.21.15.105:0/2502035343 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c0014650f0 con 0x55c001835ca0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/2502035343 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d0c0a0010 con 0x55c001835ca0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/2502035343 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3d00040450 con 0x55c001835ca0 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d1b040740 10 client.32393 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:20.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d1b040740 10 client.32393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:20.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d1b040740 20 client.32393 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:20.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c00184b7c0 0x55c00184dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d1b040740 4 client.32393 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:20.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d1b040740 1 -- 172.21.15.105:0/2502035343 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c00179eec0 con 0x55c00184b7c0 2024-03-26T17:58:20.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.118+0000 7f3d161f5640 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c00184b7c0 0x55c00184dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.119+0000 7f3cf9ffb640 10 client.32393 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:20.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.119+0000 7f3d161f5640 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c00184b7c0 0x55c00184dc20 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73dc6723c67efb7f server_cookie=b8a34b8f6ee11090 in_seq=0 out_seq=0 2024-03-26T17:58:20.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.121+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/2502035343 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c00179eec0 con 0x55c00184b7c0 2024-03-26T17:58:20.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.121+0000 7f3cf9ffb640 10 client.32393 handle_command_reply: tid=0 2024-03-26T17:58:20.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 client.32393 shutdown 2024-03-26T17:58:20.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 -- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 msgr2=0x55c00183fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c00183fa00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55c0018360a0 tx=0x7f3d00009260 comp rx=0 tx=0).stop 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 -- 172.21.15.105:0/2502035343 shutdown_connections 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c00184b7c0 0x55c00184dc20 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001846a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c00183fa00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3cf9ffb640 0 client.32393 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2502035343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c00183f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 -- 172.21.15.105:0/2502035343 >> 172.21.15.105:0/2502035343 conn(0x55c001830820 msgr2=0x55c001827210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 -- 172.21.15.105:0/2502035343 shutdown_connections 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 1 -- 172.21.15.105:0/2502035343 wait complete. 2024-03-26T17:58:20.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.122+0000 7f3d1b040740 20 client.32393 trim_cache size 0 max 16384 2024-03-26T17:58:20.134 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:19 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d1b040740 1 Processor -- start 2024-03-26T17:58:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d1b040740 1 -- start start 2024-03-26T17:58:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c001830c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c001846a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001836f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d1b040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a6fd0 con 0x55c0018352b0 2024-03-26T17:58:20.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d1b040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a8b10 con 0x55c001835ca0 2024-03-26T17:58:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d1b040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a76a0 con 0x55c00183ae10 2024-03-26T17:58:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d161f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c001830c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d161f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c001830c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35570/0 (socket says 172.21.15.105:35570) 2024-03-26T17:58:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d161f5640 1 -- 172.21.15.105:0/2103814206 learned_addr learned my addr 172.21.15.105:0/2103814206 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d169f6640 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001836f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.150+0000 7f3d159f4640 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c001846a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3d169f6640 1 -- 172.21.15.105:0/2103814206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 msgr2=0x55c001846a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3d169f6640 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c001846a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3d169f6640 1 -- 172.21.15.105:0/2103814206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 msgr2=0x55c001830c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3d169f6640 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c001830c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3d169f6640 1 -- 172.21.15.105:0/2103814206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c001831590 con 0x55c00183ae10 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3d161f5640 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c001830c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3d169f6640 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001836f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55c001831790 tx=0x7f3d08012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2692713116417607 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3cfbfff640 1 -- 172.21.15.105:0/2103814206 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08003e40 con 0x55c00183ae10 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3cfbfff640 1 -- 172.21.15.105:0/2103814206 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d08004030 con 0x55c00183ae10 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.151+0000 7f3cfbfff640 1 -- 172.21.15.105:0/2103814206 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08005280 con 0x55c00183ae10 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.152+0000 7f3d1b040740 1 -- 172.21.15.105:0/2103814206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 msgr2=0x55c001836f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.152+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001836f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55c001831790 tx=0x7f3d08012820 comp rx=0 tx=0).stop 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 -- 172.21.15.105:0/2103814206 shutdown_connections 2024-03-26T17:58:20.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c00183ae10 0x55c001836f80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c001846a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 --2- 172.21.15.105:0/2103814206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c0018352b0 0x55c001830c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 -- 172.21.15.105:0/2103814206 >> 172.21.15.105:0/2103814206 conn(0x55c001830820 msgr2=0x55c00183f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 -- 172.21.15.105:0/2103814206 shutdown_connections 2024-03-26T17:58:20.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 -- 172.21.15.105:0/2103814206 wait complete. 2024-03-26T17:58:20.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 -- start start 2024-03-26T17:58:20.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.153+0000 7f3d1b040740 1 Processor -- start 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0018352b0 0x55c001843fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c001844500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d1b040740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c001845510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d1b040740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a8440 con 0x55c00183ae10 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d1b040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a7d70 con 0x55c001835ca0 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d1b040740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d101a4fb0 con 0x55c0018352b0 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d1b040740 10 client.0 _ll_register_callbacks cb 0x7f3d00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d169f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c001845510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d169f6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c001845510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:35574/0 (socket says 172.21.15.105:35574) 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d169f6640 1 -- 172.21.15.105:0/922314333 learned_addr learned my addr 172.21.15.105:0/922314333 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d161f5640 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0018352b0 0x55c001843fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.154+0000 7f3d159f4640 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c001844500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3d169f6640 1 -- 172.21.15.105:0/922314333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0018352b0 msgr2=0x55c001843fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3d169f6640 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0018352b0 0x55c001843fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3d169f6640 1 -- 172.21.15.105:0/922314333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 msgr2=0x55c001844500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3d169f6640 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c001844500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3d169f6640 1 -- 172.21.15.105:0/922314333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c001845b00 con 0x55c00183ae10 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3d169f6640 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c001845510 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3d08007310 tx=0x7f3d08004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6b1c9c5a9b1e10 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/922314333 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d080045f0 con 0x55c00183ae10 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.155+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/922314333 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f3d0800f450 con 0x55c00183ae10 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3d1b040740 1 -- 172.21.15.105:0/922314333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d08004030 con 0x55c00183ae10 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/922314333 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3d0800f670 con 0x55c00183ae10 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3d1b040740 10 client.22719 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:20.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3d1b040740 1 -- 172.21.15.105:0/922314333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c0018b1b60 con 0x55c00183ae10 2024-03-26T17:58:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/922314333 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d08004030 con 0x55c00183ae10 2024-03-26T17:58:20.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/922314333 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3d08019030 con 0x55c00183ae10 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3d1b040740 10 client.22719 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3d1b040740 10 client.22719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3d1b040740 20 client.22719 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3d1b040740 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c0018b22e0 0x55c0018b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.156+0000 7f3d1b040740 4 client.22719 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.157+0000 7f3d1b040740 1 -- 172.21.15.105:0/922314333 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c00183e900 con 0x55c0018b22e0 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.157+0000 7f3d161f5640 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c0018b22e0 0x55c0018b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.157+0000 7f3cf9ffb640 10 client.22719 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:20.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.158+0000 7f3d161f5640 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c0018b22e0 0x55c0018b4700 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260c626655ef4dcf server_cookie=5b26157ed60397ce in_seq=0 out_seq=0 2024-03-26T17:58:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3cf9ffb640 1 -- 172.21.15.105:0/922314333 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55c00183e900 con 0x55c0018b22e0 2024-03-26T17:58:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3cf9ffb640 10 client.22719 handle_command_reply: tid=0 2024-03-26T17:58:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 client.22719 shutdown 2024-03-26T17:58:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 -- 172.21.15.105:0/922314333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 msgr2=0x55c001845510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c001845510 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3d08007310 tx=0x7f3d08004c70 comp rx=0 tx=0).stop 2024-03-26T17:58:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 -- 172.21.15.105:0/922314333 shutdown_connections 2024-03-26T17:58:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c0018b22e0 0x55c0018b4700 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3cf9ffb640 0 client.22719 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c00183ae10 0x55c001845510 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c001835ca0 0x55c001844500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 --2- 172.21.15.105:0/922314333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c0018352b0 0x55c001843fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.163+0000 7f3d1b040740 1 -- 172.21.15.105:0/922314333 >> 172.21.15.105:0/922314333 conn(0x55c001830820 msgr2=0x55c00183f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.164+0000 7f3d1b040740 1 -- 172.21.15.105:0/922314333 shutdown_connections 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.164+0000 7f3d1b040740 1 -- 172.21.15.105:0/922314333 wait complete. 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.164+0000 7f3d1b040740 20 client.22719 trim_cache size 0 max 16384 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (227 inodes in the stack)", 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stdout: "45fe17c3-260a-49e6-afd8-cadadd423adb": { 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "45fe17c3-260a-49e6-afd8-cadadd423adb", 2024-03-26T17:58:20.150 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:58:20.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:58:20.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:58:20.151 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3cfc0a2fa0 msgr2=0x7f3cfc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3cfc0a2fa0 0x7f3cfc0a5460 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3d00036040 tx=0x7f3d0002c7e0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d101039c0 msgr2=0x7f3d10199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d101039c0 0x7f3d10199db0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00e9c0 tx=0x7f3d0c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 shutdown_connections 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3cfc0a2fa0 0x7f3cfc0a5460 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3d1019f350 0x7f3d101a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d10105570 0x7f3d1019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 --2- 172.21.15.105:0/269998867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3d101039c0 0x7f3d10199db0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 >> 172.21.15.105:0/269998867 conn(0x7f3d100fef30 msgr2=0x7f3d10107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 shutdown_connections 2024-03-26T17:58:20.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:20.165+0000 7f3d171f7640 1 -- 172.21.15.105:0/269998867 wait complete. 2024-03-26T17:58:20.165 INFO:tasks.cephfs.filesystem:scrub status for tag:45fe17c3-260a-49e6-afd8-cadadd423adb - {'path': '/', 'tag': '45fe17c3-260a-49e6-afd8-cadadd423adb', 'options': 'recursive,force'} 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 -- 172.21.15.105:0/2810473380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d81002f0 msgr2=0x7fb9d8100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/2810473380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d81002f0 0x7fb9d8100760 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009ca0 tx=0x7fb9d002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 -- 172.21.15.105:0/2810473380 shutdown_connections 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/2810473380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d81002f0 0x7fb9d8100760 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/2810473380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d80ff930 0x7fb9d80ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/2810473380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d8101db0 0x7fb9d81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 -- 172.21.15.105:0/2810473380 >> 172.21.15.105:0/2810473380 conn(0x7fb9d80fb510 msgr2=0x7fb9d80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 -- 172.21.15.105:0/2810473380 shutdown_connections 2024-03-26T17:58:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 -- 172.21.15.105:0/2810473380 wait complete. 2024-03-26T17:58:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 Processor -- start 2024-03-26T17:58:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.237+0000 7fb9e01f3640 1 -- start start 2024-03-26T17:58:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9e01f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d80ff930 0x7fb9d8199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9e01f3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81002f0 0x7fb9d819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9e01f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d8101db0 0x7fb9d81a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9e01f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810f610 con 0x7fb9d8101db0 2024-03-26T17:58:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9e01f3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810f490 con 0x7fb9d81002f0 2024-03-26T17:58:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9e01f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d810f790 con 0x7fb9d80ff930 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9dd78e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81002f0 0x7fb9d819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9dd78e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81002f0 0x7fb9d819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40542/0 (socket says 172.21.15.105:40542) 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9de790640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d8101db0 0x7fb9d81a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9dd78e640 1 -- 172.21.15.105:0/557080519 learned_addr learned my addr 172.21.15.105:0/557080519 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9ddf8f640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d80ff930 0x7fb9d8199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9de790640 1 -- 172.21.15.105:0/557080519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d80ff930 msgr2=0x7fb9d8199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9de790640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d80ff930 0x7fb9d8199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9de790640 1 -- 172.21.15.105:0/557080519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81002f0 msgr2=0x7fb9d819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9de790640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81002f0 0x7fb9d819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9de790640 1 -- 172.21.15.105:0/557080519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d81a1870 con 0x7fb9d8101db0 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9ddf8f640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d80ff930 0x7fb9d8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.238+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81002f0 0x7fb9d819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.239+0000 7fb9de790640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d8101db0 0x7fb9d81a11d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fb9d002c630 tx=0x7fb9d00328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.239+0000 7fb9ceffd640 1 -- 172.21.15.105:0/557080519 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0041020 con 0x7fb9d8101db0 2024-03-26T17:58:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.239+0000 7fb9ceffd640 1 -- 172.21.15.105:0/557080519 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9d0003e20 con 0x7fb9d8101db0 2024-03-26T17:58:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.239+0000 7fb9e01f3640 1 -- 172.21.15.105:0/557080519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d81a1b00 con 0x7fb9d8101db0 2024-03-26T17:58:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.239+0000 7fb9ceffd640 1 -- 172.21.15.105:0/557080519 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0039040 con 0x7fb9d8101db0 2024-03-26T17:58:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.239+0000 7fb9e01f3640 1 -- 172.21.15.105:0/557080519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d81a2040 con 0x7fb9d8101db0 2024-03-26T17:58:20.226 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.242+0000 7fb9ceffd640 1 -- 172.21.15.105:0/557080519 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb9d0044020 con 0x7fb9d8101db0 2024-03-26T17:58:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.242+0000 7fb9e2dd7740 1 Processor -- start 2024-03-26T17:58:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.242+0000 7fb9ceffd640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb9c40a30c0 0x7fb9c40a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.242+0000 7fb9ceffd640 1 -- 172.21.15.105:0/557080519 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb9d00f1690 con 0x7fb9d8101db0 2024-03-26T17:58:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.243+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb9c40a30c0 0x7fb9c40a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.243+0000 7fb9e2dd7740 1 -- start start 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.243+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a25d0 0x5597ec5a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.243+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.243+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a8130 0x5597ec5aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5597ec5070b0 con 0x5597ec5a25d0 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597ec4af960 con 0x5597ec5a8130 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb9c40a30c0 0x7fb9c40a5580 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8011500 tx=0x7fb9c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5a7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:54996/0 (socket says 172.21.15.105:54996) 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 -- 172.21.15.105:0/1739466062 learned_addr learned my addr 172.21.15.105:0/1739466062 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9ddf8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a25d0 0x5597ec5a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9de790640 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a8130 0x5597ec5aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/1739466062 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5597ec503060 con 0x5597ec5a2fc0 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 -- 172.21.15.105:0/1739466062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a8130 msgr2=0x5597ec5aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a8130 0x5597ec5aa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 -- 172.21.15.105:0/1739466062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a25d0 msgr2=0x5597ec5a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a25d0 0x5597ec5a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9dd78e640 1 -- 172.21.15.105:0/1739466062 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ec4c1c60 con 0x5597ec5a2fc0 2024-03-26T17:58:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.244+0000 7fb9de790640 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a8130 0x5597ec5aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.245+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5a7ac0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800f780 tx=0x7fb9c8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51445e737a4725a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.245+0000 7fb9c3fff640 1 -- 172.21.15.105:0/1739466062 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c801c910 con 0x5597ec5a2fc0 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.245+0000 7fb9c3fff640 1 -- 172.21.15.105:0/1739466062 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c8025390 con 0x5597ec5a2fc0 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.245+0000 7fb9c3fff640 1 -- 172.21.15.105:0/1739466062 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c801a5f0 con 0x5597ec5a2fc0 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.246+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/1739466062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 msgr2=0x5597ec5a7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.246+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5a7ac0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800f780 tx=0x7fb9c8014000 comp rx=0 tx=0).stop 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.246+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/1739466062 shutdown_connections 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.246+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a8130 0x5597ec5aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.246+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5a7ac0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.246+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/1739466062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a25d0 0x5597ec5a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.246+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/1739466062 >> 172.21.15.105:0/1739466062 conn(0x5597ec59db40 msgr2=0x5597ec59df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.247+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/1739466062 shutdown_connections 2024-03-26T17:58:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.247+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/1739466062 wait complete. 2024-03-26T17:58:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.247+0000 7fb9e2dd7740 1 -- start start 2024-03-26T17:58:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.247+0000 7fb9e2dd7740 1 Processor -- start 2024-03-26T17:58:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a25d0 0x5597ec5ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a2fc0 0x5597ec5ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5597ec5ab130 con 0x5597ec5a8130 2024-03-26T17:58:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5597ec4c2db0 con 0x5597ec5a2fc0 2024-03-26T17:58:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5597ec5ab2b0 con 0x5597ec5a25d0 2024-03-26T17:58:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9ddf8f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a25d0 0x5597ec5ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9dd78e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a2fc0 0x5597ec5ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9ddf8f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a25d0 0x5597ec5ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55008/0 (socket says 172.21.15.105:55008) 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9ddf8f640 1 -- 172.21.15.105:0/3631832324 learned_addr learned my addr 172.21.15.105:0/3631832324 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9e2dd7740 10 client.0 _ll_register_callbacks cb 0x5597ec4c7110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9de790640 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9dd78e640 1 -- 172.21.15.105:0/3631832324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a25d0 msgr2=0x5597ec5ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a25d0 0x5597ec5ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9dd78e640 1 -- 172.21.15.105:0/3631832324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 msgr2=0x5597ec5b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9dd78e640 1 -- 172.21.15.105:0/3631832324 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ec21b210 con 0x5597ec5a2fc0 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9ddf8f640 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a25d0 0x5597ec5ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a2fc0 0x5597ec5ac920 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800f750 tx=0x7fb9c8025ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88c4cbc43d2acc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.248+0000 7fb9c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.249+0000 7fb9c17fa640 1 -- 172.21.15.105:0/3631832324 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8033060 con 0x5597ec5a2fc0 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.249+0000 7fb9c17fa640 1 -- 172.21.15.105:0/3631832324 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb9c8005260 con 0x5597ec5a2fc0 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.249+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/3631832324 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c8025390 con 0x5597ec5a2fc0 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.249+0000 7fb9c17fa640 1 -- 172.21.15.105:0/3631832324 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb9c8005460 con 0x5597ec5a2fc0 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.249+0000 7fb9e2dd7740 10 client.32411 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.249+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/3631832324 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5597ec2e7620 con 0x5597ec5a2fc0 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.249+0000 7fb9c17fa640 1 -- 172.21.15.105:0/3631832324 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c8005600 con 0x5597ec5a2fc0 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.249+0000 7fb9c17fa640 1 -- 172.21.15.105:0/3631832324 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c802e820 con 0x5597ec5a2fc0 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9e2dd7740 10 client.32411 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9e2dd7740 10 client.32411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9e2dd7740 20 client.32411 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597ec5b7e10 0x5597ec5ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9e2dd7740 4 client.32411 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/3631832324 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb9c8025390 con 0x5597ec5b7e10 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9ddf8f640 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597ec5b7e10 0x5597ec5ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9c17fa640 10 client.32411 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.250+0000 7fb9ddf8f640 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597ec5b7e10 0x5597ec5ba200 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83953af0e64e6d88 server_cookie=1dd14201b3447ad1 in_seq=0 out_seq=0 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9c17fa640 1 -- 172.21.15.105:0/3631832324 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fb9c8025390 con 0x5597ec5b7e10 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9c17fa640 10 client.32411 handle_command_reply: tid=0 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 client.32411 shutdown 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/3631832324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a2fc0 msgr2=0x5597ec5ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a2fc0 0x5597ec5ac920 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800f750 tx=0x7fb9c8025ab0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/3631832324 shutdown_connections 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597ec5b7e10 0x5597ec5ba200 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a2fc0 0x5597ec5ac920 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9c17fa640 0 client.32411 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/3631832324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a25d0 0x5597ec5ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/3631832324 >> 172.21.15.105:0/3631832324 conn(0x5597ec59db40 msgr2=0x5597ec59cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/3631832324 shutdown_connections 2024-03-26T17:58:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/3631832324 wait complete. 2024-03-26T17:58:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.252+0000 7fb9e2dd7740 20 client.32411 trim_cache size 0 max 16384 2024-03-26T17:58:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.272+0000 7fb9e2dd7740 1 Processor -- start 2024-03-26T17:58:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.272+0000 7fb9e2dd7740 1 -- start start 2024-03-26T17:58:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.272+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.272+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a8130 0x5597ec5a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81aaf60 con 0x5597ec5a2fc0 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81acaa0 con 0x5597ec5a25d0 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81ab630 con 0x5597ec5a8130 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9ddf8f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9dd78e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9ddf8f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5a39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40574/0 (socket says 172.21.15.105:40574) 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9ddf8f640 1 -- 172.21.15.105:0/2721060305 learned_addr learned my addr 172.21.15.105:0/2721060305 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9de790640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a8130 0x5597ec5a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9dd78e640 1 -- 172.21.15.105:0/2721060305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a8130 msgr2=0x5597ec5a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/2721060305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a8130 0x5597ec5a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9dd78e640 1 -- 172.21.15.105:0/2721060305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 msgr2=0x5597ec5a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/2721060305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9dd78e640 1 -- 172.21.15.105:0/2721060305 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ec599450 con 0x5597ec5a2fc0 2024-03-26T17:58:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9de790640 1 --2- 172.21.15.105:0/2721060305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a8130 0x5597ec5a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/2721060305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b3920 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8010010 tx=0x7fb9c800f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33e9e369b13086d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.273+0000 7fb9c37fe640 1 -- 172.21.15.105:0/2721060305 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c803d020 con 0x5597ec5a2fc0 2024-03-26T17:58:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9c37fe640 1 -- 172.21.15.105:0/2721060305 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c8005600 con 0x5597ec5a2fc0 2024-03-26T17:58:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9c37fe640 1 -- 172.21.15.105:0/2721060305 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8033040 con 0x5597ec5a2fc0 2024-03-26T17:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/2721060305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a2fc0 msgr2=0x5597ec5b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/2721060305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b3920 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8010010 tx=0x7fb9c800f650 comp rx=0 tx=0).stop 2024-03-26T17:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/2721060305 shutdown_connections 2024-03-26T17:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/2721060305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a8130 0x5597ec5a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/2721060305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b3920 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/2721060305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.274+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/2721060305 >> 172.21.15.105:0/2721060305 conn(0x5597ec59db40 msgr2=0x5597ec5ac3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.275+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/2721060305 shutdown_connections 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.275+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/2721060305 wait complete. 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.275+0000 7fb9e2dd7740 1 -- start start 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.275+0000 7fb9e2dd7740 1 Processor -- start 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9e2dd7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81ac3d0 con 0x5597ec5a8130 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81abd00 con 0x5597ec5a25d0 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9e2dd7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d81a8f40 con 0x5597ec5a2fc0 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9ddf8f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9e2dd7740 10 client.0 _ll_register_callbacks cb 0x7fb9d0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9ddf8f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40586/0 (socket says 172.21.15.105:40586) 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9ddf8f640 1 -- 172.21.15.105:0/427547292 learned_addr learned my addr 172.21.15.105:0/427547292 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9de790640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9dd78e640 1 -- 172.21.15.105:0/427547292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 msgr2=0x5597ec5b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9dd78e640 1 -- 172.21.15.105:0/427547292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 msgr2=0x5597ec5b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9dd78e640 1 -- 172.21.15.105:0/427547292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ec5b2a20 con 0x5597ec5a2fc0 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9ddf8f640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.276+0000 7fb9de790640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9dd78e640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b1420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800f780 tx=0x7fb9c802edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41504c1778270d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9c17fa640 1 -- 172.21.15.105:0/427547292 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c803d030 con 0x5597ec5a2fc0 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/427547292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c8005600 con 0x5597ec5a2fc0 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9c17fa640 1 -- 172.21.15.105:0/427547292 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7fb9c801c050 con 0x5597ec5a2fc0 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9c17fa640 1 -- 172.21.15.105:0/427547292 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb9c8033070 con 0x5597ec5a2fc0 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9e2dd7740 10 client.32407 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/427547292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5597ec61e140 con 0x5597ec5a2fc0 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.277+0000 7fb9c17fa640 1 -- 172.21.15.105:0/427547292 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c8005600 con 0x5597ec5a2fc0 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9c17fa640 1 -- 172.21.15.105:0/427547292 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c80267e0 con 0x5597ec5a2fc0 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9e2dd7740 10 client.32407 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9e2dd7740 10 client.32407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9e2dd7740 20 client.32407 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597ec61e880 0x5597ec620ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9e2dd7740 4 client.32407 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/427547292 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597ec5145f0 con 0x5597ec61e880 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9ddf8f640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597ec61e880 0x5597ec620ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9c17fa640 10 client.32407 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.278+0000 7fb9ddf8f640 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597ec61e880 0x5597ec620ca0 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cdf7b5734b9da40 server_cookie=550bdd0e278d6de1 in_seq=0 out_seq=0 2024-03-26T17:58:20.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:19 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:20.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:19 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:58:20.407 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:58:20.007+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003c80 [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{1=1,2=1,3=1,4=1} v10624 ap=2 RANDEPHEMERALPIN f(v286 m2024-03-26T17:57:52.084519+0000 11=11+0) n(v22 rc2024-03-26T17:57:52.084519+0000 b2861056 12=11+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@5787} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d142580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003c80:[<0x10000003bc0/COREL v50>,<0x10000003bb6/~dmtmp v48>,<0x10000000007/client8 v2078>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003c80:[<0x10000003bc0/COREL v10624>,<0x10000003bb6/~dmtmp v2231>,<0x10000000007/client8 v9603>,<0x10000000006/clients v2787>,<0x10000000005/tmp v2562>,<0x10000000003/client.0 v2458>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v2394>,<0x10000000001/sv_0 v2404>,<0x10000000000/qa v2326>,<0x1/volumes v2297>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:57:52.048759+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:57:52.084519+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v286 m2024-03-26T17:57:52.084519+0000 11=11+0)","memory_value.rstat":"n(v22 rc2024-03-26T17:57:52.084519+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T17:58:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.477+0000 7fb9c17fa640 1 -- 172.21.15.105:0/427547292 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597ec5145f0 con 0x5597ec61e880 2024-03-26T17:58:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.477+0000 7fb9c17fa640 10 client.32407 handle_command_reply: tid=0 2024-03-26T17:58:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.477+0000 7fb9e2dd7740 1 client.32407 shutdown 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/427547292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 msgr2=0x5597ec5b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b1420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800f780 tx=0x7fb9c802edb0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/427547292 shutdown_connections 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5597ec61e880 0x5597ec620ca0 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5597ec5a8130 0x5597ec5b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5597ec5a2fc0 0x5597ec5b1420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9c17fa640 0 client.32407 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 --2- 172.21.15.105:0/427547292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5597ec5a25d0 0x5597ec5b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/427547292 >> 172.21.15.105:0/427547292 conn(0x5597ec59db40 msgr2=0x5597ec5a3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/427547292 shutdown_connections 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 1 -- 172.21.15.105:0/427547292 wait complete. 2024-03-26T17:58:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.478+0000 7fb9e2dd7740 20 client.32407 trim_cache size 0 max 16384 2024-03-26T17:58:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:58:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:58:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:58:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 -- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb9c40a30c0 msgr2=0x7fb9c40a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb9c40a30c0 0x7fb9c40a5580 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8011500 tx=0x7fb9c8005f90 comp rx=0 tx=0).stop 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 -- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d8101db0 msgr2=0x7fb9d81a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d8101db0 0x7fb9d81a11d0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fb9d002c630 tx=0x7fb9d00328a0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 -- 172.21.15.105:0/557080519 shutdown_connections 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb9c40a30c0 0x7fb9c40a5580 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9d8101db0 0x7fb9d81a11d0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9d81002f0 0x7fb9d819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 --2- 172.21.15.105:0/557080519 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d80ff930 0x7fb9d8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 -- 172.21.15.105:0/557080519 >> 172.21.15.105:0/557080519 conn(0x7fb9d80fb510 msgr2=0x7fb9d80fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 -- 172.21.15.105:0/557080519 shutdown_connections 2024-03-26T17:58:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:20.479+0000 7fb9e01f3640 1 -- 172.21.15.105:0/557080519 wait complete. 2024-03-26T17:58:20.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 55453 7.05 MB/sec execute 303 sec latency 240.645 ms 2024-03-26T17:58:21.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:21.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:21.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:58:21.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:20 smithi144 ceph-mon[36095]: pgmap v950: 97 pgs: 97 active+clean; 331 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 3.2 MiB/s wr, 158 op/s 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:58:21.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:21.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:21.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:21.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:21.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:58:21.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:20 smithi105 ceph-mon[32280]: pgmap v950: 97 pgs: 97 active+clean; 331 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 3.2 MiB/s wr, 158 op/s 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:58:21.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:58:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:58:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:58:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:21.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:20 smithi135 ceph-mon[35636]: pgmap v950: 97 pgs: 97 active+clean; 331 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 3.2 MiB/s wr, 158 op/s 2024-03-26T17:58:21.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 55602 7.06 MB/sec execute 304 sec latency 134.288 ms 2024-03-26T17:58:22.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 55735 7.06 MB/sec execute 305 sec latency 131.505 ms 2024-03-26T17:58:23.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:22 smithi144 ceph-mon[36095]: pgmap v951: 97 pgs: 97 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 3.4 MiB/s wr, 226 op/s 2024-03-26T17:58:23.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:22 smithi105 ceph-mon[32280]: pgmap v951: 97 pgs: 97 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 3.4 MiB/s wr, 226 op/s 2024-03-26T17:58:23.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:22 smithi135 ceph-mon[35636]: pgmap v951: 97 pgs: 97 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 3.4 MiB/s wr, 226 op/s 2024-03-26T17:58:23.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 55865 7.07 MB/sec execute 306 sec latency 121.429 ms 2024-03-26T17:58:24.555 INFO:tasks.workunit.client.0.smithi105.stdout: 10 56000 7.07 MB/sec execute 307 sec latency 114.828 ms 2024-03-26T17:58:25.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:24 smithi144 ceph-mon[36095]: pgmap v952: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.6 MiB/s wr, 287 op/s 2024-03-26T17:58:25.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:24 smithi105 ceph-mon[32280]: pgmap v952: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.6 MiB/s wr, 287 op/s 2024-03-26T17:58:25.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:24 smithi135 ceph-mon[35636]: pgmap v952: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 775 KiB/s rd, 3.6 MiB/s wr, 287 op/s 2024-03-26T17:58:25.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 56122 7.06 MB/sec execute 308 sec latency 160.859 ms 2024-03-26T17:58:26.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 56255 7.06 MB/sec execute 309 sec latency 131.044 ms 2024-03-26T17:58:27.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:26 smithi144 ceph-mon[36095]: pgmap v953: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-26T17:58:27.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:26 smithi105 ceph-mon[32280]: pgmap v953: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-26T17:58:27.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:26 smithi135 ceph-mon[35636]: pgmap v953: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 806 KiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-03-26T17:58:27.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 56399 7.06 MB/sec execute 310 sec latency 152.891 ms 2024-03-26T17:58:28.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 56532 7.05 MB/sec execute 311 sec latency 113.255 ms 2024-03-26T17:58:29.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:28 smithi144 ceph-mon[36095]: pgmap v954: 97 pgs: 97 active+clean; 316 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-26T17:58:29.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:28 smithi105 ceph-mon[32280]: pgmap v954: 97 pgs: 97 active+clean; 316 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-26T17:58:29.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:28 smithi135 ceph-mon[35636]: pgmap v954: 97 pgs: 97 active+clean; 316 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.3 MiB/s wr, 310 op/s 2024-03-26T17:58:29.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 56680 7.05 MB/sec execute 312 sec latency 128.259 ms 2024-03-26T17:58:30.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 56867 7.05 MB/sec execute 313 sec latency 118.930 ms 2024-03-26T17:58:31.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:30 smithi144 ceph-mon[36095]: pgmap v955: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-03-26T17:58:31.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:30 smithi105 ceph-mon[32280]: pgmap v955: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-03-26T17:58:31.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:30 smithi135 ceph-mon[35636]: pgmap v955: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 2.7 MiB/s wr, 293 op/s 2024-03-26T17:58:31.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 57007 7.04 MB/sec execute 314 sec latency 117.353 ms 2024-03-26T17:58:32.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:31 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:58:32.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:31 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:58:32.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:31 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:58:32.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 57149 7.03 MB/sec execute 315 sec latency 103.747 ms 2024-03-26T17:58:33.284 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:32 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:58:33.285 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:32 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:58:33.285 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:32 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:58:33.285 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:32 smithi105 ceph-mon[32280]: pgmap v956: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-03-26T17:58:33.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:32 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:58:33.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:32 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:58:33.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:32 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:58:33.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:32 smithi144 ceph-mon[36095]: pgmap v956: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-03-26T17:58:33.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:32 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:58:33.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:32 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:58:33.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:32 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:58:33.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:32 smithi135 ceph-mon[35636]: pgmap v956: 97 pgs: 97 active+clean; 318 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-03-26T17:58:33.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 57310 7.04 MB/sec execute 316 sec latency 101.766 ms 2024-03-26T17:58:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 -- 172.21.15.105:0/3970022818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea81099d0 msgr2=0x7f6ea810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3970022818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea81099d0 0x7f6ea810bdc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009900 tx=0x7f6ea402bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 -- 172.21.15.105:0/3970022818 shutdown_connections 2024-03-26T17:58:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3970022818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ea810ec80 0x7f6ea8111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3970022818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea810c300 0x7f6ea810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3970022818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea81099d0 0x7f6ea810bdc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 -- 172.21.15.105:0/3970022818 >> 172.21.15.105:0/3970022818 conn(0x7f6ea806e900 msgr2=0x7f6ea8070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 -- 172.21.15.105:0/3970022818 shutdown_connections 2024-03-26T17:58:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.601+0000 7f6eafd10640 1 -- 172.21.15.105:0/3970022818 wait complete. 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eafd10640 1 Processor -- start 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eafd10640 1 -- start start 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eafd10640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea81099d0 0x7f6ea81093e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eafd10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ea810c300 0x7f6ea81a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eafd10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea810ec80 0x7f6ea81a7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eafd10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8114b20 con 0x7f6ea810c300 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eafd10640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81149a0 con 0x7f6ea81099d0 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eafd10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea8114ca0 con 0x7f6ea810ec80 2024-03-26T17:58:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eae2ad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea810ec80 0x7f6ea81a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eae2ad640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea810ec80 0x7f6ea81a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60958/0 (socket says 172.21.15.105:60958) 2024-03-26T17:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eae2ad640 1 -- 172.21.15.105:0/3112261922 learned_addr learned my addr 172.21.15.105:0/3112261922 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.602+0000 7f6eadaac640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea81099d0 0x7f6ea81093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.603+0000 7f6eae2ad640 1 -- 172.21.15.105:0/3112261922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea81099d0 msgr2=0x7f6ea81093e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.603+0000 7f6eae2ad640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea81099d0 0x7f6ea81093e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.603+0000 7f6eae2ad640 1 -- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ea810c300 msgr2=0x7f6ea81a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.603+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ea810c300 0x7f6ea81a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.603+0000 7f6eae2ad640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ea810c300 0x7f6ea81a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.603+0000 7f6eae2ad640 1 -- 172.21.15.105:0/3112261922 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea81a7f00 con 0x7f6ea810ec80 2024-03-26T17:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.603+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ea810c300 0x7f6ea81a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.604+0000 7f6eae2ad640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea810ec80 0x7f6ea81a7640 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000efd0 tx=0x7f6ea000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.604+0000 7f6e9effd640 1 -- 172.21.15.105:0/3112261922 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0003b70 con 0x7f6ea810ec80 2024-03-26T17:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.604+0000 7f6e9effd640 1 -- 172.21.15.105:0/3112261922 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea0003d10 con 0x7f6ea810ec80 2024-03-26T17:58:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.604+0000 7f6e9effd640 1 -- 172.21.15.105:0/3112261922 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0010980 con 0x7f6ea810ec80 2024-03-26T17:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.604+0000 7f6eafd10640 1 -- 172.21.15.105:0/3112261922 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea81a81f0 con 0x7f6ea810ec80 2024-03-26T17:58:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.604+0000 7f6eafd10640 1 -- 172.21.15.105:0/3112261922 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81a86f0 con 0x7f6ea810ec80 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.608+0000 7f6eb29d9740 1 Processor -- start 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.609+0000 7f6e9effd640 1 -- 172.21.15.105:0/3112261922 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6ea0004020 con 0x7f6ea810ec80 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.609+0000 7f6eb29d9740 1 -- start start 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.609+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x55669838a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.609+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x55669838da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.609+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838e0a0 0x55669839af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.609+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5566982ef0b0 con 0x55669838a550 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.609+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556698297960 con 0x55669838af40 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.609+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5566982eb060 con 0x55669838e0a0 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6ead2ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x55669838a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6ead2ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x55669838a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56550/0 (socket says 172.21.15.105:56550) 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6ead2ab640 1 -- 172.21.15.105:0/2415030599 learned_addr learned my addr 172.21.15.105:0/2415030599 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:33.596 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6eadaac640 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x55669838da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838e0a0 0x55669839af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6e9effd640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e940a3040 0x7f6e940a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6eadaac640 1 -- 172.21.15.105:0/2415030599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838e0a0 msgr2=0x55669839af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6eadaac640 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838e0a0 0x55669839af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6eadaac640 1 -- 172.21.15.105:0/2415030599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 msgr2=0x55669838a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6eadaac640 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x55669838a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6eadaac640 1 -- 172.21.15.105:0/2415030599 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566982a9c60 con 0x55669838af40 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6eae2ad640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e940a3040 0x7f6e940a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838e0a0 0x55669839af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.610+0000 7f6eadaac640 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x55669838da30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4000c00 tx=0x7f6ea4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad3fd471be5ec10 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.611+0000 7f6e9effd640 1 -- 172.21.15.105:0/3112261922 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f6ea00cf860 con 0x7f6ea810ec80 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.611+0000 7f6e93fff640 1 -- 172.21.15.105:0/2415030599 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4045020 con 0x55669838af40 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.611+0000 7f6e93fff640 1 -- 172.21.15.105:0/2415030599 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea4004040 con 0x55669838af40 2024-03-26T17:58:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.611+0000 7f6e93fff640 1 -- 172.21.15.105:0/2415030599 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea40046c0 con 0x55669838af40 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2415030599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 msgr2=0x55669838da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eae2ad640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e940a3040 0x7f6e940a5500 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6ea001a040 tx=0x7f6ea0009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x55669838da30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4000c00 tx=0x7f6ea4005d50 comp rx=0 tx=0).stop 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2415030599 shutdown_connections 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838e0a0 0x55669839af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x55669838da30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/2415030599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x55669838a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2415030599 >> 172.21.15.105:0/2415030599 conn(0x556698385b40 msgr2=0x556698385f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2415030599 shutdown_connections 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2415030599 wait complete. 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 -- start start 2024-03-26T17:58:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.612+0000 7f6eb29d9740 1 Processor -- start 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838a550 0x55669839cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838af40 0x55669839d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838e0a0 0x5566983a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55669839b950 con 0x55669838af40 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5566982aadb0 con 0x55669838e0a0 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55669839bad0 con 0x55669838a550 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838a550 0x55669839cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838a550 0x55669839cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60968/0 (socket says 172.21.15.105:60968) 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 -- 172.21.15.105:0/622307646 learned_addr learned my addr 172.21.15.105:0/622307646 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838e0a0 0x5566983a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eadaac640 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838af40 0x55669839d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eb29d9740 10 client.0 _ll_register_callbacks cb 0x5566982af110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 -- 172.21.15.105:0/622307646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838e0a0 msgr2=0x5566983a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838e0a0 0x5566983a40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 -- 172.21.15.105:0/622307646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838af40 msgr2=0x55669839d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838af40 0x55669839d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 -- 172.21.15.105:0/622307646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556698003210 con 0x55669838a550 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eadaac640 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838af40 0x55669839d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6eb29d9740 1 -- 172.21.15.105:0/622307646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea4004040 con 0x55669838a550 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838a550 0x55669839cb70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6e98013ec0 tx=0x7f6e9800b470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e34a11269a3c9f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.613+0000 7f6e917fa640 10 client.32413 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6e917fa640 1 -- 172.21.15.105:0/622307646 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800d5b0 con 0x55669838a550 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6e917fa640 1 -- 172.21.15.105:0/622307646 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f6e9800d750 con 0x55669838a550 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6e917fa640 1 -- 172.21.15.105:0/622307646 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6e9800d950 con 0x55669838a550 2024-03-26T17:58:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eb29d9740 10 client.32413 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eb29d9740 1 -- 172.21.15.105:0/622307646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5566980cf620 con 0x55669838a550 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6e917fa640 1 -- 172.21.15.105:0/622307646 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f6e9800daf0 con 0x55669838a550 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eb29d9740 10 client.32413 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eb29d9740 10 client.32413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eb29d9740 20 client.32413 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5566983a87c0 0x5566983aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eb29d9740 4 client.32413 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eb29d9740 1 -- 172.21.15.105:0/622307646 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566982fc5f0 con 0x5566983a87c0 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.614+0000 7f6eadaac640 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5566983a87c0 0x5566983aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.615+0000 7f6e917fa640 10 client.32413 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.615+0000 7f6eadaac640 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5566983a87c0 0x5566983aabd0 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4676d05caaf8df1d server_cookie=56481227d22116aa in_seq=0 out_seq=0 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.616+0000 7f6e917fa640 1 -- 172.21.15.105:0/622307646 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5566982fc5f0 con 0x5566983a87c0 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.616+0000 7f6e917fa640 10 client.32413 handle_command_reply: tid=0 2024-03-26T17:58:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.616+0000 7f6eb29d9740 1 client.32413 shutdown 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.616+0000 7f6eb29d9740 1 -- 172.21.15.105:0/622307646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838a550 msgr2=0x55669839cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.616+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838a550 0x55669839cb70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6e98013ec0 tx=0x7f6e9800b470 comp rx=0 tx=0).stop 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 1 -- 172.21.15.105:0/622307646 shutdown_connections 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5566983a87c0 0x5566983aabd0 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6e917fa640 0 client.32413 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838e0a0 0x5566983a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838af40 0x55669839d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/622307646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838a550 0x55669839cb70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 1 -- 172.21.15.105:0/622307646 >> 172.21.15.105:0/622307646 conn(0x556698385b40 msgr2=0x556698384df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 1 -- 172.21.15.105:0/622307646 shutdown_connections 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 1 -- 172.21.15.105:0/622307646 wait complete. 2024-03-26T17:58:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.617+0000 7f6eb29d9740 20 client.32413 trim_cache size 0 max 16384 2024-03-26T17:58:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.640+0000 7f6eb29d9740 1 Processor -- start 2024-03-26T17:58:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eb29d9740 1 -- start start 2024-03-26T17:58:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838a550 0x556698396ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838af40 0x5566983a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838e0a0 0x556698397430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81af210 con 0x55669838e0a0 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81b0d50 con 0x55669838a550 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eb29d9740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81af8e0 con 0x55669838af40 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838e0a0 0x556698397430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eadaac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838af40 0x5566983a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838e0a0 0x556698397430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56568/0 (socket says 172.21.15.105:56568) 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6eadaac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838af40 0x5566983a40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60982/0 (socket says 172.21.15.105:60982) 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 -- 172.21.15.105:0/2822557346 learned_addr learned my addr 172.21.15.105:0/2822557346 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/2822557346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838a550 0x556698396ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 -- 172.21.15.105:0/2822557346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838af40 msgr2=0x5566983a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/2822557346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838af40 0x5566983a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 -- 172.21.15.105:0/2822557346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838a550 msgr2=0x556698396ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/2822557346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838a550 0x556698396ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.641+0000 7f6ead2ab640 1 -- 172.21.15.105:0/2822557346 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55669838e480 con 0x55669838e0a0 2024-03-26T17:58:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.642+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/2822557346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838e0a0 0x556698397430 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x55669839c3e0 tx=0x7f6e98018050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1cc7b0ce83a97b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.642+0000 7f6e937fe640 1 -- 172.21.15.105:0/2822557346 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98032040 con 0x55669838e0a0 2024-03-26T17:58:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.642+0000 7f6e937fe640 1 -- 172.21.15.105:0/2822557346 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea4004040 con 0x55669838e0a0 2024-03-26T17:58:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.642+0000 7f6e937fe640 1 -- 172.21.15.105:0/2822557346 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9802a6d0 con 0x55669838e0a0 2024-03-26T17:58:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2822557346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838e0a0 msgr2=0x556698397430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/2822557346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838e0a0 0x556698397430 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x55669839c3e0 tx=0x7f6e98018050 comp rx=0 tx=0).stop 2024-03-26T17:58:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2822557346 shutdown_connections 2024-03-26T17:58:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/2822557346 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838e0a0 0x556698397430 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/2822557346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55669838af40 0x5566983a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/2822557346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838a550 0x556698396ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2822557346 >> 172.21.15.105:0/2822557346 conn(0x556698385b40 msgr2=0x556698380c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2822557346 shutdown_connections 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 -- 172.21.15.105:0/2822557346 wait complete. 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 -- start start 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.643+0000 7f6eb29d9740 1 Processor -- start 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x5566983a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x5566983a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6ead2ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x5566983a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6ead2ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x5566983a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56576/0 (socket says 172.21.15.105:56576) 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6eb29d9740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566983a31a0 0x5566983a3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6ead2ab640 1 -- 172.21.15.105:0/1179077240 learned_addr learned my addr 172.21.15.105:0/1179077240 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81b0680 con 0x55669838a550 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81affb0 con 0x55669838af40 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea81ad1f0 con 0x5566983a31a0 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6eb29d9740 10 client.0 _ll_register_callbacks cb 0x7f6e9800baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6eadaac640 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x5566983a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566983a31a0 0x5566983a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6ead2ab640 1 -- 172.21.15.105:0/1179077240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566983a31a0 msgr2=0x5566983a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566983a31a0 0x5566983a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6ead2ab640 1 -- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 msgr2=0x5566983a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.644+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x5566983a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6ead2ab640 1 -- 172.21.15.105:0/1179077240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566983a3b60 con 0x55669838a550 2024-03-26T17:58:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6eadaac640 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x5566983a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6ead2ab640 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x5566983a1ca0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f6e98012010 tx=0x7f6e98000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b839e58ee1400a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6e917fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6e917fa640 1 -- 172.21.15.105:0/1179077240 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9803c020 con 0x55669838a550 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ea4004040 con 0x55669838a550 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6e917fa640 1 -- 172.21.15.105:0/1179077240 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8293+0+0 (secure 0 0 0) 0x7f6e98013030 con 0x55669838a550 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6e917fa640 1 -- 172.21.15.105:0/1179077240 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6e98047050 con 0x55669838a550 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6eb29d9740 10 client.22752 fetch_fsmap learned FSMap version 39 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.645+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55669840f390 con 0x55669838a550 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6e917fa640 1 -- 172.21.15.105:0/1179077240 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ea4004040 con 0x55669838a550 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6e917fa640 1 -- 172.21.15.105:0/1179077240 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f6e9802a920 con 0x55669838a550 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6eb29d9740 10 client.22752 fetch_fsmap finished waiting for FSMap version 39 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6eb29d9740 10 client.22752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6eb29d9740 20 client.22752 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55669840fce0 0x5566984120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6eb29d9740 4 client.22752 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:58:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566982f6940 con 0x55669840fce0 2024-03-26T17:58:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6eadaac640 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55669840fce0 0x5566984120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.646+0000 7f6e917fa640 10 client.22752 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.647+0000 7f6eadaac640 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55669840fce0 0x5566984120d0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9eb15c80a1d764d server_cookie=cc837ac065d55f6a in_seq=0 out_seq=0 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.809+0000 7f6e917fa640 1 -- 172.21.15.105:0/1179077240 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566982f6940 con 0x55669840fce0 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.809+0000 7f6e917fa640 10 client.22752 handle_command_reply: tid=0 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.809+0000 7f6eb29d9740 1 client.22752 shutdown 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.809+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 msgr2=0x5566983a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.809+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x5566983a1ca0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f6e98012010 tx=0x7f6e98000bc0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 shutdown_connections 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55669840fce0 0x5566984120d0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5566983a31a0 0x5566983a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669838af40 0x5566983a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6e917fa640 0 client.22752 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 1 --2- 172.21.15.105:0/1179077240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55669838a550 0x5566983a1ca0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 >> 172.21.15.105:0/1179077240 conn(0x556698385b40 msgr2=0x556698396ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 shutdown_connections 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 1 -- 172.21.15.105:0/1179077240 wait complete. 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.810+0000 7f6eb29d9740 20 client.22752 trim_cache size 0 max 16384 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:58:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:58:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 -- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e940a3040 msgr2=0x7f6e940a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e940a3040 0x7f6e940a5500 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6ea001a040 tx=0x7f6ea0009f90 comp rx=0 tx=0).stop 2024-03-26T17:58:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 -- 172.21.15.105:0/3112261922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea810ec80 msgr2=0x7f6ea81a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea810ec80 0x7f6ea81a7640 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000efd0 tx=0x7f6ea000c6a0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 -- 172.21.15.105:0/3112261922 shutdown_connections 2024-03-26T17:58:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e940a3040 0x7f6e940a5500 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ea810ec80 0x7f6ea81a7640 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ea810c300 0x7f6ea81a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 --2- 172.21.15.105:0/3112261922 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6ea81099d0 0x7f6ea81093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 -- 172.21.15.105:0/3112261922 >> 172.21.15.105:0/3112261922 conn(0x7f6ea806e900 msgr2=0x7f6ea810af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 -- 172.21.15.105:0/3112261922 shutdown_connections 2024-03-26T17:58:33.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:33.811+0000 7f6eafd10640 1 -- 172.21.15.105:0/3112261922 wait complete. 2024-03-26T17:58:34.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 57458 7.05 MB/sec execute 317 sec latency 197.632 ms 2024-03-26T17:58:34.907 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:34 smithi135 ceph-mon[35636]: daemon mds.e restarted 2024-03-26T17:58:34.907 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:58:34 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setuser ceph since I am not root 2024-03-26T17:58:34.907 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:58:34 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setgroup ceph since I am not root 2024-03-26T17:58:34.907 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:58:34 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/1030448587 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f50c5b7a40 con 0x55f50c58b000 2024-03-26T17:58:34.907 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 17:58:34 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T17:58:35.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:34 smithi144 ceph-mon[36095]: daemon mds.e restarted 2024-03-26T17:58:35.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:34 smithi105 ceph-mon[32280]: daemon mds.e restarted 2024-03-26T17:58:35.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 57549 7.04 MB/sec execute 318 sec latency 130.317 ms 2024-03-26T17:58:36.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:35 smithi144 ceph-mon[36095]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T17:58:36.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:35 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/1622534523,v1:172.21.15.135:6835/1622534523] up:boot 2024-03-26T17:58:36.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:35 smithi144 ceph-mon[36095]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:58:36.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:35 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:58:36.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:35 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:58:36.090 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:35 smithi144 ceph-mon[36095]: pgmap v958: 97 pgs: 97 active+clean; 314 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-03-26T17:58:36.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:35 smithi105 ceph-mon[32280]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T17:58:36.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:35 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/1622534523,v1:172.21.15.135:6835/1622534523] up:boot 2024-03-26T17:58:36.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:35 smithi105 ceph-mon[32280]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:58:36.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:35 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:58:36.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:35 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:58:36.134 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:35 smithi105 ceph-mon[32280]: pgmap v958: 97 pgs: 97 active+clean; 314 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-03-26T17:58:36.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:35 smithi135 ceph-mon[35636]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T17:58:36.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:35 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/1622534523,v1:172.21.15.135:6835/1622534523] up:boot 2024-03-26T17:58:36.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:35 smithi135 ceph-mon[35636]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T17:58:36.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:35 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T17:58:36.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:35 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T17:58:36.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:35 smithi135 ceph-mon[35636]: pgmap v958: 97 pgs: 97 active+clean; 314 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-03-26T17:58:36.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 57695 7.06 MB/sec execute 319 sec latency 124.577 ms 2024-03-26T17:58:37.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:36 smithi144 ceph-mon[36095]: pgmap v959: 97 pgs: 97 active+clean; 316 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 2.7 MiB/s wr, 235 op/s 2024-03-26T17:58:37.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:36 smithi105 ceph-mon[32280]: pgmap v959: 97 pgs: 97 active+clean; 316 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 2.7 MiB/s wr, 235 op/s 2024-03-26T17:58:37.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:36 smithi135 ceph-mon[35636]: pgmap v959: 97 pgs: 97 active+clean; 316 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 2.7 MiB/s wr, 235 op/s 2024-03-26T17:58:37.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 57826 7.05 MB/sec execute 320 sec latency 125.036 ms 2024-03-26T17:58:38.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 58016 7.07 MB/sec execute 321 sec latency 143.272 ms 2024-03-26T17:58:39.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:38 smithi144 ceph-mon[36095]: pgmap v960: 97 pgs: 97 active+clean; 319 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.3 MiB/s wr, 268 op/s 2024-03-26T17:58:39.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:38 smithi105 ceph-mon[32280]: pgmap v960: 97 pgs: 97 active+clean; 319 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.3 MiB/s wr, 268 op/s 2024-03-26T17:58:39.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:38 smithi135 ceph-mon[35636]: pgmap v960: 97 pgs: 97 active+clean; 319 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 3.3 MiB/s wr, 268 op/s 2024-03-26T17:58:39.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 58168 7.08 MB/sec execute 322 sec latency 121.530 ms 2024-03-26T17:58:40.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 58317 7.08 MB/sec execute 323 sec latency 151.302 ms 2024-03-26T17:58:41.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:40 smithi144 ceph-mon[36095]: pgmap v961: 97 pgs: 97 active+clean; 320 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.2 MiB/s wr, 247 op/s 2024-03-26T17:58:41.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:40 smithi105 ceph-mon[32280]: pgmap v961: 97 pgs: 97 active+clean; 320 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.2 MiB/s wr, 247 op/s 2024-03-26T17:58:41.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:40 smithi135 ceph-mon[35636]: pgmap v961: 97 pgs: 97 active+clean; 320 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 3.2 MiB/s wr, 247 op/s 2024-03-26T17:58:41.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 58472 7.10 MB/sec execute 324 sec latency 112.486 ms 2024-03-26T17:58:42.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 58574 7.09 MB/sec execute 325 sec latency 131.979 ms 2024-03-26T17:58:43.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:42 smithi144 ceph-mon[36095]: pgmap v962: 97 pgs: 97 active+clean; 325 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-03-26T17:58:43.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:42 smithi105 ceph-mon[32280]: pgmap v962: 97 pgs: 97 active+clean; 325 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-03-26T17:58:43.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:42 smithi135 ceph-mon[35636]: pgmap v962: 97 pgs: 97 active+clean; 325 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.7 MiB/s wr, 249 op/s 2024-03-26T17:58:43.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 58694 7.09 MB/sec execute 326 sec latency 114.858 ms 2024-03-26T17:58:44.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 58810 7.08 MB/sec execute 327 sec latency 167.546 ms 2024-03-26T17:58:45.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:44 smithi144 ceph-mon[36095]: pgmap v963: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-03-26T17:58:45.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:44 smithi105 ceph-mon[32280]: pgmap v963: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-03-26T17:58:45.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:44 smithi135 ceph-mon[35636]: pgmap v963: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 4.0 MiB/s wr, 243 op/s 2024-03-26T17:58:45.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 58966 7.10 MB/sec execute 328 sec latency 174.682 ms 2024-03-26T17:58:46.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59080 7.11 MB/sec execute 329 sec latency 119.802 ms 2024-03-26T17:58:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 -- 172.21.15.105:0/233819622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c1056d0 msgr2=0x7f0a9c10bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/233819622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c1056d0 0x7f0a9c10bf60 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009ca0 tx=0x7f0a9402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 -- 172.21.15.105:0/233819622 shutdown_connections 2024-03-26T17:58:46.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/233819622 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c1056d0 0x7f0a9c10bf60 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/233819622 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a9c104d10 0x7f0a9c105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/233819622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a9c103b20 0x7f0a9c103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 -- 172.21.15.105:0/233819622 >> 172.21.15.105:0/233819622 conn(0x7f0a9c0ff0b0 msgr2=0x7f0a9c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 -- 172.21.15.105:0/233819622 shutdown_connections 2024-03-26T17:58:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.935+0000 7f0aa1a87640 1 -- 172.21.15.105:0/233819622 wait complete. 2024-03-26T17:58:46.920 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.936+0000 7f0aa1a87640 1 Processor -- start 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.936+0000 7f0aa1a87640 1 -- start start 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0aa1a87640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c103b20 0x7f0a9c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c103b20 0x7f0a9c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0aa1a87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a9c104d10 0x7f0a9c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0aa1a87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a9c1056d0 0x7f0a9c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0aa1a87640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c10f630 con 0x7f0a9c103b20 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0aa1a87640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c10f4b0 con 0x7f0a9c104d10 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0aa1a87640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c10f7b0 con 0x7f0a9c1056d0 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c103b20 0x7f0a9c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37012/0 (socket says 172.21.15.105:37012) 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/3232332979 learned_addr learned my addr 172.21.15.105:0/3232332979 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9affd640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a9c104d10 0x7f0a9c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a9c1056d0 0x7f0a9c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/3232332979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a9c1056d0 msgr2=0x7f0a9c1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a9c1056d0 0x7f0a9c1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/3232332979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a9c104d10 msgr2=0x7f0a9c19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a9c104d10 0x7f0a9c19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/3232332979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c1a1800 con 0x7f0a9c103b20 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c103b20 0x7f0a9c199be0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9c0 tx=0x7f0a8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a9c1056d0 0x7f0a9c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a9affd640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a9c104d10 0x7f0a9c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.937+0000 7f0a98ff9640 1 -- 172.21.15.105:0/3232332979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00cd80 con 0x7f0a9c103b20 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.938+0000 7f0a98ff9640 1 -- 172.21.15.105:0/3232332979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a8c0035f0 con 0x7f0a9c103b20 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.938+0000 7f0a98ff9640 1 -- 172.21.15.105:0/3232332979 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c010690 con 0x7f0a9c103b20 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.938+0000 7f0aa1a87640 1 -- 172.21.15.105:0/3232332979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c1a1af0 con 0x7f0a9c103b20 2024-03-26T17:58:46.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.938+0000 7f0aa1a87640 1 -- 172.21.15.105:0/3232332979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c1a1ff0 con 0x7f0a9c103b20 2024-03-26T17:58:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0a98ff9640 1 -- 172.21.15.105:0/3232332979 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0a8c0037a0 con 0x7f0a9c103b20 2024-03-26T17:58:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0aa466a740 1 Processor -- start 2024-03-26T17:58:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0aa466a740 1 -- start start 2024-03-26T17:58:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d7bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d7e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0aa466a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559b50cdb0b0 con 0x559b50d7c130 2024-03-26T17:58:46.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0a98ff9640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0a840a31b0 0x7f0a840a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0aa466a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559b50c83960 con 0x559b50d765d0 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.941+0000 7f0aa466a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559b50cd7060 con 0x559b50d76fc0 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d7bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d7bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42296/0 (socket says 172.21.15.105:42296) 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/967871188 learned_addr learned my addr 172.21.15.105:0/967871188 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d7e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a98ff9640 1 -- 172.21.15.105:0/3232332979 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0a8c0cf780 con 0x7f0a9c103b20 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9affd640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0a840a31b0 0x7f0a840a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9bfff640 1 -- 172.21.15.105:0/967871188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 msgr2=0x559b50d7bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d7bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9bfff640 1 -- 172.21.15.105:0/967871188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 msgr2=0x559b50d7e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d7e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9bfff640 1 -- 172.21.15.105:0/967871188 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b50c95c60 con 0x559b50d765d0 2024-03-26T17:58:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d7bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.942+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d769f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009430 tx=0x7f0a94004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a36f112f172714 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:46.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.943+0000 7f0a9affd640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0a840a31b0 0x7f0a840a5670 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c19b0f0 tx=0x7f0a90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.945+0000 7f0a81ffb640 1 -- 172.21.15.105:0/967871188 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94048070 con 0x559b50d765d0 2024-03-26T17:58:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.945+0000 7f0a81ffb640 1 -- 172.21.15.105:0/967871188 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a9403a040 con 0x559b50d765d0 2024-03-26T17:58:46.929 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.946+0000 7f0a81ffb640 1 -- 172.21.15.105:0/967871188 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94039070 con 0x559b50d765d0 2024-03-26T17:58:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.946+0000 7f0aa466a740 1 -- 172.21.15.105:0/967871188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 msgr2=0x559b50d769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:46.930 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.946+0000 7f0aa466a740 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d769f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009430 tx=0x7f0a94004a40 comp rx=0 tx=0).stop 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 -- 172.21.15.105:0/967871188 shutdown_connections 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d7e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d7bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 --2- 172.21.15.105:0/967871188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d769f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 -- 172.21.15.105:0/967871188 >> 172.21.15.105:0/967871188 conn(0x559b50d71b40 msgr2=0x559b50d71f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 -- 172.21.15.105:0/967871188 shutdown_connections 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 -- 172.21.15.105:0/967871188 wait complete. 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 -- start start 2024-03-26T17:58:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.947+0000 7f0aa466a740 1 Processor -- start 2024-03-26T17:58:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d80420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d80960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0aa466a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559b50d7f080 con 0x559b50d7c130 2024-03-26T17:58:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0aa466a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559b50c96db0 con 0x559b50d765d0 2024-03-26T17:58:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0aa466a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559b50d7f200 con 0x559b50d76fc0 2024-03-26T17:58:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d80960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d80420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d80420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49348/0 (socket says 172.21.15.105:49348) 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/2306403817 learned_addr learned my addr 172.21.15.105:0/2306403817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d80960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42300/0 (socket says 172.21.15.105:42300) 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0aa466a740 10 client.0 _ll_register_callbacks cb 0x559b50c9b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/2306403817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 msgr2=0x559b50d80960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d80960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/2306403817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 msgr2=0x559b50d879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/2306403817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b509ef210 con 0x559b50d765d0 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.948+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d80420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c09bb10 tx=0x7f0a8c09ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4796d946e2cf0ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0a7affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:46.935 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0a7affd640 1 -- 172.21.15.105:0/2306403817 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0abfb0 con 0x559b50d765d0 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0aa466a740 1 -- 172.21.15.105:0/2306403817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a9403a040 con 0x559b50d765d0 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0a7affd640 1 -- 172.21.15.105:0/2306403817 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0a8c0a4390 con 0x559b50d765d0 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0a7affd640 1 -- 172.21.15.105:0/2306403817 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c09be10 con 0x559b50d765d0 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0aa466a740 10 client.32453 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0aa466a740 1 -- 172.21.15.105:0/2306403817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559b50abb620 con 0x559b50d765d0 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0a7affd640 1 -- 172.21.15.105:0/2306403817 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a9403a040 con 0x559b50d765d0 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.949+0000 7f0a7affd640 1 -- 172.21.15.105:0/2306403817 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f0a8c097b60 con 0x559b50d765d0 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0aa466a740 10 client.32453 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0aa466a740 10 client.32453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0aa466a740 20 client.32453 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0aa466a740 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b50d8bf80 0x559b50d8e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0aa466a740 4 client.32453 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:46.936 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0aa466a740 1 -- 172.21.15.105:0/2306403817 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b50ce85f0 con 0x559b50d8bf80 2024-03-26T17:58:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0a9affd640 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b50d8bf80 0x559b50d8e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0a7affd640 10 client.32453 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.950+0000 7f0a9affd640 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b50d8bf80 0x559b50d8e3d0 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84a5193ed3daeb8 server_cookie=9fa6579ec4ee2044 in_seq=0 out_seq=0 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0a7affd640 1 -- 172.21.15.105:0/2306403817 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559b50ce85f0 con 0x559b50d8bf80 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0a7affd640 10 client.32453 handle_command_reply: tid=0 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 client.32453 shutdown 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 -- 172.21.15.105:0/2306403817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 msgr2=0x559b50d80420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d80420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c09bb10 tx=0x7f0a8c09ba10 comp rx=0 tx=0).stop 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 -- 172.21.15.105:0/2306403817 shutdown_connections 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b50d8bf80 0x559b50d8e3d0 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d80960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 --2- 172.21.15.105:0/2306403817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d80420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0a7affd640 0 client.32453 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:46.938 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 -- 172.21.15.105:0/2306403817 >> 172.21.15.105:0/2306403817 conn(0x559b50d71b40 msgr2=0x559b50d70df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 -- 172.21.15.105:0/2306403817 shutdown_connections 2024-03-26T17:58:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 1 -- 172.21.15.105:0/2306403817 wait complete. 2024-03-26T17:58:46.939 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.952+0000 7f0aa466a740 20 client.32453 trim_cache size 0 max 16384 2024-03-26T17:58:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0aa466a740 1 Processor -- start 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0aa466a740 1 -- start start 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d765d0 0x559b50d808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d76fc0 0x559b50d87930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d7c130 0x559b50d780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0aa466a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1aae20 con 0x559b50d76fc0 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0aa466a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1ac960 con 0x559b50d7c130 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0aa466a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1ab4f0 con 0x559b50d765d0 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d765d0 0x559b50d808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d76fc0 0x559b50d87930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0a9bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d7c130 0x559b50d780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d765d0 0x559b50d808c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42310/0 (socket says 172.21.15.105:42310) 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0a9bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d7c130 0x559b50d780d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49350/0 (socket says 172.21.15.105:49350) 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0a9b7fe640 1 -- 172.21.15.105:0/1503423749 learned_addr learned my addr 172.21.15.105:0/1503423749 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.977+0000 7f0a9affd640 1 --2- 172.21.15.105:0/1503423749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d76fc0 0x559b50d87930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37032/0 (socket says 172.21.15.105:37032) 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a9affd640 1 -- 172.21.15.105:0/1503423749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d765d0 msgr2=0x559b50d808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a9affd640 1 --2- 172.21.15.105:0/1503423749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d765d0 0x559b50d808c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a9affd640 1 -- 172.21.15.105:0/1503423749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d7c130 msgr2=0x559b50d780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a9affd640 1 --2- 172.21.15.105:0/1503423749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d7c130 0x559b50d780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a9affd640 1 -- 172.21.15.105:0/1503423749 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b50d8ea50 con 0x559b50d76fc0 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a9affd640 1 --2- 172.21.15.105:0/1503423749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d76fc0 0x559b50d87930 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x559b50d8ec50 tx=0x7f0a90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fd05348acdb5db server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a817fa640 1 -- 172.21.15.105:0/1503423749 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9002f070 con 0x559b50d76fc0 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a817fa640 1 -- 172.21.15.105:0/1503423749 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a90016070 con 0x559b50d76fc0 2024-03-26T17:58:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.978+0000 7f0a817fa640 1 -- 172.21.15.105:0/1503423749 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90003ba0 con 0x559b50d76fc0 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 -- 172.21.15.105:0/1503423749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d76fc0 msgr2=0x559b50d87930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1503423749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d76fc0 0x559b50d87930 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x559b50d8ec50 tx=0x7f0a90014000 comp rx=0 tx=0).stop 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 -- 172.21.15.105:0/1503423749 shutdown_connections 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1503423749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d7c130 0x559b50d780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1503423749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d76fc0 0x559b50d87930 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1503423749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d765d0 0x559b50d808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 -- 172.21.15.105:0/1503423749 >> 172.21.15.105:0/1503423749 conn(0x559b50d71b40 msgr2=0x559b50d779c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 -- 172.21.15.105:0/1503423749 shutdown_connections 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.979+0000 7f0aa466a740 1 -- 172.21.15.105:0/1503423749 wait complete. 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0aa466a740 1 -- start start 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0aa466a740 1 Processor -- start 2024-03-26T17:58:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d85490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0aa466a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0aa466a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1ac290 con 0x559b50d7c130 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0aa466a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1abbc0 con 0x559b50d765d0 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0aa466a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1a8e00 con 0x559b50d76fc0 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0aa466a740 10 client.0 _ll_register_callbacks cb 0x7f0a8c097a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.980+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d869e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37048/0 (socket says 172.21.15.105:37048) 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 -- 172.21.15.105:0/1985842419 learned_addr learned my addr 172.21.15.105:0/1985842419 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d85490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 -- 172.21.15.105:0/1985842419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 msgr2=0x559b50d859d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d859d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 -- 172.21.15.105:0/1985842419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 msgr2=0x559b50d85490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d85490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 -- 172.21.15.105:0/1985842419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b50d728b0 con 0x559b50d7c130 2024-03-26T17:58:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9affd640 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d859d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a9bfff640 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d869e0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x559b50d77820 tx=0x7f0a94003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425b60db70fb5af7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.981+0000 7f0a7affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.985+0000 7f0a7affd640 1 -- 172.21.15.105:0/1985842419 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94048040 con 0x559b50d7c130 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.985+0000 7f0a7affd640 1 -- 172.21.15.105:0/1985842419 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0a94004040 con 0x559b50d7c130 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.985+0000 7f0aa466a740 1 -- 172.21.15.105:0/1985842419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a90016070 con 0x559b50d7c130 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0a7affd640 1 -- 172.21.15.105:0/1985842419 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0a94039040 con 0x559b50d7c130 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0aa466a740 10 client.22776 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0aa466a740 1 -- 172.21.15.105:0/1985842419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559b50df25b0 con 0x559b50d7c130 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0a7affd640 1 -- 172.21.15.105:0/1985842419 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a90016070 con 0x559b50d7c130 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0a7affd640 1 -- 172.21.15.105:0/1985842419 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f0a940344d0 con 0x559b50d7c130 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0aa466a740 10 client.22776 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0aa466a740 10 client.22776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0aa466a740 20 client.22776 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b50df2aa0 0x559b50df4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0aa466a740 4 client.22776 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:58:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0aa466a740 1 -- 172.21.15.105:0/1985842419 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b50ce2940 con 0x559b50df2aa0 2024-03-26T17:58:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.986+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b50df2aa0 0x559b50df4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.987+0000 7f0a9b7fe640 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b50df2aa0 0x559b50df4e90 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84a74bb2f41ace5b server_cookie=40468b212d976311 in_seq=0 out_seq=0 2024-03-26T17:58:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:46.987+0000 7f0a7affd640 10 client.22776 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0a7affd640 1 -- 172.21.15.105:0/1985842419 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b50ce2940 con 0x559b50df2aa0 2024-03-26T17:58:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0a7affd640 10 client.22776 handle_command_reply: tid=0 2024-03-26T17:58:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 client.22776 shutdown 2024-03-26T17:58:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 -- 172.21.15.105:0/1985842419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 msgr2=0x559b50d869e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d869e0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x559b50d77820 tx=0x7f0a94003d50 comp rx=0 tx=0).stop 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 -- 172.21.15.105:0/1985842419 shutdown_connections 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559b50df2aa0 0x559b50df4e90 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559b50d7c130 0x559b50d869e0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559b50d76fc0 0x559b50d859d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0a7affd640 0 client.22776 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 --2- 172.21.15.105:0/1985842419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559b50d765d0 0x559b50d85490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 -- 172.21.15.105:0/1985842419 >> 172.21.15.105:0/1985842419 conn(0x559b50d71b40 msgr2=0x559b50d78250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 -- 172.21.15.105:0/1985842419 shutdown_connections 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 1 -- 172.21.15.105:0/1985842419 wait complete. 2024-03-26T17:58:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.170+0000 7f0aa466a740 20 client.22776 trim_cache size 0 max 16384 2024-03-26T17:58:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:58:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:58:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:58:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 -- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0a840a31b0 msgr2=0x7f0a840a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0a840a31b0 0x7f0a840a5670 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c19b0f0 tx=0x7f0a90005f90 comp rx=0 tx=0).stop 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 -- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c103b20 msgr2=0x7f0a9c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c103b20 0x7f0a9c199be0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9c0 tx=0x7f0a8c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 -- 172.21.15.105:0/3232332979 shutdown_connections 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0a840a31b0 0x7f0a840a5670 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a9c1056d0 0x7f0a9c1a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0a9c104d10 0x7f0a9c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 --2- 172.21.15.105:0/3232332979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a9c103b20 0x7f0a9c199be0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 -- 172.21.15.105:0/3232332979 >> 172.21.15.105:0/3232332979 conn(0x7f0a9c0ff0b0 msgr2=0x7f0a9c107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 -- 172.21.15.105:0/3232332979 shutdown_connections 2024-03-26T17:58:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:58:47.171+0000 7f0aa1a87640 1 -- 172.21.15.105:0/3232332979 wait complete. 2024-03-26T17:58:47.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:46 smithi144 ceph-mon[36095]: pgmap v964: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 3.5 MiB/s wr, 200 op/s 2024-03-26T17:58:47.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:46 smithi105 ceph-mon[32280]: pgmap v964: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 3.5 MiB/s wr, 200 op/s 2024-03-26T17:58:47.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:46 smithi135 ceph-mon[35636]: pgmap v964: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 3.5 MiB/s wr, 200 op/s 2024-03-26T17:58:47.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59223 7.13 MB/sec execute 330 sec latency 212.286 ms 2024-03-26T17:58:48.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59362 7.13 MB/sec execute 331 sec latency 123.736 ms 2024-03-26T17:58:49.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:48 smithi144 ceph-mon[36095]: pgmap v965: 97 pgs: 97 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 4.5 MiB/s wr, 226 op/s 2024-03-26T17:58:49.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:48 smithi105 ceph-mon[32280]: pgmap v965: 97 pgs: 97 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 4.5 MiB/s wr, 226 op/s 2024-03-26T17:58:49.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:48 smithi135 ceph-mon[35636]: pgmap v965: 97 pgs: 97 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 4.5 MiB/s wr, 226 op/s 2024-03-26T17:58:49.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59460 7.12 MB/sec execute 332 sec latency 126.738 ms 2024-03-26T17:58:50.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:58:50.167 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 -- 172.21.15.105:0/3225001225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4781029b0 msgr2=0x7fd478102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 --2- 172.21.15.105:0/3225001225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4781029b0 0x7fd478102e30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd460009900 tx=0x7fd46002bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 -- 172.21.15.105:0/3225001225 shutdown_connections 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 --2- 172.21.15.105:0/3225001225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd478103370 0x7fd478109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 --2- 172.21.15.105:0/3225001225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4781029b0 0x7fd478102e30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 --2- 172.21.15.105:0/3225001225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4781017c0 0x7fd478101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 -- 172.21.15.105:0/3225001225 >> 172.21.15.105:0/3225001225 conn(0x7fd4780fcd30 msgr2=0x7fd4780ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 -- 172.21.15.105:0/3225001225 shutdown_connections 2024-03-26T17:58:50.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.326+0000 7fd47759e640 1 -- 172.21.15.105:0/3225001225 wait complete. 2024-03-26T17:58:50.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd47759e640 1 Processor -- start 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd47759e640 1 -- start start 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd47759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4781017c0 0x7fd478195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd47759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4781029b0 0x7fd478195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd47759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd478103370 0x7fd47819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd47759e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd47810b110 con 0x7fd4781029b0 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd47759e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd47810af90 con 0x7fd4781017c0 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd47759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd47810b290 con 0x7fd478103370 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4781029b0 0x7fd478195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4781029b0 0x7fd478195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37070/0 (socket says 172.21.15.105:37070) 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd46ffff640 1 -- 172.21.15.105:0/3544763799 learned_addr learned my addr 172.21.15.105:0/3544763799 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.327+0000 7fd476d9d640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd478103370 0x7fd47819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd47659c640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4781017c0 0x7fd478195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd476d9d640 1 -- 172.21.15.105:0/3544763799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4781017c0 msgr2=0x7fd478195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd476d9d640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4781017c0 0x7fd478195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd476d9d640 1 -- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4781029b0 msgr2=0x7fd478195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd476d9d640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4781029b0 0x7fd478195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd476d9d640 1 -- 172.21.15.105:0/3544763799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47819d3b0 con 0x7fd478103370 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd46ffff640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4781029b0 0x7fd478195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd476d9d640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd478103370 0x7fd47819cd40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd47000ed90 tx=0x7fd470012490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd46f7fe640 1 -- 172.21.15.105:0/3544763799 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47001c070 con 0x7fd478103370 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd47759e640 1 -- 172.21.15.105:0/3544763799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47819d6a0 con 0x7fd478103370 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd46f7fe640 1 -- 172.21.15.105:0/3544763799 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd470005020 con 0x7fd478103370 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd46f7fe640 1 -- 172.21.15.105:0/3544763799 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470018660 con 0x7fd478103370 2024-03-26T17:58:50.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.328+0000 7fd47759e640 1 -- 172.21.15.105:0/3544763799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47819dbf0 con 0x7fd478103370 2024-03-26T17:58:50.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.331+0000 7fd46f7fe640 1 -- 172.21.15.105:0/3544763799 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd470017070 con 0x7fd478103370 2024-03-26T17:58:50.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47fbeb740 1 Processor -- start 2024-03-26T17:58:50.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47fbeb740 1 -- start start 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e359ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e35ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e361a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55865e2be980 con 0x55865e359aa0 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55865e266be0 con 0x55865e35f600 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55865e2ba930 con 0x55865e35a490 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd46f7fe640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4640a3020 0x7fd4640a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e359ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e359ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37082/0 (socket says 172.21.15.105:37082) 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd46ffff640 1 -- 172.21.15.105:0/891593898 learned_addr learned my addr 172.21.15.105:0/891593898 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.332+0000 7fd47659c640 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e35ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd476d9d640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4640a3020 0x7fd4640a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd46f7fe640 1 -- 172.21.15.105:0/3544763799 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd4700d32d0 con 0x7fd478103370 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd47659c640 1 -- 172.21.15.105:0/891593898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 msgr2=0x55865e361a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd47659c640 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e361a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd47659c640 1 -- 172.21.15.105:0/891593898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 msgr2=0x55865e359ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd47659c640 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e359ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd47659c640 1 -- 172.21.15.105:0/891593898 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865e09c7d0 con 0x55865e35a490 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd46ffff640 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e359ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd476d9d640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4640a3020 0x7fd4640a54e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd47009a400 tx=0x7fd4700ca620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.333+0000 7fd47659c640 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e35ef90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd468009bd0 tx=0x7fd468011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88bd119ac6ab7df6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.334+0000 7fd46cff9640 1 -- 172.21.15.105:0/891593898 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468004230 con 0x55865e35a490 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.334+0000 7fd46cff9640 1 -- 172.21.15.105:0/891593898 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd468009e40 con 0x55865e35a490 2024-03-26T17:58:50.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.334+0000 7fd46cff9640 1 -- 172.21.15.105:0/891593898 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468019790 con 0x55865e35a490 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 -- 172.21.15.105:0/891593898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 msgr2=0x55865e35ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e35ef90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd468009bd0 tx=0x7fd468011810 comp rx=0 tx=0).stop 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 -- 172.21.15.105:0/891593898 shutdown_connections 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e361a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e35ef90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/891593898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e359ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 -- 172.21.15.105:0/891593898 >> 172.21.15.105:0/891593898 conn(0x55865e355010 msgr2=0x55865e355410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 -- 172.21.15.105:0/891593898 shutdown_connections 2024-03-26T17:58:50.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 -- 172.21.15.105:0/891593898 wait complete. 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 -- start start 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.335+0000 7fd47fbeb740 1 Processor -- start 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e363d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e364280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e36b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55865e3625b0 con 0x55865e359aa0 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55865e27a160 con 0x55865e35f600 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55865e362730 con 0x55865e35a490 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd47fbeb740 10 client.0 _ll_register_callbacks cb 0x55865e2b6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e363d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e363d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37098/0 (socket says 172.21.15.105:37098) 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 -- 172.21.15.105:0/20105548 learned_addr learned my addr 172.21.15.105:0/20105548 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd47659c640 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e364280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e36b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 -- 172.21.15.105:0/20105548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 msgr2=0x55865e364280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e364280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 -- 172.21.15.105:0/20105548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 msgr2=0x55865e36b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e36b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 -- 172.21.15.105:0/20105548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865dfd23f0 con 0x55865e359aa0 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd46ffff640 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e363d40 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fd460009340 tx=0x7fd460000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6b704c07891d96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.336+0000 7fd45dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd45dffb640 1 -- 172.21.15.105:0/20105548 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460005580 con 0x55865e359aa0 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd45dffb640 1 -- 172.21.15.105:0/20105548 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd4600046c0 con 0x55865e359aa0 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 1 -- 172.21.15.105:0/20105548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55865e2cbec0 con 0x55865e359aa0 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd45dffb640 1 -- 172.21.15.105:0/20105548 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd460054050 con 0x55865e359aa0 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 10 client.22788 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 1 -- 172.21.15.105:0/20105548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55865df920f0 con 0x55865e359aa0 2024-03-26T17:58:50.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd45dffb640 1 -- 172.21.15.105:0/20105548 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55865e2cbec0 con 0x55865e359aa0 2024-03-26T17:58:50.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd45dffb640 1 -- 172.21.15.105:0/20105548 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fd460040040 con 0x55865e359aa0 2024-03-26T17:58:50.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 10 client.22788 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:50.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 10 client.22788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:50.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 20 client.22788 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:50.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55865e36fa60 0x55865e371ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 4 client.22788 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:50.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.337+0000 7fd47fbeb740 1 -- 172.21.15.105:0/20105548 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55865e2c6210 con 0x55865e36fa60 2024-03-26T17:58:50.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.338+0000 7fd47659c640 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55865e36fa60 0x55865e371ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.338+0000 7fd45dffb640 10 client.22788 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:50.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.338+0000 7fd47659c640 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55865e36fa60 0x55865e371ec0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445fbd86c1a9e91c server_cookie=27dd98aaeb2e4e9f in_seq=0 out_seq=0 2024-03-26T17:58:50.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd45dffb640 1 -- 172.21.15.105:0/20105548 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55865e2c6210 con 0x55865e36fa60 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd45dffb640 10 client.22788 handle_command_reply: tid=0 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 client.22788 shutdown 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 -- 172.21.15.105:0/20105548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 msgr2=0x55865e363d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e363d40 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fd460009340 tx=0x7fd460000bc0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 -- 172.21.15.105:0/20105548 shutdown_connections 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55865e36fa60 0x55865e371ec0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e36b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35a490 0x55865e364280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/20105548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e359aa0 0x55865e363d40 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 -- 172.21.15.105:0/20105548 >> 172.21.15.105:0/20105548 conn(0x55865e355010 msgr2=0x55865e354210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:50.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd45dffb640 0 client.22788 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:50.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 -- 172.21.15.105:0/20105548 shutdown_connections 2024-03-26T17:58:50.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 1 -- 172.21.15.105:0/20105548 wait complete. 2024-03-26T17:58:50.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.344+0000 7fd47fbeb740 20 client.22788 trim_cache size 0 max 16384 2024-03-26T17:58:50.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.372+0000 7fd47fbeb740 1 Processor -- start 2024-03-26T17:58:50.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.372+0000 7fd47fbeb740 1 -- start start 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.372+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e359aa0 0x55865e363ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.372+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 0x55865e36b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.372+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e364220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.372+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781a6990 con 0x55865e35a490 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.372+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781a84d0 con 0x55865e35f600 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.372+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781a7060 con 0x55865e359aa0 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e359aa0 0x55865e363ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e359aa0 0x55865e363ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42374/0 (socket says 172.21.15.105:42374) 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 -- 172.21.15.105:0/2687744414 learned_addr learned my addr 172.21.15.105:0/2687744414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:50.357 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 --2- 172.21.15.105:0/2687744414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e364220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 -- 172.21.15.105:0/2687744414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 msgr2=0x55865e364220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 --2- 172.21.15.105:0/2687744414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e364220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 -- 172.21.15.105:0/2687744414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 msgr2=0x55865e36b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 --2- 172.21.15.105:0/2687744414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 0x55865e36b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 -- 172.21.15.105:0/2687744414 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865e355a60 con 0x55865e359aa0 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.373+0000 7fd46ffff640 1 --2- 172.21.15.105:0/2687744414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e359aa0 0x55865e363ce0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd460005c00 tx=0x7fd46002c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed44c47b466f8211 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.374+0000 7fd45ffff640 1 -- 172.21.15.105:0/2687744414 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460003e80 con 0x55865e359aa0 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.374+0000 7fd45ffff640 1 -- 172.21.15.105:0/2687744414 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4600046c0 con 0x55865e359aa0 2024-03-26T17:58:50.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.374+0000 7fd45ffff640 1 -- 172.21.15.105:0/2687744414 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4600426e0 con 0x55865e359aa0 2024-03-26T17:58:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.374+0000 7fd47fbeb740 1 -- 172.21.15.105:0/2687744414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e359aa0 msgr2=0x55865e363ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.374+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/2687744414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e359aa0 0x55865e363ce0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd460005c00 tx=0x7fd46002c7e0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 -- 172.21.15.105:0/2687744414 shutdown_connections 2024-03-26T17:58:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/2687744414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e35f600 0x55865e364220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/2687744414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 0x55865e36b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/2687744414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e359aa0 0x55865e363ce0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 -- 172.21.15.105:0/2687744414 >> 172.21.15.105:0/2687744414 conn(0x55865e355010 msgr2=0x55865e350070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 -- 172.21.15.105:0/2687744414 shutdown_connections 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 -- 172.21.15.105:0/2687744414 wait complete. 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 -- start start 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 Processor -- start 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e359aa0 0x55865e368290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 0x55865e3687d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35f600 0x55865e369790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781a7e00 con 0x55865e35a490 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781a7730 con 0x55865e359aa0 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4781a4970 con 0x55865e35f600 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.375+0000 7fd47fbeb740 10 client.0 _ll_register_callbacks cb 0x7fd460009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:50.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e359aa0 0x55865e368290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd47659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 0x55865e3687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e359aa0 0x55865e368290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49418/0 (socket says 172.21.15.105:49418) 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 -- 172.21.15.105:0/1254755245 learned_addr learned my addr 172.21.15.105:0/1254755245 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35f600 0x55865e369790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 -- 172.21.15.105:0/1254755245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35f600 msgr2=0x55865e369790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35f600 0x55865e369790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 -- 172.21.15.105:0/1254755245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 msgr2=0x55865e3687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 0x55865e3687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 -- 172.21.15.105:0/1254755245 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865e369d80 con 0x55865e359aa0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd47659c640 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 0x55865e3687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd46ffff640 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e359aa0 0x55865e368290 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd460004590 tx=0x7fd460000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb39814b2bcc21a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd45dffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd45dffb640 1 -- 172.21.15.105:0/1254755245 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460004870 con 0x55865e359aa0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd45dffb640 1 -- 172.21.15.105:0/1254755245 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd460004a10 con 0x55865e359aa0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.376+0000 7fd47fbeb740 1 -- 172.21.15.105:0/1254755245 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4600046c0 con 0x55865e359aa0 2024-03-26T17:58:50.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd45dffb640 1 -- 172.21.15.105:0/1254755245 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd460004c10 con 0x55865e359aa0 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47fbeb740 10 client.32474 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47fbeb740 1 -- 172.21.15.105:0/1254755245 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55865e36b090 con 0x55865e359aa0 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd45dffb640 1 -- 172.21.15.105:0/1254755245 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4600046c0 con 0x55865e359aa0 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd45dffb640 1 -- 172.21.15.105:0/1254755245 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fd460066440 con 0x55865e359aa0 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47fbeb740 10 client.32474 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47fbeb740 10 client.32474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47fbeb740 20 client.32474 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55865e3d5f80 0x55865e3d83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47fbeb740 4 client.32474 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47fbeb740 1 -- 172.21.15.105:0/1254755245 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55865e243540 con 0x55865e3d5f80 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.377+0000 7fd47659c640 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55865e3d5f80 0x55865e3d83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.378+0000 7fd45dffb640 10 client.32474 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:50.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.378+0000 7fd47659c640 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55865e3d5f80 0x55865e3d83a0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=605dea71ea1f8466 server_cookie=ae7d60ed98da4d53 in_seq=0 out_seq=0 2024-03-26T17:58:50.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd45dffb640 1 -- 172.21.15.105:0/1254755245 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55865e243540 con 0x55865e3d5f80 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd45dffb640 10 client.32474 handle_command_reply: tid=0 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 client.32474 shutdown 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 -- 172.21.15.105:0/1254755245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e359aa0 msgr2=0x55865e368290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e359aa0 0x55865e368290 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd460004590 tx=0x7fd460000bc0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 -- 172.21.15.105:0/1254755245 shutdown_connections 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55865e3d5f80 0x55865e3d83a0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55865e35f600 0x55865e369790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd45dffb640 0 client.32474 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55865e35a490 0x55865e3687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 --2- 172.21.15.105:0/1254755245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55865e359aa0 0x55865e368290 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 -- 172.21.15.105:0/1254755245 >> 172.21.15.105:0/1254755245 conn(0x55865e355010 msgr2=0x55865e363ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 -- 172.21.15.105:0/1254755245 shutdown_connections 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 1 -- 172.21.15.105:0/1254755245 wait complete. 2024-03-26T17:58:50.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.380+0000 7fd47fbeb740 20 client.32474 trim_cache size 0 max 16384 2024-03-26T17:58:50.367 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:58:50.367 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:58:50.367 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:58:50.367 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 -- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4640a3020 msgr2=0x7fd4640a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4640a3020 0x7fd4640a54e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd47009a400 tx=0x7fd4700ca620 comp rx=0 tx=0).stop 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 -- 172.21.15.105:0/3544763799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd478103370 msgr2=0x7fd47819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd478103370 0x7fd47819cd40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fd47000ed90 tx=0x7fd470012490 comp rx=0 tx=0).stop 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 -- 172.21.15.105:0/3544763799 shutdown_connections 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd4640a3020 0x7fd4640a54e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd478103370 0x7fd47819cd40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd4781029b0 0x7fd478195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 --2- 172.21.15.105:0/3544763799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4781017c0 0x7fd478195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 -- 172.21.15.105:0/3544763799 >> 172.21.15.105:0/3544763799 conn(0x7fd4780fcd30 msgr2=0x7fd478105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 -- 172.21.15.105:0/3544763799 shutdown_connections 2024-03-26T17:58:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.381+0000 7fd47759e640 1 -- 172.21.15.105:0/3544763799 wait complete. 2024-03-26T17:58:50.381 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:58:50.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:58:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.529+0000 7f487088c640 1 -- 172.21.15.105:0/1438964329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c102390 msgr2=0x7f486c108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.529+0000 7f487088c640 1 --2- 172.21.15.105:0/1438964329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c102390 0x7f486c108c20 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4864009ca0 tx=0x7f486402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.530+0000 7f487088c640 1 -- 172.21.15.105:0/1438964329 shutdown_connections 2024-03-26T17:58:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.530+0000 7f487088c640 1 --2- 172.21.15.105:0/1438964329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c102390 0x7f486c108c20 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.530+0000 7f487088c640 1 --2- 172.21.15.105:0/1438964329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019d0 0x7f486c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.530+0000 7f487088c640 1 --2- 172.21.15.105:0/1438964329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c1007e0 0x7f486c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.530+0000 7f487088c640 1 -- 172.21.15.105:0/1438964329 >> 172.21.15.105:0/1438964329 conn(0x7f486c0fbd50 msgr2=0x7f486c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:50.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.530+0000 7f487088c640 1 -- 172.21.15.105:0/1438964329 shutdown_connections 2024-03-26T17:58:50.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.530+0000 7f487088c640 1 -- 172.21.15.105:0/1438964329 wait complete. 2024-03-26T17:58:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f487088c640 1 Processor -- start 2024-03-26T17:58:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f487088c640 1 -- start start 2024-03-26T17:58:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f487088c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007e0 0x7f486c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f487088c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019d0 0x7f486c196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f486a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007e0 0x7f486c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f487088c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102390 0x7f486c19df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f486a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007e0 0x7f486c1969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37124/0 (socket says 172.21.15.105:37124) 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f4869d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019d0 0x7f486c196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f4869d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019d0 0x7f486c196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42396/0 (socket says 172.21.15.105:42396) 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f486a59c640 1 -- 172.21.15.105:0/2425682105 learned_addr learned my addr 172.21.15.105:0/2425682105 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f486c10c2f0 con 0x7f486c1007e0 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f486c10c170 con 0x7f486c102390 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f486c10c470 con 0x7f486c1019d0 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.531+0000 7f486ad9d640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102390 0x7f486c19df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f4869d9b640 1 -- 172.21.15.105:0/2425682105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102390 msgr2=0x7f486c19df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f4869d9b640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102390 0x7f486c19df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f4869d9b640 1 -- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007e0 msgr2=0x7f486c1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f4869d9b640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007e0 0x7f486c1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f4869d9b640 1 -- 172.21.15.105:0/2425682105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c19e5e0 con 0x7f486c1019d0 2024-03-26T17:58:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f4869d9b640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019d0 0x7f486c196f00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f486000c970 tx=0x7f486000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f485b7fe640 1 -- 172.21.15.105:0/2425682105 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486000bcb0 con 0x7f486c1019d0 2024-03-26T17:58:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c068a80 con 0x7f486c1019d0 2024-03-26T17:58:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c069020 con 0x7f486c1019d0 2024-03-26T17:58:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f485b7fe640 1 -- 172.21.15.105:0/2425682105 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f486000be50 con 0x7f486c1019d0 2024-03-26T17:58:50.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.532+0000 7f485b7fe640 1 -- 172.21.15.105:0/2425682105 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486000e7e0 con 0x7f486c1019d0 2024-03-26T17:58:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.533+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484c0052c0 con 0x7f486c1019d0 2024-03-26T17:58:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.535+0000 7f485b7fe640 1 -- 172.21.15.105:0/2425682105 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f486000e980 con 0x7f486c1019d0 2024-03-26T17:58:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.536+0000 7f485b7fe640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48540a3020 0x7f48540a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.536+0000 7f485b7fe640 1 -- 172.21.15.105:0/2425682105 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f48600cf930 con 0x7f486c1019d0 2024-03-26T17:58:50.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.536+0000 7f486a59c640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48540a3020 0x7f48540a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:50.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.537+0000 7f486a59c640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48540a3020 0x7f48540a54e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f485c0058c0 tx=0x7f485c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:50.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.542+0000 7f485b7fe640 1 -- 172.21.15.105:0/2425682105 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4860014030 con 0x7f486c1019d0 2024-03-26T17:58:50.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59544 7.11 MB/sec execute 333 sec latency 124.259 ms 2024-03-26T17:58:50.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.872+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f484c0055b0 con 0x7f486c1019d0 2024-03-26T17:58:50.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.873+0000 7f485b7fe640 1 -- 172.21.15.105:0/2425682105 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11941 (secure 0 0 0) 0x7f486009c6c0 con 0x7f486c1019d0 2024-03-26T17:58:50.857 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:58:50.857 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:58:34.693267+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_32441":{"gid":32441,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1622534523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1622534523},{"type":"v1","addr":"172.21.15.135:6835","nonce":1622534523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:58:50.857 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 41 2024-03-26T17:58:50.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.876+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48540a3020 msgr2=0x7f48540a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.876+0000 7f487088c640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48540a3020 0x7f48540a54e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f485c0058c0 tx=0x7f485c005f90 comp rx=0 tx=0).stop 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.876+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019d0 msgr2=0x7f486c196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019d0 0x7f486c196f00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f486000c970 tx=0x7f486000ce30 comp rx=0 tx=0).stop 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 shutdown_connections 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48540a3020 0x7f48540a54e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102390 0x7f486c19df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019d0 0x7f486c196f00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 --2- 172.21.15.105:0/2425682105 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007e0 0x7f486c1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 >> 172.21.15.105:0/2425682105 conn(0x7f486c0fbd50 msgr2=0x7f486c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 shutdown_connections 2024-03-26T17:58:50.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:50.877+0000 7f487088c640 1 -- 172.21.15.105:0/2425682105 wait complete. 2024-03-26T17:58:50.877 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:58:51.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.040+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/3703530226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec104cf0 msgr2=0x7f2eec105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.040+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/3703530226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec104cf0 0x7f2eec105170 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009900 tx=0x7f2ee002bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:51.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.041+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/3703530226 shutdown_connections 2024-03-26T17:58:51.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.041+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/3703530226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2eec1056b0 0x7f2eec10bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.041+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/3703530226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec104cf0 0x7f2eec105170 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.041+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/3703530226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec103b00 0x7f2eec103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.041+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/3703530226 >> 172.21.15.105:0/3703530226 conn(0x7f2eec0ff070 msgr2=0x7f2eec1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.041+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/3703530226 shutdown_connections 2024-03-26T17:58:51.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.041+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/3703530226 wait complete. 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.041+0000 7f2ef1f5b640 1 Processor -- start 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2ef1f5b640 1 -- start start 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2ef1f5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec103b00 0x7f2eec199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2ef1f5b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2eec104cf0 0x7f2eec19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2ef1f5b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec1056b0 0x7f2eec1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2ef1f5b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f680 con 0x7f2eec103b00 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2ef1f5b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f500 con 0x7f2eec104cf0 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2ef1f5b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec10f800 con 0x7f2eec1056b0 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2eeb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec103b00 0x7f2eec199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2eeb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec103b00 0x7f2eec199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37140/0 (socket says 172.21.15.105:37140) 2024-03-26T17:58:51.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/878231171 learned_addr learned my addr 172.21.15.105:0/878231171 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2eebfff640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec1056b0 0x7f2eec1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2eec104cf0 0x7f2eec19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2eebfff640 1 -- 172.21.15.105:0/878231171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2eec104cf0 msgr2=0x7f2eec19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.042+0000 7f2eebfff640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2eec104cf0 0x7f2eec19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2eebfff640 1 -- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec103b00 msgr2=0x7f2eec199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2eebfff640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec103b00 0x7f2eec199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2eebfff640 1 -- 172.21.15.105:0/878231171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec1a19b0 con 0x7f2eec1056b0 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec103b00 0x7f2eec199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2eebfff640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec1056b0 0x7f2eec1a1310 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009fd0 tx=0x7f2ee400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2ee8ff9640 1 -- 172.21.15.105:0/878231171 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400ce20 con 0x7f2eec1056b0 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2ee8ff9640 1 -- 172.21.15.105:0/878231171 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee4005020 con 0x7f2eec1056b0 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/878231171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec075d60 con 0x7f2eec1056b0 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2ee8ff9640 1 -- 172.21.15.105:0/878231171 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4010690 con 0x7f2eec1056b0 2024-03-26T17:58:51.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.043+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/878231171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec076260 con 0x7f2eec1056b0 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.046+0000 7f2ef4c29740 1 Processor -- start 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.046+0000 7f2ee8ff9640 1 -- 172.21.15.105:0/878231171 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2ee40051d0 con 0x7f2eec1056b0 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.046+0000 7f2ef4c29740 1 -- start start 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 0x55ecbb1d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ecbb12f970 con 0x55ecbb1cb420 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ecbb0d7d70 con 0x55ecbb1caa30 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ecbb12b920 con 0x55ecbb1d0590 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49468/0 (socket says 172.21.15.105:49468) 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeaffd640 1 -- 172.21.15.105:0/3050987545 learned_addr learned my addr 172.21.15.105:0/3050987545 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 0x55ecbb1d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/3050987545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 msgr2=0x55ecbb1d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 0x55ecbb1d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/3050987545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 msgr2=0x55ecbb1cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/3050987545 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecbae433f0 con 0x55ecbb1cb420 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.047+0000 7f2ee8ff9640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ed40a2f20 0x7f2ed40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.048+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ed40a2f20 0x7f2ed40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.048+0000 7f2ee8ff9640 1 -- 172.21.15.105:0/878231171 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f2ee40cfb80 con 0x7f2eec1056b0 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.048+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1cff20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0058c0 tx=0x7f2edc011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bd39c87147171e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.048+0000 7f2ed1ffb640 1 -- 172.21.15.105:0/3050987545 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc011310 con 0x55ecbb1cb420 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.048+0000 7f2ed1ffb640 1 -- 172.21.15.105:0/3050987545 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2edc0035f0 con 0x55ecbb1cb420 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.048+0000 7f2ed1ffb640 1 -- 172.21.15.105:0/3050987545 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0044e0 con 0x55ecbb1cb420 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3050987545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 msgr2=0x55ecbb1cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ed40a2f20 0x7f2ed40a53e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0005ab0 tx=0x7f2ee002c610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1cff20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0058c0 tx=0x7f2edc011950 comp rx=0 tx=0).stop 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3050987545 shutdown_connections 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 0x55ecbb1d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1cff20 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3050987545 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3050987545 >> 172.21.15.105:0/3050987545 conn(0x55ecbb1c5fa0 msgr2=0x55ecbb1c63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3050987545 shutdown_connections 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3050987545 wait complete. 2024-03-26T17:58:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.049+0000 7f2ef4c29740 1 -- start start 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2ef4c29740 1 Processor -- start 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 0x55ecbb1dc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ecbb1d3590 con 0x55ecbb1cb420 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ecbb0eb310 con 0x55ecbb1caa30 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ecbb1d3710 con 0x55ecbb1d0590 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2eeb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2ef4c29740 10 client.0 _ll_register_callbacks cb 0x55ecbb126c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2eeb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49474/0 (socket says 172.21.15.105:49474) 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.050+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/900067326 learned_addr learned my addr 172.21.15.105:0/900067326 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eebfff640 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 0x55ecbb1dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/900067326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 msgr2=0x55ecbb1dc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 0x55ecbb1dc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 msgr2=0x55ecbb1d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/900067326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecbaf0d7d0 con 0x55ecbb1caa30 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4ee0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01b040 tx=0x7f2edc009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50138852a468f815 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2ecaffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2ecaffd640 1 -- 172.21.15.105:0/900067326 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc029020 con 0x55ecbb1caa30 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.051+0000 7f2ef4c29740 1 -- 172.21.15.105:0/900067326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2edc0035f0 con 0x55ecbb1caa30 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ecaffd640 1 -- 172.21.15.105:0/900067326 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f2edc004200 con 0x55ecbb1caa30 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ecaffd640 1 -- 172.21.15.105:0/900067326 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2edc034050 con 0x55ecbb1caa30 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ef4c29740 10 client.32492 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ef4c29740 1 -- 172.21.15.105:0/900067326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ecbb1df820 con 0x55ecbb1caa30 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ecaffd640 1 -- 172.21.15.105:0/900067326 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2edc0035f0 con 0x55ecbb1caa30 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ecaffd640 1 -- 172.21.15.105:0/900067326 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f2edc0048f0 con 0x55ecbb1caa30 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ef4c29740 10 client.32492 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ef4c29740 10 client.32492 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:58:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ef4c29740 20 client.32492 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ecbb1e0fb0 0x55ecbb1e33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ef4c29740 4 client.32492 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.052+0000 7f2ef4c29740 1 -- 172.21.15.105:0/900067326 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecbb13ceb0 con 0x55ecbb1e0fb0 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.053+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ecbb1e0fb0 0x55ecbb1e33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.053+0000 7f2ecaffd640 10 client.32492 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.053+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ecbb1e0fb0 0x55ecbb1e33a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4859c62eda5e65c0 server_cookie=61f1ce108c59aefb in_seq=0 out_seq=0 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ecaffd640 1 -- 172.21.15.105:0/900067326 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ecbb13ceb0 con 0x55ecbb1e0fb0 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ecaffd640 10 client.32492 handle_command_reply: tid=0 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 client.32492 shutdown 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 -- 172.21.15.105:0/900067326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 msgr2=0x55ecbb1d4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4ee0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01b040 tx=0x7f2edc009dd0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 -- 172.21.15.105:0/900067326 shutdown_connections 2024-03-26T17:58:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ecbb1e0fb0 0x55ecbb1e33a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1d0590 0x55ecbb1dc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ecaffd640 0 client.32492 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:58:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/900067326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4ee0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 -- 172.21.15.105:0/900067326 >> 172.21.15.105:0/900067326 conn(0x55ecbb1c5fa0 msgr2=0x55ecbb1c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 -- 172.21.15.105:0/900067326 shutdown_connections 2024-03-26T17:58:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 1 -- 172.21.15.105:0/900067326 wait complete. 2024-03-26T17:58:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.054+0000 7f2ef4c29740 20 client.32492 trim_cache size 0 max 16384 2024-03-26T17:58:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.073+0000 7f2ef4c29740 1 Processor -- start 2024-03-26T17:58:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.073+0000 7f2ef4c29740 1 -- start start 2024-03-26T17:58:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1dc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 0x55ecbb1d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1a6b00 con 0x55ecbb1cb420 2024-03-26T17:58:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1a8640 con 0x55ecbb1d0590 2024-03-26T17:58:51.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1a71d0 con 0x55ecbb1caa30 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1dc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1dc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37158/0 (socket says 172.21.15.105:37158) 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeaffd640 1 -- 172.21.15.105:0/2226044663 learned_addr learned my addr 172.21.15.105:0/2226044663 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eebfff640 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 0x55ecbb1d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeaffd640 1 -- 172.21.15.105:0/2226044663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 msgr2=0x55ecbb1d4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeaffd640 1 -- 172.21.15.105:0/2226044663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 msgr2=0x55ecbb1d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 0x55ecbb1d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.074+0000 7f2eeaffd640 1 -- 172.21.15.105:0/2226044663 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecbb1cfb10 con 0x55ecbb1cb420 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.075+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1dc380 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0036040 tx=0x7f2ee002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe2a6184a855a7f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.075+0000 7f2ed17fa640 1 -- 172.21.15.105:0/2226044663 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee005c700 con 0x55ecbb1cb420 2024-03-26T17:58:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.075+0000 7f2ed17fa640 1 -- 172.21.15.105:0/2226044663 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee0062070 con 0x55ecbb1cb420 2024-03-26T17:58:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.075+0000 7f2ed17fa640 1 -- 172.21.15.105:0/2226044663 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee00046c0 con 0x55ecbb1cb420 2024-03-26T17:58:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 -- 172.21.15.105:0/2226044663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 msgr2=0x55ecbb1dc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1dc380 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0036040 tx=0x7f2ee002c7e0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 -- 172.21.15.105:0/2226044663 shutdown_connections 2024-03-26T17:58:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 0x55ecbb1d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1dc380 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/2226044663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 -- 172.21.15.105:0/2226044663 >> 172.21.15.105:0/2226044663 conn(0x55ecbb1c5fa0 msgr2=0x55ecbb1c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 -- 172.21.15.105:0/2226044663 shutdown_connections 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 -- 172.21.15.105:0/2226044663 wait complete. 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.076+0000 7f2ef4c29740 1 -- start start 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2ef4c29740 1 Processor -- start 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2ef4c29740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 0x55ecbb1db380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1a7f70 con 0x55ecbb1cb420 2024-03-26T17:58:51.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1a78a0 con 0x55ecbb1d0590 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2ef4c29740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1a4ae0 con 0x55ecbb1caa30 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2ef4c29740 10 client.0 _ll_register_callbacks cb 0x7f2edc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeb7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42434/0 (socket says 172.21.15.105:42434) 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/3440641637 learned_addr learned my addr 172.21.15.105:0/3440641637 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eebfff640 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 0x55ecbb1db380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/3440641637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 msgr2=0x55ecbb1db380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 0x55ecbb1db380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/3440641637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 msgr2=0x55ecbb1da370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1da370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.077+0000 7f2eeb7fe640 1 -- 172.21.15.105:0/3440641637 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecbb1db970 con 0x55ecbb1caa30 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2eeb7fe640 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d9e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2edc004d60 tx=0x7f2edc003c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbef3d5e5fc605e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2ecaffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2ecaffd640 1 -- 172.21.15.105:0/3440641637 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc029040 con 0x55ecbb1caa30 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3440641637 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee0062070 con 0x55ecbb1caa30 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2ecaffd640 1 -- 172.21.15.105:0/3440641637 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f2edc0053e0 con 0x55ecbb1caa30 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2ecaffd640 1 -- 172.21.15.105:0/3440641637 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2edc048050 con 0x55ecbb1caa30 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2ef4c29740 10 client.32482 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3440641637 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ecbb247280 con 0x55ecbb1caa30 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.078+0000 7f2ecaffd640 1 -- 172.21.15.105:0/3440641637 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ee0062070 con 0x55ecbb1caa30 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2ecaffd640 1 -- 172.21.15.105:0/3440641637 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f2edc025480 con 0x55ecbb1caa30 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2ef4c29740 10 client.32482 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2ef4c29740 10 client.32482 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2ef4c29740 20 client.32482 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ecbb247ca0 0x55ecbb24a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2ef4c29740 4 client.32482 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3440641637 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecbb137200 con 0x55ecbb247ca0 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ecbb247ca0 0x55ecbb24a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.079+0000 7f2ecaffd640 10 client.32482 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2eeaffd640 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ecbb247ca0 0x55ecbb24a0c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ccc2ca845df3ba53 server_cookie=48ebbb1a34e3b4e9 in_seq=0 out_seq=0 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ecaffd640 1 -- 172.21.15.105:0/3440641637 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecbb137200 con 0x55ecbb247ca0 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ecaffd640 10 client.32482 handle_command_reply: tid=0 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 client.32482 shutdown 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3440641637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 msgr2=0x55ecbb1d9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d9e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2edc004d60 tx=0x7f2edc003c70 comp rx=0 tx=0).stop 2024-03-26T17:58:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3440641637 shutdown_connections 2024-03-26T17:58:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55ecbb247ca0 0x55ecbb24a0c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ecbb1d0590 0x55ecbb1db380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ecbb1cb420 0x55ecbb1da370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 --2- 172.21.15.105:0/3440641637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecbb1caa30 0x55ecbb1d9e30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ecaffd640 0 client.32482 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:58:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.080+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3440641637 >> 172.21.15.105:0/3440641637 conn(0x55ecbb1c5fa0 msgr2=0x55ecbb1c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.081+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3440641637 shutdown_connections 2024-03-26T17:58:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.081+0000 7f2ef4c29740 1 -- 172.21.15.105:0/3440641637 wait complete. 2024-03-26T17:58:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.081+0000 7f2ef4c29740 20 client.32482 trim_cache size 0 max 16384 2024-03-26T17:58:51.069 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.081+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ed40a2f20 msgr2=0x7f2ed40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.081+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ed40a2f20 0x7f2ed40a53e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0005ab0 tx=0x7f2ee002c610 comp rx=0 tx=0).stop 2024-03-26T17:58:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.081+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/878231171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec1056b0 msgr2=0x7f2eec1a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.081+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec1056b0 0x7f2eec1a1310 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009fd0 tx=0x7f2ee400ef10 comp rx=0 tx=0).stop 2024-03-26T17:58:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.082+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/878231171 shutdown_connections 2024-03-26T17:58:51.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.082+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2ed40a2f20 0x7f2ed40a53e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.082+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eec1056b0 0x7f2eec1a1310 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.082+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2eec104cf0 0x7f2eec19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.082+0000 7f2ef1f5b640 1 --2- 172.21.15.105:0/878231171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2eec103b00 0x7f2eec199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.082+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/878231171 >> 172.21.15.105:0/878231171 conn(0x7f2eec0ff070 msgr2=0x7f2eec108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.082+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/878231171 shutdown_connections 2024-03-26T17:58:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.082+0000 7f2ef1f5b640 1 -- 172.21.15.105:0/878231171 wait complete. 2024-03-26T17:58:51.079 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:58:51.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:50 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2425682105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:58:51.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 -- 172.21.15.105:0/36356573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574107650 msgr2=0x7f7574109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 --2- 172.21.15.105:0/36356573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574107650 0x7f7574109a40 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f756c009ca0 tx=0x7f756c02c0f0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 -- 172.21.15.105:0/36356573 shutdown_connections 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 --2- 172.21.15.105:0/36356573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574107650 0x7f7574109a40 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 --2- 172.21.15.105:0/36356573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7574104d20 0x7f7574107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 --2- 172.21.15.105:0/36356573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75741023f0 0x7f75741047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 -- 172.21.15.105:0/36356573 >> 172.21.15.105:0/36356573 conn(0x7f75740fbd60 msgr2=0x7f75740fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 -- 172.21.15.105:0/36356573 shutdown_connections 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.242+0000 7f757ab53640 1 -- 172.21.15.105:0/36356573 wait complete. 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f757ab53640 1 Processor -- start 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f757ab53640 1 -- start start 2024-03-26T17:58:51.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f757ab53640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75741023f0 0x7f7574101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f757ab53640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574104d20 0x7f75741003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f757ab53640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7574107650 0x7f75741008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f757ab53640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f757410d420 con 0x7f7574104d20 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f757ab53640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f757410d2a0 con 0x7f75741023f0 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f757ab53640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f757410d5a0 con 0x7f7574107650 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f756bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574104d20 0x7f75741003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f75788ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75741023f0 0x7f7574101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f756bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574104d20 0x7f75741003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37190/0 (socket says 172.21.15.105:37190) 2024-03-26T17:58:51.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f756bfff640 1 -- 172.21.15.105:0/499726024 learned_addr learned my addr 172.21.15.105:0/499726024 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f75790f0640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7574107650 0x7f75741008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f756bfff640 1 -- 172.21.15.105:0/499726024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7574107650 msgr2=0x7f75741008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f756bfff640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7574107650 0x7f75741008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f756bfff640 1 -- 172.21.15.105:0/499726024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75741023f0 msgr2=0x7f7574101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f756bfff640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75741023f0 0x7f7574101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f756bfff640 1 -- 172.21.15.105:0/499726024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574101170 con 0x7f7574104d20 2024-03-26T17:58:51.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.243+0000 7f75788ef640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75741023f0 0x7f7574101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.244+0000 7f756bfff640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574104d20 0x7f75741003a0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9f0 tx=0x7f756400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.244+0000 7f7569ffb640 1 -- 172.21.15.105:0/499726024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400ccc0 con 0x7f7574104d20 2024-03-26T17:58:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.244+0000 7f7569ffb640 1 -- 172.21.15.105:0/499726024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75640054d0 con 0x7f7574104d20 2024-03-26T17:58:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.244+0000 7f7569ffb640 1 -- 172.21.15.105:0/499726024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564010430 con 0x7f7574104d20 2024-03-26T17:58:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.244+0000 7f757ab53640 1 -- 172.21.15.105:0/499726024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75741a1c50 con 0x7f7574104d20 2024-03-26T17:58:51.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.244+0000 7f757ab53640 1 -- 172.21.15.105:0/499726024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75741a21a0 con 0x7f7574104d20 2024-03-26T17:58:51.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.247+0000 7f7569ffb640 1 -- 172.21.15.105:0/499726024 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7564005020 con 0x7f7574104d20 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.247+0000 7f757d81c740 1 Processor -- start 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f757d81c740 1 -- start start 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f757d81c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f16e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f757d81c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f1bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f757d81c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f1ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f757d81c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557213e7b970 con 0x557213f16a30 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f757d81c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557213e23d70 con 0x557213f1c590 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f757d81c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557213e77920 con 0x557213f17420 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f7569ffb640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f755c0a2fa0 0x7f755c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f756bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f1ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f75790f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f16e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f75790f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f16e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37206/0 (socket says 172.21.15.105:37206) 2024-03-26T17:58:51.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f75790f0640 1 -- 172.21.15.105:0/3594915990 learned_addr learned my addr 172.21.15.105:0/3594915990 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f75788ef640 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f1bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f7569ffb640 1 -- 172.21.15.105:0/499726024 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f75640cf780 con 0x7f7574104d20 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f756bfff640 1 -- 172.21.15.105:0/3594915990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 msgr2=0x557213f1bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f756bfff640 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f1bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f756bfff640 1 -- 172.21.15.105:0/3594915990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 msgr2=0x557213f16e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.248+0000 7f756bfff640 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f16e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f756bfff640 1 -- 172.21.15.105:0/3594915990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557213b8f3f0 con 0x557213f1c590 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f75788ef640 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f1bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f75790f0640 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f16e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f75790f0640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f755c0a2fa0 0x7f755c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f756bfff640 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f1ea20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75640cd290 tx=0x7f756406eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbfb506410cbcaa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f755affd640 1 -- 172.21.15.105:0/3594915990 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400ce60 con 0x557213f1c590 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f755affd640 1 -- 172.21.15.105:0/3594915990 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7564005310 con 0x557213f1c590 2024-03-26T17:58:51.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f75790f0640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f755c0a2fa0 0x7f755c0a5460 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x557213f17c80 tx=0x7f756c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.249+0000 7f755affd640 1 -- 172.21.15.105:0/3594915990 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756409ffc0 con 0x557213f1c590 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.250+0000 7f757d81c740 1 -- 172.21.15.105:0/3594915990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 msgr2=0x557213f1ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.250+0000 7f757d81c740 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f1ea20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75640cd290 tx=0x7f756406eba0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.250+0000 7f757d81c740 1 -- 172.21.15.105:0/3594915990 shutdown_connections 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.250+0000 7f757d81c740 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f1ea20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.250+0000 7f757d81c740 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f1bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.250+0000 7f757d81c740 1 --2- 172.21.15.105:0/3594915990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f16e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.250+0000 7f757d81c740 1 -- 172.21.15.105:0/3594915990 >> 172.21.15.105:0/3594915990 conn(0x557213f11fa0 msgr2=0x557213f123a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.251+0000 7f757d81c740 1 -- 172.21.15.105:0/3594915990 shutdown_connections 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.251+0000 7f757d81c740 1 -- 172.21.15.105:0/3594915990 wait complete. 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.251+0000 7f757d81c740 1 -- start start 2024-03-26T17:58:51.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.251+0000 7f757d81c740 1 Processor -- start 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f757d81c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f20ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f757d81c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f17420 0x557213f21530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f757d81c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f1c590 0x557213f28500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f757d81c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557213f1f540 con 0x557213f16a30 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f20ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f757d81c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557213e37310 con 0x557213f17420 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f757d81c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557213f1f6c0 con 0x557213f1c590 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f20ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37220/0 (socket says 172.21.15.105:37220) 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 -- 172.21.15.105:0/809206643 learned_addr learned my addr 172.21.15.105:0/809206643 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f757d81c740 10 client.0 _ll_register_callbacks cb 0x557213e72c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f756bfff640 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f17420 0x557213f21530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75790f0640 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f1c590 0x557213f28500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 -- 172.21.15.105:0/809206643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f1c590 msgr2=0x557213f28500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f1c590 0x557213f28500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 -- 172.21.15.105:0/809206643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f17420 msgr2=0x557213f21530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f17420 0x557213f21530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 -- 172.21.15.105:0/809206643 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557213c597d0 con 0x557213f16a30 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f756bfff640 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f17420 0x557213f21530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f75788ef640 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f20ff0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7570012d20 tx=0x7f757000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a61078ca72147d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f7553fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f7553fff640 1 -- 172.21.15.105:0/809206643 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700120d0 con 0x557213f16a30 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f7553fff640 1 -- 172.21.15.105:0/809206643 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f7570004020 con 0x557213f16a30 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.252+0000 7f757d81c740 1 -- 172.21.15.105:0/809206643 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557213e88eb0 con 0x557213f16a30 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.253+0000 7f7553fff640 1 -- 172.21.15.105:0/809206643 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7570012270 con 0x557213f16a30 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.253+0000 7f757d81c740 10 client.22824 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.253+0000 7f757d81c740 1 -- 172.21.15.105:0/809206643 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557213f2b920 con 0x557213f16a30 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.253+0000 7f7553fff640 1 -- 172.21.15.105:0/809206643 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x557213e88eb0 con 0x557213f16a30 2024-03-26T17:58:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.253+0000 7f7553fff640 1 -- 172.21.15.105:0/809206643 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f757001c470 con 0x557213f16a30 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.254+0000 7f757d81c740 10 client.22824 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.254+0000 7f757d81c740 10 client.22824 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.254+0000 7f757d81c740 20 client.22824 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.254+0000 7f757d81c740 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557213f2d020 0x557213f2f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.254+0000 7f757d81c740 4 client.22824 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.254+0000 7f757d81c740 1 -- 172.21.15.105:0/809206643 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557213e83200 con 0x557213f2d020 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.254+0000 7f756bfff640 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557213f2d020 0x557213f2f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.254+0000 7f7553fff640 10 client.22824 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:58:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.255+0000 7f756bfff640 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557213f2d020 0x557213f2f4c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b73de60ba49f6c9f server_cookie=2b2bbbb3dbb3979 in_seq=0 out_seq=0 2024-03-26T17:58:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f7553fff640 1 -- 172.21.15.105:0/809206643 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557213e83200 con 0x557213f2d020 2024-03-26T17:58:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f7553fff640 10 client.22824 handle_command_reply: tid=0 2024-03-26T17:58:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 client.22824 shutdown 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 -- 172.21.15.105:0/809206643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 msgr2=0x557213f20ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f20ff0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7570012d20 tx=0x7f757000f790 comp rx=0 tx=0).stop 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 -- 172.21.15.105:0/809206643 shutdown_connections 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557213f2d020 0x557213f2f4c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f1c590 0x557213f28500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f17420 0x557213f21530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 --2- 172.21.15.105:0/809206643 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f16a30 0x557213f20ff0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 -- 172.21.15.105:0/809206643 >> 172.21.15.105:0/809206643 conn(0x557213f11fa0 msgr2=0x557213f111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f7553fff640 0 client.22824 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 -- 172.21.15.105:0/809206643 shutdown_connections 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 1 -- 172.21.15.105:0/809206643 wait complete. 2024-03-26T17:58:51.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.256+0000 7f757d81c740 20 client.22824 trim_cache size 0 max 16384 2024-03-26T17:58:51.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f757d81c740 1 Processor -- start 2024-03-26T17:58:51.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f757d81c740 1 -- start start 2024-03-26T17:58:51.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f757d81c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f16a30 0x557213f21310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f757d81c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f17420 0x557213f284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f757d81c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f11040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f757d81c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75741ab090 con 0x557213f17420 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f757d81c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75741acbd0 con 0x557213f1c590 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f757d81c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75741ab760 con 0x557213f16a30 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f756bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f17420 0x557213f284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f756bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f17420 0x557213f284c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37232/0 (socket says 172.21.15.105:37232) 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f756bfff640 1 -- 172.21.15.105:0/1396558036 learned_addr learned my addr 172.21.15.105:0/1396558036 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f75788ef640 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f16a30 0x557213f21310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.277+0000 7f75790f0640 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f11040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f756bfff640 1 -- 172.21.15.105:0/1396558036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f16a30 msgr2=0x557213f21310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f756bfff640 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f16a30 0x557213f21310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f756bfff640 1 -- 172.21.15.105:0/1396558036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 msgr2=0x557213f11040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f756bfff640 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f11040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f756bfff640 1 -- 172.21.15.105:0/1396558036 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557213f12a10 con 0x557213f17420 2024-03-26T17:58:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f75788ef640 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f16a30 0x557213f21310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f756bfff640 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f17420 0x557213f284c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f756400c6a0 tx=0x7f7564004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65364d7c0b5a89e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f755a7fc640 1 -- 172.21.15.105:0/1396558036 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640a4880 con 0x557213f17420 2024-03-26T17:58:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f755a7fc640 1 -- 172.21.15.105:0/1396558036 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f756409ffc0 con 0x557213f17420 2024-03-26T17:58:51.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.278+0000 7f755a7fc640 1 -- 172.21.15.105:0/1396558036 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640babe0 con 0x557213f17420 2024-03-26T17:58:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 -- 172.21.15.105:0/1396558036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f17420 msgr2=0x557213f284c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f17420 0x557213f284c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f756400c6a0 tx=0x7f7564004960 comp rx=0 tx=0).stop 2024-03-26T17:58:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 -- 172.21.15.105:0/1396558036 shutdown_connections 2024-03-26T17:58:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f1c590 0x557213f11040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f17420 0x557213f284c0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 --2- 172.21.15.105:0/1396558036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f16a30 0x557213f21310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 -- 172.21.15.105:0/1396558036 >> 172.21.15.105:0/1396558036 conn(0x557213f11fa0 msgr2=0x557213f10930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 -- 172.21.15.105:0/1396558036 shutdown_connections 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 -- 172.21.15.105:0/1396558036 wait complete. 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 -- start start 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.279+0000 7f757d81c740 1 Processor -- start 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f757d81c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f16a30 0x557213f259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f757d81c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f25ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f757d81c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f1c590 0x557213f26f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f757d81c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f75741ac500 con 0x557213f1c590 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f757d81c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75741abe30 con 0x557213f16a30 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f757d81c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75741a9070 con 0x557213f17420 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f757d81c740 10 client.0 _ll_register_callbacks cb 0x7f7570009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75788ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f16a30 0x557213f259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f1c590 0x557213f26f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f1c590 0x557213f26f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37238/0 (socket says 172.21.15.105:37238) 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 -- 172.21.15.105:0/1716936124 learned_addr learned my addr 172.21.15.105:0/1716936124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f756bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f25ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 -- 172.21.15.105:0/1716936124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 msgr2=0x557213f25ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f25ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 -- 172.21.15.105:0/1716936124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f16a30 msgr2=0x557213f259b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f16a30 0x557213f259b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 -- 172.21.15.105:0/1716936124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557213f274f0 con 0x557213f1c590 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75788ef640 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f16a30 0x557213f259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f75790f0640 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f1c590 0x557213f26f00 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f756c032420 tx=0x7f756c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b82ad9ef2ba2a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f7553fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.280+0000 7f7553fff640 1 -- 172.21.15.105:0/1716936124 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c053020 con 0x557213f1c590 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f7553fff640 1 -- 172.21.15.105:0/1716936124 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f756c003970 con 0x557213f1c590 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 1 -- 172.21.15.105:0/1716936124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f756409ffc0 con 0x557213f1c590 2024-03-26T17:58:51.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f7553fff640 1 -- 172.21.15.105:0/1716936124 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f756c039040 con 0x557213f1c590 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 10 client.22830 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 1 -- 172.21.15.105:0/1716936124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557213f28860 con 0x557213f1c590 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f7553fff640 1 -- 172.21.15.105:0/1716936124 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f756409ffc0 con 0x557213f1c590 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f7553fff640 1 -- 172.21.15.105:0/1716936124 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f756c03f040 con 0x557213f1c590 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 10 client.22830 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 10 client.22830 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 20 client.22830 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557213f93c20 0x557213f96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 4 client.22830 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.281+0000 7f757d81c740 1 -- 172.21.15.105:0/1716936124 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557213e006a0 con 0x557213f93c20 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.282+0000 7f75788ef640 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557213f93c20 0x557213f96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.282+0000 7f7553fff640 10 client.22830 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:58:51.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.282+0000 7f75788ef640 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557213f93c20 0x557213f96040 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f211d7271a582c6 server_cookie=e4b587e20105aa10 in_seq=0 out_seq=0 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.282+0000 7f7553fff640 1 -- 172.21.15.105:0/1716936124 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557213e006a0 con 0x557213f93c20 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.282+0000 7f7553fff640 10 client.22830 handle_command_reply: tid=0 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.282+0000 7f757d81c740 1 client.22830 shutdown 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 -- 172.21.15.105:0/1716936124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f1c590 msgr2=0x557213f26f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f1c590 0x557213f26f00 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f756c032420 tx=0x7f756c009360 comp rx=0 tx=0).stop 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 -- 172.21.15.105:0/1716936124 shutdown_connections 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x557213f93c20 0x557213f96040 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557213f1c590 0x557213f26f00 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557213f17420 0x557213f25ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 --2- 172.21.15.105:0/1716936124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557213f16a30 0x557213f259b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f7553fff640 0 client.22830 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 -- 172.21.15.105:0/1716936124 >> 172.21.15.105:0/1716936124 conn(0x557213f11fa0 msgr2=0x557213f111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 -- 172.21.15.105:0/1716936124 shutdown_connections 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 1 -- 172.21.15.105:0/1716936124 wait complete. 2024-03-26T17:58:51.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.283+0000 7f757d81c740 20 client.22830 trim_cache size 0 max 16384 2024-03-26T17:58:51.272 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:51.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 -- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f755c0a2fa0 msgr2=0x7f755c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f755c0a2fa0 0x7f755c0a5460 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x557213f17c80 tx=0x7f756c02e040 comp rx=0 tx=0).stop 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 -- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574104d20 msgr2=0x7f75741003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574104d20 0x7f75741003a0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f756400e9f0 tx=0x7f756400eeb0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 -- 172.21.15.105:0/499726024 shutdown_connections 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f755c0a2fa0 0x7f755c0a5460 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7574107650 0x7f75741008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7574104d20 0x7f75741003a0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 --2- 172.21.15.105:0/499726024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f75741023f0 0x7f7574101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 -- 172.21.15.105:0/499726024 >> 172.21.15.105:0/499726024 conn(0x7f75740fbd60 msgr2=0x7f7574102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 -- 172.21.15.105:0/499726024 shutdown_connections 2024-03-26T17:58:51.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.284+0000 7f757ab53640 1 -- 172.21.15.105:0/499726024 wait complete. 2024-03-26T17:58:51.280 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:58:51.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:50 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2425682105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:58:51.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:50 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2425682105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:58:51.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.441+0000 7f307eedc640 1 -- 172.21.15.105:0/411830496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3078102390 msgr2=0x7f3078108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.441+0000 7f307eedc640 1 --2- 172.21.15.105:0/411830496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3078102390 0x7f3078108c20 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f3070009ca0 tx=0x7f307002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.441+0000 7f307eedc640 1 -- 172.21.15.105:0/411830496 shutdown_connections 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.441+0000 7f307eedc640 1 --2- 172.21.15.105:0/411830496 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3078102390 0x7f3078108c20 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.441+0000 7f307eedc640 1 --2- 172.21.15.105:0/411830496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30781019d0 0x7f3078101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.441+0000 7f307eedc640 1 --2- 172.21.15.105:0/411830496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30781007e0 0x7f3078100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.441+0000 7f307eedc640 1 -- 172.21.15.105:0/411830496 >> 172.21.15.105:0/411830496 conn(0x7f30780fbd70 msgr2=0x7f30780fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.442+0000 7f307eedc640 1 -- 172.21.15.105:0/411830496 shutdown_connections 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.442+0000 7f307eedc640 1 -- 172.21.15.105:0/411830496 wait complete. 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.442+0000 7f307eedc640 1 Processor -- start 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.442+0000 7f307eedc640 1 -- start start 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307eedc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30781007e0 0x7f3078197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307eedc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30781019d0 0x7f3078198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30781007e0 0x7f3078197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307eedc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3078102390 0x7f307819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30781007e0 0x7f3078197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37256/0 (socket says 172.21.15.105:37256) 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 -- 172.21.15.105:0/709921779 learned_addr learned my addr 172.21.15.105:0/709921779 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f307810c310 con 0x7f30781007e0 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f307810c190 con 0x7f30781019d0 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f307810c490 con 0x7f3078102390 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f3077fff640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30781019d0 0x7f3078198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307d479640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3078102390 0x7f307819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 -- 172.21.15.105:0/709921779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3078102390 msgr2=0x7f307819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3078102390 0x7f307819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 -- 172.21.15.105:0/709921779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30781019d0 msgr2=0x7f3078198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30781019d0 0x7f3078198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 -- 172.21.15.105:0/709921779 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307819f690 con 0x7f30781007e0 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307d479640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3078102390 0x7f307819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f307cc78640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30781007e0 0x7f3078197b10 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e9c0 tx=0x7f306c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f3075ffb640 1 -- 172.21.15.105:0/709921779 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00cd80 con 0x7f30781007e0 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.443+0000 7f3075ffb640 1 -- 172.21.15.105:0/709921779 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f306c0035f0 con 0x7f30781007e0 2024-03-26T17:58:51.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.444+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307819f980 con 0x7f30781007e0 2024-03-26T17:58:51.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.444+0000 7f3075ffb640 1 -- 172.21.15.105:0/709921779 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c010690 con 0x7f30781007e0 2024-03-26T17:58:51.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.444+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307819fed0 con 0x7f30781007e0 2024-03-26T17:58:51.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.446+0000 7f3075ffb640 1 -- 172.21.15.105:0/709921779 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f306c0037a0 con 0x7f30781007e0 2024-03-26T17:58:51.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.447+0000 7f3075ffb640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30680a2f20 0x7f30680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.447+0000 7f3077fff640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30680a2f20 0x7f30680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.447+0000 7f3075ffb640 1 -- 172.21.15.105:0/709921779 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f306c0cf850 con 0x7f30781007e0 2024-03-26T17:58:51.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3081ba1740 1 Processor -- start 2024-03-26T17:58:51.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3081ba1740 1 -- start start 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3077fff640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30680a2f20 0x7f30680a53e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3078198f80 tx=0x7f3064005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558728999a30 0x558728999e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55872899a420 0x55872899ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 0x5587289a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3081ba1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5587288fe970 con 0x558728999a30 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3081ba1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5587288a6d70 con 0x55872899f590 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3081ba1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5587288fa920 con 0x55872899a420 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f307cc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558728999a30 0x558728999e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3077fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55872899a420 0x55872899ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f307cc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558728999a30 0x558728999e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37272/0 (socket says 172.21.15.105:37272) 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3077fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55872899a420 0x55872899ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42504/0 (socket says 172.21.15.105:42504) 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f307cc78640 1 -- 172.21.15.105:0/3270633497 learned_addr learned my addr 172.21.15.105:0/3270633497 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f307d479640 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 0x5587289a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3077fff640 1 -- 172.21.15.105:0/3270633497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 msgr2=0x5587289a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3077fff640 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 0x5587289a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3077fff640 1 -- 172.21.15.105:0/3270633497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558728999a30 msgr2=0x558728999e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3077fff640 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558728999a30 0x558728999e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f3077fff640 1 -- 172.21.15.105:0/3270633497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587286123f0 con 0x55872899a420 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.448+0000 7f307cc78640 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558728999a30 0x558728999e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.449+0000 7f3077fff640 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55872899a420 0x55872899ef20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f30640118d0 tx=0x7f3064014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83fc5bbe4a12687 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.449+0000 7f3062ffd640 1 -- 172.21.15.105:0/3270633497 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064005020 con 0x55872899a420 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.449+0000 7f3062ffd640 1 -- 172.21.15.105:0/3270633497 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30640047a0 con 0x55872899a420 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.449+0000 7f3062ffd640 1 -- 172.21.15.105:0/3270633497 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306401e680 con 0x55872899a420 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 -- 172.21.15.105:0/3270633497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55872899a420 msgr2=0x55872899ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55872899a420 0x55872899ef20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f30640118d0 tx=0x7f3064014000 comp rx=0 tx=0).stop 2024-03-26T17:58:51.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 -- 172.21.15.105:0/3270633497 shutdown_connections 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 0x5587289a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55872899a420 0x55872899ef20 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 --2- 172.21.15.105:0/3270633497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558728999a30 0x558728999e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 -- 172.21.15.105:0/3270633497 >> 172.21.15.105:0/3270633497 conn(0x558728994fa0 msgr2=0x5587289953a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 -- 172.21.15.105:0/3270633497 shutdown_connections 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 -- 172.21.15.105:0/3270633497 wait complete. 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 -- start start 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.450+0000 7f3081ba1740 1 Processor -- start 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558728999a30 0x5587289a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f3081ba1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5587289a2540 con 0x55872899f590 2024-03-26T17:58:51.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f3081ba1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5587288ba310 con 0x55872899a420 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f3081ba1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5587289a26c0 con 0x558728999a30 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307d479640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f3081ba1740 10 client.0 _ll_register_callbacks cb 0x5587288f5c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307d479640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37288/0 (socket says 172.21.15.105:37288) 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307cc78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558728999a30 0x5587289a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307d479640 1 -- 172.21.15.105:0/558629126 learned_addr learned my addr 172.21.15.105:0/558629126 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f3077fff640 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307cc78640 1 -- 172.21.15.105:0/558629126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 msgr2=0x5587289a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307cc78640 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307cc78640 1 -- 172.21.15.105:0/558629126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 msgr2=0x5587289ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307cc78640 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.451+0000 7f307cc78640 1 -- 172.21.15.105:0/558629126 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587286dc7d0 con 0x558728999a30 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f307cc78640 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558728999a30 0x5587289a3f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f306c0a4420 tx=0x7f306c09b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c92f7d59f4e2a8df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3057fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:58:51.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3057fff640 1 -- 172.21.15.105:0/558629126 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c0a0090 con 0x558728999a30 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3057fff640 1 -- 172.21.15.105:0/558629126 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f306c0a44c0 con 0x558728999a30 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 1 -- 172.21.15.105:0/558629126 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30640047a0 con 0x558728999a30 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3057fff640 1 -- 172.21.15.105:0/558629126 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f306c0a4950 con 0x558728999a30 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 10 client.32512 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 1 -- 172.21.15.105:0/558629126 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5587289ae870 con 0x558728999a30 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3057fff640 1 -- 172.21.15.105:0/558629126 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30640047a0 con 0x558728999a30 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3057fff640 1 -- 172.21.15.105:0/558629126 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f306c09ae00 con 0x558728999a30 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 10 client.32512 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 10 client.32512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 20 client.32512 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5587289b0230 0x5587289b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 4 client.32512 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:51.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.452+0000 7f3081ba1740 1 -- 172.21.15.105:0/558629126 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55872890beb0 con 0x5587289b0230 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.453+0000 7f3077fff640 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5587289b0230 0x5587289b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.453+0000 7f3057fff640 10 client.32512 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.453+0000 7f3077fff640 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5587289b0230 0x5587289b26d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b0fcf27c995721a server_cookie=7154c235a1e34075 in_seq=0 out_seq=0 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3057fff640 1 -- 172.21.15.105:0/558629126 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55872890beb0 con 0x5587289b0230 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3057fff640 10 client.32512 handle_command_reply: tid=0 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 client.32512 shutdown 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 -- 172.21.15.105:0/558629126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558728999a30 msgr2=0x5587289a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558728999a30 0x5587289a3f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f306c0a4420 tx=0x7f306c09b960 comp rx=0 tx=0).stop 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 -- 172.21.15.105:0/558629126 shutdown_connections 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5587289b0230 0x5587289b26d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 --2- 172.21.15.105:0/558629126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558728999a30 0x5587289a3f20 secure :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f306c0a4420 tx=0x7f306c09b960 comp rx=0 tx=0).stop 2024-03-26T17:58:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3081ba1740 1 -- 172.21.15.105:0/558629126 >> 172.21.15.105:0/558629126 conn(0x558728994fa0 msgr2=0x5587289941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.454+0000 7f3057fff640 0 client.32512 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:58:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.455+0000 7f3081ba1740 1 -- 172.21.15.105:0/558629126 shutdown_connections 2024-03-26T17:58:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.455+0000 7f3081ba1740 1 -- 172.21.15.105:0/558629126 wait complete. 2024-03-26T17:58:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.455+0000 7f3081ba1740 20 client.32512 trim_cache size 0 max 16384 2024-03-26T17:58:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3081ba1740 1 Processor -- start 2024-03-26T17:58:51.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3081ba1740 1 -- start start 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x558728993930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3081ba1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f30781a8d50 con 0x55872899f590 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3081ba1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30781aa890 con 0x55872899a420 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3081ba1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30781a9420 con 0x5587289a4070 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3077fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f307d479640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x558728993930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f307cc78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3077fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37298/0 (socket says 172.21.15.105:37298) 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f307d479640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x558728993930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42528/0 (socket says 172.21.15.105:42528) 2024-03-26T17:58:51.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f307cc78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49606/0 (socket says 172.21.15.105:49606) 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.474+0000 7f3077fff640 1 -- 172.21.15.105:0/3016527294 learned_addr learned my addr 172.21.15.105:0/3016527294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f307cc78640 1 -- 172.21.15.105:0/3016527294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 msgr2=0x558728993930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f307cc78640 1 --2- 172.21.15.105:0/3016527294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x558728993930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f307cc78640 1 -- 172.21.15.105:0/3016527294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 msgr2=0x5587289ab3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f307cc78640 1 --2- 172.21.15.105:0/3016527294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f307cc78640 1 -- 172.21.15.105:0/3016527294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558728994b40 con 0x55872899a420 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f3077fff640 1 --2- 172.21.15.105:0/3016527294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f307cc78640 1 --2- 172.21.15.105:0/3016527294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a3b30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f306c0983a0 tx=0x7f306c09b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6115a226a23c3e75 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f30627fc640 1 -- 172.21.15.105:0/3016527294 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c0a0090 con 0x55872899a420 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f30627fc640 1 -- 172.21.15.105:0/3016527294 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f306c07e020 con 0x55872899a420 2024-03-26T17:58:51.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.475+0000 7f30627fc640 1 -- 172.21.15.105:0/3016527294 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c0a44c0 con 0x55872899a420 2024-03-26T17:58:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 -- 172.21.15.105:0/3016527294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 msgr2=0x5587289a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 --2- 172.21.15.105:0/3016527294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a3b30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f306c0983a0 tx=0x7f306c09b9d0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 -- 172.21.15.105:0/3016527294 shutdown_connections 2024-03-26T17:58:51.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 --2- 172.21.15.105:0/3016527294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x558728993930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 --2- 172.21.15.105:0/3016527294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899f590 0x5587289ab3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 --2- 172.21.15.105:0/3016527294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899a420 0x5587289a3b30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 -- 172.21.15.105:0/3016527294 >> 172.21.15.105:0/3016527294 conn(0x558728994fa0 msgr2=0x558728995380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 -- 172.21.15.105:0/3016527294 shutdown_connections 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.476+0000 7f3081ba1740 1 -- 172.21.15.105:0/3016527294 wait complete. 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 1 -- start start 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 1 Processor -- start 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899a420 0x5587289a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 0x5587289a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x5587289a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f30781aa1c0 con 0x55872899a420 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f30781a9af0 con 0x55872899f590 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30781a6d30 con 0x5587289a4070 2024-03-26T17:58:51.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3081ba1740 10 client.0 _ll_register_callbacks cb 0x7f306c0097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f307cc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899a420 0x5587289a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f307cc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899a420 0x5587289a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37314/0 (socket says 172.21.15.105:37314) 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f307cc78640 1 -- 172.21.15.105:0/1296531369 learned_addr learned my addr 172.21.15.105:0/1296531369 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f3077fff640 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 0x5587289a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.477+0000 7f307d479640 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x5587289a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f307cc78640 1 -- 172.21.15.105:0/1296531369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 msgr2=0x5587289a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f307cc78640 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x5587289a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f307cc78640 1 -- 172.21.15.105:0/1296531369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 msgr2=0x5587289a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f307cc78640 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 0x5587289a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f307cc78640 1 -- 172.21.15.105:0/1296531369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587289aa4c0 con 0x55872899a420 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f307cc78640 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899a420 0x5587289a69c0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f306c098910 tx=0x7f306c0a46f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbf96e6f99b8b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f3057fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:51.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f3057fff640 1 -- 172.21.15.105:0/1296531369 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c0a0090 con 0x55872899a420 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f3057fff640 1 -- 172.21.15.105:0/1296531369 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f306c09b5a0 con 0x55872899a420 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.478+0000 7f3081ba1740 1 -- 172.21.15.105:0/1296531369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306c07e020 con 0x55872899a420 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3057fff640 1 -- 172.21.15.105:0/1296531369 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f306c0850d0 con 0x55872899a420 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3081ba1740 10 client.22848 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3081ba1740 1 -- 172.21.15.105:0/1296531369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5587289ab7c0 con 0x55872899a420 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3057fff640 1 -- 172.21.15.105:0/1296531369 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f306c07e020 con 0x55872899a420 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3057fff640 1 -- 172.21.15.105:0/1296531369 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f306c096bd0 con 0x55872899a420 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3081ba1740 10 client.22848 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3081ba1740 10 client.22848 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3081ba1740 20 client.22848 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3081ba1740 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x558728a1aef0 0x558728a1d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3081ba1740 4 client.22848 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.479+0000 7f3081ba1740 1 -- 172.21.15.105:0/1296531369 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558728906200 con 0x558728a1aef0 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3077fff640 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x558728a1aef0 0x558728a1d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3077fff640 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x558728a1aef0 0x558728a1d2e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c273efc80939ffc3 server_cookie=6f007356d1289641 in_seq=0 out_seq=0 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3057fff640 10 client.22848 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3057fff640 1 -- 172.21.15.105:0/1296531369 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558728906200 con 0x558728a1aef0 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3057fff640 10 client.22848 handle_command_reply: tid=0 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3081ba1740 1 client.22848 shutdown 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3081ba1740 1 -- 172.21.15.105:0/1296531369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899a420 msgr2=0x5587289a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3081ba1740 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899a420 0x5587289a69c0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f306c098910 tx=0x7f306c0a46f0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.480+0000 7f3081ba1740 1 -- 172.21.15.105:0/1296531369 shutdown_connections 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3081ba1740 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5587289a4070 0x5587289a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3081ba1740 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x558728a1aef0 0x558728a1d2e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3081ba1740 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55872899f590 0x5587289a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3081ba1740 1 --2- 172.21.15.105:0/1296531369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55872899a420 0x5587289a69c0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3081ba1740 1 -- 172.21.15.105:0/1296531369 >> 172.21.15.105:0/1296531369 conn(0x558728994fa0 msgr2=0x558728998200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3057fff640 0 client.22848 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:58:51.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3081ba1740 1 -- 172.21.15.105:0/1296531369 shutdown_connections 2024-03-26T17:58:51.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3081ba1740 1 -- 172.21.15.105:0/1296531369 wait complete. 2024-03-26T17:58:51.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.481+0000 7f3081ba1740 20 client.22848 trim_cache size 0 max 16384 2024-03-26T17:58:51.469 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30680a2f20 msgr2=0x7f30680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30680a2f20 0x7f30680a53e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3078198f80 tx=0x7f3064005f90 comp rx=0 tx=0).stop 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30781007e0 msgr2=0x7f3078197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30781007e0 0x7f3078197b10 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f306c00e9c0 tx=0x7f306c00ee80 comp rx=0 tx=0).stop 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 shutdown_connections 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f30680a2f20 0x7f30680a53e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3078102390 0x7f307819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f30781019d0 0x7f3078198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 --2- 172.21.15.105:0/709921779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30781007e0 0x7f3078197b10 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 >> 172.21.15.105:0/709921779 conn(0x7f30780fbd70 msgr2=0x7f30780fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 shutdown_connections 2024-03-26T17:58:51.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.482+0000 7f307eedc640 1 -- 172.21.15.105:0/709921779 wait complete. 2024-03-26T17:58:51.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:58:51.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59661 7.11 MB/sec execute 334 sec latency 142.842 ms 2024-03-26T17:58:51.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2744956088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc698102370 msgr2=0x7fc698108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2744956088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc698102370 0x7fc698108c00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fc690009be0 tx=0x7fc690030070 comp rx=0 tx=0).stop 2024-03-26T17:58:51.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2744956088 shutdown_connections 2024-03-26T17:58:51.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2744956088 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc698102370 0x7fc698108c00 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2744956088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6981019b0 0x7fc698101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2744956088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6981007c0 0x7fc698100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2744956088 >> 172.21.15.105:0/2744956088 conn(0x7fc6980fbd50 msgr2=0x7fc6980fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2744956088 shutdown_connections 2024-03-26T17:58:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.633+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2744956088 wait complete. 2024-03-26T17:58:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc69dc4b640 1 Processor -- start 2024-03-26T17:58:51.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc69dc4b640 1 -- start start 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc69dc4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6981007c0 0x7fc6981958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc69dc4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6981019b0 0x7fc698195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc69dc4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc698102370 0x7fc69819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc69dc4b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810b1b0 con 0x7fc6981007c0 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc69dc4b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810b030 con 0x7fc698102370 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc69dc4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810b330 con 0x7fc6981019b0 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc6977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6981007c0 0x7fc6981958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc6977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6981007c0 0x7fc6981958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37336/0 (socket says 172.21.15.105:37336) 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc6977fe640 1 -- 172.21.15.105:0/2137220247 learned_addr learned my addr 172.21.15.105:0/2137220247 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc696ffd640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6981019b0 0x7fc698195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.634+0000 7fc697fff640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc698102370 0x7fc69819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc6977fe640 1 -- 172.21.15.105:0/2137220247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6981019b0 msgr2=0x7fc698195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc6977fe640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6981019b0 0x7fc698195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc6977fe640 1 -- 172.21.15.105:0/2137220247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc698102370 msgr2=0x7fc69819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc6977fe640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc698102370 0x7fc69819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc6977fe640 1 -- 172.21.15.105:0/2137220247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69819d500 con 0x7fc6981007c0 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc6977fe640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6981007c0 0x7fc6981958e0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc688009a50 tx=0x7fc688009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc694ff9640 1 -- 172.21.15.105:0/2137220247 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68801b070 con 0x7fc6981007c0 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc694ff9640 1 -- 172.21.15.105:0/2137220247 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc688016410 con 0x7fc6981007c0 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2137220247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69819d7f0 con 0x7fc6981007c0 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2137220247 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69819dd40 con 0x7fc6981007c0 2024-03-26T17:58:51.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.635+0000 7fc694ff9640 1 -- 172.21.15.105:0/2137220247 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688015610 con 0x7fc6981007c0 2024-03-26T17:58:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.638+0000 7fc694ff9640 1 -- 172.21.15.105:0/2137220247 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc68801e070 con 0x7fc6981007c0 2024-03-26T17:58:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.638+0000 7fc6a0832740 1 Processor -- start 2024-03-26T17:58:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.638+0000 7fc6a0832740 1 -- start start 2024-03-26T17:58:51.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc694ff9640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6800a2f20 0x7fc6800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b040a30 0x55fa0b040e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b045f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b046590 0x55fa0b048a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc6a0832740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa0afa5970 con 0x55fa0b046590 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc6a0832740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fa0af4dd70 con 0x55fa0b040a30 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc6a0832740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa0afa1920 con 0x55fa0b041420 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc694ff9640 1 -- 172.21.15.105:0/2137220247 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fc6880ce7a0 con 0x7fc6981007c0 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc697fff640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6800a2f20 0x7fc6800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b045f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b045f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42568/0 (socket says 172.21.15.105:42568) 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 -- 172.21.15.105:0/3978421503 learned_addr learned my addr 172.21.15.105:0/3978421503 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc6977fe640 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b040a30 0x55fa0b040e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc697fff640 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b046590 0x55fa0b048a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 -- 172.21.15.105:0/3978421503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b040a30 msgr2=0x55fa0b040e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b040a30 0x55fa0b040e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 -- 172.21.15.105:0/3978421503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b046590 msgr2=0x55fa0b048a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b046590 0x55fa0b048a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 -- 172.21.15.105:0/3978421503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0acb93f0 con 0x55fa0b041420 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc697fff640 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b046590 0x55fa0b048a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.639+0000 7fc696ffd640 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b045f20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009c20 tx=0x7fc68c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d51e62062708d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.640+0000 7fc697fff640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6800a2f20 0x7fc6800a53e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc690009370 tx=0x7fc6900374f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.640+0000 7fc67dffb640 1 -- 172.21.15.105:0/3978421503 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c012450 con 0x55fa0b041420 2024-03-26T17:58:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.640+0000 7fc67dffb640 1 -- 172.21.15.105:0/3978421503 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc68c012aa0 con 0x55fa0b041420 2024-03-26T17:58:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.640+0000 7fc67dffb640 1 -- 172.21.15.105:0/3978421503 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00fb10 con 0x55fa0b041420 2024-03-26T17:58:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.641+0000 7fc6a0832740 1 -- 172.21.15.105:0/3978421503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 msgr2=0x55fa0b045f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.641+0000 7fc6a0832740 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b045f20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009c20 tx=0x7fc68c0117b0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.641+0000 7fc6a0832740 1 -- 172.21.15.105:0/3978421503 shutdown_connections 2024-03-26T17:58:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.641+0000 7fc6a0832740 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b046590 0x55fa0b048a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.641+0000 7fc6a0832740 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b045f20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.641+0000 7fc6a0832740 1 --2- 172.21.15.105:0/3978421503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b040a30 0x55fa0b040e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 -- 172.21.15.105:0/3978421503 >> 172.21.15.105:0/3978421503 conn(0x55fa0b03bfa0 msgr2=0x55fa0b03c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 -- 172.21.15.105:0/3978421503 shutdown_connections 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 -- 172.21.15.105:0/3978421503 wait complete. 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 -- start start 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 Processor -- start 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b052520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa0b049540 con 0x55fa0b040a30 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fa0af61310 con 0x55fa0b046590 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.642+0000 7fc6a0832740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa0b0496c0 con 0x55fa0b041420 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6a0832740 10 client.0 _ll_register_callbacks cb 0x55fa0af9cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37354/0 (socket says 172.21.15.105:37354) 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc697fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b052520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 -- 172.21.15.105:0/671574189 learned_addr learned my addr 172.21.15.105:0/671574189 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc696ffd640 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 -- 172.21.15.105:0/671574189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 msgr2=0x55fa0b04b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 -- 172.21.15.105:0/671574189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 msgr2=0x55fa0b052520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b052520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 -- 172.21.15.105:0/671574189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0ad837d0 con 0x55fa0b040a30 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6977fe640 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04b010 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fc688097860 tx=0x7fc6880adda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece94b13dcdeb4ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc676ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc676ffd640 1 -- 172.21.15.105:0/671574189 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688016950 con 0x55fa0b040a30 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc676ffd640 1 -- 172.21.15.105:0/671574189 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fc688099190 con 0x55fa0b040a30 2024-03-26T17:58:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6a0832740 1 -- 172.21.15.105:0/671574189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68c012aa0 con 0x55fa0b040a30 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc676ffd640 1 -- 172.21.15.105:0/671574189 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc68809b590 con 0x55fa0b040a30 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.643+0000 7fc6a0832740 10 client.22860 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc6a0832740 1 -- 172.21.15.105:0/671574189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fa0b055900 con 0x55fa0b040a30 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc676ffd640 1 -- 172.21.15.105:0/671574189 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc68c012aa0 con 0x55fa0b040a30 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc676ffd640 1 -- 172.21.15.105:0/671574189 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fc688097500 con 0x55fa0b040a30 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc6a0832740 10 client.22860 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc6a0832740 10 client.22860 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc6a0832740 20 client.22860 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc6a0832740 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55fa0b056fc0 0x55fa0b059450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc6a0832740 4 client.22860 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc6a0832740 1 -- 172.21.15.105:0/671574189 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa0afb2eb0 con 0x55fa0b056fc0 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc696ffd640 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55fa0b056fc0 0x55fa0b059450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.644+0000 7fc696ffd640 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55fa0b056fc0 0x55fa0b059450 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c504d6e86d7b533 server_cookie=705ed5cd02d36fc4 in_seq=0 out_seq=0 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.645+0000 7fc676ffd640 10 client.22860 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:58:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.645+0000 7fc676ffd640 1 -- 172.21.15.105:0/671574189 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fa0afb2eb0 con 0x55fa0b056fc0 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.645+0000 7fc676ffd640 10 client.22860 handle_command_reply: tid=0 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 client.22860 shutdown 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 -- 172.21.15.105:0/671574189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 msgr2=0x55fa0b04b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04b010 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fc688097860 tx=0x7fc6880adda0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 -- 172.21.15.105:0/671574189 shutdown_connections 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55fa0b056fc0 0x55fa0b059450 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b052520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc676ffd640 0 client.22860 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 --2- 172.21.15.105:0/671574189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04b010 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 -- 172.21.15.105:0/671574189 >> 172.21.15.105:0/671574189 conn(0x55fa0b03bfa0 msgr2=0x55fa0b03b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 -- 172.21.15.105:0/671574189 shutdown_connections 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 1 -- 172.21.15.105:0/671574189 wait complete. 2024-03-26T17:58:51.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.646+0000 7fc6a0832740 20 client.22860 trim_cache size 0 max 16384 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6a0832740 1 Processor -- start 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6a0832740 1 -- start start 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b0524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b04b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6a0832740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981a6c20 con 0x55fa0b040a30 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6a0832740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981a8760 con 0x55fa0b046590 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6a0832740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981a72f0 con 0x55fa0b041420 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37360/0 (socket says 172.21.15.105:37360) 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc6977fe640 1 -- 172.21.15.105:0/1362243274 learned_addr learned my addr 172.21.15.105:0/1362243274 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.659+0000 7fc696ffd640 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b0524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc697fff640 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b04b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc6977fe640 1 -- 172.21.15.105:0/1362243274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 msgr2=0x55fa0b0524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc6977fe640 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b0524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc6977fe640 1 -- 172.21.15.105:0/1362243274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 msgr2=0x55fa0b04b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc6977fe640 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b04b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc6977fe640 1 -- 172.21.15.105:0/1362243274 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0b0365d0 con 0x55fa0b040a30 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc696ffd640 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b0524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc6977fe640 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04ac20 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc68800dfd0 tx=0x7fc6880bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c0931b16f51188 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc67d7fa640 1 -- 172.21.15.105:0/1362243274 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688097130 con 0x55fa0b040a30 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc67d7fa640 1 -- 172.21.15.105:0/1362243274 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc68809ae90 con 0x55fa0b040a30 2024-03-26T17:58:51.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.660+0000 7fc67d7fa640 1 -- 172.21.15.105:0/1362243274 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6880a3af0 con 0x55fa0b040a30 2024-03-26T17:58:51.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 -- 172.21.15.105:0/1362243274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 msgr2=0x55fa0b04ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04ac20 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fc68800dfd0 tx=0x7fc6880bb040 comp rx=0 tx=0).stop 2024-03-26T17:58:51.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 -- 172.21.15.105:0/1362243274 shutdown_connections 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b04b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b0524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1362243274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b040a30 0x55fa0b04ac20 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 -- 172.21.15.105:0/1362243274 >> 172.21.15.105:0/1362243274 conn(0x55fa0b03bfa0 msgr2=0x55fa0b03c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 -- 172.21.15.105:0/1362243274 shutdown_connections 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 -- 172.21.15.105:0/1362243274 wait complete. 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.661+0000 7fc6a0832740 1 -- start start 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6a0832740 1 Processor -- start 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b0503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6a0832740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b0513c0 0x55fa0b051840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6a0832740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981a8090 con 0x55fa0b0513c0 2024-03-26T17:58:51.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6a0832740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981a79c0 con 0x55fa0b046590 2024-03-26T17:58:51.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6a0832740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981a4c00 con 0x55fa0b041420 2024-03-26T17:58:51.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6a0832740 10 client.0 _ll_register_callbacks cb 0x7fc688016c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6977fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc697fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b0513c0 0x55fa0b051840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc696ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b0503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6977fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42584/0 (socket says 172.21.15.105:42584) 2024-03-26T17:58:51.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc6977fe640 1 -- 172.21.15.105:0/1817446774 learned_addr learned my addr 172.21.15.105:0/1817446774 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.662+0000 7fc696ffd640 1 -- 172.21.15.105:0/1817446774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 msgr2=0x55fa0b04fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc696ffd640 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc696ffd640 1 -- 172.21.15.105:0/1817446774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b0513c0 msgr2=0x55fa0b051840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc696ffd640 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b0513c0 0x55fa0b051840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc696ffd640 1 -- 172.21.15.105:0/1817446774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0b051d80 con 0x55fa0b046590 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc6977fe640 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc696ffd640 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b0503b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55fa0b03cf10 tx=0x7fc68c009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d41b141b2dc3fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc676ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc676ffd640 1 -- 172.21.15.105:0/1817446774 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0054c0 con 0x55fa0b046590 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc676ffd640 1 -- 172.21.15.105:0/1817446774 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fc68c0291d0 con 0x55fa0b046590 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc6a0832740 1 -- 172.21.15.105:0/1817446774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc68809ae90 con 0x55fa0b046590 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc676ffd640 1 -- 172.21.15.105:0/1817446774 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc68c039050 con 0x55fa0b046590 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc6a0832740 10 client.32552 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc6a0832740 1 -- 172.21.15.105:0/1817446774 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fa0b0bd9d0 con 0x55fa0b046590 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.663+0000 7fc676ffd640 1 -- 172.21.15.105:0/1817446774 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc68809ae90 con 0x55fa0b046590 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc676ffd640 1 -- 172.21.15.105:0/1817446774 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fc68c012840 con 0x55fa0b046590 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc6a0832740 10 client.32552 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc6a0832740 10 client.32552 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc6a0832740 20 client.32552 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55fa0b0be170 0x55fa0b0c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc6a0832740 4 client.32552 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc6a0832740 1 -- 172.21.15.105:0/1817446774 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa0afad200 con 0x55fa0b0be170 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc6977fe640 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55fa0b0be170 0x55fa0b0c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc676ffd640 10 client.32552 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.664+0000 7fc6977fe640 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55fa0b0be170 0x55fa0b0c0590 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=613fbfd1bb97530e server_cookie=34478691b68a5521 in_seq=0 out_seq=0 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc676ffd640 1 -- 172.21.15.105:0/1817446774 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa0afad200 con 0x55fa0b0be170 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc676ffd640 10 client.32552 handle_command_reply: tid=0 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 client.32552 shutdown 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 -- 172.21.15.105:0/1817446774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 msgr2=0x55fa0b0503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b0503b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x55fa0b03cf10 tx=0x7fc68c009530 comp rx=0 tx=0).stop 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 -- 172.21.15.105:0/1817446774 shutdown_connections 2024-03-26T17:58:51.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55fa0b0be170 0x55fa0b0c0590 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa0b0513c0 0x55fa0b051840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa0b046590 0x55fa0b0503b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 --2- 172.21.15.105:0/1817446774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa0b041420 0x55fa0b04fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc676ffd640 0 client.32552 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 -- 172.21.15.105:0/1817446774 >> 172.21.15.105:0/1817446774 conn(0x55fa0b03bfa0 msgr2=0x55fa0b04ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 -- 172.21.15.105:0/1817446774 shutdown_connections 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 1 -- 172.21.15.105:0/1817446774 wait complete. 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.665+0000 7fc6a0832740 20 client.32552 trim_cache size 0 max 16384 2024-03-26T17:58:51.656 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6800a2f20 msgr2=0x7fc6800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6800a2f20 0x7fc6800a53e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc690009370 tx=0x7fc6900374f0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6981007c0 msgr2=0x7fc6981958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6981007c0 0x7fc6981958e0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc688009a50 tx=0x7fc688009f10 comp rx=0 tx=0).stop 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2137220247 shutdown_connections 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc6800a2f20 0x7fc6800a53e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc698102370 0x7fc69819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6981019b0 0x7fc698195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 --2- 172.21.15.105:0/2137220247 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6981007c0 0x7fc6981958e0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2137220247 >> 172.21.15.105:0/2137220247 conn(0x7fc6980fbd50 msgr2=0x7fc698104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2137220247 shutdown_connections 2024-03-26T17:58:51.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.666+0000 7fc69dc4b640 1 -- 172.21.15.105:0/2137220247 wait complete. 2024-03-26T17:58:51.663 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 -- 172.21.15.105:0/1818388122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13601018e0 msgr2=0x7f1360101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 --2- 172.21.15.105:0/1818388122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13601018e0 0x7f1360101d00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1350009900 tx=0x7f135002bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 -- 172.21.15.105:0/1818388122 shutdown_connections 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 --2- 172.21.15.105:0/1818388122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1360103490 0x7f1360109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 --2- 172.21.15.105:0/1818388122 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1360102ad0 0x7f1360102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 --2- 172.21.15.105:0/1818388122 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13601018e0 0x7f1360101d00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 -- 172.21.15.105:0/1818388122 >> 172.21.15.105:0/1818388122 conn(0x7f13600fce30 msgr2=0x7f13600ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 -- 172.21.15.105:0/1818388122 shutdown_connections 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 -- 172.21.15.105:0/1818388122 wait complete. 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.819+0000 7f1365353640 1 Processor -- start 2024-03-26T17:58:51.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f1365353640 1 -- start start 2024-03-26T17:58:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f1365353640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13601018e0 0x7f1360199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f1365353640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360102ad0 0x7f136019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f1365353640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1360103490 0x7f13601a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f1365353640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f136010d460 con 0x7f1360103490 2024-03-26T17:58:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f1365353640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f136010d2e0 con 0x7f1360102ad0 2024-03-26T17:58:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f1365353640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f136010d5e0 con 0x7f13601018e0 2024-03-26T17:58:51.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1360103490 0x7f13601a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13601018e0 0x7f1360199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1360103490 0x7f13601a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37372/0 (socket says 172.21.15.105:37372) 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135f7fe640 1 -- 172.21.15.105:0/999334869 learned_addr learned my addr 172.21.15.105:0/999334869 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135e7fc640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360102ad0 0x7f136019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135effd640 1 -- 172.21.15.105:0/999334869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360102ad0 msgr2=0x7f136019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135effd640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360102ad0 0x7f136019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135effd640 1 -- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1360103490 msgr2=0x7f13601a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135effd640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1360103490 0x7f13601a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.820+0000 7f135effd640 1 -- 172.21.15.105:0/999334869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13601a19c0 con 0x7f13601018e0 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.821+0000 7f135effd640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13601018e0 0x7f1360199e00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1350036040 tx=0x7f13500042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.821+0000 7f1347fff640 1 -- 172.21.15.105:0/999334869 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350043020 con 0x7f13601018e0 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.821+0000 7f1365353640 1 -- 172.21.15.105:0/999334869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13601a1c50 con 0x7f13601018e0 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.821+0000 7f1365353640 1 -- 172.21.15.105:0/999334869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13601a21c0 con 0x7f13601018e0 2024-03-26T17:58:51.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.821+0000 7f1347fff640 1 -- 172.21.15.105:0/999334869 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1350004a50 con 0x7f13601018e0 2024-03-26T17:58:51.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.821+0000 7f1347fff640 1 -- 172.21.15.105:0/999334869 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350003b50 con 0x7f13601018e0 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f136801c740 1 Processor -- start 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f1347fff640 1 -- 172.21.15.105:0/999334869 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1350046020 con 0x7f13601018e0 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f136801c740 1 -- start start 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f136801c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d09a30 0x559e82d09e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f136801c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0a420 0x559e82d0ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f136801c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f136801c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559e82c6e970 con 0x559e82d09a30 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f136801c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559e82c16d70 con 0x559e82d0a420 2024-03-26T17:58:51.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f136801c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e82c6a920 con 0x559e82d0f590 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.824+0000 7f135f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d09a30 0x559e82d09e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d09a30 0x559e82d09e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37374/0 (socket says 172.21.15.105:37374) 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135f7fe640 1 -- 172.21.15.105:0/3688416454 learned_addr learned my addr 172.21.15.105:0/3688416454 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135effd640 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0a420 0x559e82d0ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135e7fc640 1 -- 172.21.15.105:0/3688416454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0a420 msgr2=0x559e82d0ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135e7fc640 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0a420 0x559e82d0ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135e7fc640 1 -- 172.21.15.105:0/3688416454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d09a30 msgr2=0x559e82d09e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f1347fff640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13480a2fa0 0x7f13480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135e7fc640 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d09a30 0x559e82d09e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135e7fc640 1 -- 172.21.15.105:0/3688416454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e829823f0 con 0x559e82d0f590 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135f7fe640 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d09a30 0x559e82d09e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f1347fff640 1 -- 172.21.15.105:0/999334869 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f13500f1500 con 0x7f13601018e0 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135f7fe640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13480a2fa0 0x7f13480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f135e7fc640 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d11a20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1354009c20 tx=0x7f1354011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=301226a32f0548b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f13457fa640 1 -- 172.21.15.105:0/3688416454 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354004050 con 0x559e82d0f590 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f13457fa640 1 -- 172.21.15.105:0/3688416454 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13540054d0 con 0x559e82d0f590 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.825+0000 7f13457fa640 1 -- 172.21.15.105:0/3688416454 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354004c30 con 0x559e82d0f590 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 -- 172.21.15.105:0/3688416454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 msgr2=0x559e82d11a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f135f7fe640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13480a2fa0 0x7f13480a5460 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x559e82d0ac80 tx=0x7f1358004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d11a20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1354009c20 tx=0x7f1354011810 comp rx=0 tx=0).stop 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 -- 172.21.15.105:0/3688416454 shutdown_connections 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d11a20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0a420 0x559e82d0ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 --2- 172.21.15.105:0/3688416454 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d09a30 0x559e82d09e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 -- 172.21.15.105:0/3688416454 >> 172.21.15.105:0/3688416454 conn(0x559e82d04fa0 msgr2=0x559e82d053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 -- 172.21.15.105:0/3688416454 shutdown_connections 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.826+0000 7f136801c740 1 -- 172.21.15.105:0/3688416454 wait complete. 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 1 -- start start 2024-03-26T17:58:51.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 1 Processor -- start 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d13f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d14440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d1b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559e82d12540 con 0x559e82d0a420 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559e82c2a310 con 0x559e82d09a30 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559e82d126c0 con 0x559e82d0f590 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f136801c740 10 client.0 _ll_register_callbacks cb 0x559e82c65c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f135effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d13f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d14440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f135effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d13f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49694/0 (socket says 172.21.15.105:49694) 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f135effd640 1 -- 172.21.15.105:0/2296281460 learned_addr learned my addr 172.21.15.105:0/2296281460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.827+0000 7f135f7fe640 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d1b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f135effd640 1 -- 172.21.15.105:0/2296281460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 msgr2=0x559e82d1b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f135effd640 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d1b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f135effd640 1 -- 172.21.15.105:0/2296281460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 msgr2=0x559e82d14440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f135effd640 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d14440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f135effd640 1 -- 172.21.15.105:0/2296281460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e82a4c7d0 con 0x559e82d09a30 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f135e7fc640 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d14440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f135effd640 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d13f00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1350006010 tx=0x7f135002c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11d0b506f2f73cf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f1342ff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f1342ff5640 1 -- 172.21.15.105:0/2296281460 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500b9e00 con 0x559e82d09a30 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f136801c740 1 -- 172.21.15.105:0/2296281460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13540054d0 con 0x559e82d09a30 2024-03-26T17:58:51.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f1342ff5640 1 -- 172.21.15.105:0/2296281460 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f13500ba1d0 con 0x559e82d09a30 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f1342ff5640 1 -- 172.21.15.105:0/2296281460 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f13500bbd30 con 0x559e82d09a30 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f136801c740 10 client.32564 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.828+0000 7f136801c740 1 -- 172.21.15.105:0/2296281460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559e82d1e7f0 con 0x559e82d09a30 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.829+0000 7f1342ff5640 1 -- 172.21.15.105:0/2296281460 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f13540054d0 con 0x559e82d09a30 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.829+0000 7f1342ff5640 1 -- 172.21.15.105:0/2296281460 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f13500f10d0 con 0x559e82d09a30 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.829+0000 7f136801c740 10 client.32564 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.829+0000 7f136801c740 10 client.32564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.829+0000 7f136801c740 20 client.32564 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.829+0000 7f136801c740 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559e82d20140 0x559e82d225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.829+0000 7f136801c740 4 client.32564 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.829+0000 7f136801c740 1 -- 172.21.15.105:0/2296281460 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e82c7beb0 con 0x559e82d20140 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.830+0000 7f135e7fc640 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559e82d20140 0x559e82d225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.830+0000 7f1342ff5640 10 client.32564 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.830+0000 7f135e7fc640 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559e82d20140 0x559e82d225d0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5746390e29b76c4 server_cookie=85e45a480e3745b2 in_seq=0 out_seq=0 2024-03-26T17:58:51.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f1342ff5640 1 -- 172.21.15.105:0/2296281460 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559e82c7beb0 con 0x559e82d20140 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f1342ff5640 10 client.32564 handle_command_reply: tid=0 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 client.32564 shutdown 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 -- 172.21.15.105:0/2296281460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 msgr2=0x559e82d13f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d13f00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1350006010 tx=0x7f135002c2b0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 -- 172.21.15.105:0/2296281460 shutdown_connections 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559e82d20140 0x559e82d225d0 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0f590 0x559e82d1b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f1342ff5640 0 client.32564 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d14440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 --2- 172.21.15.105:0/2296281460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d13f00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.832+0000 7f136801c740 1 -- 172.21.15.105:0/2296281460 >> 172.21.15.105:0/2296281460 conn(0x559e82d04fa0 msgr2=0x559e82d041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.833+0000 7f136801c740 1 -- 172.21.15.105:0/2296281460 shutdown_connections 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.833+0000 7f136801c740 1 -- 172.21.15.105:0/2296281460 wait complete. 2024-03-26T17:58:51.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.833+0000 7f136801c740 20 client.32564 trim_cache size 0 max 16384 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.851+0000 7f136801c740 1 Processor -- start 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.851+0000 7f136801c740 1 -- start start 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.851+0000 7f136801c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d09a30 0x559e82d13b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.851+0000 7f136801c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d1b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.851+0000 7f136801c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0f590 0x559e82d14050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.851+0000 7f136801c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ab090 con 0x559e82d0a420 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.851+0000 7f136801c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13601acbd0 con 0x559e82d0f590 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.851+0000 7f136801c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ab760 con 0x559e82d09a30 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d1b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d1b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37404/0 (socket says 172.21.15.105:37404) 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 -- 172.21.15.105:0/1836757353 learned_addr learned my addr 172.21.15.105:0/1836757353 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135f7fe640 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0f590 0x559e82d14050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135effd640 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d09a30 0x559e82d13b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 -- 172.21.15.105:0/1836757353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d09a30 msgr2=0x559e82d13b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d09a30 0x559e82d13b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 -- 172.21.15.105:0/1836757353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0f590 msgr2=0x559e82d14050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0f590 0x559e82d14050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 -- 172.21.15.105:0/1836757353 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e82d0a800 con 0x559e82d0a420 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135f7fe640 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0f590 0x559e82d14050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f135e7fc640 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d1b410 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f1354014040 tx=0x7f1354009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7d489a1660293e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f1344ff9640 1 -- 172.21.15.105:0/1836757353 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354033070 con 0x559e82d0a420 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f1344ff9640 1 -- 172.21.15.105:0/1836757353 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1354004270 con 0x559e82d0a420 2024-03-26T17:58:51.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.852+0000 7f1344ff9640 1 -- 172.21.15.105:0/1836757353 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354003830 con 0x559e82d0a420 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 -- 172.21.15.105:0/1836757353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 msgr2=0x559e82d1b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d1b410 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f1354014040 tx=0x7f1354009680 comp rx=0 tx=0).stop 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 -- 172.21.15.105:0/1836757353 shutdown_connections 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d0f590 0x559e82d14050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0a420 0x559e82d1b410 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 --2- 172.21.15.105:0/1836757353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d09a30 0x559e82d13b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 -- 172.21.15.105:0/1836757353 >> 172.21.15.105:0/1836757353 conn(0x559e82d04fa0 msgr2=0x559e82d0b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 -- 172.21.15.105:0/1836757353 shutdown_connections 2024-03-26T17:58:51.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 -- 172.21.15.105:0/1836757353 wait complete. 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.853+0000 7f136801c740 1 -- start start 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f136801c740 1 Processor -- start 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f136801c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d18640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f136801c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0a420 0x559e82d18b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f136801c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0f590 0x559e82d19b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f136801c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ac500 con 0x559e82d0f590 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f136801c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13601abe30 con 0x559e82d09a30 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f136801c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13601a9070 con 0x559e82d0a420 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f136801c740 10 client.0 _ll_register_callbacks cb 0x7f1350009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0f590 0x559e82d19b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d18640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d18640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49716/0 (socket says 172.21.15.105:49716) 2024-03-26T17:58:51.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0f590 0x559e82d19b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37406/0 (socket says 172.21.15.105:37406) 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0a420 0x559e82d18b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 -- 172.21.15.105:0/2139861675 learned_addr learned my addr 172.21.15.105:0/2139861675 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 -- 172.21.15.105:0/2139861675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0a420 msgr2=0x559e82d18b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0a420 0x559e82d18b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 -- 172.21.15.105:0/2139861675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0f590 msgr2=0x559e82d19b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0f590 0x559e82d19b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 -- 172.21.15.105:0/2139861675 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e82d1a180 con 0x559e82d09a30 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135f7fe640 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0f590 0x559e82d19b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f135effd640 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d18640 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f13500ba530 tx=0x7f13500bd2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221453e240cd8b95 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.854+0000 7f1342ff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f1342ff5640 1 -- 172.21.15.105:0/2139861675 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500bd550 con 0x559e82d09a30 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f1342ff5640 1 -- 172.21.15.105:0/2139861675 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f1350005010 con 0x559e82d09a30 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 1 -- 172.21.15.105:0/2139861675 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1354004270 con 0x559e82d09a30 2024-03-26T17:58:51.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f1342ff5640 1 -- 172.21.15.105:0/2139861675 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f13500bd730 con 0x559e82d09a30 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 10 client.32570 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 1 -- 172.21.15.105:0/2139861675 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559e82d1b4f0 con 0x559e82d09a30 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f1342ff5640 1 -- 172.21.15.105:0/2139861675 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1354004270 con 0x559e82d09a30 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f1342ff5640 1 -- 172.21.15.105:0/2139861675 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f1350005210 con 0x559e82d09a30 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 10 client.32570 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 10 client.32570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 20 client.32570 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559e82d86850 0x559e82d88c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 4 client.32570 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f136801c740 1 -- 172.21.15.105:0/2139861675 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e82c76200 con 0x559e82d86850 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.855+0000 7f135e7fc640 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559e82d86850 0x559e82d88c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.856+0000 7f1342ff5640 10 client.32570 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.856+0000 7f135e7fc640 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559e82d86850 0x559e82d88c40 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52803e077905b1e server_cookie=6c5f28fc857cc5e5 in_seq=0 out_seq=0 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f1342ff5640 1 -- 172.21.15.105:0/2139861675 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e82c76200 con 0x559e82d86850 2024-03-26T17:58:51.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f1342ff5640 10 client.32570 handle_command_reply: tid=0 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 client.32570 shutdown 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 -- 172.21.15.105:0/2139861675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 msgr2=0x559e82d18640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d18640 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f13500ba530 tx=0x7f13500bd2f0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 -- 172.21.15.105:0/2139861675 shutdown_connections 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559e82d86850 0x559e82d88c40 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559e82d0f590 0x559e82d19b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559e82d0a420 0x559e82d18b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 --2- 172.21.15.105:0/2139861675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559e82d09a30 0x559e82d18640 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 -- 172.21.15.105:0/2139861675 >> 172.21.15.105:0/2139861675 conn(0x559e82d04fa0 msgr2=0x559e82d13af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f1342ff5640 0 client.32570 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 -- 172.21.15.105:0/2139861675 shutdown_connections 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 1 -- 172.21.15.105:0/2139861675 wait complete. 2024-03-26T17:58:51.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.858+0000 7f136801c740 20 client.32570 trim_cache size 0 max 16384 2024-03-26T17:58:51.847 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:58:51.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 -- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13480a2fa0 msgr2=0x7f13480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13480a2fa0 0x7f13480a5460 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x559e82d0ac80 tx=0x7f1358004a40 comp rx=0 tx=0).stop 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 -- 172.21.15.105:0/999334869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13601018e0 msgr2=0x7f1360199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13601018e0 0x7f1360199e00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1350036040 tx=0x7f13500042f0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 -- 172.21.15.105:0/999334869 shutdown_connections 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f13480a2fa0 0x7f13480a5460 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1360103490 0x7f13601a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360102ad0 0x7f136019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 --2- 172.21.15.105:0/999334869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13601018e0 0x7f1360199e00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 -- 172.21.15.105:0/999334869 >> 172.21.15.105:0/999334869 conn(0x7f13600fce30 msgr2=0x7f1360073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:51.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 -- 172.21.15.105:0/999334869 shutdown_connections 2024-03-26T17:58:51.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:51.859+0000 7f1365353640 1 -- 172.21.15.105:0/999334869 wait complete. 2024-03-26T17:58:52.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:51 smithi105 ceph-mon[32280]: pgmap v966: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 3.4 MiB/s wr, 180 op/s 2024-03-26T17:58:52.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:51 smithi144 ceph-mon[36095]: pgmap v966: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 3.4 MiB/s wr, 180 op/s 2024-03-26T17:58:52.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:51 smithi135 ceph-mon[35636]: pgmap v966: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 440 KiB/s rd, 3.4 MiB/s wr, 180 op/s 2024-03-26T17:58:52.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59768 7.10 MB/sec execute 335 sec latency 159.891 ms 2024-03-26T17:58:52.858 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:58:52.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:58:52.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:58:53.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 -- 172.21.15.105:0/1408189861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c1019b0 msgr2=0x7f796c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 --2- 172.21.15.105:0/1408189861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c1019b0 0x7f796c101e30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7954009be0 tx=0x7f795402bef0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 -- 172.21.15.105:0/1408189861 shutdown_connections 2024-03-26T17:58:53.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 --2- 172.21.15.105:0/1408189861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f796c102370 0x7f796c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 --2- 172.21.15.105:0/1408189861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c1019b0 0x7f796c101e30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 --2- 172.21.15.105:0/1408189861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f796c1007c0 0x7f796c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 -- 172.21.15.105:0/1408189861 >> 172.21.15.105:0/1408189861 conn(0x7f796c0fbd50 msgr2=0x7f796c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 -- 172.21.15.105:0/1408189861 shutdown_connections 2024-03-26T17:58:53.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 -- 172.21.15.105:0/1408189861 wait complete. 2024-03-26T17:58:53.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.021+0000 7f7972110640 1 Processor -- start 2024-03-26T17:58:53.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f7972110640 1 -- start start 2024-03-26T17:58:53.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f7972110640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f796c1007c0 0x7f796c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f7972110640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c1019b0 0x7f796c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f7972110640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f796c102370 0x7f796c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f7972110640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f796c10d410 con 0x7f796c102370 2024-03-26T17:58:53.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f7972110640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f796c10d290 con 0x7f796c1007c0 2024-03-26T17:58:53.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f7972110640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f796c10d590 con 0x7f796c1019b0 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f796c102370 0x7f796c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f796c1007c0 0x7f796c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f796c102370 0x7f796c19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37430/0 (socket says 172.21.15.105:37430) 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f796c1007c0 0x7f796c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49746/0 (socket says 172.21.15.105:49746) 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796affd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c1019b0 0x7f796c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796bfff640 1 -- 172.21.15.105:0/785695751 learned_addr learned my addr 172.21.15.105:0/785695751 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796b7fe640 1 -- 172.21.15.105:0/785695751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c1019b0 msgr2=0x7f796c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796b7fe640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c1019b0 0x7f796c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796b7fe640 1 -- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f796c102370 msgr2=0x7f796c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796b7fe640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f796c102370 0x7f796c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796b7fe640 1 -- 172.21.15.105:0/785695751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c19f740 con 0x7f796c1007c0 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.022+0000 7f796bfff640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f796c102370 0x7f796c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.023+0000 7f796b7fe640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f796c1007c0 0x7f796c197b20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7964009b20 tx=0x7f796400b2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.023+0000 7f7968ff9640 1 -- 172.21.15.105:0/785695751 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796401e070 con 0x7f796c1007c0 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.023+0000 7f7972110640 1 -- 172.21.15.105:0/785695751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c109620 con 0x7f796c1007c0 2024-03-26T17:58:53.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.023+0000 7f7968ff9640 1 -- 172.21.15.105:0/785695751 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f796400bd00 con 0x7f796c1007c0 2024-03-26T17:58:53.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.023+0000 7f7972110640 1 -- 172.21.15.105:0/785695751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c109b40 con 0x7f796c1007c0 2024-03-26T17:58:53.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.023+0000 7f7968ff9640 1 -- 172.21.15.105:0/785695751 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964019450 con 0x7f796c1007c0 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.026+0000 7f7968ff9640 1 -- 172.21.15.105:0/785695751 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7964021070 con 0x7f796c1007c0 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.026+0000 7f7974dd8740 1 Processor -- start 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.026+0000 7f7974dd8740 1 -- start start 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.026+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcf190 0x563929dcf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.026+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcfb80 0x563929dd4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.026+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929dd7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f7974dd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563929d34630 con 0x563929dd4cf0 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f7974dd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563929cdce10 con 0x563929dcf190 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f7974dd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563929d305e0 con 0x563929dcfb80 2024-03-26T17:58:53.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcf190 0x563929dcf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcf190 0x563929dcf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49752/0 (socket says 172.21.15.105:49752) 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 -- 172.21.15.105:0/1411731680 learned_addr learned my addr 172.21.15.105:0/1411731680 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796affd640 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929dd7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796b7fe640 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcfb80 0x563929dd4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 -- 172.21.15.105:0/1411731680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcfb80 msgr2=0x563929dd4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcfb80 0x563929dd4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f7968ff9640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79580a2f20 0x7f79580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 -- 172.21.15.105:0/1411731680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 msgr2=0x563929dd7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929dd7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 -- 172.21.15.105:0/1411731680 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563929cf3740 con 0x563929dcf190 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f7968ff9640 1 -- 172.21.15.105:0/785695751 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f79640d2be0 con 0x7f796c1007c0 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79580a2f20 0x7f79580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.027+0000 7f796bfff640 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcf190 0x563929dcf5b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f795c011600 tx=0x7f795c011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66aa271dbf351894 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.028+0000 7f7961ffb640 1 -- 172.21.15.105:0/1411731680 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c00f810 con 0x563929dcf190 2024-03-26T17:58:53.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.028+0000 7f7961ffb640 1 -- 172.21.15.105:0/1411731680 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f795c01e070 con 0x563929dcf190 2024-03-26T17:58:53.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.028+0000 7f7961ffb640 1 -- 172.21.15.105:0/1411731680 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c0185f0 con 0x563929dcf190 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.028+0000 7f7974dd8740 1 -- 172.21.15.105:0/1411731680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcf190 msgr2=0x563929dcf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.028+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcf190 0x563929dcf5b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f795c011600 tx=0x7f795c011ac0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f796bfff640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79580a2f20 0x7f79580a53e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f795c01d000 tx=0x7f795c018c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 -- 172.21.15.105:0/1411731680 shutdown_connections 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929dd7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcfb80 0x563929dd4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1411731680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcf190 0x563929dcf5b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 -- 172.21.15.105:0/1411731680 >> 172.21.15.105:0/1411731680 conn(0x563929dca700 msgr2=0x563929dcab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 -- 172.21.15.105:0/1411731680 shutdown_connections 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 -- 172.21.15.105:0/1411731680 wait complete. 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 -- start start 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.029+0000 7f7974dd8740 1 Processor -- start 2024-03-26T17:58:53.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dd9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929dd9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929de0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f7974dd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563929dd7c40 con 0x563929dd4cf0 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f7974dd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563929cf01a0 con 0x563929dcfb80 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f7974dd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563929dd7dc0 con 0x563929dcf190 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f7974dd8740 10 client.0 _ll_register_callbacks cb 0x563929a47c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929de0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929de0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37452/0 (socket says 172.21.15.105:37452) 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 -- 172.21.15.105:0/2879477156 learned_addr learned my addr 172.21.15.105:0/2879477156 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796b7fe640 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dd9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796affd640 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929dd9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 -- 172.21.15.105:0/2879477156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 msgr2=0x563929dd9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dd9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 -- 172.21.15.105:0/2879477156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 msgr2=0x563929dd9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929dd9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 -- 172.21.15.105:0/2879477156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563929b107e0 con 0x563929dd4cf0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f796bfff640 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929de0ba0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f795c0058c0 tx=0x7f795c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d84f87693951755 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.030+0000 7f794affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.031+0000 7f794affd640 1 -- 172.21.15.105:0/2879477156 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c034070 con 0x563929dd4cf0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.031+0000 7f7974dd8740 1 -- 172.21.15.105:0/2879477156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f795c01e070 con 0x563929dd4cf0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.031+0000 7f794affd640 1 -- 172.21.15.105:0/2879477156 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f795c0183b0 con 0x563929dd4cf0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.031+0000 7f794affd640 1 -- 172.21.15.105:0/2879477156 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f795c040050 con 0x563929dd4cf0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.031+0000 7f7974dd8740 10 client.22896 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.031+0000 7f7974dd8740 1 -- 172.21.15.105:0/2879477156 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563929b70f70 con 0x563929dd4cf0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f794affd640 1 -- 172.21.15.105:0/2879477156 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f795c01e070 con 0x563929dd4cf0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f794affd640 1 -- 172.21.15.105:0/2879477156 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f795c00ecd0 con 0x563929dd4cf0 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f7974dd8740 10 client.22896 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f7974dd8740 10 client.22896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:53.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f7974dd8740 20 client.22896 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:53.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f7974dd8740 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563929de52d0 0x563929de7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f7974dd8740 4 client.22896 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:53.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f7974dd8740 1 -- 172.21.15.105:0/2879477156 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563929d41b70 con 0x563929de52d0 2024-03-26T17:58:53.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.032+0000 7f796b7fe640 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563929de52d0 0x563929de7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.033+0000 7f794affd640 10 client.22896 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:53.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.033+0000 7f796b7fe640 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563929de52d0 0x563929de7760 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa8737b39ccb80c server_cookie=4c60f548b18c5566 in_seq=0 out_seq=0 2024-03-26T17:58:53.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.035+0000 7f794affd640 1 -- 172.21.15.105:0/2879477156 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563929d41b70 con 0x563929de52d0 2024-03-26T17:58:53.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.035+0000 7f794affd640 10 client.22896 handle_command_reply: tid=0 2024-03-26T17:58:53.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.035+0000 7f7974dd8740 1 client.22896 shutdown 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f7974dd8740 1 -- 172.21.15.105:0/2879477156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 msgr2=0x563929de0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f7974dd8740 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929de0ba0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f795c0058c0 tx=0x7f795c018000 comp rx=0 tx=0).stop 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f7974dd8740 1 -- 172.21.15.105:0/2879477156 shutdown_connections 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f7974dd8740 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563929de52d0 0x563929de7760 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f7974dd8740 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929de0ba0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f7974dd8740 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929dd9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f7974dd8740 1 --2- 172.21.15.105:0/2879477156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dd9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f7974dd8740 1 -- 172.21.15.105:0/2879477156 >> 172.21.15.105:0/2879477156 conn(0x563929dca700 msgr2=0x563929dc9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.036+0000 7f794affd640 0 client.22896 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.037+0000 7f7974dd8740 1 -- 172.21.15.105:0/2879477156 shutdown_connections 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.037+0000 7f7974dd8740 1 -- 172.21.15.105:0/2879477156 wait complete. 2024-03-26T17:58:53.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.037+0000 7f7974dd8740 20 client.22896 trim_cache size 0 max 16384 2024-03-26T17:58:53.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f7974dd8740 1 Processor -- start 2024-03-26T17:58:53.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f7974dd8740 1 -- start start 2024-03-26T17:58:53.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929de0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929debfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f7974dd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1a6c00 con 0x563929dd4cf0 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f7974dd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1a8740 con 0x563929dcfb80 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f7974dd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1a72d0 con 0x563929dcf190 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f796bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929debfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f796affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929de0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f796b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f796b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42698/0 (socket says 172.21.15.105:42698) 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f796bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929debfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37462/0 (socket says 172.21.15.105:37462) 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f796b7fe640 1 -- 172.21.15.105:0/4082279259 learned_addr learned my addr 172.21.15.105:0/4082279259 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.064+0000 7f796affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929de0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49766/0 (socket says 172.21.15.105:49766) 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f796bfff640 1 -- 172.21.15.105:0/4082279259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 msgr2=0x563929dce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f796bfff640 1 --2- 172.21.15.105:0/4082279259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f796bfff640 1 -- 172.21.15.105:0/4082279259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 msgr2=0x563929de0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f796bfff640 1 --2- 172.21.15.105:0/4082279259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929de0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f796bfff640 1 -- 172.21.15.105:0/4082279259 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563929a08170 con 0x563929dd4cf0 2024-03-26T17:58:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f796bfff640 1 --2- 172.21.15.105:0/4082279259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929debfd0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f795c00e780 tx=0x7f795c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bb8b5eca8bb5af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f79617fa640 1 -- 172.21.15.105:0/4082279259 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c034040 con 0x563929dd4cf0 2024-03-26T17:58:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f79617fa640 1 -- 172.21.15.105:0/4082279259 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f795c01e070 con 0x563929dd4cf0 2024-03-26T17:58:53.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.065+0000 7f79617fa640 1 -- 172.21.15.105:0/4082279259 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c029070 con 0x563929dd4cf0 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 -- 172.21.15.105:0/4082279259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 msgr2=0x563929debfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 --2- 172.21.15.105:0/4082279259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929debfd0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f795c00e780 tx=0x7f795c000bc0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 -- 172.21.15.105:0/4082279259 shutdown_connections 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 --2- 172.21.15.105:0/4082279259 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dd4cf0 0x563929debfd0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 --2- 172.21.15.105:0/4082279259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929de0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 --2- 172.21.15.105:0/4082279259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dcf190 0x563929dce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 -- 172.21.15.105:0/4082279259 >> 172.21.15.105:0/4082279259 conn(0x563929dca700 msgr2=0x563929dcd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 -- 172.21.15.105:0/4082279259 shutdown_connections 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 -- 172.21.15.105:0/4082279259 wait complete. 2024-03-26T17:58:53.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.066+0000 7f7974dd8740 1 -- start start 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f7974dd8740 1 Processor -- start 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dcf190 0x563929dde850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929dded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f7974dd8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dd4cf0 0x563929ddfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f7974dd8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1a8070 con 0x563929dcf190 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f7974dd8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1a79a0 con 0x563929dcfb80 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f7974dd8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f796c1a4be0 con 0x563929dd4cf0 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f7974dd8740 10 client.0 _ll_register_callbacks cb 0x7f7964005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f796b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dcf190 0x563929dde850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f796affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929dded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f796b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dcf190 0x563929dde850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37472/0 (socket says 172.21.15.105:37472) 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f796b7fe640 1 -- 172.21.15.105:0/1651888228 learned_addr learned my addr 172.21.15.105:0/1651888228 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.067+0000 7f796bfff640 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dd4cf0 0x563929ddfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f796b7fe640 1 -- 172.21.15.105:0/1651888228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dd4cf0 msgr2=0x563929ddfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f796b7fe640 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dd4cf0 0x563929ddfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f796b7fe640 1 -- 172.21.15.105:0/1651888228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 msgr2=0x563929dded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f796b7fe640 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929dded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f796b7fe640 1 -- 172.21.15.105:0/1651888228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563929de0340 con 0x563929dcf190 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f796b7fe640 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dcf190 0x563929dde850 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f79640d06f0 tx=0x7f79640d6610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75da6e8f57b9251d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f794affd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f794affd640 1 -- 172.21.15.105:0/1651888228 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79640035f0 con 0x563929dcf190 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f794affd640 1 -- 172.21.15.105:0/1651888228 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f79640a3150 con 0x563929dcf190 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f7974dd8740 1 -- 172.21.15.105:0/1651888228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f795c01e070 con 0x563929dcf190 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f794affd640 1 -- 172.21.15.105:0/1651888228 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f796409f380 con 0x563929dcf190 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f7974dd8740 10 client.22902 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.068+0000 7f7974dd8740 1 -- 172.21.15.105:0/1651888228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563929e4bcc0 con 0x563929dcf190 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f794affd640 1 -- 172.21.15.105:0/1651888228 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f795c01e070 con 0x563929dcf190 2024-03-26T17:58:53.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f794affd640 1 -- 172.21.15.105:0/1651888228 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f796409b9f0 con 0x563929dcf190 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f7974dd8740 10 client.22902 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f7974dd8740 10 client.22902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f7974dd8740 20 client.22902 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563929e4c760 0x563929e4eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f7974dd8740 4 client.22902 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f7974dd8740 1 -- 172.21.15.105:0/1651888228 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563929d3bec0 con 0x563929e4c760 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.069+0000 7f796affd640 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563929e4c760 0x563929e4eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.070+0000 7f794affd640 10 client.22902 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:53.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.070+0000 7f796affd640 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563929e4c760 0x563929e4eb80 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9bbff5e442c93e server_cookie=a4f5cbdbf66137fe in_seq=0 out_seq=0 2024-03-26T17:58:53.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.074+0000 7f794affd640 1 -- 172.21.15.105:0/1651888228 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563929d3bec0 con 0x563929e4c760 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.074+0000 7f794affd640 10 client.22902 handle_command_reply: tid=0 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.074+0000 7f7974dd8740 1 client.22902 shutdown 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 -- 172.21.15.105:0/1651888228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dcf190 msgr2=0x563929dde850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dcf190 0x563929dde850 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f79640d06f0 tx=0x7f79640d6610 comp rx=0 tx=0).stop 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 -- 172.21.15.105:0/1651888228 shutdown_connections 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563929e4c760 0x563929e4eb80 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563929dd4cf0 0x563929ddfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f794affd640 0 client.22902 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563929dcfb80 0x563929dded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 --2- 172.21.15.105:0/1651888228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563929dcf190 0x563929dde850 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 -- 172.21.15.105:0/1651888228 >> 172.21.15.105:0/1651888228 conn(0x563929dca700 msgr2=0x563929dce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 -- 172.21.15.105:0/1651888228 shutdown_connections 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 1 -- 172.21.15.105:0/1651888228 wait complete. 2024-03-26T17:58:53.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.075+0000 7f7974dd8740 20 client.22902 trim_cache size 0 max 16384 2024-03-26T17:58:53.060 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:58:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:58:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "2fc43f82-9be4-4107-863a-580bab72600b", 2024-03-26T17:58:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:58:53.060 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 -- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79580a2f20 msgr2=0x7f79580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79580a2f20 0x7f79580a53e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f795c01d000 tx=0x7f795c018c80 comp rx=0 tx=0).stop 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 -- 172.21.15.105:0/785695751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f796c1007c0 msgr2=0x7f796c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f796c1007c0 0x7f796c197b20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7964009b20 tx=0x7f796400b2c0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 -- 172.21.15.105:0/785695751 shutdown_connections 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f79580a2f20 0x7f79580a53e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f796c102370 0x7f796c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f796c1019b0 0x7f796c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 --2- 172.21.15.105:0/785695751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f796c1007c0 0x7f796c197b20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 -- 172.21.15.105:0/785695751 >> 172.21.15.105:0/785695751 conn(0x7f796c0fbd50 msgr2=0x7f796c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 -- 172.21.15.105:0/785695751 shutdown_connections 2024-03-26T17:58:53.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.076+0000 7f7972110640 1 -- 172.21.15.105:0/785695751 wait complete. 2024-03-26T17:58:53.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:58:53.072 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:58:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.225+0000 7f764cc36640 1 -- 172.21.15.105:0/1050736712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481007c0 msgr2=0x7f7648100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.225+0000 7f764cc36640 1 --2- 172.21.15.105:0/1050736712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481007c0 0x7f7648100be0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7638009900 tx=0x7f763802bc30 comp rx=0 tx=0).stop 2024-03-26T17:58:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.226+0000 7f764cc36640 1 -- 172.21.15.105:0/1050736712 shutdown_connections 2024-03-26T17:58:53.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.226+0000 7f764cc36640 1 --2- 172.21.15.105:0/1050736712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7648102370 0x7f7648108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.226+0000 7f764cc36640 1 --2- 172.21.15.105:0/1050736712 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76481019b0 0x7f7648101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.226+0000 7f764cc36640 1 --2- 172.21.15.105:0/1050736712 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481007c0 0x7f7648100be0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.226+0000 7f764cc36640 1 -- 172.21.15.105:0/1050736712 >> 172.21.15.105:0/1050736712 conn(0x7f76480fbd50 msgr2=0x7f76480fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.226+0000 7f764cc36640 1 -- 172.21.15.105:0/1050736712 shutdown_connections 2024-03-26T17:58:53.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.226+0000 7f764cc36640 1 -- 172.21.15.105:0/1050736712 wait complete. 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.226+0000 7f764cc36640 1 Processor -- start 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764cc36640 1 -- start start 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764cc36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76481007c0 0x7f764810c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764cc36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481019b0 0x7f764810a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764cc36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7648102370 0x7f764810af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764cc36640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d410 con 0x7f76481019b0 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764cc36640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d290 con 0x7f76481007c0 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764cc36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f764810d590 con 0x7f7648102370 2024-03-26T17:58:53.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481019b0 0x7f764810a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76481007c0 0x7f764810c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7646d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7648102370 0x7f764810af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481019b0 0x7f764810a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37480/0 (socket says 172.21.15.105:37480) 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f764659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76481007c0 0x7f764810c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49792/0 (socket says 172.21.15.105:49792) 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 -- 172.21.15.105:0/3794644062 learned_addr learned my addr 172.21.15.105:0/3794644062 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7646d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7648102370 0x7f764810af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42724/0 (socket says 172.21.15.105:42724) 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 -- 172.21.15.105:0/3794644062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7648102370 msgr2=0x7f764810af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7648102370 0x7f764810af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 -- 172.21.15.105:0/3794644062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76481007c0 msgr2=0x7f764810c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76481007c0 0x7f764810c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 -- 172.21.15.105:0/3794644062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764810b7a0 con 0x7f76481019b0 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f7645d9b640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481019b0 0x7f764810a9d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f763c00ea60 tx=0x7f763c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.227+0000 7f76377fe640 1 -- 172.21.15.105:0/3794644062 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c00cde0 con 0x7f76481019b0 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.228+0000 7f764cc36640 1 -- 172.21.15.105:0/3794644062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481a1c60 con 0x7f76481019b0 2024-03-26T17:58:53.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.228+0000 7f764cc36640 1 -- 172.21.15.105:0/3794644062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a21b0 con 0x7f76481019b0 2024-03-26T17:58:53.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.228+0000 7f76377fe640 1 -- 172.21.15.105:0/3794644062 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f763c0035f0 con 0x7f76481019b0 2024-03-26T17:58:53.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.228+0000 7f76377fe640 1 -- 172.21.15.105:0/3794644062 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c010690 con 0x7f76481019b0 2024-03-26T17:58:53.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.231+0000 7f764f826740 1 Processor -- start 2024-03-26T17:58:53.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.231+0000 7f764f826740 1 -- start start 2024-03-26T17:58:53.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f76377fe640 1 -- 172.21.15.105:0/3794644062 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f763c010830 con 0x7f76481019b0 2024-03-26T17:58:53.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f764f826740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eebbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f764f826740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebc2e0 0x560c2eec0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f764f826740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eec38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f764f826740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560c2ee20980 con 0x560c2eebc2e0 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f764f826740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560c2edc8be0 con 0x560c2eec1450 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f764f826740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560c2ee1c930 con 0x560c2eebb8f0 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f7646d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eebbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f7646d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eebbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42734/0 (socket says 172.21.15.105:42734) 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f7646d9d640 1 -- 172.21.15.105:0/1153698285 learned_addr learned my addr 172.21.15.105:0/1153698285 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f764659c640 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebc2e0 0x560c2eec0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.232+0000 7f7645d9b640 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eec38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.233+0000 7f7645d9b640 1 -- 172.21.15.105:0/1153698285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 msgr2=0x560c2eebbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.233+0000 7f7645d9b640 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eebbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.233+0000 7f7645d9b640 1 -- 172.21.15.105:0/1153698285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebc2e0 msgr2=0x560c2eec0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.233+0000 7f7645d9b640 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebc2e0 0x560c2eec0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.233+0000 7f7645d9b640 1 -- 172.21.15.105:0/1153698285 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c2ebfe7d0 con 0x560c2eec1450 2024-03-26T17:58:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.233+0000 7f7645d9b640 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eec38e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f763c0cd140 tx=0x7f763c0d5f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a68d879e916b6a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.233+0000 7f76377fe640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76300a2fa0 0x7f76300a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.233+0000 7f7634ff9640 1 -- 172.21.15.105:0/1153698285 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c0de070 con 0x560c2eec1450 2024-03-26T17:58:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.234+0000 7f7634ff9640 1 -- 172.21.15.105:0/1153698285 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f763c09c4a0 con 0x560c2eec1450 2024-03-26T17:58:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.234+0000 7f7634ff9640 1 -- 172.21.15.105:0/1153698285 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c0b8cc0 con 0x560c2eec1450 2024-03-26T17:58:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.234+0000 7f76377fe640 1 -- 172.21.15.105:0/3794644062 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f763c0cf6b0 con 0x7f76481019b0 2024-03-26T17:58:53.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.234+0000 7f7646d9d640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76300a2fa0 0x7f76300a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.234+0000 7f764f826740 1 -- 172.21.15.105:0/1153698285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 msgr2=0x560c2eec38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.234+0000 7f764f826740 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eec38e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f763c0cd140 tx=0x7f763c0d5f10 comp rx=0 tx=0).stop 2024-03-26T17:58:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.234+0000 7f7646d9d640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76300a2fa0 0x7f76300a5460 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x560c2eebcb40 tx=0x7f7640009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 -- 172.21.15.105:0/1153698285 shutdown_connections 2024-03-26T17:58:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eec38e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebc2e0 0x560c2eec0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 --2- 172.21.15.105:0/1153698285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eebbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 -- 172.21.15.105:0/1153698285 >> 172.21.15.105:0/1153698285 conn(0x560c2eeb6e60 msgr2=0x560c2eeb7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 -- 172.21.15.105:0/1153698285 shutdown_connections 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 -- 172.21.15.105:0/1153698285 wait complete. 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 -- start start 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.235+0000 7f764f826740 1 Processor -- start 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764f826740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eec5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764f826740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebc2e0 0x560c2eec6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764f826740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 0x560c2eecd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764f826740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560c2eec4400 con 0x560c2eec1450 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764f826740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560c2eddc160 con 0x560c2eebc2e0 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764f826740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560c2eec4580 con 0x560c2eebb8f0 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764f826740 10 client.0 _ll_register_callbacks cb 0x560c2ee18720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eec5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eec5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42736/0 (socket says 172.21.15.105:42736) 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 -- 172.21.15.105:0/3955818698 learned_addr learned my addr 172.21.15.105:0/3955818698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f7646d9d640 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 0x560c2eecd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f7645d9b640 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebc2e0 0x560c2eec6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 -- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebc2e0 msgr2=0x560c2eec6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebc2e0 0x560c2eec6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 -- 172.21.15.105:0/3955818698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 msgr2=0x560c2eecd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 0x560c2eecd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 -- 172.21.15.105:0/3955818698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c2eb343f0 con 0x560c2eebb8f0 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f7645d9b640 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebc2e0 0x560c2eec6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f764659c640 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eec5b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7638006010 tx=0x7f7638004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36bd884dcf8a0bff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.236+0000 7f762dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.237+0000 7f762dffb640 1 -- 172.21.15.105:0/3955818698 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638005d50 con 0x560c2eebb8f0 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.237+0000 7f762dffb640 1 -- 172.21.15.105:0/3955818698 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f7638005030 con 0x560c2eebb8f0 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.237+0000 7f764f826740 1 -- 172.21.15.105:0/3955818698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c09c4a0 con 0x560c2eebb8f0 2024-03-26T17:58:53.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.237+0000 7f762dffb640 1 -- 172.21.15.105:0/3955818698 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7638005230 con 0x560c2eebb8f0 2024-03-26T17:58:53.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.237+0000 7f764f826740 10 client.32584 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:53.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.237+0000 7f764f826740 1 -- 172.21.15.105:0/3955818698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560c2eaf40f0 con 0x560c2eebb8f0 2024-03-26T17:58:53.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.237+0000 7f762dffb640 1 -- 172.21.15.105:0/3955818698 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f763c09c4a0 con 0x560c2eebb8f0 2024-03-26T17:58:53.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.238+0000 7f762dffb640 1 -- 172.21.15.105:0/3955818698 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f7638003920 con 0x560c2eebb8f0 2024-03-26T17:58:53.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.238+0000 7f764f826740 10 client.32584 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:53.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.238+0000 7f764f826740 10 client.32584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.238+0000 7f764f826740 20 client.32584 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.238+0000 7f764f826740 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c2eed1df0 0x560c2eed4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.238+0000 7f764f826740 4 client.32584 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.238+0000 7f764f826740 1 -- 172.21.15.105:0/3955818698 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c2ee2dec0 con 0x560c2eed1df0 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.238+0000 7f7645d9b640 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c2eed1df0 0x560c2eed4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.239+0000 7f7645d9b640 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c2eed1df0 0x560c2eed4280 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768476690ae3b72a server_cookie=17514ea5d8c0078e in_seq=0 out_seq=0 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.239+0000 7f762dffb640 10 client.32584 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.240+0000 7f762dffb640 1 -- 172.21.15.105:0/3955818698 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560c2ee2dec0 con 0x560c2eed1df0 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f762dffb640 10 client.32584 handle_command_reply: tid=0 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 client.32584 shutdown 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 -- 172.21.15.105:0/3955818698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 msgr2=0x560c2eec5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eec5b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7638006010 tx=0x7f7638004270 comp rx=0 tx=0).stop 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 -- 172.21.15.105:0/3955818698 shutdown_connections 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c2eed1df0 0x560c2eed4280 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 0x560c2eecd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebc2e0 0x560c2eec6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 --2- 172.21.15.105:0/3955818698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebb8f0 0x560c2eec5b00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f762dffb640 0 client.32584 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 -- 172.21.15.105:0/3955818698 >> 172.21.15.105:0/3955818698 conn(0x560c2eeb6e60 msgr2=0x560c2eeb6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 -- 172.21.15.105:0/3955818698 shutdown_connections 2024-03-26T17:58:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 1 -- 172.21.15.105:0/3955818698 wait complete. 2024-03-26T17:58:53.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.241+0000 7f764f826740 20 client.32584 trim_cache size 0 max 16384 2024-03-26T17:58:53.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.265+0000 7f764f826740 1 Processor -- start 2024-03-26T17:58:53.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.265+0000 7f764f826740 1 -- start start 2024-03-26T17:58:53.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.265+0000 7f764f826740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebb8f0 0x560c2eec5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.265+0000 7f764f826740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.265+0000 7f764f826740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 0x560c2eec5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.265+0000 7f764f826740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ab070 con 0x560c2eec1450 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.265+0000 7f764f826740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76481acbb0 con 0x560c2eebb8f0 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.265+0000 7f764f826740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ab740 con 0x560c2eebc2e0 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7646d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 0x560c2eec5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f764659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebb8f0 0x560c2eec5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42740/0 (socket says 172.21.15.105:42740) 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f764659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebb8f0 0x560c2eec5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:49826/0 (socket says 172.21.15.105:49826) 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 -- 172.21.15.105:0/2821353094 learned_addr learned my addr 172.21.15.105:0/2821353094 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 -- 172.21.15.105:0/2821353094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebb8f0 msgr2=0x560c2eec5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 --2- 172.21.15.105:0/2821353094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebb8f0 0x560c2eec5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 -- 172.21.15.105:0/2821353094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 msgr2=0x560c2eec5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 --2- 172.21.15.105:0/2821353094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 0x560c2eec5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 -- 172.21.15.105:0/2821353094 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c2eeb28d0 con 0x560c2eebc2e0 2024-03-26T17:58:53.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.266+0000 7f7645d9b640 1 --2- 172.21.15.105:0/2821353094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecd040 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f763c098200 tx=0x7f763c09bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46cc61a98d0f46b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.267+0000 7f762ffff640 1 -- 172.21.15.105:0/2821353094 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c0bd450 con 0x560c2eebc2e0 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.267+0000 7f762ffff640 1 -- 172.21.15.105:0/2821353094 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f763c003b30 con 0x560c2eebc2e0 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.267+0000 7f762ffff640 1 -- 172.21.15.105:0/2821353094 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c0bd630 con 0x560c2eebc2e0 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 -- 172.21.15.105:0/2821353094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 msgr2=0x560c2eecd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 --2- 172.21.15.105:0/2821353094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecd040 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f763c098200 tx=0x7f763c09bb70 comp rx=0 tx=0).stop 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 -- 172.21.15.105:0/2821353094 shutdown_connections 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 --2- 172.21.15.105:0/2821353094 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eec1450 0x560c2eec5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 --2- 172.21.15.105:0/2821353094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecd040 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 --2- 172.21.15.105:0/2821353094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eebb8f0 0x560c2eec5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 -- 172.21.15.105:0/2821353094 >> 172.21.15.105:0/2821353094 conn(0x560c2eeb6e60 msgr2=0x560c2eeb7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 -- 172.21.15.105:0/2821353094 shutdown_connections 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 -- 172.21.15.105:0/2821353094 wait complete. 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 -- start start 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.268+0000 7f764f826740 1 Processor -- start 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764f826740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebb8f0 0x560c2eecab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764f826740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764f826740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eecc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764f826740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ac4e0 con 0x560c2eebb8f0 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764f826740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76481abe10 con 0x560c2eec1450 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764f826740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f76481a9050 con 0x560c2eebc2e0 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764f826740 10 client.0 _ll_register_callbacks cb 0x7f7638009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebb8f0 0x560c2eecab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebb8f0 0x560c2eecab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37504/0 (socket says 172.21.15.105:37504) 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 -- 172.21.15.105:0/2280135112 learned_addr learned my addr 172.21.15.105:0/2280135112 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f7645d9b640 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f7646d9d640 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eecc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 -- 172.21.15.105:0/2280135112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 msgr2=0x560c2eecb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 -- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 msgr2=0x560c2eecc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eecc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 -- 172.21.15.105:0/2280135112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c2eec11a0 con 0x560c2eebb8f0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f7646d9d640 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eecc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f764659c640 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebb8f0 0x560c2eecab70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7638009850 tx=0x7f7638034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4400ee46227083a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.269+0000 7f762dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f762dffb640 1 -- 172.21.15.105:0/2280135112 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638009520 con 0x560c2eebb8f0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 1 -- 172.21.15.105:0/2280135112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560c2ee28210 con 0x560c2eebb8f0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f762dffb640 1 -- 172.21.15.105:0/2280135112 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f7638003800 con 0x560c2eebb8f0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f762dffb640 1 -- 172.21.15.105:0/2280135112 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7638003a00 con 0x560c2eebb8f0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 10 client.22920 fetch_fsmap learned FSMap version 41 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 1 -- 172.21.15.105:0/2280135112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560c2ef38470 con 0x560c2eebb8f0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f762dffb640 1 -- 172.21.15.105:0/2280135112 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560c2ee28210 con 0x560c2eebb8f0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f762dffb640 1 -- 172.21.15.105:0/2280135112 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f76380046c0 con 0x560c2eebb8f0 2024-03-26T17:58:53.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 10 client.22920 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:58:53.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 10 client.22920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:58:53.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 20 client.22920 populate_metadata read hostname 'smithi105' 2024-03-26T17:58:53.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c2ef38e40 0x560c2ef3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:58:53.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 4 client.22920 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:58:53.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.270+0000 7f764f826740 1 -- 172.21.15.105:0/2280135112 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c2eda5540 con 0x560c2ef38e40 2024-03-26T17:58:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.295+0000 7f7645d9b640 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c2ef38e40 0x560c2ef3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:58:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.295+0000 7f762dffb640 10 client.22920 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:53.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.295+0000 7f7645d9b640 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c2ef38e40 0x560c2ef3b260 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65dd8e93d23fed8 server_cookie=1f6779d9874428e2 in_seq=0 out_seq=0 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f762dffb640 1 -- 172.21.15.105:0/2280135112 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x560c2eda5540 con 0x560c2ef38e40 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f762dffb640 10 client.22920 handle_command_reply: tid=0 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 client.22920 shutdown 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 -- 172.21.15.105:0/2280135112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebb8f0 msgr2=0x560c2eecab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebb8f0 0x560c2eecab70 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7638009850 tx=0x7f7638034670 comp rx=0 tx=0).stop 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 -- 172.21.15.105:0/2280135112 shutdown_connections 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560c2ef38e40 0x560c2ef3b260 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560c2eec1450 0x560c2eecc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560c2eebc2e0 0x560c2eecb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 --2- 172.21.15.105:0/2280135112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560c2eebb8f0 0x560c2eecab70 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f762dffb640 0 client.22920 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 -- 172.21.15.105:0/2280135112 >> 172.21.15.105:0/2280135112 conn(0x560c2eeb6e60 msgr2=0x560c2eeb57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 -- 172.21.15.105:0/2280135112 shutdown_connections 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 1 -- 172.21.15.105:0/2280135112 wait complete. 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.297+0000 7f764f826740 20 client.22920 trim_cache size 0 max 16384 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (401 inodes in the stack)", 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "2fc43f82-9be4-4107-863a-580bab72600b": { 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "2fc43f82-9be4-4107-863a-580bab72600b", 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:58:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:58:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:58:53.283 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:58:53.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 -- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76300a2fa0 msgr2=0x7f76300a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76300a2fa0 0x7f76300a5460 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x560c2eebcb40 tx=0x7f7640009500 comp rx=0 tx=0).stop 2024-03-26T17:58:53.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 -- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481019b0 msgr2=0x7f764810a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:58:53.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481019b0 0x7f764810a9d0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f763c00ea60 tx=0x7f763c00ef20 comp rx=0 tx=0).stop 2024-03-26T17:58:53.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 -- 172.21.15.105:0/3794644062 shutdown_connections 2024-03-26T17:58:53.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76300a2fa0 0x7f76300a5460 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7648102370 0x7f764810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76481019b0 0x7f764810a9d0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 --2- 172.21.15.105:0/3794644062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76481007c0 0x7f764810c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:58:53.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 -- 172.21.15.105:0/3794644062 >> 172.21.15.105:0/3794644062 conn(0x7f76480fbd50 msgr2=0x7f7648104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:58:53.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 -- 172.21.15.105:0/3794644062 shutdown_connections 2024-03-26T17:58:53.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:58:53.298+0000 7f764cc36640 1 -- 172.21.15.105:0/3794644062 wait complete. 2024-03-26T17:58:53.296 INFO:tasks.cephfs.filesystem:scrub status for tag:2fc43f82-9be4-4107-863a-580bab72600b - {'path': '/', 'tag': '2fc43f82-9be4-4107-863a-580bab72600b', 'options': 'recursive,force'} 2024-03-26T17:58:53.406 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:58:53.145+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003c80 [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{1=1,2=1,3=1,4=1} v11648 ap=2 RANDEPHEMERALPIN f(v312 m2024-03-26T17:58:28.077729+0000 11=11+0) n(v24 rc2024-03-26T17:58:28.077729+0000 b2861056 12=11+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@6254} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d142580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003c80:[<0x10000003bc0/COREL v50>,<0x10000003bb6/~dmtmp v48>,<0x10000000007/client8 v2078>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003c80:[<0x10000003bc0/COREL v11648>,<0x10000003bb6/~dmtmp v2412>,<0x10000000007/client8 v10177>,<0x10000000006/clients v2916>,<0x10000000005/tmp v2634>,<0x10000000003/client.0 v2525>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v2458>,<0x10000000001/sv_0 v2468>,<0x10000000000/qa v2393>,<0x1/volumes v2363>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:58:28.049697+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:58:28.077729+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v312 m2024-03-26T17:58:28.077729+0000 11=11+0)","memory_value.rstat":"n(v24 rc2024-03-26T17:58:28.077729+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T17:58:53.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59860 7.09 MB/sec execute 336 sec latency 216.534 ms 2024-03-26T17:58:54.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: pgmap v967: 97 pgs: 97 active+clean; 341 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 4.1 MiB/s wr, 232 op/s 2024-03-26T17:58:54.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:58:54.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:54.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:54.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.341 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:53 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: pgmap v967: 97 pgs: 97 active+clean; 341 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 4.1 MiB/s wr, 232 op/s 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:58:54.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:54.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:58:54.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.385 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:53 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: pgmap v967: 97 pgs: 97 active+clean; 341 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 4.1 MiB/s wr, 232 op/s 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:54.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:58:54.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:54.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:58:54.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.408 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:53 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:58:54.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 59965 7.07 MB/sec execute 337 sec latency 135.423 ms 2024-03-26T17:58:55.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:54 smithi144 ceph-mon[36095]: pgmap v968: 97 pgs: 97 active+clean; 337 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-26T17:58:55.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:54 smithi105 ceph-mon[32280]: pgmap v968: 97 pgs: 97 active+clean; 337 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-26T17:58:55.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:54 smithi135 ceph-mon[35636]: pgmap v968: 97 pgs: 97 active+clean; 337 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.7 MiB/s wr, 228 op/s 2024-03-26T17:58:55.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 60078 7.06 MB/sec execute 338 sec latency 129.123 ms 2024-03-26T17:58:56.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 60205 7.06 MB/sec execute 339 sec latency 140.063 ms 2024-03-26T17:58:57.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:56 smithi144 ceph-mon[36095]: pgmap v969: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 3.4 MiB/s wr, 246 op/s 2024-03-26T17:58:57.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:56 smithi105 ceph-mon[32280]: pgmap v969: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 3.4 MiB/s wr, 246 op/s 2024-03-26T17:58:57.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:56 smithi135 ceph-mon[35636]: pgmap v969: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 3.4 MiB/s wr, 246 op/s 2024-03-26T17:58:57.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 60360 7.06 MB/sec execute 340 sec latency 138.248 ms 2024-03-26T17:58:58.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 60483 7.07 MB/sec execute 341 sec latency 113.722 ms 2024-03-26T17:58:59.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:58:58 smithi144 ceph-mon[36095]: pgmap v970: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 3.9 MiB/s wr, 303 op/s 2024-03-26T17:58:59.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:58:58 smithi105 ceph-mon[32280]: pgmap v970: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 3.9 MiB/s wr, 303 op/s 2024-03-26T17:58:59.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:58:58 smithi135 ceph-mon[35636]: pgmap v970: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 3.9 MiB/s wr, 303 op/s 2024-03-26T17:58:59.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 60599 7.07 MB/sec execute 342 sec latency 177.567 ms 2024-03-26T17:59:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 -- 172.21.15.105:0/524503833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4100190 msgr2=0x7fc6a41005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/524503833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4100190 0x7fc6a41005b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009be0 tx=0x7fc69c02bef0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 -- 172.21.15.105:0/524503833 shutdown_connections 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/524503833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a4101490 0x7fc6a4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/524503833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a4100af0 0x7fc6a4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/524503833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4100190 0x7fc6a41005b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 -- 172.21.15.105:0/524503833 >> 172.21.15.105:0/524503833 conn(0x7fc6a40fbd90 msgr2=0x7fc6a40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 -- 172.21.15.105:0/524503833 shutdown_connections 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.291+0000 7fc6aacdc640 1 -- 172.21.15.105:0/524503833 wait complete. 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6aacdc640 1 Processor -- start 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6aacdc640 1 -- start start 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6aacdc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a4100190 0x7fc6a4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6aacdc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a4100af0 0x7fc6a419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6aacdc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4101490 0x7fc6a41a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6aacdc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a410f630 con 0x7fc6a4100af0 2024-03-26T17:59:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6aacdc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a410f4b0 con 0x7fc6a4100190 2024-03-26T17:59:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6aacdc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a410f7b0 con 0x7fc6a4101490 2024-03-26T17:59:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a4100190 0x7fc6a4199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a4100af0 0x7fc6a419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6a9279640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4101490 0x7fc6a41a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a4100190 0x7fc6a4199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43060/0 (socket says 172.21.15.105:43060) 2024-03-26T17:59:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.292+0000 7fc6a8a78640 1 -- 172.21.15.105:0/462319422 learned_addr learned my addr 172.21.15.105:0/462319422 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a9279640 1 -- 172.21.15.105:0/462319422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a4100190 msgr2=0x7fc6a4199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a9279640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a4100190 0x7fc6a4199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a9279640 1 -- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a4100af0 msgr2=0x7fc6a419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a9279640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a4100af0 0x7fc6a419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a9279640 1 -- 172.21.15.105:0/462319422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a41a18c0 con 0x7fc6a4101490 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a8a78640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a4100190 0x7fc6a4199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a9279640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4101490 0x7fc6a41a1220 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e970 tx=0x7fc69800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a1ffb640 1 -- 172.21.15.105:0/462319422 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69800ccc0 con 0x7fc6a4101490 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a1ffb640 1 -- 172.21.15.105:0/462319422 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc698009e40 con 0x7fc6a4101490 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6aacdc640 1 -- 172.21.15.105:0/462319422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a41a1bb0 con 0x7fc6a4101490 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6a1ffb640 1 -- 172.21.15.105:0/462319422 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698004a00 con 0x7fc6a4101490 2024-03-26T17:59:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.293+0000 7fc6aacdc640 1 -- 172.21.15.105:0/462319422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a41a2150 con 0x7fc6a4101490 2024-03-26T17:59:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.296+0000 7fc6a1ffb640 1 -- 172.21.15.105:0/462319422 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc698004ba0 con 0x7fc6a4101490 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.297+0000 7fc6a1ffb640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc68c0a3040 0x7fc68c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.297+0000 7fc6ad9a0740 1 Processor -- start 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.297+0000 7fc6a8a78640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc68c0a3040 0x7fc68c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.297+0000 7fc6a1ffb640 1 -- 172.21.15.105:0/462319422 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fc6980cf250 con 0x7fc6a4101490 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6ad9a0740 1 -- start start 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd8145d0 0x556edd8149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 0x556edd819ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd81a130 0x556edd81c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6ad9a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556edd7790b0 con 0x556edd8145d0 2024-03-26T17:59:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6ad9a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556edd721960 con 0x556edd81a130 2024-03-26T17:59:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6ad9a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556edd775060 con 0x556edd814fc0 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a9279640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd81a130 0x556edd81c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a8a78640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc68c0a3040 0x7fc68c0a5500 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc69c006010 tx=0x7fc69c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 0x556edd819ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a9279640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd81a130 0x556edd81c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43072/0 (socket says 172.21.15.105:43072) 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd8145d0 0x556edd8149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a9279640 1 -- 172.21.15.105:0/1954778075 learned_addr learned my addr 172.21.15.105:0/1954778075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a9279640 1 -- 172.21.15.105:0/1954778075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 msgr2=0x556edd819ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a9279640 1 --2- 172.21.15.105:0/1954778075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 0x556edd819ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a9279640 1 -- 172.21.15.105:0/1954778075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd8145d0 msgr2=0x556edd8149f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a9279640 1 --2- 172.21.15.105:0/1954778075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd8145d0 0x556edd8149f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.298+0000 7fc6a9279640 1 -- 172.21.15.105:0/1954778075 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556edd733c60 con 0x556edd81a130 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.299+0000 7fc6a9279640 1 --2- 172.21.15.105:0/1954778075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd81a130 0x556edd81c5c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc698097da0 tx=0x7fc69809b360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1041b2b851a83f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.299+0000 7fc68affd640 1 -- 172.21.15.105:0/1954778075 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69801c070 con 0x556edd81a130 2024-03-26T17:59:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.299+0000 7fc68affd640 1 -- 172.21.15.105:0/1954778075 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc69809b940 con 0x556edd81a130 2024-03-26T17:59:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.299+0000 7fc68affd640 1 -- 172.21.15.105:0/1954778075 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6980860c0 con 0x556edd81a130 2024-03-26T17:59:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/1954778075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd81a130 msgr2=0x556edd81c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/1954778075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd81a130 0x556edd81c5c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc698097da0 tx=0x7fc69809b360 comp rx=0 tx=0).stop 2024-03-26T17:59:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/1954778075 shutdown_connections 2024-03-26T17:59:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/1954778075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd81a130 0x556edd81c5c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/1954778075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 0x556edd819ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/1954778075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd8145d0 0x556edd8149f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/1954778075 >> 172.21.15.105:0/1954778075 conn(0x556edd80fb40 msgr2=0x556edd80ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/1954778075 shutdown_connections 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/1954778075 wait complete. 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.300+0000 7fc6ad9a0740 1 -- start start 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.301+0000 7fc6ad9a0740 1 Processor -- start 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.301+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd8145d0 0x556edd81e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.301+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 0x556edd81e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.301+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd825890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.301+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd8145d0 0x556edd81e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.301+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd8145d0 0x556edd81e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43086/0 (socket says 172.21.15.105:43086) 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6a8a78640 1 -- 172.21.15.105:0/3535896024 learned_addr learned my addr 172.21.15.105:0/3535896024 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6a9279640 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd825890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.301+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 0x556edd81e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556edd81d020 con 0x556edd81a130 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556edd734db0 con 0x556edd8145d0 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556edd81d1a0 con 0x556edd814fc0 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6ad9a0740 10 client.0 _ll_register_callbacks cb 0x556edd739110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6a9279640 1 -- 172.21.15.105:0/3535896024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 msgr2=0x556edd81e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6a9279640 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 0x556edd81e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6a9279640 1 -- 172.21.15.105:0/3535896024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd8145d0 msgr2=0x556edd81e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6a9279640 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd8145d0 0x556edd81e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6a9279640 1 -- 172.21.15.105:0/3535896024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556edd48d210 con 0x556edd81a130 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc69809b940 con 0x556edd81a130 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc6a9279640 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd825890 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc69809fc50 tx=0x7fc69809fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6943a2468998b019 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc683fff640 10 client.22932 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc683fff640 1 -- 172.21.15.105:0/3535896024 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69801c070 con 0x556edd81a130 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.302+0000 7fc683fff640 1 -- 172.21.15.105:0/3535896024 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fc6980ce070 con 0x556edd81a130 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.303+0000 7fc683fff640 1 -- 172.21.15.105:0/3535896024 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc6980999e0 con 0x556edd81a130 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.303+0000 7fc6ad9a0740 10 client.22932 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.303+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=41}) -- 0x556edd559620 con 0x556edd81a130 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.303+0000 7fc683fff640 1 -- 172.21.15.105:0/3535896024 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fc69801f030 con 0x556edd81a130 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.303+0000 7fc6ad9a0740 10 client.22932 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.303+0000 7fc6ad9a0740 10 client.22932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.304+0000 7fc6ad9a0740 20 client.22932 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.304+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556edd82a1e0 0x556edd82c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.304+0000 7fc6ad9a0740 4 client.22932 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.304+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556edd7865f0 con 0x556edd82a1e0 2024-03-26T17:59:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.304+0000 7fc6a8a78640 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556edd82a1e0 0x556edd82c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.305+0000 7fc6a8a78640 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556edd82a1e0 0x556edd82c5f0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c93b4d1d83df27 server_cookie=78143f2d12304f2 in_seq=0 out_seq=0 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.305+0000 7fc683fff640 10 client.22932 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.306+0000 7fc683fff640 1 -- 172.21.15.105:0/3535896024 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556edd7865f0 con 0x556edd82a1e0 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.306+0000 7fc683fff640 10 client.22932 handle_command_reply: tid=0 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.306+0000 7fc6ad9a0740 1 client.22932 shutdown 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.306+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 msgr2=0x556edd825890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.306+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd825890 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc69809fc50 tx=0x7fc69809fa90 comp rx=0 tx=0).stop 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.306+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 shutdown_connections 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.306+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556edd82a1e0 0x556edd82c5f0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.306+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd825890 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.307+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd814fc0 0x556edd81e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.307+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3535896024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd8145d0 0x556edd81e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.307+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 >> 172.21.15.105:0/3535896024 conn(0x556edd80fb40 msgr2=0x556edd80edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.307+0000 7fc683fff640 0 client.22932 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.307+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 shutdown_connections 2024-03-26T17:59:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.307+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3535896024 wait complete. 2024-03-26T17:59:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.307+0000 7fc6ad9a0740 20 client.22932 trim_cache size 0 max 16384 2024-03-26T17:59:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.331+0000 7fc6ad9a0740 1 Processor -- start 2024-03-26T17:59:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.331+0000 7fc6ad9a0740 1 -- start start 2024-03-26T17:59:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.331+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd819b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.331+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd815a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.331+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd815fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.331+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd819b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.331+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd819b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38756/0 (socket says 172.21.15.105:38756) 2024-03-26T17:59:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.331+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd815a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd815a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:43088/0 (socket says 172.21.15.105:43088) 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a9279640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd815fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a9279640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd815fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52228/0 (socket says 172.21.15.105:52228) 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a8a78640 1 -- 172.21.15.105:0/945398873 learned_addr learned my addr 172.21.15.105:0/945398873 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/945398873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a41aafb0 con 0x556edd81a130 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/945398873 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a41acaf0 con 0x556edd814fc0 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/945398873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a41ab680 con 0x556edd8145d0 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a3fff640 1 -- 172.21.15.105:0/945398873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 msgr2=0x556edd819b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a3fff640 1 --2- 172.21.15.105:0/945398873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd819b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a3fff640 1 -- 172.21.15.105:0/945398873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 msgr2=0x556edd815fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a3fff640 1 --2- 172.21.15.105:0/945398873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd815fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a3fff640 1 -- 172.21.15.105:0/945398873 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556edd80b450 con 0x556edd814fc0 2024-03-26T17:59:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.332+0000 7fc6a9279640 1 --2- 172.21.15.105:0/945398873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd815fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:00.317 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.333+0000 7fc6a3fff640 1 --2- 172.21.15.105:0/945398873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd815a60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x556edd810ab0 tx=0x7fc694009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22c5aa9733b1e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.333+0000 7fc68a7fc640 1 -- 172.21.15.105:0/945398873 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694010430 con 0x556edd814fc0 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.333+0000 7fc68a7fc640 1 -- 172.21.15.105:0/945398873 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc694021070 con 0x556edd814fc0 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.333+0000 7fc68a7fc640 1 -- 172.21.15.105:0/945398873 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69401c690 con 0x556edd814fc0 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/945398873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 msgr2=0x556edd815a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/945398873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd815a60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x556edd810ab0 tx=0x7fc694009260 comp rx=0 tx=0).stop 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/945398873 shutdown_connections 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/945398873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd815fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/945398873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd815a60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/945398873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd819b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.319 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/945398873 >> 172.21.15.105:0/945398873 conn(0x556edd80fb40 msgr2=0x556edd819400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/945398873 shutdown_connections 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.334+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/945398873 wait complete. 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6ad9a0740 1 -- start start 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6ad9a0740 1 Processor -- start 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd8315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd831b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6ad9a0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd896fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd8315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6ad9a0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a41ac420 con 0x556edd81a130 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6ad9a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a41abd50 con 0x556edd814fc0 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6ad9a0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a41a8f90 con 0x556edd8145d0 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6a8a78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd8315f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38768/0 (socket says 172.21.15.105:38768) 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6a3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd831b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.335+0000 7fc6a8a78640 1 -- 172.21.15.105:0/3178132093 learned_addr learned my addr 172.21.15.105:0/3178132093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a9279640 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd896fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6ad9a0740 10 client.0 _ll_register_callbacks cb 0x7fc698005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:00.321 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a9279640 1 -- 172.21.15.105:0/3178132093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 msgr2=0x556edd8315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a9279640 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd8315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a9279640 1 -- 172.21.15.105:0/3178132093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 msgr2=0x556edd831b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a9279640 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd831b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.322 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a9279640 1 -- 172.21.15.105:0/3178132093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556edd8975d0 con 0x556edd81a130 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a3fff640 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd831b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a8a78640 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd8315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc6a9279640 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd896fe0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc698097da0 tx=0x7fc6980b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8636eaeee64fa09 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.336+0000 7fc683fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc683fff640 1 -- 172.21.15.105:0/3178132093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698097690 con 0x556edd81a130 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3178132093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc694021070 con 0x556edd81a130 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc683fff640 1 -- 172.21.15.105:0/3178132093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fc698097830 con 0x556edd81a130 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc683fff640 1 -- 172.21.15.105:0/3178132093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc69809a800 con 0x556edd81a130 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc6ad9a0740 10 client.22938 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3178132093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556edd8988e0 con 0x556edd81a130 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc683fff640 1 -- 172.21.15.105:0/3178132093 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc694021070 con 0x556edd81a130 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc683fff640 1 -- 172.21.15.105:0/3178132093 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fc69801f040 con 0x556edd81a130 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc6ad9a0740 10 client.22938 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.337+0000 7fc6ad9a0740 10 client.22938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.338+0000 7fc6ad9a0740 20 client.22938 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.338+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556edd8990e0 0x556edd89b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.338+0000 7fc6ad9a0740 4 client.22938 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:59:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.338+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3178132093 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556edd780940 con 0x556edd8990e0 2024-03-26T17:59:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.338+0000 7fc6a8a78640 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556edd8990e0 0x556edd89b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.338+0000 7fc683fff640 10 client.22938 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:00.324 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.338+0000 7fc6a8a78640 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556edd8990e0 0x556edd89b4d0 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9333e32c1a6dee7 server_cookie=6e9a94df92df20cd in_seq=0 out_seq=0 2024-03-26T17:59:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc683fff640 1 -- 172.21.15.105:0/3178132093 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556edd780940 con 0x556edd8990e0 2024-03-26T17:59:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc683fff640 10 client.22938 handle_command_reply: tid=0 2024-03-26T17:59:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 client.22938 shutdown 2024-03-26T17:59:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3178132093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 msgr2=0x556edd896fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd896fe0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc698097da0 tx=0x7fc6980b4000 comp rx=0 tx=0).stop 2024-03-26T17:59:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3178132093 shutdown_connections 2024-03-26T17:59:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556edd8990e0 0x556edd89b4d0 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc683fff640 0 client.22938 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556edd81a130 0x556edd896fe0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556edd814fc0 0x556edd831b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 --2- 172.21.15.105:0/3178132093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556edd8145d0 0x556edd8315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3178132093 >> 172.21.15.105:0/3178132093 conn(0x556edd80fb40 msgr2=0x556edd815a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.492+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3178132093 shutdown_connections 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.493+0000 7fc6ad9a0740 1 -- 172.21.15.105:0/3178132093 wait complete. 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.493+0000 7fc6ad9a0740 20 client.22938 trim_cache size 0 max 16384 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:59:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:59:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:59:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 -- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc68c0a3040 msgr2=0x7fc68c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc68c0a3040 0x7fc68c0a5500 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc69c006010 tx=0x7fc69c000f40 comp rx=0 tx=0).stop 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 -- 172.21.15.105:0/462319422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4101490 msgr2=0x7fc6a41a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4101490 0x7fc6a41a1220 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e970 tx=0x7fc69800ed70 comp rx=0 tx=0).stop 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 -- 172.21.15.105:0/462319422 shutdown_connections 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc68c0a3040 0x7fc68c0a5500 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6a4101490 0x7fc6a41a1220 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6a4100af0 0x7fc6a419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 --2- 172.21.15.105:0/462319422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc6a4100190 0x7fc6a4199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 -- 172.21.15.105:0/462319422 >> 172.21.15.105:0/462319422 conn(0x7fc6a40fbd90 msgr2=0x7fc6a40fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 -- 172.21.15.105:0/462319422 shutdown_connections 2024-03-26T17:59:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:00.494+0000 7fc6aacdc640 1 -- 172.21.15.105:0/462319422 wait complete. 2024-03-26T17:59:00.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 60727 7.06 MB/sec execute 343 sec latency 189.120 ms 2024-03-26T17:59:01.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:00 smithi144 ceph-mon[36095]: pgmap v971: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 2.7 MiB/s wr, 252 op/s 2024-03-26T17:59:01.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:00 smithi105 ceph-mon[32280]: pgmap v971: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 2.7 MiB/s wr, 252 op/s 2024-03-26T17:59:01.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:00 smithi135 ceph-mon[35636]: pgmap v971: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 2.7 MiB/s wr, 252 op/s 2024-03-26T17:59:01.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 60859 7.07 MB/sec execute 344 sec latency 126.852 ms 2024-03-26T17:59:02.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 60996 7.07 MB/sec execute 345 sec latency 130.246 ms 2024-03-26T17:59:03.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:02 smithi144 ceph-mon[36095]: pgmap v972: 97 pgs: 97 active+clean; 325 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.1 MiB/s wr, 288 op/s 2024-03-26T17:59:03.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:02 smithi105 ceph-mon[32280]: pgmap v972: 97 pgs: 97 active+clean; 325 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.1 MiB/s wr, 288 op/s 2024-03-26T17:59:03.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:02 smithi135 ceph-mon[35636]: pgmap v972: 97 pgs: 97 active+clean; 325 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 3.1 MiB/s wr, 288 op/s 2024-03-26T17:59:03.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 61136 7.05 MB/sec execute 346 sec latency 134.109 ms 2024-03-26T17:59:04.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 61296 7.04 MB/sec execute 347 sec latency 118.682 ms 2024-03-26T17:59:05.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:04 smithi144 ceph-mon[36095]: pgmap v973: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-03-26T17:59:05.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:04 smithi105 ceph-mon[32280]: pgmap v973: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-03-26T17:59:05.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:04 smithi135 ceph-mon[35636]: pgmap v973: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-03-26T17:59:05.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 61451 7.04 MB/sec execute 348 sec latency 127.187 ms 2024-03-26T17:59:06.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 61622 7.04 MB/sec execute 349 sec latency 118.720 ms 2024-03-26T17:59:07.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:06 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:59:07.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:06 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:59:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:06 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:59:07.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 61774 7.04 MB/sec execute 350 sec latency 130.949 ms 2024-03-26T17:59:08.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:07 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:59:08.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:07 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:59:08.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:07 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:59:08.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:07 smithi135 ceph-mon[35636]: pgmap v974: 97 pgs: 97 active+clean; 320 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2024-03-26T17:59:08.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:07 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:59:08.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:07 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:59:08.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:07 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:59:08.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:07 smithi144 ceph-mon[36095]: pgmap v974: 97 pgs: 97 active+clean; 320 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2024-03-26T17:59:08.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:07 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:59:08.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:07 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:59:08.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:07 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:59:08.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:07 smithi105 ceph-mon[32280]: pgmap v974: 97 pgs: 97 active+clean; 320 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 2.6 MiB/s wr, 290 op/s 2024-03-26T17:59:08.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 61906 7.03 MB/sec execute 351 sec latency 105.652 ms 2024-03-26T17:59:09.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 62053 7.05 MB/sec execute 352 sec latency 117.533 ms 2024-03-26T17:59:10.141 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:09 smithi144 ceph-mon[36095]: pgmap v975: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-03-26T17:59:10.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:09 smithi135 ceph-mon[35636]: pgmap v975: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-03-26T17:59:10.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:09 smithi105 ceph-mon[32280]: pgmap v975: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-03-26T17:59:10.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 62202 7.06 MB/sec execute 353 sec latency 112.431 ms 2024-03-26T17:59:11.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 62315 7.06 MB/sec execute 354 sec latency 124.846 ms 2024-03-26T17:59:12.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:11 smithi135 ceph-mon[35636]: pgmap v976: 97 pgs: 97 active+clean; 323 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-03-26T17:59:12.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:11 smithi144 ceph-mon[36095]: pgmap v976: 97 pgs: 97 active+clean; 323 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-03-26T17:59:12.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:11 smithi105 ceph-mon[32280]: pgmap v976: 97 pgs: 97 active+clean; 323 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-03-26T17:59:12.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 62483 7.08 MB/sec execute 355 sec latency 122.412 ms 2024-03-26T17:59:13.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 62640 7.09 MB/sec execute 356 sec latency 116.144 ms 2024-03-26T17:59:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 -- 172.21.15.105:0/3106542615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 msgr2=0x7f73b8104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 --2- 172.21.15.105:0/3106542615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 0x7f73b8104750 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009900 tx=0x7f73a802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 -- 172.21.15.105:0/3106542615 shutdown_connections 2024-03-26T17:59:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 --2- 172.21.15.105:0/3106542615 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73b81075c0 0x7f73b8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 --2- 172.21.15.105:0/3106542615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73b8104c90 0x7f73b8107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 --2- 172.21.15.105:0/3106542615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 0x7f73b8104750 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 -- 172.21.15.105:0/3106542615 >> 172.21.15.105:0/3106542615 conn(0x7f73b80fbd50 msgr2=0x7f73b80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:13.612 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 -- 172.21.15.105:0/3106542615 shutdown_connections 2024-03-26T17:59:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.611+0000 7f73be627640 1 -- 172.21.15.105:0/3106542615 wait complete. 2024-03-26T17:59:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73be627640 1 Processor -- start 2024-03-26T17:59:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73be627640 1 -- start start 2024-03-26T17:59:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73be627640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 0x7f73b8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73be627640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73b8104c90 0x7f73b8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73be627640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73b81075c0 0x7f73b819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73be627640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73b810d440 con 0x7f73b8102360 2024-03-26T17:59:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73be627640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73b810d2c0 con 0x7f73b8104c90 2024-03-26T17:59:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73be627640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73b810d5c0 con 0x7f73b81075c0 2024-03-26T17:59:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.612+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 0x7f73b8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.614 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 0x7f73b8197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38538/0 (socket says 172.21.15.105:38538) 2024-03-26T17:59:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73bcbc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73b81075c0 0x7f73b819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b7fff640 1 -- 172.21.15.105:0/3070882090 learned_addr learned my addr 172.21.15.105:0/3070882090 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b77fe640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73b8104c90 0x7f73b8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b7fff640 1 -- 172.21.15.105:0/3070882090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73b81075c0 msgr2=0x7f73b819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b7fff640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73b81075c0 0x7f73b819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b7fff640 1 -- 172.21.15.105:0/3070882090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73b8104c90 msgr2=0x7f73b8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b7fff640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73b8104c90 0x7f73b8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b7fff640 1 -- 172.21.15.105:0/3070882090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b819f6d0 con 0x7f73b8102360 2024-03-26T17:59:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b77fe640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73b8104c90 0x7f73b8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73bcbc4640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73b81075c0 0x7f73b819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b7fff640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 0x7f73b8197ab0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f73a8006010 tx=0x7f73a8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b57fa640 1 -- 172.21.15.105:0/3070882090 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8043020 con 0x7f73b8102360 2024-03-26T17:59:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b57fa640 1 -- 172.21.15.105:0/3070882090 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73a80043d0 con 0x7f73b8102360 2024-03-26T17:59:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73b57fa640 1 -- 172.21.15.105:0/3070882090 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8003b80 con 0x7f73b8102360 2024-03-26T17:59:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73be627640 1 -- 172.21.15.105:0/3070882090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b819f960 con 0x7f73b8102360 2024-03-26T17:59:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.613+0000 7f73be627640 1 -- 172.21.15.105:0/3070882090 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b819feb0 con 0x7f73b8102360 2024-03-26T17:59:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.615+0000 7f73b57fa640 1 -- 172.21.15.105:0/3070882090 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f73a8046020 con 0x7f73b8102360 2024-03-26T17:59:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.616+0000 7f73b57fa640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73a00a2fc0 0x7f73a00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.616+0000 7f73c1214740 1 Processor -- start 2024-03-26T17:59:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.616+0000 7f73c1214740 1 -- start start 2024-03-26T17:59:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.616+0000 7f73b77fe640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73a00a2fc0 0x7f73a00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.616+0000 7f73b57fa640 1 -- 172.21.15.105:0/3070882090 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f73a80f1600 con 0x7f73b8102360 2024-03-26T17:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.616+0000 7f73c1214740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fe1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.616+0000 7f73c1214740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe23d0 0x564e50fe6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.616+0000 7f73c1214740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe7540 0x564e50fe99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73c1214740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564e50f420b0 con 0x564e50fe7540 2024-03-26T17:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73c1214740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564e50eea960 con 0x564e50fe23d0 2024-03-26T17:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73c1214740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564e50f3e060 con 0x564e50fe19e0 2024-03-26T17:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fe1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.618 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73bcbc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe7540 0x564e50fe99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fe1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59284/0 (socket says 172.21.15.105:59284) 2024-03-26T17:59:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 -- 172.21.15.105:0/679548785 learned_addr learned my addr 172.21.15.105:0/679548785 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b77fe640 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe23d0 0x564e50fe6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b77fe640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73a00a2fc0 0x7f73a00a5480 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f73b8198fc0 tx=0x7f73ac011860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 -- 172.21.15.105:0/679548785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe23d0 msgr2=0x564e50fe6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe23d0 0x564e50fe6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 -- 172.21.15.105:0/679548785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe7540 msgr2=0x564e50fe99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:13.619 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe7540 0x564e50fe99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 -- 172.21.15.105:0/679548785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e50efcc60 con 0x564e50fe19e0 2024-03-26T17:59:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73bcbc4640 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe7540 0x564e50fe99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f73b7fff640 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fe1e00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f73a80c61d0 tx=0x7f73a80bd920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2aaf7e5024991d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.617+0000 7f739e7fc640 1 -- 172.21.15.105:0/679548785 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80c1e40 con 0x564e50fe19e0 2024-03-26T17:59:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f739e7fc640 1 -- 172.21.15.105:0/679548785 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73a80c6700 con 0x564e50fe19e0 2024-03-26T17:59:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f739e7fc640 1 -- 172.21.15.105:0/679548785 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80c6270 con 0x564e50fe19e0 2024-03-26T17:59:13.620 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f73c1214740 1 -- 172.21.15.105:0/679548785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 msgr2=0x564e50fe1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f73c1214740 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fe1e00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f73a80c61d0 tx=0x7f73a80bd920 comp rx=0 tx=0).stop 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f73c1214740 1 -- 172.21.15.105:0/679548785 shutdown_connections 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f73c1214740 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe7540 0x564e50fe99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f73c1214740 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe23d0 0x564e50fe6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f73c1214740 1 --2- 172.21.15.105:0/679548785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fe1e00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.618+0000 7f73c1214740 1 -- 172.21.15.105:0/679548785 >> 172.21.15.105:0/679548785 conn(0x564e50fdcf50 msgr2=0x564e50fdd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 -- 172.21.15.105:0/679548785 shutdown_connections 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 -- 172.21.15.105:0/679548785 wait complete. 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 -- start start 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 Processor -- start 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe19e0 0x564e50fe4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fe4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe7540 0x564e50fe5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564e50fd8550 con 0x564e50fe23d0 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564e50efddb0 con 0x564e50fe19e0 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.619+0000 7f73c1214740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564e50fd86d0 con 0x564e50fe7540 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73c1214740 10 client.0 _ll_register_callbacks cb 0x564e50f02110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fe4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fe4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38552/0 (socket says 172.21.15.105:38552) 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 -- 172.21.15.105:0/1893142215 learned_addr learned my addr 172.21.15.105:0/1893142215 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe19e0 0x564e50fe4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73bcbc4640 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe7540 0x564e50fe5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 -- 172.21.15.105:0/1893142215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe7540 msgr2=0x564e50fe5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe7540 0x564e50fe5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 -- 172.21.15.105:0/1893142215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe19e0 msgr2=0x564e50fe4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe19e0 0x564e50fe4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 -- 172.21.15.105:0/1893142215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e50c56210 con 0x564e50fe23d0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe19e0 0x564e50fe4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73b77fe640 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fe4840 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0036e0 tx=0x7f73ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11100dcc261e21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73977fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73977fe640 1 -- 172.21.15.105:0/1893142215 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac017040 con 0x564e50fe23d0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73977fe640 1 -- 172.21.15.105:0/1893142215 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f73ac00f4e0 con 0x564e50fe23d0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.620+0000 7f73c1214740 1 -- 172.21.15.105:0/1893142215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a80c6700 con 0x564e50fe23d0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73977fe640 1 -- 172.21.15.105:0/1893142215 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f73ac00f6e0 con 0x564e50fe23d0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73c1214740 10 client.22950 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73c1214740 1 -- 172.21.15.105:0/1893142215 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564e50d22620 con 0x564e50fe23d0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73977fe640 1 -- 172.21.15.105:0/1893142215 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73a80c6700 con 0x564e50fe23d0 2024-03-26T17:59:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73977fe640 1 -- 172.21.15.105:0/1893142215 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f73ac0318b0 con 0x564e50fe23d0 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73c1214740 10 client.22950 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73c1214740 10 client.22950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73c1214740 20 client.22950 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73c1214740 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564e50ff2f90 0x564e50ff5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73c1214740 4 client.22950 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73c1214740 1 -- 172.21.15.105:0/1893142215 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e50f4f5f0 con 0x564e50ff2f90 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.621+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564e50ff2f90 0x564e50ff5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.622+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564e50ff2f90 0x564e50ff5410 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13cfe9e01ec790b9 server_cookie=1e30b19724f43cea in_seq=0 out_seq=0 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.622+0000 7f73977fe640 10 client.22950 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.623+0000 7f73977fe640 1 -- 172.21.15.105:0/1893142215 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564e50f4f5f0 con 0x564e50ff2f90 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.623+0000 7f73977fe640 10 client.22950 handle_command_reply: tid=0 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.623+0000 7f73c1214740 1 client.22950 shutdown 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 -- 172.21.15.105:0/1893142215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 msgr2=0x564e50fe4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.624 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fe4840 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0036e0 tx=0x7f73ac004730 comp rx=0 tx=0).stop 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 -- 172.21.15.105:0/1893142215 shutdown_connections 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564e50ff2f90 0x564e50ff5410 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe7540 0x564e50fe5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fe4840 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 --2- 172.21.15.105:0/1893142215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe19e0 0x564e50fe4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 -- 172.21.15.105:0/1893142215 >> 172.21.15.105:0/1893142215 conn(0x564e50fdcf50 msgr2=0x564e50fd7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73977fe640 0 client.22950 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 -- 172.21.15.105:0/1893142215 shutdown_connections 2024-03-26T17:59:13.625 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 1 -- 172.21.15.105:0/1893142215 wait complete. 2024-03-26T17:59:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.624+0000 7f73c1214740 20 client.22950 trim_cache size 0 max 16384 2024-03-26T17:59:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.645+0000 7f73c1214740 1 Processor -- start 2024-03-26T17:59:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.645+0000 7f73c1214740 1 -- start start 2024-03-26T17:59:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73c1214740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50ff4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73c1214740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50ff52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73c1214740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 0x564e50fd8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73c1214740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a6b30 con 0x564e50fe23d0 2024-03-26T17:59:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73c1214740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a8670 con 0x564e50fe7540 2024-03-26T17:59:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73c1214740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a7200 con 0x564e50fe19e0 2024-03-26T17:59:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50ff52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50ff4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50ff4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59292/0 (socket says 172.21.15.105:59292) 2024-03-26T17:59:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50ff52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38558/0 (socket says 172.21.15.105:38558) 2024-03-26T17:59:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 -- 172.21.15.105:0/1749948407 learned_addr learned my addr 172.21.15.105:0/1749948407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73bcbc4640 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 0x564e50fd8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 -- 172.21.15.105:0/1749948407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 msgr2=0x564e50fd8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 0x564e50fd8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 -- 172.21.15.105:0/1749948407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 msgr2=0x564e50ff52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50ff52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 -- 172.21.15.105:0/1749948407 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e50fddcc0 con 0x564e50fe19e0 2024-03-26T17:59:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b77fe640 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50ff52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.646+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50ff4d70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f73a8005c20 tx=0x7f73a80095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95316f246c2c32a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.647+0000 7f739dffb640 1 -- 172.21.15.105:0/1749948407 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80bbe10 con 0x564e50fe19e0 2024-03-26T17:59:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.647+0000 7f739dffb640 1 -- 172.21.15.105:0/1749948407 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73a80bdcf0 con 0x564e50fe19e0 2024-03-26T17:59:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.647+0000 7f739dffb640 1 -- 172.21.15.105:0/1749948407 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80c6270 con 0x564e50fe19e0 2024-03-26T17:59:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 -- 172.21.15.105:0/1749948407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 msgr2=0x564e50ff4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50ff4d70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f73a8005c20 tx=0x7f73a80095e0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 -- 172.21.15.105:0/1749948407 shutdown_connections 2024-03-26T17:59:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 0x564e50fd8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50ff52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 --2- 172.21.15.105:0/1749948407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50ff4d70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 -- 172.21.15.105:0/1749948407 >> 172.21.15.105:0/1749948407 conn(0x564e50fdcf50 msgr2=0x564e50fe4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 -- 172.21.15.105:0/1749948407 shutdown_connections 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 -- 172.21.15.105:0/1749948407 wait complete. 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 -- start start 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.648+0000 7f73c1214740 1 Processor -- start 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73c1214740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fec940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73c1214740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73c1214740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 0x564e51057340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73c1214740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a7fa0 con 0x564e50fe23d0 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73c1214740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a78d0 con 0x564e50fe7540 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73c1214740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81a4b10 con 0x564e50fe19e0 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73c1214740 10 client.0 _ll_register_callbacks cb 0x7f73ac0041c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fec940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59302/0 (socket says 172.21.15.105:59302) 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 -- 172.21.15.105:0/1206542871 learned_addr learned my addr 172.21.15.105:0/1206542871 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73bcbc4640 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 0x564e51057340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 -- 172.21.15.105:0/1206542871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 msgr2=0x564e51057340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 0x564e51057340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 -- 172.21.15.105:0/1206542871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 msgr2=0x564e50fece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 -- 172.21.15.105:0/1206542871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e50fefec0 con 0x564e50fe19e0 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b77fe640 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73b7fff640 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fec940 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f73a80ba100 tx=0x7f73a80b9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e1b4e154ceb73d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.649+0000 7f73977fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.650+0000 7f73977fe640 1 -- 172.21.15.105:0/1206542871 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80c6700 con 0x564e50fe19e0 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.650+0000 7f73c1214740 1 -- 172.21.15.105:0/1206542871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a80bdcf0 con 0x564e50fe19e0 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.650+0000 7f73977fe640 1 -- 172.21.15.105:0/1206542871 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f73a80bbff0 con 0x564e50fe19e0 2024-03-26T17:59:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.650+0000 7f73977fe640 1 -- 172.21.15.105:0/1206542871 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f73a80d20a0 con 0x564e50fe19e0 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.650+0000 7f73c1214740 10 client.32626 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.650+0000 7f73c1214740 1 -- 172.21.15.105:0/1206542871 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564e51058730 con 0x564e50fe19e0 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.650+0000 7f73977fe640 1 -- 172.21.15.105:0/1206542871 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73a80bdcf0 con 0x564e50fe19e0 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.650+0000 7f73977fe640 1 -- 172.21.15.105:0/1206542871 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f73a80f10d0 con 0x564e50fe19e0 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73c1214740 10 client.32626 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73c1214740 10 client.32626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73c1214740 20 client.32626 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73c1214740 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564e51058d50 0x564e5105b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73c1214740 4 client.32626 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73c1214740 1 -- 172.21.15.105:0/1206542871 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e50f49940 con 0x564e51058d50 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73b77fe640 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564e51058d50 0x564e5105b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73977fe640 10 client.32626 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.651+0000 7f73b77fe640 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564e51058d50 0x564e5105b140 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76bb8da7cbb5ac99 server_cookie=d9108165b10008a1 in_seq=0 out_seq=0 2024-03-26T17:59:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73977fe640 1 -- 172.21.15.105:0/1206542871 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e50f49940 con 0x564e51058d50 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73977fe640 10 client.32626 handle_command_reply: tid=0 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 client.32626 shutdown 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 -- 172.21.15.105:0/1206542871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 msgr2=0x564e50fec940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fec940 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f73a80ba100 tx=0x7f73a80b9f90 comp rx=0 tx=0).stop 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 -- 172.21.15.105:0/1206542871 shutdown_connections 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564e51058d50 0x564e5105b140 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564e50fe7540 0x564e51057340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564e50fe23d0 0x564e50fece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 --2- 172.21.15.105:0/1206542871 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564e50fe19e0 0x564e50fec940 secure :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f73a80ba100 tx=0x7f73a80b9f90 comp rx=0 tx=0).stop 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 -- 172.21.15.105:0/1206542871 >> 172.21.15.105:0/1206542871 conn(0x564e50fdcf50 msgr2=0x564e50fe42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73977fe640 0 client.32626 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 -- 172.21.15.105:0/1206542871 shutdown_connections 2024-03-26T17:59:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 1 -- 172.21.15.105:0/1206542871 wait complete. 2024-03-26T17:59:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.803+0000 7f73c1214740 20 client.32626 trim_cache size 0 max 16384 2024-03-26T17:59:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:59:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:59:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:59:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:59:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.804+0000 7f73be627640 1 -- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73a00a2fc0 msgr2=0x7f73a00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.804+0000 7f73be627640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73a00a2fc0 0x7f73a00a5480 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f73b8198fc0 tx=0x7f73ac011860 comp rx=0 tx=0).stop 2024-03-26T17:59:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.804+0000 7f73be627640 1 -- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 msgr2=0x7f73b8197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.804+0000 7f73be627640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 0x7f73b8197ab0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f73a8006010 tx=0x7f73a8005e40 comp rx=0 tx=0).stop 2024-03-26T17:59:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.805+0000 7f73be627640 1 -- 172.21.15.105:0/3070882090 shutdown_connections 2024-03-26T17:59:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.805+0000 7f73be627640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73a00a2fc0 0x7f73a00a5480 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.805+0000 7f73be627640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73b81075c0 0x7f73b819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.805+0000 7f73be627640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73b8104c90 0x7f73b8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.805+0000 7f73be627640 1 --2- 172.21.15.105:0/3070882090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73b8102360 0x7f73b8197ab0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.805+0000 7f73be627640 1 -- 172.21.15.105:0/3070882090 >> 172.21.15.105:0/3070882090 conn(0x7f73b80fbd50 msgr2=0x7f73b8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.805+0000 7f73be627640 1 -- 172.21.15.105:0/3070882090 shutdown_connections 2024-03-26T17:59:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:13.805+0000 7f73be627640 1 -- 172.21.15.105:0/3070882090 wait complete. 2024-03-26T17:59:14.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:13 smithi135 ceph-mon[35636]: pgmap v977: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.1 MiB/s wr, 239 op/s 2024-03-26T17:59:14.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:13 smithi144 ceph-mon[36095]: pgmap v977: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.1 MiB/s wr, 239 op/s 2024-03-26T17:59:14.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:13 smithi105 ceph-mon[32280]: pgmap v977: 97 pgs: 97 active+clean; 328 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.1 MiB/s wr, 239 op/s 2024-03-26T17:59:14.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 62754 7.09 MB/sec execute 357 sec latency 239.451 ms 2024-03-26T17:59:15.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 62911 7.08 MB/sec execute 358 sec latency 154.787 ms 2024-03-26T17:59:16.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:15 smithi135 ceph-mon[35636]: pgmap v978: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-03-26T17:59:16.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:15 smithi144 ceph-mon[36095]: pgmap v978: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-03-26T17:59:16.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:15 smithi105 ceph-mon[32280]: pgmap v978: 97 pgs: 97 active+clean; 331 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 266 op/s 2024-03-26T17:59:16.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63011 7.08 MB/sec execute 359 sec latency 121.811 ms 2024-03-26T17:59:17.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63123 7.08 MB/sec execute 360 sec latency 128.803 ms 2024-03-26T17:59:18.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:17 smithi135 ceph-mon[35636]: pgmap v979: 97 pgs: 97 active+clean; 334 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 3.3 MiB/s wr, 219 op/s 2024-03-26T17:59:18.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:17 smithi144 ceph-mon[36095]: pgmap v979: 97 pgs: 97 active+clean; 334 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 3.3 MiB/s wr, 219 op/s 2024-03-26T17:59:18.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:17 smithi105 ceph-mon[32280]: pgmap v979: 97 pgs: 97 active+clean; 334 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 3.3 MiB/s wr, 219 op/s 2024-03-26T17:59:18.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63263 7.09 MB/sec execute 361 sec latency 113.600 ms 2024-03-26T17:59:19.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63379 7.08 MB/sec execute 362 sec latency 117.880 ms 2024-03-26T17:59:20.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:19 smithi135 ceph-mon[35636]: pgmap v980: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 4.1 MiB/s wr, 252 op/s 2024-03-26T17:59:20.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:19 smithi144 ceph-mon[36095]: pgmap v980: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 4.1 MiB/s wr, 252 op/s 2024-03-26T17:59:20.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:19 smithi105 ceph-mon[32280]: pgmap v980: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 4.1 MiB/s wr, 252 op/s 2024-03-26T17:59:20.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63525 7.10 MB/sec execute 363 sec latency 151.588 ms 2024-03-26T17:59:21.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:20 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:21.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:20 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:21.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:20 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T17:59:21.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63678 7.11 MB/sec execute 364 sec latency 129.044 ms 2024-03-26T17:59:22.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:21 smithi105 ceph-mon[32280]: pgmap v981: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-03-26T17:59:22.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:21 smithi135 ceph-mon[35636]: pgmap v981: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-03-26T17:59:22.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:21 smithi144 ceph-mon[36095]: pgmap v981: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 3.4 MiB/s wr, 204 op/s 2024-03-26T17:59:22.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63773 7.11 MB/sec execute 365 sec latency 168.398 ms 2024-03-26T17:59:23.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:59:23.298 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:59:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.467+0000 7fd285473640 1 -- 172.21.15.105:0/4263757464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 msgr2=0x7fd280108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.467+0000 7fd285473640 1 --2- 172.21.15.105:0/4263757464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 0x7fd280108c00 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fd278009ca0 tx=0x7fd27802c100 comp rx=0 tx=0).stop 2024-03-26T17:59:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.468+0000 7fd285473640 1 -- 172.21.15.105:0/4263757464 shutdown_connections 2024-03-26T17:59:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.468+0000 7fd285473640 1 --2- 172.21.15.105:0/4263757464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 0x7fd280108c00 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.468+0000 7fd285473640 1 --2- 172.21.15.105:0/4263757464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801019b0 0x7fd280101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.468+0000 7fd285473640 1 --2- 172.21.15.105:0/4263757464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2801007c0 0x7fd280100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.468+0000 7fd285473640 1 -- 172.21.15.105:0/4263757464 >> 172.21.15.105:0/4263757464 conn(0x7fd2800fbd50 msgr2=0x7fd2800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:23.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.468+0000 7fd285473640 1 -- 172.21.15.105:0/4263757464 shutdown_connections 2024-03-26T17:59:23.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.468+0000 7fd285473640 1 -- 172.21.15.105:0/4263757464 wait complete. 2024-03-26T17:59:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.468+0000 7fd285473640 1 Processor -- start 2024-03-26T17:59:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd285473640 1 -- start start 2024-03-26T17:59:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd285473640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007c0 0x7fd280199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd285473640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2801019b0 0x7fd28019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd285473640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 0x7fd2801a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd285473640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd28010b1b0 con 0x7fd280102370 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd285473640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd28010b030 con 0x7fd2801019b0 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd285473640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd28010b330 con 0x7fd2801007c0 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd27effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007c0 0x7fd280199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd27effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007c0 0x7fd280199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59452/0 (socket says 172.21.15.105:59452) 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd27effd640 1 -- 172.21.15.105:0/2923639861 learned_addr learned my addr 172.21.15.105:0/2923639861 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2801019b0 0x7fd28019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 0x7fd2801a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd27effd640 1 -- 172.21.15.105:0/2923639861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2801019b0 msgr2=0x7fd28019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.469+0000 7fd27effd640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2801019b0 0x7fd28019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.470+0000 7fd27effd640 1 -- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 msgr2=0x7fd2801a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.470+0000 7fd27effd640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 0x7fd2801a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.470+0000 7fd27effd640 1 -- 172.21.15.105:0/2923639861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2801a1920 con 0x7fd2801007c0 2024-03-26T17:59:23.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.470+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 0x7fd2801a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:23.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.470+0000 7fd27effd640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007c0 0x7fd280199d10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd27000e9c0 tx=0x7fd27000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.473+0000 7fd267fff640 1 -- 172.21.15.105:0/2923639861 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27000cd80 con 0x7fd2801007c0 2024-03-26T17:59:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.473+0000 7fd267fff640 1 -- 172.21.15.105:0/2923639861 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2700035f0 con 0x7fd2801007c0 2024-03-26T17:59:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.473+0000 7fd285473640 1 -- 172.21.15.105:0/2923639861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2801a1c10 con 0x7fd2801007c0 2024-03-26T17:59:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.473+0000 7fd267fff640 1 -- 172.21.15.105:0/2923639861 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270010690 con 0x7fd2801007c0 2024-03-26T17:59:23.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.473+0000 7fd285473640 1 -- 172.21.15.105:0/2923639861 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801a2110 con 0x7fd2801007c0 2024-03-26T17:59:23.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.476+0000 7fd267fff640 1 -- 172.21.15.105:0/2923639861 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd270004020 con 0x7fd2801007c0 2024-03-26T17:59:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.476+0000 7fd28805d740 1 Processor -- start 2024-03-26T17:59:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.476+0000 7fd267fff640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2680a3050 0x7fd2680a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.476+0000 7fd28805d740 1 -- start start 2024-03-26T17:59:23.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2680a3050 0x7fd2680a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd267fff640 1 -- 172.21.15.105:0/2923639861 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd2700cf780 con 0x7fd2801007c0 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd28805d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd28805d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd28805d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd28805d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55eb2a448980 con 0x55eb2a4e9450 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd28805d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55eb2a3f0be0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd28805d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb2a444930 con 0x55eb2a4e38f0 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4eb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59618/0 (socket says 172.21.15.105:59618) 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 -- 172.21.15.105:0/4055405776 learned_addr learned my addr 172.21.15.105:0/4055405776 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27effd640 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2680a3050 0x7fd2680a5510 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd278037040 tx=0x7fd278032850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 -- 172.21.15.105:0/4055405776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 msgr2=0x55eb2a4e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 -- 172.21.15.105:0/4055405776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e42e0 msgr2=0x55eb2a4e8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4e8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.477+0000 7fd27f7fe640 1 -- 172.21.15.105:0/4055405776 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2a2267d0 con 0x55eb2a4e9450 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.478+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4eb8e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fd278009ca0 tx=0x7fd278048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c1a0bb3f397534 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.478+0000 7fd2657fa640 1 -- 172.21.15.105:0/4055405776 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278039040 con 0x55eb2a4e9450 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.478+0000 7fd2657fa640 1 -- 172.21.15.105:0/4055405776 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd278052070 con 0x55eb2a4e9450 2024-03-26T17:59:23.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.478+0000 7fd2657fa640 1 -- 172.21.15.105:0/4055405776 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27804dbc0 con 0x55eb2a4e9450 2024-03-26T17:59:23.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 -- 172.21.15.105:0/4055405776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 msgr2=0x55eb2a4eb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4eb8e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fd278009ca0 tx=0x7fd278048000 comp rx=0 tx=0).stop 2024-03-26T17:59:23.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 -- 172.21.15.105:0/4055405776 shutdown_connections 2024-03-26T17:59:23.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4eb8e0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4e8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 --2- 172.21.15.105:0/4055405776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 -- 172.21.15.105:0/4055405776 >> 172.21.15.105:0/4055405776 conn(0x55eb2a4dee60 msgr2=0x55eb2a4df260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 -- 172.21.15.105:0/4055405776 shutdown_connections 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 -- 172.21.15.105:0/4055405776 wait complete. 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 -- start start 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.479+0000 7fd28805d740 1 Processor -- start 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd28805d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd28805d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd28805d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd28805d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55eb2a4f4bb0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd28805d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55eb2a404160 con 0x55eb2a4e38f0 2024-03-26T17:59:23.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd28805d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55eb2a4f4d30 con 0x55eb2a4e9450 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd28805d740 10 client.0 _ll_register_callbacks cb 0x55eb2a440720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59620/0 (socket says 172.21.15.105:59620) 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 -- 172.21.15.105:0/2112388318 learned_addr learned my addr 172.21.15.105:0/2112388318 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 -- 172.21.15.105:0/2112388318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e9450 msgr2=0x55eb2a4fd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 -- 172.21.15.105:0/2112388318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 msgr2=0x55eb2a4f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 -- 172.21.15.105:0/2112388318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2a15c3f0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4f67f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fd2740126e0 tx=0x7fd274012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90504fa0d02293d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:23.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.480+0000 7fd262ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:23.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.482+0000 7fd262ff5640 1 -- 172.21.15.105:0/2112388318 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274012190 con 0x55eb2a4e42e0 2024-03-26T17:59:23.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.482+0000 7fd262ff5640 1 -- 172.21.15.105:0/2112388318 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd27400f8c0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.482+0000 7fd28805d740 1 -- 172.21.15.105:0/2112388318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd278052070 con 0x55eb2a4e42e0 2024-03-26T17:59:23.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.482+0000 7fd262ff5640 1 -- 172.21.15.105:0/2112388318 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd27400fac0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.482+0000 7fd28805d740 10 client.22968 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:23.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.482+0000 7fd28805d740 1 -- 172.21.15.105:0/2112388318 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eb2a11c0f0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd262ff5640 1 -- 172.21.15.105:0/2112388318 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd278052070 con 0x55eb2a4e42e0 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd262ff5640 1 -- 172.21.15.105:0/2112388318 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fd274026c90 con 0x55eb2a4e42e0 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd28805d740 10 client.22968 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd28805d740 10 client.22968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd28805d740 20 client.22968 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd28805d740 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb2a5021d0 0x55eb2a504690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd28805d740 4 client.22968 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd28805d740 1 -- 172.21.15.105:0/2112388318 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb2a455ec0 con 0x55eb2a5021d0 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.483+0000 7fd27effd640 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb2a5021d0 0x55eb2a504690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.484+0000 7fd27effd640 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb2a5021d0 0x55eb2a504690 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1899d613d5325fc server_cookie=5d727378942db26e in_seq=0 out_seq=0 2024-03-26T17:59:23.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.484+0000 7fd262ff5640 10 client.22968 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:23.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd262ff5640 1 -- 172.21.15.105:0/2112388318 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55eb2a455ec0 con 0x55eb2a5021d0 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd262ff5640 10 client.22968 handle_command_reply: tid=0 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 client.22968 shutdown 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 -- 172.21.15.105:0/2112388318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 msgr2=0x55eb2a4f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4f67f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fd2740126e0 tx=0x7fd274012ba0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 -- 172.21.15.105:0/2112388318 shutdown_connections 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb2a5021d0 0x55eb2a504690 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4f67f0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 --2- 172.21.15.105:0/2112388318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd262ff5640 0 client.22968 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 -- 172.21.15.105:0/2112388318 >> 172.21.15.105:0/2112388318 conn(0x55eb2a4dee60 msgr2=0x55eb2a4de080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 -- 172.21.15.105:0/2112388318 shutdown_connections 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.486+0000 7fd28805d740 1 -- 172.21.15.105:0/2112388318 wait complete. 2024-03-26T17:59:23.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.487+0000 7fd28805d740 20 client.22968 trim_cache size 0 max 16384 2024-03-26T17:59:23.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.515+0000 7fd28805d740 1 Processor -- start 2024-03-26T17:59:23.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.515+0000 7fd28805d740 1 -- start start 2024-03-26T17:59:23.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.515+0000 7fd28805d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.515+0000 7fd28805d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd28805d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e9450 0x55eb2a4e5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd28805d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801a6a90 con 0x55eb2a4e42e0 2024-03-26T17:59:23.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd28805d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801a85d0 con 0x55eb2a4e9450 2024-03-26T17:59:23.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd28805d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801a7160 con 0x55eb2a4e38f0 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59628/0 (socket says 172.21.15.105:59628) 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 -- 172.21.15.105:0/144656838 learned_addr learned my addr 172.21.15.105:0/144656838 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e9450 0x55eb2a4e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27effd640 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 -- 172.21.15.105:0/144656838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 msgr2=0x55eb2a4e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 -- 172.21.15.105:0/144656838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e9450 msgr2=0x55eb2a4e5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e9450 0x55eb2a4e5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 -- 172.21.15.105:0/144656838 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2a4e87b0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fd7f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fd2740123d0 tx=0x7fd274010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3e37aa207bd99a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.516+0000 7fd264ff9640 1 -- 172.21.15.105:0/144656838 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27400fe70 con 0x55eb2a4e42e0 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.517+0000 7fd264ff9640 1 -- 172.21.15.105:0/144656838 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd27400f8d0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.517+0000 7fd264ff9640 1 -- 172.21.15.105:0/144656838 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740038f0 con 0x55eb2a4e42e0 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.517+0000 7fd28805d740 1 -- 172.21.15.105:0/144656838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 msgr2=0x55eb2a4fd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.517+0000 7fd28805d740 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fd7f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fd2740123d0 tx=0x7fd274010470 comp rx=0 tx=0).stop 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 -- 172.21.15.105:0/144656838 shutdown_connections 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e9450 0x55eb2a4e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fd7f0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 --2- 172.21.15.105:0/144656838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 -- 172.21.15.105:0/144656838 >> 172.21.15.105:0/144656838 conn(0x55eb2a4dee60 msgr2=0x55eb2a4e49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 -- 172.21.15.105:0/144656838 shutdown_connections 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 -- 172.21.15.105:0/144656838 wait complete. 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 -- start start 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 Processor -- start 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4fac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801a7f00 con 0x55eb2a4e9450 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801a7830 con 0x55eb2a4e38f0 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801a4a90 con 0x55eb2a4e42e0 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.518+0000 7fd28805d740 10 client.0 _ll_register_callbacks cb 0x7fd274009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59644/0 (socket says 172.21.15.105:59644) 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 -- 172.21.15.105:0/276277002 learned_addr learned my addr 172.21.15.105:0/276277002 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27effd640 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4fac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27e7fc640 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 -- 172.21.15.105:0/276277002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e42e0 msgr2=0x55eb2a4fb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 -- 172.21.15.105:0/276277002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 msgr2=0x55eb2a4fac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:23.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4fac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 -- 172.21.15.105:0/276277002 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2a4fc760 con 0x55eb2a4e9450 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd27f7fe640 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fc170 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fd278009ca0 tx=0x7fd2780043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a66182d1fe40b73 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.519+0000 7fd262ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.520+0000 7fd262ff5640 1 -- 172.21.15.105:0/276277002 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27805b070 con 0x55eb2a4e9450 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.520+0000 7fd28805d740 1 -- 172.21.15.105:0/276277002 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd27400f8d0 con 0x55eb2a4e9450 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.520+0000 7fd262ff5640 1 -- 172.21.15.105:0/276277002 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd27804d480 con 0x55eb2a4e9450 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.520+0000 7fd262ff5640 1 -- 172.21.15.105:0/276277002 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd278039040 con 0x55eb2a4e9450 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.520+0000 7fd28805d740 10 client.22974 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.520+0000 7fd28805d740 1 -- 172.21.15.105:0/276277002 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eb2a4fdac0 con 0x55eb2a4e9450 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd262ff5640 1 -- 172.21.15.105:0/276277002 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd27400f8d0 con 0x55eb2a4e9450 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd262ff5640 1 -- 172.21.15.105:0/276277002 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fd27803f050 con 0x55eb2a4e9450 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd28805d740 10 client.22974 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd28805d740 10 client.22974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:23.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd28805d740 20 client.22974 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:23.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd28805d740 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb2a568920 0x55eb2a56ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd28805d740 4 client.22974 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:59:23.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd28805d740 1 -- 172.21.15.105:0/276277002 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb2a450210 con 0x55eb2a568920 2024-03-26T17:59:23.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.521+0000 7fd27effd640 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb2a568920 0x55eb2a56ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.522+0000 7fd27effd640 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb2a568920 0x55eb2a56ad40 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260c38c0409351d3 server_cookie=cf2375aa8579a7bf in_seq=0 out_seq=0 2024-03-26T17:59:23.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.522+0000 7fd262ff5640 10 client.22974 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.526+0000 7fd262ff5640 1 -- 172.21.15.105:0/276277002 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eb2a450210 con 0x55eb2a568920 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd262ff5640 10 client.22974 handle_command_reply: tid=0 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 client.22974 shutdown 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 -- 172.21.15.105:0/276277002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 msgr2=0x55eb2a4fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fc170 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fd278009ca0 tx=0x7fd2780043e0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 -- 172.21.15.105:0/276277002 shutdown_connections 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55eb2a568920 0x55eb2a56ad40 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55eb2a4e9450 0x55eb2a4fc170 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55eb2a4e42e0 0x55eb2a4fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 --2- 172.21.15.105:0/276277002 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eb2a4e38f0 0x55eb2a4fac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 -- 172.21.15.105:0/276277002 >> 172.21.15.105:0/276277002 conn(0x55eb2a4dee60 msgr2=0x55eb2a4f6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd262ff5640 0 client.22974 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 -- 172.21.15.105:0/276277002 shutdown_connections 2024-03-26T17:59:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 1 -- 172.21.15.105:0/276277002 wait complete. 2024-03-26T17:59:23.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.527+0000 7fd28805d740 20 client.22974 trim_cache size 0 max 16384 2024-03-26T17:59:23.511 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:59:23.512 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:59:23.512 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:59:23.512 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 -- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2680a3050 msgr2=0x7fd2680a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2680a3050 0x7fd2680a5510 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd278037040 tx=0x7fd278032850 comp rx=0 tx=0).stop 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 -- 172.21.15.105:0/2923639861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007c0 msgr2=0x7fd280199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007c0 0x7fd280199d10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd27000e9c0 tx=0x7fd27000ee80 comp rx=0 tx=0).stop 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 -- 172.21.15.105:0/2923639861 shutdown_connections 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd2680a3050 0x7fd2680a5510 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd280102370 0x7fd2801a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd2801019b0 0x7fd28019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 --2- 172.21.15.105:0/2923639861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2801007c0 0x7fd280199d10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 -- 172.21.15.105:0/2923639861 >> 172.21.15.105:0/2923639861 conn(0x7fd2800fbd50 msgr2=0x7fd280104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 -- 172.21.15.105:0/2923639861 shutdown_connections 2024-03-26T17:59:23.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.529+0000 7fd285473640 1 -- 172.21.15.105:0/2923639861 wait complete. 2024-03-26T17:59:23.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:59:23.530 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:23.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63890 7.11 MB/sec execute 366 sec latency 140.316 ms 2024-03-26T17:59:23.633 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.633 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.633 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:23 smithi105 ceph-mon[32280]: pgmap v982: 97 pgs: 97 active+clean; 338 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-03-26T17:59:23.633 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.633 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:23 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 -- 172.21.15.105:0/2532321195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c1007e0 msgr2=0x7fdb4c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 --2- 172.21.15.105:0/2532321195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c1007e0 0x7fdb4c100c00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800d910 tx=0x7fdb4802dbb0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 -- 172.21.15.105:0/2532321195 shutdown_connections 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 --2- 172.21.15.105:0/2532321195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb4c102390 0x7fdb4c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 --2- 172.21.15.105:0/2532321195 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb4c1019d0 0x7fdb4c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 --2- 172.21.15.105:0/2532321195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c1007e0 0x7fdb4c100c00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 -- 172.21.15.105:0/2532321195 >> 172.21.15.105:0/2532321195 conn(0x7fdb4c0fbd50 msgr2=0x7fdb4c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 -- 172.21.15.105:0/2532321195 shutdown_connections 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 -- 172.21.15.105:0/2532321195 wait complete. 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.693+0000 7fdb538e8640 1 Processor -- start 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb538e8640 1 -- start start 2024-03-26T17:59:23.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb538e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb4c1007e0 0x7fdb4c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb538e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb4c1019d0 0x7fdb4c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb538e8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c102390 0x7fdb4c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb538e8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c10b220 con 0x7fdb4c1007e0 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb538e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c10b0a0 con 0x7fdb4c1019d0 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb538e8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c10b3a0 con 0x7fdb4c102390 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb4c1007e0 0x7fdb4c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb4c1007e0 0x7fdb4c195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59658/0 (socket says 172.21.15.105:59658) 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 -- 172.21.15.105:0/3373830833 learned_addr learned my addr 172.21.15.105:0/3373830833 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb50e83640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb4c1019d0 0x7fdb4c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51e85640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c102390 0x7fdb4c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 -- 172.21.15.105:0/3373830833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c102390 msgr2=0x7fdb4c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c102390 0x7fdb4c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 -- 172.21.15.105:0/3373830833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb4c1019d0 msgr2=0x7fdb4c195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:23.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb4c1019d0 0x7fdb4c195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:23.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 -- 172.21.15.105:0/3373830833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c19d590 con 0x7fdb4c1007e0 2024-03-26T17:59:23.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.694+0000 7fdb51684640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb4c1007e0 0x7fdb4c195970 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009230 tx=0x7fdb48004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:23.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.695+0000 7fdb427fc640 1 -- 172.21.15.105:0/3373830833 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48043020 con 0x7fdb4c1007e0 2024-03-26T17:59:23.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.695+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c19d820 con 0x7fdb4c1007e0 2024-03-26T17:59:23.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.695+0000 7fdb427fc640 1 -- 172.21.15.105:0/3373830833 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb48005400 con 0x7fdb4c1007e0 2024-03-26T17:59:23.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.695+0000 7fdb427fc640 1 -- 172.21.15.105:0/3373830833 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480367a0 con 0x7fdb4c1007e0 2024-03-26T17:59:23.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.695+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c19dd60 con 0x7fdb4c1007e0 2024-03-26T17:59:23.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.696+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4c107bd0 con 0x7fdb4c1007e0 2024-03-26T17:59:23.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.698+0000 7fdb427fc640 1 -- 172.21.15.105:0/3373830833 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdb48046020 con 0x7fdb4c1007e0 2024-03-26T17:59:23.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.699+0000 7fdb427fc640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb380a2fa0 0x7fdb380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:23.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.699+0000 7fdb427fc640 1 -- 172.21.15.105:0/3373830833 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fdb480f2670 con 0x7fdb4c1007e0 2024-03-26T17:59:23.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.699+0000 7fdb50e83640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb380a2fa0 0x7fdb380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:23.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.699+0000 7fdb50e83640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb380a2fa0 0x7fdb380a5460 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c196e80 tx=0x7fdb3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:23.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:23.708+0000 7fdb427fc640 1 -- 172.21.15.105:0/3373830833 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdb4804d070 con 0x7fdb4c1007e0 2024-03-26T17:59:23.839 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.839 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.839 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:23 smithi144 ceph-mon[36095]: pgmap v982: 97 pgs: 97 active+clean; 338 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-03-26T17:59:23.840 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.840 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:23 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.906 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.906 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.906 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:23 smithi135 ceph-mon[35636]: pgmap v982: 97 pgs: 97 active+clean; 338 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.9 MiB/s wr, 221 op/s 2024-03-26T17:59:23.906 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:23.906 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:23 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.066+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb4c02d240 con 0x7fdb4c1007e0 2024-03-26T17:59:24.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.067+0000 7fdb427fc640 1 -- 172.21.15.105:0/3373830833 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11941 (secure 0 0 0) 0x7fdb480bf460 con 0x7fdb4c1007e0 2024-03-26T17:59:24.050 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:59:24.050 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:58:34.693267+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_32441":{"gid":32441,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1622534523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1622534523},{"type":"v1","addr":"172.21.15.135:6835","nonce":1622534523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:59:24.050 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 41 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb380a2fa0 msgr2=0x7fdb380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb380a2fa0 0x7fdb380a5460 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c196e80 tx=0x7fdb3c005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb4c1007e0 msgr2=0x7fdb4c195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb4c1007e0 0x7fdb4c195970 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009230 tx=0x7fdb48004fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 shutdown_connections 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdb380a2fa0 0x7fdb380a5460 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb4c102390 0x7fdb4c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb4c1019d0 0x7fdb4c195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 --2- 172.21.15.105:0/3373830833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb4c1007e0 0x7fdb4c195970 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 >> 172.21.15.105:0/3373830833 conn(0x7fdb4c0fbd50 msgr2=0x7fdb4c104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 shutdown_connections 2024-03-26T17:59:24.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.069+0000 7fdb538e8640 1 -- 172.21.15.105:0/3373830833 wait complete. 2024-03-26T17:59:24.067 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:59:24.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.271+0000 7f03c404b640 1 -- 172.21.15.105:0/505218052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc108940 msgr2=0x7f03bc10ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.271+0000 7f03c404b640 1 --2- 172.21.15.105:0/505218052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc108940 0x7f03bc10ad70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009900 tx=0x7f03ac02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.271+0000 7f03c404b640 1 -- 172.21.15.105:0/505218052 shutdown_connections 2024-03-26T17:59:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.271+0000 7f03c404b640 1 --2- 172.21.15.105:0/505218052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03bc10dc60 0x7f03bc10e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.271+0000 7f03c404b640 1 --2- 172.21.15.105:0/505218052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03bc10b2b0 0x7f03bc10d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.271+0000 7f03c404b640 1 --2- 172.21.15.105:0/505218052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc108940 0x7f03bc10ad70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.271+0000 7f03c404b640 1 -- 172.21.15.105:0/505218052 >> 172.21.15.105:0/505218052 conn(0x7f03bc06f760 msgr2=0x7f03bc071ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.272+0000 7f03c404b640 1 -- 172.21.15.105:0/505218052 shutdown_connections 2024-03-26T17:59:24.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.272+0000 7f03c404b640 1 -- 172.21.15.105:0/505218052 wait complete. 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c404b640 1 Processor -- start 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c404b640 1 -- start start 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c404b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03bc108940 0x7f03bc1a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c404b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc10b2b0 0x7f03bc1a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c404b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03bc10dc60 0x7f03bc1adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c404b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc11c1f0 con 0x7f03bc10dc60 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c404b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc11c070 con 0x7f03bc108940 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c404b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc11c370 con 0x7f03bc10b2b0 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c15e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc10b2b0 0x7f03bc1a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03bc108940 0x7f03bc1a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c25e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03bc10dc60 0x7f03bc1adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03bc108940 0x7f03bc1a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45936/0 (socket says 172.21.15.105:45936) 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c1de7640 1 -- 172.21.15.105:0/850524507 learned_addr learned my addr 172.21.15.105:0/850524507 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.273+0000 7f03c25e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03bc10dc60 0x7f03bc1adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59672/0 (socket says 172.21.15.105:59672) 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03c25e8640 1 -- 172.21.15.105:0/850524507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc10b2b0 msgr2=0x7f03bc1a6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03c25e8640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc10b2b0 0x7f03bc1a6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03c25e8640 1 -- 172.21.15.105:0/850524507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03bc108940 msgr2=0x7f03bc1a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03c25e8640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03bc108940 0x7f03bc1a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03c25e8640 1 -- 172.21.15.105:0/850524507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc1ae520 con 0x7f03bc10dc60 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03c25e8640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03bc10dc60 0x7f03bc1adeb0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009fd0 tx=0x7f03b400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03b2ffd640 1 -- 172.21.15.105:0/850524507 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b400ce20 con 0x7f03bc10dc60 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03c404b640 1 -- 172.21.15.105:0/850524507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc1ae810 con 0x7f03bc10dc60 2024-03-26T17:59:24.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03b2ffd640 1 -- 172.21.15.105:0/850524507 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b4005020 con 0x7f03bc10dc60 2024-03-26T17:59:24.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.274+0000 7f03c404b640 1 -- 172.21.15.105:0/850524507 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc1aedb0 con 0x7f03bc10dc60 2024-03-26T17:59:24.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.275+0000 7f03b2ffd640 1 -- 172.21.15.105:0/850524507 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4010690 con 0x7f03bc10dc60 2024-03-26T17:59:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.277+0000 7f03b2ffd640 1 -- 172.21.15.105:0/850524507 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f03b4004a00 con 0x7f03bc10dc60 2024-03-26T17:59:24.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c6c32740 1 Processor -- start 2024-03-26T17:59:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c6c32740 1 -- start start 2024-03-26T17:59:24.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787625e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787626420 0x55978762af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x55978762da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c6c32740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55978758a970 con 0x55978762b590 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c6c32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559787532d70 con 0x559787625a30 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c6c32740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559787586920 con 0x559787626420 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.278+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787626420 0x55978762af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787626420 0x55978762af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59534/0 (socket says 172.21.15.105:59534) 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c1de7640 1 -- 172.21.15.105:0/3987469967 learned_addr learned my addr 172.21.15.105:0/3987469967 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c15e6640 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787625e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c15e6640 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x55978762da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c1de7640 1 -- 172.21.15.105:0/3987469967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 msgr2=0x559787625e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c1de7640 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787625e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c1de7640 1 -- 172.21.15.105:0/3987469967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 msgr2=0x55978762da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c1de7640 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x55978762da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c1de7640 1 -- 172.21.15.105:0/3987469967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978729e3f0 con 0x559787626420 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03b2ffd640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03a80a2e80 0x7f03a80a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03c15e6640 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x55978762da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.279+0000 7f03b2ffd640 1 -- 172.21.15.105:0/850524507 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f03b40cf970 con 0x7f03bc10dc60 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.280+0000 7f03c1de7640 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787626420 0x55978762af20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009430 tx=0x7f03ac009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204ba24909e8e887 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.280+0000 7f03c15e6640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03a80a2e80 0x7f03a80a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.280+0000 7f03a7fff640 1 -- 172.21.15.105:0/3987469967 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac045020 con 0x559787626420 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.280+0000 7f03a7fff640 1 -- 172.21.15.105:0/3987469967 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac004040 con 0x559787626420 2024-03-26T17:59:24.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.280+0000 7f03a7fff640 1 -- 172.21.15.105:0/3987469967 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac005560 con 0x559787626420 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 -- 172.21.15.105:0/3987469967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787626420 msgr2=0x55978762af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c15e6640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03a80a2e80 0x7f03a80a5340 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x559787626c80 tx=0x7f03b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787626420 0x55978762af20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009430 tx=0x7f03ac009560 comp rx=0 tx=0).stop 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 -- 172.21.15.105:0/3987469967 shutdown_connections 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x55978762da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787626420 0x55978762af20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 --2- 172.21.15.105:0/3987469967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787625e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 -- 172.21.15.105:0/3987469967 >> 172.21.15.105:0/3987469967 conn(0x559787620fa0 msgr2=0x5597876213a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 -- 172.21.15.105:0/3987469967 shutdown_connections 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.281+0000 7f03c6c32740 1 -- 172.21.15.105:0/3987469967 wait complete. 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.282+0000 7f03c6c32740 1 -- start start 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.282+0000 7f03c6c32740 1 Processor -- start 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.282+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787625a30 0x55978762fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.282+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787630430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.282+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x559787637470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.282+0000 7f03c6c32740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55978762e4e0 con 0x55978762b590 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.282+0000 7f03c6c32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559787546310 con 0x559787626420 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.282+0000 7f03c6c32740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55978762e660 con 0x559787625a30 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c15e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787630430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c15e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787630430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45956/0 (socket says 172.21.15.105:45956) 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787625a30 0x55978762fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c15e6640 1 -- 172.21.15.105:0/2294303582 learned_addr learned my addr 172.21.15.105:0/2294303582 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c25e8640 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x559787637470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c6c32740 10 client.0 _ll_register_callbacks cb 0x559787581c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:24.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c15e6640 1 -- 172.21.15.105:0/2294303582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787625a30 msgr2=0x55978762fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c15e6640 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787625a30 0x55978762fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c15e6640 1 -- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 msgr2=0x559787637470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c15e6640 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x559787637470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c15e6640 1 -- 172.21.15.105:0/2294303582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597873687d0 con 0x559787626420 2024-03-26T17:59:24.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c1de7640 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787625a30 0x55978762fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.283+0000 7f03c25e8640 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x559787637470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.284+0000 7f03c15e6640 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787630430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f03b8003870 tx=0x7f03b8009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d422b2d513a29e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.284+0000 7f03a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.284+0000 7f03a57fa640 1 -- 172.21.15.105:0/2294303582 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8009a70 con 0x559787626420 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.284+0000 7f03a57fa640 1 -- 172.21.15.105:0/2294303582 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f03b80209c0 con 0x559787626420 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.284+0000 7f03c6c32740 1 -- 172.21.15.105:0/2294303582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03ac004040 con 0x559787626420 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.284+0000 7f03a57fa640 1 -- 172.21.15.105:0/2294303582 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f03b8020bc0 con 0x559787626420 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.284+0000 7f03c6c32740 10 client.32675 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.284+0000 7f03c6c32740 1 -- 172.21.15.105:0/2294303582 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55978763a7c0 con 0x559787626420 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03a57fa640 1 -- 172.21.15.105:0/2294303582 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac004040 con 0x559787626420 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03a57fa640 1 -- 172.21.15.105:0/2294303582 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f03b8004ac0 con 0x559787626420 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03c6c32740 10 client.32675 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03c6c32740 10 client.32675 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03c6c32740 20 client.32675 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:24.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03c6c32740 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55978763bef0 0x55978763e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03c6c32740 4 client.32675 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:24.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03c6c32740 1 -- 172.21.15.105:0/2294303582 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559787597eb0 con 0x55978763bef0 2024-03-26T17:59:24.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.285+0000 7f03c1de7640 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55978763bef0 0x55978763e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.286+0000 7f03a57fa640 10 client.32675 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:59:24.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.286+0000 7f03c1de7640 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55978763bef0 0x55978763e370 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1018140e793989da server_cookie=29a5da8de8c88511 in_seq=0 out_seq=0 2024-03-26T17:59:24.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.287+0000 7f03a57fa640 1 -- 172.21.15.105:0/2294303582 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559787597eb0 con 0x55978763bef0 2024-03-26T17:59:24.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.287+0000 7f03a57fa640 10 client.32675 handle_command_reply: tid=0 2024-03-26T17:59:24.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.287+0000 7f03c6c32740 1 client.32675 shutdown 2024-03-26T17:59:24.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 -- 172.21.15.105:0/2294303582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 msgr2=0x559787630430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787630430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f03b8003870 tx=0x7f03b8009770 comp rx=0 tx=0).stop 2024-03-26T17:59:24.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 -- 172.21.15.105:0/2294303582 shutdown_connections 2024-03-26T17:59:24.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55978763bef0 0x55978763e370 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55978762b590 0x559787637470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787630430 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 --2- 172.21.15.105:0/2294303582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559787625a30 0x55978762fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03a57fa640 0 client.32675 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:59:24.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 -- 172.21.15.105:0/2294303582 >> 172.21.15.105:0/2294303582 conn(0x559787620fa0 msgr2=0x5597876201c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 -- 172.21.15.105:0/2294303582 shutdown_connections 2024-03-26T17:59:24.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 1 -- 172.21.15.105:0/2294303582 wait complete. 2024-03-26T17:59:24.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.288+0000 7f03c6c32740 20 client.32675 trim_cache size 0 max 16384 2024-03-26T17:59:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.309+0000 7f03c6c32740 1 Processor -- start 2024-03-26T17:59:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.309+0000 7f03c6c32740 1 -- start start 2024-03-26T17:59:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787625a30 0x559787621380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787637430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 0x55978762fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c6c32740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b7a10 con 0x559787625a30 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c6c32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b9550 con 0x559787626420 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c6c32740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b80e0 con 0x55978762b590 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787625a30 0x559787621380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787625a30 0x559787621380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59704/0 (socket says 172.21.15.105:59704) 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c1de7640 1 -- 172.21.15.105:0/658299619 learned_addr learned my addr 172.21.15.105:0/658299619 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c15e6640 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787637430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c25e8640 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 0x55978762fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c1de7640 1 -- 172.21.15.105:0/658299619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 msgr2=0x55978762fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c1de7640 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 0x55978762fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c1de7640 1 -- 172.21.15.105:0/658299619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 msgr2=0x559787637430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c1de7640 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787637430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.310+0000 7f03c1de7640 1 -- 172.21.15.105:0/658299619 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978762ac70 con 0x559787625a30 2024-03-26T17:59:24.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.311+0000 7f03c1de7640 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787625a30 0x559787621380 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f03ac005ec0 tx=0x7f03ac02cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f067aaa68de565 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.311+0000 7f03a77fe640 1 -- 172.21.15.105:0/658299619 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac045040 con 0x559787625a30 2024-03-26T17:59:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.311+0000 7f03a77fe640 1 -- 172.21.15.105:0/658299619 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac02c7e0 con 0x559787625a30 2024-03-26T17:59:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.311+0000 7f03a77fe640 1 -- 172.21.15.105:0/658299619 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac041850 con 0x559787625a30 2024-03-26T17:59:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 -- 172.21.15.105:0/658299619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787625a30 msgr2=0x559787621380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787625a30 0x559787621380 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f03ac005ec0 tx=0x7f03ac02cda0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 -- 172.21.15.105:0/658299619 shutdown_connections 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 0x55978762fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787626420 0x559787637430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 --2- 172.21.15.105:0/658299619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787625a30 0x559787621380 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 -- 172.21.15.105:0/658299619 >> 172.21.15.105:0/658299619 conn(0x559787620fa0 msgr2=0x559787626ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 -- 172.21.15.105:0/658299619 shutdown_connections 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.312+0000 7f03c6c32740 1 -- 172.21.15.105:0/658299619 wait complete. 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.313+0000 7f03c6c32740 1 -- start start 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.313+0000 7f03c6c32740 1 Processor -- start 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.313+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787633660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.313+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787626420 0x559787633c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.313+0000 7f03c6c32740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 0x559787636990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.313+0000 7f03c15e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787626420 0x559787633c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.313+0000 7f03c15e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787626420 0x559787633c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59714/0 (socket says 172.21.15.105:59714) 2024-03-26T17:59:24.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787633660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c1de7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787633660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45974/0 (socket says 172.21.15.105:45974) 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c15e6640 1 -- 172.21.15.105:0/293964971 learned_addr learned my addr 172.21.15.105:0/293964971 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b8e80 con 0x559787626420 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b87b0 con 0x559787625a30 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c25e8640 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 0x559787636990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc1b59f0 con 0x55978762b590 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c6c32740 10 client.0 _ll_register_callbacks cb 0x7f03b8009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c15e6640 1 -- 172.21.15.105:0/293964971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 msgr2=0x559787636990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c15e6640 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 0x559787636990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c15e6640 1 -- 172.21.15.105:0/293964971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 msgr2=0x559787633660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c15e6640 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787633660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c15e6640 1 -- 172.21.15.105:0/293964971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559787636f50 con 0x559787626420 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c1de7640 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787633660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03c15e6640 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787626420 0x559787633c00 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009460 tx=0x7f03b8009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da46a3423df9a91c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.314+0000 7f03a57fa640 1 -- 172.21.15.105:0/293964971 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b803f060 con 0x559787626420 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03a57fa640 1 -- 172.21.15.105:0/293964971 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f03b8004a50 con 0x559787626420 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03ac02c7e0 con 0x559787626420 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03a57fa640 1 -- 172.21.15.105:0/293964971 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f03b8004c50 con 0x559787626420 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 10 client.22995 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5597876a0bd0 con 0x559787626420 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03a57fa640 1 -- 172.21.15.105:0/293964971 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac02c7e0 con 0x559787626420 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03a57fa640 1 -- 172.21.15.105:0/293964971 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f03b8028800 con 0x559787626420 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 10 client.22995 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 10 client.22995 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 20 client.22995 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5597876a1560 0x5597876a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 4 client.22995 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:24.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559787592200 con 0x5597876a1560 2024-03-26T17:59:24.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.315+0000 7f03c1de7640 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5597876a1560 0x5597876a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.316+0000 7f03a57fa640 10 client.22995 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.316+0000 7f03c1de7640 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5597876a1560 0x5597876a3980 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=14cfad4c49b81e9 server_cookie=f81f7e76d6d5440c in_seq=0 out_seq=0 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.316+0000 7f03a57fa640 1 -- 172.21.15.105:0/293964971 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559787592200 con 0x5597876a1560 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03a57fa640 10 client.22995 handle_command_reply: tid=0 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 client.22995 shutdown 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787626420 msgr2=0x559787633c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787626420 0x559787633c00 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009460 tx=0x7f03b8009c80 comp rx=0 tx=0).stop 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 shutdown_connections 2024-03-26T17:59:24.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5597876a1560 0x5597876a3980 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55978762b590 0x559787636990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03a57fa640 0 client.22995 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559787626420 0x559787633c00 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 --2- 172.21.15.105:0/293964971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559787625a30 0x559787633660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 >> 172.21.15.105:0/293964971 conn(0x559787620fa0 msgr2=0x559787621380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 shutdown_connections 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 1 -- 172.21.15.105:0/293964971 wait complete. 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.317+0000 7f03c6c32740 20 client.22995 trim_cache size 0 max 16384 2024-03-26T17:59:24.304 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 -- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03a80a2e80 msgr2=0x7f03a80a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03a80a2e80 0x7f03a80a5340 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x559787626c80 tx=0x7f03b8005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 -- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03bc10dc60 msgr2=0x7f03bc1adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03bc10dc60 0x7f03bc1adeb0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009fd0 tx=0x7f03b400ef10 comp rx=0 tx=0).stop 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 -- 172.21.15.105:0/850524507 shutdown_connections 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03a80a2e80 0x7f03a80a5340 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03bc10dc60 0x7f03bc1adeb0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03bc10b2b0 0x7f03bc1a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 --2- 172.21.15.105:0/850524507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03bc108940 0x7f03bc1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 -- 172.21.15.105:0/850524507 >> 172.21.15.105:0/850524507 conn(0x7f03bc06f760 msgr2=0x7f03bc10bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 -- 172.21.15.105:0/850524507 shutdown_connections 2024-03-26T17:59:24.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.318+0000 7f03c404b640 1 -- 172.21.15.105:0/850524507 wait complete. 2024-03-26T17:59:24.314 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:59:24.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 -- 172.21.15.105:0/4054394590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe181007c0 msgr2=0x7fbe18100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/4054394590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe181007c0 0x7fbe18100be0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009900 tx=0x7fbe0802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 -- 172.21.15.105:0/4054394590 shutdown_connections 2024-03-26T17:59:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/4054394590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe18102370 0x7fbe18108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/4054394590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe181019b0 0x7fbe18101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/4054394590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe181007c0 0x7fbe18100be0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 -- 172.21.15.105:0/4054394590 >> 172.21.15.105:0/4054394590 conn(0x7fbe180fbd50 msgr2=0x7fbe180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 -- 172.21.15.105:0/4054394590 shutdown_connections 2024-03-26T17:59:24.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.473+0000 7fbe1d27e640 1 -- 172.21.15.105:0/4054394590 wait complete. 2024-03-26T17:59:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe1d27e640 1 Processor -- start 2024-03-26T17:59:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe1d27e640 1 -- start start 2024-03-26T17:59:24.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe1d27e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe181019b0 0x7fbe18197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe1d27e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe18102370 0x7fbe18198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe1d27e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1819d2f0 0x7fbe1819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe1d27e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810c2c0 con 0x7fbe18102370 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe1d27e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810c140 con 0x7fbe181019b0 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe1d27e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1810c440 con 0x7fbe1819d2f0 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe18102370 0x7fbe18198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1819d2f0 0x7fbe1819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.474+0000 7fbe0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe18102370 0x7fbe18198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59728/0 (socket says 172.21.15.105:59728) 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe0ffff640 1 -- 172.21.15.105:0/1221636210 learned_addr learned my addr 172.21.15.105:0/1221636210 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe177fe640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1819d2f0 0x7fbe1819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59584/0 (socket says 172.21.15.105:59584) 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe181019b0 0x7fbe18197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe177fe640 1 -- 172.21.15.105:0/1221636210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe181019b0 msgr2=0x7fbe18197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe177fe640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe181019b0 0x7fbe18197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe177fe640 1 -- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe18102370 msgr2=0x7fbe18198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe177fe640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe18102370 0x7fbe18198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe177fe640 1 -- 172.21.15.105:0/1221636210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe18102e90 con 0x7fbe1819d2f0 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe181019b0 0x7fbe18197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe177fe640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1819d2f0 0x7fbe1819f6e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009fd0 tx=0x7fbe1000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe14ff9640 1 -- 172.21.15.105:0/1221636210 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000ce20 con 0x7fbe1819d2f0 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe14ff9640 1 -- 172.21.15.105:0/1221636210 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe10005020 con 0x7fbe1819d2f0 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe1d27e640 1 -- 172.21.15.105:0/1221636210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe18103180 con 0x7fbe1819d2f0 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe14ff9640 1 -- 172.21.15.105:0/1221636210 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe10010690 con 0x7fbe1819d2f0 2024-03-26T17:59:24.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.475+0000 7fbe1d27e640 1 -- 172.21.15.105:0/1221636210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe18103680 con 0x7fbe1819d2f0 2024-03-26T17:59:24.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.478+0000 7fbe14ff9640 1 -- 172.21.15.105:0/1221636210 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbe100051d0 con 0x7fbe1819d2f0 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.478+0000 7fbe1fe61740 1 Processor -- start 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.478+0000 7fbe1fe61740 1 -- start start 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.478+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e39e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.478+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.478+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e41a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef94d9e970 con 0x55ef94e3a420 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef94d46d70 con 0x55ef94e3f590 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef94d9a920 con 0x55ef94e39a30 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e39e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e39e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:59596/0 (socket says 172.21.15.105:59596) 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 -- 172.21.15.105:0/4284918288 learned_addr learned my addr 172.21.15.105:0/4284918288 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59744/0 (socket says 172.21.15.105:59744) 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e41a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 -- 172.21.15.105:0/4284918288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 msgr2=0x55ef94e41a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e41a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 -- 172.21.15.105:0/4284918288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 msgr2=0x55ef94e3ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 -- 172.21.15.105:0/4284918288 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef94ab23f0 con 0x55ef94e39a30 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe14ff9640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe040a2f20 0x7fbe040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e39e50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbe00011910 tx=0x7fbe00009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d584bea6a98a18 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe040a2f20 0x7fbe040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe14ff9640 1 -- 172.21.15.105:0/1221636210 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fbe100cfc00 con 0x7fbe1819d2f0 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0d7fa640 1 -- 172.21.15.105:0/4284918288 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000f7c0 con 0x55ef94e39a30 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0d7fa640 1 -- 172.21.15.105:0/4284918288 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe0000fe10 con 0x55ef94e39a30 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.479+0000 7fbe0d7fa640 1 -- 172.21.15.105:0/4284918288 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0001fd20 con 0x55ef94e39a30 2024-03-26T17:59:24.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe1fe61740 1 -- 172.21.15.105:0/4284918288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 msgr2=0x55ef94e39e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe040a2f20 0x7fbe040a53e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009230 tx=0x7fbe08005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e39e50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbe00011910 tx=0x7fbe00009540 comp rx=0 tx=0).stop 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe1fe61740 1 -- 172.21.15.105:0/4284918288 shutdown_connections 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e41a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/4284918288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e39e50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe1fe61740 1 -- 172.21.15.105:0/4284918288 >> 172.21.15.105:0/4284918288 conn(0x55ef94e34fa0 msgr2=0x55ef94e353a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.480+0000 7fbe1fe61740 1 -- 172.21.15.105:0/4284918288 shutdown_connections 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 -- 172.21.15.105:0/4284918288 wait complete. 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 -- start start 2024-03-26T17:59:24.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 Processor -- start 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e4b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ef94e42540 con 0x55ef94e3a420 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ef94d5a310 con 0x55ef94e3f590 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ef94e426c0 con 0x55ef94e39a30 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe1fe61740 10 client.0 _ll_register_callbacks cb 0x55ef94d95c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:24.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e43fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53216/0 (socket says 172.21.15.105:53216) 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe16ffd640 1 -- 172.21.15.105:0/1108888212 learned_addr learned my addr 172.21.15.105:0/1108888212 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe177fe640 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e4b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.481+0000 7fbe0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe16ffd640 1 -- 172.21.15.105:0/1108888212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 msgr2=0x55ef94e4b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e4b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe16ffd640 1 -- 172.21.15.105:0/1108888212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 msgr2=0x55ef94e444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe16ffd640 1 -- 172.21.15.105:0/1108888212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef94b7c7d0 con 0x55ef94e39a30 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe177fe640 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e4b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e43fa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009900 tx=0x7fbe0802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9c19909235b8cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbdfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/1108888212 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0804f020 con 0x55ef94e39a30 2024-03-26T17:59:24.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/1108888212 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fbe08005040 con 0x55ef94e39a30 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.482+0000 7fbe1fe61740 1 -- 172.21.15.105:0/1108888212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe0000fe10 con 0x55ef94e39a30 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/1108888212 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbe08005240 con 0x55ef94e39a30 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe1fe61740 10 client.32680 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe1fe61740 1 -- 172.21.15.105:0/1108888212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ef94e4e900 con 0x55ef94e39a30 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/1108888212 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe0000fe10 con 0x55ef94e39a30 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/1108888212 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fbe080035f0 con 0x55ef94e39a30 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe1fe61740 10 client.32680 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe1fe61740 10 client.32680 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe1fe61740 20 client.32680 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ef94e4fef0 0x55ef94e52360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe1fe61740 4 client.32680 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe1fe61740 1 -- 172.21.15.105:0/1108888212 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef94dabeb0 con 0x55ef94e4fef0 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.483+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ef94e4fef0 0x55ef94e52360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.484+0000 7fbdfe7fc640 10 client.32680 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:59:24.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.484+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ef94e4fef0 0x55ef94e52360 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=edb8e50ed4aca951 server_cookie=b87c938513827ddf in_seq=0 out_seq=0 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.484+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/1108888212 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ef94dabeb0 con 0x55ef94e4fef0 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.484+0000 7fbdfe7fc640 10 client.32680 handle_command_reply: tid=0 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.484+0000 7fbe1fe61740 1 client.32680 shutdown 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 -- 172.21.15.105:0/1108888212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 msgr2=0x55ef94e43fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e43fa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009900 tx=0x7fbe0802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 -- 172.21.15.105:0/1108888212 shutdown_connections 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ef94e4fef0 0x55ef94e52360 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e3f590 0x55ef94e4b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/1108888212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e39a30 0x55ef94e43fa0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 -- 172.21.15.105:0/1108888212 >> 172.21.15.105:0/1108888212 conn(0x55ef94e34fa0 msgr2=0x55ef94e341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbdfe7fc640 0 client.32680 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:59:24.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 -- 172.21.15.105:0/1108888212 shutdown_connections 2024-03-26T17:59:24.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 1 -- 172.21.15.105:0/1108888212 wait complete. 2024-03-26T17:59:24.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.485+0000 7fbe1fe61740 20 client.32680 trim_cache size 0 max 16384 2024-03-26T17:59:24.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.505+0000 7fbe1fe61740 1 Processor -- start 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.505+0000 7fbe1fe61740 1 -- start start 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e3b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3f590 0x55ef94e4b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181ab5a0 con 0x55ef94e3a420 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181ad0e0 con 0x55ef94e39a30 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181abc70 con 0x55ef94e3f590 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e3b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53432/0 (socket says 172.21.15.105:53432) 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 -- 172.21.15.105:0/2566938041 learned_addr learned my addr 172.21.15.105:0/2566938041 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe177fe640 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3f590 0x55ef94e4b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 -- 172.21.15.105:0/2566938041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3f590 msgr2=0x55ef94e4b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3f590 0x55ef94e4b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 -- 172.21.15.105:0/2566938041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 msgr2=0x55ef94e3b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e3b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 -- 172.21.15.105:0/2566938041 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef94e35d10 con 0x55ef94e3a420 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe177fe640 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3f590 0x55ef94e4b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e3b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.506+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3bec0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x55ef94e3ac80 tx=0x7fbe00005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0e9a165515e2b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.507+0000 7fbe0cff9640 1 -- 172.21.15.105:0/2566938041 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00032070 con 0x55ef94e3a420 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.507+0000 7fbe0cff9640 1 -- 172.21.15.105:0/2566938041 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe00005560 con 0x55ef94e3a420 2024-03-26T17:59:24.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.507+0000 7fbe0cff9640 1 -- 172.21.15.105:0/2566938041 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000f9f0 con 0x55ef94e3a420 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 -- 172.21.15.105:0/2566938041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 msgr2=0x55ef94e3bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3bec0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x55ef94e3ac80 tx=0x7fbe00005040 comp rx=0 tx=0).stop 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 -- 172.21.15.105:0/2566938041 shutdown_connections 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3f590 0x55ef94e4b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3a420 0x55ef94e3bec0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/2566938041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e3b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 -- 172.21.15.105:0/2566938041 >> 172.21.15.105:0/2566938041 conn(0x55ef94e34fa0 msgr2=0x55ef94e35380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 -- 172.21.15.105:0/2566938041 shutdown_connections 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 -- 172.21.15.105:0/2566938041 wait complete. 2024-03-26T17:59:24.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 -- start start 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.508+0000 7fbe1fe61740 1 Processor -- start 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e4b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3a420 0x55ef94e47e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe1fe61740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3f590 0x55ef94e48340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181aca10 con 0x55ef94e3f590 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181ac340 con 0x55ef94e39a30 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe1fe61740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe181a9580 con 0x55ef94e3a420 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe1fe61740 10 client.0 _ll_register_callbacks cb 0x7fbe00005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3f590 0x55ef94e48340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe16ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e4b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3f590 0x55ef94e48340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53446/0 (socket says 172.21.15.105:53446) 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 -- 172.21.15.105:0/408907906 learned_addr learned my addr 172.21.15.105:0/408907906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe0ffff640 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3a420 0x55ef94e47e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 -- 172.21.15.105:0/408907906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3a420 msgr2=0x55ef94e47e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3a420 0x55ef94e47e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 -- 172.21.15.105:0/408907906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 msgr2=0x55ef94e4b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e4b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 -- 172.21.15.105:0/408907906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef94eb4250 con 0x55ef94e3f590 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbe177fe640 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3f590 0x55ef94e48340 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fbe1009c220 tx=0x7fbe1009c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db395a15b7c3b7c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbdfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.509+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/408907906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe100ba060 con 0x55ef94e3f590 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.510+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/408907906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fbe10005540 con 0x55ef94e3f590 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.510+0000 7fbe1fe61740 1 -- 172.21.15.105:0/408907906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe00005560 con 0x55ef94e3f590 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.510+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/408907906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fbe10067170 con 0x55ef94e3f590 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.510+0000 7fbe1fe61740 10 client.23013 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.510+0000 7fbe1fe61740 1 -- 172.21.15.105:0/408907906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ef94eb5490 con 0x55ef94e3f590 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.510+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/408907906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe1007d070 con 0x55ef94e3f590 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/408907906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fbe100cdbb0 con 0x55ef94e3f590 2024-03-26T17:59:24.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbe1fe61740 10 client.23013 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbe1fe61740 10 client.23013 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbe1fe61740 20 client.23013 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ef94eb5ec0 0x55ef94eb82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbe1fe61740 4 client.23013 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbe1fe61740 1 -- 172.21.15.105:0/408907906 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbe00005560 con 0x55ef94eb5ec0 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ef94eb5ec0 0x55ef94eb82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbdfe7fc640 10 client.23013 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbe16ffd640 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ef94eb5ec0 0x55ef94eb82b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5adf0d72685eaa0 server_cookie=b70e44fe3770f607 in_seq=0 out_seq=0 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.511+0000 7fbdfe7fc640 1 -- 172.21.15.105:0/408907906 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbe00005560 con 0x55ef94eb5ec0 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbdfe7fc640 10 client.23013 handle_command_reply: tid=0 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 client.23013 shutdown 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 -- 172.21.15.105:0/408907906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3f590 msgr2=0x55ef94e48340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3f590 0x55ef94e48340 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fbe1009c220 tx=0x7fbe1009c350 comp rx=0 tx=0).stop 2024-03-26T17:59:24.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 -- 172.21.15.105:0/408907906 shutdown_connections 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55ef94eb5ec0 0x55ef94eb82b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ef94e3f590 0x55ef94e48340 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ef94e3a420 0x55ef94e47e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 --2- 172.21.15.105:0/408907906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ef94e39a30 0x55ef94e4b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 -- 172.21.15.105:0/408907906 >> 172.21.15.105:0/408907906 conn(0x55ef94e34fa0 msgr2=0x55ef94e3b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbdfe7fc640 0 client.23013 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 -- 172.21.15.105:0/408907906 shutdown_connections 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 1 -- 172.21.15.105:0/408907906 wait complete. 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1fe61740 20 client.23013 trim_cache size 0 max 16384 2024-03-26T17:59:24.499 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1d27e640 1 -- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe040a2f20 msgr2=0x7fbe040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe040a2f20 0x7fbe040a53e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009230 tx=0x7fbe08005a20 comp rx=0 tx=0).stop 2024-03-26T17:59:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1d27e640 1 -- 172.21.15.105:0/1221636210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1819d2f0 msgr2=0x7fbe1819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.512+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1819d2f0 0x7fbe1819f6e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009fd0 tx=0x7fbe1000ef10 comp rx=0 tx=0).stop 2024-03-26T17:59:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.513+0000 7fbe1d27e640 1 -- 172.21.15.105:0/1221636210 shutdown_connections 2024-03-26T17:59:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.513+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbe040a2f20 0x7fbe040a53e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.513+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe1819d2f0 0x7fbe1819f6e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.513+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe18102370 0x7fbe18198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.513+0000 7fbe1d27e640 1 --2- 172.21.15.105:0/1221636210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe181019b0 0x7fbe18197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.513+0000 7fbe1d27e640 1 -- 172.21.15.105:0/1221636210 >> 172.21.15.105:0/1221636210 conn(0x7fbe180fbd50 msgr2=0x7fbe18068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.513+0000 7fbe1d27e640 1 -- 172.21.15.105:0/1221636210 shutdown_connections 2024-03-26T17:59:24.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.513+0000 7fbe1d27e640 1 -- 172.21.15.105:0/1221636210 wait complete. 2024-03-26T17:59:24.510 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:59:24.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 63987 7.11 MB/sec execute 367 sec latency 138.805 ms 2024-03-26T17:59:24.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.661+0000 7fe09b377640 1 -- 172.21.15.105:0/1235874703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0941018d0 msgr2=0x7fe094101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.661+0000 7fe09b377640 1 --2- 172.21.15.105:0/1235874703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0941018d0 0x7fe094101cf0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fe090009900 tx=0x7fe09002bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:24.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.661+0000 7fe09b377640 1 -- 172.21.15.105:0/1235874703 shutdown_connections 2024-03-26T17:59:24.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.661+0000 7fe09b377640 1 --2- 172.21.15.105:0/1235874703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe094103480 0x7fe094109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.661+0000 7fe09b377640 1 --2- 172.21.15.105:0/1235874703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe094102ac0 0x7fe094102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.661+0000 7fe09b377640 1 --2- 172.21.15.105:0/1235874703 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0941018d0 0x7fe094101cf0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.661+0000 7fe09b377640 1 -- 172.21.15.105:0/1235874703 >> 172.21.15.105:0/1235874703 conn(0x7fe0940fce60 msgr2=0x7fe0940ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.661+0000 7fe09b377640 1 -- 172.21.15.105:0/1235874703 shutdown_connections 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 -- 172.21.15.105:0/1235874703 wait complete. 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 Processor -- start 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 -- start start 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0941018d0 0x7fe094197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe094102ac0 0x7fe094198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe094103480 0x7fe09419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe09410d3d0 con 0x7fe094103480 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe09410d250 con 0x7fe0941018d0 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.662+0000 7fe09b377640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe09410d550 con 0x7fe094102ac0 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe094103480 0x7fe09419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe094103480 0x7fe09419f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53474/0 (socket says 172.21.15.105:53474) 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 -- 172.21.15.105:0/4050562258 learned_addr learned my addr 172.21.15.105:0/4050562258 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099113640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0941018d0 0x7fe094197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe098912640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe094102ac0 0x7fe094198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 -- 172.21.15.105:0/4050562258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe094102ac0 msgr2=0x7fe094198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe094102ac0 0x7fe094198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 -- 172.21.15.105:0/4050562258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0941018d0 msgr2=0x7fe094197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0941018d0 0x7fe094197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 -- 172.21.15.105:0/4050562258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09419f770 con 0x7fe094103480 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099113640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0941018d0 0x7fe094197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe099914640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe094103480 0x7fe09419f100 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00eb10 tx=0x7fe08c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe08a7fc640 1 -- 172.21.15.105:0/4050562258 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c004b20 con 0x7fe094103480 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe08a7fc640 1 -- 172.21.15.105:0/4050562258 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe08c004cc0 con 0x7fe094103480 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe09b377640 1 -- 172.21.15.105:0/4050562258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe094073b50 con 0x7fe094103480 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.663+0000 7fe08a7fc640 1 -- 172.21.15.105:0/4050562258 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08c010690 con 0x7fe094103480 2024-03-26T17:59:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.664+0000 7fe09b377640 1 -- 172.21.15.105:0/4050562258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0940740a0 con 0x7fe094103480 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe08a7fc640 1 -- 172.21.15.105:0/4050562258 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe08c005020 con 0x7fe094103480 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe09df63740 1 Processor -- start 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe09df63740 1 -- start start 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe09df63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2804a30 0x5625f2804e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe09df63740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f2809f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe09df63740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 0x5625f280ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe09df63740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5625f2769970 con 0x5625f2805420 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe09df63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5625f2711d70 con 0x5625f2804a30 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe09df63740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625f2765920 con 0x5625f280a590 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe099113640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f2809f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe098912640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2804a30 0x5625f2804e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe098912640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2804a30 0x5625f2804e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42064/0 (socket says 172.21.15.105:42064) 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe098912640 1 -- 172.21.15.105:0/2775069750 learned_addr learned my addr 172.21.15.105:0/2775069750 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe099113640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f2809f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53478/0 (socket says 172.21.15.105:53478) 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.667+0000 7fe098912640 1 --2- 172.21.15.105:0/2775069750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 0x5625f280ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe08a7fc640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0800a2fa0 0x7fe0800a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe098912640 1 -- 172.21.15.105:0/2775069750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 msgr2=0x5625f280ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe098912640 1 --2- 172.21.15.105:0/2775069750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 0x5625f280ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe098912640 1 -- 172.21.15.105:0/2775069750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 msgr2=0x5625f2809f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe098912640 1 --2- 172.21.15.105:0/2775069750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f2809f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe098912640 1 -- 172.21.15.105:0/2775069750 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f247d3f0 con 0x5625f2804a30 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe099914640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0800a2fa0 0x7fe0800a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe098912640 1 --2- 172.21.15.105:0/2775069750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2804a30 0x5625f2804e50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe0840058c0 tx=0x7fe084005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b51a136ab1a29c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe07f7fe640 1 -- 172.21.15.105:0/2775069750 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084023a60 con 0x5625f2804a30 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe07f7fe640 1 -- 172.21.15.105:0/2775069750 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe084023c00 con 0x5625f2804a30 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe08a7fc640 1 -- 172.21.15.105:0/4050562258 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fe08c0cf900 con 0x7fe094103480 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.668+0000 7fe07f7fe640 1 -- 172.21.15.105:0/2775069750 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084005640 con 0x5625f2804a30 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe099914640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0800a2fa0 0x7fe0800a5460 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0989c0 tx=0x7fe08c097bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 -- 172.21.15.105:0/2775069750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2804a30 msgr2=0x5625f2804e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 --2- 172.21.15.105:0/2775069750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2804a30 0x5625f2804e50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe0840058c0 tx=0x7fe084005850 comp rx=0 tx=0).stop 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 -- 172.21.15.105:0/2775069750 shutdown_connections 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 --2- 172.21.15.105:0/2775069750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 0x5625f280ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 --2- 172.21.15.105:0/2775069750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f2809f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 --2- 172.21.15.105:0/2775069750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2804a30 0x5625f2804e50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 -- 172.21.15.105:0/2775069750 >> 172.21.15.105:0/2775069750 conn(0x5625f27fffa0 msgr2=0x5625f28003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 -- 172.21.15.105:0/2775069750 shutdown_connections 2024-03-26T17:59:24.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.669+0000 7fe09df63740 1 -- 172.21.15.105:0/2775069750 wait complete. 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 1 -- start start 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 1 Processor -- start 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f280eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f280f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f280a590 0x5625f2816430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5625f280d530 con 0x5625f2805420 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5625f2725310 con 0x5625f280a590 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5625f280d6b0 con 0x5625f2804a30 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe09df63740 10 client.0 _ll_register_callbacks cb 0x5625f2760c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe099113640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f280f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe098912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f280eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe098912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f280eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53244/0 (socket says 172.21.15.105:53244) 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe098912640 1 -- 172.21.15.105:0/2113050475 learned_addr learned my addr 172.21.15.105:0/2113050475 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.670+0000 7fe098912640 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f280a590 0x5625f2816430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe098912640 1 -- 172.21.15.105:0/2113050475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f280a590 msgr2=0x5625f2816430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe098912640 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f280a590 0x5625f2816430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe098912640 1 -- 172.21.15.105:0/2113050475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 msgr2=0x5625f280f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe098912640 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f280f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe098912640 1 -- 172.21.15.105:0/2113050475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f25477d0 con 0x5625f2804a30 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe099113640 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f280f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe098912640 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f280eeb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe084005bc0 tx=0x7fe084005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=394a21a47ea5a51d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe07cff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe07cff9640 1 -- 172.21.15.105:0/2113050475 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840042a0 con 0x5625f2804a30 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe09df63740 1 -- 172.21.15.105:0/2113050475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe084023c00 con 0x5625f2804a30 2024-03-26T17:59:24.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe07cff9640 1 -- 172.21.15.105:0/2113050475 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fe084004440 con 0x5625f2804a30 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe07cff9640 1 -- 172.21.15.105:0/2113050475 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe08400f790 con 0x5625f2804a30 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe09df63740 10 client.32698 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.671+0000 7fe09df63740 1 -- 172.21.15.105:0/2113050475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5625f2819780 con 0x5625f2804a30 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe07cff9640 1 -- 172.21.15.105:0/2113050475 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe084023c00 con 0x5625f2804a30 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe07cff9640 1 -- 172.21.15.105:0/2113050475 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fe084004b00 con 0x5625f2804a30 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe09df63740 10 client.32698 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe09df63740 10 client.32698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe09df63740 20 client.32698 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe09df63740 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5625f281b1f0 0x5625f281d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe09df63740 4 client.32698 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe09df63740 1 -- 172.21.15.105:0/2113050475 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625f2776eb0 con 0x5625f281b1f0 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe099113640 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5625f281b1f0 0x5625f281d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe099113640 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5625f281b1f0 0x5625f281d650 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=825833aecff38d2f server_cookie=fdbc07e3106788ab in_seq=0 out_seq=0 2024-03-26T17:59:24.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.672+0000 7fe07cff9640 10 client.32698 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:59:24.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.673+0000 7fe07cff9640 1 -- 172.21.15.105:0/2113050475 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5625f2776eb0 con 0x5625f281b1f0 2024-03-26T17:59:24.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.673+0000 7fe07cff9640 10 client.32698 handle_command_reply: tid=0 2024-03-26T17:59:24.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.673+0000 7fe09df63740 1 client.32698 shutdown 2024-03-26T17:59:24.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.673+0000 7fe09df63740 1 -- 172.21.15.105:0/2113050475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 msgr2=0x5625f280eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f280eeb0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe084005bc0 tx=0x7fe084005020 comp rx=0 tx=0).stop 2024-03-26T17:59:24.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 -- 172.21.15.105:0/2113050475 shutdown_connections 2024-03-26T17:59:24.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5625f281b1f0 0x5625f281d650 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f280a590 0x5625f2816430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2805420 0x5625f280f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe07cff9640 0 client.32698 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:59:24.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 --2- 172.21.15.105:0/2113050475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f280eeb0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 -- 172.21.15.105:0/2113050475 >> 172.21.15.105:0/2113050475 conn(0x5625f27fffa0 msgr2=0x5625f27ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 -- 172.21.15.105:0/2113050475 shutdown_connections 2024-03-26T17:59:24.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 1 -- 172.21.15.105:0/2113050475 wait complete. 2024-03-26T17:59:24.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.674+0000 7fe09df63740 20 client.32698 trim_cache size 0 max 16384 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe09df63740 1 Processor -- start 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe09df63740 1 -- start start 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe09df63740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2804a30 0x5625f280eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe09df63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe09df63740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 0x5625f280f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe09df63740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941a6c60 con 0x5625f2804a30 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe09df63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941a87a0 con 0x5625f2805420 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe09df63740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941a7330 con 0x5625f280a590 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.697+0000 7fe098912640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2804a30 0x5625f280eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe099113640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe098912640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2804a30 0x5625f280eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53494/0 (socket says 172.21.15.105:53494) 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe098912640 1 -- 172.21.15.105:0/3002758736 learned_addr learned my addr 172.21.15.105:0/3002758736 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe099113640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28163f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42082/0 (socket says 172.21.15.105:42082) 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe098912640 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 0x5625f280f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe099113640 1 -- 172.21.15.105:0/3002758736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 msgr2=0x5625f280f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe099113640 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 0x5625f280f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe099113640 1 -- 172.21.15.105:0/3002758736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2804a30 msgr2=0x5625f280eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe099113640 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2804a30 0x5625f280eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe099113640 1 -- 172.21.15.105:0/3002758736 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f27fa5d0 con 0x5625f2805420 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe098912640 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2804a30 0x5625f280eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe099113640 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28163f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe09002c170 tx=0x7fe090005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba43d43c8d755ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe07effd640 1 -- 172.21.15.105:0/3002758736 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090005040 con 0x5625f2805420 2024-03-26T17:59:24.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe07effd640 1 -- 172.21.15.105:0/3002758736 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0900046c0 con 0x5625f2805420 2024-03-26T17:59:24.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.698+0000 7fe07effd640 1 -- 172.21.15.105:0/3002758736 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0900037f0 con 0x5625f2805420 2024-03-26T17:59:24.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.699+0000 7fe09df63740 1 -- 172.21.15.105:0/3002758736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 msgr2=0x5625f28163f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.699+0000 7fe09df63740 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28163f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe09002c170 tx=0x7fe090005b50 comp rx=0 tx=0).stop 2024-03-26T17:59:24.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.699+0000 7fe09df63740 1 -- 172.21.15.105:0/3002758736 shutdown_connections 2024-03-26T17:59:24.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.699+0000 7fe09df63740 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f280a590 0x5625f280f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.699+0000 7fe09df63740 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28163f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.699+0000 7fe09df63740 1 --2- 172.21.15.105:0/3002758736 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f2804a30 0x5625f280eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.699+0000 7fe09df63740 1 -- 172.21.15.105:0/3002758736 >> 172.21.15.105:0/3002758736 conn(0x5625f27fffa0 msgr2=0x5625f2800380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 -- 172.21.15.105:0/3002758736 shutdown_connections 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 -- 172.21.15.105:0/3002758736 wait complete. 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 -- start start 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 Processor -- start 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f2813380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f280a590 0x5625f287f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941a80d0 con 0x5625f280a590 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941a7a00 con 0x5625f2805420 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0941a4c40 con 0x5625f2804a30 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.700+0000 7fe09df63740 10 client.0 _ll_register_callbacks cb 0x7fe084005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28138c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42090/0 (socket says 172.21.15.105:42090) 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe098912640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f2813380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 -- 172.21.15.105:0/3459072620 learned_addr learned my addr 172.21.15.105:0/3459072620 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe098912640 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f280a590 0x5625f287f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 -- 172.21.15.105:0/3459072620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 msgr2=0x5625f2813380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f2813380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 -- 172.21.15.105:0/3459072620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f280a590 msgr2=0x5625f287f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f280a590 0x5625f287f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 -- 172.21.15.105:0/3459072620 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f287fac0 con 0x5625f2805420 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe099113640 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28138c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe09002ce10 tx=0x7fe090004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b085111120dcf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.701+0000 7fe07cff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe07cff9640 1 -- 172.21.15.105:0/3459072620 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090054020 con 0x5625f2805420 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe09df63740 1 -- 172.21.15.105:0/3459072620 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0900046c0 con 0x5625f2805420 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe07cff9640 1 -- 172.21.15.105:0/3459072620 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fe090004fc0 con 0x5625f2805420 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe07cff9640 1 -- 172.21.15.105:0/3459072620 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe090004c40 con 0x5625f2805420 2024-03-26T17:59:24.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe09df63740 10 client.32714 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe09df63740 1 -- 172.21.15.105:0/3459072620 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5625f2880de0 con 0x5625f2805420 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe07cff9640 1 -- 172.21.15.105:0/3459072620 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0900046c0 con 0x5625f2805420 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe07cff9640 1 -- 172.21.15.105:0/3459072620 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fe090046040 con 0x5625f2805420 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe09df63740 10 client.32714 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe09df63740 10 client.32714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe09df63740 20 client.32714 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.702+0000 7fe09df63740 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5625f281b1f0 0x5625f28837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.703+0000 7fe09df63740 4 client.32714 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.703+0000 7fe09df63740 1 -- 172.21.15.105:0/3459072620 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625f2771200 con 0x5625f281b1f0 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.703+0000 7fe098912640 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5625f281b1f0 0x5625f28837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.703+0000 7fe07cff9640 10 client.32714 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.703+0000 7fe098912640 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5625f281b1f0 0x5625f28837a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a7ad1d920e36299 server_cookie=30fb5fe5cabd2007 in_seq=0 out_seq=0 2024-03-26T17:59:24.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.703+0000 7fe07cff9640 1 -- 172.21.15.105:0/3459072620 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625f2771200 con 0x5625f281b1f0 2024-03-26T17:59:24.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.703+0000 7fe07cff9640 10 client.32714 handle_command_reply: tid=0 2024-03-26T17:59:24.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.703+0000 7fe09df63740 1 client.32714 shutdown 2024-03-26T17:59:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 -- 172.21.15.105:0/3459072620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 msgr2=0x5625f28138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28138c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe09002ce10 tx=0x7fe090004530 comp rx=0 tx=0).stop 2024-03-26T17:59:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 -- 172.21.15.105:0/3459072620 shutdown_connections 2024-03-26T17:59:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5625f280a590 0x5625f287f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5625f281b1f0 0x5625f28837a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5625f2805420 0x5625f28138c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 --2- 172.21.15.105:0/3459072620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5625f2804a30 0x5625f2813380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe07cff9640 0 client.32714 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:59:24.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 -- 172.21.15.105:0/3459072620 >> 172.21.15.105:0/3459072620 conn(0x5625f27fffa0 msgr2=0x5625f280eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.704+0000 7fe09df63740 1 -- 172.21.15.105:0/3459072620 shutdown_connections 2024-03-26T17:59:24.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.705+0000 7fe09df63740 1 -- 172.21.15.105:0/3459072620 wait complete. 2024-03-26T17:59:24.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.705+0000 7fe09df63740 20 client.32714 trim_cache size 0 max 16384 2024-03-26T17:59:24.692 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:24.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.705+0000 7fe09b377640 1 -- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0800a2fa0 msgr2=0x7fe0800a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.705+0000 7fe09b377640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0800a2fa0 0x7fe0800a5460 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0989c0 tx=0x7fe08c097bf0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 -- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe094103480 msgr2=0x7fe09419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe094103480 0x7fe09419f100 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00eb10 tx=0x7fe08c00c2d0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 -- 172.21.15.105:0/4050562258 shutdown_connections 2024-03-26T17:59:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0800a2fa0 0x7fe0800a5460 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe094103480 0x7fe09419f100 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe094102ac0 0x7fe094198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 --2- 172.21.15.105:0/4050562258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0941018d0 0x7fe094197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 -- 172.21.15.105:0/4050562258 >> 172.21.15.105:0/4050562258 conn(0x7fe0940fce60 msgr2=0x7fe094105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 -- 172.21.15.105:0/4050562258 shutdown_connections 2024-03-26T17:59:24.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.706+0000 7fe09b377640 1 -- 172.21.15.105:0/4050562258 wait complete. 2024-03-26T17:59:24.703 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 -- 172.21.15.105:0/1945776300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41007c0 msgr2=0x7f64f4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1945776300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41007c0 0x7f64f4100be0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009900 tx=0x7f64e402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 -- 172.21.15.105:0/1945776300 shutdown_connections 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1945776300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f4102370 0x7f64f4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1945776300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f41019b0 0x7f64f4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1945776300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41007c0 0x7f64f4100be0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 -- 172.21.15.105:0/1945776300 >> 172.21.15.105:0/1945776300 conn(0x7f64f40fbd50 msgr2=0x7f64f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 -- 172.21.15.105:0/1945776300 shutdown_connections 2024-03-26T17:59:24.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.858+0000 7f64f8eff640 1 -- 172.21.15.105:0/1945776300 wait complete. 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f8eff640 1 Processor -- start 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f8eff640 1 -- start start 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f8eff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f41007c0 0x7f64f4196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f8eff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41019b0 0x7f64f4196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f8eff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f4102370 0x7f64f419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f8eff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410c2c0 con 0x7f64f41019b0 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f8eff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410c140 con 0x7f64f4102370 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f8eff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64f410c440 con 0x7f64f41007c0 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41019b0 0x7f64f4196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f41007c0 0x7f64f4196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f4102370 0x7f64f419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41019b0 0x7f64f4196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53524/0 (socket says 172.21.15.105:53524) 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f1d9b640 1 -- 172.21.15.105:0/1318808793 learned_addr learned my addr 172.21.15.105:0/1318808793 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f1d9b640 1 -- 172.21.15.105:0/1318808793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f41007c0 msgr2=0x7f64f4196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f41007c0 0x7f64f4196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f1d9b640 1 -- 172.21.15.105:0/1318808793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f4102370 msgr2=0x7f64f419e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f4102370 0x7f64f419e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.859+0000 7f64f1d9b640 1 -- 172.21.15.105:0/1318808793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f419e690 con 0x7f64f41019b0 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.860+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41019b0 0x7f64f4196fb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f64e800eb70 tx=0x7f64e800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.860+0000 7f64e37fe640 1 -- 172.21.15.105:0/1318808793 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e800bc30 con 0x7f64f41019b0 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.860+0000 7f64e37fe640 1 -- 172.21.15.105:0/1318808793 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64e800bdd0 con 0x7f64f41019b0 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.860+0000 7f64f8eff640 1 -- 172.21.15.105:0/1318808793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f4068a80 con 0x7f64f41019b0 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.860+0000 7f64e37fe640 1 -- 172.21.15.105:0/1318808793 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8017410 con 0x7f64f41019b0 2024-03-26T17:59:24.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.860+0000 7f64f8eff640 1 -- 172.21.15.105:0/1318808793 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f4069020 con 0x7f64f41019b0 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.863+0000 7f64e37fe640 1 -- 172.21.15.105:0/1318808793 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f64e801e020 con 0x7f64f41019b0 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64fbbca740 1 Processor -- start 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64fbbca740 1 -- start start 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa771e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa776f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa779a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64fbbca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559afa6d6970 con 0x559afa777590 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64fbbca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559afa67ed70 con 0x559afa772420 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64fbbca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559afa6d2920 con 0x559afa771a30 2024-03-26T17:59:24.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64e37fe640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64dc0a2fa0 0x7f64dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa779a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa776f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f2d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa771e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa779a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53536/0 (socket says 172.21.15.105:53536) 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa776f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42126/0 (socket says 172.21.15.105:42126) 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f1d9b640 1 -- 172.21.15.105:0/2374757026 learned_addr learned my addr 172.21.15.105:0/2374757026 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f2d9d640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64dc0a2fa0 0x7f64dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64e37fe640 1 -- 172.21.15.105:0/1318808793 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f64e80d0a80 con 0x7f64f41019b0 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f2d9d640 1 -- 172.21.15.105:0/2374757026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 msgr2=0x559afa776f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.864+0000 7f64f2d9d640 1 --2- 172.21.15.105:0/2374757026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa776f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64f2d9d640 1 -- 172.21.15.105:0/2374757026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 msgr2=0x559afa779a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64f2d9d640 1 --2- 172.21.15.105:0/2374757026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa779a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64f2d9d640 1 -- 172.21.15.105:0/2374757026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559afa3ea3f0 con 0x559afa771a30 2024-03-26T17:59:24.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64f259c640 1 --2- 172.21.15.105:0/2374757026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa776f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64f2d9d640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64dc0a2fa0 0x7f64dc0a5460 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005fd0 tx=0x7f64ec01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64f2d9d640 1 --2- 172.21.15.105:0/2374757026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa771e50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0099a0 tx=0x7f64ec00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935d619c61f3af71 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64e0ff9640 1 -- 172.21.15.105:0/2374757026 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011e60 con 0x559afa771a30 2024-03-26T17:59:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64e0ff9640 1 -- 172.21.15.105:0/2374757026 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64ec01dbf0 con 0x559afa771a30 2024-03-26T17:59:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.865+0000 7f64e0ff9640 1 -- 172.21.15.105:0/2374757026 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0042b0 con 0x559afa771a30 2024-03-26T17:59:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.866+0000 7f64fbbca740 1 -- 172.21.15.105:0/2374757026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 msgr2=0x559afa771e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.866+0000 7f64fbbca740 1 --2- 172.21.15.105:0/2374757026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa771e50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f64ec0099a0 tx=0x7f64ec00f790 comp rx=0 tx=0).stop 2024-03-26T17:59:24.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.867+0000 7f64fbbca740 1 -- 172.21.15.105:0/2374757026 shutdown_connections 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.867+0000 7f64fbbca740 1 --2- 172.21.15.105:0/2374757026 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa779a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.867+0000 7f64fbbca740 1 --2- 172.21.15.105:0/2374757026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa776f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.867+0000 7f64fbbca740 1 --2- 172.21.15.105:0/2374757026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa771e50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.867+0000 7f64fbbca740 1 -- 172.21.15.105:0/2374757026 >> 172.21.15.105:0/2374757026 conn(0x559afa76cfa0 msgr2=0x559afa76d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.868+0000 7f64fbbca740 1 -- 172.21.15.105:0/2374757026 shutdown_connections 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.868+0000 7f64fbbca740 1 -- 172.21.15.105:0/2374757026 wait complete. 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.868+0000 7f64fbbca740 1 -- start start 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.868+0000 7f64fbbca740 1 Processor -- start 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.869+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa77bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.869+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa772420 0x559afa77c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.869+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 0x559afa783490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.869+0000 7f64fbbca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559afa77a4e0 con 0x559afa772420 2024-03-26T17:59:24.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.869+0000 7f64fbbca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559afa692310 con 0x559afa777590 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.869+0000 7f64fbbca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559afa77a660 con 0x559afa771a30 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64fbbca740 10 client.0 _ll_register_callbacks cb 0x559afa6cdc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa77bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa77bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53312/0 (socket says 172.21.15.105:53312) 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f259c640 1 -- 172.21.15.105:0/546401999 learned_addr learned my addr 172.21.15.105:0/546401999 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f2d9d640 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 0x559afa783490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa772420 0x559afa77c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f259c640 1 -- 172.21.15.105:0/546401999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 msgr2=0x559afa783490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f259c640 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 0x559afa783490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f259c640 1 -- 172.21.15.105:0/546401999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa772420 msgr2=0x559afa77c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f259c640 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa772420 0x559afa77c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.870+0000 7f64f259c640 1 -- 172.21.15.105:0/546401999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559afa4b47d0 con 0x559afa771a30 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.871+0000 7f64f259c640 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa77bf10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009230 tx=0x7f64e4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eda26a32cd4e91a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.871+0000 7f64d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.871+0000 7f64d9ffb640 1 -- 172.21.15.105:0/546401999 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4005030 con 0x559afa771a30 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.871+0000 7f64d9ffb640 1 -- 172.21.15.105:0/546401999 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f64e40051d0 con 0x559afa771a30 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.872+0000 7f64fbbca740 1 -- 172.21.15.105:0/546401999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ec01dbf0 con 0x559afa771a30 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.872+0000 7f64d9ffb640 1 -- 172.21.15.105:0/546401999 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f64e40053d0 con 0x559afa771a30 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.872+0000 7f64fbbca740 10 client.32716 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.872+0000 7f64fbbca740 1 -- 172.21.15.105:0/546401999 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559afa786870 con 0x559afa771a30 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.872+0000 7f64d9ffb640 1 -- 172.21.15.105:0/546401999 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64ec01dbf0 con 0x559afa771a30 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.872+0000 7f64d9ffb640 1 -- 172.21.15.105:0/546401999 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f64e4003680 con 0x559afa771a30 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.873+0000 7f64fbbca740 10 client.32716 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.873+0000 7f64fbbca740 10 client.32716 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:59:24.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.873+0000 7f64fbbca740 20 client.32716 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:24.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.873+0000 7f64fbbca740 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559afa787ee0 0x559afa78a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.873+0000 7f64fbbca740 4 client.32716 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:24.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.873+0000 7f64fbbca740 1 -- 172.21.15.105:0/546401999 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559afa6e3eb0 con 0x559afa787ee0 2024-03-26T17:59:24.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.873+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559afa787ee0 0x559afa78a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.874+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559afa787ee0 0x559afa78a350 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83198097f22f044f server_cookie=99a180600d62a4e1 in_seq=0 out_seq=0 2024-03-26T17:59:24.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.874+0000 7f64d9ffb640 10 client.32716 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:59:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.875+0000 7f64d9ffb640 1 -- 172.21.15.105:0/546401999 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559afa6e3eb0 con 0x559afa787ee0 2024-03-26T17:59:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.875+0000 7f64d9ffb640 10 client.32716 handle_command_reply: tid=0 2024-03-26T17:59:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.875+0000 7f64fbbca740 1 client.32716 shutdown 2024-03-26T17:59:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 -- 172.21.15.105:0/546401999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 msgr2=0x559afa77bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa77bf10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009230 tx=0x7f64e4004270 comp rx=0 tx=0).stop 2024-03-26T17:59:24.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 -- 172.21.15.105:0/546401999 shutdown_connections 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559afa787ee0 0x559afa78a350 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64d9ffb640 0 client.32716 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 0x559afa783490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa772420 0x559afa77c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 --2- 172.21.15.105:0/546401999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa77bf10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 -- 172.21.15.105:0/546401999 >> 172.21.15.105:0/546401999 conn(0x559afa76cfa0 msgr2=0x559afa76c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 -- 172.21.15.105:0/546401999 shutdown_connections 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 1 -- 172.21.15.105:0/546401999 wait complete. 2024-03-26T17:59:24.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.876+0000 7f64fbbca740 20 client.32716 trim_cache size 0 max 16384 2024-03-26T17:59:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.899+0000 7f64fbbca740 1 Processor -- start 2024-03-26T17:59:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.899+0000 7f64fbbca740 1 -- start start 2024-03-26T17:59:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.899+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa771a30 0x559afa77b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.899+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa772420 0x559afa783450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.899+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 0x559afa77ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64fbbca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41a6c10 con 0x559afa771a30 2024-03-26T17:59:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64fbbca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41a8750 con 0x559afa777590 2024-03-26T17:59:24.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64fbbca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41a72e0 con 0x559afa772420 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa771a30 0x559afa77b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 0x559afa77ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa771a30 0x559afa77b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53554/0 (socket says 172.21.15.105:53554) 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f1d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa772420 0x559afa783450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 -- 172.21.15.105:0/1972209233 learned_addr learned my addr 172.21.15.105:0/1972209233 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 -- 172.21.15.105:0/1972209233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa772420 msgr2=0x559afa783450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 --2- 172.21.15.105:0/1972209233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa772420 0x559afa783450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 -- 172.21.15.105:0/1972209233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 msgr2=0x559afa77ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 --2- 172.21.15.105:0/1972209233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 0x559afa77ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 -- 172.21.15.105:0/1972209233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559afa768820 con 0x559afa771a30 2024-03-26T17:59:24.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64f259c640 1 --2- 172.21.15.105:0/1972209233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa771a30 0x559afa77b530 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009900 tx=0x7f64e402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbe370c6f27af46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.900+0000 7f64dbfff640 1 -- 172.21.15.105:0/1972209233 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4000e60 con 0x559afa771a30 2024-03-26T17:59:24.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.901+0000 7f64dbfff640 1 -- 172.21.15.105:0/1972209233 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64e4005570 con 0x559afa771a30 2024-03-26T17:59:24.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.901+0000 7f64dbfff640 1 -- 172.21.15.105:0/1972209233 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4040c40 con 0x559afa771a30 2024-03-26T17:59:24.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.901+0000 7f64fbbca740 1 -- 172.21.15.105:0/1972209233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa771a30 msgr2=0x559afa77b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.901+0000 7f64fbbca740 1 --2- 172.21.15.105:0/1972209233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa771a30 0x559afa77b530 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009900 tx=0x7f64e402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 -- 172.21.15.105:0/1972209233 shutdown_connections 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 --2- 172.21.15.105:0/1972209233 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa777590 0x559afa77ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 --2- 172.21.15.105:0/1972209233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa772420 0x559afa783450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 --2- 172.21.15.105:0/1972209233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa771a30 0x559afa77b530 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 -- 172.21.15.105:0/1972209233 >> 172.21.15.105:0/1972209233 conn(0x559afa76cfa0 msgr2=0x559afa76c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 -- 172.21.15.105:0/1972209233 shutdown_connections 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 -- 172.21.15.105:0/1972209233 wait complete. 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 -- start start 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 Processor -- start 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa780730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.902+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa780c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64fbbca740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa781c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64fbbca740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41a8080 con 0x559afa777590 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64fbbca740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41a79b0 con 0x559afa772420 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64fbbca740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64f41a4bf0 con 0x559afa771a30 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa780c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa780c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42154/0 (socket says 172.21.15.105:42154) 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 -- 172.21.15.105:0/722651419 learned_addr learned my addr 172.21.15.105:0/722651419 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f259c640 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa780730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64fbbca740 10 client.0 _ll_register_callbacks cb 0x7f64e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f2d9d640 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa781c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 -- 172.21.15.105:0/722651419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 msgr2=0x559afa780730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa780730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 -- 172.21.15.105:0/722651419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 msgr2=0x559afa781c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa781c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 -- 172.21.15.105:0/722651419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559afa782220 con 0x559afa772420 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64f1d9b640 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa780c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f64e800b750 tx=0x7f64e809cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d061d8ffce57203d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:24.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64d9ffb640 1 -- 172.21.15.105:0/722651419 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e80ce5f0 con 0x559afa772420 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.903+0000 7f64d9ffb640 1 -- 172.21.15.105:0/722651419 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f64e80ce120 con 0x559afa772420 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 1 -- 172.21.15.105:0/722651419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e4005570 con 0x559afa772420 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64d9ffb640 1 -- 172.21.15.105:0/722651419 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f64e80b61e0 con 0x559afa772420 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 10 client.32732 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 1 -- 172.21.15.105:0/722651419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559afa783570 con 0x559afa772420 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64d9ffb640 1 -- 172.21.15.105:0/722651419 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64e4005570 con 0x559afa772420 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64d9ffb640 1 -- 172.21.15.105:0/722651419 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f64e80a5690 con 0x559afa772420 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 10 client.32732 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 10 client.32732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 20 client.32732 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559afa7ee800 0x559afa7f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:24.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 4 client.32732 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:24.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.904+0000 7f64fbbca740 1 -- 172.21.15.105:0/722651419 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559afa6de200 con 0x559afa7ee800 2024-03-26T17:59:24.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.905+0000 7f64f259c640 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559afa7ee800 0x559afa7f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:24.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.905+0000 7f64f259c640 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559afa7ee800 0x559afa7f0c20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b44d47eeaa289c93 server_cookie=475ec2fb9f2c5f01 in_seq=0 out_seq=0 2024-03-26T17:59:24.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.905+0000 7f64d9ffb640 10 client.32732 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:59:24.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64d9ffb640 1 -- 172.21.15.105:0/722651419 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559afa6de200 con 0x559afa7ee800 2024-03-26T17:59:24.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64d9ffb640 10 client.32732 handle_command_reply: tid=0 2024-03-26T17:59:24.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 client.32732 shutdown 2024-03-26T17:59:24.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 -- 172.21.15.105:0/722651419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 msgr2=0x559afa780c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa780c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f64e800b750 tx=0x7f64e809cd20 comp rx=0 tx=0).stop 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 -- 172.21.15.105:0/722651419 shutdown_connections 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x559afa7ee800 0x559afa7f0c20 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559afa777590 0x559afa781c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559afa772420 0x559afa780c70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 --2- 172.21.15.105:0/722651419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559afa771a30 0x559afa780730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 -- 172.21.15.105:0/722651419 >> 172.21.15.105:0/722651419 conn(0x559afa76cfa0 msgr2=0x559afa76b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64d9ffb640 0 client.32732 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.906+0000 7f64fbbca740 1 -- 172.21.15.105:0/722651419 shutdown_connections 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64fbbca740 1 -- 172.21.15.105:0/722651419 wait complete. 2024-03-26T17:59:24.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64fbbca740 20 client.32732 trim_cache size 0 max 16384 2024-03-26T17:59:24.894 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:24.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 -- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64dc0a2fa0 msgr2=0x7f64dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64dc0a2fa0 0x7f64dc0a5460 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f64ec005fd0 tx=0x7f64ec01e000 comp rx=0 tx=0).stop 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 -- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41019b0 msgr2=0x7f64f4196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41019b0 0x7f64f4196fb0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f64e800eb70 tx=0x7f64e800ef70 comp rx=0 tx=0).stop 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 -- 172.21.15.105:0/1318808793 shutdown_connections 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f64dc0a2fa0 0x7f64dc0a5460 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64f4102370 0x7f64f419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41019b0 0x7f64f4196fb0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 --2- 172.21.15.105:0/1318808793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f41007c0 0x7f64f4196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 -- 172.21.15.105:0/1318808793 >> 172.21.15.105:0/1318808793 conn(0x7f64f40fbd50 msgr2=0x7f64f4104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 -- 172.21.15.105:0/1318808793 shutdown_connections 2024-03-26T17:59:24.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:24.907+0000 7f64f8eff640 1 -- 172.21.15.105:0/1318808793 wait complete. 2024-03-26T17:59:24.903 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/1610036858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbc81007c0 msgr2=0x7fdbc8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/1610036858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbc81007c0 0x7fdbc8100be0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009900 tx=0x7fdbc402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/1610036858 shutdown_connections 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/1610036858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbc8102370 0x7fdbc8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/1610036858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbc81019b0 0x7fdbc8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/1610036858 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbc81007c0 0x7fdbc8100be0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/1610036858 >> 172.21.15.105:0/1610036858 conn(0x7fdbc80fbd50 msgr2=0x7fdbc80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/1610036858 shutdown_connections 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.059+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/1610036858 wait complete. 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcf4c6640 1 Processor -- start 2024-03-26T17:59:25.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcf4c6640 1 -- start start 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcf4c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbc81007c0 0x7fdbc8199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcf4c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbc81019b0 0x7fdbc819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcf4c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbc8102370 0x7fdbc81a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcf4c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8068db0 con 0x7fdbc8102370 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcf4c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8068c30 con 0x7fdbc81007c0 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcf4c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8068f30 con 0x7fdbc81019b0 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcd262640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbc81007c0 0x7fdbc8199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcd262640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbc81007c0 0x7fdbc8199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42182/0 (socket says 172.21.15.105:42182) 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcd262640 1 -- 172.21.15.105:0/2379389071 learned_addr learned my addr 172.21.15.105:0/2379389071 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:25.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcda63640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbc8102370 0x7fdbc81a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.060+0000 7fdbcca61640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbc81019b0 0x7fdbc819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbcca61640 1 -- 172.21.15.105:0/2379389071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbc81007c0 msgr2=0x7fdbc8199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbcca61640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbc81007c0 0x7fdbc8199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbcca61640 1 -- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbc8102370 msgr2=0x7fdbc81a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbcca61640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbc8102370 0x7fdbc81a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbcca61640 1 -- 172.21.15.105:0/2379389071 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc81a19d0 con 0x7fdbc81019b0 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbcca61640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbc81019b0 0x7fdbc819a2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800e9f0 tx=0x7fdbb800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbbe7fc640 1 -- 172.21.15.105:0/2379389071 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb800ccc0 con 0x7fdbc81019b0 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/2379389071 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc81a1cc0 con 0x7fdbc81019b0 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbbe7fc640 1 -- 172.21.15.105:0/2379389071 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbb80108e0 con 0x7fdbc81019b0 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbbe7fc640 1 -- 172.21.15.105:0/2379389071 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8018660 con 0x7fdbc81019b0 2024-03-26T17:59:25.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.061+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/2379389071 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc81a2200 con 0x7fdbc81019b0 2024-03-26T17:59:25.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.064+0000 7fdbd218e740 1 Processor -- start 2024-03-26T17:59:25.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.064+0000 7fdbbe7fc640 1 -- 172.21.15.105:0/2379389071 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdbb8010430 con 0x7fdbc81019b0 2024-03-26T17:59:25.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.064+0000 7fdbd218e740 1 -- start start 2024-03-26T17:59:25.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c3a30 0x55c21e7c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c4420 0x55c21e7c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c9590 0x55c21e7cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbd218e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c21e728970 con 0x55c21e7c9590 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbd218e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c21e6d0d70 con 0x55c21e7c3a30 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbd218e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c21e724920 con 0x55c21e7c4420 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c3a30 0x55c21e7c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c3a30 0x55c21e7c3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42188/0 (socket says 172.21.15.105:42188) 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 -- 172.21.15.105:0/542782219 learned_addr learned my addr 172.21.15.105:0/542782219 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcd262640 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c4420 0x55c21e7c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcca61640 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c9590 0x55c21e7cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 -- 172.21.15.105:0/542782219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c4420 msgr2=0x55c21e7c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c4420 0x55c21e7c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 -- 172.21.15.105:0/542782219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c9590 msgr2=0x55c21e7cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:25.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c9590 0x55c21e7cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 -- 172.21.15.105:0/542782219 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21e43c3f0 con 0x55c21e7c3a30 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcca61640 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c9590 0x55c21e7cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbbe7fc640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbb40a3020 0x7fdbb40a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.065+0000 7fdbcda63640 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c3a30 0x55c21e7c3e50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0011440 tx=0x7fdbc0011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c505edbcfc764e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.066+0000 7fdbcd262640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbb40a3020 0x7fdbb40a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.066+0000 7fdbbe7fc640 1 -- 172.21.15.105:0/2379389071 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fdbb80cf6c0 con 0x7fdbc81019b0 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.066+0000 7fdbb37fe640 1 -- 172.21.15.105:0/542782219 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0003c30 con 0x55c21e7c3a30 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.066+0000 7fdbb37fe640 1 -- 172.21.15.105:0/542782219 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbc0003dd0 con 0x55c21e7c3a30 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.066+0000 7fdbb37fe640 1 -- 172.21.15.105:0/542782219 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000fad0 con 0x55c21e7c3a30 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 -- 172.21.15.105:0/542782219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c3a30 msgr2=0x55c21e7c3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c3a30 0x55c21e7c3e50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0011440 tx=0x7fdbc0011900 comp rx=0 tx=0).stop 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbcd262640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbb40a3020 0x7fdbb40a54e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55c21e7c4e90 tx=0x7fdbc402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 -- 172.21.15.105:0/542782219 shutdown_connections 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c9590 0x55c21e7cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c4420 0x55c21e7c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 --2- 172.21.15.105:0/542782219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c3a30 0x55c21e7c3e50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 -- 172.21.15.105:0/542782219 >> 172.21.15.105:0/542782219 conn(0x55c21e7befa0 msgr2=0x55c21e7bf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:25.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 -- 172.21.15.105:0/542782219 shutdown_connections 2024-03-26T17:59:25.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 -- 172.21.15.105:0/542782219 wait complete. 2024-03-26T17:59:25.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 -- start start 2024-03-26T17:59:25.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.067+0000 7fdbd218e740 1 Processor -- start 2024-03-26T17:59:25.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbd218e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c21e7cc540 con 0x55c21e7c4420 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbd218e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c21e6e4310 con 0x55c21e7c9590 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbd218e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c21e7cc6c0 con 0x55c21e7c3a30 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbd218e740 10 client.0 _ll_register_callbacks cb 0x55c21e71fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbcd262640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbcca61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbcd262640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53358/0 (socket says 172.21.15.105:53358) 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbcda63640 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.068+0000 7fdbcd262640 1 -- 172.21.15.105:0/4156665387 learned_addr learned my addr 172.21.15.105:0/4156665387 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbcd262640 1 -- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 msgr2=0x55c21e7d5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbcd262640 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbcd262640 1 -- 172.21.15.105:0/4156665387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 msgr2=0x55c21e7ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbcd262640 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbcd262640 1 -- 172.21.15.105:0/4156665387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21e5067d0 con 0x55c21e7c3a30 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbcda63640 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbcd262640 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7cdfb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402c170 tx=0x7fdbc402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53be100877512bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/4156665387 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4050030 con 0x55c21e7c3a30 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbd218e740 1 -- 172.21.15.105:0/4156665387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc0003dd0 con 0x55c21e7c3a30 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/4156665387 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fdbc403a040 con 0x55c21e7c3a30 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/4156665387 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbc40046c0 con 0x55c21e7c3a30 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbd218e740 10 client.32734 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.069+0000 7fdbd218e740 1 -- 172.21.15.105:0/4156665387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c21e7d8960 con 0x55c21e7c3a30 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/4156665387 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbc0003dd0 con 0x55c21e7c3a30 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/4156665387 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fdbc4003960 con 0x55c21e7c3a30 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbd218e740 10 client.32734 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbd218e740 10 client.32734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:25.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbd218e740 20 client.32734 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:25.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbd218e740 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c21e7da200 0x55c21e7dc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbd218e740 4 client.32734 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:25.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbd218e740 1 -- 172.21.15.105:0/4156665387 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c21e735eb0 con 0x55c21e7da200 2024-03-26T17:59:25.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.070+0000 7fdbcca61640 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c21e7da200 0x55c21e7dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.071+0000 7fdbcca61640 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c21e7da200 0x55c21e7dc680 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b66055f80749a4 server_cookie=e45ecd1c86609c20 in_seq=0 out_seq=0 2024-03-26T17:59:25.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.071+0000 7fdbb0ff9640 10 client.32734 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:25.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.074+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/4156665387 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c21e735eb0 con 0x55c21e7da200 2024-03-26T17:59:25.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.074+0000 7fdbb0ff9640 10 client.32734 handle_command_reply: tid=0 2024-03-26T17:59:25.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.074+0000 7fdbd218e740 1 client.32734 shutdown 2024-03-26T17:59:25.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 -- 172.21.15.105:0/4156665387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 msgr2=0x55c21e7cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7cdfb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402c170 tx=0x7fdbc402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:25.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 -- 172.21.15.105:0/4156665387 shutdown_connections 2024-03-26T17:59:25.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c21e7da200 0x55c21e7dc680 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbb0ff9640 0 client.32734 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:25.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 --2- 172.21.15.105:0/4156665387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7cdfb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 -- 172.21.15.105:0/4156665387 >> 172.21.15.105:0/4156665387 conn(0x55c21e7befa0 msgr2=0x55c21e7be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:25.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 -- 172.21.15.105:0/4156665387 shutdown_connections 2024-03-26T17:59:25.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 1 -- 172.21.15.105:0/4156665387 wait complete. 2024-03-26T17:59:25.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.075+0000 7fdbd218e740 20 client.32734 trim_cache size 0 max 16384 2024-03-26T17:59:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbd218e740 1 Processor -- start 2024-03-26T17:59:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbd218e740 1 -- start start 2024-03-26T17:59:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbd218e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81aaf40 con 0x55c21e7c4420 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbd218e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81aca80 con 0x55c21e7c9590 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbd218e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81ab610 con 0x55c21e7c3a30 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcca61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcd262640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcca61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53620/0 (socket says 172.21.15.105:53620) 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcca61640 1 -- 172.21.15.105:0/3676148897 learned_addr learned my addr 172.21.15.105:0/3676148897 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcda63640 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcd262640 1 -- 172.21.15.105:0/3676148897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 msgr2=0x55c21e7bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcd262640 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7bd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcd262640 1 -- 172.21.15.105:0/3676148897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 msgr2=0x55c21e7d54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcd262640 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcd262640 1 -- 172.21.15.105:0/3676148897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21e7cc800 con 0x55c21e7c3a30 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.097+0000 7fdbcca61640 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.098+0000 7fdbcd262640 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7ce2d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4003d70 tx=0x7fdbc4005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3617165b735c545 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.098+0000 7fdbb2ffd640 1 -- 172.21.15.105:0/3676148897 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4050070 con 0x55c21e7c3a30 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.098+0000 7fdbb2ffd640 1 -- 172.21.15.105:0/3676148897 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbc4063030 con 0x55c21e7c3a30 2024-03-26T17:59:25.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.098+0000 7fdbb2ffd640 1 -- 172.21.15.105:0/3676148897 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc404c450 con 0x55c21e7c3a30 2024-03-26T17:59:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 -- 172.21.15.105:0/3676148897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 msgr2=0x55c21e7ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7ce2d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4003d70 tx=0x7fdbc4005720 comp rx=0 tx=0).stop 2024-03-26T17:59:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 -- 172.21.15.105:0/3676148897 shutdown_connections 2024-03-26T17:59:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 --2- 172.21.15.105:0/3676148897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7ce2d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 -- 172.21.15.105:0/3676148897 >> 172.21.15.105:0/3676148897 conn(0x55c21e7befa0 msgr2=0x55c21e7cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 -- 172.21.15.105:0/3676148897 shutdown_connections 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 -- 172.21.15.105:0/3676148897 wait complete. 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 -- start start 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.099+0000 7fdbd218e740 1 Processor -- start 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbd218e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbd218e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81ac3b0 con 0x55c21e7c4420 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbd218e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81abce0 con 0x55c21e7c9590 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbd218e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81a8f20 con 0x55c21e7c3a30 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbd218e740 10 client.0 _ll_register_callbacks cb 0x7fdbc4044050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:25.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbcd262640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbcd262640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7d2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53366/0 (socket says 172.21.15.105:53366) 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbcd262640 1 -- 172.21.15.105:0/951211727 learned_addr learned my addr 172.21.15.105:0/951211727 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbcda63640 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbcca61640 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbcd262640 1 -- 172.21.15.105:0/951211727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 msgr2=0x55c21e7d3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbcd262640 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.100+0000 7fdbcd262640 1 -- 172.21.15.105:0/951211727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 msgr2=0x55c21e7d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbcd262640 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbcd262640 1 -- 172.21.15.105:0/951211727 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21e7d4180 con 0x55c21e7c3a30 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbcd262640 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7d2640 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402ce10 tx=0x7fdbc402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e858059e345eb72 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/951211727 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc406d020 con 0x55c21e7c3a30 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/951211727 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fdbc40037f0 con 0x55c21e7c3a30 2024-03-26T17:59:25.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbd218e740 1 -- 172.21.15.105:0/951211727 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc4063030 con 0x55c21e7c3a30 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/951211727 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbc4078050 con 0x55c21e7c3a30 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbd218e740 10 client.32740 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.101+0000 7fdbd218e740 1 -- 172.21.15.105:0/951211727 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c21e7d5480 con 0x55c21e7c3a30 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/951211727 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbc4063030 con 0x55c21e7c3a30 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/951211727 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fdbc404c450 con 0x55c21e7c3a30 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbd218e740 10 client.32740 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbd218e740 10 client.32740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbd218e740 20 client.32740 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbd218e740 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c21e840830 0x55c21e842c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbd218e740 4 client.32740 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:25.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbd218e740 1 -- 172.21.15.105:0/951211727 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c21e730200 con 0x55c21e840830 2024-03-26T17:59:25.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbcca61640 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c21e840830 0x55c21e842c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:25.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbcca61640 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c21e840830 0x55c21e842c50 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb6d50981dda2307 server_cookie=d60b8791c5b85240 in_seq=0 out_seq=0 2024-03-26T17:59:25.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.102+0000 7fdbb0ff9640 10 client.32740 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:25.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbb0ff9640 1 -- 172.21.15.105:0/951211727 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c21e730200 con 0x55c21e840830 2024-03-26T17:59:25.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbb0ff9640 10 client.32740 handle_command_reply: tid=0 2024-03-26T17:59:25.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 client.32740 shutdown 2024-03-26T17:59:25.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 -- 172.21.15.105:0/951211727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 msgr2=0x55c21e7d2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7d2640 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fdbc402ce10 tx=0x7fdbc402bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 -- 172.21.15.105:0/951211727 shutdown_connections 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55c21e840830 0x55c21e842c50 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21e7c9590 0x55c21e7d3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c21e7c4420 0x55c21e7d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 --2- 172.21.15.105:0/951211727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c21e7c3a30 0x55c21e7d2640 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 -- 172.21.15.105:0/951211727 >> 172.21.15.105:0/951211727 conn(0x55c21e7befa0 msgr2=0x55c21e7ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbb0ff9640 0 client.32740 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 -- 172.21.15.105:0/951211727 shutdown_connections 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.104+0000 7fdbd218e740 1 -- 172.21.15.105:0/951211727 wait complete. 2024-03-26T17:59:25.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.105+0000 7fdbd218e740 20 client.32740 trim_cache size 0 max 16384 2024-03-26T17:59:25.094 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:25.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.105+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbb40a3020 msgr2=0x7fdbb40a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.105+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbb40a3020 0x7fdbb40a54e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x55c21e7c4e90 tx=0x7fdbc402c7e0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.105+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/2379389071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbc81019b0 msgr2=0x7fdbc819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:25.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.105+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbc81019b0 0x7fdbc819a2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800e9f0 tx=0x7fdbb800edf0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.105+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/2379389071 shutdown_connections 2024-03-26T17:59:25.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.105+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbb40a3020 0x7fdbb40a54e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.106+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbc8102370 0x7fdbc81a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.106+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbc81019b0 0x7fdbc819a2f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.106+0000 7fdbcf4c6640 1 --2- 172.21.15.105:0/2379389071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbc81007c0 0x7fdbc8199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:25.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.106+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/2379389071 >> 172.21.15.105:0/2379389071 conn(0x7fdbc80fbd50 msgr2=0x7fdbc80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:25.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.106+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/2379389071 shutdown_connections 2024-03-26T17:59:25.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:25.106+0000 7fdbcf4c6640 1 -- 172.21.15.105:0/2379389071 wait complete. 2024-03-26T17:59:25.101 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.101 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.101 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:24 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3373830833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:25.102 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:25.102 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:25.102 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:24 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3373830833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:25.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:25.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:25.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:24 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3373830833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:25.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T17:59:25.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T17:59:25.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T17:59:25.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 64113 7.10 MB/sec execute 368 sec latency 157.133 ms 2024-03-26T17:59:26.104 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:59:26.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:26.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:59:26.132 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:25 smithi105 ceph-mon[32280]: pgmap v983: 97 pgs: 97 active+clean; 343 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-03-26T17:59:26.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:25 smithi135 ceph-mon[35636]: pgmap v983: 97 pgs: 97 active+clean; 343 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-03-26T17:59:26.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 -- 172.21.15.105:0/3062174911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581019d0 msgr2=0x7f4158101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 --2- 172.21.15.105:0/3062174911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581019d0 0x7f4158101e50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f414c009900 tx=0x7f414c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 -- 172.21.15.105:0/3062174911 shutdown_connections 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 --2- 172.21.15.105:0/3062174911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4158102390 0x7f4158108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 --2- 172.21.15.105:0/3062174911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581019d0 0x7f4158101e50 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 --2- 172.21.15.105:0/3062174911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41581007e0 0x7f4158100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 -- 172.21.15.105:0/3062174911 >> 172.21.15.105:0/3062174911 conn(0x7f41580fbd70 msgr2=0x7f41580fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 -- 172.21.15.105:0/3062174911 shutdown_connections 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.268+0000 7f415d073640 1 -- 172.21.15.105:0/3062174911 wait complete. 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415d073640 1 Processor -- start 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415d073640 1 -- start start 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415d073640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581007e0 0x7f415806a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415d073640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41581019d0 0x7f415819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415d073640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4158102390 0x7f415819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415d073640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f415810c310 con 0x7f41581007e0 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415d073640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f415810c190 con 0x7f41581019d0 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415d073640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f415810c490 con 0x7f4158102390 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581007e0 0x7f415806a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581007e0 0x7f415806a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53632/0 (socket says 172.21.15.105:53632) 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f4156d9d640 1 -- 172.21.15.105:0/2622680782 learned_addr learned my addr 172.21.15.105:0/2622680782 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415759e640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4158102390 0x7f415819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.269+0000 7f415659c640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41581019d0 0x7f415819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f4156d9d640 1 -- 172.21.15.105:0/2622680782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4158102390 msgr2=0x7f415819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f4156d9d640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4158102390 0x7f415819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f4156d9d640 1 -- 172.21.15.105:0/2622680782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41581019d0 msgr2=0x7f415819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f4156d9d640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41581019d0 0x7f415819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f4156d9d640 1 -- 172.21.15.105:0/2622680782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415819f7b0 con 0x7f41581007e0 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f415659c640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41581019d0 0x7f415819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f4156d9d640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581007e0 0x7f415806a9e0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f414800e9c0 tx=0x7f414800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f413ffff640 1 -- 172.21.15.105:0/2622680782 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414800cd80 con 0x7f41581007e0 2024-03-26T17:59:26.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f413ffff640 1 -- 172.21.15.105:0/2622680782 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41480035f0 con 0x7f41581007e0 2024-03-26T17:59:26.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f415d073640 1 -- 172.21.15.105:0/2622680782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415819fa40 con 0x7f41581007e0 2024-03-26T17:59:26.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f413ffff640 1 -- 172.21.15.105:0/2622680782 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148010690 con 0x7f41581007e0 2024-03-26T17:59:26.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.270+0000 7f415d073640 1 -- 172.21.15.105:0/2622680782 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415819ff10 con 0x7f41581007e0 2024-03-26T17:59:26.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.273+0000 7f413ffff640 1 -- 172.21.15.105:0/2622680782 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f41480037a0 con 0x7f41581007e0 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415fc5e740 1 Processor -- start 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415fc5e740 1 -- start start 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f413ffff640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f41400a2fa0 0x7f41400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d90d190 0x55ad9d90d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f413ffff640 1 -- 172.21.15.105:0/2622680782 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f41480cf990 con 0x7f41581007e0 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d912680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415759e640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f41400a2fa0 0x7f41400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d90d190 0x55ad9d90d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d912680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d90d190 0x55ad9d90d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53390/0 (socket says 172.21.15.105:53390) 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f4156d9d640 1 -- 172.21.15.105:0/1278379681 learned_addr learned my addr 172.21.15.105:0/1278379681 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d915180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415fc5e740 1 -- 172.21.15.105:0/1278379681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d872630 con 0x55ad9d90db80 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415fc5e740 1 -- 172.21.15.105:0/1278379681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d81ae10 con 0x55ad9d912cf0 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415fc5e740 1 -- 172.21.15.105:0/1278379681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d86e5e0 con 0x55ad9d90d190 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.274+0000 7f415759e640 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d915180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f4156d9d640 1 -- 172.21.15.105:0/1278379681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 msgr2=0x55ad9d915180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f4156d9d640 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d915180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f4156d9d640 1 -- 172.21.15.105:0/1278379681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 msgr2=0x55ad9d912680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f4156d9d640 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d912680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f4156d9d640 1 -- 172.21.15.105:0/1278379681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9d831740 con 0x55ad9d90d190 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f415759e640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f41400a2fa0 0x7f41400a5460 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4150005fd0 tx=0x7f4150011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f4156d9d640 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d90d190 0x55ad9d90d5b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4148098480 tx=0x7f414809b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=598ca7fcdca06896 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f413d7fa640 1 -- 172.21.15.105:0/1278379681 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414809c020 con 0x55ad9d90d190 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f413d7fa640 1 -- 172.21.15.105:0/1278379681 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41480a0170 con 0x55ad9d90d190 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.275+0000 7f413d7fa640 1 -- 172.21.15.105:0/1278379681 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148097d70 con 0x55ad9d90d190 2024-03-26T17:59:26.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 -- 172.21.15.105:0/1278379681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d90d190 msgr2=0x55ad9d90d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d90d190 0x55ad9d90d5b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4148098480 tx=0x7f414809b980 comp rx=0 tx=0).stop 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 -- 172.21.15.105:0/1278379681 shutdown_connections 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d915180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d912680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1278379681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d90d190 0x55ad9d90d5b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 -- 172.21.15.105:0/1278379681 >> 172.21.15.105:0/1278379681 conn(0x55ad9d908700 msgr2=0x55ad9d908b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 -- 172.21.15.105:0/1278379681 shutdown_connections 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 -- 172.21.15.105:0/1278379681 wait complete. 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 -- start start 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.276+0000 7f415fc5e740 1 Processor -- start 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d90d190 0x55ad9d917670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d917bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d912cf0 0x55ad9d91ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415fc5e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d915ca0 con 0x55ad9d90db80 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415fc5e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d82e1a0 con 0x55ad9d90d190 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415fc5e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ad9d915e20 con 0x55ad9d912cf0 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415fc5e740 10 client.0 _ll_register_callbacks cb 0x55ad9d585c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d90d190 0x55ad9d917670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d90d190 0x55ad9d917670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42244/0 (socket says 172.21.15.105:42244) 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 -- 172.21.15.105:0/2563098136 learned_addr learned my addr 172.21.15.105:0/2563098136 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415759e640 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d912cf0 0x55ad9d91ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f415659c640 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d917bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 -- 172.21.15.105:0/2563098136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d912cf0 msgr2=0x55ad9d91ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d912cf0 0x55ad9d91ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 -- 172.21.15.105:0/2563098136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 msgr2=0x55ad9d917bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d917bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 -- 172.21.15.105:0/2563098136 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9d64e7e0 con 0x55ad9d90d190 2024-03-26T17:59:26.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f4156d9d640 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d90d190 0x55ad9d917670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f41480a00e0 tx=0x7f414809bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6487f59c21fd8275 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f413aff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.277+0000 7f413aff5640 1 -- 172.21.15.105:0/2563098136 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414809c020 con 0x55ad9d90d190 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 1 -- 172.21.15.105:0/2563098136 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41480a0170 con 0x55ad9d90d190 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f413aff5640 1 -- 172.21.15.105:0/2563098136 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f414809b680 con 0x55ad9d90d190 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f413aff5640 1 -- 172.21.15.105:0/2563098136 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4148097d70 con 0x55ad9d90d190 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 10 client.32762 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 1 -- 172.21.15.105:0/2563098136 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ad9d6aef70 con 0x55ad9d90d190 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f413aff5640 1 -- 172.21.15.105:0/2563098136 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41480a0170 con 0x55ad9d90d190 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f413aff5640 1 -- 172.21.15.105:0/2563098136 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f41480a1300 con 0x55ad9d90d190 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 10 client.32762 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 10 client.32762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 20 client.32762 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad9d9202e0 0x55ad9d9226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 4 client.32762 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:26.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415fc5e740 1 -- 172.21.15.105:0/2563098136 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad9d87fb70 con 0x55ad9d9202e0 2024-03-26T17:59:26.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.278+0000 7f415659c640 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad9d9202e0 0x55ad9d9226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.279+0000 7f413aff5640 10 client.32762 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.279+0000 7f415659c640 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad9d9202e0 0x55ad9d9226f0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e97a5891194e31 server_cookie=1d8c2e36c62be0dc in_seq=0 out_seq=0 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.280+0000 7f413aff5640 1 -- 172.21.15.105:0/2563098136 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ad9d87fb70 con 0x55ad9d9202e0 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.280+0000 7f413aff5640 10 client.32762 handle_command_reply: tid=0 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.280+0000 7f415fc5e740 1 client.32762 shutdown 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.280+0000 7f415fc5e740 1 -- 172.21.15.105:0/2563098136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d90d190 msgr2=0x55ad9d917670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d90d190 0x55ad9d917670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f41480a00e0 tx=0x7f414809bab0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 -- 172.21.15.105:0/2563098136 shutdown_connections 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad9d9202e0 0x55ad9d9226f0 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d912cf0 0x55ad9d91ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d917bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f413aff5640 0 client.32762 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 --2- 172.21.15.105:0/2563098136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d90d190 0x55ad9d917670 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 -- 172.21.15.105:0/2563098136 >> 172.21.15.105:0/2563098136 conn(0x55ad9d908700 msgr2=0x55ad9d907900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 -- 172.21.15.105:0/2563098136 shutdown_connections 2024-03-26T17:59:26.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 1 -- 172.21.15.105:0/2563098136 wait complete. 2024-03-26T17:59:26.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.281+0000 7f415fc5e740 20 client.32762 trim_cache size 0 max 16384 2024-03-26T17:59:26.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.308+0000 7f415fc5e740 1 Processor -- start 2024-03-26T17:59:26.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.308+0000 7f415fc5e740 1 -- start start 2024-03-26T17:59:26.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.308+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d917990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.308+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.308+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d907070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415fc5e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a6a70 con 0x55ad9d90db80 2024-03-26T17:59:26.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415fc5e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a85b0 con 0x55ad9d912cf0 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415fc5e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a7140 con 0x55ad9d908700 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415759e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d907070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53648/0 (socket says 172.21.15.105:53648) 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d917990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 -- 172.21.15.105:0/1246682634 learned_addr learned my addr 172.21.15.105:0/1246682634 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 -- 172.21.15.105:0/1246682634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 msgr2=0x55ad9d917990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 --2- 172.21.15.105:0/1246682634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d917990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 -- 172.21.15.105:0/1246682634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 msgr2=0x55ad9d907070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 --2- 172.21.15.105:0/1246682634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d907070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 -- 172.21.15.105:0/1246682634 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad9d90f700 con 0x55ad9d90db80 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415759e640 1 --2- 172.21.15.105:0/1246682634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d907070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:26.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f415659c640 1 --2- 172.21.15.105:0/1246682634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91ebb0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f414c02c170 tx=0x7f414c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8855be49492ab37d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.309+0000 7f413cff9640 1 -- 172.21.15.105:0/1246682634 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c04f020 con 0x55ad9d90db80 2024-03-26T17:59:26.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.310+0000 7f413cff9640 1 -- 172.21.15.105:0/1246682634 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f414c04bab0 con 0x55ad9d90db80 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.310+0000 7f413cff9640 1 -- 172.21.15.105:0/1246682634 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c049070 con 0x55ad9d90db80 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.310+0000 7f415fc5e740 1 -- 172.21.15.105:0/1246682634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 msgr2=0x55ad9d91ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.310+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1246682634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91ebb0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f414c02c170 tx=0x7f414c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 -- 172.21.15.105:0/1246682634 shutdown_connections 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1246682634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d907070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1246682634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91ebb0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1246682634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d917990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 -- 172.21.15.105:0/1246682634 >> 172.21.15.105:0/1246682634 conn(0x55ad9d922a40 msgr2=0x55ad9d917280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 -- 172.21.15.105:0/1246682634 shutdown_connections 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 -- 172.21.15.105:0/1246682634 wait complete. 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 -- start start 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 Processor -- start 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d91c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d91dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a7ee0 con 0x55ad9d90db80 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a7810 con 0x55ad9d912cf0 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41581a4a50 con 0x55ad9d908700 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415fc5e740 10 client.0 _ll_register_callbacks cb 0x7f4148005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.311+0000 7f415659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d91c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53664/0 (socket says 172.21.15.105:53664) 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415659c640 1 -- 172.21.15.105:0/1728489731 learned_addr learned my addr 172.21.15.105:0/1728489731 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415759e640 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d91dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415659c640 1 -- 172.21.15.105:0/1728489731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 msgr2=0x55ad9d91c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415659c640 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d91c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415659c640 1 -- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 msgr2=0x55ad9d91dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415659c640 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d91dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415659c640 1 -- 172.21.15.105:0/1728489731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad9d546170 con 0x55ad9d90db80 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f4156d9d640 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d91c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415759e640 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d91dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415659c640 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91cba0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f414c02ce10 tx=0x7f414c04b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bf90497237b3d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f413aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f413aff5640 1 -- 172.21.15.105:0/1728489731 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c04f040 con 0x55ad9d90db80 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f413aff5640 1 -- 172.21.15.105:0/1728489731 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f414c004380 con 0x55ad9d90db80 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.312+0000 7f415fc5e740 1 -- 172.21.15.105:0/1728489731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f414c04bab0 con 0x55ad9d90db80 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f413aff5640 1 -- 172.21.15.105:0/1728489731 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f414c005040 con 0x55ad9d90db80 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f415fc5e740 10 client.23085 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f415fc5e740 1 -- 172.21.15.105:0/1728489731 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ad9d992460 con 0x55ad9d90db80 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f413aff5640 1 -- 172.21.15.105:0/1728489731 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f414c04bab0 con 0x55ad9d90db80 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f413aff5640 1 -- 172.21.15.105:0/1728489731 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f414c004b00 con 0x55ad9d90db80 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f415fc5e740 10 client.23085 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:26.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f415fc5e740 10 client.23085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:26.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f415fc5e740 20 client.23085 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:26.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad9d992e30 0x55ad9d995250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f415fc5e740 4 client.23085 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:59:26.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f415fc5e740 1 -- 172.21.15.105:0/1728489731 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ad9d879ec0 con 0x55ad9d992e30 2024-03-26T17:59:26.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.313+0000 7f4156d9d640 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad9d992e30 0x55ad9d995250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.314+0000 7f4156d9d640 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad9d992e30 0x55ad9d995250 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33cbd598473cd58 server_cookie=eae07ffec667fe20 in_seq=0 out_seq=0 2024-03-26T17:59:26.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.314+0000 7f413aff5640 10 client.23085 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.315+0000 7f413aff5640 1 -- 172.21.15.105:0/1728489731 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ad9d879ec0 con 0x55ad9d992e30 2024-03-26T17:59:26.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.315+0000 7f413aff5640 10 client.23085 handle_command_reply: tid=0 2024-03-26T17:59:26.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.315+0000 7f415fc5e740 1 client.23085 shutdown 2024-03-26T17:59:26.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 -- 172.21.15.105:0/1728489731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 msgr2=0x55ad9d91cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91cba0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f414c02ce10 tx=0x7f414c04b830 comp rx=0 tx=0).stop 2024-03-26T17:59:26.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 -- 172.21.15.105:0/1728489731 shutdown_connections 2024-03-26T17:59:26.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ad9d992e30 0x55ad9d995250 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ad9d912cf0 0x55ad9d91dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ad9d90db80 0x55ad9d91cba0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 --2- 172.21.15.105:0/1728489731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ad9d908700 0x55ad9d91c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f413aff5640 0 client.23085 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 -- 172.21.15.105:0/1728489731 >> 172.21.15.105:0/1728489731 conn(0x55ad9d922a40 msgr2=0x55ad9d90b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 -- 172.21.15.105:0/1728489731 shutdown_connections 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 1 -- 172.21.15.105:0/1728489731 wait complete. 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.316+0000 7f415fc5e740 20 client.23085 trim_cache size 0 max 16384 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:59:26.302 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "d4d618d6-0a93-4fcd-aeee-2de78943c481", 2024-03-26T17:59:26.303 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:59:26.303 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:59:26.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 -- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f41400a2fa0 msgr2=0x7f41400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f41400a2fa0 0x7f41400a5460 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4150005fd0 tx=0x7f4150011810 comp rx=0 tx=0).stop 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 -- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581007e0 msgr2=0x7f415806a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581007e0 0x7f415806a9e0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f414800e9c0 tx=0x7f414800ee80 comp rx=0 tx=0).stop 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 -- 172.21.15.105:0/2622680782 shutdown_connections 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f41400a2fa0 0x7f41400a5460 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4158102390 0x7f415819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f41581019d0 0x7f415819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 --2- 172.21.15.105:0/2622680782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41581007e0 0x7f415806a9e0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 -- 172.21.15.105:0/2622680782 >> 172.21.15.105:0/2622680782 conn(0x7f41580fbd70 msgr2=0x7f4158104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 -- 172.21.15.105:0/2622680782 shutdown_connections 2024-03-26T17:59:26.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.317+0000 7f415d073640 1 -- 172.21.15.105:0/2622680782 wait complete. 2024-03-26T17:59:26.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:59:26.314 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:59:26.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:25 smithi144 ceph-mon[36095]: pgmap v983: 97 pgs: 97 active+clean; 343 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 4.0 MiB/s wr, 230 op/s 2024-03-26T17:59:26.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 -- 172.21.15.105:0/2437552628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 msgr2=0x7fda3c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 --2- 172.21.15.105:0/2437552628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 0x7fda3c1070c0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009900 tx=0x7fda2c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 -- 172.21.15.105:0/2437552628 shutdown_connections 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 --2- 172.21.15.105:0/2437552628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3c107600 0x7fda3c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 --2- 172.21.15.105:0/2437552628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 0x7fda3c1070c0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 --2- 172.21.15.105:0/2437552628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda3c1023a0 0x7fda3c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 -- 172.21.15.105:0/2437552628 >> 172.21.15.105:0/2437552628 conn(0x7fda3c0fbd50 msgr2=0x7fda3c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 -- 172.21.15.105:0/2437552628 shutdown_connections 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.502+0000 7fda44427640 1 -- 172.21.15.105:0/2437552628 wait complete. 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda44427640 1 Processor -- start 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda44427640 1 -- start start 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda44427640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda3c1023a0 0x7fda3c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda44427640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 0x7fda3c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda44427640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3c107600 0x7fda3c1a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda44427640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1014f0 con 0x7fda3c104cd0 2024-03-26T17:59:26.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda44427640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c101370 con 0x7fda3c1023a0 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda44427640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c101670 con 0x7fda3c107600 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda419c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 0x7fda3c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.503+0000 7fda421c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda3c1023a0 0x7fda3c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda419c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 0x7fda3c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53676/0 (socket says 172.21.15.105:53676) 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda419c2640 1 -- 172.21.15.105:0/673575510 learned_addr learned my addr 172.21.15.105:0/673575510 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda429c4640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3c107600 0x7fda3c1a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda419c2640 1 -- 172.21.15.105:0/673575510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3c107600 msgr2=0x7fda3c1a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda419c2640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3c107600 0x7fda3c1a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda419c2640 1 -- 172.21.15.105:0/673575510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda3c1023a0 msgr2=0x7fda3c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda419c2640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda3c1023a0 0x7fda3c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda419c2640 1 -- 172.21.15.105:0/673575510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c1a19d0 con 0x7fda3c104cd0 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda419c2640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 0x7fda3c19a2f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fda2c000c00 tx=0x7fda2c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda337fe640 1 -- 172.21.15.105:0/673575510 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c043020 con 0x7fda3c104cd0 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda337fe640 1 -- 172.21.15.105:0/673575510 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda2c004350 con 0x7fda3c104cd0 2024-03-26T17:59:26.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda337fe640 1 -- 172.21.15.105:0/673575510 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c003b80 con 0x7fda3c104cd0 2024-03-26T17:59:26.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda44427640 1 -- 172.21.15.105:0/673575510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c1a1c60 con 0x7fda3c104cd0 2024-03-26T17:59:26.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.504+0000 7fda44427640 1 -- 172.21.15.105:0/673575510 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a2160 con 0x7fda3c104cd0 2024-03-26T17:59:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.508+0000 7fda337fe640 1 -- 172.21.15.105:0/673575510 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fda2c046020 con 0x7fda3c104cd0 2024-03-26T17:59:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.508+0000 7fda47010740 1 Processor -- start 2024-03-26T17:59:26.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.508+0000 7fda47010740 1 -- start start 2024-03-26T17:59:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda47010740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f591d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda47010740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f596de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda47010740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda47010740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55890f4f6980 con 0x55890f5918f0 2024-03-26T17:59:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda47010740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55890f49ebe0 con 0x55890f5922e0 2024-03-26T17:59:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda47010740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55890f4f2930 con 0x55890f597450 2024-03-26T17:59:26.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda429c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f591d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda421c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f596de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda421c3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f596de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42300/0 (socket says 172.21.15.105:42300) 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda421c3640 1 -- 172.21.15.105:0/270068490 learned_addr learned my addr 172.21.15.105:0/270068490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda419c2640 1 --2- 172.21.15.105:0/270068490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda421c3640 1 -- 172.21.15.105:0/270068490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 msgr2=0x55890f5998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda421c3640 1 --2- 172.21.15.105:0/270068490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda421c3640 1 -- 172.21.15.105:0/270068490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 msgr2=0x55890f591d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda421c3640 1 --2- 172.21.15.105:0/270068490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f591d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda421c3640 1 -- 172.21.15.105:0/270068490 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890f2d47d0 con 0x55890f5922e0 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda337fe640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda280a2fa0 0x7fda280a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.509+0000 7fda429c4640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda280a2fa0 0x7fda280a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.510+0000 7fda337fe640 1 -- 172.21.15.105:0/673575510 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fda2c0f1540 con 0x7fda3c104cd0 2024-03-26T17:59:26.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.510+0000 7fda421c3640 1 --2- 172.21.15.105:0/270068490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f596de0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ce0 tx=0x7fda38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30d7dbda9421cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.510+0000 7fda30bfe640 1 -- 172.21.15.105:0/270068490 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38004220 con 0x55890f5922e0 2024-03-26T17:59:26.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.510+0000 7fda30bfe640 1 -- 172.21.15.105:0/270068490 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda380043c0 con 0x55890f5922e0 2024-03-26T17:59:26.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.510+0000 7fda30bfe640 1 -- 172.21.15.105:0/270068490 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38019810 con 0x55890f5922e0 2024-03-26T17:59:26.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 -- 172.21.15.105:0/270068490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 msgr2=0x55890f596de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda429c4640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda280a2fa0 0x7fda280a5460 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x55890f592b40 tx=0x7fda34009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 --2- 172.21.15.105:0/270068490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f596de0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ce0 tx=0x7fda38011810 comp rx=0 tx=0).stop 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 -- 172.21.15.105:0/270068490 shutdown_connections 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 --2- 172.21.15.105:0/270068490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 --2- 172.21.15.105:0/270068490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f596de0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 --2- 172.21.15.105:0/270068490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f591d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 -- 172.21.15.105:0/270068490 >> 172.21.15.105:0/270068490 conn(0x55890f58ce60 msgr2=0x55890f58d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 -- 172.21.15.105:0/270068490 shutdown_connections 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.511+0000 7fda47010740 1 -- 172.21.15.105:0/270068490 wait complete. 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 1 -- start start 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 1 Processor -- start 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f5a4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55890f59a400 con 0x55890f5918f0 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55890f4b2160 con 0x55890f5922e0 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55890f59a580 con 0x55890f597450 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda421c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda47010740 10 client.0 _ll_register_callbacks cb 0x55890f4ee720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda421c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5a4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53700/0 (socket says 172.21.15.105:53700) 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda421c3640 1 -- 172.21.15.105:0/1338203359 learned_addr learned my addr 172.21.15.105:0/1338203359 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda429c4640 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.512+0000 7fda419c2640 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f5a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda429c4640 1 -- 172.21.15.105:0/1338203359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 msgr2=0x55890f5a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda429c4640 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f5a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda429c4640 1 -- 172.21.15.105:0/1338203359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 msgr2=0x55890f5a4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda429c4640 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5a4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda429c4640 1 -- 172.21.15.105:0/1338203359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890f20a3f0 con 0x55890f597450 2024-03-26T17:59:26.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda421c3640 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda429c4640 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5ab6b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fda34005fd0 tx=0x7fda34004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a930c9fcdf7911a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda25ffb640 1 -- 172.21.15.105:0/1338203359 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3400f870 con 0x55890f597450 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda47010740 1 -- 172.21.15.105:0/1338203359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda380043c0 con 0x55890f597450 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda25ffb640 1 -- 172.21.15.105:0/1338203359 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fda3400fa10 con 0x55890f597450 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda25ffb640 1 -- 172.21.15.105:0/1338203359 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fda3400fc10 con 0x55890f597450 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda47010740 10 client.32770 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda47010740 1 -- 172.21.15.105:0/1338203359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55890f1ca0f0 con 0x55890f597450 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.513+0000 7fda25ffb640 1 -- 172.21.15.105:0/1338203359 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda380043c0 con 0x55890f597450 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.514+0000 7fda25ffb640 1 -- 172.21.15.105:0/1338203359 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fda34026070 con 0x55890f597450 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.514+0000 7fda47010740 10 client.32770 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.514+0000 7fda47010740 10 client.32770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.514+0000 7fda47010740 20 client.32770 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.514+0000 7fda47010740 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890f5b03d0 0x55890f5b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.514+0000 7fda47010740 4 client.32770 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:26.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.514+0000 7fda47010740 1 -- 172.21.15.105:0/1338203359 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55890f503ec0 con 0x55890f5b03d0 2024-03-26T17:59:26.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.514+0000 7fda421c3640 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890f5b03d0 0x55890f5b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.515+0000 7fda421c3640 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890f5b03d0 0x55890f5b2890 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6c45e4d513d4c2 server_cookie=a2a49d864033cec1 in_seq=0 out_seq=0 2024-03-26T17:59:26.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.515+0000 7fda25ffb640 10 client.32770 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda25ffb640 1 -- 172.21.15.105:0/1338203359 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55890f503ec0 con 0x55890f5b03d0 2024-03-26T17:59:26.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda25ffb640 10 client.32770 handle_command_reply: tid=0 2024-03-26T17:59:26.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 client.32770 shutdown 2024-03-26T17:59:26.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 -- 172.21.15.105:0/1338203359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 msgr2=0x55890f5ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5ab6b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fda34005fd0 tx=0x7fda34004ab0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 -- 172.21.15.105:0/1338203359 shutdown_connections 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890f5b03d0 0x55890f5b2890 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5ab6b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5922e0 0x55890f5a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 --2- 172.21.15.105:0/1338203359 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5a4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda47010740 1 -- 172.21.15.105:0/1338203359 >> 172.21.15.105:0/1338203359 conn(0x55890f58ce60 msgr2=0x55890f58c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.517+0000 7fda25ffb640 0 client.32770 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.518+0000 7fda47010740 1 -- 172.21.15.105:0/1338203359 shutdown_connections 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.518+0000 7fda47010740 1 -- 172.21.15.105:0/1338203359 wait complete. 2024-03-26T17:59:26.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.518+0000 7fda47010740 20 client.32770 trim_cache size 0 max 16384 2024-03-26T17:59:26.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.541+0000 7fda47010740 1 Processor -- start 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda47010740 1 -- start start 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda47010740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda47010740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f5922e0 0x55890f596e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda47010740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f597450 0x55890f593320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda47010740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1a6ca0 con 0x55890f5918f0 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda47010740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1a87e0 con 0x55890f597450 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda47010740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1a7370 con 0x55890f5922e0 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda421c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda421c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53702/0 (socket says 172.21.15.105:53702) 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda421c3640 1 -- 172.21.15.105:0/3626216855 learned_addr learned my addr 172.21.15.105:0/3626216855 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda429c4640 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f597450 0x55890f593320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda419c2640 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f5922e0 0x55890f596e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda421c3640 1 -- 172.21.15.105:0/3626216855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f5922e0 msgr2=0x55890f596e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda421c3640 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f5922e0 0x55890f596e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda421c3640 1 -- 172.21.15.105:0/3626216855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f597450 msgr2=0x55890f593320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda421c3640 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f597450 0x55890f593320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.542+0000 7fda421c3640 1 -- 172.21.15.105:0/3626216855 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890f58d8b0 con 0x55890f5918f0 2024-03-26T17:59:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.543+0000 7fda421c3640 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5968f0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fda38011d10 tx=0x7fda38003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba752377d717ebe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.543+0000 7fda27fff640 1 -- 172.21.15.105:0/3626216855 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3802f020 con 0x55890f5918f0 2024-03-26T17:59:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.543+0000 7fda27fff640 1 -- 172.21.15.105:0/3626216855 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda38003670 con 0x55890f5918f0 2024-03-26T17:59:26.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.543+0000 7fda27fff640 1 -- 172.21.15.105:0/3626216855 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3802b530 con 0x55890f5918f0 2024-03-26T17:59:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 -- 172.21.15.105:0/3626216855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 msgr2=0x55890f5968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5968f0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fda38011d10 tx=0x7fda38003c90 comp rx=0 tx=0).stop 2024-03-26T17:59:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 -- 172.21.15.105:0/3626216855 shutdown_connections 2024-03-26T17:59:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f597450 0x55890f593320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f5922e0 0x55890f596e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 --2- 172.21.15.105:0/3626216855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5918f0 0x55890f5968f0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 -- 172.21.15.105:0/3626216855 >> 172.21.15.105:0/3626216855 conn(0x55890f58ce60 msgr2=0x55890f587e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 -- 172.21.15.105:0/3626216855 shutdown_connections 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 -- 172.21.15.105:0/3626216855 wait complete. 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.544+0000 7fda47010740 1 -- start start 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda47010740 1 Processor -- start 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda47010740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5918f0 0x55890f5b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda47010740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5922e0 0x55890f5b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda47010740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda47010740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1a8110 con 0x55890f5922e0 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda47010740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1a7a40 con 0x55890f5918f0 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda47010740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1a4c80 con 0x55890f597450 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda47010740 10 client.0 _ll_register_callbacks cb 0x7fda34009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda419c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5922e0 0x55890f5b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda419c2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5922e0 0x55890f5b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53716/0 (socket says 172.21.15.105:53716) 2024-03-26T17:59:26.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda421c3640 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5918f0 0x55890f5b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda419c2640 1 -- 172.21.15.105:0/786889446 learned_addr learned my addr 172.21.15.105:0/786889446 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda429c4640 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda419c2640 1 -- 172.21.15.105:0/786889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 msgr2=0x55890f5b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda419c2640 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda419c2640 1 -- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5918f0 msgr2=0x55890f5b6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda419c2640 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5918f0 0x55890f5b6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda419c2640 1 -- 172.21.15.105:0/786889446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890f6146a0 con 0x55890f5922e0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.545+0000 7fda421c3640 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5918f0 0x55890f5b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda419c2640 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5922e0 0x55890f5b70d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0ba030 tx=0x7fda2c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47ee8c1bdcda67a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda25ffb640 1 -- 172.21.15.105:0/786889446 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0bd710 con 0x55890f5922e0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda25ffb640 1 -- 172.21.15.105:0/786889446 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fda2c003d20 con 0x55890f5922e0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 1 -- 172.21.15.105:0/786889446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda38003670 con 0x55890f5922e0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda25ffb640 1 -- 172.21.15.105:0/786889446 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fda2c0bd530 con 0x55890f5922e0 2024-03-26T17:59:26.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 10 client.23103 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 1 -- 172.21.15.105:0/786889446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55890f615930 con 0x55890f5922e0 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda25ffb640 1 -- 172.21.15.105:0/786889446 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda38003670 con 0x55890f5922e0 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda25ffb640 1 -- 172.21.15.105:0/786889446 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fda2c0f10e0 con 0x55890f5922e0 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 10 client.23103 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 10 client.23103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 20 client.23103 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890f6162e0 0x55890f618700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 4 client.23103 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.546+0000 7fda47010740 1 -- 172.21.15.105:0/786889446 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55890f4fe210 con 0x55890f6162e0 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.547+0000 7fda421c3640 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890f6162e0 0x55890f618700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.547+0000 7fda25ffb640 10 client.23103 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.547+0000 7fda421c3640 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890f6162e0 0x55890f618700 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b2e5089cebe097 server_cookie=4fed1b19ef697c19 in_seq=0 out_seq=0 2024-03-26T17:59:26.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda25ffb640 1 -- 172.21.15.105:0/786889446 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55890f4fe210 con 0x55890f6162e0 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda25ffb640 10 client.23103 handle_command_reply: tid=0 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 client.23103 shutdown 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 -- 172.21.15.105:0/786889446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5922e0 msgr2=0x55890f5b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5922e0 0x55890f5b70d0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0ba030 tx=0x7fda2c0045c0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 -- 172.21.15.105:0/786889446 shutdown_connections 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55890f6162e0 0x55890f618700 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55890f597450 0x55890f5b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55890f5922e0 0x55890f5b70d0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda25ffb640 0 client.23103 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 --2- 172.21.15.105:0/786889446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55890f5918f0 0x55890f5b6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 -- 172.21.15.105:0/786889446 >> 172.21.15.105:0/786889446 conn(0x55890f58ce60 msgr2=0x55890f593320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 -- 172.21.15.105:0/786889446 shutdown_connections 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 1 -- 172.21.15.105:0/786889446 wait complete. 2024-03-26T17:59:26.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.552+0000 7fda47010740 20 client.23103 trim_cache size 0 max 16384 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (314 inodes in the stack)", 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "d4d618d6-0a93-4fcd-aeee-2de78943c481": { 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "d4d618d6-0a93-4fcd-aeee-2de78943c481", 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:59:26.537 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:59:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 -- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda280a2fa0 msgr2=0x7fda280a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda280a2fa0 0x7fda280a5460 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x55890f592b40 tx=0x7fda34009500 comp rx=0 tx=0).stop 2024-03-26T17:59:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 -- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 msgr2=0x7fda3c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 0x7fda3c19a2f0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fda2c000c00 tx=0x7fda2c005dc0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 -- 172.21.15.105:0/673575510 shutdown_connections 2024-03-26T17:59:26.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fda280a2fa0 0x7fda280a5460 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda3c107600 0x7fda3c1a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda3c104cd0 0x7fda3c19a2f0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 --2- 172.21.15.105:0/673575510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda3c1023a0 0x7fda3c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 -- 172.21.15.105:0/673575510 >> 172.21.15.105:0/673575510 conn(0x7fda3c0fbd50 msgr2=0x7fda3c102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 -- 172.21.15.105:0/673575510 shutdown_connections 2024-03-26T17:59:26.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:26.553+0000 7fda44427640 1 -- 172.21.15.105:0/673575510 wait complete. 2024-03-26T17:59:26.550 INFO:tasks.cephfs.filesystem:scrub status for tag:d4d618d6-0a93-4fcd-aeee-2de78943c481 - {'path': '/', 'tag': 'd4d618d6-0a93-4fcd-aeee-2de78943c481', 'options': 'recursive,force'} 2024-03-26T17:59:26.556 INFO:tasks.workunit.client.0.smithi105.stdout: 10 64213 7.09 MB/sec execute 369 sec latency 218.649 ms 2024-03-26T17:59:26.656 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:59:26.408+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003c80 [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{1=1,2=1,3=1,4=1} v12581 ap=2 RANDEPHEMERALPIN f(v338 m2024-03-26T17:59:02.952747+0000 11=11+0) n(v26 rc2024-03-26T17:59:02.952747+0000 b2861056 12=11+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@6695} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d142580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003c80:[<0x10000003bc0/COREL v50>,<0x10000003bb6/~dmtmp v48>,<0x10000000007/client8 v2078>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003c80:[<0x10000003bc0/COREL v12581>,<0x10000003bb6/~dmtmp v2595>,<0x10000000007/client8 v10792>,<0x10000000006/clients v2995>,<0x10000000005/tmp v2702>,<0x10000000003/client.0 v2592>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v2524>,<0x10000000001/sv_0 v2534>,<0x10000000000/qa v2454>,<0x1/volumes v2422>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:59:02.924870+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:59:02.952747+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v338 m2024-03-26T17:59:02.952747+0000 11=11+0)","memory_value.rstat":"n(v26 rc2024-03-26T17:59:02.952747+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T17:59:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 -- 172.21.15.105:0/1868204637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe0109780 msgr2=0x7f0fe010bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 --2- 172.21.15.105:0/1868204637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe0109780 0x7f0fe010bc20 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009ca0 tx=0x7f0fd802c180 comp rx=0 tx=0).stop 2024-03-26T17:59:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 -- 172.21.15.105:0/1868204637 shutdown_connections 2024-03-26T17:59:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 --2- 172.21.15.105:0/1868204637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe0109780 0x7f0fe010bc20 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 --2- 172.21.15.105:0/1868204637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fe00742c0 0x7f0fe0074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 --2- 172.21.15.105:0/1868204637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fe00758a0 0x7f0fe0073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 -- 172.21.15.105:0/1868204637 >> 172.21.15.105:0/1868204637 conn(0x7f0fe00fce80 msgr2=0x7f0fe00ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 -- 172.21.15.105:0/1868204637 shutdown_connections 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.927+0000 7f0fe7085640 1 -- 172.21.15.105:0/1868204637 wait complete. 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe7085640 1 Processor -- start 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe7085640 1 -- start start 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe7085640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fe00742c0 0x7f0fe0104c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe7085640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe00758a0 0x7f0fe01051b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe7085640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fe0109780 0x7f0fe01017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe7085640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010f620 con 0x7f0fe00758a0 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe7085640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010f4a0 con 0x7f0fe00742c0 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe7085640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe010f7a0 con 0x7f0fe0109780 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe00758a0 0x7f0fe01051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe4e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fe00742c0 0x7f0fe0104c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe00758a0 0x7f0fe01051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53728/0 (socket says 172.21.15.105:53728) 2024-03-26T17:59:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fdffff640 1 -- 172.21.15.105:0/2957950077 learned_addr learned my addr 172.21.15.105:0/2957950077 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.928+0000 7f0fe5622640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fe0109780 0x7f0fe01017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fdffff640 1 -- 172.21.15.105:0/2957950077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fe0109780 msgr2=0x7f0fe01017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fdffff640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fe0109780 0x7f0fe01017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fdffff640 1 -- 172.21.15.105:0/2957950077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fe00742c0 msgr2=0x7f0fe0104c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fdffff640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fe00742c0 0x7f0fe0104c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fdffff640 1 -- 172.21.15.105:0/2957950077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe01020a0 con 0x7f0fe00758a0 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fdffff640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe00758a0 0x7f0fe01051b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00e9f0 tx=0x7f0fcc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fddffb640 1 -- 172.21.15.105:0/2957950077 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc00cd80 con 0x7f0fe00758a0 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fe7085640 1 -- 172.21.15.105:0/2957950077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe01a6040 con 0x7f0fe00758a0 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fddffb640 1 -- 172.21.15.105:0/2957950077 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fcc0054d0 con 0x7f0fe00758a0 2024-03-26T17:59:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fddffb640 1 -- 172.21.15.105:0/2957950077 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc010490 con 0x7f0fe00758a0 2024-03-26T17:59:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.929+0000 7f0fe7085640 1 -- 172.21.15.105:0/2957950077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe01a6450 con 0x7f0fe00758a0 2024-03-26T17:59:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.932+0000 7f0fddffb640 1 -- 172.21.15.105:0/2957950077 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0fcc010770 con 0x7f0fe00758a0 2024-03-26T17:59:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe9c77740 1 Processor -- start 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe9c77740 1 -- start start 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fddffb640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0fd00a3040 0x7f0fd00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x5560942609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x556094265ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x5560942685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5560941c50b0 con 0x556094266130 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55609416d960 con 0x5560942605d0 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fddffb640 1 -- 172.21.15.105:0/2957950077 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0fcc0cf780 con 0x7f0fe00758a0 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5560941c1060 con 0x556094260fc0 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe4e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x5560942609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.933+0000 7f0fe5622640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0fd00a3040 0x7f0fd00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x556094265ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x556094265ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53510/0 (socket says 172.21.15.105:53510) 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fe5622640 1 --2- 172.21.15.105:0/1501520425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x5560942685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fdffff640 1 -- 172.21.15.105:0/1501520425 learned_addr learned my addr 172.21.15.105:0/1501520425 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fe4e21640 1 -- 172.21.15.105:0/1501520425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 msgr2=0x556094265ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/1501520425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x556094265ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fe4e21640 1 -- 172.21.15.105:0/1501520425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 msgr2=0x5560942685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/1501520425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x5560942685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fe4e21640 1 -- 172.21.15.105:0/1501520425 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55609417fc60 con 0x5560942605d0 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fe5622640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0fd00a3040 0x7f0fd00a5500 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8005e30 tx=0x7f0fd8003ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/1501520425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x5560942609f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400bbf0 tx=0x7f0fd40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f59a8a6f13aa445 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.934+0000 7f0fcaffd640 1 -- 172.21.15.105:0/1501520425 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4004030 con 0x5560942605d0 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.935+0000 7f0fcaffd640 1 -- 172.21.15.105:0/1501520425 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fd40035f0 con 0x5560942605d0 2024-03-26T17:59:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.935+0000 7f0fcaffd640 1 -- 172.21.15.105:0/1501520425 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd40116a0 con 0x5560942605d0 2024-03-26T17:59:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.935+0000 7f0fe9c77740 1 -- 172.21.15.105:0/1501520425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 msgr2=0x5560942609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.935+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/1501520425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x5560942609f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400bbf0 tx=0x7f0fd40107a0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 -- 172.21.15.105:0/1501520425 shutdown_connections 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/1501520425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x5560942685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/1501520425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x556094265ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/1501520425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x5560942609f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 -- 172.21.15.105:0/1501520425 >> 172.21.15.105:0/1501520425 conn(0x55609425bb40 msgr2=0x55609425bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 -- 172.21.15.105:0/1501520425 shutdown_connections 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 -- 172.21.15.105:0/1501520425 wait complete. 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 -- start start 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.936+0000 7f0fe9c77740 1 Processor -- start 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560942605d0 0x55609426a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x55609426a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x556094271930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5560942690e0 con 0x5560942605d0 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556094180db0 con 0x556094266130 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x55609426a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe4e21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560942605d0 0x55609426a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x55609426a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53516/0 (socket says 172.21.15.105:53516) 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fdffff640 1 -- 172.21.15.105:0/3325577733 learned_addr learned my addr 172.21.15.105:0/3325577733 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556094269260 con 0x556094260fc0 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe5622640 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x556094271930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe9c77740 10 client.0 _ll_register_callbacks cb 0x556094185110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe5622640 1 -- 172.21.15.105:0/3325577733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 msgr2=0x55609426a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe5622640 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x55609426a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe5622640 1 -- 172.21.15.105:0/3325577733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560942605d0 msgr2=0x55609426a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe5622640 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560942605d0 0x55609426a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe5622640 1 -- 172.21.15.105:0/3325577733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556093ed9210 con 0x556094266130 2024-03-26T17:59:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.937+0000 7f0fe5622640 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x556094271930 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0fd802c6c0 tx=0x7f0fd802c180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0595e061519ded server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.938+0000 7f0fbffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.938+0000 7f0fbffff640 1 -- 172.21.15.105:0/3325577733 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8039040 con 0x556094266130 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.938+0000 7f0fbffff640 1 -- 172.21.15.105:0/3325577733 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0fd804d920 con 0x556094266130 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.938+0000 7f0fe9c77740 1 -- 172.21.15.105:0/3325577733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fd40035f0 con 0x556094266130 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.938+0000 7f0fbffff640 1 -- 172.21.15.105:0/3325577733 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0fd804db20 con 0x556094266130 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.938+0000 7f0fe9c77740 10 client.32798 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.938+0000 7f0fe9c77740 1 -- 172.21.15.105:0/3325577733 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556093fa5620 con 0x556094266130 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.938+0000 7f0fbffff640 1 -- 172.21.15.105:0/3325577733 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fd40035f0 con 0x556094266130 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fbffff640 1 -- 172.21.15.105:0/3325577733 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f0fd803f040 con 0x556094266130 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fe9c77740 10 client.32798 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fe9c77740 10 client.32798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fe9c77740 20 client.32798 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556094275ff0 0x556094278420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fe9c77740 4 client.32798 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fe9c77740 1 -- 172.21.15.105:0/3325577733 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560941d25f0 con 0x556094275ff0 2024-03-26T17:59:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556094275ff0 0x556094278420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556094275ff0 0x556094278420 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4901ba42fd09594 server_cookie=eb9b4ff514d33fa1 in_seq=0 out_seq=0 2024-03-26T17:59:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.939+0000 7f0fbffff640 10 client.32798 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.940+0000 7f0fbffff640 1 -- 172.21.15.105:0/3325577733 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5560941d25f0 con 0x556094275ff0 2024-03-26T17:59:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.940+0000 7f0fbffff640 10 client.32798 handle_command_reply: tid=0 2024-03-26T17:59:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.940+0000 7f0fe9c77740 1 client.32798 shutdown 2024-03-26T17:59:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 -- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 msgr2=0x556094271930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x556094271930 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0fd802c6c0 tx=0x7f0fd802c180 comp rx=0 tx=0).stop 2024-03-26T17:59:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 -- 172.21.15.105:0/3325577733 shutdown_connections 2024-03-26T17:59:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x556094275ff0 0x556094278420 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x556094271930 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fbffff640 0 client.32798 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x55609426a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/3325577733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5560942605d0 0x55609426a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 -- 172.21.15.105:0/3325577733 >> 172.21.15.105:0/3325577733 conn(0x55609425bb40 msgr2=0x55609425adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 -- 172.21.15.105:0/3325577733 shutdown_connections 2024-03-26T17:59:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 1 -- 172.21.15.105:0/3325577733 wait complete. 2024-03-26T17:59:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.941+0000 7f0fe9c77740 20 client.32798 trim_cache size 0 max 16384 2024-03-26T17:59:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe9c77740 1 Processor -- start 2024-03-26T17:59:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe9c77740 1 -- start start 2024-03-26T17:59:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x556094261850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x5560942718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x556094261d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe01af2e0 con 0x556094266130 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe01b0e20 con 0x5560942605d0 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe9c77740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe01af9b0 con 0x556094260fc0 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe5622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x556094261d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe4e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x556094261850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x5560942718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe4e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x556094261850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42372/0 (socket says 172.21.15.105:42372) 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe5622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x556094261d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53752/0 (socket says 172.21.15.105:53752) 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fdffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x5560942718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53520/0 (socket says 172.21.15.105:53520) 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.964+0000 7f0fe4e21640 1 -- 172.21.15.105:0/250041494 learned_addr learned my addr 172.21.15.105:0/250041494 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fe4e21640 1 -- 172.21.15.105:0/250041494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 msgr2=0x5560942718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/250041494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x5560942718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fe4e21640 1 -- 172.21.15.105:0/250041494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 msgr2=0x556094261d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/250041494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x556094261d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.949 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fe4e21640 1 -- 172.21.15.105:0/250041494 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560942613a0 con 0x5560942605d0 2024-03-26T17:59:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/250041494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x556094261850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400bbf0 tx=0x7f0fd4013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e114a9d4a4ba5a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fca7fc640 1 -- 172.21.15.105:0/250041494 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4030070 con 0x5560942605d0 2024-03-26T17:59:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fca7fc640 1 -- 172.21.15.105:0/250041494 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fd4005040 con 0x5560942605d0 2024-03-26T17:59:26.950 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.965+0000 7f0fca7fc640 1 -- 172.21.15.105:0/250041494 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd402a3d0 con 0x5560942605d0 2024-03-26T17:59:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 -- 172.21.15.105:0/250041494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 msgr2=0x556094261850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/250041494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x556094261850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400bbf0 tx=0x7f0fd4013040 comp rx=0 tx=0).stop 2024-03-26T17:59:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 -- 172.21.15.105:0/250041494 shutdown_connections 2024-03-26T17:59:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/250041494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094266130 0x556094261d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/250041494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556094260fc0 0x5560942718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/250041494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5560942605d0 0x556094261850 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 -- 172.21.15.105:0/250041494 >> 172.21.15.105:0/250041494 conn(0x55609425bb40 msgr2=0x556094265450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 -- 172.21.15.105:0/250041494 shutdown_connections 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 -- 172.21.15.105:0/250041494 wait complete. 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.966+0000 7f0fe9c77740 1 -- start start 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe9c77740 1 Processor -- start 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560942605d0 0x55609426e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094260fc0 0x55609426ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe9c77740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x55609426fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe4e21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560942605d0 0x55609426e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe4e21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560942605d0 0x55609426e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:53528/0 (socket says 172.21.15.105:53528) 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe4e21640 1 -- 172.21.15.105:0/2319162610 learned_addr learned my addr 172.21.15.105:0/2319162610 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe01b0750 con 0x556094260fc0 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe01b0080 con 0x556094266130 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe01ad2c0 con 0x5560942605d0 2024-03-26T17:59:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fdffff640 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094260fc0 0x55609426ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.967+0000 7f0fe9c77740 10 client.0 _ll_register_callbacks cb 0x7f0fd8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe5622640 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x55609426fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe4e21640 1 -- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 msgr2=0x55609426fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x55609426fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe4e21640 1 -- 172.21.15.105:0/2319162610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094260fc0 msgr2=0x55609426ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094260fc0 0x55609426ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe4e21640 1 -- 172.21.15.105:0/2319162610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556094270440 con 0x5560942605d0 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe5622640 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x55609426fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe4e21640 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560942605d0 0x55609426e900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0fd402e040 tx=0x7f0fd400b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7114c781d21aedd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fbffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fbffff640 1 -- 172.21.15.105:0/2319162610 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4030040 con 0x5560942605d0 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fd4005040 con 0x5560942605d0 2024-03-26T17:59:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fbffff640 1 -- 172.21.15.105:0/2319162610 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0fd4005600 con 0x5560942605d0 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fbffff640 1 -- 172.21.15.105:0/2319162610 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0fd4043050 con 0x5560942605d0 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe9c77740 10 client.32794 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.968+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556094271730 con 0x5560942605d0 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fbffff640 1 -- 172.21.15.105:0/2319162610 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fd4005040 con 0x5560942605d0 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fbffff640 1 -- 172.21.15.105:0/2319162610 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f0fd403f4f0 con 0x5560942605d0 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fe9c77740 10 client.32794 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fe9c77740 10 client.32794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fe9c77740 20 client.32794 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5560942dc540 0x5560942de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fe9c77740 4 client.32794 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560941cc940 con 0x5560942dc540 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.969+0000 7f0fdffff640 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5560942dc540 0x5560942de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.970+0000 7f0fbffff640 10 client.32794 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:26.970+0000 7f0fdffff640 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5560942dc540 0x5560942de960 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1e9ac97e0e8d5a server_cookie=9e7b2b8356e14c9e in_seq=0 out_seq=0 2024-03-26T17:59:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.118+0000 7f0fbffff640 1 -- 172.21.15.105:0/2319162610 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560941cc940 con 0x5560942dc540 2024-03-26T17:59:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.118+0000 7f0fbffff640 10 client.32794 handle_command_reply: tid=0 2024-03-26T17:59:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.118+0000 7f0fe9c77740 1 client.32794 shutdown 2024-03-26T17:59:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560942605d0 msgr2=0x55609426e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560942605d0 0x55609426e900 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0fd402e040 tx=0x7f0fd400b880 comp rx=0 tx=0).stop 2024-03-26T17:59:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 shutdown_connections 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5560942dc540 0x5560942de960 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556094266130 0x55609426fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556094260fc0 0x55609426ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fbffff640 0 client.32794 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 --2- 172.21.15.105:0/2319162610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5560942605d0 0x55609426e900 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 >> 172.21.15.105:0/2319162610 conn(0x55609425bb40 msgr2=0x5560942658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 shutdown_connections 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 1 -- 172.21.15.105:0/2319162610 wait complete. 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.119+0000 7f0fe9c77740 20 client.32794 trim_cache size 0 max 16384 2024-03-26T17:59:27.103 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:59:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:59:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:59:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:59:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 -- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0fd00a3040 msgr2=0x7f0fd00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0fd00a3040 0x7f0fd00a5500 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8005e30 tx=0x7f0fd8003ad0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 -- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe00758a0 msgr2=0x7f0fe01051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe00758a0 0x7f0fe01051b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00e9f0 tx=0x7f0fcc00edf0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 -- 172.21.15.105:0/2957950077 shutdown_connections 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0fd00a3040 0x7f0fd00a5500 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fe0109780 0x7f0fe01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0fe00758a0 0x7f0fe01051b0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 --2- 172.21.15.105:0/2957950077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fe00742c0 0x7f0fe0104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 -- 172.21.15.105:0/2957950077 >> 172.21.15.105:0/2957950077 conn(0x7f0fe00fce80 msgr2=0x7f0fe0105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 -- 172.21.15.105:0/2957950077 shutdown_connections 2024-03-26T17:59:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:27.120+0000 7f0fe7085640 1 -- 172.21.15.105:0/2957950077 wait complete. 2024-03-26T17:59:27.132 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T17:59:27.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:59:27.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:59:27.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:59:27.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:59:27.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:26 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:26 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T17:59:27.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:59:27.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:59:27.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T17:59:27.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:59:27.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:26 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:27.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 64293 7.08 MB/sec execute 370 sec latency 214.832 ms 2024-03-26T17:59:28.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:27 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:59:28.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:27 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:27 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:59:28.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:27 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:27 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T17:59:28.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:27 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.157 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:27 smithi135 ceph-mon[35636]: pgmap v984: 97 pgs: 97 active+clean; 348 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 3.5 MiB/s wr, 188 op/s 2024-03-26T17:59:28.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:27 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:59:28.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:27 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:27 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:59:28.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:27 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:27 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T17:59:28.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:27 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:27 smithi144 ceph-mon[36095]: pgmap v984: 97 pgs: 97 active+clean; 348 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 3.5 MiB/s wr, 188 op/s 2024-03-26T17:59:28.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:27 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:59:28.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:27 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:27 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:59:28.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:27 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:27 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T17:59:28.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:27 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T17:59:28.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:27 smithi105 ceph-mon[32280]: pgmap v984: 97 pgs: 97 active+clean; 348 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 3.5 MiB/s wr, 188 op/s 2024-03-26T17:59:28.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 64421 7.08 MB/sec execute 371 sec latency 139.513 ms 2024-03-26T17:59:29.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 64515 7.07 MB/sec execute 372 sec latency 231.236 ms 2024-03-26T17:59:30.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:29 smithi135 ceph-mon[35636]: pgmap v985: 97 pgs: 97 active+clean; 343 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-03-26T17:59:30.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:29 smithi144 ceph-mon[36095]: pgmap v985: 97 pgs: 97 active+clean; 343 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-03-26T17:59:30.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:29 smithi105 ceph-mon[32280]: pgmap v985: 97 pgs: 97 active+clean; 343 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 3.7 MiB/s wr, 225 op/s 2024-03-26T17:59:30.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 64628 7.06 MB/sec execute 373 sec latency 227.362 ms 2024-03-26T17:59:31.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 64752 7.06 MB/sec execute 374 sec latency 176.085 ms 2024-03-26T17:59:32.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:31 smithi135 ceph-mon[35636]: pgmap v986: 97 pgs: 97 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-03-26T17:59:32.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:31 smithi144 ceph-mon[36095]: pgmap v986: 97 pgs: 97 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-03-26T17:59:32.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:31 smithi105 ceph-mon[32280]: pgmap v986: 97 pgs: 97 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-03-26T17:59:32.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 64864 7.05 MB/sec execute 375 sec latency 125.524 ms 2024-03-26T17:59:33.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 65022 7.06 MB/sec execute 376 sec latency 125.249 ms 2024-03-26T17:59:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:33 smithi135 ceph-mon[35636]: pgmap v987: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.5 MiB/s wr, 278 op/s 2024-03-26T17:59:34.184 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:33 smithi105 ceph-mon[32280]: pgmap v987: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.5 MiB/s wr, 278 op/s 2024-03-26T17:59:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:33 smithi144 ceph-mon[36095]: pgmap v987: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 3.5 MiB/s wr, 278 op/s 2024-03-26T17:59:34.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 65130 7.06 MB/sec execute 377 sec latency 168.295 ms 2024-03-26T17:59:35.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 65243 7.05 MB/sec execute 378 sec latency 158.397 ms 2024-03-26T17:59:36.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:35 smithi135 ceph-mon[35636]: pgmap v988: 97 pgs: 97 active+clean; 334 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-03-26T17:59:36.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:35 smithi144 ceph-mon[36095]: pgmap v988: 97 pgs: 97 active+clean; 334 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-03-26T17:59:36.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:35 smithi105 ceph-mon[32280]: pgmap v988: 97 pgs: 97 active+clean; 334 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-03-26T17:59:36.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 65381 7.06 MB/sec execute 379 sec latency 134.791 ms 2024-03-26T17:59:37.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:36 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:59:37.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:36 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:59:37.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:36 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:59:37.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 65514 7.06 MB/sec execute 380 sec latency 139.364 ms 2024-03-26T17:59:38.132 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:37 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:59:38.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:37 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:59:38.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:37 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T17:59:38.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:37 smithi105 ceph-mon[32280]: pgmap v989: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.9 MiB/s wr, 293 op/s 2024-03-26T17:59:38.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:37 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:59:38.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:37 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:59:38.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:37 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T17:59:38.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:37 smithi135 ceph-mon[35636]: pgmap v989: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.9 MiB/s wr, 293 op/s 2024-03-26T17:59:38.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:37 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:59:38.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:37 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:59:38.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:37 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T17:59:38.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:37 smithi144 ceph-mon[36095]: pgmap v989: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.9 MiB/s wr, 293 op/s 2024-03-26T17:59:38.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 65641 7.05 MB/sec execute 381 sec latency 120.520 ms 2024-03-26T17:59:39.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 65794 7.04 MB/sec execute 382 sec latency 121.345 ms 2024-03-26T17:59:40.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:39 smithi135 ceph-mon[35636]: pgmap v990: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.2 MiB/s wr, 301 op/s 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.248+0000 7f1ad64dc640 1 -- 172.21.15.105:0/170520637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01007f0 msgr2=0x7f1ad0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.248+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/170520637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01007f0 0x7f1ad0100c10 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009900 tx=0x7f1ac002bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.248+0000 7f1ad64dc640 1 -- 172.21.15.105:0/170520637 shutdown_connections 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.248+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/170520637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ad01023a0 0x7f1ad0108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.248+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/170520637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad01019e0 0x7f1ad0101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.248+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/170520637 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01007f0 0x7f1ad0100c10 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.248+0000 7f1ad64dc640 1 -- 172.21.15.105:0/170520637 >> 172.21.15.105:0/170520637 conn(0x7f1ad00fbd80 msgr2=0x7f1ad00fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.249+0000 7f1ad64dc640 1 -- 172.21.15.105:0/170520637 shutdown_connections 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.249+0000 7f1ad64dc640 1 -- 172.21.15.105:0/170520637 wait complete. 2024-03-26T17:59:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.249+0000 7f1ad64dc640 1 Processor -- start 2024-03-26T17:59:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.249+0000 7f1ad64dc640 1 -- start start 2024-03-26T17:59:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.249+0000 7f1ad64dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad01007f0 0x7f1ad019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.249+0000 7f1ad64dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01019e0 0x7f1ad019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.249+0000 7f1ad64dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ad01023a0 0x7f1ad01a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1ad64dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010b1e0 con 0x7f1ad01019e0 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1ad64dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010b060 con 0x7f1ad01023a0 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1ad64dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad010b360 con 0x7f1ad01007f0 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad01007f0 0x7f1ad019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1ad4a79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ad01023a0 0x7f1ad01a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01019e0 0x7f1ad019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad01007f0 0x7f1ad019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:56968/0 (socket says 172.21.15.105:56968) 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1ad4a79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ad01023a0 0x7f1ad01a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50636/0 (socket says 172.21.15.105:50636) 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acffff640 1 -- 172.21.15.105:0/2229985924 learned_addr learned my addr 172.21.15.105:0/2229985924 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acffff640 1 -- 172.21.15.105:0/2229985924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ad01023a0 msgr2=0x7f1ad01a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acffff640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ad01023a0 0x7f1ad01a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acffff640 1 -- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01019e0 msgr2=0x7f1ad019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acffff640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01019e0 0x7f1ad019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acffff640 1 -- 172.21.15.105:0/2229985924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad01a5c70 con 0x7f1ad01007f0 2024-03-26T17:59:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.250+0000 7f1acf7fe640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01019e0 0x7f1ad019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.251+0000 7f1acffff640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad01007f0 0x7f1ad019e430 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0000c00 tx=0x7f1ac0005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.251+0000 7f1acd7fa640 1 -- 172.21.15.105:0/2229985924 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0043020 con 0x7f1ad01007f0 2024-03-26T17:59:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.251+0000 7f1ad64dc640 1 -- 172.21.15.105:0/2229985924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad01a5f00 con 0x7f1ad01007f0 2024-03-26T17:59:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.251+0000 7f1acd7fa640 1 -- 172.21.15.105:0/2229985924 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac0004350 con 0x7f1ad01007f0 2024-03-26T17:59:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.251+0000 7f1acd7fa640 1 -- 172.21.15.105:0/2229985924 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0003b50 con 0x7f1ad01007f0 2024-03-26T17:59:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.251+0000 7f1ad64dc640 1 -- 172.21.15.105:0/2229985924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad01a6370 con 0x7f1ad01007f0 2024-03-26T17:59:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.253+0000 7f1acd7fa640 1 -- 172.21.15.105:0/2229985924 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1ac0046020 con 0x7f1ad01007f0 2024-03-26T17:59:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad919f740 1 Processor -- start 2024-03-26T17:59:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad919f740 1 -- start start 2024-03-26T17:59:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1acd7fa640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1ab80a2fc0 0x7f1ab80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1a5d0 0x55aed9f1a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f1fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f20130 0x55aed9f225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad919f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55aed9e7f0b0 con 0x55aed9f20130 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad919f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55aed9e27960 con 0x55aed9f1a5d0 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad919f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aed9e7b060 con 0x55aed9f1afc0 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.254+0000 7f1ad4a79640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1ab80a2fc0 0x7f1ab80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1a5d0 0x55aed9f1a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1ad4a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f20130 0x55aed9f225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1ad4a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f20130 0x55aed9f225c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42770/0 (socket says 172.21.15.105:42770) 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1ad4a79640 1 -- 172.21.15.105:0/2985698799 learned_addr learned my addr 172.21.15.105:0/2985698799 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acd7fa640 1 -- 172.21.15.105:0/2229985924 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f1ac00f16b0 con 0x7f1ad01007f0 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acf7fe640 1 --2- 172.21.15.105:0/2985698799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f1fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acffff640 1 -- 172.21.15.105:0/2985698799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 msgr2=0x55aed9f1fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acffff640 1 --2- 172.21.15.105:0/2985698799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f1fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acffff640 1 -- 172.21.15.105:0/2985698799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f20130 msgr2=0x55aed9f225c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acffff640 1 --2- 172.21.15.105:0/2985698799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f20130 0x55aed9f225c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acffff640 1 -- 172.21.15.105:0/2985698799 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aed9e39c60 con 0x55aed9f1a5d0 2024-03-26T17:59:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1ad4a79640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1ab80a2fc0 0x7f1ab80a5480 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8011a40 tx=0x7f1ac8011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1acffff640 1 --2- 172.21.15.105:0/2985698799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1a5d0 0x55aed9f1a9f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005ab0 tx=0x7f1ac002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509dd90913b1b0e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1ab67fc640 1 -- 172.21.15.105:0/2985698799 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00be5b0 con 0x55aed9f1a5d0 2024-03-26T17:59:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.255+0000 7f1ab67fc640 1 -- 172.21.15.105:0/2985698799 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac00bf350 con 0x55aed9f1a5d0 2024-03-26T17:59:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.256+0000 7f1ab67fc640 1 -- 172.21.15.105:0/2985698799 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00bcd90 con 0x55aed9f1a5d0 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 -- 172.21.15.105:0/2985698799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1a5d0 msgr2=0x55aed9f1a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2985698799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1a5d0 0x55aed9f1a9f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005ab0 tx=0x7f1ac002bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 -- 172.21.15.105:0/2985698799 shutdown_connections 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2985698799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f20130 0x55aed9f225c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2985698799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f1fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2985698799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1a5d0 0x55aed9f1a9f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 -- 172.21.15.105:0/2985698799 >> 172.21.15.105:0/2985698799 conn(0x55aed9f15b40 msgr2=0x55aed9f15f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 -- 172.21.15.105:0/2985698799 shutdown_connections 2024-03-26T17:59:40.243 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 -- 172.21.15.105:0/2985698799 wait complete. 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 -- start start 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.257+0000 7f1ad919f740 1 Processor -- start 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f35270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f3cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad919f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55aed9f33eb0 con 0x55aed9f3a7e0 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad919f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55aed9e3adb0 con 0x55aed9f20130 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad4a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f3cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1acffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f35270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1acf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad4a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f3cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42786/0 (socket says 172.21.15.105:42786) 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad4a79640 1 -- 172.21.15.105:0/1517157464 learned_addr learned my addr 172.21.15.105:0/1517157464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad919f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aed9f34030 con 0x55aed9f1afc0 2024-03-26T17:59:40.244 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1ad919f740 10 client.0 _ll_register_callbacks cb 0x55aed9e3f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1acf7fe640 1 -- 172.21.15.105:0/1517157464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 msgr2=0x55aed9f35270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1acf7fe640 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f35270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1acf7fe640 1 -- 172.21.15.105:0/1517157464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 msgr2=0x55aed9f3cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1acf7fe640 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f3cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1acf7fe640 1 -- 172.21.15.105:0/1517157464 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aed9b93210 con 0x55aed9f20130 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.258+0000 7f1acf7fe640 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f357b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55aed9f1b3c0 tx=0x7f1ac400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0b8299bd29a48c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1aaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/1517157464 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4012110 con 0x55aed9f20130 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1ad919f740 1 -- 172.21.15.105:0/1517157464 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac00bf350 con 0x55aed9f20130 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/1517157464 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f1ac4004050 con 0x55aed9f20130 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/1517157464 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1ac40122b0 con 0x55aed9f20130 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1ad919f740 10 client.32813 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1ad919f740 1 -- 172.21.15.105:0/1517157464 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55aed9c5f620 con 0x55aed9f20130 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/1517157464 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac00bf350 con 0x55aed9f20130 2024-03-26T17:59:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.259+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/1517157464 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f1ac4010430 con 0x55aed9f20130 2024-03-26T17:59:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.260+0000 7f1ad919f740 10 client.32813 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.260+0000 7f1ad919f740 10 client.32813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.260+0000 7f1ad919f740 20 client.32813 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.260+0000 7f1ad919f740 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55aed9f418d0 0x55aed9f43d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.260+0000 7f1ad919f740 4 client.32813 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.260+0000 7f1ad919f740 1 -- 172.21.15.105:0/1517157464 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aed9e8c5f0 con 0x55aed9f418d0 2024-03-26T17:59:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.260+0000 7f1acffff640 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55aed9f418d0 0x55aed9f43d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.261+0000 7f1aaf7fe640 10 client.32813 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.261+0000 7f1acffff640 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55aed9f418d0 0x55aed9f43d90 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c20dc73d24f3262c server_cookie=ce7642f23275179 in_seq=0 out_seq=0 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.263+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/1517157464 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55aed9e8c5f0 con 0x55aed9f418d0 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.263+0000 7f1aaf7fe640 10 client.32813 handle_command_reply: tid=0 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.263+0000 7f1ad919f740 1 client.32813 shutdown 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 -- 172.21.15.105:0/1517157464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 msgr2=0x55aed9f357b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f357b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55aed9f1b3c0 tx=0x7f1ac400f790 comp rx=0 tx=0).stop 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 -- 172.21.15.105:0/1517157464 shutdown_connections 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55aed9f418d0 0x55aed9f43d90 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f3cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1aaf7fe640 0 client.32813 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f357b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 --2- 172.21.15.105:0/1517157464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f35270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 -- 172.21.15.105:0/1517157464 >> 172.21.15.105:0/1517157464 conn(0x55aed9f15b40 msgr2=0x55aed9f14df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 -- 172.21.15.105:0/1517157464 shutdown_connections 2024-03-26T17:59:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 1 -- 172.21.15.105:0/1517157464 wait complete. 2024-03-26T17:59:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.264+0000 7f1ad919f740 20 client.32813 trim_cache size 0 max 16384 2024-03-26T17:59:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad919f740 1 Processor -- start 2024-03-26T17:59:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad919f740 1 -- start start 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f18da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f192e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad4a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad4a79640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f192e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42802/0 (socket says 172.21.15.105:42802) 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1acf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f18da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad4a79640 1 -- 172.21.15.105:0/647489380 learned_addr learned my addr 172.21.15.105:0/647489380 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad919f740 1 -- 172.21.15.105:0/647489380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a6a70 con 0x55aed9f3a7e0 2024-03-26T17:59:40.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad919f740 1 -- 172.21.15.105:0/647489380 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a85b0 con 0x55aed9f20130 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1ad919f740 1 -- 172.21.15.105:0/647489380 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a7140 con 0x55aed9f1afc0 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.287+0000 7f1acffff640 1 --2- 172.21.15.105:0/647489380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.288+0000 7f1ad4a79640 1 -- 172.21.15.105:0/647489380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 msgr2=0x55aed9f496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.288+0000 7f1ad4a79640 1 --2- 172.21.15.105:0/647489380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.288+0000 7f1ad4a79640 1 -- 172.21.15.105:0/647489380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 msgr2=0x55aed9f18da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.288+0000 7f1ad4a79640 1 --2- 172.21.15.105:0/647489380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f18da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.288+0000 7f1ad4a79640 1 -- 172.21.15.105:0/647489380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aed9f34320 con 0x55aed9f3a7e0 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.288+0000 7f1ad4a79640 1 --2- 172.21.15.105:0/647489380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f192e0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800f790 tx=0x7f1ac801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3376353edd6b594c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.288+0000 7f1ab5ffb640 1 -- 172.21.15.105:0/647489380 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8004b20 con 0x55aed9f3a7e0 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.288+0000 7f1ab5ffb640 1 -- 172.21.15.105:0/647489380 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac8003e20 con 0x55aed9f3a7e0 2024-03-26T17:59:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.289+0000 7f1ab5ffb640 1 -- 172.21.15.105:0/647489380 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac801de40 con 0x55aed9f3a7e0 2024-03-26T17:59:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.289+0000 7f1ad919f740 1 -- 172.21.15.105:0/647489380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 msgr2=0x55aed9f192e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.289+0000 7f1ad919f740 1 --2- 172.21.15.105:0/647489380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f192e0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800f790 tx=0x7f1ac801f000 comp rx=0 tx=0).stop 2024-03-26T17:59:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 -- 172.21.15.105:0/647489380 shutdown_connections 2024-03-26T17:59:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 --2- 172.21.15.105:0/647489380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f192e0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 --2- 172.21.15.105:0/647489380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f20130 0x55aed9f18da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 --2- 172.21.15.105:0/647489380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f1afc0 0x55aed9f496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 -- 172.21.15.105:0/647489380 >> 172.21.15.105:0/647489380 conn(0x55aed9f15b40 msgr2=0x55aed9f1f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 -- 172.21.15.105:0/647489380 shutdown_connections 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 -- 172.21.15.105:0/647489380 wait complete. 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 -- start start 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 Processor -- start 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1afc0 0x55aed9f52b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.290+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f20130 0x55aed9f53040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1ad919f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f54050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1ad919f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a7ee0 con 0x55aed9f3a7e0 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1ad919f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad01a7810 con 0x55aed9f1afc0 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1ad919f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0072000 con 0x55aed9f20130 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1ad919f740 10 client.0 _ll_register_callbacks cb 0x7f1ac4012910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1afc0 0x55aed9f52b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1afc0 0x55aed9f52b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:50680/0 (socket says 172.21.15.105:50680) 2024-03-26T17:59:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acf7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f20130 0x55aed9f53040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 -- 172.21.15.105:0/2322614515 learned_addr learned my addr 172.21.15.105:0/2322614515 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1ad4a79640 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f54050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 -- 172.21.15.105:0/2322614515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f20130 msgr2=0x55aed9f53040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f20130 0x55aed9f53040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 -- 172.21.15.105:0/2322614515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 msgr2=0x55aed9f54050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f54050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 -- 172.21.15.105:0/2322614515 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aed9f54640 con 0x55aed9f1afc0 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1ad4a79640 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f54050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1acffff640 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1afc0 0x55aed9f52b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005ab0 tx=0x7f1ac00bd160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b295072761c00c4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.291+0000 7f1aaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/2322614515 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00be5b0 con 0x55aed9f1afc0 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/2322614515 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f1ac00ef190 con 0x55aed9f1afc0 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 1 -- 172.21.15.105:0/2322614515 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac8003e20 con 0x55aed9f1afc0 2024-03-26T17:59:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/2322614515 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1ac00bf350 con 0x55aed9f1afc0 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 10 client.32819 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 1 -- 172.21.15.105:0/2322614515 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55aed9fa7c50 con 0x55aed9f1afc0 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/2322614515 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ac8003e20 con 0x55aed9f1afc0 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/2322614515 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f1ac00beeb0 con 0x55aed9f1afc0 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 10 client.32819 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 10 client.32819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 20 client.32819 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55aed9f418d0 0x55aed9faa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 4 client.32819 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.292+0000 7f1ad919f740 1 -- 172.21.15.105:0/2322614515 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aed9e86940 con 0x55aed9f418d0 2024-03-26T17:59:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.293+0000 7f1acf7fe640 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55aed9f418d0 0x55aed9faa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.293+0000 7f1acf7fe640 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55aed9f418d0 0x55aed9faa400 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7076b81942c170c server_cookie=1f2865a42db0f0aa in_seq=0 out_seq=0 2024-03-26T17:59:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.293+0000 7f1aaf7fe640 10 client.32819 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:40.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:39 smithi144 ceph-mon[36095]: pgmap v990: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.2 MiB/s wr, 301 op/s 2024-03-26T17:59:40.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:39 smithi105 ceph-mon[32280]: pgmap v990: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 3.2 MiB/s wr, 301 op/s 2024-03-26T17:59:40.413 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1aaf7fe640 1 -- 172.21.15.105:0/2322614515 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aed9e86940 con 0x55aed9f418d0 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1aaf7fe640 10 client.32819 handle_command_reply: tid=0 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 client.32819 shutdown 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 -- 172.21.15.105:0/2322614515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1afc0 msgr2=0x55aed9f52b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1afc0 0x55aed9f52b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005ab0 tx=0x7f1ac00bd160 comp rx=0 tx=0).stop 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 -- 172.21.15.105:0/2322614515 shutdown_connections 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55aed9f418d0 0x55aed9faa400 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55aed9f3a7e0 0x55aed9f54050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aed9f20130 0x55aed9f53040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 --2- 172.21.15.105:0/2322614515 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55aed9f1afc0 0x55aed9f52b00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1aaf7fe640 0 client.32819 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 -- 172.21.15.105:0/2322614515 >> 172.21.15.105:0/2322614515 conn(0x55aed9f15b40 msgr2=0x55aed9f1f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 -- 172.21.15.105:0/2322614515 shutdown_connections 2024-03-26T17:59:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 1 -- 172.21.15.105:0/2322614515 wait complete. 2024-03-26T17:59:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.430+0000 7f1ad919f740 20 client.32819 trim_cache size 0 max 16384 2024-03-26T17:59:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:59:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:59:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:59:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:59:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 -- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1ab80a2fc0 msgr2=0x7f1ab80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1ab80a2fc0 0x7f1ab80a5480 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8011a40 tx=0x7f1ac8011440 comp rx=0 tx=0).stop 2024-03-26T17:59:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 -- 172.21.15.105:0/2229985924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad01007f0 msgr2=0x7f1ad019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad01007f0 0x7f1ad019e430 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0000c00 tx=0x7f1ac0005dc0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 -- 172.21.15.105:0/2229985924 shutdown_connections 2024-03-26T17:59:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1ab80a2fc0 0x7f1ab80a5480 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1ad01023a0 0x7f1ad01a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1ad01019e0 0x7f1ad019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 --2- 172.21.15.105:0/2229985924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad01007f0 0x7f1ad019e430 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 -- 172.21.15.105:0/2229985924 >> 172.21.15.105:0/2229985924 conn(0x7f1ad00fbd80 msgr2=0x7f1ad010dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 -- 172.21.15.105:0/2229985924 shutdown_connections 2024-03-26T17:59:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:40.431+0000 7f1ad64dc640 1 -- 172.21.15.105:0/2229985924 wait complete. 2024-03-26T17:59:40.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 65935 7.03 MB/sec execute 383 sec latency 207.904 ms 2024-03-26T17:59:41.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 66088 7.03 MB/sec execute 384 sec latency 110.029 ms 2024-03-26T17:59:42.291 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:41 smithi135 ceph-mon[35636]: pgmap v991: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-03-26T17:59:42.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:41 smithi144 ceph-mon[36095]: pgmap v991: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-03-26T17:59:42.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:41 smithi105 ceph-mon[32280]: pgmap v991: 97 pgs: 97 active+clean; 333 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-03-26T17:59:42.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 66217 7.02 MB/sec execute 385 sec latency 123.370 ms 2024-03-26T17:59:43.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 66359 7.02 MB/sec execute 386 sec latency 107.315 ms 2024-03-26T17:59:44.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:43 smithi144 ceph-mon[36095]: pgmap v992: 97 pgs: 97 active+clean; 330 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-03-26T17:59:44.377 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:43 smithi135 ceph-mon[35636]: pgmap v992: 97 pgs: 97 active+clean; 330 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-03-26T17:59:44.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:43 smithi105 ceph-mon[32280]: pgmap v992: 97 pgs: 97 active+clean; 330 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-03-26T17:59:44.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 66521 7.02 MB/sec execute 387 sec latency 123.545 ms 2024-03-26T17:59:45.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 66646 7.02 MB/sec execute 388 sec latency 120.177 ms 2024-03-26T17:59:46.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:45 smithi144 ceph-mon[36095]: pgmap v993: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-03-26T17:59:46.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:45 smithi105 ceph-mon[32280]: pgmap v993: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-03-26T17:59:46.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:45 smithi135 ceph-mon[35636]: pgmap v993: 97 pgs: 97 active+clean; 329 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 2.8 MiB/s wr, 259 op/s 2024-03-26T17:59:46.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 66825 7.05 MB/sec execute 389 sec latency 119.800 ms 2024-03-26T17:59:47.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 66956 7.05 MB/sec execute 390 sec latency 133.917 ms 2024-03-26T17:59:48.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:47 smithi144 ceph-mon[36095]: pgmap v994: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2024-03-26T17:59:48.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:47 smithi105 ceph-mon[32280]: pgmap v994: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2024-03-26T17:59:48.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:47 smithi135 ceph-mon[35636]: pgmap v994: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 2.5 MiB/s wr, 248 op/s 2024-03-26T17:59:48.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 67116 7.07 MB/sec execute 391 sec latency 136.680 ms 2024-03-26T17:59:49.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 67246 7.08 MB/sec execute 392 sec latency 145.638 ms 2024-03-26T17:59:50.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:49 smithi144 ceph-mon[36095]: pgmap v995: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-03-26T17:59:50.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:49 smithi105 ceph-mon[32280]: pgmap v995: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-03-26T17:59:50.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:49 smithi135 ceph-mon[35636]: pgmap v995: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-03-26T17:59:50.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 67382 7.08 MB/sec execute 393 sec latency 125.261 ms 2024-03-26T17:59:51.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 67506 7.07 MB/sec execute 394 sec latency 122.293 ms 2024-03-26T17:59:52.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:51 smithi144 ceph-mon[36095]: pgmap v996: 97 pgs: 97 active+clean; 344 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-03-26T17:59:52.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:51 smithi105 ceph-mon[32280]: pgmap v996: 97 pgs: 97 active+clean; 344 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-03-26T17:59:52.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:51 smithi135 ceph-mon[35636]: pgmap v996: 97 pgs: 97 active+clean; 344 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-03-26T17:59:52.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 67593 7.07 MB/sec execute 395 sec latency 114.963 ms 2024-03-26T17:59:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 -- 172.21.15.105:0/1993244943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2181019d0 msgr2=0x7ff218101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 --2- 172.21.15.105:0/1993244943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2181019d0 0x7ff218101e50 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff208009900 tx=0x7ff20802bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 -- 172.21.15.105:0/1993244943 shutdown_connections 2024-03-26T17:59:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 --2- 172.21.15.105:0/1993244943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff218102390 0x7ff218108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 --2- 172.21.15.105:0/1993244943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2181019d0 0x7ff218101e50 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 --2- 172.21.15.105:0/1993244943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2181007e0 0x7ff218100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 -- 172.21.15.105:0/1993244943 >> 172.21.15.105:0/1993244943 conn(0x7ff2180fbd50 msgr2=0x7ff2180fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 -- 172.21.15.105:0/1993244943 shutdown_connections 2024-03-26T17:59:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.545+0000 7ff21f636640 1 -- 172.21.15.105:0/1993244943 wait complete. 2024-03-26T17:59:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.546+0000 7ff21f636640 1 Processor -- start 2024-03-26T17:59:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.546+0000 7ff21f636640 1 -- start start 2024-03-26T17:59:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.546+0000 7ff21f636640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2181007e0 0x7ff218197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.546+0000 7ff21f636640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2181019d0 0x7ff218197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21f636640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff218102390 0x7ff21819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21f636640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810d470 con 0x7ff218102390 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21f636640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810d2f0 con 0x7ff2181019d0 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21f636640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff21810d5f0 con 0x7ff2181007e0 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21d3d2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2181007e0 0x7ff218197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21d3d2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2181007e0 0x7ff218197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42426/0 (socket says 172.21.15.105:42426) 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21cbd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2181019d0 0x7ff218197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21d3d2640 1 -- 172.21.15.105:0/2270914776 learned_addr learned my addr 172.21.15.105:0/2270914776 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff218102390 0x7ff21819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21cbd1640 1 -- 172.21.15.105:0/2270914776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2181007e0 msgr2=0x7ff218197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2181007e0 0x7ff218197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21cbd1640 1 -- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff218102390 msgr2=0x7ff21819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff218102390 0x7ff21819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21cbd1640 1 -- 172.21.15.105:0/2270914776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21819f690 con 0x7ff2181019d0 2024-03-26T17:59:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2181007e0 0x7ff218197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2181019d0 0x7ff218197fb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff20802c170 tx=0x7ff208005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.547+0000 7ff20e7fc640 1 -- 172.21.15.105:0/2270914776 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208043020 con 0x7ff2181019d0 2024-03-26T17:59:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.548+0000 7ff20e7fc640 1 -- 172.21.15.105:0/2270914776 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2080043d0 con 0x7ff2181019d0 2024-03-26T17:59:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.548+0000 7ff21f636640 1 -- 172.21.15.105:0/2270914776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21819f920 con 0x7ff2181019d0 2024-03-26T17:59:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.548+0000 7ff20e7fc640 1 -- 172.21.15.105:0/2270914776 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208003ae0 con 0x7ff2181019d0 2024-03-26T17:59:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.548+0000 7ff21f636640 1 -- 172.21.15.105:0/2270914776 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21819fe90 con 0x7ff2181019d0 2024-03-26T17:59:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff222226740 1 Processor -- start 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff222226740 1 -- start start 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff20e7fc640 1 -- 172.21.15.105:0/2270914776 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff208046020 con 0x7ff2181019d0 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff222226740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873740970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff222226740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873743a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff222226740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x556873750f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff222226740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5568736a50b0 con 0x5568737440a0 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff222226740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55687364d960 con 0x556873740550 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff222226740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5568736a1060 con 0x556873740f40 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21d3d2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873743a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21cbd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x556873750f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21d3d2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873743a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42436/0 (socket says 172.21.15.105:42436) 2024-03-26T17:59:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21d3d2640 1 -- 172.21.15.105:0/1691563559 learned_addr learned my addr 172.21.15.105:0/1691563559 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873740970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21d3d2640 1 -- 172.21.15.105:0/1691563559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 msgr2=0x556873740970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873740970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21d3d2640 1 -- 172.21.15.105:0/1691563559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 msgr2=0x556873750f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x556873750f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.551+0000 7ff21d3d2640 1 -- 172.21.15.105:0/1691563559 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55687365fc60 con 0x556873740f40 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.552+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873740970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.552+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873743a30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff2140115c0 tx=0x7ff214011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24118babe78bca5e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.552+0000 7ff2037fe640 1 -- 172.21.15.105:0/1691563559 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214005020 con 0x556873740f40 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.552+0000 7ff2037fe640 1 -- 172.21.15.105:0/1691563559 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2140047a0 con 0x556873740f40 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.552+0000 7ff2037fe640 1 -- 172.21.15.105:0/1691563559 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21400f8e0 con 0x556873740f40 2024-03-26T17:59:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.552+0000 7ff20e7fc640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff20e7fc640 1 -- 172.21.15.105:0/2270914776 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7ff2080f1520 con 0x7ff2181019d0 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 -- 172.21.15.105:0/1691563559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 msgr2=0x556873743a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873743a30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff2140115c0 tx=0x7ff214011a80 comp rx=0 tx=0).stop 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x5568737417a0 tx=0x7ff210009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 -- 172.21.15.105:0/1691563559 shutdown_connections 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x556873750f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873743a30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 --2- 172.21.15.105:0/1691563559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873740970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 -- 172.21.15.105:0/1691563559 >> 172.21.15.105:0/1691563559 conn(0x55687373bb40 msgr2=0x55687373bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 -- 172.21.15.105:0/1691563559 shutdown_connections 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.553+0000 7ff222226740 1 -- 172.21.15.105:0/1691563559 wait complete. 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.554+0000 7ff222226740 1 -- start start 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.554+0000 7ff222226740 1 Processor -- start 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.554+0000 7ff222226740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873752bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.554+0000 7ff222226740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x5568737530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.554+0000 7ff222226740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x55687375a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.554+0000 7ff222226740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x556873751910 con 0x5568737440a0 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.554+0000 7ff222226740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556873660db0 con 0x556873740550 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.554+0000 7ff222226740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556873751a90 con 0x556873740f40 2024-03-26T17:59:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff222226740 10 client.0 _ll_register_callbacks cb 0x556873665110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21dbd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x55687375a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21dbd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x55687375a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42090/0 (socket says 172.21.15.105:42090) 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21dbd3640 1 -- 172.21.15.105:0/2671509086 learned_addr learned my addr 172.21.15.105:0/2671509086 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x5568737530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873752bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21dbd3640 1 -- 172.21.15.105:0/2671509086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 msgr2=0x5568737530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x5568737530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21dbd3640 1 -- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 msgr2=0x556873752bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873752bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21dbd3640 1 -- 172.21.15.105:0/2671509086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568733b9210 con 0x5568737440a0 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.555+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873752bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.556+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x55687375a130 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7ff210003ea0 tx=0x7ff210004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc94888304cd4d7e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.556+0000 7ff200ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.556+0000 7ff200ff9640 1 -- 172.21.15.105:0/2671509086 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000fce0 con 0x5568737440a0 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.556+0000 7ff200ff9640 1 -- 172.21.15.105:0/2671509086 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7ff2100045c0 con 0x5568737440a0 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.556+0000 7ff222226740 1 -- 172.21.15.105:0/2671509086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2140047a0 con 0x5568737440a0 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.556+0000 7ff200ff9640 1 -- 172.21.15.105:0/2671509086 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff2100047c0 con 0x5568737440a0 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.556+0000 7ff222226740 10 client.23151 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.556+0000 7ff222226740 1 -- 172.21.15.105:0/2671509086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556873485620 con 0x5568737440a0 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff200ff9640 1 -- 172.21.15.105:0/2671509086 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2140047a0 con 0x5568737440a0 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff200ff9640 1 -- 172.21.15.105:0/2671509086 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7ff210026070 con 0x5568737440a0 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff222226740 10 client.23151 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff222226740 10 client.23151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff222226740 20 client.23151 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff222226740 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55687375eae0 0x556873760f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff222226740 4 client.23151 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff222226740 1 -- 172.21.15.105:0/2671509086 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568736b25f0 con 0x55687375eae0 2024-03-26T17:59:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.557+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55687375eae0 0x556873760f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.558+0000 7ff200ff9640 10 client.23151 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.558+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55687375eae0 0x556873760f70 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfd11b421394ab9 server_cookie=2e0706ebe69376bb in_seq=0 out_seq=0 2024-03-26T17:59:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff200ff9640 1 -- 172.21.15.105:0/2671509086 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5568736b25f0 con 0x55687375eae0 2024-03-26T17:59:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff200ff9640 10 client.23151 handle_command_reply: tid=0 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 client.23151 shutdown 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 -- 172.21.15.105:0/2671509086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 msgr2=0x55687375a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x55687375a130 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7ff210003ea0 tx=0x7ff210004ab0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 -- 172.21.15.105:0/2671509086 shutdown_connections 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55687375eae0 0x556873760f70 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x55687375a130 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x5568737530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff200ff9640 0 client.23151 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 --2- 172.21.15.105:0/2671509086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873752bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 -- 172.21.15.105:0/2671509086 >> 172.21.15.105:0/2671509086 conn(0x55687373bb40 msgr2=0x55687373adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 -- 172.21.15.105:0/2671509086 shutdown_connections 2024-03-26T17:59:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.560+0000 7ff222226740 1 -- 172.21.15.105:0/2671509086 wait complete. 2024-03-26T17:59:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.561+0000 7ff222226740 20 client.23151 trim_cache size 0 max 16384 2024-03-26T17:59:53.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 67734 7.08 MB/sec execute 396 sec latency 122.533 ms 2024-03-26T17:59:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff222226740 1 Processor -- start 2024-03-26T17:59:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff222226740 1 -- start start 2024-03-26T17:59:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff222226740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556873740550 0x55687374cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff222226740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x55687375a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff222226740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5568737440a0 0x55687374d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff222226740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181a6b60 con 0x556873740550 2024-03-26T17:59:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff222226740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181a86a0 con 0x5568737440a0 2024-03-26T17:59:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff222226740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181a7230 con 0x556873740f40 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff21d3d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556873740550 0x55687374cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff21cbd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x55687375a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff21d3d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556873740550 0x55687374cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42096/0 (socket says 172.21.15.105:42096) 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff21d3d2640 1 -- 172.21.15.105:0/1308439440 learned_addr learned my addr 172.21.15.105:0/1308439440 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.585+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5568737440a0 0x55687374d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff21cbd1640 1 -- 172.21.15.105:0/1308439440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5568737440a0 msgr2=0x55687374d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5568737440a0 0x55687374d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff21cbd1640 1 -- 172.21.15.105:0/1308439440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556873740550 msgr2=0x55687374cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556873740550 0x55687374cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff21cbd1640 1 -- 172.21.15.105:0/1308439440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55687375b5b0 con 0x556873740f40 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556873740550 0x55687374cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x55687375a0f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff2080f4fd0 tx=0x7ff208005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b69d7faf6203a1a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff202ffd640 1 -- 172.21.15.105:0/1308439440 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2080bd980 con 0x556873740f40 2024-03-26T17:59:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff202ffd640 1 -- 172.21.15.105:0/1308439440 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2080be2b0 con 0x556873740f40 2024-03-26T17:59:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.586+0000 7ff202ffd640 1 -- 172.21.15.105:0/1308439440 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2080bd510 con 0x556873740f40 2024-03-26T17:59:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 -- 172.21.15.105:0/1308439440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 msgr2=0x55687375a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x55687375a0f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff2080f4fd0 tx=0x7ff208005b50 comp rx=0 tx=0).stop 2024-03-26T17:59:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 -- 172.21.15.105:0/1308439440 shutdown_connections 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5568737440a0 0x55687374d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x55687375a0f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 --2- 172.21.15.105:0/1308439440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556873740550 0x55687374cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 -- 172.21.15.105:0/1308439440 >> 172.21.15.105:0/1308439440 conn(0x55687373bb40 msgr2=0x556873750780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 -- 172.21.15.105:0/1308439440 shutdown_connections 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 -- 172.21.15.105:0/1308439440 wait complete. 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 -- start start 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.587+0000 7ff222226740 1 Processor -- start 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff222226740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873757610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff222226740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873757b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff222226740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x556873758b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff222226740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181a7fd0 con 0x5568737440a0 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff222226740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181a7900 con 0x556873740550 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff222226740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181a4b40 con 0x556873740f40 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff222226740 10 client.0 _ll_register_callbacks cb 0x7ff210009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21cbd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873757b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21d3d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873757610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21cbd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873757b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:42472/0 (socket says 172.21.15.105:42472) 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21cbd1640 1 -- 172.21.15.105:0/1741651460 learned_addr learned my addr 172.21.15.105:0/1741651460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21dbd3640 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x556873758b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21d3d2640 1 -- 172.21.15.105:0/1741651460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 msgr2=0x556873757b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873757b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21d3d2640 1 -- 172.21.15.105:0/1741651460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 msgr2=0x556873758b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x556873758b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21d3d2640 1 -- 172.21.15.105:0/1741651460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556873759150 con 0x556873740550 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.588+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873757b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff21d3d2640 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873757610 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff21401b040 tx=0x7ff214004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979d4f1c52f6dacd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff200ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff200ff9640 1 -- 172.21.15.105:0/1741651460 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21401d040 con 0x556873740550 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff200ff9640 1 -- 172.21.15.105:0/1741651460 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7ff2140046f0 con 0x556873740550 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff222226740 1 -- 172.21.15.105:0/1741651460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2080be2b0 con 0x556873740550 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff200ff9640 1 -- 172.21.15.105:0/1741651460 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff214029230 con 0x556873740550 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff222226740 10 client.32834 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff222226740 1 -- 172.21.15.105:0/1741651460 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5568737c4eb0 con 0x556873740550 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff200ff9640 1 -- 172.21.15.105:0/1741651460 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2080be2b0 con 0x556873740550 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.589+0000 7ff200ff9640 1 -- 172.21.15.105:0/1741651460 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7ff210011d20 con 0x556873740550 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff222226740 10 client.32834 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff222226740 10 client.32834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff222226740 20 client.32834 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff222226740 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55687375eae0 0x5568737c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff222226740 4 client.32834 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T17:59:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff222226740 1 -- 172.21.15.105:0/1741651460 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568736ac940 con 0x55687375eae0 2024-03-26T17:59:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55687375eae0 0x5568737c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff21cbd1640 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55687375eae0 0x5568737c7570 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11d164d54d581e4 server_cookie=32ce6c48d1155f84 in_seq=0 out_seq=0 2024-03-26T17:59:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.590+0000 7ff200ff9640 10 client.32834 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.718+0000 7ff200ff9640 1 -- 172.21.15.105:0/1741651460 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568736ac940 con 0x55687375eae0 2024-03-26T17:59:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.718+0000 7ff200ff9640 10 client.32834 handle_command_reply: tid=0 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.718+0000 7ff222226740 1 client.32834 shutdown 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 -- 172.21.15.105:0/1741651460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 msgr2=0x556873757610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873757610 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff21401b040 tx=0x7ff214004580 comp rx=0 tx=0).stop 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 -- 172.21.15.105:0/1741651460 shutdown_connections 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55687375eae0 0x5568737c7570 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5568737440a0 0x556873758b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556873740f40 0x556873757b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 --2- 172.21.15.105:0/1741651460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556873740550 0x556873757610 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 -- 172.21.15.105:0/1741651460 >> 172.21.15.105:0/1741651460 conn(0x55687373bb40 msgr2=0x556873750350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff200ff9640 0 client.32834 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 -- 172.21.15.105:0/1741651460 shutdown_connections 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 1 -- 172.21.15.105:0/1741651460 wait complete. 2024-03-26T17:59:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.719+0000 7ff222226740 20 client.32834 trim_cache size 0 max 16384 2024-03-26T17:59:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T17:59:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T17:59:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T17:59:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T17:59:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 -- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 msgr2=0x7ff2040a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x5568737417a0 tx=0x7ff210009500 comp rx=0 tx=0).stop 2024-03-26T17:59:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 -- 172.21.15.105:0/2270914776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2181019d0 msgr2=0x7ff218197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2181019d0 0x7ff218197fb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff20802c170 tx=0x7ff208005e60 comp rx=0 tx=0).stop 2024-03-26T17:59:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 -- 172.21.15.105:0/2270914776 shutdown_connections 2024-03-26T17:59:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2040a30c0 0x7ff2040a5580 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff218102390 0x7ff21819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.706 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff2181019d0 0x7ff218197fb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 --2- 172.21.15.105:0/2270914776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2181007e0 0x7ff218197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 -- 172.21.15.105:0/2270914776 >> 172.21.15.105:0/2270914776 conn(0x7ff2180fbd50 msgr2=0x7ff218104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 -- 172.21.15.105:0/2270914776 shutdown_connections 2024-03-26T17:59:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T17:59:53.720+0000 7ff21f636640 1 -- 172.21.15.105:0/2270914776 wait complete. 2024-03-26T17:59:54.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:53 smithi144 ceph-mon[36095]: pgmap v997: 97 pgs: 97 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:59:54.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:53 smithi105 ceph-mon[32280]: pgmap v997: 97 pgs: 97 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:59:54.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:53 smithi135 ceph-mon[35636]: pgmap v997: 97 pgs: 97 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 3.5 MiB/s wr, 240 op/s 2024-03-26T17:59:54.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 67825 7.07 MB/sec execute 397 sec latency 188.604 ms 2024-03-26T17:59:55.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:54 smithi144 ceph-mon[36095]: pgmap v998: 97 pgs: 97 active+clean; 350 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.9 MiB/s wr, 242 op/s 2024-03-26T17:59:55.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:54 smithi105 ceph-mon[32280]: pgmap v998: 97 pgs: 97 active+clean; 350 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.9 MiB/s wr, 242 op/s 2024-03-26T17:59:55.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:54 smithi135 ceph-mon[35636]: pgmap v998: 97 pgs: 97 active+clean; 350 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.9 MiB/s wr, 242 op/s 2024-03-26T17:59:55.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 67975 7.08 MB/sec execute 398 sec latency 134.607 ms 2024-03-26T17:59:56.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:59:56.552 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:59:56.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 68102 7.08 MB/sec execute 399 sec latency 124.564 ms 2024-03-26T17:59:56.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.707+0000 7fb675fc4640 1 -- 172.21.15.105:0/2449792848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb670107600 msgr2=0x7fb6701099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.707+0000 7fb675fc4640 1 --2- 172.21.15.105:0/2449792848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb670107600 0x7fb6701099f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fb668009ca0 tx=0x7fb66802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.707+0000 7fb675fc4640 1 -- 172.21.15.105:0/2449792848 shutdown_connections 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.707+0000 7fb675fc4640 1 --2- 172.21.15.105:0/2449792848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb670107600 0x7fb6701099f0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.707+0000 7fb675fc4640 1 --2- 172.21.15.105:0/2449792848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb670104cd0 0x7fb6701070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.707+0000 7fb675fc4640 1 --2- 172.21.15.105:0/2449792848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6701023a0 0x7fb670104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.707+0000 7fb675fc4640 1 -- 172.21.15.105:0/2449792848 >> 172.21.15.105:0/2449792848 conn(0x7fb6700fbd50 msgr2=0x7fb6700fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.707+0000 7fb675fc4640 1 -- 172.21.15.105:0/2449792848 shutdown_connections 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.708+0000 7fb675fc4640 1 -- 172.21.15.105:0/2449792848 wait complete. 2024-03-26T17:59:56.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.708+0000 7fb675fc4640 1 Processor -- start 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.708+0000 7fb675fc4640 1 -- start start 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb675fc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6701023a0 0x7fb670199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb675fc4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb670104cd0 0x7fb67019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb675fc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb670107600 0x7fb6701a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb675fc4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb67010d3f0 con 0x7fb6701023a0 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb675fc4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb67010d270 con 0x7fb670104cd0 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb675fc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb67010d570 con 0x7fb670107600 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6701023a0 0x7fb670199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6701023a0 0x7fb670199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55660/0 (socket says 172.21.15.105:55660) 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66f7fe640 1 -- 172.21.15.105:0/1911364028 learned_addr learned my addr 172.21.15.105:0/1911364028 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66ffff640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb670107600 0x7fb6701a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66effd640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb670104cd0 0x7fb67019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66f7fe640 1 -- 172.21.15.105:0/1911364028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb670107600 msgr2=0x7fb6701a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb670107600 0x7fb6701a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66f7fe640 1 -- 172.21.15.105:0/1911364028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb670104cd0 msgr2=0x7fb67019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb670104cd0 0x7fb67019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.709+0000 7fb66f7fe640 1 -- 172.21.15.105:0/1911364028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6701a1910 con 0x7fb6701023a0 2024-03-26T17:59:56.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.710+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6701023a0 0x7fb670199d00 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb66000e9f0 tx=0x7fb66000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.710+0000 7fb66cff9640 1 -- 172.21.15.105:0/1911364028 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66000cd80 con 0x7fb6701023a0 2024-03-26T17:59:56.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.710+0000 7fb66cff9640 1 -- 172.21.15.105:0/1911364028 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6600108e0 con 0x7fb6701023a0 2024-03-26T17:59:56.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.710+0000 7fb675fc4640 1 -- 172.21.15.105:0/1911364028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6701a1c00 con 0x7fb6701023a0 2024-03-26T17:59:56.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.710+0000 7fb66cff9640 1 -- 172.21.15.105:0/1911364028 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660018920 con 0x7fb6701023a0 2024-03-26T17:59:56.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.710+0000 7fb675fc4640 1 -- 172.21.15.105:0/1911364028 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6701a21a0 con 0x7fb6701023a0 2024-03-26T17:59:56.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.713+0000 7fb66cff9640 1 -- 172.21.15.105:0/1911364028 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb660010a90 con 0x7fb6701023a0 2024-03-26T17:59:56.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.714+0000 7fb678bae740 1 Processor -- start 2024-03-26T17:59:56.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.714+0000 7fb66cff9640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6580a2fa0 0x7fb6580a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.714+0000 7fb66ffff640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6580a2fa0 0x7fb6580a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.714+0000 7fb66cff9640 1 -- 172.21.15.105:0/1911364028 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fb6600cf910 con 0x7fb6701023a0 2024-03-26T17:59:56.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.714+0000 7fb678bae740 1 -- start start 2024-03-26T17:59:56.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb66ffff640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6580a2fa0 0x7fb6580a5460 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb668006010 tx=0x7fb668032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb678bae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa68f0 0x55777caa6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb678bae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa72e0 0x55777caabde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb678bae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa68f0 0x55777caa6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa68f0 0x55777caa6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34702/0 (socket says 172.21.15.105:34702) 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb66effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa72e0 0x55777caabde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb66effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa72e0 0x55777caabde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36500/0 (socket says 172.21.15.105:36500) 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55670/0 (socket says 172.21.15.105:55670) 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb678bae740 1 -- 172.21.15.105:0/3207568349 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55777ca0b980 con 0x55777caac450 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.715+0000 7fb66f7fe640 1 -- 172.21.15.105:0/3207568349 learned_addr learned my addr 172.21.15.105:0/3207568349 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb678bae740 1 -- 172.21.15.105:0/3207568349 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55777c9b3be0 con 0x55777caa72e0 2024-03-26T17:59:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb678bae740 1 -- 172.21.15.105:0/3207568349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55777ca07930 con 0x55777caa68f0 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb66f7fe640 1 -- 172.21.15.105:0/3207568349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa72e0 msgr2=0x55777caabde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/3207568349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa72e0 0x55777caabde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb66f7fe640 1 -- 172.21.15.105:0/3207568349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 msgr2=0x55777caae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/3207568349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb66f7fe640 1 -- 172.21.15.105:0/3207568349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777c7e97d0 con 0x55777caa68f0 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb66ffff640 1 --2- 172.21.15.105:0/3207568349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.716+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/3207568349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa68f0 0x55777caa6d10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb6600988e0 tx=0x7fb66009a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7bfde284c28c179 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.717+0000 7fb655ffb640 1 -- 172.21.15.105:0/3207568349 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6600a00f0 con 0x55777caa68f0 2024-03-26T17:59:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.717+0000 7fb655ffb640 1 -- 172.21.15.105:0/3207568349 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6600a49b0 con 0x55777caa68f0 2024-03-26T17:59:56.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.717+0000 7fb655ffb640 1 -- 172.21.15.105:0/3207568349 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66009b3f0 con 0x55777caa68f0 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.718+0000 7fb678bae740 1 -- 172.21.15.105:0/3207568349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa68f0 msgr2=0x55777caa6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.718+0000 7fb678bae740 1 --2- 172.21.15.105:0/3207568349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa68f0 0x55777caa6d10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb6600988e0 tx=0x7fb66009a1d0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.718+0000 7fb678bae740 1 -- 172.21.15.105:0/3207568349 shutdown_connections 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.718+0000 7fb678bae740 1 --2- 172.21.15.105:0/3207568349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.718+0000 7fb678bae740 1 --2- 172.21.15.105:0/3207568349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa72e0 0x55777caabde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.718+0000 7fb678bae740 1 --2- 172.21.15.105:0/3207568349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa68f0 0x55777caa6d10 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.718+0000 7fb678bae740 1 -- 172.21.15.105:0/3207568349 >> 172.21.15.105:0/3207568349 conn(0x55777caa1e60 msgr2=0x55777caa2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.719+0000 7fb678bae740 1 -- 172.21.15.105:0/3207568349 shutdown_connections 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.719+0000 7fb678bae740 1 -- 172.21.15.105:0/3207568349 wait complete. 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.719+0000 7fb678bae740 1 -- start start 2024-03-26T17:59:56.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb678bae740 1 Processor -- start 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb678bae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777cab09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb678bae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caa72e0 0x55777cab0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb678bae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caac450 0x55777cab7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb678bae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55777caaf450 con 0x55777caa72e0 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb678bae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55777c9c7160 con 0x55777caa68f0 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb66effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caa72e0 0x55777cab0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777cab09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb66effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caa72e0 0x55777cab0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55678/0 (socket says 172.21.15.105:55678) 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb66effd640 1 -- 172.21.15.105:0/3739782482 learned_addr learned my addr 172.21.15.105:0/3739782482 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.720+0000 7fb66ffff640 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caac450 0x55777cab7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55777caaf5d0 con 0x55777caac450 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb678bae740 10 client.0 _ll_register_callbacks cb 0x55777ca03720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:56.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb66effd640 1 -- 172.21.15.105:0/3739782482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caac450 msgr2=0x55777cab7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb66effd640 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caac450 0x55777cab7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb66effd640 1 -- 172.21.15.105:0/3739782482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 msgr2=0x55777cab09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb66effd640 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777cab09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb66effd640 1 -- 172.21.15.105:0/3739782482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777c71f3f0 con 0x55777caa72e0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb66ffff640 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caac450 0x55777cab7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6600a49b0 con 0x55777caa72e0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb66effd640 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caa72e0 0x55777cab0f20 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb664013710 tx=0x7fb664013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e3bb397aa65c8e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb64effd640 10 client.23166 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.721+0000 7fb64effd640 1 -- 172.21.15.105:0/3739782482 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664011d70 con 0x55777caa72e0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.722+0000 7fb64effd640 1 -- 172.21.15.105:0/3739782482 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fb664004030 con 0x55777caa72e0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.722+0000 7fb64effd640 1 -- 172.21.15.105:0/3739782482 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb664004290 con 0x55777caa72e0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.722+0000 7fb678bae740 10 client.23166 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.722+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55777c6df0f0 con 0x55777caa72e0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.722+0000 7fb64effd640 1 -- 172.21.15.105:0/3739782482 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fb66401a070 con 0x55777caa72e0 2024-03-26T17:59:56.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.722+0000 7fb678bae740 10 client.23166 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.722+0000 7fb678bae740 10 client.23166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.722+0000 7fb678bae740 20 client.23166 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.723+0000 7fb678bae740 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55777cabc680 0x55777cabeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.723+0000 7fb678bae740 4 client.23166 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.723+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55777ca18ec0 con 0x55777cabc680 2024-03-26T17:59:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.723+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55777cabc680 0x55777cabeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.723+0000 7fb64effd640 10 client.23166 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:56.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.723+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55777cabc680 0x55777cabeb00 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea84497f7bdd400c server_cookie=11053bab990c552f in_seq=0 out_seq=0 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.726+0000 7fb64effd640 1 -- 172.21.15.105:0/3739782482 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55777ca18ec0 con 0x55777cabc680 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.726+0000 7fb64effd640 10 client.23166 handle_command_reply: tid=0 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.726+0000 7fb678bae740 1 client.23166 shutdown 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.726+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caa72e0 msgr2=0x55777cab0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.726+0000 7fb678bae740 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caa72e0 0x55777cab0f20 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb664013710 tx=0x7fb664013bd0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.726+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 shutdown_connections 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.726+0000 7fb678bae740 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55777cabc680 0x55777cabeb00 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.727+0000 7fb678bae740 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caac450 0x55777cab7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.727+0000 7fb64effd640 0 client.23166 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.727+0000 7fb678bae740 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caa72e0 0x55777cab0f20 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.727+0000 7fb678bae740 1 --2- 172.21.15.105:0/3739782482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777cab09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.727+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 >> 172.21.15.105:0/3739782482 conn(0x55777caa1e60 msgr2=0x55777caa1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.727+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 shutdown_connections 2024-03-26T17:59:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.727+0000 7fb678bae740 1 -- 172.21.15.105:0/3739782482 wait complete. 2024-03-26T17:59:56.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.727+0000 7fb678bae740 20 client.23166 trim_cache size 0 max 16384 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.755+0000 7fb678bae740 1 Processor -- start 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.755+0000 7fb678bae740 1 -- start start 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb678bae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777caa2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb678bae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb678bae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caa8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb678bae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701a6bb0 con 0x55777caac450 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb678bae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701a86f0 con 0x55777caa68f0 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb678bae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701a7280 con 0x55777caa72e0 2024-03-26T17:59:56.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777caa2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777caa2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36508/0 (socket says 172.21.15.105:36508) 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66f7fe640 1 -- 172.21.15.105:0/2493799536 learned_addr learned my addr 172.21.15.105:0/2493799536 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66effd640 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66ffff640 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caa8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66f7fe640 1 -- 172.21.15.105:0/2493799536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 msgr2=0x55777cab7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66f7fe640 1 -- 172.21.15.105:0/2493799536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 msgr2=0x55777caa8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caa8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66f7fe640 1 -- 172.21.15.105:0/2493799536 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777caa2da0 con 0x55777caa68f0 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.756+0000 7fb66effd640 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.757+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777caa2240 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c6a0 tx=0x7fb66009bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166c3107d85d9cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.757+0000 7fb6557fa640 1 -- 172.21.15.105:0/2493799536 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6600b4070 con 0x55777caa68f0 2024-03-26T17:59:56.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.757+0000 7fb6557fa640 1 -- 172.21.15.105:0/2493799536 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6600b0160 con 0x55777caa68f0 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.757+0000 7fb6557fa640 1 -- 172.21.15.105:0/2493799536 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660097ec0 con 0x55777caa68f0 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 -- 172.21.15.105:0/2493799536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 msgr2=0x55777caa2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777caa2240 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c6a0 tx=0x7fb66009bea0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 -- 172.21.15.105:0/2493799536 shutdown_connections 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777caa8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 --2- 172.21.15.105:0/2493799536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777caa2240 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 -- 172.21.15.105:0/2493799536 >> 172.21.15.105:0/2493799536 conn(0x55777caa1e60 msgr2=0x55777cab09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 -- 172.21.15.105:0/2493799536 shutdown_connections 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 -- 172.21.15.105:0/2493799536 wait complete. 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 -- start start 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.758+0000 7fb678bae740 1 Processor -- start 2024-03-26T17:59:56.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb678bae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777cab5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb678bae740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb678bae740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777cab6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb678bae740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701a8020 con 0x55777caac450 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb678bae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701a7950 con 0x55777caa68f0 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb678bae740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701a4b90 con 0x55777caa72e0 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777cab6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34726/0 (socket says 172.21.15.105:34726) 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66effd640 1 -- 172.21.15.105:0/1716739687 learned_addr learned my addr 172.21.15.105:0/1716739687 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb678bae740 10 client.0 _ll_register_callbacks cb 0x7fb664009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777cab5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66ffff640 1 -- 172.21.15.105:0/1716739687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 msgr2=0x55777cab57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66ffff640 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66ffff640 1 -- 172.21.15.105:0/1716739687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 msgr2=0x55777cab5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66ffff640 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777cab5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66ffff640 1 -- 172.21.15.105:0/1716739687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777cab6d30 con 0x55777caac450 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66effd640 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb66ffff640 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777cab6740 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb668037040 tx=0x7fb6680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dfa5e39c89b23f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.759+0000 7fb64effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb64effd640 1 -- 172.21.15.105:0/1716739687 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668051070 con 0x55777caac450 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb64effd640 1 -- 172.21.15.105:0/1716739687 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fb668004590 con 0x55777caac450 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 1 -- 172.21.15.105:0/1716739687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6600b0160 con 0x55777caac450 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb64effd640 1 -- 172.21.15.105:0/1716739687 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb6680047b0 con 0x55777caac450 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 10 client.23172 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 1 -- 172.21.15.105:0/1716739687 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55777cab8090 con 0x55777caac450 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb64effd640 1 -- 172.21.15.105:0/1716739687 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6600b0160 con 0x55777caac450 2024-03-26T17:59:56.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb64effd640 1 -- 172.21.15.105:0/1716739687 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fb668034730 con 0x55777caac450 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 10 client.23172 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 10 client.23172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 20 client.23172 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55777cabc680 0x55777cb24fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 4 client.23172 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.760+0000 7fb678bae740 1 -- 172.21.15.105:0/1716739687 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55777ca13210 con 0x55777cabc680 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.761+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55777cabc680 0x55777cb24fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.761+0000 7fb64effd640 10 client.23172 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.761+0000 7fb66f7fe640 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55777cabc680 0x55777cb24fd0 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6369bbb1db9c72 server_cookie=66577e015ac6a082 in_seq=0 out_seq=0 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.762+0000 7fb64effd640 1 -- 172.21.15.105:0/1716739687 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55777ca13210 con 0x55777cabc680 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.762+0000 7fb64effd640 10 client.23172 handle_command_reply: tid=0 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 client.23172 shutdown 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 -- 172.21.15.105:0/1716739687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 msgr2=0x55777cab6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777cab6740 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb668037040 tx=0x7fb6680095c0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 -- 172.21.15.105:0/1716739687 shutdown_connections 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55777cabc680 0x55777cb24fd0 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55777caac450 0x55777cab6740 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb64effd640 0 client.23172 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55777caa72e0 0x55777cab57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 --2- 172.21.15.105:0/1716739687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55777caa68f0 0x55777cab5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 -- 172.21.15.105:0/1716739687 >> 172.21.15.105:0/1716739687 conn(0x55777caa1e60 msgr2=0x55777caa8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 -- 172.21.15.105:0/1716739687 shutdown_connections 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 1 -- 172.21.15.105:0/1716739687 wait complete. 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.763+0000 7fb678bae740 20 client.23172 trim_cache size 0 max 16384 2024-03-26T17:59:56.754 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:59:56.755 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T17:59:56.755 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T17:59:56.755 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:59:56.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 -- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6580a2fa0 msgr2=0x7fb6580a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6580a2fa0 0x7fb6580a5460 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb668006010 tx=0x7fb668032340 comp rx=0 tx=0).stop 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 -- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6701023a0 msgr2=0x7fb670199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6701023a0 0x7fb670199d00 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb66000e9f0 tx=0x7fb66000eeb0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 -- 172.21.15.105:0/1911364028 shutdown_connections 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb6580a2fa0 0x7fb6580a5460 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb670107600 0x7fb6701a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb670104cd0 0x7fb67019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 --2- 172.21.15.105:0/1911364028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb6701023a0 0x7fb670199d00 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 -- 172.21.15.105:0/1911364028 >> 172.21.15.105:0/1911364028 conn(0x7fb6700fbd50 msgr2=0x7fb670101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 -- 172.21.15.105:0/1911364028 shutdown_connections 2024-03-26T17:59:56.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.764+0000 7fb675fc4640 1 -- 172.21.15.105:0/1911364028 wait complete. 2024-03-26T17:59:56.762 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T17:59:56.762 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T17:59:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 -- 172.21.15.105:0/394460010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960102200 msgr2=0x7fa960108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 --2- 172.21.15.105:0/394460010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960102200 0x7fa960108a90 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fa958009ca0 tx=0x7fa95802c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 -- 172.21.15.105:0/394460010 shutdown_connections 2024-03-26T17:59:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 --2- 172.21.15.105:0/394460010 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960102200 0x7fa960108a90 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 --2- 172.21.15.105:0/394460010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960101840 0x7fa960101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 --2- 172.21.15.105:0/394460010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa960100650 0x7fa960100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 -- 172.21.15.105:0/394460010 >> 172.21.15.105:0/394460010 conn(0x7fa9600fbba0 msgr2=0x7fa9600fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 -- 172.21.15.105:0/394460010 shutdown_connections 2024-03-26T17:59:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 -- 172.21.15.105:0/394460010 wait complete. 2024-03-26T17:59:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.957+0000 7fa9653ff640 1 Processor -- start 2024-03-26T17:59:56.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa9653ff640 1 -- start start 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa9653ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960100650 0x7fa9600695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa9653ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa960101840 0x7fa96019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa9653ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960102200 0x7fa96019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa9653ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010c180 con 0x7fa960100650 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa9653ff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010c000 con 0x7fa960102200 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa9653ff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010c300 con 0x7fa960101840 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960100650 0x7fa9600695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960100650 0x7fa9600695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55704/0 (socket says 172.21.15.105:55704) 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95effd640 1 -- 172.21.15.105:0/2943408781 learned_addr learned my addr 172.21.15.105:0/2943408781 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:56.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95e7fc640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa960101840 0x7fa96019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95f7fe640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960102200 0x7fa96019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95effd640 1 -- 172.21.15.105:0/2943408781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa960101840 msgr2=0x7fa96019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95effd640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa960101840 0x7fa96019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95effd640 1 -- 172.21.15.105:0/2943408781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960102200 msgr2=0x7fa96019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95effd640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960102200 0x7fa96019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.958+0000 7fa95effd640 1 -- 172.21.15.105:0/2943408781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96019f570 con 0x7fa960100650 2024-03-26T17:59:56.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.959+0000 7fa95effd640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960100650 0x7fa9600695f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e9c0 tx=0x7fa95000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.961+0000 7fa947fff640 1 -- 172.21.15.105:0/2943408781 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95000cd80 con 0x7fa960100650 2024-03-26T17:59:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.961+0000 7fa947fff640 1 -- 172.21.15.105:0/2943408781 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9500035f0 con 0x7fa960100650 2024-03-26T17:59:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.961+0000 7fa947fff640 1 -- 172.21.15.105:0/2943408781 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950010690 con 0x7fa960100650 2024-03-26T17:59:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.961+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96019f860 con 0x7fa960100650 2024-03-26T17:59:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.961+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96019fd60 con 0x7fa960100650 2024-03-26T17:59:56.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.963+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960107a10 con 0x7fa960100650 2024-03-26T17:59:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.964+0000 7fa947fff640 1 -- 172.21.15.105:0/2943408781 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa9500037a0 con 0x7fa960100650 2024-03-26T17:59:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.965+0000 7fa947fff640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9480a2fa0 0x7fa9480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.965+0000 7fa947fff640 1 -- 172.21.15.105:0/2943408781 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fa9500cf920 con 0x7fa960100650 2024-03-26T17:59:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.965+0000 7fa95e7fc640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9480a2fa0 0x7fa9480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:56.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.965+0000 7fa95e7fc640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9480a2fa0 0x7fa9480a5460 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa960069360 tx=0x7fa954005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:56.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:56.971+0000 7fa947fff640 1 -- 172.21.15.105:0/2943408781 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa95009c6b0 con 0x7fa960100650 2024-03-26T17:59:57.132 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:56 smithi105 ceph-mon[32280]: pgmap v999: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 3.7 MiB/s wr, 212 op/s 2024-03-26T17:59:57.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.313+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa96002d240 con 0x7fa960100650 2024-03-26T17:59:57.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.313+0000 7fa947fff640 1 -- 172.21.15.105:0/2943408781 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11941 (secure 0 0 0) 0x7fa95009bfb0 con 0x7fa960100650 2024-03-26T17:59:57.297 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T17:59:57.297 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T17:58:34.693267+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_32441":{"gid":32441,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1622534523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1622534523},{"type":"v1","addr":"172.21.15.135:6835","nonce":1622534523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T17:59:57.297 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 41 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9480a2fa0 msgr2=0x7fa9480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9480a2fa0 0x7fa9480a5460 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa960069360 tx=0x7fa954005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960100650 msgr2=0x7fa9600695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960100650 0x7fa9600695f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e9c0 tx=0x7fa95000ee80 comp rx=0 tx=0).stop 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 shutdown_connections 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9480a2fa0 0x7fa9480a5460 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa960102200 0x7fa96019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa960101840 0x7fa96019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 --2- 172.21.15.105:0/2943408781 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa960100650 0x7fa9600695f0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 >> 172.21.15.105:0/2943408781 conn(0x7fa9600fbba0 msgr2=0x7fa960104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 shutdown_connections 2024-03-26T17:59:57.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.316+0000 7fa9653ff640 1 -- 172.21.15.105:0/2943408781 wait complete. 2024-03-26T17:59:57.314 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T17:59:57.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:56 smithi144 ceph-mon[36095]: pgmap v999: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 3.7 MiB/s wr, 212 op/s 2024-03-26T17:59:57.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:56 smithi135 ceph-mon[35636]: pgmap v999: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 3.7 MiB/s wr, 212 op/s 2024-03-26T17:59:57.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 -- 172.21.15.105:0/807177443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc102370 msgr2=0x7f0bbc108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 --2- 172.21.15.105:0/807177443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc102370 0x7f0bbc108c00 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009ca0 tx=0x7f0bb402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 -- 172.21.15.105:0/807177443 shutdown_connections 2024-03-26T17:59:57.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 --2- 172.21.15.105:0/807177443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc102370 0x7f0bbc108c00 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 --2- 172.21.15.105:0/807177443 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 --2- 172.21.15.105:0/807177443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bbc1007c0 0x7f0bbc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 -- 172.21.15.105:0/807177443 >> 172.21.15.105:0/807177443 conn(0x7f0bbc0fbd50 msgr2=0x7f0bbc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 -- 172.21.15.105:0/807177443 shutdown_connections 2024-03-26T17:59:57.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.471+0000 7f0bc4034640 1 -- 172.21.15.105:0/807177443 wait complete. 2024-03-26T17:59:57.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.472+0000 7f0bc4034640 1 Processor -- start 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.472+0000 7f0bc4034640 1 -- start start 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.472+0000 7f0bc4034640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bbc1007c0 0x7f0bbc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.472+0000 7f0bc4034640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.472+0000 7f0bc4034640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bbc102370 0x7f0bbc19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.472+0000 7f0bc4034640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10c2f0 con 0x7f0bbc1019b0 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.472+0000 7f0bc4034640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10c170 con 0x7f0bbc1007c0 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.472+0000 7f0bc4034640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10c470 con 0x7f0bbc102370 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55734/0 (socket says 172.21.15.105:55734) 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 -- 172.21.15.105:0/661647355 learned_addr learned my addr 172.21.15.105:0/661647355 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bbc1007c0 0x7f0bbc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 -- 172.21.15.105:0/661647355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bbc102370 msgr2=0x7f0bbc19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bbc102370 0x7f0bbc19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bbc102370 0x7f0bbc19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 -- 172.21.15.105:0/661647355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bbc1007c0 msgr2=0x7f0bbc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bbc1007c0 0x7f0bbc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 -- 172.21.15.105:0/661647355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc19f6e0 con 0x7f0bbc1019b0 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bbc1007c0 0x7f0bbc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bbc102370 0x7f0bbc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc198050 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00ea60 tx=0x7f0bac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.473+0000 7f0bb2ffd640 1 -- 172.21.15.105:0/661647355 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac00cde0 con 0x7f0bbc1019b0 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.474+0000 7f0bb2ffd640 1 -- 172.21.15.105:0/661647355 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bac0035f0 con 0x7f0bbc1019b0 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.474+0000 7f0bb2ffd640 1 -- 172.21.15.105:0/661647355 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac010690 con 0x7f0bbc1019b0 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.474+0000 7f0bc4034640 1 -- 172.21.15.105:0/661647355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc19f9d0 con 0x7f0bbc1019b0 2024-03-26T17:59:57.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.474+0000 7f0bc4034640 1 -- 172.21.15.105:0/661647355 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc19fed0 con 0x7f0bbc1019b0 2024-03-26T17:59:57.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.476+0000 7f0bb2ffd640 1 -- 172.21.15.105:0/661647355 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0bac0037a0 con 0x7f0bbc1019b0 2024-03-26T17:59:57.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bc6c1b740 1 Processor -- start 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bc6c1b740 1 -- start start 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 0x562a70135260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a7013ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562a70095970 con 0x562a70135830 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562a7003dd70 con 0x562a70134e40 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a70091920 con 0x562a7013a9a0 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.477+0000 7f0bb2ffd640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ba80a3070 0x7f0ba80a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a7013ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc1dd0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a7013ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34772/0 (socket says 172.21.15.105:34772) 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 -- 172.21.15.105:0/4055527592 learned_addr learned my addr 172.21.15.105:0/4055527592 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc25d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 0x562a70135260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ba80a3070 0x7f0ba80a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bb2ffd640 1 -- 172.21.15.105:0/661647355 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0bac0cf670 con 0x7f0bbc1019b0 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 -- 172.21.15.105:0/4055527592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 msgr2=0x562a70135260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/4055527592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 0x562a70135260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 -- 172.21.15.105:0/4055527592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 msgr2=0x562a7013a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/4055527592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 -- 172.21.15.105:0/4055527592 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a6fda93f0 con 0x562a7013a9a0 2024-03-26T17:59:57.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/4055527592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ba80a3070 0x7f0ba80a5530 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x562a70136090 tx=0x7f0bb4032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.478+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/4055527592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a7013ce30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c760 tx=0x7f0bac00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61525515c35844c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.479+0000 7f0ba7fff640 1 -- 172.21.15.105:0/4055527592 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac09b6c0 con 0x562a7013a9a0 2024-03-26T17:59:57.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.479+0000 7f0ba7fff640 1 -- 172.21.15.105:0/4055527592 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bac003b30 con 0x562a7013a9a0 2024-03-26T17:59:57.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.479+0000 7f0ba7fff640 1 -- 172.21.15.105:0/4055527592 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac09b1b0 con 0x562a7013a9a0 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4055527592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 msgr2=0x562a7013ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4055527592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a7013ce30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c760 tx=0x7f0bac00ef90 comp rx=0 tx=0).stop 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4055527592 shutdown_connections 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4055527592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a7013ce30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4055527592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4055527592 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 0x562a70135260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4055527592 >> 172.21.15.105:0/4055527592 conn(0x562a7012bfa0 msgr2=0x562a7012c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4055527592 shutdown_connections 2024-03-26T17:59:57.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4055527592 wait complete. 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 -- start start 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.480+0000 7f0bc6c1b740 1 Processor -- start 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 0x562a7013f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a701468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562a7013d890 con 0x562a70135830 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562a70051310 con 0x562a70134e40 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562a7013da10 con 0x562a7013a9a0 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc6c1b740 10 client.0 _ll_register_callbacks cb 0x562a7008cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc15cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc1dd0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 0x562a7013f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc15cf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55760/0 (socket says 172.21.15.105:55760) 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc15cf640 1 -- 172.21.15.105:0/1625094062 learned_addr learned my addr 172.21.15.105:0/1625094062 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.481+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a701468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc15cf640 1 -- 172.21.15.105:0/1625094062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 msgr2=0x562a701468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a701468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc15cf640 1 -- 172.21.15.105:0/1625094062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 msgr2=0x562a7013f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 0x562a7013f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc15cf640 1 -- 172.21.15.105:0/1625094062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a6fe737d0 con 0x562a70135830 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013f860 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0cd180 tx=0x7f0bac098080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2537bbe0c740d9de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0ba57fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0ba57fa640 1 -- 172.21.15.105:0/1625094062 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac09b6c0 con 0x562a70135830 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0ba57fa640 1 -- 172.21.15.105:0/1625094062 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0bac099e00 con 0x562a70135830 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/1625094062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a700a2eb0 con 0x562a70135830 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0ba57fa640 1 -- 172.21.15.105:0/1625094062 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0bac09bd60 con 0x562a70135830 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc6c1b740 10 client.23190 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:57.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.482+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/1625094062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562a70149c10 con 0x562a70135830 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0ba57fa640 1 -- 172.21.15.105:0/1625094062 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bac09c460 con 0x562a70135830 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0ba57fa640 1 -- 172.21.15.105:0/1625094062 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f0bac0baad0 con 0x562a70135830 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0bc6c1b740 10 client.23190 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0bc6c1b740 10 client.23190 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0bc6c1b740 20 client.23190 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562a7014af60 0x562a7014d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0bc6c1b740 4 client.23190 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/1625094062 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a700a2eb0 con 0x562a7014af60 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.483+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562a7014af60 0x562a7014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.484+0000 7f0ba57fa640 10 client.23190 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.484+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562a7014af60 0x562a7014d350 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6210f6c3072eca34 server_cookie=922908783f347dcd in_seq=0 out_seq=0 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0ba57fa640 1 -- 172.21.15.105:0/1625094062 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562a700a2eb0 con 0x562a7014af60 2024-03-26T17:59:57.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0ba57fa640 10 client.23190 handle_command_reply: tid=0 2024-03-26T17:59:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 client.23190 shutdown 2024-03-26T17:59:57.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/1625094062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 msgr2=0x562a7013f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013f860 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0cd180 tx=0x7f0bac098080 comp rx=0 tx=0).stop 2024-03-26T17:59:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/1625094062 shutdown_connections 2024-03-26T17:59:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562a7014af60 0x562a7014d350 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0ba57fa640 0 client.23190 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:59:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a7013a9a0 0x562a701468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a70135830 0x562a7013f860 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/1625094062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70134e40 0x562a7013f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/1625094062 >> 172.21.15.105:0/1625094062 conn(0x562a7012bfa0 msgr2=0x562a7012b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/1625094062 shutdown_connections 2024-03-26T17:59:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/1625094062 wait complete. 2024-03-26T17:59:57.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.485+0000 7f0bc6c1b740 20 client.23190 trim_cache size 0 max 16384 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc6c1b740 1 Processor -- start 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc6c1b740 1 -- start start 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70127050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a70146860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a7012a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1a6b00 con 0x562a7013a9a0 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1a8640 con 0x562a70135830 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1a71d0 con 0x562a70134e40 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc25d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a7012a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.490 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc25d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a7012a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55772/0 (socket says 172.21.15.105:55772) 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc25d1640 1 -- 172.21.15.105:0/102927472 learned_addr learned my addr 172.21.15.105:0/102927472 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70127050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.505+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a70146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0bc1dd0640 1 -- 172.21.15.105:0/102927472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 msgr2=0x562a70146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a70146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0bc1dd0640 1 -- 172.21.15.105:0/102927472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 msgr2=0x562a7012a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a7012a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0bc1dd0640 1 -- 172.21.15.105:0/102927472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a70127820 con 0x562a70134e40 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a70146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a7012a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70127050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x562a7014da30 tx=0x7f0bb8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477eefae9f58e3e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0ba77fe640 1 -- 172.21.15.105:0/102927472 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb801f630 con 0x562a70134e40 2024-03-26T17:59:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0ba77fe640 1 -- 172.21.15.105:0/102927472 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bb801f7d0 con 0x562a70134e40 2024-03-26T17:59:57.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.506+0000 7f0ba77fe640 1 -- 172.21.15.105:0/102927472 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8003850 con 0x562a70134e40 2024-03-26T17:59:57.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/102927472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 msgr2=0x562a70127050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70127050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x562a7014da30 tx=0x7f0bb8005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/102927472 shutdown_connections 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a7012a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a70146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/102927472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70127050 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/102927472 >> 172.21.15.105:0/102927472 conn(0x562a7012bfa0 msgr2=0x562a70136d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/102927472 shutdown_connections 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/102927472 wait complete. 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.507+0000 7f0bc6c1b740 1 -- start start 2024-03-26T17:59:57.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc6c1b740 1 Processor -- start 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70143d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a701442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc6c1b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a701452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1a7f70 con 0x562a7013a9a0 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1a78a0 con 0x562a70135830 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc6c1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc1a4ae0 con 0x562a70134e40 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc6c1b740 10 client.0 _ll_register_callbacks cb 0x7f0bb8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc15cf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a701442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc25d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a701452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc15cf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a701442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36584/0 (socket says 172.21.15.105:36584) 2024-03-26T17:59:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc15cf640 1 -- 172.21.15.105:0/4005601722 learned_addr learned my addr 172.21.15.105:0/4005601722 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.508+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc25d1640 1 -- 172.21.15.105:0/4005601722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 msgr2=0x562a70143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70143d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc25d1640 1 -- 172.21.15.105:0/4005601722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 msgr2=0x562a701442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a701442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc25d1640 1 -- 172.21.15.105:0/4005601722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a701458b0 con 0x562a7013a9a0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc15cf640 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a701442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70143d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc25d1640 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a701452c0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x562a70146e30 tx=0x7f0bb40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6da7913c71f5699 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0ba57fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0ba57fa640 1 -- 172.21.15.105:0/4005601722 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4055020 con 0x562a7013a9a0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0ba57fa640 1 -- 172.21.15.105:0/4005601722 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0bb4004790 con 0x562a7013a9a0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4005601722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb801f7d0 con 0x562a7013a9a0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0ba57fa640 1 -- 172.21.15.105:0/4005601722 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0bb4039070 con 0x562a7013a9a0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc6c1b740 10 client.23196 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.509+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4005601722 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562a701b10c0 con 0x562a7013a9a0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0ba57fa640 1 -- 172.21.15.105:0/4005601722 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bb801f7d0 con 0x562a7013a9a0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0ba57fa640 1 -- 172.21.15.105:0/4005601722 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f0bb4034440 con 0x562a7013a9a0 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0bc6c1b740 10 client.23196 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:57.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0bc6c1b740 10 client.23196 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0bc6c1b740 20 client.23196 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562a7014af60 0x562a701b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0bc6c1b740 4 client.23196 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4005601722 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a7009d200 con 0x562a7014af60 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.510+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562a7014af60 0x562a701b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.511+0000 7f0bc1dd0640 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562a7014af60 0x562a701b3950 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4acabd54dd7e8ce3 server_cookie=7590037f2d6d0231 in_seq=0 out_seq=0 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.511+0000 7f0ba57fa640 10 client.23196 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.511+0000 7f0ba57fa640 1 -- 172.21.15.105:0/4005601722 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a7009d200 con 0x562a7014af60 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.511+0000 7f0ba57fa640 10 client.23196 handle_command_reply: tid=0 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.511+0000 7f0bc6c1b740 1 client.23196 shutdown 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.511+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4005601722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 msgr2=0x562a701452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.511+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a701452c0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x562a70146e30 tx=0x7f0bb40095c0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.512+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4005601722 shutdown_connections 2024-03-26T17:59:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.512+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x562a7014af60 0x562a701b3950 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.512+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562a7013a9a0 0x562a701452c0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.512+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a70135830 0x562a701442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.512+0000 7f0bc6c1b740 1 --2- 172.21.15.105:0/4005601722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562a70134e40 0x562a70143d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.512+0000 7f0ba57fa640 0 client.23196 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T17:59:57.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.512+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4005601722 >> 172.21.15.105:0/4005601722 conn(0x562a7012bfa0 msgr2=0x562a7012f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.513+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4005601722 shutdown_connections 2024-03-26T17:59:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.513+0000 7f0bc6c1b740 1 -- 172.21.15.105:0/4005601722 wait complete. 2024-03-26T17:59:57.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.513+0000 7f0bc6c1b740 20 client.23196 trim_cache size 0 max 16384 2024-03-26T17:59:57.502 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 -- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ba80a3070 msgr2=0x7f0ba80a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ba80a3070 0x7f0ba80a5530 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x562a70136090 tx=0x7f0bb4032340 comp rx=0 tx=0).stop 2024-03-26T17:59:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 -- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc1019b0 msgr2=0x7f0bbc198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc198050 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00ea60 tx=0x7f0bac00ef20 comp rx=0 tx=0).stop 2024-03-26T17:59:57.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 -- 172.21.15.105:0/661647355 shutdown_connections 2024-03-26T17:59:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0ba80a3070 0x7f0ba80a5530 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bbc102370 0x7f0bbc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bbc1019b0 0x7f0bbc198050 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 --2- 172.21.15.105:0/661647355 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0bbc1007c0 0x7f0bbc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 -- 172.21.15.105:0/661647355 >> 172.21.15.105:0/661647355 conn(0x7f0bbc0fbd50 msgr2=0x7f0bbc0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 -- 172.21.15.105:0/661647355 shutdown_connections 2024-03-26T17:59:57.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.514+0000 7f0bc4034640 1 -- 172.21.15.105:0/661647355 wait complete. 2024-03-26T17:59:57.512 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T17:59:57.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 68238 7.09 MB/sec execute 400 sec latency 128.630 ms 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 -- 172.21.15.105:0/3322607606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 msgr2=0x7f23ac108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 --2- 172.21.15.105:0/3322607606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 0x7f23ac108c00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f23a4009ca0 tx=0x7f23a402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 -- 172.21.15.105:0/3322607606 shutdown_connections 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 --2- 172.21.15.105:0/3322607606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 0x7f23ac108c00 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 --2- 172.21.15.105:0/3322607606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23ac1019b0 0x7f23ac101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 --2- 172.21.15.105:0/3322607606 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ac1007c0 0x7f23ac100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 -- 172.21.15.105:0/3322607606 >> 172.21.15.105:0/3322607606 conn(0x7f23ac0fbd50 msgr2=0x7f23ac0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 -- 172.21.15.105:0/3322607606 shutdown_connections 2024-03-26T17:59:57.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.674+0000 7f23b4522640 1 -- 172.21.15.105:0/3322607606 wait complete. 2024-03-26T17:59:57.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b4522640 1 Processor -- start 2024-03-26T17:59:57.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b4522640 1 -- start start 2024-03-26T17:59:57.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b4522640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ac1007c0 0x7f23ac197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b4522640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23ac1019b0 0x7f23ac198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b4522640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 0x7f23ac19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b4522640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10d3d0 con 0x7f23ac102370 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b4522640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10d250 con 0x7f23ac1007c0 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b4522640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac10d550 con 0x7f23ac1019b0 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b22be640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ac1007c0 0x7f23ac197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b2abf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 0x7f23ac19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b2abf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 0x7f23ac19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55810/0 (socket says 172.21.15.105:55810) 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b2abf640 1 -- 172.21.15.105:0/3515166018 learned_addr learned my addr 172.21.15.105:0/3515166018 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b22be640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ac1007c0 0x7f23ac197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36608/0 (socket says 172.21.15.105:36608) 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b1abd640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23ac1019b0 0x7f23ac198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b2abf640 1 -- 172.21.15.105:0/3515166018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23ac1019b0 msgr2=0x7f23ac198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b2abf640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23ac1019b0 0x7f23ac198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b2abf640 1 -- 172.21.15.105:0/3515166018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ac1007c0 msgr2=0x7f23ac197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b2abf640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ac1007c0 0x7f23ac197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.675+0000 7f23b2abf640 1 -- 172.21.15.105:0/3515166018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ac19f720 con 0x7f23ac102370 2024-03-26T17:59:57.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.676+0000 7f23b22be640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ac1007c0 0x7f23ac197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.676+0000 7f23b2abf640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 0x7f23ac19f0b0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f23a402c700 tx=0x7f23a40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.676+0000 7f23a37fe640 1 -- 172.21.15.105:0/3515166018 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4041020 con 0x7f23ac102370 2024-03-26T17:59:57.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.676+0000 7f23a37fe640 1 -- 172.21.15.105:0/3515166018 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a4003e20 con 0x7f23ac102370 2024-03-26T17:59:57.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.676+0000 7f23b4522640 1 -- 172.21.15.105:0/3515166018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ac19f9b0 con 0x7f23ac102370 2024-03-26T17:59:57.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.676+0000 7f23a37fe640 1 -- 172.21.15.105:0/3515166018 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4039040 con 0x7f23ac102370 2024-03-26T17:59:57.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.676+0000 7f23b4522640 1 -- 172.21.15.105:0/3515166018 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ac19fea0 con 0x7f23ac102370 2024-03-26T17:59:57.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.679+0000 7f23a37fe640 1 -- 172.21.15.105:0/3515166018 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f23a4044020 con 0x7f23ac102370 2024-03-26T17:59:57.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.679+0000 7f23b71ef740 1 Processor -- start 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b71ef740 1 -- start start 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d97e40 0x55c168d98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d98830 0x55c168d9d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d9d9a0 0x55c168d9fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b71ef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c168cf8970 con 0x55c168d97e40 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b71ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c168ca0d70 con 0x55c168d9d9a0 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b1abd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d97e40 0x55c168d98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b1abd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d97e40 0x55c168d98260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55820/0 (socket says 172.21.15.105:55820) 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b22be640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d98830 0x55c168d9d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23a37fe640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23980a2f20 0x7f23980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b1abd640 1 -- 172.21.15.105:0/4032584241 learned_addr learned my addr 172.21.15.105:0/4032584241 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b71ef740 1 -- 172.21.15.105:0/4032584241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c168cf4920 con 0x55c168d98830 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.680+0000 7f23b1abd640 1 --2- 172.21.15.105:0/4032584241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d9d9a0 0x55c168d9fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23a37fe640 1 -- 172.21.15.105:0/3515166018 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f23a40f15c0 con 0x7f23ac102370 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23b2abf640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23980a2f20 0x7f23980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23b22be640 1 -- 172.21.15.105:0/4032584241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d9d9a0 msgr2=0x55c168d9fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23b22be640 1 --2- 172.21.15.105:0/4032584241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d9d9a0 0x55c168d9fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23b22be640 1 -- 172.21.15.105:0/4032584241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d97e40 msgr2=0x55c168d98260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23b22be640 1 --2- 172.21.15.105:0/4032584241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d97e40 0x55c168d98260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23b22be640 1 -- 172.21.15.105:0/4032584241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c168a0c3f0 con 0x55c168d98830 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23b22be640 1 --2- 172.21.15.105:0/4032584241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d98830 0x55c168d9d330 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009fa0 tx=0x7f23a80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf02d56baa26786 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23a0bfe640 1 -- 172.21.15.105:0/4032584241 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8004310 con 0x55c168d98830 2024-03-26T17:59:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23a0bfe640 1 -- 172.21.15.105:0/4032584241 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a80044b0 con 0x55c168d98830 2024-03-26T17:59:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23b2abf640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23980a2f20 0x7f23980a53e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f23a402c6d0 tx=0x7f23a402c910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.681+0000 7f23a0bfe640 1 -- 172.21.15.105:0/4032584241 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a80035f0 con 0x55c168d98830 2024-03-26T17:59:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.682+0000 7f23b71ef740 1 -- 172.21.15.105:0/4032584241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d98830 msgr2=0x55c168d9d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.682+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4032584241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d98830 0x55c168d9d330 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009fa0 tx=0x7f23a80118d0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.682+0000 7f23b71ef740 1 -- 172.21.15.105:0/4032584241 shutdown_connections 2024-03-26T17:59:57.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.682+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4032584241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d9d9a0 0x55c168d9fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.682+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4032584241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d98830 0x55c168d9d330 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.682+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4032584241 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d97e40 0x55c168d98260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.682+0000 7f23b71ef740 1 -- 172.21.15.105:0/4032584241 >> 172.21.15.105:0/4032584241 conn(0x55c168d933b0 msgr2=0x55c168d937b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.682+0000 7f23b71ef740 1 -- 172.21.15.105:0/4032584241 shutdown_connections 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 -- 172.21.15.105:0/4032584241 wait complete. 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 -- start start 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 Processor -- start 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168da2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d9d9a0 0x55c168da96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.676 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b1abd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c168da0950 con 0x55c168d98830 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b1abd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36624/0 (socket says 172.21.15.105:36624) 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b1abd640 1 -- 172.21.15.105:0/2503507638 learned_addr learned my addr 172.21.15.105:0/2503507638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c168cb4310 con 0x55c168d97e40 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.683+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c168da0ad0 con 0x55c168d9d9a0 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b22be640 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168da2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b71ef740 10 client.0 _ll_register_callbacks cb 0x55c168cefc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b1abd640 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d9d9a0 0x55c168da96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b1abd640 1 -- 172.21.15.105:0/2503507638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d9d9a0 msgr2=0x55c168da96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b1abd640 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d9d9a0 0x55c168da96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b1abd640 1 -- 172.21.15.105:0/2503507638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 msgr2=0x55c168da2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b1abd640 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168da2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b1abd640 1 -- 172.21.15.105:0/2503507638 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c168ad67d0 con 0x55c168d97e40 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b1abd640 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da2140 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f239c009e40 tx=0x7f239c009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97123bfafff92e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f2395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f2395ffb640 1 -- 172.21.15.105:0/2503507638 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c023be0 con 0x55c168d97e40 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f2395ffb640 1 -- 172.21.15.105:0/2503507638 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f239c023d80 con 0x55c168d97e40 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.684+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a80044b0 con 0x55c168d97e40 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f2395ffb640 1 -- 172.21.15.105:0/2503507638 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f239c004020 con 0x55c168d97e40 2024-03-26T17:59:57.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b71ef740 10 client.32888 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c168d90860 con 0x55c168d97e40 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f2395ffb640 1 -- 172.21.15.105:0/2503507638 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a80044b0 con 0x55c168d97e40 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f2395ffb640 1 -- 172.21.15.105:0/2503507638 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f239c005100 con 0x55c168d97e40 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b71ef740 10 client.32888 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b71ef740 10 client.32888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b71ef740 20 client.32888 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b71ef740 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c168d91eb0 0x55c168d92360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b71ef740 4 client.32888 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c168d05eb0 con 0x55c168d91eb0 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.685+0000 7f23b22be640 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c168d91eb0 0x55c168d92360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.686+0000 7f2395ffb640 10 client.32888 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:59:57.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.686+0000 7f23b22be640 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c168d91eb0 0x55c168d92360 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f7dd1eddcccd0ef server_cookie=10910e7904566c9f in_seq=0 out_seq=0 2024-03-26T17:59:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f2395ffb640 1 -- 172.21.15.105:0/2503507638 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c168d05eb0 con 0x55c168d91eb0 2024-03-26T17:59:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f2395ffb640 10 client.32888 handle_command_reply: tid=0 2024-03-26T17:59:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 client.32888 shutdown 2024-03-26T17:59:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 msgr2=0x55c168da2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da2140 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f239c009e40 tx=0x7f239c009950 comp rx=0 tx=0).stop 2024-03-26T17:59:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 shutdown_connections 2024-03-26T17:59:57.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c168d91eb0 0x55c168d92360 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f2395ffb640 0 client.32888 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:59:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d9d9a0 0x55c168da96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168da2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 --2- 172.21.15.105:0/2503507638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da2140 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 >> 172.21.15.105:0/2503507638 conn(0x55c168d933b0 msgr2=0x55c168d8e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 shutdown_connections 2024-03-26T17:59:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 1 -- 172.21.15.105:0/2503507638 wait complete. 2024-03-26T17:59:57.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.687+0000 7f23b71ef740 20 client.32888 trim_cache size 0 max 16384 2024-03-26T17:59:57.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.711+0000 7f23b71ef740 1 Processor -- start 2024-03-26T17:59:57.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.711+0000 7f23b71ef740 1 -- start start 2024-03-26T17:59:57.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.711+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d933b0 0x55c168da2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.711+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d933b0 0x55c168da2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d933b0 0x55c168da2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34862/0 (socket says 172.21.15.105:34862) 2024-03-26T17:59:57.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 -- 172.21.15.105:0/4168097351 learned_addr learned my addr 172.21.15.105:0/4168097351 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168d8d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b71ef740 1 -- 172.21.15.105:0/4168097351 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac1a6c10 con 0x55c168d98830 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b71ef740 1 -- 172.21.15.105:0/4168097351 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac1a8750 con 0x55c168d97e40 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b71ef740 1 -- 172.21.15.105:0/4168097351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac1a72e0 con 0x55c168d933b0 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b22be640 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168d8d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 -- 172.21.15.105:0/4168097351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 msgr2=0x55c168da9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 -- 172.21.15.105:0/4168097351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 msgr2=0x55c168d8d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168d8d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.712+0000 7f23b1abd640 1 -- 172.21.15.105:0/4168097351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c168d93e00 con 0x55c168d933b0 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.713+0000 7f23b1abd640 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d933b0 0x55c168da2460 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f239c0059f0 tx=0x7f239c0137e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9db9949f8ede8d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.713+0000 7f2397fff640 1 -- 172.21.15.105:0/4168097351 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c023c70 con 0x55c168d933b0 2024-03-26T17:59:57.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.713+0000 7f2397fff640 1 -- 172.21.15.105:0/4168097351 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f239c023e10 con 0x55c168d933b0 2024-03-26T17:59:57.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.713+0000 7f2397fff640 1 -- 172.21.15.105:0/4168097351 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c025630 con 0x55c168d933b0 2024-03-26T17:59:57.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 -- 172.21.15.105:0/4168097351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d933b0 msgr2=0x55c168da2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d933b0 0x55c168da2460 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f239c0059f0 tx=0x7f239c0137e0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 -- 172.21.15.105:0/4168097351 shutdown_connections 2024-03-26T17:59:57.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168d8d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d97e40 0x55c168da9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 --2- 172.21.15.105:0/4168097351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d933b0 0x55c168da2460 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 -- 172.21.15.105:0/4168097351 >> 172.21.15.105:0/4168097351 conn(0x55c168d9d4a0 msgr2=0x55c168da1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 -- 172.21.15.105:0/4168097351 shutdown_connections 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 -- 172.21.15.105:0/4168097351 wait complete. 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 -- start start 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.714+0000 7f23b71ef740 1 Processor -- start 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d933b0 0x55c168da7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d97e40 0x55c168da7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b71ef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168da8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b71ef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac1a8080 con 0x55c168d98830 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b71ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac1a79b0 con 0x55c168d933b0 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b71ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23ac1a4bf0 con 0x55c168d97e40 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b71ef740 10 client.0 _ll_register_callbacks cb 0x7f239c025f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d97e40 0x55c168da7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b1abd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168da8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d97e40 0x55c168da7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34878/0 (socket says 172.21.15.105:34878) 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 -- 172.21.15.105:0/3358758440 learned_addr learned my addr 172.21.15.105:0/3358758440 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b1abd640 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d933b0 0x55c168da7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 -- 172.21.15.105:0/3358758440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d933b0 msgr2=0x55c168da7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d933b0 0x55c168da7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 -- 172.21.15.105:0/3358758440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 msgr2=0x55c168da8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168da8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 -- 172.21.15.105:0/3358758440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c168da8b10 con 0x55c168d97e40 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b1abd640 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d933b0 0x55c168da7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b22be640 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d97e40 0x55c168da7580 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009fd0 tx=0x7f23a8005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c448910ddf37303a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f2395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f2395ffb640 1 -- 172.21.15.105:0/3358758440 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8033060 con 0x55c168d97e40 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.715+0000 7f23b71ef740 1 -- 172.21.15.105:0/3358758440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f239c023e10 con 0x55c168d97e40 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f2395ffb640 1 -- 172.21.15.105:0/3358758440 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f23a80037f0 con 0x55c168d97e40 2024-03-26T17:59:57.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f2395ffb640 1 -- 172.21.15.105:0/3358758440 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f23a801e040 con 0x55c168d97e40 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b71ef740 10 client.32884 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b71ef740 1 -- 172.21.15.105:0/3358758440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c168e10510 con 0x55c168d97e40 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f2395ffb640 1 -- 172.21.15.105:0/3358758440 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f239c023e10 con 0x55c168d97e40 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f2395ffb640 1 -- 172.21.15.105:0/3358758440 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f23a8018070 con 0x55c168d97e40 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b71ef740 10 client.32884 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b71ef740 10 client.32884 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b71ef740 20 client.32884 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b71ef740 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c168d9d9a0 0x55c168e12d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b71ef740 4 client.32884 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b71ef740 1 -- 172.21.15.105:0/3358758440 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c168d00200 con 0x55c168d9d9a0 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.716+0000 7f23b1abd640 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c168d9d9a0 0x55c168e12d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f2395ffb640 10 client.32884 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f23b1abd640 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c168d9d9a0 0x55c168e12d90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9215310912745f3 server_cookie=f60df792c5bd9805 in_seq=0 out_seq=0 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f2395ffb640 1 -- 172.21.15.105:0/3358758440 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c168d00200 con 0x55c168d9d9a0 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f2395ffb640 10 client.32884 handle_command_reply: tid=0 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f23b71ef740 1 client.32884 shutdown 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f23b71ef740 1 -- 172.21.15.105:0/3358758440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d97e40 msgr2=0x55c168da7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f23b71ef740 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d97e40 0x55c168da7580 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009fd0 tx=0x7f23a8005850 comp rx=0 tx=0).stop 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f23b71ef740 1 -- 172.21.15.105:0/3358758440 shutdown_connections 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f23b71ef740 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c168d9d9a0 0x55c168e12d90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f23b71ef740 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c168d98830 0x55c168da8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.717+0000 7f2395ffb640 0 client.32884 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.718+0000 7f23b71ef740 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c168d97e40 0x55c168da7580 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.718+0000 7f23b71ef740 1 --2- 172.21.15.105:0/3358758440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c168d933b0 0x55c168da7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.718+0000 7f23b71ef740 1 -- 172.21.15.105:0/3358758440 >> 172.21.15.105:0/3358758440 conn(0x55c168d9d4a0 msgr2=0x55c168d96610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.718+0000 7f23b71ef740 1 -- 172.21.15.105:0/3358758440 shutdown_connections 2024-03-26T17:59:57.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.718+0000 7f23b71ef740 1 -- 172.21.15.105:0/3358758440 wait complete. 2024-03-26T17:59:57.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.718+0000 7f23b71ef740 20 client.32884 trim_cache size 0 max 16384 2024-03-26T17:59:57.710 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 -- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23980a2f20 msgr2=0x7f23980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23980a2f20 0x7f23980a53e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f23a402c6d0 tx=0x7f23a402c910 comp rx=0 tx=0).stop 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 -- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 msgr2=0x7f23ac19f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 0x7f23ac19f0b0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f23a402c700 tx=0x7f23a40328a0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 -- 172.21.15.105:0/3515166018 shutdown_connections 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23980a2f20 0x7f23980a53e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23ac102370 0x7f23ac19f0b0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23ac1019b0 0x7f23ac198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 --2- 172.21.15.105:0/3515166018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23ac1007c0 0x7f23ac197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 -- 172.21.15.105:0/3515166018 >> 172.21.15.105:0/3515166018 conn(0x7f23ac0fbd50 msgr2=0x7f23ac104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 -- 172.21.15.105:0/3515166018 shutdown_connections 2024-03-26T17:59:57.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.719+0000 7f23b4522640 1 -- 172.21.15.105:0/3515166018 wait complete. 2024-03-26T17:59:57.716 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T17:59:57.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 -- 172.21.15.105:0/1537525724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 msgr2=0x7f5618108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 --2- 172.21.15.105:0/1537525724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 0x7f5618108c00 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f5610009ca0 tx=0x7f561002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 -- 172.21.15.105:0/1537525724 shutdown_connections 2024-03-26T17:59:57.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 --2- 172.21.15.105:0/1537525724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 0x7f5618108c00 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 --2- 172.21.15.105:0/1537525724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56181019b0 0x7f5618101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 --2- 172.21.15.105:0/1537525724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56181007c0 0x7f5618100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 -- 172.21.15.105:0/1537525724 >> 172.21.15.105:0/1537525724 conn(0x7f56180fbd50 msgr2=0x7f56180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 -- 172.21.15.105:0/1537525724 shutdown_connections 2024-03-26T17:59:57.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.878+0000 7f561fdc4640 1 -- 172.21.15.105:0/1537525724 wait complete. 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561fdc4640 1 Processor -- start 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561fdc4640 1 -- start start 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561fdc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56181007c0 0x7f5618197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561fdc4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56181019b0 0x7f5618198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561fdc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 0x7f561819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561fdc4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f561810c2d0 con 0x7f5618102370 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561fdc4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f561810c150 con 0x7f56181019b0 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561fdc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f561810c450 con 0x7f56181007c0 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561e361640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 0x7f561819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561db60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56181007c0 0x7f5618197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561e361640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 0x7f561819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55872/0 (socket says 172.21.15.105:55872) 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561db60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56181007c0 0x7f5618197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34896/0 (socket says 172.21.15.105:34896) 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.879+0000 7f561e361640 1 -- 172.21.15.105:0/522488235 learned_addr learned my addr 172.21.15.105:0/522488235 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561d35f640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56181019b0 0x7f5618198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561db60640 1 -- 172.21.15.105:0/522488235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56181019b0 msgr2=0x7f5618198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561db60640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56181019b0 0x7f5618198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561db60640 1 -- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 msgr2=0x7f561819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561db60640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 0x7f561819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561db60640 1 -- 172.21.15.105:0/522488235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561819f6f0 con 0x7f56181007c0 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561e361640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 0x7f561819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561db60640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56181007c0 0x7f5618197ad0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f561400e9c0 tx=0x7f561400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f560effd640 1 -- 172.21.15.105:0/522488235 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561400ccc0 con 0x7f56181007c0 2024-03-26T17:59:57.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561fdc4640 1 -- 172.21.15.105:0/522488235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561819f9e0 con 0x7f56181007c0 2024-03-26T17:59:57.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f561fdc4640 1 -- 172.21.15.105:0/522488235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5618102e60 con 0x7f56181007c0 2024-03-26T17:59:57.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f560effd640 1 -- 172.21.15.105:0/522488235 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56140035f0 con 0x7f56181007c0 2024-03-26T17:59:57.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.880+0000 7f560effd640 1 -- 172.21.15.105:0/522488235 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561400ce60 con 0x7f56181007c0 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f560effd640 1 -- 172.21.15.105:0/522488235 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5614010690 con 0x7f56181007c0 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f56229ad740 1 Processor -- start 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f56229ad740 1 -- start start 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f56229ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f56229ad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57faf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f56229ad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f57fda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f56229ad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a8f575a970 con 0x55a8f57f6420 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f56229ad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a8f5702d70 con 0x55a8f57fb590 2024-03-26T17:59:57.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f56229ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8f5756920 con 0x55a8f57f5a30 2024-03-26T17:59:57.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f561e361640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f561e361640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34904/0 (socket says 172.21.15.105:34904) 2024-03-26T17:59:57.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f561e361640 1 -- 172.21.15.105:0/1869298329 learned_addr learned my addr 172.21.15.105:0/1869298329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f561db60640 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f561d35f640 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f57fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.884+0000 7f560effd640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56040a2fa0 0x7f56040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f561e361640 1 -- 172.21.15.105:0/1869298329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 msgr2=0x55a8f57fda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f561e361640 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f57fda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f561e361640 1 -- 172.21.15.105:0/1869298329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 msgr2=0x55a8f57faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f561e361640 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57faf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f560effd640 1 -- 172.21.15.105:0/522488235 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f56140cf7d0 con 0x7f56181007c0 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f561e361640 1 -- 172.21.15.105:0/1869298329 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f546e3f0 con 0x55a8f57f5a30 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f561db60640 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f561e361640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56040a2fa0 0x7f56040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f561e361640 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f5e50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5610037040 tx=0x7f5610009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a89351b01d7e054 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.885+0000 7f5603fff640 1 -- 172.21.15.105:0/1869298329 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610039040 con 0x55a8f57f5a30 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.886+0000 7f5603fff640 1 -- 172.21.15.105:0/1869298329 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56100046f0 con 0x55a8f57f5a30 2024-03-26T17:59:57.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.886+0000 7f5603fff640 1 -- 172.21.15.105:0/1869298329 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610034ab0 con 0x55a8f57f5a30 2024-03-26T17:59:57.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.886+0000 7f56229ad740 1 -- 172.21.15.105:0/1869298329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 msgr2=0x55a8f57f5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.886+0000 7f561e361640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56040a2fa0 0x7f56040a5460 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f561004a000 tx=0x7f5610045000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.886+0000 7f56229ad740 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f5e50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5610037040 tx=0x7f5610009880 comp rx=0 tx=0).stop 2024-03-26T17:59:57.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.887+0000 7f56229ad740 1 -- 172.21.15.105:0/1869298329 shutdown_connections 2024-03-26T17:59:57.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.887+0000 7f56229ad740 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f57fda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.887+0000 7f56229ad740 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.887+0000 7f56229ad740 1 --2- 172.21.15.105:0/1869298329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f5e50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.887+0000 7f56229ad740 1 -- 172.21.15.105:0/1869298329 >> 172.21.15.105:0/1869298329 conn(0x55a8f57f0fa0 msgr2=0x55a8f57f13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.887+0000 7f56229ad740 1 -- 172.21.15.105:0/1869298329 shutdown_connections 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.887+0000 7f56229ad740 1 -- 172.21.15.105:0/1869298329 wait complete. 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.888+0000 7f56229ad740 1 -- start start 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.888+0000 7f56229ad740 1 Processor -- start 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.888+0000 7f56229ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.888+0000 7f56229ad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57f6420 0x55a8f58004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.888+0000 7f56229ad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57fb590 0x55a8f5807520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.888+0000 7f561db60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.888+0000 7f561d35f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57f6420 0x55a8f58004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561d35f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57f6420 0x55a8f58004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36684/0 (socket says 172.21.15.105:36684) 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561d35f640 1 -- 172.21.15.105:0/4023192440 learned_addr learned my addr 172.21.15.105:0/4023192440 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a8f57fe540 con 0x55a8f57fb590 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a8f5716310 con 0x55a8f57f6420 2024-03-26T17:59:57.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8f57fe6c0 con 0x55a8f57f5a30 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561e361640 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57fb590 0x55a8f5807520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f56229ad740 10 client.0 _ll_register_callbacks cb 0x55a8f5751c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561d35f640 1 -- 172.21.15.105:0/4023192440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 msgr2=0x55a8f57fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561d35f640 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561d35f640 1 -- 172.21.15.105:0/4023192440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57fb590 msgr2=0x55a8f5807520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561d35f640 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57fb590 0x55a8f5807520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561d35f640 1 -- 172.21.15.105:0/4023192440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f55387d0 con 0x55a8f57f6420 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f561d35f640 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57f6420 0x55a8f58004e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5608017e30 tx=0x7f560800e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df2f801552497c23 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f56017fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f56017fa640 1 -- 172.21.15.105:0/4023192440 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608005220 con 0x55a8f57f6420 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f56017fa640 1 -- 172.21.15.105:0/4023192440 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f56080053c0 con 0x55a8f57f6420 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.889+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56100046f0 con 0x55a8f57f6420 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56017fa640 1 -- 172.21.15.105:0/4023192440 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f56080055c0 con 0x55a8f57f6420 2024-03-26T17:59:57.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56229ad740 10 client.32906 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a8f580a910 con 0x55a8f57f6420 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56017fa640 1 -- 172.21.15.105:0/4023192440 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5608004c50 con 0x55a8f57f6420 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56017fa640 1 -- 172.21.15.105:0/4023192440 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f560800d9f0 con 0x55a8f57f6420 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56229ad740 10 client.32906 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56229ad740 10 client.32906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56229ad740 20 client.32906 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56229ad740 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a8f580c030 0x55a8f580e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56229ad740 4 client.32906 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56100046f0 con 0x55a8f580c030 2024-03-26T17:59:57.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.890+0000 7f561db60640 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a8f580c030 0x55a8f580e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.891+0000 7f56017fa640 10 client.32906 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.891+0000 7f561db60640 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a8f580c030 0x55a8f580e4f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b1b75b3032b4e42 server_cookie=fca7503ca6703dee in_seq=0 out_seq=0 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56017fa640 1 -- 172.21.15.105:0/4023192440 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f56100046f0 con 0x55a8f580c030 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56017fa640 10 client.32906 handle_command_reply: tid=0 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 client.32906 shutdown 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57f6420 msgr2=0x55a8f58004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57f6420 0x55a8f58004e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5608017e30 tx=0x7f560800e780 comp rx=0 tx=0).stop 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 shutdown_connections 2024-03-26T17:59:57.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57fb590 0x55a8f5807520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a8f580c030 0x55a8f580e4f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56017fa640 0 client.32906 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:59:57.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57f6420 0x55a8f58004e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 --2- 172.21.15.105:0/4023192440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 >> 172.21.15.105:0/4023192440 conn(0x55a8f57f0fa0 msgr2=0x55a8f57f01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 shutdown_connections 2024-03-26T17:59:57.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 1 -- 172.21.15.105:0/4023192440 wait complete. 2024-03-26T17:59:57.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.892+0000 7f56229ad740 20 client.32906 trim_cache size 0 max 16384 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.911+0000 7f56229ad740 1 Processor -- start 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.911+0000 7f56229ad740 1 -- start start 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.911+0000 7f56229ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.911+0000 7f56229ad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.911+0000 7f56229ad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f58074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f56229ad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc00e4d0 con 0x55a8f57f6420 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f56229ad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc010010 con 0x55a8f57fb590 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f56229ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc00eba0 con 0x55a8f57f5a30 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561e361640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f58074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561e361640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f58074e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36688/0 (socket says 172.21.15.105:36688) 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561d35f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561e361640 1 -- 172.21.15.105:0/1266123362 learned_addr learned my addr 172.21.15.105:0/1266123362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561db60640 1 --2- 172.21.15.105:0/1266123362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561d35f640 1 -- 172.21.15.105:0/1266123362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 msgr2=0x55a8f57f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561d35f640 1 --2- 172.21.15.105:0/1266123362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561d35f640 1 -- 172.21.15.105:0/1266123362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 msgr2=0x55a8f58074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561d35f640 1 --2- 172.21.15.105:0/1266123362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f58074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561d35f640 1 -- 172.21.15.105:0/1266123362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8f57ff060 con 0x55a8f57f6420 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f561d35f640 1 --2- 172.21.15.105:0/1266123362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57f7e60 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f560800df00 tx=0x7f5608022700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d9cf4228066d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.912+0000 7f56037fe640 1 -- 172.21.15.105:0/1266123362 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560803a070 con 0x55a8f57f6420 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.913+0000 7f56037fe640 1 -- 172.21.15.105:0/1266123362 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5608022d30 con 0x55a8f57f6420 2024-03-26T17:59:57.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.913+0000 7f56037fe640 1 -- 172.21.15.105:0/1266123362 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5608031040 con 0x55a8f57f6420 2024-03-26T17:59:57.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.913+0000 7f56229ad740 1 -- 172.21.15.105:0/1266123362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 msgr2=0x55a8f57f7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.913+0000 7f56229ad740 1 --2- 172.21.15.105:0/1266123362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57f7e60 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f560800df00 tx=0x7f5608022700 comp rx=0 tx=0).stop 2024-03-26T17:59:57.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 -- 172.21.15.105:0/1266123362 shutdown_connections 2024-03-26T17:59:57.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 --2- 172.21.15.105:0/1266123362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f58074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 --2- 172.21.15.105:0/1266123362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f57f7e60 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 --2- 172.21.15.105:0/1266123362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f57f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 -- 172.21.15.105:0/1266123362 >> 172.21.15.105:0/1266123362 conn(0x55a8f57f0fa0 msgr2=0x55a8f57fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 -- 172.21.15.105:0/1266123362 shutdown_connections 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 -- 172.21.15.105:0/1266123362 wait complete. 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 -- start start 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.914+0000 7f56229ad740 1 Processor -- start 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f56229ad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f5804940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f56229ad740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f5804e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f56229ad740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f5805e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f56229ad740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc00f940 con 0x55a8f57f6420 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f56229ad740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc00f270 con 0x55a8f57fb590 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f56229ad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc00c4b0 con 0x55a8f57f5a30 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f56229ad740 10 client.0 _ll_register_callbacks cb 0x7f5608017d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:57.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561db60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f5804940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561d35f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f5804e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561db60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f5804940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34942/0 (socket says 172.21.15.105:34942) 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561db60640 1 -- 172.21.15.105:0/1226234951 learned_addr learned my addr 172.21.15.105:0/1226234951 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561e361640 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f5805e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561db60640 1 -- 172.21.15.105:0/1226234951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 msgr2=0x55a8f5805e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561db60640 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f5805e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561db60640 1 -- 172.21.15.105:0/1226234951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 msgr2=0x55a8f5804e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561db60640 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f5804e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561db60640 1 -- 172.21.15.105:0/1226234951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8f5806430 con 0x55a8f57f5a30 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.915+0000 7f561d35f640 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f5804e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.916+0000 7f561db60640 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f5804940 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5614098890 tx=0x7f561409bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8830e20661cd009f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.916+0000 7f56017fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.916+0000 7f56017fa640 1 -- 172.21.15.105:0/1226234951 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561409bec0 con 0x55a8f57f5a30 2024-03-26T17:59:57.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.916+0000 7f56017fa640 1 -- 172.21.15.105:0/1226234951 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f561409a1c0 con 0x55a8f57f5a30 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.916+0000 7f56229ad740 1 -- 172.21.15.105:0/1226234951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5608022d30 con 0x55a8f57f5a30 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.916+0000 7f56017fa640 1 -- 172.21.15.105:0/1226234951 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f56140a0010 con 0x55a8f57f5a30 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.916+0000 7f56229ad740 10 client.32902 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.916+0000 7f56229ad740 1 -- 172.21.15.105:0/1226234951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a8f58077a0 con 0x55a8f57f5a30 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f56017fa640 1 -- 172.21.15.105:0/1226234951 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5608022d30 con 0x55a8f57f5a30 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f56017fa640 1 -- 172.21.15.105:0/1226234951 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f56140a4440 con 0x55a8f57f5a30 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f56229ad740 10 client.32902 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f56229ad740 10 client.32902 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f56229ad740 20 client.32902 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f56229ad740 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a8f580c030 0x55a8f5874b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f56229ad740 4 client.32902 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f56229ad740 1 -- 172.21.15.105:0/1226234951 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8f5767eb0 con 0x55a8f580c030 2024-03-26T17:59:57.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.917+0000 7f561d35f640 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a8f580c030 0x55a8f5874b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f561d35f640 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a8f580c030 0x55a8f5874b60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a76b0e5a2d4cecef server_cookie=a2313eee9572f000 in_seq=0 out_seq=0 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56017fa640 10 client.32902 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56017fa640 1 -- 172.21.15.105:0/1226234951 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8f5767eb0 con 0x55a8f580c030 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56017fa640 10 client.32902 handle_command_reply: tid=0 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 client.32902 shutdown 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 -- 172.21.15.105:0/1226234951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 msgr2=0x55a8f5804940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f5804940 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f5614098890 tx=0x7f561409bac0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 -- 172.21.15.105:0/1226234951 shutdown_connections 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55a8f580c030 0x55a8f5874b60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a8f57fb590 0x55a8f5805e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56017fa640 0 client.32902 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T17:59:57.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a8f57f6420 0x55a8f5804e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 --2- 172.21.15.105:0/1226234951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a8f57f5a30 0x55a8f5804940 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 -- 172.21.15.105:0/1226234951 >> 172.21.15.105:0/1226234951 conn(0x55a8f57f0fa0 msgr2=0x55a8f57ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 -- 172.21.15.105:0/1226234951 shutdown_connections 2024-03-26T17:59:57.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 1 -- 172.21.15.105:0/1226234951 wait complete. 2024-03-26T17:59:57.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.918+0000 7f56229ad740 20 client.32902 trim_cache size 0 max 16384 2024-03-26T17:59:57.906 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:57.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 -- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56040a2fa0 msgr2=0x7f56040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56040a2fa0 0x7f56040a5460 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f561004a000 tx=0x7f5610045000 comp rx=0 tx=0).stop 2024-03-26T17:59:57.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 -- 172.21.15.105:0/522488235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56181007c0 msgr2=0x7f5618197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:57.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56181007c0 0x7f5618197ad0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f561400e9c0 tx=0x7f561400ee80 comp rx=0 tx=0).stop 2024-03-26T17:59:57.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 -- 172.21.15.105:0/522488235 shutdown_connections 2024-03-26T17:59:57.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f56040a2fa0 0x7f56040a5460 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5618102370 0x7f561819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f56181019b0 0x7f5618198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 --2- 172.21.15.105:0/522488235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56181007c0 0x7f5618197ad0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 -- 172.21.15.105:0/522488235 >> 172.21.15.105:0/522488235 conn(0x7f56180fbd50 msgr2=0x7f5618068a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 -- 172.21.15.105:0/522488235 shutdown_connections 2024-03-26T17:59:57.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:57.919+0000 7f561fdc4640 1 -- 172.21.15.105:0/522488235 wait complete. 2024-03-26T17:59:57.922 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T17:59:57.972 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:57 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2943408781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/1937866291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81019b0 msgr2=0x7fd0c8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/1937866291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81019b0 0x7fd0c8101e30 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009900 tx=0x7fd0bc02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/1937866291 shutdown_connections 2024-03-26T17:59:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/1937866291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c8102370 0x7fd0c8068a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/1937866291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81019b0 0x7fd0c8101e30 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/1937866291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c81007c0 0x7fd0c8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/1937866291 >> 172.21.15.105:0/1937866291 conn(0x7fd0c80fbd50 msgr2=0x7fd0c80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/1937866291 shutdown_connections 2024-03-26T17:59:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.077+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/1937866291 wait complete. 2024-03-26T17:59:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0cd6f1640 1 Processor -- start 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0cd6f1640 1 -- start start 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0cd6f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81007c0 0x7fd0c8199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0cd6f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c81019b0 0x7fd0c819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0cd6f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c8102370 0x7fd0c81a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0cd6f1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c810f610 con 0x7fd0c81007c0 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0cd6f1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c810f490 con 0x7fd0c8102370 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0cd6f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c810f790 con 0x7fd0c81019b0 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81007c0 0x7fd0c8199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81007c0 0x7fd0c8199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55916/0 (socket says 172.21.15.105:55916) 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.078+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/4016418224 learned_addr learned my addr 172.21.15.105:0/4016418224 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c81019b0 0x7fd0c819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c77fe640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c8102370 0x7fd0c81a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c67fc640 1 -- 172.21.15.105:0/4016418224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c8102370 msgr2=0x7fd0c81a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c8102370 0x7fd0c81a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c67fc640 1 -- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81007c0 msgr2=0x7fd0c8199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81007c0 0x7fd0c8199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c67fc640 1 -- 172.21.15.105:0/4016418224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c81a1960 con 0x7fd0c81019b0 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81007c0 0x7fd0c8199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c77fe640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c8102370 0x7fd0c81a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c81019b0 0x7fd0c819a290 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc000c00 tx=0x7fd0bc005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0affff640 1 -- 172.21.15.105:0/4016418224 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc043020 con 0x7fd0c81019b0 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/4016418224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c81a1bf0 con 0x7fd0c81019b0 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0affff640 1 -- 172.21.15.105:0/4016418224 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0bc004350 con 0x7fd0c81019b0 2024-03-26T17:59:58.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0affff640 1 -- 172.21.15.105:0/4016418224 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc003ae0 con 0x7fd0c81019b0 2024-03-26T17:59:58.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.079+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/4016418224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c81a2140 con 0x7fd0c81019b0 2024-03-26T17:59:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.082+0000 7fd0d03b8740 1 Processor -- start 2024-03-26T17:59:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.082+0000 7fd0d03b8740 1 -- start start 2024-03-26T17:59:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.082+0000 7fd0affff640 1 -- 172.21.15.105:0/4016418224 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd0bc046020 con 0x7fd0c81019b0 2024-03-26T17:59:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.082+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930df240 0x5637930df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930df240 0x5637930df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 0x5637930e7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930df240 0x5637930df660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55918/0 (socket says 172.21.15.105:55918) 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56379303b970 con 0x5637930df240 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 -- 172.21.15.105:0/2136688747 learned_addr learned my addr 172.21.15.105:0/2136688747 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2136688747 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563792fe3d70 con 0x5637930e4da0 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2136688747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563793037920 con 0x5637930dfc30 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 0x5637930e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 -- 172.21.15.105:0/2136688747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 msgr2=0x5637930e4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 -- 172.21.15.105:0/2136688747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 msgr2=0x5637930e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 0x5637930e7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 -- 172.21.15.105:0/2136688747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563792d4f3f0 con 0x5637930df240 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0c77fe640 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930df240 0x5637930df660 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0011810 tx=0x7fd0c0011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae64f0a990570d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0ad7fa640 1 -- 172.21.15.105:0/2136688747 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0011310 con 0x5637930df240 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.083+0000 7fd0affff640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0b00a2f20 0x7fd0b00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0ad7fa640 1 -- 172.21.15.105:0/2136688747 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0c000f9a0 con 0x5637930df240 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0ad7fa640 1 -- 172.21.15.105:0/2136688747 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0005650 con 0x5637930df240 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0b00a2f20 0x7fd0b00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0affff640 1 -- 172.21.15.105:0/4016418224 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd0bc0f1510 con 0x7fd0c81019b0 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2136688747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930df240 msgr2=0x5637930df660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0b00a2f20 0x7fd0b00a53e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x5637930e0030 tx=0x7fd0b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930df240 0x5637930df660 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0011810 tx=0x7fd0c0011cd0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2136688747 shutdown_connections 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 0x5637930e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/2136688747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930df240 0x5637930df660 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.084+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2136688747 >> 172.21.15.105:0/2136688747 conn(0x5637930da7b0 msgr2=0x5637930dabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2136688747 shutdown_connections 2024-03-26T17:59:58.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2136688747 wait complete. 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 -- start start 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 Processor -- start 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930df240 0x5637930e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637930e7df0 con 0x5637930e4da0 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563792ff7310 con 0x5637930df240 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637930e7f70 con 0x5637930dfc30 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.085+0000 7fd0d03b8740 10 client.0 _ll_register_callbacks cb 0x563793032c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c67fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930df240 0x5637930e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930df240 0x5637930e97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36724/0 (socket says 172.21.15.105:36724) 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/1708920883 learned_addr learned my addr 172.21.15.105:0/1708920883 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c67fc640 1 -- 172.21.15.105:0/1708920883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930df240 msgr2=0x5637930e97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930df240 0x5637930e97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c67fc640 1 -- 172.21.15.105:0/1708920883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 msgr2=0x5637930f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c67fc640 1 -- 172.21.15.105:0/1708920883 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563792e197d0 con 0x5637930dfc30 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930df240 0x5637930e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c77fe640 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e9d00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0ef020 tx=0x7fd0bc009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a027c33ca48ec8d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1708920883 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0bd560 con 0x5637930dfc30 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1708920883 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd0bc0be060 con 0x5637930dfc30 2024-03-26T17:59:58.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.086+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1708920883 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c000f9a0 con 0x5637930dfc30 2024-03-26T17:59:58.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1708920883 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd0bc0bbd40 con 0x5637930dfc30 2024-03-26T17:59:58.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0d03b8740 10 client.32914 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:58.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1708920883 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5637930e7e60 con 0x5637930dfc30 2024-03-26T17:59:58.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1708920883 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0c000f9a0 con 0x5637930dfc30 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1708920883 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fd0bc0c6180 con 0x5637930dfc30 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0d03b8740 10 client.32914 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0d03b8740 10 client.32914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0d03b8740 20 client.32914 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5637930f57a0 0x5637930f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0d03b8740 4 client.32914 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1708920883 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563793048eb0 con 0x5637930f57a0 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.087+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5637930f57a0 0x5637930f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.088+0000 7fd0aaff5640 10 client.32914 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.088+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5637930f57a0 0x5637930f7bb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1121989725087a9 server_cookie=137996fa5968298a in_seq=0 out_seq=0 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1708920883 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563793048eb0 con 0x5637930f57a0 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0aaff5640 10 client.32914 handle_command_reply: tid=0 2024-03-26T17:59:58.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 client.32914 shutdown 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1708920883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 msgr2=0x5637930e9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e9d00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0ef020 tx=0x7fd0bc009620 comp rx=0 tx=0).stop 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1708920883 shutdown_connections 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5637930f57a0 0x5637930f7bb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0aaff5640 0 client.32914 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930dfc30 0x5637930e9d00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1708920883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930df240 0x5637930e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1708920883 >> 172.21.15.105:0/1708920883 conn(0x5637930da7b0 msgr2=0x5637930d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1708920883 shutdown_connections 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1708920883 wait complete. 2024-03-26T17:59:58.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.091+0000 7fd0d03b8740 20 client.32914 trim_cache size 0 max 16384 2024-03-26T17:59:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0d03b8740 1 Processor -- start 2024-03-26T17:59:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0d03b8740 1 -- start start 2024-03-26T17:59:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930e10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930dfc30 0x5637930f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 0x5637930e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81a6b20 con 0x5637930dfc30 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81a8660 con 0x5637930e4da0 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81a71f0 con 0x5637930df240 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0c67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930dfc30 0x5637930f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930e10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0c67fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930dfc30 0x5637930f0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55932/0 (socket says 172.21.15.105:55932) 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930e10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34986/0 (socket says 172.21.15.105:34986) 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0c67fc640 1 -- 172.21.15.105:0/2299860177 learned_addr learned my addr 172.21.15.105:0/2299860177 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.111+0000 7fd0c77fe640 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 0x5637930e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/2299860177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 msgr2=0x5637930e1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 0x5637930e1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/2299860177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930dfc30 msgr2=0x5637930f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930dfc30 0x5637930f0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/2299860177 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637930e8bd0 con 0x5637930df240 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930dfc30 0x5637930f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930e10f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc005c40 tx=0x7fd0b801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6de93fc02292ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0acff9640 1 -- 172.21.15.105:0/2299860177 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800fe10 con 0x5637930df240 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0acff9640 1 -- 172.21.15.105:0/2299860177 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0b8030070 con 0x5637930df240 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.112+0000 7fd0acff9640 1 -- 172.21.15.105:0/2299860177 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b801fe00 con 0x5637930df240 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2299860177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 msgr2=0x5637930e10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930e10f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc005c40 tx=0x7fd0b801a000 comp rx=0 tx=0).stop 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2299860177 shutdown_connections 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930e4da0 0x5637930e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930dfc30 0x5637930f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/2299860177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930e10f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2299860177 >> 172.21.15.105:0/2299860177 conn(0x5637930da7b0 msgr2=0x5637930e97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2299860177 shutdown_connections 2024-03-26T17:59:58.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 -- 172.21.15.105:0/2299860177 wait complete. 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 -- start start 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.113+0000 7fd0d03b8740 1 Processor -- start 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930dfc30 0x5637930ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0d03b8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930ef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81a7f90 con 0x5637930e4da0 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81a78c0 con 0x5637930dfc30 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0d03b8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81a4b20 con 0x5637930df240 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0d03b8740 10 client.0 _ll_register_callbacks cb 0x7fd0bc0f4b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930ef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930ee1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34990/0 (socket says 172.21.15.105:34990) 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/1624300568 learned_addr learned my addr 172.21.15.105:0/1624300568 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930dfc30 0x5637930ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/1624300568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930dfc30 msgr2=0x5637930ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930dfc30 0x5637930ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/1624300568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 msgr2=0x5637930ef720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930ef720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 -- 172.21.15.105:0/1624300568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637930efd10 con 0x5637930df240 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0c6ffd640 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930ee1d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800f710 tx=0x7fd0b800fe10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a817b69eb9754ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.114+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1624300568 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8039070 con 0x5637930df240 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1624300568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b8030070 con 0x5637930df240 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1624300568 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fd0b802bb90 con 0x5637930df240 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1624300568 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd0b802bdd0 con 0x5637930df240 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 10 client.32920 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:58.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1624300568 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5637930f1010 con 0x5637930df240 2024-03-26T17:59:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1624300568 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0b8030070 con 0x5637930df240 2024-03-26T17:59:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1624300568 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fd0b80053c0 con 0x5637930df240 2024-03-26T17:59:58.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 10 client.32920 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 10 client.32920 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 20 client.32920 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5637930f57a0 0x56379315e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 4 client.32920 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.115+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1624300568 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563793043200 con 0x5637930f57a0 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.116+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5637930f57a0 0x56379315e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.116+0000 7fd0aaff5640 10 client.32920 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.116+0000 7fd0c67fc640 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5637930f57a0 0x56379315e1d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79c1780e325de4d4 server_cookie=fa5c220d841523e in_seq=0 out_seq=0 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0aaff5640 1 -- 172.21.15.105:0/1624300568 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563793043200 con 0x5637930f57a0 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0aaff5640 10 client.32920 handle_command_reply: tid=0 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0d03b8740 1 client.32920 shutdown 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1624300568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 msgr2=0x5637930ee1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930ee1d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800f710 tx=0x7fd0b800fe10 comp rx=0 tx=0).stop 2024-03-26T17:59:58.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1624300568 shutdown_connections 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x5637930f57a0 0x56379315e1d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637930e4da0 0x5637930ef720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637930dfc30 0x5637930ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.117+0000 7fd0aaff5640 0 client.32920 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0d03b8740 1 --2- 172.21.15.105:0/1624300568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637930df240 0x5637930ee1d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1624300568 >> 172.21.15.105:0/1624300568 conn(0x5637930da7b0 msgr2=0x5637930e9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1624300568 shutdown_connections 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0d03b8740 1 -- 172.21.15.105:0/1624300568 wait complete. 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0d03b8740 20 client.32920 trim_cache size 0 max 16384 2024-03-26T17:59:58.106 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:58.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0b00a2f20 msgr2=0x7fd0b00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0b00a2f20 0x7fd0b00a53e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x5637930e0030 tx=0x7fd0b8005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:58.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/4016418224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c81019b0 msgr2=0x7fd0c819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c81019b0 0x7fd0c819a290 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc000c00 tx=0x7fd0bc005dc0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/4016418224 shutdown_connections 2024-03-26T17:59:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd0b00a2f20 0x7fd0b00a53e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c8102370 0x7fd0c81a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c81019b0 0x7fd0c819a290 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 --2- 172.21.15.105:0/4016418224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0c81007c0 0x7fd0c8199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.118+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/4016418224 >> 172.21.15.105:0/4016418224 conn(0x7fd0c80fbd50 msgr2=0x7fd0c8106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.119+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/4016418224 shutdown_connections 2024-03-26T17:59:58.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.119+0000 7fd0cd6f1640 1 -- 172.21.15.105:0/4016418224 wait complete. 2024-03-26T17:59:58.115 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T17:59:58.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 -- 172.21.15.105:0/2912582375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681034a0 msgr2=0x7f3868109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 --2- 172.21.15.105:0/2912582375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681034a0 0x7f3868109d30 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3860009ca0 tx=0x7f386002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 -- 172.21.15.105:0/2912582375 shutdown_connections 2024-03-26T17:59:58.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 --2- 172.21.15.105:0/2912582375 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38681034a0 0x7f3868109d30 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 --2- 172.21.15.105:0/2912582375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3868102ae0 0x7f3868102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 --2- 172.21.15.105:0/2912582375 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38681018f0 0x7f3868101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 -- 172.21.15.105:0/2912582375 >> 172.21.15.105:0/2912582375 conn(0x7f38680fce60 msgr2=0x7f38680ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 -- 172.21.15.105:0/2912582375 shutdown_connections 2024-03-26T17:59:58.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.274+0000 7f386cb3b640 1 -- 172.21.15.105:0/2912582375 wait complete. 2024-03-26T17:59:58.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.275+0000 7f386cb3b640 1 Processor -- start 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.275+0000 7f386cb3b640 1 -- start start 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.275+0000 7f386cb3b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38681018f0 0x7f3868075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.275+0000 7f386cb3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3868102ae0 0x7f3868073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386cb3b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38681034a0 0x7f38680741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386cb3b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f386810d400 con 0x7f3868102ae0 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386cb3b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f386810d280 con 0x7f38681034a0 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386cb3b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f386810d580 con 0x7f38681018f0 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f3865d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3868102ae0 0x7f3868073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38681018f0 0x7f3868075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f3865d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3868102ae0 0x7f3868073cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55950/0 (socket says 172.21.15.105:55950) 2024-03-26T17:59:58.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f3865d9b640 1 -- 172.21.15.105:0/650635104 learned_addr learned my addr 172.21.15.105:0/650635104 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f3866d9d640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38681034a0 0x7f38680741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386659c640 1 -- 172.21.15.105:0/650635104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38681034a0 msgr2=0x7f38680741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386659c640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38681034a0 0x7f38680741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386659c640 1 -- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3868102ae0 msgr2=0x7f3868073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386659c640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3868102ae0 0x7f3868073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386659c640 1 -- 172.21.15.105:0/650635104 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3868074a80 con 0x7f38681018f0 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f3865d9b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3868102ae0 0x7f3868073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f386659c640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38681018f0 0x7f3868075670 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f385800e9c0 tx=0x7f385800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.276+0000 7f38577fe640 1 -- 172.21.15.105:0/650635104 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385800ccc0 con 0x7f38681018f0 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.277+0000 7f386cb3b640 1 -- 172.21.15.105:0/650635104 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38681a1bd0 con 0x7f38681018f0 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.277+0000 7f386cb3b640 1 -- 172.21.15.105:0/650635104 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38681a2170 con 0x7f38681018f0 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.277+0000 7f38577fe640 1 -- 172.21.15.105:0/650635104 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38580035f0 con 0x7f38681018f0 2024-03-26T17:59:58.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.277+0000 7f38577fe640 1 -- 172.21.15.105:0/650635104 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385800ce60 con 0x7f38681018f0 2024-03-26T17:59:58.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.280+0000 7f38577fe640 1 -- 172.21.15.105:0/650635104 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f38580037a0 con 0x7f38681018f0 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.280+0000 7f38577fe640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38500a2fa0 0x7f38500a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.280+0000 7f386f803740 1 Processor -- start 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f3865d9b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38500a2fa0 0x7f38500a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f38577fe640 1 -- 172.21.15.105:0/650635104 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f38580cf840 con 0x7f38681018f0 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f386f803740 1 -- start start 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f386f803740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c0f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f386f803740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c14730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f386f803740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c14da0 0x55d3e4c17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f386f803740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d3e4b6b970 con 0x55d3e4c0f240 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f386f803740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3e4b13d70 con 0x55d3e4c0fc30 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f386f803740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3e4b67920 con 0x55d3e4c14da0 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f3866d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c14da0 0x55d3e4c17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f3866d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c14da0 0x55d3e4c17230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35020/0 (socket says 172.21.15.105:35020) 2024-03-26T17:59:58.265 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f3866d9d640 1 -- 172.21.15.105:0/1889311553 learned_addr learned my addr 172.21.15.105:0/1889311553 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f386659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c0f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.281+0000 7f3865d9b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38500a2fa0 0x7f38500a5460 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f385c0095f0 tx=0x7f385c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3866d9d640 1 -- 172.21.15.105:0/1889311553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0fc30 msgr2=0x55d3e4c14730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3866d9d640 1 --2- 172.21.15.105:0/1889311553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c14730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3866d9d640 1 -- 172.21.15.105:0/1889311553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 msgr2=0x55d3e4c0f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3866d9d640 1 --2- 172.21.15.105:0/1889311553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c0f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3866d9d640 1 -- 172.21.15.105:0/1889311553 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e487f3f0 con 0x55d3e4c14da0 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f386659c640 1 --2- 172.21.15.105:0/1889311553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c0f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3866d9d640 1 --2- 172.21.15.105:0/1889311553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c14da0 0x55d3e4c17230 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3860005e30 tx=0x7f3860009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832f5ba9a79a8d9a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3854ff9640 1 -- 172.21.15.105:0/1889311553 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860043070 con 0x55d3e4c14da0 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3854ff9640 1 -- 172.21.15.105:0/1889311553 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3860032db0 con 0x55d3e4c14da0 2024-03-26T17:59:58.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.282+0000 7f3854ff9640 1 -- 172.21.15.105:0/1889311553 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860039040 con 0x55d3e4c14da0 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 -- 172.21.15.105:0/1889311553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c14da0 msgr2=0x55d3e4c17230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 --2- 172.21.15.105:0/1889311553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c14da0 0x55d3e4c17230 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3860005e30 tx=0x7f3860009800 comp rx=0 tx=0).stop 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 -- 172.21.15.105:0/1889311553 shutdown_connections 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 --2- 172.21.15.105:0/1889311553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c14da0 0x55d3e4c17230 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 --2- 172.21.15.105:0/1889311553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c14730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 --2- 172.21.15.105:0/1889311553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c0f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 -- 172.21.15.105:0/1889311553 >> 172.21.15.105:0/1889311553 conn(0x55d3e4c01fa0 msgr2=0x55d3e4c023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 -- 172.21.15.105:0/1889311553 shutdown_connections 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.283+0000 7f386f803740 1 -- 172.21.15.105:0/1889311553 wait complete. 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 1 -- start start 2024-03-26T17:59:58.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 1 Processor -- start 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c19720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c19c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c14da0 0x55d3e4c20ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d3e4c17cf0 con 0x55d3e4c0f240 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3e4b27310 con 0x55d3e4c14da0 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d3e4c17e70 con 0x55d3e4c0fc30 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386f803740 10 client.0 _ll_register_callbacks cb 0x55d3e4b62c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c19720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f3865d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c19c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c19720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55958/0 (socket says 172.21.15.105:55958) 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f386659c640 1 -- 172.21.15.105:0/2952860371 learned_addr learned my addr 172.21.15.105:0/2952860371 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.284+0000 7f3866d9d640 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c14da0 0x55d3e4c20ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f3865d9b640 1 -- 172.21.15.105:0/2952860371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c14da0 msgr2=0x55d3e4c20ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f3865d9b640 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c14da0 0x55d3e4c20ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f3865d9b640 1 -- 172.21.15.105:0/2952860371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 msgr2=0x55d3e4c19720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f3865d9b640 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c19720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f3865d9b640 1 -- 172.21.15.105:0/2952860371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e49497d0 con 0x55d3e4c0fc30 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f386659c640 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c19720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f3865d9b640 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c19c60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x55d3e4c106a0 tx=0x7f385c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95ea72ce4e874083 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f384dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f384dffb640 1 -- 172.21.15.105:0/2952860371 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c02b060 con 0x55d3e4c0fc30 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f386f803740 1 -- 172.21.15.105:0/2952860371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3860032db0 con 0x55d3e4c0fc30 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f384dffb640 1 -- 172.21.15.105:0/2952860371 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f385c01cc60 con 0x55d3e4c0fc30 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f384dffb640 1 -- 172.21.15.105:0/2952860371 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f385c01ce60 con 0x55d3e4c0fc30 2024-03-26T17:59:58.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f386f803740 10 client.32932 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.285+0000 7f386f803740 1 -- 172.21.15.105:0/2952860371 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d3e4c24160 con 0x55d3e4c0fc30 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f384dffb640 1 -- 172.21.15.105:0/2952860371 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3860032db0 con 0x55d3e4c0fc30 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f384dffb640 1 -- 172.21.15.105:0/2952860371 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f385c0305d0 con 0x55d3e4c0fc30 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f386f803740 10 client.32932 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f386f803740 10 client.32932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f386f803740 20 client.32932 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f386f803740 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3e4c25780 0x55d3e4c27c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f386f803740 4 client.32932 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f386f803740 1 -- 172.21.15.105:0/2952860371 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3e4b78eb0 con 0x55d3e4c25780 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.286+0000 7f386659c640 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3e4c25780 0x55d3e4c27c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.287+0000 7f386659c640 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3e4c25780 0x55d3e4c27c20 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362febb42f783a8f server_cookie=3aa924c282b4e731 in_seq=0 out_seq=0 2024-03-26T17:59:58.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.287+0000 7f384dffb640 10 client.32932 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:58.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f384dffb640 1 -- 172.21.15.105:0/2952860371 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d3e4b78eb0 con 0x55d3e4c25780 2024-03-26T17:59:58.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f384dffb640 10 client.32932 handle_command_reply: tid=0 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 client.32932 shutdown 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 -- 172.21.15.105:0/2952860371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 msgr2=0x55d3e4c19c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c19c60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x55d3e4c106a0 tx=0x7f385c004730 comp rx=0 tx=0).stop 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 -- 172.21.15.105:0/2952860371 shutdown_connections 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3e4c25780 0x55d3e4c27c20 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c14da0 0x55d3e4c20ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c19c60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 --2- 172.21.15.105:0/2952860371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c0f240 0x55d3e4c19720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 -- 172.21.15.105:0/2952860371 >> 172.21.15.105:0/2952860371 conn(0x55d3e4c01fa0 msgr2=0x55d3e4c011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f384dffb640 0 client.32932 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:58.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 -- 172.21.15.105:0/2952860371 shutdown_connections 2024-03-26T17:59:58.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.288+0000 7f386f803740 1 -- 172.21.15.105:0/2952860371 wait complete. 2024-03-26T17:59:58.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.289+0000 7f386f803740 20 client.32932 trim_cache size 0 max 16384 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386f803740 1 Processor -- start 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386f803740 1 -- start start 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386f803740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c00930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386f803740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c20c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386f803740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c00e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386f803740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f384800e530 con 0x55d3e4c14da0 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386f803740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3848010070 con 0x55d3e4c0f240 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386f803740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f384800ec00 con 0x55d3e4c0fc30 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f3866d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c00e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f3865d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c20c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f3866d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c00e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55960/0 (socket says 172.21.15.105:55960) 2024-03-26T17:59:58.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f386659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c00930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f3866d9d640 1 -- 172.21.15.105:0/797080092 learned_addr learned my addr 172.21.15.105:0/797080092 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.309+0000 7f3866d9d640 1 -- 172.21.15.105:0/797080092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 msgr2=0x55d3e4c20c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f3866d9d640 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c20c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f3866d9d640 1 -- 172.21.15.105:0/797080092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 msgr2=0x55d3e4c00930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f3866d9d640 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c00930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f3866d9d640 1 -- 172.21.15.105:0/797080092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e4c143d0 con 0x55d3e4c14da0 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f3865d9b640 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c20c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f386659c640 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c00930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f3866d9d640 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c00e70 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f3860032420 tx=0x7f386002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327e34c42d7c43bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f384ffff640 1 -- 172.21.15.105:0/797080092 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860039050 con 0x55d3e4c14da0 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f384ffff640 1 -- 172.21.15.105:0/797080092 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3860003970 con 0x55d3e4c14da0 2024-03-26T17:59:58.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.310+0000 7f384ffff640 1 -- 172.21.15.105:0/797080092 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860043040 con 0x55d3e4c14da0 2024-03-26T17:59:58.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 -- 172.21.15.105:0/797080092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 msgr2=0x55d3e4c00e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c00e70 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f3860032420 tx=0x7f386002c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 -- 172.21.15.105:0/797080092 shutdown_connections 2024-03-26T17:59:58.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c00e70 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c20c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 --2- 172.21.15.105:0/797080092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c00930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 -- 172.21.15.105:0/797080092 >> 172.21.15.105:0/797080092 conn(0x55d3e4c01fa0 msgr2=0x55d3e4c11130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 -- 172.21.15.105:0/797080092 shutdown_connections 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 -- 172.21.15.105:0/797080092 wait complete. 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.311+0000 7f386f803740 1 -- start start 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386f803740 1 Processor -- start 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386f803740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c1dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386f803740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c1e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386f803740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386f803740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f384800f9a0 con 0x55d3e4c14da0 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386f803740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f384800f2d0 con 0x55d3e4c0f240 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386f803740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f384800c510 con 0x55d3e4c0fc30 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386f803740 10 client.0 _ll_register_callbacks cb 0x7f385c009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3866d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3865d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c1e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3866d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c1f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55966/0 (socket says 172.21.15.105:55966) 2024-03-26T17:59:58.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3866d9d640 1 -- 172.21.15.105:0/423644407 learned_addr learned my addr 172.21.15.105:0/423644407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3865d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c1e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35048/0 (socket says 172.21.15.105:35048) 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386659c640 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c1dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3866d9d640 1 -- 172.21.15.105:0/423644407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 msgr2=0x55d3e4c1e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3866d9d640 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c1e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3866d9d640 1 -- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 msgr2=0x55d3e4c1dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3866d9d640 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c1dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3866d9d640 1 -- 172.21.15.105:0/423644407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e4c1fa20 con 0x55d3e4c14da0 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f3865d9b640 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c1e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.312+0000 7f386659c640 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c1dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.313+0000 7f3866d9d640 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c1f430 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f386002c630 tx=0x7f3860009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf354ba58b20a14f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.313+0000 7f384dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.313+0000 7f384dffb640 1 -- 172.21.15.105:0/423644407 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860043040 con 0x55d3e4c14da0 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.313+0000 7f386f803740 1 -- 172.21.15.105:0/423644407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3860003970 con 0x55d3e4c14da0 2024-03-26T17:59:58.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.313+0000 7f384dffb640 1 -- 172.21.15.105:0/423644407 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f3860003e20 con 0x55d3e4c14da0 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.313+0000 7f384dffb640 1 -- 172.21.15.105:0/423644407 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3860039030 con 0x55d3e4c14da0 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.313+0000 7f386f803740 10 client.23268 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.313+0000 7f386f803740 1 -- 172.21.15.105:0/423644407 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d3e4c20d20 con 0x55d3e4c14da0 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f384dffb640 1 -- 172.21.15.105:0/423644407 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3860004140 con 0x55d3e4c14da0 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f384dffb640 1 -- 172.21.15.105:0/423644407 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f3860048570 con 0x55d3e4c14da0 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f386f803740 10 client.23268 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f386f803740 10 client.23268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f386f803740 20 client.23268 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f386f803740 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3e4c8bc40 0x55d3e4c8e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f386f803740 4 client.23268 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f386f803740 1 -- 172.21.15.105:0/423644407 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3860003970 con 0x55d3e4c8bc40 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f386659c640 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3e4c8bc40 0x55d3e4c8e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f384dffb640 10 client.23268 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:58.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.314+0000 7f386659c640 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3e4c8bc40 0x55d3e4c8e030 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b448a0cfc74bb23e server_cookie=22270c52da986dad in_seq=0 out_seq=0 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f384dffb640 1 -- 172.21.15.105:0/423644407 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3860003970 con 0x55d3e4c8bc40 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f384dffb640 10 client.23268 handle_command_reply: tid=0 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 client.23268 shutdown 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 -- 172.21.15.105:0/423644407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 msgr2=0x55d3e4c1f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c1f430 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f386002c630 tx=0x7f3860009970 comp rx=0 tx=0).stop 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 -- 172.21.15.105:0/423644407 shutdown_connections 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d3e4c8bc40 0x55d3e4c8e030 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d3e4c14da0 0x55d3e4c1f430 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d3e4c0fc30 0x55d3e4c1e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 --2- 172.21.15.105:0/423644407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d3e4c0f240 0x55d3e4c1dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f384dffb640 0 client.23268 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 -- 172.21.15.105:0/423644407 >> 172.21.15.105:0/423644407 conn(0x55d3e4c01fa0 msgr2=0x55d3e4c05200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 -- 172.21.15.105:0/423644407 shutdown_connections 2024-03-26T17:59:58.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 1 -- 172.21.15.105:0/423644407 wait complete. 2024-03-26T17:59:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.316+0000 7f386f803740 20 client.23268 trim_cache size 0 max 16384 2024-03-26T17:59:58.305 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T17:59:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 -- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38500a2fa0 msgr2=0x7f38500a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38500a2fa0 0x7f38500a5460 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f385c0095f0 tx=0x7f385c005f90 comp rx=0 tx=0).stop 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 -- 172.21.15.105:0/650635104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38681018f0 msgr2=0x7f3868075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38681018f0 0x7f3868075670 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f385800e9c0 tx=0x7f385800ee80 comp rx=0 tx=0).stop 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 -- 172.21.15.105:0/650635104 shutdown_connections 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38500a2fa0 0x7f38500a5460 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38681034a0 0x7f38680741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3868102ae0 0x7f3868073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 --2- 172.21.15.105:0/650635104 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38681018f0 0x7f3868075670 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 -- 172.21.15.105:0/650635104 >> 172.21.15.105:0/650635104 conn(0x7f38680fce60 msgr2=0x7f3868105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 -- 172.21.15.105:0/650635104 shutdown_connections 2024-03-26T17:59:58.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:58.317+0000 7f386cb3b640 1 -- 172.21.15.105:0/650635104 wait complete. 2024-03-26T17:59:58.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:57 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2943408781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:58.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:57 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2943408781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T17:59:58.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 68364 7.08 MB/sec execute 401 sec latency 159.859 ms 2024-03-26T17:59:59.316 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T17:59:59.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T17:59:59.316 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T17:59:59.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:58 smithi144 ceph-mon[36095]: pgmap v1000: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 4.3 MiB/s wr, 226 op/s 2024-03-26T17:59:59.345 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:58 smithi105 ceph-mon[32280]: pgmap v1000: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 4.3 MiB/s wr, 226 op/s 2024-03-26T17:59:59.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:58 smithi135 ceph-mon[35636]: pgmap v1000: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 4.3 MiB/s wr, 226 op/s 2024-03-26T17:59:59.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.468+0000 7f24f45d4640 1 -- 172.21.15.105:0/2043024919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 msgr2=0x7f24ec109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.468+0000 7f24f45d4640 1 --2- 172.21.15.105:0/2043024919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 0x7f24ec109d10 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009ca0 tx=0x7f24e402c0f0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.468+0000 7f24f45d4640 1 -- 172.21.15.105:0/2043024919 shutdown_connections 2024-03-26T17:59:59.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.468+0000 7f24f45d4640 1 --2- 172.21.15.105:0/2043024919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 0x7f24ec109d10 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.468+0000 7f24f45d4640 1 --2- 172.21.15.105:0/2043024919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24ec102ac0 0x7f24ec102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.468+0000 7f24f45d4640 1 --2- 172.21.15.105:0/2043024919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec1018d0 0x7f24ec101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.468+0000 7f24f45d4640 1 -- 172.21.15.105:0/2043024919 >> 172.21.15.105:0/2043024919 conn(0x7f24ec0fce20 msgr2=0x7f24ec0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.469+0000 7f24f45d4640 1 -- 172.21.15.105:0/2043024919 shutdown_connections 2024-03-26T17:59:59.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.469+0000 7f24f45d4640 1 -- 172.21.15.105:0/2043024919 wait complete. 2024-03-26T17:59:59.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.469+0000 7f24f45d4640 1 Processor -- start 2024-03-26T17:59:59.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.469+0000 7f24f45d4640 1 -- start start 2024-03-26T17:59:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.469+0000 7f24f45d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec1018d0 0x7f24ec197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f45d4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24ec102ac0 0x7f24ec198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f45d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 0x7f24ec19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f45d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec10d3e0 con 0x7f24ec103480 2024-03-26T17:59:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f45d4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec10d260 con 0x7f24ec102ac0 2024-03-26T17:59:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f45d4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec10d560 con 0x7f24ec1018d0 2024-03-26T17:59:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2370640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec1018d0 0x7f24ec197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2b71640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 0x7f24ec19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2370640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec1018d0 0x7f24ec197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35064/0 (socket says 172.21.15.105:35064) 2024-03-26T17:59:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2370640 1 -- 172.21.15.105:0/1802590976 learned_addr learned my addr 172.21.15.105:0/1802590976 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24ec102ac0 0x7f24ec198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2b71640 1 -- 172.21.15.105:0/1802590976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec1018d0 msgr2=0x7f24ec197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2b71640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec1018d0 0x7f24ec197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2b71640 1 -- 172.21.15.105:0/1802590976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24ec102ac0 msgr2=0x7f24ec198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2b71640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24ec102ac0 0x7f24ec198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2b71640 1 -- 172.21.15.105:0/1802590976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec19f700 con 0x7f24ec103480 2024-03-26T17:59:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24ec102ac0 0x7f24ec198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2370640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec1018d0 0x7f24ec197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.470+0000 7f24f2b71640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 0x7f24ec19f090 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f24e402c630 tx=0x7f24e40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.471+0000 7f24e37fe640 1 -- 172.21.15.105:0/1802590976 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4041020 con 0x7f24ec103480 2024-03-26T17:59:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.471+0000 7f24f45d4640 1 -- 172.21.15.105:0/1802590976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ec19f990 con 0x7f24ec103480 2024-03-26T17:59:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.471+0000 7f24f45d4640 1 -- 172.21.15.105:0/1802590976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ec19fed0 con 0x7f24ec103480 2024-03-26T17:59:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.471+0000 7f24e37fe640 1 -- 172.21.15.105:0/1802590976 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24e40098b0 con 0x7f24ec103480 2024-03-26T17:59:59.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.471+0000 7f24e37fe640 1 -- 172.21.15.105:0/1802590976 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4039040 con 0x7f24ec103480 2024-03-26T17:59:59.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.474+0000 7f24f71c3740 1 Processor -- start 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.474+0000 7f24e37fe640 1 -- 172.21.15.105:0/1802590976 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f24e4044020 con 0x7f24ec103480 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f71c3740 1 -- start start 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a18cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a18f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f71c3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a0e4630 con 0x55bf3a187f90 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f71c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a08ce10 con 0x55bf3a188980 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f71c3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a0e05e0 con 0x55bf3a18d580 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f1b6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f2370640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a18cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f2370640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a18cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36832/0 (socket says 172.21.15.105:36832) 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f2370640 1 -- 172.21.15.105:0/4171056075 learned_addr learned my addr 172.21.15.105:0/4171056075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a18f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24e37fe640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24d80a2fa0 0x7f24d80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f2370640 1 -- 172.21.15.105:0/4171056075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 msgr2=0x55bf3a18f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f2370640 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a18f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f2370640 1 -- 172.21.15.105:0/4171056075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 msgr2=0x55bf3a1883b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f2370640 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1883b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f2370640 1 -- 172.21.15.105:0/4171056075 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf3a0a3740 con 0x55bf3a188980 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.475+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a18f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.476+0000 7f24e37fe640 1 -- 172.21.15.105:0/1802590976 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f24e40f13a0 con 0x7f24ec103480 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.476+0000 7f24f2370640 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a18cf10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f24e80115c0 tx=0x7f24e8011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e268ddb29bc62779 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.476+0000 7f24f2b71640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24d80a2fa0 0x7f24d80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.476+0000 7f24e0bfe640 1 -- 172.21.15.105:0/4171056075 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8005020 con 0x55bf3a188980 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.476+0000 7f24e0bfe640 1 -- 172.21.15.105:0/4171056075 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24e80047a0 con 0x55bf3a188980 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.476+0000 7f24e0bfe640 1 -- 172.21.15.105:0/4171056075 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e800f920 con 0x55bf3a188980 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 -- 172.21.15.105:0/4171056075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 msgr2=0x55bf3a18cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f2b71640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24d80a2fa0 0x7f24d80a5460 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009ca0 tx=0x7f24e40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a18cf10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f24e80115c0 tx=0x7f24e8011a80 comp rx=0 tx=0).stop 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 -- 172.21.15.105:0/4171056075 shutdown_connections 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a18f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a18cf10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 --2- 172.21.15.105:0/4171056075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1883b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 -- 172.21.15.105:0/4171056075 >> 172.21.15.105:0/4171056075 conn(0x55bf3a17a680 msgr2=0x55bf3a17aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 -- 172.21.15.105:0/4171056075 shutdown_connections 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.477+0000 7f24f71c3740 1 -- 172.21.15.105:0/4171056075 wait complete. 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 1 -- start start 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 1 Processor -- start 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a191ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a192400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a199440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a190510 con 0x55bf3a187f90 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a0a01a0 con 0x55bf3a188980 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bf3a190690 con 0x55bf3a18d580 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a191ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f71c3740 10 client.0 _ll_register_callbacks cb 0x55bf39df7c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a191ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56006/0 (socket says 172.21.15.105:56006) 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 -- 172.21.15.105:0/2527823869 learned_addr learned my addr 172.21.15.105:0/2527823869 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f2370640 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a192400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 -- 172.21.15.105:0/2527823869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 msgr2=0x55bf3a199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 -- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 msgr2=0x55bf3a192400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a192400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f1b6f640 1 -- 172.21.15.105:0/2527823869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf39ec07e0 con 0x55bf3a187f90 2024-03-26T17:59:59.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.478+0000 7f24f2370640 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a192400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a191ec0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00f8c0 tx=0x7f24dc012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccaf23ceca5e44 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24d5ffb640 1 -- 172.21.15.105:0/2527823869 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00fe40 con 0x55bf3a187f90 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24d5ffb640 1 -- 172.21.15.105:0/2527823869 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f24dc004030 con 0x55bf3a187f90 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24f71c3740 1 -- 172.21.15.105:0/2527823869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e80047a0 con 0x55bf3a187f90 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24d5ffb640 1 -- 172.21.15.105:0/2527823869 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f24dc004230 con 0x55bf3a187f90 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24f71c3740 10 client.23280 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24f71c3740 1 -- 172.21.15.105:0/2527823869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bf39f20f70 con 0x55bf3a187f90 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24d5ffb640 1 -- 172.21.15.105:0/2527823869 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24e80047a0 con 0x55bf3a187f90 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.479+0000 7f24d5ffb640 1 -- 172.21.15.105:0/2527823869 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f24dc01f040 con 0x55bf3a187f90 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24f71c3740 10 client.23280 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24f71c3740 10 client.23280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24f71c3740 20 client.23280 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24f71c3740 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf3a19dd70 0x55bf3a1a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24f71c3740 4 client.23280 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24f71c3740 1 -- 172.21.15.105:0/2527823869 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf3a0f1b70 con 0x55bf3a19dd70 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24f2370640 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf3a19dd70 0x55bf3a1a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24f2370640 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf3a19dd70 0x55bf3a1a0210 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93230acd76aaaa5f server_cookie=28e057f71ddd7fa0 in_seq=0 out_seq=0 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.480+0000 7f24d5ffb640 10 client.23280 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24d5ffb640 1 -- 172.21.15.105:0/2527823869 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bf3a0f1b70 con 0x55bf3a19dd70 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24d5ffb640 10 client.23280 handle_command_reply: tid=0 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 client.23280 shutdown 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 -- 172.21.15.105:0/2527823869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 msgr2=0x55bf3a191ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a191ec0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00f8c0 tx=0x7f24dc012040 comp rx=0 tx=0).stop 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 -- 172.21.15.105:0/2527823869 shutdown_connections 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf3a19dd70 0x55bf3a1a0210 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a192400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 --2- 172.21.15.105:0/2527823869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a191ec0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 -- 172.21.15.105:0/2527823869 >> 172.21.15.105:0/2527823869 conn(0x55bf3a17a680 msgr2=0x55bf3a187620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24d5ffb640 0 client.23280 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 -- 172.21.15.105:0/2527823869 shutdown_connections 2024-03-26T17:59:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 1 -- 172.21.15.105:0/2527823869 wait complete. 2024-03-26T17:59:59.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.482+0000 7f24f71c3740 20 client.23280 trim_cache size 0 max 16384 2024-03-26T17:59:59.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.510+0000 7f24f71c3740 1 Processor -- start 2024-03-26T17:59:59.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.510+0000 7f24f71c3740 1 -- start start 2024-03-26T17:59:59.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.510+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a187f90 0x55bf3a1921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a188980 0x55bf3a1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a18d580 0x55bf3a179070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f71c3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d000e530 con 0x55bf3a18d580 2024-03-26T17:59:59.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f71c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0010070 con 0x55bf3a187f90 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f71c3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d000ec00 con 0x55bf3a188980 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f1b6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a18d580 0x55bf3a179070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f2370640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a188980 0x55bf3a1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f1b6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a18d580 0x55bf3a179070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56014/0 (socket says 172.21.15.105:56014) 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f1b6f640 1 -- 172.21.15.105:0/3551853424 learned_addr learned my addr 172.21.15.105:0/3551853424 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a187f90 0x55bf3a1921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f2370640 1 -- 172.21.15.105:0/3551853424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a187f90 msgr2=0x55bf3a1921e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f2370640 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a187f90 0x55bf3a1921e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f2370640 1 -- 172.21.15.105:0/3551853424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a18d580 msgr2=0x55bf3a179070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f2370640 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a18d580 0x55bf3a179070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f2370640 1 -- 172.21.15.105:0/3551853424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf39db8170 con 0x55bf3a188980 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a18d580 0x55bf3a179070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.511+0000 7f24f2370640 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a188980 0x55bf3a1993d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55bf3a17b5f0 tx=0x7f24e8009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7b3f8c1d25449c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.512+0000 7f24d7fff640 1 -- 172.21.15.105:0/3551853424 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e801d040 con 0x55bf3a188980 2024-03-26T17:59:59.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.512+0000 7f24d7fff640 1 -- 172.21.15.105:0/3551853424 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24e80044e0 con 0x55bf3a188980 2024-03-26T17:59:59.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.512+0000 7f24d7fff640 1 -- 172.21.15.105:0/3551853424 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8005470 con 0x55bf3a188980 2024-03-26T17:59:59.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- 172.21.15.105:0/3551853424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a188980 msgr2=0x55bf3a1993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a188980 0x55bf3a1993d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55bf3a17b5f0 tx=0x7f24e8009530 comp rx=0 tx=0).stop 2024-03-26T17:59:59.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- 172.21.15.105:0/3551853424 shutdown_connections 2024-03-26T17:59:59.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a18d580 0x55bf3a179070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a188980 0x55bf3a1993d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 --2- 172.21.15.105:0/3551853424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a187f90 0x55bf3a1921e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- 172.21.15.105:0/3551853424 >> 172.21.15.105:0/3551853424 conn(0x55bf3a17a680 msgr2=0x55bf3a191ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- 172.21.15.105:0/3551853424 shutdown_connections 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- 172.21.15.105:0/3551853424 wait complete. 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- start start 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 Processor -- start 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1966c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a196c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a197c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f24d000f9a0 con 0x55bf3a187f90 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24d000f2d0 con 0x55bf3a188980 2024-03-26T17:59:59.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.513+0000 7f24f71c3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f24d000c510 con 0x55bf3a18d580 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f71c3740 10 client.0 _ll_register_callbacks cb 0x7f24dc010530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56022/0 (socket says 172.21.15.105:56022) 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 -- 172.21.15.105:0/542405964 learned_addr learned my addr 172.21.15.105:0/542405964 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f2370640 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 -- 172.21.15.105:0/542405964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 msgr2=0x55bf3a197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 -- 172.21.15.105:0/542405964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 msgr2=0x55bf3a196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 -- 172.21.15.105:0/542405964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf3a198200 con 0x55bf3a187f90 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24f1b6f640 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1966c0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009b50 tx=0x7f24dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4278ad07776e2573 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24d5ffb640 1 -- 172.21.15.105:0/542405964 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc03f060 con 0x55bf3a187f90 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.514+0000 7f24d5ffb640 1 -- 172.21.15.105:0/542405964 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f24dc00ebc0 con 0x55bf3a187f90 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 1 -- 172.21.15.105:0/542405964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e80044e0 con 0x55bf3a187f90 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24d5ffb640 1 -- 172.21.15.105:0/542405964 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f24dc019040 con 0x55bf3a187f90 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 10 client.23286 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 1 -- 172.21.15.105:0/542405964 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bf3a199540 con 0x55bf3a187f90 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24d5ffb640 1 -- 172.21.15.105:0/542405964 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24e80044e0 con 0x55bf3a187f90 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24d5ffb640 1 -- 172.21.15.105:0/542405964 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f24dc004f80 con 0x55bf3a187f90 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 10 client.23286 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 10 client.23286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 20 client.23286 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf3a2046d0 0x55bf3a206af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 4 client.23286 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.515+0000 7f24f71c3740 1 -- 172.21.15.105:0/542405964 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bf3a0ebec0 con 0x55bf3a2046d0 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.516+0000 7f24f2370640 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf3a2046d0 0x55bf3a206af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.516+0000 7f24f2370640 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf3a2046d0 0x55bf3a206af0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8729225f1ea7b5f server_cookie=a2701b1dd24aa2d6 in_seq=0 out_seq=0 2024-03-26T17:59:59.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.516+0000 7f24d5ffb640 10 client.23286 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24d5ffb640 1 -- 172.21.15.105:0/542405964 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bf3a0ebec0 con 0x55bf3a2046d0 2024-03-26T17:59:59.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24d5ffb640 10 client.23286 handle_command_reply: tid=0 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24f71c3740 1 client.23286 shutdown 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24f71c3740 1 -- 172.21.15.105:0/542405964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 msgr2=0x55bf3a1966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24f71c3740 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1966c0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009b50 tx=0x7f24dc004730 comp rx=0 tx=0).stop 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24f71c3740 1 -- 172.21.15.105:0/542405964 shutdown_connections 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24f71c3740 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bf3a2046d0 0x55bf3a206af0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24f71c3740 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bf3a18d580 0x55bf3a197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24f71c3740 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bf3a188980 0x55bf3a196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.518+0000 7f24f71c3740 1 --2- 172.21.15.105:0/542405964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bf3a187f90 0x55bf3a1966c0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f71c3740 1 -- 172.21.15.105:0/542405964 >> 172.21.15.105:0/542405964 conn(0x55bf3a17a680 msgr2=0x55bf3a186d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24d5ffb640 0 client.23286 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f71c3740 1 -- 172.21.15.105:0/542405964 shutdown_connections 2024-03-26T17:59:59.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f71c3740 1 -- 172.21.15.105:0/542405964 wait complete. 2024-03-26T17:59:59.506 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:59:59.506 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T17:59:59.506 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "21ec3b76-3a67-41a7-a489-bdf495446b34", 2024-03-26T17:59:59.506 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T17:59:59.506 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f71c3740 20 client.23286 trim_cache size 0 max 16384 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 -- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24d80a2fa0 msgr2=0x7f24d80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24d80a2fa0 0x7f24d80a5460 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009ca0 tx=0x7f24e40095c0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 -- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 msgr2=0x7f24ec19f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 0x7f24ec19f090 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f24e402c630 tx=0x7f24e40328a0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 -- 172.21.15.105:0/1802590976 shutdown_connections 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f24d80a2fa0 0x7f24d80a5460 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24ec103480 0x7f24ec19f090 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24ec102ac0 0x7f24ec198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.519+0000 7f24f45d4640 1 --2- 172.21.15.105:0/1802590976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec1018d0 0x7f24ec197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.520+0000 7f24f45d4640 1 -- 172.21.15.105:0/1802590976 >> 172.21.15.105:0/1802590976 conn(0x7f24ec0fce20 msgr2=0x7f24ec105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.520+0000 7f24f45d4640 1 -- 172.21.15.105:0/1802590976 shutdown_connections 2024-03-26T17:59:59.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.520+0000 7f24f45d4640 1 -- 172.21.15.105:0/1802590976 wait complete. 2024-03-26T17:59:59.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T17:59:59.517 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T17:59:59.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 68464 7.08 MB/sec execute 402 sec latency 130.191 ms 2024-03-26T17:59:59.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 -- 172.21.15.105:0/1278031630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe778101170 msgr2=0x7fe7781015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 --2- 172.21.15.105:0/1278031630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe778101170 0x7fe7781015f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009900 tx=0x7fe76c02bc30 comp rx=0 tx=0).stop 2024-03-26T17:59:59.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 -- 172.21.15.105:0/1278031630 shutdown_connections 2024-03-26T17:59:59.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 --2- 172.21.15.105:0/1278031630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe778101b30 0x7fe7781083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 --2- 172.21.15.105:0/1278031630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe778101170 0x7fe7781015f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 --2- 172.21.15.105:0/1278031630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7780fff80 0x7fe7781003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 -- 172.21.15.105:0/1278031630 >> 172.21.15.105:0/1278031630 conn(0x7fe7780fb4f0 msgr2=0x7fe7780fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 -- 172.21.15.105:0/1278031630 shutdown_connections 2024-03-26T17:59:59.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.708+0000 7fe77cd22640 1 -- 172.21.15.105:0/1278031630 wait complete. 2024-03-26T17:59:59.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.709+0000 7fe77cd22640 1 Processor -- start 2024-03-26T17:59:59.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.709+0000 7fe77cd22640 1 -- start start 2024-03-26T17:59:59.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.709+0000 7fe77cd22640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7780fff80 0x7fe778199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.709+0000 7fe77cd22640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe778101170 0x7fe77819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.709+0000 7fe77cd22640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101b30 0x7fe7781a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.709+0000 7fe77cd22640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe778109b90 con 0x7fe778101170 2024-03-26T17:59:59.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.709+0000 7fe77cd22640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe778109a10 con 0x7fe7780fff80 2024-03-26T17:59:59.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.709+0000 7fe77cd22640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe778109d10 con 0x7fe778101b30 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe778101170 0x7fe77819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe77659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7780fff80 0x7fe778199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe778101170 0x7fe77819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56028/0 (socket says 172.21.15.105:56028) 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 -- 172.21.15.105:0/2255679994 learned_addr learned my addr 172.21.15.105:0/2255679994 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe776d9d640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101b30 0x7fe7781a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 -- 172.21.15.105:0/2255679994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101b30 msgr2=0x7fe7781a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101b30 0x7fe7781a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 -- 172.21.15.105:0/2255679994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7780fff80 msgr2=0x7fe778199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7780fff80 0x7fe778199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 -- 172.21.15.105:0/2255679994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7781a1980 con 0x7fe778101170 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe776d9d640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101b30 0x7fe7781a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe77659c640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7780fff80 0x7fe778199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.710+0000 7fe775d9b640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe778101170 0x7fe77819a2a0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fe76c000c00 tx=0x7fe76c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.711+0000 7fe7677fe640 1 -- 172.21.15.105:0/2255679994 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c043020 con 0x7fe778101170 2024-03-26T17:59:59.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.711+0000 7fe7677fe640 1 -- 172.21.15.105:0/2255679994 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe76c0044c0 con 0x7fe778101170 2024-03-26T17:59:59.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.711+0000 7fe7677fe640 1 -- 172.21.15.105:0/2255679994 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c003c60 con 0x7fe778101170 2024-03-26T17:59:59.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.711+0000 7fe77cd22640 1 -- 172.21.15.105:0/2255679994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7781a1c10 con 0x7fe778101170 2024-03-26T17:59:59.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.711+0000 7fe77cd22640 1 -- 172.21.15.105:0/2255679994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7781a2110 con 0x7fe778101170 2024-03-26T17:59:59.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.714+0000 7fe7677fe640 1 -- 172.21.15.105:0/2255679994 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe76c046020 con 0x7fe778101170 2024-03-26T17:59:59.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.714+0000 7fe77f9ef740 1 Processor -- start 2024-03-26T17:59:59.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.714+0000 7fe77f9ef740 1 -- start start 2024-03-26T17:59:59.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.714+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e3cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.714+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e41de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.714+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e42450 0x563511e448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563511da1980 con 0x563511e3c8f0 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563511d49be0 con 0x563511e3d2e0 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563511d9d930 con 0x563511e42450 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e3cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe77659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e41de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e3cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56036/0 (socket says 172.21.15.105:56036) 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 -- 172.21.15.105:0/846439788 learned_addr learned my addr 172.21.15.105:0/846439788 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe775d9b640 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e42450 0x563511e448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe7677fe640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a2fa0 0x7fe7600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 -- 172.21.15.105:0/846439788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e42450 msgr2=0x563511e448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e42450 0x563511e448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 -- 172.21.15.105:0/846439788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 msgr2=0x563511e41de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e41de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 -- 172.21.15.105:0/846439788 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563511b7f7d0 con 0x563511e3c8f0 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe775d9b640 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e42450 0x563511e448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe77659c640 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e41de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe7677fe640 1 -- 172.21.15.105:0/2255679994 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fe76c0f1580 con 0x7fe778101170 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.715+0000 7fe776d9d640 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e3cd10 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fe7700117e0 tx=0x7fe770011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6bd7e210a769ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.716+0000 7fe776d9d640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a2fa0 0x7fe7600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.716+0000 7fe776d9d640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a2fa0 0x7fe7600a5460 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe77001b000 tx=0x7fe770010710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.716+0000 7fe764ff9640 1 -- 172.21.15.105:0/846439788 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770003a10 con 0x563511e3c8f0 2024-03-26T17:59:59.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.716+0000 7fe764ff9640 1 -- 172.21.15.105:0/846439788 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe77000fe50 con 0x563511e3c8f0 2024-03-26T17:59:59.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.716+0000 7fe764ff9640 1 -- 172.21.15.105:0/846439788 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770010430 con 0x563511e3c8f0 2024-03-26T17:59:59.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.717+0000 7fe77f9ef740 1 -- 172.21.15.105:0/846439788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 msgr2=0x563511e3cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.717+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e3cd10 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fe7700117e0 tx=0x7fe770011ca0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.705 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.717+0000 7fe77f9ef740 1 -- 172.21.15.105:0/846439788 shutdown_connections 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.717+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e42450 0x563511e448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.717+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e41de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.717+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/846439788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e3cd10 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.717+0000 7fe77f9ef740 1 -- 172.21.15.105:0/846439788 >> 172.21.15.105:0/846439788 conn(0x563511e37e60 msgr2=0x563511e38260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.718+0000 7fe77f9ef740 1 -- 172.21.15.105:0/846439788 shutdown_connections 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.718+0000 7fe77f9ef740 1 -- 172.21.15.105:0/846439788 wait complete. 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.718+0000 7fe77f9ef740 1 -- start start 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.718+0000 7fe77f9ef740 1 Processor -- start 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.718+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e46ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.718+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.718+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e42450 0x563511e4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563511e45400 con 0x563511e3c8f0 2024-03-26T17:59:59.706 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563511d5d160 con 0x563511e42450 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563511e45580 con 0x563511e3d2e0 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe77f9ef740 10 client.0 _ll_register_callbacks cb 0x563511d99720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe776d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e42450 0x563511e4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35158/0 (socket says 172.21.15.105:35158) 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 -- 172.21.15.105:0/679692267 learned_addr learned my addr 172.21.15.105:0/679692267 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe776d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e42450 0x563511e4e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36888/0 (socket says 172.21.15.105:36888) 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe77659c640 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e46ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 -- 172.21.15.105:0/679692267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e42450 msgr2=0x563511e4e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e42450 0x563511e4e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 -- 172.21.15.105:0/679692267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 msgr2=0x563511e46ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e46ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 -- 172.21.15.105:0/679692267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563511ab53f0 con 0x563511e3d2e0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe77659c640 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e46ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe775d9b640 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e470e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe76c02c210 tx=0x7fe76c009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e2454192db502a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe75dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.719+0000 7fe75dffb640 1 -- 172.21.15.105:0/679692267 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0bd5d0 con 0x563511e3d2e0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.720+0000 7fe77f9ef740 1 -- 172.21.15.105:0/679692267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77000fe50 con 0x563511e3d2e0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.720+0000 7fe75dffb640 1 -- 172.21.15.105:0/679692267 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fe76c0bbf70 con 0x563511e3d2e0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.720+0000 7fe75dffb640 1 -- 172.21.15.105:0/679692267 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe76c0035f0 con 0x563511e3d2e0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.720+0000 7fe77f9ef740 10 client.32968 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.720+0000 7fe77f9ef740 1 -- 172.21.15.105:0/679692267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563511a750f0 con 0x563511e3d2e0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.720+0000 7fe75dffb640 1 -- 172.21.15.105:0/679692267 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe77000fe50 con 0x563511e3d2e0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.721+0000 7fe75dffb640 1 -- 172.21.15.105:0/679692267 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fe76c0c61f0 con 0x563511e3d2e0 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.721+0000 7fe77f9ef740 10 client.32968 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.721+0000 7fe77f9ef740 10 client.32968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.721+0000 7fe77f9ef740 20 client.32968 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:59.708 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.721+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563511e529d0 0x563511e54e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.721+0000 7fe77f9ef740 4 client.32968 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T17:59:59.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.721+0000 7fe77f9ef740 1 -- 172.21.15.105:0/679692267 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563511daeec0 con 0x563511e529d0 2024-03-26T17:59:59.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.722+0000 7fe77659c640 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563511e529d0 0x563511e54e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.722+0000 7fe75dffb640 10 client.32968 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:59.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.722+0000 7fe77659c640 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563511e529d0 0x563511e54e70 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c2dd28cd7fdc4c9 server_cookie=f2f12d5923ef720f in_seq=0 out_seq=0 2024-03-26T17:59:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe75dffb640 1 -- 172.21.15.105:0/679692267 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563511daeec0 con 0x563511e529d0 2024-03-26T17:59:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe75dffb640 10 client.32968 handle_command_reply: tid=0 2024-03-26T17:59:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 client.32968 shutdown 2024-03-26T17:59:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 -- 172.21.15.105:0/679692267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 msgr2=0x563511e470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e470e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe76c02c210 tx=0x7fe76c009620 comp rx=0 tx=0).stop 2024-03-26T17:59:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 -- 172.21.15.105:0/679692267 shutdown_connections 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563511e529d0 0x563511e54e70 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e42450 0x563511e4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e470e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe75dffb640 0 client.32968 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/679692267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e3c8f0 0x563511e46ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 -- 172.21.15.105:0/679692267 >> 172.21.15.105:0/679692267 conn(0x563511e37e60 msgr2=0x563511e37080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 -- 172.21.15.105:0/679692267 shutdown_connections 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.726+0000 7fe77f9ef740 1 -- 172.21.15.105:0/679692267 wait complete. 2024-03-26T17:59:59.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.727+0000 7fe77f9ef740 20 client.32968 trim_cache size 0 max 16384 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.750+0000 7fe77f9ef740 1 Processor -- start 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.750+0000 7fe77f9ef740 1 -- start start 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.750+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3c8f0 0x563511e367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.750+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e4e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.750+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e36d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.750+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781ab080 con 0x563511e42450 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.750+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781acbc0 con 0x563511e3c8f0 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.750+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781ab750 con 0x563511e3d2e0 2024-03-26T17:59:59.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe77659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3c8f0 0x563511e367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe776d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e36d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe775d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e4e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe77659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3c8f0 0x563511e367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36904/0 (socket says 172.21.15.105:36904) 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe775d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e4e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35166/0 (socket says 172.21.15.105:35166) 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe77659c640 1 -- 172.21.15.105:0/1287544547 learned_addr learned my addr 172.21.15.105:0/1287544547 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe776d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e36d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:56046/0 (socket says 172.21.15.105:56046) 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe775d9b640 1 -- 172.21.15.105:0/1287544547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3c8f0 msgr2=0x563511e367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe775d9b640 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3c8f0 0x563511e367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe775d9b640 1 -- 172.21.15.105:0/1287544547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 msgr2=0x563511e36d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe775d9b640 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e36d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe775d9b640 1 -- 172.21.15.105:0/1287544547 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563511e554f0 con 0x563511e3d2e0 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe77659c640 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3c8f0 0x563511e367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe776d9d640 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e36d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T17:59:59.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe775d9b640 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e4e0e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0ba0d0 tx=0x7fe76c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ff7c33c66bfa50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.751+0000 7fe75ffff640 1 -- 172.21.15.105:0/1287544547 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0bd5d0 con 0x563511e3d2e0 2024-03-26T17:59:59.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe75ffff640 1 -- 172.21.15.105:0/1287544547 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe76c0be370 con 0x563511e3d2e0 2024-03-26T17:59:59.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe75ffff640 1 -- 172.21.15.105:0/1287544547 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c0bbf70 con 0x563511e3d2e0 2024-03-26T17:59:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 -- 172.21.15.105:0/1287544547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 msgr2=0x563511e4e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e4e0e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0ba0d0 tx=0x7fe76c005fd0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 -- 172.21.15.105:0/1287544547 shutdown_connections 2024-03-26T17:59:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e36d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3d2e0 0x563511e4e0e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/1287544547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3c8f0 0x563511e367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 -- 172.21.15.105:0/1287544547 >> 172.21.15.105:0/1287544547 conn(0x563511e37e60 msgr2=0x563511e41b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 -- 172.21.15.105:0/1287544547 shutdown_connections 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.752+0000 7fe77f9ef740 1 -- 172.21.15.105:0/1287544547 wait complete. 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 1 -- start start 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 1 Processor -- start 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3c8f0 0x563511e4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e4c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e4d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781ac4f0 con 0x563511e42450 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781abe20 con 0x563511e3d2e0 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7781a9060 con 0x563511e3c8f0 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77f9ef740 10 client.0 _ll_register_callbacks cb 0x7fe76c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3c8f0 0x563511e4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe776d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e4d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe775d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e4c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3c8f0 0x563511e4bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35172/0 (socket says 172.21.15.105:35172) 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe775d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e4c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:36916/0 (socket says 172.21.15.105:36916) 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77659c640 1 -- 172.21.15.105:0/3790700473 learned_addr learned my addr 172.21.15.105:0/3790700473 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77659c640 1 -- 172.21.15.105:0/3790700473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 msgr2=0x563511e4c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77659c640 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e4c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77659c640 1 -- 172.21.15.105:0/3790700473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 msgr2=0x563511e4d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77659c640 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e4d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.753+0000 7fe77659c640 1 -- 172.21.15.105:0/3790700473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563511e4db20 con 0x563511e3c8f0 2024-03-26T17:59:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe77659c640 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3c8f0 0x563511e4bfe0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x563511e3d6e0 tx=0x7fe768010ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd817e822b2e7add server_cookie=0 in_seq=0 out_seq=0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe75dffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe75dffb640 1 -- 172.21.15.105:0/3790700473 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768026060 con 0x563511e3c8f0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe75dffb640 1 -- 172.21.15.105:0/3790700473 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7fe76800f930 con 0x563511e3c8f0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe77f9ef740 1 -- 172.21.15.105:0/3790700473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe76c0be370 con 0x563511e3c8f0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe75dffb640 1 -- 172.21.15.105:0/3790700473 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe76800fb30 con 0x563511e3c8f0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe77f9ef740 10 client.32974 fetch_fsmap learned FSMap version 41 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe77f9ef740 1 -- 172.21.15.105:0/3790700473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563511eb9690 con 0x563511e3c8f0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.754+0000 7fe75dffb640 1 -- 172.21.15.105:0/3790700473 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe76c0be370 con 0x563511e3c8f0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.755+0000 7fe75dffb640 1 -- 172.21.15.105:0/3790700473 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7fe768004960 con 0x563511e3c8f0 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.755+0000 7fe77f9ef740 10 client.32974 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.755+0000 7fe77f9ef740 10 client.32974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.755+0000 7fe77f9ef740 20 client.32974 populate_metadata read hostname 'smithi105' 2024-03-26T17:59:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.755+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563511eba020 0x563511ebc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T17:59:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.755+0000 7fe77f9ef740 4 client.32974 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T17:59:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.755+0000 7fe77f9ef740 1 -- 172.21.15.105:0/3790700473 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563511e44d60 con 0x563511eba020 2024-03-26T17:59:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.755+0000 7fe775d9b640 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563511eba020 0x563511ebc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T17:59:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.756+0000 7fe775d9b640 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563511eba020 0x563511ebc440 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969683a58ce22459 server_cookie=4f09f1feefb6c862 in_seq=0 out_seq=0 2024-03-26T17:59:59.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.756+0000 7fe75dffb640 10 client.32974 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.758+0000 7fe75dffb640 1 -- 172.21.15.105:0/3790700473 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x563511e44d60 con 0x563511eba020 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.758+0000 7fe75dffb640 10 client.32974 handle_command_reply: tid=0 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.758+0000 7fe77f9ef740 1 client.32974 shutdown 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.758+0000 7fe77f9ef740 1 -- 172.21.15.105:0/3790700473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3c8f0 msgr2=0x563511e4bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.758+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3c8f0 0x563511e4bfe0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x563511e3d6e0 tx=0x7fe768010ec0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 1 -- 172.21.15.105:0/3790700473 shutdown_connections 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563511eba020 0x563511ebc440 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563511e42450 0x563511e4d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe75dffb640 0 client.32974 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563511e3d2e0 0x563511e4c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 1 --2- 172.21.15.105:0/3790700473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563511e3c8f0 0x563511e4bfe0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 1 -- 172.21.15.105:0/3790700473 >> 172.21.15.105:0/3790700473 conn(0x563511e37e60 msgr2=0x563511e3b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 1 -- 172.21.15.105:0/3790700473 shutdown_connections 2024-03-26T17:59:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 1 -- 172.21.15.105:0/3790700473 wait complete. 2024-03-26T17:59:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.759+0000 7fe77f9ef740 20 client.32974 trim_cache size 0 max 16384 2024-03-26T17:59:59.747 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T17:59:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (311 inodes in the stack)", 2024-03-26T17:59:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T17:59:59.747 INFO:teuthology.orchestra.run.smithi105.stdout: "21ec3b76-3a67-41a7-a489-bdf495446b34": { 2024-03-26T17:59:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T17:59:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "21ec3b76-3a67-41a7-a489-bdf495446b34", 2024-03-26T17:59:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T17:59:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:59:59.748 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T17:59:59.748 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 -- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a2fa0 msgr2=0x7fe7600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a2fa0 0x7fe7600a5460 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe77001b000 tx=0x7fe770010710 comp rx=0 tx=0).stop 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 -- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe778101170 msgr2=0x7fe77819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe778101170 0x7fe77819a2a0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fe76c000c00 tx=0x7fe76c0040c0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 -- 172.21.15.105:0/2255679994 shutdown_connections 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a2fa0 0x7fe7600a5460 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101b30 0x7fe7781a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe778101170 0x7fe77819a2a0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 --2- 172.21.15.105:0/2255679994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7780fff80 0x7fe778199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 -- 172.21.15.105:0/2255679994 >> 172.21.15.105:0/2255679994 conn(0x7fe7780fb4f0 msgr2=0x7fe7780fbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T17:59:59.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 -- 172.21.15.105:0/2255679994 shutdown_connections 2024-03-26T17:59:59.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T17:59:59.760+0000 7fe77cd22640 1 -- 172.21.15.105:0/2255679994 wait complete. 2024-03-26T17:59:59.756 INFO:tasks.cephfs.filesystem:scrub status for tag:21ec3b76-3a67-41a7-a489-bdf495446b34 - {'path': '/', 'tag': '21ec3b76-3a67-41a7-a489-bdf495446b34', 'options': 'recursive,force'} 2024-03-26T17:59:59.906 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T17:59:59.588+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003c80 [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{1=1,2=1,3=1,4=1} v13487 ap=2 RANDEPHEMERALPIN f(v364 m2024-03-26T17:59:38.831205+0000 11=11+0) n(v28 rc2024-03-26T17:59:38.831205+0000 b2861056 12=11+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@7151} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d142580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003c80:[<0x10000003bc0/COREL v50>,<0x10000003bb6/~dmtmp v48>,<0x10000000007/client8 v2078>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003c80:[<0x10000003bc0/COREL v13487>,<0x10000003bb6/~dmtmp v2779>,<0x10000000007/client8 v11398>,<0x10000000006/clients v3078>,<0x10000000005/tmp v2777>,<0x10000000003/client.0 v2667>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v2599>,<0x10000000001/sv_0 v2609>,<0x10000000000/qa v2529>,<0x1/volumes v2494>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T17:59:38.793224+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T17:59:38.831205+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v364 m2024-03-26T17:59:38.831205+0000 11=11+0)","memory_value.rstat":"n(v28 rc2024-03-26T17:59:38.831205+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T18:00:00.132 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T18:00:00.132 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.132 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:00.133 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 17:59:59 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:00.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T18:00:00.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:00.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 17:59:59 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:00.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T18:00:00.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:00.407 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 17:59:59 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:00.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 68569 7.08 MB/sec execute 403 sec latency 209.756 ms 2024-03-26T18:00:01.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:01.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:01.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:01.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: overall HEALTH_OK 2024-03-26T18:00:01.340 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:01 smithi144 ceph-mon[36095]: pgmap v1001: 97 pgs: 97 active+clean; 351 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-03-26T18:00:01.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:01.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:01.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:01.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: overall HEALTH_OK 2024-03-26T18:00:01.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:01 smithi105 ceph-mon[32280]: pgmap v1001: 97 pgs: 97 active+clean; 351 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-03-26T18:00:01.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:01.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:01.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:01.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:01.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: overall HEALTH_OK 2024-03-26T18:00:01.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:01 smithi135 ceph-mon[35636]: pgmap v1001: 97 pgs: 97 active+clean; 351 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 3.3 MiB/s wr, 220 op/s 2024-03-26T18:00:01.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 68697 7.08 MB/sec execute 404 sec latency 158.647 ms 2024-03-26T18:00:02.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 68817 7.07 MB/sec execute 405 sec latency 162.251 ms 2024-03-26T18:00:03.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:02 smithi144 ceph-mon[36095]: pgmap v1002: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-03-26T18:00:03.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:02 smithi105 ceph-mon[32280]: pgmap v1002: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-03-26T18:00:03.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:02 smithi135 ceph-mon[35636]: pgmap v1002: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 3.5 MiB/s wr, 233 op/s 2024-03-26T18:00:03.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 68932 7.07 MB/sec execute 406 sec latency 200.114 ms 2024-03-26T18:00:04.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 69016 7.05 MB/sec execute 407 sec latency 158.421 ms 2024-03-26T18:00:05.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:04 smithi144 ceph-mon[36095]: pgmap v1003: 97 pgs: 97 active+clean; 356 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-03-26T18:00:05.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:04 smithi105 ceph-mon[32280]: pgmap v1003: 97 pgs: 97 active+clean; 356 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-03-26T18:00:05.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:04 smithi135 ceph-mon[35636]: pgmap v1003: 97 pgs: 97 active+clean; 356 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 3.7 MiB/s wr, 276 op/s 2024-03-26T18:00:05.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 69128 7.05 MB/sec execute 408 sec latency 143.721 ms 2024-03-26T18:00:06.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 69263 7.05 MB/sec execute 409 sec latency 127.579 ms 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 -- 172.21.15.105:0/2628619620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1019d0 msgr2=0x7f4dac101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 --2- 172.21.15.105:0/2628619620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1019d0 0x7f4dac101e50 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009900 tx=0x7f4d9c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 -- 172.21.15.105:0/2628619620 shutdown_connections 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 --2- 172.21.15.105:0/2628619620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dac102390 0x7f4dac108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 --2- 172.21.15.105:0/2628619620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1019d0 0x7f4dac101e50 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 --2- 172.21.15.105:0/2628619620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4dac1007e0 0x7f4dac100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 -- 172.21.15.105:0/2628619620 >> 172.21.15.105:0/2628619620 conn(0x7f4dac0fbd50 msgr2=0x7f4dac0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 -- 172.21.15.105:0/2628619620 shutdown_connections 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 -- 172.21.15.105:0/2628619620 wait complete. 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 Processor -- start 2024-03-26T18:00:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.845+0000 7f4db4116640 1 -- start start 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db4116640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1007e0 0x7f4dac197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db4116640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dac1019d0 0x7f4dac197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db4116640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4dac102390 0x7f4dac19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db4116640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac10d430 con 0x7f4dac1007e0 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db4116640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac10d2b0 con 0x7f4dac102390 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db4116640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac10d5b0 con 0x7f4dac1019d0 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1007e0 0x7f4dac197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1007e0 0x7f4dac197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45650/0 (socket says 172.21.15.105:45650) 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 -- 172.21.15.105:0/3980239039 learned_addr learned my addr 172.21.15.105:0/3980239039 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db16b1640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dac1019d0 0x7f4dac197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db26b3640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4dac102390 0x7f4dac19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 -- 172.21.15.105:0/3980239039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dac1019d0 msgr2=0x7f4dac197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dac1019d0 0x7f4dac197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 -- 172.21.15.105:0/3980239039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4dac102390 msgr2=0x7f4dac19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4dac102390 0x7f4dac19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 -- 172.21.15.105:0/3980239039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac19f6a0 con 0x7f4dac1007e0 2024-03-26T18:00:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.846+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1007e0 0x7f4dac197a80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4da800e9c0 tx=0x7f4da800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.847+0000 7f4da2ffd640 1 -- 172.21.15.105:0/3980239039 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da800cd80 con 0x7f4dac1007e0 2024-03-26T18:00:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.847+0000 7f4da2ffd640 1 -- 172.21.15.105:0/3980239039 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da80035f0 con 0x7f4dac1007e0 2024-03-26T18:00:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.847+0000 7f4da2ffd640 1 -- 172.21.15.105:0/3980239039 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8010690 con 0x7f4dac1007e0 2024-03-26T18:00:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.847+0000 7f4db4116640 1 -- 172.21.15.105:0/3980239039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dac19f990 con 0x7f4dac1007e0 2024-03-26T18:00:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.847+0000 7f4db4116640 1 -- 172.21.15.105:0/3980239039 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dac19feb0 con 0x7f4dac1007e0 2024-03-26T18:00:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.850+0000 7f4da2ffd640 1 -- 172.21.15.105:0/3980239039 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4da80037a0 con 0x7f4dac1007e0 2024-03-26T18:00:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db6dd6740 1 Processor -- start 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db6dd6740 1 -- start start 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 0x55b37db4eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db4f130 0x55b37db515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b37daae0b0 con 0x55b37db495d0 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b37da56960 con 0x55b37db4f130 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b37daaa060 con 0x55b37db49fc0 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4da2ffd640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d980a2fc0 0x7f4d980a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db26b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db16b1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db4f130 0x55b37db515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db26b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45652/0 (socket says 172.21.15.105:45652) 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db26b3640 1 -- 172.21.15.105:0/792611160 learned_addr learned my addr 172.21.15.105:0/792611160 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.851+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 0x55b37db4eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db26b3640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d980a2fc0 0x7f4d980a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db16b1640 1 -- 172.21.15.105:0/792611160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 msgr2=0x55b37db4eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db16b1640 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 0x55b37db4eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db16b1640 1 -- 172.21.15.105:0/792611160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 msgr2=0x55b37db499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db16b1640 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db16b1640 1 -- 172.21.15.105:0/792611160 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37da68c60 con 0x55b37db4f130 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4da2ffd640 1 -- 172.21.15.105:0/3980239039 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f4da80cf7d0 con 0x7f4dac1007e0 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db26b3640 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db16b1640 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db4f130 0x55b37db515c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c000c00 tx=0x7f4d9c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232171bdc4df809c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4db26b3640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d980a2fc0 0x7f4d980a5480 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x55b37db4a820 tx=0x7f4da4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4d97fff640 1 -- 172.21.15.105:0/792611160 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c045020 con 0x55b37db4f130 2024-03-26T18:00:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.852+0000 7f4d97fff640 1 -- 172.21.15.105:0/792611160 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d9c004040 con 0x55b37db4f130 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.853+0000 7f4d97fff640 1 -- 172.21.15.105:0/792611160 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0055a0 con 0x55b37db4f130 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.853+0000 7f4db6dd6740 1 -- 172.21.15.105:0/792611160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db4f130 msgr2=0x55b37db515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.853+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db4f130 0x55b37db515c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c000c00 tx=0x7f4d9c009560 comp rx=0 tx=0).stop 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.853+0000 7f4db6dd6740 1 -- 172.21.15.105:0/792611160 shutdown_connections 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.853+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db4f130 0x55b37db515c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.853+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 0x55b37db4eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.853+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/792611160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.853+0000 7f4db6dd6740 1 -- 172.21.15.105:0/792611160 >> 172.21.15.105:0/792611160 conn(0x55b37db44b40 msgr2=0x55b37db44f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.854+0000 7f4db6dd6740 1 -- 172.21.15.105:0/792611160 shutdown_connections 2024-03-26T18:00:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.854+0000 7f4db6dd6740 1 -- 172.21.15.105:0/792611160 wait complete. 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.854+0000 7f4db6dd6740 1 -- start start 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.854+0000 7f4db6dd6740 1 Processor -- start 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db4f130 0x55b37db5a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b37db520d0 con 0x55b37db495d0 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b37da69db0 con 0x55b37db49fc0 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b37db52250 con 0x55b37db4f130 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db16b1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db16b1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38278/0 (socket says 172.21.15.105:38278) 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db16b1640 1 -- 172.21.15.105:0/2515935387 learned_addr learned my addr 172.21.15.105:0/2515935387 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db6dd6740 10 client.0 _ll_register_callbacks cb 0x55b37da6e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db26b3640 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db4f130 0x55b37db5a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db26b3640 1 -- 172.21.15.105:0/2515935387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 msgr2=0x55b37db53830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db26b3640 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db26b3640 1 -- 172.21.15.105:0/2515935387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 msgr2=0x55b37db532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db26b3640 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db26b3640 1 -- 172.21.15.105:0/2515935387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b37d7c2210 con 0x55b37db4f130 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.855+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4db26b3640 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db4f130 0x55b37db5a870 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009fd0 tx=0x7f4da4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93e06c651659676c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4d957fa640 1 -- 172.21.15.105:0/2515935387 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4026070 con 0x55b37db4f130 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4db6dd6740 1 -- 172.21.15.105:0/2515935387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d9c004040 con 0x55b37db4f130 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4d957fa640 1 -- 172.21.15.105:0/2515935387 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f4da4010d10 con 0x55b37db4f130 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4d957fa640 1 -- 172.21.15.105:0/2515935387 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4da4031050 con 0x55b37db4f130 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4db6dd6740 10 client.32986 fetch_fsmap learned FSMap version 41 2024-03-26T18:00:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4db6dd6740 1 -- 172.21.15.105:0/2515935387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b37d88e620 con 0x55b37db4f130 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.856+0000 7f4d957fa640 1 -- 172.21.15.105:0/2515935387 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d9c004040 con 0x55b37db4f130 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4d957fa640 1 -- 172.21.15.105:0/2515935387 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f4da401f050 con 0x55b37db4f130 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4db6dd6740 10 client.32986 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4db6dd6740 10 client.32986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4db6dd6740 20 client.32986 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b37db5f1c0 0x55b37db61650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4db6dd6740 4 client.32986 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4db6dd6740 1 -- 172.21.15.105:0/2515935387 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b37dabb5f0 con 0x55b37db5f1c0 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b37db5f1c0 0x55b37db61650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4d957fa640 10 client.32986 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.857+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b37db5f1c0 0x55b37db61650 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d5e93bc62f089c4 server_cookie=4876af4b2e035d9b in_seq=0 out_seq=0 2024-03-26T18:00:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4d957fa640 1 -- 172.21.15.105:0/2515935387 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b37dabb5f0 con 0x55b37db5f1c0 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4d957fa640 10 client.32986 handle_command_reply: tid=0 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 client.32986 shutdown 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 -- 172.21.15.105:0/2515935387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db4f130 msgr2=0x55b37db5a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db4f130 0x55b37db5a870 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009fd0 tx=0x7f4da4004a40 comp rx=0 tx=0).stop 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 -- 172.21.15.105:0/2515935387 shutdown_connections 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b37db5f1c0 0x55b37db61650 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db4f130 0x55b37db5a870 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4d957fa640 0 client.32986 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/2515935387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 -- 172.21.15.105:0/2515935387 >> 172.21.15.105:0/2515935387 conn(0x55b37db44b40 msgr2=0x55b37db43df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 -- 172.21.15.105:0/2515935387 shutdown_connections 2024-03-26T18:00:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 1 -- 172.21.15.105:0/2515935387 wait complete. 2024-03-26T18:00:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.861+0000 7f4db6dd6740 20 client.32986 trim_cache size 0 max 16384 2024-03-26T18:00:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db6dd6740 1 Processor -- start 2024-03-26T18:00:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db6dd6740 1 -- start start 2024-03-26T18:00:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db4e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1a8d10 con 0x55b37db4f130 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1aa850 con 0x55b37db495d0 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1a93e0 con 0x55b37db49fc0 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db26b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db4e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db16b1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db26b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db4e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45668/0 (socket says 172.21.15.105:45668) 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.884+0000 7f4db26b3640 1 -- 172.21.15.105:0/3112143280 learned_addr learned my addr 172.21.15.105:0/3112143280 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/3112143280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4db26b3640 1 -- 172.21.15.105:0/3112143280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 msgr2=0x55b37db53830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4db26b3640 1 --2- 172.21.15.105:0/3112143280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4db26b3640 1 -- 172.21.15.105:0/3112143280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db495d0 msgr2=0x55b37db532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4db26b3640 1 --2- 172.21.15.105:0/3112143280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4db26b3640 1 -- 172.21.15.105:0/3112143280 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37db4a3a0 con 0x55b37db4f130 2024-03-26T18:00:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4db26b3640 1 --2- 172.21.15.105:0/3112143280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db4e2e0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4da4005c40 tx=0x7f4da4009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c79b19dc112f999 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4d977fe640 1 -- 172.21.15.105:0/3112143280 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da403a070 con 0x55b37db4f130 2024-03-26T18:00:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4d977fe640 1 -- 172.21.15.105:0/3112143280 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da4010d10 con 0x55b37db4f130 2024-03-26T18:00:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.885+0000 7f4d977fe640 1 -- 172.21.15.105:0/3112143280 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4026040 con 0x55b37db4f130 2024-03-26T18:00:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 -- 172.21.15.105:0/3112143280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 msgr2=0x55b37db4e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/3112143280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db4e2e0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f4da4005c40 tx=0x7f4da4009f40 comp rx=0 tx=0).stop 2024-03-26T18:00:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 -- 172.21.15.105:0/3112143280 shutdown_connections 2024-03-26T18:00:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/3112143280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db4e2e0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.875 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/3112143280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db49fc0 0x55b37db53830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/3112143280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db495d0 0x55b37db532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 -- 172.21.15.105:0/3112143280 >> 172.21.15.105:0/3112143280 conn(0x55b37db44b40 msgr2=0x55b37db44f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 -- 172.21.15.105:0/3112143280 shutdown_connections 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.886+0000 7f4db6dd6740 1 -- 172.21.15.105:0/3112143280 wait complete. 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db6dd6740 1 -- start start 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db6dd6740 1 Processor -- start 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db495d0 0x55b37db57cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db58210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db6dd6740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1aa180 con 0x55b37db4f130 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1a9ab0 con 0x55b37db49fc0 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db6dd6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac1a6cf0 con 0x55b37db495d0 2024-03-26T18:00:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db1eb2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db495d0 0x55b37db57cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db1eb2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db495d0 0x55b37db57cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:52752/0 (socket says 172.21.15.105:52752) 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db1eb2640 1 -- 172.21.15.105:0/377096317 learned_addr learned my addr 172.21.15.105:0/377096317 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db16b1640 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db58210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.887+0000 7f4db26b3640 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4db6dd6740 10 client.0 _ll_register_callbacks cb 0x7f4da4003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4db1eb2640 1 -- 172.21.15.105:0/377096317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 msgr2=0x55b37db58210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db58210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4db1eb2640 1 -- 172.21.15.105:0/377096317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 msgr2=0x55b37db591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4db1eb2640 1 -- 172.21.15.105:0/377096317 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b37db597c0 con 0x55b37db495d0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4db1eb2640 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db495d0 0x55b37db57cd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4da8098840 tx=0x7f4da80bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db3ab517e9a7124d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4d957fa640 1 -- 172.21.15.105:0/377096317 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da809bec0 con 0x55b37db495d0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4d957fa640 1 -- 172.21.15.105:0/377096317 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f4da809c120 con 0x55b37db495d0 2024-03-26T18:00:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.888+0000 7f4db6dd6740 1 -- 172.21.15.105:0/377096317 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da4010d10 con 0x55b37db495d0 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4d957fa640 1 -- 172.21.15.105:0/377096317 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4da80a0010 con 0x55b37db495d0 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db6dd6740 10 client.32992 fetch_fsmap learned FSMap version 41 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db6dd6740 1 -- 172.21.15.105:0/377096317 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b37db5ab20 con 0x55b37db495d0 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4d957fa640 1 -- 172.21.15.105:0/377096317 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da80a48d0 con 0x55b37db495d0 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4d957fa640 1 -- 172.21.15.105:0/377096317 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f4da8097d10 con 0x55b37db495d0 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db6dd6740 10 client.32992 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db6dd6740 10 client.32992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db6dd6740 20 client.32992 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b37dbce3e0 0x55b37dbd0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db6dd6740 4 client.32992 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:00:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db6dd6740 1 -- 172.21.15.105:0/377096317 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4da4010d10 con 0x55b37dbce3e0 2024-03-26T18:00:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.889+0000 7f4db16b1640 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b37dbce3e0 0x55b37dbd0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.890+0000 7f4d957fa640 10 client.32992 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:06.890+0000 7f4db16b1640 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b37dbce3e0 0x55b37dbd0800 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848c1dba03ba619 server_cookie=374a54af04d2c399 in_seq=0 out_seq=0 2024-03-26T18:00:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.047+0000 7f4d957fa640 1 -- 172.21.15.105:0/377096317 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4da4010d10 con 0x55b37dbce3e0 2024-03-26T18:00:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.047+0000 7f4d957fa640 10 client.32992 handle_command_reply: tid=0 2024-03-26T18:00:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.047+0000 7f4db6dd6740 1 client.32992 shutdown 2024-03-26T18:00:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.047+0000 7f4db6dd6740 1 -- 172.21.15.105:0/377096317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db495d0 msgr2=0x55b37db57cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.047+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db495d0 0x55b37db57cd0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4da8098840 tx=0x7f4da80bc040 comp rx=0 tx=0).stop 2024-03-26T18:00:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.047+0000 7f4db6dd6740 1 -- 172.21.15.105:0/377096317 shutdown_connections 2024-03-26T18:00:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.047+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55b37dbce3e0 0x55b37dbd0800 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.048+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b37db4f130 0x55b37db591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.048+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b37db49fc0 0x55b37db58210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.048+0000 7f4db6dd6740 1 --2- 172.21.15.105:0/377096317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b37db495d0 0x55b37db57cd0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.048+0000 7f4db6dd6740 1 -- 172.21.15.105:0/377096317 >> 172.21.15.105:0/377096317 conn(0x55b37db44b40 msgr2=0x55b37db532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.048+0000 7f4d957fa640 0 client.32992 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.048+0000 7f4db6dd6740 1 -- 172.21.15.105:0/377096317 shutdown_connections 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.048+0000 7f4db6dd6740 1 -- 172.21.15.105:0/377096317 wait complete. 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.048+0000 7f4db6dd6740 20 client.32992 trim_cache size 0 max 16384 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:00:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:00:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:00:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 -- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d980a2fc0 msgr2=0x7f4d980a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d980a2fc0 0x7f4d980a5480 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x55b37db4a820 tx=0x7f4da4004ab0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 -- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1007e0 msgr2=0x7f4dac197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1007e0 0x7f4dac197a80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4da800e9c0 tx=0x7f4da800ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 -- 172.21.15.105:0/3980239039 shutdown_connections 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d980a2fc0 0x7f4d980a5480 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4dac102390 0x7f4dac19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dac1019d0 0x7f4dac197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 --2- 172.21.15.105:0/3980239039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dac1007e0 0x7f4dac197a80 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 -- 172.21.15.105:0/3980239039 >> 172.21.15.105:0/3980239039 conn(0x7f4dac0fbd50 msgr2=0x7f4dac104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 -- 172.21.15.105:0/3980239039 shutdown_connections 2024-03-26T18:00:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:07.049+0000 7f4db4116640 1 -- 172.21.15.105:0/3980239039 wait complete. 2024-03-26T18:00:07.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:06 smithi144 ceph-mon[36095]: pgmap v1004: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-03-26T18:00:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:06 smithi105 ceph-mon[32280]: pgmap v1004: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-03-26T18:00:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:06 smithi135 ceph-mon[35636]: pgmap v1004: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-03-26T18:00:07.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 69369 7.04 MB/sec execute 410 sec latency 238.215 ms 2024-03-26T18:00:08.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 69486 7.04 MB/sec execute 411 sec latency 173.950 ms 2024-03-26T18:00:09.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:08 smithi144 ceph-mon[36095]: pgmap v1005: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.6 MiB/s wr, 260 op/s 2024-03-26T18:00:09.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:08 smithi105 ceph-mon[32280]: pgmap v1005: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.6 MiB/s wr, 260 op/s 2024-03-26T18:00:09.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:08 smithi135 ceph-mon[35636]: pgmap v1005: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 3.6 MiB/s wr, 260 op/s 2024-03-26T18:00:09.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 69606 7.04 MB/sec execute 412 sec latency 127.397 ms 2024-03-26T18:00:10.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 69760 7.04 MB/sec execute 413 sec latency 130.201 ms 2024-03-26T18:00:11.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:10 smithi144 ceph-mon[36095]: pgmap v1006: 97 pgs: 97 active+clean; 345 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-26T18:00:11.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:10 smithi105 ceph-mon[32280]: pgmap v1006: 97 pgs: 97 active+clean; 345 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-26T18:00:11.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:10 smithi135 ceph-mon[35636]: pgmap v1006: 97 pgs: 97 active+clean; 345 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 2.9 MiB/s wr, 267 op/s 2024-03-26T18:00:11.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 69881 7.04 MB/sec execute 414 sec latency 118.901 ms 2024-03-26T18:00:12.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:11 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:00:12.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:11 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:00:12.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:11 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:00:12.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 70019 7.05 MB/sec execute 415 sec latency 184.563 ms 2024-03-26T18:00:13.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:12 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:00:13.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:12 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:00:13.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:12 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:00:13.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:12 smithi144 ceph-mon[36095]: pgmap v1007: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 3.0 MiB/s wr, 283 op/s 2024-03-26T18:00:13.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:12 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:00:13.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:12 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:00:13.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:12 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:00:13.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:12 smithi105 ceph-mon[32280]: pgmap v1007: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 3.0 MiB/s wr, 283 op/s 2024-03-26T18:00:13.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:12 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:00:13.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:12 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:00:13.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:12 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:00:13.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:12 smithi135 ceph-mon[35636]: pgmap v1007: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 3.0 MiB/s wr, 283 op/s 2024-03-26T18:00:13.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 70146 7.05 MB/sec execute 416 sec latency 123.969 ms 2024-03-26T18:00:14.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 70284 7.03 MB/sec execute 417 sec latency 147.150 ms 2024-03-26T18:00:15.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:14 smithi144 ceph-mon[36095]: pgmap v1008: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-03-26T18:00:15.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:14 smithi105 ceph-mon[32280]: pgmap v1008: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-03-26T18:00:15.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:14 smithi135 ceph-mon[35636]: pgmap v1008: 97 pgs: 97 active+clean; 343 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-03-26T18:00:15.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 70429 7.03 MB/sec execute 418 sec latency 128.057 ms 2024-03-26T18:00:16.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 70580 7.03 MB/sec execute 419 sec latency 124.717 ms 2024-03-26T18:00:17.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:16 smithi144 ceph-mon[36095]: pgmap v1009: 97 pgs: 97 active+clean; 344 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-03-26T18:00:17.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:16 smithi105 ceph-mon[32280]: pgmap v1009: 97 pgs: 97 active+clean; 344 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-03-26T18:00:17.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:16 smithi135 ceph-mon[35636]: pgmap v1009: 97 pgs: 97 active+clean; 344 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 2.7 MiB/s wr, 261 op/s 2024-03-26T18:00:17.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 70733 7.02 MB/sec execute 420 sec latency 124.905 ms 2024-03-26T18:00:18.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 70863 7.01 MB/sec execute 421 sec latency 120.214 ms 2024-03-26T18:00:19.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:18 smithi144 ceph-mon[36095]: pgmap v1010: 97 pgs: 97 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-03-26T18:00:19.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:18 smithi105 ceph-mon[32280]: pgmap v1010: 97 pgs: 97 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-03-26T18:00:19.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:18 smithi135 ceph-mon[35636]: pgmap v1010: 97 pgs: 97 active+clean; 342 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 286 op/s 2024-03-26T18:00:19.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 71008 7.01 MB/sec execute 422 sec latency 148.446 ms 2024-03-26T18:00:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 -- 172.21.15.105:0/4223862181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 msgr2=0x7f0d5c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 --2- 172.21.15.105:0/4223862181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 0x7f0d5c101d30 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009900 tx=0x7f0d5802bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 -- 172.21.15.105:0/4223862181 shutdown_connections 2024-03-26T18:00:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 --2- 172.21.15.105:0/4223862181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5c1034c0 0x7f0d5c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 --2- 172.21.15.105:0/4223862181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c102b00 0x7f0d5c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 --2- 172.21.15.105:0/4223862181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 0x7f0d5c101d30 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 -- 172.21.15.105:0/4223862181 >> 172.21.15.105:0/4223862181 conn(0x7f0d5c0fce80 msgr2=0x7f0d5c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 -- 172.21.15.105:0/4223862181 shutdown_connections 2024-03-26T18:00:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 -- 172.21.15.105:0/4223862181 wait complete. 2024-03-26T18:00:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.157+0000 7f0d64085640 1 Processor -- start 2024-03-26T18:00:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d64085640 1 -- start start 2024-03-26T18:00:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d64085640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 0x7f0d5c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d64085640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c102b00 0x7f0d5c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d64085640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5c1034c0 0x7f0d5c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d64085640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c10d440 con 0x7f0d5c101910 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d64085640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c10d2c0 con 0x7f0d5c102b00 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d64085640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c10d5c0 con 0x7f0d5c1034c0 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 0x7f0d5c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 0x7f0d5c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45750/0 (socket says 172.21.15.105:45750) 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61e21640 1 -- 172.21.15.105:0/2393707451 learned_addr learned my addr 172.21.15.105:0/2393707451 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61620640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c102b00 0x7f0d5c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d62622640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5c1034c0 0x7f0d5c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61e21640 1 -- 172.21.15.105:0/2393707451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5c1034c0 msgr2=0x7f0d5c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61e21640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5c1034c0 0x7f0d5c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61e21640 1 -- 172.21.15.105:0/2393707451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c102b00 msgr2=0x7f0d5c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61e21640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c102b00 0x7f0d5c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.158+0000 7f0d61e21640 1 -- 172.21.15.105:0/2393707451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5c19f6c0 con 0x7f0d5c101910 2024-03-26T18:00:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.159+0000 7f0d61e21640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 0x7f0d5c197af0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f0d58000c00 tx=0x7f0d58005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.159+0000 7f0d52ffd640 1 -- 172.21.15.105:0/2393707451 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58043020 con 0x7f0d5c101910 2024-03-26T18:00:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.159+0000 7f0d64085640 1 -- 172.21.15.105:0/2393707451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5c19f950 con 0x7f0d5c101910 2024-03-26T18:00:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.159+0000 7f0d52ffd640 1 -- 172.21.15.105:0/2393707451 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d58004350 con 0x7f0d5c101910 2024-03-26T18:00:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.159+0000 7f0d64085640 1 -- 172.21.15.105:0/2393707451 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5c19fef0 con 0x7f0d5c101910 2024-03-26T18:00:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.159+0000 7f0d52ffd640 1 -- 172.21.15.105:0/2393707451 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58003b80 con 0x7f0d5c101910 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.161+0000 7f0d52ffd640 1 -- 172.21.15.105:0/2393707451 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0d58046020 con 0x7f0d5c101910 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.162+0000 7f0d52ffd640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d480a3040 0x7f0d480a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.162+0000 7f0d61620640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d480a3040 0x7f0d480a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.162+0000 7f0d52ffd640 1 -- 172.21.15.105:0/2393707451 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0d580f14b0 con 0x7f0d5c101910 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d66c79740 1 Processor -- start 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d61620640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d480a3040 0x7f0d480a5500 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c198fb0 tx=0x7f0d4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d66c79740 1 -- start start 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a8d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8dfc0 0x559095a92ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a93130 0x559095a955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d66c79740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5590959f20b0 con 0x559095a8dfc0 2024-03-26T18:00:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d66c79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55909599a960 con 0x559095a8d5d0 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d66c79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5590959ee060 con 0x559095a93130 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a8d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a8d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42218/0 (socket says 172.21.15.105:42218) 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d61e21640 1 -- 172.21.15.105:0/3086658037 learned_addr learned my addr 172.21.15.105:0/3086658037 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d62622640 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a93130 0x559095a955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.163+0000 7f0d61620640 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8dfc0 0x559095a92ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d62622640 1 -- 172.21.15.105:0/3086658037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 msgr2=0x559095a8d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d62622640 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a8d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d62622640 1 -- 172.21.15.105:0/3086658037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8dfc0 msgr2=0x559095a92ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d62622640 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8dfc0 0x559095a92ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d62622640 1 -- 172.21.15.105:0/3086658037 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590959acc60 con 0x559095a93130 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d61e21640 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a8d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d62622640 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a93130 0x559095a955c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009f00 tx=0x7f0d54011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=697e35e39410b2b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d47fff640 1 -- 172.21.15.105:0/3086658037 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54004d20 con 0x559095a93130 2024-03-26T18:00:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d47fff640 1 -- 172.21.15.105:0/3086658037 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d54004030 con 0x559095a93130 2024-03-26T18:00:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.164+0000 7f0d47fff640 1 -- 172.21.15.105:0/3086658037 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54005280 con 0x559095a93130 2024-03-26T18:00:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 -- 172.21.15.105:0/3086658037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a93130 msgr2=0x559095a955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a93130 0x559095a955c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009f00 tx=0x7f0d54011810 comp rx=0 tx=0).stop 2024-03-26T18:00:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 -- 172.21.15.105:0/3086658037 shutdown_connections 2024-03-26T18:00:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a93130 0x559095a955c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8dfc0 0x559095a92ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3086658037 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a8d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 -- 172.21.15.105:0/3086658037 >> 172.21.15.105:0/3086658037 conn(0x559095a88b40 msgr2=0x559095a88f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 -- 172.21.15.105:0/3086658037 shutdown_connections 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.165+0000 7f0d66c79740 1 -- 172.21.15.105:0/3086658037 wait complete. 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 1 -- start start 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 1 Processor -- start 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8d5d0 0x559095a973e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a97920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a93130 0x559095a9e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559095a960e0 con 0x559095a8d5d0 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5590959addb0 con 0x559095a93130 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559095a96260 con 0x559095a8dfc0 2024-03-26T18:00:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d66c79740 10 client.0 _ll_register_callbacks cb 0x5590959b2110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8d5d0 0x559095a973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8d5d0 0x559095a973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45764/0 (socket says 172.21.15.105:45764) 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d61e21640 1 -- 172.21.15.105:0/2189664225 learned_addr learned my addr 172.21.15.105:0/2189664225 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d62622640 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a93130 0x559095a9e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.166+0000 7f0d61620640 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a97920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d62622640 1 -- 172.21.15.105:0/2189664225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 msgr2=0x559095a97920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d62622640 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a97920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d62622640 1 -- 172.21.15.105:0/2189664225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8d5d0 msgr2=0x559095a973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d62622640 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8d5d0 0x559095a973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d62622640 1 -- 172.21.15.105:0/2189664225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559095706210 con 0x559095a93130 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d61620640 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a97920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d62622640 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a93130 0x559095a9e8f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009270 tx=0x7f0d54004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6fd39add9ba33b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d457fa640 1 -- 172.21.15.105:0/2189664225 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54027020 con 0x559095a93130 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d457fa640 1 -- 172.21.15.105:0/2189664225 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0d54003a30 con 0x559095a93130 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d66c79740 1 -- 172.21.15.105:0/2189664225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d54004030 con 0x559095a93130 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d457fa640 1 -- 172.21.15.105:0/2189664225 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0d54032050 con 0x559095a93130 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d66c79740 10 client.33011 fetch_fsmap learned FSMap version 41 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.167+0000 7f0d66c79740 1 -- 172.21.15.105:0/2189664225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5590957d2620 con 0x559095a93130 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d457fa640 1 -- 172.21.15.105:0/2189664225 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d54003c50 con 0x559095a93130 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d457fa640 1 -- 172.21.15.105:0/2189664225 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f0d54018070 con 0x559095a93130 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d66c79740 10 client.33011 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d66c79740 10 client.33011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d66c79740 20 client.33011 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d66c79740 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559095aab840 0x559095aadc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d66c79740 4 client.33011 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d66c79740 1 -- 172.21.15.105:0/2189664225 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0d54004030 con 0x559095aab840 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.168+0000 7f0d61e21640 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559095aab840 0x559095aadc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.169+0000 7f0d61e21640 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559095aab840 0x559095aadc50 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5383d00097c5f42 server_cookie=c96dc80fe71f63bb in_seq=0 out_seq=0 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.169+0000 7f0d457fa640 10 client.33011 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.170+0000 7f0d457fa640 1 -- 172.21.15.105:0/2189664225 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f0d54004030 con 0x559095aab840 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.170+0000 7f0d457fa640 10 client.33011 handle_command_reply: tid=0 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.170+0000 7f0d66c79740 1 client.33011 shutdown 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.170+0000 7f0d66c79740 1 -- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a93130 msgr2=0x559095a9e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.156 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.170+0000 7f0d66c79740 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a93130 0x559095a9e8f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009270 tx=0x7f0d54004a40 comp rx=0 tx=0).stop 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.170+0000 7f0d66c79740 1 -- 172.21.15.105:0/2189664225 shutdown_connections 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.170+0000 7f0d66c79740 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559095aab840 0x559095aadc50 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.170+0000 7f0d457fa640 0 client.33011 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.171+0000 7f0d66c79740 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a93130 0x559095a9e8f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.171+0000 7f0d66c79740 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a97920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.171+0000 7f0d66c79740 1 --2- 172.21.15.105:0/2189664225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a8d5d0 0x559095a973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.171+0000 7f0d66c79740 1 -- 172.21.15.105:0/2189664225 >> 172.21.15.105:0/2189664225 conn(0x559095a88b40 msgr2=0x559095a87df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.171+0000 7f0d66c79740 1 -- 172.21.15.105:0/2189664225 shutdown_connections 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.171+0000 7f0d66c79740 1 -- 172.21.15.105:0/2189664225 wait complete. 2024-03-26T18:00:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.171+0000 7f0d66c79740 20 client.33011 trim_cache size 0 max 16384 2024-03-26T18:00:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.192+0000 7f0d66c79740 1 Processor -- start 2024-03-26T18:00:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d66c79740 1 -- start start 2024-03-26T18:00:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a9e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a92920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d66c79740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c1a6a30 con 0x559095a93130 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d66c79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c1a8570 con 0x559095a8d5d0 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d66c79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c1a7100 con 0x559095a8dfc0 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d62622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a92920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d61620640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a9e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d62622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a92920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45766/0 (socket says 172.21.15.105:45766) 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:42236/0 (socket says 172.21.15.105:42236) 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d62622640 1 -- 172.21.15.105:0/4167890977 learned_addr learned my addr 172.21.15.105:0/4167890977 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d62622640 1 -- 172.21.15.105:0/4167890977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 msgr2=0x559095a9e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d62622640 1 --2- 172.21.15.105:0/4167890977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a9e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d62622640 1 -- 172.21.15.105:0/4167890977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 msgr2=0x559095a923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d62622640 1 --2- 172.21.15.105:0/4167890977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d62622640 1 -- 172.21.15.105:0/4167890977 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559095a8d3c0 con 0x559095a93130 2024-03-26T18:00:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.193+0000 7f0d61620640 1 --2- 172.21.15.105:0/4167890977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a9e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.194+0000 7f0d62622640 1 --2- 172.21.15.105:0/4167890977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a92920 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0d5402e530 tx=0x7f0d54004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bd45210b18aa40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.194+0000 7f0d477fe640 1 -- 172.21.15.105:0/4167890977 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54004760 con 0x559095a93130 2024-03-26T18:00:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.194+0000 7f0d477fe640 1 -- 172.21.15.105:0/4167890977 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d5402e720 con 0x559095a93130 2024-03-26T18:00:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.194+0000 7f0d477fe640 1 -- 172.21.15.105:0/4167890977 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5402e0e0 con 0x559095a93130 2024-03-26T18:00:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 -- 172.21.15.105:0/4167890977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 msgr2=0x559095a92920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 --2- 172.21.15.105:0/4167890977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a92920 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0d5402e530 tx=0x7f0d54004ab0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 -- 172.21.15.105:0/4167890977 shutdown_connections 2024-03-26T18:00:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 --2- 172.21.15.105:0/4167890977 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a92920 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 --2- 172.21.15.105:0/4167890977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8dfc0 0x559095a9e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 --2- 172.21.15.105:0/4167890977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8d5d0 0x559095a923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 -- 172.21.15.105:0/4167890977 >> 172.21.15.105:0/4167890977 conn(0x559095a88b40 msgr2=0x559095a973e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 -- 172.21.15.105:0/4167890977 shutdown_connections 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 -- 172.21.15.105:0/4167890977 wait complete. 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 -- start start 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.195+0000 7f0d66c79740 1 Processor -- start 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8d5d0 0x559095a9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8dfc0 0x559095a9c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d66c79740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a9d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d66c79740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c1a7ea0 con 0x559095a93130 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d66c79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c1a77d0 con 0x559095a8dfc0 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d66c79740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5c1a4a10 con 0x559095a8d5d0 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d66c79740 10 client.0 _ll_register_callbacks cb 0x7f0d4c0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a9d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d61e21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8d5d0 0x559095a9bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a9d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45770/0 (socket says 172.21.15.105:45770) 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 -- 172.21.15.105:0/3231993493 learned_addr learned my addr 172.21.15.105:0/3231993493 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d61620640 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8dfc0 0x559095a9c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 -- 172.21.15.105:0/3231993493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8d5d0 msgr2=0x559095a9bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8d5d0 0x559095a9bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 -- 172.21.15.105:0/3231993493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8dfc0 msgr2=0x559095a9c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8dfc0 0x559095a9c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 -- 172.21.15.105:0/3231993493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559095a9d8a0 con 0x559095a93130 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d61e21640 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8d5d0 0x559095a9bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d62622640 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a9d2b0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009f00 tx=0x7f0d54004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c48e05d77e218c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.196+0000 7f0d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d457fa640 1 -- 172.21.15.105:0/3231993493 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5402e8d0 con 0x559095a93130 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d457fa640 1 -- 172.21.15.105:0/3231993493 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8293+0+0 (secure 0 0 0) 0x7f0d5402eb70 con 0x559095a93130 2024-03-26T18:00:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 1 -- 172.21.15.105:0/3231993493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d5402e720 con 0x559095a93130 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d457fa640 1 -- 172.21.15.105:0/3231993493 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0d54046050 con 0x559095a93130 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 10 client.23340 fetch_fsmap learned FSMap version 41 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 1 -- 172.21.15.105:0/3231993493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559095a9ebb0 con 0x559095a93130 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d457fa640 1 -- 172.21.15.105:0/3231993493 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d5402e720 con 0x559095a93130 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d457fa640 1 -- 172.21.15.105:0/3231993493 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f0d54018070 con 0x559095a93130 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 10 client.23340 fetch_fsmap finished waiting for FSMap version 41 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 10 client.23340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 20 client.23340 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559095b122e0 0x559095b14700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 4 client.23340 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d66c79740 1 -- 172.21.15.105:0/3231993493 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590959ff5f0 con 0x559095b122e0 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.197+0000 7f0d61e21640 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559095b122e0 0x559095b14700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.198+0000 7f0d457fa640 10 client.23340 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.198+0000 7f0d61e21640 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559095b122e0 0x559095b14700 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45badd0ec7239fd0 server_cookie=ffb316cb0e7291bf in_seq=0 out_seq=0 2024-03-26T18:00:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d457fa640 1 -- 172.21.15.105:0/3231993493 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590959ff5f0 con 0x559095b122e0 2024-03-26T18:00:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d457fa640 10 client.23340 handle_command_reply: tid=0 2024-03-26T18:00:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 client.23340 shutdown 2024-03-26T18:00:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 -- 172.21.15.105:0/3231993493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 msgr2=0x559095a9d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a9d2b0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009f00 tx=0x7f0d54004a10 comp rx=0 tx=0).stop 2024-03-26T18:00:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 -- 172.21.15.105:0/3231993493 shutdown_connections 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559095b122e0 0x559095b14700 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559095a93130 0x559095a9d2b0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559095a8dfc0 0x559095a9c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 --2- 172.21.15.105:0/3231993493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559095a8d5d0 0x559095a9bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d457fa640 0 client.23340 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 -- 172.21.15.105:0/3231993493 >> 172.21.15.105:0/3231993493 conn(0x559095a88b40 msgr2=0x559095a923e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.326+0000 7f0d66c79740 1 -- 172.21.15.105:0/3231993493 shutdown_connections 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d66c79740 1 -- 172.21.15.105:0/3231993493 wait complete. 2024-03-26T18:00:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d66c79740 20 client.23340 trim_cache size 0 max 16384 2024-03-26T18:00:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:00:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:00:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:00:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:00:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 -- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d480a3040 msgr2=0x7f0d480a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d480a3040 0x7f0d480a5500 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c198fb0 tx=0x7f0d4c005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 -- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 msgr2=0x7f0d5c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 0x7f0d5c197af0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f0d58000c00 tx=0x7f0d58005dc0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 -- 172.21.15.105:0/2393707451 shutdown_connections 2024-03-26T18:00:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f0d480a3040 0x7f0d480a5500 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0d5c1034c0 0x7f0d5c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d5c102b00 0x7f0d5c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 --2- 172.21.15.105:0/2393707451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0d5c101910 0x7f0d5c197af0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 -- 172.21.15.105:0/2393707451 >> 172.21.15.105:0/2393707451 conn(0x7f0d5c0fce80 msgr2=0x7f0d5c105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 -- 172.21.15.105:0/2393707451 shutdown_connections 2024-03-26T18:00:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:20.327+0000 7f0d64085640 1 -- 172.21.15.105:0/2393707451 wait complete. 2024-03-26T18:00:20.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 71136 7.01 MB/sec execute 423 sec latency 177.959 ms 2024-03-26T18:00:20.882 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:20 smithi105 ceph-mon[32280]: daemon mds.e restarted 2024-03-26T18:00:20.905 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:20 smithi135 ceph-mon[35636]: daemon mds.e restarted 2024-03-26T18:00:20.905 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:00:20 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setuser ceph since I am not root 2024-03-26T18:00:20.905 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:00:20 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setgroup ceph since I am not root 2024-03-26T18:00:20.906 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:00:20 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/1868569128 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db9fa31a40 con 0x55db9fa04800 2024-03-26T18:00:20.906 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:00:20 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T18:00:21.088 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:20 smithi144 ceph-mon[36095]: daemon mds.e restarted 2024-03-26T18:00:21.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 71294 7.02 MB/sec execute 424 sec latency 122.317 ms 2024-03-26T18:00:21.882 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:21 smithi105 ceph-mon[32280]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T18:00:21.882 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:21 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/966537438,v1:172.21.15.135:6835/966537438] up:boot 2024-03-26T18:00:21.882 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:21 smithi105 ceph-mon[32280]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T18:00:21.882 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:21 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:00:21.882 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:21 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:00:21.882 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:21 smithi105 ceph-mon[32280]: pgmap v1012: 97 pgs: 97 active+clean; 341 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.4 MiB/s wr, 219 op/s 2024-03-26T18:00:21.905 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:21 smithi135 ceph-mon[35636]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T18:00:21.905 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:21 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/966537438,v1:172.21.15.135:6835/966537438] up:boot 2024-03-26T18:00:21.905 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:21 smithi135 ceph-mon[35636]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T18:00:21.905 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:21 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:00:21.905 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:21 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:00:21.906 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:21 smithi135 ceph-mon[35636]: pgmap v1012: 97 pgs: 97 active+clean; 341 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.4 MiB/s wr, 219 op/s 2024-03-26T18:00:22.088 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:21 smithi144 ceph-mon[36095]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T18:00:22.089 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:21 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/966537438,v1:172.21.15.135:6835/966537438] up:boot 2024-03-26T18:00:22.089 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:21 smithi144 ceph-mon[36095]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T18:00:22.089 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:21 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:00:22.089 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:21 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:00:22.089 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:21 smithi144 ceph-mon[36095]: pgmap v1012: 97 pgs: 97 active+clean; 341 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 2.4 MiB/s wr, 219 op/s 2024-03-26T18:00:22.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 71494 7.05 MB/sec execute 425 sec latency 117.570 ms 2024-03-26T18:00:23.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:22 smithi144 ceph-mon[36095]: pgmap v1013: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-03-26T18:00:23.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:22 smithi105 ceph-mon[32280]: pgmap v1013: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-03-26T18:00:23.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:22 smithi135 ceph-mon[35636]: pgmap v1013: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-03-26T18:00:23.557 INFO:tasks.workunit.client.0.smithi105.stdout: 10 71637 7.05 MB/sec execute 426 sec latency 121.902 ms 2024-03-26T18:00:24.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 71800 7.07 MB/sec execute 427 sec latency 131.657 ms 2024-03-26T18:00:25.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:24 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:00:25.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:24 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:00:25.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:24 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:00:25.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 71942 7.08 MB/sec execute 428 sec latency 123.204 ms 2024-03-26T18:00:26.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:25 smithi144 ceph-mon[36095]: pgmap v1014: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:00:26.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:26.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:25 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:26.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:25 smithi105 ceph-mon[32280]: pgmap v1014: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:00:26.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:26.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:25 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:26.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:25 smithi135 ceph-mon[35636]: pgmap v1014: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:00:26.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:26.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:25 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:26.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72040 7.07 MB/sec execute 429 sec latency 128.625 ms 2024-03-26T18:00:27.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72126 7.06 MB/sec execute 430 sec latency 132.693 ms 2024-03-26T18:00:27.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: pgmap v1015: 97 pgs: 97 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-03-26T18:00:27.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:00:27.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:00:27.590 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: pgmap v1015: 97 pgs: 97 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:00:27.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.656 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.656 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: pgmap v1015: 97 pgs: 97 active+clean; 349 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 3.6 MiB/s wr, 244 op/s 2024-03-26T18:00:27.656 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.656 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:27.656 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:00:27.656 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:00:27.656 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:00:28.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72224 7.06 MB/sec execute 431 sec latency 129.731 ms 2024-03-26T18:00:29.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:28 smithi144 ceph-mon[36095]: pgmap v1016: 97 pgs: 97 active+clean; 353 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-03-26T18:00:29.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:28 smithi105 ceph-mon[32280]: pgmap v1016: 97 pgs: 97 active+clean; 353 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-03-26T18:00:29.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:28 smithi135 ceph-mon[35636]: pgmap v1016: 97 pgs: 97 active+clean; 353 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-03-26T18:00:29.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72348 7.06 MB/sec execute 432 sec latency 131.018 ms 2024-03-26T18:00:29.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:00:29.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:00:29.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 -- 172.21.15.105:0/3905878089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 msgr2=0x7fea2c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3905878089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 0x7fea2c101e30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fea18009900 tx=0x7fea1802bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 -- 172.21.15.105:0/3905878089 shutdown_connections 2024-03-26T18:00:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3905878089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea2c102370 0x7fea2c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3905878089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 0x7fea2c101e30 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3905878089 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea2c1007c0 0x7fea2c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 -- 172.21.15.105:0/3905878089 >> 172.21.15.105:0/3905878089 conn(0x7fea2c0fbd50 msgr2=0x7fea2c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 -- 172.21.15.105:0/3905878089 shutdown_connections 2024-03-26T18:00:29.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.910+0000 7fea32ef7640 1 -- 172.21.15.105:0/3905878089 wait complete. 2024-03-26T18:00:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea32ef7640 1 Processor -- start 2024-03-26T18:00:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea32ef7640 1 -- start start 2024-03-26T18:00:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea32ef7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea2c1007c0 0x7fea2c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea32ef7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 0x7fea2c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea32ef7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea2c102370 0x7fea2c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea32ef7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c10c340 con 0x7fea2c1019b0 2024-03-26T18:00:29.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea32ef7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c10c1c0 con 0x7fea2c102370 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea32ef7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c10c4c0 con 0x7fea2c1007c0 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea31494640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea2c102370 0x7fea2c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea31494640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea2c102370 0x7fea2c19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45524/0 (socket says 172.21.15.105:45524) 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.911+0000 7fea31494640 1 -- 172.21.15.105:0/3396690190 learned_addr learned my addr 172.21.15.105:0/3396690190 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea2bfff640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 0x7fea2c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea30c93640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea2c1007c0 0x7fea2c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea31494640 1 -- 172.21.15.105:0/3396690190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea2c1007c0 msgr2=0x7fea2c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea31494640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea2c1007c0 0x7fea2c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea31494640 1 -- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 msgr2=0x7fea2c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea31494640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 0x7fea2c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea31494640 1 -- 172.21.15.105:0/3396690190 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c19f720 con 0x7fea2c102370 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea30c93640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea2c1007c0 0x7fea2c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea2bfff640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 0x7fea2c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:29.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.912+0000 7fea31494640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea2c102370 0x7fea2c19f0b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fea2400eb10 tx=0x7fea2400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:29.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.913+0000 7fea29ffb640 1 -- 172.21.15.105:0/3396690190 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24004b20 con 0x7fea2c102370 2024-03-26T18:00:29.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.913+0000 7fea29ffb640 1 -- 172.21.15.105:0/3396690190 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea24004cc0 con 0x7fea2c102370 2024-03-26T18:00:29.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.913+0000 7fea29ffb640 1 -- 172.21.15.105:0/3396690190 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24010690 con 0x7fea2c102370 2024-03-26T18:00:29.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.913+0000 7fea32ef7640 1 -- 172.21.15.105:0/3396690190 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c19fa10 con 0x7fea2c102370 2024-03-26T18:00:29.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.914+0000 7fea32ef7640 1 -- 172.21.15.105:0/3396690190 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c19fee0 con 0x7fea2c102370 2024-03-26T18:00:29.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.917+0000 7fea29ffb640 1 -- 172.21.15.105:0/3396690190 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fea24005020 con 0x7fea2c102370 2024-03-26T18:00:29.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.917+0000 7fea35bb7740 1 Processor -- start 2024-03-26T18:00:29.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea35bb7740 1 -- start start 2024-03-26T18:00:29.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8641ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86423de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 0x55ea864268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea35bb7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ea86383980 con 0x55ea8641f2e0 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea29ffb640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea1c0a3070 0x7fea1c0a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea35bb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ea8632bbe0 con 0x55ea8641e8f0 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea35bb7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea8637f930 con 0x55ea86424450 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea30c93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86423de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8641ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea30c93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86423de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45376/0 (socket says 172.21.15.105:45376) 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea31494640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea1c0a3070 0x7fea1c0a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea30c93640 1 -- 172.21.15.105:0/3428363926 learned_addr learned my addr 172.21.15.105:0/3428363926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:29.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.918+0000 7fea2bfff640 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 0x55ea864268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea30c93640 1 -- 172.21.15.105:0/3428363926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 msgr2=0x55ea864268e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:29.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea30c93640 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 0x55ea864268e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea30c93640 1 -- 172.21.15.105:0/3428363926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 msgr2=0x55ea8641ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:29.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea30c93640 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8641ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea30c93640 1 -- 172.21.15.105:0/3428363926 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea861617d0 con 0x55ea8641f2e0 2024-03-26T18:00:29.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea29ffb640 1 -- 172.21.15.105:0/3396690190 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fea240cfaf0 con 0x7fea2c102370 2024-03-26T18:00:29.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea2bfff640 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8641ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea31494640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea1c0a3070 0x7fea1c0a5530 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fea24098b50 tx=0x7fea24004e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea30c93640 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86423de0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fea20009c20 tx=0x7fea20010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b911a2e788ebc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea16ffd640 1 -- 172.21.15.105:0/3428363926 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20005020 con 0x55ea8641f2e0 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea16ffd640 1 -- 172.21.15.105:0/3428363926 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea200047a0 con 0x55ea8641f2e0 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.919+0000 7fea16ffd640 1 -- 172.21.15.105:0/3428363926 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20012690 con 0x55ea8641f2e0 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.920+0000 7fea35bb7740 1 -- 172.21.15.105:0/3428363926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 msgr2=0x55ea86423de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.920+0000 7fea35bb7740 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86423de0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fea20009c20 tx=0x7fea20010740 comp rx=0 tx=0).stop 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.920+0000 7fea35bb7740 1 -- 172.21.15.105:0/3428363926 shutdown_connections 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.920+0000 7fea35bb7740 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 0x55ea864268e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.920+0000 7fea35bb7740 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86423de0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.920+0000 7fea35bb7740 1 --2- 172.21.15.105:0/3428363926 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8641ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.920+0000 7fea35bb7740 1 -- 172.21.15.105:0/3428363926 >> 172.21.15.105:0/3428363926 conn(0x55ea86419e60 msgr2=0x55ea8641a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.920+0000 7fea35bb7740 1 -- 172.21.15.105:0/3428363926 shutdown_connections 2024-03-26T18:00:29.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.921+0000 7fea35bb7740 1 -- 172.21.15.105:0/3428363926 wait complete. 2024-03-26T18:00:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.921+0000 7fea35bb7740 1 -- start start 2024-03-26T18:00:29.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.921+0000 7fea35bb7740 1 Processor -- start 2024-03-26T18:00:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea864289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86428f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 0x55ea8642fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea864289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea35bb7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55ea86427400 con 0x55ea8641f2e0 2024-03-26T18:00:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea35bb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ea8633f160 con 0x55ea8641e8f0 2024-03-26T18:00:29.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea35bb7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea86427580 con 0x55ea86424450 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea30c93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86428f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea35bb7740 10 client.0 _ll_register_callbacks cb 0x55ea8637b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea30c93640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86428f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45382/0 (socket says 172.21.15.105:45382) 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea30c93640 1 -- 172.21.15.105:0/1374911562 learned_addr learned my addr 172.21.15.105:0/1374911562 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea2bfff640 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 0x55ea8642fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea30c93640 1 -- 172.21.15.105:0/1374911562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 msgr2=0x55ea8642fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea30c93640 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 0x55ea8642fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea30c93640 1 -- 172.21.15.105:0/1374911562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 msgr2=0x55ea864289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea30c93640 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea864289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea30c93640 1 -- 172.21.15.105:0/1374911562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea860973f0 con 0x55ea8641f2e0 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.922+0000 7fea2bfff640 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea864289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea30c93640 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86428f20 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fea20019000 tx=0x7fea200051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39eaadaa54b5317e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:29.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea0bfff640 1 -- 172.21.15.105:0/1374911562 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2002a060 con 0x55ea8641f2e0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea35bb7740 1 -- 172.21.15.105:0/1374911562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea200047a0 con 0x55ea8641f2e0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea0bfff640 1 -- 172.21.15.105:0/1374911562 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fea20025670 con 0x55ea8641f2e0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea0bfff640 1 -- 172.21.15.105:0/1374911562 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fea20004ca0 con 0x55ea8641f2e0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea35bb7740 10 client.23358 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea35bb7740 1 -- 172.21.15.105:0/1374911562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ea860570f0 con 0x55ea8641f2e0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.923+0000 7fea0bfff640 1 -- 172.21.15.105:0/1374911562 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea200047a0 con 0x55ea8641f2e0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea0bfff640 1 -- 172.21.15.105:0/1374911562 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fea2000f040 con 0x55ea8641f2e0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea35bb7740 10 client.23358 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea35bb7740 10 client.23358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea35bb7740 20 client.23358 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea35bb7740 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea86434430 0x55ea86436890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea35bb7740 4 client.23358 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea35bb7740 1 -- 172.21.15.105:0/1374911562 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea86390ec0 con 0x55ea86434430 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea2bfff640 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea86434430 0x55ea86436890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.924+0000 7fea2bfff640 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea86434430 0x55ea86436890 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e6e7b91f54021c server_cookie=42beff998802b9ef in_seq=0 out_seq=0 2024-03-26T18:00:29.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.925+0000 7fea0bfff640 10 client.23358 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea0bfff640 1 -- 172.21.15.105:0/1374911562 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ea86390ec0 con 0x55ea86434430 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea0bfff640 10 client.23358 handle_command_reply: tid=0 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 client.23358 shutdown 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 -- 172.21.15.105:0/1374911562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 msgr2=0x55ea86428f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86428f20 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fea20019000 tx=0x7fea200051d0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 -- 172.21.15.105:0/1374911562 shutdown_connections 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea86434430 0x55ea86436890 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea86424450 0x55ea8642fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea0bfff640 0 client.23358 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641f2e0 0x55ea86428f20 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 --2- 172.21.15.105:0/1374911562 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea864289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 -- 172.21.15.105:0/1374911562 >> 172.21.15.105:0/1374911562 conn(0x55ea86419e60 msgr2=0x55ea86419080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.926+0000 7fea35bb7740 1 -- 172.21.15.105:0/1374911562 shutdown_connections 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.927+0000 7fea35bb7740 1 -- 172.21.15.105:0/1374911562 wait complete. 2024-03-26T18:00:29.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.927+0000 7fea35bb7740 20 client.23358 trim_cache size 0 max 16384 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.950+0000 7fea35bb7740 1 Processor -- start 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.950+0000 7fea35bb7740 1 -- start start 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641e8f0 0x55ea864272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea86424450 0x55ea864203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea35bb7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1a8db0 con 0x55ea8641e8f0 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea35bb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1aa8f0 con 0x55ea86424450 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea35bb7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1a9480 con 0x55ea8641f2e0 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641e8f0 0x55ea864272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641e8f0 0x55ea864272e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45392/0 (socket says 172.21.15.105:45392) 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 -- 172.21.15.105:0/2176088859 learned_addr learned my addr 172.21.15.105:0/2176088859 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea30c93640 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea86424450 0x55ea864203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 -- 172.21.15.105:0/2176088859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 msgr2=0x55ea8642feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 -- 172.21.15.105:0/2176088859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea86424450 msgr2=0x55ea864203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea86424450 0x55ea864203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 -- 172.21.15.105:0/2176088859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea8641e6e0 con 0x55ea8641e8f0 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea2bfff640 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641e8f0 0x55ea864272e0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fea18005ab0 tx=0x7fea1802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a433b49fec86b2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.951+0000 7fea167fc640 1 -- 172.21.15.105:0/2176088859 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18050020 con 0x55ea8641e8f0 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea167fc640 1 -- 172.21.15.105:0/2176088859 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea1804b710 con 0x55ea8641e8f0 2024-03-26T18:00:29.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea167fc640 1 -- 172.21.15.105:0/2176088859 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea180056d0 con 0x55ea8641e8f0 2024-03-26T18:00:29.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea35bb7740 1 -- 172.21.15.105:0/2176088859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641e8f0 msgr2=0x55ea864272e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641e8f0 0x55ea864272e0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fea18005ab0 tx=0x7fea1802bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea35bb7740 1 -- 172.21.15.105:0/2176088859 shutdown_connections 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea86424450 0x55ea864203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2176088859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea8641e8f0 0x55ea864272e0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea35bb7740 1 -- 172.21.15.105:0/2176088859 >> 172.21.15.105:0/2176088859 conn(0x55ea86419e60 msgr2=0x55ea864289e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.952+0000 7fea35bb7740 1 -- 172.21.15.105:0/2176088859 shutdown_connections 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 -- 172.21.15.105:0/2176088859 wait complete. 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 -- start start 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 Processor -- start 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8642ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea86424450 0x55ea86498780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1aa220 con 0x55ea86424450 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1a9b50 con 0x55ea8641e8f0 2024-03-26T18:00:29.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1a6d90 con 0x55ea8641f2e0 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea35bb7740 10 client.0 _ll_register_callbacks cb 0x7fea240093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea30c93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea86424450 0x55ea86498780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea30c93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46200/0 (socket says 172.21.15.105:46200) 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea30c93640 1 -- 172.21.15.105:0/2821982735 learned_addr learned my addr 172.21.15.105:0/2821982735 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.953+0000 7fea2bfff640 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8642ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea30c93640 1 -- 172.21.15.105:0/2821982735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 msgr2=0x55ea8642ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea30c93640 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8642ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea30c93640 1 -- 172.21.15.105:0/2821982735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea86424450 msgr2=0x55ea86498780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea30c93640 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea86424450 0x55ea86498780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea30c93640 1 -- 172.21.15.105:0/2821982735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea8642ffd0 con 0x55ea8641f2e0 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea2bfff640 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea86424450 0x55ea86498780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea30c93640 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642cf70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fea20000c00 tx=0x7fea20005240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7025f71bda0576 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:00:29.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea0bfff640 1 -- 172.21.15.105:0/2821982735 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2002a070 con 0x55ea8641f2e0 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea0bfff640 1 -- 172.21.15.105:0/2821982735 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fea20005580 con 0x55ea8641f2e0 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea35bb7740 1 -- 172.21.15.105:0/2821982735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1804b710 con 0x55ea8641f2e0 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea0bfff640 1 -- 172.21.15.105:0/2821982735 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fea2003f050 con 0x55ea8641f2e0 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea35bb7740 10 client.33031 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea35bb7740 1 -- 172.21.15.105:0/2821982735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ea86499be0 con 0x55ea8641f2e0 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.954+0000 7fea0bfff640 1 -- 172.21.15.105:0/2821982735 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea1804b710 con 0x55ea8641f2e0 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea0bfff640 1 -- 172.21.15.105:0/2821982735 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fea20037020 con 0x55ea8641f2e0 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea35bb7740 10 client.33031 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea35bb7740 10 client.33031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea35bb7740 20 client.33031 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea86434430 0x55ea8649c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea35bb7740 4 client.33031 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea35bb7740 1 -- 172.21.15.105:0/2821982735 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea8638b210 con 0x55ea86434430 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea2bfff640 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea86434430 0x55ea8649c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:29.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea0bfff640 10 client.33031 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.955+0000 7fea2bfff640 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea86434430 0x55ea8649c5f0 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f17469355d8093c6 server_cookie=223aabb6ead4aca9 in_seq=0 out_seq=0 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.956+0000 7fea0bfff640 1 -- 172.21.15.105:0/2821982735 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea8638b210 con 0x55ea86434430 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.956+0000 7fea0bfff640 10 client.33031 handle_command_reply: tid=0 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.956+0000 7fea35bb7740 1 client.33031 shutdown 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.956+0000 7fea35bb7740 1 -- 172.21.15.105:0/2821982735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 msgr2=0x55ea8642cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.956+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642cf70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fea20000c00 tx=0x7fea20005240 comp rx=0 tx=0).stop 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.956+0000 7fea35bb7740 1 -- 172.21.15.105:0/2821982735 shutdown_connections 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.956+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55ea86434430 0x55ea8649c5f0 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55ea86424450 0x55ea86498780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea0bfff640 0 client.33031 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea8641f2e0 0x55ea8642cf70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea35bb7740 1 --2- 172.21.15.105:0/2821982735 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ea8641e8f0 0x55ea8642ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea35bb7740 1 -- 172.21.15.105:0/2821982735 >> 172.21.15.105:0/2821982735 conn(0x55ea86419e60 msgr2=0x55ea864289e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea35bb7740 1 -- 172.21.15.105:0/2821982735 shutdown_connections 2024-03-26T18:00:29.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea35bb7740 1 -- 172.21.15.105:0/2821982735 wait complete. 2024-03-26T18:00:29.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea35bb7740 20 client.33031 trim_cache size 0 max 16384 2024-03-26T18:00:29.944 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:00:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T18:00:29.944 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T18:00:29.944 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea32ef7640 1 -- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea1c0a3070 msgr2=0x7fea1c0a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea1c0a3070 0x7fea1c0a5530 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fea24098b50 tx=0x7fea24004e70 comp rx=0 tx=0).stop 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea32ef7640 1 -- 172.21.15.105:0/3396690190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea2c102370 msgr2=0x7fea2c19f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.957+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea2c102370 0x7fea2c19f0b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fea2400eb10 tx=0x7fea2400c2d0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.958+0000 7fea32ef7640 1 -- 172.21.15.105:0/3396690190 shutdown_connections 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.958+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fea1c0a3070 0x7fea1c0a5530 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.958+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea2c102370 0x7fea2c19f0b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.958+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fea2c1019b0 0x7fea2c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.958+0000 7fea32ef7640 1 --2- 172.21.15.105:0/3396690190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea2c1007c0 0x7fea2c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.958+0000 7fea32ef7640 1 -- 172.21.15.105:0/3396690190 >> 172.21.15.105:0/3396690190 conn(0x7fea2c0fbd50 msgr2=0x7fea2c068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:29.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.958+0000 7fea32ef7640 1 -- 172.21.15.105:0/3396690190 shutdown_connections 2024-03-26T18:00:29.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:29.958+0000 7fea32ef7640 1 -- 172.21.15.105:0/3396690190 wait complete. 2024-03-26T18:00:29.953 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T18:00:29.953 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:00:30.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 -- 172.21.15.105:0/3565470468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c104c90 msgr2=0x7fda0c107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 --2- 172.21.15.105:0/3565470468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c104c90 0x7fda0c107080 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009900 tx=0x7fd9fc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:30.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 -- 172.21.15.105:0/3565470468 shutdown_connections 2024-03-26T18:00:30.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 --2- 172.21.15.105:0/3565470468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda0c1075c0 0x7fda0c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 --2- 172.21.15.105:0/3565470468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c104c90 0x7fda0c107080 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 --2- 172.21.15.105:0/3565470468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c102360 0x7fda0c104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 -- 172.21.15.105:0/3565470468 >> 172.21.15.105:0/3565470468 conn(0x7fda0c0fbd50 msgr2=0x7fda0c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 -- 172.21.15.105:0/3565470468 shutdown_connections 2024-03-26T18:00:30.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.105+0000 7fda14252640 1 -- 172.21.15.105:0/3565470468 wait complete. 2024-03-26T18:00:30.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda14252640 1 Processor -- start 2024-03-26T18:00:30.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda14252640 1 -- start start 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda14252640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda0c102360 0x7fda0c101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda14252640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c104c90 0x7fda0c100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda14252640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c1075c0 0x7fda0c100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda14252640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c10d450 con 0x7fda0c1075c0 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda14252640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c10d2d0 con 0x7fda0c102360 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda14252640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c10d5d0 con 0x7fda0c104c90 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda127ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c1075c0 0x7fda0c100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda117ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c104c90 0x7fda0c100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda117ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c104c90 0x7fda0c100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46226/0 (socket says 172.21.15.105:46226) 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda117ed640 1 -- 172.21.15.105:0/3088848931 learned_addr learned my addr 172.21.15.105:0/3088848931 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda127ef640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c1075c0 0x7fda0c100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45424/0 (socket says 172.21.15.105:45424) 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda11fee640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda0c102360 0x7fda0c101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda117ed640 1 -- 172.21.15.105:0/3088848931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda0c102360 msgr2=0x7fda0c101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda117ed640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda0c102360 0x7fda0c101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda117ed640 1 -- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c1075c0 msgr2=0x7fda0c100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda117ed640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c1075c0 0x7fda0c100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda117ed640 1 -- 172.21.15.105:0/3088848931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c100d90 con 0x7fda0c104c90 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.106+0000 7fda127ef640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c1075c0 0x7fda0c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.107+0000 7fda117ed640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c104c90 0x7fda0c100310 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc000c00 tx=0x7fd9fc005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.107+0000 7fda02ffd640 1 -- 172.21.15.105:0/3088848931 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc043020 con 0x7fda0c104c90 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.107+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c101020 con 0x7fda0c104c90 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.107+0000 7fda02ffd640 1 -- 172.21.15.105:0/3088848931 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9fc0043d0 con 0x7fda0c104c90 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.107+0000 7fda02ffd640 1 -- 172.21.15.105:0/3088848931 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc003b40 con 0x7fda0c104c90 2024-03-26T18:00:30.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.107+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1a65c0 con 0x7fda0c104c90 2024-03-26T18:00:30.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.108+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f00052c0 con 0x7fda0c104c90 2024-03-26T18:00:30.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.110+0000 7fda02ffd640 1 -- 172.21.15.105:0/3088848931 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd9fc046020 con 0x7fda0c104c90 2024-03-26T18:00:30.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.110+0000 7fda02ffd640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9f80a2f20 0x7fd9f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.111+0000 7fda11fee640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9f80a2f20 0x7fd9f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.111+0000 7fda02ffd640 1 -- 172.21.15.105:0/3088848931 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fd9fc0f14d0 con 0x7fda0c104c90 2024-03-26T18:00:30.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.111+0000 7fda11fee640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9f80a2f20 0x7fd9f80a53e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fda080095f0 tx=0x7fda08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.115+0000 7fda02ffd640 1 -- 172.21.15.105:0/3088848931 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd9fc04d020 con 0x7fda0c104c90 2024-03-26T18:00:30.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.461+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9f00055b0 con 0x7fda0c104c90 2024-03-26T18:00:30.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.462+0000 7fda02ffd640 1 -- 172.21.15.105:0/3088848931 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11938 (secure 0 0 0) 0x7fd9fc0be2c0 con 0x7fda0c104c90 2024-03-26T18:00:30.444 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:00:30.444 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:00:20.611284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_23346":{"gid":23346,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/966537438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":966537438},{"type":"v1","addr":"172.21.15.135:6835","nonce":966537438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T18:00:30.444 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 43 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9f80a2f20 msgr2=0x7fd9f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9f80a2f20 0x7fd9f80a53e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fda080095f0 tx=0x7fda08005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c104c90 msgr2=0x7fda0c100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c104c90 0x7fda0c100310 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc000c00 tx=0x7fd9fc005e60 comp rx=0 tx=0).stop 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 shutdown_connections 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9f80a2f20 0x7fd9f80a53e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda0c1075c0 0x7fda0c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c104c90 0x7fda0c100310 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 --2- 172.21.15.105:0/3088848931 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda0c102360 0x7fda0c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 >> 172.21.15.105:0/3088848931 conn(0x7fda0c0fbd50 msgr2=0x7fda0c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 shutdown_connections 2024-03-26T18:00:30.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.465+0000 7fda14252640 1 -- 172.21.15.105:0/3088848931 wait complete. 2024-03-26T18:00:30.460 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T18:00:30.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72495 7.07 MB/sec execute 433 sec latency 123.809 ms 2024-03-26T18:00:30.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 -- 172.21.15.105:0/1440625227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe028102390 msgr2=0x7fe028108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/1440625227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe028102390 0x7fe028108c20 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fe020009ca0 tx=0x7fe02002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 -- 172.21.15.105:0/1440625227 shutdown_connections 2024-03-26T18:00:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/1440625227 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe028102390 0x7fe028108c20 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/1440625227 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0281019d0 0x7fe028101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/1440625227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0281007e0 0x7fe028100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 -- 172.21.15.105:0/1440625227 >> 172.21.15.105:0/1440625227 conn(0x7fe0280fbd70 msgr2=0x7fe0280fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 -- 172.21.15.105:0/1440625227 shutdown_connections 2024-03-26T18:00:30.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.620+0000 7fe02d0f8640 1 -- 172.21.15.105:0/1440625227 wait complete. 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.621+0000 7fe02d0f8640 1 Processor -- start 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.621+0000 7fe02d0f8640 1 -- start start 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.621+0000 7fe02d0f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0281007e0 0x7fe028197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.621+0000 7fe02d0f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0281019d0 0x7fe028198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0281007e0 0x7fe028197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe02659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0281019d0 0x7fe028198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0281007e0 0x7fe028197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45600/0 (socket says 172.21.15.105:45600) 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe02659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0281019d0 0x7fe028198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45444/0 (socket says 172.21.15.105:45444) 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 -- 172.21.15.105:0/2643881737 learned_addr learned my addr 172.21.15.105:0/2643881737 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe028102390 0x7fe02819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe02810c310 con 0x7fe0281019d0 2024-03-26T18:00:30.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe02810c190 con 0x7fe0281007e0 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe02810c490 con 0x7fe028102390 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe02759e640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe028102390 0x7fe02819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 -- 172.21.15.105:0/2643881737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe028102390 msgr2=0x7fe02819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe028102390 0x7fe02819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 -- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0281019d0 msgr2=0x7fe028198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0281019d0 0x7fe028198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 -- 172.21.15.105:0/2643881737 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe02819f720 con 0x7fe0281007e0 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe02659c640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0281019d0 0x7fe028198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.622+0000 7fe026d9d640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0281007e0 0x7fe028197b00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe01800e9c0 tx=0x7fe01800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.623+0000 7fe00ffff640 1 -- 172.21.15.105:0/2643881737 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01800cd80 con 0x7fe0281007e0 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.623+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe02819fa10 con 0x7fe0281007e0 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.623+0000 7fe00ffff640 1 -- 172.21.15.105:0/2643881737 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0180035f0 con 0x7fe0281007e0 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.623+0000 7fe00ffff640 1 -- 172.21.15.105:0/2643881737 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018010690 con 0x7fe0281007e0 2024-03-26T18:00:30.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.623+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe02819ff60 con 0x7fe0281007e0 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe00ffff640 1 -- 172.21.15.105:0/2643881737 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe0180037a0 con 0x7fe0281007e0 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02fdc4740 1 Processor -- start 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02fdc4740 1 -- start start 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d30e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d3e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5579e1c92970 con 0x5579e1d31500 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5579e1c3ad70 con 0x5579e1d2d9b0 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5579e1c8e920 con 0x5579e1d2e3a0 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe00ffff640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0100a2fa0 0x7fe0100a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe026d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d30e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d3e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d3e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45450/0 (socket says 172.21.15.105:45450) 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02659c640 1 -- 172.21.15.105:0/2114089500 learned_addr learned my addr 172.21.15.105:0/2114089500 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02759e640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0100a2fa0 0x7fe0100a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.626+0000 7fe02759e640 1 --2- 172.21.15.105:0/2114089500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe02659c640 1 -- 172.21.15.105:0/2114089500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 msgr2=0x5579e1d30e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe02659c640 1 --2- 172.21.15.105:0/2114089500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d30e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe00ffff640 1 -- 172.21.15.105:0/2643881737 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fe0180cf910 con 0x7fe0281007e0 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe02659c640 1 -- 172.21.15.105:0/2114089500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 msgr2=0x5579e1d2ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe02659c640 1 --2- 172.21.15.105:0/2114089500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d2ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe02659c640 1 -- 172.21.15.105:0/2114089500 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e19a63f0 con 0x5579e1d31500 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe02659c640 1 --2- 172.21.15.105:0/2114089500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d3e370 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00bfd0 tx=0x7fe01c0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0f8a7fb0600509 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe00d7fa640 1 -- 172.21.15.105:0/2114089500 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c011760 con 0x5579e1d31500 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe02759e640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0100a2fa0 0x7fe0100a5460 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe020006010 tx=0x7fe0200093a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe00d7fa640 1 -- 172.21.15.105:0/2114089500 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe01c011db0 con 0x5579e1d31500 2024-03-26T18:00:30.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.627+0000 7fe00d7fa640 1 -- 172.21.15.105:0/2114089500 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c01a380 con 0x5579e1d31500 2024-03-26T18:00:30.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2114089500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 msgr2=0x5579e1d3e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2114089500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d3e370 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00bfd0 tx=0x7fe01c0108e0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2114089500 shutdown_connections 2024-03-26T18:00:30.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2114089500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d3e370 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2114089500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d30e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2114089500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2114089500 >> 172.21.15.105:0/2114089500 conn(0x5579e1d28fa0 msgr2=0x5579e1d293a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2114089500 shutdown_connections 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.628+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2114089500 wait complete. 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 1 -- start start 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 1 Processor -- start 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2e3a0 0x5579e1d40c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d47ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5579e1d3ed70 con 0x5579e1d31500 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5579e1c4e310 con 0x5579e1d2e3a0 2024-03-26T18:00:30.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5579e1d3eef0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe026d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe026d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46258/0 (socket says 172.21.15.105:46258) 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe026d9d640 1 -- 172.21.15.105:0/1496011219 learned_addr learned my addr 172.21.15.105:0/1496011219 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02659c640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2e3a0 0x5579e1d40c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02fdc4740 10 client.0 _ll_register_callbacks cb 0x5579e1c89c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe02759e640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d47ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe026d9d640 1 -- 172.21.15.105:0/1496011219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2e3a0 msgr2=0x5579e1d40c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.629+0000 7fe026d9d640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2e3a0 0x5579e1d40c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe026d9d640 1 -- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 msgr2=0x5579e1d47ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe026d9d640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d47ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe026d9d640 1 -- 172.21.15.105:0/1496011219 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e1a707d0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe02759e640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d47ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe02659c640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2e3a0 0x5579e1d40c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe026d9d640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40720 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe0180983b0 tx=0x7fe01809bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f58b92916673b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe00aff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe00aff5640 1 -- 172.21.15.105:0/1496011219 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01809bfa0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe00aff5640 1 -- 172.21.15.105:0/1496011219 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fe01809b3f0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe02fdc4740 1 -- 172.21.15.105:0/1496011219 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe01c011db0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe00aff5640 1 -- 172.21.15.105:0/1496011219 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe0180a00f0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe02fdc4740 10 client.33049 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.630+0000 7fe02fdc4740 1 -- 172.21.15.105:0/1496011219 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5579e1d4b0e0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe00aff5640 1 -- 172.21.15.105:0/1496011219 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe01c011db0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe00aff5640 1 -- 172.21.15.105:0/1496011219 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe01809ae60 con 0x5579e1d2d9b0 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe02fdc4740 10 client.33049 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe02fdc4740 10 client.33049 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe02fdc4740 20 client.33049 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5579e1d4c7c0 0x5579e1d4ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe02fdc4740 4 client.33049 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:30.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe02fdc4740 1 -- 172.21.15.105:0/1496011219 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579e1c9feb0 con 0x5579e1d4c7c0 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.631+0000 7fe02659c640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5579e1d4c7c0 0x5579e1d4ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.632+0000 7fe02659c640 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5579e1d4c7c0 0x5579e1d4ec50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=603fd1e8fade51ad server_cookie=c4310673cc7c519e in_seq=0 out_seq=0 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.632+0000 7fe00aff5640 10 client.33049 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.632+0000 7fe00aff5640 1 -- 172.21.15.105:0/1496011219 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5579e1c9feb0 con 0x5579e1d4c7c0 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.632+0000 7fe00aff5640 10 client.33049 handle_command_reply: tid=0 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.632+0000 7fe02fdc4740 1 client.33049 shutdown 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 -- 172.21.15.105:0/1496011219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2d9b0 msgr2=0x5579e1d40720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40720 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe0180983b0 tx=0x7fe01809bbf0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 -- 172.21.15.105:0/1496011219 shutdown_connections 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5579e1d4c7c0 0x5579e1d4ec50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d31500 0x5579e1d47ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2e3a0 0x5579e1d40c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/1496011219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40720 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 -- 172.21.15.105:0/1496011219 >> 172.21.15.105:0/1496011219 conn(0x5579e1d28fa0 msgr2=0x5579e1d281c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe00aff5640 0 client.33049 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:00:30.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 -- 172.21.15.105:0/1496011219 shutdown_connections 2024-03-26T18:00:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.633+0000 7fe02fdc4740 1 -- 172.21.15.105:0/1496011219 wait complete. 2024-03-26T18:00:30.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.634+0000 7fe02fdc4740 20 client.33049 trim_cache size 0 max 16384 2024-03-26T18:00:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.655+0000 7fe02fdc4740 1 Processor -- start 2024-03-26T18:00:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.655+0000 7fe02fdc4740 1 -- start start 2024-03-26T18:00:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.655+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.655+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d47c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.655+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d31500 0x5579e1d2c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.655+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281a6b70 con 0x5579e1d2d9b0 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.655+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281a86b0 con 0x5579e1d31500 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.655+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281a7240 con 0x5579e1d2e3a0 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe02659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d47c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45466/0 (socket says 172.21.15.105:45466) 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 -- 172.21.15.105:0/825222329 learned_addr learned my addr 172.21.15.105:0/825222329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe02759e640 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d31500 0x5579e1d2c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 -- 172.21.15.105:0/825222329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 msgr2=0x5579e1d47c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d47c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 -- 172.21.15.105:0/825222329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d31500 msgr2=0x5579e1d2c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d31500 0x5579e1d2c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 -- 172.21.15.105:0/825222329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e1d2e780 con 0x5579e1d2d9b0 2024-03-26T18:00:30.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe02659c640 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d47c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe026d9d640 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40bc0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe01809bca0 tx=0x7fe0180b5200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650a9a7c221fcbbb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe00cff9640 1 -- 172.21.15.105:0/825222329 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01807e070 con 0x5579e1d2d9b0 2024-03-26T18:00:30.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe00cff9640 1 -- 172.21.15.105:0/825222329 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0180a49b0 con 0x5579e1d2d9b0 2024-03-26T18:00:30.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.656+0000 7fe00cff9640 1 -- 172.21.15.105:0/825222329 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018097f30 con 0x5579e1d2d9b0 2024-03-26T18:00:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 -- 172.21.15.105:0/825222329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2d9b0 msgr2=0x5579e1d40bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40bc0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fe01809bca0 tx=0x7fe0180b5200 comp rx=0 tx=0).stop 2024-03-26T18:00:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 -- 172.21.15.105:0/825222329 shutdown_connections 2024-03-26T18:00:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d31500 0x5579e1d2c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d2e3a0 0x5579e1d47c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/825222329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2d9b0 0x5579e1d40bc0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 -- 172.21.15.105:0/825222329 >> 172.21.15.105:0/825222329 conn(0x5579e1d28fa0 msgr2=0x5579e1d2c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 -- 172.21.15.105:0/825222329 shutdown_connections 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.657+0000 7fe02fdc4740 1 -- 172.21.15.105:0/825222329 wait complete. 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 1 -- start start 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 1 Processor -- start 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2e3a0 0x5579e1d45b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d31500 0x5579e1d46b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281a7fe0 con 0x5579e1d2e3a0 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281a7910 con 0x5579e1d2d9b0 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281a4b50 con 0x5579e1d31500 2024-03-26T18:00:30.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02fdc4740 10 client.0 _ll_register_callbacks cb 0x7fe01809be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2e3a0 0x5579e1d45b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2e3a0 0x5579e1d45b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45478/0 (socket says 172.21.15.105:45478) 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02659c640 1 -- 172.21.15.105:0/2214002796 learned_addr learned my addr 172.21.15.105:0/2214002796 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe02759e640 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d31500 0x5579e1d46b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.658+0000 7fe026d9d640 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02659c640 1 -- 172.21.15.105:0/2214002796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d31500 msgr2=0x5579e1d46b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02659c640 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d31500 0x5579e1d46b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02659c640 1 -- 172.21.15.105:0/2214002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 msgr2=0x5579e1d455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02659c640 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02659c640 1 -- 172.21.15.105:0/2214002796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e1d47120 con 0x5579e1d2e3a0 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe026d9d640 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02659c640 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2e3a0 0x5579e1d45b20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00bfa0 tx=0x7fe01c005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a39798edddd685 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe00aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:30.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe00aff5640 1 -- 172.21.15.105:0/2214002796 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c02e070 con 0x5579e1d2e3a0 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2214002796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0180a49b0 con 0x5579e1d2e3a0 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe00aff5640 1 -- 172.21.15.105:0/2214002796 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fe01c005450 con 0x5579e1d2e3a0 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe00aff5640 1 -- 172.21.15.105:0/2214002796 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe01c03a050 con 0x5579e1d2e3a0 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02fdc4740 10 client.23385 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2214002796 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5579e1db2cf0 con 0x5579e1d2e3a0 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.659+0000 7fe00aff5640 1 -- 172.21.15.105:0/2214002796 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0180a49b0 con 0x5579e1d2e3a0 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe00aff5640 1 -- 172.21.15.105:0/2214002796 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe01c004780 con 0x5579e1d2e3a0 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe02fdc4740 10 client.23385 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe02fdc4740 10 client.23385 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe02fdc4740 20 client.23385 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5579e1db34e0 0x5579e1db5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe02fdc4740 4 client.23385 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2214002796 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579e1c9a200 con 0x5579e1db34e0 2024-03-26T18:00:30.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe026d9d640 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5579e1db34e0 0x5579e1db5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe00aff5640 10 client.23385 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.660+0000 7fe026d9d640 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5579e1db34e0 0x5579e1db5900 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2ef26d2b9ff3502 server_cookie=62493e02e3e8e148 in_seq=0 out_seq=0 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe00aff5640 1 -- 172.21.15.105:0/2214002796 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579e1c9a200 con 0x5579e1db34e0 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe00aff5640 10 client.23385 handle_command_reply: tid=0 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 client.23385 shutdown 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2214002796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2e3a0 msgr2=0x5579e1d45b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2e3a0 0x5579e1d45b20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00bfa0 tx=0x7fe01c005b20 comp rx=0 tx=0).stop 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2214002796 shutdown_connections 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5579e1db34e0 0x5579e1db5900 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe00aff5640 0 client.23385 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5579e1d31500 0x5579e1d46b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5579e1d2e3a0 0x5579e1d45b20 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 --2- 172.21.15.105:0/2214002796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5579e1d2d9b0 0x5579e1d455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2214002796 >> 172.21.15.105:0/2214002796 conn(0x5579e1d28fa0 msgr2=0x5579e1d40a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2214002796 shutdown_connections 2024-03-26T18:00:30.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 1 -- 172.21.15.105:0/2214002796 wait complete. 2024-03-26T18:00:30.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.661+0000 7fe02fdc4740 20 client.23385 trim_cache size 0 max 16384 2024-03-26T18:00:30.649 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0100a2fa0 msgr2=0x7fe0100a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0100a2fa0 0x7fe0100a5460 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe020006010 tx=0x7fe0200093a0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0281007e0 msgr2=0x7fe028197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0281007e0 0x7fe028197b00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe01800e9c0 tx=0x7fe01800ee80 comp rx=0 tx=0).stop 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 shutdown_connections 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe0100a2fa0 0x7fe0100a5460 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe028102390 0x7fe02819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0281019d0 0x7fe028198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 --2- 172.21.15.105:0/2643881737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe0281007e0 0x7fe028197b00 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 >> 172.21.15.105:0/2643881737 conn(0x7fe0280fbd70 msgr2=0x7fe0280fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 shutdown_connections 2024-03-26T18:00:30.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.662+0000 7fe02d0f8640 1 -- 172.21.15.105:0/2643881737 wait complete. 2024-03-26T18:00:30.657 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 -- 172.21.15.105:0/4135115319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781075e0 msgr2=0x7fbc78109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/4135115319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781075e0 0x7fbc78109a30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009ca0 tx=0x7fbc7002c100 comp rx=0 tx=0).stop 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 -- 172.21.15.105:0/4135115319 shutdown_connections 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/4135115319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc781075e0 0x7fbc78109a30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/4135115319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc78104cb0 0x7fbc781070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/4135115319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc78102380 0x7fbc78104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 -- 172.21.15.105:0/4135115319 >> 172.21.15.105:0/4135115319 conn(0x7fbc780fbd70 msgr2=0x7fbc780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 -- 172.21.15.105:0/4135115319 shutdown_connections 2024-03-26T18:00:30.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.815+0000 7fbc7fa44640 1 -- 172.21.15.105:0/4135115319 wait complete. 2024-03-26T18:00:30.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.816+0000 7fbc7fa44640 1 Processor -- start 2024-03-26T18:00:30.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.816+0000 7fbc7fa44640 1 -- start start 2024-03-26T18:00:30.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.816+0000 7fbc7fa44640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc78102380 0x7fbc78197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.816+0000 7fbc7fa44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc78104cb0 0x7fbc78198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.816+0000 7fbc7fa44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc781075e0 0x7fbc7819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.816+0000 7fbc7fa44640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7810d430 con 0x7fbc781075e0 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.816+0000 7fbc7fa44640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7810d2b0 con 0x7fbc78102380 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.816+0000 7fbc7fa44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7810d5b0 con 0x7fbc78104cb0 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc78102380 0x7fbc78197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc78102380 0x7fbc78197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45656/0 (socket says 172.21.15.105:45656) 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/3564386150 learned_addr learned my addr 172.21.15.105:0/3564386150 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7dfe1640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc781075e0 0x7fbc7819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc78104cb0 0x7fbc78198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7dfe1640 1 -- 172.21.15.105:0/3564386150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc78104cb0 msgr2=0x7fbc78198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7dfe1640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc78104cb0 0x7fbc78198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7dfe1640 1 -- 172.21.15.105:0/3564386150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc78102380 msgr2=0x7fbc78197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7dfe1640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc78102380 0x7fbc78197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7dfe1640 1 -- 172.21.15.105:0/3564386150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7819f760 con 0x7fbc781075e0 2024-03-26T18:00:30.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc78102380 0x7fbc78197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.817+0000 7fbc7dfe1640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc781075e0 0x7fbc7819f0f0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fbc7002c710 tx=0x7fbc70035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.818+0000 7fbc6e7fc640 1 -- 172.21.15.105:0/3564386150 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70039040 con 0x7fbc781075e0 2024-03-26T18:00:30.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.818+0000 7fbc6e7fc640 1 -- 172.21.15.105:0/3564386150 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc70044070 con 0x7fbc781075e0 2024-03-26T18:00:30.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.818+0000 7fbc7fa44640 1 -- 172.21.15.105:0/3564386150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7819f9f0 con 0x7fbc781075e0 2024-03-26T18:00:30.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.818+0000 7fbc6e7fc640 1 -- 172.21.15.105:0/3564386150 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70004330 con 0x7fbc781075e0 2024-03-26T18:00:30.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.818+0000 7fbc7fa44640 1 -- 172.21.15.105:0/3564386150 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc78100170 con 0x7fbc781075e0 2024-03-26T18:00:30.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc8262c740 1 Processor -- start 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc6e7fc640 1 -- 172.21.15.105:0/3564386150 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fbc700044d0 con 0x7fbc781075e0 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc8262c740 1 -- start start 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd98a30 0x564d0cd98e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd99420 0x564d0cd9df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc8262c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564d0ccfd970 con 0x564d0cd99420 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc8262c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564d0cca5d70 con 0x564d0cd98a30 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc8262c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d0ccf9920 con 0x564d0cd9e590 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd99420 0x564d0cd9df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd99420 0x564d0cd9df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45518/0 (socket says 172.21.15.105:45518) 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/242126997 learned_addr learned my addr 172.21.15.105:0/242126997 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.821+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd98a30 0x564d0cd98e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/242126997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 msgr2=0x564d0cda0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/242126997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd98a30 msgr2=0x564d0cd98e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd98a30 0x564d0cd98e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/242126997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0ca113f0 con 0x564d0cd99420 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd99420 0x564d0cd9df20 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011650 tx=0x7fbc74009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84614308096ccba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc6e7fc640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc640a7480 0x7fbc640a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc637fe640 1 -- 172.21.15.105:0/242126997 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7401e070 con 0x564d0cd99420 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc637fe640 1 -- 172.21.15.105:0/242126997 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc7400fb70 con 0x564d0cd99420 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc637fe640 1 -- 172.21.15.105:0/242126997 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74019c60 con 0x564d0cd99420 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc6e7fc640 1 -- 172.21.15.105:0/3564386150 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fbc700f1750 con 0x7fbc781075e0 2024-03-26T18:00:30.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.822+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc640a7480 0x7fbc640a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc640a7480 0x7fbc640a9940 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x564d0cd99c80 tx=0x7fbc68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 -- 172.21.15.105:0/242126997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd99420 msgr2=0x564d0cd9df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd99420 0x564d0cd9df20 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011650 tx=0x7fbc74009f90 comp rx=0 tx=0).stop 2024-03-26T18:00:30.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 -- 172.21.15.105:0/242126997 shutdown_connections 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd99420 0x564d0cd9df20 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 --2- 172.21.15.105:0/242126997 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd98a30 0x564d0cd98e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 -- 172.21.15.105:0/242126997 >> 172.21.15.105:0/242126997 conn(0x564d0cd93fa0 msgr2=0x564d0cd943a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 -- 172.21.15.105:0/242126997 shutdown_connections 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.823+0000 7fbc8262c740 1 -- 172.21.15.105:0/242126997 wait complete. 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 1 -- start start 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 1 Processor -- start 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cda2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd99420 0x564d0cda3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd9e590 0x564d0cdaa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564d0cda1540 con 0x564d0cd98a30 2024-03-26T18:00:30.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564d0ccb9310 con 0x564d0cd9e590 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564d0cda16c0 con 0x564d0cd99420 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc8262c740 10 client.0 _ll_register_callbacks cb 0x564d0ccf4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cda2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cda2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45522/0 (socket says 172.21.15.105:45522) 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2151405681 learned_addr learned my addr 172.21.15.105:0/2151405681 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd99420 0x564d0cda3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.824+0000 7fbc7dfe1640 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd9e590 0x564d0cdaa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2151405681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd99420 msgr2=0x564d0cda3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd99420 0x564d0cda3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2151405681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd9e590 msgr2=0x564d0cdaa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd9e590 0x564d0cdaa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2151405681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0cadb7d0 con 0x564d0cd98a30 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cda2f40 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fbc74000c00 tx=0x7fbc74009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de765d68e02e2aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:30.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2151405681 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7401e040 con 0x564d0cd98a30 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2151405681 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fbc74016070 con 0x564d0cd98a30 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc8262c740 1 -- 172.21.15.105:0/2151405681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc7400fb70 con 0x564d0cd98a30 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2151405681 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbc7400f840 con 0x564d0cd98a30 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc8262c740 10 client.23397 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.825+0000 7fbc8262c740 1 -- 172.21.15.105:0/2151405681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564d0cdad830 con 0x564d0cd98a30 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2151405681 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc7400fb70 con 0x564d0cd98a30 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2151405681 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbc74021070 con 0x564d0cd98a30 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc8262c740 10 client.23397 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc8262c740 10 client.23397 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc8262c740 20 client.23397 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x564d0cdaecd0 0x564d0cdb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc8262c740 4 client.23397 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:30.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc8262c740 1 -- 172.21.15.105:0/2151405681 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d0cd0aeb0 con 0x564d0cdaecd0 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.826+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x564d0cdaecd0 0x564d0cdb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x564d0cdaecd0 0x564d0cdb1140 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45d5bada8e6720e4 server_cookie=dc20642d34a3aff0 in_seq=0 out_seq=0 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc60ff9640 10 client.23397 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2151405681 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564d0cd0aeb0 con 0x564d0cdaecd0 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc60ff9640 10 client.23397 handle_command_reply: tid=0 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 client.23397 shutdown 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 -- 172.21.15.105:0/2151405681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 msgr2=0x564d0cda2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cda2f40 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fbc74000c00 tx=0x7fbc74009ef0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 -- 172.21.15.105:0/2151405681 shutdown_connections 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x564d0cdaecd0 0x564d0cdb1140 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd9e590 0x564d0cdaa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd99420 0x564d0cda3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2151405681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cda2f40 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.827+0000 7fbc8262c740 1 -- 172.21.15.105:0/2151405681 >> 172.21.15.105:0/2151405681 conn(0x564d0cd93fa0 msgr2=0x564d0cd931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.828+0000 7fbc60ff9640 0 client.23397 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:00:30.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.828+0000 7fbc8262c740 1 -- 172.21.15.105:0/2151405681 shutdown_connections 2024-03-26T18:00:30.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.828+0000 7fbc8262c740 1 -- 172.21.15.105:0/2151405681 wait complete. 2024-03-26T18:00:30.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.828+0000 7fbc8262c740 20 client.23397 trim_cache size 0 max 16384 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc8262c740 1 Processor -- start 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc8262c740 1 -- start start 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd98a30 0x564d0cd97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cdaa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd9e590 0x564d0cda2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc8262c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781a6be0 con 0x564d0cd9e590 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc8262c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781a8720 con 0x564d0cd99420 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc8262c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781a72b0 con 0x564d0cd98a30 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd98a30 0x564d0cd97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd98a30 0x564d0cd97910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46322/0 (socket says 172.21.15.105:46322) 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc7cfdf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cdaa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2546390269 learned_addr learned my addr 172.21.15.105:0/2546390269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.845+0000 7fbc7dfe1640 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd9e590 0x564d0cda2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2546390269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 msgr2=0x564d0cdaa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cdaa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2546390269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd9e590 msgr2=0x564d0cda2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd9e590 0x564d0cda2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2546390269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0cd8f820 con 0x564d0cd98a30 2024-03-26T18:00:30.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc7dfe1640 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd9e590 0x564d0cda2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd98a30 0x564d0cd97910 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbc740097f0 tx=0x7fbc74005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41428331665d48b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc62ffd640 1 -- 172.21.15.105:0/2546390269 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74016070 con 0x564d0cd98a30 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc62ffd640 1 -- 172.21.15.105:0/2546390269 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc74019ab0 con 0x564d0cd98a30 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.846+0000 7fbc62ffd640 1 -- 172.21.15.105:0/2546390269 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7401e030 con 0x564d0cd98a30 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 -- 172.21.15.105:0/2546390269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd98a30 msgr2=0x564d0cd97910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd98a30 0x564d0cd97910 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbc740097f0 tx=0x7fbc74005700 comp rx=0 tx=0).stop 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 -- 172.21.15.105:0/2546390269 shutdown_connections 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd9e590 0x564d0cda2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cdaa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2546390269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd98a30 0x564d0cd97910 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 -- 172.21.15.105:0/2546390269 >> 172.21.15.105:0/2546390269 conn(0x564d0cd93fa0 msgr2=0x564d0cd97200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 -- 172.21.15.105:0/2546390269 shutdown_connections 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 -- 172.21.15.105:0/2546390269 wait complete. 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 -- start start 2024-03-26T18:00:30.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.847+0000 7fbc8262c740 1 Processor -- start 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cdaa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cda6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc8262c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc8262c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781a8050 con 0x564d0cd98a30 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc8262c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781a7980 con 0x564d0cd99420 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc8262c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc781a4bc0 con 0x564d0cd9e590 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc8262c740 10 client.0 _ll_register_callbacks cb 0x7fbc7401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cdaa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7d7e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cdaa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45534/0 (socket says 172.21.15.105:45534) 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7cfdf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cda6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7d7e0640 1 -- 172.21.15.105:0/2808828076 learned_addr learned my addr 172.21.15.105:0/2808828076 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7dfe1640 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7cfdf640 1 -- 172.21.15.105:0/2808828076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 msgr2=0x564d0cda7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7cfdf640 1 -- 172.21.15.105:0/2808828076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 msgr2=0x564d0cdaa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cdaa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7cfdf640 1 -- 172.21.15.105:0/2808828076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0cdaa650 con 0x564d0cd99420 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.848+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cdaa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc7cfdf640 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cda6ac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x564d0cda4420 tx=0x7fbc680096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c15046d9f595952 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2808828076 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680044a0 con 0x564d0cd99420 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc8262c740 1 -- 172.21.15.105:0/2808828076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc74019ab0 con 0x564d0cd99420 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2808828076 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fbc6802c420 con 0x564d0cd99420 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2808828076 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbc6802c620 con 0x564d0cd99420 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc8262c740 10 client.33083 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc8262c740 1 -- 172.21.15.105:0/2808828076 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564d0ce14020 con 0x564d0cd99420 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.849+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2808828076 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc74019ab0 con 0x564d0cd99420 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2808828076 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbc7400fd20 con 0x564d0cd99420 2024-03-26T18:00:30.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc8262c740 10 client.33083 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc8262c740 10 client.33083 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc8262c740 20 client.33083 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x564d0ce149d0 0x564d0ce16df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc8262c740 4 client.33083 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc8262c740 1 -- 172.21.15.105:0/2808828076 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d0cd05200 con 0x564d0ce149d0 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x564d0ce149d0 0x564d0ce16df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc7d7e0640 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x564d0ce149d0 0x564d0ce16df0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9b9b7b518b3749b server_cookie=51348782da888ec3 in_seq=0 out_seq=0 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.850+0000 7fbc60ff9640 10 client.33083 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc60ff9640 1 -- 172.21.15.105:0/2808828076 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d0cd05200 con 0x564d0ce149d0 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc60ff9640 10 client.33083 handle_command_reply: tid=0 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 client.33083 shutdown 2024-03-26T18:00:30.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 -- 172.21.15.105:0/2808828076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 msgr2=0x564d0cda6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cda6ac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x564d0cda4420 tx=0x7fbc680096c0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 -- 172.21.15.105:0/2808828076 shutdown_connections 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x564d0ce149d0 0x564d0ce16df0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564d0cd9e590 0x564d0cda7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564d0cd99420 0x564d0cda6ac0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 --2- 172.21.15.105:0/2808828076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564d0cd98a30 0x564d0cdaa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 -- 172.21.15.105:0/2808828076 >> 172.21.15.105:0/2808828076 conn(0x564d0cd93fa0 msgr2=0x564d0cd97a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc60ff9640 0 client.33083 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 -- 172.21.15.105:0/2808828076 shutdown_connections 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 1 -- 172.21.15.105:0/2808828076 wait complete. 2024-03-26T18:00:30.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.851+0000 7fbc8262c740 20 client.33083 trim_cache size 0 max 16384 2024-03-26T18:00:30.839 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:00:30.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 -- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc640a7480 msgr2=0x7fbc640a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc640a7480 0x7fbc640a9940 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x564d0cd99c80 tx=0x7fbc68005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:30.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 -- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc781075e0 msgr2=0x7fbc7819f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:30.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc781075e0 0x7fbc7819f0f0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fbc7002c710 tx=0x7fbc70035d60 comp rx=0 tx=0).stop 2024-03-26T18:00:30.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 -- 172.21.15.105:0/3564386150 shutdown_connections 2024-03-26T18:00:30.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fbc640a7480 0x7fbc640a9940 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbc781075e0 0x7fbc7819f0f0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc78104cb0 0x7fbc78198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 --2- 172.21.15.105:0/3564386150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbc78102380 0x7fbc78197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:30.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 -- 172.21.15.105:0/3564386150 >> 172.21.15.105:0/3564386150 conn(0x7fbc780fbd70 msgr2=0x7fbc78102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:30.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 -- 172.21.15.105:0/3564386150 shutdown_connections 2024-03-26T18:00:30.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:30.852+0000 7fbc7fa44640 1 -- 172.21.15.105:0/3564386150 wait complete. 2024-03-26T18:00:30.840 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:30 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3088848931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:00:30.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:30 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3088848931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:00:30.848 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T18:00:30.905 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:30 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3088848931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:00:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3552844406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e14101dd0 msgr2=0x7f2e141021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3552844406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e14101dd0 0x7f2e141021f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009860 tx=0x7f2e0402bb70 comp rx=0 tx=0).stop 2024-03-26T18:00:31.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3552844406 shutdown_connections 2024-03-26T18:00:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3552844406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e141013e0 0x7f2e14108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3552844406 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e14100a20 0x7f2e14100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3552844406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e14101dd0 0x7f2e141021f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3552844406 >> 172.21.15.105:0/3552844406 conn(0x7f2e140fb4b0 msgr2=0x7f2e140fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3552844406 shutdown_connections 2024-03-26T18:00:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3552844406 wait complete. 2024-03-26T18:00:31.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.018+0000 7f2e1a11b640 1 Processor -- start 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e1a11b640 1 -- start start 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e1a11b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e14100a20 0x7f2e141047d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e1a11b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e141013e0 0x7f2e14102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e1a11b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e14101dd0 0x7f2e14103350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e1a11b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1410c2e0 con 0x7f2e141013e0 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e1a11b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1410c160 con 0x7f2e14101dd0 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e1a11b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1410c460 con 0x7f2e14100a20 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e141013e0 0x7f2e14102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e141013e0 0x7f2e14102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45556/0 (socket says 172.21.15.105:45556) 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e14101dd0 0x7f2e14103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e12ffd640 1 -- 172.21.15.105:0/3946292136 learned_addr learned my addr 172.21.15.105:0/3946292136 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e137fe640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e14100a20 0x7f2e141047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e12ffd640 1 -- 172.21.15.105:0/3946292136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e14100a20 msgr2=0x7f2e141047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e14100a20 0x7f2e141047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e12ffd640 1 -- 172.21.15.105:0/3946292136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e14101dd0 msgr2=0x7f2e14103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e14101dd0 0x7f2e14103350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.019+0000 7f2e12ffd640 1 -- 172.21.15.105:0/3946292136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e14103be0 con 0x7f2e141013e0 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.020+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e141013e0 0x7f2e14102e10 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800e9f0 tx=0x7f2e0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.020+0000 7f2e10ff9640 1 -- 172.21.15.105:0/3946292136 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800ccc0 con 0x7f2e141013e0 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.020+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3946292136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e141a1cd0 con 0x7f2e141013e0 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.020+0000 7f2e10ff9640 1 -- 172.21.15.105:0/3946292136 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e080035f0 con 0x7f2e141013e0 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.020+0000 7f2e10ff9640 1 -- 172.21.15.105:0/3946292136 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800ce60 con 0x7f2e141013e0 2024-03-26T18:00:31.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.020+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3946292136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e141a21c0 con 0x7f2e141013e0 2024-03-26T18:00:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.022+0000 7f2e10ff9640 1 -- 172.21.15.105:0/3946292136 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2e080037a0 con 0x7f2e141013e0 2024-03-26T18:00:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e1cde5740 1 Processor -- start 2024-03-26T18:00:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e1cde5740 1 -- start start 2024-03-26T18:00:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749ec420 0x55b2749f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b274950970 con 0x55b2749eba30 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2748f8d70 con 0x55b2749f1590 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b27494c920 con 0x55b2749ec420 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e13fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749ebe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45562/0 (socket says 172.21.15.105:45562) 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e13fff640 1 -- 172.21.15.105:0/2087467368 learned_addr learned my addr 172.21.15.105:0/2087467368 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e10ff9640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2dfc0a2f20 0x7f2dfc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e137fe640 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749ec420 0x55b2749f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e13fff640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2dfc0a2f20 0x7f2dfc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e10ff9640 1 -- 172.21.15.105:0/3946292136 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2e080cf660 con 0x7f2e141013e0 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e13fff640 1 -- 172.21.15.105:0/2087467368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749ec420 msgr2=0x55b2749f0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e13fff640 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749ec420 0x55b2749f0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e13fff640 1 -- 172.21.15.105:0/2087467368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 msgr2=0x55b2749f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.023+0000 7f2e13fff640 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749f3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.024+0000 7f2e13fff640 1 -- 172.21.15.105:0/2087467368 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2746643f0 con 0x55b2749eba30 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.024+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.024+0000 7f2e13fff640 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749ebe50 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c005c40 tx=0x7f2e0c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01cd2fd08502c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.024+0000 7f2df9ffb640 1 -- 172.21.15.105:0/2087467368 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c022020 con 0x55b2749eba30 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.024+0000 7f2e13fff640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2dfc0a2f20 0x7f2dfc0a53e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c023000 tx=0x7f2e0c021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.024+0000 7f2df9ffb640 1 -- 172.21.15.105:0/2087467368 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e0c013170 con 0x55b2749eba30 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.024+0000 7f2df9ffb640 1 -- 172.21.15.105:0/2087467368 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c010690 con 0x55b2749eba30 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2087467368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 msgr2=0x55b2749ebe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749ebe50 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c005c40 tx=0x7f2e0c00f3c0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2087467368 shutdown_connections 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749ec420 0x55b2749f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/2087467368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749ebe50 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2087467368 >> 172.21.15.105:0/2087467368 conn(0x55b2749e6fa0 msgr2=0x55b2749e73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2087467368 shutdown_connections 2024-03-26T18:00:31.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.025+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2087467368 wait complete. 2024-03-26T18:00:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 1 -- start start 2024-03-26T18:00:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 1 Processor -- start 2024-03-26T18:00:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 0x55b2749f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b2749f4540 con 0x55b2749ec420 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b27490c310 con 0x55b2749f1590 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b2749f46c0 con 0x55b2749eba30 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e1cde5740 10 client.0 _ll_register_callbacks cb 0x55b274947c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749fd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45742/0 (socket says 172.21.15.105:45742) 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e137fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 0x55b2749f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.026+0000 7f2e13fff640 1 -- 172.21.15.105:0/3416181522 learned_addr learned my addr 172.21.15.105:0/3416181522 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e13fff640 1 -- 172.21.15.105:0/3416181522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 msgr2=0x55b2749f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e13fff640 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 0x55b2749f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e13fff640 1 -- 172.21.15.105:0/3416181522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 msgr2=0x55b2749f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e13fff640 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e13fff640 1 -- 172.21.15.105:0/3416181522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b27472e7d0 con 0x55b2749f1590 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e13fff640 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749fd4f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0133d0 tx=0x7f2e0c00f9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb56497f6e90de6b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2df2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2df2ffd640 1 -- 172.21.15.105:0/3416181522 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c010640 con 0x55b2749f1590 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e1cde5740 1 -- 172.21.15.105:0/3416181522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0c013170 con 0x55b2749f1590 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2df2ffd640 1 -- 172.21.15.105:0/3416181522 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2e0c0109e0 con 0x55b2749f1590 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2df2ffd640 1 -- 172.21.15.105:0/3416181522 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2e0c010be0 con 0x55b2749f1590 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e1cde5740 10 client.33095 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2e1cde5740 1 -- 172.21.15.105:0/3416181522 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b274a00920 con 0x55b2749f1590 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.027+0000 7f2df2ffd640 1 -- 172.21.15.105:0/3416181522 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e0c013170 con 0x55b2749f1590 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2df2ffd640 1 -- 172.21.15.105:0/3416181522 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2e0c0344e0 con 0x55b2749f1590 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2e1cde5740 10 client.33095 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:31.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2e1cde5740 10 client.33095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:00:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2e1cde5740 20 client.33095 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b274a01d30 0x55b274a04190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2e1cde5740 4 client.33095 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2e1cde5740 1 -- 172.21.15.105:0/3416181522 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b27495deb0 con 0x55b274a01d30 2024-03-26T18:00:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2e137fe640 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b274a01d30 0x55b274a04190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2df2ffd640 10 client.33095 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:00:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.028+0000 7f2e137fe640 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b274a01d30 0x55b274a04190 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83bf664c86af0a4c server_cookie=ddd4f9709b3861b3 in_seq=0 out_seq=0 2024-03-26T18:00:31.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.029+0000 7f2df2ffd640 1 -- 172.21.15.105:0/3416181522 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b27495deb0 con 0x55b274a01d30 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.029+0000 7f2df2ffd640 10 client.33095 handle_command_reply: tid=0 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.029+0000 7f2e1cde5740 1 client.33095 shutdown 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 -- 172.21.15.105:0/3416181522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 msgr2=0x55b2749fd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749fd4f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0133d0 tx=0x7f2e0c00f9e0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 -- 172.21.15.105:0/3416181522 shutdown_connections 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b274a01d30 0x55b274a04190 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749fd4f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/3416181522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 0x55b2749f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 -- 172.21.15.105:0/3416181522 >> 172.21.15.105:0/3416181522 conn(0x55b2749e6fa0 msgr2=0x55b2749e61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2df2ffd640 0 client.33095 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:00:31.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 -- 172.21.15.105:0/3416181522 shutdown_connections 2024-03-26T18:00:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 1 -- 172.21.15.105:0/3416181522 wait complete. 2024-03-26T18:00:31.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.030+0000 7f2e1cde5740 20 client.33095 trim_cache size 0 max 16384 2024-03-26T18:00:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e1cde5740 1 Processor -- start 2024-03-26T18:00:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e1cde5740 1 -- start start 2024-03-26T18:00:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 0x55b2749e2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749e5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141ab0c0 con 0x55b2749ec420 2024-03-26T18:00:31.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141acc00 con 0x55b2749f1590 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141ab790 con 0x55b2749eba30 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749fd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45566/0 (socket says 172.21.15.105:45566) 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e12ffd640 1 -- 172.21.15.105:0/2831009960 learned_addr learned my addr 172.21.15.105:0/2831009960 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e137fe640 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 0x55b2749e2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e12ffd640 1 -- 172.21.15.105:0/2831009960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 msgr2=0x55b2749e2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 0x55b2749e2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e12ffd640 1 -- 172.21.15.105:0/2831009960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 msgr2=0x55b2749e5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749e5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e12ffd640 1 -- 172.21.15.105:0/2831009960 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2749e2820 con 0x55b2749ec420 2024-03-26T18:00:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.051+0000 7f2e13fff640 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.052+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749fd4b0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009530 tx=0x7f2e080cf960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce688b0729685e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.052+0000 7f2df97fa640 1 -- 172.21.15.105:0/2831009960 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0807a040 con 0x55b2749ec420 2024-03-26T18:00:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.052+0000 7f2df97fa640 1 -- 172.21.15.105:0/2831009960 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e0809fea0 con 0x55b2749ec420 2024-03-26T18:00:31.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.052+0000 7f2df97fa640 1 -- 172.21.15.105:0/2831009960 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08097aa0 con 0x55b2749ec420 2024-03-26T18:00:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.053+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2831009960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 msgr2=0x55b2749fd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.053+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749fd4b0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009530 tx=0x7f2e080cf960 comp rx=0 tx=0).stop 2024-03-26T18:00:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.053+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2831009960 shutdown_connections 2024-03-26T18:00:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.053+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749f1590 0x55b2749e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.053+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749ec420 0x55b2749fd4b0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.054+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/2831009960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749eba30 0x55b2749e2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.054+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2831009960 >> 172.21.15.105:0/2831009960 conn(0x55b2749e6fa0 msgr2=0x55b2749ed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.054+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2831009960 shutdown_connections 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.054+0000 7f2e1cde5740 1 -- 172.21.15.105:0/2831009960 wait complete. 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.054+0000 7f2e1cde5740 1 -- start start 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e1cde5740 1 Processor -- start 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749fa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749ec420 0x55b2749fada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e1cde5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749f1590 0x55b2749fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141ac530 con 0x55b2749eba30 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141abe60 con 0x55b2749ec420 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e1cde5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141a90a0 con 0x55b2749f1590 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e1cde5740 10 client.0 _ll_register_callbacks cb 0x7f2e080046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749ec420 0x55b2749fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e137fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e12ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749ec420 0x55b2749fada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45756/0 (socket says 172.21.15.105:45756) 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e12ffd640 1 -- 172.21.15.105:0/4023231110 learned_addr learned my addr 172.21.15.105:0/4023231110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.055+0000 7f2e13fff640 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749f1590 0x55b2749fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e12ffd640 1 -- 172.21.15.105:0/4023231110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749f1590 msgr2=0x55b2749fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749f1590 0x55b2749fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e12ffd640 1 -- 172.21.15.105:0/4023231110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 msgr2=0x55b2749fa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749fa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e12ffd640 1 -- 172.21.15.105:0/4023231110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2749fc3a0 con 0x55b2749ec420 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e12ffd640 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749ec420 0x55b2749fada0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2e0809bce0 tx=0x7f2e0809be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7fc71f7fd4fc14 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2df2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2df2ffd640 1 -- 172.21.15.105:0/4023231110 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080a4540 con 0x55b2749ec420 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2df2ffd640 1 -- 172.21.15.105:0/4023231110 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2e0809b1a0 con 0x55b2749ec420 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e1cde5740 1 -- 172.21.15.105:0/4023231110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e0809fea0 con 0x55b2749ec420 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2df2ffd640 1 -- 172.21.15.105:0/4023231110 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2e0809b3a0 con 0x55b2749ec420 2024-03-26T18:00:31.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e1cde5740 10 client.33101 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.056+0000 7f2e1cde5740 1 -- 172.21.15.105:0/4023231110 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b2749fd710 con 0x55b2749ec420 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.057+0000 7f2df2ffd640 1 -- 172.21.15.105:0/4023231110 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e0809fea0 con 0x55b2749ec420 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.058+0000 7f2df2ffd640 1 -- 172.21.15.105:0/4023231110 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2e08097bc0 con 0x55b2749ec420 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.058+0000 7f2e1cde5740 10 client.33101 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.058+0000 7f2e1cde5740 10 client.33101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.058+0000 7f2e1cde5740 20 client.33101 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.058+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b274a68700 0x55b274a6ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.058+0000 7f2e1cde5740 4 client.33101 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.058+0000 7f2e1cde5740 1 -- 172.21.15.105:0/4023231110 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b274958200 con 0x55b274a68700 2024-03-26T18:00:31.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.058+0000 7f2e137fe640 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b274a68700 0x55b274a6ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.060+0000 7f2df2ffd640 10 client.33101 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.060+0000 7f2e137fe640 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b274a68700 0x55b274a6ab20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdb648cfbdf69dfa server_cookie=9c1aca25d5324ca7 in_seq=0 out_seq=0 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2df2ffd640 1 -- 172.21.15.105:0/4023231110 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b274958200 con 0x55b274a68700 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2df2ffd640 10 client.33101 handle_command_reply: tid=0 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 client.33101 shutdown 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 -- 172.21.15.105:0/4023231110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749ec420 msgr2=0x55b2749fada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749ec420 0x55b2749fada0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2e0809bce0 tx=0x7f2e0809be80 comp rx=0 tx=0).stop 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 -- 172.21.15.105:0/4023231110 shutdown_connections 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b2749f1590 0x55b2749fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b274a68700 0x55b274a6ab20 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2749ec420 0x55b2749fada0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 --2- 172.21.15.105:0/4023231110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b2749eba30 0x55b2749fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2df2ffd640 0 client.33101 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:00:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 -- 172.21.15.105:0/4023231110 >> 172.21.15.105:0/4023231110 conn(0x55b2749e6fa0 msgr2=0x55b2749ea200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 -- 172.21.15.105:0/4023231110 shutdown_connections 2024-03-26T18:00:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 1 -- 172.21.15.105:0/4023231110 wait complete. 2024-03-26T18:00:31.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.061+0000 7f2e1cde5740 20 client.33101 trim_cache size 0 max 16384 2024-03-26T18:00:31.046 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2dfc0a2f20 msgr2=0x7f2dfc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2dfc0a2f20 0x7f2dfc0a53e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c023000 tx=0x7f2e0c021000 comp rx=0 tx=0).stop 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e141013e0 msgr2=0x7f2e14102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e141013e0 0x7f2e14102e10 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800e9f0 tx=0x7f2e0800eeb0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3946292136 shutdown_connections 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2dfc0a2f20 0x7f2dfc0a53e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2e14101dd0 0x7f2e14103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e141013e0 0x7f2e14102e10 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 --2- 172.21.15.105:0/3946292136 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e14100a20 0x7f2e141047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3946292136 >> 172.21.15.105:0/3946292136 conn(0x7f2e140fb4b0 msgr2=0x7f2e140ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3946292136 shutdown_connections 2024-03-26T18:00:31.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.062+0000 7f2e1a11b640 1 -- 172.21.15.105:0/3946292136 wait complete. 2024-03-26T18:00:31.058 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.217+0000 7f0153ee8640 1 -- 172.21.15.105:0/123326749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c104cd0 msgr2=0x7f014c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.217+0000 7f0153ee8640 1 --2- 172.21.15.105:0/123326749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c104cd0 0x7f014c1070c0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f013c009900 tx=0x7f013c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.217+0000 7f0153ee8640 1 -- 172.21.15.105:0/123326749 shutdown_connections 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.217+0000 7f0153ee8640 1 --2- 172.21.15.105:0/123326749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f014c107600 0x7f014c1099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.217+0000 7f0153ee8640 1 --2- 172.21.15.105:0/123326749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c104cd0 0x7f014c1070c0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.217+0000 7f0153ee8640 1 --2- 172.21.15.105:0/123326749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f014c1023a0 0x7f014c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.217+0000 7f0153ee8640 1 -- 172.21.15.105:0/123326749 >> 172.21.15.105:0/123326749 conn(0x7f014c0fbd30 msgr2=0x7f014c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.217+0000 7f0153ee8640 1 -- 172.21.15.105:0/123326749 shutdown_connections 2024-03-26T18:00:31.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 -- 172.21.15.105:0/123326749 wait complete. 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 Processor -- start 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 -- start start 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f014c1023a0 0x7f014c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f014c104cd0 0x7f014c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c107600 0x7f014c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f014c10d430 con 0x7f014c107600 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f014c10d2b0 con 0x7f014c1023a0 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.218+0000 7f0153ee8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f014c10d5b0 con 0x7f014c104cd0 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c107600 0x7f014c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0151c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f014c1023a0 0x7f014c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0151483640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f014c104cd0 0x7f014c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c107600 0x7f014c19f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45580/0 (socket says 172.21.15.105:45580) 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 -- 172.21.15.105:0/2224968362 learned_addr learned my addr 172.21.15.105:0/2224968362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 -- 172.21.15.105:0/2224968362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f014c104cd0 msgr2=0x7f014c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f014c104cd0 0x7f014c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 -- 172.21.15.105:0/2224968362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f014c1023a0 msgr2=0x7f014c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f014c1023a0 0x7f014c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 -- 172.21.15.105:0/2224968362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c19f7b0 con 0x7f014c107600 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0152485640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c107600 0x7f014c19f140 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f0144010ea0 tx=0x7f0144010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.219+0000 7f0142ffd640 1 -- 172.21.15.105:0/2224968362 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144004b20 con 0x7f014c107600 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.220+0000 7f0142ffd640 1 -- 172.21.15.105:0/2224968362 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0144004cc0 con 0x7f014c107600 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.220+0000 7f0153ee8640 1 -- 172.21.15.105:0/2224968362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c1001f0 con 0x7f014c107600 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.220+0000 7f0153ee8640 1 -- 172.21.15.105:0/2224968362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c100710 con 0x7f014c107600 2024-03-26T18:00:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.220+0000 7f0142ffd640 1 -- 172.21.15.105:0/2224968362 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144019700 con 0x7f014c107600 2024-03-26T18:00:31.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.223+0000 7f0142ffd640 1 -- 172.21.15.105:0/2224968362 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f0144005020 con 0x7f014c107600 2024-03-26T18:00:31.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.224+0000 7f0142ffd640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01380a3020 0x7f01380a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.224+0000 7f0151c84640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01380a3020 0x7f01380a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.224+0000 7f0142ffd640 1 -- 172.21.15.105:0/2224968362 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f01440d2fe0 con 0x7f014c107600 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.224+0000 7f0156baa740 1 Processor -- start 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.224+0000 7f0151c84640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01380a3020 0x7f01380a54e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0148009bd0 tx=0x7f0148005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0156baa740 1 -- start start 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0156baa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecd9a30 0x561d2ecd9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0156baa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecda420 0x561d2ecdef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0156baa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2ece1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0156baa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561d2ec3e970 con 0x561d2ecda420 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0156baa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561d2ebe6d70 con 0x561d2ecd9a30 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0156baa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561d2ec3a920 con 0x561d2ecdf590 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151483640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecda420 0x561d2ecdef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecd9a30 0x561d2ecd9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0152485640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2ece1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151483640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecda420 0x561d2ecdef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45594/0 (socket says 172.21.15.105:45594) 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151483640 1 -- 172.21.15.105:0/1269954616 learned_addr learned my addr 172.21.15.105:0/1269954616 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0152485640 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2ece1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46424/0 (socket says 172.21.15.105:46424) 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151c84640 1 -- 172.21.15.105:0/1269954616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 msgr2=0x561d2ece1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151c84640 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2ece1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151c84640 1 -- 172.21.15.105:0/1269954616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecda420 msgr2=0x561d2ecdef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151c84640 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecda420 0x561d2ecdef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151c84640 1 -- 172.21.15.105:0/1269954616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2e9523f0 con 0x561d2ecd9a30 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.225+0000 7f0151483640 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecda420 0x561d2ecdef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.226+0000 7f0151c84640 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecd9a30 0x561d2ecd9e50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0148009fd0 tx=0x7f0148014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275f4aa824c0383f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.226+0000 7f0137fff640 1 -- 172.21.15.105:0/1269954616 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148005020 con 0x561d2ecd9a30 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.226+0000 7f0137fff640 1 -- 172.21.15.105:0/1269954616 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01480047a0 con 0x561d2ecd9a30 2024-03-26T18:00:31.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.226+0000 7f0137fff640 1 -- 172.21.15.105:0/1269954616 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014801e680 con 0x561d2ecd9a30 2024-03-26T18:00:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 -- 172.21.15.105:0/1269954616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecd9a30 msgr2=0x561d2ecd9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecd9a30 0x561d2ecd9e50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0148009fd0 tx=0x7f0148014000 comp rx=0 tx=0).stop 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 -- 172.21.15.105:0/1269954616 shutdown_connections 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2ece1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecda420 0x561d2ecdef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 --2- 172.21.15.105:0/1269954616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecd9a30 0x561d2ecd9e50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 -- 172.21.15.105:0/1269954616 >> 172.21.15.105:0/1269954616 conn(0x561d2ecd4fa0 msgr2=0x561d2ecd53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 -- 172.21.15.105:0/1269954616 shutdown_connections 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 -- 172.21.15.105:0/1269954616 wait complete. 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 -- start start 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.227+0000 7f0156baa740 1 Processor -- start 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0156baa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecd9a30 0x561d2ece3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0156baa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0156baa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2eceb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0156baa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561d2ece2540 con 0x561d2ecdf590 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0156baa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561d2ebfa310 con 0x561d2ecda420 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0156baa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561d2ece26c0 con 0x561d2ecd9a30 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0151c84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecd9a30 0x561d2ece3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0151483640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0156baa740 10 client.0 _ll_register_callbacks cb 0x561d2ec35c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0151c84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecd9a30 0x561d2ece3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46426/0 (socket says 172.21.15.105:46426) 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0151c84640 1 -- 172.21.15.105:0/2758526628 learned_addr learned my addr 172.21.15.105:0/2758526628 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.228+0000 7f0152485640 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2eceb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0151c84640 1 -- 172.21.15.105:0/2758526628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 msgr2=0x561d2ece4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0151c84640 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0151c84640 1 -- 172.21.15.105:0/2758526628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 msgr2=0x561d2eceb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0151c84640 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2eceb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0151c84640 1 -- 172.21.15.105:0/2758526628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2ea1c7d0 con 0x561d2ecd9a30 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0151c84640 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecd9a30 0x561d2ece3ef0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0148005260 tx=0x7f0148005290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac9b321acf830b0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f01357fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f01357fa640 1 -- 172.21.15.105:0/2758526628 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01480054b0 con 0x561d2ecd9a30 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0156baa740 1 -- 172.21.15.105:0/2758526628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01480047a0 con 0x561d2ecd9a30 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f01357fa640 1 -- 172.21.15.105:0/2758526628 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f014801e630 con 0x561d2ecd9a30 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f01357fa640 1 -- 172.21.15.105:0/2758526628 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f014801e830 con 0x561d2ecd9a30 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0156baa740 10 client.33103 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.229+0000 7f0156baa740 1 -- 172.21.15.105:0/2758526628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561d2ecee7e0 con 0x561d2ecd9a30 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f01357fa640 1 -- 172.21.15.105:0/2758526628 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01480047a0 con 0x561d2ecd9a30 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f01357fa640 1 -- 172.21.15.105:0/2758526628 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f014803ca40 con 0x561d2ecd9a30 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f0156baa740 10 client.33103 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f0156baa740 10 client.33103 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f0156baa740 20 client.33103 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:31.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f0156baa740 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561d2ecefcd0 0x561d2ecf2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f0156baa740 4 client.33103 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f0156baa740 1 -- 172.21.15.105:0/2758526628 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d2ec4beb0 con 0x561d2ecefcd0 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.230+0000 7f0151483640 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561d2ecefcd0 0x561d2ecf2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.231+0000 7f0151483640 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561d2ecefcd0 0x561d2ecf2170 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efdf1bbf01b48a65 server_cookie=c3b556155d7afa4e in_seq=0 out_seq=0 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.231+0000 7f01357fa640 10 client.33103 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f01357fa640 1 -- 172.21.15.105:0/2758526628 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561d2ec4beb0 con 0x561d2ecefcd0 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f01357fa640 10 client.33103 handle_command_reply: tid=0 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 client.33103 shutdown 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 -- 172.21.15.105:0/2758526628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecd9a30 msgr2=0x561d2ece3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecd9a30 0x561d2ece3ef0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0148005260 tx=0x7f0148005290 comp rx=0 tx=0).stop 2024-03-26T18:00:31.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 -- 172.21.15.105:0/2758526628 shutdown_connections 2024-03-26T18:00:31.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561d2ecefcd0 0x561d2ecf2170 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2eceb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 --2- 172.21.15.105:0/2758526628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecd9a30 0x561d2ece3ef0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 -- 172.21.15.105:0/2758526628 >> 172.21.15.105:0/2758526628 conn(0x561d2ecd4fa0 msgr2=0x561d2ecd41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f01357fa640 0 client.33103 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:00:31.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 -- 172.21.15.105:0/2758526628 shutdown_connections 2024-03-26T18:00:31.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 1 -- 172.21.15.105:0/2758526628 wait complete. 2024-03-26T18:00:31.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.232+0000 7f0156baa740 20 client.33103 trim_cache size 0 max 16384 2024-03-26T18:00:31.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0156baa740 1 Processor -- start 2024-03-26T18:00:31.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0156baa740 1 -- start start 2024-03-26T18:00:31.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0156baa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0156baa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2eceb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0156baa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdc030 0x561d2ece4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0156baa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1a6b80 con 0x561d2ecdc030 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0156baa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1a86c0 con 0x561d2ecda420 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0156baa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1a7250 con 0x561d2ecdf590 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0152485640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdc030 0x561d2ece4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151483640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2eceb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151483640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2eceb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46436/0 (socket says 172.21.15.105:46436) 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0152485640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdc030 0x561d2ece4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45614/0 (socket says 172.21.15.105:45614) 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45800/0 (socket says 172.21.15.105:45800) 2024-03-26T18:00:31.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151483640 1 -- 172.21.15.105:0/4198425155 learned_addr learned my addr 172.21.15.105:0/4198425155 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151483640 1 -- 172.21.15.105:0/4198425155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 msgr2=0x561d2ece3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151483640 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151483640 1 -- 172.21.15.105:0/4198425155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdc030 msgr2=0x561d2ece4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151483640 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdc030 0x561d2ece4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.255+0000 7f0151483640 1 -- 172.21.15.105:0/4198425155 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2ecd4dd0 con 0x561d2ecdf590 2024-03-26T18:00:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.256+0000 7f0152485640 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdc030 0x561d2ece4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.256+0000 7f0151c84640 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:31.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.256+0000 7f0151483640 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2eceb430 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f013c009900 tx=0x7f013c02c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f513ff2e14a16d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.256+0000 7f01377fe640 1 -- 172.21.15.105:0/4198425155 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c02cba0 con 0x561d2ecdf590 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.256+0000 7f01377fe640 1 -- 172.21.15.105:0/4198425155 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f013c004040 con 0x561d2ecdf590 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.256+0000 7f01377fe640 1 -- 172.21.15.105:0/4198425155 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c0484f0 con 0x561d2ecdf590 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.257+0000 7f0156baa740 1 -- 172.21.15.105:0/4198425155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 msgr2=0x561d2eceb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.257+0000 7f0156baa740 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2eceb430 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f013c009900 tx=0x7f013c02c7e0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.257+0000 7f0156baa740 1 -- 172.21.15.105:0/4198425155 shutdown_connections 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.257+0000 7f0156baa740 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdc030 0x561d2ece4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.257+0000 7f0156baa740 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecdf590 0x561d2eceb430 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.257+0000 7f0156baa740 1 --2- 172.21.15.105:0/4198425155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ecda420 0x561d2ece3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.257+0000 7f0156baa740 1 -- 172.21.15.105:0/4198425155 >> 172.21.15.105:0/4198425155 conn(0x561d2ecd4fa0 msgr2=0x561d2ecdb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.258+0000 7f0156baa740 1 -- 172.21.15.105:0/4198425155 shutdown_connections 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.258+0000 7f0156baa740 1 -- 172.21.15.105:0/4198425155 wait complete. 2024-03-26T18:00:31.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.258+0000 7f0156baa740 1 -- start start 2024-03-26T18:00:31.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0156baa740 1 Processor -- start 2024-03-26T18:00:31.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0156baa740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecda420 0x561d2ece8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0156baa740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2ece8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0156baa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ece9c70 0x561d2ecea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0156baa740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1a7ff0 con 0x561d2ecdf590 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0156baa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1a7920 con 0x561d2ece9c70 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0156baa740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f014c1a4b60 con 0x561d2ecda420 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0156baa740 10 client.0 _ll_register_callbacks cb 0x7f0148005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0151483640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2ece8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0151483640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2ece8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45620/0 (socket says 172.21.15.105:45620) 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0151483640 1 -- 172.21.15.105:0/4006822584 learned_addr learned my addr 172.21.15.105:0/4006822584 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0152485640 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ece9c70 0x561d2ecea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.259+0000 7f0151c84640 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecda420 0x561d2ece8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f0151483640 1 -- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecda420 msgr2=0x561d2ece8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f0151483640 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecda420 0x561d2ece8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f0151483640 1 -- 172.21.15.105:0/4006822584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ece9c70 msgr2=0x561d2ecea0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f0151483640 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ece9c70 0x561d2ecea0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f0151483640 1 -- 172.21.15.105:0/4006822584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2ecea630 con 0x561d2ecdf590 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f0151c84640 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecda420 0x561d2ece8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f0151483640 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2ece8c60 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f013c005ab0 tx=0x7f013c02cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e89622a810c988 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f01357fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f01357fa640 1 -- 172.21.15.105:0/4006822584 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c005050 con 0x561d2ecdf590 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f01357fa640 1 -- 172.21.15.105:0/4006822584 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f013c04d070 con 0x561d2ecdf590 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.260+0000 7f0156baa740 1 -- 172.21.15.105:0/4006822584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f013c004040 con 0x561d2ecdf590 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f01357fa640 1 -- 172.21.15.105:0/4006822584 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f013c0051f0 con 0x561d2ecdf590 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f0156baa740 10 client.23439 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f0156baa740 1 -- 172.21.15.105:0/4006822584 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561d2ed5e880 con 0x561d2ecdf590 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f01357fa640 1 -- 172.21.15.105:0/4006822584 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f013c005390 con 0x561d2ecdf590 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f01357fa640 1 -- 172.21.15.105:0/4006822584 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f013c003800 con 0x561d2ecdf590 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f0156baa740 10 client.23439 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:31.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f0156baa740 10 client.23439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:00:31.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f0156baa740 20 client.23439 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:31.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f0156baa740 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561d2ed5f0a0 0x561d2ed614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f0156baa740 4 client.23439 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:00:31.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.261+0000 7f0156baa740 1 -- 172.21.15.105:0/4006822584 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f013c004040 con 0x561d2ed5f0a0 2024-03-26T18:00:31.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.262+0000 7f0151c84640 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561d2ed5f0a0 0x561d2ed614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.247 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.262+0000 7f01357fa640 10 client.23439 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.262+0000 7f0151c84640 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561d2ed5f0a0 0x561d2ed614c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd7f03208553fc95 server_cookie=1b064a2687c38000 in_seq=0 out_seq=0 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.263+0000 7f01357fa640 1 -- 172.21.15.105:0/4006822584 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f013c004040 con 0x561d2ed5f0a0 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.263+0000 7f01357fa640 10 client.23439 handle_command_reply: tid=0 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.263+0000 7f0156baa740 1 client.23439 shutdown 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.263+0000 7f0156baa740 1 -- 172.21.15.105:0/4006822584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 msgr2=0x561d2ece8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.263+0000 7f0156baa740 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2ece8c60 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f013c005ab0 tx=0x7f013c02cd00 comp rx=0 tx=0).stop 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 1 -- 172.21.15.105:0/4006822584 shutdown_connections 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561d2ed5f0a0 0x561d2ed614c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f01357fa640 0 client.23439 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561d2ece9c70 0x561d2ecea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561d2ecdf590 0x561d2ece8c60 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 1 --2- 172.21.15.105:0/4006822584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561d2ecda420 0x561d2ece8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 1 -- 172.21.15.105:0/4006822584 >> 172.21.15.105:0/4006822584 conn(0x561d2ecd4fa0 msgr2=0x561d2ece3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 1 -- 172.21.15.105:0/4006822584 shutdown_connections 2024-03-26T18:00:31.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 1 -- 172.21.15.105:0/4006822584 wait complete. 2024-03-26T18:00:31.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.264+0000 7f0156baa740 20 client.23439 trim_cache size 0 max 16384 2024-03-26T18:00:31.249 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 -- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01380a3020 msgr2=0x7f01380a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01380a3020 0x7f01380a54e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0148009bd0 tx=0x7f0148005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 -- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c107600 msgr2=0x7f014c19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c107600 0x7f014c19f140 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f0144010ea0 tx=0x7f0144010f90 comp rx=0 tx=0).stop 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 -- 172.21.15.105:0/2224968362 shutdown_connections 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f01380a3020 0x7f01380a54e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f014c107600 0x7f014c19f140 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f014c104cd0 0x7f014c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 --2- 172.21.15.105:0/2224968362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f014c1023a0 0x7f014c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.265+0000 7f0153ee8640 1 -- 172.21.15.105:0/2224968362 >> 172.21.15.105:0/2224968362 conn(0x7f014c0fbd30 msgr2=0x7f014c104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.266+0000 7f0153ee8640 1 -- 172.21.15.105:0/2224968362 shutdown_connections 2024-03-26T18:00:31.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.266+0000 7f0153ee8640 1 -- 172.21.15.105:0/2224968362 wait complete. 2024-03-26T18:00:31.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T18:00:31.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.424+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1592906218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b581019c0 msgr2=0x7f8b58101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.424+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1592906218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b581019c0 0x7f8b58101e40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c009900 tx=0x7f8b4c02bc40 comp rx=0 tx=0).stop 2024-03-26T18:00:31.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.424+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1592906218 shutdown_connections 2024-03-26T18:00:31.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.424+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1592906218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b58102380 0x7f8b58108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.424+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1592906218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b581019c0 0x7f8b58101e40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.424+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1592906218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b581007d0 0x7f8b58100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.424+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1592906218 >> 172.21.15.105:0/1592906218 conn(0x7f8b580fbd60 msgr2=0x7f8b580fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.425+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1592906218 shutdown_connections 2024-03-26T18:00:31.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.425+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1592906218 wait complete. 2024-03-26T18:00:31.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.425+0000 7f8b5dadd640 1 Processor -- start 2024-03-26T18:00:31.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.425+0000 7f8b5dadd640 1 -- start start 2024-03-26T18:00:31.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b5dadd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b581007d0 0x7f8b58195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b5dadd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b581019c0 0x7f8b58195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b5dadd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b58102380 0x7f8b5819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b5dadd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810b200 con 0x7f8b581019c0 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b5dadd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810b080 con 0x7f8b581007d0 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b5dadd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5810b380 con 0x7f8b58102380 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b57fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b58102380 0x7f8b5819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b581019c0 0x7f8b58195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b581007d0 0x7f8b58195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b57fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b58102380 0x7f8b5819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46470/0 (socket says 172.21.15.105:46470) 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b581019c0 0x7f8b58195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45656/0 (socket says 172.21.15.105:45656) 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b57fff640 1 -- 172.21.15.105:0/1694632144 learned_addr learned my addr 172.21.15.105:0/1694632144 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b57fff640 1 -- 172.21.15.105:0/1694632144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b581007d0 msgr2=0x7f8b58195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b57fff640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b581007d0 0x7f8b58195720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b57fff640 1 -- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b581019c0 msgr2=0x7f8b58195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b57fff640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b581019c0 0x7f8b58195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.426+0000 7f8b57fff640 1 -- 172.21.15.105:0/1694632144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5819d340 con 0x7f8b58102380 2024-03-26T18:00:31.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.427+0000 7f8b57fff640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b58102380 0x7f8b5819ccd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000eaa0 tx=0x7f8b5000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.427+0000 7f8b54ff9640 1 -- 172.21.15.105:0/1694632144 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000cdb0 con 0x7f8b58102380 2024-03-26T18:00:31.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.427+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1694632144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5819d630 con 0x7f8b58102380 2024-03-26T18:00:31.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.427+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1694632144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5819db80 con 0x7f8b58102380 2024-03-26T18:00:31.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.427+0000 7f8b54ff9640 1 -- 172.21.15.105:0/1694632144 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b50005020 con 0x7f8b58102380 2024-03-26T18:00:31.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.427+0000 7f8b54ff9640 1 -- 172.21.15.105:0/1694632144 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b500116d0 con 0x7f8b58102380 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b54ff9640 1 -- 172.21.15.105:0/1694632144 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8b500051d0 con 0x7f8b58102380 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b6079d740 1 Processor -- start 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b6079d740 1 -- start start 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16ee40 0x56233e16f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e174330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e176e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b6079d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56233e0cf970 con 0x56233e16ee40 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b6079d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56233e077d70 con 0x56233e16f830 2024-03-26T18:00:31.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b6079d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56233e0cb920 con 0x56233e1749a0 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.430+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16ee40 0x56233e16f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16ee40 0x56233e16f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45658/0 (socket says 172.21.15.105:45658) 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e174330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 -- 172.21.15.105:0/3303276633 learned_addr learned my addr 172.21.15.105:0/3303276633 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e176e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b54ff9640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8b400a2f20 0x7f8b400a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b57fff640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8b400a2f20 0x7f8b400a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 -- 172.21.15.105:0/3303276633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 msgr2=0x56233e176e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e176e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 -- 172.21.15.105:0/3303276633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 msgr2=0x56233e174330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b54ff9640 1 -- 172.21.15.105:0/1694632144 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f8b500cf5d0 con 0x7f8b58102380 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e174330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 -- 172.21.15.105:0/3303276633 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233dde33f0 con 0x56233e16ee40 2024-03-26T18:00:31.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b577fe640 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e174330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:31.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16ee40 0x56233e16f260 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c005c40 tx=0x7f8b4c02c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31ffa266445ba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.432+0000 7f8b3dffb640 1 -- 172.21.15.105:0/3303276633 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c04a070 con 0x56233e16ee40 2024-03-26T18:00:31.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.432+0000 7f8b3dffb640 1 -- 172.21.15.105:0/3303276633 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b4c005500 con 0x56233e16ee40 2024-03-26T18:00:31.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.432+0000 7f8b3dffb640 1 -- 172.21.15.105:0/3303276633 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c004ad0 con 0x56233e16ee40 2024-03-26T18:00:31.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.431+0000 7f8b57fff640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8b400a2f20 0x7f8b400a53e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8b500cd030 tx=0x7f8b5001eb40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 -- 172.21.15.105:0/3303276633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16ee40 msgr2=0x56233e16f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16ee40 0x56233e16f260 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c005c40 tx=0x7f8b4c02c6b0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 -- 172.21.15.105:0/3303276633 shutdown_connections 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e176e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e174330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 --2- 172.21.15.105:0/3303276633 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16ee40 0x56233e16f260 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 -- 172.21.15.105:0/3303276633 >> 172.21.15.105:0/3303276633 conn(0x56233e16a3b0 msgr2=0x56233e16a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 -- 172.21.15.105:0/3303276633 shutdown_connections 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 -- 172.21.15.105:0/3303276633 wait complete. 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.433+0000 7f8b6079d740 1 -- start start 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b6079d740 1 Processor -- start 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e1793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e179910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e180950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b6079d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56233e177990 con 0x56233e16f830 2024-03-26T18:00:31.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b6079d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56233e08b310 con 0x56233e16ee40 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b6079d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56233e177b10 con 0x56233e1749a0 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b6079d740 10 client.0 _ll_register_callbacks cb 0x56233e0c6c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e179910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e179910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45662/0 (socket says 172.21.15.105:45662) 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b577fe640 1 -- 172.21.15.105:0/949641568 learned_addr learned my addr 172.21.15.105:0/949641568 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e1793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e180950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b577fe640 1 -- 172.21.15.105:0/949641568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 msgr2=0x56233e180950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b577fe640 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e180950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b577fe640 1 -- 172.21.15.105:0/949641568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 msgr2=0x56233e1793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b577fe640 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e1793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.434+0000 7f8b577fe640 1 -- 172.21.15.105:0/949641568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233dead7d0 con 0x56233e16f830 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.435+0000 7f8b577fe640 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e179910 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8b48014690 tx=0x7f8b48014b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30138f1f67e53238 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.435+0000 7f8b36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:31.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.435+0000 7f8b36ffd640 1 -- 172.21.15.105:0/949641568 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48004230 con 0x56233e16f830 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.435+0000 7f8b36ffd640 1 -- 172.21.15.105:0/949641568 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f8b480043d0 con 0x56233e16f830 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.435+0000 7f8b6079d740 1 -- 172.21.15.105:0/949641568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4c005500 con 0x56233e16f830 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.435+0000 7f8b36ffd640 1 -- 172.21.15.105:0/949641568 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8b48027050 con 0x56233e16f830 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.435+0000 7f8b6079d740 10 client.23451 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.435+0000 7f8b6079d740 1 -- 172.21.15.105:0/949641568 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56233e183d90 con 0x56233e16f830 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b36ffd640 1 -- 172.21.15.105:0/949641568 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b4c005500 con 0x56233e16f830 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b36ffd640 1 -- 172.21.15.105:0/949641568 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8b480038e0 con 0x56233e16f830 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b6079d740 10 client.23451 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b6079d740 10 client.23451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b6079d740 20 client.23451 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b6079d740 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56233e185490 0x56233e187910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b6079d740 4 client.23451 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b6079d740 1 -- 172.21.15.105:0/949641568 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56233e0dceb0 con 0x56233e185490 2024-03-26T18:00:31.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.436+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56233e185490 0x56233e187910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.437+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56233e185490 0x56233e187910 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b47e3258c1af95 server_cookie=1924776eeda8ae87 in_seq=0 out_seq=0 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.437+0000 7f8b36ffd640 10 client.23451 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b36ffd640 1 -- 172.21.15.105:0/949641568 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56233e0dceb0 con 0x56233e185490 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b36ffd640 10 client.23451 handle_command_reply: tid=0 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 client.23451 shutdown 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 -- 172.21.15.105:0/949641568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 msgr2=0x56233e179910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e179910 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8b48014690 tx=0x7f8b48014b50 comp rx=0 tx=0).stop 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 -- 172.21.15.105:0/949641568 shutdown_connections 2024-03-26T18:00:31.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56233e185490 0x56233e187910 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e180950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e179910 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 --2- 172.21.15.105:0/949641568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e1793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b6079d740 1 -- 172.21.15.105:0/949641568 >> 172.21.15.105:0/949641568 conn(0x56233e16a3b0 msgr2=0x56233e1651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.438+0000 7f8b36ffd640 0 client.23451 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.439+0000 7f8b6079d740 1 -- 172.21.15.105:0/949641568 shutdown_connections 2024-03-26T18:00:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.439+0000 7f8b6079d740 1 -- 172.21.15.105:0/949641568 wait complete. 2024-03-26T18:00:31.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.439+0000 7f8b6079d740 20 client.23451 trim_cache size 0 max 16384 2024-03-26T18:00:31.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b6079d740 1 Processor -- start 2024-03-26T18:00:31.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b6079d740 1 -- start start 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e16ee40 0x56233e164930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e180910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e1749a0 0x56233e164e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b6079d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b581a6990 con 0x56233e1749a0 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b6079d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b581a84d0 con 0x56233e16f830 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b6079d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b581a7060 con 0x56233e16ee40 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e1749a0 0x56233e164e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e16ee40 0x56233e164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e1749a0 0x56233e164e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45664/0 (socket says 172.21.15.105:45664) 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b56ffd640 1 -- 172.21.15.105:0/3107893985 learned_addr learned my addr 172.21.15.105:0/3107893985 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.461+0000 7f8b577fe640 1 --2- 172.21.15.105:0/3107893985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e180910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b56ffd640 1 -- 172.21.15.105:0/3107893985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e16ee40 msgr2=0x56233e164930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/3107893985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e16ee40 0x56233e164930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b56ffd640 1 -- 172.21.15.105:0/3107893985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 msgr2=0x56233e180910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/3107893985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e180910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b56ffd640 1 -- 172.21.15.105:0/3107893985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233e16b120 con 0x56233e1749a0 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/3107893985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e1749a0 0x56233e164e70 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c004110 tx=0x7f8b4c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb992777197a5caa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b3d7fa640 1 -- 172.21.15.105:0/3107893985 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c05c020 con 0x56233e1749a0 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b3d7fa640 1 -- 172.21.15.105:0/3107893985 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b4c04b040 con 0x56233e1749a0 2024-03-26T18:00:31.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.462+0000 7f8b3d7fa640 1 -- 172.21.15.105:0/3107893985 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c04a040 con 0x56233e1749a0 2024-03-26T18:00:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 -- 172.21.15.105:0/3107893985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e1749a0 msgr2=0x56233e164e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 --2- 172.21.15.105:0/3107893985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e1749a0 0x56233e164e70 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c004110 tx=0x7f8b4c005bb0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 -- 172.21.15.105:0/3107893985 shutdown_connections 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 --2- 172.21.15.105:0/3107893985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e1749a0 0x56233e164e70 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 --2- 172.21.15.105:0/3107893985 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16f830 0x56233e180910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 --2- 172.21.15.105:0/3107893985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e16ee40 0x56233e164930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 -- 172.21.15.105:0/3107893985 >> 172.21.15.105:0/3107893985 conn(0x56233e16a3b0 msgr2=0x56233e170dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 -- 172.21.15.105:0/3107893985 shutdown_connections 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 -- 172.21.15.105:0/3107893985 wait complete. 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.463+0000 7f8b6079d740 1 -- start start 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b6079d740 1 Processor -- start 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e17d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e17dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b6079d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e1e9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b6079d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8b581a7e00 con 0x56233e16f830 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b6079d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8b581a7730 con 0x56233e16ee40 2024-03-26T18:00:31.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b6079d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8b581a4970 con 0x56233e1749a0 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b6079d740 10 client.0 _ll_register_callbacks cb 0x7f8b4800ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e17dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e17d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e17d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45852/0 (socket says 172.21.15.105:45852) 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 -- 172.21.15.105:0/1277094594 learned_addr learned my addr 172.21.15.105:0/1277094594 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e17dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45676/0 (socket says 172.21.15.105:45676) 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e1e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 -- 172.21.15.105:0/1277094594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 msgr2=0x56233e1e9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e1e9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 -- 172.21.15.105:0/1277094594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 msgr2=0x56233e17dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e17dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b56ffd640 1 -- 172.21.15.105:0/1277094594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233e1e9d40 con 0x56233e16ee40 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.464+0000 7f8b577fe640 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e17dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:31.450 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b56ffd640 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e17d800 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0056f0 tx=0x7f8b4c005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=334903b1afff9db6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b36ffd640 1 -- 172.21.15.105:0/1277094594 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c04a070 con 0x56233e16ee40 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b6079d740 1 -- 172.21.15.105:0/1277094594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4c04b040 con 0x56233e16ee40 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b36ffd640 1 -- 172.21.15.105:0/1277094594 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f8b4c003cc0 con 0x56233e16ee40 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b36ffd640 1 -- 172.21.15.105:0/1277094594 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8b4c05c040 con 0x56233e16ee40 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b6079d740 10 client.33137 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b6079d740 1 -- 172.21.15.105:0/1277094594 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56233e1eafa0 con 0x56233e16ee40 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.465+0000 7f8b36ffd640 1 -- 172.21.15.105:0/1277094594 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b4c04b040 con 0x56233e16ee40 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b36ffd640 1 -- 172.21.15.105:0/1277094594 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8b4c040040 con 0x56233e16ee40 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b6079d740 10 client.33137 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b6079d740 10 client.33137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b6079d740 20 client.33137 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b6079d740 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56233e1eb3e0 0x56233e1ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:31.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b6079d740 4 client.33137 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:00:31.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b6079d740 1 -- 172.21.15.105:0/1277094594 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56233e0d7200 con 0x56233e1eb3e0 2024-03-26T18:00:31.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b577fe640 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56233e1eb3e0 0x56233e1ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:31.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.466+0000 7f8b36ffd640 10 client.33137 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:31.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.467+0000 7f8b577fe640 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56233e1eb3e0 0x56233e1ed800 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868436760a1875a2 server_cookie=13dc152cdc747c08 in_seq=0 out_seq=0 2024-03-26T18:00:31.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.467+0000 7f8b36ffd640 1 -- 172.21.15.105:0/1277094594 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56233e0d7200 con 0x56233e1eb3e0 2024-03-26T18:00:31.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.467+0000 7f8b36ffd640 10 client.33137 handle_command_reply: tid=0 2024-03-26T18:00:31.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.467+0000 7f8b6079d740 1 client.33137 shutdown 2024-03-26T18:00:31.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 -- 172.21.15.105:0/1277094594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 msgr2=0x56233e17d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e17d800 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0056f0 tx=0x7f8b4c005d90 comp rx=0 tx=0).stop 2024-03-26T18:00:31.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 -- 172.21.15.105:0/1277094594 shutdown_connections 2024-03-26T18:00:31.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56233e1eb3e0 0x56233e1ed800 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56233e1749a0 0x56233e1e9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b36ffd640 0 client.33137 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:31.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56233e16f830 0x56233e17dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 --2- 172.21.15.105:0/1277094594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56233e16ee40 0x56233e17d800 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 -- 172.21.15.105:0/1277094594 >> 172.21.15.105:0/1277094594 conn(0x56233e16a3b0 msgr2=0x56233e16d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 -- 172.21.15.105:0/1277094594 shutdown_connections 2024-03-26T18:00:31.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 1 -- 172.21.15.105:0/1277094594 wait complete. 2024-03-26T18:00:31.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b6079d740 20 client.33137 trim_cache size 0 max 16384 2024-03-26T18:00:31.454 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.468+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8b400a2f20 msgr2=0x7f8b400a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8b400a2f20 0x7f8b400a53e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8b500cd030 tx=0x7f8b5001eb40 comp rx=0 tx=0).stop 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1694632144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b58102380 msgr2=0x7f8b5819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b58102380 0x7f8b5819ccd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000eaa0 tx=0x7f8b5000ef60 comp rx=0 tx=0).stop 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1694632144 shutdown_connections 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8b400a2f20 0x7f8b400a53e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b58102380 0x7f8b5819ccd0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b581019c0 0x7f8b58195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 --2- 172.21.15.105:0/1694632144 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8b581007d0 0x7f8b58195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1694632144 >> 172.21.15.105:0/1694632144 conn(0x7f8b580fbd60 msgr2=0x7f8b58104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:31.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1694632144 shutdown_connections 2024-03-26T18:00:31.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:31.469+0000 7f8b5dadd640 1 -- 172.21.15.105:0/1694632144 wait complete. 2024-03-26T18:00:31.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72609 7.06 MB/sec execute 434 sec latency 116.112 ms 2024-03-26T18:00:31.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:31 smithi105 ceph-mon[32280]: pgmap v1017: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 4.0 MiB/s wr, 232 op/s 2024-03-26T18:00:31.838 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:31 smithi144 ceph-mon[36095]: pgmap v1017: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 4.0 MiB/s wr, 232 op/s 2024-03-26T18:00:31.905 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:31 smithi135 ceph-mon[35636]: pgmap v1017: 97 pgs: 97 active+clean; 354 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 4.0 MiB/s wr, 232 op/s 2024-03-26T18:00:32.466 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T18:00:32.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:00:32.466 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T18:00:32.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72740 7.07 MB/sec execute 435 sec latency 129.824 ms 2024-03-26T18:00:32.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.628+0000 7fadf6e44640 1 -- 172.21.15.105:0/3932521015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 msgr2=0x7fadf0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.628+0000 7fadf6e44640 1 --2- 172.21.15.105:0/3932521015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 0x7fadf0102f40 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7faddc009900 tx=0x7faddc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:32.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.628+0000 7fadf6e44640 1 -- 172.21.15.105:0/3932521015 shutdown_connections 2024-03-26T18:00:32.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.628+0000 7fadf6e44640 1 --2- 172.21.15.105:0/3932521015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadf0103480 0x7fadf0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.628+0000 7fadf6e44640 1 --2- 172.21.15.105:0/3932521015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 0x7fadf0102f40 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.628+0000 7fadf6e44640 1 --2- 172.21.15.105:0/3932521015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadf01018d0 0x7fadf0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.628+0000 7fadf6e44640 1 -- 172.21.15.105:0/3932521015 >> 172.21.15.105:0/3932521015 conn(0x7fadf00fce60 msgr2=0x7fadf00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.629+0000 7fadf6e44640 1 -- 172.21.15.105:0/3932521015 shutdown_connections 2024-03-26T18:00:32.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.629+0000 7fadf6e44640 1 -- 172.21.15.105:0/3932521015 wait complete. 2024-03-26T18:00:32.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf6e44640 1 Processor -- start 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf6e44640 1 -- start start 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf6e44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadf01018d0 0x7fadf019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf6e44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 0x7fadf019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf6e44640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadf0103480 0x7fadf01a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf6e44640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fadf010f610 con 0x7fadf0102ac0 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf6e44640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fadf010f490 con 0x7fadf0103480 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf6e44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fadf010f790 con 0x7fadf01018d0 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf53e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadf0103480 0x7fadf01a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadf4be0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadf01018d0 0x7fadf019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 0x7fadf019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 0x7fadf019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45704/0 (socket says 172.21.15.105:45704) 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.630+0000 7fadeffff640 1 -- 172.21.15.105:0/523415347 learned_addr learned my addr 172.21.15.105:0/523415347 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.615 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadf53e1640 1 -- 172.21.15.105:0/523415347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadf01018d0 msgr2=0x7fadf019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadf53e1640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadf01018d0 0x7fadf019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadf53e1640 1 -- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 msgr2=0x7fadf019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadf53e1640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 0x7fadf019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadf53e1640 1 -- 172.21.15.105:0/523415347 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf01a5d40 con 0x7fadf0103480 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadeffff640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 0x7fadf019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadf4be0640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadf01018d0 0x7fadf019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadf53e1640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadf0103480 0x7fadf01a56d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fade8009fd0 tx=0x7fade800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadedffb640 1 -- 172.21.15.105:0/523415347 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade800ce20 con 0x7fadf0103480 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadf6e44640 1 -- 172.21.15.105:0/523415347 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf01a5fd0 con 0x7fadf0103480 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadedffb640 1 -- 172.21.15.105:0/523415347 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fade8005020 con 0x7fadf0103480 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.631+0000 7fadedffb640 1 -- 172.21.15.105:0/523415347 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8010690 con 0x7fadf0103480 2024-03-26T18:00:32.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.632+0000 7fadf6e44640 1 -- 172.21.15.105:0/523415347 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf01a64f0 con 0x7fadf0103480 2024-03-26T18:00:32.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.634+0000 7fadedffb640 1 -- 172.21.15.105:0/523415347 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fade80051d0 con 0x7fadf0103480 2024-03-26T18:00:32.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf9a2e740 1 Processor -- start 2024-03-26T18:00:32.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf9a2e740 1 -- start start 2024-03-26T18:00:32.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dced190 0x563b7dced5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcf2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcf2cf0 0x563b7dcf5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563b7dc52630 con 0x563b7dcedb80 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b7dbfae10 con 0x563b7dced190 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563b7dc4e5e0 con 0x563b7dcf2cf0 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadeffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcf2cf0 0x563b7dcf5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadeffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcf2cf0 0x563b7dcf5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46564/0 (socket says 172.21.15.105:46564) 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadeffff640 1 -- 172.21.15.105:0/4047461516 learned_addr learned my addr 172.21.15.105:0/4047461516 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadf4be0640 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcf2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadeffff640 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dced190 0x563b7dced5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.635+0000 7fadeffff640 1 -- 172.21.15.105:0/4047461516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dced190 msgr2=0x563b7dced5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7fadeffff640 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dced190 0x563b7dced5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7fadeffff640 1 -- 172.21.15.105:0/4047461516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 msgr2=0x563b7dcf2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7fadeffff640 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcf2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7fadeffff640 1 -- 172.21.15.105:0/4047461516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b7dc11740 con 0x563b7dcf2cf0 2024-03-26T18:00:32.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7fadedffb640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fade00a2fa0 0x7fade00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7fadeffff640 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcf2cf0 0x563b7dcf5180 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x563b7dcee3e0 tx=0x7faddc005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5f02c377e10fcf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7fadf4be0640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fade00a2fa0 0x7fade00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7faddaffd640 1 -- 172.21.15.105:0/4047461516 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc04a070 con 0x563b7dcf2cf0 2024-03-26T18:00:32.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7faddaffd640 1 -- 172.21.15.105:0/4047461516 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faddc004410 con 0x563b7dcf2cf0 2024-03-26T18:00:32.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7faddaffd640 1 -- 172.21.15.105:0/4047461516 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc043740 con 0x563b7dcf2cf0 2024-03-26T18:00:32.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.636+0000 7fadedffb640 1 -- 172.21.15.105:0/523415347 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fade80cfac0 con 0x7fadf0103480 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf4be0640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fade00a2fa0 0x7fade00a5460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x563b7dcedf80 tx=0x7fade4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 -- 172.21.15.105:0/4047461516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcf2cf0 msgr2=0x563b7dcf5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcf2cf0 0x563b7dcf5180 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x563b7dcee3e0 tx=0x7faddc005b50 comp rx=0 tx=0).stop 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 -- 172.21.15.105:0/4047461516 shutdown_connections 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcf2cf0 0x563b7dcf5180 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcf2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/4047461516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dced190 0x563b7dced5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 -- 172.21.15.105:0/4047461516 >> 172.21.15.105:0/4047461516 conn(0x563b7dce8700 msgr2=0x563b7dce8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 -- 172.21.15.105:0/4047461516 shutdown_connections 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.638+0000 7fadf9a2e740 1 -- 172.21.15.105:0/4047461516 wait complete. 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf9a2e740 1 -- start start 2024-03-26T18:00:32.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf9a2e740 1 Processor -- start 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dcf76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcedb80 0x563b7dcf7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563b7dcf5c40 con 0x563b7dcf2cf0 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b7dc0e1a0 con 0x563b7dcedb80 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563b7dcf5dc0 con 0x563b7dced190 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadeffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcedb80 0x563b7dcf7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadeffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcedb80 0x563b7dcf7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45896/0 (socket says 172.21.15.105:45896) 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadeffff640 1 -- 172.21.15.105:0/1245412481 learned_addr learned my addr 172.21.15.105:0/1245412481 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf4be0640 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dcf76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.639+0000 7fadf53e1640 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadeffff640 1 -- 172.21.15.105:0/1245412481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 msgr2=0x563b7dcf76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadeffff640 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dcf76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadeffff640 1 -- 172.21.15.105:0/1245412481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcf2cf0 msgr2=0x563b7dcfec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadeffff640 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadeffff640 1 -- 172.21.15.105:0/1245412481 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b7da2e7e0 con 0x563b7dcedb80 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadf53e1640 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadf9a2e740 10 client.0 _ll_register_callbacks cb 0x563b7d965c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1245412481 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faddc004410 con 0x563b7dcedb80 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadeffff640 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcedb80 0x563b7dcf7c20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faddc000c00 tx=0x7faddc003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=149636f4518b56f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadcffff640 10 client.33149 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.640+0000 7fadcffff640 1 -- 172.21.15.105:0/1245412481 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc04a070 con 0x563b7dcedb80 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadcffff640 1 -- 172.21.15.105:0/1245412481 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7faddc0438d0 con 0x563b7dcedb80 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadcffff640 1 -- 172.21.15.105:0/1245412481 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7faddc043b10 con 0x563b7dcedb80 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf9a2e740 10 client.33149 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1245412481 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=43}) -- 0x563b7da8ef70 con 0x563b7dcedb80 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadcffff640 1 -- 172.21.15.105:0/1245412481 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7faddc056030 con 0x563b7dcedb80 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf9a2e740 10 client.33149 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf9a2e740 10 client.33149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf9a2e740 20 client.33149 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b7dd02d50 0x563b7dd05140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf9a2e740 4 client.33149 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1245412481 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b7dc5fb70 con 0x563b7dd02d50 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.641+0000 7fadf4be0640 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b7dd02d50 0x563b7dd05140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.642+0000 7fadcffff640 10 client.33149 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.642+0000 7fadf4be0640 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b7dd02d50 0x563b7dd05140 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1434b09ecdd3608 server_cookie=b65094e5ef7f84af in_seq=0 out_seq=0 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.643+0000 7fadcffff640 1 -- 172.21.15.105:0/1245412481 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563b7dc5fb70 con 0x563b7dd02d50 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.643+0000 7fadcffff640 10 client.33149 handle_command_reply: tid=0 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.643+0000 7fadf9a2e740 1 client.33149 shutdown 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1245412481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcedb80 msgr2=0x563b7dcf7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcedb80 0x563b7dcf7c20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7faddc000c00 tx=0x7faddc003ce0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1245412481 shutdown_connections 2024-03-26T18:00:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b7dd02d50 0x563b7dd05140 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcedb80 0x563b7dcf7c20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadcffff640 0 client.33149 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1245412481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dcf76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1245412481 >> 172.21.15.105:0/1245412481 conn(0x563b7dce8700 msgr2=0x563b7dce7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1245412481 shutdown_connections 2024-03-26T18:00:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1245412481 wait complete. 2024-03-26T18:00:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.644+0000 7fadf9a2e740 20 client.33149 trim_cache size 0 max 16384 2024-03-26T18:00:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.671+0000 7fadf9a2e740 1 Processor -- start 2024-03-26T18:00:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.671+0000 7fadf9a2e740 1 -- start start 2024-03-26T18:00:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dce7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcfec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dce75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fadf01af360 con 0x563b7dcedb80 2024-03-26T18:00:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fadf01b0ea0 con 0x563b7dcf2cf0 2024-03-26T18:00:32.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fadf01afa30 con 0x563b7dced190 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcfec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcfec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45736/0 (socket says 172.21.15.105:45736) 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadeffff640 1 -- 172.21.15.105:0/1513771393 learned_addr learned my addr 172.21.15.105:0/1513771393 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf4be0640 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dce7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf53e1640 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dce75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadeffff640 1 -- 172.21.15.105:0/1513771393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 msgr2=0x563b7dce7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadeffff640 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dce7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadeffff640 1 -- 172.21.15.105:0/1513771393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 msgr2=0x563b7dce75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadeffff640 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dce75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadeffff640 1 -- 172.21.15.105:0/1513771393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b7dd054b0 con 0x563b7dcedb80 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.672+0000 7fadf4be0640 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dce7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.673+0000 7fadeffff640 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcfec20 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7faddc005920 tx=0x7faddc03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1be75b13753035 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.673+0000 7fadda7fc640 1 -- 172.21.15.105:0/1513771393 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc043cb0 con 0x563b7dcedb80 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.673+0000 7fadda7fc640 1 -- 172.21.15.105:0/1513771393 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faddc004410 con 0x563b7dcedb80 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.673+0000 7fadda7fc640 1 -- 172.21.15.105:0/1513771393 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc004920 con 0x563b7dcedb80 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1513771393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 msgr2=0x563b7dcfec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcfec20 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7faddc005920 tx=0x7faddc03a000 comp rx=0 tx=0).stop 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1513771393 shutdown_connections 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dce75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dcedb80 0x563b7dcfec20 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/1513771393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dced190 0x563b7dce7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1513771393 >> 172.21.15.105:0/1513771393 conn(0x563b7dce8700 msgr2=0x563b7dcec1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1513771393 shutdown_connections 2024-03-26T18:00:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 -- 172.21.15.105:0/1513771393 wait complete. 2024-03-26T18:00:32.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 -- start start 2024-03-26T18:00:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.674+0000 7fadf9a2e740 1 Processor -- start 2024-03-26T18:00:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dced190 0x563b7dcfc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcedb80 0x563b7dcfc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf9a2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fadf01b07d0 con 0x563b7dced190 2024-03-26T18:00:32.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fadf01b0100 con 0x563b7dcf2cf0 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf9a2e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fadf01ad340 con 0x563b7dcedb80 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf53e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf4be0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dced190 0x563b7dcfc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf53e1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45910/0 (socket says 172.21.15.105:45910) 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf4be0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dced190 0x563b7dcfc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45740/0 (socket says 172.21.15.105:45740) 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf53e1640 1 -- 172.21.15.105:0/3920726885 learned_addr learned my addr 172.21.15.105:0/3920726885 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadeffff640 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcedb80 0x563b7dcfc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf9a2e740 10 client.0 _ll_register_callbacks cb 0x7faddc048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf53e1640 1 -- 172.21.15.105:0/3920726885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcedb80 msgr2=0x563b7dcfc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf53e1640 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcedb80 0x563b7dcfc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf53e1640 1 -- 172.21.15.105:0/3920726885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dced190 msgr2=0x563b7dcfc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.675+0000 7fadf53e1640 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dced190 0x563b7dcfc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.676+0000 7fadf53e1640 1 -- 172.21.15.105:0/3920726885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b7d926170 con 0x563b7dcf2cf0 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.676+0000 7fadf53e1640 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfd960 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fade8098ad0 tx=0x7fade8098470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15cd0ac23a00f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.676+0000 7fadcffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.676+0000 7fadcffff640 1 -- 172.21.15.105:0/3920726885 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80ba060 con 0x563b7dcf2cf0 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.676+0000 7fadf9a2e740 1 -- 172.21.15.105:0/3920726885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faddc004410 con 0x563b7dcf2cf0 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.676+0000 7fadcffff640 1 -- 172.21.15.105:0/3920726885 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fade80bc0b0 con 0x563b7dcf2cf0 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadcffff640 1 -- 172.21.15.105:0/3920726885 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fade8082050 con 0x563b7dcf2cf0 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadf9a2e740 10 client.33155 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadf9a2e740 1 -- 172.21.15.105:0/3920726885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563b7dd69020 con 0x563b7dcf2cf0 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadcffff640 1 -- 172.21.15.105:0/3920726885 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faddc004410 con 0x563b7dcf2cf0 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadcffff640 1 -- 172.21.15.105:0/3920726885 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fade8097ea0 con 0x563b7dcf2cf0 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadf9a2e740 10 client.33155 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadf9a2e740 10 client.33155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadf9a2e740 20 client.33155 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b7dd69920 0x563b7dd6bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadf9a2e740 4 client.33155 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T18:00:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.677+0000 7fadf9a2e740 1 -- 172.21.15.105:0/3920726885 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563b7dc59ec0 con 0x563b7dd69920 2024-03-26T18:00:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.678+0000 7fadf4be0640 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b7dd69920 0x563b7dd6bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.678+0000 7fadf4be0640 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b7dd69920 0x563b7dd6bd10 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57102caaf189ca55 server_cookie=6b637f6d74f9948d in_seq=0 out_seq=0 2024-03-26T18:00:32.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.678+0000 7fadcffff640 10 client.33155 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.680+0000 7fadcffff640 1 -- 172.21.15.105:0/3920726885 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563b7dc59ec0 con 0x563b7dd69920 2024-03-26T18:00:32.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.680+0000 7fadcffff640 10 client.33155 handle_command_reply: tid=0 2024-03-26T18:00:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.680+0000 7fadf9a2e740 1 client.33155 shutdown 2024-03-26T18:00:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 -- 172.21.15.105:0/3920726885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 msgr2=0x563b7dcfd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfd960 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fade8098ad0 tx=0x7fade8098470 comp rx=0 tx=0).stop 2024-03-26T18:00:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 -- 172.21.15.105:0/3920726885 shutdown_connections 2024-03-26T18:00:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563b7dd69920 0x563b7dd6bd10 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b7dcf2cf0 0x563b7dcfd960 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563b7dcedb80 0x563b7dcfc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadcffff640 0 client.33155 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 --2- 172.21.15.105:0/3920726885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563b7dced190 0x563b7dcfc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 -- 172.21.15.105:0/3920726885 >> 172.21.15.105:0/3920726885 conn(0x563b7dce8700 msgr2=0x563b7dcf6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 -- 172.21.15.105:0/3920726885 shutdown_connections 2024-03-26T18:00:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 1 -- 172.21.15.105:0/3920726885 wait complete. 2024-03-26T18:00:32.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.681+0000 7fadf9a2e740 20 client.33155 trim_cache size 0 max 16384 2024-03-26T18:00:32.667 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:00:32.667 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T18:00:32.667 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "2bd68b84-2a79-4b8d-92dc-4b5a43ecc4ca", 2024-03-26T18:00:32.668 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T18:00:32.668 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:00:32.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 -- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fade00a2fa0 msgr2=0x7fade00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fade00a2fa0 0x7fade00a5460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x563b7dcedf80 tx=0x7fade4005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:32.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 -- 172.21.15.105:0/523415347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadf0103480 msgr2=0x7fadf01a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadf0103480 0x7fadf01a56d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fade8009fd0 tx=0x7fade800ef10 comp rx=0 tx=0).stop 2024-03-26T18:00:32.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 -- 172.21.15.105:0/523415347 shutdown_connections 2024-03-26T18:00:32.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fade00a2fa0 0x7fade00a5460 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fadf0103480 0x7fadf01a56d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadf0102ac0 0x7fadf019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 --2- 172.21.15.105:0/523415347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadf01018d0 0x7fadf019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 -- 172.21.15.105:0/523415347 >> 172.21.15.105:0/523415347 conn(0x7fadf00fce60 msgr2=0x7fadf010c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 -- 172.21.15.105:0/523415347 shutdown_connections 2024-03-26T18:00:32.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.682+0000 7fadf6e44640 1 -- 172.21.15.105:0/523415347 wait complete. 2024-03-26T18:00:32.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:00:32.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:00:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.835+0000 7f2fbdb59640 1 -- 172.21.15.105:0/208778888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81018f0 msgr2=0x7f2fb8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.835+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/208778888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81018f0 0x7f2fb8101d10 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009900 tx=0x7f2fa802bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.835+0000 7f2fbdb59640 1 -- 172.21.15.105:0/208778888 shutdown_connections 2024-03-26T18:00:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.835+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/208778888 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb81034a0 0x7f2fb8109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.835+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/208778888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb8102ae0 0x7f2fb8102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.835+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/208778888 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81018f0 0x7f2fb8101d10 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.835+0000 7f2fbdb59640 1 -- 172.21.15.105:0/208778888 >> 172.21.15.105:0/208778888 conn(0x7f2fb80fce60 msgr2=0x7f2fb80ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.836+0000 7f2fbdb59640 1 -- 172.21.15.105:0/208778888 shutdown_connections 2024-03-26T18:00:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.836+0000 7f2fbdb59640 1 -- 172.21.15.105:0/208778888 wait complete. 2024-03-26T18:00:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.836+0000 7f2fbdb59640 1 Processor -- start 2024-03-26T18:00:32.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.836+0000 7f2fbdb59640 1 -- start start 2024-03-26T18:00:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fbdb59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb81018f0 0x7f2fb819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fbdb59640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb8102ae0 0x7f2fb819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fbdb59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81034a0 0x7f2fb81a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fbdb59640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb80751f0 con 0x7f2fb81034a0 2024-03-26T18:00:32.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fbdb59640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8075070 con 0x7f2fb8102ae0 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fbdb59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8075370 con 0x7f2fb81018f0 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81034a0 0x7f2fb81a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb8102ae0 0x7f2fb819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81034a0 0x7f2fb81a5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45762/0 (socket says 172.21.15.105:45762) 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb8102ae0 0x7f2fb819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45932/0 (socket says 172.21.15.105:45932) 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb7fff640 1 -- 172.21.15.105:0/336584024 learned_addr learned my addr 172.21.15.105:0/336584024 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb81018f0 0x7f2fb819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb6ffd640 1 -- 172.21.15.105:0/336584024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb81018f0 msgr2=0x7f2fb819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb81018f0 0x7f2fb819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb6ffd640 1 -- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81034a0 msgr2=0x7f2fb81a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81034a0 0x7f2fb81a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb6ffd640 1 -- 172.21.15.105:0/336584024 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb81a5dd0 con 0x7f2fb8102ae0 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb7fff640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81034a0 0x7f2fb81a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.837+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb81018f0 0x7f2fb819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:32.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.838+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb8102ae0 0x7f2fb819e6f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005fd0 tx=0x7f2fac00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.838+0000 7f2fb4ff9640 1 -- 172.21.15.105:0/336584024 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac009c30 con 0x7f2fb8102ae0 2024-03-26T18:00:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.838+0000 7f2fb4ff9640 1 -- 172.21.15.105:0/336584024 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fac009dd0 con 0x7f2fb8102ae0 2024-03-26T18:00:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.838+0000 7f2fb4ff9640 1 -- 172.21.15.105:0/336584024 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac003e20 con 0x7f2fb8102ae0 2024-03-26T18:00:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.838+0000 7f2fbdb59640 1 -- 172.21.15.105:0/336584024 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb81a60c0 con 0x7f2fb8102ae0 2024-03-26T18:00:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.838+0000 7f2fbdb59640 1 -- 172.21.15.105:0/336584024 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb81a6610 con 0x7f2fb8102ae0 2024-03-26T18:00:32.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.842+0000 7f2fb4ff9640 1 -- 172.21.15.105:0/336584024 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2fac0035f0 con 0x7f2fb8102ae0 2024-03-26T18:00:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fb4ff9640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2fa00a3070 0x7f2fa00a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fc0811740 1 Processor -- start 2024-03-26T18:00:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2fa00a3070 0x7f2fa00a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fc0811740 1 -- start start 2024-03-26T18:00:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fb4ff9640 1 -- 172.21.15.105:0/336584024 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2fac0cef00 con 0x7f2fb8102ae0 2024-03-26T18:00:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8268f0 0x5582ed826d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8272e0 0x5582ed82bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed82c450 0x5582ed82e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fc0811740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5582ed78b980 con 0x5582ed82c450 2024-03-26T18:00:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fc0811740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5582ed733be0 con 0x5582ed8272e0 2024-03-26T18:00:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fc0811740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5582ed787930 con 0x5582ed8268f0 2024-03-26T18:00:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed82c450 0x5582ed82e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fb7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed82c450 0x5582ed82e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45776/0 (socket says 172.21.15.105:45776) 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.843+0000 7f2fb7fff640 1 -- 172.21.15.105:0/3328416546 learned_addr learned my addr 172.21.15.105:0/3328416546 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2fb7fff640 1 -- 172.21.15.105:0/3328416546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8268f0 msgr2=0x5582ed826d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2fb7fff640 1 --2- 172.21.15.105:0/3328416546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8268f0 0x5582ed826d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2fb7fff640 1 -- 172.21.15.105:0/3328416546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8272e0 msgr2=0x5582ed82bde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2fb7fff640 1 --2- 172.21.15.105:0/3328416546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8272e0 0x5582ed82bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2fb7fff640 1 -- 172.21.15.105:0/3328416546 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ed5697d0 con 0x5582ed82c450 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2fb7fff640 1 --2- 172.21.15.105:0/3328416546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed82c450 0x5582ed82e8e0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0011850 tx=0x7f2fb0011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468abad9e1b89e53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2fa00a3070 0x7f2fa00a5530 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2fa802c240 tx=0x7f2fa802c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2f9dffb640 1 -- 172.21.15.105:0/3328416546 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0011330 con 0x5582ed82c450 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2f9dffb640 1 -- 172.21.15.105:0/3328416546 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fb000fe50 con 0x5582ed82c450 2024-03-26T18:00:32.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.844+0000 7f2f9dffb640 1 -- 172.21.15.105:0/3328416546 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00193e0 con 0x5582ed82c450 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 -- 172.21.15.105:0/3328416546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed82c450 msgr2=0x5582ed82e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 --2- 172.21.15.105:0/3328416546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed82c450 0x5582ed82e8e0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0011850 tx=0x7f2fb0011d10 comp rx=0 tx=0).stop 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 -- 172.21.15.105:0/3328416546 shutdown_connections 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 --2- 172.21.15.105:0/3328416546 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed82c450 0x5582ed82e8e0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 --2- 172.21.15.105:0/3328416546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8272e0 0x5582ed82bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 --2- 172.21.15.105:0/3328416546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8268f0 0x5582ed826d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 -- 172.21.15.105:0/3328416546 >> 172.21.15.105:0/3328416546 conn(0x5582ed821e60 msgr2=0x5582ed822260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 -- 172.21.15.105:0/3328416546 shutdown_connections 2024-03-26T18:00:32.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.845+0000 7f2fc0811740 1 -- 172.21.15.105:0/3328416546 wait complete. 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 1 -- start start 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 1 Processor -- start 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8268f0 0x5582ed830b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8272e0 0x5582ed831040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed82c450 0x5582ed838080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5582ed82f400 con 0x5582ed8272e0 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5582ed747160 con 0x5582ed8268f0 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5582ed82f580 con 0x5582ed82c450 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fc0811740 10 client.0 _ll_register_callbacks cb 0x5582ed783720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.846+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8268f0 0x5582ed830b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8272e0 0x5582ed831040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8268f0 0x5582ed830b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45950/0 (socket says 172.21.15.105:45950) 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb77fe640 1 -- 172.21.15.105:0/2948051951 learned_addr learned my addr 172.21.15.105:0/2948051951 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb7fff640 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed82c450 0x5582ed838080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb6ffd640 1 -- 172.21.15.105:0/2948051951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed82c450 msgr2=0x5582ed838080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed82c450 0x5582ed838080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb6ffd640 1 -- 172.21.15.105:0/2948051951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8268f0 msgr2=0x5582ed830b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8268f0 0x5582ed830b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb6ffd640 1 -- 172.21.15.105:0/2948051951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ed49f3f0 con 0x5582ed8272e0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8272e0 0x5582ed831040 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x5582ed827d50 tx=0x7f2fac09bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726aaca2292e8a6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2f96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2f96ffd640 1 -- 172.21.15.105:0/2948051951 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac020040 con 0x5582ed8272e0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2f96ffd640 1 -- 172.21.15.105:0/2948051951 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2fac09aa40 con 0x5582ed8272e0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.847+0000 7f2fc0811740 1 -- 172.21.15.105:0/2948051951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb000fe50 con 0x5582ed8272e0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2f96ffd640 1 -- 172.21.15.105:0/2948051951 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2fac0a4000 con 0x5582ed8272e0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fc0811740 10 client.23487 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fc0811740 1 -- 172.21.15.105:0/2948051951 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5582ed45f0f0 con 0x5582ed8272e0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2f96ffd640 1 -- 172.21.15.105:0/2948051951 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fb000fe50 con 0x5582ed8272e0 2024-03-26T18:00:32.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2f96ffd640 1 -- 172.21.15.105:0/2948051951 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2fac0a08d0 con 0x5582ed8272e0 2024-03-26T18:00:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fc0811740 10 client.23487 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fc0811740 10 client.23487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fc0811740 20 client.23487 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fc0811740 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5582ed83c960 0x5582ed83ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fc0811740 4 client.23487 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:32.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fc0811740 1 -- 172.21.15.105:0/2948051951 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ed798ec0 con 0x5582ed83c960 2024-03-26T18:00:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.848+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5582ed83c960 0x5582ed83ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.849+0000 7f2f96ffd640 10 client.23487 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.849+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5582ed83c960 0x5582ed83ed50 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4a49b3e5322368 server_cookie=e38d366189a9583b in_seq=0 out_seq=0 2024-03-26T18:00:32.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.851+0000 7f2f96ffd640 1 -- 172.21.15.105:0/2948051951 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5582ed798ec0 con 0x5582ed83c960 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.851+0000 7f2f96ffd640 10 client.23487 handle_command_reply: tid=0 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.851+0000 7f2fc0811740 1 client.23487 shutdown 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.851+0000 7f2fc0811740 1 -- 172.21.15.105:0/2948051951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8272e0 msgr2=0x5582ed831040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.851+0000 7f2fc0811740 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8272e0 0x5582ed831040 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x5582ed827d50 tx=0x7f2fac09bcf0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.851+0000 7f2fc0811740 1 -- 172.21.15.105:0/2948051951 shutdown_connections 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2fc0811740 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5582ed83c960 0x5582ed83ed50 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2fc0811740 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed82c450 0x5582ed838080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2fc0811740 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8272e0 0x5582ed831040 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2f96ffd640 0 client.23487 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2fc0811740 1 --2- 172.21.15.105:0/2948051951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed8268f0 0x5582ed830b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2fc0811740 1 -- 172.21.15.105:0/2948051951 >> 172.21.15.105:0/2948051951 conn(0x5582ed821e60 msgr2=0x5582ed821080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2fc0811740 1 -- 172.21.15.105:0/2948051951 shutdown_connections 2024-03-26T18:00:32.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2fc0811740 1 -- 172.21.15.105:0/2948051951 wait complete. 2024-03-26T18:00:32.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.852+0000 7f2fc0811740 20 client.23487 trim_cache size 0 max 16384 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.879+0000 7f2fc0811740 1 Processor -- start 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.879+0000 7f2fc0811740 1 -- start start 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.879+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed82baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.879+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 0x5582ed838040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.879+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 0x5582ed8284c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.879+0000 7f2fc0811740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81af410 con 0x5582ed8268f0 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.879+0000 7f2fc0811740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b0f50 con 0x5582ed82c450 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.879+0000 7f2fc0811740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81afae0 con 0x5582ed8272e0 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed82baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb6ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 0x5582ed838040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed82baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45800/0 (socket says 172.21.15.105:45800) 2024-03-26T18:00:32.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 -- 172.21.15.105:0/34865132 learned_addr learned my addr 172.21.15.105:0/34865132 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb7fff640 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 0x5582ed8284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 -- 172.21.15.105:0/34865132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 msgr2=0x5582ed838040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 0x5582ed838040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 -- 172.21.15.105:0/34865132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 msgr2=0x5582ed8284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 0x5582ed8284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 -- 172.21.15.105:0/34865132 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ed827b40 con 0x5582ed8268f0 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 0x5582ed838040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed82baf0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009230 tx=0x7f2fa803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0998e71005164 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2f9d7fa640 1 -- 172.21.15.105:0/34865132 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa805a070 con 0x5582ed8268f0 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2f9d7fa640 1 -- 172.21.15.105:0/34865132 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa80054f0 con 0x5582ed8268f0 2024-03-26T18:00:32.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.880+0000 7f2f9d7fa640 1 -- 172.21.15.105:0/34865132 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8004e60 con 0x5582ed8268f0 2024-03-26T18:00:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.881+0000 7f2fc0811740 1 -- 172.21.15.105:0/34865132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 msgr2=0x5582ed82baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.881+0000 7f2fc0811740 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed82baf0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009230 tx=0x7f2fa803d000 comp rx=0 tx=0).stop 2024-03-26T18:00:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.881+0000 7f2fc0811740 1 -- 172.21.15.105:0/34865132 shutdown_connections 2024-03-26T18:00:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.881+0000 7f2fc0811740 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 0x5582ed8284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.881+0000 7f2fc0811740 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 0x5582ed838040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.881+0000 7f2fc0811740 1 --2- 172.21.15.105:0/34865132 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed82baf0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.881+0000 7f2fc0811740 1 -- 172.21.15.105:0/34865132 >> 172.21.15.105:0/34865132 conn(0x5582ed821e60 msgr2=0x5582ed822240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.882+0000 7f2fc0811740 1 -- 172.21.15.105:0/34865132 shutdown_connections 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.882+0000 7f2fc0811740 1 -- 172.21.15.105:0/34865132 wait complete. 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.882+0000 7f2fc0811740 1 -- start start 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.882+0000 7f2fc0811740 1 Processor -- start 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.882+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed8350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 0x5582ed8355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed8350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed8350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45806/0 (socket says 172.21.15.105:45806) 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fc0811740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 0x5582ed8a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 -- 172.21.15.105:0/4278036852 learned_addr learned my addr 172.21.15.105:0/4278036852 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b0880 con 0x5582ed8268f0 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81b01b0 con 0x5582ed82c450 2024-03-26T18:00:32.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81ad3f0 con 0x5582ed8272e0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fc0811740 10 client.0 _ll_register_callbacks cb 0x7f2fac09b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 -- 172.21.15.105:0/4278036852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 msgr2=0x5582ed8355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 0x5582ed8355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 -- 172.21.15.105:0/4278036852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 msgr2=0x5582ed8a0c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 0x5582ed8a0c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 -- 172.21.15.105:0/4278036852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ed827fb0 con 0x5582ed8268f0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2f96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.883+0000 7f2fb77fe640 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed8350a0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f2fa802c170 tx=0x7f2fa802caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802595ebc6972790 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.884+0000 7f2f96ffd640 1 -- 172.21.15.105:0/4278036852 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa805a030 con 0x5582ed8268f0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.884+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa80054f0 con 0x5582ed8268f0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.884+0000 7f2f96ffd640 1 -- 172.21.15.105:0/4278036852 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2fa8052040 con 0x5582ed8268f0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.884+0000 7f2f96ffd640 1 -- 172.21.15.105:0/4278036852 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2fa8004db0 con 0x5582ed8268f0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.884+0000 7f2fc0811740 10 client.23493 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.884+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5582ed8a22a0 con 0x5582ed8268f0 2024-03-26T18:00:32.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2f96ffd640 1 -- 172.21.15.105:0/4278036852 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa805c050 con 0x5582ed8268f0 2024-03-26T18:00:32.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2f96ffd640 1 -- 172.21.15.105:0/4278036852 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2fa804c450 con 0x5582ed8268f0 2024-03-26T18:00:32.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2fc0811740 10 client.23493 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:32.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2fc0811740 10 client.23493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:32.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2fc0811740 20 client.23493 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:32.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2fc0811740 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5582ed83c960 0x5582ed8a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:32.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2fc0811740 4 client.23493 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:00:32.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2fa80054f0 con 0x5582ed83c960 2024-03-26T18:00:32.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.885+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5582ed83c960 0x5582ed8a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:32.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.886+0000 7f2f96ffd640 10 client.23493 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:32.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.886+0000 7f2fb6ffd640 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5582ed83c960 0x5582ed8a4c20 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d436bd5f5ad141f server_cookie=fe8ba8778b97f1ed in_seq=0 out_seq=0 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.888+0000 7f2f96ffd640 1 -- 172.21.15.105:0/4278036852 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x7f2fa80054f0 con 0x5582ed83c960 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.888+0000 7f2f96ffd640 10 client.23493 handle_command_reply: tid=0 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.888+0000 7f2fc0811740 1 client.23493 shutdown 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 msgr2=0x5582ed8350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed8350a0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f2fa802c170 tx=0x7f2fa802caf0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 shutdown_connections 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5582ed83c960 0x5582ed8a4c20 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2f96ffd640 0 client.23493 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5582ed82c450 0x5582ed8a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5582ed8272e0 0x5582ed8355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 --2- 172.21.15.105:0/4278036852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582ed8268f0 0x5582ed8350a0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 >> 172.21.15.105:0/4278036852 conn(0x5582ed821e60 msgr2=0x5582ed830b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 shutdown_connections 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 1 -- 172.21.15.105:0/4278036852 wait complete. 2024-03-26T18:00:32.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.889+0000 7f2fc0811740 20 client.23493 trim_cache size 0 max 16384 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (340 inodes in the stack)", 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "2bd68b84-2a79-4b8d-92dc-4b5a43ecc4ca": { 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "2bd68b84-2a79-4b8d-92dc-4b5a43ecc4ca", 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:00:32.873 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:00:32.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 -- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2fa00a3070 msgr2=0x7f2fa00a5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2fa00a3070 0x7f2fa00a5530 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2fa802c240 tx=0x7f2fa802c740 comp rx=0 tx=0).stop 2024-03-26T18:00:32.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 -- 172.21.15.105:0/336584024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb8102ae0 msgr2=0x7f2fb819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:32.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb8102ae0 0x7f2fb819e6f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005fd0 tx=0x7f2fac00dde0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 -- 172.21.15.105:0/336584024 shutdown_connections 2024-03-26T18:00:32.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2fa00a3070 0x7f2fa00a5530 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2fb81034a0 0x7f2fb81a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2fb8102ae0 0x7f2fb819e6f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 --2- 172.21.15.105:0/336584024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fb81018f0 0x7f2fb819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:32.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 -- 172.21.15.105:0/336584024 >> 172.21.15.105:0/336584024 conn(0x7f2fb80fce60 msgr2=0x7f2fb810dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:32.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 -- 172.21.15.105:0/336584024 shutdown_connections 2024-03-26T18:00:32.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:00:32.890+0000 7f2fbdb59640 1 -- 172.21.15.105:0/336584024 wait complete. 2024-03-26T18:00:32.885 INFO:tasks.cephfs.filesystem:scrub status for tag:2bd68b84-2a79-4b8d-92dc-4b5a43ecc4ca - {'path': '/', 'tag': '2bd68b84-2a79-4b8d-92dc-4b5a43ecc4ca', 'options': 'recursive,force'} 2024-03-26T18:00:33.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:32 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T18:00:33.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:32 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:33.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:32 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:33.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:32 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T18:00:33.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:32 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:33.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:32 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:33.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:32 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T18:00:33.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:32 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:33.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:32 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 -- 172.21.15.105:0/210056611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 msgr2=0x7f32e0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 --2- 172.21.15.105:0/210056611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 0x7f32e0100c40 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009900 tx=0x7f32d002bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 -- 172.21.15.105:0/210056611 shutdown_connections 2024-03-26T18:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 --2- 172.21.15.105:0/210056611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e01023d0 0x7f32e0108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 --2- 172.21.15.105:0/210056611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e0101a10 0x7f32e0101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 --2- 172.21.15.105:0/210056611 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 0x7f32e0100c40 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 -- 172.21.15.105:0/210056611 >> 172.21.15.105:0/210056611 conn(0x7f32e00fbd90 msgr2=0x7f32e00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 -- 172.21.15.105:0/210056611 shutdown_connections 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 -- 172.21.15.105:0/210056611 wait complete. 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.442+0000 7f32e609a640 1 Processor -- start 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32e609a640 1 -- start start 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32e609a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 0x7f32e01047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32e609a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e0101a10 0x7f32e0102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32e609a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e01023d0 0x7f32e0103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32e609a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32e010c340 con 0x7f32e0100820 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32e609a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32e010c1c0 con 0x7f32e01023d0 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32e609a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32e010c4c0 con 0x7f32e0101a10 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 0x7f32e01047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 0x7f32e01047b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45818/0 (socket says 172.21.15.105:45818) 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 -- 172.21.15.105:0/3323719597 learned_addr learned my addr 172.21.15.105:0/3323719597 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32deffd640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e0101a10 0x7f32e0102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32dffff640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e01023d0 0x7f32e0103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 -- 172.21.15.105:0/3323719597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e0101a10 msgr2=0x7f32e0102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e0101a10 0x7f32e0102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 -- 172.21.15.105:0/3323719597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e01023d0 msgr2=0x7f32e0103330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e01023d0 0x7f32e0103330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 -- 172.21.15.105:0/3323719597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e0103bc0 con 0x7f32e0100820 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.443+0000 7f32df7fe640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 0x7f32e01047b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f32d0000c00 tx=0x7f32d0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.444+0000 7f32dcff9640 1 -- 172.21.15.105:0/3323719597 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0043020 con 0x7f32e0100820 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.444+0000 7f32e609a640 1 -- 172.21.15.105:0/3323719597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e01a1bd0 con 0x7f32e0100820 2024-03-26T18:00:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.444+0000 7f32dcff9640 1 -- 172.21.15.105:0/3323719597 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d0004350 con 0x7f32e0100820 2024-03-26T18:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.444+0000 7f32dcff9640 1 -- 172.21.15.105:0/3323719597 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0003b80 con 0x7f32e0100820 2024-03-26T18:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.444+0000 7f32e609a640 1 -- 172.21.15.105:0/3323719597 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e01a2120 con 0x7f32e0100820 2024-03-26T18:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.446+0000 7f32dcff9640 1 -- 172.21.15.105:0/3323719597 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f32d0046020 con 0x7f32e0100820 2024-03-26T18:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.447+0000 7f32dcff9640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32c80a2fc0 0x7f32c80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.447+0000 7f32dcff9640 1 -- 172.21.15.105:0/3323719597 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f32d00f11e0 con 0x7f32e0100820 2024-03-26T18:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.447+0000 7f32e8d65740 1 Processor -- start 2024-03-26T18:00:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.447+0000 7f32deffd640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32c80a2fc0 0x7f32c80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32e8d65740 1 -- start start 2024-03-26T18:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be72ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be73130 0x55a28be755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32e8d65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a28bdd20b0 con 0x55a28be73130 2024-03-26T18:00:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32e8d65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a28bd7a960 con 0x55a28be6dfc0 2024-03-26T18:00:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32e8d65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a28bdce060 con 0x55a28be6d5d0 2024-03-26T18:00:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32df7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32deffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be72ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32df7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be6d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46660/0 (socket says 172.21.15.105:46660) 2024-03-26T18:00:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32df7fe640 1 -- 172.21.15.105:0/294574577 learned_addr learned my addr 172.21.15.105:0/294574577 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32dffff640 1 --2- 172.21.15.105:0/294574577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be73130 0x55a28be755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32deffd640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32c80a2fc0 0x7f32c80a5480 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f32e0104520 tx=0x7f32d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32df7fe640 1 -- 172.21.15.105:0/294574577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 msgr2=0x55a28be72ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32df7fe640 1 --2- 172.21.15.105:0/294574577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be72ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32df7fe640 1 -- 172.21.15.105:0/294574577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be73130 msgr2=0x55a28be755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32df7fe640 1 --2- 172.21.15.105:0/294574577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be73130 0x55a28be755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.448+0000 7f32df7fe640 1 -- 172.21.15.105:0/294574577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28bd8cc60 con 0x55a28be6d5d0 2024-03-26T18:00:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.449+0000 7f32df7fe640 1 --2- 172.21.15.105:0/294574577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be6d9f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f32d00bd860 tx=0x7f32d002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d342dd6a48aa4c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.449+0000 7f32c5ffb640 1 -- 172.21.15.105:0/294574577 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d00d6070 con 0x55a28be6d5d0 2024-03-26T18:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.449+0000 7f32c5ffb640 1 -- 172.21.15.105:0/294574577 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d00bd8d0 con 0x55a28be6d5d0 2024-03-26T18:00:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.449+0000 7f32c5ffb640 1 -- 172.21.15.105:0/294574577 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d00a6e60 con 0x55a28be6d5d0 2024-03-26T18:00:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 -- 172.21.15.105:0/294574577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 msgr2=0x55a28be6d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 --2- 172.21.15.105:0/294574577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be6d9f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f32d00bd860 tx=0x7f32d002bc30 comp rx=0 tx=0).stop 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 -- 172.21.15.105:0/294574577 shutdown_connections 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 --2- 172.21.15.105:0/294574577 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be73130 0x55a28be755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 --2- 172.21.15.105:0/294574577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be72ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 --2- 172.21.15.105:0/294574577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be6d9f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 -- 172.21.15.105:0/294574577 >> 172.21.15.105:0/294574577 conn(0x55a28be68b40 msgr2=0x55a28be68f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 -- 172.21.15.105:0/294574577 shutdown_connections 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 -- 172.21.15.105:0/294574577 wait complete. 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 -- start start 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.450+0000 7f32e8d65740 1 Processor -- start 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be77120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6dfc0 0x55a28be77660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be7e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32e8d65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a28be75ea0 con 0x55a28be6dfc0 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32e8d65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a28bd8ddb0 con 0x55a28be73130 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32e8d65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a28be76020 con 0x55a28be6d5d0 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32e8d65740 10 client.0 _ll_register_callbacks cb 0x55a28bd92110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32dffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be7e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32dffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be7e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46024/0 (socket says 172.21.15.105:46024) 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32dffff640 1 -- 172.21.15.105:0/303669224 learned_addr learned my addr 172.21.15.105:0/303669224 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32df7fe640 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be77120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32deffd640 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6dfc0 0x55a28be77660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32df7fe640 1 -- 172.21.15.105:0/303669224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 msgr2=0x55a28be7e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32df7fe640 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be7e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32df7fe640 1 -- 172.21.15.105:0/303669224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6dfc0 msgr2=0x55a28be77660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32df7fe640 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6dfc0 0x55a28be77660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32df7fe640 1 -- 172.21.15.105:0/303669224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28bae6210 con 0x55a28be6d5d0 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.451+0000 7f32dffff640 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be7e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32df7fe640 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be77120 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f32d00bd860 tx=0x7f32d0005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fadfbf4303b619 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32beffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32beffd640 1 -- 172.21.15.105:0/303669224 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d00db060 con 0x55a28be6d5d0 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32beffd640 1 -- 172.21.15.105:0/303669224 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f32d00a80b0 con 0x55a28be6d5d0 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32e8d65740 1 -- 172.21.15.105:0/303669224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d00bd8d0 con 0x55a28be6d5d0 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32beffd640 1 -- 172.21.15.105:0/303669224 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f32d00d6050 con 0x55a28be6d5d0 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32e8d65740 10 client.33169 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32e8d65740 1 -- 172.21.15.105:0/303669224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a28bbb2620 con 0x55a28be6d5d0 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.452+0000 7f32beffd640 1 -- 172.21.15.105:0/303669224 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d00bd8d0 con 0x55a28be6d5d0 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32beffd640 1 -- 172.21.15.105:0/303669224 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f32d0050060 con 0x55a28be6d5d0 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32e8d65740 10 client.33169 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32e8d65740 10 client.33169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32e8d65740 20 client.33169 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32e8d65740 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28be82ca0 0x55a28be85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32e8d65740 4 client.33169 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32e8d65740 1 -- 172.21.15.105:0/303669224 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a28bddf5f0 con 0x55a28be82ca0 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32deffd640 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28be82ca0 0x55a28be85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32beffd640 10 client.33169 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.453+0000 7f32deffd640 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28be82ca0 0x55a28be85090 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e9532c110237e1 server_cookie=5ba2a624380a18a5 in_seq=0 out_seq=0 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.455+0000 7f32beffd640 1 -- 172.21.15.105:0/303669224 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a28bddf5f0 con 0x55a28be82ca0 2024-03-26T18:00:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.455+0000 7f32beffd640 10 client.33169 handle_command_reply: tid=0 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.455+0000 7f32e8d65740 1 client.33169 shutdown 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 -- 172.21.15.105:0/303669224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 msgr2=0x55a28be77120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be77120 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f32d00bd860 tx=0x7f32d0005be0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 -- 172.21.15.105:0/303669224 shutdown_connections 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28be82ca0 0x55a28be85090 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be7e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6dfc0 0x55a28be77660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 --2- 172.21.15.105:0/303669224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6d5d0 0x55a28be77120 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 -- 172.21.15.105:0/303669224 >> 172.21.15.105:0/303669224 conn(0x55a28be68b40 msgr2=0x55a28be67df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32beffd640 0 client.33169 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 -- 172.21.15.105:0/303669224 shutdown_connections 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 1 -- 172.21.15.105:0/303669224 wait complete. 2024-03-26T18:00:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.456+0000 7f32e8d65740 20 client.33169 trim_cache size 0 max 16384 2024-03-26T18:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32e8d65740 1 Processor -- start 2024-03-26T18:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32e8d65740 1 -- start start 2024-03-26T18:00:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be72b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6dfc0 0x55a28be7e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be6ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32e8d65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32e01a6ad0 con 0x55a28be6d5d0 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32e8d65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32e01a8610 con 0x55a28be73130 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32e8d65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32e01a71a0 con 0x55a28be6dfc0 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32df7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be72b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.480+0000 7f32deffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6dfc0 0x55a28be7e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32deffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6dfc0 0x55a28be7e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46688/0 (socket says 172.21.15.105:46688) 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32deffd640 1 -- 172.21.15.105:0/2808057749 learned_addr learned my addr 172.21.15.105:0/2808057749 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32dffff640 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be6ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32df7fe640 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be72b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:45842/0 (socket says 172.21.15.105:45842) 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32dffff640 1 -- 172.21.15.105:0/2808057749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6dfc0 msgr2=0x55a28be7e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32dffff640 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6dfc0 0x55a28be7e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32dffff640 1 -- 172.21.15.105:0/2808057749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 msgr2=0x55a28be72b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32dffff640 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be72b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32dffff640 1 -- 172.21.15.105:0/2808057749 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28be63c80 con 0x55a28be73130 2024-03-26T18:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32df7fe640 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be72b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32dffff640 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be6ea60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f32d8011aa0 tx=0x7f32d8011f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f862014672ff42cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32c57fa640 1 -- 172.21.15.105:0/2808057749 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8004d20 con 0x55a28be73130 2024-03-26T18:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32c57fa640 1 -- 172.21.15.105:0/2808057749 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d8003970 con 0x55a28be73130 2024-03-26T18:00:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.481+0000 7f32c57fa640 1 -- 172.21.15.105:0/2808057749 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d8004420 con 0x55a28be73130 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.482+0000 7f32e8d65740 1 -- 172.21.15.105:0/2808057749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 msgr2=0x55a28be6ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.482+0000 7f32e8d65740 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be6ea60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f32d8011aa0 tx=0x7f32d8011f60 comp rx=0 tx=0).stop 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.482+0000 7f32e8d65740 1 -- 172.21.15.105:0/2808057749 shutdown_connections 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.482+0000 7f32e8d65740 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be73130 0x55a28be6ea60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.482+0000 7f32e8d65740 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be6dfc0 0x55a28be7e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.482+0000 7f32e8d65740 1 --2- 172.21.15.105:0/2808057749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be72b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.482+0000 7f32e8d65740 1 -- 172.21.15.105:0/2808057749 >> 172.21.15.105:0/2808057749 conn(0x55a28be68b40 msgr2=0x55a28be72400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.482+0000 7f32e8d65740 1 -- 172.21.15.105:0/2808057749 shutdown_connections 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 -- 172.21.15.105:0/2808057749 wait complete. 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 -- start start 2024-03-26T18:00:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 Processor -- start 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be7bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be7c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be73130 0x55a28be7d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32e01a7f40 con 0x55a28be6d5d0 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32e01a7870 con 0x55a28be6dfc0 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32e01a4af0 con 0x55a28be73130 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.483+0000 7f32e8d65740 10 client.0 _ll_register_callbacks cb 0x7f32d00d4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32dffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be73130 0x55a28be7d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be7c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be7c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46042/0 (socket says 172.21.15.105:46042) 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32dffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be73130 0x55a28be7d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46700/0 (socket says 172.21.15.105:46700) 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 -- 172.21.15.105:0/3648990954 learned_addr learned my addr 172.21.15.105:0/3648990954 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32df7fe640 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be7bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 -- 172.21.15.105:0/3648990954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be73130 msgr2=0x55a28be7d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be73130 0x55a28be7d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 -- 172.21.15.105:0/3648990954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 msgr2=0x55a28be7bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be7bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 -- 172.21.15.105:0/3648990954 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28be7d620 con 0x55a28be6dfc0 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32df7fe640 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be7bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32dffff640 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be73130 0x55a28be7d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32deffd640 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be7c020 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f32d4022c20 tx=0x7f32d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f1f0a83c16d5e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32beffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32beffd640 1 -- 172.21.15.105:0/3648990954 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40317c0 con 0x55a28be6dfc0 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32e8d65740 1 -- 172.21.15.105:0/3648990954 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32d8003970 con 0x55a28be6dfc0 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.484+0000 7f32beffd640 1 -- 172.21.15.105:0/3648990954 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f32d40035f0 con 0x55a28be6dfc0 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32beffd640 1 -- 172.21.15.105:0/3648990954 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f32d40044d0 con 0x55a28be6dfc0 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32e8d65740 10 client.33185 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:33.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32e8d65740 1 -- 172.21.15.105:0/3648990954 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a28be7e9d0 con 0x55a28be6dfc0 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32beffd640 1 -- 172.21.15.105:0/3648990954 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d8003970 con 0x55a28be6dfc0 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32beffd640 1 -- 172.21.15.105:0/3648990954 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f32d402f950 con 0x55a28be6dfc0 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32e8d65740 10 client.33185 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32e8d65740 10 client.33185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32e8d65740 20 client.33185 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32e8d65740 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28bee9810 0x55a28beebc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32e8d65740 4 client.33185 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32e8d65740 1 -- 172.21.15.105:0/3648990954 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a28bdd9940 con 0x55a28bee9810 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.485+0000 7f32df7fe640 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28bee9810 0x55a28beebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.486+0000 7f32beffd640 10 client.33185 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:33.472 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.486+0000 7f32df7fe640 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28bee9810 0x55a28beebc30 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ca56b7ccdfffd3 server_cookie=58e64f478d29387 in_seq=0 out_seq=0 2024-03-26T18:00:33.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72846 7.07 MB/sec execute 436 sec latency 183.439 ms 2024-03-26T18:00:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.653+0000 7f32beffd640 1 -- 172.21.15.105:0/3648990954 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a28bdd9940 con 0x55a28bee9810 2024-03-26T18:00:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.653+0000 7f32beffd640 10 client.33185 handle_command_reply: tid=0 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.653+0000 7f32e8d65740 1 client.33185 shutdown 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 -- 172.21.15.105:0/3648990954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 msgr2=0x55a28be7c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be7c020 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f32d4022c20 tx=0x7f32d4004730 comp rx=0 tx=0).stop 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 -- 172.21.15.105:0/3648990954 shutdown_connections 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a28bee9810 0x55a28beebc30 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a28be73130 0x55a28be7d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32beffd640 0 client.33185 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a28be6dfc0 0x55a28be7c020 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 --2- 172.21.15.105:0/3648990954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a28be6d5d0 0x55a28be7bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 -- 172.21.15.105:0/3648990954 >> 172.21.15.105:0/3648990954 conn(0x55a28be68b40 msgr2=0x55a28be76ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 -- 172.21.15.105:0/3648990954 shutdown_connections 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 1 -- 172.21.15.105:0/3648990954 wait complete. 2024-03-26T18:00:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e8d65740 20 client.33185 trim_cache size 0 max 16384 2024-03-26T18:00:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:00:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:00:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:00:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 -- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32c80a2fc0 msgr2=0x7f32c80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32c80a2fc0 0x7f32c80a5480 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f32e0104520 tx=0x7f32d4005f90 comp rx=0 tx=0).stop 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 -- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 msgr2=0x7f32e01047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 0x7f32e01047b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f32d0000c00 tx=0x7f32d0005dc0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 -- 172.21.15.105:0/3323719597 shutdown_connections 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f32c80a2fc0 0x7f32c80a5480 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32e01023d0 0x7f32e0103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32e0101a10 0x7f32e0102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.654+0000 7f32e609a640 1 --2- 172.21.15.105:0/3323719597 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32e0100820 0x7f32e01047b0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.655+0000 7f32e609a640 1 -- 172.21.15.105:0/3323719597 >> 172.21.15.105:0/3323719597 conn(0x7f32e00fbd90 msgr2=0x7f32e00fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.655+0000 7f32e609a640 1 -- 172.21.15.105:0/3323719597 shutdown_connections 2024-03-26T18:00:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:33.655+0000 7f32e609a640 1 -- 172.21.15.105:0/3323719597 wait complete. 2024-03-26T18:00:34.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:34.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:34.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:34.155 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: pgmap v1018: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-03-26T18:00:34.156 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:33 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: pgmap v1018: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-03-26T18:00:34.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:33 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:00:34.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: pgmap v1018: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 4.1 MiB/s wr, 233 op/s 2024-03-26T18:00:34.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:33 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:00:34.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 72984 7.08 MB/sec execute 437 sec latency 236.843 ms 2024-03-26T18:00:35.282 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:34 smithi105 ceph-mon[32280]: pgmap v1019: 97 pgs: 97 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-03-26T18:00:35.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:34 smithi144 ceph-mon[36095]: pgmap v1019: 97 pgs: 97 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-03-26T18:00:35.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:34 smithi135 ceph-mon[35636]: pgmap v1019: 97 pgs: 97 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-03-26T18:00:35.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 73077 7.07 MB/sec execute 438 sec latency 150.491 ms 2024-03-26T18:00:36.558 INFO:tasks.workunit.client.0.smithi105.stdout: 10 73197 7.06 MB/sec execute 439 sec latency 130.163 ms 2024-03-26T18:00:37.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:36 smithi144 ceph-mon[36095]: pgmap v1020: 97 pgs: 97 active+clean; 361 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.4 MiB/s wr, 251 op/s 2024-03-26T18:00:37.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:36 smithi105 ceph-mon[32280]: pgmap v1020: 97 pgs: 97 active+clean; 361 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.4 MiB/s wr, 251 op/s 2024-03-26T18:00:37.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:36 smithi135 ceph-mon[35636]: pgmap v1020: 97 pgs: 97 active+clean; 361 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.4 MiB/s wr, 251 op/s 2024-03-26T18:00:37.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 73341 7.06 MB/sec execute 440 sec latency 128.205 ms 2024-03-26T18:00:38.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 73445 7.05 MB/sec execute 441 sec latency 152.155 ms 2024-03-26T18:00:39.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:38 smithi144 ceph-mon[36095]: pgmap v1021: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-03-26T18:00:39.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:38 smithi105 ceph-mon[32280]: pgmap v1021: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-03-26T18:00:39.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:38 smithi135 ceph-mon[35636]: pgmap v1021: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 3.8 MiB/s wr, 290 op/s 2024-03-26T18:00:39.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 73577 7.05 MB/sec execute 442 sec latency 157.553 ms 2024-03-26T18:00:40.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 73713 7.05 MB/sec execute 443 sec latency 156.008 ms 2024-03-26T18:00:41.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:40 smithi144 ceph-mon[36095]: pgmap v1022: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 3.0 MiB/s wr, 251 op/s 2024-03-26T18:00:41.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:40 smithi105 ceph-mon[32280]: pgmap v1022: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 3.0 MiB/s wr, 251 op/s 2024-03-26T18:00:41.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:40 smithi135 ceph-mon[35636]: pgmap v1022: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 3.0 MiB/s wr, 251 op/s 2024-03-26T18:00:41.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 73824 7.04 MB/sec execute 444 sec latency 176.763 ms 2024-03-26T18:00:42.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 73940 7.04 MB/sec execute 445 sec latency 146.163 ms 2024-03-26T18:00:43.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:42 smithi144 ceph-mon[36095]: pgmap v1023: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-03-26T18:00:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:42 smithi105 ceph-mon[32280]: pgmap v1023: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-03-26T18:00:43.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:42 smithi135 ceph-mon[35636]: pgmap v1023: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.5 MiB/s wr, 285 op/s 2024-03-26T18:00:43.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 74089 7.05 MB/sec execute 446 sec latency 113.496 ms 2024-03-26T18:00:44.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 74216 7.05 MB/sec execute 447 sec latency 128.699 ms 2024-03-26T18:00:45.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:44 smithi144 ceph-mon[36095]: pgmap v1024: 97 pgs: 97 active+clean; 355 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.6 MiB/s wr, 298 op/s 2024-03-26T18:00:45.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:44 smithi105 ceph-mon[32280]: pgmap v1024: 97 pgs: 97 active+clean; 355 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.6 MiB/s wr, 298 op/s 2024-03-26T18:00:45.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:44 smithi135 ceph-mon[35636]: pgmap v1024: 97 pgs: 97 active+clean; 355 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 3.6 MiB/s wr, 298 op/s 2024-03-26T18:00:45.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 74351 7.06 MB/sec execute 448 sec latency 116.763 ms 2024-03-26T18:00:46.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 74503 7.06 MB/sec execute 449 sec latency 132.665 ms 2024-03-26T18:00:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.772+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/3302516061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 msgr2=0x7fb1f0104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.772+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/3302516061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 0x7fb1f0104790 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009860 tx=0x7fb1ec02bb70 comp rx=0 tx=0).stop 2024-03-26T18:00:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.772+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/3302516061 shutdown_connections 2024-03-26T18:00:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.772+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/3302516061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0107600 0x7fb1f0109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.772+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/3302516061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1f0104cd0 0x7fb1f01070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.772+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/3302516061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 0x7fb1f0104790 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.772+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/3302516061 >> 172.21.15.105:0/3302516061 conn(0x7fb1f00fbd90 msgr2=0x7fb1f00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.773+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/3302516061 shutdown_connections 2024-03-26T18:00:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.773+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/3302516061 wait complete. 2024-03-26T18:00:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.773+0000 7fb1f7fe8640 1 Processor -- start 2024-03-26T18:00:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.773+0000 7fb1f7fe8640 1 -- start start 2024-03-26T18:00:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f7fe8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 0x7fb1f019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f7fe8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1f0104cd0 0x7fb1f019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f7fe8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0107600 0x7fb1f01a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f7fe8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01018c0 con 0x7fb1f0104cd0 2024-03-26T18:00:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f7fe8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0101740 con 0x7fb1f0107600 2024-03-26T18:00:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f7fe8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0101a40 con 0x7fb1f01023a0 2024-03-26T18:00:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5d84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 0x7fb1f019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5d84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 0x7fb1f019e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38960/0 (socket says 172.21.15.105:38960) 2024-03-26T18:00:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5d84640 1 -- 172.21.15.105:0/2655346046 learned_addr learned my addr 172.21.15.105:0/2655346046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5583640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1f0104cd0 0x7fb1f019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f6585640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0107600 0x7fb1f01a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5d84640 1 -- 172.21.15.105:0/2655346046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0107600 msgr2=0x7fb1f01a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5d84640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0107600 0x7fb1f01a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5d84640 1 -- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1f0104cd0 msgr2=0x7fb1f019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5d84640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1f0104cd0 0x7fb1f019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.774+0000 7fb1f5d84640 1 -- 172.21.15.105:0/2655346046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f01a5cc0 con 0x7fb1f01023a0 2024-03-26T18:00:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.775+0000 7fb1f5d84640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 0x7fb1f019e0a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0057c0 tx=0x7fb1ec0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.775+0000 7fb1e6ffd640 1 -- 172.21.15.105:0/2655346046 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec041030 con 0x7fb1f01023a0 2024-03-26T18:00:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.775+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/2655346046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f01a5f50 con 0x7fb1f01023a0 2024-03-26T18:00:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.775+0000 7fb1e6ffd640 1 -- 172.21.15.105:0/2655346046 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1ec02c9c0 con 0x7fb1f01023a0 2024-03-26T18:00:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.775+0000 7fb1e6ffd640 1 -- 172.21.15.105:0/2655346046 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec039040 con 0x7fb1f01023a0 2024-03-26T18:00:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.775+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/2655346046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a63b0 con 0x7fb1f01023a0 2024-03-26T18:00:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.778+0000 7fb1fabcc740 1 Processor -- start 2024-03-26T18:00:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.778+0000 7fb1fabcc740 1 -- start start 2024-03-26T18:00:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.778+0000 7fb1e6ffd640 1 -- 172.21.15.105:0/2655346046 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb1ec044070 con 0x7fb1f01023a0 2024-03-26T18:00:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.778+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x55723993e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993efc0 0x557239943ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x5572399465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5572398a30b0 con 0x55723993e5d0 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55723984b960 con 0x557239944130 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55723989f060 con 0x55723993efc0 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f5d84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993efc0 0x557239943ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f6585640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x55723993e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f5d84640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993efc0 0x557239943ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38968/0 (socket says 172.21.15.105:38968) 2024-03-26T18:00:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f5d84640 1 -- 172.21.15.105:0/2432196106 learned_addr learned my addr 172.21.15.105:0/2432196106 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f5583640 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x5572399465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f6585640 1 -- 172.21.15.105:0/2432196106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993efc0 msgr2=0x557239943ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f6585640 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993efc0 0x557239943ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f6585640 1 -- 172.21.15.105:0/2432196106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 msgr2=0x5572399465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f6585640 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x5572399465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f6585640 1 -- 172.21.15.105:0/2432196106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723985dc60 con 0x55723993e5d0 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f5583640 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x5572399465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1e6ffd640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1dc0a3040 0x7fb1dc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.779+0000 7fb1f6585640 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x55723993e9f0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009f30 tx=0x7fb1e8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5709ad2f1eb08e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.780+0000 7fb1e6ffd640 1 -- 172.21.15.105:0/2655346046 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fb1ec0f0ac0 con 0x7fb1f01023a0 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.780+0000 7fb1dbfff640 1 -- 172.21.15.105:0/2432196106 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8016070 con 0x55723993e5d0 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.780+0000 7fb1f5d84640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1dc0a3040 0x7fb1dc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.780+0000 7fb1dbfff640 1 -- 172.21.15.105:0/2432196106 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1e8004d20 con 0x55723993e5d0 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.780+0000 7fb1dbfff640 1 -- 172.21.15.105:0/2432196106 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8012740 con 0x55723993e5d0 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1f5d84640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1dc0a3040 0x7fb1dc0a5500 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0b95b0 tx=0x7fb1ec0dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2432196106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 msgr2=0x55723993e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x55723993e9f0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009f30 tx=0x7fb1e8010800 comp rx=0 tx=0).stop 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2432196106 shutdown_connections 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x5572399465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993efc0 0x557239943ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/2432196106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x55723993e9f0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2432196106 >> 172.21.15.105:0/2432196106 conn(0x557239939b40 msgr2=0x557239939f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2432196106 shutdown_connections 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2432196106 wait complete. 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 -- start start 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.781+0000 7fb1fabcc740 1 Processor -- start 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 0x557239948460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993efc0 0x5572399489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x55723994f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5572399470e0 con 0x55723993efc0 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1f5583640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993efc0 0x5572399489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1f5583640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993efc0 0x5572399489a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53438/0 (socket says 172.21.15.105:53438) 2024-03-26T18:00:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1f6585640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 0x557239948460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1f5583640 1 -- 172.21.15.105:0/481044431 learned_addr learned my addr 172.21.15.105:0/481044431 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55723985edb0 con 0x557239944130 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557239947260 con 0x55723993e5d0 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1f6585640 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x55723994f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.782+0000 7fb1fabcc740 10 client.0 _ll_register_callbacks cb 0x557239863110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1f5583640 1 -- 172.21.15.105:0/481044431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 msgr2=0x557239948460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1f5583640 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 0x557239948460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1f5583640 1 -- 172.21.15.105:0/481044431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 msgr2=0x55723994f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1f5583640 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x55723994f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1f5583640 1 -- 172.21.15.105:0/481044431 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572395b7210 con 0x55723993efc0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1f5583640 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993efc0 0x5572399489a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00116c0 tx=0x7fb1e0011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e07fa2a8352d2b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1d97fa640 1 -- 172.21.15.105:0/481044431 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000fad0 con 0x55723993efc0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1d97fa640 1 -- 172.21.15.105:0/481044431 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fb1e000fc70 con 0x55723993efc0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e8004d20 con 0x55723993efc0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1d97fa640 1 -- 172.21.15.105:0/481044431 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb1e001a3f0 con 0x55723993efc0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1fabcc740 10 client.23523 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.783+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557239683620 con 0x55723993efc0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1d97fa640 1 -- 172.21.15.105:0/481044431 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1e8004d20 con 0x55723993efc0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1d97fa640 1 -- 172.21.15.105:0/481044431 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb1e0019070 con 0x55723993efc0 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1fabcc740 10 client.23523 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1fabcc740 10 client.23523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1fabcc740 20 client.23523 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572399547f0 0x557239956c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1fabcc740 4 client.23523 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:00:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572398b05f0 con 0x5572399547f0 2024-03-26T18:00:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.784+0000 7fb1f6585640 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572399547f0 0x557239956c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.785+0000 7fb1f6585640 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572399547f0 0x557239956c70 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c958fafcbe92d8 server_cookie=2330ff35bec2cd5c in_seq=0 out_seq=0 2024-03-26T18:00:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.785+0000 7fb1d97fa640 10 client.23523 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1d97fa640 1 -- 172.21.15.105:0/481044431 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5572398b05f0 con 0x5572399547f0 2024-03-26T18:00:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1d97fa640 10 client.23523 handle_command_reply: tid=0 2024-03-26T18:00:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 client.23523 shutdown 2024-03-26T18:00:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993efc0 msgr2=0x5572399489a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993efc0 0x5572399489a0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00116c0 tx=0x7fb1e0011b80 comp rx=0 tx=0).stop 2024-03-26T18:00:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 shutdown_connections 2024-03-26T18:00:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572399547f0 0x557239956c70 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557239944130 0x55723994f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993efc0 0x5572399489a0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/481044431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 0x557239948460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 >> 172.21.15.105:0/481044431 conn(0x557239939b40 msgr2=0x557239938df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1d97fa640 0 client.23523 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 shutdown_connections 2024-03-26T18:00:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 1 -- 172.21.15.105:0/481044431 wait complete. 2024-03-26T18:00:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.787+0000 7fb1fabcc740 20 client.23523 trim_cache size 0 max 16384 2024-03-26T18:00:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1fabcc740 1 Processor -- start 2024-03-26T18:00:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1fabcc740 1 -- start start 2024-03-26T18:00:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x557239940170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557239944130 0x557239938560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01af390 con 0x55723993e5d0 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01b0ed0 con 0x55723993efc0 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01afa60 con 0x557239944130 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1f6585640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x557239940170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1f6585640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x557239940170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53450/0 (socket says 172.21.15.105:53450) 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1f6585640 1 -- 172.21.15.105:0/2166951583 learned_addr learned my addr 172.21.15.105:0/2166951583 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1f5583640 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.810+0000 7fb1f6585640 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557239944130 0x557239938560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1f6585640 1 -- 172.21.15.105:0/2166951583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557239944130 msgr2=0x557239938560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1f6585640 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557239944130 0x557239938560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1f6585640 1 -- 172.21.15.105:0/2166951583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 msgr2=0x55723994f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1f6585640 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1f6585640 1 -- 172.21.15.105:0/2166951583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557239935450 con 0x55723993e5d0 2024-03-26T18:00:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1f5583640 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:00:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1f6585640 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x557239940170 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009f30 tx=0x7fb1e8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bd2d1d42c5db66 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1db7fe640 1 -- 172.21.15.105:0/2166951583 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8016040 con 0x55723993e5d0 2024-03-26T18:00:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1db7fe640 1 -- 172.21.15.105:0/2166951583 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1e80059c0 con 0x55723993e5d0 2024-03-26T18:00:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.811+0000 7fb1db7fe640 1 -- 172.21.15.105:0/2166951583 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8012cd0 con 0x55723993e5d0 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2166951583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 msgr2=0x557239940170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x557239940170 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009f30 tx=0x7fb1e8010800 comp rx=0 tx=0).stop 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2166951583 shutdown_connections 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557239944130 0x557239938560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/2166951583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55723993e5d0 0x557239940170 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2166951583 >> 172.21.15.105:0/2166951583 conn(0x557239939b40 msgr2=0x55723993fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2166951583 shutdown_connections 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 -- 172.21.15.105:0/2166951583 wait complete. 2024-03-26T18:00:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.812+0000 7fb1fabcc740 1 -- start start 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1fabcc740 1 Processor -- start 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 0x55723994c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1fabcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557239944130 0x5572399b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01b0800 con 0x557239944130 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01b0130 con 0x55723993efc0 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1fabcc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01ad370 con 0x55723993e5d0 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1fabcc740 10 client.0 _ll_register_callbacks cb 0x7fb1e0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1f6585640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557239944130 0x5572399b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1f5583640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1f6585640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557239944130 0x5572399b8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53464/0 (socket says 172.21.15.105:53464) 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1f6585640 1 -- 172.21.15.105:0/891855216 learned_addr learned my addr 172.21.15.105:0/891855216 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.813+0000 7fb1f6585640 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 0x55723994c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1f6585640 1 -- 172.21.15.105:0/891855216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 msgr2=0x55723994c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1f6585640 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 0x55723994c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1f6585640 1 -- 172.21.15.105:0/891855216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 msgr2=0x55723994ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1f6585640 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1f6585640 1 -- 172.21.15.105:0/891855216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572399b9110 con 0x557239944130 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1f6585640 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557239944130 0x5572399b8ba0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8005fd0 tx=0x7fb1e8012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16137392d5c3f9ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1d97fa640 1 -- 172.21.15.105:0/891855216 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8016040 con 0x557239944130 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1d97fa640 1 -- 172.21.15.105:0/891855216 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fb1e802e100 con 0x557239944130 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1fabcc740 1 -- 172.21.15.105:0/891855216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e80059c0 con 0x557239944130 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1d97fa640 1 -- 172.21.15.105:0/891855216 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb1e802e300 con 0x557239944130 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1fabcc740 10 client.23529 fetch_fsmap learned FSMap version 43 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.814+0000 7fb1fabcc740 1 -- 172.21.15.105:0/891855216 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5572399ba400 con 0x557239944130 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.815+0000 7fb1d97fa640 1 -- 172.21.15.105:0/891855216 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1e803d420 con 0x557239944130 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.815+0000 7fb1d97fa640 1 -- 172.21.15.105:0/891855216 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb1e8028040 con 0x557239944130 2024-03-26T18:00:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.815+0000 7fb1fabcc740 10 client.23529 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:00:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.815+0000 7fb1fabcc740 10 client.23529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:00:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.815+0000 7fb1fabcc740 20 client.23529 populate_metadata read hostname 'smithi105' 2024-03-26T18:00:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.815+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572399babc0 0x5572399bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:00:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.816+0000 7fb1f5583640 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572399babc0 0x5572399bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:00:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.816+0000 7fb1fabcc740 4 client.23529 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:00:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.816+0000 7fb1fabcc740 1 -- 172.21.15.105:0/891855216 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb1e80059c0 con 0x5572399babc0 2024-03-26T18:00:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.816+0000 7fb1f5583640 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572399babc0 0x5572399bcfe0 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1722bff3786f7ea server_cookie=cb7fc1f1332fc3e4 in_seq=0 out_seq=0 2024-03-26T18:00:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.816+0000 7fb1d97fa640 10 client.23529 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.994+0000 7fb1d97fa640 1 -- 172.21.15.105:0/891855216 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fb1e80059c0 con 0x5572399babc0 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1d97fa640 10 client.23529 handle_command_reply: tid=0 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 client.23529 shutdown 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 -- 172.21.15.105:0/891855216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557239944130 msgr2=0x5572399b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557239944130 0x5572399b8ba0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8005fd0 tx=0x7fb1e8012450 comp rx=0 tx=0).stop 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 -- 172.21.15.105:0/891855216 shutdown_connections 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5572399babc0 0x5572399bcfe0 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557239944130 0x5572399b8ba0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55723993efc0 0x55723994ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 --2- 172.21.15.105:0/891855216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55723993e5d0 0x55723994c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1d97fa640 0 client.23529 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:00:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 -- 172.21.15.105:0/891855216 >> 172.21.15.105:0/891855216 conn(0x557239939b40 msgr2=0x55723993cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 -- 172.21.15.105:0/891855216 shutdown_connections 2024-03-26T18:00:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 1 -- 172.21.15.105:0/891855216 wait complete. 2024-03-26T18:00:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.995+0000 7fb1fabcc740 20 client.23529 trim_cache size 0 max 16384 2024-03-26T18:00:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:00:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:00:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:00:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1dc0a3040 msgr2=0x7fb1dc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1dc0a3040 0x7fb1dc0a5500 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0b95b0 tx=0x7fb1ec0dd040 comp rx=0 tx=0).stop 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/2655346046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 msgr2=0x7fb1f019e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 0x7fb1f019e0a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0057c0 tx=0x7fb1ec0042f0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/2655346046 shutdown_connections 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb1dc0a3040 0x7fb1dc0a5500 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb1f0107600 0x7fb1f01a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1f0104cd0 0x7fb1f019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 --2- 172.21.15.105:0/2655346046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1f01023a0 0x7fb1f019e0a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:00:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/2655346046 >> 172.21.15.105:0/2655346046 conn(0x7fb1f00fbd90 msgr2=0x7fb1f0102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:00:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/2655346046 shutdown_connections 2024-03-26T18:00:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:00:46.996+0000 7fb1f7fe8640 1 -- 172.21.15.105:0/2655346046 wait complete. 2024-03-26T18:00:47.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:46 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:00:47.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:46 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:00:47.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:46 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:00:47.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 74592 7.05 MB/sec execute 450 sec latency 229.268 ms 2024-03-26T18:00:48.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:47 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:00:48.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:47 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:00:48.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:47 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:00:48.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:47 smithi144 ceph-mon[36095]: pgmap v1025: 97 pgs: 97 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-03-26T18:00:48.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:47 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:00:48.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:47 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:00:48.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:47 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:00:48.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:47 smithi105 ceph-mon[32280]: pgmap v1025: 97 pgs: 97 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-03-26T18:00:48.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:47 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:00:48.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:47 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:00:48.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:47 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:00:48.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:47 smithi135 ceph-mon[35636]: pgmap v1025: 97 pgs: 97 active+clean; 358 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-03-26T18:00:48.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 74725 7.04 MB/sec execute 451 sec latency 120.222 ms 2024-03-26T18:00:49.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 74861 7.04 MB/sec execute 452 sec latency 122.020 ms 2024-03-26T18:00:50.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:49 smithi144 ceph-mon[36095]: pgmap v1026: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T18:00:50.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:49 smithi105 ceph-mon[32280]: pgmap v1026: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T18:00:50.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:49 smithi135 ceph-mon[35636]: pgmap v1026: 97 pgs: 97 active+clean; 356 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.4 MiB/s wr, 264 op/s 2024-03-26T18:00:50.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 74992 7.02 MB/sec execute 453 sec latency 174.825 ms 2024-03-26T18:00:51.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 75130 7.02 MB/sec execute 454 sec latency 116.230 ms 2024-03-26T18:00:52.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:51 smithi144 ceph-mon[36095]: pgmap v1027: 97 pgs: 97 active+clean; 357 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-03-26T18:00:52.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:51 smithi105 ceph-mon[32280]: pgmap v1027: 97 pgs: 97 active+clean; 357 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-03-26T18:00:52.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:51 smithi135 ceph-mon[35636]: pgmap v1027: 97 pgs: 97 active+clean; 357 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-03-26T18:00:52.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 75273 7.02 MB/sec execute 455 sec latency 134.400 ms 2024-03-26T18:00:53.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 75422 7.02 MB/sec execute 456 sec latency 112.067 ms 2024-03-26T18:00:54.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:53 smithi144 ceph-mon[36095]: pgmap v1028: 97 pgs: 97 active+clean; 349 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-26T18:00:54.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:53 smithi105 ceph-mon[32280]: pgmap v1028: 97 pgs: 97 active+clean; 349 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-26T18:00:54.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:53 smithi135 ceph-mon[35636]: pgmap v1028: 97 pgs: 97 active+clean; 349 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 3.0 MiB/s wr, 272 op/s 2024-03-26T18:00:54.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 75566 7.02 MB/sec execute 457 sec latency 114.039 ms 2024-03-26T18:00:55.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 75716 7.02 MB/sec execute 458 sec latency 126.653 ms 2024-03-26T18:00:56.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:55 smithi144 ceph-mon[36095]: pgmap v1029: 97 pgs: 97 active+clean; 347 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2024-03-26T18:00:56.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:55 smithi105 ceph-mon[32280]: pgmap v1029: 97 pgs: 97 active+clean; 347 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2024-03-26T18:00:56.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:55 smithi135 ceph-mon[35636]: pgmap v1029: 97 pgs: 97 active+clean; 347 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.9 MiB/s wr, 275 op/s 2024-03-26T18:00:56.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 75849 7.02 MB/sec execute 459 sec latency 145.549 ms 2024-03-26T18:00:57.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 76024 7.04 MB/sec execute 460 sec latency 140.321 ms 2024-03-26T18:00:58.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:57 smithi144 ceph-mon[36095]: pgmap v1030: 97 pgs: 97 active+clean; 349 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-26T18:00:58.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:57 smithi105 ceph-mon[32280]: pgmap v1030: 97 pgs: 97 active+clean; 349 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-26T18:00:58.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:57 smithi135 ceph-mon[35636]: pgmap v1030: 97 pgs: 97 active+clean; 349 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-03-26T18:00:58.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 76212 7.05 MB/sec execute 461 sec latency 132.172 ms 2024-03-26T18:00:59.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 76380 7.07 MB/sec execute 462 sec latency 165.508 ms 2024-03-26T18:01:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/3433769553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f0102390 msgr2=0x7fc4f0108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/3433769553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f0102390 0x7fc4f0108c20 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009ca0 tx=0x7fc4e802c100 comp rx=0 tx=0).stop 2024-03-26T18:01:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/3433769553 shutdown_connections 2024-03-26T18:01:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/3433769553 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f0102390 0x7fc4f0108c20 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/3433769553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4f01019d0 0x7fc4f0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/3433769553 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f01007e0 0x7fc4f0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/3433769553 >> 172.21.15.105:0/3433769553 conn(0x7fc4f00fbd50 msgr2=0x7fc4f00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/3433769553 shutdown_connections 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.120+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/3433769553 wait complete. 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f6aa0640 1 Processor -- start 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f6aa0640 1 -- start start 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f6aa0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f01007e0 0x7fc4f0196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f6aa0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f01019d0 0x7fc4f0196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f6aa0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4f0102390 0x7fc4f019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f6aa0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f010c310 con 0x7fc4f01019d0 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f6aa0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f010c190 con 0x7fc4f01007e0 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f6aa0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f010c490 con 0x7fc4f0102390 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f01019d0 0x7fc4f0196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f01019d0 0x7fc4f0196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53604/0 (socket says 172.21.15.105:53604) 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4effff640 1 -- 172.21.15.105:0/271445544 learned_addr learned my addr 172.21.15.105:0/271445544 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f483c640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f01007e0 0x7fc4f0196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.121+0000 7fc4f503d640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4f0102390 0x7fc4f019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4effff640 1 -- 172.21.15.105:0/271445544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4f0102390 msgr2=0x7fc4f019df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4effff640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4f0102390 0x7fc4f019df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4effff640 1 -- 172.21.15.105:0/271445544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f01007e0 msgr2=0x7fc4f0196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4effff640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f01007e0 0x7fc4f0196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4effff640 1 -- 172.21.15.105:0/271445544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f019e570 con 0x7fc4f01019d0 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4effff640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f01019d0 0x7fc4f0196e90 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00e9f0 tx=0x7fc4dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4edffb640 1 -- 172.21.15.105:0/271445544 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00cd80 con 0x7fc4f01019d0 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/271445544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f019e860 con 0x7fc4f01019d0 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4edffb640 1 -- 172.21.15.105:0/271445544 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4dc0054d0 con 0x7fc4f01019d0 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.122+0000 7fc4edffb640 1 -- 172.21.15.105:0/271445544 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc010430 con 0x7fc4f01019d0 2024-03-26T18:01:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.123+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/271445544 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f019ed60 con 0x7fc4f01019d0 2024-03-26T18:01:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.125+0000 7fc4edffb640 1 -- 172.21.15.105:0/271445544 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fc4dc005020 con 0x7fc4f01019d0 2024-03-26T18:01:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.125+0000 7fc4edffb640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4e00a2fc0 0x7fc4e00a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.126+0000 7fc4edffb640 1 -- 172.21.15.105:0/271445544 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fc4dc0cf800 con 0x7fc4f01019d0 2024-03-26T18:01:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.126+0000 7fc4f9685740 1 Processor -- start 2024-03-26T18:01:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.126+0000 7fc4f483c640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4e00a2fc0 0x7fc4e00a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.127+0000 7fc4f9685740 1 -- start start 2024-03-26T18:01:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.127+0000 7fc4f483c640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4e00a2fc0 0x7fc4e00a5480 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0101830 tx=0x7fc4e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.127+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe70fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fdfc0 0x564fe7102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe7103130 0x564fe71055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f9685740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564fe70620b0 con 0x564fe7103130 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f9685740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564fe700a960 con 0x564fe70fdfc0 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe70fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f9685740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564fe705e060 con 0x564fe70fd5d0 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe70fd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55428/0 (socket says 172.21.15.105:55428) 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 -- 172.21.15.105:0/1057077276 learned_addr learned my addr 172.21.15.105:0/1057077276 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4effff640 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fdfc0 0x564fe7102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f503d640 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe7103130 0x564fe71055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 -- 172.21.15.105:0/1057077276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fdfc0 msgr2=0x564fe7102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fdfc0 0x564fe7102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 -- 172.21.15.105:0/1057077276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe7103130 msgr2=0x564fe71055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe7103130 0x564fe71055c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 -- 172.21.15.105:0/1057077276 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe701cc60 con 0x564fe70fd5d0 2024-03-26T18:01:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4effff640 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fdfc0 0x564fe7102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.128+0000 7fc4f483c640 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe70fd9f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400e630 tx=0x7fc4e4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bed34f6d4ae0f79 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.129+0000 7fc4daffd640 1 -- 172.21.15.105:0/1057077276 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4026430 con 0x564fe70fd5d0 2024-03-26T18:01:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.129+0000 7fc4daffd640 1 -- 172.21.15.105:0/1057077276 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4e401c8c0 con 0x564fe70fd5d0 2024-03-26T18:01:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.129+0000 7fc4daffd640 1 -- 172.21.15.105:0/1057077276 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e401d8f0 con 0x564fe70fd5d0 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 -- 172.21.15.105:0/1057077276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 msgr2=0x564fe70fd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe70fd9f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400e630 tx=0x7fc4e4014000 comp rx=0 tx=0).stop 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 -- 172.21.15.105:0/1057077276 shutdown_connections 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe7103130 0x564fe71055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fdfc0 0x564fe7102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 --2- 172.21.15.105:0/1057077276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe70fd9f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 -- 172.21.15.105:0/1057077276 >> 172.21.15.105:0/1057077276 conn(0x564fe70f8b40 msgr2=0x564fe70f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 -- 172.21.15.105:0/1057077276 shutdown_connections 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.130+0000 7fc4f9685740 1 -- 172.21.15.105:0/1057077276 wait complete. 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.131+0000 7fc4f9685740 1 -- start start 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.131+0000 7fc4f9685740 1 Processor -- start 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.131+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fd5d0 0x564fe71072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.131+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe7107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.131+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe7103130 0x564fe710e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f9685740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564fe71060e0 con 0x564fe70fdfc0 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f9685740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564fe701ddb0 con 0x564fe70fd5d0 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f9685740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564fe7106260 con 0x564fe7103130 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe7107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe7103130 0x564fe710e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe7103130 0x564fe710e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55436/0 (socket says 172.21.15.105:55436) 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 -- 172.21.15.105:0/2541323855 learned_addr learned my addr 172.21.15.105:0/2541323855 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f483c640 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fd5d0 0x564fe71072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f9685740 10 client.0 _ll_register_callbacks cb 0x564fe7022110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 -- 172.21.15.105:0/2541323855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fd5d0 msgr2=0x564fe71072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fd5d0 0x564fe71072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 -- 172.21.15.105:0/2541323855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 msgr2=0x564fe7107820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe7107820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 -- 172.21.15.105:0/2541323855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe6d76210 con 0x564fe7103130 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f9685740 1 -- 172.21.15.105:0/2541323855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e401c8c0 con 0x564fe7103130 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4effff640 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe7107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4f503d640 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe7103130 0x564fe710e860 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8037040 tx=0x7fc4e8009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4776d586f79f1ded server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.132+0000 7fc4cffff640 10 client.33205 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4cffff640 1 -- 172.21.15.105:0/2541323855 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8044070 con 0x564fe7103130 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4cffff640 1 -- 172.21.15.105:0/2541323855 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fc4e8003970 con 0x564fe7103130 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4cffff640 1 -- 172.21.15.105:0/2541323855 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc4e8039040 con 0x564fe7103130 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4f9685740 10 client.33205 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4f9685740 1 -- 172.21.15.105:0/2541323855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=43}) -- 0x564fe6e42620 con 0x564fe7103130 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4cffff640 1 -- 172.21.15.105:0/2541323855 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc4e8003b70 con 0x564fe7103130 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4f9685740 10 client.33205 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4f9685740 10 client.33205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.133+0000 7fc4f9685740 20 client.33205 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.134+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564fe7112f60 0x564fe7115390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.134+0000 7fc4f9685740 4 client.33205 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.134+0000 7fc4f9685740 1 -- 172.21.15.105:0/2541323855 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fe706f5f0 con 0x564fe7112f60 2024-03-26T18:01:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.134+0000 7fc4f483c640 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564fe7112f60 0x564fe7115390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.134+0000 7fc4f483c640 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564fe7112f60 0x564fe7115390 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56499fee9e20a58c server_cookie=963a866edfb80eba in_seq=0 out_seq=0 2024-03-26T18:01:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.134+0000 7fc4cffff640 10 client.33205 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4cffff640 1 -- 172.21.15.105:0/2541323855 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564fe706f5f0 con 0x564fe7112f60 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4cffff640 10 client.33205 handle_command_reply: tid=0 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4f9685740 1 client.33205 shutdown 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4f9685740 1 -- 172.21.15.105:0/2541323855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe7103130 msgr2=0x564fe710e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe7103130 0x564fe710e860 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8037040 tx=0x7fc4e8009890 comp rx=0 tx=0).stop 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4f9685740 1 -- 172.21.15.105:0/2541323855 shutdown_connections 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564fe7112f60 0x564fe7115390 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe7103130 0x564fe710e860 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4cffff640 0 client.33205 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe7107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.136+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2541323855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe70fd5d0 0x564fe71072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.137+0000 7fc4f9685740 1 -- 172.21.15.105:0/2541323855 >> 172.21.15.105:0/2541323855 conn(0x564fe70f8b40 msgr2=0x564fe70f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.137+0000 7fc4f9685740 1 -- 172.21.15.105:0/2541323855 shutdown_connections 2024-03-26T18:01:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.137+0000 7fc4f9685740 1 -- 172.21.15.105:0/2541323855 wait complete. 2024-03-26T18:01:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.137+0000 7fc4f9685740 20 client.33205 trim_cache size 0 max 16384 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f9685740 1 Processor -- start 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f9685740 1 -- start start 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe711ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe710e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 0x564fe7102310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f9685740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01a6ab0 con 0x564fe70fdfc0 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f9685740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01a85f0 con 0x564fe7103130 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f9685740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01a7180 con 0x564fe70fd5d0 2024-03-26T18:01:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe710e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f483c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe711ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe710e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53630/0 (socket says 172.21.15.105:53630) 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f483c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe711ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55442/0 (socket says 172.21.15.105:55442) 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4effff640 1 -- 172.21.15.105:0/3449901451 learned_addr learned my addr 172.21.15.105:0/3449901451 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f503d640 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 0x564fe7102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f483c640 1 -- 172.21.15.105:0/3449901451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 msgr2=0x564fe7102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f483c640 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 0x564fe7102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f483c640 1 -- 172.21.15.105:0/3449901451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 msgr2=0x564fe710e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f483c640 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe710e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4f483c640 1 -- 172.21.15.105:0/3449901451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe7101f90 con 0x564fe70fd5d0 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.159+0000 7fc4effff640 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe710e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.160+0000 7fc4f483c640 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe711ac70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009a90 tx=0x7fc4e40099c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3f7acae6322df7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.160+0000 7fc4da7fc640 1 -- 172.21.15.105:0/3449901451 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e403a070 con 0x564fe70fd5d0 2024-03-26T18:01:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.160+0000 7fc4da7fc640 1 -- 172.21.15.105:0/3449901451 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4e401c410 con 0x564fe70fd5d0 2024-03-26T18:01:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.160+0000 7fc4da7fc640 1 -- 172.21.15.105:0/3449901451 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e402b040 con 0x564fe70fd5d0 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 -- 172.21.15.105:0/3449901451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 msgr2=0x564fe711ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe711ac70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009a90 tx=0x7fc4e40099c0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 -- 172.21.15.105:0/3449901451 shutdown_connections 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 0x564fe7102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fdfc0 0x564fe710e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 --2- 172.21.15.105:0/3449901451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fd5d0 0x564fe711ac70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 -- 172.21.15.105:0/3449901451 >> 172.21.15.105:0/3449901451 conn(0x564fe70f8b40 msgr2=0x564fe71072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 -- 172.21.15.105:0/3449901451 shutdown_connections 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 -- 172.21.15.105:0/3449901451 wait complete. 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 -- start start 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.161+0000 7fc4f9685740 1 Processor -- start 2024-03-26T18:01:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fd5d0 0x564fe710be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fdfc0 0x564fe710c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f9685740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 0x564fe710d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f9685740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01a7f20 con 0x564fe70fd5d0 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f9685740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01a7850 con 0x564fe7103130 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f9685740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01a4a90 con 0x564fe70fdfc0 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f9685740 10 client.0 _ll_register_callbacks cb 0x7fc4dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fd5d0 0x564fe710be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fd5d0 0x564fe710be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53632/0 (socket says 172.21.15.105:53632) 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 -- 172.21.15.105:0/2388665595 learned_addr learned my addr 172.21.15.105:0/2388665595 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4effff640 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fdfc0 0x564fe710c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f503d640 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 0x564fe710d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 -- 172.21.15.105:0/2388665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fdfc0 msgr2=0x564fe710c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fdfc0 0x564fe710c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 -- 172.21.15.105:0/2388665595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 msgr2=0x564fe710d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 0x564fe710d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 -- 172.21.15.105:0/2388665595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe710d930 con 0x564fe70fd5d0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4effff640 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fdfc0 0x564fe710c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4f483c640 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fd5d0 0x564fe710be90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc4e401d010 tx=0x7fc4e400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3e2f4cf03070b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.162+0000 7fc4cffff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4cffff640 1 -- 172.21.15.105:0/2388665595 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e403a030 con 0x564fe70fd5d0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4cffff640 1 -- 172.21.15.105:0/2388665595 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fc4e400ede0 con 0x564fe70fd5d0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4f9685740 1 -- 172.21.15.105:0/2388665595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fe7069940 con 0x564fe70fd5d0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4cffff640 1 -- 172.21.15.105:0/2388665595 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc4e401ca70 con 0x564fe70fd5d0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4f9685740 10 client.23547 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4f9685740 1 -- 172.21.15.105:0/2388665595 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564fe717d6b0 con 0x564fe70fd5d0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4cffff640 1 -- 172.21.15.105:0/2388665595 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564fe7069940 con 0x564fe70fd5d0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4cffff640 1 -- 172.21.15.105:0/2388665595 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc4e4005490 con 0x564fe70fd5d0 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.163+0000 7fc4f9685740 10 client.23547 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:00.150 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.164+0000 7fc4f9685740 10 client.23547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.164+0000 7fc4f9685740 20 client.23547 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.164+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564fe7112f60 0x564fe717fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.164+0000 7fc4f9685740 4 client.23547 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:01:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.164+0000 7fc4f9685740 1 -- 172.21.15.105:0/2388665595 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564fe6fe7770 con 0x564fe7112f60 2024-03-26T18:01:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.164+0000 7fc4effff640 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564fe7112f60 0x564fe717fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.164+0000 7fc4cffff640 10 client.23547 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:00.151 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.164+0000 7fc4effff640 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564fe7112f60 0x564fe717fb10 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16198f1077992497 server_cookie=674ddc6bb44e6bc8 in_seq=0 out_seq=0 2024-03-26T18:01:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4cffff640 1 -- 172.21.15.105:0/2388665595 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564fe6fe7770 con 0x564fe7112f60 2024-03-26T18:01:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4cffff640 10 client.23547 handle_command_reply: tid=0 2024-03-26T18:01:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 client.23547 shutdown 2024-03-26T18:01:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 -- 172.21.15.105:0/2388665595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fd5d0 msgr2=0x564fe710be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fd5d0 0x564fe710be90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fc4e401d010 tx=0x7fc4e400ec20 comp rx=0 tx=0).stop 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 -- 172.21.15.105:0/2388665595 shutdown_connections 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564fe7112f60 0x564fe717fb10 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4cffff640 0 client.23547 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564fe7103130 0x564fe710d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564fe70fdfc0 0x564fe710c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 --2- 172.21.15.105:0/2388665595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564fe70fd5d0 0x564fe710be90 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 -- 172.21.15.105:0/2388665595 >> 172.21.15.105:0/2388665595 conn(0x564fe70f8b40 msgr2=0x564fe71022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 -- 172.21.15.105:0/2388665595 shutdown_connections 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 1 -- 172.21.15.105:0/2388665595 wait complete. 2024-03-26T18:01:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.323+0000 7fc4f9685740 20 client.23547 trim_cache size 0 max 16384 2024-03-26T18:01:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:01:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:01:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:01:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4e00a2fc0 msgr2=0x7fc4e00a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4e00a2fc0 0x7fc4e00a5480 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0101830 tx=0x7fc4e4005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f01019d0 msgr2=0x7fc4f0196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f01019d0 0x7fc4f0196e90 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00e9f0 tx=0x7fc4dc00edf0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/271445544 shutdown_connections 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fc4e00a2fc0 0x7fc4e00a5480 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4f0102390 0x7fc4f019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4f01019d0 0x7fc4f0196e90 secure :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00e9f0 tx=0x7fc4dc00edf0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 --2- 172.21.15.105:0/271445544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc4f01007e0 0x7fc4f0196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.324+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/271445544 >> 172.21.15.105:0/271445544 conn(0x7fc4f00fbd50 msgr2=0x7fc4f0104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.325+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/271445544 shutdown_connections 2024-03-26T18:01:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:00.325+0000 7fc4f6aa0640 1 -- 172.21.15.105:0/271445544 wait complete. 2024-03-26T18:01:00.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:00:59 smithi144 ceph-mon[36095]: pgmap v1031: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:01:00.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:00:59 smithi105 ceph-mon[32280]: pgmap v1031: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:01:00.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:00:59 smithi135 ceph-mon[35636]: pgmap v1031: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:01:00.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 76480 7.07 MB/sec execute 463 sec latency 214.978 ms 2024-03-26T18:01:01.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 76609 7.07 MB/sec execute 464 sec latency 111.231 ms 2024-03-26T18:01:02.283 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:01 smithi105 ceph-mon[32280]: pgmap v1032: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-03-26T18:01:02.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:01 smithi144 ceph-mon[36095]: pgmap v1032: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-03-26T18:01:02.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:01 smithi135 ceph-mon[35636]: pgmap v1032: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-03-26T18:01:02.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 76704 7.07 MB/sec execute 465 sec latency 116.586 ms 2024-03-26T18:01:02.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:01:02.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:01:03.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- 172.21.15.105:0/1477101906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774100150 msgr2=0x7fe774102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 --2- 172.21.15.105:0/1477101906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774100150 0x7fe774102580 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe770009860 tx=0x7fe77002bb70 comp rx=0 tx=0).stop 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- 172.21.15.105:0/1477101906 shutdown_connections 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 --2- 172.21.15.105:0/1477101906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe774105470 0x7fe7741058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 --2- 172.21.15.105:0/1477101906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe774102ac0 0x7fe774104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 --2- 172.21.15.105:0/1477101906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774100150 0x7fe774102580 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- 172.21.15.105:0/1477101906 >> 172.21.15.105:0/1477101906 conn(0x7fe7740fbd70 msgr2=0x7fe7740fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- 172.21.15.105:0/1477101906 shutdown_connections 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- 172.21.15.105:0/1477101906 wait complete. 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 Processor -- start 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- start start 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe774100150 0x7fe77419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe774102ac0 0x7fe77419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774105470 0x7fe7741a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7741139f0 con 0x7fe774102ac0 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe774113870 con 0x7fe774100150 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.042+0000 7fe77c590640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe774113b70 con 0x7fe774105470 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe774102ac0 0x7fe77419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe774102ac0 0x7fe77419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53650/0 (socket says 172.21.15.105:53650) 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 -- 172.21.15.105:0/614973051 learned_addr learned my addr 172.21.15.105:0/614973051 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774105470 0x7fe7741a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe77a32c640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe774100150 0x7fe77419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 -- 172.21.15.105:0/614973051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774105470 msgr2=0x7fe7741a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774105470 0x7fe7741a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 -- 172.21.15.105:0/614973051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe774100150 msgr2=0x7fe77419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe774100150 0x7fe77419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 -- 172.21.15.105:0/614973051 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7741a5cd0 con 0x7fe774102ac0 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774105470 0x7fe7741a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe779b2b640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe774102ac0 0x7fe77419e640 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9c0 tx=0x7fe76400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe76b7fe640 1 -- 172.21.15.105:0/614973051 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76400cde0 con 0x7fe774102ac0 2024-03-26T18:01:03.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe77c590640 1 -- 172.21.15.105:0/614973051 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741a5fc0 con 0x7fe774102ac0 2024-03-26T18:01:03.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe77c590640 1 -- 172.21.15.105:0/614973051 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741a64d0 con 0x7fe774102ac0 2024-03-26T18:01:03.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe76b7fe640 1 -- 172.21.15.105:0/614973051 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7640035f0 con 0x7fe774102ac0 2024-03-26T18:01:03.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.043+0000 7fe76b7fe640 1 -- 172.21.15.105:0/614973051 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764010690 con 0x7fe774102ac0 2024-03-26T18:01:03.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.046+0000 7fe77f180740 1 Processor -- start 2024-03-26T18:01:03.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.046+0000 7fe77f180740 1 -- start start 2024-03-26T18:01:03.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.046+0000 7fe77f180740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831997fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77f180740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583199802e0 0x558319984de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77f180740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558319985450 0x5583199878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77f180740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5583198e4980 con 0x55831997f8f0 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77f180740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55831988cbe0 con 0x5583199802e0 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77f180740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583198e0930 con 0x558319985450 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831997fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831997fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53652/0 (socket says 172.21.15.105:53652) 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 -- 172.21.15.105:0/3527729501 learned_addr learned my addr 172.21.15.105:0/3527729501 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77a32c640 1 --2- 172.21.15.105:0/3527729501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583199802e0 0x558319984de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 -- 172.21.15.105:0/3527729501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558319985450 msgr2=0x5583199878e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/3527729501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558319985450 0x5583199878e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 -- 172.21.15.105:0/3527729501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583199802e0 msgr2=0x558319984de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/3527729501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583199802e0 0x558319984de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 -- 172.21.15.105:0/3527729501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583196c27d0 con 0x55831997f8f0 2024-03-26T18:01:03.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/3527729501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831997fd10 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0117e0 tx=0x7fe76c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0bb2bae1d7ba80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.047+0000 7fe768bfe640 1 -- 172.21.15.105:0/3527729501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c011310 con 0x55831997f8f0 2024-03-26T18:01:03.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.048+0000 7fe768bfe640 1 -- 172.21.15.105:0/3527729501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe76c003e20 con 0x55831997f8f0 2024-03-26T18:01:03.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.048+0000 7fe768bfe640 1 -- 172.21.15.105:0/3527729501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c004460 con 0x55831997f8f0 2024-03-26T18:01:03.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.048+0000 7fe76b7fe640 1 -- 172.21.15.105:0/614973051 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe764010830 con 0x7fe774102ac0 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.048+0000 7fe77f180740 1 -- 172.21.15.105:0/3527729501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 msgr2=0x55831997fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.048+0000 7fe77f180740 1 --2- 172.21.15.105:0/3527729501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831997fd10 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0117e0 tx=0x7fe76c011ca0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe76b7fe640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a3020 0x7fe7600a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77f180740 1 -- 172.21.15.105:0/3527729501 shutdown_connections 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77f180740 1 --2- 172.21.15.105:0/3527729501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558319985450 0x5583199878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77f180740 1 --2- 172.21.15.105:0/3527729501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583199802e0 0x558319984de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77f180740 1 --2- 172.21.15.105:0/3527729501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831997fd10 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77f180740 1 -- 172.21.15.105:0/3527729501 >> 172.21.15.105:0/3527729501 conn(0x55831997ae60 msgr2=0x55831997b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a3020 0x7fe7600a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77f180740 1 -- 172.21.15.105:0/3527729501 shutdown_connections 2024-03-26T18:01:03.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77f180740 1 -- 172.21.15.105:0/3527729501 wait complete. 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe76b7fe640 1 -- 172.21.15.105:0/614973051 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fe7640cf6e0 con 0x7fe774102ac0 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.049+0000 7fe77f180740 1 -- start start 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a3020 0x7fe7600a54e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe76c005ff0 tx=0x7fe76c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77f180740 1 Processor -- start 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77f180740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55831997f8f0 0x558319989b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77f180740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583199802e0 0x55831998a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77f180740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x5583199910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77f180740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558319988400 con 0x5583199802e0 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77f180740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5583198a0160 con 0x558319985450 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77f180740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558319988580 con 0x55831997f8f0 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77f180740 10 client.0 _ll_register_callbacks cb 0x5583198dc720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77ab2d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x5583199910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe779b2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583199802e0 0x55831998a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77ab2d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x5583199910b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:45972/0 (socket says 172.21.15.105:45972) 2024-03-26T18:01:03.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe779b2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583199802e0 0x55831998a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53662/0 (socket says 172.21.15.105:53662) 2024-03-26T18:01:03.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77ab2d640 1 -- 172.21.15.105:0/328060179 learned_addr learned my addr 172.21.15.105:0/328060179 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.050+0000 7fe77a32c640 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55831997f8f0 0x558319989b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe779b2b640 1 -- 172.21.15.105:0/328060179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55831997f8f0 msgr2=0x558319989b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe779b2b640 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55831997f8f0 0x558319989b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe779b2b640 1 -- 172.21.15.105:0/328060179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 msgr2=0x5583199910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe779b2b640 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x5583199910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe779b2b640 1 -- 172.21.15.105:0/328060179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583195f83f0 con 0x5583199802e0 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe779b2b640 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583199802e0 0x55831998a070 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fe76409ba70 tx=0x7fe7640037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8130d8b6570616ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe75dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe75dffb640 1 -- 172.21.15.105:0/328060179 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76409bd70 con 0x5583199802e0 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe75dffb640 1 -- 172.21.15.105:0/328060179 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fe76409b1c0 con 0x5583199802e0 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe77f180740 1 -- 172.21.15.105:0/328060179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe76c003e20 con 0x5583199802e0 2024-03-26T18:01:03.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe75dffb640 1 -- 172.21.15.105:0/328060179 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe76409fec0 con 0x5583199802e0 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.051+0000 7fe77f180740 10 client.23559 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe77f180740 1 -- 172.21.15.105:0/328060179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5583195b80f0 con 0x5583199802e0 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe75dffb640 1 -- 172.21.15.105:0/328060179 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7640b8030 con 0x5583199802e0 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe75dffb640 1 -- 172.21.15.105:0/328060179 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe764097ac0 con 0x5583199802e0 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe77f180740 10 client.23559 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe77f180740 10 client.23559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe77f180740 20 client.23559 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe77f180740 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558319995ab0 0x558319997ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe77f180740 4 client.23559 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe77f180740 1 -- 172.21.15.105:0/328060179 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583198f1ec0 con 0x558319995ab0 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.052+0000 7fe77a32c640 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558319995ab0 0x558319997ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.053+0000 7fe75dffb640 10 client.23559 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.053+0000 7fe77a32c640 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558319995ab0 0x558319997ea0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eba77d24d37bb85 server_cookie=41ca635f955be230 in_seq=0 out_seq=0 2024-03-26T18:01:03.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.054+0000 7fe75dffb640 1 -- 172.21.15.105:0/328060179 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5583198f1ec0 con 0x558319995ab0 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.054+0000 7fe75dffb640 10 client.23559 handle_command_reply: tid=0 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.054+0000 7fe77f180740 1 client.23559 shutdown 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.054+0000 7fe77f180740 1 -- 172.21.15.105:0/328060179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583199802e0 msgr2=0x55831998a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.054+0000 7fe77f180740 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583199802e0 0x55831998a070 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fe76409ba70 tx=0x7fe7640037a0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.054+0000 7fe77f180740 1 -- 172.21.15.105:0/328060179 shutdown_connections 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.054+0000 7fe77f180740 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x558319995ab0 0x558319997ea0 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.055+0000 7fe77f180740 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x5583199910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.055+0000 7fe77f180740 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583199802e0 0x55831998a070 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.055+0000 7fe77f180740 1 --2- 172.21.15.105:0/328060179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55831997f8f0 0x558319989b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.055+0000 7fe77f180740 1 -- 172.21.15.105:0/328060179 >> 172.21.15.105:0/328060179 conn(0x55831997ae60 msgr2=0x55831997a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.055+0000 7fe75dffb640 0 client.23559 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.055+0000 7fe77f180740 1 -- 172.21.15.105:0/328060179 shutdown_connections 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.055+0000 7fe77f180740 1 -- 172.21.15.105:0/328060179 wait complete. 2024-03-26T18:01:03.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.055+0000 7fe77f180740 20 client.23559 trim_cache size 0 max 16384 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.083+0000 7fe77f180740 1 Processor -- start 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.083+0000 7fe77f180740 1 -- start start 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77f180740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x558319998070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77f180740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 0x558319991070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77f180740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x558319988920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77f180740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7741af420 con 0x55831997f8f0 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77f180740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7741b0f60 con 0x558319985450 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77f180740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7741afaf0 con 0x5583199802e0 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x558319998070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x558319998070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53666/0 (socket says 172.21.15.105:53666) 2024-03-26T18:01:03.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 -- 172.21.15.105:0/1067406644 learned_addr learned my addr 172.21.15.105:0/1067406644 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x558319988920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe779b2b640 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 0x558319991070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 -- 172.21.15.105:0/1067406644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 msgr2=0x558319991070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 0x558319991070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 -- 172.21.15.105:0/1067406644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 msgr2=0x558319988920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x558319988920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 -- 172.21.15.105:0/1067406644 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558319981be0 con 0x55831997f8f0 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.084+0000 7fe77a32c640 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x558319998070 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe770000c00 tx=0x7fe7700349a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f563ba49f72b853c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.085+0000 7fe75ffff640 1 -- 172.21.15.105:0/1067406644 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77004f070 con 0x55831997f8f0 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.085+0000 7fe75ffff640 1 -- 172.21.15.105:0/1067406644 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe770004040 con 0x55831997f8f0 2024-03-26T18:01:03.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.085+0000 7fe75ffff640 1 -- 172.21.15.105:0/1067406644 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770039040 con 0x55831997f8f0 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 -- 172.21.15.105:0/1067406644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 msgr2=0x558319998070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x558319998070 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fe770000c00 tx=0x7fe7700349a0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 -- 172.21.15.105:0/1067406644 shutdown_connections 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x558319988920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 0x558319991070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 --2- 172.21.15.105:0/1067406644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x558319998070 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 -- 172.21.15.105:0/1067406644 >> 172.21.15.105:0/1067406644 conn(0x55831997ae60 msgr2=0x558319997960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 -- 172.21.15.105:0/1067406644 shutdown_connections 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 -- 172.21.15.105:0/1067406644 wait complete. 2024-03-26T18:01:03.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 -- start start 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.086+0000 7fe77f180740 1 Processor -- start 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77f180740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831998eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77f180740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 0x55831998f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77f180740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x558319990030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77f180740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7741b0890 con 0x55831997f8f0 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77f180740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7741b01c0 con 0x558319985450 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77f180740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7741ad400 con 0x5583199802e0 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77f180740 10 client.0 _ll_register_callbacks cb 0x7fe764003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831998eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831998eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53672/0 (socket says 172.21.15.105:53672) 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 -- 172.21.15.105:0/3903445362 learned_addr learned my addr 172.21.15.105:0/3903445362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe779b2b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 0x55831998f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77ab2d640 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x558319990030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 -- 172.21.15.105:0/3903445362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 msgr2=0x55831998f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 0x55831998f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 -- 172.21.15.105:0/3903445362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 msgr2=0x558319990030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x558319990030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 -- 172.21.15.105:0/3903445362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558319990620 con 0x55831997f8f0 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe77a32c640 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831998eae0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe77002e010 tx=0x7fe7700041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82935f0b287694b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.087+0000 7fe75dffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe75dffb640 1 -- 172.21.15.105:0/3903445362 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770058060 con 0x55831997f8f0 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe75dffb640 1 -- 172.21.15.105:0/3903445362 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fe770046040 con 0x55831997f8f0 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe77f180740 1 -- 172.21.15.105:0/3903445362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe770004040 con 0x55831997f8f0 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe75dffb640 1 -- 172.21.15.105:0/3903445362 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe77004f030 con 0x55831997f8f0 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe77f180740 10 client.23565 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe77f180740 1 -- 172.21.15.105:0/3903445362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5583199fbfc0 con 0x55831997f8f0 2024-03-26T18:01:03.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe75dffb640 1 -- 172.21.15.105:0/3903445362 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe770004040 con 0x55831997f8f0 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe75dffb640 1 -- 172.21.15.105:0/3903445362 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe7700035f0 con 0x55831997f8f0 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe77f180740 10 client.23565 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe77f180740 10 client.23565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.088+0000 7fe77f180740 20 client.23565 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.089+0000 7fe77f180740 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583199fc940 0x5583199fed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.089+0000 7fe77f180740 4 client.23565 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.089+0000 7fe77f180740 1 -- 172.21.15.105:0/3903445362 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583198ec210 con 0x5583199fc940 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.089+0000 7fe779b2b640 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583199fc940 0x5583199fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.089+0000 7fe75dffb640 10 client.23565 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:03.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.089+0000 7fe779b2b640 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583199fc940 0x5583199fed60 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8698b11b0afe817 server_cookie=2487bc9a04bffb49 in_seq=0 out_seq=0 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe75dffb640 1 -- 172.21.15.105:0/3903445362 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5583198ec210 con 0x5583199fc940 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe75dffb640 10 client.23565 handle_command_reply: tid=0 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe77f180740 1 client.23565 shutdown 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe77f180740 1 -- 172.21.15.105:0/3903445362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 msgr2=0x55831998eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe77f180740 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831998eae0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fe77002e010 tx=0x7fe7700041f0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe77f180740 1 -- 172.21.15.105:0/3903445362 shutdown_connections 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe77f180740 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5583199fc940 0x5583199fed60 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe75dffb640 0 client.23565 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.091+0000 7fe77f180740 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558319985450 0x558319990030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.092+0000 7fe77f180740 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583199802e0 0x55831998f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.092+0000 7fe77f180740 1 --2- 172.21.15.105:0/3903445362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55831997f8f0 0x55831998eae0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.092+0000 7fe77f180740 1 -- 172.21.15.105:0/3903445362 >> 172.21.15.105:0/3903445362 conn(0x55831997ae60 msgr2=0x55831997e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.092+0000 7fe77f180740 1 -- 172.21.15.105:0/3903445362 shutdown_connections 2024-03-26T18:01:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.092+0000 7fe77f180740 1 -- 172.21.15.105:0/3903445362 wait complete. 2024-03-26T18:01:03.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.092+0000 7fe77f180740 20 client.23565 trim_cache size 0 max 16384 2024-03-26T18:01:03.078 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:01:03.078 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T18:01:03.078 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T18:01:03.078 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:01:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 -- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a3020 msgr2=0x7fe7600a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a3020 0x7fe7600a54e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe76c005ff0 tx=0x7fe76c000f40 comp rx=0 tx=0).stop 2024-03-26T18:01:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 -- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe774102ac0 msgr2=0x7fe77419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe774102ac0 0x7fe77419e640 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e9c0 tx=0x7fe76400ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 -- 172.21.15.105:0/614973051 shutdown_connections 2024-03-26T18:01:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7600a3020 0x7fe7600a54e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe774105470 0x7fe7741a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe774102ac0 0x7fe77419e640 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 --2- 172.21.15.105:0/614973051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe774100150 0x7fe77419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 -- 172.21.15.105:0/614973051 >> 172.21.15.105:0/614973051 conn(0x7fe7740fbd70 msgr2=0x7fe7741035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 -- 172.21.15.105:0/614973051 shutdown_connections 2024-03-26T18:01:03.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.093+0000 7fe77c590640 1 -- 172.21.15.105:0/614973051 wait complete. 2024-03-26T18:01:03.088 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T18:01:03.089 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:01:03.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 -- 172.21.15.105:0/755010880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0104cf0 msgr2=0x7fe4b0105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/755010880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0104cf0 0x7fe4b0105170 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009860 tx=0x7fe4a002bb70 comp rx=0 tx=0).stop 2024-03-26T18:01:03.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 -- 172.21.15.105:0/755010880 shutdown_connections 2024-03-26T18:01:03.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/755010880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b01056b0 0x7fe4b010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/755010880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0104cf0 0x7fe4b0105170 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/755010880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4b0103b00 0x7fe4b0103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 -- 172.21.15.105:0/755010880 >> 172.21.15.105:0/755010880 conn(0x7fe4b00ff070 msgr2=0x7fe4b01014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 -- 172.21.15.105:0/755010880 shutdown_connections 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.243+0000 7fe4b79e6640 1 -- 172.21.15.105:0/755010880 wait complete. 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.244+0000 7fe4b79e6640 1 Processor -- start 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.244+0000 7fe4b79e6640 1 -- start start 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.244+0000 7fe4b79e6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0103b00 0x7fe4b0199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.244+0000 7fe4b79e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b0104cf0 0x7fe4b019a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.244+0000 7fe4b79e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4b01056b0 0x7fe4b01a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.244+0000 7fe4b79e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b010f630 con 0x7fe4b0104cf0 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.244+0000 7fe4b79e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b010f4b0 con 0x7fe4b01056b0 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.244+0000 7fe4b79e6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b010f7b0 con 0x7fe4b0103b00 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5f83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4b01056b0 0x7fe4b01a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0103b00 0x7fe4b0199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0103b00 0x7fe4b0199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55524/0 (socket says 172.21.15.105:55524) 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 -- 172.21.15.105:0/4099708934 learned_addr learned my addr 172.21.15.105:0/4099708934 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b4f81640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b0104cf0 0x7fe4b019a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 -- 172.21.15.105:0/4099708934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4b01056b0 msgr2=0x7fe4b01a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4b01056b0 0x7fe4b01a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 -- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b0104cf0 msgr2=0x7fe4b019a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b0104cf0 0x7fe4b019a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 -- 172.21.15.105:0/4099708934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b01a1980 con 0x7fe4b0103b00 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.245+0000 7fe4b5782640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0103b00 0x7fe4b0199d60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e9c0 tx=0x7fe4ac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.246+0000 7fe4a67fc640 1 -- 172.21.15.105:0/4099708934 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac00cd80 con 0x7fe4b0103b00 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.246+0000 7fe4a67fc640 1 -- 172.21.15.105:0/4099708934 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4ac0035f0 con 0x7fe4b0103b00 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.246+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b01a1c70 con 0x7fe4b0103b00 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.246+0000 7fe4a67fc640 1 -- 172.21.15.105:0/4099708934 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac010690 con 0x7fe4b0103b00 2024-03-26T18:01:03.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.246+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b01a2190 con 0x7fe4b0103b00 2024-03-26T18:01:03.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.247+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b010aef0 con 0x7fe4b0103b00 2024-03-26T18:01:03.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.248+0000 7fe4a67fc640 1 -- 172.21.15.105:0/4099708934 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe4ac0037a0 con 0x7fe4b0103b00 2024-03-26T18:01:03.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.249+0000 7fe4a67fc640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe49c0a3020 0x7fe49c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.249+0000 7fe4b4f81640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe49c0a3020 0x7fe49c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.249+0000 7fe4a67fc640 1 -- 172.21.15.105:0/4099708934 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fe4ac0cf930 con 0x7fe4b0103b00 2024-03-26T18:01:03.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.250+0000 7fe4b4f81640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe49c0a3020 0x7fe49c0a54e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0000c00 tx=0x7fe4a00349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.254+0000 7fe4a67fc640 1 -- 172.21.15.105:0/4099708934 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe4ac09c6c0 con 0x7fe4b0103b00 2024-03-26T18:01:03.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 76806 7.07 MB/sec execute 466 sec latency 107.126 ms 2024-03-26T18:01:03.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.584+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4b00624c0 con 0x7fe4b0103b00 2024-03-26T18:01:03.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.585+0000 7fe4a67fc640 1 -- 172.21.15.105:0/4099708934 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11938 (secure 0 0 0) 0x7fe4ac09bfc0 con 0x7fe4b0103b00 2024-03-26T18:01:03.567 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:01:03.567 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:00:20.611284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_23346":{"gid":23346,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/966537438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":966537438},{"type":"v1","addr":"172.21.15.135:6835","nonce":966537438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T18:01:03.568 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 43 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe49c0a3020 msgr2=0x7fe49c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe49c0a3020 0x7fe49c0a54e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0000c00 tx=0x7fe4a00349a0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0103b00 msgr2=0x7fe4b0199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0103b00 0x7fe4b0199d60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e9c0 tx=0x7fe4ac00ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 shutdown_connections 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe49c0a3020 0x7fe49c0a54e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4b01056b0 0x7fe4b01a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4b0104cf0 0x7fe4b019a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 --2- 172.21.15.105:0/4099708934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4b0103b00 0x7fe4b0199d60 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 >> 172.21.15.105:0/4099708934 conn(0x7fe4b00ff070 msgr2=0x7fe4b01080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 shutdown_connections 2024-03-26T18:01:03.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.588+0000 7fe4b79e6640 1 -- 172.21.15.105:0/4099708934 wait complete. 2024-03-26T18:01:03.585 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T18:01:03.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 -- 172.21.15.105:0/1535250061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9301007c0 msgr2=0x7fd930100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 --2- 172.21.15.105:0/1535250061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9301007c0 0x7fd930100be0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fd924009860 tx=0x7fd92402bb70 comp rx=0 tx=0).stop 2024-03-26T18:01:03.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 -- 172.21.15.105:0/1535250061 shutdown_connections 2024-03-26T18:01:03.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 --2- 172.21.15.105:0/1535250061 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd930102370 0x7fd930108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 --2- 172.21.15.105:0/1535250061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9301019b0 0x7fd930101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 --2- 172.21.15.105:0/1535250061 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9301007c0 0x7fd930100be0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 -- 172.21.15.105:0/1535250061 >> 172.21.15.105:0/1535250061 conn(0x7fd9300fbd50 msgr2=0x7fd9300fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 -- 172.21.15.105:0/1535250061 shutdown_connections 2024-03-26T18:01:03.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.743+0000 7fd9371df640 1 -- 172.21.15.105:0/1535250061 wait complete. 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd9371df640 1 Processor -- start 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd9371df640 1 -- start start 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd9371df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9301007c0 0x7fd930069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd9371df640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301019b0 0x7fd93019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd9371df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd930102370 0x7fd9301a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd9371df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010c2e0 con 0x7fd930102370 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd9371df640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010c160 con 0x7fd9301019b0 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd9371df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd93010c460 con 0x7fd9301007c0 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd93577c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd930102370 0x7fd9301a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd93577c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd930102370 0x7fd9301a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53720/0 (socket says 172.21.15.105:53720) 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd93577c640 1 -- 172.21.15.105:0/1556997490 learned_addr learned my addr 172.21.15.105:0/1556997490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd934f7b640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9301007c0 0x7fd930069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.744+0000 7fd92ffff640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301019b0 0x7fd93019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd93577c640 1 -- 172.21.15.105:0/1556997490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9301007c0 msgr2=0x7fd930069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd93577c640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9301007c0 0x7fd930069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd93577c640 1 -- 172.21.15.105:0/1556997490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301019b0 msgr2=0x7fd93019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd93577c640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301019b0 0x7fd93019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd93577c640 1 -- 172.21.15.105:0/1556997490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9301a1a10 con 0x7fd930102370 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd93577c640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd930102370 0x7fd9301a1220 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fd92800eb10 tx=0x7fd92800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd92dffb640 1 -- 172.21.15.105:0/1556997490 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92800ce20 con 0x7fd930102370 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd92dffb640 1 -- 172.21.15.105:0/1556997490 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd928005020 con 0x7fd930102370 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd92dffb640 1 -- 172.21.15.105:0/1556997490 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928010690 con 0x7fd930102370 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd9371df640 1 -- 172.21.15.105:0/1556997490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9301a1ca0 con 0x7fd930102370 2024-03-26T18:01:03.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.745+0000 7fd9371df640 1 -- 172.21.15.105:0/1556997490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9301a21a0 con 0x7fd930102370 2024-03-26T18:01:03.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.748+0000 7fd92dffb640 1 -- 172.21.15.105:0/1556997490 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd9280051d0 con 0x7fd930102370 2024-03-26T18:01:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd92dffb640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9200a2f50 0x7fd9200a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd939dc7740 1 Processor -- start 2024-03-26T18:01:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd92dffb640 1 -- 172.21.15.105:0/1556997490 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fd9280cf960 con 0x7fd930102370 2024-03-26T18:01:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd934f7b640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9200a2f50 0x7fd9200a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd939dc7740 1 -- start start 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8ae0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8ae5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd939dc7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b7e8a45970 con 0x55b7e8ae6590 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd939dc7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7e89edd70 con 0x55b7e8ae0a30 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.749+0000 7fd939dc7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7e8a41920 con 0x55b7e8ae1420 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd934f7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8ae0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd93577c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8ae5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8ae5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55546/0 (socket says 172.21.15.105:55546) 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 -- 172.21.15.105:0/3705367760 learned_addr learned my addr 172.21.15.105:0/3705367760 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd934f7b640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9200a2f50 0x7fd9200a5410 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd924009830 tx=0x7fd924034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 -- 172.21.15.105:0/3705367760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 msgr2=0x55b7e8ae0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 --2- 172.21.15.105:0/3705367760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8ae0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 -- 172.21.15.105:0/3705367760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 msgr2=0x55b7e8ae8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 --2- 172.21.15.105:0/3705367760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 -- 172.21.15.105:0/3705367760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7e87593f0 con 0x55b7e8ae1420 2024-03-26T18:01:03.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd93577c640 1 --2- 172.21.15.105:0/3705367760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:03.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.750+0000 7fd92ffff640 1 --2- 172.21.15.105:0/3705367760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8ae5f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009ca0 tx=0x7fd91c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f13e43c0998543 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.751+0000 7fd91affd640 1 -- 172.21.15.105:0/3705367760 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c004230 con 0x55b7e8ae1420 2024-03-26T18:01:03.734 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.751+0000 7fd91affd640 1 -- 172.21.15.105:0/3705367760 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c0043d0 con 0x55b7e8ae1420 2024-03-26T18:01:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.751+0000 7fd91affd640 1 -- 172.21.15.105:0/3705367760 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c004e30 con 0x55b7e8ae1420 2024-03-26T18:01:03.738 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 -- 172.21.15.105:0/3705367760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 msgr2=0x55b7e8ae5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3705367760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8ae5f20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009ca0 tx=0x7fd91c011810 comp rx=0 tx=0).stop 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 -- 172.21.15.105:0/3705367760 shutdown_connections 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3705367760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3705367760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8ae5f20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3705367760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8ae0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 -- 172.21.15.105:0/3705367760 >> 172.21.15.105:0/3705367760 conn(0x55b7e8adbfa0 msgr2=0x55b7e8adc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 -- 172.21.15.105:0/3705367760 shutdown_connections 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 -- 172.21.15.105:0/3705367760 wait complete. 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 -- start start 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.752+0000 7fd939dc7740 1 Processor -- start 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8af24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd939dc7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b7e8ae95e0 con 0x55b7e8ae6590 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd939dc7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7e8a01310 con 0x55b7e8ae0a30 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd939dc7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b7e8ae9760 con 0x55b7e8ae1420 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd934f7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd934f7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46038/0 (socket says 172.21.15.105:46038) 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd934f7b640 1 -- 172.21.15.105:0/3646665212 learned_addr learned my addr 172.21.15.105:0/3646665212 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.753+0000 7fd93577c640 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8af24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd939dc7740 10 client.0 _ll_register_callbacks cb 0x55b7e8a3cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd934f7b640 1 -- 172.21.15.105:0/3646665212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 msgr2=0x55b7e8aeb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd934f7b640 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd934f7b640 1 -- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 msgr2=0x55b7e8af24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd934f7b640 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8af24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd934f7b640 1 -- 172.21.15.105:0/3646665212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7e88237d0 con 0x55b7e8ae0a30 2024-03-26T18:01:03.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd93577c640 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8af24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd934f7b640 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeaf40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd924000c00 tx=0x7fd924005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a018490ab4bb03 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd90ffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.754+0000 7fd90ffff640 1 -- 172.21.15.105:0/3646665212 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92404f030 con 0x55b7e8ae0a30 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.755+0000 7fd939dc7740 1 -- 172.21.15.105:0/3646665212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91c0043d0 con 0x55b7e8ae0a30 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.755+0000 7fd90ffff640 1 -- 172.21.15.105:0/3646665212 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fd92402c8a0 con 0x55b7e8ae0a30 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.755+0000 7fd90ffff640 1 -- 172.21.15.105:0/3646665212 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd924044040 con 0x55b7e8ae0a30 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.755+0000 7fd939dc7740 10 client.33257 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.755+0000 7fd939dc7740 1 -- 172.21.15.105:0/3646665212 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b7e8af58d0 con 0x55b7e8ae0a30 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.755+0000 7fd90ffff640 1 -- 172.21.15.105:0/3646665212 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c0043d0 con 0x55b7e8ae0a30 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.755+0000 7fd90ffff640 1 -- 172.21.15.105:0/3646665212 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd92404a040 con 0x55b7e8ae0a30 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.756+0000 7fd939dc7740 10 client.33257 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.756+0000 7fd939dc7740 10 client.33257 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.756+0000 7fd939dc7740 20 client.33257 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:03.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.756+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b7e8af73a0 0x55b7e8af9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.756+0000 7fd939dc7740 4 client.33257 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:03.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.756+0000 7fd939dc7740 1 -- 172.21.15.105:0/3646665212 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7e8a52eb0 con 0x55b7e8af73a0 2024-03-26T18:01:03.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.756+0000 7fd92ffff640 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b7e8af73a0 0x55b7e8af9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.756+0000 7fd90ffff640 10 client.33257 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:01:03.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.757+0000 7fd92ffff640 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b7e8af73a0 0x55b7e8af9840 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d8b5034cc506ca09 server_cookie=4597e62ed8b3a2ec in_seq=0 out_seq=0 2024-03-26T18:01:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd90ffff640 1 -- 172.21.15.105:0/3646665212 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b7e8a52eb0 con 0x55b7e8af73a0 2024-03-26T18:01:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd90ffff640 10 client.33257 handle_command_reply: tid=0 2024-03-26T18:01:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 client.33257 shutdown 2024-03-26T18:01:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 -- 172.21.15.105:0/3646665212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 msgr2=0x55b7e8aeaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeaf40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd924000c00 tx=0x7fd924005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 -- 172.21.15.105:0/3646665212 shutdown_connections 2024-03-26T18:01:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b7e8af73a0 0x55b7e8af9840 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8af24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 --2- 172.21.15.105:0/3646665212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeaf40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd939dc7740 1 -- 172.21.15.105:0/3646665212 >> 172.21.15.105:0/3646665212 conn(0x55b7e8adbfa0 msgr2=0x55b7e8adb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.758+0000 7fd90ffff640 0 client.33257 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:01:03.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.759+0000 7fd939dc7740 1 -- 172.21.15.105:0/3646665212 shutdown_connections 2024-03-26T18:01:03.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.759+0000 7fd939dc7740 1 -- 172.21.15.105:0/3646665212 wait complete. 2024-03-26T18:01:03.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.759+0000 7fd939dc7740 20 client.33257 trim_cache size 0 max 16384 2024-03-26T18:01:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.779+0000 7fd939dc7740 1 Processor -- start 2024-03-26T18:01:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.779+0000 7fd939dc7740 1 -- start start 2024-03-26T18:01:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.779+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.779+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae1420 0x55b7e8af2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.779+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.779+0000 7fd939dc7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301aaf60 con 0x55b7e8ae1420 2024-03-26T18:01:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.779+0000 7fd939dc7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301acaa0 con 0x55b7e8ae6590 2024-03-26T18:01:03.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.779+0000 7fd939dc7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301ab630 con 0x55b7e8ae0a30 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae1420 0x55b7e8af2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae1420 0x55b7e8af2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53738/0 (socket says 172.21.15.105:53738) 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd93577c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 -- 172.21.15.105:0/298544485 learned_addr learned my addr 172.21.15.105:0/298544485 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd934f7b640 1 --2- 172.21.15.105:0/298544485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 -- 172.21.15.105:0/298544485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae0a30 msgr2=0x55b7e8aeb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 --2- 172.21.15.105:0/298544485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 -- 172.21.15.105:0/298544485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae6590 msgr2=0x55b7e8ae1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 --2- 172.21.15.105:0/298544485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 -- 172.21.15.105:0/298544485 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7e8adbb40 con 0x55b7e8ae1420 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd92ffff640 1 --2- 172.21.15.105:0/298544485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae1420 0x55b7e8af2480 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009c70 tx=0x7fd91c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9d9fa9d845ddd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd91a7fc640 1 -- 172.21.15.105:0/298544485 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c01e040 con 0x55b7e8ae1420 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd91a7fc640 1 -- 172.21.15.105:0/298544485 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c01f040 con 0x55b7e8ae1420 2024-03-26T18:01:03.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.780+0000 7fd91a7fc640 1 -- 172.21.15.105:0/298544485 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0043a0 con 0x55b7e8ae1420 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 -- 172.21.15.105:0/298544485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae1420 msgr2=0x55b7e8af2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 --2- 172.21.15.105:0/298544485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae1420 0x55b7e8af2480 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009c70 tx=0x7fd91c011810 comp rx=0 tx=0).stop 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 -- 172.21.15.105:0/298544485 shutdown_connections 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 --2- 172.21.15.105:0/298544485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae6590 0x55b7e8ae1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 --2- 172.21.15.105:0/298544485 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae1420 0x55b7e8af2480 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 --2- 172.21.15.105:0/298544485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae0a30 0x55b7e8aeb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 -- 172.21.15.105:0/298544485 >> 172.21.15.105:0/298544485 conn(0x55b7e8adbfa0 msgr2=0x55b7e8ad7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 -- 172.21.15.105:0/298544485 shutdown_connections 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.781+0000 7fd939dc7740 1 -- 172.21.15.105:0/298544485 wait complete. 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 1 -- start start 2024-03-26T18:01:03.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 1 Processor -- start 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8af2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8aef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301ac3d0 con 0x55b7e8ae6590 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301abd00 con 0x55b7e8ae0a30 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301a8f40 con 0x55b7e8ae1420 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd934f7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8af2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd93577c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8aef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55578/0 (socket says 172.21.15.105:55578) 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd93577c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8aef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53746/0 (socket says 172.21.15.105:53746) 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd934f7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8af2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46056/0 (socket says 172.21.15.105:46056) 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd92ffff640 1 -- 172.21.15.105:0/2556399803 learned_addr learned my addr 172.21.15.105:0/2556399803 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd939dc7740 10 client.0 _ll_register_callbacks cb 0x7fd91c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd92ffff640 1 -- 172.21.15.105:0/2556399803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 msgr2=0x55b7e8af2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd92ffff640 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8af2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd92ffff640 1 -- 172.21.15.105:0/2556399803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 msgr2=0x55b7e8aef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd92ffff640 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8aef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.782+0000 7fd92ffff640 1 -- 172.21.15.105:0/2556399803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7e8ae62e0 con 0x55b7e8ae1420 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd934f7b640 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8af2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd93577c640 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8aef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd92ffff640 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeef40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009ed0 tx=0x7fd91c009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fba0978dea2175c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd90ffff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd90ffff640 1 -- 172.21.15.105:0/2556399803 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c034030 con 0x55b7e8ae1420 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd939dc7740 1 -- 172.21.15.105:0/2556399803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91c01f040 con 0x55b7e8ae1420 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd90ffff640 1 -- 172.21.15.105:0/2556399803 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fd91c0041e0 con 0x55b7e8ae1420 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd90ffff640 1 -- 172.21.15.105:0/2556399803 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd91c040050 con 0x55b7e8ae1420 2024-03-26T18:01:03.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd939dc7740 10 client.33250 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.783+0000 7fd939dc7740 1 -- 172.21.15.105:0/2556399803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b7e8b5c7e0 con 0x55b7e8ae1420 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd90ffff640 1 -- 172.21.15.105:0/2556399803 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c01f040 con 0x55b7e8ae1420 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd90ffff640 1 -- 172.21.15.105:0/2556399803 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd91c024040 con 0x55b7e8ae1420 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd939dc7740 10 client.33250 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd939dc7740 10 client.33250 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd939dc7740 20 client.33250 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd939dc7740 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b7e8b5cef0 0x55b7e8b5f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd939dc7740 4 client.33250 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd939dc7740 1 -- 172.21.15.105:0/2556399803 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7e8a4d200 con 0x55b7e8b5cef0 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.784+0000 7fd934f7b640 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b7e8b5cef0 0x55b7e8b5f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd934f7b640 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b7e8b5cef0 0x55b7e8b5f310 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25dab312dadae4f9 server_cookie=fdabfeded7279fcd in_seq=0 out_seq=0 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd90ffff640 10 client.33250 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd90ffff640 1 -- 172.21.15.105:0/2556399803 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7e8a4d200 con 0x55b7e8b5cef0 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd90ffff640 10 client.33250 handle_command_reply: tid=0 2024-03-26T18:01:03.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 client.33250 shutdown 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 -- 172.21.15.105:0/2556399803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 msgr2=0x55b7e8aeef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeef40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009ed0 tx=0x7fd91c009f00 comp rx=0 tx=0).stop 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 -- 172.21.15.105:0/2556399803 shutdown_connections 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b7e8b5cef0 0x55b7e8b5f310 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b7e8ae6590 0x55b7e8aef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7e8ae1420 0x55b7e8aeef40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 --2- 172.21.15.105:0/2556399803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7e8ae0a30 0x55b7e8af2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd90ffff640 0 client.33250 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 -- 172.21.15.105:0/2556399803 >> 172.21.15.105:0/2556399803 conn(0x55b7e8adbfa0 msgr2=0x55b7e8adc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 -- 172.21.15.105:0/2556399803 shutdown_connections 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 1 -- 172.21.15.105:0/2556399803 wait complete. 2024-03-26T18:01:03.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.785+0000 7fd939dc7740 20 client.33250 trim_cache size 0 max 16384 2024-03-26T18:01:03.773 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:03.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 -- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9200a2f50 msgr2=0x7fd9200a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9200a2f50 0x7fd9200a5410 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd924009830 tx=0x7fd924034980 comp rx=0 tx=0).stop 2024-03-26T18:01:03.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 -- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd930102370 msgr2=0x7fd9301a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd930102370 0x7fd9301a1220 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fd92800eb10 tx=0x7fd92800ef10 comp rx=0 tx=0).stop 2024-03-26T18:01:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 -- 172.21.15.105:0/1556997490 shutdown_connections 2024-03-26T18:01:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd9200a2f50 0x7fd9200a5410 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd930102370 0x7fd9301a1220 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9301019b0 0x7fd93019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 --2- 172.21.15.105:0/1556997490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9301007c0 0x7fd930069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 -- 172.21.15.105:0/1556997490 >> 172.21.15.105:0/1556997490 conn(0x7fd9300fbd50 msgr2=0x7fd930104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 -- 172.21.15.105:0/1556997490 shutdown_connections 2024-03-26T18:01:03.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.786+0000 7fd9371df640 1 -- 172.21.15.105:0/1556997490 wait complete. 2024-03-26T18:01:03.779 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T18:01:03.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 -- 172.21.15.105:0/2171383383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224100d20 msgr2=0x7f5224101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 --2- 172.21.15.105:0/2171383383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224100d20 0x7f5224101140 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5214009900 tx=0x7f521402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:03.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 -- 172.21.15.105:0/2171383383 shutdown_connections 2024-03-26T18:01:03.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 --2- 172.21.15.105:0/2171383383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f52241023f0 0x7f5224108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 --2- 172.21.15.105:0/2171383383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5224101a30 0x7f5224101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 --2- 172.21.15.105:0/2171383383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224100d20 0x7f5224101140 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 -- 172.21.15.105:0/2171383383 >> 172.21.15.105:0/2171383383 conn(0x7f52240fb4f0 msgr2=0x7f52240fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 -- 172.21.15.105:0/2171383383 shutdown_connections 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 -- 172.21.15.105:0/2171383383 wait complete. 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 Processor -- start 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.943+0000 7f522af77640 1 -- start start 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f522af77640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5224100d20 0x7f52241003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f522af77640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224101a30 0x7f522419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f522af77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52241023f0 0x7f522419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f522af77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f522410c390 con 0x7f52241023f0 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f522af77640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f522410c210 con 0x7f5224100d20 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f522af77640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f522410c510 con 0x7f5224101a30 2024-03-26T18:01:03.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224101a30 0x7f522419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5228d13640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5224100d20 0x7f52241003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224101a30 0x7f522419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55604/0 (socket says 172.21.15.105:55604) 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 -- 172.21.15.105:0/449780989 learned_addr learned my addr 172.21.15.105:0/449780989 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5228d13640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5224100d20 0x7f52241003e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46066/0 (socket says 172.21.15.105:46066) 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5229514640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52241023f0 0x7f522419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 -- 172.21.15.105:0/449780989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5224100d20 msgr2=0x7f52241003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5224100d20 0x7f52241003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 -- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52241023f0 msgr2=0x7f522419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52241023f0 0x7f522419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 -- 172.21.15.105:0/449780989 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522419f7e0 con 0x7f5224101a30 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5228d13640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5224100d20 0x7f52241003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:03.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.944+0000 7f5223fff640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224101a30 0x7f522419c8b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f521800e9f0 tx=0x7f521800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.945+0000 7f5221ffb640 1 -- 172.21.15.105:0/449780989 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521800cd80 con 0x7f5224101a30 2024-03-26T18:01:03.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.946+0000 7f5221ffb640 1 -- 172.21.15.105:0/449780989 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52180108e0 con 0x7f5224101a30 2024-03-26T18:01:03.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.946+0000 7f5221ffb640 1 -- 172.21.15.105:0/449780989 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218018920 con 0x7f5224101a30 2024-03-26T18:01:03.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.946+0000 7f522af77640 1 -- 172.21.15.105:0/449780989 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522419fa70 con 0x7f5224101a30 2024-03-26T18:01:03.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.946+0000 7f522af77640 1 -- 172.21.15.105:0/449780989 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522419ffc0 con 0x7f5224101a30 2024-03-26T18:01:03.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.948+0000 7f5221ffb640 1 -- 172.21.15.105:0/449780989 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5218010a90 con 0x7f5224101a30 2024-03-26T18:01:03.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.949+0000 7f5221ffb640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f520c0a2f20 0x7f520c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.949+0000 7f5221ffb640 1 -- 172.21.15.105:0/449780989 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f52180cf700 con 0x7f5224101a30 2024-03-26T18:01:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.949+0000 7f522db65740 1 Processor -- start 2024-03-26T18:01:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.949+0000 7f522db65740 1 -- start start 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.949+0000 7f5228d13640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f520c0a2f20 0x7f520c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f522db65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x558394241dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f522db65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x558394244e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f522db65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x558394252370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f522db65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5583941a6970 con 0x5583942419b0 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f522db65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55839414ed70 con 0x558394245500 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f522db65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5583941a2920 con 0x5583942423a0 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5223fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x558394244e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5223fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x558394244e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55620/0 (socket says 172.21.15.105:55620) 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5223fff640 1 -- 172.21.15.105:0/2035097116 learned_addr learned my addr 172.21.15.105:0/2035097116 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5229514640 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x558394252370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5228d13640 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x558394241dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5223fff640 1 -- 172.21.15.105:0/2035097116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 msgr2=0x558394252370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5223fff640 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x558394252370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5223fff640 1 -- 172.21.15.105:0/2035097116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 msgr2=0x558394241dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5223fff640 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x558394241dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5223fff640 1 -- 172.21.15.105:0/2035097116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558393eba3f0 con 0x5583942423a0 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.950+0000 7f5228d13640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f520c0a2f20 0x7f520c0a53e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f521402c240 tx=0x7f5214005e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.951+0000 7f5223fff640 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x558394244e90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f521809be30 tx=0x7f5218010de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c7efd66387cc41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.951+0000 7f520affd640 1 -- 172.21.15.105:0/2035097116 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52180a4800 con 0x5583942423a0 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.951+0000 7f520affd640 1 -- 172.21.15.105:0/2035097116 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f521809a1a0 con 0x5583942423a0 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.951+0000 7f520affd640 1 -- 172.21.15.105:0/2035097116 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521809b4f0 con 0x5583942423a0 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 -- 172.21.15.105:0/2035097116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 msgr2=0x558394244e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x558394244e90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f521809be30 tx=0x7f5218010de0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 -- 172.21.15.105:0/2035097116 shutdown_connections 2024-03-26T18:01:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x558394252370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x558394244e90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 --2- 172.21.15.105:0/2035097116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x558394241dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 -- 172.21.15.105:0/2035097116 >> 172.21.15.105:0/2035097116 conn(0x55839423cfa0 msgr2=0x55839423d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 -- 172.21.15.105:0/2035097116 shutdown_connections 2024-03-26T18:01:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 -- 172.21.15.105:0/2035097116 wait complete. 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 -- start start 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.952+0000 7f522db65740 1 Processor -- start 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f522db65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942419b0 0x55839424be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f522db65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942423a0 0x55839424c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f522db65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558394245500 0x5583942513e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f522db65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558394245f10 con 0x558394245500 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f522db65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558394162310 con 0x5583942423a0 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f522db65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558394246090 con 0x5583942419b0 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5228d13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942419b0 0x55839424be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5228d13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942419b0 0x55839424be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55630/0 (socket says 172.21.15.105:55630) 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5228d13640 1 -- 172.21.15.105:0/3697331978 learned_addr learned my addr 172.21.15.105:0/3697331978 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f522db65740 10 client.0 _ll_register_callbacks cb 0x55839419dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5223fff640 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942423a0 0x55839424c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5229514640 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558394245500 0x5583942513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5228d13640 1 -- 172.21.15.105:0/3697331978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942423a0 msgr2=0x55839424c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5228d13640 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942423a0 0x55839424c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5228d13640 1 -- 172.21.15.105:0/3697331978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558394245500 msgr2=0x5583942513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5228d13640 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558394245500 0x5583942513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5228d13640 1 -- 172.21.15.105:0/3697331978 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558393f847d0 con 0x5583942419b0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.953+0000 7f5229514640 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558394245500 0x5583942513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f5228d13640 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942419b0 0x55839424be70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5214005ab0 tx=0x7f521402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=218e8c1824e56ca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f5203fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f5203fff640 1 -- 172.21.15.105:0/3697331978 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214053060 con 0x5583942419b0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f5203fff640 1 -- 172.21.15.105:0/3697331978 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f5214004bd0 con 0x5583942419b0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f522db65740 1 -- 172.21.15.105:0/3697331978 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f521809a1a0 con 0x5583942419b0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f5203fff640 1 -- 172.21.15.105:0/3697331978 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5214004dd0 con 0x5583942419b0 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f522db65740 10 client.33262 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:03.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f522db65740 1 -- 172.21.15.105:0/3697331978 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5583942678d0 con 0x5583942419b0 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.954+0000 7f5203fff640 1 -- 172.21.15.105:0/3697331978 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f521809a1a0 con 0x5583942419b0 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.955+0000 7f5203fff640 1 -- 172.21.15.105:0/3697331978 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f52140594b0 con 0x5583942419b0 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.955+0000 7f522db65740 10 client.33262 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.955+0000 7f522db65740 10 client.33262 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.955+0000 7f522db65740 20 client.33262 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.955+0000 7f522db65740 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558394268fa0 0x55839426b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.955+0000 7f522db65740 4 client.33262 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.955+0000 7f522db65740 1 -- 172.21.15.105:0/3697331978 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583941b3eb0 con 0x558394268fa0 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.955+0000 7f5223fff640 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558394268fa0 0x55839426b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.956+0000 7f5203fff640 10 client.33262 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.956+0000 7f5223fff640 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558394268fa0 0x55839426b440 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f6bb37621f78f78c server_cookie=b945570b09989569 in_seq=0 out_seq=0 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.956+0000 7f5203fff640 1 -- 172.21.15.105:0/3697331978 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5583941b3eb0 con 0x558394268fa0 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.956+0000 7f5203fff640 10 client.33262 handle_command_reply: tid=0 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.956+0000 7f522db65740 1 client.33262 shutdown 2024-03-26T18:01:03.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 -- 172.21.15.105:0/3697331978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942419b0 msgr2=0x55839424be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942419b0 0x55839424be70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5214005ab0 tx=0x7f521402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 -- 172.21.15.105:0/3697331978 shutdown_connections 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x558394268fa0 0x55839426b440 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558394245500 0x5583942513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942423a0 0x55839424c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f5203fff640 0 client.33262 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 --2- 172.21.15.105:0/3697331978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942419b0 0x55839424be70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 -- 172.21.15.105:0/3697331978 >> 172.21.15.105:0/3697331978 conn(0x55839423cfa0 msgr2=0x55839423c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 -- 172.21.15.105:0/3697331978 shutdown_connections 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 1 -- 172.21.15.105:0/3697331978 wait complete. 2024-03-26T18:01:03.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.957+0000 7f522db65740 20 client.33262 trim_cache size 0 max 16384 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f522db65740 1 Processor -- start 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f522db65740 1 -- start start 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f522db65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x55839424c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f522db65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x55839424abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f522db65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x55839423b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f522db65740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52241a6ab0 con 0x5583942419b0 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f522db65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52241a85f0 con 0x558394245500 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f522db65740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52241a7180 con 0x5583942423a0 2024-03-26T18:01:03.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f5228d13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x55839424c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f5228d13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x55839424c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53774/0 (socket says 172.21.15.105:53774) 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.979+0000 7f5228d13640 1 -- 172.21.15.105:0/2597278940 learned_addr learned my addr 172.21.15.105:0/2597278940 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5229514640 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x55839423b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5223fff640 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x55839424abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5228d13640 1 -- 172.21.15.105:0/2597278940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 msgr2=0x55839424abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5228d13640 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x55839424abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5228d13640 1 -- 172.21.15.105:0/2597278940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 msgr2=0x55839423b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5228d13640 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x55839423b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5228d13640 1 -- 172.21.15.105:0/2597278940 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558394242780 con 0x5583942419b0 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5229514640 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x55839423b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f5228d13640 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x55839424c310 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f5214004650 tx=0x7f5214005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0720d821de4d3e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f520a7fc640 1 -- 172.21.15.105:0/2597278940 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214053050 con 0x5583942419b0 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f520a7fc640 1 -- 172.21.15.105:0/2597278940 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52140051a0 con 0x5583942419b0 2024-03-26T18:01:03.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.980+0000 7f520a7fc640 1 -- 172.21.15.105:0/2597278940 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52140347e0 con 0x5583942419b0 2024-03-26T18:01:03.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.981+0000 7f522db65740 1 -- 172.21.15.105:0/2597278940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 msgr2=0x55839424c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.981+0000 7f522db65740 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x55839424c310 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f5214004650 tx=0x7f5214005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.981+0000 7f522db65740 1 -- 172.21.15.105:0/2597278940 shutdown_connections 2024-03-26T18:01:03.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.981+0000 7f522db65740 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558394245500 0x55839423b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.981+0000 7f522db65740 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5583942423a0 0x55839424abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.981+0000 7f522db65740 1 --2- 172.21.15.105:0/2597278940 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942419b0 0x55839424c310 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.981+0000 7f522db65740 1 -- 172.21.15.105:0/2597278940 >> 172.21.15.105:0/2597278940 conn(0x55839423cfa0 msgr2=0x55839424a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.981+0000 7f522db65740 1 -- 172.21.15.105:0/2597278940 shutdown_connections 2024-03-26T18:01:03.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.982+0000 7f522db65740 1 -- 172.21.15.105:0/2597278940 wait complete. 2024-03-26T18:01:03.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.983+0000 7f522db65740 1 -- start start 2024-03-26T18:01:03.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.983+0000 7f522db65740 1 Processor -- start 2024-03-26T18:01:03.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.983+0000 7f522db65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942419b0 0x55839427a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.983+0000 7f522db65740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942423a0 0x55839427a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.983+0000 7f522db65740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558394245500 0x55839427b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942419b0 0x55839427a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942419b0 0x55839427a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46096/0 (socket says 172.21.15.105:46096) 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 -- 172.21.15.105:0/1419698557 learned_addr learned my addr 172.21.15.105:0/1419698557 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52241a7f20 con 0x5583942423a0 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f52241a7850 con 0x5583942419b0 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52241a4a90 con 0x558394245500 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5229514640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558394245500 0x55839427b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f522db65740 10 client.0 _ll_register_callbacks cb 0x7f5214009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5223fff640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942423a0 0x55839427a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 -- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558394245500 msgr2=0x55839427b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558394245500 0x55839427b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 -- 172.21.15.105:0/1419698557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942423a0 msgr2=0x55839427a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942423a0 0x55839427a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 -- 172.21.15.105:0/1419698557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55839427bd20 con 0x5583942419b0 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5229514640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558394245500 0x55839427b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5223fff640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942423a0 0x55839427a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5228d13640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942419b0 0x55839427a2a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5214059f30 tx=0x7f5214005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6da40cc8a4d304 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.984+0000 7f5203fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.985+0000 7f5203fff640 1 -- 172.21.15.105:0/1419698557 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214053030 con 0x5583942419b0 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.985+0000 7f5203fff640 1 -- 172.21.15.105:0/1419698557 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f5214034930 con 0x5583942419b0 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.985+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583941ae200 con 0x5583942419b0 2024-03-26T18:01:03.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.985+0000 7f5203fff640 1 -- 172.21.15.105:0/1419698557 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5214034b30 con 0x5583942419b0 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.985+0000 7f522db65740 10 client.33281 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.985+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5583942cf2c0 con 0x5583942419b0 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.985+0000 7f5203fff640 1 -- 172.21.15.105:0/1419698557 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5583941ae200 con 0x5583942419b0 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.986+0000 7f5203fff640 1 -- 172.21.15.105:0/1419698557 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5214069030 con 0x5583942419b0 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.986+0000 7f522db65740 10 client.33281 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.986+0000 7f522db65740 10 client.33281 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.986+0000 7f522db65740 20 client.33281 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.986+0000 7f522db65740 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5583942cfbb0 0x5583942d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.986+0000 7f522db65740 4 client.33281 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.986+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55839412b6a0 con 0x5583942cfbb0 2024-03-26T18:01:03.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.986+0000 7f5223fff640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5583942cfbb0 0x5583942d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:03.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f5203fff640 10 client.33281 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:01:03.971 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f5223fff640 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5583942cfbb0 0x5583942d1fd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ec8639cf0e78a34 server_cookie=c9fbbc4eb423f9dd in_seq=0 out_seq=0 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f5203fff640 1 -- 172.21.15.105:0/1419698557 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55839412b6a0 con 0x5583942cfbb0 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f5203fff640 10 client.33281 handle_command_reply: tid=0 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 client.33281 shutdown 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942419b0 msgr2=0x55839427a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942419b0 0x55839427a2a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5214059f30 tx=0x7f5214005850 comp rx=0 tx=0).stop 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 shutdown_connections 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x5583942cfbb0 0x5583942d1fd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558394245500 0x55839427b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5583942423a0 0x55839427a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 --2- 172.21.15.105:0/1419698557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5583942419b0 0x55839427a2a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 >> 172.21.15.105:0/1419698557 conn(0x55839423cfa0 msgr2=0x55839424c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f5203fff640 0 client.33281 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 shutdown_connections 2024-03-26T18:01:03.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.987+0000 7f522db65740 1 -- 172.21.15.105:0/1419698557 wait complete. 2024-03-26T18:01:03.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522db65740 20 client.33281 trim_cache size 0 max 16384 2024-03-26T18:01:03.973 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:03.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 -- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f520c0a2f20 msgr2=0x7f520c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f520c0a2f20 0x7f520c0a53e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f521402c240 tx=0x7f5214005e60 comp rx=0 tx=0).stop 2024-03-26T18:01:03.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 -- 172.21.15.105:0/449780989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224101a30 msgr2=0x7f522419c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224101a30 0x7f522419c8b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f521800e9f0 tx=0x7f521800eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 -- 172.21.15.105:0/449780989 shutdown_connections 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f520c0a2f20 0x7f520c0a53e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52241023f0 0x7f522419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224101a30 0x7f522419c8b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 --2- 172.21.15.105:0/449780989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5224100d20 0x7f52241003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 -- 172.21.15.105:0/449780989 >> 172.21.15.105:0/449780989 conn(0x7f52240fb4f0 msgr2=0x7f5224104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.988+0000 7f522af77640 1 -- 172.21.15.105:0/449780989 shutdown_connections 2024-03-26T18:01:03.974 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:03.989+0000 7f522af77640 1 -- 172.21.15.105:0/449780989 wait complete. 2024-03-26T18:01:03.984 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 -- 172.21.15.105:0/2545241995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ae41019f0 msgr2=0x7f2ae4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/2545241995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ae41019f0 0x7f2ae4101e70 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009900 tx=0x7f2ad802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 -- 172.21.15.105:0/2545241995 shutdown_connections 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/2545241995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae41023b0 0x7f2ae4108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/2545241995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ae41019f0 0x7f2ae4101e70 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/2545241995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae4100800 0x7f2ae4100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 -- 172.21.15.105:0/2545241995 >> 172.21.15.105:0/2545241995 conn(0x7f2ae40fbd70 msgr2=0x7f2ae40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 -- 172.21.15.105:0/2545241995 shutdown_connections 2024-03-26T18:01:04.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.146+0000 7f2ae92dc640 1 -- 172.21.15.105:0/2545241995 wait complete. 2024-03-26T18:01:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae92dc640 1 Processor -- start 2024-03-26T18:01:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae92dc640 1 -- start start 2024-03-26T18:01:04.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae92dc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae4100800 0x7f2ae410b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae92dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae41019f0 0x7f2ae41097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae92dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ae41023b0 0x7f2ae4109d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae92dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410d450 con 0x7f2ae41023b0 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae92dc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410d2d0 con 0x7f2ae41019f0 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae92dc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410d5d0 con 0x7f2ae4100800 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae27fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae41019f0 0x7f2ae41097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae27fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae41019f0 0x7f2ae41097c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46110/0 (socket says 172.21.15.105:46110) 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae27fc640 1 -- 172.21.15.105:0/349335941 learned_addr learned my addr 172.21.15.105:0/349335941 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae4100800 0x7f2ae410b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.147+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ae41023b0 0x7f2ae4109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae27fc640 1 -- 172.21.15.105:0/349335941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae4100800 msgr2=0x7f2ae410b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae4100800 0x7f2ae410b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae27fc640 1 -- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ae41023b0 msgr2=0x7f2ae4109d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ae41023b0 0x7f2ae4109d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae27fc640 1 -- 172.21.15.105:0/349335941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae410a240 con 0x7f2ae41019f0 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae4100800 0x7f2ae410b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae41019f0 0x7f2ae41097c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8006010 tx=0x7f2ad8005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2acbfff640 1 -- 172.21.15.105:0/349335941 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8043020 con 0x7f2ae41019f0 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae92dc640 1 -- 172.21.15.105:0/349335941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae410a4d0 con 0x7f2ae41019f0 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2acbfff640 1 -- 172.21.15.105:0/349335941 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad8004350 con 0x7f2ae41019f0 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2acbfff640 1 -- 172.21.15.105:0/349335941 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8003a80 con 0x7f2ae41019f0 2024-03-26T18:01:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.148+0000 7f2ae92dc640 1 -- 172.21.15.105:0/349335941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41a6600 con 0x7f2ae41019f0 2024-03-26T18:01:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.151+0000 7f2acbfff640 1 -- 172.21.15.105:0/349335941 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2ad8046020 con 0x7f2ae41019f0 2024-03-26T18:01:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.151+0000 7f2aebf9e740 1 Processor -- start 2024-03-26T18:01:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.151+0000 7f2aebf9e740 1 -- start start 2024-03-26T18:01:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b65e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b66420 0x559539b6af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b6da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559539aca970 con 0x559539b65a30 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559539a72d70 con 0x559539b6b590 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559539ac6920 con 0x559539b66420 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b66420 0x559539b6af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b65e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b66420 0x559539b6af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55662/0 (socket says 172.21.15.105:55662) 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b65e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53800/0 (socket says 172.21.15.105:53800) 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae2ffd640 1 -- 172.21.15.105:0/648002494 learned_addr learned my addr 172.21.15.105:0/648002494 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/648002494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b6da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2acbfff640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2acc0a2f20 0x7f2acc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae37fe640 1 -- 172.21.15.105:0/648002494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b66420 msgr2=0x559539b6af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/648002494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b66420 0x559539b6af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae37fe640 1 -- 172.21.15.105:0/648002494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 msgr2=0x559539b6da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/648002494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b6da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2ae37fe640 1 -- 172.21.15.105:0/648002494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595397de3f0 con 0x559539b65a30 2024-03-26T18:01:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.152+0000 7f2acbfff640 1 -- 172.21.15.105:0/349335941 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2ad80f1620 con 0x7f2ae41019f0 2024-03-26T18:01:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.153+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/648002494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b65e50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009f00 tx=0x7f2adc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdcf364c66e8050 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.153+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2acc0a2f20 0x7f2acc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.153+0000 7f2ac97fa640 1 -- 172.21.15.105:0/648002494 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0045c0 con 0x559539b65a30 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.153+0000 7f2ac97fa640 1 -- 172.21.15.105:0/648002494 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2adc004760 con 0x559539b65a30 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.153+0000 7f2ac97fa640 1 -- 172.21.15.105:0/648002494 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc01c400 con 0x559539b65a30 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.153+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2acc0a2f20 0x7f2acc0a53e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2adc020000 tx=0x7f2adc021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 -- 172.21.15.105:0/648002494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 msgr2=0x559539b65e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/648002494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b65e50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009f00 tx=0x7f2adc011810 comp rx=0 tx=0).stop 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 -- 172.21.15.105:0/648002494 shutdown_connections 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/648002494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b6da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/648002494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b66420 0x559539b6af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/648002494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b65e50 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 -- 172.21.15.105:0/648002494 >> 172.21.15.105:0/648002494 conn(0x559539b60fa0 msgr2=0x559539b613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 -- 172.21.15.105:0/648002494 shutdown_connections 2024-03-26T18:01:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.154+0000 7f2aebf9e740 1 -- 172.21.15.105:0/648002494 wait complete. 2024-03-26T18:01:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.155+0000 7f2aebf9e740 1 -- start start 2024-03-26T18:01:04.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.155+0000 7f2aebf9e740 1 Processor -- start 2024-03-26T18:01:04.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.155+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b65a30 0x559539b70060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.155+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.155+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b77570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.155+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559539b6e540 con 0x559539b66420 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.155+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559539a86310 con 0x559539b6b590 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.155+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559539b6e6c0 con 0x559539b65a30 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b77570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b65a30 0x559539b70060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b77570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46130/0 (socket says 172.21.15.105:46130) 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 -- 172.21.15.105:0/1467978120 learned_addr learned my addr 172.21.15.105:0/1467978120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b65a30 0x559539b70060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55678/0 (socket says 172.21.15.105:55678) 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2aebf9e740 10 client.0 _ll_register_callbacks cb 0x559539ac1c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 -- 172.21.15.105:0/1467978120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b65a30 msgr2=0x559539b70060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b65a30 0x559539b70060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 -- 172.21.15.105:0/1467978120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 msgr2=0x559539b705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 -- 172.21.15.105:0/1467978120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595398a87d0 con 0x559539b6b590 2024-03-26T18:01:04.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b65a30 0x559539b70060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b77570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2adc003fc0 tx=0x7f2adc00f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4966c10acc7d7e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.156+0000 7f2ac6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/1467978120 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0045c0 con 0x559539b6b590 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/1467978120 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2adc02e070 con 0x559539b6b590 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1467978120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2adc004760 con 0x559539b6b590 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/1467978120 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2adc01c400 con 0x559539b6b590 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2aebf9e740 10 client.33293 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1467978120 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559539b7a8c0 con 0x559539b6b590 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/1467978120 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2adc004c10 con 0x559539b6b590 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/1467978120 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2adc039030 con 0x559539b6b590 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.157+0000 7f2aebf9e740 10 client.33293 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.158+0000 7f2aebf9e740 10 client.33293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.158+0000 7f2aebf9e740 20 client.33293 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:04.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.158+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559539b7be40 0x559539b7e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.158+0000 7f2aebf9e740 4 client.33293 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.158+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1467978120 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2adc004760 con 0x559539b7be40 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.158+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559539b7be40 0x559539b7e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.159+0000 7f2ac6ff5640 10 client.33293 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.159+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559539b7be40 0x559539b7e280 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9c0784e6f36620a server_cookie=dc27d7e086ffbab5 in_seq=0 out_seq=0 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.159+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/1467978120 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f2adc004760 con 0x559539b7be40 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.159+0000 7f2ac6ff5640 10 client.33293 handle_command_reply: tid=0 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.159+0000 7f2aebf9e740 1 client.33293 shutdown 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1467978120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 msgr2=0x559539b77570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b77570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2adc003fc0 tx=0x7f2adc00f9d0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1467978120 shutdown_connections 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559539b7be40 0x559539b7e280 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b6b590 0x559539b77570 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2ac6ff5640 0 client.33293 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:01:04.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1467978120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b65a30 0x559539b70060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1467978120 >> 172.21.15.105:0/1467978120 conn(0x559539b60fa0 msgr2=0x559539b601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1467978120 shutdown_connections 2024-03-26T18:01:04.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1467978120 wait complete. 2024-03-26T18:01:04.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.160+0000 7f2aebf9e740 20 client.33293 trim_cache size 0 max 16384 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.178+0000 7f2aebf9e740 1 Processor -- start 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2aebf9e740 1 -- start start 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b67a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b66420 0x559539b67f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b77530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41af480 con 0x559539b65a30 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41b0fc0 con 0x559539b66420 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41afb50 con 0x559539b6b590 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b67a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b67a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53822/0 (socket says 172.21.15.105:53822) 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae2ffd640 1 -- 172.21.15.105:0/1630766102 learned_addr learned my addr 172.21.15.105:0/1630766102 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b77530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b66420 0x559539b67f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae2ffd640 1 -- 172.21.15.105:0/1630766102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 msgr2=0x559539b77530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b77530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae2ffd640 1 -- 172.21.15.105:0/1630766102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b66420 msgr2=0x559539b67f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b66420 0x559539b67f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae2ffd640 1 -- 172.21.15.105:0/1630766102 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559539b61d10 con 0x559539b65a30 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.179+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b77530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.180+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b67a30 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x559539b66820 tx=0x7f2ad4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6877dfc10a282431 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.180+0000 7f2ac8ff9640 1 -- 172.21.15.105:0/1630766102 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4005020 con 0x559539b65a30 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.180+0000 7f2ac8ff9640 1 -- 172.21.15.105:0/1630766102 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad4024030 con 0x559539b65a30 2024-03-26T18:01:04.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.180+0000 7f2ac8ff9640 1 -- 172.21.15.105:0/1630766102 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40047a0 con 0x559539b65a30 2024-03-26T18:01:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1630766102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 msgr2=0x559539b67a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b67a30 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x559539b66820 tx=0x7f2ad4005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1630766102 shutdown_connections 2024-03-26T18:01:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b77530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b66420 0x559539b67f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/1630766102 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b65a30 0x559539b67a30 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1630766102 >> 172.21.15.105:0/1630766102 conn(0x559539b60fa0 msgr2=0x559539b70060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1630766102 shutdown_connections 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 -- 172.21.15.105:0/1630766102 wait complete. 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 -- start start 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.181+0000 7f2aebf9e740 1 Processor -- start 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b65a30 0x559539b77590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b74030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2aebf9e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b74660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41b08f0 con 0x559539b66420 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41b0220 con 0x559539b65a30 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2aebf9e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41ad460 con 0x559539b6b590 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2aebf9e740 10 client.0 _ll_register_callbacks cb 0x7f2adc01f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:04.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b74030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b74030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53836/0 (socket says 172.21.15.105:53836) 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae27fc640 1 -- 172.21.15.105:0/3515884213 learned_addr learned my addr 172.21.15.105:0/3515884213 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b65a30 0x559539b77590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae37fe640 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b74660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae27fc640 1 -- 172.21.15.105:0/3515884213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 msgr2=0x559539b74660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b74660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae27fc640 1 -- 172.21.15.105:0/3515884213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b65a30 msgr2=0x559539b77590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b65a30 0x559539b77590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.182+0000 7f2ae27fc640 1 -- 172.21.15.105:0/3515884213 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559539be01f0 con 0x559539b66420 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2ac6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2ae27fc640 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b74030 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80bda10 tx=0x7f2ad80bdae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30784b5011506a24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/3515884213 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80be170 con 0x559539b66420 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/3515884213 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f2ad8005420 con 0x559539b66420 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2aebf9e740 1 -- 172.21.15.105:0/3515884213 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ad4024030 con 0x559539b66420 2024-03-26T18:01:04.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/3515884213 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2ad80be410 con 0x559539b66420 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2aebf9e740 10 client.23625 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2aebf9e740 1 -- 172.21.15.105:0/3515884213 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559539be15c0 con 0x559539b66420 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.183+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/3515884213 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ad80c6720 con 0x559539b66420 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/3515884213 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2ad80c6320 con 0x559539b66420 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2aebf9e740 10 client.23625 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2aebf9e740 10 client.23625 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2aebf9e740 20 client.23625 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559539b7be40 0x559539be4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2aebf9e740 4 client.23625 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2aebf9e740 1 -- 172.21.15.105:0/3515884213 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2ad4024030 con 0x559539b7be40 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559539b7be40 0x559539be4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2ac6ff5640 10 client.23625 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.184+0000 7f2ae2ffd640 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559539b7be40 0x559539be4030 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ba456fbb6058ddb server_cookie=5c8de9e75fbbce7 in_seq=0 out_seq=0 2024-03-26T18:01:04.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2ac6ff5640 1 -- 172.21.15.105:0/3515884213 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2ad4024030 con 0x559539b7be40 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2ac6ff5640 10 client.23625 handle_command_reply: tid=0 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 client.23625 shutdown 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 -- 172.21.15.105:0/3515884213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 msgr2=0x559539b74030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b74030 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80bda10 tx=0x7f2ad80bdae0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 -- 172.21.15.105:0/3515884213 shutdown_connections 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559539b6b590 0x559539b74660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559539b66420 0x559539b74030 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x559539b7be40 0x559539be4030 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 --2- 172.21.15.105:0/3515884213 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559539b65a30 0x559539b77590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2ac6ff5640 0 client.23625 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 -- 172.21.15.105:0/3515884213 >> 172.21.15.105:0/3515884213 conn(0x559539b60fa0 msgr2=0x559539b64200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 -- 172.21.15.105:0/3515884213 shutdown_connections 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 1 -- 172.21.15.105:0/3515884213 wait complete. 2024-03-26T18:01:04.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.185+0000 7f2aebf9e740 20 client.23625 trim_cache size 0 max 16384 2024-03-26T18:01:04.172 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:04.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 -- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2acc0a2f20 msgr2=0x7f2acc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2acc0a2f20 0x7f2acc0a53e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2adc020000 tx=0x7f2adc021000 comp rx=0 tx=0).stop 2024-03-26T18:01:04.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 -- 172.21.15.105:0/349335941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae41019f0 msgr2=0x7f2ae41097c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae41019f0 0x7f2ae41097c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8006010 tx=0x7f2ad8005dd0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 -- 172.21.15.105:0/349335941 shutdown_connections 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2acc0a2f20 0x7f2acc0a53e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ae41023b0 0x7f2ae4109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2ae41019f0 0x7f2ae41097c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 --2- 172.21.15.105:0/349335941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ae4100800 0x7f2ae410b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 -- 172.21.15.105:0/349335941 >> 172.21.15.105:0/349335941 conn(0x7f2ae40fbd70 msgr2=0x7f2ae4104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 -- 172.21.15.105:0/349335941 shutdown_connections 2024-03-26T18:01:04.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.186+0000 7f2ae92dc640 1 -- 172.21.15.105:0/349335941 wait complete. 2024-03-26T18:01:04.178 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:03 smithi105 ceph-mon[32280]: pgmap v1033: 97 pgs: 97 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-26T18:01:04.178 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:03 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4099708934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:04.179 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T18:01:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 -- 172.21.15.105:0/2717466527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 msgr2=0x7f1e40101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 --2- 172.21.15.105:0/2717466527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 0x7f1e40101e30 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009900 tx=0x7f1e2802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 -- 172.21.15.105:0/2717466527 shutdown_connections 2024-03-26T18:01:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 --2- 172.21.15.105:0/2717466527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e40102370 0x7f1e40108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 --2- 172.21.15.105:0/2717466527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 0x7f1e40101e30 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 --2- 172.21.15.105:0/2717466527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e401007c0 0x7f1e40100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 -- 172.21.15.105:0/2717466527 >> 172.21.15.105:0/2717466527 conn(0x7f1e400fbd50 msgr2=0x7f1e400fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 -- 172.21.15.105:0/2717466527 shutdown_connections 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 -- 172.21.15.105:0/2717466527 wait complete. 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.344+0000 7f1e455df640 1 Processor -- start 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e455df640 1 -- start start 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e455df640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e401007c0 0x7f1e40069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e455df640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 0x7f1e4019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e455df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e40102370 0x7f1e4019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e455df640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010c2d0 con 0x7f1e401019b0 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e455df640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010c150 con 0x7f1e401007c0 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e455df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4010c450 con 0x7f1e40102370 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e3effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e401007c0 0x7f1e40069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e367fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 0x7f1e4019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e3effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e401007c0 0x7f1e40069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:46184/0 (socket says 172.21.15.105:46184) 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e3effd640 1 -- 172.21.15.105:0/808300859 learned_addr learned my addr 172.21.15.105:0/808300859 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e367fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 0x7f1e4019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53854/0 (socket says 172.21.15.105:53854) 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e3f7fe640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e40102370 0x7f1e4019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e367fc640 1 -- 172.21.15.105:0/808300859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e40102370 msgr2=0x7f1e4019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e367fc640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e40102370 0x7f1e4019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e367fc640 1 -- 172.21.15.105:0/808300859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e401007c0 msgr2=0x7f1e40069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e367fc640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e401007c0 0x7f1e40069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.345+0000 7f1e367fc640 1 -- 172.21.15.105:0/808300859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4019f7c0 con 0x7f1e401019b0 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.346+0000 7f1e367fc640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 0x7f1e4019c890 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000c00 tx=0x7f1e28005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.346+0000 7f1e3cff9640 1 -- 172.21.15.105:0/808300859 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28043020 con 0x7f1e401019b0 2024-03-26T18:01:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.346+0000 7f1e3cff9640 1 -- 172.21.15.105:0/808300859 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e28004350 con 0x7f1e401019b0 2024-03-26T18:01:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.346+0000 7f1e455df640 1 -- 172.21.15.105:0/808300859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4019f9f0 con 0x7f1e401019b0 2024-03-26T18:01:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.346+0000 7f1e3cff9640 1 -- 172.21.15.105:0/808300859 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28003b50 con 0x7f1e401019b0 2024-03-26T18:01:04.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.346+0000 7f1e455df640 1 -- 172.21.15.105:0/808300859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4019ff90 con 0x7f1e401019b0 2024-03-26T18:01:04.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.349+0000 7f1e3cff9640 1 -- 172.21.15.105:0/808300859 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1e28046020 con 0x7f1e401019b0 2024-03-26T18:01:04.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e481c8740 1 Processor -- start 2024-03-26T18:01:04.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e481c8740 1 -- start start 2024-03-26T18:01:04.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff831e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff836f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff839a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e481c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b9ff796970 con 0x55b9ff831a30 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e481c8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b9ff73ed70 con 0x55b9ff837590 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e481c8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b9ff792920 con 0x55b9ff832420 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e3f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff831e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e3f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff831e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53860/0 (socket says 172.21.15.105:53860) 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e3f7fe640 1 -- 172.21.15.105:0/3835304936 learned_addr learned my addr 172.21.15.105:0/3835304936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff836f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.350+0000 7f1e367fc640 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff839a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3cff9640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1e2c0a3020 0x7f1e2c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3effd640 1 -- 172.21.15.105:0/3835304936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 msgr2=0x55b9ff839a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff839a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3effd640 1 -- 172.21.15.105:0/3835304936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 msgr2=0x55b9ff831e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff831e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3effd640 1 -- 172.21.15.105:0/3835304936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ff4aa3f0 con 0x55b9ff832420 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e367fc640 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff839a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3f7fe640 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff831e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3f7fe640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1e2c0a3020 0x7f1e2c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3cff9640 1 -- 172.21.15.105:0/808300859 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f1e280f1480 con 0x7f1e401019b0 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff836f20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1e300058c0 tx=0x7f1e30011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d18ed2a8a91885a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e357fa640 1 -- 172.21.15.105:0/3835304936 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30004220 con 0x55b9ff832420 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e357fa640 1 -- 172.21.15.105:0/3835304936 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e300043c0 con 0x55b9ff832420 2024-03-26T18:01:04.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.351+0000 7f1e357fa640 1 -- 172.21.15.105:0/3835304936 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30004ce0 con 0x55b9ff832420 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 -- 172.21.15.105:0/3835304936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 msgr2=0x55b9ff836f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e3f7fe640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1e2c0a3020 0x7f1e2c0a54e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x55b9ff832c80 tx=0x7f1e38004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff836f20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1e300058c0 tx=0x7f1e30011810 comp rx=0 tx=0).stop 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 -- 172.21.15.105:0/3835304936 shutdown_connections 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff839a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff836f20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3835304936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff831e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 -- 172.21.15.105:0/3835304936 >> 172.21.15.105:0/3835304936 conn(0x55b9ff82cfa0 msgr2=0x55b9ff82d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 -- 172.21.15.105:0/3835304936 shutdown_connections 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.352+0000 7f1e481c8740 1 -- 172.21.15.105:0/3835304936 wait complete. 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e481c8740 1 -- start start 2024-03-26T18:01:04.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e481c8740 1 Processor -- start 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff840350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff840890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e481c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b9ff83e950 con 0x55b9ff832420 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e3effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff840350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e481c8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b9ff752310 con 0x55b9ff837590 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e3effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff840350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55732/0 (socket says 172.21.15.105:55732) 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e481c8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b9ff83ead0 con 0x55b9ff831a30 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e3effd640 1 -- 172.21.15.105:0/3898149742 learned_addr learned my addr 172.21.15.105:0/3898149742 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.353+0000 7f1e3f7fe640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e481c8740 10 client.0 _ll_register_callbacks cb 0x55b9ff78dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e3effd640 1 -- 172.21.15.105:0/3898149742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 msgr2=0x55b9ff8478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e3effd640 1 -- 172.21.15.105:0/3898149742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 msgr2=0x55b9ff840890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e367fc640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff840890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff840890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e3effd640 1 -- 172.21.15.105:0/3898149742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9ff5747d0 con 0x55b9ff831a30 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e3f7fe640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e367fc640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff840890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e481c8740 1 -- 172.21.15.105:0/3898149742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e300043c0 con 0x55b9ff831a30 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff840350 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1e30000c00 tx=0x7f1e30005120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c56211a24003c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e267fc640 10 client.33298 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e267fc640 1 -- 172.21.15.105:0/3898149742 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30032050 con 0x55b9ff831a30 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.354+0000 7f1e267fc640 1 -- 172.21.15.105:0/3898149742 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f1e30005290 con 0x55b9ff831a30 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e267fc640 1 -- 172.21.15.105:0/3898149742 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1e30027030 con 0x55b9ff831a30 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e481c8740 10 client.33298 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e481c8740 1 -- 172.21.15.105:0/3898149742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55b9ff84ab90 con 0x55b9ff831a30 2024-03-26T18:01:04.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e267fc640 1 -- 172.21.15.105:0/3898149742 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1e30035020 con 0x55b9ff831a30 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e481c8740 10 client.33298 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e481c8740 10 client.33298 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e481c8740 20 client.33298 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b9ff84bfa0 0x55b9ff84e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e481c8740 4 client.33298 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e481c8740 1 -- 172.21.15.105:0/3898149742 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9ff7a3eb0 con 0x55b9ff84bfa0 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.355+0000 7f1e367fc640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b9ff84bfa0 0x55b9ff84e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.356+0000 7f1e267fc640 10 client.33298 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:01:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.356+0000 7f1e367fc640 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b9ff84bfa0 0x55b9ff84e430 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95847e5384f3a38c server_cookie=58738e25b325ea50 in_seq=0 out_seq=0 2024-03-26T18:01:04.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:03 smithi144 ceph-mon[36095]: pgmap v1033: 97 pgs: 97 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-26T18:01:04.342 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:03 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4099708934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e267fc640 1 -- 172.21.15.105:0/3898149742 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b9ff7a3eb0 con 0x55b9ff84bfa0 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e267fc640 10 client.33298 handle_command_reply: tid=0 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 client.33298 shutdown 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 -- 172.21.15.105:0/3898149742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 msgr2=0x55b9ff840350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff840350 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1e30000c00 tx=0x7f1e30005120 comp rx=0 tx=0).stop 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 -- 172.21.15.105:0/3898149742 shutdown_connections 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b9ff84bfa0 0x55b9ff84e430 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e267fc640 0 client.33298 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff840890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3898149742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff840350 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 -- 172.21.15.105:0/3898149742 >> 172.21.15.105:0/3898149742 conn(0x55b9ff82cfa0 msgr2=0x55b9ff82c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 -- 172.21.15.105:0/3898149742 shutdown_connections 2024-03-26T18:01:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 1 -- 172.21.15.105:0/3898149742 wait complete. 2024-03-26T18:01:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.358+0000 7f1e481c8740 20 client.33298 trim_cache size 0 max 16384 2024-03-26T18:01:04.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.378+0000 7f1e481c8740 1 Processor -- start 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.378+0000 7f1e481c8740 1 -- start start 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff830910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff847890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff852cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e481c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401a6b40 con 0x55b9ff831a30 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e481c8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401a8680 con 0x55b9ff837590 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e481c8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401a7210 con 0x55b9ff832420 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff830910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff830910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53874/0 (socket says 172.21.15.105:53874) 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3effd640 1 -- 172.21.15.105:0/3176468467 learned_addr learned my addr 172.21.15.105:0/3176468467 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e367fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff847890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3f7fe640 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff852cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3effd640 1 -- 172.21.15.105:0/3176468467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 msgr2=0x55b9ff847890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff847890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3effd640 1 -- 172.21.15.105:0/3176468467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 msgr2=0x55b9ff852cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff852cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e3effd640 1 -- 172.21.15.105:0/3176468467 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9ff82c8f0 con 0x55b9ff831a30 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.379+0000 7f1e367fc640 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff847890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.380+0000 7f1e3effd640 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff830910 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f1e30005f60 tx=0x7f1e3001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4ff02082fd6e21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.380+0000 7f1e34ff9640 1 -- 172.21.15.105:0/3176468467 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e30032030 con 0x55b9ff831a30 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.380+0000 7f1e34ff9640 1 -- 172.21.15.105:0/3176468467 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e300043c0 con 0x55b9ff831a30 2024-03-26T18:01:04.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.380+0000 7f1e34ff9640 1 -- 172.21.15.105:0/3176468467 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3001d040 con 0x55b9ff831a30 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 -- 172.21.15.105:0/3176468467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 msgr2=0x55b9ff830910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff830910 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f1e30005f60 tx=0x7f1e3001e000 comp rx=0 tx=0).stop 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 -- 172.21.15.105:0/3176468467 shutdown_connections 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff852cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff832420 0x55b9ff847890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 --2- 172.21.15.105:0/3176468467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff831a30 0x55b9ff830910 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 -- 172.21.15.105:0/3176468467 >> 172.21.15.105:0/3176468467 conn(0x55b9ff82cfa0 msgr2=0x55b9ff830200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 -- 172.21.15.105:0/3176468467 shutdown_connections 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 -- 172.21.15.105:0/3176468467 wait complete. 2024-03-26T18:01:04.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 -- start start 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 Processor -- start 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff845190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff8456d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.381+0000 7f1e481c8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e481c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401a7fb0 con 0x55b9ff832420 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff8456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff8456d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53876/0 (socket says 172.21.15.105:53876) 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 -- 172.21.15.105:0/345932880 learned_addr learned my addr 172.21.15.105:0/345932880 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e3effd640 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff845190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e3f7fe640 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401a78e0 con 0x55b9ff837590 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e401a4b20 con 0x55b9ff831a30 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e481c8740 10 client.0 _ll_register_callbacks cb 0x7f1e38009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:04.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 -- 172.21.15.105:0/345932880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 msgr2=0x55b9ff845190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff845190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 -- 172.21.15.105:0/345932880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 msgr2=0x55b9ff8466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 -- 172.21.15.105:0/345932880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9ff83ec70 con 0x55b9ff832420 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e367fc640 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff8456d0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f1e2802c170 tx=0x7f1e280b9e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd30a61d1513ec48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e267fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.382+0000 7f1e267fc640 1 -- 172.21.15.105:0/345932880 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e280bd470 con 0x55b9ff832420 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e267fc640 1 -- 172.21.15.105:0/345932880 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f1e280bbe10 con 0x55b9ff832420 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e300043c0 con 0x55b9ff832420 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e267fc640 1 -- 172.21.15.105:0/345932880 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1e280be210 con 0x55b9ff832420 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 10 client.23640 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b9ff8b2370 con 0x55b9ff832420 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e267fc640 1 -- 172.21.15.105:0/345932880 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e300043c0 con 0x55b9ff832420 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e267fc640 1 -- 172.21.15.105:0/345932880 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1e280ba180 con 0x55b9ff832420 2024-03-26T18:01:04.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 10 client.23640 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 10 client.23640 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 20 client.23640 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b9ff8b2cc0 0x55b9ff8b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 4 client.23640 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.383+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9ff79e200 con 0x55b9ff8b2cc0 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.384+0000 7f1e3effd640 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b9ff8b2cc0 0x55b9ff8b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.384+0000 7f1e267fc640 10 client.23640 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.384+0000 7f1e3effd640 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b9ff8b2cc0 0x55b9ff8b50b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b864e952f0a9c32 server_cookie=7ee3f729556178e6 in_seq=0 out_seq=0 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.384+0000 7f1e267fc640 1 -- 172.21.15.105:0/345932880 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9ff79e200 con 0x55b9ff8b2cc0 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.384+0000 7f1e267fc640 10 client.23640 handle_command_reply: tid=0 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.384+0000 7f1e481c8740 1 client.23640 shutdown 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 msgr2=0x55b9ff8456d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff8456d0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f1e2802c170 tx=0x7f1e280b9e70 comp rx=0 tx=0).stop 2024-03-26T18:01:04.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 shutdown_connections 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55b9ff8b2cc0 0x55b9ff8b50b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b9ff837590 0x55b9ff8466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e267fc640 0 client.23640 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b9ff832420 0x55b9ff8456d0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 --2- 172.21.15.105:0/345932880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b9ff831a30 0x55b9ff845190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 >> 172.21.15.105:0/345932880 conn(0x55b9ff82cfa0 msgr2=0x55b9ff830a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 shutdown_connections 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 1 -- 172.21.15.105:0/345932880 wait complete. 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.385+0000 7f1e481c8740 20 client.23640 trim_cache size 0 max 16384 2024-03-26T18:01:04.372 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 -- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1e2c0a3020 msgr2=0x7f1e2c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1e2c0a3020 0x7f1e2c0a54e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x55b9ff832c80 tx=0x7f1e38004a40 comp rx=0 tx=0).stop 2024-03-26T18:01:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 -- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 msgr2=0x7f1e4019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 0x7f1e4019c890 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000c00 tx=0x7f1e28005dc0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 -- 172.21.15.105:0/808300859 shutdown_connections 2024-03-26T18:01:04.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1e2c0a3020 0x7f1e2c0a54e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e40102370 0x7f1e4019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e401019b0 0x7f1e4019c890 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 --2- 172.21.15.105:0/808300859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1e401007c0 0x7f1e40069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 -- 172.21.15.105:0/808300859 >> 172.21.15.105:0/808300859 conn(0x7f1e400fbd50 msgr2=0x7f1e40104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 -- 172.21.15.105:0/808300859 shutdown_connections 2024-03-26T18:01:04.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.386+0000 7f1e455df640 1 -- 172.21.15.105:0/808300859 wait complete. 2024-03-26T18:01:04.382 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T18:01:04.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:03 smithi135 ceph-mon[35636]: pgmap v1033: 97 pgs: 97 active+clean; 358 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 3.4 MiB/s wr, 237 op/s 2024-03-26T18:01:04.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:03 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4099708934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 -- 172.21.15.105:0/435321627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81007c0 msgr2=0x7f88f8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/435321627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81007c0 0x7f88f8100be0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009900 tx=0x7f88e802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 -- 172.21.15.105:0/435321627 shutdown_connections 2024-03-26T18:01:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/435321627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88f8102370 0x7f88f8108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/435321627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88f81019b0 0x7f88f8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/435321627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81007c0 0x7f88f8100be0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 -- 172.21.15.105:0/435321627 >> 172.21.15.105:0/435321627 conn(0x7f88f80fbd50 msgr2=0x7f88f80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 -- 172.21.15.105:0/435321627 shutdown_connections 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.547+0000 7f88fe4ff640 1 -- 172.21.15.105:0/435321627 wait complete. 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fe4ff640 1 Processor -- start 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fe4ff640 1 -- start start 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fe4ff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88f81007c0 0x7f88f8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fe4ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81019b0 0x7f88f8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fe4ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88f8102370 0x7f88f819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fe4ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88f810c2c0 con 0x7f88f81019b0 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fe4ff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88f810c140 con 0x7f88f8102370 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fe4ff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88f810c440 con 0x7f88f81007c0 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81019b0 0x7f88f8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88fca9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88f8102370 0x7f88f819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81019b0 0x7f88f8198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57208/0 (socket says 172.21.15.105:57208) 2024-03-26T18:01:04.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88f77fe640 1 -- 172.21.15.105:0/3600442498 learned_addr learned my addr 172.21.15.105:0/3600442498 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.548+0000 7f88f7fff640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88f81007c0 0x7f88f8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f77fe640 1 -- 172.21.15.105:0/3600442498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88f81007c0 msgr2=0x7f88f8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f77fe640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88f81007c0 0x7f88f8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f77fe640 1 -- 172.21.15.105:0/3600442498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88f8102370 msgr2=0x7f88f819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f77fe640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88f8102370 0x7f88f819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f77fe640 1 -- 172.21.15.105:0/3600442498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f819f830 con 0x7f88f81019b0 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88fca9c640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88f8102370 0x7f88f819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f77fe640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81019b0 0x7f88f8198160 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00e9f0 tx=0x7f88ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f57fa640 1 -- 172.21.15.105:0/3600442498 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00cd80 con 0x7f88f81019b0 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f57fa640 1 -- 172.21.15.105:0/3600442498 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88ec0108e0 con 0x7f88f81019b0 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88f57fa640 1 -- 172.21.15.105:0/3600442498 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec018920 con 0x7f88f81019b0 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88fe4ff640 1 -- 172.21.15.105:0/3600442498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f819fb20 con 0x7f88f81019b0 2024-03-26T18:01:04.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.549+0000 7f88fe4ff640 1 -- 172.21.15.105:0/3600442498 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f81a0020 con 0x7f88f81019b0 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.552+0000 7f88f57fa640 1 -- 172.21.15.105:0/3600442498 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f88ec010a90 con 0x7f88f81019b0 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.552+0000 7f89011c8740 1 Processor -- start 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.552+0000 7f89011c8740 1 -- start start 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f57fa640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88e00a2fa0 0x7f88e00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f89011c8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc81474e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f89011c8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc81479f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f89011c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f89011c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc813d9970 con 0x55bc8147a590 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f89011c8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc81381d70 con 0x55bc81475420 2024-03-26T18:01:04.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f89011c8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc813d5920 con 0x55bc81474a30 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88fca9c640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88e00a2fa0 0x7f88e00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f57fa640 1 -- 172.21.15.105:0/3600442498 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f88ec0cf730 con 0x7f88f81019b0 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc81474e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc81474e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60828/0 (socket says 172.21.15.105:60828) 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f7fff640 1 -- 172.21.15.105:0/3368478073 learned_addr learned my addr 172.21.15.105:0/3368478073 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f77fe640 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc81479f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88fca9c640 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f7fff640 1 -- 172.21.15.105:0/3368478073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 msgr2=0x55bc81479f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f7fff640 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc81479f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f7fff640 1 -- 172.21.15.105:0/3368478073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 msgr2=0x55bc8147ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f7fff640 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88f7fff640 1 -- 172.21.15.105:0/3368478073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc810ed3f0 con 0x55bc81474a30 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.553+0000 7f88fca9c640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88e00a2fa0 0x7f88e00a5460 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f88f0005fd0 tx=0x7f88f000eea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.554+0000 7f88f7fff640 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc81474e50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009900 tx=0x7f88e8009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67ed839c9502d396 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.554+0000 7f88de7fc640 1 -- 172.21.15.105:0/3368478073 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8045020 con 0x55bc81474a30 2024-03-26T18:01:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.554+0000 7f88de7fc640 1 -- 172.21.15.105:0/3368478073 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88e8004040 con 0x55bc81474a30 2024-03-26T18:01:04.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.554+0000 7f88de7fc640 1 -- 172.21.15.105:0/3368478073 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e80054a0 con 0x55bc81474a30 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 -- 172.21.15.105:0/3368478073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 msgr2=0x55bc81474e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc81474e50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009900 tx=0x7f88e8009610 comp rx=0 tx=0).stop 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 -- 172.21.15.105:0/3368478073 shutdown_connections 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc81479f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 --2- 172.21.15.105:0/3368478073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc81474e50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 -- 172.21.15.105:0/3368478073 >> 172.21.15.105:0/3368478073 conn(0x55bc8146ffa0 msgr2=0x55bc814703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 -- 172.21.15.105:0/3368478073 shutdown_connections 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 -- 172.21.15.105:0/3368478073 wait complete. 2024-03-26T18:01:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 -- start start 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.555+0000 7f89011c8740 1 Processor -- start 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f89011c8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f89011c8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc8147f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f89011c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc81486410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f89011c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bc8147d4e0 con 0x55bc8147a590 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f89011c8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc81395310 con 0x55bc81475420 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f89011c8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bc8147d660 con 0x55bc81474a30 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f89011c8740 10 client.0 _ll_register_callbacks cb 0x55bc813d0c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc8147f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc8147f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34154/0 (socket says 172.21.15.105:34154) 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 -- 172.21.15.105:0/691467885 learned_addr learned my addr 172.21.15.105:0/691467885 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88fca9c640 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc81486410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 -- 172.21.15.105:0/691467885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 msgr2=0x55bc8147ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 -- 172.21.15.105:0/691467885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 msgr2=0x55bc81486410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc81486410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 -- 172.21.15.105:0/691467885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc811b77d0 con 0x55bc81475420 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88fca9c640 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc81486410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f7fff640 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88f77fe640 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc8147f3d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f88ec09be60 tx=0x7f88ec09ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00a37c0bd7ef49e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.556+0000 7f88d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f88d77fe640 1 -- 172.21.15.105:0/691467885 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec019b80 con 0x55bc81475420 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f88d77fe640 1 -- 172.21.15.105:0/691467885 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f88ec010430 con 0x55bc81475420 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 1 -- 172.21.15.105:0/691467885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e8004040 con 0x55bc81475420 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f88d77fe640 1 -- 172.21.15.105:0/691467885 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f88ec0ab050 con 0x55bc81475420 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 10 client.33329 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 1 -- 172.21.15.105:0/691467885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bc81489780 con 0x55bc81475420 2024-03-26T18:01:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f88d77fe640 1 -- 172.21.15.105:0/691467885 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88e8004040 con 0x55bc81475420 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f88d77fe640 1 -- 172.21.15.105:0/691467885 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f88ec0b9070 con 0x55bc81475420 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 10 client.33329 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 10 client.33329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 20 client.33329 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bc8148ae30 0x55bc8148d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 4 client.33329 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.557+0000 7f89011c8740 1 -- 172.21.15.105:0/691467885 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc813e6eb0 con 0x55bc8148ae30 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.558+0000 7f88f7fff640 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bc8148ae30 0x55bc8148d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.558+0000 7f88d77fe640 10 client.33329 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:04.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.558+0000 7f88f7fff640 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bc8148ae30 0x55bc8148d260 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2511ba3ac96c6dbb server_cookie=382e3c39e62fd3b1 in_seq=0 out_seq=0 2024-03-26T18:01:04.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.560+0000 7f88d77fe640 1 -- 172.21.15.105:0/691467885 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc813e6eb0 con 0x55bc8148ae30 2024-03-26T18:01:04.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.560+0000 7f88d77fe640 10 client.33329 handle_command_reply: tid=0 2024-03-26T18:01:04.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.560+0000 7f89011c8740 1 client.33329 shutdown 2024-03-26T18:01:04.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 -- 172.21.15.105:0/691467885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 msgr2=0x55bc8147f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc8147f3d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f88ec09be60 tx=0x7f88ec09ba10 comp rx=0 tx=0).stop 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 -- 172.21.15.105:0/691467885 shutdown_connections 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bc8148ae30 0x55bc8148d260 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc81486410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81475420 0x55bc8147f3d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f88d77fe640 0 client.33329 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 --2- 172.21.15.105:0/691467885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 -- 172.21.15.105:0/691467885 >> 172.21.15.105:0/691467885 conn(0x55bc8146ffa0 msgr2=0x55bc8146f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 -- 172.21.15.105:0/691467885 shutdown_connections 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 1 -- 172.21.15.105:0/691467885 wait complete. 2024-03-26T18:01:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.561+0000 7f89011c8740 20 client.33329 trim_cache size 0 max 16384 2024-03-26T18:01:04.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 76907 7.06 MB/sec execute 467 sec latency 120.931 ms 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.583+0000 7f89011c8740 1 Processor -- start 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.583+0000 7f89011c8740 1 -- start start 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.583+0000 7f89011c8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.583+0000 7f89011c8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81475420 0x55bc814863d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.583+0000 7f89011c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.583+0000 7f89011c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81a6bd0 con 0x55bc8147a590 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.583+0000 7f89011c8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81a8710 con 0x55bc81474a30 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.583+0000 7f89011c8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81a72a0 con 0x55bc81475420 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88f7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81475420 0x55bc814863d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88fca9c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88f77fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81475420 0x55bc814863d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60848/0 (socket says 172.21.15.105:60848) 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88f7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34164/0 (socket says 172.21.15.105:34164) 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88f77fe640 1 -- 172.21.15.105:0/2029705158 learned_addr learned my addr 172.21.15.105:0/2029705158 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88fca9c640 1 -- 172.21.15.105:0/2029705158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81475420 msgr2=0x55bc814863d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88fca9c640 1 --2- 172.21.15.105:0/2029705158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81475420 0x55bc814863d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88fca9c640 1 -- 172.21.15.105:0/2029705158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81474a30 msgr2=0x55bc8147ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88fca9c640 1 --2- 172.21.15.105:0/2029705158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88fca9c640 1 -- 172.21.15.105:0/2029705158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc8146fdd0 con 0x55bc8147a590 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88fca9c640 1 --2- 172.21.15.105:0/2029705158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147f3d0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f88f0005cc0 tx=0x7f88f001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f64f610fd45ad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88ddffb640 1 -- 172.21.15.105:0/2029705158 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0004480 con 0x55bc8147a590 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88ddffb640 1 -- 172.21.15.105:0/2029705158 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88f0004620 con 0x55bc8147a590 2024-03-26T18:01:04.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.584+0000 7f88ddffb640 1 -- 172.21.15.105:0/2029705158 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f000f3c0 con 0x55bc8147a590 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 -- 172.21.15.105:0/2029705158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 msgr2=0x55bc8147f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 --2- 172.21.15.105:0/2029705158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147f3d0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f88f0005cc0 tx=0x7f88f001c000 comp rx=0 tx=0).stop 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 -- 172.21.15.105:0/2029705158 shutdown_connections 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 --2- 172.21.15.105:0/2029705158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc8147a590 0x55bc8147f3d0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 --2- 172.21.15.105:0/2029705158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81475420 0x55bc814863d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 --2- 172.21.15.105:0/2029705158 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc81474a30 0x55bc8147ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 -- 172.21.15.105:0/2029705158 >> 172.21.15.105:0/2029705158 conn(0x55bc8146ffa0 msgr2=0x55bc8146b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 -- 172.21.15.105:0/2029705158 shutdown_connections 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.585+0000 7f89011c8740 1 -- 172.21.15.105:0/2029705158 wait complete. 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 1 -- start start 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 1 Processor -- start 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc814834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc81475420 0x55bc814839e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8147a590 0x55bc814849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81a8040 con 0x55bc81475420 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81a7970 con 0x55bc8147a590 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88f81a4bb0 con 0x55bc81474a30 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f88f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc814834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f89011c8740 10 client.0 _ll_register_callbacks cb 0x7f88ec0bc0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f88f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc814834a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60862/0 (socket says 172.21.15.105:60862) 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f88f7fff640 1 -- 172.21.15.105:0/1942195446 learned_addr learned my addr 172.21.15.105:0/1942195446 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f88f77fe640 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc81475420 0x55bc814839e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.586+0000 7f88fca9c640 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8147a590 0x55bc814849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88f7fff640 1 -- 172.21.15.105:0/1942195446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8147a590 msgr2=0x55bc814849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88f7fff640 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8147a590 0x55bc814849f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88f7fff640 1 -- 172.21.15.105:0/1942195446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc81475420 msgr2=0x55bc814839e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88f7fff640 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc81475420 0x55bc814839e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88f7fff640 1 -- 172.21.15.105:0/1942195446 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc81484fe0 con 0x55bc81474a30 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88f77fe640 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc81475420 0x55bc814839e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88f7fff640 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc814834a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f88e8000c00 tx=0x7f88e8005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=517d8087470dd7db server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.587+0000 7f88d77fe640 1 -- 172.21.15.105:0/1942195446 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8045040 con 0x55bc81474a30 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f88d77fe640 1 -- 172.21.15.105:0/1942195446 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f88e802c8e0 con 0x55bc81474a30 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 1 -- 172.21.15.105:0/1942195446 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f0004620 con 0x55bc81474a30 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f88d77fe640 1 -- 172.21.15.105:0/1942195446 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f88e802cae0 con 0x55bc81474a30 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 10 client.33322 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 1 -- 172.21.15.105:0/1942195446 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bc81486350 con 0x55bc81474a30 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f88d77fe640 1 -- 172.21.15.105:0/1942195446 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88f0004620 con 0x55bc81474a30 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f88d77fe640 1 -- 172.21.15.105:0/1942195446 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f88e8059470 con 0x55bc81474a30 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 10 client.33322 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 10 client.33322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:04.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 20 client.33322 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:04.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bc8148ae30 0x55bc814f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:04.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 4 client.33322 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:04.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.588+0000 7f89011c8740 1 -- 172.21.15.105:0/1942195446 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc813e1200 con 0x55bc8148ae30 2024-03-26T18:01:04.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.589+0000 7f88f77fe640 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bc8148ae30 0x55bc814f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:04.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.589+0000 7f88d77fe640 10 client.33322 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:04.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.589+0000 7f88f77fe640 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bc8148ae30 0x55bc814f31e0 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed612f3159693c5 server_cookie=c848fd20bea42ad3 in_seq=0 out_seq=0 2024-03-26T18:01:04.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.590+0000 7f88d77fe640 1 -- 172.21.15.105:0/1942195446 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc813e1200 con 0x55bc8148ae30 2024-03-26T18:01:04.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.590+0000 7f88d77fe640 10 client.33322 handle_command_reply: tid=0 2024-03-26T18:01:04.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.590+0000 7f89011c8740 1 client.33322 shutdown 2024-03-26T18:01:04.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 -- 172.21.15.105:0/1942195446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 msgr2=0x55bc814834a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc814834a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f88e8000c00 tx=0x7f88e8005da0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 -- 172.21.15.105:0/1942195446 shutdown_connections 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bc8148ae30 0x55bc814f31e0 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc8147a590 0x55bc814849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f88d77fe640 0 client.33322 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bc81475420 0x55bc814839e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 --2- 172.21.15.105:0/1942195446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bc81474a30 0x55bc814834a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 -- 172.21.15.105:0/1942195446 >> 172.21.15.105:0/1942195446 conn(0x55bc8146ffa0 msgr2=0x55bc81473200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 -- 172.21.15.105:0/1942195446 shutdown_connections 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 1 -- 172.21.15.105:0/1942195446 wait complete. 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f89011c8740 20 client.33322 trim_cache size 0 max 16384 2024-03-26T18:01:04.576 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f88fe4ff640 1 -- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88e00a2fa0 msgr2=0x7f88e00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88e00a2fa0 0x7f88e00a5460 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f88f0005fd0 tx=0x7f88f000eea0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f88fe4ff640 1 -- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81019b0 msgr2=0x7f88f8198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.591+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81019b0 0x7f88f8198160 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00e9f0 tx=0x7f88ec00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.592+0000 7f88fe4ff640 1 -- 172.21.15.105:0/3600442498 shutdown_connections 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.592+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88e00a2fa0 0x7f88e00a5460 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.592+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88f8102370 0x7f88f819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.592+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88f81019b0 0x7f88f8198160 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.592+0000 7f88fe4ff640 1 --2- 172.21.15.105:0/3600442498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88f81007c0 0x7f88f8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.592+0000 7f88fe4ff640 1 -- 172.21.15.105:0/3600442498 >> 172.21.15.105:0/3600442498 conn(0x7f88f80fbd50 msgr2=0x7f88f8068ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.592+0000 7f88fe4ff640 1 -- 172.21.15.105:0/3600442498 shutdown_connections 2024-03-26T18:01:04.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:04.592+0000 7f88fe4ff640 1 -- 172.21.15.105:0/3600442498 wait complete. 2024-03-26T18:01:05.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77033 7.07 MB/sec execute 468 sec latency 127.913 ms 2024-03-26T18:01:05.588 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T18:01:05.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:01:05.589 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T18:01:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 -- 172.21.15.105:0/3888287879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff450102370 msgr2=0x7ff450108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 --2- 172.21.15.105:0/3888287879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff450102370 0x7ff450108c00 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7ff448009ca0 tx=0x7ff44802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 -- 172.21.15.105:0/3888287879 shutdown_connections 2024-03-26T18:01:05.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 --2- 172.21.15.105:0/3888287879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff450102370 0x7ff450108c00 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 --2- 172.21.15.105:0/3888287879 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4501019b0 0x7ff450101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 --2- 172.21.15.105:0/3888287879 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff4501007c0 0x7ff450100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 -- 172.21.15.105:0/3888287879 >> 172.21.15.105:0/3888287879 conn(0x7ff4500fbd50 msgr2=0x7ff4500fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 -- 172.21.15.105:0/3888287879 shutdown_connections 2024-03-26T18:01:05.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.757+0000 7ff456281640 1 -- 172.21.15.105:0/3888287879 wait complete. 2024-03-26T18:01:05.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff456281640 1 Processor -- start 2024-03-26T18:01:05.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff456281640 1 -- start start 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff456281640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4501007c0 0x7ff45019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff456281640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4501019b0 0x7ff45019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff456281640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff450102370 0x7ff4501a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff456281640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff45010d410 con 0x7ff4501007c0 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff456281640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff45010d290 con 0x7ff450102370 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff456281640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff45010d590 con 0x7ff4501019b0 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff44ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4501007c0 0x7ff45019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff44ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4501007c0 0x7ff45019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57262/0 (socket says 172.21.15.105:57262) 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff44f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4501019b0 0x7ff45019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff44ffff640 1 -- 172.21.15.105:0/2812344848 learned_addr learned my addr 172.21.15.105:0/2812344848 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.758+0000 7ff45481e640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff450102370 0x7ff4501a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44f7fe640 1 -- 172.21.15.105:0/2812344848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff450102370 msgr2=0x7ff4501a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44f7fe640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff450102370 0x7ff4501a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44f7fe640 1 -- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4501007c0 msgr2=0x7ff45019e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44f7fe640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4501007c0 0x7ff45019e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44f7fe640 1 -- 172.21.15.105:0/2812344848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4501a5d00 con 0x7ff4501019b0 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44f7fe640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4501019b0 0x7ff45019e620 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff44400e9f0 tx=0x7ff44400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44d7fa640 1 -- 172.21.15.105:0/2812344848 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44400ccc0 con 0x7ff4501019b0 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44d7fa640 1 -- 172.21.15.105:0/2812344848 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff444005020 con 0x7ff4501019b0 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff456281640 1 -- 172.21.15.105:0/2812344848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4501a5ff0 con 0x7ff4501019b0 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff44d7fa640 1 -- 172.21.15.105:0/2812344848 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444004a00 con 0x7ff4501019b0 2024-03-26T18:01:05.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.759+0000 7ff456281640 1 -- 172.21.15.105:0/2812344848 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4501a6460 con 0x7ff4501019b0 2024-03-26T18:01:05.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff458e6e740 1 Processor -- start 2024-03-26T18:01:05.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff458e6e740 1 -- start start 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff44d7fa640 1 -- 172.21.15.105:0/2812344848 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff444004ba0 con 0x7ff4501019b0 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7b80 0x55bfbdbfc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff458e6e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bfbdb5c630 con 0x55bfbdbf7b80 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff458e6e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bfbdb04e10 con 0x55bfbdbfccf0 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.762+0000 7ff458e6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bfbdb585e0 con 0x55bfbdbf7190 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff45481e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7b80 0x55bfbdbfc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff45481e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60876/0 (socket says 172.21.15.105:60876) 2024-03-26T18:01:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff45481e640 1 -- 172.21.15.105:0/1655205989 learned_addr learned my addr 172.21.15.105:0/1655205989 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44ffff640 1 -- 172.21.15.105:0/1655205989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 msgr2=0x55bfbdbf75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44ffff640 1 --2- 172.21.15.105:0/1655205989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44ffff640 1 -- 172.21.15.105:0/1655205989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbfccf0 msgr2=0x55bfbdbff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44ffff640 1 --2- 172.21.15.105:0/1655205989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44ffff640 1 -- 172.21.15.105:0/1655205989 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfbdb1b740 con 0x55bfbdbf7b80 2024-03-26T18:01:05.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44f7fe640 1 --2- 172.21.15.105:0/1655205989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44d7fa640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2f20 0x7ff4380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.763+0000 7ff44ffff640 1 --2- 172.21.15.105:0/1655205989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7b80 0x55bfbdbfc680 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7ff4400058c0 tx=0x7ff440011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bd6291a0314228 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.764+0000 7ff4367fc640 1 -- 172.21.15.105:0/1655205989 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440004220 con 0x55bfbdbf7b80 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.764+0000 7ff4367fc640 1 -- 172.21.15.105:0/1655205989 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4400043c0 con 0x55bfbdbf7b80 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.764+0000 7ff4367fc640 1 -- 172.21.15.105:0/1655205989 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440004d40 con 0x55bfbdbf7b80 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.764+0000 7ff45481e640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2f20 0x7ff4380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.764+0000 7ff44d7fa640 1 -- 172.21.15.105:0/2812344848 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7ff4440cf3f0 con 0x7ff4501019b0 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 -- 172.21.15.105:0/1655205989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7b80 msgr2=0x55bfbdbfc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff45481e640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2f20 0x7ff4380a53e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55bfbdbf83e0 tx=0x7ff4480324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 --2- 172.21.15.105:0/1655205989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7b80 0x55bfbdbfc680 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7ff4400058c0 tx=0x7ff440011810 comp rx=0 tx=0).stop 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 -- 172.21.15.105:0/1655205989 shutdown_connections 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 --2- 172.21.15.105:0/1655205989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 --2- 172.21.15.105:0/1655205989 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7b80 0x55bfbdbfc680 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 --2- 172.21.15.105:0/1655205989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 -- 172.21.15.105:0/1655205989 >> 172.21.15.105:0/1655205989 conn(0x55bfbdbf2700 msgr2=0x55bfbdbf2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 -- 172.21.15.105:0/1655205989 shutdown_connections 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.765+0000 7ff458e6e740 1 -- 172.21.15.105:0/1655205989 wait complete. 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff458e6e740 1 -- start start 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff458e6e740 1 Processor -- start 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdc01730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc01c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc08cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff458e6e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bfbdbffca0 con 0x55bfbdbf7190 2024-03-26T18:01:05.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff458e6e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bfbdb181a0 con 0x55bfbdbf7b80 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff458e6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bfbdbffe20 con 0x55bfbdbfccf0 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff45481e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc08cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.766+0000 7ff44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc01c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff45481e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc08cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60882/0 (socket says 172.21.15.105:60882) 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc01c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34188/0 (socket says 172.21.15.105:34188) 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff45481e640 1 -- 172.21.15.105:0/2231276556 learned_addr learned my addr 172.21.15.105:0/2231276556 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff458e6e740 10 client.0 _ll_register_callbacks cb 0x55bfbd86fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff44ffff640 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdc01730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff45481e640 1 -- 172.21.15.105:0/2231276556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 msgr2=0x55bfbdc01c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff45481e640 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc01c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff45481e640 1 -- 172.21.15.105:0/2231276556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 msgr2=0x55bfbdc01730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff45481e640 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdc01730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff45481e640 1 -- 172.21.15.105:0/2231276556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfbd9387e0 con 0x55bfbdbfccf0 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff45481e640 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc08cb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff44802c630 tx=0x7ff4480095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44485ccb44680051 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.767+0000 7ff42f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.768+0000 7ff42f7fe640 1 -- 172.21.15.105:0/2231276556 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448051020 con 0x55bfbdbfccf0 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.768+0000 7ff458e6e740 1 -- 172.21.15.105:0/2231276556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4400043c0 con 0x55bfbdbfccf0 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.768+0000 7ff42f7fe640 1 -- 172.21.15.105:0/2231276556 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7ff448004100 con 0x55bfbdbfccf0 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.768+0000 7ff42f7fe640 1 -- 172.21.15.105:0/2231276556 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff44805c050 con 0x55bfbdbfccf0 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.768+0000 7ff458e6e740 10 client.33334 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.768+0000 7ff458e6e740 1 -- 172.21.15.105:0/2231276556 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bfbd998f70 con 0x55bfbdbfccf0 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.768+0000 7ff42f7fe640 1 -- 172.21.15.105:0/2231276556 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4400043c0 con 0x55bfbdbfccf0 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.769+0000 7ff42f7fe640 1 -- 172.21.15.105:0/2231276556 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff44804d8c0 con 0x55bfbdbfccf0 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.769+0000 7ff458e6e740 10 client.33334 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.769+0000 7ff458e6e740 10 client.33334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.769+0000 7ff458e6e740 20 client.33334 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.769+0000 7ff458e6e740 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfbdc0da30 0x55bfbdc0fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.769+0000 7ff458e6e740 4 client.33334 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:05.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.769+0000 7ff458e6e740 1 -- 172.21.15.105:0/2231276556 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfbdb69b70 con 0x55bfbdc0da30 2024-03-26T18:01:05.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.769+0000 7ff44ffff640 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfbdc0da30 0x55bfbdc0fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.770+0000 7ff42f7fe640 10 client.33334 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:05.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.770+0000 7ff44ffff640 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfbdc0da30 0x55bfbdc0fed0 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90da7ec7ae35adf server_cookie=ad186b71080ca068 in_seq=0 out_seq=0 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.773+0000 7ff42f7fe640 1 -- 172.21.15.105:0/2231276556 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bfbdb69b70 con 0x55bfbdc0da30 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.773+0000 7ff42f7fe640 10 client.33334 handle_command_reply: tid=0 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.773+0000 7ff458e6e740 1 client.33334 shutdown 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 -- 172.21.15.105:0/2231276556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 msgr2=0x55bfbdc08cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc08cb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff44802c630 tx=0x7ff4480095c0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 -- 172.21.15.105:0/2231276556 shutdown_connections 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfbdc0da30 0x55bfbdc0fed0 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc08cb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc01c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 --2- 172.21.15.105:0/2231276556 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdc01730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 -- 172.21.15.105:0/2231276556 >> 172.21.15.105:0/2231276556 conn(0x55bfbdbf2700 msgr2=0x55bfbdbf1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff42f7fe640 0 client.33334 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 -- 172.21.15.105:0/2231276556 shutdown_connections 2024-03-26T18:01:05.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 1 -- 172.21.15.105:0/2231276556 wait complete. 2024-03-26T18:01:05.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.774+0000 7ff458e6e740 20 client.33334 trim_cache size 0 max 16384 2024-03-26T18:01:05.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff458e6e740 1 Processor -- start 2024-03-26T18:01:05.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff458e6e740 1 -- start start 2024-03-26T18:01:05.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc08c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbf15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff458e6e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501af310 con 0x55bfbdbf7190 2024-03-26T18:01:05.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff458e6e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501b0e50 con 0x55bfbdbf7b80 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff458e6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501af9e0 con 0x55bfbdbfccf0 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57298/0 (socket says 172.21.15.105:57298) 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44ffff640 1 -- 172.21.15.105:0/1055259424 learned_addr learned my addr 172.21.15.105:0/1055259424 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44f7fe640 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc08c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff45481e640 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbf15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44ffff640 1 -- 172.21.15.105:0/1055259424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 msgr2=0x55bfbdbf15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44ffff640 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbf15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44ffff640 1 -- 172.21.15.105:0/1055259424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 msgr2=0x55bfbdc08c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44ffff640 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc08c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.801+0000 7ff44ffff640 1 -- 172.21.15.105:0/1055259424 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfbd830170 con 0x55bfbdbf7190 2024-03-26T18:01:05.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.802+0000 7ff44ffff640 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf1070 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ff440000c00 tx=0x7ff440011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d5e68c6cd0f374 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.802+0000 7ff435ffb640 1 -- 172.21.15.105:0/1055259424 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440031070 con 0x55bfbdbf7190 2024-03-26T18:01:05.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.802+0000 7ff435ffb640 1 -- 172.21.15.105:0/1055259424 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff44002b810 con 0x55bfbdbf7190 2024-03-26T18:01:05.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.802+0000 7ff435ffb640 1 -- 172.21.15.105:0/1055259424 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44001d040 con 0x55bfbdbf7190 2024-03-26T18:01:05.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 -- 172.21.15.105:0/1055259424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 msgr2=0x55bfbdbf1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf1070 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ff440000c00 tx=0x7ff440011810 comp rx=0 tx=0).stop 2024-03-26T18:01:05.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 -- 172.21.15.105:0/1055259424 shutdown_connections 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbfccf0 0x55bfbdbf15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc08c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 --2- 172.21.15.105:0/1055259424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbf7190 0x55bfbdbf1070 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 -- 172.21.15.105:0/1055259424 >> 172.21.15.105:0/1055259424 conn(0x55bfbdbf2700 msgr2=0x55bfbdbf9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 -- 172.21.15.105:0/1055259424 shutdown_connections 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 -- 172.21.15.105:0/1055259424 wait complete. 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.803+0000 7ff458e6e740 1 -- start start 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff458e6e740 1 Processor -- start 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdc06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc06ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff458e6e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc07ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff458e6e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501b0780 con 0x55bfbdbfccf0 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff458e6e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501b00b0 con 0x55bfbdbf7b80 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff458e6e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501ad2f0 con 0x55bfbdbf7190 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff44ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdc06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff44f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc06ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff44ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdc06580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60906/0 (socket says 172.21.15.105:60906) 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff44ffff640 1 -- 172.21.15.105:0/3729718135 learned_addr learned my addr 172.21.15.105:0/3729718135 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff458e6e740 10 client.0 _ll_register_callbacks cb 0x7ff448044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.804+0000 7ff45481e640 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc07ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff44f7fe640 1 -- 172.21.15.105:0/3729718135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 msgr2=0x55bfbdc06580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff44f7fe640 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdc06580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff44f7fe640 1 -- 172.21.15.105:0/3729718135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbfccf0 msgr2=0x55bfbdc07ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff44f7fe640 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc07ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff44f7fe640 1 -- 172.21.15.105:0/3729718135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfbdc080c0 con 0x55bfbdbf7b80 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff44ffff640 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdc06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff45481e640 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc07ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff44f7fe640 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc06ac0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff44409b780 tx=0x7ff4440096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a84904b4057406 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff42f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:05.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff42f7fe640 1 -- 172.21.15.105:0/3729718135 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44401c040 con 0x55bfbdbf7b80 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff458e6e740 1 -- 172.21.15.105:0/3729718135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff44002b810 con 0x55bfbdbf7b80 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff42f7fe640 1 -- 172.21.15.105:0/3729718135 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7ff4440051d0 con 0x55bfbdbf7b80 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff42f7fe640 1 -- 172.21.15.105:0/3729718135 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff44409b3e0 con 0x55bfbdbf7b80 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff458e6e740 10 client.33353 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.805+0000 7ff458e6e740 1 -- 172.21.15.105:0/3729718135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55bfbdc740e0 con 0x55bfbdbf7b80 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff42f7fe640 1 -- 172.21.15.105:0/3729718135 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff44002b810 con 0x55bfbdbf7b80 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff42f7fe640 1 -- 172.21.15.105:0/3729718135 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff4440a4000 con 0x55bfbdbf7b80 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff458e6e740 10 client.33353 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff458e6e740 10 client.33353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff458e6e740 20 client.33353 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff458e6e740 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfbdc74a80 0x55bfbdc76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff458e6e740 4 client.33353 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff458e6e740 1 -- 172.21.15.105:0/3729718135 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bfbdb63ec0 con 0x55bfbdc74a80 2024-03-26T18:01:05.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.806+0000 7ff44ffff640 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfbdc74a80 0x55bfbdc76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.807+0000 7ff42f7fe640 10 client.33353 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:05.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.807+0000 7ff44ffff640 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfbdc74a80 0x55bfbdc76ea0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae94ce751cac4d16 server_cookie=5cd469000722bfd1 in_seq=0 out_seq=0 2024-03-26T18:01:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff42f7fe640 1 -- 172.21.15.105:0/3729718135 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bfbdb63ec0 con 0x55bfbdc74a80 2024-03-26T18:01:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff42f7fe640 10 client.33353 handle_command_reply: tid=0 2024-03-26T18:01:05.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 client.33353 shutdown 2024-03-26T18:01:05.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 -- 172.21.15.105:0/3729718135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 msgr2=0x55bfbdc06ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc06ac0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff44409b780 tx=0x7ff4440096e0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 -- 172.21.15.105:0/3729718135 shutdown_connections 2024-03-26T18:01:05.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55bfbdc74a80 0x55bfbdc76ea0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bfbdbfccf0 0x55bfbdc07ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bfbdbf7b80 0x55bfbdc06ac0 secure :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff44409b780 tx=0x7ff4440096e0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff42f7fe640 0 client.33353 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 --2- 172.21.15.105:0/3729718135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bfbdbf7190 0x55bfbdc06580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 -- 172.21.15.105:0/3729718135 >> 172.21.15.105:0/3729718135 conn(0x55bfbdbf2700 msgr2=0x55bfbdbf5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 -- 172.21.15.105:0/3729718135 shutdown_connections 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.815+0000 7ff458e6e740 1 -- 172.21.15.105:0/3729718135 wait complete. 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.816+0000 7ff458e6e740 20 client.33353 trim_cache size 0 max 16384 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "d4fc86d9-68e5-4809-bd18-98097c8c0d77", 2024-03-26T18:01:05.798 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T18:01:05.799 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:01:05.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 -- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2f20 msgr2=0x7ff4380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2f20 0x7ff4380a53e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55bfbdbf83e0 tx=0x7ff4480324f0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 -- 172.21.15.105:0/2812344848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4501019b0 msgr2=0x7ff45019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4501019b0 0x7ff45019e620 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff44400e9f0 tx=0x7ff44400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 -- 172.21.15.105:0/2812344848 shutdown_connections 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff4380a2f20 0x7ff4380a53e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff450102370 0x7ff4501a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4501019b0 0x7ff45019e620 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 --2- 172.21.15.105:0/2812344848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4501007c0 0x7ff45019e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 -- 172.21.15.105:0/2812344848 >> 172.21.15.105:0/2812344848 conn(0x7ff4500fbd50 msgr2=0x7ff450102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 -- 172.21.15.105:0/2812344848 shutdown_connections 2024-03-26T18:01:05.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.817+0000 7ff456281640 1 -- 172.21.15.105:0/2812344848 wait complete. 2024-03-26T18:01:05.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:01:05.811 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:01:05.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 -- 172.21.15.105:0/1929637274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 msgr2=0x7f7b7c102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1929637274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 0x7f7b7c102f60 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402bc50 comp rx=0 tx=0).stop 2024-03-26T18:01:05.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 -- 172.21.15.105:0/1929637274 shutdown_connections 2024-03-26T18:01:05.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1929637274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7c1034a0 0x7f7b7c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1929637274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 0x7f7b7c102f60 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1929637274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c1018f0 0x7f7b7c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 -- 172.21.15.105:0/1929637274 >> 172.21.15.105:0/1929637274 conn(0x7f7b7c0fce60 msgr2=0x7f7b7c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 -- 172.21.15.105:0/1929637274 shutdown_connections 2024-03-26T18:01:05.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.958+0000 7f7b80a01640 1 -- 172.21.15.105:0/1929637274 wait complete. 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.959+0000 7f7b80a01640 1 Processor -- start 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.959+0000 7f7b80a01640 1 -- start start 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.959+0000 7f7b80a01640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7c1018f0 0x7f7b7c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.959+0000 7f7b80a01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 0x7f7b7c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.959+0000 7f7b80a01640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c1034a0 0x7f7b7c1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.959+0000 7f7b80a01640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c075960 con 0x7f7b7c102ae0 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.959+0000 7f7b80a01640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c0757e0 con 0x7f7b7c1034a0 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.959+0000 7f7b80a01640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c075ae0 con 0x7f7b7c1018f0 2024-03-26T18:01:05.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 0x7f7b7c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 0x7f7b7c19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57330/0 (socket says 172.21.15.105:57330) 2024-03-26T18:01:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 -- 172.21.15.105:0/1875826209 learned_addr learned my addr 172.21.15.105:0/1875826209 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b7ad9d640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c1034a0 0x7f7b7c1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7c1018f0 0x7f7b7c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 -- 172.21.15.105:0/1875826209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7c1018f0 msgr2=0x7f7b7c19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7c1018f0 0x7f7b7c19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 -- 172.21.15.105:0/1875826209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c1034a0 msgr2=0x7f7b7c1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c1034a0 0x7f7b7c1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 -- 172.21.15.105:0/1875826209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7c1a5d30 con 0x7f7b7c102ae0 2024-03-26T18:01:05.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b7ad9d640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c1034a0 0x7f7b7c1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:05.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.960+0000 7f7b73fff640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 0x7f7b7c19e650 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f7b64006010 tx=0x7f7b640052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.961+0000 7f7b737fe640 1 -- 172.21.15.105:0/1875826209 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64041020 con 0x7f7b7c102ae0 2024-03-26T18:01:05.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.961+0000 7f7b737fe640 1 -- 172.21.15.105:0/1875826209 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b6402caa0 con 0x7f7b7c102ae0 2024-03-26T18:01:05.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.961+0000 7f7b80a01640 1 -- 172.21.15.105:0/1875826209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7c1a5fc0 con 0x7f7b7c102ae0 2024-03-26T18:01:05.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.961+0000 7f7b80a01640 1 -- 172.21.15.105:0/1875826209 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7c1a6420 con 0x7f7b7c102ae0 2024-03-26T18:01:05.947 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.961+0000 7f7b737fe640 1 -- 172.21.15.105:0/1875826209 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b640349e0 con 0x7f7b7c102ae0 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b835e0740 1 Processor -- start 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b835e0740 1 -- start start 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7c8f0 0x55612ec7cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec81de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 0x55612ec848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b835e0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55612ebe1980 con 0x55612ec7d2e0 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b835e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55612eb89be0 con 0x55612ec7c8f0 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b835e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55612ebdd930 con 0x55612ec82450 2024-03-26T18:01:05.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b7ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7c8f0 0x55612ec7cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b7a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec81de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b7ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7c8f0 0x55612ec7cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34224/0 (socket says 172.21.15.105:34224) 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b7ad9d640 1 -- 172.21.15.105:0/356306587 learned_addr learned my addr 172.21.15.105:0/356306587 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b7a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec81de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57340/0 (socket says 172.21.15.105:57340) 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.964+0000 7f7b737fe640 1 -- 172.21.15.105:0/1875826209 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7b64044020 con 0x7f7b7c102ae0 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b7a59c640 1 -- 172.21.15.105:0/356306587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 msgr2=0x55612ec848e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/356306587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 0x55612ec848e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b7a59c640 1 -- 172.21.15.105:0/356306587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7c8f0 msgr2=0x55612ec7cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/356306587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7c8f0 0x55612ec7cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b7a59c640 1 -- 172.21.15.105:0/356306587 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612e9bf7d0 con 0x55612ec7d2e0 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b7ad9d640 1 --2- 172.21.15.105:0/356306587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7c8f0 0x55612ec7cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/356306587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec81de0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009bf0 tx=0x7f7b6c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef92f33a72f5064a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b70ff9640 1 -- 172.21.15.105:0/356306587 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c009da0 con 0x55612ec7d2e0 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b70ff9640 1 -- 172.21.15.105:0/356306587 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b6c005020 con 0x55612ec7d2e0 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b737fe640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7b680a2f20 0x7f7b680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b7ad9d640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7b680a2f20 0x7f7b680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b737fe640 1 -- 172.21.15.105:0/1875826209 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f7b640f0c80 con 0x7f7b7c102ae0 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.965+0000 7f7b70ff9640 1 -- 172.21.15.105:0/356306587 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c004ad0 con 0x55612ec7d2e0 2024-03-26T18:01:05.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 -- 172.21.15.105:0/356306587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 msgr2=0x55612ec81de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 --2- 172.21.15.105:0/356306587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec81de0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009bf0 tx=0x7f7b6c011810 comp rx=0 tx=0).stop 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b7ad9d640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7b680a2f20 0x7f7b680a53e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55612ec7db40 tx=0x7f7b74009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 -- 172.21.15.105:0/356306587 shutdown_connections 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 --2- 172.21.15.105:0/356306587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 0x55612ec848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 --2- 172.21.15.105:0/356306587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec81de0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 --2- 172.21.15.105:0/356306587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7c8f0 0x55612ec7cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 -- 172.21.15.105:0/356306587 >> 172.21.15.105:0/356306587 conn(0x55612ec77e60 msgr2=0x55612ec78260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 -- 172.21.15.105:0/356306587 shutdown_connections 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 -- 172.21.15.105:0/356306587 wait complete. 2024-03-26T18:01:05.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 -- start start 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.966+0000 7f7b835e0740 1 Processor -- start 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec86ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec870e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec82450 0x55612ec8e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b835e0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55612ec854a0 con 0x55612ec7d2e0 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b835e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55612eb9d160 con 0x55612ec82450 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b835e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55612ec85620 con 0x55612ec7c8f0 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b835e0740 10 client.0 _ll_register_callbacks cb 0x55612ebd9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b7a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec86ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec870e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b7a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec86ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60932/0 (socket says 172.21.15.105:60932) 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b7a59c640 1 -- 172.21.15.105:0/1534897436 learned_addr learned my addr 172.21.15.105:0/1534897436 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec870e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57350/0 (socket says 172.21.15.105:57350) 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b7ad9d640 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec82450 0x55612ec8e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b73fff640 1 -- 172.21.15.105:0/1534897436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 msgr2=0x55612ec86ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b73fff640 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec86ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b73fff640 1 -- 172.21.15.105:0/1534897436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec82450 msgr2=0x55612ec8e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b73fff640 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec82450 0x55612ec8e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.967+0000 7f7b73fff640 1 -- 172.21.15.105:0/1534897436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612e8f53f0 con 0x55612ec7d2e0 2024-03-26T18:01:05.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b73fff640 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec870e0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f7b640eeab0 tx=0x7f7b640b96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1d652edf46ff30 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b61ffb640 1 -- 172.21.15.105:0/1534897436 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b640d6060 con 0x55612ec7d2e0 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b61ffb640 1 -- 172.21.15.105:0/1534897436 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f7b6402cc50 con 0x55612ec7d2e0 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 1 -- 172.21.15.105:0/1534897436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c005020 con 0x55612ec7d2e0 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b61ffb640 1 -- 172.21.15.105:0/1534897436 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7b640bda70 con 0x55612ec7d2e0 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 10 client.23688 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 1 -- 172.21.15.105:0/1534897436 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55612e8b50f0 con 0x55612ec7d2e0 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b61ffb640 1 -- 172.21.15.105:0/1534897436 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b6c005020 con 0x55612ec7d2e0 2024-03-26T18:01:05.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b61ffb640 1 -- 172.21.15.105:0/1534897436 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7b640d6200 con 0x55612ec7d2e0 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 10 client.23688 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 10 client.23688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 20 client.23688 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55612ec92920 0x55612ec94d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 4 client.23688 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.968+0000 7f7b835e0740 1 -- 172.21.15.105:0/1534897436 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55612ebeeec0 con 0x55612ec92920 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.969+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55612ec92920 0x55612ec94d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.969+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55612ec92920 0x55612ec94d50 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e783950d4202a9 server_cookie=dc4a7331c67bd84 in_seq=0 out_seq=0 2024-03-26T18:01:05.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.969+0000 7f7b61ffb640 10 client.23688 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:05.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b61ffb640 1 -- 172.21.15.105:0/1534897436 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55612ebeeec0 con 0x55612ec92920 2024-03-26T18:01:05.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b61ffb640 10 client.23688 handle_command_reply: tid=0 2024-03-26T18:01:05.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 client.23688 shutdown 2024-03-26T18:01:05.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 -- 172.21.15.105:0/1534897436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 msgr2=0x55612ec870e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec870e0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f7b640eeab0 tx=0x7f7b640b96d0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 -- 172.21.15.105:0/1534897436 shutdown_connections 2024-03-26T18:01:05.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55612ec92920 0x55612ec94d50 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec82450 0x55612ec8e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7d2e0 0x55612ec870e0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b61ffb640 0 client.23688 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:05.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 --2- 172.21.15.105:0/1534897436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec86ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 -- 172.21.15.105:0/1534897436 >> 172.21.15.105:0/1534897436 conn(0x55612ec77e60 msgr2=0x55612ec77080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 -- 172.21.15.105:0/1534897436 shutdown_connections 2024-03-26T18:01:05.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 1 -- 172.21.15.105:0/1534897436 wait complete. 2024-03-26T18:01:05.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:05.973+0000 7f7b835e0740 20 client.23688 trim_cache size 0 max 16384 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.000+0000 7f7b835e0740 1 Processor -- start 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.000+0000 7f7b835e0740 1 -- start start 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec78240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec82450 0x55612ec767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b835e0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1af3f0 con 0x55612ec82450 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b835e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b0f30 con 0x55612ec7d2e0 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b835e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1afac0 con 0x55612ec7c8f0 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec78240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7a59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec78240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:60938/0 (socket says 172.21.15.105:60938) 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7a59c640 1 -- 172.21.15.105:0/2477980849 learned_addr learned my addr 172.21.15.105:0/2477980849 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b73fff640 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7ad9d640 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec82450 0x55612ec767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7a59c640 1 -- 172.21.15.105:0/2477980849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 msgr2=0x55612ec8e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7a59c640 1 -- 172.21.15.105:0/2477980849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec82450 msgr2=0x55612ec767f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec82450 0x55612ec767f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7a59c640 1 -- 172.21.15.105:0/2477980849 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612ec81780 con 0x55612ec7c8f0 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.001+0000 7f7b7ad9d640 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec82450 0x55612ec767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.002+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec78240 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009230 tx=0x7f7b6c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e10f54c9e9407 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.002+0000 7f7b63fff640 1 -- 172.21.15.105:0/2477980849 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c005020 con 0x55612ec7c8f0 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.002+0000 7f7b63fff640 1 -- 172.21.15.105:0/2477980849 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b6c005430 con 0x55612ec7c8f0 2024-03-26T18:01:05.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.002+0000 7f7b63fff640 1 -- 172.21.15.105:0/2477980849 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c003850 con 0x55612ec7c8f0 2024-03-26T18:01:05.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 -- 172.21.15.105:0/2477980849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 msgr2=0x55612ec78240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec78240 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009230 tx=0x7f7b6c0096a0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 -- 172.21.15.105:0/2477980849 shutdown_connections 2024-03-26T18:01:05.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec82450 0x55612ec767f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 --2- 172.21.15.105:0/2477980849 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec7c8f0 0x55612ec78240 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 -- 172.21.15.105:0/2477980849 >> 172.21.15.105:0/2477980849 conn(0x55612ec77e60 msgr2=0x55612ec7e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 -- 172.21.15.105:0/2477980849 shutdown_connections 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 -- 172.21.15.105:0/2477980849 wait complete. 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 -- start start 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.003+0000 7f7b835e0740 1 Processor -- start 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7c8f0 0x55612ec8b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b835e0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 0x55612ec8cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b835e0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b0860 con 0x55612ec7c8f0 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b835e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1b0190 con 0x55612ec7d2e0 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b835e0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c1ad3d0 con 0x55612ec82450 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b835e0740 10 client.0 _ll_register_callbacks cb 0x7f7b64009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:05.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7c8f0 0x55612ec8b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7ad9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 0x55612ec8cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7c8f0 0x55612ec8b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57370/0 (socket says 172.21.15.105:57370) 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7a59c640 1 -- 172.21.15.105:0/527666287 learned_addr learned my addr 172.21.15.105:0/527666287 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b73fff640 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7a59c640 1 -- 172.21.15.105:0/527666287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 msgr2=0x55612ec8cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 0x55612ec8cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7a59c640 1 -- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 msgr2=0x55612ec8bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b7a59c640 1 -- 172.21.15.105:0/527666287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612ec8d260 con 0x55612ec7c8f0 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.004+0000 7f7b73fff640 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b7a59c640 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7c8f0 0x55612ec8b720 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009b40 tx=0x7f7b6c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5d8a53fad4b0b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b61ffb640 1 -- 172.21.15.105:0/527666287 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c02b860 con 0x55612ec7c8f0 2024-03-26T18:01:05.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b61ffb640 1 -- 172.21.15.105:0/527666287 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f7b6c02ba00 con 0x55612ec7c8f0 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b835e0740 1 -- 172.21.15.105:0/527666287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c005430 con 0x55612ec7c8f0 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b61ffb640 1 -- 172.21.15.105:0/527666287 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c02bc00 con 0x55612ec7c8f0 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b835e0740 10 client.23694 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b835e0740 1 -- 172.21.15.105:0/527666287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55612ecf8c90 con 0x55612ec7c8f0 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.005+0000 7f7b61ffb640 1 -- 172.21.15.105:0/527666287 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b6c005430 con 0x55612ec7c8f0 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b61ffb640 1 -- 172.21.15.105:0/527666287 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7b6c02b070 con 0x55612ec7c8f0 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b835e0740 10 client.23694 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b835e0740 10 client.23694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b835e0740 20 client.23694 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b835e0740 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55612ecf9440 0x55612ecfb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b835e0740 4 client.23694 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b835e0740 1 -- 172.21.15.105:0/527666287 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55612ebe9210 con 0x55612ecf9440 2024-03-26T18:01:05.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b73fff640 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55612ecf9440 0x55612ecfb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:05.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b61ffb640 10 client.23694 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:05.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.006+0000 7f7b73fff640 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55612ecf9440 0x55612ecfb860 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4328a85f37ea93f2 server_cookie=2c4013cb94a78c03 in_seq=0 out_seq=0 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.009+0000 7f7b61ffb640 1 -- 172.21.15.105:0/527666287 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55612ebe9210 con 0x55612ecf9440 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.009+0000 7f7b61ffb640 10 client.23694 handle_command_reply: tid=0 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.009+0000 7f7b835e0740 1 client.23694 shutdown 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.009+0000 7f7b835e0740 1 -- 172.21.15.105:0/527666287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7c8f0 msgr2=0x55612ec8b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.009+0000 7f7b835e0740 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7c8f0 0x55612ec8b720 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009b40 tx=0x7f7b6c0050b0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 1 -- 172.21.15.105:0/527666287 shutdown_connections 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55612ecf9440 0x55612ecfb860 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55612ec82450 0x55612ec8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b61ffb640 0 client.23694 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55612ec7d2e0 0x55612ec8bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 1 --2- 172.21.15.105:0/527666287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55612ec7c8f0 0x55612ec8b720 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 1 -- 172.21.15.105:0/527666287 >> 172.21.15.105:0/527666287 conn(0x55612ec77e60 msgr2=0x55612ec7b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 1 -- 172.21.15.105:0/527666287 shutdown_connections 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 1 -- 172.21.15.105:0/527666287 wait complete. 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.010+0000 7f7b835e0740 20 client.23694 trim_cache size 0 max 16384 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (456 inodes in the stack)", 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "d4fc86d9-68e5-4809-bd18-98097c8c0d77": { 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "d4fc86d9-68e5-4809-bd18-98097c8c0d77", 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T18:01:05.994 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:01:05.995 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:01:05.995 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:01:05.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 -- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7b680a2f20 msgr2=0x7f7b680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7b680a2f20 0x7f7b680a53e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55612ec7db40 tx=0x7f7b74009500 comp rx=0 tx=0).stop 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 -- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 msgr2=0x7f7b7c19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 0x7f7b7c19e650 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f7b64006010 tx=0x7f7b640052b0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 -- 172.21.15.105:0/1875826209 shutdown_connections 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7b680a2f20 0x7f7b680a53e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c1034a0 0x7f7b7c1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b7c102ae0 0x7f7b7c19e650 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 --2- 172.21.15.105:0/1875826209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b7c1018f0 0x7f7b7c19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 -- 172.21.15.105:0/1875826209 >> 172.21.15.105:0/1875826209 conn(0x7f7b7c0fce60 msgr2=0x7f7b7c10ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 -- 172.21.15.105:0/1875826209 shutdown_connections 2024-03-26T18:01:05.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:06.011+0000 7f7b80a01640 1 -- 172.21.15.105:0/1875826209 wait complete. 2024-03-26T18:01:06.006 INFO:tasks.cephfs.filesystem:scrub status for tag:d4fc86d9-68e5-4809-bd18-98097c8c0d77 - {'path': '/', 'tag': 'd4fc86d9-68e5-4809-bd18-98097c8c0d77', 'options': 'recursive,force'} 2024-03-26T18:01:06.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:05 smithi144 ceph-mon[36095]: pgmap v1034: 97 pgs: 97 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T18:01:06.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:05 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T18:01:06.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:05 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:06.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:05 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:06.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:05 smithi105 ceph-mon[32280]: pgmap v1034: 97 pgs: 97 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T18:01:06.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:05 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T18:01:06.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:05 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:06.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:05 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:06.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:05 smithi135 ceph-mon[35636]: pgmap v1034: 97 pgs: 97 active+clean; 363 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-03-26T18:01:06.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:05 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T18:01:06.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:05 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:06.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:05 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:06.405 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:01:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T18:01:05.938+0000 7f4640dfd700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000003c80 [...2,head] /volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL/ auth{1=1,2=1,3=1,4=1} v15340 ap=2 RANDEPHEMERALPIN f(v416 m2024-03-26T18:00:49.903336+0000 11=11+0) n(v32 rc2024-03-26T18:00:49.903336+0000 b2861056 12=11+1) (inest sync->mix g=1,2,3,4) caps={15330=pAsLsXsFs/-@8067} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55ef2d142580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000003c80:[<0x10000003bc0/COREL v50>,<0x10000003bb6/~dmtmp v48>,<0x10000000007/client8 v2078>,<0x10000000006/clients v1695>,<0x10000000005/tmp v1657>,<0x10000000003/client.0 v1604>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v1575>,<0x10000000001/sv_0 v1586>,<0x10000000000/qa v1535>,<0x1/volumes v1503>]//[]","memoryvalue":"(2)0x10000003c80:[<0x10000003bc0/COREL v15340>,<0x10000003bb6/~dmtmp v3150>,<0x10000000007/client8 v12592>,<0x10000000006/clients v3252>,<0x10000000005/tmp v2919>,<0x10000000003/client.0 v2809>,<0x10000000002/c5a24205-8212-466d-b5ed-c6d2b325c0fb v2738>,<0x10000000001/sv_0 v2749>,<0x10000000000/qa v2667>,<0x1/volumes v2630>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-03-26T18:00:49.878164+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-03-26T18:00:49.903336+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v416 m2024-03-26T18:00:49.903336+0000 11=11+0)","memory_value.rstat":"n(v32 rc2024-03-26T18:00:49.903336+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-03-26T18:01:06.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77143 7.06 MB/sec execute 469 sec latency 249.865 ms 2024-03-26T18:01:07.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:07.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:07.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:07.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T18:01:07.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:07.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:07.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:06 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:06 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:06 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:07.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77270 7.06 MB/sec execute 470 sec latency 161.074 ms 2024-03-26T18:01:08.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:07 smithi144 ceph-mon[36095]: pgmap v1035: 97 pgs: 97 active+clean; 365 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-03-26T18:01:08.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:07 smithi105 ceph-mon[32280]: pgmap v1035: 97 pgs: 97 active+clean; 365 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-03-26T18:01:08.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:07 smithi135 ceph-mon[35636]: pgmap v1035: 97 pgs: 97 active+clean; 365 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 3.9 MiB/s wr, 237 op/s 2024-03-26T18:01:08.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77379 7.06 MB/sec execute 471 sec latency 139.688 ms 2024-03-26T18:01:09.305 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:09 smithi135 ceph-mon[35636]: pgmap v1036: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-03-26T18:01:09.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:08 smithi144 ceph-mon[36095]: pgmap v1036: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-03-26T18:01:09.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:08 smithi105 ceph-mon[32280]: pgmap v1036: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 4.1 MiB/s wr, 267 op/s 2024-03-26T18:01:09.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77489 7.06 MB/sec execute 472 sec latency 147.975 ms 2024-03-26T18:01:10.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77629 7.07 MB/sec execute 473 sec latency 150.824 ms 2024-03-26T18:01:11.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:10 smithi144 ceph-mon[36095]: pgmap v1037: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-03-26T18:01:11.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:10 smithi105 ceph-mon[32280]: pgmap v1037: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-03-26T18:01:11.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:11 smithi135 ceph-mon[35636]: pgmap v1037: 97 pgs: 97 active+clean; 366 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 2.9 MiB/s wr, 235 op/s 2024-03-26T18:01:11.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77719 7.06 MB/sec execute 474 sec latency 138.509 ms 2024-03-26T18:01:12.559 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77863 7.06 MB/sec execute 475 sec latency 136.724 ms 2024-03-26T18:01:13.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:12 smithi144 ceph-mon[36095]: pgmap v1038: 97 pgs: 97 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:01:13.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:12 smithi105 ceph-mon[32280]: pgmap v1038: 97 pgs: 97 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:01:13.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:12 smithi135 ceph-mon[35636]: pgmap v1038: 97 pgs: 97 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 3.5 MiB/s wr, 266 op/s 2024-03-26T18:01:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.443+0000 7f88b373b640 1 -- 172.21.15.105:0/2791630114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac102c30 msgr2=0x7f88ac1030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.443+0000 7f88b373b640 1 --2- 172.21.15.105:0/2791630114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac102c30 0x7f88ac1030b0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f889c009890 tx=0x7f889c02bb70 comp rx=0 tx=0).stop 2024-03-26T18:01:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.444+0000 7f88b373b640 1 -- 172.21.15.105:0/2791630114 shutdown_connections 2024-03-26T18:01:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.444+0000 7f88b373b640 1 --2- 172.21.15.105:0/2791630114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88ac1035f0 0x7f88ac109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.444+0000 7f88b373b640 1 --2- 172.21.15.105:0/2791630114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac102c30 0x7f88ac1030b0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.444+0000 7f88b373b640 1 --2- 172.21.15.105:0/2791630114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88ac0ffbc0 0x7f88ac0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.444+0000 7f88b373b640 1 -- 172.21.15.105:0/2791630114 >> 172.21.15.105:0/2791630114 conn(0x7f88ac0fb510 msgr2=0x7f88ac0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.444+0000 7f88b373b640 1 -- 172.21.15.105:0/2791630114 shutdown_connections 2024-03-26T18:01:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.444+0000 7f88b373b640 1 -- 172.21.15.105:0/2791630114 wait complete. 2024-03-26T18:01:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.444+0000 7f88b373b640 1 Processor -- start 2024-03-26T18:01:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b373b640 1 -- start start 2024-03-26T18:01:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b373b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac0ffbc0 0x7f88ac199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b373b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88ac102c30 0x7f88ac19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b373b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88ac1035f0 0x7f88ac1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b373b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac101460 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b373b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1012e0 con 0x7f88ac1035f0 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b373b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1015e0 con 0x7f88ac102c30 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac0ffbc0 0x7f88ac199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac0ffbc0 0x7f88ac199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57392/0 (socket says 172.21.15.105:57392) 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 -- 172.21.15.105:0/3063747907 learned_addr learned my addr 172.21.15.105:0/3063747907 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b1cd8640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88ac1035f0 0x7f88ac1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88ac102c30 0x7f88ac19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 -- 172.21.15.105:0/3063747907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88ac102c30 msgr2=0x7f88ac19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88ac102c30 0x7f88ac19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 -- 172.21.15.105:0/3063747907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88ac1035f0 msgr2=0x7f88ac1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88ac1035f0 0x7f88ac1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 -- 172.21.15.105:0/3063747907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac1a1910 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.445+0000 7f88b14d7640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac0ffbc0 0x7f88ac199d50 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f88a800e9a0 tx=0x7f88a800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.446+0000 7f88a27fc640 1 -- 172.21.15.105:0/3063747907 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a800cd80 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.446+0000 7f88a27fc640 1 -- 172.21.15.105:0/3063747907 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a80108e0 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.446+0000 7f88b373b640 1 -- 172.21.15.105:0/3063747907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ac1a1c00 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.446+0000 7f88a27fc640 1 -- 172.21.15.105:0/3063747907 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8018950 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.446+0000 7f88b373b640 1 -- 172.21.15.105:0/3063747907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ac1a21a0 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.449+0000 7f88a27fc640 1 -- 172.21.15.105:0/3063747907 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f88a8010a90 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b640a740 1 Processor -- start 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b640a740 1 -- start start 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88a27fc640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88980a3040 0x7f88980a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b640a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452d5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b640a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b640a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b640a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564c452310b0 con 0x564c452d4de0 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b640a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564c451d9960 con 0x564c452da940 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b640a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564c4522d060 con 0x564c452d57d0 2024-03-26T18:01:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88b1cd8640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88980a3040 0x7f88980a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.450+0000 7f88a27fc640 1 -- 172.21.15.105:0/3063747907 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f88a80cf780 con 0x7f88ac0ffbc0 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452d5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452d5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57406/0 (socket says 172.21.15.105:57406) 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 -- 172.21.15.105:0/2622955617 learned_addr learned my addr 172.21.15.105:0/2622955617 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b1cd8640 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 -- 172.21.15.105:0/2622955617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 msgr2=0x564c452da2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452da2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 -- 172.21.15.105:0/2622955617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 msgr2=0x564c452dcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452dcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 -- 172.21.15.105:0/2622955617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c451ebc60 con 0x564c452d4de0 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b1cd8640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88980a3040 0x7f88980a5500 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005fd0 tx=0x7f88a4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88b14d7640 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452d5200 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f88a80987b0 tx=0x7f88a80981b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da94b54b47ca5a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.451+0000 7f88977fe640 1 -- 172.21.15.105:0/2622955617 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8019820 con 0x564c452d4de0 2024-03-26T18:01:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.452+0000 7f88977fe640 1 -- 172.21.15.105:0/2622955617 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a8010e20 con 0x564c452d4de0 2024-03-26T18:01:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.452+0000 7f88977fe640 1 -- 172.21.15.105:0/2622955617 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8019be0 con 0x564c452d4de0 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.452+0000 7f88b640a740 1 -- 172.21.15.105:0/2622955617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 msgr2=0x564c452d5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.452+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452d5200 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f88a80987b0 tx=0x7f88a80981b0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 -- 172.21.15.105:0/2622955617 shutdown_connections 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622955617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452d5200 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 -- 172.21.15.105:0/2622955617 >> 172.21.15.105:0/2622955617 conn(0x564c452c7b40 msgr2=0x564c452c7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 -- 172.21.15.105:0/2622955617 shutdown_connections 2024-03-26T18:01:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 -- 172.21.15.105:0/2622955617 wait complete. 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 -- start start 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.453+0000 7f88b640a740 1 Processor -- start 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b640a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452deba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b640a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452d57d0 0x564c452df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b640a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452da940 0x564c452e6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b640a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564c452dd8f0 con 0x564c452d4de0 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b640a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564c451ecdb0 con 0x564c452d57d0 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b640a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564c452dda70 con 0x564c452da940 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b640a740 10 client.0 _ll_register_callbacks cb 0x564c451f1110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b0cd6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452d57d0 0x564c452df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452deba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57408/0 (socket says 172.21.15.105:57408) 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 -- 172.21.15.105:0/1187397925 learned_addr learned my addr 172.21.15.105:0/1187397925 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b0cd6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452d57d0 0x564c452df0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34294/0 (socket says 172.21.15.105:34294) 2024-03-26T18:01:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b1cd8640 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452da940 0x564c452e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 -- 172.21.15.105:0/1187397925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452da940 msgr2=0x564c452e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452da940 0x564c452e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 -- 172.21.15.105:0/1187397925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452d57d0 msgr2=0x564c452df0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452d57d0 0x564c452df0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 -- 172.21.15.105:0/1187397925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c44f45210 con 0x564c452d4de0 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f88b14d7640 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452deba0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f88a8019610 tx=0x7f88a800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae948e767be4925 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.454+0000 7f8894ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f8894ff9640 1 -- 172.21.15.105:0/1187397925 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8019640 con 0x564c452d4de0 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f8894ff9640 1 -- 172.21.15.105:0/1187397925 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f88a80047c0 con 0x564c452d4de0 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f88b640a740 1 -- 172.21.15.105:0/1187397925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564c4523e5f0 con 0x564c452d4de0 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f8894ff9640 1 -- 172.21.15.105:0/1187397925 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f88a809ffc0 con 0x564c452d4de0 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f88b640a740 10 client.23706 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f88b640a740 1 -- 172.21.15.105:0/1187397925 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564c45011620 con 0x564c452d4de0 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f8894ff9640 1 -- 172.21.15.105:0/1187397925 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564c4523e5f0 con 0x564c452d4de0 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f8894ff9640 1 -- 172.21.15.105:0/1187397925 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f88a8012c20 con 0x564c452d4de0 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f88b640a740 10 client.23706 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f88b640a740 10 client.23706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.455+0000 7f88b640a740 20 client.23706 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.456+0000 7f88b640a740 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564c452ea890 0x564c452ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.456+0000 7f88b640a740 4 client.23706 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.456+0000 7f88b640a740 1 -- 172.21.15.105:0/1187397925 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c45238940 con 0x564c452ea890 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.456+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564c452ea890 0x564c452ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.456+0000 7f8894ff9640 10 client.23706 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.456+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564c452ea890 0x564c452ecca0 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f60f92f91a173d server_cookie=83d3c8f37b0f870 in_seq=0 out_seq=0 2024-03-26T18:01:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.458+0000 7f8894ff9640 1 -- 172.21.15.105:0/1187397925 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564c45238940 con 0x564c452ea890 2024-03-26T18:01:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.458+0000 7f8894ff9640 10 client.23706 handle_command_reply: tid=0 2024-03-26T18:01:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.458+0000 7f88b640a740 1 client.23706 shutdown 2024-03-26T18:01:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 -- 172.21.15.105:0/1187397925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 msgr2=0x564c452deba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452deba0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f88a8019610 tx=0x7f88a800cf20 comp rx=0 tx=0).stop 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 -- 172.21.15.105:0/1187397925 shutdown_connections 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564c452ea890 0x564c452ecca0 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452da940 0x564c452e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452d57d0 0x564c452df0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 --2- 172.21.15.105:0/1187397925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d4de0 0x564c452deba0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 -- 172.21.15.105:0/1187397925 >> 172.21.15.105:0/1187397925 conn(0x564c452c7b40 msgr2=0x564c452c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f8894ff9640 0 client.23706 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 -- 172.21.15.105:0/1187397925 shutdown_connections 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 1 -- 172.21.15.105:0/1187397925 wait complete. 2024-03-26T18:01:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.459+0000 7f88b640a740 20 client.23706 trim_cache size 0 max 16384 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b640a740 1 Processor -- start 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b640a740 1 -- start start 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b640a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d57d0 0x564c452d6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b640a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b640a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452de7b0 0x564c452dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b640a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1a6b00 con 0x564c452d57d0 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b640a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1a8640 con 0x564c452da940 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b640a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1a71d0 con 0x564c452de7b0 2024-03-26T18:01:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b14d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d57d0 0x564c452d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b1cd8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452de7b0 0x564c452dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b14d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d57d0 0x564c452d6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57420/0 (socket says 172.21.15.105:57420) 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b14d7640 1 -- 172.21.15.105:0/771502975 learned_addr learned my addr 172.21.15.105:0/771502975 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.480+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f88b14d7640 1 -- 172.21.15.105:0/771502975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452de7b0 msgr2=0x564c452dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f88b14d7640 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452de7b0 0x564c452dec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f88b14d7640 1 -- 172.21.15.105:0/771502975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 msgr2=0x564c452e60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f88b14d7640 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f88b14d7640 1 -- 172.21.15.105:0/771502975 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c452d9850 con 0x564c452d57d0 2024-03-26T18:01:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f88b14d7640 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d57d0 0x564c452d6860 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f88a8012f90 tx=0x7f88a809bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb02237006c6a07 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f8896ffd640 1 -- 172.21.15.105:0/771502975 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8019640 con 0x564c452d57d0 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f8896ffd640 1 -- 172.21.15.105:0/771502975 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a80a4880 con 0x564c452d57d0 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.481+0000 7f8896ffd640 1 -- 172.21.15.105:0/771502975 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a809a170 con 0x564c452d57d0 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 -- 172.21.15.105:0/771502975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d57d0 msgr2=0x564c452d6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d57d0 0x564c452d6860 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f88a8012f90 tx=0x7f88a809bae0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 -- 172.21.15.105:0/771502975 shutdown_connections 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452de7b0 0x564c452dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 --2- 172.21.15.105:0/771502975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452d57d0 0x564c452d6860 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 -- 172.21.15.105:0/771502975 >> 172.21.15.105:0/771502975 conn(0x564c452c7b40 msgr2=0x564c452d6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 -- 172.21.15.105:0/771502975 shutdown_connections 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.482+0000 7f88b640a740 1 -- 172.21.15.105:0/771502975 wait complete. 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 1 -- start start 2024-03-26T18:01:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 1 Processor -- start 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452e3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452de7b0 0x564c4534ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1a7f70 con 0x564c452de7b0 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1a78a0 con 0x564c452da940 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1a4b60 con 0x564c452d57d0 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b1cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452de7b0 0x564c4534ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b0cd6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b1cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452de7b0 0x564c4534ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:57430/0 (socket says 172.21.15.105:57430) 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b0cd6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:34314/0 (socket says 172.21.15.105:34314) 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b640a740 10 client.0 _ll_register_callbacks cb 0x7f88a8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.483+0000 7f88b1cd8640 1 -- 172.21.15.105:0/2622668215 learned_addr learned my addr 172.21.15.105:0/2622668215 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b14d7640 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b0cd6640 1 -- 172.21.15.105:0/2622668215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 msgr2=0x564c452e3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452e3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b0cd6640 1 -- 172.21.15.105:0/2622668215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452de7b0 msgr2=0x564c4534ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452de7b0 0x564c4534ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b0cd6640 1 -- 172.21.15.105:0/2622668215 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4534f110 con 0x564c452da940 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b14d7640 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452e3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b0cd6640 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e35c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f889c02c180 tx=0x7f889c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb682bf07a9c9909 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f8894ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f8894ff9640 1 -- 172.21.15.105:0/2622668215 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c039040 con 0x564c452da940 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f8894ff9640 1 -- 172.21.15.105:0/2622668215 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f889c004040 con 0x564c452da940 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.484+0000 7f88b640a740 1 -- 172.21.15.105:0/2622668215 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a80a4880 con 0x564c452da940 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f8894ff9640 1 -- 172.21.15.105:0/2622668215 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f889c004260 con 0x564c452da940 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b640a740 10 client.33386 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b640a740 1 -- 172.21.15.105:0/2622668215 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564c453504c0 con 0x564c452da940 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f8894ff9640 1 -- 172.21.15.105:0/2622668215 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a80a4880 con 0x564c452da940 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f8894ff9640 1 -- 172.21.15.105:0/2622668215 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f889c003710 con 0x564c452da940 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b640a740 10 client.33386 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b640a740 10 client.33386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b640a740 20 client.33386 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564c45350e20 0x564c45353240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b640a740 4 client.33386 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:01:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b640a740 1 -- 172.21.15.105:0/2622668215 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c451b6770 con 0x564c45350e20 2024-03-26T18:01:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.485+0000 7f88b14d7640 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564c45350e20 0x564c45353240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.486+0000 7f8894ff9640 10 client.33386 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.486+0000 7f88b14d7640 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564c45350e20 0x564c45353240 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b726edf15d06255e server_cookie=6f3e90c92c2e6f74 in_seq=0 out_seq=0 2024-03-26T18:01:13.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 77970 7.05 MB/sec execute 476 sec latency 159.054 ms 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.649+0000 7f8894ff9640 1 -- 172.21.15.105:0/2622668215 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c451b6770 con 0x564c45350e20 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.649+0000 7f8894ff9640 10 client.33386 handle_command_reply: tid=0 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.649+0000 7f88b640a740 1 client.33386 shutdown 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 -- 172.21.15.105:0/2622668215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 msgr2=0x564c452e35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e35c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f889c02c180 tx=0x7f889c000bc0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 -- 172.21.15.105:0/2622668215 shutdown_connections 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564c45350e20 0x564c45353240 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564c452de7b0 0x564c4534ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f8894ff9640 0 client.33386 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c452da940 0x564c452e35c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.632 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 --2- 172.21.15.105:0/2622668215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564c452d57d0 0x564c452e3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 -- 172.21.15.105:0/2622668215 >> 172.21.15.105:0/2622668215 conn(0x564c452c7b40 msgr2=0x564c452d6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 -- 172.21.15.105:0/2622668215 shutdown_connections 2024-03-26T18:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 1 -- 172.21.15.105:0/2622668215 wait complete. 2024-03-26T18:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.650+0000 7f88b640a740 20 client.33386 trim_cache size 0 max 16384 2024-03-26T18:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:01:13.633 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 -- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88980a3040 msgr2=0x7f88980a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88980a3040 0x7f88980a5500 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f88a4005fd0 tx=0x7f88a4009500 comp rx=0 tx=0).stop 2024-03-26T18:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 -- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac0ffbc0 msgr2=0x7f88ac199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac0ffbc0 0x7f88ac199d50 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f88a800e9a0 tx=0x7f88a800ee60 comp rx=0 tx=0).stop 2024-03-26T18:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 -- 172.21.15.105:0/3063747907 shutdown_connections 2024-03-26T18:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88980a3040 0x7f88980a5500 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88ac1035f0 0x7f88ac1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88ac102c30 0x7f88ac19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 --2- 172.21.15.105:0/3063747907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88ac0ffbc0 0x7f88ac199d50 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 -- 172.21.15.105:0/3063747907 >> 172.21.15.105:0/3063747907 conn(0x7f88ac0fb510 msgr2=0x7f88ac0fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 -- 172.21.15.105:0/3063747907 shutdown_connections 2024-03-26T18:01:13.635 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:13.651+0000 7f88b373b640 1 -- 172.21.15.105:0/3063747907 wait complete. 2024-03-26T18:01:14.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 78068 7.04 MB/sec execute 477 sec latency 223.817 ms 2024-03-26T18:01:15.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:14 smithi144 ceph-mon[36095]: pgmap v1039: 97 pgs: 97 active+clean; 369 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-03-26T18:01:15.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:14 smithi105 ceph-mon[32280]: pgmap v1039: 97 pgs: 97 active+clean; 369 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-03-26T18:01:15.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:14 smithi135 ceph-mon[35636]: pgmap v1039: 97 pgs: 97 active+clean; 369 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 3.4 MiB/s wr, 255 op/s 2024-03-26T18:01:15.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 78178 7.03 MB/sec execute 478 sec latency 198.856 ms 2024-03-26T18:01:16.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 78295 7.03 MB/sec execute 479 sec latency 139.452 ms 2024-03-26T18:01:17.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:16 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:01:17.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:16 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:01:17.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:16 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:01:17.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 78437 7.03 MB/sec execute 480 sec latency 145.988 ms 2024-03-26T18:01:17.917 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:17 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:01:17.917 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:17 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:01:17.917 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:17 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:01:18.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:17 smithi144 ceph-mon[36095]: pgmap v1040: 97 pgs: 97 active+clean; 364 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-26T18:01:18.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:17 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:01:18.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:17 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:01:18.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:17 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:01:18.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:17 smithi105 ceph-mon[32280]: pgmap v1040: 97 pgs: 97 active+clean; 364 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-26T18:01:18.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:17 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:01:18.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:17 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:01:18.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:17 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:01:18.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:17 smithi135 ceph-mon[35636]: pgmap v1040: 97 pgs: 97 active+clean; 364 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.3 MiB/s wr, 267 op/s 2024-03-26T18:01:18.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 78576 7.03 MB/sec execute 481 sec latency 131.602 ms 2024-03-26T18:01:19.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 78709 7.04 MB/sec execute 482 sec latency 115.002 ms 2024-03-26T18:01:20.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:19 smithi144 ceph-mon[36095]: pgmap v1041: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 3.4 MiB/s wr, 272 op/s 2024-03-26T18:01:20.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:19 smithi105 ceph-mon[32280]: pgmap v1041: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 3.4 MiB/s wr, 272 op/s 2024-03-26T18:01:20.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:19 smithi135 ceph-mon[35636]: pgmap v1041: 97 pgs: 97 active+clean; 357 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 3.4 MiB/s wr, 272 op/s 2024-03-26T18:01:20.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 78860 7.04 MB/sec execute 483 sec latency 128.004 ms 2024-03-26T18:01:21.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 79013 7.05 MB/sec execute 484 sec latency 129.693 ms 2024-03-26T18:01:22.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:21 smithi144 ceph-mon[36095]: pgmap v1042: 97 pgs: 97 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-03-26T18:01:22.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:21 smithi105 ceph-mon[32280]: pgmap v1042: 97 pgs: 97 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-03-26T18:01:22.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:21 smithi135 ceph-mon[35636]: pgmap v1042: 97 pgs: 97 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-03-26T18:01:22.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 79144 7.05 MB/sec execute 485 sec latency 143.992 ms 2024-03-26T18:01:23.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 79254 7.05 MB/sec execute 486 sec latency 112.536 ms 2024-03-26T18:01:24.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:23 smithi144 ceph-mon[36095]: pgmap v1043: 97 pgs: 97 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-03-26T18:01:24.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:23 smithi105 ceph-mon[32280]: pgmap v1043: 97 pgs: 97 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-03-26T18:01:24.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:23 smithi135 ceph-mon[35636]: pgmap v1043: 97 pgs: 97 active+clean; 359 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-03-26T18:01:24.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 79383 7.04 MB/sec execute 487 sec latency 130.912 ms 2024-03-26T18:01:25.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 79506 7.03 MB/sec execute 488 sec latency 119.218 ms 2024-03-26T18:01:26.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:25 smithi144 ceph-mon[36095]: pgmap v1044: 97 pgs: 97 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-03-26T18:01:26.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:25 smithi105 ceph-mon[32280]: pgmap v1044: 97 pgs: 97 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-03-26T18:01:26.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:25 smithi135 ceph-mon[35636]: pgmap v1044: 97 pgs: 97 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 3.4 MiB/s wr, 274 op/s 2024-03-26T18:01:26.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 79653 7.02 MB/sec execute 489 sec latency 117.536 ms 2024-03-26T18:01:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 -- 172.21.15.105:0/3657919110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88381007e0 msgr2=0x7f8838100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 --2- 172.21.15.105:0/3657919110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88381007e0 0x7f8838100c00 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f8834009900 tx=0x7f883402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 -- 172.21.15.105:0/3657919110 shutdown_connections 2024-03-26T18:01:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 --2- 172.21.15.105:0/3657919110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8838102390 0x7f8838108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 --2- 172.21.15.105:0/3657919110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88381019d0 0x7f8838101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 --2- 172.21.15.105:0/3657919110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88381007e0 0x7f8838100c00 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 -- 172.21.15.105:0/3657919110 >> 172.21.15.105:0/3657919110 conn(0x7f88380fbd50 msgr2=0x7f88380fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 -- 172.21.15.105:0/3657919110 shutdown_connections 2024-03-26T18:01:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.771+0000 7f883fedc640 1 -- 172.21.15.105:0/3657919110 wait complete. 2024-03-26T18:01:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883fedc640 1 Processor -- start 2024-03-26T18:01:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883fedc640 1 -- start start 2024-03-26T18:01:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883fedc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88381007e0 0x7f883819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883fedc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88381019d0 0x7f883819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883fedc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8838102390 0x7f88381a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.755 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883fedc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f883810b220 con 0x7f8838102390 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883fedc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f883810b0a0 con 0x7f88381019d0 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883fedc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f883810b3a0 con 0x7f88381007e0 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883dc78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88381007e0 0x7f883819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883dc78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88381007e0 0x7f883819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34234/0 (socket says 172.21.15.105:34234) 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883dc78640 1 -- 172.21.15.105:0/828245646 learned_addr learned my addr 172.21.15.105:0/828245646 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883e479640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8838102390 0x7f88381a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.772+0000 7f883d477640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88381019d0 0x7f883819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883dc78640 1 -- 172.21.15.105:0/828245646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88381019d0 msgr2=0x7f883819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883dc78640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88381019d0 0x7f883819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883dc78640 1 -- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8838102390 msgr2=0x7f88381a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883dc78640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8838102390 0x7f88381a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883dc78640 1 -- 172.21.15.105:0/828245646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381a5cd0 con 0x7f88381007e0 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883e479640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8838102390 0x7f88381a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883dc78640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88381007e0 0x7f883819e470 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8834000c00 tx=0x7f88340040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f882effd640 1 -- 172.21.15.105:0/828245646 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834043020 con 0x7f88381007e0 2024-03-26T18:01:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883fedc640 1 -- 172.21.15.105:0/828245646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381a5f60 con 0x7f88381007e0 2024-03-26T18:01:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f882effd640 1 -- 172.21.15.105:0/828245646 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88340044c0 con 0x7f88381007e0 2024-03-26T18:01:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f882effd640 1 -- 172.21.15.105:0/828245646 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834003c60 con 0x7f88381007e0 2024-03-26T18:01:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.773+0000 7f883fedc640 1 -- 172.21.15.105:0/828245646 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a63c0 con 0x7f88381007e0 2024-03-26T18:01:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.777+0000 7f882effd640 1 -- 172.21.15.105:0/828245646 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8834046020 con 0x7f88381007e0 2024-03-26T18:01:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.777+0000 7f8842ba1740 1 Processor -- start 2024-03-26T18:01:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.777+0000 7f8842ba1740 1 -- start start 2024-03-26T18:01:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.777+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a45d0 0x5610f09a49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.777+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a4fc0 0x5610f09a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09aa130 0x5610f09ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f882effd640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88240a2fc0 0x7f88240a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f8842ba1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5610f09090b0 con 0x5610f09aa130 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f8842ba1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610f08b1960 con 0x5610f09a4fc0 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f8842ba1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610f0905060 con 0x5610f09a45d0 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883d477640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09aa130 0x5610f09ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883dc78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a4fc0 0x5610f09a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883d477640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09aa130 0x5610f09ac5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36342/0 (socket says 172.21.15.105:36342) 2024-03-26T18:01:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883d477640 1 -- 172.21.15.105:0/2321118123 learned_addr learned my addr 172.21.15.105:0/2321118123 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883e479640 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a45d0 0x5610f09a49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f882effd640 1 -- 172.21.15.105:0/828245646 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f88340f1690 con 0x7f88381007e0 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883e479640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88240a2fc0 0x7f88240a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883d477640 1 -- 172.21.15.105:0/2321118123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a45d0 msgr2=0x5610f09a49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883d477640 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a45d0 0x5610f09a49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883d477640 1 -- 172.21.15.105:0/2321118123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a4fc0 msgr2=0x5610f09a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883d477640 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a4fc0 0x5610f09a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883d477640 1 -- 172.21.15.105:0/2321118123 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f08c3c60 con 0x5610f09aa130 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.778+0000 7f883dc78640 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a4fc0 0x5610f09a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.779+0000 7f883d477640 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09aa130 0x5610f09ac5c0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f882800bc80 tx=0x7f882800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217d2352373fd15a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.779+0000 7f883e479640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88240a2fc0 0x7f88240a5480 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x5610f09a5820 tx=0x7f8830004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.779+0000 7f8823fff640 1 -- 172.21.15.105:0/2321118123 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828015070 con 0x5610f09aa130 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.779+0000 7f8823fff640 1 -- 172.21.15.105:0/2321118123 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f882801a910 con 0x5610f09aa130 2024-03-26T18:01:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.779+0000 7f8823fff640 1 -- 172.21.15.105:0/2321118123 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280118d0 con 0x5610f09aa130 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 -- 172.21.15.105:0/2321118123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09aa130 msgr2=0x5610f09ac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09aa130 0x5610f09ac5c0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f882800bc80 tx=0x7f882800f730 comp rx=0 tx=0).stop 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 -- 172.21.15.105:0/2321118123 shutdown_connections 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09aa130 0x5610f09ac5c0 secure :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f882800bc80 tx=0x7f882800f730 comp rx=0 tx=0).stop 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a4fc0 0x5610f09a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2321118123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a45d0 0x5610f09a49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 -- 172.21.15.105:0/2321118123 >> 172.21.15.105:0/2321118123 conn(0x5610f099fb40 msgr2=0x5610f099ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 -- 172.21.15.105:0/2321118123 shutdown_connections 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 -- 172.21.15.105:0/2321118123 wait complete. 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.780+0000 7f8842ba1740 1 -- start start 2024-03-26T18:01:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f8842ba1740 1 Processor -- start 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 0x5610f09b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f8842ba1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5610f09ad0e0 con 0x5610f09a45d0 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f883dc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f8842ba1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610f08c4db0 con 0x5610f09b3b20 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f883dc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36346/0 (socket says 172.21.15.105:36346) 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f8842ba1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610f09ad260 con 0x5610f09a4fc0 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f883dc78640 1 -- 172.21.15.105:0/3968691374 learned_addr learned my addr 172.21.15.105:0/3968691374 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f883d477640 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.781+0000 7f8842ba1740 10 client.0 _ll_register_callbacks cb 0x5610f08c9110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f883e479640 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 0x5610f09b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f883d477640 1 -- 172.21.15.105:0/3968691374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 msgr2=0x5610f09b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f883d477640 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 0x5610f09b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f883d477640 1 -- 172.21.15.105:0/3968691374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 msgr2=0x5610f09ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f883d477640 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f883d477640 1 -- 172.21.15.105:0/3968691374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f061d210 con 0x5610f09a4fc0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f883d477640 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09aeaf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8828013000 tx=0x7f882800b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691f71a9fa428d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f88217fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f88217fa640 1 -- 172.21.15.105:0/3968691374 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828015070 con 0x5610f09a4fc0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f88217fa640 1 -- 172.21.15.105:0/3968691374 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f882801aac0 con 0x5610f09a4fc0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f8842ba1740 1 -- 172.21.15.105:0/3968691374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882801a910 con 0x5610f09a4fc0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f88217fa640 1 -- 172.21.15.105:0/3968691374 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f882800be70 con 0x5610f09a4fc0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f8842ba1740 10 client.33385 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.782+0000 7f8842ba1740 1 -- 172.21.15.105:0/3968691374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5610f06e9620 con 0x5610f09a4fc0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f88217fa640 1 -- 172.21.15.105:0/3968691374 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f882801a910 con 0x5610f09a4fc0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f88217fa640 1 -- 172.21.15.105:0/3968691374 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8828004b40 con 0x5610f09a4fc0 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f8842ba1740 10 client.33385 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f8842ba1740 10 client.33385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f8842ba1740 20 client.33385 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f8842ba1740 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f09ba3e0 0x5610f09bc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f8842ba1740 4 client.33385 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f8842ba1740 1 -- 172.21.15.105:0/3968691374 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610f09165f0 con 0x5610f09ba3e0 2024-03-26T18:01:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.783+0000 7f883dc78640 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f09ba3e0 0x5610f09bc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.784+0000 7f883dc78640 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f09ba3e0 0x5610f09bc7d0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0a33e56983cb01 server_cookie=b77ef22e00a254b5 in_seq=0 out_seq=0 2024-03-26T18:01:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.784+0000 7f88217fa640 10 client.33385 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.785+0000 7f88217fa640 1 -- 172.21.15.105:0/3968691374 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5610f09165f0 con 0x5610f09ba3e0 2024-03-26T18:01:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.785+0000 7f88217fa640 10 client.33385 handle_command_reply: tid=0 2024-03-26T18:01:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 client.33385 shutdown 2024-03-26T18:01:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 -- 172.21.15.105:0/3968691374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 msgr2=0x5610f09aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09aeaf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8828013000 tx=0x7f882800b9f0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 -- 172.21.15.105:0/3968691374 shutdown_connections 2024-03-26T18:01:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f09ba3e0 0x5610f09bc7d0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 0x5610f09b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f88217fa640 0 client.33385 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09aeaf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 --2- 172.21.15.105:0/3968691374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 -- 172.21.15.105:0/3968691374 >> 172.21.15.105:0/3968691374 conn(0x5610f099fb40 msgr2=0x5610f099edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 -- 172.21.15.105:0/3968691374 shutdown_connections 2024-03-26T18:01:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 1 -- 172.21.15.105:0/3968691374 wait complete. 2024-03-26T18:01:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.786+0000 7f8842ba1740 20 client.33385 trim_cache size 0 max 16384 2024-03-26T18:01:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.807+0000 7f8842ba1740 1 Processor -- start 2024-03-26T18:01:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.807+0000 7f8842ba1740 1 -- start start 2024-03-26T18:01:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.807+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a45d0 0x5610f09a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.807+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a4fc0 0x5610f099ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.807+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09b3b20 0x5610f099e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.807+0000 7f8842ba1740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88381af360 con 0x5610f09a4fc0 2024-03-26T18:01:26.790 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.807+0000 7f8842ba1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b0ea0 con 0x5610f09a45d0 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.807+0000 7f8842ba1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88381afa30 con 0x5610f09b3b20 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a4fc0 0x5610f099ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a4fc0 0x5610f099ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36360/0 (socket says 172.21.15.105:36360) 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 -- 172.21.15.105:0/2588465493 learned_addr learned my addr 172.21.15.105:0/2588465493 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883e479640 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09b3b20 0x5610f099e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883dc78640 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a45d0 0x5610f09a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 -- 172.21.15.105:0/2588465493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09b3b20 msgr2=0x5610f099e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09b3b20 0x5610f099e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 -- 172.21.15.105:0/2588465493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a45d0 msgr2=0x5610f09a34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a45d0 0x5610f09a34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 -- 172.21.15.105:0/2588465493 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f099b450 con 0x5610f09a4fc0 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f883d477640 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a4fc0 0x5610f099ac80 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f882800bc80 tx=0x7f882800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66d35c2be176dbf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.791 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.808+0000 7f88237fe640 1 -- 172.21.15.105:0/2588465493 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828015070 con 0x5610f09a4fc0 2024-03-26T18:01:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.809+0000 7f88237fe640 1 -- 172.21.15.105:0/2588465493 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8828011d20 con 0x5610f09a4fc0 2024-03-26T18:01:26.792 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.809+0000 7f88237fe640 1 -- 172.21.15.105:0/2588465493 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828005360 con 0x5610f09a4fc0 2024-03-26T18:01:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 -- 172.21.15.105:0/2588465493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a4fc0 msgr2=0x5610f099ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a4fc0 0x5610f099ac80 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f882800bc80 tx=0x7f882800f730 comp rx=0 tx=0).stop 2024-03-26T18:01:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 -- 172.21.15.105:0/2588465493 shutdown_connections 2024-03-26T18:01:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09b3b20 0x5610f099e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a4fc0 0x5610f099ac80 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2588465493 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09a45d0 0x5610f09a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 -- 172.21.15.105:0/2588465493 >> 172.21.15.105:0/2588465493 conn(0x5610f099fb40 msgr2=0x5610f09a2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 -- 172.21.15.105:0/2588465493 shutdown_connections 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.810+0000 7f8842ba1740 1 -- 172.21.15.105:0/2588465493 wait complete. 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.811+0000 7f8842ba1740 1 -- start start 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.811+0000 7f8842ba1740 1 Processor -- start 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.811+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.811+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.811+0000 7f8842ba1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 0x5610f09cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883dc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.796 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:34272/0 (socket says 172.21.15.105:34272) 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883dc78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36366/0 (socket says 172.21.15.105:36366) 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 -- 172.21.15.105:0/2108612133 learned_addr learned my addr 172.21.15.105:0/2108612133 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b07d0 con 0x5610f09a45d0 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b0100 con 0x5610f09b3b20 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f88381ad340 con 0x5610f09a4fc0 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f8842ba1740 10 client.0 _ll_register_callbacks cb 0x7f88340ba020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883e479640 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 0x5610f09cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 -- 172.21.15.105:0/2108612133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 msgr2=0x5610f09cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 0x5610f09cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 -- 172.21.15.105:0/2108612133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 msgr2=0x5610f09cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 -- 172.21.15.105:0/2108612133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f09ccef0 con 0x5610f09a4fc0 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.812+0000 7f883d477640 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09cb8f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f882800bbd0 tx=0x7f8828005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d6ae5f174df83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.813+0000 7f88217fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.813+0000 7f88217fa640 1 -- 172.21.15.105:0/2108612133 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828015030 con 0x5610f09a4fc0 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.813+0000 7f88217fa640 1 -- 172.21.15.105:0/2108612133 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f8828004890 con 0x5610f09a4fc0 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.813+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8828011d20 con 0x5610f09a4fc0 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.813+0000 7f88217fa640 1 -- 172.21.15.105:0/2108612133 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8828004a90 con 0x5610f09a4fc0 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.813+0000 7f8842ba1740 10 client.33391 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.813+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5610f0a20470 con 0x5610f09a4fc0 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.813+0000 7f88217fa640 1 -- 172.21.15.105:0/2108612133 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8828045030 con 0x5610f09a4fc0 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.814+0000 7f88217fa640 1 -- 172.21.15.105:0/2108612133 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f882803e030 con 0x5610f09a4fc0 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.814+0000 7f8842ba1740 10 client.33391 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.814+0000 7f8842ba1740 10 client.33391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.814+0000 7f8842ba1740 20 client.33391 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.814+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f0a20df0 0x5610f0a23210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.814+0000 7f8842ba1740 4 client.33391 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:01:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.814+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8828011d20 con 0x5610f0a20df0 2024-03-26T18:01:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.814+0000 7f883dc78640 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f0a20df0 0x5610f0a23210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.815+0000 7f88217fa640 10 client.33391 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:26.799 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.815+0000 7f883dc78640 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f0a20df0 0x5610f0a23210 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd1ae7108e079f25 server_cookie=c1b9694a607ca497 in_seq=0 out_seq=0 2024-03-26T18:01:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f88217fa640 1 -- 172.21.15.105:0/2108612133 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8828011d20 con 0x5610f0a20df0 2024-03-26T18:01:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f88217fa640 10 client.33391 handle_command_reply: tid=0 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 client.33391 shutdown 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 msgr2=0x5610f09cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09cb8f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f882800bbd0 tx=0x7f8828005c90 comp rx=0 tx=0).stop 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 shutdown_connections 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f0a20df0 0x5610f0a23210 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f09b3b20 0x5610f09cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f09a4fc0 0x5610f09cb8f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 --2- 172.21.15.105:0/2108612133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f09a45d0 0x5610f09cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 >> 172.21.15.105:0/2108612133 conn(0x5610f099fb40 msgr2=0x5610f09a59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f88217fa640 0 client.33391 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.975+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 shutdown_connections 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f8842ba1740 1 -- 172.21.15.105:0/2108612133 wait complete. 2024-03-26T18:01:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f8842ba1740 20 client.33391 trim_cache size 0 max 16384 2024-03-26T18:01:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:01:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:01:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:01:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:01:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 -- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88240a2fc0 msgr2=0x7f88240a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88240a2fc0 0x7f88240a5480 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x5610f09a5820 tx=0x7f8830004ab0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 -- 172.21.15.105:0/828245646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88381007e0 msgr2=0x7f883819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88381007e0 0x7f883819e470 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8834000c00 tx=0x7f88340040c0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 -- 172.21.15.105:0/828245646 shutdown_connections 2024-03-26T18:01:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f88240a2fc0 0x7f88240a5480 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8838102390 0x7f88381a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f88381019d0 0x7f883819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 --2- 172.21.15.105:0/828245646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88381007e0 0x7f883819e470 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 -- 172.21.15.105:0/828245646 >> 172.21.15.105:0/828245646 conn(0x7f88380fbd50 msgr2=0x7f883810dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 -- 172.21.15.105:0/828245646 shutdown_connections 2024-03-26T18:01:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:26.976+0000 7f883fedc640 1 -- 172.21.15.105:0/828245646 wait complete. 2024-03-26T18:01:27.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 79774 7.02 MB/sec execute 490 sec latency 218.025 ms 2024-03-26T18:01:28.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:27 smithi144 ceph-mon[36095]: pgmap v1045: 97 pgs: 97 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-03-26T18:01:28.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:27 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:01:28.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:27 smithi105 ceph-mon[32280]: pgmap v1045: 97 pgs: 97 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-03-26T18:01:28.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:27 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:01:28.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:27 smithi135 ceph-mon[35636]: pgmap v1045: 97 pgs: 97 active+clean; 362 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-03-26T18:01:28.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:27 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:01:28.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 79895 7.01 MB/sec execute 491 sec latency 131.181 ms 2024-03-26T18:01:29.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:29 smithi144 ceph-mon[36095]: pgmap v1046: 97 pgs: 97 active+clean; 360 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-26T18:01:29.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:29 smithi105 ceph-mon[32280]: pgmap v1046: 97 pgs: 97 active+clean; 360 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-26T18:01:29.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:29.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:29 smithi135 ceph-mon[35636]: pgmap v1046: 97 pgs: 97 active+clean; 360 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-26T18:01:29.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 80038 7.01 MB/sec execute 492 sec latency 122.947 ms 2024-03-26T18:01:30.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:01:30.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:01:30.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:30.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:01:30.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:01:30.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:30.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:01:30.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:01:30.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:01:30.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 80182 7.01 MB/sec execute 493 sec latency 119.861 ms 2024-03-26T18:01:31.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:31 smithi144 ceph-mon[36095]: pgmap v1047: 97 pgs: 97 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-03-26T18:01:31.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:31 smithi105 ceph-mon[32280]: pgmap v1047: 97 pgs: 97 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-03-26T18:01:31.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:31 smithi135 ceph-mon[35636]: pgmap v1047: 97 pgs: 97 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-03-26T18:01:31.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 80329 7.01 MB/sec execute 494 sec latency 170.663 ms 2024-03-26T18:01:32.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 80472 7.01 MB/sec execute 495 sec latency 139.525 ms 2024-03-26T18:01:33.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:33 smithi144 ceph-mon[36095]: pgmap v1048: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-03-26T18:01:33.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:33 smithi105 ceph-mon[32280]: pgmap v1048: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-03-26T18:01:33.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:33 smithi135 ceph-mon[35636]: pgmap v1048: 97 pgs: 97 active+clean; 356 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-03-26T18:01:33.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 80660 7.03 MB/sec execute 496 sec latency 119.800 ms 2024-03-26T18:01:34.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 80817 7.05 MB/sec execute 497 sec latency 131.481 ms 2024-03-26T18:01:35.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:35 smithi144 ceph-mon[36095]: pgmap v1049: 97 pgs: 97 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-03-26T18:01:35.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:35 smithi105 ceph-mon[32280]: pgmap v1049: 97 pgs: 97 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-03-26T18:01:35.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:35 smithi135 ceph-mon[35636]: pgmap v1049: 97 pgs: 97 active+clean; 361 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-03-26T18:01:35.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 80984 7.06 MB/sec execute 498 sec latency 147.625 ms 2024-03-26T18:01:36.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:01:36.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:01:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 -- 172.21.15.105:0/3446226981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 msgr2=0x7f486c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 --2- 172.21.15.105:0/3446226981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 0x7f486c100be0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f4868009900 tx=0x7f486802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 -- 172.21.15.105:0/3446226981 shutdown_connections 2024-03-26T18:01:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 --2- 172.21.15.105:0/3446226981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102370 0x7f486c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 --2- 172.21.15.105:0/3446226981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019b0 0x7f486c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 --2- 172.21.15.105:0/3446226981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 0x7f486c100be0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 -- 172.21.15.105:0/3446226981 >> 172.21.15.105:0/3446226981 conn(0x7f486c0fbd30 msgr2=0x7f486c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 -- 172.21.15.105:0/3446226981 shutdown_connections 2024-03-26T18:01:36.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.163+0000 7f487388c640 1 -- 172.21.15.105:0/3446226981 wait complete. 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.164+0000 7f487388c640 1 Processor -- start 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.164+0000 7f487388c640 1 -- start start 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.164+0000 7f487388c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 0x7f486c10b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.164+0000 7f487388c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019b0 0x7f486c109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.164+0000 7f487388c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102370 0x7f486c109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.164+0000 7f487388c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f486c10d470 con 0x7f486c1007c0 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.164+0000 7f487388c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f486c10d2f0 con 0x7f486c102370 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.164+0000 7f487388c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f486c10d5f0 con 0x7f486c1019b0 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 0x7f486c10b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871e29640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102370 0x7f486c109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 0x7f486c10b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46758/0 (socket says 172.21.15.105:46758) 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 -- 172.21.15.105:0/3525784 learned_addr learned my addr 172.21.15.105:0/3525784 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4870e27640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019b0 0x7f486c109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 -- 172.21.15.105:0/3525784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019b0 msgr2=0x7f486c109780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019b0 0x7f486c109780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 -- 172.21.15.105:0/3525784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102370 msgr2=0x7f486c109cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102370 0x7f486c109cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 -- 172.21.15.105:0/3525784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c10a550 con 0x7f486c1007c0 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f4871628640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 0x7f486c10b140 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f486802c610 tx=0x7f4868004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.165+0000 7f48627fc640 1 -- 172.21.15.105:0/3525784 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868043020 con 0x7f486c1007c0 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.166+0000 7f48627fc640 1 -- 172.21.15.105:0/3525784 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48680043c0 con 0x7f486c1007c0 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.166+0000 7f487388c640 1 -- 172.21.15.105:0/3525784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c1a1c50 con 0x7f486c1007c0 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.166+0000 7f48627fc640 1 -- 172.21.15.105:0/3525784 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868003bc0 con 0x7f486c1007c0 2024-03-26T18:01:36.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.166+0000 7f487388c640 1 -- 172.21.15.105:0/3525784 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c1a21a0 con 0x7f486c1007c0 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.168+0000 7f48627fc640 1 -- 172.21.15.105:0/3525784 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4868046020 con 0x7f486c1007c0 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f487647e740 1 Processor -- start 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f487647e740 1 -- start start 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f487647e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf5075d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f487647e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf507ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f487647e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf507b450 0x555cf507d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f487647e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555cf4fda980 con 0x555cf507b450 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f487647e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555cf4f82be0 con 0x555cf50758f0 2024-03-26T18:01:36.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f487647e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555cf4fd6930 con 0x555cf50762e0 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f4871628640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf507ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f4871e29640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf5075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f4870e27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf507b450 0x555cf507d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f4871628640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf507ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37808/0 (socket says 172.21.15.105:37808) 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f4871628640 1 -- 172.21.15.105:0/689082463 learned_addr learned my addr 172.21.15.105:0/689082463 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.169+0000 7f48627fc640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4871e29640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4870e27640 1 -- 172.21.15.105:0/689082463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 msgr2=0x555cf507ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4870e27640 1 --2- 172.21.15.105:0/689082463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf507ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4870e27640 1 -- 172.21.15.105:0/689082463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 msgr2=0x555cf5075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4870e27640 1 --2- 172.21.15.105:0/689082463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf5075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4870e27640 1 -- 172.21.15.105:0/689082463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf4db87d0 con 0x555cf507b450 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4871e29640 1 --2- 172.21.15.105:0/689082463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf5075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f48627fc640 1 -- 172.21.15.105:0/3525784 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f48680f1530 con 0x7f486c1007c0 2024-03-26T18:01:36.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4870e27640 1 --2- 172.21.15.105:0/689082463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf507b450 0x555cf507d8e0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f485c009bd0 tx=0x7f485c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fc528afb2e487d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f4871e29640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x555cf5076b40 tx=0x7f4864004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f48577fe640 1 -- 172.21.15.105:0/689082463 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c004230 con 0x555cf507b450 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f48577fe640 1 -- 172.21.15.105:0/689082463 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f485c009e40 con 0x555cf507b450 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.170+0000 7f48577fe640 1 -- 172.21.15.105:0/689082463 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c004ca0 con 0x555cf507b450 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 -- 172.21.15.105:0/689082463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf507b450 msgr2=0x555cf507d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 --2- 172.21.15.105:0/689082463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf507b450 0x555cf507d8e0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f485c009bd0 tx=0x7f485c011810 comp rx=0 tx=0).stop 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 -- 172.21.15.105:0/689082463 shutdown_connections 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 --2- 172.21.15.105:0/689082463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf507b450 0x555cf507d8e0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 --2- 172.21.15.105:0/689082463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf507ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 --2- 172.21.15.105:0/689082463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf5075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 -- 172.21.15.105:0/689082463 >> 172.21.15.105:0/689082463 conn(0x555cf5070e60 msgr2=0x555cf5071260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 -- 172.21.15.105:0/689082463 shutdown_connections 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.171+0000 7f487647e740 1 -- 172.21.15.105:0/689082463 wait complete. 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.172+0000 7f487647e740 1 -- start start 2024-03-26T18:01:36.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.172+0000 7f487647e740 1 Processor -- start 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.172+0000 7f487647e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.172+0000 7f487647e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 0x555cf5080080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871628640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871628640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55534/0 (socket says 172.21.15.105:55534) 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871628640 1 -- 172.21.15.105:0/206478067 learned_addr learned my addr 172.21.15.105:0/206478067 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f487647e740 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf507b450 0x555cf50870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x555cf507e400 con 0x555cf50762e0 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555cf4f96160 con 0x555cf50758f0 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555cf507e580 con 0x555cf507b450 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4870e27640 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 0x555cf5080080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871e29640 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf507b450 0x555cf50870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f487647e740 10 client.0 _ll_register_callbacks cb 0x555cf4fd2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871e29640 1 -- 172.21.15.105:0/206478067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 msgr2=0x555cf507fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871e29640 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871e29640 1 -- 172.21.15.105:0/206478067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 msgr2=0x555cf5080080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871e29640 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 0x555cf5080080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871e29640 1 -- 172.21.15.105:0/206478067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf4cee3f0 con 0x555cf507b450 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.173+0000 7f4871628640 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f4871e29640 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf507b450 0x555cf50870c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4864007310 tx=0x7f4864004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17022d3ac3f0ed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f4854ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f4854ff9640 1 -- 172.21.15.105:0/206478067 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864019070 con 0x555cf507b450 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555cf4fe7ec0 con 0x555cf507b450 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f4854ff9640 1 -- 172.21.15.105:0/206478067 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f4864004d20 con 0x555cf507b450 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f4854ff9640 1 -- 172.21.15.105:0/206478067 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4864020610 con 0x555cf507b450 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f487647e740 10 client.33403 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555cf4cae0f0 con 0x555cf507b450 2024-03-26T18:01:36.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.174+0000 7f4854ff9640 1 -- 172.21.15.105:0/206478067 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4864020c60 con 0x555cf507b450 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.175+0000 7f4854ff9640 1 -- 172.21.15.105:0/206478067 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4864005580 con 0x555cf507b450 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.175+0000 7f487647e740 10 client.33403 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.175+0000 7f487647e740 10 client.33403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.175+0000 7f487647e740 20 client.33403 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.175+0000 7f487647e740 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555cf508bef0 0x555cf508e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.175+0000 7f487647e740 4 client.33403 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.175+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cf4fe7ec0 con 0x555cf508bef0 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.175+0000 7f4871628640 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555cf508bef0 0x555cf508e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.176+0000 7f4854ff9640 10 client.33403 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:36.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.176+0000 7f4871628640 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555cf508bef0 0x555cf508e3b0 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dab6a464af0328f server_cookie=c2c5639b4f0a7414 in_seq=0 out_seq=0 2024-03-26T18:01:36.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.178+0000 7f4854ff9640 1 -- 172.21.15.105:0/206478067 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555cf4fe7ec0 con 0x555cf508bef0 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.178+0000 7f4854ff9640 10 client.33403 handle_command_reply: tid=0 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 client.33403 shutdown 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf507b450 msgr2=0x555cf50870c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf507b450 0x555cf50870c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4864007310 tx=0x7f4864004a40 comp rx=0 tx=0).stop 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 shutdown_connections 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555cf508bef0 0x555cf508e3b0 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf507b450 0x555cf50870c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 0x555cf5080080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 --2- 172.21.15.105:0/206478067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 >> 172.21.15.105:0/206478067 conn(0x555cf5070e60 msgr2=0x555cf5070080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f4854ff9640 0 client.33403 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.179+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 shutdown_connections 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.180+0000 7f487647e740 1 -- 172.21.15.105:0/206478067 wait complete. 2024-03-26T18:01:36.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.180+0000 7f487647e740 20 client.33403 trim_cache size 0 max 16384 2024-03-26T18:01:36.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.211+0000 7f487647e740 1 Processor -- start 2024-03-26T18:01:36.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.211+0000 7f487647e740 1 -- start start 2024-03-26T18:01:36.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f487647e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f487647e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf5080080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f487647e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf5087080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f487647e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f485000e530 con 0x555cf50758f0 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f487647e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4850010070 con 0x555cf507b450 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871628640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871628640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46792/0 (socket says 172.21.15.105:46792) 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871628640 1 -- 172.21.15.105:0/1687255221 learned_addr learned my addr 172.21.15.105:0/1687255221 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871e29640 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf5087080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f487647e740 1 -- 172.21.15.105:0/1687255221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f485000ec00 con 0x555cf50762e0 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4870e27640 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf5080080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871628640 1 -- 172.21.15.105:0/1687255221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 msgr2=0x555cf5080080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871628640 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf5080080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871628640 1 -- 172.21.15.105:0/1687255221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 msgr2=0x555cf5087080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871628640 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf5087080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871628640 1 -- 172.21.15.105:0/1687255221 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf50766c0 con 0x555cf50758f0 2024-03-26T18:01:36.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.212+0000 7f4871e29640 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf5087080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:36.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.213+0000 7f4871628640 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50758f0 0x555cf507fb40 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f486802c4e0 tx=0x7f48680eeec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a249023647d8b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.213+0000 7f4856ffd640 1 -- 172.21.15.105:0/1687255221 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680bd9f0 con 0x555cf50758f0 2024-03-26T18:01:36.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.213+0000 7f4856ffd640 1 -- 172.21.15.105:0/1687255221 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48680be320 con 0x555cf50758f0 2024-03-26T18:01:36.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.213+0000 7f4856ffd640 1 -- 172.21.15.105:0/1687255221 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680bd580 con 0x555cf50758f0 2024-03-26T18:01:36.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 -- 172.21.15.105:0/1687255221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50758f0 msgr2=0x555cf507fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50758f0 0x555cf507fb40 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f486802c4e0 tx=0x7f48680eeec0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 -- 172.21.15.105:0/1687255221 shutdown_connections 2024-03-26T18:01:36.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf5087080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50762e0 0x555cf5080080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 --2- 172.21.15.105:0/1687255221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50758f0 0x555cf507fb40 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 -- 172.21.15.105:0/1687255221 >> 172.21.15.105:0/1687255221 conn(0x555cf5070e60 msgr2=0x555cf507ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 -- 172.21.15.105:0/1687255221 shutdown_connections 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.214+0000 7f487647e740 1 -- 172.21.15.105:0/1687255221 wait complete. 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 1 -- start start 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 1 Processor -- start 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50758f0 0x555cf5084070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 0x555cf50845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf50f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f485000f9a0 con 0x555cf50762e0 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f485000f2d0 con 0x555cf507b450 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f485000c510 con 0x555cf50758f0 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f487647e740 10 client.0 _ll_register_callbacks cb 0x7f4864007740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:36.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f4871628640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50758f0 0x555cf5084070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f4871628640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50758f0 0x555cf5084070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37856/0 (socket says 172.21.15.105:37856) 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f4871628640 1 -- 172.21.15.105:0/763054761 learned_addr learned my addr 172.21.15.105:0/763054761 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.215+0000 7f4870e27640 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 0x555cf50845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4871e29640 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf50f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4871628640 1 -- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 msgr2=0x555cf50f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4871628640 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf50f02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4871628640 1 -- 172.21.15.105:0/763054761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 msgr2=0x555cf50845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4871628640 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 0x555cf50845b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4871628640 1 -- 172.21.15.105:0/763054761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf50f0820 con 0x555cf50758f0 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4871e29640 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf50f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4871628640 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50758f0 0x555cf5084070 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f486802c170 tx=0x7f48680035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23989514de653d3c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4854ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:36.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4854ff9640 1 -- 172.21.15.105:0/763054761 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680db060 con 0x555cf50758f0 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f487647e740 1 -- 172.21.15.105:0/763054761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48680be320 con 0x555cf50758f0 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4854ff9640 1 -- 172.21.15.105:0/763054761 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f48680ba290 con 0x555cf50758f0 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f4854ff9640 1 -- 172.21.15.105:0/763054761 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f48680bd760 con 0x555cf50758f0 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f487647e740 10 client.33409 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.216+0000 7f487647e740 1 -- 172.21.15.105:0/763054761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555cf50f1b80 con 0x555cf50758f0 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f4854ff9640 1 -- 172.21.15.105:0/763054761 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48680c1d70 con 0x555cf50758f0 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f4854ff9640 1 -- 172.21.15.105:0/763054761 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f48680c6210 con 0x555cf50758f0 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f487647e740 10 client.33409 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f487647e740 10 client.33409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f487647e740 20 client.33409 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:36.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f487647e740 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555cf50f23b0 0x555cf50f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f487647e740 4 client.33409 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:01:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f487647e740 1 -- 172.21.15.105:0/763054761 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f48680be320 con 0x555cf50f23b0 2024-03-26T18:01:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.217+0000 7f4870e27640 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555cf50f23b0 0x555cf50f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.218+0000 7f4854ff9640 10 client.33409 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:36.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.218+0000 7f4870e27640 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555cf50f23b0 0x555cf50f47d0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a3268b1de28c9d server_cookie=f1884bc69032520 in_seq=0 out_seq=0 2024-03-26T18:01:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f4854ff9640 1 -- 172.21.15.105:0/763054761 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f48680be320 con 0x555cf50f23b0 2024-03-26T18:01:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f4854ff9640 10 client.33409 handle_command_reply: tid=0 2024-03-26T18:01:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 client.33409 shutdown 2024-03-26T18:01:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 -- 172.21.15.105:0/763054761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50758f0 msgr2=0x555cf5084070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50758f0 0x555cf5084070 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f486802c170 tx=0x7f48680035f0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 -- 172.21.15.105:0/763054761 shutdown_connections 2024-03-26T18:01:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x555cf50f23b0 0x555cf50f47d0 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555cf507b450 0x555cf50f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f4854ff9640 0 client.33409 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x555cf50762e0 0x555cf50845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 --2- 172.21.15.105:0/763054761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555cf50758f0 0x555cf5084070 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 -- 172.21.15.105:0/763054761 >> 172.21.15.105:0/763054761 conn(0x555cf5070e60 msgr2=0x555cf507fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.219+0000 7f487647e740 1 -- 172.21.15.105:0/763054761 shutdown_connections 2024-03-26T18:01:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.220+0000 7f487647e740 1 -- 172.21.15.105:0/763054761 wait complete. 2024-03-26T18:01:36.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.220+0000 7f487647e740 20 client.33409 trim_cache size 0 max 16384 2024-03-26T18:01:36.209 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:01:36.210 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T18:01:36.210 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T18:01:36.210 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:01:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.221+0000 7f487388c640 1 -- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 msgr2=0x7f48580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.221+0000 7f487388c640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x555cf5076b40 tx=0x7f4864004ab0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.221+0000 7f487388c640 1 -- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 msgr2=0x7f486c10b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.221+0000 7f487388c640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 0x7f486c10b140 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f486802c610 tx=0x7f4868004040 comp rx=0 tx=0).stop 2024-03-26T18:01:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.221+0000 7f487388c640 1 -- 172.21.15.105:0/3525784 shutdown_connections 2024-03-26T18:01:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.221+0000 7f487388c640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f48580a2f20 0x7f48580a53e0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.222+0000 7f487388c640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f486c102370 0x7f486c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.222+0000 7f487388c640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1019b0 0x7f486c109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.222+0000 7f487388c640 1 --2- 172.21.15.105:0/3525784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f486c1007c0 0x7f486c10b140 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.222+0000 7f487388c640 1 -- 172.21.15.105:0/3525784 >> 172.21.15.105:0/3525784 conn(0x7f486c0fbd30 msgr2=0x7f486c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.222+0000 7f487388c640 1 -- 172.21.15.105:0/3525784 shutdown_connections 2024-03-26T18:01:36.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.222+0000 7f487388c640 1 -- 172.21.15.105:0/3525784 wait complete. 2024-03-26T18:01:36.218 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T18:01:36.218 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:01:36.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 -- 172.21.15.105:0/2217917343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1019d0 msgr2=0x7fa54c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 --2- 172.21.15.105:0/2217917343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1019d0 0x7fa54c101e50 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fa538009900 tx=0x7fa53802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:36.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 -- 172.21.15.105:0/2217917343 shutdown_connections 2024-03-26T18:01:36.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 --2- 172.21.15.105:0/2217917343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa54c102390 0x7fa54c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 --2- 172.21.15.105:0/2217917343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1019d0 0x7fa54c101e50 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 --2- 172.21.15.105:0/2217917343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa54c1007e0 0x7fa54c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 -- 172.21.15.105:0/2217917343 >> 172.21.15.105:0/2217917343 conn(0x7fa54c0fbd50 msgr2=0x7fa54c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 -- 172.21.15.105:0/2217917343 shutdown_connections 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.373+0000 7fa552d10640 1 -- 172.21.15.105:0/2217917343 wait complete. 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa552d10640 1 Processor -- start 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa552d10640 1 -- start start 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa552d10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1007e0 0x7fa54c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa552d10640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa54c1019d0 0x7fa54c19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa552d10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa54c102390 0x7fa54c1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa552d10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c10c310 con 0x7fa54c1007e0 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa552d10640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c10c190 con 0x7fa54c1019d0 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa552d10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c10c490 con 0x7fa54c102390 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa550aac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1007e0 0x7fa54c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa550aac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1007e0 0x7fa54c199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46814/0 (socket says 172.21.15.105:46814) 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa550aac640 1 -- 172.21.15.105:0/580314022 learned_addr learned my addr 172.21.15.105:0/580314022 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa54bfff640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa54c1019d0 0x7fa54c19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa5512ad640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa54c102390 0x7fa54c1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa550aac640 1 -- 172.21.15.105:0/580314022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa54c102390 msgr2=0x7fa54c1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa550aac640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa54c102390 0x7fa54c1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa550aac640 1 -- 172.21.15.105:0/580314022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa54c1019d0 msgr2=0x7fa54c19a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa550aac640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa54c1019d0 0x7fa54c19a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.374+0000 7fa550aac640 1 -- 172.21.15.105:0/580314022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54c1a19b0 con 0x7fa54c1007e0 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.375+0000 7fa550aac640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1007e0 0x7fa54c199da0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9c0 tx=0x7fa54000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.375+0000 7fa549ffb640 1 -- 172.21.15.105:0/580314022 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54000cd80 con 0x7fa54c1007e0 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.375+0000 7fa549ffb640 1 -- 172.21.15.105:0/580314022 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5400035f0 con 0x7fa54c1007e0 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.375+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54c1a1ca0 con 0x7fa54c1007e0 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.375+0000 7fa549ffb640 1 -- 172.21.15.105:0/580314022 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540010690 con 0x7fa54c1007e0 2024-03-26T18:01:36.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.375+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54c1a21a0 con 0x7fa54c1007e0 2024-03-26T18:01:36.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.377+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54c1a23d0 con 0x7fa54c1007e0 2024-03-26T18:01:36.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.378+0000 7fa549ffb640 1 -- 172.21.15.105:0/580314022 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa5400037a0 con 0x7fa54c1007e0 2024-03-26T18:01:36.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.378+0000 7fa549ffb640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa53c0a2f20 0x7fa53c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.379+0000 7fa54bfff640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa53c0a2f20 0x7fa53c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.379+0000 7fa549ffb640 1 -- 172.21.15.105:0/580314022 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fa5400cf880 con 0x7fa54c1007e0 2024-03-26T18:01:36.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.379+0000 7fa54bfff640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa53c0a2f20 0x7fa53c0a53e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa538009230 tx=0x7fa538005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.385+0000 7fa549ffb640 1 -- 172.21.15.105:0/580314022 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa54009c610 con 0x7fa54c1007e0 2024-03-26T18:01:36.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81132 7.06 MB/sec execute 499 sec latency 115.936 ms 2024-03-26T18:01:36.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.692+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa54c1a25a0 con 0x7fa54c1007e0 2024-03-26T18:01:36.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.693+0000 7fa549ffb640 1 -- 172.21.15.105:0/580314022 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11938 (secure 0 0 0) 0x7fa54009bf10 con 0x7fa54c1007e0 2024-03-26T18:01:36.675 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:01:36.675 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:00:20.611284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_23346":{"gid":23346,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/966537438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":966537438},{"type":"v1","addr":"172.21.15.135:6835","nonce":966537438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24487,24466,14628,24482,24521]},"id":1}]} 2024-03-26T18:01:36.676 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 43 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa53c0a2f20 msgr2=0x7fa53c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa53c0a2f20 0x7fa53c0a53e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa538009230 tx=0x7fa538005fd0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1007e0 msgr2=0x7fa54c199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1007e0 0x7fa54c199da0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9c0 tx=0x7fa54000ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 shutdown_connections 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa53c0a2f20 0x7fa53c0a53e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa54c102390 0x7fa54c1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa54c1019d0 0x7fa54c19a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 --2- 172.21.15.105:0/580314022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa54c1007e0 0x7fa54c199da0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 >> 172.21.15.105:0/580314022 conn(0x7fa54c0fbd50 msgr2=0x7fa54c068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 shutdown_connections 2024-03-26T18:01:36.679 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.695+0000 7fa552d10640 1 -- 172.21.15.105:0/580314022 wait complete. 2024-03-26T18:01:36.690 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T18:01:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 -- 172.21.15.105:0/3221293458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 msgr2=0x7fa910108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 --2- 172.21.15.105:0/3221293458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 0x7fa910108c10 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fa908009ca0 tx=0x7fa90802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 -- 172.21.15.105:0/3221293458 shutdown_connections 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 --2- 172.21.15.105:0/3221293458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 0x7fa910108c10 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 --2- 172.21.15.105:0/3221293458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9101019c0 0x7fa910101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 --2- 172.21.15.105:0/3221293458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9101007d0 0x7fa910100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 -- 172.21.15.105:0/3221293458 >> 172.21.15.105:0/3221293458 conn(0x7fa9100fbd60 msgr2=0x7fa9100fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 -- 172.21.15.105:0/3221293458 shutdown_connections 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 -- 172.21.15.105:0/3221293458 wait complete. 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 Processor -- start 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.849+0000 7fa916cac640 1 -- start start 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa916cac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9101007d0 0x7fa910197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa916cac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9101019c0 0x7fa910197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa916cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 0x7fa91019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa916cac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010d420 con 0x7fa910102380 2024-03-26T18:01:36.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa916cac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010d2a0 con 0x7fa9101019c0 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa916cac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa91010d5a0 con 0x7fa9101007d0 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 0x7fa91019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9101019c0 0x7fa910197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 0x7fa91019f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46828/0 (socket says 172.21.15.105:46828) 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 -- 172.21.15.105:0/321218179 learned_addr learned my addr 172.21.15.105:0/321218179 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa914a48640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9101007d0 0x7fa910197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 -- 172.21.15.105:0/321218179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9101007d0 msgr2=0x7fa910197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9101007d0 0x7fa910197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 -- 172.21.15.105:0/321218179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9101019c0 msgr2=0x7fa910197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9101019c0 0x7fa910197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 -- 172.21.15.105:0/321218179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91019f6b0 con 0x7fa910102380 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa90ffff640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9101019c0 0x7fa910197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa915249640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 0x7fa91019f040 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa90802c700 tx=0x7fa9080328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa90dffb640 1 -- 172.21.15.105:0/321218179 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908041020 con 0x7fa910102380 2024-03-26T18:01:36.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa916cac640 1 -- 172.21.15.105:0/321218179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91019f940 con 0x7fa910102380 2024-03-26T18:01:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.850+0000 7fa90dffb640 1 -- 172.21.15.105:0/321218179 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa908003e20 con 0x7fa910102380 2024-03-26T18:01:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.851+0000 7fa90dffb640 1 -- 172.21.15.105:0/321218179 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908039040 con 0x7fa910102380 2024-03-26T18:01:36.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.851+0000 7fa916cac640 1 -- 172.21.15.105:0/321218179 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91019fe80 con 0x7fa910102380 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.853+0000 7fa90dffb640 1 -- 172.21.15.105:0/321218179 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa908044020 con 0x7fa910102380 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.853+0000 7fa919971740 1 Processor -- start 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa919971740 1 -- start start 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa919971740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa32490e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa919971740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32495f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90dffb640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9000a2f20 0x7fa9000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa919971740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32496590 0x55fa32498a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa919971740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa323f5970 con 0x55fa32496590 2024-03-26T18:01:36.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa919971740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fa3239dd70 con 0x55fa32491420 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa919971740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa323f1920 con 0x55fa32490a30 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa32490e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa32490e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37900/0 (socket says 172.21.15.105:37900) 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 -- 172.21.15.105:0/2924615253 learned_addr learned my addr 172.21.15.105:0/2924615253 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90dffb640 1 -- 172.21.15.105:0/321218179 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fa9080f1600 con 0x7fa910102380 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9000a2f20 0x7fa9000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa914a48640 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32495f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 -- 172.21.15.105:0/2924615253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 msgr2=0x55fa32495f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32495f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 -- 172.21.15.105:0/2924615253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32496590 msgr2=0x55fa32498a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa915249640 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32496590 0x55fa32498a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32496590 0x55fa32498a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa90ffff640 1 -- 172.21.15.105:0/2924615253 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa321093f0 con 0x55fa32490a30 2024-03-26T18:01:36.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.854+0000 7fa915249640 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32496590 0x55fa32498a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.855+0000 7fa90ffff640 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa32490e50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc011eb0 tx=0x7fa8fc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd60ece777425493 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.855+0000 7fa90ffff640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9000a2f20 0x7fa9000a53e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00fc90 tx=0x7fa8fc018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.855+0000 7fa8faffd640 1 -- 172.21.15.105:0/2924615253 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc022070 con 0x55fa32490a30 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.855+0000 7fa8faffd640 1 -- 172.21.15.105:0/2924615253 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8fc009e20 con 0x55fa32490a30 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.855+0000 7fa8faffd640 1 -- 172.21.15.105:0/2924615253 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc01e400 con 0x55fa32490a30 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 -- 172.21.15.105:0/2924615253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 msgr2=0x55fa32490e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa32490e50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc011eb0 tx=0x7fa8fc00f790 comp rx=0 tx=0).stop 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 -- 172.21.15.105:0/2924615253 shutdown_connections 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32496590 0x55fa32498a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32495f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 --2- 172.21.15.105:0/2924615253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa32490e50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 -- 172.21.15.105:0/2924615253 >> 172.21.15.105:0/2924615253 conn(0x55fa3248bfa0 msgr2=0x55fa3248c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 -- 172.21.15.105:0/2924615253 shutdown_connections 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 -- 172.21.15.105:0/2924615253 wait complete. 2024-03-26T18:01:36.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 -- start start 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.856+0000 7fa919971740 1 Processor -- start 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa919971740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa3249af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa919971740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa919971740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32496590 0x55fa324a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa919971740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fa324994d0 con 0x55fa32491420 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa919971740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fa323b1310 con 0x55fa32496590 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa919971740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa32499650 con 0x55fa32490a30 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa919971740 10 client.0 _ll_register_callbacks cb 0x55fa323ecc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32496590 0x55fa324a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa914a48640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa3249af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32496590 0x55fa324a24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55628/0 (socket says 172.21.15.105:55628) 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46852/0 (socket says 172.21.15.105:46852) 2024-03-26T18:01:36.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 -- 172.21.15.105:0/2733226248 learned_addr learned my addr 172.21.15.105:0/2733226248 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 -- 172.21.15.105:0/2733226248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 msgr2=0x55fa3249af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa3249af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 -- 172.21.15.105:0/2733226248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 msgr2=0x55fa3249b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 -- 172.21.15.105:0/2733226248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa321d37d0 con 0x55fa32496590 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa915249640 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32496590 0x55fa324a24c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa9080ba6c0 tx=0x7fa9080bd650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7344bd297fecb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.857+0000 7fa8effff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.858+0000 7fa8effff640 1 -- 172.21.15.105:0/2733226248 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9080bb520 con 0x55fa32496590 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.858+0000 7fa8effff640 1 -- 172.21.15.105:0/2733226248 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fa9080bb740 con 0x55fa32496590 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.858+0000 7fa919971740 1 -- 172.21.15.105:0/2733226248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc009e20 con 0x55fa32496590 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.858+0000 7fa8effff640 1 -- 172.21.15.105:0/2733226248 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa9080bb340 con 0x55fa32496590 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.858+0000 7fa919971740 10 client.33434 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.858+0000 7fa919971740 1 -- 172.21.15.105:0/2733226248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa324a57a0 con 0x55fa32496590 2024-03-26T18:01:36.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa8effff640 1 -- 172.21.15.105:0/2733226248 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8fc009e20 con 0x55fa32496590 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa8effff640 1 -- 172.21.15.105:0/2733226248 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa90803c030 con 0x55fa32496590 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa919971740 10 client.33434 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa919971740 10 client.33434 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa919971740 20 client.33434 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa919971740 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55fa324a7010 0x55fa324a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa919971740 4 client.33434 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa919971740 1 -- 172.21.15.105:0/2733226248 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa32402eb0 con 0x55fa324a7010 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa914a48640 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55fa324a7010 0x55fa324a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa8effff640 10 client.33434 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:01:36.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.859+0000 7fa914a48640 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55fa324a7010 0x55fa324a9470 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=13716abea868ac29 server_cookie=3b11174f51df2b5e in_seq=0 out_seq=0 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.860+0000 7fa8effff640 1 -- 172.21.15.105:0/2733226248 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fa32402eb0 con 0x55fa324a7010 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.860+0000 7fa8effff640 10 client.33434 handle_command_reply: tid=0 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 client.33434 shutdown 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 -- 172.21.15.105:0/2733226248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32496590 msgr2=0x55fa324a24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32496590 0x55fa324a24c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa9080ba6c0 tx=0x7fa9080bd650 comp rx=0 tx=0).stop 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 -- 172.21.15.105:0/2733226248 shutdown_connections 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55fa324a7010 0x55fa324a9470 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32496590 0x55fa324a24c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 --2- 172.21.15.105:0/2733226248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32490a30 0x55fa3249af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 -- 172.21.15.105:0/2733226248 >> 172.21.15.105:0/2733226248 conn(0x55fa3248bfa0 msgr2=0x55fa3248b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa8effff640 0 client.33434 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:01:36.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 -- 172.21.15.105:0/2733226248 shutdown_connections 2024-03-26T18:01:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 1 -- 172.21.15.105:0/2733226248 wait complete. 2024-03-26T18:01:36.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.861+0000 7fa919971740 20 client.33434 trim_cache size 0 max 16384 2024-03-26T18:01:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa919971740 1 Processor -- start 2024-03-26T18:01:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa919971740 1 -- start start 2024-03-26T18:01:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa919971740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32490a30 0x55fa324929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa919971740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32492ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa919971740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa324a2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa919971740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a6b20 con 0x55fa32490a30 2024-03-26T18:01:36.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa919971740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a8660 con 0x55fa32491420 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa919971740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a71f0 con 0x55fa32496590 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa914a48640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32490a30 0x55fa324929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa914a48640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32490a30 0x55fa324929b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46860/0 (socket says 172.21.15.105:46860) 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa914a48640 1 -- 172.21.15.105:0/1149349497 learned_addr learned my addr 172.21.15.105:0/1149349497 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa90ffff640 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32492ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa915249640 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa324a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa914a48640 1 -- 172.21.15.105:0/1149349497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 msgr2=0x55fa324a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.883+0000 7fa914a48640 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa324a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa914a48640 1 -- 172.21.15.105:0/1149349497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 msgr2=0x55fa32492ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa914a48640 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32492ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa914a48640 1 -- 172.21.15.105:0/1149349497 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa32495c00 con 0x55fa32490a30 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa90ffff640 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32492ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa915249640 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa324a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa914a48640 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32490a30 0x55fa324929b0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x55fa32491820 tx=0x7fa904005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3de5170a22c429 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa8fa7fc640 1 -- 172.21.15.105:0/1149349497 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904016550 con 0x55fa32490a30 2024-03-26T18:01:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa8fa7fc640 1 -- 172.21.15.105:0/1149349497 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa904026120 con 0x55fa32490a30 2024-03-26T18:01:36.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.884+0000 7fa8fa7fc640 1 -- 172.21.15.105:0/1149349497 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904003850 con 0x55fa32490a30 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 -- 172.21.15.105:0/1149349497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32490a30 msgr2=0x55fa324929b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32490a30 0x55fa324929b0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x55fa32491820 tx=0x7fa904005f90 comp rx=0 tx=0).stop 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 -- 172.21.15.105:0/1149349497 shutdown_connections 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa324a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32491420 0x55fa32492ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 --2- 172.21.15.105:0/1149349497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32490a30 0x55fa324929b0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 -- 172.21.15.105:0/1149349497 >> 172.21.15.105:0/1149349497 conn(0x55fa3248bfa0 msgr2=0x55fa32491ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 -- 172.21.15.105:0/1149349497 shutdown_connections 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.885+0000 7fa919971740 1 -- 172.21.15.105:0/1149349497 wait complete. 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 1 -- start start 2024-03-26T18:01:36.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 1 Processor -- start 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32490a30 0x55fa3249ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa32513bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a7f90 con 0x55fa32491420 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a78c0 con 0x55fa32490a30 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a4b00 con 0x55fa32496590 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa919971740 10 client.0 _ll_register_callbacks cb 0x7fa9080bdac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa915249640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa32513bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa914a48640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32490a30 0x55fa3249ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa915249640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa32513bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37934/0 (socket says 172.21.15.105:37934) 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa914a48640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32490a30 0x55fa3249ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55634/0 (socket says 172.21.15.105:55634) 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa915249640 1 -- 172.21.15.105:0/3139089473 learned_addr learned my addr 172.21.15.105:0/3139089473 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:36.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.886+0000 7fa90ffff640 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa915249640 1 -- 172.21.15.105:0/3139089473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32490a30 msgr2=0x55fa3249ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa915249640 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32490a30 0x55fa3249ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa915249640 1 -- 172.21.15.105:0/3139089473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 msgr2=0x55fa3249f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa915249640 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa915249640 1 -- 172.21.15.105:0/3139089473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa324a2590 con 0x55fa32496590 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa915249640 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa32513bb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa9080c6500 tx=0x7fa908005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12618cf8bb567f7c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa8effff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa8effff640 1 -- 172.21.15.105:0/3139089473 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908034700 con 0x55fa32496590 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.887+0000 7fa8effff640 1 -- 172.21.15.105:0/3139089473 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fa908003b80 con 0x55fa32496590 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa919971740 1 -- 172.21.15.105:0/3139089473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa904026120 con 0x55fa32496590 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa8effff640 1 -- 172.21.15.105:0/3139089473 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa9080be3f0 con 0x55fa32496590 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa919971740 10 client.33433 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa919971740 1 -- 172.21.15.105:0/3139089473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa32515030 con 0x55fa32496590 2024-03-26T18:01:36.873 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa8effff640 1 -- 172.21.15.105:0/3139089473 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa904026120 con 0x55fa32496590 2024-03-26T18:01:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa8effff640 1 -- 172.21.15.105:0/3139089473 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa9080bd1d0 con 0x55fa32496590 2024-03-26T18:01:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa919971740 10 client.33433 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa919971740 10 client.33433 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:01:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa919971740 20 client.33433 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.888+0000 7fa919971740 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55fa32515810 0x55fa32517c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.889+0000 7fa919971740 4 client.33433 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.889+0000 7fa919971740 1 -- 172.21.15.105:0/3139089473 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa323fd200 con 0x55fa32515810 2024-03-26T18:01:36.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.889+0000 7fa914a48640 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55fa32515810 0x55fa32517c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:36.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.889+0000 7fa8effff640 10 client.33433 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:01:36.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.889+0000 7fa914a48640 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55fa32515810 0x55fa32517c30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35b1b60af586fa8e server_cookie=1453ecead27f136f in_seq=0 out_seq=0 2024-03-26T18:01:36.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.889+0000 7fa8effff640 1 -- 172.21.15.105:0/3139089473 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa323fd200 con 0x55fa32515810 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.889+0000 7fa8effff640 10 client.33433 handle_command_reply: tid=0 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.889+0000 7fa919971740 1 client.33433 shutdown 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 -- 172.21.15.105:0/3139089473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 msgr2=0x55fa32513bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa32513bb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa9080c6500 tx=0x7fa908005ef0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 -- 172.21.15.105:0/3139089473 shutdown_connections 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55fa32515810 0x55fa32517c30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa32496590 0x55fa32513bb0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa8effff640 0 client.33433 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fa32491420 0x55fa3249f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 --2- 172.21.15.105:0/3139089473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fa32490a30 0x55fa3249ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 -- 172.21.15.105:0/3139089473 >> 172.21.15.105:0/3139089473 conn(0x55fa3248bfa0 msgr2=0x55fa324929b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 -- 172.21.15.105:0/3139089473 shutdown_connections 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 1 -- 172.21.15.105:0/3139089473 wait complete. 2024-03-26T18:01:36.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.890+0000 7fa919971740 20 client.33433 trim_cache size 0 max 16384 2024-03-26T18:01:36.877 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:36.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 -- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9000a2f20 msgr2=0x7fa9000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9000a2f20 0x7fa9000a53e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00fc90 tx=0x7fa8fc018000 comp rx=0 tx=0).stop 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 -- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 msgr2=0x7fa91019f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 0x7fa91019f040 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa90802c700 tx=0x7fa9080328a0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 -- 172.21.15.105:0/321218179 shutdown_connections 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9000a2f20 0x7fa9000a53e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa910102380 0x7fa91019f040 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9101019c0 0x7fa910197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 --2- 172.21.15.105:0/321218179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9101007d0 0x7fa910197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 -- 172.21.15.105:0/321218179 >> 172.21.15.105:0/321218179 conn(0x7fa9100fbd60 msgr2=0x7fa910104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 -- 172.21.15.105:0/321218179 shutdown_connections 2024-03-26T18:01:36.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:36.891+0000 7fa916cac640 1 -- 172.21.15.105:0/321218179 wait complete. 2024-03-26T18:01:36.888 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 -- 172.21.15.105:0/1721817315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181019b0 msgr2=0x7f5818101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 --2- 172.21.15.105:0/1721817315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181019b0 0x7f5818101e30 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f580c009900 tx=0x7f580c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 -- 172.21.15.105:0/1721817315 shutdown_connections 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 --2- 172.21.15.105:0/1721817315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5818102370 0x7f5818108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 --2- 172.21.15.105:0/1721817315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181019b0 0x7f5818101e30 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 --2- 172.21.15.105:0/1721817315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58181007c0 0x7f5818100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 -- 172.21.15.105:0/1721817315 >> 172.21.15.105:0/1721817315 conn(0x7f58180fbd50 msgr2=0x7f58180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 -- 172.21.15.105:0/1721817315 shutdown_connections 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.043+0000 7f581e560640 1 -- 172.21.15.105:0/1721817315 wait complete. 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581e560640 1 Processor -- start 2024-03-26T18:01:37.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581e560640 1 -- start start 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581e560640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181007c0 0x7f58181958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581e560640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58181019b0 0x7f5818195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581e560640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5818102370 0x7f581819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581e560640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f581810b1f0 con 0x7f58181007c0 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581e560640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f581810b070 con 0x7f5818102370 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581e560640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f581810b370 con 0x7f58181019b0 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f5817fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181007c0 0x7f58181958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f5817fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181007c0 0x7f58181958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46888/0 (socket says 172.21.15.105:46888) 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f5817fff640 1 -- 172.21.15.105:0/1037422536 learned_addr learned my addr 172.21.15.105:0/1037422536 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f581cafd640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5818102370 0x7f581819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.044+0000 7f58177fe640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58181019b0 0x7f5818195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f5817fff640 1 -- 172.21.15.105:0/1037422536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58181019b0 msgr2=0x7f5818195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f5817fff640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58181019b0 0x7f5818195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f5817fff640 1 -- 172.21.15.105:0/1037422536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5818102370 msgr2=0x7f581819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f5817fff640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5818102370 0x7f581819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f5817fff640 1 -- 172.21.15.105:0/1037422536 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58181a1910 con 0x7f58181007c0 2024-03-26T18:01:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f5817fff640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181007c0 0x7f58181958e0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f580800e9c0 tx=0x7f580800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f58157fa640 1 -- 172.21.15.105:0/1037422536 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580800cd80 con 0x7f58181007c0 2024-03-26T18:01:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f58157fa640 1 -- 172.21.15.105:0/1037422536 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58080035f0 con 0x7f58181007c0 2024-03-26T18:01:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f581e560640 1 -- 172.21.15.105:0/1037422536 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58181a1c00 con 0x7f58181007c0 2024-03-26T18:01:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f58157fa640 1 -- 172.21.15.105:0/1037422536 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808010690 con 0x7f58181007c0 2024-03-26T18:01:37.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.045+0000 7f581e560640 1 -- 172.21.15.105:0/1037422536 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a2170 con 0x7f58181007c0 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f58157fa640 1 -- 172.21.15.105:0/1037422536 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f58080108f0 con 0x7f58181007c0 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f582121f740 1 Processor -- start 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f582121f740 1 -- start start 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f582121f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e972a30 0x55b25e972e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f582121f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e973420 0x55b25e977f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f582121f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e97aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f582121f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b25e8d7970 con 0x55b25e972a30 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f582121f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b25e87fd70 con 0x55b25e973420 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f582121f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b25e8d3920 con 0x55b25e978590 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f581cafd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e972a30 0x55b25e972e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f58177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e97aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f581cafd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e972a30 0x55b25e972e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46898/0 (socket says 172.21.15.105:46898) 2024-03-26T18:01:37.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f581cafd640 1 -- 172.21.15.105:0/4097002112 learned_addr learned my addr 172.21.15.105:0/4097002112 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f5817fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e973420 0x55b25e977f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.048+0000 7f58157fa640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58000a2fa0 0x7f58000a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f58177fe640 1 -- 172.21.15.105:0/4097002112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e973420 msgr2=0x55b25e977f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f58177fe640 1 --2- 172.21.15.105:0/4097002112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e973420 0x55b25e977f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f58177fe640 1 -- 172.21.15.105:0/4097002112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e972a30 msgr2=0x55b25e972e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f58177fe640 1 --2- 172.21.15.105:0/4097002112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e972a30 0x55b25e972e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f58177fe640 1 -- 172.21.15.105:0/4097002112 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25e5eb3f0 con 0x55b25e978590 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f581cafd640 1 --2- 172.21.15.105:0/4097002112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e972a30 0x55b25e972e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f581cafd640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58000a2fa0 0x7f58000a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f58157fa640 1 -- 172.21.15.105:0/1037422536 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f58080cf750 con 0x7f58181007c0 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f58177fe640 1 --2- 172.21.15.105:0/4097002112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e97aa20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f580c000c00 tx=0x7f580c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e5f3ed8e4e71eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f57fe7fc640 1 -- 172.21.15.105:0/4097002112 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c045020 con 0x55b25e978590 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f581cafd640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58000a2fa0 0x7f58000a5460 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f581000bfd0 tx=0x7f5810004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f57fe7fc640 1 -- 172.21.15.105:0/4097002112 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f580c004040 con 0x55b25e978590 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.049+0000 7f57fe7fc640 1 -- 172.21.15.105:0/4097002112 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c0055a0 con 0x55b25e978590 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 -- 172.21.15.105:0/4097002112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 msgr2=0x55b25e97aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 --2- 172.21.15.105:0/4097002112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e97aa20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f580c000c00 tx=0x7f580c009560 comp rx=0 tx=0).stop 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 -- 172.21.15.105:0/4097002112 shutdown_connections 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 --2- 172.21.15.105:0/4097002112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e97aa20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 --2- 172.21.15.105:0/4097002112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e973420 0x55b25e977f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 --2- 172.21.15.105:0/4097002112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e972a30 0x55b25e972e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 -- 172.21.15.105:0/4097002112 >> 172.21.15.105:0/4097002112 conn(0x55b25e96dfa0 msgr2=0x55b25e96e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 -- 172.21.15.105:0/4097002112 shutdown_connections 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 -- 172.21.15.105:0/4097002112 wait complete. 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.050+0000 7f582121f740 1 -- start start 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f582121f740 1 Processor -- start 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f582121f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f582121f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e973420 0x55b25e97d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f582121f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e978590 0x55b25e9844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f582121f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b25e97b4e0 con 0x55b25e978590 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f582121f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b25e893310 con 0x55b25e972a30 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f582121f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b25e97b660 con 0x55b25e973420 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f582121f740 10 client.0 _ll_register_callbacks cb 0x55b25e8cec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f581cafd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e978590 0x55b25e9844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f5817fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f581cafd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e978590 0x55b25e9844c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46904/0 (socket says 172.21.15.105:46904) 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f58177fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e973420 0x55b25e97d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f581cafd640 1 -- 172.21.15.105:0/3301566652 learned_addr learned my addr 172.21.15.105:0/3301566652 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f5817fff640 1 -- 172.21.15.105:0/3301566652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e973420 msgr2=0x55b25e97d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f5817fff640 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e973420 0x55b25e97d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f5817fff640 1 -- 172.21.15.105:0/3301566652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e978590 msgr2=0x55b25e9844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f5817fff640 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e978590 0x55b25e9844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f5817fff640 1 -- 172.21.15.105:0/3301566652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25e6b57d0 con 0x55b25e972a30 2024-03-26T18:01:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.051+0000 7f581cafd640 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e978590 0x55b25e9844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f5817fff640 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97cfb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f58080cd260 tx=0x7f5808009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=910ee8210f1a5ce0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f57f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:37.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f57f77fe640 1 -- 172.21.15.105:0/3301566652 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58080b6060 con 0x55b25e972a30 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f57f77fe640 1 -- 172.21.15.105:0/3301566652 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f58080037a0 con 0x55b25e972a30 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f582121f740 1 -- 172.21.15.105:0/3301566652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580c004040 con 0x55b25e972a30 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f57f77fe640 1 -- 172.21.15.105:0/3301566652 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f580809e040 con 0x55b25e972a30 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f582121f740 10 client.33452 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f582121f740 1 -- 172.21.15.105:0/3301566652 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b25e9878a0 con 0x55b25e972a30 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f57f77fe640 1 -- 172.21.15.105:0/3301566652 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f580c004040 con 0x55b25e972a30 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.052+0000 7f57f77fe640 1 -- 172.21.15.105:0/3301566652 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f580809c0a0 con 0x55b25e972a30 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f582121f740 10 client.33452 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f582121f740 10 client.33452 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f582121f740 20 client.33452 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f582121f740 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b25e989020 0x55b25e98b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f582121f740 4 client.33452 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f582121f740 1 -- 172.21.15.105:0/3301566652 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b25e8e4eb0 con 0x55b25e989020 2024-03-26T18:01:37.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f58177fe640 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b25e989020 0x55b25e98b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f58177fe640 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b25e989020 0x55b25e98b410 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=30cf93a5548f13b5 server_cookie=e139e0a838b4a1b9 in_seq=0 out_seq=0 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.053+0000 7f57f77fe640 10 client.33452 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f57f77fe640 1 -- 172.21.15.105:0/3301566652 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b25e8e4eb0 con 0x55b25e989020 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f57f77fe640 10 client.33452 handle_command_reply: tid=0 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 client.33452 shutdown 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 -- 172.21.15.105:0/3301566652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 msgr2=0x55b25e97cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97cfb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f58080cd260 tx=0x7f5808009700 comp rx=0 tx=0).stop 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 -- 172.21.15.105:0/3301566652 shutdown_connections 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b25e989020 0x55b25e98b410 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e978590 0x55b25e9844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e973420 0x55b25e97d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 --2- 172.21.15.105:0/3301566652 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97cfb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f57f77fe640 0 client.33452 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:01:37.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 -- 172.21.15.105:0/3301566652 >> 172.21.15.105:0/3301566652 conn(0x55b25e96dfa0 msgr2=0x55b25e96d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 -- 172.21.15.105:0/3301566652 shutdown_connections 2024-03-26T18:01:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 1 -- 172.21.15.105:0/3301566652 wait complete. 2024-03-26T18:01:37.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.054+0000 7f582121f740 20 client.33452 trim_cache size 0 max 16384 2024-03-26T18:01:37.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f582121f740 1 Processor -- start 2024-03-26T18:01:37.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f582121f740 1 -- start start 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f582121f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f582121f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e984450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f582121f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e971910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f582121f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58181aaed0 con 0x55b25e973420 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f582121f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58181aca10 con 0x55b25e972a30 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f582121f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58181ab5a0 con 0x55b25e978590 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f58177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e984450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f5817fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f5817fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55680/0 (socket says 172.21.15.105:55680) 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f58177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e984450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46912/0 (socket says 172.21.15.105:46912) 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.073+0000 7f5817fff640 1 -- 172.21.15.105:0/1167371880 learned_addr learned my addr 172.21.15.105:0/1167371880 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f581cafd640 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e971910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f58177fe640 1 -- 172.21.15.105:0/1167371880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 msgr2=0x55b25e971910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f58177fe640 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e971910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f58177fe640 1 -- 172.21.15.105:0/1167371880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 msgr2=0x55b25e97d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f58177fe640 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f58177fe640 1 -- 172.21.15.105:0/1167371880 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25e973800 con 0x55b25e973420 2024-03-26T18:01:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f5817fff640 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f58177fe640 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e984450 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f580c009900 tx=0x7f580c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7e28884b35efd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f57fdffb640 1 -- 172.21.15.105:0/1167371880 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c045040 con 0x55b25e973420 2024-03-26T18:01:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f57fdffb640 1 -- 172.21.15.105:0/1167371880 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f580c04e070 con 0x55b25e973420 2024-03-26T18:01:37.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.074+0000 7f57fdffb640 1 -- 172.21.15.105:0/1167371880 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c005040 con 0x55b25e973420 2024-03-26T18:01:37.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 -- 172.21.15.105:0/1167371880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 msgr2=0x55b25e984450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e984450 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f580c009900 tx=0x7f580c009650 comp rx=0 tx=0).stop 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 -- 172.21.15.105:0/1167371880 shutdown_connections 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e971910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e984450 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 --2- 172.21.15.105:0/1167371880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e97d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 -- 172.21.15.105:0/1167371880 >> 172.21.15.105:0/1167371880 conn(0x55b25e96dfa0 msgr2=0x55b25e971200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 -- 172.21.15.105:0/1167371880 shutdown_connections 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.075+0000 7f582121f740 1 -- 172.21.15.105:0/1167371880 wait complete. 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 1 -- start start 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 1 Processor -- start 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e982120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e982660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e983670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f58181ac340 con 0x55b25e973420 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58181abc70 con 0x55b25e972a30 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f58181a8eb0 con 0x55b25e978590 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f5817fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e982120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f5817fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e982120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55688/0 (socket says 172.21.15.105:55688) 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f5817fff640 1 -- 172.21.15.105:0/423404583 learned_addr learned my addr 172.21.15.105:0/423404583 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f58177fe640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e982660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f581cafd640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e983670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.076+0000 7f582121f740 10 client.0 _ll_register_callbacks cb 0x7f580809bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f5817fff640 1 -- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 msgr2=0x55b25e983670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f5817fff640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e983670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f5817fff640 1 -- 172.21.15.105:0/423404583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 msgr2=0x55b25e982660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f5817fff640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e982660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f5817fff640 1 -- 172.21.15.105:0/423404583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25e983c60 con 0x55b25e972a30 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f581cafd640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e983670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f58177fe640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e982660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f5817fff640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e982120 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f580800cfd0 tx=0x7f5808003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61f3d20514f4f12a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f57f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f57f77fe640 1 -- 172.21.15.105:0/423404583 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58080b6040 con 0x55b25e972a30 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f582121f740 1 -- 172.21.15.105:0/423404583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580c04e070 con 0x55b25e972a30 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f57f77fe640 1 -- 172.21.15.105:0/423404583 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f58080037a0 con 0x55b25e972a30 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f57f77fe640 1 -- 172.21.15.105:0/423404583 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f580809b980 con 0x55b25e972a30 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f582121f740 10 client.33458 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.077+0000 7f582121f740 1 -- 172.21.15.105:0/423404583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b25e9ef7b0 con 0x55b25e972a30 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f57f77fe640 1 -- 172.21.15.105:0/423404583 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f580c04e070 con 0x55b25e972a30 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f57f77fe640 1 -- 172.21.15.105:0/423404583 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5808083070 con 0x55b25e972a30 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f582121f740 10 client.33458 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f582121f740 10 client.33458 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:01:37.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f582121f740 20 client.33458 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f582121f740 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b25e9f0120 0x55b25e9f2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f582121f740 4 client.33458 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f582121f740 1 -- 172.21.15.105:0/423404583 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b25e8df200 con 0x55b25e9f0120 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.078+0000 7f58177fe640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b25e9f0120 0x55b25e9f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f58177fe640 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b25e9f0120 0x55b25e9f2540 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a687747df3c6de6e server_cookie=e3391e70e91a193a in_seq=0 out_seq=0 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f57f77fe640 10 client.33458 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f57f77fe640 1 -- 172.21.15.105:0/423404583 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b25e8df200 con 0x55b25e9f0120 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f57f77fe640 10 client.33458 handle_command_reply: tid=0 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 client.33458 shutdown 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 -- 172.21.15.105:0/423404583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 msgr2=0x55b25e982120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e982120 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f580800cfd0 tx=0x7f5808003d20 comp rx=0 tx=0).stop 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 -- 172.21.15.105:0/423404583 shutdown_connections 2024-03-26T18:01:37.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55b25e9f0120 0x55b25e9f2540 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b25e978590 0x55b25e983670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b25e973420 0x55b25e982660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f57f77fe640 0 client.33458 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 --2- 172.21.15.105:0/423404583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b25e972a30 0x55b25e982120 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 -- 172.21.15.105:0/423404583 >> 172.21.15.105:0/423404583 conn(0x55b25e96dfa0 msgr2=0x55b25e97d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 -- 172.21.15.105:0/423404583 shutdown_connections 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 1 -- 172.21.15.105:0/423404583 wait complete. 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.079+0000 7f582121f740 20 client.33458 trim_cache size 0 max 16384 2024-03-26T18:01:37.067 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 -- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58000a2fa0 msgr2=0x7f58000a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58000a2fa0 0x7f58000a5460 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f581000bfd0 tx=0x7f5810004a40 comp rx=0 tx=0).stop 2024-03-26T18:01:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 -- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181007c0 msgr2=0x7f58181958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181007c0 0x7f58181958e0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f580800e9c0 tx=0x7f580800ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 -- 172.21.15.105:0/1037422536 shutdown_connections 2024-03-26T18:01:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f58000a2fa0 0x7f58000a5460 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5818102370 0x7f581819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58181019b0 0x7f5818195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 --2- 172.21.15.105:0/1037422536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f58181007c0 0x7f58181958e0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 -- 172.21.15.105:0/1037422536 >> 172.21.15.105:0/1037422536 conn(0x7f58180fbd50 msgr2=0x7f5818104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 -- 172.21.15.105:0/1037422536 shutdown_connections 2024-03-26T18:01:37.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.080+0000 7f581e560640 1 -- 172.21.15.105:0/1037422536 wait complete. 2024-03-26T18:01:37.074 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:36 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/580314022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:37.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T18:01:37.087 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:36 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/580314022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:37.154 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:36 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/580314022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:01:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 -- 172.21.15.105:0/520817287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 msgr2=0x7f3148101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 --2- 172.21.15.105:0/520817287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 0x7f3148101d10 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f3144009900 tx=0x7f314402bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 -- 172.21.15.105:0/520817287 shutdown_connections 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 --2- 172.21.15.105:0/520817287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31481034a0 0x7f3148109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 --2- 172.21.15.105:0/520817287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3148102ae0 0x7f3148102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 --2- 172.21.15.105:0/520817287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 0x7f3148101d10 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 -- 172.21.15.105:0/520817287 >> 172.21.15.105:0/520817287 conn(0x7f31480fce60 msgr2=0x7f31480ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 -- 172.21.15.105:0/520817287 shutdown_connections 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 -- 172.21.15.105:0/520817287 wait complete. 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.229+0000 7f3150352640 1 Processor -- start 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f3150352640 1 -- start start 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f3150352640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 0x7f3148075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f3150352640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148102ae0 0x7f3148073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f3150352640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31481034a0 0x7f31480741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f3150352640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f314810d4c0 con 0x7f31481018f0 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f3150352640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f314810d340 con 0x7f3148102ae0 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f3150352640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f314810d640 con 0x7f31481034a0 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 0x7f3148075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 0x7f3148075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46932/0 (socket says 172.21.15.105:46932) 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 -- 172.21.15.105:0/2513310670 learned_addr learned my addr 172.21.15.105:0/2513310670 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314d8ed640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148102ae0 0x7f3148073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e8ef640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31481034a0 0x7f31480741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 -- 172.21.15.105:0/2513310670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31481034a0 msgr2=0x7f31480741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31481034a0 0x7f31480741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 -- 172.21.15.105:0/2513310670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148102ae0 msgr2=0x7f3148073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148102ae0 0x7f3148073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 -- 172.21.15.105:0/2513310670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3148074a80 con 0x7f31481018f0 2024-03-26T18:01:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f314e0ee640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 0x7f3148075670 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f3144005ab0 tx=0x7f31440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.230+0000 7f313f7fe640 1 -- 172.21.15.105:0/2513310670 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144043020 con 0x7f31481018f0 2024-03-26T18:01:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.231+0000 7f3150352640 1 -- 172.21.15.105:0/2513310670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31481a1cc0 con 0x7f31481018f0 2024-03-26T18:01:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.231+0000 7f313f7fe640 1 -- 172.21.15.105:0/2513310670 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31440054c0 con 0x7f31481018f0 2024-03-26T18:01:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.231+0000 7f313f7fe640 1 -- 172.21.15.105:0/2513310670 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144003d10 con 0x7f31481018f0 2024-03-26T18:01:37.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.231+0000 7f3150352640 1 -- 172.21.15.105:0/2513310670 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31481a21b0 con 0x7f31481018f0 2024-03-26T18:01:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.234+0000 7f313f7fe640 1 -- 172.21.15.105:0/2513310670 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3144046020 con 0x7f31481018f0 2024-03-26T18:01:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.234+0000 7f315300e740 1 Processor -- start 2024-03-26T18:01:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.234+0000 7f315300e740 1 -- start start 2024-03-26T18:01:37.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f315300e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c5a30 0x55d0f74c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f315300e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f315300e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74cb590 0x55d0f74cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f315300e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d0f742a970 con 0x55d0f74cb590 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f315300e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0f73d2d70 con 0x55d0f74c5a30 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f315300e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0f7426920 con 0x55d0f74c6420 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f313f7fe640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f31340a2f20 0x7f31340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e0ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e8ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c5a30 0x55d0f74c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e8ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c5a30 0x55d0f74c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55702/0 (socket says 172.21.15.105:55702) 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e8ef640 1 -- 172.21.15.105:0/2057170621 learned_addr learned my addr 172.21.15.105:0/2057170621 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f313f7fe640 1 -- 172.21.15.105:0/2513310670 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f31440f1670 con 0x7f31481018f0 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e8ef640 1 -- 172.21.15.105:0/2057170621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 msgr2=0x55d0f74caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e8ef640 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e8ef640 1 -- 172.21.15.105:0/2057170621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74cb590 msgr2=0x55d0f74cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314d8ed640 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74cb590 0x55d0f74cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e8ef640 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74cb590 0x55d0f74cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314e8ef640 1 -- 172.21.15.105:0/2057170621 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f713e3f0 con 0x55d0f74c5a30 2024-03-26T18:01:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.235+0000 7f314d8ed640 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74cb590 0x55d0f74cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:01:37.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.236+0000 7f314e8ef640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f31340a2f20 0x7f31340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.236+0000 7f314e8ef640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f31340a2f20 0x7f31340a53e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3140017000 tx=0x7f3140019000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.236+0000 7f314e8ef640 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c5a30 0x55d0f74c5e50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f314000a500 tx=0x7f314000a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5a6c1a2177f0a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.237+0000 7f313cbfe640 1 -- 172.21.15.105:0/2057170621 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314001a040 con 0x55d0f74c5a30 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.237+0000 7f313cbfe640 1 -- 172.21.15.105:0/2057170621 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3140023070 con 0x55d0f74c5a30 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.237+0000 7f313cbfe640 1 -- 172.21.15.105:0/2057170621 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314001eb20 con 0x55d0f74c5a30 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 -- 172.21.15.105:0/2057170621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c5a30 msgr2=0x55d0f74c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c5a30 0x55d0f74c5e50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f314000a500 tx=0x7f314000a9c0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 -- 172.21.15.105:0/2057170621 shutdown_connections 2024-03-26T18:01:37.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74cb590 0x55d0f74cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 --2- 172.21.15.105:0/2057170621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c5a30 0x55d0f74c5e50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 -- 172.21.15.105:0/2057170621 >> 172.21.15.105:0/2057170621 conn(0x55d0f74c0fa0 msgr2=0x55d0f74c13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 -- 172.21.15.105:0/2057170621 shutdown_connections 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 -- 172.21.15.105:0/2057170621 wait complete. 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 -- start start 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.238+0000 7f315300e740 1 Processor -- start 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f315300e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f315300e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f315300e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74cb590 0x55d0f74d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f315300e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d0f74ce4e0 con 0x55d0f74c5a30 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f315300e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0f73e6310 con 0x55d0f74c6420 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f315300e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0f74ce660 con 0x55d0f74cb590 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314d8ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314e8ef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74cb590 0x55d0f74d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f315300e740 10 client.0 _ll_register_callbacks cb 0x55d0f7421c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314e8ef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74cb590 0x55d0f74d72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38026/0 (socket says 172.21.15.105:38026) 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314e8ef640 1 -- 172.21.15.105:0/3439462072 learned_addr learned my addr 172.21.15.105:0/3439462072 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314e0ee640 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314d8ed640 1 -- 172.21.15.105:0/3439462072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74cb590 msgr2=0x55d0f74d72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314d8ed640 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74cb590 0x55d0f74d72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314d8ed640 1 -- 172.21.15.105:0/3439462072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 msgr2=0x55d0f74cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314d8ed640 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314d8ed640 1 -- 172.21.15.105:0/3439462072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f72087d0 con 0x55d0f74c6420 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314e0ee640 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f314d8ed640 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f31480753e0 tx=0x7f313800bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f8ba3e75369b77 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.239+0000 7f3131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f3131ffb640 1 -- 172.21.15.105:0/3439462072 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313801b670 con 0x55d0f74c6420 2024-03-26T18:01:37.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f3131ffb640 1 -- 172.21.15.105:0/3439462072 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f313801b810 con 0x55d0f74c6420 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 1 -- 172.21.15.105:0/3439462072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3140023070 con 0x55d0f74c6420 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f3131ffb640 1 -- 172.21.15.105:0/3439462072 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f313801ba10 con 0x55d0f74c6420 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 10 client.33467 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 1 -- 172.21.15.105:0/3439462072 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d0f74da5f0 con 0x55d0f74c6420 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f3131ffb640 1 -- 172.21.15.105:0/3439462072 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3140023070 con 0x55d0f74c6420 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f3131ffb640 1 -- 172.21.15.105:0/3439462072 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3138026420 con 0x55d0f74c6420 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 10 client.33467 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 10 client.33467 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 20 client.33467 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d0f74dbcc0 0x55d0f74de120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 4 client.33467 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.240+0000 7f315300e740 1 -- 172.21.15.105:0/3439462072 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0f7437eb0 con 0x55d0f74dbcc0 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.241+0000 7f314e0ee640 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d0f74dbcc0 0x55d0f74de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.241+0000 7f314e0ee640 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d0f74dbcc0 0x55d0f74de120 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49c80f63b3b5af8a server_cookie=22a8ab31d45c77a8 in_seq=0 out_seq=0 2024-03-26T18:01:37.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.241+0000 7f3131ffb640 10 client.33467 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f3131ffb640 1 -- 172.21.15.105:0/3439462072 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d0f7437eb0 con 0x55d0f74dbcc0 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f3131ffb640 10 client.33467 handle_command_reply: tid=0 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 client.33467 shutdown 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 -- 172.21.15.105:0/3439462072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c6420 msgr2=0x55d0f74d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f31480753e0 tx=0x7f313800bda0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 -- 172.21.15.105:0/3439462072 shutdown_connections 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74cb590 0x55d0f74d72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d0f74dbcc0 0x55d0f74de120 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f3131ffb640 0 client.33467 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 --2- 172.21.15.105:0/3439462072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 -- 172.21.15.105:0/3439462072 >> 172.21.15.105:0/3439462072 conn(0x55d0f74c0fa0 msgr2=0x55d0f74c01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.242+0000 7f315300e740 1 -- 172.21.15.105:0/3439462072 shutdown_connections 2024-03-26T18:01:37.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.243+0000 7f315300e740 1 -- 172.21.15.105:0/3439462072 wait complete. 2024-03-26T18:01:37.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.243+0000 7f315300e740 20 client.33467 trim_cache size 0 max 16384 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.261+0000 7f315300e740 1 Processor -- start 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.261+0000 7f315300e740 1 -- start start 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.261+0000 7f315300e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.261+0000 7f315300e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.261+0000 7f315300e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 0x55d0f74d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.261+0000 7f315300e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31481ab030 con 0x55d0f74c5a30 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.261+0000 7f315300e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31481acb70 con 0x55d0f74cb590 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.261+0000 7f315300e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31481ab700 con 0x55d0f74c6420 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46958/0 (socket says 172.21.15.105:46958) 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314d8ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 -- 172.21.15.105:0/1673959281 learned_addr learned my addr 172.21.15.105:0/1673959281 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e8ef640 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 0x55d0f74d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 -- 172.21.15.105:0/1673959281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 msgr2=0x55d0f74d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 -- 172.21.15.105:0/1673959281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 msgr2=0x55d0f74d7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 0x55d0f74d7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 -- 172.21.15.105:0/1673959281 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0f74c1d10 con 0x55d0f74c5a30 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314d8ed640 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f314e0ee640 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f3144005790 tx=0x7f31440ee040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f12d26b5d20a6d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f3133fff640 1 -- 172.21.15.105:0/1673959281 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31440044e0 con 0x55d0f74c5a30 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f3133fff640 1 -- 172.21.15.105:0/1673959281 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3144004c50 con 0x55d0f74c5a30 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.262+0000 7f3133fff640 1 -- 172.21.15.105:0/1673959281 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31440bc180 con 0x55d0f74c5a30 2024-03-26T18:01:37.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 -- 172.21.15.105:0/1673959281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 msgr2=0x55d0f74cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f3144005790 tx=0x7f31440ee040 comp rx=0 tx=0).stop 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 -- 172.21.15.105:0/1673959281 shutdown_connections 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 0x55d0f74d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c6420 0x55d0f74d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 --2- 172.21.15.105:0/1673959281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c5a30 0x55d0f74cfd90 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 -- 172.21.15.105:0/1673959281 >> 172.21.15.105:0/1673959281 conn(0x55d0f74c0fa0 msgr2=0x55d0f74bc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 -- 172.21.15.105:0/1673959281 shutdown_connections 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 -- 172.21.15.105:0/1673959281 wait complete. 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 -- start start 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.263+0000 7f315300e740 1 Processor -- start 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f315300e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c5a30 0x55d0f74d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f315300e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c6420 0x55d0f74d4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f315300e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 0x55d0f74d5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f315300e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31481ac4a0 con 0x55d0f74c6420 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f315300e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31481abdd0 con 0x55d0f74cb590 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f315300e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31481a9010 con 0x55d0f74c5a30 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f315300e740 10 client.0 _ll_register_callbacks cb 0x7f3138004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c6420 0x55d0f74d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314e0ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c5a30 0x55d0f74d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c6420 0x55d0f74d4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46972/0 (socket says 172.21.15.105:46972) 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 -- 172.21.15.105:0/2754999596 learned_addr learned my addr 172.21.15.105:0/2754999596 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314e8ef640 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 0x55d0f74d5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 -- 172.21.15.105:0/2754999596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c5a30 msgr2=0x55d0f74d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c5a30 0x55d0f74d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 -- 172.21.15.105:0/2754999596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 msgr2=0x55d0f74d5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 0x55d0f74d5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 -- 172.21.15.105:0/2754999596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0f74d5f10 con 0x55d0f74c6420 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f314d8ed640 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c6420 0x55d0f74d4910 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x55d0f74d1200 tx=0x7f3138026e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0ac4d687d96831 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f3131ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.264+0000 7f3131ffb640 1 -- 172.21.15.105:0/2754999596 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138036060 con 0x55d0f74c6420 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f3131ffb640 1 -- 172.21.15.105:0/2754999596 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f3138004320 con 0x55d0f74c6420 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 1 -- 172.21.15.105:0/2754999596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3144004c50 con 0x55d0f74c6420 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f3131ffb640 1 -- 172.21.15.105:0/2754999596 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3138026420 con 0x55d0f74c6420 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 10 client.23802 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 1 -- 172.21.15.105:0/2754999596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d0f74d72b0 con 0x55d0f74c6420 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f3131ffb640 1 -- 172.21.15.105:0/2754999596 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3144004c50 con 0x55d0f74c6420 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f3131ffb640 1 -- 172.21.15.105:0/2754999596 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3138005100 con 0x55d0f74c6420 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 10 client.23802 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 10 client.23802 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 20 client.23802 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:37.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d0f74dbcc0 0x55d0f75444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 4 client.23802 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f315300e740 1 -- 172.21.15.105:0/2754999596 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0f7432200 con 0x55d0f74dbcc0 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.265+0000 7f314e0ee640 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d0f74dbcc0 0x55d0f75444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f314e0ee640 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d0f74dbcc0 0x55d0f75444a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f286c20b1048627 server_cookie=9f48c821c975aec2 in_seq=0 out_seq=0 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f3131ffb640 10 client.23802 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f3131ffb640 1 -- 172.21.15.105:0/2754999596 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0f7432200 con 0x55d0f74dbcc0 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f3131ffb640 10 client.23802 handle_command_reply: tid=0 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 client.23802 shutdown 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 -- 172.21.15.105:0/2754999596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c6420 msgr2=0x55d0f74d4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c6420 0x55d0f74d4910 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x55d0f74d1200 tx=0x7f3138026e10 comp rx=0 tx=0).stop 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 -- 172.21.15.105:0/2754999596 shutdown_connections 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55d0f74dbcc0 0x55d0f75444a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0f74cb590 0x55d0f74d5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.254 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f3131ffb640 0 client.23802 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:01:37.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0f74c6420 0x55d0f74d4910 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 --2- 172.21.15.105:0/2754999596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0f74c5a30 0x55d0f74d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 -- 172.21.15.105:0/2754999596 >> 172.21.15.105:0/2754999596 conn(0x55d0f74c0fa0 msgr2=0x55d0f74cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 -- 172.21.15.105:0/2754999596 shutdown_connections 2024-03-26T18:01:37.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 1 -- 172.21.15.105:0/2754999596 wait complete. 2024-03-26T18:01:37.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.266+0000 7f315300e740 20 client.23802 trim_cache size 0 max 16384 2024-03-26T18:01:37.255 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 -- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f31340a2f20 msgr2=0x7f31340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f31340a2f20 0x7f31340a53e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3140017000 tx=0x7f3140019000 comp rx=0 tx=0).stop 2024-03-26T18:01:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 -- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 msgr2=0x7f3148075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 0x7f3148075670 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f3144005ab0 tx=0x7f31440042f0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 -- 172.21.15.105:0/2513310670 shutdown_connections 2024-03-26T18:01:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f31340a2f20 0x7f31340a53e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31481034a0 0x7f31480741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.256 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3148102ae0 0x7f3148073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 --2- 172.21.15.105:0/2513310670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31481018f0 0x7f3148075670 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 -- 172.21.15.105:0/2513310670 >> 172.21.15.105:0/2513310670 conn(0x7f31480fce60 msgr2=0x7f3148105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 -- 172.21.15.105:0/2513310670 shutdown_connections 2024-03-26T18:01:37.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.267+0000 7f3150352640 1 -- 172.21.15.105:0/2513310670 wait complete. 2024-03-26T18:01:37.263 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T18:01:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.413+0000 7fa3f8259640 1 -- 172.21.15.105:0/3079168120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3f0102ac0 msgr2=0x7fa3f0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.413+0000 7fa3f8259640 1 --2- 172.21.15.105:0/3079168120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3f0102ac0 0x7fa3f0102f40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009900 tx=0x7fa3e002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:37.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.413+0000 7fa3f8259640 1 -- 172.21.15.105:0/3079168120 shutdown_connections 2024-03-26T18:01:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.413+0000 7fa3f8259640 1 --2- 172.21.15.105:0/3079168120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3f0103480 0x7fa3f0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.413+0000 7fa3f8259640 1 --2- 172.21.15.105:0/3079168120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3f0102ac0 0x7fa3f0102f40 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.413+0000 7fa3f8259640 1 --2- 172.21.15.105:0/3079168120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f01018d0 0x7fa3f0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.413+0000 7fa3f8259640 1 -- 172.21.15.105:0/3079168120 >> 172.21.15.105:0/3079168120 conn(0x7fa3f00fce20 msgr2=0x7fa3f00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.414+0000 7fa3f8259640 1 -- 172.21.15.105:0/3079168120 shutdown_connections 2024-03-26T18:01:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.414+0000 7fa3f8259640 1 -- 172.21.15.105:0/3079168120 wait complete. 2024-03-26T18:01:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.414+0000 7fa3f8259640 1 Processor -- start 2024-03-26T18:01:37.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.414+0000 7fa3f8259640 1 -- start start 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.414+0000 7fa3f8259640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3f01018d0 0x7fa3f0199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.414+0000 7fa3f8259640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f0102ac0 0x7fa3f019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.414+0000 7fa3f8259640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3f0103480 0x7fa3f01a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f8259640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f010d400 con 0x7fa3f01018d0 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f8259640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f010d280 con 0x7fa3f0103480 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f8259640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f010d580 con 0x7fa3f0102ac0 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3f01018d0 0x7fa3f0199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f67f6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3f0103480 0x7fa3f01a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3f01018d0 0x7fa3f0199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46996/0 (socket says 172.21.15.105:46996) 2024-03-26T18:01:37.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/1871838674 learned_addr learned my addr 172.21.15.105:0/1871838674 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f0102ac0 0x7fa3f019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/1871838674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f0102ac0 msgr2=0x7fa3f019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f0102ac0 0x7fa3f019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/1871838674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3f0103480 msgr2=0x7fa3f01a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3f0103480 0x7fa3f01a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/1871838674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f01a1990 con 0x7fa3f01018d0 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f0102ac0 0x7fa3f019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3f01018d0 0x7fa3f0199d70 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00e9c0 tx=0x7fa3ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3e6ffd640 1 -- 172.21.15.105:0/1871838674 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00cd80 con 0x7fa3f01018d0 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.415+0000 7fa3e6ffd640 1 -- 172.21.15.105:0/1871838674 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3ec0035f0 con 0x7fa3f01018d0 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.416+0000 7fa3f8259640 1 -- 172.21.15.105:0/1871838674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01a1c80 con 0x7fa3f01018d0 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.416+0000 7fa3e6ffd640 1 -- 172.21.15.105:0/1871838674 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec010690 con 0x7fa3f01018d0 2024-03-26T18:01:37.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.416+0000 7fa3f8259640 1 -- 172.21.15.105:0/1871838674 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f01a21a0 con 0x7fa3f01018d0 2024-03-26T18:01:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.418+0000 7fa3e6ffd640 1 -- 172.21.15.105:0/1871838674 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa3ec0037a0 con 0x7fa3f01018d0 2024-03-26T18:01:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.419+0000 7fa3fae43740 1 Processor -- start 2024-03-26T18:01:37.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.419+0000 7fa3fae43740 1 -- start start 2024-03-26T18:01:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.419+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1bfa30 0x55d0df1bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.419+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c0420 0x55d0df1c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c5590 0x55d0df1c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1bfa30 0x55d0df1bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1bfa30 0x55d0df1bfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55754/0 (socket says 172.21.15.105:55754) 2024-03-26T18:01:37.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 -- 172.21.15.105:0/1650275312 learned_addr learned my addr 172.21.15.105:0/1650275312 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c0420 0x55d0df1c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3e6ffd640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3dc0a2fa0 0x7fa3dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3fae43740 1 -- 172.21.15.105:0/1650275312 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d0df124970 con 0x55d0df1c5590 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3fae43740 1 -- 172.21.15.105:0/1650275312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0df0ccd70 con 0x55d0df1bfa30 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3fae43740 1 -- 172.21.15.105:0/1650275312 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0df120920 con 0x55d0df1c0420 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c5590 0x55d0df1c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3dc0a2fa0 0x7fa3dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3e6ffd640 1 -- 172.21.15.105:0/1871838674 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fa3ec0cf930 con 0x7fa3f01018d0 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 -- 172.21.15.105:0/1650275312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c0420 msgr2=0x55d0df1c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c0420 0x55d0df1c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 -- 172.21.15.105:0/1650275312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c5590 msgr2=0x55d0df1c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c5590 0x55d0df1c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f67f6640 1 -- 172.21.15.105:0/1650275312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0dee383f0 con 0x55d0df1bfa30 2024-03-26T18:01:37.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.420+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c5590 0x55d0df1c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.421+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1bfa30 0x55d0df1bfe50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005c40 tx=0x7fa3e800f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18ae60e45e25cf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.421+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3dc0a2fa0 0x7fa3dc0a5460 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8008000 tx=0x7fa3e801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.421+0000 7fa3dbfff640 1 -- 172.21.15.105:0/1650275312 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800fd30 con 0x55d0df1bfa30 2024-03-26T18:01:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.421+0000 7fa3dbfff640 1 -- 172.21.15.105:0/1650275312 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3e8022030 con 0x55d0df1bfa30 2024-03-26T18:01:37.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.421+0000 7fa3dbfff640 1 -- 172.21.15.105:0/1650275312 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80108a0 con 0x55d0df1bfa30 2024-03-26T18:01:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 -- 172.21.15.105:0/1650275312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1bfa30 msgr2=0x55d0df1bfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1bfa30 0x55d0df1bfe50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005c40 tx=0x7fa3e800f3c0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 -- 172.21.15.105:0/1650275312 shutdown_connections 2024-03-26T18:01:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c5590 0x55d0df1c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c0420 0x55d0df1c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 --2- 172.21.15.105:0/1650275312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1bfa30 0x55d0df1bfe50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 -- 172.21.15.105:0/1650275312 >> 172.21.15.105:0/1650275312 conn(0x55d0df1bafa0 msgr2=0x55d0df1bb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 -- 172.21.15.105:0/1650275312 shutdown_connections 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 -- 172.21.15.105:0/1650275312 wait complete. 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.422+0000 7fa3fae43740 1 -- start start 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3fae43740 1 Processor -- start 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1bfa30 0x55d0df1c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c0420 0x55d0df1ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c5590 0x55d0df1d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3fae43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d0df1c8530 con 0x55d0df1c0420 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3f5ff5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1bfa30 0x55d0df1c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3f57f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c0420 0x55d0df1ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3f5ff5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1bfa30 0x55d0df1c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38092/0 (socket says 172.21.15.105:38092) 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/3221631672 learned_addr learned my addr 172.21.15.105:0/3221631672 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d0df0e0310 con 0x55d0df1c5590 2024-03-26T18:01:37.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c5590 0x55d0df1d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d0df1c86b0 con 0x55d0df1bfa30 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.423+0000 7fa3fae43740 10 client.0 _ll_register_callbacks cb 0x55d0df11bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/3221631672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c5590 msgr2=0x55d0df1d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c5590 0x55d0df1d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/3221631672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c0420 msgr2=0x55d0df1ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c0420 0x55d0df1ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/3221631672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0def027d0 con 0x55d0df1bfa30 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c0420 0x55d0df1ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1bfa30 0x55d0df1c9d90 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec003ec0 tx=0x7fa3ec09b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c872b272a2d925 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3d97fa640 1 -- 172.21.15.105:0/3221631672 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec09bfc0 con 0x55d0df1bfa30 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3d97fa640 1 -- 172.21.15.105:0/3221631672 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fa3ec09b410 con 0x55d0df1bfa30 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e8022030 con 0x55d0df1bfa30 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3d97fa640 1 -- 172.21.15.105:0/3221631672 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa3ec0a0110 con 0x55d0df1bfa30 2024-03-26T18:01:37.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.424+0000 7fa3fae43740 10 client.33481 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d0df1d4680 con 0x55d0df1bfa30 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3d97fa640 1 -- 172.21.15.105:0/3221631672 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3e8022030 con 0x55d0df1bfa30 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3d97fa640 1 -- 172.21.15.105:0/3221631672 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa3ec09e070 con 0x55d0df1bfa30 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3fae43740 10 client.33481 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3fae43740 10 client.33481 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3fae43740 20 client.33481 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3fae43740 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d0df1d5cf0 0x55d0df1d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3fae43740 4 client.33481 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0df131eb0 con 0x55d0df1d5cf0 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.425+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d0df1d5cf0 0x55d0df1d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.426+0000 7fa3d97fa640 10 client.33481 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.426+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d0df1d5cf0 0x55d0df1d8120 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1931c8f042378284 server_cookie=9a322f2440a3b7d in_seq=0 out_seq=0 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.426+0000 7fa3d97fa640 1 -- 172.21.15.105:0/3221631672 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d0df131eb0 con 0x55d0df1d5cf0 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.426+0000 7fa3d97fa640 10 client.33481 handle_command_reply: tid=0 2024-03-26T18:01:37.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.426+0000 7fa3fae43740 1 client.33481 shutdown 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1bfa30 msgr2=0x55d0df1c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1bfa30 0x55d0df1c9d90 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec003ec0 tx=0x7fa3ec09b920 comp rx=0 tx=0).stop 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 shutdown_connections 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d0df1d5cf0 0x55d0df1d8120 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c5590 0x55d0df1d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1c0420 0x55d0df1ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 --2- 172.21.15.105:0/3221631672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1bfa30 0x55d0df1c9d90 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3d97fa640 0 client.33481 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 >> 172.21.15.105:0/3221631672 conn(0x55d0df1bafa0 msgr2=0x55d0df1ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 shutdown_connections 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 1 -- 172.21.15.105:0/3221631672 wait complete. 2024-03-26T18:01:37.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.427+0000 7fa3fae43740 20 client.33481 trim_cache size 0 max 16384 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.443+0000 7fa3fae43740 1 Processor -- start 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.443+0000 7fa3fae43740 1 -- start start 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.443+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1c0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.443+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.443+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1ca230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.443+0000 7fa3fae43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01a6b70 con 0x55d0df1bfa30 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.443+0000 7fa3fae43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01a86b0 con 0x55d0df1c0420 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.443+0000 7fa3fae43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01a7240 con 0x55d0df1c5590 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1c0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f57f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1c0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47018/0 (socket says 172.21.15.105:47018) 2024-03-26T18:01:37.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f67f6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/2681063853 learned_addr learned my addr 172.21.15.105:0/2681063853 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/2681063853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 msgr2=0x55d0df1ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/2681063853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 msgr2=0x55d0df1d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/2681063853 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0df1ba8f0 con 0x55d0df1bfa30 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1c0ee0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec09a270 tx=0x7fa3ec097c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfad28f4a40659b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.444+0000 7fa3db7fe640 1 -- 172.21.15.105:0/2681063853 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec003b10 con 0x55d0df1bfa30 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3db7fe640 1 -- 172.21.15.105:0/2681063853 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3ec09bfc0 con 0x55d0df1bfa30 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3db7fe640 1 -- 172.21.15.105:0/2681063853 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec098090 con 0x55d0df1bfa30 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3fae43740 1 -- 172.21.15.105:0/2681063853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 msgr2=0x55d0df1c0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3fae43740 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1c0ee0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec09a270 tx=0x7fa3ec097c60 comp rx=0 tx=0).stop 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3fae43740 1 -- 172.21.15.105:0/2681063853 shutdown_connections 2024-03-26T18:01:37.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3fae43740 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3fae43740 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3fae43740 1 --2- 172.21.15.105:0/2681063853 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1c0ee0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3fae43740 1 -- 172.21.15.105:0/2681063853 >> 172.21.15.105:0/2681063853 conn(0x55d0df1bafa0 msgr2=0x55d0df1b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.445+0000 7fa3fae43740 1 -- 172.21.15.105:0/2681063853 shutdown_connections 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 -- 172.21.15.105:0/2681063853 wait complete. 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 -- start start 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 Processor -- start 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01a7fe0 con 0x55d0df1bfa30 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01a7910 con 0x55d0df1c0420 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01a4b90 con 0x55d0df1c5590 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3fae43740 10 client.0 _ll_register_callbacks cb 0x7fa3ec009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3f5ff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3f57f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3f5ff5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47034/0 (socket says 172.21.15.105:47034) 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/4065288967 learned_addr learned my addr 172.21.15.105:0/4065288967 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.446+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 msgr2=0x55d0df1cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/4065288967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 msgr2=0x55d0df1cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3f5ff5640 1 -- 172.21.15.105:0/4065288967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0df1d0230 con 0x55d0df1bfa30 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3f67f6640 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3f5ff5640 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1ce6f0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec09afc0 tx=0x7fa3ec09b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217e79f766f94c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3d97fa640 1 -- 172.21.15.105:0/4065288967 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec09bd90 con 0x55d0df1bfa30 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3d97fa640 1 -- 172.21.15.105:0/4065288967 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7fa3ec0a14a0 con 0x55d0df1bfa30 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3fae43740 1 -- 172.21.15.105:0/4065288967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3ec09bfc0 con 0x55d0df1bfa30 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3d97fa640 1 -- 172.21.15.105:0/4065288967 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa3ec003b10 con 0x55d0df1bfa30 2024-03-26T18:01:37.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3fae43740 10 client.23820 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.447+0000 7fa3fae43740 1 -- 172.21.15.105:0/4065288967 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d0df1d0230 con 0x55d0df1bfa30 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3d97fa640 1 -- 172.21.15.105:0/4065288967 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3ec09bfc0 con 0x55d0df1bfa30 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3d97fa640 1 -- 172.21.15.105:0/4065288967 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa3ec0b6030 con 0x55d0df1bfa30 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3fae43740 10 client.23820 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3fae43740 10 client.23820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3fae43740 20 client.23820 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3fae43740 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d0df23c1f0 0x55d0df23e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3fae43740 4 client.23820 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3fae43740 1 -- 172.21.15.105:0/4065288967 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0df12c200 con 0x55d0df23c1f0 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d0df23c1f0 0x55d0df23e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3d97fa640 10 client.23820 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:01:37.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.448+0000 7fa3f57f4640 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d0df23c1f0 0x55d0df23e5e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fcc26e468eaf04a server_cookie=f9653bfc64f95e71 in_seq=0 out_seq=0 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3d97fa640 1 -- 172.21.15.105:0/4065288967 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0df12c200 con 0x55d0df23c1f0 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3d97fa640 10 client.23820 handle_command_reply: tid=0 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3fae43740 1 client.23820 shutdown 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3fae43740 1 -- 172.21.15.105:0/4065288967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 msgr2=0x55d0df1ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3fae43740 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1ce6f0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec09afc0 tx=0x7fa3ec09b6c0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3fae43740 1 -- 172.21.15.105:0/4065288967 shutdown_connections 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3fae43740 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55d0df23c1f0 0x55d0df23e5e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3d97fa640 0 client.23820 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3fae43740 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d0df1c5590 0x55d0df1cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.449+0000 7fa3fae43740 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d0df1c0420 0x55d0df1cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3fae43740 1 --2- 172.21.15.105:0/4065288967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d0df1bfa30 0x55d0df1ce6f0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3fae43740 1 -- 172.21.15.105:0/4065288967 >> 172.21.15.105:0/4065288967 conn(0x55d0df1bafa0 msgr2=0x55d0df1ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3fae43740 1 -- 172.21.15.105:0/4065288967 shutdown_connections 2024-03-26T18:01:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3fae43740 1 -- 172.21.15.105:0/4065288967 wait complete. 2024-03-26T18:01:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3fae43740 20 client.23820 trim_cache size 0 max 16384 2024-03-26T18:01:37.438 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 -- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3dc0a2fa0 msgr2=0x7fa3dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3dc0a2fa0 0x7fa3dc0a5460 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8008000 tx=0x7fa3e801f000 comp rx=0 tx=0).stop 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 -- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3f01018d0 msgr2=0x7fa3f0199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3f01018d0 0x7fa3f0199d70 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00e9c0 tx=0x7fa3ec00ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 -- 172.21.15.105:0/1871838674 shutdown_connections 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3dc0a2fa0 0x7fa3dc0a5460 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3f0103480 0x7fa3f01a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f0102ac0 0x7fa3f019a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 --2- 172.21.15.105:0/1871838674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3f01018d0 0x7fa3f0199d70 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 -- 172.21.15.105:0/1871838674 >> 172.21.15.105:0/1871838674 conn(0x7fa3f00fce20 msgr2=0x7fa3f0108260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 -- 172.21.15.105:0/1871838674 shutdown_connections 2024-03-26T18:01:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.450+0000 7fa3f8259640 1 -- 172.21.15.105:0/1871838674 wait complete. 2024-03-26T18:01:37.443 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T18:01:37.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81206 7.05 MB/sec execute 500 sec latency 141.920 ms 2024-03-26T18:01:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 -- 172.21.15.105:0/737450013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1007c0 msgr2=0x7f351c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 --2- 172.21.15.105:0/737450013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1007c0 0x7f351c100be0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3510009900 tx=0x7f351002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:37.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 -- 172.21.15.105:0/737450013 shutdown_connections 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 --2- 172.21.15.105:0/737450013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c102370 0x7f351c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 --2- 172.21.15.105:0/737450013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f351c1019b0 0x7f351c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 --2- 172.21.15.105:0/737450013 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1007c0 0x7f351c100be0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 -- 172.21.15.105:0/737450013 >> 172.21.15.105:0/737450013 conn(0x7f351c0fbd50 msgr2=0x7f351c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 -- 172.21.15.105:0/737450013 shutdown_connections 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.596+0000 7f35231b3640 1 -- 172.21.15.105:0/737450013 wait complete. 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f35231b3640 1 Processor -- start 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f35231b3640 1 -- start start 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f35231b3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c1007c0 0x7f351c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f35231b3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1019b0 0x7f351c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f35231b3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f351c102370 0x7f351c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f35231b3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10d410 con 0x7f351c102370 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f35231b3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10d290 con 0x7f351c1019b0 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f35231b3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f351c10d590 con 0x7f351c1007c0 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f351bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1019b0 0x7f351c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f3521750640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f351c102370 0x7f351c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f351bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1019b0 0x7f351c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55822/0 (socket says 172.21.15.105:55822) 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f351bfff640 1 -- 172.21.15.105:0/256717124 learned_addr learned my addr 172.21.15.105:0/256717124 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.597+0000 7f3520f4f640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c1007c0 0x7f351c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3521750640 1 -- 172.21.15.105:0/256717124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c1007c0 msgr2=0x7f351c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3521750640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c1007c0 0x7f351c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3521750640 1 -- 172.21.15.105:0/256717124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1019b0 msgr2=0x7f351c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3521750640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1019b0 0x7f351c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3521750640 1 -- 172.21.15.105:0/256717124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351c1095c0 con 0x7f351c102370 2024-03-26T18:01:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f351bfff640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1019b0 0x7f351c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3520f4f640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c1007c0 0x7f351c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3521750640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f351c102370 0x7f351c19f100 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f3514009fd0 tx=0x7f351400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3519ffb640 1 -- 172.21.15.105:0/256717124 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351400ce20 con 0x7f351c102370 2024-03-26T18:01:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3519ffb640 1 -- 172.21.15.105:0/256717124 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3514005020 con 0x7f351c102370 2024-03-26T18:01:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f3519ffb640 1 -- 172.21.15.105:0/256717124 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514010690 con 0x7f351c102370 2024-03-26T18:01:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.598+0000 7f35231b3640 1 -- 172.21.15.105:0/256717124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351c1098b0 con 0x7f351c102370 2024-03-26T18:01:37.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.599+0000 7f35231b3640 1 -- 172.21.15.105:0/256717124 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351c109e50 con 0x7f351c102370 2024-03-26T18:01:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.601+0000 7f3519ffb640 1 -- 172.21.15.105:0/256717124 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f35140051d0 con 0x7f351c102370 2024-03-26T18:01:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3525d99740 1 Processor -- start 2024-03-26T18:01:37.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3525d99740 1 -- start start 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3525d99740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f84e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3525d99740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f89f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3525d99740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f8ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3525d99740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d849ee9970 con 0x55d849f8a590 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3525d99740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d849e91d70 con 0x55d849f84a30 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3525d99740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d849ee5920 con 0x55d849f85420 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.602+0000 7f3519ffb640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f350c0a2fa0 0x7f350c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f351bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f8ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f351bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f8ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47062/0 (socket says 172.21.15.105:47062) 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f351bfff640 1 -- 172.21.15.105:0/2791506415 learned_addr learned my addr 172.21.15.105:0/2791506415 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f351bfff640 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f84e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f3520f4f640 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f89f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f3521750640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f350c0a2fa0 0x7f350c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f3520f4f640 1 -- 172.21.15.105:0/2791506415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 msgr2=0x55d849f84e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f3520f4f640 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f84e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f3520f4f640 1 -- 172.21.15.105:0/2791506415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 msgr2=0x55d849f8ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f3520f4f640 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f8ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.603+0000 7f3520f4f640 1 -- 172.21.15.105:0/2791506415 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d849bfd3f0 con 0x55d849f85420 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.604+0000 7f3519ffb640 1 -- 172.21.15.105:0/256717124 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f35140cfaa0 con 0x7f351c102370 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.604+0000 7f3520f4f640 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f89f20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3510036040 tx=0x7f3510009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d2ffe3bbef9fa00 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.604+0000 7f351bfff640 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f8ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.604+0000 7f3506ffd640 1 -- 172.21.15.105:0/2791506415 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510045020 con 0x55d849f85420 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.604+0000 7f3521750640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f350c0a2fa0 0x7f350c0a5460 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3514098b00 tx=0x7f3514005810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.604+0000 7f3506ffd640 1 -- 172.21.15.105:0/2791506415 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351002cc90 con 0x55d849f85420 2024-03-26T18:01:37.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.604+0000 7f3506ffd640 1 -- 172.21.15.105:0/2791506415 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510004900 con 0x55d849f85420 2024-03-26T18:01:37.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 -- 172.21.15.105:0/2791506415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 msgr2=0x55d849f89f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f89f20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3510036040 tx=0x7f3510009580 comp rx=0 tx=0).stop 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 -- 172.21.15.105:0/2791506415 shutdown_connections 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f8ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f89f20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 --2- 172.21.15.105:0/2791506415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f84e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 -- 172.21.15.105:0/2791506415 >> 172.21.15.105:0/2791506415 conn(0x55d849f7ffa0 msgr2=0x55d849f803a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 -- 172.21.15.105:0/2791506415 shutdown_connections 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 -- 172.21.15.105:0/2791506415 wait complete. 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.605+0000 7f3525d99740 1 -- start start 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.606+0000 7f3525d99740 1 Processor -- start 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.606+0000 7f3525d99740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f8eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.606+0000 7f3525d99740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f8f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f351bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f8eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f351bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f8eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55834/0 (socket says 172.21.15.105:55834) 2024-03-26T18:01:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f3520f4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f8f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f3520f4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f8f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47074/0 (socket says 172.21.15.105:47074) 2024-03-26T18:01:37.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f351bfff640 1 -- 172.21.15.105:0/1297867164 learned_addr learned my addr 172.21.15.105:0/1297867164 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f3525d99740 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f8a590 0x55d849f96440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d849f8d4e0 con 0x55d849f85420 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d849ea5310 con 0x55d849f84a30 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d849f8d660 con 0x55d849f8a590 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f3525d99740 10 client.0 _ll_register_callbacks cb 0x55d849ee0c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.607+0000 7f351bfff640 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f8a590 0x55d849f96440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f351bfff640 1 -- 172.21.15.105:0/1297867164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f8a590 msgr2=0x55d849f96440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f351bfff640 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f8a590 0x55d849f96440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f351bfff640 1 -- 172.21.15.105:0/1297867164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 msgr2=0x55d849f8f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f351bfff640 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f8f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f351bfff640 1 -- 172.21.15.105:0/1297867164 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d849cc77d0 con 0x55d849f84a30 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f3520f4f640 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f8f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f351bfff640 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f8eec0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3508012f80 tx=0x7f3508004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9a3edb12c1917a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f34fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f34fbfff640 1 -- 172.21.15.105:0/1297867164 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35080044d0 con 0x55d849f84a30 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f351002cc90 con 0x55d849f84a30 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f34fbfff640 1 -- 172.21.15.105:0/1297867164 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f3508009040 con 0x55d849f84a30 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.608+0000 7f34fbfff640 1 -- 172.21.15.105:0/1297867164 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3508005020 con 0x55d849f84a30 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f3525d99740 10 client.33503 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d849f99850 con 0x55d849f84a30 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f34fbfff640 1 -- 172.21.15.105:0/1297867164 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351002cc90 con 0x55d849f84a30 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f34fbfff640 1 -- 172.21.15.105:0/1297867164 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f350800e5d0 con 0x55d849f84a30 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f3525d99740 10 client.33503 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f3525d99740 10 client.33503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f3525d99740 20 client.33503 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f3525d99740 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d849f9adf0 0x55d849f9d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f3525d99740 4 client.33503 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.609+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d849ef6eb0 con 0x55d849f9adf0 2024-03-26T18:01:37.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.610+0000 7f3520f4f640 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d849f9adf0 0x55d849f9d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.610+0000 7f34fbfff640 10 client.33503 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:37.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.610+0000 7f3520f4f640 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d849f9adf0 0x55d849f9d2b0 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d25156a46c241b server_cookie=137a8d3257adfdbb in_seq=0 out_seq=0 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f34fbfff640 1 -- 172.21.15.105:0/1297867164 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d849ef6eb0 con 0x55d849f9adf0 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f34fbfff640 10 client.33503 handle_command_reply: tid=0 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 client.33503 shutdown 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 msgr2=0x55d849f8eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f8eec0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3508012f80 tx=0x7f3508004730 comp rx=0 tx=0).stop 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 shutdown_connections 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d849f9adf0 0x55d849f9d2b0 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f34fbfff640 0 client.33503 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f8a590 0x55d849f96440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f8f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 --2- 172.21.15.105:0/1297867164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f8eec0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.611+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 >> 172.21.15.105:0/1297867164 conn(0x55d849f7ffa0 msgr2=0x55d849f7f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.612+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 shutdown_connections 2024-03-26T18:01:37.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.612+0000 7f3525d99740 1 -- 172.21.15.105:0/1297867164 wait complete. 2024-03-26T18:01:37.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.612+0000 7f3525d99740 20 client.33503 trim_cache size 0 max 16384 2024-03-26T18:01:37.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3525d99740 1 Processor -- start 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3525d99740 1 -- start start 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3525d99740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f84a30 0x55d849f8ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3525d99740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f96400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3525d99740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f8a590 0x55d849f8f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3525d99740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1aaf50 con 0x55d849f85420 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3525d99740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1aca90 con 0x55d849f8a590 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3525d99740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1ab620 con 0x55d849f84a30 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f96400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f96400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47086/0 (socket says 172.21.15.105:47086) 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 -- 172.21.15.105:0/1761207461 learned_addr learned my addr 172.21.15.105:0/1761207461 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.617 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f351bfff640 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f84a30 0x55d849f8ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f351bfff640 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f8a590 0x55d849f8f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 -- 172.21.15.105:0/1761207461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f84a30 msgr2=0x55d849f8ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f84a30 0x55d849f8ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 -- 172.21.15.105:0/1761207461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f8a590 msgr2=0x55d849f8f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f8a590 0x55d849f8f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 -- 172.21.15.105:0/1761207461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d849f89bd0 con 0x55d849f85420 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f351bfff640 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f8a590 0x55d849f8f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.632+0000 7f3520f4f640 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f96400 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3510009900 tx=0x7f351002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63a76bfb7adadad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.633+0000 7f35067fc640 1 -- 172.21.15.105:0/1761207461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510045040 con 0x55d849f85420 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.633+0000 7f35067fc640 1 -- 172.21.15.105:0/1761207461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351002cc90 con 0x55d849f85420 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.633+0000 7f35067fc640 1 -- 172.21.15.105:0/1761207461 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510041870 con 0x55d849f85420 2024-03-26T18:01:37.618 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 -- 172.21.15.105:0/1761207461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 msgr2=0x55d849f96400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f96400 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f3510009900 tx=0x7f351002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 -- 172.21.15.105:0/1761207461 shutdown_connections 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f8a590 0x55d849f8f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f85420 0x55d849f96400 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 --2- 172.21.15.105:0/1761207461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f84a30 0x55d849f8ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 -- 172.21.15.105:0/1761207461 >> 172.21.15.105:0/1761207461 conn(0x55d849f7ffa0 msgr2=0x55d849f86920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 -- 172.21.15.105:0/1761207461 shutdown_connections 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 -- 172.21.15.105:0/1761207461 wait complete. 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 -- start start 2024-03-26T18:01:37.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.634+0000 7f3525d99740 1 Processor -- start 2024-03-26T18:01:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3525d99740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f93580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3525d99740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f93ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3525d99740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f94ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3525d99740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1ac3c0 con 0x55d849f8a590 2024-03-26T18:01:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3525d99740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1abcf0 con 0x55d849f84a30 2024-03-26T18:01:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3525d99740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f351c1a8f30 con 0x55d849f85420 2024-03-26T18:01:37.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3525d99740 10 client.0 _ll_register_callbacks cb 0x7f3508011640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f94ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f94ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47100/0 (socket says 172.21.15.105:47100) 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 -- 172.21.15.105:0/2621548226 learned_addr learned my addr 172.21.15.105:0/2621548226 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3520f4f640 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f93ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f93580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 -- 172.21.15.105:0/2621548226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 msgr2=0x55d849f93ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f93ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 -- 172.21.15.105:0/2621548226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 msgr2=0x55d849f93580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f93580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f351bfff640 1 -- 172.21.15.105:0/2621548226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d849f950c0 con 0x55d849f8a590 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.635+0000 7f3520f4f640 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f93ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f351bfff640 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f94ad0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f350800ebe0 tx=0x7f3508004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499c5dd74b397740 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f34fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f34fbfff640 1 -- 172.21.15.105:0/2621548226 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35080044d0 con 0x55d849f8a590 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f34fbfff640 1 -- 172.21.15.105:0/2621548226 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f35080053d0 con 0x55d849f8a590 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f3525d99740 1 -- 172.21.15.105:0/2621548226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f351002cc90 con 0x55d849f8a590 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f34fbfff640 1 -- 172.21.15.105:0/2621548226 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3508003e50 con 0x55d849f8a590 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f3525d99740 10 client.23838 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f3525d99740 1 -- 172.21.15.105:0/2621548226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d849f96310 con 0x55d849f8a590 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.636+0000 7f34fbfff640 1 -- 172.21.15.105:0/2621548226 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351002cc90 con 0x55d849f8a590 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f34fbfff640 1 -- 172.21.15.105:0/2621548226 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3508004980 con 0x55d849f8a590 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f3525d99740 10 client.23838 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f3525d99740 10 client.23838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f3525d99740 20 client.23838 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f3525d99740 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d849f9adf0 0x55d84a003440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f3525d99740 4 client.23838 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f3525d99740 1 -- 172.21.15.105:0/2621548226 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d849ef1200 con 0x55d849f9adf0 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f3520f4f640 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d849f9adf0 0x55d84a003440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f34fbfff640 10 client.23838 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.637+0000 7f3520f4f640 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d849f9adf0 0x55d84a003440 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9a7ee7c99a2780 server_cookie=77ecd1a159bc0071 in_seq=0 out_seq=0 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f34fbfff640 1 -- 172.21.15.105:0/2621548226 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d849ef1200 con 0x55d849f9adf0 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f34fbfff640 10 client.23838 handle_command_reply: tid=0 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 client.23838 shutdown 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 -- 172.21.15.105:0/2621548226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 msgr2=0x55d849f94ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f94ad0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f350800ebe0 tx=0x7f3508004730 comp rx=0 tx=0).stop 2024-03-26T18:01:37.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 -- 172.21.15.105:0/2621548226 shutdown_connections 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d849f9adf0 0x55d84a003440 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d849f8a590 0x55d849f94ad0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f34fbfff640 0 client.23838 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d849f85420 0x55d849f93ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 --2- 172.21.15.105:0/2621548226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d849f84a30 0x55d849f93580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.639+0000 7f3525d99740 1 -- 172.21.15.105:0/2621548226 >> 172.21.15.105:0/2621548226 conn(0x55d849f7ffa0 msgr2=0x55d849f8ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f3525d99740 1 -- 172.21.15.105:0/2621548226 shutdown_connections 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f3525d99740 1 -- 172.21.15.105:0/2621548226 wait complete. 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f3525d99740 20 client.23838 trim_cache size 0 max 16384 2024-03-26T18:01:37.628 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:01:37.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 -- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f350c0a2fa0 msgr2=0x7f350c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f350c0a2fa0 0x7f350c0a5460 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f3514098b00 tx=0x7f3514005810 comp rx=0 tx=0).stop 2024-03-26T18:01:37.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 -- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f351c102370 msgr2=0x7f351c19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f351c102370 0x7f351c19f100 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f3514009fd0 tx=0x7f351400ef10 comp rx=0 tx=0).stop 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 -- 172.21.15.105:0/256717124 shutdown_connections 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f350c0a2fa0 0x7f350c0a5460 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f351c102370 0x7f351c19f100 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f351c1019b0 0x7f351c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 --2- 172.21.15.105:0/256717124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c1007c0 0x7f351c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.640+0000 7f35231b3640 1 -- 172.21.15.105:0/256717124 >> 172.21.15.105:0/256717124 conn(0x7f351c0fbd50 msgr2=0x7f351c104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.641+0000 7f35231b3640 1 -- 172.21.15.105:0/256717124 shutdown_connections 2024-03-26T18:01:37.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:37.641+0000 7f35231b3640 1 -- 172.21.15.105:0/256717124 wait complete. 2024-03-26T18:01:38.087 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:37 smithi144 ceph-mon[36095]: pgmap v1050: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-03-26T18:01:38.131 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:37 smithi105 ceph-mon[32280]: pgmap v1050: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-03-26T18:01:38.154 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:37 smithi135 ceph-mon[35636]: pgmap v1050: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-03-26T18:01:38.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81310 7.05 MB/sec execute 501 sec latency 148.615 ms 2024-03-26T18:01:38.637 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T18:01:38.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:01:38.637 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T18:01:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 -- 172.21.15.105:0/1920878391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84781007c0 msgr2=0x7f8478100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 --2- 172.21.15.105:0/1920878391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84781007c0 0x7f8478100be0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f8468009900 tx=0x7f846802bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 -- 172.21.15.105:0/1920878391 shutdown_connections 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 --2- 172.21.15.105:0/1920878391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8478102370 0x7f8478108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 --2- 172.21.15.105:0/1920878391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84781019b0 0x7f8478101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 --2- 172.21.15.105:0/1920878391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84781007c0 0x7f8478100be0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 -- 172.21.15.105:0/1920878391 >> 172.21.15.105:0/1920878391 conn(0x7f84780fbd50 msgr2=0x7f84780fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 -- 172.21.15.105:0/1920878391 shutdown_connections 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.800+0000 7f847d79e640 1 -- 172.21.15.105:0/1920878391 wait complete. 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f847d79e640 1 Processor -- start 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f847d79e640 1 -- start start 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f847d79e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84781007c0 0x7f8478199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f847d79e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84781019b0 0x7f847819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f847d79e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8478102370 0x7f84781a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f847d79e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f847806a5b0 con 0x7f8478102370 2024-03-26T18:01:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f847d79e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f847806a430 con 0x7f84781019b0 2024-03-26T18:01:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f847d79e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f847806a730 con 0x7f84781007c0 2024-03-26T18:01:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f84777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8478102370 0x7f84781a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84781007c0 0x7f8478199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f84777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8478102370 0x7f84781a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47120/0 (socket says 172.21.15.105:47120) 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f84777fe640 1 -- 172.21.15.105:0/563104678 learned_addr learned my addr 172.21.15.105:0/563104678 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.801+0000 7f84767fc640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84781019b0 0x7f847819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f8476ffd640 1 -- 172.21.15.105:0/563104678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84781019b0 msgr2=0x7f847819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f8476ffd640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84781019b0 0x7f847819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f8476ffd640 1 -- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8478102370 msgr2=0x7f84781a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f8476ffd640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8478102370 0x7f84781a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f8476ffd640 1 -- 172.21.15.105:0/563104678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84781a1a10 con 0x7f84781007c0 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f84777fe640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8478102370 0x7f84781a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f84767fc640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84781019b0 0x7f847819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f8476ffd640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84781007c0 0x7f8478199df0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8468036040 tx=0x7f84680419e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.802+0000 7f845ffff640 1 -- 172.21.15.105:0/563104678 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468004040 con 0x7f84781007c0 2024-03-26T18:01:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.803+0000 7f847d79e640 1 -- 172.21.15.105:0/563104678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8478102bd0 con 0x7f84781007c0 2024-03-26T18:01:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.803+0000 7f845ffff640 1 -- 172.21.15.105:0/563104678 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84680041e0 con 0x7f84781007c0 2024-03-26T18:01:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.803+0000 7f845ffff640 1 -- 172.21.15.105:0/563104678 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468003970 con 0x7f84781007c0 2024-03-26T18:01:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.803+0000 7f847d79e640 1 -- 172.21.15.105:0/563104678 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8478103160 con 0x7f84781007c0 2024-03-26T18:01:38.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.805+0000 7f845ffff640 1 -- 172.21.15.105:0/563104678 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8468052060 con 0x7f84781007c0 2024-03-26T18:01:38.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8480389740 1 Processor -- start 2024-03-26T18:01:38.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8480389740 1 -- start start 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8480389740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b81355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8480389740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b813a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8480389740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b813d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8480389740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d8b809a630 con 0x55d8b8135190 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8480389740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8b8042e10 con 0x55d8b8135b80 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8480389740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d8b80965e0 con 0x55d8b813acf0 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f84777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b81355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f84767fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b813d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f84777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b81355b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47130/0 (socket says 172.21.15.105:47130) 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f84767fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b813d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38194/0 (socket says 172.21.15.105:38194) 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f84777fe640 1 -- 172.21.15.105:0/489516548 learned_addr learned my addr 172.21.15.105:0/489516548 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f8476ffd640 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b813a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.806+0000 7f845ffff640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84600a2fa0 0x7f84600a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f84767fc640 1 -- 172.21.15.105:0/489516548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 msgr2=0x55d8b813a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f84767fc640 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b813a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f84767fc640 1 -- 172.21.15.105:0/489516548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 msgr2=0x55d8b81355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f84767fc640 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b81355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f84767fc640 1 -- 172.21.15.105:0/489516548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b8059740 con 0x55d8b813acf0 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f84777fe640 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b81355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f845ffff640 1 -- 172.21.15.105:0/563104678 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f84680f8d90 con 0x7f84781007c0 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f84777fe640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84600a2fa0 0x7f84600a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f84767fc640 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b813d180 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f846c009fa0 tx=0x7f846c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0bf3c9cd34f5368 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f845d7fa640 1 -- 172.21.15.105:0/489516548 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c004310 con 0x55d8b813acf0 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f845d7fa640 1 -- 172.21.15.105:0/489516548 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f846c0044b0 con 0x55d8b813acf0 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.807+0000 7f845d7fa640 1 -- 172.21.15.105:0/489516548 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c00f970 con 0x55d8b813acf0 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 -- 172.21.15.105:0/489516548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 msgr2=0x55d8b813d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b813d180 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f846c009fa0 tx=0x7f846c011810 comp rx=0 tx=0).stop 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f84777fe640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84600a2fa0 0x7f84600a5460 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8470009500 tx=0x7f8470004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 -- 172.21.15.105:0/489516548 shutdown_connections 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b813d180 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b813a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 --2- 172.21.15.105:0/489516548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b81355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 -- 172.21.15.105:0/489516548 >> 172.21.15.105:0/489516548 conn(0x55d8b8130700 msgr2=0x55d8b8130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 -- 172.21.15.105:0/489516548 shutdown_connections 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 -- 172.21.15.105:0/489516548 wait complete. 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.808+0000 7f8480389740 1 -- start start 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8480389740 1 Processor -- start 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8480389740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135190 0x55d8b813f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8480389740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135b80 0x55d8b813fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8480389740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 0x55d8b8146c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8480389740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d8b813dca0 con 0x55d8b8135b80 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8480389740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8b80561a0 con 0x55d8b813acf0 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8480389740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d8b813de20 con 0x55d8b8135190 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8480389740 10 client.0 _ll_register_callbacks cb 0x55d8b7dadc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135190 0x55d8b813f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f84767fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135b80 0x55d8b813fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135190 0x55d8b813f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38202/0 (socket says 172.21.15.105:38202) 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8476ffd640 1 -- 172.21.15.105:0/3087455528 learned_addr learned my addr 172.21.15.105:0/3087455528 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f84777fe640 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 0x55d8b8146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f84767fc640 1 -- 172.21.15.105:0/3087455528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135190 msgr2=0x55d8b813f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f84767fc640 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135190 0x55d8b813f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f84767fc640 1 -- 172.21.15.105:0/3087455528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 msgr2=0x55d8b8146c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f84767fc640 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 0x55d8b8146c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f84767fc640 1 -- 172.21.15.105:0/3087455528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b7e767e0 con 0x55d8b8135b80 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.809+0000 7f8476ffd640 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135190 0x55d8b813f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f84767fc640 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135b80 0x55d8b813fc10 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f846c005890 tx=0x7f846c0052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40b30e96546b7ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f845aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f845aff5640 1 -- 172.21.15.105:0/3087455528 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c028020 con 0x55d8b8135b80 2024-03-26T18:01:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f845aff5640 1 -- 172.21.15.105:0/3087455528 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f846c00f710 con 0x55d8b8135b80 2024-03-26T18:01:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f8480389740 1 -- 172.21.15.105:0/3087455528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f846c0044b0 con 0x55d8b8135b80 2024-03-26T18:01:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f845aff5640 1 -- 172.21.15.105:0/3087455528 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f846c033050 con 0x55d8b8135b80 2024-03-26T18:01:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f8480389740 10 client.23850 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f8480389740 1 -- 172.21.15.105:0/3087455528 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d8b7ed6f70 con 0x55d8b8135b80 2024-03-26T18:01:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.810+0000 7f845aff5640 1 -- 172.21.15.105:0/3087455528 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f846c0044b0 con 0x55d8b8135b80 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.811+0000 7f845aff5640 1 -- 172.21.15.105:0/3087455528 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f846c018070 con 0x55d8b8135b80 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.811+0000 7f8480389740 10 client.23850 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.811+0000 7f8480389740 10 client.23850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.811+0000 7f8480389740 20 client.23850 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.811+0000 7f8480389740 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8b814b1b0 0x55d8b814d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.811+0000 7f8480389740 4 client.23850 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.811+0000 7f8480389740 1 -- 172.21.15.105:0/3087455528 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8b80a7b70 con 0x55d8b814b1b0 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.811+0000 7f8476ffd640 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8b814b1b0 0x55d8b814d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.812+0000 7f8476ffd640 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8b814b1b0 0x55d8b814d620 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ddadaf89f7bea9 server_cookie=bb7197bf4f7fa951 in_seq=0 out_seq=0 2024-03-26T18:01:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.812+0000 7f845aff5640 10 client.23850 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.813+0000 7f845aff5640 1 -- 172.21.15.105:0/3087455528 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d8b80a7b70 con 0x55d8b814b1b0 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.813+0000 7f845aff5640 10 client.23850 handle_command_reply: tid=0 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.813+0000 7f8480389740 1 client.23850 shutdown 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.813+0000 7f8480389740 1 -- 172.21.15.105:0/3087455528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135b80 msgr2=0x55d8b813fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135b80 0x55d8b813fc10 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f846c005890 tx=0x7f846c0052e0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 -- 172.21.15.105:0/3087455528 shutdown_connections 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8b814b1b0 0x55d8b814d620 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 0x55d8b8146c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135b80 0x55d8b813fc10 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 --2- 172.21.15.105:0/3087455528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135190 0x55d8b813f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 -- 172.21.15.105:0/3087455528 >> 172.21.15.105:0/3087455528 conn(0x55d8b8130700 msgr2=0x55d8b812f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f845aff5640 0 client.23850 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 -- 172.21.15.105:0/3087455528 shutdown_connections 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 1 -- 172.21.15.105:0/3087455528 wait complete. 2024-03-26T18:01:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.814+0000 7f8480389740 20 client.23850 trim_cache size 0 max 16384 2024-03-26T18:01:38.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8480389740 1 Processor -- start 2024-03-26T18:01:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8480389740 1 -- start start 2024-03-26T18:01:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8480389740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b8137080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8480389740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135b80 0x55d8b81375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8480389740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 0x55d8b8146c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8480389740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84781a6c50 con 0x55d8b8135190 2024-03-26T18:01:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8480389740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84781a8790 con 0x55d8b813acf0 2024-03-26T18:01:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8480389740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84781a7320 con 0x55d8b8135b80 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b8137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b8137080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47148/0 (socket says 172.21.15.105:47148) 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f8476ffd640 1 -- 172.21.15.105:0/1568651909 learned_addr learned my addr 172.21.15.105:0/1568651909 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f84767fc640 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135b80 0x55d8b81375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.836+0000 7f84777fe640 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 0x55d8b8146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f8476ffd640 1 -- 172.21.15.105:0/1568651909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135b80 msgr2=0x55d8b81375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f8476ffd640 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135b80 0x55d8b81375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f8476ffd640 1 -- 172.21.15.105:0/1568651909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 msgr2=0x55d8b8146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f8476ffd640 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 0x55d8b8146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f8476ffd640 1 -- 172.21.15.105:0/1568651909 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b7d6e170 con 0x55d8b8135190 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f84767fc640 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135b80 0x55d8b81375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f8476ffd640 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b8137080 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f84680c1160 tx=0x7f84680f6720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148125f603c231cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f845cff9640 1 -- 172.21.15.105:0/1568651909 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680c1720 con 0x55d8b8135190 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f845cff9640 1 -- 172.21.15.105:0/1568651909 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84680c5b80 con 0x55d8b8135190 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.837+0000 7f845cff9640 1 -- 172.21.15.105:0/1568651909 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680c4de0 con 0x55d8b8135190 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 -- 172.21.15.105:0/1568651909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 msgr2=0x55d8b8137080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b8137080 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f84680c1160 tx=0x7f84680f6720 comp rx=0 tx=0).stop 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 -- 172.21.15.105:0/1568651909 shutdown_connections 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b813acf0 0x55d8b8146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b8135b80 0x55d8b81375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 --2- 172.21.15.105:0/1568651909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b8135190 0x55d8b8137080 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 -- 172.21.15.105:0/1568651909 >> 172.21.15.105:0/1568651909 conn(0x55d8b8130700 msgr2=0x55d8b813e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 -- 172.21.15.105:0/1568651909 shutdown_connections 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.838+0000 7f8480389740 1 -- 172.21.15.105:0/1568651909 wait complete. 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 1 -- start start 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 1 Processor -- start 2024-03-26T18:01:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b8143db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b81442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b81452b0 0x55d8b8145730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84781a80c0 con 0x55d8b81452b0 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84781a79f0 con 0x55d8b8135b80 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84781a4c50 con 0x55d8b813acf0 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8480389740 10 client.0 _ll_register_callbacks cb 0x7f846c011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b8143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f84777fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b81452b0 0x55d8b8145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b8143db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55902/0 (socket says 172.21.15.105:55902) 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8476ffd640 1 -- 172.21.15.105:0/3389087851 learned_addr learned my addr 172.21.15.105:0/3389087851 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f84767fc640 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b81442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8476ffd640 1 -- 172.21.15.105:0/3389087851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 msgr2=0x55d8b81442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8476ffd640 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b81442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8476ffd640 1 -- 172.21.15.105:0/3389087851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b81452b0 msgr2=0x55d8b8145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8476ffd640 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b81452b0 0x55d8b8145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f8476ffd640 1 -- 172.21.15.105:0/3389087851 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b8145c70 con 0x55d8b8135b80 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.839+0000 7f84777fe640 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b81452b0 0x55d8b8145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f8476ffd640 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b8143db0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8468006010 tx=0x7f84680c4f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed3df3034756b54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f845aff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f845aff5640 1 -- 172.21.15.105:0/3389087851 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680de060 con 0x55d8b8135b80 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f845aff5640 1 -- 172.21.15.105:0/3389087851 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f84680c9170 con 0x55d8b8135b80 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f8480389740 1 -- 172.21.15.105:0/3389087851 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84680c5b80 con 0x55d8b8135b80 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f845aff5640 1 -- 172.21.15.105:0/3389087851 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f84680c4fc0 con 0x55d8b8135b80 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f8480389740 10 client.33524 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f8480389740 1 -- 172.21.15.105:0/3389087851 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d8b8147020 con 0x55d8b8135b80 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.840+0000 7f845aff5640 1 -- 172.21.15.105:0/3389087851 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84680c5b80 con 0x55d8b8135b80 2024-03-26T18:01:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f845aff5640 1 -- 172.21.15.105:0/3389087851 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f84680a5050 con 0x55d8b8135b80 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f8480389740 10 client.33524 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f8480389740 10 client.33524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f8480389740 20 client.33524 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f8480389740 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8b81b1c50 0x55d8b81b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f8480389740 4 client.33524 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f8480389740 1 -- 172.21.15.105:0/3389087851 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d8b80a1ec0 con 0x55d8b81b1c50 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f84767fc640 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8b81b1c50 0x55d8b81b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f845aff5640 10 client.33524 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.841+0000 7f84767fc640 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8b81b1c50 0x55d8b81b4070 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=595541ab7341204e server_cookie=8fa3aa0bc555aa5a in_seq=0 out_seq=0 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f845aff5640 1 -- 172.21.15.105:0/3389087851 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d8b80a1ec0 con 0x55d8b81b1c50 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f845aff5640 10 client.33524 handle_command_reply: tid=0 2024-03-26T18:01:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 client.33524 shutdown 2024-03-26T18:01:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 -- 172.21.15.105:0/3389087851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 msgr2=0x55d8b8143db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b8143db0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8468006010 tx=0x7f84680c4f10 comp rx=0 tx=0).stop 2024-03-26T18:01:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 -- 172.21.15.105:0/3389087851 shutdown_connections 2024-03-26T18:01:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55d8b81b1c50 0x55d8b81b4070 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d8b81452b0 0x55d8b8145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8b813acf0 0x55d8b81442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 --2- 172.21.15.105:0/3389087851 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8b8135b80 0x55d8b8143db0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f8480389740 1 -- 172.21.15.105:0/3389087851 >> 172.21.15.105:0/3389087851 conn(0x55d8b8130700 msgr2=0x55d8b813f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.843+0000 7f845aff5640 0 client.33524 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f8480389740 1 -- 172.21.15.105:0/3389087851 shutdown_connections 2024-03-26T18:01:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f8480389740 1 -- 172.21.15.105:0/3389087851 wait complete. 2024-03-26T18:01:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f8480389740 20 client.33524 trim_cache size 0 max 16384 2024-03-26T18:01:38.835 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:01:38.835 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T18:01:38.836 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "e399dbf4-b8f6-4e36-a7a0-39f573fedd98", 2024-03-26T18:01:38.836 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T18:01:38.836 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 -- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84600a2fa0 msgr2=0x7f84600a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84600a2fa0 0x7f84600a5460 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8470009500 tx=0x7f8470004b50 comp rx=0 tx=0).stop 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 -- 172.21.15.105:0/563104678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84781007c0 msgr2=0x7f8478199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84781007c0 0x7f8478199df0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8468036040 tx=0x7f84680419e0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 -- 172.21.15.105:0/563104678 shutdown_connections 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84600a2fa0 0x7f84600a5460 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8478102370 0x7f84781a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84781019b0 0x7f847819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 --2- 172.21.15.105:0/563104678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84781007c0 0x7f8478199df0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.844+0000 7f847d79e640 1 -- 172.21.15.105:0/563104678 >> 172.21.15.105:0/563104678 conn(0x7f84780fbd50 msgr2=0x7f847810bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.845+0000 7f847d79e640 1 -- 172.21.15.105:0/563104678 shutdown_connections 2024-03-26T18:01:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.845+0000 7f847d79e640 1 -- 172.21.15.105:0/563104678 wait complete. 2024-03-26T18:01:38.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:01:38.841 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:01:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 -- 172.21.15.105:0/273446863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281019b0 msgr2=0x7f7328101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 --2- 172.21.15.105:0/273446863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281019b0 0x7f7328101e30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f731c009860 tx=0x7f731c02bb70 comp rx=0 tx=0).stop 2024-03-26T18:01:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 -- 172.21.15.105:0/273446863 shutdown_connections 2024-03-26T18:01:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 --2- 172.21.15.105:0/273446863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7328102370 0x7f7328108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 --2- 172.21.15.105:0/273446863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281019b0 0x7f7328101e30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 --2- 172.21.15.105:0/273446863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73281007c0 0x7f7328100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 -- 172.21.15.105:0/273446863 >> 172.21.15.105:0/273446863 conn(0x7f73280fbd50 msgr2=0x7f73280fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 -- 172.21.15.105:0/273446863 shutdown_connections 2024-03-26T18:01:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.989+0000 7f732d49a640 1 -- 172.21.15.105:0/273446863 wait complete. 2024-03-26T18:01:38.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.990+0000 7f732d49a640 1 Processor -- start 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.990+0000 7f732d49a640 1 -- start start 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.990+0000 7f732d49a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281007c0 0x7f7328069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.990+0000 7f732d49a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73281019b0 0x7f732819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.990+0000 7f732d49a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7328102370 0x7f732819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.990+0000 7f732d49a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f732810c340 con 0x7f73281019b0 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.990+0000 7f732d49a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f732810c1c0 con 0x7f7328102370 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.990+0000 7f732d49a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f732810c4c0 con 0x7f73281007c0 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281007c0 0x7f7328069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281007c0 0x7f7328069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38238/0 (socket says 172.21.15.105:38238) 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 -- 172.21.15.105:0/1093338627 learned_addr learned my addr 172.21.15.105:0/1093338627 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f73267fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73281019b0 0x7f732819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f73277fe640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7328102370 0x7f732819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 -- 172.21.15.105:0/1093338627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7328102370 msgr2=0x7f732819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7328102370 0x7f732819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 -- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73281019b0 msgr2=0x7f732819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73281019b0 0x7f732819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 -- 172.21.15.105:0/1093338627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73281966a0 con 0x7f73281007c0 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f73267fc640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73281019b0 0x7f732819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f7326ffd640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281007c0 0x7f7328069590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f731800e9c0 tx=0x7f731800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f730ffff640 1 -- 172.21.15.105:0/1093338627 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731800cd80 con 0x7f73281007c0 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f732d49a640 1 -- 172.21.15.105:0/1093338627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7328196930 con 0x7f73281007c0 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f730ffff640 1 -- 172.21.15.105:0/1093338627 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73180035f0 con 0x7f73281007c0 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f730ffff640 1 -- 172.21.15.105:0/1093338627 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318010690 con 0x7f73281007c0 2024-03-26T18:01:38.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.991+0000 7f732d49a640 1 -- 172.21.15.105:0/1093338627 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7328196e50 con 0x7f73281007c0 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.994+0000 7f730ffff640 1 -- 172.21.15.105:0/1093338627 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f73180037a0 con 0x7f73281007c0 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.994+0000 7f7330165740 1 Processor -- start 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.994+0000 7f7330165740 1 -- start start 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f730ffff640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73100a2f70 0x7f73100a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f7330165740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a0c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f7330165740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f7330165740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 0x559f14a140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f7330165740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f14968980 con 0x559f14a0c100 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f7330165740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559f14910be0 con 0x559f14a0caf0 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f7330165740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f14964930 con 0x559f14a11c60 2024-03-26T18:01:38.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73277fe640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73100a2f70 0x7f73100a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f730ffff640 1 -- 172.21.15.105:0/1093338627 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f73180cf990 con 0x7f73281007c0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73267fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73267fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55926/0 (socket says 172.21.15.105:55926) 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73267fc640 1 -- 172.21.15.105:0/1354127694 learned_addr learned my addr 172.21.15.105:0/1354127694 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f7326ffd640 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a0c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73277fe640 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 0x559f14a140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73267fc640 1 -- 172.21.15.105:0/1354127694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 msgr2=0x559f14a140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73267fc640 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 0x559f14a140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73267fc640 1 -- 172.21.15.105:0/1354127694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 msgr2=0x559f14a0c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73267fc640 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a0c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.995+0000 7f73267fc640 1 -- 172.21.15.105:0/1354127694 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f147467d0 con 0x559f14a0caf0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.996+0000 7f73277fe640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73100a2f70 0x7f73100a5430 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7320005fd0 tx=0x7f7320009f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.996+0000 7f73267fc640 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a115f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f731c009860 tx=0x7f731c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0de6a837106be8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.996+0000 7f730d7fa640 1 -- 172.21.15.105:0/1354127694 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c043020 con 0x559f14a0caf0 2024-03-26T18:01:38.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.996+0000 7f730d7fa640 1 -- 172.21.15.105:0/1354127694 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f731c02c510 con 0x559f14a0caf0 2024-03-26T18:01:38.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.996+0000 7f730d7fa640 1 -- 172.21.15.105:0/1354127694 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c039040 con 0x559f14a0caf0 2024-03-26T18:01:38.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.997+0000 7f7330165740 1 -- 172.21.15.105:0/1354127694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 msgr2=0x559f14a115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.997+0000 7f7330165740 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a115f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f731c009860 tx=0x7f731c009560 comp rx=0 tx=0).stop 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.997+0000 7f7330165740 1 -- 172.21.15.105:0/1354127694 shutdown_connections 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.997+0000 7f7330165740 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 0x559f14a140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.997+0000 7f7330165740 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a115f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.997+0000 7f7330165740 1 --2- 172.21.15.105:0/1354127694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a0c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.997+0000 7f7330165740 1 -- 172.21.15.105:0/1354127694 >> 172.21.15.105:0/1354127694 conn(0x559f149fee60 msgr2=0x559f149ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.998+0000 7f7330165740 1 -- 172.21.15.105:0/1354127694 shutdown_connections 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.998+0000 7f7330165740 1 -- 172.21.15.105:0/1354127694 wait complete. 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.998+0000 7f7330165740 1 -- start start 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.998+0000 7f7330165740 1 Processor -- start 2024-03-26T18:01:38.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f7330165740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 0x559f14a162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f7330165740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f7330165740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a11c60 0x559f14a1d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f7330165740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559f14a14bb0 con 0x559f14a11c60 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f7330165740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559f14924160 con 0x559f14a0caf0 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f7330165740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x559f14a14d30 con 0x559f14a0c100 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a11c60 0x559f14a1d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f7326ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 0x559f14a162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73277fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a11c60 0x559f14a1d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47200/0 (socket says 172.21.15.105:47200) 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73277fe640 1 -- 172.21.15.105:0/1541119020 learned_addr learned my addr 172.21.15.105:0/1541119020 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f7330165740 10 client.0 _ll_register_callbacks cb 0x559f14960720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73267fc640 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73277fe640 1 -- 172.21.15.105:0/1541119020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 msgr2=0x559f14a162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73277fe640 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 0x559f14a162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73277fe640 1 -- 172.21.15.105:0/1541119020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 msgr2=0x559f14a167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:38.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73277fe640 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:38.999+0000 7f73277fe640 1 -- 172.21.15.105:0/1541119020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f1467c3f0 con 0x559f14a11c60 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f73277fe640 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a11c60 0x559f14a1d830 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f732001d6a0 tx=0x7f7320004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc9d873521f1dd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f730aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f730aff5640 1 -- 172.21.15.105:0/1541119020 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732002d060 con 0x559f14a11c60 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f730aff5640 1 -- 172.21.15.105:0/1541119020 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f7320028620 con 0x559f14a11c60 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f7330165740 1 -- 172.21.15.105:0/1541119020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c02c510 con 0x559f14a11c60 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f730aff5640 1 -- 172.21.15.105:0/1541119020 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7320017040 con 0x559f14a11c60 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f7330165740 10 client.23868 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f7330165740 1 -- 172.21.15.105:0/1541119020 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559f1463c0f0 con 0x559f14a11c60 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.000+0000 7f730aff5640 1 -- 172.21.15.105:0/1541119020 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f731c02c510 con 0x559f14a11c60 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f730aff5640 1 -- 172.21.15.105:0/1541119020 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f732000b040 con 0x559f14a11c60 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f7330165740 10 client.23868 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f7330165740 10 client.23868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:38.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f7330165740 20 client.23868 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:38.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f7330165740 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f14a22200 0x559f14a24690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:38.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f7330165740 4 client.23868 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:38.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f7330165740 1 -- 172.21.15.105:0/1541119020 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f14975ec0 con 0x559f14a22200 2024-03-26T18:01:38.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f7326ffd640 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f14a22200 0x559f14a24690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:38.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f7326ffd640 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f14a22200 0x559f14a24690 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce1ab55cbe8a2623 server_cookie=59aeea466e72ece in_seq=0 out_seq=0 2024-03-26T18:01:38.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.001+0000 7f730aff5640 10 client.23868 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:38.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.003+0000 7f730aff5640 1 -- 172.21.15.105:0/1541119020 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559f14975ec0 con 0x559f14a22200 2024-03-26T18:01:38.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.003+0000 7f730aff5640 10 client.23868 handle_command_reply: tid=0 2024-03-26T18:01:38.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.003+0000 7f7330165740 1 client.23868 shutdown 2024-03-26T18:01:38.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 -- 172.21.15.105:0/1541119020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a11c60 msgr2=0x559f14a1d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:38.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a11c60 0x559f14a1d830 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f732001d6a0 tx=0x7f7320004ab0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 -- 172.21.15.105:0/1541119020 shutdown_connections 2024-03-26T18:01:38.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f14a22200 0x559f14a24690 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a11c60 0x559f14a1d830 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 --2- 172.21.15.105:0/1541119020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 0x559f14a162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:38.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 -- 172.21.15.105:0/1541119020 >> 172.21.15.105:0/1541119020 conn(0x559f149fee60 msgr2=0x559f149fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:38.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f730aff5640 0 client.23868 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:38.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 -- 172.21.15.105:0/1541119020 shutdown_connections 2024-03-26T18:01:38.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 1 -- 172.21.15.105:0/1541119020 wait complete. 2024-03-26T18:01:38.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.004+0000 7f7330165740 20 client.23868 trim_cache size 0 max 16384 2024-03-26T18:01:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.026+0000 7f7330165740 1 Processor -- start 2024-03-26T18:01:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.026+0000 7f7330165740 1 -- start start 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.026+0000 7f7330165740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 0x559f149fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.026+0000 7f7330165740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0caf0 0x559f14a1d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.026+0000 7f7330165740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a11c60 0x559f149fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f7330165740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73281a6b40 con 0x559f14a0caf0 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f7330165740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73281a8680 con 0x559f14a11c60 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f7330165740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73281a7210 con 0x559f14a0c100 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0caf0 0x559f14a1d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0caf0 0x559f14a1d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47212/0 (socket says 172.21.15.105:47212) 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f7326ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 0x559f149fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 -- 172.21.15.105:0/197441021 learned_addr learned my addr 172.21.15.105:0/197441021 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73277fe640 1 --2- 172.21.15.105:0/197441021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a11c60 0x559f149fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 -- 172.21.15.105:0/197441021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 msgr2=0x559f149fd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 --2- 172.21.15.105:0/197441021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 0x559f149fd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 -- 172.21.15.105:0/197441021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a11c60 msgr2=0x559f149fdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 --2- 172.21.15.105:0/197441021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a11c60 0x559f149fdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 -- 172.21.15.105:0/197441021 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f14a20ae0 con 0x559f14a0caf0 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f73267fc640 1 --2- 172.21.15.105:0/197441021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0caf0 0x559f14a1d7f0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f731c009860 tx=0x7f731c02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607f2fa57a26dfeb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f730cff9640 1 -- 172.21.15.105:0/197441021 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c04c030 con 0x559f14a0caf0 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f730cff9640 1 -- 172.21.15.105:0/197441021 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f731c02c510 con 0x559f14a0caf0 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.027+0000 7f730cff9640 1 -- 172.21.15.105:0/197441021 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c039040 con 0x559f14a0caf0 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 -- 172.21.15.105:0/197441021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0caf0 msgr2=0x559f14a1d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 --2- 172.21.15.105:0/197441021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0caf0 0x559f14a1d7f0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f731c009860 tx=0x7f731c02bb70 comp rx=0 tx=0).stop 2024-03-26T18:01:39.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 -- 172.21.15.105:0/197441021 shutdown_connections 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 --2- 172.21.15.105:0/197441021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a11c60 0x559f149fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 --2- 172.21.15.105:0/197441021 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0caf0 0x559f14a1d7f0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 --2- 172.21.15.105:0/197441021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a0c100 0x559f149fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 -- 172.21.15.105:0/197441021 >> 172.21.15.105:0/197441021 conn(0x559f149fee60 msgr2=0x559f149ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 -- 172.21.15.105:0/197441021 shutdown_connections 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.028+0000 7f7330165740 1 -- 172.21.15.105:0/197441021 wait complete. 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 1 -- start start 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 1 Processor -- start 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a1b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a1b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 0x559f14a1c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73281a7fb0 con 0x559f14a0c100 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73281a78e0 con 0x559f14a0caf0 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f73281a4b20 con 0x559f14a11c60 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7330165740 10 client.0 _ll_register_callbacks cb 0x7f732000eea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7326ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a1b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7326ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a1b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47228/0 (socket says 172.21.15.105:47228) 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f73267fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a1b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7326ffd640 1 -- 172.21.15.105:0/824706742 learned_addr learned my addr 172.21.15.105:0/824706742 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f73277fe640 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 0x559f14a1c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7326ffd640 1 -- 172.21.15.105:0/824706742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 msgr2=0x559f14a1c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7326ffd640 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 0x559f14a1c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7326ffd640 1 -- 172.21.15.105:0/824706742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 msgr2=0x559f14a1b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:39.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7326ffd640 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a1b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.029+0000 7f7326ffd640 1 -- 172.21.15.105:0/824706742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f14a1cde0 con 0x559f14a0c100 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f7326ffd640 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a1b2a0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f7318009810 tx=0x7f7318005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7931990b6d0167 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f730aff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f730aff5640 1 -- 172.21.15.105:0/824706742 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73180a4a30 con 0x559f14a0c100 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f730aff5640 1 -- 172.21.15.105:0/824706742 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f73180bc0c0 con 0x559f14a0c100 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f7330165740 1 -- 172.21.15.105:0/824706742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c02c510 con 0x559f14a0c100 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f730aff5640 1 -- 172.21.15.105:0/824706742 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7318098690 con 0x559f14a0c100 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f7330165740 10 client.23874 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f7330165740 1 -- 172.21.15.105:0/824706742 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559f14a889d0 con 0x559f14a0c100 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.030+0000 7f730aff5640 1 -- 172.21.15.105:0/824706742 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f731c02c510 con 0x559f14a0c100 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f730aff5640 1 -- 172.21.15.105:0/824706742 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7318096b90 con 0x559f14a0c100 2024-03-26T18:01:39.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f7330165740 10 client.23874 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f7330165740 10 client.23874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f7330165740 20 client.23874 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f7330165740 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f14a89350 0x559f14a8b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f7330165740 4 client.23874 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:01:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f7330165740 1 -- 172.21.15.105:0/824706742 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f14970210 con 0x559f14a89350 2024-03-26T18:01:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f73267fc640 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f14a89350 0x559f14a8b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f730aff5640 10 client.23874 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:39.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.031+0000 7f73267fc640 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f14a89350 0x559f14a8b770 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e35cac6c251d9c2 server_cookie=e47e8d77cefcc78e in_seq=0 out_seq=0 2024-03-26T18:01:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.033+0000 7f730aff5640 1 -- 172.21.15.105:0/824706742 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x559f14970210 con 0x559f14a89350 2024-03-26T18:01:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.033+0000 7f730aff5640 10 client.23874 handle_command_reply: tid=0 2024-03-26T18:01:39.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.033+0000 7f7330165740 1 client.23874 shutdown 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.033+0000 7f7330165740 1 -- 172.21.15.105:0/824706742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 msgr2=0x559f14a1b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.033+0000 7f7330165740 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a1b2a0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f7318009810 tx=0x7f7318005f10 comp rx=0 tx=0).stop 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 1 -- 172.21.15.105:0/824706742 shutdown_connections 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x559f14a89350 0x559f14a8b770 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x559f14a11c60 0x559f14a1c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559f14a0caf0 0x559f14a1b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 1 --2- 172.21.15.105:0/824706742 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x559f14a0c100 0x559f14a1b2a0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f730aff5640 0 client.23874 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 1 -- 172.21.15.105:0/824706742 >> 172.21.15.105:0/824706742 conn(0x559f149fee60 msgr2=0x559f14a020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 1 -- 172.21.15.105:0/824706742 shutdown_connections 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 1 -- 172.21.15.105:0/824706742 wait complete. 2024-03-26T18:01:39.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.034+0000 7f7330165740 20 client.23874 trim_cache size 0 max 16384 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (358 inodes in the stack)", 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: "e399dbf4-b8f6-4e36-a7a0-39f573fedd98": { 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "e399dbf4-b8f6-4e36-a7a0-39f573fedd98", 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:01:39.022 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:01:39.023 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:01:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 -- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73100a2f70 msgr2=0x7f73100a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73100a2f70 0x7f73100a5430 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7320005fd0 tx=0x7f7320009f00 comp rx=0 tx=0).stop 2024-03-26T18:01:39.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 -- 172.21.15.105:0/1093338627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281007c0 msgr2=0x7f7328069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281007c0 0x7f7328069590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f731800e9c0 tx=0x7f731800ee80 comp rx=0 tx=0).stop 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 -- 172.21.15.105:0/1093338627 shutdown_connections 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f73100a2f70 0x7f73100a5430 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7328102370 0x7f732819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73281019b0 0x7f732819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 --2- 172.21.15.105:0/1093338627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73281007c0 0x7f7328069590 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 -- 172.21.15.105:0/1093338627 >> 172.21.15.105:0/1093338627 conn(0x7f73280fbd50 msgr2=0x7f7328104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 -- 172.21.15.105:0/1093338627 shutdown_connections 2024-03-26T18:01:39.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:01:39.035+0000 7f732d49a640 1 -- 172.21.15.105:0/1093338627 wait complete. 2024-03-26T18:01:39.031 INFO:tasks.cephfs.filesystem:scrub status for tag:e399dbf4-b8f6-4e36-a7a0-39f573fedd98 - {'path': '/', 'tag': 'e399dbf4-b8f6-4e36-a7a0-39f573fedd98', 'options': 'recursive,force'} 2024-03-26T18:01:39.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:38 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T18:01:39.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:38 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:39.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:38 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:39.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:38 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T18:01:39.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:38 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:39.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:38 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:39.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:38 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T18:01:39.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:38 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:39.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:38 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:39.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81410 7.05 MB/sec execute 502 sec latency 198.499 ms 2024-03-26T18:01:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 -- 172.21.15.105:0/1083437901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6960102ba0 msgr2=0x7f6960103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 --2- 172.21.15.105:0/1083437901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6960102ba0 0x7f6960103020 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f6950009930 tx=0x7f695002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 -- 172.21.15.105:0/1083437901 shutdown_connections 2024-03-26T18:01:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 --2- 172.21.15.105:0/1083437901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960103560 0x7f6960109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 --2- 172.21.15.105:0/1083437901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6960102ba0 0x7f6960103020 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 --2- 172.21.15.105:0/1083437901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69600ffbd0 0x7f69600ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 -- 172.21.15.105:0/1083437901 >> 172.21.15.105:0/1083437901 conn(0x7f69600fb500 msgr2=0x7f69600fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 -- 172.21.15.105:0/1083437901 shutdown_connections 2024-03-26T18:01:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.093+0000 7f6967348640 1 -- 172.21.15.105:0/1083437901 wait complete. 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f6967348640 1 Processor -- start 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f6967348640 1 -- start start 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f6967348640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69600ffbd0 0x7f6960199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f6967348640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6960102ba0 0x7f696019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f6967348640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960103560 0x7f69601a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f6967348640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f696010d460 con 0x7f69600ffbd0 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f6967348640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f696010d2e0 con 0x7f6960102ba0 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f6967348640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f696010d5e0 con 0x7f6960103560 2024-03-26T18:01:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69600ffbd0 0x7f6960199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f69648e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6960102ba0 0x7f696019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f69648e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6960102ba0 0x7f696019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55958/0 (socket says 172.21.15.105:55958) 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69600ffbd0 0x7f6960199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47252/0 (socket says 172.21.15.105:47252) 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f69648e3640 1 -- 172.21.15.105:0/3762831312 learned_addr learned my addr 172.21.15.105:0/3762831312 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.094+0000 7f69658e5640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960103560 0x7f69601a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69648e3640 1 -- 172.21.15.105:0/3762831312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960103560 msgr2=0x7f69601a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69648e3640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960103560 0x7f69601a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69648e3640 1 -- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69600ffbd0 msgr2=0x7f6960199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69648e3640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69600ffbd0 0x7f6960199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69648e3640 1 -- 172.21.15.105:0/3762831312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69601a1920 con 0x7f6960102ba0 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69650e4640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69600ffbd0 0x7f6960199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69658e5640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960103560 0x7f69601a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69648e3640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6960102ba0 0x7f696019a250 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6950009230 tx=0x7f6950005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69567fc640 1 -- 172.21.15.105:0/3762831312 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950043020 con 0x7f6960102ba0 2024-03-26T18:01:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f6967348640 1 -- 172.21.15.105:0/3762831312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69601a1bb0 con 0x7f6960102ba0 2024-03-26T18:01:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f6967348640 1 -- 172.21.15.105:0/3762831312 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69601a2150 con 0x7f6960102ba0 2024-03-26T18:01:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69567fc640 1 -- 172.21.15.105:0/3762831312 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69500043d0 con 0x7f6960102ba0 2024-03-26T18:01:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.095+0000 7f69567fc640 1 -- 172.21.15.105:0/3762831312 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950003af0 con 0x7f6960102ba0 2024-03-26T18:01:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.098+0000 7f69567fc640 1 -- 172.21.15.105:0/3762831312 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6950046020 con 0x7f6960102ba0 2024-03-26T18:01:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.099+0000 7f696a00b740 1 Processor -- start 2024-03-26T18:01:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.099+0000 7f696a00b740 1 -- start start 2024-03-26T18:01:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.099+0000 7f69567fc640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f694c0a3040 0x7f694c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.099+0000 7f696a00b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.099+0000 7f696a00b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0c6fc0 0x562cfd0cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.099+0000 7f69567fc640 1 -- 172.21.15.105:0/3762831312 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f69500f1610 con 0x7f6960102ba0 2024-03-26T18:01:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.099+0000 7f696a00b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0cc130 0x562cfd0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.099+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47260/0 (socket says 172.21.15.105:47260) 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69650e4640 1 -- 172.21.15.105:0/3685664591 learned_addr learned my addr 172.21.15.105:0/3685664591 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f696a00b740 1 -- 172.21.15.105:0/3685664591 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562cfd02b0b0 con 0x562cfd0c65d0 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f696a00b740 1 -- 172.21.15.105:0/3685664591 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562cfcfd3960 con 0x562cfd0cc130 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f696a00b740 1 -- 172.21.15.105:0/3685664591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562cfd027060 con 0x562cfd0c6fc0 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69658e5640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f694c0a3040 0x7f694c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69648e3640 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0c6fc0 0x562cfd0cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69658e5640 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0cc130 0x562cfd0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69650e4640 1 -- 172.21.15.105:0/3685664591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0c6fc0 msgr2=0x562cfd0cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69650e4640 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0c6fc0 0x562cfd0cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69650e4640 1 -- 172.21.15.105:0/3685664591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0cc130 msgr2=0x562cfd0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69650e4640 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0cc130 0x562cfd0ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69650e4640 1 -- 172.21.15.105:0/3685664591 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfcfe5c60 con 0x562cfd0c65d0 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69658e5640 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0cc130 0x562cfd0ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:01:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.100+0000 7f69648e3640 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0c6fc0 0x562cfd0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.101+0000 7f69658e5640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f694c0a3040 0x7f694c0a5500 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6958009fd0 tx=0x7f695800f580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.101+0000 7f69650e4640 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0c69f0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f695c0114f0 tx=0x7f695c0118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4500bcfb531f256 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.101+0000 7f694b7fe640 1 -- 172.21.15.105:0/3685664591 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c004310 con 0x562cfd0c65d0 2024-03-26T18:01:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.101+0000 7f694b7fe640 1 -- 172.21.15.105:0/3685664591 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695c0044b0 con 0x562cfd0c65d0 2024-03-26T18:01:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.101+0000 7f694b7fe640 1 -- 172.21.15.105:0/3685664591 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c0035f0 con 0x562cfd0c65d0 2024-03-26T18:01:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 -- 172.21.15.105:0/3685664591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 msgr2=0x562cfd0c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0c69f0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f695c0114f0 tx=0x7f695c0118f0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 -- 172.21.15.105:0/3685664591 shutdown_connections 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0cc130 0x562cfd0ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0c6fc0 0x562cfd0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 --2- 172.21.15.105:0/3685664591 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0c69f0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 -- 172.21.15.105:0/3685664591 >> 172.21.15.105:0/3685664591 conn(0x562cfd0c1b40 msgr2=0x562cfd0c1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 -- 172.21.15.105:0/3685664591 shutdown_connections 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 -- 172.21.15.105:0/3685664591 wait complete. 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 -- start start 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.102+0000 7f696a00b740 1 Processor -- start 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f696a00b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f696a00b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f696a00b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f696a00b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x562cfd0cf0e0 con 0x562cfd0c65d0 2024-03-26T18:01:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f696a00b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562cfcfe6db0 con 0x562cfd0c6fc0 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f696a00b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562cfd0cf260 con 0x562cfd0cc130 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f696a00b740 10 client.0 _ll_register_callbacks cb 0x562cfcfeb110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69648e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69648e3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55972/0 (socket says 172.21.15.105:55972) 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69658e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69648e3640 1 -- 172.21.15.105:0/2736258638 learned_addr learned my addr 172.21.15.105:0/2736258638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69658e5640 1 -- 172.21.15.105:0/2736258638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 msgr2=0x562cfd0d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69658e5640 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69658e5640 1 -- 172.21.15.105:0/2736258638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 msgr2=0x562cfd0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69658e5640 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.103+0000 7f69658e5640 1 -- 172.21.15.105:0/2736258638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfcd3f210 con 0x562cfd0cc130 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f69658e5640 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0d78a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6958003bf0 tx=0x7f6958004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb09c71031b41b61 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f6948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f6948ff9640 1 -- 172.21.15.105:0/2736258638 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695801f410 con 0x562cfd0cc130 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f6948ff9640 1 -- 172.21.15.105:0/2736258638 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f695801f5b0 con 0x562cfd0cc130 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f696a00b740 1 -- 172.21.15.105:0/2736258638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695c0044b0 con 0x562cfd0cc130 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f6948ff9640 1 -- 172.21.15.105:0/2736258638 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f695801f7b0 con 0x562cfd0cc130 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f696a00b740 10 client.33553 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f696a00b740 1 -- 172.21.15.105:0/2736258638 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562cfce0b620 con 0x562cfd0cc130 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.104+0000 7f6948ff9640 1 -- 172.21.15.105:0/2736258638 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695c0044b0 con 0x562cfd0cc130 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f6948ff9640 1 -- 172.21.15.105:0/2736258638 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f695801b070 con 0x562cfd0cc130 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f696a00b740 10 client.33553 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f696a00b740 10 client.33553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f696a00b740 20 client.33553 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f696a00b740 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562cfd0dc1c0 0x562cfd0de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f696a00b740 4 client.33553 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f696a00b740 1 -- 172.21.15.105:0/2736258638 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cfd0385f0 con 0x562cfd0dc1c0 2024-03-26T18:01:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f69650e4640 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562cfd0dc1c0 0x562cfd0de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f6948ff9640 10 client.33553 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.105+0000 7f69650e4640 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562cfd0dc1c0 0x562cfd0de5f0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1531c3d29797a9bd server_cookie=c2112fa6645d0f89 in_seq=0 out_seq=0 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.107+0000 7f6948ff9640 1 -- 172.21.15.105:0/2736258638 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562cfd0385f0 con 0x562cfd0dc1c0 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.107+0000 7f6948ff9640 10 client.33553 handle_command_reply: tid=0 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.107+0000 7f696a00b740 1 client.33553 shutdown 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.107+0000 7f696a00b740 1 -- 172.21.15.105:0/2736258638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 msgr2=0x562cfd0d78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.107+0000 7f696a00b740 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0d78a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6958003bf0 tx=0x7f6958004a40 comp rx=0 tx=0).stop 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 1 -- 172.21.15.105:0/2736258638 shutdown_connections 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562cfd0dc1c0 0x562cfd0de5f0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0d78a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 1 --2- 172.21.15.105:0/2736258638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 1 -- 172.21.15.105:0/2736258638 >> 172.21.15.105:0/2736258638 conn(0x562cfd0c1b40 msgr2=0x562cfd0c0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f6948ff9640 0 client.33553 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 1 -- 172.21.15.105:0/2736258638 shutdown_connections 2024-03-26T18:01:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 1 -- 172.21.15.105:0/2736258638 wait complete. 2024-03-26T18:01:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.108+0000 7f696a00b740 20 client.33553 trim_cache size 0 max 16384 2024-03-26T18:01:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f696a00b740 1 Processor -- start 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f696a00b740 1 -- start start 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f696a00b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f696a00b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f696a00b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f696a00b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69601aaf00 con 0x562cfd0c65d0 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f696a00b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69601aca40 con 0x562cfd0c6fc0 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f696a00b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69601ab5d0 con 0x562cfd0cc130 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47280/0 (socket says 172.21.15.105:47280) 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f69650e4640 1 -- 172.21.15.105:0/3190506869 learned_addr learned my addr 172.21.15.105:0/3190506869 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f69648e3640 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.129+0000 7f69658e5640 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f69648e3640 1 -- 172.21.15.105:0/3190506869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 msgr2=0x562cfd0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f69648e3640 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f69648e3640 1 -- 172.21.15.105:0/3190506869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 msgr2=0x562cfd0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f69648e3640 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f69648e3640 1 -- 172.21.15.105:0/3190506869 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfd0c73a0 con 0x562cfd0c6fc0 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f69650e4640 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f69658e5640 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f69648e3640 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d7860 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f69500ba640 tx=0x7f6950041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce4a210a905950a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f694affd640 1 -- 172.21.15.105:0/3190506869 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500be400 con 0x562cfd0c6fc0 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f694affd640 1 -- 172.21.15.105:0/3190506869 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6950003cf0 con 0x562cfd0c6fc0 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.130+0000 7f694affd640 1 -- 172.21.15.105:0/3190506869 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69500c64e0 con 0x562cfd0c6fc0 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 -- 172.21.15.105:0/3190506869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 msgr2=0x562cfd0d7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d7860 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f69500ba640 tx=0x7f6950041040 comp rx=0 tx=0).stop 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 -- 172.21.15.105:0/3190506869 shutdown_connections 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d7860 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 --2- 172.21.15.105:0/3190506869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 -- 172.21.15.105:0/3190506869 >> 172.21.15.105:0/3190506869 conn(0x562cfd0c1b40 msgr2=0x562cfd0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 -- 172.21.15.105:0/3190506869 shutdown_connections 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 -- 172.21.15.105:0/3190506869 wait complete. 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.131+0000 7f696a00b740 1 -- start start 2024-03-26T18:01:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f696a00b740 1 Processor -- start 2024-03-26T18:01:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f696a00b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f696a00b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f696a00b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f696a00b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69601ac370 con 0x562cfd0c65d0 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f696a00b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69601abca0 con 0x562cfd0c6fc0 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f696a00b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69601a8ee0 con 0x562cfd0cc130 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f696a00b740 10 client.0 _ll_register_callbacks cb 0x7f695800fe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69650e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47282/0 (socket says 172.21.15.105:47282) 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69650e4640 1 -- 172.21.15.105:0/3064404224 learned_addr learned my addr 172.21.15.105:0/3064404224 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69658e5640 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69648e3640 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69650e4640 1 -- 172.21.15.105:0/3064404224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 msgr2=0x562cfd140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69650e4640 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69650e4640 1 -- 172.21.15.105:0/3064404224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 msgr2=0x562cfd0d4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69650e4640 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69650e4640 1 -- 172.21.15.105:0/3064404224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfd0d7b00 con 0x562cfd0c65d0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.132+0000 7f69658e5640 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f69650e4640 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d44f0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f695c000c00 tx=0x7f695c0111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a8b113836279a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f6948ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f6948ff9640 1 -- 172.21.15.105:0/3064404224 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c032020 con 0x562cfd0c65d0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f6948ff9640 1 -- 172.21.15.105:0/3064404224 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f695c004190 con 0x562cfd0c65d0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f696a00b740 1 -- 172.21.15.105:0/3064404224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6950003cf0 con 0x562cfd0c65d0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f6948ff9640 1 -- 172.21.15.105:0/3064404224 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f695c03d050 con 0x562cfd0c65d0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f696a00b740 10 client.23892 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f696a00b740 1 -- 172.21.15.105:0/3064404224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562cfd141b40 con 0x562cfd0c65d0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.133+0000 7f6948ff9640 1 -- 172.21.15.105:0/3064404224 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6950003cf0 con 0x562cfd0c65d0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f6948ff9640 1 -- 172.21.15.105:0/3064404224 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f695c02e580 con 0x562cfd0c65d0 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f696a00b740 10 client.23892 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f696a00b740 10 client.23892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f696a00b740 20 client.23892 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f696a00b740 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562cfd142400 0x562cfd144820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f696a00b740 4 client.23892 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:01:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f696a00b740 1 -- 172.21.15.105:0/3064404224 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562cfd032940 con 0x562cfd142400 2024-03-26T18:01:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f69648e3640 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562cfd142400 0x562cfd144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f6948ff9640 10 client.23892 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.134+0000 7f69648e3640 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562cfd142400 0x562cfd144820 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c050e398fbfe44a server_cookie=c8ecf3f155563540 in_seq=0 out_seq=0 2024-03-26T18:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f6948ff9640 1 -- 172.21.15.105:0/3064404224 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562cfd032940 con 0x562cfd142400 2024-03-26T18:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f6948ff9640 10 client.23892 handle_command_reply: tid=0 2024-03-26T18:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 client.23892 shutdown 2024-03-26T18:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 -- 172.21.15.105:0/3064404224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 msgr2=0x562cfd0d44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d44f0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f695c000c00 tx=0x7f695c0111d0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 -- 172.21.15.105:0/3064404224 shutdown_connections 2024-03-26T18:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x562cfd142400 0x562cfd144820 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562cfd0cc130 0x562cfd140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f6948ff9640 0 client.23892 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562cfd0c6fc0 0x562cfd0d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 --2- 172.21.15.105:0/3064404224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562cfd0c65d0 0x562cfd0d44f0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 -- 172.21.15.105:0/3064404224 >> 172.21.15.105:0/3064404224 conn(0x562cfd0c1b40 msgr2=0x562cfd0c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 -- 172.21.15.105:0/3064404224 shutdown_connections 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.276+0000 7f696a00b740 1 -- 172.21.15.105:0/3064404224 wait complete. 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f696a00b740 20 client.23892 trim_cache size 0 max 16384 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:01:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:01:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 -- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f694c0a3040 msgr2=0x7f694c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f694c0a3040 0x7f694c0a5500 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6958009fd0 tx=0x7f695800f580 comp rx=0 tx=0).stop 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 -- 172.21.15.105:0/3762831312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6960102ba0 msgr2=0x7f696019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6960102ba0 0x7f696019a250 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6950009230 tx=0x7f6950005e40 comp rx=0 tx=0).stop 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 -- 172.21.15.105:0/3762831312 shutdown_connections 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f694c0a3040 0x7f694c0a5500 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960103560 0x7f69601a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6960102ba0 0x7f696019a250 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 --2- 172.21.15.105:0/3762831312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69600ffbd0 0x7f6960199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 -- 172.21.15.105:0/3762831312 >> 172.21.15.105:0/3762831312 conn(0x7f69600fb500 msgr2=0x7f6960100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 -- 172.21.15.105:0/3762831312 shutdown_connections 2024-03-26T18:01:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:40.277+0000 7f6967348640 1 -- 172.21.15.105:0/3762831312 wait complete. 2024-03-26T18:01:40.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: pgmap v1051: 97 pgs: 97 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-03-26T18:01:40.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:39 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:40.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:40.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:40.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:40.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:40.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:01:40.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: pgmap v1051: 97 pgs: 97 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-03-26T18:01:40.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:39 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:40.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: pgmap v1051: 97 pgs: 97 active+clean; 374 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.8 MiB/s wr, 237 op/s 2024-03-26T18:01:40.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:39 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:01:40.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81511 7.05 MB/sec execute 503 sec latency 206.925 ms 2024-03-26T18:01:41.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81639 7.06 MB/sec execute 504 sec latency 126.581 ms 2024-03-26T18:01:42.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:41 smithi144 ceph-mon[36095]: pgmap v1052: 97 pgs: 97 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-26T18:01:42.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:41 smithi105 ceph-mon[32280]: pgmap v1052: 97 pgs: 97 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-26T18:01:42.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:41 smithi135 ceph-mon[35636]: pgmap v1052: 97 pgs: 97 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.7 MiB/s wr, 209 op/s 2024-03-26T18:01:42.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81759 7.05 MB/sec execute 505 sec latency 148.197 ms 2024-03-26T18:01:43.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81892 7.06 MB/sec execute 506 sec latency 134.589 ms 2024-03-26T18:01:44.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:43 smithi144 ceph-mon[36095]: pgmap v1053: 97 pgs: 97 active+clean; 377 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 3.7 MiB/s wr, 230 op/s 2024-03-26T18:01:44.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:43 smithi105 ceph-mon[32280]: pgmap v1053: 97 pgs: 97 active+clean; 377 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 3.7 MiB/s wr, 230 op/s 2024-03-26T18:01:44.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:43 smithi135 ceph-mon[35636]: pgmap v1053: 97 pgs: 97 active+clean; 377 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 3.7 MiB/s wr, 230 op/s 2024-03-26T18:01:44.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 81994 7.05 MB/sec execute 507 sec latency 143.912 ms 2024-03-26T18:01:45.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 82135 7.05 MB/sec execute 508 sec latency 137.466 ms 2024-03-26T18:01:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:45 smithi144 ceph-mon[36095]: pgmap v1054: 97 pgs: 97 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 4.1 MiB/s wr, 269 op/s 2024-03-26T18:01:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:45 smithi105 ceph-mon[32280]: pgmap v1054: 97 pgs: 97 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 4.1 MiB/s wr, 269 op/s 2024-03-26T18:01:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:45 smithi135 ceph-mon[35636]: pgmap v1054: 97 pgs: 97 active+clean; 375 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 4.1 MiB/s wr, 269 op/s 2024-03-26T18:01:46.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 82217 7.04 MB/sec execute 509 sec latency 137.575 ms 2024-03-26T18:01:47.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 82334 7.05 MB/sec execute 510 sec latency 153.605 ms 2024-03-26T18:01:48.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:47 smithi144 ceph-mon[36095]: pgmap v1055: 97 pgs: 97 active+clean; 376 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-03-26T18:01:48.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:47 smithi105 ceph-mon[32280]: pgmap v1055: 97 pgs: 97 active+clean; 376 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-03-26T18:01:48.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:47 smithi135 ceph-mon[35636]: pgmap v1055: 97 pgs: 97 active+clean; 376 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 3.5 MiB/s wr, 258 op/s 2024-03-26T18:01:48.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 82444 7.04 MB/sec execute 511 sec latency 195.672 ms 2024-03-26T18:01:49.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:49 smithi144 ceph-mon[36095]: pgmap v1056: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-03-26T18:01:49.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:49 smithi105 ceph-mon[32280]: pgmap v1056: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-03-26T18:01:49.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:49 smithi135 ceph-mon[35636]: pgmap v1056: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 3.7 MiB/s wr, 258 op/s 2024-03-26T18:01:49.561 INFO:tasks.workunit.client.0.smithi105.stdout: 10 82577 7.04 MB/sec execute 512 sec latency 144.904 ms 2024-03-26T18:01:50.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 82707 7.03 MB/sec execute 513 sec latency 139.457 ms 2024-03-26T18:01:51.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:51 smithi144 ceph-mon[36095]: pgmap v1057: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-03-26T18:01:51.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:51 smithi105 ceph-mon[32280]: pgmap v1057: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-03-26T18:01:51.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:51 smithi135 ceph-mon[35636]: pgmap v1057: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.1 MiB/s wr, 242 op/s 2024-03-26T18:01:51.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 82832 7.03 MB/sec execute 514 sec latency 121.428 ms 2024-03-26T18:01:52.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:52 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:01:52.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:52 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:01:52.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:52 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:01:52.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 82950 7.03 MB/sec execute 515 sec latency 117.609 ms 2024-03-26T18:01:53.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:53 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:01:53.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:53 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:01:53.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:53 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:01:53.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:53 smithi144 ceph-mon[36095]: pgmap v1058: 97 pgs: 97 active+clean; 377 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-03-26T18:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 -- 172.21.15.105:0/1579034397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 msgr2=0x7f2364100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 --2- 172.21.15.105:0/1579034397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 0x7f2364100c40 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f2360009900 tx=0x7f236002bc30 comp rx=0 tx=0).stop 2024-03-26T18:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 -- 172.21.15.105:0/1579034397 shutdown_connections 2024-03-26T18:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 --2- 172.21.15.105:0/1579034397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23641023d0 0x7f2364108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 --2- 172.21.15.105:0/1579034397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2364101a10 0x7f2364101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 --2- 172.21.15.105:0/1579034397 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 0x7f2364100c40 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 -- 172.21.15.105:0/1579034397 >> 172.21.15.105:0/1579034397 conn(0x7f23640fbd90 msgr2=0x7f23640fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 -- 172.21.15.105:0/1579034397 shutdown_connections 2024-03-26T18:01:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.399+0000 7f236bf77640 1 -- 172.21.15.105:0/1579034397 wait complete. 2024-03-26T18:01:53.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:53 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:01:53.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:53 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:01:53.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:53 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:01:53.383 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:53 smithi105 ceph-mon[32280]: pgmap v1058: 97 pgs: 97 active+clean; 377 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f236bf77640 1 Processor -- start 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f236bf77640 1 -- start start 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f236bf77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 0x7f2364199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f236bf77640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2364101a10 0x7f236419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f236bf77640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23641023d0 0x7f23641a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f236bf77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f236410c350 con 0x7f2364100820 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f236bf77640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f236410c1d0 con 0x7f23641023d0 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f236bf77640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f236410c4d0 con 0x7f2364101a10 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f2369d13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 0x7f2364199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f2369d13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 0x7f2364199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43694/0 (socket says 172.21.15.105:43694) 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f2369d13640 1 -- 172.21.15.105:0/1698690422 learned_addr learned my addr 172.21.15.105:0/1698690422 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.400+0000 7f2369512640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2364101a10 0x7f236419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f236a514640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23641023d0 0x7f23641a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f2369d13640 1 -- 172.21.15.105:0/1698690422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2364101a10 msgr2=0x7f236419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f2369d13640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2364101a10 0x7f236419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f2369d13640 1 -- 172.21.15.105:0/1698690422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23641023d0 msgr2=0x7f23641a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f2369d13640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23641023d0 0x7f23641a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f2369d13640 1 -- 172.21.15.105:0/1698690422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23641a1900 con 0x7f2364100820 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f2369512640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2364101a10 0x7f236419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f236a514640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23641023d0 0x7f23641a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f2369d13640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 0x7f2364199cc0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f2360000c00 tx=0x7f2360005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f235affd640 1 -- 172.21.15.105:0/1698690422 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360043020 con 0x7f2364100820 2024-03-26T18:01:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f236bf77640 1 -- 172.21.15.105:0/1698690422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23641a1b90 con 0x7f2364100820 2024-03-26T18:01:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f235affd640 1 -- 172.21.15.105:0/1698690422 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2360004350 con 0x7f2364100820 2024-03-26T18:01:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f235affd640 1 -- 172.21.15.105:0/1698690422 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360003b80 con 0x7f2364100820 2024-03-26T18:01:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.401+0000 7f236bf77640 1 -- 172.21.15.105:0/1698690422 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23641a20e0 con 0x7f2364100820 2024-03-26T18:01:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.404+0000 7f235affd640 1 -- 172.21.15.105:0/1698690422 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2360046020 con 0x7f2364100820 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.404+0000 7f235affd640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23500a2fc0 0x7f23500a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f236ec3a740 1 Processor -- start 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f235affd640 1 -- 172.21.15.105:0/1698690422 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f23600f1580 con 0x7f2364100820 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f236ec3a740 1 -- start start 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f8059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f80aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f80d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f236ec3a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56543f76a0b0 con 0x56543f80b130 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f2369512640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23500a2fc0 0x7f23500a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f236ec3a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56543f712960 con 0x56543f8055d0 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f236ec3a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56543f766060 con 0x56543f805fc0 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f2369d13640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f8059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f2369d13640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f8059f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40486/0 (socket says 172.21.15.105:40486) 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.405+0000 7f2369d13640 1 -- 172.21.15.105:0/2760134616 learned_addr learned my addr 172.21.15.105:0/2760134616 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.406+0000 7f236a514640 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f80d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.406+0000 7f2369d13640 1 -- 172.21.15.105:0/2760134616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 msgr2=0x56543f80aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.406+0000 7f2369d13640 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f80aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.406+0000 7f2369d13640 1 -- 172.21.15.105:0/2760134616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 msgr2=0x56543f80d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.406+0000 7f2369d13640 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f80d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.406+0000 7f2369d13640 1 -- 172.21.15.105:0/2760134616 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543f724c60 con 0x56543f8055d0 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.406+0000 7f236a514640 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f80d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.406+0000 7f2369512640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23500a2fc0 0x7f23500a5480 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f236419b1c0 tx=0x7f2354011860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.407+0000 7f2369d13640 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f8059f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f23600bdc00 tx=0x7f23600bdb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9319c485b505c32b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.407+0000 7f234ffff640 1 -- 172.21.15.105:0/2760134616 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23600bdc70 con 0x56543f8055d0 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.407+0000 7f234ffff640 1 -- 172.21.15.105:0/2760134616 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23600c1dc0 con 0x56543f8055d0 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.407+0000 7f234ffff640 1 -- 172.21.15.105:0/2760134616 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23600ba2e0 con 0x56543f8055d0 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 -- 172.21.15.105:0/2760134616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 msgr2=0x56543f8059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f8059f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f23600bdc00 tx=0x7f23600bdb00 comp rx=0 tx=0).stop 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 -- 172.21.15.105:0/2760134616 shutdown_connections 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f80d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f80aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 --2- 172.21.15.105:0/2760134616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f8059f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 -- 172.21.15.105:0/2760134616 >> 172.21.15.105:0/2760134616 conn(0x56543f800b40 msgr2=0x56543f800f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 -- 172.21.15.105:0/2760134616 shutdown_connections 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 -- 172.21.15.105:0/2760134616 wait complete. 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 -- start start 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.408+0000 7f236ec3a740 1 Processor -- start 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f8055d0 0x56543f80f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f805fc0 0x56543f80f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f8165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236ec3a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56543f80dea0 con 0x56543f805fc0 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236ec3a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56543f725db0 con 0x56543f80b130 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236ec3a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56543f80e020 con 0x56543f8055d0 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236a514640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f8165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236a514640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f8165d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:40492/0 (socket says 172.21.15.105:40492) 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236a514640 1 -- 172.21.15.105:0/3296617315 learned_addr learned my addr 172.21.15.105:0/3296617315 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236ec3a740 10 client.0 _ll_register_callbacks cb 0x56543f72a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f2369512640 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f805fc0 0x56543f80f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236a514640 1 -- 172.21.15.105:0/3296617315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f8055d0 msgr2=0x56543f80f050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236a514640 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f8055d0 0x56543f80f050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236a514640 1 -- 172.21.15.105:0/3296617315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f805fc0 msgr2=0x56543f80f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236a514640 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f805fc0 0x56543f80f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.409+0000 7f236a514640 1 -- 172.21.15.105:0/3296617315 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543f47e210 con 0x56543f80b130 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.410+0000 7f2369512640 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f805fc0 0x56543f80f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.410+0000 7f236a514640 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f8165d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f235c00fe40 tx=0x7f235c012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85839f93f2b73147 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.410+0000 7f234d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.410+0000 7f234d7fa640 1 -- 172.21.15.105:0/3296617315 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c003970 con 0x56543f80b130 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.410+0000 7f234d7fa640 1 -- 172.21.15.105:0/3296617315 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f235c003b10 con 0x56543f80b130 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.410+0000 7f236ec3a740 1 -- 172.21.15.105:0/3296617315 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23600c1dc0 con 0x56543f80b130 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f234d7fa640 1 -- 172.21.15.105:0/3296617315 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f235c003d10 con 0x56543f80b130 2024-03-26T18:01:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f236ec3a740 10 client.33569 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f236ec3a740 1 -- 172.21.15.105:0/3296617315 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56543f54a620 con 0x56543f80b130 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f234d7fa640 1 -- 172.21.15.105:0/3296617315 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f235c0044e0 con 0x56543f80b130 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f234d7fa640 1 -- 172.21.15.105:0/3296617315 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f235c01c790 con 0x56543f80b130 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f236ec3a740 10 client.33569 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f236ec3a740 10 client.33569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f236ec3a740 20 client.33569 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f236ec3a740 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56543f81b340 0x56543f81d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f236ec3a740 4 client.33569 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f236ec3a740 1 -- 172.21.15.105:0/3296617315 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f23600c1dc0 con 0x56543f81b340 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.411+0000 7f2369d13640 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56543f81b340 0x56543f81d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.412+0000 7f234d7fa640 10 client.33569 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.412+0000 7f2369d13640 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56543f81b340 0x56543f81d7a0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e54e36ef4ec4e52 server_cookie=1701c974909cbc6 in_seq=0 out_seq=0 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.413+0000 7f234d7fa640 1 -- 172.21.15.105:0/3296617315 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f23600c1dc0 con 0x56543f81b340 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.413+0000 7f234d7fa640 10 client.33569 handle_command_reply: tid=0 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.413+0000 7f236ec3a740 1 client.33569 shutdown 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.413+0000 7f236ec3a740 1 -- 172.21.15.105:0/3296617315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 msgr2=0x56543f8165d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.413+0000 7f236ec3a740 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f8165d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f235c00fe40 tx=0x7f235c012820 comp rx=0 tx=0).stop 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.413+0000 7f236ec3a740 1 -- 172.21.15.105:0/3296617315 shutdown_connections 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.413+0000 7f236ec3a740 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56543f81b340 0x56543f81d7a0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.414+0000 7f236ec3a740 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f8165d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.414+0000 7f236ec3a740 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f805fc0 0x56543f80f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.414+0000 7f236ec3a740 1 --2- 172.21.15.105:0/3296617315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f8055d0 0x56543f80f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.414+0000 7f236ec3a740 1 -- 172.21.15.105:0/3296617315 >> 172.21.15.105:0/3296617315 conn(0x56543f800b40 msgr2=0x56543f7ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.414+0000 7f234d7fa640 0 client.33569 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.414+0000 7f236ec3a740 1 -- 172.21.15.105:0/3296617315 shutdown_connections 2024-03-26T18:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.414+0000 7f236ec3a740 1 -- 172.21.15.105:0/3296617315 wait complete. 2024-03-26T18:01:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.414+0000 7f236ec3a740 20 client.33569 trim_cache size 0 max 16384 2024-03-26T18:01:53.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:53 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:01:53.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:53 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:01:53.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:53 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:01:53.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:53 smithi135 ceph-mon[35636]: pgmap v1058: 97 pgs: 97 active+clean; 377 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 3.3 MiB/s wr, 261 op/s 2024-03-26T18:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.436+0000 7f236ec3a740 1 Processor -- start 2024-03-26T18:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f236ec3a740 1 -- start start 2024-03-26T18:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f8055d0 0x56543f806a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f816590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f806fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f236ec3a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23641aaf00 con 0x56543f8055d0 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f236ec3a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23641aca40 con 0x56543f80b130 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f2369512640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f816590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f2369d13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f8055d0 0x56543f806a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f2369512640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f816590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:41104/0 (socket says 172.21.15.105:41104) 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f2369512640 1 -- 172.21.15.105:0/2453804587 learned_addr learned my addr 172.21.15.105:0/2453804587 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f236ec3a740 1 -- 172.21.15.105:0/2453804587 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23641ab5d0 con 0x56543f805fc0 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f236a514640 1 --2- 172.21.15.105:0/2453804587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f806fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f2369512640 1 -- 172.21.15.105:0/2453804587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 msgr2=0x56543f806fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.437+0000 7f2369512640 1 --2- 172.21.15.105:0/2453804587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f806fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.438+0000 7f2369512640 1 -- 172.21.15.105:0/2453804587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f8055d0 msgr2=0x56543f806a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.438+0000 7f2369512640 1 --2- 172.21.15.105:0/2453804587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f8055d0 0x56543f806a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.438+0000 7f2369512640 1 -- 172.21.15.105:0/2453804587 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543f8018b0 con 0x56543f805fc0 2024-03-26T18:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.438+0000 7f2369512640 1 --2- 172.21.15.105:0/2453804587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f816590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2354004370 tx=0x7f235401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef18771ca224769 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.438+0000 7f234f7fe640 1 -- 172.21.15.105:0/2453804587 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354023070 con 0x56543f805fc0 2024-03-26T18:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.438+0000 7f234f7fe640 1 -- 172.21.15.105:0/2453804587 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2354019050 con 0x56543f805fc0 2024-03-26T18:01:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.438+0000 7f234f7fe640 1 -- 172.21.15.105:0/2453804587 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354017030 con 0x56543f805fc0 2024-03-26T18:01:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 -- 172.21.15.105:0/2453804587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 msgr2=0x56543f816590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 --2- 172.21.15.105:0/2453804587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f816590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2354004370 tx=0x7f235401a000 comp rx=0 tx=0).stop 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 -- 172.21.15.105:0/2453804587 shutdown_connections 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 --2- 172.21.15.105:0/2453804587 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f80b130 0x56543f806fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 --2- 172.21.15.105:0/2453804587 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f816590 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 --2- 172.21.15.105:0/2453804587 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f8055d0 0x56543f806a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 -- 172.21.15.105:0/2453804587 >> 172.21.15.105:0/2453804587 conn(0x56543f800b40 msgr2=0x56543f80a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 -- 172.21.15.105:0/2453804587 shutdown_connections 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.439+0000 7f236ec3a740 1 -- 172.21.15.105:0/2453804587 wait complete. 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.440+0000 7f236ec3a740 1 -- start start 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.440+0000 7f236ec3a740 1 Processor -- start 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.440+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f813220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.440+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f813760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.440+0000 7f236ec3a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f87f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.440+0000 7f236ec3a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23641ac370 con 0x56543f80b130 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.440+0000 7f236ec3a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f23641abca0 con 0x56543f8055d0 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.440+0000 7f236ec3a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23641a8ee0 con 0x56543f805fc0 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f236a514640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f87f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f236a514640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f87f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:43720/0 (socket says 172.21.15.105:43720) 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f236a514640 1 -- 172.21.15.105:0/1114421324 learned_addr learned my addr 172.21.15.105:0/1114421324 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369512640 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f813760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f236ec3a740 10 client.0 _ll_register_callbacks cb 0x7f235c00f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369d13640 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f813220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369512640 1 -- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 msgr2=0x56543f813220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369512640 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f813220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369512640 1 -- 172.21.15.105:0/1114421324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 msgr2=0x56543f87f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369512640 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f87f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369512640 1 -- 172.21.15.105:0/1114421324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543f8167e0 con 0x56543f805fc0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369d13640 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f813220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f2369512640 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f813760 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x56543f8005c0 tx=0x7f23540056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2091a9f0f411d5b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.441+0000 7f234d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f234d7fa640 1 -- 172.21.15.105:0/1114421324 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2354023040 con 0x56543f805fc0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f234d7fa640 1 -- 172.21.15.105:0/1114421324 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f235400f7a0 con 0x56543f805fc0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f236ec3a740 1 -- 172.21.15.105:0/1114421324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2354019050 con 0x56543f805fc0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f234d7fa640 1 -- 172.21.15.105:0/1114421324 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f235400f9a0 con 0x56543f805fc0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f236ec3a740 10 client.33571 fetch_fsmap learned FSMap version 43 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f236ec3a740 1 -- 172.21.15.105:0/1114421324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56543f880c00 con 0x56543f805fc0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f234d7fa640 1 -- 172.21.15.105:0/1114421324 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2354019050 con 0x56543f805fc0 2024-03-26T18:01:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f234d7fa640 1 -- 172.21.15.105:0/1114421324 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f23540284d0 con 0x56543f805fc0 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f236ec3a740 10 client.33571 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f236ec3a740 10 client.33571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f236ec3a740 20 client.33571 populate_metadata read hostname 'smithi105' 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.442+0000 7f236ec3a740 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56543f881500 0x56543f883920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.443+0000 7f236ec3a740 4 client.33571 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.443+0000 7f236ec3a740 1 -- 172.21.15.105:0/1114421324 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56543f7775f0 con 0x56543f881500 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.443+0000 7f2369d13640 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56543f881500 0x56543f883920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.443+0000 7f2369d13640 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56543f881500 0x56543f883920 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b92aab605c6be8 server_cookie=8ca0ac125e8190a4 in_seq=0 out_seq=0 2024-03-26T18:01:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.443+0000 7f234d7fa640 10 client.33571 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:53.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 83086 7.03 MB/sec execute 516 sec latency 174.630 ms 2024-03-26T18:01:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f234d7fa640 1 -- 172.21.15.105:0/1114421324 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56543f7775f0 con 0x56543f881500 2024-03-26T18:01:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f234d7fa640 10 client.33571 handle_command_reply: tid=0 2024-03-26T18:01:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 client.33571 shutdown 2024-03-26T18:01:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 -- 172.21.15.105:0/1114421324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 msgr2=0x56543f813760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f813760 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x56543f8005c0 tx=0x7f23540056a0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 -- 172.21.15.105:0/1114421324 shutdown_connections 2024-03-26T18:01:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56543f881500 0x56543f883920 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56543f80b130 0x56543f87f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56543f805fc0 0x56543f813760 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 --2- 172.21.15.105:0/1114421324 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56543f8055d0 0x56543f813220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f234d7fa640 0 client.33571 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 -- 172.21.15.105:0/1114421324 >> 172.21.15.105:0/1114421324 conn(0x56543f800b40 msgr2=0x56543f803da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 -- 172.21.15.105:0/1114421324 shutdown_connections 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.581+0000 7f236ec3a740 1 -- 172.21.15.105:0/1114421324 wait complete. 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236ec3a740 20 client.33571 trim_cache size 0 max 16384 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:01:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:01:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:01:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:01:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 -- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23500a2fc0 msgr2=0x7f23500a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23500a2fc0 0x7f23500a5480 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f236419b1c0 tx=0x7f2354011860 comp rx=0 tx=0).stop 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 -- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 msgr2=0x7f2364199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 0x7f2364199cc0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f2360000c00 tx=0x7f2360005dc0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 -- 172.21.15.105:0/1698690422 shutdown_connections 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f23500a2fc0 0x7f23500a5480 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f23641023d0 0x7f23641a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2364101a10 0x7f236419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 --2- 172.21.15.105:0/1698690422 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2364100820 0x7f2364199cc0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 -- 172.21.15.105:0/1698690422 >> 172.21.15.105:0/1698690422 conn(0x7f23640fbd90 msgr2=0x7f23640fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 -- 172.21.15.105:0/1698690422 shutdown_connections 2024-03-26T18:01:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:01:53.582+0000 7f236bf77640 1 -- 172.21.15.105:0/1698690422 wait complete. 2024-03-26T18:01:54.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 83233 7.03 MB/sec execute 517 sec latency 210.421 ms 2024-03-26T18:01:55.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:55 smithi144 ceph-mon[36095]: pgmap v1059: 97 pgs: 97 active+clean; 372 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-26T18:01:55.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:55 smithi105 ceph-mon[32280]: pgmap v1059: 97 pgs: 97 active+clean; 372 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-26T18:01:55.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:55 smithi135 ceph-mon[35636]: pgmap v1059: 97 pgs: 97 active+clean; 372 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-26T18:01:55.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 83356 7.03 MB/sec execute 518 sec latency 153.130 ms 2024-03-26T18:01:56.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 83490 7.04 MB/sec execute 519 sec latency 121.607 ms 2024-03-26T18:01:57.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:57 smithi144 ceph-mon[36095]: pgmap v1060: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 3.0 MiB/s wr, 212 op/s 2024-03-26T18:01:57.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:57 smithi105 ceph-mon[32280]: pgmap v1060: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 3.0 MiB/s wr, 212 op/s 2024-03-26T18:01:57.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:57 smithi135 ceph-mon[35636]: pgmap v1060: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 3.0 MiB/s wr, 212 op/s 2024-03-26T18:01:57.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 83666 7.04 MB/sec execute 520 sec latency 117.241 ms 2024-03-26T18:01:58.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 83779 7.04 MB/sec execute 521 sec latency 114.799 ms 2024-03-26T18:01:59.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:01:59 smithi144 ceph-mon[36095]: pgmap v1061: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-26T18:01:59.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:01:59 smithi105 ceph-mon[32280]: pgmap v1061: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-26T18:01:59.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:01:59 smithi135 ceph-mon[35636]: pgmap v1061: 97 pgs: 97 active+clean; 369 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 3.6 MiB/s wr, 246 op/s 2024-03-26T18:01:59.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 83903 7.04 MB/sec execute 522 sec latency 121.747 ms 2024-03-26T18:02:00.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 84019 7.03 MB/sec execute 523 sec latency 124.858 ms 2024-03-26T18:02:01.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:01 smithi144 ceph-mon[36095]: pgmap v1062: 97 pgs: 97 active+clean; 370 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-03-26T18:02:01.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:01 smithi105 ceph-mon[32280]: pgmap v1062: 97 pgs: 97 active+clean; 370 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-03-26T18:02:01.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:01 smithi135 ceph-mon[35636]: pgmap v1062: 97 pgs: 97 active+clean; 370 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 641 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-03-26T18:02:01.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 84152 7.02 MB/sec execute 524 sec latency 134.469 ms 2024-03-26T18:02:02.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 84277 7.01 MB/sec execute 525 sec latency 133.090 ms 2024-03-26T18:02:03.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:03 smithi144 ceph-mon[36095]: pgmap v1063: 97 pgs: 97 active+clean; 369 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-03-26T18:02:03.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:03 smithi105 ceph-mon[32280]: pgmap v1063: 97 pgs: 97 active+clean; 369 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-03-26T18:02:03.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:03 smithi135 ceph-mon[35636]: pgmap v1063: 97 pgs: 97 active+clean; 369 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-03-26T18:02:03.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 84409 7.01 MB/sec execute 526 sec latency 209.946 ms 2024-03-26T18:02:04.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 84544 7.01 MB/sec execute 527 sec latency 110.967 ms 2024-03-26T18:02:05.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:05 smithi144 ceph-mon[36095]: pgmap v1064: 97 pgs: 97 active+clean; 372 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-03-26T18:02:05.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:05 smithi105 ceph-mon[32280]: pgmap v1064: 97 pgs: 97 active+clean; 372 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-03-26T18:02:05.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:05 smithi135 ceph-mon[35636]: pgmap v1064: 97 pgs: 97 active+clean; 372 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 2.8 MiB/s wr, 247 op/s 2024-03-26T18:02:05.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 84701 7.02 MB/sec execute 528 sec latency 111.833 ms 2024-03-26T18:02:06.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 84853 7.02 MB/sec execute 529 sec latency 121.757 ms 2024-03-26T18:02:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.701+0000 7f7f03962640 1 -- 172.21.15.105:0/2902927734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc102b00 msgr2=0x7f7efc102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.701+0000 7f7f03962640 1 --2- 172.21.15.105:0/2902927734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc102b00 0x7f7efc102f80 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009900 tx=0x7f7eec02bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.701+0000 7f7f03962640 1 -- 172.21.15.105:0/2902927734 shutdown_connections 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.701+0000 7f7f03962640 1 --2- 172.21.15.105:0/2902927734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7efc1034c0 0x7f7efc109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.701+0000 7f7f03962640 1 --2- 172.21.15.105:0/2902927734 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc102b00 0x7f7efc102f80 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.701+0000 7f7f03962640 1 --2- 172.21.15.105:0/2902927734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc101910 0x7f7efc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.701+0000 7f7f03962640 1 -- 172.21.15.105:0/2902927734 >> 172.21.15.105:0/2902927734 conn(0x7f7efc0fce80 msgr2=0x7f7efc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.701+0000 7f7f03962640 1 -- 172.21.15.105:0/2902927734 shutdown_connections 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 -- 172.21.15.105:0/2902927734 wait complete. 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 Processor -- start 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 -- start start 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc101910 0x7f7efc199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.685 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7efc102b00 0x7f7efc19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc1034c0 0x7f7efc1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc10d470 con 0x7f7efc1034c0 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc10d2f0 con 0x7f7efc101910 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f03962640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc10d5f0 con 0x7f7efc102b00 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f016fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc101910 0x7f7efc199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.702+0000 7f7f01eff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc1034c0 0x7f7efc1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f016fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc101910 0x7f7efc199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41492/0 (socket says 172.21.15.105:41492) 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f016fe640 1 -- 172.21.15.105:0/433608072 learned_addr learned my addr 172.21.15.105:0/433608072 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:06.686 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f01eff640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc1034c0 0x7f7efc1a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38038/0 (socket says 172.21.15.105:38038) 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f00efd640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7efc102b00 0x7f7efc19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f00efd640 1 -- 172.21.15.105:0/433608072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc101910 msgr2=0x7f7efc199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f00efd640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc101910 0x7f7efc199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f00efd640 1 -- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc1034c0 msgr2=0x7f7efc1a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f00efd640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc1034c0 0x7f7efc1a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f00efd640 1 -- 172.21.15.105:0/433608072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc1a18e0 con 0x7f7efc102b00 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f01eff640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc1034c0 0x7f7efc1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f00efd640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7efc102b00 0x7f7efc19a230 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7eec02c170 tx=0x7f7eec005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7ef27fc640 1 -- 172.21.15.105:0/433608072 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec043020 con 0x7f7efc102b00 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7ef27fc640 1 -- 172.21.15.105:0/433608072 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7eec0043d0 con 0x7f7efc102b00 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f03962640 1 -- 172.21.15.105:0/433608072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7efc1a1b70 con 0x7f7efc102b00 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7ef27fc640 1 -- 172.21.15.105:0/433608072 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec003ae0 con 0x7f7efc102b00 2024-03-26T18:02:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.703+0000 7f7f03962640 1 -- 172.21.15.105:0/433608072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7efc1a2090 con 0x7f7efc102b00 2024-03-26T18:02:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.706+0000 7f7ef27fc640 1 -- 172.21.15.105:0/433608072 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f7eec046020 con 0x7f7efc102b00 2024-03-26T18:02:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f0662b740 1 Processor -- start 2024-03-26T18:02:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f0662b740 1 -- start start 2024-03-26T18:02:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd625d0 0x55a1cdd629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd67ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd6a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f0662b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a1cdcc70b0 con 0x55a1cdd62fc0 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f0662b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a1cdc6f960 con 0x55a1cdd625d0 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f0662b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a1cdcc3060 con 0x55a1cdd68130 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f01eff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd625d0 0x55a1cdd629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f016fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd67ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f01eff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd625d0 0x55a1cdd629f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41496/0 (socket says 172.21.15.105:41496) 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f01eff640 1 -- 172.21.15.105:0/2531877662 learned_addr learned my addr 172.21.15.105:0/2531877662 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7ef27fc640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ee80a2fc0 0x7f7ee80a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.707+0000 7f7f00efd640 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd6a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f01eff640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ee80a2fc0 0x7f7ee80a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7ef27fc640 1 -- 172.21.15.105:0/433608072 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f7eec0f1540 con 0x7f7efc102b00 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f01eff640 1 -- 172.21.15.105:0/2531877662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 msgr2=0x55a1cdd6a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f01eff640 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd6a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f01eff640 1 -- 172.21.15.105:0/2531877662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 msgr2=0x55a1cdd67ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f01eff640 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd67ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f01eff640 1 -- 172.21.15.105:0/2531877662 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1cdc81c60 con 0x55a1cdd625d0 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f016fe640 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd67ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f01eff640 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd625d0 0x55a1cdd629f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009fd0 tx=0x7f7ef4020cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cea5e48ea02615 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7f01eff640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ee80a2fc0 0x7f7ee80a5480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7ef402a000 tx=0x7f7ef4027000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2531877662 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef402d070 con 0x55a1cdd625d0 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2531877662 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef400f850 con 0x55a1cdd625d0 2024-03-26T18:02:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.708+0000 7f7ee77fe640 1 -- 172.21.15.105:0/2531877662 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4010790 con 0x55a1cdd625d0 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 -- 172.21.15.105:0/2531877662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd625d0 msgr2=0x55a1cdd629f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd625d0 0x55a1cdd629f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009fd0 tx=0x7f7ef4020cc0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 -- 172.21.15.105:0/2531877662 shutdown_connections 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd6a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd67ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2531877662 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd625d0 0x55a1cdd629f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 -- 172.21.15.105:0/2531877662 >> 172.21.15.105:0/2531877662 conn(0x55a1cdd5db40 msgr2=0x55a1cdd5df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 -- 172.21.15.105:0/2531877662 shutdown_connections 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.709+0000 7f7f0662b740 1 -- 172.21.15.105:0/2531877662 wait complete. 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 1 -- start start 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 1 Processor -- start 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd625d0 0x55a1cdd6c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd6c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd68130 0x55a1cdd739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a1cdd6b0e0 con 0x55a1cdd62fc0 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a1cdc82db0 con 0x55a1cdd68130 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a1cdd6b260 con 0x55a1cdd625d0 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f0662b740 10 client.0 _ll_register_callbacks cb 0x55a1cdc87110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f01eff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd68130 0x55a1cdd739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f01eff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd68130 0x55a1cdd739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41498/0 (socket says 172.21.15.105:41498) 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.710+0000 7f7f01eff640 1 -- 172.21.15.105:0/2191046438 learned_addr learned my addr 172.21.15.105:0/2191046438 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f00efd640 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd6c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f016fe640 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd625d0 0x55a1cdd6c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f01eff640 1 -- 172.21.15.105:0/2191046438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd625d0 msgr2=0x55a1cdd6c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f01eff640 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd625d0 0x55a1cdd6c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f01eff640 1 -- 172.21.15.105:0/2191046438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 msgr2=0x55a1cdd6c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f01eff640 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd6c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f01eff640 1 -- 172.21.15.105:0/2191046438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1cd9db210 con 0x55a1cdd68130 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f00efd640 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd6c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f01eff640 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd68130 0x55a1cdd739a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009560 tx=0x7f7ef4004300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29e83939f293722 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7ee4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/2191046438 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef402d030 con 0x55a1cdd68130 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7f0662b740 1 -- 172.21.15.105:0/2191046438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef400f850 con 0x55a1cdd68130 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.711+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/2191046438 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f7ef4004c50 con 0x55a1cdd68130 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/2191046438 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7ef4004e50 con 0x55a1cdd68130 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f0662b740 10 client.33587 fetch_fsmap learned FSMap version 43 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f0662b740 1 -- 172.21.15.105:0/2191046438 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a1cdaa7620 con 0x55a1cdd68130 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/2191046438 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef400f850 con 0x55a1cdd68130 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/2191046438 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7ef4027070 con 0x55a1cdd68130 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f0662b740 10 client.33587 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:02:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f0662b740 10 client.33587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f0662b740 20 client.33587 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a1cdd780e0 0x55a1cdd7a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f0662b740 4 client.33587 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f0662b740 1 -- 172.21.15.105:0/2191046438 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1cdcd45f0 con 0x55a1cdd780e0 2024-03-26T18:02:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.712+0000 7f7f016fe640 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a1cdd780e0 0x55a1cdd7a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.713+0000 7f7ee4ff9640 10 client.33587 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.713+0000 7f7f016fe640 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a1cdd780e0 0x55a1cdd7a4d0 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46455a6f4d543fc3 server_cookie=55ccf350adc22706 in_seq=0 out_seq=0 2024-03-26T18:02:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.714+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/2191046438 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a1cdcd45f0 con 0x55a1cdd780e0 2024-03-26T18:02:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.714+0000 7f7ee4ff9640 10 client.33587 handle_command_reply: tid=0 2024-03-26T18:02:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.714+0000 7f7f0662b740 1 client.33587 shutdown 2024-03-26T18:02:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 -- 172.21.15.105:0/2191046438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd68130 msgr2=0x55a1cdd739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd68130 0x55a1cdd739a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009560 tx=0x7f7ef4004300 comp rx=0 tx=0).stop 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 -- 172.21.15.105:0/2191046438 shutdown_connections 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a1cdd780e0 0x55a1cdd7a4d0 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd68130 0x55a1cdd739a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7ee4ff9640 0 client.33587 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd6c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 --2- 172.21.15.105:0/2191046438 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd625d0 0x55a1cdd6c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 -- 172.21.15.105:0/2191046438 >> 172.21.15.105:0/2191046438 conn(0x55a1cdd5db40 msgr2=0x55a1cdd5cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 -- 172.21.15.105:0/2191046438 shutdown_connections 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 1 -- 172.21.15.105:0/2191046438 wait complete. 2024-03-26T18:02:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.715+0000 7f7f0662b740 20 client.33587 trim_cache size 0 max 16384 2024-03-26T18:02:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.738+0000 7f7f0662b740 1 Processor -- start 2024-03-26T18:02:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.738+0000 7f7f0662b740 1 -- start start 2024-03-26T18:02:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd63a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd73960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd63fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f0662b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1a6b60 con 0x55a1cdd625d0 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f0662b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1a86a0 con 0x55a1cdd62fc0 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f0662b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1a7230 con 0x55a1cdd68130 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f016fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd63a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f01eff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd63fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f016fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd63a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38068/0 (socket says 172.21.15.105:38068) 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f016fe640 1 -- 172.21.15.105:0/944962897 learned_addr learned my addr 172.21.15.105:0/944962897 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f00efd640 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd73960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f016fe640 1 -- 172.21.15.105:0/944962897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 msgr2=0x55a1cdd63fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f016fe640 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd63fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f016fe640 1 -- 172.21.15.105:0/944962897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 msgr2=0x55a1cdd73960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f016fe640 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd73960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f016fe640 1 -- 172.21.15.105:0/944962897 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1cdd7a840 con 0x55a1cdd625d0 2024-03-26T18:02:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.739+0000 7f7f01eff640 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd63fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.740+0000 7f7f016fe640 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd63a60 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0ff170 tx=0x7f7ef8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c70b6ae664dc2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.740+0000 7f7ee6ffd640 1 -- 172.21.15.105:0/944962897 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8024070 con 0x55a1cdd625d0 2024-03-26T18:02:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.740+0000 7f7ee6ffd640 1 -- 172.21.15.105:0/944962897 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef80054d0 con 0x55a1cdd625d0 2024-03-26T18:02:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.740+0000 7f7ee6ffd640 1 -- 172.21.15.105:0/944962897 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8004ce0 con 0x55a1cdd625d0 2024-03-26T18:02:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 -- 172.21.15.105:0/944962897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 msgr2=0x55a1cdd63a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd63a60 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0ff170 tx=0x7f7ef8005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 -- 172.21.15.105:0/944962897 shutdown_connections 2024-03-26T18:02:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdd63fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd73960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 --2- 172.21.15.105:0/944962897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd63a60 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 -- 172.21.15.105:0/944962897 >> 172.21.15.105:0/944962897 conn(0x55a1cdd5db40 msgr2=0x55a1cdd67430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 -- 172.21.15.105:0/944962897 shutdown_connections 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 -- 172.21.15.105:0/944962897 wait complete. 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 -- start start 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.741+0000 7f7f0662b740 1 Processor -- start 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd70660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd70ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f0662b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdddc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f0662b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1a7fd0 con 0x55a1cdd625d0 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f0662b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1a7900 con 0x55a1cdd62fc0 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f0662b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc1a4b80 con 0x55a1cdd68130 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f0662b740 10 client.0 _ll_register_callbacks cb 0x7f7ef402b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f016fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd70660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f016fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd70660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38076/0 (socket says 172.21.15.105:38076) 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f016fe640 1 -- 172.21.15.105:0/3734101173 learned_addr learned my addr 172.21.15.105:0/3734101173 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f01eff640 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdddc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f00efd640 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd70ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f016fe640 1 -- 172.21.15.105:0/3734101173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 msgr2=0x55a1cdddc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f016fe640 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdddc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f016fe640 1 -- 172.21.15.105:0/3734101173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 msgr2=0x55a1cdd70ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f016fe640 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd70ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.742+0000 7f7f016fe640 1 -- 172.21.15.105:0/3734101173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1cdd73c20 con 0x55a1cdd625d0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.743+0000 7f7f016fe640 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd70660 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8000c00 tx=0x7f7ef8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb270ca314f59d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.743+0000 7f7ee4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.743+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/3734101173 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8024050 con 0x55a1cdd625d0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.743+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/3734101173 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8293+0+0 (secure 0 0 0) 0x7f7ef8004a60 con 0x55a1cdd625d0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.743+0000 7f7f0662b740 1 -- 172.21.15.105:0/3734101173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef80054d0 con 0x55a1cdd625d0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.743+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/3734101173 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7ef8004c60 con 0x55a1cdd625d0 2024-03-26T18:02:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.743+0000 7f7f0662b740 10 client.23925 fetch_fsmap learned FSMap version 43 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.743+0000 7f7f0662b740 1 -- 172.21.15.105:0/3734101173 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a1cdddd980 con 0x55a1cdd625d0 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/3734101173 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ef80054d0 con 0x55a1cdd625d0 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/3734101173 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7ef8020620 con 0x55a1cdd625d0 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7f0662b740 10 client.23925 fetch_fsmap finished waiting for FSMap version 43 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7f0662b740 10 client.23925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7f0662b740 20 client.23925 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7f0662b740 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a1cdd780e0 0x55a1cdde0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7f0662b740 4 client.23925 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7f0662b740 1 -- 172.21.15.105:0/3734101173 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1cdcce940 con 0x55a1cdd780e0 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7f00efd640 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a1cdd780e0 0x55a1cdde0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7ee4ff9640 10 client.23925 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.744+0000 7f7f00efd640 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a1cdd780e0 0x55a1cdde0410 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf6bf3ecf3458c6 server_cookie=7f5fdab6dea2a8cc in_seq=0 out_seq=0 2024-03-26T18:02:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.894+0000 7f7ee4ff9640 1 -- 172.21.15.105:0/3734101173 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1cdcce940 con 0x55a1cdd780e0 2024-03-26T18:02:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.894+0000 7f7ee4ff9640 10 client.23925 handle_command_reply: tid=0 2024-03-26T18:02:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.894+0000 7f7f0662b740 1 client.23925 shutdown 2024-03-26T18:02:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.894+0000 7f7f0662b740 1 -- 172.21.15.105:0/3734101173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 msgr2=0x55a1cdd70660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.894+0000 7f7f0662b740 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd70660 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8000c00 tx=0x7f7ef8009bf0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7f0662b740 1 -- 172.21.15.105:0/3734101173 shutdown_connections 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7f0662b740 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55a1cdd780e0 0x55a1cdde0410 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7f0662b740 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a1cdd68130 0x55a1cdddc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7f0662b740 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1cdd62fc0 0x55a1cdd70ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7f0662b740 1 --2- 172.21.15.105:0/3734101173 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a1cdd625d0 0x55a1cdd70660 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7f0662b740 1 -- 172.21.15.105:0/3734101173 >> 172.21.15.105:0/3734101173 conn(0x55a1cdd5db40 msgr2=0x55a1cdd60da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7ee4ff9640 0 client.23925 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7f0662b740 1 -- 172.21.15.105:0/3734101173 shutdown_connections 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.895+0000 7f7f0662b740 1 -- 172.21.15.105:0/3734101173 wait complete. 2024-03-26T18:02:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.896+0000 7f7f0662b740 20 client.23925 trim_cache size 0 max 16384 2024-03-26T18:02:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:02:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:02:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:02:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.896+0000 7f7f03962640 1 -- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ee80a2fc0 msgr2=0x7f7ee80a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.896+0000 7f7f03962640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ee80a2fc0 0x7f7ee80a5480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7ef402a000 tx=0x7f7ef4027000 comp rx=0 tx=0).stop 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.896+0000 7f7f03962640 1 -- 172.21.15.105:0/433608072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7efc102b00 msgr2=0x7f7efc19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.896+0000 7f7f03962640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7efc102b00 0x7f7efc19a230 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7eec02c170 tx=0x7f7eec005e60 comp rx=0 tx=0).stop 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.897+0000 7f7f03962640 1 -- 172.21.15.105:0/433608072 shutdown_connections 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.897+0000 7f7f03962640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ee80a2fc0 0x7f7ee80a5480 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.897+0000 7f7f03962640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7efc1034c0 0x7f7efc1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.897+0000 7f7f03962640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7efc102b00 0x7f7efc19a230 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.897+0000 7f7f03962640 1 --2- 172.21.15.105:0/433608072 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7efc101910 0x7f7efc199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.897+0000 7f7f03962640 1 -- 172.21.15.105:0/433608072 >> 172.21.15.105:0/433608072 conn(0x7f7efc0fce80 msgr2=0x7f7efc073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.897+0000 7f7f03962640 1 -- 172.21.15.105:0/433608072 shutdown_connections 2024-03-26T18:02:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:06.897+0000 7f7f03962640 1 -- 172.21.15.105:0/433608072 wait complete. 2024-03-26T18:02:07.313 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:07 smithi135 ceph-mon[35636]: pgmap v1065: 97 pgs: 97 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-03-26T18:02:07.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:07 smithi144 ceph-mon[36095]: pgmap v1065: 97 pgs: 97 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-03-26T18:02:07.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:07 smithi105 ceph-mon[32280]: pgmap v1065: 97 pgs: 97 active+clean; 371 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 3.0 MiB/s wr, 249 op/s 2024-03-26T18:02:07.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 84979 7.01 MB/sec execute 530 sec latency 217.457 ms 2024-03-26T18:02:07.654 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:02:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setuser ceph since I am not root 2024-03-26T18:02:07.654 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:02:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setgroup ceph since I am not root 2024-03-26T18:02:07.654 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:02:07 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/3249445194 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564ec8bba40 con 0x5564ec88e800 2024-03-26T18:02:07.654 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:02:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T18:02:08.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:08 smithi144 ceph-mon[36095]: daemon mds.e restarted 2024-03-26T18:02:08.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:08 smithi105 ceph-mon[32280]: daemon mds.e restarted 2024-03-26T18:02:08.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:08 smithi135 ceph-mon[35636]: daemon mds.e restarted 2024-03-26T18:02:08.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 85128 7.02 MB/sec execute 531 sec latency 124.911 ms 2024-03-26T18:02:09.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:02:09.033 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:02:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/4024525150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f18101a30 msgr2=0x7f9f18101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/4024525150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f18101a30 0x7f9f18101eb0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009900 tx=0x7f9f0002bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/4024525150 shutdown_connections 2024-03-26T18:02:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/4024525150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f181023f0 0x7f9f18108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/4024525150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f18101a30 0x7f9f18101eb0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/4024525150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18100840 0x7f9f18100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/4024525150 >> 172.21.15.105:0/4024525150 conn(0x7f9f180fbdf0 msgr2=0x7f9f180fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/4024525150 shutdown_connections 2024-03-26T18:02:09.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.193+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/4024525150 wait complete. 2024-03-26T18:02:09.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.194+0000 7f9f1dbf3640 1 Processor -- start 2024-03-26T18:02:09.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.194+0000 7f9f1dbf3640 1 -- start start 2024-03-26T18:02:09.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.194+0000 7f9f1dbf3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18100840 0x7f9f18197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.194+0000 7f9f1dbf3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18101a30 0x7f9f181980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.194+0000 7f9f1dbf3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181023f0 0x7f9f1819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.194+0000 7f9f1dbf3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810c300 con 0x7f9f181023f0 2024-03-26T18:02:09.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.194+0000 7f9f1dbf3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810c180 con 0x7f9f18100840 2024-03-26T18:02:09.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.194+0000 7f9f1dbf3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810c480 con 0x7f9f18101a30 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18100840 0x7f9f18197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181023f0 0x7f9f1819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18100840 0x7f9f18197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41540/0 (socket says 172.21.15.105:41540) 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181023f0 0x7f9f1819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38094/0 (socket says 172.21.15.105:38094) 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18101a30 0x7f9f181980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18101a30 0x7f9f181980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47410/0 (socket says 172.21.15.105:47410) 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 -- 172.21.15.105:0/721781895 learned_addr learned my addr 172.21.15.105:0/721781895 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 -- 172.21.15.105:0/721781895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18101a30 msgr2=0x7f9f181980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18101a30 0x7f9f181980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 -- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181023f0 msgr2=0x7f9f1819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181023f0 0x7f9f1819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 -- 172.21.15.105:0/721781895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1819f750 con 0x7f9f18100840 2024-03-26T18:02:09.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f17fff640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181023f0 0x7f9f1819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18101a30 0x7f9f181980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f177fe640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18100840 0x7f9f18197b80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800d770 tx=0x7f9f0800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.195+0000 7f9f157fa640 1 -- 172.21.15.105:0/721781895 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08017070 con 0x7f9f18100840 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.196+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/721781895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1819fa40 con 0x7f9f18100840 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.196+0000 7f9f157fa640 1 -- 172.21.15.105:0/721781895 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f08009c30 con 0x7f9f18100840 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.196+0000 7f9f157fa640 1 -- 172.21.15.105:0/721781895 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08003d00 con 0x7f9f18100840 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.196+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/721781895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1819ff60 con 0x7f9f18100840 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f207db740 1 Processor -- start 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f207db740 1 -- start start 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f157fa640 1 -- 172.21.15.105:0/721781895 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9f08003ea0 con 0x7f9f18100840 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f207db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f726f8f0 0x55e5f726fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f207db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7274de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f207db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 0x55e5f72778e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f207db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e5f71d4980 con 0x55e5f72702e0 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f207db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e5f717cbe0 con 0x55e5f7275450 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.198+0000 7f9f207db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e5f71d0930 con 0x55e5f726f8f0 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f726f8f0 0x55e5f726fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f177fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7274de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f726f8f0 0x55e5f726fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47414/0 (socket says 172.21.15.105:47414) 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 -- 172.21.15.105:0/2890921452 learned_addr learned my addr 172.21.15.105:0/2890921452 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/2890921452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 0x55e5f72778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 -- 172.21.15.105:0/2890921452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 msgr2=0x55e5f72778e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 --2- 172.21.15.105:0/2890921452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 0x55e5f72778e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 -- 172.21.15.105:0/2890921452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 msgr2=0x55e5f7274de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 --2- 172.21.15.105:0/2890921452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7274de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 -- 172.21.15.105:0/2890921452 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f6fb27d0 con 0x55e5f726f8f0 2024-03-26T18:02:09.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f157fa640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f040a2f20 0x7f9f040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f17fff640 1 --2- 172.21.15.105:0/2890921452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f726f8f0 0x55e5f726fd10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9f100117e0 tx=0x7f9f10011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae690e6e2ced444 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f0dffb640 1 -- 172.21.15.105:0/2890921452 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10011310 con 0x55e5f726f8f0 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f177fe640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f040a2f20 0x7f9f040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f0dffb640 1 -- 172.21.15.105:0/2890921452 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f10003e20 con 0x55e5f726f8f0 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f0dffb640 1 -- 172.21.15.105:0/2890921452 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100044f0 con 0x55e5f726f8f0 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.199+0000 7f9f157fa640 1 -- 172.21.15.105:0/721781895 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f9f080cf070 con 0x7f9f18100840 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 -- 172.21.15.105:0/2890921452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f726f8f0 msgr2=0x55e5f726fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f177fe640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f040a2f20 0x7f9f040a53e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9f080ccea0 tx=0x7f9f080d2b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 --2- 172.21.15.105:0/2890921452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f726f8f0 0x55e5f726fd10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9f100117e0 tx=0x7f9f10011ca0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 -- 172.21.15.105:0/2890921452 shutdown_connections 2024-03-26T18:02:09.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 --2- 172.21.15.105:0/2890921452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 0x55e5f72778e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 --2- 172.21.15.105:0/2890921452 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7274de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 --2- 172.21.15.105:0/2890921452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f726f8f0 0x55e5f726fd10 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 -- 172.21.15.105:0/2890921452 >> 172.21.15.105:0/2890921452 conn(0x55e5f726ae60 msgr2=0x55e5f726b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 -- 172.21.15.105:0/2890921452 shutdown_connections 2024-03-26T18:02:09.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.200+0000 7f9f207db740 1 -- 172.21.15.105:0/2890921452 wait complete. 2024-03-26T18:02:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 1 -- start start 2024-03-26T18:02:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 1 Processor -- start 2024-03-26T18:02:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f726f8f0 0x55e5f72720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f72702e0 0x55e5f7272630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 0x55e5f72735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55e5f727c810 con 0x55e5f726f8f0 2024-03-26T18:02:09.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e5f7190160 con 0x55e5f7275450 2024-03-26T18:02:09.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e5f727c990 con 0x55e5f72702e0 2024-03-26T18:02:09.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.201+0000 7f9f207db740 10 client.0 _ll_register_callbacks cb 0x55e5f71cc720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:09.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f726f8f0 0x55e5f72720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f72702e0 0x55e5f7272630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f726f8f0 0x55e5f72720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38114/0 (socket says 172.21.15.105:38114) 2024-03-26T18:02:09.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f17fff640 1 -- 172.21.15.105:0/1862587178 learned_addr learned my addr 172.21.15.105:0/1862587178 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 0x55e5f72735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f0ffff640 1 -- 172.21.15.105:0/1862587178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 msgr2=0x55e5f72735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 0x55e5f72735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f0ffff640 1 -- 172.21.15.105:0/1862587178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f726f8f0 msgr2=0x55e5f72720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f726f8f0 0x55e5f72720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f0ffff640 1 -- 172.21.15.105:0/1862587178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f6ee83f0 con 0x55e5f72702e0 2024-03-26T18:02:09.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f726f8f0 0x55e5f72720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:09.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f72702e0 0x55e5f7272630 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f00005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158b4d3002c437a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9ef6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:09.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/1862587178 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00005d50 con 0x55e5f72702e0 2024-03-26T18:02:09.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/1862587178 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f9f000046c0 con 0x55e5f72702e0 2024-03-26T18:02:09.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.202+0000 7f9f207db740 1 -- 172.21.15.105:0/1862587178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f10003e20 con 0x55e5f72702e0 2024-03-26T18:02:09.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/1862587178 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9f0002cc90 con 0x55e5f72702e0 2024-03-26T18:02:09.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f207db740 10 client.33607 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:09.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f207db740 1 -- 172.21.15.105:0/1862587178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e5f6ea80f0 con 0x55e5f72702e0 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/1862587178 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f10003e20 con 0x55e5f72702e0 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/1862587178 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f9f00003970 con 0x55e5f72702e0 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f207db740 10 client.33607 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f207db740 10 client.33607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f207db740 20 client.33607 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f207db740 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e5f728e0c0 0x55e5f72904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f207db740 4 client.33607 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f207db740 1 -- 172.21.15.105:0/1862587178 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f71e1ec0 con 0x55e5f728e0c0 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.203+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e5f728e0c0 0x55e5f72904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.204+0000 7f9f17fff640 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e5f728e0c0 0x55e5f72904f0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda9fac92e7b1f05 server_cookie=aa0e344f6ce2891b in_seq=0 out_seq=0 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.204+0000 7f9ef6ffd640 10 client.33607 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.205+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/1862587178 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e5f71e1ec0 con 0x55e5f728e0c0 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.205+0000 7f9ef6ffd640 10 client.33607 handle_command_reply: tid=0 2024-03-26T18:02:09.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 client.33607 shutdown 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 -- 172.21.15.105:0/1862587178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f72702e0 msgr2=0x55e5f7272630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f72702e0 0x55e5f7272630 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f00005270 comp rx=0 tx=0).stop 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 -- 172.21.15.105:0/1862587178 shutdown_connections 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e5f728e0c0 0x55e5f72904f0 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9ef6ffd640 0 client.33607 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f7275450 0x55e5f72735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f72702e0 0x55e5f7272630 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 --2- 172.21.15.105:0/1862587178 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f726f8f0 0x55e5f72720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 -- 172.21.15.105:0/1862587178 >> 172.21.15.105:0/1862587178 conn(0x55e5f726ae60 msgr2=0x55e5f726a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 -- 172.21.15.105:0/1862587178 shutdown_connections 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 1 -- 172.21.15.105:0/1862587178 wait complete. 2024-03-26T18:02:09.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.206+0000 7f9f207db740 20 client.33607 trim_cache size 0 max 16384 2024-03-26T18:02:09.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f207db740 1 Processor -- start 2024-03-26T18:02:09.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f207db740 1 -- start start 2024-03-26T18:02:09.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f207db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f726b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f207db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f72697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f207db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f7269d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f207db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a6bd0 con 0x55e5f72702e0 2024-03-26T18:02:09.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f207db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a8710 con 0x55e5f726f8f0 2024-03-26T18:02:09.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f207db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a72a0 con 0x55e5f7275450 2024-03-26T18:02:09.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f72697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f72697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38118/0 (socket says 172.21.15.105:38118) 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f0ffff640 1 -- 172.21.15.105:0/987392476 learned_addr learned my addr 172.21.15.105:0/987392476 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f726b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f17fff640 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f7269d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f0ffff640 1 -- 172.21.15.105:0/987392476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 msgr2=0x55e5f7269d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f7269d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f0ffff640 1 -- 172.21.15.105:0/987392476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 msgr2=0x55e5f726b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f726b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f0ffff640 1 -- 172.21.15.105:0/987392476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f7270b40 con 0x55e5f72702e0 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.236+0000 7f9f17fff640 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f7269d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.237+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f72697f0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f9f0002c170 tx=0x7f9f00009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6befef5efada856c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.237+0000 7f9f0d7fa640 1 -- 172.21.15.105:0/987392476 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00059030 con 0x55e5f72702e0 2024-03-26T18:02:09.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.237+0000 7f9f0d7fa640 1 -- 172.21.15.105:0/987392476 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f0002cc90 con 0x55e5f72702e0 2024-03-26T18:02:09.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.237+0000 7f9f0d7fa640 1 -- 172.21.15.105:0/987392476 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00034ac0 con 0x55e5f72702e0 2024-03-26T18:02:09.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 -- 172.21.15.105:0/987392476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 msgr2=0x55e5f72697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f72697f0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f9f0002c170 tx=0x7f9f00009610 comp rx=0 tx=0).stop 2024-03-26T18:02:09.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 -- 172.21.15.105:0/987392476 shutdown_connections 2024-03-26T18:02:09.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f7269d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f72697f0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 --2- 172.21.15.105:0/987392476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f726b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 -- 172.21.15.105:0/987392476 >> 172.21.15.105:0/987392476 conn(0x55e5f726ae60 msgr2=0x55e5f7266100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 -- 172.21.15.105:0/987392476 shutdown_connections 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 -- 172.21.15.105:0/987392476 wait complete. 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.238+0000 7f9f207db740 1 -- start start 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f207db740 1 Processor -- start 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f207db740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f7288f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f207db740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7289490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f207db740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f728a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f207db740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a8040 con 0x55e5f72702e0 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f207db740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a7970 con 0x55e5f726f8f0 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f207db740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9f181a4bb0 con 0x55e5f7275450 2024-03-26T18:02:09.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f207db740 10 client.0 _ll_register_callbacks cb 0x7f9f00009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7289490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7289490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38128/0 (socket says 172.21.15.105:38128) 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f0ffff640 1 -- 172.21.15.105:0/3065102725 learned_addr learned my addr 172.21.15.105:0/3065102725 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f17fff640 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f7288f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f17fff640 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f728a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f0ffff640 1 -- 172.21.15.105:0/3065102725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 msgr2=0x55e5f728a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f728a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f0ffff640 1 -- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 msgr2=0x55e5f7288f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f7288f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f0ffff640 1 -- 172.21.15.105:0/3065102725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f728aa90 con 0x55e5f72702e0 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.239+0000 7f9f17fff640 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f7288f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.240+0000 7f9f0ffff640 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7289490 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f9f00003f80 tx=0x7f9f000038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c1e5e9c0b44339 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.240+0000 7f9ef6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:09.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.240+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/3065102725 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f00062020 con 0x55e5f72702e0 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.240+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/3065102725 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f9f000347c0 con 0x55e5f72702e0 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.240+0000 7f9f207db740 1 -- 172.21.15.105:0/3065102725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f0002cc90 con 0x55e5f72702e0 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.240+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/3065102725 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9f00034a00 con 0x55e5f72702e0 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.240+0000 7f9f207db740 10 client.23949 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.240+0000 7f9f207db740 1 -- 172.21.15.105:0/3065102725 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e5f72f4b80 con 0x55e5f72702e0 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/3065102725 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f0002cc90 con 0x55e5f72702e0 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/3065102725 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f9f00055c20 con 0x55e5f72702e0 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9f207db740 10 client.23949 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9f207db740 10 client.23949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9f207db740 20 client.23949 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:09.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9f207db740 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e5f728e0c0 0x55e5f72f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9f207db740 4 client.23949 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:02:09.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9f207db740 1 -- 172.21.15.105:0/3065102725 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5f71dc210 con 0x55e5f728e0c0 2024-03-26T18:02:09.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.241+0000 7f9f17fff640 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e5f728e0c0 0x55e5f72f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.242+0000 7f9f17fff640 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e5f728e0c0 0x55e5f72f74d0 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af38151459127e4 server_cookie=a961a1f3ec1b6777 in_seq=0 out_seq=0 2024-03-26T18:02:09.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.242+0000 7f9ef6ffd640 10 client.23949 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:09.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.243+0000 7f9ef6ffd640 1 -- 172.21.15.105:0/3065102725 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e5f71dc210 con 0x55e5f728e0c0 2024-03-26T18:02:09.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.243+0000 7f9ef6ffd640 10 client.23949 handle_command_reply: tid=0 2024-03-26T18:02:09.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.243+0000 7f9f207db740 1 client.23949 shutdown 2024-03-26T18:02:09.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.243+0000 7f9f207db740 1 -- 172.21.15.105:0/3065102725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 msgr2=0x55e5f7289490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.243+0000 7f9f207db740 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7289490 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f9f00003f80 tx=0x7f9f000038a0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.243+0000 7f9f207db740 1 -- 172.21.15.105:0/3065102725 shutdown_connections 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f207db740 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55e5f728e0c0 0x55e5f72f74d0 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9ef6ffd640 0 client.23949 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f207db740 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e5f7275450 0x55e5f728a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f207db740 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55e5f72702e0 0x55e5f7289490 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f207db740 1 --2- 172.21.15.105:0/3065102725 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e5f726f8f0 0x55e5f7288f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f207db740 1 -- 172.21.15.105:0/3065102725 >> 172.21.15.105:0/3065102725 conn(0x55e5f726ae60 msgr2=0x55e5f7266080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f207db740 1 -- 172.21.15.105:0/3065102725 shutdown_connections 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f207db740 1 -- 172.21.15.105:0/3065102725 wait complete. 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f207db740 20 client.23949 trim_cache size 0 max 16384 2024-03-26T18:02:09.229 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:02:09.230 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T18:02:09.230 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T18:02:09.230 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:02:09.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f040a2f20 msgr2=0x7f9f040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f040a2f20 0x7f9f040a53e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9f080ccea0 tx=0x7f9f080d2b50 comp rx=0 tx=0).stop 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/721781895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18100840 msgr2=0x7f9f18197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.244+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18100840 0x7f9f18197b80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800d770 tx=0x7f9f0800dc30 comp rx=0 tx=0).stop 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.245+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/721781895 shutdown_connections 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.245+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9f040a2f20 0x7f9f040a53e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.245+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9f181023f0 0x7f9f1819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.245+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f18101a30 0x7f9f181980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.245+0000 7f9f1dbf3640 1 --2- 172.21.15.105:0/721781895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f18100840 0x7f9f18197b80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.245+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/721781895 >> 172.21.15.105:0/721781895 conn(0x7f9f180fbdf0 msgr2=0x7f9f180fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.245+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/721781895 shutdown_connections 2024-03-26T18:02:09.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.245+0000 7f9f1dbf3640 1 -- 172.21.15.105:0/721781895 wait complete. 2024-03-26T18:02:09.238 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:09 smithi105 ceph-mon[32280]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T18:02:09.238 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:09 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/1455536711,v1:172.21.15.135:6835/1455536711] up:boot 2024-03-26T18:02:09.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:09 smithi105 ceph-mon[32280]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T18:02:09.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:09 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:02:09.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:09 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:09.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:09 smithi105 ceph-mon[32280]: pgmap v1067: 97 pgs: 97 active+clean; 363 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.8 MiB/s wr, 236 op/s 2024-03-26T18:02:09.240 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T18:02:09.240 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:02:09.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:09 smithi135 ceph-mon[35636]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T18:02:09.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:09 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/1455536711,v1:172.21.15.135:6835/1455536711] up:boot 2024-03-26T18:02:09.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:09 smithi135 ceph-mon[35636]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T18:02:09.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:09 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:02:09.304 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:09 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:09.305 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:09 smithi135 ceph-mon[35636]: pgmap v1067: 97 pgs: 97 active+clean; 363 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.8 MiB/s wr, 236 op/s 2024-03-26T18:02:09.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:09 smithi144 ceph-mon[36095]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T18:02:09.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:09 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/1455536711,v1:172.21.15.135:6835/1455536711] up:boot 2024-03-26T18:02:09.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:09 smithi144 ceph-mon[36095]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-26T18:02:09.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:09 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:02:09.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:09 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:02:09.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:09 smithi144 ceph-mon[36095]: pgmap v1067: 97 pgs: 97 active+clean; 363 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 2.8 MiB/s wr, 236 op/s 2024-03-26T18:02:09.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 -- 172.21.15.105:0/1395525595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41019d0 msgr2=0x7f6fa4101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/1395525595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41019d0 0x7f6fa4101e50 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009900 tx=0x7f6f9802bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:09.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 -- 172.21.15.105:0/1395525595 shutdown_connections 2024-03-26T18:02:09.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/1395525595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa4102390 0x7f6fa4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/1395525595 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41019d0 0x7f6fa4101e50 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/1395525595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa41007e0 0x7f6fa4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 -- 172.21.15.105:0/1395525595 >> 172.21.15.105:0/1395525595 conn(0x7f6fa40fbd50 msgr2=0x7f6fa40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 -- 172.21.15.105:0/1395525595 shutdown_connections 2024-03-26T18:02:09.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.402+0000 7f6faa9ab640 1 -- 172.21.15.105:0/1395525595 wait complete. 2024-03-26T18:02:09.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6faa9ab640 1 Processor -- start 2024-03-26T18:02:09.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6faa9ab640 1 -- start start 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6faa9ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41007e0 0x7f6fa4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6faa9ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa41019d0 0x7f6fa41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6faa9ab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa4102390 0x7f6fa419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6faa9ab640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410c310 con 0x7f6fa41007e0 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6faa9ab640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410c190 con 0x7f6fa4102390 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6faa9ab640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410c490 con 0x7f6fa41019d0 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa41019d0 0x7f6fa41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41007e0 0x7f6fa4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6fa37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa41019d0 0x7f6fa41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47466/0 (socket says 172.21.15.105:47466) 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.403+0000 7f6fa37fe640 1 -- 172.21.15.105:0/3236924910 learned_addr learned my addr 172.21.15.105:0/3236924910 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa8f48640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa4102390 0x7f6fa419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa37fe640 1 -- 172.21.15.105:0/3236924910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa4102390 msgr2=0x7f6fa419afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa37fe640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa4102390 0x7f6fa419afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa37fe640 1 -- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41007e0 msgr2=0x7f6fa4197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa37fe640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41007e0 0x7f6fa4197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa37fe640 1 -- 172.21.15.105:0/3236924910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa419b590 con 0x7f6fa41019d0 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa3fff640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41007e0 0x7f6fa4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:09.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa8f48640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa4102390 0x7f6fa419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa37fe640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa41019d0 0x7f6fa41980d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f98006010 tx=0x7f6f98005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa17fa640 1 -- 172.21.15.105:0/3236924910 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98054020 con 0x7f6fa41019d0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa17fa640 1 -- 172.21.15.105:0/3236924910 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f980043d0 con 0x7f6fa41019d0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6fa17fa640 1 -- 172.21.15.105:0/3236924910 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98003b80 con 0x7f6fa41019d0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa419b820 con 0x7f6fa41019d0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.404+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa419bd00 con 0x7f6fa41019d0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.406+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa419bf60 con 0x7f6fa41019d0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.407+0000 7f6fa17fa640 1 -- 172.21.15.105:0/3236924910 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6f98057020 con 0x7f6fa41019d0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.407+0000 7f6fa17fa640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f8c0a2fa0 0x7f6f8c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.408+0000 7f6fa3fff640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f8c0a2fa0 0x7f6f8c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.408+0000 7f6fa17fa640 1 -- 172.21.15.105:0/3236924910 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6f981026a0 con 0x7f6fa41019d0 2024-03-26T18:02:09.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.408+0000 7f6fa3fff640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f8c0a2fa0 0x7f6f8c0a5460 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4101830 tx=0x7f6f94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.413+0000 7f6fa17fa640 1 -- 172.21.15.105:0/3236924910 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6f9805e020 con 0x7f6fa41019d0 2024-03-26T18:02:09.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 85311 7.03 MB/sec execute 532 sec latency 137.651 ms 2024-03-26T18:02:09.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.754+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fa419c160 con 0x7f6fa41019d0 2024-03-26T18:02:09.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.755+0000 7f6fa17fa640 1 -- 172.21.15.105:0/3236924910 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11941 (secure 0 0 0) 0x7f6f980cf490 con 0x7f6fa41019d0 2024-03-26T18:02:09.736 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:02:09.736 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:02:08.013819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_33599":{"gid":33599,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1455536711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1455536711},{"type":"v1","addr":"172.21.15.135:6835","nonce":1455536711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T18:02:09.736 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 45 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f8c0a2fa0 msgr2=0x7f6f8c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f8c0a2fa0 0x7f6f8c0a5460 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4101830 tx=0x7f6f94005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa41019d0 msgr2=0x7f6fa41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa41019d0 0x7f6fa41980d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f98006010 tx=0x7f6f98005e40 comp rx=0 tx=0).stop 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 shutdown_connections 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6f8c0a2fa0 0x7f6f8c0a5460 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa4102390 0x7f6fa419afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa41019d0 0x7f6fa41980d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 --2- 172.21.15.105:0/3236924910 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fa41007e0 0x7f6fa4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 >> 172.21.15.105:0/3236924910 conn(0x7f6fa40fbd50 msgr2=0x7f6fa4068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 shutdown_connections 2024-03-26T18:02:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.758+0000 7f6faa9ab640 1 -- 172.21.15.105:0/3236924910 wait complete. 2024-03-26T18:02:09.753 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 -- 172.21.15.105:0/2543536393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500102ac0 msgr2=0x7f4500102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 --2- 172.21.15.105:0/2543536393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500102ac0 0x7f4500102f40 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009900 tx=0x7f44f002bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 -- 172.21.15.105:0/2543536393 shutdown_connections 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 --2- 172.21.15.105:0/2543536393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500103480 0x7f4500109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 --2- 172.21.15.105:0/2543536393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500102ac0 0x7f4500102f40 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 --2- 172.21.15.105:0/2543536393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001018d0 0x7f4500101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 -- 172.21.15.105:0/2543536393 >> 172.21.15.105:0/2543536393 conn(0x7f45000fce60 msgr2=0x7f45000ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 -- 172.21.15.105:0/2543536393 shutdown_connections 2024-03-26T18:02:09.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.916+0000 7f45078ae640 1 -- 172.21.15.105:0/2543536393 wait complete. 2024-03-26T18:02:09.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f45078ae640 1 Processor -- start 2024-03-26T18:02:09.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f45078ae640 1 -- start start 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f45078ae640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001018d0 0x7f4500197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f45078ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102ac0 0x7f4500198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f45078ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500103480 0x7f450019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f45078ae640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f450010d400 con 0x7f4500103480 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f45078ae640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f450010d280 con 0x7f45001018d0 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f45078ae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f450010d580 con 0x7f4500102ac0 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.917+0000 7f450564a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001018d0 0x7f4500197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f4505e4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500103480 0x7f450019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f450564a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001018d0 0x7f4500197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41604/0 (socket says 172.21.15.105:41604) 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f450564a640 1 -- 172.21.15.105:0/363103294 learned_addr learned my addr 172.21.15.105:0/363103294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f4504e49640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102ac0 0x7f4500198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f450564a640 1 -- 172.21.15.105:0/363103294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102ac0 msgr2=0x7f4500198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f450564a640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102ac0 0x7f4500198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f450564a640 1 -- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500103480 msgr2=0x7f450019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f450564a640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500103480 0x7f450019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f450564a640 1 -- 172.21.15.105:0/363103294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450019f730 con 0x7f45001018d0 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f4505e4b640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500103480 0x7f450019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f450564a640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001018d0 0x7f4500197b10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00e9c0 tx=0x7f44fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f44f67fc640 1 -- 172.21.15.105:0/363103294 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00cde0 con 0x7f45001018d0 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f45078ae640 1 -- 172.21.15.105:0/363103294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4500073b50 con 0x7f45001018d0 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f45078ae640 1 -- 172.21.15.105:0/363103294 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45000740c0 con 0x7f45001018d0 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f44f67fc640 1 -- 172.21.15.105:0/363103294 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44fc0108e0 con 0x7f45001018d0 2024-03-26T18:02:09.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.918+0000 7f44f67fc640 1 -- 172.21.15.105:0/363103294 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0188e0 con 0x7f45001018d0 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.921+0000 7f44f67fc640 1 -- 172.21.15.105:0/363103294 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f44fc01f020 con 0x7f45001018d0 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.922+0000 7f450a570740 1 Processor -- start 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.922+0000 7f450a570740 1 -- start start 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.922+0000 7f450a570740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd501d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.922+0000 7f450a570740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5022730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.922+0000 7f450a570740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd5025230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.922+0000 7f450a570740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561cd4f79970 con 0x561cd501dc30 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.922+0000 7f450a570740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561cd4f21d70 con 0x561cd501d240 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.922+0000 7f450a570740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561cd4f75920 con 0x561cd5022da0 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f4505e4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd501d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5022730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5022730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38182/0 (socket says 172.21.15.105:38182) 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f4505e4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd501d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41620/0 (socket says 172.21.15.105:41620) 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 -- 172.21.15.105:0/2553578698 learned_addr learned my addr 172.21.15.105:0/2553578698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f44f67fc640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44ec0a2fa0 0x7f44ec0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f4504e49640 1 --2- 172.21.15.105:0/2553578698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd5025230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 -- 172.21.15.105:0/2553578698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 msgr2=0x561cd5025230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 --2- 172.21.15.105:0/2553578698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd5025230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 -- 172.21.15.105:0/2553578698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 msgr2=0x561cd501d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 --2- 172.21.15.105:0/2553578698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd501d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f4505e4b640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44ec0a2fa0 0x7f44ec0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 -- 172.21.15.105:0/2553578698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd4c8d3f0 con 0x561cd501dc30 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.923+0000 7f450564a640 1 --2- 172.21.15.105:0/2553578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5022730 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0ccf20 tx=0x7f44fc066760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499e22a33370be3a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.924+0000 7f44eb7fe640 1 -- 172.21.15.105:0/2553578698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc010a90 con 0x561cd501dc30 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.924+0000 7f44eb7fe640 1 -- 172.21.15.105:0/2553578698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44fc010e20 con 0x561cd501dc30 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.924+0000 7f44eb7fe640 1 -- 172.21.15.105:0/2553578698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc013190 con 0x561cd501dc30 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.924+0000 7f44f67fc640 1 -- 172.21.15.105:0/363103294 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f44fc0cf520 con 0x7f45001018d0 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 -- 172.21.15.105:0/2553578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 msgr2=0x561cd5022730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 --2- 172.21.15.105:0/2553578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5022730 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0ccf20 tx=0x7f44fc066760 comp rx=0 tx=0).stop 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f4505e4b640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44ec0a2fa0 0x7f44ec0a5460 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x561cd501e490 tx=0x7f44f8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 -- 172.21.15.105:0/2553578698 shutdown_connections 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 --2- 172.21.15.105:0/2553578698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd5025230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 --2- 172.21.15.105:0/2553578698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5022730 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 --2- 172.21.15.105:0/2553578698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd501d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 -- 172.21.15.105:0/2553578698 >> 172.21.15.105:0/2553578698 conn(0x561cd50187b0 msgr2=0x561cd5018bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 -- 172.21.15.105:0/2553578698 shutdown_connections 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 -- 172.21.15.105:0/2553578698 wait complete. 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 -- start start 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.925+0000 7f450a570740 1 Processor -- start 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450a570740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501d240 0x561cd50386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450a570740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5038be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450a570740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd5022da0 0x561cd503fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450a570740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561cd5036d00 con 0x561cd501dc30 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450a570740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561cd4f35310 con 0x561cd5022da0 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450a570740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561cd5036e80 con 0x561cd501d240 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450a570740 10 client.0 _ll_register_callbacks cb 0x561cd4f70c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:09.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f4504e49640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5038be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f4505e4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd5022da0 0x561cd503fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450564a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501d240 0x561cd50386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450564a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501d240 0x561cd50386a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47516/0 (socket says 172.21.15.105:47516) 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f4505e4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd5022da0 0x561cd503fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41632/0 (socket says 172.21.15.105:41632) 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f4504e49640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5038be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38194/0 (socket says 172.21.15.105:38194) 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450564a640 1 -- 172.21.15.105:0/3309282852 learned_addr learned my addr 172.21.15.105:0/3309282852 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450564a640 1 -- 172.21.15.105:0/3309282852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd5022da0 msgr2=0x561cd503fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450564a640 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd5022da0 0x561cd503fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450564a640 1 -- 172.21.15.105:0/3309282852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 msgr2=0x561cd5038be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450564a640 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5038be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f450564a640 1 -- 172.21.15.105:0/3309282852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd4d577d0 con 0x561cd501d240 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.926+0000 7f4505e4b640 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd5022da0 0x561cd503fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450564a640 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501d240 0x561cd50386a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f44fc09b910 tx=0x7f44fc0cf820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19081bb962884b73 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f44e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:09.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f44e8ff9640 1 -- 172.21.15.105:0/3309282852 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc013370 con 0x561cd501d240 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f44e8ff9640 1 -- 172.21.15.105:0/3309282852 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f44fc0b3380 con 0x561cd501d240 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 1 -- 172.21.15.105:0/3309282852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561cd4f86eb0 con 0x561cd501d240 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f44e8ff9640 1 -- 172.21.15.105:0/3309282852 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f44fc013560 con 0x561cd501d240 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 10 client.33631 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 1 -- 172.21.15.105:0/3309282852 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561cd5036d00 con 0x561cd501d240 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f44e8ff9640 1 -- 172.21.15.105:0/3309282852 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561cd4f86eb0 con 0x561cd501d240 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f44e8ff9640 1 -- 172.21.15.105:0/3309282852 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f44fc098280 con 0x561cd501d240 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 10 client.33631 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 10 client.33631 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 20 client.33631 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561cd5044160 0x561cd50465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 4 client.33631 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f450a570740 1 -- 172.21.15.105:0/3309282852 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cd4f81200 con 0x561cd5044160 2024-03-26T18:02:09.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.927+0000 7f4504e49640 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561cd5044160 0x561cd50465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.928+0000 7f44e8ff9640 10 client.33631 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:02:09.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.928+0000 7f4504e49640 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561cd5044160 0x561cd50465b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62ebf592d6e0a18f server_cookie=da06aab96f4db640 in_seq=0 out_seq=0 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.928+0000 7f44e8ff9640 1 -- 172.21.15.105:0/3309282852 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561cd4f81200 con 0x561cd5044160 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.928+0000 7f44e8ff9640 10 client.33631 handle_command_reply: tid=0 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.928+0000 7f450a570740 1 client.33631 shutdown 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 -- 172.21.15.105:0/3309282852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501d240 msgr2=0x561cd50386a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501d240 0x561cd50386a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f44fc09b910 tx=0x7f44fc0cf820 comp rx=0 tx=0).stop 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 -- 172.21.15.105:0/3309282852 shutdown_connections 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561cd5044160 0x561cd50465b0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd5022da0 0x561cd503fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd5038be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 --2- 172.21.15.105:0/3309282852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501d240 0x561cd50386a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f44e8ff9640 0 client.33631 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:02:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 -- 172.21.15.105:0/3309282852 >> 172.21.15.105:0/3309282852 conn(0x561cd50187b0 msgr2=0x561cd500f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 -- 172.21.15.105:0/3309282852 shutdown_connections 2024-03-26T18:02:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 1 -- 172.21.15.105:0/3309282852 wait complete. 2024-03-26T18:02:09.915 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.929+0000 7f450a570740 20 client.33631 trim_cache size 0 max 16384 2024-03-26T18:02:09.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450a570740 1 Processor -- start 2024-03-26T18:02:09.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450a570740 1 -- start start 2024-03-26T18:02:09.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450a570740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd50382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450a570740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501dc30 0x561cd503fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450a570740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd5022da0 0x561cd50387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450a570740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44e400e440 con 0x561cd5022da0 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450a570740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44e400ff80 con 0x561cd501d240 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450a570740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44e400eb10 con 0x561cd501dc30 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4504e49640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501dc30 0x561cd503fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4505e4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd5022da0 0x561cd50387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450564a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd50382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4505e4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd5022da0 0x561cd50387f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38210/0 (socket says 172.21.15.105:38210) 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f450564a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd50382b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41646/0 (socket says 172.21.15.105:41646) 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4504e49640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501dc30 0x561cd503fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47522/0 (socket says 172.21.15.105:47522) 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4505e4b640 1 -- 172.21.15.105:0/1138753252 learned_addr learned my addr 172.21.15.105:0/1138753252 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4504e49640 1 -- 172.21.15.105:0/1138753252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 msgr2=0x561cd50382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4504e49640 1 --2- 172.21.15.105:0/1138753252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd50382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4504e49640 1 -- 172.21.15.105:0/1138753252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd5022da0 msgr2=0x561cd50387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4504e49640 1 --2- 172.21.15.105:0/1138753252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd5022da0 0x561cd50387f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.947+0000 7f4504e49640 1 -- 172.21.15.105:0/1138753252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd50185c0 con 0x561cd501dc30 2024-03-26T18:02:09.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.948+0000 7f4504e49640 1 --2- 172.21.15.105:0/1138753252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501dc30 0x561cd503fbe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f44f0000c00 tx=0x7f44f0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f13071ad82306e30 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.948+0000 7f44eaffd640 1 -- 172.21.15.105:0/1138753252 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f002cae0 con 0x561cd501dc30 2024-03-26T18:02:09.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.948+0000 7f44eaffd640 1 -- 172.21.15.105:0/1138753252 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44f002cc80 con 0x561cd501dc30 2024-03-26T18:02:09.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.948+0000 7f44eaffd640 1 -- 172.21.15.105:0/1138753252 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f00041c0 con 0x561cd501dc30 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 -- 172.21.15.105:0/1138753252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501dc30 msgr2=0x561cd503fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 --2- 172.21.15.105:0/1138753252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501dc30 0x561cd503fbe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f44f0000c00 tx=0x7f44f0005fd0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 -- 172.21.15.105:0/1138753252 shutdown_connections 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 --2- 172.21.15.105:0/1138753252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd5022da0 0x561cd50387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 --2- 172.21.15.105:0/1138753252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd501dc30 0x561cd503fbe0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 --2- 172.21.15.105:0/1138753252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd50382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 -- 172.21.15.105:0/1138753252 >> 172.21.15.105:0/1138753252 conn(0x561cd50187b0 msgr2=0x561cd501f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 -- 172.21.15.105:0/1138753252 shutdown_connections 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 -- 172.21.15.105:0/1138753252 wait complete. 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 -- start start 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.949+0000 7f450a570740 1 Processor -- start 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450a570740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd503cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450a570740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd503d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450a570740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd503e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450a570740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f44e400f8b0 con 0x561cd501dc30 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450a570740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f44e400f1e0 con 0x561cd501d240 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450a570740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44e400c420 con 0x561cd5022da0 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f4504e49640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd503d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450564a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd503cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450564a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd503cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41652/0 (socket says 172.21.15.105:41652) 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450a570740 10 client.0 _ll_register_callbacks cb 0x7f44f80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450564a640 1 -- 172.21.15.105:0/2362393635 learned_addr learned my addr 172.21.15.105:0/2362393635 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f4504e49640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd503d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38218/0 (socket says 172.21.15.105:38218) 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f4505e4b640 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd503e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450564a640 1 -- 172.21.15.105:0/2362393635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 msgr2=0x561cd503e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450564a640 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd503e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450564a640 1 -- 172.21.15.105:0/2362393635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 msgr2=0x561cd503d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:09.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450564a640 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd503d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f450564a640 1 -- 172.21.15.105:0/2362393635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd503e8f0 con 0x561cd501d240 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f4504e49640 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd503d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.950+0000 7f4505e4b640 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd503e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f450564a640 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd503cdb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f44fc098070 tx=0x7f44fc09bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc1282d9167a512 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f44e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f44e8ff9640 1 -- 172.21.15.105:0/2362393635 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc013770 con 0x561cd501d240 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f450a570740 1 -- 172.21.15.105:0/2362393635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561cd4efe6a0 con 0x561cd501d240 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f44e8ff9640 1 -- 172.21.15.105:0/2362393635 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f44fc013190 con 0x561cd501d240 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f44e8ff9640 1 -- 172.21.15.105:0/2362393635 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f44fc0139b0 con 0x561cd501d240 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f450a570740 10 client.33638 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f450a570740 1 -- 172.21.15.105:0/2362393635 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561cd503e8f0 con 0x561cd501d240 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f44e8ff9640 1 -- 172.21.15.105:0/2362393635 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561cd4efe6a0 con 0x561cd501d240 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f44e8ff9640 1 -- 172.21.15.105:0/2362393635 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f44fc0cf0d0 con 0x561cd501d240 2024-03-26T18:02:09.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f450a570740 10 client.33638 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f450a570740 10 client.33638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f450a570740 20 client.33638 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.951+0000 7f450a570740 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561cd50aa4b0 0x561cd50ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 4 client.33638 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 -- 172.21.15.105:0/2362393635 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cd4f4b000 con 0x561cd50aa4b0 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f4504e49640 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561cd50aa4b0 0x561cd50ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f4504e49640 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561cd50aa4b0 0x561cd50ac8a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1acc53524ded9f1 server_cookie=2d4a157c00695b99 in_seq=0 out_seq=0 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f44e8ff9640 10 client.33638 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f44e8ff9640 1 -- 172.21.15.105:0/2362393635 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cd4f4b000 con 0x561cd50aa4b0 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f44e8ff9640 10 client.33638 handle_command_reply: tid=0 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 client.33638 shutdown 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 -- 172.21.15.105:0/2362393635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 msgr2=0x561cd503cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd503cdb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f44fc098070 tx=0x7f44fc09bd50 comp rx=0 tx=0).stop 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 -- 172.21.15.105:0/2362393635 shutdown_connections 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x561cd50aa4b0 0x561cd50ac8a0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561cd5022da0 0x561cd503e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561cd501dc30 0x561cd503d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f450a570740 1 --2- 172.21.15.105:0/2362393635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561cd501d240 0x561cd503cdb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.952+0000 7f44e8ff9640 0 client.33638 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.953+0000 7f450a570740 1 -- 172.21.15.105:0/2362393635 >> 172.21.15.105:0/2362393635 conn(0x561cd50187b0 msgr2=0x561cd50382b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.953+0000 7f450a570740 1 -- 172.21.15.105:0/2362393635 shutdown_connections 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.953+0000 7f450a570740 1 -- 172.21.15.105:0/2362393635 wait complete. 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.953+0000 7f450a570740 20 client.33638 trim_cache size 0 max 16384 2024-03-26T18:02:09.939 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:09.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 -- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44ec0a2fa0 msgr2=0x7f44ec0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44ec0a2fa0 0x7f44ec0a5460 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x561cd501e490 tx=0x7f44f8004ab0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 -- 172.21.15.105:0/363103294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001018d0 msgr2=0x7f4500197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:09.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001018d0 0x7f4500197b10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00e9c0 tx=0x7f44fc00ee80 comp rx=0 tx=0).stop 2024-03-26T18:02:09.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 -- 172.21.15.105:0/363103294 shutdown_connections 2024-03-26T18:02:09.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f44ec0a2fa0 0x7f44ec0a5460 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4500103480 0x7f450019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4500102ac0 0x7f4500198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 --2- 172.21.15.105:0/363103294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f45001018d0 0x7f4500197b10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:09.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 -- 172.21.15.105:0/363103294 >> 172.21.15.105:0/363103294 conn(0x7f45000fce60 msgr2=0x7f4500105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:09.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 -- 172.21.15.105:0/363103294 shutdown_connections 2024-03-26T18:02:09.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:09.954+0000 7f45078ae640 1 -- 172.21.15.105:0/363103294 wait complete. 2024-03-26T18:02:09.949 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T18:02:10.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 -- 172.21.15.105:0/1918004967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8107620 msgr2=0x7f84c8109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 --2- 172.21.15.105:0/1918004967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8107620 0x7f84c8109a10 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009ca0 tx=0x7f84c002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 -- 172.21.15.105:0/1918004967 shutdown_connections 2024-03-26T18:02:10.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 --2- 172.21.15.105:0/1918004967 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8107620 0x7f84c8109a10 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 --2- 172.21.15.105:0/1918004967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84c8104cf0 0x7f84c81070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 --2- 172.21.15.105:0/1918004967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c81023c0 0x7f84c81047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 -- 172.21.15.105:0/1918004967 >> 172.21.15.105:0/1918004967 conn(0x7f84c80fbd50 msgr2=0x7f84c80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 -- 172.21.15.105:0/1918004967 shutdown_connections 2024-03-26T18:02:10.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.111+0000 7f84cd20b640 1 -- 172.21.15.105:0/1918004967 wait complete. 2024-03-26T18:02:10.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.112+0000 7f84cd20b640 1 Processor -- start 2024-03-26T18:02:10.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.112+0000 7f84cd20b640 1 -- start start 2024-03-26T18:02:10.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84cd20b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c81023c0 0x7f84c8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84cd20b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8104cf0 0x7f84c8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84cd20b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84c8107620 0x7f84c819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84cd20b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84c810d3f0 con 0x7f84c8104cf0 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84cd20b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84c810d270 con 0x7f84c8107620 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84cd20b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84c810d570 con 0x7f84c81023c0 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8104cf0 0x7f84c8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c81023c0 0x7f84c8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c81023c0 0x7f84c8197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47560/0 (socket says 172.21.15.105:47560) 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8104cf0 0x7f84c8198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38240/0 (socket says 172.21.15.105:38240) 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 -- 172.21.15.105:0/3758204897 learned_addr learned my addr 172.21.15.105:0/3758204897 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c759e640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84c8107620 0x7f84c819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 -- 172.21.15.105:0/3758204897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84c8107620 msgr2=0x7f84c819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84c8107620 0x7f84c819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 -- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8104cf0 msgr2=0x7f84c8198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8104cf0 0x7f84c8198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 -- 172.21.15.105:0/3758204897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c81002b0 con 0x7f84c81023c0 2024-03-26T18:02:10.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.113+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c81023c0 0x7f84c8197ad0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f84b800e9f0 tx=0x7f84b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.114+0000 7f84affff640 1 -- 172.21.15.105:0/3758204897 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b800cd80 con 0x7f84c81023c0 2024-03-26T18:02:10.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.114+0000 7f84affff640 1 -- 172.21.15.105:0/3758204897 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84b80108e0 con 0x7f84c81023c0 2024-03-26T18:02:10.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.114+0000 7f84affff640 1 -- 172.21.15.105:0/3758204897 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8018920 con 0x7f84c81023c0 2024-03-26T18:02:10.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.114+0000 7f84cd20b640 1 -- 172.21.15.105:0/3758204897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c81005a0 con 0x7f84c81023c0 2024-03-26T18:02:10.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.114+0000 7f84cd20b640 1 -- 172.21.15.105:0/3758204897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c8100aa0 con 0x7f84c81023c0 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.117+0000 7f84cfdfc740 1 Processor -- start 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.117+0000 7f84cfdfc740 1 -- start start 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.117+0000 7f84affff640 1 -- 172.21.15.105:0/3758204897 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f84b8010a90 con 0x7f84c81023c0 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f48e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f4df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f50a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84cfdfc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563fc3ead970 con 0x563fc3f49420 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84cfdfc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563fc3e55d70 con 0x563fc3f4e590 2024-03-26T18:02:10.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84cfdfc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563fc3ea9920 con 0x563fc3f48a30 2024-03-26T18:02:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84c759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f48e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84c759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f48e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47566/0 (socket says 172.21.15.105:47566) 2024-03-26T18:02:10.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84c759e640 1 -- 172.21.15.105:0/2840710407 learned_addr learned my addr 172.21.15.105:0/2840710407 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.118+0000 7f84c659c640 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f50a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c659c640 1 -- 172.21.15.105:0/2840710407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 msgr2=0x563fc3f48e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c659c640 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f48e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c659c640 1 -- 172.21.15.105:0/2840710407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 msgr2=0x563fc3f4df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f4df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c659c640 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f4df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c659c640 1 -- 172.21.15.105:0/2840710407 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc3bc13f0 con 0x563fc3f4e590 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f4df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84affff640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84b00a2fa0 0x7f84b00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c759e640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84b00a2fa0 0x7f84b00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84affff640 1 -- 172.21.15.105:0/3758204897 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f84b80cf680 con 0x7f84c81023c0 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84c659c640 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f50a20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f84c8069500 tx=0x7f84bc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc09f9e535c61f09 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84ad7fa640 1 -- 172.21.15.105:0/2840710407 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc004110 con 0x563fc3f4e590 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84ad7fa640 1 -- 172.21.15.105:0/2840710407 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84bc0054d0 con 0x563fc3f4e590 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.119+0000 7f84ad7fa640 1 -- 172.21.15.105:0/2840710407 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc004e40 con 0x563fc3f4e590 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84c759e640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84b00a2fa0 0x7f84b00a5460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x563fc3f49c80 tx=0x7f84c00324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2840710407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 msgr2=0x563fc3f50a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f50a20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f84c8069500 tx=0x7f84bc005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2840710407 shutdown_connections 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f50a20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f4df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2840710407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f48e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2840710407 >> 172.21.15.105:0/2840710407 conn(0x563fc3f43fa0 msgr2=0x563fc3f443a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2840710407 shutdown_connections 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2840710407 wait complete. 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 -- start start 2024-03-26T18:02:10.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.121+0000 7f84cfdfc740 1 Processor -- start 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f52d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f52d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f52d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47568/0 (socket says 172.21.15.105:47568) 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c6d9d640 1 -- 172.21.15.105:0/2353681146 learned_addr learned my addr 172.21.15.105:0/2353681146 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563fc3f51540 con 0x563fc3f49420 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563fc3e69310 con 0x563fc3f4e590 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563fc3f516c0 con 0x563fc3f48a30 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c759e640 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84cfdfc740 10 client.0 _ll_register_callbacks cb 0x563fc3ea4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c6d9d640 1 -- 172.21.15.105:0/2353681146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 msgr2=0x563fc3f5a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f5a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c6d9d640 1 -- 172.21.15.105:0/2353681146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 msgr2=0x563fc3f532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.122+0000 7f84c6d9d640 1 -- 172.21.15.105:0/2353681146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc3c8b7d0 con 0x563fc3f48a30 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84bc0054d0 con 0x563fc3f48a30 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f52d70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f84b8000c00 tx=0x7f84b8010ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e1c20a2d8d59b20 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84aaff5640 10 client.33646 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84aaff5640 1 -- 172.21.15.105:0/2353681146 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8010630 con 0x563fc3f48a30 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84aaff5640 1 -- 172.21.15.105:0/2353681146 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f84b809b270 con 0x563fc3f48a30 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84aaff5640 1 -- 172.21.15.105:0/2353681146 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f84b809b4f0 con 0x563fc3f48a30 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84cfdfc740 10 client.33646 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({fsmap=45}) -- 0x563fc3f5d5b0 con 0x563fc3f48a30 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.123+0000 7f84aaff5640 1 -- 172.21.15.105:0/2353681146 <== mon.1 v2:172.21.15.144:3300/0 4 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f84b80cfb30 con 0x563fc3f48a30 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84cfdfc740 10 client.33646 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:10.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84cfdfc740 10 client.33646 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84cfdfc740 20 client.33646 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x563fc3f5eac0 0x563fc3f60f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84cfdfc740 4 client.33646 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f84bc0054d0 con 0x563fc3f5eac0 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84c659c640 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x563fc3f5eac0 0x563fc3f60f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84c659c640 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x563fc3f5eac0 0x563fc3f60f10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96d4241727ccdb8d server_cookie=6decff5c0d2f9d98 in_seq=0 out_seq=0 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.124+0000 7f84aaff5640 10 client.33646 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84aaff5640 1 -- 172.21.15.105:0/2353681146 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f84bc0054d0 con 0x563fc3f5eac0 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84aaff5640 10 client.33646 handle_command_reply: tid=0 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 client.33646 shutdown 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 msgr2=0x563fc3f52d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f52d70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f84b8000c00 tx=0x7f84b8010ea0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 shutdown_connections 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x563fc3f5eac0 0x563fc3f60f10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f4e590 0x563fc3f5a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f49420 0x563fc3f532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/2353681146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f52d70 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84aaff5640 0 client.33646 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:02:10.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.125+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 >> 172.21.15.105:0/2353681146 conn(0x563fc3f43fa0 msgr2=0x563fc3f431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.126+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 shutdown_connections 2024-03-26T18:02:10.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.126+0000 7f84cfdfc740 1 -- 172.21.15.105:0/2353681146 wait complete. 2024-03-26T18:02:10.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.126+0000 7f84cfdfc740 20 client.33646 trim_cache size 0 max 16384 2024-03-26T18:02:10.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84cfdfc740 1 Processor -- start 2024-03-26T18:02:10.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84cfdfc740 1 -- start start 2024-03-26T18:02:10.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f47910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 0x563fc3f5a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f4e590 0x563fc3f52980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84cfdfc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81a6d30 con 0x563fc3f4e590 2024-03-26T18:02:10.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f47910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84c6d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f47910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47570/0 (socket says 172.21.15.105:47570) 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84c6d9d640 1 -- 172.21.15.105:0/3008386391 learned_addr learned my addr 172.21.15.105:0/3008386391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84c659c640 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 0x563fc3f5a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84c759e640 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f4e590 0x563fc3f52980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84cfdfc740 1 -- 172.21.15.105:0/3008386391 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81a8870 con 0x563fc3f49420 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.147+0000 7f84cfdfc740 1 -- 172.21.15.105:0/3008386391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81a7400 con 0x563fc3f48a30 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84c759e640 1 -- 172.21.15.105:0/3008386391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 msgr2=0x563fc3f47910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84c759e640 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f47910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84c759e640 1 -- 172.21.15.105:0/3008386391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 msgr2=0x563fc3f5a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84c759e640 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 0x563fc3f5a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84c759e640 1 -- 172.21.15.105:0/3008386391 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc3f3f820 con 0x563fc3f4e590 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f47910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84c759e640 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f4e590 0x563fc3f52980 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f84c0032380 tx=0x7f84c0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53beed5268a32dd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84acff9640 1 -- 172.21.15.105:0/3008386391 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0039040 con 0x563fc3f4e590 2024-03-26T18:02:10.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84acff9640 1 -- 172.21.15.105:0/3008386391 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84c0004040 con 0x563fc3f4e590 2024-03-26T18:02:10.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.148+0000 7f84acff9640 1 -- 172.21.15.105:0/3008386391 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c004e6c0 con 0x563fc3f4e590 2024-03-26T18:02:10.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.149+0000 7f84cfdfc740 1 -- 172.21.15.105:0/3008386391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f4e590 msgr2=0x563fc3f52980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.149+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f4e590 0x563fc3f52980 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f84c0032380 tx=0x7f84c0041000 comp rx=0 tx=0).stop 2024-03-26T18:02:10.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.149+0000 7f84cfdfc740 1 -- 172.21.15.105:0/3008386391 shutdown_connections 2024-03-26T18:02:10.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.149+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f4e590 0x563fc3f52980 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.149+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 0x563fc3f5a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.149+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/3008386391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f48a30 0x563fc3f47910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.149+0000 7f84cfdfc740 1 -- 172.21.15.105:0/3008386391 >> 172.21.15.105:0/3008386391 conn(0x563fc3f43fa0 msgr2=0x563fc3f47200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.149+0000 7f84cfdfc740 1 -- 172.21.15.105:0/3008386391 shutdown_connections 2024-03-26T18:02:10.132 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:10 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3236924910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 -- 172.21.15.105:0/3008386391 wait complete. 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 -- start start 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 Processor -- start 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f48a30 0x563fc3f57ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 0x563fc3f58420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f4e590 0x563fc3f59430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81a81a0 con 0x563fc3f48a30 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81a7ad0 con 0x563fc3f49420 2024-03-26T18:02:10.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84c81a4d10 con 0x563fc3f4e590 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84cfdfc740 10 client.0 _ll_register_callbacks cb 0x7f84b8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84c759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f4e590 0x563fc3f59430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84c759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f4e590 0x563fc3f59430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47580/0 (socket says 172.21.15.105:47580) 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84c759e640 1 -- 172.21.15.105:0/4181257322 learned_addr learned my addr 172.21.15.105:0/4181257322 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f48a30 0x563fc3f57ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.150+0000 7f84c659c640 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 0x563fc3f58420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84c759e640 1 -- 172.21.15.105:0/4181257322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 msgr2=0x563fc3f58420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84c759e640 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 0x563fc3f58420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84c759e640 1 -- 172.21.15.105:0/4181257322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f48a30 msgr2=0x563fc3f57ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84c759e640 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f48a30 0x563fc3f57ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84c759e640 1 -- 172.21.15.105:0/4181257322 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc3f59a20 con 0x563fc3f4e590 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84c759e640 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f4e590 0x563fc3f59430 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f84c0032420 tx=0x7f84c0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63949f02d655ec9c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84aaff5640 1 -- 172.21.15.105:0/4181257322 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0039040 con 0x563fc3f4e590 2024-03-26T18:02:10.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84aaff5640 1 -- 172.21.15.105:0/4181257322 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f84c00041f0 con 0x563fc3f4e590 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84cfdfc740 1 -- 172.21.15.105:0/4181257322 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c0004040 con 0x563fc3f4e590 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84aaff5640 1 -- 172.21.15.105:0/4181257322 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f84c0004430 con 0x563fc3f4e590 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84cfdfc740 10 client.33652 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.151+0000 7f84cfdfc740 1 -- 172.21.15.105:0/4181257322 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563fc3fc5370 con 0x563fc3f4e590 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84aaff5640 1 -- 172.21.15.105:0/4181257322 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84c0004040 con 0x563fc3f4e590 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84aaff5640 1 -- 172.21.15.105:0/4181257322 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f84c004e460 con 0x563fc3f4e590 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84cfdfc740 10 client.33652 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84cfdfc740 10 client.33652 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84cfdfc740 20 client.33652 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x563fc3fc5ca0 0x563fc3fc80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84cfdfc740 4 client.33652 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84cfdfc740 1 -- 172.21.15.105:0/4181257322 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc3ebaeb0 con 0x563fc3fc5ca0 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x563fc3fc5ca0 0x563fc3fc80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84aaff5640 10 client.33652 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.152+0000 7f84c6d9d640 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x563fc3fc5ca0 0x563fc3fc80c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69378d2c574e06e3 server_cookie=b914f1e237d79e03 in_seq=0 out_seq=0 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84aaff5640 1 -- 172.21.15.105:0/4181257322 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc3ebaeb0 con 0x563fc3fc5ca0 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84aaff5640 10 client.33652 handle_command_reply: tid=0 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 client.33652 shutdown 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 -- 172.21.15.105:0/4181257322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f4e590 msgr2=0x563fc3f59430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f4e590 0x563fc3f59430 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f84c0032420 tx=0x7f84c0005fd0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 -- 172.21.15.105:0/4181257322 shutdown_connections 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x563fc3fc5ca0 0x563fc3fc80c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84aaff5640 0 client.33652 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563fc3f4e590 0x563fc3f59430 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563fc3f49420 0x563fc3f58420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 --2- 172.21.15.105:0/4181257322 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fc3f48a30 0x563fc3f57ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 -- 172.21.15.105:0/4181257322 >> 172.21.15.105:0/4181257322 conn(0x563fc3f43fa0 msgr2=0x563fc3f52980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 -- 172.21.15.105:0/4181257322 shutdown_connections 2024-03-26T18:02:10.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 1 -- 172.21.15.105:0/4181257322 wait complete. 2024-03-26T18:02:10.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.153+0000 7f84cfdfc740 20 client.33652 trim_cache size 0 max 16384 2024-03-26T18:02:10.140 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:10.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 -- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84b00a2fa0 msgr2=0x7f84b00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84b00a2fa0 0x7f84b00a5460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x563fc3f49c80 tx=0x7f84c00324f0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 -- 172.21.15.105:0/3758204897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c81023c0 msgr2=0x7f84c8197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c81023c0 0x7f84c8197ad0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f84b800e9f0 tx=0x7f84b800eeb0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 -- 172.21.15.105:0/3758204897 shutdown_connections 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f84b00a2fa0 0x7f84b00a5460 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f84c8107620 0x7f84c819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84c8104cf0 0x7f84c8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 --2- 172.21.15.105:0/3758204897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84c81023c0 0x7f84c8197ad0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 -- 172.21.15.105:0/3758204897 >> 172.21.15.105:0/3758204897 conn(0x7f84c80fbd50 msgr2=0x7f84c8102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 -- 172.21.15.105:0/3758204897 shutdown_connections 2024-03-26T18:02:10.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.154+0000 7f84cd20b640 1 -- 172.21.15.105:0/3758204897 wait complete. 2024-03-26T18:02:10.148 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T18:02:10.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 -- 172.21.15.105:0/3922410971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c0102370 msgr2=0x7f66c0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 --2- 172.21.15.105:0/3922410971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c0102370 0x7f66c0108c00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009be0 tx=0x7f66b802bef0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 -- 172.21.15.105:0/3922410971 shutdown_connections 2024-03-26T18:02:10.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 --2- 172.21.15.105:0/3922410971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c0102370 0x7f66c0108c00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 --2- 172.21.15.105:0/3922410971 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01019b0 0x7f66c0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 --2- 172.21.15.105:0/3922410971 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c01007c0 0x7f66c0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 -- 172.21.15.105:0/3922410971 >> 172.21.15.105:0/3922410971 conn(0x7f66c00fbd50 msgr2=0x7f66c00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 -- 172.21.15.105:0/3922410971 shutdown_connections 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.313+0000 7f66c7753640 1 -- 172.21.15.105:0/3922410971 wait complete. 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c7753640 1 Processor -- start 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c7753640 1 -- start start 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c7753640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01007c0 0x7f66c0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c7753640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01019b0 0x7f66c0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c7753640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0102370 0x7f66c019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c7753640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010c320 con 0x7f66c0102370 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c7753640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010c1a0 con 0x7f66c01007c0 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c54ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01007c0 0x7f66c0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c7753640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c010c4a0 con 0x7f66c01019b0 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c54ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01007c0 0x7f66c0197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41690/0 (socket says 172.21.15.105:41690) 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c5cf0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0102370 0x7f66c019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c54ef640 1 -- 172.21.15.105:0/2699327177 learned_addr learned my addr 172.21.15.105:0/2699327177 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c4cee640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01019b0 0x7f66c0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c54ef640 1 -- 172.21.15.105:0/2699327177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01019b0 msgr2=0x7f66c0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.314+0000 7f66c54ef640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01019b0 0x7f66c0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66c54ef640 1 -- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0102370 msgr2=0x7f66c019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66c54ef640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0102370 0x7f66c019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66c54ef640 1 -- 172.21.15.105:0/2699327177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c019f6f0 con 0x7f66c01007c0 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66c4cee640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01019b0 0x7f66c0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66c54ef640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01007c0 0x7f66c0197b20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e9a0 tx=0x7f66bc00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66b67fc640 1 -- 172.21.15.105:0/2699327177 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00cd80 con 0x7f66c01007c0 2024-03-26T18:02:10.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66b67fc640 1 -- 172.21.15.105:0/2699327177 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66bc0108e0 con 0x7f66c01007c0 2024-03-26T18:02:10.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66c7753640 1 -- 172.21.15.105:0/2699327177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c019f9e0 con 0x7f66c01007c0 2024-03-26T18:02:10.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66b67fc640 1 -- 172.21.15.105:0/2699327177 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc018950 con 0x7f66c01007c0 2024-03-26T18:02:10.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.315+0000 7f66c7753640 1 -- 172.21.15.105:0/2699327177 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c019ff30 con 0x7f66c01007c0 2024-03-26T18:02:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.318+0000 7f66b67fc640 1 -- 172.21.15.105:0/2699327177 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f66bc010a90 con 0x7f66c01007c0 2024-03-26T18:02:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.318+0000 7f66ca41e740 1 Processor -- start 2024-03-26T18:02:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.318+0000 7f66ca41e740 1 -- start start 2024-03-26T18:02:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.318+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 0x5638b851fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8524f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8527a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66ca41e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5638b8484970 con 0x5638b851fa30 2024-03-26T18:02:10.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66ca41e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5638b842cd70 con 0x5638b8525590 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66ca41e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638b8480920 con 0x5638b8520420 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c5cf0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 0x5638b851fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8527a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8527a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41704/0 (socket says 172.21.15.105:41704) 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c5cf0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 0x5638b851fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38290/0 (socket says 172.21.15.105:38290) 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 -- 172.21.15.105:0/3321598857 learned_addr learned my addr 172.21.15.105:0/3321598857 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c54ef640 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8524f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 -- 172.21.15.105:0/3321598857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 msgr2=0x5638b8524f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8524f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 -- 172.21.15.105:0/3321598857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 msgr2=0x5638b851fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 0x5638b851fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66b67fc640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66ac0a2fa0 0x7f66ac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 -- 172.21.15.105:0/3321598857 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b81983f0 con 0x5638b8525590 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c54ef640 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8524f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c5cf0640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66ac0a2fa0 0x7f66ac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66b67fc640 1 -- 172.21.15.105:0/2699327177 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f66bc0cf720 con 0x7f66c01007c0 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.319+0000 7f66c4cee640 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8527a20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f66b00114f0 tx=0x7f66b00119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7cb9db0f2908598 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.320+0000 7f66ab7fe640 1 -- 172.21.15.105:0/3321598857 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b001e070 con 0x5638b8525590 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.320+0000 7f66ab7fe640 1 -- 172.21.15.105:0/3321598857 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b00035f0 con 0x5638b8525590 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.320+0000 7f66ab7fe640 1 -- 172.21.15.105:0/3321598857 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000f820 con 0x5638b8525590 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 -- 172.21.15.105:0/3321598857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 msgr2=0x5638b8527a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66c5cf0640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66ac0a2fa0 0x7f66ac0a5460 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f66b8034fd0 tx=0x7f66b8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8527a20 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f66b00114f0 tx=0x7f66b00119b0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 -- 172.21.15.105:0/3321598857 shutdown_connections 2024-03-26T18:02:10.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8527a20 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8524f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 --2- 172.21.15.105:0/3321598857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 0x5638b851fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 -- 172.21.15.105:0/3321598857 >> 172.21.15.105:0/3321598857 conn(0x5638b851afa0 msgr2=0x5638b851b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 -- 172.21.15.105:0/3321598857 shutdown_connections 2024-03-26T18:02:10.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 -- 172.21.15.105:0/3321598857 wait complete. 2024-03-26T18:02:10.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 -- start start 2024-03-26T18:02:10.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.321+0000 7f66ca41e740 1 Processor -- start 2024-03-26T18:02:10.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b8529f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8520420 0x5638b852a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8531490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66ca41e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5638b8528540 con 0x5638b8520420 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66ca41e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5638b8440310 con 0x5638b8525590 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66ca41e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638b85286c0 con 0x5638b851fa30 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c4cee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8520420 0x5638b852a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66ca41e740 10 client.0 _ll_register_callbacks cb 0x5638b847bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c5cf0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8531490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c5cf0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8531490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41720/0 (socket says 172.21.15.105:41720) 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c4cee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8520420 0x5638b852a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38302/0 (socket says 172.21.15.105:38302) 2024-03-26T18:02:10.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c5cf0640 1 -- 172.21.15.105:0/599841867 learned_addr learned my addr 172.21.15.105:0/599841867 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c54ef640 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b8529f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c4cee640 1 -- 172.21.15.105:0/599841867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 msgr2=0x5638b8529f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c4cee640 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b8529f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c4cee640 1 -- 172.21.15.105:0/599841867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 msgr2=0x5638b8531490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c4cee640 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8531490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c4cee640 1 -- 172.21.15.105:0/599841867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b82627d0 con 0x5638b8520420 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.322+0000 7f66c5cf0640 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8531490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.323+0000 7f66c4cee640 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8520420 0x5638b852a450 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f66b0011eb0 tx=0x7f66b00041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e402306507696836 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.323+0000 7f66a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.323+0000 7f66a8ff9640 1 -- 172.21.15.105:0/599841867 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b001e040 con 0x5638b8520420 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.323+0000 7f66a8ff9640 1 -- 172.21.15.105:0/599841867 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f66b0016050 con 0x5638b8520420 2024-03-26T18:02:10.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.323+0000 7f66ca41e740 1 -- 172.21.15.105:0/599841867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b00035f0 con 0x5638b8520420 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.323+0000 7f66a8ff9640 1 -- 172.21.15.105:0/599841867 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f66b0005040 con 0x5638b8520420 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.323+0000 7f66ca41e740 10 client.24000 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.323+0000 7f66ca41e740 1 -- 172.21.15.105:0/599841867 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5638b8534800 con 0x5638b8520420 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66a8ff9640 1 -- 172.21.15.105:0/599841867 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b00046c0 con 0x5638b8520420 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66a8ff9640 1 -- 172.21.15.105:0/599841867 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f66b0021030 con 0x5638b8520420 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66ca41e740 10 client.24000 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66ca41e740 10 client.24000 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66ca41e740 20 client.24000 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66ca41e740 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5638b8535d90 0x5638b85381d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66ca41e740 4 client.24000 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66ca41e740 1 -- 172.21.15.105:0/599841867 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f66b00035f0 con 0x5638b8535d90 2024-03-26T18:02:10.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.324+0000 7f66c54ef640 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5638b8535d90 0x5638b85381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.325+0000 7f66a8ff9640 10 client.24000 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.325+0000 7f66c54ef640 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5638b8535d90 0x5638b85381d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48e23073502e2b99 server_cookie=c65481061276b9a9 in_seq=0 out_seq=0 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.325+0000 7f66a8ff9640 1 -- 172.21.15.105:0/599841867 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f66b00035f0 con 0x5638b8535d90 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.325+0000 7f66a8ff9640 10 client.24000 handle_command_reply: tid=0 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.325+0000 7f66ca41e740 1 client.24000 shutdown 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 -- 172.21.15.105:0/599841867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8520420 msgr2=0x5638b852a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8520420 0x5638b852a450 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f66b0011eb0 tx=0x7f66b00041e0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 -- 172.21.15.105:0/599841867 shutdown_connections 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5638b8535d90 0x5638b85381d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8531490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8520420 0x5638b852a450 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66a8ff9640 0 client.24000 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:02:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 --2- 172.21.15.105:0/599841867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b8529f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 -- 172.21.15.105:0/599841867 >> 172.21.15.105:0/599841867 conn(0x5638b851afa0 msgr2=0x5638b851a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 -- 172.21.15.105:0/599841867 shutdown_connections 2024-03-26T18:02:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 1 -- 172.21.15.105:0/599841867 wait complete. 2024-03-26T18:02:10.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.326+0000 7f66ca41e740 20 client.24000 trim_cache size 0 max 16384 2024-03-26T18:02:10.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.349+0000 7f66ca41e740 1 Processor -- start 2024-03-26T18:02:10.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66ca41e740 1 -- start start 2024-03-26T18:02:10.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 0x5638b8519930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8531450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8519e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66ca41e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a6b90 con 0x5638b851fa30 2024-03-26T18:02:10.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66ca41e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a86d0 con 0x5638b8525590 2024-03-26T18:02:10.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66ca41e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a7260 con 0x5638b8520420 2024-03-26T18:02:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66c4cee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8531450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66c4cee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8531450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47606/0 (socket says 172.21.15.105:47606) 2024-03-26T18:02:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.350+0000 7f66c4cee640 1 -- 172.21.15.105:0/2963865391 learned_addr learned my addr 172.21.15.105:0/2963865391 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.351+0000 7f66c4cee640 1 -- 172.21.15.105:0/2963865391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 msgr2=0x5638b8519e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:02:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.351+0000 7f66c4cee640 1 --2- 172.21.15.105:0/2963865391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8519e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.351+0000 7f66c4cee640 1 -- 172.21.15.105:0/2963865391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 msgr2=0x5638b8519930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.351+0000 7f66c4cee640 1 --2- 172.21.15.105:0/2963865391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 0x5638b8519930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.351+0000 7f66c4cee640 1 -- 172.21.15.105:0/2963865391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b8516820 con 0x5638b8520420 2024-03-26T18:02:10.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.351+0000 7f66c4cee640 1 --2- 172.21.15.105:0/2963865391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8531450 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f66b0004450 tx=0x7f66b00096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95c3498439408e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.352+0000 7f66aaffd640 1 -- 172.21.15.105:0/2963865391 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b001e070 con 0x5638b8520420 2024-03-26T18:02:10.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.352+0000 7f66aaffd640 1 -- 172.21.15.105:0/2963865391 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b0016020 con 0x5638b8520420 2024-03-26T18:02:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.352+0000 7f66aaffd640 1 -- 172.21.15.105:0/2963865391 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b000fb20 con 0x5638b8520420 2024-03-26T18:02:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.353+0000 7f66ca41e740 1 -- 172.21.15.105:0/2963865391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 msgr2=0x5638b8531450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.353+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2963865391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8531450 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f66b0004450 tx=0x7f66b00096c0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.355+0000 7f66ca41e740 1 -- 172.21.15.105:0/2963865391 shutdown_connections 2024-03-26T18:02:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.355+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2963865391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8525590 0x5638b8519e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.355+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2963865391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b8520420 0x5638b8531450 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.355+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2963865391 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b851fa30 0x5638b8519930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.355+0000 7f66ca41e740 1 -- 172.21.15.105:0/2963865391 >> 172.21.15.105:0/2963865391 conn(0x5638b851afa0 msgr2=0x5638b85218e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.355+0000 7f66ca41e740 1 -- 172.21.15.105:0/2963865391 shutdown_connections 2024-03-26T18:02:10.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.355+0000 7f66ca41e740 1 -- 172.21.15.105:0/2963865391 wait complete. 2024-03-26T18:02:10.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:10 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3236924910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.356+0000 7f66ca41e740 1 -- start start 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.356+0000 7f66ca41e740 1 Processor -- start 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.356+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b852e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.356+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8520420 0x5638b852ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.356+0000 7f66ca41e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8525590 0x5638b852fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66ca41e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a8000 con 0x5638b8525590 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66ca41e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a7930 con 0x5638b8520420 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66ca41e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66c01a4b70 con 0x5638b851fa30 2024-03-26T18:02:10.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c5cf0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8525590 0x5638b852fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b852e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b852e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47610/0 (socket says 172.21.15.105:47610) 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66ca41e740 10 client.0 _ll_register_callbacks cb 0x7f66b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c5cf0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8525590 0x5638b852fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38330/0 (socket says 172.21.15.105:38330) 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 -- 172.21.15.105:0/2476499434 learned_addr learned my addr 172.21.15.105:0/2476499434 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c4cee640 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8520420 0x5638b852ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 -- 172.21.15.105:0/2476499434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8520420 msgr2=0x5638b852ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8520420 0x5638b852ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 -- 172.21.15.105:0/2476499434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8525590 msgr2=0x5638b852fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8525590 0x5638b852fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 -- 172.21.15.105:0/2476499434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b8530330 con 0x5638b851fa30 2024-03-26T18:02:10.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c5cf0640 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8525590 0x5638b852fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66c54ef640 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b852e7f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f66bc09bb10 tx=0x7f66bc09b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66de147c60af119a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.357+0000 7f66a8ff9640 1 -- 172.21.15.105:0/2476499434 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc09be10 con 0x5638b851fa30 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.358+0000 7f66a8ff9640 1 -- 172.21.15.105:0/2476499434 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f66bc09a110 con 0x5638b851fa30 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.358+0000 7f66ca41e740 1 -- 172.21.15.105:0/2476499434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b0016020 con 0x5638b851fa30 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.358+0000 7f66a8ff9640 1 -- 172.21.15.105:0/2476499434 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f66bc09ff60 con 0x5638b851fa30 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.358+0000 7f66ca41e740 10 client.33670 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.358+0000 7f66ca41e740 1 -- 172.21.15.105:0/2476499434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5638b8531700 con 0x5638b851fa30 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.358+0000 7f66a8ff9640 1 -- 172.21.15.105:0/2476499434 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66b0016020 con 0x5638b851fa30 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.359+0000 7f66a8ff9640 1 -- 172.21.15.105:0/2476499434 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f66bc0cfbd0 con 0x5638b851fa30 2024-03-26T18:02:10.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.359+0000 7f66ca41e740 10 client.33670 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.359+0000 7f66ca41e740 10 client.33670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:02:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.359+0000 7f66ca41e740 20 client.33670 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.359+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5638b859c880 0x5638b859eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.359+0000 7f66ca41e740 4 client.33670 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.359+0000 7f66ca41e740 1 -- 172.21.15.105:0/2476499434 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638b8491eb0 con 0x5638b859c880 2024-03-26T18:02:10.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.359+0000 7f66c4cee640 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5638b859c880 0x5638b859eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66c4cee640 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5638b859c880 0x5638b859eca0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45c211f5f94b6143 server_cookie=190dbfbdc7180246 in_seq=0 out_seq=0 2024-03-26T18:02:10.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66a8ff9640 10 client.33670 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66a8ff9640 1 -- 172.21.15.105:0/2476499434 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638b8491eb0 con 0x5638b859c880 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66a8ff9640 10 client.33670 handle_command_reply: tid=0 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 client.33670 shutdown 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 -- 172.21.15.105:0/2476499434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 msgr2=0x5638b852e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b852e7f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f66bc09bb10 tx=0x7f66bc09b830 comp rx=0 tx=0).stop 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 -- 172.21.15.105:0/2476499434 shutdown_connections 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5638b8525590 0x5638b852fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5638b859c880 0x5638b859eca0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5638b8520420 0x5638b852ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 --2- 172.21.15.105:0/2476499434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b851fa30 0x5638b852e7f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 -- 172.21.15.105:0/2476499434 >> 172.21.15.105:0/2476499434 conn(0x5638b851afa0 msgr2=0x5638b851e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66a8ff9640 0 client.33670 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:02:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 -- 172.21.15.105:0/2476499434 shutdown_connections 2024-03-26T18:02:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.360+0000 7f66ca41e740 1 -- 172.21.15.105:0/2476499434 wait complete. 2024-03-26T18:02:10.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66ca41e740 20 client.33670 trim_cache size 0 max 16384 2024-03-26T18:02:10.346 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 -- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66ac0a2fa0 msgr2=0x7f66ac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66ac0a2fa0 0x7f66ac0a5460 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f66b8034fd0 tx=0x7f66b8009500 comp rx=0 tx=0).stop 2024-03-26T18:02:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 -- 172.21.15.105:0/2699327177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01007c0 msgr2=0x7f66c0197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01007c0 0x7f66c0197b20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e9a0 tx=0x7f66bc00ee60 comp rx=0 tx=0).stop 2024-03-26T18:02:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 -- 172.21.15.105:0/2699327177 shutdown_connections 2024-03-26T18:02:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f66ac0a2fa0 0x7f66ac0a5460 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66c0102370 0x7f66c019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66c01019b0 0x7f66c0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 --2- 172.21.15.105:0/2699327177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66c01007c0 0x7f66c0197b20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 -- 172.21.15.105:0/2699327177 >> 172.21.15.105:0/2699327177 conn(0x7f66c00fbd50 msgr2=0x7f66c00fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 -- 172.21.15.105:0/2699327177 shutdown_connections 2024-03-26T18:02:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.361+0000 7f66c7753640 1 -- 172.21.15.105:0/2699327177 wait complete. 2024-03-26T18:02:10.356 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T18:02:10.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:10 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3236924910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 -- 172.21.15.105:0/1741697513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c18102390 msgr2=0x7f9c18108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 --2- 172.21.15.105:0/1741697513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c18102390 0x7f9c18108c20 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009ca0 tx=0x7f9c1002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 -- 172.21.15.105:0/1741697513 shutdown_connections 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 --2- 172.21.15.105:0/1741697513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c18102390 0x7f9c18108c20 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 --2- 172.21.15.105:0/1741697513 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c181019d0 0x7f9c18101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 --2- 172.21.15.105:0/1741697513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c181007e0 0x7f9c18100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 -- 172.21.15.105:0/1741697513 >> 172.21.15.105:0/1741697513 conn(0x7f9c180fbd70 msgr2=0x7f9c180fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 -- 172.21.15.105:0/1741697513 shutdown_connections 2024-03-26T18:02:10.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.499+0000 7f9c200a0640 1 -- 172.21.15.105:0/1741697513 wait complete. 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c200a0640 1 Processor -- start 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c200a0640 1 -- start start 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c200a0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c181007e0 0x7f9c18197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c200a0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c181019d0 0x7f9c18198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c200a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c18102390 0x7f9c1819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c200a0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1810c310 con 0x7f9c181007e0 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c200a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1810c190 con 0x7f9c18102390 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c200a0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1810c490 con 0x7f9c181019d0 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1de3c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c181007e0 0x7f9c18197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1de3c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c181007e0 0x7f9c18197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38334/0 (socket says 172.21.15.105:38334) 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1de3c640 1 -- 172.21.15.105:0/2025122490 learned_addr learned my addr 172.21.15.105:0/2025122490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c181019d0 0x7f9c18198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c18102390 0x7f9c1819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1de3c640 1 -- 172.21.15.105:0/2025122490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c181019d0 msgr2=0x7f9c18198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c181019d0 0x7f9c18198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1de3c640 1 -- 172.21.15.105:0/2025122490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c18102390 msgr2=0x7f9c1819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c18102390 0x7f9c1819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.500+0000 7f9c1de3c640 1 -- 172.21.15.105:0/2025122490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1819f6e0 con 0x7f9c181007e0 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.501+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c181007e0 0x7f9c18197b10 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400e9c0 tx=0x7f9c1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.501+0000 7f9c0effd640 1 -- 172.21.15.105:0/2025122490 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400cd80 con 0x7f9c181007e0 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.501+0000 7f9c0effd640 1 -- 172.21.15.105:0/2025122490 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c140035f0 con 0x7f9c181007e0 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.501+0000 7f9c200a0640 1 -- 172.21.15.105:0/2025122490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1819f9d0 con 0x7f9c181007e0 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.501+0000 7f9c0effd640 1 -- 172.21.15.105:0/2025122490 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c14010690 con 0x7f9c181007e0 2024-03-26T18:02:10.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.501+0000 7f9c200a0640 1 -- 172.21.15.105:0/2025122490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1819ff20 con 0x7f9c181007e0 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.504+0000 7f9c0effd640 1 -- 172.21.15.105:0/2025122490 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f9c140037a0 con 0x7f9c181007e0 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.504+0000 7f9c22c85740 1 Processor -- start 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.504+0000 7f9c22c85740 1 -- start start 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bf1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c22c85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561b32b51970 con 0x561b32bed420 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c22c85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561b32af9d70 con 0x561b32beca30 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c22c85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b32b4d920 con 0x561b32bf2590 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c0effd640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9c040a2fa0 0x7f9c040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1de3c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bf1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1de3c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bf1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38342/0 (socket says 172.21.15.105:38342) 2024-03-26T18:02:10.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1de3c640 1 -- 172.21.15.105:0/1824201275 learned_addr learned my addr 172.21.15.105:0/1824201275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9c040a2fa0 0x7f9c040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1d63b640 1 -- 172.21.15.105:0/1824201275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 msgr2=0x561b32bece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1d63b640 1 -- 172.21.15.105:0/1824201275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 msgr2=0x561b32bf1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bf1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1d63b640 1 -- 172.21.15.105:0/1824201275 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b328653f0 con 0x561b32bf2590 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c0effd640 1 -- 172.21.15.105:0/2025122490 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f9c140cf8c0 con 0x7f9c181007e0 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.505+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bf1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.506+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf4a20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009bf0 tx=0x7f9c08011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cbd4186273f30b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.506+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9c040a2fa0 0x7f9c040a5460 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9c10006010 tx=0x7f9c100324f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.506+0000 7f9c03fff640 1 -- 172.21.15.105:0/1824201275 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08004050 con 0x561b32bf2590 2024-03-26T18:02:10.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.506+0000 7f9c03fff640 1 -- 172.21.15.105:0/1824201275 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c080054d0 con 0x561b32bf2590 2024-03-26T18:02:10.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.506+0000 7f9c03fff640 1 -- 172.21.15.105:0/1824201275 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08004cf0 con 0x561b32bf2590 2024-03-26T18:02:10.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 -- 172.21.15.105:0/1824201275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 msgr2=0x561b32bf4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf4a20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009bf0 tx=0x7f9c08011810 comp rx=0 tx=0).stop 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 -- 172.21.15.105:0/1824201275 shutdown_connections 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf4a20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bf1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 --2- 172.21.15.105:0/1824201275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 -- 172.21.15.105:0/1824201275 >> 172.21.15.105:0/1824201275 conn(0x561b32be7fa0 msgr2=0x561b32be83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 -- 172.21.15.105:0/1824201275 shutdown_connections 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 -- 172.21.15.105:0/1824201275 wait complete. 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 -- start start 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.507+0000 7f9c22c85740 1 Processor -- start 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bed420 0x561b32bf7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bfe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c22c85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561b32bf5540 con 0x561b32beca30 2024-03-26T18:02:10.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c22c85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561b32b0d310 con 0x561b32bed420 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c22c85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b32bf56c0 con 0x561b32bf2590 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c22c85740 10 client.0 _ll_register_callbacks cb 0x561b32b48c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c1e63d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bfe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c1e63d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bfe490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47648/0 (socket says 172.21.15.105:47648) 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c1e63d640 1 -- 172.21.15.105:0/3412162955 learned_addr learned my addr 172.21.15.105:0/3412162955 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bed420 0x561b32bf7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.508+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c1e63d640 1 -- 172.21.15.105:0/3412162955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bed420 msgr2=0x561b32bf7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bed420 0x561b32bf7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c1e63d640 1 -- 172.21.15.105:0/3412162955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32beca30 msgr2=0x561b32bf6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c1e63d640 1 -- 172.21.15.105:0/3412162955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b3292f7d0 con 0x561b32bf2590 2024-03-26T18:02:10.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bfe490 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009ca0 tx=0x7f9c100095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed516bfd43d245e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c017fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c017fa640 1 -- 172.21.15.105:0/3412162955 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10054060 con 0x561b32bf2590 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.509+0000 7f9c017fa640 1 -- 172.21.15.105:0/3412162955 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f9c10004510 con 0x561b32bf2590 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.510+0000 7f9c22c85740 1 -- 172.21.15.105:0/3412162955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c080054d0 con 0x561b32bf2590 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.510+0000 7f9c017fa640 1 -- 172.21.15.105:0/3412162955 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9c10039040 con 0x561b32bf2590 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.510+0000 7f9c22c85740 10 client.33682 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.510+0000 7f9c22c85740 1 -- 172.21.15.105:0/3412162955 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561b32c002b0 con 0x561b32bf2590 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c017fa640 1 -- 172.21.15.105:0/3412162955 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c080054d0 con 0x561b32bf2590 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c017fa640 1 -- 172.21.15.105:0/3412162955 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f9c10057050 con 0x561b32bf2590 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c22c85740 10 client.33682 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c22c85740 10 client.33682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c22c85740 20 client.33682 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:10.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561b32c01ed0 0x561b32c02320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c22c85740 4 client.33682 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c22c85740 1 -- 172.21.15.105:0/3412162955 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b32b5eeb0 con 0x561b32c01ed0 2024-03-26T18:02:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.511+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561b32c01ed0 0x561b32c02320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.512+0000 7f9c017fa640 10 client.33682 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:02:10.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.512+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561b32c01ed0 0x561b32c02320 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45c0aa3ac0ce728e server_cookie=8d010233466d17eb in_seq=0 out_seq=0 2024-03-26T18:02:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c017fa640 1 -- 172.21.15.105:0/3412162955 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561b32b5eeb0 con 0x561b32c01ed0 2024-03-26T18:02:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c017fa640 10 client.33682 handle_command_reply: tid=0 2024-03-26T18:02:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 client.33682 shutdown 2024-03-26T18:02:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 -- 172.21.15.105:0/3412162955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 msgr2=0x561b32bfe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bfe490 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009ca0 tx=0x7f9c100095c0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 -- 172.21.15.105:0/3412162955 shutdown_connections 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561b32c01ed0 0x561b32c02320 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bfe490 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c017fa640 0 client.33682 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bed420 0x561b32bf7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3412162955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.513+0000 7f9c22c85740 1 -- 172.21.15.105:0/3412162955 >> 172.21.15.105:0/3412162955 conn(0x561b32be7fa0 msgr2=0x561b32be71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.514+0000 7f9c22c85740 1 -- 172.21.15.105:0/3412162955 shutdown_connections 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.514+0000 7f9c22c85740 1 -- 172.21.15.105:0/3412162955 wait complete. 2024-03-26T18:02:10.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.514+0000 7f9c22c85740 20 client.33682 trim_cache size 0 max 16384 2024-03-26T18:02:10.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c22c85740 1 Processor -- start 2024-03-26T18:02:10.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c22c85740 1 -- start start 2024-03-26T18:02:10.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c22c85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181a6c70 con 0x561b32bed420 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c22c85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181a87b0 con 0x561b32beca30 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c22c85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181a7340 con 0x561b32bf2590 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c1d63b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c1d63b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38360/0 (socket says 172.21.15.105:38360) 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c1d63b640 1 -- 172.21.15.105:0/1466888367 learned_addr learned my addr 172.21.15.105:0/1466888367 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.534+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c1d63b640 1 -- 172.21.15.105:0/1466888367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 msgr2=0x561b32bf7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c1d63b640 1 -- 172.21.15.105:0/1466888367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 msgr2=0x561b32bf6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c1d63b640 1 -- 172.21.15.105:0/1466888367 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b32be3a10 con 0x561b32bed420 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:10.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfe450 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009250 tx=0x7f9c08009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13882e58554ffff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c037fe640 1 -- 172.21.15.105:0/1466888367 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08004050 con 0x561b32bed420 2024-03-26T18:02:10.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c037fe640 1 -- 172.21.15.105:0/1466888367 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c08005500 con 0x561b32bed420 2024-03-26T18:02:10.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.535+0000 7f9c037fe640 1 -- 172.21.15.105:0/1466888367 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08004c80 con 0x561b32bed420 2024-03-26T18:02:10.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 -- 172.21.15.105:0/1466888367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 msgr2=0x561b32bfe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfe450 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009250 tx=0x7f9c08009930 comp rx=0 tx=0).stop 2024-03-26T18:02:10.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 -- 172.21.15.105:0/1466888367 shutdown_connections 2024-03-26T18:02:10.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32bf2590 0x561b32bf7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfe450 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 --2- 172.21.15.105:0/1466888367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32beca30 0x561b32bf6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 -- 172.21.15.105:0/1466888367 >> 172.21.15.105:0/1466888367 conn(0x561b32be7fa0 msgr2=0x561b32be8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 -- 172.21.15.105:0/1466888367 shutdown_connections 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.536+0000 7f9c22c85740 1 -- 172.21.15.105:0/1466888367 wait complete. 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 1 -- start start 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 1 Processor -- start 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32beca30 0x561b32bfb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bf2590 0x561b32bfca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181a80e0 con 0x561b32bed420 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181a7a10 con 0x561b32bf2590 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181a4c50 con 0x561b32beca30 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c22c85740 10 client.0 _ll_register_callbacks cb 0x7f9c14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c1de3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32beca30 0x561b32bfb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c1de3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32beca30 0x561b32bfb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47678/0 (socket says 172.21.15.105:47678) 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c1de3c640 1 -- 172.21.15.105:0/3945145044 learned_addr learned my addr 172.21.15.105:0/3945145044 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.537+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c1e63d640 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bf2590 0x561b32bfca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c1de3c640 1 -- 172.21.15.105:0/3945145044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bf2590 msgr2=0x561b32bfca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bf2590 0x561b32bfca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c1de3c640 1 -- 172.21.15.105:0/3945145044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 msgr2=0x561b32bfb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c1de3c640 1 -- 172.21.15.105:0/3945145044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b32bfcff0 con 0x561b32beca30 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c1de3c640 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32beca30 0x561b32bfb4b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009fd0 tx=0x7f9c140cd250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a7d2139702cc66a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c017fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c017fa640 1 -- 172.21.15.105:0/3945145044 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c140a0100 con 0x561b32beca30 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c22c85740 1 -- 172.21.15.105:0/3945145044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c08005500 con 0x561b32beca30 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c017fa640 1 -- 172.21.15.105:0/3945145044 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f9c140bc0c0 con 0x561b32beca30 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c017fa640 1 -- 172.21.15.105:0/3945145044 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9c140a49c0 con 0x561b32beca30 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c22c85740 10 client.33688 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.538+0000 7f9c22c85740 1 -- 172.21.15.105:0/3945145044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561b32bfe3c0 con 0x561b32beca30 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c017fa640 1 -- 172.21.15.105:0/3945145044 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c08005500 con 0x561b32beca30 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c017fa640 1 -- 172.21.15.105:0/3945145044 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f9c140b0130 con 0x561b32beca30 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c22c85740 10 client.33688 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c22c85740 10 client.33688 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:02:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c22c85740 20 client.33688 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561b32c01ed0 0x561b32c746c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c22c85740 4 client.33688 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c22c85740 1 -- 172.21.15.105:0/3945145044 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b32b59200 con 0x561b32c01ed0 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561b32c01ed0 0x561b32c746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c017fa640 10 client.33688 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.539+0000 7f9c1d63b640 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561b32c01ed0 0x561b32c746c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e1a2e5b794c9641 server_cookie=22d815b4a5924ab7 in_seq=0 out_seq=0 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c017fa640 1 -- 172.21.15.105:0/3945145044 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9c08030020 con 0x561b32c01ed0 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c017fa640 10 client.33688 handle_command_reply: tid=0 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 client.33688 shutdown 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 -- 172.21.15.105:0/3945145044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32beca30 msgr2=0x561b32bfb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32beca30 0x561b32bfb4b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9c14009fd0 tx=0x7f9c140cd250 comp rx=0 tx=0).stop 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 -- 172.21.15.105:0/3945145044 shutdown_connections 2024-03-26T18:02:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x561b32c01ed0 0x561b32c746c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561b32bf2590 0x561b32bfca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561b32bed420 0x561b32bfb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c017fa640 0 client.33688 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 --2- 172.21.15.105:0/3945145044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b32beca30 0x561b32bfb4b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 -- 172.21.15.105:0/3945145044 >> 172.21.15.105:0/3945145044 conn(0x561b32be7fa0 msgr2=0x561b32bf6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 -- 172.21.15.105:0/3945145044 shutdown_connections 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 1 -- 172.21.15.105:0/3945145044 wait complete. 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c22c85740 20 client.33688 trim_cache size 0 max 16384 2024-03-26T18:02:10.527 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c200a0640 1 -- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9c040a2fa0 msgr2=0x7f9c040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c200a0640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9c040a2fa0 0x7f9c040a5460 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9c10006010 tx=0x7f9c100324f0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c200a0640 1 -- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c181007e0 msgr2=0x7f9c18197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.540+0000 7f9c200a0640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c181007e0 0x7f9c18197b10 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400e9c0 tx=0x7f9c1400ee80 comp rx=0 tx=0).stop 2024-03-26T18:02:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.541+0000 7f9c200a0640 1 -- 172.21.15.105:0/2025122490 shutdown_connections 2024-03-26T18:02:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.541+0000 7f9c200a0640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f9c040a2fa0 0x7f9c040a5460 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.541+0000 7f9c200a0640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c18102390 0x7f9c1819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.541+0000 7f9c200a0640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c181019d0 0x7f9c18198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.541+0000 7f9c200a0640 1 --2- 172.21.15.105:0/2025122490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c181007e0 0x7f9c18197b10 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.541+0000 7f9c200a0640 1 -- 172.21.15.105:0/2025122490 >> 172.21.15.105:0/2025122490 conn(0x7f9c180fbd70 msgr2=0x7f9c180fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.541+0000 7f9c200a0640 1 -- 172.21.15.105:0/2025122490 shutdown_connections 2024-03-26T18:02:10.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.541+0000 7f9c200a0640 1 -- 172.21.15.105:0/2025122490 wait complete. 2024-03-26T18:02:10.535 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T18:02:10.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 85447 7.04 MB/sec execute 533 sec latency 144.769 ms 2024-03-26T18:02:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 -- 172.21.15.105:0/2074274777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1017c0 msgr2=0x7f654c101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 --2- 172.21.15.105:0/2074274777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1017c0 0x7f654c101be0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f653c009900 tx=0x7f653c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 -- 172.21.15.105:0/2074274777 shutdown_connections 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 --2- 172.21.15.105:0/2074274777 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f654c103370 0x7f654c109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 --2- 172.21.15.105:0/2074274777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c1029b0 0x7f654c102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 --2- 172.21.15.105:0/2074274777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1017c0 0x7f654c101be0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 -- 172.21.15.105:0/2074274777 >> 172.21.15.105:0/2074274777 conn(0x7f654c0fcd30 msgr2=0x7f654c0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 -- 172.21.15.105:0/2074274777 shutdown_connections 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.696+0000 7f655159d640 1 -- 172.21.15.105:0/2074274777 wait complete. 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f655159d640 1 Processor -- start 2024-03-26T18:02:10.678 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f655159d640 1 -- start start 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f655159d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c1017c0 0x7f654c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f655159d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1029b0 0x7f654c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f655159d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f654c103370 0x7f654c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f655159d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f654c10d2e0 con 0x7f654c1029b0 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f655159d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f654c10d160 con 0x7f654c103370 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f655159d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f654c10d460 con 0x7f654c1017c0 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f654b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1029b0 0x7f654c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f654bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c1017c0 0x7f654c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f654b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1029b0 0x7f654c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38384/0 (socket says 172.21.15.105:38384) 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f654bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c1017c0 0x7f654c197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47696/0 (socket says 172.21.15.105:47696) 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f654b7fe640 1 -- 172.21.15.105:0/4143139631 learned_addr learned my addr 172.21.15.105:0/4143139631 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.697+0000 7f6550d9c640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f654c103370 0x7f654c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f654b7fe640 1 -- 172.21.15.105:0/4143139631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c1017c0 msgr2=0x7f654c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f654b7fe640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c1017c0 0x7f654c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f654b7fe640 1 -- 172.21.15.105:0/4143139631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f654c103370 msgr2=0x7f654c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f654b7fe640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f654c103370 0x7f654c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f654b7fe640 1 -- 172.21.15.105:0/4143139631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c19f630 con 0x7f654c1029b0 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f654b7fe640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1029b0 0x7f654c197f50 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f6540009c40 tx=0x7f654000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f65497fa640 1 -- 172.21.15.105:0/4143139631 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540005020 con 0x7f654c1029b0 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f655159d640 1 -- 172.21.15.105:0/4143139631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654c073b50 con 0x7f654c1029b0 2024-03-26T18:02:10.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f655159d640 1 -- 172.21.15.105:0/4143139631 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654c0740a0 con 0x7f654c1029b0 2024-03-26T18:02:10.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f65497fa640 1 -- 172.21.15.105:0/4143139631 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6540005670 con 0x7f654c1029b0 2024-03-26T18:02:10.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.698+0000 7f65497fa640 1 -- 172.21.15.105:0/4143139631 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540019670 con 0x7f654c1029b0 2024-03-26T18:02:10.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65553eb740 1 Processor -- start 2024-03-26T18:02:10.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65553eb740 1 -- start start 2024-03-26T18:02:10.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65553eb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfe9d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65553eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9d630 0x5622cfea2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65553eb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfea27a0 0x5622cfea4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65553eb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5622cfe01970 con 0x5622cfea27a0 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65553eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5622cfda9d70 con 0x5622cfe9d630 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65553eb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5622cfdfd920 con 0x5622cfe9cc40 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.701+0000 7f65497fa640 1 -- 172.21.15.105:0/4143139631 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f65400051c0 con 0x7f654c1029b0 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f654bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9d630 0x5622cfea2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f654bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9d630 0x5622cfea2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41808/0 (socket says 172.21.15.105:41808) 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f654bfff640 1 -- 172.21.15.105:0/3706358822 learned_addr learned my addr 172.21.15.105:0/3706358822 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f6550d9c640 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfe9d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f654b7fe640 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfea27a0 0x5622cfea4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f6550d9c640 1 -- 172.21.15.105:0/3706358822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9d630 msgr2=0x5622cfea2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f6550d9c640 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9d630 0x5622cfea2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f6550d9c640 1 -- 172.21.15.105:0/3706358822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfea27a0 msgr2=0x5622cfea4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f6550d9c640 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfea27a0 0x5622cfea4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f6550d9c640 1 -- 172.21.15.105:0/3706358822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cfb153f0 con 0x5622cfe9cc40 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f654b7fe640 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfea27a0 0x5622cfea4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f6550d9c640 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfe9d060 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6544009f10 tx=0x7f6544011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=639aa36cec557146 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f65327fc640 1 -- 172.21.15.105:0/3706358822 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544004d20 con 0x5622cfe9cc40 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.702+0000 7f65327fc640 1 -- 172.21.15.105:0/3706358822 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6544004030 con 0x5622cfe9cc40 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65327fc640 1 -- 172.21.15.105:0/3706358822 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544005260 con 0x5622cfe9cc40 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65497fa640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65340a2fa0 0x7f65340a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f654bfff640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65340a2fa0 0x7f65340a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65497fa640 1 -- 172.21.15.105:0/4143139631 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f65400d2d10 con 0x7f654c1029b0 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65553eb740 1 -- 172.21.15.105:0/3706358822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 msgr2=0x5622cfe9d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65553eb740 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfe9d060 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6544009f10 tx=0x7f6544011810 comp rx=0 tx=0).stop 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f654bfff640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65340a2fa0 0x7f65340a5460 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x5622cfe9da30 tx=0x7f653c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65553eb740 1 -- 172.21.15.105:0/3706358822 shutdown_connections 2024-03-26T18:02:10.690 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65553eb740 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfea27a0 0x5622cfea4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65553eb740 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9d630 0x5622cfea2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65553eb740 1 --2- 172.21.15.105:0/3706358822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfe9d060 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.703+0000 7f65553eb740 1 -- 172.21.15.105:0/3706358822 >> 172.21.15.105:0/3706358822 conn(0x5622cfe981b0 msgr2=0x5622cfe985b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 -- 172.21.15.105:0/3706358822 shutdown_connections 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 -- 172.21.15.105:0/3706358822 wait complete. 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 -- start start 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 Processor -- start 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfea7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfeae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5622cfea57a0 con 0x5622cfe9d630 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5622cfdbd310 con 0x5622cfea27a0 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5622cfea5920 con 0x5622cfe9cc40 2024-03-26T18:02:10.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.704+0000 7f65553eb740 10 client.0 _ll_register_callbacks cb 0x5622cfdf8c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f654bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f654b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfea7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfeae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfeae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41816/0 (socket says 172.21.15.105:41816) 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 -- 172.21.15.105:0/3293453885 learned_addr learned my addr 172.21.15.105:0/3293453885 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 -- 172.21.15.105:0/3293453885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 msgr2=0x5622cfea71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 -- 172.21.15.105:0/3293453885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 msgr2=0x5622cfea7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfea7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 -- 172.21.15.105:0/3293453885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cfbdf7d0 con 0x5622cfea27a0 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f654bfff640 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f6550d9c640 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfeae750 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6544004d20 tx=0x7f6544004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e655df488629d30 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f652b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:10.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f652b7fe640 1 -- 172.21.15.105:0/3293453885 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6544027020 con 0x5622cfea27a0 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f652b7fe640 1 -- 172.21.15.105:0/3293453885 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6544003c30 con 0x5622cfea27a0 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.705+0000 7f65553eb740 1 -- 172.21.15.105:0/3293453885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6544004030 con 0x5622cfea27a0 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f652b7fe640 1 -- 172.21.15.105:0/3293453885 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6544032050 con 0x5622cfea27a0 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f65553eb740 10 client.33701 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f65553eb740 1 -- 172.21.15.105:0/3293453885 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5622cfeb1ac0 con 0x5622cfea27a0 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f652b7fe640 1 -- 172.21.15.105:0/3293453885 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6544004030 con 0x5622cfea27a0 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f652b7fe640 1 -- 172.21.15.105:0/3293453885 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f6544018070 con 0x5622cfea27a0 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f65553eb740 10 client.33701 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f65553eb740 10 client.33701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f65553eb740 20 client.33701 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f65553eb740 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5622cfeb30a0 0x5622cfeb54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f65553eb740 4 client.33701 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f65553eb740 1 -- 172.21.15.105:0/3293453885 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622cfe0eeb0 con 0x5622cfeb30a0 2024-03-26T18:02:10.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.706+0000 7f654bfff640 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5622cfeb30a0 0x5622cfeb54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.707+0000 7f652b7fe640 10 client.33701 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:10.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.707+0000 7f654bfff640 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5622cfeb30a0 0x5622cfeb54c0 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff200de82665d68a server_cookie=74d932ff6be33c75 in_seq=0 out_seq=0 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f652b7fe640 1 -- 172.21.15.105:0/3293453885 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5622cfe0eeb0 con 0x5622cfeb30a0 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f652b7fe640 10 client.33701 handle_command_reply: tid=0 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 client.33701 shutdown 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 -- 172.21.15.105:0/3293453885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 msgr2=0x5622cfeae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfeae750 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6544004d20 tx=0x7f6544004d50 comp rx=0 tx=0).stop 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 -- 172.21.15.105:0/3293453885 shutdown_connections 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5622cfeb30a0 0x5622cfeb54c0 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfeae750 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfea7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f652b7fe640 0 client.33701 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:10.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 --2- 172.21.15.105:0/3293453885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 -- 172.21.15.105:0/3293453885 >> 172.21.15.105:0/3293453885 conn(0x5622cfe981b0 msgr2=0x5622cfe973d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 -- 172.21.15.105:0/3293453885 shutdown_connections 2024-03-26T18:02:10.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 1 -- 172.21.15.105:0/3293453885 wait complete. 2024-03-26T18:02:10.696 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.709+0000 7f65553eb740 20 client.33701 trim_cache size 0 max 16384 2024-03-26T18:02:10.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.727+0000 7f65553eb740 1 Processor -- start 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.727+0000 7f65553eb740 1 -- start start 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.727+0000 7f65553eb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f65553eb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f65553eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfea6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f65553eb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f653800e530 con 0x5622cfe9d630 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f65553eb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6538010070 con 0x5622cfea27a0 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f65553eb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f653800ec00 con 0x5622cfe9cc40 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47732/0 (socket says 172.21.15.105:47732) 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 -- 172.21.15.105:0/1332882230 learned_addr learned my addr 172.21.15.105:0/1332882230 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654b7fe640 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f6550d9c640 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfea6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 -- 172.21.15.105:0/1332882230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 msgr2=0x5622cfea6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfea6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 -- 172.21.15.105:0/1332882230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 msgr2=0x5622cfeae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 -- 172.21.15.105:0/1332882230 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622cfe97b00 con 0x5622cfe9cc40 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.728+0000 7f654bfff640 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea5d50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f653c036040 tx=0x7f653c045000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0583a891f77c8ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.729+0000 7f6531ffb640 1 -- 172.21.15.105:0/1332882230 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c054af0 con 0x5622cfe9cc40 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.729+0000 7f6531ffb640 1 -- 172.21.15.105:0/1332882230 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f653c0054f0 con 0x5622cfe9cc40 2024-03-26T18:02:10.711 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.729+0000 7f6531ffb640 1 -- 172.21.15.105:0/1332882230 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c054580 con 0x5622cfe9cc40 2024-03-26T18:02:10.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 -- 172.21.15.105:0/1332882230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 msgr2=0x5622cfea5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea5d50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f653c036040 tx=0x7f653c045000 comp rx=0 tx=0).stop 2024-03-26T18:02:10.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 -- 172.21.15.105:0/1332882230 shutdown_connections 2024-03-26T18:02:10.715 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfea27a0 0x5622cfea6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 --2- 172.21.15.105:0/1332882230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfe9cc40 0x5622cfea5d50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 -- 172.21.15.105:0/1332882230 >> 172.21.15.105:0/1332882230 conn(0x5622cfe981b0 msgr2=0x5622cfe96b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 -- 172.21.15.105:0/1332882230 shutdown_connections 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 -- 172.21.15.105:0/1332882230 wait complete. 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 -- start start 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.730+0000 7f65553eb740 1 Processor -- start 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f65553eb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9cc40 0x5622cfeac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f65553eb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f654bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9cc40 0x5622cfeac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f654bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9cc40 0x5622cfeac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41844/0 (socket says 172.21.15.105:41844) 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f654bfff640 1 -- 172.21.15.105:0/721498111 learned_addr learned my addr 172.21.15.105:0/721498111 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:10.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f654b7fe640 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f65553eb740 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfea27a0 0x5622cfead780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f653800f9a0 con 0x5622cfe9d630 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f653800f2d0 con 0x5622cfe9cc40 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f653800c510 con 0x5622cfea27a0 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f65553eb740 10 client.0 _ll_register_callbacks cb 0x7f6540005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.731+0000 7f6550d9c640 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfea27a0 0x5622cfead780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f654bfff640 1 -- 172.21.15.105:0/721498111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfea27a0 msgr2=0x5622cfead780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f654bfff640 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfea27a0 0x5622cfead780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f654bfff640 1 -- 172.21.15.105:0/721498111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 msgr2=0x5622cfeac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f654bfff640 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f654bfff640 1 -- 172.21.15.105:0/721498111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622cfe98f20 con 0x5622cfe9cc40 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f654b7fe640 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:10.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f654bfff640 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9cc40 0x5622cfeac230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f653c02cd00 tx=0x7f653c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b48e7f091e95e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f652b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f652b7fe640 1 -- 172.21.15.105:0/721498111 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c060070 con 0x5622cfe9cc40 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f652b7fe640 1 -- 172.21.15.105:0/721498111 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f653c054500 con 0x5622cfe9cc40 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f653c0054f0 con 0x5622cfe9cc40 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f652b7fe640 1 -- 172.21.15.105:0/721498111 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f653c06a050 con 0x5622cfe9cc40 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f65553eb740 10 client.33707 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.732+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5622cff21de0 con 0x5622cfe9cc40 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f652b7fe640 1 -- 172.21.15.105:0/721498111 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f653c054740 con 0x5622cfe9cc40 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f652b7fe640 1 -- 172.21.15.105:0/721498111 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f653c034560 con 0x5622cfe9cc40 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f65553eb740 10 client.33707 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:10.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f65553eb740 10 client.33707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:10.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f65553eb740 20 client.33707 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:10.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f65553eb740 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5622cff22800 0x5622cff24c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:10.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f65553eb740 4 client.33707 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:10.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f653c0054f0 con 0x5622cff22800 2024-03-26T18:02:10.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.733+0000 7f654b7fe640 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5622cff22800 0x5622cff24c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:10.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.734+0000 7f654b7fe640 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5622cff22800 0x5622cff24c20 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6ea27a8b2004b3 server_cookie=91a274483a7b24a in_seq=0 out_seq=0 2024-03-26T18:02:10.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.734+0000 7f652b7fe640 10 client.33707 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:10.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f652b7fe640 1 -- 172.21.15.105:0/721498111 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f653c0054f0 con 0x5622cff22800 2024-03-26T18:02:10.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f652b7fe640 10 client.33707 handle_command_reply: tid=0 2024-03-26T18:02:10.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 client.33707 shutdown 2024-03-26T18:02:10.720 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9cc40 msgr2=0x5622cfeac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9cc40 0x5622cfeac230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f653c02cd00 tx=0x7f653c005720 comp rx=0 tx=0).stop 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 shutdown_connections 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5622cff22800 0x5622cff24c20 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f652b7fe640 0 client.33707 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5622cfea27a0 0x5622cfead780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5622cfe9d630 0x5622cfeac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 --2- 172.21.15.105:0/721498111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5622cfe9cc40 0x5622cfeac230 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 >> 172.21.15.105:0/721498111 conn(0x5622cfe981b0 msgr2=0x5622cfe9bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 shutdown_connections 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 1 -- 172.21.15.105:0/721498111 wait complete. 2024-03-26T18:02:10.721 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.737+0000 7f65553eb740 20 client.33707 trim_cache size 0 max 16384 2024-03-26T18:02:10.722 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:10.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 -- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65340a2fa0 msgr2=0x7f65340a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65340a2fa0 0x7f65340a5460 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x5622cfe9da30 tx=0x7f653c005fd0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 -- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1029b0 msgr2=0x7f654c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:10.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1029b0 0x7f654c197f50 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f6540009c40 tx=0x7f654000b690 comp rx=0 tx=0).stop 2024-03-26T18:02:10.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 -- 172.21.15.105:0/4143139631 shutdown_connections 2024-03-26T18:02:10.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f65340a2fa0 0x7f65340a5460 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f654c103370 0x7f654c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f654c1029b0 0x7f654c197f50 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 --2- 172.21.15.105:0/4143139631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c1017c0 0x7f654c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:10.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 -- 172.21.15.105:0/4143139631 >> 172.21.15.105:0/4143139631 conn(0x7f654c0fcd30 msgr2=0x7f654c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:10.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 -- 172.21.15.105:0/4143139631 shutdown_connections 2024-03-26T18:02:10.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:10.738+0000 7f655159d640 1 -- 172.21.15.105:0/4143139631 wait complete. 2024-03-26T18:02:11.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:11 smithi144 ceph-mon[36095]: pgmap v1068: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-03-26T18:02:11.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:11 smithi105 ceph-mon[32280]: pgmap v1068: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-03-26T18:02:11.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:11 smithi135 ceph-mon[35636]: pgmap v1068: 97 pgs: 97 active+clean; 368 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-03-26T18:02:11.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 85593 7.05 MB/sec execute 534 sec latency 116.782 ms 2024-03-26T18:02:11.733 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T18:02:11.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:11.734 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T18:02:11.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 -- 172.21.15.105:0/2485463999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 msgr2=0x7f5920101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 --2- 172.21.15.105:0/2485463999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 0x7f5920101cf0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f591c009860 tx=0x7f591c02bb70 comp rx=0 tx=0).stop 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 -- 172.21.15.105:0/2485463999 shutdown_connections 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 --2- 172.21.15.105:0/2485463999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5920103480 0x7f5920109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 --2- 172.21.15.105:0/2485463999 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5920102ac0 0x7f5920102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 --2- 172.21.15.105:0/2485463999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 0x7f5920101cf0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 -- 172.21.15.105:0/2485463999 >> 172.21.15.105:0/2485463999 conn(0x7f59200fce20 msgr2=0x7f59200ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 -- 172.21.15.105:0/2485463999 shutdown_connections 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.893+0000 7f5927659640 1 -- 172.21.15.105:0/2485463999 wait complete. 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5927659640 1 Processor -- start 2024-03-26T18:02:11.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5927659640 1 -- start start 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5927659640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 0x7f5920075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5927659640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5920102ac0 0x7f5920073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5927659640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5920103480 0x7f59200741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5927659640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f592010d3d0 con 0x7f5920103480 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5927659640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f592010d250 con 0x7f5920102ac0 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5927659640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f592010d550 con 0x7f59201018d0 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5925bf6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5920103480 0x7f59200741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f59253f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 0x7f5920075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5925bf6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5920103480 0x7f59200741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38430/0 (socket says 172.21.15.105:38430) 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5925bf6640 1 -- 172.21.15.105:0/3380553535 learned_addr learned my addr 172.21.15.105:0/3380553535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:11.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.894+0000 7f5924bf4640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5920102ac0 0x7f5920073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59253f5640 1 -- 172.21.15.105:0/3380553535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5920102ac0 msgr2=0x7f5920073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59253f5640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5920102ac0 0x7f5920073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59253f5640 1 -- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5920103480 msgr2=0x7f59200741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59253f5640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5920103480 0x7f59200741f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59253f5640 1 -- 172.21.15.105:0/3380553535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5920074a80 con 0x7f59201018d0 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59253f5640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 0x7f5920075670 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f591c037040 tx=0x7f591c0040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59167fc640 1 -- 172.21.15.105:0/3380553535 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c041020 con 0x7f59201018d0 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59167fc640 1 -- 172.21.15.105:0/3380553535 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f591c0044d0 con 0x7f59201018d0 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f5927659640 1 -- 172.21.15.105:0/3380553535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59201a1c50 con 0x7f59201018d0 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f59167fc640 1 -- 172.21.15.105:0/3380553535 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c039040 con 0x7f59201018d0 2024-03-26T18:02:11.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.895+0000 7f5927659640 1 -- 172.21.15.105:0/3380553535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59201a21a0 con 0x7f59201018d0 2024-03-26T18:02:11.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.898+0000 7f59167fc640 1 -- 172.21.15.105:0/3380553535 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f591c044020 con 0x7f59201018d0 2024-03-26T18:02:11.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f592a240740 1 Processor -- start 2024-03-26T18:02:11.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f592a240740 1 -- start start 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f592a240740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cdbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f592a240740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdc390 0x563564ce0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f592a240740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ce3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f592a240740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563564c38630 con 0x563564cdb9a0 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f592a240740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563564be0e10 con 0x563564ce1500 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f592a240740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563564c345e0 con 0x563564cdc390 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f5925bf6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cdbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f5925bf6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cdbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38436/0 (socket says 172.21.15.105:38436) 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f5925bf6640 1 -- 172.21.15.105:0/3348453109 learned_addr learned my addr 172.21.15.105:0/3348453109 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f5924bf4640 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ce3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f59253f5640 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdc390 0x563564ce0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.899+0000 7f59167fc640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f590c0a3020 0x7f590c0a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f5925bf6640 1 -- 172.21.15.105:0/3348453109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdc390 msgr2=0x563564ce0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f5925bf6640 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdc390 0x563564ce0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f5925bf6640 1 -- 172.21.15.105:0/3348453109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 msgr2=0x563564ce3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f5925bf6640 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ce3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f5925bf6640 1 -- 172.21.15.105:0/3348453109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563564bf7740 con 0x563564cdb9a0 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f5924bf4640 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ce3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f59167fc640 1 -- 172.21.15.105:0/3380553535 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f591c0f0c20 con 0x7f59201018d0 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f5925bf6640 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cdbdc0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f5918011620 tx=0x7f5918011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11d4e17117210b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f5925bf6640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f590c0a3020 0x7f590c0a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.900+0000 7f590b7fe640 1 -- 172.21.15.105:0/3348453109 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591801f020 con 0x563564cdb9a0 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.901+0000 7f590b7fe640 1 -- 172.21.15.105:0/3348453109 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f591801b430 con 0x563564cdb9a0 2024-03-26T18:02:11.883 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.901+0000 7f590b7fe640 1 -- 172.21.15.105:0/3348453109 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918017050 con 0x563564cdb9a0 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 -- 172.21.15.105:0/3348453109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 msgr2=0x563564cdbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f5925bf6640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f590c0a3020 0x7f590c0a54e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f591800ffd0 tx=0x7f591800ff40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cdbdc0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f5918011620 tx=0x7f5918011ae0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 -- 172.21.15.105:0/3348453109 shutdown_connections 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ce3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdc390 0x563564ce0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 --2- 172.21.15.105:0/3348453109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cdbdc0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 -- 172.21.15.105:0/3348453109 >> 172.21.15.105:0/3348453109 conn(0x563564cd6f10 msgr2=0x563564cd7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 -- 172.21.15.105:0/3348453109 shutdown_connections 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 -- 172.21.15.105:0/3348453109 wait complete. 2024-03-26T18:02:11.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 -- start start 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.902+0000 7f592a240740 1 Processor -- start 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f592a240740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564ce5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f592a240740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 0x563564ce6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f592a240740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ced3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f592a240740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563564ce44b0 con 0x563564cdb9a0 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f592a240740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563564bf41a0 con 0x563564cdc390 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f592a240740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563564ce4630 con 0x563564ce1500 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f5925bf6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ced3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f59253f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564ce5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f5924bf4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 0x563564ce6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f5925bf6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ced3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47766/0 (socket says 172.21.15.105:47766) 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f5925bf6640 1 -- 172.21.15.105:0/2065542863 learned_addr learned my addr 172.21.15.105:0/2065542863 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f592a240740 10 client.0 _ll_register_callbacks cb 0x56356494bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:11.888 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f5925bf6640 1 -- 172.21.15.105:0/2065542863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 msgr2=0x563564ce6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f5925bf6640 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 0x563564ce6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.903+0000 7f5925bf6640 1 -- 172.21.15.105:0/2065542863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 msgr2=0x563564ce5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f5925bf6640 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564ce5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f5925bf6640 1 -- 172.21.15.105:0/2065542863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563564a147e0 con 0x563564ce1500 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f59253f5640 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564ce5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f5925bf6640 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ced3d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f59180115f0 tx=0x7f591801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375ec27e7eebb029 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f5908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f5908ff9640 1 -- 172.21.15.105:0/2065542863 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918033020 con 0x563564ce1500 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f592a240740 1 -- 172.21.15.105:0/2065542863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591801b430 con 0x563564ce1500 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f5908ff9640 1 -- 172.21.15.105:0/2065542863 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f5918011050 con 0x563564ce1500 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f5908ff9640 1 -- 172.21.15.105:0/2065542863 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f591803e050 con 0x563564ce1500 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f592a240740 10 client.33718 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.904+0000 7f592a240740 1 -- 172.21.15.105:0/2065542863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563564a74f70 con 0x563564ce1500 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f5908ff9640 1 -- 172.21.15.105:0/2065542863 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f591800ee50 con 0x563564ce1500 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f5908ff9640 1 -- 172.21.15.105:0/2065542863 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f591800fa30 con 0x563564ce1500 2024-03-26T18:02:11.889 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f592a240740 10 client.33718 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:11.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f592a240740 10 client.33718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:11.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f592a240740 20 client.33718 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:11.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f592a240740 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563564cf2000 0x563564cf44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f592a240740 4 client.33718 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:11.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f592a240740 1 -- 172.21.15.105:0/2065542863 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f591801b430 con 0x563564cf2000 2024-03-26T18:02:11.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.905+0000 7f59253f5640 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563564cf2000 0x563564cf44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.906+0000 7f5908ff9640 10 client.33718 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:11.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.906+0000 7f59253f5640 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563564cf2000 0x563564cf44c0 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5de843ef002458c server_cookie=82c1529b39298378 in_seq=0 out_seq=0 2024-03-26T18:02:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f5908ff9640 1 -- 172.21.15.105:0/2065542863 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f591801b430 con 0x563564cf2000 2024-03-26T18:02:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f5908ff9640 10 client.33718 handle_command_reply: tid=0 2024-03-26T18:02:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 client.33718 shutdown 2024-03-26T18:02:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 -- 172.21.15.105:0/2065542863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 msgr2=0x563564ced3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ced3d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f59180115f0 tx=0x7f591801c000 comp rx=0 tx=0).stop 2024-03-26T18:02:11.892 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 -- 172.21.15.105:0/2065542863 shutdown_connections 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563564cf2000 0x563564cf44c0 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f5908ff9640 0 client.33718 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ced3d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 0x563564ce6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 --2- 172.21.15.105:0/2065542863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564ce5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 -- 172.21.15.105:0/2065542863 >> 172.21.15.105:0/2065542863 conn(0x563564cd6f10 msgr2=0x563564ccd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 -- 172.21.15.105:0/2065542863 shutdown_connections 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 1 -- 172.21.15.105:0/2065542863 wait complete. 2024-03-26T18:02:11.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.908+0000 7f592a240740 20 client.33718 trim_cache size 0 max 16384 2024-03-26T18:02:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f592a240740 1 Processor -- start 2024-03-26T18:02:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f592a240740 1 -- start start 2024-03-26T18:02:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f592a240740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cda880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f592a240740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 0x563564ced390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f592a240740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ccd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f592a240740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59201a6c70 con 0x563564cdb9a0 2024-03-26T18:02:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f592a240740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59201a87b0 con 0x563564cdc390 2024-03-26T18:02:11.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f592a240740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59201a7340 con 0x563564ce1500 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f59253f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cda880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f59253f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cda880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38448/0 (socket says 172.21.15.105:38448) 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f59253f5640 1 -- 172.21.15.105:0/1989196648 learned_addr learned my addr 172.21.15.105:0/1989196648 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f5924bf4640 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 0x563564ced390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f5925bf6640 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ccd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f59253f5640 1 -- 172.21.15.105:0/1989196648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 msgr2=0x563564ccd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f59253f5640 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ccd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.937+0000 7f59253f5640 1 -- 172.21.15.105:0/1989196648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 msgr2=0x563564ced390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.938+0000 7f59253f5640 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 0x563564ced390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.938+0000 7f59253f5640 1 -- 172.21.15.105:0/1989196648 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356490c170 con 0x563564cdb9a0 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.938+0000 7f5925bf6640 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ccd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.938+0000 7f59253f5640 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cda880 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f591c0b9c80 tx=0x7f591c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfea8219ba605fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.938+0000 7f590affd640 1 -- 172.21.15.105:0/1989196648 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0bcdf0 con 0x563564cdb9a0 2024-03-26T18:02:11.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.938+0000 7f590affd640 1 -- 172.21.15.105:0/1989196648 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f591c0f6020 con 0x563564cdb9a0 2024-03-26T18:02:11.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.938+0000 7f590affd640 1 -- 172.21.15.105:0/1989196648 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0bcc10 con 0x563564cdb9a0 2024-03-26T18:02:11.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 -- 172.21.15.105:0/1989196648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 msgr2=0x563564cda880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cda880 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f591c0b9c80 tx=0x7f591c02e040 comp rx=0 tx=0).stop 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 -- 172.21.15.105:0/1989196648 shutdown_connections 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564ce1500 0x563564ccd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564cdc390 0x563564ced390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 --2- 172.21.15.105:0/1989196648 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdb9a0 0x563564cda880 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 -- 172.21.15.105:0/1989196648 >> 172.21.15.105:0/1989196648 conn(0x563564cd6f10 msgr2=0x563564cda170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 -- 172.21.15.105:0/1989196648 shutdown_connections 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 -- 172.21.15.105:0/1989196648 wait complete. 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.939+0000 7f592a240740 1 -- start start 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f592a240740 1 Processor -- start 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f592a240740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdb9a0 0x563564cea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f592a240740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdc390 0x563564ceaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f592a240740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ceba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f592a240740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59201a80e0 con 0x563564cdc390 2024-03-26T18:02:11.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f592a240740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59201a7a10 con 0x563564ce1500 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f592a240740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59201a4c50 con 0x563564cdb9a0 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f592a240740 10 client.0 _ll_register_callbacks cb 0x7f59180114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f5924bf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdc390 0x563564ceaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f59253f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdb9a0 0x563564cea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f5924bf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdc390 0x563564ceaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38456/0 (socket says 172.21.15.105:38456) 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f5925bf6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ceba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f5924bf4640 1 -- 172.21.15.105:0/2932959906 learned_addr learned my addr 172.21.15.105:0/2932959906 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.940+0000 7f59253f5640 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdb9a0 0x563564cea500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47786/0 (socket says 172.21.15.105:47786) 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5924bf4640 1 -- 172.21.15.105:0/2932959906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdb9a0 msgr2=0x563564cea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5924bf4640 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdb9a0 0x563564cea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5924bf4640 1 -- 172.21.15.105:0/2932959906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 msgr2=0x563564ceba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5924bf4640 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ceba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5924bf4640 1 -- 172.21.15.105:0/2932959906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563564cec040 con 0x563564cdc390 2024-03-26T18:02:11.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f59253f5640 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdb9a0 0x563564cea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5925bf6640 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ceba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5924bf4640 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdc390 0x563564ceaa40 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fd0 tx=0x7f5910012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4e03561044c62d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5908ff9640 1 -- 172.21.15.105:0/2932959906 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910022060 con 0x563564cdc390 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f592a240740 1 -- 172.21.15.105:0/2932959906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591c0f6020 con 0x563564cdc390 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5908ff9640 1 -- 172.21.15.105:0/2932959906 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f5910004220 con 0x563564cdc390 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f5908ff9640 1 -- 172.21.15.105:0/2932959906 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5910004420 con 0x563564cdc390 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f592a240740 10 client.24057 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.941+0000 7f592a240740 1 -- 172.21.15.105:0/2932959906 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563564ced480 con 0x563564cdc390 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f5908ff9640 1 -- 172.21.15.105:0/2932959906 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f591c0f6020 con 0x563564cdc390 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f5908ff9640 1 -- 172.21.15.105:0/2932959906 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f5910005020 con 0x563564cdc390 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f592a240740 10 client.24057 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f592a240740 10 client.24057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:11.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f592a240740 20 client.24057 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f592a240740 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563564cf2000 0x563564d5aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f592a240740 4 client.24057 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T18:02:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f592a240740 1 -- 172.21.15.105:0/2932959906 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563564c45b70 con 0x563564cf2000 2024-03-26T18:02:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f59253f5640 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563564cf2000 0x563564d5aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f5908ff9640 10 client.24057 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:11.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.942+0000 7f59253f5640 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563564cf2000 0x563564d5aa90 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fd6c3517a1bffe server_cookie=db7ba540b751796a in_seq=0 out_seq=0 2024-03-26T18:02:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f5908ff9640 1 -- 172.21.15.105:0/2932959906 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563564c45b70 con 0x563564cf2000 2024-03-26T18:02:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f5908ff9640 10 client.24057 handle_command_reply: tid=0 2024-03-26T18:02:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 client.24057 shutdown 2024-03-26T18:02:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 -- 172.21.15.105:0/2932959906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdc390 msgr2=0x563564ceaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdc390 0x563564ceaa40 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fd0 tx=0x7f5910012d70 comp rx=0 tx=0).stop 2024-03-26T18:02:11.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 -- 172.21.15.105:0/2932959906 shutdown_connections 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563564cf2000 0x563564d5aa90 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563564ce1500 0x563564ceba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563564cdc390 0x563564ceaa40 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f5908ff9640 0 client.24057 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 --2- 172.21.15.105:0/2932959906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563564cdb9a0 0x563564cea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 -- 172.21.15.105:0/2932959906 >> 172.21.15.105:0/2932959906 conn(0x563564cd6f10 msgr2=0x563564cdaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 -- 172.21.15.105:0/2932959906 shutdown_connections 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 1 -- 172.21.15.105:0/2932959906 wait complete. 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.945+0000 7f592a240740 20 client.24057 trim_cache size 0 max 16384 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:02:11.929 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T18:02:11.930 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "07226078-0c4b-4c8b-ad67-d20acb5b89b2", 2024-03-26T18:02:11.930 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T18:02:11.930 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:02:11.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 -- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f590c0a3020 msgr2=0x7f590c0a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f590c0a3020 0x7f590c0a54e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f591800ffd0 tx=0x7f591800ff40 comp rx=0 tx=0).stop 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 -- 172.21.15.105:0/3380553535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 msgr2=0x7f5920075670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 0x7f5920075670 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f591c037040 tx=0x7f591c0040e0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 -- 172.21.15.105:0/3380553535 shutdown_connections 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f590c0a3020 0x7f590c0a54e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5920103480 0x7f59200741f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5920102ac0 0x7f5920073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 --2- 172.21.15.105:0/3380553535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59201018d0 0x7f5920075670 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 -- 172.21.15.105:0/3380553535 >> 172.21.15.105:0/3380553535 conn(0x7f59200fce20 msgr2=0x7f5920105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 -- 172.21.15.105:0/3380553535 shutdown_connections 2024-03-26T18:02:11.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:11.946+0000 7f5927659640 1 -- 172.21.15.105:0/3380553535 wait complete. 2024-03-26T18:02:11.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:02:11.940 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:02:12.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 -- 172.21.15.105:0/2045871729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09041007e0 msgr2=0x7f0904100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 --2- 172.21.15.105:0/2045871729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09041007e0 0x7f0904100c00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009860 tx=0x7f08f402bb70 comp rx=0 tx=0).stop 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 -- 172.21.15.105:0/2045871729 shutdown_connections 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 --2- 172.21.15.105:0/2045871729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0904102390 0x7f0904108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 --2- 172.21.15.105:0/2045871729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09041019d0 0x7f0904101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 --2- 172.21.15.105:0/2045871729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09041007e0 0x7f0904100c00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 -- 172.21.15.105:0/2045871729 >> 172.21.15.105:0/2045871729 conn(0x7f09040fbd50 msgr2=0x7f09040fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 -- 172.21.15.105:0/2045871729 shutdown_connections 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 -- 172.21.15.105:0/2045871729 wait complete. 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.096+0000 7f090a875640 1 Processor -- start 2024-03-26T18:02:12.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f090a875640 1 -- start start 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f090a875640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09041007e0 0x7f0904199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f090a875640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09041019d0 0x7f090419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f090a875640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0904102390 0x7f09041a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f090a875640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f090410b210 con 0x7f09041019d0 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f090a875640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f090410b090 con 0x7f09041007e0 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f090a875640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f090410b390 con 0x7f0904102390 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f0903fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09041007e0 0x7f0904199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f0903fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09041007e0 0x7f0904199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41904/0 (socket says 172.21.15.105:41904) 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f0903fff640 1 -- 172.21.15.105:0/3493257699 learned_addr learned my addr 172.21.15.105:0/3493257699 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:12.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f09037fe640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09041019d0 0x7f090419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.097+0000 7f0908e12640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0904102390 0x7f09041a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f0903fff640 1 -- 172.21.15.105:0/3493257699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0904102390 msgr2=0x7f09041a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f0903fff640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0904102390 0x7f09041a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f0903fff640 1 -- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09041019d0 msgr2=0x7f090419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f0903fff640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09041019d0 0x7f090419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f0903fff640 1 -- 172.21.15.105:0/3493257699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09041a19d0 con 0x7f09041007e0 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f09037fe640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09041019d0 0x7f090419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f0903fff640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09041007e0 0x7f0904199dc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005940 tx=0x7f08f40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f09017fa640 1 -- 172.21.15.105:0/3493257699 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4041020 con 0x7f09041007e0 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f090a875640 1 -- 172.21.15.105:0/3493257699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09041a1c60 con 0x7f09041007e0 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.098+0000 7f09017fa640 1 -- 172.21.15.105:0/3493257699 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08f40035f0 con 0x7f09041007e0 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.099+0000 7f09017fa640 1 -- 172.21.15.105:0/3493257699 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4039040 con 0x7f09041007e0 2024-03-26T18:02:12.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.099+0000 7f090a875640 1 -- 172.21.15.105:0/3493257699 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09041a21a0 con 0x7f09041007e0 2024-03-26T18:02:12.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.101+0000 7f09017fa640 1 -- 172.21.15.105:0/3493257699 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f08f4044020 con 0x7f09041007e0 2024-03-26T18:02:12.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.101+0000 7f09017fa640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08ec0a2f20 0x7f08ec0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.102+0000 7f09037fe640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08ec0a2f20 0x7f08ec0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.102+0000 7f09017fa640 1 -- 172.21.15.105:0/3493257699 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f08f40f0ec0 con 0x7f09041007e0 2024-03-26T18:02:12.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.102+0000 7f09037fe640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08ec0a2f20 0x7f08ec0a53e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009fc0 tx=0x7f08f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:12.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.102+0000 7f090d45a740 1 Processor -- start 2024-03-26T18:02:12.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.102+0000 7f090d45a740 1 -- start start 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f090d45a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded0238f0 0x561ded023d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f090d45a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0242e0 0x561ded028de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f090d45a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded029450 0x561ded02b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f090d45a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561decf88980 con 0x561ded029450 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f090d45a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561decf30be0 con 0x561ded0242e0 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f090d45a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561decf84930 con 0x561ded0238f0 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded029450 0x561ded02b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded029450 0x561ded02b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38494/0 (socket says 172.21.15.105:38494) 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 -- 172.21.15.105:0/1079600087 learned_addr learned my addr 172.21.15.105:0/1079600087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f09037fe640 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0242e0 0x561ded028de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0903fff640 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded0238f0 0x561ded023d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 -- 172.21.15.105:0/1079600087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded0238f0 msgr2=0x561ded023d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded0238f0 0x561ded023d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 -- 172.21.15.105:0/1079600087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0242e0 msgr2=0x561ded028de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:12.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0242e0 0x561ded028de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 -- 172.21.15.105:0/1079600087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561decd667d0 con 0x561ded029450 2024-03-26T18:02:12.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0903fff640 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded0238f0 0x561ded023d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:02:12.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f0908e12640 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded029450 0x561ded02b8e0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f08fc005c40 tx=0x7f08fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab5ad84f0520427 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:12.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.103+0000 7f08ea7fc640 1 -- 172.21.15.105:0/1079600087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00c800 con 0x561ded029450 2024-03-26T18:02:12.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.104+0000 7f08ea7fc640 1 -- 172.21.15.105:0/1079600087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08fc01c070 con 0x561ded029450 2024-03-26T18:02:12.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.104+0000 7f08ea7fc640 1 -- 172.21.15.105:0/1079600087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc017d00 con 0x561ded029450 2024-03-26T18:02:12.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 -- 172.21.15.105:0/1079600087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded029450 msgr2=0x561ded02b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded029450 0x561ded02b8e0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f08fc005c40 tx=0x7f08fc0095c0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 -- 172.21.15.105:0/1079600087 shutdown_connections 2024-03-26T18:02:12.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded029450 0x561ded02b8e0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0242e0 0x561ded028de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 --2- 172.21.15.105:0/1079600087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded0238f0 0x561ded023d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 -- 172.21.15.105:0/1079600087 >> 172.21.15.105:0/1079600087 conn(0x561ded01ee60 msgr2=0x561ded01f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:12.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 -- 172.21.15.105:0/1079600087 shutdown_connections 2024-03-26T18:02:12.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.105+0000 7f090d45a740 1 -- 172.21.15.105:0/1079600087 wait complete. 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f090d45a740 1 -- start start 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f090d45a740 1 Processor -- start 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f090d45a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0238f0 0x561ded02dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f090d45a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0242e0 0x561ded02e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f090d45a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f090d45a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561ded02c400 con 0x561ded0242e0 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f09037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0242e0 0x561ded02e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f090d45a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561decf44160 con 0x561ded0238f0 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f090d45a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561ded02c580 con 0x561ded029450 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.106+0000 7f09037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0242e0 0x561ded02e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38502/0 (socket says 172.21.15.105:38502) 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f0908e12640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f09037fe640 1 -- 172.21.15.105:0/2197665370 learned_addr learned my addr 172.21.15.105:0/2197665370 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f0903fff640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0238f0 0x561ded02dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f090d45a740 10 client.0 _ll_register_callbacks cb 0x561decf80720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f09037fe640 1 -- 172.21.15.105:0/2197665370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 msgr2=0x561ded034fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f09037fe640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f09037fe640 1 -- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0238f0 msgr2=0x561ded02dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f09037fe640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0238f0 0x561ded02dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f09037fe640 1 -- 172.21.15.105:0/2197665370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561decc9c3f0 con 0x561ded0242e0 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f0908e12640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f0903fff640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0238f0 0x561ded02dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f09037fe640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0242e0 0x561ded02e010 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f08f8003730 tx=0x7f08f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a42e7e51402fcd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f08e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f08e37fe640 1 -- 172.21.15.105:0/2197665370 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f800f940 con 0x561ded0242e0 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f08e37fe640 1 -- 172.21.15.105:0/2197665370 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f08f800fae0 con 0x561ded0242e0 2024-03-26T18:02:12.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.107+0000 7f090d45a740 1 -- 172.21.15.105:0/2197665370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08fc01c070 con 0x561ded0242e0 2024-03-26T18:02:12.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f08e37fe640 1 -- 172.21.15.105:0/2197665370 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f08f800fd20 con 0x561ded0242e0 2024-03-26T18:02:12.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f090d45a740 10 client.24069 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:12.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f090d45a740 1 -- 172.21.15.105:0/2197665370 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561decc5c0f0 con 0x561ded0242e0 2024-03-26T18:02:12.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f08e37fe640 1 -- 172.21.15.105:0/2197665370 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08fc01c070 con 0x561ded0242e0 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f08e37fe640 1 -- 172.21.15.105:0/2197665370 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f08f801a050 con 0x561ded0242e0 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f090d45a740 10 client.24069 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f090d45a740 10 client.24069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f090d45a740 20 client.24069 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f090d45a740 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561ded0398a0 0x561ded03bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f090d45a740 4 client.24069 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f090d45a740 1 -- 172.21.15.105:0/2197665370 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561decf95ec0 con 0x561ded0398a0 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.108+0000 7f0903fff640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561ded0398a0 0x561ded03bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.109+0000 7f08e37fe640 10 client.24069 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.109+0000 7f0903fff640 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561ded0398a0 0x561ded03bd20 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71dd4dc4a8c5c557 server_cookie=29bbcf732b727e59 in_seq=0 out_seq=0 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f08e37fe640 1 -- 172.21.15.105:0/2197665370 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561decf95ec0 con 0x561ded0398a0 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f08e37fe640 10 client.24069 handle_command_reply: tid=0 2024-03-26T18:02:12.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 client.24069 shutdown 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 -- 172.21.15.105:0/2197665370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0242e0 msgr2=0x561ded02e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0242e0 0x561ded02e010 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f08f8003730 tx=0x7f08f8004730 comp rx=0 tx=0).stop 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 -- 172.21.15.105:0/2197665370 shutdown_connections 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561ded0398a0 0x561ded03bd20 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0242e0 0x561ded02e010 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 --2- 172.21.15.105:0/2197665370 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0238f0 0x561ded02dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 -- 172.21.15.105:0/2197665370 >> 172.21.15.105:0/2197665370 conn(0x561ded01ee60 msgr2=0x561ded01e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f08e37fe640 0 client.24069 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 -- 172.21.15.105:0/2197665370 shutdown_connections 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 1 -- 172.21.15.105:0/2197665370 wait complete. 2024-03-26T18:02:12.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.111+0000 7f090d45a740 20 client.24069 trim_cache size 0 max 16384 2024-03-26T18:02:12.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f090d45a740 1 Processor -- start 2024-03-26T18:02:12.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f090d45a740 1 -- start start 2024-03-26T18:02:12.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f090d45a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded028af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f090d45a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f090d45a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded025950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f090d45a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ab020 con 0x561ded0238f0 2024-03-26T18:02:12.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f090d45a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09041acb60 con 0x561ded0254c0 2024-03-26T18:02:12.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f090d45a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ab6f0 con 0x561ded029450 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f0903fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded028af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f0903fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded028af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38510/0 (socket says 172.21.15.105:38510) 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f0903fff640 1 -- 172.21.15.105:0/901220911 learned_addr learned my addr 172.21.15.105:0/901220911 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f09037fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.137+0000 7f0908e12640 1 --2- 172.21.15.105:0/901220911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded025950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f0903fff640 1 -- 172.21.15.105:0/901220911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 msgr2=0x561ded034fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f0903fff640 1 --2- 172.21.15.105:0/901220911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f0903fff640 1 -- 172.21.15.105:0/901220911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 msgr2=0x561ded025950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f0903fff640 1 --2- 172.21.15.105:0/901220911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded025950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f0903fff640 1 -- 172.21.15.105:0/901220911 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ded0246c0 con 0x561ded0238f0 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f0903fff640 1 --2- 172.21.15.105:0/901220911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded028af0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f08f40bd4e0 tx=0x7f08f40ee7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b8bb792d55b8a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f08e9ffb640 1 -- 172.21.15.105:0/901220911 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40d6070 con 0x561ded0238f0 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f08e9ffb640 1 -- 172.21.15.105:0/901220911 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08f40bdc50 con 0x561ded0238f0 2024-03-26T18:02:12.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.138+0000 7f08e9ffb640 1 -- 172.21.15.105:0/901220911 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40a6070 con 0x561ded0238f0 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 -- 172.21.15.105:0/901220911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 msgr2=0x561ded028af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 --2- 172.21.15.105:0/901220911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded028af0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f08f40bd4e0 tx=0x7f08f40ee7f0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 -- 172.21.15.105:0/901220911 shutdown_connections 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 --2- 172.21.15.105:0/901220911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded025950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 --2- 172.21.15.105:0/901220911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded034fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 --2- 172.21.15.105:0/901220911 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded028af0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 -- 172.21.15.105:0/901220911 >> 172.21.15.105:0/901220911 conn(0x561ded01ee60 msgr2=0x561ded02dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 -- 172.21.15.105:0/901220911 shutdown_connections 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 -- 172.21.15.105:0/901220911 wait complete. 2024-03-26T18:02:12.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.139+0000 7f090d45a740 1 -- start start 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f090d45a740 1 Processor -- start 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f090d45a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded032380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f090d45a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded0328c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f090d45a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded0337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f090d45a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09041ac490 con 0x561ded0238f0 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f090d45a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09041abdc0 con 0x561ded0254c0 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f090d45a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09041a9000 con 0x561ded029450 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f090d45a740 10 client.0 _ll_register_callbacks cb 0x7f08f80035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0903fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded032380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f09037fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded0328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0903fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded032380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38516/0 (socket says 172.21.15.105:38516) 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0903fff640 1 -- 172.21.15.105:0/100995629 learned_addr learned my addr 172.21.15.105:0/100995629 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:12.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0908e12640 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded0337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0903fff640 1 -- 172.21.15.105:0/100995629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 msgr2=0x561ded0337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0903fff640 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded0337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0903fff640 1 -- 172.21.15.105:0/100995629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 msgr2=0x561ded0328c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0903fff640 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded0328c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f0903fff640 1 -- 172.21.15.105:0/100995629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ded033dd0 con 0x561ded0238f0 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.140+0000 7f09037fe640 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded0328c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f0903fff640 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded032380 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f08f40c5a30 tx=0x7f08f4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c1b77cd2e6ccdd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f08e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f08e37fe640 1 -- 172.21.15.105:0/100995629 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40db060 con 0x561ded0238f0 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f08e37fe640 1 -- 172.21.15.105:0/100995629 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f08f40c5ad0 con 0x561ded0238f0 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f090d45a740 1 -- 172.21.15.105:0/100995629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f40bdc50 con 0x561ded0238f0 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f08e37fe640 1 -- 172.21.15.105:0/100995629 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f08f40a6040 con 0x561ded0238f0 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f090d45a740 10 client.24075 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:12.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f090d45a740 1 -- 172.21.15.105:0/100995629 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561ded0350e0 con 0x561ded0238f0 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f08e37fe640 1 -- 172.21.15.105:0/100995629 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08f40bdc50 con 0x561ded0238f0 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.141+0000 7f08e37fe640 1 -- 172.21.15.105:0/100995629 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f08f403c030 con 0x561ded0238f0 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f090d45a740 10 client.24075 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f090d45a740 10 client.24075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f090d45a740 20 client.24075 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f090d45a740 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561ded0a00b0 0x561ded0a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f090d45a740 4 client.24075 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f090d45a740 1 -- 172.21.15.105:0/100995629 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561decf90210 con 0x561ded0a00b0 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f09037fe640 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561ded0a00b0 0x561ded0a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f08e37fe640 10 client.24075 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:12.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.142+0000 7f09037fe640 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561ded0a00b0 0x561ded0a24d0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e1ee095e20bff0 server_cookie=3b0987c7713d1606 in_seq=0 out_seq=0 2024-03-26T18:02:12.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f08e37fe640 1 -- 172.21.15.105:0/100995629 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x561decf90210 con 0x561ded0a00b0 2024-03-26T18:02:12.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f08e37fe640 10 client.24075 handle_command_reply: tid=0 2024-03-26T18:02:12.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 client.24075 shutdown 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 -- 172.21.15.105:0/100995629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 msgr2=0x561ded032380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded032380 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f08f40c5a30 tx=0x7f08f4004610 comp rx=0 tx=0).stop 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 -- 172.21.15.105:0/100995629 shutdown_connections 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561ded0a00b0 0x561ded0a24d0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561ded029450 0x561ded0337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ded0254c0 0x561ded0328c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f08e37fe640 0 client.24075 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 --2- 172.21.15.105:0/100995629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561ded0238f0 0x561ded032380 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 -- 172.21.15.105:0/100995629 >> 172.21.15.105:0/100995629 conn(0x561ded01ee60 msgr2=0x561ded0220c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 -- 172.21.15.105:0/100995629 shutdown_connections 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.144+0000 7f090d45a740 1 -- 172.21.15.105:0/100995629 wait complete. 2024-03-26T18:02:12.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090d45a740 20 client.24075 trim_cache size 0 max 16384 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (310 inodes in the stack)", 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout: "07226078-0c4b-4c8b-ad67-d20acb5b89b2": { 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "07226078-0c4b-4c8b-ad67-d20acb5b89b2", 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:02:12.130 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:02:12.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 -- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08ec0a2f20 msgr2=0x7f08ec0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08ec0a2f20 0x7f08ec0a53e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009fc0 tx=0x7f08f8005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:12.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 -- 172.21.15.105:0/3493257699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09041007e0 msgr2=0x7f0904199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:12.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09041007e0 0x7f0904199dc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005940 tx=0x7f08f40042f0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 -- 172.21.15.105:0/3493257699 shutdown_connections 2024-03-26T18:02:12.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f08ec0a2f20 0x7f08ec0a53e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0904102390 0x7f09041a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09041019d0 0x7f090419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 --2- 172.21.15.105:0/3493257699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09041007e0 0x7f0904199dc0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:12.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 -- 172.21.15.105:0/3493257699 >> 172.21.15.105:0/3493257699 conn(0x7f09040fbd50 msgr2=0x7f09041071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:12.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.145+0000 7f090a875640 1 -- 172.21.15.105:0/3493257699 shutdown_connections 2024-03-26T18:02:12.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:12.146+0000 7f090a875640 1 -- 172.21.15.105:0/3493257699 wait complete. 2024-03-26T18:02:12.139 INFO:tasks.cephfs.filesystem:scrub status for tag:07226078-0c4b-4c8b-ad67-d20acb5b89b2 - {'path': '/', 'tag': '07226078-0c4b-4c8b-ad67-d20acb5b89b2', 'options': 'recursive,force'} 2024-03-26T18:02:12.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 85720 7.05 MB/sec execute 535 sec latency 191.977 ms 2024-03-26T18:02:12.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:12 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:12.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:12 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:12.633 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:12 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:12.634 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:12 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:12.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:12 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:12.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:12 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 85837 7.05 MB/sec execute 536 sec latency 113.092 ms 2024-03-26T18:02:13.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T18:02:13.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:13.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:13 smithi144 ceph-mon[36095]: pgmap v1069: 97 pgs: 97 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.7 MiB/s wr, 265 op/s 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.632 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:13 smithi105 ceph-mon[32280]: pgmap v1069: 97 pgs: 97 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.7 MiB/s wr, 265 op/s 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:13.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:13.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:13.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:13 smithi135 ceph-mon[35636]: pgmap v1069: 97 pgs: 97 active+clean; 374 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 3.7 MiB/s wr, 265 op/s 2024-03-26T18:02:14.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 85922 7.04 MB/sec execute 537 sec latency 125.771 ms 2024-03-26T18:02:15.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:15 smithi144 ceph-mon[36095]: pgmap v1070: 97 pgs: 97 active+clean; 378 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 3.9 MiB/s wr, 276 op/s 2024-03-26T18:02:15.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:15 smithi105 ceph-mon[32280]: pgmap v1070: 97 pgs: 97 active+clean; 378 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 3.9 MiB/s wr, 276 op/s 2024-03-26T18:02:15.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:15 smithi135 ceph-mon[35636]: pgmap v1070: 97 pgs: 97 active+clean; 378 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 3.9 MiB/s wr, 276 op/s 2024-03-26T18:02:15.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86053 7.05 MB/sec execute 538 sec latency 125.958 ms 2024-03-26T18:02:16.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86176 7.05 MB/sec execute 539 sec latency 159.909 ms 2024-03-26T18:02:17.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:17 smithi144 ceph-mon[36095]: pgmap v1071: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-03-26T18:02:17.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:17 smithi105 ceph-mon[32280]: pgmap v1071: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-03-26T18:02:17.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:17 smithi135 ceph-mon[35636]: pgmap v1071: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-03-26T18:02:17.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86303 7.05 MB/sec execute 540 sec latency 129.976 ms 2024-03-26T18:02:18.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86437 7.06 MB/sec execute 541 sec latency 141.989 ms 2024-03-26T18:02:19.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:19 smithi144 ceph-mon[36095]: pgmap v1072: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-03-26T18:02:19.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:19 smithi105 ceph-mon[32280]: pgmap v1072: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-03-26T18:02:19.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:19 smithi135 ceph-mon[35636]: pgmap v1072: 97 pgs: 97 active+clean; 381 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 4.3 MiB/s wr, 294 op/s 2024-03-26T18:02:19.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86530 7.05 MB/sec execute 542 sec latency 134.250 ms 2024-03-26T18:02:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.024+0000 7f638873b640 1 -- 172.21.15.105:0/3708098252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380104c90 msgr2=0x7f6380107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.024+0000 7f638873b640 1 --2- 172.21.15.105:0/3708098252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380104c90 0x7f6380107080 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f6370009900 tx=0x7f637002bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.024+0000 7f638873b640 1 -- 172.21.15.105:0/3708098252 shutdown_connections 2024-03-26T18:02:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.024+0000 7f638873b640 1 --2- 172.21.15.105:0/3708098252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63801075c0 0x7f6380109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.024+0000 7f638873b640 1 --2- 172.21.15.105:0/3708098252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380104c90 0x7f6380107080 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.024+0000 7f638873b640 1 --2- 172.21.15.105:0/3708098252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6380102360 0x7f6380104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.024+0000 7f638873b640 1 -- 172.21.15.105:0/3708098252 >> 172.21.15.105:0/3708098252 conn(0x7f63800fbd50 msgr2=0x7f63800fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.025+0000 7f638873b640 1 -- 172.21.15.105:0/3708098252 shutdown_connections 2024-03-26T18:02:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.025+0000 7f638873b640 1 -- 172.21.15.105:0/3708098252 wait complete. 2024-03-26T18:02:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.025+0000 7f638873b640 1 Processor -- start 2024-03-26T18:02:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.025+0000 7f638873b640 1 -- start start 2024-03-26T18:02:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f638873b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380102360 0x7f6380199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f638873b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6380104c90 0x7f638019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f638873b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63801075c0 0x7f63801a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f638873b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f638010d420 con 0x7f6380102360 2024-03-26T18:02:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f638873b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f638010d2a0 con 0x7f6380104c90 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f638873b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f638010d5a0 con 0x7f63801075c0 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380102360 0x7f6380199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380102360 0x7f6380199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60658/0 (socket says 172.21.15.105:60658) 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 -- 172.21.15.105:0/2981688566 learned_addr learned my addr 172.21.15.105:0/2981688566 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f6385cd6640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6380104c90 0x7f638019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f6386cd8640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63801075c0 0x7f63801a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 -- 172.21.15.105:0/2981688566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63801075c0 msgr2=0x7f63801a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63801075c0 0x7f63801a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 -- 172.21.15.105:0/2981688566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6380104c90 msgr2=0x7f638019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6380104c90 0x7f638019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 -- 172.21.15.105:0/2981688566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63801a1900 con 0x7f6380102360 2024-03-26T18:02:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.026+0000 7f63864d7640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380102360 0x7f6380199cf0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f637c00e9f0 tx=0x7f637c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.027+0000 7f63777fe640 1 -- 172.21.15.105:0/2981688566 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c00cd80 con 0x7f6380102360 2024-03-26T18:02:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.027+0000 7f638873b640 1 -- 172.21.15.105:0/2981688566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63801a1bf0 con 0x7f6380102360 2024-03-26T18:02:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.027+0000 7f63777fe640 1 -- 172.21.15.105:0/2981688566 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f637c0054d0 con 0x7f6380102360 2024-03-26T18:02:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.027+0000 7f63777fe640 1 -- 172.21.15.105:0/2981688566 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c010490 con 0x7f6380102360 2024-03-26T18:02:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.027+0000 7f638873b640 1 -- 172.21.15.105:0/2981688566 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63801a2140 con 0x7f6380102360 2024-03-26T18:02:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f638b409740 1 Processor -- start 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f63777fe640 1 -- 172.21.15.105:0/2981688566 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f637c005020 con 0x7f6380102360 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f638b409740 1 -- start start 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f638b409740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b01499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f638b409740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f638b409740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b014f130 0x5645b01515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f638b409740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5645b00ae0b0 con 0x5645b0149fc0 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f638b409740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5645b0056960 con 0x5645b01495d0 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f638b409740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5645b00aa060 con 0x5645b014f130 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f63864d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f6386cd8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b01499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f63864d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b014eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60662/0 (socket says 172.21.15.105:60662) 2024-03-26T18:02:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f63864d7640 1 -- 172.21.15.105:0/3207002050 learned_addr learned my addr 172.21.15.105:0/3207002050 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.030+0000 7f6385cd6640 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b014f130 0x5645b01515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f63864d7640 1 -- 172.21.15.105:0/3207002050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b014f130 msgr2=0x5645b01515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f63864d7640 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b014f130 0x5645b01515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f63864d7640 1 -- 172.21.15.105:0/3207002050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 msgr2=0x5645b01499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f63864d7640 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b01499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f63864d7640 1 -- 172.21.15.105:0/3207002050 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b0068c60 con 0x5645b0149fc0 2024-03-26T18:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f63777fe640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f636c0a2fc0 0x7f636c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f6386cd8640 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b01499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f6386cd8640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f636c0a2fc0 0x7f636c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f63864d7640 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b014eac0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f637c0d5a90 tx=0x7f637c0d5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cfc2e41cccc2d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.031+0000 7f63777fe640 1 -- 172.21.15.105:0/2981688566 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f637c0cf8f0 con 0x7f6380102360 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.032+0000 7f6374ff9640 1 -- 172.21.15.105:0/3207002050 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c0d2e60 con 0x5645b0149fc0 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.032+0000 7f6374ff9640 1 -- 172.21.15.105:0/3207002050 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f637c005310 con 0x5645b0149fc0 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.032+0000 7f6374ff9640 1 -- 172.21.15.105:0/3207002050 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c09bac0 con 0x5645b0149fc0 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.032+0000 7f638b409740 1 -- 172.21.15.105:0/3207002050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 msgr2=0x5645b014eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.032+0000 7f638b409740 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b014eac0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f637c0d5a90 tx=0x7f637c0d5b80 comp rx=0 tx=0).stop 2024-03-26T18:02:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.032+0000 7f6386cd8640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f636c0a2fc0 0x7f636c0a5480 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x5645b014a820 tx=0x7f6378009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 -- 172.21.15.105:0/3207002050 shutdown_connections 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b014f130 0x5645b01515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b014eac0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 --2- 172.21.15.105:0/3207002050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b01499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 -- 172.21.15.105:0/3207002050 >> 172.21.15.105:0/3207002050 conn(0x5645b0144b40 msgr2=0x5645b0144f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 -- 172.21.15.105:0/3207002050 shutdown_connections 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 -- 172.21.15.105:0/3207002050 wait complete. 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 -- start start 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.033+0000 7f638b409740 1 Processor -- start 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f638b409740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b0153380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f638b409740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b0149fc0 0x5645b01538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f638b409740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b015a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f6386cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f6386cd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b015a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60664/0 (socket says 172.21.15.105:60664) 2024-03-26T18:02:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f63864d7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b0153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f6386cd8640 1 -- 172.21.15.105:0/2010693723 learned_addr learned my addr 172.21.15.105:0/2010693723 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f6385cd6640 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b0149fc0 0x5645b01538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5645b01520e0 con 0x5645b014f130 2024-03-26T18:02:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5645b0069db0 con 0x5645b01495d0 2024-03-26T18:02:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5645b0152260 con 0x5645b0149fc0 2024-03-26T18:02:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.034+0000 7f638b409740 10 client.0 _ll_register_callbacks cb 0x5645b006e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6386cd8640 1 -- 172.21.15.105:0/2010693723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b0149fc0 msgr2=0x5645b01538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6386cd8640 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b0149fc0 0x5645b01538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6386cd8640 1 -- 172.21.15.105:0/2010693723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 msgr2=0x5645b0153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6386cd8640 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b0153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6386cd8640 1 -- 172.21.15.105:0/2010693723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645afdc2210 con 0x5645b014f130 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6386cd8640 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b015a900 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f6378010bf0 tx=0x7f6378004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea4db7177f5540d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6369ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6369ffb640 1 -- 172.21.15.105:0/2010693723 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637802c060 con 0x5645b014f130 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f6369ffb640 1 -- 172.21.15.105:0/2010693723 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6378017030 con 0x5645b014f130 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.035+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5645b00bb5f0 con 0x5645b014f130 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f6369ffb640 1 -- 172.21.15.105:0/2010693723 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f637801fbc0 con 0x5645b014f130 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f638b409740 10 client.24087 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5645afe8e620 con 0x5645b014f130 2024-03-26T18:02:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f6369ffb640 1 -- 172.21.15.105:0/2010693723 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5645b00bb5f0 con 0x5645b014f130 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f6369ffb640 1 -- 172.21.15.105:0/2010693723 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f637801c3f0 con 0x5645b014f130 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f638b409740 10 client.24087 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f638b409740 10 client.24087 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f638b409740 20 client.24087 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f638b409740 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5645b015f260 0x5645b0161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f638b409740 4 client.24087 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645b00b5940 con 0x5645b015f260 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.036+0000 7f63864d7640 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5645b015f260 0x5645b0161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.037+0000 7f6369ffb640 10 client.24087 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.037+0000 7f63864d7640 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5645b015f260 0x5645b0161650 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbe07b78ce28745 server_cookie=5f3e2c21588f4e7a in_seq=0 out_seq=0 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f6369ffb640 1 -- 172.21.15.105:0/2010693723 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5645b00b5940 con 0x5645b015f260 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f6369ffb640 10 client.24087 handle_command_reply: tid=0 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 client.24087 shutdown 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 msgr2=0x5645b015a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b015a900 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f6378010bf0 tx=0x7f6378004ab0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 shutdown_connections 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5645b015f260 0x5645b0161650 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b015a900 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b0149fc0 0x5645b01538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f6369ffb640 0 client.24087 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 --2- 172.21.15.105:0/2010693723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b01495d0 0x5645b0153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 >> 172.21.15.105:0/2010693723 conn(0x5645b0144b40 msgr2=0x5645b0143df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.039+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 shutdown_connections 2024-03-26T18:02:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.040+0000 7f638b409740 1 -- 172.21.15.105:0/2010693723 wait complete. 2024-03-26T18:02:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.040+0000 7f638b409740 20 client.24087 trim_cache size 0 max 16384 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f638b409740 1 Processor -- start 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f638b409740 1 -- start start 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f638b409740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b014b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f638b409740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f638b409740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b014f130 0x5645b0165f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f638b409740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63801aaf70 con 0x5645b0149fc0 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f638b409740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63801acab0 con 0x5645b014f130 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f638b409740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63801ab640 con 0x5645b01495d0 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f6385cd6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f6385cd6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b015a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60676/0 (socket says 172.21.15.105:60676) 2024-03-26T18:02:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f6385cd6640 1 -- 172.21.15.105:0/2439820446 learned_addr learned my addr 172.21.15.105:0/2439820446 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f63864d7640 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b014b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.065+0000 7f6386cd8640 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b014f130 0x5645b0165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f6385cd6640 1 -- 172.21.15.105:0/2439820446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 msgr2=0x5645b014b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f6385cd6640 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b014b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f6385cd6640 1 -- 172.21.15.105:0/2439820446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b014f130 msgr2=0x5645b0165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f6385cd6640 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b014f130 0x5645b0165f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f6385cd6640 1 -- 172.21.15.105:0/2439820446 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645b014a3a0 con 0x5645b0149fc0 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f63864d7640 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f6385cd6640 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b015a8c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f6380069aa0 tx=0x7f6370009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c274d2b67d9ba435 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f636bfff640 1 -- 172.21.15.105:0/2439820446 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370044070 con 0x5645b0149fc0 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f636bfff640 1 -- 172.21.15.105:0/2439820446 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f637002cbc0 con 0x5645b0149fc0 2024-03-26T18:02:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.066+0000 7f636bfff640 1 -- 172.21.15.105:0/2439820446 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370005640 con 0x5645b0149fc0 2024-03-26T18:02:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 -- 172.21.15.105:0/2439820446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 msgr2=0x5645b015a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b015a8c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f6380069aa0 tx=0x7f6370009610 comp rx=0 tx=0).stop 2024-03-26T18:02:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 -- 172.21.15.105:0/2439820446 shutdown_connections 2024-03-26T18:02:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b014f130 0x5645b0165f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b0149fc0 0x5645b015a8c0 secure :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f6380069aa0 tx=0x7f6370009610 comp rx=0 tx=0).stop 2024-03-26T18:02:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 --2- 172.21.15.105:0/2439820446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 -- 172.21.15.105:0/2439820446 >> 172.21.15.105:0/2439820446 conn(0x5645b0144b40 msgr2=0x5645b014aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 -- 172.21.15.105:0/2439820446 shutdown_connections 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.067+0000 7f638b409740 1 -- 172.21.15.105:0/2439820446 wait complete. 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 1 -- start start 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 1 Processor -- start 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b015a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b0158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b0159220 0x5645b01596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f63864d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63801ac3e0 con 0x5645b014f130 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63801abd10 con 0x5645b0159220 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63801a8f50 con 0x5645b01495d0 2024-03-26T18:02:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f63864d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b015a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36846/0 (socket says 172.21.15.105:36846) 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f63864d7640 1 -- 172.21.15.105:0/1903102431 learned_addr learned my addr 172.21.15.105:0/1903102431 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f638b409740 10 client.0 _ll_register_callbacks cb 0x7f6378009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f6385cd6640 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b0158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f63864d7640 1 -- 172.21.15.105:0/1903102431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b0159220 msgr2=0x5645b01596a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f63864d7640 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b0159220 0x5645b01596a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f63864d7640 1 -- 172.21.15.105:0/1903102431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 msgr2=0x5645b0158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.068+0000 7f63864d7640 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b0158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f63864d7640 1 -- 172.21.15.105:0/1903102431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645b01cdda0 con 0x5645b01495d0 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f6385cd6640 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b0158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f63864d7640 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b015a660 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f637c0d7670 tx=0x7f637c097b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b46cd84cda28b385 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f6369ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f6369ffb640 1 -- 172.21.15.105:0/1903102431 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c0d2e60 con 0x5645b01495d0 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f6369ffb640 1 -- 172.21.15.105:0/1903102431 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f637c0046c0 con 0x5645b01495d0 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f638b409740 1 -- 172.21.15.105:0/1903102431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f637002cbc0 con 0x5645b01495d0 2024-03-26T18:02:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f6369ffb640 1 -- 172.21.15.105:0/1903102431 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f637c09a0c0 con 0x5645b01495d0 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f638b409740 10 client.33757 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.069+0000 7f638b409740 1 -- 172.21.15.105:0/1903102431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5645b01cf000 con 0x5645b01495d0 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f6369ffb640 1 -- 172.21.15.105:0/1903102431 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f637c005680 con 0x5645b01495d0 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f6369ffb640 1 -- 172.21.15.105:0/1903102431 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f637c0cfce0 con 0x5645b01495d0 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f638b409740 10 client.33757 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f638b409740 10 client.33757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f638b409740 20 client.33757 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f638b409740 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5645b01cf570 0x5645b01d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f638b409740 4 client.33757 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f638b409740 1 -- 172.21.15.105:0/1903102431 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f637002cbc0 con 0x5645b01cf570 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.070+0000 7f6385cd6640 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5645b01cf570 0x5645b01d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.071+0000 7f6369ffb640 10 client.33757 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.071+0000 7f6385cd6640 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5645b01cf570 0x5645b01d1960 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1993c298814aa578 server_cookie=e6b310abd14e52d9 in_seq=0 out_seq=0 2024-03-26T18:02:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f6369ffb640 1 -- 172.21.15.105:0/1903102431 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f637002cbc0 con 0x5645b01cf570 2024-03-26T18:02:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f6369ffb640 10 client.33757 handle_command_reply: tid=0 2024-03-26T18:02:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 client.33757 shutdown 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 -- 172.21.15.105:0/1903102431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 msgr2=0x5645b015a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b015a660 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f637c0d7670 tx=0x7f637c097b20 comp rx=0 tx=0).stop 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 -- 172.21.15.105:0/1903102431 shutdown_connections 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5645b01cf570 0x5645b01d1960 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5645b0159220 0x5645b01596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f6369ffb640 0 client.33757 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5645b014f130 0x5645b0158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 --2- 172.21.15.105:0/1903102431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645b01495d0 0x5645b015a660 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 -- 172.21.15.105:0/1903102431 >> 172.21.15.105:0/1903102431 conn(0x5645b0144b40 msgr2=0x5645b0143560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 -- 172.21.15.105:0/1903102431 shutdown_connections 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 1 -- 172.21.15.105:0/1903102431 wait complete. 2024-03-26T18:02:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.269+0000 7f638b409740 20 client.33757 trim_cache size 0 max 16384 2024-03-26T18:02:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:02:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:02:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:02:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 -- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f636c0a2fc0 msgr2=0x7f636c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f636c0a2fc0 0x7f636c0a5480 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x5645b014a820 tx=0x7f6378009500 comp rx=0 tx=0).stop 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 -- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380102360 msgr2=0x7f6380199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380102360 0x7f6380199cf0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f637c00e9f0 tx=0x7f637c00edf0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 -- 172.21.15.105:0/2981688566 shutdown_connections 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f636c0a2fc0 0x7f636c0a5480 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63801075c0 0x7f63801a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6380104c90 0x7f638019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 --2- 172.21.15.105:0/2981688566 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6380102360 0x7f6380199cf0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 -- 172.21.15.105:0/2981688566 >> 172.21.15.105:0/2981688566 conn(0x7f63800fbd50 msgr2=0x7f6380101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 -- 172.21.15.105:0/2981688566 shutdown_connections 2024-03-26T18:02:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:20.270+0000 7f638873b640 1 -- 172.21.15.105:0/2981688566 wait complete. 2024-03-26T18:02:20.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86616 7.04 MB/sec execute 543 sec latency 240.029 ms 2024-03-26T18:02:21.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:21 smithi144 ceph-mon[36095]: pgmap v1073: 97 pgs: 97 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-03-26T18:02:21.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:21 smithi105 ceph-mon[32280]: pgmap v1073: 97 pgs: 97 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-03-26T18:02:21.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:21 smithi135 ceph-mon[35636]: pgmap v1073: 97 pgs: 97 active+clean; 382 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 3.4 MiB/s wr, 235 op/s 2024-03-26T18:02:21.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86743 7.04 MB/sec execute 544 sec latency 142.925 ms 2024-03-26T18:02:22.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86826 7.03 MB/sec execute 545 sec latency 137.093 ms 2024-03-26T18:02:23.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:23 smithi144 ceph-mon[36095]: pgmap v1074: 97 pgs: 97 active+clean; 386 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-03-26T18:02:23.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:23 smithi105 ceph-mon[32280]: pgmap v1074: 97 pgs: 97 active+clean; 386 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-03-26T18:02:23.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:23 smithi135 ceph-mon[35636]: pgmap v1074: 97 pgs: 97 active+clean; 386 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 3.4 MiB/s wr, 247 op/s 2024-03-26T18:02:23.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 86944 7.03 MB/sec execute 546 sec latency 137.783 ms 2024-03-26T18:02:24.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 87078 7.04 MB/sec execute 547 sec latency 138.083 ms 2024-03-26T18:02:25.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:25 smithi144 ceph-mon[36095]: pgmap v1075: 97 pgs: 97 active+clean; 387 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-03-26T18:02:25.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:25 smithi105 ceph-mon[32280]: pgmap v1075: 97 pgs: 97 active+clean; 387 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-03-26T18:02:25.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:25 smithi135 ceph-mon[35636]: pgmap v1075: 97 pgs: 97 active+clean; 387 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 3.4 MiB/s wr, 266 op/s 2024-03-26T18:02:25.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 87207 7.03 MB/sec execute 548 sec latency 131.726 ms 2024-03-26T18:02:26.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 87337 7.03 MB/sec execute 549 sec latency 146.699 ms 2024-03-26T18:02:27.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:26 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:02:27.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:26 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:02:27.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:26 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:02:27.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 87454 7.02 MB/sec execute 550 sec latency 130.515 ms 2024-03-26T18:02:28.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:02:28.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:02:28.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:27 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:02:28.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:27 smithi144 ceph-mon[36095]: pgmap v1076: 97 pgs: 97 active+clean; 385 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-03-26T18:02:28.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:02:28.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:02:28.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:27 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:02:28.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:27 smithi105 ceph-mon[32280]: pgmap v1076: 97 pgs: 97 active+clean; 385 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-03-26T18:02:28.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:02:28.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:02:28.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:27 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:02:28.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:27 smithi135 ceph-mon[35636]: pgmap v1076: 97 pgs: 97 active+clean; 385 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 3.3 MiB/s wr, 262 op/s 2024-03-26T18:02:28.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 87582 7.02 MB/sec execute 551 sec latency 136.578 ms 2024-03-26T18:02:29.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 87704 7.02 MB/sec execute 552 sec latency 139.239 ms 2024-03-26T18:02:30.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:29 smithi144 ceph-mon[36095]: pgmap v1077: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-26T18:02:30.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:29 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:30.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:29 smithi105 ceph-mon[32280]: pgmap v1077: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-26T18:02:30.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:29 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:30.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:29 smithi135 ceph-mon[35636]: pgmap v1077: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-03-26T18:02:30.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:29 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:02:30.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 87831 7.02 MB/sec execute 553 sec latency 146.169 ms 2024-03-26T18:02:31.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:31.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:31.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:30 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:02:31.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:31.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:31.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:30 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:02:31.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:02:31.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:02:31.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:30 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:02:31.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 87983 7.03 MB/sec execute 554 sec latency 136.344 ms 2024-03-26T18:02:32.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:31 smithi144 ceph-mon[36095]: pgmap v1078: 97 pgs: 97 active+clean; 381 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-03-26T18:02:32.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:31 smithi105 ceph-mon[32280]: pgmap v1078: 97 pgs: 97 active+clean; 381 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-03-26T18:02:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:31 smithi135 ceph-mon[35636]: pgmap v1078: 97 pgs: 97 active+clean; 381 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-03-26T18:02:32.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 88115 7.02 MB/sec execute 555 sec latency 124.704 ms 2024-03-26T18:02:33.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:32 smithi144 ceph-mon[36095]: pgmap v1079: 97 pgs: 97 active+clean; 379 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 3.1 MiB/s wr, 248 op/s 2024-03-26T18:02:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.395+0000 7f3b4b020640 1 -- 172.21.15.105:0/4223524807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 msgr2=0x7f3b44108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.395+0000 7f3b4b020640 1 --2- 172.21.15.105:0/4223524807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 0x7f3b44108380 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009ca0 tx=0x7f3b3c02c100 comp rx=0 tx=0).stop 2024-03-26T18:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.396+0000 7f3b4b020640 1 -- 172.21.15.105:0/4223524807 shutdown_connections 2024-03-26T18:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.396+0000 7f3b4b020640 1 --2- 172.21.15.105:0/4223524807 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 0x7f3b44108380 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.396+0000 7f3b4b020640 1 --2- 172.21.15.105:0/4223524807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b44101130 0x7f3b441015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.396+0000 7f3b4b020640 1 --2- 172.21.15.105:0/4223524807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b44100140 0x7f3b44100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.396+0000 7f3b4b020640 1 -- 172.21.15.105:0/4223524807 >> 172.21.15.105:0/4223524807 conn(0x7f3b440fb0a0 msgr2=0x7f3b440fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.396+0000 7f3b4b020640 1 -- 172.21.15.105:0/4223524807 shutdown_connections 2024-03-26T18:02:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.396+0000 7f3b4b020640 1 -- 172.21.15.105:0/4223524807 wait complete. 2024-03-26T18:02:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.397+0000 7f3b4b020640 1 Processor -- start 2024-03-26T18:02:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.397+0000 7f3b4b020640 1 -- start start 2024-03-26T18:02:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.397+0000 7f3b4b020640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b44100140 0x7f3b44195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.397+0000 7f3b4b020640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b44101130 0x7f3b44195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.397+0000 7f3b4b020640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 0x7f3b4419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b4b020640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410b270 con 0x7f3b44101af0 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b48dbc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b44100140 0x7f3b44195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b48dbc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b44100140 0x7f3b44195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41654/0 (socket says 172.21.15.105:41654) 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b48dbc640 1 -- 172.21.15.105:0/1223054729 learned_addr learned my addr 172.21.15.105:0/1223054729 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b495bd640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 0x7f3b4419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410b0f0 con 0x7f3b44100140 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410b3f0 con 0x7f3b44101130 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b43fff640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b44101130 0x7f3b44195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b495bd640 1 -- 172.21.15.105:0/1223054729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b44101130 msgr2=0x7f3b44195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b495bd640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b44101130 0x7f3b44195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b495bd640 1 -- 172.21.15.105:0/1223054729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b44100140 msgr2=0x7f3b44195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b495bd640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b44100140 0x7f3b44195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b495bd640 1 -- 172.21.15.105:0/1223054729 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4419d440 con 0x7f3b44101af0 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b44100140 0x7f3b44195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b43fff640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b44101130 0x7f3b44195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.398+0000 7f3b495bd640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 0x7f3b4419cdd0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009bf0 tx=0x7f3b3c035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:33.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:32 smithi105 ceph-mon[32280]: pgmap v1079: 97 pgs: 97 active+clean; 379 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 3.1 MiB/s wr, 248 op/s 2024-03-26T18:02:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.399+0000 7f3b41ffb640 1 -- 172.21.15.105:0/1223054729 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c039040 con 0x7f3b44101af0 2024-03-26T18:02:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.399+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4419d6d0 con 0x7f3b44101af0 2024-03-26T18:02:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.399+0000 7f3b41ffb640 1 -- 172.21.15.105:0/1223054729 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b3c034740 con 0x7f3b44101af0 2024-03-26T18:02:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.399+0000 7f3b41ffb640 1 -- 172.21.15.105:0/1223054729 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c004500 con 0x7f3b44101af0 2024-03-26T18:02:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.399+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4419db50 con 0x7f3b44101af0 2024-03-26T18:02:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.402+0000 7f3b41ffb640 1 -- 172.21.15.105:0/1223054729 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3b3c046030 con 0x7f3b44101af0 2024-03-26T18:02:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b41ffb640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b34003830 0x7f3b34005cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b4dc0d740 1 Processor -- start 2024-03-26T18:02:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b4dc0d740 1 -- start start 2024-03-26T18:02:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b34003830 0x7f3b34005cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee81365d0 0x55fee81369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee813bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fee809b0b0 con 0x55fee81365d0 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fee8043960 con 0x55fee8136fc0 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fee8097060 con 0x55fee813c130 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee813bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b48dbc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee81365d0 0x55fee81369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b495bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee813bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41668/0 (socket says 172.21.15.105:41668) 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b495bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38964/0 (socket says 172.21.15.105:38964) 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.403+0000 7f3b43fff640 1 -- 172.21.15.105:0/3853404490 learned_addr learned my addr 172.21.15.105:0/3853404490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b495bd640 1 -- 172.21.15.105:0/3853404490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 msgr2=0x55fee813bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b495bd640 1 --2- 172.21.15.105:0/3853404490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee813bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b495bd640 1 -- 172.21.15.105:0/3853404490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee81365d0 msgr2=0x55fee81369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b495bd640 1 --2- 172.21.15.105:0/3853404490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee81365d0 0x55fee81369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b495bd640 1 -- 172.21.15.105:0/3853404490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee8055c60 con 0x55fee813c130 2024-03-26T18:02:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b495bd640 1 --2- 172.21.15.105:0/3853404490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813e5c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0eec00 tx=0x7f3b3c08f4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37117e13ded1788 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b2effd640 1 -- 172.21.15.105:0/3853404490 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0bd1d0 con 0x55fee813c130 2024-03-26T18:02:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b34003830 0x7f3b34005cf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3b380058c0 tx=0x7f3b38009bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b41ffb640 1 -- 172.21.15.105:0/1223054729 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f3b3c0f10f0 con 0x7f3b44101af0 2024-03-26T18:02:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b2effd640 1 -- 172.21.15.105:0/3853404490 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b3c035e50 con 0x55fee813c130 2024-03-26T18:02:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.404+0000 7f3b2effd640 1 -- 172.21.15.105:0/3853404490 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0ad830 con 0x55fee813c130 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3853404490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 msgr2=0x55fee813e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3853404490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813e5c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0eec00 tx=0x7f3b3c08f4f0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3853404490 shutdown_connections 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3853404490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813e5c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3853404490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee813bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3853404490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee81365d0 0x55fee81369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3853404490 >> 172.21.15.105:0/3853404490 conn(0x55fee8131b40 msgr2=0x55fee8131f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3853404490 shutdown_connections 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3853404490 wait complete. 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 -- start start 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.405+0000 7f3b4dc0d740 1 Processor -- start 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee81365d0 0x55fee8140060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee81405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee81475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fee813eea0 con 0x55fee813c130 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fee8056db0 con 0x55fee8136fc0 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b48dbc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee81365d0 0x55fee8140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b48dbc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee81365d0 0x55fee8140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38980/0 (socket says 172.21.15.105:38980) 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b48dbc640 1 -- 172.21.15.105:0/4085757613 learned_addr learned my addr 172.21.15.105:0/4085757613 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b43fff640 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee81405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fee813f020 con 0x55fee81365d0 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b495bd640 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee81475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b4dc0d740 10 client.0 _ll_register_callbacks cb 0x55fee805b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b48dbc640 1 -- 172.21.15.105:0/4085757613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 msgr2=0x55fee81405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee81405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b48dbc640 1 -- 172.21.15.105:0/4085757613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 msgr2=0x55fee81475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee81475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b48dbc640 1 -- 172.21.15.105:0/4085757613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee7daf210 con 0x55fee81365d0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.406+0000 7f3b495bd640 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee81475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee81365d0 0x55fee8140060 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800fbb0 tx=0x7f3b3801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6175247e9d8af447 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b23fff640 1 -- 172.21.15.105:0/4085757613 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3801f920 con 0x55fee81365d0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b23fff640 1 -- 172.21.15.105:0/4085757613 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f3b3801fac0 con 0x55fee81365d0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b3c035e50 con 0x55fee81365d0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b23fff640 1 -- 172.21.15.105:0/4085757613 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3b38032050 con 0x55fee81365d0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b4dc0d740 10 client.33769 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fee7e7b620 con 0x55fee81365d0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b23fff640 1 -- 172.21.15.105:0/4085757613 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b3c035e50 con 0x55fee81365d0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b23fff640 1 -- 172.21.15.105:0/4085757613 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f3b38020050 con 0x55fee81365d0 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.407+0000 7f3b4dc0d740 10 client.33769 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.408+0000 7f3b4dc0d740 10 client.33769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.408+0000 7f3b4dc0d740 20 client.33769 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.408+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fee814c400 0x55fee814e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.408+0000 7f3b4dc0d740 4 client.33769 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.408+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fee80a85f0 con 0x55fee814c400 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.408+0000 7f3b43fff640 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fee814c400 0x55fee814e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.408+0000 7f3b43fff640 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fee814c400 0x55fee814e8c0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd07403459f63bc3 server_cookie=99d86a13b6d94f22 in_seq=0 out_seq=0 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.408+0000 7f3b23fff640 10 client.33769 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b23fff640 1 -- 172.21.15.105:0/4085757613 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fee80a85f0 con 0x55fee814c400 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b23fff640 10 client.33769 handle_command_reply: tid=0 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 client.33769 shutdown 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee81365d0 msgr2=0x55fee8140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee81365d0 0x55fee8140060 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800fbb0 tx=0x7f3b3801d000 comp rx=0 tx=0).stop 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 shutdown_connections 2024-03-26T18:02:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fee814c400 0x55fee814e8c0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee81475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee8136fc0 0x55fee81405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/4085757613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee81365d0 0x55fee8140060 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 >> 172.21.15.105:0/4085757613 conn(0x55fee8131b40 msgr2=0x55fee8130df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.410+0000 7f3b23fff640 0 client.33769 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.411+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 shutdown_connections 2024-03-26T18:02:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.411+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/4085757613 wait complete. 2024-03-26T18:02:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.411+0000 7f3b4dc0d740 20 client.33769 trim_cache size 0 max 16384 2024-03-26T18:02:33.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:32 smithi135 ceph-mon[35636]: pgmap v1079: 97 pgs: 97 active+clean; 379 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 3.1 MiB/s wr, 248 op/s 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b4dc0d740 1 Processor -- start 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b4dc0d740 1 -- start start 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8138170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee8136fc0 0x55fee8147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2800e530 con 0x55fee8136fc0 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b28010070 con 0x55fee81365d0 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2800ec00 con 0x55fee813c130 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee8136fc0 0x55fee8147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b495bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b48dbc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8138170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b495bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:38982/0 (socket says 172.21.15.105:38982) 2024-03-26T18:02:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b43fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee8136fc0 0x55fee8147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53540/0 (socket says 172.21.15.105:53540) 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.431+0000 7f3b495bd640 1 -- 172.21.15.105:0/676154379 learned_addr learned my addr 172.21.15.105:0/676154379 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b495bd640 1 -- 172.21.15.105:0/676154379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 msgr2=0x55fee8138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b495bd640 1 --2- 172.21.15.105:0/676154379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8138170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b495bd640 1 -- 172.21.15.105:0/676154379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee8136fc0 msgr2=0x55fee8147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b495bd640 1 --2- 172.21.15.105:0/676154379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee8136fc0 0x55fee8147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b495bd640 1 -- 172.21.15.105:0/676154379 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee81386b0 con 0x55fee813c130 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b495bd640 1 --2- 172.21.15.105:0/676154379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813fc70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c004150 tx=0x7f3b3c0a83d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a029167d551f30 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b2e7fc640 1 -- 172.21.15.105:0/676154379 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0aac30 con 0x55fee813c130 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b2e7fc640 1 -- 172.21.15.105:0/676154379 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b3c035e50 con 0x55fee813c130 2024-03-26T18:02:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.432+0000 7f3b2e7fc640 1 -- 172.21.15.105:0/676154379 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0aae30 con 0x55fee813c130 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/676154379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 msgr2=0x55fee813fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/676154379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813fc70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c004150 tx=0x7f3b3c0a83d0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/676154379 shutdown_connections 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/676154379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee813c130 0x55fee813fc70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/676154379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee8136fc0 0x55fee8147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/676154379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8138170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/676154379 >> 172.21.15.105:0/676154379 conn(0x55fee8131b40 msgr2=0x55fee8137a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/676154379 shutdown_connections 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/676154379 wait complete. 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.433+0000 7f3b4dc0d740 1 -- start start 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b4dc0d740 1 Processor -- start 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee8136fc0 0x55fee8144b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b4dc0d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee8145b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2800f9a0 con 0x55fee813c130 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2800f2d0 con 0x55fee81365d0 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b4dc0d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2800c510 con 0x55fee8136fc0 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b48dbc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b48dbc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:41696/0 (socket says 172.21.15.105:41696) 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b48dbc640 1 -- 172.21.15.105:0/3907138404 learned_addr learned my addr 172.21.15.105:0/3907138404 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b4dc0d740 10 client.0 _ll_register_callbacks cb 0x7f3b38009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b43fff640 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee8136fc0 0x55fee8144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.434+0000 7f3b495bd640 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee8145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b48dbc640 1 -- 172.21.15.105:0/3907138404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee8136fc0 msgr2=0x55fee8144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee8136fc0 0x55fee8144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b48dbc640 1 -- 172.21.15.105:0/3907138404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 msgr2=0x55fee8145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee8145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b48dbc640 1 -- 172.21.15.105:0/3907138404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee8146140 con 0x55fee81365d0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b495bd640 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee8145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b48dbc640 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8144650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3b38011ea0 tx=0x7f3b38021360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=420319a72b072995 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b23fff640 1 -- 172.21.15.105:0/3907138404 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380216f0 con 0x55fee81365d0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b23fff640 1 -- 172.21.15.105:0/3907138404 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f3b380218b0 con 0x55fee81365d0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3907138404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b3c035e50 con 0x55fee81365d0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b23fff640 1 -- 172.21.15.105:0/3907138404 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3b38021ab0 con 0x55fee81365d0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b4dc0d740 10 client.33773 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.435+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3907138404 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fee8147450 con 0x55fee81365d0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.436+0000 7f3b23fff640 1 -- 172.21.15.105:0/3907138404 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b3c035e50 con 0x55fee81365d0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.436+0000 7f3b23fff640 1 -- 172.21.15.105:0/3907138404 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f3b3801f7f0 con 0x55fee81365d0 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.436+0000 7f3b4dc0d740 10 client.33773 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.436+0000 7f3b4dc0d740 10 client.33773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.436+0000 7f3b4dc0d740 20 client.33773 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.436+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fee81b2b40 0x55fee81b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.436+0000 7f3b4dc0d740 4 client.33773 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:02:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.436+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3907138404 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fee80a2940 con 0x55fee81b2b40 2024-03-26T18:02:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.437+0000 7f3b43fff640 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fee81b2b40 0x55fee81b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.437+0000 7f3b43fff640 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fee81b2b40 0x55fee81b4f60 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6848d2b6e8e2411d server_cookie=6ac54aa2523a2e97 in_seq=0 out_seq=0 2024-03-26T18:02:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.437+0000 7f3b23fff640 10 client.33773 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:33.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 88272 7.03 MB/sec execute 556 sec latency 188.625 ms 2024-03-26T18:02:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.619+0000 7f3b23fff640 1 -- 172.21.15.105:0/3907138404 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fee80a2940 con 0x55fee81b2b40 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.619+0000 7f3b23fff640 10 client.33773 handle_command_reply: tid=0 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.619+0000 7f3b4dc0d740 1 client.33773 shutdown 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.619+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3907138404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 msgr2=0x55fee8144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.619+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8144650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3b38011ea0 tx=0x7f3b38021360 comp rx=0 tx=0).stop 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3907138404 shutdown_connections 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fee81b2b40 0x55fee81b4f60 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fee813c130 0x55fee8145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fee8136fc0 0x55fee8144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 1 --2- 172.21.15.105:0/3907138404 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fee81365d0 0x55fee8144650 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3907138404 >> 172.21.15.105:0/3907138404 conn(0x55fee8131b40 msgr2=0x55fee813fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b23fff640 0 client.33773 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3907138404 shutdown_connections 2024-03-26T18:02:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 1 -- 172.21.15.105:0/3907138404 wait complete. 2024-03-26T18:02:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.620+0000 7f3b4dc0d740 20 client.33773 trim_cache size 0 max 16384 2024-03-26T18:02:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:02:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:02:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:02:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:02:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b34003830 msgr2=0x7f3b34005cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b34003830 0x7f3b34005cf0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3b380058c0 tx=0x7f3b38009bd0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 msgr2=0x7f3b4419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 0x7f3b4419cdd0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009bf0 tx=0x7f3b3c035ce0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 shutdown_connections 2024-03-26T18:02:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3b34003830 0x7f3b34005cf0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b44101af0 0x7f3b4419cdd0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b44101130 0x7f3b44195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 --2- 172.21.15.105:0/1223054729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3b44100140 0x7f3b44195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 >> 172.21.15.105:0/1223054729 conn(0x7f3b440fb0a0 msgr2=0x7f3b44104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 shutdown_connections 2024-03-26T18:02:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:33.621+0000 7f3b4b020640 1 -- 172.21.15.105:0/1223054729 wait complete. 2024-03-26T18:02:34.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 88385 7.03 MB/sec execute 557 sec latency 263.092 ms 2024-03-26T18:02:35.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:35 smithi144 ceph-mon[36095]: pgmap v1080: 97 pgs: 97 active+clean; 380 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-03-26T18:02:35.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:35 smithi105 ceph-mon[32280]: pgmap v1080: 97 pgs: 97 active+clean; 380 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-03-26T18:02:35.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:35 smithi135 ceph-mon[35636]: pgmap v1080: 97 pgs: 97 active+clean; 380 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.2 MiB/s wr, 248 op/s 2024-03-26T18:02:35.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 88520 7.03 MB/sec execute 558 sec latency 126.694 ms 2024-03-26T18:02:36.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 88646 7.02 MB/sec execute 559 sec latency 109.546 ms 2024-03-26T18:02:37.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:37 smithi144 ceph-mon[36095]: pgmap v1081: 97 pgs: 97 active+clean; 377 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.0 MiB/s wr, 244 op/s 2024-03-26T18:02:37.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:37 smithi105 ceph-mon[32280]: pgmap v1081: 97 pgs: 97 active+clean; 377 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.0 MiB/s wr, 244 op/s 2024-03-26T18:02:37.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:37 smithi135 ceph-mon[35636]: pgmap v1081: 97 pgs: 97 active+clean; 377 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.0 MiB/s wr, 244 op/s 2024-03-26T18:02:37.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 88777 7.01 MB/sec execute 560 sec latency 129.385 ms 2024-03-26T18:02:38.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 88900 7.01 MB/sec execute 561 sec latency 108.834 ms 2024-03-26T18:02:39.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:39 smithi144 ceph-mon[36095]: pgmap v1082: 97 pgs: 97 active+clean; 374 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-03-26T18:02:39.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:39 smithi105 ceph-mon[32280]: pgmap v1082: 97 pgs: 97 active+clean; 374 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-03-26T18:02:39.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:39 smithi135 ceph-mon[35636]: pgmap v1082: 97 pgs: 97 active+clean; 374 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-03-26T18:02:39.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 89031 7.01 MB/sec execute 562 sec latency 128.143 ms 2024-03-26T18:02:40.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 89173 7.00 MB/sec execute 563 sec latency 104.898 ms 2024-03-26T18:02:41.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:41 smithi144 ceph-mon[36095]: pgmap v1083: 97 pgs: 97 active+clean; 376 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-03-26T18:02:41.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:41 smithi105 ceph-mon[32280]: pgmap v1083: 97 pgs: 97 active+clean; 376 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-03-26T18:02:41.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:41 smithi135 ceph-mon[35636]: pgmap v1083: 97 pgs: 97 active+clean; 376 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-03-26T18:02:41.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 89330 7.01 MB/sec execute 564 sec latency 107.245 ms 2024-03-26T18:02:42.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:02:42.140 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:02:42.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 -- 172.21.15.105:0/4232079328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 msgr2=0x7fcdc4107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/4232079328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 0x7fcdc4107120 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009900 tx=0x7fcdb802bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 -- 172.21.15.105:0/4232079328 shutdown_connections 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/4232079328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc4107660 0x7fcdc4109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/4232079328 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 0x7fcdc4107120 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/4232079328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdc4102400 0x7fcdc41047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 -- 172.21.15.105:0/4232079328 >> 172.21.15.105:0/4232079328 conn(0x7fcdc40fbd70 msgr2=0x7fcdc40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 -- 172.21.15.105:0/4232079328 shutdown_connections 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.307+0000 7fcdc96c6640 1 -- 172.21.15.105:0/4232079328 wait complete. 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.308+0000 7fcdc96c6640 1 Processor -- start 2024-03-26T18:02:42.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.308+0000 7fcdc96c6640 1 -- start start 2024-03-26T18:02:42.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.308+0000 7fcdc96c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc4102400 0x7fcdc4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.308+0000 7fcdc96c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 0x7fcdc4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc96c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdc4107660 0x7fcdc419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc96c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc410d480 con 0x7fcdc4104d30 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc96c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc410d300 con 0x7fcdc4107660 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc96c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc410d600 con 0x7fcdc4102400 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc4102400 0x7fcdc4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 0x7fcdc4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc2ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc4102400 0x7fcdc4197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47274/0 (socket says 172.21.15.105:47274) 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/2503876638 learned_addr learned my addr 172.21.15.105:0/2503876638 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdc4107660 0x7fcdc419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc27fc640 1 -- 172.21.15.105:0/2503876638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc4102400 msgr2=0x7fcdc4197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc27fc640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc4102400 0x7fcdc4197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc27fc640 1 -- 172.21.15.105:0/2503876638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdc4107660 msgr2=0x7fcdc419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc27fc640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdc4107660 0x7fcdc419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc27fc640 1 -- 172.21.15.105:0/2503876638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc419f7e0 con 0x7fcdc4104d30 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc4102400 0x7fcdc4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:42.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdc4107660 0x7fcdc419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:42.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc27fc640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 0x7fcdc4198100 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8036040 tx=0x7fcdb8005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:42.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdabfff640 1 -- 172.21.15.105:0/2503876638 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8043020 con 0x7fcdc4104d30 2024-03-26T18:02:42.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc96c6640 1 -- 172.21.15.105:0/2503876638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc419fa70 con 0x7fcdc4104d30 2024-03-26T18:02:42.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdabfff640 1 -- 172.21.15.105:0/2503876638 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdb8004350 con 0x7fcdc4104d30 2024-03-26T18:02:42.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdabfff640 1 -- 172.21.15.105:0/2503876638 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8003a50 con 0x7fcdc4104d30 2024-03-26T18:02:42.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.309+0000 7fcdc96c6640 1 -- 172.21.15.105:0/2503876638 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc419ffe0 con 0x7fcdc4104d30 2024-03-26T18:02:42.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdabfff640 1 -- 172.21.15.105:0/2503876638 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fcdb8046020 con 0x7fcdc4104d30 2024-03-26T18:02:42.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdcc38c740 1 Processor -- start 2024-03-26T18:02:42.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdcc38c740 1 -- start start 2024-03-26T18:02:42.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f0651d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f0656de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdabfff640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdac0a2f20 0x7fcdac0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5643f05b6980 con 0x5643f06522e0 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5643f055ebe0 con 0x5643f0657450 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.312+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5643f05b2930 con 0x5643f06518f0 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f0656de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f0656de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48580/0 (socket says 172.21.15.105:48580) 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/3987677064 learned_addr learned my addr 172.21.15.105:0/3987677064 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc27fc640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdac0a2f20 0x7fcdac0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f0651d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdabfff640 1 -- 172.21.15.105:0/2503876638 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fcdb80f1490 con 0x7fcdc4104d30 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/3987677064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 msgr2=0x5643f0651d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f0651d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/3987677064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 msgr2=0x5643f06598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/3987677064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f03947d0 con 0x5643f06522e0 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f0656de0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4011650 tx=0x7fcdb4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d71e6b54bfbf0d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcdc27fc640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdac0a2f20 0x7fcdac0a53e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80bd910 tx=0x7fcdb803ae30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcda97fa640 1 -- 172.21.15.105:0/3987677064 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb401e070 con 0x5643f06522e0 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcda97fa640 1 -- 172.21.15.105:0/3987677064 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdb400fb70 con 0x5643f06522e0 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.313+0000 7fcda97fa640 1 -- 172.21.15.105:0/3987677064 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4019c30 con 0x5643f06522e0 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 -- 172.21.15.105:0/3987677064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 msgr2=0x5643f0656de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f0656de0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4011650 tx=0x7fcdb4009f90 comp rx=0 tx=0).stop 2024-03-26T18:02:42.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 -- 172.21.15.105:0/3987677064 shutdown_connections 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f0656de0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/3987677064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f0651d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 -- 172.21.15.105:0/3987677064 >> 172.21.15.105:0/3987677064 conn(0x5643f064ce60 msgr2=0x5643f064d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 -- 172.21.15.105:0/3987677064 shutdown_connections 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 -- 172.21.15.105:0/3987677064 wait complete. 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 -- start start 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.314+0000 7fcdcc38c740 1 Processor -- start 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f065bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f065c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5643f065a400 con 0x5643f06522e0 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5643f0572160 con 0x5643f0657450 2024-03-26T18:02:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5643f065a580 con 0x5643f06518f0 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdcc38c740 10 client.0 _ll_register_callbacks cb 0x5643f05ae720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f065c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f065bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f065c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48592/0 (socket says 172.21.15.105:48592) 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/900620936 learned_addr learned my addr 172.21.15.105:0/900620936 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc37fe640 1 -- 172.21.15.105:0/900620936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 msgr2=0x5643f06630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc37fe640 1 -- 172.21.15.105:0/900620936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 msgr2=0x5643f065c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f065c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc37fe640 1 -- 172.21.15.105:0/900620936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f02ca3f0 con 0x5643f06518f0 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f065c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:42.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f065bb70 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005c40 tx=0x7fcdbc009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f4c12287949618e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.315+0000 7fcda6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcda6ff5640 1 -- 172.21.15.105:0/900620936 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc027060 con 0x5643f06518f0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 1 -- 172.21.15.105:0/900620936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb400fb70 con 0x5643f06518f0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcda6ff5640 1 -- 172.21.15.105:0/900620936 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fcdbc00ebb0 con 0x5643f06518f0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcda6ff5640 1 -- 172.21.15.105:0/900620936 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcdbc00edb0 con 0x5643f06518f0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 10 client.33787 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 1 -- 172.21.15.105:0/900620936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5643f028a0f0 con 0x5643f06518f0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcda6ff5640 1 -- 172.21.15.105:0/900620936 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdb400fb70 con 0x5643f06518f0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcda6ff5640 1 -- 172.21.15.105:0/900620936 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fcdbc020040 con 0x5643f06518f0 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 10 client.33787 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 10 client.33787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 20 client.33787 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5643f0667970 0x5643f0669de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 4 client.33787 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.316+0000 7fcdcc38c740 1 -- 172.21.15.105:0/900620936 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643f05c3ec0 con 0x5643f0667970 2024-03-26T18:02:42.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.317+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5643f0667970 0x5643f0669de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.317+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5643f0667970 0x5643f0669de0 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d629b84bc1619623 server_cookie=eba779a3d1abd944 in_seq=0 out_seq=0 2024-03-26T18:02:42.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.317+0000 7fcda6ff5640 10 client.33787 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcda6ff5640 1 -- 172.21.15.105:0/900620936 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5643f05c3ec0 con 0x5643f0667970 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcda6ff5640 10 client.33787 handle_command_reply: tid=0 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 client.33787 shutdown 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 -- 172.21.15.105:0/900620936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 msgr2=0x5643f065bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f065bb70 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005c40 tx=0x7fcdbc009500 comp rx=0 tx=0).stop 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 -- 172.21.15.105:0/900620936 shutdown_connections 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5643f0667970 0x5643f0669de0 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06522e0 0x5643f065c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/900620936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06518f0 0x5643f065bb70 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 -- 172.21.15.105:0/900620936 >> 172.21.15.105:0/900620936 conn(0x5643f064ce60 msgr2=0x5643f064c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcda6ff5640 0 client.33787 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:42.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.319+0000 7fcdcc38c740 1 -- 172.21.15.105:0/900620936 shutdown_connections 2024-03-26T18:02:42.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.320+0000 7fcdcc38c740 1 -- 172.21.15.105:0/900620936 wait complete. 2024-03-26T18:02:42.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.320+0000 7fcdcc38c740 20 client.33787 trim_cache size 0 max 16384 2024-03-26T18:02:42.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.346+0000 7fcdcc38c740 1 Processor -- start 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.346+0000 7fcdcc38c740 1 -- start start 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.346+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f0656b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.346+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f06522e0 0x5643f0663080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.346+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f0657450 0x5643f065b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.346+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41a8e50 con 0x5643f06518f0 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.346+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41aa990 con 0x5643f06522e0 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41a9520 con 0x5643f0657450 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f0656b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f06522e0 0x5643f0663080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f06522e0 0x5643f0663080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:60960/0 (socket says 172.21.15.105:60960) 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f0656b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48594/0 (socket says 172.21.15.105:48594) 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/2122230255 learned_addr learned my addr 172.21.15.105:0/2122230255 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:42.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f0657450 0x5643f065b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/2122230255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f0657450 msgr2=0x5643f065b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f0657450 0x5643f065b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/2122230255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 msgr2=0x5643f0656b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f0656b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 -- 172.21.15.105:0/2122230255 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f06663a0 con 0x5643f06522e0 2024-03-26T18:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f0656b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:42.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f06522e0 0x5643f0663080 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4013fc0 tx=0x7fcdb4009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74579198f0d320cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.347+0000 7fcda8ff9640 1 -- 172.21.15.105:0/2122230255 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb401e040 con 0x5643f06522e0 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcda8ff9640 1 -- 172.21.15.105:0/2122230255 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdb402a740 con 0x5643f06522e0 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcda8ff9640 1 -- 172.21.15.105:0/2122230255 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4005590 con 0x5643f06522e0 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcdcc38c740 1 -- 172.21.15.105:0/2122230255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f06522e0 msgr2=0x5643f0663080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f06522e0 0x5643f0663080 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4013fc0 tx=0x7fcdb4009df0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcdcc38c740 1 -- 172.21.15.105:0/2122230255 shutdown_connections 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f0657450 0x5643f065b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f06522e0 0x5643f0663080 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/2122230255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f0656b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcdcc38c740 1 -- 172.21.15.105:0/2122230255 >> 172.21.15.105:0/2122230255 conn(0x5643f064ce60 msgr2=0x5643f06529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.348+0000 7fcdcc38c740 1 -- 172.21.15.105:0/2122230255 shutdown_connections 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 -- 172.21.15.105:0/2122230255 wait complete. 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 -- start start 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 Processor -- start 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f065ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06522e0 0x5643f06604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41aa2c0 con 0x5643f06518f0 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41a9bf0 con 0x5643f0657450 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc41a6e30 con 0x5643f06522e0 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdc37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f065ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdcc38c740 10 client.0 _ll_register_callbacks cb 0x7fcdbc011950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:42.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdc37fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f065ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48598/0 (socket says 172.21.15.105:48598) 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdc37fe640 1 -- 172.21.15.105:0/1396148062 learned_addr learned my addr 172.21.15.105:0/1396148062 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06522e0 0x5643f06604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.349+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdc37fe640 1 -- 172.21.15.105:0/1396148062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06522e0 msgr2=0x5643f06604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06522e0 0x5643f06604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdc37fe640 1 -- 172.21.15.105:0/1396148062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 msgr2=0x5643f06cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdc37fe640 1 -- 172.21.15.105:0/1396148062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f06cc210 con 0x5643f06518f0 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdc37fe640 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f065ffb0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009e20 tx=0x7fcdbc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f881b65635d0c98c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcda6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcda6ff5640 1 -- 172.21.15.105:0/1396148062 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc027040 con 0x5643f06518f0 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcda6ff5640 1 -- 172.21.15.105:0/1396148062 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fcdbc011040 con 0x5643f06518f0 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdcc38c740 1 -- 172.21.15.105:0/1396148062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb402a740 con 0x5643f06518f0 2024-03-26T18:02:42.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcda6ff5640 1 -- 172.21.15.105:0/1396148062 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcdbc011240 con 0x5643f06518f0 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdcc38c740 10 client.24126 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.350+0000 7fcdcc38c740 1 -- 172.21.15.105:0/1396148062 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5643f06cd460 con 0x5643f06518f0 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcda6ff5640 1 -- 172.21.15.105:0/1396148062 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdb402a740 con 0x5643f06518f0 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcda6ff5640 1 -- 172.21.15.105:0/1396148062 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fcdbc004460 con 0x5643f06518f0 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcdcc38c740 10 client.24126 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcdcc38c740 10 client.24126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcdcc38c740 20 client.24126 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5643f06cdd90 0x5643f06d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcdcc38c740 4 client.24126 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcdcc38c740 1 -- 172.21.15.105:0/1396148062 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643f05be210 con 0x5643f06cdd90 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.351+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5643f06cdd90 0x5643f06d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.352+0000 7fcda6ff5640 10 client.24126 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:42.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.352+0000 7fcdc2ffd640 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5643f06cdd90 0x5643f06d01b0 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b868f4890394b68 server_cookie=b001f325dfff7cb7 in_seq=0 out_seq=0 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.354+0000 7fcda6ff5640 1 -- 172.21.15.105:0/1396148062 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5643f05be210 con 0x5643f06cdd90 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.354+0000 7fcda6ff5640 10 client.24126 handle_command_reply: tid=0 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.354+0000 7fcdcc38c740 1 client.24126 shutdown 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 -- 172.21.15.105:0/1396148062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 msgr2=0x5643f065ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f065ffb0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009e20 tx=0x7fcdbc005fd0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 -- 172.21.15.105:0/1396148062 shutdown_connections 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5643f06cdd90 0x5643f06d01b0 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5643f0657450 0x5643f06cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643f06522e0 0x5643f06604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 --2- 172.21.15.105:0/1396148062 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5643f06518f0 0x5643f065ffb0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcda6ff5640 0 client.24126 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 -- 172.21.15.105:0/1396148062 >> 172.21.15.105:0/1396148062 conn(0x5643f064ce60 msgr2=0x5643f0653500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 -- 172.21.15.105:0/1396148062 shutdown_connections 2024-03-26T18:02:42.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 1 -- 172.21.15.105:0/1396148062 wait complete. 2024-03-26T18:02:42.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.355+0000 7fcdcc38c740 20 client.24126 trim_cache size 0 max 16384 2024-03-26T18:02:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:02:42.339 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T18:02:42.339 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T18:02:42.339 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 -- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdac0a2f20 msgr2=0x7fcdac0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdac0a2f20 0x7fcdac0a53e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80bd910 tx=0x7fcdb803ae30 comp rx=0 tx=0).stop 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 -- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 msgr2=0x7fcdc4198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 0x7fcdc4198100 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8036040 tx=0x7fcdb8005df0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 -- 172.21.15.105:0/2503876638 shutdown_connections 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fcdac0a2f20 0x7fcdac0a53e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcdc4107660 0x7fcdc419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcdc4104d30 0x7fcdc4198100 secure :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8036040 tx=0x7fcdb8005df0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 --2- 172.21.15.105:0/2503876638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdc4102400 0x7fcdc4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 -- 172.21.15.105:0/2503876638 >> 172.21.15.105:0/2503876638 conn(0x7fcdc40fbd70 msgr2=0x7fcdc4104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 -- 172.21.15.105:0/2503876638 shutdown_connections 2024-03-26T18:02:42.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.356+0000 7fcdc96c6640 1 -- 172.21.15.105:0/2503876638 wait complete. 2024-03-26T18:02:42.350 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T18:02:42.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:02:42.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 -- 172.21.15.105:0/2753663477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100ab0 msgr2=0x7fe26c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 --2- 172.21.15.105:0/2753663477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100ab0 0x7fe26c100f10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe260009860 tx=0x7fe26002bb70 comp rx=0 tx=0).stop 2024-03-26T18:02:42.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 -- 172.21.15.105:0/2753663477 shutdown_connections 2024-03-26T18:02:42.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 --2- 172.21.15.105:0/2753663477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe26c101450 0x7fe26c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 --2- 172.21.15.105:0/2753663477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100ab0 0x7fe26c100f10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 --2- 172.21.15.105:0/2753663477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26c100150 0x7fe26c100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 -- 172.21.15.105:0/2753663477 >> 172.21.15.105:0/2753663477 conn(0x7fe26c0fbd50 msgr2=0x7fe26c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 -- 172.21.15.105:0/2753663477 shutdown_connections 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 -- 172.21.15.105:0/2753663477 wait complete. 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.514+0000 7fe272177640 1 Processor -- start 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe272177640 1 -- start start 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe272177640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100150 0x7fe26c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe272177640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26c100ab0 0x7fe26c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe272177640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe26c101450 0x7fe26c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe272177640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c10f5d0 con 0x7fe26c100ab0 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe272177640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c10f450 con 0x7fe26c101450 2024-03-26T18:02:42.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe272177640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c10f750 con 0x7fe26c100150 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100150 0x7fe26c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26b7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100150 0x7fe26c199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47344/0 (socket says 172.21.15.105:47344) 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26b7fe640 1 -- 172.21.15.105:0/3148303841 learned_addr learned my addr 172.21.15.105:0/3148303841 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe26c101450 0x7fe26c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26affd640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26c100ab0 0x7fe26c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26affd640 1 -- 172.21.15.105:0/3148303841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100150 msgr2=0x7fe26c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26affd640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100150 0x7fe26c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26affd640 1 -- 172.21.15.105:0/3148303841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe26c101450 msgr2=0x7fe26c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26affd640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe26c101450 0x7fe26c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26affd640 1 -- 172.21.15.105:0/3148303841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c1a1930 con 0x7fe26c100ab0 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26b7fe640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100150 0x7fe26c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.515+0000 7fe26bfff640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe26c101450 0x7fe26c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.516+0000 7fe26affd640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26c100ab0 0x7fe26c19a260 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fe260037040 tx=0x7fe2600040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.516+0000 7fe268ff9640 1 -- 172.21.15.105:0/3148303841 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260041020 con 0x7fe26c100ab0 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.516+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c1a1bc0 con 0x7fe26c100ab0 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.516+0000 7fe268ff9640 1 -- 172.21.15.105:0/3148303841 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2600044d0 con 0x7fe26c100ab0 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.516+0000 7fe268ff9640 1 -- 172.21.15.105:0/3148303841 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260039040 con 0x7fe26c100ab0 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.516+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c1a2100 con 0x7fe26c100ab0 2024-03-26T18:02:42.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.517+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c10b120 con 0x7fe26c100ab0 2024-03-26T18:02:42.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.519+0000 7fe268ff9640 1 -- 172.21.15.105:0/3148303841 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe260044020 con 0x7fe26c100ab0 2024-03-26T18:02:42.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.519+0000 7fe268ff9640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2540a2f20 0x7fe2540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:42.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.519+0000 7fe26b7fe640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2540a2f20 0x7fe2540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:42.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.519+0000 7fe268ff9640 1 -- 172.21.15.105:0/3148303841 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fe2600f0cd0 con 0x7fe26c100ab0 2024-03-26T18:02:42.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.520+0000 7fe26b7fe640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2540a2f20 0x7fe2540a53e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe26c19b1f0 tx=0x7fe25c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:42.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.527+0000 7fe268ff9640 1 -- 172.21.15.105:0/3148303841 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe2600f6020 con 0x7fe26c100ab0 2024-03-26T18:02:42.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 89476 7.01 MB/sec execute 565 sec latency 134.467 ms 2024-03-26T18:02:42.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.862+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe26c0624c0 con 0x7fe26c100ab0 2024-03-26T18:02:42.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.863+0000 7fe268ff9640 1 -- 172.21.15.105:0/3148303841 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11941 (secure 0 0 0) 0x7fe26004b020 con 0x7fe26c100ab0 2024-03-26T18:02:42.844 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:02:42.844 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:02:08.013819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_33599":{"gid":33599,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1455536711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1455536711},{"type":"v1","addr":"172.21.15.135:6835","nonce":1455536711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T18:02:42.844 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 45 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2540a2f20 msgr2=0x7fe2540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2540a2f20 0x7fe2540a53e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe26c19b1f0 tx=0x7fe25c009260 comp rx=0 tx=0).stop 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26c100ab0 msgr2=0x7fe26c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26c100ab0 0x7fe26c19a260 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fe260037040 tx=0x7fe2600040e0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 shutdown_connections 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe2540a2f20 0x7fe2540a53e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe26c101450 0x7fe26c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe26c100ab0 0x7fe26c19a260 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 --2- 172.21.15.105:0/3148303841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26c100150 0x7fe26c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 >> 172.21.15.105:0/3148303841 conn(0x7fe26c0fbd50 msgr2=0x7fe26c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 shutdown_connections 2024-03-26T18:02:42.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:42.865+0000 7fe272177640 1 -- 172.21.15.105:0/3148303841 wait complete. 2024-03-26T18:02:42.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T18:02:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 -- 172.21.15.105:0/3697191038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c0102370 msgr2=0x7fd5c0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3697191038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c0102370 0x7fd5c0108c00 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009ca0 tx=0x7fd5b802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 -- 172.21.15.105:0/3697191038 shutdown_connections 2024-03-26T18:02:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3697191038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c0102370 0x7fd5c0108c00 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3697191038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5c01019b0 0x7fd5c0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3697191038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c01007c0 0x7fd5c0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 -- 172.21.15.105:0/3697191038 >> 172.21.15.105:0/3697191038 conn(0x7fd5c00fbd50 msgr2=0x7fd5c00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:42.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 -- 172.21.15.105:0/3697191038 shutdown_connections 2024-03-26T18:02:42.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.017+0000 7fd5c4987640 1 -- 172.21.15.105:0/3697191038 wait complete. 2024-03-26T18:02:43.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5c4987640 1 Processor -- start 2024-03-26T18:02:43.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5c4987640 1 -- start start 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5c4987640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5c01007c0 0x7fd5c01959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5c4987640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c01019b0 0x7fd5c0195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5c4987640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c0102370 0x7fd5c019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5c4987640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010c2f0 con 0x7fd5c01019b0 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5c4987640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010c170 con 0x7fd5c01007c0 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5c4987640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c010c470 con 0x7fd5c0102370 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c01019b0 0x7fd5c0195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5be59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5c01007c0 0x7fd5c01959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.018+0000 7fd5bed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c0102370 0x7fd5c019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c01019b0 0x7fd5c0195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48646/0 (socket says 172.21.15.105:48646) 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/3139960502 learned_addr learned my addr 172.21.15.105:0/3139960502 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5bed9d640 1 -- 172.21.15.105:0/3139960502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5c01007c0 msgr2=0x7fd5c01959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5bed9d640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5c01007c0 0x7fd5c01959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5bed9d640 1 -- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c01019b0 msgr2=0x7fd5c0195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5bed9d640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c01019b0 0x7fd5c0195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5bed9d640 1 -- 172.21.15.105:0/3139960502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c019d5c0 con 0x7fd5c0102370 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5bed9d640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c0102370 0x7fd5c019cf50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd5b802c630 tx=0x7fd5b80328a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5b7fff640 1 -- 172.21.15.105:0/3139960502 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8041020 con 0x7fd5c0102370 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5c4987640 1 -- 172.21.15.105:0/3139960502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c019d850 con 0x7fd5c0102370 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5b7fff640 1 -- 172.21.15.105:0/3139960502 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5b8003e20 con 0x7fd5c0102370 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5b7fff640 1 -- 172.21.15.105:0/3139960502 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8039040 con 0x7fd5c0102370 2024-03-26T18:02:43.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.019+0000 7fd5c4987640 1 -- 172.21.15.105:0/3139960502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c019dd90 con 0x7fd5c0102370 2024-03-26T18:02:43.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.022+0000 7fd5b7fff640 1 -- 172.21.15.105:0/3139960502 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd5b8044020 con 0x7fd5c0102370 2024-03-26T18:02:43.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5c7570740 1 Processor -- start 2024-03-26T18:02:43.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5c7570740 1 -- start start 2024-03-26T18:02:43.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b32400ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b324011f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 0x55b324014a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5c7570740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b323f71970 con 0x55b32400ca30 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5c7570740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b323f19d70 con 0x55b32400d420 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5c7570740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b323f6d920 con 0x55b324012590 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5b7fff640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5ac0a2fa0 0x7fd5ac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b32400ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b32400ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48662/0 (socket says 172.21.15.105:48662) 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.023+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/2961139907 learned_addr learned my addr 172.21.15.105:0/2961139907 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5bed9d640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5ac0a2fa0 0x7fd5ac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5be59c640 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b324011f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b7fff640 1 -- 172.21.15.105:0/3139960502 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fd5b80f14e0 con 0x7fd5c0102370 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/2961139907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 msgr2=0x55b324014a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 0x55b324014a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/2961139907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 msgr2=0x55b324011f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b324011f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/2961139907 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b323c853f0 con 0x55b32400ca30 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5be59c640 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b324011f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b32400ce50 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009d30 tx=0x7fd5a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43ee2942821f072 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5bed9d640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5ac0a2fa0 0x7fd5ac0a5460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8006010 tx=0x7fd5b804bbd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b4d99640 1 -- 172.21.15.105:0/2961139907 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8004030 con 0x55b32400ca30 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b4d99640 1 -- 172.21.15.105:0/2961139907 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5a80054d0 con 0x55b32400ca30 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.024+0000 7fd5b4d99640 1 -- 172.21.15.105:0/2961139907 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8004b20 con 0x55b32400ca30 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 -- 172.21.15.105:0/2961139907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 msgr2=0x55b32400ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b32400ce50 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009d30 tx=0x7fd5a8011810 comp rx=0 tx=0).stop 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 -- 172.21.15.105:0/2961139907 shutdown_connections 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 0x55b324014a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b324011f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 --2- 172.21.15.105:0/2961139907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b32400ce50 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 -- 172.21.15.105:0/2961139907 >> 172.21.15.105:0/2961139907 conn(0x55b324007fa0 msgr2=0x55b3240083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 -- 172.21.15.105:0/2961139907 shutdown_connections 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.025+0000 7fd5c7570740 1 -- 172.21.15.105:0/2961139907 wait complete. 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 1 -- start start 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 1 Processor -- start 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400ca30 0x55b324016fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400d420 0x55b324017510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b324012590 0x55b32401e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b324015540 con 0x55b32400d420 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b323f2d310 con 0x55b324012590 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3240156c0 con 0x55b32400ca30 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5c7570740 10 client.0 _ll_register_callbacks cb 0x55b323f68c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400ca30 0x55b324016fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400ca30 0x55b324016fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47382/0 (socket says 172.21.15.105:47382) 2024-03-26T18:02:43.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/438233705 learned_addr learned my addr 172.21.15.105:0/438233705 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b324012590 0x55b32401e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.026+0000 7fd5be59c640 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400d420 0x55b324017510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/438233705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b324012590 msgr2=0x55b32401e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b324012590 0x55b32401e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/438233705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400d420 msgr2=0x55b324017510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400d420 0x55b324017510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/438233705 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b323d4f7d0 con 0x55b32400ca30 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400ca30 0x55b324016fd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8005490 tx=0x7fd5a8005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de98209230ce0c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/438233705 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8028450 con 0x55b32400ca30 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/438233705 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fd5a80285f0 con 0x55b32400ca30 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.027+0000 7fd5c7570740 1 -- 172.21.15.105:0/438233705 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5a80054d0 con 0x55b32400ca30 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.028+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/438233705 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd5a8028a00 con 0x55b32400ca30 2024-03-26T18:02:43.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.028+0000 7fd5c7570740 10 client.33808 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.028+0000 7fd5c7570740 1 -- 172.21.15.105:0/438233705 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b3240218c0 con 0x55b32400ca30 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.028+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/438233705 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5a80054d0 con 0x55b32400ca30 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.028+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/438233705 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fd5a8019b60 con 0x55b32400ca30 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5c7570740 10 client.33808 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5c7570740 10 client.33808 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5c7570740 20 client.33808 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5c7570740 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b324022de0 0x55b3240251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5c7570740 4 client.33808 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5c7570740 1 -- 172.21.15.105:0/438233705 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b323f7eeb0 con 0x55b324022de0 2024-03-26T18:02:43.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5be59c640 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b324022de0 0x55b3240251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5be59c640 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b324022de0 0x55b3240251f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=663f21d95c9384d9 server_cookie=e9ffe7e217a56f7d in_seq=0 out_seq=0 2024-03-26T18:02:43.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.029+0000 7fd5a5ffb640 10 client.33808 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/438233705 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b323f7eeb0 con 0x55b324022de0 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5a5ffb640 10 client.33808 handle_command_reply: tid=0 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 client.33808 shutdown 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 -- 172.21.15.105:0/438233705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400ca30 msgr2=0x55b324016fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400ca30 0x55b324016fd0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8005490 tx=0x7fd5a8005d30 comp rx=0 tx=0).stop 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 -- 172.21.15.105:0/438233705 shutdown_connections 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b324022de0 0x55b3240251f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b324012590 0x55b32401e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400d420 0x55b324017510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5a5ffb640 0 client.33808 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 --2- 172.21.15.105:0/438233705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400ca30 0x55b324016fd0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 -- 172.21.15.105:0/438233705 >> 172.21.15.105:0/438233705 conn(0x55b324007fa0 msgr2=0x55b3240071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.030+0000 7fd5c7570740 1 -- 172.21.15.105:0/438233705 shutdown_connections 2024-03-26T18:02:43.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.031+0000 7fd5c7570740 1 -- 172.21.15.105:0/438233705 wait complete. 2024-03-26T18:02:43.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.031+0000 7fd5c7570740 20 client.33808 trim_cache size 0 max 16384 2024-03-26T18:02:43.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.049+0000 7fd5c7570740 1 Processor -- start 2024-03-26T18:02:43.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.049+0000 7fd5c7570740 1 -- start start 2024-03-26T18:02:43.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.049+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b3240172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.049+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b32401e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.049+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 0x55b324015a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5c7570740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b7b00 con 0x55b32400ca30 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5c7570740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b9640 con 0x55b32400d420 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5c7570740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b81d0 con 0x55b324012590 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b32401e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b3240172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b32401e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32772/0 (socket says 172.21.15.105:32772) 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 -- 172.21.15.105:0/231161205 learned_addr learned my addr 172.21.15.105:0/231161205 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b3240172f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48678/0 (socket says 172.21.15.105:48678) 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 0x55b324015a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 -- 172.21.15.105:0/231161205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 msgr2=0x55b324015a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 0x55b324015a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 -- 172.21.15.105:0/231161205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 msgr2=0x55b3240172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b3240172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 -- 172.21.15.105:0/231161205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b324008d10 con 0x55b32400d420 2024-03-26T18:02:43.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b3240172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5be59c640 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b32401e4e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x55b32400d820 tx=0x7fd5b0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7afa79c9b8bd246 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.050+0000 7fd5a7fff640 1 -- 172.21.15.105:0/231161205 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0004250 con 0x55b32400d420 2024-03-26T18:02:43.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5a7fff640 1 -- 172.21.15.105:0/231161205 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5b00043f0 con 0x55b32400d420 2024-03-26T18:02:43.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5a7fff640 1 -- 172.21.15.105:0/231161205 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0020110 con 0x55b32400d420 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5c7570740 1 -- 172.21.15.105:0/231161205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 msgr2=0x55b32401e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5c7570740 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b32401e4e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x55b32400d820 tx=0x7fd5b0005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5c7570740 1 -- 172.21.15.105:0/231161205 shutdown_connections 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5c7570740 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b324012590 0x55b324015a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5c7570740 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400d420 0x55b32401e4e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5c7570740 1 --2- 172.21.15.105:0/231161205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b32400ca30 0x55b3240172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.051+0000 7fd5c7570740 1 -- 172.21.15.105:0/231161205 >> 172.21.15.105:0/231161205 conn(0x55b324007fa0 msgr2=0x55b324016be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.052+0000 7fd5c7570740 1 -- 172.21.15.105:0/231161205 shutdown_connections 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.052+0000 7fd5c7570740 1 -- 172.21.15.105:0/231161205 wait complete. 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.052+0000 7fd5c7570740 1 -- start start 2024-03-26T18:02:43.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.052+0000 7fd5c7570740 1 Processor -- start 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400ca30 0x55b32401e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400d420 0x55b32401ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5c7570740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b324012590 0x55b32401b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5c7570740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b8f70 con 0x55b324012590 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5c7570740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b88a0 con 0x55b32400ca30 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5c7570740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b5b90 con 0x55b32400d420 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5c7570740 10 client.0 _ll_register_callbacks cb 0x7fd5a8009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400ca30 0x55b32401e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400ca30 0x55b32401e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32782/0 (socket says 172.21.15.105:32782) 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/535550636 learned_addr learned my addr 172.21.15.105:0/535550636 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b324012590 0x55b32401b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5be59c640 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400d420 0x55b32401ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/535550636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400d420 msgr2=0x55b32401ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400d420 0x55b32401ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/535550636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400ca30 msgr2=0x55b32401e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400ca30 0x55b32401e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.053+0000 7fd5b5d9b640 1 -- 172.21.15.105:0/535550636 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3240870e0 con 0x55b324012590 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.054+0000 7fd5b5d9b640 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b324012590 0x55b32401b2b0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800a010 tx=0x7fd5a8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5354e602083edea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.054+0000 7fd5a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.054+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/535550636 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a80277e0 con 0x55b324012590 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.054+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/535550636 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fd5a8028ba0 con 0x55b324012590 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.054+0000 7fd5c7570740 1 -- 172.21.15.105:0/535550636 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b00043f0 con 0x55b324012590 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.054+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/535550636 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd5a8028da0 con 0x55b324012590 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.054+0000 7fd5c7570740 10 client.24150 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.054+0000 7fd5c7570740 1 -- 172.21.15.105:0/535550636 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b324088190 con 0x55b324012590 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/535550636 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5b00043f0 con 0x55b324012590 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/535550636 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fd5a804a020 con 0x55b324012590 2024-03-26T18:02:43.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5c7570740 10 client.24150 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5c7570740 10 client.24150 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5c7570740 20 client.24150 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5c7570740 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b324088ad0 0x55b32408aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5c7570740 4 client.24150 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5c7570740 1 -- 172.21.15.105:0/535550636 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b323f79200 con 0x55b324088ad0 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5be59c640 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b324088ad0 0x55b32408aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5be59c640 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b324088ad0 0x55b32408aef0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bfd5983940a63f16 server_cookie=e215bc4d79a7d0f2 in_seq=0 out_seq=0 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.055+0000 7fd5a5ffb640 10 client.24150 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5a5ffb640 1 -- 172.21.15.105:0/535550636 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b323f79200 con 0x55b324088ad0 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5a5ffb640 10 client.24150 handle_command_reply: tid=0 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 client.24150 shutdown 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 -- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b324012590 msgr2=0x55b32401b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b324012590 0x55b32401b2b0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800a010 tx=0x7fd5a8005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:43.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 -- 172.21.15.105:0/535550636 shutdown_connections 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x55b324088ad0 0x55b32408aef0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b324012590 0x55b32401b2b0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5a5ffb640 0 client.24150 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b32400d420 0x55b32401ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 --2- 172.21.15.105:0/535550636 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b32400ca30 0x55b32401e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 -- 172.21.15.105:0/535550636 >> 172.21.15.105:0/535550636 conn(0x55b324007fa0 msgr2=0x55b32400b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 -- 172.21.15.105:0/535550636 shutdown_connections 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 1 -- 172.21.15.105:0/535550636 wait complete. 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.056+0000 7fd5c7570740 20 client.24150 trim_cache size 0 max 16384 2024-03-26T18:02:43.042 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:43.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 -- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5ac0a2fa0 msgr2=0x7fd5ac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5ac0a2fa0 0x7fd5ac0a5460 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8006010 tx=0x7fd5b804bbd0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 -- 172.21.15.105:0/3139960502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c0102370 msgr2=0x7fd5c019cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c0102370 0x7fd5c019cf50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd5b802c630 tx=0x7fd5b80328a0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 -- 172.21.15.105:0/3139960502 shutdown_connections 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5ac0a2fa0 0x7fd5ac0a5460 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5c0102370 0x7fd5c019cf50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c01019b0 0x7fd5c0195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 --2- 172.21.15.105:0/3139960502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5c01007c0 0x7fd5c01959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 -- 172.21.15.105:0/3139960502 >> 172.21.15.105:0/3139960502 conn(0x7fd5c00fbd50 msgr2=0x7fd5c00fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 -- 172.21.15.105:0/3139960502 shutdown_connections 2024-03-26T18:02:43.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.057+0000 7fd5c4987640 1 -- 172.21.15.105:0/3139960502 wait complete. 2024-03-26T18:02:43.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T18:02:43.130 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:42 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3148303841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:02:43.154 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:42 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3148303841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 -- 172.21.15.105:0/3926862384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 msgr2=0x7f6e60108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 --2- 172.21.15.105:0/3926862384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 0x7f6e60108c00 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009ca0 tx=0x7f6e5802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 -- 172.21.15.105:0/3926862384 shutdown_connections 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 --2- 172.21.15.105:0/3926862384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 0x7f6e60108c00 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 --2- 172.21.15.105:0/3926862384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e601019b0 0x7f6e60101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 --2- 172.21.15.105:0/3926862384 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e601007c0 0x7f6e60100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 -- 172.21.15.105:0/3926862384 >> 172.21.15.105:0/3926862384 conn(0x7f6e600fbd50 msgr2=0x7f6e600fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 -- 172.21.15.105:0/3926862384 shutdown_connections 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.210+0000 7f6e68297640 1 -- 172.21.15.105:0/3926862384 wait complete. 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e68297640 1 Processor -- start 2024-03-26T18:02:43.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e68297640 1 -- start start 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e68297640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e601007c0 0x7f6e60197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e68297640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e601019b0 0x7f6e60198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e68297640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 0x7f6e6019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e68297640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010c2d0 con 0x7f6e60102370 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e68297640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010c150 con 0x7f6e601019b0 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e68297640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6010c450 con 0x7f6e601007c0 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66834640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 0x7f6e6019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66834640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 0x7f6e6019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48702/0 (socket says 172.21.15.105:48702) 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66834640 1 -- 172.21.15.105:0/2312223142 learned_addr learned my addr 172.21.15.105:0/2312223142 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66033640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e601007c0 0x7f6e60197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e65832640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e601019b0 0x7f6e60198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66834640 1 -- 172.21.15.105:0/2312223142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e601007c0 msgr2=0x7f6e60197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66834640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e601007c0 0x7f6e60197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66834640 1 -- 172.21.15.105:0/2312223142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e601019b0 msgr2=0x7f6e60198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66834640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e601019b0 0x7f6e60198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.211+0000 7f6e66834640 1 -- 172.21.15.105:0/2312223142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6019f740 con 0x7f6e60102370 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.212+0000 7f6e66834640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 0x7f6e6019f0d0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f6e5802c700 tx=0x7f6e580328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.212+0000 7f6e577fe640 1 -- 172.21.15.105:0/2312223142 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58041020 con 0x7f6e60102370 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.212+0000 7f6e577fe640 1 -- 172.21.15.105:0/2312223142 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e58003e20 con 0x7f6e60102370 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.212+0000 7f6e68297640 1 -- 172.21.15.105:0/2312223142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6019f9d0 con 0x7f6e60102370 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.212+0000 7f6e577fe640 1 -- 172.21.15.105:0/2312223142 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58039040 con 0x7f6e60102370 2024-03-26T18:02:43.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.212+0000 7f6e68297640 1 -- 172.21.15.105:0/2312223142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6019ff10 con 0x7f6e60102370 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.214+0000 7f6e577fe640 1 -- 172.21.15.105:0/2312223142 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6e58044020 con 0x7f6e60102370 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.215+0000 7f6e577fe640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e4c0a2fa0 0x7f6e4c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.215+0000 7f6e577fe640 1 -- 172.21.15.105:0/2312223142 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6e580f1540 con 0x7f6e60102370 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.215+0000 7f6e66033640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e4c0a2fa0 0x7f6e4c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e6ab22740 1 Processor -- start 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e66033640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e4c0a2fa0 0x7f6e4c0a5460 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6e60101810 tx=0x7f6e5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e6ab22740 1 -- start start 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613274e40 0x55c613275260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.197 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 0x55c61327a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61327ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c6131d5970 con 0x55c613274e40 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c61317dd70 con 0x55c61327a9a0 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c6131d1920 con 0x55c613275830 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e66033640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613274e40 0x55c613275260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e66033640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613274e40 0x55c613275260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48710/0 (socket says 172.21.15.105:48710) 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e66033640 1 -- 172.21.15.105:0/1117210226 learned_addr learned my addr 172.21.15.105:0/1117210226 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e66834640 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61327ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.216+0000 7f6e65832640 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 0x55c61327a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e66033640 1 -- 172.21.15.105:0/1117210226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 msgr2=0x55c61327a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e66033640 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 0x55c61327a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e66033640 1 -- 172.21.15.105:0/1117210226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 msgr2=0x55c61327ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e66033640 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61327ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e66033640 1 -- 172.21.15.105:0/1117210226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c612ee93f0 con 0x55c613274e40 2024-03-26T18:02:43.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e66033640 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613274e40 0x55c613275260 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00e630 tx=0x7f6e5c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6317b4bdc690494 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e54bfe640 1 -- 172.21.15.105:0/1117210226 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c026430 con 0x55c613274e40 2024-03-26T18:02:43.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e54bfe640 1 -- 172.21.15.105:0/1117210226 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e5c01c8c0 con 0x55c613274e40 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.217+0000 7f6e54bfe640 1 -- 172.21.15.105:0/1117210226 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c01d920 con 0x55c613274e40 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 -- 172.21.15.105:0/1117210226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613274e40 msgr2=0x55c613275260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613274e40 0x55c613275260 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00e630 tx=0x7f6e5c014000 comp rx=0 tx=0).stop 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 -- 172.21.15.105:0/1117210226 shutdown_connections 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61327ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 0x55c61327a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/1117210226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613274e40 0x55c613275260 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 -- 172.21.15.105:0/1117210226 >> 172.21.15.105:0/1117210226 conn(0x55c6132703b0 msgr2=0x55c6132707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 -- 172.21.15.105:0/1117210226 shutdown_connections 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.218+0000 7f6e6ab22740 1 -- 172.21.15.105:0/1117210226 wait complete. 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 1 -- start start 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 1 Processor -- start 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 0x55c61327f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61327fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613284ae0 0x55c613286ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c61327d9a0 con 0x55c613284ae0 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c613191310 con 0x55c61327a9a0 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c61327db20 con 0x55c613275830 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e6ab22740 10 client.0 _ll_register_callbacks cb 0x55c6131ccc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66834640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613284ae0 0x55c613286ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66033640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 0x55c61327f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66834640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613284ae0 0x55c613286ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48718/0 (socket says 172.21.15.105:48718) 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66834640 1 -- 172.21.15.105:0/4291293548 learned_addr learned my addr 172.21.15.105:0/4291293548 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e65832640 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61327fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66834640 1 -- 172.21.15.105:0/4291293548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 msgr2=0x55c61327f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66834640 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 0x55c61327f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66834640 1 -- 172.21.15.105:0/4291293548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 msgr2=0x55c61327fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66834640 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61327fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.219+0000 7f6e66834640 1 -- 172.21.15.105:0/4291293548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c612fb37d0 con 0x55c613284ae0 2024-03-26T18:02:43.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e66834640 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613284ae0 0x55c613286ed0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f6e580ef050 tx=0x7f6e580bd590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4496eb710fe83c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e49ffb640 1 -- 172.21.15.105:0/4291293548 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580bdc30 con 0x55c613284ae0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e49ffb640 1 -- 172.21.15.105:0/4291293548 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6e580f1110 con 0x55c613284ae0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e6ab22740 1 -- 172.21.15.105:0/4291293548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e5c01c8c0 con 0x55c613284ae0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e49ffb640 1 -- 172.21.15.105:0/4291293548 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6e580c1d80 con 0x55c613284ae0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e6ab22740 10 client.24162 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e6ab22740 1 -- 172.21.15.105:0/4291293548 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c61328a240 con 0x55c613284ae0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e49ffb640 1 -- 172.21.15.105:0/4291293548 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e5c01c8c0 con 0x55c613284ae0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.220+0000 7f6e49ffb640 1 -- 172.21.15.105:0/4291293548 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f6e5803c030 con 0x55c613284ae0 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e6ab22740 10 client.24162 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e6ab22740 10 client.24162 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e6ab22740 20 client.24162 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c61328b9a0 0x55c61328ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e6ab22740 4 client.24162 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e6ab22740 1 -- 172.21.15.105:0/4291293548 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6131e2eb0 con 0x55c61328b9a0 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e66033640 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c61328b9a0 0x55c61328ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e49ffb640 10 client.24162 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.221+0000 7f6e66033640 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c61328b9a0 0x55c61328ddc0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=abf878025b3eb5d0 server_cookie=dce6b5a969a9b293 in_seq=0 out_seq=0 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e49ffb640 1 -- 172.21.15.105:0/4291293548 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c6131e2eb0 con 0x55c61328b9a0 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e49ffb640 10 client.24162 handle_command_reply: tid=0 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 client.24162 shutdown 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 -- 172.21.15.105:0/4291293548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613284ae0 msgr2=0x55c613286ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613284ae0 0x55c613286ed0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f6e580ef050 tx=0x7f6e580bd590 comp rx=0 tx=0).stop 2024-03-26T18:02:43.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 -- 172.21.15.105:0/4291293548 shutdown_connections 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c61328b9a0 0x55c61328ddc0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e49ffb640 0 client.24162 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613284ae0 0x55c613286ed0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61327fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/4291293548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613275830 0x55c61327f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 -- 172.21.15.105:0/4291293548 >> 172.21.15.105:0/4291293548 conn(0x55c6132703b0 msgr2=0x55c61326b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 -- 172.21.15.105:0/4291293548 shutdown_connections 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.222+0000 7f6e6ab22740 1 -- 172.21.15.105:0/4291293548 wait complete. 2024-03-26T18:02:43.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.223+0000 7f6e6ab22740 20 client.24162 trim_cache size 0 max 16384 2024-03-26T18:02:43.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.243+0000 7f6e6ab22740 1 Processor -- start 2024-03-26T18:02:43.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.243+0000 7f6e6ab22740 1 -- start start 2024-03-26T18:02:43.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c613276f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61327a9a0 0x55c613277450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c613284ae0 0x55c61326a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a6b90 con 0x55c613275830 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a86d0 con 0x55c613284ae0 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a7260 con 0x55c61327a9a0 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c613276f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c613276f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48730/0 (socket says 172.21.15.105:48730) 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e65832640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61327a9a0 0x55c613277450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 -- 172.21.15.105:0/2034925439 learned_addr learned my addr 172.21.15.105:0/2034925439 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66834640 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c613284ae0 0x55c61326a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 -- 172.21.15.105:0/2034925439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61327a9a0 msgr2=0x55c613277450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61327a9a0 0x55c613277450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 -- 172.21.15.105:0/2034925439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c613284ae0 msgr2=0x55c61326a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c613284ae0 0x55c61326a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 -- 172.21.15.105:0/2034925439 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c613271120 con 0x55c613275830 2024-03-26T18:02:43.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e65832640 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61327a9a0 0x55c613277450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66834640 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c613284ae0 0x55c61326a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.244+0000 7f6e66033640 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c613276f10 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c026000 tx=0x7f6e5c011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c21ddea765983cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.245+0000 7f6e4bfff640 1 -- 172.21.15.105:0/2034925439 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c02b040 con 0x55c613275830 2024-03-26T18:02:43.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.245+0000 7f6e4bfff640 1 -- 172.21.15.105:0/2034925439 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e5c0054e0 con 0x55c613275830 2024-03-26T18:02:43.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.245+0000 7f6e4bfff640 1 -- 172.21.15.105:0/2034925439 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c01c670 con 0x55c613275830 2024-03-26T18:02:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 -- 172.21.15.105:0/2034925439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 msgr2=0x55c613276f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c613276f10 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c026000 tx=0x7f6e5c011950 comp rx=0 tx=0).stop 2024-03-26T18:02:43.231 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 -- 172.21.15.105:0/2034925439 shutdown_connections 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c613284ae0 0x55c61326a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c61327a9a0 0x55c613277450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/2034925439 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c613276f10 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 -- 172.21.15.105:0/2034925439 >> 172.21.15.105:0/2034925439 conn(0x55c6132703b0 msgr2=0x55c61327f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 -- 172.21.15.105:0/2034925439 shutdown_connections 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 -- 172.21.15.105:0/2034925439 wait complete. 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 -- start start 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.246+0000 7f6e6ab22740 1 Processor -- start 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c61329c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61329c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e6ab22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613284ae0 0x55c61329d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a8000 con 0x55c613275830 2024-03-26T18:02:43.232 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a7930 con 0x55c61327a9a0 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e6ab22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601a4b70 con 0x55c613284ae0 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e6ab22740 10 client.0 _ll_register_callbacks cb 0x7f6e580bdaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c61329c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c61329c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48736/0 (socket says 172.21.15.105:48736) 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 -- 172.21.15.105:0/3627587752 learned_addr learned my addr 172.21.15.105:0/3627587752 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66834640 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613284ae0 0x55c61329d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e65832640 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61329c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 -- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613284ae0 msgr2=0x55c61329d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613284ae0 0x55c61329d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 -- 172.21.15.105:0/3627587752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 msgr2=0x55c61329c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61329c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.233 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 -- 172.21.15.105:0/3627587752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61329df60 con 0x55c613275830 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66834640 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613284ae0 0x55c61329d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e66033640 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c61329c420 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0118d0 tx=0x7f6e5c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b78a0874b2b268 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.247+0000 7f6e49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e49ffb640 1 -- 172.21.15.105:0/3627587752 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c035120 con 0x55c613275830 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e49ffb640 1 -- 172.21.15.105:0/3627587752 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6e5c0355b0 con 0x55c613275830 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 1 -- 172.21.15.105:0/3627587752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e5c0054e0 con 0x55c613275830 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e49ffb640 1 -- 172.21.15.105:0/3627587752 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6e5c0357b0 con 0x55c613275830 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 10 client.24168 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 1 -- 172.21.15.105:0/3627587752 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c61329f260 con 0x55c613275830 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e49ffb640 1 -- 172.21.15.105:0/3627587752 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e5c0054e0 con 0x55c613275830 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e49ffb640 1 -- 172.21.15.105:0/3627587752 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f6e5c03e6b0 con 0x55c613275830 2024-03-26T18:02:43.234 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 10 client.24168 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 10 client.24168 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 20 client.24168 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c6132f1b10 0x55c6132f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 4 client.24168 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.248+0000 7f6e6ab22740 1 -- 172.21.15.105:0/3627587752 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6131dd200 con 0x55c6132f1b10 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.249+0000 7f6e65832640 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c6132f1b10 0x55c6132f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.249+0000 7f6e49ffb640 10 client.24168 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.249+0000 7f6e65832640 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c6132f1b10 0x55c6132f3f30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b5b850e0d8ad4ba1 server_cookie=b848c34d80fbe71f in_seq=0 out_seq=0 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.249+0000 7f6e49ffb640 1 -- 172.21.15.105:0/3627587752 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6131dd200 con 0x55c6132f1b10 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.249+0000 7f6e49ffb640 10 client.24168 handle_command_reply: tid=0 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.249+0000 7f6e6ab22740 1 client.24168 shutdown 2024-03-26T18:02:43.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.249+0000 7f6e6ab22740 1 -- 172.21.15.105:0/3627587752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 msgr2=0x55c61329c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.249+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c61329c420 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0118d0 tx=0x7f6e5c01c4c0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 1 -- 172.21.15.105:0/3627587752 shutdown_connections 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c6132f1b10 0x55c6132f3f30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c613284ae0 0x55c61329d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e49ffb640 0 client.24168 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61327a9a0 0x55c61329c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 1 --2- 172.21.15.105:0/3627587752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c613275830 0x55c61329c420 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 1 -- 172.21.15.105:0/3627587752 >> 172.21.15.105:0/3627587752 conn(0x55c6132703b0 msgr2=0x55c613276f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 1 -- 172.21.15.105:0/3627587752 shutdown_connections 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 1 -- 172.21.15.105:0/3627587752 wait complete. 2024-03-26T18:02:43.236 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.250+0000 7f6e6ab22740 20 client.24168 trim_cache size 0 max 16384 2024-03-26T18:02:43.237 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:43.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 -- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e4c0a2fa0 msgr2=0x7f6e4c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e4c0a2fa0 0x7f6e4c0a5460 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6e60101810 tx=0x7f6e5c005f90 comp rx=0 tx=0).stop 2024-03-26T18:02:43.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 -- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 msgr2=0x7f6e6019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.237 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 0x7f6e6019f0d0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f6e5802c700 tx=0x7f6e580328a0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 -- 172.21.15.105:0/2312223142 shutdown_connections 2024-03-26T18:02:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6e4c0a2fa0 0x7f6e4c0a5460 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e60102370 0x7f6e6019f0d0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6e601019b0 0x7f6e60198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 --2- 172.21.15.105:0/2312223142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e601007c0 0x7f6e60197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.251+0000 7f6e68297640 1 -- 172.21.15.105:0/2312223142 >> 172.21.15.105:0/2312223142 conn(0x7f6e600fbd50 msgr2=0x7f6e600fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.252+0000 7f6e68297640 1 -- 172.21.15.105:0/2312223142 shutdown_connections 2024-03-26T18:02:43.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.252+0000 7f6e68297640 1 -- 172.21.15.105:0/2312223142 wait complete. 2024-03-26T18:02:43.247 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T18:02:43.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:42 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3148303841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 -- 172.21.15.105:0/2681347571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201007c0 msgr2=0x7ff820100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 --2- 172.21.15.105:0/2681347571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201007c0 0x7ff820100be0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7ff810009900 tx=0x7ff81002bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 -- 172.21.15.105:0/2681347571 shutdown_connections 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 --2- 172.21.15.105:0/2681347571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff820102370 0x7ff820108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 --2- 172.21.15.105:0/2681347571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8201019b0 0x7ff820101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 --2- 172.21.15.105:0/2681347571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201007c0 0x7ff820100be0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 -- 172.21.15.105:0/2681347571 >> 172.21.15.105:0/2681347571 conn(0x7ff8200fbd50 msgr2=0x7ff8200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 -- 172.21.15.105:0/2681347571 shutdown_connections 2024-03-26T18:02:43.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.411+0000 7ff824a05640 1 -- 172.21.15.105:0/2681347571 wait complete. 2024-03-26T18:02:43.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff824a05640 1 Processor -- start 2024-03-26T18:02:43.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff824a05640 1 -- start start 2024-03-26T18:02:43.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff824a05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8201007c0 0x7ff820196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff824a05640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201019b0 0x7ff820196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff824a05640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff820102370 0x7ff82019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff824a05640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010c2c0 con 0x7ff8201019b0 2024-03-26T18:02:43.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff824a05640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010c140 con 0x7ff8201007c0 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff824a05640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010c440 con 0x7ff820102370 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff81dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201019b0 0x7ff820196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff81e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8201007c0 0x7ff820196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff81dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201019b0 0x7ff820196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48766/0 (socket says 172.21.15.105:48766) 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff81dd9b640 1 -- 172.21.15.105:0/1650755229 learned_addr learned my addr 172.21.15.105:0/1650755229 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.412+0000 7ff81ed9d640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff820102370 0x7ff82019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff81dd9b640 1 -- 172.21.15.105:0/1650755229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff820102370 msgr2=0x7ff82019dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff820102370 0x7ff82019dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff81dd9b640 1 -- 172.21.15.105:0/1650755229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8201007c0 msgr2=0x7ff820196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8201007c0 0x7ff820196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff81dd9b640 1 -- 172.21.15.105:0/1650755229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82019e650 con 0x7ff8201019b0 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff81e59c640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8201007c0 0x7ff820196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201019b0 0x7ff820196f70 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7ff81400ea60 tx=0x7ff81400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff80f7fe640 1 -- 172.21.15.105:0/1650755229 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400cde0 con 0x7ff8201019b0 2024-03-26T18:02:43.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff824a05640 1 -- 172.21.15.105:0/1650755229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82019e940 con 0x7ff8201019b0 2024-03-26T18:02:43.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff80f7fe640 1 -- 172.21.15.105:0/1650755229 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8140035f0 con 0x7ff8201019b0 2024-03-26T18:02:43.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff80f7fe640 1 -- 172.21.15.105:0/1650755229 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814010690 con 0x7ff8201019b0 2024-03-26T18:02:43.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.413+0000 7ff824a05640 1 -- 172.21.15.105:0/1650755229 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82019eee0 con 0x7ff8201019b0 2024-03-26T18:02:43.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.416+0000 7ff80f7fe640 1 -- 172.21.15.105:0/1650755229 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff8140037a0 con 0x7ff8201019b0 2024-03-26T18:02:43.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.416+0000 7ff8275f2740 1 Processor -- start 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff8275f2740 1 -- start start 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff8275f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637ca026970 con 0x5637ca0c4630 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff8275f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5637c9fced70 con 0x5637ca0c97a0 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff8275f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637ca022920 con 0x5637ca0c3c40 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff81dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff81ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff81ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47464/0 (socket says 172.21.15.105:47464) 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff81e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.417+0000 7ff81ed9d640 1 -- 172.21.15.105:0/401544859 learned_addr learned my addr 172.21.15.105:0/401544859 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81dd9b640 1 -- 172.21.15.105:0/401544859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 msgr2=0x5637ca0c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81dd9b640 1 -- 172.21.15.105:0/401544859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 msgr2=0x5637ca0c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff80f7fe640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8080a2f20 0x7ff8080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81dd9b640 1 -- 172.21.15.105:0/401544859 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637c9d3a3f0 con 0x5637ca0c97a0 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81e59c640 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81ed9d640 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81ed9d640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8080a2f20 0x7ff8080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0cbc30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff8140cd180 tx=0x7ff814098140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaf704ad52ce9d90 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff80f7fe640 1 -- 172.21.15.105:0/1650755229 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff8140cf6f0 con 0x7ff8201019b0 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff80cff9640 1 -- 172.21.15.105:0/401544859 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81409bbb0 con 0x5637ca0c97a0 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff81ed9d640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8080a2f20 0x7ff8080a53e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x5637ca0c4e90 tx=0x7ff818009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff80cff9640 1 -- 172.21.15.105:0/401544859 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff81409f070 con 0x5637ca0c97a0 2024-03-26T18:02:43.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.418+0000 7ff80cff9640 1 -- 172.21.15.105:0/401544859 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81409b740 con 0x5637ca0c97a0 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 -- 172.21.15.105:0/401544859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 msgr2=0x5637ca0cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0cbc30 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff8140cd180 tx=0x7ff814098140 comp rx=0 tx=0).stop 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 -- 172.21.15.105:0/401544859 shutdown_connections 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0cbc30 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 --2- 172.21.15.105:0/401544859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 -- 172.21.15.105:0/401544859 >> 172.21.15.105:0/401544859 conn(0x5637ca0bf1b0 msgr2=0x5637ca0bf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 -- 172.21.15.105:0/401544859 shutdown_connections 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.419+0000 7ff8275f2740 1 -- 172.21.15.105:0/401544859 wait complete. 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 1 -- start start 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 1 Processor -- start 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c4630 0x5637ca0d9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c97a0 0x5637ca0dbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5637ca0cc750 con 0x5637ca0c3c40 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5637c9fe2310 con 0x5637ca0c4630 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5637ca0cc8d0 con 0x5637ca0c97a0 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff8275f2740 10 client.0 _ll_register_callbacks cb 0x5637ca01dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff81e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff81dd9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c4630 0x5637ca0d9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff81e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48784/0 (socket says 172.21.15.105:48784) 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff81e59c640 1 -- 172.21.15.105:0/105749718 learned_addr learned my addr 172.21.15.105:0/105749718 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.420+0000 7ff81ed9d640 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c97a0 0x5637ca0dbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff81dd9b640 1 -- 172.21.15.105:0/105749718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c97a0 msgr2=0x5637ca0dbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c97a0 0x5637ca0dbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff81dd9b640 1 -- 172.21.15.105:0/105749718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c3c40 msgr2=0x5637ca0c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff81dd9b640 1 -- 172.21.15.105:0/105749718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637c9e047d0 con 0x5637ca0c4630 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff81e59c640 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c4630 0x5637ca0d9520 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff8140cd1b0 tx=0x7ff8140cf9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca5c30bbab9f4eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff805ffb640 1 -- 172.21.15.105:0/105749718 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81409b920 con 0x5637ca0c4630 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff8275f2740 1 -- 172.21.15.105:0/105749718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81409f070 con 0x5637ca0c4630 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff805ffb640 1 -- 172.21.15.105:0/105749718 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff814098450 con 0x5637ca0c4630 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff805ffb640 1 -- 172.21.15.105:0/105749718 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff81409b740 con 0x5637ca0c4630 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff8275f2740 10 client.33842 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.421+0000 7ff8275f2740 1 -- 172.21.15.105:0/105749718 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5637ca0df0b0 con 0x5637ca0c4630 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff805ffb640 1 -- 172.21.15.105:0/105749718 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff81409f070 con 0x5637ca0c4630 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff805ffb640 1 -- 172.21.15.105:0/105749718 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff8140d2ce0 con 0x5637ca0c4630 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff8275f2740 10 client.33842 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff8275f2740 10 client.33842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff8275f2740 20 client.33842 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff8275f2740 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5637ca0e07e0 0x5637ca0e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff8275f2740 4 client.33842 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:43.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff8275f2740 1 -- 172.21.15.105:0/105749718 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637ca033eb0 con 0x5637ca0e07e0 2024-03-26T18:02:43.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff81e59c640 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5637ca0e07e0 0x5637ca0e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff81e59c640 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5637ca0e07e0 0x5637ca0e2c00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79b1234852441187 server_cookie=cd9749bfd87bf4e0 in_seq=0 out_seq=0 2024-03-26T18:02:43.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.422+0000 7ff805ffb640 10 client.33842 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:02:43.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff805ffb640 1 -- 172.21.15.105:0/105749718 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5637ca033eb0 con 0x5637ca0e07e0 2024-03-26T18:02:43.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff805ffb640 10 client.33842 handle_command_reply: tid=0 2024-03-26T18:02:43.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 client.33842 shutdown 2024-03-26T18:02:43.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 -- 172.21.15.105:0/105749718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c4630 msgr2=0x5637ca0d9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c4630 0x5637ca0d9520 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff8140cd1b0 tx=0x7ff8140cf9f0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 -- 172.21.15.105:0/105749718 shutdown_connections 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c97a0 0x5637ca0dbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5637ca0e07e0 0x5637ca0e2c00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c4630 0x5637ca0d9520 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 --2- 172.21.15.105:0/105749718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.423+0000 7ff8275f2740 1 -- 172.21.15.105:0/105749718 >> 172.21.15.105:0/105749718 conn(0x5637ca0bf1b0 msgr2=0x5637ca0bc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.424+0000 7ff805ffb640 0 client.33842 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.424+0000 7ff8275f2740 1 -- 172.21.15.105:0/105749718 shutdown_connections 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.424+0000 7ff8275f2740 1 -- 172.21.15.105:0/105749718 wait complete. 2024-03-26T18:02:43.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.424+0000 7ff8275f2740 20 client.33842 trim_cache size 0 max 16384 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.445+0000 7ff8275f2740 1 Processor -- start 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.445+0000 7ff8275f2740 1 -- start start 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.445+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.445+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.445+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff8275f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201a6cd0 con 0x5637ca0c4630 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff8275f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201a8810 con 0x5637ca0c97a0 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff8275f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201a73a0 con 0x5637ca0c3c40 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47476/0 (socket says 172.21.15.105:47476) 2024-03-26T18:02:43.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 -- 172.21.15.105:0/2497414808 learned_addr learned my addr 172.21.15.105:0/2497414808 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81ed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 -- 172.21.15.105:0/2497414808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 msgr2=0x5637ca0bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 -- 172.21.15.105:0/2497414808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 msgr2=0x5637ca0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 -- 172.21.15.105:0/2497414808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ca0d9890 con 0x5637ca0c3c40 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.446+0000 7ff81e59c640 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff810036040 tx=0x7ff810005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cd8b4999b111ce3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff807fff640 1 -- 172.21.15.105:0/2497414808 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81004e020 con 0x5637ca0c3c40 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff807fff640 1 -- 172.21.15.105:0/2497414808 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8100043d0 con 0x5637ca0c3c40 2024-03-26T18:02:43.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff807fff640 1 -- 172.21.15.105:0/2497414808 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810005040 con 0x5637ca0c3c40 2024-03-26T18:02:43.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff8275f2740 1 -- 172.21.15.105:0/2497414808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 msgr2=0x5637ca0c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff810036040 tx=0x7ff810005fd0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff8275f2740 1 -- 172.21.15.105:0/2497414808 shutdown_connections 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca0bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2497414808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0c9120 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.447+0000 7ff8275f2740 1 -- 172.21.15.105:0/2497414808 >> 172.21.15.105:0/2497414808 conn(0x5637ca0bf1b0 msgr2=0x5637ca0c2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 -- 172.21.15.105:0/2497414808 shutdown_connections 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 -- 172.21.15.105:0/2497414808 wait complete. 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 -- start start 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 Processor -- start 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca1444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201a8140 con 0x5637ca0c4630 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201a7a70 con 0x5637ca0c97a0 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201a4cb0 con 0x5637ca0c3c40 2024-03-26T18:02:43.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff8275f2740 10 client.0 _ll_register_callbacks cb 0x7ff818009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff81dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.448+0000 7ff81e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0f0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47486/0 (socket says 172.21.15.105:47486) 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81e59c640 1 -- 172.21.15.105:0/2453019895 learned_addr learned my addr 172.21.15.105:0/2453019895 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48798/0 (socket says 172.21.15.105:48798) 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81dd9b640 1 -- 172.21.15.105:0/2453019895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 msgr2=0x5637ca0f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0f0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81dd9b640 1 -- 172.21.15.105:0/2453019895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 msgr2=0x5637ca1444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca1444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81dd9b640 1 -- 172.21.15.105:0/2453019895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ca0dbe80 con 0x5637ca0c4630 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff81dd9b640 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0f1200 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7ff81409bae0 tx=0x7ff8140cfa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46309f812be3408 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff805ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff805ffb640 1 -- 172.21.15.105:0/2453019895 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81409bbb0 con 0x5637ca0c4630 2024-03-26T18:02:43.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff805ffb640 1 -- 172.21.15.105:0/2453019895 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff81409c040 con 0x5637ca0c4630 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.449+0000 7ff8275f2740 1 -- 172.21.15.105:0/2453019895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8100043d0 con 0x5637ca0c4630 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff805ffb640 1 -- 172.21.15.105:0/2453019895 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff81409b920 con 0x5637ca0c4630 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff8275f2740 10 client.24186 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff8275f2740 1 -- 172.21.15.105:0/2453019895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5637ca145b60 con 0x5637ca0c4630 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff805ffb640 1 -- 172.21.15.105:0/2453019895 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8100043d0 con 0x5637ca0c4630 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff805ffb640 1 -- 172.21.15.105:0/2453019895 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff8140d2ce0 con 0x5637ca0c4630 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff8275f2740 10 client.24186 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff8275f2740 10 client.24186 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff8275f2740 20 client.24186 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5637ca146180 0x5637ca1485a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff8275f2740 4 client.24186 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:43.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff8275f2740 1 -- 172.21.15.105:0/2453019895 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637ca02e200 con 0x5637ca146180 2024-03-26T18:02:43.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.450+0000 7ff81e59c640 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5637ca146180 0x5637ca1485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff81e59c640 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5637ca146180 0x5637ca1485a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=199fbf0f26f284eb server_cookie=e1f95f366dd1e189 in_seq=0 out_seq=0 2024-03-26T18:02:43.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff805ffb640 10 client.24186 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff805ffb640 1 -- 172.21.15.105:0/2453019895 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637ca02e200 con 0x5637ca146180 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff805ffb640 10 client.24186 handle_command_reply: tid=0 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff8275f2740 1 client.24186 shutdown 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff8275f2740 1 -- 172.21.15.105:0/2453019895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 msgr2=0x5637ca0f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0f1200 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7ff81409bae0 tx=0x7ff8140cfa60 comp rx=0 tx=0).stop 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff8275f2740 1 -- 172.21.15.105:0/2453019895 shutdown_connections 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5637ca146180 0x5637ca1485a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5637ca0c97a0 0x5637ca1444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.452+0000 7ff805ffb640 0 client.24186 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.451+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5637ca0c4630 0x5637ca0f1200 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.452+0000 7ff8275f2740 1 --2- 172.21.15.105:0/2453019895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5637ca0c3c40 0x5637ca0f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.452+0000 7ff8275f2740 1 -- 172.21.15.105:0/2453019895 >> 172.21.15.105:0/2453019895 conn(0x5637ca0bf1b0 msgr2=0x5637ca0c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.452+0000 7ff8275f2740 1 -- 172.21.15.105:0/2453019895 shutdown_connections 2024-03-26T18:02:43.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.452+0000 7ff8275f2740 1 -- 172.21.15.105:0/2453019895 wait complete. 2024-03-26T18:02:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.452+0000 7ff8275f2740 20 client.24186 trim_cache size 0 max 16384 2024-03-26T18:02:43.437 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:43.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 -- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8080a2f20 msgr2=0x7ff8080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8080a2f20 0x7ff8080a53e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x5637ca0c4e90 tx=0x7ff818009600 comp rx=0 tx=0).stop 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 -- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201019b0 msgr2=0x7ff820196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201019b0 0x7ff820196f70 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7ff81400ea60 tx=0x7ff81400ef20 comp rx=0 tx=0).stop 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 -- 172.21.15.105:0/1650755229 shutdown_connections 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff8080a2f20 0x7ff8080a53e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff820102370 0x7ff82019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8201019b0 0x7ff820196f70 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 --2- 172.21.15.105:0/1650755229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff8201007c0 0x7ff820196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 -- 172.21.15.105:0/1650755229 >> 172.21.15.105:0/1650755229 conn(0x7ff8200fbd50 msgr2=0x7ff820104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 -- 172.21.15.105:0/1650755229 shutdown_connections 2024-03-26T18:02:43.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.453+0000 7ff824a05640 1 -- 172.21.15.105:0/1650755229 wait complete. 2024-03-26T18:02:43.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T18:02:43.562 INFO:tasks.workunit.client.0.smithi105.stdout: 10 89628 7.02 MB/sec execute 566 sec latency 125.286 ms 2024-03-26T18:02:43.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 -- 172.21.15.105:0/3743433297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c1023c0 msgr2=0x7f1f8c1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/3743433297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c1023c0 0x7f1f8c1047b0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009900 tx=0x7f1f8002bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:43.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 -- 172.21.15.105:0/3743433297 shutdown_connections 2024-03-26T18:02:43.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/3743433297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8c107620 0x7f1f8c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/3743433297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c104cf0 0x7f1f8c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/3743433297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c1023c0 0x7f1f8c1047b0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 -- 172.21.15.105:0/3743433297 >> 172.21.15.105:0/3743433297 conn(0x7f1f8c0fbd50 msgr2=0x7f1f8c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 -- 172.21.15.105:0/3743433297 shutdown_connections 2024-03-26T18:02:43.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.602+0000 7f1f92f0c640 1 -- 172.21.15.105:0/3743433297 wait complete. 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.603+0000 7f1f92f0c640 1 Processor -- start 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.603+0000 7f1f92f0c640 1 -- start start 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.603+0000 7f1f92f0c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8c1023c0 0x7f1f8c101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.603+0000 7f1f92f0c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c104cf0 0x7f1f8c100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.603+0000 7f1f92f0c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c107620 0x7f1f8c1008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f92f0c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10d410 con 0x7f1f8c104cf0 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f92f0c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10d290 con 0x7f1f8c107620 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f92f0c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c10d590 con 0x7f1f8c1023c0 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8c1023c0 0x7f1f8c101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c104cf0 0x7f1f8c100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8c1023c0 0x7f1f8c101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47514/0 (socket says 172.21.15.105:47514) 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 -- 172.21.15.105:0/919201701 learned_addr learned my addr 172.21.15.105:0/919201701 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f914a9640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c107620 0x7f1f8c1008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 -- 172.21.15.105:0/919201701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c107620 msgr2=0x7f1f8c1008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c107620 0x7f1f8c1008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 -- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c104cf0 msgr2=0x7f1f8c100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c104cf0 0x7f1f8c100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 -- 172.21.15.105:0/919201701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c101140 con 0x7f1f8c1023c0 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c104cf0 0x7f1f8c100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8c1023c0 0x7f1f8c101d30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1f80006010 tx=0x7f1f80005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f89ffb640 1 -- 172.21.15.105:0/919201701 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80043020 con 0x7f1f8c1023c0 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f92f0c640 1 -- 172.21.15.105:0/919201701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c1a1ca0 con 0x7f1f8c1023c0 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f89ffb640 1 -- 172.21.15.105:0/919201701 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f800043d0 con 0x7f1f8c1023c0 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.604+0000 7f1f89ffb640 1 -- 172.21.15.105:0/919201701 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80003b80 con 0x7f1f8c1023c0 2024-03-26T18:02:43.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.605+0000 7f1f92f0c640 1 -- 172.21.15.105:0/919201701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c1a2190 con 0x7f1f8c1023c0 2024-03-26T18:02:43.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.607+0000 7f1f95bcc740 1 Processor -- start 2024-03-26T18:02:43.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.607+0000 7f1f95bcc740 1 -- start start 2024-03-26T18:02:43.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.607+0000 7f1f89ffb640 1 -- 172.21.15.105:0/919201701 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f1f80046020 con 0x7f1f8c1023c0 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.607+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a608e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.607+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a609420 0x55bb2a60df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a610a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bb2a56d970 con 0x55bb2a609420 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bb2a515d70 con 0x55bb2a60e590 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bb2a569920 con 0x55bb2a608a30 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f90ca8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a609420 0x55bb2a60df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f914a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a608e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a610a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f914a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a608e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47532/0 (socket says 172.21.15.105:47532) 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a610a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32940/0 (socket says 172.21.15.105:32940) 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f90ca8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a609420 0x55bb2a60df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48840/0 (socket says 172.21.15.105:48840) 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f914a9640 1 -- 172.21.15.105:0/1746291242 learned_addr learned my addr 172.21.15.105:0/1746291242 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f8bfff640 1 -- 172.21.15.105:0/1746291242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 msgr2=0x55bb2a608e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.591 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a608e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f8bfff640 1 -- 172.21.15.105:0/1746291242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a609420 msgr2=0x55bb2a60df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a609420 0x55bb2a60df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f8bfff640 1 -- 172.21.15.105:0/1746291242 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb2a2813f0 con 0x55bb2a60e590 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f914a9640 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a608e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a609420 0x55bb2a60df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f89ffb640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f7c0a2f20 0x7f1f7c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a610a20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c069500 tx=0x7f1f780118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc37449670164961 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.608+0000 7f1f914a9640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f7c0a2f20 0x7f1f7c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f89ffb640 1 -- 172.21.15.105:0/919201701 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f1f800f1620 con 0x7f1f8c1023c0 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f76ffd640 1 -- 172.21.15.105:0/1746291242 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78004030 con 0x55bb2a60e590 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f76ffd640 1 -- 172.21.15.105:0/1746291242 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f780054d0 con 0x55bb2a60e590 2024-03-26T18:02:43.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f76ffd640 1 -- 172.21.15.105:0/1746291242 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78004c80 con 0x55bb2a60e590 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f95bcc740 1 -- 172.21.15.105:0/1746291242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 msgr2=0x55bb2a610a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f914a9640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f7c0a2f20 0x7f1f7c0a53e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x55bb2a609c80 tx=0x7f1f84009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a610a20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c069500 tx=0x7f1f780118d0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f95bcc740 1 -- 172.21.15.105:0/1746291242 shutdown_connections 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.609+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a610a20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a609420 0x55bb2a60df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/1746291242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a608e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 -- 172.21.15.105:0/1746291242 >> 172.21.15.105:0/1746291242 conn(0x55bb2a603fa0 msgr2=0x55bb2a6043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 -- 172.21.15.105:0/1746291242 shutdown_connections 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 -- 172.21.15.105:0/1746291242 wait complete. 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 -- start start 2024-03-26T18:02:43.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 Processor -- start 2024-03-26T18:02:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a609420 0x55bb2a613430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.610+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a61a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55bb2a611540 con 0x55bb2a608a30 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bb2a529310 con 0x55bb2a60e590 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55bb2a6116c0 con 0x55bb2a609420 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f95bcc740 10 client.0 _ll_register_callbacks cb 0x55bb2a564c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a609420 0x55bb2a613430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f914a9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a61a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a609420 0x55bb2a613430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47534/0 (socket says 172.21.15.105:47534) 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 -- 172.21.15.105:0/2103031232 learned_addr learned my addr 172.21.15.105:0/2103031232 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 -- 172.21.15.105:0/2103031232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 msgr2=0x55bb2a61a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a61a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 -- 172.21.15.105:0/2103031232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 msgr2=0x55bb2a612ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 -- 172.21.15.105:0/2103031232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2a34b7d0 con 0x55bb2a609420 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a609420 0x55bb2a613430 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009730 tx=0x7f1f78003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=572276565285d87b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f6bfff640 1 -- 172.21.15.105:0/2103031232 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78027020 con 0x55bb2a609420 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f6bfff640 1 -- 172.21.15.105:0/2103031232 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f1f78016040 con 0x55bb2a609420 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.611+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2103031232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f780054d0 con 0x55bb2a609420 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f6bfff640 1 -- 172.21.15.105:0/2103031232 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1f78032050 con 0x55bb2a609420 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f95bcc740 10 client.33862 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2103031232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bb2a61d7e0 con 0x55bb2a609420 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f6bfff640 1 -- 172.21.15.105:0/2103031232 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f780054d0 con 0x55bb2a609420 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f6bfff640 1 -- 172.21.15.105:0/2103031232 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f1f7800f9f0 con 0x55bb2a609420 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f95bcc740 10 client.33862 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f95bcc740 10 client.33862 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f95bcc740 20 client.33862 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bb2a61ecc0 0x55bb2a621160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f95bcc740 4 client.33862 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2103031232 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb2a57aeb0 con 0x55bb2a61ecc0 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.612+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bb2a61ecc0 0x55bb2a621160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.613+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bb2a61ecc0 0x55bb2a621160 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5d10e3f08bc297e server_cookie=26d2e91d2641e481 in_seq=0 out_seq=0 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.613+0000 7f1f6bfff640 10 client.33862 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f6bfff640 1 -- 172.21.15.105:0/2103031232 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bb2a57aeb0 con 0x55bb2a61ecc0 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f6bfff640 10 client.33862 handle_command_reply: tid=0 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 client.33862 shutdown 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2103031232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a609420 msgr2=0x55bb2a613430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a609420 0x55bb2a613430 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009730 tx=0x7f1f78003820 comp rx=0 tx=0).stop 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2103031232 shutdown_connections 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bb2a61ecc0 0x55bb2a621160 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a60e590 0x55bb2a61a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a609420 0x55bb2a613430 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f6bfff640 0 client.33862 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:02:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2103031232 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2103031232 >> 172.21.15.105:0/2103031232 conn(0x55bb2a603fa0 msgr2=0x55bb2a6031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2103031232 shutdown_connections 2024-03-26T18:02:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2103031232 wait complete. 2024-03-26T18:02:43.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.614+0000 7f1f95bcc740 20 client.33862 trim_cache size 0 max 16384 2024-03-26T18:02:43.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.637+0000 7f1f95bcc740 1 Processor -- start 2024-03-26T18:02:43.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f95bcc740 1 -- start start 2024-03-26T18:02:43.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a613430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.619 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a60e590 0x55bb2a61a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f914a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a60e590 0x55bb2a61a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1ab050 con 0x55bb2a60e590 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1acb90 con 0x55bb2a609420 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1ab720 con 0x55bb2a608a30 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f914a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a60e590 0x55bb2a61a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48868/0 (socket says 172.21.15.105:48868) 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f914a9640 1 -- 172.21.15.105:0/2952278291 learned_addr learned my addr 172.21.15.105:0/2952278291 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a613430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f8bfff640 1 -- 172.21.15.105:0/2952278291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 msgr2=0x55bb2a612ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f8bfff640 1 -- 172.21.15.105:0/2952278291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a60e590 msgr2=0x55bb2a61a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a60e590 0x55bb2a61a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f8bfff640 1 -- 172.21.15.105:0/2952278291 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb2a6038f0 con 0x55bb2a609420 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.638+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.639+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a613430 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800fe30 tx=0x7f1f78009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dac1a4665326e70 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.639+0000 7f1f767fc640 1 -- 172.21.15.105:0/2952278291 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78027070 con 0x55bb2a609420 2024-03-26T18:02:43.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.639+0000 7f1f767fc640 1 -- 172.21.15.105:0/2952278291 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f78003c20 con 0x55bb2a609420 2024-03-26T18:02:43.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.639+0000 7f1f767fc640 1 -- 172.21.15.105:0/2952278291 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800fae0 con 0x55bb2a609420 2024-03-26T18:02:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2952278291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 msgr2=0x55bb2a613430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a613430 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800fe30 tx=0x7f1f78009e20 comp rx=0 tx=0).stop 2024-03-26T18:02:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2952278291 shutdown_connections 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a60e590 0x55bb2a61a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a613430 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/2952278291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a608a30 0x55bb2a612ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2952278291 >> 172.21.15.105:0/2952278291 conn(0x55bb2a603fa0 msgr2=0x55bb2a60a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2952278291 shutdown_connections 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 -- 172.21.15.105:0/2952278291 wait complete. 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 -- start start 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.640+0000 7f1f95bcc740 1 Processor -- start 2024-03-26T18:02:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a617520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a617a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f95bcc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a60e590 0x55bb2a683050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1ac4c0 con 0x55bb2a608a30 2024-03-26T18:02:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1abdf0 con 0x55bb2a609420 2024-03-26T18:02:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f95bcc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1a9030 con 0x55bb2a60e590 2024-03-26T18:02:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f95bcc740 10 client.0 _ll_register_callbacks cb 0x7f1f800f7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a617a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a617a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32960/0 (socket says 172.21.15.105:32960) 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f90ca8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a617520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f8bfff640 1 -- 172.21.15.105:0/3952052459 learned_addr learned my addr 172.21.15.105:0/3952052459 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f914a9640 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a60e590 0x55bb2a683050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f8bfff640 1 -- 172.21.15.105:0/3952052459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a60e590 msgr2=0x55bb2a683050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a60e590 0x55bb2a683050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f8bfff640 1 -- 172.21.15.105:0/3952052459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 msgr2=0x55bb2a617520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a617520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f8bfff640 1 -- 172.21.15.105:0/3952052459 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb2a683610 con 0x55bb2a609420 2024-03-26T18:02:43.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.641+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a617520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f8bfff640 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a617a60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1f78011910 tx=0x7f1f7800f480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee8212c9813a5188 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f6bfff640 1 -- 172.21.15.105:0/3952052459 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78027040 con 0x55bb2a609420 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f6bfff640 1 -- 172.21.15.105:0/3952052459 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f1f7800fa20 con 0x55bb2a609420 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f95bcc740 1 -- 172.21.15.105:0/3952052459 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f78003c20 con 0x55bb2a609420 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f6bfff640 1 -- 172.21.15.105:0/3952052459 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1f7800fc20 con 0x55bb2a609420 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f95bcc740 10 client.33863 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f95bcc740 1 -- 172.21.15.105:0/3952052459 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bb2a684910 con 0x55bb2a609420 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.642+0000 7f1f6bfff640 1 -- 172.21.15.105:0/3952052459 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f78045030 con 0x55bb2a609420 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f6bfff640 1 -- 172.21.15.105:0/3952052459 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f1f78023b30 con 0x55bb2a609420 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f95bcc740 10 client.33863 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f95bcc740 10 client.33863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f95bcc740 20 client.33863 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bb2a685210 0x55bb2a687630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f95bcc740 4 client.33863 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f95bcc740 1 -- 172.21.15.105:0/3952052459 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1f78003c20 con 0x55bb2a685210 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bb2a685210 0x55bb2a687630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f6bfff640 10 client.33863 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.643+0000 7f1f90ca8640 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bb2a685210 0x55bb2a687630 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4941911b3a36148b server_cookie=5f2fa7386659a811 in_seq=0 out_seq=0 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f6bfff640 1 -- 172.21.15.105:0/3952052459 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1f78003c20 con 0x55bb2a685210 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f6bfff640 10 client.33863 handle_command_reply: tid=0 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 client.33863 shutdown 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 -- 172.21.15.105:0/3952052459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 msgr2=0x55bb2a617a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a617a60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1f78011910 tx=0x7f1f7800f480 comp rx=0 tx=0).stop 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 -- 172.21.15.105:0/3952052459 shutdown_connections 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x55bb2a685210 0x55bb2a687630 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f6bfff640 0 client.33863 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55bb2a60e590 0x55bb2a683050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bb2a609420 0x55bb2a617a60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 --2- 172.21.15.105:0/3952052459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55bb2a608a30 0x55bb2a617520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.644+0000 7f1f95bcc740 1 -- 172.21.15.105:0/3952052459 >> 172.21.15.105:0/3952052459 conn(0x55bb2a603fa0 msgr2=0x55bb2a612eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f95bcc740 1 -- 172.21.15.105:0/3952052459 shutdown_connections 2024-03-26T18:02:43.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f95bcc740 1 -- 172.21.15.105:0/3952052459 wait complete. 2024-03-26T18:02:43.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f95bcc740 20 client.33863 trim_cache size 0 max 16384 2024-03-26T18:02:43.631 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 -- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f7c0a2f20 msgr2=0x7f1f7c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f7c0a2f20 0x7f1f7c0a53e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x55bb2a609c80 tx=0x7f1f84009500 comp rx=0 tx=0).stop 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 -- 172.21.15.105:0/919201701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8c1023c0 msgr2=0x7f1f8c101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8c1023c0 0x7f1f8c101d30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1f80006010 tx=0x7f1f80005e40 comp rx=0 tx=0).stop 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 -- 172.21.15.105:0/919201701 shutdown_connections 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f1f7c0a2f20 0x7f1f7c0a53e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1f8c107620 0x7f1f8c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f8c104cf0 0x7f1f8c100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 --2- 172.21.15.105:0/919201701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8c1023c0 0x7f1f8c101d30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.645+0000 7f1f92f0c640 1 -- 172.21.15.105:0/919201701 >> 172.21.15.105:0/919201701 conn(0x7f1f8c0fbd50 msgr2=0x7f1f8c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.646+0000 7f1f92f0c640 1 -- 172.21.15.105:0/919201701 shutdown_connections 2024-03-26T18:02:43.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.646+0000 7f1f92f0c640 1 -- 172.21.15.105:0/919201701 wait complete. 2024-03-26T18:02:43.639 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T18:02:43.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.802+0000 7f592fd7e640 1 -- 172.21.15.105:0/912768473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5928110140 msgr2=0x7f5928112530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.802+0000 7f592fd7e640 1 --2- 172.21.15.105:0/912768473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5928110140 0x7f5928112530 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5920009ca0 tx=0x7f592002c100 comp rx=0 tx=0).stop 2024-03-26T18:02:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.803+0000 7f592fd7e640 1 -- 172.21.15.105:0/912768473 shutdown_connections 2024-03-26T18:02:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.803+0000 7f592fd7e640 1 --2- 172.21.15.105:0/912768473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5928110140 0x7f5928112530 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.803+0000 7f592fd7e640 1 --2- 172.21.15.105:0/912768473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5928109220 0x7f592810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.803+0000 7f592fd7e640 1 --2- 172.21.15.105:0/912768473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59281088c0 0x7f5928108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.803+0000 7f592fd7e640 1 -- 172.21.15.105:0/912768473 >> 172.21.15.105:0/912768473 conn(0x7f592806eac0 msgr2=0x7f5928070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.803+0000 7f592fd7e640 1 -- 172.21.15.105:0/912768473 shutdown_connections 2024-03-26T18:02:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.803+0000 7f592fd7e640 1 -- 172.21.15.105:0/912768473 wait complete. 2024-03-26T18:02:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592fd7e640 1 Processor -- start 2024-03-26T18:02:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592fd7e640 1 -- start start 2024-03-26T18:02:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592fd7e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59281088c0 0x7f59281a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592fd7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5928109220 0x7f59281a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592fd7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5928110140 0x7f59281a7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592fd7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5928115c10 con 0x7f5928110140 2024-03-26T18:02:43.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592fd7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5928115a90 con 0x7f5928109220 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592fd7e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5928115d90 con 0x7f59281088c0 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592d319640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5928109220 0x7f59281a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592db1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59281088c0 0x7f59281a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592db1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59281088c0 0x7f59281a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47578/0 (socket says 172.21.15.105:47578) 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592e31b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5928110140 0x7f59281a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592db1a640 1 -- 172.21.15.105:0/3333236029 learned_addr learned my addr 172.21.15.105:0/3333236029 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592d319640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5928109220 0x7f59281a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32984/0 (socket says 172.21.15.105:32984) 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592db1a640 1 -- 172.21.15.105:0/3333236029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5928109220 msgr2=0x7f59281a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592db1a640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5928109220 0x7f59281a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592db1a640 1 -- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5928110140 msgr2=0x7f59281a7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.804+0000 7f592db1a640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5928110140 0x7f59281a7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.805+0000 7f592db1a640 1 -- 172.21.15.105:0/3333236029 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59281a7cf0 con 0x7f59281088c0 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.805+0000 7f592db1a640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59281088c0 0x7f59281a0120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f592400ea60 tx=0x7f592400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.805+0000 7f591effd640 1 -- 172.21.15.105:0/3333236029 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592400cde0 con 0x7f59281088c0 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.805+0000 7f591effd640 1 -- 172.21.15.105:0/3333236029 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59240035f0 con 0x7f59281088c0 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.805+0000 7f591effd640 1 -- 172.21.15.105:0/3333236029 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924010690 con 0x7f59281088c0 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.805+0000 7f592fd7e640 1 -- 172.21.15.105:0/3333236029 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59281a7fe0 con 0x7f59281088c0 2024-03-26T18:02:43.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.805+0000 7f592fd7e640 1 -- 172.21.15.105:0/3333236029 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59281a8390 con 0x7f59281088c0 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.808+0000 7f593296b740 1 Processor -- start 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f593296b740 1 -- start start 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f593296b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc0a30 0x563cf4bc0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f593296b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc1420 0x563cf4bc5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f593296b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc6590 0x563cf4bc8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f593296b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563cf4b25970 con 0x563cf4bc1420 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f593296b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563cf4acdd70 con 0x563cf4bc6590 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f593296b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563cf4b21920 con 0x563cf4bc0a30 2024-03-26T18:02:43.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f591effd640 1 -- 172.21.15.105:0/3333236029 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f59240037a0 con 0x7f59281088c0 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592d319640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc6590 0x563cf4bc8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592e31b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc0a30 0x563cf4bc0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592d319640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc6590 0x563cf4bc8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:32992/0 (socket says 172.21.15.105:32992) 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592e31b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc0a30 0x563cf4bc0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47584/0 (socket says 172.21.15.105:47584) 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592d319640 1 -- 172.21.15.105:0/4129981460 learned_addr learned my addr 172.21.15.105:0/4129981460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592db1a640 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc1420 0x563cf4bc5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592e31b640 1 -- 172.21.15.105:0/4129981460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc6590 msgr2=0x563cf4bc8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592e31b640 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc6590 0x563cf4bc8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592e31b640 1 -- 172.21.15.105:0/4129981460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc1420 msgr2=0x563cf4bc5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592e31b640 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc1420 0x563cf4bc5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592e31b640 1 -- 172.21.15.105:0/4129981460 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf48393f0 con 0x563cf4bc0a30 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592d319640 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc6590 0x563cf4bc8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:43.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.809+0000 7f592db1a640 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc1420 0x563cf4bc5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.810+0000 7f592e31b640 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc0a30 0x563cf4bc0e50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5920009bf0 tx=0x7f59200093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9607a071c5511c64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.810+0000 7f591effd640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59140a2f20 0x7f59140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.810+0000 7f5913fff640 1 -- 172.21.15.105:0/4129981460 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920043070 con 0x563cf4bc0a30 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.810+0000 7f592db1a640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59140a2f20 0x7f59140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.810+0000 7f5913fff640 1 -- 172.21.15.105:0/4129981460 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5920032dc0 con 0x563cf4bc0a30 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.810+0000 7f5913fff640 1 -- 172.21.15.105:0/4129981460 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920039040 con 0x563cf4bc0a30 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.810+0000 7f591effd640 1 -- 172.21.15.105:0/3333236029 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f59240cf670 con 0x7f59281088c0 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 -- 172.21.15.105:0/4129981460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc0a30 msgr2=0x563cf4bc0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f592db1a640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59140a2f20 0x7f59140a53e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f59240cd4a0 tx=0x7f592400ef90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc0a30 0x563cf4bc0e50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5920009bf0 tx=0x7f59200093f0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 -- 172.21.15.105:0/4129981460 shutdown_connections 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc6590 0x563cf4bc8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc1420 0x563cf4bc5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 --2- 172.21.15.105:0/4129981460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc0a30 0x563cf4bc0e50 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 -- 172.21.15.105:0/4129981460 >> 172.21.15.105:0/4129981460 conn(0x563cf4bbbfa0 msgr2=0x563cf4bbc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 -- 172.21.15.105:0/4129981460 shutdown_connections 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 -- 172.21.15.105:0/4129981460 wait complete. 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 -- start start 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.811+0000 7f593296b740 1 Processor -- start 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f593296b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc0a30 0x563cf4bcaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f593296b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc1420 0x563cf4bcb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f593296b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc6590 0x563cf4bd24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f593296b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563cf4bc94e0 con 0x563cf4bc6590 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f593296b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563cf4ae1310 con 0x563cf4bc0a30 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f593296b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563cf4bc9660 con 0x563cf4bc1420 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f593296b740 10 client.0 _ll_register_callbacks cb 0x563cf4b1cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592d319640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc1420 0x563cf4bcb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592d319640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc1420 0x563cf4bcb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:47592/0 (socket says 172.21.15.105:47592) 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592d319640 1 -- 172.21.15.105:0/1022197419 learned_addr learned my addr 172.21.15.105:0/1022197419 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592e31b640 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc0a30 0x563cf4bcaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592d319640 1 -- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc0a30 msgr2=0x563cf4bcaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592d319640 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc0a30 0x563cf4bcaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592d319640 1 -- 172.21.15.105:0/1022197419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc6590 msgr2=0x563cf4bd24d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592d319640 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc6590 0x563cf4bd24d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592d319640 1 -- 172.21.15.105:0/1022197419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf49037d0 con 0x563cf4bc1420 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.812+0000 7f592e31b640 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc0a30 0x563cf4bcaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f592d319640 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc1420 0x563cf4bcb490 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5918012700 tx=0x7f5918012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5bb8944dfb2b5e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f59117fa640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f59117fa640 1 -- 172.21.15.105:0/1022197419 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591800f940 con 0x563cf4bc1420 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f59117fa640 1 -- 172.21.15.105:0/1022197419 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f591800fae0 con 0x563cf4bc1420 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 1 -- 172.21.15.105:0/1022197419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5920032dc0 con 0x563cf4bc1420 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f59117fa640 1 -- 172.21.15.105:0/1022197419 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f591800fce0 con 0x563cf4bc1420 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 10 client.33880 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 1 -- 172.21.15.105:0/1022197419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563cf4bd4720 con 0x563cf4bc1420 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f59117fa640 1 -- 172.21.15.105:0/1022197419 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5920032dc0 con 0x563cf4bc1420 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f59117fa640 1 -- 172.21.15.105:0/1022197419 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f5918010430 con 0x563cf4bc1420 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 10 client.33880 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 10 client.33880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 20 client.33880 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563cf4bd5e00 0x563cf4bd6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 4 client.33880 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.813+0000 7f593296b740 1 -- 172.21.15.105:0/1022197419 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cf4b32eb0 con 0x563cf4bd5e00 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.814+0000 7f592e31b640 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563cf4bd5e00 0x563cf4bd6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.814+0000 7f59117fa640 10 client.33880 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.814+0000 7f592e31b640 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563cf4bd5e00 0x563cf4bd6250 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1777218cc2047 server_cookie=990b5d6c8e19ef64 in_seq=0 out_seq=0 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f59117fa640 1 -- 172.21.15.105:0/1022197419 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563cf4b32eb0 con 0x563cf4bd5e00 2024-03-26T18:02:43.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f59117fa640 10 client.33880 handle_command_reply: tid=0 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 client.33880 shutdown 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 -- 172.21.15.105:0/1022197419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc1420 msgr2=0x563cf4bcb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc1420 0x563cf4bcb490 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5918012700 tx=0x7f5918012bc0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 -- 172.21.15.105:0/1022197419 shutdown_connections 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563cf4bd5e00 0x563cf4bd6250 crc :-1 s=READY pgs=608 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc6590 0x563cf4bd24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc1420 0x563cf4bcb490 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 --2- 172.21.15.105:0/1022197419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc0a30 0x563cf4bcaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 -- 172.21.15.105:0/1022197419 >> 172.21.15.105:0/1022197419 conn(0x563cf4bbbfa0 msgr2=0x563cf4bbb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f59117fa640 0 client.33880 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 -- 172.21.15.105:0/1022197419 shutdown_connections 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.815+0000 7f593296b740 1 -- 172.21.15.105:0/1022197419 wait complete. 2024-03-26T18:02:43.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.816+0000 7f593296b740 20 client.33880 trim_cache size 0 max 16384 2024-03-26T18:02:43.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f593296b740 1 Processor -- start 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f593296b740 1 -- start start 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f593296b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bb7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f593296b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bd2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f593296b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 0x563cf4bcaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f593296b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59281af130 con 0x563cf4bc0a30 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f593296b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59281b0c70 con 0x563cf4bc1420 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f593296b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59281af800 con 0x563cf4bc6590 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f592e31b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bb7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f592e31b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bb7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48914/0 (socket says 172.21.15.105:48914) 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.833+0000 7f592e31b640 1 -- 172.21.15.105:0/2952405754 learned_addr learned my addr 172.21.15.105:0/2952405754 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f592d319640 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bd2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f592e31b640 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 0x563cf4bcaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f592e31b640 1 -- 172.21.15.105:0/2952405754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 msgr2=0x563cf4bcaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f592e31b640 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 0x563cf4bcaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f592e31b640 1 -- 172.21.15.105:0/2952405754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 msgr2=0x563cf4bd2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f592e31b640 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bd2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f592e31b640 1 -- 172.21.15.105:0/2952405754 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf4bbcd10 con 0x563cf4bc0a30 2024-03-26T18:02:43.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f592e31b640 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bb7050 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f59200323d0 tx=0x7f592002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfb5dd5d7b05f76 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f59137fe640 1 -- 172.21.15.105:0/2952405754 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920039070 con 0x563cf4bc0a30 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f59137fe640 1 -- 172.21.15.105:0/2952405754 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f592004f050 con 0x563cf4bc0a30 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.834+0000 7f59137fe640 1 -- 172.21.15.105:0/2952405754 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920043040 con 0x563cf4bc0a30 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.835+0000 7f593296b740 1 -- 172.21.15.105:0/2952405754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 msgr2=0x563cf4bb7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.835+0000 7f593296b740 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bb7050 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f59200323d0 tx=0x7f592002e040 comp rx=0 tx=0).stop 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.835+0000 7f593296b740 1 -- 172.21.15.105:0/2952405754 shutdown_connections 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.835+0000 7f593296b740 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 0x563cf4bcaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.835+0000 7f593296b740 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bd2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.835+0000 7f593296b740 1 --2- 172.21.15.105:0/2952405754 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bb7050 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.835+0000 7f593296b740 1 -- 172.21.15.105:0/2952405754 >> 172.21.15.105:0/2952405754 conn(0x563cf4bbbfa0 msgr2=0x563cf4bbc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 -- 172.21.15.105:0/2952405754 shutdown_connections 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 -- 172.21.15.105:0/2952405754 wait complete. 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 -- start start 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 Processor -- start 2024-03-26T18:02:43.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bd2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bcece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 0x563cf4bcf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59281b05a0 con 0x563cf4bc0a30 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f59281afed0 con 0x563cf4bc1420 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.836+0000 7f593296b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59281ad110 con 0x563cf4bc6590 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f593296b740 10 client.0 _ll_register_callbacks cb 0x7f5918009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592e31b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bd2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592e31b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bd2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48926/0 (socket says 172.21.15.105:48926) 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592e31b640 1 -- 172.21.15.105:0/3766893947 learned_addr learned my addr 172.21.15.105:0/3766893947 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592d319640 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bcece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592e31b640 1 -- 172.21.15.105:0/3766893947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 msgr2=0x563cf4bcf220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592e31b640 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 0x563cf4bcf220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592e31b640 1 -- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 msgr2=0x563cf4bcece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592e31b640 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bcece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.837+0000 7f592e31b640 1 -- 172.21.15.105:0/3766893947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf4bcf760 con 0x563cf4bc0a30 2024-03-26T18:02:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.838+0000 7f592d319640 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bcece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:02:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.838+0000 7f592e31b640 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bd2020 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f5920032f50 tx=0x7f5920005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1128941aba15d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.838+0000 7f59117fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.838+0000 7f59117fa640 1 -- 172.21.15.105:0/3766893947 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920043030 con 0x563cf4bc0a30 2024-03-26T18:02:43.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.838+0000 7f593296b740 1 -- 172.21.15.105:0/3766893947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592004f050 con 0x563cf4bc0a30 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.838+0000 7f59117fa640 1 -- 172.21.15.105:0/3766893947 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f59200504f0 con 0x563cf4bc0a30 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f59117fa640 1 -- 172.21.15.105:0/3766893947 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5920039040 con 0x563cf4bc0a30 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f593296b740 10 client.24219 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f593296b740 1 -- 172.21.15.105:0/3766893947 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563cf4c44e10 con 0x563cf4bc0a30 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f59117fa640 1 -- 172.21.15.105:0/3766893947 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f592004f050 con 0x563cf4bc0a30 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f59117fa640 1 -- 172.21.15.105:0/3766893947 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f59200484e0 con 0x563cf4bc0a30 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f593296b740 10 client.24219 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f593296b740 10 client.24219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f593296b740 20 client.24219 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f593296b740 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563cf4c451f0 0x563cf4c47610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f593296b740 4 client.24219 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f593296b740 1 -- 172.21.15.105:0/3766893947 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cf4b2d200 con 0x563cf4c451f0 2024-03-26T18:02:43.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.839+0000 7f592d319640 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563cf4c451f0 0x563cf4c47610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:43.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.840+0000 7f592d319640 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563cf4c451f0 0x563cf4c47610 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8362e735970fff9 server_cookie=5c1d96ec44ca0af3 in_seq=0 out_seq=0 2024-03-26T18:02:43.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.840+0000 7f59117fa640 10 client.24219 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f59117fa640 1 -- 172.21.15.105:0/3766893947 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cf4b2d200 con 0x563cf4c451f0 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f59117fa640 10 client.24219 handle_command_reply: tid=0 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 client.24219 shutdown 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 -- 172.21.15.105:0/3766893947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 msgr2=0x563cf4bd2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bd2020 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f5920032f50 tx=0x7f5920005fd0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 -- 172.21.15.105:0/3766893947 shutdown_connections 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x563cf4c451f0 0x563cf4c47610 crc :-1 s=READY pgs=609 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563cf4bc6590 0x563cf4bcf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cf4bc1420 0x563cf4bcece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 --2- 172.21.15.105:0/3766893947 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563cf4bc0a30 0x563cf4bd2020 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 -- 172.21.15.105:0/3766893947 >> 172.21.15.105:0/3766893947 conn(0x563cf4bbbfa0 msgr2=0x563cf4bb7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f59117fa640 0 client.24219 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:43.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 -- 172.21.15.105:0/3766893947 shutdown_connections 2024-03-26T18:02:43.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.841+0000 7f593296b740 1 -- 172.21.15.105:0/3766893947 wait complete. 2024-03-26T18:02:43.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f593296b740 20 client.24219 trim_cache size 0 max 16384 2024-03-26T18:02:43.825 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:02:43.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 -- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59140a2f20 msgr2=0x7f59140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59140a2f20 0x7f59140a53e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f59240cd4a0 tx=0x7f592400ef90 comp rx=0 tx=0).stop 2024-03-26T18:02:43.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 -- 172.21.15.105:0/3333236029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59281088c0 msgr2=0x7f59281a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:43.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59281088c0 0x7f59281a0120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f592400ea60 tx=0x7f592400ef20 comp rx=0 tx=0).stop 2024-03-26T18:02:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 -- 172.21.15.105:0/3333236029 shutdown_connections 2024-03-26T18:02:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f59140a2f20 0x7f59140a53e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5928110140 0x7f59281a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5928109220 0x7f59281a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 --2- 172.21.15.105:0/3333236029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59281088c0 0x7f59281a0120 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 -- 172.21.15.105:0/3333236029 >> 172.21.15.105:0/3333236029 conn(0x7f592806eac0 msgr2=0x7f592810bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 -- 172.21.15.105:0/3333236029 shutdown_connections 2024-03-26T18:02:43.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:43.842+0000 7f592fd7e640 1 -- 172.21.15.105:0/3333236029 wait complete. 2024-03-26T18:02:43.900 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:43 smithi105 ceph-mon[32280]: pgmap v1084: 97 pgs: 97 active+clean; 376 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.9 MiB/s wr, 242 op/s 2024-03-26T18:02:44.153 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:43 smithi135 ceph-mon[35636]: pgmap v1084: 97 pgs: 97 active+clean; 376 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.9 MiB/s wr, 242 op/s 2024-03-26T18:02:44.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:43 smithi144 ceph-mon[36095]: pgmap v1084: 97 pgs: 97 active+clean; 376 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.9 MiB/s wr, 242 op/s 2024-03-26T18:02:44.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 89801 7.02 MB/sec execute 567 sec latency 120.697 ms 2024-03-26T18:02:44.836 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T18:02:44.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:02:44.837 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 -- 172.21.15.105:0/2212529019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 msgr2=0x7ff594109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2212529019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 0x7ff594109d10 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009ca0 tx=0x7ff58c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 -- 172.21.15.105:0/2212529019 shutdown_connections 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2212529019 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 0x7ff594109d10 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2212529019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff594102ac0 0x7ff594102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2212529019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5941018d0 0x7ff594101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 -- 172.21.15.105:0/2212529019 >> 172.21.15.105:0/2212529019 conn(0x7ff5940fce40 msgr2=0x7ff5940ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 -- 172.21.15.105:0/2212529019 shutdown_connections 2024-03-26T18:02:44.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.000+0000 7ff59aee8640 1 -- 172.21.15.105:0/2212529019 wait complete. 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff59aee8640 1 Processor -- start 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff59aee8640 1 -- start start 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff59aee8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5941018d0 0x7ff594197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff59aee8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff594102ac0 0x7ff594198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff59aee8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 0x7ff59419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff59aee8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff59410d400 con 0x7ff594103480 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff59aee8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff59410d280 con 0x7ff5941018d0 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff59aee8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff59410d580 con 0x7ff594102ac0 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff599485640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 0x7ff59419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff599485640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 0x7ff59419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36394/0 (socket says 172.21.15.105:36394) 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff598c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5941018d0 0x7ff594197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff599485640 1 -- 172.21.15.105:0/2004194868 learned_addr learned my addr 172.21.15.105:0/2004194868 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff593fff640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff594102ac0 0x7ff594198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff599485640 1 -- 172.21.15.105:0/2004194868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff594102ac0 msgr2=0x7ff594198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff599485640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff594102ac0 0x7ff594198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff599485640 1 -- 172.21.15.105:0/2004194868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5941018d0 msgr2=0x7ff594197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff599485640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5941018d0 0x7ff594197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff599485640 1 -- 172.21.15.105:0/2004194868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59419f6f0 con 0x7ff594103480 2024-03-26T18:02:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.001+0000 7ff598c84640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5941018d0 0x7ff594197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:44.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.002+0000 7ff599485640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 0x7ff59419f080 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7ff58c032380 tx=0x7ff58c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.004+0000 7ff591ffb640 1 -- 172.21.15.105:0/2004194868 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c041020 con 0x7ff594103480 2024-03-26T18:02:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.004+0000 7ff591ffb640 1 -- 172.21.15.105:0/2004194868 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff58c003e20 con 0x7ff594103480 2024-03-26T18:02:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.004+0000 7ff591ffb640 1 -- 172.21.15.105:0/2004194868 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c039040 con 0x7ff594103480 2024-03-26T18:02:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.004+0000 7ff59aee8640 1 -- 172.21.15.105:0/2004194868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59419f980 con 0x7ff594103480 2024-03-26T18:02:44.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.005+0000 7ff59aee8640 1 -- 172.21.15.105:0/2004194868 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59419fe20 con 0x7ff594103480 2024-03-26T18:02:44.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.007+0000 7ff591ffb640 1 -- 172.21.15.105:0/2004194868 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff58c044020 con 0x7ff594103480 2024-03-26T18:02:44.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff59dbab740 1 Processor -- start 2024-03-26T18:02:44.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff59dbab740 1 -- start start 2024-03-26T18:02:44.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff591ffb640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5840a2fa0 0x7ff5840a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0228f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c02294680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c02297180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff59dbab740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c021f4630 con 0x561c02294cf0 2024-03-26T18:02:44.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff59dbab740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561c0219ce10 con 0x561c0228f190 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff59dbab740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c021f05e0 con 0x561c0228fb80 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff593fff640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5840a2fa0 0x7ff5840a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff591ffb640 1 -- 172.21.15.105:0/2004194868 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff58c0f13b0 con 0x7ff594103480 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff598c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0228f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff598c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0228f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55756/0 (socket says 172.21.15.105:55756) 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff598c84640 1 -- 172.21.15.105:0/2991869248 learned_addr learned my addr 172.21.15.105:0/2991869248 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.008+0000 7ff593fff640 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c02294680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff599485640 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c02297180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff598c84640 1 -- 172.21.15.105:0/2991869248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 msgr2=0x561c02294680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff598c84640 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c02294680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff598c84640 1 -- 172.21.15.105:0/2991869248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 msgr2=0x561c02297180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff598c84640 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c02297180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff598c84640 1 -- 172.21.15.105:0/2991869248 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c021b3740 con 0x561c0228f190 2024-03-26T18:02:44.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff599485640 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c02297180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:44.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff593fff640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5840a2fa0 0x7ff5840a5460 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff594198fe0 tx=0x7ff580009ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:44.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.009+0000 7ff598c84640 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0228f5b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff5880114f0 tx=0x7ff5880119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e023c07395d473ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:44.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.010+0000 7ff57effd640 1 -- 172.21.15.105:0/2991869248 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588004020 con 0x561c0228f190 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.010+0000 7ff57effd640 1 -- 172.21.15.105:0/2991869248 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5880041c0 con 0x561c0228f190 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.010+0000 7ff57effd640 1 -- 172.21.15.105:0/2991869248 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58800f8a0 con 0x561c0228f190 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 -- 172.21.15.105:0/2991869248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 msgr2=0x561c0228f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0228f5b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff5880114f0 tx=0x7ff5880119b0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 -- 172.21.15.105:0/2991869248 shutdown_connections 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c02297180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c02294680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 --2- 172.21.15.105:0/2991869248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0228f5b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 -- 172.21.15.105:0/2991869248 >> 172.21.15.105:0/2991869248 conn(0x561c0228a700 msgr2=0x561c0228ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 -- 172.21.15.105:0/2991869248 shutdown_connections 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.011+0000 7ff59dbab740 1 -- 172.21.15.105:0/2991869248 wait complete. 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 1 -- start start 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 1 Processor -- start 2024-03-26T18:02:44.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0229db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c0229e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c022a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561c0229c150 con 0x561c02294cf0 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561c021b01a0 con 0x561c0228f190 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561c0229c2d0 con 0x561c0228fb80 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff59dbab740 10 client.0 _ll_register_callbacks cb 0x561c01f07c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff598c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0229db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff593fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c0229e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff593fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c0229e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36872/0 (socket says 172.21.15.105:36872) 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff593fff640 1 -- 172.21.15.105:0/3937702919 learned_addr learned my addr 172.21.15.105:0/3937702919 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.012+0000 7ff599485640 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c022a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff593fff640 1 -- 172.21.15.105:0/3937702919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 msgr2=0x561c0229db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff593fff640 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0229db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff593fff640 1 -- 172.21.15.105:0/3937702919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 msgr2=0x561c022a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff593fff640 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c022a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff593fff640 1 -- 172.21.15.105:0/3937702919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c01fd07e0 con 0x561c0228fb80 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff598c84640 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0229db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff593fff640 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c0229e090 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff580004270 tx=0x7ff580004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=133c1731cf3d9fd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff573fff640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff573fff640 1 -- 172.21.15.105:0/3937702919 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580013150 con 0x561c0228fb80 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff59dbab740 1 -- 172.21.15.105:0/3937702919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5880041c0 con 0x561c0228fb80 2024-03-26T18:02:44.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff573fff640 1 -- 172.21.15.105:0/3937702919 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff5800035f0 con 0x561c0228fb80 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.013+0000 7ff573fff640 1 -- 172.21.15.105:0/3937702919 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff5800132f0 con 0x561c0228fb80 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff59dbab740 10 client.33895 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff59dbab740 1 -- 172.21.15.105:0/3937702919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561c02030f70 con 0x561c0228fb80 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff573fff640 1 -- 172.21.15.105:0/3937702919 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff5880041c0 con 0x561c0228fb80 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff573fff640 1 -- 172.21.15.105:0/3937702919 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff580020050 con 0x561c0228fb80 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff59dbab740 10 client.33895 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff59dbab740 10 client.33895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff59dbab740 20 client.33895 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c022a98a0 0x561c022abd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff59dbab740 4 client.33895 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff59dbab740 1 -- 172.21.15.105:0/3937702919 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c02201b70 con 0x561c022a98a0 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.014+0000 7ff598c84640 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c022a98a0 0x561c022abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.015+0000 7ff598c84640 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c022a98a0 0x561c022abd20 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dbf3b6734aeeba6 server_cookie=10950e0f11480fff in_seq=0 out_seq=0 2024-03-26T18:02:44.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.015+0000 7ff573fff640 10 client.33895 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:45.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff573fff640 1 -- 172.21.15.105:0/3937702919 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c02201b70 con 0x561c022a98a0 2024-03-26T18:02:45.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff573fff640 10 client.33895 handle_command_reply: tid=0 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 client.33895 shutdown 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 -- 172.21.15.105:0/3937702919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 msgr2=0x561c0229e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c0229e090 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff580004270 tx=0x7ff580004730 comp rx=0 tx=0).stop 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 -- 172.21.15.105:0/3937702919 shutdown_connections 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c022a98a0 0x561c022abd20 crc :-1 s=READY pgs=610 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff573fff640 0 client.33895 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c022a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c0229e090 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3937702919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c0229db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.016+0000 7ff59dbab740 1 -- 172.21.15.105:0/3937702919 >> 172.21.15.105:0/3937702919 conn(0x561c0228a700 msgr2=0x561c02289900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.017+0000 7ff59dbab740 1 -- 172.21.15.105:0/3937702919 shutdown_connections 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.017+0000 7ff59dbab740 1 -- 172.21.15.105:0/3937702919 wait complete. 2024-03-26T18:02:45.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.017+0000 7ff59dbab740 20 client.33895 trim_cache size 0 max 16384 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff59dbab740 1 Processor -- start 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff59dbab740 1 -- start start 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c02289070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c0228fb80 0x561c022a5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c02294cf0 0x561c022895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff59dbab740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941a6af0 con 0x561c0228fb80 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff59dbab740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941a8630 con 0x561c0228f190 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff59dbab740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941a71c0 con 0x561c02294cf0 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff593fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c0228fb80 0x561c022a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff593fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c0228fb80 0x561c022a5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36404/0 (socket says 172.21.15.105:36404) 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff593fff640 1 -- 172.21.15.105:0/1042040657 learned_addr learned my addr 172.21.15.105:0/1042040657 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff598c84640 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c02289070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff599485640 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c02294cf0 0x561c022895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff593fff640 1 -- 172.21.15.105:0/1042040657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c02294cf0 msgr2=0x561c022895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff593fff640 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c02294cf0 0x561c022895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff593fff640 1 -- 172.21.15.105:0/1042040657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 msgr2=0x561c02289070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff593fff640 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c02289070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.047+0000 7ff593fff640 1 -- 172.21.15.105:0/1042040657 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c01ec8170 con 0x561c0228fb80 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.048+0000 7ff593fff640 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c0228fb80 0x561c022a5020 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7ff580014fd0 tx=0x7ff580009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88184d34468a442c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.048+0000 7ff57e7fc640 1 -- 172.21.15.105:0/1042040657 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580013150 con 0x561c0228fb80 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.048+0000 7ff57e7fc640 1 -- 172.21.15.105:0/1042040657 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff58002f030 con 0x561c0228fb80 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.048+0000 7ff57e7fc640 1 -- 172.21.15.105:0/1042040657 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580003d00 con 0x561c0228fb80 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.048+0000 7ff59dbab740 1 -- 172.21.15.105:0/1042040657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c0228fb80 msgr2=0x561c022a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.048+0000 7ff59dbab740 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c0228fb80 0x561c022a5020 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7ff580014fd0 tx=0x7ff580009e20 comp rx=0 tx=0).stop 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 -- 172.21.15.105:0/1042040657 shutdown_connections 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c02294cf0 0x561c022895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c0228fb80 0x561c022a5020 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 --2- 172.21.15.105:0/1042040657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c02289070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 -- 172.21.15.105:0/1042040657 >> 172.21.15.105:0/1042040657 conn(0x561c0228a700 msgr2=0x561c0229db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 -- 172.21.15.105:0/1042040657 shutdown_connections 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 -- 172.21.15.105:0/1042040657 wait complete. 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 -- start start 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 Processor -- start 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c022a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c022a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.049+0000 7ff59dbab740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c022a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff59dbab740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941a7f60 con 0x561c02294cf0 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff59dbab740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941a7890 con 0x561c0228f190 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff59dbab740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941a4ad0 con 0x561c0228fb80 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff59dbab740 10 client.0 _ll_register_callbacks cb 0x7ff580009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c022a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c022a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55790/0 (socket says 172.21.15.105:55790) 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 -- 172.21.15.105:0/3997752262 learned_addr learned my addr 172.21.15.105:0/3997752262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff593fff640 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c022a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff599485640 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c022a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 -- 172.21.15.105:0/3997752262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 msgr2=0x561c022a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c022a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 -- 172.21.15.105:0/3997752262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 msgr2=0x561c022a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c022a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 -- 172.21.15.105:0/3997752262 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c022a4320 con 0x561c0228f190 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff598c84640 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c022a27e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff588000c00 tx=0x7ff588009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bfe661fc5cd4a36 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff573fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff573fff640 1 -- 172.21.15.105:0/3997752262 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588003d30 con 0x561c0228f190 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff59dbab740 1 -- 172.21.15.105:0/3997752262 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58002f030 con 0x561c0228f190 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.050+0000 7ff573fff640 1 -- 172.21.15.105:0/3997752262 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff588016040 con 0x561c0228f190 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff573fff640 1 -- 172.21.15.105:0/3997752262 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff5880050b0 con 0x561c0228f190 2024-03-26T18:02:45.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff59dbab740 10 client.33893 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff59dbab740 1 -- 172.21.15.105:0/3997752262 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561c0230fd20 con 0x561c0228f190 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff573fff640 1 -- 172.21.15.105:0/3997752262 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff58002f030 con 0x561c0228f190 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff573fff640 1 -- 172.21.15.105:0/3997752262 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff58802e020 con 0x561c0228f190 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff59dbab740 10 client.33893 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff59dbab740 10 client.33893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff59dbab740 20 client.33893 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c022a98a0 0x561c023124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff59dbab740 4 client.33893 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff59dbab740 1 -- 172.21.15.105:0/3997752262 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c021fbec0 con 0x561c022a98a0 2024-03-26T18:02:45.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.051+0000 7ff593fff640 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c022a98a0 0x561c023124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.052+0000 7ff573fff640 10 client.33893 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.052+0000 7ff593fff640 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c022a98a0 0x561c023124f0 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6dcea8767e1ea6e server_cookie=9d676b28ebbd61da in_seq=0 out_seq=0 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff573fff640 1 -- 172.21.15.105:0/3997752262 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c021fbec0 con 0x561c022a98a0 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff573fff640 10 client.33893 handle_command_reply: tid=0 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 client.33893 shutdown 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 -- 172.21.15.105:0/3997752262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 msgr2=0x561c022a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c022a27e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff588000c00 tx=0x7ff588009be0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 -- 172.21.15.105:0/3997752262 shutdown_connections 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561c022a98a0 0x561c023124f0 crc :-1 s=READY pgs=611 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff573fff640 0 client.33893 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561c02294cf0 0x561c022a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561c0228fb80 0x561c022a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 --2- 172.21.15.105:0/3997752262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561c0228f190 0x561c022a27e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.054+0000 7ff59dbab740 1 -- 172.21.15.105:0/3997752262 >> 172.21.15.105:0/3997752262 conn(0x561c0228a700 msgr2=0x561c02289070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.055+0000 7ff59dbab740 1 -- 172.21.15.105:0/3997752262 shutdown_connections 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.055+0000 7ff59dbab740 1 -- 172.21.15.105:0/3997752262 wait complete. 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.055+0000 7ff59dbab740 20 client.33893 trim_cache size 0 max 16384 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T18:02:45.039 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "9b4e28b1-7cd1-4fbb-ac49-f45316ec38fe", 2024-03-26T18:02:45.040 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T18:02:45.040 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:02:45.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 -- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5840a2fa0 msgr2=0x7ff5840a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5840a2fa0 0x7ff5840a5460 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff594198fe0 tx=0x7ff580009ad0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 -- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 msgr2=0x7ff59419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 0x7ff59419f080 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7ff58c032380 tx=0x7ff58c032820 comp rx=0 tx=0).stop 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 -- 172.21.15.105:0/2004194868 shutdown_connections 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff5840a2fa0 0x7ff5840a5460 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff594103480 0x7ff59419f080 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff594102ac0 0x7ff594198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 --2- 172.21.15.105:0/2004194868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff5941018d0 0x7ff594197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 -- 172.21.15.105:0/2004194868 >> 172.21.15.105:0/2004194868 conn(0x7ff5940fce40 msgr2=0x7ff594105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 -- 172.21.15.105:0/2004194868 shutdown_connections 2024-03-26T18:02:45.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.056+0000 7ff59aee8640 1 -- 172.21.15.105:0/2004194868 wait complete. 2024-03-26T18:02:45.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:02:45.049 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:02:45.130 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:45 smithi105 ceph-mon[32280]: pgmap v1085: 97 pgs: 97 active+clean; 377 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-03-26T18:02:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 -- 172.21.15.105:0/1702447883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db0102370 msgr2=0x7f4db0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1702447883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db0102370 0x7f4db0108c00 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009ca0 tx=0x7f4da802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 -- 172.21.15.105:0/1702447883 shutdown_connections 2024-03-26T18:02:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1702447883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db0102370 0x7f4db0108c00 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1702447883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01019b0 0x7f4db0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1702447883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db01007c0 0x7f4db0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 -- 172.21.15.105:0/1702447883 >> 172.21.15.105:0/1702447883 conn(0x7f4db00fbd50 msgr2=0x7f4db00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 -- 172.21.15.105:0/1702447883 shutdown_connections 2024-03-26T18:02:45.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.192+0000 7f4db79ab640 1 -- 172.21.15.105:0/1702447883 wait complete. 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db79ab640 1 Processor -- start 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db79ab640 1 -- start start 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db79ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01007c0 0x7f4db0104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db79ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db01019b0 0x7f4db0102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db79ab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db0102370 0x7f4db01032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db79ab640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4db010c2d0 con 0x7f4db01019b0 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db79ab640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4db010c150 con 0x7f4db0102370 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db79ab640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4db010c450 con 0x7f4db01007c0 2024-03-26T18:02:45.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db5747640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01007c0 0x7f4db0104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.193+0000 7f4db4f46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db01019b0 0x7f4db0102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5747640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01007c0 0x7f4db0104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36922/0 (socket says 172.21.15.105:36922) 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db4f46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db01019b0 0x7f4db0102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36446/0 (socket says 172.21.15.105:36446) 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5747640 1 -- 172.21.15.105:0/1761768535 learned_addr learned my addr 172.21.15.105:0/1761768535 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5f48640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db0102370 0x7f4db01032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5747640 1 -- 172.21.15.105:0/1761768535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db0102370 msgr2=0x7f4db01032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5747640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db0102370 0x7f4db01032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5747640 1 -- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db01019b0 msgr2=0x7f4db0102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5747640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db01019b0 0x7f4db0102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5747640 1 -- 172.21.15.105:0/1761768535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db0103b60 con 0x7f4db01007c0 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5f48640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db0102370 0x7f4db01032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db5747640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01007c0 0x7f4db0104750 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00e9c0 tx=0x7f4dac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4da67fc640 1 -- 172.21.15.105:0/1761768535 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00cd80 con 0x7f4db01007c0 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db79ab640 1 -- 172.21.15.105:0/1761768535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db01a1bd0 con 0x7f4db01007c0 2024-03-26T18:02:45.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4da67fc640 1 -- 172.21.15.105:0/1761768535 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dac0035f0 con 0x7f4db01007c0 2024-03-26T18:02:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4da67fc640 1 -- 172.21.15.105:0/1761768535 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac010690 con 0x7f4db01007c0 2024-03-26T18:02:45.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.194+0000 7f4db79ab640 1 -- 172.21.15.105:0/1761768535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01a2170 con 0x7f4db01007c0 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.197+0000 7f4dba58b740 1 Processor -- start 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.197+0000 7f4dba58b740 1 -- start start 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4da67fc640 1 -- 172.21.15.105:0/1761768535 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f4dac0037a0 con 0x7f4db01007c0 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cc9f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cc9f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4dba58b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5590cc957980 con 0x5590cc9f28f0 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4dba58b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5590cc8ffbe0 con 0x5590cc9f8450 2024-03-26T18:02:45.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4dba58b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5590cc953930 con 0x5590cc9f32e0 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4db5747640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cc9f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4db4f46640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4db5747640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cc9f7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36932/0 (socket says 172.21.15.105:36932) 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4db5747640 1 -- 172.21.15.105:0/2895144386 learned_addr learned my addr 172.21.15.105:0/2895144386 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4db5f48640 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cc9f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4db4f46640 1 -- 172.21.15.105:0/2895144386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 msgr2=0x5590cc9f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4da67fc640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d9c0a2fa0 0x7f4d9c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.198+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cc9f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4db4f46640 1 -- 172.21.15.105:0/2895144386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 msgr2=0x5590cc9f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cc9f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4db4f46640 1 -- 172.21.15.105:0/2895144386 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590cc7357d0 con 0x5590cc9f8450 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4db5f48640 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cc9f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4da67fc640 1 -- 172.21.15.105:0/1761768535 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f4dac0cf830 con 0x7f4db01007c0 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9fa8e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009bf0 tx=0x7f4da0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65014022d62fe152 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:45.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4db5f48640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d9c0a2fa0 0x7f4d9c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4d9b7fe640 1 -- 172.21.15.105:0/2895144386 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0004050 con 0x5590cc9f8450 2024-03-26T18:02:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.199+0000 7f4d9b7fe640 1 -- 172.21.15.105:0/2895144386 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da00054d0 con 0x5590cc9f8450 2024-03-26T18:02:45.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4d9b7fe640 1 -- 172.21.15.105:0/2895144386 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0004d30 con 0x5590cc9f8450 2024-03-26T18:02:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4db5f48640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d9c0a2fa0 0x7f4d9c0a5460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x5590cc9f3b40 tx=0x7f4da80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 -- 172.21.15.105:0/2895144386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 msgr2=0x5590cc9fa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9fa8e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009bf0 tx=0x7f4da0011810 comp rx=0 tx=0).stop 2024-03-26T18:02:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 -- 172.21.15.105:0/2895144386 shutdown_connections 2024-03-26T18:02:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9fa8e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cc9f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2895144386 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cc9f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 -- 172.21.15.105:0/2895144386 >> 172.21.15.105:0/2895144386 conn(0x5590cc9ede60 msgr2=0x5590cc9ee260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 -- 172.21.15.105:0/2895144386 shutdown_connections 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.200+0000 7f4dba58b740 1 -- 172.21.15.105:0/2895144386 wait complete. 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 1 -- start start 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 1 Processor -- start 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f28f0 0x5590cc9fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f32e0 0x5590cc9fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca04030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5590cc9fb450 con 0x5590cc9f32e0 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5590cc913160 con 0x5590cc9f8450 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5590cc9fb5d0 con 0x5590cc9f28f0 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4dba58b740 10 client.0 _ll_register_callbacks cb 0x5590cc94f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4db4f46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f32e0 0x5590cc9fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4db5747640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f28f0 0x5590cc9fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4db4f46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f32e0 0x5590cc9fcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36468/0 (socket says 172.21.15.105:36468) 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4db4f46640 1 -- 172.21.15.105:0/2208941248 learned_addr learned my addr 172.21.15.105:0/2208941248 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.201+0000 7f4db5f48640 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca04030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4db4f46640 1 -- 172.21.15.105:0/2208941248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f28f0 msgr2=0x5590cc9fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f28f0 0x5590cc9fcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4db4f46640 1 -- 172.21.15.105:0/2208941248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 msgr2=0x5590cca04030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca04030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4db4f46640 1 -- 172.21.15.105:0/2208941248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590cc66b3f0 con 0x5590cc9f32e0 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f32e0 0x5590cc9fcff0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f4da001b040 tx=0x7f4da0009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c954da229e2f772 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4d98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2208941248 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0029070 con 0x5590cc9f32e0 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4dba58b740 1 -- 172.21.15.105:0/2208941248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da00054d0 con 0x5590cc9f32e0 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2208941248 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f4da0003a00 con 0x5590cc9f32e0 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2208941248 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4da0035050 con 0x5590cc9f32e0 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4dba58b740 10 client.24249 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:45.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.202+0000 7f4dba58b740 1 -- 172.21.15.105:0/2208941248 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5590cc62b0f0 con 0x5590cc9f32e0 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2208941248 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4da00054d0 con 0x5590cc9f32e0 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2208941248 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f4da0005150 con 0x5590cc9f32e0 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4dba58b740 10 client.24249 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4dba58b740 10 client.24249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4dba58b740 20 client.24249 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5590cca08750 0x5590cca0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4dba58b740 4 client.24249 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4dba58b740 1 -- 172.21.15.105:0/2208941248 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590cc964ec0 con 0x5590cca08750 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.203+0000 7f4db5747640 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5590cca08750 0x5590cca0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.204+0000 7f4d98ff9640 10 client.24249 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:45.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.204+0000 7f4db5747640 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5590cca08750 0x5590cca0abb0 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543e30812ccd654c server_cookie=6c6aad806da0a019 in_seq=0 out_seq=0 2024-03-26T18:02:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.205+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2208941248 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5590cc964ec0 con 0x5590cca08750 2024-03-26T18:02:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.205+0000 7f4d98ff9640 10 client.24249 handle_command_reply: tid=0 2024-03-26T18:02:45.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.205+0000 7f4dba58b740 1 client.24249 shutdown 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.205+0000 7f4dba58b740 1 -- 172.21.15.105:0/2208941248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f32e0 msgr2=0x5590cc9fcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.205+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f32e0 0x5590cc9fcff0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f4da001b040 tx=0x7f4da0009fd0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 1 -- 172.21.15.105:0/2208941248 shutdown_connections 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5590cca08750 0x5590cca0abb0 crc :-1 s=READY pgs=612 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca04030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f32e0 0x5590cc9fcff0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2208941248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f28f0 0x5590cc9fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 1 -- 172.21.15.105:0/2208941248 >> 172.21.15.105:0/2208941248 conn(0x5590cc9ede60 msgr2=0x5590cc9ed080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4d98ff9640 0 client.24249 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 1 -- 172.21.15.105:0/2208941248 shutdown_connections 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 1 -- 172.21.15.105:0/2208941248 wait complete. 2024-03-26T18:02:45.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.206+0000 7f4dba58b740 20 client.24249 trim_cache size 0 max 16384 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4dba58b740 1 Processor -- start 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4dba58b740 1 -- start start 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca0a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cca03ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4dba58b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01ab010 con 0x5590cc9f28f0 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4dba58b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01acb50 con 0x5590cc9f8450 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4dba58b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01ab6e0 con 0x5590cc9f32e0 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4db5747640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca0a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4db5747640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca0a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36470/0 (socket says 172.21.15.105:36470) 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4db5747640 1 -- 172.21.15.105:0/4018600890 learned_addr learned my addr 172.21.15.105:0/4018600890 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:45.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4db5f48640 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.236+0000 7f4db4f46640 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cca03ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4db5747640 1 -- 172.21.15.105:0/4018600890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 msgr2=0x5590cca03ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4db5747640 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cca03ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4db5747640 1 -- 172.21.15.105:0/4018600890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 msgr2=0x5590cc9ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4db5747640 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4db5747640 1 -- 172.21.15.105:0/4018600890 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590cc9f7680 con 0x5590cc9f28f0 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4db5747640 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca0a760 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00cfd0 tx=0x7f4dac0d1d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c84363e6621c80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4d9affd640 1 -- 172.21.15.105:0/4018600890 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac09bf20 con 0x5590cc9f28f0 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4d9affd640 1 -- 172.21.15.105:0/4018600890 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dac0a0070 con 0x5590cc9f28f0 2024-03-26T18:02:45.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.237+0000 7f4d9affd640 1 -- 172.21.15.105:0/4018600890 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac09a2b0 con 0x5590cc9f28f0 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 -- 172.21.15.105:0/4018600890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 msgr2=0x5590cca0a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca0a760 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00cfd0 tx=0x7f4dac0d1d40 comp rx=0 tx=0).stop 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 -- 172.21.15.105:0/4018600890 shutdown_connections 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cc9ec7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cca03ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 --2- 172.21.15.105:0/4018600890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca0a760 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 -- 172.21.15.105:0/4018600890 >> 172.21.15.105:0/4018600890 conn(0x5590cc9ede60 msgr2=0x5590cc9f43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 -- 172.21.15.105:0/4018600890 shutdown_connections 2024-03-26T18:02:45.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.238+0000 7f4dba58b740 1 -- 172.21.15.105:0/4018600890 wait complete. 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 1 -- start start 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 1 Processor -- start 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca01580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cca01ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca02ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01ac480 con 0x5590cc9f28f0 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4db5747640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca01580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01abdb0 con 0x5590cc9f8450 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01a8ff0 con 0x5590cc9f32e0 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4db5747640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca01580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36486/0 (socket says 172.21.15.105:36486) 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4db5747640 1 -- 172.21.15.105:0/2136491362 learned_addr learned my addr 172.21.15.105:0/2136491362 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4dba58b740 10 client.0 _ll_register_callbacks cb 0x7f4da8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cca01ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.239+0000 7f4db5f48640 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca02ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4db5747640 1 -- 172.21.15.105:0/2136491362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 msgr2=0x5590cca01ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4db5747640 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cca01ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4db5747640 1 -- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 msgr2=0x5590cca02ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4db5747640 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca02ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4db5747640 1 -- 172.21.15.105:0/2136491362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590cca030c0 con 0x5590cc9f28f0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4db5f48640 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca02ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4db5747640 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca01580 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f4dac000c00 tx=0x7f4dac09bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58947b57960acf40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4d98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2136491362 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac099fc0 con 0x5590cc9f28f0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2136491362 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f4dac0a44a0 con 0x5590cc9f28f0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4dba58b740 1 -- 172.21.15.105:0/2136491362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dac0a0070 con 0x5590cc9f28f0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2136491362 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4dac09bf20 con 0x5590cc9f28f0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4dba58b740 10 client.24255 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.240+0000 7f4dba58b740 1 -- 172.21.15.105:0/2136491362 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5590cca6eab0 con 0x5590cc9f28f0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2136491362 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dac0a0070 con 0x5590cc9f28f0 2024-03-26T18:02:45.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2136491362 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f4dac09f030 con 0x5590cc9f28f0 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4dba58b740 10 client.24255 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4dba58b740 10 client.24255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4dba58b740 20 client.24255 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5590cca6f270 0x5590cca71690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4dba58b740 4 client.24255 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4dba58b740 1 -- 172.21.15.105:0/2136491362 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5590cc95f210 con 0x5590cca6f270 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5590cca6f270 0x5590cca71690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.241+0000 7f4d98ff9640 10 client.24255 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:45.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.242+0000 7f4db4f46640 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5590cca6f270 0x5590cca71690 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b7143cb4471994 server_cookie=d91ad1877afe227 in_seq=0 out_seq=0 2024-03-26T18:02:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4d98ff9640 1 -- 172.21.15.105:0/2136491362 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5590cc95f210 con 0x5590cca6f270 2024-03-26T18:02:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4d98ff9640 10 client.24255 handle_command_reply: tid=0 2024-03-26T18:02:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 client.24255 shutdown 2024-03-26T18:02:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 -- 172.21.15.105:0/2136491362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 msgr2=0x5590cca01580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca01580 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f4dac000c00 tx=0x7f4dac09bd70 comp rx=0 tx=0).stop 2024-03-26T18:02:45.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 -- 172.21.15.105:0/2136491362 shutdown_connections 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5590cca6f270 0x5590cca71690 crc :-1 s=READY pgs=613 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590cc9f8450 0x5590cca02ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4d98ff9640 0 client.24255 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5590cc9f32e0 0x5590cca01ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 --2- 172.21.15.105:0/2136491362 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5590cc9f28f0 0x5590cca01580 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.244+0000 7f4dba58b740 1 -- 172.21.15.105:0/2136491362 >> 172.21.15.105:0/2136491362 conn(0x5590cc9ede60 msgr2=0x5590cc9f10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.245+0000 7f4dba58b740 1 -- 172.21.15.105:0/2136491362 shutdown_connections 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.245+0000 7f4dba58b740 1 -- 172.21.15.105:0/2136491362 wait complete. 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.245+0000 7f4dba58b740 20 client.24255 trim_cache size 0 max 16384 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (339 inodes in the stack)", 2024-03-26T18:02:45.227 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T18:02:45.228 INFO:teuthology.orchestra.run.smithi105.stdout: "9b4e28b1-7cd1-4fbb-ac49-f45316ec38fe": { 2024-03-26T18:02:45.228 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T18:02:45.228 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "9b4e28b1-7cd1-4fbb-ac49-f45316ec38fe", 2024-03-26T18:02:45.228 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T18:02:45.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:02:45.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:02:45.228 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.245+0000 7f4db79ab640 1 -- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d9c0a2fa0 msgr2=0x7f4d9c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.245+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d9c0a2fa0 0x7f4d9c0a5460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x5590cc9f3b40 tx=0x7f4da80095c0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.245+0000 7f4db79ab640 1 -- 172.21.15.105:0/1761768535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01007c0 msgr2=0x7f4db0104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.245+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01007c0 0x7f4db0104750 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00e9c0 tx=0x7f4dac00ee80 comp rx=0 tx=0).stop 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.246+0000 7f4db79ab640 1 -- 172.21.15.105:0/1761768535 shutdown_connections 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.246+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f4d9c0a2fa0 0x7f4d9c0a5460 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.246+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4db0102370 0x7f4db01032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.246+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4db01019b0 0x7f4db0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.246+0000 7f4db79ab640 1 --2- 172.21.15.105:0/1761768535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4db01007c0 0x7f4db0104750 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.246+0000 7f4db79ab640 1 -- 172.21.15.105:0/1761768535 >> 172.21.15.105:0/1761768535 conn(0x7f4db00fbd50 msgr2=0x7f4db00fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.246+0000 7f4db79ab640 1 -- 172.21.15.105:0/1761768535 shutdown_connections 2024-03-26T18:02:45.229 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:02:45.246+0000 7f4db79ab640 1 -- 172.21.15.105:0/1761768535 wait complete. 2024-03-26T18:02:45.240 INFO:tasks.cephfs.filesystem:scrub status for tag:9b4e28b1-7cd1-4fbb-ac49-f45316ec38fe - {'path': '/', 'tag': '9b4e28b1-7cd1-4fbb-ac49-f45316ec38fe', 'options': 'recursive,force'} 2024-03-26T18:02:45.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:45 smithi144 ceph-mon[36095]: pgmap v1085: 97 pgs: 97 active+clean; 377 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-03-26T18:02:45.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:45 smithi135 ceph-mon[35636]: pgmap v1085: 97 pgs: 97 active+clean; 377 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 3.3 MiB/s wr, 256 op/s 2024-03-26T18:02:45.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 89934 7.03 MB/sec execute 568 sec latency 188.300 ms 2024-03-26T18:02:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T18:02:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:46.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.338 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:46 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:02:46.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:46 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:46.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:46.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:02:46.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:46 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:02:46.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 90127 7.04 MB/sec execute 569 sec latency 109.566 ms 2024-03-26T18:02:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 -- 172.21.15.105:0/2311477043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3340101730 msgr2=0x7f3340101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 --2- 172.21.15.105:0/2311477043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3340101730 0x7f3340101b50 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f3330009900 tx=0x7f333002bc30 comp rx=0 tx=0).stop 2024-03-26T18:02:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 -- 172.21.15.105:0/2311477043 shutdown_connections 2024-03-26T18:02:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 --2- 172.21.15.105:0/2311477043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f33401032e0 0x7f3340109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 --2- 172.21.15.105:0/2311477043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3340102920 0x7f3340102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 --2- 172.21.15.105:0/2311477043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3340101730 0x7f3340101b50 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 -- 172.21.15.105:0/2311477043 >> 172.21.15.105:0/2311477043 conn(0x7f33400fcca0 msgr2=0x7f33400ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 -- 172.21.15.105:0/2311477043 shutdown_connections 2024-03-26T18:02:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.736+0000 7f3344f9d640 1 -- 172.21.15.105:0/2311477043 wait complete. 2024-03-26T18:02:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f3344f9d640 1 Processor -- start 2024-03-26T18:02:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f3344f9d640 1 -- start start 2024-03-26T18:02:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f3344f9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3340101730 0x7f33401979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f3344f9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3340102920 0x7f3340197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f3344f9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33401032e0 0x7f334019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.720 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f3344f9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f334010d250 con 0x7f33401032e0 2024-03-26T18:02:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f3344f9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f334010d0d0 con 0x7f3340102920 2024-03-26T18:02:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f3344f9d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f334010d3d0 con 0x7f3340101730 2024-03-26T18:02:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f333effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3340102920 0x7f3340197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f333effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3340102920 0x7f3340197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55852/0 (socket says 172.21.15.105:55852) 2024-03-26T18:02:46.721 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f333ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33401032e0 0x7f334019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f333effd640 1 -- 172.21.15.105:0/1193188543 learned_addr learned my addr 172.21.15.105:0/1193188543 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.737+0000 7f333f7fe640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3340101730 0x7f33401979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333effd640 1 -- 172.21.15.105:0/1193188543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3340101730 msgr2=0x7f33401979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333effd640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3340101730 0x7f33401979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333effd640 1 -- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33401032e0 msgr2=0x7f334019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.722 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333effd640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33401032e0 0x7f334019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333effd640 1 -- 172.21.15.105:0/1193188543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334019f5f0 con 0x7f3340102920 2024-03-26T18:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333f7fe640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3340101730 0x7f33401979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333effd640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3340102920 0x7f3340197f10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f333400e9f0 tx=0x7f333400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333cff9640 1 -- 172.21.15.105:0/1193188543 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333400cd80 con 0x7f3340102920 2024-03-26T18:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f3344f9d640 1 -- 172.21.15.105:0/1193188543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334019f8e0 con 0x7f3340102920 2024-03-26T18:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333cff9640 1 -- 172.21.15.105:0/1193188543 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33340054d0 con 0x7f3340102920 2024-03-26T18:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f333cff9640 1 -- 172.21.15.105:0/1193188543 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334010490 con 0x7f3340102920 2024-03-26T18:02:46.723 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.738+0000 7f3344f9d640 1 -- 172.21.15.105:0/1193188543 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334019fe30 con 0x7f3340102920 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.741+0000 7f333cff9640 1 -- 172.21.15.105:0/1193188543 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3334005020 con 0x7f3340102920 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.741+0000 7f3348deb740 1 Processor -- start 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.741+0000 7f3348deb740 1 -- start start 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.741+0000 7f333cff9640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33280a2f90 0x7f33280a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f3348deb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc55790 0x55796cc55bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333ffff640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33280a2f90 0x7f33280a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f3348deb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f3348deb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 0x55796cc5d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f3348deb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55796cbba0b0 con 0x55796cc5b2f0 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333cff9640 1 -- 172.21.15.105:0/1193188543 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f33340cf6f0 con 0x7f3340102920 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f3348deb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55796cb62960 con 0x55796cc55790 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f3348deb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55796cbb6060 con 0x55796cc56180 2024-03-26T18:02:46.728 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc55790 0x55796cc55bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 0x55796cc5d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc55790 0x55796cc55bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:55864/0 (socket says 172.21.15.105:55864) 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333f7fe640 1 -- 172.21.15.105:0/535213941 learned_addr learned my addr 172.21.15.105:0/535213941 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333f7fe640 1 -- 172.21.15.105:0/535213941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 msgr2=0x55796cc5ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333f7fe640 1 --2- 172.21.15.105:0/535213941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333f7fe640 1 -- 172.21.15.105:0/535213941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 msgr2=0x55796cc5d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333f7fe640 1 --2- 172.21.15.105:0/535213941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 0x55796cc5d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333f7fe640 1 -- 172.21.15.105:0/535213941 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55796cb74c60 con 0x55796cc55790 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.742+0000 7f333ffff640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33280a2f90 0x7f33280a5450 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3338005fd0 tx=0x7f3338013750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.743+0000 7f333f7fe640 1 --2- 172.21.15.105:0/535213941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc55790 0x55796cc55bb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3330009900 tx=0x7f33300095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5fe639148800b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.743+0000 7f3325ffb640 1 -- 172.21.15.105:0/535213941 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330045020 con 0x55796cc55790 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.743+0000 7f3325ffb640 1 -- 172.21.15.105:0/535213941 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f333002cc90 con 0x55796cc55790 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.743+0000 7f3325ffb640 1 -- 172.21.15.105:0/535213941 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33300048c0 con 0x55796cc55790 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 -- 172.21.15.105:0/535213941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc55790 msgr2=0x55796cc55bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.729 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 --2- 172.21.15.105:0/535213941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc55790 0x55796cc55bb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3330009900 tx=0x7f33300095f0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 -- 172.21.15.105:0/535213941 shutdown_connections 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 --2- 172.21.15.105:0/535213941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 0x55796cc5d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 --2- 172.21.15.105:0/535213941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 --2- 172.21.15.105:0/535213941 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc55790 0x55796cc55bb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 -- 172.21.15.105:0/535213941 >> 172.21.15.105:0/535213941 conn(0x55796cc50d00 msgr2=0x55796cc51100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 -- 172.21.15.105:0/535213941 shutdown_connections 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 -- 172.21.15.105:0/535213941 wait complete. 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 -- start start 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.744+0000 7f3348deb740 1 Processor -- start 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f3348deb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc55790 0x55796cc5f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f3348deb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f3348deb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc66a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f3348deb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55796cc5e240 con 0x55796cc55790 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f3348deb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55796cb75db0 con 0x55796cc5b2f0 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f3348deb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55796cc5e3c0 con 0x55796cc56180 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f3348deb740 10 client.0 _ll_register_callbacks cb 0x55796cb7a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc66a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36974/0 (socket says 172.21.15.105:36974) 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333effd640 1 -- 172.21.15.105:0/3242076075 learned_addr learned my addr 172.21.15.105:0/3242076075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333f7fe640 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc55790 0x55796cc5f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333ffff640 1 -- 172.21.15.105:0/3242076075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 msgr2=0x55796cc5f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333ffff640 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333ffff640 1 -- 172.21.15.105:0/3242076075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc55790 msgr2=0x55796cc5f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333ffff640 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc55790 0x55796cc5f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333ffff640 1 -- 172.21.15.105:0/3242076075 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55796c8ce210 con 0x55796cc5b2f0 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333effd640 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.745+0000 7f333ffff640 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc66a10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3338009240 tx=0x7f3338004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d73160ae7cb759 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:46.731 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f331effd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f331effd640 1 -- 172.21.15.105:0/3242076075 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338029020 con 0x55796cc5b2f0 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f331effd640 1 -- 172.21.15.105:0/3242076075 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f3338004690 con 0x55796cc5b2f0 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 1 -- 172.21.15.105:0/3242076075 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333002cc90 con 0x55796cc5b2f0 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f331effd640 1 -- 172.21.15.105:0/3242076075 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3338034050 con 0x55796cc5b2f0 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 10 client.33920 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 1 -- 172.21.15.105:0/3242076075 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55796c99a620 con 0x55796cc5b2f0 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f331effd640 1 -- 172.21.15.105:0/3242076075 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f333002cc90 con 0x55796cc5b2f0 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f331effd640 1 -- 172.21.15.105:0/3242076075 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f3338020050 con 0x55796cc5b2f0 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 10 client.33920 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 10 client.33920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 20 client.33920 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55796cc6b3a0 0x55796cc6d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 4 client.33920 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.746+0000 7f3348deb740 1 -- 172.21.15.105:0/3242076075 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55796cbc75f0 con 0x55796cc6b3a0 2024-03-26T18:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.747+0000 7f333f7fe640 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55796cc6b3a0 0x55796cc6d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.747+0000 7f331effd640 10 client.33920 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.747+0000 7f333f7fe640 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55796cc6b3a0 0x55796cc6d7b0 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf9360792b3483d server_cookie=7e8409ba2328936 in_seq=0 out_seq=0 2024-03-26T18:02:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f331effd640 1 -- 172.21.15.105:0/3242076075 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55796cbc75f0 con 0x55796cc6b3a0 2024-03-26T18:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f331effd640 10 client.33920 handle_command_reply: tid=0 2024-03-26T18:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 client.33920 shutdown 2024-03-26T18:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 -- 172.21.15.105:0/3242076075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 msgr2=0x55796cc66a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc66a10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3338009240 tx=0x7f3338004ab0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 -- 172.21.15.105:0/3242076075 shutdown_connections 2024-03-26T18:02:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55796cc6b3a0 0x55796cc6d7b0 crc :-1 s=READY pgs=614 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc66a10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc56180 0x55796cc5f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 --2- 172.21.15.105:0/3242076075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc55790 0x55796cc5f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 -- 172.21.15.105:0/3242076075 >> 172.21.15.105:0/3242076075 conn(0x55796cc50d00 msgr2=0x55796cc500a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f331effd640 0 client.33920 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 -- 172.21.15.105:0/3242076075 shutdown_connections 2024-03-26T18:02:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 1 -- 172.21.15.105:0/3242076075 wait complete. 2024-03-26T18:02:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.749+0000 7f3348deb740 20 client.33920 trim_cache size 0 max 16384 2024-03-26T18:02:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.769+0000 7f3348deb740 1 Processor -- start 2024-03-26T18:02:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f3348deb740 1 -- start start 2024-03-26T18:02:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f3348deb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f3348deb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc56180 0x55796cc669d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f3348deb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 0x55796cc56f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f3348deb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33401a6900 con 0x55796cc5b2f0 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f3348deb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33401a8440 con 0x55796cc56180 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f3348deb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33401a6fd0 con 0x55796cc55790 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 0x55796cc56f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc56180 0x55796cc669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc56a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:36980/0 (socket says 172.21.15.105:36980) 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 -- 172.21.15.105:0/400164547 learned_addr learned my addr 172.21.15.105:0/400164547 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 -- 172.21.15.105:0/400164547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc56180 msgr2=0x55796cc669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 --2- 172.21.15.105:0/400164547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc56180 0x55796cc669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 -- 172.21.15.105:0/400164547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 msgr2=0x55796cc56f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 --2- 172.21.15.105:0/400164547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 0x55796cc56f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 -- 172.21.15.105:0/400164547 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55796cc51a70 con 0x55796cc55790 2024-03-26T18:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.770+0000 7f333f7fe640 1 --2- 172.21.15.105:0/400164547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc56a10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3330005ab0 tx=0x7f3330005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1fb409b04b3c827 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f33257fa640 1 -- 172.21.15.105:0/400164547 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330045040 con 0x55796cc55790 2024-03-26T18:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f33257fa640 1 -- 172.21.15.105:0/400164547 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f333002cc90 con 0x55796cc55790 2024-03-26T18:02:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f33257fa640 1 -- 172.21.15.105:0/400164547 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330004bb0 con 0x55796cc55790 2024-03-26T18:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 -- 172.21.15.105:0/400164547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 msgr2=0x55796cc56a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 --2- 172.21.15.105:0/400164547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc56a10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3330005ab0 tx=0x7f3330005b50 comp rx=0 tx=0).stop 2024-03-26T18:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 -- 172.21.15.105:0/400164547 shutdown_connections 2024-03-26T18:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 --2- 172.21.15.105:0/400164547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc5b2f0 0x55796cc56f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 --2- 172.21.15.105:0/400164547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc56180 0x55796cc669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 --2- 172.21.15.105:0/400164547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc56a10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 -- 172.21.15.105:0/400164547 >> 172.21.15.105:0/400164547 conn(0x55796cc50d00 msgr2=0x55796cc5f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 -- 172.21.15.105:0/400164547 shutdown_connections 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.771+0000 7f3348deb740 1 -- 172.21.15.105:0/400164547 wait complete. 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 1 -- start start 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 1 Processor -- start 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc63a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc56180 0x55796cc63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc64f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f33401a7d70 con 0x55796cc56180 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33401a76a0 con 0x55796cc5b2f0 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33401a48e0 con 0x55796cc55790 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f3348deb740 10 client.0 _ll_register_callbacks cb 0x7f33380138f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f333effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc56180 0x55796cc63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f333effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc56180 0x55796cc63f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:36530/0 (socket says 172.21.15.105:36530) 2024-03-26T18:02:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f333f7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc63a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f333effd640 1 -- 172.21.15.105:0/2769197491 learned_addr learned my addr 172.21.15.105:0/2769197491 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.772+0000 7f333ffff640 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc64f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f333effd640 1 -- 172.21.15.105:0/2769197491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 msgr2=0x55796cc63a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f333effd640 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc63a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f333effd640 1 -- 172.21.15.105:0/2769197491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 msgr2=0x55796cc64f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f333effd640 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc64f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f333effd640 1 -- 172.21.15.105:0/2769197491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55796cc65550 con 0x55796cc56180 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f333effd640 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc56180 0x55796cc63f50 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f33340987b0 tx=0x7f333409b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec5b65091759b6a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f331effd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f331effd640 1 -- 172.21.15.105:0/2769197491 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333409bde0 con 0x55796cc56180 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f331effd640 1 -- 172.21.15.105:0/2769197491 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f33340bab50 con 0x55796cc56180 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f3348deb740 1 -- 172.21.15.105:0/2769197491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333002cc90 con 0x55796cc56180 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f331effd640 1 -- 172.21.15.105:0/2769197491 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f333409ff30 con 0x55796cc56180 2024-03-26T18:02:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.773+0000 7f3348deb740 10 client.24273 fetch_fsmap learned FSMap version 45 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f3348deb740 1 -- 172.21.15.105:0/2769197491 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55796cc66930 con 0x55796cc56180 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f331effd640 1 -- 172.21.15.105:0/2769197491 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f333409f040 con 0x55796cc56180 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f331effd640 1 -- 172.21.15.105:0/2769197491 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f3334097b30 con 0x55796cc56180 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f3348deb740 10 client.24273 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f3348deb740 10 client.24273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f3348deb740 20 client.24273 populate_metadata read hostname 'smithi105' 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f3348deb740 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55796ccd1b00 0x55796ccd3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f3348deb740 4 client.24273 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f3348deb740 1 -- 172.21.15.105:0/2769197491 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f333002cc90 con 0x55796ccd1b00 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.774+0000 7f333f7fe640 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55796ccd1b00 0x55796ccd3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:02:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.775+0000 7f333f7fe640 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55796ccd1b00 0x55796ccd3f20 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b68118d0d12944c server_cookie=74c51d679f083670 in_seq=0 out_seq=0 2024-03-26T18:02:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.775+0000 7f331effd640 10 client.24273 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.931+0000 7f331effd640 1 -- 172.21.15.105:0/2769197491 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f333002cc90 con 0x55796ccd1b00 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.931+0000 7f331effd640 10 client.24273 handle_command_reply: tid=0 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.931+0000 7f3348deb740 1 client.24273 shutdown 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 -- 172.21.15.105:0/2769197491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc56180 msgr2=0x55796cc63f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc56180 0x55796cc63f50 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f33340987b0 tx=0x7f333409b870 comp rx=0 tx=0).stop 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 -- 172.21.15.105:0/2769197491 shutdown_connections 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55796ccd1b00 0x55796ccd3f20 crc :-1 s=READY pgs=615 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f331effd640 0 client.24273 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55796cc5b2f0 0x55796cc64f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55796cc56180 0x55796cc63f50 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.913 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 --2- 172.21.15.105:0/2769197491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55796cc55790 0x55796cc63a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 -- 172.21.15.105:0/2769197491 >> 172.21.15.105:0/2769197491 conn(0x55796cc50d00 msgr2=0x55796cc53f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 -- 172.21.15.105:0/2769197491 shutdown_connections 2024-03-26T18:02:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 1 -- 172.21.15.105:0/2769197491 wait complete. 2024-03-26T18:02:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.932+0000 7f3348deb740 20 client.24273 trim_cache size 0 max 16384 2024-03-26T18:02:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:02:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:02:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:02:46.914 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:02:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 -- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33280a2f90 msgr2=0x7f33280a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33280a2f90 0x7f33280a5450 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3338005fd0 tx=0x7f3338013750 comp rx=0 tx=0).stop 2024-03-26T18:02:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 -- 172.21.15.105:0/1193188543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3340102920 msgr2=0x7f3340197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:02:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3340102920 0x7f3340197f10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f333400e9f0 tx=0x7f333400edf0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 -- 172.21.15.105:0/1193188543 shutdown_connections 2024-03-26T18:02:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f33280a2f90 0x7f33280a5450 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33401032e0 0x7f334019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3340102920 0x7f3340197f10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 --2- 172.21.15.105:0/1193188543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3340101730 0x7f33401979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:02:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 -- 172.21.15.105:0/1193188543 >> 172.21.15.105:0/1193188543 conn(0x7f33400fcca0 msgr2=0x7f3340105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:02:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 -- 172.21.15.105:0/1193188543 shutdown_connections 2024-03-26T18:02:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:02:46.933+0000 7f3344f9d640 1 -- 172.21.15.105:0/1193188543 wait complete. 2024-03-26T18:02:47.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:47 smithi144 ceph-mon[36095]: pgmap v1086: 97 pgs: 97 active+clean; 381 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.2 MiB/s wr, 264 op/s 2024-03-26T18:02:47.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:47 smithi105 ceph-mon[32280]: pgmap v1086: 97 pgs: 97 active+clean; 381 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.2 MiB/s wr, 264 op/s 2024-03-26T18:02:47.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:47 smithi135 ceph-mon[35636]: pgmap v1086: 97 pgs: 97 active+clean; 381 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 3.2 MiB/s wr, 264 op/s 2024-03-26T18:02:47.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 90218 7.04 MB/sec execute 570 sec latency 231.231 ms 2024-03-26T18:02:48.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 90322 7.03 MB/sec execute 571 sec latency 109.328 ms 2024-03-26T18:02:49.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:49 smithi144 ceph-mon[36095]: pgmap v1087: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.7 MiB/s wr, 281 op/s 2024-03-26T18:02:49.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:49 smithi105 ceph-mon[32280]: pgmap v1087: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.7 MiB/s wr, 281 op/s 2024-03-26T18:02:49.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:49 smithi135 ceph-mon[35636]: pgmap v1087: 97 pgs: 97 active+clean; 383 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.7 MiB/s wr, 281 op/s 2024-03-26T18:02:49.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 90450 7.04 MB/sec execute 572 sec latency 130.757 ms 2024-03-26T18:02:50.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 90563 7.04 MB/sec execute 573 sec latency 120.445 ms 2024-03-26T18:02:51.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:51 smithi144 ceph-mon[36095]: pgmap v1088: 97 pgs: 97 active+clean; 386 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.5 MiB/s wr, 274 op/s 2024-03-26T18:02:51.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:51 smithi105 ceph-mon[32280]: pgmap v1088: 97 pgs: 97 active+clean; 386 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.5 MiB/s wr, 274 op/s 2024-03-26T18:02:51.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:51 smithi135 ceph-mon[35636]: pgmap v1088: 97 pgs: 97 active+clean; 386 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 3.5 MiB/s wr, 274 op/s 2024-03-26T18:02:51.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 90674 7.04 MB/sec execute 574 sec latency 131.146 ms 2024-03-26T18:02:52.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 90810 7.05 MB/sec execute 575 sec latency 139.947 ms 2024-03-26T18:02:53.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:53 smithi144 ceph-mon[36095]: pgmap v1089: 97 pgs: 97 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-03-26T18:02:53.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:53 smithi105 ceph-mon[32280]: pgmap v1089: 97 pgs: 97 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-03-26T18:02:53.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:53 smithi135 ceph-mon[35636]: pgmap v1089: 97 pgs: 97 active+clean; 389 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 3.6 MiB/s wr, 273 op/s 2024-03-26T18:02:53.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 90962 7.05 MB/sec execute 576 sec latency 161.255 ms 2024-03-26T18:02:54.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91075 7.05 MB/sec execute 577 sec latency 159.211 ms 2024-03-26T18:02:55.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:55 smithi144 ceph-mon[36095]: pgmap v1090: 97 pgs: 97 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-03-26T18:02:55.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:55 smithi105 ceph-mon[32280]: pgmap v1090: 97 pgs: 97 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-03-26T18:02:55.406 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:55 smithi135 ceph-mon[35636]: pgmap v1090: 97 pgs: 97 active+clean; 391 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 3.9 MiB/s wr, 274 op/s 2024-03-26T18:02:55.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91173 7.04 MB/sec execute 578 sec latency 118.713 ms 2024-03-26T18:02:56.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91277 7.04 MB/sec execute 579 sec latency 171.474 ms 2024-03-26T18:02:57.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:56 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:02:57.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:56 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:02:57.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:56 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:02:57.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91377 7.03 MB/sec execute 580 sec latency 138.077 ms 2024-03-26T18:02:58.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:57 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:02:58.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:57 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:02:58.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:57 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:02:58.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:57 smithi144 ceph-mon[36095]: pgmap v1091: 97 pgs: 97 active+clean; 392 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.6 MiB/s wr, 249 op/s 2024-03-26T18:02:58.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:57 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:02:58.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:57 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:02:58.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:57 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:02:58.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:57 smithi105 ceph-mon[32280]: pgmap v1091: 97 pgs: 97 active+clean; 392 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.6 MiB/s wr, 249 op/s 2024-03-26T18:02:58.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:57 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:02:58.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:57 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:02:58.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:57 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:02:58.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:57 smithi135 ceph-mon[35636]: pgmap v1091: 97 pgs: 97 active+clean; 392 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 3.6 MiB/s wr, 249 op/s 2024-03-26T18:02:58.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91466 7.03 MB/sec execute 581 sec latency 134.616 ms 2024-03-26T18:02:59.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91599 7.03 MB/sec execute 582 sec latency 136.932 ms 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 -- 172.21.15.105:0/2849440412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01007e0 msgr2=0x7faed0100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 --2- 172.21.15.105:0/2849440412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01007e0 0x7faed0100c00 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7faecc009900 tx=0x7faecc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 -- 172.21.15.105:0/2849440412 shutdown_connections 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 --2- 172.21.15.105:0/2849440412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed0102390 0x7faed0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 --2- 172.21.15.105:0/2849440412 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed01019d0 0x7faed0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 --2- 172.21.15.105:0/2849440412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01007e0 0x7faed0100c00 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 -- 172.21.15.105:0/2849440412 >> 172.21.15.105:0/2849440412 conn(0x7faed00fbd50 msgr2=0x7faed00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 -- 172.21.15.105:0/2849440412 shutdown_connections 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.058+0000 7faed7285640 1 -- 172.21.15.105:0/2849440412 wait complete. 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed7285640 1 Processor -- start 2024-03-26T18:03:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed7285640 1 -- start start 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed7285640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed01007e0 0x7faed0069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed7285640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01019d0 0x7faed019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed7285640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed0102390 0x7faed019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed7285640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faed010c330 con 0x7faed01019d0 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed7285640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faed010c1b0 con 0x7faed01007e0 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed7285640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faed010c4b0 con 0x7faed0102390 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed4820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01019d0 0x7faed019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed5021640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed01007e0 0x7faed0069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed4820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01019d0 0x7faed019c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50226/0 (socket says 172.21.15.105:50226) 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed4820640 1 -- 172.21.15.105:0/2685918773 learned_addr learned my addr 172.21.15.105:0/2685918773 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed5822640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed0102390 0x7faed019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed4820640 1 -- 172.21.15.105:0/2685918773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed0102390 msgr2=0x7faed019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed4820640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed0102390 0x7faed019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed4820640 1 -- 172.21.15.105:0/2685918773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed01007e0 msgr2=0x7faed0069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.059+0000 7faed4820640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed01007e0 0x7faed0069ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.060+0000 7faed4820640 1 -- 172.21.15.105:0/2685918773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed019f750 con 0x7faed01019d0 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.060+0000 7faed4820640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01019d0 0x7faed019c820 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7faec000e9f0 tx=0x7faec000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.060+0000 7faec67fc640 1 -- 172.21.15.105:0/2685918773 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec000ccc0 con 0x7faed01019d0 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.060+0000 7faed7285640 1 -- 172.21.15.105:0/2685918773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed019f9e0 con 0x7faed01019d0 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.060+0000 7faec67fc640 1 -- 172.21.15.105:0/2685918773 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec00108e0 con 0x7faed01019d0 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.060+0000 7faec67fc640 1 -- 172.21.15.105:0/2685918773 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec00187c0 con 0x7faed01019d0 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.060+0000 7faed7285640 1 -- 172.21.15.105:0/2685918773 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed019ff80 con 0x7faed01019d0 2024-03-26T18:03:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.063+0000 7faec67fc640 1 -- 172.21.15.105:0/2685918773 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7faec0010a90 con 0x7faed01019d0 2024-03-26T18:03:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.064+0000 7faec67fc640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faebc0a2fc0 0x7faebc0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.064+0000 7faed9e77740 1 Processor -- start 2024-03-26T18:03:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.064+0000 7faed5021640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faebc0a2fc0 0x7faebc0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.064+0000 7faec67fc640 1 -- 172.21.15.105:0/2685918773 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7faec00cf810 con 0x7faed01019d0 2024-03-26T18:03:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.064+0000 7faed9e77740 1 -- start start 2024-03-26T18:03:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed9e77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e6a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed9e77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed5021640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faebc0a2fc0 0x7faebc0a5480 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faecc02c240 tx=0x7faecc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed4820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed5021640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e6a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed4820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50240/0 (socket says 172.21.15.105:50240) 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed5021640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e6a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37488/0 (socket says 172.21.15.105:37488) 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed4820640 1 -- 172.21.15.105:0/2301639675 learned_addr learned my addr 172.21.15.105:0/2301639675 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed9e77740 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e70130 0x557bf8e725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed9e77740 1 -- 172.21.15.105:0/2301639675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557bf8dcf0b0 con 0x557bf8e6afc0 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed9e77740 1 -- 172.21.15.105:0/2301639675 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557bf8d77960 con 0x557bf8e70130 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed9e77740 1 -- 172.21.15.105:0/2301639675 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557bf8dcb060 con 0x557bf8e6a5d0 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed5822640 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e70130 0x557bf8e725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed4820640 1 -- 172.21.15.105:0/2301639675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 msgr2=0x557bf8e6a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed4820640 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e6a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed4820640 1 -- 172.21.15.105:0/2301639675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e70130 msgr2=0x557bf8e725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed4820640 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e70130 0x557bf8e725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed4820640 1 -- 172.21.15.105:0/2301639675 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf8d89c60 con 0x557bf8e6afc0 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.065+0000 7faed5021640 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e6a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:00.047 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.066+0000 7faed4820640 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6fac0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7faec009be90 tx=0x7faec000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d368555d80f5ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.066+0000 7faebb7fe640 1 -- 172.21.15.105:0/2301639675 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec00a4910 con 0x557bf8e6afc0 2024-03-26T18:03:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.066+0000 7faebb7fe640 1 -- 172.21.15.105:0/2301639675 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec009a200 con 0x557bf8e6afc0 2024-03-26T18:03:00.048 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.066+0000 7faebb7fe640 1 -- 172.21.15.105:0/2301639675 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0097eb0 con 0x557bf8e6afc0 2024-03-26T18:03:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 -- 172.21.15.105:0/2301639675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 msgr2=0x557bf8e6fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6fac0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7faec009be90 tx=0x7faec000ce60 comp rx=0 tx=0).stop 2024-03-26T18:03:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 -- 172.21.15.105:0/2301639675 shutdown_connections 2024-03-26T18:03:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e70130 0x557bf8e725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6fac0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 --2- 172.21.15.105:0/2301639675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e6a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.050 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 -- 172.21.15.105:0/2301639675 >> 172.21.15.105:0/2301639675 conn(0x557bf8e65b40 msgr2=0x557bf8e65f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 -- 172.21.15.105:0/2301639675 shutdown_connections 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.067+0000 7faed9e77740 1 -- 172.21.15.105:0/2301639675 wait complete. 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 1 -- start start 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 1 Processor -- start 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e74410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8e74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e70130 0x557bf8e7b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x557bf8e730e0 con 0x557bf8e70130 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557bf8d8adb0 con 0x557bf8e6afc0 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557bf8e73260 con 0x557bf8e6a5d0 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed9e77740 10 client.0 _ll_register_callbacks cb 0x557bf8d8f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed5021640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e74410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed5822640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e70130 0x557bf8e7b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.051 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed5021640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e74410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:37490/0 (socket says 172.21.15.105:37490) 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.068+0000 7faed5021640 1 -- 172.21.15.105:0/1804574817 learned_addr learned my addr 172.21.15.105:0/1804574817 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed4820640 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8e74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed4820640 1 -- 172.21.15.105:0/1804574817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 msgr2=0x557bf8e74410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed4820640 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e74410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed4820640 1 -- 172.21.15.105:0/1804574817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e70130 msgr2=0x557bf8e7b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed4820640 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e70130 0x557bf8e7b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed4820640 1 -- 172.21.15.105:0/1804574817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf8ae3210 con 0x557bf8e6afc0 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed5021640 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e74410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed5822640 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e70130 0x557bf8e7b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faed4820640 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8e74950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7faec0023000 tx=0x7faec00097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5957960a251a20c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faeb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:03:00.052 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.069+0000 7faeb8ff9640 1 -- 172.21.15.105:0/1804574817 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0010630 con 0x557bf8e6afc0 2024-03-26T18:03:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.070+0000 7faeb8ff9640 1 -- 172.21.15.105:0/1804574817 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7faec0097c50 con 0x557bf8e6afc0 2024-03-26T18:03:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.070+0000 7faed9e77740 1 -- 172.21.15.105:0/1804574817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec009a200 con 0x557bf8e6afc0 2024-03-26T18:03:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.071+0000 7faeb8ff9640 1 -- 172.21.15.105:0/1804574817 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7faec00a4910 con 0x557bf8e6afc0 2024-03-26T18:03:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.071+0000 7faed9e77740 10 client.33938 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.071+0000 7faed9e77740 1 -- 172.21.15.105:0/1804574817 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557bf8baf620 con 0x557bf8e6afc0 2024-03-26T18:03:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.071+0000 7faeb8ff9640 1 -- 172.21.15.105:0/1804574817 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec009a200 con 0x557bf8e6afc0 2024-03-26T18:03:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.072+0000 7faeb8ff9640 1 -- 172.21.15.105:0/1804574817 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7faec00a13e0 con 0x557bf8e6afc0 2024-03-26T18:03:00.054 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.072+0000 7faed9e77740 10 client.33938 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.072+0000 7faed9e77740 10 client.33938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.072+0000 7faed9e77740 20 client.33938 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.072+0000 7faed9e77740 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557bf8e7fe10 0x557bf8e82200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.072+0000 7faed9e77740 4 client.33938 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.072+0000 7faed9e77740 1 -- 172.21.15.105:0/1804574817 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bf8ddc5f0 con 0x557bf8e7fe10 2024-03-26T18:03:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.072+0000 7faed5021640 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557bf8e7fe10 0x557bf8e82200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.073+0000 7faeb8ff9640 10 client.33938 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:00.055 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.073+0000 7faed5021640 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557bf8e7fe10 0x557bf8e82200 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1534676df24067b server_cookie=89da8c64af065b7f in_seq=0 out_seq=0 2024-03-26T18:03:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.074+0000 7faeb8ff9640 1 -- 172.21.15.105:0/1804574817 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557bf8ddc5f0 con 0x557bf8e7fe10 2024-03-26T18:03:00.056 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.074+0000 7faeb8ff9640 10 client.33938 handle_command_reply: tid=0 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.074+0000 7faed9e77740 1 client.33938 shutdown 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 -- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 msgr2=0x557bf8e74950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8e74950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7faec0023000 tx=0x7faec00097a0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 -- 172.21.15.105:0/1804574817 shutdown_connections 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557bf8e7fe10 0x557bf8e82200 crc :-1 s=READY pgs=616 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e70130 0x557bf8e7b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8e74950 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 --2- 172.21.15.105:0/1804574817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e6a5d0 0x557bf8e74410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faeb8ff9640 0 client.33938 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 -- 172.21.15.105:0/1804574817 >> 172.21.15.105:0/1804574817 conn(0x557bf8e65b40 msgr2=0x557bf8e64df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 -- 172.21.15.105:0/1804574817 shutdown_connections 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 1 -- 172.21.15.105:0/1804574817 wait complete. 2024-03-26T18:03:00.057 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.075+0000 7faed9e77740 20 client.33938 trim_cache size 0 max 16384 2024-03-26T18:03:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed9e77740 1 Processor -- start 2024-03-26T18:03:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed9e77740 1 -- start start 2024-03-26T18:03:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed9e77740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6a5d0 0x557bf8e60c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed9e77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed9e77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 0x557bf8e6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed4820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed4820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50256/0 (socket says 172.21.15.105:50256) 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed9e77740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faed01a8ed0 con 0x557bf8e6afc0 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed4820640 1 -- 172.21.15.105:0/3249395055 learned_addr learned my addr 172.21.15.105:0/3249395055 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed9e77740 1 -- 172.21.15.105:0/3249395055 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faed01aaa10 con 0x557bf8e6a5d0 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed9e77740 1 -- 172.21.15.105:0/3249395055 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faed01a95a0 con 0x557bf8e70130 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.100+0000 7faed5021640 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6a5d0 0x557bf8e60c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faed5822640 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 0x557bf8e6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faed4820640 1 -- 172.21.15.105:0/3249395055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 msgr2=0x557bf8e6bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faed4820640 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 0x557bf8e6bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faed4820640 1 -- 172.21.15.105:0/3249395055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6a5d0 msgr2=0x557bf8e60c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faed4820640 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6a5d0 0x557bf8e60c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.082 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faed4820640 1 -- 172.21.15.105:0/3249395055 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf8e61450 con 0x557bf8e6afc0 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faed4820640 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6ba60 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7faec0098360 tx=0x7faec000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b43c796f0caef1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faebaffd640 1 -- 172.21.15.105:0/3249395055 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec009a200 con 0x557bf8e6afc0 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faebaffd640 1 -- 172.21.15.105:0/3249395055 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faec0010e00 con 0x557bf8e6afc0 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.101+0000 7faebaffd640 1 -- 172.21.15.105:0/3249395055 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec00a11e0 con 0x557bf8e6afc0 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.102+0000 7faed9e77740 1 -- 172.21.15.105:0/3249395055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 msgr2=0x557bf8e6ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.102+0000 7faed9e77740 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6ba60 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7faec0098360 tx=0x7faec000ce60 comp rx=0 tx=0).stop 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.103+0000 7faed9e77740 1 -- 172.21.15.105:0/3249395055 shutdown_connections 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.103+0000 7faed9e77740 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 0x557bf8e6bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.103+0000 7faed9e77740 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6afc0 0x557bf8e6ba60 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.084 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.103+0000 7faed9e77740 1 --2- 172.21.15.105:0/3249395055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6a5d0 0x557bf8e60c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.103+0000 7faed9e77740 1 -- 172.21.15.105:0/3249395055 >> 172.21.15.105:0/3249395055 conn(0x557bf8e65b40 msgr2=0x557bf8e6f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.103+0000 7faed9e77740 1 -- 172.21.15.105:0/3249395055 shutdown_connections 2024-03-26T18:03:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.103+0000 7faed9e77740 1 -- 172.21.15.105:0/3249395055 wait complete. 2024-03-26T18:03:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.104+0000 7faed9e77740 1 -- start start 2024-03-26T18:03:00.085 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.104+0000 7faed9e77740 1 Processor -- start 2024-03-26T18:03:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.104+0000 7faed9e77740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6a5d0 0x557bf8ee12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.104+0000 7faed9e77740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8ee1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.104+0000 7faed9e77740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 0x557bf8ee8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.104+0000 7faed5021640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6a5d0 0x557bf8ee12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.104+0000 7faed5021640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6a5d0 0x557bf8ee12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50272/0 (socket says 172.21.15.105:50272) 2024-03-26T18:03:00.088 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faed01aa340 con 0x557bf8e6a5d0 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed5021640 1 -- 172.21.15.105:0/2492292583 learned_addr learned my addr 172.21.15.105:0/2492292583 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed4820640 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8ee1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faed01a9c70 con 0x557bf8e6afc0 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed5822640 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 0x557bf8ee8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faed01a6eb0 con 0x557bf8e70130 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed9e77740 10 client.0 _ll_register_callbacks cb 0x7faec8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed4820640 1 -- 172.21.15.105:0/2492292583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 msgr2=0x557bf8ee8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed4820640 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 0x557bf8ee8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed4820640 1 -- 172.21.15.105:0/2492292583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6a5d0 msgr2=0x557bf8ee12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed4820640 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6a5d0 0x557bf8ee12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed4820640 1 -- 172.21.15.105:0/2492292583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf8ee8e30 con 0x557bf8e6afc0 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.105+0000 7faed5021640 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6a5d0 0x557bf8ee12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faed4820640 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8ee1800 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faec00a0210 tx=0x7faec00097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=755ce25a74af7e4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:00.089 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faeb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faeb8ff9640 1 -- 172.21.15.105:0/2492292583 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec00a4b60 con 0x557bf8e6afc0 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faeb8ff9640 1 -- 172.21.15.105:0/2492292583 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7faec00a4d00 con 0x557bf8e6afc0 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557bf8dd6940 con 0x557bf8e6afc0 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faeb8ff9640 1 -- 172.21.15.105:0/2492292583 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7faec0010430 con 0x557bf8e6afc0 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faed9e77740 10 client.33944 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557bf8ee8e30 con 0x557bf8e6afc0 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faeb8ff9640 1 -- 172.21.15.105:0/2492292583 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x557bf8dd6940 con 0x557bf8e6afc0 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faeb8ff9640 1 -- 172.21.15.105:0/2492292583 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7faec0018960 con 0x557bf8e6afc0 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faed9e77740 10 client.33944 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.106+0000 7faed9e77740 10 client.33944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.107+0000 7faed9e77740 20 client.33944 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.107+0000 7faed9e77740 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557bf8e7fe10 0x557bf8e79fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:00.090 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.107+0000 7faed9e77740 4 client.33944 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:03:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.107+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557bf8d54770 con 0x557bf8e7fe10 2024-03-26T18:03:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.107+0000 7faed5021640 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557bf8e7fe10 0x557bf8e79fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.107+0000 7faeb8ff9640 10 client.33944 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.107+0000 7faed5021640 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557bf8e7fe10 0x557bf8e79fa0 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98de0fb4523c5006 server_cookie=c4b34aad9b0db8e8 in_seq=0 out_seq=0 2024-03-26T18:03:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faeb8ff9640 1 -- 172.21.15.105:0/2492292583 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557bf8d54770 con 0x557bf8e7fe10 2024-03-26T18:03:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faeb8ff9640 10 client.33944 handle_command_reply: tid=0 2024-03-26T18:03:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 client.33944 shutdown 2024-03-26T18:03:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 msgr2=0x557bf8ee1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8ee1800 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faec00a0210 tx=0x7faec00097a0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 shutdown_connections 2024-03-26T18:03:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x557bf8e7fe10 0x557bf8e79fa0 crc :-1 s=READY pgs=617 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bf8e70130 0x557bf8ee8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557bf8e6afc0 0x557bf8ee1800 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 --2- 172.21.15.105:0/2492292583 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x557bf8e6a5d0 0x557bf8ee12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faeb8ff9640 0 client.33944 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 >> 172.21.15.105:0/2492292583 conn(0x557bf8e65b40 msgr2=0x557bf8e6ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 shutdown_connections 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 1 -- 172.21.15.105:0/2492292583 wait complete. 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.287+0000 7faed9e77740 20 client.33944 trim_cache size 0 max 16384 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:03:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:03:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 -- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faebc0a2fc0 msgr2=0x7faebc0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faebc0a2fc0 0x7faebc0a5480 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faecc02c240 tx=0x7faecc02c7e0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 -- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01019d0 msgr2=0x7faed019c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01019d0 0x7faed019c820 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7faec000e9f0 tx=0x7faec000edf0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 -- 172.21.15.105:0/2685918773 shutdown_connections 2024-03-26T18:03:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7faebc0a2fc0 0x7faebc0a5480 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed0102390 0x7faed019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faed01019d0 0x7faed019c820 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 --2- 172.21.15.105:0/2685918773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed01007e0 0x7faed0069ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 -- 172.21.15.105:0/2685918773 >> 172.21.15.105:0/2685918773 conn(0x7faed00fbd50 msgr2=0x7faed0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 -- 172.21.15.105:0/2685918773 shutdown_connections 2024-03-26T18:03:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:00.288+0000 7faed7285640 1 -- 172.21.15.105:0/2685918773 wait complete. 2024-03-26T18:03:00.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:02:59 smithi144 ceph-mon[36095]: pgmap v1092: 97 pgs: 97 active+clean; 395 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 3.8 MiB/s wr, 230 op/s 2024-03-26T18:03:00.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:02:59 smithi105 ceph-mon[32280]: pgmap v1092: 97 pgs: 97 active+clean; 395 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 3.8 MiB/s wr, 230 op/s 2024-03-26T18:03:00.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:02:59 smithi135 ceph-mon[35636]: pgmap v1092: 97 pgs: 97 active+clean; 395 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 3.8 MiB/s wr, 230 op/s 2024-03-26T18:03:00.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91718 7.03 MB/sec execute 583 sec latency 203.872 ms 2024-03-26T18:03:01.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91839 7.02 MB/sec execute 584 sec latency 153.105 ms 2024-03-26T18:03:02.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:01 smithi144 ceph-mon[36095]: pgmap v1093: 97 pgs: 97 active+clean; 396 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-03-26T18:03:02.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:01 smithi105 ceph-mon[32280]: pgmap v1093: 97 pgs: 97 active+clean; 396 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-03-26T18:03:02.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:01 smithi135 ceph-mon[35636]: pgmap v1093: 97 pgs: 97 active+clean; 396 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 3.3 MiB/s wr, 202 op/s 2024-03-26T18:03:02.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 91981 7.02 MB/sec execute 585 sec latency 140.376 ms 2024-03-26T18:03:03.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:02 smithi144 ceph-mon[36095]: pgmap v1094: 97 pgs: 97 active+clean; 395 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-03-26T18:03:03.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:02 smithi105 ceph-mon[32280]: pgmap v1094: 97 pgs: 97 active+clean; 395 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-03-26T18:03:03.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:02 smithi135 ceph-mon[35636]: pgmap v1094: 97 pgs: 97 active+clean; 395 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-03-26T18:03:03.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 92093 7.01 MB/sec execute 586 sec latency 267.593 ms 2024-03-26T18:03:04.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 92221 7.02 MB/sec execute 587 sec latency 125.620 ms 2024-03-26T18:03:05.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:05 smithi144 ceph-mon[36095]: pgmap v1095: 97 pgs: 97 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-03-26T18:03:05.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:05 smithi105 ceph-mon[32280]: pgmap v1095: 97 pgs: 97 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-03-26T18:03:05.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:05 smithi135 ceph-mon[35636]: pgmap v1095: 97 pgs: 97 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 3.3 MiB/s wr, 233 op/s 2024-03-26T18:03:05.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 92350 7.01 MB/sec execute 588 sec latency 125.590 ms 2024-03-26T18:03:06.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 92468 7.01 MB/sec execute 589 sec latency 124.147 ms 2024-03-26T18:03:07.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:07 smithi144 ceph-mon[36095]: pgmap v1096: 97 pgs: 97 active+clean; 383 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.1 MiB/s wr, 244 op/s 2024-03-26T18:03:07.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:07 smithi105 ceph-mon[32280]: pgmap v1096: 97 pgs: 97 active+clean; 383 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.1 MiB/s wr, 244 op/s 2024-03-26T18:03:07.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:07 smithi135 ceph-mon[35636]: pgmap v1096: 97 pgs: 97 active+clean; 383 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 3.1 MiB/s wr, 244 op/s 2024-03-26T18:03:07.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 92605 7.02 MB/sec execute 590 sec latency 151.234 ms 2024-03-26T18:03:08.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 92782 7.02 MB/sec execute 591 sec latency 149.474 ms 2024-03-26T18:03:09.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:09 smithi144 ceph-mon[36095]: pgmap v1097: 97 pgs: 97 active+clean; 382 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-03-26T18:03:09.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:09 smithi105 ceph-mon[32280]: pgmap v1097: 97 pgs: 97 active+clean; 382 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-03-26T18:03:09.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:09 smithi135 ceph-mon[35636]: pgmap v1097: 97 pgs: 97 active+clean; 382 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 3.3 MiB/s wr, 260 op/s 2024-03-26T18:03:09.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 92939 7.03 MB/sec execute 592 sec latency 118.554 ms 2024-03-26T18:03:10.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 93047 7.02 MB/sec execute 593 sec latency 135.273 ms 2024-03-26T18:03:11.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:11 smithi144 ceph-mon[36095]: pgmap v1098: 97 pgs: 97 active+clean; 383 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-03-26T18:03:11.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:11 smithi105 ceph-mon[32280]: pgmap v1098: 97 pgs: 97 active+clean; 383 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-03-26T18:03:11.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:11 smithi135 ceph-mon[35636]: pgmap v1098: 97 pgs: 97 active+clean; 383 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-03-26T18:03:11.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 93202 7.02 MB/sec execute 594 sec latency 108.286 ms 2024-03-26T18:03:12.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 93344 7.02 MB/sec execute 595 sec latency 114.485 ms 2024-03-26T18:03:13.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:13 smithi144 ceph-mon[36095]: pgmap v1099: 97 pgs: 97 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-03-26T18:03:13.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:13 smithi105 ceph-mon[32280]: pgmap v1099: 97 pgs: 97 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-03-26T18:03:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.402+0000 7fdbf50b2640 1 -- 172.21.15.105:0/1680011006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0104cf0 msgr2=0x7fdbf01070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.402+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/1680011006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0104cf0 0x7fdbf01070e0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009900 tx=0x7fdbd802bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.402+0000 7fdbf50b2640 1 -- 172.21.15.105:0/1680011006 shutdown_connections 2024-03-26T18:03:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.402+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/1680011006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf0107620 0x7fdbf0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.402+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/1680011006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0104cf0 0x7fdbf01070e0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.402+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/1680011006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf01023c0 0x7fdbf01047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.402+0000 7fdbf50b2640 1 -- 172.21.15.105:0/1680011006 >> 172.21.15.105:0/1680011006 conn(0x7fdbf00fbd50 msgr2=0x7fdbf00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.403+0000 7fdbf50b2640 1 -- 172.21.15.105:0/1680011006 shutdown_connections 2024-03-26T18:03:13.385 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.403+0000 7fdbf50b2640 1 -- 172.21.15.105:0/1680011006 wait complete. 2024-03-26T18:03:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.403+0000 7fdbf50b2640 1 Processor -- start 2024-03-26T18:03:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbf50b2640 1 -- start start 2024-03-26T18:03:13.386 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbf50b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf01023c0 0x7fdbf019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbf50b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf0104cf0 0x7fdbf019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbf50b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0107620 0x7fdbf01a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbf50b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf0101e70 con 0x7fdbf0107620 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbf50b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf0101cf0 con 0x7fdbf0104cf0 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbf50b2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf0101ff0 con 0x7fdbf01023c0 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbef59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0107620 0x7fdbf01a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf01023c0 0x7fdbf019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf0104cf0 0x7fdbf019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf01023c0 0x7fdbf019e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39008/0 (socket says 172.21.15.105:39008) 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbef59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0107620 0x7fdbf01a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37414/0 (socket says 172.21.15.105:37414) 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.404+0000 7fdbeed9d640 1 -- 172.21.15.105:0/2031948963 learned_addr learned my addr 172.21.15.105:0/2031948963 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbeed9d640 1 -- 172.21.15.105:0/2031948963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf0104cf0 msgr2=0x7fdbf019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf0104cf0 0x7fdbf019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbeed9d640 1 -- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0107620 msgr2=0x7fdbf01a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0107620 0x7fdbf01a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbeed9d640 1 -- 172.21.15.105:0/2031948963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf01a5d40 con 0x7fdbf01023c0 2024-03-26T18:03:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf01023c0 0x7fdbf019e120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e9f0 tx=0x7fdbe000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbe7fff640 1 -- 172.21.15.105:0/2031948963 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000ccc0 con 0x7fdbf01023c0 2024-03-26T18:03:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbe7fff640 1 -- 172.21.15.105:0/2031948963 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe000ce60 con 0x7fdbf01023c0 2024-03-26T18:03:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.405+0000 7fdbe7fff640 1 -- 172.21.15.105:0/2031948963 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0010430 con 0x7fdbf01023c0 2024-03-26T18:03:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.406+0000 7fdbf50b2640 1 -- 172.21.15.105:0/2031948963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf01a5fd0 con 0x7fdbf01023c0 2024-03-26T18:03:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.406+0000 7fdbf50b2640 1 -- 172.21.15.105:0/2031948963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf01a63e0 con 0x7fdbf01023c0 2024-03-26T18:03:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.409+0000 7fdbe7fff640 1 -- 172.21.15.105:0/2031948963 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fdbe0005020 con 0x7fdbf01023c0 2024-03-26T18:03:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.410+0000 7fdbe7fff640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbdc0a30c0 0x7fdbdc0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.410+0000 7fdbee59c640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbdc0a30c0 0x7fdbdc0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.410+0000 7fdbe7fff640 1 -- 172.21.15.105:0/2031948963 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fdbe00cf790 con 0x7fdbf01023c0 2024-03-26T18:03:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.411+0000 7fdbf7d7c740 1 Processor -- start 2024-03-26T18:03:13.392 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.411+0000 7fdbf7d7c740 1 -- start start 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.411+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e278f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.411+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e278fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.411+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278fd130 0x561e278ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbf7d7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e2785c0b0 con 0x561e278fd130 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbf7d7c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e27804960 con 0x561e278f7fc0 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbf7d7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e27858060 con 0x561e278f75d0 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbee59c640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbdc0a30c0 0x7fdbdc0a5580 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdbd802c240 tx=0x7fdbd802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbef59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278fd130 0x561e278ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e278f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e278f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39016/0 (socket says 172.21.15.105:39016) 2024-03-26T18:03:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 -- 172.21.15.105:0/339029040 learned_addr learned my addr 172.21.15.105:0/339029040 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbee59c640 1 --2- 172.21.15.105:0/339029040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e278fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 -- 172.21.15.105:0/339029040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 msgr2=0x561e278fcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/339029040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e278fcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 -- 172.21.15.105:0/339029040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278fd130 msgr2=0x561e278ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/339029040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278fd130 0x561e278ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 -- 172.21.15.105:0/339029040 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e27816c60 con 0x561e278f75d0 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.412+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/339029040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e278f79f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00a4300 tx=0x7fdbe009c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abd0b867f8b5437e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.413+0000 7fdbe57fa640 1 -- 172.21.15.105:0/339029040 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00a4830 con 0x561e278f75d0 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.413+0000 7fdbe57fa640 1 -- 172.21.15.105:0/339029040 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe00cd300 con 0x561e278f75d0 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.413+0000 7fdbe57fa640 1 -- 172.21.15.105:0/339029040 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00a43a0 con 0x561e278f75d0 2024-03-26T18:03:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/339029040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 msgr2=0x561e278f79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/339029040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e278f79f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00a4300 tx=0x7fdbe009c520 comp rx=0 tx=0).stop 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/339029040 shutdown_connections 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/339029040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278fd130 0x561e278ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/339029040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e278fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/339029040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e278f79f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/339029040 >> 172.21.15.105:0/339029040 conn(0x561e278f2b40 msgr2=0x561e278f2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/339029040 shutdown_connections 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.414+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/339029040 wait complete. 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 1 -- start start 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 1 Processor -- start 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e27901490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 0x561e27908a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x561e27900180 con 0x561e278f75d0 2024-03-26T18:03:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561e27817db0 con 0x561e278f7fc0 2024-03-26T18:03:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561e27900300 con 0x561e278fd130 2024-03-26T18:03:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbf7d7c740 10 client.0 _ll_register_callbacks cb 0x561e2781c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e27901490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e27901490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37432/0 (socket says 172.21.15.105:37432) 2024-03-26T18:03:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbeed9d640 1 -- 172.21.15.105:0/482424043 learned_addr learned my addr 172.21.15.105:0/482424043 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.415+0000 7fdbee59c640 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbef59e640 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 0x561e27908a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbeed9d640 1 -- 172.21.15.105:0/482424043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 msgr2=0x561e27908a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 0x561e27908a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbeed9d640 1 -- 172.21.15.105:0/482424043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 msgr2=0x561e279019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbeed9d640 1 -- 172.21.15.105:0/482424043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e27570210 con 0x561e278f75d0 2024-03-26T18:03:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e27901490 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00987f0 tx=0x7fdbe00048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e82c5025e2a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.416+0000 7fdbd67fc640 1 -- 172.21.15.105:0/482424043 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00b9070 con 0x561e278f75d0 2024-03-26T18:03:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.418+0000 7fdbd67fc640 1 -- 172.21.15.105:0/482424043 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fdbe00a45b0 con 0x561e278f75d0 2024-03-26T18:03:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.418+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/482424043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe00cd300 con 0x561e278f75d0 2024-03-26T18:03:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.418+0000 7fdbd67fc640 1 -- 172.21.15.105:0/482424043 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdbe00a4830 con 0x561e278f75d0 2024-03-26T18:03:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.418+0000 7fdbf7d7c740 10 client.24303 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.418+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/482424043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561e2763c620 con 0x561e278f75d0 2024-03-26T18:03:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbd67fc640 1 -- 172.21.15.105:0/482424043 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe00bab90 con 0x561e278f75d0 2024-03-26T18:03:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbd67fc640 1 -- 172.21.15.105:0/482424043 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fdbe009ace0 con 0x561e278f75d0 2024-03-26T18:03:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbf7d7c740 10 client.24303 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbf7d7c740 10 client.24303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbf7d7c740 20 client.24303 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e2790cf70 0x561e2790f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbf7d7c740 4 client.24303 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/482424043 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdbe00cd300 con 0x561e2790cf70 2024-03-26T18:03:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.419+0000 7fdbee59c640 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e2790cf70 0x561e2790f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.420+0000 7fdbee59c640 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e2790cf70 0x561e2790f360 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f7aec82c0c8d7a server_cookie=2c67ae75e43f80f5 in_seq=0 out_seq=0 2024-03-26T18:03:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.420+0000 7fdbd67fc640 10 client.24303 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.421+0000 7fdbd67fc640 1 -- 172.21.15.105:0/482424043 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fdbe00cd300 con 0x561e2790cf70 2024-03-26T18:03:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.421+0000 7fdbd67fc640 10 client.24303 handle_command_reply: tid=0 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.421+0000 7fdbf7d7c740 1 client.24303 shutdown 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.421+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/482424043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 msgr2=0x561e27901490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.421+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e27901490 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00987f0 tx=0x7fdbe00048d0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/482424043 shutdown_connections 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e2790cf70 0x561e2790f360 crc :-1 s=READY pgs=618 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 0x561e27908a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbd67fc640 0 client.24303 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/482424043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e27901490 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/482424043 >> 172.21.15.105:0/482424043 conn(0x561e278f2b40 msgr2=0x561e278f1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/482424043 shutdown_connections 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/482424043 wait complete. 2024-03-26T18:03:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.422+0000 7fdbf7d7c740 20 client.24303 trim_cache size 0 max 16384 2024-03-26T18:03:13.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:13 smithi135 ceph-mon[35636]: pgmap v1099: 97 pgs: 97 active+clean; 385 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-03-26T18:03:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.449+0000 7fdbf7d7c740 1 Processor -- start 2024-03-26T18:03:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.449+0000 7fdbf7d7c740 1 -- start start 2024-03-26T18:03:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.449+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e279010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.449+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 0x561e279015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e279010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e279010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37436/0 (socket says 172.21.15.105:37436) 2024-03-26T18:03:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:59658/0 (socket says 172.21.15.105:59658) 2024-03-26T18:03:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbeed9d640 1 -- 172.21.15.105:0/1385551435 learned_addr learned my addr 172.21.15.105:0/1385551435 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/1385551435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01af3a0 con 0x561e278f75d0 2024-03-26T18:03:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/1385551435 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01b0ee0 con 0x561e278f7fc0 2024-03-26T18:03:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbef59e640 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 0x561e279015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/1385551435 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01afa70 con 0x561e278fd130 2024-03-26T18:03:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbee59c640 1 -- 172.21.15.105:0/1385551435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 msgr2=0x561e279015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbee59c640 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 0x561e279015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbee59c640 1 -- 172.21.15.105:0/1385551435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 msgr2=0x561e279010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.450+0000 7fdbee59c640 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e279010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.451+0000 7fdbee59c640 1 -- 172.21.15.105:0/1385551435 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e278ee450 con 0x561e278f7fc0 2024-03-26T18:03:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.451+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e279010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.451+0000 7fdbee59c640 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279089d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8036040 tx=0x7fdbd8043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b65504cd360b541f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.451+0000 7fdbe4ff9640 1 -- 172.21.15.105:0/1385551435 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8055b60 con 0x561e278f7fc0 2024-03-26T18:03:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.451+0000 7fdbe4ff9640 1 -- 172.21.15.105:0/1385551435 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbd80557f0 con 0x561e278f7fc0 2024-03-26T18:03:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.451+0000 7fdbe4ff9640 1 -- 172.21.15.105:0/1385551435 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80046c0 con 0x561e278f7fc0 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.452+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/1385551435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 msgr2=0x561e279089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.452+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279089d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8036040 tx=0x7fdbd8043000 comp rx=0 tx=0).stop 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.452+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/1385551435 shutdown_connections 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.452+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278fd130 0x561e279015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.452+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278f7fc0 0x561e279089d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.452+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/1385551435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f75d0 0x561e279010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.453+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/1385551435 >> 172.21.15.105:0/1385551435 conn(0x561e278f2b40 msgr2=0x561e278fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.453+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/1385551435 shutdown_connections 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.453+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/1385551435 wait complete. 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.453+0000 7fdbf7d7c740 1 -- start start 2024-03-26T18:03:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.453+0000 7fdbf7d7c740 1 Processor -- start 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e279057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f7fc0 0x561e27905d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbf7d7c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278fd130 0x561e279711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e279057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbf7d7c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01b0810 con 0x561e278f7fc0 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbeed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e279057c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:39042/0 (socket says 172.21.15.105:39042) 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbeed9d640 1 -- 172.21.15.105:0/4187066979 learned_addr learned my addr 172.21.15.105:0/4187066979 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbee59c640 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f7fc0 0x561e27905d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbf7d7c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01b0140 con 0x561e278fd130 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf01ad380 con 0x561e278f75d0 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbf7d7c740 10 client.0 _ll_register_callbacks cb 0x7fdbe8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbef59e640 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278fd130 0x561e279711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbee59c640 1 -- 172.21.15.105:0/4187066979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 msgr2=0x561e279057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbee59c640 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e279057c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbee59c640 1 -- 172.21.15.105:0/4187066979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278fd130 msgr2=0x561e279711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbee59c640 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278fd130 0x561e279711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.454+0000 7fdbee59c640 1 -- 172.21.15.105:0/4187066979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e27971710 con 0x561e278f7fc0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.455+0000 7fdbee59c640 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f7fc0 0x561e27905d00 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8004650 tx=0x7fdbd80053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475b875976b93c2a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.455+0000 7fdbd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.455+0000 7fdbd67fc640 1 -- 172.21.15.105:0/4187066979 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd803a040 con 0x561e278f7fc0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.455+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd80557f0 con 0x561e278f7fc0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.455+0000 7fdbd67fc640 1 -- 172.21.15.105:0/4187066979 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fdbd8055340 con 0x561e278f7fc0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.455+0000 7fdbd67fc640 1 -- 172.21.15.105:0/4187066979 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdbd8055540 con 0x561e278f7fc0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.455+0000 7fdbf7d7c740 10 client.24309 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.455+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561e279728f0 con 0x561e278f7fc0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbd67fc640 1 -- 172.21.15.105:0/4187066979 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbd80557f0 con 0x561e278f7fc0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbd67fc640 1 -- 172.21.15.105:0/4187066979 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fdbd8003850 con 0x561e278f7fc0 2024-03-26T18:03:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbf7d7c740 10 client.24309 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbf7d7c740 10 client.24309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbf7d7c740 20 client.24309 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e2790cf70 0x561e27975270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbf7d7c740 4 client.24309 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T18:03:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e278695f0 con 0x561e2790cf70 2024-03-26T18:03:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.456+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e2790cf70 0x561e27975270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.457+0000 7fdbd67fc640 10 client.24309 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.457+0000 7fdbeed9d640 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e2790cf70 0x561e27975270 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a00ef1bfea0ce6b server_cookie=45b951e7fc7cb32e in_seq=0 out_seq=0 2024-03-26T18:03:13.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 93437 7.01 MB/sec execute 596 sec latency 149.489 ms 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbd67fc640 1 -- 172.21.15.105:0/4187066979 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e278695f0 con 0x561e2790cf70 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbd67fc640 10 client.24309 handle_command_reply: tid=0 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 client.24309 shutdown 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f7fc0 msgr2=0x561e27905d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f7fc0 0x561e27905d00 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8004650 tx=0x7fdbd80053b0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 shutdown_connections 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x561e2790cf70 0x561e27975270 crc :-1 s=READY pgs=619 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561e278fd130 0x561e279711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x561e278f7fc0 0x561e27905d00 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 --2- 172.21.15.105:0/4187066979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561e278f75d0 0x561e279057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 >> 172.21.15.105:0/4187066979 conn(0x561e278f2b40 msgr2=0x561e27901010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbd67fc640 0 client.24309 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 shutdown_connections 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 1 -- 172.21.15.105:0/4187066979 wait complete. 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.602+0000 7fdbf7d7c740 20 client.24309 trim_cache size 0 max 16384 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:{ 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "message": "", 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout: "return_code": 0 2024-03-26T18:03:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stdout:} 2024-03-26T18:03:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.603+0000 7fdbf50b2640 1 -- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbdc0a30c0 msgr2=0x7fdbdc0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.603+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbdc0a30c0 0x7fdbdc0a5580 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdbd802c240 tx=0x7fdbd802c7e0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.603+0000 7fdbf50b2640 1 -- 172.21.15.105:0/2031948963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf01023c0 msgr2=0x7fdbf019e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.603+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf01023c0 0x7fdbf019e120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e9f0 tx=0x7fdbe000edf0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.604+0000 7fdbf50b2640 1 -- 172.21.15.105:0/2031948963 shutdown_connections 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.604+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fdbdc0a30c0 0x7fdbdc0a5580 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.604+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdbf0107620 0x7fdbf01a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.604+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbf0104cf0 0x7fdbf019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.604+0000 7fdbf50b2640 1 --2- 172.21.15.105:0/2031948963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdbf01023c0 0x7fdbf019e120 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.604+0000 7fdbf50b2640 1 -- 172.21.15.105:0/2031948963 >> 172.21.15.105:0/2031948963 conn(0x7fdbf00fbd50 msgr2=0x7fdbf0102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.604+0000 7fdbf50b2640 1 -- 172.21.15.105:0/2031948963 shutdown_connections 2024-03-26T18:03:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi105.front.sepia.ceph.com.smithi105.stderr:2024-03-26T18:03:13.604+0000 7fdbf50b2640 1 -- 172.21.15.105:0/2031948963 wait complete. 2024-03-26T18:03:14.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 93564 7.01 MB/sec execute 597 sec latency 194.041 ms 2024-03-26T18:03:15.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:03:15.241 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:03:15.273 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:15 smithi105 ceph-mon[32280]: pgmap v1100: 97 pgs: 97 active+clean; 386 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-03-26T18:03:15.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:15 smithi144 ceph-mon[36095]: pgmap v1100: 97 pgs: 97 active+clean; 386 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-03-26T18:03:15.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.405+0000 7ff6d6436640 1 -- 172.21.15.105:0/2659851200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 msgr2=0x7ff6d01018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.405+0000 7ff6d6436640 1 --2- 172.21.15.105:0/2659851200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 0x7ff6d01018f0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009ca0 tx=0x7ff6c802c140 comp rx=0 tx=0).stop 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 -- 172.21.15.105:0/2659851200 shutdown_connections 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 --2- 172.21.15.105:0/2659851200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 0x7ff6d01018f0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 --2- 172.21.15.105:0/2659851200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d0100ad0 0x7ff6d0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 --2- 172.21.15.105:0/2659851200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d0100170 0x7ff6d0100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 -- 172.21.15.105:0/2659851200 >> 172.21.15.105:0/2659851200 conn(0x7ff6d00fbd30 msgr2=0x7ff6d00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 -- 172.21.15.105:0/2659851200 shutdown_connections 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 -- 172.21.15.105:0/2659851200 wait complete. 2024-03-26T18:03:15.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 Processor -- start 2024-03-26T18:03:15.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.406+0000 7ff6d6436640 1 -- start start 2024-03-26T18:03:15.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d6436640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d0100170 0x7ff6d0199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d6436640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d0100ad0 0x7ff6d019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d6436640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 0x7ff6d01a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d6436640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010f5d0 con 0x7ff6d0101470 2024-03-26T18:03:15.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d6436640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010f450 con 0x7ff6d0100ad0 2024-03-26T18:03:15.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d6436640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010f750 con 0x7ff6d0100170 2024-03-26T18:03:15.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d49d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 0x7ff6d01a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d0100170 0x7ff6d0199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d49d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 0x7ff6d01a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55508/0 (socket says 172.21.15.105:55508) 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d49d3640 1 -- 172.21.15.105:0/1455153196 learned_addr learned my addr 172.21.15.105:0/1455153196 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d0100ad0 0x7ff6d019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6cffff640 1 -- 172.21.15.105:0/1455153196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d0100ad0 msgr2=0x7ff6d019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6cffff640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d0100ad0 0x7ff6d019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6cffff640 1 -- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 msgr2=0x7ff6d01a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6cffff640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 0x7ff6d01a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6cffff640 1 -- 172.21.15.105:0/1455153196 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d01a1950 con 0x7ff6d0100170 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.407+0000 7ff6d49d3640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 0x7ff6d01a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.408+0000 7ff6cffff640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d0100170 0x7ff6d0199d90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000e9f0 tx=0x7ff6c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.408+0000 7ff6cd7fa640 1 -- 172.21.15.105:0/1455153196 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000ccc0 con 0x7ff6d0100170 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.408+0000 7ff6d6436640 1 -- 172.21.15.105:0/1455153196 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d01a1c40 con 0x7ff6d0100170 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.408+0000 7ff6cd7fa640 1 -- 172.21.15.105:0/1455153196 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6c00108e0 con 0x7ff6d0100170 2024-03-26T18:03:15.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.408+0000 7ff6cd7fa640 1 -- 172.21.15.105:0/1455153196 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0018880 con 0x7ff6d0100170 2024-03-26T18:03:15.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.408+0000 7ff6d6436640 1 -- 172.21.15.105:0/1455153196 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d01a2190 con 0x7ff6d0100170 2024-03-26T18:03:15.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.411+0000 7ff6cd7fa640 1 -- 172.21.15.105:0/1455153196 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff6c0010a90 con 0x7ff6d0100170 2024-03-26T18:03:15.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.411+0000 7ff6d9023740 1 Processor -- start 2024-03-26T18:03:15.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.411+0000 7ff6d9023740 1 -- start start 2024-03-26T18:03:15.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26617d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe2661cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe2661f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6d9023740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fe2657c980 con 0x55fe266182e0 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6d9023740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fe26524be0 con 0x55fe2661d450 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6d9023740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fe26578930 con 0x55fe266178f0 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cd7fa640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6b80a2f20 0x7ff6b80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6d49d3640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6b80a2f20 0x7ff6b80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe2661cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe2661f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6d49d3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26617d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe2661f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38230/0 (socket says 172.21.15.105:38230) 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/3157483752 learned_addr learned my addr 172.21.15.105:0/3157483752 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cd7fa640 1 -- 172.21.15.105:0/1455153196 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff6c00cf7f0 con 0x7ff6d0100170 2024-03-26T18:03:15.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/3157483752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 msgr2=0x55fe26617d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/3157483752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26617d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/3157483752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 msgr2=0x55fe2661cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/3157483752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe2661cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.412+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/3157483752 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2635a7d0 con 0x55fe2661d450 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.413+0000 7ff6d49d3640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6b80a2f20 0x7ff6b80a53e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009a60 tx=0x7ff6c8005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.413+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/3157483752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe2661f8e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009cf0 tx=0x7ff6c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75835be4b0007ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.413+0000 7ff6b67fc640 1 -- 172.21.15.105:0/3157483752 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4004230 con 0x55fe2661d450 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.413+0000 7ff6b67fc640 1 -- 172.21.15.105:0/3157483752 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6c40043d0 con 0x55fe2661d450 2024-03-26T18:03:15.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.413+0000 7ff6b67fc640 1 -- 172.21.15.105:0/3157483752 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4019880 con 0x55fe2661d450 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 -- 172.21.15.105:0/3157483752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 msgr2=0x55fe2661f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3157483752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe2661f8e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009cf0 tx=0x7ff6c4011810 comp rx=0 tx=0).stop 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 -- 172.21.15.105:0/3157483752 shutdown_connections 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3157483752 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe2661f8e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3157483752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe2661cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3157483752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26617d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 -- 172.21.15.105:0/3157483752 >> 172.21.15.105:0/3157483752 conn(0x55fe26612e60 msgr2=0x55fe26613260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 -- 172.21.15.105:0/3157483752 shutdown_connections 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.414+0000 7ff6d9023740 1 -- 172.21.15.105:0/3157483752 wait complete. 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 1 -- start start 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 1 Processor -- start 2024-03-26T18:03:15.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26621a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26621fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55fe26620400 con 0x55fe266182e0 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fe26538160 con 0x55fe2661d450 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fe26620580 con 0x55fe266178f0 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6d9023740 10 client.0 _ll_register_callbacks cb 0x55fe26574720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.415+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26621fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6d49d3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26621a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26621fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55532/0 (socket says 172.21.15.105:55532) 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/3625550531 learned_addr learned my addr 172.21.15.105:0/3625550531 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/3625550531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 msgr2=0x55fe26621a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26621a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/3625550531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 msgr2=0x55fe26628ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/3625550531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe262903f0 con 0x55fe266182e0 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cffff640 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26621a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6d49d3640 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26621fb0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009700 tx=0x7ff6c40052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c5b1fdbf3b8b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6af7fe640 1 -- 172.21.15.105:0/3625550531 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4027070 con 0x55fe266182e0 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6af7fe640 1 -- 172.21.15.105:0/3625550531 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff6c40037f0 con 0x55fe266182e0 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.416+0000 7ff6d9023740 1 -- 172.21.15.105:0/3625550531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c40043d0 con 0x55fe266182e0 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6af7fe640 1 -- 172.21.15.105:0/3625550531 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff6c4033050 con 0x55fe266182e0 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6d9023740 10 client.24321 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6d9023740 1 -- 172.21.15.105:0/3625550531 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fe262500f0 con 0x55fe266182e0 2024-03-26T18:03:15.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6af7fe640 1 -- 172.21.15.105:0/3625550531 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6c40043d0 con 0x55fe266182e0 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6af7fe640 1 -- 172.21.15.105:0/3625550531 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff6c4004b70 con 0x55fe266182e0 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6d9023740 10 client.24321 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6d9023740 10 client.24321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6d9023740 20 client.24321 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fe2662d6e0 0x55fe2662fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6d9023740 4 client.24321 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6d9023740 1 -- 172.21.15.105:0/3625550531 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe26589ec0 con 0x55fe2662d6e0 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.417+0000 7ff6cffff640 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fe2662d6e0 0x55fe2662fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.418+0000 7ff6af7fe640 10 client.24321 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.418+0000 7ff6cffff640 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fe2662d6e0 0x55fe2662fb40 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af8ca297b5effff server_cookie=2014b1e35d0338d2 in_seq=0 out_seq=0 2024-03-26T18:03:15.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:15 smithi135 ceph-mon[35636]: pgmap v1100: 97 pgs: 97 active+clean; 386 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 2.9 MiB/s wr, 232 op/s 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.419+0000 7ff6af7fe640 1 -- 172.21.15.105:0/3625550531 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fe26589ec0 con 0x55fe2662d6e0 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.419+0000 7ff6af7fe640 10 client.24321 handle_command_reply: tid=0 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.419+0000 7ff6d9023740 1 client.24321 shutdown 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 -- 172.21.15.105:0/3625550531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 msgr2=0x55fe26621fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26621fb0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009700 tx=0x7ff6c40052f0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 -- 172.21.15.105:0/3625550531 shutdown_connections 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fe2662d6e0 0x55fe2662fb40 crc :-1 s=READY pgs=620 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6af7fe640 0 client.24321 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26621fb0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 --2- 172.21.15.105:0/3625550531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26621a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 -- 172.21.15.105:0/3625550531 >> 172.21.15.105:0/3625550531 conn(0x55fe26612e60 msgr2=0x55fe26612080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:15.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 -- 172.21.15.105:0/3625550531 shutdown_connections 2024-03-26T18:03:15.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 1 -- 172.21.15.105:0/3625550531 wait complete. 2024-03-26T18:03:15.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.420+0000 7ff6d9023740 20 client.24321 trim_cache size 0 max 16384 2024-03-26T18:03:15.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d9023740 1 Processor -- start 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d9023740 1 -- start start 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266178f0 0x55fe26619460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266182e0 0x55fe266199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d9023740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01ab0c0 con 0x55fe266178f0 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d9023740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01acc00 con 0x55fe2661d450 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d9023740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01ab790 con 0x55fe266182e0 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266178f0 0x55fe26619460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6cffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266178f0 0x55fe26619460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55534/0 (socket says 172.21.15.105:55534) 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6cffff640 1 -- 172.21.15.105:0/2515607046 learned_addr learned my addr 172.21.15.105:0/2515607046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266182e0 0x55fe266199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.446+0000 7ff6d49d3640 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6cffff640 1 -- 172.21.15.105:0/2515607046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266182e0 msgr2=0x55fe266199a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6cffff640 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266182e0 0x55fe266199a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6cffff640 1 -- 172.21.15.105:0/2515607046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 msgr2=0x55fe26628fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6cffff640 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6cffff640 1 -- 172.21.15.105:0/2515607046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2660de20 con 0x55fe266178f0 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6d49d3640 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6cffff640 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266178f0 0x55fe26619460 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0098360 tx=0x7ff6c00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326eda50cdc34572 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6b5ffb640 1 -- 172.21.15.105:0/2515607046 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c009bf90 con 0x55fe266178f0 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6b5ffb640 1 -- 172.21.15.105:0/2515607046 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6c00a0030 con 0x55fe266178f0 2024-03-26T18:03:15.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.447+0000 7ff6b5ffb640 1 -- 172.21.15.105:0/2515607046 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0018c80 con 0x55fe266178f0 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 -- 172.21.15.105:0/2515607046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266178f0 msgr2=0x55fe26619460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266178f0 0x55fe26619460 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0098360 tx=0x7ff6c00bc040 comp rx=0 tx=0).stop 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 -- 172.21.15.105:0/2515607046 shutdown_connections 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26628fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266182e0 0x55fe266199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2515607046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266178f0 0x55fe26619460 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 -- 172.21.15.105:0/2515607046 >> 172.21.15.105:0/2515607046 conn(0x55fe26612e60 msgr2=0x55fe26621a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 -- 172.21.15.105:0/2515607046 shutdown_connections 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.448+0000 7ff6d9023740 1 -- 172.21.15.105:0/2515607046 wait complete. 2024-03-26T18:03:15.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 1 -- start start 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 1 Processor -- start 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26625ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26626410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26691a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01ac530 con 0x55fe266182e0 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01abe60 con 0x55fe2661d450 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d01a90a0 con 0x55fe266178f0 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d9023740 10 client.0 _ll_register_callbacks cb 0x7ff6c000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26626410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6cf7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26626410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55550/0 (socket says 172.21.15.105:55550) 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/2935400820 learned_addr learned my addr 172.21.15.105:0/2935400820 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6d49d3640 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26691a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.449+0000 7ff6cffff640 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26625ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/2935400820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 msgr2=0x55fe26625ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:15.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26625ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 msgr2=0x55fe26691a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26691a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6cf7fe640 1 -- 172.21.15.105:0/2935400820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe26691f70 con 0x55fe266182e0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6d49d3640 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26691a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6cf7fe640 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26626410 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4000c00 tx=0x7ff6c40199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bcd62ef632abbb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6af7fe640 1 -- 172.21.15.105:0/2935400820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c403c060 con 0x55fe266182e0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6af7fe640 1 -- 172.21.15.105:0/2935400820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff6c4003800 con 0x55fe266182e0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6d9023740 1 -- 172.21.15.105:0/2935400820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c00a0030 con 0x55fe266182e0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6af7fe640 1 -- 172.21.15.105:0/2935400820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff6c4027040 con 0x55fe266182e0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6d9023740 10 client.24327 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.450+0000 7ff6d9023740 1 -- 172.21.15.105:0/2935400820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fe266931e0 con 0x55fe266182e0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6af7fe640 1 -- 172.21.15.105:0/2935400820 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6c00a0030 con 0x55fe266182e0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6af7fe640 1 -- 172.21.15.105:0/2935400820 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff6c4003a40 con 0x55fe266182e0 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6d9023740 10 client.24327 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:15.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6d9023740 10 client.24327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:15.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6d9023740 20 client.24327 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:15.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fe2662d6e0 0x55fe26695b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6d9023740 4 client.24327 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:03:15.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6d9023740 1 -- 172.21.15.105:0/2935400820 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe26584210 con 0x55fe2662d6e0 2024-03-26T18:03:15.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.451+0000 7ff6cffff640 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fe2662d6e0 0x55fe26695b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.452+0000 7ff6af7fe640 10 client.24327 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:15.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.452+0000 7ff6cffff640 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fe2662d6e0 0x55fe26695b70 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e3d0f3bbc7ff29 server_cookie=9b49889304271e52 in_seq=0 out_seq=0 2024-03-26T18:03:15.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.453+0000 7ff6af7fe640 1 -- 172.21.15.105:0/2935400820 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fe26584210 con 0x55fe2662d6e0 2024-03-26T18:03:15.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.453+0000 7ff6af7fe640 10 client.24327 handle_command_reply: tid=0 2024-03-26T18:03:15.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.453+0000 7ff6d9023740 1 client.24327 shutdown 2024-03-26T18:03:15.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 -- 172.21.15.105:0/2935400820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 msgr2=0x55fe26626410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26626410 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4000c00 tx=0x7ff6c40199b0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 -- 172.21.15.105:0/2935400820 shutdown_connections 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55fe2662d6e0 0x55fe26695b70 crc :-1 s=READY pgs=621 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fe2661d450 0x55fe26691a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fe266182e0 0x55fe26626410 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 --2- 172.21.15.105:0/2935400820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fe266178f0 0x55fe26625ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6af7fe640 0 client.24327 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 -- 172.21.15.105:0/2935400820 >> 172.21.15.105:0/2935400820 conn(0x55fe26612e60 msgr2=0x55fe26621a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 -- 172.21.15.105:0/2935400820 shutdown_connections 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 1 -- 172.21.15.105:0/2935400820 wait complete. 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.454+0000 7ff6d9023740 20 client.24327 trim_cache size 0 max 16384 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:03:15.439 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T18:03:15.440 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T18:03:15.440 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:03:15.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 -- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6b80a2f20 msgr2=0x7ff6b80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6b80a2f20 0x7ff6b80a53e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009a60 tx=0x7ff6c8005f60 comp rx=0 tx=0).stop 2024-03-26T18:03:15.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 -- 172.21.15.105:0/1455153196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d0100170 msgr2=0x7ff6d0199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d0100170 0x7ff6d0199d90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000e9f0 tx=0x7ff6c000eeb0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 -- 172.21.15.105:0/1455153196 shutdown_connections 2024-03-26T18:03:15.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff6b80a2f20 0x7ff6b80a53e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6d0101470 0x7ff6d01a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff6d0100ad0 0x7ff6d019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 --2- 172.21.15.105:0/1455153196 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6d0100170 0x7ff6d0199d90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.455+0000 7ff6d6436640 1 -- 172.21.15.105:0/1455153196 >> 172.21.15.105:0/1455153196 conn(0x7ff6d00fbd30 msgr2=0x7ff6d00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:15.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.456+0000 7ff6d6436640 1 -- 172.21.15.105:0/1455153196 shutdown_connections 2024-03-26T18:03:15.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.456+0000 7ff6d6436640 1 -- 172.21.15.105:0/1455153196 wait complete. 2024-03-26T18:03:15.450 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T18:03:15.450 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:03:15.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 93689 7.00 MB/sec execute 598 sec latency 129.267 ms 2024-03-26T18:03:15.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.650+0000 7f04fba75640 1 -- 172.21.15.105:0/109196783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f4103b00 msgr2=0x7f04f4103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.650+0000 7f04fba75640 1 --2- 172.21.15.105:0/109196783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f4103b00 0x7f04f4103f20 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.650+0000 7f04fba75640 1 -- 172.21.15.105:0/109196783 shutdown_connections 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.650+0000 7f04fba75640 1 --2- 172.21.15.105:0/109196783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04f41056b0 0x7f04f410bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.650+0000 7f04fba75640 1 --2- 172.21.15.105:0/109196783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04f4104cf0 0x7f04f4105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.650+0000 7f04fba75640 1 --2- 172.21.15.105:0/109196783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f4103b00 0x7f04f4103f20 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.650+0000 7f04fba75640 1 -- 172.21.15.105:0/109196783 >> 172.21.15.105:0/109196783 conn(0x7f04f40ff070 msgr2=0x7f04f41014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.651+0000 7f04fba75640 1 -- 172.21.15.105:0/109196783 shutdown_connections 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.651+0000 7f04fba75640 1 -- 172.21.15.105:0/109196783 wait complete. 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.651+0000 7f04fba75640 1 Processor -- start 2024-03-26T18:03:15.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.651+0000 7f04fba75640 1 -- start start 2024-03-26T18:03:15.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fba75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04f4103b00 0x7f04f4199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fba75640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04f4104cf0 0x7f04f419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fba75640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f41056b0 0x7f04f41a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fba75640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04f410f630 con 0x7f04f41056b0 2024-03-26T18:03:15.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fba75640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f04f410f4b0 con 0x7f04f4104cf0 2024-03-26T18:03:15.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fba75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04f410f7b0 con 0x7f04f4103b00 2024-03-26T18:03:15.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f41056b0 0x7f04f41a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f41056b0 0x7f04f41a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55572/0 (socket says 172.21.15.105:55572) 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 -- 172.21.15.105:0/3437967203 learned_addr learned my addr 172.21.15.105:0/3437967203 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04f9010640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04f4104cf0 0x7f04f419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 -- 172.21.15.105:0/3437967203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04f4103b00 msgr2=0x7f04f4199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04f4103b00 0x7f04f4199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 -- 172.21.15.105:0/3437967203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04f4104cf0 msgr2=0x7f04f419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04f4104cf0 0x7f04f419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 -- 172.21.15.105:0/3437967203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f41a1960 con 0x7f04f41056b0 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04f9010640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04f4104cf0 0x7f04f419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.652+0000 7f04fa012640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f41056b0 0x7f04f41a12c0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00eb10 tx=0x7f04ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.653+0000 7f04eaffd640 1 -- 172.21.15.105:0/3437967203 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec004b20 con 0x7f04f41056b0 2024-03-26T18:03:15.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.653+0000 7f04eaffd640 1 -- 172.21.15.105:0/3437967203 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04ec004cc0 con 0x7f04f41056b0 2024-03-26T18:03:15.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.653+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f41a1c50 con 0x7f04f41056b0 2024-03-26T18:03:15.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.653+0000 7f04eaffd640 1 -- 172.21.15.105:0/3437967203 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec010720 con 0x7f04f41056b0 2024-03-26T18:03:15.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.653+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f41a21a0 con 0x7f04f41056b0 2024-03-26T18:03:15.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.655+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f410aea0 con 0x7f04f41056b0 2024-03-26T18:03:15.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.656+0000 7f04eaffd640 1 -- 172.21.15.105:0/3437967203 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f04ec005020 con 0x7f04f41056b0 2024-03-26T18:03:15.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.657+0000 7f04eaffd640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f04e00a2fa0 0x7f04e00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:15.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.657+0000 7f04eaffd640 1 -- 172.21.15.105:0/3437967203 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f04ec0cfa00 con 0x7f04f41056b0 2024-03-26T18:03:15.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.657+0000 7f04f9811640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f04e00a2fa0 0x7f04e00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:15.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.657+0000 7f04f9811640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f04e00a2fa0 0x7f04e00a5460 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f04f0036040 tx=0x7f04f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:15.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:15.665+0000 7f04eaffd640 1 -- 172.21.15.105:0/3437967203 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f04ec09c7f0 con 0x7f04f41056b0 2024-03-26T18:03:15.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.004+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04f40624c0 con 0x7f04f41056b0 2024-03-26T18:03:15.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.006+0000 7f04eaffd640 1 -- 172.21.15.105:0/3437967203 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11941 (secure 0 0 0) 0x7f04ec09c0f0 con 0x7f04f41056b0 2024-03-26T18:03:15.987 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:03:15.987 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:02:08.013819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_33599":{"gid":33599,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1455536711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1455536711},{"type":"v1","addr":"172.21.15.135:6835","nonce":1455536711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24466,24487,14628,24521,24482]},"id":1}]} 2024-03-26T18:03:15.987 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 45 2024-03-26T18:03:15.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f04e00a2fa0 msgr2=0x7f04e00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f04e00a2fa0 0x7f04e00a5460 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f04f0036040 tx=0x7f04f0000bc0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f41056b0 msgr2=0x7f04f41a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:15.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f41056b0 0x7f04f41a12c0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00eb10 tx=0x7f04ec00c2d0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 shutdown_connections 2024-03-26T18:03:15.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f04e00a2fa0 0x7f04e00a5460 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04f41056b0 0x7f04f41a12c0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f04f4104cf0 0x7f04f419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 --2- 172.21.15.105:0/3437967203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04f4103b00 0x7f04f4199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:15.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 >> 172.21.15.105:0/3437967203 conn(0x7f04f40ff070 msgr2=0x7f04f4108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:15.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 shutdown_connections 2024-03-26T18:03:15.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.009+0000 7f04fba75640 1 -- 172.21.15.105:0/3437967203 wait complete. 2024-03-26T18:03:16.004 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T18:03:16.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 -- 172.21.15.105:0/3185662291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a0103480 msgr2=0x7ff3a0109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/3185662291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a0103480 0x7ff3a0109d10 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7ff398009ca0 tx=0x7ff39802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 -- 172.21.15.105:0/3185662291 shutdown_connections 2024-03-26T18:03:16.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/3185662291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a0103480 0x7ff3a0109d10 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/3185662291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a0102ac0 0x7ff3a0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/3185662291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a01018d0 0x7ff3a0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 -- 172.21.15.105:0/3185662291 >> 172.21.15.105:0/3185662291 conn(0x7ff3a00fce40 msgr2=0x7ff3a00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 -- 172.21.15.105:0/3185662291 shutdown_connections 2024-03-26T18:03:16.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.160+0000 7ff3a7f10640 1 -- 172.21.15.105:0/3185662291 wait complete. 2024-03-26T18:03:16.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a7f10640 1 Processor -- start 2024-03-26T18:03:16.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a7f10640 1 -- start start 2024-03-26T18:03:16.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a7f10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a01018d0 0x7ff3a0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a7f10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a0102ac0 0x7ff3a019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a7f10640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a0103480 0x7ff3a01a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a7f10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a010d3e0 con 0x7ff3a01018d0 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a7f10640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a010d260 con 0x7ff3a0103480 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a7f10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a010d560 con 0x7ff3a0102ac0 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a5cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a01018d0 0x7ff3a0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a64ad640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a0103480 0x7ff3a01a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a54ab640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a0102ac0 0x7ff3a019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a5cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a01018d0 0x7ff3a0199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55580/0 (socket says 172.21.15.105:55580) 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.161+0000 7ff3a5cac640 1 -- 172.21.15.105:0/2952406156 learned_addr learned my addr 172.21.15.105:0/2952406156 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff3a54ab640 1 -- 172.21.15.105:0/2952406156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a0103480 msgr2=0x7ff3a01a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a0103480 0x7ff3a01a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff3a54ab640 1 -- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a01018d0 msgr2=0x7ff3a0199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a01018d0 0x7ff3a0199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff3a54ab640 1 -- 172.21.15.105:0/2952406156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a01a1900 con 0x7ff3a0102ac0 2024-03-26T18:03:16.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a0102ac0 0x7ff3a019a200 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e9f0 tx=0x7ff39000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff396ffd640 1 -- 172.21.15.105:0/2952406156 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39000ccc0 con 0x7ff3a0102ac0 2024-03-26T18:03:16.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff396ffd640 1 -- 172.21.15.105:0/2952406156 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3900108e0 con 0x7ff3a0102ac0 2024-03-26T18:03:16.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff3a7f10640 1 -- 172.21.15.105:0/2952406156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a01a1bf0 con 0x7ff3a0102ac0 2024-03-26T18:03:16.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff396ffd640 1 -- 172.21.15.105:0/2952406156 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390018880 con 0x7ff3a0102ac0 2024-03-26T18:03:16.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.162+0000 7ff3a7f10640 1 -- 172.21.15.105:0/2952406156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a01a2140 con 0x7ff3a0102ac0 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.165+0000 7ff3aabda740 1 Processor -- start 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff396ffd640 1 -- 172.21.15.105:0/2952406156 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff390010a90 con 0x7ff3a0102ac0 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3aabda740 1 -- start start 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9bae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9bb420 0x56146b9bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9c0590 0x56146b9c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3aabda740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56146b91f970 con 0x56146b9bb420 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3aabda740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56146b8c7d70 con 0x56146b9c0590 2024-03-26T18:03:16.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3aabda740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56146b91b920 con 0x56146b9baa30 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3a54ab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9c0590 0x56146b9c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3a54ab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9c0590 0x56146b9c2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38296/0 (socket says 172.21.15.105:38296) 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3a54ab640 1 -- 172.21.15.105:0/3969363832 learned_addr learned my addr 172.21.15.105:0/3969363832 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3a64ad640 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9bae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.166+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9bb420 0x56146b9bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a54ab640 1 -- 172.21.15.105:0/3969363832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 msgr2=0x56146b9bae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9bae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a54ab640 1 -- 172.21.15.105:0/3969363832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9bb420 msgr2=0x56146b9bff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9bb420 0x56146b9bff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a54ab640 1 -- 172.21.15.105:0/3969363832 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56146b6333f0 con 0x56146b9c0590 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a64ad640 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9bb420 0x56146b9bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff396ffd640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff38c0a2fa0 0x7ff38c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a64ad640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff38c0a2fa0 0x7ff38c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9c0590 0x56146b9c2a20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff3900d5c20 tx=0x7ff3900d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=827240662c00765 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff396ffd640 1 -- 172.21.15.105:0/2952406156 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff3900cf780 con 0x7ff3a0102ac0 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.167+0000 7ff38bfff640 1 -- 172.21.15.105:0/3969363832 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390026030 con 0x56146b9c0590 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.168+0000 7ff38bfff640 1 -- 172.21.15.105:0/3969363832 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff390010e00 con 0x56146b9c0590 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.168+0000 7ff38bfff640 1 -- 172.21.15.105:0/3969363832 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3900a4820 con 0x56146b9c0590 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 -- 172.21.15.105:0/3969363832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9c0590 msgr2=0x56146b9c2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3a64ad640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff38c0a2fa0 0x7ff38c0a5460 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff398006010 tx=0x7ff39802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9c0590 0x56146b9c2a20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff3900d5c20 tx=0x7ff3900d7670 comp rx=0 tx=0).stop 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 -- 172.21.15.105:0/3969363832 shutdown_connections 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9c0590 0x56146b9c2a20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9bb420 0x56146b9bff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3969363832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 -- 172.21.15.105:0/3969363832 >> 172.21.15.105:0/3969363832 conn(0x56146b9b5fa0 msgr2=0x56146b9b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 -- 172.21.15.105:0/3969363832 shutdown_connections 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 -- 172.21.15.105:0/3969363832 wait complete. 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 -- start start 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.169+0000 7ff3aabda740 1 Processor -- start 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9c0590 0x56146b9cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3aabda740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56146b9c3540 con 0x56146b9baa30 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3aabda740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56146b8db310 con 0x56146b9bb420 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3aabda740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56146b9c36c0 con 0x56146b9c0590 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3aabda740 10 client.0 _ll_register_callbacks cb 0x56146b916c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a54ab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a5cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a5cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55600/0 (socket says 172.21.15.105:55600) 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a54ab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38298/0 (socket says 172.21.15.105:38298) 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a5cac640 1 -- 172.21.15.105:0/210424598 learned_addr learned my addr 172.21.15.105:0/210424598 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a64ad640 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9c0590 0x56146b9cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a5cac640 1 -- 172.21.15.105:0/210424598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9c0590 msgr2=0x56146b9cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9c0590 0x56146b9cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a5cac640 1 -- 172.21.15.105:0/210424598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 msgr2=0x56146b9c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a5cac640 1 -- 172.21.15.105:0/210424598 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56146b6fd7d0 con 0x56146b9baa30 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.170+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.171+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c4f50 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0058c0 tx=0x7ff39c012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2da420ea2f062e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.171+0000 7ff3897fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.171+0000 7ff3897fa640 1 -- 172.21.15.105:0/210424598 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c004030 con 0x56146b9baa30 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.171+0000 7ff3aabda740 1 -- 172.21.15.105:0/210424598 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56146b92ceb0 con 0x56146b9baa30 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.171+0000 7ff3897fa640 1 -- 172.21.15.105:0/210424598 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff39c0041d0 con 0x56146b9baa30 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.171+0000 7ff3897fa640 1 -- 172.21.15.105:0/210424598 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff39c0043d0 con 0x56146b9baa30 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.171+0000 7ff3aabda740 10 client.24345 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.171+0000 7ff3aabda740 1 -- 172.21.15.105:0/210424598 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56146b9d8170 con 0x56146b9baa30 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3897fa640 1 -- 172.21.15.105:0/210424598 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56146b92ceb0 con 0x56146b9baa30 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3897fa640 1 -- 172.21.15.105:0/210424598 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff39c003ac0 con 0x56146b9baa30 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3aabda740 10 client.24345 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3aabda740 10 client.24345 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3aabda740 20 client.24345 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3aabda740 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56146b9d9ca0 0x56146b9dc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3aabda740 4 client.24345 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3aabda740 1 -- 172.21.15.105:0/210424598 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56146b927200 con 0x56146b9d9ca0 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.172+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56146b9d9ca0 0x56146b9dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.173+0000 7ff3897fa640 10 client.24345 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.173+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56146b9d9ca0 0x56146b9dc140 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=91351b1a263a8401 server_cookie=7b0ec5ff2198c0d4 in_seq=0 out_seq=0 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.173+0000 7ff3897fa640 1 -- 172.21.15.105:0/210424598 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56146b927200 con 0x56146b9d9ca0 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3897fa640 10 client.24345 handle_command_reply: tid=0 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 client.24345 shutdown 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 -- 172.21.15.105:0/210424598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 msgr2=0x56146b9c4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c4f50 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0058c0 tx=0x7ff39c012bf0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 -- 172.21.15.105:0/210424598 shutdown_connections 2024-03-26T18:03:16.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56146b9d9ca0 0x56146b9dc140 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9c0590 0x56146b9cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 --2- 172.21.15.105:0/210424598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c4f50 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 -- 172.21.15.105:0/210424598 >> 172.21.15.105:0/210424598 conn(0x56146b9b5fa0 msgr2=0x56146b9b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3897fa640 0 client.24345 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:03:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 -- 172.21.15.105:0/210424598 shutdown_connections 2024-03-26T18:03:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 1 -- 172.21.15.105:0/210424598 wait complete. 2024-03-26T18:03:16.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.174+0000 7ff3aabda740 20 client.24345 trim_cache size 0 max 16384 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3aabda740 1 Processor -- start 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3aabda740 1 -- start start 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9b5fa0 0x56146b9b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3aabda740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a01a6b40 con 0x56146b9b5fa0 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3a5cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9b5fa0 0x56146b9b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3a5cac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9b5fa0 0x56146b9b9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55604/0 (socket says 172.21.15.105:55604) 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3aabda740 1 -- 172.21.15.105:0/2988274477 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a01a8680 con 0x56146b9bb420 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3aabda740 1 -- 172.21.15.105:0/2988274477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a01a7210 con 0x56146b9baa30 2024-03-26T18:03:16.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3a5cac640 1 -- 172.21.15.105:0/2988274477 learned_addr learned my addr 172.21.15.105:0/2988274477 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.199+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff3a64ad640 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff3a54ab640 1 -- 172.21.15.105:0/2988274477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 msgr2=0x56146b9c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff3a54ab640 1 -- 172.21.15.105:0/2988274477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9b5fa0 msgr2=0x56146b9b9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9b5fa0 0x56146b9b9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff3a54ab640 1 -- 172.21.15.105:0/2988274477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56146b9bb800 con 0x56146b9baa30 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9cc490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff39001a040 tx=0x7ff390009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2710b2a8564047ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff38b7fe640 1 -- 172.21.15.105:0/2988274477 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39009bec0 con 0x56146b9baa30 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff38b7fe640 1 -- 172.21.15.105:0/2988274477 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff39009ff60 con 0x56146b9baa30 2024-03-26T18:03:16.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.200+0000 7ff38b7fe640 1 -- 172.21.15.105:0/2988274477 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3900d7d20 con 0x56146b9baa30 2024-03-26T18:03:16.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 -- 172.21.15.105:0/2988274477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 msgr2=0x56146b9cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9cc490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff39001a040 tx=0x7ff390009700 comp rx=0 tx=0).stop 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 -- 172.21.15.105:0/2988274477 shutdown_connections 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9baa30 0x56146b9cc490 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 --2- 172.21.15.105:0/2988274477 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9b5fa0 0x56146b9b9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 -- 172.21.15.105:0/2988274477 >> 172.21.15.105:0/2988274477 conn(0x56146b9bc1c0 msgr2=0x56146b9b9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 -- 172.21.15.105:0/2988274477 shutdown_connections 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.201+0000 7ff3aabda740 1 -- 172.21.15.105:0/2988274477 wait complete. 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.202+0000 7ff3aabda740 1 -- start start 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.202+0000 7ff3aabda740 1 Processor -- start 2024-03-26T18:03:16.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.202+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9b5fa0 0x56146b9c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.202+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55620/0 (socket says 172.21.15.105:55620) 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3aabda740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 -- 172.21.15.105:0/3579210087 learned_addr learned my addr 172.21.15.105:0/3579210087 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a01a7fb0 con 0x56146b9baa30 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a01a78e0 con 0x56146b9bb420 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a01a4b60 con 0x56146b9b5fa0 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3aabda740 10 client.0 _ll_register_callbacks cb 0x7ff39c009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9b5fa0 0x56146b9c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a64ad640 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 -- 172.21.15.105:0/3579210087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9b5fa0 msgr2=0x56146b9c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9b5fa0 0x56146b9c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 -- 172.21.15.105:0/3579210087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 msgr2=0x56146b9caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 -- 172.21.15.105:0/3579210087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56146b9cb010 con 0x56146b9baa30 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9b5fa0 0x56146b9c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3a54ab640 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c9a10 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7ff39009be50 tx=0x7ff3900d5ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8b539d58132447 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.203+0000 7ff3897fa640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3897fa640 1 -- 172.21.15.105:0/3579210087 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3900a4820 con 0x56146b9baa30 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3897fa640 1 -- 172.21.15.105:0/3579210087 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff3900d5d10 con 0x56146b9baa30 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff39009ff60 con 0x56146b9baa30 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3897fa640 1 -- 172.21.15.105:0/3579210087 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff39009bec0 con 0x56146b9baa30 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3aabda740 10 client.24351 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56146b9cc310 con 0x56146b9baa30 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3897fa640 1 -- 172.21.15.105:0/3579210087 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff39009ff60 con 0x56146b9baa30 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3897fa640 1 -- 172.21.15.105:0/3579210087 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff39009c120 con 0x56146b9baa30 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3aabda740 10 client.24351 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3aabda740 10 client.24351 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3aabda740 20 client.24351 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56146b9c0590 0x56146ba42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.204+0000 7ff3aabda740 4 client.24351 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.205+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56146b8a46a0 con 0x56146b9c0590 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.205+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56146b9c0590 0x56146ba42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.205+0000 7ff3a5cac640 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56146b9c0590 0x56146ba42090 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb238d2ae987304a server_cookie=e4f8d978873b4356 in_seq=0 out_seq=0 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.205+0000 7ff3897fa640 10 client.24351 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3897fa640 1 -- 172.21.15.105:0/3579210087 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56146b8a46a0 con 0x56146b9c0590 2024-03-26T18:03:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3897fa640 10 client.24351 handle_command_reply: tid=0 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 client.24351 shutdown 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 msgr2=0x56146b9c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c9a10 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7ff39009be50 tx=0x7ff3900d5ba0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 shutdown_connections 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x56146b9c0590 0x56146ba42090 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56146b9bb420 0x56146b9caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3897fa640 0 client.24351 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56146b9baa30 0x56146b9c9a10 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 --2- 172.21.15.105:0/3579210087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56146b9b5fa0 0x56146b9c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.206+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 >> 172.21.15.105:0/3579210087 conn(0x56146b9bc1c0 msgr2=0x56146b9b9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 shutdown_connections 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3aabda740 1 -- 172.21.15.105:0/3579210087 wait complete. 2024-03-26T18:03:16.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3aabda740 20 client.24351 trim_cache size 0 max 16384 2024-03-26T18:03:16.194 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 -- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff38c0a2fa0 msgr2=0x7ff38c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff38c0a2fa0 0x7ff38c0a5460 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff398006010 tx=0x7ff39802e040 comp rx=0 tx=0).stop 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 -- 172.21.15.105:0/2952406156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a0102ac0 msgr2=0x7ff3a019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a0102ac0 0x7ff3a019a200 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff39000e9f0 tx=0x7ff39000eeb0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 -- 172.21.15.105:0/2952406156 shutdown_connections 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff38c0a2fa0 0x7ff38c0a5460 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3a0103480 0x7ff3a01a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3a0102ac0 0x7ff3a019a200 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 --2- 172.21.15.105:0/2952406156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3a01018d0 0x7ff3a0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.207+0000 7ff3a7f10640 1 -- 172.21.15.105:0/2952406156 >> 172.21.15.105:0/2952406156 conn(0x7ff3a00fce40 msgr2=0x7ff3a0073af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.208+0000 7ff3a7f10640 1 -- 172.21.15.105:0/2952406156 shutdown_connections 2024-03-26T18:03:16.196 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.208+0000 7ff3a7f10640 1 -- 172.21.15.105:0/2952406156 wait complete. 2024-03-26T18:03:16.203 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T18:03:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.347+0000 7f6cda1b2640 1 -- 172.21.15.105:0/2840310710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4104c70 msgr2=0x7f6cd4107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.347+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/2840310710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4104c70 0x7f6cd4107060 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009900 tx=0x7f6cbc02bc50 comp rx=0 tx=0).stop 2024-03-26T18:03:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 -- 172.21.15.105:0/2840310710 shutdown_connections 2024-03-26T18:03:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/2840310710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cd41075a0 0x7f6cd41099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/2840310710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4104c70 0x7f6cd4107060 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/2840310710 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cd4102340 0x7f6cd4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 -- 172.21.15.105:0/2840310710 >> 172.21.15.105:0/2840310710 conn(0x7f6cd40fbd50 msgr2=0x7f6cd40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 -- 172.21.15.105:0/2840310710 shutdown_connections 2024-03-26T18:03:16.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 -- 172.21.15.105:0/2840310710 wait complete. 2024-03-26T18:03:16.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 Processor -- start 2024-03-26T18:03:16.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.348+0000 7f6cda1b2640 1 -- start start 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cda1b2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4102340 0x7f6cd4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cda1b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cd4104c70 0x7f6cd41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cda1b2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cd41075a0 0x7f6cd419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cda1b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd410d430 con 0x7f6cd4104c70 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cda1b2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd410d2b0 con 0x7f6cd41075a0 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4102340 0x7f6cd4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4102340 0x7f6cd4197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55172/0 (socket says 172.21.15.105:55172) 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cd37fe640 1 -- 172.21.15.105:0/1829430673 learned_addr learned my addr 172.21.15.105:0/1829430673 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cd3fff640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cd41075a0 0x7f6cd419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd410d5b0 con 0x7f6cd4102340 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.349+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cd4104c70 0x7f6cd41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.350+0000 7f6cd3fff640 1 -- 172.21.15.105:0/1829430673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4102340 msgr2=0x7f6cd4197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.350+0000 7f6cd3fff640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4102340 0x7f6cd4197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.350+0000 7f6cd3fff640 1 -- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cd4104c70 msgr2=0x7f6cd41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.350+0000 7f6cd3fff640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cd4104c70 0x7f6cd41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.350+0000 7f6cd3fff640 1 -- 172.21.15.105:0/1829430673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd419f780 con 0x7f6cd41075a0 2024-03-26T18:03:16.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.350+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cd4104c70 0x7f6cd41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.351+0000 7f6cd3fff640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cd41075a0 0x7f6cd419f110 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00ec50 tx=0x7f6ccc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.351+0000 7f6cd17fa640 1 -- 172.21.15.105:0/1829430673 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc003b70 con 0x7f6cd41075a0 2024-03-26T18:03:16.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.351+0000 7f6cd17fa640 1 -- 172.21.15.105:0/1829430673 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ccc003d10 con 0x7f6cd41075a0 2024-03-26T18:03:16.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.351+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd419fa70 con 0x7f6cd41075a0 2024-03-26T18:03:16.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.351+0000 7f6cd17fa640 1 -- 172.21.15.105:0/1829430673 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc010760 con 0x7f6cd41075a0 2024-03-26T18:03:16.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.351+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd41003c0 con 0x7f6cd41075a0 2024-03-26T18:03:16.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.354+0000 7f6cd17fa640 1 -- 172.21.15.105:0/1829430673 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6ccc004020 con 0x7f6cd41075a0 2024-03-26T18:03:16.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.354+0000 7f6cdcda0740 1 Processor -- start 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.354+0000 7f6cdcda0740 1 -- start start 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1f6a30 0x55c7ff1f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff1fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c7ff15b970 con 0x55c7ff1f6a30 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c7ff103d70 con 0x55c7ff1f7420 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c7ff157920 con 0x55c7ff1fc590 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1f6a30 0x55c7ff1f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6cd17fa640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6cc00ab7b0 0x7f6cc00adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1f6a30 0x55c7ff1f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55656/0 (socket says 172.21.15.105:55656) 2024-03-26T18:03:16.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 -- 172.21.15.105:0/4225014091 learned_addr learned my addr 172.21.15.105:0/4225014091 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff1fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 -- 172.21.15.105:0/4225014091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 msgr2=0x55c7ff1fbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff1fbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 -- 172.21.15.105:0/4225014091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1f6a30 msgr2=0x55c7ff1f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1f6a30 0x55c7ff1f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.355+0000 7f6ccbfff640 1 -- 172.21.15.105:0/4225014091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7fee6f3f0 con 0x55c7ff1fc590 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.356+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fea20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc005ec0 tx=0x7f6cbc02bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4213ddf1482c09 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.356+0000 7f6cd17fa640 1 -- 172.21.15.105:0/1829430673 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6ccc0cf940 con 0x7f6cd41075a0 2024-03-26T18:03:16.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:16 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3437967203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:03:16.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.356+0000 7f6cd3fff640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6cc00ab7b0 0x7f6cc00adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.356+0000 7f6cc9ffb640 1 -- 172.21.15.105:0/4225014091 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc004410 con 0x55c7ff1fc590 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.356+0000 7f6cc9ffb640 1 -- 172.21.15.105:0/4225014091 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cbc0054f0 con 0x55c7ff1fc590 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.356+0000 7f6cc9ffb640 1 -- 172.21.15.105:0/4225014091 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc004cd0 con 0x55c7ff1fc590 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cdcda0740 1 -- 172.21.15.105:0/4225014091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1fc590 msgr2=0x55c7ff1fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cd3fff640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6cc00ab7b0 0x7f6cc00adc70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0cfca0 tx=0x7f6ccc00ce70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fea20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc005ec0 tx=0x7f6cbc02bc50 comp rx=0 tx=0).stop 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cdcda0740 1 -- 172.21.15.105:0/4225014091 shutdown_connections 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fea20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff1fbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/4225014091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1f6a30 0x55c7ff1f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cdcda0740 1 -- 172.21.15.105:0/4225014091 >> 172.21.15.105:0/4225014091 conn(0x55c7ff1f1fa0 msgr2=0x55c7ff1f23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.357+0000 7f6cdcda0740 1 -- 172.21.15.105:0/4225014091 shutdown_connections 2024-03-26T18:03:16.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.358+0000 7f6cdcda0740 1 -- 172.21.15.105:0/4225014091 wait complete. 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.358+0000 7f6cdcda0740 1 -- start start 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.358+0000 7f6cdcda0740 1 Processor -- start 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.359+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 0x55c7ff200fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.359+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff2014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.359+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff208530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.359+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55c7ff1ff580 con 0x55c7ff1fc590 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.359+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c7ff117310 con 0x55c7ff1f7420 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.359+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c7ff1ff700 con 0x55c7ff1f6a30 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.359+0000 7f6cdcda0740 10 client.0 _ll_register_callbacks cb 0x55c7ff152c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff2014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff2014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38352/0 (socket says 172.21.15.105:38352) 2024-03-26T18:03:16.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 -- 172.21.15.105:0/3069006944 learned_addr learned my addr 172.21.15.105:0/3069006944 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 -- 172.21.15.105:0/3069006944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 msgr2=0x55c7ff200fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:03:16.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 0x55c7ff200fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 -- 172.21.15.105:0/3069006944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 msgr2=0x55c7ff208530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff208530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff208530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 -- 172.21.15.105:0/3069006944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7fef397d0 con 0x55c7ff1f7420 2024-03-26T18:03:16.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff208530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:03:16.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff2014f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4012790 tx=0x7f6cc4012c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=525ef0ce64aca248 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.360+0000 7f6cb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:03:16.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.361+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/3069006944 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400f920 con 0x55c7ff1f7420 2024-03-26T18:03:16.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.361+0000 7f6cdcda0740 1 -- 172.21.15.105:0/3069006944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cbc0054f0 con 0x55c7ff1f7420 2024-03-26T18:03:16.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.361+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/3069006944 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6cc400fac0 con 0x55c7ff1f7420 2024-03-26T18:03:16.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.362+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/3069006944 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6cc400fcc0 con 0x55c7ff1f7420 2024-03-26T18:03:16.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.363+0000 7f6cdcda0740 10 client.34016 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.363+0000 7f6cdcda0740 1 -- 172.21.15.105:0/3069006944 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c7ff20b960 con 0x55c7ff1f7420 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.364+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/3069006944 <== mon.2 v2:172.21.15.135:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cbc0054f0 con 0x55c7ff1f7420 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.364+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/3069006944 <== mon.2 v2:172.21.15.135:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f6cc4004bb0 con 0x55c7ff1f7420 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.364+0000 7f6cdcda0740 10 client.34016 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.364+0000 7f6cdcda0740 10 client.34016 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.364+0000 7f6cdcda0740 20 client.34016 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.364+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c7ff20d040 0x55c7ff20f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.364+0000 7f6cdcda0740 4 client.34016 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.364+0000 7f6cdcda0740 1 -- 172.21.15.105:0/3069006944 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7ff168eb0 con 0x55c7ff20d040 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.365+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c7ff20d040 0x55c7ff20f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.365+0000 7f6cb2ffd640 10 client.34016 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:03:16.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.366+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c7ff20d040 0x55c7ff20f4e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=962dec176918aae2 server_cookie=d369d56e44d2a6ca in_seq=0 out_seq=0 2024-03-26T18:03:16.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.366+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/3069006944 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c7ff168eb0 con 0x55c7ff20d040 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.366+0000 7f6cb2ffd640 10 client.34016 handle_command_reply: tid=0 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 client.34016 shutdown 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 -- 172.21.15.105:0/3069006944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 msgr2=0x55c7ff2014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff2014f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4012790 tx=0x7f6cc4012c50 comp rx=0 tx=0).stop 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 -- 172.21.15.105:0/3069006944 shutdown_connections 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c7ff20d040 0x55c7ff20f4e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff208530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cb2ffd640 0 client.34016 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff2014f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/3069006944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 0x55c7ff200fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 -- 172.21.15.105:0/3069006944 >> 172.21.15.105:0/3069006944 conn(0x55c7ff1f1fa0 msgr2=0x55c7ff1f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 -- 172.21.15.105:0/3069006944 shutdown_connections 2024-03-26T18:03:16.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 1 -- 172.21.15.105:0/3069006944 wait complete. 2024-03-26T18:03:16.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.367+0000 7f6cdcda0740 20 client.34016 trim_cache size 0 max 16384 2024-03-26T18:03:16.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.391+0000 7f6cdcda0740 1 Processor -- start 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cdcda0740 1 -- start start 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f6a30 0x55c7ff1ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f7420 0x55c7ff2084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41a6bd0 con 0x55c7ff1fc590 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41a8710 con 0x55c7ff1f6a30 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41a72a0 con 0x55c7ff1f7420 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f7420 0x55c7ff2084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f7420 0x55c7ff2084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55212/0 (socket says 172.21.15.105:55212) 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6cd37fe640 1 -- 172.21.15.105:0/394838657 learned_addr learned my addr 172.21.15.105:0/394838657 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.392+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cd37fe640 1 -- 172.21.15.105:0/394838657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f6a30 msgr2=0x55c7ff1ffaa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f6a30 0x55c7ff1ffaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cd37fe640 1 -- 172.21.15.105:0/394838657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 msgr2=0x55c7ff1fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cd37fe640 1 -- 172.21.15.105:0/394838657 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ff1f2d10 con 0x55c7ff1f7420 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f7420 0x55c7ff2084f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4026640 tx=0x7f6cc4009860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7829ddcba89b4479 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cc97fa640 1 -- 172.21.15.105:0/394838657 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400f920 con 0x55c7ff1f7420 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cc97fa640 1 -- 172.21.15.105:0/394838657 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cc400fac0 con 0x55c7ff1f7420 2024-03-26T18:03:16.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.393+0000 7f6cc97fa640 1 -- 172.21.15.105:0/394838657 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40107d0 con 0x55c7ff1f7420 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 -- 172.21.15.105:0/394838657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f7420 msgr2=0x55c7ff2084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f7420 0x55c7ff2084f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4026640 tx=0x7f6cc4009860 comp rx=0 tx=0).stop 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 -- 172.21.15.105:0/394838657 shutdown_connections 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff1fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f7420 0x55c7ff2084f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/394838657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f6a30 0x55c7ff1ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 -- 172.21.15.105:0/394838657 >> 172.21.15.105:0/394838657 conn(0x55c7ff1f1fa0 msgr2=0x55c7ff1f89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 -- 172.21.15.105:0/394838657 shutdown_connections 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 -- 172.21.15.105:0/394838657 wait complete. 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.394+0000 7f6cdcda0740 1 -- start start 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cdcda0740 1 Processor -- start 2024-03-26T18:03:16.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 0x55c7ff208390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff204e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cdcda0740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff2053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41a8040 con 0x55c7ff1fc590 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41a7970 con 0x55c7ff1f7420 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cdcda0740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd41a4bb0 con 0x55c7ff1f6a30 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cdcda0740 10 client.0 _ll_register_callbacks cb 0x7f6cc40099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff204e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 0x55c7ff208390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff204e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38362/0 (socket says 172.21.15.105:38362) 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6ccbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff2053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.395+0000 7f6cd37fe640 1 -- 172.21.15.105:0/223139194 learned_addr learned my addr 172.21.15.105:0/223139194 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6ccbfff640 1 -- 172.21.15.105:0/223139194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 msgr2=0x55c7ff208390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 0x55c7ff208390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6ccbfff640 1 -- 172.21.15.105:0/223139194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 msgr2=0x55c7ff204e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff204e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6ccbfff640 1 -- 172.21.15.105:0/223139194 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ff271410 con 0x55c7ff1fc590 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff204e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6ccbfff640 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff2053a0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc02c190 tx=0x7f6cbc02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ab29f711f5d6f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6cb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/223139194 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc054c30 con 0x55c7ff1fc590 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/223139194 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6cbc054dd0 con 0x55c7ff1fc590 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6cdcda0740 1 -- 172.21.15.105:0/223139194 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc400fac0 con 0x55c7ff1fc590 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.396+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/223139194 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f6cbc054170 con 0x55c7ff1fc590 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cdcda0740 10 client.24366 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cdcda0740 1 -- 172.21.15.105:0/223139194 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c7ff272720 con 0x55c7ff1fc590 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/223139194 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cc400fac0 con 0x55c7ff1fc590 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/223139194 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f6cbc0347b0 con 0x55c7ff1fc590 2024-03-26T18:03:16.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cdcda0740 10 client.24366 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cdcda0740 10 client.24366 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cdcda0740 20 client.24366 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c7ff272c90 0x55c7ff275080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cdcda0740 4 client.24366 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cdcda0740 1 -- 172.21.15.105:0/223139194 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7ff163200 con 0x55c7ff272c90 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.397+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c7ff272c90 0x55c7ff275080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.398+0000 7f6cd37fe640 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c7ff272c90 0x55c7ff275080 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4ca1fab00a1bdc14 server_cookie=156ec7ca6242b7ac in_seq=0 out_seq=0 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.398+0000 7f6cb2ffd640 10 client.24366 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.398+0000 7f6cb2ffd640 1 -- 172.21.15.105:0/223139194 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7ff163200 con 0x55c7ff272c90 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.398+0000 7f6cb2ffd640 10 client.24366 handle_command_reply: tid=0 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.398+0000 7f6cdcda0740 1 client.24366 shutdown 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.398+0000 7f6cdcda0740 1 -- 172.21.15.105:0/223139194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 msgr2=0x55c7ff2053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.398+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff2053a0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc02c190 tx=0x7f6cbc02bc50 comp rx=0 tx=0).stop 2024-03-26T18:03:16.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 1 -- 172.21.15.105:0/223139194 shutdown_connections 2024-03-26T18:03:16.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55c7ff272c90 0x55c7ff275080 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55c7ff1fc590 0x55c7ff2053a0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c7ff1f7420 0x55c7ff204e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 1 --2- 172.21.15.105:0/223139194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c7ff1f6a30 0x55c7ff208390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.384 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:16 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3437967203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:03:16.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 1 -- 172.21.15.105:0/223139194 >> 172.21.15.105:0/223139194 conn(0x55c7ff1f1fa0 msgr2=0x55c7ff1f5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cb2ffd640 0 client.24366 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:03:16.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 1 -- 172.21.15.105:0/223139194 shutdown_connections 2024-03-26T18:03:16.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 1 -- 172.21.15.105:0/223139194 wait complete. 2024-03-26T18:03:16.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cdcda0740 20 client.24366 trim_cache size 0 max 16384 2024-03-26T18:03:16.385 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6cc00ab7b0 msgr2=0x7f6cc00adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.399+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6cc00ab7b0 0x7f6cc00adc70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0cfca0 tx=0x7f6ccc00ce70 comp rx=0 tx=0).stop 2024-03-26T18:03:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cd41075a0 msgr2=0x7f6cd419f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cd41075a0 0x7f6cd419f110 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00ec50 tx=0x7f6ccc00c6a0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 shutdown_connections 2024-03-26T18:03:16.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f6cc00ab7b0 0x7f6cc00adc70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6cd41075a0 0x7f6cd419f110 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cd4104c70 0x7f6cd41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 --2- 172.21.15.105:0/1829430673 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6cd4102340 0x7f6cd4197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 >> 172.21.15.105:0/1829430673 conn(0x7f6cd40fbd50 msgr2=0x7f6cd4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 shutdown_connections 2024-03-26T18:03:16.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.400+0000 7f6cda1b2640 1 -- 172.21.15.105:0/1829430673 wait complete. 2024-03-26T18:03:16.395 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T18:03:16.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:16 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3437967203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 -- 172.21.15.105:0/1182601878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 msgr2=0x7f3f54104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/1182601878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 0x7f3f54104790 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009900 tx=0x7f3f5002bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 -- 172.21.15.105:0/1182601878 shutdown_connections 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/1182601878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f54107600 0x7f3f541099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/1182601878 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f54104cd0 0x7f3f541070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/1182601878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 0x7f3f54104790 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 -- 172.21.15.105:0/1182601878 >> 172.21.15.105:0/1182601878 conn(0x7f3f540fbd50 msgr2=0x7f3f540fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 -- 172.21.15.105:0/1182601878 shutdown_connections 2024-03-26T18:03:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.566+0000 7f3f5ba36640 1 -- 172.21.15.105:0/1182601878 wait complete. 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f5ba36640 1 Processor -- start 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f5ba36640 1 -- start start 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f5ba36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 0x7f3f54197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f5ba36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f54104cd0 0x7f3f541980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f5ba36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f54107600 0x7f3f5419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f5ba36640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410d410 con 0x7f3f541023a0 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f5ba36640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410d290 con 0x7f3f54104cd0 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f5ba36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410d590 con 0x7f3f54107600 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f597d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 0x7f3f54197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f597d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 0x7f3f54197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55708/0 (socket says 172.21.15.105:55708) 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f597d2640 1 -- 172.21.15.105:0/3170044208 learned_addr learned my addr 172.21.15.105:0/3170044208 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f54104cd0 0x7f3f541980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.567+0000 7f3f59fd3640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f54107600 0x7f3f5419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f597d2640 1 -- 172.21.15.105:0/3170044208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f54107600 msgr2=0x7f3f5419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f597d2640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f54107600 0x7f3f5419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f597d2640 1 -- 172.21.15.105:0/3170044208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f54104cd0 msgr2=0x7f3f541980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f597d2640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f54104cd0 0x7f3f541980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f597d2640 1 -- 172.21.15.105:0/3170044208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5419f7a0 con 0x7f3f541023a0 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f54104cd0 0x7f3f541980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f597d2640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 0x7f3f54197b80 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f3f50006010 tx=0x7f3f50005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f4a7fc640 1 -- 172.21.15.105:0/3170044208 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50043020 con 0x7f3f541023a0 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f4a7fc640 1 -- 172.21.15.105:0/3170044208 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f500043d0 con 0x7f3f541023a0 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f5ba36640 1 -- 172.21.15.105:0/3170044208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5419fa30 con 0x7f3f541023a0 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f4a7fc640 1 -- 172.21.15.105:0/3170044208 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50003b80 con 0x7f3f541023a0 2024-03-26T18:03:16.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.568+0000 7f3f5ba36640 1 -- 172.21.15.105:0/3170044208 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5419ffd0 con 0x7f3f541023a0 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.571+0000 7f3f4a7fc640 1 -- 172.21.15.105:0/3170044208 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3f50046020 con 0x7f3f541023a0 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f5e626740 1 Processor -- start 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f5e626740 1 -- start start 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75bddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f5e626740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5573c7522970 con 0x5573c75bd9b0 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f5e626740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5573c74cad70 con 0x5573c75c1500 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f5e626740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5573c751e920 con 0x5573c75be3a0 2024-03-26T18:03:16.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f59fd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75bddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f58fd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f59fd3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75bddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55714/0 (socket says 172.21.15.105:55714) 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f59fd3640 1 -- 172.21.15.105:0/4200155932 learned_addr learned my addr 172.21.15.105:0/4200155932 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.572+0000 7f3f597d2640 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f58fd1640 1 -- 172.21.15.105:0/4200155932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 msgr2=0x5573c75c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f58fd1640 1 -- 172.21.15.105:0/4200155932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 msgr2=0x5573c75bddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75bddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f58fd1640 1 -- 172.21.15.105:0/4200155932 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c72363f0 con 0x5573c75c1500 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f597d2640 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f59fd3640 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75bddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75ce370 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f54069520 tx=0x7f3f440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35240597cb471fbf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f4a7fc640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f400a2fa0 0x7f3f400a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f59fd3640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f400a2fa0 0x7f3f400a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f3f7fe640 1 -- 172.21.15.105:0/4200155932 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44004030 con 0x5573c75c1500 2024-03-26T18:03:16.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.573+0000 7f3f3f7fe640 1 -- 172.21.15.105:0/4200155932 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f440054d0 con 0x5573c75c1500 2024-03-26T18:03:16.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f3f7fe640 1 -- 172.21.15.105:0/4200155932 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44004ce0 con 0x5573c75c1500 2024-03-26T18:03:16.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f4a7fc640 1 -- 172.21.15.105:0/3170044208 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f3f500f15a0 con 0x7f3f541023a0 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f5e626740 1 -- 172.21.15.105:0/4200155932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 msgr2=0x5573c75ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f59fd3640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f400a2fa0 0x7f3f400a5460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x5573c75bec00 tx=0x7f3f4c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75ce370 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f54069520 tx=0x7f3f440118d0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f5e626740 1 -- 172.21.15.105:0/4200155932 shutdown_connections 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75ce370 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4200155932 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75bddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.574+0000 7f3f5e626740 1 -- 172.21.15.105:0/4200155932 >> 172.21.15.105:0/4200155932 conn(0x5573c75b8fa0 msgr2=0x5573c75b93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.575+0000 7f3f5e626740 1 -- 172.21.15.105:0/4200155932 shutdown_connections 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.575+0000 7f3f5e626740 1 -- 172.21.15.105:0/4200155932 wait complete. 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.575+0000 7f3f5e626740 1 -- start start 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.575+0000 7f3f5e626740 1 Processor -- start 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.575+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.575+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 0x5573c75d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f5e626740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5573c75cee10 con 0x5573c75bd9b0 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f5e626740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5573c74de310 con 0x5573c75be3a0 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f5e626740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5573c75cef90 con 0x5573c75c1500 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38398/0 (socket says 172.21.15.105:38398) 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 -- 172.21.15.105:0/4249876185 learned_addr learned my addr 172.21.15.105:0/4249876185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f59fd3640 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 0x5573c75d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f597d2640 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f5e626740 10 client.0 _ll_register_callbacks cb 0x5573c7519c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 -- 172.21.15.105:0/4249876185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 msgr2=0x5573c75d7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 0x5573c75d7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 -- 172.21.15.105:0/4249876185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 msgr2=0x5573c75d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 -- 172.21.15.105:0/4249876185 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c73007d0 con 0x5573c75be3a0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f597d2640 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f5e626740 1 -- 172.21.15.105:0/4249876185 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f440054d0 con 0x5573c75be3a0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009560 tx=0x7f3f44003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e0392898108afd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.576+0000 7f3f3cff9640 10 client.34031 ms_handle_connect on v2:172.21.15.135:3300/0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f3cff9640 1 -- 172.21.15.105:0/4249876185 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44028020 con 0x5573c75be3a0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f3cff9640 1 -- 172.21.15.105:0/4249876185 <== mon.2 v2:172.21.15.135:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f3f44016040 con 0x5573c75be3a0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f3cff9640 1 -- 172.21.15.105:0/4249876185 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3f44003ba0 con 0x5573c75be3a0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f5e626740 10 client.34031 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f5e626740 1 -- 172.21.15.105:0/4249876185 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=45}) -- 0x5573c75dafb0 con 0x5573c75be3a0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f3cff9640 1 -- 172.21.15.105:0/4249876185 <== mon.2 v2:172.21.15.135:3300/0 4 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f3f440049e0 con 0x5573c75be3a0 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f5e626740 10 client.34031 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f5e626740 10 client.34031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:03:16.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f5e626740 20 client.34031 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5573c75dc8c0 0x5573c75ded80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f5e626740 4 client.34031 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:16.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f5e626740 1 -- 172.21.15.105:0/4249876185 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573c752feb0 con 0x5573c75dc8c0 2024-03-26T18:03:16.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.577+0000 7f3f597d2640 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5573c75dc8c0 0x5573c75ded80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.578+0000 7f3f597d2640 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5573c75dc8c0 0x5573c75ded80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3f4b5eefdd94dfd server_cookie=e6a57e82db9717f1 in_seq=0 out_seq=0 2024-03-26T18:03:16.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.578+0000 7f3f3cff9640 10 client.34031 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:03:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f3cff9640 1 -- 172.21.15.105:0/4249876185 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5573c752feb0 con 0x5573c75dc8c0 2024-03-26T18:03:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f3cff9640 10 client.34031 handle_command_reply: tid=0 2024-03-26T18:03:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 client.34031 shutdown 2024-03-26T18:03:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 -- 172.21.15.105:0/4249876185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75be3a0 msgr2=0x5573c75d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009560 tx=0x7f3f44003820 comp rx=0 tx=0).stop 2024-03-26T18:03:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 -- 172.21.15.105:0/4249876185 shutdown_connections 2024-03-26T18:03:16.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 0x5573c75d7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5573c75dc8c0 0x5573c75ded80 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 --2- 172.21.15.105:0/4249876185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 -- 172.21.15.105:0/4249876185 >> 172.21.15.105:0/4249876185 conn(0x5573c75b8fa0 msgr2=0x5573c75b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f3cff9640 0 client.34031 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:03:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 -- 172.21.15.105:0/4249876185 shutdown_connections 2024-03-26T18:03:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 1 -- 172.21.15.105:0/4249876185 wait complete. 2024-03-26T18:03:16.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.579+0000 7f3f5e626740 20 client.34031 trim_cache size 0 max 16384 2024-03-26T18:03:16.565 INFO:tasks.workunit.client.0.smithi105.stdout: 10 93816 7.00 MB/sec execute 599 sec latency 144.865 ms 2024-03-26T18:03:16.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f5e626740 1 Processor -- start 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f5e626740 1 -- start start 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 0x5573c75d7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f5e626740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f541a6ce0 con 0x5573c75be3a0 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f5e626740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f541a8820 con 0x5573c75bd9b0 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f5e626740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f541a73b0 con 0x5573c75c1500 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f58fd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f597d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.602+0000 7f3f597d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38404/0 (socket says 172.21.15.105:38404) 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f597d2640 1 -- 172.21.15.105:0/3842079470 learned_addr learned my addr 172.21.15.105:0/3842079470 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f58fd1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55746/0 (socket says 172.21.15.105:55746) 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f59fd3640 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 0x5573c75d7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f58fd1640 1 -- 172.21.15.105:0/3842079470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 msgr2=0x5573c75d7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 0x5573c75d7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f58fd1640 1 -- 172.21.15.105:0/3842079470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75bd9b0 msgr2=0x5573c75d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f58fd1640 1 -- 172.21.15.105:0/3842079470 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c75ce6e0 con 0x5573c75be3a0 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f597d2640 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f3f44000c00 tx=0x7f3f44024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf925e51e6d3a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f3effd640 1 -- 172.21.15.105:0/3842079470 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44028040 con 0x5573c75be3a0 2024-03-26T18:03:16.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f3effd640 1 -- 172.21.15.105:0/3842079470 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f440054d0 con 0x5573c75be3a0 2024-03-26T18:03:16.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.603+0000 7f3f3effd640 1 -- 172.21.15.105:0/3842079470 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f440196b0 con 0x5573c75be3a0 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 -- 172.21.15.105:0/3842079470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75be3a0 msgr2=0x5573c75d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f3f44000c00 tx=0x7f3f44024000 comp rx=0 tx=0).stop 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 -- 172.21.15.105:0/3842079470 shutdown_connections 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75c1500 0x5573c75d7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75be3a0 0x5573c75d0c60 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3842079470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75bd9b0 0x5573c75d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 -- 172.21.15.105:0/3842079470 >> 172.21.15.105:0/3842079470 conn(0x5573c75b8fa0 msgr2=0x5573c75b9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 -- 172.21.15.105:0/3842079470 shutdown_connections 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.604+0000 7f3f5e626740 1 -- 172.21.15.105:0/3842079470 wait complete. 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f5e626740 1 -- start start 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f5e626740 1 Processor -- start 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.592 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75d7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f5e626740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f5e626740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f541a8150 con 0x5573c75bd9b0 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f5e626740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3f541a7a80 con 0x5573c75c1500 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f5e626740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f541a4cc0 con 0x5573c75be3a0 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f597d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.605+0000 7f3f58fd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75d7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f58fd1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75d7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55262/0 (socket says 172.21.15.105:55262) 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f597d2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55748/0 (socket says 172.21.15.105:55748) 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f58fd1640 1 -- 172.21.15.105:0/3878035945 learned_addr learned my addr 172.21.15.105:0/3878035945 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f59fd3640 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f5e626740 10 client.0 _ll_register_callbacks cb 0x7f3f4c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f58fd1640 1 -- 172.21.15.105:0/3878035945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 msgr2=0x5573c75d4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75d4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f58fd1640 1 -- 172.21.15.105:0/3878035945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 msgr2=0x5573c75d7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f58fd1640 1 -- 172.21.15.105:0/3878035945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c75d47e0 con 0x5573c75be3a0 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f597d2640 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f58fd1640 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75d7aa0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3f44011dd0 tx=0x7f3f44004cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bd5c9f47224dc61 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f3cff9640 1 -- 172.21.15.105:0/3878035945 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44028070 con 0x5573c75be3a0 2024-03-26T18:03:16.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.606+0000 7f3f3cff9640 1 -- 172.21.15.105:0/3878035945 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f3f44019860 con 0x5573c75be3a0 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 1 -- 172.21.15.105:0/3878035945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f440054d0 con 0x5573c75be3a0 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f3cff9640 1 -- 172.21.15.105:0/3878035945 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3f44019a60 con 0x5573c75be3a0 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 10 client.34045 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 1 -- 172.21.15.105:0/3878035945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5573c7641920 con 0x5573c75be3a0 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f3cff9640 1 -- 172.21.15.105:0/3878035945 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f440054d0 con 0x5573c75be3a0 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f3cff9640 1 -- 172.21.15.105:0/3878035945 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f3f44023900 con 0x5573c75be3a0 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 10 client.34045 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 10 client.34045 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 20 client.34045 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5573c7642070 0x5573c7644460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 4 client.34045 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.607+0000 7f3f5e626740 1 -- 172.21.15.105:0/3878035945 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573c752a200 con 0x5573c7642070 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.608+0000 7f3f597d2640 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5573c7642070 0x5573c7644460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.608+0000 7f3f597d2640 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5573c7642070 0x5573c7644460 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e50a85ea179530b7 server_cookie=14cab83ebee1d631 in_seq=0 out_seq=0 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.608+0000 7f3f3cff9640 10 client.34045 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.608+0000 7f3f3cff9640 1 -- 172.21.15.105:0/3878035945 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573c752a200 con 0x5573c7642070 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.608+0000 7f3f3cff9640 10 client.34045 handle_command_reply: tid=0 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.608+0000 7f3f5e626740 1 client.34045 shutdown 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.608+0000 7f3f5e626740 1 -- 172.21.15.105:0/3878035945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 msgr2=0x5573c75d7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.608+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75d7aa0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3f44011dd0 tx=0x7f3f44004cd0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 1 -- 172.21.15.105:0/3878035945 shutdown_connections 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x5573c7642070 0x5573c7644460 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5573c75c1500 0x5573c75d4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5573c75be3a0 0x5573c75d7aa0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f3cff9640 0 client.34045 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:03:16.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 1 --2- 172.21.15.105:0/3878035945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5573c75bd9b0 0x5573c75d7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 1 -- 172.21.15.105:0/3878035945 >> 172.21.15.105:0/3878035945 conn(0x5573c75b8fa0 msgr2=0x5573c75cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 1 -- 172.21.15.105:0/3878035945 shutdown_connections 2024-03-26T18:03:16.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 1 -- 172.21.15.105:0/3878035945 wait complete. 2024-03-26T18:03:16.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5e626740 20 client.34045 trim_cache size 0 max 16384 2024-03-26T18:03:16.597 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5ba36640 1 -- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f400a2fa0 msgr2=0x7f3f400a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.609+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f400a2fa0 0x7f3f400a5460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x5573c75bec00 tx=0x7f3f4c009500 comp rx=0 tx=0).stop 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 -- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 msgr2=0x7f3f54197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 0x7f3f54197b80 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f3f50006010 tx=0x7f3f50005e40 comp rx=0 tx=0).stop 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 -- 172.21.15.105:0/3170044208 shutdown_connections 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3f400a2fa0 0x7f3f400a5460 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f54107600 0x7f3f5419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3f54104cd0 0x7f3f541980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.598 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 --2- 172.21.15.105:0/3170044208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f541023a0 0x7f3f54197b80 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 -- 172.21.15.105:0/3170044208 >> 172.21.15.105:0/3170044208 conn(0x7f3f540fbd50 msgr2=0x7f3f54102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 -- 172.21.15.105:0/3170044208 shutdown_connections 2024-03-26T18:03:16.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.610+0000 7f3f5ba36640 1 -- 172.21.15.105:0/3170044208 wait complete. 2024-03-26T18:03:16.606 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T18:03:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 -- 172.21.15.105:0/1075432479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601018f0 msgr2=0x7eff60101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 --2- 172.21.15.105:0/1075432479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601018f0 0x7eff60101d10 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7eff50009900 tx=0x7eff5002bc40 comp rx=0 tx=0).stop 2024-03-26T18:03:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 -- 172.21.15.105:0/1075432479 shutdown_connections 2024-03-26T18:03:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 --2- 172.21.15.105:0/1075432479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff601034a0 0x7eff60109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 --2- 172.21.15.105:0/1075432479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff60102ae0 0x7eff60102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 --2- 172.21.15.105:0/1075432479 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601018f0 0x7eff60101d10 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 -- 172.21.15.105:0/1075432479 >> 172.21.15.105:0/1075432479 conn(0x7eff600fce60 msgr2=0x7eff600ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 -- 172.21.15.105:0/1075432479 shutdown_connections 2024-03-26T18:03:16.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.766+0000 7eff656a1640 1 -- 172.21.15.105:0/1075432479 wait complete. 2024-03-26T18:03:16.751 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.767+0000 7eff656a1640 1 Processor -- start 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.767+0000 7eff656a1640 1 -- start start 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff656a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff601018f0 0x7eff60075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff656a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff60102ae0 0x7eff60073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff656a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601034a0 0x7eff600741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff656a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010d440 con 0x7eff601034a0 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff656a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010d2c0 con 0x7eff601018f0 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff656a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff6010d5c0 con 0x7eff60102ae0 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff601018f0 0x7eff60075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff601018f0 0x7eff60075670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38428/0 (socket says 172.21.15.105:38428) 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5effd640 1 -- 172.21.15.105:0/510553692 learned_addr learned my addr 172.21.15.105:0/510553692 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5f7fe640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601034a0 0x7eff600741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff60102ae0 0x7eff60073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5f7fe640 1 -- 172.21.15.105:0/510553692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff60102ae0 msgr2=0x7eff60073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5f7fe640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff60102ae0 0x7eff60073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5f7fe640 1 -- 172.21.15.105:0/510553692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff601018f0 msgr2=0x7eff60075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5f7fe640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff601018f0 0x7eff60075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5f7fe640 1 -- 172.21.15.105:0/510553692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff60074a80 con 0x7eff601034a0 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.768+0000 7eff5f7fe640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601034a0 0x7eff600741f0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7eff5800ec50 tx=0x7eff5800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.769+0000 7eff47fff640 1 -- 172.21.15.105:0/510553692 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58003b70 con 0x7eff601034a0 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.769+0000 7eff47fff640 1 -- 172.21.15.105:0/510553692 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff58003d10 con 0x7eff601034a0 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.769+0000 7eff47fff640 1 -- 172.21.15.105:0/510553692 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58010760 con 0x7eff601034a0 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.769+0000 7eff656a1640 1 -- 172.21.15.105:0/510553692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601a1cc0 con 0x7eff601034a0 2024-03-26T18:03:16.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.769+0000 7eff656a1640 1 -- 172.21.15.105:0/510553692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff601a21c0 con 0x7eff601034a0 2024-03-26T18:03:16.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff6836e740 1 Processor -- start 2024-03-26T18:03:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff47fff640 1 -- 172.21.15.105:0/510553692 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7eff58004020 con 0x7eff601034a0 2024-03-26T18:03:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff6836e740 1 -- start start 2024-03-26T18:03:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff6836e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f6a30 0x56273c9f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff6836e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273c9fbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff6836e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9fc590 0x56273c9fea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff6836e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56273c95b970 con 0x56273c9fc590 2024-03-26T18:03:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff6836e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56273c903d70 con 0x56273c9f6a30 2024-03-26T18:03:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.773+0000 7eff6836e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56273c957920 con 0x56273c9f7420 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f6a30 0x56273c9f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273c9fbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f6a30 0x56273c9f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38432/0 (socket says 172.21.15.105:38432) 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5e7fc640 1 -- 172.21.15.105:0/115251343 learned_addr learned my addr 172.21.15.105:0/115251343 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9fc590 0x56273c9fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5effd640 1 -- 172.21.15.105:0/115251343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f6a30 msgr2=0x56273c9f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5effd640 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f6a30 0x56273c9f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5effd640 1 -- 172.21.15.105:0/115251343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9fc590 msgr2=0x56273c9fea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff47fff640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7eff480a2fa0 0x7eff480a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5effd640 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9fc590 0x56273c9fea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5effd640 1 -- 172.21.15.105:0/115251343 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273c66f3f0 con 0x56273c9f7420 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f6a30 0x56273c9f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff47fff640 1 -- 172.21.15.105:0/510553692 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7eff580cf990 con 0x7eff601034a0 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5f7fe640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7eff480a2fa0 0x7eff480a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff5effd640 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273c9fbf20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7eff50005ab0 tx=0x7eff50005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=559b0ad715621cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.774+0000 7eff457fa640 1 -- 172.21.15.105:0/115251343 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50045020 con 0x56273c9f7420 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.775+0000 7eff457fa640 1 -- 172.21.15.105:0/115251343 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff5002c9b0 con 0x56273c9f7420 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.775+0000 7eff5f7fe640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7eff480a2fa0 0x7eff480a5460 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7eff580cd7c0 tx=0x7eff580045e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.775+0000 7eff457fa640 1 -- 172.21.15.105:0/115251343 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50005570 con 0x56273c9f7420 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.775+0000 7eff6836e740 1 -- 172.21.15.105:0/115251343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 msgr2=0x56273c9fbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.775+0000 7eff6836e740 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273c9fbf20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7eff50005ab0 tx=0x7eff50005df0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 -- 172.21.15.105:0/115251343 shutdown_connections 2024-03-26T18:03:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9fc590 0x56273c9fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273c9fbf20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 --2- 172.21.15.105:0/115251343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f6a30 0x56273c9f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 -- 172.21.15.105:0/115251343 >> 172.21.15.105:0/115251343 conn(0x56273c9f1fa0 msgr2=0x56273c9f23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 -- 172.21.15.105:0/115251343 shutdown_connections 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 -- 172.21.15.105:0/115251343 wait complete. 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 -- start start 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 Processor -- start 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca00ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273ca01430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9fc590 0x56273ca08400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56273c9ff4e0 con 0x56273c9f6a30 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56273c917310 con 0x56273c9fc590 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56273c9ff660 con 0x56273c9f7420 2024-03-26T18:03:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca00ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff6836e740 10 client.0 _ll_register_callbacks cb 0x56273c952c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273ca01430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff5effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273ca01430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55300/0 (socket says 172.21.15.105:55300) 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.776+0000 7eff5effd640 1 -- 172.21.15.105:0/3814441116 learned_addr learned my addr 172.21.15.105:0/3814441116 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff5effd640 1 -- 172.21.15.105:0/3814441116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9fc590 msgr2=0x56273ca08400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff5effd640 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9fc590 0x56273ca08400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff5effd640 1 -- 172.21.15.105:0/3814441116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 msgr2=0x56273ca00ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff5effd640 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca00ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff5effd640 1 -- 172.21.15.105:0/3814441116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56273c7397d0 con 0x56273c9f7420 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca00ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff5effd640 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273ca01430 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7eff50006010 tx=0x7eff50005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc61be9a78f9d71 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:03:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff42ff5640 1 -- 172.21.15.105:0/3814441116 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5004e070 con 0x56273c9f7420 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff6836e740 1 -- 172.21.15.105:0/3814441116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5002c9b0 con 0x56273c9f7420 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff42ff5640 1 -- 172.21.15.105:0/3814441116 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7eff500046c0 con 0x56273c9f7420 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff42ff5640 1 -- 172.21.15.105:0/3814441116 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7eff5005a050 con 0x56273c9f7420 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff6836e740 10 client.34057 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff6836e740 1 -- 172.21.15.105:0/3814441116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56273c9ff5a0 con 0x56273c9f7420 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.777+0000 7eff42ff5640 1 -- 172.21.15.105:0/3814441116 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff5002c9b0 con 0x56273c9f7420 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.778+0000 7eff42ff5640 1 -- 172.21.15.105:0/3814441116 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7eff50003a00 con 0x56273c9f7420 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.778+0000 7eff6836e740 10 client.34057 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.778+0000 7eff6836e740 10 client.34057 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.778+0000 7eff6836e740 20 client.34057 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.778+0000 7eff6836e740 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56273ca0c9c0 0x56273ca0ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.778+0000 7eff6836e740 4 client.34057 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.778+0000 7eff6836e740 1 -- 172.21.15.105:0/3814441116 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56273c968eb0 con 0x56273ca0c9c0 2024-03-26T18:03:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.778+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56273ca0c9c0 0x56273ca0ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.779+0000 7eff42ff5640 10 client.34057 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:03:16.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.779+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56273ca0c9c0 0x56273ca0ee30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11aae4928e5d5d9f server_cookie=f67de2a5ab6244f6 in_seq=0 out_seq=0 2024-03-26T18:03:16.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff42ff5640 1 -- 172.21.15.105:0/3814441116 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56273c968eb0 con 0x56273ca0c9c0 2024-03-26T18:03:16.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff42ff5640 10 client.34057 handle_command_reply: tid=0 2024-03-26T18:03:16.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 client.34057 shutdown 2024-03-26T18:03:16.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 -- 172.21.15.105:0/3814441116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 msgr2=0x56273ca01430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273ca01430 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7eff50006010 tx=0x7eff50005770 comp rx=0 tx=0).stop 2024-03-26T18:03:16.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 -- 172.21.15.105:0/3814441116 shutdown_connections 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56273ca0c9c0 0x56273ca0ee30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9fc590 0x56273ca08400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff42ff5640 0 client.34057 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f7420 0x56273ca01430 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 --2- 172.21.15.105:0/3814441116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca00ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 -- 172.21.15.105:0/3814441116 >> 172.21.15.105:0/3814441116 conn(0x56273c9f1fa0 msgr2=0x56273c9f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 -- 172.21.15.105:0/3814441116 shutdown_connections 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 1 -- 172.21.15.105:0/3814441116 wait complete. 2024-03-26T18:03:16.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.780+0000 7eff6836e740 20 client.34057 trim_cache size 0 max 16384 2024-03-26T18:03:16.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff6836e740 1 Processor -- start 2024-03-26T18:03:16.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff6836e740 1 -- start start 2024-03-26T18:03:16.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff6836e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f1fa0 0x56273c9f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff6836e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f1fa0 0x56273c9f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f1fa0 0x56273c9f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55314/0 (socket says 172.21.15.105:55314) 2024-03-26T18:03:16.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff6836e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5e7fc640 1 -- 172.21.15.105:0/2442770326 learned_addr learned my addr 172.21.15.105:0/2442770326 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff6836e740 1 -- 172.21.15.105:0/2442770326 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff601aafa0 con 0x56273c9f6a30 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff6836e740 1 -- 172.21.15.105:0/2442770326 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7eff601acae0 con 0x56273c9f7420 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff6836e740 1 -- 172.21.15.105:0/2442770326 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff601ab670 con 0x56273c9f1fa0 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5effd640 1 -- 172.21.15.105:0/2442770326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f1fa0 msgr2=0x56273c9f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5effd640 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f1fa0 0x56273c9f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5effd640 1 -- 172.21.15.105:0/2442770326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 msgr2=0x56273ca136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5effd640 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5effd640 1 -- 172.21.15.105:0/2442770326 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273c9f1dd0 con 0x56273c9f6a30 2024-03-26T18:03:16.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.799+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T18:03:16.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.800+0000 7eff5effd640 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca083c0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7eff50036040 tx=0x7eff5002bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2374d5a70b6181 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.800+0000 7eff44ff9640 1 -- 172.21.15.105:0/2442770326 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50045040 con 0x56273c9f6a30 2024-03-26T18:03:16.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.800+0000 7eff44ff9640 1 -- 172.21.15.105:0/2442770326 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff5002c8f0 con 0x56273c9f6a30 2024-03-26T18:03:16.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.800+0000 7eff44ff9640 1 -- 172.21.15.105:0/2442770326 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5004e040 con 0x56273c9f6a30 2024-03-26T18:03:16.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 -- 172.21.15.105:0/2442770326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 msgr2=0x56273ca083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca083c0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7eff50036040 tx=0x7eff5002bc40 comp rx=0 tx=0).stop 2024-03-26T18:03:16.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 -- 172.21.15.105:0/2442770326 shutdown_connections 2024-03-26T18:03:16.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f6a30 0x56273ca083c0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 --2- 172.21.15.105:0/2442770326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f1fa0 0x56273c9f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 -- 172.21.15.105:0/2442770326 >> 172.21.15.105:0/2442770326 conn(0x56273c9f8200 msgr2=0x56273c9f5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 -- 172.21.15.105:0/2442770326 shutdown_connections 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.801+0000 7eff6836e740 1 -- 172.21.15.105:0/2442770326 wait complete. 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 1 -- start start 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 1 Processor -- start 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f1fa0 0x56273ca05c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f1fa0 0x56273ca05c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff5e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f1fa0 0x56273ca05c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55812/0 (socket says 172.21.15.105:55812) 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f6a30 0x56273ca06180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff5e7fc640 1 -- 172.21.15.105:0/3772490292 learned_addr learned my addr 172.21.15.105:0/3772490292 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff601ac410 con 0x56273c9f1fa0 2024-03-26T18:03:16.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7eff601abd40 con 0x56273c9f7420 2024-03-26T18:03:16.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff601a8f80 con 0x56273c9f6a30 2024-03-26T18:03:16.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff5effd640 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f6a30 0x56273ca06180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff5e7fc640 1 -- 172.21.15.105:0/3772490292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f6a30 msgr2=0x56273ca06180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f6a30 0x56273ca06180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff5e7fc640 1 -- 172.21.15.105:0/3772490292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 msgr2=0x56273ca07190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca07190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff5e7fc640 1 -- 172.21.15.105:0/3772490292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56273ca07780 con 0x56273c9f1fa0 2024-03-26T18:03:16.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.802+0000 7eff6836e740 10 client.0 _ll_register_callbacks cb 0x7eff54009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff5e7fc640 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f1fa0 0x56273ca05c40 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x56273ca0f6a0 tx=0x7eff54009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4397893515051440 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:16.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5002c8f0 con 0x56273c9f1fa0 2024-03-26T18:03:16.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff42ff5640 1 -- 172.21.15.105:0/3772490292 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54035050 con 0x56273c9f1fa0 2024-03-26T18:03:16.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff42ff5640 1 -- 172.21.15.105:0/3772490292 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7eff54003b10 con 0x56273c9f1fa0 2024-03-26T18:03:16.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff42ff5640 1 -- 172.21.15.105:0/3772490292 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7eff54003d70 con 0x56273c9f1fa0 2024-03-26T18:03:16.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff6836e740 10 client.24402 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=45}) -- 0x56273ca72df0 con 0x56273c9f1fa0 2024-03-26T18:03:16.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.803+0000 7eff42ff5640 1 -- 172.21.15.105:0/3772490292 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7eff54011550 con 0x56273c9f1fa0 2024-03-26T18:03:16.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff6836e740 10 client.24402 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff6836e740 10 client.24402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff6836e740 20 client.24402 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff6836e740 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56273c9fc590 0x56273ca75700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff6836e740 4 client.24402 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7eff5002c8f0 con 0x56273c9fc590 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff5effd640 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56273c9fc590 0x56273ca75700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff42ff5640 10 client.24402 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff5effd640 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56273c9fc590 0x56273ca75700 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44f95646073a0f95 server_cookie=97d8fc31edc8e3c1 in_seq=0 out_seq=0 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff42ff5640 1 -- 172.21.15.105:0/3772490292 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7eff5002c8f0 con 0x56273c9fc590 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff42ff5640 10 client.24402 handle_command_reply: tid=0 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.804+0000 7eff6836e740 1 client.24402 shutdown 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f1fa0 msgr2=0x56273ca05c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f1fa0 0x56273ca05c40 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x56273ca0f6a0 tx=0x7eff54009370 comp rx=0 tx=0).stop 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 shutdown_connections 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x56273c9fc590 0x56273ca75700 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56273c9f7420 0x56273ca07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56273c9f6a30 0x56273ca06180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 --2- 172.21.15.105:0/3772490292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56273c9f1fa0 0x56273ca05c40 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 >> 172.21.15.105:0/3772490292 conn(0x56273c9f8200 msgr2=0x56273c9f5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff42ff5640 0 client.24402 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 shutdown_connections 2024-03-26T18:03:16.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 1 -- 172.21.15.105:0/3772490292 wait complete. 2024-03-26T18:03:16.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.805+0000 7eff6836e740 20 client.24402 trim_cache size 0 max 16384 2024-03-26T18:03:16.799 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:16.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 -- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7eff480a2fa0 msgr2=0x7eff480a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7eff480a2fa0 0x7eff480a5460 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7eff580cd7c0 tx=0x7eff580045e0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 -- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601034a0 msgr2=0x7eff600741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601034a0 0x7eff600741f0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7eff5800ec50 tx=0x7eff5800c6a0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 -- 172.21.15.105:0/510553692 shutdown_connections 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7eff480a2fa0 0x7eff480a5460 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff601034a0 0x7eff600741f0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff60102ae0 0x7eff60073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 --2- 172.21.15.105:0/510553692 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff601018f0 0x7eff60075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 -- 172.21.15.105:0/510553692 >> 172.21.15.105:0/510553692 conn(0x7eff600fce60 msgr2=0x7eff60105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 -- 172.21.15.105:0/510553692 shutdown_connections 2024-03-26T18:03:16.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.806+0000 7eff656a1640 1 -- 172.21.15.105:0/510553692 wait complete. 2024-03-26T18:03:16.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T18:03:16.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.963+0000 7f90c159a640 1 -- 172.21.15.105:0/1966223708 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8034d80 con 0x7f90c410b230 2024-03-26T18:03:16.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 -- 172.21.15.105:0/1966223708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b230 msgr2=0x7f90c4112000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 --2- 172.21.15.105:0/1966223708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b230 0x7f90c4112000 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009860 tx=0x7f90b802bb70 comp rx=0 tx=0).stop 2024-03-26T18:03:16.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 -- 172.21.15.105:0/1966223708 shutdown_connections 2024-03-26T18:03:16.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 --2- 172.21.15.105:0/1966223708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c410b6b0 0x7f90c4114550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.945 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 --2- 172.21.15.105:0/1966223708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b230 0x7f90c4112000 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 --2- 172.21.15.105:0/1966223708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41088c0 0x7f90c410acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 -- 172.21.15.105:0/1966223708 >> 172.21.15.105:0/1966223708 conn(0x7f90c406efc0 msgr2=0x7f90c4071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 -- 172.21.15.105:0/1966223708 shutdown_connections 2024-03-26T18:03:16.946 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.964+0000 7f90c8d10640 1 -- 172.21.15.105:0/1966223708 wait complete. 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c8d10640 1 Processor -- start 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c8d10640 1 -- start start 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c8d10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41088c0 0x7f90c41a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c8d10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c410b230 0x7f90c41a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c8d10640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b6b0 0x7f90c41a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c8d10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4117e10 con 0x7f90c41088c0 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c8d10640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4117c90 con 0x7f90c410b6b0 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c8d10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4117f90 con 0x7f90c410b230 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41088c0 0x7f90c41a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41088c0 0x7f90c41a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55836/0 (socket says 172.21.15.105:55836) 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c259c640 1 -- 172.21.15.105:0/218218015 learned_addr learned my addr 172.21.15.105:0/218218015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b6b0 0x7f90c41a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.965+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c410b230 0x7f90c41a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c1d9b640 1 -- 172.21.15.105:0/218218015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b6b0 msgr2=0x7f90c41a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b6b0 0x7f90c41a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c1d9b640 1 -- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41088c0 msgr2=0x7f90c41a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41088c0 0x7f90c41a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c1d9b640 1 -- 172.21.15.105:0/218218015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c41aa120 con 0x7f90c410b230 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b6b0 0x7f90c41a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c259c640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41088c0 0x7f90c41a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c410b230 0x7f90c41a2a90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f90b802c0b0 tx=0x7f90b80040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90b37fe640 1 -- 172.21.15.105:0/218218015 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8041020 con 0x7f90c410b230 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c8d10640 1 -- 172.21.15.105:0/218218015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c41aa3b0 con 0x7f90c410b230 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90c8d10640 1 -- 172.21.15.105:0/218218015 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c41aa8f0 con 0x7f90c410b230 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90b37fe640 1 -- 172.21.15.105:0/218218015 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90b80044d0 con 0x7f90c410b230 2024-03-26T18:03:16.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.966+0000 7f90b37fe640 1 -- 172.21.15.105:0/218218015 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8039040 con 0x7f90c410b230 2024-03-26T18:03:16.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.969+0000 7f90b37fe640 1 -- 172.21.15.105:0/218218015 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f90b8044020 con 0x7f90c410b230 2024-03-26T18:03:16.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90b37fe640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a2fa0 0x7f90ac0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90cb9dc740 1 Processor -- start 2024-03-26T18:03:16.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90c259c640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a2fa0 0x7f90ac0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90cb9dc740 1 -- start start 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90b37fe640 1 -- 172.21.15.105:0/218218015 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f90b80f0bf0 con 0x7f90c410b230 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dc3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc4420 0x55f3e6dc8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dcba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6d28970 con 0x55f3e6dc3a30 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6cd0d70 con 0x55f3e6dc4420 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6d24920 con 0x55f3e6dc9590 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dcba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90c259c640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a2fa0 0x7f90ac0a5460 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005c40 tx=0x7f90bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dcba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55336/0 (socket says 172.21.15.105:55336) 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90c259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dc3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.970+0000 7f90c2d9d640 1 -- 172.21.15.105:0/2009383672 learned_addr learned my addr 172.21.15.105:0/2009383672 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc4420 0x55f3e6dc8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c2d9d640 1 -- 172.21.15.105:0/2009383672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc4420 msgr2=0x55f3e6dc8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc4420 0x55f3e6dc8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c2d9d640 1 -- 172.21.15.105:0/2009383672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc3a30 msgr2=0x55f3e6dc3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dc3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c2d9d640 1 -- 172.21.15.105:0/2009383672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e6a3c3f0 con 0x55f3e6dc9590 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc4420 0x55f3e6dc8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c259c640 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dc3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dcba20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009fd0 tx=0x7f90b4011920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2454f815fbbb885 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90b0ff9640 1 -- 172.21.15.105:0/2009383672 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b401e070 con 0x55f3e6dc9590 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90b0ff9640 1 -- 172.21.15.105:0/2009383672 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90b400f9f0 con 0x55f3e6dc9590 2024-03-26T18:03:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.971+0000 7f90b0ff9640 1 -- 172.21.15.105:0/2009383672 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4019aa0 con 0x55f3e6dc9590 2024-03-26T18:03:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.972+0000 7f90cb9dc740 1 -- 172.21.15.105:0/2009383672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 msgr2=0x55f3e6dcba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.972+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dcba20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009fd0 tx=0x7f90b4011920 comp rx=0 tx=0).stop 2024-03-26T18:03:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.972+0000 7f90cb9dc740 1 -- 172.21.15.105:0/2009383672 shutdown_connections 2024-03-26T18:03:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.972+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dcba20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.972+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc4420 0x55f3e6dc8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.972+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/2009383672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dc3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.972+0000 7f90cb9dc740 1 -- 172.21.15.105:0/2009383672 >> 172.21.15.105:0/2009383672 conn(0x55f3e6dbefa0 msgr2=0x55f3e6dbf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.973+0000 7f90cb9dc740 1 -- 172.21.15.105:0/2009383672 shutdown_connections 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.973+0000 7f90cb9dc740 1 -- 172.21.15.105:0/2009383672 wait complete. 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.973+0000 7f90cb9dc740 1 -- start start 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.973+0000 7f90cb9dc740 1 Processor -- start 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.973+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.973+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38494/0 (socket says 172.21.15.105:38494) 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c1d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55852/0 (socket says 172.21.15.105:55852) 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c259c640 1 -- 172.21.15.105:0/3448347027 learned_addr learned my addr 172.21.15.105:0/3448347027 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6dcc540 con 0x55f3e6dc4420 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6ce4310 con 0x55f3e6dc3a30 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f3e6dcc6c0 con 0x55f3e6dc9590 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c1d9b640 1 -- 172.21.15.105:0/3448347027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 msgr2=0x55f3e6dd5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c1d9b640 1 -- 172.21.15.105:0/3448347027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 msgr2=0x55f3e6dcdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c1d9b640 1 -- 172.21.15.105:0/3448347027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e6b067d0 con 0x55f3e6dc4420 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c259c640 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009780 tx=0x7f90b80bcbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d89f9258498ef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.974+0000 7f90a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 10 client.0 _ll_register_callbacks cb 0x55f3e6d1fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b400f9f0 con 0x55f3e6dc4420 2024-03-26T18:03:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3448347027 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b80d6060 con 0x55f3e6dc4420 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3448347027 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f90b80bb580 con 0x55f3e6dc4420 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3448347027 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f90b80c13d0 con 0x55f3e6dc4420 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 10 client.24414 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({fsmap=45}) -- 0x55f3e6dd8520 con 0x55f3e6dc4420 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3448347027 <== mon.0 v2:172.21.15.105:3300/0 4 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f90b803c030 con 0x55f3e6dc4420 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 10 client.24414 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 10 client.24414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 20 client.24414 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f3e6dd9cd0 0x55f3e6ddc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 4 client.24414 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.975+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3e6d35eb0 con 0x55f3e6dd9cd0 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.976+0000 7f90c259c640 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f3e6dd9cd0 0x55f3e6ddc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.976+0000 7f90c259c640 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f3e6dd9cd0 0x55f3e6ddc190 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4409a5a49a8e5b70 server_cookie=120560d6f61c5749 in_seq=0 out_seq=0 2024-03-26T18:03:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.976+0000 7f90a9ffb640 10 client.24414 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.978+0000 7f90a9ffb640 1 -- 172.21.15.105:0/3448347027 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f3e6d35eb0 con 0x55f3e6dd9cd0 2024-03-26T18:03:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.979+0000 7f90a9ffb640 10 client.24414 handle_command_reply: tid=0 2024-03-26T18:03:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.979+0000 7f90cb9dc740 1 client.24414 shutdown 2024-03-26T18:03:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 msgr2=0x55f3e6dce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009780 tx=0x7f90b80bcbe0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 shutdown_connections 2024-03-26T18:03:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f3e6dd9cd0 0x55f3e6ddc190 crc :-1 s=READY pgs=622 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90a9ffb640 0 client.24414 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/3448347027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.980+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 >> 172.21.15.105:0/3448347027 conn(0x55f3e6dbefa0 msgr2=0x55f3e6dbe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.981+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 shutdown_connections 2024-03-26T18:03:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.981+0000 7f90cb9dc740 1 -- 172.21.15.105:0/3448347027 wait complete. 2024-03-26T18:03:16.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:16.981+0000 7f90cb9dc740 20 client.24414 trim_cache size 0 max 16384 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.003+0000 7f90cb9dc740 1 Processor -- start 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.003+0000 7f90cb9dc740 1 -- start start 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b37a0 con 0x55f3e6dc4420 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b52e0 con 0x55f3e6dc3a30 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b3e70 con 0x55f3e6dc9590 2024-03-26T18:03:16.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38496/0 (socket says 172.21.15.105:38496) 2024-03-26T18:03:16.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 -- 172.21.15.105:0/664337125 learned_addr learned my addr 172.21.15.105:0/664337125 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 -- 172.21.15.105:0/664337125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 msgr2=0x55f3e6dd5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 -- 172.21.15.105:0/664337125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 msgr2=0x55f3e6dce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 -- 172.21.15.105:0/664337125 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3e6dbfd10 con 0x55f3e6dc3a30 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.004+0000 7f90c259c640 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f90bc01d850 tx=0x7f90bc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=565f40ebb5c3ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90abfff640 1 -- 172.21.15.105:0/664337125 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc003b70 con 0x55f3e6dc3a30 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90abfff640 1 -- 172.21.15.105:0/664337125 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90bc030070 con 0x55f3e6dc3a30 2024-03-26T18:03:16.990 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90abfff640 1 -- 172.21.15.105:0/664337125 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc004440 con 0x55f3e6dc3a30 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90cb9dc740 1 -- 172.21.15.105:0/664337125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 msgr2=0x55f3e6dcdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f90bc01d850 tx=0x7f90bc014000 comp rx=0 tx=0).stop 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90cb9dc740 1 -- 172.21.15.105:0/664337125 shutdown_connections 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc4420 0x55f3e6dce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/664337125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dcdf10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.005+0000 7f90cb9dc740 1 -- 172.21.15.105:0/664337125 >> 172.21.15.105:0/664337125 conn(0x55f3e6dbefa0 msgr2=0x55f3e6dbf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 -- 172.21.15.105:0/664337125 shutdown_connections 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 -- 172.21.15.105:0/664337125 wait complete. 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 -- start start 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 Processor -- start 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dd26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc4420 0x55f3e6dd2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b4c10 con 0x55f3e6dc9590 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b4540 con 0x55f3e6dc3a30 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b1780 con 0x55f3e6dc4420 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90cb9dc740 10 client.0 _ll_register_callbacks cb 0x7f90b8005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55872/0 (socket says 172.21.15.105:55872) 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90c2d9d640 1 -- 172.21.15.105:0/1248330459 learned_addr learned my addr 172.21.15.105:0/1248330459 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90c1d9b640 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc4420 0x55f3e6dd2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.006+0000 7f90c259c640 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dd26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90c2d9d640 1 -- 172.21.15.105:0/1248330459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc4420 msgr2=0x55f3e6dd2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc4420 0x55f3e6dd2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90c2d9d640 1 -- 172.21.15.105:0/1248330459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 msgr2=0x55f3e6dd26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dd26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90c2d9d640 1 -- 172.21.15.105:0/1248330459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3e6dd4210 con 0x55f3e6dc9590 2024-03-26T18:03:16.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90c2d9d640 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd3c20 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f90b4011fd0 tx=0x7f90b400fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75448703f719d53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1248330459 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b401e040 con 0x55f3e6dc9590 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90cb9dc740 1 -- 172.21.15.105:0/1248330459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90bc030070 con 0x55f3e6dc9590 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1248330459 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f90b400f600 con 0x55f3e6dc9590 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1248330459 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f90b4032050 con 0x55f3e6dc9590 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90cb9dc740 10 client.24420 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.007+0000 7f90cb9dc740 1 -- 172.21.15.105:0/1248330459 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f3e6dd5510 con 0x55f3e6dc9590 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1248330459 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90bc030070 con 0x55f3e6dc9590 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1248330459 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f90b4003a20 con 0x55f3e6dc9590 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90cb9dc740 10 client.24420 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90cb9dc740 10 client.24420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90cb9dc740 20 client.24420 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f3e6dd9cd0 0x55f3e6e422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90cb9dc740 4 client.24420 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:16.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90cb9dc740 1 -- 172.21.15.105:0/1248330459 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3e6d30200 con 0x55f3e6dd9cd0 2024-03-26T18:03:16.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90c259c640 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f3e6dd9cd0 0x55f3e6e422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:16.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90a9ffb640 10 client.24420 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:16.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.008+0000 7f90c259c640 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f3e6dd9cd0 0x55f3e6e422c0 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67040919d7d45fc8 server_cookie=bb4a454da4f1bfda in_seq=0 out_seq=0 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.009+0000 7f90a9ffb640 1 -- 172.21.15.105:0/1248330459 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3e6d30200 con 0x55f3e6dd9cd0 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.009+0000 7f90a9ffb640 10 client.24420 handle_command_reply: tid=0 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.009+0000 7f90cb9dc740 1 client.24420 shutdown 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.009+0000 7f90cb9dc740 1 -- 172.21.15.105:0/1248330459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc9590 msgr2=0x55f3e6dd3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.009+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd3c20 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f90b4011fd0 tx=0x7f90b400fe60 comp rx=0 tx=0).stop 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.009+0000 7f90cb9dc740 1 -- 172.21.15.105:0/1248330459 shutdown_connections 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x55f3e6dd9cd0 0x55f3e6e422c0 crc :-1 s=READY pgs=623 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f3e6dc9590 0x55f3e6dd3c20 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f3e6dc4420 0x55f3e6dd2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90cb9dc740 1 --2- 172.21.15.105:0/1248330459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3e6dc3a30 0x55f3e6dd26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90cb9dc740 1 -- 172.21.15.105:0/1248330459 >> 172.21.15.105:0/1248330459 conn(0x55f3e6dbefa0 msgr2=0x55f3e6dc59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90a9ffb640 0 client.24420 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:16.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90cb9dc740 1 -- 172.21.15.105:0/1248330459 shutdown_connections 2024-03-26T18:03:16.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90cb9dc740 1 -- 172.21.15.105:0/1248330459 wait complete. 2024-03-26T18:03:16.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90cb9dc740 20 client.24420 trim_cache size 0 max 16384 2024-03-26T18:03:16.996 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90c8d10640 1 -- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a2fa0 msgr2=0x7f90ac0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90c8d10640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a2fa0 0x7f90ac0a5460 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f90bc005c40 tx=0x7f90bc009500 comp rx=0 tx=0).stop 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90c8d10640 1 -- 172.21.15.105:0/218218015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c410b230 msgr2=0x7f90c41a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.010+0000 7f90c8d10640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c410b230 0x7f90c41a2a90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f90b802c0b0 tx=0x7f90b80040e0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.011+0000 7f90c8d10640 1 -- 172.21.15.105:0/218218015 shutdown_connections 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.011+0000 7f90c8d10640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90ac0a2fa0 0x7f90ac0a5460 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.011+0000 7f90c8d10640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f90c410b6b0 0x7f90c41a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.011+0000 7f90c8d10640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90c410b230 0x7f90c41a2a90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.011+0000 7f90c8d10640 1 --2- 172.21.15.105:0/218218015 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90c41088c0 0x7f90c41a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.011+0000 7f90c8d10640 1 -- 172.21.15.105:0/218218015 >> 172.21.15.105:0/218218015 conn(0x7f90c406efc0 msgr2=0x7f90c4071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.011+0000 7f90c8d10640 1 -- 172.21.15.105:0/218218015 shutdown_connections 2024-03-26T18:03:16.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:17.011+0000 7f90c8d10640 1 -- 172.21.15.105:0/218218015 wait complete. 2024-03-26T18:03:17.130 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:17 smithi105 ceph-mon[32280]: pgmap v1101: 97 pgs: 97 active+clean; 384 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 3.0 MiB/s wr, 243 op/s 2024-03-26T18:03:17.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:17 smithi144 ceph-mon[36095]: pgmap v1101: 97 pgs: 97 active+clean; 384 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 3.0 MiB/s wr, 243 op/s 2024-03-26T18:03:17.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:17 smithi135 ceph-mon[35636]: pgmap v1101: 97 pgs: 97 active+clean; 384 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 3.0 MiB/s wr, 243 op/s 2024-03-26T18:03:17.563 INFO:tasks.workunit.client.0.smithi105.stdout: 10 cleanup 600 sec 2024-03-26T18:03:18.008 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T18:03:18.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:03:18.008 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T18:03:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.161+0000 7f5bce19d640 1 -- 172.21.15.105:0/2169825219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc81006a0 msgr2=0x7f5bc8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.161+0000 7f5bce19d640 1 --2- 172.21.15.105:0/2169825219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc81006a0 0x7f5bc8100ac0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009900 tx=0x7f5bc402bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.161+0000 7f5bce19d640 1 -- 172.21.15.105:0/2169825219 shutdown_connections 2024-03-26T18:03:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.161+0000 7f5bce19d640 1 --2- 172.21.15.105:0/2169825219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc8102250 0x7f5bc8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.161+0000 7f5bce19d640 1 --2- 172.21.15.105:0/2169825219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc8101890 0x7f5bc8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.161+0000 7f5bce19d640 1 --2- 172.21.15.105:0/2169825219 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc81006a0 0x7f5bc8100ac0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.161+0000 7f5bce19d640 1 -- 172.21.15.105:0/2169825219 >> 172.21.15.105:0/2169825219 conn(0x7f5bc80fbc30 msgr2=0x7f5bc80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.162+0000 7f5bce19d640 1 -- 172.21.15.105:0/2169825219 shutdown_connections 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.162+0000 7f5bce19d640 1 -- 172.21.15.105:0/2169825219 wait complete. 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.162+0000 7f5bce19d640 1 Processor -- start 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.162+0000 7f5bce19d640 1 -- start start 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.162+0000 7f5bce19d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc81006a0 0x7f5bc80695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.162+0000 7f5bce19d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc8101890 0x7f5bc819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bce19d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc8102250 0x7f5bc819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc8101890 0x7f5bc819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcd19b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc81006a0 0x7f5bc80695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bce19d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810c1d0 con 0x7f5bc8101890 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc8101890 0x7f5bc819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55892/0 (socket says 172.21.15.105:55892) 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bce19d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810c050 con 0x7f5bc8102250 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc810c350 con 0x7f5bc81006a0 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 -- 172.21.15.105:0/902300221 learned_addr learned my addr 172.21.15.105:0/902300221 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcd99c640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc8102250 0x7f5bc819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 -- 172.21.15.105:0/902300221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc81006a0 msgr2=0x7f5bc80695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc81006a0 0x7f5bc80695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 -- 172.21.15.105:0/902300221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc8102250 msgr2=0x7f5bc819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc8102250 0x7f5bc819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 -- 172.21.15.105:0/902300221 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc819f620 con 0x7f5bc8101890 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcd19b640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc81006a0 0x7f5bc80695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc8101890 0x7f5bc819c6f0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009f20 tx=0x7f5bb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bbe7fc640 1 -- 172.21.15.105:0/902300221 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8020070 con 0x7f5bc8101890 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bbe7fc640 1 -- 172.21.15.105:0/902300221 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bb800cd80 con 0x7f5bc8101890 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.163+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc8196520 con 0x7f5bc8101890 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.164+0000 7f5bbe7fc640 1 -- 172.21.15.105:0/902300221 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb801bae0 con 0x7f5bc8101890 2024-03-26T18:03:18.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.164+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc8196ac0 con 0x7f5bc8101890 2024-03-26T18:03:18.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bbe7fc640 1 -- 172.21.15.105:0/902300221 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f5bb8023070 con 0x7f5bc8101890 2024-03-26T18:03:18.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bd1fef740 1 Processor -- start 2024-03-26T18:03:18.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bd1fef740 1 -- start start 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3db2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3db5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560da3d12630 con 0x560da3db2e90 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560da3cbae10 con 0x560da3dad330 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560da3d0e5e0 con 0x560da3dadd20 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bcc99a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3db5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bcd99c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bcd19b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3db2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bcc99a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3db5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55896/0 (socket says 172.21.15.105:55896) 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bcd99c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38526/0 (socket says 172.21.15.105:38526) 2024-03-26T18:03:18.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bcc99a640 1 -- 172.21.15.105:0/2868455828 learned_addr learned my addr 172.21.15.105:0/2868455828 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.167+0000 7f5bbe7fc640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5bb40a2fa0 0x7f5bb40a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bcd99c640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5bb40a2fa0 0x7f5bb40a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bbe7fc640 1 -- 172.21.15.105:0/902300221 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f5bb80d3550 con 0x7f5bc8101890 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bcc99a640 1 -- 172.21.15.105:0/2868455828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 msgr2=0x560da3db2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/2868455828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3db2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bcc99a640 1 -- 172.21.15.105:0/2868455828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 msgr2=0x560da3dad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/2868455828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bcc99a640 1 -- 172.21.15.105:0/2868455828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da3cd1740 con 0x560da3db2e90 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bcd19b640 1 --2- 172.21.15.105:0/2868455828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3db2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/2868455828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3db5320 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f5bb809c150 tx=0x7f5bb801b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22be00bbc54ccfa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bb37fe640 1 -- 172.21.15.105:0/2868455828 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb809f830 con 0x560da3db2e90 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bb37fe640 1 -- 172.21.15.105:0/2868455828 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bb8013da0 con 0x560da3db2e90 2024-03-26T18:03:18.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.168+0000 7f5bb37fe640 1 -- 172.21.15.105:0/2868455828 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb809f650 con 0x560da3db2e90 2024-03-26T18:03:18.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2868455828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 msgr2=0x560da3db5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2868455828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3db5320 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f5bb809c150 tx=0x7f5bb801b440 comp rx=0 tx=0).stop 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bcd99c640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5bb40a2fa0 0x7f5bb40a5460 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x560da3dae580 tx=0x7f5bc0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2868455828 shutdown_connections 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2868455828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3db5320 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2868455828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3db2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2868455828 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2868455828 >> 172.21.15.105:0/2868455828 conn(0x560da3da88a0 msgr2=0x560da3da8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.169+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2868455828 shutdown_connections 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.170+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2868455828 wait complete. 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.170+0000 7f5bd1fef740 1 -- start start 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.170+0000 7f5bd1fef740 1 Processor -- start 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3db77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3dadd20 0x560da3db7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3db2e90 0x560da3dbed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x560da3db5d80 con 0x560da3dadd20 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560da3cce1a0 con 0x560da3db2e90 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560da3db5f00 con 0x560da3dad330 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3dadd20 0x560da3db7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3dadd20 0x560da3db7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55900/0 (socket says 172.21.15.105:55900) 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcd19b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3db77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 -- 172.21.15.105:0/2256784011 learned_addr learned my addr 172.21.15.105:0/2256784011 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcd99c640 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3db2e90 0x560da3dbed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bd1fef740 10 client.0 _ll_register_callbacks cb 0x560da3a25c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 -- 172.21.15.105:0/2256784011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 msgr2=0x560da3db77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3db77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 -- 172.21.15.105:0/2256784011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3db2e90 msgr2=0x560da3dbed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3db2e90 0x560da3dbed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 -- 172.21.15.105:0/2256784011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da3aee7e0 con 0x560da3dadd20 2024-03-26T18:03:18.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3dadd20 0x560da3db7cf0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f5bb809fc80 tx=0x7f5bb809fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bfaea3c7c93321 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.171+0000 7f5bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/2256784011 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb809f650 con 0x560da3dadd20 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2256784011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560da3d1fb70 con 0x560da3dadd20 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/2256784011 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f5bb80a83d0 con 0x560da3dadd20 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/2256784011 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5bb809f830 con 0x560da3dadd20 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 10 client.24432 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2256784011 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560da3b4ef70 con 0x560da3dadd20 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/2256784011 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560da3d1fb70 con 0x560da3dadd20 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/2256784011 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f5bb80b6070 con 0x560da3dadd20 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 10 client.24432 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 10 client.24432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 20 client.24432 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560da3dc31e0 0x560da3dc55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 4 client.24432 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:18.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.172+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2256784011 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560da3d19ec0 con 0x560da3dc31e0 2024-03-26T18:03:18.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.173+0000 7f5bcd19b640 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560da3dc31e0 0x560da3dc55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.173+0000 7f5bcd19b640 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560da3dc31e0 0x560da3dc55d0 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eabb4557aeaa4cc server_cookie=43aa4f96b5f65102 in_seq=0 out_seq=0 2024-03-26T18:03:18.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.173+0000 7f5bb0ff9640 10 client.24432 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/2256784011 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560da3d19ec0 con 0x560da3dc31e0 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bb0ff9640 10 client.24432 handle_command_reply: tid=0 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 client.24432 shutdown 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2256784011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3dadd20 msgr2=0x560da3db7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3dadd20 0x560da3db7cf0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f5bb809fc80 tx=0x7f5bb809fd90 comp rx=0 tx=0).stop 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2256784011 shutdown_connections 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560da3dc31e0 0x560da3dc55d0 crc :-1 s=READY pgs=624 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3db2e90 0x560da3dbed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bb0ff9640 0 client.24432 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:18.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3dadd20 0x560da3db7cf0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2256784011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3db77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2256784011 >> 172.21.15.105:0/2256784011 conn(0x560da3da88a0 msgr2=0x560da3da7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.175+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2256784011 shutdown_connections 2024-03-26T18:03:18.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.176+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2256784011 wait complete. 2024-03-26T18:03:18.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.176+0000 7f5bd1fef740 20 client.24432 trim_cache size 0 max 16384 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.198+0000 7f5bd1fef740 1 Processor -- start 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.198+0000 7f5bd1fef740 1 -- start start 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.198+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3da7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.198+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dadd20 0x560da3dbecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.198+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3da7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.198+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a69d0 con 0x560da3db2e90 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.198+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a8510 con 0x560da3dadd20 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.198+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a70a0 con 0x560da3dad330 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3da7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd19b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3da7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3da7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55902/0 (socket says 172.21.15.105:55902) 2024-03-26T18:03:18.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 -- 172.21.15.105:0/2959496187 learned_addr learned my addr 172.21.15.105:0/2959496187 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd19b640 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3da7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55416/0 (socket says 172.21.15.105:55416) 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dadd20 0x560da3dbecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 -- 172.21.15.105:0/2959496187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 msgr2=0x560da3da7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3da7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 -- 172.21.15.105:0/2959496187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dadd20 msgr2=0x560da3dbecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dadd20 0x560da3dbecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 -- 172.21.15.105:0/2959496187 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da39e6170 con 0x560da3db2e90 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bcd99c640 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3da7750 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00039a0 tx=0x7f5bc0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfc8c11d2b01ec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bb2ffd640 1 -- 172.21.15.105:0/2959496187 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0004250 con 0x560da3db2e90 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bb2ffd640 1 -- 172.21.15.105:0/2959496187 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bc0026030 con 0x560da3db2e90 2024-03-26T18:03:18.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.199+0000 7f5bb2ffd640 1 -- 172.21.15.105:0/2959496187 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0022cd0 con 0x560da3db2e90 2024-03-26T18:03:18.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.200+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2959496187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 msgr2=0x560da3da7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.200+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3da7750 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00039a0 tx=0x7f5bc0005bd0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.200+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2959496187 shutdown_connections 2024-03-26T18:03:18.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.200+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3da7750 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.200+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dadd20 0x560da3dbecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.200+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/2959496187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dad330 0x560da3da7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.200+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2959496187 >> 172.21.15.105:0/2959496187 conn(0x560da3da88a0 msgr2=0x560da3daf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2959496187 shutdown_connections 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 -- 172.21.15.105:0/2959496187 wait complete. 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 -- start start 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 Processor -- start 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dbc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3dbc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3dbd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a7e40 con 0x560da3db2e90 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a7770 con 0x560da3dad330 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81a49b0 con 0x560da3dadd20 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.201+0000 7f5bd1fef740 10 client.0 _ll_register_callbacks cb 0x7f5bc0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3dbc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcd19b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dbc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3dbc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55428/0 (socket says 172.21.15.105:55428) 2024-03-26T18:03:18.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 -- 172.21.15.105:0/3487234385 learned_addr learned my addr 172.21.15.105:0/3487234385 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcd99c640 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3dbd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 -- 172.21.15.105:0/3487234385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 msgr2=0x560da3dbc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dbc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 -- 172.21.15.105:0/3487234385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 msgr2=0x560da3dbd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3dbd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 -- 172.21.15.105:0/3487234385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da3dbdef0 con 0x560da3dadd20 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bcc99a640 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3dbc940 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5bb809c670 tx=0x7f5bb8009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2f9efe9a207e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/3487234385 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb809f650 con 0x560da3dadd20 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bd1fef740 1 -- 172.21.15.105:0/3487234385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc0026030 con 0x560da3dadd20 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.202+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/3487234385 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f5bb809f140 con 0x560da3dadd20 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/3487234385 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5bb809ba40 con 0x560da3dadd20 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bd1fef740 10 client.34099 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bd1fef740 1 -- 172.21.15.105:0/3487234385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560da3e292f0 con 0x560da3dadd20 2024-03-26T18:03:18.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/3487234385 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bc0026030 con 0x560da3dadd20 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/3487234385 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f5bb80b6070 con 0x560da3dadd20 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bd1fef740 10 client.34099 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bd1fef740 10 client.34099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bd1fef740 20 client.34099 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560da3e29b20 0x560da3e2bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bd1fef740 4 client.34099 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bd1fef740 1 -- 172.21.15.105:0/3487234385 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560da3db2c00 con 0x560da3e29b20 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bcd19b640 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560da3e29b20 0x560da3e2bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bb0ff9640 10 client.34099 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:18.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.203+0000 7f5bcd19b640 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560da3e29b20 0x560da3e2bf40 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ba0269ad0105e8 server_cookie=91fc42cfc91e216a in_seq=0 out_seq=0 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.206+0000 7f5bb0ff9640 1 -- 172.21.15.105:0/3487234385 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560da3db2c00 con 0x560da3e29b20 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.206+0000 7f5bb0ff9640 10 client.34099 handle_command_reply: tid=0 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 client.34099 shutdown 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 -- 172.21.15.105:0/3487234385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 msgr2=0x560da3dbc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3dbc940 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5bb809c670 tx=0x7f5bb8009700 comp rx=0 tx=0).stop 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 -- 172.21.15.105:0/3487234385 shutdown_connections 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x560da3e29b20 0x560da3e2bf40 crc :-1 s=READY pgs=625 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x560da3db2e90 0x560da3dbd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560da3dadd20 0x560da3dbc940 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 --2- 172.21.15.105:0/3487234385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560da3dad330 0x560da3dbc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 -- 172.21.15.105:0/3487234385 >> 172.21.15.105:0/3487234385 conn(0x560da3da88a0 msgr2=0x560da3dabb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bb0ff9640 0 client.34099 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:18.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 -- 172.21.15.105:0/3487234385 shutdown_connections 2024-03-26T18:03:18.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 1 -- 172.21.15.105:0/3487234385 wait complete. 2024-03-26T18:03:18.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.207+0000 7f5bd1fef740 20 client.34099 trim_cache size 0 max 16384 2024-03-26T18:03:18.193 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:03:18.193 INFO:teuthology.orchestra.run.smithi105.stdout: "return_code": 0, 2024-03-26T18:03:18.193 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_tag": "f4a53e9e-2b67-4810-8649-fa2ab62cae3a", 2024-03-26T18:03:18.193 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": "asynchronous" 2024-03-26T18:03:18.193 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:03:18.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5bb40a2fa0 msgr2=0x7f5bb40a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5bb40a2fa0 0x7f5bb40a5460 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x560da3dae580 tx=0x7f5bc0004ab0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc8101890 msgr2=0x7f5bc819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc8101890 0x7f5bc819c6f0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009f20 tx=0x7f5bb800ee80 comp rx=0 tx=0).stop 2024-03-26T18:03:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 shutdown_connections 2024-03-26T18:03:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f5bb40a2fa0 0x7f5bb40a5460 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5bc8102250 0x7f5bc819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5bc8101890 0x7f5bc819c6f0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 --2- 172.21.15.105:0/902300221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bc81006a0 0x7f5bc80695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 >> 172.21.15.105:0/902300221 conn(0x7f5bc80fbc30 msgr2=0x7f5bc8104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 shutdown_connections 2024-03-26T18:03:18.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.208+0000 7f5bce19d640 1 -- 172.21.15.105:0/902300221 wait complete. 2024-03-26T18:03:18.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:03:18.201 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:03:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.343+0000 7fa9b54c6640 1 -- 172.21.15.105:0/240958819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0102370 msgr2=0x7fa9b0108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.343+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/240958819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0102370 0x7fa9b0108c00 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8009ca0 tx=0x7fa9a802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 -- 172.21.15.105:0/240958819 shutdown_connections 2024-03-26T18:03:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/240958819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b0102370 0x7fa9b0108c00 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/240958819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b01019b0 0x7fa9b0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/240958819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b01007c0 0x7fa9b0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 -- 172.21.15.105:0/240958819 >> 172.21.15.105:0/240958819 conn(0x7fa9b00fbd50 msgr2=0x7fa9b00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 -- 172.21.15.105:0/240958819 shutdown_connections 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 -- 172.21.15.105:0/240958819 wait complete. 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 Processor -- start 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.344+0000 7fa9b54c6640 1 -- start start 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9b54c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b01007c0 0x7fa9b0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9b54c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b01019b0 0x7fa9b01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9b54c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b0102370 0x7fa9b019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9b54c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b010c2f0 con 0x7fa9b01007c0 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9b54c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b010c170 con 0x7fa9b0102370 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9b54c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b010c470 con 0x7fa9b01019b0 2024-03-26T18:03:18.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b01007c0 0x7fa9b0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b01007c0 0x7fa9b0197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55926/0 (socket says 172.21.15.105:55926) 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 -- 172.21.15.105:0/1047446287 learned_addr learned my addr 172.21.15.105:0/1047446287 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9ae7fc640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b01019b0 0x7fa9b01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b0102370 0x7fa9b019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 -- 172.21.15.105:0/1047446287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b01019b0 msgr2=0x7fa9b01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b01019b0 0x7fa9b01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 -- 172.21.15.105:0/1047446287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b0102370 msgr2=0x7fa9b019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b0102370 0x7fa9b019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 -- 172.21.15.105:0/1047446287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b019f7a0 con 0x7fa9b01007c0 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b0102370 0x7fa9b019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.345+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b01007c0 0x7fa9b0197b80 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400e9c0 tx=0x7fa9a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.346+0000 7fa997fff640 1 -- 172.21.15.105:0/1047446287 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400cd80 con 0x7fa9b01007c0 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.346+0000 7fa9b54c6640 1 -- 172.21.15.105:0/1047446287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b019fa90 con 0x7fa9b01007c0 2024-03-26T18:03:18.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.346+0000 7fa997fff640 1 -- 172.21.15.105:0/1047446287 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a40035f0 con 0x7fa9b01007c0 2024-03-26T18:03:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.346+0000 7fa997fff640 1 -- 172.21.15.105:0/1047446287 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4010690 con 0x7fa9b01007c0 2024-03-26T18:03:18.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.346+0000 7fa9b54c6640 1 -- 172.21.15.105:0/1047446287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01a0030 con 0x7fa9b01007c0 2024-03-26T18:03:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa9b8191740 1 Processor -- start 2024-03-26T18:03:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa9b8191740 1 -- start start 2024-03-26T18:03:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa997fff640 1 -- 172.21.15.105:0/1047446287 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa9a40037a0 con 0x7fa9b01007c0 2024-03-26T18:03:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f4ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4f2e0 0x5610f3f53de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa9b8191740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5610f3eb3980 con 0x5610f3f4f2e0 2024-03-26T18:03:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa9b8191740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610f3e5bbe0 con 0x5610f3f4e8f0 2024-03-26T18:03:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.349+0000 7fa9b8191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610f3eaf930 con 0x5610f3f54450 2024-03-26T18:03:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9af7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f4ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9ae7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9af7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f4ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:38580/0 (socket says 172.21.15.105:38580) 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9af7fe640 1 -- 172.21.15.105:0/273037115 learned_addr learned my addr 172.21.15.105:0/273037115 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9ae7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:55460/0 (socket says 172.21.15.105:55460) 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/273037115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4f2e0 0x5610f3f53de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa997fff640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9980a2fa0 0x7fa9980a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9ae7fc640 1 -- 172.21.15.105:0/273037115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 msgr2=0x5610f3f4ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9ae7fc640 1 --2- 172.21.15.105:0/273037115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f4ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9ae7fc640 1 -- 172.21.15.105:0/273037115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4f2e0 msgr2=0x5610f3f53de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9ae7fc640 1 --2- 172.21.15.105:0/273037115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4f2e0 0x5610f3f53de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9ae7fc640 1 -- 172.21.15.105:0/273037115 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f3c917d0 con 0x5610f3f54450 2024-03-26T18:03:18.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa997fff640 1 -- 172.21.15.105:0/1047446287 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fa9a40cf800 con 0x7fa9b01007c0 2024-03-26T18:03:18.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.350+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9980a2fa0 0x7fa9980a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.351+0000 7fa9ae7fc640 1 --2- 172.21.15.105:0/273037115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f568e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009bd0 tx=0x7fa9a0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=196eb3e5000e8aef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.351+0000 7fa9957fa640 1 -- 172.21.15.105:0/273037115 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0004230 con 0x5610f3f54450 2024-03-26T18:03:18.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.351+0000 7fa9957fa640 1 -- 172.21.15.105:0/273037115 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a0009e40 con 0x5610f3f54450 2024-03-26T18:03:18.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.351+0000 7fa9957fa640 1 -- 172.21.15.105:0/273037115 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0004ca0 con 0x5610f3f54450 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9b8191740 1 -- 172.21.15.105:0/273037115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 msgr2=0x5610f3f568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9b8191740 1 --2- 172.21.15.105:0/273037115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f568e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009bd0 tx=0x7fa9a0011810 comp rx=0 tx=0).stop 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9980a2fa0 0x7fa9980a5460 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x5610f3f4fb40 tx=0x7fa9a802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9b8191740 1 -- 172.21.15.105:0/273037115 shutdown_connections 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9b8191740 1 --2- 172.21.15.105:0/273037115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f568e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9b8191740 1 --2- 172.21.15.105:0/273037115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4f2e0 0x5610f3f53de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9b8191740 1 --2- 172.21.15.105:0/273037115 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f4ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9b8191740 1 -- 172.21.15.105:0/273037115 >> 172.21.15.105:0/273037115 conn(0x5610f3f49e60 msgr2=0x5610f3f4a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.352+0000 7fa9b8191740 1 -- 172.21.15.105:0/273037115 shutdown_connections 2024-03-26T18:03:18.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 -- 172.21.15.105:0/273037115 wait complete. 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 -- start start 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 Processor -- start 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f58b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4f2e0 0x5610f3f59060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5610f3f57450 con 0x5610f3f54450 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610f3e6f160 con 0x5610f3f4e8f0 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5610f3f575d0 con 0x5610f3f4f2e0 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9b8191740 10 client.0 _ll_register_callbacks cb 0x5610f3eab720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55944/0 (socket says 172.21.15.105:55944) 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9ae7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4f2e0 0x5610f3f59060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f58b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.353+0000 7fa9af7fe640 1 -- 172.21.15.105:0/3162708112 learned_addr learned my addr 172.21.15.105:0/3162708112 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9af7fe640 1 -- 172.21.15.105:0/3162708112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4f2e0 msgr2=0x5610f3f59060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4f2e0 0x5610f3f59060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9af7fe640 1 -- 172.21.15.105:0/3162708112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 msgr2=0x5610f3f58b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f58b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9af7fe640 1 -- 172.21.15.105:0/3162708112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f3bc73f0 con 0x5610f3f54450 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f58b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f600a0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fa9a802c630 tx=0x7fa9a8043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344601426ee21da4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa992ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa992ff5640 1 -- 172.21.15.105:0/3162708112 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8039040 con 0x5610f3f54450 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa992ff5640 1 -- 172.21.15.105:0/3162708112 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fa9a8003d80 con 0x5610f3f54450 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9b8191740 1 -- 172.21.15.105:0/3162708112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5610f3ec0ec0 con 0x5610f3f54450 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa992ff5640 1 -- 172.21.15.105:0/3162708112 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa9a8004050 con 0x5610f3f54450 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9b8191740 10 client.24450 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.354+0000 7fa9b8191740 1 -- 172.21.15.105:0/3162708112 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5610f3b870f0 con 0x5610f3f54450 2024-03-26T18:03:18.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa992ff5640 1 -- 172.21.15.105:0/3162708112 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5610f3ec0ec0 con 0x5610f3f54450 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa992ff5640 1 -- 172.21.15.105:0/3162708112 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fa9a8034a40 con 0x5610f3f54450 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa9b8191740 10 client.24450 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa9b8191740 10 client.24450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa9b8191740 20 client.24450 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f3f64900 0x5610f3f66dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa9b8191740 4 client.24450 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa9b8191740 1 -- 172.21.15.105:0/3162708112 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610f3ebb210 con 0x5610f3f64900 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f3f64900 0x5610f3f66dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.355+0000 7fa992ff5640 10 client.24450 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:18.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.356+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f3f64900 0x5610f3f66dc0 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e86afb7fedab69f server_cookie=e3e95d53cc3a708d in_seq=0 out_seq=0 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa992ff5640 1 -- 172.21.15.105:0/3162708112 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5610f3ebb210 con 0x5610f3f64900 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa992ff5640 10 client.24450 handle_command_reply: tid=0 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 client.24450 shutdown 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 -- 172.21.15.105:0/3162708112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 msgr2=0x5610f3f600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f600a0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fa9a802c630 tx=0x7fa9a8043040 comp rx=0 tx=0).stop 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 -- 172.21.15.105:0/3162708112 shutdown_connections 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f3f64900 0x5610f3f66dc0 crc :-1 s=READY pgs=626 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f600a0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4f2e0 0x5610f3f59060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa992ff5640 0 client.24450 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:18.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3162708112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4e8f0 0x5610f3f58b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 -- 172.21.15.105:0/3162708112 >> 172.21.15.105:0/3162708112 conn(0x5610f3f49e60 msgr2=0x5610f3f49080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 -- 172.21.15.105:0/3162708112 shutdown_connections 2024-03-26T18:03:18.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 1 -- 172.21.15.105:0/3162708112 wait complete. 2024-03-26T18:03:18.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.361+0000 7fa9b8191740 20 client.24450 trim_cache size 0 max 16384 2024-03-26T18:03:18.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9b8191740 1 Processor -- start 2024-03-26T18:03:18.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9b8191740 1 -- start start 2024-03-26T18:03:18.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4e8f0 0x5610f3f50460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 0x5610f3f60060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9b8191740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01a6b90 con 0x5610f3f54450 2024-03-26T18:03:18.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9b8191740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01a86d0 con 0x5610f3f4f2e0 2024-03-26T18:03:18.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9b8191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01a7260 con 0x5610f3f4e8f0 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9af7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55952/0 (socket says 172.21.15.105:55952) 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4e8f0 0x5610f3f50460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9af7fe640 1 -- 172.21.15.105:0/3098761573 learned_addr learned my addr 172.21.15.105:0/3098761573 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9ae7fc640 1 --2- 172.21.15.105:0/3098761573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 0x5610f3f60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9af7fe640 1 -- 172.21.15.105:0/3098761573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4e8f0 msgr2=0x5610f3f50460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/3098761573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4e8f0 0x5610f3f50460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9af7fe640 1 -- 172.21.15.105:0/3098761573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 msgr2=0x5610f3f60060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/3098761573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 0x5610f3f60060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.382+0000 7fa9af7fe640 1 -- 172.21.15.105:0/3098761573 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f3f44e20 con 0x5610f3f54450 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/3098761573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f487f0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8034de0 tx=0x7fa9a8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cb949e77e8cba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa994ff9640 1 -- 172.21.15.105:0/3098761573 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8039040 con 0x5610f3f54450 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa994ff9640 1 -- 172.21.15.105:0/3098761573 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a8003d80 con 0x5610f3f54450 2024-03-26T18:03:18.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa994ff9640 1 -- 172.21.15.105:0/3098761573 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8034b70 con 0x5610f3f54450 2024-03-26T18:03:18.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa9b8191740 1 -- 172.21.15.105:0/3098761573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 msgr2=0x5610f3f487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3098761573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f487f0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8034de0 tx=0x7fa9a8009620 comp rx=0 tx=0).stop 2024-03-26T18:03:18.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa9b8191740 1 -- 172.21.15.105:0/3098761573 shutdown_connections 2024-03-26T18:03:18.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3098761573 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f54450 0x5610f3f487f0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3098761573 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 0x5610f3f60060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.383+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3098761573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f4e8f0 0x5610f3f50460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 -- 172.21.15.105:0/3098761573 >> 172.21.15.105:0/3098761573 conn(0x5610f3f49e60 msgr2=0x5610f3f4a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 -- 172.21.15.105:0/3098761573 shutdown_connections 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 -- 172.21.15.105:0/3098761573 wait complete. 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 -- start start 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 Processor -- start 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4e8f0 0x5610f3f5d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 0x5610f3f5dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f5ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01a8000 con 0x5610f3f4e8f0 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01a7930 con 0x5610f3f4f2e0 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9b8191740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01a4b70 con 0x5610f3f54450 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.384+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4e8f0 0x5610f3f5d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9aeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4e8f0 0x5610f3f5d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55958/0 (socket says 172.21.15.105:55958) 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9aeffd640 1 -- 172.21.15.105:0/3184752023 learned_addr learned my addr 172.21.15.105:0/3184752023 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9ae7fc640 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 0x5610f3f5dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9af7fe640 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f5ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9b8191740 10 client.0 _ll_register_callbacks cb 0x7fa9a0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9aeffd640 1 -- 172.21.15.105:0/3184752023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 msgr2=0x5610f3f5ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f5ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9aeffd640 1 -- 172.21.15.105:0/3184752023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 msgr2=0x5610f3f5dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 0x5610f3f5dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9aeffd640 1 -- 172.21.15.105:0/3184752023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f3f5f340 con 0x5610f3f4e8f0 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9aeffd640 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4e8f0 0x5610f3f5d800 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4018000 tx=0x7fa9a4003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34973d4218113452 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa992ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa992ff5640 1 -- 172.21.15.105:0/3184752023 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a409b7f0 con 0x5610f3f4e8f0 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa992ff5640 1 -- 172.21.15.105:0/3184752023 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fa9a40a4410 con 0x5610f3f4e8f0 2024-03-26T18:03:18.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9b8191740 1 -- 172.21.15.105:0/3184752023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9a8003d80 con 0x5610f3f4e8f0 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa992ff5640 1 -- 172.21.15.105:0/3184752023 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa9a40bc0b0 con 0x5610f3f4e8f0 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9b8191740 10 client.24456 fetch_fsmap learned FSMap version 45 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.385+0000 7fa9b8191740 1 -- 172.21.15.105:0/3184752023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5610f3fcae30 con 0x5610f3f4e8f0 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa992ff5640 1 -- 172.21.15.105:0/3184752023 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9a8003d80 con 0x5610f3f4e8f0 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa992ff5640 1 -- 172.21.15.105:0/3184752023 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fa9a40cfbb0 con 0x5610f3f4e8f0 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa9b8191740 10 client.24456 fetch_fsmap finished waiting for FSMap version 45 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa9b8191740 10 client.24456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa9b8191740 20 client.24456 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f3fcb830 0x5610f3fcdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa9b8191740 4 client.24456 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa9b8191740 1 -- 172.21.15.105:0/3184752023 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610f3e38540 con 0x5610f3fcb830 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa9ae7fc640 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f3fcb830 0x5610f3fcdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa992ff5640 10 client.24456 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:18.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.386+0000 7fa9ae7fc640 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f3fcb830 0x5610f3fcdc50 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3533bcb4dabe4fe server_cookie=825cfdd844c939b3 in_seq=0 out_seq=0 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa992ff5640 1 -- 172.21.15.105:0/3184752023 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5610f3e38540 con 0x5610f3fcb830 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa992ff5640 10 client.24456 handle_command_reply: tid=0 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 client.24456 shutdown 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 -- 172.21.15.105:0/3184752023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4e8f0 msgr2=0x5610f3f5d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4e8f0 0x5610f3f5d800 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4018000 tx=0x7fa9a4003e30 comp rx=0 tx=0).stop 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 -- 172.21.15.105:0/3184752023 shutdown_connections 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x5610f3fcb830 0x5610f3fcdc50 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5610f3f54450 0x5610f3f5ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610f3f4f2e0 0x5610f3f5dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 --2- 172.21.15.105:0/3184752023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5610f3f4e8f0 0x5610f3f5d800 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 -- 172.21.15.105:0/3184752023 >> 172.21.15.105:0/3184752023 conn(0x5610f3f49e60 msgr2=0x5610f3f53a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa992ff5640 0 client.24456 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 -- 172.21.15.105:0/3184752023 shutdown_connections 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 1 -- 172.21.15.105:0/3184752023 wait complete. 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.389+0000 7fa9b8191740 20 client.24456 trim_cache size 0 max 16384 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "scrub active (183 inodes in the stack)", 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": { 2024-03-26T18:03:18.375 INFO:teuthology.orchestra.run.smithi105.stdout: "f4a53e9e-2b67-4810-8649-fa2ab62cae3a": { 2024-03-26T18:03:18.376 INFO:teuthology.orchestra.run.smithi105.stdout: "path": "/", 2024-03-26T18:03:18.376 INFO:teuthology.orchestra.run.smithi105.stdout: "tag": "f4a53e9e-2b67-4810-8649-fa2ab62cae3a", 2024-03-26T18:03:18.376 INFO:teuthology.orchestra.run.smithi105.stdout: "options": "recursive,force" 2024-03-26T18:03:18.376 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:18.376 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:18.376 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 -- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9980a2fa0 msgr2=0x7fa9980a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9980a2fa0 0x7fa9980a5460 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x5610f3f4fb40 tx=0x7fa9a802e040 comp rx=0 tx=0).stop 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 -- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b01007c0 msgr2=0x7fa9b0197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b01007c0 0x7fa9b0197b80 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400e9c0 tx=0x7fa9a400ee80 comp rx=0 tx=0).stop 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 -- 172.21.15.105:0/1047446287 shutdown_connections 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa9980a2fa0 0x7fa9980a5460 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa9b0102370 0x7fa9b019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9b01019b0 0x7fa9b01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 --2- 172.21.15.105:0/1047446287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9b01007c0 0x7fa9b0197b80 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 -- 172.21.15.105:0/1047446287 >> 172.21.15.105:0/1047446287 conn(0x7fa9b00fbd50 msgr2=0x7fa9b00fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 -- 172.21.15.105:0/1047446287 shutdown_connections 2024-03-26T18:03:18.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:18.390+0000 7fa9b54c6640 1 -- 172.21.15.105:0/1047446287 wait complete. 2024-03-26T18:03:18.384 INFO:tasks.cephfs.filesystem:scrub status for tag:f4a53e9e-2b67-4810-8649-fa2ab62cae3a - {'path': '/', 'tag': 'f4a53e9e-2b67-4810-8649-fa2ab62cae3a', 'options': 'recursive,force'} 2024-03-26T18:03:18.563 INFO:tasks.workunit.client.0.smithi105.stdout: 9 cleanup 601 sec 2024-03-26T18:03:18.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:18 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:03:18.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:18 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:18.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:18 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:03:18.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:18 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:18.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:18 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:03:18.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:18 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.563 INFO:tasks.workunit.client.0.smithi105.stdout: 8 cleanup 602 sec 2024-03-26T18:03:19.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub queued for path: / 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: active paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:19 smithi144 ceph-mon[36095]: pgmap v1102: 97 pgs: 97 active+clean; 382 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-26T18:03:19.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub queued for path: / 2024-03-26T18:03:19.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:03:19.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:03:19.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:03:19.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:03:19.630 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: active paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.631 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:19 smithi105 ceph-mon[32280]: pgmap v1102: 97 pgs: 97 active+clean; 382 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-26T18:03:19.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub queued for path: / 2024-03-26T18:03:19.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: active paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: scrub summary: idle+waiting paths [/] 2024-03-26T18:03:19.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:19 smithi135 ceph-mon[35636]: pgmap v1102: 97 pgs: 97 active+clean; 382 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 2.9 MiB/s wr, 247 op/s 2024-03-26T18:03:20.048 INFO:tasks.workunit.client.0.smithi105.stdout: 0 cleanup 603 sec 2024-03-26T18:03:20.048 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-03-26T18:03:20.048 INFO:tasks.workunit.client.0.smithi105.stdout: Operation Count AvgLat MaxLat 2024-03-26T18:03:20.048 INFO:tasks.workunit.client.0.smithi105.stdout: ---------------------------------------- 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: NTCreateX 134881 10.286 161.531 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Close 99167 0.111 83.902 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Rename 5711 25.024 171.661 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Unlink 27257 8.108 127.754 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Qpathinfo 122427 6.237 170.568 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Qfileinfo 21353 0.192 4.350 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Qfsinfo 22408 0.554 44.631 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Sfileinfo 10996 1.545 82.497 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Find 47294 70.251 267.587 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: WriteX 66723 0.407 86.818 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: ReadX 211551 0.005 1.085 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: LockX 440 3.262 13.574 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: UnlockX 440 3.089 23.813 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: Flush 9451 8.991 163.528 2024-03-26T18:03:20.049 INFO:tasks.workunit.client.0.smithi105.stdout: 2024-03-26T18:03:20.050 INFO:tasks.workunit.client.0.smithi105.stdout:Throughput 7.00313 MB/sec 10 clients 10 procs max_latency=267.593 ms 2024-03-26T18:03:20.050 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T18:03:20.050 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T18:03:20.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:20 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:20.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:20 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:20.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:20 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:21.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:21 smithi144 ceph-mon[36095]: pgmap v1103: 97 pgs: 97 active+clean; 363 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T18:03:21.614 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:21 smithi105 ceph-mon[32280]: pgmap v1103: 97 pgs: 97 active+clean; 363 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T18:03:21.615 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-03-26T18:03:21.615 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-26T18:03:21.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:21 smithi135 ceph-mon[35636]: pgmap v1103: 97 pgs: 97 active+clean; 363 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 745 KiB/s rd, 2.7 MiB/s wr, 269 op/s 2024-03-26T18:03:22.182 DEBUG:teuthology.parallel:result is None 2024-03-26T18:03:22.183 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T18:03:22.220 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T18:03:22.221 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-26T18:03:22.231 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T18:03:22.269 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-26T18:03:22.278 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-26T18:03:24.836 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:24 smithi144 ceph-mon[36095]: pgmap v1104: 97 pgs: 97 active+clean; 306 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.6 MiB/s wr, 288 op/s 2024-03-26T18:03:24.836 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:24 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:24.880 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:24 smithi105 ceph-mon[32280]: pgmap v1104: 97 pgs: 97 active+clean; 306 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.6 MiB/s wr, 288 op/s 2024-03-26T18:03:24.880 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:24 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:24.903 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:24 smithi135 ceph-mon[35636]: pgmap v1104: 97 pgs: 97 active+clean; 306 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 2.6 MiB/s wr, 288 op/s 2024-03-26T18:03:24.903 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:24 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:26.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:26 smithi144 ceph-mon[36095]: pgmap v1105: 97 pgs: 97 active+clean; 194 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-26T18:03:26.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:26 smithi144 ceph-mon[36095]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:26.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:26 smithi105 ceph-mon[32280]: pgmap v1105: 97 pgs: 97 active+clean; 194 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-26T18:03:26.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:26 smithi105 ceph-mon[32280]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:26.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:26 smithi135 ceph-mon[35636]: pgmap v1105: 97 pgs: 97 active+clean; 194 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-26T18:03:26.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:26 smithi135 ceph-mon[35636]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-26T18:03:28.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:28 smithi144 ceph-mon[36095]: pgmap v1106: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.8 MiB/s wr, 276 op/s 2024-03-26T18:03:28.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:28 smithi105 ceph-mon[32280]: pgmap v1106: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.8 MiB/s wr, 276 op/s 2024-03-26T18:03:28.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:28 smithi135 ceph-mon[35636]: pgmap v1106: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 1.8 MiB/s wr, 276 op/s 2024-03-26T18:03:30.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:30 smithi144 ceph-mon[36095]: pgmap v1107: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-03-26T18:03:30.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:30 smithi105 ceph-mon[32280]: pgmap v1107: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-03-26T18:03:30.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:30 smithi135 ceph-mon[35636]: pgmap v1107: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 214 op/s 2024-03-26T18:03:31.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:31 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:03:31.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:31 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:03:31.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:31 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:03:32.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:32 smithi144 ceph-mon[36095]: pgmap v1108: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 706 KiB/s wr, 167 op/s 2024-03-26T18:03:32.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:03:32.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:03:32.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:03:32.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:32 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:03:32.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:32 smithi105 ceph-mon[32280]: pgmap v1108: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 706 KiB/s wr, 167 op/s 2024-03-26T18:03:32.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:03:32.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:03:32.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:03:32.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:32 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:03:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:32 smithi135 ceph-mon[35636]: pgmap v1108: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 706 KiB/s wr, 167 op/s 2024-03-26T18:03:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:03:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:03:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:03:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:32 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:03:33.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:33 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:03:33.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:33 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:03:33.337 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:33 smithi144 ceph-mon[36095]: scrub summary: idle 2024-03-26T18:03:33.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:33 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:03:33.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:33 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:03:33.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:33 smithi105 ceph-mon[32280]: scrub summary: idle 2024-03-26T18:03:33.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:33 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:03:33.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:33 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:03:33.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:33 smithi135 ceph-mon[35636]: scrub summary: idle 2024-03-26T18:03:34.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:34 smithi144 ceph-mon[36095]: pgmap v1109: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 251 KiB/s wr, 119 op/s 2024-03-26T18:03:34.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:34 smithi105 ceph-mon[32280]: pgmap v1109: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 251 KiB/s wr, 119 op/s 2024-03-26T18:03:34.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:34 smithi135 ceph-mon[35636]: pgmap v1109: 97 pgs: 97 active+clean; 180 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 251 KiB/s wr, 119 op/s 2024-03-26T18:03:36.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:36 smithi144 ceph-mon[36095]: pgmap v1110: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 78 op/s 2024-03-26T18:03:36.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:36 smithi105 ceph-mon[32280]: pgmap v1110: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 78 op/s 2024-03-26T18:03:36.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:36 smithi135 ceph-mon[35636]: pgmap v1110: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 91 KiB/s wr, 78 op/s 2024-03-26T18:03:38.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:38 smithi144 ceph-mon[36095]: pgmap v1111: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-03-26T18:03:38.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:38 smithi105 ceph-mon[32280]: pgmap v1111: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-03-26T18:03:38.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:38 smithi135 ceph-mon[35636]: pgmap v1111: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 71 KiB/s wr, 32 op/s 2024-03-26T18:03:40.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:40 smithi144 ceph-mon[36095]: pgmap v1112: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-03-26T18:03:40.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:40 smithi105 ceph-mon[32280]: pgmap v1112: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-03-26T18:03:40.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:40 smithi135 ceph-mon[35636]: pgmap v1112: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 18 op/s 2024-03-26T18:03:42.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:42 smithi144 ceph-mon[36095]: pgmap v1113: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-03-26T18:03:42.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:42 smithi105 ceph-mon[32280]: pgmap v1113: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-03-26T18:03:42.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:42 smithi135 ceph-mon[35636]: pgmap v1113: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-03-26T18:03:44.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:44 smithi144 ceph-mon[36095]: pgmap v1114: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T18:03:44.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:44 smithi105 ceph-mon[32280]: pgmap v1114: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T18:03:44.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:44 smithi135 ceph-mon[35636]: pgmap v1114: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-03-26T18:03:46.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:46 smithi144 ceph-mon[36095]: pgmap v1115: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-26T18:03:46.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:46 smithi105 ceph-mon[32280]: pgmap v1115: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-26T18:03:46.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:46 smithi135 ceph-mon[35636]: pgmap v1115: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-26T18:03:48.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:48 smithi144 ceph-mon[36095]: pgmap v1116: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-26T18:03:48.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:48 smithi105 ceph-mon[32280]: pgmap v1116: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-26T18:03:48.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T18:03:48.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T18:03:48.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:48 smithi135 ceph-mon[35636]: pgmap v1116: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 11 op/s 2024-03-26T18:03:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 -- 172.21.15.105:0/3907984231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 msgr2=0x7f96a4109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 --2- 172.21.15.105:0/3907984231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 0x7f96a4109d30 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f969c009ca0 tx=0x7f969c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 -- 172.21.15.105:0/3907984231 shutdown_connections 2024-03-26T18:03:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 --2- 172.21.15.105:0/3907984231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 0x7f96a4109d30 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 --2- 172.21.15.105:0/3907984231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a4102ae0 0x7f96a4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 --2- 172.21.15.105:0/3907984231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41018f0 0x7f96a4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 -- 172.21.15.105:0/3907984231 >> 172.21.15.105:0/3907984231 conn(0x7f96a40fce60 msgr2=0x7f96a40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 -- 172.21.15.105:0/3907984231 shutdown_connections 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.521+0000 7f96aaac6640 1 -- 172.21.15.105:0/3907984231 wait complete. 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96aaac6640 1 Processor -- start 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96aaac6640 1 -- start start 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96aaac6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41018f0 0x7f96a4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96aaac6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a4102ae0 0x7f96a4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96aaac6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 0x7f96a419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96aaac6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d420 con 0x7f96a41034a0 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96aaac6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d2a0 con 0x7f96a41018f0 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96aaac6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96a410d5a0 con 0x7f96a4102ae0 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a9063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 0x7f96a419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a8862640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41018f0 0x7f96a4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a9063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 0x7f96a419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49600/0 (socket says 172.21.15.105:49600) 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a4102ae0 0x7f96a4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a9063640 1 -- 172.21.15.105:0/1425465990 learned_addr learned my addr 172.21.15.105:0/1425465990 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a8862640 1 -- 172.21.15.105:0/1425465990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a4102ae0 msgr2=0x7f96a4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a8862640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a4102ae0 0x7f96a4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a8862640 1 -- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 msgr2=0x7f96a419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a8862640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 0x7f96a419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a8862640 1 -- 172.21.15.105:0/1425465990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a419f6e0 con 0x7f96a41018f0 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a3fff640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a4102ae0 0x7f96a4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a9063640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 0x7f96a419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.522+0000 7f96a8862640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41018f0 0x7f96a4197ac0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f969800e9c0 tx=0x7f969800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.523+0000 7f96a1ffb640 1 -- 172.21.15.105:0/1425465990 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969800ccc0 con 0x7f96a41018f0 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.523+0000 7f96aaac6640 1 -- 172.21.15.105:0/1425465990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a419f9d0 con 0x7f96a41018f0 2024-03-26T18:03:48.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.523+0000 7f96a1ffb640 1 -- 172.21.15.105:0/1425465990 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96980035f0 con 0x7f96a41018f0 2024-03-26T18:03:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.523+0000 7f96a1ffb640 1 -- 172.21.15.105:0/1425465990 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969800ce60 con 0x7f96a41018f0 2024-03-26T18:03:48.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.523+0000 7f96aaac6640 1 -- 172.21.15.105:0/1425465990 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a419ff70 con 0x7f96a41018f0 2024-03-26T18:03:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.525+0000 7f96a1ffb640 1 -- 172.21.15.105:0/1425465990 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f96980037a0 con 0x7f96a41018f0 2024-03-26T18:03:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.525+0000 7f96a1ffb640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96940a2f20 0x7f96940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96a3fff640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96940a2f20 0x7f96940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96a1ffb640 1 -- 172.21.15.105:0/1425465990 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f96980cf820 con 0x7f96a41018f0 2024-03-26T18:03:48.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96a3fff640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96940a2f20 0x7f96940a53e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f96a4198fd0 tx=0x7f9690005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96ad793740 1 Processor -- start 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96ad793740 1 -- start start 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96ad793740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd529ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96ad793740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd529fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96ad793740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96ad793740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564dd51ff980 con 0x564dd529a8f0 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96ad793740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564dd51a7be0 con 0x564dd529b2e0 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.526+0000 7f96ad793740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564dd51fb930 con 0x564dd52a0450 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd529ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd529ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49602/0 (socket says 172.21.15.105:49602) 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 -- 172.21.15.105:0/3484576393 learned_addr learned my addr 172.21.15.105:0/3484576393 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a3fff640 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd529fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a9063640 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 -- 172.21.15.105:0/3484576393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 msgr2=0x564dd52a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 -- 172.21.15.105:0/3484576393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 msgr2=0x564dd529fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd529fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 -- 172.21.15.105:0/3484576393 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd4fdd7d0 con 0x564dd529a8f0 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f96a8862640 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd529ad10 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f9698098370 tx=0x7f969809b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e6de365a72d6c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f968effd640 1 -- 172.21.15.105:0/3484576393 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980a0060 con 0x564dd529a8f0 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f968effd640 1 -- 172.21.15.105:0/3484576393 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96980a4920 con 0x564dd529a8f0 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.527+0000 7f968effd640 1 -- 172.21.15.105:0/3484576393 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698097ec0 con 0x564dd529a8f0 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 -- 172.21.15.105:0/3484576393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 msgr2=0x564dd529ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd529ad10 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f9698098370 tx=0x7f969809b970 comp rx=0 tx=0).stop 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 -- 172.21.15.105:0/3484576393 shutdown_connections 2024-03-26T18:03:48.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd529fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 --2- 172.21.15.105:0/3484576393 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd529ad10 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 -- 172.21.15.105:0/3484576393 >> 172.21.15.105:0/3484576393 conn(0x564dd5295e60 msgr2=0x564dd5296260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 -- 172.21.15.105:0/3484576393 shutdown_connections 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 -- 172.21.15.105:0/3484576393 wait complete. 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 -- start start 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.528+0000 7f96ad793740 1 Processor -- start 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96ad793740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd52a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96ad793740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96ad793740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52ac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96ad793740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564dd52a3400 con 0x564dd529a8f0 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96ad793740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564dd51bb160 con 0x564dd529b2e0 2024-03-26T18:03:48.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96ad793740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564dd52a3580 con 0x564dd52a0450 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96ad793740 10 client.0 _ll_register_callbacks cb 0x564dd51f7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd52a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd52a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49608/0 (socket says 172.21.15.105:49608) 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 -- 172.21.15.105:0/169023845 learned_addr learned my addr 172.21.15.105:0/169023845 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a9063640 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a3fff640 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 -- 172.21.15.105:0/169023845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 msgr2=0x564dd52ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 -- 172.21.15.105:0/169023845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 msgr2=0x564dd52a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 -- 172.21.15.105:0/169023845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd4f133f0 con 0x564dd529a8f0 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96a8862640 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd52a4bd0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f96980cfb40 tx=0x7f9698098580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28b90c5a77041a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:48.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f9683fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f9683fff640 1 -- 172.21.15.105:0/169023845 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969809b360 con 0x564dd529a8f0 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f9683fff640 1 -- 172.21.15.105:0/169023845 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f969809b500 con 0x564dd529a8f0 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.529+0000 7f96ad793740 1 -- 172.21.15.105:0/169023845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96980a4920 con 0x564dd529a8f0 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f9683fff640 1 -- 172.21.15.105:0/169023845 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9698097c60 con 0x564dd529a8f0 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96ad793740 10 client.24468 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96ad793740 1 -- 172.21.15.105:0/169023845 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564dd4ed30f0 con 0x564dd529a8f0 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f9683fff640 1 -- 172.21.15.105:0/169023845 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96980a4920 con 0x564dd529a8f0 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f9683fff640 1 -- 172.21.15.105:0/169023845 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f96980a11f0 con 0x564dd529a8f0 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96ad793740 10 client.24468 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96ad793740 10 client.24468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96ad793740 20 client.24468 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:48.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96ad793740 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564dd52b0640 0x564dd52b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96ad793740 4 client.24468 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96ad793740 1 -- 172.21.15.105:0/169023845 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dd520cec0 con 0x564dd52b0640 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.530+0000 7f96a3fff640 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564dd52b0640 0x564dd52b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f9683fff640 10 client.24468 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96a3fff640 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564dd52b0640 0x564dd52b2a30 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef7a11c8b0885ee server_cookie=25cbf870cd180c16 in_seq=0 out_seq=0 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f9683fff640 1 -- 172.21.15.105:0/169023845 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564dd520cec0 con 0x564dd52b0640 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f9683fff640 10 client.24468 handle_command_reply: tid=0 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 client.24468 shutdown 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 -- 172.21.15.105:0/169023845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 msgr2=0x564dd52a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd52a4bd0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f96980cfb40 tx=0x7f9698098580 comp rx=0 tx=0).stop 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 -- 172.21.15.105:0/169023845 shutdown_connections 2024-03-26T18:03:48.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564dd52b0640 0x564dd52b2a30 crc :-1 s=READY pgs=628 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f9683fff640 0 client.24468 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 --2- 172.21.15.105:0/169023845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd52a4bd0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.531+0000 7f96ad793740 1 -- 172.21.15.105:0/169023845 >> 172.21.15.105:0/169023845 conn(0x564dd5295e60 msgr2=0x564dd5295080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.532+0000 7f96ad793740 1 -- 172.21.15.105:0/169023845 shutdown_connections 2024-03-26T18:03:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.532+0000 7f96ad793740 1 -- 172.21.15.105:0/169023845 wait complete. 2024-03-26T18:03:48.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.532+0000 7f96ad793740 20 client.24468 trim_cache size 0 max 16384 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.550+0000 7f96ad793740 1 Processor -- start 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.550+0000 7f96ad793740 1 -- start start 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.550+0000 7f96ad793740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd5296240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.550+0000 7f96ad793740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96ad793740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52a5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96ad793740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a8e00 con 0x564dd529a8f0 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96ad793740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41aa9a0 con 0x564dd529b2e0 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96ad793740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a9500 con 0x564dd52a0450 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd5296240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd5296240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49610/0 (socket says 172.21.15.105:49610) 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 -- 172.21.15.105:0/2323237471 learned_addr learned my addr 172.21.15.105:0/2323237471 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a3fff640 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a9063640 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 -- 172.21.15.105:0/2323237471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 msgr2=0x564dd52a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 -- 172.21.15.105:0/2323237471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 msgr2=0x564dd52ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 -- 172.21.15.105:0/2323237471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dd529f850 con 0x564dd529a8f0 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f96a8862640 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd5296240 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f969809a1c0 tx=0x7f969809bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec7298d0fd942e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f968e7fc640 1 -- 172.21.15.105:0/2323237471 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980981b0 con 0x564dd529a8f0 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f968e7fc640 1 -- 172.21.15.105:0/2323237471 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96980a4920 con 0x564dd529a8f0 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.551+0000 7f968e7fc640 1 -- 172.21.15.105:0/2323237471 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969809ba50 con 0x564dd529a8f0 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 -- 172.21.15.105:0/2323237471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 msgr2=0x564dd5296240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd5296240 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f969809a1c0 tx=0x7f969809bce0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 -- 172.21.15.105:0/2323237471 shutdown_connections 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd52a0450 0x564dd52a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 --2- 172.21.15.105:0/2323237471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd529a8f0 0x564dd5296240 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 -- 172.21.15.105:0/2323237471 >> 172.21.15.105:0/2323237471 conn(0x564dd5295e60 msgr2=0x564dd52a3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 -- 172.21.15.105:0/2323237471 shutdown_connections 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 -- 172.21.15.105:0/2323237471 wait complete. 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 -- start start 2024-03-26T18:03:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 Processor -- start 2024-03-26T18:03:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd529a8f0 0x564dd52a9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd52a0450 0x564dd52aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41aa2d0 con 0x564dd52a0450 2024-03-26T18:03:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.552+0000 7f96ad793740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a9c00 con 0x564dd529b2e0 2024-03-26T18:03:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96ad793740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96a41a6de0 con 0x564dd529a8f0 2024-03-26T18:03:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96ad793740 10 client.0 _ll_register_callbacks cb 0x7f9690009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd52a0450 0x564dd52aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd52a0450 0x564dd52aae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49618/0 (socket says 172.21.15.105:49618) 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 -- 172.21.15.105:0/3170650808 learned_addr learned my addr 172.21.15.105:0/3170650808 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a3fff640 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a8862640 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd529a8f0 0x564dd52a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 -- 172.21.15.105:0/3170650808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd529a8f0 msgr2=0x564dd52a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd529a8f0 0x564dd52a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 -- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 msgr2=0x564dd52a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 -- 172.21.15.105:0/3170650808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dd52ab450 con 0x564dd52a0450 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a3fff640 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96a9063640 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd52a0450 0x564dd52aae60 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f969c02c700 tx=0x7f969c009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bd78bf1b921305 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f9683fff640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f9683fff640 1 -- 172.21.15.105:0/3170650808 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c039040 con 0x564dd52a0450 2024-03-26T18:03:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f9683fff640 1 -- 172.21.15.105:0/3170650808 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f969c003cc0 con 0x564dd52a0450 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96ad793740 1 -- 172.21.15.105:0/3170650808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96980a4920 con 0x564dd52a0450 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f9683fff640 1 -- 172.21.15.105:0/3170650808 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f969c004040 con 0x564dd52a0450 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96ad793740 10 client.24474 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.553+0000 7f96ad793740 1 -- 172.21.15.105:0/3170650808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564dd5316920 con 0x564dd52a0450 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f9683fff640 1 -- 172.21.15.105:0/3170650808 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96980a4920 con 0x564dd52a0450 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f9683fff640 1 -- 172.21.15.105:0/3170650808 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f969c034830 con 0x564dd52a0450 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f96ad793740 10 client.24474 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f96ad793740 10 client.24474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f96ad793740 20 client.24474 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f96ad793740 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564dd5317370 0x564dd5319790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f96ad793740 4 client.24474 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f96ad793740 1 -- 172.21.15.105:0/3170650808 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564dd5207210 con 0x564dd5317370 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f96a8862640 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564dd5317370 0x564dd5319790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f9683fff640 10 client.24474 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f96a8862640 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564dd5317370 0x564dd5319790 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3adebed5008d2e server_cookie=5a83912f7d54a9ac in_seq=0 out_seq=0 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.554+0000 7f9683fff640 1 -- 172.21.15.105:0/3170650808 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564dd5207210 con 0x564dd5317370 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f9683fff640 10 client.24474 handle_command_reply: tid=0 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 client.24474 shutdown 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 -- 172.21.15.105:0/3170650808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd52a0450 msgr2=0x564dd52aae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd52a0450 0x564dd52aae60 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f969c02c700 tx=0x7f969c009a10 comp rx=0 tx=0).stop 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 -- 172.21.15.105:0/3170650808 shutdown_connections 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x564dd5317370 0x564dd5319790 crc :-1 s=READY pgs=629 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x564dd52a0450 0x564dd52aae60 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564dd529b2e0 0x564dd52a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f9683fff640 0 client.24474 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 --2- 172.21.15.105:0/3170650808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564dd529a8f0 0x564dd52a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 -- 172.21.15.105:0/3170650808 >> 172.21.15.105:0/3170650808 conn(0x564dd5295e60 msgr2=0x564dd52990c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 -- 172.21.15.105:0/3170650808 shutdown_connections 2024-03-26T18:03:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 1 -- 172.21.15.105:0/3170650808 wait complete. 2024-03-26T18:03:48.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96ad793740 20 client.24474 trim_cache size 0 max 16384 2024-03-26T18:03:48.542 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:03:48.542 INFO:teuthology.orchestra.run.smithi105.stdout: "status": "no active scrubs running", 2024-03-26T18:03:48.542 INFO:teuthology.orchestra.run.smithi105.stdout: "scrubs": {} 2024-03-26T18:03:48.542 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:03:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 -- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96940a2f20 msgr2=0x7f96940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96940a2f20 0x7f96940a53e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f96a4198fd0 tx=0x7f9690005f90 comp rx=0 tx=0).stop 2024-03-26T18:03:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 -- 172.21.15.105:0/1425465990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41018f0 msgr2=0x7f96a4197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41018f0 0x7f96a4197ac0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f969800e9c0 tx=0x7f969800ee80 comp rx=0 tx=0).stop 2024-03-26T18:03:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 -- 172.21.15.105:0/1425465990 shutdown_connections 2024-03-26T18:03:48.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f96940a2f20 0x7f96940a53e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96a41034a0 0x7f96a419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a4102ae0 0x7f96a4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 --2- 172.21.15.105:0/1425465990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96a41018f0 0x7f96a4197ac0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.555+0000 7f96aaac6640 1 -- 172.21.15.105:0/1425465990 >> 172.21.15.105:0/1425465990 conn(0x7f96a40fce60 msgr2=0x7f96a4105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.556+0000 7f96aaac6640 1 -- 172.21.15.105:0/1425465990 shutdown_connections 2024-03-26T18:03:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.556+0000 7f96aaac6640 1 -- 172.21.15.105:0/1425465990 wait complete. 2024-03-26T18:03:48.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T18:03:48.547 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:03:48.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 -- 172.21.15.105:0/1362407784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97d8104d10 msgr2=0x7f97d8107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 --2- 172.21.15.105:0/1362407784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97d8104d10 0x7f97d8107100 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009860 tx=0x7f97c402bb70 comp rx=0 tx=0).stop 2024-03-26T18:03:48.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 -- 172.21.15.105:0/1362407784 shutdown_connections 2024-03-26T18:03:48.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 --2- 172.21.15.105:0/1362407784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d8107640 0x7f97d8109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 --2- 172.21.15.105:0/1362407784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97d8104d10 0x7f97d8107100 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 --2- 172.21.15.105:0/1362407784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97d81023e0 0x7f97d81047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 -- 172.21.15.105:0/1362407784 >> 172.21.15.105:0/1362407784 conn(0x7f97d80fbd70 msgr2=0x7f97d80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 -- 172.21.15.105:0/1362407784 shutdown_connections 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.685+0000 7f97df061640 1 -- 172.21.15.105:0/1362407784 wait complete. 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97df061640 1 Processor -- start 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97df061640 1 -- start start 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97df061640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d81023e0 0x7f97d8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97df061640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97d8104d10 0x7f97d81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97df061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97d8107640 0x7f97d819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97df061640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97d810d400 con 0x7f97d81023e0 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97df061640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97d810d280 con 0x7f97d8104d10 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97df061640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97d810d580 con 0x7f97d8107640 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d81023e0 0x7f97d8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d81023e0 0x7f97d8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49644/0 (socket says 172.21.15.105:49644) 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 -- 172.21.15.105:0/2172967258 learned_addr learned my addr 172.21.15.105:0/2172967258 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dd5fe640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97d8107640 0x7f97d819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.668 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97d7fff640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97d8104d10 0x7f97d81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 -- 172.21.15.105:0/2172967258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97d8107640 msgr2=0x7f97d819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97d8107640 0x7f97d819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 -- 172.21.15.105:0/2172967258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97d8104d10 msgr2=0x7f97d81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97d8104d10 0x7f97d81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 -- 172.21.15.105:0/2172967258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d819f7c0 con 0x7f97d81023e0 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97dcdfd640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d81023e0 0x7f97d8197ba0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e9f0 tx=0x7f97cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.686+0000 7f97d5ffb640 1 -- 172.21.15.105:0/2172967258 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc00cd80 con 0x7f97d81023e0 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.687+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d819fab0 con 0x7f97d81023e0 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.687+0000 7f97d5ffb640 1 -- 172.21.15.105:0/2172967258 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97cc0108e0 con 0x7f97d81023e0 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.687+0000 7f97d5ffb640 1 -- 172.21.15.105:0/2172967258 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc018920 con 0x7f97d81023e0 2024-03-26T18:03:48.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.687+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d81a0020 con 0x7f97d81023e0 2024-03-26T18:03:48.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.688+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d80fd860 con 0x7f97d81023e0 2024-03-26T18:03:48.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.689+0000 7f97d5ffb640 1 -- 172.21.15.105:0/2172967258 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f97cc010a90 con 0x7f97d81023e0 2024-03-26T18:03:48.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.689+0000 7f97d5ffb640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97c80a2f20 0x7f97c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:48.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.690+0000 7f97d7fff640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97c80a2f20 0x7f97c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:48.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.690+0000 7f97d5ffb640 1 -- 172.21.15.105:0/2172967258 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f97cc0cf700 con 0x7f97d81023e0 2024-03-26T18:03:48.671 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.690+0000 7f97d7fff640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97c80a2f20 0x7f97c80a53e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f97c402c180 tx=0x7f97c40349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:48.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.697+0000 7f97d5ffb640 1 -- 172.21.15.105:0/2172967258 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f97cc09bea0 con 0x7f97d81023e0 2024-03-26T18:03:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.917+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97d80624f0 con 0x7f97d81023e0 2024-03-26T18:03:48.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.918+0000 7f97d5ffb640 1 -- 172.21.15.105:0/2172967258 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11946 (secure 0 0 0) 0x7f97cc09ff40 con 0x7f97d81023e0 2024-03-26T18:03:48.898 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:03:48.898 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:03:24.037550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_33599":{"gid":33599,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1455536711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1455536711},{"type":"v1","addr":"172.21.15.135:6835","nonce":1455536711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24482,14628,24487,24466]},"id":1}]} 2024-03-26T18:03:48.898 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 48 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97c80a2f20 msgr2=0x7f97c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97c80a2f20 0x7f97c80a53e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f97c402c180 tx=0x7f97c40349a0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d81023e0 msgr2=0x7f97d8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d81023e0 0x7f97d8197ba0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e9f0 tx=0x7f97cc00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 shutdown_connections 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f97c80a2f20 0x7f97c80a53e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97d8107640 0x7f97d819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97d8104d10 0x7f97d81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 --2- 172.21.15.105:0/2172967258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97d81023e0 0x7f97d8197ba0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 >> 172.21.15.105:0/2172967258 conn(0x7f97d80fbd70 msgr2=0x7f97d8102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:48.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 shutdown_connections 2024-03-26T18:03:48.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:48.920+0000 7f97df061640 1 -- 172.21.15.105:0/2172967258 wait complete. 2024-03-26T18:03:48.915 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-26T18:03:49.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 -- 172.21.15.105:0/2952979458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041034a0 msgr2=0x7f8d04109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 --2- 172.21.15.105:0/2952979458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041034a0 0x7f8d04109d30 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009900 tx=0x7f8cfc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:49.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 -- 172.21.15.105:0/2952979458 shutdown_connections 2024-03-26T18:03:49.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 --2- 172.21.15.105:0/2952979458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041034a0 0x7f8d04109d30 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 --2- 172.21.15.105:0/2952979458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d04102ae0 0x7f8d04102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 --2- 172.21.15.105:0/2952979458 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d041018f0 0x7f8d04101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 -- 172.21.15.105:0/2952979458 >> 172.21.15.105:0/2952979458 conn(0x7f8d040fce60 msgr2=0x7f8d040ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 -- 172.21.15.105:0/2952979458 shutdown_connections 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 -- 172.21.15.105:0/2952979458 wait complete. 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.041+0000 7f8d08c36640 1 Processor -- start 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d08c36640 1 -- start start 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d08c36640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041018f0 0x7f8d04197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d08c36640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d04102ae0 0x7f8d04197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d08c36640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d041034a0 0x7f8d0419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d08c36640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410d400 con 0x7f8d041018f0 2024-03-26T18:03:49.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d08c36640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410d280 con 0x7f8d041034a0 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d08c36640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410d580 con 0x7f8d04102ae0 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041018f0 0x7f8d04197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041018f0 0x7f8d04197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49658/0 (socket says 172.21.15.105:49658) 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 -- 172.21.15.105:0/3811747624 learned_addr learned my addr 172.21.15.105:0/3811747624 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8cf9d9b640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d04102ae0 0x7f8d04197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d041034a0 0x7f8d0419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 -- 172.21.15.105:0/3811747624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d04102ae0 msgr2=0x7f8d04197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d04102ae0 0x7f8d04197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 -- 172.21.15.105:0/3811747624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d041034a0 msgr2=0x7f8d0419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d041034a0 0x7f8d0419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 -- 172.21.15.105:0/3811747624 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0419f6c0 con 0x7f8d041018f0 2024-03-26T18:03:49.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8cf9d9b640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d04102ae0 0x7f8d04197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.042+0000 7f8d0259c640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041018f0 0x7f8d04197aa0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400ebc0 tx=0x7f8cf400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.043+0000 7f8cfbfff640 1 -- 172.21.15.105:0/3811747624 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4003b70 con 0x7f8d041018f0 2024-03-26T18:03:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.043+0000 7f8d08c36640 1 -- 172.21.15.105:0/3811747624 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0419f9b0 con 0x7f8d041018f0 2024-03-26T18:03:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.043+0000 7f8cfbfff640 1 -- 172.21.15.105:0/3811747624 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cf4003d10 con 0x7f8d041018f0 2024-03-26T18:03:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.043+0000 7f8cfbfff640 1 -- 172.21.15.105:0/3811747624 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4019820 con 0x7f8d041018f0 2024-03-26T18:03:49.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.043+0000 7f8d08c36640 1 -- 172.21.15.105:0/3811747624 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0419ff50 con 0x7f8d041018f0 2024-03-26T18:03:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.045+0000 7f8cfbfff640 1 -- 172.21.15.105:0/3811747624 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8cf40199c0 con 0x7f8d041018f0 2024-03-26T18:03:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8d0b81d740 1 Processor -- start 2024-03-26T18:03:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8d0b81d740 1 -- start start 2024-03-26T18:03:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7704e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7709f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 0x5640d770ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5640d7669970 con 0x5640d7705420 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5640d7611d70 con 0x5640d770a590 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5640d7665920 con 0x5640d7704a30 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.046+0000 7f8cfbfff640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cf00a2f20 0x7f8cf00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d0259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7709f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7704e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7704e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35764/0 (socket says 172.21.15.105:35764) 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d0259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7709f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49672/0 (socket says 172.21.15.105:49672) 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 -- 172.21.15.105:0/3328809242 learned_addr learned my addr 172.21.15.105:0/3328809242 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8cf9d9b640 1 --2- 172.21.15.105:0/3328809242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 0x5640d770ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cf00a2f20 0x7f8cf00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8cfbfff640 1 -- 172.21.15.105:0/3811747624 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f8cf40cfcb0 con 0x7f8d041018f0 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 -- 172.21.15.105:0/3328809242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 msgr2=0x5640d770ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3328809242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 0x5640d770ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 -- 172.21.15.105:0/3328809242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 msgr2=0x5640d7709f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3328809242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7709f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 -- 172.21.15.105:0/3328809242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d737d3f0 con 0x5640d7704a30 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3328809242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7704e50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0096c0 tx=0x7f8cfc004330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a59852acd0690a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.047+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cf00a2f20 0x7f8cf00a53e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005c40 tx=0x7f8cfc03b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8cf8d99640 1 -- 172.21.15.105:0/3328809242 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc02ca90 con 0x5640d7704a30 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8cf8d99640 1 -- 172.21.15.105:0/3328809242 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cfc051070 con 0x5640d7704a30 2024-03-26T18:03:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8cf8d99640 1 -- 172.21.15.105:0/3328809242 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc003850 con 0x5640d7704a30 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3328809242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 msgr2=0x5640d7704e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3328809242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7704e50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0096c0 tx=0x7f8cfc004330 comp rx=0 tx=0).stop 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3328809242 shutdown_connections 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3328809242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 0x5640d770ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3328809242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7709f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3328809242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7704e50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.048+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3328809242 >> 172.21.15.105:0/3328809242 conn(0x5640d76fffa0 msgr2=0x5640d77003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3328809242 shutdown_connections 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3328809242 wait complete. 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 -- start start 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 Processor -- start 2024-03-26T18:03:49.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7704a30 0x5640d770efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7705420 0x5640d770f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d77164b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5640d770d4e0 con 0x5640d770a590 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5640d7625310 con 0x5640d7704a30 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5640d770d660 con 0x5640d7705420 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0b81d740 10 client.0 _ll_register_callbacks cb 0x5640d7660c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d02d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d77164b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7704a30 0x5640d770efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8cf9d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7705420 0x5640d770f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7704a30 0x5640d770efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54098/0 (socket says 172.21.15.105:54098) 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d02d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d77164b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49676/0 (socket says 172.21.15.105:49676) 2024-03-26T18:03:49.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.049+0000 7f8d0259c640 1 -- 172.21.15.105:0/3254631284 learned_addr learned my addr 172.21.15.105:0/3254631284 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8d02d9d640 1 -- 172.21.15.105:0/3254631284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7705420 msgr2=0x5640d770f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7705420 0x5640d770f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8d02d9d640 1 -- 172.21.15.105:0/3254631284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7704a30 msgr2=0x5640d770efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7704a30 0x5640d770efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8d02d9d640 1 -- 172.21.15.105:0/3254631284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d74477d0 con 0x5640d770a590 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8cf9d9b640 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7705420 0x5640d770f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8d0259c640 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7704a30 0x5640d770efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d77164b0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0096c0 tx=0x7f8cfc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c86216241e51b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.050+0000 7f8ce9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.051+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/3254631284 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0048c0 con 0x5640d770a590 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.051+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/3254631284 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f8cfc004a60 con 0x5640d770a590 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.051+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3254631284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cfc051070 con 0x5640d770a590 2024-03-26T18:03:49.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.051+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/3254631284 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8cfc004e50 con 0x5640d770a590 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.051+0000 7f8d0b81d740 10 client.24492 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.051+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3254631284 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5640d770d4e0 con 0x5640d770a590 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.051+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/3254631284 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cfc051070 con 0x5640d770a590 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.051+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/3254631284 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f8cfc044ae0 con 0x5640d770a590 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8d0b81d740 10 client.24492 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8d0b81d740 10 client.24492 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8d0b81d740 20 client.24492 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5640d771ab20 0x5640d771cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8d0b81d740 4 client.24492 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3254631284 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640d7676eb0 con 0x5640d771ab20 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8d0259c640 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5640d771ab20 0x5640d771cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8d0259c640 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5640d771ab20 0x5640d771cf50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0cd82b1c54ba000 server_cookie=b8d701bb740c3d7d in_seq=0 out_seq=0 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.052+0000 7f8ce9ffb640 10 client.24492 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/3254631284 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5640d7676eb0 con 0x5640d771ab20 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8ce9ffb640 10 client.24492 handle_command_reply: tid=0 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 client.24492 shutdown 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3254631284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 msgr2=0x5640d77164b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d77164b0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0096c0 tx=0x7f8cfc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3254631284 shutdown_connections 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5640d771ab20 0x5640d771cf50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d77164b0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7705420 0x5640d770f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8ce9ffb640 0 client.24492 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/3254631284 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7704a30 0x5640d770efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.040 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3254631284 >> 172.21.15.105:0/3254631284 conn(0x5640d76fffa0 msgr2=0x5640d76ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3254631284 shutdown_connections 2024-03-26T18:03:49.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 1 -- 172.21.15.105:0/3254631284 wait complete. 2024-03-26T18:03:49.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.053+0000 7f8d0b81d740 20 client.24492 trim_cache size 0 max 16384 2024-03-26T18:03:49.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0b81d740 1 Processor -- start 2024-03-26T18:03:49.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0b81d740 1 -- start start 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d770da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7716470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 0x5640d770df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041a6b00 con 0x5640d7705420 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041a86a0 con 0x5640d770a590 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041a7200 con 0x5640d7704a30 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7716470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7716470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49686/0 (socket says 172.21.15.105:49686) 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 -- 172.21.15.105:0/1419907914 learned_addr learned my addr 172.21.15.105:0/1419907914 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d0259c640 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d770da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 0x5640d770df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 -- 172.21.15.105:0/1419907914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 msgr2=0x5640d770da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d770da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 -- 172.21.15.105:0/1419907914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 msgr2=0x5640d770df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 0x5640d770df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 -- 172.21.15.105:0/1419907914 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d7709ca0 con 0x5640d7705420 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.076+0000 7f8cf9d9b640 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7716470 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f8cec012870 tx=0x7f8cec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6b7d980c592181 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8cebfff640 1 -- 172.21.15.105:0/1419907914 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec022070 con 0x5640d7705420 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8cebfff640 1 -- 172.21.15.105:0/1419907914 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cec00f920 con 0x5640d7705420 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8cebfff640 1 -- 172.21.15.105:0/1419907914 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec003810 con 0x5640d7705420 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 -- 172.21.15.105:0/1419907914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 msgr2=0x5640d7716470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7716470 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f8cec012870 tx=0x7f8cec005f90 comp rx=0 tx=0).stop 2024-03-26T18:03:49.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 -- 172.21.15.105:0/1419907914 shutdown_connections 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d770a590 0x5640d770df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d7705420 0x5640d7716470 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/1419907914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d770da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 -- 172.21.15.105:0/1419907914 >> 172.21.15.105:0/1419907914 conn(0x5640d76fffa0 msgr2=0x5640d7700380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 -- 172.21.15.105:0/1419907914 shutdown_connections 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 -- 172.21.15.105:0/1419907914 wait complete. 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.077+0000 7f8d0b81d740 1 -- start start 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 1 Processor -- start 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7713480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7705420 0x5640d77139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d777ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041a7fd0 con 0x5640d770a590 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041a7900 con 0x5640d7705420 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041a4ae0 con 0x5640d7704a30 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 10 client.0 _ll_register_callbacks cb 0x7f8cf4097f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d777ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d777ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49692/0 (socket says 172.21.15.105:49692) 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 -- 172.21.15.105:0/2864822400 learned_addr learned my addr 172.21.15.105:0/2864822400 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8cf9d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7705420 0x5640d77139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0259c640 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7713480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 -- 172.21.15.105:0/2864822400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 msgr2=0x5640d7713480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7713480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 -- 172.21.15.105:0/2864822400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7705420 msgr2=0x5640d77139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7705420 0x5640d77139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 -- 172.21.15.105:0/2864822400 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d777f440 con 0x5640d770a590 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d02d9d640 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d777ee80 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc046010 tx=0x7f8cfc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c355970947f96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8ce9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/2864822400 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc004ad0 con 0x5640d770a590 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/2864822400 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f8cfc044460 con 0x5640d770a590 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.078+0000 7f8d0b81d740 1 -- 172.21.15.105:0/2864822400 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cec00f920 con 0x5640d770a590 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/2864822400 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8cfc044660 con 0x5640d770a590 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0b81d740 10 client.24498 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0b81d740 1 -- 172.21.15.105:0/2864822400 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5640d7780740 con 0x5640d770a590 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/2864822400 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cec00f920 con 0x5640d770a590 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/2864822400 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f8cfc069060 con 0x5640d770a590 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0b81d740 10 client.24498 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0b81d740 10 client.24498 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0b81d740 20 client.24498 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5640d7780f50 0x5640d7783370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0b81d740 4 client.24498 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0b81d740 1 -- 172.21.15.105:0/2864822400 --> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640d7706500 con 0x5640d7780f50 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0259c640 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5640d7780f50 0x5640d7783370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8ce9ffb640 10 client.24498 ms_handle_connect on v2:172.21.15.105:6840/3843959520 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.079+0000 7f8d0259c640 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5640d7780f50 0x5640d7783370 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95138e2e196b07c1 server_cookie=a5904bf632978e20 in_seq=0 out_seq=0 2024-03-26T18:03:49.067 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8ce9ffb640 1 -- 172.21.15.105:0/2864822400 <== mds.4 v2:172.21.15.105:6840/3843959520 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640d7706500 con 0x5640d7780f50 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8ce9ffb640 10 client.24498 handle_command_reply: tid=0 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 client.24498 shutdown 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 -- 172.21.15.105:0/2864822400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 msgr2=0x5640d777ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d777ee80 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc046010 tx=0x7f8cfc005bd0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 -- 172.21.15.105:0/2864822400 shutdown_connections 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.105:6840/3843959520,v1:172.21.15.105:6841/3843959520] conn(0x5640d7780f50 0x5640d7783370 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5640d770a590 0x5640d777ee80 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5640d7705420 0x5640d77139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 --2- 172.21.15.105:0/2864822400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5640d7704a30 0x5640d7713480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 -- 172.21.15.105:0/2864822400 >> 172.21.15.105:0/2864822400 conn(0x5640d76fffa0 msgr2=0x5640d770da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8ce9ffb640 0 client.24498 ms_handle_reset on v2:172.21.15.105:6840/3843959520 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 -- 172.21.15.105:0/2864822400 shutdown_connections 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 1 -- 172.21.15.105:0/2864822400 wait complete. 2024-03-26T18:03:49.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d0b81d740 20 client.24498 trim_cache size 0 max 16384 2024-03-26T18:03:49.069 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:49.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d08c36640 1 -- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cf00a2f20 msgr2=0x7f8cf00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d08c36640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cf00a2f20 0x7f8cf00a53e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005c40 tx=0x7f8cfc03b000 comp rx=0 tx=0).stop 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d08c36640 1 -- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041018f0 msgr2=0x7f8d04197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d08c36640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041018f0 0x7f8d04197aa0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400ebc0 tx=0x7f8cf400c6a0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d08c36640 1 -- 172.21.15.105:0/3811747624 shutdown_connections 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.080+0000 7f8d08c36640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8cf00a2f20 0x7f8cf00a53e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.081+0000 7f8d08c36640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8d041034a0 0x7f8d0419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.081+0000 7f8d08c36640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d04102ae0 0x7f8d04197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.081+0000 7f8d08c36640 1 --2- 172.21.15.105:0/3811747624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d041018f0 0x7f8d04197aa0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.081+0000 7f8d08c36640 1 -- 172.21.15.105:0/3811747624 >> 172.21.15.105:0/3811747624 conn(0x7f8d040fce60 msgr2=0x7f8d04105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.081+0000 7f8d08c36640 1 -- 172.21.15.105:0/3811747624 shutdown_connections 2024-03-26T18:03:49.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.081+0000 7f8d08c36640 1 -- 172.21.15.105:0/3811747624 wait complete. 2024-03-26T18:03:49.071 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-26T18:03:49.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 -- 172.21.15.105:0/171493340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6564104d30 msgr2=0x7f6564107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 --2- 172.21.15.105:0/171493340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6564104d30 0x7f6564107120 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6558009860 tx=0x7f655802bb70 comp rx=0 tx=0).stop 2024-03-26T18:03:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 -- 172.21.15.105:0/171493340 shutdown_connections 2024-03-26T18:03:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 --2- 172.21.15.105:0/171493340 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6564107660 0x7f6564109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 --2- 172.21.15.105:0/171493340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6564104d30 0x7f6564107120 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 --2- 172.21.15.105:0/171493340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6564102400 0x7f65641047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 -- 172.21.15.105:0/171493340 >> 172.21.15.105:0/171493340 conn(0x7f65640fbd70 msgr2=0x7f65640fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 -- 172.21.15.105:0/171493340 shutdown_connections 2024-03-26T18:03:49.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.196+0000 7f656a337640 1 -- 172.21.15.105:0/171493340 wait complete. 2024-03-26T18:03:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.197+0000 7f656a337640 1 Processor -- start 2024-03-26T18:03:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.197+0000 7f656a337640 1 -- start start 2024-03-26T18:03:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f656a337640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6564102400 0x7f6564197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f656a337640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6564104d30 0x7f65641980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f656a337640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6564107660 0x7f656419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f656a337640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f656410d400 con 0x7f6564104d30 2024-03-26T18:03:49.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f656a337640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f656410d280 con 0x7f6564107660 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f656a337640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f656410d580 con 0x7f6564102400 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6564104d30 0x7f65641980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f6563fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6564102400 0x7f6564197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6564104d30 0x7f65641980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49698/0 (socket says 172.21.15.105:49698) 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 -- 172.21.15.105:0/4124996942 learned_addr learned my addr 172.21.15.105:0/4124996942 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65688d4640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6564107660 0x7f656419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 -- 172.21.15.105:0/4124996942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6564102400 msgr2=0x7f6564197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6564102400 0x7f6564197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 -- 172.21.15.105:0/4124996942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6564107660 msgr2=0x7f656419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6564107660 0x7f656419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 -- 172.21.15.105:0/4124996942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656419f790 con 0x7f6564104d30 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65637fe640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6564104d30 0x7f65641980b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f6558005940 tx=0x7f65580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65617fa640 1 -- 172.21.15.105:0/4124996942 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558041020 con 0x7f6564104d30 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f656a337640 1 -- 172.21.15.105:0/4124996942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656419fa20 con 0x7f6564104d30 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65617fa640 1 -- 172.21.15.105:0/4124996942 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65580035f0 con 0x7f6564104d30 2024-03-26T18:03:49.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f65617fa640 1 -- 172.21.15.105:0/4124996942 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558039040 con 0x7f6564104d30 2024-03-26T18:03:49.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.198+0000 7f656a337640 1 -- 172.21.15.105:0/4124996942 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656419ff10 con 0x7f6564104d30 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.200+0000 7f65617fa640 1 -- 172.21.15.105:0/4124996942 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f6558044020 con 0x7f6564104d30 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.201+0000 7f65617fa640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f654c0a2f20 0x7f654c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.201+0000 7f6563fff640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f654c0a2f20 0x7f654c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.201+0000 7f65617fa640 1 -- 172.21.15.105:0/4124996942 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f65580f0dd0 con 0x7f6564104d30 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f656cffc740 1 Processor -- start 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f6563fff640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f654c0a2f20 0x7f654c0a53e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6554009bb0 tx=0x7f6554005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f656cffc740 1 -- start start 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f656cffc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3761e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f656cffc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3766f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f656cffc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 0x55d6f3769a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f656cffc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d6f36c6970 con 0x55d6f3762420 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f656cffc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d6f366ed70 con 0x55d6f3767590 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f656cffc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d6f36c2920 con 0x55d6f3761a30 2024-03-26T18:03:49.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3766f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3766f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49714/0 (socket says 172.21.15.105:49714) 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 -- 172.21.15.105:0/1997384859 learned_addr learned my addr 172.21.15.105:0/1997384859 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f6563fff640 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3761e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65688d4640 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 0x55d6f3769a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 -- 172.21.15.105:0/1997384859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 msgr2=0x55d6f3761e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3761e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 -- 172.21.15.105:0/1997384859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 msgr2=0x55d6f3769a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 0x55d6f3769a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 -- 172.21.15.105:0/1997384859 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6f33da3f0 con 0x55d6f3762420 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f65637fe640 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3766f20 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f65580bd450 tx=0x7f65580bd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c2cab3380a336d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.202+0000 7f654a7fc640 1 -- 172.21.15.105:0/1997384859 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580bdbc0 con 0x55d6f3762420 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f654a7fc640 1 -- 172.21.15.105:0/1997384859 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65580bd4c0 con 0x55d6f3762420 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f654a7fc640 1 -- 172.21.15.105:0/1997384859 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580b9b30 con 0x55d6f3762420 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 -- 172.21.15.105:0/1997384859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 msgr2=0x55d6f3766f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3766f20 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f65580bd450 tx=0x7f65580bd000 comp rx=0 tx=0).stop 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 -- 172.21.15.105:0/1997384859 shutdown_connections 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 0x55d6f3769a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3766f20 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 --2- 172.21.15.105:0/1997384859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3761e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 -- 172.21.15.105:0/1997384859 >> 172.21.15.105:0/1997384859 conn(0x55d6f375cfa0 msgr2=0x55d6f375d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 -- 172.21.15.105:0/1997384859 shutdown_connections 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 -- 172.21.15.105:0/1997384859 wait complete. 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.203+0000 7f656cffc740 1 -- start start 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f656cffc740 1 Processor -- start 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f656cffc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f376bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f656cffc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f376c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f656cffc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 0x55d6f3773180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f656cffc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55d6f376a300 con 0x55d6f3762420 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f656cffc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d6f3682310 con 0x55d6f3767590 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f656cffc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d6f376a480 con 0x55d6f3761a30 2024-03-26T18:03:49.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f656cffc740 10 client.0 _ll_register_callbacks cb 0x55d6f36bdc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f376c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f6563fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f376bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f376c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49726/0 (socket says 172.21.15.105:49726) 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 -- 172.21.15.105:0/2955201198 learned_addr learned my addr 172.21.15.105:0/2955201198 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65688d4640 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 0x55d6f3773180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f6563fff640 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f376bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35828/0 (socket says 172.21.15.105:35828) 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 -- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 msgr2=0x55d6f376bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f376bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 -- 172.21.15.105:0/2955201198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 msgr2=0x55d6f3773180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 0x55d6f3773180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 -- 172.21.15.105:0/2955201198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6f34a47d0 con 0x55d6f3762420 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f6563fff640 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f376bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65637fe640 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f376c140 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f65580bba10 tx=0x7f6558003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2da44ea445f6478 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.204+0000 7f65437fe640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:49.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f65437fe640 1 -- 172.21.15.105:0/2955201198 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580c5ed0 con 0x55d6f3762420 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f65437fe640 1 -- 172.21.15.105:0/2955201198 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f65580bb560 con 0x55d6f3762420 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 1 -- 172.21.15.105:0/2955201198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65580bd4c0 con 0x55d6f3762420 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f65437fe640 1 -- 172.21.15.105:0/2955201198 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f65580bdbc0 con 0x55d6f3762420 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 10 client.24510 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 1 -- 172.21.15.105:0/2955201198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d6f376a300 con 0x55d6f3762420 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f65437fe640 1 -- 172.21.15.105:0/2955201198 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65580bd4c0 con 0x55d6f3762420 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f65437fe640 1 -- 172.21.15.105:0/2955201198 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f655803c030 con 0x55d6f3762420 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 10 client.24510 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 10 client.24510 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 20 client.24510 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55d6f3777a10 0x55d6f3779e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 4 client.24510 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f656cffc740 1 -- 172.21.15.105:0/2955201198 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6f36d3eb0 con 0x55d6f3777a10 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.205+0000 7f6563fff640 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55d6f3777a10 0x55d6f3779e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.206+0000 7f6563fff640 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55d6f3777a10 0x55d6f3779e50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6b42441bea3adcd server_cookie=98ecb591d5589a9c in_seq=0 out_seq=0 2024-03-26T18:03:49.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.206+0000 7f65437fe640 10 client.24510 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.206+0000 7f65437fe640 1 -- 172.21.15.105:0/2955201198 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d6f36d3eb0 con 0x55d6f3777a10 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.206+0000 7f65437fe640 10 client.24510 handle_command_reply: tid=0 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.206+0000 7f656cffc740 1 client.24510 shutdown 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 -- 172.21.15.105:0/2955201198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 msgr2=0x55d6f376c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f376c140 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f65580bba10 tx=0x7f6558003d40 comp rx=0 tx=0).stop 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 -- 172.21.15.105:0/2955201198 shutdown_connections 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55d6f3777a10 0x55d6f3779e50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3767590 0x55d6f3773180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f65437fe640 0 client.24510 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f376c140 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 --2- 172.21.15.105:0/2955201198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f376bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 -- 172.21.15.105:0/2955201198 >> 172.21.15.105:0/2955201198 conn(0x55d6f375cfa0 msgr2=0x55d6f375c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 -- 172.21.15.105:0/2955201198 shutdown_connections 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 1 -- 172.21.15.105:0/2955201198 wait complete. 2024-03-26T18:03:49.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.207+0000 7f656cffc740 20 client.24510 trim_cache size 0 max 16384 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f656cffc740 1 Processor -- start 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f656cffc740 1 -- start start 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f656cffc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3761a30 0x55d6f3779e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f656cffc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3773140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f656cffc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3767590 0x55d6f375b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f656cffc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65641a8dd0 con 0x55d6f3762420 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f656cffc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65641aa970 con 0x55d6f3761a30 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f656cffc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65641a94d0 con 0x55d6f3767590 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3773140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3773140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49732/0 (socket says 172.21.15.105:49732) 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 -- 172.21.15.105:0/689547570 learned_addr learned my addr 172.21.15.105:0/689547570 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f6563fff640 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3761a30 0x55d6f3779e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65688d4640 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3767590 0x55d6f375b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 -- 172.21.15.105:0/689547570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3767590 msgr2=0x55d6f375b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3767590 0x55d6f375b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 -- 172.21.15.105:0/689547570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3761a30 msgr2=0x55d6f3779e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3761a30 0x55d6f3779e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 -- 172.21.15.105:0/689547570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6f3758a10 con 0x55d6f3762420 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.220+0000 7f65637fe640 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3773140 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f65580b9920 tx=0x7f65580bd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b21561f06f2383 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f6549ffb640 1 -- 172.21.15.105:0/689547570 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580bb830 con 0x55d6f3762420 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f6549ffb640 1 -- 172.21.15.105:0/689547570 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65580c5ed0 con 0x55d6f3762420 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f6549ffb640 1 -- 172.21.15.105:0/689547570 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580d65f0 con 0x55d6f3762420 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 -- 172.21.15.105:0/689547570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 msgr2=0x55d6f3773140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3773140 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f65580b9920 tx=0x7f65580bd000 comp rx=0 tx=0).stop 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 -- 172.21.15.105:0/689547570 shutdown_connections 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3767590 0x55d6f375b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3762420 0x55d6f3773140 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 --2- 172.21.15.105:0/689547570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3761a30 0x55d6f3779e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 -- 172.21.15.105:0/689547570 >> 172.21.15.105:0/689547570 conn(0x55d6f375cfa0 msgr2=0x55d6f3762b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 -- 172.21.15.105:0/689547570 shutdown_connections 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 -- 172.21.15.105:0/689547570 wait complete. 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.221+0000 7f656cffc740 1 -- start start 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f656cffc740 1 Processor -- start 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f656cffc740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3770370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f656cffc740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3762420 0x55d6f37708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f656cffc740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3767590 0x55d6f37718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f656cffc740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65641aa2a0 con 0x55d6f3767590 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f656cffc740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f65641a9bd0 con 0x55d6f3762420 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f656cffc740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65641a6db0 con 0x55d6f3761a30 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f656cffc740 10 client.0 _ll_register_callbacks cb 0x7f655c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f6563fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3770370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f65688d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3767590 0x55d6f37718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f6563fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3770370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35850/0 (socket says 172.21.15.105:35850) 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f6563fff640 1 -- 172.21.15.105:0/2143627473 learned_addr learned my addr 172.21.15.105:0/2143627473 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f65637fe640 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3762420 0x55d6f37708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f6563fff640 1 -- 172.21.15.105:0/2143627473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3762420 msgr2=0x55d6f37708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f6563fff640 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3762420 0x55d6f37708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f6563fff640 1 -- 172.21.15.105:0/2143627473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3767590 msgr2=0x55d6f37718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f6563fff640 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3767590 0x55d6f37718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f6563fff640 1 -- 172.21.15.105:0/2143627473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6f3771eb0 con 0x55d6f3761a30 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.222+0000 7f65688d4640 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3767590 0x55d6f37718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f6563fff640 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3770370 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6554009a70 tx=0x7f6554004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7f08eef44b1b031 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f65437fe640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f65437fe640 1 -- 172.21.15.105:0/2143627473 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655402c440 con 0x55d6f3761a30 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f65437fe640 1 -- 172.21.15.105:0/2143627473 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f6554009050 con 0x55d6f3761a30 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 1 -- 172.21.15.105:0/2143627473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65580c5ed0 con 0x55d6f3761a30 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f65437fe640 1 -- 172.21.15.105:0/2143627473 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f655403b050 con 0x55d6f3761a30 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 10 client.34177 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 1 -- 172.21.15.105:0/2143627473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d6f3773280 con 0x55d6f3761a30 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f65437fe640 1 -- 172.21.15.105:0/2143627473 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65580c5ed0 con 0x55d6f3761a30 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f65437fe640 1 -- 172.21.15.105:0/2143627473 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f6554028410 con 0x55d6f3761a30 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 10 client.34177 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 10 client.34177 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 20 client.34177 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55d6f37de350 0x55d6f37e0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 4 client.34177 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f656cffc740 1 -- 172.21.15.105:0/2143627473 --> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6f36ce200 con 0x55d6f37de350 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.223+0000 7f65637fe640 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55d6f37de350 0x55d6f37e0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f65437fe640 10 client.34177 ms_handle_connect on v2:172.21.15.144:6836/184508646 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f65637fe640 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55d6f37de350 0x55d6f37e0740 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e9cc06857c53f50b server_cookie=7c11d6d3fe185cca in_seq=0 out_seq=0 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f65437fe640 1 -- 172.21.15.105:0/2143627473 <== mds.3 v2:172.21.15.144:6836/184508646 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6f36ce200 con 0x55d6f37de350 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f65437fe640 10 client.34177 handle_command_reply: tid=0 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 client.34177 shutdown 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 -- 172.21.15.105:0/2143627473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 msgr2=0x55d6f3770370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3770370 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6554009a70 tx=0x7f6554004730 comp rx=0 tx=0).stop 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 -- 172.21.15.105:0/2143627473 shutdown_connections 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.144:6836/184508646,v1:172.21.15.144:6837/184508646] conn(0x55d6f37de350 0x55d6f37e0740 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d6f3767590 0x55d6f37718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d6f3762420 0x55d6f37708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 --2- 172.21.15.105:0/2143627473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d6f3761a30 0x55d6f3770370 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f65437fe640 0 client.34177 ms_handle_reset on v2:172.21.15.144:6836/184508646 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 -- 172.21.15.105:0/2143627473 >> 172.21.15.105:0/2143627473 conn(0x55d6f375cfa0 msgr2=0x55d6f3760200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 -- 172.21.15.105:0/2143627473 shutdown_connections 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 1 -- 172.21.15.105:0/2143627473 wait complete. 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.224+0000 7f656cffc740 20 client.34177 trim_cache size 0 max 16384 2024-03-26T18:03:49.210 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 -- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f654c0a2f20 msgr2=0x7f654c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f654c0a2f20 0x7f654c0a53e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6554009bb0 tx=0x7f6554005f90 comp rx=0 tx=0).stop 2024-03-26T18:03:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 -- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6564104d30 msgr2=0x7f65641980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6564104d30 0x7f65641980b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f6558005940 tx=0x7f65580042f0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 -- 172.21.15.105:0/4124996942 shutdown_connections 2024-03-26T18:03:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f654c0a2f20 0x7f654c0a53e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6564107660 0x7f656419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6564104d30 0x7f65641980b0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 --2- 172.21.15.105:0/4124996942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6564102400 0x7f6564197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 -- 172.21.15.105:0/4124996942 >> 172.21.15.105:0/4124996942 conn(0x7f65640fbd70 msgr2=0x7f6564102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 -- 172.21.15.105:0/4124996942 shutdown_connections 2024-03-26T18:03:49.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.225+0000 7f656a337640 1 -- 172.21.15.105:0/4124996942 wait complete. 2024-03-26T18:03:49.214 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:49 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2172967258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:03:49.216 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T18:03:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 -- 172.21.15.105:0/1321418319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2874102ad0 msgr2=0x7f2874102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 --2- 172.21.15.105:0/1321418319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2874102ad0 0x7f2874102f50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f285c009900 tx=0x7f285c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 -- 172.21.15.105:0/1321418319 shutdown_connections 2024-03-26T18:03:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 --2- 172.21.15.105:0/1321418319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2874103490 0x7f2874109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 --2- 172.21.15.105:0/1321418319 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2874102ad0 0x7f2874102f50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 --2- 172.21.15.105:0/1321418319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28741018e0 0x7f2874101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 -- 172.21.15.105:0/1321418319 >> 172.21.15.105:0/1321418319 conn(0x7f28740fce50 msgr2=0x7f28740ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 -- 172.21.15.105:0/1321418319 shutdown_connections 2024-03-26T18:03:49.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.354+0000 7f2879a92640 1 -- 172.21.15.105:0/1321418319 wait complete. 2024-03-26T18:03:49.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2879a92640 1 Processor -- start 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2879a92640 1 -- start start 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2879a92640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28741018e0 0x7f2874197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2879a92640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2874102ad0 0x7f2874198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2879a92640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2874103490 0x7f287419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2879a92640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f287410d410 con 0x7f2874102ad0 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2879a92640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f287410d290 con 0x7f28741018e0 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2879a92640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f287410d590 con 0x7f2874103490 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2874102ad0 0x7f2874198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f28737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28741018e0 0x7f2874197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2874102ad0 0x7f2874198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49766/0 (socket says 172.21.15.105:49766) 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 -- 172.21.15.105:0/2370692426 learned_addr learned my addr 172.21.15.105:0/2370692426 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f2873fff640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2874103490 0x7f287419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 -- 172.21.15.105:0/2370692426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2874103490 msgr2=0x7f287419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2874103490 0x7f287419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 -- 172.21.15.105:0/2370692426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28741018e0 msgr2=0x7f2874197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28741018e0 0x7f2874197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 -- 172.21.15.105:0/2370692426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287419f6f0 con 0x7f2874102ad0 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.355+0000 7f286bfff640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2874102ad0 0x7f2874198010 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f285c006010 tx=0x7f285c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.356+0000 7f28717fa640 1 -- 172.21.15.105:0/2370692426 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c043020 con 0x7f2874102ad0 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.356+0000 7f2879a92640 1 -- 172.21.15.105:0/2370692426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287419f980 con 0x7f2874102ad0 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.356+0000 7f28717fa640 1 -- 172.21.15.105:0/2370692426 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f285c0043d0 con 0x7f2874102ad0 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.356+0000 7f28717fa640 1 -- 172.21.15.105:0/2370692426 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c003b80 con 0x7f2874102ad0 2024-03-26T18:03:49.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.356+0000 7f2879a92640 1 -- 172.21.15.105:0/2370692426 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287419ff20 con 0x7f2874102ad0 2024-03-26T18:03:49.339 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:49 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2172967258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f287c67d740 1 Processor -- start 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f28717fa640 1 -- 172.21.15.105:0/2370692426 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f285c046020 con 0x7f2874102ad0 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f287c67d740 1 -- start start 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f287c67d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c0649a30 0x55b0c0649e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f287c67d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c064a420 0x55b0c064ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f287c67d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c0651a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f287c67d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b0c05ae970 con 0x55b0c064f590 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f287c67d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0556d70 con 0x55b0c064a420 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.358+0000 7f287c67d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0c05aa920 con 0x55b0c0649a30 2024-03-26T18:03:49.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f286bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c0651a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c0649a30 0x55b0c0649e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c0649a30 0x55b0c0649e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35864/0 (socket says 172.21.15.105:35864) 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f28737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c064a420 0x55b0c064ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 -- 172.21.15.105:0/3400724795 learned_addr learned my addr 172.21.15.105:0/3400724795 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 -- 172.21.15.105:0/3400724795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c064a420 msgr2=0x55b0c064ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c064a420 0x55b0c064ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 -- 172.21.15.105:0/3400724795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 msgr2=0x55b0c0651a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c0651a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 -- 172.21.15.105:0/3400724795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0c02c23f0 con 0x55b0c0649a30 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f28737fe640 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c064a420 0x55b0c064ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f286bfff640 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c0651a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f28717fa640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28600a2f20 0x7f28600a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2873fff640 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c0649a30 0x55b0c0649e50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f286c009fd0 tx=0x7f286c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=403f865d141fb662 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f28737fe640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28600a2f20 0x7f28600a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f28717fa640 1 -- 172.21.15.105:0/2370692426 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f285c0f1650 con 0x7f2874102ad0 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2869ffb640 1 -- 172.21.15.105:0/3400724795 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c004d20 con 0x55b0c0649a30 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2869ffb640 1 -- 172.21.15.105:0/3400724795 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f286c004050 con 0x55b0c0649a30 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.359+0000 7f2869ffb640 1 -- 172.21.15.105:0/3400724795 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c005280 con 0x55b0c0649a30 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 -- 172.21.15.105:0/3400724795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c0649a30 msgr2=0x55b0c0649e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f28737fe640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28600a2f20 0x7f28600a53e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55b0c064a820 tx=0x7f2864005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c0649a30 0x55b0c0649e50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f286c009fd0 tx=0x7f286c0118d0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 -- 172.21.15.105:0/3400724795 shutdown_connections 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c0651a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c064a420 0x55b0c064ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 --2- 172.21.15.105:0/3400724795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c0649a30 0x55b0c0649e50 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 -- 172.21.15.105:0/3400724795 >> 172.21.15.105:0/3400724795 conn(0x55b0c0644fa0 msgr2=0x55b0c06453a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 -- 172.21.15.105:0/3400724795 shutdown_connections 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 -- 172.21.15.105:0/3400724795 wait complete. 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.360+0000 7f287c67d740 1 -- start start 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f287c67d740 1 Processor -- start 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f287c67d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0653fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f287c67d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 0x55b0c0654510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f287c67d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f287c67d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0652540 con 0x55b0c064f590 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f287c67d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b0c056a310 con 0x55b0c0649a30 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f287c67d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0c06526c0 con 0x55b0c064a420 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f287c67d740 10 client.0 _ll_register_callbacks cb 0x55b0c05a5c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f28737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0653fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2873fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f28737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0653fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54206/0 (socket says 172.21.15.105:54206) 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f286bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 0x55b0c0654510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f28737fe640 1 -- 172.21.15.105:0/238273576 learned_addr learned my addr 172.21.15.105:0/238273576 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2873fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49784/0 (socket says 172.21.15.105:49784) 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2873fff640 1 -- 172.21.15.105:0/238273576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 msgr2=0x55b0c0654510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2873fff640 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 0x55b0c0654510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2873fff640 1 -- 172.21.15.105:0/238273576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 msgr2=0x55b0c0653fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2873fff640 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0653fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2873fff640 1 -- 172.21.15.105:0/238273576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c038c7d0 con 0x55b0c064f590 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2873fff640 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b550 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f286c01b040 tx=0x7f286c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a259366294a0d3b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.361+0000 7f2852ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f2852ffd640 1 -- 172.21.15.105:0/238273576 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c028020 con 0x55b0c064f590 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f2852ffd640 1 -- 172.21.15.105:0/238273576 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f286c004a40 con 0x55b0c064f590 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 1 -- 172.21.15.105:0/238273576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f286c004050 con 0x55b0c064f590 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f2852ffd640 1 -- 172.21.15.105:0/238273576 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f286c033050 con 0x55b0c064f590 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 10 client.24528 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 1 -- 172.21.15.105:0/238273576 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b0c065e8c0 con 0x55b0c064f590 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f2852ffd640 1 -- 172.21.15.105:0/238273576 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f286c004050 con 0x55b0c064f590 2024-03-26T18:03:49.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f2852ffd640 1 -- 172.21.15.105:0/238273576 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f286c0057f0 con 0x55b0c064f590 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 10 client.24528 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 10 client.24528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 20 client.24528 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b0c065fde0 0x55b0c0662210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 4 client.24528 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f287c67d740 1 -- 172.21.15.105:0/238273576 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0c05bbeb0 con 0x55b0c065fde0 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.362+0000 7f28737fe640 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b0c065fde0 0x55b0c0662210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.363+0000 7f2852ffd640 10 client.24528 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.363+0000 7f28737fe640 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b0c065fde0 0x55b0c0662210 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76c02a1ea91f218 server_cookie=28bb37e913bb116e in_seq=0 out_seq=0 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f2852ffd640 1 -- 172.21.15.105:0/238273576 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b0c05bbeb0 con 0x55b0c065fde0 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f2852ffd640 10 client.24528 handle_command_reply: tid=0 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 client.24528 shutdown 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 -- 172.21.15.105:0/238273576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 msgr2=0x55b0c065b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b550 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f286c01b040 tx=0x7f286c004430 comp rx=0 tx=0).stop 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 -- 172.21.15.105:0/238273576 shutdown_connections 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b550 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 0x55b0c0654510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b0c065fde0 0x55b0c0662210 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 --2- 172.21.15.105:0/238273576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0653fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 -- 172.21.15.105:0/238273576 >> 172.21.15.105:0/238273576 conn(0x55b0c0644fa0 msgr2=0x55b0c06441c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f2852ffd640 0 client.24528 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 -- 172.21.15.105:0/238273576 shutdown_connections 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 1 -- 172.21.15.105:0/238273576 wait complete. 2024-03-26T18:03:49.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.364+0000 7f287c67d740 20 client.24528 trim_cache size 0 max 16384 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f287c67d740 1 Processor -- start 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f287c67d740 1 -- start start 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f287c67d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0652a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f287c67d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 0x55b0c0652fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f287c67d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f287c67d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a8da0 con 0x55b0c064f590 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f287c67d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28741aa940 con 0x55b0c0649a30 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f287c67d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a94a0 con 0x55b0c064a420 2024-03-26T18:03:49.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f2873fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f28737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0652a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f2873fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49786/0 (socket says 172.21.15.105:49786) 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f286bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 0x55b0c0652fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f2873fff640 1 -- 172.21.15.105:0/3501280549 learned_addr learned my addr 172.21.15.105:0/3501280549 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.377+0000 7f2873fff640 1 -- 172.21.15.105:0/3501280549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 msgr2=0x55b0c0652fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.378+0000 7f2873fff640 1 --2- 172.21.15.105:0/3501280549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 0x55b0c0652fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.378+0000 7f2873fff640 1 -- 172.21.15.105:0/3501280549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 msgr2=0x55b0c0652a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.378+0000 7f2873fff640 1 --2- 172.21.15.105:0/3501280549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0652a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.378+0000 7f2873fff640 1 -- 172.21.15.105:0/3501280549 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0c0652420 con 0x55b0c064f590 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.378+0000 7f2873fff640 1 --2- 172.21.15.105:0/3501280549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b510 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f286c00f3f0 tx=0x7f286c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf8fe9fb448fed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.378+0000 7f28697fa640 1 -- 172.21.15.105:0/3501280549 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c028040 con 0x55b0c064f590 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.378+0000 7f28697fa640 1 -- 172.21.15.105:0/3501280549 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f286c009b30 con 0x55b0c064f590 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.378+0000 7f28697fa640 1 -- 172.21.15.105:0/3501280549 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c019830 con 0x55b0c064f590 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 -- 172.21.15.105:0/3501280549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 msgr2=0x55b0c065b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 --2- 172.21.15.105:0/3501280549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b510 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f286c00f3f0 tx=0x7f286c003970 comp rx=0 tx=0).stop 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 -- 172.21.15.105:0/3501280549 shutdown_connections 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 --2- 172.21.15.105:0/3501280549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064f590 0x55b0c065b510 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 --2- 172.21.15.105:0/3501280549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064a420 0x55b0c0652fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 --2- 172.21.15.105:0/3501280549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c0652a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 -- 172.21.15.105:0/3501280549 >> 172.21.15.105:0/3501280549 conn(0x55b0c0644fa0 msgr2=0x55b0c064b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 -- 172.21.15.105:0/3501280549 shutdown_connections 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 -- 172.21.15.105:0/3501280549 wait complete. 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 -- start start 2024-03-26T18:03:49.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.379+0000 7f287c67d740 1 Processor -- start 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c065b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064a420 0x55b0c0657c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064f590 0x55b0c0658190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f28741aa270 con 0x55b0c064a420 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a9ba0 con 0x55b0c0649a30 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a6d80 con 0x55b0c064f590 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 10 client.0 _ll_register_callbacks cb 0x7f285c0f7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064a420 0x55b0c0657c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f28737fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c065b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064a420 0x55b0c0657c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49790/0 (socket says 172.21.15.105:49790) 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 -- 172.21.15.105:0/22955064 learned_addr learned my addr 172.21.15.105:0/22955064 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f2873fff640 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064f590 0x55b0c0658190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 -- 172.21.15.105:0/22955064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064f590 msgr2=0x55b0c0658190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064f590 0x55b0c0658190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 -- 172.21.15.105:0/22955064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 msgr2=0x55b0c065b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c065b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 -- 172.21.15.105:0/22955064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c0658750 con 0x55b0c064a420 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f28737fe640 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c065b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f286bfff640 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064a420 0x55b0c0657c50 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f285c0bda40 tx=0x7f285c0ba0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2677910acdc8f74a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f2852ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f2852ffd640 1 -- 172.21.15.105:0/22955064 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0bd6a0 con 0x55b0c064a420 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f2852ffd640 1 -- 172.21.15.105:0/22955064 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f285c0054b0 con 0x55b0c064a420 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 1 -- 172.21.15.105:0/22955064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f286c009b30 con 0x55b0c064a420 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f2852ffd640 1 -- 172.21.15.105:0/22955064 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f285c0056b0 con 0x55b0c064a420 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 10 client.24534 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.380+0000 7f287c67d740 1 -- 172.21.15.105:0/22955064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b0c06c5210 con 0x55b0c064a420 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f2852ffd640 1 -- 172.21.15.105:0/22955064 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f286c009b30 con 0x55b0c064a420 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f2852ffd640 1 -- 172.21.15.105:0/22955064 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f285c0c62c0 con 0x55b0c064a420 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f287c67d740 10 client.24534 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f287c67d740 10 client.24534 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f287c67d740 20 client.24534 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f287c67d740 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b0c06c5c90 0x55b0c06c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f287c67d740 4 client.24534 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f287c67d740 1 -- 172.21.15.105:0/22955064 --> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0c0653dc0 con 0x55b0c06c5c90 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f28737fe640 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b0c06c5c90 0x55b0c06c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f2852ffd640 10 client.24534 ms_handle_connect on v2:172.21.15.135:6836/388686483 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f28737fe640 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b0c06c5c90 0x55b0c06c8080 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a20ab1924ff7bd9b server_cookie=a75079ef0bab44e6 in_seq=0 out_seq=0 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f2852ffd640 1 -- 172.21.15.105:0/22955064 <== mds.2 v2:172.21.15.135:6836/388686483 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0c0653dc0 con 0x55b0c06c5c90 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.381+0000 7f2852ffd640 10 client.24534 handle_command_reply: tid=0 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 client.24534 shutdown 2024-03-26T18:03:49.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 -- 172.21.15.105:0/22955064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064a420 msgr2=0x55b0c0657c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064a420 0x55b0c0657c50 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f285c0bda40 tx=0x7f285c0ba0a0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 -- 172.21.15.105:0/22955064 shutdown_connections 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0c064f590 0x55b0c0658190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55b0c064a420 0x55b0c0657c50 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.135:6836/388686483,v1:172.21.15.135:6837/388686483] conn(0x55b0c06c5c90 0x55b0c06c8080 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 --2- 172.21.15.105:0/22955064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b0c0649a30 0x55b0c065b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 -- 172.21.15.105:0/22955064 >> 172.21.15.105:0/22955064 conn(0x55b0c0644fa0 msgr2=0x55b0c064b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2852ffd640 0 client.24534 ms_handle_reset on v2:172.21.15.135:6836/388686483 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 -- 172.21.15.105:0/22955064 shutdown_connections 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 1 -- 172.21.15.105:0/22955064 wait complete. 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f287c67d740 20 client.24534 trim_cache size 0 max 16384 2024-03-26T18:03:49.368 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:49.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 -- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28600a2f20 msgr2=0x7f28600a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28600a2f20 0x7f28600a53e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55b0c064a820 tx=0x7f2864005f90 comp rx=0 tx=0).stop 2024-03-26T18:03:49.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 -- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2874102ad0 msgr2=0x7f2874198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2874102ad0 0x7f2874198010 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f285c006010 tx=0x7f285c005e40 comp rx=0 tx=0).stop 2024-03-26T18:03:49.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 -- 172.21.15.105:0/2370692426 shutdown_connections 2024-03-26T18:03:49.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f28600a2f20 0x7f28600a53e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2874103490 0x7f287419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2874102ad0 0x7f2874198010 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 --2- 172.21.15.105:0/2370692426 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f28741018e0 0x7f2874197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 -- 172.21.15.105:0/2370692426 >> 172.21.15.105:0/2370692426 conn(0x7f28740fce50 msgr2=0x7f2874105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.382+0000 7f2879a92640 1 -- 172.21.15.105:0/2370692426 shutdown_connections 2024-03-26T18:03:49.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.383+0000 7f2879a92640 1 -- 172.21.15.105:0/2370692426 wait complete. 2024-03-26T18:03:49.373 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T18:03:49.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:49 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2172967258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 -- 172.21.15.105:0/843841518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3804102380 msgr2=0x7f3804108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 --2- 172.21.15.105:0/843841518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3804102380 0x7f3804108c10 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009ca0 tx=0x7f37fc02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 -- 172.21.15.105:0/843841518 shutdown_connections 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 --2- 172.21.15.105:0/843841518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3804102380 0x7f3804108c10 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 --2- 172.21.15.105:0/843841518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38041019c0 0x7f3804101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 --2- 172.21.15.105:0/843841518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38041007d0 0x7f3804100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 -- 172.21.15.105:0/843841518 >> 172.21.15.105:0/843841518 conn(0x7f38040fbd60 msgr2=0x7f38040fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 -- 172.21.15.105:0/843841518 shutdown_connections 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 -- 172.21.15.105:0/843841518 wait complete. 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 Processor -- start 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 -- start start 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38041007d0 0x7f380406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38041019c0 0x7f380419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804102380 0x7f380419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.501+0000 7f380977e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f380410c300 con 0x7f38041007d0 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f380977e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f380410c180 con 0x7f38041019c0 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f380977e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f380410c480 con 0x7f3804102380 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38041007d0 0x7f380406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38041007d0 0x7f380406aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49800/0 (socket says 172.21.15.105:49800) 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 -- 172.21.15.105:0/1146504399 learned_addr learned my addr 172.21.15.105:0/1146504399 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f38037fe640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804102380 0x7f380419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f37fbfff640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38041019c0 0x7f380419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 -- 172.21.15.105:0/1146504399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804102380 msgr2=0x7f380419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804102380 0x7f380419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 -- 172.21.15.105:0/1146504399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38041019c0 msgr2=0x7f380419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38041019c0 0x7f380419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 -- 172.21.15.105:0/1146504399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380419f720 con 0x7f38041007d0 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f38037fe640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804102380 0x7f380419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38041007d0 0x7f380406aa40 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f37f400e9c0 tx=0x7f37f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.484 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3800ff9640 1 -- 172.21.15.105:0/1146504399 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f400cde0 con 0x7f38041007d0 2024-03-26T18:03:49.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3800ff9640 1 -- 172.21.15.105:0/1146504399 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37f40108e0 con 0x7f38041007d0 2024-03-26T18:03:49.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f380977e640 1 -- 172.21.15.105:0/1146504399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380419fa10 con 0x7f38041007d0 2024-03-26T18:03:49.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f3800ff9640 1 -- 172.21.15.105:0/1146504399 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f40188e0 con 0x7f38041007d0 2024-03-26T18:03:49.485 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.502+0000 7f380977e640 1 -- 172.21.15.105:0/1146504399 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380419ff60 con 0x7f38041007d0 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3800ff9640 1 -- 172.21.15.105:0/1146504399 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f37f401f020 con 0x7f38041007d0 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f380c36a740 1 Processor -- start 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f380c36a740 1 -- start start 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f380c36a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a60e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f380c36a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a61420 0x558779a65f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f380c36a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a66590 0x558779a68a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f380c36a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5587799c5970 con 0x558779a60a30 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f380c36a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55877996dd70 con 0x558779a66590 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f380c36a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5587799c1920 con 0x558779a61420 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a61420 0x558779a65f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f38037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a60e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a61420 0x558779a65f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35912/0 (socket says 172.21.15.105:35912) 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f38037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a60e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49810/0 (socket says 172.21.15.105:49810) 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f37fbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a66590 0x558779a68a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3802ffd640 1 -- 172.21.15.105:0/1567325610 learned_addr learned my addr 172.21.15.105:0/1567325610 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3800ff9640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f37f00a2f20 0x7f37f00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3802ffd640 1 -- 172.21.15.105:0/1567325610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a66590 msgr2=0x558779a68a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1567325610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a66590 0x558779a68a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3802ffd640 1 -- 172.21.15.105:0/1567325610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 msgr2=0x558779a60e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1567325610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a60e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f3802ffd640 1 -- 172.21.15.105:0/1567325610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587796d93f0 con 0x558779a61420 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.505+0000 7f38037fe640 1 --2- 172.21.15.105:0/1567325610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a60e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.506+0000 7f38037fe640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f37f00a2f20 0x7f37f00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.506+0000 7f3800ff9640 1 -- 172.21.15.105:0/1146504399 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f37f40cf5b0 con 0x7f38041007d0 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.506+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1567325610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a61420 0x558779a65f20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f37f4051ba0 tx=0x7f37f4022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a9c894d0b82d6ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.506+0000 7f37f97fa640 1 -- 172.21.15.105:0/1567325610 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f409b600 con 0x558779a61420 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.506+0000 7f38037fe640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f37f00a2f20 0x7f37f00a53e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f37fc006010 tx=0x7f37fc02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.506+0000 7f37f97fa640 1 -- 172.21.15.105:0/1567325610 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37f4052a50 con 0x558779a61420 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.506+0000 7f37f97fa640 1 -- 172.21.15.105:0/1567325610 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4014d70 con 0x558779a61420 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- 172.21.15.105:0/1567325610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a61420 msgr2=0x558779a65f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 --2- 172.21.15.105:0/1567325610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a61420 0x558779a65f20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f37f4051ba0 tx=0x7f37f4022000 comp rx=0 tx=0).stop 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- 172.21.15.105:0/1567325610 shutdown_connections 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 --2- 172.21.15.105:0/1567325610 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a66590 0x558779a68a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 --2- 172.21.15.105:0/1567325610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a61420 0x558779a65f20 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 --2- 172.21.15.105:0/1567325610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a60e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- 172.21.15.105:0/1567325610 >> 172.21.15.105:0/1567325610 conn(0x558779a5bfa0 msgr2=0x558779a5c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- 172.21.15.105:0/1567325610 shutdown_connections 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- 172.21.15.105:0/1567325610 wait complete. 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- start start 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 Processor -- start 2024-03-26T18:03:49.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a6ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 0x558779a6b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a66590 0x558779a72400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558779a694e0 con 0x558779a66590 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558779981310 con 0x558779a61420 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558779a69660 con 0x558779a60a30 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.507+0000 7f380c36a740 10 client.0 _ll_register_callbacks cb 0x5587799bcc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a6ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f38037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a66590 0x558779a72400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a6ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:35924/0 (socket says 172.21.15.105:35924) 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 -- 172.21.15.105:0/2661461832 learned_addr learned my addr 172.21.15.105:0/2661461832 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f37fbfff640 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 0x558779a6b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 -- 172.21.15.105:0/2661461832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 msgr2=0x558779a6b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 0x558779a6b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 -- 172.21.15.105:0/2661461832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a66590 msgr2=0x558779a72400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a66590 0x558779a72400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 -- 172.21.15.105:0/2661461832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587797a37d0 con 0x558779a60a30 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f3802ffd640 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a6ae80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f37f40cd2d0 tx=0x7f37f409b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23e1127c8b85b597 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f37ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.144:3300/0 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f37ea7fc640 1 -- 172.21.15.105:0/2661461832 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f409b600 con 0x558779a60a30 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f380c36a740 1 -- 172.21.15.105:0/2661461832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587799d2eb0 con 0x558779a60a30 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f37ea7fc640 1 -- 172.21.15.105:0/2661461832 <== mon.1 v2:172.21.15.144:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f37f40affb0 con 0x558779a60a30 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f37ea7fc640 1 -- 172.21.15.105:0/2661461832 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f37f40b9050 con 0x558779a60a30 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f380c36a740 10 client.34207 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f380c36a740 1 -- 172.21.15.105:0/2661461832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558779a695a0 con 0x558779a60a30 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.508+0000 7f37ea7fc640 1 -- 172.21.15.105:0/2661461832 <== mon.1 v2:172.21.15.144:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5587799d2eb0 con 0x558779a60a30 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f37ea7fc640 1 -- 172.21.15.105:0/2661461832 <== mon.1 v2:172.21.15.144:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f37f40b1020 con 0x558779a60a30 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f380c36a740 10 client.34207 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f380c36a740 10 client.34207 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f380c36a740 20 client.34207 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f380c36a740 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558779a769a0 0x558779a78dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f380c36a740 4 client.34207 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f380c36a740 1 -- 172.21.15.105:0/2661461832 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587799cd200 con 0x558779a769a0 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f37fbfff640 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558779a769a0 0x558779a78dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f37ea7fc640 10 client.34207 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.509+0000 7f37fbfff640 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558779a769a0 0x558779a78dd0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95589563000f3c5a server_cookie=8d5cddc36255818f in_seq=0 out_seq=0 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f37ea7fc640 1 -- 172.21.15.105:0/2661461832 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5587799cd200 con 0x558779a769a0 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f37ea7fc640 10 client.34207 handle_command_reply: tid=0 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 client.34207 shutdown 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 -- 172.21.15.105:0/2661461832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 msgr2=0x558779a6ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a6ae80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f37f40cd2d0 tx=0x7f37f409b860 comp rx=0 tx=0).stop 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 -- 172.21.15.105:0/2661461832 shutdown_connections 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558779a769a0 0x558779a78dd0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a66590 0x558779a72400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f37ea7fc640 0 client.34207 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:03:49.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 0x558779a6b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 --2- 172.21.15.105:0/2661461832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a6ae80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 -- 172.21.15.105:0/2661461832 >> 172.21.15.105:0/2661461832 conn(0x558779a5bfa0 msgr2=0x558779a5b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 -- 172.21.15.105:0/2661461832 shutdown_connections 2024-03-26T18:03:49.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 1 -- 172.21.15.105:0/2661461832 wait complete. 2024-03-26T18:03:49.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.510+0000 7f380c36a740 20 client.34207 trim_cache size 0 max 16384 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.523+0000 7f380c36a740 1 Processor -- start 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.523+0000 7f380c36a740 1 -- start start 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f380c36a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a5bfa0 0x558779a6b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f380c36a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a72390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f380c36a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 0x558779a69a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f380c36a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f38041a6c90 con 0x558779a60a30 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f380c36a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f38041a8830 con 0x558779a61420 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f380c36a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38041a7390 con 0x558779a5bfa0 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a72390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a5bfa0 0x558779a6b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a72390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49824/0 (socket says 172.21.15.105:49824) 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 -- 172.21.15.105:0/3745347983 learned_addr learned my addr 172.21.15.105:0/3745347983 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f38037fe640 1 --2- 172.21.15.105:0/3745347983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 0x558779a69a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 -- 172.21.15.105:0/3745347983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a5bfa0 msgr2=0x558779a6b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 --2- 172.21.15.105:0/3745347983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a5bfa0 0x558779a6b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 -- 172.21.15.105:0/3745347983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 msgr2=0x558779a69a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 --2- 172.21.15.105:0/3745347983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 0x558779a69a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 -- 172.21.15.105:0/3745347983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558779a69770 con 0x558779a60a30 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37fbfff640 1 --2- 172.21.15.105:0/3745347983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a72390 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x558779a5cf10 tx=0x7f37ec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3388612caf269485 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37f8ff9640 1 -- 172.21.15.105:0/3745347983 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec01f050 con 0x558779a60a30 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37f8ff9640 1 -- 172.21.15.105:0/3745347983 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37ec01f6a0 con 0x558779a60a30 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.524+0000 7f37f8ff9640 1 -- 172.21.15.105:0/3745347983 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec010b80 con 0x558779a60a30 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 -- 172.21.15.105:0/3745347983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 msgr2=0x558779a72390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 --2- 172.21.15.105:0/3745347983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a72390 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x558779a5cf10 tx=0x7f37ec005f90 comp rx=0 tx=0).stop 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 -- 172.21.15.105:0/3745347983 shutdown_connections 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 --2- 172.21.15.105:0/3745347983 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a61420 0x558779a69a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 --2- 172.21.15.105:0/3745347983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a60a30 0x558779a72390 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 --2- 172.21.15.105:0/3745347983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a5bfa0 0x558779a6b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 -- 172.21.15.105:0/3745347983 >> 172.21.15.105:0/3745347983 conn(0x558779a62200 msgr2=0x558779a5f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 -- 172.21.15.105:0/3745347983 shutdown_connections 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 -- 172.21.15.105:0/3745347983 wait complete. 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 -- start start 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.525+0000 7f380c36a740 1 Processor -- start 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f380c36a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a5bfa0 0x558779a6ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f380c36a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f380c36a740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a61420 0x558779a714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f380c36a740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f38041a8160 con 0x558779a61420 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f380c36a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f38041a7a90 con 0x558779a5bfa0 2024-03-26T18:03:49.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f380c36a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38041a4c70 con 0x558779a60a30 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f380c36a740 10 client.0 _ll_register_callbacks cb 0x7f37f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a61420 0x558779a714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a5bfa0 0x558779a6ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f37fbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a61420 0x558779a714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49830/0 (socket says 172.21.15.105:49830) 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 -- 172.21.15.105:0/1918993323 learned_addr learned my addr 172.21.15.105:0/1918993323 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f3802ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a5bfa0 0x558779a6ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54272/0 (socket says 172.21.15.105:54272) 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 -- 172.21.15.105:0/1918993323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 msgr2=0x558779a704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 -- 172.21.15.105:0/1918993323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a5bfa0 msgr2=0x558779a6ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a5bfa0 0x558779a6ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 -- 172.21.15.105:0/1918993323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558779a71ab0 con 0x558779a61420 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a5bfa0 0x558779a6ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f37fbfff640 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:49.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f38037fe640 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a61420 0x558779a714c0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009ca0 tx=0x7f37fc0324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68effe9d24d2e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f37ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f37ea7fc640 1 -- 172.21.15.105:0/1918993323 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc05a060 con 0x558779a61420 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f380c36a740 1 -- 172.21.15.105:0/1918993323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ec01f6a0 con 0x558779a61420 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.526+0000 7f37ea7fc640 1 -- 172.21.15.105:0/1918993323 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f37fc003d80 con 0x558779a61420 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f37ea7fc640 1 -- 172.21.15.105:0/1918993323 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f37fc039040 con 0x558779a61420 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f380c36a740 10 client.24552 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f380c36a740 1 -- 172.21.15.105:0/1918993323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558779add180 con 0x558779a61420 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f37ea7fc640 1 -- 172.21.15.105:0/1918993323 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37ec01f6a0 con 0x558779a61420 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f37ea7fc640 1 -- 172.21.15.105:0/1918993323 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f37fc034670 con 0x558779a61420 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f380c36a740 10 client.24552 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f380c36a740 10 client.24552 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f380c36a740 20 client.24552 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f380c36a740 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558779a66590 0x558779adfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f380c36a740 4 client.24552 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:49.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f380c36a740 1 -- 172.21.15.105:0/1918993323 --> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55877994a6a0 con 0x558779a66590 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558779a66590 0x558779adfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f37ea7fc640 10 client.24552 ms_handle_connect on v2:172.21.15.144:6838/1175380833 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.527+0000 7f3802ffd640 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558779a66590 0x558779adfbd0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44ccc44b2ba46bd9 server_cookie=ccb31ae2d28b512 in_seq=0 out_seq=0 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f37ea7fc640 1 -- 172.21.15.105:0/1918993323 <== mds.1 v2:172.21.15.144:6838/1175380833 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55877994a6a0 con 0x558779a66590 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f37ea7fc640 10 client.24552 handle_command_reply: tid=0 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 client.24552 shutdown 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 -- 172.21.15.105:0/1918993323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a61420 msgr2=0x558779a714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a61420 0x558779a714c0 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009ca0 tx=0x7f37fc0324f0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 -- 172.21.15.105:0/1918993323 shutdown_connections 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.144:6838/1175380833,v1:172.21.15.144:6839/1175380833] conn(0x558779a66590 0x558779adfbd0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558779a61420 0x558779a714c0 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558779a60a30 0x558779a704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 --2- 172.21.15.105:0/1918993323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558779a5bfa0 0x558779a6ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 -- 172.21.15.105:0/1918993323 >> 172.21.15.105:0/1918993323 conn(0x558779a62200 msgr2=0x558779a6b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f37ea7fc640 0 client.24552 ms_handle_reset on v2:172.21.15.144:6838/1175380833 2024-03-26T18:03:49.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 -- 172.21.15.105:0/1918993323 shutdown_connections 2024-03-26T18:03:49.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 1 -- 172.21.15.105:0/1918993323 wait complete. 2024-03-26T18:03:49.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380c36a740 20 client.24552 trim_cache size 0 max 16384 2024-03-26T18:03:49.515 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 -- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f37f00a2f20 msgr2=0x7f37f00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f37f00a2f20 0x7f37f00a53e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f37fc006010 tx=0x7f37fc02e040 comp rx=0 tx=0).stop 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 -- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38041007d0 msgr2=0x7f380406aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38041007d0 0x7f380406aa40 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f37f400e9c0 tx=0x7f37f400ee80 comp rx=0 tx=0).stop 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 -- 172.21.15.105:0/1146504399 shutdown_connections 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f37f00a2f20 0x7f37f00a53e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3804102380 0x7f380419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38041019c0 0x7f380419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 --2- 172.21.15.105:0/1146504399 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38041007d0 0x7f380406aa40 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.528+0000 7f380977e640 1 -- 172.21.15.105:0/1146504399 >> 172.21.15.105:0/1146504399 conn(0x7f38040fbd60 msgr2=0x7f3804104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.529+0000 7f380977e640 1 -- 172.21.15.105:0/1146504399 shutdown_connections 2024-03-26T18:03:49.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.529+0000 7f380977e640 1 -- 172.21.15.105:0/1146504399 wait complete. 2024-03-26T18:03:49.520 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T18:03:49.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 -- 172.21.15.105:0/1878746194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 msgr2=0x7f3190109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 --2- 172.21.15.105:0/1878746194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 0x7f3190109d10 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f3188009ca0 tx=0x7f318802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 -- 172.21.15.105:0/1878746194 shutdown_connections 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 --2- 172.21.15.105:0/1878746194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 0x7f3190109d10 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 --2- 172.21.15.105:0/1878746194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3190102ac0 0x7f3190102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 --2- 172.21.15.105:0/1878746194 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f31901018d0 0x7f3190101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 -- 172.21.15.105:0/1878746194 >> 172.21.15.105:0/1878746194 conn(0x7f31900fce60 msgr2=0x7f31900ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 -- 172.21.15.105:0/1878746194 shutdown_connections 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.647+0000 7f319571b640 1 -- 172.21.15.105:0/1878746194 wait complete. 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f319571b640 1 Processor -- start 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f319571b640 1 -- start start 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f319571b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31901018d0 0x7f3190075670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f319571b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3190102ac0 0x7f3190073cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f319571b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 0x7f31900741f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f319571b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f319010d400 con 0x7f3190103480 2024-03-26T18:03:49.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f319571b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f319010d280 con 0x7f3190102ac0 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f319571b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f319010d580 con 0x7f31901018d0 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f318f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 0x7f31900741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f318f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 0x7f31900741f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49852/0 (socket says 172.21.15.105:49852) 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f318f7fe640 1 -- 172.21.15.105:0/2041881466 learned_addr learned my addr 172.21.15.105:0/2041881466 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f318effd640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31901018d0 0x7f3190075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.648+0000 7f31867fc640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3190102ac0 0x7f3190073cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318f7fe640 1 -- 172.21.15.105:0/2041881466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31901018d0 msgr2=0x7f3190075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318f7fe640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31901018d0 0x7f3190075670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318f7fe640 1 -- 172.21.15.105:0/2041881466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3190102ac0 msgr2=0x7f3190073cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318f7fe640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3190102ac0 0x7f3190073cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318f7fe640 1 -- 172.21.15.105:0/2041881466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3190074a80 con 0x7f3190103480 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318f7fe640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 0x7f31900741f0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f318802c700 tx=0x7f31880328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318cff9640 1 -- 172.21.15.105:0/2041881466 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188041020 con 0x7f3190103480 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318cff9640 1 -- 172.21.15.105:0/2041881466 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3188003e20 con 0x7f3190103480 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f319571b640 1 -- 172.21.15.105:0/2041881466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31901a1b70 con 0x7f3190103480 2024-03-26T18:03:49.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f318cff9640 1 -- 172.21.15.105:0/2041881466 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188039040 con 0x7f3190103480 2024-03-26T18:03:49.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.649+0000 7f319571b640 1 -- 172.21.15.105:0/2041881466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31901a2110 con 0x7f3190103480 2024-03-26T18:03:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.651+0000 7f318cff9640 1 -- 172.21.15.105:0/2041881466 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3188044020 con 0x7f3190103480 2024-03-26T18:03:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31983e8740 1 Processor -- start 2024-03-26T18:03:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31983e8740 1 -- start start 2024-03-26T18:03:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31983e8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258947e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31983e8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258948420 0x56525894cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31983e8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56525894d590 0x56525894fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31983e8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5652588ac970 con 0x565258947a30 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31983e8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565258854d70 con 0x565258948420 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31983e8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5652588a8920 con 0x56525894d590 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f318cff9640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f317c0a2fa0 0x7f317c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258947e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258947e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49864/0 (socket says 172.21.15.105:49864) 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f31867fc640 1 -- 172.21.15.105:0/1613378356 learned_addr learned my addr 172.21.15.105:0/1613378356 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f318f7fe640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f317c0a2fa0 0x7f317c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.652+0000 7f318effd640 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258948420 0x56525894cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f31867fc640 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56525894d590 0x56525894fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f318cff9640 1 -- 172.21.15.105:0/2041881466 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f31880f1520 con 0x7f3190103480 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f318effd640 1 -- 172.21.15.105:0/1613378356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56525894d590 msgr2=0x56525894fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f318effd640 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56525894d590 0x56525894fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f318effd640 1 -- 172.21.15.105:0/1613378356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 msgr2=0x565258947e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f318effd640 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258947e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f318effd640 1 -- 172.21.15.105:0/1613378356 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652585c03f0 con 0x565258948420 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f318f7fe640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f317c0a2fa0 0x7f317c0a5460 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f31880ef350 tx=0x7f3188088760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f318effd640 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258948420 0x56525894cf20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3180009bf0 tx=0x7f3180011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880abc0ad156119f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f31857fa640 1 -- 172.21.15.105:0/1613378356 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180009da0 con 0x565258948420 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f31857fa640 1 -- 172.21.15.105:0/1613378356 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3180005020 con 0x565258948420 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.653+0000 7f31857fa640 1 -- 172.21.15.105:0/1613378356 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31800049e0 con 0x565258948420 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 -- 172.21.15.105:0/1613378356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258948420 msgr2=0x56525894cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258948420 0x56525894cf20 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3180009bf0 tx=0x7f3180011810 comp rx=0 tx=0).stop 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 -- 172.21.15.105:0/1613378356 shutdown_connections 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56525894d590 0x56525894fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258948420 0x56525894cf20 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 --2- 172.21.15.105:0/1613378356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258947e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 -- 172.21.15.105:0/1613378356 >> 172.21.15.105:0/1613378356 conn(0x565258942fa0 msgr2=0x5652589433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 -- 172.21.15.105:0/1613378356 shutdown_connections 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.654+0000 7f31983e8740 1 -- 172.21.15.105:0/1613378356 wait complete. 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 1 -- start start 2024-03-26T18:03:49.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 1 Processor -- start 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258951f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 0x565258952490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 0x5652589594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565258950540 con 0x565258947a30 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565258868310 con 0x56525894d590 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5652589506c0 con 0x565258948420 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31983e8740 10 client.0 _ll_register_callbacks cb 0x5652588a3c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258951f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258951f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49880/0 (socket says 172.21.15.105:49880) 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 -- 172.21.15.105:0/2044017464 learned_addr learned my addr 172.21.15.105:0/2044017464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f318effd640 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 0x565258952490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 0x5652589594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 -- 172.21.15.105:0/2044017464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 msgr2=0x565258952490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 0x565258952490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 -- 172.21.15.105:0/2044017464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 msgr2=0x5652589594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 0x5652589594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.655+0000 7f31867fc640 1 -- 172.21.15.105:0/2044017464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525868a7d0 con 0x565258947a30 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31867fc640 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258951f50 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f31780038e0 tx=0x7f3178015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690fff230e305d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31767fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31767fc640 1 -- 172.21.15.105:0/2044017464 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178005020 con 0x565258947a30 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31767fc640 1 -- 172.21.15.105:0/2044017464 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f31780051c0 con 0x565258947a30 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 1 -- 172.21.15.105:0/2044017464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3180005020 con 0x565258947a30 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31767fc640 1 -- 172.21.15.105:0/2044017464 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f31780053c0 con 0x565258947a30 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 10 client.24564 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 1 -- 172.21.15.105:0/2044017464 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56525895c8b0 con 0x565258947a30 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31767fc640 1 -- 172.21.15.105:0/2044017464 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3180005020 con 0x565258947a30 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31767fc640 1 -- 172.21.15.105:0/2044017464 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f3178013ce0 con 0x565258947a30 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 10 client.24564 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 10 client.24564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 20 client.24564 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56525895def0 0x565258960390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 4 client.24564 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f31983e8740 1 -- 172.21.15.105:0/2044017464 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652588b9eb0 con 0x56525895def0 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.656+0000 7f318effd640 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56525895def0 0x565258960390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.657+0000 7f318effd640 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56525895def0 0x565258960390 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aa479191d83cd53 server_cookie=3dda642d22650066 in_seq=0 out_seq=0 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.657+0000 7f31767fc640 10 client.24564 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.657+0000 7f31767fc640 1 -- 172.21.15.105:0/2044017464 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5652588b9eb0 con 0x56525895def0 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.657+0000 7f31767fc640 10 client.24564 handle_command_reply: tid=0 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 client.24564 shutdown 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 -- 172.21.15.105:0/2044017464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 msgr2=0x565258951f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258951f50 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f31780038e0 tx=0x7f3178015040 comp rx=0 tx=0).stop 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 -- 172.21.15.105:0/2044017464 shutdown_connections 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56525895def0 0x565258960390 crc :-1 s=READY pgs=630 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 0x5652589594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31767fc640 0 client.24564 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 0x565258952490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 --2- 172.21.15.105:0/2044017464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258947a30 0x565258951f50 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 -- 172.21.15.105:0/2044017464 >> 172.21.15.105:0/2044017464 conn(0x565258942fa0 msgr2=0x5652589421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 -- 172.21.15.105:0/2044017464 shutdown_connections 2024-03-26T18:03:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 1 -- 172.21.15.105:0/2044017464 wait complete. 2024-03-26T18:03:49.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.658+0000 7f31983e8740 20 client.24564 trim_cache size 0 max 16384 2024-03-26T18:03:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.672+0000 7f31983e8740 1 Processor -- start 2024-03-26T18:03:49.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.672+0000 7f31983e8740 1 -- start start 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31983e8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258947a30 0x5652589523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31983e8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 0x565258959490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31983e8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56525894d590 0x565258946910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31983e8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31901aaf30 con 0x56525894d590 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31983e8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31901acad0 con 0x565258947a30 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31983e8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31901ab630 con 0x565258948420 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56525894d590 0x565258946910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56525894d590 0x565258946910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49892/0 (socket says 172.21.15.105:49892) 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 -- 172.21.15.105:0/1888755304 learned_addr learned my addr 172.21.15.105:0/1888755304 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f318effd640 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 0x565258959490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258947a30 0x5652589523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 -- 172.21.15.105:0/1888755304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 msgr2=0x565258959490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 0x565258959490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 -- 172.21.15.105:0/1888755304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258947a30 msgr2=0x5652589523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258947a30 0x5652589523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 -- 172.21.15.105:0/1888755304 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565258942b40 con 0x56525894d590 2024-03-26T18:03:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f31867fc640 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56525894d590 0x565258946910 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f3178013aa0 tx=0x7f3178009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4016deb23abbc59 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f3184ff9640 1 -- 172.21.15.105:0/1888755304 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178005610 con 0x56525894d590 2024-03-26T18:03:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f3184ff9640 1 -- 172.21.15.105:0/1888755304 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3178032070 con 0x56525894d590 2024-03-26T18:03:49.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.673+0000 7f3184ff9640 1 -- 172.21.15.105:0/1888755304 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31780229c0 con 0x56525894d590 2024-03-26T18:03:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 -- 172.21.15.105:0/1888755304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56525894d590 msgr2=0x565258946910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56525894d590 0x565258946910 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f3178013aa0 tx=0x7f3178009780 comp rx=0 tx=0).stop 2024-03-26T18:03:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 -- 172.21.15.105:0/1888755304 shutdown_connections 2024-03-26T18:03:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56525894d590 0x565258946910 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258948420 0x565258959490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 --2- 172.21.15.105:0/1888755304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565258947a30 0x5652589523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 -- 172.21.15.105:0/1888755304 >> 172.21.15.105:0/1888755304 conn(0x565258942fa0 msgr2=0x565258946200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 -- 172.21.15.105:0/1888755304 shutdown_connections 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.674+0000 7f31983e8740 1 -- 172.21.15.105:0/1888755304 wait complete. 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 1 -- start start 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 1 Processor -- start 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258947a30 0x565258956ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258948420 0x565258957020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 0x565258958030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f31901ac400 con 0x565258948420 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31901abd30 con 0x56525894d590 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31901a8f10 con 0x565258947a30 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31983e8740 10 client.0 _ll_register_callbacks cb 0x7f31780045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258948420 0x565258957020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258948420 0x565258957020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49902/0 (socket says 172.21.15.105:49902) 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 -- 172.21.15.105:0/3634440203 learned_addr learned my addr 172.21.15.105:0/3634440203 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31867fc640 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 0x565258958030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31867fc640 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258947a30 0x565258956ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 -- 172.21.15.105:0/3634440203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258947a30 msgr2=0x565258956ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258947a30 0x565258956ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 -- 172.21.15.105:0/3634440203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 msgr2=0x565258958030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 0x565258958030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 -- 172.21.15.105:0/3634440203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565258958620 con 0x565258948420 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31867fc640 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258947a30 0x565258956ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f318effd640 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258948420 0x565258957020 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f3180000c00 tx=0x7f31800045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e44faadec39ddc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.675+0000 7f31767fc640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31767fc640 1 -- 172.21.15.105:0/3634440203 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180005440 con 0x565258948420 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31767fc640 1 -- 172.21.15.105:0/3634440203 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f31800055e0 con 0x565258948420 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 1 -- 172.21.15.105:0/3634440203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3178032070 con 0x565258948420 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31767fc640 1 -- 172.21.15.105:0/3634440203 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f318002ba30 con 0x565258948420 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 10 client.24570 fetch_fsmap learned FSMap version 48 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 1 -- 172.21.15.105:0/3634440203 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5652589c4290 con 0x565258948420 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31767fc640 1 -- 172.21.15.105:0/3634440203 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3178032070 con 0x565258948420 2024-03-26T18:03:49.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31767fc640 1 -- 172.21.15.105:0/3634440203 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f3180004930 con 0x565258948420 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 10 client.24570 fetch_fsmap finished waiting for FSMap version 48 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 10 client.24570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 20 client.24570 populate_metadata read hostname 'smithi105' 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56525895def0 0x5652589c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 4 client.24570 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31983e8740 1 -- 172.21.15.105:0/3634440203 --> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652588b4200 con 0x56525895def0 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.676+0000 7f31867fc640 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56525895def0 0x5652589c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31767fc640 10 client.24570 ms_handle_connect on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31867fc640 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56525895def0 0x5652589c6b40 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f43e812116290335 server_cookie=a568364a23ea463f in_seq=0 out_seq=0 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31767fc640 1 -- 172.21.15.105:0/3634440203 <== mds.0 v2:172.21.15.135:6838/338591305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652588b4200 con 0x56525895def0 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31767fc640 10 client.24570 handle_command_reply: tid=0 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 client.24570 shutdown 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 -- 172.21.15.105:0/3634440203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258948420 msgr2=0x565258957020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258948420 0x565258957020 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f3180000c00 tx=0x7f31800045c0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 -- 172.21.15.105:0/3634440203 shutdown_connections 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.135:6838/338591305,v1:172.21.15.135:6839/338591305] conn(0x56525895def0 0x5652589c6b40 crc :-1 s=READY pgs=631 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56525894d590 0x565258958030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565258948420 0x565258957020 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31767fc640 0 client.24570 ms_handle_reset on v2:172.21.15.135:6838/338591305 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 --2- 172.21.15.105:0/3634440203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x565258947a30 0x565258956ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 -- 172.21.15.105:0/3634440203 >> 172.21.15.105:0/3634440203 conn(0x565258942fa0 msgr2=0x565258946a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 -- 172.21.15.105:0/3634440203 shutdown_connections 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 1 -- 172.21.15.105:0/3634440203 wait complete. 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.677+0000 7f31983e8740 20 client.24570 trim_cache size 0 max 16384 2024-03-26T18:03:49.663 INFO:teuthology.orchestra.run.smithi105.stdout:[] 2024-03-26T18:03:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 -- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f317c0a2fa0 msgr2=0x7f317c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f317c0a2fa0 0x7f317c0a5460 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f31880ef350 tx=0x7f3188088760 comp rx=0 tx=0).stop 2024-03-26T18:03:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 -- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 msgr2=0x7f31900741f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 0x7f31900741f0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f318802c700 tx=0x7f31880328a0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 -- 172.21.15.105:0/2041881466 shutdown_connections 2024-03-26T18:03:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f317c0a2fa0 0x7f317c0a5460 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3190103480 0x7f31900741f0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3190102ac0 0x7f3190073cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 --2- 172.21.15.105:0/2041881466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31901018d0 0x7f3190075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 -- 172.21.15.105:0/2041881466 >> 172.21.15.105:0/2041881466 conn(0x7f31900fce60 msgr2=0x7f3190105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 -- 172.21.15.105:0/2041881466 shutdown_connections 2024-03-26T18:03:49.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:49.678+0000 7f319571b640 1 -- 172.21.15.105:0/2041881466 wait complete. 2024-03-26T18:03:50.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:50 smithi144 ceph-mon[36095]: pgmap v1117: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:03:50.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:50 smithi105 ceph-mon[32280]: pgmap v1117: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:03:50.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:50 smithi135 ceph-mon[35636]: pgmap v1117: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:03:50.669 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-26T18:03:50.669 INFO:tasks.fwd_scrub:done joining 2024-03-26T18:03:50.670 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-26T18:03:50.680 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 -- 172.21.15.105:0/3865111034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 msgr2=0x7ff25c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3865111034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 0x7ff25c100c00 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009900 tx=0x7ff24c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 -- 172.21.15.105:0/3865111034 shutdown_connections 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3865111034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff25c102390 0x7ff25c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3865111034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff25c1019d0 0x7ff25c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3865111034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 0x7ff25c100c00 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 -- 172.21.15.105:0/3865111034 >> 172.21.15.105:0/3865111034 conn(0x7ff25c0fbd50 msgr2=0x7ff25c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 -- 172.21.15.105:0/3865111034 shutdown_connections 2024-03-26T18:03:50.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.809+0000 7ff260dc6640 1 -- 172.21.15.105:0/3865111034 wait complete. 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.810+0000 7ff260dc6640 1 Processor -- start 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.810+0000 7ff260dc6640 1 -- start start 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.810+0000 7ff260dc6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 0x7ff25c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.810+0000 7ff260dc6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff25c1019d0 0x7ff25c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.810+0000 7ff260dc6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff25c102390 0x7ff25c1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.810+0000 7ff260dc6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c10c310 con 0x7ff25c1007e0 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.810+0000 7ff260dc6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c10c190 con 0x7ff25c102390 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.810+0000 7ff260dc6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c10c490 con 0x7ff25c1019d0 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff25a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 0x7ff25c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff25a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 0x7ff25c199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49924/0 (socket says 172.21.15.105:49924) 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff25a59c640 1 -- 172.21.15.105:0/3980338368 learned_addr learned my addr 172.21.15.105:0/3980338368 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff251d9b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff25c1019d0 0x7ff25c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:50.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff25ad9d640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff25c102390 0x7ff25c1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff251d9b640 1 -- 172.21.15.105:0/3980338368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff25c102390 msgr2=0x7ff25c1a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff251d9b640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff25c102390 0x7ff25c1a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff251d9b640 1 -- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 msgr2=0x7ff25c199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff251d9b640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 0x7ff25c199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff251d9b640 1 -- 172.21.15.105:0/3980338368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c1a1980 con 0x7ff25c1019d0 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff25a59c640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 0x7ff25c199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff25ad9d640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff25c102390 0x7ff25c1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff251d9b640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff25c1019d0 0x7ff25c19a2a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e9f0 tx=0x7ff24400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff253fff640 1 -- 172.21.15.105:0/3980338368 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400cd80 con 0x7ff25c1019d0 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff253fff640 1 -- 172.21.15.105:0/3980338368 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2440035f0 con 0x7ff25c1019d0 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c1a1c70 con 0x7ff25c1019d0 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff253fff640 1 -- 172.21.15.105:0/3980338368 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244010670 con 0x7ff25c1019d0 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.811+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c1a21c0 con 0x7ff25c1019d0 2024-03-26T18:03:50.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.812+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25c1a23f0 con 0x7ff25c1019d0 2024-03-26T18:03:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.813+0000 7ff253fff640 1 -- 172.21.15.105:0/3980338368 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff2440037a0 con 0x7ff25c1019d0 2024-03-26T18:03:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.814+0000 7ff253fff640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2480a2f20 0x7ff2480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.814+0000 7ff25a59c640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2480a2f20 0x7ff2480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.814+0000 7ff253fff640 1 -- 172.21.15.105:0/3980338368 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff2440cf6a0 con 0x7ff25c1019d0 2024-03-26T18:03:50.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.815+0000 7ff25a59c640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2480a2f20 0x7ff2480a53e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff24c02c840 tx=0x7ff24c0095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:50.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:50.817+0000 7ff253fff640 1 -- 172.21.15.105:0/3980338368 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff24409c490 con 0x7ff25c1019d0 2024-03-26T18:03:50.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.001+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff25c1a25c0 con 0x7ff25c1019d0 2024-03-26T18:03:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.002+0000 7ff253fff640 1 -- 172.21.15.105:0/3980338368 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+22833 (secure 0 0 0) 0x7ff24409bd90 con 0x7ff25c1019d0 2024-03-26T18:03:50.982 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:03:50.983 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":96,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T18:02:07.352921+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3f","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:03:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2480a2f20 msgr2=0x7ff2480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2480a2f20 0x7ff2480a53e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff24c02c840 tx=0x7ff24c0095e0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff25c1019d0 msgr2=0x7ff25c19a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:50.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff25c1019d0 0x7ff25c19a2a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff24400e9f0 tx=0x7ff24400eeb0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 shutdown_connections 2024-03-26T18:03:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff2480a2f20 0x7ff2480a53e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff25c102390 0x7ff25c1a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff25c1019d0 0x7ff25c19a2a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 --2- 172.21.15.105:0/3980338368 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff25c1007e0 0x7ff25c199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 >> 172.21.15.105:0/3980338368 conn(0x7ff25c0fbd50 msgr2=0x7ff25c068ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 shutdown_connections 2024-03-26T18:03:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.004+0000 7ff260dc6640 1 -- 172.21.15.105:0/3980338368 wait complete. 2024-03-26T18:03:50.996 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:03:50.996 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 -- 172.21.15.105:0/1387416083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c20100b10 msgr2=0x7f2c20100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 --2- 172.21.15.105:0/1387416083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c20100b10 0x7f2c20100f70 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009900 tx=0x7f2c1c02bcd0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 -- 172.21.15.105:0/1387416083 shutdown_connections 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 --2- 172.21.15.105:0/1387416083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c201014b0 0x7f2c20101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 --2- 172.21.15.105:0/1387416083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c20100b10 0x7f2c20100f70 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 --2- 172.21.15.105:0/1387416083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c201001b0 0x7f2c201005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 -- 172.21.15.105:0/1387416083 >> 172.21.15.105:0/1387416083 conn(0x7f2c200fbd50 msgr2=0x7f2c200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 -- 172.21.15.105:0/1387416083 shutdown_connections 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.152+0000 7f2c27787640 1 -- 172.21.15.105:0/1387416083 wait complete. 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c27787640 1 Processor -- start 2024-03-26T18:03:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c27787640 1 -- start start 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c27787640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c201001b0 0x7f2c20199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c27787640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c20100b10 0x7f2c2019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c27787640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c201014b0 0x7f2c201a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c27787640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2010f5f0 con 0x7f2c201001b0 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c27787640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2010f470 con 0x7f2c201014b0 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c27787640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2010f770 con 0x7f2c20100b10 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c25523640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c201001b0 0x7f2c20199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c25523640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c201001b0 0x7f2c20199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49954/0 (socket says 172.21.15.105:49954) 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c25523640 1 -- 172.21.15.105:0/1873191862 learned_addr learned my addr 172.21.15.105:0/1873191862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c24d22640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c20100b10 0x7f2c2019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.153+0000 7f2c25d24640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c201014b0 0x7f2c201a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c25523640 1 -- 172.21.15.105:0/1873191862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c20100b10 msgr2=0x7f2c2019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c25523640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c20100b10 0x7f2c2019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c25523640 1 -- 172.21.15.105:0/1873191862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c201014b0 msgr2=0x7f2c201a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c25523640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c201014b0 0x7f2c201a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c25523640 1 -- 172.21.15.105:0/1873191862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c201a1950 con 0x7f2c201001b0 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c24d22640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c20100b10 0x7f2c2019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c25d24640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c201014b0 0x7f2c201a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c25523640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c201001b0 0x7f2c20199d40 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000e9f0 tx=0x7f2c1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c167fc640 1 -- 172.21.15.105:0/1873191862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000cd80 con 0x7f2c201001b0 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c167fc640 1 -- 172.21.15.105:0/1873191862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c100108e0 con 0x7f2c201001b0 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c201a1c40 con 0x7f2c201001b0 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c167fc640 1 -- 172.21.15.105:0/1873191862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10018920 con 0x7f2c201001b0 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.154+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c201a2190 con 0x7f2c201001b0 2024-03-26T18:03:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.155+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2010b160 con 0x7f2c201001b0 2024-03-26T18:03:51.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.157+0000 7f2c167fc640 1 -- 172.21.15.105:0/1873191862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2c10010a90 con 0x7f2c201001b0 2024-03-26T18:03:51.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.158+0000 7f2c167fc640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c0c0a2f20 0x7f2c0c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.158+0000 7f2c24d22640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c0c0a2f20 0x7f2c0c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.158+0000 7f2c167fc640 1 -- 172.21.15.105:0/1873191862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f2c100cfc00 con 0x7f2c201001b0 2024-03-26T18:03:51.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.158+0000 7f2c24d22640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c0c0a2f20 0x7f2c0c0a53e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009230 tx=0x7f2c1c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:51.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.161+0000 7f2c167fc640 1 -- 172.21.15.105:0/1873191862 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2c1009c3a0 con 0x7f2c201001b0 2024-03-26T18:03:51.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.340+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2c2002d4f0 con 0x7f2c201001b0 2024-03-26T18:03:51.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.340+0000 7f2c167fc640 1 -- 172.21.15.105:0/1873191862 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f2c100a0440 con 0x7f2c201001b0 2024-03-26T18:03:51.320 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T18:03:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c0c0a2f20 msgr2=0x7f2c0c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c0c0a2f20 0x7f2c0c0a53e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009230 tx=0x7f2c1c005fd0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c201001b0 msgr2=0x7f2c20199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c201001b0 0x7f2c20199d40 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000e9f0 tx=0x7f2c1000eeb0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 shutdown_connections 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2c0c0a2f20 0x7f2c0c0a53e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c201014b0 0x7f2c201a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c20100b10 0x7f2c2019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 --2- 172.21.15.105:0/1873191862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c201001b0 0x7f2c20199d40 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 >> 172.21.15.105:0/1873191862 conn(0x7f2c200fbd50 msgr2=0x7f2c200fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 shutdown_connections 2024-03-26T18:03:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.342+0000 7f2c27787640 1 -- 172.21.15.105:0/1873191862 wait complete. 2024-03-26T18:03:51.332 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:51 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3980338368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:03:51.333 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:03:51.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:51 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3980338368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:03:51.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:51 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3980338368' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:03:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 -- 172.21.15.105:0/592105863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95541007c0 msgr2=0x7f9554100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 --2- 172.21.15.105:0/592105863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95541007c0 0x7f9554100be0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f9548009900 tx=0x7f954802bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 -- 172.21.15.105:0/592105863 shutdown_connections 2024-03-26T18:03:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 --2- 172.21.15.105:0/592105863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9554102370 0x7f9554108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 --2- 172.21.15.105:0/592105863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95541019b0 0x7f9554101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 --2- 172.21.15.105:0/592105863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95541007c0 0x7f9554100be0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 -- 172.21.15.105:0/592105863 >> 172.21.15.105:0/592105863 conn(0x7f95540fbd50 msgr2=0x7f95540fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 -- 172.21.15.105:0/592105863 shutdown_connections 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.460+0000 7f955ae6c640 1 -- 172.21.15.105:0/592105863 wait complete. 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f955ae6c640 1 Processor -- start 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f955ae6c640 1 -- start start 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f955ae6c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95541007c0 0x7f9554069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f955ae6c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95541019b0 0x7f955419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f955ae6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554102370 0x7f955419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f955ae6c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f955410c2c0 con 0x7f9554102370 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f955ae6c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f955410c140 con 0x7f95541007c0 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f955ae6c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f955410c440 con 0x7f95541019b0 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9559409640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554102370 0x7f955419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9558c08640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95541007c0 0x7f9554069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9559409640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554102370 0x7f955419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49984/0 (socket says 172.21.15.105:49984) 2024-03-26T18:03:51.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9559409640 1 -- 172.21.15.105:0/1012757783 learned_addr learned my addr 172.21.15.105:0/1012757783 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9553fff640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95541019b0 0x7f955419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9559409640 1 -- 172.21.15.105:0/1012757783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95541019b0 msgr2=0x7f955419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9559409640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95541019b0 0x7f955419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9559409640 1 -- 172.21.15.105:0/1012757783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95541007c0 msgr2=0x7f9554069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.461+0000 7f9559409640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95541007c0 0x7f9554069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.462+0000 7f9559409640 1 -- 172.21.15.105:0/1012757783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955419f7b0 con 0x7f9554102370 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.462+0000 7f9559409640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554102370 0x7f955419eea0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f954c00eb10 tx=0x7f954c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.462+0000 7f9551ffb640 1 -- 172.21.15.105:0/1012757783 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c004b20 con 0x7f9554102370 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.462+0000 7f9551ffb640 1 -- 172.21.15.105:0/1012757783 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f954c004cc0 con 0x7f9554102370 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.462+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955419fa40 con 0x7f9554102370 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.462+0000 7f9551ffb640 1 -- 172.21.15.105:0/1012757783 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c010690 con 0x7f9554102370 2024-03-26T18:03:51.444 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.462+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955419ff40 con 0x7f9554102370 2024-03-26T18:03:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.463+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9554107b50 con 0x7f9554102370 2024-03-26T18:03:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.464+0000 7f9551ffb640 1 -- 172.21.15.105:0/1012757783 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f954c005020 con 0x7f9554102370 2024-03-26T18:03:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.465+0000 7f9551ffb640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95440a2fa0 0x7f95440a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.445 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.465+0000 7f9558c08640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95440a2fa0 0x7f95440a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.465+0000 7f9551ffb640 1 -- 172.21.15.105:0/1012757783 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f954c0cf9e0 con 0x7f9554102370 2024-03-26T18:03:51.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.465+0000 7f9558c08640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95440a2fa0 0x7f95440a5460 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9548005ab0 tx=0x7f9548005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:51.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.469+0000 7f9551ffb640 1 -- 172.21.15.105:0/1012757783 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f954c09c770 con 0x7f9554102370 2024-03-26T18:03:51.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.649+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95540624c0 con 0x7f9554102370 2024-03-26T18:03:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T18:03:51.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.649+0000 7f9551ffb640 1 -- 172.21.15.105:0/1012757783 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f954c09c070 con 0x7f9554102370 2024-03-26T18:03:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95440a2fa0 msgr2=0x7f95440a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95440a2fa0 0x7f95440a5460 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9548005ab0 tx=0x7f9548005fd0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554102370 msgr2=0x7f955419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554102370 0x7f955419eea0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f954c00eb10 tx=0x7f954c00c2d0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 shutdown_connections 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f95440a2fa0 0x7f95440a5460 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554102370 0x7f955419eea0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95541019b0 0x7f955419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 --2- 172.21.15.105:0/1012757783 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95541007c0 0x7f9554069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 >> 172.21.15.105:0/1012757783 conn(0x7f95540fbd50 msgr2=0x7f9554104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 shutdown_connections 2024-03-26T18:03:51.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.652+0000 7f955ae6c640 1 -- 172.21.15.105:0/1012757783 wait complete. 2024-03-26T18:03:51.644 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:03:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 -- 172.21.15.105:0/4157940111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a841019d0 msgr2=0x7f2a84101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/4157940111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a841019d0 0x7f2a84101e50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009860 tx=0x7f2a7402bb70 comp rx=0 tx=0).stop 2024-03-26T18:03:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 -- 172.21.15.105:0/4157940111 shutdown_connections 2024-03-26T18:03:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/4157940111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a84102390 0x7f2a84108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/4157940111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a841019d0 0x7f2a84101e50 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/4157940111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a841007e0 0x7f2a84100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 -- 172.21.15.105:0/4157940111 >> 172.21.15.105:0/4157940111 conn(0x7f2a840fbd70 msgr2=0x7f2a840fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 -- 172.21.15.105:0/4157940111 shutdown_connections 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 -- 172.21.15.105:0/4157940111 wait complete. 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 Processor -- start 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.771+0000 7f2a88bf4640 1 -- start start 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a88bf4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a841007e0 0x7f2a84197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a88bf4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a841019d0 0x7f2a841980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a88bf4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a84102390 0x7f2a8419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a88bf4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410c310 con 0x7f2a841019d0 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a88bf4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410c190 con 0x7f2a84102390 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a88bf4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410c490 con 0x7f2a841007e0 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a841019d0 0x7f2a841980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a841019d0 0x7f2a841980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:49996/0 (socket says 172.21.15.105:49996) 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 -- 172.21.15.105:0/2518017689 learned_addr learned my addr 172.21.15.105:0/2518017689 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a8259c640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a841007e0 0x7f2a84197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a82d9d640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a84102390 0x7f2a8419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 -- 172.21.15.105:0/2518017689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a841007e0 msgr2=0x7f2a84197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a841007e0 0x7f2a84197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 -- 172.21.15.105:0/2518017689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a84102390 msgr2=0x7f2a8419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a84102390 0x7f2a8419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 -- 172.21.15.105:0/2518017689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8419f770 con 0x7f2a841019d0 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a81d9b640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a841019d0 0x7f2a841980e0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f2a74005940 tx=0x7f2a74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a7b7fe640 1 -- 172.21.15.105:0/2518017689 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74041020 con 0x7f2a841019d0 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8419fa00 con 0x7f2a841019d0 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a7b7fe640 1 -- 172.21.15.105:0/2518017689 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a7402c9c0 con 0x7f2a841019d0 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a7b7fe640 1 -- 172.21.15.105:0/2518017689 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74039040 con 0x7f2a841019d0 2024-03-26T18:03:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.772+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8419ff40 con 0x7f2a841019d0 2024-03-26T18:03:51.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.773+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a841a0110 con 0x7f2a841019d0 2024-03-26T18:03:51.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.775+0000 7f2a7b7fe640 1 -- 172.21.15.105:0/2518017689 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f2a74044020 con 0x7f2a841019d0 2024-03-26T18:03:51.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.775+0000 7f2a7b7fe640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2a700a2f20 0x7f2a700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:51.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.776+0000 7f2a7b7fe640 1 -- 172.21.15.105:0/2518017689 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f2a740f11f0 con 0x7f2a841019d0 2024-03-26T18:03:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.776+0000 7f2a8259c640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2a700a2f20 0x7f2a700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.776+0000 7f2a8259c640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2a700a2f20 0x7f2a700a53e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2a84101830 tx=0x7f2a6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.779+0000 7f2a7b7fe640 1 -- 172.21.15.105:0/2518017689 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2a7404b020 con 0x7f2a841019d0 2024-03-26T18:03:51.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.977+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a841a02e0 con 0x7f2a841019d0 2024-03-26T18:03:51.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.978+0000 7f2a7b7fe640 1 -- 172.21.15.105:0/2518017689 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f2a740bdf80 con 0x7f2a841019d0 2024-03-26T18:03:51.958 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.979+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2a700a2f20 msgr2=0x7f2a700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2a700a2f20 0x7f2a700a53e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2a84101830 tx=0x7f2a6c005f90 comp rx=0 tx=0).stop 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a841019d0 msgr2=0x7f2a841980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a841019d0 0x7f2a841980e0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f2a74005940 tx=0x7f2a74004270 comp rx=0 tx=0).stop 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 shutdown_connections 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f2a700a2f20 0x7f2a700a53e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a84102390 0x7f2a8419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2a841019d0 0x7f2a841980e0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 --2- 172.21.15.105:0/2518017689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a841007e0 0x7f2a84197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:51.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 >> 172.21.15.105:0/2518017689 conn(0x7f2a840fbd70 msgr2=0x7f2a84069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:51.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 shutdown_connections 2024-03-26T18:03:51.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:51.980+0000 7f2a88bf4640 1 -- 172.21.15.105:0/2518017689 wait complete. 2024-03-26T18:03:51.972 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-26T18:03:52.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 -- 172.21.15.105:0/1503889090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1007e0 msgr2=0x7f764c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:52.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 --2- 172.21.15.105:0/1503889090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1007e0 0x7f764c100c00 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f763c009900 tx=0x7f763c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:03:52.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 -- 172.21.15.105:0/1503889090 shutdown_connections 2024-03-26T18:03:52.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 --2- 172.21.15.105:0/1503889090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c102390 0x7f764c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:52.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 --2- 172.21.15.105:0/1503889090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c1019d0 0x7f764c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 --2- 172.21.15.105:0/1503889090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1007e0 0x7f764c100c00 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 -- 172.21.15.105:0/1503889090 >> 172.21.15.105:0/1503889090 conn(0x7f764c0fbd50 msgr2=0x7f764c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 -- 172.21.15.105:0/1503889090 shutdown_connections 2024-03-26T18:03:52.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.099+0000 7f765115b640 1 -- 172.21.15.105:0/1503889090 wait complete. 2024-03-26T18:03:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f765115b640 1 Processor -- start 2024-03-26T18:03:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f765115b640 1 -- start start 2024-03-26T18:03:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f765115b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c1007e0 0x7f764c196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:52.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f765115b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1019d0 0x7f764c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f765115b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102390 0x7f764c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f765115b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10c310 con 0x7f764c1019d0 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f765115b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10c190 con 0x7f764c102390 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f765115b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10c490 con 0x7f764c1007e0 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f764a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1019d0 0x7f764c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f764a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1019d0 0x7f764c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50008/0 (socket says 172.21.15.105:50008) 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f764a59c640 1 -- 172.21.15.105:0/688985526 learned_addr learned my addr 172.21.15.105:0/688985526 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f764ad9d640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c1007e0 0x7f764c196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.100+0000 7f764b59e640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102390 0x7f764c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f764a59c640 1 -- 172.21.15.105:0/688985526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c1007e0 msgr2=0x7f764c196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f764a59c640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c1007e0 0x7f764c196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f764a59c640 1 -- 172.21.15.105:0/688985526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102390 msgr2=0x7f764c19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:52.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f764a59c640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102390 0x7f764c19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f764a59c640 1 -- 172.21.15.105:0/688985526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c19e660 con 0x7f764c1019d0 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f764ad9d640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c1007e0 0x7f764c196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f764a59c640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1019d0 0x7f764c196f80 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f764000ea60 tx=0x7f764000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f7633fff640 1 -- 172.21.15.105:0/688985526 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764000cde0 con 0x7f764c1019d0 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c19e950 con 0x7f764c1019d0 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f7633fff640 1 -- 172.21.15.105:0/688985526 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76400035f0 con 0x7f764c1019d0 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f7633fff640 1 -- 172.21.15.105:0/688985526 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640010690 con 0x7f764c1019d0 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.101+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c19eea0 con 0x7f764c1019d0 2024-03-26T18:03:52.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.102+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764c107c70 con 0x7f764c1019d0 2024-03-26T18:03:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.103+0000 7f7633fff640 1 -- 172.21.15.105:0/688985526 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f76400037a0 con 0x7f764c1019d0 2024-03-26T18:03:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.104+0000 7f7633fff640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76340a2f20 0x7f76340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:03:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.104+0000 7f764ad9d640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76340a2f20 0x7f76340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:03:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.104+0000 7f7633fff640 1 -- 172.21.15.105:0/688985526 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f76400cf690 con 0x7f764c1019d0 2024-03-26T18:03:52.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.104+0000 7f764ad9d640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76340a2f20 0x7f76340a53e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f763c005ab0 tx=0x7f763c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:03:52.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.108+0000 7f7633fff640 1 -- 172.21.15.105:0/688985526 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f764009c480 con 0x7f764c1019d0 2024-03-26T18:03:52.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:52 smithi144 ceph-mon[36095]: pgmap v1118: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:03:52.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:52 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1873191862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:03:52.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:52 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1012757783' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:03:52.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:52 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2518017689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:03:52.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.366+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f764c0624c0 con 0x7f764c1019d0 2024-03-26T18:03:52.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.375+0000 7f7633fff640 1 -- 172.21.15.105:0/688985526 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+741851 (secure 0 0 0) 0x7f764009bd80 con 0x7f764c1019d0 2024-03-26T18:03:52.365 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:03:52.365 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:03:52.365 INFO:teuthology.orchestra.run.smithi105.stdout: "epoch": 20, 2024-03-26T18:03:52.365 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "active_gid": 14152, 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "active_name": "x", 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "active_addrs": { 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6800", 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2294330938 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:6801", 2024-03-26T18:03:52.366 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2294330938 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "active_addr": "172.21.15.105:6801/2294330938", 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "active_change": "2024-03-26T17:29:26.501651+0000", 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "active_mgr_features": 4540701547738038271, 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "available": true, 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "standbys": [ 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "gid": 24109, 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "z", 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr_features": 4540701547738038271, 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: "available_modules": [ 2024-03-26T18:03:52.367 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "alerts", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T18:03:52.368 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.369 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.370 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.371 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.372 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.373 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_destination": { 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_destination", 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.374 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email address to send alerts to", 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_from_name": { 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_from_name", 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "Ceph", 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.375 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email From: name", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_host": { 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_host", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP server", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.376 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_password": { 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_password", 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Password to authenticate with", 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.377 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_port": { 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_port", 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "465", 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP port", 2024-03-26T18:03:52.378 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_sender": { 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_sender", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP envelope sender", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.379 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_ssl": { 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_ssl", 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.380 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_user": { 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_user", 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "User to authenticate as", 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.381 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "balancer", 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "active": { 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "active", 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.382 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_time": { 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_time", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0000", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.383 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_weekday": { 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_weekday", 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.384 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_max_iterations": { 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_max_iterations", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "25", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:52.385 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "250", 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_metrics": { 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_metrics", 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.386 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_step": { 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_step", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.5", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0.001", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "0.999", 2024-03-26T18:03:52.387 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "aggressiveness of optimization", 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "end_time": { 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_time", 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2359", 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.388 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "end_weekday": { 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_weekday", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T18:03:52.389 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.390 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.391 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.392 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.393 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "min_score": { 2024-03-26T18:03:52.394 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_score", 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-03-26T18:03:52.395 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "upmap", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "crush-compat", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "none", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "read", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap-read" 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Balancer mode", 2024-03-26T18:03:52.396 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_ids": { 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_ids", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.397 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.398 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_deviation": { 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_deviation", 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.399 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_optimizations": { 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_optimizations", 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_down_multiplier": { 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_down_multiplier", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T18:03:52.401 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_refresh_rate": { 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_refresh_rate", 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "20", 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T18:03:52.402 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_starting_port": { 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_starting_port", 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4721", 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.403 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_ptrace": { 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_ptrace", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T18:03:52.404 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_interval": { 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_interval", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.405 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_memory_target_ratio": { 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.7", 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.406 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm_log_destination": { 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm_log_destination", 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "file", 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "file,syslog", 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "syslog" 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.407 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "cgroups_split": { 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cgroups_split", 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.408 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "config_checks_enabled": { 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_checks_enabled", 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.409 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "config_dashboard": { 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_dashboard", 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.410 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_alertmanager": { 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_alertmanager", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:52.411 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_base": { 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_base", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Container image name, without the tag", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.412 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_elasticsearch": { 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_elasticsearch", 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "elasticsearch container image", 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.413 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_grafana": { 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_grafana", 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.414 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_haproxy": { 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_haproxy", 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "HAproxy container image", 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.415 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_agent": { 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_agent", 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger agent container image", 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_collector": { 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_collector", 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger collector container image", 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_query": { 2024-03-26T18:03:52.417 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_query", 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger query container image", 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_keepalived": { 2024-03-26T18:03:52.418 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_keepalived", 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Keepalived container image", 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_loki": { 2024-03-26T18:03:52.419 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_loki", 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Loki container image", 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_node_exporter": { 2024-03-26T18:03:52.420 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_node_exporter", 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_nvmeof": { 2024-03-26T18:03:52.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_nvmeof", 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Nvme-of container image", 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_prometheus": { 2024-03-26T18:03:52.422 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_prometheus", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_promtail": { 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_promtail", 2024-03-26T18:03:52.423 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Promtail container image", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_snmp_gateway": { 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_snmp_gateway", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.424 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SNMP Gateway container image", 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "container_init": { 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_init", 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.425 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "daemon_cache_timeout": { 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "daemon_cache_timeout", 2024-03-26T18:03:52.426 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.427 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "default_cephadm_command_timeout": { 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "900", 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.428 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "default_registry": { 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_registry", 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io", 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "device_cache_timeout": { 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_cache_timeout", 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1800", 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache device inventory", 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "device_enhanced_scan": { 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_enhanced_scan", 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "facts_cache_timeout": { 2024-03-26T18:03:52.431 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "facts_cache_timeout", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache host facts data", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "grafana_dashboards_path": { 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "grafana_dashboards_path", 2024-03-26T18:03:52.432 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "host_check_interval": { 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "host_check_interval", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.433 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to perform a host check", 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "hw_monitoring": { 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hw_monitoring", 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.434 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "inventory_list_all": { 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "inventory_list_all", 2024-03-26T18:03:52.435 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.436 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.437 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "log_refresh_metadata": { 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_refresh_metadata", 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.438 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.439 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.440 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.441 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.442 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "*", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T18:03:52.443 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "max_count_per_host": { 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_count_per_host", 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.444 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of daemons per service per host", 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "max_osd_draining_count": { 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_osd_draining_count", 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.445 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "migration_current": { 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "migration_current", 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.446 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "internal - do not modify", 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "root", 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.447 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm-package", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "root" 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "oob_default_addr": { 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "oob_default_addr", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "169.254.1.1", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.448 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "prometheus_alerts_path": { 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus_alerts_path", 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.449 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_insecure": { 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_insecure", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T18:03:52.450 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_password": { 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_password", 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T18:03:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_url": { 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_url", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.452 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_username": { 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_username", 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.453 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "secure_monitoring_stack": { 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secure_monitoring_stack", 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.454 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "service_discovery_port": { 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "service_discovery_port", 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8765", 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "cephadm service discovery port", 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.455 INFO:teuthology.orchestra.run.smithi105.stdout: "ssh_config_file": { 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssh_config_file", 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "use_agent": { 2024-03-26T18:03:52.456 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_agent", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "use_repo_digest": { 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_repo_digest", 2024-03-26T18:03:52.457 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_failed_host_check": { 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_failed_host_check", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.458 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_daemons": { 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_daemons", 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.459 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_hosts": { 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_hosts", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.460 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crash", 2024-03-26T18:03:52.461 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.462 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.464 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.465 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "retain_interval": { 2024-03-26T18:03:52.466 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retain_interval", 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "31536000", 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.467 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_recent_interval": { 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_recent_interval", 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1209600", 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.468 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dashboard", 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.469 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.470 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.471 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_ENABLED": { 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.472 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.473 INFO:teuthology.orchestra.run.smithi105.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.474 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.475 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.476 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T18:03:52.477 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T18:03:52.478 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.479 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T18:03:52.480 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T18:03:52.481 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T18:03:52.482 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.483 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_URL": { 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_URL", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.484 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:52.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.486 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.487 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.488 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.489 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "{}", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.490 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.492 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T18:03:52.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.494 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.495 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.496 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.497 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T18:03:52.498 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.499 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.500 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.501 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8", 2024-03-26T18:03:52.502 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "45", 2024-03-26T18:03:52.503 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.504 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:52.505 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.506 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.507 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.508 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.509 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.510 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.511 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "cross_origin_url": { 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cross_origin_url", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.512 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "crt_file": { 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crt_file", 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.513 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "debug": { 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "debug", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.514 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable/disable debug options", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "jwt_token_ttl": { 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "jwt_token_ttl", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "28800", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.515 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.516 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.517 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.518 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.519 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.520 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "motd": { 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "motd", 2024-03-26T18:03:52.521 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "The message of the day", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect_resolve_ip_addr": { 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.522 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.523 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.524 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8080", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.525 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl_server_port": { 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl_server_port", 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.526 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8443", 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.527 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "redirect", 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect" 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-03-26T18:03:52.528 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "url_prefix": { 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url_prefix", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.529 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "devicehealth", 2024-03-26T18:03:52.530 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_monitoring": { 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_monitoring", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "monitor device health metrics", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.531 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.532 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.533 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.534 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.535 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "mark_out_threshold": { 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mark_out_threshold", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.536 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2419200", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_name": { 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_name", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.537 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "device_health_metrics", 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "retention_period": { 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retention_period", 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.538 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15552000", 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain device health metrics", 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_frequency": { 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_frequency", 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.539 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "self_heal": { 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "self_heal", 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.540 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.541 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_threshold": { 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_threshold", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7257600", 2024-03-26T18:03:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "diskprediction_local", 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.543 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.544 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.545 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.546 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.547 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "predict_interval": { 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predict_interval", 2024-03-26T18:03:52.548 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "predictor_model": { 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predictor_model", 2024-03-26T18:03:52.549 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "prophetstor", 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.550 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "influx", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": false, 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "influxdb python module not found", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "batch_size": { 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "batch_size", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5000", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T18:03:52.552 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "database": { 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "database", 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "ceph", 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T18:03:52.553 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": { 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hostname", 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server hostname", 2024-03-26T18:03:52.554 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "30", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "5", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.555 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.556 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.557 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.558 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.560 INFO:teuthology.orchestra.run.smithi105.stdout: "password": { 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "password", 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "password of InfluxDB server user", 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "port": { 2024-03-26T18:03:52.561 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "port", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8086", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server port", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-03-26T18:03:52.562 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "false", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "threads": { 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threads", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.563 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "32", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "username": { 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "username", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.564 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "username of InfluxDB server user", 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "verify_ssl": { 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "verify_ssl", 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.565 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "true", 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "insights", 2024-03-26T18:03:52.566 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.567 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.568 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.570 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.571 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "iostat", 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.572 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.573 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.574 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.575 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.576 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "k8sevents", 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_event_retention_days": { 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ceph_event_retention_days", 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7", 2024-03-26T18:03:52.577 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "config_check_secs": { 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_check_secs", 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.578 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "10", 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.579 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.581 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.582 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.583 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "localpool", 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "failure_domain": { 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "failure_domain", 2024-03-26T18:03:52.584 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "host", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "failure domain for any created local pool", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.585 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.586 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.587 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.588 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.589 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "min_size": { 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_size", 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.590 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default min_size for any created local pool", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "num_rep": { 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "num_rep", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default replica count for any created local pool", 2024-03-26T18:03:52.591 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_num": { 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_num", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "128", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.592 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "prefix": { 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prefix", 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name prefix for any created local pool", 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "subtree": { 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "subtree", 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "rack", 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T18:03:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mds_autoscaler", 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.595 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.596 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.597 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.598 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.599 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirroring", 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.600 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.601 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.603 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.604 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "nfs", 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.605 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.606 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.607 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.608 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.609 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.610 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "fail_fs": { 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "fail_fs", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.611 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.612 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.613 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.614 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.615 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator": { 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm", 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "rook", 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: "test_orchestrator" 2024-03-26T18:03:52.617 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Orchestrator backend", 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_perf_query", 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.618 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.619 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.620 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.621 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.622 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_support", 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.623 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.624 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.626 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.627 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.628 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_autoscaler", 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.629 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.630 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.632 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "threshold": { 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threshold", 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1.0", 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "scaling threshold", 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.635 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "progress", 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_pg_recovery_event": { 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_pg_recovery_event", 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.637 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.639 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.640 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.641 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "max_completed_events": { 2024-03-26T18:03:52.642 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_completed_events", 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "50", 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "number of past completed events to remember", 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:52.643 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long the module is going to sleep", 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "cache": { 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cache", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.645 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "exclude_perf_counters": { 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "exclude_perf_counters", 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.646 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.647 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.649 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.650 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.651 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools": { 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools", 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.652 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "300", 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_interval": { 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_interval", 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15.0", 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.654 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "9283", 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.656 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "stale_cache_strategy": { 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stale_cache_strategy", 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "log", 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.657 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "default", 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "default", 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "error" 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.658 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.659 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_support", 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.660 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.661 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.662 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.663 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.664 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_snap_create": { 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_snap_create", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.665 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "mirror_snapshot_schedule": { 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.666 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "trash_purge_schedule": { 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "trash_purge_schedule", 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.667 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "restful", 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_auth": { 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_auth", 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.668 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.669 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.670 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.671 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.672 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.673 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.674 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.675 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.676 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rgw", 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.677 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.678 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.679 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.680 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.681 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rook", 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.682 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.683 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.684 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.685 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.686 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.687 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "storage_class": { 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "storage_class", 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "local", 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.688 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "selftest", 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.689 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.690 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.691 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.692 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.693 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "roption1": { 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption1", 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.694 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "roption2": { 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption2", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.695 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "xyz", 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption1": { 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption1", 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.696 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption2": { 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption2", 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.697 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption3": { 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption3", 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.698 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption4": { 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption4", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.699 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption5": { 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption5", 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.700 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption6": { 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption6", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.701 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption7": { 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption7", 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.702 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "42", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "testkey": { 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testkey", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.703 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "testlkey": { 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testlkey", 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.704 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "testnewline": { 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testnewline", 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.705 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snap_schedule", 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.706 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_m_granularity": { 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_m_granularity", 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.707 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "dump_on_update": { 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dump_on_update", 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "dump database to debug log on update", 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.708 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.709 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.710 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.711 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.712 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.713 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stats", 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.714 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.715 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.716 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.717 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.718 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.719 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "status", 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.720 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.721 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.722 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.723 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.724 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.725 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telegraf", 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "address": { 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "address", 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.726 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.727 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.728 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.729 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.730 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.731 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telemetry", 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_basic": { 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_basic", 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.733 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_crash": { 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_crash", 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.734 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_device": { 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_device", 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_ident": { 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_ident", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_perf": { 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_perf", 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "contact": { 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "contact", 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.738 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "description": { 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "description", 2024-03-26T18:03:52.739 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "device_url": { 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_url", 2024-03-26T18:03:52.740 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.741 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:52.742 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "24", 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "8", 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "last_opt_revision": { 2024-03-26T18:03:52.743 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "last_opt_revision", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard": { 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard", 2024-03-26T18:03:52.744 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard_description": { 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard_description", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.745 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.746 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.748 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.749 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.750 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "organization": { 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "organization", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.751 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "proxy": { 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "proxy", 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.752 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "url": { 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.753 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "test_orchestrator", 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.754 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.755 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.756 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.757 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.758 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "volumes", 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.759 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.760 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.761 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.762 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.764 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_clones": { 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_clones", 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4", 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.765 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "periodic_async_work": { 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "periodic_async_work", 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.766 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Periodically check for async work", 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_delay": { 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_delay", 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.767 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_no_wait": { 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.768 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix", 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "discovery_interval": { 2024-03-26T18:03:52.769 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "discovery_interval", 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "100", 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.770 INFO:teuthology.orchestra.run.smithi105.stdout: "identifier": { 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "identifier", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:52.771 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.772 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.773 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.775 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.776 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_host": { 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_host", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.777 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_port": { 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_port", 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.778 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10051", 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_sender": { 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_sender", 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.779 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.780 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "gid": 24110, 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "y", 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr_features": 4540701547738038271, 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "available_modules": [ 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "alerts", 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.781 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.782 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.783 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.784 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.785 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.786 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_destination": { 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_destination", 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.787 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email address to send alerts to", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_from_name": { 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_from_name", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "Ceph", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email From: name", 2024-03-26T18:03:52.788 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_host": { 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_host", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP server", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.789 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_password": { 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_password", 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Password to authenticate with", 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.790 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_port": { 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_port", 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "465", 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP port", 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.791 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_sender": { 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_sender", 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP envelope sender", 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.792 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_ssl": { 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_ssl", 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_user": { 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_user", 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "User to authenticate as", 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.794 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "balancer", 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "active": { 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "active", 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.795 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_time": { 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_time", 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0000", 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.796 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_weekday": { 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_weekday", 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-03-26T18:03:52.797 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_max_iterations": { 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_max_iterations", 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "25", 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:52.798 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "250", 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_metrics": { 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_metrics", 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.799 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.801 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.801 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T18:03:52.801 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T18:03:52.801 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.801 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_step": { 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_step", 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.5", 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0.001", 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "0.999", 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "aggressiveness of optimization", 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T18:03:52.802 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "end_time": { 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_time", 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2359", 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:03:52.803 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "end_weekday": { 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_weekday", 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:03:52.804 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.805 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.806 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.807 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.808 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.809 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "min_score": { 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_score", 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.810 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "upmap", 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "crush-compat", 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "none", 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "read", 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap", 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap-read" 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Balancer mode", 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_ids": { 2024-03-26T18:03:52.812 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_ids", 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:52.813 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_deviation": { 2024-03-26T18:03:52.814 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_deviation", 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_optimizations": { 2024-03-26T18:03:52.815 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_optimizations", 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_down_multiplier": { 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_down_multiplier", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T18:03:52.817 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_refresh_rate": { 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_refresh_rate", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "20", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.818 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_starting_port": { 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_starting_port", 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4721", 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.819 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_ptrace": { 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_ptrace", 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T18:03:52.820 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_interval": { 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_interval", 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.821 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_memory_target_ratio": { 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.7", 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.822 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm_log_destination": { 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm_log_destination", 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "file", 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "file,syslog", 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: "syslog" 2024-03-26T18:03:52.823 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "cgroups_split": { 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cgroups_split", 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.824 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "config_checks_enabled": { 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_checks_enabled", 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.825 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "config_dashboard": { 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_dashboard", 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.826 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_alertmanager": { 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_alertmanager", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.827 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_base": { 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_base", 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.828 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Container image name, without the tag", 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_elasticsearch": { 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_elasticsearch", 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.829 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "elasticsearch container image", 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_grafana": { 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_grafana", 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.830 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_haproxy": { 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_haproxy", 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.831 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "HAproxy container image", 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_agent": { 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_agent", 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.832 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger agent container image", 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_collector": { 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_collector", 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.833 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger collector container image", 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_query": { 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_query", 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.834 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger query container image", 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_keepalived": { 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_keepalived", 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.835 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Keepalived container image", 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_loki": { 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_loki", 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.836 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Loki container image", 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_node_exporter": { 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_node_exporter", 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.837 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_nvmeof": { 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_nvmeof", 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.838 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Nvme-of container image", 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_prometheus": { 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_prometheus", 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.839 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_promtail": { 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_promtail", 2024-03-26T18:03:52.840 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Promtail container image", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_snmp_gateway": { 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_snmp_gateway", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.841 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SNMP Gateway container image", 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "container_init": { 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_init", 2024-03-26T18:03:52.842 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "daemon_cache_timeout": { 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "daemon_cache_timeout", 2024-03-26T18:03:52.843 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "default_cephadm_command_timeout": { 2024-03-26T18:03:52.844 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "900", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "default_registry": { 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_registry", 2024-03-26T18:03:52.845 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io", 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "device_cache_timeout": { 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_cache_timeout", 2024-03-26T18:03:52.846 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1800", 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache device inventory", 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "device_enhanced_scan": { 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_enhanced_scan", 2024-03-26T18:03:52.847 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "facts_cache_timeout": { 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "facts_cache_timeout", 2024-03-26T18:03:52.848 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache host facts data", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "grafana_dashboards_path": { 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "grafana_dashboards_path", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.849 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "host_check_interval": { 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "host_check_interval", 2024-03-26T18:03:52.850 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to perform a host check", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "hw_monitoring": { 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hw_monitoring", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.851 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "inventory_list_all": { 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "inventory_list_all", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.852 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.853 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.854 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "log_refresh_metadata": { 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_refresh_metadata", 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.856 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.857 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.858 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.859 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "*", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.860 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "max_count_per_host": { 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_count_per_host", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.861 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of daemons per service per host", 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "max_osd_draining_count": { 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_osd_draining_count", 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.862 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "migration_current": { 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "migration_current", 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.863 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "internal - do not modify", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "root", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm-package", 2024-03-26T18:03:52.864 INFO:teuthology.orchestra.run.smithi105.stdout: "root" 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "oob_default_addr": { 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "oob_default_addr", 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "169.254.1.1", 2024-03-26T18:03:52.865 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "prometheus_alerts_path": { 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus_alerts_path", 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.866 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_insecure": { 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_insecure", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.867 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_password": { 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_password", 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.868 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_url": { 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_url", 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.869 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_username": { 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_username", 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.870 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "secure_monitoring_stack": { 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secure_monitoring_stack", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.871 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "service_discovery_port": { 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "service_discovery_port", 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8765", 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.872 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "cephadm service discovery port", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "ssh_config_file": { 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssh_config_file", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.873 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "use_agent": { 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_agent", 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.874 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "use_repo_digest": { 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_repo_digest", 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_failed_host_check": { 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_failed_host_check", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T18:03:52.876 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_daemons": { 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_daemons", 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T18:03:52.877 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_hosts": { 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_hosts", 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T18:03:52.878 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crash", 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.879 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.880 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.881 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.882 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.883 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "retain_interval": { 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retain_interval", 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.884 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "31536000", 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_recent_interval": { 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_recent_interval", 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.885 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1209600", 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dashboard", 2024-03-26T18:03:52.886 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.887 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.888 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.889 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_ENABLED": { 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.890 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T18:03:52.891 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T18:03:52.892 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T18:03:52.893 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.894 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.895 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T18:03:52.896 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T18:03:52.897 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.898 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.899 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.900 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.901 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_URL": { 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_URL", 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.902 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T18:03:52.903 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T18:03:52.904 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.905 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T18:03:52.906 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T18:03:52.907 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.908 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "{}", 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.909 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.910 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.911 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.912 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.913 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.914 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.915 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.916 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.917 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.918 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.919 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.920 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.921 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8", 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T18:03:52.922 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "45", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.923 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T18:03:52.924 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T18:03:52.925 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T18:03:52.926 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T18:03:52.927 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T18:03:52.928 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.929 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T18:03:52.930 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "cross_origin_url": { 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cross_origin_url", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.931 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "crt_file": { 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crt_file", 2024-03-26T18:03:52.932 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "debug": { 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "debug", 2024-03-26T18:03:52.933 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable/disable debug options", 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "jwt_token_ttl": { 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "jwt_token_ttl", 2024-03-26T18:03:52.934 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "28800", 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-03-26T18:03:52.935 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.936 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.937 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.938 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.939 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.940 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "motd": { 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "motd", 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.941 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "The message of the day", 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect_resolve_ip_addr": { 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.942 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.943 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.944 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8080", 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.945 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl_server_port": { 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl_server_port", 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.946 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8443", 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.947 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "redirect", 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect" 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-03-26T18:03:52.948 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "url_prefix": { 2024-03-26T18:03:52.949 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url_prefix", 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.950 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "devicehealth", 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_monitoring": { 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_monitoring", 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.951 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "monitor device health metrics", 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.952 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.953 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.954 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.955 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.956 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "mark_out_threshold": { 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mark_out_threshold", 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2419200", 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.957 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_name": { 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_name", 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "device_health_metrics", 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.958 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "retention_period": { 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retention_period", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15552000", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.959 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain device health metrics", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_frequency": { 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_frequency", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T18:03:52.960 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "self_heal": { 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "self_heal", 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T18:03:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.962 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_threshold": { 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_threshold", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7257600", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.963 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "diskprediction_local", 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.964 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.965 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.966 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.967 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.968 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "predict_interval": { 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predict_interval", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-03-26T18:03:52.969 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "predictor_model": { 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predictor_model", 2024-03-26T18:03:52.970 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "prophetstor", 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.971 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.972 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "influx", 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": false, 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "influxdb python module not found", 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "batch_size": { 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "batch_size", 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.973 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5000", 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.974 INFO:teuthology.orchestra.run.smithi105.stdout: "database": { 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "database", 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "ceph", 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.975 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": { 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hostname", 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server hostname", 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.976 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "30", 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "5", 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.978 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.979 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.980 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.981 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.982 INFO:teuthology.orchestra.run.smithi105.stdout: "password": { 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "password", 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "password of InfluxDB server user", 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.983 INFO:teuthology.orchestra.run.smithi105.stdout: "port": { 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "port", 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8086", 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server port", 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-03-26T18:03:52.984 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "false", 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "threads": { 2024-03-26T18:03:52.985 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threads", 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "32", 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "username": { 2024-03-26T18:03:52.986 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "username", 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "username of InfluxDB server user", 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "verify_ssl": { 2024-03-26T18:03:52.987 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "verify_ssl", 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "true", 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.988 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "insights", 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.989 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.991 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.992 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.993 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "iostat", 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:52.994 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.995 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.996 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.997 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:52.998 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:52.999 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "k8sevents", 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_event_retention_days": { 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ceph_event_retention_days", 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7", 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.000 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "config_check_secs": { 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_check_secs", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "10", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.001 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.002 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.003 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.004 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.005 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.006 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "localpool", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "failure_domain": { 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "failure_domain", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "host", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "failure domain for any created local pool", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T18:03:53.007 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.008 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.009 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.010 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.011 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.012 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "min_size": { 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_size", 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.013 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default min_size for any created local pool", 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: "num_rep": { 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "num_rep", 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.014 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3", 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default replica count for any created local pool", 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_num": { 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_num", 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.015 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "128", 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "prefix": { 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prefix", 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.016 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name prefix for any created local pool", 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "subtree": { 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "subtree", 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.017 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "rack", 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.018 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mds_autoscaler", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.019 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.020 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.021 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.022 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.023 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirroring", 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.024 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.025 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.026 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.027 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.028 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "nfs", 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.029 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.030 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.031 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.034 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "fail_fs": { 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "fail_fs", 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.035 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.036 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.037 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.038 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.039 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.040 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator": { 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm", 2024-03-26T18:03:53.041 INFO:teuthology.orchestra.run.smithi105.stdout: "rook", 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: "test_orchestrator" 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Orchestrator backend", 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_perf_query", 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.043 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.044 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.045 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.046 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.047 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_support", 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.048 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.049 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.050 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.051 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.052 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.053 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_autoscaler", 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.054 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.055 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.056 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.057 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.058 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.059 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "threshold": { 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threshold", 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.060 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1.0", 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "scaling threshold", 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.061 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "progress", 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_pg_recovery_event": { 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_pg_recovery_event", 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.062 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.063 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.064 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.065 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.066 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.067 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.068 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "max_completed_events": { 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_completed_events", 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "50", 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "number of past completed events to remember", 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.069 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long the module is going to sleep", 2024-03-26T18:03:53.070 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus", 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "cache": { 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cache", 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.071 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "exclude_perf_counters": { 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "exclude_perf_counters", 2024-03-26T18:03:53.072 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.073 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.074 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.075 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.076 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.077 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.078 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools": { 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools", 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.079 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "300", 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.080 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_interval": { 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_interval", 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15.0", 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.081 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T18:03:53.082 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "9283", 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T18:03:53.086 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "stale_cache_strategy": { 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stale_cache_strategy", 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "log", 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.087 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "default", 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.088 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "default", 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "error" 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.089 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_support", 2024-03-26T18:03:53.090 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.091 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.092 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.093 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.094 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.095 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_snap_create": { 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_snap_create", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.096 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "mirror_snapshot_schedule": { 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.097 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "trash_purge_schedule": { 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "trash_purge_schedule", 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.098 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "restful", 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_auth": { 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_auth", 2024-03-26T18:03:53.099 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-03-26T18:03:53.100 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.101 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.102 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.103 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.104 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.105 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.106 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.107 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rgw", 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.109 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.110 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.111 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.112 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.113 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rook", 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.114 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.115 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.116 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.117 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.118 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "storage_class": { 2024-03-26T18:03:53.119 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "storage_class", 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "local", 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "selftest", 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.121 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.122 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.123 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.124 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.125 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "roption1": { 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption1", 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.126 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "roption2": { 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption2", 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "xyz", 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.127 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption1": { 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption1", 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.128 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption2": { 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption2", 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.129 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption3": { 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption3", 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.130 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption4": { 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption4", 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.131 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption5": { 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption5", 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.132 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption6": { 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption6", 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.133 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption7": { 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption7", 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.134 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "42", 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "testkey": { 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testkey", 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.135 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "testlkey": { 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testlkey", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.136 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "testnewline": { 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testnewline", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.137 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snap_schedule", 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.138 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_m_granularity": { 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_m_granularity", 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "dump_on_update": { 2024-03-26T18:03:53.139 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dump_on_update", 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "dump database to debug log on update", 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.140 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.141 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.142 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.143 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.144 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.145 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stats", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.146 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.147 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.148 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.149 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.150 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "status", 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.151 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.152 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.153 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.154 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.155 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.156 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telegraf", 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "address": { 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "address", 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.157 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15", 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.158 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.159 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.160 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.161 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.162 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.163 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telemetry", 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_basic": { 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_basic", 2024-03-26T18:03:53.164 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_crash": { 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_crash", 2024-03-26T18:03:53.165 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_device": { 2024-03-26T18:03:53.166 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_device", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_ident": { 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_ident", 2024-03-26T18:03:53.167 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_perf": { 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_perf", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.168 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "contact": { 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "contact", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.169 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "description": { 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "description", 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.170 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "device_url": { 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_url", 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.171 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.172 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.173 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "24", 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "8", 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "last_opt_revision": { 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "last_opt_revision", 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.174 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1", 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard": { 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard", 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.175 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard_description": { 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard_description", 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.176 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.177 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.178 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.179 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.180 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.181 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "organization": { 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "organization", 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.182 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "proxy": { 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "proxy", 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.183 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "url": { 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T18:03:53.184 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "test_orchestrator", 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.185 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.186 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.187 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.188 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.189 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.190 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "volumes", 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.191 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.192 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.193 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.194 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.195 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_clones": { 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_clones", 2024-03-26T18:03:53.196 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4", 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "periodic_async_work": { 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "periodic_async_work", 2024-03-26T18:03:53.197 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Periodically check for async work", 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.198 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_delay": { 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_delay", 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.199 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_no_wait": { 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.200 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix", 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "discovery_interval": { 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "discovery_interval", 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:53.201 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "100", 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "identifier": { 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "identifier", 2024-03-26T18:03:53.202 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:53.203 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.204 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.205 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.206 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.207 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.208 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.209 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_host": { 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_host", 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.210 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_port": { 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_port", 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10051", 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.211 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_sender": { 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_sender", 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.212 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: "modules": [ 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm", 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: "dashboard", 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: "iostat", 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: "nfs", 2024-03-26T18:03:53.213 INFO:teuthology.orchestra.run.smithi105.stdout: "restful" 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "available_modules": [ 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "alerts", 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:53.214 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.215 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.216 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.217 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.218 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.219 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_destination": { 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_destination", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.220 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email address to send alerts to", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_from_name": { 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_from_name", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "Ceph", 2024-03-26T18:03:53.221 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Email From: name", 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_host": { 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_host", 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.222 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP server", 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_password": { 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_password", 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.223 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Password to authenticate with", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_port": { 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_port", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "465", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.224 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP port", 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_sender": { 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_sender", 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.225 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SMTP envelope sender", 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_ssl": { 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_ssl", 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.226 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "smtp_user": { 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "smtp_user", 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.227 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "User to authenticate as", 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "balancer", 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.228 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "active": { 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "active", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.229 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_time": { 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_time", 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0000", 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.230 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "begin_weekday": { 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "begin_weekday", 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.231 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_max_iterations": { 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_max_iterations", 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "25", 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "250", 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.232 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_metrics": { 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_metrics", 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T18:03:53.233 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "crush_compat_step": { 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crush_compat_step", 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.5", 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0.001", 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "0.999", 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "aggressiveness of optimization", 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T18:03:53.234 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "end_time": { 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_time", 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2359", 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T18:03:53.235 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "end_weekday": { 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "end_weekday", 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "0", 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "6", 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.236 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.237 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.238 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.239 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.240 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.241 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "min_score": { 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_score", 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.242 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "upmap", 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.243 INFO:teuthology.orchestra.run.smithi105.stdout: "crush-compat", 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "none", 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "read", 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap", 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap-read" 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Balancer mode", 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_ids": { 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_ids", 2024-03-26T18:03:53.244 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:53.245 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_deviation": { 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_deviation", 2024-03-26T18:03:53.246 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "upmap_max_optimizations": { 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "upmap_max_optimizations", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.247 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm", 2024-03-26T18:03:53.248 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_down_multiplier": { 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_down_multiplier", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.249 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_refresh_rate": { 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_refresh_rate", 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "20", 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.250 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "agent_starting_port": { 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "agent_starting_port", 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4721", 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.251 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_ptrace": { 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_ptrace", 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.252 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_interval": { 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_interval", 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.253 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "autotune_memory_target_ratio": { 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0.7", 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.254 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm_log_destination": { 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephadm_log_destination", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "file", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "file,syslog", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "syslog" 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.255 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "cgroups_split": { 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cgroups_split", 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.256 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "config_checks_enabled": { 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_checks_enabled", 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.257 INFO:teuthology.orchestra.run.smithi105.stdout: "config_dashboard": { 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_dashboard", 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.258 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_alertmanager": { 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_alertmanager", 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_base": { 2024-03-26T18:03:53.259 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_base", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Container image name, without the tag", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_elasticsearch": { 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_elasticsearch", 2024-03-26T18:03:53.260 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "elasticsearch container image", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_grafana": { 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_grafana", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.261 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_haproxy": { 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_haproxy", 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.262 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "HAproxy container image", 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_agent": { 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_agent", 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.263 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger agent container image", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_collector": { 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_collector", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.264 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger collector container image", 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_jaeger_query": { 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_jaeger_query", 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.265 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Jaeger query container image", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_keepalived": { 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_keepalived", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T18:03:53.266 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Keepalived container image", 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_loki": { 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_loki", 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.267 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Loki container image", 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_node_exporter": { 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_node_exporter", 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.268 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_nvmeof": { 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_nvmeof", 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.269 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Nvme-of container image", 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_prometheus": { 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_prometheus", 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.270 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Prometheus container image", 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_promtail": { 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_promtail", 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.271 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Promtail container image", 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "container_image_snmp_gateway": { 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_image_snmp_gateway", 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.272 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "SNMP Gateway container image", 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "container_init": { 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "container_init", 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.273 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "daemon_cache_timeout": { 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "daemon_cache_timeout", 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.274 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "default_cephadm_command_timeout": { 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.275 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "900", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "default_registry": { 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "default_registry", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.276 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "docker.io", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "device_cache_timeout": { 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_cache_timeout", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1800", 2024-03-26T18:03:53.277 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache device inventory", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "device_enhanced_scan": { 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_enhanced_scan", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.278 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "facts_cache_timeout": { 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "facts_cache_timeout", 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:53.279 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "seconds to cache host facts data", 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "grafana_dashboards_path": { 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "grafana_dashboards_path", 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.280 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "host_check_interval": { 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "host_check_interval", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:53.281 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to perform a host check", 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "hw_monitoring": { 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hw_monitoring", 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.282 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "inventory_list_all": { 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "inventory_list_all", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.283 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.284 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.285 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "log_refresh_metadata": { 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_refresh_metadata", 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.286 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.287 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.288 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.289 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.290 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "*", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T18:03:53.291 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "max_count_per_host": { 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_count_per_host", 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of daemons per service per host", 2024-03-26T18:03:53.292 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "max_osd_draining_count": { 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_osd_draining_count", 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T18:03:53.293 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "migration_current": { 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "migration_current", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "internal - do not modify", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.294 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "mode": { 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mode", 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "root", 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm-package", 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "root" 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T18:03:53.295 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "oob_default_addr": { 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "oob_default_addr", 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "169.254.1.1", 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.296 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "prometheus_alerts_path": { 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus_alerts_path", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.297 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_insecure": { 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_insecure", 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T18:03:53.298 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_password": { 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_password", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.299 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_url": { 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_url", 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.300 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "registry_username": { 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "registry_username", 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.301 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "secure_monitoring_stack": { 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "secure_monitoring_stack", 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T18:03:53.302 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "service_discovery_port": { 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "service_discovery_port", 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8765", 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "cephadm service discovery port", 2024-03-26T18:03:53.303 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "ssh_config_file": { 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssh_config_file", 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.304 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "use_agent": { 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_agent", 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.305 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "use_repo_digest": { 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "use_repo_digest", 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.306 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_failed_host_check": { 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_failed_host_check", 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.307 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_daemons": { 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_daemons", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.308 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_on_stray_hosts": { 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_on_stray_hosts", 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.309 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crash", 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.310 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.311 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.312 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.313 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.314 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.315 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "retain_interval": { 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retain_interval", 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "31536000", 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.316 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_recent_interval": { 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_recent_interval", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1209600", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.317 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dashboard", 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T18:03:53.318 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T18:03:53.319 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.320 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_ENABLED": { 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.321 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.322 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.323 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.324 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.325 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.326 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.327 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.328 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.329 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.330 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.331 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.332 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.333 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_URL": { 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_URL", 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.334 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.335 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.336 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.337 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.338 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.339 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "{}", 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.340 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.341 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.342 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.343 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.344 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.345 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.346 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.347 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.348 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.349 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.350 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.351 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.352 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8", 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.353 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "45", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.354 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.355 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "admin", 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.356 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.357 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.358 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.359 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.360 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T18:03:53.361 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "cross_origin_url": { 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cross_origin_url", 2024-03-26T18:03:53.362 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "crt_file": { 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "crt_file", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.363 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "debug": { 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "debug", 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.364 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Enable/disable debug options", 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "jwt_token_ttl": { 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "jwt_token_ttl", 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.365 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "28800", 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.366 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.367 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.368 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.369 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.370 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.371 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "motd": { 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "motd", 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.372 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "The message of the day", 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect_resolve_ip_addr": { 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.373 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.374 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.375 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8080", 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.376 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl_server_port": { 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl_server_port", 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.377 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8443", 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-03-26T18:03:53.378 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "redirect", 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "redirect" 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.379 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.380 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "url_prefix": { 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url_prefix", 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.381 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "devicehealth", 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_monitoring": { 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_monitoring", 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.382 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "monitor device health metrics", 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.383 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.384 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.385 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.386 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.387 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "mark_out_threshold": { 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mark_out_threshold", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "2419200", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.388 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "pool_name": { 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pool_name", 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.389 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "device_health_metrics", 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "retention_period": { 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "retention_period", 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.390 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15552000", 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long to retain device health metrics", 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_frequency": { 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_frequency", 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.391 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "self_heal": { 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "self_heal", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.392 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.393 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "warn_threshold": { 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "warn_threshold", 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.394 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7257600", 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "diskprediction_local", 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.395 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.396 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.397 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.398 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.399 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.400 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "predict_interval": { 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predict_interval", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "86400", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.401 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "predictor_model": { 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "predictor_model", 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "prophetstor", 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.402 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "600", 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.403 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "influx", 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": false, 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "influxdb python module not found", 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "batch_size": { 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "batch_size", 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.404 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5000", 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "database": { 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "database", 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.405 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "ceph", 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": { 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "hostname", 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.406 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server hostname", 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:53.407 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "30", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "5", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.408 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.409 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.410 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.411 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.412 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "password": { 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "password", 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.413 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "password of InfluxDB server user", 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "port": { 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "port", 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.414 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "8086", 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "InfluxDB server port", 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "ssl": { 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ssl", 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.415 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "false", 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "threads": { 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threads", 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.416 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "32", 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "username": { 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "username", 2024-03-26T18:03:53.417 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "username of InfluxDB server user", 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "verify_ssl": { 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "verify_ssl", 2024-03-26T18:03:53.418 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "true", 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.419 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "insights", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.420 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.421 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.422 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.423 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.424 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "iostat", 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.425 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.426 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.427 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.428 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.429 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "k8sevents", 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_event_retention_days": { 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "ceph_event_retention_days", 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.430 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "7", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "config_check_secs": { 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "config_check_secs", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.431 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "10", 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.432 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.433 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.434 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.435 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.436 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.436 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.436 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.436 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.436 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.436 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.436 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.436 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.437 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.437 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.437 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.437 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.437 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.437 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.437 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.437 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.438 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.438 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.438 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.438 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.438 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.438 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.438 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.439 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.439 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.439 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.439 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.439 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "localpool", 2024-03-26T18:03:53.439 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.439 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.439 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.440 INFO:teuthology.orchestra.run.smithi105.stdout: "failure_domain": { 2024-03-26T18:03:53.440 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "failure_domain", 2024-03-26T18:03:53.440 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.440 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.440 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.440 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "host", 2024-03-26T18:03:53.440 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "failure domain for any created local pool", 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.441 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.442 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.443 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.444 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.445 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "min_size": { 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "min_size", 2024-03-26T18:03:53.446 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default min_size for any created local pool", 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "num_rep": { 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "num_rep", 2024-03-26T18:03:53.447 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default replica count for any created local pool", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_num": { 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_num", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.448 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "128", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "prefix": { 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prefix", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.449 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "name prefix for any created local pool", 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "subtree": { 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "subtree", 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.450 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "rack", 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.451 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mds_autoscaler", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.452 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.453 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.454 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirroring", 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.457 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.458 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.459 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.460 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.461 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "nfs", 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.462 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.463 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.464 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.465 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.466 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.467 INFO:teuthology.orchestra.run.smithi105.stdout: "fail_fs": { 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "fail_fs", 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.468 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.469 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.470 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.471 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.472 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.473 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator": { 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "orchestrator", 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "cephadm", 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "rook", 2024-03-26T18:03:53.474 INFO:teuthology.orchestra.run.smithi105.stdout: "test_orchestrator" 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Orchestrator backend", 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_perf_query", 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.475 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.476 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.477 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.478 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.479 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.480 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "osd_support", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.481 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.482 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.483 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.484 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.485 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "pg_autoscaler", 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.486 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.487 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.488 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.489 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.490 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.491 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "threshold": { 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "threshold", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "3.0", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1.0", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "scaling threshold", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T18:03:53.492 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "progress", 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_pg_recovery_event": { 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_pg_recovery_event", 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.493 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.494 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.495 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.496 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.497 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.498 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.499 INFO:teuthology.orchestra.run.smithi105.stdout: "max_completed_events": { 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_completed_events", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "50", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "number of past completed events to remember", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "sleep_interval": { 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "sleep_interval", 2024-03-26T18:03:53.500 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "5", 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "how long the module is going to sleep", 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.501 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "prometheus", 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "cache": { 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cache", 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.502 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "exclude_perf_counters": { 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "exclude_perf_counters", 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.503 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.504 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.505 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.506 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.507 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.508 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.509 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools": { 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools", 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.510 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T18:03:53.511 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "300", 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.512 INFO:teuthology.orchestra.run.smithi105.stdout: "scrape_interval": { 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "scrape_interval", 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15.0", 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.513 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "::", 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.514 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "9283", 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.515 INFO:teuthology.orchestra.run.smithi105.stdout: "stale_cache_strategy": { 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stale_cache_strategy", 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "log", 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.516 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_behaviour": { 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_behaviour", 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "default", 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "default", 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "error" 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.517 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "standby_error_status_code": { 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "standby_error_status_code", 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "500", 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "400", 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "599", 2024-03-26T18:03:53.518 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_support", 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.519 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.520 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.521 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.522 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.523 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.524 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_snap_create": { 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_snap_create", 2024-03-26T18:03:53.525 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10", 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "mirror_snapshot_schedule": { 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T18:03:53.526 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "trash_purge_schedule": { 2024-03-26T18:03:53.527 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "trash_purge_schedule", 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.528 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "restful", 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "enable_auth": { 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enable_auth", 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.529 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "key_file": { 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "key_file", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.530 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.531 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.531 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.531 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.531 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.531 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.531 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.531 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.531 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.532 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.533 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.534 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.535 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.536 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "server_addr": { 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_addr", 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.537 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "server_port": { 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "server_port", 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.538 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rgw", 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.540 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.541 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.542 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.543 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rook", 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.544 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.545 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.546 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.547 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.548 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.549 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "storage_class": { 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "storage_class", 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "local", 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.550 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "selftest", 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.551 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.552 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.553 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.554 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.555 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.556 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.557 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "roption1": { 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption1", 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.558 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "roption2": { 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "roption2", 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "xyz", 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.559 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption1": { 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption1", 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.560 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption2": { 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption2", 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.561 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption3": { 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption3", 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "float", 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.562 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption4": { 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption4", 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.563 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption5": { 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption5", 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.564 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.565 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption6": { 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption6", 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.566 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "rwoption7": { 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rwoption7", 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "1", 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "42", 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.567 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "testkey": { 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testkey", 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.568 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "testlkey": { 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testlkey", 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.569 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "testnewline": { 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "testnewline", 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.570 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snap_schedule", 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "allow_m_granularity": { 2024-03-26T18:03:53.571 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "allow_m_granularity", 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.572 INFO:teuthology.orchestra.run.smithi105.stdout: "dump_on_update": { 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "dump_on_update", 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "dump database to debug log on update", 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.573 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.574 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.575 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.576 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.577 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "stats", 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.579 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.580 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.581 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.582 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.583 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "status", 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.584 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.585 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.586 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.587 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.588 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.589 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telegraf", 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.590 INFO:teuthology.orchestra.run.smithi105.stdout: "address": { 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "address", 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:53.591 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "15", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.592 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.593 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.594 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.595 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.596 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "telemetry", 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.597 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_basic": { 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_basic", 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.598 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_crash": { 2024-03-26T18:03:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_crash", 2024-03-26T18:03:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.599 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_device": { 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_device", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.600 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_ident": { 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_ident", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T18:03:53.601 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "channel_perf": { 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "channel_perf", 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.602 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "contact": { 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "contact", 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.603 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "description": { 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "description", 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.604 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "device_url": { 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "device_url", 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.605 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "enabled": { 2024-03-26T18:03:53.606 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "enabled", 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.607 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "24", 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "8", 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.608 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "last_opt_revision": { 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "last_opt_revision", 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "1", 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.609 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard": { 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard", 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.610 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "leaderboard_description": { 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "leaderboard_description", 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.611 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.612 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.613 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.614 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.615 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.616 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "organization": { 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "organization", 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.617 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "proxy": { 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "proxy", 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.618 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "url": { 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "url", 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.619 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "test_orchestrator", 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.620 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.621 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.622 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.623 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.624 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.625 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "volumes", 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.626 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.627 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.628 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.629 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.630 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.631 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "max_concurrent_clones": { 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "max_concurrent_clones", 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "4", 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.632 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "periodic_async_work": { 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "periodic_async_work", 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.633 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Periodically check for async work", 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_delay": { 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_delay", 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "0", 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.634 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "snapshot_clone_no_wait": { 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "True", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T18:03:53.635 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix", 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "can_run": true, 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "error_string": "", 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "module_options": { 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "discovery_interval": { 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "discovery_interval", 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "uint", 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.636 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "100", 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "identifier": { 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "identifier", 2024-03-26T18:03:53.637 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.638 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "interval": { 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "interval", 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "secs", 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "60", 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.639 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "log_level": { 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_level", 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.640 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster": { 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster", 2024-03-26T18:03:53.641 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.642 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_cluster_level": { 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_cluster_level", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "info", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [ 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "critical", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "debug", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "error", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "info", 2024-03-26T18:03:53.643 INFO:teuthology.orchestra.run.smithi105.stdout: "warning" 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "log_to_file": { 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "log_to_file", 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "bool", 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 1, 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "False", 2024-03-26T18:03:53.644 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_host": { 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_host", 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "", 2024-03-26T18:03:53.645 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_port": { 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_port", 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "int", 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "10051", 2024-03-26T18:03:53.646 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "zabbix_sender": { 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "zabbix_sender", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "str", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "level": "advanced", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "flags": 0, 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "min": "", 2024-03-26T18:03:53.647 INFO:teuthology.orchestra.run.smithi105.stdout: "max": "", 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "enum_allowed": [], 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "desc": "", 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "long_desc": "", 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "tags": [], 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "see_also": [] 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "services": { 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "dashboard": "https://172.21.15.105:8443/" 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "always_on_modules": { 2024-03-26T18:03:53.648 INFO:teuthology.orchestra.run.smithi105.stdout: "octopus": [ 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "pacific": [ 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-03-26T18:03:53.649 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "quincy": [ 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-03-26T18:03:53.650 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "reef": [ 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-03-26T18:03:53.651 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: ], 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "squid": [ 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "balancer", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "crash", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "devicehealth", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "orchestrator", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "pg_autoscaler", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "progress", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "rbd_support", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "status", 2024-03-26T18:03:53.652 INFO:teuthology.orchestra.run.smithi105.stdout: "telemetry", 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "volumes" 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "last_failure_osd_epoch": 3, 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "active_clients": [ 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "devicehealth", 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 2281139888 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.653 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "libcephsqlite", 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1759068294 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "rbd_support", 2024-03-26T18:03:53.654 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 1080855561 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "volumes", 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "addrvec": [ 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-03-26T18:03:53.655 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v2", 2024-03-26T18:03:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "172.21.15.105:0", 2024-03-26T18:03:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 3729821015 2024-03-26T18:03:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:53.656 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-26T18:03:53.656 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:03:53.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:53 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/688985526' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T18:03:53.659 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:52 smithi135 ceph-mon[35636]: pgmap v1118: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:03:53.659 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:52 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1873191862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:03:53.659 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:52 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1012757783' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:03:53.659 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:52 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2518017689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:03:53.659 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:53 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/688985526' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T18:03:53.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76340a2f20 msgr2=0x7f76340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:53.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76340a2f20 0x7f76340a53e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f763c005ab0 tx=0x7f763c005fd0 comp rx=0 tx=0).stop 2024-03-26T18:03:53.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1019d0 msgr2=0x7f764c196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:03:53.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1019d0 0x7f764c196f80 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f764000ea60 tx=0x7f764000ef20 comp rx=0 tx=0).stop 2024-03-26T18:03:53.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 shutdown_connections 2024-03-26T18:03:53.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f76340a2f20 0x7f76340a53e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f764c102390 0x7f764c19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f764c1019d0 0x7f764c196f80 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 --2- 172.21.15.105:0/688985526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f764c1007e0 0x7f764c196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:03:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 >> 172.21.15.105:0/688985526 conn(0x7f764c0fbd50 msgr2=0x7f764c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:03:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 shutdown_connections 2024-03-26T18:03:53.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:03:52.380+0000 7f765115b640 1 -- 172.21.15.105:0/688985526 wait complete. 2024-03-26T18:03:53.660 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:52 smithi105 ceph-mon[32280]: pgmap v1118: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:03:53.660 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:52 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1873191862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:03:53.660 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:52 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1012757783' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:03:53.661 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:52 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2518017689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:03:53.666 DEBUG:tasks.check_counter:Getting stats from a 2024-03-26T18:03:53.667 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-26T18:03:53.667 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:53 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/688985526' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T18:03:53.930 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:03:54.310 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:54 smithi105 ceph-mon[32280]: pgmap v1119: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:03:54.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:54 smithi144 ceph-mon[36095]: pgmap v1119: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:03:54.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:54 smithi135 ceph-mon[35636]: pgmap v1119: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:03:55.737 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:03:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 11553, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 17212, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 3778149, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 13309437, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 18, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 6, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 1.123258312, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.532034658, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.607540999, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.077987264, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 17206, 2024-03-26T18:03:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.364033927, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000021157 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 21488, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.004604204, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000214 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 255869, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 128161, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 69954643, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 21665332, 2024-03-26T18:03:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 10, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 6, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 10.449293635, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 4.133002608, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 8.840320161, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.030760494, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 128164, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.863838585, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000014542 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 380558, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.046383642, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000121 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 1008224, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 299136 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 90095, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 54021, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 24353233, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 17874771, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 106, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 4.322648870, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 1.692474718, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 3.468131033, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.080265525, 2024-03-26T18:03:55.741 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 54020, 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.912752313, 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000016896 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 138204, 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.019687759, 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000142 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 5872, 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 772800 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.742 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7022, 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 3.162453444, 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000450363 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T18:03:55.743 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3587, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.173220534, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000048291 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 6136, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 5861, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5861, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 31.568243187, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.005386153 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 275, 2024-03-26T18:03:55.744 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 17, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 7, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 10, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 13, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 232, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 4, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 16352, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 5325, 2024-03-26T18:03:55.745 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 273, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 123, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 1, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 20, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 1, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 3, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 75, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 692, 2024-03-26T18:03:55.746 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 212797333, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 75, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 75, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 1708, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 3187, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:03:55.747 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 1963, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 42, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 221, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 4335 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 132, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 482, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 298, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 387, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 135, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 1446, 2024-03-26T18:03:55.748 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 1446, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 1, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-26T18:03:55.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 9304, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 9304, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 11, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 11, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-26T18:03:55.750 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 14242794, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3287, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 10.252741810, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003119179 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 22, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1584, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 1562, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 21, 2024-03-26T18:03:55.751 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 217, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 196, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 17, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 2677, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 2660, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 1455, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 1455, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 59948, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 223612 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 15556, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 7, 2024-03-26T18:03:55.752 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 6136, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 22, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 3152, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1443, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 12.332201663, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.008546224 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000252410, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000252410 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.753 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1236, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 4.215386085, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003410506 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.754 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.756 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-26T18:03:55.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.756 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.756 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-26T18:03:55.756 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.757 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 251, 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.193377845, 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000770429 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1484, 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.720164974, 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000485286 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-26T18:03:55.758 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-26T18:03:55.759 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.760 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-26T18:03:55.761 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.762 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1446, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 14.106860210, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.009755781 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-03-26T18:03:55.763 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 1, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 736, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 736, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 1535, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:03:55.764 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:03:55.765 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 293599, 2024-03-26T18:03:55.766 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 123, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7744, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 165, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-26T18:03:55.767 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 88693, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 399, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 6154, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 10268163, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 33, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 6154, 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-26T18:03:55.768 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6154, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 18.656619772, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.003031624 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6154, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 6546 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 6154, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 439, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 5715, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-26T18:03:55.769 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 364, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 3651, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 259, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 1460, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 75, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2024-03-26T18:03:55.770 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-26T18:03:55.771 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 96, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 11, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 12, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 17, 2024-03-26T18:03:55.772 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 5, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 518, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 189 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-26T18:03:55.773 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 6, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 1446, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 1446, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 357513, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 65878347, 2024-03-26T18:03:55.774 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 357513, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 357513, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 65878347, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:03:55.775 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 6154, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 10414209, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 6154, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 10414209, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.776 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 6154, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 6154, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 6154, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 6154, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.777 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 1446, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 146046, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 1446, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 364, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 146046, 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:03:55.778 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x5596e7ea6500": { 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 9304, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 10048490, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 9304, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 3287, 2024-03-26T18:03:55.779 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 10048490, 2024-03-26T18:03:55.780 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:03:55.780 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:55.780 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:55.780 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:55.780 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.780 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:55.780 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:03:56.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:56 smithi144 ceph-mon[36095]: pgmap v1120: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:03:56.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:56 smithi105 ceph-mon[32280]: pgmap v1120: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:03:56.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:56 smithi135 ceph-mon[35636]: pgmap v1120: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:03:56.546 INFO:tasks.check_counter:Daemon mds.a mds.exported=1 2024-03-26T18:03:56.546 INFO:tasks.check_counter:Daemon mds.a mds.imported=1 2024-03-26T18:03:56.546 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=132 2024-03-26T18:03:56.546 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=482 2024-03-26T18:03:56.546 DEBUG:tasks.check_counter:Getting stats from d 2024-03-26T18:03:56.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-26T18:03:56.812 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:03:58.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:03:58 smithi144 ceph-mon[36095]: pgmap v1121: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:03:58.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:03:58 smithi105 ceph-mon[32280]: pgmap v1121: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:03:58.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:03:58 smithi135 ceph-mon[35636]: pgmap v1121: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 4, 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 4, 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 3040, 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 448, 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 0, 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.001536104, 2024-03-26T18:03:59.043 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000032284, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.000623863, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000724208, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000241402 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.044 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 451, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 446, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 820448, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 276112, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 2, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.050597531, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.019728512, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.022545489, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.008418588, 2024-03-26T18:03:59.045 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 445, 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.010481595, 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000023554 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 820448, 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 276112 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.046 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 341, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 320, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 215952, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.018604592, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.014883360, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.001606160, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 339, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.009019310, 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000026605 2024-03-26T18:03:59.047 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 215952 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 0, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:03:59.048 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:03:59.049 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:03:59.050 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 148744, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 30, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-26T18:03:59.051 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 928, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 2, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:03:59.052 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 453, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 756677, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 453, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 453, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 756677, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:03:59.053 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:03:59.054 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:59.054 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:03:59.054 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:04:00.031 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-26T18:04:00.031 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-26T18:04:00.031 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-26T18:04:00.031 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-26T18:04:00.031 DEBUG:tasks.check_counter:Getting stats from g 2024-03-26T18:04:00.031 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-26T18:04:00.286 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:04:00.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:00 smithi144 ceph-mon[36095]: pgmap v1122: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:00.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:00 smithi105 ceph-mon[32280]: pgmap v1122: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:00.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:00 smithi135 ceph-mon[35636]: pgmap v1122: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:01.984 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2283, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2256, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1575202, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 848406, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 5, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.198740063, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.072961985, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.059868412, 2024-03-26T18:04:01.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.056743121, 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2255, 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.051544408, 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000022857 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 984640, 2024-03-26T18:04:01.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 276960 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 3363, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 3698, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 10904590, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1749805, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 5, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.209537134, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.066392140, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.057886096, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.070306150, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3697, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.079513585, 2024-03-26T18:04:01.987 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000021507 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 681904 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 4, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 6, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1629, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 1284, 2024-03-26T18:04:01.988 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.001812895, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000066464, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.001399652, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000083832, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001721480, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000430370 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:01.989 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 8740, 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.183891914, 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000021040 2024-03-26T18:04:01.990 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-26T18:04:01.991 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-26T18:04:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 1456, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 1456, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 1456, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 1, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-26T18:04:01.993 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 0, 2024-03-26T18:04:01.994 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:01.995 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-26T18:04:01.996 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.997 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 9304, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 11, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-26T18:04:01.998 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 14242794, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 14240807, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 9304 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 13, 2024-03-26T18:04:01.999 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 1469, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 1456, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 11, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 22, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 11, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 1456, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 1486, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 30, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 55700, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215420 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-26T18:04:02.000 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:02.001 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.002 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:02.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.004 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.010 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:02.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 1, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 3819, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:02.012 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:02.013 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:02.014 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 198287, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 57, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:02.015 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 165, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 1068408, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 5952, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-26T18:04:02.016 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 5159, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 21, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 3, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 5159, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5159, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.240954109, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000434377 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5159, 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 5160 2024-03-26T18:04:02.017 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 5159, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 5159, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 3191, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 1966, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-03-26T18:04:02.018 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 1, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-26T18:04:02.019 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-26T18:04:02.020 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 96, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 11, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 6, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 8, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 2, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-26T18:04:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:02.022 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 5646, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 11945437, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 5646, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5646, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 11945437, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:02.023 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 5159, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 10048495, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5159, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 10048495, 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:04:02.024 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 5159, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 5159, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5159, 2024-03-26T18:04:02.025 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 5159, 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:04:02.026 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.027 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55e9df8fa6e0": { 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T18:04:02.028 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-26T18:04:02.029 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-26T18:04:02.029 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:04:02.029 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:02.029 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:02.029 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:02.029 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:02.029 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:02.029 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:04:02.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:02 smithi144 ceph-mon[36095]: pgmap v1123: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:02.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:02 smithi105 ceph-mon[32280]: pgmap v1123: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:02.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:02 smithi135 ceph-mon[35636]: pgmap v1123: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:02.778 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-26T18:04:02.778 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-26T18:04:02.779 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-26T18:04:02.779 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-03-26T18:04:02.779 DEBUG:tasks.check_counter:Getting stats from j 2024-03-26T18:04:02.779 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-26T18:04:03.027 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T18:04:04.129 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:04 smithi105 ceph-mon[32280]: pgmap v1124: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:04.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:04 smithi144 ceph-mon[36095]: pgmap v1124: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:04.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:04 smithi135 ceph-mon[35636]: pgmap v1124: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:04.777 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-26T18:04:04.777 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:04.777 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 0, 2024-03-26T18:04:04.777 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2, 2024-03-26T18:04:04.777 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 0, 2024-03-26T18:04:04.777 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 0, 2024-03-26T18:04:04.777 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-03-26T18:04:04.777 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 0, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.001160544, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.000292724, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.778 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 343, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 320, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 216304, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 4, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 2, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.017882472, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.014259568, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.001524004, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 341, 2024-03-26T18:04:04.779 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.010691871, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000031354 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 216304 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 458, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 451, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 824304, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 278480, 2024-03-26T18:04:04.780 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 2, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.051342439, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.020072540, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.023312575, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.008393789, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 451, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.011571269, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000025656 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.781 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 824304, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 278480 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 0, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:04.782 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:04.783 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:04.784 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 148744, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 30, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:04.785 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 928, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 2, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 456, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 757067, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:04.786 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 456, 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 456, 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 757067, 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-26T18:04:04.787 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-26T18:04:05.502 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-26T18:04:05.502 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-26T18:04:05.502 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-26T18:04:05.502 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-26T18:04:05.502 DEBUG:tasks.check_counter:Getting stats from b 2024-03-26T18:04:05.502 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-26T18:04:05.959 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:04:06.331 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:06 smithi135 ceph-mon[35636]: pgmap v1125: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:06.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:06 smithi144 ceph-mon[36095]: pgmap v1125: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:06.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:06 smithi105 ceph-mon[32280]: pgmap v1125: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:08.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:08 smithi144 ceph-mon[36095]: pgmap v1126: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:08.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:08 smithi105 ceph-mon[32280]: pgmap v1126: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:08.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:08 smithi135 ceph-mon[35636]: pgmap v1126: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:08.509 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-03-26T18:04:08.509 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:08.509 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 172543, 2024-03-26T18:04:08.509 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 300250, 2024-03-26T18:04:08.509 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 35894645, 2024-03-26T18:04:08.509 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 242213250, 2024-03-26T18:04:08.509 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 36, 2024-03-26T18:04:08.509 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 16.299885829, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 7.820987607, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 7.538046958, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.534036159, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 300250, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 16.719191301, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000055684 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 380875, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.075701824, 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000198 2024-03-26T18:04:08.510 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 1046048, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 343664 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 305765, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 558933, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 61071684, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 309479285, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 10, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 6, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 19.516937643, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 8.250192068, 2024-03-26T18:04:08.511 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 10.369881553, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.589350173, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 558931, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 23.359487384, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000041793 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 766444, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.122259110, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000159 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 693440 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.512 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 2295874, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 2542409, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 664312322, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 1612609402, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 593, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 5, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 96.243612539, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 41.050761533, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 65.722870413, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.801708751, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2542407, 2024-03-26T18:04:08.513 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 44.765622419, 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000017607 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4683274, 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.638369310, 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000136 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 89008, 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 31872 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-03-26T18:04:08.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 114218, 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 179.835445596, 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001574493 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 66576, 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 13.620532037, 2024-03-26T18:04:08.515 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000204586 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 1884542, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 1881514, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1881514, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 5172.976835508, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002749369 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 3028, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 4, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 1, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 5890, 2024-03-26T18:04:08.516 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 147, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 51, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 85, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 11, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 73, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 225, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 2, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 2, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 3, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 3682926, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 3575512, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 1642, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 12, 2024-03-26T18:04:08.517 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 1920, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 15, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 183, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 6, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 10, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 15316, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 4, 2024-03-26T18:04:08.518 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 234, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 15316, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 2742, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 48, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 2694, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 12574, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 8, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 89454, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 100182, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:08.519 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 143706, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 42872, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 1445, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 125448 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 444, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 27, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 18, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 22, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 1120, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 16, 2024-03-26T18:04:08.520 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 34511, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 34495, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:08.521 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 48, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 15, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 215031, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 212973, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 212973, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 2058, 2024-03-26T18:04:08.522 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 2359, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 329, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 326, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 326, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 3, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 1, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 4, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 451419332, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 455510942, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 56450, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 213.283174492, 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003778267 2024-03-26T18:04:08.523 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 43, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 34786, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 34743, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 38, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 147, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 109, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 147, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 75960, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 75813, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 2, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 34722, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 34720, 2024-03-26T18:04:08.524 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 81532, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 231804 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 2014928, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 3023, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 1884542, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 3767, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 8956, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 34585, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 153.341365652, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.004433753 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-03-26T18:04:08.525 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1680255, 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4166.247709261, 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002479532 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.526 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 58179, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 220.690607300, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003793303 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.527 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.528 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.529 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 119, 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.810245748, 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.006808787 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.530 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 19625, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 67.612521698, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003445224 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 45868, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 366.284608849, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.007985624 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 7232, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 39.145984117, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.005412884 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.531 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 105, 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.854882970, 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.008141742 2024-03-26T18:04:08.532 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000841223, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000841223 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.533 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1120, 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 2.857040873, 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002550929 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.534 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 8, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.020288752, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002536094 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 34417, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 155.110739065, 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.004506805 2024-03-26T18:04:08.535 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 3, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 2, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 128791, 2024-03-26T18:04:08.536 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 128791, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 1548, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:08.537 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:08.538 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 310807, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 106, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-03-26T18:04:08.539 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 165, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 261715, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 3646, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:08.540 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 132878, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 479843375, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 37, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 132878, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 132878, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 818.737035805, 2024-03-26T18:04:08.541 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.006161569 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 132878, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 171023 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 132878, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 18236, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 114642, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 5784, 2024-03-26T18:04:08.542 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 10684, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 2867, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 59269, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 1906, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 35220, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 15369, 2024-03-26T18:04:08.543 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 21368, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-03-26T18:04:08.544 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 96, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-03-26T18:04:08.545 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 12, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 12, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 12, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 14369, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 10, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 4177 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-03-26T18:04:08.546 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 6, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 34495, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 34495, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:08.547 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 2774178, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 511558744, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 2774178, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 2774178, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 511558744, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.548 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 132878, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 483327302, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 132878, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 483327302, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.549 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 132878, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 132878, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 132878, 2024-03-26T18:04:08.550 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 132878, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 34495, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 3483995, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 34495, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:08.551 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 2819, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 3483995, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x55ef2c286500": { 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 215031, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 451316638, 2024-03-26T18:04:08.552 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 215031, 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 56450, 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 451316638, 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:08.553 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-03-26T18:04:09.496 INFO:tasks.check_counter:Daemon mds.b mds.exported=15 2024-03-26T18:04:09.496 INFO:tasks.check_counter:Daemon mds.b mds.imported=6 2024-03-26T18:04:09.496 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=444 2024-03-26T18:04:09.496 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=27 2024-03-26T18:04:09.496 DEBUG:tasks.check_counter:Getting stats from e 2024-03-26T18:04:09.496 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-26T18:04:09.755 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:04:10.153 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:10 smithi135 ceph-mon[35636]: pgmap v1127: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:10.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:10 smithi105 ceph-mon[32280]: pgmap v1127: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:10.380 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:10 smithi144 ceph-mon[36095]: pgmap v1127: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:11.921 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 156, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 158, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 4245123, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 49627, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 7, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.016812138, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.003931829, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.007034219, 2024-03-26T18:04:11.922 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.003755842, 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 156, 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.006532074, 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000041872 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:04:11.923 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 174, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 168, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 12729148, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 62389, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 5, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.020582562, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.004096249, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.014099220, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.003233875, 2024-03-26T18:04:11.924 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:11.925 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 168, 2024-03-26T18:04:11.925 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.011714998, 2024-03-26T18:04:11.925 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000069732 2024-03-26T18:04:11.925 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.925 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:11.925 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.925 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.925 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 190144, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 23536 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 182, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 209, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 10960787, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 189959, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 5, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-03-26T18:04:11.926 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.019522633, 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.004173232, 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.010423000, 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.003846322, 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 207, 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.005738616, 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000027722 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.927 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 132400 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-03-26T18:04:11.928 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 764, 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.046875365, 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000061355 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.929 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 0, 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 0, 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.930 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 2635, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 1262, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 1373, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 831, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-03-26T18:04:11.931 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 2, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 0, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 0, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:11.932 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 4, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 234, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-03-26T18:04:11.933 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:04:11.934 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-03-26T18:04:11.935 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-03-26T18:04:11.936 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 0, 2024-03-26T18:04:11.937 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 13171, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 3, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 6, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 451419332, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 455510942, 2024-03-26T18:04:11.938 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 455509804, 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 13171 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-03-26T18:04:11.939 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 33, 2024-03-26T18:04:11.940 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 2480, 2024-03-26T18:04:11.940 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 2447, 2024-03-26T18:04:11.940 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 32, 2024-03-26T18:04:11.940 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 110, 2024-03-26T18:04:11.940 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 78, 2024-03-26T18:04:11.940 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 2635, 2024-03-26T18:04:11.940 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 3620, 2024-03-26T18:04:11.940 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 985, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 0, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 52772, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 215396 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 0, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 0, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 0, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-03-26T18:04:11.941 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.942 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:11.943 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.944 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.945 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.946 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-03-26T18:04:11.947 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.948 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-03-26T18:04:11.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.950 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.952 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.952 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:11.952 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.952 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.952 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.952 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.952 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-03-26T18:04:11.952 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-03-26T18:04:11.953 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:11.954 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 1, 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 0, 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-03-26T18:04:11.955 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 0, 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 0, 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 2684, 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:11.956 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:11.957 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:11.958 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 214741, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 59, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-03-26T18:04:11.959 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 165, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 1970278, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 12194, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:11.960 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 459, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 26, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 4, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 459, 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-03-26T18:04:11.961 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 459, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.301993733, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000657938 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 459, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 460 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 459, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 459, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 0, 2024-03-26T18:04:11.962 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 247, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 0, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 209, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 0, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 0, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-03-26T18:04:11.963 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 2, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-03-26T18:04:11.964 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-03-26T18:04:11.965 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 8, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 8, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-03-26T18:04:11.966 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 0, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 2, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 0 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 0, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:04:11.967 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 508, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 27883851, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 508, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:11.968 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 508, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 27883851, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:11.969 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 459, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 27616307, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 459, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 27616307, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:11.970 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 459, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 459, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 459, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 459, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:11.971 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x5564ed7d86e0": { 2024-03-26T18:04:11.972 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:11.973 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-03-26T18:04:12.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:12 smithi144 ceph-mon[36095]: pgmap v1128: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:12.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:12 smithi105 ceph-mon[32280]: pgmap v1128: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:12.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:12 smithi135 ceph-mon[35636]: pgmap v1128: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:13.056 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-26T18:04:13.056 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-26T18:04:13.056 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-26T18:04:13.056 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-26T18:04:13.056 DEBUG:tasks.check_counter:Getting stats from h 2024-03-26T18:04:13.056 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-26T18:04:13.379 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:04:14.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:14 smithi144 ceph-mon[36095]: pgmap v1129: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:14.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:14 smithi105 ceph-mon[32280]: pgmap v1129: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:14.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:14 smithi135 ceph-mon[35636]: pgmap v1129: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 255522, 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 127747, 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 69929829, 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 21468859, 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 9, 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 12.375027164, 2024-03-26T18:04:15.182 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 5.554782542, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 9.486152835, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.013947539, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 127750, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1.890858739, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000014801 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 381174, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.053809631, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000141 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.183 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 1016496, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 284800 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 84628, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 42658, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 22930109, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 8845184, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 105, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 8, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 3.251882912, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 1.214225314, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 2.800116211, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000367748, 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:15.184 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 42657, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.619431036, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000014521 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 126775, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.016018180, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000126 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 687280 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1012, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1030, 2024-03-26T18:04:15.185 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 362011, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 521460, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 14, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 6, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.060145632, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.024062501, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.042183273, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.000199957, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1024, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.027695630, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000027046 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:15.186 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2001, 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000320931, 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000160 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-03-26T18:04:15.187 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 48, 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.001329153, 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000027690 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000004269, 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000002134 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 556, 2024-03-26T18:04:15.188 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 270, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 270, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4.585769953, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.016984333 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 286, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 11, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 18, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 2, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 16, 2024-03-26T18:04:15.189 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 13, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 226, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 1, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 1, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 4, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 1201, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 792, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 286, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 123, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-03-26T18:04:15.190 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 692, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 212797333, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:15.191 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 276, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 3, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 277, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 3, 2024-03-26T18:04:15.192 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 604, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 358, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 514, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:15.193 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-03-26T18:04:15.194 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 3, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 3, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 2, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 2, 2024-03-26T18:04:15.195 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 2, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 4195385, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.003233133, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001616566 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 23, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 136, 2024-03-26T18:04:15.196 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 113, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 22, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 281, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 259, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 18, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 244, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 226, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 1, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 7, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 6, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 56256, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 223612 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-03-26T18:04:15.197 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 1370, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 7, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 556, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 18, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 282, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000337858, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000337858 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.198 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 269, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4.585432095, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.017046215 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.199 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.200 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.201 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.202 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.203 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:15.204 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:15.205 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:15.206 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.207 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.208 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 1, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 45, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 45, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 1552, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:15.209 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:15.210 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:15.211 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 259475, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 110, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 165, 2024-03-26T18:04:15.212 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 92789, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 358, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 41, 2024-03-26T18:04:15.213 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 12354, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 28, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 41, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 41, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.226995190, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.005536468 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 41, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 68 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 41, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 0, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 41, 2024-03-26T18:04:15.214 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 10, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 2, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 6, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 6, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:15.215 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 20, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-03-26T18:04:15.216 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 96, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 11, 2024-03-26T18:04:15.217 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 8, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 13, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 5, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 21, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 3 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-03-26T18:04:15.218 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 341158, 2024-03-26T18:04:15.219 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 62488161, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 341158, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 341158, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 62488161, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-03-26T18:04:15.220 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 41, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 12354, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 41, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 12354, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:15.221 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 41, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 41, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 41, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 41, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.222 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.223 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x561ac7ede500": { 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 3, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 1081, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 3, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 2, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1081, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:15.224 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:15.225 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:15.225 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.225 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:15.225 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-03-26T18:04:16.094 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-26T18:04:16.094 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-26T18:04:16.094 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-26T18:04:16.094 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=604 2024-03-26T18:04:16.094 DEBUG:tasks.check_counter:Getting stats from k 2024-03-26T18:04:16.095 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-26T18:04:16.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:16 smithi144 ceph-mon[36095]: pgmap v1130: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:16.361 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T18:04:16.382 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:16 smithi105 ceph-mon[32280]: pgmap v1130: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:16.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:16 smithi135 ceph-mon[35636]: pgmap v1130: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:18.083 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 500, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 470, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 1006018, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 283684, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 6, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 1, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.053243258, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.018676731, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.027089764, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.008692162, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 469, 2024-03-26T18:04:18.084 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.012353184, 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000026339 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 1005184, 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 283360 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1565, 2024-03-26T18:04:18.085 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1907, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 386982, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 1184069, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 5, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.121924972, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.038974076, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.025052639, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.046672828, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1906, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.044459185, 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000023325 2024-03-26T18:04:18.086 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 687984 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 3340, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 3342, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 999628, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 1062178, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 4, 2024-03-26T18:04:18.087 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.213402727, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.071736345, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.052594928, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.072608848, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3340, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.072308470, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000021649 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.088 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 8104, 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.183266395, 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000022614 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.089 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 0, 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 0, 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.090 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 0, 2024-03-26T18:04:18.091 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 1, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-03-26T18:04:18.092 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-03-26T18:04:18.093 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-03-26T18:04:18.094 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:18.095 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 3, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-03-26T18:04:18.096 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 0, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 2, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 2, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 4195385, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 4194892, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.097 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 3 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 3, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 3, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 0, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 1, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 1, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 0, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 0, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 0, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 0, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 0, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-03-26T18:04:18.098 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 50264, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 215420 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 0, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 0, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 0, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.099 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.100 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.101 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.102 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.103 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-03-26T18:04:18.103 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.103 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.103 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.103 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.104 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.105 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.106 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.107 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.108 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.109 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 1, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 0, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 0, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 2691, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.110 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:18.111 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:18.112 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 190167, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 52, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:18.113 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 165, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 8464, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 13, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-03-26T18:04:18.114 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 4901, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 21, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 3, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 4901, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4901, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 2.120955879, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000432759 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-03-26T18:04:18.115 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4901, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4902 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 4901, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 4901, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 0, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 3226, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 0, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 1673, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 0, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 0, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-03-26T18:04:18.116 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 1, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-03-26T18:04:18.117 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-03-26T18:04:18.118 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 96, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 11, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 5, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 6, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 1, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 0, 2024-03-26T18:04:18.119 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 2, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 0 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 0, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:04:18.120 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 5401, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 1878047, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 5401, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:18.121 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 5401, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1878047, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:18.122 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 4901, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 1086, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4901, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1086, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:18.123 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 4901, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 4901, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4901, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 4901, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.124 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:18.125 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x55f8e57a66e0": { 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:18.126 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-03-26T18:04:18.127 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-03-26T18:04:18.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:18 smithi144 ceph-mon[36095]: pgmap v1131: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:18.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:18 smithi105 ceph-mon[32280]: pgmap v1131: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:18.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:18 smithi135 ceph-mon[35636]: pgmap v1131: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:18.745 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-26T18:04:18.745 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-26T18:04:18.745 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-26T18:04:18.745 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-26T18:04:18.745 DEBUG:tasks.check_counter:Getting stats from c 2024-03-26T18:04:18.746 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-26T18:04:19.143 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:04:20.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:20 smithi144 ceph-mon[36095]: pgmap v1132: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:20.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:20 smithi105 ceph-mon[32280]: pgmap v1132: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:20.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:20 smithi135 ceph-mon[35636]: pgmap v1132: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:20.925 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T18:04:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:20.925 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 3177, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 3330, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 906456, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 6391205, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 62, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.243913809, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.085170916, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.065737507, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.073816517, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3324, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.081545403, 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000024532 2024-03-26T18:04:20.926 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 338, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000051500, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000152 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 375952 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 342721, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 175422, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 92610453, 2024-03-26T18:04:20.927 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 33301676, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 14, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 6, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 13.199991666, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 5.264614413, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 10.795953293, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.049836733, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 175425, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2.367802268, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000013497 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 512931, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.066718602, 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000130 2024-03-26T18:04:20.928 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 402528 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 8257, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 10789, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 3510552, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 11596878, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 58, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 7, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.529925323, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.218121309, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.355190959, 2024-03-26T18:04:20.929 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.013070857, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 10788, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.194567773, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000018035 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 16769, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.002889521, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000172 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1020928, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 313520 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.930 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5542, 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.878414199, 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000338941 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:20.931 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2886, 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.097586609, 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000033813 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 4449, 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 4292, 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 4292, 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 16.248139408, 2024-03-26T18:04:20.932 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.003785680 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 157, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 11, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 18, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 3, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 15, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 13, 2024-03-26T18:04:20.933 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 226, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 4, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 10358, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 5180, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 157, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 114, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-26T18:04:20.934 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 2, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 2, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 31, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 758, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 692, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 212797333, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 758, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:20.935 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 758, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 629, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 2147, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 2346, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 1093, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 256, 2024-03-26T18:04:20.936 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 2078 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 188, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 474, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 293, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 367, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 215, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 602, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 602, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:20.937 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 2, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.938 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 6310, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 6310, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 8, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 8, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-26T18:04:20.939 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 11415173, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2299, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 5.779338354, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002513848 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 23, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 738, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 715, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 22, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 207, 2024-03-26T18:04:20.940 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 185, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 18, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 1401, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 1383, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 608, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 608, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 62080, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223612 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 8972, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 7, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 4449, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 22, 2024-03-26T18:04:20.941 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 1523, 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 571, 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 4.695082083, 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.008222560 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:20.942 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1381, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 4.134143590, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002993586 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:20.943 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.944 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-26T18:04:20.945 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.946 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 778, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.544462982, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000699823 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 859, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.742028905, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000863828 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.947 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 101, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.952517256, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.009430863 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.948 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-26T18:04:20.949 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-26T18:04:20.950 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.951 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 602, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 5.179904592, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.008604492 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-26T18:04:20.952 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 603, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 603, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 1559, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:20.953 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:20.954 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:20.955 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 284981, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 119, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:20.956 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 165, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 92701, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 391, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 5229, 2024-03-26T18:04:20.957 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 7433073, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 33, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 5229, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5229, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 10.957815265, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002095585 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5229, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 5797 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 5229, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 1163, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 4066, 2024-03-26T18:04:20.958 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 405, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 2704, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 308, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 615, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 758, 2024-03-26T18:04:20.959 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-26T18:04:20.960 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 96, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 11, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 12, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 17, 2024-03-26T18:04:20.961 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 5, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 670, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 307 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-26T18:04:20.962 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 6, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 602, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 602, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 354151, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 65121918, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:20.963 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 354151, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 354151, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 65121918, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 5229, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 7493875, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 5229, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 7493875, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-26T18:04:20.965 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 5229, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 5229, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 5229, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 5229, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.966 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 602, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 60802, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 602, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 405, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 60802, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.967 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55f7352f2500": { 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 6310, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 7220869, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 6310, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 2299, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 7220869, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:20.968 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.969 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:20.969 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T18:04:21.683 INFO:tasks.check_counter:Daemon mds.c mds.exported=2 2024-03-26T18:04:21.684 INFO:tasks.check_counter:Daemon mds.c mds.imported=2 2024-03-26T18:04:21.684 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=188 2024-03-26T18:04:21.684 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=474 2024-03-26T18:04:21.684 DEBUG:tasks.check_counter:Getting stats from f 2024-03-26T18:04:21.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-26T18:04:21.993 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:04:22.294 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:22 smithi144 ceph-mon[36095]: pgmap v1133: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:22.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:22 smithi105 ceph-mon[32280]: pgmap v1133: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:22.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:22 smithi135 ceph-mon[35636]: pgmap v1133: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:24.094 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T18:04:24.094 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:24.094 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 2732, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2700, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 38697211, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 991986, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.218456256, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.061984793, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.087085657, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.058292682, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2700, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.061031635, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000022604 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.095 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1007680, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 282528 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 687, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 1030, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 50508179, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 994468, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.091219762, 2024-03-26T18:04:24.096 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.021625485, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.049455690, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.014347394, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1028, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.025146404, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000024461 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 776576 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.097 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 2726, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2728, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 71996160, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 866875, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.190977918, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.045272992, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.080410604, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.054919515, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2726, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.060482057, 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000022187 2024-03-26T18:04:24.098 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T18:04:24.099 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 9648, 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.180092170, 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000018666 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-26T18:04:24.100 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 11393, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-26T18:04:24.101 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 11393, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 11393, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-26T18:04:24.102 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:04:24.103 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-26T18:04:24.104 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-26T18:04:24.105 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 81393, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-26T18:04:24.106 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 114, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 8, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 162815303, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 162814165, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.107 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 81393 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 15, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 11386, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 11371, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 13, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 50, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 37, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 11393, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 11713, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 320, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 63160, 2024-03-26T18:04:24.108 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.110 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:24.111 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.111 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.111 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.111 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.111 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-26T18:04:24.111 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.111 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.111 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.112 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.113 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.115 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:24.116 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.117 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.118 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-26T18:04:24.119 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 1, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 2692, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:24.120 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:24.121 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:24.122 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 232215, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 78, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 165, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:24.123 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 8071063, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 45886, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 5642, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 5642, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5642, 2024-03-26T18:04:24.124 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 3.266944595, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000579040 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5642, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 5643 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 5642, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 5642, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 3220, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 2420, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-26T18:04:24.125 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:24.126 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-26T18:04:24.127 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 96, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 11, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 10, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 11, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 1, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-26T18:04:24.128 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:04:24.129 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 6141, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 160620430, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 6141, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:24.130 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 6141, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 160620430, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:24.131 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 5642, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 158624296, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 5642, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 158624296, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-26T18:04:24.132 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 5642, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 5642, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 5642, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 5642, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:24.133 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.134 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55950ecb26e0": { 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:24.135 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:24.136 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.136 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:24.136 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T18:04:24.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:24 smithi144 ceph-mon[36095]: pgmap v1134: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:24.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:24 smithi105 ceph-mon[32280]: pgmap v1134: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:24.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:24 smithi135 ceph-mon[35636]: pgmap v1134: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:25.048 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-26T18:04:25.048 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-26T18:04:25.048 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-26T18:04:25.049 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-26T18:04:25.049 DEBUG:tasks.check_counter:Getting stats from i 2024-03-26T18:04:25.049 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-26T18:04:25.349 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:04:26.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:26 smithi144 ceph-mon[36095]: pgmap v1135: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:26.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:26 smithi105 ceph-mon[32280]: pgmap v1135: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:26.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:26 smithi135 ceph-mon[35636]: pgmap v1135: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:27.103 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T18:04:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 363407, 2024-03-26T18:04:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 317762, 2024-03-26T18:04:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 88631733, 2024-03-26T18:04:27.103 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 119524696, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 10, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 5, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 16.272717874, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 7.010825443, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 11.433456799, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.314915364, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 317761, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 4.247861793, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000013368 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.104 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 652520, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.090272602, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000138 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 673901, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 762500, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 195432485, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 502152803, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 13, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 7, 2024-03-26T18:04:27.105 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 29.165129293, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 12.415460057, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 19.904128506, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.359717590, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 762499, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 10.538179686, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000013820 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1391598, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.181168172, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000130 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.106 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1028368, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 314704 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 58554, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 101204, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 12291596, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 85747263, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 105, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 8, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 4.426819223, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 1.831732098, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 2.209203496, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.451664641, 2024-03-26T18:04:27.107 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 101202, 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.696434766, 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000016762 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 126676, 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.025307827, 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000199 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 779648 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.108 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 50143, 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 50.575988922, 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.001008635 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T18:04:27.109 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 23470, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 1.969802951, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000083928 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 555900, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 553964, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 553964, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 353.830929784, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000638725 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-26T18:04:27.110 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 1936, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 11, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 21, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 4, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 3, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 14, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 14, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 208, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 2, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 2, 2024-03-26T18:04:27.111 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 3, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 1083930, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 1044307, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 452, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 119, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 711, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 4, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 10, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 13, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 169, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:27.112 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 3774, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 692, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 212797333, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 3774, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 418, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 418, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 3356, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 8, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 28286, 2024-03-26T18:04:27.113 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 34219, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 50392, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 16874, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 719, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 41733 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 28, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 454, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 282, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 361, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 181, 2024-03-26T18:04:27.114 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 2, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 11354, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 11352, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:04:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 4, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 81393, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 81393, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-26T18:04:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 114, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 114, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 8, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 162815303, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 0, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 24190, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 69.589842029, 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002876802 2024-03-26T18:04:27.117 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 0 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 19, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 11469, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 11450, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 18, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 203, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 185, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 21, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 18518, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 18497, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 2, 2024-03-26T18:04:27.118 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 11366, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 11364, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 64940, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 223612 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 597885, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 5756, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 555900, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 1392, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 12511, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 11205, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 33.524725932, 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002991943 2024-03-26T18:04:27.119 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 484554, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 185.860283480, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000383569 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.120 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 20093, 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 16.390046198, 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000815709 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.121 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.122 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-26T18:04:27.123 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-26T18:04:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 8297, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 8.044116119, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000969521 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 15458, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 70.507948199, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.004561259 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-26T18:04:27.125 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2606, 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 9.390920754, 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.003603576 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.126 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 19, 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.142029816, 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.007475253 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.127 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 1, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.009517666, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.009517666 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 396, 2024-03-26T18:04:27.128 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.197612776, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000499022 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.129 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 11335, 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 29.763728844, 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002625825 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.130 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 1, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 0, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 63977, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 63977, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 1565, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.131 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:27.132 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:27.133 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 303511, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 109, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-26T18:04:27.134 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 165, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 83863, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 670, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:27.135 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 43480, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 160647297, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 28, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 43480, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 43480, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 122.826931761, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.002824906 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.136 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 43480, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 44941 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 43480, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 5100, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 38380, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 0, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 10, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 1326, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 25516, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 717, 2024-03-26T18:04:27.137 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 11401, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 3774, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 20, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-26T18:04:27.138 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-26T18:04:27.139 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 96, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 11, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 12, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 13, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 1, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 1456, 2024-03-26T18:04:27.140 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 0, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 721 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 1, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 6, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-26T18:04:27.141 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 6, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 11352, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 11352, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 1095858, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 197696857, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 1095858, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:27.142 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1095858, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 197696857, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:27.143 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 43480, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 161793849, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 43480, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 161793849, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:27.144 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 43480, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 43480, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 43480, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 43480, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:27.145 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 11352, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 1146552, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 11352, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 1326, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 1146552, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.146 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55863743e500": { 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 81393, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 158620999, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 81393, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 24190, 2024-03-26T18:04:27.147 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 158620999, 2024-03-26T18:04:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.148 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:27.148 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T18:04:27.815 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2024-03-26T18:04:27.815 INFO:tasks.check_counter:Daemon mds.i mds.imported=13 2024-03-26T18:04:27.815 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=28 2024-03-26T18:04:27.815 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=454 2024-03-26T18:04:27.815 DEBUG:tasks.check_counter:Getting stats from l 2024-03-26T18:04:27.815 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab shell --fsid 15cee578-eb96-11ee-95d0-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-26T18:04:28.077 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:28 smithi144 ceph-mon[36095]: pgmap v1136: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:28.102 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T18:04:28.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:28 smithi105 ceph-mon[32280]: pgmap v1136: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:28.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:28 smithi135 ceph-mon[35636]: pgmap v1136: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 3582, 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 3584, 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 5250045, 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 1139077, 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 7, 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.232436382, 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.073261633, 2024-03-26T18:04:29.977 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.060331280, 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.080262367, 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 3582, 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.072273088, 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000020176 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.978 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 9, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 354, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 4750, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 780366, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 8, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 4, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.015245123, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.010479384, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.002568272, 2024-03-26T18:04:29.979 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.000073632, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 353, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.012916712, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000036591 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 779344 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T18:04:29.980 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_messages": 2237, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages": 2207, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_bytes": 4465173, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_bytes": 837010, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_created_connections": 5, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_active_connections": 3, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_total_time": 0.175274328, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_send_time": 0.058777417, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_recv_time": 0.055644772, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_running_fast_dispatch_time": 0.053737939, 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T18:04:29.981 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 2206, 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.045744072, 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000020736 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_handle_ack_lat": { 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_recv_encrypted_bytes": 1006000, 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "msgr_send_encrypted_bytes": 285280 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "cct": { 2024-03-26T18:04:29.982 INFO:teuthology.orchestra.run.smithi144.stdout: "total_workers": 1, 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "unhealthy_workers": 0 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-MDSRank": { 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 8856, 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.187202625, 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000021138 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "finisher-PurgeQueue": { 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "queue_len": 0, 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "complete_latency": { 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.983 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "request": 0, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "reply": 0, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "reply_latency": { 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_reply": 0, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "forward": 0, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_complete": 0, 2024-03-26T18:04:29.984 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_fetch_keys": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_commit": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_split": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_merge": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes": 609, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_top": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_bottom": 609, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pin_tail": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_pinned": 609, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_expired": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "inodes_with_caps": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "caps": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "subtrees": 1, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse": 0, 2024-03-26T18:04:29.985 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_hit": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_forward": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_discover": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_dir_fetch": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_remote_ino": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "traverse_lock": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "load_cent": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "q": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "exported": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "exported_inodes": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "imported": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "imported_inodes": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_dir_fetch": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_backtrace_fetch": 0, 2024-03-26T18:04:29.986 INFO:teuthology.orchestra.run.smithi144.stdout: "openino_peer_discover": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rfiles": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rbytes": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "root_rsnaps": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_set_tag": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_inotable_repaired": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_inodes": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "scrub_file_inodes": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_inode_file_caps": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_grant": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T18:04:29.987 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_caps_dirty": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_cap_release": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "process_request_cap_release": 0 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_cache": { 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_update_receipt": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_try_discover": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_send_discover": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "dir_handle_discover": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_delayed": 0, 2024-03-26T18:04:29.988 INFO:teuthology.orchestra.run.smithi144.stdout: "num_strays_enqueuing": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_created": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_enqueued": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_reintegrated": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "strays_migrated": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_processing": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_enqueued": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "num_recovering_prioritized": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_started": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "recovery_completed": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_path": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_quiesce_inode": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_exportdir": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_flush": 0, 2024-03-26T18:04:29.989 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragmentdir": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_fragstats": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "ireq_inodestats": 0 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_log": { 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "evlrg": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "evadd": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "evex": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "evtrm": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "ev": 6310, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "evexg": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "evexd": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "segadd": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "segex": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "segtrm": 0, 2024-03-26T18:04:29.990 INFO:teuthology.orchestra.run.smithi144.stdout: "seg": 8, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "segmjr": 2, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "segexg": 0, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "segexd": 0, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "expos": 4194304, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "wrpos": 11415173, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "rdpos": 11414035, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "jlat": { 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: "replayed": 6310 2024-03-26T18:04:29.991 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_mem": { 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "ino": 10, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "ino+": 628, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "ino-": 618, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "dir": 8, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "dir+": 25, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "dir-": 17, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "dn": 609, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "dn+": 680, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "dn-": 71, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "cap": 0, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "cap+": 0, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "cap-": 0, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "rss": 55540, 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: "heap": 215420 2024-03-26T18:04:29.992 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_server": { 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_client_request": 0, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "dispatch_server_request": 0, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_request": 0, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_client_session": 0, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "handle_peer_request": 0, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "req_create_latency": { 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getattr_latency": { 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.993 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getfilelock_latency": { 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "req_link_latency": { 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookup_latency": { 2024-03-26T18:04:29.994 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookuphash_latency": { 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupino_latency": { 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.995 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupname_latency": { 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupparent_latency": { 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lookupsnap_latency": { 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.996 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "req_lssnap_latency": { 2024-03-26T18:04:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.997 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mkdir_latency": { 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mknod_latency": { 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "req_mksnap_latency": { 2024-03-26T18:04:29.998 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "req_open_latency": { 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "req_readdir_latency": { 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:29.999 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rename_latency": { 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "req_renamesnap_latency": { 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "req_snapdiff_latency": { 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.000 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmdir_latency": { 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmsnap_latency": { 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "req_rmxattr_latency": { 2024-03-26T18:04:30.001 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setattr_latency": { 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setdirlayout_latency": { 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.002 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setfilelock_latency": { 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setlayout_latency": { 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "req_setxattr_latency": { 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.003 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "req_symlink_latency": { 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "req_unlink_latency": { 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_revoke_eviction": 0, 2024-03-26T18:04:30.004 INFO:teuthology.orchestra.run.smithi144.stdout: "cap_acquisition_throttle": 0, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "req_getvxattr_latency": { 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_sessions": { 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "session_count": 1, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "session_add": 2, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "session_remove": 1, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_open": 1, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "sessions_stale": 0, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "total_load": 0, 2024-03-26T18:04:30.005 INFO:teuthology.orchestra.run.smithi144.stdout: "average_load": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "avg_session_uptime": 2699, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "mdthresh_evicted": 0 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "mempool": { 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_bytes": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bloom_filter_items": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_alloc_items": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_data_items": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T18:04:30.006 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_other_items": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_bytes": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_extent_items": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_bytes": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_blob_items": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T18:04:30.007 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_fsck_items": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_bytes": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_txc_items": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_bytes": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluestore_writing_items": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_bytes": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_items": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_reader_items": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "bluefs_file_writer_items": 0, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_bytes": 207879, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_anon_items": 63, 2024-03-26T18:04:30.008 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_bytes": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "buffer_meta_items": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_bytes": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_items": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_bytes": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_mapbl_items": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_bytes": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_pglog_items": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_bytes": 7744, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_items": 165, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "osdmap_mapping_items": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_bytes": 0, 2024-03-26T18:04:30.009 INFO:teuthology.orchestra.run.smithi144.stdout: "pgmap_items": 0, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_bytes": 461656, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_co_items": 2531, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_bytes": 0, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_1_items": 0, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_bytes": 0, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "unittest_2_items": 0 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "objecter": { 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "op_active": 0, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "op_laggy": 0, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send": 5320, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "op_send_bytes": 21, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "op_resend": 3, 2024-03-26T18:04:30.010 INFO:teuthology.orchestra.run.smithi144.stdout: "op_reply": 5320, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "op_latency": { 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5320, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 2.136963672, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000401684 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "op_inflight": 0, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "oplen_avg": { 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 5320, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 5321 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "op": 5320, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "op_r": 5320, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "op_w": 0, 2024-03-26T18:04:30.011 INFO:teuthology.orchestra.run.smithi144.stdout: "op_rmw": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "op_pg": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_stat": 3271, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_create": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_read": 2047, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_write": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writefull": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_writesame": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_append": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_zero": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_truncate": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_delete": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_mapext": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_sparse_read": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_clonerange": 0, 2024-03-26T18:04:30.012 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_getxattr": 1, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_setxattr": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_cmpxattr": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_rmxattr": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_resetxattrs": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_call": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_watch": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_notify": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_pgls_filter": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "osdop_other": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_active": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_send": 0, 2024-03-26T18:04:30.013 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_resend": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "linger_ping": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_active": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_send": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "poolop_resend": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_active": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_send": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "poolstat_resend": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_active": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_send": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "statfs_resend": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "command_active": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "command_send": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "command_resend": 0, 2024-03-26T18:04:30.014 INFO:teuthology.orchestra.run.smithi144.stdout: "map_epoch": 96, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "map_full": 0, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "map_inc": 11, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_sessions": 7, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_open": 10, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_session_close": 3, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_laggy": 0, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_wr": 0, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_rd": 2, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_del": 0 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "oft": { 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_objs": 0, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_kv_pairs": 0, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_updates": 0, 2024-03-26T18:04:30.015 INFO:teuthology.orchestra.run.smithi144.stdout: "omap_total_removes": 0 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "osd-slow-ops": { 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "slow_ops_count": 0 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "purge_queue": { 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops": 0, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing": 0, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executing_high_water": 0, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed_ops": 0, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_executed": 0, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "pq_item_in_journal": 0 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T18:04:30.016 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 5824, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 9167118, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 5824, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 5824, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 9167118, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:30.017 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_bytes": { 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 104857600, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 5320, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 7223024, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 5320, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 7223024, 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:30.018 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-objecter_ops": { 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 1024, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 5320, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 5320, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 5320, 2024-03-26T18:04:30.019 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 5320, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle": { 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:30.020 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "throttle-write_buf_throttle-0x55f196f2a6e0": { 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "max": 3758096384, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "get_started": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "get": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "get_sum": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_fail": 0, 2024-03-26T18:04:30.021 INFO:teuthology.orchestra.run.smithi144.stdout: "get_or_fail_success": 0, 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "take": 0, 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "take_sum": 0, 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "put": 0, 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "put_sum": 0, 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "wait": { 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "avgcount": 0, 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "sum": 0.000000000, 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: "avgtime": 0.000000000 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-03-26T18:04:30.022 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-03-26T18:04:30.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:30 smithi144 ceph-mon[36095]: pgmap v1137: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:30.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:30 smithi105 ceph-mon[32280]: pgmap v1137: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:30.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:30 smithi135 ceph-mon[35636]: pgmap v1137: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-26T18:04:30.773 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-26T18:04:30.773 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-26T18:04:30.773 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-26T18:04:30.773 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-03-26T18:04:30.774 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-26T18:04:30.785 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-26T18:04:30.785 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:04:30.785 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T18:04:30.812 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:04:30.813 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T18:04:30.870 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 -- 172.21.15.105:0/554821273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 msgr2=0x7f6a10102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 --2- 172.21.15.105:0/554821273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 0x7f6a10102f80 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009900 tx=0x7f69f802bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 -- 172.21.15.105:0/554821273 shutdown_connections 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 --2- 172.21.15.105:0/554821273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a101034c0 0x7f6a10109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 --2- 172.21.15.105:0/554821273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 0x7f6a10102f80 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 --2- 172.21.15.105:0/554821273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a10101910 0x7f6a10101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 -- 172.21.15.105:0/554821273 >> 172.21.15.105:0/554821273 conn(0x7f6a100fce60 msgr2=0x7f6a100ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 -- 172.21.15.105:0/554821273 shutdown_connections 2024-03-26T18:04:31.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.039+0000 7f6a15a61640 1 -- 172.21.15.105:0/554821273 wait complete. 2024-03-26T18:04:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a15a61640 1 Processor -- start 2024-03-26T18:04:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a15a61640 1 -- start start 2024-03-26T18:04:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a15a61640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a10101910 0x7f6a10197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a15a61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 0x7f6a10198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a15a61640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a101034c0 0x7f6a1019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a15a61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1010d440 con 0x7f6a10102b00 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a15a61640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1010d2c0 con 0x7f6a10101910 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a15a61640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1010d5c0 con 0x7f6a101034c0 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a0effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a10101910 0x7f6a10197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 0x7f6a10198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a0effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a10101910 0x7f6a10197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54676/0 (socket says 172.21.15.105:54676) 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a07fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 0x7f6a10198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37868/0 (socket says 172.21.15.105:37868) 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a0effd640 1 -- 172.21.15.105:0/3883033728 learned_addr learned my addr 172.21.15.105:0/3883033728 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a0f7fe640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a101034c0 0x7f6a1019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a07fff640 1 -- 172.21.15.105:0/3883033728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a101034c0 msgr2=0x7f6a1019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a07fff640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a101034c0 0x7f6a1019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a07fff640 1 -- 172.21.15.105:0/3883033728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a10101910 msgr2=0x7f6a10197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.040+0000 7f6a07fff640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a10101910 0x7f6a10197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.022 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.041+0000 7f6a07fff640 1 -- 172.21.15.105:0/3883033728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1019f760 con 0x7f6a10102b00 2024-03-26T18:04:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.041+0000 7f6a0effd640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a10101910 0x7f6a10197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.041+0000 7f6a07fff640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 0x7f6a10198080 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f69f8000c00 tx=0x7f69f8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.041+0000 7f6a14a5f640 1 -- 172.21.15.105:0/3883033728 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8043020 con 0x7f6a10102b00 2024-03-26T18:04:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.041+0000 7f6a14a5f640 1 -- 172.21.15.105:0/3883033728 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69f8004350 con 0x7f6a10102b00 2024-03-26T18:04:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.041+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1019f9f0 con 0x7f6a10102b00 2024-03-26T18:04:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.041+0000 7f6a14a5f640 1 -- 172.21.15.105:0/3883033728 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8003b50 con 0x7f6a10102b00 2024-03-26T18:04:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.041+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a1019ff40 con 0x7f6a10102b00 2024-03-26T18:04:31.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.042+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a100686b0 con 0x7f6a10102b00 2024-03-26T18:04:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.044+0000 7f6a14a5f640 1 -- 172.21.15.105:0/3883033728 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f69f8046020 con 0x7f6a10102b00 2024-03-26T18:04:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.044+0000 7f6a14a5f640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69fc0a2f20 0x7f69fc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.044+0000 7f6a0effd640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69fc0a2f20 0x7f69fc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.045+0000 7f6a14a5f640 1 -- 172.21.15.105:0/3883033728 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f69f80f1ae0 con 0x7f6a10102b00 2024-03-26T18:04:31.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.045+0000 7f6a0effd640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69fc0a2f20 0x7f69fc0a53e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6a000058c0 tx=0x7f6a00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:31.026 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.047+0000 7f6a14a5f640 1 -- 172.21.15.105:0/3883033728 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f69f804d020 con 0x7f6a10102b00 2024-03-26T18:04:31.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.227+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a1002d2c0 con 0x7f6a10102b00 2024-03-26T18:04:31.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.229+0000 7f6a14a5f640 1 -- 172.21.15.105:0/3883033728 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+22833 (secure 0 0 0) 0x7f69f80be8d0 con 0x7f6a10102b00 2024-03-26T18:04:31.208 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:04:31.209 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":96,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T18:02:07.352921+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3f","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69fc0a2f20 msgr2=0x7f69fc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69fc0a2f20 0x7f69fc0a53e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6a000058c0 tx=0x7f6a00005f90 comp rx=0 tx=0).stop 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 msgr2=0x7f6a10198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 0x7f6a10198080 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f69f8000c00 tx=0x7f69f8005dc0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 shutdown_connections 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f69fc0a2f20 0x7f69fc0a53e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a101034c0 0x7f6a1019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a10102b00 0x7f6a10198080 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 --2- 172.21.15.105:0/3883033728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6a10101910 0x7f6a10197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 >> 172.21.15.105:0/3883033728 conn(0x7f6a100fce60 msgr2=0x7f6a10105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 shutdown_connections 2024-03-26T18:04:31.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.231+0000 7f6a15a61640 1 -- 172.21.15.105:0/3883033728 wait complete. 2024-03-26T18:04:31.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:04:31.223 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 -- 172.21.15.105:0/1091308218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c102aa0 msgr2=0x7f230c104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 --2- 172.21.15.105:0/1091308218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c102aa0 0x7f230c104f10 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009900 tx=0x7f22fc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 -- 172.21.15.105:0/1091308218 shutdown_connections 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 --2- 172.21.15.105:0/1091308218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f230c105450 0x7f230c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 --2- 172.21.15.105:0/1091308218 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c102aa0 0x7f230c104f10 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 --2- 172.21.15.105:0/1091308218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c100130 0x7f230c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 -- 172.21.15.105:0/1091308218 >> 172.21.15.105:0/1091308218 conn(0x7f230c0fbd50 msgr2=0x7f230c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 -- 172.21.15.105:0/1091308218 shutdown_connections 2024-03-26T18:04:31.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.364+0000 7f2314212640 1 -- 172.21.15.105:0/1091308218 wait complete. 2024-03-26T18:04:31.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2314212640 1 Processor -- start 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2314212640 1 -- start start 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2314212640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c100130 0x7f230c19e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2314212640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f230c102aa0 0x7f230c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2314212640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c105450 0x7f230c1a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2314212640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1139e0 con 0x7f230c100130 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2314212640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f230c113860 con 0x7f230c105450 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2314212640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f230c113b60 con 0x7f230c102aa0 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2311fae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c100130 0x7f230c19e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2311fae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c100130 0x7f230c19e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37880/0 (socket says 172.21.15.105:37880) 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f2311fae640 1 -- 172.21.15.105:0/1928334984 learned_addr learned my addr 172.21.15.105:0/1928334984 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f23117ad640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f230c102aa0 0x7f230c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.365+0000 7f23127af640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c105450 0x7f230c1a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2311fae640 1 -- 172.21.15.105:0/1928334984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f230c102aa0 msgr2=0x7f230c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2311fae640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f230c102aa0 0x7f230c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2311fae640 1 -- 172.21.15.105:0/1928334984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c105450 msgr2=0x7f230c1a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2311fae640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c105450 0x7f230c1a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2311fae640 1 -- 172.21.15.105:0/1928334984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c1a5d10 con 0x7f230c100130 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f23117ad640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f230c102aa0 0x7f230c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2311fae640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c100130 0x7f230c19e0f0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f230800d950 tx=0x7f230800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2302ffd640 1 -- 172.21.15.105:0/1928334984 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308017030 con 0x7f230c100130 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c1a5fa0 con 0x7f230c100130 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2302ffd640 1 -- 172.21.15.105:0/1928334984 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2308004020 con 0x7f230c100130 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2302ffd640 1 -- 172.21.15.105:0/1928334984 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080047a0 con 0x7f230c100130 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.366+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c1a64b0 con 0x7f230c100130 2024-03-26T18:04:31.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.367+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230c0fd840 con 0x7f230c100130 2024-03-26T18:04:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.368+0000 7f2302ffd640 1 -- 172.21.15.105:0/1928334984 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f23080041d0 con 0x7f230c100130 2024-03-26T18:04:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.369+0000 7f2302ffd640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22f80a2f20 0x7f22f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.369+0000 7f23117ad640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22f80a2f20 0x7f22f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.369+0000 7f2302ffd640 1 -- 172.21.15.105:0/1928334984 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f23080cf510 con 0x7f230c100130 2024-03-26T18:04:31.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.370+0000 7f23117ad640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22f80a2f20 0x7f22f80a53e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009230 tx=0x7f22fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:31.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.372+0000 7f2302ffd640 1 -- 172.21.15.105:0/1928334984 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f230809c300 con 0x7f230c100130 2024-03-26T18:04:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.553+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f230c0624c0 con 0x7f230c100130 2024-03-26T18:04:31.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.553+0000 7f2302ffd640 1 -- 172.21.15.105:0/1928334984 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f230809bc00 con 0x7f230c100130 2024-03-26T18:04:31.533 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T18:04:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22f80a2f20 msgr2=0x7f22f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22f80a2f20 0x7f22f80a53e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009230 tx=0x7f22fc005fd0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c100130 msgr2=0x7f230c19e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c100130 0x7f230c19e0f0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f230800d950 tx=0x7f230800de10 comp rx=0 tx=0).stop 2024-03-26T18:04:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 shutdown_connections 2024-03-26T18:04:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f22f80a2f20 0x7f22f80a53e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f230c105450 0x7f230c1a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f230c102aa0 0x7f230c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 --2- 172.21.15.105:0/1928334984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f230c100130 0x7f230c19e0f0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 >> 172.21.15.105:0/1928334984 conn(0x7f230c0fbd50 msgr2=0x7f230c100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 shutdown_connections 2024-03-26T18:04:31.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.557+0000 7f2314212640 1 -- 172.21.15.105:0/1928334984 wait complete. 2024-03-26T18:04:31.548 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 -- 172.21.15.105:0/950755569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1001b0 msgr2=0x7f8a4c1005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 --2- 172.21.15.105:0/950755569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1001b0 0x7f8a4c1005d0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009900 tx=0x7f8a4802bcd0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 -- 172.21.15.105:0/950755569 shutdown_connections 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 --2- 172.21.15.105:0/950755569 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c1014b0 0x7f8a4c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 --2- 172.21.15.105:0/950755569 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c100b10 0x7f8a4c100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 --2- 172.21.15.105:0/950755569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1001b0 0x7f8a4c1005d0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 -- 172.21.15.105:0/950755569 >> 172.21.15.105:0/950755569 conn(0x7f8a4c0fbd50 msgr2=0x7f8a4c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 -- 172.21.15.105:0/950755569 shutdown_connections 2024-03-26T18:04:31.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.678+0000 7f8a5435b640 1 -- 172.21.15.105:0/950755569 wait complete. 2024-03-26T18:04:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a5435b640 1 Processor -- start 2024-03-26T18:04:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a5435b640 1 -- start start 2024-03-26T18:04:31.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a5435b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c1001b0 0x7f8a4c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a5435b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c100b10 0x7f8a4c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a5435b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1014b0 0x7f8a4c1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a5435b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10f5f0 con 0x7f8a4c1014b0 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a5435b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10f470 con 0x7f8a4c100b10 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a5435b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10f770 con 0x7f8a4c1001b0 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.679+0000 7f8a528f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1014b0 0x7f8a4c1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a528f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1014b0 0x7f8a4c1a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37890/0 (socket says 172.21.15.105:37890) 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a528f8640 1 -- 172.21.15.105:0/3766106275 learned_addr learned my addr 172.21.15.105:0/3766106275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a520f7640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c1001b0 0x7f8a4c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a518f6640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c100b10 0x7f8a4c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a528f8640 1 -- 172.21.15.105:0/3766106275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c1001b0 msgr2=0x7f8a4c199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a528f8640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c1001b0 0x7f8a4c199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a528f8640 1 -- 172.21.15.105:0/3766106275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c100b10 msgr2=0x7f8a4c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a528f8640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c100b10 0x7f8a4c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a528f8640 1 -- 172.21.15.105:0/3766106275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c1a1960 con 0x7f8a4c1014b0 2024-03-26T18:04:31.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a528f8640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1014b0 0x7f8a4c1a12f0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400eb10 tx=0x7f8a4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:31.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a437fe640 1 -- 172.21.15.105:0/3766106275 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44004b20 con 0x7f8a4c1014b0 2024-03-26T18:04:31.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c1a1c50 con 0x7f8a4c1014b0 2024-03-26T18:04:31.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a437fe640 1 -- 172.21.15.105:0/3766106275 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a44004cc0 con 0x7f8a4c1014b0 2024-03-26T18:04:31.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a437fe640 1 -- 172.21.15.105:0/3766106275 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44010690 con 0x7f8a4c1014b0 2024-03-26T18:04:31.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.680+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c1a21a0 con 0x7f8a4c1014b0 2024-03-26T18:04:31.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.682+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a4c10b1e0 con 0x7f8a4c1014b0 2024-03-26T18:04:31.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.682+0000 7f8a437fe640 1 -- 172.21.15.105:0/3766106275 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8a44005020 con 0x7f8a4c1014b0 2024-03-26T18:04:31.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.683+0000 7f8a437fe640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a380a2f20 0x7f8a380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:31.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.683+0000 7f8a437fe640 1 -- 172.21.15.105:0/3766106275 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f8a440cf980 con 0x7f8a4c1014b0 2024-03-26T18:04:31.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.683+0000 7f8a520f7640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a380a2f20 0x7f8a380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:31.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.683+0000 7f8a520f7640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a380a2f20 0x7f8a380a53e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009230 tx=0x7f8a48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:31.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.687+0000 7f8a437fe640 1 -- 172.21.15.105:0/3766106275 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a4409c770 con 0x7f8a4c1014b0 2024-03-26T18:04:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.962+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8a4c0624c0 con 0x7f8a4c1014b0 2024-03-26T18:04:31.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.962+0000 7f8a437fe640 1 -- 172.21.15.105:0/3766106275 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f8a4409c070 con 0x7f8a4c1014b0 2024-03-26T18:04:31.942 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a380a2f20 msgr2=0x7f8a380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a380a2f20 0x7f8a380a53e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009230 tx=0x7f8a48005fd0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1014b0 msgr2=0x7f8a4c1a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1014b0 0x7f8a4c1a12f0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400eb10 tx=0x7f8a4400c2d0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 shutdown_connections 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f8a380a2f20 0x7f8a380a53e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a4c1014b0 0x7f8a4c1a12f0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a4c100b10 0x7f8a4c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 --2- 172.21.15.105:0/3766106275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a4c1001b0 0x7f8a4c199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 >> 172.21.15.105:0/3766106275 conn(0x7f8a4c0fbd50 msgr2=0x7f8a4c0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 shutdown_connections 2024-03-26T18:04:31.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:31.964+0000 7f8a5435b640 1 -- 172.21.15.105:0/3766106275 wait complete. 2024-03-26T18:04:31.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:04:32.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 -- 172.21.15.105:0/850026418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 msgr2=0x7fa350101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 --2- 172.21.15.105:0/850026418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 0x7fa350101d10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa344009900 tx=0x7fa34402bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 -- 172.21.15.105:0/850026418 shutdown_connections 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 --2- 172.21.15.105:0/850026418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa3501034a0 0x7fa350109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 --2- 172.21.15.105:0/850026418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa350102ae0 0x7fa350102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 --2- 172.21.15.105:0/850026418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 0x7fa350101d10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 -- 172.21.15.105:0/850026418 >> 172.21.15.105:0/850026418 conn(0x7fa3500fce60 msgr2=0x7fa3500ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 -- 172.21.15.105:0/850026418 shutdown_connections 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.088+0000 7fa3558f7640 1 -- 172.21.15.105:0/850026418 wait complete. 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa3558f7640 1 Processor -- start 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa3558f7640 1 -- start start 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa3558f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 0x7fa350197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa3558f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa350102ae0 0x7fa350198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa3558f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3501034a0 0x7fa35019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa3558f7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa35010d3f0 con 0x7fa350102ae0 2024-03-26T18:04:32.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa3558f7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa35010d270 con 0x7fa3501034a0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa3558f7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa35010d570 con 0x7fa3501018f0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa350102ae0 0x7fa350198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 0x7fa350197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3501034a0 0x7fa35019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34effd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 0x7fa350197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:40946/0 (socket says 172.21.15.105:40946) 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34effd640 1 -- 172.21.15.105:0/4013014464 learned_addr learned my addr 172.21.15.105:0/4013014464 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34effd640 1 -- 172.21.15.105:0/4013014464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3501034a0 msgr2=0x7fa35019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34effd640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3501034a0 0x7fa35019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34effd640 1 -- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa350102ae0 msgr2=0x7fa350198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34effd640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa350102ae0 0x7fa350198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.089+0000 7fa34effd640 1 -- 172.21.15.105:0/4013014464 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35019f750 con 0x7fa3501018f0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.090+0000 7fa34effd640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 0x7fa350197b30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa344000c00 tx=0x7fa344005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.090+0000 7fa3548f5640 1 -- 172.21.15.105:0/4013014464 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344043020 con 0x7fa3501018f0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.090+0000 7fa3548f5640 1 -- 172.21.15.105:0/4013014464 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa344004350 con 0x7fa3501018f0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.090+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35019f9e0 con 0x7fa3501018f0 2024-03-26T18:04:32.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.090+0000 7fa3548f5640 1 -- 172.21.15.105:0/4013014464 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344003b80 con 0x7fa3501018f0 2024-03-26T18:04:32.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.090+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35019ff80 con 0x7fa3501018f0 2024-03-26T18:04:32.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.091+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa350108d30 con 0x7fa3501018f0 2024-03-26T18:04:32.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.092+0000 7fa3548f5640 1 -- 172.21.15.105:0/4013014464 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fa344046020 con 0x7fa3501018f0 2024-03-26T18:04:32.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.093+0000 7fa3548f5640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3380a2fa0 0x7fa3380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.093+0000 7fa34e7fc640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3380a2fa0 0x7fa3380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.093+0000 7fa3548f5640 1 -- 172.21.15.105:0/4013014464 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fa3440f14d0 con 0x7fa3501018f0 2024-03-26T18:04:32.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.093+0000 7fa34e7fc640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3380a2fa0 0x7fa3380a5460 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa3400095f0 tx=0x7fa340005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:32.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.097+0000 7fa3548f5640 1 -- 172.21.15.105:0/4013014464 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa34404d020 con 0x7fa3501018f0 2024-03-26T18:04:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.294+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3500624c0 con 0x7fa3501018f0 2024-03-26T18:04:32.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.295+0000 7fa3548f5640 1 -- 172.21.15.105:0/4013014464 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7fa3440be2c0 con 0x7fa3501018f0 2024-03-26T18:04:32.275 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T18:04:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3380a2fa0 msgr2=0x7fa3380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3380a2fa0 0x7fa3380a5460 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa3400095f0 tx=0x7fa340005f90 comp rx=0 tx=0).stop 2024-03-26T18:04:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 msgr2=0x7fa350197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 0x7fa350197b30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa344000c00 tx=0x7fa344005dc0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 shutdown_connections 2024-03-26T18:04:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fa3380a2fa0 0x7fa3380a5460 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa3501034a0 0x7fa35019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa350102ae0 0x7fa350198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 --2- 172.21.15.105:0/4013014464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3501018f0 0x7fa350197b30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.297+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 >> 172.21.15.105:0/4013014464 conn(0x7fa3500fce60 msgr2=0x7fa350105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.298+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 shutdown_connections 2024-03-26T18:04:32.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.298+0000 7fa3558f7640 1 -- 172.21.15.105:0/4013014464 wait complete. 2024-03-26T18:04:32.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:32 smithi105 ceph-mon[32280]: pgmap v1138: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:32.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:32 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3883033728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:32.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:32 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:04:32.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:32 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1928334984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:04:32.289 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:32 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3766106275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:04:32.291 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T18:04:32.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:32 smithi144 ceph-mon[36095]: pgmap v1138: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:32.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:32 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3883033728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:32.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:32 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:04:32.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:32 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1928334984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:04:32.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:32 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3766106275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 -- 172.21.15.105:0/2535444425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 msgr2=0x7f3ec8108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 --2- 172.21.15.105:0/2535444425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 0x7f3ec8108c00 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009ca0 tx=0x7f3ec002c0f0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 -- 172.21.15.105:0/2535444425 shutdown_connections 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 --2- 172.21.15.105:0/2535444425 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 0x7f3ec8108c00 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 --2- 172.21.15.105:0/2535444425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ec81019b0 0x7f3ec8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 --2- 172.21.15.105:0/2535444425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec81007c0 0x7f3ec8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 -- 172.21.15.105:0/2535444425 >> 172.21.15.105:0/2535444425 conn(0x7f3ec80fbd50 msgr2=0x7f3ec80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 -- 172.21.15.105:0/2535444425 shutdown_connections 2024-03-26T18:04:32.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.417+0000 7f3ed0369640 1 -- 172.21.15.105:0/2535444425 wait complete. 2024-03-26T18:04:32.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ed0369640 1 Processor -- start 2024-03-26T18:04:32.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ed0369640 1 -- start start 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ed0369640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ec81007c0 0x7f3ec8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ed0369640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec81019b0 0x7f3ec819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ed0369640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 0x7f3ec819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ed0369640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810c2f0 con 0x7f3ec8102370 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ed0369640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810c170 con 0x7f3ec81007c0 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ed0369640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec810c470 con 0x7f3ec81019b0 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ece906640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 0x7f3ec819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ece105640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ec81007c0 0x7f3ec8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ece906640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 0x7f3ec819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37936/0 (socket says 172.21.15.105:37936) 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ece906640 1 -- 172.21.15.105:0/1320451582 learned_addr learned my addr 172.21.15.105:0/1320451582 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ecd904640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec81019b0 0x7f3ec819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ece906640 1 -- 172.21.15.105:0/1320451582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec81019b0 msgr2=0x7f3ec819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ece906640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec81019b0 0x7f3ec819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.418+0000 7f3ece906640 1 -- 172.21.15.105:0/1320451582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ec81007c0 msgr2=0x7f3ec8069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ece906640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ec81007c0 0x7f3ec8069590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ece906640 1 -- 172.21.15.105:0/1320451582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec819f790 con 0x7f3ec8102370 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ece105640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ec81007c0 0x7f3ec8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ece906640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 0x7f3ec819ee80 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f3ec002c700 tx=0x7f3ec00328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ebf7fe640 1 -- 172.21.15.105:0/1320451582 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0041020 con 0x7f3ec8102370 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ebf7fe640 1 -- 172.21.15.105:0/1320451582 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec0003e20 con 0x7f3ec8102370 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec819f9c0 con 0x7f3ec8102370 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ebf7fe640 1 -- 172.21.15.105:0/1320451582 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0039040 con 0x7f3ec8102370 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.419+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec819feb0 con 0x7f3ec8102370 2024-03-26T18:04:32.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.420+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec8107c00 con 0x7f3ec8102370 2024-03-26T18:04:32.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.421+0000 7f3ebf7fe640 1 -- 172.21.15.105:0/1320451582 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f3ec0044020 con 0x7f3ec8102370 2024-03-26T18:04:32.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.422+0000 7f3ebf7fe640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3eb40a2f20 0x7f3eb40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.422+0000 7f3ece105640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3eb40a2f20 0x7f3eb40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.422+0000 7f3ebf7fe640 1 -- 172.21.15.105:0/1320451582 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f3ec00f1a00 con 0x7f3ec8102370 2024-03-26T18:04:32.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.422+0000 7f3ece105640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3eb40a2f20 0x7f3eb40a53e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8101810 tx=0x7f3ec4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:32 smithi135 ceph-mon[35636]: pgmap v1138: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:32 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3883033728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:32 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T18:04:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:32 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1928334984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:04:32.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:32 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3766106275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:04:32.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.428+0000 7f3ebf7fe640 1 -- 172.21.15.105:0/1320451582 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3ec004b020 con 0x7f3ec8102370 2024-03-26T18:04:32.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.622+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3ec802d400 con 0x7f3ec8102370 2024-03-26T18:04:32.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.622+0000 7f3ebf7fe640 1 -- 172.21.15.105:0/1320451582 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+601 (secure 0 0 0) 0x7f3ec00be7f0 con 0x7f3ec8102370 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/1242681476 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/976039762 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6801/1556474900 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/1151470472 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/3821116771 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6800/3642413152 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/2484248025 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/2017206095 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:32.602 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6801/3642413152 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:32.603 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6800/1556474900 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:32.603 INFO:teuthology.orchestra.run.smithi105.stderr:listed 10 entries 2024-03-26T18:04:32.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3eb40a2f20 msgr2=0x7f3eb40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3eb40a2f20 0x7f3eb40a53e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8101810 tx=0x7f3ec4005f90 comp rx=0 tx=0).stop 2024-03-26T18:04:32.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 msgr2=0x7f3ec819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 0x7f3ec819ee80 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f3ec002c700 tx=0x7f3ec00328a0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 shutdown_connections 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f3eb40a2f20 0x7f3eb40a53e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec8102370 0x7f3ec819ee80 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec81019b0 0x7f3ec819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 --2- 172.21.15.105:0/1320451582 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ec81007c0 0x7f3ec8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 >> 172.21.15.105:0/1320451582 conn(0x7f3ec80fbd50 msgr2=0x7f3ec8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 shutdown_connections 2024-03-26T18:04:32.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.625+0000 7f3ed0369640 1 -- 172.21.15.105:0/1320451582 wait complete. 2024-03-26T18:04:32.616 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-26T18:04:32.616 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T18:04:32.632 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:04:32.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 -- 172.21.15.105:0/4125990630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d8104c90 msgr2=0x7f12d8107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/4125990630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d8104c90 0x7f12d8107080 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009900 tx=0x7f12c002bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:32.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 -- 172.21.15.105:0/4125990630 shutdown_connections 2024-03-26T18:04:32.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/4125990630 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12d81075c0 0x7f12d8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/4125990630 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d8104c90 0x7f12d8107080 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/4125990630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d8102360 0x7f12d8104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 -- 172.21.15.105:0/4125990630 >> 172.21.15.105:0/4125990630 conn(0x7f12d80fbd50 msgr2=0x7f12d80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 -- 172.21.15.105:0/4125990630 shutdown_connections 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 -- 172.21.15.105:0/4125990630 wait complete. 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 Processor -- start 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.814+0000 7f12dcbd4640 1 -- start start 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12dcbd4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d8102360 0x7f12d8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12dcbd4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12d8104c90 0x7f12d81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12dcbd4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d81075c0 0x7f12d819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12dcbd4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f12d810d420 con 0x7f12d81075c0 2024-03-26T18:04:32.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12dcbd4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12d810d2a0 con 0x7f12d8104c90 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12dcbd4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12d810d5a0 con 0x7f12d8102360 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d81075c0 0x7f12d819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d81075c0 0x7f12d819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37954/0 (socket says 172.21.15.105:37954) 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d659c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d8102360 0x7f12d8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 -- 172.21.15.105:0/3011903165 learned_addr learned my addr 172.21.15.105:0/3011903165 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12cffff640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12d8104c90 0x7f12d81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 -- 172.21.15.105:0/3011903165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d8102360 msgr2=0x7f12d8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d8102360 0x7f12d8197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 -- 172.21.15.105:0/3011903165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12d8104c90 msgr2=0x7f12d81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12d8104c90 0x7f12d81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 -- 172.21.15.105:0/3011903165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d819f770 con 0x7f12d81075c0 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d659c640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d8102360 0x7f12d8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12cffff640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12d8104c90 0x7f12d81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:32.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12d6d9d640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d81075c0 0x7f12d819f100 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f12d000efd0 tx=0x7f12d000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12cf7fe640 1 -- 172.21.15.105:0/3011903165 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0003b70 con 0x7f12d81075c0 2024-03-26T18:04:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12cf7fe640 1 -- 172.21.15.105:0/3011903165 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12d0003d10 con 0x7f12d81075c0 2024-03-26T18:04:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12cf7fe640 1 -- 172.21.15.105:0/3011903165 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0010980 con 0x7f12d81075c0 2024-03-26T18:04:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.815+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d819fa60 con 0x7f12d81075c0 2024-03-26T18:04:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.816+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d819ff30 con 0x7f12d81075c0 2024-03-26T18:04:32.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.817+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12d80fd840 con 0x7f12d81075c0 2024-03-26T18:04:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.818+0000 7f12cf7fe640 1 -- 172.21.15.105:0/3011903165 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f12d0004020 con 0x7f12d81075c0 2024-03-26T18:04:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.818+0000 7f12cf7fe640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12c40a2f20 0x7f12c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.818+0000 7f12d659c640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12c40a2f20 0x7f12c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.818+0000 7f12cf7fe640 1 -- 172.21.15.105:0/3011903165 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f12d00cf830 con 0x7f12d81075c0 2024-03-26T18:04:32.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.819+0000 7f12d659c640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12c40a2f20 0x7f12c40a53e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009fc0 tx=0x7f12c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:32.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:32.826+0000 7f12cf7fe640 1 -- 172.21.15.105:0/3011903165 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f12d009c620 con 0x7f12d81075c0 2024-03-26T18:04:32.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.010+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12d80624c0 con 0x7f12d81075c0 2024-03-26T18:04:32.991 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.011+0000 7f12cf7fe640 1 -- 172.21.15.105:0/3011903165 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+22833 (secure 0 0 0) 0x7f12d009bf20 con 0x7f12d81075c0 2024-03-26T18:04:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:04:32.991 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":96,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T18:02:07.352921+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3f","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:04:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12c40a2f20 msgr2=0x7f12c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12c40a2f20 0x7f12c40a53e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009fc0 tx=0x7f12c8005f90 comp rx=0 tx=0).stop 2024-03-26T18:04:32.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d81075c0 msgr2=0x7f12d819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d81075c0 0x7f12d819f100 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f12d000efd0 tx=0x7f12d000c6a0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 shutdown_connections 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f12c40a2f20 0x7f12c40a53e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12d81075c0 0x7f12d819f100 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12d8104c90 0x7f12d81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 --2- 172.21.15.105:0/3011903165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d8102360 0x7f12d8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 >> 172.21.15.105:0/3011903165 conn(0x7f12d80fbd50 msgr2=0x7f12d8102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 shutdown_connections 2024-03-26T18:04:32.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.014+0000 7f12dcbd4640 1 -- 172.21.15.105:0/3011903165 wait complete. 2024-03-26T18:04:33.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T17:31:50.834121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T17:36:20.108300+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T17:36:21.285220+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T18:04:33.005 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T18:04:33.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.136+0000 7f03f80c6640 1 -- 172.21.15.105:0/3532744803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0107610 msgr2=0x7f03f0109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.136+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3532744803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0107610 0x7f03f0109a00 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009ca0 tx=0x7f03e802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.136+0000 7f03f80c6640 1 -- 172.21.15.105:0/3532744803 shutdown_connections 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.136+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3532744803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f0107610 0x7f03f0109a00 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.136+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3532744803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f0104ce0 0x7f03f01070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.136+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3532744803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f01023b0 0x7f03f01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.136+0000 7f03f80c6640 1 -- 172.21.15.105:0/3532744803 >> 172.21.15.105:0/3532744803 conn(0x7f03f00fbd60 msgr2=0x7f03f00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 -- 172.21.15.105:0/3532744803 shutdown_connections 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 -- 172.21.15.105:0/3532744803 wait complete. 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 Processor -- start 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 -- start start 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f01023b0 0x7f03f0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f0104ce0 0x7f03f0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f0107610 0x7f03f019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f03f010d400 con 0x7f03f01023b0 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03f010d280 con 0x7f03f0104ce0 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f80c6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f03f010d580 con 0x7f03f0107610 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f5e62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f01023b0 0x7f03f0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f5e62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f01023b0 0x7f03f0197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37968/0 (socket says 172.21.15.105:37968) 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f5e62640 1 -- 172.21.15.105:0/3848259863 learned_addr learned my addr 172.21.15.105:0/3848259863 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.137+0000 7f03f6663640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f0107610 0x7f03f019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f5661640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f0104ce0 0x7f03f0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f5e62640 1 -- 172.21.15.105:0/3848259863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f0107610 msgr2=0x7f03f019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f5e62640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f0107610 0x7f03f019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f5e62640 1 -- 172.21.15.105:0/3848259863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f0104ce0 msgr2=0x7f03f0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f5e62640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f0104ce0 0x7f03f0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f5e62640 1 -- 172.21.15.105:0/3848259863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f019f750 con 0x7f03f01023b0 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f5661640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f0104ce0 0x7f03f0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f5e62640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f01023b0 0x7f03f0197b30 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00e9f0 tx=0x7f03ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:33.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03e6ffd640 1 -- 172.21.15.105:0/3848259863 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00cd80 con 0x7f03f01023b0 2024-03-26T18:04:33.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03e6ffd640 1 -- 172.21.15.105:0/3848259863 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ec0108e0 con 0x7f03f01023b0 2024-03-26T18:04:33.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f019fa40 con 0x7f03f01023b0 2024-03-26T18:04:33.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03e6ffd640 1 -- 172.21.15.105:0/3848259863 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec018920 con 0x7f03f01023b0 2024-03-26T18:04:33.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.138+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f019ffe0 con 0x7f03f01023b0 2024-03-26T18:04:33.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.139+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f0068b00 con 0x7f03f01023b0 2024-03-26T18:04:33.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.140+0000 7f03e6ffd640 1 -- 172.21.15.105:0/3848259863 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f03ec010a90 con 0x7f03f01023b0 2024-03-26T18:04:33.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.141+0000 7f03e6ffd640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03dc0a2fa0 0x7f03dc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.141+0000 7f03f5661640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03dc0a2fa0 0x7f03dc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.141+0000 7f03e6ffd640 1 -- 172.21.15.105:0/3848259863 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f03ec0cf750 con 0x7f03f01023b0 2024-03-26T18:04:33.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.141+0000 7f03f5661640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03dc0a2fa0 0x7f03dc0a5460 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f03f0199040 tx=0x7f03e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:33.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.145+0000 7f03e6ffd640 1 -- 172.21.15.105:0/3848259863 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03ec09c540 con 0x7f03f01023b0 2024-03-26T18:04:33.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.327+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03f00624c0 con 0x7f03f01023b0 2024-03-26T18:04:33.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.327+0000 7f03e6ffd640 1 -- 172.21.15.105:0/3848259863 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f03ec09be40 con 0x7f03f01023b0 2024-03-26T18:04:33.307 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-03-26T18:04:33.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.329+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03dc0a2fa0 msgr2=0x7f03dc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.329+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03dc0a2fa0 0x7f03dc0a5460 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f03f0199040 tx=0x7f03e0005f90 comp rx=0 tx=0).stop 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.329+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f01023b0 msgr2=0x7f03f0197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.329+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f01023b0 0x7f03f0197b30 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00e9f0 tx=0x7f03ec00eeb0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.329+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 shutdown_connections 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.329+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f03dc0a2fa0 0x7f03dc0a5460 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.330+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03f0107610 0x7f03f019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.330+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03f0104ce0 0x7f03f0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.330+0000 7f03f80c6640 1 --2- 172.21.15.105:0/3848259863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03f01023b0 0x7f03f0197b30 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.330+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 >> 172.21.15.105:0/3848259863 conn(0x7f03f00fbd60 msgr2=0x7f03f0102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.330+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 shutdown_connections 2024-03-26T18:04:33.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.330+0000 7f03f80c6640 1 -- 172.21.15.105:0/3848259863 wait complete. 2024-03-26T18:04:33.323 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:33 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/4013014464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:04:33.323 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:04:33.323 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:04:33.323 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:33 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:04:33.323 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:33 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1320451582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:04:33.323 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:33 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3011903165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:33.324 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T18:04:33.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:33 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/4013014464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:04:33.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:04:33.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:04:33.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:33 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:04:33.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:33 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1320451582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:04:33.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:33 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3011903165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:33.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:33 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/4013014464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:04:33.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T18:04:33.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T18:04:33.402 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:33 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' 2024-03-26T18:04:33.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:33 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1320451582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:04:33.403 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:33 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3011903165' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/4260600467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f41007c0 msgr2=0x7fe7f4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/4260600467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f41007c0 0x7fe7f4100be0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009900 tx=0x7fe7e402bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/4260600467 shutdown_connections 2024-03-26T18:04:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/4260600467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7f4102370 0x7fe7f4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/4260600467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7f41019b0 0x7fe7f4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/4260600467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f41007c0 0x7fe7f4100be0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/4260600467 >> 172.21.15.105:0/4260600467 conn(0x7fe7f40fbd50 msgr2=0x7fe7f40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:33.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/4260600467 shutdown_connections 2024-03-26T18:04:33.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.451+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/4260600467 wait complete. 2024-03-26T18:04:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f8dc4640 1 Processor -- start 2024-03-26T18:04:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f8dc4640 1 -- start start 2024-03-26T18:04:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f8dc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7f41007c0 0x7fe7f4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f8dc4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7f41019b0 0x7fe7f41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f8dc4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f4102370 0x7fe7f419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f8dc4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f410c330 con 0x7fe7f4102370 2024-03-26T18:04:33.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f8dc4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f410c1b0 con 0x7fe7f41019b0 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f8dc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f410c4b0 con 0x7fe7f41007c0 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.452+0000 7fe7f2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f4102370 0x7fe7f419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f4102370 0x7fe7f419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37978/0 (socket says 172.21.15.105:37978) 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f2d9d640 1 -- 172.21.15.105:0/3365110006 learned_addr learned my addr 172.21.15.105:0/3365110006 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f259c640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7f41007c0 0x7fe7f4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7ebfff640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7f41019b0 0x7fe7f41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f2d9d640 1 -- 172.21.15.105:0/3365110006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7f41007c0 msgr2=0x7fe7f4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f2d9d640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7f41007c0 0x7fe7f4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f2d9d640 1 -- 172.21.15.105:0/3365110006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7f41019b0 msgr2=0x7fe7f41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f2d9d640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7f41019b0 0x7fe7f41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f2d9d640 1 -- 172.21.15.105:0/3365110006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f419f7a0 con 0x7fe7f4102370 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7ebfff640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7f41019b0 0x7fe7f41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f259c640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7f41007c0 0x7fe7f4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:33.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f2d9d640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f4102370 0x7fe7f419f130 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00eb10 tx=0x7fe7ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7eb7fe640 1 -- 172.21.15.105:0/3365110006 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec004b20 con 0x7fe7f4102370 2024-03-26T18:04:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7eb7fe640 1 -- 172.21.15.105:0/3365110006 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7ec004cc0 con 0x7fe7f4102370 2024-03-26T18:04:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f419fa90 con 0x7fe7f4102370 2024-03-26T18:04:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7eb7fe640 1 -- 172.21.15.105:0/3365110006 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec010690 con 0x7fe7f4102370 2024-03-26T18:04:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.453+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f419ffe0 con 0x7fe7f4102370 2024-03-26T18:04:33.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.455+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f41a0270 con 0x7fe7f4102370 2024-03-26T18:04:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.455+0000 7fe7eb7fe640 1 -- 172.21.15.105:0/3365110006 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fe7ec005020 con 0x7fe7f4102370 2024-03-26T18:04:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.456+0000 7fe7eb7fe640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7e00a2fa0 0x7fe7e00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.456+0000 7fe7f259c640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7e00a2fa0 0x7fe7e00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.456+0000 7fe7eb7fe640 1 -- 172.21.15.105:0/3365110006 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fe7ec0cf900 con 0x7fe7f4102370 2024-03-26T18:04:33.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.456+0000 7fe7f259c640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7e00a2fa0 0x7fe7e00a5460 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe7e402c240 tx=0x7fe7e402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:33.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.460+0000 7fe7eb7fe640 1 -- 172.21.15.105:0/3365110006 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe7ec09c6f0 con 0x7fe7f4102370 2024-03-26T18:04:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.641+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7f41a0440 con 0x7fe7f4102370 2024-03-26T18:04:33.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.641+0000 7fe7eb7fe640 1 -- 172.21.15.105:0/3365110006 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7fe7ec09bff0 con 0x7fe7f4102370 2024-03-26T18:04:33.621 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-03-26T18:04:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.643+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7e00a2fa0 msgr2=0x7fe7e00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.643+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7e00a2fa0 0x7fe7e00a5460 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe7e402c240 tx=0x7fe7e402c7e0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.643+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f4102370 msgr2=0x7fe7f419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.643+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f4102370 0x7fe7f419f130 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00eb10 tx=0x7fe7ec00c2d0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.643+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 shutdown_connections 2024-03-26T18:04:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.643+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fe7e00a2fa0 0x7fe7e00a5460 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.644+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f4102370 0x7fe7f419f130 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.644+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe7f41019b0 0x7fe7f41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.644+0000 7fe7f8dc4640 1 --2- 172.21.15.105:0/3365110006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7f41007c0 0x7fe7f4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.644+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 >> 172.21.15.105:0/3365110006 conn(0x7fe7f40fbd50 msgr2=0x7fe7f4068a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:33.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.644+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 shutdown_connections 2024-03-26T18:04:33.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.644+0000 7fe7f8dc4640 1 -- 172.21.15.105:0/3365110006 wait complete. 2024-03-26T18:04:33.635 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T18:04:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 -- 172.21.15.105:0/4107675155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 msgr2=0x7f907c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 --2- 172.21.15.105:0/4107675155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 0x7f907c100be0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f906c009900 tx=0x7f906c02bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 -- 172.21.15.105:0/4107675155 shutdown_connections 2024-03-26T18:04:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 --2- 172.21.15.105:0/4107675155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c102370 0x7f907c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 --2- 172.21.15.105:0/4107675155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f907c1019b0 0x7f907c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 --2- 172.21.15.105:0/4107675155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 0x7f907c100be0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 -- 172.21.15.105:0/4107675155 >> 172.21.15.105:0/4107675155 conn(0x7f907c0fbd50 msgr2=0x7f907c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:33.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 -- 172.21.15.105:0/4107675155 shutdown_connections 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.762+0000 7f90818ac640 1 -- 172.21.15.105:0/4107675155 wait complete. 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f90818ac640 1 Processor -- start 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f90818ac640 1 -- start start 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f90818ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 0x7f907c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f90818ac640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c1019b0 0x7f907c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f90818ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f907c102370 0x7f907c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f90818ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f907c10d410 con 0x7f907c102370 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f90818ac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f907c10d290 con 0x7f907c1007c0 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f90818ac640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f907c10d590 con 0x7f907c1019b0 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 0x7f907c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f907c102370 0x7f907c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 0x7f907c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54820/0 (socket says 172.21.15.105:54820) 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c1019b0 0x7f907c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907affd640 1 -- 172.21.15.105:0/1502147820 learned_addr learned my addr 172.21.15.105:0/1502147820 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907affd640 1 -- 172.21.15.105:0/1502147820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c1019b0 msgr2=0x7f907c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907affd640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c1019b0 0x7f907c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907affd640 1 -- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f907c102370 msgr2=0x7f907c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907affd640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f907c102370 0x7f907c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.763+0000 7f907affd640 1 -- 172.21.15.105:0/1502147820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c19f770 con 0x7f907c1007c0 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.764+0000 7f907affd640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 0x7f907c197b50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f906c000c00 tx=0x7f906c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.764+0000 7f90808aa640 1 -- 172.21.15.105:0/1502147820 <== mon.2 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c043020 con 0x7f907c1007c0 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.764+0000 7f90808aa640 1 -- 172.21.15.105:0/1502147820 <== mon.2 v2:172.21.15.135:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f906c004350 con 0x7f907c1007c0 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.764+0000 7f90808aa640 1 -- 172.21.15.105:0/1502147820 <== mon.2 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c003b80 con 0x7f907c1007c0 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.764+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c19fa00 con 0x7f907c1007c0 2024-03-26T18:04:33.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.764+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c19ff50 con 0x7f907c1007c0 2024-03-26T18:04:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.765+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907c107c20 con 0x7f907c1007c0 2024-03-26T18:04:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.766+0000 7f90808aa640 1 -- 172.21.15.105:0/1502147820 <== mon.2 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f906c046020 con 0x7f907c1007c0 2024-03-26T18:04:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.766+0000 7f90808aa640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90640a2f20 0x7f90640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.766+0000 7f907a7fc640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90640a2f20 0x7f90640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.766+0000 7f90808aa640 1 -- 172.21.15.105:0/1502147820 <== mon.2 v2:172.21.15.135:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f906c0f1580 con 0x7f907c1007c0 2024-03-26T18:04:33.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.767+0000 7f907a7fc640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90640a2f20 0x7f90640a53e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f907c199060 tx=0x7f9070005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:33.750 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.771+0000 7f90808aa640 1 -- 172.21.15.105:0/1502147820 <== mon.2 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f906c04d020 con 0x7f907c1007c0 2024-03-26T18:04:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.960+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f907c02d480 con 0x7f907c1007c0 2024-03-26T18:04:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.961+0000 7f90808aa640 1 -- 172.21.15.105:0/1502147820 <== mon.2 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f906c0be370 con 0x7f907c1007c0 2024-03-26T18:04:33.940 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-03-26T18:04:33.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90640a2f20 msgr2=0x7f90640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90640a2f20 0x7f90640a53e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f907c199060 tx=0x7f9070005f90 comp rx=0 tx=0).stop 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 msgr2=0x7f907c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 0x7f907c197b50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f906c000c00 tx=0x7f906c005dc0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 shutdown_connections 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f90640a2f20 0x7f90640a53e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f907c102370 0x7f907c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f907c1019b0 0x7f907c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 --2- 172.21.15.105:0/1502147820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f907c1007c0 0x7f907c197b50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 >> 172.21.15.105:0/1502147820 conn(0x7f907c0fbd50 msgr2=0x7f907c104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 shutdown_connections 2024-03-26T18:04:33.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:33.963+0000 7f90818ac640 1 -- 172.21.15.105:0/1502147820 wait complete. 2024-03-26T18:04:33.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 -- 172.21.15.105:0/657120605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d01019c0 msgr2=0x7f40d0101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 --2- 172.21.15.105:0/657120605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d01019c0 0x7f40d0101e40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009860 tx=0x7f40c002bb70 comp rx=0 tx=0).stop 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 -- 172.21.15.105:0/657120605 shutdown_connections 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 --2- 172.21.15.105:0/657120605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40d0102380 0x7f40d0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 --2- 172.21.15.105:0/657120605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d01019c0 0x7f40d0101e40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 --2- 172.21.15.105:0/657120605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40d01007d0 0x7f40d0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 -- 172.21.15.105:0/657120605 >> 172.21.15.105:0/657120605 conn(0x7f40d00fbd60 msgr2=0x7f40d00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 -- 172.21.15.105:0/657120605 shutdown_connections 2024-03-26T18:04:34.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.092+0000 7f40d62a1640 1 -- 172.21.15.105:0/657120605 wait complete. 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d62a1640 1 Processor -- start 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d62a1640 1 -- start start 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d62a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40d01007d0 0x7f40d0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d62a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d01019c0 0x7f40d0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d62a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40d0102380 0x7f40d019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d62a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010d420 con 0x7f40d0102380 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d62a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010d2a0 con 0x7f40d01007d0 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d62a1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40d010d5a0 con 0x7f40d01019c0 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d483e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40d0102380 0x7f40d019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40d01007d0 0x7f40d0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d483e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40d0102380 0x7f40d019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38028/0 (socket says 172.21.15.105:38028) 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40d483e640 1 -- 172.21.15.105:0/1420715625 learned_addr learned my addr 172.21.15.105:0/1420715625 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40cffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40d01007d0 0x7f40d0197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.105:54844/0 (socket says 172.21.15.105:54844) 2024-03-26T18:04:34.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.093+0000 7f40cf7fe640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d01019c0 0x7f40d0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40d483e640 1 -- 172.21.15.105:0/1420715625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d01019c0 msgr2=0x7f40d0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40d483e640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d01019c0 0x7f40d0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40d483e640 1 -- 172.21.15.105:0/1420715625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40d01007d0 msgr2=0x7f40d0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40d483e640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40d01007d0 0x7f40d0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40d483e640 1 -- 172.21.15.105:0/1420715625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d019f720 con 0x7f40d0102380 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40cffff640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40d01007d0 0x7f40d0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40d483e640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40d0102380 0x7f40d019f0b0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f40c800eb10 tx=0x7f40c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40cd7fa640 1 -- 172.21.15.105:0/1420715625 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8004b20 con 0x7f40d0102380 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d019fa10 con 0x7f40d0102380 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40cd7fa640 1 -- 172.21.15.105:0/1420715625 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40c8004cc0 con 0x7f40d0102380 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40cd7fa640 1 -- 172.21.15.105:0/1420715625 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c8010690 con 0x7f40d0102380 2024-03-26T18:04:34.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.094+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d019ff80 con 0x7f40d0102380 2024-03-26T18:04:34.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.095+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d0107bb0 con 0x7f40d0102380 2024-03-26T18:04:34.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.096+0000 7f40cd7fa640 1 -- 172.21.15.105:0/1420715625 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f40c8005000 con 0x7f40d0102380 2024-03-26T18:04:34.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.097+0000 7f40cd7fa640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40b80a2f20 0x7f40b80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:34.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.097+0000 7f40cffff640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40b80a2f20 0x7f40b80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:34.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.097+0000 7f40cd7fa640 1 -- 172.21.15.105:0/1420715625 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f40c80d00e0 con 0x7f40d0102380 2024-03-26T18:04:34.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.098+0000 7f40cffff640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40b80a2f20 0x7f40b80a53e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f40c40096c0 tx=0x7f40c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:34.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.101+0000 7f40cd7fa640 1 -- 172.21.15.105:0/1420715625 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f40c809ced0 con 0x7f40d0102380 2024-03-26T18:04:34.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.289+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40d002d240 con 0x7f40d0102380 2024-03-26T18:04:34.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.290+0000 7f40cd7fa640 1 -- 172.21.15.105:0/1420715625 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+601 (secure 0 0 0) 0x7f40c809c7d0 con 0x7f40d0102380 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/1242681476 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/976039762 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6801/1556474900 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/1151470472 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/3821116771 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6800/3642413152 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/2484248025 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:0/2017206095 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6801/3642413152 2024-03-27T17:28:59.817767+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105:6800/1556474900 2024-03-27T17:29:26.501525+0000 2024-03-26T18:04:34.270 INFO:teuthology.orchestra.run.smithi105.stderr:listed 10 entries 2024-03-26T18:04:34.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40b80a2f20 msgr2=0x7f40b80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:34.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40b80a2f20 0x7f40b80a53e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f40c40096c0 tx=0x7f40c4009260 comp rx=0 tx=0).stop 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40d0102380 msgr2=0x7f40d019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40d0102380 0x7f40d019f0b0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f40c800eb10 tx=0x7f40c800c2d0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 shutdown_connections 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f40b80a2f20 0x7f40b80a53e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40d0102380 0x7f40d019f0b0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40d01019c0 0x7f40d0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 --2- 172.21.15.105:0/1420715625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40d01007d0 0x7f40d0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 >> 172.21.15.105:0/1420715625 conn(0x7f40d00fbd60 msgr2=0x7f40d0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 shutdown_connections 2024-03-26T18:04:34.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:34.292+0000 7f40d62a1640 1 -- 172.21.15.105:0/1420715625 wait complete. 2024-03-26T18:04:34.285 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-03-26T18:04:34.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:04:34.285 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-26T18:04:34.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi105.stderr:ceph-fuse[83984]: fuse finished with error 0 and tester_r 0 2024-03-26T18:04:34.334 INFO:teuthology.orchestra.run:waiting for 300 2024-03-26T18:04:34.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:34 smithi144 ceph-mon[36095]: pgmap v1139: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:34.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:34 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3848259863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:34 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3365110006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:34 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1502147820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:34 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/1420715625' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:04:34.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:34 smithi105 ceph-mon[32280]: pgmap v1139: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:34.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:34 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3848259863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:34 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3365110006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:34 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1502147820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:34 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/1420715625' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:04:34.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:34 smithi135 ceph-mon[35636]: pgmap v1139: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-03-26T18:04:34.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:34 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3848259863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:34 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3365110006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:34 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1502147820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T18:04:34.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:34 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/1420715625' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T18:04:36.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:36 smithi144 ceph-mon[36095]: pgmap v1140: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:36.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:36 smithi105 ceph-mon[32280]: pgmap v1140: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:36.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:36 smithi135 ceph-mon[35636]: pgmap v1140: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-26T18:04:38.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:38 smithi144 ceph-mon[36095]: pgmap v1141: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T18:04:38.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:38 smithi105 ceph-mon[32280]: pgmap v1141: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T18:04:38.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:38 smithi135 ceph-mon[35636]: pgmap v1141: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 11 op/s 2024-03-26T18:04:40.435 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T18:04:40.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:04:40.436 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-26T18:04:40.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:04:40.468 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-03-26T18:04:40.523 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-26T18:04:40.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:04:40.523 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T18:04:40.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:40 smithi144 ceph-mon[36095]: pgmap v1142: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-26T18:04:40.586 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:40 smithi105 ceph-mon[32280]: pgmap v1142: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-26T18:04:40.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T18:04:40.590 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-03-26T18:04:40.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:40 smithi135 ceph-mon[35636]: pgmap v1142: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-26T18:04:40.688 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-26T18:04:40.698 INFO:tasks.cephadm:Teardown begin 2024-03-26T18:04:40.699 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-26T18:04:40.699 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:04:40.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 -- 172.21.15.105:0/873341978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc102360 msgr2=0x7f38cc104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:40.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 --2- 172.21.15.105:0/873341978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc102360 0x7f38cc104750 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009900 tx=0x7f38bc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:40.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 -- 172.21.15.105:0/873341978 shutdown_connections 2024-03-26T18:04:40.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 --2- 172.21.15.105:0/873341978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38cc1075c0 0x7f38cc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:40.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 --2- 172.21.15.105:0/873341978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38cc104c90 0x7f38cc107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:40.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 --2- 172.21.15.105:0/873341978 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc102360 0x7f38cc104750 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:40.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 -- 172.21.15.105:0/873341978 >> 172.21.15.105:0/873341978 conn(0x7f38cc0fbd50 msgr2=0x7f38cc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:40.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 -- 172.21.15.105:0/873341978 shutdown_connections 2024-03-26T18:04:40.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.826+0000 7f38d131b640 1 -- 172.21.15.105:0/873341978 wait complete. 2024-03-26T18:04:40.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.827+0000 7f38d131b640 1 Processor -- start 2024-03-26T18:04:40.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.827+0000 7f38d131b640 1 -- start start 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38d131b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38cc102360 0x7f38cc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38d131b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38cc104c90 0x7f38cc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38d131b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc1075c0 0x7f38cc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38d131b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc10d410 con 0x7f38cc1075c0 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38d131b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc10d290 con 0x7f38cc104c90 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38d131b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc10d590 con 0x7f38cc102360 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc1075c0 0x7f38cc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38caffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38cc102360 0x7f38cc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38caffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38cc102360 0x7f38cc197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.105:46202/0 (socket says 172.21.15.105:46202) 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38c27fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38cc104c90 0x7f38cc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38caffd640 1 -- 172.21.15.105:0/2875604711 learned_addr learned my addr 172.21.15.105:0/2875604711 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc1075c0 0x7f38cc19f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34446/0 (socket says 172.21.15.105:34446) 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38cb7fe640 1 -- 172.21.15.105:0/2875604711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38cc102360 msgr2=0x7f38cc197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38cb7fe640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38cc102360 0x7f38cc197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:40.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38cb7fe640 1 -- 172.21.15.105:0/2875604711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38cc104c90 msgr2=0x7f38cc1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38cb7fe640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38cc104c90 0x7f38cc1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38cb7fe640 1 -- 172.21.15.105:0/2875604711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc19f780 con 0x7f38cc1075c0 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38caffd640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38cc102360 0x7f38cc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38cb7fe640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc1075c0 0x7f38cc19f110 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f38c400da10 tx=0x7f38c400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38c8ff9640 1 -- 172.21.15.105:0/2875604711 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c400bd70 con 0x7f38cc1075c0 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38c8ff9640 1 -- 172.21.15.105:0/2875604711 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c4003e20 con 0x7f38cc1075c0 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38cc19fa70 con 0x7f38cc1075c0 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38c8ff9640 1 -- 172.21.15.105:0/2875604711 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c40106c0 con 0x7f38cc1075c0 2024-03-26T18:04:40.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.828+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38cc19ff70 con 0x7f38cc1075c0 2024-03-26T18:04:40.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.830+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38cc0fd840 con 0x7f38cc1075c0 2024-03-26T18:04:40.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.831+0000 7f38c8ff9640 1 -- 172.21.15.105:0/2875604711 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f38c4003970 con 0x7f38cc1075c0 2024-03-26T18:04:40.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.832+0000 7f38c8ff9640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38b80a2f20 0x7f38b80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:40.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.832+0000 7f38caffd640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38b80a2f20 0x7f38b80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:40.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.832+0000 7f38c8ff9640 1 -- 172.21.15.105:0/2875604711 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f38c40cf750 con 0x7f38cc1075c0 2024-03-26T18:04:40.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.833+0000 7f38caffd640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38b80a2f20 0x7f38b80a53e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0096c0 tx=0x7f38bc02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:40.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:40.837+0000 7f38c8ff9640 1 -- 172.21.15.105:0/2875604711 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f38c409c540 con 0x7f38cc1075c0 2024-03-26T18:04:41.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.056+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38cc0624c0 con 0x7f38cc1075c0 2024-03-26T18:04:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.056+0000 7f38c8ff9640 1 -- 172.21.15.105:0/2875604711 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11946 (secure 0 0 0) 0x7f38c409be40 con 0x7f38cc1075c0 2024-03-26T18:04:41.036 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:04:41.036 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:03:24.037550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_33599":{"gid":33599,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1455536711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1455536711},{"type":"v1","addr":"172.21.15.135:6835","nonce":1455536711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24482,14628,24487,24466]},"id":1}]} 2024-03-26T18:04:41.036 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 48 2024-03-26T18:04:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.058+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38b80a2f20 msgr2=0x7f38b80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.058+0000 7f38d131b640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38b80a2f20 0x7f38b80a53e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0096c0 tx=0x7f38bc02c7e0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.058+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc1075c0 msgr2=0x7f38cc19f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.058+0000 7f38d131b640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc1075c0 0x7f38cc19f110 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f38c400da10 tx=0x7f38c400ded0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.058+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 shutdown_connections 2024-03-26T18:04:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.059+0000 7f38d131b640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f38b80a2f20 0x7f38b80a53e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.038 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.059+0000 7f38d131b640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38cc1075c0 0x7f38cc19f110 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.059+0000 7f38d131b640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f38cc104c90 0x7f38cc1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.059+0000 7f38d131b640 1 --2- 172.21.15.105:0/2875604711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38cc102360 0x7f38cc197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.059+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 >> 172.21.15.105:0/2875604711 conn(0x7f38cc0fbd50 msgr2=0x7f38cc102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.059+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 shutdown_connections 2024-03-26T18:04:41.039 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.059+0000 7f38d131b640 1 -- 172.21.15.105:0/2875604711 wait complete. 2024-03-26T18:04:41.050 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 -- 172.21.15.105:0/2317552778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3754102390 msgr2=0x7f3754108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 --2- 172.21.15.105:0/2317552778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3754102390 0x7f3754108c20 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f374c009ca0 tx=0x7f374c02c0f0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 -- 172.21.15.105:0/2317552778 shutdown_connections 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 --2- 172.21.15.105:0/2317552778 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3754102390 0x7f3754108c20 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 --2- 172.21.15.105:0/2317552778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37541019d0 0x7f3754101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 --2- 172.21.15.105:0/2317552778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37541007e0 0x7f3754100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 -- 172.21.15.105:0/2317552778 >> 172.21.15.105:0/2317552778 conn(0x7f37540fbd50 msgr2=0x7f37540fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 -- 172.21.15.105:0/2317552778 shutdown_connections 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 -- 172.21.15.105:0/2317552778 wait complete. 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 Processor -- start 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 -- start start 2024-03-26T18:04:41.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37541007e0 0x7f3754196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37541019d0 0x7f3754196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3754102390 0x7f375419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f375410c310 con 0x7f37541019d0 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.174+0000 7f375a885640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f375410c190 con 0x7f3754102390 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f375a885640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f375410c490 con 0x7f37541007e0 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37541019d0 0x7f3754196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37541019d0 0x7f3754196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34468/0 (socket says 172.21.15.105:34468) 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f3753fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37541007e0 0x7f3754196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 -- 172.21.15.105:0/2653102369 learned_addr learned my addr 172.21.15.105:0/2653102369 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f3758e22640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3754102390 0x7f375419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 -- 172.21.15.105:0/2653102369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37541007e0 msgr2=0x7f3754196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37541007e0 0x7f3754196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 -- 172.21.15.105:0/2653102369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3754102390 msgr2=0x7f375419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3754102390 0x7f375419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 -- 172.21.15.105:0/2653102369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375419e630 con 0x7f37541019d0 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37537fe640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37541019d0 0x7f3754196f50 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f374800d950 tx=0x7f374800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37517fa640 1 -- 172.21.15.105:0/2653102369 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748017030 con 0x7f37541019d0 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375419e920 con 0x7f37541019d0 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37517fa640 1 -- 172.21.15.105:0/2653102369 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3748004020 con 0x7f37541019d0 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f37517fa640 1 -- 172.21.15.105:0/2653102369 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480047a0 con 0x7f37541019d0 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.175+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375419eec0 con 0x7f37541019d0 2024-03-26T18:04:41.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.176+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3754107b00 con 0x7f37541019d0 2024-03-26T18:04:41.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.178+0000 7f37517fa640 1 -- 172.21.15.105:0/2653102369 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f37480041d0 con 0x7f37541019d0 2024-03-26T18:04:41.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.178+0000 7f37517fa640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f373c0a2f20 0x7f373c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.178+0000 7f3753fff640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f373c0a2f20 0x7f373c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.178+0000 7f37517fa640 1 -- 172.21.15.105:0/2653102369 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7f37480cf460 con 0x7f37541019d0 2024-03-26T18:04:41.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.179+0000 7f3753fff640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f373c0a2f20 0x7f373c0a53e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3754101830 tx=0x7f3744005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:41.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.182+0000 7f37517fa640 1 -- 172.21.15.105:0/2653102369 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f374809c250 con 0x7f37541019d0 2024-03-26T18:04:41.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.399+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f375402d240 con 0x7f37541019d0 2024-03-26T18:04:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.399+0000 7f37517fa640 1 -- 172.21.15.105:0/2653102369 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11946 (secure 0 0 0) 0x7f374809bb50 con 0x7f37541019d0 2024-03-26T18:04:41.379 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:04:41.379 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3032565642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":3032565642},{"type":"v1","addr":"172.21.15.105:6835","nonce":3032565642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2045143293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2045143293},{"type":"v1","addr":"172.21.15.105:6837","nonce":2045143293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T17:36:22.059727+0000","modified":"2024-03-26T18:03:24.037550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24521,"mds_1":24487,"mds_2":24482,"mds_3":24466,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6839/3053296748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":3053296748},{"type":"v1","addr":"172.21.15.105:6839","nonce":3053296748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":24,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6841/3843959520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6840","nonce":3843959520},{"type":"v1","addr":"172.21.15.105:6841","nonce":3843959520}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6833/3569777961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3569777961},{"type":"v1","addr":"172.21.15.144:6833","nonce":3569777961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6833/1499648854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":1499648854},{"type":"v1","addr":"172.21.15.135:6833","nonce":1499648854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6835/29089236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":29089236},{"type":"v1","addr":"172.21.15.144:6835","nonce":29089236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":22,"state":"up:active","state_seq":8,"addr":"172.21.15.144:6837/184508646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":184508646},{"type":"v1","addr":"172.21.15.144:6837","nonce":184508646}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6837/388686483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":388686483},{"type":"v1","addr":"172.21.15.135:6837","nonce":388686483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6839/1175380833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6838","nonce":1175380833},{"type":"v1","addr":"172.21.15.144:6839","nonce":1175380833}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.135:6839/338591305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":338591305},{"type":"v1","addr":"172.21.15.135:6839","nonce":338591305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_33599":{"gid":33599,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/1455536711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":1455536711},{"type":"v1","addr":"172.21.15.135:6835","nonce":1455536711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24521,24482,14628,24487,24466]},"id":1}]} 2024-03-26T18:04:41.379 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 48 2024-03-26T18:04:41.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.401+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f373c0a2f20 msgr2=0x7f373c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.401+0000 7f375a885640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f373c0a2f20 0x7f373c0a53e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3754101830 tx=0x7f3744005f90 comp rx=0 tx=0).stop 2024-03-26T18:04:41.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.401+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37541019d0 msgr2=0x7f3754196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.401+0000 7f375a885640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37541019d0 0x7f3754196f50 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f374800d950 tx=0x7f374800de10 comp rx=0 tx=0).stop 2024-03-26T18:04:41.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.402+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 shutdown_connections 2024-03-26T18:04:41.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.402+0000 7f375a885640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f373c0a2f20 0x7f373c0a53e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.402+0000 7f375a885640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3754102390 0x7f375419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.402+0000 7f375a885640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37541019d0 0x7f3754196f50 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.402+0000 7f375a885640 1 --2- 172.21.15.105:0/2653102369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37541007e0 0x7f3754196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.402+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 >> 172.21.15.105:0/2653102369 conn(0x7f37540fbd50 msgr2=0x7f3754104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:41.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.402+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 shutdown_connections 2024-03-26T18:04:41.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.402+0000 7f375a885640 1 -- 172.21.15.105:0/2653102369 wait complete. 2024-03-26T18:04:41.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T18:04:41.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 -- 172.21.15.105:0/3315674067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec101800 msgr2=0x7fd5ec101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3315674067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec101800 0x7fd5ec101c80 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009900 tx=0x7fd5dc02bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 -- 172.21.15.105:0/3315674067 shutdown_connections 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3315674067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ec1021c0 0x7fd5ec108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3315674067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec101800 0x7fd5ec101c80 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3315674067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5ec100610 0x7fd5ec100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 -- 172.21.15.105:0/3315674067 >> 172.21.15.105:0/3315674067 conn(0x7fd5ec0fbba0 msgr2=0x7fd5ec0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 -- 172.21.15.105:0/3315674067 shutdown_connections 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 -- 172.21.15.105:0/3315674067 wait complete. 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 Processor -- start 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.519+0000 7fd5f339d640 1 -- start start 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f339d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec100610 0x7fd5ec0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f339d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5ec101800 0x7fd5ec19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f339d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ec1021c0 0x7fd5ec19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f339d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec10c150 con 0x7fd5ec100610 2024-03-26T18:04:41.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f339d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec10bfd0 con 0x7fd5ec101800 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f339d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec10c2d0 con 0x7fd5ec1021c0 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec100610 0x7fd5ec0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec100610 0x7fd5ec0695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34484/0 (socket says 172.21.15.105:34484) 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 -- 172.21.15.105:0/3054656387 learned_addr learned my addr 172.21.15.105:0/3054656387 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f1b9a640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5ec101800 0x7fd5ec19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f2b9c640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ec1021c0 0x7fd5ec19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 -- 172.21.15.105:0/3054656387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ec1021c0 msgr2=0x7fd5ec19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ec1021c0 0x7fd5ec19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 -- 172.21.15.105:0/3054656387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5ec101800 msgr2=0x7fd5ec19c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5ec101800 0x7fd5ec19c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 -- 172.21.15.105:0/3054656387 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec19f530 con 0x7fd5ec100610 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f239b640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec100610 0x7fd5ec0695f0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800e9c0 tx=0x7fd5e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5e37fe640 1 -- 172.21.15.105:0/3054656387 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800cd80 con 0x7fd5ec100610 2024-03-26T18:04:41.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ec19f820 con 0x7fd5ec100610 2024-03-26T18:04:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5e37fe640 1 -- 172.21.15.105:0/3054656387 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5e80035f0 con 0x7fd5ec100610 2024-03-26T18:04:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5e37fe640 1 -- 172.21.15.105:0/3054656387 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8010690 con 0x7fd5ec100610 2024-03-26T18:04:41.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.520+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ec19fd40 con 0x7fd5ec100610 2024-03-26T18:04:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.521+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ec107a00 con 0x7fd5ec100610 2024-03-26T18:04:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.523+0000 7fd5e37fe640 1 -- 172.21.15.105:0/3054656387 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd5e80037a0 con 0x7fd5ec100610 2024-03-26T18:04:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.524+0000 7fd5e37fe640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5d80a2f20 0x7fd5d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.524+0000 7fd5f1b9a640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5d80a2f20 0x7fd5d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.524+0000 7fd5e37fe640 1 -- 172.21.15.105:0/3054656387 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7fd5e80cf7a0 con 0x7fd5ec100610 2024-03-26T18:04:41.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.524+0000 7fd5f1b9a640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5d80a2f20 0x7fd5d80a53e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009230 tx=0x7fd5dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:41.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.528+0000 7fd5e37fe640 1 -- 172.21.15.105:0/3054656387 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd5e809c590 con 0x7fd5ec100610 2024-03-26T18:04:41.585 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:41 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2875604711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:04:41.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:41 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2875604711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:04:41.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:41 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2875604711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:04:41.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.710+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5ec02d240 con 0x7fd5ec100610 2024-03-26T18:04:41.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.712+0000 7fd5e37fe640 1 -- 172.21.15.105:0/3054656387 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+22833 (secure 0 0 0) 0x7fd5e809be90 con 0x7fd5ec100610 2024-03-26T18:04:41.692 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-26T18:04:41.692 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":96,"fsid":"15cee578-eb96-11ee-95d0-87774f69a715","created":"2024-03-26T17:28:32.621716+0000","modified":"2024-03-26T18:02:07.352921+0000","last_up_change":"2024-03-26T17:35:32.806664+0000","last_in_change":"2024-03-26T17:35:12.034267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T17:31:50.834121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T17:36:20.108300+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T17:36:21.285220+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42769940-d924-4456-ab99-af1c060516a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6803","nonce":101464397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6805","nonce":101464397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6809","nonce":101464397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":101464397},{"type":"v1","addr":"172.21.15.105:6807","nonce":101464397}]},"public_addr":"172.21.15.105:6803/101464397","cluster_addr":"172.21.15.105:6805/101464397","heartbeat_back_addr":"172.21.15.105:6809/101464397","heartbeat_front_addr":"172.21.15.105:6807/101464397","state":["exists","up"]},{"osd":1,"uuid":"33937b10-f283-4680-a12c-ee35827a2bd2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6801","nonce":932873598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6803","nonce":932873598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6807","nonce":932873598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":932873598},{"type":"v1","addr":"172.21.15.135:6805","nonce":932873598}]},"public_addr":"172.21.15.135:6801/932873598","cluster_addr":"172.21.15.135:6803/932873598","heartbeat_back_addr":"172.21.15.135:6807/932873598","heartbeat_front_addr":"172.21.15.135:6805/932873598","state":["exists","up"]},{"osd":2,"uuid":"f434f0a6-ad51-45e8-b9e9-411cead27014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6801","nonce":386966961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6803","nonce":386966961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6807","nonce":386966961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":386966961},{"type":"v1","addr":"172.21.15.144:6805","nonce":386966961}]},"public_addr":"172.21.15.144:6801/386966961","cluster_addr":"172.21.15.144:6803/386966961","heartbeat_back_addr":"172.21.15.144:6807/386966961","heartbeat_front_addr":"172.21.15.144:6805/386966961","state":["exists","up"]},{"osd":3,"uuid":"f52f7e5b-63cd-43ea-a2d9-5b3282dbb996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6811","nonce":2769664525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6813","nonce":2769664525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6817","nonce":2769664525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2769664525},{"type":"v1","addr":"172.21.15.105:6815","nonce":2769664525}]},"public_addr":"172.21.15.105:6811/2769664525","cluster_addr":"172.21.15.105:6813/2769664525","heartbeat_back_addr":"172.21.15.105:6817/2769664525","heartbeat_front_addr":"172.21.15.105:6815/2769664525","state":["exists","up"]},{"osd":4,"uuid":"a9450083-abb4-4382-b2eb-96f679b03bdb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6809","nonce":507590464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6811","nonce":507590464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6815","nonce":507590464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":507590464},{"type":"v1","addr":"172.21.15.135:6813","nonce":507590464}]},"public_addr":"172.21.15.135:6809/507590464","cluster_addr":"172.21.15.135:6811/507590464","heartbeat_back_addr":"172.21.15.135:6815/507590464","heartbeat_front_addr":"172.21.15.135:6813/507590464","state":["exists","up"]},{"osd":5,"uuid":"c13547ad-d6de-4be7-8548-7444afdf81e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6809","nonce":3886741876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6811","nonce":3886741876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6815","nonce":3886741876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":3886741876},{"type":"v1","addr":"172.21.15.144:6813","nonce":3886741876}]},"public_addr":"172.21.15.144:6809/3886741876","cluster_addr":"172.21.15.144:6811/3886741876","heartbeat_back_addr":"172.21.15.144:6815/3886741876","heartbeat_front_addr":"172.21.15.144:6813/3886741876","state":["exists","up"]},{"osd":6,"uuid":"342c60b1-acab-414f-b483-bc6ab61d927e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6819","nonce":3456307470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6821","nonce":3456307470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6825","nonce":3456307470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":3456307470},{"type":"v1","addr":"172.21.15.105:6823","nonce":3456307470}]},"public_addr":"172.21.15.105:6819/3456307470","cluster_addr":"172.21.15.105:6821/3456307470","heartbeat_back_addr":"172.21.15.105:6825/3456307470","heartbeat_front_addr":"172.21.15.105:6823/3456307470","state":["exists","up"]},{"osd":7,"uuid":"28f0b5ba-25fb-4bdb-a2d8-7221cb8345fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6817","nonce":788617133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6819","nonce":788617133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6823","nonce":788617133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":788617133},{"type":"v1","addr":"172.21.15.135:6821","nonce":788617133}]},"public_addr":"172.21.15.135:6817/788617133","cluster_addr":"172.21.15.135:6819/788617133","heartbeat_back_addr":"172.21.15.135:6823/788617133","heartbeat_front_addr":"172.21.15.135:6821/788617133","state":["exists","up"]},{"osd":8,"uuid":"779c4b05-662f-4332-994a-5cb93ba4aabf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6817","nonce":4074146904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6819","nonce":4074146904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6823","nonce":4074146904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":4074146904},{"type":"v1","addr":"172.21.15.144:6821","nonce":4074146904}]},"public_addr":"172.21.15.144:6817/4074146904","cluster_addr":"172.21.15.144:6819/4074146904","heartbeat_back_addr":"172.21.15.144:6823/4074146904","heartbeat_front_addr":"172.21.15.144:6821/4074146904","state":["exists","up"]},{"osd":9,"uuid":"0f1218e3-f488-426a-8ac6-afece38016e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6827","nonce":1324177840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6829","nonce":1324177840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6833","nonce":1324177840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":1324177840},{"type":"v1","addr":"172.21.15.105:6831","nonce":1324177840}]},"public_addr":"172.21.15.105:6827/1324177840","cluster_addr":"172.21.15.105:6829/1324177840","heartbeat_back_addr":"172.21.15.105:6833/1324177840","heartbeat_front_addr":"172.21.15.105:6831/1324177840","state":["exists","up"]},{"osd":10,"uuid":"47f4a8a1-7c2d-4e70-a83e-03296b92743c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6825","nonce":3021911997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6827","nonce":3021911997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6831","nonce":3021911997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3021911997},{"type":"v1","addr":"172.21.15.135:6829","nonce":3021911997}]},"public_addr":"172.21.15.135:6825/3021911997","cluster_addr":"172.21.15.135:6827/3021911997","heartbeat_back_addr":"172.21.15.135:6831/3021911997","heartbeat_front_addr":"172.21.15.135:6829/3021911997","state":["exists","up"]},{"osd":11,"uuid":"82fd99c0-5cc2-4744-b31a-45dac10ca1db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6825","nonce":4293375044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6827","nonce":4293375044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6831","nonce":4293375044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4293375044},{"type":"v1","addr":"172.21.15.144:6829","nonce":4293375044}]},"public_addr":"172.21.15.144:6825/4293375044","cluster_addr":"172.21.15.144:6827/4293375044","heartbeat_back_addr":"172.21.15.144:6831/4293375044","heartbeat_front_addr":"172.21.15.144:6829/4293375044","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:30:59.330304+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:23.229426+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:31:47.162665+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:12.311129+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:32:36.248493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:00.741685+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:25.305713+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:33:49.777375+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:14.209545+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:34:39.375713+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:04.967670+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T17:35:30.075132+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9},{"from":4,"to":6}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":7}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3f","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:0/1242681476":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/976039762":"2024-03-27T17:29:26.501525+0000","172.21.15.105:6801/1556474900":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/1151470472":"2024-03-27T17:29:26.501525+0000","172.21.15.105:0/3821116771":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2484248025":"2024-03-27T17:28:59.817767+0000","172.21.15.105:0/2017206095":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6801/3642413152":"2024-03-27T17:28:59.817767+0000","172.21.15.105:6800/1556474900":"2024-03-27T17:29:26.501525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T18:04:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.714+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5d80a2f20 msgr2=0x7fd5d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.714+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5d80a2f20 0x7fd5d80a53e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009230 tx=0x7fd5dc005fd0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.714+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec100610 msgr2=0x7fd5ec0695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.714+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec100610 0x7fd5ec0695f0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fd5e800e9c0 tx=0x7fd5e800ee80 comp rx=0 tx=0).stop 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.715+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 shutdown_connections 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.715+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd5d80a2f20 0x7fd5d80a53e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.715+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5ec1021c0 0x7fd5ec19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.715+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5ec101800 0x7fd5ec19c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.715+0000 7fd5f339d640 1 --2- 172.21.15.105:0/3054656387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5ec100610 0x7fd5ec0695f0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.715+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 >> 172.21.15.105:0/3054656387 conn(0x7fd5ec0fbba0 msgr2=0x7fd5ec104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.715+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 shutdown_connections 2024-03-26T18:04:41.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.715+0000 7fd5f339d640 1 -- 172.21.15.105:0/3054656387 wait complete. 2024-03-26T18:04:41.709 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-26T18:04:41.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 -- 172.21.15.105:0/2708217334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01007d0 msgr2=0x7ff9c0100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2708217334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01007d0 0x7ff9c0100bf0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009900 tx=0x7ff9b002bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:41.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 -- 172.21.15.105:0/2708217334 shutdown_connections 2024-03-26T18:04:41.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2708217334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c0102380 0x7ff9c0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2708217334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9c01019c0 0x7ff9c0101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2708217334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01007d0 0x7ff9c0100bf0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 -- 172.21.15.105:0/2708217334 >> 172.21.15.105:0/2708217334 conn(0x7ff9c00fbd60 msgr2=0x7ff9c00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:41.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 -- 172.21.15.105:0/2708217334 shutdown_connections 2024-03-26T18:04:41.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.834+0000 7ff9c5012640 1 -- 172.21.15.105:0/2708217334 wait complete. 2024-03-26T18:04:41.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9c5012640 1 Processor -- start 2024-03-26T18:04:41.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9c5012640 1 -- start start 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9c5012640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c01007d0 0x7ff9c0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9c5012640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01019c0 0x7ff9c0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9c5012640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9c0102380 0x7ff9c019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9c5012640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010c2f0 con 0x7ff9c01019c0 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9c5012640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010c170 con 0x7ff9c01007d0 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9c5012640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c010c470 con 0x7ff9c0102380 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9be59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01019c0 0x7ff9c0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9be59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01019c0 0x7ff9c0198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34502/0 (socket says 172.21.15.105:34502) 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9be59c640 1 -- 172.21.15.105:0/2071093262 learned_addr learned my addr 172.21.15.105:0/2071093262 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9bed9d640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c01007d0 0x7ff9c0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.835+0000 7ff9bf59e640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9c0102380 0x7ff9c019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9be59c640 1 -- 172.21.15.105:0/2071093262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9c0102380 msgr2=0x7ff9c019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9be59c640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9c0102380 0x7ff9c019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9be59c640 1 -- 172.21.15.105:0/2071093262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c01007d0 msgr2=0x7ff9c0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9be59c640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c01007d0 0x7ff9c0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9be59c640 1 -- 172.21.15.105:0/2071093262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c019f770 con 0x7ff9c01019c0 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9be59c640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01019c0 0x7ff9c0198090 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e9f0 tx=0x7ff9b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9a7fff640 1 -- 172.21.15.105:0/2071093262 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400cd80 con 0x7ff9c01019c0 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c019fa60 con 0x7ff9c01019c0 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9a7fff640 1 -- 172.21.15.105:0/2071093262 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9b40054d0 con 0x7ff9c01019c0 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9a7fff640 1 -- 172.21.15.105:0/2071093262 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4010490 con 0x7ff9c01019c0 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.836+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c019ff60 con 0x7ff9c01019c0 2024-03-26T18:04:41.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.837+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c01a01f0 con 0x7ff9c01019c0 2024-03-26T18:04:41.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.838+0000 7ff9a7fff640 1 -- 172.21.15.105:0/2071093262 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7ff9b4005020 con 0x7ff9c01019c0 2024-03-26T18:04:41.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.839+0000 7ff9a7fff640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9a80a2f20 0x7ff9a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:41.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.839+0000 7ff9bed9d640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9a80a2f20 0x7ff9a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:41.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.839+0000 7ff9a7fff640 1 -- 172.21.15.105:0/2071093262 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8293+0+0 (secure 0 0 0) 0x7ff9b40cf770 con 0x7ff9c01019c0 2024-03-26T18:04:41.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.839+0000 7ff9bed9d640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9a80a2f20 0x7ff9a80a53e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005ab0 tx=0x7ff9b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:41.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:41.844+0000 7ff9a7fff640 1 -- 172.21.15.105:0/2071093262 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff9b409c560 con 0x7ff9c01019c0 2024-03-26T18:04:42.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.101+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff9c01a03c0 con 0x7ff9c01019c0 2024-03-26T18:04:42.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.103+0000 7ff9a7fff640 1 -- 172.21.15.105:0/2071093262 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7ff9b409be60 con 0x7ff9c01019c0 2024-03-26T18:04:42.083 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-26T18:04:42.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9a80a2f20 msgr2=0x7ff9a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:42.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9a80a2f20 0x7ff9a80a53e0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005ab0 tx=0x7ff9b0005fd0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01019c0 msgr2=0x7ff9c0198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01019c0 0x7ff9c0198090 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400e9f0 tx=0x7ff9b400edf0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 shutdown_connections 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7ff9a80a2f20 0x7ff9a80a53e0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9c0102380 0x7ff9c019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff9c01019c0 0x7ff9c0198090 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 --2- 172.21.15.105:0/2071093262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff9c01007d0 0x7ff9c0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 >> 172.21.15.105:0/2071093262 conn(0x7ff9c00fbd60 msgr2=0x7ff9c0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 shutdown_connections 2024-03-26T18:04:42.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.107+0000 7ff9c5012640 1 -- 172.21.15.105:0/2071093262 wait complete. 2024-03-26T18:04:42.101 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-26T18:04:42.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.233+0000 7fb48ce0b640 1 -- 172.21.15.105:0/4209251767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 msgr2=0x7fb488104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:42.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.233+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/4209251767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 0x7fb488104770 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fb480009900 tx=0x7fb48002bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:42.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 -- 172.21.15.105:0/4209251767 shutdown_connections 2024-03-26T18:04:42.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/4209251767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4881075e0 0x7fb4881099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/4209251767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb488104cb0 0x7fb4881070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/4209251767 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 0x7fb488104770 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 -- 172.21.15.105:0/4209251767 >> 172.21.15.105:0/4209251767 conn(0x7fb4880fbd50 msgr2=0x7fb4880fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 -- 172.21.15.105:0/4209251767 shutdown_connections 2024-03-26T18:04:42.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 -- 172.21.15.105:0/4209251767 wait complete. 2024-03-26T18:04:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 Processor -- start 2024-03-26T18:04:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.234+0000 7fb48ce0b640 1 -- start start 2024-03-26T18:04:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48ce0b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 0x7fb488197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48ce0b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb488104cb0 0x7fb4881980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48ce0b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4881075e0 0x7fb48819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:42.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48ce0b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810d3e0 con 0x7fb488102380 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48ce0b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810d260 con 0x7fb488104cb0 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48ce0b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810d560 con 0x7fb4881075e0 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 0x7fb488197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 0x7fb488197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34562/0 (socket says 172.21.15.105:34562) 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 -- 172.21.15.105:0/2775385078 learned_addr learned my addr 172.21.15.105:0/2775385078 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb485d9b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb488104cb0 0x7fb4881980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb486d9d640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4881075e0 0x7fb48819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 -- 172.21.15.105:0/2775385078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4881075e0 msgr2=0x7fb48819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4881075e0 0x7fb48819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 -- 172.21.15.105:0/2775385078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb488104cb0 msgr2=0x7fb4881980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb488104cb0 0x7fb4881980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 -- 172.21.15.105:0/2775385078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48819f7c0 con 0x7fb488102380 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48659c640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 0x7fb488197ba0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fb480006010 tx=0x7fb4800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:42.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb4777fe640 1 -- 172.21.15.105:0/2775385078 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480043020 con 0x7fb488102380 2024-03-26T18:04:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48819fa50 con 0x7fb488102380 2024-03-26T18:04:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb4777fe640 1 -- 172.21.15.105:0/2775385078 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb480004aa0 con 0x7fb488102380 2024-03-26T18:04:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb4777fe640 1 -- 172.21.15.105:0/2775385078 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480003c60 con 0x7fb488102380 2024-03-26T18:04:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.235+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48819fff0 con 0x7fb488102380 2024-03-26T18:04:42.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.236+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4880686b0 con 0x7fb488102380 2024-03-26T18:04:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.238+0000 7fb4777fe640 1 -- 172.21.15.105:0/2775385078 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fb480046020 con 0x7fb488102380 2024-03-26T18:04:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.239+0000 7fb4777fe640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4700a2f20 0x7fb4700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.239+0000 7fb485d9b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4700a2f20 0x7fb4700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.239+0000 7fb4777fe640 1 -- 172.21.15.105:0/2775385078 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8723+0+0 (secure 0 0 0) 0x7fb4800f2070 con 0x7fb488102380 2024-03-26T18:04:42.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.239+0000 7fb485d9b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4700a2f20 0x7fb4700a53e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb4881990b0 tx=0x7fb47c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:42.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.241+0000 7fb4777fe640 1 -- 172.21.15.105:0/2775385078 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb48004d020 con 0x7fb488102380 2024-03-26T18:04:42.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a[32276]: 2024-03-26T18:04:42.102+0000 7f68febe4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T18:04:42.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: pgmap v1143: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-26T18:04:42.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2653102369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:04:42.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/3054656387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:42.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2071093262' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T18:04:42.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2071093262' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mon[32280]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a[73602]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a[73602]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mds[73606]: -- 172.21.15.105:0/2799073026 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7e0d47a40 con 0x55d7e0d1b000 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a[73602]: starting mds.a at 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g[73240]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.380 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g[73240]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.381 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-mds[73246]: -- 172.21.15.105:0/2217233246 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557e0333a40 con 0x5557e0306800 2024-03-26T18:04:42.381 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:04:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g[73240]: starting mds.g at 2024-03-26T18:04:42.402 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.402 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.403 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mds[65706]: -- 172.21.15.135:0/1871807025 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582e27bba40 con 0x5582e278ec00 2024-03-26T18:04:42.403 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: starting mds.b at 2024-03-26T18:04:42.403 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.403 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.403 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mds[65086]: -- 172.21.15.135:0/1811129820 <== mon.0 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563142131a40 con 0x563142104800 2024-03-26T18:04:42.403 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: starting mds.e at 2024-03-26T18:04:42.403 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k[64760]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k[64760]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mds[64764]: -- 172.21.15.135:0/3437220723 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560a4593a40 con 0x5560a4566800 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k[64760]: starting mds.k at 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: pgmap v1143: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2653102369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/3054656387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2071093262' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T18:04:42.404 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2071093262' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T18:04:42.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T18:04:42.405 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mon[35636]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T18:04:42.405 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h[65386]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.405 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h[65386]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.405 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-mds[65390]: -- 172.21.15.135:0/269929369 <== mon.2 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555cef249a40 con 0x555cef21d000 2024-03-26T18:04:42.405 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:04:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h[65386]: starting mds.h at 2024-03-26T18:04:42.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:42.468+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb48802d240 con 0x7fb488102380 2024-03-26T18:04:42.586 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i[67075]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.586 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i[67075]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.586 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i[67075]: starting mds.i at 2024-03-26T18:04:42.587 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c[66756]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.587 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c[66756]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.587 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mds[66760]: -- 172.21.15.144:0/2303808207 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ddf66b5a40 con 0x55ddf6688c00 2024-03-26T18:04:42.587 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c[66756]: starting mds.c at 2024-03-26T18:04:42.587 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-f[66445]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.587 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-f[66445]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.587 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mds[66449]: -- 172.21.15.144:0/753494030 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff471d1a40 con 0x55ff471a4800 2024-03-26T18:04:42.587 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-f[66445]: starting mds.f at 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l[66133]: ignoring --setuser ceph since I am not root 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l[66133]: ignoring --setgroup ceph since I am not root 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mds[66137]: -- 172.21.15.144:0/822735825 <== mon.1 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d6e5f1a40 con 0x563d6e5c4800 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l[66133]: starting mds.l at 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: pgmap v1143: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2653102369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/3054656387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2071093262' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T18:04:42.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2071093262' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T18:04:42.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T18:04:42.589 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:42 smithi144 ceph-mon[36095]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-26T18:04:43.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.105+0000 7fb4777fe640 1 -- 172.21.15.105:0/2775385078 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7fb4800becb0 con 0x7fb488102380 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4700a2f20 msgr2=0x7fb4700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4700a2f20 0x7fb4700a53e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb4881990b0 tx=0x7fb47c009680 comp rx=0 tx=0).stop 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 msgr2=0x7fb488197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 0x7fb488197ba0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fb480006010 tx=0x7fb4800042f0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 shutdown_connections 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fb4700a2f20 0x7fb4700a53e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4881075e0 0x7fb48819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb488104cb0 0x7fb4881980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 --2- 172.21.15.105:0/2775385078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb488102380 0x7fb488197ba0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 >> 172.21.15.105:0/2775385078 conn(0x7fb4880fbd50 msgr2=0x7fb488102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:43.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 shutdown_connections 2024-03-26T18:04:43.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.107+0000 7fb48ce0b640 1 -- 172.21.15.105:0/2775385078 wait complete. 2024-03-26T18:04:43.102 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-26T18:04:43.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.229+0000 7fd945560640 1 -- 172.21.15.105:0/1937592790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 msgr2=0x7fd940109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:43.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.229+0000 7fd945560640 1 --2- 172.21.15.105:0/1937592790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 0x7fd940109d30 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fd938009ca0 tx=0x7fd93802c0f0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.229+0000 7fd945560640 1 -- 172.21.15.105:0/1937592790 shutdown_connections 2024-03-26T18:04:43.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 --2- 172.21.15.105:0/1937592790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 0x7fd940109d30 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 --2- 172.21.15.105:0/1937592790 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd940102ae0 0x7fd940102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 --2- 172.21.15.105:0/1937592790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9401018f0 0x7fd940101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 -- 172.21.15.105:0/1937592790 >> 172.21.15.105:0/1937592790 conn(0x7fd9400fce60 msgr2=0x7fd9400ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 -- 172.21.15.105:0/1937592790 shutdown_connections 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 -- 172.21.15.105:0/1937592790 wait complete. 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 Processor -- start 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 -- start start 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9401018f0 0x7fd940197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd940102ae0 0x7fd940198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 0x7fd94019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd94010d420 con 0x7fd9401034a0 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd94010d2a0 con 0x7fd9401018f0 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd945560640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd94010d5a0 con 0x7fd940102ae0 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd93f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 0x7fd94019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:43.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd93f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 0x7fd94019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34576/0 (socket says 172.21.15.105:34576) 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd93f7fe640 1 -- 172.21.15.105:0/2370623075 learned_addr learned my addr 172.21.15.105:0/2370623075 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.230+0000 7fd93effd640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9401018f0 0x7fd940197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd937fff640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd940102ae0 0x7fd940198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93f7fe640 1 -- 172.21.15.105:0/2370623075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd940102ae0 msgr2=0x7fd940198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93f7fe640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd940102ae0 0x7fd940198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93f7fe640 1 -- 172.21.15.105:0/2370623075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9401018f0 msgr2=0x7fd940197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93f7fe640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9401018f0 0x7fd940197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93f7fe640 1 -- 172.21.15.105:0/2370623075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94019f710 con 0x7fd9401034a0 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd937fff640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd940102ae0 0x7fd940198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93f7fe640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 0x7fd94019f0a0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fd93802c700 tx=0x7fd9380328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93cff9640 1 -- 172.21.15.105:0/2370623075 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938041020 con 0x7fd9401034a0 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94019f9a0 con 0x7fd9401034a0 2024-03-26T18:04:43.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93cff9640 1 -- 172.21.15.105:0/2370623075 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd938003e20 con 0x7fd9401034a0 2024-03-26T18:04:43.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd93cff9640 1 -- 172.21.15.105:0/2370623075 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938039040 con 0x7fd9401034a0 2024-03-26T18:04:43.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.231+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94019fe90 con 0x7fd9401034a0 2024-03-26T18:04:43.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.232+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd940108cb0 con 0x7fd9401034a0 2024-03-26T18:04:43.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.233+0000 7fd93cff9640 1 -- 172.21.15.105:0/2370623075 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7fd938044020 con 0x7fd9401034a0 2024-03-26T18:04:43.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.234+0000 7fd93cff9640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd92c0a2f20 0x7fd92c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:43.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.234+0000 7fd93effd640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd92c0a2f20 0x7fd92c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:43.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.234+0000 7fd93cff9640 1 -- 172.21.15.105:0/2370623075 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8723+0+0 (secure 0 0 0) 0x7fd9380f19f0 con 0x7fd9401034a0 2024-03-26T18:04:43.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.234+0000 7fd93effd640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd92c0a2f20 0x7fd92c0a53e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd930005fd0 tx=0x7fd93000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:43.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.238+0000 7fd93cff9640 1 -- 172.21.15.105:0/2370623075 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd93804b020 con 0x7fd9401034a0 2024-03-26T18:04:43.379 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2775385078' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: Cluster is now healthy 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2775385078' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6835/1697792400,v1:172.21.15.135:6837/1697792400] up:boot 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6838/1981728757,v1:172.21.15.135:6839/1981728757] up:boot 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6836/2034359139,v1:172.21.15.144:6837/2034359139] up:boot 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6838/2328137496,v1:172.21.15.144:6839/2328137496] up:boot 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6832/1812589378,v1:172.21.15.135:6833/1812589378] up:boot 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6832/4291800819,v1:172.21.15.144:6834/4291800819] up:boot 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.144:6833/2856311251,v1:172.21.15.144:6835/2856311251] up:boot 2024-03-26T18:04:43.380 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.135:6834/2217704470,v1:172.21.15.135:6836/2217704470] up:boot 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.105:6839/4109662702,v1:172.21.15.105:6841/4109662702] up:boot 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: mds.? [v2:172.21.15.105:6838/1268932534,v1:172.21.15.105:6840/1268932534] up:boot 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: fsmap 12 up:standby 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:04:43.381 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:43 smithi105 ceph-mon[32280]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:04:43.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:43.419+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd9400624c0 con 0x7fd9401034a0 2024-03-26T18:04:43.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2775385078' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:04:43.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T18:04:43.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: Cluster is now healthy 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2775385078' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6835/1697792400,v1:172.21.15.135:6837/1697792400] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6838/1981728757,v1:172.21.15.135:6839/1981728757] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6836/2034359139,v1:172.21.15.144:6837/2034359139] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6838/2328137496,v1:172.21.15.144:6839/2328137496] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6832/1812589378,v1:172.21.15.135:6833/1812589378] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6832/4291800819,v1:172.21.15.144:6834/4291800819] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.144:6833/2856311251,v1:172.21.15.144:6835/2856311251] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.135:6834/2217704470,v1:172.21.15.135:6836/2217704470] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.105:6839/4109662702,v1:172.21.15.105:6841/4109662702] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: mds.? [v2:172.21.15.105:6838/1268932534,v1:172.21.15.105:6840/1268932534] up:boot 2024-03-26T18:04:43.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: fsmap 12 up:standby 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:04:43.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:43 smithi144 ceph-mon[36095]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:04:43.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2775385078' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T18:04:43.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T18:04:43.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T18:04:43.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: Cluster is now healthy 2024-03-26T18:04:43.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2775385078' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T18:04:43.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6835/1697792400,v1:172.21.15.135:6837/1697792400] up:boot 2024-03-26T18:04:43.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6838/1981728757,v1:172.21.15.135:6839/1981728757] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6836/2034359139,v1:172.21.15.144:6837/2034359139] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6838/2328137496,v1:172.21.15.144:6839/2328137496] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6832/1812589378,v1:172.21.15.135:6833/1812589378] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6832/4291800819,v1:172.21.15.144:6834/4291800819] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.144:6833/2856311251,v1:172.21.15.144:6835/2856311251] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.135:6834/2217704470,v1:172.21.15.135:6836/2217704470] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.105:6839/4109662702,v1:172.21.15.105:6841/4109662702] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: mds.? [v2:172.21.15.105:6838/1268932534,v1:172.21.15.105:6840/1268932534] up:boot 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: fsmap 12 up:standby 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T18:04:43.654 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T18:04:43.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T18:04:43.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T18:04:43.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T18:04:43.655 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:43 smithi135 ceph-mon[35636]: from='mgr.14152 172.21.15.105:0/4006762139' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T18:04:44.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.341+0000 7fd93cff9640 1 -- 172.21.15.105:0/2370623075 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7fd9380be630 con 0x7fd9401034a0 2024-03-26T18:04:44.321 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-03-26T18:04:44.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd92c0a2f20 msgr2=0x7fd92c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:44.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd92c0a2f20 0x7fd92c0a53e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd930005fd0 tx=0x7fd93000b390 comp rx=0 tx=0).stop 2024-03-26T18:04:44.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 msgr2=0x7fd94019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:44.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 0x7fd94019f0a0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fd93802c700 tx=0x7fd9380328a0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 shutdown_connections 2024-03-26T18:04:44.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7fd92c0a2f20 0x7fd92c0a53e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9401034a0 0x7fd94019f0a0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd940102ae0 0x7fd940198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 --2- 172.21.15.105:0/2370623075 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd9401018f0 0x7fd940197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 >> 172.21.15.105:0/2370623075 conn(0x7fd9400fce60 msgr2=0x7fd940105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:44.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 shutdown_connections 2024-03-26T18:04:44.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.344+0000 7fd945560640 1 -- 172.21.15.105:0/2370623075 wait complete. 2024-03-26T18:04:44.343 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 -- 172.21.15.105:0/367951274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c102360 msgr2=0x7f800c104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 --2- 172.21.15.105:0/367951274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c102360 0x7f800c104750 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f8000009900 tx=0x7f800002bc30 comp rx=0 tx=0).stop 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 -- 172.21.15.105:0/367951274 shutdown_connections 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 --2- 172.21.15.105:0/367951274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f800c1075c0 0x7f800c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 --2- 172.21.15.105:0/367951274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c104c90 0x7f800c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 --2- 172.21.15.105:0/367951274 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c102360 0x7f800c104750 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 -- 172.21.15.105:0/367951274 >> 172.21.15.105:0/367951274 conn(0x7f800c0fbd70 msgr2=0x7f800c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 -- 172.21.15.105:0/367951274 shutdown_connections 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.477+0000 7f8011cea640 1 -- 172.21.15.105:0/367951274 wait complete. 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f8011cea640 1 Processor -- start 2024-03-26T18:04:44.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f8011cea640 1 -- start start 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f8011cea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c102360 0x7f800c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f8011cea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f800c104c90 0x7f800c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f8011cea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c1075c0 0x7f800c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f8011cea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10d410 con 0x7f800c1075c0 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f8011cea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10d290 con 0x7f800c104c90 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f8011cea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10d590 con 0x7f800c102360 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c1075c0 0x7f800c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c1075c0 0x7f800c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34588/0 (socket says 172.21.15.105:34588) 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 -- 172.21.15.105:0/926923338 learned_addr learned my addr 172.21.15.105:0/926923338 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800b7fe640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c102360 0x7f800c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800affd640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f800c104c90 0x7f800c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 -- 172.21.15.105:0/926923338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c102360 msgr2=0x7f800c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:44.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c102360 0x7f800c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 -- 172.21.15.105:0/926923338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f800c104c90 msgr2=0x7f800c198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f800c104c90 0x7f800c198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 -- 172.21.15.105:0/926923338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c19f6e0 con 0x7f800c1075c0 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.478+0000 7f800bfff640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c1075c0 0x7f800c19f070 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f800400cce0 tx=0x7f800400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.479+0000 7f8008ff9640 1 -- 172.21.15.105:0/926923338 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800400f450 con 0x7f800c1075c0 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.479+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c19f9d0 con 0x7f800c1075c0 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.479+0000 7f8008ff9640 1 -- 172.21.15.105:0/926923338 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f800400faa0 con 0x7f800c1075c0 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.479+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c19fed0 con 0x7f800c1075c0 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.479+0000 7f8008ff9640 1 -- 172.21.15.105:0/926923338 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004018b10 con 0x7f800c1075c0 2024-03-26T18:04:44.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.480+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c0fd860 con 0x7f800c1075c0 2024-03-26T18:04:44.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.481+0000 7f8008ff9640 1 -- 172.21.15.105:0/926923338 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) ==== 137855+0+0 (secure 0 0 0) 0x7f8004018cb0 con 0x7f800c1075c0 2024-03-26T18:04:44.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.481+0000 7f8008ff9640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ff40a2f20 0x7f7ff40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T18:04:44.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.482+0000 7f800b7fe640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ff40a2f20 0x7f7ff40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T18:04:44.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.482+0000 7f8008ff9640 1 -- 172.21.15.105:0/926923338 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8190+0+0 (secure 0 0 0) 0x7f80040cf950 con 0x7f800c1075c0 2024-03-26T18:04:44.463 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.482+0000 7f800b7fe640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ff40a2f20 0x7f7ff40a53e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8000009230 tx=0x7f8000005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T18:04:44.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.485+0000 7f8008ff9640 1 -- 172.21.15.105:0/926923338 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8004014030 con 0x7f800c1075c0 2024-03-26T18:04:44.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:44 smithi144 ceph-mon[36095]: pgmap v1145: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-26T18:04:44.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:44 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2370623075' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:04:44.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:44 smithi105 ceph-mon[32280]: pgmap v1145: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-26T18:04:44.629 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:44 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2370623075' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:04:44.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:44.662+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f800c02d240 con 0x7f800c1075c0 2024-03-26T18:04:44.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:44 smithi135 ceph-mon[35636]: pgmap v1145: 97 pgs: 97 active+clean; 179 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-03-26T18:04:44.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:44 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2370623075' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:04:45.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.347+0000 7f8008ff9640 1 -- 172.21.15.105:0/926923338 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7f80040bae20 con 0x7f800c1075c0 2024-03-26T18:04:45.328 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-03-26T18:04:45.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ff40a2f20 msgr2=0x7f7ff40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:45.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ff40a2f20 0x7f7ff40a53e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8000009230 tx=0x7f8000005fd0 comp rx=0 tx=0).stop 2024-03-26T18:04:45.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c1075c0 msgr2=0x7f800c19f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T18:04:45.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c1075c0 0x7f800c19f070 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f800400cce0 tx=0x7f800400a620 comp rx=0 tx=0).stop 2024-03-26T18:04:45.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 shutdown_connections 2024-03-26T18:04:45.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:6800/2294330938,v1:172.21.15.105:6801/2294330938] conn(0x7f7ff40a2f20 0x7f7ff40a53e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:45.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f800c1075c0 0x7f800c19f070 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:45.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f800c104c90 0x7f800c198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:45.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 --2- 172.21.15.105:0/926923338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c102360 0x7f800c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T18:04:45.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 >> 172.21.15.105:0/926923338 conn(0x7f800c0fbd70 msgr2=0x7f800c102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T18:04:45.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 shutdown_connections 2024-03-26T18:04:45.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-26T18:04:45.349+0000 7f8011cea640 1 -- 172.21.15.105:0/926923338 wait complete. 2024-03-26T18:04:45.342 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:04:45.386 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:04:45.426 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:04:45.479 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-26T18:04:45.479 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T18:04:45.503 DEBUG:teuthology.orchestra.run.smithi135:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T18:04:45.517 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T18:04:45.531 INFO:tasks.cephadm:Stopping all daemons... 2024-03-26T18:04:45.531 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-26T18:04:45.531 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.a 2024-03-26T18:04:45.585 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/2370623075' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:04:45.585 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:45 smithi105 ceph-mon[32280]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T18:04:45.585 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:45 smithi105 ceph-mon[32280]: from='client.? 172.21.15.105:0/926923338' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:04:45.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/2370623075' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:04:45.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:45 smithi144 ceph-mon[36095]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T18:04:45.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:45 smithi144 ceph-mon[36095]: from='client.? 172.21.15.105:0/926923338' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:04:45.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/2370623075' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T18:04:45.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:45 smithi135 ceph-mon[35636]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T18:04:45.652 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:45 smithi135 ceph-mon[35636]: from='client.? 172.21.15.105:0/926923338' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T18:04:45.867 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:45 smithi105 systemd[1]: Stopping Ceph mon.a for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:04:46.129 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a[32276]: 2024-03-26T18:04:45.886+0000 7f69043ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:04:46.129 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a[32276]: 2024-03-26T18:04:45.886+0000 7f69043ef700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-26T18:04:46.129 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:45 smithi105 podman[124631]: 2024-03-26 18:04:45.930807688 +0000 UTC m=+0.165596176 container died beae2e9cfeec292fb059fbb4fa10f199eec28b6dffceee3623bb4902f0e34be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:04:46.602 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:46 smithi105 podman[124631]: 2024-03-26 18:04:46.313008128 +0000 UTC m=+0.547796617 container cleanup beae2e9cfeec292fb059fbb4fa10f199eec28b6dffceee3623bb4902f0e34be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T18:04:46.602 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:46 smithi105 bash[124631]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a 2024-03-26T18:04:46.879 INFO:journalctl@ceph.mon.a.smithi105.stdout:Mar 26 18:04:46 smithi105 podman[124648]: 2024-03-26 18:04:46.622915077 +0000 UTC m=+0.694780679 container remove beae2e9cfeec292fb059fbb4fa10f199eec28b6dffceee3623bb4902f0e34be2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-03-26T18:04:46.913 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.a.service' 2024-03-26T18:04:47.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:04:47.040 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-26T18:04:47.040 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-26T18:04:47.040 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.b 2024-03-26T18:04:47.360 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:47 smithi135 systemd[1]: Stopping Ceph mon.b for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:04:47.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-b[35632]: 2024-03-26T18:04:47.381+0000 7fe05ebd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:04:47.653 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-b[35632]: 2024-03-26T18:04:47.381+0000 7fe05ebd0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-26T18:04:47.910 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:47 smithi135 podman[74191]: 2024-03-26 18:04:47.676530412 +0000 UTC m=+0.390215519 container died efb123e3c39fbb4909b9fa9aafc65bf317d5d419f30f4059637de9c6e8f9c38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0) 2024-03-26T18:04:48.310 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:48 smithi135 podman[74191]: 2024-03-26 18:04:48.03049229 +0000 UTC m=+0.744177405 container cleanup efb123e3c39fbb4909b9fa9aafc65bf317d5d419f30f4059637de9c6e8f9c38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T18:04:48.310 INFO:journalctl@ceph.mon.b.smithi135.stdout:Mar 26 18:04:48 smithi135 bash[74191]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-b 2024-03-26T18:04:48.434 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.b.service' 2024-03-26T18:04:48.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:04:48.492 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-26T18:04:48.492 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-26T18:04:48.492 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.c 2024-03-26T18:04:48.836 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:48 smithi144 systemd[1]: Stopping Ceph mon.c for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:04:49.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-c[36078]: 2024-03-26T18:04:48.907+0000 7f1dda55c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:04:49.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-c[36078]: 2024-03-26T18:04:48.907+0000 7f1dda55c700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-26T18:04:49.336 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:49 smithi144 podman[75130]: 2024-03-26 18:04:49.092564222 +0000 UTC m=+0.321086510 container died e121d0b2a0cf4f5cf45b366975db0858524e1f9688df6536d56618e7a4f0be89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-26T18:04:49.730 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:49 smithi144 podman[75130]: 2024-03-26 18:04:49.516789494 +0000 UTC m=+0.745311765 container cleanup e121d0b2a0cf4f5cf45b366975db0858524e1f9688df6536d56618e7a4f0be89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-c, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T18:04:49.731 INFO:journalctl@ceph.mon.c.smithi144.stdout:Mar 26 18:04:49 smithi144 bash[75130]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mon-c 2024-03-26T18:04:49.865 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mon.c.service' 2024-03-26T18:04:49.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:04:49.912 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-26T18:04:49.912 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-26T18:04:49.912 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.x 2024-03-26T18:04:50.243 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 18:04:49 smithi105 systemd[1]: Stopping Ceph mgr.x for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:04:50.853 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 18:04:50 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x[32509]: 2024-03-26T18:04:50.549+0000 7f46047be700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:04:50.854 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 18:04:50 smithi105 podman[127478]: 2024-03-26 18:04:50.580939441 +0000 UTC m=+0.133619182 container died f6be31cfa6808843b95f1a08e92759009815bb7e6ea97e9c8d6bcd186dd1ff2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-26T18:04:51.129 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 18:04:50 smithi105 podman[127478]: 2024-03-26 18:04:50.975676824 +0000 UTC m=+0.528356569 container cleanup f6be31cfa6808843b95f1a08e92759009815bb7e6ea97e9c8d6bcd186dd1ff2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:360516069d9393362c4cc6eb9371680fe16d66ab, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:04:51.129 INFO:journalctl@ceph.mgr.x.smithi105.stdout:Mar 26 18:04:50 smithi105 bash[127478]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-x 2024-03-26T18:04:51.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-26T18:04:51.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:04:51.426 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-26T18:04:51.426 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-26T18:04:51.426 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.y 2024-03-26T18:04:51.742 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 18:04:51 smithi135 systemd[1]: Stopping Ceph mgr.y for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:04:52.152 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 18:04:51 smithi135 podman[74312]: 2024-03-26 18:04:51.763786064 +0000 UTC m=+0.124577871 container died ef9dc14d24170298a0dfcb18286fd453520adecbc1f4d251186d631801d7995f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:04:52.490 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 18:04:52 smithi135 podman[74312]: 2024-03-26 18:04:52.260029854 +0000 UTC m=+0.620821648 container cleanup ef9dc14d24170298a0dfcb18286fd453520adecbc1f4d251186d631801d7995f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-03-26T18:04:52.490 INFO:journalctl@ceph.mgr.y.smithi135.stdout:Mar 26 18:04:52 smithi135 bash[74312]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-y 2024-03-26T18:04:52.601 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-26T18:04:52.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:04:52.638 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-26T18:04:52.638 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-26T18:04:52.638 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.z 2024-03-26T18:04:52.954 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:04:52 smithi144 systemd[1]: Stopping Ceph mgr.z for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:04:53.234 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:04:52 smithi144 podman[75251]: 2024-03-26 18:04:52.975379585 +0000 UTC m=+0.124823141 container died 89c606a2afae007cb963516bb9c060a88b045f91ecd160f9cb0fd52c32a9f25e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T18:04:53.586 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:04:53 smithi144 podman[75251]: 2024-03-26 18:04:53.339258018 +0000 UTC m=+0.488701577 container cleanup 89c606a2afae007cb963516bb9c060a88b045f91ecd160f9cb0fd52c32a9f25e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T18:04:53.586 INFO:journalctl@ceph.mgr.z.smithi144.stdout:Mar 26 18:04:53 smithi144 bash[75251]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mgr-z 2024-03-26T18:04:53.730 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-26T18:04:53.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:04:53.767 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-26T18:04:53.767 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-26T18:04:53.768 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.0 2024-03-26T18:04:54.059 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:04:53 smithi105 systemd[1]: Stopping Ceph osd.0 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:04:54.379 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:04:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0[46771]: 2024-03-26T18:04:54.079+0000 7f416b2a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:04:54.379 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:04:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0[46771]: 2024-03-26T18:04:54.079+0000 7f416b2a3700 -1 osd.0 99 *** Got signal Terminated *** 2024-03-26T18:04:54.379 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:04:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0[46771]: 2024-03-26T18:04:54.079+0000 7f416b2a3700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:04:59.711 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:04:59 smithi105 podman[127599]: 2024-03-26 18:04:59.4593009 +0000 UTC m=+5.473419776 container died 1e6d9cd1e8dabfdda8c61d2dabb0f616b00b4d80d6f4d8b96fd9df6b08467ba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T18:05:00.102 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:04:59 smithi105 podman[127599]: 2024-03-26 18:04:59.846914264 +0000 UTC m=+5.861033142 container cleanup 1e6d9cd1e8dabfdda8c61d2dabb0f616b00b4d80d6f4d8b96fd9df6b08467ba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True) 2024-03-26T18:05:00.102 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:04:59 smithi105 bash[127599]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0 2024-03-26T18:05:00.379 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:05:00 smithi105 podman[127616]: 2024-03-26 18:05:00.123581855 +0000 UTC m=+0.657108059 container remove 1e6d9cd1e8dabfdda8c61d2dabb0f616b00b4d80d6f4d8b96fd9df6b08467ba3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:05:00.698 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:05:00 smithi105 podman[127709]: 2024-03-26 18:05:00.358078253 +0000 UTC m=+0.017467257 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:05:00.699 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:05:00 smithi105 podman[127709]: 2024-03-26 18:05:00.457825748 +0000 UTC m=+0.117214723 container create 1e5b5f0029878f47a385b3f2536a344fca300f09ec684207233ee19ca9ec2caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS) 2024-03-26T18:05:00.699 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:05:00 smithi105 podman[127709]: 2024-03-26 18:05:00.719294599 +0000 UTC m=+0.378683582 container init 1e5b5f0029878f47a385b3f2536a344fca300f09ec684207233ee19ca9ec2caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-26T18:05:01.027 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:05:00 smithi105 podman[127709]: 2024-03-26 18:05:00.723039258 +0000 UTC m=+0.382428233 container start 1e5b5f0029878f47a385b3f2536a344fca300f09ec684207233ee19ca9ec2caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-26T18:05:01.028 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:05:00 smithi105 podman[127709]: 2024-03-26 18:05:00.80224743 +0000 UTC m=+0.461636420 container attach 1e5b5f0029878f47a385b3f2536a344fca300f09ec684207233ee19ca9ec2caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-03-26T18:05:01.309 INFO:journalctl@ceph.osd.0.smithi105.stdout:Mar 26 18:05:01 smithi105 podman[127709]: 2024-03-26 18:05:01.048768302 +0000 UTC m=+0.708157297 container died 1e5b5f0029878f47a385b3f2536a344fca300f09ec684207233ee19ca9ec2caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-0-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-26T18:05:01.614 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.0.service' 2024-03-26T18:05:01.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:05:01.656 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-26T18:05:01.656 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-26T18:05:01.656 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.1 2024-03-26T18:05:01.960 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:01 smithi135 systemd[1]: Stopping Ceph osd.1 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:05:02.402 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:01 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1[40246]: 2024-03-26T18:05:01.979+0000 7f239318c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:05:02.402 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:01 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1[40246]: 2024-03-26T18:05:01.979+0000 7f239318c700 -1 osd.1 99 *** Got signal Terminated *** 2024-03-26T18:05:02.402 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:01 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1[40246]: 2024-03-26T18:05:01.979+0000 7f239318c700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:05:07.744 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:07 smithi135 podman[74438]: 2024-03-26 18:05:07.454332046 +0000 UTC m=+5.582380139 container died 2f44de1ee6840501d6a57d1d7006d2a703ce549671c660cc5ccbbb4b31908845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-26T18:05:08.068 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:07 smithi135 podman[74438]: 2024-03-26 18:05:07.871979553 +0000 UTC m=+6.000027646 container cleanup 2f44de1ee6840501d6a57d1d7006d2a703ce549671c660cc5ccbbb4b31908845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:05:08.069 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:07 smithi135 bash[74438]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1 2024-03-26T18:05:08.402 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:08 smithi135 podman[74455]: 2024-03-26 18:05:08.090108792 +0000 UTC m=+0.632800039 container remove 2f44de1ee6840501d6a57d1d7006d2a703ce549671c660cc5ccbbb4b31908845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T18:05:08.856 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:08 smithi135 podman[74546]: 2024-03-26 18:05:08.416297831 +0000 UTC m=+0.018330574 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:05:08.856 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:08 smithi135 podman[74546]: 2024-03-26 18:05:08.524163024 +0000 UTC m=+0.126195756 container create e44633a8b10ca67be6a87c8c76dcc52ddc8388abe2496969f52f4f8da91d325f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:05:08.856 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:08 smithi135 podman[74546]: 2024-03-26 18:05:08.726685963 +0000 UTC m=+0.328718691 container init e44633a8b10ca67be6a87c8c76dcc52ddc8388abe2496969f52f4f8da91d325f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:05:08.856 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:08 smithi135 podman[74546]: 2024-03-26 18:05:08.72984476 +0000 UTC m=+0.331877490 container start e44633a8b10ca67be6a87c8c76dcc52ddc8388abe2496969f52f4f8da91d325f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T18:05:08.856 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:08 smithi135 podman[74546]: 2024-03-26 18:05:08.876457544 +0000 UTC m=+0.478490273 container attach e44633a8b10ca67be6a87c8c76dcc52ddc8388abe2496969f52f4f8da91d325f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:05:10.583 INFO:journalctl@ceph.osd.1.smithi135.stdout:Mar 26 18:05:10 smithi135 podman[74546]: 2024-03-26 18:05:10.286148353 +0000 UTC m=+1.888181084 container died e44633a8b10ca67be6a87c8c76dcc52ddc8388abe2496969f52f4f8da91d325f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T18:05:10.906 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.1.service' 2024-03-26T18:05:10.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:05:10.947 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-26T18:05:10.947 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-26T18:05:10.948 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.2 2024-03-26T18:05:11.247 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:11 smithi144 systemd[1]: Stopping Ceph osd.2 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:05:11.586 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2[41252]: 2024-03-26T18:05:11.267+0000 7f87b7a67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:05:11.586 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2[41252]: 2024-03-26T18:05:11.267+0000 7f87b7a67700 -1 osd.2 99 *** Got signal Terminated *** 2024-03-26T18:05:11.586 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2[41252]: 2024-03-26T18:05:11.267+0000 7f87b7a67700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:05:17.113 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:16 smithi144 podman[75375]: 2024-03-26 18:05:16.873098642 +0000 UTC m=+5.715970216 container died 23b2844412299baffed759c70bea2bf0e8aa5c3526174488bdb078fc0547a377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T18:05:17.469 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:17 smithi144 podman[75375]: 2024-03-26 18:05:17.242309921 +0000 UTC m=+6.085181494 container cleanup 23b2844412299baffed759c70bea2bf0e8aa5c3526174488bdb078fc0547a377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T18:05:17.469 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:17 smithi144 bash[75375]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2 2024-03-26T18:05:17.826 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:17 smithi144 podman[75393]: 2024-03-26 18:05:17.519759597 +0000 UTC m=+0.647188371 container remove 23b2844412299baffed759c70bea2bf0e8aa5c3526174488bdb078fc0547a377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T18:05:18.076 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:17 smithi144 podman[75490]: 2024-03-26 18:05:17.75003396 +0000 UTC m=+0.017575154 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:05:18.076 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:17 smithi144 podman[75490]: 2024-03-26 18:05:17.87041503 +0000 UTC m=+0.137956214 container create 96791637092b888e8cdd7e917c3883470fcead7dc670389ab5ead83b0a1fd226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-03-26T18:05:18.077 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:18 smithi144 podman[75490]: 2024-03-26 18:05:18.097870944 +0000 UTC m=+0.365412139 container init 96791637092b888e8cdd7e917c3883470fcead7dc670389ab5ead83b0a1fd226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T18:05:18.336 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:18 smithi144 podman[75490]: 2024-03-26 18:05:18.101201502 +0000 UTC m=+0.368742691 container start 96791637092b888e8cdd7e917c3883470fcead7dc670389ab5ead83b0a1fd226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T18:05:18.336 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:18 smithi144 podman[75490]: 2024-03-26 18:05:18.181203779 +0000 UTC m=+0.448744975 container attach 96791637092b888e8cdd7e917c3883470fcead7dc670389ab5ead83b0a1fd226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T18:05:18.771 INFO:journalctl@ceph.osd.2.smithi144.stdout:Mar 26 18:05:18 smithi144 podman[75490]: 2024-03-26 18:05:18.451579501 +0000 UTC m=+0.719120693 container died 96791637092b888e8cdd7e917c3883470fcead7dc670389ab5ead83b0a1fd226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T18:05:19.074 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.2.service' 2024-03-26T18:05:19.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:05:19.118 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-26T18:05:19.118 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-26T18:05:19.119 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.3 2024-03-26T18:05:19.437 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:19 smithi105 systemd[1]: Stopping Ceph osd.3 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:05:19.704 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:19 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T18:05:19.458+0000 7fd9fda21700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:05:19.704 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:19 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T18:05:19.458+0000 7fd9fda21700 -1 osd.3 99 *** Got signal Terminated *** 2024-03-26T18:05:19.704 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:19 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T18:05:19.458+0000 7fd9fda21700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:05:20.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:20 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:20.007+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:21.266 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:20 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:20.990+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:21.266 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:21.191+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:21.629 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:21 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T18:05:21.287+0000 7fd9fa015700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:56.674136+0000 front 2024-03-26T18:04:56.674169+0000 (oldest deadline 2024-03-26T18:05:20.774263+0000) 2024-03-26T18:05:22.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:21 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:21.815+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:22.307 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:22 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:22.193+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:22.308 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:22 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:22.017+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:22.629 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:22 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T18:05:22.328+0000 7fd9fa015700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:56.674136+0000 front 2024-03-26T18:04:56.674169+0000 (oldest deadline 2024-03-26T18:05:20.774263+0000) 2024-03-26T18:05:22.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:22 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:22.541+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:23.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:22.855+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:23.326 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:23 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:23.029+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:23.326 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:23 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:23.226+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:23.629 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:23 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T18:05:23.346+0000 7fd9fa015700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:56.674136+0000 front 2024-03-26T18:04:56.674169+0000 (oldest deadline 2024-03-26T18:05:20.774263+0000) 2024-03-26T18:05:23.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:23 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:23.548+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:24.308 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:24 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:24.194+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:24.308 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:24 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:24.020+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:24.335 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:23 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:23.875+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:24.629 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:24 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3[53439]: 2024-03-26T18:05:24.329+0000 7fd9fa015700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:56.674136+0000 front 2024-03-26T18:04:56.674169+0000 (oldest deadline 2024-03-26T18:05:20.774263+0000) 2024-03-26T18:05:24.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:24 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:24.570+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:25.122 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:24 smithi105 podman[127814]: 2024-03-26 18:05:24.860436224 +0000 UTC m=+5.532151555 container died 4f5aeb51041523ba9d444aeda4d11b9fa84e15a6524f71cdf73d7e6313b65751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-26T18:05:25.122 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:25 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:25.019+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:25.335 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:24 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:24.886+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:25.378 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:25 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:25.180+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:25.379 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:25 smithi105 podman[127814]: 2024-03-26 18:05:25.233319279 +0000 UTC m=+5.905034608 container cleanup 4f5aeb51041523ba9d444aeda4d11b9fa84e15a6524f71cdf73d7e6313b65751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-26T18:05:25.379 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:25 smithi105 bash[127814]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3 2024-03-26T18:05:25.792 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:25 smithi105 podman[127843]: 2024-03-26 18:05:25.509642165 +0000 UTC m=+0.644586701 container remove 4f5aeb51041523ba9d444aeda4d11b9fa84e15a6524f71cdf73d7e6313b65751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:05:25.818 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:25.370+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:25.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:25 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:25.557+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:26.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:25.839+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:26.100 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:26 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:26.005+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:26.100 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:25 smithi105 podman[127934]: 2024-03-26 18:05:25.716652305 +0000 UTC m=+0.017936266 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:05:26.100 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:25 smithi105 podman[127934]: 2024-03-26 18:05:25.836604269 +0000 UTC m=+0.137888263 container create 0b71ee02b1c9fcf8e9a62358017ac6f7282612b92faeba43df32b4634fa3eb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-26T18:05:26.100 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:26 smithi105 podman[127934]: 2024-03-26 18:05:26.052007183 +0000 UTC m=+0.353291133 container init 0b71ee02b1c9fcf8e9a62358017ac6f7282612b92faeba43df32b4634fa3eb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:05:26.100 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:26 smithi105 podman[127934]: 2024-03-26 18:05:26.055737124 +0000 UTC m=+0.357021077 container start 0b71ee02b1c9fcf8e9a62358017ac6f7282612b92faeba43df32b4634fa3eb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T18:05:26.100 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:26 smithi105 podman[127934]: 2024-03-26 18:05:26.121457586 +0000 UTC m=+0.422741559 container attach 0b71ee02b1c9fcf8e9a62358017ac6f7282612b92faeba43df32b4634fa3eb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T18:05:26.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:26 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:26.211+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:26.379 INFO:journalctl@ceph.osd.3.smithi105.stdout:Mar 26 18:05:26 smithi105 podman[127934]: 2024-03-26 18:05:26.347334108 +0000 UTC m=+0.648618060 container died 0b71ee02b1c9fcf8e9a62358017ac6f7282612b92faeba43df32b4634fa3eb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-26T18:05:26.800 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:26.415+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:26.894 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.3.service' 2024-03-26T18:05:26.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:26 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:26.592+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:26.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:05:26.936 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-26T18:05:26.937 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-26T18:05:26.937 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.4 2024-03-26T18:05:27.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:26.821+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:27.128 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:27 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:27.028+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:27.245 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:27 smithi135 systemd[1]: Stopping Ceph osd.4 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:05:27.558 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:27 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4[46915]: 2024-03-26T18:05:27.265+0000 7f312a462700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:05:27.558 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:27 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4[46915]: 2024-03-26T18:05:27.265+0000 7f312a462700 -1 osd.4 99 *** Got signal Terminated *** 2024-03-26T18:05:27.558 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:27 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4[46915]: 2024-03-26T18:05:27.265+0000 7f312a462700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:05:27.629 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:27 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:27.169+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:27.779 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:27.391+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:27.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:27 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:27.579+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:28.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:27.851+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:28.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:28 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:28.123+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:28.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:28 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:28.033+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:28.659 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:28 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:28.388+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:28.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:28 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:28.542+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:28.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:28 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:28.542+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:29.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:28 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:28.680+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:29.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:28 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:28.861+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:29.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:29 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:29.163+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:29.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:29 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:29.019+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:29.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:29 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:29.019+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:29.642 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:29 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:29.367+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:29.642 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:29 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:29.367+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:29.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:29 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:29.494+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:29.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:29 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:29.494+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:30.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:29 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:29.663+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:30.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:29 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:29.887+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:30.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:29 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:29.887+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:30.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:30 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:30.017+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:30.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:30 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:30.017+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:30.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:30 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:30.120+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:30.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:30 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:30.120+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:30.835 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:30 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:30.617+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:30.836 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:30 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:30.374+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:30.836 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:30 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:30.374+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:30.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:30 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:30.473+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:30.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:30 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:30.490+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:30.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:30 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:30.490+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:31.321 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:30 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:30.884+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:31.321 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:30 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:30.884+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:31.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:31 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:31.102+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:31.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:31 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:31.102+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:31.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:31 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:31.008+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:31.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:31 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:31.008+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:31.586 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:31 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:31.342+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:31.586 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:31 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:31.342+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:31.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:31 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:31.454+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:31.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:31 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:31.540+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:31.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:31 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:31.540+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:32.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:31 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:31.619+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:32.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:31 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:31.861+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:32.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:31 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:31.861+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:32.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:32 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:32.132+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:32.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:32 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:32.132+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:32.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:32 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:32.057+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:32.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:32 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:32.057+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:32.402 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:31 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4[46915]: 2024-03-26T18:05:31.948+0000 7f3122a4e700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.031354+0000 front 2024-03-26T18:05:06.031417+0000 (oldest deadline 2024-03-26T18:05:31.331477+0000) 2024-03-26T18:05:32.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:32 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:32.381+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:32.614 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:32 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:32.381+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:32.720 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:32 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:32.430+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:32.720 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:32 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:32.559+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:32.721 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:32 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:32.559+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:33.035 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:32 smithi135 podman[74664]: 2024-03-26 18:05:32.742090573 +0000 UTC m=+5.590314407 container died 7694c0d225dd546aae88ca56debbce87668b6d050934d9c7237e52a664d00923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-26T18:05:33.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:32 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:32.635+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:33.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:32 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:32.816+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:33.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:32 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:32.816+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:33.328 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:33 smithi135 podman[74664]: 2024-03-26 18:05:33.157179194 +0000 UTC m=+6.005403038 container cleanup 7694c0d225dd546aae88ca56debbce87668b6d050934d9c7237e52a664d00923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-26T18:05:33.329 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:33 smithi135 bash[74664]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4 2024-03-26T18:05:33.378 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:33 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:33.072+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:33.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:33 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:33.072+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:33.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:33 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:33.104+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:33.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:33 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:33.104+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:33.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:33 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:33.350+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:33.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:33 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:33.350+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:33.652 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:33 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:33.380+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:33.652 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:33 smithi135 podman[74681]: 2024-03-26 18:05:33.349970273 +0000 UTC m=+0.596039384 container remove 7694c0d225dd546aae88ca56debbce87668b6d050934d9c7237e52a664d00923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T18:05:33.653 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:33 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:33.586+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:33.653 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:33 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:33.586+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:34.049 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:33 smithi135 podman[74775]: 2024-03-26 18:05:33.648481274 +0000 UTC m=+0.017791835 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:05:34.049 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:33 smithi135 podman[74775]: 2024-03-26 18:05:33.750968813 +0000 UTC m=+0.120279376 container create b8f99b6c9f3723f3ef17b9d2e21bbc0027d074518aa7c207f4a46270abab518c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:05:34.049 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:33 smithi135 podman[74775]: 2024-03-26 18:05:33.98685167 +0000 UTC m=+0.356162236 container init b8f99b6c9f3723f3ef17b9d2e21bbc0027d074518aa7c207f4a46270abab518c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:05:34.049 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:33 smithi135 podman[74775]: 2024-03-26 18:05:33.990002125 +0000 UTC m=+0.359312692 container start b8f99b6c9f3723f3ef17b9d2e21bbc0027d074518aa7c207f4a46270abab518c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:05:34.050 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:34 smithi135 podman[74775]: 2024-03-26 18:05:34.069975206 +0000 UTC m=+0.439285770 container attach b8f99b6c9f3723f3ef17b9d2e21bbc0027d074518aa7c207f4a46270abab518c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-03-26T18:05:34.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:33 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:33.625+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:34.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:33 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:33.839+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:34.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:33 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:33.839+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:34.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:34 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:34.035+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:34.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:34 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:34.035+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:34.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:34 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:34.103+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:34.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:34 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:34.103+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:34.392 INFO:journalctl@ceph.osd.4.smithi135.stdout:Mar 26 18:05:34 smithi135 podman[74775]: 2024-03-26 18:05:34.279717543 +0000 UTC m=+0.649028100 container died b8f99b6c9f3723f3ef17b9d2e21bbc0027d074518aa7c207f4a46270abab518c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-03-26T18:05:34.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:34 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:34.323+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:34.586 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:34 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:34.323+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:34.652 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:34 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:34.414+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:34.652 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:34 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:34.585+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:34.652 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:34 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:34.585+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:34.772 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.4.service' 2024-03-26T18:05:34.813 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:05:34.813 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-26T18:05:34.813 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-26T18:05:34.813 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.5 2024-03-26T18:05:35.086 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:34 smithi144 systemd[1]: Stopping Ceph osd.5 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:05:35.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:34 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:34.647+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:35.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:34 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:34.858+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:35.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:34 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:34.858+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:35.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:35 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:35.124+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:35.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:35 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:35.124+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:35.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:35 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:35.037+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:35.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:35 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:35.037+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:35.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:35.145+0000 7fd227ff9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:05:35.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:35.145+0000 7fd227ff9700 -1 osd.5 99 *** Got signal Terminated *** 2024-03-26T18:05:35.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:35.145+0000 7fd227ff9700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:05:35.586 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:35.333+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:35.586 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:35.333+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:35.881 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:35.634+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:35.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:35 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:35.455+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:35.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:35 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:35.583+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:35.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:35 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:35.583+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:36.298 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:35.902+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:36.298 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:35 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:35.902+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:36.378 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:36 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:36.100+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:36.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:36 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:36.100+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:36.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:36 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:36.003+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:36.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:36 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:36.003+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:36.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:36.319+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:36.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:36.319+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:36.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:36 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:36.467+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:36.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:36 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:36.585+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:36.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:36 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:36.585+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:37.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:36.649+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:37.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:36.889+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:37.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:36 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:36.889+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:37.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:37 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:37.031+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:37.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:37 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:37.031+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:37.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:37 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:37.054+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:37.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:37 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:37.054+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:37.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:37 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:37.352+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:37.586 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:37 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:37.352+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:37.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:37 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:37.516+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:37.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:37 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:37.590+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:37.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:37 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:37.590+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:38.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:37 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:37.692+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:38.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:37 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:37.913+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:38.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:37 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:37.913+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:38.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:38 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:38.021+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:38.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:38 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:38.021+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:38.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:38 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:38.045+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:38.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:38 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:38.045+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:38.681 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:38.403+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:38.681 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:38.403+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:38.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:38.515+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:38.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:38.565+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:38.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:38 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:38.565+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:39.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:38.702+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:39.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:38.909+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:39.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:38 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:38.909+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:39.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:39 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:39.019+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:39.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:39 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:39.019+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:39.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:39 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:39.006+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:39.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:39 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:39.006+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:39.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:39 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:39.006+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:39.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:39 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:39.380+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:59.046677+0000 front 2024-03-26T18:04:59.046692+0000 (oldest deadline 2024-03-26T18:05:24.346784+0000) 2024-03-26T18:05:39.701 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:39 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5[48162]: 2024-03-26T18:05:39.380+0000 7fd2205e5700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:04.347447+0000 front 2024-03-26T18:05:04.347366+0000 (oldest deadline 2024-03-26T18:05:29.047506+0000) 2024-03-26T18:05:39.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:39 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:39.549+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:39.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:39 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:39.549+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:39.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:39 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:39.612+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:39.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:39 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:39.612+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:39.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:39 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:39.612+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:40.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:39 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:39.722+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:40.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:39 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:39.909+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:40.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:39 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:39.909+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:40.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:40.044+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:40.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:40.044+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:40.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:40.011+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:40.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:40.011+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:40.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:40.011+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:40.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:40.591+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:40.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:40.591+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:40.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:40.632+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:40.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:40.632+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:40.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:40 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:40.632+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:40.926 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:40 smithi144 podman[75599]: 2024-03-26 18:05:40.662572679 +0000 UTC m=+5.636166801 container died 92990fba8e8759429b56044bdd48688a65c463eb6147b05ab4254254517fd7e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T18:05:40.926 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:40 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:40.715+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:40.926 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:40 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:40.715+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:40.926 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:40 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:40.876+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:40.926 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:40 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:40.876+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:41.291 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:41 smithi144 podman[75599]: 2024-03-26 18:05:41.045188193 +0000 UTC m=+6.018782316 container cleanup 92990fba8e8759429b56044bdd48688a65c463eb6147b05ab4254254517fd7e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T18:05:41.291 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:41 smithi144 bash[75599]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5 2024-03-26T18:05:41.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:41.022+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:41.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:41.022+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:41.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:41.032+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:41.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:41.032+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:41.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:41 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:41.032+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:41.585 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:41 smithi144 podman[75616]: 2024-03-26 18:05:41.313088874 +0000 UTC m=+0.649245627 container remove 92990fba8e8759429b56044bdd48688a65c463eb6147b05ab4254254517fd7e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T18:05:41.823 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:41.546+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:41.823 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:41.546+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:41.824 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:41.594+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:41.824 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:41.594+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:41.824 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:41 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:41.594+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:41.895 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:41 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:41.866+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:41.895 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:41 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:41.866+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:41.895 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:41 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:41.866+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:41.896 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:41 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:41.761+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:41.896 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:41 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:41.761+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:41.896 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:41 smithi144 podman[75707]: 2024-03-26 18:05:41.561472631 +0000 UTC m=+0.017892316 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:05:41.896 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:41 smithi144 podman[75707]: 2024-03-26 18:05:41.697449422 +0000 UTC m=+0.153869100 container create f20c084be6cc93e10cae2ca7d42e58f00f21bcf722430d8a1dcdd22d7ea7dae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T18:05:41.896 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:41 smithi144 podman[75707]: 2024-03-26 18:05:41.916411913 +0000 UTC m=+0.372831592 container init f20c084be6cc93e10cae2ca7d42e58f00f21bcf722430d8a1dcdd22d7ea7dae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-26T18:05:42.184 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:41 smithi144 podman[75707]: 2024-03-26 18:05:41.920074875 +0000 UTC m=+0.376494565 container start f20c084be6cc93e10cae2ca7d42e58f00f21bcf722430d8a1dcdd22d7ea7dae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T18:05:42.185 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:41 smithi144 podman[75707]: 2024-03-26 18:05:41.999767625 +0000 UTC m=+0.456187304 container attach f20c084be6cc93e10cae2ca7d42e58f00f21bcf722430d8a1dcdd22d7ea7dae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-26T18:05:42.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:42.009+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:42.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:42.009+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:42.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:42.009+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:42.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:42.021+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:42.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:42.021+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:42.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:42.021+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.973693+0000 front 2024-03-26T18:05:15.973845+0000 (oldest deadline 2024-03-26T18:05:41.873815+0000) 2024-03-26T18:05:42.481 INFO:journalctl@ceph.osd.5.smithi144.stdout:Mar 26 18:05:42 smithi144 podman[75707]: 2024-03-26 18:05:42.206256264 +0000 UTC m=+0.662675950 container died f20c084be6cc93e10cae2ca7d42e58f00f21bcf722430d8a1dcdd22d7ea7dae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-26T18:05:42.765 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.5.service' 2024-03-26T18:05:42.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:05:42.808 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-26T18:05:42.808 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-26T18:05:42.808 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.6 2024-03-26T18:05:42.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:42.504+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:42.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:42.504+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:42.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:42.618+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:42.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:42.618+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:42.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:42 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:42.618+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:43.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:42.870+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:43.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:42.870+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:43.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:42.870+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:43.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:42.763+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:43.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:42 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:42.763+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:43.122 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:42.979+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:43.122 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:42.979+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:43.123 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:42 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:42.979+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:43.123 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:42 smithi105 systemd[1]: Stopping Ceph osd.6 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:05:43.123 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:43.054+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:43.123 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:43.054+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:43.123 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:43.054+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.973693+0000 front 2024-03-26T18:05:15.973845+0000 (oldest deadline 2024-03-26T18:05:41.873815+0000) 2024-03-26T18:05:43.123 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:43.142+0000 7fe3305dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:05:43.123 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:43.142+0000 7fe3305dc700 -1 osd.6 99 *** Got signal Terminated *** 2024-03-26T18:05:43.123 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:43.142+0000 7fe3305dc700 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:05:43.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:43 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:43.468+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:43.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:43 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:43.468+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:43.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:43 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:43.662+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:43.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:43 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:43.662+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:43.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:43 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:43.663+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:44.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:43 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:43.743+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:44.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:43 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:43.743+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:44.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:43 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:43.822+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:44.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:43 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:43.822+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:44.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:43 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:43.822+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:44.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:44.053+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:44.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:44.053+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:44.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:44.053+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.973693+0000 front 2024-03-26T18:05:15.973845+0000 (oldest deadline 2024-03-26T18:05:41.873815+0000) 2024-03-26T18:05:44.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:44.023+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:44.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:44.023+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:44.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:44 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:44.023+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:44.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:44 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:44.496+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:44.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:44 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:44.496+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:44.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:44 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:44.621+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:44.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:44 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:44.621+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:44.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:44 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:44.621+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:45.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:44 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:44.741+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:45.087 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:44 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:44.741+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:45.087 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:44 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:44.780+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:45.088 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:44 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:44.780+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:45.088 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:44 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:44.780+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:45.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:45.062+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:45.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:45.062+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:45.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:45.062+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:45.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:45.102+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:45.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:45.102+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:45.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:45 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:45.102+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.973693+0000 front 2024-03-26T18:05:15.973845+0000 (oldest deadline 2024-03-26T18:05:41.873815+0000) 2024-03-26T18:05:45.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:45 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:45.469+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:45.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:45 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:45.469+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:45.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:45 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:45.664+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:45.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:45 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:45.664+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:45.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:45 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:45.664+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:46.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:45 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:45.710+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:46.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:45 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:45.710+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:46.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:45 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:45.793+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:46.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:45 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:45.793+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:46.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:45 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:45.793+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:46.378 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:46.102+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:46.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:46.102+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:46.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:46.102+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:46.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:46.131+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:46.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:46.131+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:46.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:46.131+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.973693+0000 front 2024-03-26T18:05:15.973845+0000 (oldest deadline 2024-03-26T18:05:41.873815+0000) 2024-03-26T18:05:46.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:46 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:46.131+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:21.874220+0000 front 2024-03-26T18:05:21.874185+0000 (oldest deadline 2024-03-26T18:05:45.374367+0000) 2024-03-26T18:05:46.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:46 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:46.493+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:46.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:46 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:46.493+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:46.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:46 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:46.616+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:46.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:46 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:46.616+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:46.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:46 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:46.616+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:46.987 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:46 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:46.707+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:46.987 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:46 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:46.707+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:46.987 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:46 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:46.791+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:46.987 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:46 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:46.791+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:46.987 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:46 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:46.791+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:47.378 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:47 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:47.057+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:47.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:47 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:47.057+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:47.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:47 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:47.057+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:47.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:47 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:47.092+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:47.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:47 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:47.092+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:47.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:47 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:47.092+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.973693+0000 front 2024-03-26T18:05:15.973845+0000 (oldest deadline 2024-03-26T18:05:41.873815+0000) 2024-03-26T18:05:47.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:47 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:47.092+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:21.874220+0000 front 2024-03-26T18:05:21.874185+0000 (oldest deadline 2024-03-26T18:05:45.374367+0000) 2024-03-26T18:05:47.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:47.465+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:47.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:47.465+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:47.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:47.661+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:47.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:47.661+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:47.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:47 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:47.661+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:48.003 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:47 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:47.704+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:48.003 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:47 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:47.704+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:48.003 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:47 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:47.745+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:48.004 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:47 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:47.745+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:48.004 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:47 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:47.745+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:48.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:48 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:48.123+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.970331+0000 front 2024-03-26T18:04:57.970375+0000 (oldest deadline 2024-03-26T18:05:19.670674+0000) 2024-03-26T18:05:48.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:48 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:48.123+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:03.771853+0000 front 2024-03-26T18:05:03.771811+0000 (oldest deadline 2024-03-26T18:05:28.471636+0000) 2024-03-26T18:05:48.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:48 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:48.123+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.973693+0000 front 2024-03-26T18:05:15.973845+0000 (oldest deadline 2024-03-26T18:05:41.873815+0000) 2024-03-26T18:05:48.379 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:48 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6[59988]: 2024-03-26T18:05:48.123+0000 7fe32cbd0700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:21.874220+0000 front 2024-03-26T18:05:21.874185+0000 (oldest deadline 2024-03-26T18:05:45.374367+0000) 2024-03-26T18:05:48.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:48 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:48.040+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:48.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:48 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:48.040+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:48.380 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:48 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:48.040+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:48.380 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:48 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:48.040+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:48.899 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:48 smithi105 podman[128042]: 2024-03-26 18:05:48.668412704 +0000 UTC m=+5.644257279 container died f05eaf2111126e9ae391da21755fd8d2a452d36755b3143f9b7ab4aa140a1585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:05:48.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:48 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:48.494+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:48.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:48 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:48.494+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:48.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:48 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:48.613+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:48.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:48 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:48.613+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:48.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:48 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:48.613+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:48.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:48 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:48.613+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:49.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:48.743+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:49.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:48.743+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:49.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:48.769+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:49.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:48.769+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:49.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:48 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:48.769+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:49.182 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:49 smithi105 podman[128042]: 2024-03-26 18:05:49.010734065 +0000 UTC m=+5.986578639 container cleanup f05eaf2111126e9ae391da21755fd8d2a452d36755b3143f9b7ab4aa140a1585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T18:05:49.182 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:49 smithi105 bash[128042]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6 2024-03-26T18:05:49.182 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:49 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:49.022+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:49.182 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:49 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:49.022+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:49.182 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:49 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:49.022+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:49.183 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:49 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:49.022+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:49.592 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:49 smithi105 podman[128059]: 2024-03-26 18:05:49.203897396 +0000 UTC m=+0.535555543 container remove f05eaf2111126e9ae391da21755fd8d2a452d36755b3143f9b7ab4aa140a1585 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T18:05:49.849 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:49 smithi105 podman[128153]: 2024-03-26 18:05:49.517473234 +0000 UTC m=+0.017657842 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:05:49.850 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:49 smithi105 podman[128153]: 2024-03-26 18:05:49.644139829 +0000 UTC m=+0.144324454 container create 3f13884a747bf5fa6831d5302fc9c36addf13b48d74e5b1b8b27463688d22306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-26T18:05:49.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:49 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:49.494+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:49.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:49 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:49.494+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:49.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:49 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:49.494+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.945445+0000 front 2024-03-26T18:05:22.945318+0000 (oldest deadline 2024-03-26T18:05:48.845211+0000) 2024-03-26T18:05:49.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:49 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:49.656+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:49.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:49 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:49.656+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:49.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:49 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:49.656+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:49.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:49 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:49.656+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:50.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:49 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:49.698+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:50.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:49 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:49.698+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:50.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:49 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:49.698+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:50.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:49 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:49.720+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:50.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:49 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:49.720+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:50.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:49 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:49.720+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:50.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:49 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:49.972+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:50.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:49 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:49.972+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:50.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:49 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:49.972+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:50.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:49 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:49.972+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:50.129 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:49 smithi105 podman[128153]: 2024-03-26 18:05:49.871365635 +0000 UTC m=+0.371550219 container init 3f13884a747bf5fa6831d5302fc9c36addf13b48d74e5b1b8b27463688d22306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-26T18:05:50.129 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:49 smithi105 podman[128153]: 2024-03-26 18:05:49.87495544 +0000 UTC m=+0.375140026 container start 3f13884a747bf5fa6831d5302fc9c36addf13b48d74e5b1b8b27463688d22306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, CEPH_POINT_RELEASE=) 2024-03-26T18:05:50.129 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:49 smithi105 podman[128153]: 2024-03-26 18:05:49.946382854 +0000 UTC m=+0.446567438 container attach 3f13884a747bf5fa6831d5302fc9c36addf13b48d74e5b1b8b27463688d22306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-26T18:05:50.416 INFO:journalctl@ceph.osd.6.smithi105.stdout:Mar 26 18:05:50 smithi105 podman[128153]: 2024-03-26 18:05:50.164742205 +0000 UTC m=+0.664926801 container died 3f13884a747bf5fa6831d5302fc9c36addf13b48d74e5b1b8b27463688d22306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:05:50.732 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.6.service' 2024-03-26T18:05:50.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:05:50.774 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-26T18:05:50.774 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-26T18:05:50.774 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.7 2024-03-26T18:05:50.804 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:50.690+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:50.805 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:50.690+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:50.805 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:50.690+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:50.805 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:50.690+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:50.805 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:50.526+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:50.805 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:50.526+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:50.805 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:50.526+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.945445+0000 front 2024-03-26T18:05:22.945318+0000 (oldest deadline 2024-03-26T18:05:48.845211+0000) 2024-03-26T18:05:51.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:50 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:50.757+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:51.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:50 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:50.757+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:51.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:50 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:50.757+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:51.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:50 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:50.735+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:51.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:50 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:50.735+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:51.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:50 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:50.735+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:51.096 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:50 smithi135 systemd[1]: Stopping Ceph osd.7 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:05:51.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:50 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:50.974+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:51.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:50 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:50.974+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:51.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:50 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:50.974+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:51.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:50 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:50.974+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:51.402 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:51.116+0000 7f4f90851700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:05:51.402 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:51.116+0000 7f4f90851700 -1 osd.7 99 *** Got signal Terminated *** 2024-03-26T18:05:51.402 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:51.116+0000 7f4f90851700 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:05:51.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:51.511+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:51.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:51.511+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:51.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:51.511+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.945445+0000 front 2024-03-26T18:05:22.945318+0000 (oldest deadline 2024-03-26T18:05:48.845211+0000) 2024-03-26T18:05:51.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:51.671+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:51.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:51.671+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:51.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:51.671+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:51.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:51 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:51.671+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:52.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:51 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:51.757+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:52.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:51 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:51.757+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:52.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:51 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:51.757+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:52.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:51 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:51.714+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:52.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:51 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:51.714+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:52.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:51 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:51.714+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:52.378 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:51 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:51.940+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:52.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:51 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:51.940+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:52.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:51 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:51.940+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:52.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:51 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:51.940+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:52.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:52 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:52.514+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:52.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:52 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:52.514+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:52.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:52 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:52.514+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.945445+0000 front 2024-03-26T18:05:22.945318+0000 (oldest deadline 2024-03-26T18:05:48.845211+0000) 2024-03-26T18:05:52.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:52 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:52.702+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:52.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:52 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:52.702+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:52.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:52 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:52.702+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:52.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:52 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:52.702+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:53.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:52 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:52.783+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:53.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:52 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:52.783+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:53.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:52 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:52.783+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:53.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:52 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:52.742+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:53.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:52 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:52.742+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:53.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:52 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:52.742+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:53.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:52 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:52.927+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:53.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:52 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:52.927+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:53.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:52 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:52.927+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:53.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:52 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:52.927+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:53.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:53.493+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:53.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:53.493+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:53.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:53.493+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.945445+0000 front 2024-03-26T18:05:22.945318+0000 (oldest deadline 2024-03-26T18:05:48.845211+0000) 2024-03-26T18:05:53.902 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:53.493+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.746012+0000 front 2024-03-26T18:05:31.746059+0000 (oldest deadline 2024-03-26T18:05:53.446170+0000) 2024-03-26T18:05:53.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:53.711+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:53.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:53.711+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:53.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:53.711+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:53.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:53.711+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:53.903 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:53 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:53.711+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:05:54.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:53 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:53.723+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:54.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:53 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:53.723+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:54.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:53 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:53.723+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:54.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:53 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:53.742+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:54.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:53 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:53.742+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:54.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:53 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:53.742+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:54.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:53 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:53.743+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:05:54.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:53 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:53.884+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:54.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:53 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:53.884+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:54.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:53 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:53.884+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:54.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:53 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:53.884+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:54.724 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:54.478+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:54.725 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:54.478+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:54.725 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:54.478+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.945445+0000 front 2024-03-26T18:05:22.945318+0000 (oldest deadline 2024-03-26T18:05:48.845211+0000) 2024-03-26T18:05:54.725 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:54.478+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.746012+0000 front 2024-03-26T18:05:31.746059+0000 (oldest deadline 2024-03-26T18:05:53.446170+0000) 2024-03-26T18:05:55.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:54 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:54.728+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:55.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:54 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:54.728+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:55.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:54 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:54.728+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:55.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:54 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:54.751+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:55.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:54 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:54.751+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:55.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:54 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:54.751+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:55.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:54 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:54.751+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:05:55.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:54.843+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:55.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:54.843+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:55.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:54.843+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:55.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:54 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:54.843+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:55.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:54.745+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:55.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:54.745+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:55.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:54.745+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:55.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:54.745+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:55.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:54 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:54.745+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:05:55.767 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:55.485+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.543014+0000 front 2024-03-26T18:05:06.543129+0000 (oldest deadline 2024-03-26T18:05:30.043176+0000) 2024-03-26T18:05:55.767 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:55.485+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.944176+0000 front 2024-03-26T18:05:15.944152+0000 (oldest deadline 2024-03-26T18:05:39.444248+0000) 2024-03-26T18:05:55.767 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:55.485+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.945445+0000 front 2024-03-26T18:05:22.945318+0000 (oldest deadline 2024-03-26T18:05:48.845211+0000) 2024-03-26T18:05:55.767 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7[53610]: 2024-03-26T18:05:55.485+0000 7f4f88e3d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.746012+0000 front 2024-03-26T18:05:31.746059+0000 (oldest deadline 2024-03-26T18:05:53.446170+0000) 2024-03-26T18:05:56.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:55 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:55.798+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:56.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:55 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:55.798+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:56.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:55 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:55.798+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:56.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:55 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:55.798+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:05:56.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:55 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:55.765+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:56.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:55 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:55.765+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:56.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:55 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:55.765+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:56.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:55 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:55.765+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:05:56.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:55.866+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:56.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:55.866+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:56.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:55.866+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:56.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:55.866+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:56.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:55 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:55.866+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:05:56.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:55.787+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:56.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:55.787+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:56.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:55.787+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:56.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:55.787+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:56.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:55.787+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:05:56.897 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:56 smithi135 podman[74893]: 2024-03-26 18:05:56.628662845 +0000 UTC m=+5.628669950 container died 6d56f89f29cb3d4f35da9af2da6f30d38dc3a31ab41300b264c552c3ea353eec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-26T18:05:56.897 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:56 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:56.738+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:56.898 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:56 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:56.738+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:56.898 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:56 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:56.738+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:56.898 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:56 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:56.738+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:56.898 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:56 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:56.738+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:05:57.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:56 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:56.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:57.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:56 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:56.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:57.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:56 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:56.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:57.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:56 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:56.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:05:57.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:56 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:56.776+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:57.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:56 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:56.776+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:57.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:56 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:56.776+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:57.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:56 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:56.776+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:05:57.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:56.885+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:57.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:56.885+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:57.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:56.885+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:57.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:56.885+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:57.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:56 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:56.885+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:05:57.152 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:57 smithi135 podman[74893]: 2024-03-26 18:05:57.034411513 +0000 UTC m=+6.034418607 container cleanup 6d56f89f29cb3d4f35da9af2da6f30d38dc3a31ab41300b264c552c3ea353eec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T18:05:57.152 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:57 smithi135 bash[74893]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7 2024-03-26T18:05:57.586 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:57 smithi135 podman[74912]: 2024-03-26 18:05:57.28547227 +0000 UTC m=+0.655487986 container remove 6d56f89f29cb3d4f35da9af2da6f30d38dc3a31ab41300b264c552c3ea353eec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-26T18:05:57.858 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:57 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:57.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:57.858 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:57 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:57.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:57.858 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:57 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:57.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:57.858 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:57 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:57.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:57.858 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:57 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:57.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:05:57.859 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:57 smithi135 podman[75002]: 2024-03-26 18:05:57.510974075 +0000 UTC m=+0.018288949 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:05:57.859 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:57 smithi135 podman[75002]: 2024-03-26 18:05:57.611101676 +0000 UTC m=+0.118416532 container create 88b39d988e3156c7fba1f8577c5400ae714d1f067c4df42567956648bd22392f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-26T18:05:57.859 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:57 smithi135 podman[75002]: 2024-03-26 18:05:57.805208114 +0000 UTC m=+0.312522970 container init 88b39d988e3156c7fba1f8577c5400ae714d1f067c4df42567956648bd22392f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7-deactivate, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T18:05:57.859 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:57 smithi135 podman[75002]: 2024-03-26 18:05:57.808306155 +0000 UTC m=+0.315621010 container start 88b39d988e3156c7fba1f8577c5400ae714d1f067c4df42567956648bd22392f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T18:05:57.859 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:57 smithi135 podman[75002]: 2024-03-26 18:05:57.880140737 +0000 UTC m=+0.387455595 container attach 88b39d988e3156c7fba1f8577c5400ae714d1f067c4df42567956648bd22392f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7-deactivate, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-26T18:05:58.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:57 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:57.888+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:58.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:57 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:57.888+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:58.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:57 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:57.888+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:58.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:57 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:57.888+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:05:58.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:57 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:57.785+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:58.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:57 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:57.785+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:58.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:57 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:57.785+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:58.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:57 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:57.785+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:05:58.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:57.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:58.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:57.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:58.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:57.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:58.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:57.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:58.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:57 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:57.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:05:58.151 INFO:journalctl@ceph.osd.7.smithi135.stdout:Mar 26 18:05:58 smithi135 podman[75002]: 2024-03-26 18:05:58.099456119 +0000 UTC m=+0.606770975 container died 88b39d988e3156c7fba1f8577c5400ae714d1f067c4df42567956648bd22392f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T18:05:58.659 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.7.service' 2024-03-26T18:05:58.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:05:58.741 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-26T18:05:58.741 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-26T18:05:58.742 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.8 2024-03-26T18:05:58.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:58 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:58.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:05:58.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:58 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:58.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:05:58.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:58 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:58.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:05:58.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:58 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:58.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:05:58.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:58 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:58.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:05:59.017 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:58 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:58.750+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:05:59.018 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:58 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:58.750+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:05:59.018 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:58 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:58.750+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:05:59.018 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:58 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:58.750+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:05:59.018 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:58 smithi144 systemd[1]: Stopping Ceph osd.8 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:05:59.018 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:58 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:58.863+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:05:59.018 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:58 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:58.863+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:05:59.018 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:58 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:58.863+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:05:59.018 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:58 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:58.863+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:05:59.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:58 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:58.880+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:05:59.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:58 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:58.880+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:05:59.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:58 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:58.880+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:05:59.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:58 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:58.880+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:05:59.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:58 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:58.880+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:05:59.335 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:59.038+0000 7fc2fa9ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:05:59.335 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:59.038+0000 7fc2fa9ca700 -1 osd.8 99 *** Got signal Terminated *** 2024-03-26T18:05:59.335 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:59.038+0000 7fc2fa9ca700 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:06:00.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:59.855+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:06:00.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:59.855+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:06:00.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:59.855+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:06:00.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:05:59.855+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:06:00.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:59.719+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:00.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:59.719+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:00.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:59.719+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:00.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:05:59 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:05:59.720+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:00.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:59.899+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:00.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:59.899+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:00.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:59.899+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:00.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:59.899+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:00.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:05:59 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:05:59.899+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:00.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:59 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:59.792+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:00.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:59 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:59.792+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:00.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:59 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:59.792+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:00.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:59 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:59.792+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:00.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:05:59 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:05:59.792+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:01.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:00 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:00.887+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:06:01.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:00 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:00.887+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:06:01.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:00 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:00.887+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:06:01.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:00 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:00.887+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:06:01.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:00 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:00.698+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:01.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:00 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:00.698+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:01.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:00 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:00.698+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:01.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:00 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:00.698+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:01.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:00 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:00.858+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:01.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:00 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:00.858+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:01.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:00 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:00.858+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:01.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:00 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:00.858+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:01.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:00 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:00.858+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:01.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:00 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:00.821+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:01.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:00 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:00.821+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:01.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:00 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:00.821+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:01.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:00 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:00.821+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:01.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:00 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:00.821+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:02.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:01.894+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:06:02.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:01.894+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:06:02.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:01.894+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:06:02.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:01.894+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:06:02.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:01.894+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:35.218019+0000 front 2024-03-26T18:05:35.218083+0000 (oldest deadline 2024-03-26T18:06:01.118153+0000) 2024-03-26T18:06:02.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:01.656+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:02.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:01.656+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:02.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:01.656+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:02.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:01 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:01.656+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:02.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:01 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:01.898+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:02.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:01 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:01.898+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:02.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:01 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:01.898+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:02.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:01 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:01.898+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:02.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:01 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:01.898+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:02.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:01 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:01.832+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:02.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:01 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:01.832+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:02.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:01 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:01.832+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:02.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:01 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:01.832+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:02.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:01 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:01.832+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:03.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:02.856+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:06:03.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:02.856+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:06:03.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:02.856+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:06:03.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:02.856+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:06:03.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:02.856+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:35.218019+0000 front 2024-03-26T18:05:35.218083+0000 (oldest deadline 2024-03-26T18:06:01.118153+0000) 2024-03-26T18:06:03.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:02.613+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:03.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:02.613+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:03.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:02.613+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:03.086 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:02.613+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:03.087 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:02 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:02.613+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:03.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:02 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:02.830+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:03.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:02 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:02.830+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:03.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:02 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:02.830+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:03.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:02 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:02.830+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:03.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:02 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:02.830+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:03.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:02 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:02.830+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:03.378 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:02 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:02.901+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:03.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:02 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:02.901+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:03.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:02 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:02.901+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:03.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:02 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:02.901+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:03.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:02 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:02.901+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:03.822 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:03.573+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:03.822 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:03.573+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:03.822 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:03.573+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:03.823 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:03.573+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:03.823 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:03.573+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:04.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:03.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.113403+0000 front 2024-03-26T18:05:06.113248+0000 (oldest deadline 2024-03-26T18:05:28.413336+0000) 2024-03-26T18:06:04.085 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:03.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:15.414834+0000 front 2024-03-26T18:05:15.414797+0000 (oldest deadline 2024-03-26T18:05:40.115031+0000) 2024-03-26T18:06:04.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:03.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.016180+0000 front 2024-03-26T18:05:23.016306+0000 (oldest deadline 2024-03-26T18:05:48.916101+0000) 2024-03-26T18:06:04.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:03.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.617254+0000 front 2024-03-26T18:05:30.617159+0000 (oldest deadline 2024-03-26T18:05:53.517142+0000) 2024-03-26T18:06:04.086 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8[54704]: 2024-03-26T18:06:03.843+0000 7fc2f6fbe700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:35.218019+0000 front 2024-03-26T18:05:35.218083+0000 (oldest deadline 2024-03-26T18:06:01.118153+0000) 2024-03-26T18:06:04.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:03 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:03.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:04.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:03 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:03.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:04.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:03 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:03.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:04.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:03 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:03.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:04.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:03 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:03.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:04.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:03 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:03.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:04.378 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:03 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:03.900+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:04.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:03 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:03.900+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:04.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:03 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:03.900+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:04.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:03 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:03.900+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:04.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:03 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:03.900+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:04.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:03 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:03.900+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:04.676 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:04 smithi144 podman[75825]: 2024-03-26 18:06:04.439669473 +0000 UTC m=+5.478047027 container died 3777dac6b6fce6cd916a5920d1c0356c5d63d79d4a317be40ed5b11be8c66f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-03-26T18:06:04.676 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:04 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:04.575+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:04.677 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:04 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:04.575+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:04.677 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:04 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:04.575+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:04.677 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:04 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:04.575+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:04.677 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:04 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:04.575+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:04.993 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:04 smithi144 podman[75825]: 2024-03-26 18:06:04.779992772 +0000 UTC m=+5.818370337 container cleanup 3777dac6b6fce6cd916a5920d1c0356c5d63d79d4a317be40ed5b11be8c66f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T18:06:04.993 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:04 smithi144 bash[75825]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8 2024-03-26T18:06:05.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:04 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:04.795+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:05.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:04 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:04.795+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:05.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:04 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:04.795+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:05.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:04 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:04.795+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:05.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:04 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:04.795+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:05.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:04 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:04.795+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:05.335 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:05 smithi144 podman[75846]: 2024-03-26 18:06:05.014862531 +0000 UTC m=+0.573417740 container remove 3777dac6b6fce6cd916a5920d1c0356c5d63d79d4a317be40ed5b11be8c66f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T18:06:05.378 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:04.914+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:05.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:04.914+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:05.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:04.914+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:05.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:04.914+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:05.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:04.914+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:05.379 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:04 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:04.914+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:05.666 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:05 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:05.615+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:05.666 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:05 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:05.615+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:05.666 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:05 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:05.615+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:05.666 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:05 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:05.615+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:05.666 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:05 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:05.615+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:05.666 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:05 smithi144 podman[75936]: 2024-03-26 18:06:05.333940402 +0000 UTC m=+0.018013579 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:06:05.667 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:05 smithi144 podman[75936]: 2024-03-26 18:06:05.432329155 +0000 UTC m=+0.116402398 container create 5ad021f0548a25366510dcdb108dc59571803d367b47efce774ebd1060060a19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:06:05.667 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:05 smithi144 podman[75936]: 2024-03-26 18:06:05.687562096 +0000 UTC m=+0.371635281 container init 5ad021f0548a25366510dcdb108dc59571803d367b47efce774ebd1060060a19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-03-26T18:06:05.966 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:05 smithi144 podman[75936]: 2024-03-26 18:06:05.690871116 +0000 UTC m=+0.374944294 container start 5ad021f0548a25366510dcdb108dc59571803d367b47efce774ebd1060060a19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T18:06:05.967 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:05 smithi144 podman[75936]: 2024-03-26 18:06:05.784983972 +0000 UTC m=+0.469057150 container attach 5ad021f0548a25366510dcdb108dc59571803d367b47efce774ebd1060060a19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:06:06.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:05 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:05.870+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:06.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:05 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:05.870+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:06.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:05 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:05.870+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:06.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:05 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:05.870+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:06.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:05 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:05.870+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:06.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:05 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:05.870+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:06.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:05.826+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:06.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:05.826+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:06.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:05.826+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:06.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:05.826+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:06.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:05.826+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:06.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:05 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:05.826+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:06.244 INFO:journalctl@ceph.osd.8.smithi144.stdout:Mar 26 18:06:05 smithi144 podman[75936]: 2024-03-26 18:06:05.988565421 +0000 UTC m=+0.672638599 container died 5ad021f0548a25366510dcdb108dc59571803d367b47efce774ebd1060060a19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:06:06.568 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.8.service' 2024-03-26T18:06:06.608 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:06.608 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-26T18:06:06.608 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-26T18:06:06.608 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.9 2024-03-26T18:06:06.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:06 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:06.583+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:06.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:06 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:06.583+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:06.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:06 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:06.583+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:06.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:06 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:06.583+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:06.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:06 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:06.583+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:06.923 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 systemd[1]: Stopping Ceph osd.9 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:06.923 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:06.923 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:06.923 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:06.923 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:06.923 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:06.923 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.871+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:06.923 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.944+0000 7fb814a19700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:06.924 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.944+0000 7fb814a19700 -1 osd.9 99 *** Got signal Terminated *** 2024-03-26T18:06:06.924 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:06 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:06.944+0000 7fb814a19700 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:06:07.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:06 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:06.798+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:07.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:06 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:06.798+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:07.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:06 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:06.798+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:07.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:06 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:06.798+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:07.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:06 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:06.798+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:07.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:06 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:06.798+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:07.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:07 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:07.536+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:07.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:07 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:07.536+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:07.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:07 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:07.536+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:07.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:07 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:07.536+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:07.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:07 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:07.536+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:08.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:07 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:07.828+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:08.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:07 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:07.828+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:08.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:07 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:07.828+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:08.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:07 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:07.828+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:08.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:07 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:07.828+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:08.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:07 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:07.828+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:08.151 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:07.784+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:08.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:07.784+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:08.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:07.784+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:08.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:07.784+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:08.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:07.784+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:08.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:07 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:07.784+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:08.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:08 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:08.541+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:08.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:08 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:08.541+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:08.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:08 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:08.541+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:08.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:08 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:08.541+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:08.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:08 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:08.541+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:09.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:08 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:08.844+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:09.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:08 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:08.844+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:09.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:08 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:08.844+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:09.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:08 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:08.844+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:09.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:08 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:08.844+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:09.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:08 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:08.844+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:09.151 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:08 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:08.758+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:09.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:08 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:08.758+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:09.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:08 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:08.758+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:09.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:08 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:08.758+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:09.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:08 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:08.758+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:09.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:08 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:08.758+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:09.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:09 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:09.591+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:09.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:09 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:09.591+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:09.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:09 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:09.591+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:09.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:09 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:09.591+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:09.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:09 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:09.591+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:09.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:09 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:09.591+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:10.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:09 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:09.841+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:10.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:09 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:09.841+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:10.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:09 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:09.841+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:10.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:09 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:09.841+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:10.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:09 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:09.841+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:10.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:09 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:09.841+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:10.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:09 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:09.841+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:43.276503+0000 front 2024-03-26T18:05:43.276547+0000 (oldest deadline 2024-03-26T18:06:09.077070+0000) 2024-03-26T18:06:10.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:09 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:09.771+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:10.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:09 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:09.771+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:10.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:09 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:09.771+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:10.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:09 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:09.771+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:10.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:09 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:09.771+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:10.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:09 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:09.771+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:10.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:10 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:10.570+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:10.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:10 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:10.570+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:10.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:10 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:10.570+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:10.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:10 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:10.570+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:10.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:10 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:10.570+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:10.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:10 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:10.570+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:11.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:10 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:10.865+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:11.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:10 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:10.865+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:11.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:10 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:10.865+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:11.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:10 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:10.865+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:11.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:10 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:10.865+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:11.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:10 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:10.865+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:11.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:10 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:10.865+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:43.276503+0000 front 2024-03-26T18:05:43.276547+0000 (oldest deadline 2024-03-26T18:06:09.077070+0000) 2024-03-26T18:06:11.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:10 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:10.820+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:11.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:10 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:10.820+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:11.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:10 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:10.820+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:11.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:10 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:10.820+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:11.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:10 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:10.820+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:11.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:10 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:10.820+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:11.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:11.553+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:11.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:11.553+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:11.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:11.553+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:11.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:11.553+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:11.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:11.553+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:11.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:11.553+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:12.128 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:11 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:11.862+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.567135+0000 front 2024-03-26T18:04:57.567094+0000 (oldest deadline 2024-03-26T18:05:20.467165+0000) 2024-03-26T18:06:12.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:11 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:11.862+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.668730+0000 front 2024-03-26T18:05:05.668763+0000 (oldest deadline 2024-03-26T18:05:29.768797+0000) 2024-03-26T18:06:12.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:11 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:11.862+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.071413+0000 front 2024-03-26T18:05:13.071339+0000 (oldest deadline 2024-03-26T18:05:38.371453+0000) 2024-03-26T18:06:12.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:11 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:11.862+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:22.372769+0000 front 2024-03-26T18:05:22.372822+0000 (oldest deadline 2024-03-26T18:05:47.073037+0000) 2024-03-26T18:06:12.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:11 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:11.862+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.074745+0000 front 2024-03-26T18:05:31.074564+0000 (oldest deadline 2024-03-26T18:05:55.174504+0000) 2024-03-26T18:06:12.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:11 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:11.862+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.976101+0000 front 2024-03-26T18:05:37.976113+0000 (oldest deadline 2024-03-26T18:06:03.276080+0000) 2024-03-26T18:06:12.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:11 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9[67043]: 2024-03-26T18:06:11.862+0000 7fb80d005700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:43.276503+0000 front 2024-03-26T18:05:43.276547+0000 (oldest deadline 2024-03-26T18:06:09.077070+0000) 2024-03-26T18:06:12.151 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:11 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:11.863+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:12.151 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:11 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:11.863+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:12.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:11 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:11.863+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:12.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:11 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:11.863+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:12.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:11 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:11.863+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:12.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:11 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:11.863+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:12.657 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:12 smithi105 podman[128271]: 2024-03-26 18:06:12.421137117 +0000 UTC m=+5.596778384 container died 70f2f136e305f894387beed6835bd217216e3e818ee704c80ab46455b3b699f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-26T18:06:12.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:12 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:12.522+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:12.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:12 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:12.522+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:12.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:12 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:12.522+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:12.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:12 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:12.522+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:12.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:12 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:12.522+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:12.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:12 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:12.522+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:12.956 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:12 smithi105 podman[128271]: 2024-03-26 18:06:12.768971918 +0000 UTC m=+5.944613188 container cleanup 70f2f136e305f894387beed6835bd217216e3e818ee704c80ab46455b3b699f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-26T18:06:12.956 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:12 smithi105 bash[128271]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9 2024-03-26T18:06:13.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:12 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:12.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:13.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:12 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:12.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:13.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:12 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:12.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:13.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:12 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:12.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:13.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:12 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:12.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:13.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:12 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:12.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:13.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:12 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:12.814+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:45.997761+0000 front 2024-03-26T18:05:45.997737+0000 (oldest deadline 2024-03-26T18:06:11.897607+0000) 2024-03-26T18:06:13.354 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:12 smithi105 podman[128288]: 2024-03-26 18:06:12.978621571 +0000 UTC m=+0.551447317 container remove 70f2f136e305f894387beed6835bd217216e3e818ee704c80ab46455b3b699f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:06:13.628 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:13 smithi105 podman[128376]: 2024-03-26 18:06:13.27935454 +0000 UTC m=+0.018096368 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:06:13.628 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:13 smithi105 podman[128376]: 2024-03-26 18:06:13.387744836 +0000 UTC m=+0.126486729 container create 6f33b748e60b9da5b90fa69ca389f0e433e0f85c736e4287b8b79ee2f6aaf029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T18:06:13.629 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:13 smithi105 podman[128376]: 2024-03-26 18:06:13.606995297 +0000 UTC m=+0.345737124 container init 6f33b748e60b9da5b90fa69ca389f0e433e0f85c736e4287b8b79ee2f6aaf029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:06:13.629 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:13 smithi105 podman[128376]: 2024-03-26 18:06:13.610213692 +0000 UTC m=+0.348955511 container start 6f33b748e60b9da5b90fa69ca389f0e433e0f85c736e4287b8b79ee2f6aaf029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-26T18:06:13.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:13 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:13.476+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:13.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:13 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:13.476+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:13.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:13 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:13.476+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:13.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:13 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:13.476+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:13.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:13 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:13.476+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:13.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:13 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:13.476+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:14.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:13 smithi105 podman[128376]: 2024-03-26 18:06:13.698762647 +0000 UTC m=+0.437504474 container attach 6f33b748e60b9da5b90fa69ca389f0e433e0f85c736e4287b8b79ee2f6aaf029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:06:14.129 INFO:journalctl@ceph.osd.9.smithi105.stdout:Mar 26 18:06:13 smithi105 podman[128376]: 2024-03-26 18:06:13.8986492 +0000 UTC m=+0.637391019 container died 6f33b748e60b9da5b90fa69ca389f0e433e0f85c736e4287b8b79ee2f6aaf029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:06:14.151 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:13.801+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:14.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:13.801+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:14.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:13.801+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:14.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:13.801+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:14.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:13.801+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:14.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:13.801+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:14.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:13 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:13.801+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:45.997761+0000 front 2024-03-26T18:05:45.997737+0000 (oldest deadline 2024-03-26T18:06:11.897607+0000) 2024-03-26T18:06:14.468 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.9.service' 2024-03-26T18:06:14.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:14.509 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-26T18:06:14.509 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-26T18:06:14.509 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.10 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 systemd[1]: Stopping Ceph osd.10 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.806+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.806+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.806+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.806+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.806+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.806+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.806+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:45.997761+0000 front 2024-03-26T18:05:45.997737+0000 (oldest deadline 2024-03-26T18:06:11.897607+0000) 2024-03-26T18:06:14.809 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.829+0000 7f5ef1be1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:14.810 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.829+0000 7f5ef1be1700 -1 osd.10 99 *** Got signal Terminated *** 2024-03-26T18:06:14.810 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:14 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:14.829+0000 7f5ef1be1700 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:06:14.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:14 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:14.487+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:14.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:14 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:14.487+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:14.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:14 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:14.487+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:14.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:14 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:14.487+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:14.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:14 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:14.487+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:14.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:14 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:14.487+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:15.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:15.519+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:15.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:15.519+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:15.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:15.519+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:15.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:15.519+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:15.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:15.519+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:15.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:15.519+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:16.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:15 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:15.785+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:16.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:15 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:15.786+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:16.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:15 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:15.786+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:16.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:15 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:15.786+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:16.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:15 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:15.786+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:16.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:15 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:15.786+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:16.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:15 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:15.786+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:45.997761+0000 front 2024-03-26T18:05:45.997737+0000 (oldest deadline 2024-03-26T18:06:11.897607+0000) 2024-03-26T18:06:16.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:16 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:16.516+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:16.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:16 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:16.516+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:16.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:16 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:16.516+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:16.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:16 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:16.516+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:16.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:16 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:16.516+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:16.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:16 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:16.516+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:17.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:16 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:16.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:17.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:16 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:16.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:17.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:16 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:16.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:17.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:16 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:16.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:17.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:16 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:16.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:17.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:16 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:16.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:17.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:16 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:16.764+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:45.997761+0000 front 2024-03-26T18:05:45.997737+0000 (oldest deadline 2024-03-26T18:06:11.897607+0000) 2024-03-26T18:06:17.585 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:17 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:17.484+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:17.585 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:17 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:17.484+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:17.585 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:17 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:17.484+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:17.586 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:17 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:17.484+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:17.586 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:17 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:17.484+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:17.586 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:17 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:17.484+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:18.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:17 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:17.788+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:18.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:17 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:17.788+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:18.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:17 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:17.788+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:18.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:17 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:17.788+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:18.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:17 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:17.788+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:18.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:17 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:17.788+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:18.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:17 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:17.788+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:45.997761+0000 front 2024-03-26T18:05:45.997737+0000 (oldest deadline 2024-03-26T18:06:11.897607+0000) 2024-03-26T18:06:18.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:18 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:18.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:18.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:18 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:18.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:18.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:18 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:18.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:18.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:18 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:18.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:18.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:18 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:18.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:18.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:18 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:18.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:19.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:18 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:18.763+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:19.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:18 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:18.763+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:19.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:18 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:18.763+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:19.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:18 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:18.763+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:19.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:18 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:18.763+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:19.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:18 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:18.763+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:19.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:18 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:18.763+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:45.997761+0000 front 2024-03-26T18:05:45.997737+0000 (oldest deadline 2024-03-26T18:06:11.897607+0000) 2024-03-26T18:06:19.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:19 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:19.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:19.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:19 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:19.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:19.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:19 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:19.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:19.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:19 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:19.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:19.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:19 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:19.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:19.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:19 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:19.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:19.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:19 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:19.523+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:20.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:19 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:19.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:57.189178+0000 front 2024-03-26T18:04:57.189235+0000 (oldest deadline 2024-03-26T18:05:21.889152+0000) 2024-03-26T18:06:20.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:19 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:19.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:05.990139+0000 front 2024-03-26T18:05:05.990225+0000 (oldest deadline 2024-03-26T18:05:27.690334+0000) 2024-03-26T18:06:20.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:19 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:19.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:13.492211+0000 front 2024-03-26T18:05:13.492137+0000 (oldest deadline 2024-03-26T18:05:39.391826+0000) 2024-03-26T18:06:20.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:19 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:19.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6814 osd.3 since back 2024-03-26T18:05:23.893845+0000 front 2024-03-26T18:05:23.893865+0000 (oldest deadline 2024-03-26T18:05:47.993832+0000) 2024-03-26T18:06:20.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:19 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:19.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:31.494785+0000 front 2024-03-26T18:05:31.494803+0000 (oldest deadline 2024-03-26T18:05:53.194983+0000) 2024-03-26T18:06:20.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:19 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:19.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.196835+0000 front 2024-03-26T18:05:37.196949+0000 (oldest deadline 2024-03-26T18:06:02.496806+0000) 2024-03-26T18:06:20.153 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:19 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10[60194]: 2024-03-26T18:06:19.802+0000 7f5eee1d5700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:45.997761+0000 front 2024-03-26T18:05:45.997737+0000 (oldest deadline 2024-03-26T18:06:11.897607+0000) 2024-03-26T18:06:20.651 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:20 smithi135 podman[75120]: 2024-03-26 18:06:20.375815037 +0000 UTC m=+5.647138431 container died edad98145e63ef7052b6ea4abf7a477013b0fa2c4b2f7b89bc847c419e7c2173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-26T18:06:20.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:20 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:20.509+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:20.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:20 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:20.509+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:20.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:20 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:20.509+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:20.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:20 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:20.510+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:20.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:20 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:20.510+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:20.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:20 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:20.510+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:20.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:20 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:20.510+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:20.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:20 smithi135 podman[75120]: 2024-03-26 18:06:20.771431057 +0000 UTC m=+6.042754428 container cleanup edad98145e63ef7052b6ea4abf7a477013b0fa2c4b2f7b89bc847c419e7c2173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-26T18:06:20.902 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:20 smithi135 bash[75120]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10 2024-03-26T18:06:21.364 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:21 smithi135 podman[75138]: 2024-03-26 18:06:21.03123012 +0000 UTC m=+0.637891438 container remove edad98145e63ef7052b6ea4abf7a477013b0fa2c4b2f7b89bc847c419e7c2173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:06:21.652 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:21 smithi135 podman[75226]: 2024-03-26 18:06:21.289442144 +0000 UTC m=+0.017994372 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:06:21.652 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:21 smithi135 podman[75226]: 2024-03-26 18:06:21.39017614 +0000 UTC m=+0.118728363 container create 039df08b98be0b2db017ef77407fa8d9590724d42b9c4615ee94685808f4d277 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10-deactivate, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:06:21.652 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:21 smithi135 podman[75226]: 2024-03-26 18:06:21.609136572 +0000 UTC m=+0.337688797 container init 039df08b98be0b2db017ef77407fa8d9590724d42b9c4615ee94685808f4d277 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:06:21.652 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:21 smithi135 podman[75226]: 2024-03-26 18:06:21.61217412 +0000 UTC m=+0.340726343 container start 039df08b98be0b2db017ef77407fa8d9590724d42b9c4615ee94685808f4d277 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T18:06:21.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:21 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:21.515+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:21.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:21 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:21.515+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:21.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:21 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:21.515+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:21.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:21 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:21.515+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:21.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:21 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:21.515+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:21.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:21 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:21.515+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:21.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:21 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:21.515+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:22.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:21 smithi135 podman[75226]: 2024-03-26 18:06:21.683978591 +0000 UTC m=+0.412530842 container attach 039df08b98be0b2db017ef77407fa8d9590724d42b9c4615ee94685808f4d277 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:06:22.152 INFO:journalctl@ceph.osd.10.smithi135.stdout:Mar 26 18:06:21 smithi135 podman[75226]: 2024-03-26 18:06:21.901923844 +0000 UTC m=+0.630476068 container died 039df08b98be0b2db017ef77407fa8d9590724d42b9c4615ee94685808f4d277 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T18:06:22.446 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.10.service' 2024-03-26T18:06:22.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:22.487 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-26T18:06:22.487 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-26T18:06:22.487 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.11 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.545+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.545+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.545+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.545+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.545+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.545+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.545+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 systemd[1]: Stopping Ceph osd.11 for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:22.828 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.848+0000 7f123a47b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:22.829 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.848+0000 7f123a47b700 -1 osd.11 99 *** Got signal Terminated *** 2024-03-26T18:06:22.829 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:22 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:22.848+0000 7f123a47b700 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T18:06:23.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:23 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:23.525+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:23.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:23 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:23.525+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:23.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:23 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:23.525+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:23.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:23 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:23.525+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:23.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:23 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:23.525+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:23.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:23 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:23.525+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:23.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:23 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:23.525+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:24.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:24 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:24.488+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:24.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:24 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:24.488+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:24.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:24 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:24.488+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:24.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:24 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:24.488+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:24.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:24 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:24.488+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:24.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:24 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:24.488+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:24.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:24 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:24.488+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:25.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:25.466+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:25.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:25.466+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:25.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:25.466+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:25.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:25.466+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:25.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:25.466+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:25.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:25.466+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:25.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:25 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:25.466+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:26.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:26.495+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:26.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:26.495+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:26.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:26.495+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:26.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:26.495+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:26.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:26.495+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:26.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:26.495+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:26.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:26 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:26.495+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:27.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:27.503+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6806 osd.0 since back 2024-03-26T18:04:54.103667+0000 front 2024-03-26T18:04:54.103839+0000 (oldest deadline 2024-03-26T18:05:21.104439+0000) 2024-03-26T18:06:27.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:27.503+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6804 osd.1 since back 2024-03-26T18:05:06.306352+0000 front 2024-03-26T18:05:06.306313+0000 (oldest deadline 2024-03-26T18:05:29.806475+0000) 2024-03-26T18:06:27.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:27.503+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6804 osd.2 since back 2024-03-26T18:05:14.908848+0000 front 2024-03-26T18:05:14.908894+0000 (oldest deadline 2024-03-26T18:05:41.309826+0000) 2024-03-26T18:06:27.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:27.503+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6812 osd.4 since back 2024-03-26T18:05:30.711534+0000 front 2024-03-26T18:05:30.711555+0000 (oldest deadline 2024-03-26T18:05:55.411441+0000) 2024-03-26T18:06:27.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:27.503+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.144:6812 osd.5 since back 2024-03-26T18:05:37.112295+0000 front 2024-03-26T18:05:37.112396+0000 (oldest deadline 2024-03-26T18:06:02.412486+0000) 2024-03-26T18:06:27.835 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:27.503+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.105:6822 osd.6 since back 2024-03-26T18:05:44.614129+0000 front 2024-03-26T18:05:44.614080+0000 (oldest deadline 2024-03-26T18:06:08.713978+0000) 2024-03-26T18:06:27.836 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:27 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11[61558]: 2024-03-26T18:06:27.503+0000 7f1232a67700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.135:6820 osd.7 since back 2024-03-26T18:05:54.515319+0000 front 2024-03-26T18:05:54.515250+0000 (oldest deadline 2024-03-26T18:06:18.615182+0000) 2024-03-26T18:06:28.464 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:28 smithi144 podman[76055]: 2024-03-26 18:06:28.23153761 +0000 UTC m=+5.529571064 container died 0b651b33db3c119478302c176bdee1651b716a45192f8a54675e24895daafe7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T18:06:28.761 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:28 smithi144 podman[76055]: 2024-03-26 18:06:28.573569074 +0000 UTC m=+5.871602534 container cleanup 0b651b33db3c119478302c176bdee1651b716a45192f8a54675e24895daafe7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T18:06:28.761 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:28 smithi144 bash[76055]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11 2024-03-26T18:06:29.085 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:28 smithi144 podman[76073]: 2024-03-26 18:06:28.783049574 +0000 UTC m=+0.547168010 container remove 0b651b33db3c119478302c176bdee1651b716a45192f8a54675e24895daafe7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T18:06:29.464 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:29 smithi144 podman[76165]: 2024-03-26 18:06:29.087821998 +0000 UTC m=+0.017490458 image pull 497bb87bc5da137bba1d1c5a539351c8d0563edd13b9adcc04bcd9377cdbb016 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9 2024-03-26T18:06:29.465 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:29 smithi144 podman[76165]: 2024-03-26 18:06:29.241907667 +0000 UTC m=+0.171576128 container create 7acdc85ab78426aba873aeaa12f34319cd9a4f5cdcca1a6bf70ac2e1769d0f16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T18:06:29.756 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:29 smithi144 podman[76165]: 2024-03-26 18:06:29.486490477 +0000 UTC m=+0.416158937 container init 7acdc85ab78426aba873aeaa12f34319cd9a4f5cdcca1a6bf70ac2e1769d0f16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T18:06:29.756 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:29 smithi144 podman[76165]: 2024-03-26 18:06:29.489818113 +0000 UTC m=+0.419486584 container start 7acdc85ab78426aba873aeaa12f34319cd9a4f5cdcca1a6bf70ac2e1769d0f16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-26T18:06:29.757 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:29 smithi144 podman[76165]: 2024-03-26 18:06:29.586898572 +0000 UTC m=+0.516567042 container attach 7acdc85ab78426aba873aeaa12f34319cd9a4f5cdcca1a6bf70ac2e1769d0f16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-26T18:06:30.051 INFO:journalctl@ceph.osd.11.smithi144.stdout:Mar 26 18:06:29 smithi144 podman[76165]: 2024-03-26 18:06:29.778234193 +0000 UTC m=+0.707902663 container died 7acdc85ab78426aba873aeaa12f34319cd9a4f5cdcca1a6bf70ac2e1769d0f16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T18:06:30.336 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@osd.11.service' 2024-03-26T18:06:30.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:30.376 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-26T18:06:30.377 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-26T18:06:30.377 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.a 2024-03-26T18:06:30.675 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:06:30 smithi105 systemd[1]: Stopping Ceph mds.a for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:31.128 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:06:30 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a[73602]: 2024-03-26T18:06:30.695+0000 7f8b036fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:31.128 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:06:30 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a[73602]: 2024-03-26T18:06:30.695+0000 7f8b036fb700 -1 mds.a *** got signal Terminated *** 2024-03-26T18:06:34.533 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:06:34 smithi105 podman[128493]: 2024-03-26 18:06:34.199521612 +0000 UTC m=+3.608227496 container died 4276e5e49d5470b605e313f3b6bfb61efb7fc0fb43e2417160c56e5aa28a208a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD) 2024-03-26T18:06:34.858 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:06:34 smithi105 podman[128493]: 2024-03-26 18:06:34.654365641 +0000 UTC m=+4.063071570 container cleanup 4276e5e49d5470b605e313f3b6bfb61efb7fc0fb43e2417160c56e5aa28a208a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-26T18:06:34.858 INFO:journalctl@ceph.mds.a.smithi105.stdout:Mar 26 18:06:34 smithi105 bash[128493]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-a 2024-03-26T18:06:34.981 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.a.service' 2024-03-26T18:06:35.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:35.017 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-26T18:06:35.017 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-26T18:06:35.018 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.d 2024-03-26T18:06:35.128 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:35 smithi105 systemd[1]: Stopping Ceph mds.d for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:35.628 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:35 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-d[72632]: 2024-03-26T18:06:35.373+0000 7f542f85e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:35.628 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:35 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-d[72632]: 2024-03-26T18:06:35.373+0000 7f542f85e700 -1 mds.d *** got signal Terminated *** 2024-03-26T18:06:39.628 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:39 smithi105 podman[128613]: 2024-03-26 18:06:39.319469041 +0000 UTC m=+4.067997096 container died c526cc4369abec0c67684a356c4df1c08b1ad623dd1e7d4696a1cd7ee0d3d9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-26T18:06:39.953 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:39 smithi105 podman[128613]: 2024-03-26 18:06:39.749536185 +0000 UTC m=+4.498064249 container cleanup c526cc4369abec0c67684a356c4df1c08b1ad623dd1e7d4696a1cd7ee0d3d9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:06:39.953 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:39 smithi105 bash[128613]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-d 2024-03-26T18:06:40.196 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.d.service' 2024-03-26T18:06:40.220 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:39 smithi105 podman[128627]: 2024-03-26 18:06:39.97575685 +0000 UTC m=+0.635419038 container remove c526cc4369abec0c67684a356c4df1c08b1ad623dd1e7d4696a1cd7ee0d3d9d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-d, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-26T18:06:40.220 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:40 smithi105 systemd[1]: ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.d.service: Deactivated successfully. 2024-03-26T18:06:40.220 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:40 smithi105 systemd[1]: Stopped Ceph mds.d for 15cee578-eb96-11ee-95d0-87774f69a715. 2024-03-26T18:06:40.220 INFO:journalctl@ceph.mds.d.smithi105.stdout:Mar 26 18:06:40 smithi105 systemd[1]: ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.d.service: Consumed 1.140s CPU time. 2024-03-26T18:06:40.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:40.232 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-26T18:06:40.233 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-26T18:06:40.233 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.g 2024-03-26T18:06:40.556 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:06:40 smithi105 systemd[1]: Stopping Ceph mds.g for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:40.878 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:06:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g[73240]: 2024-03-26T18:06:40.576+0000 7ff07a2d5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:40.878 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:06:40 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g[73240]: 2024-03-26T18:06:40.576+0000 7ff07a2d5700 -1 mds.g *** got signal Terminated *** 2024-03-26T18:06:42.532 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:06:42 smithi105 podman[128734]: 2024-03-26 18:06:42.197182143 +0000 UTC m=+1.727880170 container died ce2f700b9024bc43e1194e8bd119c9c794fe8ef12bf6ec9b20183cbab808c685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240320.145742-3605160) 2024-03-26T18:06:42.849 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:06:42 smithi105 podman[128734]: 2024-03-26 18:06:42.628651676 +0000 UTC m=+2.159349627 container cleanup ce2f700b9024bc43e1194e8bd119c9c794fe8ef12bf6ec9b20183cbab808c685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-26T18:06:42.849 INFO:journalctl@ceph.mds.g.smithi105.stdout:Mar 26 18:06:42 smithi105 bash[128734]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-g 2024-03-26T18:06:42.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.g.service' 2024-03-26T18:06:43.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:43.014 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-26T18:06:43.014 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-26T18:06:43.014 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.j 2024-03-26T18:06:43.128 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 18:06:43 smithi105 systemd[1]: Stopping Ceph mds.j for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:43.628 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 18:06:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j[72284]: 2024-03-26T18:06:43.375+0000 7fdbf3dd4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:43.628 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 18:06:43 smithi105 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j[72284]: 2024-03-26T18:06:43.375+0000 7fdbf3dd4700 -1 mds.j *** got signal Terminated *** 2024-03-26T18:06:44.810 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 18:06:44 smithi105 podman[128856]: 2024-03-26 18:06:44.552035396 +0000 UTC m=+1.303561114 container died 772e1ba46c6523514a84a35c4c980551467436bbb024aa5db363d908ba63b2df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-26T18:06:45.061 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 18:06:44 smithi105 podman[128856]: 2024-03-26 18:06:44.832044871 +0000 UTC m=+1.583570592 container cleanup 772e1ba46c6523514a84a35c4c980551467436bbb024aa5db363d908ba63b2df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-26T18:06:45.061 INFO:journalctl@ceph.mds.j.smithi105.stdout:Mar 26 18:06:44 smithi105 bash[128856]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-j 2024-03-26T18:06:45.184 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.j.service' 2024-03-26T18:06:45.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:45.219 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-26T18:06:45.219 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-26T18:06:45.219 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.b 2024-03-26T18:06:45.534 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:06:45 smithi135 systemd[1]: Stopping Ceph mds.b for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:45.901 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:06:45 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T18:06:45.555+0000 7fa422d6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:45.901 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:06:45 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b[65702]: 2024-03-26T18:06:45.555+0000 7fa422d6d700 -1 mds.b *** got signal Terminated *** 2024-03-26T18:06:49.818 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:06:49 smithi135 podman[75332]: 2024-03-26 18:06:49.580708235 +0000 UTC m=+4.140324060 container died 3d18c077baf57d81bd2d21245f57cfd21390f5d704dc6db981ecbdc3d7fc7e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:06:50.151 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:06:49 smithi135 podman[75332]: 2024-03-26 18:06:49.938825943 +0000 UTC m=+4.498441777 container cleanup 3d18c077baf57d81bd2d21245f57cfd21390f5d704dc6db981ecbdc3d7fc7e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-26T18:06:50.151 INFO:journalctl@ceph.mds.b.smithi135.stdout:Mar 26 18:06:49 smithi135 bash[75332]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-b 2024-03-26T18:06:50.334 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.b.service' 2024-03-26T18:06:50.370 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:50.370 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-26T18:06:50.370 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-26T18:06:50.370 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.e 2024-03-26T18:06:50.651 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:06:50 smithi135 systemd[1]: Stopping Ceph mds.e for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:51.151 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:06:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: 2024-03-26T18:06:50.775+0000 7f2cedb8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:51.151 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:06:50 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e[65082]: 2024-03-26T18:06:50.775+0000 7f2cedb8b700 -1 mds.e *** got signal Terminated *** 2024-03-26T18:06:54.449 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:06:54 smithi135 podman[75452]: 2024-03-26 18:06:54.187229306 +0000 UTC m=+3.571545611 container died 8a72d34a3e5ed7b4d139507d4c8ed461e8e8621867c750ef5a1c9398e070bf44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T18:06:54.814 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:06:54 smithi135 podman[75452]: 2024-03-26 18:06:54.5428995 +0000 UTC m=+3.927215709 container cleanup 8a72d34a3e5ed7b4d139507d4c8ed461e8e8621867c750ef5a1c9398e070bf44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-26T18:06:54.814 INFO:journalctl@ceph.mds.e.smithi135.stdout:Mar 26 18:06:54 smithi135 bash[75452]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-e 2024-03-26T18:06:54.921 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.e.service' 2024-03-26T18:06:54.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:54.955 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-26T18:06:54.955 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-26T18:06:54.956 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.h 2024-03-26T18:06:55.151 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:06:55 smithi135 systemd[1]: Stopping Ceph mds.h for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:55.651 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:06:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h[65386]: 2024-03-26T18:06:55.304+0000 7fba6ed02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:55.651 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:06:55 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h[65386]: 2024-03-26T18:06:55.304+0000 7fba6ed02700 -1 mds.h *** got signal Terminated *** 2024-03-26T18:06:58.434 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:06:58 smithi135 podman[75573]: 2024-03-26 18:06:58.182193386 +0000 UTC m=+2.984178913 container died 6677b0ca0560f57293849320bd45326263ae223b6c4913a895bfcb3fdf532dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T18:06:58.900 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:06:58 smithi135 podman[75573]: 2024-03-26 18:06:58.52160235 +0000 UTC m=+3.323587879 container cleanup 6677b0ca0560f57293849320bd45326263ae223b6c4913a895bfcb3fdf532dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-26T18:06:58.900 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:06:58 smithi135 bash[75573]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h 2024-03-26T18:06:58.900 INFO:journalctl@ceph.mds.h.smithi135.stdout:Mar 26 18:06:58 smithi135 podman[75587]: 2024-03-26 18:06:58.697689717 +0000 UTC m=+0.518295489 container remove 6677b0ca0560f57293849320bd45326263ae223b6c4913a895bfcb3fdf532dfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T18:06:58.923 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.h.service' 2024-03-26T18:06:58.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:06:58.957 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-26T18:06:58.957 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-26T18:06:58.957 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.k 2024-03-26T18:06:59.269 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:06:59 smithi135 systemd[1]: Stopping Ceph mds.k for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:06:59.651 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:06:59 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k[64760]: 2024-03-26T18:06:59.291+0000 7fabf0b4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:06:59.651 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:06:59 smithi135 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k[64760]: 2024-03-26T18:06:59.291+0000 7fabf0b4b700 -1 mds.k *** got signal Terminated *** 2024-03-26T18:07:02.437 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:07:02 smithi135 podman[75694]: 2024-03-26 18:07:02.183335393 +0000 UTC m=+2.987038214 container died 34eb9a26b5abaa12b788cd846c29331fedac8177f42ab09135861155d454e24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T18:07:02.698 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:07:02 smithi135 podman[75694]: 2024-03-26 18:07:02.533860167 +0000 UTC m=+3.337563028 container cleanup 34eb9a26b5abaa12b788cd846c29331fedac8177f42ab09135861155d454e24e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T18:07:02.698 INFO:journalctl@ceph.mds.k.smithi135.stdout:Mar 26 18:07:02 smithi135 bash[75694]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-k 2024-03-26T18:07:02.896 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.k.service' 2024-03-26T18:07:02.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:07:02.932 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-26T18:07:02.932 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-26T18:07:02.932 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.c 2024-03-26T18:07:03.259 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:07:03 smithi144 systemd[1]: Stopping Ceph mds.c for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:07:03.585 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:07:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c[66756]: 2024-03-26T18:07:03.280+0000 7fec112d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:07:03.585 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:07:03 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c[66756]: 2024-03-26T18:07:03.280+0000 7fec112d2700 -1 mds.c *** got signal Terminated *** 2024-03-26T18:07:06.517 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:07:06 smithi144 podman[76271]: 2024-03-26 18:07:06.179848985 +0000 UTC m=+3.016004952 container died 04c50fd54e37fc25b7f126f06859bee609c1d0704835665f871e1ecbaf2f91f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-26T18:07:06.834 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:07:06 smithi144 podman[76271]: 2024-03-26 18:07:06.638620219 +0000 UTC m=+3.474776193 container cleanup 04c50fd54e37fc25b7f126f06859bee609c1d0704835665f871e1ecbaf2f91f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T18:07:06.835 INFO:journalctl@ceph.mds.c.smithi144.stdout:Mar 26 18:07:06 smithi144 bash[76271]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-c 2024-03-26T18:07:07.082 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.c.service' 2024-03-26T18:07:07.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:07:07.121 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-26T18:07:07.121 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-26T18:07:07.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.f 2024-03-26T18:07:07.585 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:07:07 smithi144 systemd[1]: Stopping Ceph mds.f for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:07:07.585 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:07:07 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-f[66445]: 2024-03-26T18:07:07.450+0000 7f0b273a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:07:07.585 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:07:07 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-f[66445]: 2024-03-26T18:07:07.450+0000 7f0b273a9700 -1 mds.f *** got signal Terminated *** 2024-03-26T18:07:10.421 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:07:10 smithi144 podman[76403]: 2024-03-26 18:07:10.181564113 +0000 UTC m=+2.820028247 container died 8332b20c980fb6bd2eadf2aa34a32a755ea8167ca94127730a053d501d0bb8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240320.145742-3605160, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T18:07:10.796 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:07:10 smithi144 podman[76403]: 2024-03-26 18:07:10.567483333 +0000 UTC m=+3.205947478 container cleanup 8332b20c980fb6bd2eadf2aa34a32a755ea8167ca94127730a053d501d0bb8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240320.145742-3605160, io.buildah.version=1.33.5) 2024-03-26T18:07:10.796 INFO:journalctl@ceph.mds.f.smithi144.stdout:Mar 26 18:07:10 smithi144 bash[76403]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-f 2024-03-26T18:07:10.919 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.f.service' 2024-03-26T18:07:10.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:07:10.955 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-26T18:07:10.955 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-26T18:07:10.955 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.i 2024-03-26T18:07:11.084 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:07:11 smithi144 systemd[1]: Stopping Ceph mds.i for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:07:11.584 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:07:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i[67075]: 2024-03-26T18:07:11.295+0000 7f96d853b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:07:11.585 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:07:11 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i[67075]: 2024-03-26T18:07:11.295+0000 7f96d853b700 -1 mds.i *** got signal Terminated *** 2024-03-26T18:07:14.459 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:07:14 smithi144 podman[76524]: 2024-03-26 18:07:14.180537505 +0000 UTC m=+2.988076190 container died fe945a7b850fe4a87f2948605956b4ef211c9d14ec9ec0f93026ab2773da923f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i, RELEASE=wip-batrick-testing-20240320.145742-3605160, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T18:07:14.766 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:07:14 smithi144 podman[76524]: 2024-03-26 18:07:14.571323668 +0000 UTC m=+3.378862362 container cleanup fe945a7b850fe4a87f2948605956b4ef211c9d14ec9ec0f93026ab2773da923f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T18:07:14.766 INFO:journalctl@ceph.mds.i.smithi144.stdout:Mar 26 18:07:14 smithi144 bash[76524]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-i 2024-03-26T18:07:14.882 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.i.service' 2024-03-26T18:07:14.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:07:14.917 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-26T18:07:14.917 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-26T18:07:14.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.l 2024-03-26T18:07:15.084 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:07:15 smithi144 systemd[1]: Stopping Ceph mds.l for 15cee578-eb96-11ee-95d0-87774f69a715... 2024-03-26T18:07:15.584 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:07:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l[66133]: 2024-03-26T18:07:15.249+0000 7f79cb2f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T18:07:15.585 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:07:15 smithi144 ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l[66133]: 2024-03-26T18:07:15.249+0000 7f79cb2f2700 -1 mds.l *** got signal Terminated *** 2024-03-26T18:07:18.486 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:07:18 smithi144 podman[76645]: 2024-03-26 18:07:18.180469443 +0000 UTC m=+3.027706594 container died a7b6aeb1d2e201644d2f559532c1858d1d1cd2f92b0d9857a2fcebc2a218ab22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240320.145742-3605160, GIT_CLEAN=True) 2024-03-26T18:07:18.835 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:07:18 smithi144 podman[76645]: 2024-03-26 18:07:18.591023146 +0000 UTC m=+3.438260338 container cleanup a7b6aeb1d2e201644d2f559532c1858d1d1cd2f92b0d9857a2fcebc2a218ab22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d8dd1e9aa1093beda52502a2a16674060866f86acfc8d3eb725f25f5e1398ac9, name=ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240320.145742-3605160, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-26T18:07:18.835 INFO:journalctl@ceph.mds.l.smithi144.stdout:Mar 26 18:07:18 smithi144 bash[76645]: ceph-15cee578-eb96-11ee-95d0-87774f69a715-mds-l 2024-03-26T18:07:18.987 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-15cee578-eb96-11ee-95d0-87774f69a715@mds.l.service' 2024-03-26T18:07:19.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T18:07:19.021 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-26T18:07:19.021 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 15cee578-eb96-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T18:07:19.211 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T18:07:22.164 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 15cee578-eb96-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T18:07:22.354 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T18:07:25.525 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 15cee578-eb96-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T18:07:25.714 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T18:07:28.283 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:07:28.320 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:07:28.354 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T18:07:28.384 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-26T18:07:28.386 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi105/crash 2024-03-26T18:07:28.387 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash -- . 2024-03-26T18:07:28.730 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T18:07:28.731 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now 2024-03-26T18:07:28.733 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi135/crash 2024-03-26T18:07:28.734 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash -- . 2024-03-26T18:07:28.886 INFO:teuthology.orchestra.run.smithi135.stderr:tar: /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T18:07:28.886 INFO:teuthology.orchestra.run.smithi135.stderr:tar: Error is not recoverable: exiting now 2024-03-26T18:07:28.888 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi144/crash 2024-03-26T18:07:28.889 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash -- . 2024-03-26T18:07:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/15cee578-eb96-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T18:07:28.931 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-03-26T18:07:28.932 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-26T18:07:28.932 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T18:07:28.978 INFO:teuthology.orchestra.run.smithi105.stdout:2024-03-26T17:52:14.058729+0000 mds.b (mds.0) 139 : cluster [WRN] Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T18:07:28.978 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-26T18:07:28.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[SEC\]' /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T18:07:29.016 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]' /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T18:07:29.092 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[WRN\]' /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v '\(PG_DEGRADED' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T18:07:29.167 INFO:teuthology.orchestra.run.smithi105.stdout:2024-03-26T17:52:14.058729+0000 mds.b (mds.0) 139 : cluster [WRN] Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-03-26T18:07:29.167 INFO:tasks.cephadm:Compressing logs... 2024-03-26T18:07:29.167 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T18:07:29.209 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T18:07:29.212 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T18:07:29.254 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T18:07:29.254 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T18:07:29.255 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-26T18:07:29.255 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T18:07:29.255 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-26T18:07:29.255 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-26T18:07:29.258 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-26T18:07:29.258 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-26T18:07:29.258 INFO:teuthology.orchestra.run.smithi144.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T18:07:29.258 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-26T18:07:29.258 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-26T18:07:29.258 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T18:07:29.258 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log 2024-03-26T18:07:29.259 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-26T18:07:29.259 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-26T18:07:29.259 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T18:07:29.259 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-26T18:07:29.260 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-03-26T18:07:29.260 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/cephadm.log 2024-03-26T18:07:29.270 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-26T18:07:29.271 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-26T18:07:29.271 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-26T18:07:29.271 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T18:07:29.275 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.6.log: /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-26T18:07:29.275 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-26T18:07:29.275 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.g.log: -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-26T18:07:29.275 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-26T18:07:29.275 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-26T18:07:29.276 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T18:07:29.276 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log 2024-03-26T18:07:29.276 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T18:07:29.276 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-26T18:07:29.276 INFO:teuthology.orchestra.run.smithi105.stderr: 83.7% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T18:07:29.276 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-26T18:07:29.276 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-26T18:07:29.277 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.d.log: /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.a.log: 87.3% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T18:07:29.277 INFO:teuthology.orchestra.run.smithi135.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T18:07:29.277 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T18:07:29.277 INFO:teuthology.orchestra.run.smithi105.stderr: 90.4% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T18:07:29.279 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-26T18:07:29.281 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T18:07:29.281 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-26T18:07:29.281 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-26T18:07:29.281 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-26T18:07:29.281 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-26T18:07:29.283 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-26T18:07:29.284 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-26T18:07:29.284 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-26T18:07:29.284 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-26T18:07:29.284 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T18:07:29.284 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log 2024-03-26T18:07:29.284 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T18:07:29.284 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-26T18:07:29.380 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T18:07:29.464 INFO:teuthology.orchestra.run.smithi105.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T18:07:29.470 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T18:07:29.492 INFO:teuthology.orchestra.run.smithi135.stderr: 93.4% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-26T18:07:29.504 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T18:07:29.517 INFO:teuthology.orchestra.run.smithi144.stderr: 93.5% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-26T18:07:29.530 INFO:teuthology.orchestra.run.smithi144.stderr: 83.7% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T18:07:29.533 INFO:teuthology.orchestra.run.smithi144.stderr: 90.6% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T18:07:29.534 INFO:teuthology.orchestra.run.smithi144.stderr: 87.3% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T18:07:29.559 INFO:teuthology.orchestra.run.smithi105.stderr: 90.4% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-26T18:07:29.666 INFO:teuthology.orchestra.run.smithi105.stderr: 94.7% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-26T18:07:29.676 INFO:teuthology.orchestra.run.smithi135.stderr: 83.7% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T18:07:29.719 INFO:teuthology.orchestra.run.smithi105.stderr: 94.8% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-26T18:07:29.780 INFO:teuthology.orchestra.run.smithi144.stderr: 94.3% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-26T18:07:29.803 INFO:teuthology.orchestra.run.smithi135.stderr: 90.6% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T18:07:29.803 INFO:teuthology.orchestra.run.smithi135.stderr: 91.0% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T18:07:29.831 INFO:teuthology.orchestra.run.smithi135.stderr: 87.3% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T18:07:30.563 INFO:teuthology.orchestra.run.smithi144.stderr: 91.9% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-26T18:07:31.277 INFO:teuthology.orchestra.run.smithi105.stderr: 91.0% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-26T18:07:32.109 INFO:teuthology.orchestra.run.smithi135.stderr: 94.8% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-26T18:07:34.767 INFO:teuthology.orchestra.run.smithi144.stderr: 95.2% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-26T18:07:38.271 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-26T18:07:39.561 INFO:teuthology.orchestra.run.smithi144.stderr: 93.6% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-26T18:07:40.111 INFO:teuthology.orchestra.run.smithi144.stderr: 93.9% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-26T18:07:42.041 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-26T18:07:43.722 INFO:teuthology.orchestra.run.smithi144.stderr: 93.9% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-26T18:07:45.526 INFO:teuthology.orchestra.run.smithi105.stderr: 95.0% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-26T18:07:45.868 INFO:teuthology.orchestra.run.smithi144.stderr: 95.3% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-26T18:07:50.237 INFO:teuthology.orchestra.run.smithi135.stderr: 92.0% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-26T18:08:18.728 INFO:teuthology.orchestra.run.smithi105.stderr: 93.6% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-26T18:08:25.933 INFO:teuthology.orchestra.run.smithi105.stderr: 93.5% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-26T18:08:59.677 INFO:teuthology.orchestra.run.smithi105.stderr: 93.6% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-26T18:09:12.002 INFO:teuthology.orchestra.run.smithi105.stderr: 95.0% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-26T18:09:33.945 INFO:teuthology.orchestra.run.smithi135.stderr: 93.6% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-26T18:09:38.527 INFO:teuthology.orchestra.run.smithi135.stderr: 93.6% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-26T18:09:38.975 INFO:teuthology.orchestra.run.smithi135.stderr: 96.4% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-26T18:10:01.966 INFO:teuthology.orchestra.run.smithi144.stderr: 94.6% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-26T18:10:01.968 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-03-26T18:10:01.968 INFO:teuthology.orchestra.run.smithi144.stderr:real 2m32.742s 2024-03-26T18:10:01.968 INFO:teuthology.orchestra.run.smithi144.stderr:user 3m19.446s 2024-03-26T18:10:01.968 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m7.108s 2024-03-26T18:10:09.979 INFO:teuthology.orchestra.run.smithi135.stderr: 94.8% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-26T18:10:21.707 INFO:teuthology.orchestra.run.smithi135.stderr: 93.8% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-26T18:10:39.724 INFO:teuthology.orchestra.run.smithi135.stderr: 93.9% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-26T18:15:10.198 INFO:teuthology.orchestra.run.smithi105.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-26T18:15:10.221 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-26T18:15:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:real 7m41.002s 2024-03-26T18:15:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:user 6m9.224s 2024-03-26T18:15:10.221 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m16.422s 2024-03-26T18:19:21.395 INFO:teuthology.orchestra.run.smithi135.stderr: 94.9% -- replaced with /var/log/ceph/15cee578-eb96-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-26T18:19:21.398 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-03-26T18:19:21.398 INFO:teuthology.orchestra.run.smithi135.stderr:real 11m52.174s 2024-03-26T18:19:21.398 INFO:teuthology.orchestra.run.smithi135.stderr:user 8m42.973s 2024-03-26T18:19:21.398 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m22.454s 2024-03-26T18:19:21.399 INFO:tasks.cephadm:Archiving logs... 2024-03-26T18:19:21.400 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi105/log 2024-03-26T18:19:21.401 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T18:20:24.333 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi135/log 2024-03-26T18:20:24.428 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T18:21:38.932 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi144/log 2024-03-26T18:21:38.934 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T18:22:06.283 INFO:tasks.cephadm:Removing cluster... 2024-03-26T18:22:06.284 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 15cee578-eb96-11ee-95d0-87774f69a715 --force 2024-03-26T18:22:06.571 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T18:22:06.961 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 15cee578-eb96-11ee-95d0-87774f69a715 --force 2024-03-26T18:22:07.178 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T18:22:07.820 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 15cee578-eb96-11ee-95d0-87774f69a715 --force 2024-03-26T18:22:08.012 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 15cee578-eb96-11ee-95d0-87774f69a715 2024-03-26T18:22:08.497 INFO:tasks.cephadm:Removing cephadm ... 2024-03-26T18:22:08.498 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T18:22:08.541 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T18:22:08.558 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T18:22:08.597 INFO:tasks.cephadm:Teardown complete 2024-03-26T18:22:08.597 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-26T18:22:08.609 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-26T18:22:08.610 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T18:22:08.613 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T18:22:08.615 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T18:22:08.741 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-26T18:22:08.752 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-26T18:22:08.752 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:22:08.755 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:22:08.758 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T18:22:08.771 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:22:08.771 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:22:08.790 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-26T18:22:08.797 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:22:08.797 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-03-26T18:22:08.797 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 59 +968us[ +968us] +/- 78ms 2024-03-26T18:22:08.797 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv02.front.sepia.ceph.com 3 6 377 63 -241us[ -220us] +/- 73ms 2024-03-26T18:22:08.797 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 -615us[ -615us] +/- 68ms 2024-03-26T18:22:08.797 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:22:08.802 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:22:08.802 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-03-26T18:22:08.802 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +973us[ +973us] +/- 78ms 2024-03-26T18:22:08.802 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv02.front.sepia.ceph.com 3 6 377 60 -181us[ -170us] +/- 73ms 2024-03-26T18:22:08.802 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -611us[ -600us] +/- 68ms 2024-03-26T18:22:08.802 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:22:08.817 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T18:22:08.818 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-26T18:22:08.818 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 +1007us[+1046us] +/- 78ms 2024-03-26T18:22:08.818 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -201us[ -162us] +/- 73ms 2024-03-26T18:22:08.818 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -572us[ -572us] +/- 68ms 2024-03-26T18:22:08.818 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T18:22:08.819 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-26T18:22:08.829 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-26T18:22:08.830 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-26T18:22:08.840 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T18:22:09.051 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T18:22:09.329 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T18:22:09.434 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:22:09.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:22:09.469 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:22:09.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:22:09.505 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T18:22:09.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:22:09.536 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-26T18:22:09.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-26T18:22:09.558 INFO:teuthology.task.internal:Duration was 3705.342796 seconds 2024-03-26T18:22:09.559 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-26T18:22:09.570 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-26T18:22:09.570 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T18:22:09.573 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T18:22:09.576 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T18:22:09.646 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:22:09.650 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:22:09.678 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T18:22:10.141 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-26T18:22:10.141 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-03-26T18:22:10.142 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T18:22:10.177 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-03-26T18:22:10.177 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T18:22:10.216 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-03-26T18:22:10.216 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T18:22:10.252 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-26T18:22:10.253 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T18:22:10.256 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T18:22:10.259 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T18:22:10.336 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-26T18:22:10.337 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T18:22:10.339 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T18:22:10.342 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T18:22:10.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-26T18:22:10.771 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-26T18:22:10.771 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T18:22:10.799 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T18:22:10.837 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T18:22:10.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-26T18:22:10.882 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:22:10.885 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:22:10.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:22:10.914 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-03-26T18:22:10.918 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-03-26T18:22:10.939 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-03-26T18:22:10.959 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:22:10.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:22:10.992 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:22:11.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:22:11.006 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T18:22:11.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T18:22:11.020 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-26T18:22:11.031 INFO:teuthology.task.internal:Transferring archived files... 2024-03-26T18:22:11.033 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi105 2024-03-26T18:22:11.033 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T18:22:11.080 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi135 2024-03-26T18:22:11.080 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T18:22:11.121 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746/remote/smithi144 2024-03-26T18:22:11.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T18:22:11.161 INFO:teuthology.task.internal:Removing archive directory... 2024-03-26T18:22:11.161 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T18:22:11.164 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T18:22:11.166 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T18:22:11.210 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-26T18:22:11.222 INFO:teuthology.task.internal:Not uploading archives. 2024-03-26T18:22:11.222 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-26T18:22:11.250 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-26T18:22:11.250 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T18:22:11.253 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T18:22:11.255 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T18:22:11.265 INFO:teuthology.orchestra.run.smithi105.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 18:22 /home/ubuntu/cephtest 2024-03-26T18:22:11.267 INFO:teuthology.orchestra.run.smithi135.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 18:22 /home/ubuntu/cephtest 2024-03-26T18:22:11.269 INFO:teuthology.orchestra.run.smithi144.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 18:22 /home/ubuntu/cephtest 2024-03-26T18:22:11.287 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-26T18:22:11.298 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-26T18:22:11.323 INFO:teuthology.nuke:Checking targets against current locks 2024-03-26T18:22:11.348 DEBUG:teuthology.nuke:shortname: smithi105 2024-03-26T18:22:11.349 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T18:22:11.369 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:11:42.884831', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMJkJ/zNzhJMU8rIIvlIXthVYsZ4BrKzZ4FpTaWdJ4itQTssydhBKDBCQynlNS3fdGCiBTJFDtnYiFB4g19oNx0='} 2024-03-26T18:22:11.375 DEBUG:teuthology.nuke:shortname: smithi135 2024-03-26T18:22:11.375 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T18:22:11.397 DEBUG:teuthology.nuke:shortname: smithi144 2024-03-26T18:22:11.397 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T18:22:11.400 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:11:42.886026', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGmWgygJpbWFGl1WNLgOHy3mouoIX1uBTaGSpq9L6pDJasjwvgZBr+TMwjG7Gk1OODvDxb7FJYuOG13N5bVFggU='} 2024-03-26T18:22:11.412 INFO:teuthology.orchestra.console.smithi105:Power off 2024-03-26T18:22:11.412 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T18:22:11.423 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-03-26_17:10:07-fs-wip-batrick-testing-20240320.145742-distro-default-smithi/7623746', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 17:11:42.883247', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGG/8Zcx0ZZbBdPTy0I0nOjTN5Ygmm++UXaETsRL3K+50Wl0R38H93gRXYIWTzQZIfmN310YtsXsjIWLW34c/8Y='} 2024-03-26T18:22:11.438 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-03-26T18:22:11.438 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:11.457 INFO:teuthology.orchestra.console.smithi135:Power off 2024-03-26T18:22:11.457 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T18:22:11.467 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-26T18:22:11.480 DEBUG:teuthology.orchestra.console.smithi135:power off output: Chassis Power Control: Down/Off 2024-03-26T18:22:11.480 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:11.494 INFO:teuthology.orchestra.console.smithi144:Power off 2024-03-26T18:22:11.494 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T18:22:11.505 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2024-03-26T18:22:11.518 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-03-26T18:22:11.518 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:11.547 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-26T18:22:15.468 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:15.506 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:15.549 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:15.580 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-26T18:22:15.617 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on 2024-03-26T18:22:15.659 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-03-26T18:22:19.582 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:19.618 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:19.660 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T18:22:19.692 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-03-26T18:22:19.729 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off 2024-03-26T18:22:19.770 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-03-26T18:22:19.793 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-03-26T18:22:19.830 INFO:teuthology.orchestra.console.smithi135:Power off completed 2024-03-26T18:22:19.870 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-03-26T18:22:19.977 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} duration: 3705.342796087265 failure_reason: '"2024-03-26T17:52:14.058729+0000 mds.b (mds.0) 139 : cluster [WRN] Scrub error on inode 0x10000003c80 (/volumes/qa/sv_0/c5a24205-8212-466d-b5ed-c6d2b325c0fb/client.0/tmp/clients/client8/~dmtmp/COREL) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_lflores@teuthology success: false 2024-03-26T18:22:19.977 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T18:22:20.069 INFO:teuthology.run:FAIL